2024-03-21T09:56:45.283 INFO:root:teuthology version: 0.0.1.dev262+ge691533 2024-03-21T09:56:45.284 DEBUG:teuthology.run:Teuthology command: teuthology --owner scheduled_vshankar@teuthology --archive /home/teuthworker/archive/vshankar-2024-03-20_11:21:18-fs-wip-vshankar-testing-20240320.053002-testing-default-smithi/7612221 --description fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate} clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} mount/fuse objectstore-ec/bluestore-ec-root omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/automatic export-check n/5 replication/default} standby-replay tasks/{0-subvolume/{with-namespace-isolated} 1-check-counter 2-scrub/yes 3-snaps/yes 4-flush/yes 5-workunit/suites/fsync-tester}} --name vshankar-2024-03-20_11:21:18-fs-wip-vshankar-testing-20240320.053002-testing-default-smithi --verbose -- /home/teuthworker/archive/vshankar-2024-03-20_11:21:18-fs-wip-vshankar-testing-20240320.053002-testing-default-smithi/7612221/orig.config.yaml 2024-03-21T09:56:45.342 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-03-21T09:56:45.433 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/vshankar-2024-03-20_11:21:18-fs-wip-vshankar-testing-20240320.053002-testing-default-smithi/7612221 branch: wip-vshankar-testing-20240320.053002 description: fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate} clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} mount/fuse objectstore-ec/bluestore-ec-root omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/automatic export-check n/5 replication/default} standby-replay tasks/{0-subvolume/{with-namespace-isolated} 1-check-counter 2-scrub/yes 3-snaps/yes 4-flush/yes 5-workunit/suites/fsync-tester}} email: vshankar@redhat.com first_in_suite: false job_id: '7612221' kernel: flavor: default kdb: true sha1: 5ec3e6e88b2e2783e09498588bb1f433ae9c8986 last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi name: vshankar-2024-03-20_11:21:18-fs-wip-vshankar-testing-20240320.053002-testing-default-smithi no_nested_subset: false nuke-on-error: true openstack: - volumes: count: 4 size: 20 - machine: disk: 200 os_type: centos os_version: 9.stream overrides: admin_socket: branch: wip-vshankar-testing-20240320.053002 ceph: cephfs: ec_profile: - m=2 - k=2 - crush-failure-domain=osd max_mds: 5 session_timeout: 300 standby_replay: true conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 rados mon op timeout: 900 rados osd op timeout: 900 mds: debug mds: 20 debug mds balancer: 20 debug ms: 1 mds bal fragment size max: 10000 mds bal merge size: 5 mds bal split bits: 3 mds bal split size: 100 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: cephsqlite lock renewal timeout: 900000 debug cephsqlite: 20 debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon op complaint time: 120 osd: bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 20 debug bluestore: 20 debug ms: 1 debug osd: 20 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore osd op complaint time: 180 osd shutdown pgref assert: true osd_max_omap_entries_per_request: 10000 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - FS_DEGRADED - FS_INLINE_DATA_DEPRECATED - FS_WITH_FAILED_MDS - MDS_ALL_DOWN - MDS_DAMAGE - MDS_DEGRADED - MDS_FAILED - MDS_INSUFFICIENT_STANDBY - MDS_UP_LESS_THAN_MAX - filesystem is online with fewer MDS than max_mds - POOL_APP_NOT_ENABLED - overall HEALTH_ - Replacing daemon - deprecated feature inline_data - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding - MON_DOWN - slow metadata IO - SLOW_OPS - slow request sha1: 2edfee68ad7243bed422a049b6f5f58619300f11 subvols: create: 2 subvol_options: --namespace-isolated ceph-deploy: bluestore: true conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: osd default pool size: 2 osd: bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 20 debug bluestore: 20 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore fs: xfs ceph-fuse: client.0: mount_subvol_num: 0 check-counter: counters: mds: - mds.exported - mds.imported - mds.root_rsnaps - mds_server.req_mksnap_latency.avgcount - mds_server.req_rmsnap_latency.avgcount dry_run: true install: ceph: flavor: default sha1: 2edfee68ad7243bed422a049b6f5f58619300f11 kclient: client.0: mount_subvol_num: 1 selinux: allowlist: - scontext=system_u:system_r:logrotate_t:s0 - scontext=system_u:system_r:getty_t:s0 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: wip-vshankar-testing-20240320.053002 sha1: 2edfee68ad7243bed422a049b6f5f58619300f11 owner: scheduled_vshankar@teuthology priority: 100 repo: https://git.ceph.com/ceph-ci.git roles: - - mon.a - mgr.x - mds.a - mds.d - mds.g - mds.j - osd.0 - osd.3 - osd.6 - osd.9 - client.0 - - mon.b - mgr.y - mds.b - mds.e - mds.h - mds.k - osd.1 - osd.4 - osd.7 - osd.10 - - mon.c - mgr.z - mds.c - mds.f - mds.i - mds.l - osd.2 - osd.5 - osd.8 - osd.11 seed: 8073 sha1: 2edfee68ad7243bed422a049b6f5f58619300f11 sleep_before_teardown: 0 subset: 203/1024 suite: fs suite_branch: wip-vshankar-testing-20240320.053002 suite_path: /home/teuthworker/src/git.ceph.com_ceph-c_2edfee68ad7243bed422a049b6f5f58619300f11/qa suite_relpath: qa suite_repo: https://git.ceph.com/ceph-ci.git suite_sha1: 2edfee68ad7243bed422a049b6f5f58619300f11 syslog: ignorelist: - WARNING*.*check_session_state - WARNING*.*__ceph_remove_cap targets: smithi047.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBHNXJSw2oJjrIRoEo1Osgd8J/rbxEni4kT0rPJ7iyFCCmLmOeHp7qFWpLHBfaLStn9uQ5h8B5wjIeEzYPyxoawU= smithi100.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBCtiuUJf14dOt1EXUNXu5LKRKerqT+HKFTSH10265CedGplUrp3qL+/yj4PXM8//sBfj+Pra2spb7RAgx1LOEBw= smithi157.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBM/llVo/dKuUfYnnYg3GbzDRHmUgFjiTlkAr3eSJvGV/iWSij9WSYdIjABt+AMkcLr9HLbGhSL7qCEIaxaDwTmk= tasks: - install: extra_packages: deb: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror rpm: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev - postgresql - postgresql-client - postgresql-common - postgresql-contrib rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - postgresql - postgresql-server - postgresql-contrib - cephadm: roleless: false - cephadm.shell: mon.a: - ceph orch status - ceph orch ps - ceph orch ls - ceph orch host ls - ceph orch device ls - cephadm.shell: mon.a: - ceph fs dump - ceph osd dump - fs.ready: timeout: 300 - ceph-fuse: null - exec: mon.a: - ceph fs set cephfs balance_automate true - check-counter: {} - fwd_scrub: scrub_timeout: 900 sleep_between_iterations: 1 - exec: mon.a: - ceph mgr module enable snap_schedule - ceph config set mgr mgr/snap_schedule/allow_m_granularity true - ceph config set mgr mgr/snap_schedule/dump_on_update true - ceph fs snap-schedule add --fs=cephfs --path=/ --snap_schedule=1m - ceph fs snap-schedule retention add --fs=cephfs --path=/ --retention-spec-or-period=6m3h - ceph fs snap-schedule status --fs=cephfs --path=/ - ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true - date +%s > START_TIME - full_sequential_finally: - exec: mon.a: - date +%s > END_TIME - START_TIME=$(cat START_TIME); END_TIME=$(cat END_TIME); DIFF_TIME=$((600-(END_TIME-START_TIME))); if [ "$DIFF_TIME" -gt 0 ]; then sleep "$DIFF_TIME"; fi - ceph fs snap-schedule status --fs=cephfs --path=/ - ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true - background_exec: mon.a: - while sleep 13; do ceph tell mds.cephfs:0 flush journal; done - workunit: clients: all: - suites/fsync-tester.sh teuthology: fragments_dropped: - /cephfs/home/vshankar/src/git.ceph.com_ceph-c_wip-vshankar-testing-20240320.053002/qa/suites/fs/workload/overrides/pg_health.yaml meta: {} postmerge: - local function is_kupstream() return false end - local function is_kdistro() return false end - local function is_fuse() return true end - local function syntax_version() return '' end teuthology_branch: main teuthology_sha1: e691533f9cbb33d85b2187bba20d7102f098636d timestamp: 2024-03-20_11:21:18 tube: smithi user: vshankar verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.2226885 2024-03-21T09:56:45.433 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph-c_2edfee68ad7243bed422a049b6f5f58619300f11/qa; will attempt to use it 2024-03-21T09:56:45.434 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph-c_2edfee68ad7243bed422a049b6f5f58619300f11/qa/tasks 2024-03-21T09:56:45.434 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-03-21T09:56:45.436 INFO:teuthology.task.internal:Checking packages... 2024-03-21T09:56:45.460 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'default' and ceph hash '2edfee68ad7243bed422a049b6f5f58619300f11' 2024-03-21T09:56:45.460 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-03-21T09:56:45.460 INFO:teuthology.packaging:ref: None 2024-03-21T09:56:45.461 INFO:teuthology.packaging:tag: None 2024-03-21T09:56:45.461 INFO:teuthology.packaging:branch: wip-vshankar-testing-20240320.053002 2024-03-21T09:56:45.461 INFO:teuthology.packaging:sha1: 2edfee68ad7243bed422a049b6f5f58619300f11 2024-03-21T09:56:45.461 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=wip-vshankar-testing-20240320.053002 2024-03-21T09:56:45.597 INFO:teuthology.task.internal:Found packages for ceph version 19.0.0-2231.g2edfee68 2024-03-21T09:56:45.599 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-03-21T09:56:45.606 INFO:teuthology.task.internal:no buildpackages task found 2024-03-21T09:56:45.606 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-03-21T09:56:45.614 INFO:teuthology.task.internal:Saving configuration 2024-03-21T09:56:45.636 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-03-21T09:56:45.642 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-03-21T09:56:45.668 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi047.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/vshankar-2024-03-20_11:21:18-fs-wip-vshankar-testing-20240320.053002-testing-default-smithi/7612221', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-03-21 09:51:20.906566', 'locked_by': 'scheduled_vshankar@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBHNXJSw2oJjrIRoEo1Osgd8J/rbxEni4kT0rPJ7iyFCCmLmOeHp7qFWpLHBfaLStn9uQ5h8B5wjIeEzYPyxoawU='} 2024-03-21T09:56:45.692 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi100.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/vshankar-2024-03-20_11:21:18-fs-wip-vshankar-testing-20240320.053002-testing-default-smithi/7612221', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-03-21 09:51:20.904946', 'locked_by': 'scheduled_vshankar@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBCtiuUJf14dOt1EXUNXu5LKRKerqT+HKFTSH10265CedGplUrp3qL+/yj4PXM8//sBfj+Pra2spb7RAgx1LOEBw='} 2024-03-21T09:56:45.714 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi157.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/vshankar-2024-03-20_11:21:18-fs-wip-vshankar-testing-20240320.053002-testing-default-smithi/7612221', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-03-21 09:51:20.907815', 'locked_by': 'scheduled_vshankar@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBM/llVo/dKuUfYnnYg3GbzDRHmUgFjiTlkAr3eSJvGV/iWSij9WSYdIjABt+AMkcLr9HLbGhSL7qCEIaxaDwTmk='} 2024-03-21T09:56:45.714 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-03-21T09:56:45.721 INFO:teuthology.task.internal:roles: ubuntu@smithi047.front.sepia.ceph.com - ['mon.a', 'mgr.x', 'mds.a', 'mds.d', 'mds.g', 'mds.j', 'osd.0', 'osd.3', 'osd.6', 'osd.9', 'client.0'] 2024-03-21T09:56:45.721 INFO:teuthology.task.internal:roles: ubuntu@smithi100.front.sepia.ceph.com - ['mon.b', 'mgr.y', 'mds.b', 'mds.e', 'mds.h', 'mds.k', 'osd.1', 'osd.4', 'osd.7', 'osd.10'] 2024-03-21T09:56:45.721 INFO:teuthology.task.internal:roles: ubuntu@smithi157.front.sepia.ceph.com - ['mon.c', 'mgr.z', 'mds.c', 'mds.f', 'mds.i', 'mds.l', 'osd.2', 'osd.5', 'osd.8', 'osd.11'] 2024-03-21T09:56:45.721 INFO:teuthology.run_tasks:Running task console_log... 2024-03-21T09:56:45.838 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f15f3f8f160>, signals=[15]) 2024-03-21T09:56:45.839 INFO:teuthology.run_tasks:Running task internal.connect... 2024-03-21T09:56:45.846 INFO:teuthology.task.internal:Opening connections... 2024-03-21T09:56:45.847 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi047.front.sepia.ceph.com 2024-03-21T09:56:45.849 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi047.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-21T09:56:45.994 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi100.front.sepia.ceph.com 2024-03-21T09:56:45.995 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi100.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-21T09:56:46.083 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi157.front.sepia.ceph.com 2024-03-21T09:56:46.084 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi157.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-21T09:56:46.174 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-03-21T09:56:46.180 DEBUG:teuthology.orchestra.run.smithi047:> uname -m 2024-03-21T09:56:46.199 INFO:teuthology.orchestra.run.smithi047.stdout:x86_64 2024-03-21T09:56:46.199 DEBUG:teuthology.orchestra.run.smithi047:> cat /etc/os-release 2024-03-21T09:56:46.257 INFO:teuthology.orchestra.run.smithi047.stdout:NAME="CentOS Stream" 2024-03-21T09:56:46.257 INFO:teuthology.orchestra.run.smithi047.stdout:VERSION="9" 2024-03-21T09:56:46.257 INFO:teuthology.orchestra.run.smithi047.stdout:ID="centos" 2024-03-21T09:56:46.257 INFO:teuthology.orchestra.run.smithi047.stdout:ID_LIKE="rhel fedora" 2024-03-21T09:56:46.258 INFO:teuthology.orchestra.run.smithi047.stdout:VERSION_ID="9" 2024-03-21T09:56:46.258 INFO:teuthology.orchestra.run.smithi047.stdout:PLATFORM_ID="platform:el9" 2024-03-21T09:56:46.258 INFO:teuthology.orchestra.run.smithi047.stdout:PRETTY_NAME="CentOS Stream 9" 2024-03-21T09:56:46.258 INFO:teuthology.orchestra.run.smithi047.stdout:ANSI_COLOR="0;31" 2024-03-21T09:56:46.258 INFO:teuthology.orchestra.run.smithi047.stdout:LOGO="fedora-logo-icon" 2024-03-21T09:56:46.258 INFO:teuthology.orchestra.run.smithi047.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-03-21T09:56:46.258 INFO:teuthology.orchestra.run.smithi047.stdout:HOME_URL="https://centos.org/" 2024-03-21T09:56:46.258 INFO:teuthology.orchestra.run.smithi047.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-03-21T09:56:46.258 INFO:teuthology.orchestra.run.smithi047.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-03-21T09:56:46.258 INFO:teuthology.orchestra.run.smithi047.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-03-21T09:56:46.259 INFO:teuthology.lock.ops:Updating smithi047.front.sepia.ceph.com on lock server 2024-03-21T09:56:46.283 DEBUG:teuthology.orchestra.run.smithi100:> uname -m 2024-03-21T09:56:46.301 INFO:teuthology.orchestra.run.smithi100.stdout:x86_64 2024-03-21T09:56:46.301 DEBUG:teuthology.orchestra.run.smithi100:> cat /etc/os-release 2024-03-21T09:56:46.358 INFO:teuthology.orchestra.run.smithi100.stdout:NAME="CentOS Stream" 2024-03-21T09:56:46.359 INFO:teuthology.orchestra.run.smithi100.stdout:VERSION="9" 2024-03-21T09:56:46.359 INFO:teuthology.orchestra.run.smithi100.stdout:ID="centos" 2024-03-21T09:56:46.359 INFO:teuthology.orchestra.run.smithi100.stdout:ID_LIKE="rhel fedora" 2024-03-21T09:56:46.359 INFO:teuthology.orchestra.run.smithi100.stdout:VERSION_ID="9" 2024-03-21T09:56:46.359 INFO:teuthology.orchestra.run.smithi100.stdout:PLATFORM_ID="platform:el9" 2024-03-21T09:56:46.359 INFO:teuthology.orchestra.run.smithi100.stdout:PRETTY_NAME="CentOS Stream 9" 2024-03-21T09:56:46.359 INFO:teuthology.orchestra.run.smithi100.stdout:ANSI_COLOR="0;31" 2024-03-21T09:56:46.359 INFO:teuthology.orchestra.run.smithi100.stdout:LOGO="fedora-logo-icon" 2024-03-21T09:56:46.359 INFO:teuthology.orchestra.run.smithi100.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-03-21T09:56:46.359 INFO:teuthology.orchestra.run.smithi100.stdout:HOME_URL="https://centos.org/" 2024-03-21T09:56:46.359 INFO:teuthology.orchestra.run.smithi100.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-03-21T09:56:46.360 INFO:teuthology.orchestra.run.smithi100.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-03-21T09:56:46.360 INFO:teuthology.orchestra.run.smithi100.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-03-21T09:56:46.360 INFO:teuthology.lock.ops:Updating smithi100.front.sepia.ceph.com on lock server 2024-03-21T09:56:46.383 DEBUG:teuthology.orchestra.run.smithi157:> uname -m 2024-03-21T09:56:46.401 INFO:teuthology.orchestra.run.smithi157.stdout:x86_64 2024-03-21T09:56:46.402 DEBUG:teuthology.orchestra.run.smithi157:> cat /etc/os-release 2024-03-21T09:56:46.459 INFO:teuthology.orchestra.run.smithi157.stdout:NAME="CentOS Stream" 2024-03-21T09:56:46.459 INFO:teuthology.orchestra.run.smithi157.stdout:VERSION="9" 2024-03-21T09:56:46.459 INFO:teuthology.orchestra.run.smithi157.stdout:ID="centos" 2024-03-21T09:56:46.459 INFO:teuthology.orchestra.run.smithi157.stdout:ID_LIKE="rhel fedora" 2024-03-21T09:56:46.459 INFO:teuthology.orchestra.run.smithi157.stdout:VERSION_ID="9" 2024-03-21T09:56:46.459 INFO:teuthology.orchestra.run.smithi157.stdout:PLATFORM_ID="platform:el9" 2024-03-21T09:56:46.459 INFO:teuthology.orchestra.run.smithi157.stdout:PRETTY_NAME="CentOS Stream 9" 2024-03-21T09:56:46.459 INFO:teuthology.orchestra.run.smithi157.stdout:ANSI_COLOR="0;31" 2024-03-21T09:56:46.459 INFO:teuthology.orchestra.run.smithi157.stdout:LOGO="fedora-logo-icon" 2024-03-21T09:56:46.459 INFO:teuthology.orchestra.run.smithi157.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-03-21T09:56:46.459 INFO:teuthology.orchestra.run.smithi157.stdout:HOME_URL="https://centos.org/" 2024-03-21T09:56:46.459 INFO:teuthology.orchestra.run.smithi157.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-03-21T09:56:46.459 INFO:teuthology.orchestra.run.smithi157.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-03-21T09:56:46.460 INFO:teuthology.orchestra.run.smithi157.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-03-21T09:56:46.460 INFO:teuthology.lock.ops:Updating smithi157.front.sepia.ceph.com on lock server 2024-03-21T09:56:46.483 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-03-21T09:56:46.493 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-03-21T09:56:46.499 INFO:teuthology.task.internal:Checking for old test directory... 2024-03-21T09:56:46.500 DEBUG:teuthology.orchestra.run.smithi047:> test '!' -e /home/ubuntu/cephtest 2024-03-21T09:56:46.502 DEBUG:teuthology.orchestra.run.smithi100:> test '!' -e /home/ubuntu/cephtest 2024-03-21T09:56:46.504 DEBUG:teuthology.orchestra.run.smithi157:> test '!' -e /home/ubuntu/cephtest 2024-03-21T09:56:46.521 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-03-21T09:56:46.529 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-03-21T09:56:46.529 DEBUG:teuthology.orchestra.run.smithi047:> test -z $(ls -A /var/lib/ceph) 2024-03-21T09:56:46.560 DEBUG:teuthology.orchestra.run.smithi100:> test -z $(ls -A /var/lib/ceph) 2024-03-21T09:56:46.562 DEBUG:teuthology.orchestra.run.smithi157:> test -z $(ls -A /var/lib/ceph) 2024-03-21T09:56:46.597 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-03-21T09:56:46.674 INFO:teuthology.run_tasks:Running task kernel... 2024-03-21T09:56:46.689 INFO:teuthology.task.kernel:normalize config orig: {'flavor': 'default', 'kdb': True, 'sha1': '5ec3e6e88b2e2783e09498588bb1f433ae9c8986'} 2024-03-21T09:56:46.689 INFO:teuthology.task.kernel:config {'mon.a': {'flavor': 'default', 'kdb': True, 'sha1': '5ec3e6e88b2e2783e09498588bb1f433ae9c8986'}, 'mon.b': {'flavor': 'default', 'kdb': True, 'sha1': '5ec3e6e88b2e2783e09498588bb1f433ae9c8986'}, 'mon.c': {'flavor': 'default', 'kdb': True, 'sha1': '5ec3e6e88b2e2783e09498588bb1f433ae9c8986'}}, timeout 300 2024-03-21T09:56:46.689 DEBUG:teuthology.orchestra.run.smithi047:> test -f /run/.containerenv -o -f /.dockerenv 2024-03-21T09:56:46.690 DEBUG:teuthology.orchestra.run.smithi100:> test -f /run/.containerenv -o -f /.dockerenv 2024-03-21T09:56:46.690 DEBUG:teuthology.orchestra.run.smithi157:> test -f /run/.containerenv -o -f /.dockerenv 2024-03-21T09:56:46.706 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-21T09:56:46.707 DEBUG:teuthology.task.kernel:sha1 for mon.b is 5ec3e6e88b2e2783e09498588bb1f433ae9c8986 2024-03-21T09:56:46.707 INFO:teuthology.task.kernel:Checking kernel version of mon.b, want "5ec3e6e88b2e2783e09498588bb1f433ae9c8986"... 2024-03-21T09:56:46.707 DEBUG:teuthology.orchestra.run.smithi100:> uname -r 2024-03-21T09:56:46.708 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-21T09:56:46.708 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-21T09:56:46.709 DEBUG:teuthology.task.kernel:sha1 for mon.c is 5ec3e6e88b2e2783e09498588bb1f433ae9c8986 2024-03-21T09:56:46.709 INFO:teuthology.task.kernel:Checking kernel version of mon.c, want "5ec3e6e88b2e2783e09498588bb1f433ae9c8986"... 2024-03-21T09:56:46.709 DEBUG:teuthology.orchestra.run.smithi157:> uname -r 2024-03-21T09:56:46.709 DEBUG:teuthology.task.kernel:sha1 for mon.a is 5ec3e6e88b2e2783e09498588bb1f433ae9c8986 2024-03-21T09:56:46.709 INFO:teuthology.task.kernel:Checking kernel version of mon.a, want "5ec3e6e88b2e2783e09498588bb1f433ae9c8986"... 2024-03-21T09:56:46.709 DEBUG:teuthology.orchestra.run.smithi047:> uname -r 2024-03-21T09:56:46.763 INFO:teuthology.orchestra.run.smithi100.stdout:5.14.0-361.el9.x86_64 2024-03-21T09:56:46.764 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-361.el9.x86_64 vs 5ec3e6e88b2e2783e09498588bb1f433ae9c8986 2024-03-21T09:56:46.764 DEBUG:teuthology.task.kernel:failed to parse current kernel version 2024-03-21T09:56:46.764 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F9%2Fx86_64&sha1=5ec3e6e88b2e2783e09498588bb1f433ae9c8986 2024-03-21T09:56:46.769 INFO:teuthology.orchestra.run.smithi047.stdout:5.14.0-361.el9.x86_64 2024-03-21T09:56:46.769 INFO:teuthology.orchestra.run.smithi157.stdout:5.14.0-361.el9.x86_64 2024-03-21T09:56:46.769 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-361.el9.x86_64 vs 5ec3e6e88b2e2783e09498588bb1f433ae9c8986 2024-03-21T09:56:46.769 DEBUG:teuthology.task.kernel:failed to parse current kernel version 2024-03-21T09:56:46.769 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F9%2Fx86_64&sha1=5ec3e6e88b2e2783e09498588bb1f433ae9c8986 2024-03-21T09:56:46.770 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-361.el9.x86_64 vs 5ec3e6e88b2e2783e09498588bb1f433ae9c8986 2024-03-21T09:56:46.770 DEBUG:teuthology.task.kernel:failed to parse current kernel version 2024-03-21T09:56:46.771 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F9%2Fx86_64&sha1=5ec3e6e88b2e2783e09498588bb1f433ae9c8986 2024-03-21T09:56:46.916 DEBUG:teuthology.orchestra.run.smithi047:> sudo yum upgrade -y linux-firmware 2024-03-21T09:56:46.928 DEBUG:teuthology.orchestra.run.smithi100:> sudo yum upgrade -y linux-firmware 2024-03-21T09:56:46.946 DEBUG:teuthology.orchestra.run.smithi157:> sudo yum upgrade -y linux-firmware 2024-03-21T09:56:48.985 INFO:teuthology.orchestra.run.smithi157.stdout:CentOS-9 - AppStream 15 MB/s | 6.7 MB 00:00 2024-03-21T09:56:49.077 INFO:teuthology.orchestra.run.smithi100.stdout:CentOS-9 - AppStream 13 MB/s | 6.7 MB 00:00 2024-03-21T09:56:49.097 INFO:teuthology.orchestra.run.smithi047.stdout:CentOS-9 - AppStream 12 MB/s | 6.7 MB 00:00 2024-03-21T09:56:50.385 INFO:teuthology.orchestra.run.smithi157.stdout:CentOS-9 - BaseOS 6.4 MB/s | 2.0 MB 00:00 2024-03-21T09:56:50.539 INFO:teuthology.orchestra.run.smithi047.stdout:CentOS-9 - BaseOS 6.7 MB/s | 2.0 MB 00:00 2024-03-21T09:56:50.701 INFO:teuthology.orchestra.run.smithi100.stdout:CentOS-9 - BaseOS 3.9 MB/s | 2.0 MB 00:00 2024-03-21T09:56:51.184 INFO:teuthology.orchestra.run.smithi157.stdout:CentOS Stream 9 - BaseOS 19 MB/s | 8.0 MB 00:00 2024-03-21T09:56:51.939 INFO:teuthology.orchestra.run.smithi047.stdout:CentOS Stream 9 - BaseOS 7.7 MB/s | 8.0 MB 00:01 2024-03-21T09:57:03.148 INFO:teuthology.orchestra.run.smithi100.stdout:CentOS Stream 9 - BaseOS 681 kB/s | 8.0 MB 00:12 2024-03-21T09:57:05.016 INFO:teuthology.orchestra.run.smithi100.stdout:CentOS Stream 9 - AppStream 23 MB/s | 19 MB 00:00 2024-03-21T09:57:08.183 INFO:teuthology.orchestra.run.smithi157.stdout:CentOS Stream 9 - AppStream 1.2 MB/s | 19 MB 00:15 2024-03-21T09:57:08.983 INFO:teuthology.orchestra.run.smithi100.stdout:CentOS Stream 9 - CRB 12 MB/s | 6.1 MB 00:00 2024-03-21T09:57:10.351 INFO:teuthology.orchestra.run.smithi100.stdout:CentOS Stream 9 - Extras packages 71 kB/s | 16 kB 00:00 2024-03-21T09:57:11.655 INFO:teuthology.orchestra.run.smithi100.stdout:Extra Packages for Enterprise Linux 17 MB/s | 21 MB 00:01 2024-03-21T09:57:13.487 INFO:teuthology.orchestra.run.smithi047.stdout:CentOS Stream 9 - AppStream 942 kB/s | 19 MB 00:20 2024-03-21T09:57:13.875 INFO:teuthology.orchestra.run.smithi157.stdout:CentOS Stream 9 - CRB 2.7 MB/s | 6.1 MB 00:02 2024-03-21T09:57:15.325 INFO:teuthology.orchestra.run.smithi157.stdout:CentOS Stream 9 - Extras packages 51 kB/s | 16 kB 00:00 2024-03-21T09:57:16.543 INFO:teuthology.orchestra.run.smithi100.stdout:lab-extras 33 kB/s | 1.7 kB 00:00 2024-03-21T09:57:18.146 INFO:teuthology.orchestra.run.smithi047.stdout:CentOS Stream 9 - CRB 5.7 MB/s | 6.1 MB 00:01 2024-03-21T09:57:18.745 INFO:teuthology.orchestra.run.smithi100.stdout:Dependencies resolved. 2024-03-21T09:57:18.750 INFO:teuthology.orchestra.run.smithi100.stdout:================================================================================ 2024-03-21T09:57:18.751 INFO:teuthology.orchestra.run.smithi100.stdout: Package Arch Version Repository Size 2024-03-21T09:57:18.751 INFO:teuthology.orchestra.run.smithi100.stdout:================================================================================ 2024-03-21T09:57:18.751 INFO:teuthology.orchestra.run.smithi100.stdout:Upgrading: 2024-03-21T09:57:18.751 INFO:teuthology.orchestra.run.smithi100.stdout: linux-firmware noarch 20240219-143.el9 CentOS-BaseOS 375 M 2024-03-21T09:57:18.751 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-03-21T09:57:18.751 INFO:teuthology.orchestra.run.smithi100.stdout:Transaction Summary 2024-03-21T09:57:18.751 INFO:teuthology.orchestra.run.smithi100.stdout:================================================================================ 2024-03-21T09:57:18.751 INFO:teuthology.orchestra.run.smithi100.stdout:Upgrade 1 Package 2024-03-21T09:57:18.751 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-03-21T09:57:18.751 INFO:teuthology.orchestra.run.smithi100.stdout:Total download size: 375 M 2024-03-21T09:57:18.751 INFO:teuthology.orchestra.run.smithi100.stdout:Downloading Packages: 2024-03-21T09:57:19.605 INFO:teuthology.orchestra.run.smithi047.stdout:CentOS Stream 9 - Extras packages 60 kB/s | 16 kB 00:00 2024-03-21T09:57:20.557 INFO:teuthology.orchestra.run.smithi047.stdout:Extra Packages for Enterprise Linux 25 MB/s | 21 MB 00:00 2024-03-21T09:57:25.565 INFO:teuthology.orchestra.run.smithi047.stdout:lab-extras 31 kB/s | 1.7 kB 00:00 2024-03-21T09:57:26.736 INFO:teuthology.orchestra.run.smithi100.stdout:linux-firmware-20240219-143.el9.noarch.rpm 47 MB/s | 375 MB 00:07 2024-03-21T09:57:26.737 INFO:teuthology.orchestra.run.smithi100.stdout:-------------------------------------------------------------------------------- 2024-03-21T09:57:26.737 INFO:teuthology.orchestra.run.smithi100.stdout:Total 47 MB/s | 375 MB 00:07 2024-03-21T09:57:26.737 INFO:teuthology.orchestra.run.smithi100.stdout:Running transaction check 2024-03-21T09:57:26.820 INFO:teuthology.orchestra.run.smithi100.stdout:Transaction check succeeded. 2024-03-21T09:57:26.820 INFO:teuthology.orchestra.run.smithi100.stdout:Running transaction test 2024-03-21T09:57:27.806 INFO:teuthology.orchestra.run.smithi047.stdout:Dependencies resolved. 2024-03-21T09:57:27.814 INFO:teuthology.orchestra.run.smithi047.stdout:================================================================================ 2024-03-21T09:57:27.815 INFO:teuthology.orchestra.run.smithi047.stdout: Package Arch Version Repository Size 2024-03-21T09:57:27.815 INFO:teuthology.orchestra.run.smithi047.stdout:================================================================================ 2024-03-21T09:57:27.815 INFO:teuthology.orchestra.run.smithi047.stdout:Upgrading: 2024-03-21T09:57:27.815 INFO:teuthology.orchestra.run.smithi047.stdout: linux-firmware noarch 20240219-143.el9 CentOS-BaseOS 375 M 2024-03-21T09:57:27.815 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-21T09:57:27.815 INFO:teuthology.orchestra.run.smithi047.stdout:Transaction Summary 2024-03-21T09:57:27.815 INFO:teuthology.orchestra.run.smithi047.stdout:================================================================================ 2024-03-21T09:57:27.815 INFO:teuthology.orchestra.run.smithi047.stdout:Upgrade 1 Package 2024-03-21T09:57:27.815 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-21T09:57:27.815 INFO:teuthology.orchestra.run.smithi047.stdout:Total download size: 375 M 2024-03-21T09:57:27.815 INFO:teuthology.orchestra.run.smithi047.stdout:Downloading Packages: 2024-03-21T09:57:28.950 INFO:teuthology.orchestra.run.smithi157.stdout:Extra Packages for Enterprise Linux 1.5 MB/s | 21 MB 00:13 2024-03-21T09:57:29.285 INFO:teuthology.orchestra.run.smithi100.stdout:Transaction test succeeded. 2024-03-21T09:57:29.285 INFO:teuthology.orchestra.run.smithi100.stdout:Running transaction 2024-03-21T09:57:33.950 INFO:teuthology.orchestra.run.smithi157.stdout:lab-extras 16 kB/s | 1.7 kB 00:00 2024-03-21T09:57:35.490 INFO:teuthology.orchestra.run.smithi100.stdout: Preparing : 1/1 2024-03-21T09:57:35.694 INFO:teuthology.orchestra.run.smithi100.stdout: Upgrading : linux-firmware-20240219-143.el9.noarch 1/2 2024-03-21T09:57:35.768 INFO:teuthology.orchestra.run.smithi100.stdout: Cleanup : linux-firmware-20230814-139.el9.noarch 2/2 2024-03-21T09:57:36.109 INFO:teuthology.orchestra.run.smithi157.stdout:Dependencies resolved. 2024-03-21T09:57:36.123 INFO:teuthology.orchestra.run.smithi157.stdout:================================================================================ 2024-03-21T09:57:36.123 INFO:teuthology.orchestra.run.smithi157.stdout: Package Arch Version Repository Size 2024-03-21T09:57:36.123 INFO:teuthology.orchestra.run.smithi157.stdout:================================================================================ 2024-03-21T09:57:36.123 INFO:teuthology.orchestra.run.smithi157.stdout:Upgrading: 2024-03-21T09:57:36.123 INFO:teuthology.orchestra.run.smithi157.stdout: linux-firmware noarch 20240219-143.el9 CentOS-BaseOS 375 M 2024-03-21T09:57:36.123 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-03-21T09:57:36.123 INFO:teuthology.orchestra.run.smithi157.stdout:Transaction Summary 2024-03-21T09:57:36.123 INFO:teuthology.orchestra.run.smithi157.stdout:================================================================================ 2024-03-21T09:57:36.123 INFO:teuthology.orchestra.run.smithi157.stdout:Upgrade 1 Package 2024-03-21T09:57:36.124 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-03-21T09:57:36.124 INFO:teuthology.orchestra.run.smithi157.stdout:Total download size: 375 M 2024-03-21T09:57:36.124 INFO:teuthology.orchestra.run.smithi157.stdout:Downloading Packages: 2024-03-21T09:57:37.212 INFO:teuthology.orchestra.run.smithi100.stdout: Running scriptlet: linux-firmware-20230814-139.el9.noarch 2/2 2024-03-21T09:57:37.213 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : linux-firmware-20240219-143.el9.noarch 1/2 2024-03-21T09:57:37.593 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : linux-firmware-20230814-139.el9.noarch 2/2 2024-03-21T09:57:37.593 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-03-21T09:57:37.593 INFO:teuthology.orchestra.run.smithi100.stdout:Upgraded: 2024-03-21T09:57:37.593 INFO:teuthology.orchestra.run.smithi100.stdout: linux-firmware-20240219-143.el9.noarch 2024-03-21T09:57:37.593 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-03-21T09:57:37.593 INFO:teuthology.orchestra.run.smithi100.stdout:Complete! 2024-03-21T09:57:37.876 INFO:teuthology.task.kernel:Downloading kernel 5ec3e6e88b2e2783e09498588bb1f433ae9c8986 on mon.b... 2024-03-21T09:57:37.877 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F9%2Fx86_64&sha1=5ec3e6e88b2e2783e09498588bb1f433ae9c8986 2024-03-21T09:57:38.222 INFO:teuthology.task.kernel:fetching, builder baseurl is https://4.chacra.ceph.com/r/kernel/testing/5ec3e6e88b2e2783e09498588bb1f433ae9c8986/centos/9/flavors/default/x86_64/ 2024-03-21T09:57:38.222 DEBUG:teuthology.orchestra.run.smithi100:> rm -f /tmp/kernel.x86_64.rpm && echo kernel-6.8.0_g5ec3e6e88b2e-1.x86_64.rpm | wget -nv -O /tmp/kernel.x86_64.rpm --base=https://4.chacra.ceph.com/r/kernel/testing/5ec3e6e88b2e2783e09498588bb1f433ae9c8986/centos/9/flavors/default/x86_64/ --input-file=- 2024-03-21T09:57:38.224 DEBUG:teuthology.task.kernel:Waiting for download/copy to ubuntu@smithi100.front.sepia.ceph.com to complete... 2024-03-21T09:57:38.732 INFO:teuthology.orchestra.run.smithi047.stdout:linux-firmware-20240219-143.el9.noarch.rpm 34 MB/s | 375 MB 00:10 2024-03-21T09:57:38.733 INFO:teuthology.orchestra.run.smithi047.stdout:-------------------------------------------------------------------------------- 2024-03-21T09:57:38.733 INFO:teuthology.orchestra.run.smithi047.stdout:Total 34 MB/s | 375 MB 00:10 2024-03-21T09:57:38.734 INFO:teuthology.orchestra.run.smithi047.stdout:Running transaction check 2024-03-21T09:57:38.840 INFO:teuthology.orchestra.run.smithi047.stdout:Transaction check succeeded. 2024-03-21T09:57:38.840 INFO:teuthology.orchestra.run.smithi047.stdout:Running transaction test 2024-03-21T09:57:41.341 INFO:teuthology.orchestra.run.smithi047.stdout:Transaction test succeeded. 2024-03-21T09:57:41.341 INFO:teuthology.orchestra.run.smithi047.stdout:Running transaction 2024-03-21T09:57:47.050 INFO:teuthology.orchestra.run.smithi157.stdout:linux-firmware-20240219-143.el9.noarch.rpm 34 MB/s | 375 MB 00:10 2024-03-21T09:57:47.051 INFO:teuthology.orchestra.run.smithi157.stdout:-------------------------------------------------------------------------------- 2024-03-21T09:57:47.051 INFO:teuthology.orchestra.run.smithi157.stdout:Total 34 MB/s | 375 MB 00:10 2024-03-21T09:57:47.052 INFO:teuthology.orchestra.run.smithi157.stdout:Running transaction check 2024-03-21T09:57:47.133 INFO:teuthology.orchestra.run.smithi157.stdout:Transaction check succeeded. 2024-03-21T09:57:47.134 INFO:teuthology.orchestra.run.smithi157.stdout:Running transaction test 2024-03-21T09:57:47.673 INFO:teuthology.orchestra.run.smithi047.stdout: Preparing : 1/1 2024-03-21T09:57:47.893 INFO:teuthology.orchestra.run.smithi047.stdout: Upgrading : linux-firmware-20240219-143.el9.noarch 1/2 2024-03-21T09:57:47.994 INFO:teuthology.orchestra.run.smithi047.stdout: Cleanup : linux-firmware-20230814-139.el9.noarch 2/2 2024-03-21T09:57:49.338 INFO:teuthology.orchestra.run.smithi047.stdout: Running scriptlet: linux-firmware-20230814-139.el9.noarch 2/2 2024-03-21T09:57:49.338 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : linux-firmware-20240219-143.el9.noarch 1/2 2024-03-21T09:57:49.535 INFO:teuthology.orchestra.run.smithi157.stdout:Transaction test succeeded. 2024-03-21T09:57:49.536 INFO:teuthology.orchestra.run.smithi157.stdout:Running transaction 2024-03-21T09:57:49.759 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : linux-firmware-20230814-139.el9.noarch 2/2 2024-03-21T09:57:49.759 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-21T09:57:49.759 INFO:teuthology.orchestra.run.smithi047.stdout:Upgraded: 2024-03-21T09:57:49.759 INFO:teuthology.orchestra.run.smithi047.stdout: linux-firmware-20240219-143.el9.noarch 2024-03-21T09:57:49.759 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-21T09:57:49.759 INFO:teuthology.orchestra.run.smithi047.stdout:Complete! 2024-03-21T09:57:50.038 INFO:teuthology.task.kernel:Downloading kernel 5ec3e6e88b2e2783e09498588bb1f433ae9c8986 on mon.a... 2024-03-21T09:57:50.039 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F9%2Fx86_64&sha1=5ec3e6e88b2e2783e09498588bb1f433ae9c8986 2024-03-21T09:57:50.184 INFO:teuthology.task.kernel:fetching, builder baseurl is https://4.chacra.ceph.com/r/kernel/testing/5ec3e6e88b2e2783e09498588bb1f433ae9c8986/centos/9/flavors/default/x86_64/ 2024-03-21T09:57:50.184 DEBUG:teuthology.orchestra.run.smithi047:> rm -f /tmp/kernel.x86_64.rpm && echo kernel-6.8.0_g5ec3e6e88b2e-1.x86_64.rpm | wget -nv -O /tmp/kernel.x86_64.rpm --base=https://4.chacra.ceph.com/r/kernel/testing/5ec3e6e88b2e2783e09498588bb1f433ae9c8986/centos/9/flavors/default/x86_64/ --input-file=- 2024-03-21T09:57:50.187 DEBUG:teuthology.task.kernel:Waiting for download/copy to ubuntu@smithi047.front.sepia.ceph.com to complete... 2024-03-21T09:57:55.875 INFO:teuthology.orchestra.run.smithi157.stdout: Preparing : 1/1 2024-03-21T09:57:56.164 INFO:teuthology.orchestra.run.smithi157.stdout: Upgrading : linux-firmware-20240219-143.el9.noarch 1/2 2024-03-21T09:57:56.258 INFO:teuthology.orchestra.run.smithi157.stdout: Cleanup : linux-firmware-20230814-139.el9.noarch 2/2 2024-03-21T09:57:57.525 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: linux-firmware-20230814-139.el9.noarch 2/2 2024-03-21T09:57:57.525 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : linux-firmware-20240219-143.el9.noarch 1/2 2024-03-21T09:57:57.911 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : linux-firmware-20230814-139.el9.noarch 2/2 2024-03-21T09:57:57.911 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-03-21T09:57:57.911 INFO:teuthology.orchestra.run.smithi157.stdout:Upgraded: 2024-03-21T09:57:57.911 INFO:teuthology.orchestra.run.smithi157.stdout: linux-firmware-20240219-143.el9.noarch 2024-03-21T09:57:57.911 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-03-21T09:57:57.911 INFO:teuthology.orchestra.run.smithi157.stdout:Complete! 2024-03-21T09:57:58.201 INFO:teuthology.task.kernel:Downloading kernel 5ec3e6e88b2e2783e09498588bb1f433ae9c8986 on mon.c... 2024-03-21T09:57:58.202 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F9%2Fx86_64&sha1=5ec3e6e88b2e2783e09498588bb1f433ae9c8986 2024-03-21T09:57:58.348 INFO:teuthology.task.kernel:fetching, builder baseurl is https://4.chacra.ceph.com/r/kernel/testing/5ec3e6e88b2e2783e09498588bb1f433ae9c8986/centos/9/flavors/default/x86_64/ 2024-03-21T09:57:58.348 DEBUG:teuthology.orchestra.run.smithi157:> rm -f /tmp/kernel.x86_64.rpm && echo kernel-6.8.0_g5ec3e6e88b2e-1.x86_64.rpm | wget -nv -O /tmp/kernel.x86_64.rpm --base=https://4.chacra.ceph.com/r/kernel/testing/5ec3e6e88b2e2783e09498588bb1f433ae9c8986/centos/9/flavors/default/x86_64/ --input-file=- 2024-03-21T09:57:58.351 DEBUG:teuthology.task.kernel:Waiting for download/copy to ubuntu@smithi157.front.sepia.ceph.com to complete... 2024-03-21T09:57:59.488 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-21 09:57:59 URL:https://4.chacra.ceph.com/r/kernel/testing/5ec3e6e88b2e2783e09498588bb1f433ae9c8986/centos/9/flavors/default/x86_64/kernel-6.8.0_g5ec3e6e88b2e-1.x86_64.rpm [336309687/336309687] -> "/tmp/kernel.x86_64.rpm" [1] 2024-03-21T09:57:59.488 INFO:teuthology.orchestra.run.smithi100.stderr:FINISHED --2024-03-21 09:57:59-- 2024-03-21T09:57:59.488 INFO:teuthology.orchestra.run.smithi100.stderr:Total wall clock time: 21s 2024-03-21T09:57:59.488 INFO:teuthology.orchestra.run.smithi100.stderr:Downloaded: 1 files, 321M in 21s (15.4 MB/s) 2024-03-21T09:57:59.490 INFO:teuthology.task.kernel:Installing kernel 5ec3e6e88b2e2783e09498588bb1f433ae9c8986 on mon.b... 2024-03-21T09:57:59.490 DEBUG:teuthology.orchestra.run.smithi100:> sudo rpm -ivh --oldpackage --replacefiles --replacepkgs /tmp/kernel.x86_64.rpm 2024-03-21T09:58:01.428 INFO:teuthology.orchestra.run.smithi100.stdout:Verifying... ######################################## 2024-03-21T09:58:01.478 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing... ######################################## 2024-03-21T09:58:01.557 INFO:teuthology.orchestra.run.smithi100.stdout:Updating / installing... 2024-03-21T09:58:08.258 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21 09:58:08 URL:https://4.chacra.ceph.com/r/kernel/testing/5ec3e6e88b2e2783e09498588bb1f433ae9c8986/centos/9/flavors/default/x86_64/kernel-6.8.0_g5ec3e6e88b2e-1.x86_64.rpm [336309687/336309687] -> "/tmp/kernel.x86_64.rpm" [1] 2024-03-21T09:58:08.258 INFO:teuthology.orchestra.run.smithi047.stderr:FINISHED --2024-03-21 09:58:08-- 2024-03-21T09:58:08.258 INFO:teuthology.orchestra.run.smithi047.stderr:Total wall clock time: 18s 2024-03-21T09:58:08.258 INFO:teuthology.orchestra.run.smithi047.stderr:Downloaded: 1 files, 321M in 18s (18.1 MB/s) 2024-03-21T09:58:08.261 INFO:teuthology.task.kernel:Installing kernel 5ec3e6e88b2e2783e09498588bb1f433ae9c8986 on mon.a... 2024-03-21T09:58:08.261 DEBUG:teuthology.orchestra.run.smithi047:> sudo rpm -ivh --oldpackage --replacefiles --replacepkgs /tmp/kernel.x86_64.rpm 2024-03-21T09:58:09.081 INFO:teuthology.orchestra.run.smithi100.stdout:kernel-6.8.0_g5ec3e6e88b2e-1 ######################################## 2024-03-21T09:58:10.289 INFO:teuthology.orchestra.run.smithi047.stdout:Verifying... ######################################## 2024-03-21T09:58:10.356 INFO:teuthology.orchestra.run.smithi047.stdout:Preparing... ######################################## 2024-03-21T09:58:10.441 INFO:teuthology.orchestra.run.smithi047.stdout:Updating / installing... 2024-03-21T09:58:15.724 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-21 09:58:15 URL:https://4.chacra.ceph.com/r/kernel/testing/5ec3e6e88b2e2783e09498588bb1f433ae9c8986/centos/9/flavors/default/x86_64/kernel-6.8.0_g5ec3e6e88b2e-1.x86_64.rpm [336309687/336309687] -> "/tmp/kernel.x86_64.rpm" [1] 2024-03-21T09:58:15.724 INFO:teuthology.orchestra.run.smithi157.stderr:FINISHED --2024-03-21 09:58:15-- 2024-03-21T09:58:15.724 INFO:teuthology.orchestra.run.smithi157.stderr:Total wall clock time: 17s 2024-03-21T09:58:15.724 INFO:teuthology.orchestra.run.smithi157.stderr:Downloaded: 1 files, 321M in 17s (19.1 MB/s) 2024-03-21T09:58:15.727 INFO:teuthology.task.kernel:Installing kernel 5ec3e6e88b2e2783e09498588bb1f433ae9c8986 on mon.c... 2024-03-21T09:58:15.727 DEBUG:teuthology.orchestra.run.smithi157:> sudo rpm -ivh --oldpackage --replacefiles --replacepkgs /tmp/kernel.x86_64.rpm 2024-03-21T09:58:17.667 INFO:teuthology.orchestra.run.smithi157.stdout:Verifying... ######################################## 2024-03-21T09:58:17.765 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing... ######################################## 2024-03-21T09:58:17.846 INFO:teuthology.orchestra.run.smithi157.stdout:Updating / installing... 2024-03-21T09:58:18.319 INFO:teuthology.orchestra.run.smithi047.stdout:kernel-6.8.0_g5ec3e6e88b2e-1 ######################################## 2024-03-21T09:58:24.919 INFO:teuthology.orchestra.run.smithi100.stderr:dracut: Disabling early microcode, because kernel does not support it. CONFIG_MICROCODE_[AMD|INTEL]!=y 2024-03-21T09:58:25.393 INFO:teuthology.orchestra.run.smithi157.stdout:kernel-6.8.0_g5ec3e6e88b2e-1 ######################################## 2024-03-21T09:58:34.309 INFO:teuthology.orchestra.run.smithi047.stderr:dracut: Disabling early microcode, because kernel does not support it. CONFIG_MICROCODE_[AMD|INTEL]!=y 2024-03-21T09:58:41.535 INFO:teuthology.orchestra.run.smithi157.stderr:dracut: Disabling early microcode, because kernel does not support it. CONFIG_MICROCODE_[AMD|INTEL]!=y 2024-03-21T09:58:47.292 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi100.front.sepia.ceph.com, path=/tmp/kernel.x86_64.rpm, version=None) 2024-03-21T09:58:47.292 DEBUG:teuthology.orchestra.run.smithi100:> rpm -qlp /tmp/kernel.x86_64.rpm 2024-03-21T09:58:47.364 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e 2024-03-21T09:58:47.364 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/System.map 2024-03-21T09:58:47.365 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/config 2024-03-21T09:58:47.365 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel 2024-03-21T09:58:47.365 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/arch 2024-03-21T09:58:47.365 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/arch/x86 2024-03-21T09:58:47.365 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/arch/x86/crypto 2024-03-21T09:58:47.365 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/arch/x86/crypto/blowfish-x86_64.ko 2024-03-21T09:58:47.365 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/arch/x86/crypto/camellia-aesni-avx-x86_64.ko 2024-03-21T09:58:47.365 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/arch/x86/crypto/camellia-aesni-avx2.ko 2024-03-21T09:58:47.365 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/arch/x86/crypto/camellia-x86_64.ko 2024-03-21T09:58:47.365 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/arch/x86/crypto/cast5-avx-x86_64.ko 2024-03-21T09:58:47.365 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/arch/x86/crypto/cast6-avx-x86_64.ko 2024-03-21T09:58:47.365 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/arch/x86/crypto/crc32-pclmul.ko 2024-03-21T09:58:47.365 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/arch/x86/crypto/crc32c-intel.ko 2024-03-21T09:58:47.365 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/arch/x86/crypto/crct10dif-pclmul.ko 2024-03-21T09:58:47.365 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/arch/x86/crypto/des3_ede-x86_64.ko 2024-03-21T09:58:47.366 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/arch/x86/crypto/ghash-clmulni-intel.ko 2024-03-21T09:58:47.366 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/arch/x86/crypto/serpent-avx-x86_64.ko 2024-03-21T09:58:47.366 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/arch/x86/crypto/serpent-avx2.ko 2024-03-21T09:58:47.366 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/arch/x86/crypto/serpent-sse2-x86_64.ko 2024-03-21T09:58:47.366 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/arch/x86/crypto/sha1-ssse3.ko 2024-03-21T09:58:47.366 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/arch/x86/crypto/sha256-ssse3.ko 2024-03-21T09:58:47.366 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/arch/x86/crypto/sha512-ssse3.ko 2024-03-21T09:58:47.366 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/arch/x86/crypto/twofish-avx-x86_64.ko 2024-03-21T09:58:47.366 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/arch/x86/crypto/twofish-x86_64-3way.ko 2024-03-21T09:58:47.366 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/arch/x86/crypto/twofish-x86_64.ko 2024-03-21T09:58:47.366 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/arch/x86/kernel 2024-03-21T09:58:47.366 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/arch/x86/kernel/cpu 2024-03-21T09:58:47.367 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/arch/x86/kernel/cpu/mce 2024-03-21T09:58:47.367 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/arch/x86/kernel/cpu/mce/mce-inject.ko 2024-03-21T09:58:47.367 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/arch/x86/kvm 2024-03-21T09:58:47.367 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/arch/x86/kvm/kvm-amd.ko 2024-03-21T09:58:47.367 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/arch/x86/kvm/kvm-intel.ko 2024-03-21T09:58:47.367 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/arch/x86/kvm/kvm.ko 2024-03-21T09:58:47.367 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto 2024-03-21T09:58:47.367 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/ansi_cprng.ko 2024-03-21T09:58:47.367 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/anubis.ko 2024-03-21T09:58:47.367 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/arc4.ko 2024-03-21T09:58:47.367 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/async_tx 2024-03-21T09:58:47.367 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/async_tx/async_memcpy.ko 2024-03-21T09:58:47.367 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/async_tx/async_pq.ko 2024-03-21T09:58:47.367 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/async_tx/async_raid6_recov.ko 2024-03-21T09:58:47.367 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/async_tx/async_tx.ko 2024-03-21T09:58:47.368 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/async_tx/async_xor.ko 2024-03-21T09:58:47.368 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/async_tx/raid6test.ko 2024-03-21T09:58:47.368 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/authenc.ko 2024-03-21T09:58:47.368 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/authencesn.ko 2024-03-21T09:58:47.368 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/blake2b_generic.ko 2024-03-21T09:58:47.368 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/blowfish_common.ko 2024-03-21T09:58:47.368 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/blowfish_generic.ko 2024-03-21T09:58:47.368 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/camellia_generic.ko 2024-03-21T09:58:47.368 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/cast5_generic.ko 2024-03-21T09:58:47.368 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/cast6_generic.ko 2024-03-21T09:58:47.368 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/cast_common.ko 2024-03-21T09:58:47.368 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/ccm.ko 2024-03-21T09:58:47.368 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/cmac.ko 2024-03-21T09:58:47.369 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/crc32_generic.ko 2024-03-21T09:58:47.369 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/crypto_engine.ko 2024-03-21T09:58:47.369 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/crypto_user.ko 2024-03-21T09:58:47.369 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/cts.ko 2024-03-21T09:58:47.369 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/des_generic.ko 2024-03-21T09:58:47.369 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/dh_generic.ko 2024-03-21T09:58:47.369 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/echainiv.ko 2024-03-21T09:58:47.369 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/essiv.ko 2024-03-21T09:58:47.369 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/fcrypt.ko 2024-03-21T09:58:47.369 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/khazad.ko 2024-03-21T09:58:47.369 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/lz4.ko 2024-03-21T09:58:47.369 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/lz4hc.ko 2024-03-21T09:58:47.369 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/md4.ko 2024-03-21T09:58:47.369 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/michael_mic.ko 2024-03-21T09:58:47.370 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/pcbc.ko 2024-03-21T09:58:47.370 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/pcrypt.ko 2024-03-21T09:58:47.370 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/rmd160.ko 2024-03-21T09:58:47.370 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/seed.ko 2024-03-21T09:58:47.370 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/serpent_generic.ko 2024-03-21T09:58:47.370 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/tcrypt.ko 2024-03-21T09:58:47.370 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/tea.ko 2024-03-21T09:58:47.370 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/twofish_common.ko 2024-03-21T09:58:47.370 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/twofish_generic.ko 2024-03-21T09:58:47.370 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/vmac.ko 2024-03-21T09:58:47.370 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/wp512.ko 2024-03-21T09:58:47.370 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/xcbc.ko 2024-03-21T09:58:47.370 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/xor.ko 2024-03-21T09:58:47.371 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/xxhash_generic.ko 2024-03-21T09:58:47.371 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers 2024-03-21T09:58:47.371 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/acpi 2024-03-21T09:58:47.371 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/acpi/acpi_ipmi.ko 2024-03-21T09:58:47.371 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/acpi/acpi_pad.ko 2024-03-21T09:58:47.371 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/acpi/custom_method.ko 2024-03-21T09:58:47.371 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/acpi/ec_sys.ko 2024-03-21T09:58:47.371 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/acpi/platform_profile.ko 2024-03-21T09:58:47.371 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/acpi/sbs.ko 2024-03-21T09:58:47.371 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/acpi/sbshc.ko 2024-03-21T09:58:47.371 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/acpi/video.ko 2024-03-21T09:58:47.371 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata 2024-03-21T09:58:47.371 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/acard-ahci.ko 2024-03-21T09:58:47.371 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/ahci_platform.ko 2024-03-21T09:58:47.371 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/ata_generic.ko 2024-03-21T09:58:47.372 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/libahci_platform.ko 2024-03-21T09:58:47.372 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_acpi.ko 2024-03-21T09:58:47.372 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_ali.ko 2024-03-21T09:58:47.372 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_amd.ko 2024-03-21T09:58:47.372 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_artop.ko 2024-03-21T09:58:47.372 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_atiixp.ko 2024-03-21T09:58:47.372 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_atp867x.ko 2024-03-21T09:58:47.372 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_cmd640.ko 2024-03-21T09:58:47.372 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_cmd64x.ko 2024-03-21T09:58:47.372 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_cypress.ko 2024-03-21T09:58:47.372 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_efar.ko 2024-03-21T09:58:47.372 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_hpt366.ko 2024-03-21T09:58:47.372 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_hpt37x.ko 2024-03-21T09:58:47.372 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_hpt3x2n.ko 2024-03-21T09:58:47.373 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_hpt3x3.ko 2024-03-21T09:58:47.373 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_it8213.ko 2024-03-21T09:58:47.373 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_it821x.ko 2024-03-21T09:58:47.373 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_jmicron.ko 2024-03-21T09:58:47.373 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_marvell.ko 2024-03-21T09:58:47.373 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_mpiix.ko 2024-03-21T09:58:47.373 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_netcell.ko 2024-03-21T09:58:47.373 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_ninja32.ko 2024-03-21T09:58:47.373 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_ns87410.ko 2024-03-21T09:58:47.373 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_ns87415.ko 2024-03-21T09:58:47.373 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_oldpiix.ko 2024-03-21T09:58:47.373 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_opti.ko 2024-03-21T09:58:47.373 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_optidma.ko 2024-03-21T09:58:47.373 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_pcmcia.ko 2024-03-21T09:58:47.374 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_pdc2027x.ko 2024-03-21T09:58:47.374 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_pdc202xx_old.ko 2024-03-21T09:58:47.374 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_piccolo.ko 2024-03-21T09:58:47.374 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_rdc.ko 2024-03-21T09:58:47.374 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_sch.ko 2024-03-21T09:58:47.374 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_serverworks.ko 2024-03-21T09:58:47.374 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_sil680.ko 2024-03-21T09:58:47.374 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_sis.ko 2024-03-21T09:58:47.374 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_sl82c105.ko 2024-03-21T09:58:47.374 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_triflex.ko 2024-03-21T09:58:47.374 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_via.ko 2024-03-21T09:58:47.374 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pdc_adma.ko 2024-03-21T09:58:47.374 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/sata_inic162x.ko 2024-03-21T09:58:47.375 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/sata_mv.ko 2024-03-21T09:58:47.375 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/sata_nv.ko 2024-03-21T09:58:47.375 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/sata_promise.ko 2024-03-21T09:58:47.375 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/sata_qstor.ko 2024-03-21T09:58:47.375 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/sata_sil.ko 2024-03-21T09:58:47.375 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/sata_sil24.ko 2024-03-21T09:58:47.375 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/sata_sis.ko 2024-03-21T09:58:47.375 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/sata_svw.ko 2024-03-21T09:58:47.375 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/sata_sx4.ko 2024-03-21T09:58:47.375 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/sata_uli.ko 2024-03-21T09:58:47.375 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/sata_via.ko 2024-03-21T09:58:47.375 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/sata_vsc.ko 2024-03-21T09:58:47.375 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/auxdisplay 2024-03-21T09:58:47.375 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/auxdisplay/ks0108.ko 2024-03-21T09:58:47.376 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/base 2024-03-21T09:58:47.376 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/base/regmap 2024-03-21T09:58:47.376 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/base/regmap/regmap-i2c.ko 2024-03-21T09:58:47.376 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/base/regmap/regmap-mmio.ko 2024-03-21T09:58:47.376 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/bcma 2024-03-21T09:58:47.376 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/bcma/bcma.ko 2024-03-21T09:58:47.376 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/block 2024-03-21T09:58:47.376 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/block/aoe 2024-03-21T09:58:47.376 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/block/aoe/aoe.ko 2024-03-21T09:58:47.376 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/block/brd.ko 2024-03-21T09:58:47.376 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/block/drbd 2024-03-21T09:58:47.376 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/block/drbd/drbd.ko 2024-03-21T09:58:47.376 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/block/floppy.ko 2024-03-21T09:58:47.376 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/block/loop.ko 2024-03-21T09:58:47.377 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/block/mtip32xx 2024-03-21T09:58:47.377 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/block/mtip32xx/mtip32xx.ko 2024-03-21T09:58:47.377 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/block/nbd.ko 2024-03-21T09:58:47.377 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/block/null_blk 2024-03-21T09:58:47.377 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/block/null_blk/null_blk.ko 2024-03-21T09:58:47.377 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/block/pktcdvd.ko 2024-03-21T09:58:47.377 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/block/rbd.ko 2024-03-21T09:58:47.377 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/block/virtio_blk.ko 2024-03-21T09:58:47.377 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/block/xen-blkback 2024-03-21T09:58:47.377 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/block/xen-blkback/xen-blkback.ko 2024-03-21T09:58:47.377 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/block/xen-blkfront.ko 2024-03-21T09:58:47.377 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/block/zram 2024-03-21T09:58:47.377 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/block/zram/zram.ko 2024-03-21T09:58:47.377 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/char 2024-03-21T09:58:47.377 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/char/hangcheck-timer.ko 2024-03-21T09:58:47.378 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/char/hw_random 2024-03-21T09:58:47.378 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/char/hw_random/amd-rng.ko 2024-03-21T09:58:47.378 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/char/hw_random/intel-rng.ko 2024-03-21T09:58:47.378 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/char/hw_random/timeriomem-rng.ko 2024-03-21T09:58:47.378 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/char/hw_random/via-rng.ko 2024-03-21T09:58:47.378 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/char/hw_random/virtio-rng.ko 2024-03-21T09:58:47.378 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/char/ipmi 2024-03-21T09:58:47.378 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/char/ipmi/ipmi_devintf.ko 2024-03-21T09:58:47.378 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/char/ipmi/ipmi_msghandler.ko 2024-03-21T09:58:47.378 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/char/ipmi/ipmi_poweroff.ko 2024-03-21T09:58:47.378 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/char/ipmi/ipmi_si.ko 2024-03-21T09:58:47.378 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/char/ipmi/ipmi_ssif.ko 2024-03-21T09:58:47.378 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/char/ipmi/ipmi_watchdog.ko 2024-03-21T09:58:47.378 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/char/lp.ko 2024-03-21T09:58:47.379 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/char/mwave 2024-03-21T09:58:47.379 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/char/mwave/mwave.ko 2024-03-21T09:58:47.379 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/char/ppdev.ko 2024-03-21T09:58:47.379 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/char/tlclk.ko 2024-03-21T09:58:47.379 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/char/tpm 2024-03-21T09:58:47.379 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/char/tpm/tpm.ko 2024-03-21T09:58:47.379 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/char/tpm/tpm_atmel.ko 2024-03-21T09:58:47.379 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/char/tpm/tpm_infineon.ko 2024-03-21T09:58:47.379 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/char/tpm/tpm_nsc.ko 2024-03-21T09:58:47.379 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/char/tpm/tpm_tis.ko 2024-03-21T09:58:47.379 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/char/tpm/tpm_tis_core.ko 2024-03-21T09:58:47.379 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/char/uv_mmtimer.ko 2024-03-21T09:58:47.379 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/char/virtio_console.ko 2024-03-21T09:58:47.379 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/cpufreq 2024-03-21T09:58:47.379 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/cpufreq/acpi-cpufreq.ko 2024-03-21T09:58:47.380 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/cpufreq/amd_freq_sensitivity.ko 2024-03-21T09:58:47.380 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/cpufreq/p4-clockmod.ko 2024-03-21T09:58:47.380 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/cpufreq/pcc-cpufreq.ko 2024-03-21T09:58:47.380 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/cpufreq/powernow-k8.ko 2024-03-21T09:58:47.380 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/cpufreq/speedstep-lib.ko 2024-03-21T09:58:47.380 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/crypto 2024-03-21T09:58:47.380 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/crypto/ccp 2024-03-21T09:58:47.380 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/crypto/ccp/ccp-crypto.ko 2024-03-21T09:58:47.380 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/crypto/ccp/ccp.ko 2024-03-21T09:58:47.380 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/crypto/intel 2024-03-21T09:58:47.380 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/crypto/intel/qat 2024-03-21T09:58:47.380 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/crypto/intel/qat/qat_common 2024-03-21T09:58:47.380 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/crypto/intel/qat/qat_common/intel_qat.ko 2024-03-21T09:58:47.380 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/crypto/intel/qat/qat_dh895xcc 2024-03-21T09:58:47.381 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/crypto/intel/qat/qat_dh895xcc/qat_dh895xcc.ko 2024-03-21T09:58:47.381 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/crypto/padlock-aes.ko 2024-03-21T09:58:47.381 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/crypto/padlock-sha.ko 2024-03-21T09:58:47.381 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/crypto/virtio 2024-03-21T09:58:47.381 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/crypto/virtio/virtio_crypto.ko 2024-03-21T09:58:47.381 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/dca 2024-03-21T09:58:47.381 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/dca/dca.ko 2024-03-21T09:58:47.381 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/devfreq 2024-03-21T09:58:47.381 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/devfreq/governor_simpleondemand.ko 2024-03-21T09:58:47.381 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/dma 2024-03-21T09:58:47.381 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/dma/dw 2024-03-21T09:58:47.381 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/dma/dw/dw_dmac.ko 2024-03-21T09:58:47.381 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/dma/ioat 2024-03-21T09:58:47.381 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/dma/ioat/ioatdma.ko 2024-03-21T09:58:47.381 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/edac 2024-03-21T09:58:47.382 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/edac/amd64_edac.ko 2024-03-21T09:58:47.382 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/edac/e752x_edac.ko 2024-03-21T09:58:47.382 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/edac/edac_mce_amd.ko 2024-03-21T09:58:47.382 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/edac/i3000_edac.ko 2024-03-21T09:58:47.382 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/edac/i3200_edac.ko 2024-03-21T09:58:47.382 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/edac/i5100_edac.ko 2024-03-21T09:58:47.382 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/edac/i5400_edac.ko 2024-03-21T09:58:47.382 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/edac/i7300_edac.ko 2024-03-21T09:58:47.382 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/edac/i7core_edac.ko 2024-03-21T09:58:47.382 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/edac/i82975x_edac.ko 2024-03-21T09:58:47.382 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/edac/ie31200_edac.ko 2024-03-21T09:58:47.382 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/edac/sb_edac.ko 2024-03-21T09:58:47.382 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/edac/x38_edac.ko 2024-03-21T09:58:47.382 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/firmware 2024-03-21T09:58:47.383 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/firmware/edd.ko 2024-03-21T09:58:47.383 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/firmware/iscsi_ibft.ko 2024-03-21T09:58:47.383 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/gpio 2024-03-21T09:58:47.383 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/gpio/gpio-ich.ko 2024-03-21T09:58:47.383 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/gpio/gpio-viperboard.ko 2024-03-21T09:58:47.383 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid 2024-03-21T09:58:47.383 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-appleir.ko 2024-03-21T09:58:47.383 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-aureal.ko 2024-03-21T09:58:47.383 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-axff.ko 2024-03-21T09:58:47.383 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-dr.ko 2024-03-21T09:58:47.383 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-elecom.ko 2024-03-21T09:58:47.383 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-elo.ko 2024-03-21T09:58:47.383 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-emsff.ko 2024-03-21T09:58:47.383 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-gaff.ko 2024-03-21T09:58:47.384 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-gt683r.ko 2024-03-21T09:58:47.384 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-gyration.ko 2024-03-21T09:58:47.384 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-holtek-kbd.ko 2024-03-21T09:58:47.384 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-holtek-mouse.ko 2024-03-21T09:58:47.384 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-holtekff.ko 2024-03-21T09:58:47.384 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-hyperv.ko 2024-03-21T09:58:47.384 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-icade.ko 2024-03-21T09:58:47.384 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-keytouch.ko 2024-03-21T09:58:47.384 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-kye.ko 2024-03-21T09:58:47.384 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-lcpower.ko 2024-03-21T09:58:47.384 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-led.ko 2024-03-21T09:58:47.384 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-lenovo.ko 2024-03-21T09:58:47.384 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-logitech-dj.ko 2024-03-21T09:58:47.384 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-logitech-hidpp.ko 2024-03-21T09:58:47.385 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-multitouch.ko 2024-03-21T09:58:47.385 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-ortek.ko 2024-03-21T09:58:47.385 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-penmount.ko 2024-03-21T09:58:47.385 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-petalynx.ko 2024-03-21T09:58:47.385 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-picolcd.ko 2024-03-21T09:58:47.385 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-pl.ko 2024-03-21T09:58:47.385 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-plantronics.ko 2024-03-21T09:58:47.385 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-primax.ko 2024-03-21T09:58:47.385 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-rmi.ko 2024-03-21T09:58:47.385 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-roccat-arvo.ko 2024-03-21T09:58:47.385 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-roccat-common.ko 2024-03-21T09:58:47.385 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-roccat-isku.ko 2024-03-21T09:58:47.385 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-roccat-kone.ko 2024-03-21T09:58:47.385 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-roccat-koneplus.ko 2024-03-21T09:58:47.386 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-roccat-konepure.ko 2024-03-21T09:58:47.386 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-roccat-kovaplus.ko 2024-03-21T09:58:47.386 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-roccat-lua.ko 2024-03-21T09:58:47.386 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-roccat-pyra.ko 2024-03-21T09:58:47.386 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-roccat-ryos.ko 2024-03-21T09:58:47.386 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-roccat-savu.ko 2024-03-21T09:58:47.386 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-roccat.ko 2024-03-21T09:58:47.386 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-saitek.ko 2024-03-21T09:58:47.386 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-samsung.ko 2024-03-21T09:58:47.386 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-sensor-hub.ko 2024-03-21T09:58:47.386 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-sjoy.ko 2024-03-21T09:58:47.386 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-sony.ko 2024-03-21T09:58:47.386 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-speedlink.ko 2024-03-21T09:58:47.387 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-steelseries.ko 2024-03-21T09:58:47.387 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-sunplus.ko 2024-03-21T09:58:47.387 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-thrustmaster.ko 2024-03-21T09:58:47.387 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-tivo.ko 2024-03-21T09:58:47.387 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-tmff.ko 2024-03-21T09:58:47.387 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-topseed.ko 2024-03-21T09:58:47.387 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-twinhan.ko 2024-03-21T09:58:47.387 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-uclogic.ko 2024-03-21T09:58:47.387 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-waltop.ko 2024-03-21T09:58:47.387 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-wiimote.ko 2024-03-21T09:58:47.387 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-xinmo.ko 2024-03-21T09:58:47.387 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-zpff.ko 2024-03-21T09:58:47.387 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-zydacron.ko 2024-03-21T09:58:47.387 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/uhid.ko 2024-03-21T09:58:47.387 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/wacom.ko 2024-03-21T09:58:47.388 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hv 2024-03-21T09:58:47.388 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hv/hv_balloon.ko 2024-03-21T09:58:47.388 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hv/hv_utils.ko 2024-03-21T09:58:47.388 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hv/hv_vmbus.ko 2024-03-21T09:58:47.388 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon 2024-03-21T09:58:47.388 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/abituguru.ko 2024-03-21T09:58:47.388 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/abituguru3.ko 2024-03-21T09:58:47.388 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/acpi_power_meter.ko 2024-03-21T09:58:47.388 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/ad7414.ko 2024-03-21T09:58:47.388 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/ad7418.ko 2024-03-21T09:58:47.388 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/adc128d818.ko 2024-03-21T09:58:47.388 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/adm1025.ko 2024-03-21T09:58:47.388 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/adm1026.ko 2024-03-21T09:58:47.388 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/adm1029.ko 2024-03-21T09:58:47.389 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/adm1031.ko 2024-03-21T09:58:47.389 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/adm9240.ko 2024-03-21T09:58:47.389 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/ads7828.ko 2024-03-21T09:58:47.389 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/adt7410.ko 2024-03-21T09:58:47.389 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/adt7411.ko 2024-03-21T09:58:47.389 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/adt7462.ko 2024-03-21T09:58:47.389 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/adt7470.ko 2024-03-21T09:58:47.389 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/adt7475.ko 2024-03-21T09:58:47.389 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/adt7x10.ko 2024-03-21T09:58:47.389 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/amc6821.ko 2024-03-21T09:58:47.389 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/applesmc.ko 2024-03-21T09:58:47.389 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/asb100.ko 2024-03-21T09:58:47.389 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/asc7621.ko 2024-03-21T09:58:47.389 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/asus_atk0110.ko 2024-03-21T09:58:47.389 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/atxp1.ko 2024-03-21T09:58:47.390 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/coretemp.ko 2024-03-21T09:58:47.390 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/dell-smm-hwmon.ko 2024-03-21T09:58:47.390 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/dme1737.ko 2024-03-21T09:58:47.390 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/ds1621.ko 2024-03-21T09:58:47.390 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/ds620.ko 2024-03-21T09:58:47.390 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/emc1403.ko 2024-03-21T09:58:47.390 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/emc6w201.ko 2024-03-21T09:58:47.390 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/f71805f.ko 2024-03-21T09:58:47.390 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/f71882fg.ko 2024-03-21T09:58:47.390 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/f75375s.ko 2024-03-21T09:58:47.390 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/fam15h_power.ko 2024-03-21T09:58:47.390 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/fschmd.ko 2024-03-21T09:58:47.390 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/g760a.ko 2024-03-21T09:58:47.390 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/g762.ko 2024-03-21T09:58:47.391 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/gl518sm.ko 2024-03-21T09:58:47.391 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/gl520sm.ko 2024-03-21T09:58:47.391 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/hwmon-vid.ko 2024-03-21T09:58:47.391 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/i5500_temp.ko 2024-03-21T09:58:47.391 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/i5k_amb.ko 2024-03-21T09:58:47.391 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/ibmaem.ko 2024-03-21T09:58:47.391 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/ibmpex.ko 2024-03-21T09:58:47.391 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/ina209.ko 2024-03-21T09:58:47.391 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/ina2xx.ko 2024-03-21T09:58:47.391 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/it87.ko 2024-03-21T09:58:47.391 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/k10temp.ko 2024-03-21T09:58:47.391 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/k8temp.ko 2024-03-21T09:58:47.391 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/lineage-pem.ko 2024-03-21T09:58:47.391 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/lm63.ko 2024-03-21T09:58:47.391 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/lm73.ko 2024-03-21T09:58:47.392 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/lm75.ko 2024-03-21T09:58:47.392 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/lm77.ko 2024-03-21T09:58:47.392 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/lm78.ko 2024-03-21T09:58:47.392 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/lm80.ko 2024-03-21T09:58:47.392 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/lm83.ko 2024-03-21T09:58:47.392 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/lm85.ko 2024-03-21T09:58:47.392 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/lm87.ko 2024-03-21T09:58:47.392 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/lm90.ko 2024-03-21T09:58:47.392 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/lm92.ko 2024-03-21T09:58:47.392 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/lm93.ko 2024-03-21T09:58:47.392 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/lm95234.ko 2024-03-21T09:58:47.392 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/lm95241.ko 2024-03-21T09:58:47.392 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/lm95245.ko 2024-03-21T09:58:47.392 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/ltc2945.ko 2024-03-21T09:58:47.392 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/ltc4151.ko 2024-03-21T09:58:47.393 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/ltc4215.ko 2024-03-21T09:58:47.393 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/ltc4222.ko 2024-03-21T09:58:47.393 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/ltc4245.ko 2024-03-21T09:58:47.393 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/ltc4260.ko 2024-03-21T09:58:47.393 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/ltc4261.ko 2024-03-21T09:58:47.393 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/max16065.ko 2024-03-21T09:58:47.393 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/max1619.ko 2024-03-21T09:58:47.393 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/max1668.ko 2024-03-21T09:58:47.393 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/max197.ko 2024-03-21T09:58:47.393 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/max6639.ko 2024-03-21T09:58:47.393 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/max6650.ko 2024-03-21T09:58:47.393 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/max6697.ko 2024-03-21T09:58:47.393 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/mcp3021.ko 2024-03-21T09:58:47.393 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/nct6683.ko 2024-03-21T09:58:47.394 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/nct6775-core.ko 2024-03-21T09:58:47.394 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/nct6775.ko 2024-03-21T09:58:47.394 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/nct7802.ko 2024-03-21T09:58:47.394 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/pc87360.ko 2024-03-21T09:58:47.394 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/pc87427.ko 2024-03-21T09:58:47.394 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/pcf8591.ko 2024-03-21T09:58:47.394 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/pmbus 2024-03-21T09:58:47.394 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/pmbus/adm1275.ko 2024-03-21T09:58:47.394 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/pmbus/lm25066.ko 2024-03-21T09:58:47.394 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/pmbus/ltc2978.ko 2024-03-21T09:58:47.394 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/pmbus/max16064.ko 2024-03-21T09:58:47.394 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/pmbus/max34440.ko 2024-03-21T09:58:47.394 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/pmbus/max8688.ko 2024-03-21T09:58:47.394 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/pmbus/pmbus.ko 2024-03-21T09:58:47.395 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/pmbus/pmbus_core.ko 2024-03-21T09:58:47.395 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/pmbus/tps40422.ko 2024-03-21T09:58:47.395 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/pmbus/ucd9000.ko 2024-03-21T09:58:47.395 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/pmbus/ucd9200.ko 2024-03-21T09:58:47.395 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/pmbus/zl6100.ko 2024-03-21T09:58:47.395 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/powr1220.ko 2024-03-21T09:58:47.395 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/sch5627.ko 2024-03-21T09:58:47.395 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/sch5636.ko 2024-03-21T09:58:47.395 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/sch56xx-common.ko 2024-03-21T09:58:47.395 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/sht15.ko 2024-03-21T09:58:47.395 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/sht21.ko 2024-03-21T09:58:47.395 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/shtc1.ko 2024-03-21T09:58:47.395 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/sis5595.ko 2024-03-21T09:58:47.396 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/smsc47b397.ko 2024-03-21T09:58:47.396 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/smsc47m1.ko 2024-03-21T09:58:47.396 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/smsc47m192.ko 2024-03-21T09:58:47.396 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/thmc50.ko 2024-03-21T09:58:47.396 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/tmp102.ko 2024-03-21T09:58:47.396 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/tmp103.ko 2024-03-21T09:58:47.396 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/tmp401.ko 2024-03-21T09:58:47.396 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/tmp421.ko 2024-03-21T09:58:47.396 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/via-cputemp.ko 2024-03-21T09:58:47.396 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/via686a.ko 2024-03-21T09:58:47.396 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/vt1211.ko 2024-03-21T09:58:47.396 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/vt8231.ko 2024-03-21T09:58:47.396 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/w83627ehf.ko 2024-03-21T09:58:47.396 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/w83627hf.ko 2024-03-21T09:58:47.397 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/w83781d.ko 2024-03-21T09:58:47.397 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/w83791d.ko 2024-03-21T09:58:47.397 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/w83792d.ko 2024-03-21T09:58:47.397 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/w83793.ko 2024-03-21T09:58:47.397 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/w83795.ko 2024-03-21T09:58:47.397 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/w83l785ts.ko 2024-03-21T09:58:47.397 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/w83l786ng.ko 2024-03-21T09:58:47.397 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/i2c 2024-03-21T09:58:47.397 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/i2c/algos 2024-03-21T09:58:47.397 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/i2c/algos/i2c-algo-bit.ko 2024-03-21T09:58:47.397 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/i2c/algos/i2c-algo-pca.ko 2024-03-21T09:58:47.397 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/i2c/busses 2024-03-21T09:58:47.397 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/i2c/busses/i2c-amd756-s4882.ko 2024-03-21T09:58:47.397 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/i2c/busses/i2c-amd756.ko 2024-03-21T09:58:47.398 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/i2c/busses/i2c-amd8111.ko 2024-03-21T09:58:47.398 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/i2c/busses/i2c-ccgx-ucsi.ko 2024-03-21T09:58:47.398 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/i2c/busses/i2c-designware-core.ko 2024-03-21T09:58:47.398 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/i2c/busses/i2c-designware-pci.ko 2024-03-21T09:58:47.398 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/i2c/busses/i2c-designware-platform.ko 2024-03-21T09:58:47.398 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/i2c/busses/i2c-diolan-u2c.ko 2024-03-21T09:58:47.398 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/i2c/busses/i2c-i801.ko 2024-03-21T09:58:47.398 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/i2c/busses/i2c-isch.ko 2024-03-21T09:58:47.398 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/i2c/busses/i2c-ismt.ko 2024-03-21T09:58:47.398 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/i2c/busses/i2c-nforce2-s4985.ko 2024-03-21T09:58:47.398 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/i2c/busses/i2c-nforce2.ko 2024-03-21T09:58:47.398 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/i2c/busses/i2c-parport.ko 2024-03-21T09:58:47.398 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/i2c/busses/i2c-pca-platform.ko 2024-03-21T09:58:47.398 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/i2c/busses/i2c-piix4.ko 2024-03-21T09:58:47.399 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/i2c/busses/i2c-scmi.ko 2024-03-21T09:58:47.399 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/i2c/busses/i2c-simtec.ko 2024-03-21T09:58:47.399 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/i2c/busses/i2c-sis96x.ko 2024-03-21T09:58:47.399 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/i2c/busses/i2c-tiny-usb.ko 2024-03-21T09:58:47.399 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/i2c/busses/i2c-via.ko 2024-03-21T09:58:47.399 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/i2c/busses/i2c-viapro.ko 2024-03-21T09:58:47.399 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/i2c/busses/i2c-viperboard.ko 2024-03-21T09:58:47.399 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/i2c/i2c-dev.ko 2024-03-21T09:58:47.399 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/i2c/i2c-mux.ko 2024-03-21T09:58:47.399 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/i2c/i2c-slave-eeprom.ko 2024-03-21T09:58:47.399 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/i2c/i2c-smbus.ko 2024-03-21T09:58:47.399 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/i2c/i2c-stub.ko 2024-03-21T09:58:47.399 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/infiniband 2024-03-21T09:58:47.399 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/infiniband/core 2024-03-21T09:58:47.399 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/infiniband/core/ib_cm.ko 2024-03-21T09:58:47.400 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/infiniband/core/ib_core.ko 2024-03-21T09:58:47.400 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/infiniband/core/ib_umad.ko 2024-03-21T09:58:47.400 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/infiniband/core/ib_uverbs.ko 2024-03-21T09:58:47.400 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/infiniband/core/iw_cm.ko 2024-03-21T09:58:47.400 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/infiniband/core/rdma_cm.ko 2024-03-21T09:58:47.400 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/infiniband/core/rdma_ucm.ko 2024-03-21T09:58:47.400 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/infiniband/hw 2024-03-21T09:58:47.400 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/infiniband/hw/cxgb4 2024-03-21T09:58:47.400 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/infiniband/hw/cxgb4/iw_cxgb4.ko 2024-03-21T09:58:47.400 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/infiniband/hw/mlx4 2024-03-21T09:58:47.400 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/infiniband/hw/mlx4/mlx4_ib.ko 2024-03-21T09:58:47.400 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/infiniband/hw/mlx5 2024-03-21T09:58:47.400 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/infiniband/hw/mlx5/mlx5_ib.ko 2024-03-21T09:58:47.400 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/infiniband/hw/mthca 2024-03-21T09:58:47.400 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/infiniband/hw/mthca/ib_mthca.ko 2024-03-21T09:58:47.401 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/infiniband/ulp 2024-03-21T09:58:47.401 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/infiniband/ulp/ipoib 2024-03-21T09:58:47.401 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/infiniband/ulp/ipoib/ib_ipoib.ko 2024-03-21T09:58:47.401 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/infiniband/ulp/iser 2024-03-21T09:58:47.401 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/infiniband/ulp/iser/ib_iser.ko 2024-03-21T09:58:47.401 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/infiniband/ulp/isert 2024-03-21T09:58:47.401 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/infiniband/ulp/isert/ib_isert.ko 2024-03-21T09:58:47.401 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/infiniband/ulp/srp 2024-03-21T09:58:47.401 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/infiniband/ulp/srp/ib_srp.ko 2024-03-21T09:58:47.401 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/infiniband/ulp/srpt 2024-03-21T09:58:47.401 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/infiniband/ulp/srpt/ib_srpt.ko 2024-03-21T09:58:47.401 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/input 2024-03-21T09:58:47.401 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/input/gameport 2024-03-21T09:58:47.401 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/input/gameport/emu10k1-gp.ko 2024-03-21T09:58:47.401 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/input/gameport/fm801-gp.ko 2024-03-21T09:58:47.402 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/input/gameport/gameport.ko 2024-03-21T09:58:47.402 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/input/joydev.ko 2024-03-21T09:58:47.402 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/input/keyboard 2024-03-21T09:58:47.402 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/input/keyboard/gpio_keys.ko 2024-03-21T09:58:47.402 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/input/mouse 2024-03-21T09:58:47.402 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/input/mouse/appletouch.ko 2024-03-21T09:58:47.402 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/input/mouse/bcm5974.ko 2024-03-21T09:58:47.402 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/input/mouse/cyapatp.ko 2024-03-21T09:58:47.402 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/input/mouse/elan_i2c.ko 2024-03-21T09:58:47.402 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/input/mouse/sermouse.ko 2024-03-21T09:58:47.402 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/input/mouse/synaptics_i2c.ko 2024-03-21T09:58:47.402 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/input/mouse/synaptics_usb.ko 2024-03-21T09:58:47.402 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/input/mouse/vsxxxaa.ko 2024-03-21T09:58:47.402 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/input/rmi4 2024-03-21T09:58:47.403 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/input/rmi4/rmi_core.ko 2024-03-21T09:58:47.403 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/input/serio 2024-03-21T09:58:47.403 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/input/serio/altera_ps2.ko 2024-03-21T09:58:47.403 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/input/serio/arc_ps2.ko 2024-03-21T09:58:47.403 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/input/serio/hyperv-keyboard.ko 2024-03-21T09:58:47.403 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/input/serio/serio_raw.ko 2024-03-21T09:58:47.403 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/input/sparse-keymap.ko 2024-03-21T09:58:47.403 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/leds 2024-03-21T09:58:47.403 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/leds/leds-blinkm.ko 2024-03-21T09:58:47.403 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/leds/leds-lm3530.ko 2024-03-21T09:58:47.403 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/leds/leds-lp3944.ko 2024-03-21T09:58:47.403 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/leds/leds-ss4200.ko 2024-03-21T09:58:47.403 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/leds/trigger 2024-03-21T09:58:47.403 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/leds/trigger/ledtrig-audio.ko 2024-03-21T09:58:47.404 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/leds/trigger/ledtrig-backlight.ko 2024-03-21T09:58:47.404 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/leds/trigger/ledtrig-camera.ko 2024-03-21T09:58:47.404 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/leds/trigger/ledtrig-default-on.ko 2024-03-21T09:58:47.404 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/leds/trigger/ledtrig-gpio.ko 2024-03-21T09:58:47.404 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/leds/trigger/ledtrig-heartbeat.ko 2024-03-21T09:58:47.404 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/leds/trigger/ledtrig-oneshot.ko 2024-03-21T09:58:47.404 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/leds/trigger/ledtrig-timer.ko 2024-03-21T09:58:47.404 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/leds/trigger/ledtrig-transient.ko 2024-03-21T09:58:47.404 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/md 2024-03-21T09:58:47.404 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/md/bcache 2024-03-21T09:58:47.404 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/md/bcache/bcache.ko 2024-03-21T09:58:47.404 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/md/dm-bio-prison.ko 2024-03-21T09:58:47.404 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/md/dm-cache-smq.ko 2024-03-21T09:58:47.404 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/md/dm-cache.ko 2024-03-21T09:58:47.405 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/md/dm-crypt.ko 2024-03-21T09:58:47.405 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/md/dm-delay.ko 2024-03-21T09:58:47.405 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/md/dm-flakey.ko 2024-03-21T09:58:47.405 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/md/dm-log-userspace.ko 2024-03-21T09:58:47.405 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/md/dm-multipath.ko 2024-03-21T09:58:47.405 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/md/dm-queue-length.ko 2024-03-21T09:58:47.405 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/md/dm-raid.ko 2024-03-21T09:58:47.405 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/md/dm-round-robin.ko 2024-03-21T09:58:47.405 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/md/dm-service-time.ko 2024-03-21T09:58:47.405 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/md/dm-switch.ko 2024-03-21T09:58:47.405 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/md/dm-thin-pool.ko 2024-03-21T09:58:47.405 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/md/dm-verity.ko 2024-03-21T09:58:47.405 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/md/persistent-data 2024-03-21T09:58:47.405 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/md/persistent-data/dm-persistent-data.ko 2024-03-21T09:58:47.405 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/md/raid0.ko 2024-03-21T09:58:47.405 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/md/raid1.ko 2024-03-21T09:58:47.406 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/md/raid10.ko 2024-03-21T09:58:47.406 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/md/raid456.ko 2024-03-21T09:58:47.406 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/message 2024-03-21T09:58:47.406 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/message/fusion 2024-03-21T09:58:47.406 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/message/fusion/mptbase.ko 2024-03-21T09:58:47.406 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/message/fusion/mptctl.ko 2024-03-21T09:58:47.406 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/message/fusion/mptfc.ko 2024-03-21T09:58:47.406 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/message/fusion/mptlan.ko 2024-03-21T09:58:47.406 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/message/fusion/mptsas.ko 2024-03-21T09:58:47.406 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/message/fusion/mptscsih.ko 2024-03-21T09:58:47.406 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/message/fusion/mptspi.ko 2024-03-21T09:58:47.406 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/mfd 2024-03-21T09:58:47.406 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/mfd/lpc_ich.ko 2024-03-21T09:58:47.406 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/mfd/lpc_sch.ko 2024-03-21T09:58:47.407 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/mfd/mfd-core.ko 2024-03-21T09:58:47.407 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/mfd/sm501.ko 2024-03-21T09:58:47.407 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/mfd/viperboard.ko 2024-03-21T09:58:47.407 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/mfd/vx855.ko 2024-03-21T09:58:47.407 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/mfd/wl1273-core.ko 2024-03-21T09:58:47.407 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/misc 2024-03-21T09:58:47.407 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/misc/altera-stapl 2024-03-21T09:58:47.407 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/misc/altera-stapl/altera-stapl.ko 2024-03-21T09:58:47.407 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/misc/apds9802als.ko 2024-03-21T09:58:47.407 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/misc/apds990x.ko 2024-03-21T09:58:47.407 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/misc/bh1770glc.ko 2024-03-21T09:58:47.407 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/misc/cb710 2024-03-21T09:58:47.407 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/misc/cb710/cb710.ko 2024-03-21T09:58:47.407 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/misc/eeprom 2024-03-21T09:58:47.407 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/misc/eeprom/at24.ko 2024-03-21T09:58:47.407 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/misc/eeprom/eeprom_93cx6.ko 2024-03-21T09:58:47.408 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/misc/eeprom/max6875.ko 2024-03-21T09:58:47.408 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/misc/enclosure.ko 2024-03-21T09:58:47.408 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/misc/hpilo.ko 2024-03-21T09:58:47.408 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/misc/isl29003.ko 2024-03-21T09:58:47.408 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/misc/isl29020.ko 2024-03-21T09:58:47.408 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/misc/lis3lv02d 2024-03-21T09:58:47.408 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/misc/lis3lv02d/lis3lv02d.ko 2024-03-21T09:58:47.408 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/misc/lis3lv02d/lis3lv02d_i2c.ko 2024-03-21T09:58:47.408 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/misc/mei 2024-03-21T09:58:47.408 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/misc/mei/mei-me.ko 2024-03-21T09:58:47.408 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/misc/mei/mei-txe.ko 2024-03-21T09:58:47.408 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/misc/mei/mei.ko 2024-03-21T09:58:47.408 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/misc/sgi-gru 2024-03-21T09:58:47.408 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/misc/sgi-gru/gru.ko 2024-03-21T09:58:47.409 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/misc/sgi-xp 2024-03-21T09:58:47.409 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/misc/sgi-xp/xp.ko 2024-03-21T09:58:47.409 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/misc/sgi-xp/xpc.ko 2024-03-21T09:58:47.409 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/misc/sgi-xp/xpnet.ko 2024-03-21T09:58:47.409 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/misc/tifm_7xx1.ko 2024-03-21T09:58:47.409 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/misc/tifm_core.ko 2024-03-21T09:58:47.409 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/misc/tsl2550.ko 2024-03-21T09:58:47.409 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/misc/vmw_balloon.ko 2024-03-21T09:58:47.409 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/misc/vmw_vmci 2024-03-21T09:58:47.409 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/misc/vmw_vmci/vmw_vmci.ko 2024-03-21T09:58:47.409 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net 2024-03-21T09:58:47.409 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/bonding 2024-03-21T09:58:47.409 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/bonding/bonding.ko 2024-03-21T09:58:47.409 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/dsa 2024-03-21T09:58:47.410 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/dsa/b53 2024-03-21T09:58:47.410 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/dsa/b53/b53_common.ko 2024-03-21T09:58:47.410 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/dsa/bcm-sf2.ko 2024-03-21T09:58:47.410 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/dsa/mv88e6060.ko 2024-03-21T09:58:47.410 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/dsa/mv88e6xxx 2024-03-21T09:58:47.410 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/dsa/mv88e6xxx/mv88e6xxx.ko 2024-03-21T09:58:47.410 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/dummy.ko 2024-03-21T09:58:47.410 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/eql.ko 2024-03-21T09:58:47.410 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet 2024-03-21T09:58:47.410 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/3com 2024-03-21T09:58:47.410 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/3com/3c574_cs.ko 2024-03-21T09:58:47.410 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/3com/3c589_cs.ko 2024-03-21T09:58:47.410 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/3com/3c59x.ko 2024-03-21T09:58:47.410 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/3com/typhoon.ko 2024-03-21T09:58:47.411 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/8390 2024-03-21T09:58:47.411 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/8390/8390.ko 2024-03-21T09:58:47.411 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/8390/axnet_cs.ko 2024-03-21T09:58:47.411 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/8390/ne2k-pci.ko 2024-03-21T09:58:47.411 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/8390/pcnet_cs.ko 2024-03-21T09:58:47.411 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/adaptec 2024-03-21T09:58:47.411 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/adaptec/starfire.ko 2024-03-21T09:58:47.411 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/agere 2024-03-21T09:58:47.411 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/agere/et131x.ko 2024-03-21T09:58:47.411 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/alteon 2024-03-21T09:58:47.411 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/alteon/acenic.ko 2024-03-21T09:58:47.411 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/altera 2024-03-21T09:58:47.411 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/altera/altera_tse.ko 2024-03-21T09:58:47.411 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/amd 2024-03-21T09:58:47.412 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/amd/amd8111e.ko 2024-03-21T09:58:47.412 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/amd/nmclan_cs.ko 2024-03-21T09:58:47.412 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/amd/pcnet32.ko 2024-03-21T09:58:47.412 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/atheros 2024-03-21T09:58:47.412 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/atheros/alx 2024-03-21T09:58:47.412 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/atheros/alx/alx.ko 2024-03-21T09:58:47.412 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/atheros/atl1c 2024-03-21T09:58:47.412 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/atheros/atl1c/atl1c.ko 2024-03-21T09:58:47.412 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/atheros/atl1e 2024-03-21T09:58:47.412 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/atheros/atl1e/atl1e.ko 2024-03-21T09:58:47.412 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/atheros/atlx 2024-03-21T09:58:47.412 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/atheros/atlx/atl1.ko 2024-03-21T09:58:47.412 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/atheros/atlx/atl2.ko 2024-03-21T09:58:47.412 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/broadcom 2024-03-21T09:58:47.413 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/broadcom/b44.ko 2024-03-21T09:58:47.413 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/broadcom/bnx2.ko 2024-03-21T09:58:47.413 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/broadcom/bnx2x 2024-03-21T09:58:47.413 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/broadcom/bnx2x/bnx2x.ko 2024-03-21T09:58:47.413 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/broadcom/cnic.ko 2024-03-21T09:58:47.413 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/broadcom/genet 2024-03-21T09:58:47.413 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/broadcom/genet/genet.ko 2024-03-21T09:58:47.413 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/broadcom/tg3.ko 2024-03-21T09:58:47.413 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/brocade 2024-03-21T09:58:47.413 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/brocade/bna 2024-03-21T09:58:47.413 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/brocade/bna/bna.ko 2024-03-21T09:58:47.413 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/chelsio 2024-03-21T09:58:47.413 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/chelsio/cxgb 2024-03-21T09:58:47.413 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/chelsio/cxgb/cxgb.ko 2024-03-21T09:58:47.413 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/chelsio/cxgb3 2024-03-21T09:58:47.414 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/chelsio/cxgb3/cxgb3.ko 2024-03-21T09:58:47.414 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/chelsio/cxgb4 2024-03-21T09:58:47.414 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/chelsio/cxgb4/cxgb4.ko 2024-03-21T09:58:47.414 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/chelsio/cxgb4vf 2024-03-21T09:58:47.414 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/chelsio/cxgb4vf/cxgb4vf.ko 2024-03-21T09:58:47.414 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/chelsio/libcxgb 2024-03-21T09:58:47.414 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/chelsio/libcxgb/libcxgb.ko 2024-03-21T09:58:47.414 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/cisco 2024-03-21T09:58:47.414 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/cisco/enic 2024-03-21T09:58:47.414 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/cisco/enic/enic.ko 2024-03-21T09:58:47.414 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/dec 2024-03-21T09:58:47.414 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/dec/tulip 2024-03-21T09:58:47.414 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/dec/tulip/de2104x.ko 2024-03-21T09:58:47.415 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/dec/tulip/dmfe.ko 2024-03-21T09:58:47.415 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/dec/tulip/tulip.ko 2024-03-21T09:58:47.415 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/dec/tulip/uli526x.ko 2024-03-21T09:58:47.415 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/dec/tulip/winbond-840.ko 2024-03-21T09:58:47.415 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/dec/tulip/xircom_cb.ko 2024-03-21T09:58:47.415 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/dlink 2024-03-21T09:58:47.415 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/dlink/dl2k.ko 2024-03-21T09:58:47.415 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/dlink/sundance.ko 2024-03-21T09:58:47.415 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/dnet.ko 2024-03-21T09:58:47.415 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/emulex 2024-03-21T09:58:47.415 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/emulex/benet 2024-03-21T09:58:47.415 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/emulex/benet/be2net.ko 2024-03-21T09:58:47.415 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/ethoc.ko 2024-03-21T09:58:47.415 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/fealnx.ko 2024-03-21T09:58:47.415 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/intel 2024-03-21T09:58:47.416 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/intel/e100.ko 2024-03-21T09:58:47.416 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/intel/e1000 2024-03-21T09:58:47.416 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/intel/e1000/e1000.ko 2024-03-21T09:58:47.416 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/intel/e1000e 2024-03-21T09:58:47.416 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/intel/e1000e/e1000e.ko 2024-03-21T09:58:47.416 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/intel/fm10k 2024-03-21T09:58:47.416 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/intel/fm10k/fm10k.ko 2024-03-21T09:58:47.416 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/intel/i40e 2024-03-21T09:58:47.416 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/intel/i40e/i40e.ko 2024-03-21T09:58:47.416 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/intel/iavf 2024-03-21T09:58:47.416 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/intel/iavf/iavf.ko 2024-03-21T09:58:47.416 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/intel/igb 2024-03-21T09:58:47.416 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/intel/igb/igb.ko 2024-03-21T09:58:47.416 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/intel/igbvf 2024-03-21T09:58:47.417 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/intel/igbvf/igbvf.ko 2024-03-21T09:58:47.417 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/intel/ixgbe 2024-03-21T09:58:47.417 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/intel/ixgbe/ixgbe.ko 2024-03-21T09:58:47.417 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/intel/ixgbevf 2024-03-21T09:58:47.417 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/intel/ixgbevf/ixgbevf.ko 2024-03-21T09:58:47.417 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/jme.ko 2024-03-21T09:58:47.417 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/marvell 2024-03-21T09:58:47.417 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/marvell/mvmdio.ko 2024-03-21T09:58:47.417 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/marvell/skge.ko 2024-03-21T09:58:47.417 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/marvell/sky2.ko 2024-03-21T09:58:47.417 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/mellanox 2024-03-21T09:58:47.417 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/mellanox/mlx4 2024-03-21T09:58:47.417 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_core.ko 2024-03-21T09:58:47.417 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_en.ko 2024-03-21T09:58:47.418 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/mellanox/mlx5 2024-03-21T09:58:47.418 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/mellanox/mlx5/core 2024-03-21T09:58:47.418 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/mellanox/mlx5/core/mlx5_core.ko 2024-03-21T09:58:47.418 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/micrel 2024-03-21T09:58:47.418 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/micrel/ksz884x.ko 2024-03-21T09:58:47.418 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/myricom 2024-03-21T09:58:47.418 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/myricom/myri10ge 2024-03-21T09:58:47.418 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/myricom/myri10ge/myri10ge.ko 2024-03-21T09:58:47.418 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/natsemi 2024-03-21T09:58:47.418 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/natsemi/natsemi.ko 2024-03-21T09:58:47.418 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/natsemi/ns83820.ko 2024-03-21T09:58:47.418 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/neterion 2024-03-21T09:58:47.418 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/neterion/s2io.ko 2024-03-21T09:58:47.419 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/nvidia 2024-03-21T09:58:47.419 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/nvidia/forcedeth.ko 2024-03-21T09:58:47.419 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/packetengines 2024-03-21T09:58:47.419 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/packetengines/hamachi.ko 2024-03-21T09:58:47.419 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/packetengines/yellowfin.ko 2024-03-21T09:58:47.419 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/qlogic 2024-03-21T09:58:47.419 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/qlogic/netxen 2024-03-21T09:58:47.419 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/qlogic/netxen/netxen_nic.ko 2024-03-21T09:58:47.419 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/qlogic/qla3xxx.ko 2024-03-21T09:58:47.419 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/qlogic/qlcnic 2024-03-21T09:58:47.419 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/qlogic/qlcnic/qlcnic.ko 2024-03-21T09:58:47.419 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/rdc 2024-03-21T09:58:47.419 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/rdc/r6040.ko 2024-03-21T09:58:47.419 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/realtek 2024-03-21T09:58:47.420 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/realtek/8139cp.ko 2024-03-21T09:58:47.420 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/realtek/8139too.ko 2024-03-21T09:58:47.420 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/realtek/atp.ko 2024-03-21T09:58:47.420 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/realtek/r8169.ko 2024-03-21T09:58:47.420 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/rocker 2024-03-21T09:58:47.420 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/rocker/rocker.ko 2024-03-21T09:58:47.420 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/sfc 2024-03-21T09:58:47.420 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/sfc/sfc.ko 2024-03-21T09:58:47.420 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/silan 2024-03-21T09:58:47.420 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/silan/sc92031.ko 2024-03-21T09:58:47.420 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/sis 2024-03-21T09:58:47.420 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/sis/sis190.ko 2024-03-21T09:58:47.420 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/sis/sis900.ko 2024-03-21T09:58:47.420 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/smsc 2024-03-21T09:58:47.421 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/smsc/epic100.ko 2024-03-21T09:58:47.421 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/smsc/smc91c92_cs.ko 2024-03-21T09:58:47.421 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/smsc/smsc911x.ko 2024-03-21T09:58:47.421 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/smsc/smsc9420.ko 2024-03-21T09:58:47.421 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/stmicro 2024-03-21T09:58:47.421 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/stmicro/stmmac 2024-03-21T09:58:47.421 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/stmicro/stmmac/dwmac-intel.ko 2024-03-21T09:58:47.421 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/stmicro/stmmac/stmmac.ko 2024-03-21T09:58:47.421 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/sun 2024-03-21T09:58:47.421 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/sun/cassini.ko 2024-03-21T09:58:47.421 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/sun/niu.ko 2024-03-21T09:58:47.421 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/sun/sungem.ko 2024-03-21T09:58:47.421 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/sun/sunhme.ko 2024-03-21T09:58:47.421 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/tehuti 2024-03-21T09:58:47.421 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/tehuti/tehuti.ko 2024-03-21T09:58:47.422 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/ti 2024-03-21T09:58:47.422 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/ti/tlan.ko 2024-03-21T09:58:47.422 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/via 2024-03-21T09:58:47.422 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/via/via-rhine.ko 2024-03-21T09:58:47.422 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/via/via-velocity.ko 2024-03-21T09:58:47.422 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/wiznet 2024-03-21T09:58:47.422 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/wiznet/w5100.ko 2024-03-21T09:58:47.422 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/wiznet/w5300.ko 2024-03-21T09:58:47.422 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/xircom 2024-03-21T09:58:47.422 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/xircom/xirc2ps_cs.ko 2024-03-21T09:58:47.422 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/geneve.ko 2024-03-21T09:58:47.422 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/hyperv 2024-03-21T09:58:47.422 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/hyperv/hv_netvsc.ko 2024-03-21T09:58:47.422 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ieee802154 2024-03-21T09:58:47.423 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ieee802154/fakelb.ko 2024-03-21T09:58:47.423 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ifb.ko 2024-03-21T09:58:47.423 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ipvlan 2024-03-21T09:58:47.423 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ipvlan/ipvlan.ko 2024-03-21T09:58:47.423 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/macvlan.ko 2024-03-21T09:58:47.423 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/macvtap.ko 2024-03-21T09:58:47.423 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/mdio 2024-03-21T09:58:47.423 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/mdio.ko 2024-03-21T09:58:47.423 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/mdio/mdio-bcm-unimac.ko 2024-03-21T09:58:47.423 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/mdio/mdio-bitbang.ko 2024-03-21T09:58:47.423 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/mdio/mdio-regmap.ko 2024-03-21T09:58:47.423 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/mii.ko 2024-03-21T09:58:47.423 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/net_failover.ko 2024-03-21T09:58:47.423 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/netconsole.ko 2024-03-21T09:58:47.424 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/nlmon.ko 2024-03-21T09:58:47.424 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/pcs 2024-03-21T09:58:47.424 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/pcs/pcs-lynx.ko 2024-03-21T09:58:47.424 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/pcs/pcs_xpcs.ko 2024-03-21T09:58:47.424 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/phy 2024-03-21T09:58:47.424 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/phy/amd.ko 2024-03-21T09:58:47.424 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/phy/bcm-phy-lib.ko 2024-03-21T09:58:47.424 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/phy/bcm-phy-ptp.ko 2024-03-21T09:58:47.424 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/phy/bcm7xxx.ko 2024-03-21T09:58:47.424 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/phy/bcm87xx.ko 2024-03-21T09:58:47.424 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/phy/broadcom.ko 2024-03-21T09:58:47.424 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/phy/cicada.ko 2024-03-21T09:58:47.424 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/phy/davicom.ko 2024-03-21T09:58:47.424 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/phy/dp83640.ko 2024-03-21T09:58:47.424 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/phy/et1011c.ko 2024-03-21T09:58:47.425 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/phy/icplus.ko 2024-03-21T09:58:47.425 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/phy/lxt.ko 2024-03-21T09:58:47.425 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/phy/marvell.ko 2024-03-21T09:58:47.425 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/phy/micrel.ko 2024-03-21T09:58:47.425 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/phy/national.ko 2024-03-21T09:58:47.425 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/phy/phylink.ko 2024-03-21T09:58:47.425 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/phy/qsemi.ko 2024-03-21T09:58:47.425 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/phy/realtek.ko 2024-03-21T09:58:47.425 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/phy/smsc.ko 2024-03-21T09:58:47.425 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/phy/ste10Xp.ko 2024-03-21T09:58:47.425 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/phy/vitesse.ko 2024-03-21T09:58:47.425 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ppp 2024-03-21T09:58:47.425 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ppp/bsd_comp.ko 2024-03-21T09:58:47.426 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ppp/ppp_async.ko 2024-03-21T09:58:47.426 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ppp/ppp_deflate.ko 2024-03-21T09:58:47.426 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ppp/ppp_generic.ko 2024-03-21T09:58:47.426 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ppp/ppp_mppe.ko 2024-03-21T09:58:47.426 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ppp/ppp_synctty.ko 2024-03-21T09:58:47.426 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ppp/pppoe.ko 2024-03-21T09:58:47.426 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ppp/pppox.ko 2024-03-21T09:58:47.426 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ppp/pptp.ko 2024-03-21T09:58:47.426 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/slip 2024-03-21T09:58:47.426 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/slip/slhc.ko 2024-03-21T09:58:47.426 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/slip/slip.ko 2024-03-21T09:58:47.426 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/sungem_phy.ko 2024-03-21T09:58:47.426 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/tap.ko 2024-03-21T09:58:47.426 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/team 2024-03-21T09:58:47.426 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/team/team.ko 2024-03-21T09:58:47.427 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/team/team_mode_activebackup.ko 2024-03-21T09:58:47.427 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/team/team_mode_broadcast.ko 2024-03-21T09:58:47.427 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/team/team_mode_loadbalance.ko 2024-03-21T09:58:47.427 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/team/team_mode_random.ko 2024-03-21T09:58:47.427 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/team/team_mode_roundrobin.ko 2024-03-21T09:58:47.427 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/tun.ko 2024-03-21T09:58:47.427 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/veth.ko 2024-03-21T09:58:47.427 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/virtio_net.ko 2024-03-21T09:58:47.427 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/vmxnet3 2024-03-21T09:58:47.427 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/vmxnet3/vmxnet3.ko 2024-03-21T09:58:47.427 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/vxlan 2024-03-21T09:58:47.427 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/vxlan/vxlan.ko 2024-03-21T09:58:47.427 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/xen-netback 2024-03-21T09:58:47.427 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/xen-netback/xen-netback.ko 2024-03-21T09:58:47.427 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/xen-netfront.ko 2024-03-21T09:58:47.427 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ntb 2024-03-21T09:58:47.428 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ntb/ntb.ko 2024-03-21T09:58:47.428 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/nvme 2024-03-21T09:58:47.428 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/nvme/host 2024-03-21T09:58:47.428 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/nvme/host/nvme-core.ko 2024-03-21T09:58:47.428 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/nvme/host/nvme.ko 2024-03-21T09:58:47.428 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/parport 2024-03-21T09:58:47.428 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/parport/parport.ko 2024-03-21T09:58:47.428 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/parport/parport_cs.ko 2024-03-21T09:58:47.428 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/parport/parport_pc.ko 2024-03-21T09:58:47.428 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/parport/parport_serial.ko 2024-03-21T09:58:47.428 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/pci 2024-03-21T09:58:47.428 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/pci/hotplug 2024-03-21T09:58:47.428 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/pci/hotplug/acpiphp_ibm.ko 2024-03-21T09:58:47.428 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/pci/pcie 2024-03-21T09:58:47.428 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/pci/pcie/aer_inject.ko 2024-03-21T09:58:47.429 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/pci/xen-pcifront.ko 2024-03-21T09:58:47.429 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/pcmcia 2024-03-21T09:58:47.429 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/pcmcia/i82092.ko 2024-03-21T09:58:47.429 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/pcmcia/pd6729.ko 2024-03-21T09:58:47.429 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/pcmcia/yenta_socket.ko 2024-03-21T09:58:47.429 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/pinctrl 2024-03-21T09:58:47.429 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/pinctrl/intel 2024-03-21T09:58:47.429 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/pinctrl/intel/pinctrl-cherryview.ko 2024-03-21T09:58:47.429 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/platform 2024-03-21T09:58:47.429 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/platform/x86 2024-03-21T09:58:47.429 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/platform/x86/acer-wmi.ko 2024-03-21T09:58:47.429 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/platform/x86/acerhdf.ko 2024-03-21T09:58:47.429 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/platform/x86/apple-gmux.ko 2024-03-21T09:58:47.429 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/platform/x86/asus-laptop.ko 2024-03-21T09:58:47.429 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/platform/x86/asus-nb-wmi.ko 2024-03-21T09:58:47.429 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/platform/x86/asus-wmi.ko 2024-03-21T09:58:47.429 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/platform/x86/classmate-laptop.ko 2024-03-21T09:58:47.430 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/platform/x86/eeepc-laptop.ko 2024-03-21T09:58:47.430 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/platform/x86/eeepc-wmi.ko 2024-03-21T09:58:47.430 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/platform/x86/fujitsu-laptop.ko 2024-03-21T09:58:47.430 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/platform/x86/fujitsu-tablet.ko 2024-03-21T09:58:47.430 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/platform/x86/hdaps.ko 2024-03-21T09:58:47.430 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/platform/x86/intel 2024-03-21T09:58:47.430 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/platform/x86/intel/intel-rst.ko 2024-03-21T09:58:47.430 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/platform/x86/intel_ips.ko 2024-03-21T09:58:47.430 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/platform/x86/msi-wmi.ko 2024-03-21T09:58:47.430 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/platform/x86/mxm-wmi.ko 2024-03-21T09:58:47.430 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/platform/x86/panasonic-laptop.ko 2024-03-21T09:58:47.430 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/platform/x86/samsung-laptop.ko 2024-03-21T09:58:47.430 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/platform/x86/samsung-q10.ko 2024-03-21T09:58:47.430 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/platform/x86/topstar-laptop.ko 2024-03-21T09:58:47.431 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/platform/x86/toshiba_bluetooth.ko 2024-03-21T09:58:47.431 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/platform/x86/toshiba_haps.ko 2024-03-21T09:58:47.431 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/platform/x86/wmi-bmof.ko 2024-03-21T09:58:47.431 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/platform/x86/wmi.ko 2024-03-21T09:58:47.431 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/powercap 2024-03-21T09:58:47.431 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/powercap/intel_rapl_common.ko 2024-03-21T09:58:47.431 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/powercap/intel_rapl_msr.ko 2024-03-21T09:58:47.431 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/pps 2024-03-21T09:58:47.431 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/pps/clients 2024-03-21T09:58:47.431 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/pps/clients/pps-gpio.ko 2024-03-21T09:58:47.431 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/pps/clients/pps-ldisc.ko 2024-03-21T09:58:47.431 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/pps/clients/pps_parport.ko 2024-03-21T09:58:47.431 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/pps/pps_core.ko 2024-03-21T09:58:47.431 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ptp 2024-03-21T09:58:47.431 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ptp/ptp.ko 2024-03-21T09:58:47.432 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ptp/ptp_kvm.ko 2024-03-21T09:58:47.432 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/rtc 2024-03-21T09:58:47.432 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/rtc/rtc-bq32k.ko 2024-03-21T09:58:47.432 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/rtc/rtc-ds1286.ko 2024-03-21T09:58:47.432 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/rtc/rtc-ds1307.ko 2024-03-21T09:58:47.432 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/rtc/rtc-ds1374.ko 2024-03-21T09:58:47.432 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/rtc/rtc-ds1511.ko 2024-03-21T09:58:47.432 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/rtc/rtc-ds1553.ko 2024-03-21T09:58:47.432 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/rtc/rtc-ds1672.ko 2024-03-21T09:58:47.432 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/rtc/rtc-ds1742.ko 2024-03-21T09:58:47.432 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/rtc/rtc-ds2404.ko 2024-03-21T09:58:47.432 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/rtc/rtc-ds3232.ko 2024-03-21T09:58:47.432 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/rtc/rtc-em3027.ko 2024-03-21T09:58:47.432 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/rtc/rtc-fm3130.ko 2024-03-21T09:58:47.432 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/rtc/rtc-isl12022.ko 2024-03-21T09:58:47.433 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/rtc/rtc-isl1208.ko 2024-03-21T09:58:47.433 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/rtc/rtc-m41t80.ko 2024-03-21T09:58:47.433 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/rtc/rtc-m48t35.ko 2024-03-21T09:58:47.433 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/rtc/rtc-m48t59.ko 2024-03-21T09:58:47.433 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/rtc/rtc-max6900.ko 2024-03-21T09:58:47.433 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/rtc/rtc-msm6242.ko 2024-03-21T09:58:47.433 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/rtc/rtc-pcf2127.ko 2024-03-21T09:58:47.433 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/rtc/rtc-pcf85063.ko 2024-03-21T09:58:47.433 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/rtc/rtc-pcf8523.ko 2024-03-21T09:58:47.433 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/rtc/rtc-pcf8563.ko 2024-03-21T09:58:47.433 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/rtc/rtc-pcf8583.ko 2024-03-21T09:58:47.433 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/rtc/rtc-rp5c01.ko 2024-03-21T09:58:47.433 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/rtc/rtc-rs5c372.ko 2024-03-21T09:58:47.433 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/rtc/rtc-rv3029c2.ko 2024-03-21T09:58:47.433 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/rtc/rtc-rx8025.ko 2024-03-21T09:58:47.434 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/rtc/rtc-rx8581.ko 2024-03-21T09:58:47.434 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/rtc/rtc-stk17ta8.ko 2024-03-21T09:58:47.434 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/rtc/rtc-x1205.ko 2024-03-21T09:58:47.434 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi 2024-03-21T09:58:47.434 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/3w-9xxx.ko 2024-03-21T09:58:47.434 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/3w-sas.ko 2024-03-21T09:58:47.434 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/3w-xxxx.ko 2024-03-21T09:58:47.434 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/BusLogic.ko 2024-03-21T09:58:47.434 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/a100u2w.ko 2024-03-21T09:58:47.434 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/aacraid 2024-03-21T09:58:47.434 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/aacraid/aacraid.ko 2024-03-21T09:58:47.434 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/advansys.ko 2024-03-21T09:58:47.434 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/aic7xxx 2024-03-21T09:58:47.434 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/aic7xxx/aic79xx.ko 2024-03-21T09:58:47.435 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/aic7xxx/aic7xxx.ko 2024-03-21T09:58:47.435 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/am53c974.ko 2024-03-21T09:58:47.435 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/arcmsr 2024-03-21T09:58:47.435 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/arcmsr/arcmsr.ko 2024-03-21T09:58:47.435 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/atp870u.ko 2024-03-21T09:58:47.435 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/be2iscsi 2024-03-21T09:58:47.435 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/be2iscsi/be2iscsi.ko 2024-03-21T09:58:47.435 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/bfa 2024-03-21T09:58:47.435 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/bfa/bfa.ko 2024-03-21T09:58:47.435 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/bnx2fc 2024-03-21T09:58:47.435 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/bnx2fc/bnx2fc.ko 2024-03-21T09:58:47.435 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/bnx2i 2024-03-21T09:58:47.435 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/bnx2i/bnx2i.ko 2024-03-21T09:58:47.435 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/ch.ko 2024-03-21T09:58:47.435 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/csiostor 2024-03-21T09:58:47.436 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/csiostor/csiostor.ko 2024-03-21T09:58:47.436 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/cxgbi 2024-03-21T09:58:47.436 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/cxgbi/cxgb3i 2024-03-21T09:58:47.436 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/cxgbi/cxgb3i/cxgb3i.ko 2024-03-21T09:58:47.436 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/cxgbi/cxgb4i 2024-03-21T09:58:47.436 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/cxgbi/cxgb4i/cxgb4i.ko 2024-03-21T09:58:47.436 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/cxgbi/libcxgbi.ko 2024-03-21T09:58:47.436 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/dc395x.ko 2024-03-21T09:58:47.436 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/device_handler 2024-03-21T09:58:47.436 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/device_handler/scsi_dh_alua.ko 2024-03-21T09:58:47.436 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/device_handler/scsi_dh_emc.ko 2024-03-21T09:58:47.436 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/device_handler/scsi_dh_hp_sw.ko 2024-03-21T09:58:47.436 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/device_handler/scsi_dh_rdac.ko 2024-03-21T09:58:47.436 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/dmx3191d.ko 2024-03-21T09:58:47.437 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/esas2r 2024-03-21T09:58:47.437 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/esas2r/esas2r.ko 2024-03-21T09:58:47.437 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/esp_scsi.ko 2024-03-21T09:58:47.437 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/fcoe 2024-03-21T09:58:47.437 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/fcoe/fcoe.ko 2024-03-21T09:58:47.437 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/fcoe/libfcoe.ko 2024-03-21T09:58:47.437 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/fnic 2024-03-21T09:58:47.437 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/fnic/fnic.ko 2024-03-21T09:58:47.437 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/hpsa.ko 2024-03-21T09:58:47.437 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/hptiop.ko 2024-03-21T09:58:47.437 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/hv_storvsc.ko 2024-03-21T09:58:47.437 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/initio.ko 2024-03-21T09:58:47.437 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/ipr.ko 2024-03-21T09:58:47.437 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/ips.ko 2024-03-21T09:58:47.438 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/isci 2024-03-21T09:58:47.438 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/isci/isci.ko 2024-03-21T09:58:47.438 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/iscsi_boot_sysfs.ko 2024-03-21T09:58:47.438 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/iscsi_tcp.ko 2024-03-21T09:58:47.438 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/libfc 2024-03-21T09:58:47.438 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/libfc/libfc.ko 2024-03-21T09:58:47.438 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/libiscsi.ko 2024-03-21T09:58:47.438 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/libiscsi_tcp.ko 2024-03-21T09:58:47.438 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/libsas 2024-03-21T09:58:47.438 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/libsas/libsas.ko 2024-03-21T09:58:47.438 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/lpfc 2024-03-21T09:58:47.438 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/lpfc/lpfc.ko 2024-03-21T09:58:47.438 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/megaraid 2024-03-21T09:58:47.438 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/megaraid.ko 2024-03-21T09:58:47.439 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/megaraid/megaraid_mbox.ko 2024-03-21T09:58:47.439 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/megaraid/megaraid_mm.ko 2024-03-21T09:58:47.439 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/megaraid/megaraid_sas.ko 2024-03-21T09:58:47.439 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/mpt3sas 2024-03-21T09:58:47.439 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/mpt3sas/mpt3sas.ko 2024-03-21T09:58:47.439 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/mvsas 2024-03-21T09:58:47.439 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/mvsas/mvsas.ko 2024-03-21T09:58:47.439 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/mvumi.ko 2024-03-21T09:58:47.439 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/pm8001 2024-03-21T09:58:47.439 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/pm8001/pm80xx.ko 2024-03-21T09:58:47.439 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/pmcraid.ko 2024-03-21T09:58:47.439 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/qla1280.ko 2024-03-21T09:58:47.439 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/qla2xxx 2024-03-21T09:58:47.439 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/qla2xxx/qla2xxx.ko 2024-03-21T09:58:47.439 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/qla2xxx/tcm_qla2xxx.ko 2024-03-21T09:58:47.440 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/qla4xxx 2024-03-21T09:58:47.440 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/qla4xxx/qla4xxx.ko 2024-03-21T09:58:47.440 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/raid_class.ko 2024-03-21T09:58:47.440 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/scsi_debug.ko 2024-03-21T09:58:47.440 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/scsi_transport_fc.ko 2024-03-21T09:58:47.440 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/scsi_transport_iscsi.ko 2024-03-21T09:58:47.440 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/scsi_transport_sas.ko 2024-03-21T09:58:47.440 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/scsi_transport_spi.ko 2024-03-21T09:58:47.440 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/scsi_transport_srp.ko 2024-03-21T09:58:47.440 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/ses.ko 2024-03-21T09:58:47.440 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/st.ko 2024-03-21T09:58:47.440 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/stex.ko 2024-03-21T09:58:47.440 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/sym53c8xx_2 2024-03-21T09:58:47.440 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/sym53c8xx_2/sym53c8xx.ko 2024-03-21T09:58:47.440 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/virtio_scsi.ko 2024-03-21T09:58:47.441 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/vmw_pvscsi.ko 2024-03-21T09:58:47.441 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/wd719x.ko 2024-03-21T09:58:47.441 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ssb 2024-03-21T09:58:47.441 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ssb/ssb.ko 2024-03-21T09:58:47.441 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/target 2024-03-21T09:58:47.441 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/target/iscsi 2024-03-21T09:58:47.441 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/target/iscsi/iscsi_target_mod.ko 2024-03-21T09:58:47.441 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/target/loopback 2024-03-21T09:58:47.441 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/target/loopback/tcm_loop.ko 2024-03-21T09:58:47.441 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/target/target_core_file.ko 2024-03-21T09:58:47.441 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/target/target_core_iblock.ko 2024-03-21T09:58:47.441 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/target/target_core_mod.ko 2024-03-21T09:58:47.441 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/target/target_core_pscsi.ko 2024-03-21T09:58:47.441 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/target/target_core_user.ko 2024-03-21T09:58:47.441 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/target/tcm_fc 2024-03-21T09:58:47.442 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/target/tcm_fc/tcm_fc.ko 2024-03-21T09:58:47.442 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/thermal 2024-03-21T09:58:47.442 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/thermal/intel 2024-03-21T09:58:47.442 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/thermal/intel/int340x_thermal 2024-03-21T09:58:47.442 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/thermal/intel/int340x_thermal/acpi_thermal_rel.ko 2024-03-21T09:58:47.442 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/thermal/intel/int340x_thermal/int3400_thermal.ko 2024-03-21T09:58:47.442 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/thermal/intel/int340x_thermal/int3401_thermal.ko 2024-03-21T09:58:47.442 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/thermal/intel/int340x_thermal/int3402_thermal.ko 2024-03-21T09:58:47.442 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/thermal/intel/int340x_thermal/int3403_thermal.ko 2024-03-21T09:58:47.442 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/thermal/intel/int340x_thermal/int340x_thermal_zone.ko 2024-03-21T09:58:47.442 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device.ko 2024-03-21T09:58:47.442 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci.ko 2024-03-21T09:58:47.442 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci_legacy.ko 2024-03-21T09:58:47.443 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_mbox.ko 2024-03-21T09:58:47.443 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_power_floor.ko 2024-03-21T09:58:47.443 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rapl.ko 2024-03-21T09:58:47.443 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rfim.ko 2024-03-21T09:58:47.443 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_wt_hint.ko 2024-03-21T09:58:47.443 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_wt_req.ko 2024-03-21T09:58:47.443 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/thermal/intel/intel_soc_dts_iosf.ko 2024-03-21T09:58:47.443 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/thermal/intel/intel_soc_dts_thermal.ko 2024-03-21T09:58:47.443 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/thermal/intel/x86_pkg_temp_thermal.ko 2024-03-21T09:58:47.443 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/tty 2024-03-21T09:58:47.443 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/tty/ipwireless 2024-03-21T09:58:47.443 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/tty/ipwireless/ipwireless.ko 2024-03-21T09:58:47.443 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/tty/n_gsm.ko 2024-03-21T09:58:47.443 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/tty/n_hdlc.ko 2024-03-21T09:58:47.444 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/tty/nozomi.ko 2024-03-21T09:58:47.444 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/tty/serial 2024-03-21T09:58:47.444 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/tty/serial/8250 2024-03-21T09:58:47.444 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/tty/serial/8250/serial_cs.ko 2024-03-21T09:58:47.444 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/tty/serial/arc_uart.ko 2024-03-21T09:58:47.444 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/tty/serial/jsm 2024-03-21T09:58:47.444 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/tty/serial/jsm/jsm.ko 2024-03-21T09:58:47.444 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ufs 2024-03-21T09:58:47.444 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ufs/core 2024-03-21T09:58:47.444 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ufs/core/ufshcd-core.ko 2024-03-21T09:58:47.444 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ufs/host 2024-03-21T09:58:47.444 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ufs/host/ufshcd-pci.ko 2024-03-21T09:58:47.444 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/uio 2024-03-21T09:58:47.444 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/uio/uio.ko 2024-03-21T09:58:47.444 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/uio/uio_aec.ko 2024-03-21T09:58:47.445 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/uio/uio_cif.ko 2024-03-21T09:58:47.445 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/uio/uio_pci_generic.ko 2024-03-21T09:58:47.445 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/uio/uio_sercos3.ko 2024-03-21T09:58:47.445 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb 2024-03-21T09:58:47.445 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/class 2024-03-21T09:58:47.445 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/class/cdc-acm.ko 2024-03-21T09:58:47.445 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/class/cdc-wdm.ko 2024-03-21T09:58:47.445 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/class/usblp.ko 2024-03-21T09:58:47.445 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/class/usbtmc.ko 2024-03-21T09:58:47.445 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/host 2024-03-21T09:58:47.445 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/host/sl811-hcd.ko 2024-03-21T09:58:47.445 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/image 2024-03-21T09:58:47.445 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/image/mdc800.ko 2024-03-21T09:58:47.445 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/image/microtek.ko 2024-03-21T09:58:47.445 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/misc 2024-03-21T09:58:47.446 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/misc/adutux.ko 2024-03-21T09:58:47.446 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/misc/appledisplay.ko 2024-03-21T09:58:47.446 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/misc/emi26.ko 2024-03-21T09:58:47.446 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/misc/emi62.ko 2024-03-21T09:58:47.446 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/misc/ezusb.ko 2024-03-21T09:58:47.446 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/misc/idmouse.ko 2024-03-21T09:58:47.446 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/misc/iowarrior.ko 2024-03-21T09:58:47.446 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/misc/isight_firmware.ko 2024-03-21T09:58:47.446 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/misc/ldusb.ko 2024-03-21T09:58:47.446 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/misc/legousbtower.ko 2024-03-21T09:58:47.446 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/misc/sisusbvga 2024-03-21T09:58:47.446 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/misc/sisusbvga/sisusbvga.ko 2024-03-21T09:58:47.446 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/misc/trancevibrator.ko 2024-03-21T09:58:47.446 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/misc/usb3503.ko 2024-03-21T09:58:47.447 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/misc/usblcd.ko 2024-03-21T09:58:47.447 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/misc/usbsevseg.ko 2024-03-21T09:58:47.447 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/misc/uss720.ko 2024-03-21T09:58:47.447 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/misc/yurex.ko 2024-03-21T09:58:47.447 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/phy 2024-03-21T09:58:47.447 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/phy/phy-generic.ko 2024-03-21T09:58:47.447 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial 2024-03-21T09:58:47.447 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/aircable.ko 2024-03-21T09:58:47.447 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/ark3116.ko 2024-03-21T09:58:47.447 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/belkin_sa.ko 2024-03-21T09:58:47.447 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/ch341.ko 2024-03-21T09:58:47.447 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/cp210x.ko 2024-03-21T09:58:47.447 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/cyberjack.ko 2024-03-21T09:58:47.447 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/cypress_m8.ko 2024-03-21T09:58:47.447 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/digi_acceleport.ko 2024-03-21T09:58:47.448 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/empeg.ko 2024-03-21T09:58:47.448 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/ftdi_sio.ko 2024-03-21T09:58:47.448 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/garmin_gps.ko 2024-03-21T09:58:47.448 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/io_edgeport.ko 2024-03-21T09:58:47.448 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/io_ti.ko 2024-03-21T09:58:47.448 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/ipaq.ko 2024-03-21T09:58:47.448 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/ipw.ko 2024-03-21T09:58:47.448 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/ir-usb.ko 2024-03-21T09:58:47.448 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/iuu_phoenix.ko 2024-03-21T09:58:47.448 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/keyspan.ko 2024-03-21T09:58:47.448 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/keyspan_pda.ko 2024-03-21T09:58:47.448 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/kl5kusb105.ko 2024-03-21T09:58:47.448 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/kobil_sct.ko 2024-03-21T09:58:47.448 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/mct_u232.ko 2024-03-21T09:58:47.448 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/mos7720.ko 2024-03-21T09:58:47.449 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/mos7840.ko 2024-03-21T09:58:47.449 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/navman.ko 2024-03-21T09:58:47.449 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/omninet.ko 2024-03-21T09:58:47.449 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/opticon.ko 2024-03-21T09:58:47.449 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/option.ko 2024-03-21T09:58:47.449 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/oti6858.ko 2024-03-21T09:58:47.449 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/pl2303.ko 2024-03-21T09:58:47.449 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/qcaux.ko 2024-03-21T09:58:47.449 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/qcserial.ko 2024-03-21T09:58:47.449 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/quatech2.ko 2024-03-21T09:58:47.449 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/safe_serial.ko 2024-03-21T09:58:47.449 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/sierra.ko 2024-03-21T09:58:47.449 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/spcp8x5.ko 2024-03-21T09:58:47.449 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/ssu100.ko 2024-03-21T09:58:47.449 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/symbolserial.ko 2024-03-21T09:58:47.450 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/ti_usb_3410_5052.ko 2024-03-21T09:58:47.450 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/usb-serial-simple.ko 2024-03-21T09:58:47.450 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/usb_debug.ko 2024-03-21T09:58:47.450 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/usb_wwan.ko 2024-03-21T09:58:47.450 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/visor.ko 2024-03-21T09:58:47.450 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/whiteheat.ko 2024-03-21T09:58:47.450 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/xsens_mt.ko 2024-03-21T09:58:47.450 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/storage 2024-03-21T09:58:47.450 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/storage/uas.ko 2024-03-21T09:58:47.450 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/storage/ums-alauda.ko 2024-03-21T09:58:47.450 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/storage/ums-cypress.ko 2024-03-21T09:58:47.450 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/storage/ums-datafab.ko 2024-03-21T09:58:47.450 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/storage/ums-eneub6250.ko 2024-03-21T09:58:47.451 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/storage/ums-freecom.ko 2024-03-21T09:58:47.451 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/storage/ums-isd200.ko 2024-03-21T09:58:47.451 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/storage/ums-jumpshot.ko 2024-03-21T09:58:47.451 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/storage/ums-karma.ko 2024-03-21T09:58:47.451 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/storage/ums-onetouch.ko 2024-03-21T09:58:47.451 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/storage/ums-realtek.ko 2024-03-21T09:58:47.451 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/storage/ums-sddr09.ko 2024-03-21T09:58:47.451 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/storage/ums-sddr55.ko 2024-03-21T09:58:47.451 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/storage/ums-usbat.ko 2024-03-21T09:58:47.451 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/storage/usb-storage.ko 2024-03-21T09:58:47.451 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/usbip 2024-03-21T09:58:47.451 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/usbip/usbip-core.ko 2024-03-21T09:58:47.451 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/usbip/usbip-host.ko 2024-03-21T09:58:47.451 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/usbip/vhci-hcd.ko 2024-03-21T09:58:47.451 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/vfio 2024-03-21T09:58:47.452 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/vfio/pci 2024-03-21T09:58:47.452 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/vfio/pci/vfio-pci-core.ko 2024-03-21T09:58:47.452 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/vfio/pci/vfio-pci.ko 2024-03-21T09:58:47.452 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/vfio/vfio.ko 2024-03-21T09:58:47.452 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/vfio/vfio_iommu_type1.ko 2024-03-21T09:58:47.452 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/vhost 2024-03-21T09:58:47.452 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/vhost/vhost.ko 2024-03-21T09:58:47.452 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/vhost/vhost_iotlb.ko 2024-03-21T09:58:47.452 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/vhost/vhost_net.ko 2024-03-21T09:58:47.452 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/vhost/vhost_scsi.ko 2024-03-21T09:58:47.452 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/video 2024-03-21T09:58:47.452 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/video/backlight 2024-03-21T09:58:47.452 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/video/backlight/apple_bl.ko 2024-03-21T09:58:47.452 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/video/backlight/lcd.ko 2024-03-21T09:58:47.452 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/video/backlight/platform_lcd.ko 2024-03-21T09:58:47.453 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/virtio 2024-03-21T09:58:47.453 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/virtio/virtio.ko 2024-03-21T09:58:47.453 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/virtio/virtio_balloon.ko 2024-03-21T09:58:47.453 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/virtio/virtio_mmio.ko 2024-03-21T09:58:47.453 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/virtio/virtio_pci.ko 2024-03-21T09:58:47.453 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/virtio/virtio_pci_legacy_dev.ko 2024-03-21T09:58:47.453 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/virtio/virtio_pci_modern_dev.ko 2024-03-21T09:58:47.453 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/virtio/virtio_ring.ko 2024-03-21T09:58:47.453 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/w1 2024-03-21T09:58:47.453 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/w1/masters 2024-03-21T09:58:47.453 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/w1/masters/ds2482.ko 2024-03-21T09:58:47.453 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/w1/masters/ds2490.ko 2024-03-21T09:58:47.453 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/w1/slaves 2024-03-21T09:58:47.453 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/w1/slaves/w1_ds2406.ko 2024-03-21T09:58:47.453 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/w1/slaves/w1_ds2408.ko 2024-03-21T09:58:47.454 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/w1/slaves/w1_ds2413.ko 2024-03-21T09:58:47.454 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/w1/slaves/w1_ds2423.ko 2024-03-21T09:58:47.454 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/w1/slaves/w1_ds2431.ko 2024-03-21T09:58:47.454 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/w1/slaves/w1_ds2433.ko 2024-03-21T09:58:47.454 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/w1/slaves/w1_ds2780.ko 2024-03-21T09:58:47.454 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/w1/slaves/w1_ds2781.ko 2024-03-21T09:58:47.454 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/w1/slaves/w1_ds28e04.ko 2024-03-21T09:58:47.454 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/w1/slaves/w1_smem.ko 2024-03-21T09:58:47.454 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/w1/slaves/w1_therm.ko 2024-03-21T09:58:47.454 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/w1/wire.ko 2024-03-21T09:58:47.454 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/watchdog 2024-03-21T09:58:47.454 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/watchdog/alim1535_wdt.ko 2024-03-21T09:58:47.454 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/watchdog/alim7101_wdt.ko 2024-03-21T09:58:47.454 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/watchdog/f71808e_wdt.ko 2024-03-21T09:58:47.455 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/watchdog/hpwdt.ko 2024-03-21T09:58:47.455 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/watchdog/i6300esb.ko 2024-03-21T09:58:47.455 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/watchdog/iTCO_vendor_support.ko 2024-03-21T09:58:47.455 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/watchdog/iTCO_wdt.ko 2024-03-21T09:58:47.455 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/watchdog/ib700wdt.ko 2024-03-21T09:58:47.455 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/watchdog/ibmasr.ko 2024-03-21T09:58:47.455 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/watchdog/ie6xx_wdt.ko 2024-03-21T09:58:47.455 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/watchdog/it8712f_wdt.ko 2024-03-21T09:58:47.455 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/watchdog/it87_wdt.ko 2024-03-21T09:58:47.455 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/watchdog/machzwd.ko 2024-03-21T09:58:47.455 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/watchdog/nv_tco.ko 2024-03-21T09:58:47.455 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/watchdog/pcwd_pci.ko 2024-03-21T09:58:47.455 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/watchdog/pcwd_usb.ko 2024-03-21T09:58:47.455 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/watchdog/sbc_fitpc2_wdt.ko 2024-03-21T09:58:47.455 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/watchdog/sch311x_wdt.ko 2024-03-21T09:58:47.456 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/watchdog/softdog.ko 2024-03-21T09:58:47.456 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/watchdog/sp5100_tco.ko 2024-03-21T09:58:47.456 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/watchdog/via_wdt.ko 2024-03-21T09:58:47.456 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/watchdog/w83627hf_wdt.ko 2024-03-21T09:58:47.456 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/watchdog/w83877f_wdt.ko 2024-03-21T09:58:47.456 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/watchdog/w83977f_wdt.ko 2024-03-21T09:58:47.456 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/watchdog/wdt_pci.ko 2024-03-21T09:58:47.456 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/watchdog/xen_wdt.ko 2024-03-21T09:58:47.456 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/xen 2024-03-21T09:58:47.456 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/xen/xen-acpi-processor.ko 2024-03-21T09:58:47.456 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/xen/xen-evtchn.ko 2024-03-21T09:58:47.456 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/xen/xen-gntalloc.ko 2024-03-21T09:58:47.456 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/xen/xen-gntdev.ko 2024-03-21T09:58:47.456 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/xen/xen-pciback 2024-03-21T09:58:47.456 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/xen/xen-pciback/xen-pciback.ko 2024-03-21T09:58:47.457 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/xen/xen-privcmd.ko 2024-03-21T09:58:47.457 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/xen/xenfs 2024-03-21T09:58:47.457 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/xen/xenfs/xenfs.ko 2024-03-21T09:58:47.457 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs 2024-03-21T09:58:47.457 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/9p 2024-03-21T09:58:47.457 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/9p/9p.ko 2024-03-21T09:58:47.457 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/affs 2024-03-21T09:58:47.457 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/affs/affs.ko 2024-03-21T09:58:47.457 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/befs 2024-03-21T09:58:47.457 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/befs/befs.ko 2024-03-21T09:58:47.457 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/binfmt_misc.ko 2024-03-21T09:58:47.457 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/btrfs 2024-03-21T09:58:47.457 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/btrfs/btrfs.ko 2024-03-21T09:58:47.457 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/cachefiles 2024-03-21T09:58:47.457 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/cachefiles/cachefiles.ko 2024-03-21T09:58:47.458 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/ceph 2024-03-21T09:58:47.458 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/ceph/ceph.ko 2024-03-21T09:58:47.458 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/coda 2024-03-21T09:58:47.458 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/coda/coda.ko 2024-03-21T09:58:47.458 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/cramfs 2024-03-21T09:58:47.458 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/cramfs/cramfs.ko 2024-03-21T09:58:47.458 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/dlm 2024-03-21T09:58:47.458 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/dlm/dlm.ko 2024-03-21T09:58:47.458 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/ecryptfs 2024-03-21T09:58:47.458 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/ecryptfs/ecryptfs.ko 2024-03-21T09:58:47.458 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/f2fs 2024-03-21T09:58:47.458 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/f2fs/f2fs.ko 2024-03-21T09:58:47.458 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/fat 2024-03-21T09:58:47.459 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/fat/fat.ko 2024-03-21T09:58:47.459 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/fat/msdos.ko 2024-03-21T09:58:47.459 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/fat/vfat.ko 2024-03-21T09:58:47.459 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/fuse 2024-03-21T09:58:47.459 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/fuse/cuse.ko 2024-03-21T09:58:47.459 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/fuse/fuse.ko 2024-03-21T09:58:47.459 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/gfs2 2024-03-21T09:58:47.459 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/gfs2/gfs2.ko 2024-03-21T09:58:47.459 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/hfs 2024-03-21T09:58:47.459 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/hfs/hfs.ko 2024-03-21T09:58:47.459 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/hfsplus 2024-03-21T09:58:47.459 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/hfsplus/hfsplus.ko 2024-03-21T09:58:47.459 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/isofs 2024-03-21T09:58:47.459 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/isofs/isofs.ko 2024-03-21T09:58:47.459 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/jfs 2024-03-21T09:58:47.460 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/jfs/jfs.ko 2024-03-21T09:58:47.460 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/lockd 2024-03-21T09:58:47.460 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/lockd/lockd.ko 2024-03-21T09:58:47.460 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/minix 2024-03-21T09:58:47.460 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/minix/minix.ko 2024-03-21T09:58:47.460 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/netfs 2024-03-21T09:58:47.460 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/netfs/netfs.ko 2024-03-21T09:58:47.460 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nfs 2024-03-21T09:58:47.460 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nfs/blocklayout 2024-03-21T09:58:47.460 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nfs/blocklayout/blocklayoutdriver.ko 2024-03-21T09:58:47.460 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nfs/filelayout 2024-03-21T09:58:47.460 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nfs/filelayout/nfs_layout_nfsv41_files.ko 2024-03-21T09:58:47.460 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nfs/flexfilelayout 2024-03-21T09:58:47.460 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nfs/flexfilelayout/nfs_layout_flexfiles.ko 2024-03-21T09:58:47.460 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nfs/nfs.ko 2024-03-21T09:58:47.460 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nfs/nfsv3.ko 2024-03-21T09:58:47.461 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nfs/nfsv4.ko 2024-03-21T09:58:47.461 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nfs_common 2024-03-21T09:58:47.461 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nfs_common/grace.ko 2024-03-21T09:58:47.461 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nfs_common/nfs_acl.ko 2024-03-21T09:58:47.461 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nfsd 2024-03-21T09:58:47.461 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nfsd/nfsd.ko 2024-03-21T09:58:47.461 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nilfs2 2024-03-21T09:58:47.461 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nilfs2/nilfs2.ko 2024-03-21T09:58:47.461 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls 2024-03-21T09:58:47.461 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/mac-celtic.ko 2024-03-21T09:58:47.461 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/mac-centeuro.ko 2024-03-21T09:58:47.461 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/mac-croatian.ko 2024-03-21T09:58:47.461 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/mac-cyrillic.ko 2024-03-21T09:58:47.461 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/mac-gaelic.ko 2024-03-21T09:58:47.462 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/mac-greek.ko 2024-03-21T09:58:47.462 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/mac-iceland.ko 2024-03-21T09:58:47.462 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/mac-inuit.ko 2024-03-21T09:58:47.462 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/mac-roman.ko 2024-03-21T09:58:47.462 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/mac-romanian.ko 2024-03-21T09:58:47.462 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/mac-turkish.ko 2024-03-21T09:58:47.462 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_cp1250.ko 2024-03-21T09:58:47.462 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_cp1251.ko 2024-03-21T09:58:47.462 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_cp1255.ko 2024-03-21T09:58:47.462 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_cp737.ko 2024-03-21T09:58:47.462 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_cp775.ko 2024-03-21T09:58:47.462 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_cp850.ko 2024-03-21T09:58:47.462 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_cp852.ko 2024-03-21T09:58:47.462 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_cp855.ko 2024-03-21T09:58:47.463 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_cp857.ko 2024-03-21T09:58:47.463 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_cp860.ko 2024-03-21T09:58:47.463 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_cp861.ko 2024-03-21T09:58:47.463 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_cp862.ko 2024-03-21T09:58:47.463 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_cp863.ko 2024-03-21T09:58:47.463 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_cp864.ko 2024-03-21T09:58:47.463 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_cp865.ko 2024-03-21T09:58:47.463 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_cp866.ko 2024-03-21T09:58:47.463 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_cp869.ko 2024-03-21T09:58:47.463 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_cp874.ko 2024-03-21T09:58:47.463 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_cp932.ko 2024-03-21T09:58:47.463 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_cp936.ko 2024-03-21T09:58:47.463 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_cp949.ko 2024-03-21T09:58:47.463 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_cp950.ko 2024-03-21T09:58:47.464 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_euc-jp.ko 2024-03-21T09:58:47.464 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_iso8859-1.ko 2024-03-21T09:58:47.464 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_iso8859-13.ko 2024-03-21T09:58:47.464 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_iso8859-14.ko 2024-03-21T09:58:47.464 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_iso8859-15.ko 2024-03-21T09:58:47.464 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_iso8859-2.ko 2024-03-21T09:58:47.464 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_iso8859-3.ko 2024-03-21T09:58:47.464 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_iso8859-4.ko 2024-03-21T09:58:47.464 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_iso8859-5.ko 2024-03-21T09:58:47.464 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_iso8859-6.ko 2024-03-21T09:58:47.464 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_iso8859-7.ko 2024-03-21T09:58:47.464 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_iso8859-9.ko 2024-03-21T09:58:47.464 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_koi8-r.ko 2024-03-21T09:58:47.464 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_koi8-ru.ko 2024-03-21T09:58:47.464 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_koi8-u.ko 2024-03-21T09:58:47.465 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_ucs2_utils.ko 2024-03-21T09:58:47.465 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_utf8.ko 2024-03-21T09:58:47.465 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/ocfs2 2024-03-21T09:58:47.465 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/ocfs2/cluster 2024-03-21T09:58:47.465 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/ocfs2/cluster/ocfs2_nodemanager.ko 2024-03-21T09:58:47.465 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/ocfs2/dlm 2024-03-21T09:58:47.465 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/ocfs2/dlm/ocfs2_dlm.ko 2024-03-21T09:58:47.465 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/ocfs2/dlmfs 2024-03-21T09:58:47.465 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/ocfs2/dlmfs/ocfs2_dlmfs.ko 2024-03-21T09:58:47.465 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/ocfs2/ocfs2.ko 2024-03-21T09:58:47.465 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/ocfs2/ocfs2_stack_o2cb.ko 2024-03-21T09:58:47.465 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/ocfs2/ocfs2_stack_user.ko 2024-03-21T09:58:47.465 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/ocfs2/ocfs2_stackglue.ko 2024-03-21T09:58:47.466 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/overlayfs 2024-03-21T09:58:47.466 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/overlayfs/overlay.ko 2024-03-21T09:58:47.466 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/pstore 2024-03-21T09:58:47.466 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/pstore/ramoops.ko 2024-03-21T09:58:47.466 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/reiserfs 2024-03-21T09:58:47.466 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/reiserfs/reiserfs.ko 2024-03-21T09:58:47.466 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/romfs 2024-03-21T09:58:47.466 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/romfs/romfs.ko 2024-03-21T09:58:47.466 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/smb 2024-03-21T09:58:47.466 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/smb/client 2024-03-21T09:58:47.466 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/smb/client/cifs.ko 2024-03-21T09:58:47.466 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/smb/common 2024-03-21T09:58:47.466 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/smb/common/cifs_arc4.ko 2024-03-21T09:58:47.467 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/smb/common/cifs_md4.ko 2024-03-21T09:58:47.467 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/squashfs 2024-03-21T09:58:47.467 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/squashfs/squashfs.ko 2024-03-21T09:58:47.467 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/sysv 2024-03-21T09:58:47.467 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/sysv/sysv.ko 2024-03-21T09:58:47.467 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/udf 2024-03-21T09:58:47.467 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/udf/udf.ko 2024-03-21T09:58:47.467 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/ufs 2024-03-21T09:58:47.467 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/ufs/ufs.ko 2024-03-21T09:58:47.467 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/xfs 2024-03-21T09:58:47.467 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/xfs/xfs.ko 2024-03-21T09:58:47.467 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/kernel 2024-03-21T09:58:47.467 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/kernel/rcu 2024-03-21T09:58:47.467 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/kernel/rcu/rcutorture.ko 2024-03-21T09:58:47.468 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/kernel/torture.ko 2024-03-21T09:58:47.468 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/kernel/trace 2024-03-21T09:58:47.468 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/kernel/trace/ring_buffer_benchmark.ko 2024-03-21T09:58:47.468 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/lib 2024-03-21T09:58:47.468 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/lib/asn1_encoder.ko 2024-03-21T09:58:47.468 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/lib/crc-itu-t.ko 2024-03-21T09:58:47.468 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/lib/crc8.ko 2024-03-21T09:58:47.468 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/lib/crypto 2024-03-21T09:58:47.468 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/lib/crypto/libarc4.ko 2024-03-21T09:58:47.468 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/lib/crypto/libdes.ko 2024-03-21T09:58:47.468 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/lib/libcrc32c.ko 2024-03-21T09:58:47.468 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/lib/lru_cache.ko 2024-03-21T09:58:47.468 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/lib/lz4 2024-03-21T09:58:47.469 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/lib/lz4/lz4_compress.ko 2024-03-21T09:58:47.469 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/lib/lz4/lz4hc_compress.ko 2024-03-21T09:58:47.469 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/lib/math 2024-03-21T09:58:47.469 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/lib/math/cordic.ko 2024-03-21T09:58:47.469 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/lib/raid6 2024-03-21T09:58:47.469 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/lib/raid6/raid6_pq.ko 2024-03-21T09:58:47.469 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/lib/reed_solomon 2024-03-21T09:58:47.469 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/lib/reed_solomon/reed_solomon.ko 2024-03-21T09:58:47.469 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/lib/ts_bm.ko 2024-03-21T09:58:47.469 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/lib/ts_fsm.ko 2024-03-21T09:58:47.469 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/lib/ts_kmp.ko 2024-03-21T09:58:47.469 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/lib/zstd 2024-03-21T09:58:47.469 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/lib/zstd/zstd_compress.ko 2024-03-21T09:58:47.469 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/mm 2024-03-21T09:58:47.470 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/mm/hwpoison-inject.ko 2024-03-21T09:58:47.470 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net 2024-03-21T09:58:47.470 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/6lowpan 2024-03-21T09:58:47.470 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/6lowpan/6lowpan.ko 2024-03-21T09:58:47.470 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/6lowpan/nhc_dest.ko 2024-03-21T09:58:47.470 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/6lowpan/nhc_fragment.ko 2024-03-21T09:58:47.470 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/6lowpan/nhc_hop.ko 2024-03-21T09:58:47.470 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/6lowpan/nhc_ipv6.ko 2024-03-21T09:58:47.470 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/6lowpan/nhc_mobility.ko 2024-03-21T09:58:47.470 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/6lowpan/nhc_routing.ko 2024-03-21T09:58:47.470 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/6lowpan/nhc_udp.ko 2024-03-21T09:58:47.470 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/802 2024-03-21T09:58:47.470 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/802/garp.ko 2024-03-21T09:58:47.470 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/802/mrp.ko 2024-03-21T09:58:47.471 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/802/p8022.ko 2024-03-21T09:58:47.471 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/802/psnap.ko 2024-03-21T09:58:47.471 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/802/stp.ko 2024-03-21T09:58:47.471 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/8021q 2024-03-21T09:58:47.471 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/8021q/8021q.ko 2024-03-21T09:58:47.471 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/9p 2024-03-21T09:58:47.471 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/9p/9pnet.ko 2024-03-21T09:58:47.471 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/9p/9pnet_fd.ko 2024-03-21T09:58:47.471 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/9p/9pnet_rdma.ko 2024-03-21T09:58:47.471 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/9p/9pnet_virtio.ko 2024-03-21T09:58:47.471 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/batman-adv 2024-03-21T09:58:47.471 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/batman-adv/batman-adv.ko 2024-03-21T09:58:47.471 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/bridge 2024-03-21T09:58:47.471 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/bridge/br_netfilter.ko 2024-03-21T09:58:47.471 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/bridge/bridge.ko 2024-03-21T09:58:47.472 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/bridge/netfilter 2024-03-21T09:58:47.472 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/bridge/netfilter/ebt_802_3.ko 2024-03-21T09:58:47.472 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/bridge/netfilter/ebt_among.ko 2024-03-21T09:58:47.472 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/bridge/netfilter/ebt_arp.ko 2024-03-21T09:58:47.472 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/bridge/netfilter/ebt_arpreply.ko 2024-03-21T09:58:47.472 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/bridge/netfilter/ebt_dnat.ko 2024-03-21T09:58:47.472 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/bridge/netfilter/ebt_ip.ko 2024-03-21T09:58:47.472 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/bridge/netfilter/ebt_ip6.ko 2024-03-21T09:58:47.472 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/bridge/netfilter/ebt_limit.ko 2024-03-21T09:58:47.472 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/bridge/netfilter/ebt_log.ko 2024-03-21T09:58:47.472 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/bridge/netfilter/ebt_mark.ko 2024-03-21T09:58:47.472 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/bridge/netfilter/ebt_mark_m.ko 2024-03-21T09:58:47.472 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/bridge/netfilter/ebt_nflog.ko 2024-03-21T09:58:47.472 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/bridge/netfilter/ebt_pkttype.ko 2024-03-21T09:58:47.472 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/bridge/netfilter/ebt_redirect.ko 2024-03-21T09:58:47.473 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/bridge/netfilter/ebt_snat.ko 2024-03-21T09:58:47.473 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/bridge/netfilter/ebt_stp.ko 2024-03-21T09:58:47.473 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/bridge/netfilter/ebt_vlan.ko 2024-03-21T09:58:47.473 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/bridge/netfilter/ebtable_broute.ko 2024-03-21T09:58:47.473 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/bridge/netfilter/ebtable_filter.ko 2024-03-21T09:58:47.473 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/bridge/netfilter/ebtable_nat.ko 2024-03-21T09:58:47.473 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/bridge/netfilter/ebtables.ko 2024-03-21T09:58:47.473 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/bridge/netfilter/nf_conntrack_bridge.ko 2024-03-21T09:58:47.473 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/bridge/netfilter/nft_meta_bridge.ko 2024-03-21T09:58:47.473 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/bridge/netfilter/nft_reject_bridge.ko 2024-03-21T09:58:47.473 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ceph 2024-03-21T09:58:47.473 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ceph/libceph.ko 2024-03-21T09:58:47.473 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/core 2024-03-21T09:58:47.473 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/core/failover.ko 2024-03-21T09:58:47.473 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/core/pktgen.ko 2024-03-21T09:58:47.474 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/dccp 2024-03-21T09:58:47.474 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/dccp/dccp.ko 2024-03-21T09:58:47.474 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/dccp/dccp_diag.ko 2024-03-21T09:58:47.474 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/dccp/dccp_ipv4.ko 2024-03-21T09:58:47.474 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/dccp/dccp_ipv6.ko 2024-03-21T09:58:47.474 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/dns_resolver 2024-03-21T09:58:47.474 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/dns_resolver/dns_resolver.ko 2024-03-21T09:58:47.474 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/dsa 2024-03-21T09:58:47.474 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/dsa/dsa_core.ko 2024-03-21T09:58:47.474 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/dsa/tag_brcm.ko 2024-03-21T09:58:47.474 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/dsa/tag_dsa.ko 2024-03-21T09:58:47.474 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/dsa/tag_none.ko 2024-03-21T09:58:47.474 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/dsa/tag_trailer.ko 2024-03-21T09:58:47.474 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ieee802154 2024-03-21T09:58:47.475 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ieee802154/6lowpan 2024-03-21T09:58:47.475 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ieee802154/6lowpan/ieee802154_6lowpan.ko 2024-03-21T09:58:47.475 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ieee802154/ieee802154.ko 2024-03-21T09:58:47.475 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ieee802154/ieee802154_socket.ko 2024-03-21T09:58:47.475 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4 2024-03-21T09:58:47.475 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/ah4.ko 2024-03-21T09:58:47.475 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/esp4.ko 2024-03-21T09:58:47.475 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/fou.ko 2024-03-21T09:58:47.475 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/gre.ko 2024-03-21T09:58:47.475 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/inet_diag.ko 2024-03-21T09:58:47.475 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/ip_gre.ko 2024-03-21T09:58:47.475 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/ip_tunnel.ko 2024-03-21T09:58:47.475 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/ip_vti.ko 2024-03-21T09:58:47.475 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/ipcomp.ko 2024-03-21T09:58:47.476 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/ipip.ko 2024-03-21T09:58:47.476 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/netfilter 2024-03-21T09:58:47.476 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/netfilter/arp_tables.ko 2024-03-21T09:58:47.476 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/netfilter/arpt_mangle.ko 2024-03-21T09:58:47.476 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/netfilter/arptable_filter.ko 2024-03-21T09:58:47.476 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/netfilter/ipt_ECN.ko 2024-03-21T09:58:47.476 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/netfilter/ipt_SYNPROXY.ko 2024-03-21T09:58:47.476 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/netfilter/ipt_ah.ko 2024-03-21T09:58:47.476 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/netfilter/ipt_rpfilter.ko 2024-03-21T09:58:47.476 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/netfilter/iptable_mangle.ko 2024-03-21T09:58:47.476 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/netfilter/iptable_nat.ko 2024-03-21T09:58:47.476 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/netfilter/iptable_raw.ko 2024-03-21T09:58:47.476 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/netfilter/iptable_security.ko 2024-03-21T09:58:47.476 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/netfilter/nf_defrag_ipv4.ko 2024-03-21T09:58:47.476 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/netfilter/nf_dup_ipv4.ko 2024-03-21T09:58:47.477 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/netfilter/nf_nat_h323.ko 2024-03-21T09:58:47.477 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/netfilter/nf_nat_pptp.ko 2024-03-21T09:58:47.477 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/netfilter/nf_nat_snmp_basic.ko 2024-03-21T09:58:47.477 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/netfilter/nf_socket_ipv4.ko 2024-03-21T09:58:47.477 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/netfilter/nf_tproxy_ipv4.ko 2024-03-21T09:58:47.477 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/netfilter/nft_dup_ipv4.ko 2024-03-21T09:58:47.477 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/netfilter/nft_fib_ipv4.ko 2024-03-21T09:58:47.477 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/netfilter/nft_reject_ipv4.ko 2024-03-21T09:58:47.477 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/tcp_bic.ko 2024-03-21T09:58:47.477 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/tcp_dctcp.ko 2024-03-21T09:58:47.477 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/tcp_diag.ko 2024-03-21T09:58:47.477 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/tcp_highspeed.ko 2024-03-21T09:58:47.477 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/tcp_htcp.ko 2024-03-21T09:58:47.477 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/tcp_hybla.ko 2024-03-21T09:58:47.478 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/tcp_illinois.ko 2024-03-21T09:58:47.478 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/tcp_lp.ko 2024-03-21T09:58:47.478 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/tcp_scalable.ko 2024-03-21T09:58:47.478 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/tcp_vegas.ko 2024-03-21T09:58:47.478 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/tcp_veno.ko 2024-03-21T09:58:47.478 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/tcp_westwood.ko 2024-03-21T09:58:47.478 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/tcp_yeah.ko 2024-03-21T09:58:47.478 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/tunnel4.ko 2024-03-21T09:58:47.478 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/udp_diag.ko 2024-03-21T09:58:47.478 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/udp_tunnel.ko 2024-03-21T09:58:47.478 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/xfrm4_tunnel.ko 2024-03-21T09:58:47.478 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv6 2024-03-21T09:58:47.478 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv6/ah6.ko 2024-03-21T09:58:47.478 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv6/esp6.ko 2024-03-21T09:58:47.479 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv6/fou6.ko 2024-03-21T09:58:47.479 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv6/ip6_tunnel.ko 2024-03-21T09:58:47.479 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv6/ip6_udp_tunnel.ko 2024-03-21T09:58:47.479 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv6/ip6_vti.ko 2024-03-21T09:58:47.479 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv6/ipcomp6.ko 2024-03-21T09:58:47.479 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv6/netfilter 2024-03-21T09:58:47.479 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv6/netfilter/ip6_tables.ko 2024-03-21T09:58:47.479 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv6/netfilter/ip6t_REJECT.ko 2024-03-21T09:58:47.479 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv6/netfilter/ip6t_SYNPROXY.ko 2024-03-21T09:58:47.479 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv6/netfilter/ip6t_ah.ko 2024-03-21T09:58:47.479 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv6/netfilter/ip6t_eui64.ko 2024-03-21T09:58:47.479 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv6/netfilter/ip6t_frag.ko 2024-03-21T09:58:47.479 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv6/netfilter/ip6t_hbh.ko 2024-03-21T09:58:47.480 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv6/netfilter/ip6t_ipv6header.ko 2024-03-21T09:58:47.480 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv6/netfilter/ip6t_mh.ko 2024-03-21T09:58:47.480 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv6/netfilter/ip6t_rpfilter.ko 2024-03-21T09:58:47.480 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv6/netfilter/ip6t_rt.ko 2024-03-21T09:58:47.480 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv6/netfilter/ip6table_filter.ko 2024-03-21T09:58:47.480 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv6/netfilter/ip6table_mangle.ko 2024-03-21T09:58:47.480 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv6/netfilter/ip6table_nat.ko 2024-03-21T09:58:47.480 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv6/netfilter/ip6table_raw.ko 2024-03-21T09:58:47.480 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv6/netfilter/ip6table_security.ko 2024-03-21T09:58:47.480 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv6/netfilter/nf_defrag_ipv6.ko 2024-03-21T09:58:47.480 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv6/netfilter/nf_dup_ipv6.ko 2024-03-21T09:58:47.480 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv6/netfilter/nf_reject_ipv6.ko 2024-03-21T09:58:47.480 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv6/netfilter/nf_socket_ipv6.ko 2024-03-21T09:58:47.480 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv6/netfilter/nf_tproxy_ipv6.ko 2024-03-21T09:58:47.480 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv6/netfilter/nft_dup_ipv6.ko 2024-03-21T09:58:47.480 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv6/netfilter/nft_fib_ipv6.ko 2024-03-21T09:58:47.481 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv6/netfilter/nft_reject_ipv6.ko 2024-03-21T09:58:47.481 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv6/sit.ko 2024-03-21T09:58:47.481 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv6/tunnel6.ko 2024-03-21T09:58:47.481 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv6/xfrm6_tunnel.ko 2024-03-21T09:58:47.481 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/key 2024-03-21T09:58:47.481 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/key/af_key.ko 2024-03-21T09:58:47.481 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/l2tp 2024-03-21T09:58:47.481 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/l2tp/l2tp_core.ko 2024-03-21T09:58:47.481 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/l2tp/l2tp_debugfs.ko 2024-03-21T09:58:47.481 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/l2tp/l2tp_eth.ko 2024-03-21T09:58:47.481 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/l2tp/l2tp_ip.ko 2024-03-21T09:58:47.481 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/l2tp/l2tp_ip6.ko 2024-03-21T09:58:47.481 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/l2tp/l2tp_netlink.ko 2024-03-21T09:58:47.481 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/l2tp/l2tp_ppp.ko 2024-03-21T09:58:47.481 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/llc 2024-03-21T09:58:47.482 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/llc/llc.ko 2024-03-21T09:58:47.482 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/mac802154 2024-03-21T09:58:47.482 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/mac802154/mac802154.ko 2024-03-21T09:58:47.482 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/mpls 2024-03-21T09:58:47.482 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/mpls/mpls_gso.ko 2024-03-21T09:58:47.482 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter 2024-03-21T09:58:47.482 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/ipset 2024-03-21T09:58:47.482 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/ipset/ip_set.ko 2024-03-21T09:58:47.482 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/ipset/ip_set_bitmap_ip.ko 2024-03-21T09:58:47.482 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/ipset/ip_set_bitmap_ipmac.ko 2024-03-21T09:58:47.482 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/ipset/ip_set_bitmap_port.ko 2024-03-21T09:58:47.482 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/ipset/ip_set_hash_ip.ko 2024-03-21T09:58:47.482 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/ipset/ip_set_hash_ipmark.ko 2024-03-21T09:58:47.482 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/ipset/ip_set_hash_ipport.ko 2024-03-21T09:58:47.483 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/ipset/ip_set_hash_ipportip.ko 2024-03-21T09:58:47.483 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/ipset/ip_set_hash_ipportnet.ko 2024-03-21T09:58:47.483 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/ipset/ip_set_hash_mac.ko 2024-03-21T09:58:47.483 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/ipset/ip_set_hash_net.ko 2024-03-21T09:58:47.483 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/ipset/ip_set_hash_netiface.ko 2024-03-21T09:58:47.483 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/ipset/ip_set_hash_netnet.ko 2024-03-21T09:58:47.483 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/ipset/ip_set_hash_netport.ko 2024-03-21T09:58:47.483 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/ipset/ip_set_hash_netportnet.ko 2024-03-21T09:58:47.483 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/ipset/ip_set_list_set.ko 2024-03-21T09:58:47.483 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/ipvs 2024-03-21T09:58:47.483 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/ipvs/ip_vs.ko 2024-03-21T09:58:47.483 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/ipvs/ip_vs_dh.ko 2024-03-21T09:58:47.483 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/ipvs/ip_vs_fo.ko 2024-03-21T09:58:47.483 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/ipvs/ip_vs_ftp.ko 2024-03-21T09:58:47.483 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/ipvs/ip_vs_lblc.ko 2024-03-21T09:58:47.483 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/ipvs/ip_vs_lblcr.ko 2024-03-21T09:58:47.484 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/ipvs/ip_vs_lc.ko 2024-03-21T09:58:47.484 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/ipvs/ip_vs_nq.ko 2024-03-21T09:58:47.484 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/ipvs/ip_vs_pe_sip.ko 2024-03-21T09:58:47.484 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/ipvs/ip_vs_rr.ko 2024-03-21T09:58:47.484 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/ipvs/ip_vs_sed.ko 2024-03-21T09:58:47.484 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/ipvs/ip_vs_sh.ko 2024-03-21T09:58:47.484 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/ipvs/ip_vs_wlc.ko 2024-03-21T09:58:47.484 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/ipvs/ip_vs_wrr.ko 2024-03-21T09:58:47.484 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nf_conncount.ko 2024-03-21T09:58:47.484 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nf_conntrack.ko 2024-03-21T09:58:47.484 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nf_conntrack_amanda.ko 2024-03-21T09:58:47.484 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nf_conntrack_broadcast.ko 2024-03-21T09:58:47.484 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nf_conntrack_ftp.ko 2024-03-21T09:58:47.484 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nf_conntrack_h323.ko 2024-03-21T09:58:47.484 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nf_conntrack_irc.ko 2024-03-21T09:58:47.485 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nf_conntrack_netbios_ns.ko 2024-03-21T09:58:47.485 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nf_conntrack_netlink.ko 2024-03-21T09:58:47.485 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nf_conntrack_pptp.ko 2024-03-21T09:58:47.485 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nf_conntrack_sane.ko 2024-03-21T09:58:47.485 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nf_conntrack_sip.ko 2024-03-21T09:58:47.485 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nf_conntrack_snmp.ko 2024-03-21T09:58:47.485 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nf_conntrack_tftp.ko 2024-03-21T09:58:47.485 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nf_dup_netdev.ko 2024-03-21T09:58:47.485 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nf_flow_table.ko 2024-03-21T09:58:47.485 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nf_flow_table_inet.ko 2024-03-21T09:58:47.485 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nf_log_syslog.ko 2024-03-21T09:58:47.485 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nf_nat.ko 2024-03-21T09:58:47.485 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nf_nat_amanda.ko 2024-03-21T09:58:47.485 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nf_nat_ftp.ko 2024-03-21T09:58:47.485 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nf_nat_irc.ko 2024-03-21T09:58:47.486 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nf_nat_sip.ko 2024-03-21T09:58:47.486 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nf_nat_tftp.ko 2024-03-21T09:58:47.486 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nf_synproxy_core.ko 2024-03-21T09:58:47.486 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nf_tables.ko 2024-03-21T09:58:47.486 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nfnetlink.ko 2024-03-21T09:58:47.486 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nfnetlink_acct.ko 2024-03-21T09:58:47.486 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nfnetlink_log.ko 2024-03-21T09:58:47.486 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nfnetlink_osf.ko 2024-03-21T09:58:47.486 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nfnetlink_queue.ko 2024-03-21T09:58:47.486 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nft_chain_nat.ko 2024-03-21T09:58:47.486 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nft_compat.ko 2024-03-21T09:58:47.486 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nft_ct.ko 2024-03-21T09:58:47.486 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nft_dup_netdev.ko 2024-03-21T09:58:47.486 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nft_fib.ko 2024-03-21T09:58:47.487 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nft_fwd_netdev.ko 2024-03-21T09:58:47.487 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nft_hash.ko 2024-03-21T09:58:47.487 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nft_limit.ko 2024-03-21T09:58:47.487 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nft_log.ko 2024-03-21T09:58:47.487 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nft_masq.ko 2024-03-21T09:58:47.487 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nft_nat.ko 2024-03-21T09:58:47.487 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nft_numgen.ko 2024-03-21T09:58:47.487 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nft_queue.ko 2024-03-21T09:58:47.487 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nft_quota.ko 2024-03-21T09:58:47.487 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nft_redir.ko 2024-03-21T09:58:47.487 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nft_reject.ko 2024-03-21T09:58:47.487 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nft_reject_inet.ko 2024-03-21T09:58:47.487 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nft_socket.ko 2024-03-21T09:58:47.487 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nft_tunnel.ko 2024-03-21T09:58:47.487 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_AUDIT.ko 2024-03-21T09:58:47.488 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_CHECKSUM.ko 2024-03-21T09:58:47.488 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_CLASSIFY.ko 2024-03-21T09:58:47.488 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_CONNSECMARK.ko 2024-03-21T09:58:47.488 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_CT.ko 2024-03-21T09:58:47.488 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_DSCP.ko 2024-03-21T09:58:47.488 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_HL.ko 2024-03-21T09:58:47.488 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_HMARK.ko 2024-03-21T09:58:47.488 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_IDLETIMER.ko 2024-03-21T09:58:47.488 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_LED.ko 2024-03-21T09:58:47.488 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_LOG.ko 2024-03-21T09:58:47.488 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_MASQUERADE.ko 2024-03-21T09:58:47.488 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_NETMAP.ko 2024-03-21T09:58:47.488 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_NFLOG.ko 2024-03-21T09:58:47.488 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_NFQUEUE.ko 2024-03-21T09:58:47.488 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_RATEEST.ko 2024-03-21T09:58:47.489 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_REDIRECT.ko 2024-03-21T09:58:47.489 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_SECMARK.ko 2024-03-21T09:58:47.489 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_TCPMSS.ko 2024-03-21T09:58:47.489 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_TCPOPTSTRIP.ko 2024-03-21T09:58:47.489 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_TEE.ko 2024-03-21T09:58:47.489 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_TPROXY.ko 2024-03-21T09:58:47.489 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_TRACE.ko 2024-03-21T09:58:47.489 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_addrtype.ko 2024-03-21T09:58:47.489 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_bpf.ko 2024-03-21T09:58:47.489 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_cgroup.ko 2024-03-21T09:58:47.489 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_cluster.ko 2024-03-21T09:58:47.489 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_comment.ko 2024-03-21T09:58:47.489 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_connbytes.ko 2024-03-21T09:58:47.489 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_connlabel.ko 2024-03-21T09:58:47.489 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_connlimit.ko 2024-03-21T09:58:47.489 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_connmark.ko 2024-03-21T09:58:47.490 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_conntrack.ko 2024-03-21T09:58:47.490 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_cpu.ko 2024-03-21T09:58:47.490 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_dccp.ko 2024-03-21T09:58:47.490 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_devgroup.ko 2024-03-21T09:58:47.490 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_dscp.ko 2024-03-21T09:58:47.490 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_ecn.ko 2024-03-21T09:58:47.490 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_esp.ko 2024-03-21T09:58:47.490 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_hashlimit.ko 2024-03-21T09:58:47.490 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_helper.ko 2024-03-21T09:58:47.490 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_hl.ko 2024-03-21T09:58:47.490 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_ipcomp.ko 2024-03-21T09:58:47.490 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_iprange.ko 2024-03-21T09:58:47.490 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_ipvs.ko 2024-03-21T09:58:47.490 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_l2tp.ko 2024-03-21T09:58:47.491 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_length.ko 2024-03-21T09:58:47.491 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_limit.ko 2024-03-21T09:58:47.491 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_mac.ko 2024-03-21T09:58:47.491 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_mark.ko 2024-03-21T09:58:47.491 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_multiport.ko 2024-03-21T09:58:47.491 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_nat.ko 2024-03-21T09:58:47.491 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_nfacct.ko 2024-03-21T09:58:47.491 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_osf.ko 2024-03-21T09:58:47.491 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_owner.ko 2024-03-21T09:58:47.491 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_physdev.ko 2024-03-21T09:58:47.491 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_pkttype.ko 2024-03-21T09:58:47.491 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_policy.ko 2024-03-21T09:58:47.491 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_quota.ko 2024-03-21T09:58:47.491 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_rateest.ko 2024-03-21T09:58:47.491 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_realm.ko 2024-03-21T09:58:47.491 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_recent.ko 2024-03-21T09:58:47.492 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_sctp.ko 2024-03-21T09:58:47.492 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_set.ko 2024-03-21T09:58:47.492 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_socket.ko 2024-03-21T09:58:47.492 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_state.ko 2024-03-21T09:58:47.492 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_statistic.ko 2024-03-21T09:58:47.492 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_string.ko 2024-03-21T09:58:47.492 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_tcpmss.ko 2024-03-21T09:58:47.492 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_time.ko 2024-03-21T09:58:47.492 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_u32.ko 2024-03-21T09:58:47.492 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netlink 2024-03-21T09:58:47.492 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netlink/netlink_diag.ko 2024-03-21T09:58:47.492 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/nsh 2024-03-21T09:58:47.492 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/nsh/nsh.ko 2024-03-21T09:58:47.492 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/openvswitch 2024-03-21T09:58:47.492 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/openvswitch/openvswitch.ko 2024-03-21T09:58:47.492 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/openvswitch/vport-geneve.ko 2024-03-21T09:58:47.493 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/openvswitch/vport-gre.ko 2024-03-21T09:58:47.493 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/openvswitch/vport-vxlan.ko 2024-03-21T09:58:47.493 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/packet 2024-03-21T09:58:47.493 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/packet/af_packet_diag.ko 2024-03-21T09:58:47.493 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/rds 2024-03-21T09:58:47.493 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/rds/rds.ko 2024-03-21T09:58:47.493 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/rds/rds_rdma.ko 2024-03-21T09:58:47.493 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/rds/rds_tcp.ko 2024-03-21T09:58:47.493 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched 2024-03-21T09:58:47.493 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/act_csum.ko 2024-03-21T09:58:47.493 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/act_gact.ko 2024-03-21T09:58:47.493 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/act_mirred.ko 2024-03-21T09:58:47.493 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/act_nat.ko 2024-03-21T09:58:47.493 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/act_pedit.ko 2024-03-21T09:58:47.493 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/act_police.ko 2024-03-21T09:58:47.493 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/act_simple.ko 2024-03-21T09:58:47.494 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/act_skbedit.ko 2024-03-21T09:58:47.494 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/act_vlan.ko 2024-03-21T09:58:47.494 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/cls_basic.ko 2024-03-21T09:58:47.494 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/cls_bpf.ko 2024-03-21T09:58:47.494 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/cls_flow.ko 2024-03-21T09:58:47.494 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/cls_fw.ko 2024-03-21T09:58:47.494 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/cls_route.ko 2024-03-21T09:58:47.494 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/cls_u32.ko 2024-03-21T09:58:47.494 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/em_cmp.ko 2024-03-21T09:58:47.494 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/em_ipset.ko 2024-03-21T09:58:47.494 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/em_meta.ko 2024-03-21T09:58:47.494 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/em_nbyte.ko 2024-03-21T09:58:47.494 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/em_text.ko 2024-03-21T09:58:47.494 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/em_u32.ko 2024-03-21T09:58:47.494 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/sch_choke.ko 2024-03-21T09:58:47.494 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/sch_codel.ko 2024-03-21T09:58:47.495 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/sch_drr.ko 2024-03-21T09:58:47.495 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/sch_fq.ko 2024-03-21T09:58:47.495 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/sch_gred.ko 2024-03-21T09:58:47.495 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/sch_hfsc.ko 2024-03-21T09:58:47.495 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/sch_hhf.ko 2024-03-21T09:58:47.495 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/sch_htb.ko 2024-03-21T09:58:47.495 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/sch_ingress.ko 2024-03-21T09:58:47.495 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/sch_mqprio.ko 2024-03-21T09:58:47.495 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/sch_mqprio_lib.ko 2024-03-21T09:58:47.495 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/sch_multiq.ko 2024-03-21T09:58:47.495 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/sch_netem.ko 2024-03-21T09:58:47.495 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/sch_pie.ko 2024-03-21T09:58:47.495 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/sch_plug.ko 2024-03-21T09:58:47.495 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/sch_prio.ko 2024-03-21T09:58:47.496 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/sch_qfq.ko 2024-03-21T09:58:47.496 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/sch_red.ko 2024-03-21T09:58:47.496 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/sch_sfb.ko 2024-03-21T09:58:47.496 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/sch_sfq.ko 2024-03-21T09:58:47.496 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/sch_tbf.ko 2024-03-21T09:58:47.496 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/sch_teql.ko 2024-03-21T09:58:47.496 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sctp 2024-03-21T09:58:47.496 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sctp/sctp.ko 2024-03-21T09:58:47.496 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sctp/sctp_diag.ko 2024-03-21T09:58:47.496 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sunrpc 2024-03-21T09:58:47.496 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sunrpc/auth_gss 2024-03-21T09:58:47.496 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sunrpc/auth_gss/auth_rpcgss.ko 2024-03-21T09:58:47.496 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sunrpc/auth_gss/rpcsec_gss_krb5.ko 2024-03-21T09:58:47.496 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sunrpc/sunrpc.ko 2024-03-21T09:58:47.496 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sunrpc/xprtrdma 2024-03-21T09:58:47.497 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sunrpc/xprtrdma/rpcrdma.ko 2024-03-21T09:58:47.497 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/tipc 2024-03-21T09:58:47.497 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/tipc/diag.ko 2024-03-21T09:58:47.497 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/tipc/tipc.ko 2024-03-21T09:58:47.497 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/unix 2024-03-21T09:58:47.497 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/unix/unix_diag.ko 2024-03-21T09:58:47.497 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/vmw_vsock 2024-03-21T09:58:47.497 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/vmw_vsock/vmw_vsock_virtio_transport_common.ko 2024-03-21T09:58:47.497 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/vmw_vsock/vmw_vsock_vmci_transport.ko 2024-03-21T09:58:47.497 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/vmw_vsock/vsock.ko 2024-03-21T09:58:47.497 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/vmw_vsock/vsock_diag.ko 2024-03-21T09:58:47.497 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/vmw_vsock/vsock_loopback.ko 2024-03-21T09:58:47.497 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/xfrm 2024-03-21T09:58:47.497 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/xfrm/xfrm_ipcomp.ko 2024-03-21T09:58:47.497 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/security 2024-03-21T09:58:47.497 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/security/keys 2024-03-21T09:58:47.498 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/security/keys/encrypted-keys 2024-03-21T09:58:47.498 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/security/keys/encrypted-keys/encrypted-keys.ko 2024-03-21T09:58:47.498 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/security/keys/trusted-keys 2024-03-21T09:58:47.498 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/security/keys/trusted-keys/trusted.ko 2024-03-21T09:58:47.498 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/virt 2024-03-21T09:58:47.498 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/virt/lib 2024-03-21T09:58:47.498 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/virt/lib/irqbypass.ko 2024-03-21T09:58:47.498 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/modules.alias 2024-03-21T09:58:47.498 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/modules.alias.bin 2024-03-21T09:58:47.498 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/modules.builtin 2024-03-21T09:58:47.498 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/modules.builtin.alias.bin 2024-03-21T09:58:47.498 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/modules.builtin.bin 2024-03-21T09:58:47.498 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/modules.builtin.modinfo 2024-03-21T09:58:47.498 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/modules.dep 2024-03-21T09:58:47.498 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/modules.dep.bin 2024-03-21T09:58:47.499 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/modules.devname 2024-03-21T09:58:47.499 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/modules.order 2024-03-21T09:58:47.499 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/modules.softdep 2024-03-21T09:58:47.499 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/modules.symbols 2024-03-21T09:58:47.499 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/modules.symbols.bin 2024-03-21T09:58:47.499 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/vmlinuz 2024-03-21T09:58:47.502 DEBUG:teuthology.task.kernel:get_image_version: 6.8.0-g5ec3e6e88b2e 2024-03-21T09:58:47.503 DEBUG:teuthology.orchestra.run.smithi100:> rpm --scripts -qp /tmp/kernel.x86_64.rpm 2024-03-21T09:58:47.525 INFO:teuthology.orchestra.run.smithi100.stdout:postinstall scriptlet (using /bin/sh): 2024-03-21T09:58:47.526 INFO:teuthology.orchestra.run.smithi100.stdout:if [ -x /usr/bin/kernel-install ]; then 2024-03-21T09:58:47.526 INFO:teuthology.orchestra.run.smithi100.stdout: /usr/bin/kernel-install add 6.8.0-g5ec3e6e88b2e /lib/modules/6.8.0-g5ec3e6e88b2e/vmlinuz 2024-03-21T09:58:47.526 INFO:teuthology.orchestra.run.smithi100.stdout:fi 2024-03-21T09:58:47.526 INFO:teuthology.orchestra.run.smithi100.stdout:for file in vmlinuz System.map config; do 2024-03-21T09:58:47.526 INFO:teuthology.orchestra.run.smithi100.stdout: if ! cmp --silent "/lib/modules/6.8.0-g5ec3e6e88b2e/${file}" "/boot/${file}-6.8.0-g5ec3e6e88b2e"; then 2024-03-21T09:58:47.526 INFO:teuthology.orchestra.run.smithi100.stdout: cp "/lib/modules/6.8.0-g5ec3e6e88b2e/${file}" "/boot/${file}-6.8.0-g5ec3e6e88b2e" 2024-03-21T09:58:47.526 INFO:teuthology.orchestra.run.smithi100.stdout: fi 2024-03-21T09:58:47.526 INFO:teuthology.orchestra.run.smithi100.stdout:done 2024-03-21T09:58:47.526 INFO:teuthology.orchestra.run.smithi100.stdout:preuninstall scriptlet (using /bin/sh): 2024-03-21T09:58:47.526 INFO:teuthology.orchestra.run.smithi100.stdout:if [ -x /sbin/new-kernel-pkg ]; then 2024-03-21T09:58:47.526 INFO:teuthology.orchestra.run.smithi100.stdout:new-kernel-pkg --remove 6.8.0-g5ec3e6e88b2e --rminitrd --initrdfile=/boot/initramfs-6.8.0-g5ec3e6e88b2e.img 2024-03-21T09:58:47.526 INFO:teuthology.orchestra.run.smithi100.stdout:elif [ -x /usr/bin/kernel-install ]; then 2024-03-21T09:58:47.526 INFO:teuthology.orchestra.run.smithi100.stdout:kernel-install remove 6.8.0-g5ec3e6e88b2e 2024-03-21T09:58:47.527 INFO:teuthology.orchestra.run.smithi100.stdout:fi 2024-03-21T09:58:47.527 INFO:teuthology.orchestra.run.smithi100.stdout:postuninstall scriptlet (using /bin/sh): 2024-03-21T09:58:47.527 INFO:teuthology.orchestra.run.smithi100.stdout:if [ -x /sbin/update-bootloader ]; then 2024-03-21T09:58:47.527 INFO:teuthology.orchestra.run.smithi100.stdout:/sbin/update-bootloader --remove 6.8.0-g5ec3e6e88b2e 2024-03-21T09:58:47.527 INFO:teuthology.orchestra.run.smithi100.stdout:fi 2024-03-21T09:58:47.528 DEBUG:teuthology.orchestra.run.smithi100:> sudo rpm -qi grub2-tools 2024-03-21T09:58:47.604 INFO:teuthology.orchestra.run.smithi100.stdout:Name : grub2-tools 2024-03-21T09:58:47.604 INFO:teuthology.orchestra.run.smithi100.stdout:Epoch : 1 2024-03-21T09:58:47.604 INFO:teuthology.orchestra.run.smithi100.stdout:Version : 2.06 2024-03-21T09:58:47.604 INFO:teuthology.orchestra.run.smithi100.stdout:Release : 68.el9 2024-03-21T09:58:47.604 INFO:teuthology.orchestra.run.smithi100.stdout:Architecture: x86_64 2024-03-21T09:58:47.604 INFO:teuthology.orchestra.run.smithi100.stdout:Install Date: Fri 01 Sep 2023 09:14:30 PM UTC 2024-03-21T09:58:47.604 INFO:teuthology.orchestra.run.smithi100.stdout:Group : Unspecified 2024-03-21T09:58:47.604 INFO:teuthology.orchestra.run.smithi100.stdout:Size : 8264254 2024-03-21T09:58:47.604 INFO:teuthology.orchestra.run.smithi100.stdout:License : GPLv3+ 2024-03-21T09:58:47.605 INFO:teuthology.orchestra.run.smithi100.stdout:Signature : RSA/SHA256, Mon 07 Aug 2023 01:57:16 PM UTC, Key ID 05b555b38483c65d 2024-03-21T09:58:47.605 INFO:teuthology.orchestra.run.smithi100.stdout:Source RPM : grub2-2.06-68.el9.src.rpm 2024-03-21T09:58:47.605 INFO:teuthology.orchestra.run.smithi100.stdout:Build Date : Wed 02 Aug 2023 03:15:48 PM UTC 2024-03-21T09:58:47.605 INFO:teuthology.orchestra.run.smithi100.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-03-21T09:58:47.605 INFO:teuthology.orchestra.run.smithi100.stdout:Packager : builder@centos.org 2024-03-21T09:58:47.605 INFO:teuthology.orchestra.run.smithi100.stdout:Vendor : CentOS 2024-03-21T09:58:47.605 INFO:teuthology.orchestra.run.smithi100.stdout:URL : http://www.gnu.org/software/grub/ 2024-03-21T09:58:47.605 INFO:teuthology.orchestra.run.smithi100.stdout:Summary : Support tools for GRUB. 2024-03-21T09:58:47.605 INFO:teuthology.orchestra.run.smithi100.stdout:Description : 2024-03-21T09:58:47.605 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-03-21T09:58:47.605 INFO:teuthology.orchestra.run.smithi100.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-03-21T09:58:47.605 INFO:teuthology.orchestra.run.smithi100.stdout:customizable bootloader with modular architecture. It supports a rich 2024-03-21T09:58:47.605 INFO:teuthology.orchestra.run.smithi100.stdout:variety of kernel formats, file systems, computer architectures and 2024-03-21T09:58:47.605 INFO:teuthology.orchestra.run.smithi100.stdout:hardware devices. 2024-03-21T09:58:47.605 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-03-21T09:58:47.605 INFO:teuthology.orchestra.run.smithi100.stdout:This subpackage provides tools for support of all platforms. 2024-03-21T09:58:47.606 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-03-21T09:58:47.609 INFO:teuthology.task.kernel:Updating grub on smithi100 to boot 6.8.0-g5ec3e6e88b2e 2024-03-21T09:58:47.609 DEBUG:teuthology.orchestra.run.smithi100:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-03-21T09:58:48.198 INFO:teuthology.orchestra.run.smithi100.stderr:Generating grub configuration file ... 2024-03-21T09:58:49.583 INFO:teuthology.orchestra.run.smithi100.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-03-21T09:58:49.615 INFO:teuthology.orchestra.run.smithi100.stderr:done 2024-03-21T09:58:49.617 DEBUG:teuthology.orchestra.run.smithi100:> mktemp 2024-03-21T09:58:49.631 INFO:teuthology.orchestra.run.smithi100.stdout:/tmp/tmp.taKyfheDSb 2024-03-21T09:58:49.632 DEBUG:teuthology.orchestra.run.smithi100:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.taKyfheDSb 2024-03-21T09:58:49.701 DEBUG:teuthology.orchestra.run.smithi100:> sudo chmod 0666 /tmp/tmp.taKyfheDSb 2024-03-21T09:58:49.831 DEBUG:teuthology.orchestra.remote:smithi100:/tmp/tmp.taKyfheDSb is 6KB 2024-03-21T09:58:49.880 DEBUG:teuthology.orchestra.run.smithi100:> rm -fr /tmp/tmp.taKyfheDSb 2024-03-21T09:58:49.894 DEBUG:teuthology.orchestra.run.smithi100:> sudo /bin/ls /boot/loader/entries || true 2024-03-21T09:58:49.961 INFO:teuthology.orchestra.run.smithi100.stdout:15949e3c88704300882482eddfd7baae-0-rescue.conf 2024-03-21T09:58:49.961 INFO:teuthology.orchestra.run.smithi100.stdout:15949e3c88704300882482eddfd7baae-5.14.0-239.el9.x86_64.conf 2024-03-21T09:58:49.961 INFO:teuthology.orchestra.run.smithi100.stdout:15949e3c88704300882482eddfd7baae-5.14.0-361.el9.x86_64.conf 2024-03-21T09:58:49.961 INFO:teuthology.orchestra.run.smithi100.stdout:15949e3c88704300882482eddfd7baae-6.8.0-g5ec3e6e88b2e.conf 2024-03-21T09:58:49.962 DEBUG:teuthology.orchestra.run.smithi100:> sudo grub2-set-default 15949e3c88704300882482eddfd7baae-6.8.0-g5ec3e6e88b2e 2024-03-21T09:58:50.128 DEBUG:teuthology.orchestra.run.smithi100:> sudo shutdown -r now 2024-03-21T09:58:57.690 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi047.front.sepia.ceph.com, path=/tmp/kernel.x86_64.rpm, version=None) 2024-03-21T09:58:57.690 DEBUG:teuthology.orchestra.run.smithi047:> rpm -qlp /tmp/kernel.x86_64.rpm 2024-03-21T09:58:57.778 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e 2024-03-21T09:58:57.778 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/System.map 2024-03-21T09:58:57.778 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/config 2024-03-21T09:58:57.778 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel 2024-03-21T09:58:57.778 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/arch 2024-03-21T09:58:57.778 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/arch/x86 2024-03-21T09:58:57.778 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/arch/x86/crypto 2024-03-21T09:58:57.778 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/arch/x86/crypto/blowfish-x86_64.ko 2024-03-21T09:58:57.778 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/arch/x86/crypto/camellia-aesni-avx-x86_64.ko 2024-03-21T09:58:57.778 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/arch/x86/crypto/camellia-aesni-avx2.ko 2024-03-21T09:58:57.779 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/arch/x86/crypto/camellia-x86_64.ko 2024-03-21T09:58:57.779 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/arch/x86/crypto/cast5-avx-x86_64.ko 2024-03-21T09:58:57.779 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/arch/x86/crypto/cast6-avx-x86_64.ko 2024-03-21T09:58:57.779 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/arch/x86/crypto/crc32-pclmul.ko 2024-03-21T09:58:57.779 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/arch/x86/crypto/crc32c-intel.ko 2024-03-21T09:58:57.779 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/arch/x86/crypto/crct10dif-pclmul.ko 2024-03-21T09:58:57.779 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/arch/x86/crypto/des3_ede-x86_64.ko 2024-03-21T09:58:57.779 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/arch/x86/crypto/ghash-clmulni-intel.ko 2024-03-21T09:58:57.779 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/arch/x86/crypto/serpent-avx-x86_64.ko 2024-03-21T09:58:57.779 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/arch/x86/crypto/serpent-avx2.ko 2024-03-21T09:58:57.779 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/arch/x86/crypto/serpent-sse2-x86_64.ko 2024-03-21T09:58:57.779 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/arch/x86/crypto/sha1-ssse3.ko 2024-03-21T09:58:57.779 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/arch/x86/crypto/sha256-ssse3.ko 2024-03-21T09:58:57.779 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/arch/x86/crypto/sha512-ssse3.ko 2024-03-21T09:58:57.779 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/arch/x86/crypto/twofish-avx-x86_64.ko 2024-03-21T09:58:57.780 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/arch/x86/crypto/twofish-x86_64-3way.ko 2024-03-21T09:58:57.780 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/arch/x86/crypto/twofish-x86_64.ko 2024-03-21T09:58:57.780 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/arch/x86/kernel 2024-03-21T09:58:57.780 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/arch/x86/kernel/cpu 2024-03-21T09:58:57.780 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/arch/x86/kernel/cpu/mce 2024-03-21T09:58:57.780 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/arch/x86/kernel/cpu/mce/mce-inject.ko 2024-03-21T09:58:57.780 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/arch/x86/kvm 2024-03-21T09:58:57.780 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/arch/x86/kvm/kvm-amd.ko 2024-03-21T09:58:57.780 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/arch/x86/kvm/kvm-intel.ko 2024-03-21T09:58:57.780 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/arch/x86/kvm/kvm.ko 2024-03-21T09:58:57.780 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto 2024-03-21T09:58:57.780 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/ansi_cprng.ko 2024-03-21T09:58:57.780 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/anubis.ko 2024-03-21T09:58:57.780 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/arc4.ko 2024-03-21T09:58:57.780 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/async_tx 2024-03-21T09:58:57.781 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/async_tx/async_memcpy.ko 2024-03-21T09:58:57.781 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/async_tx/async_pq.ko 2024-03-21T09:58:57.781 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/async_tx/async_raid6_recov.ko 2024-03-21T09:58:57.781 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/async_tx/async_tx.ko 2024-03-21T09:58:57.781 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/async_tx/async_xor.ko 2024-03-21T09:58:57.781 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/async_tx/raid6test.ko 2024-03-21T09:58:57.781 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/authenc.ko 2024-03-21T09:58:57.781 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/authencesn.ko 2024-03-21T09:58:57.781 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/blake2b_generic.ko 2024-03-21T09:58:57.781 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/blowfish_common.ko 2024-03-21T09:58:57.781 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/blowfish_generic.ko 2024-03-21T09:58:57.781 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/camellia_generic.ko 2024-03-21T09:58:57.781 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/cast5_generic.ko 2024-03-21T09:58:57.781 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/cast6_generic.ko 2024-03-21T09:58:57.782 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/cast_common.ko 2024-03-21T09:58:57.782 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/ccm.ko 2024-03-21T09:58:57.782 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/cmac.ko 2024-03-21T09:58:57.782 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/crc32_generic.ko 2024-03-21T09:58:57.782 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/crypto_engine.ko 2024-03-21T09:58:57.782 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/crypto_user.ko 2024-03-21T09:58:57.782 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/cts.ko 2024-03-21T09:58:57.782 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/des_generic.ko 2024-03-21T09:58:57.782 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/dh_generic.ko 2024-03-21T09:58:57.782 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/echainiv.ko 2024-03-21T09:58:57.782 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/essiv.ko 2024-03-21T09:58:57.782 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/fcrypt.ko 2024-03-21T09:58:57.782 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/khazad.ko 2024-03-21T09:58:57.783 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/lz4.ko 2024-03-21T09:58:57.783 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/lz4hc.ko 2024-03-21T09:58:57.783 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/md4.ko 2024-03-21T09:58:57.783 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/michael_mic.ko 2024-03-21T09:58:57.783 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/pcbc.ko 2024-03-21T09:58:57.783 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/pcrypt.ko 2024-03-21T09:58:57.783 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/rmd160.ko 2024-03-21T09:58:57.783 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/seed.ko 2024-03-21T09:58:57.783 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/serpent_generic.ko 2024-03-21T09:58:57.783 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/tcrypt.ko 2024-03-21T09:58:57.783 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/tea.ko 2024-03-21T09:58:57.783 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/twofish_common.ko 2024-03-21T09:58:57.783 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/twofish_generic.ko 2024-03-21T09:58:57.783 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/vmac.ko 2024-03-21T09:58:57.784 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/wp512.ko 2024-03-21T09:58:57.784 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/xcbc.ko 2024-03-21T09:58:57.784 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/xor.ko 2024-03-21T09:58:57.784 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/xxhash_generic.ko 2024-03-21T09:58:57.784 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers 2024-03-21T09:58:57.784 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/acpi 2024-03-21T09:58:57.784 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/acpi/acpi_ipmi.ko 2024-03-21T09:58:57.784 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/acpi/acpi_pad.ko 2024-03-21T09:58:57.784 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/acpi/custom_method.ko 2024-03-21T09:58:57.784 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/acpi/ec_sys.ko 2024-03-21T09:58:57.784 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/acpi/platform_profile.ko 2024-03-21T09:58:57.784 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/acpi/sbs.ko 2024-03-21T09:58:57.784 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/acpi/sbshc.ko 2024-03-21T09:58:57.784 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/acpi/video.ko 2024-03-21T09:58:57.785 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata 2024-03-21T09:58:57.785 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/acard-ahci.ko 2024-03-21T09:58:57.785 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/ahci_platform.ko 2024-03-21T09:58:57.785 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/ata_generic.ko 2024-03-21T09:58:57.785 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/libahci_platform.ko 2024-03-21T09:58:57.785 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_acpi.ko 2024-03-21T09:58:57.785 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_ali.ko 2024-03-21T09:58:57.785 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_amd.ko 2024-03-21T09:58:57.785 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_artop.ko 2024-03-21T09:58:57.785 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_atiixp.ko 2024-03-21T09:58:57.785 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_atp867x.ko 2024-03-21T09:58:57.785 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_cmd640.ko 2024-03-21T09:58:57.785 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_cmd64x.ko 2024-03-21T09:58:57.785 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_cypress.ko 2024-03-21T09:58:57.785 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_efar.ko 2024-03-21T09:58:57.785 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_hpt366.ko 2024-03-21T09:58:57.786 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_hpt37x.ko 2024-03-21T09:58:57.786 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_hpt3x2n.ko 2024-03-21T09:58:57.786 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_hpt3x3.ko 2024-03-21T09:58:57.786 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_it8213.ko 2024-03-21T09:58:57.786 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_it821x.ko 2024-03-21T09:58:57.786 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_jmicron.ko 2024-03-21T09:58:57.786 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_marvell.ko 2024-03-21T09:58:57.786 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_mpiix.ko 2024-03-21T09:58:57.786 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_netcell.ko 2024-03-21T09:58:57.786 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_ninja32.ko 2024-03-21T09:58:57.786 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_ns87410.ko 2024-03-21T09:58:57.786 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_ns87415.ko 2024-03-21T09:58:57.786 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_oldpiix.ko 2024-03-21T09:58:57.786 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_opti.ko 2024-03-21T09:58:57.787 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_optidma.ko 2024-03-21T09:58:57.787 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_pcmcia.ko 2024-03-21T09:58:57.787 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_pdc2027x.ko 2024-03-21T09:58:57.787 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_pdc202xx_old.ko 2024-03-21T09:58:57.787 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_piccolo.ko 2024-03-21T09:58:57.787 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_rdc.ko 2024-03-21T09:58:57.787 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_sch.ko 2024-03-21T09:58:57.787 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_serverworks.ko 2024-03-21T09:58:57.787 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_sil680.ko 2024-03-21T09:58:57.787 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_sis.ko 2024-03-21T09:58:57.787 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_sl82c105.ko 2024-03-21T09:58:57.787 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_triflex.ko 2024-03-21T09:58:57.787 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_via.ko 2024-03-21T09:58:57.788 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pdc_adma.ko 2024-03-21T09:58:57.788 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/sata_inic162x.ko 2024-03-21T09:58:57.788 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/sata_mv.ko 2024-03-21T09:58:57.788 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/sata_nv.ko 2024-03-21T09:58:57.788 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/sata_promise.ko 2024-03-21T09:58:57.788 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/sata_qstor.ko 2024-03-21T09:58:57.788 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/sata_sil.ko 2024-03-21T09:58:57.788 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/sata_sil24.ko 2024-03-21T09:58:57.788 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/sata_sis.ko 2024-03-21T09:58:57.788 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/sata_svw.ko 2024-03-21T09:58:57.788 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/sata_sx4.ko 2024-03-21T09:58:57.788 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/sata_uli.ko 2024-03-21T09:58:57.788 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/sata_via.ko 2024-03-21T09:58:57.788 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/sata_vsc.ko 2024-03-21T09:58:57.789 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/auxdisplay 2024-03-21T09:58:57.789 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/auxdisplay/ks0108.ko 2024-03-21T09:58:57.789 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/base 2024-03-21T09:58:57.789 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/base/regmap 2024-03-21T09:58:57.789 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/base/regmap/regmap-i2c.ko 2024-03-21T09:58:57.789 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/base/regmap/regmap-mmio.ko 2024-03-21T09:58:57.789 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/bcma 2024-03-21T09:58:57.789 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/bcma/bcma.ko 2024-03-21T09:58:57.789 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/block 2024-03-21T09:58:57.789 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/block/aoe 2024-03-21T09:58:57.789 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/block/aoe/aoe.ko 2024-03-21T09:58:57.789 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/block/brd.ko 2024-03-21T09:58:57.789 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/block/drbd 2024-03-21T09:58:57.789 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/block/drbd/drbd.ko 2024-03-21T09:58:57.790 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/block/floppy.ko 2024-03-21T09:58:57.790 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/block/loop.ko 2024-03-21T09:58:57.790 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/block/mtip32xx 2024-03-21T09:58:57.790 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/block/mtip32xx/mtip32xx.ko 2024-03-21T09:58:57.790 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/block/nbd.ko 2024-03-21T09:58:57.790 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/block/null_blk 2024-03-21T09:58:57.790 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/block/null_blk/null_blk.ko 2024-03-21T09:58:57.790 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/block/pktcdvd.ko 2024-03-21T09:58:57.790 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/block/rbd.ko 2024-03-21T09:58:57.790 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/block/virtio_blk.ko 2024-03-21T09:58:57.790 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/block/xen-blkback 2024-03-21T09:58:57.790 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/block/xen-blkback/xen-blkback.ko 2024-03-21T09:58:57.790 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/block/xen-blkfront.ko 2024-03-21T09:58:57.790 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/block/zram 2024-03-21T09:58:57.791 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/block/zram/zram.ko 2024-03-21T09:58:57.791 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/char 2024-03-21T09:58:57.791 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/char/hangcheck-timer.ko 2024-03-21T09:58:57.791 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/char/hw_random 2024-03-21T09:58:57.791 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/char/hw_random/amd-rng.ko 2024-03-21T09:58:57.791 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/char/hw_random/intel-rng.ko 2024-03-21T09:58:57.791 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/char/hw_random/timeriomem-rng.ko 2024-03-21T09:58:57.791 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/char/hw_random/via-rng.ko 2024-03-21T09:58:57.791 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/char/hw_random/virtio-rng.ko 2024-03-21T09:58:57.791 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/char/ipmi 2024-03-21T09:58:57.791 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/char/ipmi/ipmi_devintf.ko 2024-03-21T09:58:57.792 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/char/ipmi/ipmi_msghandler.ko 2024-03-21T09:58:57.792 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/char/ipmi/ipmi_poweroff.ko 2024-03-21T09:58:57.792 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/char/ipmi/ipmi_si.ko 2024-03-21T09:58:57.792 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/char/ipmi/ipmi_ssif.ko 2024-03-21T09:58:57.792 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/char/ipmi/ipmi_watchdog.ko 2024-03-21T09:58:57.792 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/char/lp.ko 2024-03-21T09:58:57.792 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/char/mwave 2024-03-21T09:58:57.792 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/char/mwave/mwave.ko 2024-03-21T09:58:57.792 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/char/ppdev.ko 2024-03-21T09:58:57.793 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/char/tlclk.ko 2024-03-21T09:58:57.793 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/char/tpm 2024-03-21T09:58:57.793 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/char/tpm/tpm.ko 2024-03-21T09:58:57.793 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/char/tpm/tpm_atmel.ko 2024-03-21T09:58:57.793 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/char/tpm/tpm_infineon.ko 2024-03-21T09:58:57.793 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/char/tpm/tpm_nsc.ko 2024-03-21T09:58:57.793 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/char/tpm/tpm_tis.ko 2024-03-21T09:58:57.793 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/char/tpm/tpm_tis_core.ko 2024-03-21T09:58:57.793 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/char/uv_mmtimer.ko 2024-03-21T09:58:57.793 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/char/virtio_console.ko 2024-03-21T09:58:57.793 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/cpufreq 2024-03-21T09:58:57.793 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/cpufreq/acpi-cpufreq.ko 2024-03-21T09:58:57.793 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/cpufreq/amd_freq_sensitivity.ko 2024-03-21T09:58:57.793 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/cpufreq/p4-clockmod.ko 2024-03-21T09:58:57.794 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/cpufreq/pcc-cpufreq.ko 2024-03-21T09:58:57.794 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/cpufreq/powernow-k8.ko 2024-03-21T09:58:57.794 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/cpufreq/speedstep-lib.ko 2024-03-21T09:58:57.794 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/crypto 2024-03-21T09:58:57.794 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/crypto/ccp 2024-03-21T09:58:57.794 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/crypto/ccp/ccp-crypto.ko 2024-03-21T09:58:57.794 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/crypto/ccp/ccp.ko 2024-03-21T09:58:57.794 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/crypto/intel 2024-03-21T09:58:57.794 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/crypto/intel/qat 2024-03-21T09:58:57.794 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/crypto/intel/qat/qat_common 2024-03-21T09:58:57.794 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/crypto/intel/qat/qat_common/intel_qat.ko 2024-03-21T09:58:57.794 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/crypto/intel/qat/qat_dh895xcc 2024-03-21T09:58:57.794 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/crypto/intel/qat/qat_dh895xcc/qat_dh895xcc.ko 2024-03-21T09:58:57.795 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/crypto/padlock-aes.ko 2024-03-21T09:58:57.795 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/crypto/padlock-sha.ko 2024-03-21T09:58:57.795 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/crypto/virtio 2024-03-21T09:58:57.795 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/crypto/virtio/virtio_crypto.ko 2024-03-21T09:58:57.795 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/dca 2024-03-21T09:58:57.795 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/dca/dca.ko 2024-03-21T09:58:57.795 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/devfreq 2024-03-21T09:58:57.795 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/devfreq/governor_simpleondemand.ko 2024-03-21T09:58:57.795 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/dma 2024-03-21T09:58:57.795 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/dma/dw 2024-03-21T09:58:57.795 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/dma/dw/dw_dmac.ko 2024-03-21T09:58:57.795 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/dma/ioat 2024-03-21T09:58:57.795 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/dma/ioat/ioatdma.ko 2024-03-21T09:58:57.795 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/edac 2024-03-21T09:58:57.795 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/edac/amd64_edac.ko 2024-03-21T09:58:57.796 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/edac/e752x_edac.ko 2024-03-21T09:58:57.796 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/edac/edac_mce_amd.ko 2024-03-21T09:58:57.796 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/edac/i3000_edac.ko 2024-03-21T09:58:57.796 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/edac/i3200_edac.ko 2024-03-21T09:58:57.796 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/edac/i5100_edac.ko 2024-03-21T09:58:57.796 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/edac/i5400_edac.ko 2024-03-21T09:58:57.796 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/edac/i7300_edac.ko 2024-03-21T09:58:57.796 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/edac/i7core_edac.ko 2024-03-21T09:58:57.796 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/edac/i82975x_edac.ko 2024-03-21T09:58:57.796 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/edac/ie31200_edac.ko 2024-03-21T09:58:57.796 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/edac/sb_edac.ko 2024-03-21T09:58:57.796 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/edac/x38_edac.ko 2024-03-21T09:58:57.796 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/firmware 2024-03-21T09:58:57.796 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/firmware/edd.ko 2024-03-21T09:58:57.796 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/firmware/iscsi_ibft.ko 2024-03-21T09:58:57.797 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/gpio 2024-03-21T09:58:57.797 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/gpio/gpio-ich.ko 2024-03-21T09:58:57.797 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/gpio/gpio-viperboard.ko 2024-03-21T09:58:57.797 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid 2024-03-21T09:58:57.797 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-appleir.ko 2024-03-21T09:58:57.797 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-aureal.ko 2024-03-21T09:58:57.797 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-axff.ko 2024-03-21T09:58:57.797 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-dr.ko 2024-03-21T09:58:57.797 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-elecom.ko 2024-03-21T09:58:57.797 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-elo.ko 2024-03-21T09:58:57.797 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-emsff.ko 2024-03-21T09:58:57.797 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-gaff.ko 2024-03-21T09:58:57.797 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-gt683r.ko 2024-03-21T09:58:57.797 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-gyration.ko 2024-03-21T09:58:57.798 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-holtek-kbd.ko 2024-03-21T09:58:57.798 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-holtek-mouse.ko 2024-03-21T09:58:57.798 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-holtekff.ko 2024-03-21T09:58:57.798 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-hyperv.ko 2024-03-21T09:58:57.798 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-icade.ko 2024-03-21T09:58:57.798 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-keytouch.ko 2024-03-21T09:58:57.798 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-kye.ko 2024-03-21T09:58:57.798 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-lcpower.ko 2024-03-21T09:58:57.798 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-led.ko 2024-03-21T09:58:57.798 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-lenovo.ko 2024-03-21T09:58:57.798 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-logitech-dj.ko 2024-03-21T09:58:57.798 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-logitech-hidpp.ko 2024-03-21T09:58:57.798 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-multitouch.ko 2024-03-21T09:58:57.798 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-ortek.ko 2024-03-21T09:58:57.799 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-penmount.ko 2024-03-21T09:58:57.799 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-petalynx.ko 2024-03-21T09:58:57.799 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-picolcd.ko 2024-03-21T09:58:57.799 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-pl.ko 2024-03-21T09:58:57.799 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-plantronics.ko 2024-03-21T09:58:57.799 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-primax.ko 2024-03-21T09:58:57.799 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-rmi.ko 2024-03-21T09:58:57.799 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-roccat-arvo.ko 2024-03-21T09:58:57.799 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-roccat-common.ko 2024-03-21T09:58:57.799 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-roccat-isku.ko 2024-03-21T09:58:57.799 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-roccat-kone.ko 2024-03-21T09:58:57.799 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-roccat-koneplus.ko 2024-03-21T09:58:57.799 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-roccat-konepure.ko 2024-03-21T09:58:57.799 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-roccat-kovaplus.ko 2024-03-21T09:58:57.799 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-roccat-lua.ko 2024-03-21T09:58:57.800 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-roccat-pyra.ko 2024-03-21T09:58:57.800 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-roccat-ryos.ko 2024-03-21T09:58:57.800 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-roccat-savu.ko 2024-03-21T09:58:57.800 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-roccat.ko 2024-03-21T09:58:57.800 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-saitek.ko 2024-03-21T09:58:57.800 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-samsung.ko 2024-03-21T09:58:57.800 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-sensor-hub.ko 2024-03-21T09:58:57.800 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-sjoy.ko 2024-03-21T09:58:57.800 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-sony.ko 2024-03-21T09:58:57.800 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-speedlink.ko 2024-03-21T09:58:57.800 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-steelseries.ko 2024-03-21T09:58:57.800 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-sunplus.ko 2024-03-21T09:58:57.800 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-thrustmaster.ko 2024-03-21T09:58:57.800 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-tivo.ko 2024-03-21T09:58:57.800 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-tmff.ko 2024-03-21T09:58:57.801 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-topseed.ko 2024-03-21T09:58:57.801 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-twinhan.ko 2024-03-21T09:58:57.801 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-uclogic.ko 2024-03-21T09:58:57.801 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-waltop.ko 2024-03-21T09:58:57.801 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-wiimote.ko 2024-03-21T09:58:57.801 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-xinmo.ko 2024-03-21T09:58:57.801 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-zpff.ko 2024-03-21T09:58:57.801 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-zydacron.ko 2024-03-21T09:58:57.801 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/uhid.ko 2024-03-21T09:58:57.801 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/wacom.ko 2024-03-21T09:58:57.801 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hv 2024-03-21T09:58:57.801 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hv/hv_balloon.ko 2024-03-21T09:58:57.801 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hv/hv_utils.ko 2024-03-21T09:58:57.801 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hv/hv_vmbus.ko 2024-03-21T09:58:57.801 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon 2024-03-21T09:58:57.802 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/abituguru.ko 2024-03-21T09:58:57.802 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/abituguru3.ko 2024-03-21T09:58:57.802 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/acpi_power_meter.ko 2024-03-21T09:58:57.802 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/ad7414.ko 2024-03-21T09:58:57.802 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/ad7418.ko 2024-03-21T09:58:57.802 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/adc128d818.ko 2024-03-21T09:58:57.802 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/adm1025.ko 2024-03-21T09:58:57.802 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/adm1026.ko 2024-03-21T09:58:57.802 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/adm1029.ko 2024-03-21T09:58:57.802 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/adm1031.ko 2024-03-21T09:58:57.802 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/adm9240.ko 2024-03-21T09:58:57.802 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/ads7828.ko 2024-03-21T09:58:57.802 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/adt7410.ko 2024-03-21T09:58:57.803 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/adt7411.ko 2024-03-21T09:58:57.803 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/adt7462.ko 2024-03-21T09:58:57.803 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/adt7470.ko 2024-03-21T09:58:57.803 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/adt7475.ko 2024-03-21T09:58:57.803 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/adt7x10.ko 2024-03-21T09:58:57.803 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/amc6821.ko 2024-03-21T09:58:57.803 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/applesmc.ko 2024-03-21T09:58:57.803 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/asb100.ko 2024-03-21T09:58:57.803 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/asc7621.ko 2024-03-21T09:58:57.803 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/asus_atk0110.ko 2024-03-21T09:58:57.803 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/atxp1.ko 2024-03-21T09:58:57.803 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/coretemp.ko 2024-03-21T09:58:57.803 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/dell-smm-hwmon.ko 2024-03-21T09:58:57.803 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/dme1737.ko 2024-03-21T09:58:57.803 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/ds1621.ko 2024-03-21T09:58:57.803 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/ds620.ko 2024-03-21T09:58:57.804 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/emc1403.ko 2024-03-21T09:58:57.804 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/emc6w201.ko 2024-03-21T09:58:57.804 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/f71805f.ko 2024-03-21T09:58:57.804 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/f71882fg.ko 2024-03-21T09:58:57.804 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/f75375s.ko 2024-03-21T09:58:57.804 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/fam15h_power.ko 2024-03-21T09:58:57.804 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/fschmd.ko 2024-03-21T09:58:57.804 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/g760a.ko 2024-03-21T09:58:57.804 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/g762.ko 2024-03-21T09:58:57.804 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/gl518sm.ko 2024-03-21T09:58:57.804 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/gl520sm.ko 2024-03-21T09:58:57.804 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/hwmon-vid.ko 2024-03-21T09:58:57.804 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/i5500_temp.ko 2024-03-21T09:58:57.804 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/i5k_amb.ko 2024-03-21T09:58:57.804 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/ibmaem.ko 2024-03-21T09:58:57.804 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/ibmpex.ko 2024-03-21T09:58:57.805 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/ina209.ko 2024-03-21T09:58:57.805 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/ina2xx.ko 2024-03-21T09:58:57.805 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/it87.ko 2024-03-21T09:58:57.805 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/k10temp.ko 2024-03-21T09:58:57.805 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/k8temp.ko 2024-03-21T09:58:57.805 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/lineage-pem.ko 2024-03-21T09:58:57.805 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/lm63.ko 2024-03-21T09:58:57.805 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/lm73.ko 2024-03-21T09:58:57.805 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/lm75.ko 2024-03-21T09:58:57.805 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/lm77.ko 2024-03-21T09:58:57.805 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/lm78.ko 2024-03-21T09:58:57.805 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/lm80.ko 2024-03-21T09:58:57.805 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/lm83.ko 2024-03-21T09:58:57.805 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/lm85.ko 2024-03-21T09:58:57.806 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/lm87.ko 2024-03-21T09:58:57.806 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/lm90.ko 2024-03-21T09:58:57.806 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/lm92.ko 2024-03-21T09:58:57.806 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/lm93.ko 2024-03-21T09:58:57.806 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/lm95234.ko 2024-03-21T09:58:57.806 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/lm95241.ko 2024-03-21T09:58:57.806 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/lm95245.ko 2024-03-21T09:58:57.806 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/ltc2945.ko 2024-03-21T09:58:57.806 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/ltc4151.ko 2024-03-21T09:58:57.806 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/ltc4215.ko 2024-03-21T09:58:57.806 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/ltc4222.ko 2024-03-21T09:58:57.806 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/ltc4245.ko 2024-03-21T09:58:57.806 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/ltc4260.ko 2024-03-21T09:58:57.806 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/ltc4261.ko 2024-03-21T09:58:57.807 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/max16065.ko 2024-03-21T09:58:57.807 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/max1619.ko 2024-03-21T09:58:57.807 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/max1668.ko 2024-03-21T09:58:57.807 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/max197.ko 2024-03-21T09:58:57.807 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/max6639.ko 2024-03-21T09:58:57.807 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/max6650.ko 2024-03-21T09:58:57.807 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/max6697.ko 2024-03-21T09:58:57.807 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/mcp3021.ko 2024-03-21T09:58:57.807 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/nct6683.ko 2024-03-21T09:58:57.807 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/nct6775-core.ko 2024-03-21T09:58:57.807 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/nct6775.ko 2024-03-21T09:58:57.807 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/nct7802.ko 2024-03-21T09:58:57.807 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/pc87360.ko 2024-03-21T09:58:57.807 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/pc87427.ko 2024-03-21T09:58:57.808 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/pcf8591.ko 2024-03-21T09:58:57.808 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/pmbus 2024-03-21T09:58:57.808 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/pmbus/adm1275.ko 2024-03-21T09:58:57.808 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/pmbus/lm25066.ko 2024-03-21T09:58:57.808 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/pmbus/ltc2978.ko 2024-03-21T09:58:57.808 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/pmbus/max16064.ko 2024-03-21T09:58:57.808 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/pmbus/max34440.ko 2024-03-21T09:58:57.808 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/pmbus/max8688.ko 2024-03-21T09:58:57.808 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/pmbus/pmbus.ko 2024-03-21T09:58:57.808 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/pmbus/pmbus_core.ko 2024-03-21T09:58:57.808 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/pmbus/tps40422.ko 2024-03-21T09:58:57.808 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/pmbus/ucd9000.ko 2024-03-21T09:58:57.808 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/pmbus/ucd9200.ko 2024-03-21T09:58:57.808 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/pmbus/zl6100.ko 2024-03-21T09:58:57.809 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/powr1220.ko 2024-03-21T09:58:57.809 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/sch5627.ko 2024-03-21T09:58:57.809 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/sch5636.ko 2024-03-21T09:58:57.809 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/sch56xx-common.ko 2024-03-21T09:58:57.809 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/sht15.ko 2024-03-21T09:58:57.809 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/sht21.ko 2024-03-21T09:58:57.809 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/shtc1.ko 2024-03-21T09:58:57.809 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/sis5595.ko 2024-03-21T09:58:57.809 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/smsc47b397.ko 2024-03-21T09:58:57.809 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/smsc47m1.ko 2024-03-21T09:58:57.809 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/smsc47m192.ko 2024-03-21T09:58:57.809 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/thmc50.ko 2024-03-21T09:58:57.809 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/tmp102.ko 2024-03-21T09:58:57.809 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/tmp103.ko 2024-03-21T09:58:57.810 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/tmp401.ko 2024-03-21T09:58:57.810 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/tmp421.ko 2024-03-21T09:58:57.810 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/via-cputemp.ko 2024-03-21T09:58:57.810 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/via686a.ko 2024-03-21T09:58:57.810 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/vt1211.ko 2024-03-21T09:58:57.810 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/vt8231.ko 2024-03-21T09:58:57.810 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/w83627ehf.ko 2024-03-21T09:58:57.810 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/w83627hf.ko 2024-03-21T09:58:57.810 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/w83781d.ko 2024-03-21T09:58:57.810 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/w83791d.ko 2024-03-21T09:58:57.810 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/w83792d.ko 2024-03-21T09:58:57.810 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/w83793.ko 2024-03-21T09:58:57.810 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/w83795.ko 2024-03-21T09:58:57.810 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/w83l785ts.ko 2024-03-21T09:58:57.811 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/w83l786ng.ko 2024-03-21T09:58:57.811 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/i2c 2024-03-21T09:58:57.811 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/i2c/algos 2024-03-21T09:58:57.811 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/i2c/algos/i2c-algo-bit.ko 2024-03-21T09:58:57.811 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/i2c/algos/i2c-algo-pca.ko 2024-03-21T09:58:57.811 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/i2c/busses 2024-03-21T09:58:57.811 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/i2c/busses/i2c-amd756-s4882.ko 2024-03-21T09:58:57.811 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/i2c/busses/i2c-amd756.ko 2024-03-21T09:58:57.811 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/i2c/busses/i2c-amd8111.ko 2024-03-21T09:58:57.811 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/i2c/busses/i2c-ccgx-ucsi.ko 2024-03-21T09:58:57.811 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/i2c/busses/i2c-designware-core.ko 2024-03-21T09:58:57.811 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/i2c/busses/i2c-designware-pci.ko 2024-03-21T09:58:57.811 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/i2c/busses/i2c-designware-platform.ko 2024-03-21T09:58:57.811 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/i2c/busses/i2c-diolan-u2c.ko 2024-03-21T09:58:57.812 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/i2c/busses/i2c-i801.ko 2024-03-21T09:58:57.812 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/i2c/busses/i2c-isch.ko 2024-03-21T09:58:57.812 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/i2c/busses/i2c-ismt.ko 2024-03-21T09:58:57.812 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/i2c/busses/i2c-nforce2-s4985.ko 2024-03-21T09:58:57.812 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/i2c/busses/i2c-nforce2.ko 2024-03-21T09:58:57.812 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/i2c/busses/i2c-parport.ko 2024-03-21T09:58:57.812 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/i2c/busses/i2c-pca-platform.ko 2024-03-21T09:58:57.812 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/i2c/busses/i2c-piix4.ko 2024-03-21T09:58:57.812 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/i2c/busses/i2c-scmi.ko 2024-03-21T09:58:57.812 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/i2c/busses/i2c-simtec.ko 2024-03-21T09:58:57.812 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/i2c/busses/i2c-sis96x.ko 2024-03-21T09:58:57.812 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/i2c/busses/i2c-tiny-usb.ko 2024-03-21T09:58:57.812 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/i2c/busses/i2c-via.ko 2024-03-21T09:58:57.812 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/i2c/busses/i2c-viapro.ko 2024-03-21T09:58:57.813 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/i2c/busses/i2c-viperboard.ko 2024-03-21T09:58:57.813 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/i2c/i2c-dev.ko 2024-03-21T09:58:57.813 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/i2c/i2c-mux.ko 2024-03-21T09:58:57.813 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/i2c/i2c-slave-eeprom.ko 2024-03-21T09:58:57.813 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/i2c/i2c-smbus.ko 2024-03-21T09:58:57.813 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/i2c/i2c-stub.ko 2024-03-21T09:58:57.813 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/infiniband 2024-03-21T09:58:57.813 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/infiniband/core 2024-03-21T09:58:57.813 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/infiniband/core/ib_cm.ko 2024-03-21T09:58:57.813 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/infiniband/core/ib_core.ko 2024-03-21T09:58:57.813 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/infiniband/core/ib_umad.ko 2024-03-21T09:58:57.813 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/infiniband/core/ib_uverbs.ko 2024-03-21T09:58:57.813 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/infiniband/core/iw_cm.ko 2024-03-21T09:58:57.813 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/infiniband/core/rdma_cm.ko 2024-03-21T09:58:57.813 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/infiniband/core/rdma_ucm.ko 2024-03-21T09:58:57.814 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/infiniband/hw 2024-03-21T09:58:57.814 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/infiniband/hw/cxgb4 2024-03-21T09:58:57.814 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/infiniband/hw/cxgb4/iw_cxgb4.ko 2024-03-21T09:58:57.814 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/infiniband/hw/mlx4 2024-03-21T09:58:57.814 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/infiniband/hw/mlx4/mlx4_ib.ko 2024-03-21T09:58:57.814 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/infiniband/hw/mlx5 2024-03-21T09:58:57.814 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/infiniband/hw/mlx5/mlx5_ib.ko 2024-03-21T09:58:57.814 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/infiniband/hw/mthca 2024-03-21T09:58:57.814 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/infiniband/hw/mthca/ib_mthca.ko 2024-03-21T09:58:57.814 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/infiniband/ulp 2024-03-21T09:58:57.814 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/infiniband/ulp/ipoib 2024-03-21T09:58:57.814 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/infiniband/ulp/ipoib/ib_ipoib.ko 2024-03-21T09:58:57.814 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/infiniband/ulp/iser 2024-03-21T09:58:57.815 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/infiniband/ulp/iser/ib_iser.ko 2024-03-21T09:58:57.815 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/infiniband/ulp/isert 2024-03-21T09:58:57.815 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/infiniband/ulp/isert/ib_isert.ko 2024-03-21T09:58:57.815 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/infiniband/ulp/srp 2024-03-21T09:58:57.815 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/infiniband/ulp/srp/ib_srp.ko 2024-03-21T09:58:57.815 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/infiniband/ulp/srpt 2024-03-21T09:58:57.815 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/infiniband/ulp/srpt/ib_srpt.ko 2024-03-21T09:58:57.815 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/input 2024-03-21T09:58:57.815 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/input/gameport 2024-03-21T09:58:57.815 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/input/gameport/emu10k1-gp.ko 2024-03-21T09:58:57.815 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/input/gameport/fm801-gp.ko 2024-03-21T09:58:57.815 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/input/gameport/gameport.ko 2024-03-21T09:58:57.815 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/input/joydev.ko 2024-03-21T09:58:57.815 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/input/keyboard 2024-03-21T09:58:57.816 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/input/keyboard/gpio_keys.ko 2024-03-21T09:58:57.816 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/input/mouse 2024-03-21T09:58:57.816 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/input/mouse/appletouch.ko 2024-03-21T09:58:57.816 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/input/mouse/bcm5974.ko 2024-03-21T09:58:57.816 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/input/mouse/cyapatp.ko 2024-03-21T09:58:57.816 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/input/mouse/elan_i2c.ko 2024-03-21T09:58:57.816 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/input/mouse/sermouse.ko 2024-03-21T09:58:57.816 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/input/mouse/synaptics_i2c.ko 2024-03-21T09:58:57.816 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/input/mouse/synaptics_usb.ko 2024-03-21T09:58:57.816 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/input/mouse/vsxxxaa.ko 2024-03-21T09:58:57.816 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/input/rmi4 2024-03-21T09:58:57.816 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/input/rmi4/rmi_core.ko 2024-03-21T09:58:57.816 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/input/serio 2024-03-21T09:58:57.816 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/input/serio/altera_ps2.ko 2024-03-21T09:58:57.817 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/input/serio/arc_ps2.ko 2024-03-21T09:58:57.817 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/input/serio/hyperv-keyboard.ko 2024-03-21T09:58:57.817 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/input/serio/serio_raw.ko 2024-03-21T09:58:57.817 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/input/sparse-keymap.ko 2024-03-21T09:58:57.817 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/leds 2024-03-21T09:58:57.817 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/leds/leds-blinkm.ko 2024-03-21T09:58:57.817 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/leds/leds-lm3530.ko 2024-03-21T09:58:57.817 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/leds/leds-lp3944.ko 2024-03-21T09:58:57.817 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/leds/leds-ss4200.ko 2024-03-21T09:58:57.817 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/leds/trigger 2024-03-21T09:58:57.817 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/leds/trigger/ledtrig-audio.ko 2024-03-21T09:58:57.817 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/leds/trigger/ledtrig-backlight.ko 2024-03-21T09:58:57.817 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/leds/trigger/ledtrig-camera.ko 2024-03-21T09:58:57.817 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/leds/trigger/ledtrig-default-on.ko 2024-03-21T09:58:57.818 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/leds/trigger/ledtrig-gpio.ko 2024-03-21T09:58:57.818 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/leds/trigger/ledtrig-heartbeat.ko 2024-03-21T09:58:57.818 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/leds/trigger/ledtrig-oneshot.ko 2024-03-21T09:58:57.818 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/leds/trigger/ledtrig-timer.ko 2024-03-21T09:58:57.818 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/leds/trigger/ledtrig-transient.ko 2024-03-21T09:58:57.818 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/md 2024-03-21T09:58:57.818 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/md/bcache 2024-03-21T09:58:57.818 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/md/bcache/bcache.ko 2024-03-21T09:58:57.818 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/md/dm-bio-prison.ko 2024-03-21T09:58:57.818 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/md/dm-cache-smq.ko 2024-03-21T09:58:57.818 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/md/dm-cache.ko 2024-03-21T09:58:57.818 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/md/dm-crypt.ko 2024-03-21T09:58:57.818 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/md/dm-delay.ko 2024-03-21T09:58:57.818 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/md/dm-flakey.ko 2024-03-21T09:58:57.819 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/md/dm-log-userspace.ko 2024-03-21T09:58:57.819 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/md/dm-multipath.ko 2024-03-21T09:58:57.819 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/md/dm-queue-length.ko 2024-03-21T09:58:57.819 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/md/dm-raid.ko 2024-03-21T09:58:57.819 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/md/dm-round-robin.ko 2024-03-21T09:58:57.819 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/md/dm-service-time.ko 2024-03-21T09:58:57.819 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/md/dm-switch.ko 2024-03-21T09:58:57.819 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/md/dm-thin-pool.ko 2024-03-21T09:58:57.819 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/md/dm-verity.ko 2024-03-21T09:58:57.819 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/md/persistent-data 2024-03-21T09:58:57.819 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/md/persistent-data/dm-persistent-data.ko 2024-03-21T09:58:57.819 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/md/raid0.ko 2024-03-21T09:58:57.819 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/md/raid1.ko 2024-03-21T09:58:57.819 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/md/raid10.ko 2024-03-21T09:58:57.820 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/md/raid456.ko 2024-03-21T09:58:57.820 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/message 2024-03-21T09:58:57.820 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/message/fusion 2024-03-21T09:58:57.820 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/message/fusion/mptbase.ko 2024-03-21T09:58:57.820 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/message/fusion/mptctl.ko 2024-03-21T09:58:57.820 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/message/fusion/mptfc.ko 2024-03-21T09:58:57.820 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/message/fusion/mptlan.ko 2024-03-21T09:58:57.820 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/message/fusion/mptsas.ko 2024-03-21T09:58:57.820 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/message/fusion/mptscsih.ko 2024-03-21T09:58:57.820 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/message/fusion/mptspi.ko 2024-03-21T09:58:57.820 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/mfd 2024-03-21T09:58:57.820 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/mfd/lpc_ich.ko 2024-03-21T09:58:57.820 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/mfd/lpc_sch.ko 2024-03-21T09:58:57.820 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/mfd/mfd-core.ko 2024-03-21T09:58:57.820 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/mfd/sm501.ko 2024-03-21T09:58:57.820 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/mfd/viperboard.ko 2024-03-21T09:58:57.821 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/mfd/vx855.ko 2024-03-21T09:58:57.821 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/mfd/wl1273-core.ko 2024-03-21T09:58:57.821 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/misc 2024-03-21T09:58:57.821 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/misc/altera-stapl 2024-03-21T09:58:57.821 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/misc/altera-stapl/altera-stapl.ko 2024-03-21T09:58:57.821 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/misc/apds9802als.ko 2024-03-21T09:58:57.821 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/misc/apds990x.ko 2024-03-21T09:58:57.821 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/misc/bh1770glc.ko 2024-03-21T09:58:57.821 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/misc/cb710 2024-03-21T09:58:57.821 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/misc/cb710/cb710.ko 2024-03-21T09:58:57.821 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/misc/eeprom 2024-03-21T09:58:57.821 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/misc/eeprom/at24.ko 2024-03-21T09:58:57.821 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/misc/eeprom/eeprom_93cx6.ko 2024-03-21T09:58:57.821 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/misc/eeprom/max6875.ko 2024-03-21T09:58:57.822 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/misc/enclosure.ko 2024-03-21T09:58:57.822 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/misc/hpilo.ko 2024-03-21T09:58:57.822 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/misc/isl29003.ko 2024-03-21T09:58:57.822 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/misc/isl29020.ko 2024-03-21T09:58:57.822 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/misc/lis3lv02d 2024-03-21T09:58:57.822 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/misc/lis3lv02d/lis3lv02d.ko 2024-03-21T09:58:57.822 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/misc/lis3lv02d/lis3lv02d_i2c.ko 2024-03-21T09:58:57.822 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/misc/mei 2024-03-21T09:58:57.822 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/misc/mei/mei-me.ko 2024-03-21T09:58:57.822 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/misc/mei/mei-txe.ko 2024-03-21T09:58:57.822 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/misc/mei/mei.ko 2024-03-21T09:58:57.822 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/misc/sgi-gru 2024-03-21T09:58:57.822 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/misc/sgi-gru/gru.ko 2024-03-21T09:58:57.822 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/misc/sgi-xp 2024-03-21T09:58:57.823 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/misc/sgi-xp/xp.ko 2024-03-21T09:58:57.823 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/misc/sgi-xp/xpc.ko 2024-03-21T09:58:57.823 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/misc/sgi-xp/xpnet.ko 2024-03-21T09:58:57.823 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/misc/tifm_7xx1.ko 2024-03-21T09:58:57.823 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/misc/tifm_core.ko 2024-03-21T09:58:57.823 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/misc/tsl2550.ko 2024-03-21T09:58:57.823 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/misc/vmw_balloon.ko 2024-03-21T09:58:57.823 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/misc/vmw_vmci 2024-03-21T09:58:57.823 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/misc/vmw_vmci/vmw_vmci.ko 2024-03-21T09:58:57.823 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net 2024-03-21T09:58:57.823 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/bonding 2024-03-21T09:58:57.823 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/bonding/bonding.ko 2024-03-21T09:58:57.823 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/dsa 2024-03-21T09:58:57.823 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/dsa/b53 2024-03-21T09:58:57.824 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/dsa/b53/b53_common.ko 2024-03-21T09:58:57.824 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/dsa/bcm-sf2.ko 2024-03-21T09:58:57.824 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/dsa/mv88e6060.ko 2024-03-21T09:58:57.824 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/dsa/mv88e6xxx 2024-03-21T09:58:57.824 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/dsa/mv88e6xxx/mv88e6xxx.ko 2024-03-21T09:58:57.824 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/dummy.ko 2024-03-21T09:58:57.824 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/eql.ko 2024-03-21T09:58:57.824 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet 2024-03-21T09:58:57.824 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/3com 2024-03-21T09:58:57.824 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/3com/3c574_cs.ko 2024-03-21T09:58:57.824 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/3com/3c589_cs.ko 2024-03-21T09:58:57.824 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/3com/3c59x.ko 2024-03-21T09:58:57.824 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/3com/typhoon.ko 2024-03-21T09:58:57.824 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/8390 2024-03-21T09:58:57.824 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/8390/8390.ko 2024-03-21T09:58:57.825 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/8390/axnet_cs.ko 2024-03-21T09:58:57.825 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/8390/ne2k-pci.ko 2024-03-21T09:58:57.825 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/8390/pcnet_cs.ko 2024-03-21T09:58:57.825 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/adaptec 2024-03-21T09:58:57.825 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/adaptec/starfire.ko 2024-03-21T09:58:57.825 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/agere 2024-03-21T09:58:57.825 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/agere/et131x.ko 2024-03-21T09:58:57.825 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/alteon 2024-03-21T09:58:57.825 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/alteon/acenic.ko 2024-03-21T09:58:57.825 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/altera 2024-03-21T09:58:57.825 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/altera/altera_tse.ko 2024-03-21T09:58:57.825 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/amd 2024-03-21T09:58:57.825 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/amd/amd8111e.ko 2024-03-21T09:58:57.825 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/amd/nmclan_cs.ko 2024-03-21T09:58:57.825 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/amd/pcnet32.ko 2024-03-21T09:58:57.826 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/atheros 2024-03-21T09:58:57.826 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/atheros/alx 2024-03-21T09:58:57.826 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/atheros/alx/alx.ko 2024-03-21T09:58:57.826 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/atheros/atl1c 2024-03-21T09:58:57.826 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/atheros/atl1c/atl1c.ko 2024-03-21T09:58:57.826 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/atheros/atl1e 2024-03-21T09:58:57.826 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/atheros/atl1e/atl1e.ko 2024-03-21T09:58:57.826 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/atheros/atlx 2024-03-21T09:58:57.826 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/atheros/atlx/atl1.ko 2024-03-21T09:58:57.826 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/atheros/atlx/atl2.ko 2024-03-21T09:58:57.826 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/broadcom 2024-03-21T09:58:57.826 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/broadcom/b44.ko 2024-03-21T09:58:57.826 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/broadcom/bnx2.ko 2024-03-21T09:58:57.826 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/broadcom/bnx2x 2024-03-21T09:58:57.827 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/broadcom/bnx2x/bnx2x.ko 2024-03-21T09:58:57.827 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/broadcom/cnic.ko 2024-03-21T09:58:57.827 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/broadcom/genet 2024-03-21T09:58:57.827 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/broadcom/genet/genet.ko 2024-03-21T09:58:57.827 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/broadcom/tg3.ko 2024-03-21T09:58:57.827 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/brocade 2024-03-21T09:58:57.827 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/brocade/bna 2024-03-21T09:58:57.827 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/brocade/bna/bna.ko 2024-03-21T09:58:57.827 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/chelsio 2024-03-21T09:58:57.827 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/chelsio/cxgb 2024-03-21T09:58:57.827 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/chelsio/cxgb/cxgb.ko 2024-03-21T09:58:57.827 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/chelsio/cxgb3 2024-03-21T09:58:57.827 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/chelsio/cxgb3/cxgb3.ko 2024-03-21T09:58:57.827 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/chelsio/cxgb4 2024-03-21T09:58:57.827 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/chelsio/cxgb4/cxgb4.ko 2024-03-21T09:58:57.828 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/chelsio/cxgb4vf 2024-03-21T09:58:57.828 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/chelsio/cxgb4vf/cxgb4vf.ko 2024-03-21T09:58:57.828 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/chelsio/libcxgb 2024-03-21T09:58:57.828 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/chelsio/libcxgb/libcxgb.ko 2024-03-21T09:58:57.828 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/cisco 2024-03-21T09:58:57.828 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/cisco/enic 2024-03-21T09:58:57.828 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/cisco/enic/enic.ko 2024-03-21T09:58:57.828 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/dec 2024-03-21T09:58:57.828 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/dec/tulip 2024-03-21T09:58:57.828 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/dec/tulip/de2104x.ko 2024-03-21T09:58:57.828 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/dec/tulip/dmfe.ko 2024-03-21T09:58:57.828 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/dec/tulip/tulip.ko 2024-03-21T09:58:57.828 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/dec/tulip/uli526x.ko 2024-03-21T09:58:57.829 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/dec/tulip/winbond-840.ko 2024-03-21T09:58:57.829 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/dec/tulip/xircom_cb.ko 2024-03-21T09:58:57.829 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/dlink 2024-03-21T09:58:57.829 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/dlink/dl2k.ko 2024-03-21T09:58:57.829 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/dlink/sundance.ko 2024-03-21T09:58:57.829 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/dnet.ko 2024-03-21T09:58:57.829 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/emulex 2024-03-21T09:58:57.829 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/emulex/benet 2024-03-21T09:58:57.829 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/emulex/benet/be2net.ko 2024-03-21T09:58:57.829 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/ethoc.ko 2024-03-21T09:58:57.829 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/fealnx.ko 2024-03-21T09:58:57.829 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/intel 2024-03-21T09:58:57.829 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/intel/e100.ko 2024-03-21T09:58:57.829 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/intel/e1000 2024-03-21T09:58:57.829 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/intel/e1000/e1000.ko 2024-03-21T09:58:57.830 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/intel/e1000e 2024-03-21T09:58:57.830 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/intel/e1000e/e1000e.ko 2024-03-21T09:58:57.830 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/intel/fm10k 2024-03-21T09:58:57.830 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/intel/fm10k/fm10k.ko 2024-03-21T09:58:57.830 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/intel/i40e 2024-03-21T09:58:57.830 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/intel/i40e/i40e.ko 2024-03-21T09:58:57.830 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/intel/iavf 2024-03-21T09:58:57.830 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/intel/iavf/iavf.ko 2024-03-21T09:58:57.830 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/intel/igb 2024-03-21T09:58:57.830 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/intel/igb/igb.ko 2024-03-21T09:58:57.830 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/intel/igbvf 2024-03-21T09:58:57.830 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/intel/igbvf/igbvf.ko 2024-03-21T09:58:57.830 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/intel/ixgbe 2024-03-21T09:58:57.830 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/intel/ixgbe/ixgbe.ko 2024-03-21T09:58:57.831 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/intel/ixgbevf 2024-03-21T09:58:57.831 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/intel/ixgbevf/ixgbevf.ko 2024-03-21T09:58:57.831 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/jme.ko 2024-03-21T09:58:57.831 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/marvell 2024-03-21T09:58:57.831 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/marvell/mvmdio.ko 2024-03-21T09:58:57.831 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/marvell/skge.ko 2024-03-21T09:58:57.831 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/marvell/sky2.ko 2024-03-21T09:58:57.831 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/mellanox 2024-03-21T09:58:57.831 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/mellanox/mlx4 2024-03-21T09:58:57.831 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_core.ko 2024-03-21T09:58:57.831 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_en.ko 2024-03-21T09:58:57.831 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/mellanox/mlx5 2024-03-21T09:58:57.831 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/mellanox/mlx5/core 2024-03-21T09:58:57.831 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/mellanox/mlx5/core/mlx5_core.ko 2024-03-21T09:58:57.832 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/micrel 2024-03-21T09:58:57.832 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/micrel/ksz884x.ko 2024-03-21T09:58:57.832 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/myricom 2024-03-21T09:58:57.832 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/myricom/myri10ge 2024-03-21T09:58:57.832 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/myricom/myri10ge/myri10ge.ko 2024-03-21T09:58:57.832 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/natsemi 2024-03-21T09:58:57.832 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/natsemi/natsemi.ko 2024-03-21T09:58:57.832 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/natsemi/ns83820.ko 2024-03-21T09:58:57.832 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/neterion 2024-03-21T09:58:57.832 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/neterion/s2io.ko 2024-03-21T09:58:57.832 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/nvidia 2024-03-21T09:58:57.832 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/nvidia/forcedeth.ko 2024-03-21T09:58:57.832 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/packetengines 2024-03-21T09:58:57.832 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/packetengines/hamachi.ko 2024-03-21T09:58:57.833 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/packetengines/yellowfin.ko 2024-03-21T09:58:57.833 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/qlogic 2024-03-21T09:58:57.833 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/qlogic/netxen 2024-03-21T09:58:57.833 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/qlogic/netxen/netxen_nic.ko 2024-03-21T09:58:57.833 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/qlogic/qla3xxx.ko 2024-03-21T09:58:57.833 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/qlogic/qlcnic 2024-03-21T09:58:57.833 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/qlogic/qlcnic/qlcnic.ko 2024-03-21T09:58:57.833 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/rdc 2024-03-21T09:58:57.833 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/rdc/r6040.ko 2024-03-21T09:58:57.833 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/realtek 2024-03-21T09:58:57.833 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/realtek/8139cp.ko 2024-03-21T09:58:57.833 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/realtek/8139too.ko 2024-03-21T09:58:57.833 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/realtek/atp.ko 2024-03-21T09:58:57.833 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/realtek/r8169.ko 2024-03-21T09:58:57.833 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/rocker 2024-03-21T09:58:57.834 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/rocker/rocker.ko 2024-03-21T09:58:57.834 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/sfc 2024-03-21T09:58:57.834 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/sfc/sfc.ko 2024-03-21T09:58:57.834 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/silan 2024-03-21T09:58:57.834 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/silan/sc92031.ko 2024-03-21T09:58:57.834 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/sis 2024-03-21T09:58:57.834 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/sis/sis190.ko 2024-03-21T09:58:57.834 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/sis/sis900.ko 2024-03-21T09:58:57.834 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/smsc 2024-03-21T09:58:57.834 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/smsc/epic100.ko 2024-03-21T09:58:57.834 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/smsc/smc91c92_cs.ko 2024-03-21T09:58:57.834 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/smsc/smsc911x.ko 2024-03-21T09:58:57.834 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/smsc/smsc9420.ko 2024-03-21T09:58:57.835 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/stmicro 2024-03-21T09:58:57.835 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/stmicro/stmmac 2024-03-21T09:58:57.835 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/stmicro/stmmac/dwmac-intel.ko 2024-03-21T09:58:57.835 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/stmicro/stmmac/stmmac.ko 2024-03-21T09:58:57.835 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/sun 2024-03-21T09:58:57.835 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/sun/cassini.ko 2024-03-21T09:58:57.835 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/sun/niu.ko 2024-03-21T09:58:57.835 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/sun/sungem.ko 2024-03-21T09:58:57.835 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/sun/sunhme.ko 2024-03-21T09:58:57.835 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/tehuti 2024-03-21T09:58:57.835 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/tehuti/tehuti.ko 2024-03-21T09:58:57.835 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/ti 2024-03-21T09:58:57.835 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/ti/tlan.ko 2024-03-21T09:58:57.835 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/via 2024-03-21T09:58:57.835 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/via/via-rhine.ko 2024-03-21T09:58:57.835 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/via/via-velocity.ko 2024-03-21T09:58:57.836 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/wiznet 2024-03-21T09:58:57.836 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/wiznet/w5100.ko 2024-03-21T09:58:57.836 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/wiznet/w5300.ko 2024-03-21T09:58:57.836 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/xircom 2024-03-21T09:58:57.836 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/xircom/xirc2ps_cs.ko 2024-03-21T09:58:57.836 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/geneve.ko 2024-03-21T09:58:57.836 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/hyperv 2024-03-21T09:58:57.836 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/hyperv/hv_netvsc.ko 2024-03-21T09:58:57.836 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ieee802154 2024-03-21T09:58:57.836 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ieee802154/fakelb.ko 2024-03-21T09:58:57.836 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ifb.ko 2024-03-21T09:58:57.836 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ipvlan 2024-03-21T09:58:57.836 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ipvlan/ipvlan.ko 2024-03-21T09:58:57.836 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/macvlan.ko 2024-03-21T09:58:57.836 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/macvtap.ko 2024-03-21T09:58:57.837 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/mdio 2024-03-21T09:58:57.837 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/mdio.ko 2024-03-21T09:58:57.837 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/mdio/mdio-bcm-unimac.ko 2024-03-21T09:58:57.837 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/mdio/mdio-bitbang.ko 2024-03-21T09:58:57.837 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/mdio/mdio-regmap.ko 2024-03-21T09:58:57.837 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/mii.ko 2024-03-21T09:58:57.837 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/net_failover.ko 2024-03-21T09:58:57.837 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/netconsole.ko 2024-03-21T09:58:57.837 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/nlmon.ko 2024-03-21T09:58:57.837 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/pcs 2024-03-21T09:58:57.837 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/pcs/pcs-lynx.ko 2024-03-21T09:58:57.837 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/pcs/pcs_xpcs.ko 2024-03-21T09:58:57.837 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/phy 2024-03-21T09:58:57.837 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/phy/amd.ko 2024-03-21T09:58:57.838 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/phy/bcm-phy-lib.ko 2024-03-21T09:58:57.838 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/phy/bcm-phy-ptp.ko 2024-03-21T09:58:57.838 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/phy/bcm7xxx.ko 2024-03-21T09:58:57.838 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/phy/bcm87xx.ko 2024-03-21T09:58:57.838 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/phy/broadcom.ko 2024-03-21T09:58:57.838 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/phy/cicada.ko 2024-03-21T09:58:57.838 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/phy/davicom.ko 2024-03-21T09:58:57.838 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/phy/dp83640.ko 2024-03-21T09:58:57.838 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/phy/et1011c.ko 2024-03-21T09:58:57.838 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/phy/icplus.ko 2024-03-21T09:58:57.838 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/phy/lxt.ko 2024-03-21T09:58:57.838 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/phy/marvell.ko 2024-03-21T09:58:57.838 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/phy/micrel.ko 2024-03-21T09:58:57.838 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/phy/national.ko 2024-03-21T09:58:57.839 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/phy/phylink.ko 2024-03-21T09:58:57.839 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/phy/qsemi.ko 2024-03-21T09:58:57.839 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/phy/realtek.ko 2024-03-21T09:58:57.839 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/phy/smsc.ko 2024-03-21T09:58:57.839 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/phy/ste10Xp.ko 2024-03-21T09:58:57.839 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/phy/vitesse.ko 2024-03-21T09:58:57.839 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ppp 2024-03-21T09:58:57.839 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ppp/bsd_comp.ko 2024-03-21T09:58:57.839 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ppp/ppp_async.ko 2024-03-21T09:58:57.839 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ppp/ppp_deflate.ko 2024-03-21T09:58:57.839 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ppp/ppp_generic.ko 2024-03-21T09:58:57.839 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ppp/ppp_mppe.ko 2024-03-21T09:58:57.839 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ppp/ppp_synctty.ko 2024-03-21T09:58:57.839 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ppp/pppoe.ko 2024-03-21T09:58:57.839 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ppp/pppox.ko 2024-03-21T09:58:57.840 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ppp/pptp.ko 2024-03-21T09:58:57.840 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/slip 2024-03-21T09:58:57.840 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/slip/slhc.ko 2024-03-21T09:58:57.840 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/slip/slip.ko 2024-03-21T09:58:57.840 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/sungem_phy.ko 2024-03-21T09:58:57.840 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/tap.ko 2024-03-21T09:58:57.840 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/team 2024-03-21T09:58:57.840 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/team/team.ko 2024-03-21T09:58:57.840 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/team/team_mode_activebackup.ko 2024-03-21T09:58:57.840 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/team/team_mode_broadcast.ko 2024-03-21T09:58:57.840 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/team/team_mode_loadbalance.ko 2024-03-21T09:58:57.840 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/team/team_mode_random.ko 2024-03-21T09:58:57.840 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/team/team_mode_roundrobin.ko 2024-03-21T09:58:57.840 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/tun.ko 2024-03-21T09:58:57.840 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/veth.ko 2024-03-21T09:58:57.841 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/virtio_net.ko 2024-03-21T09:58:57.841 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/vmxnet3 2024-03-21T09:58:57.841 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/vmxnet3/vmxnet3.ko 2024-03-21T09:58:57.841 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/vxlan 2024-03-21T09:58:57.841 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/vxlan/vxlan.ko 2024-03-21T09:58:57.841 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/xen-netback 2024-03-21T09:58:57.841 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/xen-netback/xen-netback.ko 2024-03-21T09:58:57.841 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/xen-netfront.ko 2024-03-21T09:58:57.841 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ntb 2024-03-21T09:58:57.841 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ntb/ntb.ko 2024-03-21T09:58:57.841 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/nvme 2024-03-21T09:58:57.841 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/nvme/host 2024-03-21T09:58:57.841 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/nvme/host/nvme-core.ko 2024-03-21T09:58:57.841 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/nvme/host/nvme.ko 2024-03-21T09:58:57.842 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/parport 2024-03-21T09:58:57.842 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/parport/parport.ko 2024-03-21T09:58:57.842 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/parport/parport_cs.ko 2024-03-21T09:58:57.842 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/parport/parport_pc.ko 2024-03-21T09:58:57.842 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/parport/parport_serial.ko 2024-03-21T09:58:57.842 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/pci 2024-03-21T09:58:57.842 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/pci/hotplug 2024-03-21T09:58:57.842 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/pci/hotplug/acpiphp_ibm.ko 2024-03-21T09:58:57.842 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/pci/pcie 2024-03-21T09:58:57.842 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/pci/pcie/aer_inject.ko 2024-03-21T09:58:57.842 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/pci/xen-pcifront.ko 2024-03-21T09:58:57.842 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/pcmcia 2024-03-21T09:58:57.842 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/pcmcia/i82092.ko 2024-03-21T09:58:57.842 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/pcmcia/pd6729.ko 2024-03-21T09:58:57.843 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/pcmcia/yenta_socket.ko 2024-03-21T09:58:57.843 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/pinctrl 2024-03-21T09:58:57.843 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/pinctrl/intel 2024-03-21T09:58:57.843 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/pinctrl/intel/pinctrl-cherryview.ko 2024-03-21T09:58:57.843 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/platform 2024-03-21T09:58:57.843 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/platform/x86 2024-03-21T09:58:57.843 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/platform/x86/acer-wmi.ko 2024-03-21T09:58:57.843 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/platform/x86/acerhdf.ko 2024-03-21T09:58:57.843 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/platform/x86/apple-gmux.ko 2024-03-21T09:58:57.843 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/platform/x86/asus-laptop.ko 2024-03-21T09:58:57.843 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/platform/x86/asus-nb-wmi.ko 2024-03-21T09:58:57.843 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/platform/x86/asus-wmi.ko 2024-03-21T09:58:57.843 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/platform/x86/classmate-laptop.ko 2024-03-21T09:58:57.843 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/platform/x86/eeepc-laptop.ko 2024-03-21T09:58:57.843 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/platform/x86/eeepc-wmi.ko 2024-03-21T09:58:57.844 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/platform/x86/fujitsu-laptop.ko 2024-03-21T09:58:57.844 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/platform/x86/fujitsu-tablet.ko 2024-03-21T09:58:57.844 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/platform/x86/hdaps.ko 2024-03-21T09:58:57.844 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/platform/x86/intel 2024-03-21T09:58:57.844 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/platform/x86/intel/intel-rst.ko 2024-03-21T09:58:57.844 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/platform/x86/intel_ips.ko 2024-03-21T09:58:57.844 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/platform/x86/msi-wmi.ko 2024-03-21T09:58:57.844 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/platform/x86/mxm-wmi.ko 2024-03-21T09:58:57.844 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/platform/x86/panasonic-laptop.ko 2024-03-21T09:58:57.844 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/platform/x86/samsung-laptop.ko 2024-03-21T09:58:57.844 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/platform/x86/samsung-q10.ko 2024-03-21T09:58:57.844 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/platform/x86/topstar-laptop.ko 2024-03-21T09:58:57.844 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/platform/x86/toshiba_bluetooth.ko 2024-03-21T09:58:57.844 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/platform/x86/toshiba_haps.ko 2024-03-21T09:58:57.844 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/platform/x86/wmi-bmof.ko 2024-03-21T09:58:57.844 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/platform/x86/wmi.ko 2024-03-21T09:58:57.844 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/powercap 2024-03-21T09:58:57.845 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/powercap/intel_rapl_common.ko 2024-03-21T09:58:57.845 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/powercap/intel_rapl_msr.ko 2024-03-21T09:58:57.845 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/pps 2024-03-21T09:58:57.845 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/pps/clients 2024-03-21T09:58:57.845 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/pps/clients/pps-gpio.ko 2024-03-21T09:58:57.845 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/pps/clients/pps-ldisc.ko 2024-03-21T09:58:57.845 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/pps/clients/pps_parport.ko 2024-03-21T09:58:57.845 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/pps/pps_core.ko 2024-03-21T09:58:57.845 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ptp 2024-03-21T09:58:57.845 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ptp/ptp.ko 2024-03-21T09:58:57.845 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ptp/ptp_kvm.ko 2024-03-21T09:58:57.845 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/rtc 2024-03-21T09:58:57.845 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/rtc/rtc-bq32k.ko 2024-03-21T09:58:57.845 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/rtc/rtc-ds1286.ko 2024-03-21T09:58:57.845 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/rtc/rtc-ds1307.ko 2024-03-21T09:58:57.846 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/rtc/rtc-ds1374.ko 2024-03-21T09:58:57.846 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/rtc/rtc-ds1511.ko 2024-03-21T09:58:57.846 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/rtc/rtc-ds1553.ko 2024-03-21T09:58:57.846 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/rtc/rtc-ds1672.ko 2024-03-21T09:58:57.846 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/rtc/rtc-ds1742.ko 2024-03-21T09:58:57.846 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/rtc/rtc-ds2404.ko 2024-03-21T09:58:57.846 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/rtc/rtc-ds3232.ko 2024-03-21T09:58:57.846 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/rtc/rtc-em3027.ko 2024-03-21T09:58:57.846 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/rtc/rtc-fm3130.ko 2024-03-21T09:58:57.846 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/rtc/rtc-isl12022.ko 2024-03-21T09:58:57.846 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/rtc/rtc-isl1208.ko 2024-03-21T09:58:57.846 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/rtc/rtc-m41t80.ko 2024-03-21T09:58:57.846 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/rtc/rtc-m48t35.ko 2024-03-21T09:58:57.846 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/rtc/rtc-m48t59.ko 2024-03-21T09:58:57.847 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/rtc/rtc-max6900.ko 2024-03-21T09:58:57.847 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/rtc/rtc-msm6242.ko 2024-03-21T09:58:57.847 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/rtc/rtc-pcf2127.ko 2024-03-21T09:58:57.847 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/rtc/rtc-pcf85063.ko 2024-03-21T09:58:57.847 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/rtc/rtc-pcf8523.ko 2024-03-21T09:58:57.847 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/rtc/rtc-pcf8563.ko 2024-03-21T09:58:57.847 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/rtc/rtc-pcf8583.ko 2024-03-21T09:58:57.847 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/rtc/rtc-rp5c01.ko 2024-03-21T09:58:57.847 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/rtc/rtc-rs5c372.ko 2024-03-21T09:58:57.847 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/rtc/rtc-rv3029c2.ko 2024-03-21T09:58:57.847 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/rtc/rtc-rx8025.ko 2024-03-21T09:58:57.847 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/rtc/rtc-rx8581.ko 2024-03-21T09:58:57.847 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/rtc/rtc-stk17ta8.ko 2024-03-21T09:58:57.847 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/rtc/rtc-x1205.ko 2024-03-21T09:58:57.847 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi 2024-03-21T09:58:57.848 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/3w-9xxx.ko 2024-03-21T09:58:57.848 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/3w-sas.ko 2024-03-21T09:58:57.848 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/3w-xxxx.ko 2024-03-21T09:58:57.848 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/BusLogic.ko 2024-03-21T09:58:57.848 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/a100u2w.ko 2024-03-21T09:58:57.848 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/aacraid 2024-03-21T09:58:57.848 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/aacraid/aacraid.ko 2024-03-21T09:58:57.848 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/advansys.ko 2024-03-21T09:58:57.848 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/aic7xxx 2024-03-21T09:58:57.848 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/aic7xxx/aic79xx.ko 2024-03-21T09:58:57.848 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/aic7xxx/aic7xxx.ko 2024-03-21T09:58:57.848 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/am53c974.ko 2024-03-21T09:58:57.848 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/arcmsr 2024-03-21T09:58:57.848 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/arcmsr/arcmsr.ko 2024-03-21T09:58:57.849 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/atp870u.ko 2024-03-21T09:58:57.849 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/be2iscsi 2024-03-21T09:58:57.849 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/be2iscsi/be2iscsi.ko 2024-03-21T09:58:57.849 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/bfa 2024-03-21T09:58:57.849 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/bfa/bfa.ko 2024-03-21T09:58:57.849 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/bnx2fc 2024-03-21T09:58:57.849 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/bnx2fc/bnx2fc.ko 2024-03-21T09:58:57.849 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/bnx2i 2024-03-21T09:58:57.849 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/bnx2i/bnx2i.ko 2024-03-21T09:58:57.849 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/ch.ko 2024-03-21T09:58:57.849 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/csiostor 2024-03-21T09:58:57.849 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/csiostor/csiostor.ko 2024-03-21T09:58:57.849 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/cxgbi 2024-03-21T09:58:57.849 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/cxgbi/cxgb3i 2024-03-21T09:58:57.850 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/cxgbi/cxgb3i/cxgb3i.ko 2024-03-21T09:58:57.850 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/cxgbi/cxgb4i 2024-03-21T09:58:57.850 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/cxgbi/cxgb4i/cxgb4i.ko 2024-03-21T09:58:57.850 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/cxgbi/libcxgbi.ko 2024-03-21T09:58:57.850 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/dc395x.ko 2024-03-21T09:58:57.850 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/device_handler 2024-03-21T09:58:57.850 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/device_handler/scsi_dh_alua.ko 2024-03-21T09:58:57.850 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/device_handler/scsi_dh_emc.ko 2024-03-21T09:58:57.850 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/device_handler/scsi_dh_hp_sw.ko 2024-03-21T09:58:57.850 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/device_handler/scsi_dh_rdac.ko 2024-03-21T09:58:57.850 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/dmx3191d.ko 2024-03-21T09:58:57.850 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/esas2r 2024-03-21T09:58:57.850 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/esas2r/esas2r.ko 2024-03-21T09:58:57.850 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/esp_scsi.ko 2024-03-21T09:58:57.851 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/fcoe 2024-03-21T09:58:57.851 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/fcoe/fcoe.ko 2024-03-21T09:58:57.851 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/fcoe/libfcoe.ko 2024-03-21T09:58:57.851 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/fnic 2024-03-21T09:58:57.851 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/fnic/fnic.ko 2024-03-21T09:58:57.851 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/hpsa.ko 2024-03-21T09:58:57.851 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/hptiop.ko 2024-03-21T09:58:57.851 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/hv_storvsc.ko 2024-03-21T09:58:57.851 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/initio.ko 2024-03-21T09:58:57.851 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/ipr.ko 2024-03-21T09:58:57.851 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/ips.ko 2024-03-21T09:58:57.851 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/isci 2024-03-21T09:58:57.851 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/isci/isci.ko 2024-03-21T09:58:57.851 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/iscsi_boot_sysfs.ko 2024-03-21T09:58:57.851 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/iscsi_tcp.ko 2024-03-21T09:58:57.852 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/libfc 2024-03-21T09:58:57.852 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/libfc/libfc.ko 2024-03-21T09:58:57.852 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/libiscsi.ko 2024-03-21T09:58:57.852 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/libiscsi_tcp.ko 2024-03-21T09:58:57.852 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/libsas 2024-03-21T09:58:57.852 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/libsas/libsas.ko 2024-03-21T09:58:57.852 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/lpfc 2024-03-21T09:58:57.852 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/lpfc/lpfc.ko 2024-03-21T09:58:57.852 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/megaraid 2024-03-21T09:58:57.852 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/megaraid.ko 2024-03-21T09:58:57.852 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/megaraid/megaraid_mbox.ko 2024-03-21T09:58:57.852 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/megaraid/megaraid_mm.ko 2024-03-21T09:58:57.852 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/megaraid/megaraid_sas.ko 2024-03-21T09:58:57.852 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/mpt3sas 2024-03-21T09:58:57.852 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/mpt3sas/mpt3sas.ko 2024-03-21T09:58:57.853 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/mvsas 2024-03-21T09:58:57.853 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/mvsas/mvsas.ko 2024-03-21T09:58:57.853 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/mvumi.ko 2024-03-21T09:58:57.853 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/pm8001 2024-03-21T09:58:57.853 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/pm8001/pm80xx.ko 2024-03-21T09:58:57.853 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/pmcraid.ko 2024-03-21T09:58:57.853 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/qla1280.ko 2024-03-21T09:58:57.853 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/qla2xxx 2024-03-21T09:58:57.853 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/qla2xxx/qla2xxx.ko 2024-03-21T09:58:57.853 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/qla2xxx/tcm_qla2xxx.ko 2024-03-21T09:58:57.853 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/qla4xxx 2024-03-21T09:58:57.853 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/qla4xxx/qla4xxx.ko 2024-03-21T09:58:57.853 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/raid_class.ko 2024-03-21T09:58:57.853 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/scsi_debug.ko 2024-03-21T09:58:57.853 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/scsi_transport_fc.ko 2024-03-21T09:58:57.853 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/scsi_transport_iscsi.ko 2024-03-21T09:58:57.853 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/scsi_transport_sas.ko 2024-03-21T09:58:57.854 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/scsi_transport_spi.ko 2024-03-21T09:58:57.854 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/scsi_transport_srp.ko 2024-03-21T09:58:57.854 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/ses.ko 2024-03-21T09:58:57.854 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/st.ko 2024-03-21T09:58:57.854 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/stex.ko 2024-03-21T09:58:57.854 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/sym53c8xx_2 2024-03-21T09:58:57.854 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/sym53c8xx_2/sym53c8xx.ko 2024-03-21T09:58:57.854 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/virtio_scsi.ko 2024-03-21T09:58:57.854 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/vmw_pvscsi.ko 2024-03-21T09:58:57.854 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/wd719x.ko 2024-03-21T09:58:57.854 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ssb 2024-03-21T09:58:57.854 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ssb/ssb.ko 2024-03-21T09:58:57.854 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/target 2024-03-21T09:58:57.854 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/target/iscsi 2024-03-21T09:58:57.855 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/target/iscsi/iscsi_target_mod.ko 2024-03-21T09:58:57.855 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/target/loopback 2024-03-21T09:58:57.855 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/target/loopback/tcm_loop.ko 2024-03-21T09:58:57.855 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/target/target_core_file.ko 2024-03-21T09:58:57.855 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/target/target_core_iblock.ko 2024-03-21T09:58:57.855 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/target/target_core_mod.ko 2024-03-21T09:58:57.855 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/target/target_core_pscsi.ko 2024-03-21T09:58:57.855 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/target/target_core_user.ko 2024-03-21T09:58:57.855 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/target/tcm_fc 2024-03-21T09:58:57.855 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/target/tcm_fc/tcm_fc.ko 2024-03-21T09:58:57.855 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/thermal 2024-03-21T09:58:57.855 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/thermal/intel 2024-03-21T09:58:57.855 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/thermal/intel/int340x_thermal 2024-03-21T09:58:57.855 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/thermal/intel/int340x_thermal/acpi_thermal_rel.ko 2024-03-21T09:58:57.855 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/thermal/intel/int340x_thermal/int3400_thermal.ko 2024-03-21T09:58:57.855 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/thermal/intel/int340x_thermal/int3401_thermal.ko 2024-03-21T09:58:57.856 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/thermal/intel/int340x_thermal/int3402_thermal.ko 2024-03-21T09:58:57.856 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/thermal/intel/int340x_thermal/int3403_thermal.ko 2024-03-21T09:58:57.856 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/thermal/intel/int340x_thermal/int340x_thermal_zone.ko 2024-03-21T09:58:57.856 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device.ko 2024-03-21T09:58:57.856 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci.ko 2024-03-21T09:58:57.856 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci_legacy.ko 2024-03-21T09:58:57.856 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_mbox.ko 2024-03-21T09:58:57.856 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_power_floor.ko 2024-03-21T09:58:57.856 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rapl.ko 2024-03-21T09:58:57.856 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rfim.ko 2024-03-21T09:58:57.856 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_wt_hint.ko 2024-03-21T09:58:57.856 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_wt_req.ko 2024-03-21T09:58:57.856 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/thermal/intel/intel_soc_dts_iosf.ko 2024-03-21T09:58:57.856 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/thermal/intel/intel_soc_dts_thermal.ko 2024-03-21T09:58:57.856 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/thermal/intel/x86_pkg_temp_thermal.ko 2024-03-21T09:58:57.857 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/tty 2024-03-21T09:58:57.857 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/tty/ipwireless 2024-03-21T09:58:57.857 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/tty/ipwireless/ipwireless.ko 2024-03-21T09:58:57.857 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/tty/n_gsm.ko 2024-03-21T09:58:57.857 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/tty/n_hdlc.ko 2024-03-21T09:58:57.857 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/tty/nozomi.ko 2024-03-21T09:58:57.857 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/tty/serial 2024-03-21T09:58:57.857 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/tty/serial/8250 2024-03-21T09:58:57.857 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/tty/serial/8250/serial_cs.ko 2024-03-21T09:58:57.857 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/tty/serial/arc_uart.ko 2024-03-21T09:58:57.857 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/tty/serial/jsm 2024-03-21T09:58:57.857 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/tty/serial/jsm/jsm.ko 2024-03-21T09:58:57.857 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ufs 2024-03-21T09:58:57.857 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ufs/core 2024-03-21T09:58:57.858 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ufs/core/ufshcd-core.ko 2024-03-21T09:58:57.858 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ufs/host 2024-03-21T09:58:57.858 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ufs/host/ufshcd-pci.ko 2024-03-21T09:58:57.858 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/uio 2024-03-21T09:58:57.858 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/uio/uio.ko 2024-03-21T09:58:57.858 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/uio/uio_aec.ko 2024-03-21T09:58:57.858 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/uio/uio_cif.ko 2024-03-21T09:58:57.858 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/uio/uio_pci_generic.ko 2024-03-21T09:58:57.858 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/uio/uio_sercos3.ko 2024-03-21T09:58:57.858 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb 2024-03-21T09:58:57.858 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/class 2024-03-21T09:58:57.858 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/class/cdc-acm.ko 2024-03-21T09:58:57.858 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/class/cdc-wdm.ko 2024-03-21T09:58:57.858 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/class/usblp.ko 2024-03-21T09:58:57.858 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/class/usbtmc.ko 2024-03-21T09:58:57.859 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/host 2024-03-21T09:58:57.859 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/host/sl811-hcd.ko 2024-03-21T09:58:57.859 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/image 2024-03-21T09:58:57.859 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/image/mdc800.ko 2024-03-21T09:58:57.859 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/image/microtek.ko 2024-03-21T09:58:57.859 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/misc 2024-03-21T09:58:57.859 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/misc/adutux.ko 2024-03-21T09:58:57.859 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/misc/appledisplay.ko 2024-03-21T09:58:57.859 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/misc/emi26.ko 2024-03-21T09:58:57.859 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/misc/emi62.ko 2024-03-21T09:58:57.859 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/misc/ezusb.ko 2024-03-21T09:58:57.859 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/misc/idmouse.ko 2024-03-21T09:58:57.859 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/misc/iowarrior.ko 2024-03-21T09:58:57.859 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/misc/isight_firmware.ko 2024-03-21T09:58:57.860 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/misc/ldusb.ko 2024-03-21T09:58:57.860 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/misc/legousbtower.ko 2024-03-21T09:58:57.860 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/misc/sisusbvga 2024-03-21T09:58:57.860 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/misc/sisusbvga/sisusbvga.ko 2024-03-21T09:58:57.860 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/misc/trancevibrator.ko 2024-03-21T09:58:57.860 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/misc/usb3503.ko 2024-03-21T09:58:57.860 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/misc/usblcd.ko 2024-03-21T09:58:57.860 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/misc/usbsevseg.ko 2024-03-21T09:58:57.860 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/misc/uss720.ko 2024-03-21T09:58:57.860 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/misc/yurex.ko 2024-03-21T09:58:57.860 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/phy 2024-03-21T09:58:57.860 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/phy/phy-generic.ko 2024-03-21T09:58:57.860 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial 2024-03-21T09:58:57.860 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/aircable.ko 2024-03-21T09:58:57.860 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/ark3116.ko 2024-03-21T09:58:57.860 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/belkin_sa.ko 2024-03-21T09:58:57.861 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/ch341.ko 2024-03-21T09:58:57.861 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/cp210x.ko 2024-03-21T09:58:57.861 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/cyberjack.ko 2024-03-21T09:58:57.861 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/cypress_m8.ko 2024-03-21T09:58:57.861 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/digi_acceleport.ko 2024-03-21T09:58:57.861 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/empeg.ko 2024-03-21T09:58:57.861 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/ftdi_sio.ko 2024-03-21T09:58:57.861 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/garmin_gps.ko 2024-03-21T09:58:57.861 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/io_edgeport.ko 2024-03-21T09:58:57.861 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/io_ti.ko 2024-03-21T09:58:57.861 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/ipaq.ko 2024-03-21T09:58:57.861 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/ipw.ko 2024-03-21T09:58:57.861 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/ir-usb.ko 2024-03-21T09:58:57.861 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/iuu_phoenix.ko 2024-03-21T09:58:57.861 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/keyspan.ko 2024-03-21T09:58:57.862 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/keyspan_pda.ko 2024-03-21T09:58:57.862 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/kl5kusb105.ko 2024-03-21T09:58:57.862 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/kobil_sct.ko 2024-03-21T09:58:57.862 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/mct_u232.ko 2024-03-21T09:58:57.862 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/mos7720.ko 2024-03-21T09:58:57.862 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/mos7840.ko 2024-03-21T09:58:57.862 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/navman.ko 2024-03-21T09:58:57.862 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/omninet.ko 2024-03-21T09:58:57.862 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/opticon.ko 2024-03-21T09:58:57.862 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/option.ko 2024-03-21T09:58:57.862 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/oti6858.ko 2024-03-21T09:58:57.862 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/pl2303.ko 2024-03-21T09:58:57.862 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/qcaux.ko 2024-03-21T09:58:57.862 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/qcserial.ko 2024-03-21T09:58:57.863 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/quatech2.ko 2024-03-21T09:58:57.863 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/safe_serial.ko 2024-03-21T09:58:57.863 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/sierra.ko 2024-03-21T09:58:57.863 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/spcp8x5.ko 2024-03-21T09:58:57.863 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/ssu100.ko 2024-03-21T09:58:57.863 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/symbolserial.ko 2024-03-21T09:58:57.863 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/ti_usb_3410_5052.ko 2024-03-21T09:58:57.863 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/usb-serial-simple.ko 2024-03-21T09:58:57.863 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/usb_debug.ko 2024-03-21T09:58:57.863 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/usb_wwan.ko 2024-03-21T09:58:57.863 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/visor.ko 2024-03-21T09:58:57.863 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/whiteheat.ko 2024-03-21T09:58:57.863 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/xsens_mt.ko 2024-03-21T09:58:57.863 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/storage 2024-03-21T09:58:57.863 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/storage/uas.ko 2024-03-21T09:58:57.864 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/storage/ums-alauda.ko 2024-03-21T09:58:57.864 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/storage/ums-cypress.ko 2024-03-21T09:58:57.864 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/storage/ums-datafab.ko 2024-03-21T09:58:57.864 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/storage/ums-eneub6250.ko 2024-03-21T09:58:57.864 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/storage/ums-freecom.ko 2024-03-21T09:58:57.864 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/storage/ums-isd200.ko 2024-03-21T09:58:57.864 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/storage/ums-jumpshot.ko 2024-03-21T09:58:57.864 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/storage/ums-karma.ko 2024-03-21T09:58:57.864 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/storage/ums-onetouch.ko 2024-03-21T09:58:57.864 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/storage/ums-realtek.ko 2024-03-21T09:58:57.864 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/storage/ums-sddr09.ko 2024-03-21T09:58:57.864 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/storage/ums-sddr55.ko 2024-03-21T09:58:57.864 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/storage/ums-usbat.ko 2024-03-21T09:58:57.864 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/storage/usb-storage.ko 2024-03-21T09:58:57.865 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/usbip 2024-03-21T09:58:57.865 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/usbip/usbip-core.ko 2024-03-21T09:58:57.865 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/usbip/usbip-host.ko 2024-03-21T09:58:57.865 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/usbip/vhci-hcd.ko 2024-03-21T09:58:57.865 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/vfio 2024-03-21T09:58:57.865 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/vfio/pci 2024-03-21T09:58:57.865 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/vfio/pci/vfio-pci-core.ko 2024-03-21T09:58:57.865 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/vfio/pci/vfio-pci.ko 2024-03-21T09:58:57.865 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/vfio/vfio.ko 2024-03-21T09:58:57.865 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/vfio/vfio_iommu_type1.ko 2024-03-21T09:58:57.865 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/vhost 2024-03-21T09:58:57.865 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/vhost/vhost.ko 2024-03-21T09:58:57.865 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/vhost/vhost_iotlb.ko 2024-03-21T09:58:57.865 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/vhost/vhost_net.ko 2024-03-21T09:58:57.865 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/vhost/vhost_scsi.ko 2024-03-21T09:58:57.866 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/video 2024-03-21T09:58:57.866 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/video/backlight 2024-03-21T09:58:57.866 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/video/backlight/apple_bl.ko 2024-03-21T09:58:57.866 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/video/backlight/lcd.ko 2024-03-21T09:58:57.866 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/video/backlight/platform_lcd.ko 2024-03-21T09:58:57.866 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/virtio 2024-03-21T09:58:57.866 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/virtio/virtio.ko 2024-03-21T09:58:57.866 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/virtio/virtio_balloon.ko 2024-03-21T09:58:57.866 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/virtio/virtio_mmio.ko 2024-03-21T09:58:57.866 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/virtio/virtio_pci.ko 2024-03-21T09:58:57.866 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/virtio/virtio_pci_legacy_dev.ko 2024-03-21T09:58:57.866 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/virtio/virtio_pci_modern_dev.ko 2024-03-21T09:58:57.866 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/virtio/virtio_ring.ko 2024-03-21T09:58:57.866 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/w1 2024-03-21T09:58:57.867 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/w1/masters 2024-03-21T09:58:57.867 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/w1/masters/ds2482.ko 2024-03-21T09:58:57.867 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/w1/masters/ds2490.ko 2024-03-21T09:58:57.867 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/w1/slaves 2024-03-21T09:58:57.867 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/w1/slaves/w1_ds2406.ko 2024-03-21T09:58:57.867 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/w1/slaves/w1_ds2408.ko 2024-03-21T09:58:57.867 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/w1/slaves/w1_ds2413.ko 2024-03-21T09:58:57.867 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/w1/slaves/w1_ds2423.ko 2024-03-21T09:58:57.867 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/w1/slaves/w1_ds2431.ko 2024-03-21T09:58:57.867 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/w1/slaves/w1_ds2433.ko 2024-03-21T09:58:57.867 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/w1/slaves/w1_ds2780.ko 2024-03-21T09:58:57.867 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/w1/slaves/w1_ds2781.ko 2024-03-21T09:58:57.867 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/w1/slaves/w1_ds28e04.ko 2024-03-21T09:58:57.867 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/w1/slaves/w1_smem.ko 2024-03-21T09:58:57.867 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/w1/slaves/w1_therm.ko 2024-03-21T09:58:57.868 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/w1/wire.ko 2024-03-21T09:58:57.868 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/watchdog 2024-03-21T09:58:57.868 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/watchdog/alim1535_wdt.ko 2024-03-21T09:58:57.868 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/watchdog/alim7101_wdt.ko 2024-03-21T09:58:57.868 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/watchdog/f71808e_wdt.ko 2024-03-21T09:58:57.868 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/watchdog/hpwdt.ko 2024-03-21T09:58:57.868 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/watchdog/i6300esb.ko 2024-03-21T09:58:57.868 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/watchdog/iTCO_vendor_support.ko 2024-03-21T09:58:57.868 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/watchdog/iTCO_wdt.ko 2024-03-21T09:58:57.868 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/watchdog/ib700wdt.ko 2024-03-21T09:58:57.868 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/watchdog/ibmasr.ko 2024-03-21T09:58:57.868 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/watchdog/ie6xx_wdt.ko 2024-03-21T09:58:57.868 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/watchdog/it8712f_wdt.ko 2024-03-21T09:58:57.868 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/watchdog/it87_wdt.ko 2024-03-21T09:58:57.869 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/watchdog/machzwd.ko 2024-03-21T09:58:57.869 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/watchdog/nv_tco.ko 2024-03-21T09:58:57.869 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/watchdog/pcwd_pci.ko 2024-03-21T09:58:57.869 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/watchdog/pcwd_usb.ko 2024-03-21T09:58:57.869 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/watchdog/sbc_fitpc2_wdt.ko 2024-03-21T09:58:57.869 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/watchdog/sch311x_wdt.ko 2024-03-21T09:58:57.869 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/watchdog/softdog.ko 2024-03-21T09:58:57.869 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/watchdog/sp5100_tco.ko 2024-03-21T09:58:57.869 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/watchdog/via_wdt.ko 2024-03-21T09:58:57.869 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/watchdog/w83627hf_wdt.ko 2024-03-21T09:58:57.869 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/watchdog/w83877f_wdt.ko 2024-03-21T09:58:57.869 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/watchdog/w83977f_wdt.ko 2024-03-21T09:58:57.869 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/watchdog/wdt_pci.ko 2024-03-21T09:58:57.869 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/watchdog/xen_wdt.ko 2024-03-21T09:58:57.869 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/xen 2024-03-21T09:58:57.870 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/xen/xen-acpi-processor.ko 2024-03-21T09:58:57.870 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/xen/xen-evtchn.ko 2024-03-21T09:58:57.870 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/xen/xen-gntalloc.ko 2024-03-21T09:58:57.870 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/xen/xen-gntdev.ko 2024-03-21T09:58:57.870 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/xen/xen-pciback 2024-03-21T09:58:57.870 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/xen/xen-pciback/xen-pciback.ko 2024-03-21T09:58:57.870 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/xen/xen-privcmd.ko 2024-03-21T09:58:57.870 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/xen/xenfs 2024-03-21T09:58:57.870 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/xen/xenfs/xenfs.ko 2024-03-21T09:58:57.870 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs 2024-03-21T09:58:57.870 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/9p 2024-03-21T09:58:57.870 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/9p/9p.ko 2024-03-21T09:58:57.870 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/affs 2024-03-21T09:58:57.871 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/affs/affs.ko 2024-03-21T09:58:57.871 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/befs 2024-03-21T09:58:57.871 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/befs/befs.ko 2024-03-21T09:58:57.871 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/binfmt_misc.ko 2024-03-21T09:58:57.871 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/btrfs 2024-03-21T09:58:57.871 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/btrfs/btrfs.ko 2024-03-21T09:58:57.871 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/cachefiles 2024-03-21T09:58:57.871 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/cachefiles/cachefiles.ko 2024-03-21T09:58:57.871 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/ceph 2024-03-21T09:58:57.871 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/ceph/ceph.ko 2024-03-21T09:58:57.871 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/coda 2024-03-21T09:58:57.871 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/coda/coda.ko 2024-03-21T09:58:57.871 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/cramfs 2024-03-21T09:58:57.871 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/cramfs/cramfs.ko 2024-03-21T09:58:57.871 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/dlm 2024-03-21T09:58:57.872 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/dlm/dlm.ko 2024-03-21T09:58:57.872 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/ecryptfs 2024-03-21T09:58:57.872 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/ecryptfs/ecryptfs.ko 2024-03-21T09:58:57.872 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/f2fs 2024-03-21T09:58:57.872 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/f2fs/f2fs.ko 2024-03-21T09:58:57.872 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/fat 2024-03-21T09:58:57.872 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/fat/fat.ko 2024-03-21T09:58:57.872 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/fat/msdos.ko 2024-03-21T09:58:57.872 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/fat/vfat.ko 2024-03-21T09:58:57.872 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/fuse 2024-03-21T09:58:57.872 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/fuse/cuse.ko 2024-03-21T09:58:57.872 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/fuse/fuse.ko 2024-03-21T09:58:57.872 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/gfs2 2024-03-21T09:58:57.872 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/gfs2/gfs2.ko 2024-03-21T09:58:57.873 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/hfs 2024-03-21T09:58:57.873 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/hfs/hfs.ko 2024-03-21T09:58:57.873 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/hfsplus 2024-03-21T09:58:57.873 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/hfsplus/hfsplus.ko 2024-03-21T09:58:57.873 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/isofs 2024-03-21T09:58:57.873 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/isofs/isofs.ko 2024-03-21T09:58:57.873 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/jfs 2024-03-21T09:58:57.873 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/jfs/jfs.ko 2024-03-21T09:58:57.873 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/lockd 2024-03-21T09:58:57.873 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/lockd/lockd.ko 2024-03-21T09:58:57.873 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/minix 2024-03-21T09:58:57.873 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/minix/minix.ko 2024-03-21T09:58:57.873 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/netfs 2024-03-21T09:58:57.873 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/netfs/netfs.ko 2024-03-21T09:58:57.873 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nfs 2024-03-21T09:58:57.874 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nfs/blocklayout 2024-03-21T09:58:57.874 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nfs/blocklayout/blocklayoutdriver.ko 2024-03-21T09:58:57.874 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nfs/filelayout 2024-03-21T09:58:57.874 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nfs/filelayout/nfs_layout_nfsv41_files.ko 2024-03-21T09:58:57.874 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nfs/flexfilelayout 2024-03-21T09:58:57.874 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nfs/flexfilelayout/nfs_layout_flexfiles.ko 2024-03-21T09:58:57.874 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nfs/nfs.ko 2024-03-21T09:58:57.874 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nfs/nfsv3.ko 2024-03-21T09:58:57.874 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nfs/nfsv4.ko 2024-03-21T09:58:57.874 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nfs_common 2024-03-21T09:58:57.874 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nfs_common/grace.ko 2024-03-21T09:58:57.874 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nfs_common/nfs_acl.ko 2024-03-21T09:58:57.874 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nfsd 2024-03-21T09:58:57.874 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nfsd/nfsd.ko 2024-03-21T09:58:57.875 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nilfs2 2024-03-21T09:58:57.875 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nilfs2/nilfs2.ko 2024-03-21T09:58:57.875 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls 2024-03-21T09:58:57.875 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/mac-celtic.ko 2024-03-21T09:58:57.875 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/mac-centeuro.ko 2024-03-21T09:58:57.875 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/mac-croatian.ko 2024-03-21T09:58:57.875 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/mac-cyrillic.ko 2024-03-21T09:58:57.875 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/mac-gaelic.ko 2024-03-21T09:58:57.875 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/mac-greek.ko 2024-03-21T09:58:57.875 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/mac-iceland.ko 2024-03-21T09:58:57.875 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/mac-inuit.ko 2024-03-21T09:58:57.875 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/mac-roman.ko 2024-03-21T09:58:57.875 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/mac-romanian.ko 2024-03-21T09:58:57.875 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/mac-turkish.ko 2024-03-21T09:58:57.876 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_cp1250.ko 2024-03-21T09:58:57.876 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_cp1251.ko 2024-03-21T09:58:57.876 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_cp1255.ko 2024-03-21T09:58:57.876 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_cp737.ko 2024-03-21T09:58:57.876 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_cp775.ko 2024-03-21T09:58:57.876 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_cp850.ko 2024-03-21T09:58:57.876 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_cp852.ko 2024-03-21T09:58:57.876 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_cp855.ko 2024-03-21T09:58:57.876 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_cp857.ko 2024-03-21T09:58:57.876 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_cp860.ko 2024-03-21T09:58:57.876 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_cp861.ko 2024-03-21T09:58:57.876 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_cp862.ko 2024-03-21T09:58:57.876 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_cp863.ko 2024-03-21T09:58:57.876 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_cp864.ko 2024-03-21T09:58:57.876 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_cp865.ko 2024-03-21T09:58:57.877 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_cp866.ko 2024-03-21T09:58:57.877 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_cp869.ko 2024-03-21T09:58:57.877 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_cp874.ko 2024-03-21T09:58:57.877 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_cp932.ko 2024-03-21T09:58:57.877 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_cp936.ko 2024-03-21T09:58:57.877 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_cp949.ko 2024-03-21T09:58:57.877 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_cp950.ko 2024-03-21T09:58:57.877 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_euc-jp.ko 2024-03-21T09:58:57.877 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_iso8859-1.ko 2024-03-21T09:58:57.877 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_iso8859-13.ko 2024-03-21T09:58:57.877 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_iso8859-14.ko 2024-03-21T09:58:57.877 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_iso8859-15.ko 2024-03-21T09:58:57.877 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_iso8859-2.ko 2024-03-21T09:58:57.877 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_iso8859-3.ko 2024-03-21T09:58:57.877 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_iso8859-4.ko 2024-03-21T09:58:57.878 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_iso8859-5.ko 2024-03-21T09:58:57.878 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_iso8859-6.ko 2024-03-21T09:58:57.878 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_iso8859-7.ko 2024-03-21T09:58:57.878 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_iso8859-9.ko 2024-03-21T09:58:57.878 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_koi8-r.ko 2024-03-21T09:58:57.878 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_koi8-ru.ko 2024-03-21T09:58:57.878 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_koi8-u.ko 2024-03-21T09:58:57.878 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_ucs2_utils.ko 2024-03-21T09:58:57.878 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_utf8.ko 2024-03-21T09:58:57.878 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/ocfs2 2024-03-21T09:58:57.878 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/ocfs2/cluster 2024-03-21T09:58:57.878 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/ocfs2/cluster/ocfs2_nodemanager.ko 2024-03-21T09:58:57.878 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/ocfs2/dlm 2024-03-21T09:58:57.878 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/ocfs2/dlm/ocfs2_dlm.ko 2024-03-21T09:58:57.879 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/ocfs2/dlmfs 2024-03-21T09:58:57.879 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/ocfs2/dlmfs/ocfs2_dlmfs.ko 2024-03-21T09:58:57.879 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/ocfs2/ocfs2.ko 2024-03-21T09:58:57.879 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/ocfs2/ocfs2_stack_o2cb.ko 2024-03-21T09:58:57.879 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/ocfs2/ocfs2_stack_user.ko 2024-03-21T09:58:57.879 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/ocfs2/ocfs2_stackglue.ko 2024-03-21T09:58:57.879 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/overlayfs 2024-03-21T09:58:57.879 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/overlayfs/overlay.ko 2024-03-21T09:58:57.879 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/pstore 2024-03-21T09:58:57.879 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/pstore/ramoops.ko 2024-03-21T09:58:57.879 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/reiserfs 2024-03-21T09:58:57.879 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/reiserfs/reiserfs.ko 2024-03-21T09:58:57.879 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/romfs 2024-03-21T09:58:57.879 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/romfs/romfs.ko 2024-03-21T09:58:57.880 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/smb 2024-03-21T09:58:57.880 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/smb/client 2024-03-21T09:58:57.880 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/smb/client/cifs.ko 2024-03-21T09:58:57.880 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/smb/common 2024-03-21T09:58:57.880 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/smb/common/cifs_arc4.ko 2024-03-21T09:58:57.880 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/smb/common/cifs_md4.ko 2024-03-21T09:58:57.880 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/squashfs 2024-03-21T09:58:57.880 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/squashfs/squashfs.ko 2024-03-21T09:58:57.880 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/sysv 2024-03-21T09:58:57.880 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/sysv/sysv.ko 2024-03-21T09:58:57.880 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/udf 2024-03-21T09:58:57.880 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/udf/udf.ko 2024-03-21T09:58:57.880 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/ufs 2024-03-21T09:58:57.880 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/ufs/ufs.ko 2024-03-21T09:58:57.881 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/xfs 2024-03-21T09:58:57.881 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/xfs/xfs.ko 2024-03-21T09:58:57.881 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/kernel 2024-03-21T09:58:57.881 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/kernel/rcu 2024-03-21T09:58:57.881 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/kernel/rcu/rcutorture.ko 2024-03-21T09:58:57.881 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/kernel/torture.ko 2024-03-21T09:58:57.881 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/kernel/trace 2024-03-21T09:58:57.881 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/kernel/trace/ring_buffer_benchmark.ko 2024-03-21T09:58:57.881 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/lib 2024-03-21T09:58:57.881 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/lib/asn1_encoder.ko 2024-03-21T09:58:57.881 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/lib/crc-itu-t.ko 2024-03-21T09:58:57.881 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/lib/crc8.ko 2024-03-21T09:58:57.881 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/lib/crypto 2024-03-21T09:58:57.881 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/lib/crypto/libarc4.ko 2024-03-21T09:58:57.882 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/lib/crypto/libdes.ko 2024-03-21T09:58:57.882 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/lib/libcrc32c.ko 2024-03-21T09:58:57.882 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/lib/lru_cache.ko 2024-03-21T09:58:57.882 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/lib/lz4 2024-03-21T09:58:57.882 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/lib/lz4/lz4_compress.ko 2024-03-21T09:58:57.882 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/lib/lz4/lz4hc_compress.ko 2024-03-21T09:58:57.882 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/lib/math 2024-03-21T09:58:57.882 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/lib/math/cordic.ko 2024-03-21T09:58:57.882 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/lib/raid6 2024-03-21T09:58:57.882 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/lib/raid6/raid6_pq.ko 2024-03-21T09:58:57.882 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/lib/reed_solomon 2024-03-21T09:58:57.882 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/lib/reed_solomon/reed_solomon.ko 2024-03-21T09:58:57.882 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/lib/ts_bm.ko 2024-03-21T09:58:57.882 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/lib/ts_fsm.ko 2024-03-21T09:58:57.883 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/lib/ts_kmp.ko 2024-03-21T09:58:57.883 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/lib/zstd 2024-03-21T09:58:57.883 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/lib/zstd/zstd_compress.ko 2024-03-21T09:58:57.883 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/mm 2024-03-21T09:58:57.883 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/mm/hwpoison-inject.ko 2024-03-21T09:58:57.883 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net 2024-03-21T09:58:57.883 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/6lowpan 2024-03-21T09:58:57.883 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/6lowpan/6lowpan.ko 2024-03-21T09:58:57.883 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/6lowpan/nhc_dest.ko 2024-03-21T09:58:57.883 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/6lowpan/nhc_fragment.ko 2024-03-21T09:58:57.883 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/6lowpan/nhc_hop.ko 2024-03-21T09:58:57.883 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/6lowpan/nhc_ipv6.ko 2024-03-21T09:58:57.883 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/6lowpan/nhc_mobility.ko 2024-03-21T09:58:57.883 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/6lowpan/nhc_routing.ko 2024-03-21T09:58:57.883 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/6lowpan/nhc_udp.ko 2024-03-21T09:58:57.883 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/802 2024-03-21T09:58:57.884 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/802/garp.ko 2024-03-21T09:58:57.884 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/802/mrp.ko 2024-03-21T09:58:57.884 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/802/p8022.ko 2024-03-21T09:58:57.884 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/802/psnap.ko 2024-03-21T09:58:57.884 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/802/stp.ko 2024-03-21T09:58:57.884 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/8021q 2024-03-21T09:58:57.884 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/8021q/8021q.ko 2024-03-21T09:58:57.884 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/9p 2024-03-21T09:58:57.884 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/9p/9pnet.ko 2024-03-21T09:58:57.884 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/9p/9pnet_fd.ko 2024-03-21T09:58:57.884 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/9p/9pnet_rdma.ko 2024-03-21T09:58:57.884 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/9p/9pnet_virtio.ko 2024-03-21T09:58:57.884 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/batman-adv 2024-03-21T09:58:57.884 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/batman-adv/batman-adv.ko 2024-03-21T09:58:57.884 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/bridge 2024-03-21T09:58:57.885 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/bridge/br_netfilter.ko 2024-03-21T09:58:57.885 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/bridge/bridge.ko 2024-03-21T09:58:57.885 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/bridge/netfilter 2024-03-21T09:58:57.885 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/bridge/netfilter/ebt_802_3.ko 2024-03-21T09:58:57.885 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/bridge/netfilter/ebt_among.ko 2024-03-21T09:58:57.885 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/bridge/netfilter/ebt_arp.ko 2024-03-21T09:58:57.885 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/bridge/netfilter/ebt_arpreply.ko 2024-03-21T09:58:57.885 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/bridge/netfilter/ebt_dnat.ko 2024-03-21T09:58:57.885 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/bridge/netfilter/ebt_ip.ko 2024-03-21T09:58:57.885 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/bridge/netfilter/ebt_ip6.ko 2024-03-21T09:58:57.885 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/bridge/netfilter/ebt_limit.ko 2024-03-21T09:58:57.885 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/bridge/netfilter/ebt_log.ko 2024-03-21T09:58:57.885 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/bridge/netfilter/ebt_mark.ko 2024-03-21T09:58:57.885 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/bridge/netfilter/ebt_mark_m.ko 2024-03-21T09:58:57.885 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/bridge/netfilter/ebt_nflog.ko 2024-03-21T09:58:57.885 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/bridge/netfilter/ebt_pkttype.ko 2024-03-21T09:58:57.886 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/bridge/netfilter/ebt_redirect.ko 2024-03-21T09:58:57.886 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/bridge/netfilter/ebt_snat.ko 2024-03-21T09:58:57.886 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/bridge/netfilter/ebt_stp.ko 2024-03-21T09:58:57.886 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/bridge/netfilter/ebt_vlan.ko 2024-03-21T09:58:57.886 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/bridge/netfilter/ebtable_broute.ko 2024-03-21T09:58:57.886 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/bridge/netfilter/ebtable_filter.ko 2024-03-21T09:58:57.886 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/bridge/netfilter/ebtable_nat.ko 2024-03-21T09:58:57.886 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/bridge/netfilter/ebtables.ko 2024-03-21T09:58:57.886 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/bridge/netfilter/nf_conntrack_bridge.ko 2024-03-21T09:58:57.886 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/bridge/netfilter/nft_meta_bridge.ko 2024-03-21T09:58:57.886 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/bridge/netfilter/nft_reject_bridge.ko 2024-03-21T09:58:57.886 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ceph 2024-03-21T09:58:57.886 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ceph/libceph.ko 2024-03-21T09:58:57.887 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/core 2024-03-21T09:58:57.887 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/core/failover.ko 2024-03-21T09:58:57.887 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/core/pktgen.ko 2024-03-21T09:58:57.887 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/dccp 2024-03-21T09:58:57.887 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/dccp/dccp.ko 2024-03-21T09:58:57.887 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/dccp/dccp_diag.ko 2024-03-21T09:58:57.887 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/dccp/dccp_ipv4.ko 2024-03-21T09:58:57.887 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/dccp/dccp_ipv6.ko 2024-03-21T09:58:57.887 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/dns_resolver 2024-03-21T09:58:57.887 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/dns_resolver/dns_resolver.ko 2024-03-21T09:58:57.887 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/dsa 2024-03-21T09:58:57.887 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/dsa/dsa_core.ko 2024-03-21T09:58:57.887 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/dsa/tag_brcm.ko 2024-03-21T09:58:57.887 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/dsa/tag_dsa.ko 2024-03-21T09:58:57.888 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/dsa/tag_none.ko 2024-03-21T09:58:57.888 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/dsa/tag_trailer.ko 2024-03-21T09:58:57.888 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ieee802154 2024-03-21T09:58:57.888 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ieee802154/6lowpan 2024-03-21T09:58:57.888 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ieee802154/6lowpan/ieee802154_6lowpan.ko 2024-03-21T09:58:57.888 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ieee802154/ieee802154.ko 2024-03-21T09:58:57.888 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ieee802154/ieee802154_socket.ko 2024-03-21T09:58:57.888 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4 2024-03-21T09:58:57.888 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/ah4.ko 2024-03-21T09:58:57.888 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/esp4.ko 2024-03-21T09:58:57.888 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/fou.ko 2024-03-21T09:58:57.888 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/gre.ko 2024-03-21T09:58:57.888 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/inet_diag.ko 2024-03-21T09:58:57.888 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/ip_gre.ko 2024-03-21T09:58:57.889 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/ip_tunnel.ko 2024-03-21T09:58:57.889 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/ip_vti.ko 2024-03-21T09:58:57.889 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/ipcomp.ko 2024-03-21T09:58:57.889 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/ipip.ko 2024-03-21T09:58:57.889 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/netfilter 2024-03-21T09:58:57.889 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/netfilter/arp_tables.ko 2024-03-21T09:58:57.889 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/netfilter/arpt_mangle.ko 2024-03-21T09:58:57.889 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/netfilter/arptable_filter.ko 2024-03-21T09:58:57.889 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/netfilter/ipt_ECN.ko 2024-03-21T09:58:57.889 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/netfilter/ipt_SYNPROXY.ko 2024-03-21T09:58:57.889 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/netfilter/ipt_ah.ko 2024-03-21T09:58:57.889 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/netfilter/ipt_rpfilter.ko 2024-03-21T09:58:57.889 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/netfilter/iptable_mangle.ko 2024-03-21T09:58:57.889 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/netfilter/iptable_nat.ko 2024-03-21T09:58:57.889 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/netfilter/iptable_raw.ko 2024-03-21T09:58:57.890 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/netfilter/iptable_security.ko 2024-03-21T09:58:57.890 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/netfilter/nf_defrag_ipv4.ko 2024-03-21T09:58:57.890 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/netfilter/nf_dup_ipv4.ko 2024-03-21T09:58:57.890 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/netfilter/nf_nat_h323.ko 2024-03-21T09:58:57.890 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/netfilter/nf_nat_pptp.ko 2024-03-21T09:58:57.890 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/netfilter/nf_nat_snmp_basic.ko 2024-03-21T09:58:57.890 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/netfilter/nf_socket_ipv4.ko 2024-03-21T09:58:57.890 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/netfilter/nf_tproxy_ipv4.ko 2024-03-21T09:58:57.890 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/netfilter/nft_dup_ipv4.ko 2024-03-21T09:58:57.890 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/netfilter/nft_fib_ipv4.ko 2024-03-21T09:58:57.890 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/netfilter/nft_reject_ipv4.ko 2024-03-21T09:58:57.890 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/tcp_bic.ko 2024-03-21T09:58:57.890 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/tcp_dctcp.ko 2024-03-21T09:58:57.891 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/tcp_diag.ko 2024-03-21T09:58:57.891 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/tcp_highspeed.ko 2024-03-21T09:58:57.891 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/tcp_htcp.ko 2024-03-21T09:58:57.891 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/tcp_hybla.ko 2024-03-21T09:58:57.891 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/tcp_illinois.ko 2024-03-21T09:58:57.891 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/tcp_lp.ko 2024-03-21T09:58:57.891 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/tcp_scalable.ko 2024-03-21T09:58:57.891 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/tcp_vegas.ko 2024-03-21T09:58:57.891 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/tcp_veno.ko 2024-03-21T09:58:57.891 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/tcp_westwood.ko 2024-03-21T09:58:57.891 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/tcp_yeah.ko 2024-03-21T09:58:57.891 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/tunnel4.ko 2024-03-21T09:58:57.891 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/udp_diag.ko 2024-03-21T09:58:57.891 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/udp_tunnel.ko 2024-03-21T09:58:57.891 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/xfrm4_tunnel.ko 2024-03-21T09:58:57.892 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv6 2024-03-21T09:58:57.892 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv6/ah6.ko 2024-03-21T09:58:57.892 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv6/esp6.ko 2024-03-21T09:58:57.892 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv6/fou6.ko 2024-03-21T09:58:57.892 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv6/ip6_tunnel.ko 2024-03-21T09:58:57.892 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv6/ip6_udp_tunnel.ko 2024-03-21T09:58:57.892 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv6/ip6_vti.ko 2024-03-21T09:58:57.892 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv6/ipcomp6.ko 2024-03-21T09:58:57.892 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv6/netfilter 2024-03-21T09:58:57.892 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv6/netfilter/ip6_tables.ko 2024-03-21T09:58:57.892 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv6/netfilter/ip6t_REJECT.ko 2024-03-21T09:58:57.892 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv6/netfilter/ip6t_SYNPROXY.ko 2024-03-21T09:58:57.892 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv6/netfilter/ip6t_ah.ko 2024-03-21T09:58:57.892 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv6/netfilter/ip6t_eui64.ko 2024-03-21T09:58:57.893 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv6/netfilter/ip6t_frag.ko 2024-03-21T09:58:57.893 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv6/netfilter/ip6t_hbh.ko 2024-03-21T09:58:57.893 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv6/netfilter/ip6t_ipv6header.ko 2024-03-21T09:58:57.893 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv6/netfilter/ip6t_mh.ko 2024-03-21T09:58:57.893 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv6/netfilter/ip6t_rpfilter.ko 2024-03-21T09:58:57.893 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv6/netfilter/ip6t_rt.ko 2024-03-21T09:58:57.893 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv6/netfilter/ip6table_filter.ko 2024-03-21T09:58:57.893 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv6/netfilter/ip6table_mangle.ko 2024-03-21T09:58:57.893 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv6/netfilter/ip6table_nat.ko 2024-03-21T09:58:57.893 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv6/netfilter/ip6table_raw.ko 2024-03-21T09:58:57.893 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv6/netfilter/ip6table_security.ko 2024-03-21T09:58:57.893 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv6/netfilter/nf_defrag_ipv6.ko 2024-03-21T09:58:57.893 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv6/netfilter/nf_dup_ipv6.ko 2024-03-21T09:58:57.894 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv6/netfilter/nf_reject_ipv6.ko 2024-03-21T09:58:57.894 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv6/netfilter/nf_socket_ipv6.ko 2024-03-21T09:58:57.894 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv6/netfilter/nf_tproxy_ipv6.ko 2024-03-21T09:58:57.894 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv6/netfilter/nft_dup_ipv6.ko 2024-03-21T09:58:57.894 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv6/netfilter/nft_fib_ipv6.ko 2024-03-21T09:58:57.894 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv6/netfilter/nft_reject_ipv6.ko 2024-03-21T09:58:57.894 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv6/sit.ko 2024-03-21T09:58:57.894 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv6/tunnel6.ko 2024-03-21T09:58:57.894 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv6/xfrm6_tunnel.ko 2024-03-21T09:58:57.894 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/key 2024-03-21T09:58:57.894 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/key/af_key.ko 2024-03-21T09:58:57.894 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/l2tp 2024-03-21T09:58:57.894 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/l2tp/l2tp_core.ko 2024-03-21T09:58:57.894 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/l2tp/l2tp_debugfs.ko 2024-03-21T09:58:57.894 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/l2tp/l2tp_eth.ko 2024-03-21T09:58:57.895 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/l2tp/l2tp_ip.ko 2024-03-21T09:58:57.895 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/l2tp/l2tp_ip6.ko 2024-03-21T09:58:57.895 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/l2tp/l2tp_netlink.ko 2024-03-21T09:58:57.895 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/l2tp/l2tp_ppp.ko 2024-03-21T09:58:57.895 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/llc 2024-03-21T09:58:57.895 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/llc/llc.ko 2024-03-21T09:58:57.895 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/mac802154 2024-03-21T09:58:57.895 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/mac802154/mac802154.ko 2024-03-21T09:58:57.895 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/mpls 2024-03-21T09:58:57.895 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/mpls/mpls_gso.ko 2024-03-21T09:58:57.895 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter 2024-03-21T09:58:57.895 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/ipset 2024-03-21T09:58:57.895 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/ipset/ip_set.ko 2024-03-21T09:58:57.895 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/ipset/ip_set_bitmap_ip.ko 2024-03-21T09:58:57.895 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/ipset/ip_set_bitmap_ipmac.ko 2024-03-21T09:58:57.896 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/ipset/ip_set_bitmap_port.ko 2024-03-21T09:58:57.896 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/ipset/ip_set_hash_ip.ko 2024-03-21T09:58:57.896 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/ipset/ip_set_hash_ipmark.ko 2024-03-21T09:58:57.896 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/ipset/ip_set_hash_ipport.ko 2024-03-21T09:58:57.896 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/ipset/ip_set_hash_ipportip.ko 2024-03-21T09:58:57.896 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/ipset/ip_set_hash_ipportnet.ko 2024-03-21T09:58:57.896 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/ipset/ip_set_hash_mac.ko 2024-03-21T09:58:57.896 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/ipset/ip_set_hash_net.ko 2024-03-21T09:58:57.896 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/ipset/ip_set_hash_netiface.ko 2024-03-21T09:58:57.896 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/ipset/ip_set_hash_netnet.ko 2024-03-21T09:58:57.896 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/ipset/ip_set_hash_netport.ko 2024-03-21T09:58:57.896 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/ipset/ip_set_hash_netportnet.ko 2024-03-21T09:58:57.896 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/ipset/ip_set_list_set.ko 2024-03-21T09:58:57.896 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/ipvs 2024-03-21T09:58:57.896 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/ipvs/ip_vs.ko 2024-03-21T09:58:57.896 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/ipvs/ip_vs_dh.ko 2024-03-21T09:58:57.897 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/ipvs/ip_vs_fo.ko 2024-03-21T09:58:57.897 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/ipvs/ip_vs_ftp.ko 2024-03-21T09:58:57.897 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/ipvs/ip_vs_lblc.ko 2024-03-21T09:58:57.897 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/ipvs/ip_vs_lblcr.ko 2024-03-21T09:58:57.897 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/ipvs/ip_vs_lc.ko 2024-03-21T09:58:57.897 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/ipvs/ip_vs_nq.ko 2024-03-21T09:58:57.897 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/ipvs/ip_vs_pe_sip.ko 2024-03-21T09:58:57.897 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/ipvs/ip_vs_rr.ko 2024-03-21T09:58:57.897 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/ipvs/ip_vs_sed.ko 2024-03-21T09:58:57.897 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/ipvs/ip_vs_sh.ko 2024-03-21T09:58:57.897 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/ipvs/ip_vs_wlc.ko 2024-03-21T09:58:57.897 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/ipvs/ip_vs_wrr.ko 2024-03-21T09:58:57.897 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nf_conncount.ko 2024-03-21T09:58:57.897 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nf_conntrack.ko 2024-03-21T09:58:57.897 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nf_conntrack_amanda.ko 2024-03-21T09:58:57.897 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nf_conntrack_broadcast.ko 2024-03-21T09:58:57.898 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nf_conntrack_ftp.ko 2024-03-21T09:58:57.898 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nf_conntrack_h323.ko 2024-03-21T09:58:57.898 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nf_conntrack_irc.ko 2024-03-21T09:58:57.898 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nf_conntrack_netbios_ns.ko 2024-03-21T09:58:57.898 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nf_conntrack_netlink.ko 2024-03-21T09:58:57.898 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nf_conntrack_pptp.ko 2024-03-21T09:58:57.898 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nf_conntrack_sane.ko 2024-03-21T09:58:57.898 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nf_conntrack_sip.ko 2024-03-21T09:58:57.898 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nf_conntrack_snmp.ko 2024-03-21T09:58:57.898 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nf_conntrack_tftp.ko 2024-03-21T09:58:57.898 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nf_dup_netdev.ko 2024-03-21T09:58:57.898 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nf_flow_table.ko 2024-03-21T09:58:57.898 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nf_flow_table_inet.ko 2024-03-21T09:58:57.898 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nf_log_syslog.ko 2024-03-21T09:58:57.899 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nf_nat.ko 2024-03-21T09:58:57.899 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nf_nat_amanda.ko 2024-03-21T09:58:57.899 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nf_nat_ftp.ko 2024-03-21T09:58:57.899 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nf_nat_irc.ko 2024-03-21T09:58:57.899 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nf_nat_sip.ko 2024-03-21T09:58:57.899 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nf_nat_tftp.ko 2024-03-21T09:58:57.899 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nf_synproxy_core.ko 2024-03-21T09:58:57.899 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nf_tables.ko 2024-03-21T09:58:57.899 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nfnetlink.ko 2024-03-21T09:58:57.899 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nfnetlink_acct.ko 2024-03-21T09:58:57.899 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nfnetlink_log.ko 2024-03-21T09:58:57.899 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nfnetlink_osf.ko 2024-03-21T09:58:57.899 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nfnetlink_queue.ko 2024-03-21T09:58:57.899 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nft_chain_nat.ko 2024-03-21T09:58:57.900 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nft_compat.ko 2024-03-21T09:58:57.900 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nft_ct.ko 2024-03-21T09:58:57.900 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nft_dup_netdev.ko 2024-03-21T09:58:57.900 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nft_fib.ko 2024-03-21T09:58:57.900 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nft_fwd_netdev.ko 2024-03-21T09:58:57.900 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nft_hash.ko 2024-03-21T09:58:57.900 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nft_limit.ko 2024-03-21T09:58:57.900 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nft_log.ko 2024-03-21T09:58:57.900 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nft_masq.ko 2024-03-21T09:58:57.900 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nft_nat.ko 2024-03-21T09:58:57.900 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nft_numgen.ko 2024-03-21T09:58:57.900 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nft_queue.ko 2024-03-21T09:58:57.900 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nft_quota.ko 2024-03-21T09:58:57.900 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nft_redir.ko 2024-03-21T09:58:57.900 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nft_reject.ko 2024-03-21T09:58:57.900 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nft_reject_inet.ko 2024-03-21T09:58:57.901 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nft_socket.ko 2024-03-21T09:58:57.901 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nft_tunnel.ko 2024-03-21T09:58:57.901 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_AUDIT.ko 2024-03-21T09:58:57.901 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_CHECKSUM.ko 2024-03-21T09:58:57.901 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_CLASSIFY.ko 2024-03-21T09:58:57.901 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_CONNSECMARK.ko 2024-03-21T09:58:57.901 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_CT.ko 2024-03-21T09:58:57.901 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_DSCP.ko 2024-03-21T09:58:57.901 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_HL.ko 2024-03-21T09:58:57.901 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_HMARK.ko 2024-03-21T09:58:57.901 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_IDLETIMER.ko 2024-03-21T09:58:57.901 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_LED.ko 2024-03-21T09:58:57.901 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_LOG.ko 2024-03-21T09:58:57.901 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_MASQUERADE.ko 2024-03-21T09:58:57.902 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_NETMAP.ko 2024-03-21T09:58:57.902 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_NFLOG.ko 2024-03-21T09:58:57.902 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_NFQUEUE.ko 2024-03-21T09:58:57.902 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_RATEEST.ko 2024-03-21T09:58:57.902 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_REDIRECT.ko 2024-03-21T09:58:57.902 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_SECMARK.ko 2024-03-21T09:58:57.902 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_TCPMSS.ko 2024-03-21T09:58:57.902 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_TCPOPTSTRIP.ko 2024-03-21T09:58:57.902 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_TEE.ko 2024-03-21T09:58:57.902 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_TPROXY.ko 2024-03-21T09:58:57.902 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_TRACE.ko 2024-03-21T09:58:57.902 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_addrtype.ko 2024-03-21T09:58:57.902 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_bpf.ko 2024-03-21T09:58:57.902 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_cgroup.ko 2024-03-21T09:58:57.902 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_cluster.ko 2024-03-21T09:58:57.903 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_comment.ko 2024-03-21T09:58:57.903 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_connbytes.ko 2024-03-21T09:58:57.903 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_connlabel.ko 2024-03-21T09:58:57.903 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_connlimit.ko 2024-03-21T09:58:57.903 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_connmark.ko 2024-03-21T09:58:57.903 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_conntrack.ko 2024-03-21T09:58:57.903 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_cpu.ko 2024-03-21T09:58:57.903 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_dccp.ko 2024-03-21T09:58:57.903 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_devgroup.ko 2024-03-21T09:58:57.903 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_dscp.ko 2024-03-21T09:58:57.903 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_ecn.ko 2024-03-21T09:58:57.903 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_esp.ko 2024-03-21T09:58:57.903 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_hashlimit.ko 2024-03-21T09:58:57.903 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_helper.ko 2024-03-21T09:58:57.904 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_hl.ko 2024-03-21T09:58:57.904 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_ipcomp.ko 2024-03-21T09:58:57.904 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_iprange.ko 2024-03-21T09:58:57.904 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_ipvs.ko 2024-03-21T09:58:57.904 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_l2tp.ko 2024-03-21T09:58:57.904 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_length.ko 2024-03-21T09:58:57.904 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_limit.ko 2024-03-21T09:58:57.904 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_mac.ko 2024-03-21T09:58:57.904 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_mark.ko 2024-03-21T09:58:57.904 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_multiport.ko 2024-03-21T09:58:57.904 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_nat.ko 2024-03-21T09:58:57.904 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_nfacct.ko 2024-03-21T09:58:57.904 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_osf.ko 2024-03-21T09:58:57.904 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_owner.ko 2024-03-21T09:58:57.904 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_physdev.ko 2024-03-21T09:58:57.905 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_pkttype.ko 2024-03-21T09:58:57.905 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_policy.ko 2024-03-21T09:58:57.905 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_quota.ko 2024-03-21T09:58:57.905 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_rateest.ko 2024-03-21T09:58:57.905 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_realm.ko 2024-03-21T09:58:57.905 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_recent.ko 2024-03-21T09:58:57.905 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_sctp.ko 2024-03-21T09:58:57.905 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_set.ko 2024-03-21T09:58:57.905 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_socket.ko 2024-03-21T09:58:57.905 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_state.ko 2024-03-21T09:58:57.905 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_statistic.ko 2024-03-21T09:58:57.905 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_string.ko 2024-03-21T09:58:57.905 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_tcpmss.ko 2024-03-21T09:58:57.905 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_time.ko 2024-03-21T09:58:57.905 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_u32.ko 2024-03-21T09:58:57.905 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netlink 2024-03-21T09:58:57.906 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netlink/netlink_diag.ko 2024-03-21T09:58:57.906 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/nsh 2024-03-21T09:58:57.906 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/nsh/nsh.ko 2024-03-21T09:58:57.906 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/openvswitch 2024-03-21T09:58:57.906 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/openvswitch/openvswitch.ko 2024-03-21T09:58:57.906 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/openvswitch/vport-geneve.ko 2024-03-21T09:58:57.906 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/openvswitch/vport-gre.ko 2024-03-21T09:58:57.906 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/openvswitch/vport-vxlan.ko 2024-03-21T09:58:57.906 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/packet 2024-03-21T09:58:57.906 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/packet/af_packet_diag.ko 2024-03-21T09:58:57.906 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/rds 2024-03-21T09:58:57.906 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/rds/rds.ko 2024-03-21T09:58:57.906 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/rds/rds_rdma.ko 2024-03-21T09:58:57.906 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/rds/rds_tcp.ko 2024-03-21T09:58:57.907 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched 2024-03-21T09:58:57.907 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/act_csum.ko 2024-03-21T09:58:57.907 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/act_gact.ko 2024-03-21T09:58:57.907 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/act_mirred.ko 2024-03-21T09:58:57.907 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/act_nat.ko 2024-03-21T09:58:57.907 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/act_pedit.ko 2024-03-21T09:58:57.907 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/act_police.ko 2024-03-21T09:58:57.907 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/act_simple.ko 2024-03-21T09:58:57.907 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/act_skbedit.ko 2024-03-21T09:58:57.907 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/act_vlan.ko 2024-03-21T09:58:57.907 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/cls_basic.ko 2024-03-21T09:58:57.907 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/cls_bpf.ko 2024-03-21T09:58:57.907 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/cls_flow.ko 2024-03-21T09:58:57.907 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/cls_fw.ko 2024-03-21T09:58:57.908 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/cls_route.ko 2024-03-21T09:58:57.908 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/cls_u32.ko 2024-03-21T09:58:57.908 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/em_cmp.ko 2024-03-21T09:58:57.908 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/em_ipset.ko 2024-03-21T09:58:57.908 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/em_meta.ko 2024-03-21T09:58:57.908 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/em_nbyte.ko 2024-03-21T09:58:57.908 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/em_text.ko 2024-03-21T09:58:57.908 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/em_u32.ko 2024-03-21T09:58:57.908 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/sch_choke.ko 2024-03-21T09:58:57.908 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/sch_codel.ko 2024-03-21T09:58:57.908 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/sch_drr.ko 2024-03-21T09:58:57.908 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/sch_fq.ko 2024-03-21T09:58:57.908 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/sch_gred.ko 2024-03-21T09:58:57.908 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/sch_hfsc.ko 2024-03-21T09:58:57.909 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/sch_hhf.ko 2024-03-21T09:58:57.909 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/sch_htb.ko 2024-03-21T09:58:57.909 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/sch_ingress.ko 2024-03-21T09:58:57.909 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/sch_mqprio.ko 2024-03-21T09:58:57.909 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/sch_mqprio_lib.ko 2024-03-21T09:58:57.909 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/sch_multiq.ko 2024-03-21T09:58:57.909 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/sch_netem.ko 2024-03-21T09:58:57.909 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/sch_pie.ko 2024-03-21T09:58:57.909 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/sch_plug.ko 2024-03-21T09:58:57.909 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/sch_prio.ko 2024-03-21T09:58:57.909 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/sch_qfq.ko 2024-03-21T09:58:57.909 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/sch_red.ko 2024-03-21T09:58:57.909 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/sch_sfb.ko 2024-03-21T09:58:57.909 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/sch_sfq.ko 2024-03-21T09:58:57.909 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/sch_tbf.ko 2024-03-21T09:58:57.910 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/sch_teql.ko 2024-03-21T09:58:57.910 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sctp 2024-03-21T09:58:57.910 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sctp/sctp.ko 2024-03-21T09:58:57.910 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sctp/sctp_diag.ko 2024-03-21T09:58:57.910 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sunrpc 2024-03-21T09:58:57.910 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sunrpc/auth_gss 2024-03-21T09:58:57.910 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sunrpc/auth_gss/auth_rpcgss.ko 2024-03-21T09:58:57.910 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sunrpc/auth_gss/rpcsec_gss_krb5.ko 2024-03-21T09:58:57.910 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sunrpc/sunrpc.ko 2024-03-21T09:58:57.910 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sunrpc/xprtrdma 2024-03-21T09:58:57.910 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sunrpc/xprtrdma/rpcrdma.ko 2024-03-21T09:58:57.910 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/tipc 2024-03-21T09:58:57.910 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/tipc/diag.ko 2024-03-21T09:58:57.910 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/tipc/tipc.ko 2024-03-21T09:58:57.911 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/unix 2024-03-21T09:58:57.911 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/unix/unix_diag.ko 2024-03-21T09:58:57.911 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/vmw_vsock 2024-03-21T09:58:57.911 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/vmw_vsock/vmw_vsock_virtio_transport_common.ko 2024-03-21T09:58:57.911 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/vmw_vsock/vmw_vsock_vmci_transport.ko 2024-03-21T09:58:57.911 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/vmw_vsock/vsock.ko 2024-03-21T09:58:57.911 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/vmw_vsock/vsock_diag.ko 2024-03-21T09:58:57.911 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/vmw_vsock/vsock_loopback.ko 2024-03-21T09:58:57.911 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/xfrm 2024-03-21T09:58:57.911 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/xfrm/xfrm_ipcomp.ko 2024-03-21T09:58:57.911 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/security 2024-03-21T09:58:57.911 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/security/keys 2024-03-21T09:58:57.911 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/security/keys/encrypted-keys 2024-03-21T09:58:57.911 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/security/keys/encrypted-keys/encrypted-keys.ko 2024-03-21T09:58:57.912 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/security/keys/trusted-keys 2024-03-21T09:58:57.912 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/security/keys/trusted-keys/trusted.ko 2024-03-21T09:58:57.912 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/virt 2024-03-21T09:58:57.912 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/virt/lib 2024-03-21T09:58:57.912 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/virt/lib/irqbypass.ko 2024-03-21T09:58:57.912 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/modules.alias 2024-03-21T09:58:57.912 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/modules.alias.bin 2024-03-21T09:58:57.912 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/modules.builtin 2024-03-21T09:58:57.912 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/modules.builtin.alias.bin 2024-03-21T09:58:57.912 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/modules.builtin.bin 2024-03-21T09:58:57.912 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/modules.builtin.modinfo 2024-03-21T09:58:57.912 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/modules.dep 2024-03-21T09:58:57.912 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/modules.dep.bin 2024-03-21T09:58:57.912 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/modules.devname 2024-03-21T09:58:57.912 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/modules.order 2024-03-21T09:58:57.912 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/modules.softdep 2024-03-21T09:58:57.913 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/modules.symbols 2024-03-21T09:58:57.913 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/modules.symbols.bin 2024-03-21T09:58:57.913 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/vmlinuz 2024-03-21T09:58:57.916 DEBUG:teuthology.task.kernel:get_image_version: 6.8.0-g5ec3e6e88b2e 2024-03-21T09:58:57.916 DEBUG:teuthology.orchestra.run.smithi047:> rpm --scripts -qp /tmp/kernel.x86_64.rpm 2024-03-21T09:58:57.939 INFO:teuthology.orchestra.run.smithi047.stdout:postinstall scriptlet (using /bin/sh): 2024-03-21T09:58:57.939 INFO:teuthology.orchestra.run.smithi047.stdout:if [ -x /usr/bin/kernel-install ]; then 2024-03-21T09:58:57.939 INFO:teuthology.orchestra.run.smithi047.stdout: /usr/bin/kernel-install add 6.8.0-g5ec3e6e88b2e /lib/modules/6.8.0-g5ec3e6e88b2e/vmlinuz 2024-03-21T09:58:57.939 INFO:teuthology.orchestra.run.smithi047.stdout:fi 2024-03-21T09:58:57.939 INFO:teuthology.orchestra.run.smithi047.stdout:for file in vmlinuz System.map config; do 2024-03-21T09:58:57.939 INFO:teuthology.orchestra.run.smithi047.stdout: if ! cmp --silent "/lib/modules/6.8.0-g5ec3e6e88b2e/${file}" "/boot/${file}-6.8.0-g5ec3e6e88b2e"; then 2024-03-21T09:58:57.939 INFO:teuthology.orchestra.run.smithi047.stdout: cp "/lib/modules/6.8.0-g5ec3e6e88b2e/${file}" "/boot/${file}-6.8.0-g5ec3e6e88b2e" 2024-03-21T09:58:57.940 INFO:teuthology.orchestra.run.smithi047.stdout: fi 2024-03-21T09:58:57.940 INFO:teuthology.orchestra.run.smithi047.stdout:done 2024-03-21T09:58:57.940 INFO:teuthology.orchestra.run.smithi047.stdout:preuninstall scriptlet (using /bin/sh): 2024-03-21T09:58:57.940 INFO:teuthology.orchestra.run.smithi047.stdout:if [ -x /sbin/new-kernel-pkg ]; then 2024-03-21T09:58:57.940 INFO:teuthology.orchestra.run.smithi047.stdout:new-kernel-pkg --remove 6.8.0-g5ec3e6e88b2e --rminitrd --initrdfile=/boot/initramfs-6.8.0-g5ec3e6e88b2e.img 2024-03-21T09:58:57.940 INFO:teuthology.orchestra.run.smithi047.stdout:elif [ -x /usr/bin/kernel-install ]; then 2024-03-21T09:58:57.940 INFO:teuthology.orchestra.run.smithi047.stdout:kernel-install remove 6.8.0-g5ec3e6e88b2e 2024-03-21T09:58:57.940 INFO:teuthology.orchestra.run.smithi047.stdout:fi 2024-03-21T09:58:57.940 INFO:teuthology.orchestra.run.smithi047.stdout:postuninstall scriptlet (using /bin/sh): 2024-03-21T09:58:57.940 INFO:teuthology.orchestra.run.smithi047.stdout:if [ -x /sbin/update-bootloader ]; then 2024-03-21T09:58:57.940 INFO:teuthology.orchestra.run.smithi047.stdout:/sbin/update-bootloader --remove 6.8.0-g5ec3e6e88b2e 2024-03-21T09:58:57.940 INFO:teuthology.orchestra.run.smithi047.stdout:fi 2024-03-21T09:58:57.941 DEBUG:teuthology.orchestra.run.smithi047:> sudo rpm -qi grub2-tools 2024-03-21T09:58:58.020 INFO:teuthology.orchestra.run.smithi047.stdout:Name : grub2-tools 2024-03-21T09:58:58.020 INFO:teuthology.orchestra.run.smithi047.stdout:Epoch : 1 2024-03-21T09:58:58.021 INFO:teuthology.orchestra.run.smithi047.stdout:Version : 2.06 2024-03-21T09:58:58.021 INFO:teuthology.orchestra.run.smithi047.stdout:Release : 68.el9 2024-03-21T09:58:58.021 INFO:teuthology.orchestra.run.smithi047.stdout:Architecture: x86_64 2024-03-21T09:58:58.021 INFO:teuthology.orchestra.run.smithi047.stdout:Install Date: Fri 01 Sep 2023 09:14:30 PM UTC 2024-03-21T09:58:58.021 INFO:teuthology.orchestra.run.smithi047.stdout:Group : Unspecified 2024-03-21T09:58:58.021 INFO:teuthology.orchestra.run.smithi047.stdout:Size : 8264254 2024-03-21T09:58:58.021 INFO:teuthology.orchestra.run.smithi047.stdout:License : GPLv3+ 2024-03-21T09:58:58.021 INFO:teuthology.orchestra.run.smithi047.stdout:Signature : RSA/SHA256, Mon 07 Aug 2023 01:57:16 PM UTC, Key ID 05b555b38483c65d 2024-03-21T09:58:58.021 INFO:teuthology.orchestra.run.smithi047.stdout:Source RPM : grub2-2.06-68.el9.src.rpm 2024-03-21T09:58:58.021 INFO:teuthology.orchestra.run.smithi047.stdout:Build Date : Wed 02 Aug 2023 03:15:48 PM UTC 2024-03-21T09:58:58.021 INFO:teuthology.orchestra.run.smithi047.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-03-21T09:58:58.021 INFO:teuthology.orchestra.run.smithi047.stdout:Packager : builder@centos.org 2024-03-21T09:58:58.021 INFO:teuthology.orchestra.run.smithi047.stdout:Vendor : CentOS 2024-03-21T09:58:58.021 INFO:teuthology.orchestra.run.smithi047.stdout:URL : http://www.gnu.org/software/grub/ 2024-03-21T09:58:58.021 INFO:teuthology.orchestra.run.smithi047.stdout:Summary : Support tools for GRUB. 2024-03-21T09:58:58.021 INFO:teuthology.orchestra.run.smithi047.stdout:Description : 2024-03-21T09:58:58.022 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-21T09:58:58.022 INFO:teuthology.orchestra.run.smithi047.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-03-21T09:58:58.022 INFO:teuthology.orchestra.run.smithi047.stdout:customizable bootloader with modular architecture. It supports a rich 2024-03-21T09:58:58.022 INFO:teuthology.orchestra.run.smithi047.stdout:variety of kernel formats, file systems, computer architectures and 2024-03-21T09:58:58.022 INFO:teuthology.orchestra.run.smithi047.stdout:hardware devices. 2024-03-21T09:58:58.022 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-21T09:58:58.022 INFO:teuthology.orchestra.run.smithi047.stdout:This subpackage provides tools for support of all platforms. 2024-03-21T09:58:58.023 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-03-21T09:58:58.023 INFO:teuthology.task.kernel:Updating grub on smithi047 to boot 6.8.0-g5ec3e6e88b2e 2024-03-21T09:58:58.023 DEBUG:teuthology.orchestra.run.smithi047:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-03-21T09:58:58.648 INFO:teuthology.orchestra.run.smithi047.stderr:Generating grub configuration file ... 2024-03-21T09:59:00.214 INFO:teuthology.orchestra.run.smithi047.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-03-21T09:59:00.245 INFO:teuthology.orchestra.run.smithi047.stderr:done 2024-03-21T09:59:00.247 DEBUG:teuthology.orchestra.run.smithi047:> mktemp 2024-03-21T09:59:00.262 INFO:teuthology.orchestra.run.smithi047.stdout:/tmp/tmp.LwIgFaH6Zf 2024-03-21T09:59:00.263 DEBUG:teuthology.orchestra.run.smithi047:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.LwIgFaH6Zf 2024-03-21T09:59:00.340 DEBUG:teuthology.orchestra.run.smithi047:> sudo chmod 0666 /tmp/tmp.LwIgFaH6Zf 2024-03-21T09:59:00.486 DEBUG:teuthology.orchestra.remote:smithi047:/tmp/tmp.LwIgFaH6Zf is 6KB 2024-03-21T09:59:00.535 DEBUG:teuthology.orchestra.run.smithi047:> rm -fr /tmp/tmp.LwIgFaH6Zf 2024-03-21T09:59:00.550 DEBUG:teuthology.orchestra.run.smithi047:> sudo /bin/ls /boot/loader/entries || true 2024-03-21T09:59:00.621 INFO:teuthology.orchestra.run.smithi047.stdout:15949e3c88704300882482eddfd7baae-0-rescue.conf 2024-03-21T09:59:00.622 INFO:teuthology.orchestra.run.smithi047.stdout:15949e3c88704300882482eddfd7baae-5.14.0-239.el9.x86_64.conf 2024-03-21T09:59:00.622 INFO:teuthology.orchestra.run.smithi047.stdout:15949e3c88704300882482eddfd7baae-5.14.0-361.el9.x86_64.conf 2024-03-21T09:59:00.622 INFO:teuthology.orchestra.run.smithi047.stdout:15949e3c88704300882482eddfd7baae-6.8.0-g5ec3e6e88b2e.conf 2024-03-21T09:59:00.623 DEBUG:teuthology.orchestra.run.smithi047:> sudo grub2-set-default 15949e3c88704300882482eddfd7baae-6.8.0-g5ec3e6e88b2e 2024-03-21T09:59:00.808 DEBUG:teuthology.orchestra.run.smithi047:> sudo shutdown -r now 2024-03-21T09:59:04.032 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi157.front.sepia.ceph.com, path=/tmp/kernel.x86_64.rpm, version=None) 2024-03-21T09:59:04.032 DEBUG:teuthology.orchestra.run.smithi157:> rpm -qlp /tmp/kernel.x86_64.rpm 2024-03-21T09:59:04.109 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e 2024-03-21T09:59:04.110 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/System.map 2024-03-21T09:59:04.110 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/config 2024-03-21T09:59:04.110 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel 2024-03-21T09:59:04.110 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/arch 2024-03-21T09:59:04.110 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/arch/x86 2024-03-21T09:59:04.110 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/arch/x86/crypto 2024-03-21T09:59:04.110 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/arch/x86/crypto/blowfish-x86_64.ko 2024-03-21T09:59:04.110 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/arch/x86/crypto/camellia-aesni-avx-x86_64.ko 2024-03-21T09:59:04.111 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/arch/x86/crypto/camellia-aesni-avx2.ko 2024-03-21T09:59:04.111 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/arch/x86/crypto/camellia-x86_64.ko 2024-03-21T09:59:04.111 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/arch/x86/crypto/cast5-avx-x86_64.ko 2024-03-21T09:59:04.111 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/arch/x86/crypto/cast6-avx-x86_64.ko 2024-03-21T09:59:04.111 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/arch/x86/crypto/crc32-pclmul.ko 2024-03-21T09:59:04.111 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/arch/x86/crypto/crc32c-intel.ko 2024-03-21T09:59:04.111 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/arch/x86/crypto/crct10dif-pclmul.ko 2024-03-21T09:59:04.111 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/arch/x86/crypto/des3_ede-x86_64.ko 2024-03-21T09:59:04.111 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/arch/x86/crypto/ghash-clmulni-intel.ko 2024-03-21T09:59:04.111 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/arch/x86/crypto/serpent-avx-x86_64.ko 2024-03-21T09:59:04.112 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/arch/x86/crypto/serpent-avx2.ko 2024-03-21T09:59:04.112 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/arch/x86/crypto/serpent-sse2-x86_64.ko 2024-03-21T09:59:04.112 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/arch/x86/crypto/sha1-ssse3.ko 2024-03-21T09:59:04.112 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/arch/x86/crypto/sha256-ssse3.ko 2024-03-21T09:59:04.112 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/arch/x86/crypto/sha512-ssse3.ko 2024-03-21T09:59:04.112 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/arch/x86/crypto/twofish-avx-x86_64.ko 2024-03-21T09:59:04.112 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/arch/x86/crypto/twofish-x86_64-3way.ko 2024-03-21T09:59:04.112 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/arch/x86/crypto/twofish-x86_64.ko 2024-03-21T09:59:04.112 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/arch/x86/kernel 2024-03-21T09:59:04.112 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/arch/x86/kernel/cpu 2024-03-21T09:59:04.113 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/arch/x86/kernel/cpu/mce 2024-03-21T09:59:04.113 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/arch/x86/kernel/cpu/mce/mce-inject.ko 2024-03-21T09:59:04.113 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/arch/x86/kvm 2024-03-21T09:59:04.113 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/arch/x86/kvm/kvm-amd.ko 2024-03-21T09:59:04.113 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/arch/x86/kvm/kvm-intel.ko 2024-03-21T09:59:04.113 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/arch/x86/kvm/kvm.ko 2024-03-21T09:59:04.113 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto 2024-03-21T09:59:04.113 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/ansi_cprng.ko 2024-03-21T09:59:04.113 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/anubis.ko 2024-03-21T09:59:04.113 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/arc4.ko 2024-03-21T09:59:04.113 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/async_tx 2024-03-21T09:59:04.114 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/async_tx/async_memcpy.ko 2024-03-21T09:59:04.114 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/async_tx/async_pq.ko 2024-03-21T09:59:04.114 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/async_tx/async_raid6_recov.ko 2024-03-21T09:59:04.114 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/async_tx/async_tx.ko 2024-03-21T09:59:04.114 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/async_tx/async_xor.ko 2024-03-21T09:59:04.114 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/async_tx/raid6test.ko 2024-03-21T09:59:04.114 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/authenc.ko 2024-03-21T09:59:04.114 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/authencesn.ko 2024-03-21T09:59:04.114 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/blake2b_generic.ko 2024-03-21T09:59:04.114 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/blowfish_common.ko 2024-03-21T09:59:04.114 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/blowfish_generic.ko 2024-03-21T09:59:04.114 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/camellia_generic.ko 2024-03-21T09:59:04.114 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/cast5_generic.ko 2024-03-21T09:59:04.115 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/cast6_generic.ko 2024-03-21T09:59:04.115 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/cast_common.ko 2024-03-21T09:59:04.115 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/ccm.ko 2024-03-21T09:59:04.115 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/cmac.ko 2024-03-21T09:59:04.115 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/crc32_generic.ko 2024-03-21T09:59:04.115 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/crypto_engine.ko 2024-03-21T09:59:04.115 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/crypto_user.ko 2024-03-21T09:59:04.115 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/cts.ko 2024-03-21T09:59:04.115 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/des_generic.ko 2024-03-21T09:59:04.115 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/dh_generic.ko 2024-03-21T09:59:04.115 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/echainiv.ko 2024-03-21T09:59:04.115 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/essiv.ko 2024-03-21T09:59:04.115 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/fcrypt.ko 2024-03-21T09:59:04.115 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/khazad.ko 2024-03-21T09:59:04.116 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/lz4.ko 2024-03-21T09:59:04.116 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/lz4hc.ko 2024-03-21T09:59:04.116 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/md4.ko 2024-03-21T09:59:04.116 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/michael_mic.ko 2024-03-21T09:59:04.116 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/pcbc.ko 2024-03-21T09:59:04.116 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/pcrypt.ko 2024-03-21T09:59:04.116 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/rmd160.ko 2024-03-21T09:59:04.116 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/seed.ko 2024-03-21T09:59:04.116 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/serpent_generic.ko 2024-03-21T09:59:04.116 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/tcrypt.ko 2024-03-21T09:59:04.116 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/tea.ko 2024-03-21T09:59:04.116 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/twofish_common.ko 2024-03-21T09:59:04.116 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/twofish_generic.ko 2024-03-21T09:59:04.116 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/vmac.ko 2024-03-21T09:59:04.117 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/wp512.ko 2024-03-21T09:59:04.117 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/xcbc.ko 2024-03-21T09:59:04.117 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/xor.ko 2024-03-21T09:59:04.117 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/xxhash_generic.ko 2024-03-21T09:59:04.117 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers 2024-03-21T09:59:04.117 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/acpi 2024-03-21T09:59:04.117 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/acpi/acpi_ipmi.ko 2024-03-21T09:59:04.117 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/acpi/acpi_pad.ko 2024-03-21T09:59:04.117 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/acpi/custom_method.ko 2024-03-21T09:59:04.117 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/acpi/ec_sys.ko 2024-03-21T09:59:04.117 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/acpi/platform_profile.ko 2024-03-21T09:59:04.117 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/acpi/sbs.ko 2024-03-21T09:59:04.117 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/acpi/sbshc.ko 2024-03-21T09:59:04.117 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/acpi/video.ko 2024-03-21T09:59:04.117 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata 2024-03-21T09:59:04.118 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/acard-ahci.ko 2024-03-21T09:59:04.118 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/ahci_platform.ko 2024-03-21T09:59:04.118 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/ata_generic.ko 2024-03-21T09:59:04.118 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/libahci_platform.ko 2024-03-21T09:59:04.118 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_acpi.ko 2024-03-21T09:59:04.118 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_ali.ko 2024-03-21T09:59:04.118 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_amd.ko 2024-03-21T09:59:04.118 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_artop.ko 2024-03-21T09:59:04.118 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_atiixp.ko 2024-03-21T09:59:04.118 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_atp867x.ko 2024-03-21T09:59:04.118 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_cmd640.ko 2024-03-21T09:59:04.118 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_cmd64x.ko 2024-03-21T09:59:04.118 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_cypress.ko 2024-03-21T09:59:04.118 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_efar.ko 2024-03-21T09:59:04.119 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_hpt366.ko 2024-03-21T09:59:04.119 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_hpt37x.ko 2024-03-21T09:59:04.119 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_hpt3x2n.ko 2024-03-21T09:59:04.119 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_hpt3x3.ko 2024-03-21T09:59:04.119 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_it8213.ko 2024-03-21T09:59:04.119 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_it821x.ko 2024-03-21T09:59:04.119 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_jmicron.ko 2024-03-21T09:59:04.119 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_marvell.ko 2024-03-21T09:59:04.119 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_mpiix.ko 2024-03-21T09:59:04.119 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_netcell.ko 2024-03-21T09:59:04.119 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_ninja32.ko 2024-03-21T09:59:04.119 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_ns87410.ko 2024-03-21T09:59:04.119 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_ns87415.ko 2024-03-21T09:59:04.119 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_oldpiix.ko 2024-03-21T09:59:04.120 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_opti.ko 2024-03-21T09:59:04.120 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_optidma.ko 2024-03-21T09:59:04.120 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_pcmcia.ko 2024-03-21T09:59:04.120 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_pdc2027x.ko 2024-03-21T09:59:04.120 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_pdc202xx_old.ko 2024-03-21T09:59:04.120 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_piccolo.ko 2024-03-21T09:59:04.120 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_rdc.ko 2024-03-21T09:59:04.120 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_sch.ko 2024-03-21T09:59:04.120 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_serverworks.ko 2024-03-21T09:59:04.120 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_sil680.ko 2024-03-21T09:59:04.120 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_sis.ko 2024-03-21T09:59:04.120 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_sl82c105.ko 2024-03-21T09:59:04.120 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_triflex.ko 2024-03-21T09:59:04.121 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_via.ko 2024-03-21T09:59:04.121 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pdc_adma.ko 2024-03-21T09:59:04.121 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/sata_inic162x.ko 2024-03-21T09:59:04.121 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/sata_mv.ko 2024-03-21T09:59:04.121 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/sata_nv.ko 2024-03-21T09:59:04.121 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/sata_promise.ko 2024-03-21T09:59:04.121 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/sata_qstor.ko 2024-03-21T09:59:04.121 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/sata_sil.ko 2024-03-21T09:59:04.121 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/sata_sil24.ko 2024-03-21T09:59:04.121 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/sata_sis.ko 2024-03-21T09:59:04.121 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/sata_svw.ko 2024-03-21T09:59:04.121 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/sata_sx4.ko 2024-03-21T09:59:04.121 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/sata_uli.ko 2024-03-21T09:59:04.121 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/sata_via.ko 2024-03-21T09:59:04.121 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/sata_vsc.ko 2024-03-21T09:59:04.122 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/auxdisplay 2024-03-21T09:59:04.122 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/auxdisplay/ks0108.ko 2024-03-21T09:59:04.122 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/base 2024-03-21T09:59:04.122 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/base/regmap 2024-03-21T09:59:04.122 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/base/regmap/regmap-i2c.ko 2024-03-21T09:59:04.122 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/base/regmap/regmap-mmio.ko 2024-03-21T09:59:04.122 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/bcma 2024-03-21T09:59:04.122 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/bcma/bcma.ko 2024-03-21T09:59:04.122 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/block 2024-03-21T09:59:04.122 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/block/aoe 2024-03-21T09:59:04.122 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/block/aoe/aoe.ko 2024-03-21T09:59:04.122 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/block/brd.ko 2024-03-21T09:59:04.122 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/block/drbd 2024-03-21T09:59:04.123 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/block/drbd/drbd.ko 2024-03-21T09:59:04.123 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/block/floppy.ko 2024-03-21T09:59:04.123 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/block/loop.ko 2024-03-21T09:59:04.123 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/block/mtip32xx 2024-03-21T09:59:04.123 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/block/mtip32xx/mtip32xx.ko 2024-03-21T09:59:04.123 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/block/nbd.ko 2024-03-21T09:59:04.123 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/block/null_blk 2024-03-21T09:59:04.123 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/block/null_blk/null_blk.ko 2024-03-21T09:59:04.123 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/block/pktcdvd.ko 2024-03-21T09:59:04.123 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/block/rbd.ko 2024-03-21T09:59:04.123 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/block/virtio_blk.ko 2024-03-21T09:59:04.123 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/block/xen-blkback 2024-03-21T09:59:04.123 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/block/xen-blkback/xen-blkback.ko 2024-03-21T09:59:04.123 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/block/xen-blkfront.ko 2024-03-21T09:59:04.124 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/block/zram 2024-03-21T09:59:04.124 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/block/zram/zram.ko 2024-03-21T09:59:04.124 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/char 2024-03-21T09:59:04.124 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/char/hangcheck-timer.ko 2024-03-21T09:59:04.124 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/char/hw_random 2024-03-21T09:59:04.124 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/char/hw_random/amd-rng.ko 2024-03-21T09:59:04.124 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/char/hw_random/intel-rng.ko 2024-03-21T09:59:04.124 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/char/hw_random/timeriomem-rng.ko 2024-03-21T09:59:04.124 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/char/hw_random/via-rng.ko 2024-03-21T09:59:04.124 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/char/hw_random/virtio-rng.ko 2024-03-21T09:59:04.124 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/char/ipmi 2024-03-21T09:59:04.124 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/char/ipmi/ipmi_devintf.ko 2024-03-21T09:59:04.124 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/char/ipmi/ipmi_msghandler.ko 2024-03-21T09:59:04.124 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/char/ipmi/ipmi_poweroff.ko 2024-03-21T09:59:04.125 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/char/ipmi/ipmi_si.ko 2024-03-21T09:59:04.125 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/char/ipmi/ipmi_ssif.ko 2024-03-21T09:59:04.125 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/char/ipmi/ipmi_watchdog.ko 2024-03-21T09:59:04.125 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/char/lp.ko 2024-03-21T09:59:04.125 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/char/mwave 2024-03-21T09:59:04.125 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/char/mwave/mwave.ko 2024-03-21T09:59:04.125 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/char/ppdev.ko 2024-03-21T09:59:04.125 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/char/tlclk.ko 2024-03-21T09:59:04.125 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/char/tpm 2024-03-21T09:59:04.125 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/char/tpm/tpm.ko 2024-03-21T09:59:04.125 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/char/tpm/tpm_atmel.ko 2024-03-21T09:59:04.125 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/char/tpm/tpm_infineon.ko 2024-03-21T09:59:04.125 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/char/tpm/tpm_nsc.ko 2024-03-21T09:59:04.126 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/char/tpm/tpm_tis.ko 2024-03-21T09:59:04.126 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/char/tpm/tpm_tis_core.ko 2024-03-21T09:59:04.126 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/char/uv_mmtimer.ko 2024-03-21T09:59:04.126 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/char/virtio_console.ko 2024-03-21T09:59:04.126 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/cpufreq 2024-03-21T09:59:04.126 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/cpufreq/acpi-cpufreq.ko 2024-03-21T09:59:04.126 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/cpufreq/amd_freq_sensitivity.ko 2024-03-21T09:59:04.126 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/cpufreq/p4-clockmod.ko 2024-03-21T09:59:04.126 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/cpufreq/pcc-cpufreq.ko 2024-03-21T09:59:04.126 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/cpufreq/powernow-k8.ko 2024-03-21T09:59:04.126 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/cpufreq/speedstep-lib.ko 2024-03-21T09:59:04.126 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/crypto 2024-03-21T09:59:04.127 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/crypto/ccp 2024-03-21T09:59:04.127 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/crypto/ccp/ccp-crypto.ko 2024-03-21T09:59:04.127 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/crypto/ccp/ccp.ko 2024-03-21T09:59:04.127 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/crypto/intel 2024-03-21T09:59:04.127 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/crypto/intel/qat 2024-03-21T09:59:04.127 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/crypto/intel/qat/qat_common 2024-03-21T09:59:04.127 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/crypto/intel/qat/qat_common/intel_qat.ko 2024-03-21T09:59:04.127 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/crypto/intel/qat/qat_dh895xcc 2024-03-21T09:59:04.127 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/crypto/intel/qat/qat_dh895xcc/qat_dh895xcc.ko 2024-03-21T09:59:04.127 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/crypto/padlock-aes.ko 2024-03-21T09:59:04.127 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/crypto/padlock-sha.ko 2024-03-21T09:59:04.127 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/crypto/virtio 2024-03-21T09:59:04.127 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/crypto/virtio/virtio_crypto.ko 2024-03-21T09:59:04.128 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/dca 2024-03-21T09:59:04.128 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/dca/dca.ko 2024-03-21T09:59:04.128 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/devfreq 2024-03-21T09:59:04.128 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/devfreq/governor_simpleondemand.ko 2024-03-21T09:59:04.128 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/dma 2024-03-21T09:59:04.128 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/dma/dw 2024-03-21T09:59:04.128 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/dma/dw/dw_dmac.ko 2024-03-21T09:59:04.128 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/dma/ioat 2024-03-21T09:59:04.128 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/dma/ioat/ioatdma.ko 2024-03-21T09:59:04.128 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/edac 2024-03-21T09:59:04.128 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/edac/amd64_edac.ko 2024-03-21T09:59:04.128 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/edac/e752x_edac.ko 2024-03-21T09:59:04.128 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/edac/edac_mce_amd.ko 2024-03-21T09:59:04.128 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/edac/i3000_edac.ko 2024-03-21T09:59:04.129 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/edac/i3200_edac.ko 2024-03-21T09:59:04.129 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/edac/i5100_edac.ko 2024-03-21T09:59:04.129 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/edac/i5400_edac.ko 2024-03-21T09:59:04.129 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/edac/i7300_edac.ko 2024-03-21T09:59:04.129 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/edac/i7core_edac.ko 2024-03-21T09:59:04.129 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/edac/i82975x_edac.ko 2024-03-21T09:59:04.129 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/edac/ie31200_edac.ko 2024-03-21T09:59:04.129 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/edac/sb_edac.ko 2024-03-21T09:59:04.129 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/edac/x38_edac.ko 2024-03-21T09:59:04.129 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/firmware 2024-03-21T09:59:04.129 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/firmware/edd.ko 2024-03-21T09:59:04.129 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/firmware/iscsi_ibft.ko 2024-03-21T09:59:04.129 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/gpio 2024-03-21T09:59:04.129 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/gpio/gpio-ich.ko 2024-03-21T09:59:04.130 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/gpio/gpio-viperboard.ko 2024-03-21T09:59:04.130 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid 2024-03-21T09:59:04.130 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-appleir.ko 2024-03-21T09:59:04.130 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-aureal.ko 2024-03-21T09:59:04.130 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-axff.ko 2024-03-21T09:59:04.130 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-dr.ko 2024-03-21T09:59:04.130 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-elecom.ko 2024-03-21T09:59:04.130 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-elo.ko 2024-03-21T09:59:04.130 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-emsff.ko 2024-03-21T09:59:04.130 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-gaff.ko 2024-03-21T09:59:04.130 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-gt683r.ko 2024-03-21T09:59:04.130 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-gyration.ko 2024-03-21T09:59:04.130 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-holtek-kbd.ko 2024-03-21T09:59:04.130 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-holtek-mouse.ko 2024-03-21T09:59:04.131 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-holtekff.ko 2024-03-21T09:59:04.131 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-hyperv.ko 2024-03-21T09:59:04.131 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-icade.ko 2024-03-21T09:59:04.131 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-keytouch.ko 2024-03-21T09:59:04.131 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-kye.ko 2024-03-21T09:59:04.131 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-lcpower.ko 2024-03-21T09:59:04.131 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-led.ko 2024-03-21T09:59:04.131 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-lenovo.ko 2024-03-21T09:59:04.131 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-logitech-dj.ko 2024-03-21T09:59:04.131 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-logitech-hidpp.ko 2024-03-21T09:59:04.131 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-multitouch.ko 2024-03-21T09:59:04.131 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-ortek.ko 2024-03-21T09:59:04.131 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-penmount.ko 2024-03-21T09:59:04.131 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-petalynx.ko 2024-03-21T09:59:04.132 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-picolcd.ko 2024-03-21T09:59:04.132 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-pl.ko 2024-03-21T09:59:04.132 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-plantronics.ko 2024-03-21T09:59:04.132 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-primax.ko 2024-03-21T09:59:04.132 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-rmi.ko 2024-03-21T09:59:04.132 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-roccat-arvo.ko 2024-03-21T09:59:04.132 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-roccat-common.ko 2024-03-21T09:59:04.132 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-roccat-isku.ko 2024-03-21T09:59:04.132 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-roccat-kone.ko 2024-03-21T09:59:04.132 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-roccat-koneplus.ko 2024-03-21T09:59:04.132 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-roccat-konepure.ko 2024-03-21T09:59:04.132 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-roccat-kovaplus.ko 2024-03-21T09:59:04.132 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-roccat-lua.ko 2024-03-21T09:59:04.132 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-roccat-pyra.ko 2024-03-21T09:59:04.133 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-roccat-ryos.ko 2024-03-21T09:59:04.133 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-roccat-savu.ko 2024-03-21T09:59:04.133 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-roccat.ko 2024-03-21T09:59:04.133 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-saitek.ko 2024-03-21T09:59:04.133 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-samsung.ko 2024-03-21T09:59:04.133 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-sensor-hub.ko 2024-03-21T09:59:04.133 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-sjoy.ko 2024-03-21T09:59:04.133 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-sony.ko 2024-03-21T09:59:04.133 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-speedlink.ko 2024-03-21T09:59:04.133 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-steelseries.ko 2024-03-21T09:59:04.133 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-sunplus.ko 2024-03-21T09:59:04.134 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-thrustmaster.ko 2024-03-21T09:59:04.134 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-tivo.ko 2024-03-21T09:59:04.134 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-tmff.ko 2024-03-21T09:59:04.134 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-topseed.ko 2024-03-21T09:59:04.134 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-twinhan.ko 2024-03-21T09:59:04.134 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-uclogic.ko 2024-03-21T09:59:04.134 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-waltop.ko 2024-03-21T09:59:04.134 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-wiimote.ko 2024-03-21T09:59:04.134 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-xinmo.ko 2024-03-21T09:59:04.134 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-zpff.ko 2024-03-21T09:59:04.134 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-zydacron.ko 2024-03-21T09:59:04.134 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/uhid.ko 2024-03-21T09:59:04.134 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/wacom.ko 2024-03-21T09:59:04.135 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hv 2024-03-21T09:59:04.135 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hv/hv_balloon.ko 2024-03-21T09:59:04.135 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hv/hv_utils.ko 2024-03-21T09:59:04.135 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hv/hv_vmbus.ko 2024-03-21T09:59:04.135 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon 2024-03-21T09:59:04.135 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/abituguru.ko 2024-03-21T09:59:04.135 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/abituguru3.ko 2024-03-21T09:59:04.135 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/acpi_power_meter.ko 2024-03-21T09:59:04.135 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/ad7414.ko 2024-03-21T09:59:04.135 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/ad7418.ko 2024-03-21T09:59:04.135 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/adc128d818.ko 2024-03-21T09:59:04.136 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/adm1025.ko 2024-03-21T09:59:04.136 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/adm1026.ko 2024-03-21T09:59:04.136 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/adm1029.ko 2024-03-21T09:59:04.136 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/adm1031.ko 2024-03-21T09:59:04.136 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/adm9240.ko 2024-03-21T09:59:04.136 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/ads7828.ko 2024-03-21T09:59:04.136 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/adt7410.ko 2024-03-21T09:59:04.136 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/adt7411.ko 2024-03-21T09:59:04.136 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/adt7462.ko 2024-03-21T09:59:04.136 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/adt7470.ko 2024-03-21T09:59:04.136 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/adt7475.ko 2024-03-21T09:59:04.136 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/adt7x10.ko 2024-03-21T09:59:04.136 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/amc6821.ko 2024-03-21T09:59:04.137 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/applesmc.ko 2024-03-21T09:59:04.137 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/asb100.ko 2024-03-21T09:59:04.137 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/asc7621.ko 2024-03-21T09:59:04.137 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/asus_atk0110.ko 2024-03-21T09:59:04.137 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/atxp1.ko 2024-03-21T09:59:04.137 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/coretemp.ko 2024-03-21T09:59:04.137 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/dell-smm-hwmon.ko 2024-03-21T09:59:04.137 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/dme1737.ko 2024-03-21T09:59:04.137 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/ds1621.ko 2024-03-21T09:59:04.137 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/ds620.ko 2024-03-21T09:59:04.137 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/emc1403.ko 2024-03-21T09:59:04.138 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/emc6w201.ko 2024-03-21T09:59:04.138 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/f71805f.ko 2024-03-21T09:59:04.138 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/f71882fg.ko 2024-03-21T09:59:04.138 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/f75375s.ko 2024-03-21T09:59:04.138 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/fam15h_power.ko 2024-03-21T09:59:04.138 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/fschmd.ko 2024-03-21T09:59:04.138 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/g760a.ko 2024-03-21T09:59:04.138 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/g762.ko 2024-03-21T09:59:04.138 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/gl518sm.ko 2024-03-21T09:59:04.138 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/gl520sm.ko 2024-03-21T09:59:04.138 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/hwmon-vid.ko 2024-03-21T09:59:04.138 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/i5500_temp.ko 2024-03-21T09:59:04.139 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/i5k_amb.ko 2024-03-21T09:59:04.139 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/ibmaem.ko 2024-03-21T09:59:04.139 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/ibmpex.ko 2024-03-21T09:59:04.139 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/ina209.ko 2024-03-21T09:59:04.139 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/ina2xx.ko 2024-03-21T09:59:04.139 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/it87.ko 2024-03-21T09:59:04.139 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/k10temp.ko 2024-03-21T09:59:04.139 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/k8temp.ko 2024-03-21T09:59:04.139 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/lineage-pem.ko 2024-03-21T09:59:04.139 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/lm63.ko 2024-03-21T09:59:04.139 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/lm73.ko 2024-03-21T09:59:04.139 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/lm75.ko 2024-03-21T09:59:04.139 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/lm77.ko 2024-03-21T09:59:04.139 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/lm78.ko 2024-03-21T09:59:04.140 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/lm80.ko 2024-03-21T09:59:04.140 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/lm83.ko 2024-03-21T09:59:04.140 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/lm85.ko 2024-03-21T09:59:04.140 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/lm87.ko 2024-03-21T09:59:04.140 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/lm90.ko 2024-03-21T09:59:04.140 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/lm92.ko 2024-03-21T09:59:04.140 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/lm93.ko 2024-03-21T09:59:04.140 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/lm95234.ko 2024-03-21T09:59:04.140 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/lm95241.ko 2024-03-21T09:59:04.140 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/lm95245.ko 2024-03-21T09:59:04.140 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/ltc2945.ko 2024-03-21T09:59:04.140 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/ltc4151.ko 2024-03-21T09:59:04.140 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/ltc4215.ko 2024-03-21T09:59:04.140 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/ltc4222.ko 2024-03-21T09:59:04.141 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/ltc4245.ko 2024-03-21T09:59:04.141 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/ltc4260.ko 2024-03-21T09:59:04.141 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/ltc4261.ko 2024-03-21T09:59:04.141 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/max16065.ko 2024-03-21T09:59:04.141 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/max1619.ko 2024-03-21T09:59:04.141 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/max1668.ko 2024-03-21T09:59:04.141 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/max197.ko 2024-03-21T09:59:04.141 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/max6639.ko 2024-03-21T09:59:04.141 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/max6650.ko 2024-03-21T09:59:04.141 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/max6697.ko 2024-03-21T09:59:04.141 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/mcp3021.ko 2024-03-21T09:59:04.142 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/nct6683.ko 2024-03-21T09:59:04.142 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/nct6775-core.ko 2024-03-21T09:59:04.142 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/nct6775.ko 2024-03-21T09:59:04.142 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/nct7802.ko 2024-03-21T09:59:04.142 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/pc87360.ko 2024-03-21T09:59:04.142 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/pc87427.ko 2024-03-21T09:59:04.142 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/pcf8591.ko 2024-03-21T09:59:04.142 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/pmbus 2024-03-21T09:59:04.142 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/pmbus/adm1275.ko 2024-03-21T09:59:04.142 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/pmbus/lm25066.ko 2024-03-21T09:59:04.142 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/pmbus/ltc2978.ko 2024-03-21T09:59:04.142 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/pmbus/max16064.ko 2024-03-21T09:59:04.143 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/pmbus/max34440.ko 2024-03-21T09:59:04.143 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/pmbus/max8688.ko 2024-03-21T09:59:04.143 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/pmbus/pmbus.ko 2024-03-21T09:59:04.143 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/pmbus/pmbus_core.ko 2024-03-21T09:59:04.143 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/pmbus/tps40422.ko 2024-03-21T09:59:04.143 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/pmbus/ucd9000.ko 2024-03-21T09:59:04.143 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/pmbus/ucd9200.ko 2024-03-21T09:59:04.143 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/pmbus/zl6100.ko 2024-03-21T09:59:04.143 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/powr1220.ko 2024-03-21T09:59:04.143 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/sch5627.ko 2024-03-21T09:59:04.144 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/sch5636.ko 2024-03-21T09:59:04.144 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/sch56xx-common.ko 2024-03-21T09:59:04.144 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/sht15.ko 2024-03-21T09:59:04.144 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/sht21.ko 2024-03-21T09:59:04.144 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/shtc1.ko 2024-03-21T09:59:04.144 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/sis5595.ko 2024-03-21T09:59:04.144 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/smsc47b397.ko 2024-03-21T09:59:04.144 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/smsc47m1.ko 2024-03-21T09:59:04.144 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/smsc47m192.ko 2024-03-21T09:59:04.144 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/thmc50.ko 2024-03-21T09:59:04.145 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/tmp102.ko 2024-03-21T09:59:04.145 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/tmp103.ko 2024-03-21T09:59:04.145 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/tmp401.ko 2024-03-21T09:59:04.145 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/tmp421.ko 2024-03-21T09:59:04.145 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/via-cputemp.ko 2024-03-21T09:59:04.145 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/via686a.ko 2024-03-21T09:59:04.145 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/vt1211.ko 2024-03-21T09:59:04.145 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/vt8231.ko 2024-03-21T09:59:04.145 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/w83627ehf.ko 2024-03-21T09:59:04.145 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/w83627hf.ko 2024-03-21T09:59:04.145 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/w83781d.ko 2024-03-21T09:59:04.146 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/w83791d.ko 2024-03-21T09:59:04.146 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/w83792d.ko 2024-03-21T09:59:04.146 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/w83793.ko 2024-03-21T09:59:04.146 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/w83795.ko 2024-03-21T09:59:04.146 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/w83l785ts.ko 2024-03-21T09:59:04.146 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/w83l786ng.ko 2024-03-21T09:59:04.146 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/i2c 2024-03-21T09:59:04.146 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/i2c/algos 2024-03-21T09:59:04.146 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/i2c/algos/i2c-algo-bit.ko 2024-03-21T09:59:04.147 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/i2c/algos/i2c-algo-pca.ko 2024-03-21T09:59:04.147 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/i2c/busses 2024-03-21T09:59:04.147 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/i2c/busses/i2c-amd756-s4882.ko 2024-03-21T09:59:04.147 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/i2c/busses/i2c-amd756.ko 2024-03-21T09:59:04.147 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/i2c/busses/i2c-amd8111.ko 2024-03-21T09:59:04.147 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/i2c/busses/i2c-ccgx-ucsi.ko 2024-03-21T09:59:04.147 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/i2c/busses/i2c-designware-core.ko 2024-03-21T09:59:04.147 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/i2c/busses/i2c-designware-pci.ko 2024-03-21T09:59:04.147 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/i2c/busses/i2c-designware-platform.ko 2024-03-21T09:59:04.147 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/i2c/busses/i2c-diolan-u2c.ko 2024-03-21T09:59:04.148 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/i2c/busses/i2c-i801.ko 2024-03-21T09:59:04.148 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/i2c/busses/i2c-isch.ko 2024-03-21T09:59:04.148 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/i2c/busses/i2c-ismt.ko 2024-03-21T09:59:04.148 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/i2c/busses/i2c-nforce2-s4985.ko 2024-03-21T09:59:04.148 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/i2c/busses/i2c-nforce2.ko 2024-03-21T09:59:04.148 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/i2c/busses/i2c-parport.ko 2024-03-21T09:59:04.148 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/i2c/busses/i2c-pca-platform.ko 2024-03-21T09:59:04.148 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/i2c/busses/i2c-piix4.ko 2024-03-21T09:59:04.148 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/i2c/busses/i2c-scmi.ko 2024-03-21T09:59:04.148 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/i2c/busses/i2c-simtec.ko 2024-03-21T09:59:04.148 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/i2c/busses/i2c-sis96x.ko 2024-03-21T09:59:04.148 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/i2c/busses/i2c-tiny-usb.ko 2024-03-21T09:59:04.149 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/i2c/busses/i2c-via.ko 2024-03-21T09:59:04.149 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/i2c/busses/i2c-viapro.ko 2024-03-21T09:59:04.149 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/i2c/busses/i2c-viperboard.ko 2024-03-21T09:59:04.149 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/i2c/i2c-dev.ko 2024-03-21T09:59:04.149 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/i2c/i2c-mux.ko 2024-03-21T09:59:04.149 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/i2c/i2c-slave-eeprom.ko 2024-03-21T09:59:04.149 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/i2c/i2c-smbus.ko 2024-03-21T09:59:04.149 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/i2c/i2c-stub.ko 2024-03-21T09:59:04.149 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/infiniband 2024-03-21T09:59:04.149 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/infiniband/core 2024-03-21T09:59:04.149 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/infiniband/core/ib_cm.ko 2024-03-21T09:59:04.149 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/infiniband/core/ib_core.ko 2024-03-21T09:59:04.149 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/infiniband/core/ib_umad.ko 2024-03-21T09:59:04.149 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/infiniband/core/ib_uverbs.ko 2024-03-21T09:59:04.150 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/infiniband/core/iw_cm.ko 2024-03-21T09:59:04.150 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/infiniband/core/rdma_cm.ko 2024-03-21T09:59:04.150 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/infiniband/core/rdma_ucm.ko 2024-03-21T09:59:04.150 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/infiniband/hw 2024-03-21T09:59:04.150 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/infiniband/hw/cxgb4 2024-03-21T09:59:04.150 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/infiniband/hw/cxgb4/iw_cxgb4.ko 2024-03-21T09:59:04.150 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/infiniband/hw/mlx4 2024-03-21T09:59:04.150 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/infiniband/hw/mlx4/mlx4_ib.ko 2024-03-21T09:59:04.150 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/infiniband/hw/mlx5 2024-03-21T09:59:04.150 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/infiniband/hw/mlx5/mlx5_ib.ko 2024-03-21T09:59:04.150 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/infiniband/hw/mthca 2024-03-21T09:59:04.150 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/infiniband/hw/mthca/ib_mthca.ko 2024-03-21T09:59:04.150 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/infiniband/ulp 2024-03-21T09:59:04.151 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/infiniband/ulp/ipoib 2024-03-21T09:59:04.151 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/infiniband/ulp/ipoib/ib_ipoib.ko 2024-03-21T09:59:04.151 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/infiniband/ulp/iser 2024-03-21T09:59:04.151 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/infiniband/ulp/iser/ib_iser.ko 2024-03-21T09:59:04.151 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/infiniband/ulp/isert 2024-03-21T09:59:04.151 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/infiniband/ulp/isert/ib_isert.ko 2024-03-21T09:59:04.151 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/infiniband/ulp/srp 2024-03-21T09:59:04.151 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/infiniband/ulp/srp/ib_srp.ko 2024-03-21T09:59:04.151 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/infiniband/ulp/srpt 2024-03-21T09:59:04.151 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/infiniband/ulp/srpt/ib_srpt.ko 2024-03-21T09:59:04.151 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/input 2024-03-21T09:59:04.151 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/input/gameport 2024-03-21T09:59:04.151 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/input/gameport/emu10k1-gp.ko 2024-03-21T09:59:04.151 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/input/gameport/fm801-gp.ko 2024-03-21T09:59:04.151 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/input/gameport/gameport.ko 2024-03-21T09:59:04.152 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/input/joydev.ko 2024-03-21T09:59:04.152 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/input/keyboard 2024-03-21T09:59:04.152 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/input/keyboard/gpio_keys.ko 2024-03-21T09:59:04.152 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/input/mouse 2024-03-21T09:59:04.152 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/input/mouse/appletouch.ko 2024-03-21T09:59:04.152 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/input/mouse/bcm5974.ko 2024-03-21T09:59:04.152 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/input/mouse/cyapatp.ko 2024-03-21T09:59:04.152 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/input/mouse/elan_i2c.ko 2024-03-21T09:59:04.152 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/input/mouse/sermouse.ko 2024-03-21T09:59:04.152 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/input/mouse/synaptics_i2c.ko 2024-03-21T09:59:04.152 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/input/mouse/synaptics_usb.ko 2024-03-21T09:59:04.152 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/input/mouse/vsxxxaa.ko 2024-03-21T09:59:04.152 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/input/rmi4 2024-03-21T09:59:04.152 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/input/rmi4/rmi_core.ko 2024-03-21T09:59:04.153 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/input/serio 2024-03-21T09:59:04.153 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/input/serio/altera_ps2.ko 2024-03-21T09:59:04.153 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/input/serio/arc_ps2.ko 2024-03-21T09:59:04.153 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/input/serio/hyperv-keyboard.ko 2024-03-21T09:59:04.153 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/input/serio/serio_raw.ko 2024-03-21T09:59:04.153 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/input/sparse-keymap.ko 2024-03-21T09:59:04.153 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/leds 2024-03-21T09:59:04.153 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/leds/leds-blinkm.ko 2024-03-21T09:59:04.153 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/leds/leds-lm3530.ko 2024-03-21T09:59:04.153 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/leds/leds-lp3944.ko 2024-03-21T09:59:04.153 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/leds/leds-ss4200.ko 2024-03-21T09:59:04.153 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/leds/trigger 2024-03-21T09:59:04.153 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/leds/trigger/ledtrig-audio.ko 2024-03-21T09:59:04.153 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/leds/trigger/ledtrig-backlight.ko 2024-03-21T09:59:04.153 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/leds/trigger/ledtrig-camera.ko 2024-03-21T09:59:04.154 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/leds/trigger/ledtrig-default-on.ko 2024-03-21T09:59:04.154 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/leds/trigger/ledtrig-gpio.ko 2024-03-21T09:59:04.154 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/leds/trigger/ledtrig-heartbeat.ko 2024-03-21T09:59:04.154 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/leds/trigger/ledtrig-oneshot.ko 2024-03-21T09:59:04.154 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/leds/trigger/ledtrig-timer.ko 2024-03-21T09:59:04.154 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/leds/trigger/ledtrig-transient.ko 2024-03-21T09:59:04.154 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/md 2024-03-21T09:59:04.154 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/md/bcache 2024-03-21T09:59:04.154 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/md/bcache/bcache.ko 2024-03-21T09:59:04.154 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/md/dm-bio-prison.ko 2024-03-21T09:59:04.154 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/md/dm-cache-smq.ko 2024-03-21T09:59:04.154 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/md/dm-cache.ko 2024-03-21T09:59:04.154 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/md/dm-crypt.ko 2024-03-21T09:59:04.155 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/md/dm-delay.ko 2024-03-21T09:59:04.155 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/md/dm-flakey.ko 2024-03-21T09:59:04.155 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/md/dm-log-userspace.ko 2024-03-21T09:59:04.155 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/md/dm-multipath.ko 2024-03-21T09:59:04.155 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/md/dm-queue-length.ko 2024-03-21T09:59:04.155 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/md/dm-raid.ko 2024-03-21T09:59:04.155 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/md/dm-round-robin.ko 2024-03-21T09:59:04.155 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/md/dm-service-time.ko 2024-03-21T09:59:04.155 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/md/dm-switch.ko 2024-03-21T09:59:04.155 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/md/dm-thin-pool.ko 2024-03-21T09:59:04.155 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/md/dm-verity.ko 2024-03-21T09:59:04.155 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/md/persistent-data 2024-03-21T09:59:04.155 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/md/persistent-data/dm-persistent-data.ko 2024-03-21T09:59:04.155 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/md/raid0.ko 2024-03-21T09:59:04.155 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/md/raid1.ko 2024-03-21T09:59:04.156 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/md/raid10.ko 2024-03-21T09:59:04.156 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/md/raid456.ko 2024-03-21T09:59:04.156 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/message 2024-03-21T09:59:04.156 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/message/fusion 2024-03-21T09:59:04.156 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/message/fusion/mptbase.ko 2024-03-21T09:59:04.156 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/message/fusion/mptctl.ko 2024-03-21T09:59:04.156 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/message/fusion/mptfc.ko 2024-03-21T09:59:04.156 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/message/fusion/mptlan.ko 2024-03-21T09:59:04.156 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/message/fusion/mptsas.ko 2024-03-21T09:59:04.156 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/message/fusion/mptscsih.ko 2024-03-21T09:59:04.156 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/message/fusion/mptspi.ko 2024-03-21T09:59:04.156 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/mfd 2024-03-21T09:59:04.156 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/mfd/lpc_ich.ko 2024-03-21T09:59:04.156 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/mfd/lpc_sch.ko 2024-03-21T09:59:04.157 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/mfd/mfd-core.ko 2024-03-21T09:59:04.157 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/mfd/sm501.ko 2024-03-21T09:59:04.157 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/mfd/viperboard.ko 2024-03-21T09:59:04.157 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/mfd/vx855.ko 2024-03-21T09:59:04.157 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/mfd/wl1273-core.ko 2024-03-21T09:59:04.157 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/misc 2024-03-21T09:59:04.157 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/misc/altera-stapl 2024-03-21T09:59:04.157 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/misc/altera-stapl/altera-stapl.ko 2024-03-21T09:59:04.157 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/misc/apds9802als.ko 2024-03-21T09:59:04.157 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/misc/apds990x.ko 2024-03-21T09:59:04.157 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/misc/bh1770glc.ko 2024-03-21T09:59:04.157 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/misc/cb710 2024-03-21T09:59:04.157 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/misc/cb710/cb710.ko 2024-03-21T09:59:04.157 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/misc/eeprom 2024-03-21T09:59:04.157 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/misc/eeprom/at24.ko 2024-03-21T09:59:04.158 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/misc/eeprom/eeprom_93cx6.ko 2024-03-21T09:59:04.158 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/misc/eeprom/max6875.ko 2024-03-21T09:59:04.158 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/misc/enclosure.ko 2024-03-21T09:59:04.158 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/misc/hpilo.ko 2024-03-21T09:59:04.158 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/misc/isl29003.ko 2024-03-21T09:59:04.158 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/misc/isl29020.ko 2024-03-21T09:59:04.158 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/misc/lis3lv02d 2024-03-21T09:59:04.158 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/misc/lis3lv02d/lis3lv02d.ko 2024-03-21T09:59:04.158 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/misc/lis3lv02d/lis3lv02d_i2c.ko 2024-03-21T09:59:04.158 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/misc/mei 2024-03-21T09:59:04.158 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/misc/mei/mei-me.ko 2024-03-21T09:59:04.158 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/misc/mei/mei-txe.ko 2024-03-21T09:59:04.158 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/misc/mei/mei.ko 2024-03-21T09:59:04.159 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/misc/sgi-gru 2024-03-21T09:59:04.159 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/misc/sgi-gru/gru.ko 2024-03-21T09:59:04.159 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/misc/sgi-xp 2024-03-21T09:59:04.159 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/misc/sgi-xp/xp.ko 2024-03-21T09:59:04.159 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/misc/sgi-xp/xpc.ko 2024-03-21T09:59:04.159 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/misc/sgi-xp/xpnet.ko 2024-03-21T09:59:04.159 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/misc/tifm_7xx1.ko 2024-03-21T09:59:04.159 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/misc/tifm_core.ko 2024-03-21T09:59:04.159 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/misc/tsl2550.ko 2024-03-21T09:59:04.159 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/misc/vmw_balloon.ko 2024-03-21T09:59:04.159 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/misc/vmw_vmci 2024-03-21T09:59:04.159 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/misc/vmw_vmci/vmw_vmci.ko 2024-03-21T09:59:04.159 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net 2024-03-21T09:59:04.160 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/bonding 2024-03-21T09:59:04.160 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/bonding/bonding.ko 2024-03-21T09:59:04.160 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/dsa 2024-03-21T09:59:04.160 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/dsa/b53 2024-03-21T09:59:04.160 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/dsa/b53/b53_common.ko 2024-03-21T09:59:04.160 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/dsa/bcm-sf2.ko 2024-03-21T09:59:04.160 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/dsa/mv88e6060.ko 2024-03-21T09:59:04.160 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/dsa/mv88e6xxx 2024-03-21T09:59:04.160 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/dsa/mv88e6xxx/mv88e6xxx.ko 2024-03-21T09:59:04.160 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/dummy.ko 2024-03-21T09:59:04.160 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/eql.ko 2024-03-21T09:59:04.160 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet 2024-03-21T09:59:04.160 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/3com 2024-03-21T09:59:04.160 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/3com/3c574_cs.ko 2024-03-21T09:59:04.160 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/3com/3c589_cs.ko 2024-03-21T09:59:04.161 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/3com/3c59x.ko 2024-03-21T09:59:04.161 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/3com/typhoon.ko 2024-03-21T09:59:04.161 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/8390 2024-03-21T09:59:04.161 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/8390/8390.ko 2024-03-21T09:59:04.161 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/8390/axnet_cs.ko 2024-03-21T09:59:04.161 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/8390/ne2k-pci.ko 2024-03-21T09:59:04.161 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/8390/pcnet_cs.ko 2024-03-21T09:59:04.161 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/adaptec 2024-03-21T09:59:04.161 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/adaptec/starfire.ko 2024-03-21T09:59:04.161 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/agere 2024-03-21T09:59:04.161 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/agere/et131x.ko 2024-03-21T09:59:04.161 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/alteon 2024-03-21T09:59:04.161 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/alteon/acenic.ko 2024-03-21T09:59:04.161 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/altera 2024-03-21T09:59:04.162 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/altera/altera_tse.ko 2024-03-21T09:59:04.162 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/amd 2024-03-21T09:59:04.162 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/amd/amd8111e.ko 2024-03-21T09:59:04.162 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/amd/nmclan_cs.ko 2024-03-21T09:59:04.162 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/amd/pcnet32.ko 2024-03-21T09:59:04.162 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/atheros 2024-03-21T09:59:04.162 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/atheros/alx 2024-03-21T09:59:04.162 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/atheros/alx/alx.ko 2024-03-21T09:59:04.162 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/atheros/atl1c 2024-03-21T09:59:04.162 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/atheros/atl1c/atl1c.ko 2024-03-21T09:59:04.162 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/atheros/atl1e 2024-03-21T09:59:04.162 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/atheros/atl1e/atl1e.ko 2024-03-21T09:59:04.162 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/atheros/atlx 2024-03-21T09:59:04.162 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/atheros/atlx/atl1.ko 2024-03-21T09:59:04.163 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/atheros/atlx/atl2.ko 2024-03-21T09:59:04.163 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/broadcom 2024-03-21T09:59:04.163 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/broadcom/b44.ko 2024-03-21T09:59:04.163 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/broadcom/bnx2.ko 2024-03-21T09:59:04.163 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/broadcom/bnx2x 2024-03-21T09:59:04.163 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/broadcom/bnx2x/bnx2x.ko 2024-03-21T09:59:04.163 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/broadcom/cnic.ko 2024-03-21T09:59:04.163 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/broadcom/genet 2024-03-21T09:59:04.163 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/broadcom/genet/genet.ko 2024-03-21T09:59:04.163 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/broadcom/tg3.ko 2024-03-21T09:59:04.163 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/brocade 2024-03-21T09:59:04.163 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/brocade/bna 2024-03-21T09:59:04.163 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/brocade/bna/bna.ko 2024-03-21T09:59:04.163 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/chelsio 2024-03-21T09:59:04.163 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/chelsio/cxgb 2024-03-21T09:59:04.164 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/chelsio/cxgb/cxgb.ko 2024-03-21T09:59:04.164 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/chelsio/cxgb3 2024-03-21T09:59:04.164 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/chelsio/cxgb3/cxgb3.ko 2024-03-21T09:59:04.164 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/chelsio/cxgb4 2024-03-21T09:59:04.164 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/chelsio/cxgb4/cxgb4.ko 2024-03-21T09:59:04.164 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/chelsio/cxgb4vf 2024-03-21T09:59:04.164 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/chelsio/cxgb4vf/cxgb4vf.ko 2024-03-21T09:59:04.164 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/chelsio/libcxgb 2024-03-21T09:59:04.164 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/chelsio/libcxgb/libcxgb.ko 2024-03-21T09:59:04.164 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/cisco 2024-03-21T09:59:04.164 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/cisco/enic 2024-03-21T09:59:04.164 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/cisco/enic/enic.ko 2024-03-21T09:59:04.164 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/dec 2024-03-21T09:59:04.164 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/dec/tulip 2024-03-21T09:59:04.165 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/dec/tulip/de2104x.ko 2024-03-21T09:59:04.165 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/dec/tulip/dmfe.ko 2024-03-21T09:59:04.165 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/dec/tulip/tulip.ko 2024-03-21T09:59:04.165 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/dec/tulip/uli526x.ko 2024-03-21T09:59:04.165 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/dec/tulip/winbond-840.ko 2024-03-21T09:59:04.165 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/dec/tulip/xircom_cb.ko 2024-03-21T09:59:04.165 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/dlink 2024-03-21T09:59:04.165 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/dlink/dl2k.ko 2024-03-21T09:59:04.165 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/dlink/sundance.ko 2024-03-21T09:59:04.165 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/dnet.ko 2024-03-21T09:59:04.165 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/emulex 2024-03-21T09:59:04.165 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/emulex/benet 2024-03-21T09:59:04.165 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/emulex/benet/be2net.ko 2024-03-21T09:59:04.165 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/ethoc.ko 2024-03-21T09:59:04.165 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/fealnx.ko 2024-03-21T09:59:04.165 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/intel 2024-03-21T09:59:04.166 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/intel/e100.ko 2024-03-21T09:59:04.166 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/intel/e1000 2024-03-21T09:59:04.166 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/intel/e1000/e1000.ko 2024-03-21T09:59:04.166 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/intel/e1000e 2024-03-21T09:59:04.166 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/intel/e1000e/e1000e.ko 2024-03-21T09:59:04.166 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/intel/fm10k 2024-03-21T09:59:04.166 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/intel/fm10k/fm10k.ko 2024-03-21T09:59:04.166 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/intel/i40e 2024-03-21T09:59:04.166 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/intel/i40e/i40e.ko 2024-03-21T09:59:04.166 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/intel/iavf 2024-03-21T09:59:04.166 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/intel/iavf/iavf.ko 2024-03-21T09:59:04.166 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/intel/igb 2024-03-21T09:59:04.166 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/intel/igb/igb.ko 2024-03-21T09:59:04.167 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/intel/igbvf 2024-03-21T09:59:04.167 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/intel/igbvf/igbvf.ko 2024-03-21T09:59:04.167 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/intel/ixgbe 2024-03-21T09:59:04.167 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/intel/ixgbe/ixgbe.ko 2024-03-21T09:59:04.167 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/intel/ixgbevf 2024-03-21T09:59:04.167 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/intel/ixgbevf/ixgbevf.ko 2024-03-21T09:59:04.167 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/jme.ko 2024-03-21T09:59:04.167 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/marvell 2024-03-21T09:59:04.167 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/marvell/mvmdio.ko 2024-03-21T09:59:04.167 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/marvell/skge.ko 2024-03-21T09:59:04.167 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/marvell/sky2.ko 2024-03-21T09:59:04.167 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/mellanox 2024-03-21T09:59:04.167 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/mellanox/mlx4 2024-03-21T09:59:04.167 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_core.ko 2024-03-21T09:59:04.168 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_en.ko 2024-03-21T09:59:04.168 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/mellanox/mlx5 2024-03-21T09:59:04.168 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/mellanox/mlx5/core 2024-03-21T09:59:04.168 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/mellanox/mlx5/core/mlx5_core.ko 2024-03-21T09:59:04.168 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/micrel 2024-03-21T09:59:04.168 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/micrel/ksz884x.ko 2024-03-21T09:59:04.168 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/myricom 2024-03-21T09:59:04.168 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/myricom/myri10ge 2024-03-21T09:59:04.168 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/myricom/myri10ge/myri10ge.ko 2024-03-21T09:59:04.168 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/natsemi 2024-03-21T09:59:04.168 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/natsemi/natsemi.ko 2024-03-21T09:59:04.169 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/natsemi/ns83820.ko 2024-03-21T09:59:04.169 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/neterion 2024-03-21T09:59:04.169 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/neterion/s2io.ko 2024-03-21T09:59:04.169 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/nvidia 2024-03-21T09:59:04.169 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/nvidia/forcedeth.ko 2024-03-21T09:59:04.169 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/packetengines 2024-03-21T09:59:04.169 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/packetengines/hamachi.ko 2024-03-21T09:59:04.169 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/packetengines/yellowfin.ko 2024-03-21T09:59:04.169 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/qlogic 2024-03-21T09:59:04.169 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/qlogic/netxen 2024-03-21T09:59:04.169 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/qlogic/netxen/netxen_nic.ko 2024-03-21T09:59:04.169 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/qlogic/qla3xxx.ko 2024-03-21T09:59:04.170 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/qlogic/qlcnic 2024-03-21T09:59:04.170 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/qlogic/qlcnic/qlcnic.ko 2024-03-21T09:59:04.170 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/rdc 2024-03-21T09:59:04.170 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/rdc/r6040.ko 2024-03-21T09:59:04.170 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/realtek 2024-03-21T09:59:04.170 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/realtek/8139cp.ko 2024-03-21T09:59:04.170 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/realtek/8139too.ko 2024-03-21T09:59:04.170 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/realtek/atp.ko 2024-03-21T09:59:04.170 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/realtek/r8169.ko 2024-03-21T09:59:04.170 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/rocker 2024-03-21T09:59:04.170 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/rocker/rocker.ko 2024-03-21T09:59:04.170 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/sfc 2024-03-21T09:59:04.171 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/sfc/sfc.ko 2024-03-21T09:59:04.171 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/silan 2024-03-21T09:59:04.171 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/silan/sc92031.ko 2024-03-21T09:59:04.171 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/sis 2024-03-21T09:59:04.171 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/sis/sis190.ko 2024-03-21T09:59:04.171 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/sis/sis900.ko 2024-03-21T09:59:04.171 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/smsc 2024-03-21T09:59:04.171 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/smsc/epic100.ko 2024-03-21T09:59:04.171 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/smsc/smc91c92_cs.ko 2024-03-21T09:59:04.171 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/smsc/smsc911x.ko 2024-03-21T09:59:04.171 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/smsc/smsc9420.ko 2024-03-21T09:59:04.172 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/stmicro 2024-03-21T09:59:04.172 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/stmicro/stmmac 2024-03-21T09:59:04.172 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/stmicro/stmmac/dwmac-intel.ko 2024-03-21T09:59:04.172 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/stmicro/stmmac/stmmac.ko 2024-03-21T09:59:04.172 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/sun 2024-03-21T09:59:04.172 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/sun/cassini.ko 2024-03-21T09:59:04.172 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/sun/niu.ko 2024-03-21T09:59:04.172 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/sun/sungem.ko 2024-03-21T09:59:04.172 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/sun/sunhme.ko 2024-03-21T09:59:04.172 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/tehuti 2024-03-21T09:59:04.172 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/tehuti/tehuti.ko 2024-03-21T09:59:04.172 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/ti 2024-03-21T09:59:04.172 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/ti/tlan.ko 2024-03-21T09:59:04.172 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/via 2024-03-21T09:59:04.173 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/via/via-rhine.ko 2024-03-21T09:59:04.173 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/via/via-velocity.ko 2024-03-21T09:59:04.173 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/wiznet 2024-03-21T09:59:04.173 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/wiznet/w5100.ko 2024-03-21T09:59:04.173 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/wiznet/w5300.ko 2024-03-21T09:59:04.173 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/xircom 2024-03-21T09:59:04.173 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/xircom/xirc2ps_cs.ko 2024-03-21T09:59:04.173 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/geneve.ko 2024-03-21T09:59:04.173 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/hyperv 2024-03-21T09:59:04.173 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/hyperv/hv_netvsc.ko 2024-03-21T09:59:04.174 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ieee802154 2024-03-21T09:59:04.174 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ieee802154/fakelb.ko 2024-03-21T09:59:04.174 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ifb.ko 2024-03-21T09:59:04.174 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ipvlan 2024-03-21T09:59:04.174 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ipvlan/ipvlan.ko 2024-03-21T09:59:04.174 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/macvlan.ko 2024-03-21T09:59:04.174 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/macvtap.ko 2024-03-21T09:59:04.174 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/mdio 2024-03-21T09:59:04.174 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/mdio.ko 2024-03-21T09:59:04.174 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/mdio/mdio-bcm-unimac.ko 2024-03-21T09:59:04.174 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/mdio/mdio-bitbang.ko 2024-03-21T09:59:04.175 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/mdio/mdio-regmap.ko 2024-03-21T09:59:04.175 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/mii.ko 2024-03-21T09:59:04.175 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/net_failover.ko 2024-03-21T09:59:04.175 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/netconsole.ko 2024-03-21T09:59:04.175 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/nlmon.ko 2024-03-21T09:59:04.175 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/pcs 2024-03-21T09:59:04.175 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/pcs/pcs-lynx.ko 2024-03-21T09:59:04.175 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/pcs/pcs_xpcs.ko 2024-03-21T09:59:04.175 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/phy 2024-03-21T09:59:04.175 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/phy/amd.ko 2024-03-21T09:59:04.175 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/phy/bcm-phy-lib.ko 2024-03-21T09:59:04.175 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/phy/bcm-phy-ptp.ko 2024-03-21T09:59:04.176 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/phy/bcm7xxx.ko 2024-03-21T09:59:04.176 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/phy/bcm87xx.ko 2024-03-21T09:59:04.176 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/phy/broadcom.ko 2024-03-21T09:59:04.176 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/phy/cicada.ko 2024-03-21T09:59:04.176 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/phy/davicom.ko 2024-03-21T09:59:04.176 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/phy/dp83640.ko 2024-03-21T09:59:04.176 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/phy/et1011c.ko 2024-03-21T09:59:04.176 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/phy/icplus.ko 2024-03-21T09:59:04.176 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/phy/lxt.ko 2024-03-21T09:59:04.176 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/phy/marvell.ko 2024-03-21T09:59:04.177 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/phy/micrel.ko 2024-03-21T09:59:04.177 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/phy/national.ko 2024-03-21T09:59:04.177 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/phy/phylink.ko 2024-03-21T09:59:04.177 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/phy/qsemi.ko 2024-03-21T09:59:04.177 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/phy/realtek.ko 2024-03-21T09:59:04.177 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/phy/smsc.ko 2024-03-21T09:59:04.177 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/phy/ste10Xp.ko 2024-03-21T09:59:04.177 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/phy/vitesse.ko 2024-03-21T09:59:04.177 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ppp 2024-03-21T09:59:04.177 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ppp/bsd_comp.ko 2024-03-21T09:59:04.177 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ppp/ppp_async.ko 2024-03-21T09:59:04.178 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ppp/ppp_deflate.ko 2024-03-21T09:59:04.178 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ppp/ppp_generic.ko 2024-03-21T09:59:04.178 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ppp/ppp_mppe.ko 2024-03-21T09:59:04.178 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ppp/ppp_synctty.ko 2024-03-21T09:59:04.178 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ppp/pppoe.ko 2024-03-21T09:59:04.178 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ppp/pppox.ko 2024-03-21T09:59:04.178 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ppp/pptp.ko 2024-03-21T09:59:04.178 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/slip 2024-03-21T09:59:04.178 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/slip/slhc.ko 2024-03-21T09:59:04.179 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/slip/slip.ko 2024-03-21T09:59:04.179 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/sungem_phy.ko 2024-03-21T09:59:04.179 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/tap.ko 2024-03-21T09:59:04.179 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/team 2024-03-21T09:59:04.179 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/team/team.ko 2024-03-21T09:59:04.179 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/team/team_mode_activebackup.ko 2024-03-21T09:59:04.179 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/team/team_mode_broadcast.ko 2024-03-21T09:59:04.179 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/team/team_mode_loadbalance.ko 2024-03-21T09:59:04.179 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/team/team_mode_random.ko 2024-03-21T09:59:04.179 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/team/team_mode_roundrobin.ko 2024-03-21T09:59:04.180 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/tun.ko 2024-03-21T09:59:04.180 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/veth.ko 2024-03-21T09:59:04.180 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/virtio_net.ko 2024-03-21T09:59:04.180 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/vmxnet3 2024-03-21T09:59:04.180 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/vmxnet3/vmxnet3.ko 2024-03-21T09:59:04.180 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/vxlan 2024-03-21T09:59:04.180 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/vxlan/vxlan.ko 2024-03-21T09:59:04.180 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/xen-netback 2024-03-21T09:59:04.180 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/xen-netback/xen-netback.ko 2024-03-21T09:59:04.180 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/xen-netfront.ko 2024-03-21T09:59:04.180 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ntb 2024-03-21T09:59:04.181 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ntb/ntb.ko 2024-03-21T09:59:04.181 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/nvme 2024-03-21T09:59:04.181 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/nvme/host 2024-03-21T09:59:04.181 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/nvme/host/nvme-core.ko 2024-03-21T09:59:04.181 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/nvme/host/nvme.ko 2024-03-21T09:59:04.181 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/parport 2024-03-21T09:59:04.181 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/parport/parport.ko 2024-03-21T09:59:04.181 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/parport/parport_cs.ko 2024-03-21T09:59:04.181 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/parport/parport_pc.ko 2024-03-21T09:59:04.181 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/parport/parport_serial.ko 2024-03-21T09:59:04.182 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/pci 2024-03-21T09:59:04.182 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/pci/hotplug 2024-03-21T09:59:04.182 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/pci/hotplug/acpiphp_ibm.ko 2024-03-21T09:59:04.182 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/pci/pcie 2024-03-21T09:59:04.182 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/pci/pcie/aer_inject.ko 2024-03-21T09:59:04.182 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/pci/xen-pcifront.ko 2024-03-21T09:59:04.182 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/pcmcia 2024-03-21T09:59:04.182 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/pcmcia/i82092.ko 2024-03-21T09:59:04.182 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/pcmcia/pd6729.ko 2024-03-21T09:59:04.182 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/pcmcia/yenta_socket.ko 2024-03-21T09:59:04.182 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/pinctrl 2024-03-21T09:59:04.182 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/pinctrl/intel 2024-03-21T09:59:04.182 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/pinctrl/intel/pinctrl-cherryview.ko 2024-03-21T09:59:04.182 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/platform 2024-03-21T09:59:04.183 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/platform/x86 2024-03-21T09:59:04.183 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/platform/x86/acer-wmi.ko 2024-03-21T09:59:04.183 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/platform/x86/acerhdf.ko 2024-03-21T09:59:04.183 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/platform/x86/apple-gmux.ko 2024-03-21T09:59:04.183 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/platform/x86/asus-laptop.ko 2024-03-21T09:59:04.183 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/platform/x86/asus-nb-wmi.ko 2024-03-21T09:59:04.183 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/platform/x86/asus-wmi.ko 2024-03-21T09:59:04.183 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/platform/x86/classmate-laptop.ko 2024-03-21T09:59:04.183 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/platform/x86/eeepc-laptop.ko 2024-03-21T09:59:04.183 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/platform/x86/eeepc-wmi.ko 2024-03-21T09:59:04.183 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/platform/x86/fujitsu-laptop.ko 2024-03-21T09:59:04.183 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/platform/x86/fujitsu-tablet.ko 2024-03-21T09:59:04.183 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/platform/x86/hdaps.ko 2024-03-21T09:59:04.183 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/platform/x86/intel 2024-03-21T09:59:04.184 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/platform/x86/intel/intel-rst.ko 2024-03-21T09:59:04.184 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/platform/x86/intel_ips.ko 2024-03-21T09:59:04.184 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/platform/x86/msi-wmi.ko 2024-03-21T09:59:04.184 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/platform/x86/mxm-wmi.ko 2024-03-21T09:59:04.184 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/platform/x86/panasonic-laptop.ko 2024-03-21T09:59:04.184 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/platform/x86/samsung-laptop.ko 2024-03-21T09:59:04.184 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/platform/x86/samsung-q10.ko 2024-03-21T09:59:04.184 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/platform/x86/topstar-laptop.ko 2024-03-21T09:59:04.184 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/platform/x86/toshiba_bluetooth.ko 2024-03-21T09:59:04.184 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/platform/x86/toshiba_haps.ko 2024-03-21T09:59:04.184 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/platform/x86/wmi-bmof.ko 2024-03-21T09:59:04.184 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/platform/x86/wmi.ko 2024-03-21T09:59:04.184 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/powercap 2024-03-21T09:59:04.184 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/powercap/intel_rapl_common.ko 2024-03-21T09:59:04.184 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/powercap/intel_rapl_msr.ko 2024-03-21T09:59:04.185 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/pps 2024-03-21T09:59:04.185 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/pps/clients 2024-03-21T09:59:04.185 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/pps/clients/pps-gpio.ko 2024-03-21T09:59:04.185 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/pps/clients/pps-ldisc.ko 2024-03-21T09:59:04.185 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/pps/clients/pps_parport.ko 2024-03-21T09:59:04.185 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/pps/pps_core.ko 2024-03-21T09:59:04.185 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ptp 2024-03-21T09:59:04.185 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ptp/ptp.ko 2024-03-21T09:59:04.185 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ptp/ptp_kvm.ko 2024-03-21T09:59:04.185 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/rtc 2024-03-21T09:59:04.185 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/rtc/rtc-bq32k.ko 2024-03-21T09:59:04.185 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/rtc/rtc-ds1286.ko 2024-03-21T09:59:04.185 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/rtc/rtc-ds1307.ko 2024-03-21T09:59:04.185 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/rtc/rtc-ds1374.ko 2024-03-21T09:59:04.186 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/rtc/rtc-ds1511.ko 2024-03-21T09:59:04.186 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/rtc/rtc-ds1553.ko 2024-03-21T09:59:04.186 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/rtc/rtc-ds1672.ko 2024-03-21T09:59:04.186 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/rtc/rtc-ds1742.ko 2024-03-21T09:59:04.186 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/rtc/rtc-ds2404.ko 2024-03-21T09:59:04.186 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/rtc/rtc-ds3232.ko 2024-03-21T09:59:04.186 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/rtc/rtc-em3027.ko 2024-03-21T09:59:04.186 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/rtc/rtc-fm3130.ko 2024-03-21T09:59:04.186 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/rtc/rtc-isl12022.ko 2024-03-21T09:59:04.186 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/rtc/rtc-isl1208.ko 2024-03-21T09:59:04.186 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/rtc/rtc-m41t80.ko 2024-03-21T09:59:04.186 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/rtc/rtc-m48t35.ko 2024-03-21T09:59:04.186 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/rtc/rtc-m48t59.ko 2024-03-21T09:59:04.186 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/rtc/rtc-max6900.ko 2024-03-21T09:59:04.187 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/rtc/rtc-msm6242.ko 2024-03-21T09:59:04.187 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/rtc/rtc-pcf2127.ko 2024-03-21T09:59:04.187 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/rtc/rtc-pcf85063.ko 2024-03-21T09:59:04.187 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/rtc/rtc-pcf8523.ko 2024-03-21T09:59:04.187 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/rtc/rtc-pcf8563.ko 2024-03-21T09:59:04.187 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/rtc/rtc-pcf8583.ko 2024-03-21T09:59:04.187 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/rtc/rtc-rp5c01.ko 2024-03-21T09:59:04.187 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/rtc/rtc-rs5c372.ko 2024-03-21T09:59:04.187 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/rtc/rtc-rv3029c2.ko 2024-03-21T09:59:04.187 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/rtc/rtc-rx8025.ko 2024-03-21T09:59:04.187 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/rtc/rtc-rx8581.ko 2024-03-21T09:59:04.187 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/rtc/rtc-stk17ta8.ko 2024-03-21T09:59:04.187 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/rtc/rtc-x1205.ko 2024-03-21T09:59:04.187 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi 2024-03-21T09:59:04.187 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/3w-9xxx.ko 2024-03-21T09:59:04.188 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/3w-sas.ko 2024-03-21T09:59:04.188 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/3w-xxxx.ko 2024-03-21T09:59:04.189 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/BusLogic.ko 2024-03-21T09:59:04.189 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/a100u2w.ko 2024-03-21T09:59:04.189 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/aacraid 2024-03-21T09:59:04.189 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/aacraid/aacraid.ko 2024-03-21T09:59:04.189 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/advansys.ko 2024-03-21T09:59:04.189 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/aic7xxx 2024-03-21T09:59:04.189 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/aic7xxx/aic79xx.ko 2024-03-21T09:59:04.190 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/aic7xxx/aic7xxx.ko 2024-03-21T09:59:04.190 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/am53c974.ko 2024-03-21T09:59:04.190 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/arcmsr 2024-03-21T09:59:04.190 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/arcmsr/arcmsr.ko 2024-03-21T09:59:04.190 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/atp870u.ko 2024-03-21T09:59:04.190 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/be2iscsi 2024-03-21T09:59:04.190 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/be2iscsi/be2iscsi.ko 2024-03-21T09:59:04.190 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/bfa 2024-03-21T09:59:04.190 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/bfa/bfa.ko 2024-03-21T09:59:04.190 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/bnx2fc 2024-03-21T09:59:04.190 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/bnx2fc/bnx2fc.ko 2024-03-21T09:59:04.190 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/bnx2i 2024-03-21T09:59:04.190 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/bnx2i/bnx2i.ko 2024-03-21T09:59:04.191 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/ch.ko 2024-03-21T09:59:04.191 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/csiostor 2024-03-21T09:59:04.191 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/csiostor/csiostor.ko 2024-03-21T09:59:04.191 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/cxgbi 2024-03-21T09:59:04.191 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/cxgbi/cxgb3i 2024-03-21T09:59:04.191 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/cxgbi/cxgb3i/cxgb3i.ko 2024-03-21T09:59:04.191 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/cxgbi/cxgb4i 2024-03-21T09:59:04.191 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/cxgbi/cxgb4i/cxgb4i.ko 2024-03-21T09:59:04.191 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/cxgbi/libcxgbi.ko 2024-03-21T09:59:04.191 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/dc395x.ko 2024-03-21T09:59:04.191 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/device_handler 2024-03-21T09:59:04.191 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/device_handler/scsi_dh_alua.ko 2024-03-21T09:59:04.191 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/device_handler/scsi_dh_emc.ko 2024-03-21T09:59:04.191 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/device_handler/scsi_dh_hp_sw.ko 2024-03-21T09:59:04.191 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/device_handler/scsi_dh_rdac.ko 2024-03-21T09:59:04.192 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/dmx3191d.ko 2024-03-21T09:59:04.192 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/esas2r 2024-03-21T09:59:04.192 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/esas2r/esas2r.ko 2024-03-21T09:59:04.192 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/esp_scsi.ko 2024-03-21T09:59:04.192 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/fcoe 2024-03-21T09:59:04.192 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/fcoe/fcoe.ko 2024-03-21T09:59:04.192 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/fcoe/libfcoe.ko 2024-03-21T09:59:04.192 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/fnic 2024-03-21T09:59:04.192 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/fnic/fnic.ko 2024-03-21T09:59:04.192 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/hpsa.ko 2024-03-21T09:59:04.192 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/hptiop.ko 2024-03-21T09:59:04.192 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/hv_storvsc.ko 2024-03-21T09:59:04.192 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/initio.ko 2024-03-21T09:59:04.192 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/ipr.ko 2024-03-21T09:59:04.193 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/ips.ko 2024-03-21T09:59:04.193 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/isci 2024-03-21T09:59:04.193 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/isci/isci.ko 2024-03-21T09:59:04.193 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/iscsi_boot_sysfs.ko 2024-03-21T09:59:04.193 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/iscsi_tcp.ko 2024-03-21T09:59:04.193 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/libfc 2024-03-21T09:59:04.193 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/libfc/libfc.ko 2024-03-21T09:59:04.193 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/libiscsi.ko 2024-03-21T09:59:04.193 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/libiscsi_tcp.ko 2024-03-21T09:59:04.193 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/libsas 2024-03-21T09:59:04.193 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/libsas/libsas.ko 2024-03-21T09:59:04.193 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/lpfc 2024-03-21T09:59:04.193 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/lpfc/lpfc.ko 2024-03-21T09:59:04.193 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/megaraid 2024-03-21T09:59:04.193 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/megaraid.ko 2024-03-21T09:59:04.194 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/megaraid/megaraid_mbox.ko 2024-03-21T09:59:04.194 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/megaraid/megaraid_mm.ko 2024-03-21T09:59:04.194 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/megaraid/megaraid_sas.ko 2024-03-21T09:59:04.194 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/mpt3sas 2024-03-21T09:59:04.194 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/mpt3sas/mpt3sas.ko 2024-03-21T09:59:04.194 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/mvsas 2024-03-21T09:59:04.194 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/mvsas/mvsas.ko 2024-03-21T09:59:04.194 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/mvumi.ko 2024-03-21T09:59:04.194 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/pm8001 2024-03-21T09:59:04.194 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/pm8001/pm80xx.ko 2024-03-21T09:59:04.194 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/pmcraid.ko 2024-03-21T09:59:04.194 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/qla1280.ko 2024-03-21T09:59:04.194 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/qla2xxx 2024-03-21T09:59:04.194 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/qla2xxx/qla2xxx.ko 2024-03-21T09:59:04.195 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/qla2xxx/tcm_qla2xxx.ko 2024-03-21T09:59:04.195 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/qla4xxx 2024-03-21T09:59:04.195 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/qla4xxx/qla4xxx.ko 2024-03-21T09:59:04.195 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/raid_class.ko 2024-03-21T09:59:04.195 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/scsi_debug.ko 2024-03-21T09:59:04.195 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/scsi_transport_fc.ko 2024-03-21T09:59:04.195 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/scsi_transport_iscsi.ko 2024-03-21T09:59:04.195 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/scsi_transport_sas.ko 2024-03-21T09:59:04.195 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/scsi_transport_spi.ko 2024-03-21T09:59:04.195 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/scsi_transport_srp.ko 2024-03-21T09:59:04.195 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/ses.ko 2024-03-21T09:59:04.195 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/st.ko 2024-03-21T09:59:04.195 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/stex.ko 2024-03-21T09:59:04.195 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/sym53c8xx_2 2024-03-21T09:59:04.196 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/sym53c8xx_2/sym53c8xx.ko 2024-03-21T09:59:04.196 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/virtio_scsi.ko 2024-03-21T09:59:04.196 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/vmw_pvscsi.ko 2024-03-21T09:59:04.196 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/wd719x.ko 2024-03-21T09:59:04.196 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ssb 2024-03-21T09:59:04.196 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ssb/ssb.ko 2024-03-21T09:59:04.196 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/target 2024-03-21T09:59:04.196 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/target/iscsi 2024-03-21T09:59:04.196 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/target/iscsi/iscsi_target_mod.ko 2024-03-21T09:59:04.196 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/target/loopback 2024-03-21T09:59:04.196 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/target/loopback/tcm_loop.ko 2024-03-21T09:59:04.196 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/target/target_core_file.ko 2024-03-21T09:59:04.196 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/target/target_core_iblock.ko 2024-03-21T09:59:04.197 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/target/target_core_mod.ko 2024-03-21T09:59:04.197 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/target/target_core_pscsi.ko 2024-03-21T09:59:04.197 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/target/target_core_user.ko 2024-03-21T09:59:04.197 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/target/tcm_fc 2024-03-21T09:59:04.197 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/target/tcm_fc/tcm_fc.ko 2024-03-21T09:59:04.197 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/thermal 2024-03-21T09:59:04.197 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/thermal/intel 2024-03-21T09:59:04.197 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/thermal/intel/int340x_thermal 2024-03-21T09:59:04.197 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/thermal/intel/int340x_thermal/acpi_thermal_rel.ko 2024-03-21T09:59:04.197 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/thermal/intel/int340x_thermal/int3400_thermal.ko 2024-03-21T09:59:04.197 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/thermal/intel/int340x_thermal/int3401_thermal.ko 2024-03-21T09:59:04.197 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/thermal/intel/int340x_thermal/int3402_thermal.ko 2024-03-21T09:59:04.197 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/thermal/intel/int340x_thermal/int3403_thermal.ko 2024-03-21T09:59:04.197 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/thermal/intel/int340x_thermal/int340x_thermal_zone.ko 2024-03-21T09:59:04.198 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device.ko 2024-03-21T09:59:04.198 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci.ko 2024-03-21T09:59:04.198 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci_legacy.ko 2024-03-21T09:59:04.198 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_mbox.ko 2024-03-21T09:59:04.198 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_power_floor.ko 2024-03-21T09:59:04.198 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rapl.ko 2024-03-21T09:59:04.198 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rfim.ko 2024-03-21T09:59:04.198 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_wt_hint.ko 2024-03-21T09:59:04.198 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_wt_req.ko 2024-03-21T09:59:04.198 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/thermal/intel/intel_soc_dts_iosf.ko 2024-03-21T09:59:04.198 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/thermal/intel/intel_soc_dts_thermal.ko 2024-03-21T09:59:04.198 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/thermal/intel/x86_pkg_temp_thermal.ko 2024-03-21T09:59:04.198 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/tty 2024-03-21T09:59:04.199 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/tty/ipwireless 2024-03-21T09:59:04.199 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/tty/ipwireless/ipwireless.ko 2024-03-21T09:59:04.199 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/tty/n_gsm.ko 2024-03-21T09:59:04.199 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/tty/n_hdlc.ko 2024-03-21T09:59:04.199 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/tty/nozomi.ko 2024-03-21T09:59:04.199 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/tty/serial 2024-03-21T09:59:04.199 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/tty/serial/8250 2024-03-21T09:59:04.199 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/tty/serial/8250/serial_cs.ko 2024-03-21T09:59:04.199 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/tty/serial/arc_uart.ko 2024-03-21T09:59:04.199 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/tty/serial/jsm 2024-03-21T09:59:04.199 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/tty/serial/jsm/jsm.ko 2024-03-21T09:59:04.199 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ufs 2024-03-21T09:59:04.199 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ufs/core 2024-03-21T09:59:04.199 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ufs/core/ufshcd-core.ko 2024-03-21T09:59:04.199 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ufs/host 2024-03-21T09:59:04.200 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ufs/host/ufshcd-pci.ko 2024-03-21T09:59:04.200 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/uio 2024-03-21T09:59:04.200 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/uio/uio.ko 2024-03-21T09:59:04.200 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/uio/uio_aec.ko 2024-03-21T09:59:04.200 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/uio/uio_cif.ko 2024-03-21T09:59:04.200 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/uio/uio_pci_generic.ko 2024-03-21T09:59:04.200 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/uio/uio_sercos3.ko 2024-03-21T09:59:04.200 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb 2024-03-21T09:59:04.200 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/class 2024-03-21T09:59:04.200 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/class/cdc-acm.ko 2024-03-21T09:59:04.200 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/class/cdc-wdm.ko 2024-03-21T09:59:04.200 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/class/usblp.ko 2024-03-21T09:59:04.200 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/class/usbtmc.ko 2024-03-21T09:59:04.200 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/host 2024-03-21T09:59:04.201 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/host/sl811-hcd.ko 2024-03-21T09:59:04.201 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/image 2024-03-21T09:59:04.201 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/image/mdc800.ko 2024-03-21T09:59:04.201 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/image/microtek.ko 2024-03-21T09:59:04.201 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/misc 2024-03-21T09:59:04.201 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/misc/adutux.ko 2024-03-21T09:59:04.201 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/misc/appledisplay.ko 2024-03-21T09:59:04.201 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/misc/emi26.ko 2024-03-21T09:59:04.201 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/misc/emi62.ko 2024-03-21T09:59:04.201 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/misc/ezusb.ko 2024-03-21T09:59:04.201 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/misc/idmouse.ko 2024-03-21T09:59:04.201 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/misc/iowarrior.ko 2024-03-21T09:59:04.201 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/misc/isight_firmware.ko 2024-03-21T09:59:04.201 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/misc/ldusb.ko 2024-03-21T09:59:04.201 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/misc/legousbtower.ko 2024-03-21T09:59:04.202 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/misc/sisusbvga 2024-03-21T09:59:04.202 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/misc/sisusbvga/sisusbvga.ko 2024-03-21T09:59:04.202 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/misc/trancevibrator.ko 2024-03-21T09:59:04.202 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/misc/usb3503.ko 2024-03-21T09:59:04.202 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/misc/usblcd.ko 2024-03-21T09:59:04.202 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/misc/usbsevseg.ko 2024-03-21T09:59:04.202 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/misc/uss720.ko 2024-03-21T09:59:04.202 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/misc/yurex.ko 2024-03-21T09:59:04.202 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/phy 2024-03-21T09:59:04.202 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/phy/phy-generic.ko 2024-03-21T09:59:04.202 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial 2024-03-21T09:59:04.202 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/aircable.ko 2024-03-21T09:59:04.202 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/ark3116.ko 2024-03-21T09:59:04.203 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/belkin_sa.ko 2024-03-21T09:59:04.203 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/ch341.ko 2024-03-21T09:59:04.203 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/cp210x.ko 2024-03-21T09:59:04.203 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/cyberjack.ko 2024-03-21T09:59:04.203 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/cypress_m8.ko 2024-03-21T09:59:04.203 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/digi_acceleport.ko 2024-03-21T09:59:04.203 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/empeg.ko 2024-03-21T09:59:04.203 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/ftdi_sio.ko 2024-03-21T09:59:04.203 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/garmin_gps.ko 2024-03-21T09:59:04.203 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/io_edgeport.ko 2024-03-21T09:59:04.203 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/io_ti.ko 2024-03-21T09:59:04.203 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/ipaq.ko 2024-03-21T09:59:04.203 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/ipw.ko 2024-03-21T09:59:04.203 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/ir-usb.ko 2024-03-21T09:59:04.204 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/iuu_phoenix.ko 2024-03-21T09:59:04.204 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/keyspan.ko 2024-03-21T09:59:04.204 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/keyspan_pda.ko 2024-03-21T09:59:04.204 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/kl5kusb105.ko 2024-03-21T09:59:04.204 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/kobil_sct.ko 2024-03-21T09:59:04.204 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/mct_u232.ko 2024-03-21T09:59:04.204 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/mos7720.ko 2024-03-21T09:59:04.204 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/mos7840.ko 2024-03-21T09:59:04.204 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/navman.ko 2024-03-21T09:59:04.204 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/omninet.ko 2024-03-21T09:59:04.205 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/opticon.ko 2024-03-21T09:59:04.205 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/option.ko 2024-03-21T09:59:04.205 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/oti6858.ko 2024-03-21T09:59:04.205 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/pl2303.ko 2024-03-21T09:59:04.205 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/qcaux.ko 2024-03-21T09:59:04.205 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/qcserial.ko 2024-03-21T09:59:04.205 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/quatech2.ko 2024-03-21T09:59:04.205 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/safe_serial.ko 2024-03-21T09:59:04.205 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/sierra.ko 2024-03-21T09:59:04.205 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/spcp8x5.ko 2024-03-21T09:59:04.205 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/ssu100.ko 2024-03-21T09:59:04.205 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/symbolserial.ko 2024-03-21T09:59:04.206 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/ti_usb_3410_5052.ko 2024-03-21T09:59:04.206 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/usb-serial-simple.ko 2024-03-21T09:59:04.206 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/usb_debug.ko 2024-03-21T09:59:04.206 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/usb_wwan.ko 2024-03-21T09:59:04.206 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/visor.ko 2024-03-21T09:59:04.206 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/whiteheat.ko 2024-03-21T09:59:04.206 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/xsens_mt.ko 2024-03-21T09:59:04.206 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/storage 2024-03-21T09:59:04.206 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/storage/uas.ko 2024-03-21T09:59:04.206 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/storage/ums-alauda.ko 2024-03-21T09:59:04.207 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/storage/ums-cypress.ko 2024-03-21T09:59:04.207 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/storage/ums-datafab.ko 2024-03-21T09:59:04.207 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/storage/ums-eneub6250.ko 2024-03-21T09:59:04.207 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/storage/ums-freecom.ko 2024-03-21T09:59:04.207 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/storage/ums-isd200.ko 2024-03-21T09:59:04.207 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/storage/ums-jumpshot.ko 2024-03-21T09:59:04.207 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/storage/ums-karma.ko 2024-03-21T09:59:04.207 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/storage/ums-onetouch.ko 2024-03-21T09:59:04.207 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/storage/ums-realtek.ko 2024-03-21T09:59:04.207 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/storage/ums-sddr09.ko 2024-03-21T09:59:04.207 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/storage/ums-sddr55.ko 2024-03-21T09:59:04.207 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/storage/ums-usbat.ko 2024-03-21T09:59:04.207 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/storage/usb-storage.ko 2024-03-21T09:59:04.207 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/usbip 2024-03-21T09:59:04.208 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/usbip/usbip-core.ko 2024-03-21T09:59:04.208 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/usbip/usbip-host.ko 2024-03-21T09:59:04.208 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/usbip/vhci-hcd.ko 2024-03-21T09:59:04.208 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/vfio 2024-03-21T09:59:04.208 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/vfio/pci 2024-03-21T09:59:04.208 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/vfio/pci/vfio-pci-core.ko 2024-03-21T09:59:04.208 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/vfio/pci/vfio-pci.ko 2024-03-21T09:59:04.208 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/vfio/vfio.ko 2024-03-21T09:59:04.208 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/vfio/vfio_iommu_type1.ko 2024-03-21T09:59:04.208 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/vhost 2024-03-21T09:59:04.208 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/vhost/vhost.ko 2024-03-21T09:59:04.209 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/vhost/vhost_iotlb.ko 2024-03-21T09:59:04.209 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/vhost/vhost_net.ko 2024-03-21T09:59:04.209 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/vhost/vhost_scsi.ko 2024-03-21T09:59:04.209 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/video 2024-03-21T09:59:04.209 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/video/backlight 2024-03-21T09:59:04.209 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/video/backlight/apple_bl.ko 2024-03-21T09:59:04.209 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/video/backlight/lcd.ko 2024-03-21T09:59:04.209 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/video/backlight/platform_lcd.ko 2024-03-21T09:59:04.209 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/virtio 2024-03-21T09:59:04.209 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/virtio/virtio.ko 2024-03-21T09:59:04.209 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/virtio/virtio_balloon.ko 2024-03-21T09:59:04.209 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/virtio/virtio_mmio.ko 2024-03-21T09:59:04.209 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/virtio/virtio_pci.ko 2024-03-21T09:59:04.210 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/virtio/virtio_pci_legacy_dev.ko 2024-03-21T09:59:04.210 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/virtio/virtio_pci_modern_dev.ko 2024-03-21T09:59:04.210 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/virtio/virtio_ring.ko 2024-03-21T09:59:04.210 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/w1 2024-03-21T09:59:04.210 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/w1/masters 2024-03-21T09:59:04.210 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/w1/masters/ds2482.ko 2024-03-21T09:59:04.210 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/w1/masters/ds2490.ko 2024-03-21T09:59:04.210 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/w1/slaves 2024-03-21T09:59:04.210 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/w1/slaves/w1_ds2406.ko 2024-03-21T09:59:04.210 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/w1/slaves/w1_ds2408.ko 2024-03-21T09:59:04.210 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/w1/slaves/w1_ds2413.ko 2024-03-21T09:59:04.211 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/w1/slaves/w1_ds2423.ko 2024-03-21T09:59:04.211 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/w1/slaves/w1_ds2431.ko 2024-03-21T09:59:04.211 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/w1/slaves/w1_ds2433.ko 2024-03-21T09:59:04.211 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/w1/slaves/w1_ds2780.ko 2024-03-21T09:59:04.211 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/w1/slaves/w1_ds2781.ko 2024-03-21T09:59:04.211 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/w1/slaves/w1_ds28e04.ko 2024-03-21T09:59:04.211 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/w1/slaves/w1_smem.ko 2024-03-21T09:59:04.211 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/w1/slaves/w1_therm.ko 2024-03-21T09:59:04.211 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/w1/wire.ko 2024-03-21T09:59:04.211 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/watchdog 2024-03-21T09:59:04.211 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/watchdog/alim1535_wdt.ko 2024-03-21T09:59:04.211 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/watchdog/alim7101_wdt.ko 2024-03-21T09:59:04.211 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/watchdog/f71808e_wdt.ko 2024-03-21T09:59:04.211 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/watchdog/hpwdt.ko 2024-03-21T09:59:04.212 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/watchdog/i6300esb.ko 2024-03-21T09:59:04.212 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/watchdog/iTCO_vendor_support.ko 2024-03-21T09:59:04.212 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/watchdog/iTCO_wdt.ko 2024-03-21T09:59:04.212 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/watchdog/ib700wdt.ko 2024-03-21T09:59:04.212 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/watchdog/ibmasr.ko 2024-03-21T09:59:04.212 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/watchdog/ie6xx_wdt.ko 2024-03-21T09:59:04.212 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/watchdog/it8712f_wdt.ko 2024-03-21T09:59:04.212 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/watchdog/it87_wdt.ko 2024-03-21T09:59:04.212 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/watchdog/machzwd.ko 2024-03-21T09:59:04.212 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/watchdog/nv_tco.ko 2024-03-21T09:59:04.212 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/watchdog/pcwd_pci.ko 2024-03-21T09:59:04.212 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/watchdog/pcwd_usb.ko 2024-03-21T09:59:04.212 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/watchdog/sbc_fitpc2_wdt.ko 2024-03-21T09:59:04.212 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/watchdog/sch311x_wdt.ko 2024-03-21T09:59:04.213 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/watchdog/softdog.ko 2024-03-21T09:59:04.213 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/watchdog/sp5100_tco.ko 2024-03-21T09:59:04.213 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/watchdog/via_wdt.ko 2024-03-21T09:59:04.213 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/watchdog/w83627hf_wdt.ko 2024-03-21T09:59:04.213 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/watchdog/w83877f_wdt.ko 2024-03-21T09:59:04.213 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/watchdog/w83977f_wdt.ko 2024-03-21T09:59:04.213 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/watchdog/wdt_pci.ko 2024-03-21T09:59:04.213 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/watchdog/xen_wdt.ko 2024-03-21T09:59:04.213 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/xen 2024-03-21T09:59:04.213 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/xen/xen-acpi-processor.ko 2024-03-21T09:59:04.213 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/xen/xen-evtchn.ko 2024-03-21T09:59:04.213 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/xen/xen-gntalloc.ko 2024-03-21T09:59:04.213 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/xen/xen-gntdev.ko 2024-03-21T09:59:04.213 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/xen/xen-pciback 2024-03-21T09:59:04.214 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/xen/xen-pciback/xen-pciback.ko 2024-03-21T09:59:04.214 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/xen/xen-privcmd.ko 2024-03-21T09:59:04.214 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/xen/xenfs 2024-03-21T09:59:04.214 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/xen/xenfs/xenfs.ko 2024-03-21T09:59:04.214 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs 2024-03-21T09:59:04.214 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/9p 2024-03-21T09:59:04.214 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/9p/9p.ko 2024-03-21T09:59:04.214 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/affs 2024-03-21T09:59:04.214 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/affs/affs.ko 2024-03-21T09:59:04.215 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/befs 2024-03-21T09:59:04.215 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/befs/befs.ko 2024-03-21T09:59:04.215 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/binfmt_misc.ko 2024-03-21T09:59:04.215 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/btrfs 2024-03-21T09:59:04.215 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/btrfs/btrfs.ko 2024-03-21T09:59:04.215 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/cachefiles 2024-03-21T09:59:04.215 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/cachefiles/cachefiles.ko 2024-03-21T09:59:04.215 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/ceph 2024-03-21T09:59:04.215 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/ceph/ceph.ko 2024-03-21T09:59:04.215 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/coda 2024-03-21T09:59:04.215 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/coda/coda.ko 2024-03-21T09:59:04.215 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/cramfs 2024-03-21T09:59:04.216 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/cramfs/cramfs.ko 2024-03-21T09:59:04.216 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/dlm 2024-03-21T09:59:04.216 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/dlm/dlm.ko 2024-03-21T09:59:04.216 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/ecryptfs 2024-03-21T09:59:04.216 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/ecryptfs/ecryptfs.ko 2024-03-21T09:59:04.216 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/f2fs 2024-03-21T09:59:04.216 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/f2fs/f2fs.ko 2024-03-21T09:59:04.216 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/fat 2024-03-21T09:59:04.216 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/fat/fat.ko 2024-03-21T09:59:04.217 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/fat/msdos.ko 2024-03-21T09:59:04.217 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/fat/vfat.ko 2024-03-21T09:59:04.217 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/fuse 2024-03-21T09:59:04.217 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/fuse/cuse.ko 2024-03-21T09:59:04.217 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/fuse/fuse.ko 2024-03-21T09:59:04.217 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/gfs2 2024-03-21T09:59:04.217 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/gfs2/gfs2.ko 2024-03-21T09:59:04.217 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/hfs 2024-03-21T09:59:04.217 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/hfs/hfs.ko 2024-03-21T09:59:04.217 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/hfsplus 2024-03-21T09:59:04.217 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/hfsplus/hfsplus.ko 2024-03-21T09:59:04.218 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/isofs 2024-03-21T09:59:04.218 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/isofs/isofs.ko 2024-03-21T09:59:04.218 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/jfs 2024-03-21T09:59:04.218 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/jfs/jfs.ko 2024-03-21T09:59:04.218 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/lockd 2024-03-21T09:59:04.218 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/lockd/lockd.ko 2024-03-21T09:59:04.218 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/minix 2024-03-21T09:59:04.218 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/minix/minix.ko 2024-03-21T09:59:04.218 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/netfs 2024-03-21T09:59:04.219 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/netfs/netfs.ko 2024-03-21T09:59:04.219 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nfs 2024-03-21T09:59:04.219 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nfs/blocklayout 2024-03-21T09:59:04.219 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nfs/blocklayout/blocklayoutdriver.ko 2024-03-21T09:59:04.219 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nfs/filelayout 2024-03-21T09:59:04.219 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nfs/filelayout/nfs_layout_nfsv41_files.ko 2024-03-21T09:59:04.219 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nfs/flexfilelayout 2024-03-21T09:59:04.219 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nfs/flexfilelayout/nfs_layout_flexfiles.ko 2024-03-21T09:59:04.219 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nfs/nfs.ko 2024-03-21T09:59:04.220 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nfs/nfsv3.ko 2024-03-21T09:59:04.220 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nfs/nfsv4.ko 2024-03-21T09:59:04.220 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nfs_common 2024-03-21T09:59:04.220 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nfs_common/grace.ko 2024-03-21T09:59:04.220 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nfs_common/nfs_acl.ko 2024-03-21T09:59:04.220 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nfsd 2024-03-21T09:59:04.220 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nfsd/nfsd.ko 2024-03-21T09:59:04.220 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nilfs2 2024-03-21T09:59:04.220 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nilfs2/nilfs2.ko 2024-03-21T09:59:04.220 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls 2024-03-21T09:59:04.220 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/mac-celtic.ko 2024-03-21T09:59:04.220 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/mac-centeuro.ko 2024-03-21T09:59:04.220 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/mac-croatian.ko 2024-03-21T09:59:04.220 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/mac-cyrillic.ko 2024-03-21T09:59:04.221 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/mac-gaelic.ko 2024-03-21T09:59:04.221 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/mac-greek.ko 2024-03-21T09:59:04.221 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/mac-iceland.ko 2024-03-21T09:59:04.221 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/mac-inuit.ko 2024-03-21T09:59:04.221 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/mac-roman.ko 2024-03-21T09:59:04.221 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/mac-romanian.ko 2024-03-21T09:59:04.221 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/mac-turkish.ko 2024-03-21T09:59:04.221 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_cp1250.ko 2024-03-21T09:59:04.221 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_cp1251.ko 2024-03-21T09:59:04.221 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_cp1255.ko 2024-03-21T09:59:04.221 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_cp737.ko 2024-03-21T09:59:04.221 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_cp775.ko 2024-03-21T09:59:04.221 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_cp850.ko 2024-03-21T09:59:04.222 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_cp852.ko 2024-03-21T09:59:04.222 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_cp855.ko 2024-03-21T09:59:04.222 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_cp857.ko 2024-03-21T09:59:04.222 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_cp860.ko 2024-03-21T09:59:04.222 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_cp861.ko 2024-03-21T09:59:04.222 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_cp862.ko 2024-03-21T09:59:04.222 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_cp863.ko 2024-03-21T09:59:04.222 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_cp864.ko 2024-03-21T09:59:04.222 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_cp865.ko 2024-03-21T09:59:04.222 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_cp866.ko 2024-03-21T09:59:04.222 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_cp869.ko 2024-03-21T09:59:04.222 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_cp874.ko 2024-03-21T09:59:04.222 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_cp932.ko 2024-03-21T09:59:04.222 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_cp936.ko 2024-03-21T09:59:04.223 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_cp949.ko 2024-03-21T09:59:04.223 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_cp950.ko 2024-03-21T09:59:04.223 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_euc-jp.ko 2024-03-21T09:59:04.223 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_iso8859-1.ko 2024-03-21T09:59:04.223 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_iso8859-13.ko 2024-03-21T09:59:04.223 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_iso8859-14.ko 2024-03-21T09:59:04.223 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_iso8859-15.ko 2024-03-21T09:59:04.223 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_iso8859-2.ko 2024-03-21T09:59:04.223 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_iso8859-3.ko 2024-03-21T09:59:04.223 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_iso8859-4.ko 2024-03-21T09:59:04.223 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_iso8859-5.ko 2024-03-21T09:59:04.223 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_iso8859-6.ko 2024-03-21T09:59:04.223 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_iso8859-7.ko 2024-03-21T09:59:04.223 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_iso8859-9.ko 2024-03-21T09:59:04.224 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_koi8-r.ko 2024-03-21T09:59:04.224 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_koi8-ru.ko 2024-03-21T09:59:04.224 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_koi8-u.ko 2024-03-21T09:59:04.224 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_ucs2_utils.ko 2024-03-21T09:59:04.224 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_utf8.ko 2024-03-21T09:59:04.224 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/ocfs2 2024-03-21T09:59:04.224 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/ocfs2/cluster 2024-03-21T09:59:04.224 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/ocfs2/cluster/ocfs2_nodemanager.ko 2024-03-21T09:59:04.224 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/ocfs2/dlm 2024-03-21T09:59:04.224 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/ocfs2/dlm/ocfs2_dlm.ko 2024-03-21T09:59:04.224 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/ocfs2/dlmfs 2024-03-21T09:59:04.224 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/ocfs2/dlmfs/ocfs2_dlmfs.ko 2024-03-21T09:59:04.224 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/ocfs2/ocfs2.ko 2024-03-21T09:59:04.224 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/ocfs2/ocfs2_stack_o2cb.ko 2024-03-21T09:59:04.225 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/ocfs2/ocfs2_stack_user.ko 2024-03-21T09:59:04.225 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/ocfs2/ocfs2_stackglue.ko 2024-03-21T09:59:04.225 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/overlayfs 2024-03-21T09:59:04.225 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/overlayfs/overlay.ko 2024-03-21T09:59:04.225 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/pstore 2024-03-21T09:59:04.225 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/pstore/ramoops.ko 2024-03-21T09:59:04.225 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/reiserfs 2024-03-21T09:59:04.225 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/reiserfs/reiserfs.ko 2024-03-21T09:59:04.225 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/romfs 2024-03-21T09:59:04.225 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/romfs/romfs.ko 2024-03-21T09:59:04.225 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/smb 2024-03-21T09:59:04.225 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/smb/client 2024-03-21T09:59:04.225 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/smb/client/cifs.ko 2024-03-21T09:59:04.226 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/smb/common 2024-03-21T09:59:04.226 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/smb/common/cifs_arc4.ko 2024-03-21T09:59:04.226 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/smb/common/cifs_md4.ko 2024-03-21T09:59:04.226 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/squashfs 2024-03-21T09:59:04.226 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/squashfs/squashfs.ko 2024-03-21T09:59:04.226 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/sysv 2024-03-21T09:59:04.226 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/sysv/sysv.ko 2024-03-21T09:59:04.226 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/udf 2024-03-21T09:59:04.226 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/udf/udf.ko 2024-03-21T09:59:04.226 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/ufs 2024-03-21T09:59:04.226 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/ufs/ufs.ko 2024-03-21T09:59:04.226 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/xfs 2024-03-21T09:59:04.226 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/xfs/xfs.ko 2024-03-21T09:59:04.226 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/kernel 2024-03-21T09:59:04.227 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/kernel/rcu 2024-03-21T09:59:04.227 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/kernel/rcu/rcutorture.ko 2024-03-21T09:59:04.227 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/kernel/torture.ko 2024-03-21T09:59:04.227 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/kernel/trace 2024-03-21T09:59:04.227 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/kernel/trace/ring_buffer_benchmark.ko 2024-03-21T09:59:04.227 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/lib 2024-03-21T09:59:04.227 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/lib/asn1_encoder.ko 2024-03-21T09:59:04.227 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/lib/crc-itu-t.ko 2024-03-21T09:59:04.227 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/lib/crc8.ko 2024-03-21T09:59:04.227 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/lib/crypto 2024-03-21T09:59:04.227 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/lib/crypto/libarc4.ko 2024-03-21T09:59:04.227 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/lib/crypto/libdes.ko 2024-03-21T09:59:04.227 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/lib/libcrc32c.ko 2024-03-21T09:59:04.227 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/lib/lru_cache.ko 2024-03-21T09:59:04.228 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/lib/lz4 2024-03-21T09:59:04.228 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/lib/lz4/lz4_compress.ko 2024-03-21T09:59:04.228 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/lib/lz4/lz4hc_compress.ko 2024-03-21T09:59:04.228 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/lib/math 2024-03-21T09:59:04.228 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/lib/math/cordic.ko 2024-03-21T09:59:04.228 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/lib/raid6 2024-03-21T09:59:04.228 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/lib/raid6/raid6_pq.ko 2024-03-21T09:59:04.228 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/lib/reed_solomon 2024-03-21T09:59:04.228 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/lib/reed_solomon/reed_solomon.ko 2024-03-21T09:59:04.228 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/lib/ts_bm.ko 2024-03-21T09:59:04.228 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/lib/ts_fsm.ko 2024-03-21T09:59:04.228 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/lib/ts_kmp.ko 2024-03-21T09:59:04.228 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/lib/zstd 2024-03-21T09:59:04.228 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/lib/zstd/zstd_compress.ko 2024-03-21T09:59:04.229 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/mm 2024-03-21T09:59:04.229 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/mm/hwpoison-inject.ko 2024-03-21T09:59:04.229 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net 2024-03-21T09:59:04.229 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/6lowpan 2024-03-21T09:59:04.229 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/6lowpan/6lowpan.ko 2024-03-21T09:59:04.229 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/6lowpan/nhc_dest.ko 2024-03-21T09:59:04.229 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/6lowpan/nhc_fragment.ko 2024-03-21T09:59:04.229 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/6lowpan/nhc_hop.ko 2024-03-21T09:59:04.229 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/6lowpan/nhc_ipv6.ko 2024-03-21T09:59:04.229 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/6lowpan/nhc_mobility.ko 2024-03-21T09:59:04.229 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/6lowpan/nhc_routing.ko 2024-03-21T09:59:04.229 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/6lowpan/nhc_udp.ko 2024-03-21T09:59:04.229 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/802 2024-03-21T09:59:04.229 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/802/garp.ko 2024-03-21T09:59:04.230 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/802/mrp.ko 2024-03-21T09:59:04.230 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/802/p8022.ko 2024-03-21T09:59:04.230 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/802/psnap.ko 2024-03-21T09:59:04.230 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/802/stp.ko 2024-03-21T09:59:04.230 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/8021q 2024-03-21T09:59:04.230 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/8021q/8021q.ko 2024-03-21T09:59:04.230 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/9p 2024-03-21T09:59:04.230 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/9p/9pnet.ko 2024-03-21T09:59:04.230 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/9p/9pnet_fd.ko 2024-03-21T09:59:04.230 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/9p/9pnet_rdma.ko 2024-03-21T09:59:04.230 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/9p/9pnet_virtio.ko 2024-03-21T09:59:04.230 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/batman-adv 2024-03-21T09:59:04.230 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/batman-adv/batman-adv.ko 2024-03-21T09:59:04.231 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/bridge 2024-03-21T09:59:04.231 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/bridge/br_netfilter.ko 2024-03-21T09:59:04.231 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/bridge/bridge.ko 2024-03-21T09:59:04.231 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/bridge/netfilter 2024-03-21T09:59:04.231 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/bridge/netfilter/ebt_802_3.ko 2024-03-21T09:59:04.231 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/bridge/netfilter/ebt_among.ko 2024-03-21T09:59:04.231 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/bridge/netfilter/ebt_arp.ko 2024-03-21T09:59:04.231 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/bridge/netfilter/ebt_arpreply.ko 2024-03-21T09:59:04.231 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/bridge/netfilter/ebt_dnat.ko 2024-03-21T09:59:04.231 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/bridge/netfilter/ebt_ip.ko 2024-03-21T09:59:04.231 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/bridge/netfilter/ebt_ip6.ko 2024-03-21T09:59:04.231 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/bridge/netfilter/ebt_limit.ko 2024-03-21T09:59:04.231 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/bridge/netfilter/ebt_log.ko 2024-03-21T09:59:04.231 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/bridge/netfilter/ebt_mark.ko 2024-03-21T09:59:04.231 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/bridge/netfilter/ebt_mark_m.ko 2024-03-21T09:59:04.231 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/bridge/netfilter/ebt_nflog.ko 2024-03-21T09:59:04.232 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/bridge/netfilter/ebt_pkttype.ko 2024-03-21T09:59:04.232 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/bridge/netfilter/ebt_redirect.ko 2024-03-21T09:59:04.232 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/bridge/netfilter/ebt_snat.ko 2024-03-21T09:59:04.232 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/bridge/netfilter/ebt_stp.ko 2024-03-21T09:59:04.232 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/bridge/netfilter/ebt_vlan.ko 2024-03-21T09:59:04.232 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/bridge/netfilter/ebtable_broute.ko 2024-03-21T09:59:04.232 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/bridge/netfilter/ebtable_filter.ko 2024-03-21T09:59:04.232 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/bridge/netfilter/ebtable_nat.ko 2024-03-21T09:59:04.232 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/bridge/netfilter/ebtables.ko 2024-03-21T09:59:04.232 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/bridge/netfilter/nf_conntrack_bridge.ko 2024-03-21T09:59:04.232 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/bridge/netfilter/nft_meta_bridge.ko 2024-03-21T09:59:04.232 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/bridge/netfilter/nft_reject_bridge.ko 2024-03-21T09:59:04.232 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ceph 2024-03-21T09:59:04.232 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ceph/libceph.ko 2024-03-21T09:59:04.232 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/core 2024-03-21T09:59:04.233 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/core/failover.ko 2024-03-21T09:59:04.233 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/core/pktgen.ko 2024-03-21T09:59:04.233 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/dccp 2024-03-21T09:59:04.233 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/dccp/dccp.ko 2024-03-21T09:59:04.233 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/dccp/dccp_diag.ko 2024-03-21T09:59:04.233 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/dccp/dccp_ipv4.ko 2024-03-21T09:59:04.233 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/dccp/dccp_ipv6.ko 2024-03-21T09:59:04.233 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/dns_resolver 2024-03-21T09:59:04.233 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/dns_resolver/dns_resolver.ko 2024-03-21T09:59:04.233 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/dsa 2024-03-21T09:59:04.233 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/dsa/dsa_core.ko 2024-03-21T09:59:04.233 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/dsa/tag_brcm.ko 2024-03-21T09:59:04.233 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/dsa/tag_dsa.ko 2024-03-21T09:59:04.233 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/dsa/tag_none.ko 2024-03-21T09:59:04.233 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/dsa/tag_trailer.ko 2024-03-21T09:59:04.234 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ieee802154 2024-03-21T09:59:04.234 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ieee802154/6lowpan 2024-03-21T09:59:04.234 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ieee802154/6lowpan/ieee802154_6lowpan.ko 2024-03-21T09:59:04.234 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ieee802154/ieee802154.ko 2024-03-21T09:59:04.234 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ieee802154/ieee802154_socket.ko 2024-03-21T09:59:04.234 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4 2024-03-21T09:59:04.234 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/ah4.ko 2024-03-21T09:59:04.234 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/esp4.ko 2024-03-21T09:59:04.234 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/fou.ko 2024-03-21T09:59:04.234 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/gre.ko 2024-03-21T09:59:04.234 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/inet_diag.ko 2024-03-21T09:59:04.234 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/ip_gre.ko 2024-03-21T09:59:04.234 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/ip_tunnel.ko 2024-03-21T09:59:04.234 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/ip_vti.ko 2024-03-21T09:59:04.235 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/ipcomp.ko 2024-03-21T09:59:04.235 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/ipip.ko 2024-03-21T09:59:04.235 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/netfilter 2024-03-21T09:59:04.235 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/netfilter/arp_tables.ko 2024-03-21T09:59:04.235 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/netfilter/arpt_mangle.ko 2024-03-21T09:59:04.235 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/netfilter/arptable_filter.ko 2024-03-21T09:59:04.235 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/netfilter/ipt_ECN.ko 2024-03-21T09:59:04.235 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/netfilter/ipt_SYNPROXY.ko 2024-03-21T09:59:04.235 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/netfilter/ipt_ah.ko 2024-03-21T09:59:04.235 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/netfilter/ipt_rpfilter.ko 2024-03-21T09:59:04.235 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/netfilter/iptable_mangle.ko 2024-03-21T09:59:04.235 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/netfilter/iptable_nat.ko 2024-03-21T09:59:04.235 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/netfilter/iptable_raw.ko 2024-03-21T09:59:04.235 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/netfilter/iptable_security.ko 2024-03-21T09:59:04.236 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/netfilter/nf_defrag_ipv4.ko 2024-03-21T09:59:04.236 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/netfilter/nf_dup_ipv4.ko 2024-03-21T09:59:04.236 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/netfilter/nf_nat_h323.ko 2024-03-21T09:59:04.236 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/netfilter/nf_nat_pptp.ko 2024-03-21T09:59:04.236 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/netfilter/nf_nat_snmp_basic.ko 2024-03-21T09:59:04.236 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/netfilter/nf_socket_ipv4.ko 2024-03-21T09:59:04.236 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/netfilter/nf_tproxy_ipv4.ko 2024-03-21T09:59:04.236 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/netfilter/nft_dup_ipv4.ko 2024-03-21T09:59:04.236 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/netfilter/nft_fib_ipv4.ko 2024-03-21T09:59:04.236 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/netfilter/nft_reject_ipv4.ko 2024-03-21T09:59:04.236 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/tcp_bic.ko 2024-03-21T09:59:04.236 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/tcp_dctcp.ko 2024-03-21T09:59:04.236 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/tcp_diag.ko 2024-03-21T09:59:04.236 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/tcp_highspeed.ko 2024-03-21T09:59:04.236 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/tcp_htcp.ko 2024-03-21T09:59:04.237 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/tcp_hybla.ko 2024-03-21T09:59:04.237 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/tcp_illinois.ko 2024-03-21T09:59:04.237 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/tcp_lp.ko 2024-03-21T09:59:04.237 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/tcp_scalable.ko 2024-03-21T09:59:04.237 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/tcp_vegas.ko 2024-03-21T09:59:04.237 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/tcp_veno.ko 2024-03-21T09:59:04.237 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/tcp_westwood.ko 2024-03-21T09:59:04.237 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/tcp_yeah.ko 2024-03-21T09:59:04.237 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/tunnel4.ko 2024-03-21T09:59:04.237 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/udp_diag.ko 2024-03-21T09:59:04.237 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/udp_tunnel.ko 2024-03-21T09:59:04.237 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/xfrm4_tunnel.ko 2024-03-21T09:59:04.237 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv6 2024-03-21T09:59:04.237 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv6/ah6.ko 2024-03-21T09:59:04.238 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv6/esp6.ko 2024-03-21T09:59:04.238 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv6/fou6.ko 2024-03-21T09:59:04.238 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv6/ip6_tunnel.ko 2024-03-21T09:59:04.238 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv6/ip6_udp_tunnel.ko 2024-03-21T09:59:04.238 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv6/ip6_vti.ko 2024-03-21T09:59:04.238 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv6/ipcomp6.ko 2024-03-21T09:59:04.238 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv6/netfilter 2024-03-21T09:59:04.238 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv6/netfilter/ip6_tables.ko 2024-03-21T09:59:04.238 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv6/netfilter/ip6t_REJECT.ko 2024-03-21T09:59:04.238 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv6/netfilter/ip6t_SYNPROXY.ko 2024-03-21T09:59:04.238 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv6/netfilter/ip6t_ah.ko 2024-03-21T09:59:04.238 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv6/netfilter/ip6t_eui64.ko 2024-03-21T09:59:04.238 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv6/netfilter/ip6t_frag.ko 2024-03-21T09:59:04.238 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv6/netfilter/ip6t_hbh.ko 2024-03-21T09:59:04.238 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv6/netfilter/ip6t_ipv6header.ko 2024-03-21T09:59:04.239 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv6/netfilter/ip6t_mh.ko 2024-03-21T09:59:04.239 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv6/netfilter/ip6t_rpfilter.ko 2024-03-21T09:59:04.239 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv6/netfilter/ip6t_rt.ko 2024-03-21T09:59:04.239 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv6/netfilter/ip6table_filter.ko 2024-03-21T09:59:04.239 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv6/netfilter/ip6table_mangle.ko 2024-03-21T09:59:04.239 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv6/netfilter/ip6table_nat.ko 2024-03-21T09:59:04.239 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv6/netfilter/ip6table_raw.ko 2024-03-21T09:59:04.239 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv6/netfilter/ip6table_security.ko 2024-03-21T09:59:04.239 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv6/netfilter/nf_defrag_ipv6.ko 2024-03-21T09:59:04.239 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv6/netfilter/nf_dup_ipv6.ko 2024-03-21T09:59:04.239 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv6/netfilter/nf_reject_ipv6.ko 2024-03-21T09:59:04.239 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv6/netfilter/nf_socket_ipv6.ko 2024-03-21T09:59:04.239 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv6/netfilter/nf_tproxy_ipv6.ko 2024-03-21T09:59:04.239 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv6/netfilter/nft_dup_ipv6.ko 2024-03-21T09:59:04.240 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv6/netfilter/nft_fib_ipv6.ko 2024-03-21T09:59:04.240 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv6/netfilter/nft_reject_ipv6.ko 2024-03-21T09:59:04.240 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv6/sit.ko 2024-03-21T09:59:04.240 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv6/tunnel6.ko 2024-03-21T09:59:04.240 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv6/xfrm6_tunnel.ko 2024-03-21T09:59:04.240 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/key 2024-03-21T09:59:04.240 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/key/af_key.ko 2024-03-21T09:59:04.240 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/l2tp 2024-03-21T09:59:04.240 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/l2tp/l2tp_core.ko 2024-03-21T09:59:04.240 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/l2tp/l2tp_debugfs.ko 2024-03-21T09:59:04.241 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/l2tp/l2tp_eth.ko 2024-03-21T09:59:04.241 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/l2tp/l2tp_ip.ko 2024-03-21T09:59:04.241 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/l2tp/l2tp_ip6.ko 2024-03-21T09:59:04.241 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/l2tp/l2tp_netlink.ko 2024-03-21T09:59:04.241 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/l2tp/l2tp_ppp.ko 2024-03-21T09:59:04.241 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/llc 2024-03-21T09:59:04.241 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/llc/llc.ko 2024-03-21T09:59:04.241 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/mac802154 2024-03-21T09:59:04.241 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/mac802154/mac802154.ko 2024-03-21T09:59:04.241 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/mpls 2024-03-21T09:59:04.241 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/mpls/mpls_gso.ko 2024-03-21T09:59:04.241 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter 2024-03-21T09:59:04.241 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/ipset 2024-03-21T09:59:04.241 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/ipset/ip_set.ko 2024-03-21T09:59:04.242 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/ipset/ip_set_bitmap_ip.ko 2024-03-21T09:59:04.242 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/ipset/ip_set_bitmap_ipmac.ko 2024-03-21T09:59:04.242 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/ipset/ip_set_bitmap_port.ko 2024-03-21T09:59:04.242 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/ipset/ip_set_hash_ip.ko 2024-03-21T09:59:04.242 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/ipset/ip_set_hash_ipmark.ko 2024-03-21T09:59:04.242 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/ipset/ip_set_hash_ipport.ko 2024-03-21T09:59:04.242 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/ipset/ip_set_hash_ipportip.ko 2024-03-21T09:59:04.242 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/ipset/ip_set_hash_ipportnet.ko 2024-03-21T09:59:04.242 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/ipset/ip_set_hash_mac.ko 2024-03-21T09:59:04.242 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/ipset/ip_set_hash_net.ko 2024-03-21T09:59:04.243 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/ipset/ip_set_hash_netiface.ko 2024-03-21T09:59:04.243 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/ipset/ip_set_hash_netnet.ko 2024-03-21T09:59:04.243 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/ipset/ip_set_hash_netport.ko 2024-03-21T09:59:04.243 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/ipset/ip_set_hash_netportnet.ko 2024-03-21T09:59:04.243 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/ipset/ip_set_list_set.ko 2024-03-21T09:59:04.243 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/ipvs 2024-03-21T09:59:04.243 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/ipvs/ip_vs.ko 2024-03-21T09:59:04.243 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/ipvs/ip_vs_dh.ko 2024-03-21T09:59:04.243 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/ipvs/ip_vs_fo.ko 2024-03-21T09:59:04.243 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/ipvs/ip_vs_ftp.ko 2024-03-21T09:59:04.243 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/ipvs/ip_vs_lblc.ko 2024-03-21T09:59:04.243 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/ipvs/ip_vs_lblcr.ko 2024-03-21T09:59:04.243 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/ipvs/ip_vs_lc.ko 2024-03-21T09:59:04.244 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/ipvs/ip_vs_nq.ko 2024-03-21T09:59:04.244 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/ipvs/ip_vs_pe_sip.ko 2024-03-21T09:59:04.244 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/ipvs/ip_vs_rr.ko 2024-03-21T09:59:04.244 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/ipvs/ip_vs_sed.ko 2024-03-21T09:59:04.244 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/ipvs/ip_vs_sh.ko 2024-03-21T09:59:04.244 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/ipvs/ip_vs_wlc.ko 2024-03-21T09:59:04.244 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/ipvs/ip_vs_wrr.ko 2024-03-21T09:59:04.244 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nf_conncount.ko 2024-03-21T09:59:04.244 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nf_conntrack.ko 2024-03-21T09:59:04.244 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nf_conntrack_amanda.ko 2024-03-21T09:59:04.244 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nf_conntrack_broadcast.ko 2024-03-21T09:59:04.245 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nf_conntrack_ftp.ko 2024-03-21T09:59:04.245 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nf_conntrack_h323.ko 2024-03-21T09:59:04.245 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nf_conntrack_irc.ko 2024-03-21T09:59:04.245 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nf_conntrack_netbios_ns.ko 2024-03-21T09:59:04.245 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nf_conntrack_netlink.ko 2024-03-21T09:59:04.245 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nf_conntrack_pptp.ko 2024-03-21T09:59:04.245 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nf_conntrack_sane.ko 2024-03-21T09:59:04.245 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nf_conntrack_sip.ko 2024-03-21T09:59:04.245 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nf_conntrack_snmp.ko 2024-03-21T09:59:04.245 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nf_conntrack_tftp.ko 2024-03-21T09:59:04.245 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nf_dup_netdev.ko 2024-03-21T09:59:04.245 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nf_flow_table.ko 2024-03-21T09:59:04.246 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nf_flow_table_inet.ko 2024-03-21T09:59:04.246 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nf_log_syslog.ko 2024-03-21T09:59:04.246 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nf_nat.ko 2024-03-21T09:59:04.246 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nf_nat_amanda.ko 2024-03-21T09:59:04.246 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nf_nat_ftp.ko 2024-03-21T09:59:04.246 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nf_nat_irc.ko 2024-03-21T09:59:04.246 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nf_nat_sip.ko 2024-03-21T09:59:04.246 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nf_nat_tftp.ko 2024-03-21T09:59:04.246 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nf_synproxy_core.ko 2024-03-21T09:59:04.246 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nf_tables.ko 2024-03-21T09:59:04.247 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nfnetlink.ko 2024-03-21T09:59:04.247 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nfnetlink_acct.ko 2024-03-21T09:59:04.247 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nfnetlink_log.ko 2024-03-21T09:59:04.247 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nfnetlink_osf.ko 2024-03-21T09:59:04.247 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nfnetlink_queue.ko 2024-03-21T09:59:04.247 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nft_chain_nat.ko 2024-03-21T09:59:04.247 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nft_compat.ko 2024-03-21T09:59:04.247 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nft_ct.ko 2024-03-21T09:59:04.247 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nft_dup_netdev.ko 2024-03-21T09:59:04.247 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nft_fib.ko 2024-03-21T09:59:04.247 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nft_fwd_netdev.ko 2024-03-21T09:59:04.247 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nft_hash.ko 2024-03-21T09:59:04.248 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nft_limit.ko 2024-03-21T09:59:04.248 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nft_log.ko 2024-03-21T09:59:04.248 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nft_masq.ko 2024-03-21T09:59:04.248 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nft_nat.ko 2024-03-21T09:59:04.248 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nft_numgen.ko 2024-03-21T09:59:04.248 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nft_queue.ko 2024-03-21T09:59:04.248 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nft_quota.ko 2024-03-21T09:59:04.248 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nft_redir.ko 2024-03-21T09:59:04.248 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nft_reject.ko 2024-03-21T09:59:04.248 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nft_reject_inet.ko 2024-03-21T09:59:04.249 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nft_socket.ko 2024-03-21T09:59:04.249 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nft_tunnel.ko 2024-03-21T09:59:04.249 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_AUDIT.ko 2024-03-21T09:59:04.249 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_CHECKSUM.ko 2024-03-21T09:59:04.249 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_CLASSIFY.ko 2024-03-21T09:59:04.249 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_CONNSECMARK.ko 2024-03-21T09:59:04.249 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_CT.ko 2024-03-21T09:59:04.249 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_DSCP.ko 2024-03-21T09:59:04.249 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_HL.ko 2024-03-21T09:59:04.249 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_HMARK.ko 2024-03-21T09:59:04.250 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_IDLETIMER.ko 2024-03-21T09:59:04.250 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_LED.ko 2024-03-21T09:59:04.250 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_LOG.ko 2024-03-21T09:59:04.250 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_MASQUERADE.ko 2024-03-21T09:59:04.250 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_NETMAP.ko 2024-03-21T09:59:04.250 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_NFLOG.ko 2024-03-21T09:59:04.250 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_NFQUEUE.ko 2024-03-21T09:59:04.250 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_RATEEST.ko 2024-03-21T09:59:04.250 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_REDIRECT.ko 2024-03-21T09:59:04.250 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_SECMARK.ko 2024-03-21T09:59:04.251 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_TCPMSS.ko 2024-03-21T09:59:04.251 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_TCPOPTSTRIP.ko 2024-03-21T09:59:04.251 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_TEE.ko 2024-03-21T09:59:04.251 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_TPROXY.ko 2024-03-21T09:59:04.251 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_TRACE.ko 2024-03-21T09:59:04.251 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_addrtype.ko 2024-03-21T09:59:04.251 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_bpf.ko 2024-03-21T09:59:04.251 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_cgroup.ko 2024-03-21T09:59:04.251 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_cluster.ko 2024-03-21T09:59:04.251 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_comment.ko 2024-03-21T09:59:04.252 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_connbytes.ko 2024-03-21T09:59:04.252 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_connlabel.ko 2024-03-21T09:59:04.252 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_connlimit.ko 2024-03-21T09:59:04.252 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_connmark.ko 2024-03-21T09:59:04.252 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_conntrack.ko 2024-03-21T09:59:04.252 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_cpu.ko 2024-03-21T09:59:04.252 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_dccp.ko 2024-03-21T09:59:04.252 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_devgroup.ko 2024-03-21T09:59:04.252 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_dscp.ko 2024-03-21T09:59:04.252 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_ecn.ko 2024-03-21T09:59:04.252 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_esp.ko 2024-03-21T09:59:04.253 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_hashlimit.ko 2024-03-21T09:59:04.253 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_helper.ko 2024-03-21T09:59:04.253 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_hl.ko 2024-03-21T09:59:04.253 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_ipcomp.ko 2024-03-21T09:59:04.253 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_iprange.ko 2024-03-21T09:59:04.253 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_ipvs.ko 2024-03-21T09:59:04.253 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_l2tp.ko 2024-03-21T09:59:04.253 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_length.ko 2024-03-21T09:59:04.253 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_limit.ko 2024-03-21T09:59:04.253 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_mac.ko 2024-03-21T09:59:04.253 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_mark.ko 2024-03-21T09:59:04.254 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_multiport.ko 2024-03-21T09:59:04.254 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_nat.ko 2024-03-21T09:59:04.254 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_nfacct.ko 2024-03-21T09:59:04.254 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_osf.ko 2024-03-21T09:59:04.254 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_owner.ko 2024-03-21T09:59:04.254 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_physdev.ko 2024-03-21T09:59:04.254 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_pkttype.ko 2024-03-21T09:59:04.254 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_policy.ko 2024-03-21T09:59:04.254 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_quota.ko 2024-03-21T09:59:04.254 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_rateest.ko 2024-03-21T09:59:04.254 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_realm.ko 2024-03-21T09:59:04.254 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_recent.ko 2024-03-21T09:59:04.254 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_sctp.ko 2024-03-21T09:59:04.255 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_set.ko 2024-03-21T09:59:04.255 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_socket.ko 2024-03-21T09:59:04.255 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_state.ko 2024-03-21T09:59:04.255 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_statistic.ko 2024-03-21T09:59:04.255 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_string.ko 2024-03-21T09:59:04.255 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_tcpmss.ko 2024-03-21T09:59:04.255 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_time.ko 2024-03-21T09:59:04.255 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_u32.ko 2024-03-21T09:59:04.255 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netlink 2024-03-21T09:59:04.255 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netlink/netlink_diag.ko 2024-03-21T09:59:04.255 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/nsh 2024-03-21T09:59:04.255 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/nsh/nsh.ko 2024-03-21T09:59:04.255 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/openvswitch 2024-03-21T09:59:04.255 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/openvswitch/openvswitch.ko 2024-03-21T09:59:04.256 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/openvswitch/vport-geneve.ko 2024-03-21T09:59:04.256 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/openvswitch/vport-gre.ko 2024-03-21T09:59:04.256 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/openvswitch/vport-vxlan.ko 2024-03-21T09:59:04.256 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/packet 2024-03-21T09:59:04.256 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/packet/af_packet_diag.ko 2024-03-21T09:59:04.256 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/rds 2024-03-21T09:59:04.256 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/rds/rds.ko 2024-03-21T09:59:04.256 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/rds/rds_rdma.ko 2024-03-21T09:59:04.256 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/rds/rds_tcp.ko 2024-03-21T09:59:04.256 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched 2024-03-21T09:59:04.256 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/act_csum.ko 2024-03-21T09:59:04.256 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/act_gact.ko 2024-03-21T09:59:04.256 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/act_mirred.ko 2024-03-21T09:59:04.256 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/act_nat.ko 2024-03-21T09:59:04.257 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/act_pedit.ko 2024-03-21T09:59:04.257 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/act_police.ko 2024-03-21T09:59:04.257 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/act_simple.ko 2024-03-21T09:59:04.257 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/act_skbedit.ko 2024-03-21T09:59:04.257 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/act_vlan.ko 2024-03-21T09:59:04.257 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/cls_basic.ko 2024-03-21T09:59:04.257 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/cls_bpf.ko 2024-03-21T09:59:04.257 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/cls_flow.ko 2024-03-21T09:59:04.257 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/cls_fw.ko 2024-03-21T09:59:04.257 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/cls_route.ko 2024-03-21T09:59:04.257 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/cls_u32.ko 2024-03-21T09:59:04.257 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/em_cmp.ko 2024-03-21T09:59:04.257 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/em_ipset.ko 2024-03-21T09:59:04.257 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/em_meta.ko 2024-03-21T09:59:04.258 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/em_nbyte.ko 2024-03-21T09:59:04.258 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/em_text.ko 2024-03-21T09:59:04.258 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/em_u32.ko 2024-03-21T09:59:04.258 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/sch_choke.ko 2024-03-21T09:59:04.258 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/sch_codel.ko 2024-03-21T09:59:04.258 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/sch_drr.ko 2024-03-21T09:59:04.258 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/sch_fq.ko 2024-03-21T09:59:04.258 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/sch_gred.ko 2024-03-21T09:59:04.258 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/sch_hfsc.ko 2024-03-21T09:59:04.258 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/sch_hhf.ko 2024-03-21T09:59:04.258 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/sch_htb.ko 2024-03-21T09:59:04.258 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/sch_ingress.ko 2024-03-21T09:59:04.258 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/sch_mqprio.ko 2024-03-21T09:59:04.258 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/sch_mqprio_lib.ko 2024-03-21T09:59:04.259 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/sch_multiq.ko 2024-03-21T09:59:04.259 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/sch_netem.ko 2024-03-21T09:59:04.259 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/sch_pie.ko 2024-03-21T09:59:04.259 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/sch_plug.ko 2024-03-21T09:59:04.259 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/sch_prio.ko 2024-03-21T09:59:04.259 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/sch_qfq.ko 2024-03-21T09:59:04.259 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/sch_red.ko 2024-03-21T09:59:04.259 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/sch_sfb.ko 2024-03-21T09:59:04.259 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/sch_sfq.ko 2024-03-21T09:59:04.259 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/sch_tbf.ko 2024-03-21T09:59:04.259 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/sch_teql.ko 2024-03-21T09:59:04.259 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sctp 2024-03-21T09:59:04.259 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sctp/sctp.ko 2024-03-21T09:59:04.259 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sctp/sctp_diag.ko 2024-03-21T09:59:04.259 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sunrpc 2024-03-21T09:59:04.260 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sunrpc/auth_gss 2024-03-21T09:59:04.260 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sunrpc/auth_gss/auth_rpcgss.ko 2024-03-21T09:59:04.260 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sunrpc/auth_gss/rpcsec_gss_krb5.ko 2024-03-21T09:59:04.260 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sunrpc/sunrpc.ko 2024-03-21T09:59:04.260 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sunrpc/xprtrdma 2024-03-21T09:59:04.260 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sunrpc/xprtrdma/rpcrdma.ko 2024-03-21T09:59:04.260 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/tipc 2024-03-21T09:59:04.260 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/tipc/diag.ko 2024-03-21T09:59:04.260 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/tipc/tipc.ko 2024-03-21T09:59:04.260 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/unix 2024-03-21T09:59:04.260 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/unix/unix_diag.ko 2024-03-21T09:59:04.260 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/vmw_vsock 2024-03-21T09:59:04.260 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/vmw_vsock/vmw_vsock_virtio_transport_common.ko 2024-03-21T09:59:04.260 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/vmw_vsock/vmw_vsock_vmci_transport.ko 2024-03-21T09:59:04.261 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/vmw_vsock/vsock.ko 2024-03-21T09:59:04.261 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/vmw_vsock/vsock_diag.ko 2024-03-21T09:59:04.261 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/vmw_vsock/vsock_loopback.ko 2024-03-21T09:59:04.261 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/xfrm 2024-03-21T09:59:04.261 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/xfrm/xfrm_ipcomp.ko 2024-03-21T09:59:04.261 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/security 2024-03-21T09:59:04.261 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/security/keys 2024-03-21T09:59:04.261 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/security/keys/encrypted-keys 2024-03-21T09:59:04.261 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/security/keys/encrypted-keys/encrypted-keys.ko 2024-03-21T09:59:04.261 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/security/keys/trusted-keys 2024-03-21T09:59:04.261 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/security/keys/trusted-keys/trusted.ko 2024-03-21T09:59:04.261 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/virt 2024-03-21T09:59:04.261 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/virt/lib 2024-03-21T09:59:04.261 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/virt/lib/irqbypass.ko 2024-03-21T09:59:04.262 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/modules.alias 2024-03-21T09:59:04.262 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/modules.alias.bin 2024-03-21T09:59:04.262 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/modules.builtin 2024-03-21T09:59:04.262 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/modules.builtin.alias.bin 2024-03-21T09:59:04.262 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/modules.builtin.bin 2024-03-21T09:59:04.262 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/modules.builtin.modinfo 2024-03-21T09:59:04.262 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/modules.dep 2024-03-21T09:59:04.262 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/modules.dep.bin 2024-03-21T09:59:04.262 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/modules.devname 2024-03-21T09:59:04.262 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/modules.order 2024-03-21T09:59:04.262 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/modules.softdep 2024-03-21T09:59:04.262 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/modules.symbols 2024-03-21T09:59:04.262 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/modules.symbols.bin 2024-03-21T09:59:04.262 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/vmlinuz 2024-03-21T09:59:04.266 DEBUG:teuthology.task.kernel:get_image_version: 6.8.0-g5ec3e6e88b2e 2024-03-21T09:59:04.266 DEBUG:teuthology.orchestra.run.smithi157:> rpm --scripts -qp /tmp/kernel.x86_64.rpm 2024-03-21T09:59:04.288 INFO:teuthology.orchestra.run.smithi157.stdout:postinstall scriptlet (using /bin/sh): 2024-03-21T09:59:04.288 INFO:teuthology.orchestra.run.smithi157.stdout:if [ -x /usr/bin/kernel-install ]; then 2024-03-21T09:59:04.288 INFO:teuthology.orchestra.run.smithi157.stdout: /usr/bin/kernel-install add 6.8.0-g5ec3e6e88b2e /lib/modules/6.8.0-g5ec3e6e88b2e/vmlinuz 2024-03-21T09:59:04.288 INFO:teuthology.orchestra.run.smithi157.stdout:fi 2024-03-21T09:59:04.288 INFO:teuthology.orchestra.run.smithi157.stdout:for file in vmlinuz System.map config; do 2024-03-21T09:59:04.288 INFO:teuthology.orchestra.run.smithi157.stdout: if ! cmp --silent "/lib/modules/6.8.0-g5ec3e6e88b2e/${file}" "/boot/${file}-6.8.0-g5ec3e6e88b2e"; then 2024-03-21T09:59:04.289 INFO:teuthology.orchestra.run.smithi157.stdout: cp "/lib/modules/6.8.0-g5ec3e6e88b2e/${file}" "/boot/${file}-6.8.0-g5ec3e6e88b2e" 2024-03-21T09:59:04.289 INFO:teuthology.orchestra.run.smithi157.stdout: fi 2024-03-21T09:59:04.289 INFO:teuthology.orchestra.run.smithi157.stdout:done 2024-03-21T09:59:04.289 INFO:teuthology.orchestra.run.smithi157.stdout:preuninstall scriptlet (using /bin/sh): 2024-03-21T09:59:04.289 INFO:teuthology.orchestra.run.smithi157.stdout:if [ -x /sbin/new-kernel-pkg ]; then 2024-03-21T09:59:04.289 INFO:teuthology.orchestra.run.smithi157.stdout:new-kernel-pkg --remove 6.8.0-g5ec3e6e88b2e --rminitrd --initrdfile=/boot/initramfs-6.8.0-g5ec3e6e88b2e.img 2024-03-21T09:59:04.289 INFO:teuthology.orchestra.run.smithi157.stdout:elif [ -x /usr/bin/kernel-install ]; then 2024-03-21T09:59:04.289 INFO:teuthology.orchestra.run.smithi157.stdout:kernel-install remove 6.8.0-g5ec3e6e88b2e 2024-03-21T09:59:04.289 INFO:teuthology.orchestra.run.smithi157.stdout:fi 2024-03-21T09:59:04.290 INFO:teuthology.orchestra.run.smithi157.stdout:postuninstall scriptlet (using /bin/sh): 2024-03-21T09:59:04.290 INFO:teuthology.orchestra.run.smithi157.stdout:if [ -x /sbin/update-bootloader ]; then 2024-03-21T09:59:04.290 INFO:teuthology.orchestra.run.smithi157.stdout:/sbin/update-bootloader --remove 6.8.0-g5ec3e6e88b2e 2024-03-21T09:59:04.290 INFO:teuthology.orchestra.run.smithi157.stdout:fi 2024-03-21T09:59:04.291 DEBUG:teuthology.orchestra.run.smithi157:> sudo rpm -qi grub2-tools 2024-03-21T09:59:04.365 INFO:teuthology.orchestra.run.smithi157.stdout:Name : grub2-tools 2024-03-21T09:59:04.365 INFO:teuthology.orchestra.run.smithi157.stdout:Epoch : 1 2024-03-21T09:59:04.365 INFO:teuthology.orchestra.run.smithi157.stdout:Version : 2.06 2024-03-21T09:59:04.366 INFO:teuthology.orchestra.run.smithi157.stdout:Release : 68.el9 2024-03-21T09:59:04.366 INFO:teuthology.orchestra.run.smithi157.stdout:Architecture: x86_64 2024-03-21T09:59:04.366 INFO:teuthology.orchestra.run.smithi157.stdout:Install Date: Fri 01 Sep 2023 09:14:30 PM UTC 2024-03-21T09:59:04.366 INFO:teuthology.orchestra.run.smithi157.stdout:Group : Unspecified 2024-03-21T09:59:04.366 INFO:teuthology.orchestra.run.smithi157.stdout:Size : 8264254 2024-03-21T09:59:04.366 INFO:teuthology.orchestra.run.smithi157.stdout:License : GPLv3+ 2024-03-21T09:59:04.366 INFO:teuthology.orchestra.run.smithi157.stdout:Signature : RSA/SHA256, Mon 07 Aug 2023 01:57:16 PM UTC, Key ID 05b555b38483c65d 2024-03-21T09:59:04.366 INFO:teuthology.orchestra.run.smithi157.stdout:Source RPM : grub2-2.06-68.el9.src.rpm 2024-03-21T09:59:04.366 INFO:teuthology.orchestra.run.smithi157.stdout:Build Date : Wed 02 Aug 2023 03:15:48 PM UTC 2024-03-21T09:59:04.366 INFO:teuthology.orchestra.run.smithi157.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-03-21T09:59:04.366 INFO:teuthology.orchestra.run.smithi157.stdout:Packager : builder@centos.org 2024-03-21T09:59:04.366 INFO:teuthology.orchestra.run.smithi157.stdout:Vendor : CentOS 2024-03-21T09:59:04.366 INFO:teuthology.orchestra.run.smithi157.stdout:URL : http://www.gnu.org/software/grub/ 2024-03-21T09:59:04.366 INFO:teuthology.orchestra.run.smithi157.stdout:Summary : Support tools for GRUB. 2024-03-21T09:59:04.367 INFO:teuthology.orchestra.run.smithi157.stdout:Description : 2024-03-21T09:59:04.367 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-03-21T09:59:04.367 INFO:teuthology.orchestra.run.smithi157.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-03-21T09:59:04.367 INFO:teuthology.orchestra.run.smithi157.stdout:customizable bootloader with modular architecture. It supports a rich 2024-03-21T09:59:04.367 INFO:teuthology.orchestra.run.smithi157.stdout:variety of kernel formats, file systems, computer architectures and 2024-03-21T09:59:04.367 INFO:teuthology.orchestra.run.smithi157.stdout:hardware devices. 2024-03-21T09:59:04.367 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-03-21T09:59:04.367 INFO:teuthology.orchestra.run.smithi157.stdout:This subpackage provides tools for support of all platforms. 2024-03-21T09:59:04.368 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-03-21T09:59:04.368 INFO:teuthology.task.kernel:Updating grub on smithi157 to boot 6.8.0-g5ec3e6e88b2e 2024-03-21T09:59:04.369 DEBUG:teuthology.orchestra.run.smithi157:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-03-21T09:59:04.988 INFO:teuthology.orchestra.run.smithi157.stderr:Generating grub configuration file ... 2024-03-21T09:59:06.336 INFO:teuthology.orchestra.run.smithi157.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-03-21T09:59:06.369 INFO:teuthology.orchestra.run.smithi157.stderr:done 2024-03-21T09:59:06.371 DEBUG:teuthology.orchestra.run.smithi157:> mktemp 2024-03-21T09:59:06.386 INFO:teuthology.orchestra.run.smithi157.stdout:/tmp/tmp.MQ0hWdEWRf 2024-03-21T09:59:06.386 DEBUG:teuthology.orchestra.run.smithi157:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.MQ0hWdEWRf 2024-03-21T09:59:06.455 DEBUG:teuthology.orchestra.run.smithi157:> sudo chmod 0666 /tmp/tmp.MQ0hWdEWRf 2024-03-21T09:59:06.598 DEBUG:teuthology.orchestra.remote:smithi157:/tmp/tmp.MQ0hWdEWRf is 6KB 2024-03-21T09:59:06.646 DEBUG:teuthology.orchestra.run.smithi157:> rm -fr /tmp/tmp.MQ0hWdEWRf 2024-03-21T09:59:06.661 DEBUG:teuthology.orchestra.run.smithi157:> sudo /bin/ls /boot/loader/entries || true 2024-03-21T09:59:06.734 INFO:teuthology.orchestra.run.smithi157.stdout:15949e3c88704300882482eddfd7baae-0-rescue.conf 2024-03-21T09:59:06.734 INFO:teuthology.orchestra.run.smithi157.stdout:15949e3c88704300882482eddfd7baae-5.14.0-239.el9.x86_64.conf 2024-03-21T09:59:06.734 INFO:teuthology.orchestra.run.smithi157.stdout:15949e3c88704300882482eddfd7baae-5.14.0-361.el9.x86_64.conf 2024-03-21T09:59:06.734 INFO:teuthology.orchestra.run.smithi157.stdout:15949e3c88704300882482eddfd7baae-6.8.0-g5ec3e6e88b2e.conf 2024-03-21T09:59:06.735 DEBUG:teuthology.orchestra.run.smithi157:> sudo grub2-set-default 15949e3c88704300882482eddfd7baae-6.8.0-g5ec3e6e88b2e 2024-03-21T09:59:06.900 DEBUG:teuthology.orchestra.run.smithi157:> sudo shutdown -r now 2024-03-21T09:59:20.144 INFO:teuthology.task.kernel:Checking client mon.b for new kernel version... 2024-03-21T09:59:20.145 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi100.front.sepia.ceph.com' 2024-03-21T09:59:20.146 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi100.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-21T09:59:30.818 INFO:teuthology.task.kernel:Checking client mon.a for new kernel version... 2024-03-21T09:59:30.819 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi047.front.sepia.ceph.com' 2024-03-21T09:59:30.819 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi047.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-21T09:59:36.907 INFO:teuthology.task.kernel:Checking client mon.c for new kernel version... 2024-03-21T09:59:36.907 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi157.front.sepia.ceph.com' 2024-03-21T09:59:36.908 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi157.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-21T09:59:38.526 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.100 2024-03-21T09:59:47.536 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi100.front.sepia.ceph.com' 2024-03-21T09:59:47.537 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi100.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-21T09:59:49.278 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.47 2024-03-21T09:59:50.594 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.100 2024-03-21T09:59:55.422 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.157 2024-03-21T09:59:58.282 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi047.front.sepia.ceph.com' 2024-03-21T09:59:58.283 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi047.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-21T10:00:01.342 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.47 2024-03-21T10:00:02.596 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi100.front.sepia.ceph.com' 2024-03-21T10:00:02.597 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi100.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-21T10:00:04.425 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi157.front.sepia.ceph.com' 2024-03-21T10:00:04.426 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi157.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-21T10:00:07.486 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.157 2024-03-21T10:00:13.348 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi047.front.sepia.ceph.com' 2024-03-21T10:00:13.349 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi047.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-21T10:00:19.490 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi157.front.sepia.ceph.com' 2024-03-21T10:00:19.491 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi157.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-21T10:00:21.022 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.100 2024-03-21T10:00:22.558 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.157 2024-03-21T10:00:36.034 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi100.front.sepia.ceph.com' 2024-03-21T10:00:36.035 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi100.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-21T10:00:36.418 DEBUG:teuthology.orchestra.run.smithi100:> true 2024-03-21T10:00:37.028 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi100.front.sepia.ceph.com' 2024-03-21T10:00:37.028 INFO:teuthology.task.kernel:Checking kernel version of mon.b, want "6.8.0-g5ec3e6e88b2e"... 2024-03-21T10:00:37.028 DEBUG:teuthology.orchestra.run.smithi100:> uname -r 2024-03-21T10:00:37.066 INFO:teuthology.orchestra.run.smithi100.stdout:6.8.0-g5ec3e6e88b2e 2024-03-21T10:00:37.066 DEBUG:teuthology.task.kernel:current kernel version is 6.8.0-g5ec3e6e88b2e vs 6.8.0-g5ec3e6e88b2e 2024-03-21T10:00:37.066 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-03-21T10:00:37.067 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-03-21T10:00:37.559 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi157.front.sepia.ceph.com' 2024-03-21T10:00:37.560 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi157.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-21T10:00:38.067 INFO:teuthology.task.kernel:Enabling kdb on mon.b... 2024-03-21T10:00:38.068 DEBUG:teuthology.orchestra.run.smithi100:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-03-21T10:00:38.185 INFO:teuthology.orchestra.run.smithi100.stdout:ttyS1 2024-03-21T10:00:38.229 DEBUG:teuthology.parallel:result is None 2024-03-21T10:00:45.230 DEBUG:teuthology.orchestra.run.smithi047:> true 2024-03-21T10:00:45.889 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi047.front.sepia.ceph.com' 2024-03-21T10:00:45.890 INFO:teuthology.task.kernel:Checking kernel version of mon.a, want "6.8.0-g5ec3e6e88b2e"... 2024-03-21T10:00:45.890 DEBUG:teuthology.orchestra.run.smithi047:> uname -r 2024-03-21T10:00:45.928 INFO:teuthology.orchestra.run.smithi047.stdout:6.8.0-g5ec3e6e88b2e 2024-03-21T10:00:45.928 DEBUG:teuthology.task.kernel:current kernel version is 6.8.0-g5ec3e6e88b2e vs 6.8.0-g5ec3e6e88b2e 2024-03-21T10:00:45.928 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-03-21T10:00:45.928 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-03-21T10:00:46.929 INFO:teuthology.task.kernel:Enabling kdb on mon.a... 2024-03-21T10:00:46.929 DEBUG:teuthology.orchestra.run.smithi047:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-03-21T10:00:47.102 INFO:teuthology.orchestra.run.smithi047.stdout:ttyS1 2024-03-21T10:00:47.155 DEBUG:teuthology.parallel:result is None 2024-03-21T10:00:56.094 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.157 2024-03-21T10:01:14.096 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi157.front.sepia.ceph.com' 2024-03-21T10:01:14.097 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi157.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-21T10:01:14.555 DEBUG:teuthology.orchestra.run.smithi157:> true 2024-03-21T10:01:15.259 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi157.front.sepia.ceph.com' 2024-03-21T10:01:15.259 INFO:teuthology.task.kernel:Checking kernel version of mon.c, want "6.8.0-g5ec3e6e88b2e"... 2024-03-21T10:01:15.259 DEBUG:teuthology.orchestra.run.smithi157:> uname -r 2024-03-21T10:01:15.297 INFO:teuthology.orchestra.run.smithi157.stdout:6.8.0-g5ec3e6e88b2e 2024-03-21T10:01:15.297 DEBUG:teuthology.task.kernel:current kernel version is 6.8.0-g5ec3e6e88b2e vs 6.8.0-g5ec3e6e88b2e 2024-03-21T10:01:15.298 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-03-21T10:01:15.298 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-03-21T10:01:16.299 INFO:teuthology.task.kernel:Enabling kdb on mon.c... 2024-03-21T10:01:16.299 DEBUG:teuthology.orchestra.run.smithi157:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-03-21T10:01:16.433 INFO:teuthology.orchestra.run.smithi157.stdout:ttyS1 2024-03-21T10:01:16.482 DEBUG:teuthology.parallel:result is None 2024-03-21T10:01:16.482 INFO:teuthology.run_tasks:Running task internal.base... 2024-03-21T10:01:16.489 INFO:teuthology.task.internal:Creating test directory... 2024-03-21T10:01:16.489 DEBUG:teuthology.orchestra.run.smithi047:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-03-21T10:01:16.499 DEBUG:teuthology.orchestra.run.smithi100:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-03-21T10:01:16.507 DEBUG:teuthology.orchestra.run.smithi157:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-03-21T10:01:16.556 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-03-21T10:01:16.564 INFO:teuthology.run_tasks:Running task internal.archive... 2024-03-21T10:01:16.569 INFO:teuthology.task.internal:Creating archive directory... 2024-03-21T10:01:16.569 DEBUG:teuthology.orchestra.run.smithi047:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-03-21T10:01:16.575 DEBUG:teuthology.orchestra.run.smithi100:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-03-21T10:01:16.580 DEBUG:teuthology.orchestra.run.smithi157:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-03-21T10:01:16.650 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-03-21T10:01:16.657 INFO:teuthology.task.internal:Enabling coredump saving... 2024-03-21T10:01:16.657 DEBUG:teuthology.orchestra.run.smithi047:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-03-21T10:01:16.665 DEBUG:teuthology.orchestra.run.smithi100:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-03-21T10:01:16.672 DEBUG:teuthology.orchestra.run.smithi157:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-03-21T10:01:16.746 INFO:teuthology.orchestra.run.smithi157.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-03-21T10:01:16.770 INFO:teuthology.orchestra.run.smithi100.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-03-21T10:01:16.772 INFO:teuthology.orchestra.run.smithi047.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-03-21T10:01:16.781 INFO:teuthology.orchestra.run.smithi157.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-03-21T10:01:16.813 INFO:teuthology.orchestra.run.smithi100.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-03-21T10:01:16.814 INFO:teuthology.orchestra.run.smithi047.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-03-21T10:01:16.818 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-03-21T10:01:16.826 INFO:teuthology.task.internal:Configuring sudo... 2024-03-21T10:01:16.826 DEBUG:teuthology.orchestra.run.smithi047:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-03-21T10:01:16.834 DEBUG:teuthology.orchestra.run.smithi100:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-03-21T10:01:16.838 DEBUG:teuthology.orchestra.run.smithi157:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-03-21T10:01:16.927 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-03-21T10:01:16.936 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-03-21T10:01:16.936 DEBUG:teuthology.orchestra.run.smithi047:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-03-21T10:01:16.943 DEBUG:teuthology.orchestra.run.smithi100:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-03-21T10:01:16.950 DEBUG:teuthology.orchestra.run.smithi157:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-03-21T10:01:16.998 DEBUG:teuthology.orchestra.run.smithi047:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-03-21T10:01:17.083 DEBUG:teuthology.orchestra.run.smithi047:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-03-21T10:01:17.232 DEBUG:teuthology.orchestra.run.smithi047:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-03-21T10:01:17.320 DEBUG:teuthology.orchestra.run.smithi047:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-03-21T10:01:17.424 DEBUG:teuthology.orchestra.run.smithi047:> set -ex 2024-03-21T10:01:17.424 DEBUG:teuthology.orchestra.run.smithi047:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-03-21T10:01:17.522 DEBUG:teuthology.orchestra.run.smithi100:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-03-21T10:01:17.608 DEBUG:teuthology.orchestra.run.smithi100:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-03-21T10:01:17.734 DEBUG:teuthology.orchestra.run.smithi100:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-03-21T10:01:17.817 DEBUG:teuthology.orchestra.run.smithi100:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-03-21T10:01:17.914 DEBUG:teuthology.orchestra.run.smithi100:> set -ex 2024-03-21T10:01:17.914 DEBUG:teuthology.orchestra.run.smithi100:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-03-21T10:01:18.006 DEBUG:teuthology.orchestra.run.smithi157:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-03-21T10:01:18.080 DEBUG:teuthology.orchestra.run.smithi157:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-03-21T10:01:18.178 DEBUG:teuthology.orchestra.run.smithi157:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-03-21T10:01:18.245 DEBUG:teuthology.orchestra.run.smithi157:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-03-21T10:01:18.331 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2024-03-21T10:01:18.331 DEBUG:teuthology.orchestra.run.smithi157:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-03-21T10:01:18.394 DEBUG:teuthology.orchestra.run.smithi047:> sudo service rsyslog restart 2024-03-21T10:01:18.402 DEBUG:teuthology.orchestra.run.smithi100:> sudo service rsyslog restart 2024-03-21T10:01:18.409 DEBUG:teuthology.orchestra.run.smithi157:> sudo service rsyslog restart 2024-03-21T10:01:18.520 INFO:teuthology.orchestra.run.smithi047.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-03-21T10:01:18.525 INFO:teuthology.orchestra.run.smithi100.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-03-21T10:01:18.532 INFO:teuthology.orchestra.run.smithi157.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-03-21T10:01:19.102 INFO:teuthology.run_tasks:Running task internal.timer... 2024-03-21T10:01:19.109 INFO:teuthology.task.internal:Starting timer... 2024-03-21T10:01:19.109 INFO:teuthology.run_tasks:Running task pcp... 2024-03-21T10:01:19.119 INFO:teuthology.run_tasks:Running task selinux... 2024-03-21T10:01:19.128 DEBUG:teuthology.task:Applying overrides for task selinux: {'allowlist': ['scontext=system_u:system_r:logrotate_t:s0', 'scontext=system_u:system_r:getty_t:s0']} 2024-03-21T10:01:19.128 DEBUG:teuthology.orchestra.run.smithi047:> sudo service auditd rotate 2024-03-21T10:01:19.282 INFO:teuthology.orchestra.run.smithi047.stdout:Rotating logs: 2024-03-21T10:01:19.287 DEBUG:teuthology.orchestra.run.smithi100:> sudo service auditd rotate 2024-03-21T10:01:19.435 INFO:teuthology.orchestra.run.smithi100.stdout:Rotating logs: 2024-03-21T10:01:19.441 DEBUG:teuthology.orchestra.run.smithi157:> sudo service auditd rotate 2024-03-21T10:01:19.551 INFO:teuthology.orchestra.run.smithi157.stdout:Rotating logs: 2024-03-21T10:01:19.556 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-03-21T10:01:19.556 DEBUG:teuthology.orchestra.run.smithi047:> /usr/sbin/getenforce 2024-03-21T10:01:19.609 INFO:teuthology.orchestra.run.smithi047.stdout:Permissive 2024-03-21T10:01:19.610 DEBUG:teuthology.orchestra.run.smithi100:> /usr/sbin/getenforce 2024-03-21T10:01:19.661 INFO:teuthology.orchestra.run.smithi100.stdout:Permissive 2024-03-21T10:01:19.661 DEBUG:teuthology.orchestra.run.smithi157:> /usr/sbin/getenforce 2024-03-21T10:01:19.703 INFO:teuthology.orchestra.run.smithi157.stdout:Permissive 2024-03-21T10:01:19.704 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi047.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi100.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi157.front.sepia.ceph.com': 'permissive'} 2024-03-21T10:01:19.704 DEBUG:teuthology.orchestra.run.smithi047:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-03-21T10:01:19.791 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-21T10:01:19.792 DEBUG:teuthology.orchestra.run.smithi100:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-03-21T10:01:19.877 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-21T10:01:19.878 DEBUG:teuthology.orchestra.run.smithi157:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-03-21T10:01:19.964 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-21T10:01:19.965 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-03-21T10:01:19.965 DEBUG:teuthology.orchestra.run.smithi047:> sudo /usr/sbin/setenforce permissive 2024-03-21T10:01:20.063 DEBUG:teuthology.orchestra.run.smithi100:> sudo /usr/sbin/setenforce permissive 2024-03-21T10:01:20.149 DEBUG:teuthology.orchestra.run.smithi157:> sudo /usr/sbin/setenforce permissive 2024-03-21T10:01:20.251 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-03-21T10:01:20.263 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-03-21T10:01:20.280 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-03-21T10:01:20.281 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi047.front.sepia.ceph.com,smithi100.front.sepia.ceph.com,smithi157.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-03-21T10:07:33.215 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi047.front.sepia.ceph.com'), Remote(name='ubuntu@smithi100.front.sepia.ceph.com'), Remote(name='ubuntu@smithi157.front.sepia.ceph.com')] 2024-03-21T10:07:33.217 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi047.front.sepia.ceph.com' 2024-03-21T10:07:33.219 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi047.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-21T10:07:33.381 DEBUG:teuthology.orchestra.run.smithi047:> true 2024-03-21T10:07:33.543 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi047.front.sepia.ceph.com' 2024-03-21T10:07:33.544 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi100.front.sepia.ceph.com' 2024-03-21T10:07:33.544 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi100.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-21T10:07:33.710 DEBUG:teuthology.orchestra.run.smithi100:> true 2024-03-21T10:07:33.866 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi100.front.sepia.ceph.com' 2024-03-21T10:07:33.866 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi157.front.sepia.ceph.com' 2024-03-21T10:07:33.867 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi157.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-21T10:07:34.005 DEBUG:teuthology.orchestra.run.smithi157:> true 2024-03-21T10:07:34.146 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi157.front.sepia.ceph.com' 2024-03-21T10:07:34.146 INFO:teuthology.run_tasks:Running task clock... 2024-03-21T10:07:34.156 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-03-21T10:07:34.156 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-03-21T10:07:34.157 DEBUG:teuthology.orchestra.run.smithi047:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-03-21T10:07:34.164 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-03-21T10:07:34.164 DEBUG:teuthology.orchestra.run.smithi100:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-03-21T10:07:34.171 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-03-21T10:07:34.171 DEBUG:teuthology.orchestra.run.smithi157:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-03-21T10:07:34.257 INFO:teuthology.orchestra.run.smithi047.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-03-21T10:07:34.265 INFO:teuthology.orchestra.run.smithi100.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-03-21T10:07:34.281 INFO:teuthology.orchestra.run.smithi157.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-03-21T10:07:34.312 INFO:teuthology.orchestra.run.smithi047.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-03-21T10:07:34.317 INFO:teuthology.orchestra.run.smithi100.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-03-21T10:07:34.331 INFO:teuthology.orchestra.run.smithi157.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-03-21T10:07:34.408 INFO:teuthology.orchestra.run.smithi157.stderr:sudo: ntpd: command not found 2024-03-21T10:07:34.409 INFO:teuthology.orchestra.run.smithi100.stderr:sudo: ntpd: command not found 2024-03-21T10:07:34.410 INFO:teuthology.orchestra.run.smithi047.stderr:sudo: ntpd: command not found 2024-03-21T10:07:34.444 INFO:teuthology.orchestra.run.smithi157.stdout:506 Cannot talk to daemon 2024-03-21T10:07:34.453 INFO:teuthology.orchestra.run.smithi100.stdout:506 Cannot talk to daemon 2024-03-21T10:07:34.454 INFO:teuthology.orchestra.run.smithi047.stdout:506 Cannot talk to daemon 2024-03-21T10:07:34.501 INFO:teuthology.orchestra.run.smithi157.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-03-21T10:07:34.505 INFO:teuthology.orchestra.run.smithi100.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-03-21T10:07:34.506 INFO:teuthology.orchestra.run.smithi047.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-03-21T10:07:34.546 INFO:teuthology.orchestra.run.smithi157.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-03-21T10:07:34.559 INFO:teuthology.orchestra.run.smithi100.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-03-21T10:07:34.563 INFO:teuthology.orchestra.run.smithi047.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-03-21T10:07:34.680 INFO:teuthology.orchestra.run.smithi157.stderr:bash: line 1: ntpq: command not found 2024-03-21T10:07:34.688 INFO:teuthology.orchestra.run.smithi100.stderr:bash: line 1: ntpq: command not found 2024-03-21T10:07:34.688 INFO:teuthology.orchestra.run.smithi157.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-03-21T10:07:34.688 INFO:teuthology.orchestra.run.smithi157.stdout:=============================================================================== 2024-03-21T10:07:34.688 INFO:teuthology.orchestra.run.smithi157.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-21T10:07:34.688 INFO:teuthology.orchestra.run.smithi157.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-21T10:07:34.688 INFO:teuthology.orchestra.run.smithi157.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-21T10:07:34.688 INFO:teuthology.orchestra.run.smithi157.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-21T10:07:34.697 INFO:teuthology.orchestra.run.smithi100.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-03-21T10:07:34.697 INFO:teuthology.orchestra.run.smithi100.stdout:=============================================================================== 2024-03-21T10:07:34.698 INFO:teuthology.orchestra.run.smithi100.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-21T10:07:34.698 INFO:teuthology.orchestra.run.smithi100.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-21T10:07:34.698 INFO:teuthology.orchestra.run.smithi100.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-21T10:07:34.698 INFO:teuthology.orchestra.run.smithi100.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-21T10:07:34.704 INFO:teuthology.orchestra.run.smithi047.stderr:bash: line 1: ntpq: command not found 2024-03-21T10:07:34.714 INFO:teuthology.orchestra.run.smithi047.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-03-21T10:07:34.714 INFO:teuthology.orchestra.run.smithi047.stdout:=============================================================================== 2024-03-21T10:07:34.714 INFO:teuthology.orchestra.run.smithi047.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-21T10:07:34.714 INFO:teuthology.orchestra.run.smithi047.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-21T10:07:34.714 INFO:teuthology.orchestra.run.smithi047.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-21T10:07:34.714 INFO:teuthology.orchestra.run.smithi047.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-21T10:07:34.715 INFO:teuthology.run_tasks:Running task install... 2024-03-21T10:07:34.724 DEBUG:teuthology.task.install:project ceph 2024-03-21T10:07:34.724 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': '2edfee68ad7243bed422a049b6f5f58619300f11'}} 2024-03-21T10:07:34.724 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': '2edfee68ad7243bed422a049b6f5f58619300f11'} 2024-03-21T10:07:34.724 INFO:teuthology.task.install:Using flavor: default 2024-03-21T10:07:34.730 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-03-21T10:07:34.730 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']} 2024-03-21T10:07:34.731 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '2edfee68ad7243bed422a049b6f5f58619300f11', 'tag': None, 'wait_for_package': False} 2024-03-21T10:07:34.731 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=2edfee68ad7243bed422a049b6f5f58619300f11 2024-03-21T10:07:34.734 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '2edfee68ad7243bed422a049b6f5f58619300f11', 'tag': None, 'wait_for_package': False} 2024-03-21T10:07:34.734 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=2edfee68ad7243bed422a049b6f5f58619300f11 2024-03-21T10:07:34.735 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '2edfee68ad7243bed422a049b6f5f58619300f11', 'tag': None, 'wait_for_package': False} 2024-03-21T10:07:34.735 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=2edfee68ad7243bed422a049b6f5f58619300f11 2024-03-21T10:07:34.879 INFO:teuthology.task.install.rpm:Pulling from https://2.chacra.ceph.com/r/ceph/wip-vshankar-testing-20240320.053002/2edfee68ad7243bed422a049b6f5f58619300f11/centos/9/flavors/default/ 2024-03-21T10:07:34.879 INFO:teuthology.task.install.rpm:Package version is 19.0.0-2231.g2edfee68 2024-03-21T10:07:34.895 INFO:teuthology.task.install.rpm:Pulling from https://2.chacra.ceph.com/r/ceph/wip-vshankar-testing-20240320.053002/2edfee68ad7243bed422a049b6f5f58619300f11/centos/9/flavors/default/ 2024-03-21T10:07:34.895 INFO:teuthology.task.install.rpm:Package version is 19.0.0-2231.g2edfee68 2024-03-21T10:07:34.904 INFO:teuthology.task.install.rpm:Pulling from https://2.chacra.ceph.com/r/ceph/wip-vshankar-testing-20240320.053002/2edfee68ad7243bed422a049b6f5f58619300f11/centos/9/flavors/default/ 2024-03-21T10:07:34.904 INFO:teuthology.task.install.rpm:Package version is 19.0.0-2231.g2edfee68 2024-03-21T10:07:34.999 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://2.chacra.ceph.com/r/ceph/wip-vshankar-testing-20240320.053002/2edfee68ad7243bed422a049b6f5f58619300f11/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-vshankar-testing-20240320.053002/2edfee68ad7243bed422a049b6f5f58619300f11/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-vshankar-testing-20240320.053002/2edfee68ad7243bed422a049b6f5f58619300f11/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-03-21T10:07:34.999 DEBUG:teuthology.orchestra.run.smithi047:> set -ex 2024-03-21T10:07:34.999 DEBUG:teuthology.orchestra.run.smithi047:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-03-21T10:07:35.003 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://2.chacra.ceph.com/r/ceph/wip-vshankar-testing-20240320.053002/2edfee68ad7243bed422a049b6f5f58619300f11/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-vshankar-testing-20240320.053002/2edfee68ad7243bed422a049b6f5f58619300f11/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-vshankar-testing-20240320.053002/2edfee68ad7243bed422a049b6f5f58619300f11/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-03-21T10:07:35.003 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2024-03-21T10:07:35.003 DEBUG:teuthology.orchestra.run.smithi157:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-03-21T10:07:35.009 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://2.chacra.ceph.com/r/ceph/wip-vshankar-testing-20240320.053002/2edfee68ad7243bed422a049b6f5f58619300f11/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-vshankar-testing-20240320.053002/2edfee68ad7243bed422a049b6f5f58619300f11/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-vshankar-testing-20240320.053002/2edfee68ad7243bed422a049b6f5f58619300f11/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-03-21T10:07:35.009 DEBUG:teuthology.orchestra.run.smithi100:> set -ex 2024-03-21T10:07:35.009 DEBUG:teuthology.orchestra.run.smithi100:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-03-21T10:07:35.092 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-03-21T10:07:35.092 DEBUG:teuthology.orchestra.run.smithi157:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/2edfee68ad7243bed422a049b6f5f58619300f11/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-03-21T10:07:35.106 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-03-21T10:07:35.106 DEBUG:teuthology.orchestra.run.smithi047:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/2edfee68ad7243bed422a049b6f5f58619300f11/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-03-21T10:07:35.108 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-03-21T10:07:35.108 DEBUG:teuthology.orchestra.run.smithi100:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/2edfee68ad7243bed422a049b6f5f58619300f11/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-03-21T10:07:35.224 DEBUG:teuthology.orchestra.run.smithi157:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-03-21T10:07:35.245 DEBUG:teuthology.orchestra.run.smithi100:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-03-21T10:07:35.251 DEBUG:teuthology.orchestra.run.smithi047:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-03-21T10:07:35.336 DEBUG:teuthology.orchestra.run.smithi157:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-03-21T10:07:35.375 DEBUG:teuthology.orchestra.run.smithi100:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-03-21T10:07:35.387 DEBUG:teuthology.orchestra.run.smithi047:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-03-21T10:07:35.406 INFO:teuthology.orchestra.run.smithi157.stdout:check_obsoletes = 1 2024-03-21T10:07:35.410 DEBUG:teuthology.orchestra.run.smithi157:> sudo yum clean all 2024-03-21T10:07:35.463 INFO:teuthology.orchestra.run.smithi100.stdout:check_obsoletes = 1 2024-03-21T10:07:35.468 DEBUG:teuthology.orchestra.run.smithi100:> sudo yum clean all 2024-03-21T10:07:35.485 INFO:teuthology.orchestra.run.smithi047.stdout:check_obsoletes = 1 2024-03-21T10:07:35.490 DEBUG:teuthology.orchestra.run.smithi047:> sudo yum clean all 2024-03-21T10:07:35.741 INFO:teuthology.orchestra.run.smithi157.stdout:74 files removed 2024-03-21T10:07:35.779 DEBUG:teuthology.orchestra.run.smithi157:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-03-21T10:07:35.811 INFO:teuthology.orchestra.run.smithi100.stdout:74 files removed 2024-03-21T10:07:35.849 DEBUG:teuthology.orchestra.run.smithi100:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-03-21T10:07:35.858 INFO:teuthology.orchestra.run.smithi047.stdout:74 files removed 2024-03-21T10:07:35.899 DEBUG:teuthology.orchestra.run.smithi047:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-03-21T10:07:36.648 INFO:teuthology.orchestra.run.smithi157.stdout:ceph packages for x86_64 211 kB/s | 83 kB 00:00 2024-03-21T10:07:36.730 INFO:teuthology.orchestra.run.smithi100.stdout:ceph packages for x86_64 223 kB/s | 83 kB 00:00 2024-03-21T10:07:36.834 INFO:teuthology.orchestra.run.smithi047.stdout:ceph packages for x86_64 202 kB/s | 83 kB 00:00 2024-03-21T10:07:37.165 INFO:teuthology.orchestra.run.smithi157.stdout:ceph noarch packages 24 kB/s | 10 kB 00:00 2024-03-21T10:07:37.223 INFO:teuthology.orchestra.run.smithi100.stdout:ceph noarch packages 25 kB/s | 10 kB 00:00 2024-03-21T10:07:37.351 INFO:teuthology.orchestra.run.smithi047.stdout:ceph noarch packages 24 kB/s | 10 kB 00:00 2024-03-21T10:07:37.649 INFO:teuthology.orchestra.run.smithi157.stdout:ceph source packages 4.6 kB/s | 1.9 kB 00:00 2024-03-21T10:07:37.698 INFO:teuthology.orchestra.run.smithi100.stdout:ceph source packages 4.7 kB/s | 1.9 kB 00:00 2024-03-21T10:07:37.843 INFO:teuthology.orchestra.run.smithi047.stdout:ceph source packages 4.6 kB/s | 1.9 kB 00:00 2024-03-21T10:07:38.258 INFO:teuthology.orchestra.run.smithi100.stdout:CentOS-9 - AppStream 14 MB/s | 6.7 MB 00:00 2024-03-21T10:07:38.428 INFO:teuthology.orchestra.run.smithi157.stdout:CentOS-9 - AppStream 9.5 MB/s | 6.7 MB 00:00 2024-03-21T10:07:38.746 INFO:teuthology.orchestra.run.smithi047.stdout:CentOS-9 - AppStream 8.1 MB/s | 6.7 MB 00:00 2024-03-21T10:07:39.886 INFO:teuthology.orchestra.run.smithi100.stdout:CentOS-9 - BaseOS 5.9 MB/s | 2.0 MB 00:00 2024-03-21T10:07:40.162 INFO:teuthology.orchestra.run.smithi157.stdout:CentOS-9 - BaseOS 4.5 MB/s | 2.0 MB 00:00 2024-03-21T10:07:40.447 INFO:teuthology.orchestra.run.smithi047.stdout:CentOS-9 - BaseOS 5.2 MB/s | 2.0 MB 00:00 2024-03-21T10:07:41.956 INFO:teuthology.orchestra.run.smithi047.stdout:CentOS Stream 9 - BaseOS 7.4 MB/s | 8.0 MB 00:01 2024-03-21T10:07:42.232 INFO:teuthology.orchestra.run.smithi157.stdout:CentOS Stream 9 - BaseOS 4.8 MB/s | 8.0 MB 00:01 2024-03-21T10:07:44.136 INFO:teuthology.orchestra.run.smithi047.stdout:CentOS Stream 9 - AppStream 19 MB/s | 19 MB 00:00 2024-03-21T10:07:49.156 INFO:teuthology.orchestra.run.smithi100.stdout:CentOS Stream 9 - BaseOS 925 kB/s | 8.0 MB 00:08 2024-03-21T10:07:49.403 INFO:teuthology.orchestra.run.smithi047.stdout:CentOS Stream 9 - CRB 4.4 MB/s | 6.1 MB 00:01 2024-03-21T10:07:51.238 INFO:teuthology.orchestra.run.smithi047.stdout:CentOS Stream 9 - Extras packages 33 kB/s | 16 kB 00:00 2024-03-21T10:07:52.045 INFO:teuthology.orchestra.run.smithi100.stdout:CentOS Stream 9 - AppStream 11 MB/s | 19 MB 00:01 2024-03-21T10:07:53.137 INFO:teuthology.orchestra.run.smithi047.stdout:Extra Packages for Enterprise Linux 12 MB/s | 21 MB 00:01 2024-03-21T10:07:57.998 INFO:teuthology.orchestra.run.smithi100.stdout:CentOS Stream 9 - CRB 2.8 MB/s | 6.1 MB 00:02 2024-03-21T10:07:59.106 INFO:teuthology.orchestra.run.smithi047.stdout:lab-extras 37 kB/s | 1.7 kB 00:00 2024-03-21T10:08:00.549 INFO:teuthology.orchestra.run.smithi100.stdout:CentOS Stream 9 - Extras packages 18 kB/s | 16 kB 00:00 2024-03-21T10:08:01.618 INFO:teuthology.orchestra.run.smithi100.stdout:Extra Packages for Enterprise Linux 23 MB/s | 21 MB 00:00 2024-03-21T10:08:01.825 INFO:teuthology.orchestra.run.smithi047.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-03-21T10:08:01.827 INFO:teuthology.orchestra.run.smithi047.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-03-21T10:08:01.851 INFO:teuthology.orchestra.run.smithi047.stdout:Package NetworkManager-1:1.44.0-2.el9.x86_64 is already installed. 2024-03-21T10:08:01.852 INFO:teuthology.orchestra.run.smithi047.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-03-21T10:08:01.854 INFO:teuthology.orchestra.run.smithi047.stdout:Package util-linux-2.37.4-15.el9.x86_64 is already installed. 2024-03-21T10:08:01.855 INFO:teuthology.orchestra.run.smithi047.stdout:Package libacl-devel-2.3.1-3.el9.x86_64 is already installed. 2024-03-21T10:08:01.858 INFO:teuthology.orchestra.run.smithi047.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-03-21T10:08:01.862 INFO:teuthology.orchestra.run.smithi047.stdout:Package xfsdump-3.1.12-3.el9.x86_64 is already installed. 2024-03-21T10:08:01.864 INFO:teuthology.orchestra.run.smithi047.stdout:Package xfsprogs-5.19.0-4.el9.x86_64 is already installed. 2024-03-21T10:08:01.962 INFO:teuthology.orchestra.run.smithi047.stdout:Dependencies resolved. 2024-03-21T10:08:01.987 INFO:teuthology.orchestra.run.smithi047.stdout:============================================================================================ 2024-03-21T10:08:01.987 INFO:teuthology.orchestra.run.smithi047.stdout: Package Arch Version Repository Size 2024-03-21T10:08:01.987 INFO:teuthology.orchestra.run.smithi047.stdout:============================================================================================ 2024-03-21T10:08:01.987 INFO:teuthology.orchestra.run.smithi047.stdout:Installing: 2024-03-21T10:08:01.987 INFO:teuthology.orchestra.run.smithi047.stdout: bison x86_64 3.7.4-5.el9 CentOS-AppStream 920 k 2024-03-21T10:08:01.987 INFO:teuthology.orchestra.run.smithi047.stdout: ceph x86_64 2:19.0.0-2231.g2edfee68.el9 ceph 6.6 k 2024-03-21T10:08:01.987 INFO:teuthology.orchestra.run.smithi047.stdout: ceph-base x86_64 2:19.0.0-2231.g2edfee68.el9 ceph 5.4 M 2024-03-21T10:08:01.987 INFO:teuthology.orchestra.run.smithi047.stdout: ceph-fuse x86_64 2:19.0.0-2231.g2edfee68.el9 ceph 890 k 2024-03-21T10:08:01.987 INFO:teuthology.orchestra.run.smithi047.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-2231.g2edfee68.el9 ceph 145 k 2024-03-21T10:08:01.987 INFO:teuthology.orchestra.run.smithi047.stdout: ceph-mgr x86_64 2:19.0.0-2231.g2edfee68.el9 ceph 1.5 M 2024-03-21T10:08:01.988 INFO:teuthology.orchestra.run.smithi047.stdout: ceph-mgr-cephadm noarch 2:19.0.0-2231.g2edfee68.el9 ceph-noarch 140 k 2024-03-21T10:08:01.988 INFO:teuthology.orchestra.run.smithi047.stdout: ceph-mgr-dashboard noarch 2:19.0.0-2231.g2edfee68.el9 ceph-noarch 1.8 M 2024-03-21T10:08:01.988 INFO:teuthology.orchestra.run.smithi047.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-2231.g2edfee68.el9 ceph-noarch 7.4 M 2024-03-21T10:08:01.988 INFO:teuthology.orchestra.run.smithi047.stdout: ceph-mgr-rook noarch 2:19.0.0-2231.g2edfee68.el9 ceph-noarch 49 k 2024-03-21T10:08:01.988 INFO:teuthology.orchestra.run.smithi047.stdout: ceph-radosgw x86_64 2:19.0.0-2231.g2edfee68.el9 ceph 9.9 M 2024-03-21T10:08:01.988 INFO:teuthology.orchestra.run.smithi047.stdout: ceph-test x86_64 2:19.0.0-2231.g2edfee68.el9 ceph 48 M 2024-03-21T10:08:01.988 INFO:teuthology.orchestra.run.smithi047.stdout: ceph-volume noarch 2:19.0.0-2231.g2edfee68.el9 ceph-noarch 264 k 2024-03-21T10:08:01.988 INFO:teuthology.orchestra.run.smithi047.stdout: cephadm noarch 2:19.0.0-2231.g2edfee68.el9 ceph-noarch 750 k 2024-03-21T10:08:01.988 INFO:teuthology.orchestra.run.smithi047.stdout: cephfs-mirror x86_64 2:19.0.0-2231.g2edfee68.el9 ceph 220 k 2024-03-21T10:08:01.988 INFO:teuthology.orchestra.run.smithi047.stdout: cephfs-shell x86_64 2:19.0.0-2231.g2edfee68.el9 ceph 27 k 2024-03-21T10:08:01.988 INFO:teuthology.orchestra.run.smithi047.stdout: cephfs-top noarch 2:19.0.0-2231.g2edfee68.el9 ceph-noarch 18 k 2024-03-21T10:08:01.988 INFO:teuthology.orchestra.run.smithi047.stdout: elfutils-libelf-devel x86_64 0.190-2.el9 appstream 21 k 2024-03-21T10:08:01.988 INFO:teuthology.orchestra.run.smithi047.stdout: flex x86_64 2.6.4-9.el9 CentOS-AppStream 307 k 2024-03-21T10:08:01.988 INFO:teuthology.orchestra.run.smithi047.stdout: libaio-devel x86_64 0.3.111-13.el9 CentOS-AppStream 10 k 2024-03-21T10:08:01.989 INFO:teuthology.orchestra.run.smithi047.stdout: libcephfs-devel x86_64 2:19.0.0-2231.g2edfee68.el9 ceph 31 k 2024-03-21T10:08:01.989 INFO:teuthology.orchestra.run.smithi047.stdout: libcephfs2 x86_64 2:19.0.0-2231.g2edfee68.el9 ceph 747 k 2024-03-21T10:08:01.989 INFO:teuthology.orchestra.run.smithi047.stdout: librados-devel x86_64 2:19.0.0-2231.g2edfee68.el9 ceph 127 k 2024-03-21T10:08:01.989 INFO:teuthology.orchestra.run.smithi047.stdout: libtool x86_64 2.4.6-45.el9 CentOS-AppStream 578 k 2024-03-21T10:08:01.989 INFO:teuthology.orchestra.run.smithi047.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-03-21T10:08:01.989 INFO:teuthology.orchestra.run.smithi047.stdout: openssl-devel x86_64 1:3.0.7-27.el9 appstream 4.1 M 2024-03-21T10:08:01.989 INFO:teuthology.orchestra.run.smithi047.stdout: postgresql x86_64 13.11-1.el9 CentOS-AppStream 1.5 M 2024-03-21T10:08:01.989 INFO:teuthology.orchestra.run.smithi047.stdout: postgresql-contrib x86_64 13.11-1.el9 CentOS-AppStream 811 k 2024-03-21T10:08:01.989 INFO:teuthology.orchestra.run.smithi047.stdout: postgresql-server x86_64 13.11-1.el9 CentOS-AppStream 5.7 M 2024-03-21T10:08:01.989 INFO:teuthology.orchestra.run.smithi047.stdout: python3-cephfs x86_64 2:19.0.0-2231.g2edfee68.el9 ceph 161 k 2024-03-21T10:08:01.989 INFO:teuthology.orchestra.run.smithi047.stdout: python3-rados x86_64 2:19.0.0-2231.g2edfee68.el9 ceph 320 k 2024-03-21T10:08:01.989 INFO:teuthology.orchestra.run.smithi047.stdout: python3-rbd x86_64 2:19.0.0-2231.g2edfee68.el9 ceph 297 k 2024-03-21T10:08:01.989 INFO:teuthology.orchestra.run.smithi047.stdout: python3-rgw x86_64 2:19.0.0-2231.g2edfee68.el9 ceph 99 k 2024-03-21T10:08:01.990 INFO:teuthology.orchestra.run.smithi047.stdout: rbd-fuse x86_64 2:19.0.0-2231.g2edfee68.el9 ceph 85 k 2024-03-21T10:08:01.990 INFO:teuthology.orchestra.run.smithi047.stdout: rbd-mirror x86_64 2:19.0.0-2231.g2edfee68.el9 ceph 3.1 M 2024-03-21T10:08:01.990 INFO:teuthology.orchestra.run.smithi047.stdout: rbd-nbd x86_64 2:19.0.0-2231.g2edfee68.el9 ceph 171 k 2024-03-21T10:08:01.990 INFO:teuthology.orchestra.run.smithi047.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-03-21T10:08:01.990 INFO:teuthology.orchestra.run.smithi047.stdout:Upgrading: 2024-03-21T10:08:01.990 INFO:teuthology.orchestra.run.smithi047.stdout: NetworkManager x86_64 1:1.47.2-1.el9 baseos 2.3 M 2024-03-21T10:08:01.990 INFO:teuthology.orchestra.run.smithi047.stdout: NetworkManager-libnm x86_64 1:1.47.2-1.el9 baseos 1.8 M 2024-03-21T10:08:01.990 INFO:teuthology.orchestra.run.smithi047.stdout: NetworkManager-team x86_64 1:1.47.2-1.el9 baseos 42 k 2024-03-21T10:08:01.990 INFO:teuthology.orchestra.run.smithi047.stdout: NetworkManager-tui x86_64 1:1.47.2-1.el9 baseos 248 k 2024-03-21T10:08:01.990 INFO:teuthology.orchestra.run.smithi047.stdout: acl x86_64 2.3.1-4.el9 baseos 71 k 2024-03-21T10:08:01.990 INFO:teuthology.orchestra.run.smithi047.stdout: elfutils-debuginfod-client x86_64 0.190-2.el9 baseos 36 k 2024-03-21T10:08:01.990 INFO:teuthology.orchestra.run.smithi047.stdout: elfutils-libelf x86_64 0.190-2.el9 baseos 192 k 2024-03-21T10:08:01.990 INFO:teuthology.orchestra.run.smithi047.stdout: elfutils-libs x86_64 0.190-2.el9 baseos 256 k 2024-03-21T10:08:01.991 INFO:teuthology.orchestra.run.smithi047.stdout: libacl x86_64 2.3.1-4.el9 baseos 23 k 2024-03-21T10:08:01.991 INFO:teuthology.orchestra.run.smithi047.stdout: libacl-devel x86_64 2.3.1-4.el9 appstream 74 k 2024-03-21T10:08:01.991 INFO:teuthology.orchestra.run.smithi047.stdout: libblkid x86_64 2.37.4-18.el9 baseos 108 k 2024-03-21T10:08:01.991 INFO:teuthology.orchestra.run.smithi047.stdout: libfdisk x86_64 2.37.4-18.el9 baseos 155 k 2024-03-21T10:08:01.991 INFO:teuthology.orchestra.run.smithi047.stdout: libmount x86_64 2.37.4-18.el9 baseos 136 k 2024-03-21T10:08:01.991 INFO:teuthology.orchestra.run.smithi047.stdout: librados2 x86_64 2:19.0.0-2231.g2edfee68.el9 ceph 3.4 M 2024-03-21T10:08:01.991 INFO:teuthology.orchestra.run.smithi047.stdout: librbd1 x86_64 2:19.0.0-2231.g2edfee68.el9 ceph 3.1 M 2024-03-21T10:08:01.991 INFO:teuthology.orchestra.run.smithi047.stdout: libsmartcols x86_64 2.37.4-18.el9 baseos 64 k 2024-03-21T10:08:01.991 INFO:teuthology.orchestra.run.smithi047.stdout: libuuid x86_64 2.37.4-18.el9 baseos 29 k 2024-03-21T10:08:01.991 INFO:teuthology.orchestra.run.smithi047.stdout: openssl x86_64 1:3.0.7-27.el9 baseos 1.2 M 2024-03-21T10:08:01.991 INFO:teuthology.orchestra.run.smithi047.stdout: openssl-libs x86_64 1:3.0.7-27.el9 baseos 2.2 M 2024-03-21T10:08:01.992 INFO:teuthology.orchestra.run.smithi047.stdout: util-linux x86_64 2.37.4-18.el9 baseos 2.3 M 2024-03-21T10:08:01.992 INFO:teuthology.orchestra.run.smithi047.stdout: util-linux-core x86_64 2.37.4-18.el9 baseos 465 k 2024-03-21T10:08:01.992 INFO:teuthology.orchestra.run.smithi047.stdout: util-linux-user x86_64 2.37.4-18.el9 baseos 31 k 2024-03-21T10:08:01.992 INFO:teuthology.orchestra.run.smithi047.stdout: xfsdump x86_64 3.1.12-4.el9 CentOS-BaseOS 321 k 2024-03-21T10:08:01.992 INFO:teuthology.orchestra.run.smithi047.stdout: xfsprogs x86_64 6.3.0-1.el9 baseos 1.1 M 2024-03-21T10:08:01.992 INFO:teuthology.orchestra.run.smithi047.stdout:Installing dependencies: 2024-03-21T10:08:01.992 INFO:teuthology.orchestra.run.smithi047.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-03-21T10:08:01.992 INFO:teuthology.orchestra.run.smithi047.stdout: automake noarch 1.16.2-8.el9 CentOS-AppStream 662 k 2024-03-21T10:08:01.992 INFO:teuthology.orchestra.run.smithi047.stdout: boost-program-options x86_64 1.75.0-8.el9 CentOS-AppStream 106 k 2024-03-21T10:08:01.992 INFO:teuthology.orchestra.run.smithi047.stdout: ceph-common x86_64 2:19.0.0-2231.g2edfee68.el9 ceph 21 M 2024-03-21T10:08:01.992 INFO:teuthology.orchestra.run.smithi047.stdout: ceph-grafana-dashboards noarch 2:19.0.0-2231.g2edfee68.el9 ceph-noarch 34 k 2024-03-21T10:08:01.992 INFO:teuthology.orchestra.run.smithi047.stdout: ceph-mds x86_64 2:19.0.0-2231.g2edfee68.el9 ceph 2.3 M 2024-03-21T10:08:01.992 INFO:teuthology.orchestra.run.smithi047.stdout: ceph-mgr-modules-core noarch 2:19.0.0-2231.g2edfee68.el9 ceph-noarch 248 k 2024-03-21T10:08:01.992 INFO:teuthology.orchestra.run.smithi047.stdout: ceph-mon x86_64 2:19.0.0-2231.g2edfee68.el9 ceph 4.7 M 2024-03-21T10:08:01.993 INFO:teuthology.orchestra.run.smithi047.stdout: ceph-osd x86_64 2:19.0.0-2231.g2edfee68.el9 ceph 17 M 2024-03-21T10:08:01.993 INFO:teuthology.orchestra.run.smithi047.stdout: ceph-prometheus-alerts noarch 2:19.0.0-2231.g2edfee68.el9 ceph-noarch 16 k 2024-03-21T10:08:01.993 INFO:teuthology.orchestra.run.smithi047.stdout: ceph-selinux x86_64 2:19.0.0-2231.g2edfee68.el9 ceph 25 k 2024-03-21T10:08:01.993 INFO:teuthology.orchestra.run.smithi047.stdout: flexiblas x86_64 3.0.4-8.el9 CentOS-AppStream 32 k 2024-03-21T10:08:01.993 INFO:teuthology.orchestra.run.smithi047.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 CentOS-AppStream 3.0 M 2024-03-21T10:08:01.993 INFO:teuthology.orchestra.run.smithi047.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 CentOS-AppStream 17 k 2024-03-21T10:08:01.993 INFO:teuthology.orchestra.run.smithi047.stdout: fuse x86_64 2.9.9-15.el9 CentOS-BaseOS 79 k 2024-03-21T10:08:01.993 INFO:teuthology.orchestra.run.smithi047.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-03-21T10:08:01.993 INFO:teuthology.orchestra.run.smithi047.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-03-21T10:08:01.993 INFO:teuthology.orchestra.run.smithi047.stdout: jq x86_64 1.6-16.el9 baseos 187 k 2024-03-21T10:08:01.993 INFO:teuthology.orchestra.run.smithi047.stdout: libarrow x86_64 9.0.0-10.el9 epel 4.4 M 2024-03-21T10:08:01.993 INFO:teuthology.orchestra.run.smithi047.stdout: libarrow-doc noarch 9.0.0-10.el9 epel 26 k 2024-03-21T10:08:01.993 INFO:teuthology.orchestra.run.smithi047.stdout: libcephsqlite x86_64 2:19.0.0-2231.g2edfee68.el9 ceph 163 k 2024-03-21T10:08:01.993 INFO:teuthology.orchestra.run.smithi047.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-03-21T10:08:01.994 INFO:teuthology.orchestra.run.smithi047.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-03-21T10:08:01.994 INFO:teuthology.orchestra.run.smithi047.stdout: libpmemobj x86_64 1.12.1-1.el9 CentOS-AppStream 159 k 2024-03-21T10:08:01.994 INFO:teuthology.orchestra.run.smithi047.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-03-21T10:08:01.994 INFO:teuthology.orchestra.run.smithi047.stdout: librabbitmq x86_64 0.11.0-7.el9 CentOS-AppStream 44 k 2024-03-21T10:08:01.994 INFO:teuthology.orchestra.run.smithi047.stdout: libradosstriper1 x86_64 2:19.0.0-2231.g2edfee68.el9 ceph 501 k 2024-03-21T10:08:01.994 INFO:teuthology.orchestra.run.smithi047.stdout: librdkafka x86_64 1.6.1-102.el9 CentOS-AppStream 661 k 2024-03-21T10:08:01.994 INFO:teuthology.orchestra.run.smithi047.stdout: librgw2 x86_64 2:19.0.0-2231.g2edfee68.el9 ceph 5.1 M 2024-03-21T10:08:01.994 INFO:teuthology.orchestra.run.smithi047.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-03-21T10:08:01.994 INFO:teuthology.orchestra.run.smithi047.stdout: libzstd-devel x86_64 1.5.1-2.el9 CentOS-AppStream 46 k 2024-03-21T10:08:01.994 INFO:teuthology.orchestra.run.smithi047.stdout: lttng-ust x86_64 2.12.0-6.el9 CentOS-AppStream 282 k 2024-03-21T10:08:01.994 INFO:teuthology.orchestra.run.smithi047.stdout: lua x86_64 5.4.4-4.el9 CentOS-AppStream 187 k 2024-03-21T10:08:01.994 INFO:teuthology.orchestra.run.smithi047.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-03-21T10:08:01.994 INFO:teuthology.orchestra.run.smithi047.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-03-21T10:08:01.995 INFO:teuthology.orchestra.run.smithi047.stdout: oniguruma x86_64 6.9.6-1.el9.5 CentOS-AppStream 217 k 2024-03-21T10:08:01.995 INFO:teuthology.orchestra.run.smithi047.stdout: openblas x86_64 0.3.21-2.el9 CentOS-AppStream 34 k 2024-03-21T10:08:01.995 INFO:teuthology.orchestra.run.smithi047.stdout: openblas-openmp x86_64 0.3.21-2.el9 CentOS-AppStream 4.7 M 2024-03-21T10:08:01.995 INFO:teuthology.orchestra.run.smithi047.stdout: parquet-libs x86_64 9.0.0-10.el9 epel 838 k 2024-03-21T10:08:01.995 INFO:teuthology.orchestra.run.smithi047.stdout: postgresql-private-libs x86_64 13.11-1.el9 CentOS-AppStream 134 k 2024-03-21T10:08:01.995 INFO:teuthology.orchestra.run.smithi047.stdout: protobuf x86_64 3.14.0-13.el9 CentOS-AppStream 1.0 M 2024-03-21T10:08:01.995 INFO:teuthology.orchestra.run.smithi047.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-03-21T10:08:01.995 INFO:teuthology.orchestra.run.smithi047.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-03-21T10:08:01.995 INFO:teuthology.orchestra.run.smithi047.stdout: python3-attrs noarch 20.3.0-7.el9 CentOS-AppStream 78 k 2024-03-21T10:08:01.995 INFO:teuthology.orchestra.run.smithi047.stdout: python3-babel noarch 2.9.1-2.el9 CentOS-AppStream 5.8 M 2024-03-21T10:08:01.995 INFO:teuthology.orchestra.run.smithi047.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-03-21T10:08:01.995 INFO:teuthology.orchestra.run.smithi047.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-03-21T10:08:01.995 INFO:teuthology.orchestra.run.smithi047.stdout: python3-ceph-argparse x86_64 2:19.0.0-2231.g2edfee68.el9 ceph 45 k 2024-03-21T10:08:01.995 INFO:teuthology.orchestra.run.smithi047.stdout: python3-ceph-common x86_64 2:19.0.0-2231.g2edfee68.el9 ceph 131 k 2024-03-21T10:08:01.996 INFO:teuthology.orchestra.run.smithi047.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-03-21T10:08:01.996 INFO:teuthology.orchestra.run.smithi047.stdout: python3-cffi x86_64 1.14.5-5.el9 CentOS-BaseOS 241 k 2024-03-21T10:08:01.996 INFO:teuthology.orchestra.run.smithi047.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-03-21T10:08:01.996 INFO:teuthology.orchestra.run.smithi047.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-03-21T10:08:01.996 INFO:teuthology.orchestra.run.smithi047.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-03-21T10:08:01.996 INFO:teuthology.orchestra.run.smithi047.stdout: python3-colorama noarch 0.4.4-12.el9 epel 35 k 2024-03-21T10:08:01.996 INFO:teuthology.orchestra.run.smithi047.stdout: python3-cryptography x86_64 36.0.1-4.el9 CentOS-BaseOS 1.1 M 2024-03-21T10:08:01.996 INFO:teuthology.orchestra.run.smithi047.stdout: python3-devel x86_64 3.9.17-2.el9 appstream 245 k 2024-03-21T10:08:01.996 INFO:teuthology.orchestra.run.smithi047.stdout: python3-google-auth noarch 1:2.28.2-1.el9 epel 203 k 2024-03-21T10:08:01.996 INFO:teuthology.orchestra.run.smithi047.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-03-21T10:08:01.996 INFO:teuthology.orchestra.run.smithi047.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-03-21T10:08:01.996 INFO:teuthology.orchestra.run.smithi047.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-03-21T10:08:01.996 INFO:teuthology.orchestra.run.smithi047.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-03-21T10:08:01.996 INFO:teuthology.orchestra.run.smithi047.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-03-21T10:08:01.997 INFO:teuthology.orchestra.run.smithi047.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-03-21T10:08:01.997 INFO:teuthology.orchestra.run.smithi047.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-03-21T10:08:01.997 INFO:teuthology.orchestra.run.smithi047.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-03-21T10:08:01.997 INFO:teuthology.orchestra.run.smithi047.stdout: python3-jsonpatch noarch 1.21-16.el9 CentOS-AppStream 24 k 2024-03-21T10:08:01.997 INFO:teuthology.orchestra.run.smithi047.stdout: python3-jsonpointer noarch 2.0-4.el9 CentOS-AppStream 18 k 2024-03-21T10:08:01.997 INFO:teuthology.orchestra.run.smithi047.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-03-21T10:08:01.997 INFO:teuthology.orchestra.run.smithi047.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-03-21T10:08:01.997 INFO:teuthology.orchestra.run.smithi047.stdout: python3-mako noarch 1.1.4-6.el9 CentOS-AppStream 144 k 2024-03-21T10:08:01.997 INFO:teuthology.orchestra.run.smithi047.stdout: python3-markupsafe x86_64 1.1.1-12.el9 CentOS-AppStream 32 k 2024-03-21T10:08:01.997 INFO:teuthology.orchestra.run.smithi047.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-03-21T10:08:01.997 INFO:teuthology.orchestra.run.smithi047.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-03-21T10:08:01.998 INFO:teuthology.orchestra.run.smithi047.stdout: python3-numpy x86_64 1:1.20.1-5.el9 CentOS-AppStream 4.8 M 2024-03-21T10:08:01.998 INFO:teuthology.orchestra.run.smithi047.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 CentOS-AppStream 217 k 2024-03-21T10:08:01.998 INFO:teuthology.orchestra.run.smithi047.stdout: python3-oauthlib noarch 3.1.1-5.el9 CentOS-AppStream 174 k 2024-03-21T10:08:01.998 INFO:teuthology.orchestra.run.smithi047.stdout: python3-packaging noarch 20.9-5.el9 CentOS-AppStream 69 k 2024-03-21T10:08:01.998 INFO:teuthology.orchestra.run.smithi047.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-03-21T10:08:01.998 INFO:teuthology.orchestra.run.smithi047.stdout: python3-ply noarch 3.11-14.el9 CentOS-BaseOS 103 k 2024-03-21T10:08:01.998 INFO:teuthology.orchestra.run.smithi047.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-03-21T10:08:01.998 INFO:teuthology.orchestra.run.smithi047.stdout: python3-prettytable noarch 0.7.2-27.el9 CentOS-AppStream 41 k 2024-03-21T10:08:01.998 INFO:teuthology.orchestra.run.smithi047.stdout: python3-protobuf noarch 3.14.0-13.el9 CentOS-AppStream 238 k 2024-03-21T10:08:01.998 INFO:teuthology.orchestra.run.smithi047.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-03-21T10:08:01.998 INFO:teuthology.orchestra.run.smithi047.stdout: python3-pyasn1 noarch 0.4.8-6.el9 CentOS-AppStream 132 k 2024-03-21T10:08:01.998 INFO:teuthology.orchestra.run.smithi047.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 CentOS-AppStream 211 k 2024-03-21T10:08:01.998 INFO:teuthology.orchestra.run.smithi047.stdout: python3-pycparser noarch 2.20-6.el9 CentOS-BaseOS 124 k 2024-03-21T10:08:01.998 INFO:teuthology.orchestra.run.smithi047.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-03-21T10:08:01.999 INFO:teuthology.orchestra.run.smithi047.stdout: python3-pytz noarch 2021.1-5.el9 CentOS-AppStream 47 k 2024-03-21T10:08:01.999 INFO:teuthology.orchestra.run.smithi047.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-03-21T10:08:01.999 INFO:teuthology.orchestra.run.smithi047.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-03-21T10:08:01.999 INFO:teuthology.orchestra.run.smithi047.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 CentOS-AppStream 43 k 2024-03-21T10:08:01.999 INFO:teuthology.orchestra.run.smithi047.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-03-21T10:08:01.999 INFO:teuthology.orchestra.run.smithi047.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-03-21T10:08:01.999 INFO:teuthology.orchestra.run.smithi047.stdout: python3-scipy x86_64 1.6.2-8.el9 CentOS-AppStream 16 M 2024-03-21T10:08:01.999 INFO:teuthology.orchestra.run.smithi047.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-03-21T10:08:01.999 INFO:teuthology.orchestra.run.smithi047.stdout: python3-toml noarch 0.10.2-6.el9 CentOS-AppStream 39 k 2024-03-21T10:08:01.999 INFO:teuthology.orchestra.run.smithi047.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-03-21T10:08:01.999 INFO:teuthology.orchestra.run.smithi047.stdout: python3-urllib3 noarch 1.26.5-5.el9 CentOS-BaseOS 187 k 2024-03-21T10:08:01.999 INFO:teuthology.orchestra.run.smithi047.stdout: python3-wcwidth noarch 0.2.5-8.el9 CentOS-AppStream 41 k 2024-03-21T10:08:01.999 INFO:teuthology.orchestra.run.smithi047.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-03-21T10:08:01.999 INFO:teuthology.orchestra.run.smithi047.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-03-21T10:08:02.000 INFO:teuthology.orchestra.run.smithi047.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-03-21T10:08:02.000 INFO:teuthology.orchestra.run.smithi047.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-03-21T10:08:02.000 INFO:teuthology.orchestra.run.smithi047.stdout: qatlib x86_64 23.11.0-1.el9 appstream 216 k 2024-03-21T10:08:02.000 INFO:teuthology.orchestra.run.smithi047.stdout: qatzip-libs x86_64 1.1.2-2.el9 appstream 44 k 2024-03-21T10:08:02.000 INFO:teuthology.orchestra.run.smithi047.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-03-21T10:08:02.000 INFO:teuthology.orchestra.run.smithi047.stdout: socat x86_64 1.7.4.1-5.el9 CentOS-AppStream 300 k 2024-03-21T10:08:02.000 INFO:teuthology.orchestra.run.smithi047.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-03-21T10:08:02.000 INFO:teuthology.orchestra.run.smithi047.stdout: uuid x86_64 1.6.2-55.el9 CentOS-AppStream 56 k 2024-03-21T10:08:02.000 INFO:teuthology.orchestra.run.smithi047.stdout: xmlstarlet x86_64 1.6.1-20.el9 CentOS-AppStream 63 k 2024-03-21T10:08:02.000 INFO:teuthology.orchestra.run.smithi047.stdout: zlib-devel x86_64 1.2.11-40.el9 CentOS-AppStream 44 k 2024-03-21T10:08:02.000 INFO:teuthology.orchestra.run.smithi047.stdout:Installing weak dependencies: 2024-03-21T10:08:02.000 INFO:teuthology.orchestra.run.smithi047.stdout: qatlib-service x86_64 23.11.0-1.el9 appstream 35 k 2024-03-21T10:08:02.000 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-21T10:08:02.000 INFO:teuthology.orchestra.run.smithi047.stdout:Transaction Summary 2024-03-21T10:08:02.001 INFO:teuthology.orchestra.run.smithi047.stdout:============================================================================================ 2024-03-21T10:08:02.001 INFO:teuthology.orchestra.run.smithi047.stdout:Install 150 Packages 2024-03-21T10:08:02.001 INFO:teuthology.orchestra.run.smithi047.stdout:Upgrade 24 Packages 2024-03-21T10:08:02.001 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-21T10:08:02.001 INFO:teuthology.orchestra.run.smithi047.stdout:Total download size: 224 M 2024-03-21T10:08:02.001 INFO:teuthology.orchestra.run.smithi047.stdout:Downloading Packages: 2024-03-21T10:08:02.983 INFO:teuthology.orchestra.run.smithi047.stdout:(1/174): ceph-19.0.0-2231.g2edfee68.el9.x86_64. 22 kB/s | 6.6 kB 00:00 2024-03-21T10:08:03.308 INFO:teuthology.orchestra.run.smithi047.stdout:(2/174): ceph-base-19.0.0-2231.g2edfee68.el9.x8 8.7 MB/s | 5.4 MB 00:00 2024-03-21T10:08:03.358 INFO:teuthology.orchestra.run.smithi047.stdout:(3/174): ceph-immutable-object-cache-19.0.0-223 2.9 MB/s | 145 kB 00:00 2024-03-21T10:08:03.517 INFO:teuthology.orchestra.run.smithi047.stdout:(4/174): ceph-mds-19.0.0-2231.g2edfee68.el9.x86 15 MB/s | 2.3 MB 00:00 2024-03-21T10:08:03.642 INFO:teuthology.orchestra.run.smithi047.stdout:(5/174): ceph-mgr-19.0.0-2231.g2edfee68.el9.x86 12 MB/s | 1.5 MB 00:00 2024-03-21T10:08:03.918 INFO:teuthology.orchestra.run.smithi047.stdout:(6/174): ceph-mon-19.0.0-2231.g2edfee68.el9.x86 17 MB/s | 4.7 MB 00:00 2024-03-21T10:08:03.959 INFO:teuthology.orchestra.run.smithi047.stdout:(7/174): ceph-fuse-19.0.0-2231.g2edfee68.el9.x8 913 kB/s | 890 kB 00:00 2024-03-21T10:08:04.809 INFO:teuthology.orchestra.run.smithi047.stdout:(8/174): ceph-osd-19.0.0-2231.g2edfee68.el9.x86 19 MB/s | 17 MB 00:00 2024-03-21T10:08:04.859 INFO:teuthology.orchestra.run.smithi047.stdout:(9/174): ceph-selinux-19.0.0-2231.g2edfee68.el9 516 kB/s | 25 kB 00:00 2024-03-21T10:08:07.268 INFO:teuthology.orchestra.run.smithi047.stdout:(10/174): ceph-test-19.0.0-2231.g2edfee68.el9.x 20 MB/s | 48 MB 00:02 2024-03-21T10:08:07.326 INFO:teuthology.orchestra.run.smithi047.stdout:(11/174): cephfs-mirror-19.0.0-2231.g2edfee68.e 3.8 MB/s | 220 kB 00:00 2024-03-21T10:08:07.377 INFO:teuthology.orchestra.run.smithi047.stdout:(12/174): cephfs-shell-19.0.0-2231.g2edfee68.el 552 kB/s | 27 kB 00:00 2024-03-21T10:08:07.427 INFO:teuthology.orchestra.run.smithi047.stdout:(13/174): libcephfs-devel-19.0.0-2231.g2edfee68 639 kB/s | 31 kB 00:00 2024-03-21T10:08:07.442 INFO:teuthology.orchestra.run.smithi100.stdout:lab-extras 27 kB/s | 1.7 kB 00:00 2024-03-21T10:08:07.519 INFO:teuthology.orchestra.run.smithi047.stdout:(14/174): libcephfs2-19.0.0-2231.g2edfee68.el9. 8.1 MB/s | 747 kB 00:00 2024-03-21T10:08:07.578 INFO:teuthology.orchestra.run.smithi047.stdout:(15/174): libcephsqlite-19.0.0-2231.g2edfee68.e 2.8 MB/s | 163 kB 00:00 2024-03-21T10:08:07.628 INFO:teuthology.orchestra.run.smithi047.stdout:(16/174): librados-devel-19.0.0-2231.g2edfee68. 2.5 MB/s | 127 kB 00:00 2024-03-21T10:08:07.694 INFO:teuthology.orchestra.run.smithi047.stdout:(17/174): libradosstriper1-19.0.0-2231.g2edfee6 7.5 MB/s | 501 kB 00:00 2024-03-21T10:08:08.020 INFO:teuthology.orchestra.run.smithi047.stdout:(18/174): librgw2-19.0.0-2231.g2edfee68.el9.x86 16 MB/s | 5.1 MB 00:00 2024-03-21T10:08:08.104 INFO:teuthology.orchestra.run.smithi047.stdout:(19/174): python3-ceph-argparse-19.0.0-2231.g2e 549 kB/s | 45 kB 00:00 2024-03-21T10:08:08.196 INFO:teuthology.orchestra.run.smithi047.stdout:(20/174): python3-ceph-common-19.0.0-2231.g2edf 1.4 MB/s | 131 kB 00:00 2024-03-21T10:08:08.287 INFO:teuthology.orchestra.run.smithi047.stdout:(21/174): python3-cephfs-19.0.0-2231.g2edfee68. 1.7 MB/s | 161 kB 00:00 2024-03-21T10:08:08.388 INFO:teuthology.orchestra.run.smithi047.stdout:(22/174): python3-rados-19.0.0-2231.g2edfee68.e 3.2 MB/s | 320 kB 00:00 2024-03-21T10:08:08.496 INFO:teuthology.orchestra.run.smithi047.stdout:(23/174): python3-rbd-19.0.0-2231.g2edfee68.el9 2.7 MB/s | 297 kB 00:00 2024-03-21T10:08:08.588 INFO:teuthology.orchestra.run.smithi047.stdout:(24/174): python3-rgw-19.0.0-2231.g2edfee68.el9 1.1 MB/s | 99 kB 00:00 2024-03-21T10:08:08.672 INFO:teuthology.orchestra.run.smithi047.stdout:(25/174): rbd-fuse-19.0.0-2231.g2edfee68.el9.x8 1.0 MB/s | 85 kB 00:00 2024-03-21T10:08:08.914 INFO:teuthology.orchestra.run.smithi047.stdout:(26/174): rbd-mirror-19.0.0-2231.g2edfee68.el9. 13 MB/s | 3.1 MB 00:00 2024-03-21T10:08:09.006 INFO:teuthology.orchestra.run.smithi047.stdout:(27/174): rbd-nbd-19.0.0-2231.g2edfee68.el9.x86 1.8 MB/s | 171 kB 00:00 2024-03-21T10:08:09.119 INFO:teuthology.orchestra.run.smithi047.stdout:(28/174): ceph-grafana-dashboards-19.0.0-2231.g 301 kB/s | 34 kB 00:00 2024-03-21T10:08:09.215 INFO:teuthology.orchestra.run.smithi047.stdout:(29/174): ceph-mgr-cephadm-19.0.0-2231.g2edfee6 1.4 MB/s | 140 kB 00:00 2024-03-21T10:08:09.398 INFO:teuthology.orchestra.run.smithi047.stdout:(30/174): ceph-mgr-dashboard-19.0.0-2231.g2edfe 9.8 MB/s | 1.8 MB 00:00 2024-03-21T10:08:09.874 INFO:teuthology.orchestra.run.smithi047.stdout:(31/174): ceph-mgr-diskprediction-local-19.0.0- 16 MB/s | 7.4 MB 00:00 2024-03-21T10:08:10.002 INFO:teuthology.orchestra.run.smithi100.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-03-21T10:08:10.004 INFO:teuthology.orchestra.run.smithi100.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-03-21T10:08:10.026 INFO:teuthology.orchestra.run.smithi100.stdout:Package NetworkManager-1:1.44.0-2.el9.x86_64 is already installed. 2024-03-21T10:08:10.028 INFO:teuthology.orchestra.run.smithi100.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-03-21T10:08:10.029 INFO:teuthology.orchestra.run.smithi100.stdout:Package util-linux-2.37.4-15.el9.x86_64 is already installed. 2024-03-21T10:08:10.031 INFO:teuthology.orchestra.run.smithi100.stdout:Package libacl-devel-2.3.1-3.el9.x86_64 is already installed. 2024-03-21T10:08:10.033 INFO:teuthology.orchestra.run.smithi100.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-03-21T10:08:10.037 INFO:teuthology.orchestra.run.smithi100.stdout:Package xfsdump-3.1.12-3.el9.x86_64 is already installed. 2024-03-21T10:08:10.038 INFO:teuthology.orchestra.run.smithi100.stdout:Package xfsprogs-5.19.0-4.el9.x86_64 is already installed. 2024-03-21T10:08:10.126 INFO:teuthology.orchestra.run.smithi100.stdout:Dependencies resolved. 2024-03-21T10:08:10.141 INFO:teuthology.orchestra.run.smithi047.stdout:(32/174): ceph-common-19.0.0-2231.g2edfee68.el9 2.8 MB/s | 21 MB 00:07 2024-03-21T10:08:10.147 INFO:teuthology.orchestra.run.smithi100.stdout:============================================================================================ 2024-03-21T10:08:10.148 INFO:teuthology.orchestra.run.smithi100.stdout: Package Arch Version Repository Size 2024-03-21T10:08:10.148 INFO:teuthology.orchestra.run.smithi100.stdout:============================================================================================ 2024-03-21T10:08:10.148 INFO:teuthology.orchestra.run.smithi100.stdout:Installing: 2024-03-21T10:08:10.148 INFO:teuthology.orchestra.run.smithi100.stdout: bison x86_64 3.7.4-5.el9 CentOS-AppStream 920 k 2024-03-21T10:08:10.148 INFO:teuthology.orchestra.run.smithi100.stdout: ceph x86_64 2:19.0.0-2231.g2edfee68.el9 ceph 6.6 k 2024-03-21T10:08:10.148 INFO:teuthology.orchestra.run.smithi100.stdout: ceph-base x86_64 2:19.0.0-2231.g2edfee68.el9 ceph 5.4 M 2024-03-21T10:08:10.148 INFO:teuthology.orchestra.run.smithi100.stdout: ceph-fuse x86_64 2:19.0.0-2231.g2edfee68.el9 ceph 890 k 2024-03-21T10:08:10.148 INFO:teuthology.orchestra.run.smithi100.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-2231.g2edfee68.el9 ceph 145 k 2024-03-21T10:08:10.148 INFO:teuthology.orchestra.run.smithi100.stdout: ceph-mgr x86_64 2:19.0.0-2231.g2edfee68.el9 ceph 1.5 M 2024-03-21T10:08:10.148 INFO:teuthology.orchestra.run.smithi100.stdout: ceph-mgr-cephadm noarch 2:19.0.0-2231.g2edfee68.el9 ceph-noarch 140 k 2024-03-21T10:08:10.148 INFO:teuthology.orchestra.run.smithi100.stdout: ceph-mgr-dashboard noarch 2:19.0.0-2231.g2edfee68.el9 ceph-noarch 1.8 M 2024-03-21T10:08:10.148 INFO:teuthology.orchestra.run.smithi100.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-2231.g2edfee68.el9 ceph-noarch 7.4 M 2024-03-21T10:08:10.148 INFO:teuthology.orchestra.run.smithi100.stdout: ceph-mgr-rook noarch 2:19.0.0-2231.g2edfee68.el9 ceph-noarch 49 k 2024-03-21T10:08:10.149 INFO:teuthology.orchestra.run.smithi100.stdout: ceph-radosgw x86_64 2:19.0.0-2231.g2edfee68.el9 ceph 9.9 M 2024-03-21T10:08:10.149 INFO:teuthology.orchestra.run.smithi100.stdout: ceph-test x86_64 2:19.0.0-2231.g2edfee68.el9 ceph 48 M 2024-03-21T10:08:10.149 INFO:teuthology.orchestra.run.smithi100.stdout: ceph-volume noarch 2:19.0.0-2231.g2edfee68.el9 ceph-noarch 264 k 2024-03-21T10:08:10.149 INFO:teuthology.orchestra.run.smithi100.stdout: cephadm noarch 2:19.0.0-2231.g2edfee68.el9 ceph-noarch 750 k 2024-03-21T10:08:10.149 INFO:teuthology.orchestra.run.smithi100.stdout: cephfs-mirror x86_64 2:19.0.0-2231.g2edfee68.el9 ceph 220 k 2024-03-21T10:08:10.149 INFO:teuthology.orchestra.run.smithi100.stdout: cephfs-shell x86_64 2:19.0.0-2231.g2edfee68.el9 ceph 27 k 2024-03-21T10:08:10.149 INFO:teuthology.orchestra.run.smithi100.stdout: cephfs-top noarch 2:19.0.0-2231.g2edfee68.el9 ceph-noarch 18 k 2024-03-21T10:08:10.149 INFO:teuthology.orchestra.run.smithi100.stdout: elfutils-libelf-devel x86_64 0.190-2.el9 appstream 21 k 2024-03-21T10:08:10.149 INFO:teuthology.orchestra.run.smithi100.stdout: flex x86_64 2.6.4-9.el9 CentOS-AppStream 307 k 2024-03-21T10:08:10.149 INFO:teuthology.orchestra.run.smithi100.stdout: libaio-devel x86_64 0.3.111-13.el9 CentOS-AppStream 10 k 2024-03-21T10:08:10.149 INFO:teuthology.orchestra.run.smithi100.stdout: libcephfs-devel x86_64 2:19.0.0-2231.g2edfee68.el9 ceph 31 k 2024-03-21T10:08:10.149 INFO:teuthology.orchestra.run.smithi100.stdout: libcephfs2 x86_64 2:19.0.0-2231.g2edfee68.el9 ceph 747 k 2024-03-21T10:08:10.149 INFO:teuthology.orchestra.run.smithi100.stdout: librados-devel x86_64 2:19.0.0-2231.g2edfee68.el9 ceph 127 k 2024-03-21T10:08:10.149 INFO:teuthology.orchestra.run.smithi100.stdout: libtool x86_64 2.4.6-45.el9 CentOS-AppStream 578 k 2024-03-21T10:08:10.150 INFO:teuthology.orchestra.run.smithi100.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-03-21T10:08:10.150 INFO:teuthology.orchestra.run.smithi100.stdout: openssl-devel x86_64 1:3.0.7-27.el9 appstream 4.1 M 2024-03-21T10:08:10.150 INFO:teuthology.orchestra.run.smithi100.stdout: postgresql x86_64 13.11-1.el9 CentOS-AppStream 1.5 M 2024-03-21T10:08:10.150 INFO:teuthology.orchestra.run.smithi100.stdout: postgresql-contrib x86_64 13.11-1.el9 CentOS-AppStream 811 k 2024-03-21T10:08:10.150 INFO:teuthology.orchestra.run.smithi100.stdout: postgresql-server x86_64 13.11-1.el9 CentOS-AppStream 5.7 M 2024-03-21T10:08:10.150 INFO:teuthology.orchestra.run.smithi100.stdout: python3-cephfs x86_64 2:19.0.0-2231.g2edfee68.el9 ceph 161 k 2024-03-21T10:08:10.150 INFO:teuthology.orchestra.run.smithi100.stdout: python3-rados x86_64 2:19.0.0-2231.g2edfee68.el9 ceph 320 k 2024-03-21T10:08:10.150 INFO:teuthology.orchestra.run.smithi100.stdout: python3-rbd x86_64 2:19.0.0-2231.g2edfee68.el9 ceph 297 k 2024-03-21T10:08:10.151 INFO:teuthology.orchestra.run.smithi100.stdout: python3-rgw x86_64 2:19.0.0-2231.g2edfee68.el9 ceph 99 k 2024-03-21T10:08:10.151 INFO:teuthology.orchestra.run.smithi100.stdout: rbd-fuse x86_64 2:19.0.0-2231.g2edfee68.el9 ceph 85 k 2024-03-21T10:08:10.151 INFO:teuthology.orchestra.run.smithi100.stdout: rbd-mirror x86_64 2:19.0.0-2231.g2edfee68.el9 ceph 3.1 M 2024-03-21T10:08:10.151 INFO:teuthology.orchestra.run.smithi100.stdout: rbd-nbd x86_64 2:19.0.0-2231.g2edfee68.el9 ceph 171 k 2024-03-21T10:08:10.151 INFO:teuthology.orchestra.run.smithi100.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-03-21T10:08:10.151 INFO:teuthology.orchestra.run.smithi100.stdout:Upgrading: 2024-03-21T10:08:10.151 INFO:teuthology.orchestra.run.smithi100.stdout: NetworkManager x86_64 1:1.47.2-1.el9 baseos 2.3 M 2024-03-21T10:08:10.151 INFO:teuthology.orchestra.run.smithi100.stdout: NetworkManager-libnm x86_64 1:1.47.2-1.el9 baseos 1.8 M 2024-03-21T10:08:10.151 INFO:teuthology.orchestra.run.smithi100.stdout: NetworkManager-team x86_64 1:1.47.2-1.el9 baseos 42 k 2024-03-21T10:08:10.151 INFO:teuthology.orchestra.run.smithi100.stdout: NetworkManager-tui x86_64 1:1.47.2-1.el9 baseos 248 k 2024-03-21T10:08:10.151 INFO:teuthology.orchestra.run.smithi100.stdout: acl x86_64 2.3.1-4.el9 baseos 71 k 2024-03-21T10:08:10.151 INFO:teuthology.orchestra.run.smithi100.stdout: elfutils-debuginfod-client x86_64 0.190-2.el9 baseos 36 k 2024-03-21T10:08:10.151 INFO:teuthology.orchestra.run.smithi100.stdout: elfutils-libelf x86_64 0.190-2.el9 baseos 192 k 2024-03-21T10:08:10.152 INFO:teuthology.orchestra.run.smithi100.stdout: elfutils-libs x86_64 0.190-2.el9 baseos 256 k 2024-03-21T10:08:10.152 INFO:teuthology.orchestra.run.smithi100.stdout: libacl x86_64 2.3.1-4.el9 baseos 23 k 2024-03-21T10:08:10.152 INFO:teuthology.orchestra.run.smithi100.stdout: libacl-devel x86_64 2.3.1-4.el9 appstream 74 k 2024-03-21T10:08:10.152 INFO:teuthology.orchestra.run.smithi100.stdout: libblkid x86_64 2.37.4-18.el9 baseos 108 k 2024-03-21T10:08:10.152 INFO:teuthology.orchestra.run.smithi100.stdout: libfdisk x86_64 2.37.4-18.el9 baseos 155 k 2024-03-21T10:08:10.152 INFO:teuthology.orchestra.run.smithi100.stdout: libmount x86_64 2.37.4-18.el9 baseos 136 k 2024-03-21T10:08:10.152 INFO:teuthology.orchestra.run.smithi100.stdout: librados2 x86_64 2:19.0.0-2231.g2edfee68.el9 ceph 3.4 M 2024-03-21T10:08:10.152 INFO:teuthology.orchestra.run.smithi100.stdout: librbd1 x86_64 2:19.0.0-2231.g2edfee68.el9 ceph 3.1 M 2024-03-21T10:08:10.152 INFO:teuthology.orchestra.run.smithi100.stdout: libsmartcols x86_64 2.37.4-18.el9 baseos 64 k 2024-03-21T10:08:10.152 INFO:teuthology.orchestra.run.smithi100.stdout: libuuid x86_64 2.37.4-18.el9 baseos 29 k 2024-03-21T10:08:10.152 INFO:teuthology.orchestra.run.smithi100.stdout: openssl x86_64 1:3.0.7-27.el9 baseos 1.2 M 2024-03-21T10:08:10.152 INFO:teuthology.orchestra.run.smithi100.stdout: openssl-libs x86_64 1:3.0.7-27.el9 baseos 2.2 M 2024-03-21T10:08:10.153 INFO:teuthology.orchestra.run.smithi100.stdout: util-linux x86_64 2.37.4-18.el9 baseos 2.3 M 2024-03-21T10:08:10.153 INFO:teuthology.orchestra.run.smithi100.stdout: util-linux-core x86_64 2.37.4-18.el9 baseos 465 k 2024-03-21T10:08:10.153 INFO:teuthology.orchestra.run.smithi100.stdout: util-linux-user x86_64 2.37.4-18.el9 baseos 31 k 2024-03-21T10:08:10.153 INFO:teuthology.orchestra.run.smithi100.stdout: xfsdump x86_64 3.1.12-4.el9 CentOS-BaseOS 321 k 2024-03-21T10:08:10.153 INFO:teuthology.orchestra.run.smithi100.stdout: xfsprogs x86_64 6.3.0-1.el9 baseos 1.1 M 2024-03-21T10:08:10.153 INFO:teuthology.orchestra.run.smithi100.stdout:Installing dependencies: 2024-03-21T10:08:10.153 INFO:teuthology.orchestra.run.smithi100.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-03-21T10:08:10.153 INFO:teuthology.orchestra.run.smithi100.stdout: automake noarch 1.16.2-8.el9 CentOS-AppStream 662 k 2024-03-21T10:08:10.153 INFO:teuthology.orchestra.run.smithi100.stdout: boost-program-options x86_64 1.75.0-8.el9 CentOS-AppStream 106 k 2024-03-21T10:08:10.153 INFO:teuthology.orchestra.run.smithi100.stdout: ceph-common x86_64 2:19.0.0-2231.g2edfee68.el9 ceph 21 M 2024-03-21T10:08:10.153 INFO:teuthology.orchestra.run.smithi100.stdout: ceph-grafana-dashboards noarch 2:19.0.0-2231.g2edfee68.el9 ceph-noarch 34 k 2024-03-21T10:08:10.153 INFO:teuthology.orchestra.run.smithi100.stdout: ceph-mds x86_64 2:19.0.0-2231.g2edfee68.el9 ceph 2.3 M 2024-03-21T10:08:10.153 INFO:teuthology.orchestra.run.smithi100.stdout: ceph-mgr-modules-core noarch 2:19.0.0-2231.g2edfee68.el9 ceph-noarch 248 k 2024-03-21T10:08:10.153 INFO:teuthology.orchestra.run.smithi100.stdout: ceph-mon x86_64 2:19.0.0-2231.g2edfee68.el9 ceph 4.7 M 2024-03-21T10:08:10.154 INFO:teuthology.orchestra.run.smithi100.stdout: ceph-osd x86_64 2:19.0.0-2231.g2edfee68.el9 ceph 17 M 2024-03-21T10:08:10.154 INFO:teuthology.orchestra.run.smithi100.stdout: ceph-prometheus-alerts noarch 2:19.0.0-2231.g2edfee68.el9 ceph-noarch 16 k 2024-03-21T10:08:10.154 INFO:teuthology.orchestra.run.smithi100.stdout: ceph-selinux x86_64 2:19.0.0-2231.g2edfee68.el9 ceph 25 k 2024-03-21T10:08:10.154 INFO:teuthology.orchestra.run.smithi100.stdout: flexiblas x86_64 3.0.4-8.el9 CentOS-AppStream 32 k 2024-03-21T10:08:10.154 INFO:teuthology.orchestra.run.smithi100.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 CentOS-AppStream 3.0 M 2024-03-21T10:08:10.154 INFO:teuthology.orchestra.run.smithi100.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 CentOS-AppStream 17 k 2024-03-21T10:08:10.154 INFO:teuthology.orchestra.run.smithi100.stdout: fuse x86_64 2.9.9-15.el9 CentOS-BaseOS 79 k 2024-03-21T10:08:10.154 INFO:teuthology.orchestra.run.smithi100.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-03-21T10:08:10.154 INFO:teuthology.orchestra.run.smithi100.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-03-21T10:08:10.154 INFO:teuthology.orchestra.run.smithi100.stdout: jq x86_64 1.6-16.el9 baseos 187 k 2024-03-21T10:08:10.154 INFO:teuthology.orchestra.run.smithi100.stdout: libarrow x86_64 9.0.0-10.el9 epel 4.4 M 2024-03-21T10:08:10.154 INFO:teuthology.orchestra.run.smithi100.stdout: libarrow-doc noarch 9.0.0-10.el9 epel 26 k 2024-03-21T10:08:10.154 INFO:teuthology.orchestra.run.smithi100.stdout: libcephsqlite x86_64 2:19.0.0-2231.g2edfee68.el9 ceph 163 k 2024-03-21T10:08:10.155 INFO:teuthology.orchestra.run.smithi100.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-03-21T10:08:10.155 INFO:teuthology.orchestra.run.smithi100.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-03-21T10:08:10.155 INFO:teuthology.orchestra.run.smithi100.stdout: libpmemobj x86_64 1.12.1-1.el9 CentOS-AppStream 159 k 2024-03-21T10:08:10.155 INFO:teuthology.orchestra.run.smithi100.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-03-21T10:08:10.155 INFO:teuthology.orchestra.run.smithi100.stdout: librabbitmq x86_64 0.11.0-7.el9 CentOS-AppStream 44 k 2024-03-21T10:08:10.155 INFO:teuthology.orchestra.run.smithi100.stdout: libradosstriper1 x86_64 2:19.0.0-2231.g2edfee68.el9 ceph 501 k 2024-03-21T10:08:10.155 INFO:teuthology.orchestra.run.smithi100.stdout: librdkafka x86_64 1.6.1-102.el9 CentOS-AppStream 661 k 2024-03-21T10:08:10.155 INFO:teuthology.orchestra.run.smithi100.stdout: librgw2 x86_64 2:19.0.0-2231.g2edfee68.el9 ceph 5.1 M 2024-03-21T10:08:10.155 INFO:teuthology.orchestra.run.smithi100.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-03-21T10:08:10.156 INFO:teuthology.orchestra.run.smithi100.stdout: libzstd-devel x86_64 1.5.1-2.el9 CentOS-AppStream 46 k 2024-03-21T10:08:10.156 INFO:teuthology.orchestra.run.smithi100.stdout: lttng-ust x86_64 2.12.0-6.el9 CentOS-AppStream 282 k 2024-03-21T10:08:10.156 INFO:teuthology.orchestra.run.smithi100.stdout: lua x86_64 5.4.4-4.el9 CentOS-AppStream 187 k 2024-03-21T10:08:10.156 INFO:teuthology.orchestra.run.smithi100.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-03-21T10:08:10.156 INFO:teuthology.orchestra.run.smithi100.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-03-21T10:08:10.156 INFO:teuthology.orchestra.run.smithi100.stdout: oniguruma x86_64 6.9.6-1.el9.5 CentOS-AppStream 217 k 2024-03-21T10:08:10.156 INFO:teuthology.orchestra.run.smithi100.stdout: openblas x86_64 0.3.21-2.el9 CentOS-AppStream 34 k 2024-03-21T10:08:10.156 INFO:teuthology.orchestra.run.smithi100.stdout: openblas-openmp x86_64 0.3.21-2.el9 CentOS-AppStream 4.7 M 2024-03-21T10:08:10.156 INFO:teuthology.orchestra.run.smithi100.stdout: parquet-libs x86_64 9.0.0-10.el9 epel 838 k 2024-03-21T10:08:10.156 INFO:teuthology.orchestra.run.smithi100.stdout: postgresql-private-libs x86_64 13.11-1.el9 CentOS-AppStream 134 k 2024-03-21T10:08:10.156 INFO:teuthology.orchestra.run.smithi100.stdout: protobuf x86_64 3.14.0-13.el9 CentOS-AppStream 1.0 M 2024-03-21T10:08:10.156 INFO:teuthology.orchestra.run.smithi100.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-03-21T10:08:10.156 INFO:teuthology.orchestra.run.smithi100.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-03-21T10:08:10.156 INFO:teuthology.orchestra.run.smithi100.stdout: python3-attrs noarch 20.3.0-7.el9 CentOS-AppStream 78 k 2024-03-21T10:08:10.157 INFO:teuthology.orchestra.run.smithi100.stdout: python3-babel noarch 2.9.1-2.el9 CentOS-AppStream 5.8 M 2024-03-21T10:08:10.157 INFO:teuthology.orchestra.run.smithi100.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-03-21T10:08:10.157 INFO:teuthology.orchestra.run.smithi100.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-03-21T10:08:10.157 INFO:teuthology.orchestra.run.smithi100.stdout: python3-ceph-argparse x86_64 2:19.0.0-2231.g2edfee68.el9 ceph 45 k 2024-03-21T10:08:10.157 INFO:teuthology.orchestra.run.smithi100.stdout: python3-ceph-common x86_64 2:19.0.0-2231.g2edfee68.el9 ceph 131 k 2024-03-21T10:08:10.157 INFO:teuthology.orchestra.run.smithi100.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-03-21T10:08:10.157 INFO:teuthology.orchestra.run.smithi100.stdout: python3-cffi x86_64 1.14.5-5.el9 CentOS-BaseOS 241 k 2024-03-21T10:08:10.157 INFO:teuthology.orchestra.run.smithi100.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-03-21T10:08:10.157 INFO:teuthology.orchestra.run.smithi100.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-03-21T10:08:10.158 INFO:teuthology.orchestra.run.smithi100.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-03-21T10:08:10.158 INFO:teuthology.orchestra.run.smithi100.stdout: python3-colorama noarch 0.4.4-12.el9 epel 35 k 2024-03-21T10:08:10.158 INFO:teuthology.orchestra.run.smithi100.stdout: python3-cryptography x86_64 36.0.1-4.el9 CentOS-BaseOS 1.1 M 2024-03-21T10:08:10.158 INFO:teuthology.orchestra.run.smithi100.stdout: python3-devel x86_64 3.9.17-2.el9 appstream 245 k 2024-03-21T10:08:10.158 INFO:teuthology.orchestra.run.smithi100.stdout: python3-google-auth noarch 1:2.28.2-1.el9 epel 203 k 2024-03-21T10:08:10.158 INFO:teuthology.orchestra.run.smithi100.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-03-21T10:08:10.158 INFO:teuthology.orchestra.run.smithi100.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-03-21T10:08:10.158 INFO:teuthology.orchestra.run.smithi100.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-03-21T10:08:10.158 INFO:teuthology.orchestra.run.smithi100.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-03-21T10:08:10.158 INFO:teuthology.orchestra.run.smithi100.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-03-21T10:08:10.158 INFO:teuthology.orchestra.run.smithi100.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-03-21T10:08:10.158 INFO:teuthology.orchestra.run.smithi100.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-03-21T10:08:10.158 INFO:teuthology.orchestra.run.smithi100.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-03-21T10:08:10.159 INFO:teuthology.orchestra.run.smithi100.stdout: python3-jsonpatch noarch 1.21-16.el9 CentOS-AppStream 24 k 2024-03-21T10:08:10.159 INFO:teuthology.orchestra.run.smithi100.stdout: python3-jsonpointer noarch 2.0-4.el9 CentOS-AppStream 18 k 2024-03-21T10:08:10.159 INFO:teuthology.orchestra.run.smithi100.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-03-21T10:08:10.159 INFO:teuthology.orchestra.run.smithi100.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-03-21T10:08:10.159 INFO:teuthology.orchestra.run.smithi100.stdout: python3-mako noarch 1.1.4-6.el9 CentOS-AppStream 144 k 2024-03-21T10:08:10.159 INFO:teuthology.orchestra.run.smithi100.stdout: python3-markupsafe x86_64 1.1.1-12.el9 CentOS-AppStream 32 k 2024-03-21T10:08:10.159 INFO:teuthology.orchestra.run.smithi100.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-03-21T10:08:10.159 INFO:teuthology.orchestra.run.smithi100.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-03-21T10:08:10.159 INFO:teuthology.orchestra.run.smithi100.stdout: python3-numpy x86_64 1:1.20.1-5.el9 CentOS-AppStream 4.8 M 2024-03-21T10:08:10.159 INFO:teuthology.orchestra.run.smithi100.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 CentOS-AppStream 217 k 2024-03-21T10:08:10.159 INFO:teuthology.orchestra.run.smithi100.stdout: python3-oauthlib noarch 3.1.1-5.el9 CentOS-AppStream 174 k 2024-03-21T10:08:10.160 INFO:teuthology.orchestra.run.smithi100.stdout: python3-packaging noarch 20.9-5.el9 CentOS-AppStream 69 k 2024-03-21T10:08:10.160 INFO:teuthology.orchestra.run.smithi100.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-03-21T10:08:10.160 INFO:teuthology.orchestra.run.smithi100.stdout: python3-ply noarch 3.11-14.el9 CentOS-BaseOS 103 k 2024-03-21T10:08:10.160 INFO:teuthology.orchestra.run.smithi100.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-03-21T10:08:10.160 INFO:teuthology.orchestra.run.smithi100.stdout: python3-prettytable noarch 0.7.2-27.el9 CentOS-AppStream 41 k 2024-03-21T10:08:10.160 INFO:teuthology.orchestra.run.smithi100.stdout: python3-protobuf noarch 3.14.0-13.el9 CentOS-AppStream 238 k 2024-03-21T10:08:10.160 INFO:teuthology.orchestra.run.smithi100.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-03-21T10:08:10.160 INFO:teuthology.orchestra.run.smithi100.stdout: python3-pyasn1 noarch 0.4.8-6.el9 CentOS-AppStream 132 k 2024-03-21T10:08:10.160 INFO:teuthology.orchestra.run.smithi100.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 CentOS-AppStream 211 k 2024-03-21T10:08:10.160 INFO:teuthology.orchestra.run.smithi100.stdout: python3-pycparser noarch 2.20-6.el9 CentOS-BaseOS 124 k 2024-03-21T10:08:10.160 INFO:teuthology.orchestra.run.smithi100.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-03-21T10:08:10.160 INFO:teuthology.orchestra.run.smithi100.stdout: python3-pytz noarch 2021.1-5.el9 CentOS-AppStream 47 k 2024-03-21T10:08:10.161 INFO:teuthology.orchestra.run.smithi100.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-03-21T10:08:10.161 INFO:teuthology.orchestra.run.smithi100.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-03-21T10:08:10.161 INFO:teuthology.orchestra.run.smithi100.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 CentOS-AppStream 43 k 2024-03-21T10:08:10.161 INFO:teuthology.orchestra.run.smithi100.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-03-21T10:08:10.161 INFO:teuthology.orchestra.run.smithi100.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-03-21T10:08:10.161 INFO:teuthology.orchestra.run.smithi100.stdout: python3-scipy x86_64 1.6.2-8.el9 CentOS-AppStream 16 M 2024-03-21T10:08:10.161 INFO:teuthology.orchestra.run.smithi100.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-03-21T10:08:10.161 INFO:teuthology.orchestra.run.smithi100.stdout: python3-toml noarch 0.10.2-6.el9 CentOS-AppStream 39 k 2024-03-21T10:08:10.161 INFO:teuthology.orchestra.run.smithi100.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-03-21T10:08:10.161 INFO:teuthology.orchestra.run.smithi100.stdout: python3-urllib3 noarch 1.26.5-5.el9 CentOS-BaseOS 187 k 2024-03-21T10:08:10.161 INFO:teuthology.orchestra.run.smithi100.stdout: python3-wcwidth noarch 0.2.5-8.el9 CentOS-AppStream 41 k 2024-03-21T10:08:10.162 INFO:teuthology.orchestra.run.smithi100.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-03-21T10:08:10.162 INFO:teuthology.orchestra.run.smithi100.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-03-21T10:08:10.162 INFO:teuthology.orchestra.run.smithi100.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-03-21T10:08:10.162 INFO:teuthology.orchestra.run.smithi100.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-03-21T10:08:10.162 INFO:teuthology.orchestra.run.smithi100.stdout: qatlib x86_64 23.11.0-1.el9 appstream 216 k 2024-03-21T10:08:10.162 INFO:teuthology.orchestra.run.smithi100.stdout: qatzip-libs x86_64 1.1.2-2.el9 appstream 44 k 2024-03-21T10:08:10.162 INFO:teuthology.orchestra.run.smithi100.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-03-21T10:08:10.162 INFO:teuthology.orchestra.run.smithi100.stdout: socat x86_64 1.7.4.1-5.el9 CentOS-AppStream 300 k 2024-03-21T10:08:10.162 INFO:teuthology.orchestra.run.smithi100.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-03-21T10:08:10.163 INFO:teuthology.orchestra.run.smithi100.stdout: uuid x86_64 1.6.2-55.el9 CentOS-AppStream 56 k 2024-03-21T10:08:10.163 INFO:teuthology.orchestra.run.smithi100.stdout: xmlstarlet x86_64 1.6.1-20.el9 CentOS-AppStream 63 k 2024-03-21T10:08:10.163 INFO:teuthology.orchestra.run.smithi100.stdout: zlib-devel x86_64 1.2.11-40.el9 CentOS-AppStream 44 k 2024-03-21T10:08:10.163 INFO:teuthology.orchestra.run.smithi100.stdout:Installing weak dependencies: 2024-03-21T10:08:10.163 INFO:teuthology.orchestra.run.smithi100.stdout: qatlib-service x86_64 23.11.0-1.el9 appstream 35 k 2024-03-21T10:08:10.163 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-03-21T10:08:10.163 INFO:teuthology.orchestra.run.smithi100.stdout:Transaction Summary 2024-03-21T10:08:10.163 INFO:teuthology.orchestra.run.smithi100.stdout:============================================================================================ 2024-03-21T10:08:10.163 INFO:teuthology.orchestra.run.smithi100.stdout:Install 150 Packages 2024-03-21T10:08:10.164 INFO:teuthology.orchestra.run.smithi100.stdout:Upgrade 24 Packages 2024-03-21T10:08:10.164 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-03-21T10:08:10.164 INFO:teuthology.orchestra.run.smithi100.stdout:Total download size: 224 M 2024-03-21T10:08:10.165 INFO:teuthology.orchestra.run.smithi100.stdout:Downloading Packages: 2024-03-21T10:08:10.174 INFO:teuthology.orchestra.run.smithi047.stdout:(33/174): ceph-mgr-modules-core-19.0.0-2231.g2e 831 kB/s | 248 kB 00:00 2024-03-21T10:08:10.233 INFO:teuthology.orchestra.run.smithi047.stdout:(34/174): ceph-mgr-rook-19.0.0-2231.g2edfee68.e 544 kB/s | 49 kB 00:00 2024-03-21T10:08:10.258 INFO:teuthology.orchestra.run.smithi047.stdout:(35/174): ceph-prometheus-alerts-19.0.0-2231.g2 198 kB/s | 16 kB 00:00 2024-03-21T10:08:10.390 INFO:teuthology.orchestra.run.smithi047.stdout:(36/174): cephadm-19.0.0-2231.g2edfee68.el9.noa 5.6 MB/s | 750 kB 00:00 2024-03-21T10:08:10.425 INFO:teuthology.orchestra.run.smithi047.stdout:(37/174): ceph-volume-19.0.0-2231.g2edfee68.el9 1.3 MB/s | 264 kB 00:00 2024-03-21T10:08:10.534 INFO:teuthology.orchestra.run.smithi047.stdout:(38/174): cephfs-top-19.0.0-2231.g2edfee68.el9. 128 kB/s | 18 kB 00:00 2024-03-21T10:08:10.676 INFO:teuthology.orchestra.run.smithi047.stdout:(39/174): ceph-radosgw-19.0.0-2231.g2edfee68.el 1.5 MB/s | 9.9 MB 00:06 2024-03-21T10:08:10.851 INFO:teuthology.orchestra.run.smithi047.stdout:(40/174): automake-1.16.2-8.el9.noarch.rpm 1.5 MB/s | 662 kB 00:00 2024-03-21T10:08:10.901 INFO:teuthology.orchestra.run.smithi047.stdout:(41/174): bison-3.7.4-5.el9.x86_64.rpm 2.5 MB/s | 920 kB 00:00 2024-03-21T10:08:10.918 INFO:teuthology.orchestra.run.smithi047.stdout:(42/174): boost-program-options-1.75.0-8.el9.x8 440 kB/s | 106 kB 00:00 2024-03-21T10:08:10.952 INFO:teuthology.orchestra.run.smithi047.stdout:(43/174): flex-2.6.4-9.el9.x86_64.rpm 3.1 MB/s | 307 kB 00:00 2024-03-21T10:08:10.985 INFO:teuthology.orchestra.run.smithi047.stdout:(44/174): flexiblas-3.0.4-8.el9.x86_64.rpm 501 kB/s | 32 kB 00:00 2024-03-21T10:08:11.144 INFO:teuthology.orchestra.run.smithi047.stdout:(45/174): flexiblas-netlib-3.0.4-8.el9.x86_64.r 13 MB/s | 3.0 MB 00:00 2024-03-21T10:08:11.169 INFO:teuthology.orchestra.run.smithi047.stdout:(46/174): flexiblas-openblas-openmp-3.0.4-8.el9 82 kB/s | 17 kB 00:00 2024-03-21T10:08:11.194 INFO:teuthology.orchestra.run.smithi047.stdout:(47/174): libaio-devel-0.3.111-13.el9.x86_64.rp 49 kB/s | 10 kB 00:00 2024-03-21T10:08:11.320 INFO:teuthology.orchestra.run.smithi047.stdout:(48/174): librabbitmq-0.11.0-7.el9.x86_64.rpm 357 kB/s | 44 kB 00:00 2024-03-21T10:08:11.353 INFO:teuthology.orchestra.run.smithi047.stdout:(49/174): libpmemobj-1.12.1-1.el9.x86_64.rpm 1.0 MB/s | 159 kB 00:00 2024-03-21T10:08:11.387 INFO:teuthology.orchestra.run.smithi047.stdout:(50/174): librdkafka-1.6.1-102.el9.x86_64.rpm 3.4 MB/s | 661 kB 00:00 2024-03-21T10:08:11.429 INFO:teuthology.orchestra.run.smithi047.stdout:(51/174): libtool-2.4.6-45.el9.x86_64.rpm 5.6 MB/s | 578 kB 00:00 2024-03-21T10:08:11.454 INFO:teuthology.orchestra.run.smithi047.stdout:(52/174): libzstd-devel-1.5.1-2.el9.x86_64.rpm 763 kB/s | 46 kB 00:00 2024-03-21T10:08:11.479 INFO:teuthology.orchestra.run.smithi047.stdout:(53/174): lttng-ust-2.12.0-6.el9.x86_64.rpm 3.2 MB/s | 282 kB 00:00 2024-03-21T10:08:11.504 INFO:teuthology.orchestra.run.smithi047.stdout:(54/174): lua-5.4.4-4.el9.x86_64.rpm 2.6 MB/s | 187 kB 00:00 2024-03-21T10:08:11.530 INFO:teuthology.orchestra.run.smithi047.stdout:(55/174): openblas-0.3.21-2.el9.x86_64.rpm 724 kB/s | 34 kB 00:00 2024-03-21T10:08:11.571 INFO:teuthology.orchestra.run.smithi047.stdout:(56/174): oniguruma-6.9.6-1.el9.5.x86_64.rpm 2.4 MB/s | 217 kB 00:00 2024-03-21T10:08:11.660 INFO:teuthology.orchestra.run.smithi100.stdout:(1/174): ceph-19.0.0-2231.g2edfee68.el9.x86_64. 14 kB/s | 6.6 kB 00:00 2024-03-21T10:08:11.772 INFO:teuthology.orchestra.run.smithi047.stdout:(57/174): openblas-openmp-0.3.21-2.el9.x86_64.r 18 MB/s | 4.7 MB 00:00 2024-03-21T10:08:11.839 INFO:teuthology.orchestra.run.smithi047.stdout:(58/174): postgresql-13.11-1.el9.x86_64.rpm 5.1 MB/s | 1.5 MB 00:00 2024-03-21T10:08:11.872 INFO:teuthology.orchestra.run.smithi047.stdout:(59/174): postgresql-contrib-13.11-1.el9.x86_64 2.8 MB/s | 811 kB 00:00 2024-03-21T10:08:11.898 INFO:teuthology.orchestra.run.smithi047.stdout:(60/174): postgresql-private-libs-13.11-1.el9.x 1.2 MB/s | 134 kB 00:00 2024-03-21T10:08:11.981 INFO:teuthology.orchestra.run.smithi047.stdout:(61/174): protobuf-3.14.0-13.el9.x86_64.rpm 9.5 MB/s | 1.0 MB 00:00 2024-03-21T10:08:12.165 INFO:teuthology.orchestra.run.smithi047.stdout:(62/174): postgresql-server-13.11-1.el9.x86_64. 20 MB/s | 5.7 MB 00:00 2024-03-21T10:08:12.207 INFO:teuthology.orchestra.run.smithi047.stdout:(63/174): python3-attrs-20.3.0-7.el9.noarch.rpm 255 kB/s | 78 kB 00:00 2024-03-21T10:08:12.235 INFO:teuthology.orchestra.run.smithi100.stdout:(2/174): ceph-fuse-19.0.0-2231.g2edfee68.el9.x8 1.5 MB/s | 890 kB 00:00 2024-03-21T10:08:12.365 INFO:teuthology.orchestra.run.smithi047.stdout:(64/174): python3-babel-2.9.1-2.el9.noarch.rpm 16 MB/s | 5.8 MB 00:00 2024-03-21T10:08:12.390 INFO:teuthology.orchestra.run.smithi047.stdout:(65/174): python3-jsonpatch-1.21-16.el9.noarch. 117 kB/s | 24 kB 00:00 2024-03-21T10:08:12.394 INFO:teuthology.orchestra.run.smithi100.stdout:(3/174): ceph-base-19.0.0-2231.g2edfee68.el9.x8 4.5 MB/s | 5.4 MB 00:01 2024-03-21T10:08:12.416 INFO:teuthology.orchestra.run.smithi047.stdout:(66/174): python3-jsonpointer-2.0-4.el9.noarch. 95 kB/s | 18 kB 00:00 2024-03-21T10:08:12.427 INFO:teuthology.orchestra.run.smithi100.stdout:(4/174): ceph-immutable-object-cache-19.0.0-223 760 kB/s | 145 kB 00:00 2024-03-21T10:08:12.441 INFO:teuthology.orchestra.run.smithi047.stdout:(67/174): python3-markupsafe-1.1.1-12.el9.x86_6 650 kB/s | 32 kB 00:00 2024-03-21T10:08:12.474 INFO:teuthology.orchestra.run.smithi047.stdout:(68/174): python3-mako-1.1.4-6.el9.noarch.rpm 1.7 MB/s | 144 kB 00:00 2024-03-21T10:08:12.661 INFO:teuthology.orchestra.run.smithi100.stdout:(5/174): ceph-mds-19.0.0-2231.g2edfee68.el9.x86 8.7 MB/s | 2.3 MB 00:00 2024-03-21T10:08:12.692 INFO:teuthology.orchestra.run.smithi047.stdout:(69/174): python3-numpy-1.20.1-5.el9.x86_64.rpm 17 MB/s | 4.8 MB 00:00 2024-03-21T10:08:12.725 INFO:teuthology.orchestra.run.smithi047.stdout:(70/174): python3-numpy-f2py-1.20.1-5.el9.x86_6 786 kB/s | 217 kB 00:00 2024-03-21T10:08:12.750 INFO:teuthology.orchestra.run.smithi047.stdout:(71/174): python3-oauthlib-3.1.1-5.el9.noarch.r 669 kB/s | 174 kB 00:00 2024-03-21T10:08:12.792 INFO:teuthology.orchestra.run.smithi047.stdout:(72/174): python3-packaging-20.9-5.el9.noarch.r 1.6 MB/s | 69 kB 00:00 2024-03-21T10:08:12.817 INFO:teuthology.orchestra.run.smithi047.stdout:(73/174): python3-prettytable-0.7.2-27.el9.noar 626 kB/s | 41 kB 00:00 2024-03-21T10:08:12.851 INFO:teuthology.orchestra.run.smithi047.stdout:(74/174): python3-protobuf-3.14.0-13.el9.noarch 2.3 MB/s | 238 kB 00:00 2024-03-21T10:08:12.876 INFO:teuthology.orchestra.run.smithi047.stdout:(75/174): python3-pyasn1-0.4.8-6.el9.noarch.rpm 2.3 MB/s | 132 kB 00:00 2024-03-21T10:08:12.901 INFO:teuthology.orchestra.run.smithi047.stdout:(76/174): python3-pyasn1-modules-0.4.8-6.el9.no 2.5 MB/s | 211 kB 00:00 2024-03-21T10:08:12.911 INFO:teuthology.orchestra.run.smithi100.stdout:(6/174): ceph-mgr-19.0.0-2231.g2edfee68.el9.x86 3.1 MB/s | 1.5 MB 00:00 2024-03-21T10:08:12.952 INFO:teuthology.orchestra.run.smithi047.stdout:(77/174): python3-pytz-2021.1-5.el9.noarch.rpm 475 kB/s | 47 kB 00:00 2024-03-21T10:08:12.994 INFO:teuthology.orchestra.run.smithi047.stdout:(78/174): python3-requests-oauthlib-1.3.0-12.el 473 kB/s | 43 kB 00:00 2024-03-21T10:08:13.144 INFO:teuthology.orchestra.run.smithi047.stdout:(79/174): python3-toml-0.10.2-6.el9.noarch.rpm 220 kB/s | 39 kB 00:00 2024-03-21T10:08:13.211 INFO:teuthology.orchestra.run.smithi047.stdout:(80/174): python3-wcwidth-0.2.5-8.el9.noarch.rp 207 kB/s | 41 kB 00:00 2024-03-21T10:08:13.245 INFO:teuthology.orchestra.run.smithi100.stdout:(7/174): ceph-mon-19.0.0-2231.g2edfee68.el9.x86 8.1 MB/s | 4.7 MB 00:00 2024-03-21T10:08:13.420 INFO:teuthology.orchestra.run.smithi047.stdout:(81/174): python3-scipy-1.6.2-8.el9.x86_64.rpm 30 MB/s | 16 MB 00:00 2024-03-21T10:08:13.444 INFO:teuthology.orchestra.run.smithi047.stdout:(82/174): socat-1.7.4.1-5.el9.x86_64.rpm 1.0 MB/s | 300 kB 00:00 2024-03-21T10:08:13.470 INFO:teuthology.orchestra.run.smithi047.stdout:(83/174): uuid-1.6.2-55.el9.x86_64.rpm 225 kB/s | 56 kB 00:00 2024-03-21T10:08:13.495 INFO:teuthology.orchestra.run.smithi047.stdout:(84/174): zlib-devel-1.2.11-40.el9.x86_64.rpm 910 kB/s | 44 kB 00:00 2024-03-21T10:08:13.520 INFO:teuthology.orchestra.run.smithi047.stdout:(85/174): xmlstarlet-1.6.1-20.el9.x86_64.rpm 846 kB/s | 63 kB 00:00 2024-03-21T10:08:13.546 INFO:teuthology.orchestra.run.smithi047.stdout:(86/174): fuse-2.9.9-15.el9.x86_64.rpm 1.0 MB/s | 79 kB 00:00 2024-03-21T10:08:13.602 INFO:teuthology.orchestra.run.smithi047.stdout:(87/174): python3-cffi-1.14.5-5.el9.x86_64.rpm 2.2 MB/s | 241 kB 00:00 2024-03-21T10:08:13.638 INFO:teuthology.orchestra.run.smithi047.stdout:(88/174): python3-ply-3.11-14.el9.noarch.rpm 1.1 MB/s | 103 kB 00:00 2024-03-21T10:08:13.696 INFO:teuthology.orchestra.run.smithi047.stdout:(89/174): python3-cryptography-36.0.1-4.el9.x86 7.8 MB/s | 1.1 MB 00:00 2024-03-21T10:08:13.721 INFO:teuthology.orchestra.run.smithi047.stdout:(90/174): python3-pycparser-2.20-6.el9.noarch.r 1.0 MB/s | 124 kB 00:00 2024-03-21T10:08:13.755 INFO:teuthology.orchestra.run.smithi047.stdout:(91/174): python3-urllib3-1.26.5-5.el9.noarch.r 1.7 MB/s | 187 kB 00:00 2024-03-21T10:08:13.854 INFO:teuthology.orchestra.run.smithi100.stdout:(8/174): ceph-common-19.0.0-2231.g2edfee68.el9. 7.7 MB/s | 21 MB 00:02 2024-03-21T10:08:13.912 INFO:teuthology.orchestra.run.smithi100.stdout:(9/174): ceph-selinux-19.0.0-2231.g2edfee68.el9 440 kB/s | 25 kB 00:00 2024-03-21T10:08:13.997 INFO:teuthology.orchestra.run.smithi047.stdout:(92/174): libquadmath-11.4.1-3.el9.x86_64.rpm 811 kB/s | 195 kB 00:00 2024-03-21T10:08:14.022 INFO:teuthology.orchestra.run.smithi047.stdout:(93/174): jq-1.6-16.el9.x86_64.rpm 629 kB/s | 187 kB 00:00 2024-03-21T10:08:14.081 INFO:teuthology.orchestra.run.smithi047.stdout:(94/174): libgfortran-11.4.1-3.el9.x86_64.rpm 2.2 MB/s | 803 kB 00:00 2024-03-21T10:08:14.123 INFO:teuthology.orchestra.run.smithi047.stdout:(95/174): python3-requests-2.25.1-8.el9.noarch. 998 kB/s | 125 kB 00:00 2024-03-21T10:08:14.190 INFO:teuthology.orchestra.run.smithi047.stdout:(96/174): elfutils-libelf-devel-0.190-2.el9.x86 129 kB/s | 21 kB 00:00 2024-03-21T10:08:14.215 INFO:teuthology.orchestra.run.smithi047.stdout:(97/174): libuuid-devel-2.37.4-18.el9.x86_64.rp 191 kB/s | 25 kB 00:00 2024-03-21T10:08:14.316 INFO:teuthology.orchestra.run.smithi047.stdout:(98/174): python3-devel-3.9.17-2.el9.x86_64.rpm 1.9 MB/s | 245 kB 00:00 2024-03-21T10:08:14.349 INFO:teuthology.orchestra.run.smithi047.stdout:(99/174): python3-jinja2-2.11.3-5.el9.noarch.rp 1.8 MB/s | 248 kB 00:00 2024-03-21T10:08:14.383 INFO:teuthology.orchestra.run.smithi047.stdout:(100/174): qatlib-23.11.0-1.el9.x86_64.rpm 3.2 MB/s | 216 kB 00:00 2024-03-21T10:08:14.400 INFO:teuthology.orchestra.run.smithi047.stdout:(101/174): qatlib-service-23.11.0-1.el9.x86_64. 707 kB/s | 35 kB 00:00 2024-03-21T10:08:14.425 INFO:teuthology.orchestra.run.smithi047.stdout:(102/174): qatzip-libs-1.1.2-2.el9.x86_64.rpm 1.1 MB/s | 44 kB 00:00 2024-03-21T10:08:14.450 INFO:teuthology.orchestra.run.smithi047.stdout:(103/174): xfsprogs-devel-6.3.0-1.el9.x86_64.rp 1.8 MB/s | 94 kB 00:00 2024-03-21T10:08:14.592 INFO:teuthology.orchestra.run.smithi047.stdout:(104/174): openssl-devel-3.0.7-27.el9.x86_64.rp 8.8 MB/s | 4.1 MB 00:00 2024-03-21T10:08:14.637 INFO:teuthology.orchestra.run.smithi100.stdout:(10/174): ceph-radosgw-19.0.0-2231.g2edfee68.el 7.1 MB/s | 9.9 MB 00:01 2024-03-21T10:08:14.696 INFO:teuthology.orchestra.run.smithi100.stdout:(11/174): cephfs-mirror-19.0.0-2231.g2edfee68.e 3.8 MB/s | 220 kB 00:00 2024-03-21T10:08:14.763 INFO:teuthology.orchestra.run.smithi100.stdout:(12/174): cephfs-shell-19.0.0-2231.g2edfee68.el 410 kB/s | 27 kB 00:00 2024-03-21T10:08:14.813 INFO:teuthology.orchestra.run.smithi100.stdout:(13/174): libcephfs-devel-19.0.0-2231.g2edfee68 638 kB/s | 31 kB 00:00 2024-03-21T10:08:14.955 INFO:teuthology.orchestra.run.smithi100.stdout:(14/174): libcephfs2-19.0.0-2231.g2edfee68.el9. 5.2 MB/s | 747 kB 00:00 2024-03-21T10:08:14.959 INFO:teuthology.orchestra.run.smithi047.stdout:(105/174): abseil-cpp-20211102.0-3.el9.x86_64.r 1.5 MB/s | 548 kB 00:00 2024-03-21T10:08:15.039 INFO:teuthology.orchestra.run.smithi100.stdout:(15/174): libcephsqlite-19.0.0-2231.g2edfee68.e 1.9 MB/s | 163 kB 00:00 2024-03-21T10:08:15.059 INFO:teuthology.orchestra.run.smithi047.stdout:(106/174): gperftools-libs-2.9.1-2.el9.x86_64.r 3.1 MB/s | 309 kB 00:00 2024-03-21T10:08:15.093 INFO:teuthology.orchestra.run.smithi047.stdout:(107/174): grpc-data-1.46.7-10.el9.noarch.rpm 601 kB/s | 19 kB 00:00 2024-03-21T10:08:15.122 INFO:teuthology.orchestra.run.smithi100.stdout:(16/174): librados-devel-19.0.0-2231.g2edfee68. 1.5 MB/s | 127 kB 00:00 2024-03-21T10:08:15.126 INFO:teuthology.orchestra.run.smithi047.stdout:(108/174): lua-devel-5.4.4-4.el9.x86_64.rpm 32 kB/s | 22 kB 00:00 2024-03-21T10:08:15.264 INFO:teuthology.orchestra.run.smithi100.stdout:(17/174): libradosstriper1-19.0.0-2231.g2edfee6 3.5 MB/s | 501 kB 00:00 2024-03-21T10:08:15.318 INFO:teuthology.orchestra.run.smithi047.stdout:(109/174): libarrow-doc-9.0.0-10.el9.noarch.rpm 136 kB/s | 26 kB 00:00 2024-03-21T10:08:15.369 INFO:teuthology.orchestra.run.smithi047.stdout:(110/174): liboath-2.6.7-2.el9.x86_64.rpm 989 kB/s | 49 kB 00:00 2024-03-21T10:08:15.427 INFO:teuthology.orchestra.run.smithi047.stdout:(111/174): libunwind-1.6.2-1.el9.x86_64.rpm 1.1 MB/s | 67 kB 00:00 2024-03-21T10:08:15.503 INFO:teuthology.orchestra.run.smithi047.stdout:(112/174): luarocks-3.9.2-1.el9.noarch.rpm 2.0 MB/s | 151 kB 00:00 2024-03-21T10:08:15.695 INFO:teuthology.orchestra.run.smithi047.stdout:(113/174): parquet-libs-9.0.0-10.el9.x86_64.rpm 4.3 MB/s | 838 kB 00:00 2024-03-21T10:08:15.739 INFO:teuthology.orchestra.run.smithi100.stdout:(18/174): ceph-osd-19.0.0-2231.g2edfee68.el9.x8 6.0 MB/s | 17 MB 00:02 2024-03-21T10:08:15.781 INFO:teuthology.orchestra.run.smithi100.stdout:(19/174): python3-ceph-argparse-19.0.0-2231.g2e 1.1 MB/s | 45 kB 00:00 2024-03-21T10:08:15.828 INFO:teuthology.orchestra.run.smithi047.stdout:(114/174): python3-asyncssh-2.13.2-5.el9.noarch 4.1 MB/s | 548 kB 00:00 2024-03-21T10:08:15.848 INFO:teuthology.orchestra.run.smithi100.stdout:(20/174): python3-ceph-common-19.0.0-2231.g2edf 2.0 MB/s | 131 kB 00:00 2024-03-21T10:08:15.870 INFO:teuthology.orchestra.run.smithi047.stdout:(115/174): python3-bcrypt-3.2.2-1.el9.x86_64.rp 1.0 MB/s | 43 kB 00:00 2024-03-21T10:08:15.912 INFO:teuthology.orchestra.run.smithi047.stdout:(116/174): python3-cachetools-4.2.4-1.el9.noarc 791 kB/s | 32 kB 00:00 2024-03-21T10:08:15.954 INFO:teuthology.orchestra.run.smithi047.stdout:(117/174): python3-certifi-2021.10.8-2.el9.noar 367 kB/s | 15 kB 00:00 2024-03-21T10:08:15.982 INFO:teuthology.orchestra.run.smithi100.stdout:(21/174): librgw2-19.0.0-2231.g2edfee68.el9.x86 7.2 MB/s | 5.1 MB 00:00 2024-03-21T10:08:16.007 INFO:teuthology.orchestra.run.smithi100.stdout:(22/174): python3-cephfs-19.0.0-2231.g2edfee68. 1.0 MB/s | 161 kB 00:00 2024-03-21T10:08:16.046 INFO:teuthology.orchestra.run.smithi047.stdout:(118/174): libarrow-9.0.0-10.el9.x86_64.rpm 4.6 MB/s | 4.4 MB 00:00 2024-03-21T10:08:16.057 INFO:teuthology.orchestra.run.smithi100.stdout:(23/174): python3-rados-19.0.0-2231.g2edfee68.e 4.2 MB/s | 320 kB 00:00 2024-03-21T10:08:16.096 INFO:teuthology.orchestra.run.smithi047.stdout:(119/174): python3-cheroot-8.6.0-4.el9.noarch.r 1.2 MB/s | 172 kB 00:00 2024-03-21T10:08:16.099 INFO:teuthology.orchestra.run.smithi100.stdout:(24/174): python3-rbd-19.0.0-2231.g2edfee68.el9 3.2 MB/s | 297 kB 00:00 2024-03-21T10:08:16.133 INFO:teuthology.orchestra.run.smithi100.stdout:(25/174): python3-rgw-19.0.0-2231.g2edfee68.el9 1.3 MB/s | 99 kB 00:00 2024-03-21T10:08:16.166 INFO:teuthology.orchestra.run.smithi100.stdout:(26/174): rbd-fuse-19.0.0-2231.g2edfee68.el9.x8 1.3 MB/s | 85 kB 00:00 2024-03-21T10:08:16.180 INFO:teuthology.orchestra.run.smithi047.stdout:(120/174): python3-cherrypy-18.6.1-2.el9.noarch 2.6 MB/s | 358 kB 00:00 2024-03-21T10:08:16.233 INFO:teuthology.orchestra.run.smithi100.stdout:(27/174): rbd-nbd-19.0.0-2231.g2edfee68.el9.x86 2.5 MB/s | 171 kB 00:00 2024-03-21T10:08:16.247 INFO:teuthology.orchestra.run.smithi047.stdout:(121/174): python3-cmd2-2.4.2-2.el9.noarch.rpm 2.0 MB/s | 303 kB 00:00 2024-03-21T10:08:16.272 INFO:teuthology.orchestra.run.smithi047.stdout:(122/174): python3-colorama-0.4.4-12.el9.noarch 387 kB/s | 35 kB 00:00 2024-03-21T10:08:16.309 INFO:teuthology.orchestra.run.smithi100.stdout:(28/174): ceph-grafana-dashboards-19.0.0-2231.g 458 kB/s | 34 kB 00:00 2024-03-21T10:08:16.382 INFO:teuthology.orchestra.run.smithi047.stdout:(123/174): python3-google-auth-2.28.2-1.el9.noa 1.5 MB/s | 203 kB 00:00 2024-03-21T10:08:16.401 INFO:teuthology.orchestra.run.smithi100.stdout:(29/174): ceph-mgr-cephadm-19.0.0-2231.g2edfee6 1.5 MB/s | 140 kB 00:00 2024-03-21T10:08:16.448 INFO:teuthology.orchestra.run.smithi047.stdout:(124/174): python3-grpcio-tools-1.46.7-10.el9.x 2.1 MB/s | 144 kB 00:00 2024-03-21T10:08:16.490 INFO:teuthology.orchestra.run.smithi047.stdout:(125/174): protobuf-compiler-3.14.0-13.el9.x86_ 423 kB/s | 863 kB 00:02 2024-03-21T10:08:16.515 INFO:teuthology.orchestra.run.smithi047.stdout:(126/174): python3-jaraco-8.2.1-3.el9.noarch.rp 161 kB/s | 11 kB 00:00 2024-03-21T10:08:16.541 INFO:teuthology.orchestra.run.smithi047.stdout:(127/174): python3-jaraco-classes-3.2.1-5.el9.n 358 kB/s | 18 kB 00:00 2024-03-21T10:08:16.576 INFO:teuthology.orchestra.run.smithi100.stdout:(30/174): rbd-mirror-19.0.0-2231.g2edfee68.el9. 7.0 MB/s | 3.1 MB 00:00 2024-03-21T10:08:16.657 INFO:teuthology.orchestra.run.smithi047.stdout:(128/174): python3-jaraco-functools-3.5.0-2.el9 436 kB/s | 19 kB 00:00 2024-03-21T10:08:16.699 INFO:teuthology.orchestra.run.smithi047.stdout:(129/174): python3-jaraco-text-3.2.0-6.el9.noar 514 kB/s | 20 kB 00:00 2024-03-21T10:08:16.733 INFO:teuthology.orchestra.run.smithi047.stdout:(130/174): python3-jaraco-collections-3.0.0-8.e 107 kB/s | 23 kB 00:00 2024-03-21T10:08:16.792 INFO:teuthology.orchestra.run.smithi047.stdout:(131/174): python3-logutils-0.3.5-21.el9.noarch 807 kB/s | 46 kB 00:00 2024-03-21T10:08:16.875 INFO:teuthology.orchestra.run.smithi047.stdout:(132/174): python3-more-itertools-8.12.0-2.el9. 957 kB/s | 79 kB 00:00 2024-03-21T10:08:16.917 INFO:teuthology.orchestra.run.smithi047.stdout:(133/174): python3-natsort-7.1.1-5.el9.noarch.r 1.4 MB/s | 58 kB 00:00 2024-03-21T10:08:17.001 INFO:teuthology.orchestra.run.smithi047.stdout:(134/174): python3-pecan-1.4.2-3.el9.noarch.rpm 3.2 MB/s | 272 kB 00:00 2024-03-21T10:08:17.043 INFO:teuthology.orchestra.run.smithi047.stdout:(135/174): python3-portend-3.1.0-2.el9.noarch.r 403 kB/s | 16 kB 00:00 2024-03-21T10:08:17.076 INFO:teuthology.orchestra.run.smithi100.stdout:(31/174): ceph-test-19.0.0-2231.g2edfee68.el9.x 15 MB/s | 48 MB 00:03 2024-03-21T10:08:17.093 INFO:teuthology.orchestra.run.smithi047.stdout:(136/174): python3-pyOpenSSL-21.0.0-1.el9.noarc 1.8 MB/s | 90 kB 00:00 2024-03-21T10:08:17.135 INFO:teuthology.orchestra.run.smithi047.stdout:(137/174): python3-pyperclip-1.8.2-2.el9.noarch 695 kB/s | 28 kB 00:00 2024-03-21T10:08:17.143 INFO:teuthology.orchestra.run.smithi100.stdout:(32/174): ceph-mgr-dashboard-19.0.0-2231.g2edfe 2.4 MB/s | 1.8 MB 00:00 2024-03-21T10:08:17.177 INFO:teuthology.orchestra.run.smithi100.stdout:(33/174): ceph-mgr-modules-core-19.0.0-2231.g2e 2.4 MB/s | 248 kB 00:00 2024-03-21T10:08:17.177 INFO:teuthology.orchestra.run.smithi047.stdout:(138/174): python3-repoze-lru-0.7-16.el9.noarch 756 kB/s | 31 kB 00:00 2024-03-21T10:08:17.202 INFO:teuthology.orchestra.run.smithi100.stdout:(34/174): ceph-mgr-rook-19.0.0-2231.g2edfee68.e 855 kB/s | 49 kB 00:00 2024-03-21T10:08:17.219 INFO:teuthology.orchestra.run.smithi100.stdout:(35/174): ceph-prometheus-alerts-19.0.0-2231.g2 401 kB/s | 16 kB 00:00 2024-03-21T10:08:17.244 INFO:teuthology.orchestra.run.smithi047.stdout:(139/174): python3-grpcio-1.46.7-10.el9.x86_64. 2.1 MB/s | 2.0 MB 00:00 2024-03-21T10:08:17.286 INFO:teuthology.orchestra.run.smithi047.stdout:(140/174): python3-rsa-4.9-2.el9.noarch.rpm 1.4 MB/s | 59 kB 00:00 2024-03-21T10:08:17.294 INFO:teuthology.orchestra.run.smithi100.stdout:(36/174): ceph-volume-19.0.0-2231.g2edfee68.el9 2.8 MB/s | 264 kB 00:00 2024-03-21T10:08:17.319 INFO:teuthology.orchestra.run.smithi047.stdout:(141/174): python3-routes-2.5.1-5.el9.noarch.rp 1.3 MB/s | 188 kB 00:00 2024-03-21T10:08:17.328 INFO:teuthology.orchestra.run.smithi100.stdout:(37/174): cephadm-19.0.0-2231.g2edfee68.el9.noa 6.8 MB/s | 750 kB 00:00 2024-03-21T10:08:17.336 INFO:teuthology.orchestra.run.smithi047.stdout:(142/174): python3-tempora-5.0.0-2.el9.noarch.r 725 kB/s | 36 kB 00:00 2024-03-21T10:08:17.361 INFO:teuthology.orchestra.run.smithi100.stdout:(38/174): cephfs-top-19.0.0-2231.g2edfee68.el9. 276 kB/s | 18 kB 00:00 2024-03-21T10:08:17.362 INFO:teuthology.orchestra.run.smithi047.stdout:(143/174): python3-typing-extensions-4.4.0-2.el 1.2 MB/s | 51 kB 00:00 2024-03-21T10:08:17.429 INFO:teuthology.orchestra.run.smithi047.stdout:(144/174): python3-webob-1.8.7-6.el9.noarch.rpm 2.5 MB/s | 230 kB 00:00 2024-03-21T10:08:17.463 INFO:teuthology.orchestra.run.smithi047.stdout:(145/174): python3-websocket-client-1.2.3-2.el9 893 kB/s | 90 kB 00:00 2024-03-21T10:08:17.522 INFO:teuthology.orchestra.run.smithi047.stdout:(146/174): python3-zc-lockfile-2.0-10.el9.noarc 349 kB/s | 20 kB 00:00 2024-03-21T10:08:17.581 INFO:teuthology.orchestra.run.smithi047.stdout:(147/174): python3-kubernetes-26.1.0-2.el9.noar 1.2 MB/s | 1.0 MB 00:00 2024-03-21T10:08:17.603 INFO:teuthology.orchestra.run.smithi100.stdout:(39/174): ceph-mgr-diskprediction-local-19.0.0- 7.2 MB/s | 7.4 MB 00:01 2024-03-21T10:08:17.614 INFO:teuthology.orchestra.run.smithi047.stdout:(148/174): re2-20211101-3.el9.x86_64.rpm 2.1 MB/s | 192 kB 00:00 2024-03-21T10:08:17.653 INFO:teuthology.orchestra.run.smithi100.stdout:(40/174): automake-1.16.2-8.el9.noarch.rpm 2.0 MB/s | 662 kB 00:00 2024-03-21T10:08:17.665 INFO:teuthology.orchestra.run.smithi047.stdout:(149/174): python3-werkzeug-2.0.3-3.el9.1.noarc 1.8 MB/s | 427 kB 00:00 2024-03-21T10:08:17.679 INFO:teuthology.orchestra.run.smithi100.stdout:(41/174): boost-program-options-1.75.0-8.el9.x8 1.5 MB/s | 106 kB 00:00 2024-03-21T10:08:17.713 INFO:teuthology.orchestra.run.smithi100.stdout:(42/174): flex-2.6.4-9.el9.x86_64.rpm 5.4 MB/s | 307 kB 00:00 2024-03-21T10:08:17.738 INFO:teuthology.orchestra.run.smithi100.stdout:(43/174): flexiblas-3.0.4-8.el9.x86_64.rpm 593 kB/s | 32 kB 00:00 2024-03-21T10:08:17.780 INFO:teuthology.orchestra.run.smithi100.stdout:(44/174): bison-3.7.4-5.el9.x86_64.rpm 2.2 MB/s | 920 kB 00:00 2024-03-21T10:08:17.905 INFO:teuthology.orchestra.run.smithi100.stdout:(45/174): flexiblas-netlib-3.0.4-8.el9.x86_64.r 16 MB/s | 3.0 MB 00:00 2024-03-21T10:08:17.930 INFO:teuthology.orchestra.run.smithi100.stdout:(46/174): flexiblas-openblas-openmp-3.0.4-8.el9 91 kB/s | 17 kB 00:00 2024-03-21T10:08:17.955 INFO:teuthology.orchestra.run.smithi100.stdout:(47/174): libaio-devel-0.3.111-13.el9.x86_64.rp 62 kB/s | 10 kB 00:00 2024-03-21T10:08:17.958 INFO:teuthology.orchestra.run.smithi047.stdout:(150/174): librbd1-19.0.0-2231.g2edfee68.el9.x8 11 MB/s | 3.1 MB 00:00 2024-03-21T10:08:17.987 INFO:teuthology.orchestra.run.smithi157.stdout:CentOS Stream 9 - AppStream 558 kB/s | 19 MB 00:34 2024-03-21T10:08:17.997 INFO:teuthology.orchestra.run.smithi100.stdout:(48/174): libpmemobj-1.12.1-1.el9.x86_64.rpm 3.9 MB/s | 159 kB 00:00 2024-03-21T10:08:17.999 INFO:teuthology.orchestra.run.smithi047.stdout:(151/174): xfsdump-3.1.12-4.el9.x86_64.rpm 7.7 MB/s | 321 kB 00:00 2024-03-21T10:08:18.031 INFO:teuthology.orchestra.run.smithi100.stdout:(49/174): librabbitmq-0.11.0-7.el9.x86_64.rpm 599 kB/s | 44 kB 00:00 2024-03-21T10:08:18.065 INFO:teuthology.orchestra.run.smithi100.stdout:(50/174): librdkafka-1.6.1-102.el9.x86_64.rpm 6.0 MB/s | 661 kB 00:00 2024-03-21T10:08:18.106 INFO:teuthology.orchestra.run.smithi100.stdout:(51/174): libtool-2.4.6-45.el9.x86_64.rpm 5.5 MB/s | 578 kB 00:00 2024-03-21T10:08:18.132 INFO:teuthology.orchestra.run.smithi100.stdout:(52/174): libzstd-devel-1.5.1-2.el9.x86_64.rpm 759 kB/s | 46 kB 00:00 2024-03-21T10:08:18.157 INFO:teuthology.orchestra.run.smithi100.stdout:(53/174): lttng-ust-2.12.0-6.el9.x86_64.rpm 3.2 MB/s | 282 kB 00:00 2024-03-21T10:08:18.191 INFO:teuthology.orchestra.run.smithi047.stdout:(152/174): NetworkManager-1.47.2-1.el9.x86_64.r 13 MB/s | 2.3 MB 00:00 2024-03-21T10:08:18.192 INFO:teuthology.orchestra.run.smithi100.stdout:(54/174): lua-5.4.4-4.el9.x86_64.rpm 2.3 MB/s | 187 kB 00:00 2024-03-21T10:08:18.215 INFO:teuthology.orchestra.run.smithi100.stdout:(55/174): oniguruma-6.9.6-1.el9.5.x86_64.rpm 3.8 MB/s | 217 kB 00:00 2024-03-21T10:08:18.232 INFO:teuthology.orchestra.run.smithi100.stdout:(56/174): openblas-0.3.21-2.el9.x86_64.rpm 470 kB/s | 34 kB 00:00 2024-03-21T10:08:18.341 INFO:teuthology.orchestra.run.smithi047.stdout:(153/174): NetworkManager-libnm-1.47.2-1.el9.x8 12 MB/s | 1.8 MB 00:00 2024-03-21T10:08:18.391 INFO:teuthology.orchestra.run.smithi100.stdout:(57/174): openblas-openmp-0.3.21-2.el9.x86_64.r 24 MB/s | 4.7 MB 00:00 2024-03-21T10:08:18.399 INFO:teuthology.orchestra.run.smithi047.stdout:(154/174): NetworkManager-team-1.47.2-1.el9.x86 735 kB/s | 42 kB 00:00 2024-03-21T10:08:18.466 INFO:teuthology.orchestra.run.smithi100.stdout:(58/174): postgresql-13.11-1.el9.x86_64.rpm 7.0 MB/s | 1.5 MB 00:00 2024-03-21T10:08:18.483 INFO:teuthology.orchestra.run.smithi047.stdout:(155/174): NetworkManager-tui-1.47.2-1.el9.x86_ 3.0 MB/s | 248 kB 00:00 2024-03-21T10:08:18.500 INFO:teuthology.orchestra.run.smithi100.stdout:(59/174): postgresql-contrib-13.11-1.el9.x86_64 3.1 MB/s | 811 kB 00:00 2024-03-21T10:08:18.525 INFO:teuthology.orchestra.run.smithi100.stdout:(60/174): postgresql-private-libs-13.11-1.el9.x 1.1 MB/s | 134 kB 00:00 2024-03-21T10:08:18.583 INFO:teuthology.orchestra.run.smithi047.stdout:(156/174): librados2-19.0.0-2231.g2edfee68.el9. 3.5 MB/s | 3.4 MB 00:00 2024-03-21T10:08:18.584 INFO:teuthology.orchestra.run.smithi100.stdout:(61/174): protobuf-3.14.0-13.el9.x86_64.rpm 12 MB/s | 1.0 MB 00:00 2024-03-21T10:08:18.608 INFO:teuthology.orchestra.run.smithi047.stdout:(157/174): acl-2.3.1-4.el9.x86_64.rpm 575 kB/s | 71 kB 00:00 2024-03-21T10:08:18.659 INFO:teuthology.orchestra.run.smithi047.stdout:(158/174): thrift-0.15.0-2.el9.x86_64.rpm 1.5 MB/s | 1.6 MB 00:01 2024-03-21T10:08:18.684 INFO:teuthology.orchestra.run.smithi047.stdout:(159/174): elfutils-debuginfod-client-0.190-2.e 364 kB/s | 36 kB 00:00 2024-03-21T10:08:18.734 INFO:teuthology.orchestra.run.smithi100.stdout:(62/174): postgresql-server-13.11-1.el9.x86_64. 25 MB/s | 5.7 MB 00:00 2024-03-21T10:08:18.735 INFO:teuthology.orchestra.run.smithi047.stdout:(160/174): libacl-2.3.1-4.el9.x86_64.rpm 467 kB/s | 23 kB 00:00 2024-03-21T10:08:18.759 INFO:teuthology.orchestra.run.smithi100.stdout:(63/174): python3-attrs-20.3.0-7.el9.noarch.rpm 349 kB/s | 78 kB 00:00 2024-03-21T10:08:18.780 INFO:teuthology.orchestra.run.smithi047.stdout:(161/174): elfutils-libelf-0.190-2.el9.x86_64.r 1.1 MB/s | 192 kB 00:00 2024-03-21T10:08:18.818 INFO:teuthology.orchestra.run.smithi047.stdout:(162/174): elfutils-libs-0.190-2.el9.x86_64.rpm 1.6 MB/s | 256 kB 00:00 2024-03-21T10:08:18.860 INFO:teuthology.orchestra.run.smithi047.stdout:(163/174): libblkid-2.37.4-18.el9.x86_64.rpm 871 kB/s | 108 kB 00:00 2024-03-21T10:08:18.893 INFO:teuthology.orchestra.run.smithi100.stdout:(64/174): python3-babel-2.9.1-2.el9.noarch.rpm 20 MB/s | 5.8 MB 00:00 2024-03-21T10:08:18.902 INFO:teuthology.orchestra.run.smithi047.stdout:(164/174): libmount-2.37.4-18.el9.x86_64.rpm 1.6 MB/s | 136 kB 00:00 2024-03-21T10:08:18.918 INFO:teuthology.orchestra.run.smithi100.stdout:(65/174): python3-jsonpatch-1.21-16.el9.noarch. 169 kB/s | 24 kB 00:00 2024-03-21T10:08:18.928 INFO:teuthology.orchestra.run.smithi047.stdout:(165/174): libfdisk-2.37.4-18.el9.x86_64.rpm 1.0 MB/s | 155 kB 00:00 2024-03-21T10:08:18.943 INFO:teuthology.orchestra.run.smithi100.stdout:(66/174): python3-jsonpointer-2.0-4.el9.noarch. 107 kB/s | 18 kB 00:00 2024-03-21T10:08:18.953 INFO:teuthology.orchestra.run.smithi047.stdout:(166/174): libsmartcols-2.37.4-18.el9.x86_64.rp 700 kB/s | 64 kB 00:00 2024-03-21T10:08:18.978 INFO:teuthology.orchestra.run.smithi047.stdout:(167/174): libuuid-2.37.4-18.el9.x86_64.rpm 385 kB/s | 29 kB 00:00 2024-03-21T10:08:18.993 INFO:teuthology.orchestra.run.smithi100.stdout:(67/174): python3-mako-1.1.4-6.el9.noarch.rpm 2.9 MB/s | 144 kB 00:00 2024-03-21T10:08:19.052 INFO:teuthology.orchestra.run.smithi100.stdout:(68/174): python3-markupsafe-1.1.1-12.el9.x86_6 295 kB/s | 32 kB 00:00 2024-03-21T10:08:19.087 INFO:teuthology.orchestra.run.smithi047.stdout:(168/174): openssl-3.0.7-27.el9.x86_64.rpm 7.7 MB/s | 1.2 MB 00:00 2024-03-21T10:08:19.169 INFO:teuthology.orchestra.run.smithi100.stdout:(69/174): python3-numpy-1.20.1-5.el9.x86_64.rpm 21 MB/s | 4.8 MB 00:00 2024-03-21T10:08:19.187 INFO:teuthology.orchestra.run.smithi047.stdout:(169/174): openssl-libs-3.0.7-27.el9.x86_64.rpm 9.3 MB/s | 2.2 MB 00:00 2024-03-21T10:08:19.194 INFO:teuthology.orchestra.run.smithi100.stdout:(70/174): python3-numpy-f2py-1.20.1-5.el9.x86_6 1.1 MB/s | 217 kB 00:00 2024-03-21T10:08:19.228 INFO:teuthology.orchestra.run.smithi100.stdout:(71/174): python3-oauthlib-3.1.1-5.el9.noarch.r 1.0 MB/s | 174 kB 00:00 2024-03-21T10:08:19.246 INFO:teuthology.orchestra.run.smithi047.stdout:(170/174): util-linux-2.37.4-18.el9.x86_64.rpm 8.6 MB/s | 2.3 MB 00:00 2024-03-21T10:08:19.247 INFO:teuthology.orchestra.run.smithi100.stdout:(72/174): python3-packaging-20.9-5.el9.noarch.r 1.4 MB/s | 69 kB 00:00 2024-03-21T10:08:19.261 INFO:teuthology.orchestra.run.smithi100.stdout:(73/174): python3-prettytable-0.7.2-27.el9.noar 640 kB/s | 41 kB 00:00 2024-03-21T10:08:19.279 INFO:teuthology.orchestra.run.smithi047.stdout:(171/174): util-linux-core-2.37.4-18.el9.x86_64 2.4 MB/s | 465 kB 00:00 2024-03-21T10:08:19.295 INFO:teuthology.orchestra.run.smithi100.stdout:(74/174): python3-protobuf-3.14.0-13.el9.noarch 3.6 MB/s | 238 kB 00:00 2024-03-21T10:08:19.304 INFO:teuthology.orchestra.run.smithi047.stdout:(172/174): util-linux-user-2.37.4-18.el9.x86_64 267 kB/s | 31 kB 00:00 2024-03-21T10:08:19.328 INFO:teuthology.orchestra.run.smithi100.stdout:(75/174): python3-pyasn1-modules-0.4.8-6.el9.no 3.2 MB/s | 211 kB 00:00 2024-03-21T10:08:19.354 INFO:teuthology.orchestra.run.smithi100.stdout:(76/174): python3-pyasn1-0.4.8-6.el9.noarch.rpm 1.4 MB/s | 132 kB 00:00 2024-03-21T10:08:19.363 INFO:teuthology.orchestra.run.smithi047.stdout:(173/174): libacl-devel-2.3.1-4.el9.x86_64.rpm 899 kB/s | 74 kB 00:00 2024-03-21T10:08:19.370 INFO:teuthology.orchestra.run.smithi100.stdout:(77/174): python3-pytz-2021.1-5.el9.noarch.rpm 648 kB/s | 47 kB 00:00 2024-03-21T10:08:19.387 INFO:teuthology.orchestra.run.smithi100.stdout:(78/174): python3-requests-oauthlib-1.3.0-12.el 754 kB/s | 43 kB 00:00 2024-03-21T10:08:19.404 INFO:teuthology.orchestra.run.smithi100.stdout:(79/174): python3-toml-0.10.2-6.el9.noarch.rpm 1.2 MB/s | 39 kB 00:00 2024-03-21T10:08:19.463 INFO:teuthology.orchestra.run.smithi047.stdout:(174/174): xfsprogs-6.3.0-1.el9.x86_64.rpm 5.0 MB/s | 1.1 MB 00:00 2024-03-21T10:08:19.478 INFO:teuthology.orchestra.run.smithi047.stdout:-------------------------------------------------------------------------------- 2024-03-21T10:08:19.478 INFO:teuthology.orchestra.run.smithi047.stdout:Total 13 MB/s | 224 MB 00:17 2024-03-21T10:08:19.522 INFO:teuthology.orchestra.run.smithi100.stdout:(80/174): python3-wcwidth-0.2.5-8.el9.noarch.rp 309 kB/s | 41 kB 00:00 2024-03-21T10:08:19.755 INFO:teuthology.orchestra.run.smithi100.stdout:(81/174): python3-scipy-1.6.2-8.el9.x86_64.rpm 41 MB/s | 16 MB 00:00 2024-03-21T10:08:19.788 INFO:teuthology.orchestra.run.smithi100.stdout:(82/174): socat-1.7.4.1-5.el9.x86_64.rpm 814 kB/s | 300 kB 00:00 2024-03-21T10:08:19.805 INFO:teuthology.orchestra.run.smithi100.stdout:(83/174): uuid-1.6.2-55.el9.x86_64.rpm 208 kB/s | 56 kB 00:00 2024-03-21T10:08:19.822 INFO:teuthology.orchestra.run.smithi100.stdout:(84/174): xmlstarlet-1.6.1-20.el9.x86_64.rpm 980 kB/s | 63 kB 00:00 2024-03-21T10:08:19.839 INFO:teuthology.orchestra.run.smithi100.stdout:(85/174): zlib-devel-1.2.11-40.el9.x86_64.rpm 1.3 MB/s | 44 kB 00:00 2024-03-21T10:08:19.864 INFO:teuthology.orchestra.run.smithi100.stdout:(86/174): fuse-2.9.9-15.el9.x86_64.rpm 1.3 MB/s | 79 kB 00:00 2024-03-21T10:08:19.898 INFO:teuthology.orchestra.run.smithi100.stdout:(87/174): python3-cffi-1.14.5-5.el9.x86_64.rpm 3.2 MB/s | 241 kB 00:00 2024-03-21T10:08:19.923 INFO:teuthology.orchestra.run.smithi100.stdout:(88/174): python3-ply-3.11-14.el9.noarch.rpm 1.8 MB/s | 103 kB 00:00 2024-03-21T10:08:19.974 INFO:teuthology.orchestra.run.smithi100.stdout:(89/174): python3-cryptography-36.0.1-4.el9.x86 11 MB/s | 1.1 MB 00:00 2024-03-21T10:08:19.999 INFO:teuthology.orchestra.run.smithi100.stdout:(90/174): python3-pycparser-2.20-6.el9.noarch.r 1.2 MB/s | 124 kB 00:00 2024-03-21T10:08:20.032 INFO:teuthology.orchestra.run.smithi100.stdout:(91/174): python3-urllib3-1.26.5-5.el9.noarch.r 1.9 MB/s | 187 kB 00:00 2024-03-21T10:08:20.399 INFO:teuthology.orchestra.run.smithi100.stdout:(92/174): jq-1.6-16.el9.x86_64.rpm 471 kB/s | 187 kB 00:00 2024-03-21T10:08:20.433 INFO:teuthology.orchestra.run.smithi100.stdout:(93/174): libquadmath-11.4.1-3.el9.x86_64.rpm 489 kB/s | 195 kB 00:00 2024-03-21T10:08:20.491 INFO:teuthology.orchestra.run.smithi100.stdout:(94/174): libgfortran-11.4.1-3.el9.x86_64.rpm 1.6 MB/s | 803 kB 00:00 2024-03-21T10:08:20.525 INFO:teuthology.orchestra.run.smithi100.stdout:(95/174): python3-requests-2.25.1-8.el9.noarch. 1.0 MB/s | 125 kB 00:00 2024-03-21T10:08:20.709 INFO:teuthology.orchestra.run.smithi100.stdout:(96/174): elfutils-libelf-devel-0.190-2.el9.x86 78 kB/s | 21 kB 00:00 2024-03-21T10:08:20.792 INFO:teuthology.orchestra.run.smithi100.stdout:(97/174): libuuid-devel-2.37.4-18.el9.x86_64.rp 85 kB/s | 25 kB 00:00 2024-03-21T10:08:20.959 INFO:teuthology.orchestra.run.smithi047.stdout:Running transaction check 2024-03-21T10:08:21.043 INFO:teuthology.orchestra.run.smithi100.stdout:(98/174): python3-devel-3.9.17-2.el9.x86_64.rpm 738 kB/s | 245 kB 00:00 2024-03-21T10:08:21.134 INFO:teuthology.orchestra.run.smithi100.stdout:(99/174): qatlib-23.11.0-1.el9.x86_64.rpm 2.3 MB/s | 216 kB 00:00 2024-03-21T10:08:21.185 INFO:teuthology.orchestra.run.smithi100.stdout:(100/174): qatlib-service-23.11.0-1.el9.x86_64. 709 kB/s | 35 kB 00:00 2024-03-21T10:08:21.358 INFO:teuthology.orchestra.run.smithi047.stdout:Transaction check succeeded. 2024-03-21T10:08:21.358 INFO:teuthology.orchestra.run.smithi047.stdout:Running transaction test 2024-03-21T10:08:21.385 INFO:teuthology.orchestra.run.smithi100.stdout:(101/174): qatzip-libs-1.1.2-2.el9.x86_64.rpm 223 kB/s | 44 kB 00:00 2024-03-21T10:08:21.443 INFO:teuthology.orchestra.run.smithi100.stdout:(102/174): python3-jinja2-2.11.3-5.el9.noarch.r 381 kB/s | 248 kB 00:00 2024-03-21T10:08:21.477 INFO:teuthology.orchestra.run.smithi100.stdout:(103/174): xfsprogs-devel-6.3.0-1.el9.x86_64.rp 1.0 MB/s | 94 kB 00:00 2024-03-21T10:08:21.711 INFO:teuthology.orchestra.run.smithi100.stdout:(104/174): lua-devel-5.4.4-4.el9.x86_64.rpm 84 kB/s | 22 kB 00:00 2024-03-21T10:08:21.936 INFO:teuthology.orchestra.run.smithi100.stdout:(105/174): abseil-cpp-20211102.0-3.el9.x86_64.r 2.4 MB/s | 548 kB 00:00 2024-03-21T10:08:22.028 INFO:teuthology.orchestra.run.smithi100.stdout:(106/174): gperftools-libs-2.9.1-2.el9.x86_64.r 3.3 MB/s | 309 kB 00:00 2024-03-21T10:08:22.070 INFO:teuthology.orchestra.run.smithi100.stdout:(107/174): grpc-data-1.46.7-10.el9.noarch.rpm 474 kB/s | 19 kB 00:00 2024-03-21T10:08:22.246 INFO:teuthology.orchestra.run.smithi157.stdout:CentOS Stream 9 - CRB 12 MB/s | 6.1 MB 00:00 2024-03-21T10:08:22.254 INFO:teuthology.orchestra.run.smithi100.stdout:(108/174): protobuf-compiler-3.14.0-13.el9.x86_ 1.1 MB/s | 863 kB 00:00 2024-03-21T10:08:22.329 INFO:teuthology.orchestra.run.smithi100.stdout:(109/174): libarrow-doc-9.0.0-10.el9.noarch.rpm 351 kB/s | 26 kB 00:00 2024-03-21T10:08:22.430 INFO:teuthology.orchestra.run.smithi100.stdout:(110/174): liboath-2.6.7-2.el9.x86_64.rpm 488 kB/s | 49 kB 00:00 2024-03-21T10:08:22.488 INFO:teuthology.orchestra.run.smithi100.stdout:(111/174): libunwind-1.6.2-1.el9.x86_64.rpm 1.1 MB/s | 67 kB 00:00 2024-03-21T10:08:22.555 INFO:teuthology.orchestra.run.smithi100.stdout:(112/174): luarocks-3.9.2-1.el9.noarch.rpm 2.2 MB/s | 151 kB 00:00 2024-03-21T10:08:22.738 INFO:teuthology.orchestra.run.smithi100.stdout:(113/174): parquet-libs-9.0.0-10.el9.x86_64.rpm 4.5 MB/s | 838 kB 00:00 2024-03-21T10:08:22.897 INFO:teuthology.orchestra.run.smithi100.stdout:(114/174): python3-asyncssh-2.13.2-5.el9.noarch 3.4 MB/s | 548 kB 00:00 2024-03-21T10:08:22.939 INFO:teuthology.orchestra.run.smithi100.stdout:(115/174): python3-bcrypt-3.2.2-1.el9.x86_64.rp 1.0 MB/s | 43 kB 00:00 2024-03-21T10:08:22.981 INFO:teuthology.orchestra.run.smithi100.stdout:(116/174): python3-cachetools-4.2.4-1.el9.noarc 787 kB/s | 32 kB 00:00 2024-03-21T10:08:23.023 INFO:teuthology.orchestra.run.smithi100.stdout:(117/174): python3-certifi-2021.10.8-2.el9.noar 366 kB/s | 15 kB 00:00 2024-03-21T10:08:23.115 INFO:teuthology.orchestra.run.smithi100.stdout:(118/174): libarrow-9.0.0-10.el9.x86_64.rpm 4.2 MB/s | 4.4 MB 00:01 2024-03-21T10:08:23.149 INFO:teuthology.orchestra.run.smithi100.stdout:(119/174): python3-cheroot-8.6.0-4.el9.noarch.r 1.3 MB/s | 172 kB 00:00 2024-03-21T10:08:23.215 INFO:teuthology.orchestra.run.smithi100.stdout:(120/174): python3-cherrypy-18.6.1-2.el9.noarch 3.5 MB/s | 358 kB 00:00 2024-03-21T10:08:23.250 INFO:teuthology.orchestra.run.smithi100.stdout:(121/174): python3-cmd2-2.4.2-2.el9.noarch.rpm 3.0 MB/s | 303 kB 00:00 2024-03-21T10:08:23.282 INFO:teuthology.orchestra.run.smithi100.stdout:(122/174): python3-colorama-0.4.4-12.el9.noarch 534 kB/s | 35 kB 00:00 2024-03-21T10:08:23.333 INFO:teuthology.orchestra.run.smithi100.stdout:(123/174): python3-google-auth-2.28.2-1.el9.noa 2.4 MB/s | 203 kB 00:00 2024-03-21T10:08:23.391 INFO:teuthology.orchestra.run.smithi100.stdout:(124/174): python3-grpcio-tools-1.46.7-10.el9.x 2.4 MB/s | 144 kB 00:00 2024-03-21T10:08:23.434 INFO:teuthology.orchestra.run.smithi100.stdout:(125/174): python3-jaraco-8.2.1-3.el9.noarch.rp 261 kB/s | 11 kB 00:00 2024-03-21T10:08:23.475 INFO:teuthology.orchestra.run.smithi100.stdout:(126/174): python3-jaraco-classes-3.2.1-5.el9.n 433 kB/s | 18 kB 00:00 2024-03-21T10:08:23.592 INFO:teuthology.orchestra.run.smithi100.stdout:(127/174): python3-jaraco-collections-3.0.0-8.e 200 kB/s | 23 kB 00:00 2024-03-21T10:08:23.634 INFO:teuthology.orchestra.run.smithi100.stdout:(128/174): python3-jaraco-functools-3.5.0-2.el9 476 kB/s | 19 kB 00:00 2024-03-21T10:08:23.676 INFO:teuthology.orchestra.run.smithi100.stdout:(129/174): python3-jaraco-text-3.2.0-6.el9.noar 480 kB/s | 20 kB 00:00 2024-03-21T10:08:23.816 INFO:teuthology.orchestra.run.smithi047.stdout:Transaction test succeeded. 2024-03-21T10:08:23.817 INFO:teuthology.orchestra.run.smithi047.stdout:Running transaction 2024-03-21T10:08:23.906 INFO:teuthology.orchestra.run.smithi157.stdout:CentOS Stream 9 - Extras packages 44 kB/s | 16 kB 00:00 2024-03-21T10:08:23.926 INFO:teuthology.orchestra.run.smithi100.stdout:(130/174): python3-kubernetes-26.1.0-2.el9.noar 4.1 MB/s | 1.0 MB 00:00 2024-03-21T10:08:23.985 INFO:teuthology.orchestra.run.smithi100.stdout:(131/174): python3-grpcio-1.46.7-10.el9.x86_64. 2.9 MB/s | 2.0 MB 00:00 2024-03-21T10:08:24.002 INFO:teuthology.orchestra.run.smithi100.stdout:(132/174): python3-logutils-0.3.5-21.el9.noarch 623 kB/s | 46 kB 00:00 2024-03-21T10:08:24.036 INFO:teuthology.orchestra.run.smithi100.stdout:(133/174): python3-more-itertools-8.12.0-2.el9. 1.6 MB/s | 79 kB 00:00 2024-03-21T10:08:24.086 INFO:teuthology.orchestra.run.smithi100.stdout:(134/174): python3-natsort-7.1.1-5.el9.noarch.r 692 kB/s | 58 kB 00:00 2024-03-21T10:08:24.128 INFO:teuthology.orchestra.run.smithi100.stdout:(135/174): python3-portend-3.1.0-2.el9.noarch.r 402 kB/s | 16 kB 00:00 2024-03-21T10:08:24.170 INFO:teuthology.orchestra.run.smithi100.stdout:(136/174): python3-pecan-1.4.2-3.el9.noarch.rpm 2.0 MB/s | 272 kB 00:00 2024-03-21T10:08:24.203 INFO:teuthology.orchestra.run.smithi100.stdout:(137/174): python3-pyOpenSSL-21.0.0-1.el9.noarc 1.2 MB/s | 90 kB 00:00 2024-03-21T10:08:24.229 INFO:teuthology.orchestra.run.smithi100.stdout:(138/174): python3-pyperclip-1.8.2-2.el9.noarch 488 kB/s | 28 kB 00:00 2024-03-21T10:08:24.254 INFO:teuthology.orchestra.run.smithi100.stdout:(139/174): python3-repoze-lru-0.7-16.el9.noarch 619 kB/s | 31 kB 00:00 2024-03-21T10:08:24.296 INFO:teuthology.orchestra.run.smithi100.stdout:(140/174): python3-routes-2.5.1-5.el9.noarch.rp 2.8 MB/s | 188 kB 00:00 2024-03-21T10:08:24.321 INFO:teuthology.orchestra.run.smithi100.stdout:(141/174): python3-rsa-4.9-2.el9.noarch.rpm 893 kB/s | 59 kB 00:00 2024-03-21T10:08:24.347 INFO:teuthology.orchestra.run.smithi100.stdout:(142/174): python3-tempora-5.0.0-2.el9.noarch.r 721 kB/s | 36 kB 00:00 2024-03-21T10:08:24.372 INFO:teuthology.orchestra.run.smithi100.stdout:(143/174): python3-typing-extensions-4.4.0-2.el 1.0 MB/s | 51 kB 00:00 2024-03-21T10:08:24.414 INFO:teuthology.orchestra.run.smithi100.stdout:(144/174): python3-webob-1.8.7-6.el9.noarch.rpm 3.4 MB/s | 230 kB 00:00 2024-03-21T10:08:24.439 INFO:teuthology.orchestra.run.smithi100.stdout:(145/174): python3-websocket-client-1.2.3-2.el9 1.3 MB/s | 90 kB 00:00 2024-03-21T10:08:24.481 INFO:teuthology.orchestra.run.smithi100.stdout:(146/174): python3-zc-lockfile-2.0-10.el9.noarc 490 kB/s | 20 kB 00:00 2024-03-21T10:08:24.548 INFO:teuthology.orchestra.run.smithi100.stdout:(147/174): python3-werkzeug-2.0.3-3.el9.1.noarc 3.1 MB/s | 427 kB 00:00 2024-03-21T10:08:24.573 INFO:teuthology.orchestra.run.smithi100.stdout:(148/174): re2-20211101-3.el9.x86_64.rpm 2.1 MB/s | 192 kB 00:00 2024-03-21T10:08:25.049 INFO:teuthology.orchestra.run.smithi100.stdout:(149/174): librados2-19.0.0-2231.g2edfee68.el9. 7.2 MB/s | 3.4 MB 00:00 2024-03-21T10:08:25.241 INFO:teuthology.orchestra.run.smithi100.stdout:(150/174): librbd1-19.0.0-2231.g2edfee68.el9.x8 16 MB/s | 3.1 MB 00:00 2024-03-21T10:08:25.408 INFO:teuthology.orchestra.run.smithi100.stdout:(151/174): thrift-0.15.0-2.el9.x86_64.rpm 1.8 MB/s | 1.6 MB 00:00 2024-03-21T10:08:25.441 INFO:teuthology.orchestra.run.smithi100.stdout:(152/174): xfsdump-3.1.12-4.el9.x86_64.rpm 1.6 MB/s | 321 kB 00:00 2024-03-21T10:08:25.579 INFO:teuthology.orchestra.run.smithi157.stdout:Extra Packages for Enterprise Linux 14 MB/s | 21 MB 00:01 2024-03-21T10:08:25.833 INFO:teuthology.orchestra.run.smithi100.stdout:(153/174): NetworkManager-libnm-1.47.2-1.el9.x8 4.7 MB/s | 1.8 MB 00:00 2024-03-21T10:08:25.908 INFO:teuthology.orchestra.run.smithi100.stdout:(154/174): NetworkManager-1.47.2-1.el9.x86_64.r 4.6 MB/s | 2.3 MB 00:00 2024-03-21T10:08:25.934 INFO:teuthology.orchestra.run.smithi100.stdout:(155/174): NetworkManager-team-1.47.2-1.el9.x86 422 kB/s | 42 kB 00:00 2024-03-21T10:08:25.986 INFO:teuthology.orchestra.run.smithi100.stdout:(156/174): NetworkManager-tui-1.47.2-1.el9.x86_ 3.3 MB/s | 248 kB 00:00 2024-03-21T10:08:26.009 INFO:teuthology.orchestra.run.smithi100.stdout:(157/174): acl-2.3.1-4.el9.x86_64.rpm 958 kB/s | 71 kB 00:00 2024-03-21T10:08:26.051 INFO:teuthology.orchestra.run.smithi100.stdout:(158/174): elfutils-debuginfod-client-0.190-2.e 550 kB/s | 36 kB 00:00 2024-03-21T10:08:26.085 INFO:teuthology.orchestra.run.smithi100.stdout:(159/174): elfutils-libelf-0.190-2.el9.x86_64.r 2.5 MB/s | 192 kB 00:00 2024-03-21T10:08:26.143 INFO:teuthology.orchestra.run.smithi100.stdout:(160/174): elfutils-libs-0.190-2.el9.x86_64.rpm 2.7 MB/s | 256 kB 00:00 2024-03-21T10:08:26.169 INFO:teuthology.orchestra.run.smithi100.stdout:(161/174): libacl-2.3.1-4.el9.x86_64.rpm 278 kB/s | 23 kB 00:00 2024-03-21T10:08:26.211 INFO:teuthology.orchestra.run.smithi100.stdout:(162/174): libblkid-2.37.4-18.el9.x86_64.rpm 1.6 MB/s | 108 kB 00:00 2024-03-21T10:08:26.244 INFO:teuthology.orchestra.run.smithi100.stdout:(163/174): libfdisk-2.37.4-18.el9.x86_64.rpm 2.0 MB/s | 155 kB 00:00 2024-03-21T10:08:26.294 INFO:teuthology.orchestra.run.smithi100.stdout:(164/174): libmount-2.37.4-18.el9.x86_64.rpm 1.6 MB/s | 136 kB 00:00 2024-03-21T10:08:26.323 INFO:teuthology.orchestra.run.smithi100.stdout:(165/174): libsmartcols-2.37.4-18.el9.x86_64.rp 823 kB/s | 64 kB 00:00 2024-03-21T10:08:26.365 INFO:teuthology.orchestra.run.smithi100.stdout:(166/174): libuuid-2.37.4-18.el9.x86_64.rpm 413 kB/s | 29 kB 00:00 2024-03-21T10:08:26.440 INFO:teuthology.orchestra.run.smithi100.stdout:(167/174): openssl-3.0.7-27.el9.x86_64.rpm 10 MB/s | 1.2 MB 00:00 2024-03-21T10:08:26.549 INFO:teuthology.orchestra.run.smithi100.stdout:(168/174): openssl-libs-3.0.7-27.el9.x86_64.rpm 12 MB/s | 2.2 MB 00:00 2024-03-21T10:08:26.633 INFO:teuthology.orchestra.run.smithi100.stdout:(169/174): util-linux-2.37.4-18.el9.x86_64.rpm 12 MB/s | 2.3 MB 00:00 2024-03-21T10:08:26.675 INFO:teuthology.orchestra.run.smithi100.stdout:(170/174): util-linux-core-2.37.4-18.el9.x86_64 3.6 MB/s | 465 kB 00:00 2024-03-21T10:08:26.842 INFO:teuthology.orchestra.run.smithi100.stdout:(171/174): util-linux-user-2.37.4-18.el9.x86_64 150 kB/s | 31 kB 00:00 2024-03-21T10:08:26.909 INFO:teuthology.orchestra.run.smithi100.stdout:(172/174): xfsprogs-6.3.0-1.el9.x86_64.rpm 4.6 MB/s | 1.1 MB 00:00 2024-03-21T10:08:27.001 INFO:teuthology.orchestra.run.smithi100.stdout:(173/174): libacl-devel-2.3.1-4.el9.x86_64.rpm 471 kB/s | 74 kB 00:00 2024-03-21T10:08:29.759 INFO:teuthology.orchestra.run.smithi100.stdout:(174/174): openssl-devel-3.0.7-27.el9.x86_64.rp 458 kB/s | 4.1 MB 00:09 2024-03-21T10:08:29.769 INFO:teuthology.orchestra.run.smithi100.stdout:-------------------------------------------------------------------------------- 2024-03-21T10:08:29.770 INFO:teuthology.orchestra.run.smithi100.stdout:Total 11 MB/s | 224 MB 00:19 2024-03-21T10:08:31.189 INFO:teuthology.orchestra.run.smithi100.stdout:Running transaction check 2024-03-21T10:08:31.440 INFO:teuthology.orchestra.run.smithi157.stdout:lab-extras 33 kB/s | 1.7 kB 00:00 2024-03-21T10:08:31.532 INFO:teuthology.orchestra.run.smithi047.stdout: Preparing : 1/1 2024-03-21T10:08:31.571 INFO:teuthology.orchestra.run.smithi100.stdout:Transaction check succeeded. 2024-03-21T10:08:31.571 INFO:teuthology.orchestra.run.smithi100.stdout:Running transaction test 2024-03-21T10:08:31.681 INFO:teuthology.orchestra.run.smithi047.stdout: Upgrading : openssl-libs-1:3.0.7-27.el9.x86_64 1/198 2024-03-21T10:08:31.728 INFO:teuthology.orchestra.run.smithi047.stdout: Upgrading : libuuid-2.37.4-18.el9.x86_64 2/198 2024-03-21T10:08:31.765 INFO:teuthology.orchestra.run.smithi047.stdout: Upgrading : libblkid-2.37.4-18.el9.x86_64 3/198 2024-03-21T10:08:31.811 INFO:teuthology.orchestra.run.smithi047.stdout: Running scriptlet: libblkid-2.37.4-18.el9.x86_64 3/198 2024-03-21T10:08:31.903 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 4/198 2024-03-21T10:08:32.014 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 5/198 2024-03-21T10:08:32.068 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : thrift-0.15.0-2.el9.x86_64 6/198 2024-03-21T10:08:32.114 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 7/198 2024-03-21T10:08:32.220 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 8/198 2024-03-21T10:08:32.697 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 9/198 2024-03-21T10:08:32.747 INFO:teuthology.orchestra.run.smithi047.stdout: Upgrading : librados2-2:19.0.0-2231.g2edfee68.el9.x86_64 10/198 2024-03-21T10:08:32.902 INFO:teuthology.orchestra.run.smithi047.stdout: Running scriptlet: librados2-2:19.0.0-2231.g2edfee68.el9.x86_64 10/198 2024-03-21T10:08:32.932 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : libcephfs2-2:19.0.0-2231.g2edfee68.el9.x86_64 11/198 2024-03-21T10:08:33.066 INFO:teuthology.orchestra.run.smithi047.stdout: Running scriptlet: libcephfs2-2:19.0.0-2231.g2edfee68.el9.x86_64 11/198 2024-03-21T10:08:33.099 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-rados-2:19.0.0-2231.g2edfee68.el9.x86_64 12/198 2024-03-21T10:08:33.178 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : postgresql-private-libs-13.11-1.el9.x86_64 13/198 2024-03-21T10:08:33.493 INFO:teuthology.orchestra.run.smithi047.stdout: Upgrading : elfutils-libelf-0.190-2.el9.x86_64 14/198 2024-03-21T10:08:33.545 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : postgresql-13.11-1.el9.x86_64 15/198 2024-03-21T10:08:33.579 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : libcephsqlite-2:19.0.0-2231.g2edfee68.el9.x86_64 16/198 2024-03-21T10:08:33.734 INFO:teuthology.orchestra.run.smithi047.stdout: Running scriptlet: libcephsqlite-2:19.0.0-2231.g2edfee68.el9.x86_64 16/198 2024-03-21T10:08:33.887 INFO:teuthology.orchestra.run.smithi100.stdout:Transaction test succeeded. 2024-03-21T10:08:33.888 INFO:teuthology.orchestra.run.smithi100.stdout:Running transaction 2024-03-21T10:08:34.012 INFO:teuthology.orchestra.run.smithi157.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-03-21T10:08:34.013 INFO:teuthology.orchestra.run.smithi157.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-03-21T10:08:34.032 INFO:teuthology.orchestra.run.smithi157.stdout:Package NetworkManager-1:1.44.0-2.el9.x86_64 is already installed. 2024-03-21T10:08:34.033 INFO:teuthology.orchestra.run.smithi157.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-03-21T10:08:34.034 INFO:teuthology.orchestra.run.smithi157.stdout:Package util-linux-2.37.4-15.el9.x86_64 is already installed. 2024-03-21T10:08:34.036 INFO:teuthology.orchestra.run.smithi157.stdout:Package libacl-devel-2.3.1-3.el9.x86_64 is already installed. 2024-03-21T10:08:34.038 INFO:teuthology.orchestra.run.smithi157.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-03-21T10:08:34.041 INFO:teuthology.orchestra.run.smithi157.stdout:Package xfsdump-3.1.12-3.el9.x86_64 is already installed. 2024-03-21T10:08:34.042 INFO:teuthology.orchestra.run.smithi157.stdout:Package xfsprogs-5.19.0-4.el9.x86_64 is already installed. 2024-03-21T10:08:34.138 INFO:teuthology.orchestra.run.smithi157.stdout:Dependencies resolved. 2024-03-21T10:08:34.158 INFO:teuthology.orchestra.run.smithi157.stdout:============================================================================================ 2024-03-21T10:08:34.158 INFO:teuthology.orchestra.run.smithi157.stdout: Package Arch Version Repository Size 2024-03-21T10:08:34.158 INFO:teuthology.orchestra.run.smithi157.stdout:============================================================================================ 2024-03-21T10:08:34.158 INFO:teuthology.orchestra.run.smithi157.stdout:Installing: 2024-03-21T10:08:34.158 INFO:teuthology.orchestra.run.smithi157.stdout: bison x86_64 3.7.4-5.el9 CentOS-AppStream 920 k 2024-03-21T10:08:34.158 INFO:teuthology.orchestra.run.smithi157.stdout: ceph x86_64 2:19.0.0-2231.g2edfee68.el9 ceph 6.6 k 2024-03-21T10:08:34.158 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-base x86_64 2:19.0.0-2231.g2edfee68.el9 ceph 5.4 M 2024-03-21T10:08:34.159 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-fuse x86_64 2:19.0.0-2231.g2edfee68.el9 ceph 890 k 2024-03-21T10:08:34.159 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-2231.g2edfee68.el9 ceph 145 k 2024-03-21T10:08:34.159 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-mgr x86_64 2:19.0.0-2231.g2edfee68.el9 ceph 1.5 M 2024-03-21T10:08:34.159 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-mgr-cephadm noarch 2:19.0.0-2231.g2edfee68.el9 ceph-noarch 140 k 2024-03-21T10:08:34.159 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-mgr-dashboard noarch 2:19.0.0-2231.g2edfee68.el9 ceph-noarch 1.8 M 2024-03-21T10:08:34.159 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-2231.g2edfee68.el9 ceph-noarch 7.4 M 2024-03-21T10:08:34.159 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-mgr-rook noarch 2:19.0.0-2231.g2edfee68.el9 ceph-noarch 49 k 2024-03-21T10:08:34.159 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-radosgw x86_64 2:19.0.0-2231.g2edfee68.el9 ceph 9.9 M 2024-03-21T10:08:34.159 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-test x86_64 2:19.0.0-2231.g2edfee68.el9 ceph 48 M 2024-03-21T10:08:34.159 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-volume noarch 2:19.0.0-2231.g2edfee68.el9 ceph-noarch 264 k 2024-03-21T10:08:34.159 INFO:teuthology.orchestra.run.smithi157.stdout: cephadm noarch 2:19.0.0-2231.g2edfee68.el9 ceph-noarch 750 k 2024-03-21T10:08:34.159 INFO:teuthology.orchestra.run.smithi157.stdout: cephfs-mirror x86_64 2:19.0.0-2231.g2edfee68.el9 ceph 220 k 2024-03-21T10:08:34.159 INFO:teuthology.orchestra.run.smithi157.stdout: cephfs-shell x86_64 2:19.0.0-2231.g2edfee68.el9 ceph 27 k 2024-03-21T10:08:34.160 INFO:teuthology.orchestra.run.smithi157.stdout: cephfs-top noarch 2:19.0.0-2231.g2edfee68.el9 ceph-noarch 18 k 2024-03-21T10:08:34.160 INFO:teuthology.orchestra.run.smithi157.stdout: elfutils-libelf-devel x86_64 0.190-2.el9 appstream 21 k 2024-03-21T10:08:34.160 INFO:teuthology.orchestra.run.smithi157.stdout: flex x86_64 2.6.4-9.el9 CentOS-AppStream 307 k 2024-03-21T10:08:34.160 INFO:teuthology.orchestra.run.smithi157.stdout: libaio-devel x86_64 0.3.111-13.el9 CentOS-AppStream 10 k 2024-03-21T10:08:34.160 INFO:teuthology.orchestra.run.smithi157.stdout: libcephfs-devel x86_64 2:19.0.0-2231.g2edfee68.el9 ceph 31 k 2024-03-21T10:08:34.160 INFO:teuthology.orchestra.run.smithi157.stdout: libcephfs2 x86_64 2:19.0.0-2231.g2edfee68.el9 ceph 747 k 2024-03-21T10:08:34.160 INFO:teuthology.orchestra.run.smithi157.stdout: librados-devel x86_64 2:19.0.0-2231.g2edfee68.el9 ceph 127 k 2024-03-21T10:08:34.160 INFO:teuthology.orchestra.run.smithi157.stdout: libtool x86_64 2.4.6-45.el9 CentOS-AppStream 578 k 2024-03-21T10:08:34.160 INFO:teuthology.orchestra.run.smithi157.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-03-21T10:08:34.160 INFO:teuthology.orchestra.run.smithi157.stdout: openssl-devel x86_64 1:3.0.7-27.el9 appstream 4.1 M 2024-03-21T10:08:34.160 INFO:teuthology.orchestra.run.smithi157.stdout: postgresql x86_64 13.11-1.el9 CentOS-AppStream 1.5 M 2024-03-21T10:08:34.160 INFO:teuthology.orchestra.run.smithi157.stdout: postgresql-contrib x86_64 13.11-1.el9 CentOS-AppStream 811 k 2024-03-21T10:08:34.160 INFO:teuthology.orchestra.run.smithi157.stdout: postgresql-server x86_64 13.11-1.el9 CentOS-AppStream 5.7 M 2024-03-21T10:08:34.161 INFO:teuthology.orchestra.run.smithi157.stdout: python3-cephfs x86_64 2:19.0.0-2231.g2edfee68.el9 ceph 161 k 2024-03-21T10:08:34.161 INFO:teuthology.orchestra.run.smithi157.stdout: python3-rados x86_64 2:19.0.0-2231.g2edfee68.el9 ceph 320 k 2024-03-21T10:08:34.161 INFO:teuthology.orchestra.run.smithi157.stdout: python3-rbd x86_64 2:19.0.0-2231.g2edfee68.el9 ceph 297 k 2024-03-21T10:08:34.161 INFO:teuthology.orchestra.run.smithi157.stdout: python3-rgw x86_64 2:19.0.0-2231.g2edfee68.el9 ceph 99 k 2024-03-21T10:08:34.161 INFO:teuthology.orchestra.run.smithi157.stdout: rbd-fuse x86_64 2:19.0.0-2231.g2edfee68.el9 ceph 85 k 2024-03-21T10:08:34.161 INFO:teuthology.orchestra.run.smithi157.stdout: rbd-mirror x86_64 2:19.0.0-2231.g2edfee68.el9 ceph 3.1 M 2024-03-21T10:08:34.161 INFO:teuthology.orchestra.run.smithi157.stdout: rbd-nbd x86_64 2:19.0.0-2231.g2edfee68.el9 ceph 171 k 2024-03-21T10:08:34.161 INFO:teuthology.orchestra.run.smithi157.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-03-21T10:08:34.161 INFO:teuthology.orchestra.run.smithi157.stdout:Upgrading: 2024-03-21T10:08:34.161 INFO:teuthology.orchestra.run.smithi157.stdout: NetworkManager x86_64 1:1.47.2-1.el9 baseos 2.3 M 2024-03-21T10:08:34.161 INFO:teuthology.orchestra.run.smithi157.stdout: NetworkManager-libnm x86_64 1:1.47.2-1.el9 baseos 1.8 M 2024-03-21T10:08:34.161 INFO:teuthology.orchestra.run.smithi157.stdout: NetworkManager-team x86_64 1:1.47.2-1.el9 baseos 42 k 2024-03-21T10:08:34.162 INFO:teuthology.orchestra.run.smithi157.stdout: NetworkManager-tui x86_64 1:1.47.2-1.el9 baseos 248 k 2024-03-21T10:08:34.162 INFO:teuthology.orchestra.run.smithi157.stdout: acl x86_64 2.3.1-4.el9 baseos 71 k 2024-03-21T10:08:34.162 INFO:teuthology.orchestra.run.smithi157.stdout: elfutils-debuginfod-client x86_64 0.190-2.el9 baseos 36 k 2024-03-21T10:08:34.162 INFO:teuthology.orchestra.run.smithi157.stdout: elfutils-libelf x86_64 0.190-2.el9 baseos 192 k 2024-03-21T10:08:34.162 INFO:teuthology.orchestra.run.smithi157.stdout: elfutils-libs x86_64 0.190-2.el9 baseos 256 k 2024-03-21T10:08:34.162 INFO:teuthology.orchestra.run.smithi157.stdout: libacl x86_64 2.3.1-4.el9 baseos 23 k 2024-03-21T10:08:34.162 INFO:teuthology.orchestra.run.smithi157.stdout: libacl-devel x86_64 2.3.1-4.el9 appstream 74 k 2024-03-21T10:08:34.162 INFO:teuthology.orchestra.run.smithi157.stdout: libblkid x86_64 2.37.4-18.el9 baseos 108 k 2024-03-21T10:08:34.162 INFO:teuthology.orchestra.run.smithi157.stdout: libfdisk x86_64 2.37.4-18.el9 baseos 155 k 2024-03-21T10:08:34.162 INFO:teuthology.orchestra.run.smithi157.stdout: libmount x86_64 2.37.4-18.el9 baseos 136 k 2024-03-21T10:08:34.162 INFO:teuthology.orchestra.run.smithi157.stdout: librados2 x86_64 2:19.0.0-2231.g2edfee68.el9 ceph 3.4 M 2024-03-21T10:08:34.162 INFO:teuthology.orchestra.run.smithi157.stdout: librbd1 x86_64 2:19.0.0-2231.g2edfee68.el9 ceph 3.1 M 2024-03-21T10:08:34.163 INFO:teuthology.orchestra.run.smithi157.stdout: libsmartcols x86_64 2.37.4-18.el9 baseos 64 k 2024-03-21T10:08:34.163 INFO:teuthology.orchestra.run.smithi157.stdout: libuuid x86_64 2.37.4-18.el9 baseos 29 k 2024-03-21T10:08:34.163 INFO:teuthology.orchestra.run.smithi157.stdout: openssl x86_64 1:3.0.7-27.el9 baseos 1.2 M 2024-03-21T10:08:34.163 INFO:teuthology.orchestra.run.smithi157.stdout: openssl-libs x86_64 1:3.0.7-27.el9 baseos 2.2 M 2024-03-21T10:08:34.163 INFO:teuthology.orchestra.run.smithi157.stdout: util-linux x86_64 2.37.4-18.el9 baseos 2.3 M 2024-03-21T10:08:34.163 INFO:teuthology.orchestra.run.smithi157.stdout: util-linux-core x86_64 2.37.4-18.el9 baseos 465 k 2024-03-21T10:08:34.163 INFO:teuthology.orchestra.run.smithi157.stdout: util-linux-user x86_64 2.37.4-18.el9 baseos 31 k 2024-03-21T10:08:34.163 INFO:teuthology.orchestra.run.smithi157.stdout: xfsdump x86_64 3.1.12-4.el9 CentOS-BaseOS 321 k 2024-03-21T10:08:34.163 INFO:teuthology.orchestra.run.smithi157.stdout: xfsprogs x86_64 6.3.0-1.el9 baseos 1.1 M 2024-03-21T10:08:34.163 INFO:teuthology.orchestra.run.smithi157.stdout:Installing dependencies: 2024-03-21T10:08:34.163 INFO:teuthology.orchestra.run.smithi157.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-03-21T10:08:34.163 INFO:teuthology.orchestra.run.smithi157.stdout: automake noarch 1.16.2-8.el9 CentOS-AppStream 662 k 2024-03-21T10:08:34.163 INFO:teuthology.orchestra.run.smithi157.stdout: boost-program-options x86_64 1.75.0-8.el9 CentOS-AppStream 106 k 2024-03-21T10:08:34.163 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-common x86_64 2:19.0.0-2231.g2edfee68.el9 ceph 21 M 2024-03-21T10:08:34.164 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-grafana-dashboards noarch 2:19.0.0-2231.g2edfee68.el9 ceph-noarch 34 k 2024-03-21T10:08:34.164 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-mds x86_64 2:19.0.0-2231.g2edfee68.el9 ceph 2.3 M 2024-03-21T10:08:34.164 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-mgr-modules-core noarch 2:19.0.0-2231.g2edfee68.el9 ceph-noarch 248 k 2024-03-21T10:08:34.164 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-mon x86_64 2:19.0.0-2231.g2edfee68.el9 ceph 4.7 M 2024-03-21T10:08:34.164 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-osd x86_64 2:19.0.0-2231.g2edfee68.el9 ceph 17 M 2024-03-21T10:08:34.164 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-prometheus-alerts noarch 2:19.0.0-2231.g2edfee68.el9 ceph-noarch 16 k 2024-03-21T10:08:34.164 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-selinux x86_64 2:19.0.0-2231.g2edfee68.el9 ceph 25 k 2024-03-21T10:08:34.164 INFO:teuthology.orchestra.run.smithi157.stdout: flexiblas x86_64 3.0.4-8.el9 CentOS-AppStream 32 k 2024-03-21T10:08:34.164 INFO:teuthology.orchestra.run.smithi157.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 CentOS-AppStream 3.0 M 2024-03-21T10:08:34.164 INFO:teuthology.orchestra.run.smithi157.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 CentOS-AppStream 17 k 2024-03-21T10:08:34.164 INFO:teuthology.orchestra.run.smithi157.stdout: fuse x86_64 2.9.9-15.el9 CentOS-BaseOS 79 k 2024-03-21T10:08:34.164 INFO:teuthology.orchestra.run.smithi157.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-03-21T10:08:34.164 INFO:teuthology.orchestra.run.smithi157.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-03-21T10:08:34.164 INFO:teuthology.orchestra.run.smithi157.stdout: jq x86_64 1.6-16.el9 baseos 187 k 2024-03-21T10:08:34.165 INFO:teuthology.orchestra.run.smithi157.stdout: libarrow x86_64 9.0.0-10.el9 epel 4.4 M 2024-03-21T10:08:34.165 INFO:teuthology.orchestra.run.smithi157.stdout: libarrow-doc noarch 9.0.0-10.el9 epel 26 k 2024-03-21T10:08:34.165 INFO:teuthology.orchestra.run.smithi157.stdout: libcephsqlite x86_64 2:19.0.0-2231.g2edfee68.el9 ceph 163 k 2024-03-21T10:08:34.165 INFO:teuthology.orchestra.run.smithi157.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-03-21T10:08:34.165 INFO:teuthology.orchestra.run.smithi157.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-03-21T10:08:34.165 INFO:teuthology.orchestra.run.smithi157.stdout: libpmemobj x86_64 1.12.1-1.el9 CentOS-AppStream 159 k 2024-03-21T10:08:34.165 INFO:teuthology.orchestra.run.smithi157.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-03-21T10:08:34.165 INFO:teuthology.orchestra.run.smithi157.stdout: librabbitmq x86_64 0.11.0-7.el9 CentOS-AppStream 44 k 2024-03-21T10:08:34.165 INFO:teuthology.orchestra.run.smithi157.stdout: libradosstriper1 x86_64 2:19.0.0-2231.g2edfee68.el9 ceph 501 k 2024-03-21T10:08:34.165 INFO:teuthology.orchestra.run.smithi157.stdout: librdkafka x86_64 1.6.1-102.el9 CentOS-AppStream 661 k 2024-03-21T10:08:34.165 INFO:teuthology.orchestra.run.smithi157.stdout: librgw2 x86_64 2:19.0.0-2231.g2edfee68.el9 ceph 5.1 M 2024-03-21T10:08:34.165 INFO:teuthology.orchestra.run.smithi157.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-03-21T10:08:34.165 INFO:teuthology.orchestra.run.smithi157.stdout: libzstd-devel x86_64 1.5.1-2.el9 CentOS-AppStream 46 k 2024-03-21T10:08:34.166 INFO:teuthology.orchestra.run.smithi157.stdout: lttng-ust x86_64 2.12.0-6.el9 CentOS-AppStream 282 k 2024-03-21T10:08:34.166 INFO:teuthology.orchestra.run.smithi157.stdout: lua x86_64 5.4.4-4.el9 CentOS-AppStream 187 k 2024-03-21T10:08:34.166 INFO:teuthology.orchestra.run.smithi157.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-03-21T10:08:34.166 INFO:teuthology.orchestra.run.smithi157.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-03-21T10:08:34.166 INFO:teuthology.orchestra.run.smithi157.stdout: oniguruma x86_64 6.9.6-1.el9.5 CentOS-AppStream 217 k 2024-03-21T10:08:34.166 INFO:teuthology.orchestra.run.smithi157.stdout: openblas x86_64 0.3.21-2.el9 CentOS-AppStream 34 k 2024-03-21T10:08:34.166 INFO:teuthology.orchestra.run.smithi157.stdout: openblas-openmp x86_64 0.3.21-2.el9 CentOS-AppStream 4.7 M 2024-03-21T10:08:34.166 INFO:teuthology.orchestra.run.smithi157.stdout: parquet-libs x86_64 9.0.0-10.el9 epel 838 k 2024-03-21T10:08:34.166 INFO:teuthology.orchestra.run.smithi157.stdout: postgresql-private-libs x86_64 13.11-1.el9 CentOS-AppStream 134 k 2024-03-21T10:08:34.166 INFO:teuthology.orchestra.run.smithi157.stdout: protobuf x86_64 3.14.0-13.el9 CentOS-AppStream 1.0 M 2024-03-21T10:08:34.166 INFO:teuthology.orchestra.run.smithi157.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-03-21T10:08:34.166 INFO:teuthology.orchestra.run.smithi157.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-03-21T10:08:34.166 INFO:teuthology.orchestra.run.smithi157.stdout: python3-attrs noarch 20.3.0-7.el9 CentOS-AppStream 78 k 2024-03-21T10:08:34.167 INFO:teuthology.orchestra.run.smithi157.stdout: python3-babel noarch 2.9.1-2.el9 CentOS-AppStream 5.8 M 2024-03-21T10:08:34.167 INFO:teuthology.orchestra.run.smithi157.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-03-21T10:08:34.167 INFO:teuthology.orchestra.run.smithi157.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-03-21T10:08:34.167 INFO:teuthology.orchestra.run.smithi157.stdout: python3-ceph-argparse x86_64 2:19.0.0-2231.g2edfee68.el9 ceph 45 k 2024-03-21T10:08:34.167 INFO:teuthology.orchestra.run.smithi157.stdout: python3-ceph-common x86_64 2:19.0.0-2231.g2edfee68.el9 ceph 131 k 2024-03-21T10:08:34.167 INFO:teuthology.orchestra.run.smithi157.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-03-21T10:08:34.167 INFO:teuthology.orchestra.run.smithi157.stdout: python3-cffi x86_64 1.14.5-5.el9 CentOS-BaseOS 241 k 2024-03-21T10:08:34.167 INFO:teuthology.orchestra.run.smithi157.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-03-21T10:08:34.167 INFO:teuthology.orchestra.run.smithi157.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-03-21T10:08:34.167 INFO:teuthology.orchestra.run.smithi157.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-03-21T10:08:34.167 INFO:teuthology.orchestra.run.smithi157.stdout: python3-colorama noarch 0.4.4-12.el9 epel 35 k 2024-03-21T10:08:34.167 INFO:teuthology.orchestra.run.smithi157.stdout: python3-cryptography x86_64 36.0.1-4.el9 CentOS-BaseOS 1.1 M 2024-03-21T10:08:34.167 INFO:teuthology.orchestra.run.smithi157.stdout: python3-devel x86_64 3.9.17-2.el9 appstream 245 k 2024-03-21T10:08:34.167 INFO:teuthology.orchestra.run.smithi157.stdout: python3-google-auth noarch 1:2.28.2-1.el9 epel 203 k 2024-03-21T10:08:34.168 INFO:teuthology.orchestra.run.smithi157.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-03-21T10:08:34.168 INFO:teuthology.orchestra.run.smithi157.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-03-21T10:08:34.168 INFO:teuthology.orchestra.run.smithi157.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-03-21T10:08:34.168 INFO:teuthology.orchestra.run.smithi157.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-03-21T10:08:34.168 INFO:teuthology.orchestra.run.smithi157.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-03-21T10:08:34.168 INFO:teuthology.orchestra.run.smithi157.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-03-21T10:08:34.168 INFO:teuthology.orchestra.run.smithi157.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-03-21T10:08:34.168 INFO:teuthology.orchestra.run.smithi157.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-03-21T10:08:34.168 INFO:teuthology.orchestra.run.smithi157.stdout: python3-jsonpatch noarch 1.21-16.el9 CentOS-AppStream 24 k 2024-03-21T10:08:34.168 INFO:teuthology.orchestra.run.smithi157.stdout: python3-jsonpointer noarch 2.0-4.el9 CentOS-AppStream 18 k 2024-03-21T10:08:34.168 INFO:teuthology.orchestra.run.smithi157.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-03-21T10:08:34.168 INFO:teuthology.orchestra.run.smithi157.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-03-21T10:08:34.168 INFO:teuthology.orchestra.run.smithi157.stdout: python3-mako noarch 1.1.4-6.el9 CentOS-AppStream 144 k 2024-03-21T10:08:34.169 INFO:teuthology.orchestra.run.smithi157.stdout: python3-markupsafe x86_64 1.1.1-12.el9 CentOS-AppStream 32 k 2024-03-21T10:08:34.169 INFO:teuthology.orchestra.run.smithi157.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-03-21T10:08:34.169 INFO:teuthology.orchestra.run.smithi157.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-03-21T10:08:34.169 INFO:teuthology.orchestra.run.smithi157.stdout: python3-numpy x86_64 1:1.20.1-5.el9 CentOS-AppStream 4.8 M 2024-03-21T10:08:34.169 INFO:teuthology.orchestra.run.smithi157.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 CentOS-AppStream 217 k 2024-03-21T10:08:34.169 INFO:teuthology.orchestra.run.smithi157.stdout: python3-oauthlib noarch 3.1.1-5.el9 CentOS-AppStream 174 k 2024-03-21T10:08:34.169 INFO:teuthology.orchestra.run.smithi157.stdout: python3-packaging noarch 20.9-5.el9 CentOS-AppStream 69 k 2024-03-21T10:08:34.169 INFO:teuthology.orchestra.run.smithi157.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-03-21T10:08:34.169 INFO:teuthology.orchestra.run.smithi157.stdout: python3-ply noarch 3.11-14.el9 CentOS-BaseOS 103 k 2024-03-21T10:08:34.169 INFO:teuthology.orchestra.run.smithi157.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-03-21T10:08:34.169 INFO:teuthology.orchestra.run.smithi157.stdout: python3-prettytable noarch 0.7.2-27.el9 CentOS-AppStream 41 k 2024-03-21T10:08:34.169 INFO:teuthology.orchestra.run.smithi157.stdout: python3-protobuf noarch 3.14.0-13.el9 CentOS-AppStream 238 k 2024-03-21T10:08:34.169 INFO:teuthology.orchestra.run.smithi157.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-03-21T10:08:34.169 INFO:teuthology.orchestra.run.smithi157.stdout: python3-pyasn1 noarch 0.4.8-6.el9 CentOS-AppStream 132 k 2024-03-21T10:08:34.170 INFO:teuthology.orchestra.run.smithi157.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 CentOS-AppStream 211 k 2024-03-21T10:08:34.170 INFO:teuthology.orchestra.run.smithi157.stdout: python3-pycparser noarch 2.20-6.el9 CentOS-BaseOS 124 k 2024-03-21T10:08:34.170 INFO:teuthology.orchestra.run.smithi157.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-03-21T10:08:34.170 INFO:teuthology.orchestra.run.smithi157.stdout: python3-pytz noarch 2021.1-5.el9 CentOS-AppStream 47 k 2024-03-21T10:08:34.170 INFO:teuthology.orchestra.run.smithi157.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-03-21T10:08:34.170 INFO:teuthology.orchestra.run.smithi157.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-03-21T10:08:34.170 INFO:teuthology.orchestra.run.smithi157.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 CentOS-AppStream 43 k 2024-03-21T10:08:34.170 INFO:teuthology.orchestra.run.smithi157.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-03-21T10:08:34.170 INFO:teuthology.orchestra.run.smithi157.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-03-21T10:08:34.170 INFO:teuthology.orchestra.run.smithi157.stdout: python3-scipy x86_64 1.6.2-8.el9 CentOS-AppStream 16 M 2024-03-21T10:08:34.170 INFO:teuthology.orchestra.run.smithi157.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-03-21T10:08:34.170 INFO:teuthology.orchestra.run.smithi157.stdout: python3-toml noarch 0.10.2-6.el9 CentOS-AppStream 39 k 2024-03-21T10:08:34.170 INFO:teuthology.orchestra.run.smithi157.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-03-21T10:08:34.170 INFO:teuthology.orchestra.run.smithi157.stdout: python3-urllib3 noarch 1.26.5-5.el9 CentOS-BaseOS 187 k 2024-03-21T10:08:34.170 INFO:teuthology.orchestra.run.smithi157.stdout: python3-wcwidth noarch 0.2.5-8.el9 CentOS-AppStream 41 k 2024-03-21T10:08:34.171 INFO:teuthology.orchestra.run.smithi157.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-03-21T10:08:34.171 INFO:teuthology.orchestra.run.smithi157.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-03-21T10:08:34.171 INFO:teuthology.orchestra.run.smithi157.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-03-21T10:08:34.171 INFO:teuthology.orchestra.run.smithi157.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-03-21T10:08:34.171 INFO:teuthology.orchestra.run.smithi157.stdout: qatlib x86_64 23.11.0-1.el9 appstream 216 k 2024-03-21T10:08:34.171 INFO:teuthology.orchestra.run.smithi157.stdout: qatzip-libs x86_64 1.1.2-2.el9 appstream 44 k 2024-03-21T10:08:34.171 INFO:teuthology.orchestra.run.smithi157.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-03-21T10:08:34.171 INFO:teuthology.orchestra.run.smithi157.stdout: socat x86_64 1.7.4.1-5.el9 CentOS-AppStream 300 k 2024-03-21T10:08:34.171 INFO:teuthology.orchestra.run.smithi157.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-03-21T10:08:34.171 INFO:teuthology.orchestra.run.smithi157.stdout: uuid x86_64 1.6.2-55.el9 CentOS-AppStream 56 k 2024-03-21T10:08:34.171 INFO:teuthology.orchestra.run.smithi157.stdout: xmlstarlet x86_64 1.6.1-20.el9 CentOS-AppStream 63 k 2024-03-21T10:08:34.171 INFO:teuthology.orchestra.run.smithi157.stdout: zlib-devel x86_64 1.2.11-40.el9 CentOS-AppStream 44 k 2024-03-21T10:08:34.172 INFO:teuthology.orchestra.run.smithi157.stdout:Installing weak dependencies: 2024-03-21T10:08:34.172 INFO:teuthology.orchestra.run.smithi157.stdout: qatlib-service x86_64 23.11.0-1.el9 appstream 35 k 2024-03-21T10:08:34.172 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-03-21T10:08:34.172 INFO:teuthology.orchestra.run.smithi157.stdout:Transaction Summary 2024-03-21T10:08:34.172 INFO:teuthology.orchestra.run.smithi157.stdout:============================================================================================ 2024-03-21T10:08:34.172 INFO:teuthology.orchestra.run.smithi157.stdout:Install 150 Packages 2024-03-21T10:08:34.172 INFO:teuthology.orchestra.run.smithi157.stdout:Upgrade 24 Packages 2024-03-21T10:08:34.172 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-03-21T10:08:34.173 INFO:teuthology.orchestra.run.smithi157.stdout:Total download size: 224 M 2024-03-21T10:08:34.173 INFO:teuthology.orchestra.run.smithi157.stdout:Downloading Packages: 2024-03-21T10:08:34.176 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : libradosstriper1-2:19.0.0-2231.g2edfee68.el9.x86 17/198 2024-03-21T10:08:34.265 INFO:teuthology.orchestra.run.smithi047.stdout: Running scriptlet: libradosstriper1-2:19.0.0-2231.g2edfee68.el9.x86 17/198 2024-03-21T10:08:34.321 INFO:teuthology.orchestra.run.smithi047.stdout: Upgrading : libmount-2.37.4-18.el9.x86_64 18/198 2024-03-21T10:08:34.348 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : qatlib-service-23.11.0-1.el9.x86_64 19/198 2024-03-21T10:08:34.405 INFO:teuthology.orchestra.run.smithi047.stdout: Running scriptlet: qatlib-service-23.11.0-1.el9.x86_64 19/198 2024-03-21T10:08:34.666 INFO:teuthology.orchestra.run.smithi047.stdout: Running scriptlet: qatlib-23.11.0-1.el9.x86_64 20/198 2024-03-21T10:08:34.706 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : qatlib-23.11.0-1.el9.x86_64 20/198 2024-03-21T10:08:34.756 INFO:teuthology.orchestra.run.smithi047.stdout: Upgrading : libsmartcols-2.37.4-18.el9.x86_64 21/198 2024-03-21T10:08:35.108 INFO:teuthology.orchestra.run.smithi047.stdout: Upgrading : libacl-2.3.1-4.el9.x86_64 22/198 2024-03-21T10:08:35.264 INFO:teuthology.orchestra.run.smithi047.stdout: Upgrading : NetworkManager-libnm-1:1.47.2-1.el9.x86_64 23/198 2024-03-21T10:08:35.337 INFO:teuthology.orchestra.run.smithi047.stdout: Running scriptlet: NetworkManager-libnm-1:1.47.2-1.el9.x86_64 23/198 2024-03-21T10:08:35.412 INFO:teuthology.orchestra.run.smithi157.stdout:(1/174): ceph-19.0.0-2231.g2edfee68.el9.x86_64. 24 kB/s | 6.6 kB 00:00 2024-03-21T10:08:35.541 INFO:teuthology.orchestra.run.smithi047.stdout: Running scriptlet: NetworkManager-1:1.47.2-1.el9.x86_64 24/198 2024-03-21T10:08:35.583 INFO:teuthology.orchestra.run.smithi047.stdout: Upgrading : NetworkManager-1:1.47.2-1.el9.x86_64 24/198 2024-03-21T10:08:35.734 INFO:teuthology.orchestra.run.smithi157.stdout:(2/174): ceph-fuse-19.0.0-2231.g2edfee68.el9.x8 2.7 MB/s | 890 kB 00:00 2024-03-21T10:08:35.792 INFO:teuthology.orchestra.run.smithi157.stdout:(3/174): ceph-immutable-object-cache-19.0.0-223 2.5 MB/s | 145 kB 00:00 2024-03-21T10:08:35.924 INFO:teuthology.orchestra.run.smithi047.stdout: Running scriptlet: NetworkManager-1:1.47.2-1.el9.x86_64 24/198 2024-03-21T10:08:36.109 INFO:teuthology.orchestra.run.smithi157.stdout:(4/174): ceph-mds-19.0.0-2231.g2edfee68.el9.x86 7.3 MB/s | 2.3 MB 00:00 2024-03-21T10:08:36.112 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : re2-1:20211101-3.el9.x86_64 25/198 2024-03-21T10:08:36.242 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : libarrow-9.0.0-10.el9.x86_64 26/198 2024-03-21T10:08:36.281 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 27/198 2024-03-21T10:08:36.284 INFO:teuthology.orchestra.run.smithi157.stdout:(5/174): ceph-base-19.0.0-2231.g2edfee68.el9.x8 4.7 MB/s | 5.4 MB 00:01 2024-03-21T10:08:36.318 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : liboath-2.6.7-2.el9.x86_64 28/198 2024-03-21T10:08:36.359 INFO:teuthology.orchestra.run.smithi157.stdout:(6/174): ceph-mgr-19.0.0-2231.g2edfee68.el9.x86 6.0 MB/s | 1.5 MB 00:00 2024-03-21T10:08:36.377 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 29/198 2024-03-21T10:08:36.493 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 30/198 2024-03-21T10:08:36.567 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 31/198 2024-03-21T10:08:36.615 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 32/198 2024-03-21T10:08:36.720 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 33/198 2024-03-21T10:08:36.796 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 34/198 2024-03-21T10:08:36.876 INFO:teuthology.orchestra.run.smithi157.stdout:(7/174): ceph-mon-19.0.0-2231.g2edfee68.el9.x86 7.9 MB/s | 4.7 MB 00:00 2024-03-21T10:08:36.966 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 35/198 2024-03-21T10:08:37.026 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 36/198 2024-03-21T10:08:37.068 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : lua-5.4.4-4.el9.x86_64 37/198 2024-03-21T10:08:37.197 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 38/198 2024-03-21T10:08:37.236 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-ceph-common-2:19.0.0-2231.g2edfee68.el9. 39/198 2024-03-21T10:08:37.293 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-ceph-argparse-2:19.0.0-2231.g2edfee68.el 40/198 2024-03-21T10:08:37.327 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-cephfs-2:19.0.0-2231.g2edfee68.el9.x86_6 41/198 2024-03-21T10:08:37.454 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 42/198 2024-03-21T10:08:37.524 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : luarocks-3.9.2-1.el9.noarch 43/198 2024-03-21T10:08:37.639 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 44/198 2024-03-21T10:08:37.823 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 45/198 2024-03-21T10:08:37.889 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 46/198 2024-03-21T10:08:37.951 INFO:teuthology.orchestra.run.smithi157.stdout:(8/174): ceph-radosgw-19.0.0-2231.g2edfee68.el9 9.2 MB/s | 9.9 MB 00:01 2024-03-21T10:08:38.219 INFO:teuthology.orchestra.run.smithi157.stdout:(9/174): ceph-common-19.0.0-2231.g2edfee68.el9. 6.7 MB/s | 21 MB 00:03 2024-03-21T10:08:38.243 INFO:teuthology.orchestra.run.smithi157.stdout:(10/174): ceph-selinux-19.0.0-2231.g2edfee68.el 86 kB/s | 25 kB 00:00 2024-03-21T10:08:38.311 INFO:teuthology.orchestra.run.smithi157.stdout:(11/174): cephfs-mirror-19.0.0-2231.g2edfee68.e 3.3 MB/s | 220 kB 00:00 2024-03-21T10:08:38.377 INFO:teuthology.orchestra.run.smithi157.stdout:(12/174): cephfs-shell-19.0.0-2231.g2edfee68.el 411 kB/s | 27 kB 00:00 2024-03-21T10:08:38.588 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-rsa-4.9-2.el9.noarch 47/198 2024-03-21T10:08:38.594 INFO:teuthology.orchestra.run.smithi157.stdout:(13/174): ceph-osd-19.0.0-2231.g2edfee68.el9.x8 7.5 MB/s | 17 MB 00:02 2024-03-21T10:08:38.620 INFO:teuthology.orchestra.run.smithi157.stdout:(14/174): libcephfs-devel-19.0.0-2231.g2edfee68 130 kB/s | 31 kB 00:00 2024-03-21T10:08:38.687 INFO:teuthology.orchestra.run.smithi157.stdout:(15/174): libcephsqlite-19.0.0-2231.g2edfee68.e 2.4 MB/s | 163 kB 00:00 2024-03-21T10:08:38.691 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 48/198 2024-03-21T10:08:38.728 INFO:teuthology.orchestra.run.smithi157.stdout:(16/174): libcephfs2-19.0.0-2231.g2edfee68.el9. 5.5 MB/s | 747 kB 00:00 2024-03-21T10:08:38.762 INFO:teuthology.orchestra.run.smithi157.stdout:(17/174): librados-devel-19.0.0-2231.g2edfee68. 1.7 MB/s | 127 kB 00:00 2024-03-21T10:08:38.808 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 49/198 2024-03-21T10:08:38.812 INFO:teuthology.orchestra.run.smithi157.stdout:(18/174): libradosstriper1-19.0.0-2231.g2edfee6 5.9 MB/s | 501 kB 00:00 2024-03-21T10:08:38.863 INFO:teuthology.orchestra.run.smithi047.stdout: Upgrading : util-linux-core-2.37.4-18.el9.x86_64 50/198 2024-03-21T10:08:38.888 INFO:teuthology.orchestra.run.smithi157.stdout:(19/174): python3-ceph-argparse-19.0.0-2231.g2e 605 kB/s | 45 kB 00:00 2024-03-21T10:08:38.889 INFO:teuthology.orchestra.run.smithi047.stdout: Running scriptlet: util-linux-core-2.37.4-18.el9.x86_64 50/198 2024-03-21T10:08:38.939 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : qatzip-libs-1.1.2-2.el9.x86_64 51/198 2024-03-21T10:08:38.971 INFO:teuthology.orchestra.run.smithi157.stdout:(20/174): python3-ceph-common-19.0.0-2231.g2edf 1.6 MB/s | 131 kB 00:00 2024-03-21T10:08:38.994 INFO:teuthology.orchestra.run.smithi047.stdout: Upgrading : elfutils-libs-0.190-2.el9.x86_64 52/198 2024-03-21T10:08:39.047 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : librados-devel-2:19.0.0-2231.g2edfee68.el9.x86_6 53/198 2024-03-21T10:08:39.063 INFO:teuthology.orchestra.run.smithi157.stdout:(21/174): python3-cephfs-19.0.0-2231.g2edfee68. 1.7 MB/s | 161 kB 00:00 2024-03-21T10:08:39.086 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 54/198 2024-03-21T10:08:39.172 INFO:teuthology.orchestra.run.smithi157.stdout:(22/174): python3-rados-19.0.0-2231.g2edfee68.e 2.9 MB/s | 320 kB 00:00 2024-03-21T10:08:39.314 INFO:teuthology.orchestra.run.smithi157.stdout:(23/174): librgw2-19.0.0-2231.g2edfee68.el9.x86 9.3 MB/s | 5.1 MB 00:00 2024-03-21T10:08:39.348 INFO:teuthology.orchestra.run.smithi157.stdout:(24/174): python3-rbd-19.0.0-2231.g2edfee68.el9 1.7 MB/s | 297 kB 00:00 2024-03-21T10:08:39.381 INFO:teuthology.orchestra.run.smithi157.stdout:(25/174): python3-rgw-19.0.0-2231.g2edfee68.el9 1.5 MB/s | 99 kB 00:00 2024-03-21T10:08:39.406 INFO:teuthology.orchestra.run.smithi157.stdout:(26/174): rbd-fuse-19.0.0-2231.g2edfee68.el9.x8 1.5 MB/s | 85 kB 00:00 2024-03-21T10:08:39.490 INFO:teuthology.orchestra.run.smithi157.stdout:(27/174): rbd-nbd-19.0.0-2231.g2edfee68.el9.x86 2.0 MB/s | 171 kB 00:00 2024-03-21T10:08:39.512 INFO:teuthology.orchestra.run.smithi047.stdout: Upgrading : libfdisk-2.37.4-18.el9.x86_64 55/198 2024-03-21T10:08:39.566 INFO:teuthology.orchestra.run.smithi157.stdout:(28/174): ceph-grafana-dashboards-19.0.0-2231.g 456 kB/s | 34 kB 00:00 2024-03-21T10:08:39.682 INFO:teuthology.orchestra.run.smithi157.stdout:(29/174): rbd-mirror-19.0.0-2231.g2edfee68.el9. 10 MB/s | 3.1 MB 00:00 2024-03-21T10:08:39.708 INFO:teuthology.orchestra.run.smithi157.stdout:(30/174): ceph-mgr-cephadm-19.0.0-2231.g2edfee6 993 kB/s | 140 kB 00:00 2024-03-21T10:08:39.743 INFO:teuthology.orchestra.run.smithi047.stdout: Upgrading : util-linux-2.37.4-18.el9.x86_64 56/198 2024-03-21T10:08:39.827 INFO:teuthology.orchestra.run.smithi047.stdout: Upgrading : xfsprogs-6.3.0-1.el9.x86_64 57/198 2024-03-21T10:08:39.858 INFO:teuthology.orchestra.run.smithi157.stdout:(31/174): ceph-mgr-dashboard-19.0.0-2231.g2edfe 10 MB/s | 1.8 MB 00:00 2024-03-21T10:08:39.895 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 58/198 2024-03-21T10:08:39.942 INFO:teuthology.orchestra.run.smithi157.stdout:(32/174): ceph-mgr-modules-core-19.0.0-2231.g2e 2.9 MB/s | 248 kB 00:00 2024-03-21T10:08:40.017 INFO:teuthology.orchestra.run.smithi157.stdout:(33/174): ceph-mgr-rook-19.0.0-2231.g2edfee68.e 664 kB/s | 49 kB 00:00 2024-03-21T10:08:40.084 INFO:teuthology.orchestra.run.smithi157.stdout:(34/174): ceph-prometheus-alerts-19.0.0-2231.g2 249 kB/s | 16 kB 00:00 2024-03-21T10:08:40.113 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 59/198 2024-03-21T10:08:40.164 INFO:teuthology.orchestra.run.smithi047.stdout: Upgrading : openssl-1:3.0.7-27.el9.x86_64 60/198 2024-03-21T10:08:40.167 INFO:teuthology.orchestra.run.smithi157.stdout:(35/174): ceph-volume-19.0.0-2231.g2edfee68.el9 3.1 MB/s | 264 kB 00:00 2024-03-21T10:08:40.247 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 61/198 2024-03-21T10:08:40.301 INFO:teuthology.orchestra.run.smithi157.stdout:(36/174): cephadm-19.0.0-2231.g2edfee68.el9.noa 5.5 MB/s | 750 kB 00:00 2024-03-21T10:08:40.347 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 62/198 2024-03-21T10:08:40.385 INFO:teuthology.orchestra.run.smithi157.stdout:(37/174): cephfs-top-19.0.0-2231.g2edfee68.el9. 221 kB/s | 18 kB 00:00 2024-03-21T10:08:40.388 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 63/198 2024-03-21T10:08:40.428 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 64/198 2024-03-21T10:08:40.498 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 65/198 2024-03-21T10:08:40.545 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 66/198 2024-03-21T10:08:40.608 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 67/198 2024-03-21T10:08:40.660 INFO:teuthology.orchestra.run.smithi157.stdout:(38/174): ceph-mgr-diskprediction-local-19.0.0- 7.8 MB/s | 7.4 MB 00:00 2024-03-21T10:08:40.679 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 68/198 2024-03-21T10:08:40.702 INFO:teuthology.orchestra.run.smithi157.stdout:(39/174): automake-1.16.2-8.el9.noarch.rpm 2.0 MB/s | 662 kB 00:00 2024-03-21T10:08:40.744 INFO:teuthology.orchestra.run.smithi157.stdout:(40/174): bison-3.7.4-5.el9.x86_64.rpm 11 MB/s | 920 kB 00:00 2024-03-21T10:08:40.777 INFO:teuthology.orchestra.run.smithi157.stdout:(41/174): boost-program-options-1.75.0-8.el9.x8 1.6 MB/s | 106 kB 00:00 2024-03-21T10:08:40.811 INFO:teuthology.orchestra.run.smithi157.stdout:(42/174): flex-2.6.4-9.el9.x86_64.rpm 4.6 MB/s | 307 kB 00:00 2024-03-21T10:08:40.836 INFO:teuthology.orchestra.run.smithi157.stdout:(43/174): flexiblas-3.0.4-8.el9.x86_64.rpm 572 kB/s | 32 kB 00:00 2024-03-21T10:08:40.882 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 69/198 2024-03-21T10:08:40.934 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 70/198 2024-03-21T10:08:40.945 INFO:teuthology.orchestra.run.smithi157.stdout:(44/174): flexiblas-netlib-3.0.4-8.el9.x86_64.r 23 MB/s | 3.0 MB 00:00 2024-03-21T10:08:40.962 INFO:teuthology.orchestra.run.smithi157.stdout:(45/174): flexiblas-openblas-openmp-3.0.4-8.el9 155 kB/s | 17 kB 00:00 2024-03-21T10:08:40.971 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-colorama-0.4.4-12.el9.noarch 71/198 2024-03-21T10:08:40.995 INFO:teuthology.orchestra.run.smithi157.stdout:(46/174): libaio-devel-0.3.111-13.el9.x86_64.rp 313 kB/s | 10 kB 00:00 2024-03-21T10:08:41.021 INFO:teuthology.orchestra.run.smithi157.stdout:(47/174): libpmemobj-1.12.1-1.el9.x86_64.rpm 2.7 MB/s | 159 kB 00:00 2024-03-21T10:08:41.030 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 72/198 2024-03-21T10:08:41.046 INFO:teuthology.orchestra.run.smithi157.stdout:(48/174): librabbitmq-0.11.0-7.el9.x86_64.rpm 916 kB/s | 44 kB 00:00 2024-03-21T10:08:41.088 INFO:teuthology.orchestra.run.smithi157.stdout:(49/174): librdkafka-1.6.1-102.el9.x86_64.rpm 9.8 MB/s | 661 kB 00:00 2024-03-21T10:08:41.113 INFO:teuthology.orchestra.run.smithi100.stdout: Preparing : 1/1 2024-03-21T10:08:41.138 INFO:teuthology.orchestra.run.smithi157.stdout:(50/174): libtool-2.4.6-45.el9.x86_64.rpm 6.6 MB/s | 578 kB 00:00 2024-03-21T10:08:41.164 INFO:teuthology.orchestra.run.smithi157.stdout:(51/174): libzstd-devel-1.5.1-2.el9.x86_64.rpm 663 kB/s | 46 kB 00:00 2024-03-21T10:08:41.173 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 73/198 2024-03-21T10:08:41.197 INFO:teuthology.orchestra.run.smithi157.stdout:(52/174): lttng-ust-2.12.0-6.el9.x86_64.rpm 4.8 MB/s | 282 kB 00:00 2024-03-21T10:08:41.220 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-google-auth-1:2.28.2-1.el9.noarch 74/198 2024-03-21T10:08:41.234 INFO:teuthology.orchestra.run.smithi100.stdout: Upgrading : openssl-libs-1:3.0.7-27.el9.x86_64 1/198 2024-03-21T10:08:41.239 INFO:teuthology.orchestra.run.smithi157.stdout:(53/174): lua-5.4.4-4.el9.x86_64.rpm 2.5 MB/s | 187 kB 00:00 2024-03-21T10:08:41.273 INFO:teuthology.orchestra.run.smithi157.stdout:(54/174): oniguruma-6.9.6-1.el9.5.x86_64.rpm 2.9 MB/s | 217 kB 00:00 2024-03-21T10:08:41.273 INFO:teuthology.orchestra.run.smithi100.stdout: Upgrading : libuuid-2.37.4-18.el9.x86_64 2/198 2024-03-21T10:08:41.290 INFO:teuthology.orchestra.run.smithi157.stdout:(55/174): openblas-0.3.21-2.el9.x86_64.rpm 718 kB/s | 34 kB 00:00 2024-03-21T10:08:41.292 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 75/198 2024-03-21T10:08:41.311 INFO:teuthology.orchestra.run.smithi100.stdout: Upgrading : libblkid-2.37.4-18.el9.x86_64 3/198 2024-03-21T10:08:41.328 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 76/198 2024-03-21T10:08:41.355 INFO:teuthology.orchestra.run.smithi100.stdout: Running scriptlet: libblkid-2.37.4-18.el9.x86_64 3/198 2024-03-21T10:08:41.361 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : libarrow-doc-9.0.0-10.el9.noarch 77/198 2024-03-21T10:08:41.424 INFO:teuthology.orchestra.run.smithi157.stdout:(56/174): openblas-openmp-0.3.21-2.el9.x86_64.r 32 MB/s | 4.7 MB 00:00 2024-03-21T10:08:41.448 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 4/198 2024-03-21T10:08:41.474 INFO:teuthology.orchestra.run.smithi157.stdout:(57/174): postgresql-13.11-1.el9.x86_64.rpm 9.1 MB/s | 1.5 MB 00:00 2024-03-21T10:08:41.516 INFO:teuthology.orchestra.run.smithi157.stdout:(58/174): postgresql-contrib-13.11-1.el9.x86_64 8.7 MB/s | 811 kB 00:00 2024-03-21T10:08:41.541 INFO:teuthology.orchestra.run.smithi157.stdout:(59/174): postgresql-private-libs-13.11-1.el9.x 2.2 MB/s | 134 kB 00:00 2024-03-21T10:08:41.550 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 78/198 2024-03-21T10:08:41.559 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 5/198 2024-03-21T10:08:41.620 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : thrift-0.15.0-2.el9.x86_64 6/198 2024-03-21T10:08:41.651 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 7/198 2024-03-21T10:08:41.683 INFO:teuthology.orchestra.run.smithi157.stdout:(60/174): postgresql-server-13.11-1.el9.x86_64. 34 MB/s | 5.7 MB 00:00 2024-03-21T10:08:41.725 INFO:teuthology.orchestra.run.smithi157.stdout:(61/174): protobuf-3.14.0-13.el9.x86_64.rpm 6.0 MB/s | 1.0 MB 00:00 2024-03-21T10:08:41.739 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 79/198 2024-03-21T10:08:41.741 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 8/198 2024-03-21T10:08:41.775 INFO:teuthology.orchestra.run.smithi157.stdout:(62/174): python3-attrs-20.3.0-7.el9.noarch.rpm 1.6 MB/s | 78 kB 00:00 2024-03-21T10:08:41.873 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 80/198 2024-03-21T10:08:41.909 INFO:teuthology.orchestra.run.smithi157.stdout:(63/174): python3-babel-2.9.1-2.el9.noarch.rpm 32 MB/s | 5.8 MB 00:00 2024-03-21T10:08:41.934 INFO:teuthology.orchestra.run.smithi157.stdout:(64/174): python3-jsonpatch-1.21-16.el9.noarch. 170 kB/s | 24 kB 00:00 2024-03-21T10:08:41.938 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-devel-3.9.17-2.el9.x86_64 81/198 2024-03-21T10:08:41.967 INFO:teuthology.orchestra.run.smithi157.stdout:(65/174): python3-jsonpointer-2.0-4.el9.noarch. 563 kB/s | 18 kB 00:00 2024-03-21T10:08:41.993 INFO:teuthology.orchestra.run.smithi157.stdout:(66/174): python3-mako-1.1.4-6.el9.noarch.rpm 2.5 MB/s | 144 kB 00:00 2024-03-21T10:08:42.002 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-ply-3.11-14.el9.noarch 82/198 2024-03-21T10:08:42.018 INFO:teuthology.orchestra.run.smithi157.stdout:(67/174): python3-markupsafe-1.1.1-12.el9.x86_6 655 kB/s | 32 kB 00:00 2024-03-21T10:08:42.078 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 83/198 2024-03-21T10:08:42.152 INFO:teuthology.orchestra.run.smithi157.stdout:(68/174): python3-numpy-1.20.1-5.el9.x86_64.rpm 30 MB/s | 4.8 MB 00:00 2024-03-21T10:08:42.163 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 9/198 2024-03-21T10:08:42.191 INFO:teuthology.orchestra.run.smithi100.stdout: Upgrading : librados2-2:19.0.0-2231.g2edfee68.el9.x86_64 10/198 2024-03-21T10:08:42.202 INFO:teuthology.orchestra.run.smithi157.stdout:(69/174): python3-numpy-f2py-1.20.1-5.el9.x86_6 1.3 MB/s | 217 kB 00:00 2024-03-21T10:08:42.244 INFO:teuthology.orchestra.run.smithi157.stdout:(70/174): python3-oauthlib-3.1.1-5.el9.noarch.r 4.2 MB/s | 174 kB 00:00 2024-03-21T10:08:42.269 INFO:teuthology.orchestra.run.smithi157.stdout:(71/174): python3-packaging-20.9-5.el9.noarch.r 1.0 MB/s | 69 kB 00:00 2024-03-21T10:08:42.303 INFO:teuthology.orchestra.run.smithi157.stdout:(72/174): python3-prettytable-0.7.2-27.el9.noar 1.2 MB/s | 41 kB 00:00 2024-03-21T10:08:42.332 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 84/198 2024-03-21T10:08:42.336 INFO:teuthology.orchestra.run.smithi157.stdout:(73/174): python3-protobuf-3.14.0-13.el9.noarch 3.5 MB/s | 238 kB 00:00 2024-03-21T10:08:42.340 INFO:teuthology.orchestra.run.smithi100.stdout: Running scriptlet: librados2-2:19.0.0-2231.g2edfee68.el9.x86_64 10/198 2024-03-21T10:08:42.361 INFO:teuthology.orchestra.run.smithi157.stdout:(74/174): python3-pyasn1-0.4.8-6.el9.noarch.rpm 2.3 MB/s | 132 kB 00:00 2024-03-21T10:08:42.367 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : libcephfs2-2:19.0.0-2231.g2edfee68.el9.x86_64 11/198 2024-03-21T10:08:42.392 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 85/198 2024-03-21T10:08:42.395 INFO:teuthology.orchestra.run.smithi157.stdout:(75/174): python3-pyasn1-modules-0.4.8-6.el9.no 3.6 MB/s | 211 kB 00:00 2024-03-21T10:08:42.420 INFO:teuthology.orchestra.run.smithi157.stdout:(76/174): python3-pytz-2021.1-5.el9.noarch.rpm 844 kB/s | 47 kB 00:00 2024-03-21T10:08:42.446 INFO:teuthology.orchestra.run.smithi157.stdout:(77/174): python3-requests-oauthlib-1.3.0-12.el 869 kB/s | 43 kB 00:00 2024-03-21T10:08:42.485 INFO:teuthology.orchestra.run.smithi100.stdout: Running scriptlet: libcephfs2-2:19.0.0-2231.g2edfee68.el9.x86_64 11/198 2024-03-21T10:08:42.525 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-rados-2:19.0.0-2231.g2edfee68.el9.x86_64 12/198 2024-03-21T10:08:42.569 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 86/198 2024-03-21T10:08:42.590 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : postgresql-private-libs-13.11-1.el9.x86_64 13/198 2024-03-21T10:08:42.623 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 87/198 2024-03-21T10:08:42.662 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 88/198 2024-03-21T10:08:42.700 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : fuse-2.9.9-15.el9.x86_64 89/198 2024-03-21T10:08:42.742 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : zlib-devel-1.2.11-40.el9.x86_64 90/198 2024-03-21T10:08:42.754 INFO:teuthology.orchestra.run.smithi157.stdout:(78/174): python3-scipy-1.6.2-8.el9.x86_64.rpm 47 MB/s | 16 MB 00:00 2024-03-21T10:08:42.782 INFO:teuthology.orchestra.run.smithi157.stdout:(79/174): python3-toml-0.10.2-6.el9.noarch.rpm 121 kB/s | 39 kB 00:00 2024-03-21T10:08:42.789 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 91/198 2024-03-21T10:08:42.813 INFO:teuthology.orchestra.run.smithi157.stdout:(80/174): python3-wcwidth-0.2.5-8.el9.noarch.rp 711 kB/s | 41 kB 00:00 2024-03-21T10:08:42.847 INFO:teuthology.orchestra.run.smithi157.stdout:(81/174): socat-1.7.4.1-5.el9.x86_64.rpm 4.4 MB/s | 300 kB 00:00 2024-03-21T10:08:42.872 INFO:teuthology.orchestra.run.smithi157.stdout:(82/174): uuid-1.6.2-55.el9.x86_64.rpm 1.0 MB/s | 56 kB 00:00 2024-03-21T10:08:42.877 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : uuid-1.6.2-55.el9.x86_64 92/198 2024-03-21T10:08:42.881 INFO:teuthology.orchestra.run.smithi100.stdout: Upgrading : elfutils-libelf-0.190-2.el9.x86_64 14/198 2024-03-21T10:08:42.898 INFO:teuthology.orchestra.run.smithi157.stdout:(83/174): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.3 MB/s | 63 kB 00:00 2024-03-21T10:08:42.923 INFO:teuthology.orchestra.run.smithi157.stdout:(84/174): zlib-devel-1.2.11-40.el9.x86_64.rpm 902 kB/s | 44 kB 00:00 2024-03-21T10:08:42.937 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 93/198 2024-03-21T10:08:42.946 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : postgresql-13.11-1.el9.x86_64 15/198 2024-03-21T10:08:42.948 INFO:teuthology.orchestra.run.smithi157.stdout:(85/174): fuse-2.9.9-15.el9.x86_64.rpm 1.6 MB/s | 79 kB 00:00 2024-03-21T10:08:42.972 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : libcephsqlite-2:19.0.0-2231.g2edfee68.el9.x86_64 16/198 2024-03-21T10:08:42.982 INFO:teuthology.orchestra.run.smithi157.stdout:(86/174): python3-cffi-1.14.5-5.el9.x86_64.rpm 4.1 MB/s | 241 kB 00:00 2024-03-21T10:08:42.982 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 94/198 2024-03-21T10:08:43.032 INFO:teuthology.orchestra.run.smithi157.stdout:(87/174): python3-cryptography-36.0.1-4.el9.x86 14 MB/s | 1.1 MB 00:00 2024-03-21T10:08:43.057 INFO:teuthology.orchestra.run.smithi157.stdout:(88/174): python3-ply-3.11-14.el9.noarch.rpm 1.5 MB/s | 103 kB 00:00 2024-03-21T10:08:43.083 INFO:teuthology.orchestra.run.smithi157.stdout:(89/174): python3-pycparser-2.20-6.el9.noarch.r 2.5 MB/s | 124 kB 00:00 2024-03-21T10:08:43.116 INFO:teuthology.orchestra.run.smithi157.stdout:(90/174): python3-urllib3-1.26.5-5.el9.noarch.r 3.2 MB/s | 187 kB 00:00 2024-03-21T10:08:43.126 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 95/198 2024-03-21T10:08:43.128 INFO:teuthology.orchestra.run.smithi100.stdout: Running scriptlet: libcephsqlite-2:19.0.0-2231.g2edfee68.el9.x86_64 16/198 2024-03-21T10:08:43.157 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : libradosstriper1-2:19.0.0-2231.g2edfee68.el9.x86 17/198 2024-03-21T10:08:43.193 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 96/198 2024-03-21T10:08:43.229 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 97/198 2024-03-21T10:08:43.246 INFO:teuthology.orchestra.run.smithi100.stdout: Running scriptlet: libradosstriper1-2:19.0.0-2231.g2edfee68.el9.x86 17/198 2024-03-21T10:08:43.295 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 98/198 2024-03-21T10:08:43.300 INFO:teuthology.orchestra.run.smithi100.stdout: Upgrading : libmount-2.37.4-18.el9.x86_64 18/198 2024-03-21T10:08:43.325 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : qatlib-service-23.11.0-1.el9.x86_64 19/198 2024-03-21T10:08:43.340 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 99/198 2024-03-21T10:08:43.397 INFO:teuthology.orchestra.run.smithi100.stdout: Running scriptlet: qatlib-service-23.11.0-1.el9.x86_64 19/198 2024-03-21T10:08:43.494 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 100/198 2024-03-21T10:08:43.613 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 101/198 2024-03-21T10:08:43.633 INFO:teuthology.orchestra.run.smithi157.stdout:(91/174): ceph-test-19.0.0-2231.g2edfee68.el9.x 8.8 MB/s | 48 MB 00:05 2024-03-21T10:08:43.662 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 102/198 2024-03-21T10:08:43.705 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 103/198 2024-03-21T10:08:43.717 INFO:teuthology.orchestra.run.smithi157.stdout:(92/174): jq-1.6-16.el9.x86_64.rpm 296 kB/s | 187 kB 00:00 2024-03-21T10:08:43.726 INFO:teuthology.orchestra.run.smithi100.stdout: Running scriptlet: qatlib-23.11.0-1.el9.x86_64 20/198 2024-03-21T10:08:43.764 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 104/198 2024-03-21T10:08:43.766 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : qatlib-23.11.0-1.el9.x86_64 20/198 2024-03-21T10:08:43.802 INFO:teuthology.orchestra.run.smithi157.stdout:(93/174): python3-requests-2.25.1-8.el9.noarch. 1.5 MB/s | 125 kB 00:00 2024-03-21T10:08:43.826 INFO:teuthology.orchestra.run.smithi100.stdout: Upgrading : libsmartcols-2.37.4-18.el9.x86_64 21/198 2024-03-21T10:08:43.834 INFO:teuthology.orchestra.run.smithi157.stdout:(94/174): libquadmath-11.4.1-3.el9.x86_64.rpm 979 kB/s | 195 kB 00:00 2024-03-21T10:08:43.876 INFO:teuthology.orchestra.run.smithi157.stdout:(95/174): libgfortran-11.4.1-3.el9.x86_64.rpm 1.0 MB/s | 803 kB 00:00 2024-03-21T10:08:43.946 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-packaging-20.9-5.el9.noarch 105/198 2024-03-21T10:08:43.969 INFO:teuthology.orchestra.run.smithi157.stdout:(96/174): elfutils-libelf-devel-0.190-2.el9.x86 129 kB/s | 21 kB 00:00 2024-03-21T10:08:43.993 INFO:teuthology.orchestra.run.smithi157.stdout:(97/174): libuuid-devel-2.37.4-18.el9.x86_64.rp 161 kB/s | 25 kB 00:00 2024-03-21T10:08:44.013 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 106/198 2024-03-21T10:08:44.152 INFO:teuthology.orchestra.run.smithi100.stdout: Upgrading : libacl-2.3.1-4.el9.x86_64 22/198 2024-03-21T10:08:44.160 INFO:teuthology.orchestra.run.smithi157.stdout:(98/174): python3-devel-3.9.17-2.el9.x86_64.rpm 1.4 MB/s | 245 kB 00:00 2024-03-21T10:08:44.185 INFO:teuthology.orchestra.run.smithi157.stdout:(99/174): python3-jinja2-2.11.3-5.el9.noarch.rp 1.3 MB/s | 248 kB 00:00 2024-03-21T10:08:44.244 INFO:teuthology.orchestra.run.smithi157.stdout:(100/174): qatlib-service-23.11.0-1.el9.x86_64. 608 kB/s | 35 kB 00:00 2024-03-21T10:08:44.277 INFO:teuthology.orchestra.run.smithi157.stdout:(101/174): qatlib-23.11.0-1.el9.x86_64.rpm 2.3 MB/s | 216 kB 00:00 2024-03-21T10:08:44.303 INFO:teuthology.orchestra.run.smithi157.stdout:(102/174): qatzip-libs-1.1.2-2.el9.x86_64.rpm 768 kB/s | 44 kB 00:00 2024-03-21T10:08:44.336 INFO:teuthology.orchestra.run.smithi157.stdout:(103/174): xfsprogs-devel-6.3.0-1.el9.x86_64.rp 1.6 MB/s | 94 kB 00:00 2024-03-21T10:08:44.399 INFO:teuthology.orchestra.run.smithi100.stdout: Upgrading : NetworkManager-libnm-1:1.47.2-1.el9.x86_64 23/198 2024-03-21T10:08:44.445 INFO:teuthology.orchestra.run.smithi157.stdout:(104/174): openssl-devel-3.0.7-27.el9.x86_64.rp 7.3 MB/s | 4.1 MB 00:00 2024-03-21T10:08:44.488 INFO:teuthology.orchestra.run.smithi100.stdout: Running scriptlet: NetworkManager-libnm-1:1.47.2-1.el9.x86_64 23/198 2024-03-21T10:08:44.653 INFO:teuthology.orchestra.run.smithi157.stdout:(105/174): lua-devel-5.4.4-4.el9.x86_64.rpm 64 kB/s | 22 kB 00:00 2024-03-21T10:08:44.678 INFO:teuthology.orchestra.run.smithi100.stdout: Running scriptlet: NetworkManager-1:1.47.2-1.el9.x86_64 24/198 2024-03-21T10:08:44.736 INFO:teuthology.orchestra.run.smithi100.stdout: Upgrading : NetworkManager-1:1.47.2-1.el9.x86_64 24/198 2024-03-21T10:08:44.837 INFO:teuthology.orchestra.run.smithi157.stdout:(106/174): protobuf-compiler-3.14.0-13.el9.x86_ 1.7 MB/s | 863 kB 00:00 2024-03-21T10:08:44.992 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 107/198 2024-03-21T10:08:45.045 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 108/198 2024-03-21T10:08:45.055 INFO:teuthology.orchestra.run.smithi100.stdout: Running scriptlet: NetworkManager-1:1.47.2-1.el9.x86_64 24/198 2024-03-21T10:08:45.103 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 109/198 2024-03-21T10:08:45.163 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 110/198 2024-03-21T10:08:45.237 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : re2-1:20211101-3.el9.x86_64 25/198 2024-03-21T10:08:45.238 INFO:teuthology.orchestra.run.smithi157.stdout:(107/174): abseil-cpp-20211102.0-3.el9.x86_64.r 754 kB/s | 548 kB 00:00 2024-03-21T10:08:45.254 INFO:teuthology.orchestra.run.smithi157.stdout:(108/174): grpc-data-1.46.7-10.el9.noarch.rpm 47 kB/s | 19 kB 00:00 2024-03-21T10:08:45.281 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 111/198 2024-03-21T10:08:45.321 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 112/198 2024-03-21T10:08:45.350 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : libarrow-9.0.0-10.el9.x86_64 26/198 2024-03-21T10:08:45.380 INFO:teuthology.orchestra.run.smithi157.stdout:(109/174): gperftools-libs-2.9.1-2.el9.x86_64.r 426 kB/s | 309 kB 00:00 2024-03-21T10:08:45.391 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 27/198 2024-03-21T10:08:45.405 INFO:teuthology.orchestra.run.smithi157.stdout:(110/174): libarrow-doc-9.0.0-10.el9.noarch.rpm 173 kB/s | 26 kB 00:00 2024-03-21T10:08:45.419 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : liboath-2.6.7-2.el9.x86_64 28/198 2024-03-21T10:08:45.469 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 29/198 2024-03-21T10:08:45.489 INFO:teuthology.orchestra.run.smithi157.stdout:(111/174): liboath-2.6.7-2.el9.x86_64.rpm 451 kB/s | 49 kB 00:00 2024-03-21T10:08:45.574 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : openblas-0.3.21-2.el9.x86_64 113/198 2024-03-21T10:08:45.576 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 30/198 2024-03-21T10:08:45.601 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 114/198 2024-03-21T10:08:45.614 INFO:teuthology.orchestra.run.smithi157.stdout:(112/174): luarocks-3.9.2-1.el9.noarch.rpm 1.2 MB/s | 151 kB 00:00 2024-03-21T10:08:45.639 INFO:teuthology.orchestra.run.smithi157.stdout:(113/174): libunwind-1.6.2-1.el9.x86_64.rpm 289 kB/s | 67 kB 00:00 2024-03-21T10:08:45.641 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 31/198 2024-03-21T10:08:45.691 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 32/198 2024-03-21T10:08:45.697 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 115/198 2024-03-21T10:08:45.781 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 33/198 2024-03-21T10:08:45.781 INFO:teuthology.orchestra.run.smithi157.stdout:(114/174): libarrow-9.0.0-10.el9.x86_64.rpm 8.2 MB/s | 4.4 MB 00:00 2024-03-21T10:08:45.830 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 34/198 2024-03-21T10:08:45.892 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 35/198 2024-03-21T10:08:45.898 INFO:teuthology.orchestra.run.smithi157.stdout:(115/174): parquet-libs-9.0.0-10.el9.x86_64.rpm 2.9 MB/s | 838 kB 00:00 2024-03-21T10:08:45.923 INFO:teuthology.orchestra.run.smithi157.stdout:(116/174): python3-bcrypt-3.2.2-1.el9.x86_64.rp 308 kB/s | 43 kB 00:00 2024-03-21T10:08:45.934 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 36/198 2024-03-21T10:08:45.966 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : lua-5.4.4-4.el9.x86_64 37/198 2024-03-21T10:08:45.999 INFO:teuthology.orchestra.run.smithi157.stdout:(117/174): python3-cachetools-4.2.4-1.el9.noarc 325 kB/s | 32 kB 00:00 2024-03-21T10:08:46.041 INFO:teuthology.orchestra.run.smithi157.stdout:(118/174): python3-certifi-2021.10.8-2.el9.noar 129 kB/s | 15 kB 00:00 2024-03-21T10:08:46.070 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 38/198 2024-03-21T10:08:46.074 INFO:teuthology.orchestra.run.smithi157.stdout:(119/174): python3-asyncssh-2.13.2-5.el9.noarch 1.2 MB/s | 548 kB 00:00 2024-03-21T10:08:46.108 INFO:teuthology.orchestra.run.smithi157.stdout:(120/174): python3-cheroot-8.6.0-4.el9.noarch.r 1.6 MB/s | 172 kB 00:00 2024-03-21T10:08:46.111 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-ceph-common-2:19.0.0-2231.g2edfee68.el9. 39/198 2024-03-21T10:08:46.166 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-ceph-argparse-2:19.0.0-2231.g2edfee68.el 40/198 2024-03-21T10:08:46.166 INFO:teuthology.orchestra.run.smithi157.stdout:(121/174): python3-cherrypy-18.6.1-2.el9.noarch 2.8 MB/s | 358 kB 00:00 2024-03-21T10:08:46.202 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-cephfs-2:19.0.0-2231.g2edfee68.el9.x86_6 41/198 2024-03-21T10:08:46.208 INFO:teuthology.orchestra.run.smithi157.stdout:(122/174): python3-cmd2-2.4.2-2.el9.noarch.rpm 2.2 MB/s | 303 kB 00:00 2024-03-21T10:08:46.234 INFO:teuthology.orchestra.run.smithi157.stdout:(123/174): python3-colorama-0.4.4-12.el9.noarch 282 kB/s | 35 kB 00:00 2024-03-21T10:08:46.275 INFO:teuthology.orchestra.run.smithi157.stdout:(124/174): python3-google-auth-2.28.2-1.el9.noa 1.8 MB/s | 203 kB 00:00 2024-03-21T10:08:46.321 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 42/198 2024-03-21T10:08:46.351 INFO:teuthology.orchestra.run.smithi157.stdout:(125/174): python3-grpcio-tools-1.46.7-10.el9.x 1.2 MB/s | 144 kB 00:00 2024-03-21T10:08:46.376 INFO:teuthology.orchestra.run.smithi157.stdout:(126/174): python3-jaraco-8.2.1-3.el9.noarch.rp 107 kB/s | 11 kB 00:00 2024-03-21T10:08:46.379 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : luarocks-3.9.2-1.el9.noarch 43/198 2024-03-21T10:08:46.435 INFO:teuthology.orchestra.run.smithi157.stdout:(127/174): python3-grpcio-1.46.7-10.el9.x86_64. 9.0 MB/s | 2.0 MB 00:00 2024-03-21T10:08:46.460 INFO:teuthology.orchestra.run.smithi157.stdout:(128/174): python3-jaraco-classes-3.2.1-5.el9.n 164 kB/s | 18 kB 00:00 2024-03-21T10:08:46.478 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 44/198 2024-03-21T10:08:46.485 INFO:teuthology.orchestra.run.smithi157.stdout:(129/174): python3-jaraco-collections-3.0.0-8.e 214 kB/s | 23 kB 00:00 2024-03-21T10:08:46.535 INFO:teuthology.orchestra.run.smithi157.stdout:(130/174): python3-jaraco-functools-3.5.0-2.el9 195 kB/s | 19 kB 00:00 2024-03-21T10:08:46.655 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 45/198 2024-03-21T10:08:46.655 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 116/198 2024-03-21T10:08:46.656 INFO:teuthology.orchestra.run.smithi157.stdout:(131/174): python3-jaraco-text-3.2.0-6.el9.noar 197 kB/s | 20 kB 00:00 2024-03-21T10:08:46.656 INFO:teuthology.orchestra.run.smithi157.stdout:(132/174): python3-logutils-0.3.5-21.el9.noarch 465 kB/s | 46 kB 00:00 2024-03-21T10:08:46.656 INFO:teuthology.orchestra.run.smithi157.stdout:(133/174): python3-more-itertools-8.12.0-2.el9. 863 kB/s | 79 kB 00:00 2024-03-21T10:08:46.692 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 117/198 2024-03-21T10:08:46.709 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 46/198 2024-03-21T10:08:46.720 INFO:teuthology.orchestra.run.smithi157.stdout:(134/174): python3-kubernetes-26.1.0-2.el9.noar 4.4 MB/s | 1.0 MB 00:00 2024-03-21T10:08:46.736 INFO:teuthology.orchestra.run.smithi157.stdout:(135/174): python3-natsort-7.1.1-5.el9.noarch.r 580 kB/s | 58 kB 00:00 2024-03-21T10:08:46.770 INFO:teuthology.orchestra.run.smithi157.stdout:(136/174): python3-pecan-1.4.2-3.el9.noarch.rpm 2.3 MB/s | 272 kB 00:00 2024-03-21T10:08:46.812 INFO:teuthology.orchestra.run.smithi157.stdout:(137/174): python3-portend-3.1.0-2.el9.noarch.r 179 kB/s | 16 kB 00:00 2024-03-21T10:08:46.838 INFO:teuthology.orchestra.run.smithi157.stdout:(138/174): python3-pyOpenSSL-21.0.0-1.el9.noarc 899 kB/s | 90 kB 00:00 2024-03-21T10:08:46.872 INFO:teuthology.orchestra.run.smithi157.stdout:(139/174): python3-pyperclip-1.8.2-2.el9.noarch 281 kB/s | 28 kB 00:00 2024-03-21T10:08:46.914 INFO:teuthology.orchestra.run.smithi157.stdout:(140/174): python3-repoze-lru-0.7-16.el9.noarch 307 kB/s | 31 kB 00:00 2024-03-21T10:08:46.947 INFO:teuthology.orchestra.run.smithi157.stdout:(141/174): python3-routes-2.5.1-5.el9.noarch.rp 1.7 MB/s | 188 kB 00:00 2024-03-21T10:08:47.006 INFO:teuthology.orchestra.run.smithi157.stdout:(142/174): python3-rsa-4.9-2.el9.noarch.rpm 443 kB/s | 59 kB 00:00 2024-03-21T10:08:47.039 INFO:teuthology.orchestra.run.smithi157.stdout:(143/174): python3-tempora-5.0.0-2.el9.noarch.r 286 kB/s | 36 kB 00:00 2024-03-21T10:08:47.065 INFO:teuthology.orchestra.run.smithi157.stdout:(144/174): python3-typing-extensions-4.4.0-2.el 438 kB/s | 51 kB 00:00 2024-03-21T10:08:47.123 INFO:teuthology.orchestra.run.smithi157.stdout:(145/174): python3-webob-1.8.7-6.el9.noarch.rpm 1.9 MB/s | 230 kB 00:00 2024-03-21T10:08:47.148 INFO:teuthology.orchestra.run.smithi157.stdout:(146/174): python3-websocket-client-1.2.3-2.el9 829 kB/s | 90 kB 00:00 2024-03-21T10:08:47.182 INFO:teuthology.orchestra.run.smithi157.stdout:(147/174): python3-werkzeug-2.0.3-3.el9.1.noarc 3.6 MB/s | 427 kB 00:00 2024-03-21T10:08:47.224 INFO:teuthology.orchestra.run.smithi157.stdout:(148/174): python3-zc-lockfile-2.0-10.el9.noarc 200 kB/s | 20 kB 00:00 2024-03-21T10:08:47.257 INFO:teuthology.orchestra.run.smithi157.stdout:(149/174): re2-20211101-3.el9.x86_64.rpm 1.7 MB/s | 192 kB 00:00 2024-03-21T10:08:47.395 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-rsa-4.9-2.el9.noarch 47/198 2024-03-21T10:08:47.433 INFO:teuthology.orchestra.run.smithi157.stdout:(150/174): thrift-0.15.0-2.el9.x86_64.rpm 6.3 MB/s | 1.6 MB 00:00 2024-03-21T10:08:47.489 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 48/198 2024-03-21T10:08:47.591 INFO:teuthology.orchestra.run.smithi157.stdout:(151/174): xfsdump-3.1.12-4.el9.x86_64.rpm 2.0 MB/s | 321 kB 00:00 2024-03-21T10:08:47.609 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 49/198 2024-03-21T10:08:47.666 INFO:teuthology.orchestra.run.smithi100.stdout: Upgrading : util-linux-core-2.37.4-18.el9.x86_64 50/198 2024-03-21T10:08:47.689 INFO:teuthology.orchestra.run.smithi100.stdout: Running scriptlet: util-linux-core-2.37.4-18.el9.x86_64 50/198 2024-03-21T10:08:47.732 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : qatzip-libs-1.1.2-2.el9.x86_64 51/198 2024-03-21T10:08:47.799 INFO:teuthology.orchestra.run.smithi100.stdout: Upgrading : elfutils-libs-0.190-2.el9.x86_64 52/198 2024-03-21T10:08:47.849 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : librados-devel-2:19.0.0-2231.g2edfee68.el9.x86_6 53/198 2024-03-21T10:08:47.889 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 54/198 2024-03-21T10:08:48.313 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 118/198 2024-03-21T10:08:48.322 INFO:teuthology.orchestra.run.smithi100.stdout: Upgrading : libfdisk-2.37.4-18.el9.x86_64 55/198 2024-03-21T10:08:48.400 INFO:teuthology.orchestra.run.smithi157.stdout:(152/174): NetworkManager-1.47.2-1.el9.x86_64.r 2.9 MB/s | 2.3 MB 00:00 2024-03-21T10:08:48.413 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 119/198 2024-03-21T10:08:48.452 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : oniguruma-6.9.6-1.el9.5.x86_64 120/198 2024-03-21T10:08:48.482 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : jq-1.6-16.el9.x86_64 121/198 2024-03-21T10:08:48.523 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 122/198 2024-03-21T10:08:48.567 INFO:teuthology.orchestra.run.smithi100.stdout: Upgrading : util-linux-2.37.4-18.el9.x86_64 56/198 2024-03-21T10:08:48.645 INFO:teuthology.orchestra.run.smithi100.stdout: Upgrading : xfsprogs-6.3.0-1.el9.x86_64 57/198 2024-03-21T10:08:48.700 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 58/198 2024-03-21T10:08:48.800 INFO:teuthology.orchestra.run.smithi157.stdout:(153/174): NetworkManager-libnm-1.47.2-1.el9.x8 4.6 MB/s | 1.8 MB 00:00 2024-03-21T10:08:48.859 INFO:teuthology.orchestra.run.smithi157.stdout:(154/174): NetworkManager-team-1.47.2-1.el9.x86 735 kB/s | 42 kB 00:00 2024-03-21T10:08:48.903 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 59/198 2024-03-21T10:08:48.919 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 123/198 2024-03-21T10:08:48.954 INFO:teuthology.orchestra.run.smithi047.stdout: Upgrading : librbd1-2:19.0.0-2231.g2edfee68.el9.x86_64 124/198 2024-03-21T10:08:48.959 INFO:teuthology.orchestra.run.smithi100.stdout: Upgrading : openssl-1:3.0.7-27.el9.x86_64 60/198 2024-03-21T10:08:49.009 INFO:teuthology.orchestra.run.smithi157.stdout:(155/174): NetworkManager-tui-1.47.2-1.el9.x86_ 1.6 MB/s | 248 kB 00:00 2024-03-21T10:08:49.033 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 61/198 2024-03-21T10:08:49.059 INFO:teuthology.orchestra.run.smithi157.stdout:(156/174): acl-2.3.1-4.el9.x86_64.rpm 1.4 MB/s | 71 kB 00:00 2024-03-21T10:08:49.074 INFO:teuthology.orchestra.run.smithi047.stdout: Running scriptlet: librbd1-2:19.0.0-2231.g2edfee68.el9.x86_64 124/198 2024-03-21T10:08:49.118 INFO:teuthology.orchestra.run.smithi157.stdout:(157/174): elfutils-debuginfod-client-0.190-2.e 634 kB/s | 36 kB 00:00 2024-03-21T10:08:49.118 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 62/198 2024-03-21T10:08:49.153 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-rbd-2:19.0.0-2231.g2edfee68.el9.x86_64 125/198 2024-03-21T10:08:49.158 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 63/198 2024-03-21T10:08:49.201 INFO:teuthology.orchestra.run.smithi157.stdout:(158/174): elfutils-libelf-0.190-2.el9.x86_64.r 2.3 MB/s | 192 kB 00:00 2024-03-21T10:08:49.205 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 64/198 2024-03-21T10:08:49.281 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 65/198 2024-03-21T10:08:49.302 INFO:teuthology.orchestra.run.smithi157.stdout:(159/174): librbd1-19.0.0-2231.g2edfee68.el9.x8 1.5 MB/s | 3.1 MB 00:02 2024-03-21T10:08:49.322 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 66/198 2024-03-21T10:08:49.335 INFO:teuthology.orchestra.run.smithi157.stdout:(160/174): elfutils-libs-0.190-2.el9.x86_64.rpm 1.9 MB/s | 256 kB 00:00 2024-03-21T10:08:49.384 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 67/198 2024-03-21T10:08:49.419 INFO:teuthology.orchestra.run.smithi157.stdout:(161/174): librados2-19.0.0-2231.g2edfee68.el9. 1.5 MB/s | 3.4 MB 00:02 2024-03-21T10:08:49.443 INFO:teuthology.orchestra.run.smithi157.stdout:(162/174): libacl-2.3.1-4.el9.x86_64.rpm 165 kB/s | 23 kB 00:00 2024-03-21T10:08:49.444 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 68/198 2024-03-21T10:08:49.468 INFO:teuthology.orchestra.run.smithi157.stdout:(163/174): libblkid-2.37.4-18.el9.x86_64.rpm 823 kB/s | 108 kB 00:00 2024-03-21T10:08:49.526 INFO:teuthology.orchestra.run.smithi157.stdout:(164/174): libsmartcols-2.37.4-18.el9.x86_64.rp 1.1 MB/s | 64 kB 00:00 2024-03-21T10:08:49.535 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 126/198 2024-03-21T10:08:49.585 INFO:teuthology.orchestra.run.smithi157.stdout:(165/174): libuuid-2.37.4-18.el9.x86_64.rpm 497 kB/s | 29 kB 00:00 2024-03-21T10:08:49.611 INFO:teuthology.orchestra.run.smithi157.stdout:(166/174): libmount-2.37.4-18.el9.x86_64.rpm 811 kB/s | 136 kB 00:00 2024-03-21T10:08:49.632 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 69/198 2024-03-21T10:08:49.636 INFO:teuthology.orchestra.run.smithi157.stdout:(167/174): libfdisk-2.37.4-18.el9.x86_64.rpm 717 kB/s | 155 kB 00:00 2024-03-21T10:08:49.696 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 70/198 2024-03-21T10:08:49.733 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-colorama-0.4.4-12.el9.noarch 71/198 2024-03-21T10:08:49.799 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 72/198 2024-03-21T10:08:49.836 INFO:teuthology.orchestra.run.smithi157.stdout:(168/174): openssl-3.0.7-27.el9.x86_64.rpm 4.8 MB/s | 1.2 MB 00:00 2024-03-21T10:08:49.928 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 73/198 2024-03-21T10:08:49.970 INFO:teuthology.orchestra.run.smithi157.stdout:(169/174): util-linux-core-2.37.4-18.el9.x86_64 3.7 MB/s | 465 kB 00:00 2024-03-21T10:08:49.973 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-google-auth-1:2.28.2-1.el9.noarch 74/198 2024-03-21T10:08:50.019 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 75/198 2024-03-21T10:08:50.028 INFO:teuthology.orchestra.run.smithi157.stdout:(170/174): util-linux-user-2.37.4-18.el9.x86_64 542 kB/s | 31 kB 00:00 2024-03-21T10:08:50.057 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 76/198 2024-03-21T10:08:50.089 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : libarrow-doc-9.0.0-10.el9.noarch 77/198 2024-03-21T10:08:50.125 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : parquet-libs-9.0.0-10.el9.x86_64 127/198 2024-03-21T10:08:50.153 INFO:teuthology.orchestra.run.smithi157.stdout:(171/174): openssl-libs-3.0.7-27.el9.x86_64.rpm 4.0 MB/s | 2.2 MB 00:00 2024-03-21T10:08:50.165 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : librgw2-2:19.0.0-2231.g2edfee68.el9.x86_64 128/198 2024-03-21T10:08:50.220 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 78/198 2024-03-21T10:08:50.229 INFO:teuthology.orchestra.run.smithi157.stdout:(172/174): util-linux-2.37.4-18.el9.x86_64.rpm 3.9 MB/s | 2.3 MB 00:00 2024-03-21T10:08:50.265 INFO:teuthology.orchestra.run.smithi047.stdout: Running scriptlet: librgw2-2:19.0.0-2231.g2edfee68.el9.x86_64 128/198 2024-03-21T10:08:50.271 INFO:teuthology.orchestra.run.smithi157.stdout:(173/174): xfsprogs-6.3.0-1.el9.x86_64.rpm 4.4 MB/s | 1.1 MB 00:00 2024-03-21T10:08:50.292 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-rgw-2:19.0.0-2231.g2edfee68.el9.x86_64 129/198 2024-03-21T10:08:50.354 INFO:teuthology.orchestra.run.smithi157.stdout:(174/174): libacl-devel-2.3.1-4.el9.x86_64.rpm 373 kB/s | 74 kB 00:00 2024-03-21T10:08:50.361 INFO:teuthology.orchestra.run.smithi157.stdout:-------------------------------------------------------------------------------- 2024-03-21T10:08:50.362 INFO:teuthology.orchestra.run.smithi157.stdout:Total 14 MB/s | 224 MB 00:16 2024-03-21T10:08:50.392 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 79/198 2024-03-21T10:08:50.520 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 80/198 2024-03-21T10:08:50.574 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-devel-3.9.17-2.el9.x86_64 81/198 2024-03-21T10:08:50.635 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-ply-3.11-14.el9.noarch 82/198 2024-03-21T10:08:50.718 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 83/198 2024-03-21T10:08:50.957 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 84/198 2024-03-21T10:08:51.012 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 85/198 2024-03-21T10:08:51.184 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 86/198 2024-03-21T10:08:51.242 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 87/198 2024-03-21T10:08:51.351 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 88/198 2024-03-21T10:08:51.694 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : fuse-2.9.9-15.el9.x86_64 89/198 2024-03-21T10:08:51.756 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : zlib-devel-1.2.11-40.el9.x86_64 90/198 2024-03-21T10:08:51.807 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 91/198 2024-03-21T10:08:51.821 INFO:teuthology.orchestra.run.smithi157.stdout:Running transaction check 2024-03-21T10:08:51.875 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : uuid-1.6.2-55.el9.x86_64 92/198 2024-03-21T10:08:51.928 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 93/198 2024-03-21T10:08:51.977 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 94/198 2024-03-21T10:08:52.113 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 95/198 2024-03-21T10:08:52.155 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 96/198 2024-03-21T10:08:52.189 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 97/198 2024-03-21T10:08:52.200 INFO:teuthology.orchestra.run.smithi157.stdout:Transaction check succeeded. 2024-03-21T10:08:52.200 INFO:teuthology.orchestra.run.smithi157.stdout:Running transaction test 2024-03-21T10:08:52.250 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 98/198 2024-03-21T10:08:52.284 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 99/198 2024-03-21T10:08:52.422 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 100/198 2024-03-21T10:08:52.541 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 101/198 2024-03-21T10:08:52.603 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 102/198 2024-03-21T10:08:52.635 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 103/198 2024-03-21T10:08:52.692 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 104/198 2024-03-21T10:08:52.840 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-packaging-20.9-5.el9.noarch 105/198 2024-03-21T10:08:52.910 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 106/198 2024-03-21T10:08:52.981 INFO:teuthology.orchestra.run.smithi047.stdout: Running scriptlet: ceph-common-2:19.0.0-2231.g2edfee68.el9.x86_64 130/198 2024-03-21T10:08:53.022 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : ceph-common-2:19.0.0-2231.g2edfee68.el9.x86_64 130/198 2024-03-21T10:08:53.682 INFO:teuthology.orchestra.run.smithi047.stdout: Running scriptlet: ceph-common-2:19.0.0-2231.g2edfee68.el9.x86_64 130/198 2024-03-21T10:08:53.734 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : ceph-base-2:19.0.0-2231.g2edfee68.el9.x86_64 131/198 2024-03-21T10:08:53.856 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 107/198 2024-03-21T10:08:53.874 INFO:teuthology.orchestra.run.smithi047.stdout: Running scriptlet: ceph-base-2:19.0.0-2231.g2edfee68.el9.x86_64 131/198 2024-03-21T10:08:53.875 INFO:teuthology.orchestra.run.smithi047.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-03-21T10:08:53.875 INFO:teuthology.orchestra.run.smithi047.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-03-21T10:08:53.875 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-21T10:08:53.910 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : ceph-selinux-2:19.0.0-2231.g2edfee68.el9.x86_64 132/198 2024-03-21T10:08:53.916 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 108/198 2024-03-21T10:08:53.950 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 109/198 2024-03-21T10:08:54.008 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 110/198 2024-03-21T10:08:54.126 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 111/198 2024-03-21T10:08:54.167 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 112/198 2024-03-21T10:08:54.400 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : openblas-0.3.21-2.el9.x86_64 113/198 2024-03-21T10:08:54.431 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 114/198 2024-03-21T10:08:54.524 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 115/198 2024-03-21T10:08:54.564 INFO:teuthology.orchestra.run.smithi157.stdout:Transaction test succeeded. 2024-03-21T10:08:54.564 INFO:teuthology.orchestra.run.smithi157.stdout:Running transaction 2024-03-21T10:08:55.327 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 116/198 2024-03-21T10:08:55.468 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 117/198 2024-03-21T10:08:56.970 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 118/198 2024-03-21T10:08:57.077 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 119/198 2024-03-21T10:08:57.141 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : oniguruma-6.9.6-1.el9.5.x86_64 120/198 2024-03-21T10:08:57.172 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : jq-1.6-16.el9.x86_64 121/198 2024-03-21T10:08:57.213 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 122/198 2024-03-21T10:08:57.593 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 123/198 2024-03-21T10:08:57.626 INFO:teuthology.orchestra.run.smithi100.stdout: Upgrading : librbd1-2:19.0.0-2231.g2edfee68.el9.x86_64 124/198 2024-03-21T10:08:57.745 INFO:teuthology.orchestra.run.smithi100.stdout: Running scriptlet: librbd1-2:19.0.0-2231.g2edfee68.el9.x86_64 124/198 2024-03-21T10:08:57.774 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-rbd-2:19.0.0-2231.g2edfee68.el9.x86_64 125/198 2024-03-21T10:08:57.828 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 126/198 2024-03-21T10:08:58.340 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : parquet-libs-9.0.0-10.el9.x86_64 127/198 2024-03-21T10:08:58.371 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : librgw2-2:19.0.0-2231.g2edfee68.el9.x86_64 128/198 2024-03-21T10:08:58.462 INFO:teuthology.orchestra.run.smithi100.stdout: Running scriptlet: librgw2-2:19.0.0-2231.g2edfee68.el9.x86_64 128/198 2024-03-21T10:08:58.489 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-rgw-2:19.0.0-2231.g2edfee68.el9.x86_64 129/198 2024-03-21T10:09:01.047 INFO:teuthology.orchestra.run.smithi100.stdout: Running scriptlet: ceph-common-2:19.0.0-2231.g2edfee68.el9.x86_64 130/198 2024-03-21T10:09:01.086 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : ceph-common-2:19.0.0-2231.g2edfee68.el9.x86_64 130/198 2024-03-21T10:09:01.705 INFO:teuthology.orchestra.run.smithi100.stdout: Running scriptlet: ceph-common-2:19.0.0-2231.g2edfee68.el9.x86_64 130/198 2024-03-21T10:09:01.748 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : ceph-base-2:19.0.0-2231.g2edfee68.el9.x86_64 131/198 2024-03-21T10:09:01.791 INFO:teuthology.orchestra.run.smithi157.stdout: Preparing : 1/1 2024-03-21T10:09:01.912 INFO:teuthology.orchestra.run.smithi157.stdout: Upgrading : openssl-libs-1:3.0.7-27.el9.x86_64 1/198 2024-03-21T10:09:01.927 INFO:teuthology.orchestra.run.smithi100.stdout: Running scriptlet: ceph-base-2:19.0.0-2231.g2edfee68.el9.x86_64 131/198 2024-03-21T10:09:01.927 INFO:teuthology.orchestra.run.smithi100.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-03-21T10:09:01.928 INFO:teuthology.orchestra.run.smithi100.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-03-21T10:09:01.928 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-03-21T10:09:01.960 INFO:teuthology.orchestra.run.smithi157.stdout: Upgrading : libuuid-2.37.4-18.el9.x86_64 2/198 2024-03-21T10:09:01.998 INFO:teuthology.orchestra.run.smithi157.stdout: Upgrading : libblkid-2.37.4-18.el9.x86_64 3/198 2024-03-21T10:09:01.999 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : ceph-selinux-2:19.0.0-2231.g2edfee68.el9.x86_64 132/198 2024-03-21T10:09:02.043 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: libblkid-2.37.4-18.el9.x86_64 3/198 2024-03-21T10:09:02.128 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 4/198 2024-03-21T10:09:02.228 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 5/198 2024-03-21T10:09:02.280 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : thrift-0.15.0-2.el9.x86_64 6/198 2024-03-21T10:09:02.313 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 7/198 2024-03-21T10:09:02.395 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 8/198 2024-03-21T10:09:02.801 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 9/198 2024-03-21T10:09:02.836 INFO:teuthology.orchestra.run.smithi157.stdout: Upgrading : librados2-2:19.0.0-2231.g2edfee68.el9.x86_64 10/198 2024-03-21T10:09:02.994 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: librados2-2:19.0.0-2231.g2edfee68.el9.x86_64 10/198 2024-03-21T10:09:03.020 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : libcephfs2-2:19.0.0-2231.g2edfee68.el9.x86_64 11/198 2024-03-21T10:09:03.122 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: libcephfs2-2:19.0.0-2231.g2edfee68.el9.x86_64 11/198 2024-03-21T10:09:03.150 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-rados-2:19.0.0-2231.g2edfee68.el9.x86_64 12/198 2024-03-21T10:09:03.201 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : postgresql-private-libs-13.11-1.el9.x86_64 13/198 2024-03-21T10:09:03.488 INFO:teuthology.orchestra.run.smithi157.stdout: Upgrading : elfutils-libelf-0.190-2.el9.x86_64 14/198 2024-03-21T10:09:03.543 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : postgresql-13.11-1.el9.x86_64 15/198 2024-03-21T10:09:03.569 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : libcephsqlite-2:19.0.0-2231.g2edfee68.el9.x86_64 16/198 2024-03-21T10:09:03.712 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: libcephsqlite-2:19.0.0-2231.g2edfee68.el9.x86_64 16/198 2024-03-21T10:09:03.745 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : libradosstriper1-2:19.0.0-2231.g2edfee68.el9.x86 17/198 2024-03-21T10:09:03.834 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: libradosstriper1-2:19.0.0-2231.g2edfee68.el9.x86 17/198 2024-03-21T10:09:03.880 INFO:teuthology.orchestra.run.smithi157.stdout: Upgrading : libmount-2.37.4-18.el9.x86_64 18/198 2024-03-21T10:09:03.905 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : qatlib-service-23.11.0-1.el9.x86_64 19/198 2024-03-21T10:09:03.977 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: qatlib-service-23.11.0-1.el9.x86_64 19/198 2024-03-21T10:09:04.269 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: qatlib-23.11.0-1.el9.x86_64 20/198 2024-03-21T10:09:04.316 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : qatlib-23.11.0-1.el9.x86_64 20/198 2024-03-21T10:09:04.366 INFO:teuthology.orchestra.run.smithi157.stdout: Upgrading : libsmartcols-2.37.4-18.el9.x86_64 21/198 2024-03-21T10:09:04.730 INFO:teuthology.orchestra.run.smithi157.stdout: Upgrading : libacl-2.3.1-4.el9.x86_64 22/198 2024-03-21T10:09:04.923 INFO:teuthology.orchestra.run.smithi157.stdout: Upgrading : NetworkManager-libnm-1:1.47.2-1.el9.x86_64 23/198 2024-03-21T10:09:05.012 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: NetworkManager-libnm-1:1.47.2-1.el9.x86_64 23/198 2024-03-21T10:09:05.207 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: NetworkManager-1:1.47.2-1.el9.x86_64 24/198 2024-03-21T10:09:05.250 INFO:teuthology.orchestra.run.smithi157.stdout: Upgrading : NetworkManager-1:1.47.2-1.el9.x86_64 24/198 2024-03-21T10:09:05.580 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: NetworkManager-1:1.47.2-1.el9.x86_64 24/198 2024-03-21T10:09:05.759 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : re2-1:20211101-3.el9.x86_64 25/198 2024-03-21T10:09:05.885 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : libarrow-9.0.0-10.el9.x86_64 26/198 2024-03-21T10:09:05.923 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 27/198 2024-03-21T10:09:05.960 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : liboath-2.6.7-2.el9.x86_64 28/198 2024-03-21T10:09:06.018 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 29/198 2024-03-21T10:09:06.128 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 30/198 2024-03-21T10:09:06.199 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 31/198 2024-03-21T10:09:06.248 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 32/198 2024-03-21T10:09:06.339 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 33/198 2024-03-21T10:09:06.386 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 34/198 2024-03-21T10:09:06.448 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 35/198 2024-03-21T10:09:06.492 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 36/198 2024-03-21T10:09:06.523 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : lua-5.4.4-4.el9.x86_64 37/198 2024-03-21T10:09:06.625 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 38/198 2024-03-21T10:09:06.652 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-ceph-common-2:19.0.0-2231.g2edfee68.el9. 39/198 2024-03-21T10:09:06.700 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-ceph-argparse-2:19.0.0-2231.g2edfee68.el 40/198 2024-03-21T10:09:06.733 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-cephfs-2:19.0.0-2231.g2edfee68.el9.x86_6 41/198 2024-03-21T10:09:06.846 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 42/198 2024-03-21T10:09:06.904 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : luarocks-3.9.2-1.el9.noarch 43/198 2024-03-21T10:09:07.003 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 44/198 2024-03-21T10:09:07.174 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 45/198 2024-03-21T10:09:07.240 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 46/198 2024-03-21T10:09:07.495 INFO:teuthology.orchestra.run.smithi047.stdout: Running scriptlet: ceph-selinux-2:19.0.0-2231.g2edfee68.el9.x86_64 132/198 2024-03-21T10:09:07.495 INFO:teuthology.orchestra.run.smithi047.stdout:skipping the directory /sys 2024-03-21T10:09:07.495 INFO:teuthology.orchestra.run.smithi047.stdout:skipping the directory /proc 2024-03-21T10:09:07.495 INFO:teuthology.orchestra.run.smithi047.stdout:skipping the directory /mnt 2024-03-21T10:09:07.495 INFO:teuthology.orchestra.run.smithi047.stdout:skipping the directory /var/tmp 2024-03-21T10:09:07.495 INFO:teuthology.orchestra.run.smithi047.stdout:skipping the directory /home 2024-03-21T10:09:07.495 INFO:teuthology.orchestra.run.smithi047.stdout:skipping the directory /root 2024-03-21T10:09:07.495 INFO:teuthology.orchestra.run.smithi047.stdout:skipping the directory /tmp 2024-03-21T10:09:07.495 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-21T10:09:07.733 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : ceph-volume-2:19.0.0-2231.g2edfee68.el9.noarch 133/198 2024-03-21T10:09:07.783 INFO:teuthology.orchestra.run.smithi047.stdout: Running scriptlet: ceph-volume-2:19.0.0-2231.g2edfee68.el9.noarch 133/198 2024-03-21T10:09:07.783 INFO:teuthology.orchestra.run.smithi047.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-21T10:09:07.783 INFO:teuthology.orchestra.run.smithi047.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-03-21T10:09:07.783 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-21T10:09:07.969 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-rsa-4.9-2.el9.noarch 47/198 2024-03-21T10:09:08.060 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 48/198 2024-03-21T10:09:08.169 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 49/198 2024-03-21T10:09:08.239 INFO:teuthology.orchestra.run.smithi157.stdout: Upgrading : util-linux-core-2.37.4-18.el9.x86_64 50/198 2024-03-21T10:09:08.263 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: util-linux-core-2.37.4-18.el9.x86_64 50/198 2024-03-21T10:09:08.312 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : qatzip-libs-1.1.2-2.el9.x86_64 51/198 2024-03-21T10:09:08.380 INFO:teuthology.orchestra.run.smithi157.stdout: Upgrading : elfutils-libs-0.190-2.el9.x86_64 52/198 2024-03-21T10:09:08.431 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : librados-devel-2:19.0.0-2231.g2edfee68.el9.x86_6 53/198 2024-03-21T10:09:08.478 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 54/198 2024-03-21T10:09:08.883 INFO:teuthology.orchestra.run.smithi157.stdout: Upgrading : libfdisk-2.37.4-18.el9.x86_64 55/198 2024-03-21T10:09:09.114 INFO:teuthology.orchestra.run.smithi157.stdout: Upgrading : util-linux-2.37.4-18.el9.x86_64 56/198 2024-03-21T10:09:09.228 INFO:teuthology.orchestra.run.smithi157.stdout: Upgrading : xfsprogs-6.3.0-1.el9.x86_64 57/198 2024-03-21T10:09:09.297 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 58/198 2024-03-21T10:09:09.485 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : ceph-osd-2:19.0.0-2231.g2edfee68.el9.x86_64 134/198 2024-03-21T10:09:09.517 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 59/198 2024-03-21T10:09:09.566 INFO:teuthology.orchestra.run.smithi047.stdout: Running scriptlet: ceph-osd-2:19.0.0-2231.g2edfee68.el9.x86_64 134/198 2024-03-21T10:09:09.566 INFO:teuthology.orchestra.run.smithi047.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-21T10:09:09.567 INFO:teuthology.orchestra.run.smithi047.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-03-21T10:09:09.567 INFO:teuthology.orchestra.run.smithi047.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-03-21T10:09:09.567 INFO:teuthology.orchestra.run.smithi047.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-03-21T10:09:09.567 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-21T10:09:09.757 INFO:teuthology.orchestra.run.smithi157.stdout: Upgrading : openssl-1:3.0.7-27.el9.x86_64 60/198 2024-03-21T10:09:09.827 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : ceph-mds-2:19.0.0-2231.g2edfee68.el9.x86_64 135/198 2024-03-21T10:09:09.843 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 61/198 2024-03-21T10:09:09.901 INFO:teuthology.orchestra.run.smithi047.stdout: Running scriptlet: ceph-mds-2:19.0.0-2231.g2edfee68.el9.x86_64 135/198 2024-03-21T10:09:09.901 INFO:teuthology.orchestra.run.smithi047.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-21T10:09:09.901 INFO:teuthology.orchestra.run.smithi047.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-03-21T10:09:09.901 INFO:teuthology.orchestra.run.smithi047.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-03-21T10:09:09.901 INFO:teuthology.orchestra.run.smithi047.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-03-21T10:09:09.901 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-21T10:09:09.956 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 62/198 2024-03-21T10:09:09.996 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 63/198 2024-03-21T10:09:10.036 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 64/198 2024-03-21T10:09:10.104 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 65/198 2024-03-21T10:09:10.144 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 66/198 2024-03-21T10:09:10.206 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 67/198 2024-03-21T10:09:10.272 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 68/198 2024-03-21T10:09:10.403 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : ceph-mon-2:19.0.0-2231.g2edfee68.el9.x86_64 136/198 2024-03-21T10:09:10.462 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 69/198 2024-03-21T10:09:10.475 INFO:teuthology.orchestra.run.smithi047.stdout: Running scriptlet: ceph-mon-2:19.0.0-2231.g2edfee68.el9.x86_64 136/198 2024-03-21T10:09:10.475 INFO:teuthology.orchestra.run.smithi047.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-21T10:09:10.475 INFO:teuthology.orchestra.run.smithi047.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-03-21T10:09:10.475 INFO:teuthology.orchestra.run.smithi047.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-03-21T10:09:10.475 INFO:teuthology.orchestra.run.smithi047.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-03-21T10:09:10.475 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-21T10:09:10.517 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 70/198 2024-03-21T10:09:10.555 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-colorama-0.4.4-12.el9.noarch 71/198 2024-03-21T10:09:10.588 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : automake-1.16.2-8.el9.noarch 137/198 2024-03-21T10:09:10.612 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 72/198 2024-03-21T10:09:10.740 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 73/198 2024-03-21T10:09:10.769 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-google-auth-1:2.28.2-1.el9.noarch 74/198 2024-03-21T10:09:10.822 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 75/198 2024-03-21T10:09:10.860 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 76/198 2024-03-21T10:09:10.893 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : libarrow-doc-9.0.0-10.el9.noarch 77/198 2024-03-21T10:09:11.061 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 78/198 2024-03-21T10:09:11.205 INFO:teuthology.orchestra.run.smithi047.stdout: Running scriptlet: cephadm-2:19.0.0-2231.g2edfee68.el9.noarch 138/198 2024-03-21T10:09:11.232 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 79/198 2024-03-21T10:09:11.241 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : cephadm-2:19.0.0-2231.g2edfee68.el9.noarch 138/198 2024-03-21T10:09:11.292 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-2231.g2edfee68.e 139/198 2024-03-21T10:09:11.375 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 80/198 2024-03-21T10:09:11.395 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-2231.g2edfee68. 140/198 2024-03-21T10:09:11.420 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-devel-3.9.17-2.el9.x86_64 81/198 2024-03-21T10:09:11.428 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-2231.g2edfee68.el9.noa 141/198 2024-03-21T10:09:11.483 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-ply-3.11-14.el9.noarch 82/198 2024-03-21T10:09:11.561 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 83/198 2024-03-21T10:09:11.769 INFO:teuthology.orchestra.run.smithi047.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-2231.g2edfee68.el9.noa 141/198 2024-03-21T10:09:11.804 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-2231.g2edfee68.el9.n 142/198 2024-03-21T10:09:11.806 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 84/198 2024-03-21T10:09:11.866 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 85/198 2024-03-21T10:09:12.040 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 86/198 2024-03-21T10:09:12.088 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 87/198 2024-03-21T10:09:12.148 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 88/198 2024-03-21T10:09:12.192 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : fuse-2.9.9-15.el9.x86_64 89/198 2024-03-21T10:09:12.234 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : zlib-devel-1.2.11-40.el9.x86_64 90/198 2024-03-21T10:09:12.280 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 91/198 2024-03-21T10:09:12.348 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : uuid-1.6.2-55.el9.x86_64 92/198 2024-03-21T10:09:12.405 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 93/198 2024-03-21T10:09:12.455 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 94/198 2024-03-21T10:09:12.580 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 95/198 2024-03-21T10:09:12.624 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 96/198 2024-03-21T10:09:12.659 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 97/198 2024-03-21T10:09:12.711 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 98/198 2024-03-21T10:09:12.753 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 99/198 2024-03-21T10:09:12.896 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 100/198 2024-03-21T10:09:12.975 INFO:teuthology.orchestra.run.smithi047.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-2231.g2edfee68.el9.n 142/198 2024-03-21T10:09:13.004 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-2231.g2ed 143/198 2024-03-21T10:09:13.014 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 101/198 2024-03-21T10:09:13.075 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 102/198 2024-03-21T10:09:13.144 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 103/198 2024-03-21T10:09:13.153 INFO:teuthology.orchestra.run.smithi047.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-2231.g2ed 143/198 2024-03-21T10:09:13.205 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 104/198 2024-03-21T10:09:13.342 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-2231.g2edfee68.el 144/198 2024-03-21T10:09:13.355 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-packaging-20.9-5.el9.noarch 105/198 2024-03-21T10:09:13.372 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : ceph-mgr-2:19.0.0-2231.g2edfee68.el9.x86_64 145/198 2024-03-21T10:09:13.418 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 106/198 2024-03-21T10:09:13.443 INFO:teuthology.orchestra.run.smithi047.stdout: Running scriptlet: ceph-mgr-2:19.0.0-2231.g2edfee68.el9.x86_64 145/198 2024-03-21T10:09:13.443 INFO:teuthology.orchestra.run.smithi047.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-21T10:09:13.444 INFO:teuthology.orchestra.run.smithi047.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-03-21T10:09:13.444 INFO:teuthology.orchestra.run.smithi047.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-03-21T10:09:13.444 INFO:teuthology.orchestra.run.smithi047.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-03-21T10:09:13.444 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-21T10:09:13.507 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : ceph-mgr-rook-2:19.0.0-2231.g2edfee68.el9.noarch 146/198 2024-03-21T10:09:13.545 INFO:teuthology.orchestra.run.smithi047.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-2231.g2edfee68.el9.noarch 146/198 2024-03-21T10:09:13.641 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : ceph-2:19.0.0-2231.g2edfee68.el9.x86_64 147/198 2024-03-21T10:09:14.353 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 107/198 2024-03-21T10:09:14.401 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 108/198 2024-03-21T10:09:14.434 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 109/198 2024-03-21T10:09:14.489 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 110/198 2024-03-21T10:09:14.601 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 111/198 2024-03-21T10:09:14.651 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 112/198 2024-03-21T10:09:14.673 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : libtool-2.4.6-45.el9.x86_64 148/198 2024-03-21T10:09:14.709 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : ceph-radosgw-2:19.0.0-2231.g2edfee68.el9.x86_64 149/198 2024-03-21T10:09:14.781 INFO:teuthology.orchestra.run.smithi047.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-2231.g2edfee68.el9.x86_64 149/198 2024-03-21T10:09:14.781 INFO:teuthology.orchestra.run.smithi047.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-21T10:09:14.781 INFO:teuthology.orchestra.run.smithi047.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-03-21T10:09:14.781 INFO:teuthology.orchestra.run.smithi047.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-03-21T10:09:14.781 INFO:teuthology.orchestra.run.smithi047.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-03-21T10:09:14.782 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-21T10:09:14.835 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-2231.g2edfe 150/198 2024-03-21T10:09:14.849 INFO:teuthology.orchestra.run.smithi100.stdout: Running scriptlet: ceph-selinux-2:19.0.0-2231.g2edfee68.el9.x86_64 132/198 2024-03-21T10:09:14.849 INFO:teuthology.orchestra.run.smithi100.stdout:skipping the directory /sys 2024-03-21T10:09:14.850 INFO:teuthology.orchestra.run.smithi100.stdout:skipping the directory /proc 2024-03-21T10:09:14.850 INFO:teuthology.orchestra.run.smithi100.stdout:skipping the directory /mnt 2024-03-21T10:09:14.850 INFO:teuthology.orchestra.run.smithi100.stdout:skipping the directory /var/tmp 2024-03-21T10:09:14.850 INFO:teuthology.orchestra.run.smithi100.stdout:skipping the directory /home 2024-03-21T10:09:14.850 INFO:teuthology.orchestra.run.smithi100.stdout:skipping the directory /root 2024-03-21T10:09:14.850 INFO:teuthology.orchestra.run.smithi100.stdout:skipping the directory /tmp 2024-03-21T10:09:14.850 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-03-21T10:09:14.885 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : openblas-0.3.21-2.el9.x86_64 113/198 2024-03-21T10:09:14.906 INFO:teuthology.orchestra.run.smithi047.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-2231.g2edfe 150/198 2024-03-21T10:09:14.906 INFO:teuthology.orchestra.run.smithi047.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-21T10:09:14.906 INFO:teuthology.orchestra.run.smithi047.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-03-21T10:09:14.906 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-21T10:09:14.923 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 114/198 2024-03-21T10:09:14.978 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : cephfs-mirror-2:19.0.0-2231.g2edfee68.el9.x86_64 151/198 2024-03-21T10:09:15.019 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 115/198 2024-03-21T10:09:15.054 INFO:teuthology.orchestra.run.smithi047.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-2231.g2edfee68.el9.x86_64 151/198 2024-03-21T10:09:15.055 INFO:teuthology.orchestra.run.smithi047.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-21T10:09:15.055 INFO:teuthology.orchestra.run.smithi047.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-03-21T10:09:15.055 INFO:teuthology.orchestra.run.smithi047.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-03-21T10:09:15.055 INFO:teuthology.orchestra.run.smithi047.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-03-21T10:09:15.055 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-21T10:09:15.104 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : ceph-volume-2:19.0.0-2231.g2edfee68.el9.noarch 133/198 2024-03-21T10:09:15.150 INFO:teuthology.orchestra.run.smithi100.stdout: Running scriptlet: ceph-volume-2:19.0.0-2231.g2edfee68.el9.noarch 133/198 2024-03-21T10:09:15.150 INFO:teuthology.orchestra.run.smithi100.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-21T10:09:15.150 INFO:teuthology.orchestra.run.smithi100.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-03-21T10:09:15.150 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-03-21T10:09:15.428 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : rbd-mirror-2:19.0.0-2231.g2edfee68.el9.x86_64 152/198 2024-03-21T10:09:15.502 INFO:teuthology.orchestra.run.smithi047.stdout: Running scriptlet: rbd-mirror-2:19.0.0-2231.g2edfee68.el9.x86_64 152/198 2024-03-21T10:09:15.502 INFO:teuthology.orchestra.run.smithi047.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-21T10:09:15.502 INFO:teuthology.orchestra.run.smithi047.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-03-21T10:09:15.502 INFO:teuthology.orchestra.run.smithi047.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-03-21T10:09:15.502 INFO:teuthology.orchestra.run.smithi047.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-03-21T10:09:15.502 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-21T10:09:15.866 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 116/198 2024-03-21T10:09:16.001 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 117/198 2024-03-21T10:09:16.791 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : ceph-osd-2:19.0.0-2231.g2edfee68.el9.x86_64 134/198 2024-03-21T10:09:16.890 INFO:teuthology.orchestra.run.smithi100.stdout: Running scriptlet: ceph-osd-2:19.0.0-2231.g2edfee68.el9.x86_64 134/198 2024-03-21T10:09:16.890 INFO:teuthology.orchestra.run.smithi100.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-21T10:09:16.891 INFO:teuthology.orchestra.run.smithi100.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-03-21T10:09:16.891 INFO:teuthology.orchestra.run.smithi100.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-03-21T10:09:16.891 INFO:teuthology.orchestra.run.smithi100.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-03-21T10:09:16.891 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-03-21T10:09:17.150 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : ceph-mds-2:19.0.0-2231.g2edfee68.el9.x86_64 135/198 2024-03-21T10:09:17.218 INFO:teuthology.orchestra.run.smithi100.stdout: Running scriptlet: ceph-mds-2:19.0.0-2231.g2edfee68.el9.x86_64 135/198 2024-03-21T10:09:17.218 INFO:teuthology.orchestra.run.smithi100.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-21T10:09:17.218 INFO:teuthology.orchestra.run.smithi100.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-03-21T10:09:17.218 INFO:teuthology.orchestra.run.smithi100.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-03-21T10:09:17.218 INFO:teuthology.orchestra.run.smithi100.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-03-21T10:09:17.218 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-03-21T10:09:17.544 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 118/198 2024-03-21T10:09:17.635 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 119/198 2024-03-21T10:09:17.673 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : oniguruma-6.9.6-1.el9.5.x86_64 120/198 2024-03-21T10:09:17.700 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : ceph-mon-2:19.0.0-2231.g2edfee68.el9.x86_64 136/198 2024-03-21T10:09:17.704 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : jq-1.6-16.el9.x86_64 121/198 2024-03-21T10:09:17.745 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 122/198 2024-03-21T10:09:17.769 INFO:teuthology.orchestra.run.smithi100.stdout: Running scriptlet: ceph-mon-2:19.0.0-2231.g2edfee68.el9.x86_64 136/198 2024-03-21T10:09:17.770 INFO:teuthology.orchestra.run.smithi100.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-21T10:09:17.770 INFO:teuthology.orchestra.run.smithi100.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-03-21T10:09:17.770 INFO:teuthology.orchestra.run.smithi100.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-03-21T10:09:17.770 INFO:teuthology.orchestra.run.smithi100.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-03-21T10:09:17.770 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-03-21T10:09:17.886 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : automake-1.16.2-8.el9.noarch 137/198 2024-03-21T10:09:18.120 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 123/198 2024-03-21T10:09:18.151 INFO:teuthology.orchestra.run.smithi157.stdout: Upgrading : librbd1-2:19.0.0-2231.g2edfee68.el9.x86_64 124/198 2024-03-21T10:09:18.270 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: librbd1-2:19.0.0-2231.g2edfee68.el9.x86_64 124/198 2024-03-21T10:09:18.298 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-rbd-2:19.0.0-2231.g2edfee68.el9.x86_64 125/198 2024-03-21T10:09:18.351 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 126/198 2024-03-21T10:09:18.490 INFO:teuthology.orchestra.run.smithi100.stdout: Running scriptlet: cephadm-2:19.0.0-2231.g2edfee68.el9.noarch 138/198 2024-03-21T10:09:18.529 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : cephadm-2:19.0.0-2231.g2edfee68.el9.noarch 138/198 2024-03-21T10:09:18.580 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-2231.g2edfee68.e 139/198 2024-03-21T10:09:18.680 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-2231.g2edfee68. 140/198 2024-03-21T10:09:18.707 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-2231.g2edfee68.el9.noa 141/198 2024-03-21T10:09:18.861 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : parquet-libs-9.0.0-10.el9.x86_64 127/198 2024-03-21T10:09:18.904 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : librgw2-2:19.0.0-2231.g2edfee68.el9.x86_64 128/198 2024-03-21T10:09:19.011 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: librgw2-2:19.0.0-2231.g2edfee68.el9.x86_64 128/198 2024-03-21T10:09:19.035 INFO:teuthology.orchestra.run.smithi100.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-2231.g2edfee68.el9.noa 141/198 2024-03-21T10:09:19.039 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-rgw-2:19.0.0-2231.g2edfee68.el9.x86_64 129/198 2024-03-21T10:09:19.067 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-2231.g2edfee68.el9.n 142/198 2024-03-21T10:09:20.193 INFO:teuthology.orchestra.run.smithi100.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-2231.g2edfee68.el9.n 142/198 2024-03-21T10:09:20.225 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-2231.g2ed 143/198 2024-03-21T10:09:20.367 INFO:teuthology.orchestra.run.smithi100.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-2231.g2ed 143/198 2024-03-21T10:09:20.539 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-2231.g2edfee68.el 144/198 2024-03-21T10:09:20.569 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : ceph-mgr-2:19.0.0-2231.g2edfee68.el9.x86_64 145/198 2024-03-21T10:09:20.593 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : ceph-test-2:19.0.0-2231.g2edfee68.el9.x86_64 153/198 2024-03-21T10:09:20.637 INFO:teuthology.orchestra.run.smithi100.stdout: Running scriptlet: ceph-mgr-2:19.0.0-2231.g2edfee68.el9.x86_64 145/198 2024-03-21T10:09:20.638 INFO:teuthology.orchestra.run.smithi100.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-21T10:09:20.638 INFO:teuthology.orchestra.run.smithi100.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-03-21T10:09:20.638 INFO:teuthology.orchestra.run.smithi100.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-03-21T10:09:20.638 INFO:teuthology.orchestra.run.smithi100.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-03-21T10:09:20.638 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-03-21T10:09:20.646 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : rbd-fuse-2:19.0.0-2231.g2edfee68.el9.x86_64 154/198 2024-03-21T10:09:20.682 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : rbd-nbd-2:19.0.0-2231.g2edfee68.el9.x86_64 155/198 2024-03-21T10:09:20.695 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : ceph-mgr-rook-2:19.0.0-2231.g2edfee68.el9.noarch 146/198 2024-03-21T10:09:20.713 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : elfutils-libelf-devel-0.190-2.el9.x86_64 156/198 2024-03-21T10:09:20.731 INFO:teuthology.orchestra.run.smithi100.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-2231.g2edfee68.el9.noarch 146/198 2024-03-21T10:09:20.788 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : ceph-2:19.0.0-2231.g2edfee68.el9.x86_64 147/198 2024-03-21T10:09:21.001 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : cephfs-shell-2:19.0.0-2231.g2edfee68.el9.x86_64 157/198 2024-03-21T10:09:21.133 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : postgresql-contrib-13.11-1.el9.x86_64 158/198 2024-03-21T10:09:21.185 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : ceph-fuse-2:19.0.0-2231.g2edfee68.el9.x86_64 159/198 2024-03-21T10:09:21.244 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 160/198 2024-03-21T10:09:21.317 INFO:teuthology.orchestra.run.smithi047.stdout: Upgrading : xfsdump-3.1.12-4.el9.x86_64 161/198 2024-03-21T10:09:21.362 INFO:teuthology.orchestra.run.smithi047.stdout: Upgrading : util-linux-user-2.37.4-18.el9.x86_64 162/198 2024-03-21T10:09:21.397 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : libcephfs-devel-2:19.0.0-2231.g2edfee68.el9.x86_ 163/198 2024-03-21T10:09:21.442 INFO:teuthology.orchestra.run.smithi047.stdout: Upgrading : elfutils-debuginfod-client-0.190-2.el9.x86_64 164/198 2024-03-21T10:09:21.494 INFO:teuthology.orchestra.run.smithi047.stdout: Upgrading : NetworkManager-team-1:1.47.2-1.el9.x86_64 165/198 2024-03-21T10:09:21.588 INFO:teuthology.orchestra.run.smithi047.stdout: Upgrading : NetworkManager-tui-1:1.47.2-1.el9.x86_64 166/198 2024-03-21T10:09:21.641 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: ceph-common-2:19.0.0-2231.g2edfee68.el9.x86_64 130/198 2024-03-21T10:09:21.685 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : ceph-common-2:19.0.0-2231.g2edfee68.el9.x86_64 130/198 2024-03-21T10:09:21.720 INFO:teuthology.orchestra.run.smithi047.stdout: Upgrading : acl-2.3.1-4.el9.x86_64 167/198 2024-03-21T10:09:21.754 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : libtool-2.4.6-45.el9.x86_64 148/198 2024-03-21T10:09:21.758 INFO:teuthology.orchestra.run.smithi047.stdout: Upgrading : libacl-devel-2.3.1-4.el9.x86_64 168/198 2024-03-21T10:09:21.788 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : ceph-radosgw-2:19.0.0-2231.g2edfee68.el9.x86_64 149/198 2024-03-21T10:09:21.874 INFO:teuthology.orchestra.run.smithi100.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-2231.g2edfee68.el9.x86_64 149/198 2024-03-21T10:09:21.874 INFO:teuthology.orchestra.run.smithi100.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-21T10:09:21.874 INFO:teuthology.orchestra.run.smithi100.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-03-21T10:09:21.874 INFO:teuthology.orchestra.run.smithi100.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-03-21T10:09:21.874 INFO:teuthology.orchestra.run.smithi100.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-03-21T10:09:21.874 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-03-21T10:09:21.923 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-2231.g2edfe 150/198 2024-03-21T10:09:22.016 INFO:teuthology.orchestra.run.smithi100.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-2231.g2edfe 150/198 2024-03-21T10:09:22.017 INFO:teuthology.orchestra.run.smithi100.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-21T10:09:22.017 INFO:teuthology.orchestra.run.smithi100.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-03-21T10:09:22.017 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-03-21T10:09:22.098 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : cephfs-mirror-2:19.0.0-2231.g2edfee68.el9.x86_64 151/198 2024-03-21T10:09:22.200 INFO:teuthology.orchestra.run.smithi100.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-2231.g2edfee68.el9.x86_64 151/198 2024-03-21T10:09:22.200 INFO:teuthology.orchestra.run.smithi100.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-21T10:09:22.201 INFO:teuthology.orchestra.run.smithi100.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-03-21T10:09:22.201 INFO:teuthology.orchestra.run.smithi100.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-03-21T10:09:22.201 INFO:teuthology.orchestra.run.smithi100.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-03-21T10:09:22.201 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-03-21T10:09:22.310 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: ceph-common-2:19.0.0-2231.g2edfee68.el9.x86_64 130/198 2024-03-21T10:09:22.364 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : ceph-base-2:19.0.0-2231.g2edfee68.el9.x86_64 131/198 2024-03-21T10:09:22.544 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: ceph-base-2:19.0.0-2231.g2edfee68.el9.x86_64 131/198 2024-03-21T10:09:22.545 INFO:teuthology.orchestra.run.smithi157.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-03-21T10:09:22.545 INFO:teuthology.orchestra.run.smithi157.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-03-21T10:09:22.545 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-03-21T10:09:22.589 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : ceph-selinux-2:19.0.0-2231.g2edfee68.el9.x86_64 132/198 2024-03-21T10:09:22.590 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : rbd-mirror-2:19.0.0-2231.g2edfee68.el9.x86_64 152/198 2024-03-21T10:09:22.673 INFO:teuthology.orchestra.run.smithi100.stdout: Running scriptlet: rbd-mirror-2:19.0.0-2231.g2edfee68.el9.x86_64 152/198 2024-03-21T10:09:22.673 INFO:teuthology.orchestra.run.smithi100.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-21T10:09:22.673 INFO:teuthology.orchestra.run.smithi100.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-03-21T10:09:22.674 INFO:teuthology.orchestra.run.smithi100.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-03-21T10:09:22.674 INFO:teuthology.orchestra.run.smithi100.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-03-21T10:09:22.674 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-03-21T10:09:22.698 INFO:teuthology.orchestra.run.smithi047.stdout: Running scriptlet: postgresql-server-13.11-1.el9.x86_64 169/198 2024-03-21T10:09:22.739 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : postgresql-server-13.11-1.el9.x86_64 169/198 2024-03-21T10:09:22.797 INFO:teuthology.orchestra.run.smithi047.stdout: Running scriptlet: postgresql-server-13.11-1.el9.x86_64 169/198 2024-03-21T10:09:24.197 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : cephfs-top-2:19.0.0-2231.g2edfee68.el9.noarch 170/198 2024-03-21T10:09:24.294 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : openssl-devel-1:3.0.7-27.el9.x86_64 171/198 2024-03-21T10:09:24.352 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 172/198 2024-03-21T10:09:24.487 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : flex-2.6.4-9.el9.x86_64 173/198 2024-03-21T10:09:24.522 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : bison-3.7.4-5.el9.x86_64 174/198 2024-03-21T10:09:24.555 INFO:teuthology.orchestra.run.smithi047.stdout: Cleanup : NetworkManager-tui-1:1.44.0-2.el9.x86_64 175/198 2024-03-21T10:09:24.589 INFO:teuthology.orchestra.run.smithi047.stdout: Cleanup : elfutils-debuginfod-client-0.189-3.el9.x86_64 176/198 2024-03-21T10:09:24.631 INFO:teuthology.orchestra.run.smithi047.stdout: Cleanup : elfutils-libs-0.189-3.el9.x86_64 177/198 2024-03-21T10:09:24.631 INFO:teuthology.orchestra.run.smithi047.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 178/198 2024-03-21T10:09:24.777 INFO:teuthology.orchestra.run.smithi047.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 178/198 2024-03-21T10:09:24.852 INFO:teuthology.orchestra.run.smithi047.stdout: Cleanup : openssl-1:3.0.7-24.el9.x86_64 179/198 2024-03-21T10:09:24.852 INFO:teuthology.orchestra.run.smithi047.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 180/198 2024-03-21T10:09:25.023 INFO:teuthology.orchestra.run.smithi047.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 180/198 2024-03-21T10:09:25.073 INFO:teuthology.orchestra.run.smithi047.stdout: Cleanup : xfsdump-3.1.12-3.el9.x86_64 181/198 2024-03-21T10:09:25.122 INFO:teuthology.orchestra.run.smithi047.stdout: Cleanup : xfsprogs-5.19.0-4.el9.x86_64 182/198 2024-03-21T10:09:25.163 INFO:teuthology.orchestra.run.smithi047.stdout: Cleanup : acl-2.3.1-3.el9.x86_64 183/198 2024-03-21T10:09:25.196 INFO:teuthology.orchestra.run.smithi047.stdout: Cleanup : libacl-devel-2.3.1-3.el9.x86_64 184/198 2024-03-21T10:09:25.275 INFO:teuthology.orchestra.run.smithi047.stdout: Cleanup : util-linux-user-2.37.4-15.el9.x86_64 185/198 2024-03-21T10:09:25.353 INFO:teuthology.orchestra.run.smithi047.stdout: Cleanup : util-linux-2.37.4-15.el9.x86_64 186/198 2024-03-21T10:09:25.401 INFO:teuthology.orchestra.run.smithi047.stdout: Cleanup : util-linux-core-2.37.4-15.el9.x86_64 187/198 2024-03-21T10:09:25.434 INFO:teuthology.orchestra.run.smithi047.stdout: Cleanup : libfdisk-2.37.4-15.el9.x86_64 188/198 2024-03-21T10:09:25.492 INFO:teuthology.orchestra.run.smithi047.stdout: Cleanup : libmount-2.37.4-15.el9.x86_64 189/198 2024-03-21T10:09:25.526 INFO:teuthology.orchestra.run.smithi047.stdout: Cleanup : libblkid-2.37.4-15.el9.x86_64 190/198 2024-03-21T10:09:25.559 INFO:teuthology.orchestra.run.smithi047.stdout: Cleanup : NetworkManager-team-1:1.44.0-2.el9.x86_64 191/198 2024-03-21T10:09:25.596 INFO:teuthology.orchestra.run.smithi047.stdout: Running scriptlet: NetworkManager-1:1.44.0-2.el9.x86_64 192/198 2024-03-21T10:09:25.597 INFO:teuthology.orchestra.run.smithi047.stdout: Cleanup : NetworkManager-1:1.44.0-2.el9.x86_64 192/198 2024-03-21T10:09:25.918 INFO:teuthology.orchestra.run.smithi047.stdout: Running scriptlet: NetworkManager-1:1.44.0-2.el9.x86_64 192/198 2024-03-21T10:09:25.919 INFO:teuthology.orchestra.run.smithi047.stdout: Cleanup : NetworkManager-libnm-1:1.44.0-2.el9.x86_64 193/198 2024-03-21T10:09:26.066 INFO:teuthology.orchestra.run.smithi047.stdout: Running scriptlet: NetworkManager-libnm-1:1.44.0-2.el9.x86_64 193/198 2024-03-21T10:09:26.129 INFO:teuthology.orchestra.run.smithi047.stdout: Cleanup : libuuid-2.37.4-15.el9.x86_64 194/198 2024-03-21T10:09:26.183 INFO:teuthology.orchestra.run.smithi047.stdout: Cleanup : libsmartcols-2.37.4-15.el9.x86_64 195/198 2024-03-21T10:09:26.215 INFO:teuthology.orchestra.run.smithi047.stdout: Cleanup : libacl-2.3.1-3.el9.x86_64 196/198 2024-03-21T10:09:26.255 INFO:teuthology.orchestra.run.smithi047.stdout: Cleanup : openssl-libs-1:3.0.7-24.el9.x86_64 197/198 2024-03-21T10:09:26.378 INFO:teuthology.orchestra.run.smithi047.stdout: Cleanup : elfutils-libelf-0.189-3.el9.x86_64 198/198 2024-03-21T10:09:26.635 INFO:teuthology.orchestra.run.smithi047.stdout: Running scriptlet: elfutils-libelf-0.189-3.el9.x86_64 198/198 2024-03-21T10:09:26.635 INFO:teuthology.orchestra.run.smithi047.stdout:Couldn't write '0' to 'kernel/yama/ptrace_scope', ignoring: No such file or directory 2024-03-21T10:09:26.635 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-21T10:09:27.471 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : ceph-test-2:19.0.0-2231.g2edfee68.el9.x86_64 153/198 2024-03-21T10:09:27.516 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : rbd-fuse-2:19.0.0-2231.g2edfee68.el9.x86_64 154/198 2024-03-21T10:09:27.552 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : rbd-nbd-2:19.0.0-2231.g2edfee68.el9.x86_64 155/198 2024-03-21T10:09:27.590 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : elfutils-libelf-devel-0.190-2.el9.x86_64 156/198 2024-03-21T10:09:27.864 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : cephfs-shell-2:19.0.0-2231.g2edfee68.el9.x86_64 157/198 2024-03-21T10:09:27.981 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : postgresql-contrib-13.11-1.el9.x86_64 158/198 2024-03-21T10:09:28.030 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : ceph-fuse-2:19.0.0-2231.g2edfee68.el9.x86_64 159/198 2024-03-21T10:09:28.084 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 160/198 2024-03-21T10:09:28.146 INFO:teuthology.orchestra.run.smithi100.stdout: Upgrading : xfsdump-3.1.12-4.el9.x86_64 161/198 2024-03-21T10:09:28.199 INFO:teuthology.orchestra.run.smithi100.stdout: Upgrading : util-linux-user-2.37.4-18.el9.x86_64 162/198 2024-03-21T10:09:28.250 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : libcephfs-devel-2:19.0.0-2231.g2edfee68.el9.x86_ 163/198 2024-03-21T10:09:28.296 INFO:teuthology.orchestra.run.smithi100.stdout: Upgrading : elfutils-debuginfod-client-0.190-2.el9.x86_64 164/198 2024-03-21T10:09:28.348 INFO:teuthology.orchestra.run.smithi100.stdout: Upgrading : NetworkManager-team-1:1.47.2-1.el9.x86_64 165/198 2024-03-21T10:09:28.438 INFO:teuthology.orchestra.run.smithi100.stdout: Upgrading : NetworkManager-tui-1:1.47.2-1.el9.x86_64 166/198 2024-03-21T10:09:28.540 INFO:teuthology.orchestra.run.smithi100.stdout: Upgrading : acl-2.3.1-4.el9.x86_64 167/198 2024-03-21T10:09:28.580 INFO:teuthology.orchestra.run.smithi100.stdout: Upgrading : libacl-devel-2.3.1-4.el9.x86_64 168/198 2024-03-21T10:09:29.452 INFO:teuthology.orchestra.run.smithi100.stdout: Running scriptlet: postgresql-server-13.11-1.el9.x86_64 169/198 2024-03-21T10:09:29.519 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : postgresql-server-13.11-1.el9.x86_64 169/198 2024-03-21T10:09:29.576 INFO:teuthology.orchestra.run.smithi100.stdout: Running scriptlet: postgresql-server-13.11-1.el9.x86_64 169/198 2024-03-21T10:09:30.894 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : cephfs-top-2:19.0.0-2231.g2edfee68.el9.noarch 170/198 2024-03-21T10:09:30.993 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : openssl-devel-1:3.0.7-27.el9.x86_64 171/198 2024-03-21T10:09:31.033 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 172/198 2024-03-21T10:09:31.171 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : flex-2.6.4-9.el9.x86_64 173/198 2024-03-21T10:09:31.216 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : bison-3.7.4-5.el9.x86_64 174/198 2024-03-21T10:09:31.245 INFO:teuthology.orchestra.run.smithi100.stdout: Cleanup : NetworkManager-tui-1:1.44.0-2.el9.x86_64 175/198 2024-03-21T10:09:31.279 INFO:teuthology.orchestra.run.smithi100.stdout: Cleanup : elfutils-debuginfod-client-0.189-3.el9.x86_64 176/198 2024-03-21T10:09:31.320 INFO:teuthology.orchestra.run.smithi100.stdout: Cleanup : elfutils-libs-0.189-3.el9.x86_64 177/198 2024-03-21T10:09:31.320 INFO:teuthology.orchestra.run.smithi100.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 178/198 2024-03-21T10:09:31.330 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : ceph-2:19.0.0-2231.g2edfee68.el9.x86_64 1/198 2024-03-21T10:09:31.330 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : ceph-base-2:19.0.0-2231.g2edfee68.el9.x86_64 2/198 2024-03-21T10:09:31.330 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : ceph-common-2:19.0.0-2231.g2edfee68.el9.x86_64 3/198 2024-03-21T10:09:31.330 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : ceph-fuse-2:19.0.0-2231.g2edfee68.el9.x86_64 4/198 2024-03-21T10:09:31.331 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-2231.g2edfe 5/198 2024-03-21T10:09:31.331 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : ceph-mds-2:19.0.0-2231.g2edfee68.el9.x86_64 6/198 2024-03-21T10:09:31.331 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : ceph-mgr-2:19.0.0-2231.g2edfee68.el9.x86_64 7/198 2024-03-21T10:09:31.331 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : ceph-mon-2:19.0.0-2231.g2edfee68.el9.x86_64 8/198 2024-03-21T10:09:31.331 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : ceph-osd-2:19.0.0-2231.g2edfee68.el9.x86_64 9/198 2024-03-21T10:09:31.331 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : ceph-radosgw-2:19.0.0-2231.g2edfee68.el9.x86_64 10/198 2024-03-21T10:09:31.331 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : ceph-selinux-2:19.0.0-2231.g2edfee68.el9.x86_64 11/198 2024-03-21T10:09:31.331 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : ceph-test-2:19.0.0-2231.g2edfee68.el9.x86_64 12/198 2024-03-21T10:09:31.331 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : cephfs-mirror-2:19.0.0-2231.g2edfee68.el9.x86_64 13/198 2024-03-21T10:09:31.331 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : cephfs-shell-2:19.0.0-2231.g2edfee68.el9.x86_64 14/198 2024-03-21T10:09:31.331 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : libcephfs-devel-2:19.0.0-2231.g2edfee68.el9.x86_ 15/198 2024-03-21T10:09:31.331 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : libcephfs2-2:19.0.0-2231.g2edfee68.el9.x86_64 16/198 2024-03-21T10:09:31.332 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : libcephsqlite-2:19.0.0-2231.g2edfee68.el9.x86_64 17/198 2024-03-21T10:09:31.332 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : librados-devel-2:19.0.0-2231.g2edfee68.el9.x86_6 18/198 2024-03-21T10:09:31.332 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : libradosstriper1-2:19.0.0-2231.g2edfee68.el9.x86 19/198 2024-03-21T10:09:31.332 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : librgw2-2:19.0.0-2231.g2edfee68.el9.x86_64 20/198 2024-03-21T10:09:31.332 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-ceph-argparse-2:19.0.0-2231.g2edfee68.el 21/198 2024-03-21T10:09:31.332 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-ceph-common-2:19.0.0-2231.g2edfee68.el9. 22/198 2024-03-21T10:09:31.332 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-cephfs-2:19.0.0-2231.g2edfee68.el9.x86_6 23/198 2024-03-21T10:09:31.332 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-rados-2:19.0.0-2231.g2edfee68.el9.x86_64 24/198 2024-03-21T10:09:31.332 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-rbd-2:19.0.0-2231.g2edfee68.el9.x86_64 25/198 2024-03-21T10:09:31.332 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-rgw-2:19.0.0-2231.g2edfee68.el9.x86_64 26/198 2024-03-21T10:09:31.332 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : rbd-fuse-2:19.0.0-2231.g2edfee68.el9.x86_64 27/198 2024-03-21T10:09:31.332 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : rbd-mirror-2:19.0.0-2231.g2edfee68.el9.x86_64 28/198 2024-03-21T10:09:31.332 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : rbd-nbd-2:19.0.0-2231.g2edfee68.el9.x86_64 29/198 2024-03-21T10:09:31.333 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-2231.g2edfee68. 30/198 2024-03-21T10:09:31.333 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-2231.g2edfee68.el9.noa 31/198 2024-03-21T10:09:31.347 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-2231.g2edfee68.el9.n 32/198 2024-03-21T10:09:31.348 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-2231.g2ed 33/198 2024-03-21T10:09:31.348 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-2231.g2edfee68.el 34/198 2024-03-21T10:09:31.348 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : ceph-mgr-rook-2:19.0.0-2231.g2edfee68.el9.noarch 35/198 2024-03-21T10:09:31.348 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-2231.g2edfee68.e 36/198 2024-03-21T10:09:31.348 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : ceph-volume-2:19.0.0-2231.g2edfee68.el9.noarch 37/198 2024-03-21T10:09:31.348 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : cephadm-2:19.0.0-2231.g2edfee68.el9.noarch 38/198 2024-03-21T10:09:31.348 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : cephfs-top-2:19.0.0-2231.g2edfee68.el9.noarch 39/198 2024-03-21T10:09:31.348 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : automake-1.16.2-8.el9.noarch 40/198 2024-03-21T10:09:31.348 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : bison-3.7.4-5.el9.x86_64 41/198 2024-03-21T10:09:31.348 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 42/198 2024-03-21T10:09:31.348 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : flex-2.6.4-9.el9.x86_64 43/198 2024-03-21T10:09:31.348 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 44/198 2024-03-21T10:09:31.348 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 45/198 2024-03-21T10:09:31.348 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 46/198 2024-03-21T10:09:31.348 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 47/198 2024-03-21T10:09:31.349 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 48/198 2024-03-21T10:09:31.349 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 49/198 2024-03-21T10:09:31.349 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 50/198 2024-03-21T10:09:31.349 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 51/198 2024-03-21T10:09:31.349 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 52/198 2024-03-21T10:09:31.349 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 53/198 2024-03-21T10:09:31.349 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : lua-5.4.4-4.el9.x86_64 54/198 2024-03-21T10:09:31.349 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : oniguruma-6.9.6-1.el9.5.x86_64 55/198 2024-03-21T10:09:31.349 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 56/198 2024-03-21T10:09:31.349 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 57/198 2024-03-21T10:09:31.349 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : postgresql-13.11-1.el9.x86_64 58/198 2024-03-21T10:09:31.349 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : postgresql-contrib-13.11-1.el9.x86_64 59/198 2024-03-21T10:09:31.349 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : postgresql-private-libs-13.11-1.el9.x86_64 60/198 2024-03-21T10:09:31.350 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : postgresql-server-13.11-1.el9.x86_64 61/198 2024-03-21T10:09:31.350 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 62/198 2024-03-21T10:09:31.350 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 63/198 2024-03-21T10:09:31.350 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 64/198 2024-03-21T10:09:31.350 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 65/198 2024-03-21T10:09:31.350 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 66/198 2024-03-21T10:09:31.350 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 67/198 2024-03-21T10:09:31.350 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 68/198 2024-03-21T10:09:31.350 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 69/198 2024-03-21T10:09:31.350 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 70/198 2024-03-21T10:09:31.350 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 71/198 2024-03-21T10:09:31.350 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 72/198 2024-03-21T10:09:31.350 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 73/198 2024-03-21T10:09:31.350 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 74/198 2024-03-21T10:09:31.351 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 75/198 2024-03-21T10:09:31.351 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 76/198 2024-03-21T10:09:31.351 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 77/198 2024-03-21T10:09:31.351 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 78/198 2024-03-21T10:09:31.351 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 79/198 2024-03-21T10:09:31.351 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 80/198 2024-03-21T10:09:31.351 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 81/198 2024-03-21T10:09:31.351 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 82/198 2024-03-21T10:09:31.351 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 83/198 2024-03-21T10:09:31.351 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 84/198 2024-03-21T10:09:31.351 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : zlib-devel-1.2.11-40.el9.x86_64 85/198 2024-03-21T10:09:31.351 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 86/198 2024-03-21T10:09:31.351 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 87/198 2024-03-21T10:09:31.351 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 88/198 2024-03-21T10:09:31.351 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-ply-3.11-14.el9.noarch 89/198 2024-03-21T10:09:31.352 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 90/198 2024-03-21T10:09:31.352 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 91/198 2024-03-21T10:09:31.352 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : jq-1.6-16.el9.x86_64 92/198 2024-03-21T10:09:31.352 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 93/198 2024-03-21T10:09:31.352 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 94/198 2024-03-21T10:09:31.352 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 95/198 2024-03-21T10:09:31.352 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : elfutils-libelf-devel-0.190-2.el9.x86_64 96/198 2024-03-21T10:09:31.352 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 97/198 2024-03-21T10:09:31.352 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : openssl-devel-1:3.0.7-27.el9.x86_64 98/198 2024-03-21T10:09:31.352 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-devel-3.9.17-2.el9.x86_64 99/198 2024-03-21T10:09:31.352 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 100/198 2024-03-21T10:09:31.352 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : qatlib-23.11.0-1.el9.x86_64 101/198 2024-03-21T10:09:31.352 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : qatlib-service-23.11.0-1.el9.x86_64 102/198 2024-03-21T10:09:31.352 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : qatzip-libs-1.1.2-2.el9.x86_64 103/198 2024-03-21T10:09:31.352 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 104/198 2024-03-21T10:09:31.353 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 105/198 2024-03-21T10:09:31.353 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 106/198 2024-03-21T10:09:31.353 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 107/198 2024-03-21T10:09:31.353 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 108/198 2024-03-21T10:09:31.353 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 109/198 2024-03-21T10:09:31.353 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : libarrow-9.0.0-10.el9.x86_64 110/198 2024-03-21T10:09:31.353 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : libarrow-doc-9.0.0-10.el9.noarch 111/198 2024-03-21T10:09:31.353 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 112/198 2024-03-21T10:09:31.353 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 113/198 2024-03-21T10:09:31.353 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 114/198 2024-03-21T10:09:31.353 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : parquet-libs-9.0.0-10.el9.x86_64 115/198 2024-03-21T10:09:31.353 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 116/198 2024-03-21T10:09:31.353 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 117/198 2024-03-21T10:09:31.353 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 118/198 2024-03-21T10:09:31.353 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 119/198 2024-03-21T10:09:31.353 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 120/198 2024-03-21T10:09:31.353 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 121/198 2024-03-21T10:09:31.354 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 122/198 2024-03-21T10:09:31.354 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-colorama-0.4.4-12.el9.noarch 123/198 2024-03-21T10:09:31.354 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-google-auth-1:2.28.2-1.el9.noarch 124/198 2024-03-21T10:09:31.354 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 125/198 2024-03-21T10:09:31.354 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 126/198 2024-03-21T10:09:31.354 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 127/198 2024-03-21T10:09:31.354 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 128/198 2024-03-21T10:09:31.354 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 129/198 2024-03-21T10:09:31.354 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 130/198 2024-03-21T10:09:31.354 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 131/198 2024-03-21T10:09:31.354 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 132/198 2024-03-21T10:09:31.354 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 133/198 2024-03-21T10:09:31.354 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 134/198 2024-03-21T10:09:31.354 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 135/198 2024-03-21T10:09:31.355 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 136/198 2024-03-21T10:09:31.355 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 137/198 2024-03-21T10:09:31.355 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 138/198 2024-03-21T10:09:31.355 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 139/198 2024-03-21T10:09:31.355 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 140/198 2024-03-21T10:09:31.355 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 141/198 2024-03-21T10:09:31.355 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 142/198 2024-03-21T10:09:31.355 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 143/198 2024-03-21T10:09:31.355 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 144/198 2024-03-21T10:09:31.355 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 145/198 2024-03-21T10:09:31.355 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 146/198 2024-03-21T10:09:31.355 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 147/198 2024-03-21T10:09:31.355 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 148/198 2024-03-21T10:09:31.355 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : re2-1:20211101-3.el9.x86_64 149/198 2024-03-21T10:09:31.355 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 150/198 2024-03-21T10:09:31.355 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : librados2-2:19.0.0-2231.g2edfee68.el9.x86_64 151/198 2024-03-21T10:09:31.356 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 152/198 2024-03-21T10:09:31.356 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : librbd1-2:19.0.0-2231.g2edfee68.el9.x86_64 153/198 2024-03-21T10:09:31.356 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 154/198 2024-03-21T10:09:31.356 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : xfsdump-3.1.12-4.el9.x86_64 155/198 2024-03-21T10:09:31.356 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : xfsdump-3.1.12-3.el9.x86_64 156/198 2024-03-21T10:09:31.356 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : NetworkManager-1:1.47.2-1.el9.x86_64 157/198 2024-03-21T10:09:31.356 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : NetworkManager-1:1.44.0-2.el9.x86_64 158/198 2024-03-21T10:09:31.356 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : NetworkManager-libnm-1:1.47.2-1.el9.x86_64 159/198 2024-03-21T10:09:31.356 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : NetworkManager-libnm-1:1.44.0-2.el9.x86_64 160/198 2024-03-21T10:09:31.356 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : NetworkManager-team-1:1.47.2-1.el9.x86_64 161/198 2024-03-21T10:09:31.356 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : NetworkManager-team-1:1.44.0-2.el9.x86_64 162/198 2024-03-21T10:09:31.356 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : NetworkManager-tui-1:1.47.2-1.el9.x86_64 163/198 2024-03-21T10:09:31.356 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : NetworkManager-tui-1:1.44.0-2.el9.x86_64 164/198 2024-03-21T10:09:31.356 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : acl-2.3.1-4.el9.x86_64 165/198 2024-03-21T10:09:31.356 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : acl-2.3.1-3.el9.x86_64 166/198 2024-03-21T10:09:31.357 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : elfutils-debuginfod-client-0.190-2.el9.x86_64 167/198 2024-03-21T10:09:31.357 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : elfutils-debuginfod-client-0.189-3.el9.x86_64 168/198 2024-03-21T10:09:31.357 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : elfutils-libelf-0.190-2.el9.x86_64 169/198 2024-03-21T10:09:31.357 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : elfutils-libelf-0.189-3.el9.x86_64 170/198 2024-03-21T10:09:31.357 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : elfutils-libs-0.190-2.el9.x86_64 171/198 2024-03-21T10:09:31.357 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : elfutils-libs-0.189-3.el9.x86_64 172/198 2024-03-21T10:09:31.357 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : libacl-2.3.1-4.el9.x86_64 173/198 2024-03-21T10:09:31.357 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : libacl-2.3.1-3.el9.x86_64 174/198 2024-03-21T10:09:31.357 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : libblkid-2.37.4-18.el9.x86_64 175/198 2024-03-21T10:09:31.357 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : libblkid-2.37.4-15.el9.x86_64 176/198 2024-03-21T10:09:31.357 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : libfdisk-2.37.4-18.el9.x86_64 177/198 2024-03-21T10:09:31.357 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : libfdisk-2.37.4-15.el9.x86_64 178/198 2024-03-21T10:09:31.357 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : libmount-2.37.4-18.el9.x86_64 179/198 2024-03-21T10:09:31.357 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : libmount-2.37.4-15.el9.x86_64 180/198 2024-03-21T10:09:31.357 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : libsmartcols-2.37.4-18.el9.x86_64 181/198 2024-03-21T10:09:31.358 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : libsmartcols-2.37.4-15.el9.x86_64 182/198 2024-03-21T10:09:31.358 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : libuuid-2.37.4-18.el9.x86_64 183/198 2024-03-21T10:09:31.358 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : libuuid-2.37.4-15.el9.x86_64 184/198 2024-03-21T10:09:31.358 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 185/198 2024-03-21T10:09:31.358 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : openssl-1:3.0.7-24.el9.x86_64 186/198 2024-03-21T10:09:31.358 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 187/198 2024-03-21T10:09:31.358 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : openssl-libs-1:3.0.7-24.el9.x86_64 188/198 2024-03-21T10:09:31.358 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : util-linux-2.37.4-18.el9.x86_64 189/198 2024-03-21T10:09:31.358 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : util-linux-2.37.4-15.el9.x86_64 190/198 2024-03-21T10:09:31.358 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : util-linux-core-2.37.4-18.el9.x86_64 191/198 2024-03-21T10:09:31.359 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : util-linux-core-2.37.4-15.el9.x86_64 192/198 2024-03-21T10:09:31.359 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : util-linux-user-2.37.4-18.el9.x86_64 193/198 2024-03-21T10:09:31.359 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : util-linux-user-2.37.4-15.el9.x86_64 194/198 2024-03-21T10:09:31.359 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : xfsprogs-6.3.0-1.el9.x86_64 195/198 2024-03-21T10:09:31.359 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : xfsprogs-5.19.0-4.el9.x86_64 196/198 2024-03-21T10:09:31.359 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : libacl-devel-2.3.1-4.el9.x86_64 197/198 2024-03-21T10:09:31.458 INFO:teuthology.orchestra.run.smithi100.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 178/198 2024-03-21T10:09:31.501 INFO:teuthology.orchestra.run.smithi100.stdout: Cleanup : openssl-1:3.0.7-24.el9.x86_64 179/198 2024-03-21T10:09:31.501 INFO:teuthology.orchestra.run.smithi100.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 180/198 2024-03-21T10:09:31.630 INFO:teuthology.orchestra.run.smithi100.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 180/198 2024-03-21T10:09:31.691 INFO:teuthology.orchestra.run.smithi100.stdout: Cleanup : xfsdump-3.1.12-3.el9.x86_64 181/198 2024-03-21T10:09:31.746 INFO:teuthology.orchestra.run.smithi100.stdout: Cleanup : xfsprogs-5.19.0-4.el9.x86_64 182/198 2024-03-21T10:09:31.787 INFO:teuthology.orchestra.run.smithi100.stdout: Cleanup : acl-2.3.1-3.el9.x86_64 183/198 2024-03-21T10:09:31.819 INFO:teuthology.orchestra.run.smithi100.stdout: Cleanup : libacl-devel-2.3.1-3.el9.x86_64 184/198 2024-03-21T10:09:31.886 INFO:teuthology.orchestra.run.smithi100.stdout: Cleanup : util-linux-user-2.37.4-15.el9.x86_64 185/198 2024-03-21T10:09:31.958 INFO:teuthology.orchestra.run.smithi100.stdout: Cleanup : util-linux-2.37.4-15.el9.x86_64 186/198 2024-03-21T10:09:32.024 INFO:teuthology.orchestra.run.smithi100.stdout: Cleanup : util-linux-core-2.37.4-15.el9.x86_64 187/198 2024-03-21T10:09:32.058 INFO:teuthology.orchestra.run.smithi100.stdout: Cleanup : libfdisk-2.37.4-15.el9.x86_64 188/198 2024-03-21T10:09:32.091 INFO:teuthology.orchestra.run.smithi100.stdout: Cleanup : libmount-2.37.4-15.el9.x86_64 189/198 2024-03-21T10:09:32.125 INFO:teuthology.orchestra.run.smithi100.stdout: Cleanup : libblkid-2.37.4-15.el9.x86_64 190/198 2024-03-21T10:09:32.158 INFO:teuthology.orchestra.run.smithi100.stdout: Cleanup : NetworkManager-team-1:1.44.0-2.el9.x86_64 191/198 2024-03-21T10:09:32.195 INFO:teuthology.orchestra.run.smithi100.stdout: Running scriptlet: NetworkManager-1:1.44.0-2.el9.x86_64 192/198 2024-03-21T10:09:32.195 INFO:teuthology.orchestra.run.smithi100.stdout: Cleanup : NetworkManager-1:1.44.0-2.el9.x86_64 192/198 2024-03-21T10:09:32.516 INFO:teuthology.orchestra.run.smithi100.stdout: Running scriptlet: NetworkManager-1:1.44.0-2.el9.x86_64 192/198 2024-03-21T10:09:32.517 INFO:teuthology.orchestra.run.smithi100.stdout: Cleanup : NetworkManager-libnm-1:1.44.0-2.el9.x86_64 193/198 2024-03-21T10:09:32.648 INFO:teuthology.orchestra.run.smithi100.stdout: Running scriptlet: NetworkManager-libnm-1:1.44.0-2.el9.x86_64 193/198 2024-03-21T10:09:32.678 INFO:teuthology.orchestra.run.smithi100.stdout: Cleanup : libuuid-2.37.4-15.el9.x86_64 194/198 2024-03-21T10:09:32.733 INFO:teuthology.orchestra.run.smithi100.stdout: Cleanup : libsmartcols-2.37.4-15.el9.x86_64 195/198 2024-03-21T10:09:32.789 INFO:teuthology.orchestra.run.smithi100.stdout: Cleanup : libacl-2.3.1-3.el9.x86_64 196/198 2024-03-21T10:09:32.829 INFO:teuthology.orchestra.run.smithi100.stdout: Cleanup : openssl-libs-1:3.0.7-24.el9.x86_64 197/198 2024-03-21T10:09:32.949 INFO:teuthology.orchestra.run.smithi100.stdout: Cleanup : elfutils-libelf-0.189-3.el9.x86_64 198/198 2024-03-21T10:09:33.190 INFO:teuthology.orchestra.run.smithi100.stdout: Running scriptlet: elfutils-libelf-0.189-3.el9.x86_64 198/198 2024-03-21T10:09:33.191 INFO:teuthology.orchestra.run.smithi100.stdout:Couldn't write '0' to 'kernel/yama/ptrace_scope', ignoring: No such file or directory 2024-03-21T10:09:33.191 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-03-21T10:09:33.807 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : libacl-devel-2.3.1-3.el9.x86_64 198/198 2024-03-21T10:09:33.807 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-21T10:09:33.807 INFO:teuthology.orchestra.run.smithi047.stdout:Upgraded: 2024-03-21T10:09:33.807 INFO:teuthology.orchestra.run.smithi047.stdout: NetworkManager-1:1.47.2-1.el9.x86_64 2024-03-21T10:09:33.807 INFO:teuthology.orchestra.run.smithi047.stdout: NetworkManager-libnm-1:1.47.2-1.el9.x86_64 2024-03-21T10:09:33.807 INFO:teuthology.orchestra.run.smithi047.stdout: NetworkManager-team-1:1.47.2-1.el9.x86_64 2024-03-21T10:09:33.808 INFO:teuthology.orchestra.run.smithi047.stdout: NetworkManager-tui-1:1.47.2-1.el9.x86_64 2024-03-21T10:09:33.808 INFO:teuthology.orchestra.run.smithi047.stdout: acl-2.3.1-4.el9.x86_64 2024-03-21T10:09:33.808 INFO:teuthology.orchestra.run.smithi047.stdout: elfutils-debuginfod-client-0.190-2.el9.x86_64 2024-03-21T10:09:33.808 INFO:teuthology.orchestra.run.smithi047.stdout: elfutils-libelf-0.190-2.el9.x86_64 2024-03-21T10:09:33.808 INFO:teuthology.orchestra.run.smithi047.stdout: elfutils-libs-0.190-2.el9.x86_64 2024-03-21T10:09:33.808 INFO:teuthology.orchestra.run.smithi047.stdout: libacl-2.3.1-4.el9.x86_64 2024-03-21T10:09:33.808 INFO:teuthology.orchestra.run.smithi047.stdout: libacl-devel-2.3.1-4.el9.x86_64 2024-03-21T10:09:33.808 INFO:teuthology.orchestra.run.smithi047.stdout: libblkid-2.37.4-18.el9.x86_64 2024-03-21T10:09:33.808 INFO:teuthology.orchestra.run.smithi047.stdout: libfdisk-2.37.4-18.el9.x86_64 2024-03-21T10:09:33.808 INFO:teuthology.orchestra.run.smithi047.stdout: libmount-2.37.4-18.el9.x86_64 2024-03-21T10:09:33.808 INFO:teuthology.orchestra.run.smithi047.stdout: librados2-2:19.0.0-2231.g2edfee68.el9.x86_64 2024-03-21T10:09:33.808 INFO:teuthology.orchestra.run.smithi047.stdout: librbd1-2:19.0.0-2231.g2edfee68.el9.x86_64 2024-03-21T10:09:33.809 INFO:teuthology.orchestra.run.smithi047.stdout: libsmartcols-2.37.4-18.el9.x86_64 2024-03-21T10:09:33.809 INFO:teuthology.orchestra.run.smithi047.stdout: libuuid-2.37.4-18.el9.x86_64 2024-03-21T10:09:33.809 INFO:teuthology.orchestra.run.smithi047.stdout: openssl-1:3.0.7-27.el9.x86_64 2024-03-21T10:09:33.809 INFO:teuthology.orchestra.run.smithi047.stdout: openssl-libs-1:3.0.7-27.el9.x86_64 2024-03-21T10:09:33.809 INFO:teuthology.orchestra.run.smithi047.stdout: util-linux-2.37.4-18.el9.x86_64 2024-03-21T10:09:33.809 INFO:teuthology.orchestra.run.smithi047.stdout: util-linux-core-2.37.4-18.el9.x86_64 2024-03-21T10:09:33.809 INFO:teuthology.orchestra.run.smithi047.stdout: util-linux-user-2.37.4-18.el9.x86_64 2024-03-21T10:09:33.809 INFO:teuthology.orchestra.run.smithi047.stdout: xfsdump-3.1.12-4.el9.x86_64 2024-03-21T10:09:33.809 INFO:teuthology.orchestra.run.smithi047.stdout: xfsprogs-6.3.0-1.el9.x86_64 2024-03-21T10:09:33.809 INFO:teuthology.orchestra.run.smithi047.stdout:Installed: 2024-03-21T10:09:33.809 INFO:teuthology.orchestra.run.smithi047.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-03-21T10:09:33.809 INFO:teuthology.orchestra.run.smithi047.stdout: automake-1.16.2-8.el9.noarch 2024-03-21T10:09:33.809 INFO:teuthology.orchestra.run.smithi047.stdout: bison-3.7.4-5.el9.x86_64 2024-03-21T10:09:33.809 INFO:teuthology.orchestra.run.smithi047.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-03-21T10:09:33.809 INFO:teuthology.orchestra.run.smithi047.stdout: ceph-2:19.0.0-2231.g2edfee68.el9.x86_64 2024-03-21T10:09:33.810 INFO:teuthology.orchestra.run.smithi047.stdout: ceph-base-2:19.0.0-2231.g2edfee68.el9.x86_64 2024-03-21T10:09:33.810 INFO:teuthology.orchestra.run.smithi047.stdout: ceph-common-2:19.0.0-2231.g2edfee68.el9.x86_64 2024-03-21T10:09:33.810 INFO:teuthology.orchestra.run.smithi047.stdout: ceph-fuse-2:19.0.0-2231.g2edfee68.el9.x86_64 2024-03-21T10:09:33.810 INFO:teuthology.orchestra.run.smithi047.stdout: ceph-grafana-dashboards-2:19.0.0-2231.g2edfee68.el9.noarch 2024-03-21T10:09:33.810 INFO:teuthology.orchestra.run.smithi047.stdout: ceph-immutable-object-cache-2:19.0.0-2231.g2edfee68.el9.x86_64 2024-03-21T10:09:33.810 INFO:teuthology.orchestra.run.smithi047.stdout: ceph-mds-2:19.0.0-2231.g2edfee68.el9.x86_64 2024-03-21T10:09:33.810 INFO:teuthology.orchestra.run.smithi047.stdout: ceph-mgr-2:19.0.0-2231.g2edfee68.el9.x86_64 2024-03-21T10:09:33.810 INFO:teuthology.orchestra.run.smithi047.stdout: ceph-mgr-cephadm-2:19.0.0-2231.g2edfee68.el9.noarch 2024-03-21T10:09:33.810 INFO:teuthology.orchestra.run.smithi047.stdout: ceph-mgr-dashboard-2:19.0.0-2231.g2edfee68.el9.noarch 2024-03-21T10:09:33.810 INFO:teuthology.orchestra.run.smithi047.stdout: ceph-mgr-diskprediction-local-2:19.0.0-2231.g2edfee68.el9.noarch 2024-03-21T10:09:33.810 INFO:teuthology.orchestra.run.smithi047.stdout: ceph-mgr-modules-core-2:19.0.0-2231.g2edfee68.el9.noarch 2024-03-21T10:09:33.810 INFO:teuthology.orchestra.run.smithi047.stdout: ceph-mgr-rook-2:19.0.0-2231.g2edfee68.el9.noarch 2024-03-21T10:09:33.810 INFO:teuthology.orchestra.run.smithi047.stdout: ceph-mon-2:19.0.0-2231.g2edfee68.el9.x86_64 2024-03-21T10:09:33.811 INFO:teuthology.orchestra.run.smithi047.stdout: ceph-osd-2:19.0.0-2231.g2edfee68.el9.x86_64 2024-03-21T10:09:33.811 INFO:teuthology.orchestra.run.smithi047.stdout: ceph-prometheus-alerts-2:19.0.0-2231.g2edfee68.el9.noarch 2024-03-21T10:09:33.811 INFO:teuthology.orchestra.run.smithi047.stdout: ceph-radosgw-2:19.0.0-2231.g2edfee68.el9.x86_64 2024-03-21T10:09:33.811 INFO:teuthology.orchestra.run.smithi047.stdout: ceph-selinux-2:19.0.0-2231.g2edfee68.el9.x86_64 2024-03-21T10:09:33.811 INFO:teuthology.orchestra.run.smithi047.stdout: ceph-test-2:19.0.0-2231.g2edfee68.el9.x86_64 2024-03-21T10:09:33.811 INFO:teuthology.orchestra.run.smithi047.stdout: ceph-volume-2:19.0.0-2231.g2edfee68.el9.noarch 2024-03-21T10:09:33.811 INFO:teuthology.orchestra.run.smithi047.stdout: cephadm-2:19.0.0-2231.g2edfee68.el9.noarch 2024-03-21T10:09:33.811 INFO:teuthology.orchestra.run.smithi047.stdout: cephfs-mirror-2:19.0.0-2231.g2edfee68.el9.x86_64 2024-03-21T10:09:33.811 INFO:teuthology.orchestra.run.smithi047.stdout: cephfs-shell-2:19.0.0-2231.g2edfee68.el9.x86_64 2024-03-21T10:09:33.811 INFO:teuthology.orchestra.run.smithi047.stdout: cephfs-top-2:19.0.0-2231.g2edfee68.el9.noarch 2024-03-21T10:09:33.811 INFO:teuthology.orchestra.run.smithi047.stdout: elfutils-libelf-devel-0.190-2.el9.x86_64 2024-03-21T10:09:33.811 INFO:teuthology.orchestra.run.smithi047.stdout: flex-2.6.4-9.el9.x86_64 2024-03-21T10:09:33.811 INFO:teuthology.orchestra.run.smithi047.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-03-21T10:09:33.811 INFO:teuthology.orchestra.run.smithi047.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-03-21T10:09:33.812 INFO:teuthology.orchestra.run.smithi047.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-03-21T10:09:33.812 INFO:teuthology.orchestra.run.smithi047.stdout: fuse-2.9.9-15.el9.x86_64 2024-03-21T10:09:33.812 INFO:teuthology.orchestra.run.smithi047.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-03-21T10:09:33.812 INFO:teuthology.orchestra.run.smithi047.stdout: grpc-data-1.46.7-10.el9.noarch 2024-03-21T10:09:33.812 INFO:teuthology.orchestra.run.smithi047.stdout: jq-1.6-16.el9.x86_64 2024-03-21T10:09:33.812 INFO:teuthology.orchestra.run.smithi047.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-03-21T10:09:33.812 INFO:teuthology.orchestra.run.smithi047.stdout: libarrow-9.0.0-10.el9.x86_64 2024-03-21T10:09:33.812 INFO:teuthology.orchestra.run.smithi047.stdout: libarrow-doc-9.0.0-10.el9.noarch 2024-03-21T10:09:33.812 INFO:teuthology.orchestra.run.smithi047.stdout: libcephfs-devel-2:19.0.0-2231.g2edfee68.el9.x86_64 2024-03-21T10:09:33.812 INFO:teuthology.orchestra.run.smithi047.stdout: libcephfs2-2:19.0.0-2231.g2edfee68.el9.x86_64 2024-03-21T10:09:33.812 INFO:teuthology.orchestra.run.smithi047.stdout: libcephsqlite-2:19.0.0-2231.g2edfee68.el9.x86_64 2024-03-21T10:09:33.812 INFO:teuthology.orchestra.run.smithi047.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-03-21T10:09:33.813 INFO:teuthology.orchestra.run.smithi047.stdout: liboath-2.6.7-2.el9.x86_64 2024-03-21T10:09:33.813 INFO:teuthology.orchestra.run.smithi047.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-03-21T10:09:33.813 INFO:teuthology.orchestra.run.smithi047.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-03-21T10:09:33.813 INFO:teuthology.orchestra.run.smithi047.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-03-21T10:09:33.813 INFO:teuthology.orchestra.run.smithi047.stdout: librados-devel-2:19.0.0-2231.g2edfee68.el9.x86_64 2024-03-21T10:09:33.813 INFO:teuthology.orchestra.run.smithi047.stdout: libradosstriper1-2:19.0.0-2231.g2edfee68.el9.x86_64 2024-03-21T10:09:33.813 INFO:teuthology.orchestra.run.smithi047.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-03-21T10:09:33.813 INFO:teuthology.orchestra.run.smithi047.stdout: librgw2-2:19.0.0-2231.g2edfee68.el9.x86_64 2024-03-21T10:09:33.813 INFO:teuthology.orchestra.run.smithi047.stdout: libtool-2.4.6-45.el9.x86_64 2024-03-21T10:09:33.813 INFO:teuthology.orchestra.run.smithi047.stdout: libunwind-1.6.2-1.el9.x86_64 2024-03-21T10:09:33.813 INFO:teuthology.orchestra.run.smithi047.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-03-21T10:09:33.813 INFO:teuthology.orchestra.run.smithi047.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-03-21T10:09:33.813 INFO:teuthology.orchestra.run.smithi047.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-03-21T10:09:33.813 INFO:teuthology.orchestra.run.smithi047.stdout: lua-5.4.4-4.el9.x86_64 2024-03-21T10:09:33.813 INFO:teuthology.orchestra.run.smithi047.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-03-21T10:09:33.814 INFO:teuthology.orchestra.run.smithi047.stdout: luarocks-3.9.2-1.el9.noarch 2024-03-21T10:09:33.814 INFO:teuthology.orchestra.run.smithi047.stdout: oniguruma-6.9.6-1.el9.5.x86_64 2024-03-21T10:09:33.814 INFO:teuthology.orchestra.run.smithi047.stdout: openblas-0.3.21-2.el9.x86_64 2024-03-21T10:09:33.814 INFO:teuthology.orchestra.run.smithi047.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-03-21T10:09:33.814 INFO:teuthology.orchestra.run.smithi047.stdout: openssl-devel-1:3.0.7-27.el9.x86_64 2024-03-21T10:09:33.814 INFO:teuthology.orchestra.run.smithi047.stdout: parquet-libs-9.0.0-10.el9.x86_64 2024-03-21T10:09:33.814 INFO:teuthology.orchestra.run.smithi047.stdout: postgresql-13.11-1.el9.x86_64 2024-03-21T10:09:33.814 INFO:teuthology.orchestra.run.smithi047.stdout: postgresql-contrib-13.11-1.el9.x86_64 2024-03-21T10:09:33.814 INFO:teuthology.orchestra.run.smithi047.stdout: postgresql-private-libs-13.11-1.el9.x86_64 2024-03-21T10:09:33.814 INFO:teuthology.orchestra.run.smithi047.stdout: postgresql-server-13.11-1.el9.x86_64 2024-03-21T10:09:33.814 INFO:teuthology.orchestra.run.smithi047.stdout: protobuf-3.14.0-13.el9.x86_64 2024-03-21T10:09:33.814 INFO:teuthology.orchestra.run.smithi047.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-03-21T10:09:33.814 INFO:teuthology.orchestra.run.smithi047.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-03-21T10:09:33.814 INFO:teuthology.orchestra.run.smithi047.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-03-21T10:09:33.814 INFO:teuthology.orchestra.run.smithi047.stdout: python3-babel-2.9.1-2.el9.noarch 2024-03-21T10:09:33.815 INFO:teuthology.orchestra.run.smithi047.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-03-21T10:09:33.815 INFO:teuthology.orchestra.run.smithi047.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-03-21T10:09:33.815 INFO:teuthology.orchestra.run.smithi047.stdout: python3-ceph-argparse-2:19.0.0-2231.g2edfee68.el9.x86_64 2024-03-21T10:09:33.815 INFO:teuthology.orchestra.run.smithi047.stdout: python3-ceph-common-2:19.0.0-2231.g2edfee68.el9.x86_64 2024-03-21T10:09:33.815 INFO:teuthology.orchestra.run.smithi047.stdout: python3-cephfs-2:19.0.0-2231.g2edfee68.el9.x86_64 2024-03-21T10:09:33.815 INFO:teuthology.orchestra.run.smithi047.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-03-21T10:09:33.815 INFO:teuthology.orchestra.run.smithi047.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-03-21T10:09:33.815 INFO:teuthology.orchestra.run.smithi047.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-03-21T10:09:33.815 INFO:teuthology.orchestra.run.smithi047.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-03-21T10:09:33.815 INFO:teuthology.orchestra.run.smithi047.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-03-21T10:09:33.815 INFO:teuthology.orchestra.run.smithi047.stdout: python3-colorama-0.4.4-12.el9.noarch 2024-03-21T10:09:33.815 INFO:teuthology.orchestra.run.smithi047.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-03-21T10:09:33.815 INFO:teuthology.orchestra.run.smithi047.stdout: python3-devel-3.9.17-2.el9.x86_64 2024-03-21T10:09:33.815 INFO:teuthology.orchestra.run.smithi047.stdout: python3-google-auth-1:2.28.2-1.el9.noarch 2024-03-21T10:09:33.815 INFO:teuthology.orchestra.run.smithi047.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-03-21T10:09:33.816 INFO:teuthology.orchestra.run.smithi047.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-03-21T10:09:33.816 INFO:teuthology.orchestra.run.smithi047.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-03-21T10:09:33.816 INFO:teuthology.orchestra.run.smithi047.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-03-21T10:09:33.816 INFO:teuthology.orchestra.run.smithi047.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-03-21T10:09:33.816 INFO:teuthology.orchestra.run.smithi047.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-03-21T10:09:33.816 INFO:teuthology.orchestra.run.smithi047.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-03-21T10:09:33.816 INFO:teuthology.orchestra.run.smithi047.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-03-21T10:09:33.816 INFO:teuthology.orchestra.run.smithi047.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-03-21T10:09:33.816 INFO:teuthology.orchestra.run.smithi047.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-03-21T10:09:33.816 INFO:teuthology.orchestra.run.smithi047.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-03-21T10:09:33.816 INFO:teuthology.orchestra.run.smithi047.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-03-21T10:09:33.816 INFO:teuthology.orchestra.run.smithi047.stdout: python3-mako-1.1.4-6.el9.noarch 2024-03-21T10:09:33.816 INFO:teuthology.orchestra.run.smithi047.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-03-21T10:09:33.816 INFO:teuthology.orchestra.run.smithi047.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-03-21T10:09:33.816 INFO:teuthology.orchestra.run.smithi047.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-03-21T10:09:33.817 INFO:teuthology.orchestra.run.smithi047.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-03-21T10:09:33.817 INFO:teuthology.orchestra.run.smithi047.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-03-21T10:09:33.817 INFO:teuthology.orchestra.run.smithi047.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-03-21T10:09:33.817 INFO:teuthology.orchestra.run.smithi047.stdout: python3-packaging-20.9-5.el9.noarch 2024-03-21T10:09:33.817 INFO:teuthology.orchestra.run.smithi047.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-03-21T10:09:33.817 INFO:teuthology.orchestra.run.smithi047.stdout: python3-ply-3.11-14.el9.noarch 2024-03-21T10:09:33.817 INFO:teuthology.orchestra.run.smithi047.stdout: python3-portend-3.1.0-2.el9.noarch 2024-03-21T10:09:33.817 INFO:teuthology.orchestra.run.smithi047.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-03-21T10:09:33.817 INFO:teuthology.orchestra.run.smithi047.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-03-21T10:09:33.817 INFO:teuthology.orchestra.run.smithi047.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-03-21T10:09:33.817 INFO:teuthology.orchestra.run.smithi047.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-03-21T10:09:33.817 INFO:teuthology.orchestra.run.smithi047.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-03-21T10:09:33.817 INFO:teuthology.orchestra.run.smithi047.stdout: python3-pycparser-2.20-6.el9.noarch 2024-03-21T10:09:33.817 INFO:teuthology.orchestra.run.smithi047.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-03-21T10:09:33.818 INFO:teuthology.orchestra.run.smithi047.stdout: python3-pytz-2021.1-5.el9.noarch 2024-03-21T10:09:33.818 INFO:teuthology.orchestra.run.smithi047.stdout: python3-rados-2:19.0.0-2231.g2edfee68.el9.x86_64 2024-03-21T10:09:33.818 INFO:teuthology.orchestra.run.smithi047.stdout: python3-rbd-2:19.0.0-2231.g2edfee68.el9.x86_64 2024-03-21T10:09:33.818 INFO:teuthology.orchestra.run.smithi047.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-03-21T10:09:33.818 INFO:teuthology.orchestra.run.smithi047.stdout: python3-requests-2.25.1-8.el9.noarch 2024-03-21T10:09:33.818 INFO:teuthology.orchestra.run.smithi047.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-03-21T10:09:33.818 INFO:teuthology.orchestra.run.smithi047.stdout: python3-rgw-2:19.0.0-2231.g2edfee68.el9.x86_64 2024-03-21T10:09:33.818 INFO:teuthology.orchestra.run.smithi047.stdout: python3-routes-2.5.1-5.el9.noarch 2024-03-21T10:09:33.818 INFO:teuthology.orchestra.run.smithi047.stdout: python3-rsa-4.9-2.el9.noarch 2024-03-21T10:09:33.818 INFO:teuthology.orchestra.run.smithi047.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-03-21T10:09:33.818 INFO:teuthology.orchestra.run.smithi047.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-03-21T10:09:33.819 INFO:teuthology.orchestra.run.smithi047.stdout: python3-toml-0.10.2-6.el9.noarch 2024-03-21T10:09:33.819 INFO:teuthology.orchestra.run.smithi047.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-03-21T10:09:33.819 INFO:teuthology.orchestra.run.smithi047.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-03-21T10:09:33.819 INFO:teuthology.orchestra.run.smithi047.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-03-21T10:09:33.819 INFO:teuthology.orchestra.run.smithi047.stdout: python3-webob-1.8.7-6.el9.noarch 2024-03-21T10:09:33.819 INFO:teuthology.orchestra.run.smithi047.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-03-21T10:09:33.819 INFO:teuthology.orchestra.run.smithi047.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-03-21T10:09:33.819 INFO:teuthology.orchestra.run.smithi047.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-03-21T10:09:33.819 INFO:teuthology.orchestra.run.smithi047.stdout: qatlib-23.11.0-1.el9.x86_64 2024-03-21T10:09:33.819 INFO:teuthology.orchestra.run.smithi047.stdout: qatlib-service-23.11.0-1.el9.x86_64 2024-03-21T10:09:33.819 INFO:teuthology.orchestra.run.smithi047.stdout: qatzip-libs-1.1.2-2.el9.x86_64 2024-03-21T10:09:33.819 INFO:teuthology.orchestra.run.smithi047.stdout: rbd-fuse-2:19.0.0-2231.g2edfee68.el9.x86_64 2024-03-21T10:09:33.819 INFO:teuthology.orchestra.run.smithi047.stdout: rbd-mirror-2:19.0.0-2231.g2edfee68.el9.x86_64 2024-03-21T10:09:33.819 INFO:teuthology.orchestra.run.smithi047.stdout: rbd-nbd-2:19.0.0-2231.g2edfee68.el9.x86_64 2024-03-21T10:09:33.820 INFO:teuthology.orchestra.run.smithi047.stdout: re2-1:20211101-3.el9.x86_64 2024-03-21T10:09:33.820 INFO:teuthology.orchestra.run.smithi047.stdout: socat-1.7.4.1-5.el9.x86_64 2024-03-21T10:09:33.820 INFO:teuthology.orchestra.run.smithi047.stdout: thrift-0.15.0-2.el9.x86_64 2024-03-21T10:09:33.820 INFO:teuthology.orchestra.run.smithi047.stdout: uuid-1.6.2-55.el9.x86_64 2024-03-21T10:09:33.820 INFO:teuthology.orchestra.run.smithi047.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-03-21T10:09:33.820 INFO:teuthology.orchestra.run.smithi047.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-03-21T10:09:33.820 INFO:teuthology.orchestra.run.smithi047.stdout: zlib-devel-1.2.11-40.el9.x86_64 2024-03-21T10:09:33.820 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-21T10:09:33.820 INFO:teuthology.orchestra.run.smithi047.stdout:Complete! 2024-03-21T10:09:34.557 DEBUG:teuthology.parallel:result is None 2024-03-21T10:09:35.608 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: ceph-selinux-2:19.0.0-2231.g2edfee68.el9.x86_64 132/198 2024-03-21T10:09:35.608 INFO:teuthology.orchestra.run.smithi157.stdout:skipping the directory /sys 2024-03-21T10:09:35.608 INFO:teuthology.orchestra.run.smithi157.stdout:skipping the directory /proc 2024-03-21T10:09:35.609 INFO:teuthology.orchestra.run.smithi157.stdout:skipping the directory /mnt 2024-03-21T10:09:35.609 INFO:teuthology.orchestra.run.smithi157.stdout:skipping the directory /var/tmp 2024-03-21T10:09:35.609 INFO:teuthology.orchestra.run.smithi157.stdout:skipping the directory /home 2024-03-21T10:09:35.609 INFO:teuthology.orchestra.run.smithi157.stdout:skipping the directory /root 2024-03-21T10:09:35.609 INFO:teuthology.orchestra.run.smithi157.stdout:skipping the directory /tmp 2024-03-21T10:09:35.609 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-03-21T10:09:35.828 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : ceph-volume-2:19.0.0-2231.g2edfee68.el9.noarch 133/198 2024-03-21T10:09:35.877 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: ceph-volume-2:19.0.0-2231.g2edfee68.el9.noarch 133/198 2024-03-21T10:09:35.878 INFO:teuthology.orchestra.run.smithi157.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-21T10:09:35.878 INFO:teuthology.orchestra.run.smithi157.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-03-21T10:09:35.878 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-03-21T10:09:37.472 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : ceph-osd-2:19.0.0-2231.g2edfee68.el9.x86_64 134/198 2024-03-21T10:09:37.555 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: ceph-osd-2:19.0.0-2231.g2edfee68.el9.x86_64 134/198 2024-03-21T10:09:37.555 INFO:teuthology.orchestra.run.smithi157.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-21T10:09:37.555 INFO:teuthology.orchestra.run.smithi157.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-03-21T10:09:37.555 INFO:teuthology.orchestra.run.smithi157.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-03-21T10:09:37.555 INFO:teuthology.orchestra.run.smithi157.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-03-21T10:09:37.555 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-03-21T10:09:37.823 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : ceph-mds-2:19.0.0-2231.g2edfee68.el9.x86_64 135/198 2024-03-21T10:09:37.907 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: ceph-mds-2:19.0.0-2231.g2edfee68.el9.x86_64 135/198 2024-03-21T10:09:37.907 INFO:teuthology.orchestra.run.smithi157.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-21T10:09:37.908 INFO:teuthology.orchestra.run.smithi157.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-03-21T10:09:37.908 INFO:teuthology.orchestra.run.smithi157.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-03-21T10:09:37.908 INFO:teuthology.orchestra.run.smithi157.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-03-21T10:09:37.908 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-03-21T10:09:38.046 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : ceph-2:19.0.0-2231.g2edfee68.el9.x86_64 1/198 2024-03-21T10:09:38.047 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : ceph-base-2:19.0.0-2231.g2edfee68.el9.x86_64 2/198 2024-03-21T10:09:38.047 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : ceph-common-2:19.0.0-2231.g2edfee68.el9.x86_64 3/198 2024-03-21T10:09:38.047 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : ceph-fuse-2:19.0.0-2231.g2edfee68.el9.x86_64 4/198 2024-03-21T10:09:38.047 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-2231.g2edfe 5/198 2024-03-21T10:09:38.047 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : ceph-mds-2:19.0.0-2231.g2edfee68.el9.x86_64 6/198 2024-03-21T10:09:38.047 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : ceph-mgr-2:19.0.0-2231.g2edfee68.el9.x86_64 7/198 2024-03-21T10:09:38.047 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : ceph-mon-2:19.0.0-2231.g2edfee68.el9.x86_64 8/198 2024-03-21T10:09:38.047 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : ceph-osd-2:19.0.0-2231.g2edfee68.el9.x86_64 9/198 2024-03-21T10:09:38.047 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : ceph-radosgw-2:19.0.0-2231.g2edfee68.el9.x86_64 10/198 2024-03-21T10:09:38.047 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : ceph-selinux-2:19.0.0-2231.g2edfee68.el9.x86_64 11/198 2024-03-21T10:09:38.047 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : ceph-test-2:19.0.0-2231.g2edfee68.el9.x86_64 12/198 2024-03-21T10:09:38.048 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : cephfs-mirror-2:19.0.0-2231.g2edfee68.el9.x86_64 13/198 2024-03-21T10:09:38.048 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : cephfs-shell-2:19.0.0-2231.g2edfee68.el9.x86_64 14/198 2024-03-21T10:09:38.048 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : libcephfs-devel-2:19.0.0-2231.g2edfee68.el9.x86_ 15/198 2024-03-21T10:09:38.048 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : libcephfs2-2:19.0.0-2231.g2edfee68.el9.x86_64 16/198 2024-03-21T10:09:38.048 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : libcephsqlite-2:19.0.0-2231.g2edfee68.el9.x86_64 17/198 2024-03-21T10:09:38.048 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : librados-devel-2:19.0.0-2231.g2edfee68.el9.x86_6 18/198 2024-03-21T10:09:38.048 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : libradosstriper1-2:19.0.0-2231.g2edfee68.el9.x86 19/198 2024-03-21T10:09:38.048 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : librgw2-2:19.0.0-2231.g2edfee68.el9.x86_64 20/198 2024-03-21T10:09:38.048 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-ceph-argparse-2:19.0.0-2231.g2edfee68.el 21/198 2024-03-21T10:09:38.048 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-ceph-common-2:19.0.0-2231.g2edfee68.el9. 22/198 2024-03-21T10:09:38.048 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-cephfs-2:19.0.0-2231.g2edfee68.el9.x86_6 23/198 2024-03-21T10:09:38.049 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-rados-2:19.0.0-2231.g2edfee68.el9.x86_64 24/198 2024-03-21T10:09:38.049 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-rbd-2:19.0.0-2231.g2edfee68.el9.x86_64 25/198 2024-03-21T10:09:38.049 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-rgw-2:19.0.0-2231.g2edfee68.el9.x86_64 26/198 2024-03-21T10:09:38.049 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : rbd-fuse-2:19.0.0-2231.g2edfee68.el9.x86_64 27/198 2024-03-21T10:09:38.049 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : rbd-mirror-2:19.0.0-2231.g2edfee68.el9.x86_64 28/198 2024-03-21T10:09:38.049 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : rbd-nbd-2:19.0.0-2231.g2edfee68.el9.x86_64 29/198 2024-03-21T10:09:38.049 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-2231.g2edfee68. 30/198 2024-03-21T10:09:38.049 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-2231.g2edfee68.el9.noa 31/198 2024-03-21T10:09:38.049 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-2231.g2edfee68.el9.n 32/198 2024-03-21T10:09:38.049 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-2231.g2ed 33/198 2024-03-21T10:09:38.049 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-2231.g2edfee68.el 34/198 2024-03-21T10:09:38.050 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : ceph-mgr-rook-2:19.0.0-2231.g2edfee68.el9.noarch 35/198 2024-03-21T10:09:38.050 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-2231.g2edfee68.e 36/198 2024-03-21T10:09:38.050 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : ceph-volume-2:19.0.0-2231.g2edfee68.el9.noarch 37/198 2024-03-21T10:09:38.050 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : cephadm-2:19.0.0-2231.g2edfee68.el9.noarch 38/198 2024-03-21T10:09:38.050 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : cephfs-top-2:19.0.0-2231.g2edfee68.el9.noarch 39/198 2024-03-21T10:09:38.050 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : automake-1.16.2-8.el9.noarch 40/198 2024-03-21T10:09:38.050 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : bison-3.7.4-5.el9.x86_64 41/198 2024-03-21T10:09:38.050 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 42/198 2024-03-21T10:09:38.050 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : flex-2.6.4-9.el9.x86_64 43/198 2024-03-21T10:09:38.050 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 44/198 2024-03-21T10:09:38.050 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 45/198 2024-03-21T10:09:38.051 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 46/198 2024-03-21T10:09:38.051 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 47/198 2024-03-21T10:09:38.051 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 48/198 2024-03-21T10:09:38.051 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 49/198 2024-03-21T10:09:38.051 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 50/198 2024-03-21T10:09:38.051 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 51/198 2024-03-21T10:09:38.051 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 52/198 2024-03-21T10:09:38.051 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 53/198 2024-03-21T10:09:38.051 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : lua-5.4.4-4.el9.x86_64 54/198 2024-03-21T10:09:38.051 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : oniguruma-6.9.6-1.el9.5.x86_64 55/198 2024-03-21T10:09:38.051 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 56/198 2024-03-21T10:09:38.051 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 57/198 2024-03-21T10:09:38.052 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : postgresql-13.11-1.el9.x86_64 58/198 2024-03-21T10:09:38.052 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : postgresql-contrib-13.11-1.el9.x86_64 59/198 2024-03-21T10:09:38.052 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : postgresql-private-libs-13.11-1.el9.x86_64 60/198 2024-03-21T10:09:38.052 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : postgresql-server-13.11-1.el9.x86_64 61/198 2024-03-21T10:09:38.052 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 62/198 2024-03-21T10:09:38.052 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 63/198 2024-03-21T10:09:38.052 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 64/198 2024-03-21T10:09:38.052 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 65/198 2024-03-21T10:09:38.052 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 66/198 2024-03-21T10:09:38.052 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 67/198 2024-03-21T10:09:38.052 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 68/198 2024-03-21T10:09:38.052 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 69/198 2024-03-21T10:09:38.053 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 70/198 2024-03-21T10:09:38.053 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 71/198 2024-03-21T10:09:38.053 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 72/198 2024-03-21T10:09:38.053 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 73/198 2024-03-21T10:09:38.053 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 74/198 2024-03-21T10:09:38.053 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 75/198 2024-03-21T10:09:38.053 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 76/198 2024-03-21T10:09:38.053 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 77/198 2024-03-21T10:09:38.053 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 78/198 2024-03-21T10:09:38.053 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 79/198 2024-03-21T10:09:38.053 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 80/198 2024-03-21T10:09:38.053 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 81/198 2024-03-21T10:09:38.053 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 82/198 2024-03-21T10:09:38.054 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 83/198 2024-03-21T10:09:38.054 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 84/198 2024-03-21T10:09:38.054 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : zlib-devel-1.2.11-40.el9.x86_64 85/198 2024-03-21T10:09:38.054 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 86/198 2024-03-21T10:09:38.054 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 87/198 2024-03-21T10:09:38.054 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 88/198 2024-03-21T10:09:38.054 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-ply-3.11-14.el9.noarch 89/198 2024-03-21T10:09:38.054 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 90/198 2024-03-21T10:09:38.054 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 91/198 2024-03-21T10:09:38.054 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : jq-1.6-16.el9.x86_64 92/198 2024-03-21T10:09:38.054 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 93/198 2024-03-21T10:09:38.055 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 94/198 2024-03-21T10:09:38.055 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 95/198 2024-03-21T10:09:38.055 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : elfutils-libelf-devel-0.190-2.el9.x86_64 96/198 2024-03-21T10:09:38.055 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 97/198 2024-03-21T10:09:38.055 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : openssl-devel-1:3.0.7-27.el9.x86_64 98/198 2024-03-21T10:09:38.055 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-devel-3.9.17-2.el9.x86_64 99/198 2024-03-21T10:09:38.055 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 100/198 2024-03-21T10:09:38.055 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : qatlib-23.11.0-1.el9.x86_64 101/198 2024-03-21T10:09:38.055 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : qatlib-service-23.11.0-1.el9.x86_64 102/198 2024-03-21T10:09:38.055 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : qatzip-libs-1.1.2-2.el9.x86_64 103/198 2024-03-21T10:09:38.055 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 104/198 2024-03-21T10:09:38.056 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 105/198 2024-03-21T10:09:38.056 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 106/198 2024-03-21T10:09:38.056 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 107/198 2024-03-21T10:09:38.056 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 108/198 2024-03-21T10:09:38.056 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 109/198 2024-03-21T10:09:38.056 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : libarrow-9.0.0-10.el9.x86_64 110/198 2024-03-21T10:09:38.056 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : libarrow-doc-9.0.0-10.el9.noarch 111/198 2024-03-21T10:09:38.056 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 112/198 2024-03-21T10:09:38.056 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 113/198 2024-03-21T10:09:38.056 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 114/198 2024-03-21T10:09:38.056 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : parquet-libs-9.0.0-10.el9.x86_64 115/198 2024-03-21T10:09:38.056 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 116/198 2024-03-21T10:09:38.057 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 117/198 2024-03-21T10:09:38.057 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 118/198 2024-03-21T10:09:38.057 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 119/198 2024-03-21T10:09:38.057 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 120/198 2024-03-21T10:09:38.057 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 121/198 2024-03-21T10:09:38.057 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 122/198 2024-03-21T10:09:38.057 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-colorama-0.4.4-12.el9.noarch 123/198 2024-03-21T10:09:38.057 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-google-auth-1:2.28.2-1.el9.noarch 124/198 2024-03-21T10:09:38.057 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 125/198 2024-03-21T10:09:38.057 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 126/198 2024-03-21T10:09:38.057 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 127/198 2024-03-21T10:09:38.057 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 128/198 2024-03-21T10:09:38.057 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 129/198 2024-03-21T10:09:38.058 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 130/198 2024-03-21T10:09:38.058 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 131/198 2024-03-21T10:09:38.058 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 132/198 2024-03-21T10:09:38.058 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 133/198 2024-03-21T10:09:38.058 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 134/198 2024-03-21T10:09:38.058 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 135/198 2024-03-21T10:09:38.058 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 136/198 2024-03-21T10:09:38.058 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 137/198 2024-03-21T10:09:38.058 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 138/198 2024-03-21T10:09:38.058 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 139/198 2024-03-21T10:09:38.058 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 140/198 2024-03-21T10:09:38.061 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 141/198 2024-03-21T10:09:38.061 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 142/198 2024-03-21T10:09:38.061 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 143/198 2024-03-21T10:09:38.061 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 144/198 2024-03-21T10:09:38.061 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 145/198 2024-03-21T10:09:38.062 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 146/198 2024-03-21T10:09:38.062 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 147/198 2024-03-21T10:09:38.062 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 148/198 2024-03-21T10:09:38.062 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : re2-1:20211101-3.el9.x86_64 149/198 2024-03-21T10:09:38.062 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 150/198 2024-03-21T10:09:38.062 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : librados2-2:19.0.0-2231.g2edfee68.el9.x86_64 151/198 2024-03-21T10:09:38.062 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 152/198 2024-03-21T10:09:38.062 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : librbd1-2:19.0.0-2231.g2edfee68.el9.x86_64 153/198 2024-03-21T10:09:38.062 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 154/198 2024-03-21T10:09:38.062 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : xfsdump-3.1.12-4.el9.x86_64 155/198 2024-03-21T10:09:38.062 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : xfsdump-3.1.12-3.el9.x86_64 156/198 2024-03-21T10:09:38.062 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : NetworkManager-1:1.47.2-1.el9.x86_64 157/198 2024-03-21T10:09:38.063 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : NetworkManager-1:1.44.0-2.el9.x86_64 158/198 2024-03-21T10:09:38.063 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : NetworkManager-libnm-1:1.47.2-1.el9.x86_64 159/198 2024-03-21T10:09:38.063 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : NetworkManager-libnm-1:1.44.0-2.el9.x86_64 160/198 2024-03-21T10:09:38.063 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : NetworkManager-team-1:1.47.2-1.el9.x86_64 161/198 2024-03-21T10:09:38.063 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : NetworkManager-team-1:1.44.0-2.el9.x86_64 162/198 2024-03-21T10:09:38.063 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : NetworkManager-tui-1:1.47.2-1.el9.x86_64 163/198 2024-03-21T10:09:38.063 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : NetworkManager-tui-1:1.44.0-2.el9.x86_64 164/198 2024-03-21T10:09:38.063 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : acl-2.3.1-4.el9.x86_64 165/198 2024-03-21T10:09:38.063 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : acl-2.3.1-3.el9.x86_64 166/198 2024-03-21T10:09:38.063 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : elfutils-debuginfod-client-0.190-2.el9.x86_64 167/198 2024-03-21T10:09:38.063 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : elfutils-debuginfod-client-0.189-3.el9.x86_64 168/198 2024-03-21T10:09:38.063 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : elfutils-libelf-0.190-2.el9.x86_64 169/198 2024-03-21T10:09:38.064 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : elfutils-libelf-0.189-3.el9.x86_64 170/198 2024-03-21T10:09:38.064 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : elfutils-libs-0.190-2.el9.x86_64 171/198 2024-03-21T10:09:38.064 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : elfutils-libs-0.189-3.el9.x86_64 172/198 2024-03-21T10:09:38.064 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : libacl-2.3.1-4.el9.x86_64 173/198 2024-03-21T10:09:38.064 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : libacl-2.3.1-3.el9.x86_64 174/198 2024-03-21T10:09:38.064 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : libblkid-2.37.4-18.el9.x86_64 175/198 2024-03-21T10:09:38.064 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : libblkid-2.37.4-15.el9.x86_64 176/198 2024-03-21T10:09:38.064 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : libfdisk-2.37.4-18.el9.x86_64 177/198 2024-03-21T10:09:38.064 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : libfdisk-2.37.4-15.el9.x86_64 178/198 2024-03-21T10:09:38.064 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : libmount-2.37.4-18.el9.x86_64 179/198 2024-03-21T10:09:38.064 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : libmount-2.37.4-15.el9.x86_64 180/198 2024-03-21T10:09:38.064 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : libsmartcols-2.37.4-18.el9.x86_64 181/198 2024-03-21T10:09:38.065 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : libsmartcols-2.37.4-15.el9.x86_64 182/198 2024-03-21T10:09:38.065 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : libuuid-2.37.4-18.el9.x86_64 183/198 2024-03-21T10:09:38.065 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : libuuid-2.37.4-15.el9.x86_64 184/198 2024-03-21T10:09:38.065 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 185/198 2024-03-21T10:09:38.065 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : openssl-1:3.0.7-24.el9.x86_64 186/198 2024-03-21T10:09:38.065 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 187/198 2024-03-21T10:09:38.065 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : openssl-libs-1:3.0.7-24.el9.x86_64 188/198 2024-03-21T10:09:38.065 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : util-linux-2.37.4-18.el9.x86_64 189/198 2024-03-21T10:09:38.065 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : util-linux-2.37.4-15.el9.x86_64 190/198 2024-03-21T10:09:38.065 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : util-linux-core-2.37.4-18.el9.x86_64 191/198 2024-03-21T10:09:38.065 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : util-linux-core-2.37.4-15.el9.x86_64 192/198 2024-03-21T10:09:38.066 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : util-linux-user-2.37.4-18.el9.x86_64 193/198 2024-03-21T10:09:38.066 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : util-linux-user-2.37.4-15.el9.x86_64 194/198 2024-03-21T10:09:38.066 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : xfsprogs-6.3.0-1.el9.x86_64 195/198 2024-03-21T10:09:38.066 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : xfsprogs-5.19.0-4.el9.x86_64 196/198 2024-03-21T10:09:38.066 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : libacl-devel-2.3.1-4.el9.x86_64 197/198 2024-03-21T10:09:38.398 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : ceph-mon-2:19.0.0-2231.g2edfee68.el9.x86_64 136/198 2024-03-21T10:09:38.476 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: ceph-mon-2:19.0.0-2231.g2edfee68.el9.x86_64 136/198 2024-03-21T10:09:38.476 INFO:teuthology.orchestra.run.smithi157.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-21T10:09:38.476 INFO:teuthology.orchestra.run.smithi157.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-03-21T10:09:38.476 INFO:teuthology.orchestra.run.smithi157.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-03-21T10:09:38.476 INFO:teuthology.orchestra.run.smithi157.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-03-21T10:09:38.476 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-03-21T10:09:38.599 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : automake-1.16.2-8.el9.noarch 137/198 2024-03-21T10:09:39.196 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: cephadm-2:19.0.0-2231.g2edfee68.el9.noarch 138/198 2024-03-21T10:09:39.243 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : cephadm-2:19.0.0-2231.g2edfee68.el9.noarch 138/198 2024-03-21T10:09:39.296 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-2231.g2edfee68.e 139/198 2024-03-21T10:09:39.396 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-2231.g2edfee68. 140/198 2024-03-21T10:09:39.438 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-2231.g2edfee68.el9.noa 141/198 2024-03-21T10:09:39.758 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-2231.g2edfee68.el9.noa 141/198 2024-03-21T10:09:39.805 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-2231.g2edfee68.el9.n 142/198 2024-03-21T10:09:40.465 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : libacl-devel-2.3.1-3.el9.x86_64 198/198 2024-03-21T10:09:40.466 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-03-21T10:09:40.466 INFO:teuthology.orchestra.run.smithi100.stdout:Upgraded: 2024-03-21T10:09:40.466 INFO:teuthology.orchestra.run.smithi100.stdout: NetworkManager-1:1.47.2-1.el9.x86_64 2024-03-21T10:09:40.466 INFO:teuthology.orchestra.run.smithi100.stdout: NetworkManager-libnm-1:1.47.2-1.el9.x86_64 2024-03-21T10:09:40.466 INFO:teuthology.orchestra.run.smithi100.stdout: NetworkManager-team-1:1.47.2-1.el9.x86_64 2024-03-21T10:09:40.466 INFO:teuthology.orchestra.run.smithi100.stdout: NetworkManager-tui-1:1.47.2-1.el9.x86_64 2024-03-21T10:09:40.466 INFO:teuthology.orchestra.run.smithi100.stdout: acl-2.3.1-4.el9.x86_64 2024-03-21T10:09:40.466 INFO:teuthology.orchestra.run.smithi100.stdout: elfutils-debuginfod-client-0.190-2.el9.x86_64 2024-03-21T10:09:40.466 INFO:teuthology.orchestra.run.smithi100.stdout: elfutils-libelf-0.190-2.el9.x86_64 2024-03-21T10:09:40.466 INFO:teuthology.orchestra.run.smithi100.stdout: elfutils-libs-0.190-2.el9.x86_64 2024-03-21T10:09:40.466 INFO:teuthology.orchestra.run.smithi100.stdout: libacl-2.3.1-4.el9.x86_64 2024-03-21T10:09:40.467 INFO:teuthology.orchestra.run.smithi100.stdout: libacl-devel-2.3.1-4.el9.x86_64 2024-03-21T10:09:40.467 INFO:teuthology.orchestra.run.smithi100.stdout: libblkid-2.37.4-18.el9.x86_64 2024-03-21T10:09:40.467 INFO:teuthology.orchestra.run.smithi100.stdout: libfdisk-2.37.4-18.el9.x86_64 2024-03-21T10:09:40.467 INFO:teuthology.orchestra.run.smithi100.stdout: libmount-2.37.4-18.el9.x86_64 2024-03-21T10:09:40.467 INFO:teuthology.orchestra.run.smithi100.stdout: librados2-2:19.0.0-2231.g2edfee68.el9.x86_64 2024-03-21T10:09:40.467 INFO:teuthology.orchestra.run.smithi100.stdout: librbd1-2:19.0.0-2231.g2edfee68.el9.x86_64 2024-03-21T10:09:40.467 INFO:teuthology.orchestra.run.smithi100.stdout: libsmartcols-2.37.4-18.el9.x86_64 2024-03-21T10:09:40.467 INFO:teuthology.orchestra.run.smithi100.stdout: libuuid-2.37.4-18.el9.x86_64 2024-03-21T10:09:40.467 INFO:teuthology.orchestra.run.smithi100.stdout: openssl-1:3.0.7-27.el9.x86_64 2024-03-21T10:09:40.467 INFO:teuthology.orchestra.run.smithi100.stdout: openssl-libs-1:3.0.7-27.el9.x86_64 2024-03-21T10:09:40.467 INFO:teuthology.orchestra.run.smithi100.stdout: util-linux-2.37.4-18.el9.x86_64 2024-03-21T10:09:40.467 INFO:teuthology.orchestra.run.smithi100.stdout: util-linux-core-2.37.4-18.el9.x86_64 2024-03-21T10:09:40.467 INFO:teuthology.orchestra.run.smithi100.stdout: util-linux-user-2.37.4-18.el9.x86_64 2024-03-21T10:09:40.468 INFO:teuthology.orchestra.run.smithi100.stdout: xfsdump-3.1.12-4.el9.x86_64 2024-03-21T10:09:40.468 INFO:teuthology.orchestra.run.smithi100.stdout: xfsprogs-6.3.0-1.el9.x86_64 2024-03-21T10:09:40.468 INFO:teuthology.orchestra.run.smithi100.stdout:Installed: 2024-03-21T10:09:40.468 INFO:teuthology.orchestra.run.smithi100.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-03-21T10:09:40.468 INFO:teuthology.orchestra.run.smithi100.stdout: automake-1.16.2-8.el9.noarch 2024-03-21T10:09:40.468 INFO:teuthology.orchestra.run.smithi100.stdout: bison-3.7.4-5.el9.x86_64 2024-03-21T10:09:40.468 INFO:teuthology.orchestra.run.smithi100.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-03-21T10:09:40.468 INFO:teuthology.orchestra.run.smithi100.stdout: ceph-2:19.0.0-2231.g2edfee68.el9.x86_64 2024-03-21T10:09:40.468 INFO:teuthology.orchestra.run.smithi100.stdout: ceph-base-2:19.0.0-2231.g2edfee68.el9.x86_64 2024-03-21T10:09:40.468 INFO:teuthology.orchestra.run.smithi100.stdout: ceph-common-2:19.0.0-2231.g2edfee68.el9.x86_64 2024-03-21T10:09:40.468 INFO:teuthology.orchestra.run.smithi100.stdout: ceph-fuse-2:19.0.0-2231.g2edfee68.el9.x86_64 2024-03-21T10:09:40.468 INFO:teuthology.orchestra.run.smithi100.stdout: ceph-grafana-dashboards-2:19.0.0-2231.g2edfee68.el9.noarch 2024-03-21T10:09:40.468 INFO:teuthology.orchestra.run.smithi100.stdout: ceph-immutable-object-cache-2:19.0.0-2231.g2edfee68.el9.x86_64 2024-03-21T10:09:40.468 INFO:teuthology.orchestra.run.smithi100.stdout: ceph-mds-2:19.0.0-2231.g2edfee68.el9.x86_64 2024-03-21T10:09:40.469 INFO:teuthology.orchestra.run.smithi100.stdout: ceph-mgr-2:19.0.0-2231.g2edfee68.el9.x86_64 2024-03-21T10:09:40.469 INFO:teuthology.orchestra.run.smithi100.stdout: ceph-mgr-cephadm-2:19.0.0-2231.g2edfee68.el9.noarch 2024-03-21T10:09:40.469 INFO:teuthology.orchestra.run.smithi100.stdout: ceph-mgr-dashboard-2:19.0.0-2231.g2edfee68.el9.noarch 2024-03-21T10:09:40.469 INFO:teuthology.orchestra.run.smithi100.stdout: ceph-mgr-diskprediction-local-2:19.0.0-2231.g2edfee68.el9.noarch 2024-03-21T10:09:40.469 INFO:teuthology.orchestra.run.smithi100.stdout: ceph-mgr-modules-core-2:19.0.0-2231.g2edfee68.el9.noarch 2024-03-21T10:09:40.469 INFO:teuthology.orchestra.run.smithi100.stdout: ceph-mgr-rook-2:19.0.0-2231.g2edfee68.el9.noarch 2024-03-21T10:09:40.469 INFO:teuthology.orchestra.run.smithi100.stdout: ceph-mon-2:19.0.0-2231.g2edfee68.el9.x86_64 2024-03-21T10:09:40.469 INFO:teuthology.orchestra.run.smithi100.stdout: ceph-osd-2:19.0.0-2231.g2edfee68.el9.x86_64 2024-03-21T10:09:40.469 INFO:teuthology.orchestra.run.smithi100.stdout: ceph-prometheus-alerts-2:19.0.0-2231.g2edfee68.el9.noarch 2024-03-21T10:09:40.469 INFO:teuthology.orchestra.run.smithi100.stdout: ceph-radosgw-2:19.0.0-2231.g2edfee68.el9.x86_64 2024-03-21T10:09:40.469 INFO:teuthology.orchestra.run.smithi100.stdout: ceph-selinux-2:19.0.0-2231.g2edfee68.el9.x86_64 2024-03-21T10:09:40.470 INFO:teuthology.orchestra.run.smithi100.stdout: ceph-test-2:19.0.0-2231.g2edfee68.el9.x86_64 2024-03-21T10:09:40.470 INFO:teuthology.orchestra.run.smithi100.stdout: ceph-volume-2:19.0.0-2231.g2edfee68.el9.noarch 2024-03-21T10:09:40.470 INFO:teuthology.orchestra.run.smithi100.stdout: cephadm-2:19.0.0-2231.g2edfee68.el9.noarch 2024-03-21T10:09:40.470 INFO:teuthology.orchestra.run.smithi100.stdout: cephfs-mirror-2:19.0.0-2231.g2edfee68.el9.x86_64 2024-03-21T10:09:40.470 INFO:teuthology.orchestra.run.smithi100.stdout: cephfs-shell-2:19.0.0-2231.g2edfee68.el9.x86_64 2024-03-21T10:09:40.470 INFO:teuthology.orchestra.run.smithi100.stdout: cephfs-top-2:19.0.0-2231.g2edfee68.el9.noarch 2024-03-21T10:09:40.470 INFO:teuthology.orchestra.run.smithi100.stdout: elfutils-libelf-devel-0.190-2.el9.x86_64 2024-03-21T10:09:40.470 INFO:teuthology.orchestra.run.smithi100.stdout: flex-2.6.4-9.el9.x86_64 2024-03-21T10:09:40.470 INFO:teuthology.orchestra.run.smithi100.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-03-21T10:09:40.470 INFO:teuthology.orchestra.run.smithi100.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-03-21T10:09:40.470 INFO:teuthology.orchestra.run.smithi100.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-03-21T10:09:40.470 INFO:teuthology.orchestra.run.smithi100.stdout: fuse-2.9.9-15.el9.x86_64 2024-03-21T10:09:40.470 INFO:teuthology.orchestra.run.smithi100.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-03-21T10:09:40.471 INFO:teuthology.orchestra.run.smithi100.stdout: grpc-data-1.46.7-10.el9.noarch 2024-03-21T10:09:40.471 INFO:teuthology.orchestra.run.smithi100.stdout: jq-1.6-16.el9.x86_64 2024-03-21T10:09:40.471 INFO:teuthology.orchestra.run.smithi100.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-03-21T10:09:40.471 INFO:teuthology.orchestra.run.smithi100.stdout: libarrow-9.0.0-10.el9.x86_64 2024-03-21T10:09:40.471 INFO:teuthology.orchestra.run.smithi100.stdout: libarrow-doc-9.0.0-10.el9.noarch 2024-03-21T10:09:40.471 INFO:teuthology.orchestra.run.smithi100.stdout: libcephfs-devel-2:19.0.0-2231.g2edfee68.el9.x86_64 2024-03-21T10:09:40.471 INFO:teuthology.orchestra.run.smithi100.stdout: libcephfs2-2:19.0.0-2231.g2edfee68.el9.x86_64 2024-03-21T10:09:40.471 INFO:teuthology.orchestra.run.smithi100.stdout: libcephsqlite-2:19.0.0-2231.g2edfee68.el9.x86_64 2024-03-21T10:09:40.471 INFO:teuthology.orchestra.run.smithi100.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-03-21T10:09:40.471 INFO:teuthology.orchestra.run.smithi100.stdout: liboath-2.6.7-2.el9.x86_64 2024-03-21T10:09:40.471 INFO:teuthology.orchestra.run.smithi100.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-03-21T10:09:40.471 INFO:teuthology.orchestra.run.smithi100.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-03-21T10:09:40.471 INFO:teuthology.orchestra.run.smithi100.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-03-21T10:09:40.472 INFO:teuthology.orchestra.run.smithi100.stdout: librados-devel-2:19.0.0-2231.g2edfee68.el9.x86_64 2024-03-21T10:09:40.472 INFO:teuthology.orchestra.run.smithi100.stdout: libradosstriper1-2:19.0.0-2231.g2edfee68.el9.x86_64 2024-03-21T10:09:40.472 INFO:teuthology.orchestra.run.smithi100.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-03-21T10:09:40.472 INFO:teuthology.orchestra.run.smithi100.stdout: librgw2-2:19.0.0-2231.g2edfee68.el9.x86_64 2024-03-21T10:09:40.472 INFO:teuthology.orchestra.run.smithi100.stdout: libtool-2.4.6-45.el9.x86_64 2024-03-21T10:09:40.472 INFO:teuthology.orchestra.run.smithi100.stdout: libunwind-1.6.2-1.el9.x86_64 2024-03-21T10:09:40.472 INFO:teuthology.orchestra.run.smithi100.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-03-21T10:09:40.472 INFO:teuthology.orchestra.run.smithi100.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-03-21T10:09:40.472 INFO:teuthology.orchestra.run.smithi100.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-03-21T10:09:40.472 INFO:teuthology.orchestra.run.smithi100.stdout: lua-5.4.4-4.el9.x86_64 2024-03-21T10:09:40.473 INFO:teuthology.orchestra.run.smithi100.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-03-21T10:09:40.473 INFO:teuthology.orchestra.run.smithi100.stdout: luarocks-3.9.2-1.el9.noarch 2024-03-21T10:09:40.473 INFO:teuthology.orchestra.run.smithi100.stdout: oniguruma-6.9.6-1.el9.5.x86_64 2024-03-21T10:09:40.473 INFO:teuthology.orchestra.run.smithi100.stdout: openblas-0.3.21-2.el9.x86_64 2024-03-21T10:09:40.473 INFO:teuthology.orchestra.run.smithi100.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-03-21T10:09:40.473 INFO:teuthology.orchestra.run.smithi100.stdout: openssl-devel-1:3.0.7-27.el9.x86_64 2024-03-21T10:09:40.473 INFO:teuthology.orchestra.run.smithi100.stdout: parquet-libs-9.0.0-10.el9.x86_64 2024-03-21T10:09:40.473 INFO:teuthology.orchestra.run.smithi100.stdout: postgresql-13.11-1.el9.x86_64 2024-03-21T10:09:40.473 INFO:teuthology.orchestra.run.smithi100.stdout: postgresql-contrib-13.11-1.el9.x86_64 2024-03-21T10:09:40.473 INFO:teuthology.orchestra.run.smithi100.stdout: postgresql-private-libs-13.11-1.el9.x86_64 2024-03-21T10:09:40.473 INFO:teuthology.orchestra.run.smithi100.stdout: postgresql-server-13.11-1.el9.x86_64 2024-03-21T10:09:40.473 INFO:teuthology.orchestra.run.smithi100.stdout: protobuf-3.14.0-13.el9.x86_64 2024-03-21T10:09:40.473 INFO:teuthology.orchestra.run.smithi100.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-03-21T10:09:40.473 INFO:teuthology.orchestra.run.smithi100.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-03-21T10:09:40.473 INFO:teuthology.orchestra.run.smithi100.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-03-21T10:09:40.473 INFO:teuthology.orchestra.run.smithi100.stdout: python3-babel-2.9.1-2.el9.noarch 2024-03-21T10:09:40.474 INFO:teuthology.orchestra.run.smithi100.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-03-21T10:09:40.474 INFO:teuthology.orchestra.run.smithi100.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-03-21T10:09:40.474 INFO:teuthology.orchestra.run.smithi100.stdout: python3-ceph-argparse-2:19.0.0-2231.g2edfee68.el9.x86_64 2024-03-21T10:09:40.474 INFO:teuthology.orchestra.run.smithi100.stdout: python3-ceph-common-2:19.0.0-2231.g2edfee68.el9.x86_64 2024-03-21T10:09:40.474 INFO:teuthology.orchestra.run.smithi100.stdout: python3-cephfs-2:19.0.0-2231.g2edfee68.el9.x86_64 2024-03-21T10:09:40.474 INFO:teuthology.orchestra.run.smithi100.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-03-21T10:09:40.474 INFO:teuthology.orchestra.run.smithi100.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-03-21T10:09:40.474 INFO:teuthology.orchestra.run.smithi100.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-03-21T10:09:40.474 INFO:teuthology.orchestra.run.smithi100.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-03-21T10:09:40.474 INFO:teuthology.orchestra.run.smithi100.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-03-21T10:09:40.474 INFO:teuthology.orchestra.run.smithi100.stdout: python3-colorama-0.4.4-12.el9.noarch 2024-03-21T10:09:40.475 INFO:teuthology.orchestra.run.smithi100.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-03-21T10:09:40.475 INFO:teuthology.orchestra.run.smithi100.stdout: python3-devel-3.9.17-2.el9.x86_64 2024-03-21T10:09:40.475 INFO:teuthology.orchestra.run.smithi100.stdout: python3-google-auth-1:2.28.2-1.el9.noarch 2024-03-21T10:09:40.475 INFO:teuthology.orchestra.run.smithi100.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-03-21T10:09:40.475 INFO:teuthology.orchestra.run.smithi100.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-03-21T10:09:40.475 INFO:teuthology.orchestra.run.smithi100.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-03-21T10:09:40.475 INFO:teuthology.orchestra.run.smithi100.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-03-21T10:09:40.475 INFO:teuthology.orchestra.run.smithi100.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-03-21T10:09:40.475 INFO:teuthology.orchestra.run.smithi100.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-03-21T10:09:40.475 INFO:teuthology.orchestra.run.smithi100.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-03-21T10:09:40.476 INFO:teuthology.orchestra.run.smithi100.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-03-21T10:09:40.476 INFO:teuthology.orchestra.run.smithi100.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-03-21T10:09:40.476 INFO:teuthology.orchestra.run.smithi100.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-03-21T10:09:40.476 INFO:teuthology.orchestra.run.smithi100.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-03-21T10:09:40.476 INFO:teuthology.orchestra.run.smithi100.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-03-21T10:09:40.476 INFO:teuthology.orchestra.run.smithi100.stdout: python3-mako-1.1.4-6.el9.noarch 2024-03-21T10:09:40.476 INFO:teuthology.orchestra.run.smithi100.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-03-21T10:09:40.476 INFO:teuthology.orchestra.run.smithi100.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-03-21T10:09:40.476 INFO:teuthology.orchestra.run.smithi100.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-03-21T10:09:40.476 INFO:teuthology.orchestra.run.smithi100.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-03-21T10:09:40.476 INFO:teuthology.orchestra.run.smithi100.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-03-21T10:09:40.476 INFO:teuthology.orchestra.run.smithi100.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-03-21T10:09:40.476 INFO:teuthology.orchestra.run.smithi100.stdout: python3-packaging-20.9-5.el9.noarch 2024-03-21T10:09:40.476 INFO:teuthology.orchestra.run.smithi100.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-03-21T10:09:40.477 INFO:teuthology.orchestra.run.smithi100.stdout: python3-ply-3.11-14.el9.noarch 2024-03-21T10:09:40.477 INFO:teuthology.orchestra.run.smithi100.stdout: python3-portend-3.1.0-2.el9.noarch 2024-03-21T10:09:40.477 INFO:teuthology.orchestra.run.smithi100.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-03-21T10:09:40.477 INFO:teuthology.orchestra.run.smithi100.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-03-21T10:09:40.477 INFO:teuthology.orchestra.run.smithi100.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-03-21T10:09:40.477 INFO:teuthology.orchestra.run.smithi100.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-03-21T10:09:40.477 INFO:teuthology.orchestra.run.smithi100.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-03-21T10:09:40.477 INFO:teuthology.orchestra.run.smithi100.stdout: python3-pycparser-2.20-6.el9.noarch 2024-03-21T10:09:40.477 INFO:teuthology.orchestra.run.smithi100.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-03-21T10:09:40.477 INFO:teuthology.orchestra.run.smithi100.stdout: python3-pytz-2021.1-5.el9.noarch 2024-03-21T10:09:40.478 INFO:teuthology.orchestra.run.smithi100.stdout: python3-rados-2:19.0.0-2231.g2edfee68.el9.x86_64 2024-03-21T10:09:40.478 INFO:teuthology.orchestra.run.smithi100.stdout: python3-rbd-2:19.0.0-2231.g2edfee68.el9.x86_64 2024-03-21T10:09:40.478 INFO:teuthology.orchestra.run.smithi100.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-03-21T10:09:40.478 INFO:teuthology.orchestra.run.smithi100.stdout: python3-requests-2.25.1-8.el9.noarch 2024-03-21T10:09:40.478 INFO:teuthology.orchestra.run.smithi100.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-03-21T10:09:40.478 INFO:teuthology.orchestra.run.smithi100.stdout: python3-rgw-2:19.0.0-2231.g2edfee68.el9.x86_64 2024-03-21T10:09:40.478 INFO:teuthology.orchestra.run.smithi100.stdout: python3-routes-2.5.1-5.el9.noarch 2024-03-21T10:09:40.478 INFO:teuthology.orchestra.run.smithi100.stdout: python3-rsa-4.9-2.el9.noarch 2024-03-21T10:09:40.478 INFO:teuthology.orchestra.run.smithi100.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-03-21T10:09:40.478 INFO:teuthology.orchestra.run.smithi100.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-03-21T10:09:40.478 INFO:teuthology.orchestra.run.smithi100.stdout: python3-toml-0.10.2-6.el9.noarch 2024-03-21T10:09:40.478 INFO:teuthology.orchestra.run.smithi100.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-03-21T10:09:40.478 INFO:teuthology.orchestra.run.smithi100.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-03-21T10:09:40.479 INFO:teuthology.orchestra.run.smithi100.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-03-21T10:09:40.479 INFO:teuthology.orchestra.run.smithi100.stdout: python3-webob-1.8.7-6.el9.noarch 2024-03-21T10:09:40.479 INFO:teuthology.orchestra.run.smithi100.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-03-21T10:09:40.479 INFO:teuthology.orchestra.run.smithi100.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-03-21T10:09:40.479 INFO:teuthology.orchestra.run.smithi100.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-03-21T10:09:40.479 INFO:teuthology.orchestra.run.smithi100.stdout: qatlib-23.11.0-1.el9.x86_64 2024-03-21T10:09:40.479 INFO:teuthology.orchestra.run.smithi100.stdout: qatlib-service-23.11.0-1.el9.x86_64 2024-03-21T10:09:40.479 INFO:teuthology.orchestra.run.smithi100.stdout: qatzip-libs-1.1.2-2.el9.x86_64 2024-03-21T10:09:40.479 INFO:teuthology.orchestra.run.smithi100.stdout: rbd-fuse-2:19.0.0-2231.g2edfee68.el9.x86_64 2024-03-21T10:09:40.479 INFO:teuthology.orchestra.run.smithi100.stdout: rbd-mirror-2:19.0.0-2231.g2edfee68.el9.x86_64 2024-03-21T10:09:40.479 INFO:teuthology.orchestra.run.smithi100.stdout: rbd-nbd-2:19.0.0-2231.g2edfee68.el9.x86_64 2024-03-21T10:09:40.479 INFO:teuthology.orchestra.run.smithi100.stdout: re2-1:20211101-3.el9.x86_64 2024-03-21T10:09:40.480 INFO:teuthology.orchestra.run.smithi100.stdout: socat-1.7.4.1-5.el9.x86_64 2024-03-21T10:09:40.480 INFO:teuthology.orchestra.run.smithi100.stdout: thrift-0.15.0-2.el9.x86_64 2024-03-21T10:09:40.480 INFO:teuthology.orchestra.run.smithi100.stdout: uuid-1.6.2-55.el9.x86_64 2024-03-21T10:09:40.480 INFO:teuthology.orchestra.run.smithi100.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-03-21T10:09:40.480 INFO:teuthology.orchestra.run.smithi100.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-03-21T10:09:40.480 INFO:teuthology.orchestra.run.smithi100.stdout: zlib-devel-1.2.11-40.el9.x86_64 2024-03-21T10:09:40.480 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-03-21T10:09:40.480 INFO:teuthology.orchestra.run.smithi100.stdout:Complete! 2024-03-21T10:09:40.939 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-2231.g2edfee68.el9.n 142/198 2024-03-21T10:09:40.963 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-2231.g2ed 143/198 2024-03-21T10:09:41.104 DEBUG:teuthology.parallel:result is None 2024-03-21T10:09:41.107 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-2231.g2ed 143/198 2024-03-21T10:09:41.277 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-2231.g2edfee68.el 144/198 2024-03-21T10:09:41.324 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : ceph-mgr-2:19.0.0-2231.g2edfee68.el9.x86_64 145/198 2024-03-21T10:09:41.404 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: ceph-mgr-2:19.0.0-2231.g2edfee68.el9.x86_64 145/198 2024-03-21T10:09:41.404 INFO:teuthology.orchestra.run.smithi157.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-21T10:09:41.404 INFO:teuthology.orchestra.run.smithi157.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-03-21T10:09:41.405 INFO:teuthology.orchestra.run.smithi157.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-03-21T10:09:41.405 INFO:teuthology.orchestra.run.smithi157.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-03-21T10:09:41.405 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-03-21T10:09:41.466 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : ceph-mgr-rook-2:19.0.0-2231.g2edfee68.el9.noarch 146/198 2024-03-21T10:09:41.505 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-2231.g2edfee68.el9.noarch 146/198 2024-03-21T10:09:41.597 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : ceph-2:19.0.0-2231.g2edfee68.el9.x86_64 147/198 2024-03-21T10:09:42.583 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : libtool-2.4.6-45.el9.x86_64 148/198 2024-03-21T10:09:42.619 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : ceph-radosgw-2:19.0.0-2231.g2edfee68.el9.x86_64 149/198 2024-03-21T10:09:42.693 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-2231.g2edfee68.el9.x86_64 149/198 2024-03-21T10:09:42.693 INFO:teuthology.orchestra.run.smithi157.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-21T10:09:42.693 INFO:teuthology.orchestra.run.smithi157.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-03-21T10:09:42.693 INFO:teuthology.orchestra.run.smithi157.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-03-21T10:09:42.693 INFO:teuthology.orchestra.run.smithi157.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-03-21T10:09:42.693 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-03-21T10:09:42.744 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-2231.g2edfe 150/198 2024-03-21T10:09:42.828 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-2231.g2edfe 150/198 2024-03-21T10:09:42.828 INFO:teuthology.orchestra.run.smithi157.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-21T10:09:42.829 INFO:teuthology.orchestra.run.smithi157.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-03-21T10:09:42.829 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-03-21T10:09:42.903 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : cephfs-mirror-2:19.0.0-2231.g2edfee68.el9.x86_64 151/198 2024-03-21T10:09:43.007 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-2231.g2edfee68.el9.x86_64 151/198 2024-03-21T10:09:43.008 INFO:teuthology.orchestra.run.smithi157.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-21T10:09:43.008 INFO:teuthology.orchestra.run.smithi157.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-03-21T10:09:43.008 INFO:teuthology.orchestra.run.smithi157.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-03-21T10:09:43.008 INFO:teuthology.orchestra.run.smithi157.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-03-21T10:09:43.008 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-03-21T10:09:43.379 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : rbd-mirror-2:19.0.0-2231.g2edfee68.el9.x86_64 152/198 2024-03-21T10:09:43.474 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: rbd-mirror-2:19.0.0-2231.g2edfee68.el9.x86_64 152/198 2024-03-21T10:09:43.475 INFO:teuthology.orchestra.run.smithi157.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-21T10:09:43.475 INFO:teuthology.orchestra.run.smithi157.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-03-21T10:09:43.475 INFO:teuthology.orchestra.run.smithi157.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-03-21T10:09:43.475 INFO:teuthology.orchestra.run.smithi157.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-03-21T10:09:43.475 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-03-21T10:09:48.327 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : ceph-test-2:19.0.0-2231.g2edfee68.el9.x86_64 153/198 2024-03-21T10:09:48.380 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : rbd-fuse-2:19.0.0-2231.g2edfee68.el9.x86_64 154/198 2024-03-21T10:09:48.436 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : rbd-nbd-2:19.0.0-2231.g2edfee68.el9.x86_64 155/198 2024-03-21T10:09:48.480 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : elfutils-libelf-devel-0.190-2.el9.x86_64 156/198 2024-03-21T10:09:48.770 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : cephfs-shell-2:19.0.0-2231.g2edfee68.el9.x86_64 157/198 2024-03-21T10:09:48.895 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : postgresql-contrib-13.11-1.el9.x86_64 158/198 2024-03-21T10:09:48.942 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : ceph-fuse-2:19.0.0-2231.g2edfee68.el9.x86_64 159/198 2024-03-21T10:09:48.995 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 160/198 2024-03-21T10:09:49.058 INFO:teuthology.orchestra.run.smithi157.stdout: Upgrading : xfsdump-3.1.12-4.el9.x86_64 161/198 2024-03-21T10:09:49.094 INFO:teuthology.orchestra.run.smithi157.stdout: Upgrading : util-linux-user-2.37.4-18.el9.x86_64 162/198 2024-03-21T10:09:49.120 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : libcephfs-devel-2:19.0.0-2231.g2edfee68.el9.x86_ 163/198 2024-03-21T10:09:49.166 INFO:teuthology.orchestra.run.smithi157.stdout: Upgrading : elfutils-debuginfod-client-0.190-2.el9.x86_64 164/198 2024-03-21T10:09:49.219 INFO:teuthology.orchestra.run.smithi157.stdout: Upgrading : NetworkManager-team-1:1.47.2-1.el9.x86_64 165/198 2024-03-21T10:09:49.304 INFO:teuthology.orchestra.run.smithi157.stdout: Upgrading : NetworkManager-tui-1:1.47.2-1.el9.x86_64 166/198 2024-03-21T10:09:49.421 INFO:teuthology.orchestra.run.smithi157.stdout: Upgrading : acl-2.3.1-4.el9.x86_64 167/198 2024-03-21T10:09:49.459 INFO:teuthology.orchestra.run.smithi157.stdout: Upgrading : libacl-devel-2.3.1-4.el9.x86_64 168/198 2024-03-21T10:09:50.337 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: postgresql-server-13.11-1.el9.x86_64 169/198 2024-03-21T10:09:50.399 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : postgresql-server-13.11-1.el9.x86_64 169/198 2024-03-21T10:09:50.466 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: postgresql-server-13.11-1.el9.x86_64 169/198 2024-03-21T10:09:51.799 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : cephfs-top-2:19.0.0-2231.g2edfee68.el9.noarch 170/198 2024-03-21T10:09:51.896 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : openssl-devel-1:3.0.7-27.el9.x86_64 171/198 2024-03-21T10:09:51.953 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 172/198 2024-03-21T10:09:52.108 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : flex-2.6.4-9.el9.x86_64 173/198 2024-03-21T10:09:52.149 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : bison-3.7.4-5.el9.x86_64 174/198 2024-03-21T10:09:52.182 INFO:teuthology.orchestra.run.smithi157.stdout: Cleanup : NetworkManager-tui-1:1.44.0-2.el9.x86_64 175/198 2024-03-21T10:09:52.216 INFO:teuthology.orchestra.run.smithi157.stdout: Cleanup : elfutils-debuginfod-client-0.189-3.el9.x86_64 176/198 2024-03-21T10:09:52.277 INFO:teuthology.orchestra.run.smithi157.stdout: Cleanup : elfutils-libs-0.189-3.el9.x86_64 177/198 2024-03-21T10:09:52.277 INFO:teuthology.orchestra.run.smithi157.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 178/198 2024-03-21T10:09:52.428 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 178/198 2024-03-21T10:09:52.470 INFO:teuthology.orchestra.run.smithi157.stdout: Cleanup : openssl-1:3.0.7-24.el9.x86_64 179/198 2024-03-21T10:09:52.471 INFO:teuthology.orchestra.run.smithi157.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 180/198 2024-03-21T10:09:52.608 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 180/198 2024-03-21T10:09:52.668 INFO:teuthology.orchestra.run.smithi157.stdout: Cleanup : xfsdump-3.1.12-3.el9.x86_64 181/198 2024-03-21T10:09:52.716 INFO:teuthology.orchestra.run.smithi157.stdout: Cleanup : xfsprogs-5.19.0-4.el9.x86_64 182/198 2024-03-21T10:09:52.757 INFO:teuthology.orchestra.run.smithi157.stdout: Cleanup : acl-2.3.1-3.el9.x86_64 183/198 2024-03-21T10:09:52.789 INFO:teuthology.orchestra.run.smithi157.stdout: Cleanup : libacl-devel-2.3.1-3.el9.x86_64 184/198 2024-03-21T10:09:52.860 INFO:teuthology.orchestra.run.smithi157.stdout: Cleanup : util-linux-user-2.37.4-15.el9.x86_64 185/198 2024-03-21T10:09:52.936 INFO:teuthology.orchestra.run.smithi157.stdout: Cleanup : util-linux-2.37.4-15.el9.x86_64 186/198 2024-03-21T10:09:52.994 INFO:teuthology.orchestra.run.smithi157.stdout: Cleanup : util-linux-core-2.37.4-15.el9.x86_64 187/198 2024-03-21T10:09:53.027 INFO:teuthology.orchestra.run.smithi157.stdout: Cleanup : libfdisk-2.37.4-15.el9.x86_64 188/198 2024-03-21T10:09:53.061 INFO:teuthology.orchestra.run.smithi157.stdout: Cleanup : libmount-2.37.4-15.el9.x86_64 189/198 2024-03-21T10:09:53.094 INFO:teuthology.orchestra.run.smithi157.stdout: Cleanup : libblkid-2.37.4-15.el9.x86_64 190/198 2024-03-21T10:09:53.128 INFO:teuthology.orchestra.run.smithi157.stdout: Cleanup : NetworkManager-team-1:1.44.0-2.el9.x86_64 191/198 2024-03-21T10:09:53.165 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: NetworkManager-1:1.44.0-2.el9.x86_64 192/198 2024-03-21T10:09:53.165 INFO:teuthology.orchestra.run.smithi157.stdout: Cleanup : NetworkManager-1:1.44.0-2.el9.x86_64 192/198 2024-03-21T10:09:53.486 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: NetworkManager-1:1.44.0-2.el9.x86_64 192/198 2024-03-21T10:09:53.486 INFO:teuthology.orchestra.run.smithi157.stdout: Cleanup : NetworkManager-libnm-1:1.44.0-2.el9.x86_64 193/198 2024-03-21T10:09:53.609 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: NetworkManager-libnm-1:1.44.0-2.el9.x86_64 193/198 2024-03-21T10:09:53.639 INFO:teuthology.orchestra.run.smithi157.stdout: Cleanup : libuuid-2.37.4-15.el9.x86_64 194/198 2024-03-21T10:09:53.693 INFO:teuthology.orchestra.run.smithi157.stdout: Cleanup : libsmartcols-2.37.4-15.el9.x86_64 195/198 2024-03-21T10:09:53.725 INFO:teuthology.orchestra.run.smithi157.stdout: Cleanup : libacl-2.3.1-3.el9.x86_64 196/198 2024-03-21T10:09:53.765 INFO:teuthology.orchestra.run.smithi157.stdout: Cleanup : openssl-libs-1:3.0.7-24.el9.x86_64 197/198 2024-03-21T10:09:53.882 INFO:teuthology.orchestra.run.smithi157.stdout: Cleanup : elfutils-libelf-0.189-3.el9.x86_64 198/198 2024-03-21T10:09:54.137 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: elfutils-libelf-0.189-3.el9.x86_64 198/198 2024-03-21T10:09:54.137 INFO:teuthology.orchestra.run.smithi157.stdout:Couldn't write '0' to 'kernel/yama/ptrace_scope', ignoring: No such file or directory 2024-03-21T10:09:54.137 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-03-21T10:09:59.020 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : ceph-2:19.0.0-2231.g2edfee68.el9.x86_64 1/198 2024-03-21T10:09:59.021 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : ceph-base-2:19.0.0-2231.g2edfee68.el9.x86_64 2/198 2024-03-21T10:09:59.021 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : ceph-common-2:19.0.0-2231.g2edfee68.el9.x86_64 3/198 2024-03-21T10:09:59.021 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : ceph-fuse-2:19.0.0-2231.g2edfee68.el9.x86_64 4/198 2024-03-21T10:09:59.021 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-2231.g2edfe 5/198 2024-03-21T10:09:59.021 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : ceph-mds-2:19.0.0-2231.g2edfee68.el9.x86_64 6/198 2024-03-21T10:09:59.021 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : ceph-mgr-2:19.0.0-2231.g2edfee68.el9.x86_64 7/198 2024-03-21T10:09:59.021 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : ceph-mon-2:19.0.0-2231.g2edfee68.el9.x86_64 8/198 2024-03-21T10:09:59.021 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : ceph-osd-2:19.0.0-2231.g2edfee68.el9.x86_64 9/198 2024-03-21T10:09:59.021 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : ceph-radosgw-2:19.0.0-2231.g2edfee68.el9.x86_64 10/198 2024-03-21T10:09:59.021 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : ceph-selinux-2:19.0.0-2231.g2edfee68.el9.x86_64 11/198 2024-03-21T10:09:59.021 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : ceph-test-2:19.0.0-2231.g2edfee68.el9.x86_64 12/198 2024-03-21T10:09:59.021 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : cephfs-mirror-2:19.0.0-2231.g2edfee68.el9.x86_64 13/198 2024-03-21T10:09:59.022 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : cephfs-shell-2:19.0.0-2231.g2edfee68.el9.x86_64 14/198 2024-03-21T10:09:59.022 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : libcephfs-devel-2:19.0.0-2231.g2edfee68.el9.x86_ 15/198 2024-03-21T10:09:59.022 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : libcephfs2-2:19.0.0-2231.g2edfee68.el9.x86_64 16/198 2024-03-21T10:09:59.022 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : libcephsqlite-2:19.0.0-2231.g2edfee68.el9.x86_64 17/198 2024-03-21T10:09:59.022 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : librados-devel-2:19.0.0-2231.g2edfee68.el9.x86_6 18/198 2024-03-21T10:09:59.022 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : libradosstriper1-2:19.0.0-2231.g2edfee68.el9.x86 19/198 2024-03-21T10:09:59.022 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : librgw2-2:19.0.0-2231.g2edfee68.el9.x86_64 20/198 2024-03-21T10:09:59.022 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-ceph-argparse-2:19.0.0-2231.g2edfee68.el 21/198 2024-03-21T10:09:59.022 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-ceph-common-2:19.0.0-2231.g2edfee68.el9. 22/198 2024-03-21T10:09:59.022 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-cephfs-2:19.0.0-2231.g2edfee68.el9.x86_6 23/198 2024-03-21T10:09:59.022 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-rados-2:19.0.0-2231.g2edfee68.el9.x86_64 24/198 2024-03-21T10:09:59.022 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-rbd-2:19.0.0-2231.g2edfee68.el9.x86_64 25/198 2024-03-21T10:09:59.022 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-rgw-2:19.0.0-2231.g2edfee68.el9.x86_64 26/198 2024-03-21T10:09:59.023 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : rbd-fuse-2:19.0.0-2231.g2edfee68.el9.x86_64 27/198 2024-03-21T10:09:59.038 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : rbd-mirror-2:19.0.0-2231.g2edfee68.el9.x86_64 28/198 2024-03-21T10:09:59.038 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : rbd-nbd-2:19.0.0-2231.g2edfee68.el9.x86_64 29/198 2024-03-21T10:09:59.038 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-2231.g2edfee68. 30/198 2024-03-21T10:09:59.038 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-2231.g2edfee68.el9.noa 31/198 2024-03-21T10:09:59.038 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-2231.g2edfee68.el9.n 32/198 2024-03-21T10:09:59.038 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-2231.g2ed 33/198 2024-03-21T10:09:59.038 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-2231.g2edfee68.el 34/198 2024-03-21T10:09:59.038 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : ceph-mgr-rook-2:19.0.0-2231.g2edfee68.el9.noarch 35/198 2024-03-21T10:09:59.038 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-2231.g2edfee68.e 36/198 2024-03-21T10:09:59.039 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : ceph-volume-2:19.0.0-2231.g2edfee68.el9.noarch 37/198 2024-03-21T10:09:59.039 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : cephadm-2:19.0.0-2231.g2edfee68.el9.noarch 38/198 2024-03-21T10:09:59.039 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : cephfs-top-2:19.0.0-2231.g2edfee68.el9.noarch 39/198 2024-03-21T10:09:59.039 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : automake-1.16.2-8.el9.noarch 40/198 2024-03-21T10:09:59.039 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : bison-3.7.4-5.el9.x86_64 41/198 2024-03-21T10:09:59.039 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 42/198 2024-03-21T10:09:59.039 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : flex-2.6.4-9.el9.x86_64 43/198 2024-03-21T10:09:59.039 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 44/198 2024-03-21T10:09:59.039 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 45/198 2024-03-21T10:09:59.039 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 46/198 2024-03-21T10:09:59.039 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 47/198 2024-03-21T10:09:59.039 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 48/198 2024-03-21T10:09:59.039 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 49/198 2024-03-21T10:09:59.039 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 50/198 2024-03-21T10:09:59.039 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 51/198 2024-03-21T10:09:59.040 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 52/198 2024-03-21T10:09:59.040 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 53/198 2024-03-21T10:09:59.040 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : lua-5.4.4-4.el9.x86_64 54/198 2024-03-21T10:09:59.040 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : oniguruma-6.9.6-1.el9.5.x86_64 55/198 2024-03-21T10:09:59.040 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 56/198 2024-03-21T10:09:59.040 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 57/198 2024-03-21T10:09:59.040 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : postgresql-13.11-1.el9.x86_64 58/198 2024-03-21T10:09:59.040 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : postgresql-contrib-13.11-1.el9.x86_64 59/198 2024-03-21T10:09:59.040 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : postgresql-private-libs-13.11-1.el9.x86_64 60/198 2024-03-21T10:09:59.040 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : postgresql-server-13.11-1.el9.x86_64 61/198 2024-03-21T10:09:59.040 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 62/198 2024-03-21T10:09:59.040 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 63/198 2024-03-21T10:09:59.040 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 64/198 2024-03-21T10:09:59.040 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 65/198 2024-03-21T10:09:59.041 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 66/198 2024-03-21T10:09:59.041 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 67/198 2024-03-21T10:09:59.041 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 68/198 2024-03-21T10:09:59.041 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 69/198 2024-03-21T10:09:59.041 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 70/198 2024-03-21T10:09:59.041 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 71/198 2024-03-21T10:09:59.041 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 72/198 2024-03-21T10:09:59.041 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 73/198 2024-03-21T10:09:59.041 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 74/198 2024-03-21T10:09:59.041 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 75/198 2024-03-21T10:09:59.041 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 76/198 2024-03-21T10:09:59.041 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 77/198 2024-03-21T10:09:59.041 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 78/198 2024-03-21T10:09:59.041 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 79/198 2024-03-21T10:09:59.041 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 80/198 2024-03-21T10:09:59.042 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 81/198 2024-03-21T10:09:59.042 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 82/198 2024-03-21T10:09:59.042 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 83/198 2024-03-21T10:09:59.042 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 84/198 2024-03-21T10:09:59.042 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : zlib-devel-1.2.11-40.el9.x86_64 85/198 2024-03-21T10:09:59.042 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 86/198 2024-03-21T10:09:59.042 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 87/198 2024-03-21T10:09:59.042 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 88/198 2024-03-21T10:09:59.042 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-ply-3.11-14.el9.noarch 89/198 2024-03-21T10:09:59.042 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 90/198 2024-03-21T10:09:59.042 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 91/198 2024-03-21T10:09:59.042 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : jq-1.6-16.el9.x86_64 92/198 2024-03-21T10:09:59.042 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 93/198 2024-03-21T10:09:59.042 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 94/198 2024-03-21T10:09:59.043 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 95/198 2024-03-21T10:09:59.043 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : elfutils-libelf-devel-0.190-2.el9.x86_64 96/198 2024-03-21T10:09:59.043 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 97/198 2024-03-21T10:09:59.043 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : openssl-devel-1:3.0.7-27.el9.x86_64 98/198 2024-03-21T10:09:59.043 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-devel-3.9.17-2.el9.x86_64 99/198 2024-03-21T10:09:59.043 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 100/198 2024-03-21T10:09:59.043 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : qatlib-23.11.0-1.el9.x86_64 101/198 2024-03-21T10:09:59.043 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : qatlib-service-23.11.0-1.el9.x86_64 102/198 2024-03-21T10:09:59.043 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : qatzip-libs-1.1.2-2.el9.x86_64 103/198 2024-03-21T10:09:59.043 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 104/198 2024-03-21T10:09:59.043 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 105/198 2024-03-21T10:09:59.043 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 106/198 2024-03-21T10:09:59.044 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 107/198 2024-03-21T10:09:59.044 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 108/198 2024-03-21T10:09:59.044 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 109/198 2024-03-21T10:09:59.044 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : libarrow-9.0.0-10.el9.x86_64 110/198 2024-03-21T10:09:59.044 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : libarrow-doc-9.0.0-10.el9.noarch 111/198 2024-03-21T10:09:59.044 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 112/198 2024-03-21T10:09:59.044 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 113/198 2024-03-21T10:09:59.044 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 114/198 2024-03-21T10:09:59.044 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : parquet-libs-9.0.0-10.el9.x86_64 115/198 2024-03-21T10:09:59.044 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 116/198 2024-03-21T10:09:59.044 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 117/198 2024-03-21T10:09:59.044 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 118/198 2024-03-21T10:09:59.044 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 119/198 2024-03-21T10:09:59.044 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 120/198 2024-03-21T10:09:59.044 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 121/198 2024-03-21T10:09:59.045 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 122/198 2024-03-21T10:09:59.045 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-colorama-0.4.4-12.el9.noarch 123/198 2024-03-21T10:09:59.045 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-google-auth-1:2.28.2-1.el9.noarch 124/198 2024-03-21T10:09:59.045 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 125/198 2024-03-21T10:09:59.045 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 126/198 2024-03-21T10:09:59.045 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 127/198 2024-03-21T10:09:59.045 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 128/198 2024-03-21T10:09:59.045 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 129/198 2024-03-21T10:09:59.045 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 130/198 2024-03-21T10:09:59.045 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 131/198 2024-03-21T10:09:59.045 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 132/198 2024-03-21T10:09:59.045 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 133/198 2024-03-21T10:09:59.045 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 134/198 2024-03-21T10:09:59.045 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 135/198 2024-03-21T10:09:59.045 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 136/198 2024-03-21T10:09:59.046 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 137/198 2024-03-21T10:09:59.046 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 138/198 2024-03-21T10:09:59.046 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 139/198 2024-03-21T10:09:59.046 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 140/198 2024-03-21T10:09:59.046 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 141/198 2024-03-21T10:09:59.046 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 142/198 2024-03-21T10:09:59.046 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 143/198 2024-03-21T10:09:59.046 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 144/198 2024-03-21T10:09:59.046 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 145/198 2024-03-21T10:09:59.046 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 146/198 2024-03-21T10:09:59.046 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 147/198 2024-03-21T10:09:59.046 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 148/198 2024-03-21T10:09:59.046 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : re2-1:20211101-3.el9.x86_64 149/198 2024-03-21T10:09:59.047 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 150/198 2024-03-21T10:09:59.047 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : librados2-2:19.0.0-2231.g2edfee68.el9.x86_64 151/198 2024-03-21T10:09:59.047 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 152/198 2024-03-21T10:09:59.047 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : librbd1-2:19.0.0-2231.g2edfee68.el9.x86_64 153/198 2024-03-21T10:09:59.047 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 154/198 2024-03-21T10:09:59.047 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : xfsdump-3.1.12-4.el9.x86_64 155/198 2024-03-21T10:09:59.047 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : xfsdump-3.1.12-3.el9.x86_64 156/198 2024-03-21T10:09:59.047 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : NetworkManager-1:1.47.2-1.el9.x86_64 157/198 2024-03-21T10:09:59.047 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : NetworkManager-1:1.44.0-2.el9.x86_64 158/198 2024-03-21T10:09:59.047 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : NetworkManager-libnm-1:1.47.2-1.el9.x86_64 159/198 2024-03-21T10:09:59.047 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : NetworkManager-libnm-1:1.44.0-2.el9.x86_64 160/198 2024-03-21T10:09:59.047 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : NetworkManager-team-1:1.47.2-1.el9.x86_64 161/198 2024-03-21T10:09:59.047 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : NetworkManager-team-1:1.44.0-2.el9.x86_64 162/198 2024-03-21T10:09:59.047 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : NetworkManager-tui-1:1.47.2-1.el9.x86_64 163/198 2024-03-21T10:09:59.047 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : NetworkManager-tui-1:1.44.0-2.el9.x86_64 164/198 2024-03-21T10:09:59.048 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : acl-2.3.1-4.el9.x86_64 165/198 2024-03-21T10:09:59.048 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : acl-2.3.1-3.el9.x86_64 166/198 2024-03-21T10:09:59.048 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : elfutils-debuginfod-client-0.190-2.el9.x86_64 167/198 2024-03-21T10:09:59.048 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : elfutils-debuginfod-client-0.189-3.el9.x86_64 168/198 2024-03-21T10:09:59.048 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : elfutils-libelf-0.190-2.el9.x86_64 169/198 2024-03-21T10:09:59.048 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : elfutils-libelf-0.189-3.el9.x86_64 170/198 2024-03-21T10:09:59.048 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : elfutils-libs-0.190-2.el9.x86_64 171/198 2024-03-21T10:09:59.048 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : elfutils-libs-0.189-3.el9.x86_64 172/198 2024-03-21T10:09:59.048 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : libacl-2.3.1-4.el9.x86_64 173/198 2024-03-21T10:09:59.048 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : libacl-2.3.1-3.el9.x86_64 174/198 2024-03-21T10:09:59.048 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : libblkid-2.37.4-18.el9.x86_64 175/198 2024-03-21T10:09:59.048 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : libblkid-2.37.4-15.el9.x86_64 176/198 2024-03-21T10:09:59.048 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : libfdisk-2.37.4-18.el9.x86_64 177/198 2024-03-21T10:09:59.048 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : libfdisk-2.37.4-15.el9.x86_64 178/198 2024-03-21T10:09:59.048 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : libmount-2.37.4-18.el9.x86_64 179/198 2024-03-21T10:09:59.048 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : libmount-2.37.4-15.el9.x86_64 180/198 2024-03-21T10:09:59.049 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : libsmartcols-2.37.4-18.el9.x86_64 181/198 2024-03-21T10:09:59.049 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : libsmartcols-2.37.4-15.el9.x86_64 182/198 2024-03-21T10:09:59.049 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : libuuid-2.37.4-18.el9.x86_64 183/198 2024-03-21T10:09:59.049 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : libuuid-2.37.4-15.el9.x86_64 184/198 2024-03-21T10:09:59.049 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 185/198 2024-03-21T10:09:59.049 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : openssl-1:3.0.7-24.el9.x86_64 186/198 2024-03-21T10:09:59.049 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 187/198 2024-03-21T10:09:59.049 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : openssl-libs-1:3.0.7-24.el9.x86_64 188/198 2024-03-21T10:09:59.049 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : util-linux-2.37.4-18.el9.x86_64 189/198 2024-03-21T10:09:59.049 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : util-linux-2.37.4-15.el9.x86_64 190/198 2024-03-21T10:09:59.049 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : util-linux-core-2.37.4-18.el9.x86_64 191/198 2024-03-21T10:09:59.049 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : util-linux-core-2.37.4-15.el9.x86_64 192/198 2024-03-21T10:09:59.049 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : util-linux-user-2.37.4-18.el9.x86_64 193/198 2024-03-21T10:09:59.049 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : util-linux-user-2.37.4-15.el9.x86_64 194/198 2024-03-21T10:09:59.049 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : xfsprogs-6.3.0-1.el9.x86_64 195/198 2024-03-21T10:09:59.050 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : xfsprogs-5.19.0-4.el9.x86_64 196/198 2024-03-21T10:09:59.050 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : libacl-devel-2.3.1-4.el9.x86_64 197/198 2024-03-21T10:10:01.484 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : libacl-devel-2.3.1-3.el9.x86_64 198/198 2024-03-21T10:10:01.484 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-03-21T10:10:01.484 INFO:teuthology.orchestra.run.smithi157.stdout:Upgraded: 2024-03-21T10:10:01.484 INFO:teuthology.orchestra.run.smithi157.stdout: NetworkManager-1:1.47.2-1.el9.x86_64 2024-03-21T10:10:01.484 INFO:teuthology.orchestra.run.smithi157.stdout: NetworkManager-libnm-1:1.47.2-1.el9.x86_64 2024-03-21T10:10:01.484 INFO:teuthology.orchestra.run.smithi157.stdout: NetworkManager-team-1:1.47.2-1.el9.x86_64 2024-03-21T10:10:01.484 INFO:teuthology.orchestra.run.smithi157.stdout: NetworkManager-tui-1:1.47.2-1.el9.x86_64 2024-03-21T10:10:01.484 INFO:teuthology.orchestra.run.smithi157.stdout: acl-2.3.1-4.el9.x86_64 2024-03-21T10:10:01.484 INFO:teuthology.orchestra.run.smithi157.stdout: elfutils-debuginfod-client-0.190-2.el9.x86_64 2024-03-21T10:10:01.484 INFO:teuthology.orchestra.run.smithi157.stdout: elfutils-libelf-0.190-2.el9.x86_64 2024-03-21T10:10:01.485 INFO:teuthology.orchestra.run.smithi157.stdout: elfutils-libs-0.190-2.el9.x86_64 2024-03-21T10:10:01.485 INFO:teuthology.orchestra.run.smithi157.stdout: libacl-2.3.1-4.el9.x86_64 2024-03-21T10:10:01.485 INFO:teuthology.orchestra.run.smithi157.stdout: libacl-devel-2.3.1-4.el9.x86_64 2024-03-21T10:10:01.485 INFO:teuthology.orchestra.run.smithi157.stdout: libblkid-2.37.4-18.el9.x86_64 2024-03-21T10:10:01.485 INFO:teuthology.orchestra.run.smithi157.stdout: libfdisk-2.37.4-18.el9.x86_64 2024-03-21T10:10:01.485 INFO:teuthology.orchestra.run.smithi157.stdout: libmount-2.37.4-18.el9.x86_64 2024-03-21T10:10:01.485 INFO:teuthology.orchestra.run.smithi157.stdout: librados2-2:19.0.0-2231.g2edfee68.el9.x86_64 2024-03-21T10:10:01.485 INFO:teuthology.orchestra.run.smithi157.stdout: librbd1-2:19.0.0-2231.g2edfee68.el9.x86_64 2024-03-21T10:10:01.485 INFO:teuthology.orchestra.run.smithi157.stdout: libsmartcols-2.37.4-18.el9.x86_64 2024-03-21T10:10:01.485 INFO:teuthology.orchestra.run.smithi157.stdout: libuuid-2.37.4-18.el9.x86_64 2024-03-21T10:10:01.485 INFO:teuthology.orchestra.run.smithi157.stdout: openssl-1:3.0.7-27.el9.x86_64 2024-03-21T10:10:01.485 INFO:teuthology.orchestra.run.smithi157.stdout: openssl-libs-1:3.0.7-27.el9.x86_64 2024-03-21T10:10:01.486 INFO:teuthology.orchestra.run.smithi157.stdout: util-linux-2.37.4-18.el9.x86_64 2024-03-21T10:10:01.486 INFO:teuthology.orchestra.run.smithi157.stdout: util-linux-core-2.37.4-18.el9.x86_64 2024-03-21T10:10:01.486 INFO:teuthology.orchestra.run.smithi157.stdout: util-linux-user-2.37.4-18.el9.x86_64 2024-03-21T10:10:01.486 INFO:teuthology.orchestra.run.smithi157.stdout: xfsdump-3.1.12-4.el9.x86_64 2024-03-21T10:10:01.486 INFO:teuthology.orchestra.run.smithi157.stdout: xfsprogs-6.3.0-1.el9.x86_64 2024-03-21T10:10:01.486 INFO:teuthology.orchestra.run.smithi157.stdout:Installed: 2024-03-21T10:10:01.486 INFO:teuthology.orchestra.run.smithi157.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-03-21T10:10:01.486 INFO:teuthology.orchestra.run.smithi157.stdout: automake-1.16.2-8.el9.noarch 2024-03-21T10:10:01.486 INFO:teuthology.orchestra.run.smithi157.stdout: bison-3.7.4-5.el9.x86_64 2024-03-21T10:10:01.486 INFO:teuthology.orchestra.run.smithi157.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-03-21T10:10:01.486 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-2:19.0.0-2231.g2edfee68.el9.x86_64 2024-03-21T10:10:01.486 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-base-2:19.0.0-2231.g2edfee68.el9.x86_64 2024-03-21T10:10:01.486 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-common-2:19.0.0-2231.g2edfee68.el9.x86_64 2024-03-21T10:10:01.487 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-fuse-2:19.0.0-2231.g2edfee68.el9.x86_64 2024-03-21T10:10:01.487 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-grafana-dashboards-2:19.0.0-2231.g2edfee68.el9.noarch 2024-03-21T10:10:01.487 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-immutable-object-cache-2:19.0.0-2231.g2edfee68.el9.x86_64 2024-03-21T10:10:01.487 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-mds-2:19.0.0-2231.g2edfee68.el9.x86_64 2024-03-21T10:10:01.487 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-mgr-2:19.0.0-2231.g2edfee68.el9.x86_64 2024-03-21T10:10:01.487 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-mgr-cephadm-2:19.0.0-2231.g2edfee68.el9.noarch 2024-03-21T10:10:01.487 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-mgr-dashboard-2:19.0.0-2231.g2edfee68.el9.noarch 2024-03-21T10:10:01.487 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-mgr-diskprediction-local-2:19.0.0-2231.g2edfee68.el9.noarch 2024-03-21T10:10:01.487 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-mgr-modules-core-2:19.0.0-2231.g2edfee68.el9.noarch 2024-03-21T10:10:01.487 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-mgr-rook-2:19.0.0-2231.g2edfee68.el9.noarch 2024-03-21T10:10:01.487 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-mon-2:19.0.0-2231.g2edfee68.el9.x86_64 2024-03-21T10:10:01.487 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-osd-2:19.0.0-2231.g2edfee68.el9.x86_64 2024-03-21T10:10:01.487 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-prometheus-alerts-2:19.0.0-2231.g2edfee68.el9.noarch 2024-03-21T10:10:01.488 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-radosgw-2:19.0.0-2231.g2edfee68.el9.x86_64 2024-03-21T10:10:01.488 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-selinux-2:19.0.0-2231.g2edfee68.el9.x86_64 2024-03-21T10:10:01.488 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-test-2:19.0.0-2231.g2edfee68.el9.x86_64 2024-03-21T10:10:01.488 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-volume-2:19.0.0-2231.g2edfee68.el9.noarch 2024-03-21T10:10:01.488 INFO:teuthology.orchestra.run.smithi157.stdout: cephadm-2:19.0.0-2231.g2edfee68.el9.noarch 2024-03-21T10:10:01.488 INFO:teuthology.orchestra.run.smithi157.stdout: cephfs-mirror-2:19.0.0-2231.g2edfee68.el9.x86_64 2024-03-21T10:10:01.488 INFO:teuthology.orchestra.run.smithi157.stdout: cephfs-shell-2:19.0.0-2231.g2edfee68.el9.x86_64 2024-03-21T10:10:01.488 INFO:teuthology.orchestra.run.smithi157.stdout: cephfs-top-2:19.0.0-2231.g2edfee68.el9.noarch 2024-03-21T10:10:01.488 INFO:teuthology.orchestra.run.smithi157.stdout: elfutils-libelf-devel-0.190-2.el9.x86_64 2024-03-21T10:10:01.488 INFO:teuthology.orchestra.run.smithi157.stdout: flex-2.6.4-9.el9.x86_64 2024-03-21T10:10:01.488 INFO:teuthology.orchestra.run.smithi157.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-03-21T10:10:01.488 INFO:teuthology.orchestra.run.smithi157.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-03-21T10:10:01.489 INFO:teuthology.orchestra.run.smithi157.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-03-21T10:10:01.489 INFO:teuthology.orchestra.run.smithi157.stdout: fuse-2.9.9-15.el9.x86_64 2024-03-21T10:10:01.489 INFO:teuthology.orchestra.run.smithi157.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-03-21T10:10:01.489 INFO:teuthology.orchestra.run.smithi157.stdout: grpc-data-1.46.7-10.el9.noarch 2024-03-21T10:10:01.489 INFO:teuthology.orchestra.run.smithi157.stdout: jq-1.6-16.el9.x86_64 2024-03-21T10:10:01.489 INFO:teuthology.orchestra.run.smithi157.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-03-21T10:10:01.489 INFO:teuthology.orchestra.run.smithi157.stdout: libarrow-9.0.0-10.el9.x86_64 2024-03-21T10:10:01.489 INFO:teuthology.orchestra.run.smithi157.stdout: libarrow-doc-9.0.0-10.el9.noarch 2024-03-21T10:10:01.489 INFO:teuthology.orchestra.run.smithi157.stdout: libcephfs-devel-2:19.0.0-2231.g2edfee68.el9.x86_64 2024-03-21T10:10:01.489 INFO:teuthology.orchestra.run.smithi157.stdout: libcephfs2-2:19.0.0-2231.g2edfee68.el9.x86_64 2024-03-21T10:10:01.489 INFO:teuthology.orchestra.run.smithi157.stdout: libcephsqlite-2:19.0.0-2231.g2edfee68.el9.x86_64 2024-03-21T10:10:01.489 INFO:teuthology.orchestra.run.smithi157.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-03-21T10:10:01.489 INFO:teuthology.orchestra.run.smithi157.stdout: liboath-2.6.7-2.el9.x86_64 2024-03-21T10:10:01.490 INFO:teuthology.orchestra.run.smithi157.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-03-21T10:10:01.490 INFO:teuthology.orchestra.run.smithi157.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-03-21T10:10:01.490 INFO:teuthology.orchestra.run.smithi157.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-03-21T10:10:01.490 INFO:teuthology.orchestra.run.smithi157.stdout: librados-devel-2:19.0.0-2231.g2edfee68.el9.x86_64 2024-03-21T10:10:01.490 INFO:teuthology.orchestra.run.smithi157.stdout: libradosstriper1-2:19.0.0-2231.g2edfee68.el9.x86_64 2024-03-21T10:10:01.490 INFO:teuthology.orchestra.run.smithi157.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-03-21T10:10:01.490 INFO:teuthology.orchestra.run.smithi157.stdout: librgw2-2:19.0.0-2231.g2edfee68.el9.x86_64 2024-03-21T10:10:01.490 INFO:teuthology.orchestra.run.smithi157.stdout: libtool-2.4.6-45.el9.x86_64 2024-03-21T10:10:01.490 INFO:teuthology.orchestra.run.smithi157.stdout: libunwind-1.6.2-1.el9.x86_64 2024-03-21T10:10:01.490 INFO:teuthology.orchestra.run.smithi157.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-03-21T10:10:01.490 INFO:teuthology.orchestra.run.smithi157.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-03-21T10:10:01.490 INFO:teuthology.orchestra.run.smithi157.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-03-21T10:10:01.490 INFO:teuthology.orchestra.run.smithi157.stdout: lua-5.4.4-4.el9.x86_64 2024-03-21T10:10:01.491 INFO:teuthology.orchestra.run.smithi157.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-03-21T10:10:01.491 INFO:teuthology.orchestra.run.smithi157.stdout: luarocks-3.9.2-1.el9.noarch 2024-03-21T10:10:01.491 INFO:teuthology.orchestra.run.smithi157.stdout: oniguruma-6.9.6-1.el9.5.x86_64 2024-03-21T10:10:01.491 INFO:teuthology.orchestra.run.smithi157.stdout: openblas-0.3.21-2.el9.x86_64 2024-03-21T10:10:01.491 INFO:teuthology.orchestra.run.smithi157.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-03-21T10:10:01.491 INFO:teuthology.orchestra.run.smithi157.stdout: openssl-devel-1:3.0.7-27.el9.x86_64 2024-03-21T10:10:01.491 INFO:teuthology.orchestra.run.smithi157.stdout: parquet-libs-9.0.0-10.el9.x86_64 2024-03-21T10:10:01.491 INFO:teuthology.orchestra.run.smithi157.stdout: postgresql-13.11-1.el9.x86_64 2024-03-21T10:10:01.491 INFO:teuthology.orchestra.run.smithi157.stdout: postgresql-contrib-13.11-1.el9.x86_64 2024-03-21T10:10:01.491 INFO:teuthology.orchestra.run.smithi157.stdout: postgresql-private-libs-13.11-1.el9.x86_64 2024-03-21T10:10:01.491 INFO:teuthology.orchestra.run.smithi157.stdout: postgresql-server-13.11-1.el9.x86_64 2024-03-21T10:10:01.491 INFO:teuthology.orchestra.run.smithi157.stdout: protobuf-3.14.0-13.el9.x86_64 2024-03-21T10:10:01.491 INFO:teuthology.orchestra.run.smithi157.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-03-21T10:10:01.492 INFO:teuthology.orchestra.run.smithi157.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-03-21T10:10:01.492 INFO:teuthology.orchestra.run.smithi157.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-03-21T10:10:01.492 INFO:teuthology.orchestra.run.smithi157.stdout: python3-babel-2.9.1-2.el9.noarch 2024-03-21T10:10:01.492 INFO:teuthology.orchestra.run.smithi157.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-03-21T10:10:01.492 INFO:teuthology.orchestra.run.smithi157.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-03-21T10:10:01.492 INFO:teuthology.orchestra.run.smithi157.stdout: python3-ceph-argparse-2:19.0.0-2231.g2edfee68.el9.x86_64 2024-03-21T10:10:01.492 INFO:teuthology.orchestra.run.smithi157.stdout: python3-ceph-common-2:19.0.0-2231.g2edfee68.el9.x86_64 2024-03-21T10:10:01.492 INFO:teuthology.orchestra.run.smithi157.stdout: python3-cephfs-2:19.0.0-2231.g2edfee68.el9.x86_64 2024-03-21T10:10:01.492 INFO:teuthology.orchestra.run.smithi157.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-03-21T10:10:01.492 INFO:teuthology.orchestra.run.smithi157.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-03-21T10:10:01.492 INFO:teuthology.orchestra.run.smithi157.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-03-21T10:10:01.493 INFO:teuthology.orchestra.run.smithi157.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-03-21T10:10:01.493 INFO:teuthology.orchestra.run.smithi157.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-03-21T10:10:01.493 INFO:teuthology.orchestra.run.smithi157.stdout: python3-colorama-0.4.4-12.el9.noarch 2024-03-21T10:10:01.493 INFO:teuthology.orchestra.run.smithi157.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-03-21T10:10:01.493 INFO:teuthology.orchestra.run.smithi157.stdout: python3-devel-3.9.17-2.el9.x86_64 2024-03-21T10:10:01.493 INFO:teuthology.orchestra.run.smithi157.stdout: python3-google-auth-1:2.28.2-1.el9.noarch 2024-03-21T10:10:01.493 INFO:teuthology.orchestra.run.smithi157.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-03-21T10:10:01.493 INFO:teuthology.orchestra.run.smithi157.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-03-21T10:10:01.493 INFO:teuthology.orchestra.run.smithi157.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-03-21T10:10:01.493 INFO:teuthology.orchestra.run.smithi157.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-03-21T10:10:01.493 INFO:teuthology.orchestra.run.smithi157.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-03-21T10:10:01.493 INFO:teuthology.orchestra.run.smithi157.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-03-21T10:10:01.494 INFO:teuthology.orchestra.run.smithi157.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-03-21T10:10:01.494 INFO:teuthology.orchestra.run.smithi157.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-03-21T10:10:01.494 INFO:teuthology.orchestra.run.smithi157.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-03-21T10:10:01.494 INFO:teuthology.orchestra.run.smithi157.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-03-21T10:10:01.494 INFO:teuthology.orchestra.run.smithi157.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-03-21T10:10:01.494 INFO:teuthology.orchestra.run.smithi157.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-03-21T10:10:01.494 INFO:teuthology.orchestra.run.smithi157.stdout: python3-mako-1.1.4-6.el9.noarch 2024-03-21T10:10:01.494 INFO:teuthology.orchestra.run.smithi157.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-03-21T10:10:01.494 INFO:teuthology.orchestra.run.smithi157.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-03-21T10:10:01.494 INFO:teuthology.orchestra.run.smithi157.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-03-21T10:10:01.494 INFO:teuthology.orchestra.run.smithi157.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-03-21T10:10:01.494 INFO:teuthology.orchestra.run.smithi157.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-03-21T10:10:01.494 INFO:teuthology.orchestra.run.smithi157.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-03-21T10:10:01.495 INFO:teuthology.orchestra.run.smithi157.stdout: python3-packaging-20.9-5.el9.noarch 2024-03-21T10:10:01.495 INFO:teuthology.orchestra.run.smithi157.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-03-21T10:10:01.495 INFO:teuthology.orchestra.run.smithi157.stdout: python3-ply-3.11-14.el9.noarch 2024-03-21T10:10:01.495 INFO:teuthology.orchestra.run.smithi157.stdout: python3-portend-3.1.0-2.el9.noarch 2024-03-21T10:10:01.495 INFO:teuthology.orchestra.run.smithi157.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-03-21T10:10:01.495 INFO:teuthology.orchestra.run.smithi157.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-03-21T10:10:01.495 INFO:teuthology.orchestra.run.smithi157.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-03-21T10:10:01.495 INFO:teuthology.orchestra.run.smithi157.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-03-21T10:10:01.495 INFO:teuthology.orchestra.run.smithi157.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-03-21T10:10:01.495 INFO:teuthology.orchestra.run.smithi157.stdout: python3-pycparser-2.20-6.el9.noarch 2024-03-21T10:10:01.495 INFO:teuthology.orchestra.run.smithi157.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-03-21T10:10:01.495 INFO:teuthology.orchestra.run.smithi157.stdout: python3-pytz-2021.1-5.el9.noarch 2024-03-21T10:10:01.495 INFO:teuthology.orchestra.run.smithi157.stdout: python3-rados-2:19.0.0-2231.g2edfee68.el9.x86_64 2024-03-21T10:10:01.495 INFO:teuthology.orchestra.run.smithi157.stdout: python3-rbd-2:19.0.0-2231.g2edfee68.el9.x86_64 2024-03-21T10:10:01.496 INFO:teuthology.orchestra.run.smithi157.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-03-21T10:10:01.496 INFO:teuthology.orchestra.run.smithi157.stdout: python3-requests-2.25.1-8.el9.noarch 2024-03-21T10:10:01.496 INFO:teuthology.orchestra.run.smithi157.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-03-21T10:10:01.496 INFO:teuthology.orchestra.run.smithi157.stdout: python3-rgw-2:19.0.0-2231.g2edfee68.el9.x86_64 2024-03-21T10:10:01.496 INFO:teuthology.orchestra.run.smithi157.stdout: python3-routes-2.5.1-5.el9.noarch 2024-03-21T10:10:01.496 INFO:teuthology.orchestra.run.smithi157.stdout: python3-rsa-4.9-2.el9.noarch 2024-03-21T10:10:01.496 INFO:teuthology.orchestra.run.smithi157.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-03-21T10:10:01.496 INFO:teuthology.orchestra.run.smithi157.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-03-21T10:10:01.496 INFO:teuthology.orchestra.run.smithi157.stdout: python3-toml-0.10.2-6.el9.noarch 2024-03-21T10:10:01.496 INFO:teuthology.orchestra.run.smithi157.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-03-21T10:10:01.496 INFO:teuthology.orchestra.run.smithi157.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-03-21T10:10:01.496 INFO:teuthology.orchestra.run.smithi157.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-03-21T10:10:01.497 INFO:teuthology.orchestra.run.smithi157.stdout: python3-webob-1.8.7-6.el9.noarch 2024-03-21T10:10:01.497 INFO:teuthology.orchestra.run.smithi157.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-03-21T10:10:01.497 INFO:teuthology.orchestra.run.smithi157.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-03-21T10:10:01.497 INFO:teuthology.orchestra.run.smithi157.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-03-21T10:10:01.497 INFO:teuthology.orchestra.run.smithi157.stdout: qatlib-23.11.0-1.el9.x86_64 2024-03-21T10:10:01.497 INFO:teuthology.orchestra.run.smithi157.stdout: qatlib-service-23.11.0-1.el9.x86_64 2024-03-21T10:10:01.497 INFO:teuthology.orchestra.run.smithi157.stdout: qatzip-libs-1.1.2-2.el9.x86_64 2024-03-21T10:10:01.497 INFO:teuthology.orchestra.run.smithi157.stdout: rbd-fuse-2:19.0.0-2231.g2edfee68.el9.x86_64 2024-03-21T10:10:01.497 INFO:teuthology.orchestra.run.smithi157.stdout: rbd-mirror-2:19.0.0-2231.g2edfee68.el9.x86_64 2024-03-21T10:10:01.497 INFO:teuthology.orchestra.run.smithi157.stdout: rbd-nbd-2:19.0.0-2231.g2edfee68.el9.x86_64 2024-03-21T10:10:01.497 INFO:teuthology.orchestra.run.smithi157.stdout: re2-1:20211101-3.el9.x86_64 2024-03-21T10:10:01.497 INFO:teuthology.orchestra.run.smithi157.stdout: socat-1.7.4.1-5.el9.x86_64 2024-03-21T10:10:01.498 INFO:teuthology.orchestra.run.smithi157.stdout: thrift-0.15.0-2.el9.x86_64 2024-03-21T10:10:01.498 INFO:teuthology.orchestra.run.smithi157.stdout: uuid-1.6.2-55.el9.x86_64 2024-03-21T10:10:01.498 INFO:teuthology.orchestra.run.smithi157.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-03-21T10:10:01.498 INFO:teuthology.orchestra.run.smithi157.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-03-21T10:10:01.498 INFO:teuthology.orchestra.run.smithi157.stdout: zlib-devel-1.2.11-40.el9.x86_64 2024-03-21T10:10:01.498 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-03-21T10:10:01.498 INFO:teuthology.orchestra.run.smithi157.stdout:Complete! 2024-03-21T10:10:02.261 DEBUG:teuthology.parallel:result is None 2024-03-21T10:10:02.262 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=2edfee68ad7243bed422a049b6f5f58619300f11 2024-03-21T10:10:02.476 DEBUG:teuthology.orchestra.run.smithi047:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-03-21T10:10:02.553 INFO:teuthology.orchestra.run.smithi047.stdout:19.0.0-2231.g2edfee68.el9 2024-03-21T10:10:02.554 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-2231.g2edfee68.el9 2024-03-21T10:10:02.554 INFO:teuthology.task.install:The correct ceph version 19.0.0-2231.g2edfee68 is installed. 2024-03-21T10:10:02.556 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=2edfee68ad7243bed422a049b6f5f58619300f11 2024-03-21T10:10:02.720 DEBUG:teuthology.orchestra.run.smithi100:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-03-21T10:10:02.791 INFO:teuthology.orchestra.run.smithi100.stdout:19.0.0-2231.g2edfee68.el9 2024-03-21T10:10:02.791 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-2231.g2edfee68.el9 2024-03-21T10:10:02.791 INFO:teuthology.task.install:The correct ceph version 19.0.0-2231.g2edfee68 is installed. 2024-03-21T10:10:02.793 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=2edfee68ad7243bed422a049b6f5f58619300f11 2024-03-21T10:10:02.957 DEBUG:teuthology.orchestra.run.smithi157:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-03-21T10:10:03.023 INFO:teuthology.orchestra.run.smithi157.stdout:19.0.0-2231.g2edfee68.el9 2024-03-21T10:10:03.023 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-2231.g2edfee68.el9 2024-03-21T10:10:03.023 INFO:teuthology.task.install:The correct ceph version 19.0.0-2231.g2edfee68 is installed. 2024-03-21T10:10:03.026 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-03-21T10:10:03.026 DEBUG:teuthology.orchestra.run.smithi047:> set -ex 2024-03-21T10:10:03.026 DEBUG:teuthology.orchestra.run.smithi047:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-03-21T10:10:03.138 DEBUG:teuthology.orchestra.run.smithi100:> set -ex 2024-03-21T10:10:03.138 DEBUG:teuthology.orchestra.run.smithi100:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-03-21T10:10:03.240 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2024-03-21T10:10:03.241 DEBUG:teuthology.orchestra.run.smithi157:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-03-21T10:10:03.325 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-03-21T10:10:03.326 DEBUG:teuthology.orchestra.run.smithi047:> set -ex 2024-03-21T10:10:03.326 DEBUG:teuthology.orchestra.run.smithi047:> sudo dd of=/usr/bin/daemon-helper 2024-03-21T10:10:03.432 DEBUG:teuthology.orchestra.run.smithi047:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-03-21T10:10:03.573 DEBUG:teuthology.orchestra.run.smithi100:> set -ex 2024-03-21T10:10:03.573 DEBUG:teuthology.orchestra.run.smithi100:> sudo dd of=/usr/bin/daemon-helper 2024-03-21T10:10:03.674 DEBUG:teuthology.orchestra.run.smithi100:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-03-21T10:10:03.806 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2024-03-21T10:10:03.806 DEBUG:teuthology.orchestra.run.smithi157:> sudo dd of=/usr/bin/daemon-helper 2024-03-21T10:10:03.891 DEBUG:teuthology.orchestra.run.smithi157:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-03-21T10:10:04.008 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-03-21T10:10:04.008 DEBUG:teuthology.orchestra.run.smithi047:> set -ex 2024-03-21T10:10:04.008 DEBUG:teuthology.orchestra.run.smithi047:> sudo dd of=/usr/bin/adjust-ulimits 2024-03-21T10:10:04.110 DEBUG:teuthology.orchestra.run.smithi047:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-03-21T10:10:04.254 DEBUG:teuthology.orchestra.run.smithi100:> set -ex 2024-03-21T10:10:04.254 DEBUG:teuthology.orchestra.run.smithi100:> sudo dd of=/usr/bin/adjust-ulimits 2024-03-21T10:10:04.351 DEBUG:teuthology.orchestra.run.smithi100:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-03-21T10:10:04.488 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2024-03-21T10:10:04.488 DEBUG:teuthology.orchestra.run.smithi157:> sudo dd of=/usr/bin/adjust-ulimits 2024-03-21T10:10:04.557 DEBUG:teuthology.orchestra.run.smithi157:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-03-21T10:10:04.668 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-03-21T10:10:04.669 DEBUG:teuthology.orchestra.run.smithi047:> set -ex 2024-03-21T10:10:04.669 DEBUG:teuthology.orchestra.run.smithi047:> sudo dd of=/usr/bin/stdin-killer 2024-03-21T10:10:04.780 DEBUG:teuthology.orchestra.run.smithi047:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-03-21T10:10:04.881 DEBUG:teuthology.orchestra.run.smithi100:> set -ex 2024-03-21T10:10:04.881 DEBUG:teuthology.orchestra.run.smithi100:> sudo dd of=/usr/bin/stdin-killer 2024-03-21T10:10:04.977 DEBUG:teuthology.orchestra.run.smithi100:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-03-21T10:10:05.067 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2024-03-21T10:10:05.067 DEBUG:teuthology.orchestra.run.smithi157:> sudo dd of=/usr/bin/stdin-killer 2024-03-21T10:10:05.137 DEBUG:teuthology.orchestra.run.smithi157:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-03-21T10:10:05.248 INFO:teuthology.run_tasks:Running task cephadm... 2024-03-21T10:10:05.357 INFO:tasks.cephadm:Config: {'roleless': False, 'cephfs': {'ec_profile': ['m=2', 'k=2', 'crush-failure-domain=osd'], 'max_mds': 5, 'session_timeout': 300, 'standby_replay': True}, 'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds bal fragment size max': 10000, 'mds bal merge size': 5, 'mds bal split bits': 3, 'mds bal split size': 100, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'cephsqlite lock renewal timeout': 900000, 'debug cephsqlite': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120}, 'osd': {'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180, 'osd shutdown pgref assert': True, 'osd_max_omap_entries_per_request': 10000}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'MON_DOWN', 'slow metadata IO', 'SLOW_OPS', 'slow request'], 'sha1': '2edfee68ad7243bed422a049b6f5f58619300f11', 'subvols': {'create': 2, 'subvol_options': '--namespace-isolated'}} 2024-03-21T10:10:05.357 INFO:tasks.cephadm:Cluster image is quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2edfee68ad7243bed422a049b6f5f58619300f11 2024-03-21T10:10:05.357 INFO:tasks.cephadm:Cluster fsid is 3056d51a-e76b-11ee-95cd-87774f69a715 2024-03-21T10:10:05.358 INFO:tasks.cephadm:Choosing monitor IPs and ports... 2024-03-21T10:10:05.358 INFO:tasks.cephadm:Monitor IPs: {'mon.a': '172.21.15.47', 'mon.b': '172.21.15.100', 'mon.c': '172.21.15.157'} 2024-03-21T10:10:05.358 INFO:tasks.cephadm:First mon is mon.a on smithi047 2024-03-21T10:10:05.358 INFO:tasks.cephadm:First mgr is x 2024-03-21T10:10:05.358 INFO:tasks.cephadm:Normalizing hostnames... 2024-03-21T10:10:05.358 DEBUG:teuthology.orchestra.run.smithi047:> sudo hostname $(hostname -s) 2024-03-21T10:10:05.466 DEBUG:teuthology.orchestra.run.smithi100:> sudo hostname $(hostname -s) 2024-03-21T10:10:05.563 DEBUG:teuthology.orchestra.run.smithi157:> sudo hostname $(hostname -s) 2024-03-21T10:10:05.647 INFO:tasks.cephadm:Downloading "compiled" cephadm from cachra 2024-03-21T10:10:05.647 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=2edfee68ad7243bed422a049b6f5f58619300f11 2024-03-21T10:10:05.879 INFO:tasks.cephadm:builder_project result: [{'status': 'ready', 'sha1': '2edfee68ad7243bed422a049b6f5f58619300f11', 'extra': {'build_url': 'https://jenkins.ceph.com/job/ceph-dev-new-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic/77791/', 'root_build_cause': 'SCMTRIGGER', 'version': '19.0.0-2231-g2edfee68', 'node_name': '172.21.5.34+adami04', 'job_name': 'ceph-dev-new-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic', 'package_manager_version': '19.0.0-2231.g2edfee68'}, 'url': 'https://2.chacra.ceph.com/r/ceph/wip-vshankar-testing-20240320.053002/2edfee68ad7243bed422a049b6f5f58619300f11/centos/9/flavors/default/', 'distro_codename': None, 'modified': '2024-03-20 07:25:08.579979', 'distro_version': '9', 'project': 'ceph', 'flavor': 'default', 'ref': 'wip-vshankar-testing-20240320.053002', 'chacra_url': 'https://2.chacra.ceph.com/repos/ceph/wip-vshankar-testing-20240320.053002/2edfee68ad7243bed422a049b6f5f58619300f11/centos/9/flavors/default/', 'archs': ['x86_64', 'source'], 'distro': 'centos'}] 2024-03-21T10:10:06.082 INFO:tasks.util.chacra:got chacra host 2.chacra.ceph.com, ref wip-vshankar-testing-20240320.053002, sha1 2edfee68ad7243bed422a049b6f5f58619300f11 from https://shaman.ceph.com/api/search/?project=ceph&distros=centos%2F9%2Fx86_64&flavor=default&sha1=2edfee68ad7243bed422a049b6f5f58619300f11 2024-03-21T10:10:06.084 INFO:tasks.cephadm:Discovered cachra url: https://2.chacra.ceph.com/binaries/ceph/wip-vshankar-testing-20240320.053002/2edfee68ad7243bed422a049b6f5f58619300f11/centos/9/x86_64/flavors/default/cephadm 2024-03-21T10:10:06.084 DEBUG:teuthology.orchestra.run.smithi047:> curl --silent -L https://2.chacra.ceph.com/binaries/ceph/wip-vshankar-testing-20240320.053002/2edfee68ad7243bed422a049b6f5f58619300f11/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-03-21T10:10:06.802 INFO:teuthology.orchestra.run.smithi047.stdout:-rw-r--r--. 1 ubuntu ubuntu 769188 Mar 21 10:10 /home/ubuntu/cephtest/cephadm 2024-03-21T10:10:06.803 DEBUG:teuthology.orchestra.run.smithi100:> curl --silent -L https://2.chacra.ceph.com/binaries/ceph/wip-vshankar-testing-20240320.053002/2edfee68ad7243bed422a049b6f5f58619300f11/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-03-21T10:10:07.793 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r--. 1 ubuntu ubuntu 769188 Mar 21 10:10 /home/ubuntu/cephtest/cephadm 2024-03-21T10:10:07.794 DEBUG:teuthology.orchestra.run.smithi157:> curl --silent -L https://2.chacra.ceph.com/binaries/ceph/wip-vshankar-testing-20240320.053002/2edfee68ad7243bed422a049b6f5f58619300f11/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-03-21T10:10:09.054 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r--. 1 ubuntu ubuntu 769188 Mar 21 10:10 /home/ubuntu/cephtest/cephadm 2024-03-21T10:10:09.055 DEBUG:teuthology.orchestra.run.smithi047:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-03-21T10:10:09.119 DEBUG:teuthology.orchestra.run.smithi100:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-03-21T10:10:09.180 DEBUG:teuthology.orchestra.run.smithi157:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-03-21T10:10:09.278 INFO:tasks.cephadm:Pulling image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2edfee68ad7243bed422a049b6f5f58619300f11 on all hosts... 2024-03-21T10:10:09.278 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2edfee68ad7243bed422a049b6f5f58619300f11 pull 2024-03-21T10:10:09.286 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2edfee68ad7243bed422a049b6f5f58619300f11 pull 2024-03-21T10:10:09.293 DEBUG:teuthology.orchestra.run.smithi157:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2edfee68ad7243bed422a049b6f5f58619300f11 pull 2024-03-21T10:10:10.412 INFO:teuthology.orchestra.run.smithi100.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2edfee68ad7243bed422a049b6f5f58619300f11... 2024-03-21T10:10:10.426 INFO:teuthology.orchestra.run.smithi047.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2edfee68ad7243bed422a049b6f5f58619300f11... 2024-03-21T10:10:10.650 INFO:teuthology.orchestra.run.smithi157.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2edfee68ad7243bed422a049b6f5f58619300f11... 2024-03-21T10:10:49.761 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-03-21T10:10:49.762 INFO:teuthology.orchestra.run.smithi047.stdout: "ceph_version": "ceph version 19.0.0-2231-g2edfee68 (2edfee68ad7243bed422a049b6f5f58619300f11) squid (dev)", 2024-03-21T10:10:49.762 INFO:teuthology.orchestra.run.smithi047.stdout: "image_id": "237e43c7a65ff48534d94b97c1bfdbe4b7be5672e3d9a613745083ea3702aa54", 2024-03-21T10:10:49.762 INFO:teuthology.orchestra.run.smithi047.stdout: "repo_digests": [ 2024-03-21T10:10:49.762 INFO:teuthology.orchestra.run.smithi047.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232" 2024-03-21T10:10:49.762 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-03-21T10:10:49.762 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-03-21T10:10:50.135 INFO:teuthology.orchestra.run.smithi100.stdout:{ 2024-03-21T10:10:50.135 INFO:teuthology.orchestra.run.smithi100.stdout: "ceph_version": "ceph version 19.0.0-2231-g2edfee68 (2edfee68ad7243bed422a049b6f5f58619300f11) squid (dev)", 2024-03-21T10:10:50.136 INFO:teuthology.orchestra.run.smithi100.stdout: "image_id": "237e43c7a65ff48534d94b97c1bfdbe4b7be5672e3d9a613745083ea3702aa54", 2024-03-21T10:10:50.136 INFO:teuthology.orchestra.run.smithi100.stdout: "repo_digests": [ 2024-03-21T10:10:50.136 INFO:teuthology.orchestra.run.smithi100.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232" 2024-03-21T10:10:50.136 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-03-21T10:10:50.136 INFO:teuthology.orchestra.run.smithi100.stdout:} 2024-03-21T10:11:04.432 INFO:teuthology.orchestra.run.smithi157.stdout:{ 2024-03-21T10:11:04.432 INFO:teuthology.orchestra.run.smithi157.stdout: "ceph_version": "ceph version 19.0.0-2231-g2edfee68 (2edfee68ad7243bed422a049b6f5f58619300f11) squid (dev)", 2024-03-21T10:11:04.432 INFO:teuthology.orchestra.run.smithi157.stdout: "image_id": "237e43c7a65ff48534d94b97c1bfdbe4b7be5672e3d9a613745083ea3702aa54", 2024-03-21T10:11:04.432 INFO:teuthology.orchestra.run.smithi157.stdout: "repo_digests": [ 2024-03-21T10:11:04.432 INFO:teuthology.orchestra.run.smithi157.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232" 2024-03-21T10:11:04.432 INFO:teuthology.orchestra.run.smithi157.stdout: ] 2024-03-21T10:11:04.432 INFO:teuthology.orchestra.run.smithi157.stdout:} 2024-03-21T10:11:04.472 DEBUG:teuthology.orchestra.run.smithi047:> sudo mkdir -p /etc/ceph 2024-03-21T10:11:04.580 DEBUG:teuthology.orchestra.run.smithi100:> sudo mkdir -p /etc/ceph 2024-03-21T10:11:04.684 DEBUG:teuthology.orchestra.run.smithi157:> sudo mkdir -p /etc/ceph 2024-03-21T10:11:04.782 DEBUG:teuthology.orchestra.run.smithi047:> sudo chmod 777 /etc/ceph 2024-03-21T10:11:04.882 DEBUG:teuthology.orchestra.run.smithi100:> sudo chmod 777 /etc/ceph 2024-03-21T10:11:04.979 DEBUG:teuthology.orchestra.run.smithi157:> sudo chmod 777 /etc/ceph 2024-03-21T10:11:05.075 INFO:tasks.cephadm:Writing seed config... 2024-03-21T10:11:05.076 INFO:tasks.cephadm: override: [client] client mount timeout = 600 2024-03-21T10:11:05.076 INFO:tasks.cephadm: override: [client] debug client = 20 2024-03-21T10:11:05.076 INFO:tasks.cephadm: override: [client] debug ms = 1 2024-03-21T10:11:05.076 INFO:tasks.cephadm: override: [client] rados mon op timeout = 900 2024-03-21T10:11:05.076 INFO:tasks.cephadm: override: [client] rados osd op timeout = 900 2024-03-21T10:11:05.076 INFO:tasks.cephadm: override: [mds] debug mds = 20 2024-03-21T10:11:05.077 INFO:tasks.cephadm: override: [mds] debug mds balancer = 20 2024-03-21T10:11:05.077 INFO:tasks.cephadm: override: [mds] debug ms = 1 2024-03-21T10:11:05.077 INFO:tasks.cephadm: override: [mds] mds bal fragment size max = 10000 2024-03-21T10:11:05.077 INFO:tasks.cephadm: override: [mds] mds bal merge size = 5 2024-03-21T10:11:05.077 INFO:tasks.cephadm: override: [mds] mds bal split bits = 3 2024-03-21T10:11:05.077 INFO:tasks.cephadm: override: [mds] mds bal split size = 100 2024-03-21T10:11:05.077 INFO:tasks.cephadm: override: [mds] mds debug frag = True 2024-03-21T10:11:05.077 INFO:tasks.cephadm: override: [mds] mds debug scatterstat = True 2024-03-21T10:11:05.077 INFO:tasks.cephadm: override: [mds] mds op complaint time = 180 2024-03-21T10:11:05.077 INFO:tasks.cephadm: override: [mds] mds verify scatter = True 2024-03-21T10:11:05.077 INFO:tasks.cephadm: override: [mds] osd op complaint time = 180 2024-03-21T10:11:05.077 INFO:tasks.cephadm: override: [mds] rados mon op timeout = 900 2024-03-21T10:11:05.077 INFO:tasks.cephadm: override: [mds] rados osd op timeout = 900 2024-03-21T10:11:05.078 INFO:tasks.cephadm: override: [mgr] cephsqlite lock renewal timeout = 900000 2024-03-21T10:11:05.078 INFO:tasks.cephadm: override: [mgr] debug cephsqlite = 20 2024-03-21T10:11:05.078 INFO:tasks.cephadm: override: [mgr] debug mgr = 20 2024-03-21T10:11:05.078 INFO:tasks.cephadm: override: [mgr] debug ms = 1 2024-03-21T10:11:05.078 INFO:tasks.cephadm: override: [mon] debug mon = 20 2024-03-21T10:11:05.078 INFO:tasks.cephadm: override: [mon] debug ms = 1 2024-03-21T10:11:05.078 INFO:tasks.cephadm: override: [mon] debug paxos = 20 2024-03-21T10:11:05.078 INFO:tasks.cephadm: override: [mon] mon op complaint time = 120 2024-03-21T10:11:05.078 INFO:tasks.cephadm: override: [osd] bluestore block size = 96636764160 2024-03-21T10:11:05.078 INFO:tasks.cephadm: override: [osd] bluestore fsck on mount = True 2024-03-21T10:11:05.078 INFO:tasks.cephadm: override: [osd] debug bluefs = 20 2024-03-21T10:11:05.078 INFO:tasks.cephadm: override: [osd] debug bluestore = 20 2024-03-21T10:11:05.078 INFO:tasks.cephadm: override: [osd] debug ms = 1 2024-03-21T10:11:05.078 INFO:tasks.cephadm: override: [osd] debug osd = 20 2024-03-21T10:11:05.079 INFO:tasks.cephadm: override: [osd] debug rocksdb = 10 2024-03-21T10:11:05.079 INFO:tasks.cephadm: override: [osd] mon osd backfillfull_ratio = 0.85 2024-03-21T10:11:05.079 INFO:tasks.cephadm: override: [osd] mon osd full ratio = 0.9 2024-03-21T10:11:05.079 INFO:tasks.cephadm: override: [osd] mon osd nearfull ratio = 0.8 2024-03-21T10:11:05.079 INFO:tasks.cephadm: override: [osd] osd failsafe full ratio = 0.95 2024-03-21T10:11:05.079 INFO:tasks.cephadm: override: [osd] osd objectstore = bluestore 2024-03-21T10:11:05.079 INFO:tasks.cephadm: override: [osd] osd op complaint time = 180 2024-03-21T10:11:05.079 INFO:tasks.cephadm: override: [osd] osd shutdown pgref assert = True 2024-03-21T10:11:05.079 INFO:tasks.cephadm: override: [osd] osd_max_omap_entries_per_request = 10000 2024-03-21T10:11:05.080 DEBUG:teuthology.orchestra.run.smithi047:> set -ex 2024-03-21T10:11:05.080 DEBUG:teuthology.orchestra.run.smithi047:> dd of=/home/ubuntu/cephtest/seed.ceph.conf 2024-03-21T10:11:05.138 DEBUG:tasks.cephadm:Final config: [global] # make logging friendly to teuthology log_to_file = true log_to_stderr = false log to journald = false mon cluster log to file = true mon cluster log file level = debug mon clock drift allowed = 1.000 # replicate across OSDs, not hosts osd crush chooseleaf type = 0 #osd pool default size = 2 osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd # enable some debugging auth debug = true ms die on old message = true ms die on bug = true debug asserts on shutdown = true # adjust warnings mon max pg per osd = 10000# >= luminous mon pg warn max object skew = 0 mon osd allow primary affinity = true mon osd allow pg remap = true mon warn on legacy crush tunables = false mon warn on crush straw calc version zero = false mon warn on no sortbitwise = false mon warn on osd down out interval zero = false mon warn on too few osds = false mon_warn_on_pool_pg_num_not_power_of_two = false # disable pg_autoscaler by default for new pools osd_pool_default_pg_autoscale_mode = off # tests delete pools mon allow pool delete = true fsid = 3056d51a-e76b-11ee-95cd-87774f69a715 [osd] osd scrub load threshold = 5.0 osd scrub max interval = 600 osd mclock profile = high_recovery_ops osd recover clone overlap = true osd recovery max chunk = 1048576 osd deep scrub update digest min age = 30 osd map max advance = 10 osd memory target autotune = true # debugging osd debug shutdown = true osd debug op order = true osd debug verify stray on activate = true osd debug pg log writeout = true osd debug verify cached snaps = true osd debug verify missing on start = true osd debug misdirected ops = true osd op queue = debug_random osd op queue cut off = debug_random osd shutdown pgref assert = True bdev debug aio = true osd sloppy crc = true bluestore block size = 96636764160 bluestore fsck on mount = True debug bluefs = 20 debug bluestore = 20 debug ms = 1 debug osd = 20 debug rocksdb = 10 mon osd backfillfull_ratio = 0.85 mon osd full ratio = 0.9 mon osd nearfull ratio = 0.8 osd failsafe full ratio = 0.95 osd objectstore = bluestore osd op complaint time = 180 osd_max_omap_entries_per_request = 10000 [mgr] mon reweight min pgs per osd = 4 mon reweight min bytes per osd = 10 mgr/telemetry/nag = false cephsqlite lock renewal timeout = 900000 debug cephsqlite = 20 debug mgr = 20 debug ms = 1 [mon] mon data avail warn = 5 mon mgr mkfs grace = 240 mon reweight min pgs per osd = 4 mon osd reporter subtree level = osd mon osd prime pg temp = true mon reweight min bytes per osd = 10 # rotate auth tickets quickly to exercise renewal paths auth mon ticket ttl = 660# 11m auth service ticket ttl = 240# 4m # don't complain about global id reclaim mon_warn_on_insecure_global_id_reclaim = false mon_warn_on_insecure_global_id_reclaim_allowed = false debug mon = 20 debug ms = 1 debug paxos = 20 mon op complaint time = 120 [client.rgw] rgw cache enabled = true rgw enable ops log = true rgw enable usage log = true [client] client mount timeout = 600 debug client = 20 debug ms = 1 rados mon op timeout = 900 rados osd op timeout = 900 [mds] debug mds = 20 debug mds balancer = 20 debug ms = 1 mds bal fragment size max = 10000 mds bal merge size = 5 mds bal split bits = 3 mds bal split size = 100 mds debug frag = True mds debug scatterstat = True mds op complaint time = 180 mds verify scatter = True osd op complaint time = 180 rados mon op timeout = 900 rados osd op timeout = 900 2024-03-21T10:11:05.138 DEBUG:teuthology.orchestra.run.smithi047:mon.a> sudo journalctl -f -n 0 -u ceph-3056d51a-e76b-11ee-95cd-87774f69a715@mon.a.service 2024-03-21T10:11:05.146 DEBUG:teuthology.orchestra.run.smithi047:mgr.x> sudo journalctl -f -n 0 -u ceph-3056d51a-e76b-11ee-95cd-87774f69a715@mgr.x.service 2024-03-21T10:11:05.191 INFO:tasks.cephadm:Bootstrapping... 2024-03-21T10:11:05.191 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2edfee68ad7243bed422a049b6f5f58619300f11 -v bootstrap --fsid 3056d51a-e76b-11ee-95cd-87774f69a715 --config /home/ubuntu/cephtest/seed.ceph.conf --output-config /etc/ceph/ceph.conf --output-keyring /etc/ceph/ceph.client.admin.keyring --output-pub-ssh-key /home/ubuntu/cephtest/ceph.pub --mon-id a --mgr-id x --orphan-initial-daemons --skip-monitoring-stack --mon-ip 172.21.15.47 --skip-admin-label && sudo chmod +r /etc/ceph/ceph.client.admin.keyring 2024-03-21T10:11:05.483 INFO:teuthology.orchestra.run.smithi047.stdout:-------------------------------------------------------------------------------- 2024-03-21T10:11:05.483 INFO:teuthology.orchestra.run.smithi047.stdout:cephadm ['--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2edfee68ad7243bed422a049b6f5f58619300f11', '-v', 'bootstrap', '--fsid', '3056d51a-e76b-11ee-95cd-87774f69a715', '--config', '/home/ubuntu/cephtest/seed.ceph.conf', '--output-config', '/etc/ceph/ceph.conf', '--output-keyring', '/etc/ceph/ceph.client.admin.keyring', '--output-pub-ssh-key', '/home/ubuntu/cephtest/ceph.pub', '--mon-id', 'a', '--mgr-id', 'x', '--orphan-initial-daemons', '--skip-monitoring-stack', '--mon-ip', '172.21.15.47', '--skip-admin-label'] 2024-03-21T10:11:05.483 INFO:teuthology.orchestra.run.smithi047.stderr:Specifying an fsid for your cluster offers no advantages and may increase the likelihood of fsid conflicts. 2024-03-21T10:11:05.483 INFO:teuthology.orchestra.run.smithi047.stdout:Verifying podman|docker is present... 2024-03-21T10:11:05.541 INFO:teuthology.orchestra.run.smithi047.stdout:/bin/podman: stdout 4.9.3-dev 2024-03-21T10:11:05.541 INFO:teuthology.orchestra.run.smithi047.stdout:Verifying lvm2 is present... 2024-03-21T10:11:05.541 INFO:teuthology.orchestra.run.smithi047.stdout:Verifying time synchronization is in place... 2024-03-21T10:11:05.558 INFO:teuthology.orchestra.run.smithi047.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-03-21T10:11:05.559 INFO:teuthology.orchestra.run.smithi047.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-03-21T10:11:05.575 INFO:teuthology.orchestra.run.smithi047.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-03-21T10:11:05.575 INFO:teuthology.orchestra.run.smithi047.stdout:systemctl: stdout inactive 2024-03-21T10:11:05.600 INFO:teuthology.orchestra.run.smithi047.stdout:systemctl: stdout enabled 2024-03-21T10:11:05.622 INFO:teuthology.orchestra.run.smithi047.stdout:systemctl: stdout active 2024-03-21T10:11:05.622 INFO:teuthology.orchestra.run.smithi047.stdout:Unit chronyd.service is enabled and running 2024-03-21T10:11:05.623 INFO:teuthology.orchestra.run.smithi047.stdout:Repeating the final host check... 2024-03-21T10:11:05.685 INFO:teuthology.orchestra.run.smithi047.stdout:/bin/podman: stdout 4.9.3-dev 2024-03-21T10:11:05.685 INFO:teuthology.orchestra.run.smithi047.stdout:podman (/bin/podman) version 4.9.3 is present 2024-03-21T10:11:05.685 INFO:teuthology.orchestra.run.smithi047.stdout:systemctl is present 2024-03-21T10:11:05.685 INFO:teuthology.orchestra.run.smithi047.stdout:lvcreate is present 2024-03-21T10:11:05.699 INFO:teuthology.orchestra.run.smithi047.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-03-21T10:11:05.699 INFO:teuthology.orchestra.run.smithi047.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-03-21T10:11:05.718 INFO:teuthology.orchestra.run.smithi047.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-03-21T10:11:05.719 INFO:teuthology.orchestra.run.smithi047.stdout:systemctl: stdout inactive 2024-03-21T10:11:05.738 INFO:teuthology.orchestra.run.smithi047.stdout:systemctl: stdout enabled 2024-03-21T10:11:05.758 INFO:teuthology.orchestra.run.smithi047.stdout:systemctl: stdout active 2024-03-21T10:11:05.759 INFO:teuthology.orchestra.run.smithi047.stdout:Unit chronyd.service is enabled and running 2024-03-21T10:11:05.759 INFO:teuthology.orchestra.run.smithi047.stdout:Host looks OK 2024-03-21T10:11:05.759 INFO:teuthology.orchestra.run.smithi047.stdout:Cluster fsid: 3056d51a-e76b-11ee-95cd-87774f69a715 2024-03-21T10:11:05.760 INFO:teuthology.orchestra.run.smithi047.stdout:Acquiring lock 140315914228688 on /run/cephadm/3056d51a-e76b-11ee-95cd-87774f69a715.lock 2024-03-21T10:11:05.760 INFO:teuthology.orchestra.run.smithi047.stdout:Lock 140315914228688 acquired on /run/cephadm/3056d51a-e76b-11ee-95cd-87774f69a715.lock 2024-03-21T10:11:05.762 INFO:teuthology.orchestra.run.smithi047.stdout:Verifying IP 172.21.15.47 port 3300 ... 2024-03-21T10:11:05.763 INFO:teuthology.orchestra.run.smithi047.stdout:Verifying IP 172.21.15.47 port 6789 ... 2024-03-21T10:11:05.763 INFO:teuthology.orchestra.run.smithi047.stdout:Base mon IP(s) is [172.21.15.47:3300, 172.21.15.47:6789], mon addrv is [v2:172.21.15.47:3300,v1:172.21.15.47:6789] 2024-03-21T10:11:05.779 INFO:teuthology.orchestra.run.smithi047.stdout:/sbin/ip: stdout default via 172.21.15.254 dev ens1f0 proto dhcp src 172.21.15.47 metric 100 2024-03-21T10:11:05.779 INFO:teuthology.orchestra.run.smithi047.stdout:/sbin/ip: stdout 172.21.0.0/20 dev ens1f0 proto kernel scope link src 172.21.15.47 metric 100 2024-03-21T10:11:05.792 INFO:teuthology.orchestra.run.smithi047.stdout:/sbin/ip: stdout ::1 dev lo proto kernel metric 256 pref medium 2024-03-21T10:11:05.792 INFO:teuthology.orchestra.run.smithi047.stdout:/sbin/ip: stdout fe80::/64 dev ens1f0 proto kernel metric 1024 pref medium 2024-03-21T10:11:05.792 INFO:teuthology.orchestra.run.smithi047.stdout:/sbin/ip: stdout default via fe80::327c:5e00:6487:71e0 dev ens1f0 proto ra metric 100 pref medium 2024-03-21T10:11:05.803 INFO:teuthology.orchestra.run.smithi047.stdout:/sbin/ip: stdout 1: lo: mtu 65536 state UNKNOWN qlen 1000 2024-03-21T10:11:05.804 INFO:teuthology.orchestra.run.smithi047.stdout:/sbin/ip: stdout inet6 ::1/128 scope host 2024-03-21T10:11:05.804 INFO:teuthology.orchestra.run.smithi047.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-03-21T10:11:05.804 INFO:teuthology.orchestra.run.smithi047.stdout:/sbin/ip: stdout 4: ens1f0: mtu 1500 state UP qlen 1000 2024-03-21T10:11:05.804 INFO:teuthology.orchestra.run.smithi047.stdout:/sbin/ip: stdout inet6 fe80::ec4:7aff:febd:148e/64 scope link noprefixroute 2024-03-21T10:11:05.804 INFO:teuthology.orchestra.run.smithi047.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-03-21T10:11:05.806 INFO:teuthology.orchestra.run.smithi047.stdout:Mon IP `172.21.15.47` is in CIDR network `172.21.0.0/20` 2024-03-21T10:11:05.806 INFO:teuthology.orchestra.run.smithi047.stdout:Mon IP `172.21.15.47` is in CIDR network `172.21.0.0/20` 2024-03-21T10:11:05.807 INFO:teuthology.orchestra.run.smithi047.stdout:Inferred mon public CIDR from local network configuration ['172.21.0.0/20', '172.21.0.0/20'] 2024-03-21T10:11:05.809 INFO:teuthology.orchestra.run.smithi047.stdout:Internal network (--cluster-network) has not been provided, OSD replication will default to the public_network 2024-03-21T10:11:05.812 INFO:teuthology.orchestra.run.smithi047.stdout:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2edfee68ad7243bed422a049b6f5f58619300f11... 2024-03-21T10:11:06.440 INFO:teuthology.orchestra.run.smithi047.stdout:/bin/podman: stdout 237e43c7a65ff48534d94b97c1bfdbe4b7be5672e3d9a613745083ea3702aa54 2024-03-21T10:11:06.441 INFO:teuthology.orchestra.run.smithi047.stdout:/bin/podman: stderr Trying to pull quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2edfee68ad7243bed422a049b6f5f58619300f11... 2024-03-21T10:11:06.441 INFO:teuthology.orchestra.run.smithi047.stdout:/bin/podman: stderr Getting image source signatures 2024-03-21T10:11:06.441 INFO:teuthology.orchestra.run.smithi047.stdout:/bin/podman: stderr Copying blob sha256:96a207f569d15e36f8e9e3b2feafd8af7b93861f24c2b60ada6c9667dd5405be 2024-03-21T10:11:06.441 INFO:teuthology.orchestra.run.smithi047.stdout:/bin/podman: stderr Copying blob sha256:9516588f908cdb14b5e4062a6063db2d9291b355c3f32e2a99864608a4a7b649 2024-03-21T10:11:06.442 INFO:teuthology.orchestra.run.smithi047.stdout:/bin/podman: stderr Copying config sha256:237e43c7a65ff48534d94b97c1bfdbe4b7be5672e3d9a613745083ea3702aa54 2024-03-21T10:11:06.442 INFO:teuthology.orchestra.run.smithi047.stdout:/bin/podman: stderr Writing manifest to image destination 2024-03-21T10:11:07.738 INFO:teuthology.orchestra.run.smithi047.stdout:ceph: stdout ceph version 19.0.0-2231-g2edfee68 (2edfee68ad7243bed422a049b6f5f58619300f11) squid (dev) 2024-03-21T10:11:07.738 INFO:teuthology.orchestra.run.smithi047.stdout:Ceph version: ceph version 19.0.0-2231-g2edfee68 (2edfee68ad7243bed422a049b6f5f58619300f11) squid (dev) 2024-03-21T10:11:07.738 INFO:teuthology.orchestra.run.smithi047.stdout:Extracting ceph user uid/gid from container image... 2024-03-21T10:11:08.882 INFO:teuthology.orchestra.run.smithi047.stdout:stat: stdout 167 167 2024-03-21T10:11:08.882 INFO:teuthology.orchestra.run.smithi047.stdout:Creating initial keys... 2024-03-21T10:11:10.015 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph-authtool: stdout AQC9B/xltzYOFRAA8yph8BaVvxeePoTp4LrtAQ== 2024-03-21T10:11:11.097 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph-authtool: stdout AQC+B/xlegGuGhAAAzmzggxLCTHYUNhOcrxU1w== 2024-03-21T10:11:12.360 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph-authtool: stdout AQC/B/xlXAvwIhAAld1Rm6Pf7aoR/cgwkh1gpQ== 2024-03-21T10:11:12.362 INFO:teuthology.orchestra.run.smithi047.stdout:Creating initial monmap... 2024-03-21T10:11:13.569 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: monmap file /tmp/monmap 2024-03-21T10:11:13.570 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/monmaptool: stdout setting min_mon_release = quincy 2024-03-21T10:11:13.570 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: set fsid to 3056d51a-e76b-11ee-95cd-87774f69a715 2024-03-21T10:11:13.570 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-03-21T10:11:13.570 INFO:teuthology.orchestra.run.smithi047.stdout:monmaptool for a [v2:172.21.15.47:3300,v1:172.21.15.47:6789] on /usr/bin/monmaptool: monmap file /tmp/monmap 2024-03-21T10:11:13.570 INFO:teuthology.orchestra.run.smithi047.stdout:setting min_mon_release = quincy 2024-03-21T10:11:13.570 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/monmaptool: set fsid to 3056d51a-e76b-11ee-95cd-87774f69a715 2024-03-21T10:11:13.570 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-03-21T10:11:13.571 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-21T10:11:13.571 INFO:teuthology.orchestra.run.smithi047.stdout:Creating mon... 2024-03-21T10:11:14.678 INFO:teuthology.orchestra.run.smithi047.stdout:create mon.a on 2024-03-21T10:11:15.074 INFO:teuthology.orchestra.run.smithi047.stdout:systemctl: stderr Removed "/etc/systemd/system/multi-user.target.wants/ceph.target". 2024-03-21T10:11:15.318 INFO:teuthology.orchestra.run.smithi047.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /etc/systemd/system/ceph.target. 2024-03-21T10:11:15.622 INFO:teuthology.orchestra.run.smithi047.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph-3056d51a-e76b-11ee-95cd-87774f69a715.target → /etc/systemd/system/ceph-3056d51a-e76b-11ee-95cd-87774f69a715.target. 2024-03-21T10:11:15.622 INFO:teuthology.orchestra.run.smithi047.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph.target.wants/ceph-3056d51a-e76b-11ee-95cd-87774f69a715.target → /etc/systemd/system/ceph-3056d51a-e76b-11ee-95cd-87774f69a715.target. 2024-03-21T10:11:16.023 INFO:teuthology.orchestra.run.smithi047.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-3056d51a-e76b-11ee-95cd-87774f69a715@mon.a 2024-03-21T10:11:16.023 INFO:teuthology.orchestra.run.smithi047.stdout:systemctl: stderr Failed to reset failed state of unit ceph-3056d51a-e76b-11ee-95cd-87774f69a715@mon.a.service: Unit ceph-3056d51a-e76b-11ee-95cd-87774f69a715@mon.a.service not loaded. 2024-03-21T10:11:16.329 INFO:teuthology.orchestra.run.smithi047.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-3056d51a-e76b-11ee-95cd-87774f69a715.target.wants/ceph-3056d51a-e76b-11ee-95cd-87774f69a715@mon.a.service → /etc/systemd/system/ceph-3056d51a-e76b-11ee-95cd-87774f69a715@.service. 2024-03-21T10:11:16.965 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:11:16 smithi047 podman[31150]: 2024-03-21 10:11:16.940354983 +0000 UTC m=+0.345464158 container start af0d1f022cee7a39b29672d1c55b8b994c852494e248268eeee96ffc08262a38 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2edfee68ad7243bed422a049b6f5f58619300f11, name=ceph-3056d51a-e76b-11ee-95cd-87774f69a715-mon-a, GIT_CLEAN=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, CEPH_POINT_RELEASE=, ceph=True, GIT_BRANCH=HEAD, org.label-schema.build-date=20240319) 2024-03-21T10:11:17.027 INFO:teuthology.orchestra.run.smithi047.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-03-21T10:11:17.027 INFO:teuthology.orchestra.run.smithi047.stdout:systemctl: stdout disabled 2024-03-21T10:11:17.042 INFO:teuthology.orchestra.run.smithi047.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-03-21T10:11:17.042 INFO:teuthology.orchestra.run.smithi047.stdout:systemctl: stdout inactive 2024-03-21T10:11:17.042 INFO:teuthology.orchestra.run.smithi047.stdout:firewalld.service is not enabled 2024-03-21T10:11:17.042 INFO:teuthology.orchestra.run.smithi047.stdout:Not possible to enable service . firewalld.service is not available 2024-03-21T10:11:17.043 INFO:teuthology.orchestra.run.smithi047.stdout:Waiting for mon to start... 2024-03-21T10:11:17.043 INFO:teuthology.orchestra.run.smithi047.stdout:Waiting for mon... 2024-03-21T10:11:17.358 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:11:16 smithi047 bash[31150]: af0d1f022cee7a39b29672d1c55b8b994c852494e248268eeee96ffc08262a38 2024-03-21T10:11:17.358 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:11:17 smithi047 systemd[1]: Started Ceph mon.a for 3056d51a-e76b-11ee-95cd-87774f69a715. 2024-03-21T10:11:17.358 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:11:17 smithi047 ceph-mon[31169]: mkfs 3056d51a-e76b-11ee-95cd-87774f69a715 2024-03-21T10:11:17.358 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:11:17 smithi047 ceph-mon[31169]: mon.a is new leader, mons a in quorum (ranks 0) 2024-03-21T10:11:18.370 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:11:18 smithi047 ceph-mon[31169]: mon.a is new leader, mons a in quorum (ranks 0) 2024-03-21T10:11:18.370 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:11:18 smithi047 ceph-mon[31169]: monmap e1: 1 mons at {a=[v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-03-21T10:11:18.370 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:11:18 smithi047 ceph-mon[31169]: fsmap 2024-03-21T10:11:18.370 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:11:18 smithi047 ceph-mon[31169]: osdmap e1: 0 total, 0 up, 0 in 2024-03-21T10:11:18.370 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:11:18 smithi047 ceph-mon[31169]: mgrmap e1: no daemons active 2024-03-21T10:11:18.370 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:11:18 smithi047 ceph-mon[31169]: from='client.? 172.21.15.47:0/750475052' entity='client.admin' cmd=[{"prefix": "status"}]: dispatch 2024-03-21T10:11:18.390 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout cluster: 2024-03-21T10:11:18.391 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout id: 3056d51a-e76b-11ee-95cd-87774f69a715 2024-03-21T10:11:18.391 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout health: HEALTH_OK 2024-03-21T10:11:18.391 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout 2024-03-21T10:11:18.392 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout services: 2024-03-21T10:11:18.392 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout mon: 1 daemons, quorum a (age 0.731466s) 2024-03-21T10:11:18.392 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout mgr: no daemons active 2024-03-21T10:11:18.392 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout osd: 0 osds: 0 up, 0 in 2024-03-21T10:11:18.392 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout 2024-03-21T10:11:18.393 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout data: 2024-03-21T10:11:18.393 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout pools: 0 pools, 0 pgs 2024-03-21T10:11:18.393 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout objects: 0 objects, 0 B 2024-03-21T10:11:18.393 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout usage: 0 B used, 0 B / 0 B avail 2024-03-21T10:11:18.393 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout pgs: 2024-03-21T10:11:18.393 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout 2024-03-21T10:11:18.395 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:17.654+0000 7f860f200700 1 Processor -- start 2024-03-21T10:11:18.395 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:17.654+0000 7f860f200700 1 -- start start 2024-03-21T10:11:18.395 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:17.655+0000 7f860f200700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f861007ce30 0x7f861007b2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:11:18.395 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:17.655+0000 7f860f200700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f861007b8b0 con 0x7f861007ce30 2024-03-21T10:11:18.395 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:17.656+0000 7f860de00700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f861007ce30 0x7f861007b2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:11:18.395 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:17.656+0000 7f860de00700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f861007ce30 0x7f861007b2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:43722/0 (socket says 172.21.15.47:43722) 2024-03-21T10:11:18.395 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:17.656+0000 7f860de00700 1 -- 172.21.15.47:0/3925899331 learned_addr learned my addr 172.21.15.47:0/3925899331 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:11:18.395 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:17.657+0000 7f860de00700 1 -- 172.21.15.47:0/3925899331 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f861007ba30 con 0x7f861007ce30 2024-03-21T10:11:18.395 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:17.657+0000 7f860de00700 1 --2- 172.21.15.47:0/3925899331 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f861007ce30 0x7f861007b2e0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f86000098a0 tx=0x7f8600009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b6b9fc1cfdc88001 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:11:18.395 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:17.657+0000 7f860d400700 1 -- 172.21.15.47:0/3925899331 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8600004160 con 0x7f861007ce30 2024-03-21T10:11:18.396 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:17.658+0000 7f860d400700 1 -- 172.21.15.47:0/3925899331 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8600004300 con 0x7f861007ce30 2024-03-21T10:11:18.396 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:17.658+0000 7f860d400700 1 -- 172.21.15.47:0/3925899331 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f86000036a0 con 0x7f861007ce30 2024-03-21T10:11:18.396 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:17.658+0000 7f860f200700 1 -- 172.21.15.47:0/3925899331 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f861007ce30 msgr2=0x7f861007b2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:11:18.396 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:17.658+0000 7f860f200700 1 --2- 172.21.15.47:0/3925899331 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f861007ce30 0x7f861007b2e0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f86000098a0 tx=0x7f8600009bb0 comp rx=0 tx=0).stop 2024-03-21T10:11:18.397 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:17.658+0000 7f860f200700 1 -- 172.21.15.47:0/3925899331 shutdown_connections 2024-03-21T10:11:18.397 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:17.658+0000 7f860f200700 1 --2- 172.21.15.47:0/3925899331 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f861007ce30 0x7f861007b2e0 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:11:18.397 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:17.658+0000 7f860f200700 1 -- 172.21.15.47:0/3925899331 >> 172.21.15.47:0/3925899331 conn(0x7f8610102cd0 msgr2=0x7f86101050f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:11:18.397 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:17.659+0000 7f860f200700 1 -- 172.21.15.47:0/3925899331 shutdown_connections 2024-03-21T10:11:18.398 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:17.659+0000 7f860f200700 1 -- 172.21.15.47:0/3925899331 wait complete. 2024-03-21T10:11:18.398 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:17.660+0000 7f860f200700 1 Processor -- start 2024-03-21T10:11:18.398 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:17.660+0000 7f860f200700 1 -- start start 2024-03-21T10:11:18.398 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:17.661+0000 7f860f200700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f861007ce30 0x7f861019cf20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:11:18.398 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:17.661+0000 7f860f200700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f861007bf60 con 0x7f861007ce30 2024-03-21T10:11:18.398 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:17.661+0000 7f860de00700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f861007ce30 0x7f861019cf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:11:18.399 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:17.661+0000 7f860de00700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f861007ce30 0x7f861019cf20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:43726/0 (socket says 172.21.15.47:43726) 2024-03-21T10:11:18.399 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:17.661+0000 7f860de00700 1 -- 172.21.15.47:0/750475052 learned_addr learned my addr 172.21.15.47:0/750475052 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:11:18.399 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:17.662+0000 7f860de00700 1 -- 172.21.15.47:0/750475052 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f861019d460 con 0x7f861007ce30 2024-03-21T10:11:18.399 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:17.662+0000 7f860de00700 1 --2- 172.21.15.47:0/750475052 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f861007ce30 0x7f861019cf20 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f8600009fb0 tx=0x7f8600004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:11:18.399 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:17.662+0000 7f8606a00700 1 -- 172.21.15.47:0/750475052 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8600003de0 con 0x7f861007ce30 2024-03-21T10:11:18.401 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:17.662+0000 7f8606a00700 1 -- 172.21.15.47:0/750475052 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8600011630 con 0x7f861007ce30 2024-03-21T10:11:18.401 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:17.662+0000 7f860f200700 1 -- 172.21.15.47:0/750475052 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f861019d6f0 con 0x7f861007ce30 2024-03-21T10:11:18.401 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:17.662+0000 7f8606a00700 1 -- 172.21.15.47:0/750475052 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8600011930 con 0x7f861007ce30 2024-03-21T10:11:18.401 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:17.663+0000 7f860f200700 1 -- 172.21.15.47:0/750475052 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f86101a03e0 con 0x7f861007ce30 2024-03-21T10:11:18.401 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:17.663+0000 7f8606a00700 1 -- 172.21.15.47:0/750475052 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (secure 0 0 0) 0x7f8600011b60 con 0x7f861007ce30 2024-03-21T10:11:18.401 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:17.663+0000 7f8606a00700 1 -- 172.21.15.47:0/750475052 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f860001b790 con 0x7f861007ce30 2024-03-21T10:11:18.401 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:17.664+0000 7f860f200700 1 -- 172.21.15.47:0/750475052 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f86100637c0 con 0x7f861007ce30 2024-03-21T10:11:18.401 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:17.667+0000 7f8606a00700 1 -- 172.21.15.47:0/750475052 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73141 (secure 0 0 0) 0x7f8600004260 con 0x7f861007ce30 2024-03-21T10:11:18.401 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:17.748+0000 7f860f200700 1 -- 172.21.15.47:0/750475052 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "status"} v 0) v1 -- 0x7f86101a05b0 con 0x7f861007ce30 2024-03-21T10:11:18.402 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:17.748+0000 7f8606a00700 1 -- 172.21.15.47:0/750475052 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "status"}]=0 v0) v1 ==== 54+0+317 (secure 0 0 0) 0x7f86101a05b0 con 0x7f861007ce30 2024-03-21T10:11:18.402 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:17.750+0000 7f860f200700 1 -- 172.21.15.47:0/750475052 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f861007ce30 msgr2=0x7f861019cf20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:11:18.402 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:17.750+0000 7f860f200700 1 --2- 172.21.15.47:0/750475052 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f861007ce30 0x7f861019cf20 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f8600009fb0 tx=0x7f8600004750 comp rx=0 tx=0).stop 2024-03-21T10:11:18.402 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:17.750+0000 7f860f200700 1 -- 172.21.15.47:0/750475052 shutdown_connections 2024-03-21T10:11:18.402 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:17.750+0000 7f860f200700 1 --2- 172.21.15.47:0/750475052 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f861007ce30 0x7f861019cf20 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:11:18.402 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:17.750+0000 7f860f200700 1 -- 172.21.15.47:0/750475052 >> 172.21.15.47:0/750475052 conn(0x7f8610102cd0 msgr2=0x7f8610103ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:11:18.402 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:17.750+0000 7f860f200700 1 -- 172.21.15.47:0/750475052 shutdown_connections 2024-03-21T10:11:18.402 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:17.750+0000 7f860f200700 1 -- 172.21.15.47:0/750475052 wait complete. 2024-03-21T10:11:18.402 INFO:teuthology.orchestra.run.smithi047.stdout:mon is available 2024-03-21T10:11:18.403 INFO:teuthology.orchestra.run.smithi047.stdout:Assimilating anything we can from ceph.conf... 2024-03-21T10:11:19.408 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:11:19 smithi047 ceph-mon[31169]: from='client.? 172.21.15.47:0/3320628520' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-03-21T10:11:19.409 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:11:19 smithi047 ceph-mon[31169]: from='client.? 172.21.15.47:0/3320628520' entity='client.admin' cmd='[{"prefix": "config assimilate-conf"}]': finished 2024-03-21T10:11:19.737 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout 2024-03-21T10:11:19.738 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout [global] 2024-03-21T10:11:19.738 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout fsid = 3056d51a-e76b-11ee-95cd-87774f69a715 2024-03-21T10:11:19.738 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout mon_host = [v2:172.21.15.47:3300,v1:172.21.15.47:6789] 2024-03-21T10:11:19.738 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-03-21T10:11:19.738 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-03-21T10:11:19.738 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-03-21T10:11:19.738 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-03-21T10:11:19.738 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout 2024-03-21T10:11:19.739 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout [mgr] 2024-03-21T10:11:19.739 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-03-21T10:11:19.739 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout 2024-03-21T10:11:19.739 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout [osd] 2024-03-21T10:11:19.739 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-03-21T10:11:19.739 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-03-21T10:11:19.739 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-03-21T10:11:19.739 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-03-21T10:11:19.739 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-03-21T10:11:19.739 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:18.979+0000 7fe872e00700 1 Processor -- start 2024-03-21T10:11:19.739 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:18.980+0000 7fe872e00700 1 -- start start 2024-03-21T10:11:19.740 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:18.980+0000 7fe872e00700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe86c07ce30 0x7fe86c07b2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:11:19.740 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:18.980+0000 7fe872e00700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe86c07b8b0 con 0x7fe86c07ce30 2024-03-21T10:11:19.740 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:18.980+0000 7fe86be00700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe86c07ce30 0x7fe86c07b2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:11:19.740 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:18.981+0000 7fe86be00700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe86c07ce30 0x7fe86c07b2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:43732/0 (socket says 172.21.15.47:43732) 2024-03-21T10:11:19.740 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:18.981+0000 7fe86be00700 1 -- 172.21.15.47:0/2552690663 learned_addr learned my addr 172.21.15.47:0/2552690663 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:11:19.740 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:18.981+0000 7fe86be00700 1 -- 172.21.15.47:0/2552690663 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe86c07ba30 con 0x7fe86c07ce30 2024-03-21T10:11:19.740 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:18.981+0000 7fe86be00700 1 --2- 172.21.15.47:0/2552690663 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe86c07ce30 0x7fe86c07b2e0 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7fe85c009e80 tx=0x7fe85c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b5750dcb59e7e6a5 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:11:19.740 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:18.982+0000 7fe86b400700 1 -- 172.21.15.47:0/2552690663 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fe85c017070 con 0x7fe86c07ce30 2024-03-21T10:11:19.740 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:18.982+0000 7fe86b400700 1 -- 172.21.15.47:0/2552690663 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe85c003a20 con 0x7fe86c07ce30 2024-03-21T10:11:19.740 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:18.982+0000 7fe86b400700 1 -- 172.21.15.47:0/2552690663 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fe85c003d20 con 0x7fe86c07ce30 2024-03-21T10:11:19.740 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:18.982+0000 7fe872e00700 1 -- 172.21.15.47:0/2552690663 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe86c07ce30 msgr2=0x7fe86c07b2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:11:19.740 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:18.982+0000 7fe872e00700 1 --2- 172.21.15.47:0/2552690663 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe86c07ce30 0x7fe86c07b2e0 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7fe85c009e80 tx=0x7fe85c00b330 comp rx=0 tx=0).stop 2024-03-21T10:11:19.741 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:18.983+0000 7fe872e00700 1 -- 172.21.15.47:0/2552690663 shutdown_connections 2024-03-21T10:11:19.741 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:18.983+0000 7fe872e00700 1 --2- 172.21.15.47:0/2552690663 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe86c07ce30 0x7fe86c07b2e0 unknown :-1 s=CLOSED pgs=3 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:11:19.741 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:18.983+0000 7fe872e00700 1 -- 172.21.15.47:0/2552690663 >> 172.21.15.47:0/2552690663 conn(0x7fe86c102f50 msgr2=0x7fe86c105370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:11:19.741 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:18.983+0000 7fe872e00700 1 -- 172.21.15.47:0/2552690663 shutdown_connections 2024-03-21T10:11:19.741 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:18.983+0000 7fe872e00700 1 -- 172.21.15.47:0/2552690663 wait complete. 2024-03-21T10:11:19.741 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:18.984+0000 7fe872e00700 1 Processor -- start 2024-03-21T10:11:19.741 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:18.984+0000 7fe872e00700 1 -- start start 2024-03-21T10:11:19.741 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:18.984+0000 7fe872e00700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe86c07ce30 0x7fe86c19d1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:11:19.741 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:18.984+0000 7fe872e00700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe86c07bf60 con 0x7fe86c07ce30 2024-03-21T10:11:19.741 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:18.985+0000 7fe86be00700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe86c07ce30 0x7fe86c19d1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:11:19.741 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:18.985+0000 7fe86be00700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe86c07ce30 0x7fe86c19d1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:43744/0 (socket says 172.21.15.47:43744) 2024-03-21T10:11:19.741 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:18.985+0000 7fe86be00700 1 -- 172.21.15.47:0/3320628520 learned_addr learned my addr 172.21.15.47:0/3320628520 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:11:19.742 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:18.985+0000 7fe86be00700 1 -- 172.21.15.47:0/3320628520 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe86c19d710 con 0x7fe86c07ce30 2024-03-21T10:11:19.742 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:18.985+0000 7fe86be00700 1 --2- 172.21.15.47:0/3320628520 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe86c07ce30 0x7fe86c19d1d0 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7fe86c07c350 tx=0x7fe85c004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:11:19.742 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:18.986+0000 7fe869600700 1 -- 172.21.15.47:0/3320628520 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fe85c022020 con 0x7fe86c07ce30 2024-03-21T10:11:19.742 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:18.986+0000 7fe869600700 1 -- 172.21.15.47:0/3320628520 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe85c018040 con 0x7fe86c07ce30 2024-03-21T10:11:19.742 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:18.986+0000 7fe872e00700 1 -- 172.21.15.47:0/3320628520 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe86c19d9a0 con 0x7fe86c07ce30 2024-03-21T10:11:19.742 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:18.986+0000 7fe869600700 1 -- 172.21.15.47:0/3320628520 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fe85c017070 con 0x7fe86c07ce30 2024-03-21T10:11:19.742 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:18.986+0000 7fe872e00700 1 -- 172.21.15.47:0/3320628520 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe86c1a0690 con 0x7fe86c07ce30 2024-03-21T10:11:19.742 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:18.986+0000 7fe869600700 1 -- 172.21.15.47:0/3320628520 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (secure 0 0 0) 0x7fe85c025020 con 0x7fe86c07ce30 2024-03-21T10:11:19.742 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:18.986+0000 7fe869600700 1 -- 172.21.15.47:0/3320628520 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7fe85c00f050 con 0x7fe86c07ce30 2024-03-21T10:11:19.742 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:18.987+0000 7fe872e00700 1 -- 172.21.15.47:0/3320628520 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe86c0637c0 con 0x7fe86c07ce30 2024-03-21T10:11:19.742 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:18.990+0000 7fe869600700 1 -- 172.21.15.47:0/3320628520 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73141 (secure 0 0 0) 0x7fe85c02c030 con 0x7fe86c07ce30 2024-03-21T10:11:19.742 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:19.069+0000 7fe872e00700 1 -- 172.21.15.47:0/3320628520 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) v1 -- 0x7fe86c02dbf0 con 0x7fe86c07ce30 2024-03-21T10:11:19.742 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:19.077+0000 7fe869600700 1 -- 172.21.15.47:0/3320628520 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v2) v1 ==== 70+0+429 (secure 0 0 0) 0x7fe85c01e480 con 0x7fe86c07ce30 2024-03-21T10:11:19.743 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:19.078+0000 7fe869600700 1 -- 172.21.15.47:0/3320628520 <== mon.0 v2:172.21.15.47:3300/0 8 ==== config(26 keys) v1 ==== 1071+0+0 (secure 0 0 0) 0x7fe85c003a20 con 0x7fe86c07ce30 2024-03-21T10:11:19.743 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:19.078+0000 7fe872e00700 1 -- 172.21.15.47:0/3320628520 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe86c07ce30 msgr2=0x7fe86c19d1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:11:19.743 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:19.078+0000 7fe872e00700 1 --2- 172.21.15.47:0/3320628520 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe86c07ce30 0x7fe86c19d1d0 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7fe86c07c350 tx=0x7fe85c004ad0 comp rx=0 tx=0).stop 2024-03-21T10:11:19.743 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:19.078+0000 7fe872e00700 1 -- 172.21.15.47:0/3320628520 shutdown_connections 2024-03-21T10:11:19.743 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:19.078+0000 7fe872e00700 1 --2- 172.21.15.47:0/3320628520 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe86c07ce30 0x7fe86c19d1d0 unknown :-1 s=CLOSED pgs=4 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:11:19.743 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:19.078+0000 7fe872e00700 1 -- 172.21.15.47:0/3320628520 >> 172.21.15.47:0/3320628520 conn(0x7fe86c102f50 msgr2=0x7fe86c103e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:11:19.743 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:19.078+0000 7fe872e00700 1 -- 172.21.15.47:0/3320628520 shutdown_connections 2024-03-21T10:11:19.743 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:19.078+0000 7fe872e00700 1 -- 172.21.15.47:0/3320628520 wait complete. 2024-03-21T10:11:19.743 INFO:teuthology.orchestra.run.smithi047.stdout:Generating new minimal ceph.conf... 2024-03-21T10:11:20.653 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:11:20 smithi047 ceph-mon[31169]: from='client.? 172.21.15.47:0/187559545' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T10:11:20.969 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:20.239+0000 7fc187200700 1 Processor -- start 2024-03-21T10:11:20.969 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:20.239+0000 7fc187200700 1 -- start start 2024-03-21T10:11:20.969 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:20.240+0000 7fc187200700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc188108150 0x7fc18810a540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:11:20.969 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:20.240+0000 7fc187200700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc18807ae10 con 0x7fc188108150 2024-03-21T10:11:20.969 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:20.240+0000 7fc185e00700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc188108150 0x7fc18810a540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:11:20.969 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:20.240+0000 7fc185e00700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc188108150 0x7fc18810a540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:43750/0 (socket says 172.21.15.47:43750) 2024-03-21T10:11:20.970 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:20.240+0000 7fc185e00700 1 -- 172.21.15.47:0/1167789408 learned_addr learned my addr 172.21.15.47:0/1167789408 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:11:20.970 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:20.241+0000 7fc185e00700 1 -- 172.21.15.47:0/1167789408 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc18810aa80 con 0x7fc188108150 2024-03-21T10:11:20.970 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:20.241+0000 7fc185e00700 1 --2- 172.21.15.47:0/1167789408 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc188108150 0x7fc18810a540 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7fc1780098a0 tx=0x7fc178009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=504ef1371c4237df server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:11:20.970 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:20.241+0000 7fc184a00700 1 -- 172.21.15.47:0/1167789408 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc178004160 con 0x7fc188108150 2024-03-21T10:11:20.970 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:20.241+0000 7fc184a00700 1 -- 172.21.15.47:0/1167789408 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) v1 ==== 1071+0+0 (secure 0 0 0) 0x7fc178004300 con 0x7fc188108150 2024-03-21T10:11:20.970 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:20.241+0000 7fc184a00700 1 -- 172.21.15.47:0/1167789408 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc178011a80 con 0x7fc188108150 2024-03-21T10:11:20.970 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:20.242+0000 7fc187200700 1 -- 172.21.15.47:0/1167789408 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc188108150 msgr2=0x7fc18810a540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:11:20.970 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:20.242+0000 7fc187200700 1 --2- 172.21.15.47:0/1167789408 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc188108150 0x7fc18810a540 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7fc1780098a0 tx=0x7fc178009bb0 comp rx=0 tx=0).stop 2024-03-21T10:11:20.970 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:20.242+0000 7fc187200700 1 -- 172.21.15.47:0/1167789408 shutdown_connections 2024-03-21T10:11:20.970 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:20.242+0000 7fc187200700 1 --2- 172.21.15.47:0/1167789408 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc188108150 0x7fc18810a540 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:11:20.970 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:20.242+0000 7fc187200700 1 -- 172.21.15.47:0/1167789408 >> 172.21.15.47:0/1167789408 conn(0x7fc188101b60 msgr2=0x7fc188103f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:11:20.970 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:20.242+0000 7fc187200700 1 -- 172.21.15.47:0/1167789408 shutdown_connections 2024-03-21T10:11:20.970 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:20.242+0000 7fc187200700 1 -- 172.21.15.47:0/1167789408 wait complete. 2024-03-21T10:11:20.971 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:20.243+0000 7fc187200700 1 Processor -- start 2024-03-21T10:11:20.971 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:20.244+0000 7fc187200700 1 -- start start 2024-03-21T10:11:20.971 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:20.244+0000 7fc187200700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc188108150 0x7fc18819d780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:11:20.971 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:20.244+0000 7fc187200700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc18810b9e0 con 0x7fc188108150 2024-03-21T10:11:20.971 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:20.244+0000 7fc185e00700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc188108150 0x7fc18819d780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:11:20.971 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:20.244+0000 7fc185e00700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc188108150 0x7fc18819d780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:43752/0 (socket says 172.21.15.47:43752) 2024-03-21T10:11:20.971 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:20.244+0000 7fc185e00700 1 -- 172.21.15.47:0/187559545 learned_addr learned my addr 172.21.15.47:0/187559545 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:11:20.971 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:20.245+0000 7fc185e00700 1 -- 172.21.15.47:0/187559545 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc18819dcc0 con 0x7fc188108150 2024-03-21T10:11:20.971 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:20.245+0000 7fc185e00700 1 --2- 172.21.15.47:0/187559545 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc188108150 0x7fc18819d780 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fc178011a50 tx=0x7fc178011b30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:11:20.971 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:20.245+0000 7fc17ea00700 1 -- 172.21.15.47:0/187559545 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc178003a80 con 0x7fc188108150 2024-03-21T10:11:20.971 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:20.245+0000 7fc17ea00700 1 -- 172.21.15.47:0/187559545 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) v1 ==== 1071+0+0 (secure 0 0 0) 0x7fc178003c20 con 0x7fc188108150 2024-03-21T10:11:20.971 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:20.245+0000 7fc187200700 1 -- 172.21.15.47:0/187559545 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc18819df50 con 0x7fc188108150 2024-03-21T10:11:20.971 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:20.245+0000 7fc17ea00700 1 -- 172.21.15.47:0/187559545 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc17801a650 con 0x7fc188108150 2024-03-21T10:11:20.971 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:20.245+0000 7fc187200700 1 -- 172.21.15.47:0/187559545 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc1881a0c40 con 0x7fc188108150 2024-03-21T10:11:20.972 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:20.246+0000 7fc17ea00700 1 -- 172.21.15.47:0/187559545 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (secure 0 0 0) 0x7fc17801ad00 con 0x7fc188108150 2024-03-21T10:11:20.972 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:20.246+0000 7fc17ea00700 1 -- 172.21.15.47:0/187559545 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7fc178019700 con 0x7fc188108150 2024-03-21T10:11:20.972 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:20.247+0000 7fc187200700 1 -- 172.21.15.47:0/187559545 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc1880637c0 con 0x7fc188108150 2024-03-21T10:11:20.972 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:20.250+0000 7fc17ea00700 1 -- 172.21.15.47:0/187559545 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73141 (secure 0 0 0) 0x7fc178024ad0 con 0x7fc188108150 2024-03-21T10:11:20.972 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:20.325+0000 7fc187200700 1 -- 172.21.15.47:0/187559545 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) v1 -- 0x7fc1881a1180 con 0x7fc188108150 2024-03-21T10:11:20.972 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:20.326+0000 7fc17ea00700 1 -- 172.21.15.47:0/187559545 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v2) v1 ==== 76+0+175 (secure 0 0 0) 0x7fc178024440 con 0x7fc188108150 2024-03-21T10:11:20.972 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:20.328+0000 7fc187200700 1 -- 172.21.15.47:0/187559545 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc188108150 msgr2=0x7fc18819d780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:11:20.972 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:20.328+0000 7fc187200700 1 --2- 172.21.15.47:0/187559545 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc188108150 0x7fc18819d780 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fc178011a50 tx=0x7fc178011b30 comp rx=0 tx=0).stop 2024-03-21T10:11:20.972 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:20.328+0000 7fc187200700 1 -- 172.21.15.47:0/187559545 shutdown_connections 2024-03-21T10:11:20.972 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:20.328+0000 7fc187200700 1 --2- 172.21.15.47:0/187559545 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc188108150 0x7fc18819d780 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:11:20.972 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:20.328+0000 7fc187200700 1 -- 172.21.15.47:0/187559545 >> 172.21.15.47:0/187559545 conn(0x7fc188101b60 msgr2=0x7fc1881025c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:11:20.972 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:20.328+0000 7fc187200700 1 -- 172.21.15.47:0/187559545 shutdown_connections 2024-03-21T10:11:20.972 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:20.328+0000 7fc187200700 1 -- 172.21.15.47:0/187559545 wait complete. 2024-03-21T10:11:20.972 INFO:teuthology.orchestra.run.smithi047.stdout:Restarting the monitor... 2024-03-21T10:11:21.214 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:11:21 smithi047 systemd[1]: Stopping Ceph mon.a for 3056d51a-e76b-11ee-95cd-87774f69a715... 2024-03-21T10:11:21.823 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:11:21 smithi047 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-mon-a[31165]: 2024-03-21T10:11:21.480+0000 7f04a1600700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-03-21T10:11:21.823 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:11:21 smithi047 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-mon-a[31165]: 2024-03-21T10:11:21.480+0000 7f04a1600700 -1 mon.a@0(leader) e1 *** Got Signal Terminated *** 2024-03-21T10:11:21.823 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:11:21 smithi047 podman[31423]: 2024-03-21 10:11:21.5132487 +0000 UTC m=+0.152265285 container died af0d1f022cee7a39b29672d1c55b8b994c852494e248268eeee96ffc08262a38 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2edfee68ad7243bed422a049b6f5f58619300f11, name=ceph-3056d51a-e76b-11ee-95cd-87774f69a715-mon-a, org.label-schema.build-date=20240319, GIT_CLEAN=True, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, org.label-schema.vendor=CentOS, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5) 2024-03-21T10:11:22.101 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:11:21 smithi047 podman[31423]: 2024-03-21 10:11:21.932548876 +0000 UTC m=+0.571565475 container cleanup af0d1f022cee7a39b29672d1c55b8b994c852494e248268eeee96ffc08262a38 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2edfee68ad7243bed422a049b6f5f58619300f11, name=ceph-3056d51a-e76b-11ee-95cd-87774f69a715-mon-a, GIT_CLEAN=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.build-date=20240319, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS) 2024-03-21T10:11:22.101 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:11:21 smithi047 bash[31423]: ceph-3056d51a-e76b-11ee-95cd-87774f69a715-mon-a 2024-03-21T10:11:22.464 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:11:22 smithi047 podman[31437]: 2024-03-21 10:11:22.100014273 +0000 UTC m=+0.573577206 container remove af0d1f022cee7a39b29672d1c55b8b994c852494e248268eeee96ffc08262a38 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2edfee68ad7243bed422a049b6f5f58619300f11, name=ceph-3056d51a-e76b-11ee-95cd-87774f69a715-mon-a, GIT_BRANCH=HEAD, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240319, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-21T10:11:22.964 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:11:22 smithi047 systemd[1]: ceph-3056d51a-e76b-11ee-95cd-87774f69a715@mon.a.service: Deactivated successfully. 2024-03-21T10:11:22.964 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:11:22 smithi047 systemd[1]: Stopped Ceph mon.a for 3056d51a-e76b-11ee-95cd-87774f69a715. 2024-03-21T10:11:22.964 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:11:22 smithi047 systemd[1]: ceph-3056d51a-e76b-11ee-95cd-87774f69a715@mon.a.service: Consumed 2.001s CPU time. 2024-03-21T10:11:22.964 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:11:22 smithi047 systemd[1]: Starting Ceph mon.a for 3056d51a-e76b-11ee-95cd-87774f69a715... 2024-03-21T10:11:23.273 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:11:23 smithi047 podman[31529]: 2024-03-21 10:11:22.92607288 +0000 UTC m=+0.037952051 image pull 237e43c7a65ff48534d94b97c1bfdbe4b7be5672e3d9a613745083ea3702aa54 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2edfee68ad7243bed422a049b6f5f58619300f11 2024-03-21T10:11:23.273 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:11:23 smithi047 podman[31529]: 2024-03-21 10:11:23.059187076 +0000 UTC m=+0.171066286 container create e43a05e9bf7b17a59e76c9f2a93da51a6c01f99431f509aeb2d6d80f76fc6414 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2edfee68ad7243bed422a049b6f5f58619300f11, name=ceph-3056d51a-e76b-11ee-95cd-87774f69a715-mon-a, org.label-schema.build-date=20240319, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_CLEAN=True, maintainer=Guillaume Abrioux , ceph=True, CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-21T10:11:23.273 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:11:23 smithi047 podman[31529]: 2024-03-21 10:11:23.271679594 +0000 UTC m=+0.383558811 container init e43a05e9bf7b17a59e76c9f2a93da51a6c01f99431f509aeb2d6d80f76fc6414 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2edfee68ad7243bed422a049b6f5f58619300f11, name=ceph-3056d51a-e76b-11ee-95cd-87774f69a715-mon-a, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240319, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.vendor=CentOS, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0) 2024-03-21T10:11:23.363 INFO:teuthology.orchestra.run.smithi047.stdout:Setting public_network to 172.21.0.0/20 in mon config section 2024-03-21T10:11:23.681 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:11:23 smithi047 podman[31529]: 2024-03-21 10:11:23.283527076 +0000 UTC m=+0.395406258 container start e43a05e9bf7b17a59e76c9f2a93da51a6c01f99431f509aeb2d6d80f76fc6414 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2edfee68ad7243bed422a049b6f5f58619300f11, name=ceph-3056d51a-e76b-11ee-95cd-87774f69a715-mon-a, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.build-date=20240319, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, io.buildah.version=1.33.5, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True) 2024-03-21T10:11:23.682 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:11:23 smithi047 ceph-mon[31547]: set uid:gid to 167:167 (ceph:ceph) 2024-03-21T10:11:23.682 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:11:23 smithi047 ceph-mon[31547]: ceph version 19.0.0-2231-g2edfee68 (2edfee68ad7243bed422a049b6f5f58619300f11) squid (dev), process ceph-mon, pid 2 2024-03-21T10:11:23.682 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:11:23 smithi047 ceph-mon[31547]: pidfile_write: ignore empty --pid-file 2024-03-21T10:11:23.682 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:11:23 smithi047 ceph-mon[31547]: load: jerasure load: lrc 2024-03-21T10:11:23.682 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:11:23 smithi047 ceph-mon[31547]: rocksdb: RocksDB version: 7.9.2 2024-03-21T10:11:23.682 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:11:23 smithi047 ceph-mon[31547]: rocksdb: Git sha 0 2024-03-21T10:11:23.682 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:11:23 smithi047 ceph-mon[31547]: rocksdb: Compile date 2024-03-20 06:23:48 2024-03-21T10:11:23.682 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:11:23 smithi047 ceph-mon[31547]: rocksdb: DB SUMMARY 2024-03-21T10:11:23.682 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:11:23 smithi047 ceph-mon[31547]: rocksdb: DB Session ID: WBXTRXU9QVE5LE0ACB7F 2024-03-21T10:11:23.682 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:11:23 smithi047 ceph-mon[31547]: rocksdb: CURRENT file: CURRENT 2024-03-21T10:11:23.683 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:11:23 smithi047 ceph-mon[31547]: rocksdb: IDENTITY file: IDENTITY 2024-03-21T10:11:23.683 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:11:23 smithi047 ceph-mon[31547]: rocksdb: MANIFEST file: MANIFEST-000010 size: 179 Bytes 2024-03-21T10:11:23.683 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:11:23 smithi047 ceph-mon[31547]: rocksdb: SST files in /var/lib/ceph/mon/ceph-a/store.db dir, Total Num: 1, files: 000008.sst 2024-03-21T10:11:23.683 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:11:23 smithi047 ceph-mon[31547]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-a/store.db: 000009.log size: 98954 ; 2024-03-21T10:11:23.683 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:11:23 smithi047 ceph-mon[31547]: rocksdb: Options.error_if_exists: 0 2024-03-21T10:11:23.683 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:11:23 smithi047 ceph-mon[31547]: rocksdb: Options.create_if_missing: 0 2024-03-21T10:11:23.683 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:11:23 smithi047 ceph-mon[31547]: rocksdb: Options.paranoid_checks: 1 2024-03-21T10:11:23.683 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:11:23 smithi047 ceph-mon[31547]: rocksdb: Options.flush_verify_memtable_count: 1 2024-03-21T10:11:23.683 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:11:23 smithi047 ceph-mon[31547]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-03-21T10:11:23.683 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:11:23 smithi047 ceph-mon[31547]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-03-21T10:11:23.683 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:11:23 smithi047 ceph-mon[31547]: rocksdb: Options.env: 0x558008a0db20 2024-03-21T10:11:23.683 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:11:23 smithi047 ceph-mon[31547]: rocksdb: Options.fs: PosixFileSystem 2024-03-21T10:11:23.683 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:11:23 smithi047 ceph-mon[31547]: rocksdb: Options.info_log: 0x55800a649960 2024-03-21T10:11:23.684 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:11:23 smithi047 ceph-mon[31547]: rocksdb: Options.max_file_opening_threads: 16 2024-03-21T10:11:23.684 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:11:23 smithi047 ceph-mon[31547]: rocksdb: Options.statistics: (nil) 2024-03-21T10:11:23.684 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:11:23 smithi047 ceph-mon[31547]: rocksdb: Options.use_fsync: 0 2024-03-21T10:11:23.684 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:11:23 smithi047 ceph-mon[31547]: rocksdb: Options.max_log_file_size: 0 2024-03-21T10:11:23.684 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:11:23 smithi047 ceph-mon[31547]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-03-21T10:11:23.684 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:11:23 smithi047 ceph-mon[31547]: rocksdb: Options.log_file_time_to_roll: 0 2024-03-21T10:11:23.684 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:11:23 smithi047 ceph-mon[31547]: rocksdb: Options.keep_log_file_num: 1000 2024-03-21T10:11:23.684 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:11:23 smithi047 ceph-mon[31547]: rocksdb: Options.recycle_log_file_num: 0 2024-03-21T10:11:23.684 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:11:23 smithi047 ceph-mon[31547]: rocksdb: Options.allow_fallocate: 1 2024-03-21T10:11:23.685 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:11:23 smithi047 ceph-mon[31547]: rocksdb: Options.allow_mmap_reads: 0 2024-03-21T10:11:23.685 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:11:23 smithi047 ceph-mon[31547]: rocksdb: Options.allow_mmap_writes: 0 2024-03-21T10:11:23.685 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:11:23 smithi047 ceph-mon[31547]: rocksdb: Options.use_direct_reads: 0 2024-03-21T10:11:23.685 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:11:23 smithi047 ceph-mon[31547]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-03-21T10:11:23.685 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:11:23 smithi047 ceph-mon[31547]: rocksdb: Options.create_missing_column_families: 0 2024-03-21T10:11:23.685 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:11:23 smithi047 ceph-mon[31547]: rocksdb: Options.db_log_dir: 2024-03-21T10:11:23.685 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:11:23 smithi047 ceph-mon[31547]: rocksdb: Options.wal_dir: 2024-03-21T10:11:23.685 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:11:23 smithi047 ceph-mon[31547]: rocksdb: Options.table_cache_numshardbits: 6 2024-03-21T10:11:23.686 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:11:23 smithi047 ceph-mon[31547]: rocksdb: Options.WAL_ttl_seconds: 0 2024-03-21T10:11:23.686 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:11:23 smithi047 ceph-mon[31547]: rocksdb: Options.WAL_size_limit_MB: 0 2024-03-21T10:11:23.686 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:11:23 smithi047 ceph-mon[31547]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-03-21T10:11:23.686 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:11:23 smithi047 ceph-mon[31547]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-03-21T10:11:23.686 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:11:23 smithi047 ceph-mon[31547]: rocksdb: Options.is_fd_close_on_exec: 1 2024-03-21T10:11:23.686 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:11:23 smithi047 ceph-mon[31547]: rocksdb: Options.advise_random_on_open: 1 2024-03-21T10:11:23.686 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:11:23 smithi047 ceph-mon[31547]: rocksdb: Options.db_write_buffer_size: 0 2024-03-21T10:11:23.686 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:11:23 smithi047 ceph-mon[31547]: rocksdb: Options.write_buffer_manager: 0x55800a61c1e0 2024-03-21T10:11:23.686 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:11:23 smithi047 ceph-mon[31547]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-03-21T10:11:23.686 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:11:23 smithi047 ceph-mon[31547]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-03-21T10:11:23.686 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:11:23 smithi047 ceph-mon[31547]: rocksdb: Options.use_adaptive_mutex: 0 2024-03-21T10:11:23.686 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:11:23 smithi047 ceph-mon[31547]: rocksdb: Options.rate_limiter: (nil) 2024-03-21T10:11:23.687 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:11:23 smithi047 ceph-mon[31547]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-03-21T10:11:23.687 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:11:23 smithi047 ceph-mon[31547]: rocksdb: Options.wal_recovery_mode: 2 2024-03-21T10:11:23.687 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:11:23 smithi047 ceph-mon[31547]: rocksdb: Options.enable_thread_tracking: 0 2024-03-21T10:11:23.687 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:11:23 smithi047 ceph-mon[31547]: rocksdb: Options.enable_pipelined_write: 0 2024-03-21T10:11:23.687 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:11:23 smithi047 ceph-mon[31547]: rocksdb: Options.unordered_write: 0 2024-03-21T10:11:23.687 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:11:23 smithi047 ceph-mon[31547]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-03-21T10:11:23.687 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:11:23 smithi047 ceph-mon[31547]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-03-21T10:11:23.687 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:11:23 smithi047 ceph-mon[31547]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-03-21T10:11:23.687 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:11:23 smithi047 ceph-mon[31547]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-03-21T10:11:23.687 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:11:23 smithi047 ceph-mon[31547]: rocksdb: Options.row_cache: None 2024-03-21T10:11:23.687 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:11:23 smithi047 ceph-mon[31547]: rocksdb: Options.wal_filter: None 2024-03-21T10:11:23.687 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:11:23 smithi047 ceph-mon[31547]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-03-21T10:11:23.687 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:11:23 smithi047 ceph-mon[31547]: rocksdb: Options.allow_ingest_behind: 0 2024-03-21T10:11:23.688 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:11:23 smithi047 ceph-mon[31547]: rocksdb: Options.two_write_queues: 0 2024-03-21T10:11:23.688 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:11:23 smithi047 ceph-mon[31547]: rocksdb: Options.manual_wal_flush: 0 2024-03-21T10:11:23.688 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:11:23 smithi047 ceph-mon[31547]: rocksdb: Options.wal_compression: 0 2024-03-21T10:11:23.688 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:11:23 smithi047 ceph-mon[31547]: rocksdb: Options.atomic_flush: 0 2024-03-21T10:11:23.688 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:11:23 smithi047 ceph-mon[31547]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-03-21T10:11:23.688 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:11:23 smithi047 ceph-mon[31547]: rocksdb: Options.persist_stats_to_disk: 0 2024-03-21T10:11:23.688 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:11:23 smithi047 ceph-mon[31547]: rocksdb: Options.write_dbid_to_manifest: 0 2024-03-21T10:11:23.688 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:11:23 smithi047 ceph-mon[31547]: rocksdb: Options.log_readahead_size: 0 2024-03-21T10:11:23.688 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:11:23 smithi047 ceph-mon[31547]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-03-21T10:11:23.688 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:11:23 smithi047 ceph-mon[31547]: rocksdb: Options.best_efforts_recovery: 0 2024-03-21T10:11:23.688 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:11:23 smithi047 ceph-mon[31547]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-03-21T10:11:23.689 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:11:23 smithi047 ceph-mon[31547]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-03-21T10:11:23.689 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:11:23 smithi047 ceph-mon[31547]: rocksdb: Options.allow_data_in_errors: 0 2024-03-21T10:11:23.689 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:11:23 smithi047 ceph-mon[31547]: rocksdb: Options.db_host_id: __hostname__ 2024-03-21T10:11:23.689 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:11:23 smithi047 ceph-mon[31547]: rocksdb: Options.enforce_single_del_contracts: true 2024-03-21T10:11:23.689 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:11:23 smithi047 ceph-mon[31547]: rocksdb: Options.max_background_jobs: 2 2024-03-21T10:11:23.689 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:11:23 smithi047 ceph-mon[31547]: rocksdb: Options.max_background_compactions: -1 2024-03-21T10:11:23.689 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:11:23 smithi047 ceph-mon[31547]: rocksdb: Options.max_subcompactions: 1 2024-03-21T10:11:23.689 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:11:23 smithi047 ceph-mon[31547]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-03-21T10:11:23.689 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:11:23 smithi047 ceph-mon[31547]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-03-21T10:11:23.689 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:11:23 smithi047 ceph-mon[31547]: rocksdb: Options.delayed_write_rate : 16777216 2024-03-21T10:11:23.689 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:11:23 smithi047 ceph-mon[31547]: rocksdb: Options.max_total_wal_size: 0 2024-03-21T10:11:23.689 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:11:23 smithi047 ceph-mon[31547]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-03-21T10:11:23.689 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:11:23 smithi047 ceph-mon[31547]: rocksdb: Options.stats_dump_period_sec: 600 2024-03-21T10:11:23.689 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:11:23 smithi047 ceph-mon[31547]: rocksdb: Options.stats_persist_period_sec: 600 2024-03-21T10:11:23.690 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:11:23 smithi047 ceph-mon[31547]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-03-21T10:11:23.690 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:11:23 smithi047 ceph-mon[31547]: rocksdb: Options.max_open_files: -1 2024-03-21T10:11:23.690 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:11:23 smithi047 ceph-mon[31547]: rocksdb: Options.bytes_per_sync: 0 2024-03-21T10:11:23.690 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:11:23 smithi047 ceph-mon[31547]: rocksdb: Options.wal_bytes_per_sync: 0 2024-03-21T10:11:23.690 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:11:23 smithi047 ceph-mon[31547]: rocksdb: Options.strict_bytes_per_sync: 0 2024-03-21T10:11:23.690 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:11:23 smithi047 ceph-mon[31547]: rocksdb: Options.compaction_readahead_size: 0 2024-03-21T10:11:23.690 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:11:23 smithi047 ceph-mon[31547]: rocksdb: Options.max_background_flushes: -1 2024-03-21T10:11:23.690 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:11:23 smithi047 ceph-mon[31547]: rocksdb: Compression algorithms supported: 2024-03-21T10:11:23.690 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:11:23 smithi047 ceph-mon[31547]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-03-21T10:11:23.690 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:11:23 smithi047 ceph-mon[31547]: rocksdb: kZSTD supported: 0 2024-03-21T10:11:23.690 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:11:23 smithi047 ceph-mon[31547]: rocksdb: kXpressCompression supported: 0 2024-03-21T10:11:23.690 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:11:23 smithi047 ceph-mon[31547]: rocksdb: kLZ4HCCompression supported: 1 2024-03-21T10:11:23.690 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:11:23 smithi047 ceph-mon[31547]: rocksdb: kZlibCompression supported: 1 2024-03-21T10:11:23.691 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:11:23 smithi047 ceph-mon[31547]: rocksdb: kSnappyCompression supported: 1 2024-03-21T10:11:23.691 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:11:23 smithi047 ceph-mon[31547]: rocksdb: kLZ4Compression supported: 1 2024-03-21T10:11:23.691 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:11:23 smithi047 ceph-mon[31547]: rocksdb: kBZip2Compression supported: 0 2024-03-21T10:11:23.691 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:11:23 smithi047 ceph-mon[31547]: rocksdb: Fast CRC32 supported: Supported on x86 2024-03-21T10:11:23.691 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:11:23 smithi047 ceph-mon[31547]: rocksdb: DMutex implementation: pthread_mutex_t 2024-03-21T10:11:23.691 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:11:23 smithi047 ceph-mon[31547]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 2024-03-21T10:11:23.691 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:11:23 smithi047 ceph-mon[31547]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-03-21T10:11:23.691 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:11:23 smithi047 ceph-mon[31547]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-03-21T10:11:23.691 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:11:23 smithi047 ceph-mon[31547]: rocksdb: Options.merge_operator: 2024-03-21T10:11:23.691 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:11:23 smithi047 ceph-mon[31547]: rocksdb: Options.compaction_filter: None 2024-03-21T10:11:23.691 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:11:23 smithi047 ceph-mon[31547]: rocksdb: Options.compaction_filter_factory: None 2024-03-21T10:11:23.691 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:11:23 smithi047 ceph-mon[31547]: rocksdb: Options.sst_partitioner_factory: None 2024-03-21T10:11:23.691 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:11:23 smithi047 ceph-mon[31547]: rocksdb: Options.memtable_factory: SkipListFactory 2024-03-21T10:11:23.691 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:11:23 smithi047 ceph-mon[31547]: rocksdb: Options.table_factory: BlockBasedTable 2024-03-21T10:11:23.692 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:11:23 smithi047 ceph-mon[31547]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x55800a649a60) 2024-03-21T10:11:23.692 INFO:journalctl@ceph.mon.a.smithi047.stdout: cache_index_and_filter_blocks: 1 2024-03-21T10:11:23.692 INFO:journalctl@ceph.mon.a.smithi047.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-03-21T10:11:23.692 INFO:journalctl@ceph.mon.a.smithi047.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-03-21T10:11:23.692 INFO:journalctl@ceph.mon.a.smithi047.stdout: pin_top_level_index_and_filter: 1 2024-03-21T10:11:23.692 INFO:journalctl@ceph.mon.a.smithi047.stdout: index_type: 0 2024-03-21T10:11:23.692 INFO:journalctl@ceph.mon.a.smithi047.stdout: data_block_index_type: 0 2024-03-21T10:11:23.692 INFO:journalctl@ceph.mon.a.smithi047.stdout: index_shortening: 1 2024-03-21T10:11:23.692 INFO:journalctl@ceph.mon.a.smithi047.stdout: data_block_hash_table_util_ratio: 0.750000 2024-03-21T10:11:23.692 INFO:journalctl@ceph.mon.a.smithi047.stdout: checksum: 4 2024-03-21T10:11:23.692 INFO:journalctl@ceph.mon.a.smithi047.stdout: no_block_cache: 0 2024-03-21T10:11:23.692 INFO:journalctl@ceph.mon.a.smithi047.stdout: block_cache: 0x55800a64f090 2024-03-21T10:11:23.692 INFO:journalctl@ceph.mon.a.smithi047.stdout: block_cache_name: BinnedLRUCache 2024-03-21T10:11:23.692 INFO:journalctl@ceph.mon.a.smithi047.stdout: block_cache_options: 2024-03-21T10:11:23.693 INFO:journalctl@ceph.mon.a.smithi047.stdout: capacity : 536870912 2024-03-21T10:11:23.693 INFO:journalctl@ceph.mon.a.smithi047.stdout: num_shard_bits : 4 2024-03-21T10:11:23.693 INFO:journalctl@ceph.mon.a.smithi047.stdout: strict_capacity_limit : 0 2024-03-21T10:11:23.693 INFO:journalctl@ceph.mon.a.smithi047.stdout: high_pri_pool_ratio: 0.000 2024-03-21T10:11:23.693 INFO:journalctl@ceph.mon.a.smithi047.stdout: block_cache_compressed: (nil) 2024-03-21T10:11:23.693 INFO:journalctl@ceph.mon.a.smithi047.stdout: persistent_cache: (nil) 2024-03-21T10:11:23.693 INFO:journalctl@ceph.mon.a.smithi047.stdout: block_size: 4096 2024-03-21T10:11:23.693 INFO:journalctl@ceph.mon.a.smithi047.stdout: block_size_deviation: 10 2024-03-21T10:11:23.693 INFO:journalctl@ceph.mon.a.smithi047.stdout: block_restart_interval: 16 2024-03-21T10:11:23.693 INFO:journalctl@ceph.mon.a.smithi047.stdout: index_block_restart_interval: 1 2024-03-21T10:11:23.693 INFO:journalctl@ceph.mon.a.smithi047.stdout: metadata_block_size: 4096 2024-03-21T10:11:23.693 INFO:journalctl@ceph.mon.a.smithi047.stdout: partition_filters: 0 2024-03-21T10:11:23.693 INFO:journalctl@ceph.mon.a.smithi047.stdout: use_delta_encoding: 1 2024-03-21T10:11:23.693 INFO:journalctl@ceph.mon.a.smithi047.stdout: filter_policy: bloomfilter 2024-03-21T10:11:23.694 INFO:journalctl@ceph.mon.a.smithi047.stdout: whole_key_filtering: 1 2024-03-21T10:11:23.694 INFO:journalctl@ceph.mon.a.smithi047.stdout: verify_compression: 0 2024-03-21T10:11:23.694 INFO:journalctl@ceph.mon.a.smithi047.stdout: read_amp_bytes_per_bit: 0 2024-03-21T10:11:23.694 INFO:journalctl@ceph.mon.a.smithi047.stdout: format_version: 5 2024-03-21T10:11:23.694 INFO:journalctl@ceph.mon.a.smithi047.stdout: enable_index_compression: 1 2024-03-21T10:11:23.694 INFO:journalctl@ceph.mon.a.smithi047.stdout: block_align: 0 2024-03-21T10:11:23.694 INFO:journalctl@ceph.mon.a.smithi047.stdout: max_auto_readahead_size: 262144 2024-03-21T10:11:23.694 INFO:journalctl@ceph.mon.a.smithi047.stdout: prepopulate_block_cache: 0 2024-03-21T10:11:23.694 INFO:journalctl@ceph.mon.a.smithi047.stdout: initial_auto_readahead_size: 8192 2024-03-21T10:11:23.694 INFO:journalctl@ceph.mon.a.smithi047.stdout: num_file_reads_for_auto_readahead: 2 2024-03-21T10:11:23.694 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:11:23 smithi047 ceph-mon[31547]: rocksdb: Options.write_buffer_size: 33554432 2024-03-21T10:11:23.694 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:11:23 smithi047 ceph-mon[31547]: rocksdb: Options.max_write_buffer_number: 2 2024-03-21T10:11:23.695 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:11:23 smithi047 ceph-mon[31547]: rocksdb: Options.compression: NoCompression 2024-03-21T10:11:23.695 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:11:23 smithi047 ceph-mon[31547]: rocksdb: Options.bottommost_compression: Disabled 2024-03-21T10:11:23.695 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:11:23 smithi047 ceph-mon[31547]: rocksdb: Options.prefix_extractor: nullptr 2024-03-21T10:11:23.695 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:11:23 smithi047 ceph-mon[31547]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-03-21T10:11:23.695 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:11:23 smithi047 ceph-mon[31547]: rocksdb: Options.num_levels: 7 2024-03-21T10:11:23.695 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:11:23 smithi047 ceph-mon[31547]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-03-21T10:11:23.696 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:11:23 smithi047 ceph-mon[31547]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-03-21T10:11:23.696 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:11:23 smithi047 ceph-mon[31547]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-03-21T10:11:23.696 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:11:23 smithi047 ceph-mon[31547]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-03-21T10:11:23.696 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:11:23 smithi047 ceph-mon[31547]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-03-21T10:11:23.696 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:11:23 smithi047 ceph-mon[31547]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-03-21T10:11:23.696 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:11:23 smithi047 ceph-mon[31547]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-03-21T10:11:23.696 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:11:23 smithi047 ceph-mon[31547]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-03-21T10:11:23.696 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:11:23 smithi047 ceph-mon[31547]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-03-21T10:11:23.696 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:11:23 smithi047 ceph-mon[31547]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-03-21T10:11:23.696 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:11:23 smithi047 ceph-mon[31547]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-03-21T10:11:23.697 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:11:23 smithi047 ceph-mon[31547]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-03-21T10:11:23.697 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:11:23 smithi047 ceph-mon[31547]: rocksdb: Options.compression_opts.window_bits: -14 2024-03-21T10:11:23.697 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:11:23 smithi047 ceph-mon[31547]: rocksdb: Options.compression_opts.level: 32767 2024-03-21T10:11:23.697 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:11:23 smithi047 ceph-mon[31547]: rocksdb: Options.compression_opts.strategy: 0 2024-03-21T10:11:23.697 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:11:23 smithi047 ceph-mon[31547]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-03-21T10:11:23.697 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:11:23 smithi047 ceph-mon[31547]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-03-21T10:11:23.697 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:11:23 smithi047 ceph-mon[31547]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-03-21T10:11:23.697 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:11:23 smithi047 ceph-mon[31547]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-03-21T10:11:23.697 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:11:23 smithi047 ceph-mon[31547]: rocksdb: Options.compression_opts.enabled: false 2024-03-21T10:11:23.697 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:11:23 smithi047 ceph-mon[31547]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-03-21T10:11:23.697 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:11:23 smithi047 ceph-mon[31547]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-03-21T10:11:23.697 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:11:23 smithi047 ceph-mon[31547]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-03-21T10:11:23.698 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:11:23 smithi047 ceph-mon[31547]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-03-21T10:11:23.698 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:11:23 smithi047 ceph-mon[31547]: rocksdb: Options.target_file_size_base: 67108864 2024-03-21T10:11:23.698 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:11:23 smithi047 ceph-mon[31547]: rocksdb: Options.target_file_size_multiplier: 1 2024-03-21T10:11:23.698 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:11:23 smithi047 ceph-mon[31547]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-03-21T10:11:23.698 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:11:23 smithi047 ceph-mon[31547]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-03-21T10:11:23.698 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:11:23 smithi047 ceph-mon[31547]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-03-21T10:11:23.698 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:11:23 smithi047 ceph-mon[31547]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-03-21T10:11:23.698 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:11:23 smithi047 ceph-mon[31547]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-03-21T10:11:23.698 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:11:23 smithi047 ceph-mon[31547]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-03-21T10:11:23.698 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:11:23 smithi047 ceph-mon[31547]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-03-21T10:11:23.698 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:11:23 smithi047 ceph-mon[31547]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-03-21T10:11:23.699 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:11:23 smithi047 ceph-mon[31547]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-03-21T10:11:23.699 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:11:23 smithi047 ceph-mon[31547]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-03-21T10:11:23.699 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:11:23 smithi047 ceph-mon[31547]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-03-21T10:11:23.699 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:11:23 smithi047 ceph-mon[31547]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-03-21T10:11:23.699 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:11:23 smithi047 ceph-mon[31547]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-03-21T10:11:23.699 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:11:23 smithi047 ceph-mon[31547]: rocksdb: Options.arena_block_size: 1048576 2024-03-21T10:11:23.699 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:11:23 smithi047 ceph-mon[31547]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-03-21T10:11:23.699 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:11:23 smithi047 ceph-mon[31547]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-03-21T10:11:23.699 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:11:23 smithi047 ceph-mon[31547]: rocksdb: Options.disable_auto_compactions: 0 2024-03-21T10:11:23.699 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:11:23 smithi047 ceph-mon[31547]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-03-21T10:11:23.699 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:11:23 smithi047 ceph-mon[31547]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-03-21T10:11:23.699 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:11:23 smithi047 ceph-mon[31547]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-03-21T10:11:23.699 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:11:23 smithi047 ceph-mon[31547]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-03-21T10:11:23.700 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:11:23 smithi047 ceph-mon[31547]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-03-21T10:11:23.700 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:11:23 smithi047 ceph-mon[31547]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-03-21T10:11:23.700 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:11:23 smithi047 ceph-mon[31547]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-03-21T10:11:23.700 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:11:23 smithi047 ceph-mon[31547]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-03-21T10:11:23.700 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:11:23 smithi047 ceph-mon[31547]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-03-21T10:11:23.700 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:11:23 smithi047 ceph-mon[31547]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-03-21T10:11:23.700 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:11:23 smithi047 ceph-mon[31547]: rocksdb: Options.table_properties_collectors: 2024-03-21T10:11:23.700 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:11:23 smithi047 ceph-mon[31547]: rocksdb: Options.inplace_update_support: 0 2024-03-21T10:11:23.700 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:11:23 smithi047 ceph-mon[31547]: rocksdb: Options.inplace_update_num_locks: 10000 2024-03-21T10:11:23.700 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:11:23 smithi047 ceph-mon[31547]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-03-21T10:11:23.700 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:11:23 smithi047 ceph-mon[31547]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-03-21T10:11:23.701 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:11:23 smithi047 ceph-mon[31547]: rocksdb: Options.memtable_huge_page_size: 0 2024-03-21T10:11:23.701 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:11:23 smithi047 ceph-mon[31547]: rocksdb: Options.bloom_locality: 0 2024-03-21T10:11:23.701 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:11:23 smithi047 ceph-mon[31547]: rocksdb: Options.max_successive_merges: 0 2024-03-21T10:11:23.701 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:11:23 smithi047 ceph-mon[31547]: rocksdb: Options.optimize_filters_for_hits: 0 2024-03-21T10:11:23.701 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:11:23 smithi047 ceph-mon[31547]: rocksdb: Options.paranoid_file_checks: 0 2024-03-21T10:11:23.701 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:11:23 smithi047 ceph-mon[31547]: rocksdb: Options.force_consistency_checks: 1 2024-03-21T10:11:23.701 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:11:23 smithi047 ceph-mon[31547]: rocksdb: Options.report_bg_io_stats: 0 2024-03-21T10:11:23.701 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:11:23 smithi047 ceph-mon[31547]: rocksdb: Options.ttl: 2592000 2024-03-21T10:11:23.701 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:11:23 smithi047 ceph-mon[31547]: rocksdb: Options.periodic_compaction_seconds: 0 2024-03-21T10:11:23.701 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:11:23 smithi047 ceph-mon[31547]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-03-21T10:11:23.701 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:11:23 smithi047 ceph-mon[31547]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-03-21T10:11:23.702 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:11:23 smithi047 ceph-mon[31547]: rocksdb: Options.enable_blob_files: false 2024-03-21T10:11:23.702 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:11:23 smithi047 ceph-mon[31547]: rocksdb: Options.min_blob_size: 0 2024-03-21T10:11:23.702 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:11:23 smithi047 ceph-mon[31547]: rocksdb: Options.blob_file_size: 268435456 2024-03-21T10:11:23.702 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:11:23 smithi047 ceph-mon[31547]: rocksdb: Options.blob_compression_type: NoCompression 2024-03-21T10:11:23.702 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:11:23 smithi047 ceph-mon[31547]: rocksdb: Options.enable_blob_garbage_collection: false 2024-03-21T10:11:23.702 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:11:23 smithi047 ceph-mon[31547]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-03-21T10:11:23.702 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:11:23 smithi047 ceph-mon[31547]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-03-21T10:11:23.702 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:11:23 smithi047 ceph-mon[31547]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-03-21T10:11:23.702 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:11:23 smithi047 ceph-mon[31547]: rocksdb: Options.blob_file_starting_level: 0 2024-03-21T10:11:23.702 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:11:23 smithi047 ceph-mon[31547]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-03-21T10:11:23.702 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:11:23 smithi047 ceph-mon[31547]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 succeeded,manifest_file_number is 10, next_file_number is 12, last_sequence is 5, log_number is 5,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 5 2024-03-21T10:11:23.702 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:11:23 smithi047 ceph-mon[31547]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 5 2024-03-21T10:11:23.703 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:11:23 smithi047 ceph-mon[31547]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: 753fb1d3-56b6-42e0-81b2-6302e699ae68 2024-03-21T10:11:23.703 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:11:23 smithi047 ceph-mon[31547]: rocksdb: EVENT_LOG_v1 {"time_micros": 1711015883342306, "job": 1, "event": "recovery_started", "wal_files": [9]} 2024-03-21T10:11:23.703 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:11:23 smithi047 ceph-mon[31547]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #9 mode 2 2024-03-21T10:11:23.703 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:11:23 smithi047 ceph-mon[31547]: rocksdb: EVENT_LOG_v1 {"time_micros": 1711015883344151, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 13, "file_size": 94226, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 8, "largest_seqno": 312, "table_properties": {"data_size": 92254, "index_size": 254, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 709, "raw_key_size": 13505, "raw_average_key_size": 49, "raw_value_size": 84890, "raw_average_value_size": 313, "num_data_blocks": 11, "num_entries": 271, "num_filter_entries": 271, "num_deletions": 3, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1711015883, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "753fb1d3-56b6-42e0-81b2-6302e699ae68", "db_session_id": "WBXTRXU9QVE5LE0ACB7F", "orig_file_number": 13, "seqno_to_time_mapping": "N/A"}} 2024-03-21T10:11:23.703 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:11:23 smithi047 ceph-mon[31547]: rocksdb: EVENT_LOG_v1 {"time_micros": 1711015883344304, "job": 1, "event": "recovery_finished"} 2024-03-21T10:11:23.703 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:11:23 smithi047 ceph-mon[31547]: rocksdb: [db/version_set.cc:5047] Creating manifest 15 2024-03-21T10:11:23.703 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:11:23 smithi047 ceph-mon[31547]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-a/store.db/000009.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-03-21T10:11:23.703 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:11:23 smithi047 ceph-mon[31547]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x55800a742000 2024-03-21T10:11:23.703 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:11:23 smithi047 ceph-mon[31547]: rocksdb: DB pointer 0x55800a72c000 2024-03-21T10:11:23.703 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:11:23 smithi047 ceph-mon[31547]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-03-21T10:11:23.703 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:11:23 smithi047 ceph-mon[31547]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-03-21T10:11:23.703 INFO:journalctl@ceph.mon.a.smithi047.stdout: ** DB Stats ** 2024-03-21T10:11:23.703 INFO:journalctl@ceph.mon.a.smithi047.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-03-21T10:11:23.704 INFO:journalctl@ceph.mon.a.smithi047.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-03-21T10:11:23.704 INFO:journalctl@ceph.mon.a.smithi047.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-03-21T10:11:23.704 INFO:journalctl@ceph.mon.a.smithi047.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-03-21T10:11:23.704 INFO:journalctl@ceph.mon.a.smithi047.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-03-21T10:11:23.704 INFO:journalctl@ceph.mon.a.smithi047.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-03-21T10:11:23.704 INFO:journalctl@ceph.mon.a.smithi047.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-03-21T10:11:23.704 INFO:journalctl@ceph.mon.a.smithi047.stdout: 2024-03-21T10:11:23.704 INFO:journalctl@ceph.mon.a.smithi047.stdout: ** Compaction Stats [default] ** 2024-03-21T10:11:23.704 INFO:journalctl@ceph.mon.a.smithi047.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-03-21T10:11:23.704 INFO:journalctl@ceph.mon.a.smithi047.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-03-21T10:11:23.704 INFO:journalctl@ceph.mon.a.smithi047.stdout: L0 2/0 93.85 KB 0.5 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 75.6 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-03-21T10:11:23.704 INFO:journalctl@ceph.mon.a.smithi047.stdout: Sum 2/0 93.85 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 75.6 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-03-21T10:11:23.704 INFO:journalctl@ceph.mon.a.smithi047.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 75.6 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-03-21T10:11:23.704 INFO:journalctl@ceph.mon.a.smithi047.stdout: 2024-03-21T10:11:23.705 INFO:journalctl@ceph.mon.a.smithi047.stdout: ** Compaction Stats [default] ** 2024-03-21T10:11:23.705 INFO:journalctl@ceph.mon.a.smithi047.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-03-21T10:11:23.705 INFO:journalctl@ceph.mon.a.smithi047.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-03-21T10:11:23.705 INFO:journalctl@ceph.mon.a.smithi047.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 75.6 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-03-21T10:11:23.705 INFO:journalctl@ceph.mon.a.smithi047.stdout: 2024-03-21T10:11:23.705 INFO:journalctl@ceph.mon.a.smithi047.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-03-21T10:11:23.705 INFO:journalctl@ceph.mon.a.smithi047.stdout: 2024-03-21T10:11:23.705 INFO:journalctl@ceph.mon.a.smithi047.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-03-21T10:11:23.705 INFO:journalctl@ceph.mon.a.smithi047.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-03-21T10:11:23.705 INFO:journalctl@ceph.mon.a.smithi047.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-03-21T10:11:23.705 INFO:journalctl@ceph.mon.a.smithi047.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-03-21T10:11:23.705 INFO:journalctl@ceph.mon.a.smithi047.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-03-21T10:11:23.705 INFO:journalctl@ceph.mon.a.smithi047.stdout: AddFile(Keys): cumulative 0, interval 0 2024-03-21T10:11:23.706 INFO:journalctl@ceph.mon.a.smithi047.stdout: Cumulative compaction: 0.00 GB write, 13.06 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-03-21T10:11:23.706 INFO:journalctl@ceph.mon.a.smithi047.stdout: Interval compaction: 0.00 GB write, 13.06 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-03-21T10:11:23.706 INFO:journalctl@ceph.mon.a.smithi047.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-03-21T10:11:23.706 INFO:journalctl@ceph.mon.a.smithi047.stdout: Block cache BinnedLRUCache@0x55800a64f090#2 capacity: 512.00 MB usage: 1.33 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 2.8e-05 secs_since: 0 2024-03-21T10:11:23.706 INFO:journalctl@ceph.mon.a.smithi047.stdout: Block cache entry stats(count,size,portion): FilterBlock(2,0.89 KB,0.000169873%) IndexBlock(2,0.44 KB,8.34465e-05%) Misc(1,0.00 KB,0%) 2024-03-21T10:11:23.706 INFO:journalctl@ceph.mon.a.smithi047.stdout: 2024-03-21T10:11:23.706 INFO:journalctl@ceph.mon.a.smithi047.stdout: ** File Read Latency Histogram By Level [default] ** 2024-03-21T10:11:23.706 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:11:23 smithi047 ceph-mon[31547]: starting mon.a rank 0 at public addrs [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] at bind addrs [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon_data /var/lib/ceph/mon/ceph-a fsid 3056d51a-e76b-11ee-95cd-87774f69a715 2024-03-21T10:11:23.706 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:11:23 smithi047 ceph-mon[31547]: mon.a@-1(???) e1 preinit fsid 3056d51a-e76b-11ee-95cd-87774f69a715 2024-03-21T10:11:23.706 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:11:23 smithi047 ceph-mon[31547]: mon.a@-1(???).mds e1 new map 2024-03-21T10:11:23.706 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:11:23 smithi047 ceph-mon[31547]: mon.a@-1(???).mds e1 print_map 2024-03-21T10:11:23.707 INFO:journalctl@ceph.mon.a.smithi047.stdout: e1 2024-03-21T10:11:23.707 INFO:journalctl@ceph.mon.a.smithi047.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-03-21T10:11:23.707 INFO:journalctl@ceph.mon.a.smithi047.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments} 2024-03-21T10:11:23.707 INFO:journalctl@ceph.mon.a.smithi047.stdout: legacy client fscid: -1 2024-03-21T10:11:23.707 INFO:journalctl@ceph.mon.a.smithi047.stdout: 2024-03-21T10:11:23.707 INFO:journalctl@ceph.mon.a.smithi047.stdout: No filesystems configured 2024-03-21T10:11:23.707 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:11:23 smithi047 ceph-mon[31547]: mon.a@-1(???).osd e1 crush map has features 3314932999778484224, adjusting msgr requires 2024-03-21T10:11:23.707 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:11:23 smithi047 ceph-mon[31547]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-03-21T10:11:23.707 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:11:23 smithi047 ceph-mon[31547]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-03-21T10:11:23.707 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:11:23 smithi047 ceph-mon[31547]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-03-21T10:11:23.707 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:11:23 smithi047 ceph-mon[31547]: mon.a@-1(???).paxosservice(auth 1..2) refresh upgraded, format 0 -> 3 2024-03-21T10:11:23.707 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:11:23 smithi047 bash[31529]: e43a05e9bf7b17a59e76c9f2a93da51a6c01f99431f509aeb2d6d80f76fc6414 2024-03-21T10:11:23.707 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:11:23 smithi047 systemd[1]: Started Ceph mon.a for 3056d51a-e76b-11ee-95cd-87774f69a715. 2024-03-21T10:11:23.707 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:11:23 smithi047 ceph-mon[31547]: mon.a is new leader, mons a in quorum (ranks 0) 2024-03-21T10:11:23.708 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:11:23 smithi047 ceph-mon[31547]: monmap e1: 1 mons at {a=[v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-03-21T10:11:23.708 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:11:23 smithi047 ceph-mon[31547]: fsmap 2024-03-21T10:11:23.708 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:11:23 smithi047 ceph-mon[31547]: osdmap e1: 0 total, 0 up, 0 in 2024-03-21T10:11:23.708 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:11:23 smithi047 ceph-mon[31547]: mgrmap e1: no daemons active 2024-03-21T10:11:24.725 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:23.988+0000 7f7f5fa00700 1 Processor -- start 2024-03-21T10:11:24.725 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:23.988+0000 7f7f5fa00700 1 -- start start 2024-03-21T10:11:24.725 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:23.988+0000 7f7f5fa00700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7f58109ba0 0x7f7f58109fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:11:24.725 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:23.988+0000 7f7f5fa00700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f5810a590 con 0x7f7f58109ba0 2024-03-21T10:11:24.725 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:23.989+0000 7f7f5d000700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7f58109ba0 0x7f7f58109fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:11:24.725 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:23.989+0000 7f7f5d000700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7f58109ba0 0x7f7f58109fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:57464/0 (socket says 172.21.15.47:57464) 2024-03-21T10:11:24.725 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:23.989+0000 7f7f5d000700 1 -- 172.21.15.47:0/3306997771 learned_addr learned my addr 172.21.15.47:0/3306997771 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:11:24.726 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:23.989+0000 7f7f5d000700 1 -- 172.21.15.47:0/3306997771 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f5810ad00 con 0x7f7f58109ba0 2024-03-21T10:11:24.726 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:23.990+0000 7f7f5d000700 1 --2- 172.21.15.47:0/3306997771 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7f58109ba0 0x7f7f58109fc0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f7f4c0098a0 tx=0x7f7f4c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=45ae25804188a512 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:11:24.726 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:23.990+0000 7f7f57400700 1 -- 172.21.15.47:0/3306997771 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f7f4c004160 con 0x7f7f58109ba0 2024-03-21T10:11:24.729 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:23.990+0000 7f7f57400700 1 -- 172.21.15.47:0/3306997771 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) v1 ==== 1071+0+0 (secure 0 0 0) 0x7f7f4c004300 con 0x7f7f58109ba0 2024-03-21T10:11:24.729 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:23.990+0000 7f7f57400700 1 -- 172.21.15.47:0/3306997771 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f7f4c011a80 con 0x7f7f58109ba0 2024-03-21T10:11:24.729 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:23.991+0000 7f7f5fa00700 1 -- 172.21.15.47:0/3306997771 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7f58109ba0 msgr2=0x7f7f58109fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:11:24.730 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:23.991+0000 7f7f5fa00700 1 --2- 172.21.15.47:0/3306997771 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7f58109ba0 0x7f7f58109fc0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f7f4c0098a0 tx=0x7f7f4c009bb0 comp rx=0 tx=0).stop 2024-03-21T10:11:24.730 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:23.991+0000 7f7f5fa00700 1 -- 172.21.15.47:0/3306997771 shutdown_connections 2024-03-21T10:11:24.730 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:23.991+0000 7f7f5fa00700 1 --2- 172.21.15.47:0/3306997771 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7f58109ba0 0x7f7f58109fc0 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:11:24.730 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:23.991+0000 7f7f5fa00700 1 -- 172.21.15.47:0/3306997771 >> 172.21.15.47:0/3306997771 conn(0x7f7f581054a0 msgr2=0x7f7f58107890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:11:24.730 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:23.991+0000 7f7f5fa00700 1 -- 172.21.15.47:0/3306997771 shutdown_connections 2024-03-21T10:11:24.730 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:23.991+0000 7f7f5fa00700 1 -- 172.21.15.47:0/3306997771 wait complete. 2024-03-21T10:11:24.730 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:23.992+0000 7f7f5fa00700 1 Processor -- start 2024-03-21T10:11:24.730 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:23.993+0000 7f7f5fa00700 1 -- start start 2024-03-21T10:11:24.730 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:23.993+0000 7f7f5fa00700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7f58109ba0 0x7f7f5819d9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:11:24.730 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:23.993+0000 7f7f5fa00700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f5810e0b0 con 0x7f7f58109ba0 2024-03-21T10:11:24.730 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:23.993+0000 7f7f5d000700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7f58109ba0 0x7f7f5819d9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:11:24.730 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:23.993+0000 7f7f5d000700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7f58109ba0 0x7f7f5819d9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:57476/0 (socket says 172.21.15.47:57476) 2024-03-21T10:11:24.731 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:23.993+0000 7f7f5d000700 1 -- 172.21.15.47:0/3647841326 learned_addr learned my addr 172.21.15.47:0/3647841326 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:11:24.731 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:23.994+0000 7f7f5d000700 1 -- 172.21.15.47:0/3647841326 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f5819df30 con 0x7f7f58109ba0 2024-03-21T10:11:24.731 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:23.994+0000 7f7f5d000700 1 --2- 172.21.15.47:0/3647841326 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7f58109ba0 0x7f7f5819d9f0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f7f4c000c00 tx=0x7f7f4c004290 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:11:24.731 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:23.994+0000 7f7f55600700 1 -- 172.21.15.47:0/3647841326 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f7f4c0039c0 con 0x7f7f58109ba0 2024-03-21T10:11:24.731 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:23.994+0000 7f7f55600700 1 -- 172.21.15.47:0/3647841326 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) v1 ==== 1071+0+0 (secure 0 0 0) 0x7f7f4c003b60 con 0x7f7f58109ba0 2024-03-21T10:11:24.731 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:23.994+0000 7f7f55600700 1 -- 172.21.15.47:0/3647841326 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f7f4c01a6f0 con 0x7f7f58109ba0 2024-03-21T10:11:24.731 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:23.994+0000 7f7f5fa00700 1 -- 172.21.15.47:0/3647841326 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7f5819e1c0 con 0x7f7f58109ba0 2024-03-21T10:11:24.731 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:23.995+0000 7f7f5fa00700 1 -- 172.21.15.47:0/3647841326 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7f581a0eb0 con 0x7f7f58109ba0 2024-03-21T10:11:24.731 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:23.995+0000 7f7f55600700 1 -- 172.21.15.47:0/3647841326 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (secure 0 0 0) 0x7f7f4c01ac20 con 0x7f7f58109ba0 2024-03-21T10:11:24.731 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:23.995+0000 7f7f55600700 1 -- 172.21.15.47:0/3647841326 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f7f4c019700 con 0x7f7f58109ba0 2024-03-21T10:11:24.731 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:23.996+0000 7f7f5fa00700 1 -- 172.21.15.47:0/3647841326 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7f580637c0 con 0x7f7f58109ba0 2024-03-21T10:11:24.731 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:23.999+0000 7f7f55600700 1 -- 172.21.15.47:0/3647841326 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73141 (secure 0 0 0) 0x7f7f4c004300 con 0x7f7f58109ba0 2024-03-21T10:11:24.731 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:24.076+0000 7f7f5fa00700 1 -- 172.21.15.47:0/3647841326 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command([{prefix=config set, name=public_network}] v 0) v1 -- 0x7f7f581a1080 con 0x7f7f58109ba0 2024-03-21T10:11:24.732 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:24.080+0000 7f7f55600700 1 -- 172.21.15.47:0/3647841326 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{prefix=config set, name=public_network}]=0 v3)=0 v3) v1 ==== 124+0+0 (secure 0 0 0) 0x7f7f4c024a40 con 0x7f7f58109ba0 2024-03-21T10:11:24.732 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:24.081+0000 7f7f5fa00700 1 -- 172.21.15.47:0/3647841326 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7f58109ba0 msgr2=0x7f7f5819d9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:11:24.732 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:24.081+0000 7f7f5fa00700 1 --2- 172.21.15.47:0/3647841326 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7f58109ba0 0x7f7f5819d9f0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f7f4c000c00 tx=0x7f7f4c004290 comp rx=0 tx=0).stop 2024-03-21T10:11:24.732 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:24.082+0000 7f7f5fa00700 1 -- 172.21.15.47:0/3647841326 shutdown_connections 2024-03-21T10:11:24.732 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:24.082+0000 7f7f5fa00700 1 --2- 172.21.15.47:0/3647841326 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7f58109ba0 0x7f7f5819d9f0 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:11:24.732 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:24.082+0000 7f7f5fa00700 1 -- 172.21.15.47:0/3647841326 >> 172.21.15.47:0/3647841326 conn(0x7f7f581054a0 msgr2=0x7f7f58106ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:11:24.732 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:24.082+0000 7f7f5fa00700 1 -- 172.21.15.47:0/3647841326 shutdown_connections 2024-03-21T10:11:24.732 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:24.082+0000 7f7f5fa00700 1 -- 172.21.15.47:0/3647841326 wait complete. 2024-03-21T10:11:24.732 INFO:teuthology.orchestra.run.smithi047.stdout:Wrote config to /etc/ceph/ceph.conf 2024-03-21T10:11:24.759 INFO:teuthology.orchestra.run.smithi047.stdout:Wrote keyring to /etc/ceph/ceph.client.admin.keyring 2024-03-21T10:11:24.759 INFO:teuthology.orchestra.run.smithi047.stdout:Creating mgr... 2024-03-21T10:11:24.760 INFO:teuthology.orchestra.run.smithi047.stdout:Verifying port 0.0.0.0:9283 ... 2024-03-21T10:11:24.762 INFO:teuthology.orchestra.run.smithi047.stdout:Verifying port 0.0.0.0:8765 ... 2024-03-21T10:11:25.210 INFO:teuthology.orchestra.run.smithi047.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-3056d51a-e76b-11ee-95cd-87774f69a715@mgr.x 2024-03-21T10:11:25.210 INFO:teuthology.orchestra.run.smithi047.stdout:systemctl: stderr Failed to reset failed state of unit ceph-3056d51a-e76b-11ee-95cd-87774f69a715@mgr.x.service: Unit ceph-3056d51a-e76b-11ee-95cd-87774f69a715@mgr.x.service not loaded. 2024-03-21T10:11:25.463 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:11:25 smithi047 ceph-mon[31547]: from='client.? 172.21.15.47:0/3647841326' entity='client.admin' 2024-03-21T10:11:25.485 INFO:teuthology.orchestra.run.smithi047.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-3056d51a-e76b-11ee-95cd-87774f69a715.target.wants/ceph-3056d51a-e76b-11ee-95cd-87774f69a715@mgr.x.service → /etc/systemd/system/ceph-3056d51a-e76b-11ee-95cd-87774f69a715@.service. 2024-03-21T10:11:25.888 INFO:journalctl@ceph.mgr.x.smithi047.stdout:Mar 21 10:11:25 smithi047 systemd[1]: Starting Ceph mgr.x for 3056d51a-e76b-11ee-95cd-87774f69a715... 2024-03-21T10:11:26.142 INFO:journalctl@ceph.mgr.x.smithi047.stdout:Mar 21 10:11:25 smithi047 podman[31758]: 2024-03-21 10:11:25.794154425 +0000 UTC m=+0.036653013 image pull 237e43c7a65ff48534d94b97c1bfdbe4b7be5672e3d9a613745083ea3702aa54 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2edfee68ad7243bed422a049b6f5f58619300f11 2024-03-21T10:11:26.142 INFO:journalctl@ceph.mgr.x.smithi047.stdout:Mar 21 10:11:25 smithi047 podman[31758]: 2024-03-21 10:11:25.902535603 +0000 UTC m=+0.145034210 container create d2faba9824e03dbc0ebbad478602be6bfd4e2c2dafcee4d67e10d3e703ebaea9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2edfee68ad7243bed422a049b6f5f58619300f11, name=ceph-3056d51a-e76b-11ee-95cd-87774f69a715-mgr-x, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240319, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS) 2024-03-21T10:11:26.143 INFO:journalctl@ceph.mgr.x.smithi047.stdout:Mar 21 10:11:26 smithi047 podman[31758]: 2024-03-21 10:11:26.131417513 +0000 UTC m=+0.373916153 container init d2faba9824e03dbc0ebbad478602be6bfd4e2c2dafcee4d67e10d3e703ebaea9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2edfee68ad7243bed422a049b6f5f58619300f11, name=ceph-3056d51a-e76b-11ee-95cd-87774f69a715-mgr-x, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240319, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, ceph=True, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0) 2024-03-21T10:11:26.143 INFO:journalctl@ceph.mgr.x.smithi047.stdout:Mar 21 10:11:26 smithi047 podman[31758]: 2024-03-21 10:11:26.141701582 +0000 UTC m=+0.384200193 container start d2faba9824e03dbc0ebbad478602be6bfd4e2c2dafcee4d67e10d3e703ebaea9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2edfee68ad7243bed422a049b6f5f58619300f11, name=ceph-3056d51a-e76b-11ee-95cd-87774f69a715-mgr-x, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_CLEAN=True, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.build-date=20240319, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True) 2024-03-21T10:11:26.307 INFO:teuthology.orchestra.run.smithi047.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-03-21T10:11:26.307 INFO:teuthology.orchestra.run.smithi047.stdout:systemctl: stdout disabled 2024-03-21T10:11:26.327 INFO:teuthology.orchestra.run.smithi047.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-03-21T10:11:26.328 INFO:teuthology.orchestra.run.smithi047.stdout:systemctl: stdout inactive 2024-03-21T10:11:26.328 INFO:teuthology.orchestra.run.smithi047.stdout:firewalld.service is not enabled 2024-03-21T10:11:26.328 INFO:teuthology.orchestra.run.smithi047.stdout:Not possible to enable service . firewalld.service is not available 2024-03-21T10:11:26.354 INFO:teuthology.orchestra.run.smithi047.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-03-21T10:11:26.358 INFO:teuthology.orchestra.run.smithi047.stdout:systemctl: stdout disabled 2024-03-21T10:11:26.377 INFO:teuthology.orchestra.run.smithi047.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-03-21T10:11:26.377 INFO:teuthology.orchestra.run.smithi047.stdout:systemctl: stdout inactive 2024-03-21T10:11:26.377 INFO:teuthology.orchestra.run.smithi047.stdout:firewalld.service is not enabled 2024-03-21T10:11:26.378 INFO:teuthology.orchestra.run.smithi047.stdout:Not possible to open ports <[9283, 8765]>. firewalld.service is not available 2024-03-21T10:11:26.378 INFO:teuthology.orchestra.run.smithi047.stdout:Waiting for mgr to start... 2024-03-21T10:11:26.378 INFO:teuthology.orchestra.run.smithi047.stdout:Waiting for mgr... 2024-03-21T10:11:26.464 INFO:journalctl@ceph.mgr.x.smithi047.stdout:Mar 21 10:11:26 smithi047 bash[31758]: d2faba9824e03dbc0ebbad478602be6bfd4e2c2dafcee4d67e10d3e703ebaea9 2024-03-21T10:11:26.464 INFO:journalctl@ceph.mgr.x.smithi047.stdout:Mar 21 10:11:26 smithi047 systemd[1]: Started Ceph mgr.x for 3056d51a-e76b-11ee-95cd-87774f69a715. 2024-03-21T10:11:27.102 INFO:journalctl@ceph.mgr.x.smithi047.stdout:Mar 21 10:11:26 smithi047 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-mgr-x[31773]: 2024-03-21T10:11:26.933+0000 7fefd0081200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-03-21T10:11:27.102 INFO:journalctl@ceph.mgr.x.smithi047.stdout:Mar 21 10:11:27 smithi047 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-mgr-x[31773]: 2024-03-21T10:11:27.059+0000 7fefd0081200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-03-21T10:11:27.387 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:11:27 smithi047 ceph-mon[31547]: from='client.? 172.21.15.47:0/17345182' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-03-21T10:11:27.928 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout 2024-03-21T10:11:27.930 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout { 2024-03-21T10:11:27.930 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "fsid": "3056d51a-e76b-11ee-95cd-87774f69a715", 2024-03-21T10:11:27.930 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "health": { 2024-03-21T10:11:27.930 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-03-21T10:11:27.930 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-03-21T10:11:27.930 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-03-21T10:11:27.930 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout }, 2024-03-21T10:11:27.931 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-03-21T10:11:27.931 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-03-21T10:11:27.931 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout 0 2024-03-21T10:11:27.931 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout ], 2024-03-21T10:11:27.931 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-03-21T10:11:27.931 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "a" 2024-03-21T10:11:27.931 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout ], 2024-03-21T10:11:27.931 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "quorum_age": 3, 2024-03-21T10:11:27.933 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "monmap": { 2024-03-21T10:11:27.933 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-21T10:11:27.933 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-03-21T10:11:27.933 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-03-21T10:11:27.933 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout }, 2024-03-21T10:11:27.933 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-03-21T10:11:27.933 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-21T10:11:27.933 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-03-21T10:11:27.933 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-03-21T10:11:27.933 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-03-21T10:11:27.941 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-03-21T10:11:27.942 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-03-21T10:11:27.942 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-03-21T10:11:27.942 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout }, 2024-03-21T10:11:27.942 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-03-21T10:11:27.942 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-03-21T10:11:27.942 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-03-21T10:11:27.942 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-03-21T10:11:27.942 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-03-21T10:11:27.942 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-03-21T10:11:27.942 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-03-21T10:11:27.942 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-03-21T10:11:27.942 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-03-21T10:11:27.943 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout }, 2024-03-21T10:11:27.943 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-03-21T10:11:27.943 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-21T10:11:27.943 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-03-21T10:11:27.943 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-03-21T10:11:27.943 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout }, 2024-03-21T10:11:27.943 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-03-21T10:11:27.943 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "available": false, 2024-03-21T10:11:27.943 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-03-21T10:11:27.943 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "modules": [ 2024-03-21T10:11:27.943 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "iostat", 2024-03-21T10:11:27.943 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "nfs", 2024-03-21T10:11:27.943 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "restful" 2024-03-21T10:11:27.943 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout ], 2024-03-21T10:11:27.944 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "services": {} 2024-03-21T10:11:27.944 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout }, 2024-03-21T10:11:27.944 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-03-21T10:11:27.944 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-21T10:11:27.944 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "modified": "2024-03-21T10:11:17.020553+0000", 2024-03-21T10:11:27.944 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "services": {} 2024-03-21T10:11:27.944 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout }, 2024-03-21T10:11:27.944 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-03-21T10:11:27.944 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout } 2024-03-21T10:11:27.944 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:27.005+0000 7fc652000700 1 Processor -- start 2024-03-21T10:11:27.944 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:27.005+0000 7fc652000700 1 -- start start 2024-03-21T10:11:27.944 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:27.006+0000 7fc652000700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc64c108330 0x7fc64c10a720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:11:27.944 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:27.006+0000 7fc652000700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc64c07ae50 con 0x7fc64c108330 2024-03-21T10:11:27.944 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:27.006+0000 7fc64b400700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc64c108330 0x7fc64c10a720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:11:27.945 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:27.006+0000 7fc64b400700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc64c108330 0x7fc64c10a720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:57506/0 (socket says 172.21.15.47:57506) 2024-03-21T10:11:27.945 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:27.006+0000 7fc64b400700 1 -- 172.21.15.47:0/3964224643 learned_addr learned my addr 172.21.15.47:0/3964224643 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:11:27.945 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:27.007+0000 7fc64b400700 1 -- 172.21.15.47:0/3964224643 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc64c10ac60 con 0x7fc64c108330 2024-03-21T10:11:27.945 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:27.007+0000 7fc64b400700 1 --2- 172.21.15.47:0/3964224643 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc64c108330 0x7fc64c10a720 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7fc63c0098a0 tx=0x7fc63c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=13703e2a8e420d60 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:11:27.945 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:27.007+0000 7fc64a000700 1 -- 172.21.15.47:0/3964224643 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc63c004160 con 0x7fc64c108330 2024-03-21T10:11:27.945 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:27.007+0000 7fc64a000700 1 -- 172.21.15.47:0/3964224643 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) v1 ==== 1071+0+0 (secure 0 0 0) 0x7fc63c004300 con 0x7fc64c108330 2024-03-21T10:11:27.945 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:27.007+0000 7fc64a000700 1 -- 172.21.15.47:0/3964224643 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc63c011a80 con 0x7fc64c108330 2024-03-21T10:11:27.945 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:27.008+0000 7fc652000700 1 -- 172.21.15.47:0/3964224643 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc64c108330 msgr2=0x7fc64c10a720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:11:27.945 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:27.008+0000 7fc652000700 1 --2- 172.21.15.47:0/3964224643 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc64c108330 0x7fc64c10a720 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7fc63c0098a0 tx=0x7fc63c009bb0 comp rx=0 tx=0).stop 2024-03-21T10:11:27.945 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:27.008+0000 7fc652000700 1 -- 172.21.15.47:0/3964224643 shutdown_connections 2024-03-21T10:11:27.945 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:27.008+0000 7fc652000700 1 --2- 172.21.15.47:0/3964224643 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc64c108330 0x7fc64c10a720 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:11:27.945 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:27.008+0000 7fc652000700 1 -- 172.21.15.47:0/3964224643 >> 172.21.15.47:0/3964224643 conn(0x7fc64c101de0 msgr2=0x7fc64c104200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:11:27.945 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:27.009+0000 7fc652000700 1 -- 172.21.15.47:0/3964224643 shutdown_connections 2024-03-21T10:11:27.945 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:27.009+0000 7fc652000700 1 -- 172.21.15.47:0/3964224643 wait complete. 2024-03-21T10:11:27.946 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:27.010+0000 7fc652000700 1 Processor -- start 2024-03-21T10:11:27.946 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:27.010+0000 7fc652000700 1 -- start start 2024-03-21T10:11:27.946 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:27.010+0000 7fc652000700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc64c108330 0x7fc64c107bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:11:27.946 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:27.010+0000 7fc652000700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc64c10bc20 con 0x7fc64c108330 2024-03-21T10:11:27.946 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:27.010+0000 7fc64b400700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc64c108330 0x7fc64c107bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:11:27.946 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:27.010+0000 7fc64b400700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc64c108330 0x7fc64c107bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:57520/0 (socket says 172.21.15.47:57520) 2024-03-21T10:11:27.946 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:27.010+0000 7fc64b400700 1 -- 172.21.15.47:0/17345182 learned_addr learned my addr 172.21.15.47:0/17345182 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:11:27.946 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:27.011+0000 7fc64b400700 1 -- 172.21.15.47:0/17345182 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc64c1080f0 con 0x7fc64c108330 2024-03-21T10:11:27.946 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:27.011+0000 7fc64b400700 1 --2- 172.21.15.47:0/17345182 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc64c108330 0x7fc64c107bb0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fc63c011a50 tx=0x7fc63c004290 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:11:27.946 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:27.011+0000 7fc643e00700 1 -- 172.21.15.47:0/17345182 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc63c0039f0 con 0x7fc64c108330 2024-03-21T10:11:27.946 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:27.011+0000 7fc643e00700 1 -- 172.21.15.47:0/17345182 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) v1 ==== 1071+0+0 (secure 0 0 0) 0x7fc63c003b90 con 0x7fc64c108330 2024-03-21T10:11:27.946 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:27.011+0000 7fc652000700 1 -- 172.21.15.47:0/17345182 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc64c1062e0 con 0x7fc64c108330 2024-03-21T10:11:27.946 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:27.011+0000 7fc643e00700 1 -- 172.21.15.47:0/17345182 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc63c01a600 con 0x7fc64c108330 2024-03-21T10:11:27.947 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:27.011+0000 7fc652000700 1 -- 172.21.15.47:0/17345182 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc64c106820 con 0x7fc64c108330 2024-03-21T10:11:27.947 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:27.012+0000 7fc643e00700 1 -- 172.21.15.47:0/17345182 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (secure 0 0 0) 0x7fc63c01ac60 con 0x7fc64c108330 2024-03-21T10:11:27.947 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:27.012+0000 7fc643e00700 1 -- 172.21.15.47:0/17345182 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7fc63c019700 con 0x7fc64c108330 2024-03-21T10:11:27.947 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:27.012+0000 7fc652000700 1 -- 172.21.15.47:0/17345182 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc64c0637c0 con 0x7fc64c108330 2024-03-21T10:11:27.947 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:27.015+0000 7fc643e00700 1 -- 172.21.15.47:0/17345182 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73141 (secure 0 0 0) 0x7fc63c024a40 con 0x7fc64c108330 2024-03-21T10:11:27.947 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:27.095+0000 7fc652000700 1 -- 172.21.15.47:0/17345182 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) v1 -- 0x7fc64c106ad0 con 0x7fc64c108330 2024-03-21T10:11:27.947 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:27.096+0000 7fc643e00700 1 -- 172.21.15.47:0/17345182 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) v1 ==== 79+0+1239 (secure 0 0 0) 0x7fc63c024440 con 0x7fc64c108330 2024-03-21T10:11:27.947 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:27.097+0000 7fc652000700 1 -- 172.21.15.47:0/17345182 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc64c108330 msgr2=0x7fc64c107bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:11:27.947 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:27.097+0000 7fc652000700 1 --2- 172.21.15.47:0/17345182 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc64c108330 0x7fc64c107bb0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fc63c011a50 tx=0x7fc63c004290 comp rx=0 tx=0).stop 2024-03-21T10:11:27.947 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:27.097+0000 7fc652000700 1 -- 172.21.15.47:0/17345182 shutdown_connections 2024-03-21T10:11:27.947 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:27.097+0000 7fc652000700 1 --2- 172.21.15.47:0/17345182 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc64c108330 0x7fc64c107bb0 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:11:27.947 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:27.097+0000 7fc652000700 1 -- 172.21.15.47:0/17345182 >> 172.21.15.47:0/17345182 conn(0x7fc64c101de0 msgr2=0x7fc64c1028d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:11:27.947 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:27.098+0000 7fc652000700 1 -- 172.21.15.47:0/17345182 shutdown_connections 2024-03-21T10:11:27.947 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:27.098+0000 7fc652000700 1 -- 172.21.15.47:0/17345182 wait complete. 2024-03-21T10:11:27.948 INFO:teuthology.orchestra.run.smithi047.stdout:mgr not available, waiting (1/15)... 2024-03-21T10:11:28.214 INFO:journalctl@ceph.mgr.x.smithi047.stdout:Mar 21 10:11:27 smithi047 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-mgr-x[31773]: 2024-03-21T10:11:27.939+0000 7fefd0081200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-03-21T10:11:28.214 INFO:journalctl@ceph.mgr.x.smithi047.stdout:Mar 21 10:11:28 smithi047 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-mgr-x[31773]: 2024-03-21T10:11:28.063+0000 7fefd0081200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-03-21T10:11:28.714 INFO:journalctl@ceph.mgr.x.smithi047.stdout:Mar 21 10:11:28 smithi047 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-mgr-x[31773]: 2024-03-21T10:11:28.243+0000 7fefd0081200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-03-21T10:11:28.714 INFO:journalctl@ceph.mgr.x.smithi047.stdout:Mar 21 10:11:28 smithi047 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-mgr-x[31773]: 2024-03-21T10:11:28.369+0000 7fefd0081200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-03-21T10:11:28.714 INFO:journalctl@ceph.mgr.x.smithi047.stdout:Mar 21 10:11:28 smithi047 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-mgr-x[31773]: 2024-03-21T10:11:28.483+0000 7fefd0081200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-03-21T10:11:29.086 INFO:journalctl@ceph.mgr.x.smithi047.stdout:Mar 21 10:11:28 smithi047 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-mgr-x[31773]: 2024-03-21T10:11:28.825+0000 7fefd0081200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-03-21T10:11:29.086 INFO:journalctl@ceph.mgr.x.smithi047.stdout:Mar 21 10:11:28 smithi047 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-mgr-x[31773]: 2024-03-21T10:11:28.970+0000 7fefd0081200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-03-21T10:11:29.086 INFO:journalctl@ceph.mgr.x.smithi047.stdout:Mar 21 10:11:29 smithi047 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-mgr-x[31773]: 2024-03-21T10:11:29.084+0000 7fefd0081200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-03-21T10:11:29.434 INFO:journalctl@ceph.mgr.x.smithi047.stdout:Mar 21 10:11:29 smithi047 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-mgr-x[31773]: 2024-03-21T10:11:29.212+0000 7fefd0081200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-03-21T10:11:29.434 INFO:journalctl@ceph.mgr.x.smithi047.stdout:Mar 21 10:11:29 smithi047 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-mgr-x[31773]: 2024-03-21T10:11:29.432+0000 7fefd0081200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-03-21T10:11:30.930 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:11:30 smithi047 ceph-mon[31547]: from='client.? 172.21.15.47:0/1413333410' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-03-21T10:11:30.930 INFO:journalctl@ceph.mgr.x.smithi047.stdout:Mar 21 10:11:30 smithi047 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-mgr-x[31773]: 2024-03-21T10:11:30.813+0000 7fefd0081200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-03-21T10:11:31.214 INFO:journalctl@ceph.mgr.x.smithi047.stdout:Mar 21 10:11:30 smithi047 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-mgr-x[31773]: 2024-03-21T10:11:30.993+0000 7fefd0081200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-03-21T10:11:31.337 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout 2024-03-21T10:11:31.337 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout { 2024-03-21T10:11:31.338 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "fsid": "3056d51a-e76b-11ee-95cd-87774f69a715", 2024-03-21T10:11:31.338 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "health": { 2024-03-21T10:11:31.338 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-03-21T10:11:31.338 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-03-21T10:11:31.338 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-03-21T10:11:31.343 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout }, 2024-03-21T10:11:31.343 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-03-21T10:11:31.343 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-03-21T10:11:31.343 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout 0 2024-03-21T10:11:31.343 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout ], 2024-03-21T10:11:31.343 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-03-21T10:11:31.343 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "a" 2024-03-21T10:11:31.344 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout ], 2024-03-21T10:11:31.344 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "quorum_age": 7, 2024-03-21T10:11:31.344 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "monmap": { 2024-03-21T10:11:31.344 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-21T10:11:31.344 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-03-21T10:11:31.344 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-03-21T10:11:31.344 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout }, 2024-03-21T10:11:31.344 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-03-21T10:11:31.344 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-21T10:11:31.344 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-03-21T10:11:31.344 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-03-21T10:11:31.344 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-03-21T10:11:31.344 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-03-21T10:11:31.344 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-03-21T10:11:31.344 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-03-21T10:11:31.345 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout }, 2024-03-21T10:11:31.345 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-03-21T10:11:31.345 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-03-21T10:11:31.345 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-03-21T10:11:31.345 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-03-21T10:11:31.345 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-03-21T10:11:31.345 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-03-21T10:11:31.345 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-03-21T10:11:31.345 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-03-21T10:11:31.345 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-03-21T10:11:31.345 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout }, 2024-03-21T10:11:31.345 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-03-21T10:11:31.345 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-21T10:11:31.345 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-03-21T10:11:31.345 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-03-21T10:11:31.346 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout }, 2024-03-21T10:11:31.346 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-03-21T10:11:31.346 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "available": false, 2024-03-21T10:11:31.346 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-03-21T10:11:31.346 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "modules": [ 2024-03-21T10:11:31.346 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "iostat", 2024-03-21T10:11:31.346 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "nfs", 2024-03-21T10:11:31.346 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "restful" 2024-03-21T10:11:31.346 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout ], 2024-03-21T10:11:31.346 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "services": {} 2024-03-21T10:11:31.346 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout }, 2024-03-21T10:11:31.346 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-03-21T10:11:31.346 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-21T10:11:31.351 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "modified": "2024-03-21T10:11:17.020553+0000", 2024-03-21T10:11:31.352 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "services": {} 2024-03-21T10:11:31.352 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout }, 2024-03-21T10:11:31.352 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-03-21T10:11:31.352 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout } 2024-03-21T10:11:31.352 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:30.561+0000 7ff429200700 1 Processor -- start 2024-03-21T10:11:31.352 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:30.561+0000 7ff429200700 1 -- start start 2024-03-21T10:11:31.352 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:30.562+0000 7ff429200700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff424108310 0x7ff42410a700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:11:31.352 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:30.562+0000 7ff429200700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff42407ae70 con 0x7ff424108310 2024-03-21T10:11:31.352 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:30.562+0000 7ff421e00700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff424108310 0x7ff42410a700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:11:31.352 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:30.562+0000 7ff421e00700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff424108310 0x7ff42410a700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:57532/0 (socket says 172.21.15.47:57532) 2024-03-21T10:11:31.352 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:30.562+0000 7ff421e00700 1 -- 172.21.15.47:0/2804738595 learned_addr learned my addr 172.21.15.47:0/2804738595 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:11:31.352 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:30.563+0000 7ff421e00700 1 -- 172.21.15.47:0/2804738595 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff42410ac40 con 0x7ff424108310 2024-03-21T10:11:31.352 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:30.563+0000 7ff421e00700 1 --2- 172.21.15.47:0/2804738595 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff424108310 0x7ff42410a700 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7ff4140098a0 tx=0x7ff414009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1d091f678fb5ab80 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:11:31.353 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:30.563+0000 7ff420a00700 1 -- 172.21.15.47:0/2804738595 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ff414004160 con 0x7ff424108310 2024-03-21T10:11:31.353 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:30.563+0000 7ff420a00700 1 -- 172.21.15.47:0/2804738595 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) v1 ==== 1071+0+0 (secure 0 0 0) 0x7ff414004300 con 0x7ff424108310 2024-03-21T10:11:31.353 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:30.563+0000 7ff420a00700 1 -- 172.21.15.47:0/2804738595 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ff414011a80 con 0x7ff424108310 2024-03-21T10:11:31.353 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:30.564+0000 7ff429200700 1 -- 172.21.15.47:0/2804738595 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff424108310 msgr2=0x7ff42410a700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:11:31.353 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:30.564+0000 7ff429200700 1 --2- 172.21.15.47:0/2804738595 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff424108310 0x7ff42410a700 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7ff4140098a0 tx=0x7ff414009bb0 comp rx=0 tx=0).stop 2024-03-21T10:11:31.353 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:30.564+0000 7ff429200700 1 -- 172.21.15.47:0/2804738595 shutdown_connections 2024-03-21T10:11:31.353 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:30.564+0000 7ff429200700 1 --2- 172.21.15.47:0/2804738595 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff424108310 0x7ff42410a700 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:11:31.353 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:30.564+0000 7ff429200700 1 -- 172.21.15.47:0/2804738595 >> 172.21.15.47:0/2804738595 conn(0x7ff424101de0 msgr2=0x7ff424104200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:11:31.353 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:30.564+0000 7ff429200700 1 -- 172.21.15.47:0/2804738595 shutdown_connections 2024-03-21T10:11:31.353 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:30.564+0000 7ff429200700 1 -- 172.21.15.47:0/2804738595 wait complete. 2024-03-21T10:11:31.353 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:30.565+0000 7ff429200700 1 Processor -- start 2024-03-21T10:11:31.353 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:30.565+0000 7ff429200700 1 -- start start 2024-03-21T10:11:31.353 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:30.566+0000 7ff429200700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff424108310 0x7ff42419d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:11:31.354 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:30.566+0000 7ff429200700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff42410bb90 con 0x7ff424108310 2024-03-21T10:11:31.354 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:30.566+0000 7ff421e00700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff424108310 0x7ff42419d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:11:31.354 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:30.566+0000 7ff421e00700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff424108310 0x7ff42419d990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:57540/0 (socket says 172.21.15.47:57540) 2024-03-21T10:11:31.354 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:30.566+0000 7ff421e00700 1 -- 172.21.15.47:0/1413333410 learned_addr learned my addr 172.21.15.47:0/1413333410 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:11:31.354 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:30.566+0000 7ff421e00700 1 -- 172.21.15.47:0/1413333410 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff42419ded0 con 0x7ff424108310 2024-03-21T10:11:31.354 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:30.567+0000 7ff421e00700 1 --2- 172.21.15.47:0/1413333410 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff424108310 0x7ff42419d990 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7ff414011a50 tx=0x7ff414004290 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:11:31.354 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:30.567+0000 7ff41aa00700 1 -- 172.21.15.47:0/1413333410 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ff4140039f0 con 0x7ff424108310 2024-03-21T10:11:31.354 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:30.567+0000 7ff41aa00700 1 -- 172.21.15.47:0/1413333410 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) v1 ==== 1071+0+0 (secure 0 0 0) 0x7ff414003b90 con 0x7ff424108310 2024-03-21T10:11:31.354 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:30.567+0000 7ff41aa00700 1 -- 172.21.15.47:0/1413333410 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ff41401a5a0 con 0x7ff424108310 2024-03-21T10:11:31.354 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:30.567+0000 7ff429200700 1 -- 172.21.15.47:0/1413333410 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff42419e1c0 con 0x7ff424108310 2024-03-21T10:11:31.354 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:30.567+0000 7ff429200700 1 -- 172.21.15.47:0/1413333410 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff4241a0e50 con 0x7ff424108310 2024-03-21T10:11:31.354 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:30.567+0000 7ff41aa00700 1 -- 172.21.15.47:0/1413333410 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (secure 0 0 0) 0x7ff41401abe0 con 0x7ff424108310 2024-03-21T10:11:31.354 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:30.567+0000 7ff41aa00700 1 -- 172.21.15.47:0/1413333410 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7ff414019700 con 0x7ff424108310 2024-03-21T10:11:31.355 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:30.568+0000 7ff429200700 1 -- 172.21.15.47:0/1413333410 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff4240637c0 con 0x7ff424108310 2024-03-21T10:11:31.355 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:30.571+0000 7ff41aa00700 1 -- 172.21.15.47:0/1413333410 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73141 (secure 0 0 0) 0x7ff414024a40 con 0x7ff424108310 2024-03-21T10:11:31.355 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:30.650+0000 7ff429200700 1 -- 172.21.15.47:0/1413333410 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) v1 -- 0x7ff4241a1080 con 0x7ff424108310 2024-03-21T10:11:31.355 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:30.651+0000 7ff41aa00700 1 -- 172.21.15.47:0/1413333410 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) v1 ==== 79+0+1239 (secure 0 0 0) 0x7ff414024440 con 0x7ff424108310 2024-03-21T10:11:31.355 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:30.653+0000 7ff429200700 1 -- 172.21.15.47:0/1413333410 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff424108310 msgr2=0x7ff42419d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:11:31.355 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:30.653+0000 7ff429200700 1 --2- 172.21.15.47:0/1413333410 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff424108310 0x7ff42419d990 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7ff414011a50 tx=0x7ff414004290 comp rx=0 tx=0).stop 2024-03-21T10:11:31.355 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:30.653+0000 7ff429200700 1 -- 172.21.15.47:0/1413333410 shutdown_connections 2024-03-21T10:11:31.355 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:30.653+0000 7ff429200700 1 --2- 172.21.15.47:0/1413333410 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff424108310 0x7ff42419d990 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:11:31.355 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:30.653+0000 7ff429200700 1 -- 172.21.15.47:0/1413333410 >> 172.21.15.47:0/1413333410 conn(0x7ff424101de0 msgr2=0x7ff424102890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:11:31.355 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:30.653+0000 7ff429200700 1 -- 172.21.15.47:0/1413333410 shutdown_connections 2024-03-21T10:11:31.355 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:30.653+0000 7ff429200700 1 -- 172.21.15.47:0/1413333410 wait complete. 2024-03-21T10:11:31.355 INFO:teuthology.orchestra.run.smithi047.stdout:mgr not available, waiting (2/15)... 2024-03-21T10:11:31.713 INFO:journalctl@ceph.mgr.x.smithi047.stdout:Mar 21 10:11:31 smithi047 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-mgr-x[31773]: 2024-03-21T10:11:31.426+0000 7fefd0081200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-03-21T10:11:31.714 INFO:journalctl@ceph.mgr.x.smithi047.stdout:Mar 21 10:11:31 smithi047 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-mgr-x[31773]: 2024-03-21T10:11:31.547+0000 7fefd0081200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-03-21T10:11:32.065 INFO:journalctl@ceph.mgr.x.smithi047.stdout:Mar 21 10:11:31 smithi047 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-mgr-x[31773]: 2024-03-21T10:11:31.811+0000 7fefd0081200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-03-21T10:11:32.451 INFO:journalctl@ceph.mgr.x.smithi047.stdout:Mar 21 10:11:32 smithi047 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-mgr-x[31773]: 2024-03-21T10:11:32.064+0000 7fefd0081200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-03-21T10:11:32.713 INFO:journalctl@ceph.mgr.x.smithi047.stdout:Mar 21 10:11:32 smithi047 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-mgr-x[31773]: 2024-03-21T10:11:32.449+0000 7fefd0081200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-03-21T10:11:33.088 INFO:journalctl@ceph.mgr.x.smithi047.stdout:Mar 21 10:11:32 smithi047 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-mgr-x[31773]: 2024-03-21T10:11:32.770+0000 7fefd0081200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-03-21T10:11:33.463 INFO:journalctl@ceph.mgr.x.smithi047.stdout:Mar 21 10:11:33 smithi047 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-mgr-x[31773]: 2024-03-21T10:11:33.086+0000 7fefd0081200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-03-21T10:11:34.400 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:11:34 smithi047 ceph-mon[31547]: from='client.? 172.21.15.47:0/4272450069' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-03-21T10:11:34.401 INFO:journalctl@ceph.mgr.x.smithi047.stdout:Mar 21 10:11:34 smithi047 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-mgr-x[31773]: 2024-03-21T10:11:34.173+0000 7fefd0081200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-03-21T10:11:34.757 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout 2024-03-21T10:11:34.758 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout { 2024-03-21T10:11:34.758 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "fsid": "3056d51a-e76b-11ee-95cd-87774f69a715", 2024-03-21T10:11:34.758 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "health": { 2024-03-21T10:11:34.758 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-03-21T10:11:34.759 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-03-21T10:11:34.759 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-03-21T10:11:34.759 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout }, 2024-03-21T10:11:34.759 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-03-21T10:11:34.759 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-03-21T10:11:34.760 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout 0 2024-03-21T10:11:34.760 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout ], 2024-03-21T10:11:34.760 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-03-21T10:11:34.760 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "a" 2024-03-21T10:11:34.760 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout ], 2024-03-21T10:11:34.761 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "quorum_age": 10, 2024-03-21T10:11:34.761 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "monmap": { 2024-03-21T10:11:34.761 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-21T10:11:34.761 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-03-21T10:11:34.761 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-03-21T10:11:34.762 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout }, 2024-03-21T10:11:34.762 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-03-21T10:11:34.762 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-21T10:11:34.762 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-03-21T10:11:34.762 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-03-21T10:11:34.762 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-03-21T10:11:34.763 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-03-21T10:11:34.763 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-03-21T10:11:34.763 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-03-21T10:11:34.763 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout }, 2024-03-21T10:11:34.763 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-03-21T10:11:34.764 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-03-21T10:11:34.764 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-03-21T10:11:34.764 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-03-21T10:11:34.765 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-03-21T10:11:34.765 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-03-21T10:11:34.765 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-03-21T10:11:34.766 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-03-21T10:11:34.766 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-03-21T10:11:34.766 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout }, 2024-03-21T10:11:34.766 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-03-21T10:11:34.766 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-21T10:11:34.766 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-03-21T10:11:34.766 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-03-21T10:11:34.766 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout }, 2024-03-21T10:11:34.766 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-03-21T10:11:34.766 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "available": false, 2024-03-21T10:11:34.768 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-03-21T10:11:34.768 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "modules": [ 2024-03-21T10:11:34.768 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "iostat", 2024-03-21T10:11:34.768 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "nfs", 2024-03-21T10:11:34.768 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "restful" 2024-03-21T10:11:34.768 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout ], 2024-03-21T10:11:34.768 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "services": {} 2024-03-21T10:11:34.768 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout }, 2024-03-21T10:11:34.769 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-03-21T10:11:34.769 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-21T10:11:34.769 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "modified": "2024-03-21T10:11:17.020553+0000", 2024-03-21T10:11:34.769 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "services": {} 2024-03-21T10:11:34.769 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout }, 2024-03-21T10:11:34.769 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-03-21T10:11:34.772 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout } 2024-03-21T10:11:34.772 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:33.924+0000 7f0756e00700 1 Processor -- start 2024-03-21T10:11:34.772 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:33.924+0000 7f0756e00700 1 -- start start 2024-03-21T10:11:34.773 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:33.925+0000 7f0756e00700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f075007ce10 0x7f075007b2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:11:34.773 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:33.925+0000 7f0756e00700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f075007b800 con 0x7f075007ce10 2024-03-21T10:11:34.773 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:33.925+0000 7f074fe00700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f075007ce10 0x7f075007b2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:11:34.773 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:33.925+0000 7f074fe00700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f075007ce10 0x7f075007b2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:47426/0 (socket says 172.21.15.47:47426) 2024-03-21T10:11:34.773 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:33.925+0000 7f074fe00700 1 -- 172.21.15.47:0/801287582 learned_addr learned my addr 172.21.15.47:0/801287582 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:11:34.773 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:33.926+0000 7f074fe00700 1 -- 172.21.15.47:0/801287582 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f075007b980 con 0x7f075007ce10 2024-03-21T10:11:34.773 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:33.926+0000 7f074fe00700 1 --2- 172.21.15.47:0/801287582 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f075007ce10 0x7f075007b2c0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f07400098a0 tx=0x7f0740009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=76c60c352d80aa0b server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:11:34.773 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:33.926+0000 7f074ea00700 1 -- 172.21.15.47:0/801287582 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f0740004160 con 0x7f075007ce10 2024-03-21T10:11:34.773 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:33.927+0000 7f074ea00700 1 -- 172.21.15.47:0/801287582 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) v1 ==== 1071+0+0 (secure 0 0 0) 0x7f0740004300 con 0x7f075007ce10 2024-03-21T10:11:34.773 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:33.927+0000 7f074ea00700 1 -- 172.21.15.47:0/801287582 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f0740011a80 con 0x7f075007ce10 2024-03-21T10:11:34.773 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:33.927+0000 7f0756e00700 1 -- 172.21.15.47:0/801287582 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f075007ce10 msgr2=0x7f075007b2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:11:34.773 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:33.927+0000 7f0756e00700 1 --2- 172.21.15.47:0/801287582 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f075007ce10 0x7f075007b2c0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f07400098a0 tx=0x7f0740009bb0 comp rx=0 tx=0).stop 2024-03-21T10:11:34.773 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:33.927+0000 7f0756e00700 1 -- 172.21.15.47:0/801287582 shutdown_connections 2024-03-21T10:11:34.773 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:33.927+0000 7f0756e00700 1 --2- 172.21.15.47:0/801287582 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f075007ce10 0x7f075007b2c0 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:11:34.774 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:33.927+0000 7f0756e00700 1 -- 172.21.15.47:0/801287582 >> 172.21.15.47:0/801287582 conn(0x7f0750102f50 msgr2=0x7f0750105370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:11:34.774 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:33.927+0000 7f0756e00700 1 -- 172.21.15.47:0/801287582 shutdown_connections 2024-03-21T10:11:34.774 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:33.928+0000 7f0756e00700 1 -- 172.21.15.47:0/801287582 wait complete. 2024-03-21T10:11:34.774 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:33.928+0000 7f0756e00700 1 Processor -- start 2024-03-21T10:11:34.774 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:33.929+0000 7f0756e00700 1 -- start start 2024-03-21T10:11:34.774 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:33.929+0000 7f0756e00700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f075007ce10 0x7f075019da00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:11:34.774 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:33.929+0000 7f0756e00700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f075010de20 con 0x7f075007ce10 2024-03-21T10:11:34.774 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:33.929+0000 7f074fe00700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f075007ce10 0x7f075019da00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:11:34.774 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:33.929+0000 7f074fe00700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f075007ce10 0x7f075019da00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:47432/0 (socket says 172.21.15.47:47432) 2024-03-21T10:11:34.774 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:33.929+0000 7f074fe00700 1 -- 172.21.15.47:0/4272450069 learned_addr learned my addr 172.21.15.47:0/4272450069 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:11:34.774 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:33.930+0000 7f074fe00700 1 -- 172.21.15.47:0/4272450069 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f075019df40 con 0x7f075007ce10 2024-03-21T10:11:34.774 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:33.930+0000 7f074fe00700 1 --2- 172.21.15.47:0/4272450069 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f075007ce10 0x7f075019da00 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f0740011a50 tx=0x7f0740004290 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:11:34.775 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:33.930+0000 7f074cc00700 1 -- 172.21.15.47:0/4272450069 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f07400039f0 con 0x7f075007ce10 2024-03-21T10:11:34.775 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:33.930+0000 7f074cc00700 1 -- 172.21.15.47:0/4272450069 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) v1 ==== 1071+0+0 (secure 0 0 0) 0x7f0740003b90 con 0x7f075007ce10 2024-03-21T10:11:34.775 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:33.930+0000 7f074cc00700 1 -- 172.21.15.47:0/4272450069 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f074001a600 con 0x7f075007ce10 2024-03-21T10:11:34.775 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:33.930+0000 7f0756e00700 1 -- 172.21.15.47:0/4272450069 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f075019e1d0 con 0x7f075007ce10 2024-03-21T10:11:34.776 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:33.930+0000 7f0756e00700 1 -- 172.21.15.47:0/4272450069 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f07501a0ec0 con 0x7f075007ce10 2024-03-21T10:11:34.776 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:33.931+0000 7f074cc00700 1 -- 172.21.15.47:0/4272450069 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (secure 0 0 0) 0x7f074001ab30 con 0x7f075007ce10 2024-03-21T10:11:34.777 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:33.931+0000 7f074cc00700 1 -- 172.21.15.47:0/4272450069 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f0740019700 con 0x7f075007ce10 2024-03-21T10:11:34.777 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:33.931+0000 7f0756e00700 1 -- 172.21.15.47:0/4272450069 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f07500637c0 con 0x7f075007ce10 2024-03-21T10:11:34.777 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:33.934+0000 7f074cc00700 1 -- 172.21.15.47:0/4272450069 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73141 (secure 0 0 0) 0x7f0740024a40 con 0x7f075007ce10 2024-03-21T10:11:34.777 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:34.014+0000 7f0756e00700 1 -- 172.21.15.47:0/4272450069 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) v1 -- 0x7f07501a1480 con 0x7f075007ce10 2024-03-21T10:11:34.777 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:34.015+0000 7f074cc00700 1 -- 172.21.15.47:0/4272450069 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) v1 ==== 79+0+1240 (secure 0 0 0) 0x7f0740032030 con 0x7f075007ce10 2024-03-21T10:11:34.777 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:34.016+0000 7f0756e00700 1 -- 172.21.15.47:0/4272450069 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f075007ce10 msgr2=0x7f075019da00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:11:34.777 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:34.016+0000 7f0756e00700 1 --2- 172.21.15.47:0/4272450069 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f075007ce10 0x7f075019da00 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f0740011a50 tx=0x7f0740004290 comp rx=0 tx=0).stop 2024-03-21T10:11:34.777 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:34.016+0000 7f0756e00700 1 -- 172.21.15.47:0/4272450069 shutdown_connections 2024-03-21T10:11:34.777 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:34.016+0000 7f0756e00700 1 --2- 172.21.15.47:0/4272450069 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f075007ce10 0x7f075019da00 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:11:34.777 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:34.016+0000 7f0756e00700 1 -- 172.21.15.47:0/4272450069 >> 172.21.15.47:0/4272450069 conn(0x7f0750102f50 msgr2=0x7f0750103ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:11:34.777 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:34.017+0000 7f0756e00700 1 -- 172.21.15.47:0/4272450069 shutdown_connections 2024-03-21T10:11:34.777 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:34.017+0000 7f0756e00700 1 -- 172.21.15.47:0/4272450069 wait complete. 2024-03-21T10:11:34.777 INFO:teuthology.orchestra.run.smithi047.stdout:mgr not available, waiting (3/15)... 2024-03-21T10:11:35.033 INFO:journalctl@ceph.mgr.x.smithi047.stdout:Mar 21 10:11:34 smithi047 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-mgr-x[31773]: 2024-03-21T10:11:34.899+0000 7fefd0081200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-03-21T10:11:35.033 INFO:journalctl@ceph.mgr.x.smithi047.stdout:Mar 21 10:11:35 smithi047 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-mgr-x[31773]: 2024-03-21T10:11:35.031+0000 7fefd0081200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-03-21T10:11:35.714 INFO:journalctl@ceph.mgr.x.smithi047.stdout:Mar 21 10:11:35 smithi047 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-mgr-x[31773]: 2024-03-21T10:11:35.323+0000 7fefd0081200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-03-21T10:11:35.714 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:11:35 smithi047 ceph-mon[31547]: Activating manager daemon x 2024-03-21T10:11:35.714 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:11:35 smithi047 ceph-mon[31547]: mgrmap e2: x(active, starting, since 0.0121631s) 2024-03-21T10:11:35.714 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:11:35 smithi047 ceph-mon[31547]: from='mgr.14100 172.21.15.47:0/1370599564' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-03-21T10:11:35.714 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:11:35 smithi047 ceph-mon[31547]: from='mgr.14100 172.21.15.47:0/1370599564' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-03-21T10:11:35.714 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:11:35 smithi047 ceph-mon[31547]: from='mgr.14100 172.21.15.47:0/1370599564' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-03-21T10:11:35.714 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:11:35 smithi047 ceph-mon[31547]: from='mgr.14100 172.21.15.47:0/1370599564' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-21T10:11:35.714 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:11:35 smithi047 ceph-mon[31547]: from='mgr.14100 172.21.15.47:0/1370599564' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-03-21T10:11:35.714 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:11:35 smithi047 ceph-mon[31547]: Manager daemon x is now available 2024-03-21T10:11:35.714 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:11:35 smithi047 ceph-mon[31547]: from='mgr.14100 172.21.15.47:0/1370599564' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-03-21T10:11:35.715 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:11:35 smithi047 ceph-mon[31547]: from='mgr.14100 172.21.15.47:0/1370599564' entity='mgr.x' 2024-03-21T10:11:35.715 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:11:35 smithi047 ceph-mon[31547]: from='mgr.14100 172.21.15.47:0/1370599564' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-03-21T10:11:35.715 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:11:35 smithi047 ceph-mon[31547]: from='mgr.14100 172.21.15.47:0/1370599564' entity='mgr.x' 2024-03-21T10:11:35.715 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:11:35 smithi047 ceph-mon[31547]: from='mgr.14100 172.21.15.47:0/1370599564' entity='mgr.x' 2024-03-21T10:11:37.464 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:11:37 smithi047 ceph-mon[31547]: mgrmap e3: x(active, since 1.02231s) 2024-03-21T10:11:38.447 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:11:38 smithi047 ceph-mon[31547]: mgrmap e4: x(active, since 2s) 2024-03-21T10:11:38.447 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:11:38 smithi047 ceph-mon[31547]: from='client.? 172.21.15.47:0/1672444375' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-03-21T10:11:38.466 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout 2024-03-21T10:11:38.466 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout { 2024-03-21T10:11:38.466 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "fsid": "3056d51a-e76b-11ee-95cd-87774f69a715", 2024-03-21T10:11:38.466 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "health": { 2024-03-21T10:11:38.468 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-03-21T10:11:38.468 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-03-21T10:11:38.468 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-03-21T10:11:38.468 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout }, 2024-03-21T10:11:38.468 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-03-21T10:11:38.468 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-03-21T10:11:38.468 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout 0 2024-03-21T10:11:38.468 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout ], 2024-03-21T10:11:38.468 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-03-21T10:11:38.469 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "a" 2024-03-21T10:11:38.469 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout ], 2024-03-21T10:11:38.469 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "quorum_age": 14, 2024-03-21T10:11:38.469 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "monmap": { 2024-03-21T10:11:38.470 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-21T10:11:38.470 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-03-21T10:11:38.470 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-03-21T10:11:38.470 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout }, 2024-03-21T10:11:38.470 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-03-21T10:11:38.471 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-21T10:11:38.471 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-03-21T10:11:38.471 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-03-21T10:11:38.471 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-03-21T10:11:38.471 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-03-21T10:11:38.473 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-03-21T10:11:38.473 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-03-21T10:11:38.473 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout }, 2024-03-21T10:11:38.473 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-03-21T10:11:38.473 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-03-21T10:11:38.473 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-03-21T10:11:38.473 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-03-21T10:11:38.473 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-03-21T10:11:38.473 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-03-21T10:11:38.473 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-03-21T10:11:38.473 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-03-21T10:11:38.474 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-03-21T10:11:38.474 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout }, 2024-03-21T10:11:38.474 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-03-21T10:11:38.474 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-21T10:11:38.474 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-03-21T10:11:38.474 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-03-21T10:11:38.474 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout }, 2024-03-21T10:11:38.478 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-03-21T10:11:38.478 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "available": true, 2024-03-21T10:11:38.478 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-03-21T10:11:38.478 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "modules": [ 2024-03-21T10:11:38.479 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "iostat", 2024-03-21T10:11:38.479 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "nfs", 2024-03-21T10:11:38.479 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "restful" 2024-03-21T10:11:38.479 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout ], 2024-03-21T10:11:38.479 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "services": {} 2024-03-21T10:11:38.479 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout }, 2024-03-21T10:11:38.479 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-03-21T10:11:38.479 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-21T10:11:38.479 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "modified": "2024-03-21T10:11:17.020553+0000", 2024-03-21T10:11:38.479 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "services": {} 2024-03-21T10:11:38.479 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout }, 2024-03-21T10:11:38.479 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-03-21T10:11:38.480 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout } 2024-03-21T10:11:38.480 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:37.349+0000 7f5463200700 1 Processor -- start 2024-03-21T10:11:38.480 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:37.349+0000 7f5463200700 1 -- start start 2024-03-21T10:11:38.480 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:37.349+0000 7f5463200700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5464107c10 0x7f5464108030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:11:38.480 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:37.349+0000 7f5463200700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5464108600 con 0x7f5464107c10 2024-03-21T10:11:38.480 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:37.350+0000 7f5461e00700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5464107c10 0x7f5464108030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:11:38.480 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:37.350+0000 7f5461e00700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5464107c10 0x7f5464108030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:47504/0 (socket says 172.21.15.47:47504) 2024-03-21T10:11:38.480 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:37.350+0000 7f5461e00700 1 -- 172.21.15.47:0/808967201 learned_addr learned my addr 172.21.15.47:0/808967201 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:11:38.480 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:37.350+0000 7f5461e00700 1 -- 172.21.15.47:0/808967201 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5464106870 con 0x7f5464107c10 2024-03-21T10:11:38.480 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:37.351+0000 7f5461e00700 1 --2- 172.21.15.47:0/808967201 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5464107c10 0x7f5464108030 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f5454009e80 tx=0x7f545400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d964b8d561b61db2 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:11:38.480 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:37.351+0000 7f5461400700 1 -- 172.21.15.47:0/808967201 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f5454017070 con 0x7f5464107c10 2024-03-21T10:11:38.480 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:37.351+0000 7f5461400700 1 -- 172.21.15.47:0/808967201 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) v1 ==== 1071+0+0 (secure 0 0 0) 0x7f545400b8c0 con 0x7f5464107c10 2024-03-21T10:11:38.480 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:37.351+0000 7f5461400700 1 -- 172.21.15.47:0/808967201 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f5454004520 con 0x7f5464107c10 2024-03-21T10:11:38.481 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:37.352+0000 7f5463200700 1 -- 172.21.15.47:0/808967201 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5464107c10 msgr2=0x7f5464108030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:11:38.481 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:37.352+0000 7f5463200700 1 --2- 172.21.15.47:0/808967201 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5464107c10 0x7f5464108030 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f5454009e80 tx=0x7f545400b330 comp rx=0 tx=0).stop 2024-03-21T10:11:38.481 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:37.352+0000 7f5463200700 1 -- 172.21.15.47:0/808967201 shutdown_connections 2024-03-21T10:11:38.481 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:37.352+0000 7f5463200700 1 --2- 172.21.15.47:0/808967201 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5464107c10 0x7f5464108030 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:11:38.481 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:37.352+0000 7f5463200700 1 -- 172.21.15.47:0/808967201 >> 172.21.15.47:0/808967201 conn(0x7f54641013c0 msgr2=0x7f54641037e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:11:38.481 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:37.352+0000 7f5463200700 1 -- 172.21.15.47:0/808967201 shutdown_connections 2024-03-21T10:11:38.481 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:37.352+0000 7f5463200700 1 -- 172.21.15.47:0/808967201 wait complete. 2024-03-21T10:11:38.481 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:37.353+0000 7f5463200700 1 Processor -- start 2024-03-21T10:11:38.481 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:37.353+0000 7f5463200700 1 -- start start 2024-03-21T10:11:38.481 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:37.353+0000 7f5463200700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5464107c10 0x7f546419d810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:11:38.481 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:37.353+0000 7f5463200700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f546410ac30 con 0x7f5464107c10 2024-03-21T10:11:38.485 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:37.354+0000 7f5461e00700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5464107c10 0x7f546419d810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:11:38.485 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:37.354+0000 7f5461e00700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5464107c10 0x7f546419d810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:47520/0 (socket says 172.21.15.47:47520) 2024-03-21T10:11:38.485 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:37.354+0000 7f5461e00700 1 -- 172.21.15.47:0/1672444375 learned_addr learned my addr 172.21.15.47:0/1672444375 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:11:38.485 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:37.354+0000 7f5461e00700 1 -- 172.21.15.47:0/1672444375 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f546419dd50 con 0x7f5464107c10 2024-03-21T10:11:38.485 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:37.354+0000 7f5461e00700 1 --2- 172.21.15.47:0/1672444375 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5464107c10 0x7f546419d810 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f545400b7b0 tx=0x7f5454004670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:11:38.485 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:37.354+0000 7f545b400700 1 -- 172.21.15.47:0/1672444375 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f5454020020 con 0x7f5464107c10 2024-03-21T10:11:38.485 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:37.355+0000 7f545b400700 1 -- 172.21.15.47:0/1672444375 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) v1 ==== 1071+0+0 (secure 0 0 0) 0x7f545400bcb0 con 0x7f5464107c10 2024-03-21T10:11:38.485 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:37.355+0000 7f545b400700 1 -- 172.21.15.47:0/1672444375 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f5454017040 con 0x7f5464107c10 2024-03-21T10:11:38.485 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:37.355+0000 7f5463200700 1 -- 172.21.15.47:0/1672444375 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f546419dfe0 con 0x7f5464107c10 2024-03-21T10:11:38.485 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:37.355+0000 7f5463200700 1 -- 172.21.15.47:0/1672444375 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f54641a0c80 con 0x7f5464107c10 2024-03-21T10:11:38.485 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:37.356+0000 7f5463200700 1 -- 172.21.15.47:0/1672444375 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f54640637c0 con 0x7f5464107c10 2024-03-21T10:11:38.486 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:37.356+0000 7f545b400700 1 -- 172.21.15.47:0/1672444375 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 4) v1 ==== 46283+0+0 (secure 0 0 0) 0x7f5454023020 con 0x7f5464107c10 2024-03-21T10:11:38.486 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:37.356+0000 7f545b400700 1 --2- 172.21.15.47:0/1672444375 >> [v2:172.21.15.47:6800/3942658432,v1:172.21.15.47:6801/3942658432] conn(0x7f5448039240 0x7f544803b700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:11:38.486 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:37.357+0000 7f545b400700 1 -- 172.21.15.47:0/1672444375 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f545400f050 con 0x7f5464107c10 2024-03-21T10:11:38.486 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:37.357+0000 7f5459400700 1 --2- 172.21.15.47:0/1672444375 >> [v2:172.21.15.47:6800/3942658432,v1:172.21.15.47:6801/3942658432] conn(0x7f5448039240 0x7f544803b700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:11:38.486 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:37.357+0000 7f5459400700 1 --2- 172.21.15.47:0/1672444375 >> [v2:172.21.15.47:6800/3942658432,v1:172.21.15.47:6801/3942658432] conn(0x7f5448039240 0x7f544803b700 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f544c006fd0 tx=0x7f544c006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:11:38.486 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:37.363+0000 7f545b400700 1 -- 172.21.15.47:0/1672444375 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f545402a020 con 0x7f5464107c10 2024-03-21T10:11:38.486 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:37.677+0000 7f5463200700 1 -- 172.21.15.47:0/1672444375 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) v1 -- 0x7f546402dc00 con 0x7f5464107c10 2024-03-21T10:11:38.486 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:37.678+0000 7f545b400700 1 -- 172.21.15.47:0/1672444375 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) v1 ==== 79+0+1239 (secure 0 0 0) 0x7f545402a200 con 0x7f5464107c10 2024-03-21T10:11:38.486 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:37.681+0000 7f5463200700 1 -- 172.21.15.47:0/1672444375 >> [v2:172.21.15.47:6800/3942658432,v1:172.21.15.47:6801/3942658432] conn(0x7f5448039240 msgr2=0x7f544803b700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:11:38.487 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:37.681+0000 7f5463200700 1 --2- 172.21.15.47:0/1672444375 >> [v2:172.21.15.47:6800/3942658432,v1:172.21.15.47:6801/3942658432] conn(0x7f5448039240 0x7f544803b700 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f544c006fd0 tx=0x7f544c006e40 comp rx=0 tx=0).stop 2024-03-21T10:11:38.487 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:37.681+0000 7f5463200700 1 -- 172.21.15.47:0/1672444375 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5464107c10 msgr2=0x7f546419d810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:11:38.487 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:37.681+0000 7f5463200700 1 --2- 172.21.15.47:0/1672444375 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5464107c10 0x7f546419d810 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f545400b7b0 tx=0x7f5454004670 comp rx=0 tx=0).stop 2024-03-21T10:11:38.487 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:37.681+0000 7f5463200700 1 -- 172.21.15.47:0/1672444375 shutdown_connections 2024-03-21T10:11:38.487 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:37.681+0000 7f5463200700 1 --2- 172.21.15.47:0/1672444375 >> [v2:172.21.15.47:6800/3942658432,v1:172.21.15.47:6801/3942658432] conn(0x7f5448039240 0x7f544803b700 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:11:38.487 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:37.681+0000 7f5463200700 1 --2- 172.21.15.47:0/1672444375 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5464107c10 0x7f546419d810 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:11:38.487 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:37.681+0000 7f5463200700 1 -- 172.21.15.47:0/1672444375 >> 172.21.15.47:0/1672444375 conn(0x7f54641013c0 msgr2=0x7f5464102100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:11:38.487 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:37.681+0000 7f5463200700 1 -- 172.21.15.47:0/1672444375 shutdown_connections 2024-03-21T10:11:38.487 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:37.681+0000 7f5463200700 1 -- 172.21.15.47:0/1672444375 wait complete. 2024-03-21T10:11:38.487 INFO:teuthology.orchestra.run.smithi047.stdout:mgr is available 2024-03-21T10:11:39.870 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:11:39 smithi047 ceph-mon[31547]: from='client.? 172.21.15.47:0/232268730' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-03-21T10:11:40.171 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout 2024-03-21T10:11:40.171 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout [global] 2024-03-21T10:11:40.172 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout fsid = 3056d51a-e76b-11ee-95cd-87774f69a715 2024-03-21T10:11:40.172 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-03-21T10:11:40.172 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-03-21T10:11:40.172 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-03-21T10:11:40.172 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-03-21T10:11:40.172 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout 2024-03-21T10:11:40.172 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout [mgr] 2024-03-21T10:11:40.172 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-03-21T10:11:40.172 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout 2024-03-21T10:11:40.172 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout [osd] 2024-03-21T10:11:40.172 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-03-21T10:11:40.172 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-03-21T10:11:40.172 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-03-21T10:11:40.173 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-03-21T10:11:40.173 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-03-21T10:11:40.173 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:39.287+0000 7fd8f5400700 1 Processor -- start 2024-03-21T10:11:40.173 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:39.287+0000 7fd8f5400700 1 -- start start 2024-03-21T10:11:40.173 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:39.287+0000 7fd8f5400700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd8f0106780 0x7fd8f0106ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:11:40.173 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:39.287+0000 7fd8f5400700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8f0107170 con 0x7fd8f0106780 2024-03-21T10:11:40.173 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:39.288+0000 7fd8ee800700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd8f0106780 0x7fd8f0106ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:11:40.173 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:39.288+0000 7fd8ee800700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd8f0106780 0x7fd8f0106ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:47536/0 (socket says 172.21.15.47:47536) 2024-03-21T10:11:40.173 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:39.288+0000 7fd8ee800700 1 -- 172.21.15.47:0/4240055406 learned_addr learned my addr 172.21.15.47:0/4240055406 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:11:40.173 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:39.288+0000 7fd8ee800700 1 -- 172.21.15.47:0/4240055406 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8f0107980 con 0x7fd8f0106780 2024-03-21T10:11:40.173 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:39.289+0000 7fd8ee800700 1 --2- 172.21.15.47:0/4240055406 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd8f0106780 0x7fd8f0106ba0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fd8e0009e80 tx=0x7fd8e000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3a2239285124fe23 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:11:40.173 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:39.289+0000 7fd8ede00700 1 -- 172.21.15.47:0/4240055406 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd8e0017070 con 0x7fd8f0106780 2024-03-21T10:11:40.173 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:39.289+0000 7fd8ede00700 1 -- 172.21.15.47:0/4240055406 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) v1 ==== 1071+0+0 (secure 0 0 0) 0x7fd8e000b8c0 con 0x7fd8f0106780 2024-03-21T10:11:40.174 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:39.289+0000 7fd8ede00700 1 -- 172.21.15.47:0/4240055406 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd8e0004520 con 0x7fd8f0106780 2024-03-21T10:11:40.174 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:39.290+0000 7fd8f5400700 1 -- 172.21.15.47:0/4240055406 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd8f0106780 msgr2=0x7fd8f0106ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:11:40.174 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:39.290+0000 7fd8f5400700 1 --2- 172.21.15.47:0/4240055406 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd8f0106780 0x7fd8f0106ba0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fd8e0009e80 tx=0x7fd8e000b330 comp rx=0 tx=0).stop 2024-03-21T10:11:40.174 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:39.290+0000 7fd8f5400700 1 -- 172.21.15.47:0/4240055406 shutdown_connections 2024-03-21T10:11:40.174 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:39.290+0000 7fd8f5400700 1 --2- 172.21.15.47:0/4240055406 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd8f0106780 0x7fd8f0106ba0 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:11:40.174 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:39.290+0000 7fd8f5400700 1 -- 172.21.15.47:0/4240055406 >> 172.21.15.47:0/4240055406 conn(0x7fd8f0101de0 msgr2=0x7fd8f0104200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:11:40.174 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:39.290+0000 7fd8f5400700 1 -- 172.21.15.47:0/4240055406 shutdown_connections 2024-03-21T10:11:40.174 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:39.290+0000 7fd8f5400700 1 -- 172.21.15.47:0/4240055406 wait complete. 2024-03-21T10:11:40.174 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:39.291+0000 7fd8f5400700 1 Processor -- start 2024-03-21T10:11:40.174 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:39.291+0000 7fd8f5400700 1 -- start start 2024-03-21T10:11:40.174 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:39.292+0000 7fd8f5400700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd8f0106780 0x7fd8f019da10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:11:40.174 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:39.292+0000 7fd8f5400700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8f010ad30 con 0x7fd8f0106780 2024-03-21T10:11:40.174 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:39.292+0000 7fd8ee800700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd8f0106780 0x7fd8f019da10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:11:40.175 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:39.292+0000 7fd8ee800700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd8f0106780 0x7fd8f019da10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:47548/0 (socket says 172.21.15.47:47548) 2024-03-21T10:11:40.175 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:39.292+0000 7fd8ee800700 1 -- 172.21.15.47:0/232268730 learned_addr learned my addr 172.21.15.47:0/232268730 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:11:40.175 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:39.292+0000 7fd8ee800700 1 -- 172.21.15.47:0/232268730 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8f019df50 con 0x7fd8f0106780 2024-03-21T10:11:40.175 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:39.292+0000 7fd8ee800700 1 --2- 172.21.15.47:0/232268730 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd8f0106780 0x7fd8f019da10 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fd8e0000f80 tx=0x7fd8e0009640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:11:40.175 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:39.293+0000 7fd8e7e00700 1 -- 172.21.15.47:0/232268730 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd8e0020020 con 0x7fd8f0106780 2024-03-21T10:11:40.175 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:39.293+0000 7fd8e7e00700 1 -- 172.21.15.47:0/232268730 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) v1 ==== 1071+0+0 (secure 0 0 0) 0x7fd8e000bba0 con 0x7fd8f0106780 2024-03-21T10:11:40.175 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:39.293+0000 7fd8e7e00700 1 -- 172.21.15.47:0/232268730 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd8e0017040 con 0x7fd8f0106780 2024-03-21T10:11:40.175 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:39.293+0000 7fd8f5400700 1 -- 172.21.15.47:0/232268730 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd8f019e1e0 con 0x7fd8f0106780 2024-03-21T10:11:40.175 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:39.293+0000 7fd8f5400700 1 -- 172.21.15.47:0/232268730 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd8f01a0ed0 con 0x7fd8f0106780 2024-03-21T10:11:40.175 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:39.294+0000 7fd8e7e00700 1 -- 172.21.15.47:0/232268730 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 4) v1 ==== 46283+0+0 (secure 0 0 0) 0x7fd8e0023020 con 0x7fd8f0106780 2024-03-21T10:11:40.175 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:39.294+0000 7fd8f5400700 1 -- 172.21.15.47:0/232268730 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd8f00637c0 con 0x7fd8f0106780 2024-03-21T10:11:40.175 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:39.294+0000 7fd8e7e00700 1 --2- 172.21.15.47:0/232268730 >> [v2:172.21.15.47:6800/3942658432,v1:172.21.15.47:6801/3942658432] conn(0x7fd8d4039180 0x7fd8d403b640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:11:40.175 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:39.294+0000 7fd8e7e00700 1 -- 172.21.15.47:0/232268730 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7fd8e000f050 con 0x7fd8f0106780 2024-03-21T10:11:40.175 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:39.295+0000 7fd8e5e00700 1 --2- 172.21.15.47:0/232268730 >> [v2:172.21.15.47:6800/3942658432,v1:172.21.15.47:6801/3942658432] conn(0x7fd8d4039180 0x7fd8d403b640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:11:40.176 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:39.295+0000 7fd8e5e00700 1 --2- 172.21.15.47:0/232268730 >> [v2:172.21.15.47:6800/3942658432,v1:172.21.15.47:6801/3942658432] conn(0x7fd8d4039180 0x7fd8d403b640 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fd8d8006fd0 tx=0x7fd8d8006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:11:40.176 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:39.300+0000 7fd8e7e00700 1 -- 172.21.15.47:0/232268730 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fd8e002a020 con 0x7fd8f0106780 2024-03-21T10:11:40.176 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:39.512+0000 7fd8f5400700 1 -- 172.21.15.47:0/232268730 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) v1 -- 0x7fd8f002dd70 con 0x7fd8f0106780 2024-03-21T10:11:40.176 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:39.514+0000 7fd8e7e00700 1 -- 172.21.15.47:0/232268730 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v3) v1 ==== 70+0+373 (secure 0 0 0) 0x7fd8e002c300 con 0x7fd8f0106780 2024-03-21T10:11:40.176 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:39.517+0000 7fd8f5400700 1 -- 172.21.15.47:0/232268730 >> [v2:172.21.15.47:6800/3942658432,v1:172.21.15.47:6801/3942658432] conn(0x7fd8d4039180 msgr2=0x7fd8d403b640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:11:40.176 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:39.517+0000 7fd8f5400700 1 --2- 172.21.15.47:0/232268730 >> [v2:172.21.15.47:6800/3942658432,v1:172.21.15.47:6801/3942658432] conn(0x7fd8d4039180 0x7fd8d403b640 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fd8d8006fd0 tx=0x7fd8d8006e40 comp rx=0 tx=0).stop 2024-03-21T10:11:40.176 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:39.517+0000 7fd8f5400700 1 -- 172.21.15.47:0/232268730 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd8f0106780 msgr2=0x7fd8f019da10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:11:40.176 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:39.517+0000 7fd8f5400700 1 --2- 172.21.15.47:0/232268730 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd8f0106780 0x7fd8f019da10 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fd8e0000f80 tx=0x7fd8e0009640 comp rx=0 tx=0).stop 2024-03-21T10:11:40.176 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:39.517+0000 7fd8f5400700 1 -- 172.21.15.47:0/232268730 shutdown_connections 2024-03-21T10:11:40.176 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:39.517+0000 7fd8f5400700 1 --2- 172.21.15.47:0/232268730 >> [v2:172.21.15.47:6800/3942658432,v1:172.21.15.47:6801/3942658432] conn(0x7fd8d4039180 0x7fd8d403b640 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:11:40.176 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:39.517+0000 7fd8f5400700 1 --2- 172.21.15.47:0/232268730 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd8f0106780 0x7fd8f019da10 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:11:40.176 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:39.517+0000 7fd8f5400700 1 -- 172.21.15.47:0/232268730 >> 172.21.15.47:0/232268730 conn(0x7fd8f0101de0 msgr2=0x7fd8f0102c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:11:40.176 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:39.518+0000 7fd8f5400700 1 -- 172.21.15.47:0/232268730 shutdown_connections 2024-03-21T10:11:40.176 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:39.518+0000 7fd8f5400700 1 -- 172.21.15.47:0/232268730 wait complete. 2024-03-21T10:11:40.177 INFO:teuthology.orchestra.run.smithi047.stdout:Enabling cephadm module... 2024-03-21T10:11:41.463 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:11:41 smithi047 ceph-mon[31547]: from='client.? 172.21.15.47:0/1573881771' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "cephadm"}]: dispatch 2024-03-21T10:11:41.464 INFO:journalctl@ceph.mgr.x.smithi047.stdout:Mar 21 10:11:41 smithi047 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-mgr-x[31773]: ignoring --setuser ceph since I am not root 2024-03-21T10:11:41.464 INFO:journalctl@ceph.mgr.x.smithi047.stdout:Mar 21 10:11:41 smithi047 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-mgr-x[31773]: ignoring --setgroup ceph since I am not root 2024-03-21T10:11:41.865 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:40.849+0000 7f0575e00700 1 Processor -- start 2024-03-21T10:11:41.865 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:40.850+0000 7f0575e00700 1 -- start start 2024-03-21T10:11:41.865 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:40.850+0000 7f0575e00700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f057007cdf0 0x7f057007b2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:11:41.865 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:40.850+0000 7f0575e00700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f057007b7e0 con 0x7f057007cdf0 2024-03-21T10:11:41.865 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:40.850+0000 7f056ea00700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f057007cdf0 0x7f057007b2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:11:41.865 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:40.850+0000 7f056ea00700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f057007cdf0 0x7f057007b2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:47550/0 (socket says 172.21.15.47:47550) 2024-03-21T10:11:41.865 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:40.850+0000 7f056ea00700 1 -- 172.21.15.47:0/3722986506 learned_addr learned my addr 172.21.15.47:0/3722986506 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:11:41.865 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:40.851+0000 7f056ea00700 1 -- 172.21.15.47:0/3722986506 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f057007b960 con 0x7f057007cdf0 2024-03-21T10:11:41.865 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:40.851+0000 7f056ea00700 1 --2- 172.21.15.47:0/3722986506 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f057007cdf0 0x7f057007b2a0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f05600098a0 tx=0x7f0560009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5caba6aa1fe8d271 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:11:41.866 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:40.852+0000 7f056e000700 1 -- 172.21.15.47:0/3722986506 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f0560004160 con 0x7f057007cdf0 2024-03-21T10:11:41.866 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:40.852+0000 7f056e000700 1 -- 172.21.15.47:0/3722986506 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) v1 ==== 1071+0+0 (secure 0 0 0) 0x7f0560004300 con 0x7f057007cdf0 2024-03-21T10:11:41.866 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:40.852+0000 7f056e000700 1 -- 172.21.15.47:0/3722986506 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f0560011a80 con 0x7f057007cdf0 2024-03-21T10:11:41.866 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:40.852+0000 7f0575e00700 1 -- 172.21.15.47:0/3722986506 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f057007cdf0 msgr2=0x7f057007b2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:11:41.866 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:40.852+0000 7f0575e00700 1 --2- 172.21.15.47:0/3722986506 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f057007cdf0 0x7f057007b2a0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f05600098a0 tx=0x7f0560009bb0 comp rx=0 tx=0).stop 2024-03-21T10:11:41.866 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:40.853+0000 7f0575e00700 1 -- 172.21.15.47:0/3722986506 shutdown_connections 2024-03-21T10:11:41.866 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:40.853+0000 7f0575e00700 1 --2- 172.21.15.47:0/3722986506 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f057007cdf0 0x7f057007b2a0 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:11:41.866 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:40.853+0000 7f0575e00700 1 -- 172.21.15.47:0/3722986506 >> 172.21.15.47:0/3722986506 conn(0x7f0570102f50 msgr2=0x7f0570105370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:11:41.866 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:40.853+0000 7f0575e00700 1 -- 172.21.15.47:0/3722986506 shutdown_connections 2024-03-21T10:11:41.866 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:40.853+0000 7f0575e00700 1 -- 172.21.15.47:0/3722986506 wait complete. 2024-03-21T10:11:41.866 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:40.854+0000 7f0575e00700 1 Processor -- start 2024-03-21T10:11:41.866 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:40.854+0000 7f0575e00700 1 -- start start 2024-03-21T10:11:41.866 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:40.854+0000 7f0575e00700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f057007cdf0 0x7f057019d9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:11:41.866 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:40.854+0000 7f0575e00700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f057010de20 con 0x7f057007cdf0 2024-03-21T10:11:41.867 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:40.855+0000 7f056ea00700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f057007cdf0 0x7f057019d9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:11:41.867 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:40.855+0000 7f056ea00700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f057007cdf0 0x7f057019d9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:47552/0 (socket says 172.21.15.47:47552) 2024-03-21T10:11:41.867 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:40.855+0000 7f056ea00700 1 -- 172.21.15.47:0/1573881771 learned_addr learned my addr 172.21.15.47:0/1573881771 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:11:41.867 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:40.855+0000 7f056ea00700 1 -- 172.21.15.47:0/1573881771 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f057019dee0 con 0x7f057007cdf0 2024-03-21T10:11:41.867 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:40.855+0000 7f056ea00700 1 --2- 172.21.15.47:0/1573881771 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f057007cdf0 0x7f057019d9a0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f0560011a50 tx=0x7f0560011b30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:11:41.867 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:40.856+0000 7f0574a00700 1 -- 172.21.15.47:0/1573881771 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f0560003a80 con 0x7f057007cdf0 2024-03-21T10:11:41.867 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:40.856+0000 7f0574a00700 1 -- 172.21.15.47:0/1573881771 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) v1 ==== 1071+0+0 (secure 0 0 0) 0x7f0560003c20 con 0x7f057007cdf0 2024-03-21T10:11:41.867 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:40.856+0000 7f0574a00700 1 -- 172.21.15.47:0/1573881771 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f056001a650 con 0x7f057007cdf0 2024-03-21T10:11:41.867 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:40.856+0000 7f0575e00700 1 -- 172.21.15.47:0/1573881771 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f057019e170 con 0x7f057007cdf0 2024-03-21T10:11:41.867 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:40.856+0000 7f0575e00700 1 -- 172.21.15.47:0/1573881771 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f05701a0e60 con 0x7f057007cdf0 2024-03-21T10:11:41.867 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:40.857+0000 7f0575e00700 1 -- 172.21.15.47:0/1573881771 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f05700637c0 con 0x7f057007cdf0 2024-03-21T10:11:41.867 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:40.857+0000 7f0574a00700 1 -- 172.21.15.47:0/1573881771 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 4) v1 ==== 46283+0+0 (secure 0 0 0) 0x7f0560024ad0 con 0x7f057007cdf0 2024-03-21T10:11:41.867 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:40.858+0000 7f0574a00700 1 --2- 172.21.15.47:0/1573881771 >> [v2:172.21.15.47:6800/3942658432,v1:172.21.15.47:6801/3942658432] conn(0x7f0554039180 0x7f055403b640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:11:41.867 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:40.858+0000 7f0574a00700 1 -- 172.21.15.47:0/1573881771 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f056004d5a0 con 0x7f057007cdf0 2024-03-21T10:11:41.868 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:40.858+0000 7f0566000700 1 --2- 172.21.15.47:0/1573881771 >> [v2:172.21.15.47:6800/3942658432,v1:172.21.15.47:6801/3942658432] conn(0x7f0554039180 0x7f055403b640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:11:41.868 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:40.859+0000 7f0566000700 1 --2- 172.21.15.47:0/1573881771 >> [v2:172.21.15.47:6800/3942658432,v1:172.21.15.47:6801/3942658432] conn(0x7f0554039180 0x7f055403b640 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f055800ad30 tx=0x7f05580093f0 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:11:41.868 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:40.863+0000 7f0574a00700 1 -- 172.21.15.47:0/1573881771 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f056000e260 con 0x7f057007cdf0 2024-03-21T10:11:41.868 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:41.148+0000 7f0575e00700 1 -- 172.21.15.47:0/1573881771 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "cephadm"} v 0) v1 -- 0x7f05701a1030 con 0x7f057007cdf0 2024-03-21T10:11:41.868 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:41.202+0000 7f0574a00700 1 -- 172.21.15.47:0/1573881771 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "cephadm"}]=0 v5) v1 ==== 86+0+0 (secure 0 0 0) 0x7f05701a1030 con 0x7f057007cdf0 2024-03-21T10:11:41.868 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:41.205+0000 7f0574a00700 1 -- 172.21.15.47:0/1573881771 <== mon.0 v2:172.21.15.47:3300/0 8 ==== mgrmap(e 5) v1 ==== 46294+0+0 (secure 0 0 0) 0x7f056002d3f0 con 0x7f057007cdf0 2024-03-21T10:11:41.868 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:41.206+0000 7f0575e00700 1 -- 172.21.15.47:0/1573881771 >> [v2:172.21.15.47:6800/3942658432,v1:172.21.15.47:6801/3942658432] conn(0x7f0554039180 msgr2=0x7f055403b640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:11:41.868 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:41.206+0000 7f0575e00700 1 --2- 172.21.15.47:0/1573881771 >> [v2:172.21.15.47:6800/3942658432,v1:172.21.15.47:6801/3942658432] conn(0x7f0554039180 0x7f055403b640 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f055800ad30 tx=0x7f05580093f0 comp rx=0 tx=0).stop 2024-03-21T10:11:41.868 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:41.206+0000 7f0575e00700 1 -- 172.21.15.47:0/1573881771 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f057007cdf0 msgr2=0x7f057019d9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:11:41.868 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:41.206+0000 7f0575e00700 1 --2- 172.21.15.47:0/1573881771 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f057007cdf0 0x7f057019d9a0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f0560011a50 tx=0x7f0560011b30 comp rx=0 tx=0).stop 2024-03-21T10:11:41.868 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:41.207+0000 7f0575e00700 1 -- 172.21.15.47:0/1573881771 shutdown_connections 2024-03-21T10:11:41.868 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:41.207+0000 7f0575e00700 1 --2- 172.21.15.47:0/1573881771 >> [v2:172.21.15.47:6800/3942658432,v1:172.21.15.47:6801/3942658432] conn(0x7f0554039180 0x7f055403b640 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:11:41.868 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:41.208+0000 7f0575e00700 1 --2- 172.21.15.47:0/1573881771 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f057007cdf0 0x7f057019d9a0 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:11:41.868 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:41.208+0000 7f0575e00700 1 -- 172.21.15.47:0/1573881771 >> 172.21.15.47:0/1573881771 conn(0x7f0570102f50 msgr2=0x7f0570103a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:11:41.868 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:41.208+0000 7f0575e00700 1 -- 172.21.15.47:0/1573881771 shutdown_connections 2024-03-21T10:11:41.869 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:41.208+0000 7f0575e00700 1 -- 172.21.15.47:0/1573881771 wait complete. 2024-03-21T10:11:42.137 INFO:journalctl@ceph.mgr.x.smithi047.stdout:Mar 21 10:11:42 smithi047 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-mgr-x[31773]: 2024-03-21T10:11:42.001+0000 7f00c8d9d200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-03-21T10:11:42.137 INFO:journalctl@ceph.mgr.x.smithi047.stdout:Mar 21 10:11:42 smithi047 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-mgr-x[31773]: 2024-03-21T10:11:42.136+0000 7f00c8d9d200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-03-21T10:11:42.403 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:11:42 smithi047 ceph-mon[31547]: from='client.? 172.21.15.47:0/1573881771' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "cephadm"}]': finished 2024-03-21T10:11:42.403 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:11:42 smithi047 ceph-mon[31547]: mgrmap e5: x(active, since 5s) 2024-03-21T10:11:43.280 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:11:43 smithi047 ceph-mon[31547]: from='client.? 172.21.15.47:0/61187293' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-03-21T10:11:43.280 INFO:journalctl@ceph.mgr.x.smithi047.stdout:Mar 21 10:11:42 smithi047 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-mgr-x[31773]: 2024-03-21T10:11:42.969+0000 7f00c8d9d200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-03-21T10:11:43.280 INFO:journalctl@ceph.mgr.x.smithi047.stdout:Mar 21 10:11:43 smithi047 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-mgr-x[31773]: 2024-03-21T10:11:43.096+0000 7f00c8d9d200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-03-21T10:11:43.470 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout { 2024-03-21T10:11:43.470 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "epoch": 5, 2024-03-21T10:11:43.470 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "available": true, 2024-03-21T10:11:43.470 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "active_name": "x", 2024-03-21T10:11:43.470 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-03-21T10:11:43.470 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout } 2024-03-21T10:11:43.470 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:42.476+0000 7fc976c00700 1 Processor -- start 2024-03-21T10:11:43.471 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:42.476+0000 7fc976c00700 1 -- start start 2024-03-21T10:11:43.471 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:42.477+0000 7fc976c00700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc97007ce10 0x7fc97007b2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:11:43.471 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:42.477+0000 7fc976c00700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc97007b800 con 0x7fc97007ce10 2024-03-21T10:11:43.471 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:42.477+0000 7fc96fe00700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc97007ce10 0x7fc97007b2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:11:43.471 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:42.477+0000 7fc96fe00700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc97007ce10 0x7fc97007b2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:44936/0 (socket says 172.21.15.47:44936) 2024-03-21T10:11:43.471 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:42.477+0000 7fc96fe00700 1 -- 172.21.15.47:0/89057109 learned_addr learned my addr 172.21.15.47:0/89057109 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:11:43.471 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:42.478+0000 7fc96fe00700 1 -- 172.21.15.47:0/89057109 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc97007b980 con 0x7fc97007ce10 2024-03-21T10:11:43.471 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:42.478+0000 7fc96fe00700 1 --2- 172.21.15.47:0/89057109 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc97007ce10 0x7fc97007b2c0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fc9600098a0 tx=0x7fc960009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=86497cc0ec172814 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:11:43.471 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:42.478+0000 7fc96ea00700 1 -- 172.21.15.47:0/89057109 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc960004160 con 0x7fc97007ce10 2024-03-21T10:11:43.471 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:42.479+0000 7fc96ea00700 1 -- 172.21.15.47:0/89057109 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) v1 ==== 1071+0+0 (secure 0 0 0) 0x7fc960004300 con 0x7fc97007ce10 2024-03-21T10:11:43.471 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:42.479+0000 7fc96ea00700 1 -- 172.21.15.47:0/89057109 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc960011a80 con 0x7fc97007ce10 2024-03-21T10:11:43.471 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:42.479+0000 7fc976c00700 1 -- 172.21.15.47:0/89057109 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc97007ce10 msgr2=0x7fc97007b2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:11:43.471 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:42.479+0000 7fc976c00700 1 --2- 172.21.15.47:0/89057109 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc97007ce10 0x7fc97007b2c0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fc9600098a0 tx=0x7fc960009bb0 comp rx=0 tx=0).stop 2024-03-21T10:11:43.471 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:42.479+0000 7fc976c00700 1 -- 172.21.15.47:0/89057109 shutdown_connections 2024-03-21T10:11:43.471 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:42.479+0000 7fc976c00700 1 --2- 172.21.15.47:0/89057109 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc97007ce10 0x7fc97007b2c0 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:11:43.472 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:42.479+0000 7fc976c00700 1 -- 172.21.15.47:0/89057109 >> 172.21.15.47:0/89057109 conn(0x7fc970102f50 msgr2=0x7fc970105370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:11:43.472 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:42.479+0000 7fc976c00700 1 -- 172.21.15.47:0/89057109 shutdown_connections 2024-03-21T10:11:43.472 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:42.479+0000 7fc976c00700 1 -- 172.21.15.47:0/89057109 wait complete. 2024-03-21T10:11:43.472 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:42.480+0000 7fc976c00700 1 Processor -- start 2024-03-21T10:11:43.472 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:42.481+0000 7fc976c00700 1 -- start start 2024-03-21T10:11:43.472 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:42.481+0000 7fc976c00700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc97007ce10 0x7fc97019da00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:11:43.472 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:42.481+0000 7fc976c00700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc97010de20 con 0x7fc97007ce10 2024-03-21T10:11:43.472 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:42.481+0000 7fc96fe00700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc97007ce10 0x7fc97019da00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:11:43.472 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:42.481+0000 7fc96fe00700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc97007ce10 0x7fc97019da00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:44938/0 (socket says 172.21.15.47:44938) 2024-03-21T10:11:43.472 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:42.481+0000 7fc96fe00700 1 -- 172.21.15.47:0/61187293 learned_addr learned my addr 172.21.15.47:0/61187293 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:11:43.472 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:42.482+0000 7fc96fe00700 1 -- 172.21.15.47:0/61187293 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc97019df40 con 0x7fc97007ce10 2024-03-21T10:11:43.472 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:42.482+0000 7fc96fe00700 1 --2- 172.21.15.47:0/61187293 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc97007ce10 0x7fc97019da00 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fc960011a50 tx=0x7fc960004290 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:11:43.472 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:42.482+0000 7fc96cc00700 1 -- 172.21.15.47:0/61187293 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc9600039f0 con 0x7fc97007ce10 2024-03-21T10:11:43.472 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:42.482+0000 7fc96cc00700 1 -- 172.21.15.47:0/61187293 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) v1 ==== 1071+0+0 (secure 0 0 0) 0x7fc960003b90 con 0x7fc97007ce10 2024-03-21T10:11:43.472 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:42.482+0000 7fc96cc00700 1 -- 172.21.15.47:0/61187293 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc96001a600 con 0x7fc97007ce10 2024-03-21T10:11:43.473 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:42.482+0000 7fc976c00700 1 -- 172.21.15.47:0/61187293 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc97019e1d0 con 0x7fc97007ce10 2024-03-21T10:11:43.473 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:42.483+0000 7fc976c00700 1 -- 172.21.15.47:0/61187293 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc9701a0ec0 con 0x7fc97007ce10 2024-03-21T10:11:43.473 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:42.484+0000 7fc976c00700 1 -- 172.21.15.47:0/61187293 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc9700637c0 con 0x7fc97007ce10 2024-03-21T10:11:43.473 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:42.484+0000 7fc96cc00700 1 -- 172.21.15.47:0/61187293 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 46294+0+0 (secure 0 0 0) 0x7fc960024a40 con 0x7fc97007ce10 2024-03-21T10:11:43.473 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:42.485+0000 7fc96cc00700 1 --2- 172.21.15.47:0/61187293 >> [v2:172.21.15.47:6800/3942658432,v1:172.21.15.47:6801/3942658432] conn(0x7fc954039180 0x7fc95403b640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:11:43.473 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:42.485+0000 7fc96cc00700 1 -- 172.21.15.47:0/61187293 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7fc96004d5b0 con 0x7fc97007ce10 2024-03-21T10:11:43.473 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:42.485+0000 7fc96f400700 1 -- 172.21.15.47:0/61187293 >> [v2:172.21.15.47:6800/3942658432,v1:172.21.15.47:6801/3942658432] conn(0x7fc954039180 msgr2=0x7fc95403b640 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.47:6800/3942658432 2024-03-21T10:11:43.473 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:42.485+0000 7fc96f400700 1 --2- 172.21.15.47:0/61187293 >> [v2:172.21.15.47:6800/3942658432,v1:172.21.15.47:6801/3942658432] conn(0x7fc954039180 0x7fc95403b640 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-21T10:11:43.473 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:42.490+0000 7fc96cc00700 1 -- 172.21.15.47:0/61187293 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fc96004cfd0 con 0x7fc97007ce10 2024-03-21T10:11:43.473 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:42.686+0000 7fc96f400700 1 -- 172.21.15.47:0/61187293 >> [v2:172.21.15.47:6800/3942658432,v1:172.21.15.47:6801/3942658432] conn(0x7fc954039180 msgr2=0x7fc95403b640 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.47:6800/3942658432 2024-03-21T10:11:43.473 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:42.686+0000 7fc96f400700 1 --2- 172.21.15.47:0/61187293 >> [v2:172.21.15.47:6800/3942658432,v1:172.21.15.47:6801/3942658432] conn(0x7fc954039180 0x7fc95403b640 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-21T10:11:43.473 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:42.778+0000 7fc976c00700 1 -- 172.21.15.47:0/61187293 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) v1 -- 0x7fc9701a1090 con 0x7fc97007ce10 2024-03-21T10:11:43.473 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:42.778+0000 7fc96cc00700 1 -- 172.21.15.47:0/61187293 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v5) v1 ==== 56+0+88 (secure 0 0 0) 0x7fc9701a1090 con 0x7fc97007ce10 2024-03-21T10:11:43.473 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:42.781+0000 7fc976c00700 1 -- 172.21.15.47:0/61187293 >> [v2:172.21.15.47:6800/3942658432,v1:172.21.15.47:6801/3942658432] conn(0x7fc954039180 msgr2=0x7fc95403b640 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-21T10:11:43.473 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:42.781+0000 7fc976c00700 1 --2- 172.21.15.47:0/61187293 >> [v2:172.21.15.47:6800/3942658432,v1:172.21.15.47:6801/3942658432] conn(0x7fc954039180 0x7fc95403b640 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:11:43.474 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:42.781+0000 7fc976c00700 1 -- 172.21.15.47:0/61187293 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc97007ce10 msgr2=0x7fc97019da00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:11:43.474 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:42.781+0000 7fc976c00700 1 --2- 172.21.15.47:0/61187293 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc97007ce10 0x7fc97019da00 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fc960011a50 tx=0x7fc960004290 comp rx=0 tx=0).stop 2024-03-21T10:11:43.474 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:42.781+0000 7fc976c00700 1 -- 172.21.15.47:0/61187293 shutdown_connections 2024-03-21T10:11:43.474 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:42.781+0000 7fc976c00700 1 --2- 172.21.15.47:0/61187293 >> [v2:172.21.15.47:6800/3942658432,v1:172.21.15.47:6801/3942658432] conn(0x7fc954039180 0x7fc95403b640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:11:43.474 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:42.781+0000 7fc976c00700 1 --2- 172.21.15.47:0/61187293 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc97007ce10 0x7fc97019da00 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:11:43.474 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:42.781+0000 7fc976c00700 1 -- 172.21.15.47:0/61187293 >> 172.21.15.47:0/61187293 conn(0x7fc970102f50 msgr2=0x7fc970103ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:11:43.474 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:42.782+0000 7fc976c00700 1 -- 172.21.15.47:0/61187293 shutdown_connections 2024-03-21T10:11:43.474 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:42.782+0000 7fc976c00700 1 -- 172.21.15.47:0/61187293 wait complete. 2024-03-21T10:11:43.474 INFO:teuthology.orchestra.run.smithi047.stdout:Waiting for the mgr to restart... 2024-03-21T10:11:43.474 INFO:teuthology.orchestra.run.smithi047.stdout:Waiting for mgr epoch 5... 2024-03-21T10:11:43.617 INFO:journalctl@ceph.mgr.x.smithi047.stdout:Mar 21 10:11:43 smithi047 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-mgr-x[31773]: 2024-03-21T10:11:43.278+0000 7f00c8d9d200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-03-21T10:11:43.617 INFO:journalctl@ceph.mgr.x.smithi047.stdout:Mar 21 10:11:43 smithi047 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-mgr-x[31773]: 2024-03-21T10:11:43.406+0000 7f00c8d9d200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-03-21T10:11:43.617 INFO:journalctl@ceph.mgr.x.smithi047.stdout:Mar 21 10:11:43 smithi047 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-mgr-x[31773]: 2024-03-21T10:11:43.525+0000 7f00c8d9d200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-03-21T10:11:44.138 INFO:journalctl@ceph.mgr.x.smithi047.stdout:Mar 21 10:11:43 smithi047 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-mgr-x[31773]: 2024-03-21T10:11:43.873+0000 7f00c8d9d200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-03-21T10:11:44.138 INFO:journalctl@ceph.mgr.x.smithi047.stdout:Mar 21 10:11:44 smithi047 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-mgr-x[31773]: 2024-03-21T10:11:44.020+0000 7f00c8d9d200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-03-21T10:11:44.138 INFO:journalctl@ceph.mgr.x.smithi047.stdout:Mar 21 10:11:44 smithi047 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-mgr-x[31773]: 2024-03-21T10:11:44.136+0000 7f00c8d9d200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-03-21T10:11:44.463 INFO:journalctl@ceph.mgr.x.smithi047.stdout:Mar 21 10:11:44 smithi047 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-mgr-x[31773]: 2024-03-21T10:11:44.263+0000 7f00c8d9d200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-03-21T10:11:44.963 INFO:journalctl@ceph.mgr.x.smithi047.stdout:Mar 21 10:11:44 smithi047 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-mgr-x[31773]: 2024-03-21T10:11:44.486+0000 7f00c8d9d200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-03-21T10:11:46.214 INFO:journalctl@ceph.mgr.x.smithi047.stdout:Mar 21 10:11:45 smithi047 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-mgr-x[31773]: 2024-03-21T10:11:45.794+0000 7f00c8d9d200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-03-21T10:11:46.214 INFO:journalctl@ceph.mgr.x.smithi047.stdout:Mar 21 10:11:45 smithi047 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-mgr-x[31773]: 2024-03-21T10:11:45.972+0000 7f00c8d9d200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-03-21T10:11:46.714 INFO:journalctl@ceph.mgr.x.smithi047.stdout:Mar 21 10:11:46 smithi047 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-mgr-x[31773]: 2024-03-21T10:11:46.399+0000 7f00c8d9d200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-03-21T10:11:46.714 INFO:journalctl@ceph.mgr.x.smithi047.stdout:Mar 21 10:11:46 smithi047 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-mgr-x[31773]: 2024-03-21T10:11:46.519+0000 7f00c8d9d200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-03-21T10:11:47.035 INFO:journalctl@ceph.mgr.x.smithi047.stdout:Mar 21 10:11:46 smithi047 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-mgr-x[31773]: 2024-03-21T10:11:46.781+0000 7f00c8d9d200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-03-21T10:11:47.421 INFO:journalctl@ceph.mgr.x.smithi047.stdout:Mar 21 10:11:47 smithi047 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-mgr-x[31773]: 2024-03-21T10:11:47.033+0000 7f00c8d9d200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-03-21T10:11:47.713 INFO:journalctl@ceph.mgr.x.smithi047.stdout:Mar 21 10:11:47 smithi047 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-mgr-x[31773]: 2024-03-21T10:11:47.418+0000 7f00c8d9d200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-03-21T10:11:48.080 INFO:journalctl@ceph.mgr.x.smithi047.stdout:Mar 21 10:11:47 smithi047 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-mgr-x[31773]: 2024-03-21T10:11:47.762+0000 7f00c8d9d200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-03-21T10:11:48.438 INFO:journalctl@ceph.mgr.x.smithi047.stdout:Mar 21 10:11:48 smithi047 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-mgr-x[31773]: 2024-03-21T10:11:48.078+0000 7f00c8d9d200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-03-21T10:11:49.464 INFO:journalctl@ceph.mgr.x.smithi047.stdout:Mar 21 10:11:49 smithi047 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-mgr-x[31773]: 2024-03-21T10:11:49.157+0000 7f00c8d9d200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-03-21T10:11:49.963 INFO:journalctl@ceph.mgr.x.smithi047.stdout:Mar 21 10:11:49 smithi047 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-mgr-x[31773]: 2024-03-21T10:11:49.876+0000 7f00c8d9d200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-03-21T10:11:50.287 INFO:journalctl@ceph.mgr.x.smithi047.stdout:Mar 21 10:11:50 smithi047 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-mgr-x[31773]: 2024-03-21T10:11:50.008+0000 7f00c8d9d200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-03-21T10:11:50.713 INFO:journalctl@ceph.mgr.x.smithi047.stdout:Mar 21 10:11:50 smithi047 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-mgr-x[31773]: 2024-03-21T10:11:50.299+0000 7f00c8d9d200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-03-21T10:11:50.714 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:11:50 smithi047 ceph-mon[31547]: Active manager daemon x restarted 2024-03-21T10:11:50.714 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:11:50 smithi047 ceph-mon[31547]: Activating manager daemon x 2024-03-21T10:11:50.714 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:11:50 smithi047 ceph-mon[31547]: osdmap e2: 0 total, 0 up, 0 in 2024-03-21T10:11:50.714 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:11:50 smithi047 ceph-mon[31547]: mgrmap e6: x(active, starting, since 0.00563416s) 2024-03-21T10:11:50.714 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:11:50 smithi047 ceph-mon[31547]: from='mgr.14120 172.21.15.47:0/2543639975' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-21T10:11:50.714 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:11:50 smithi047 ceph-mon[31547]: from='mgr.14120 172.21.15.47:0/2543639975' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-03-21T10:11:50.714 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:11:50 smithi047 ceph-mon[31547]: from='mgr.14120 172.21.15.47:0/2543639975' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-03-21T10:11:50.714 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:11:50 smithi047 ceph-mon[31547]: from='mgr.14120 172.21.15.47:0/2543639975' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-03-21T10:11:50.714 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:11:50 smithi047 ceph-mon[31547]: from='mgr.14120 172.21.15.47:0/2543639975' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-03-21T10:11:50.714 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:11:50 smithi047 ceph-mon[31547]: Manager daemon x is now available 2024-03-21T10:11:50.714 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:11:50 smithi047 ceph-mon[31547]: from='mgr.14120 172.21.15.47:0/2543639975' entity='mgr.x' 2024-03-21T10:11:51.714 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:11:51 smithi047 ceph-mon[31547]: Found migration_current of "None". Setting to last migration. 2024-03-21T10:11:51.714 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:11:51 smithi047 ceph-mon[31547]: from='mgr.14120 172.21.15.47:0/2543639975' entity='mgr.x' 2024-03-21T10:11:51.714 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:11:51 smithi047 ceph-mon[31547]: from='mgr.14120 172.21.15.47:0/2543639975' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T10:11:51.714 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:11:51 smithi047 ceph-mon[31547]: from='mgr.14120 172.21.15.47:0/2543639975' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T10:11:51.714 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:11:51 smithi047 ceph-mon[31547]: from='mgr.14120 172.21.15.47:0/2543639975' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-03-21T10:11:51.714 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:11:51 smithi047 ceph-mon[31547]: from='mgr.14120 172.21.15.47:0/2543639975' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-03-21T10:11:51.714 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:11:51 smithi047 ceph-mon[31547]: from='mgr.14120 172.21.15.47:0/2543639975' entity='mgr.x' 2024-03-21T10:11:51.714 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:11:51 smithi047 ceph-mon[31547]: from='mgr.14120 172.21.15.47:0/2543639975' entity='mgr.x' 2024-03-21T10:11:51.714 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:11:51 smithi047 ceph-mon[31547]: from='client.14124 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-03-21T10:11:51.714 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:11:51 smithi047 ceph-mon[31547]: mgrmap e7: x(active, since 1.01674s) 2024-03-21T10:11:52.124 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout { 2024-03-21T10:11:52.125 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 7, 2024-03-21T10:11:52.125 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "initialized": true 2024-03-21T10:11:52.125 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout } 2024-03-21T10:11:52.126 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:44.068+0000 7f4f98000700 1 Processor -- start 2024-03-21T10:11:52.126 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:44.068+0000 7f4f98000700 1 -- start start 2024-03-21T10:11:52.126 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:44.069+0000 7f4f98000700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4f901060e0 0x7f4f90106500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:11:52.126 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:44.069+0000 7f4f98000700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f9007ae90 con 0x7f4f901060e0 2024-03-21T10:11:52.126 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:44.069+0000 7f4f95600700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4f901060e0 0x7f4f90106500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:11:52.126 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:44.069+0000 7f4f95600700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4f901060e0 0x7f4f90106500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:44946/0 (socket says 172.21.15.47:44946) 2024-03-21T10:11:52.138 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:44.069+0000 7f4f95600700 1 -- 172.21.15.47:0/3447532931 learned_addr learned my addr 172.21.15.47:0/3447532931 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:11:52.138 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:44.070+0000 7f4f95600700 1 -- 172.21.15.47:0/3447532931 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4f90106a40 con 0x7f4f901060e0 2024-03-21T10:11:52.139 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:44.070+0000 7f4f95600700 1 --2- 172.21.15.47:0/3447532931 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4f901060e0 0x7f4f90106500 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f4f840098a0 tx=0x7f4f84009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dc4b762bcb4c1f2d server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:11:52.139 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:44.070+0000 7f4f8fe00700 1 -- 172.21.15.47:0/3447532931 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f4f84004160 con 0x7f4f901060e0 2024-03-21T10:11:52.139 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:44.070+0000 7f4f8fe00700 1 -- 172.21.15.47:0/3447532931 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) v1 ==== 1071+0+0 (secure 0 0 0) 0x7f4f84004300 con 0x7f4f901060e0 2024-03-21T10:11:52.139 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:44.070+0000 7f4f8fe00700 1 -- 172.21.15.47:0/3447532931 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f4f84011a80 con 0x7f4f901060e0 2024-03-21T10:11:52.139 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:44.071+0000 7f4f98000700 1 -- 172.21.15.47:0/3447532931 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4f901060e0 msgr2=0x7f4f90106500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:11:52.139 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:44.071+0000 7f4f98000700 1 --2- 172.21.15.47:0/3447532931 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4f901060e0 0x7f4f90106500 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f4f840098a0 tx=0x7f4f84009bb0 comp rx=0 tx=0).stop 2024-03-21T10:11:52.140 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:44.071+0000 7f4f98000700 1 -- 172.21.15.47:0/3447532931 shutdown_connections 2024-03-21T10:11:52.140 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:44.071+0000 7f4f98000700 1 --2- 172.21.15.47:0/3447532931 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4f901060e0 0x7f4f90106500 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:11:52.140 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:44.071+0000 7f4f98000700 1 -- 172.21.15.47:0/3447532931 >> 172.21.15.47:0/3447532931 conn(0x7f4f90101de0 msgr2=0x7f4f90104200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:11:52.140 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:44.071+0000 7f4f98000700 1 -- 172.21.15.47:0/3447532931 shutdown_connections 2024-03-21T10:11:52.140 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:44.072+0000 7f4f98000700 1 -- 172.21.15.47:0/3447532931 wait complete. 2024-03-21T10:11:52.140 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:44.073+0000 7f4f98000700 1 Processor -- start 2024-03-21T10:11:52.140 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:44.073+0000 7f4f98000700 1 -- start start 2024-03-21T10:11:52.140 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:44.074+0000 7f4f98000700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4f901060e0 0x7f4f9010bb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:11:52.140 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:44.074+0000 7f4f98000700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f90107a70 con 0x7f4f901060e0 2024-03-21T10:11:52.140 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:44.074+0000 7f4f95600700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4f901060e0 0x7f4f9010bb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:11:52.140 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:44.074+0000 7f4f95600700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4f901060e0 0x7f4f9010bb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:44958/0 (socket says 172.21.15.47:44958) 2024-03-21T10:11:52.140 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:44.074+0000 7f4f95600700 1 -- 172.21.15.47:0/870331445 learned_addr learned my addr 172.21.15.47:0/870331445 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:11:52.141 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:44.074+0000 7f4f95600700 1 -- 172.21.15.47:0/870331445 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4f9010c0c0 con 0x7f4f901060e0 2024-03-21T10:11:52.141 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:44.074+0000 7f4f95600700 1 --2- 172.21.15.47:0/870331445 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4f901060e0 0x7f4f9010bb80 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f4f84011a50 tx=0x7f4f84004290 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:11:52.141 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:44.075+0000 7f4f8e000700 1 -- 172.21.15.47:0/870331445 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f4f840039f0 con 0x7f4f901060e0 2024-03-21T10:11:52.141 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:44.075+0000 7f4f8e000700 1 -- 172.21.15.47:0/870331445 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) v1 ==== 1071+0+0 (secure 0 0 0) 0x7f4f84003b90 con 0x7f4f901060e0 2024-03-21T10:11:52.141 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:44.075+0000 7f4f98000700 1 -- 172.21.15.47:0/870331445 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4f9010c350 con 0x7f4f901060e0 2024-03-21T10:11:52.141 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:44.075+0000 7f4f8e000700 1 -- 172.21.15.47:0/870331445 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f4f8401a600 con 0x7f4f901060e0 2024-03-21T10:11:52.141 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:44.075+0000 7f4f98000700 1 -- 172.21.15.47:0/870331445 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4f90108910 con 0x7f4f901060e0 2024-03-21T10:11:52.141 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:44.076+0000 7f4f8e000700 1 -- 172.21.15.47:0/870331445 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 46294+0+0 (secure 0 0 0) 0x7f4f84024a40 con 0x7f4f901060e0 2024-03-21T10:11:52.141 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:44.077+0000 7f4f8e000700 1 --2- 172.21.15.47:0/870331445 >> [v2:172.21.15.47:6800/3942658432,v1:172.21.15.47:6801/3942658432] conn(0x7f4f78039180 0x7f4f7803b640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:11:52.141 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:44.077+0000 7f4f8e000700 1 -- 172.21.15.47:0/870331445 --> [v2:172.21.15.47:6800/3942658432,v1:172.21.15.47:6801/3942658432] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7f4f7803bd50 con 0x7f4f78039180 2024-03-21T10:11:52.141 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:44.077+0000 7f4f8e000700 1 -- 172.21.15.47:0/870331445 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f4f8404d5b0 con 0x7f4f901060e0 2024-03-21T10:11:52.141 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:44.077+0000 7f4f94c00700 1 -- 172.21.15.47:0/870331445 >> [v2:172.21.15.47:6800/3942658432,v1:172.21.15.47:6801/3942658432] conn(0x7f4f78039180 msgr2=0x7f4f7803b640 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.47:6800/3942658432 2024-03-21T10:11:52.141 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:44.077+0000 7f4f94c00700 1 --2- 172.21.15.47:0/870331445 >> [v2:172.21.15.47:6800/3942658432,v1:172.21.15.47:6801/3942658432] conn(0x7f4f78039180 0x7f4f7803b640 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-21T10:11:52.141 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:44.278+0000 7f4f94c00700 1 -- 172.21.15.47:0/870331445 >> [v2:172.21.15.47:6800/3942658432,v1:172.21.15.47:6801/3942658432] conn(0x7f4f78039180 msgr2=0x7f4f7803b640 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.47:6800/3942658432 2024-03-21T10:11:52.141 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:44.278+0000 7f4f94c00700 1 --2- 172.21.15.47:0/870331445 >> [v2:172.21.15.47:6800/3942658432,v1:172.21.15.47:6801/3942658432] conn(0x7f4f78039180 0x7f4f7803b640 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-21T10:11:52.142 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:44.679+0000 7f4f94c00700 1 -- 172.21.15.47:0/870331445 >> [v2:172.21.15.47:6800/3942658432,v1:172.21.15.47:6801/3942658432] conn(0x7f4f78039180 msgr2=0x7f4f7803b640 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.47:6800/3942658432 2024-03-21T10:11:52.142 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:44.679+0000 7f4f94c00700 1 --2- 172.21.15.47:0/870331445 >> [v2:172.21.15.47:6800/3942658432,v1:172.21.15.47:6801/3942658432] conn(0x7f4f78039180 0x7f4f7803b640 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-21T10:11:52.142 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:45.480+0000 7f4f94c00700 1 -- 172.21.15.47:0/870331445 >> [v2:172.21.15.47:6800/3942658432,v1:172.21.15.47:6801/3942658432] conn(0x7f4f78039180 msgr2=0x7f4f7803b640 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.47:6800/3942658432 2024-03-21T10:11:52.142 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:45.480+0000 7f4f94c00700 1 --2- 172.21.15.47:0/870331445 >> [v2:172.21.15.47:6800/3942658432,v1:172.21.15.47:6801/3942658432] conn(0x7f4f78039180 0x7f4f7803b640 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-21T10:11:52.142 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:47.082+0000 7f4f94c00700 1 -- 172.21.15.47:0/870331445 >> [v2:172.21.15.47:6800/3942658432,v1:172.21.15.47:6801/3942658432] conn(0x7f4f78039180 msgr2=0x7f4f7803b640 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.47:6800/3942658432 2024-03-21T10:11:52.142 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:47.082+0000 7f4f94c00700 1 --2- 172.21.15.47:0/870331445 >> [v2:172.21.15.47:6800/3942658432,v1:172.21.15.47:6801/3942658432] conn(0x7f4f78039180 0x7f4f7803b640 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2024-03-21T10:11:52.142 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:50.285+0000 7f4f94c00700 1 -- 172.21.15.47:0/870331445 >> [v2:172.21.15.47:6800/3942658432,v1:172.21.15.47:6801/3942658432] conn(0x7f4f78039180 msgr2=0x7f4f7803b640 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.47:6800/3942658432 2024-03-21T10:11:52.142 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:50.285+0000 7f4f94c00700 1 --2- 172.21.15.47:0/870331445 >> [v2:172.21.15.47:6800/3942658432,v1:172.21.15.47:6801/3942658432] conn(0x7f4f78039180 0x7f4f7803b640 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 6.400000 2024-03-21T10:11:52.142 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:50.307+0000 7f4f8e000700 1 -- 172.21.15.47:0/870331445 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mgrmap(e 6) v1 ==== 46061+0+0 (secure 0 0 0) 0x7f4f84024440 con 0x7f4f901060e0 2024-03-21T10:11:52.142 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:50.307+0000 7f4f8e000700 1 -- 172.21.15.47:0/870331445 >> [v2:172.21.15.47:6800/3942658432,v1:172.21.15.47:6801/3942658432] conn(0x7f4f78039180 msgr2=0x7f4f7803b640 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-21T10:11:52.142 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:50.307+0000 7f4f8e000700 1 --2- 172.21.15.47:0/870331445 >> [v2:172.21.15.47:6800/3942658432,v1:172.21.15.47:6801/3942658432] conn(0x7f4f78039180 0x7f4f7803b640 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:11:52.143 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:51.312+0000 7f4f8e000700 1 -- 172.21.15.47:0/870331445 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mgrmap(e 7) v1 ==== 46188+0+0 (secure 0 0 0) 0x7f4f8404ecb0 con 0x7f4f901060e0 2024-03-21T10:11:52.143 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:51.313+0000 7f4f8e000700 1 --2- 172.21.15.47:0/870331445 >> [v2:172.21.15.47:6800/3836903972,v1:172.21.15.47:6801/3836903972] conn(0x7f4f7803ca70 0x7f4f7803ee60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:11:52.143 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:51.313+0000 7f4f8e000700 1 -- 172.21.15.47:0/870331445 --> [v2:172.21.15.47:6800/3836903972,v1:172.21.15.47:6801/3836903972] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7f4f8400af30 con 0x7f4f7803ca70 2024-03-21T10:11:52.143 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:51.314+0000 7f4f94c00700 1 --2- 172.21.15.47:0/870331445 >> [v2:172.21.15.47:6800/3836903972,v1:172.21.15.47:6801/3836903972] conn(0x7f4f7803ca70 0x7f4f7803ee60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:11:52.143 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:51.316+0000 7f4f94c00700 1 --2- 172.21.15.47:0/870331445 >> [v2:172.21.15.47:6800/3836903972,v1:172.21.15.47:6801/3836903972] conn(0x7f4f7803ca70 0x7f4f7803ee60 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f4f7c003a10 tx=0x7f4f7c009290 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:11:52.143 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:51.318+0000 7f4f8e000700 1 -- 172.21.15.47:0/870331445 <== mgr.14120 v2:172.21.15.47:6800/3836903972 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+7774 (secure 0 0 0) 0x7f4f8400af30 con 0x7f4f7803ca70 2024-03-21T10:11:52.143 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:51.327+0000 7f4f98000700 1 -- 172.21.15.47:0/870331445 --> [v2:172.21.15.47:6800/3836903972,v1:172.21.15.47:6801/3836903972] -- command(tid 1: {"prefix": "mgr_status"}) v1 -- 0x7f4f9010c520 con 0x7f4f7803ca70 2024-03-21T10:11:52.143 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:51.327+0000 7f4f8e000700 1 -- 172.21.15.47:0/870331445 <== mgr.14120 v2:172.21.15.47:6800/3836903972 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+51 (secure 0 0 0) 0x7f4f9010c520 con 0x7f4f7803ca70 2024-03-21T10:11:52.143 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:51.328+0000 7f4f98000700 1 -- 172.21.15.47:0/870331445 >> [v2:172.21.15.47:6800/3836903972,v1:172.21.15.47:6801/3836903972] conn(0x7f4f7803ca70 msgr2=0x7f4f7803ee60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:11:52.143 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:51.328+0000 7f4f98000700 1 --2- 172.21.15.47:0/870331445 >> [v2:172.21.15.47:6800/3836903972,v1:172.21.15.47:6801/3836903972] conn(0x7f4f7803ca70 0x7f4f7803ee60 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f4f7c003a10 tx=0x7f4f7c009290 comp rx=0 tx=0).stop 2024-03-21T10:11:52.143 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:51.328+0000 7f4f98000700 1 -- 172.21.15.47:0/870331445 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4f901060e0 msgr2=0x7f4f9010bb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:11:52.143 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:51.328+0000 7f4f98000700 1 --2- 172.21.15.47:0/870331445 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4f901060e0 0x7f4f9010bb80 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f4f84011a50 tx=0x7f4f84004290 comp rx=0 tx=0).stop 2024-03-21T10:11:52.143 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:51.328+0000 7f4f98000700 1 -- 172.21.15.47:0/870331445 shutdown_connections 2024-03-21T10:11:52.143 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:51.328+0000 7f4f98000700 1 --2- 172.21.15.47:0/870331445 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4f901060e0 0x7f4f9010bb80 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:11:52.144 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:51.328+0000 7f4f98000700 1 --2- 172.21.15.47:0/870331445 >> [v2:172.21.15.47:6800/3836903972,v1:172.21.15.47:6801/3836903972] conn(0x7f4f7803ca70 0x7f4f7803ee60 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:11:52.144 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:51.328+0000 7f4f98000700 1 --2- 172.21.15.47:0/870331445 >> [v2:172.21.15.47:6800/3942658432,v1:172.21.15.47:6801/3942658432] conn(0x7f4f78039180 0x7f4f7803b640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:11:52.144 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:51.328+0000 7f4f98000700 1 -- 172.21.15.47:0/870331445 >> 172.21.15.47:0/870331445 conn(0x7f4f90101de0 msgr2=0x7f4f90194590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:11:52.144 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:51.328+0000 7f4f98000700 1 -- 172.21.15.47:0/870331445 shutdown_connections 2024-03-21T10:11:52.144 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:51.328+0000 7f4f98000700 1 -- 172.21.15.47:0/870331445 wait complete. 2024-03-21T10:11:52.144 INFO:teuthology.orchestra.run.smithi047.stdout:mgr epoch 5 is available 2024-03-21T10:11:52.144 INFO:teuthology.orchestra.run.smithi047.stdout:Setting orchestrator backend to cephadm... 2024-03-21T10:11:52.422 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:11:52 smithi047 ceph-mon[31547]: from='client.14124 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-03-21T10:11:52.422 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:11:52 smithi047 ceph-mon[31547]: [21/Mar/2024:10:11:51] ENGINE Bus STARTING 2024-03-21T10:11:52.422 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:11:52 smithi047 ceph-mon[31547]: [21/Mar/2024:10:11:52] ENGINE Serving on http://172.21.15.47:8765 2024-03-21T10:11:52.422 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:11:52 smithi047 ceph-mon[31547]: [21/Mar/2024:10:11:52] ENGINE Serving on https://172.21.15.47:7150 2024-03-21T10:11:52.423 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:11:52 smithi047 ceph-mon[31547]: [21/Mar/2024:10:11:52] ENGINE Bus STARTED 2024-03-21T10:11:52.423 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:11:52 smithi047 ceph-mon[31547]: from='mgr.14120 172.21.15.47:0/2543639975' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T10:11:53.714 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:11:53 smithi047 ceph-mon[31547]: mgrmap e8: x(active, since 2s) 2024-03-21T10:11:53.714 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:11:53 smithi047 ceph-mon[31547]: from='client.14132 -' entity='client.admin' cmd=[{"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:11:53.714 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:11:53 smithi047 ceph-mon[31547]: from='mgr.14120 172.21.15.47:0/2543639975' entity='mgr.x' 2024-03-21T10:11:53.714 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:11:53 smithi047 ceph-mon[31547]: from='mgr.14120 172.21.15.47:0/2543639975' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T10:11:53.841 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:52.828+0000 7ff539400700 1 Processor -- start 2024-03-21T10:11:53.841 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:52.829+0000 7ff539400700 1 -- start start 2024-03-21T10:11:53.841 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:52.829+0000 7ff539400700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff534109ba0 0x7ff534109fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:11:53.841 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:52.829+0000 7ff539400700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff53410a590 con 0x7ff534109ba0 2024-03-21T10:11:53.841 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:52.830+0000 7ff532800700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff534109ba0 0x7ff534109fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:11:53.842 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:52.830+0000 7ff532800700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff534109ba0 0x7ff534109fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:40674/0 (socket says 172.21.15.47:40674) 2024-03-21T10:11:53.842 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:52.830+0000 7ff532800700 1 -- 172.21.15.47:0/4081718991 learned_addr learned my addr 172.21.15.47:0/4081718991 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:11:53.846 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:52.830+0000 7ff532800700 1 -- 172.21.15.47:0/4081718991 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff53410ad00 con 0x7ff534109ba0 2024-03-21T10:11:53.847 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:52.830+0000 7ff532800700 1 --2- 172.21.15.47:0/4081718991 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff534109ba0 0x7ff534109fc0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7ff524009e80 tx=0x7ff52400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=23d6f3a9dad2d38e server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:11:53.847 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:52.831+0000 7ff531e00700 1 -- 172.21.15.47:0/4081718991 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ff524017070 con 0x7ff534109ba0 2024-03-21T10:11:53.847 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:52.831+0000 7ff531e00700 1 -- 172.21.15.47:0/4081718991 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) v1 ==== 1071+0+0 (secure 0 0 0) 0x7ff52400b8c0 con 0x7ff534109ba0 2024-03-21T10:11:53.847 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:52.831+0000 7ff531e00700 1 -- 172.21.15.47:0/4081718991 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ff524004520 con 0x7ff534109ba0 2024-03-21T10:11:53.847 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:52.831+0000 7ff539400700 1 -- 172.21.15.47:0/4081718991 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff534109ba0 msgr2=0x7ff534109fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:11:53.847 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:52.831+0000 7ff539400700 1 --2- 172.21.15.47:0/4081718991 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff534109ba0 0x7ff534109fc0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7ff524009e80 tx=0x7ff52400b330 comp rx=0 tx=0).stop 2024-03-21T10:11:53.847 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:52.832+0000 7ff539400700 1 -- 172.21.15.47:0/4081718991 shutdown_connections 2024-03-21T10:11:53.847 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:52.832+0000 7ff539400700 1 --2- 172.21.15.47:0/4081718991 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff534109ba0 0x7ff534109fc0 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:11:53.847 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:52.832+0000 7ff539400700 1 -- 172.21.15.47:0/4081718991 >> 172.21.15.47:0/4081718991 conn(0x7ff5341054a0 msgr2=0x7ff534107890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:11:53.847 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:52.832+0000 7ff539400700 1 -- 172.21.15.47:0/4081718991 shutdown_connections 2024-03-21T10:11:53.847 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:52.832+0000 7ff539400700 1 -- 172.21.15.47:0/4081718991 wait complete. 2024-03-21T10:11:53.847 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:52.833+0000 7ff539400700 1 Processor -- start 2024-03-21T10:11:53.847 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:52.833+0000 7ff539400700 1 -- start start 2024-03-21T10:11:53.848 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:52.833+0000 7ff539400700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff534109ba0 0x7ff53419d900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:11:53.848 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:52.833+0000 7ff539400700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff53410e0b0 con 0x7ff534109ba0 2024-03-21T10:11:53.848 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:52.834+0000 7ff532800700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff534109ba0 0x7ff53419d900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:11:53.848 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:52.834+0000 7ff532800700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff534109ba0 0x7ff53419d900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:40686/0 (socket says 172.21.15.47:40686) 2024-03-21T10:11:53.848 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:52.834+0000 7ff532800700 1 -- 172.21.15.47:0/613838233 learned_addr learned my addr 172.21.15.47:0/613838233 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:11:53.848 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:52.834+0000 7ff532800700 1 -- 172.21.15.47:0/613838233 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff53419de40 con 0x7ff534109ba0 2024-03-21T10:11:53.848 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:52.834+0000 7ff532800700 1 --2- 172.21.15.47:0/613838233 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff534109ba0 0x7ff53419d900 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7ff524003f60 tx=0x7ff524004670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:11:53.848 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:52.835+0000 7ff52be00700 1 -- 172.21.15.47:0/613838233 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ff524020020 con 0x7ff534109ba0 2024-03-21T10:11:53.848 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:52.835+0000 7ff52be00700 1 -- 172.21.15.47:0/613838233 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) v1 ==== 1071+0+0 (secure 0 0 0) 0x7ff52400bcb0 con 0x7ff534109ba0 2024-03-21T10:11:53.848 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:52.835+0000 7ff52be00700 1 -- 172.21.15.47:0/613838233 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ff524017040 con 0x7ff534109ba0 2024-03-21T10:11:53.848 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:52.835+0000 7ff539400700 1 -- 172.21.15.47:0/613838233 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff53419e0d0 con 0x7ff534109ba0 2024-03-21T10:11:53.848 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:52.835+0000 7ff539400700 1 -- 172.21.15.47:0/613838233 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff5341a0dc0 con 0x7ff534109ba0 2024-03-21T10:11:53.848 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:52.836+0000 7ff52be00700 1 -- 172.21.15.47:0/613838233 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 8) v1 ==== 46294+0+0 (secure 0 0 0) 0x7ff524023020 con 0x7ff534109ba0 2024-03-21T10:11:53.848 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:52.836+0000 7ff539400700 1 -- 172.21.15.47:0/613838233 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff5340637c0 con 0x7ff534109ba0 2024-03-21T10:11:53.849 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:52.837+0000 7ff52be00700 1 --2- 172.21.15.47:0/613838233 >> [v2:172.21.15.47:6800/3836903972,v1:172.21.15.47:6801/3836903972] conn(0x7ff518039290 0x7ff51803b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:11:53.849 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:52.837+0000 7ff52be00700 1 -- 172.21.15.47:0/613838233 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7ff52400f050 con 0x7ff534109ba0 2024-03-21T10:11:53.849 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:52.837+0000 7ff529e00700 1 --2- 172.21.15.47:0/613838233 >> [v2:172.21.15.47:6800/3836903972,v1:172.21.15.47:6801/3836903972] conn(0x7ff518039290 0x7ff51803b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:11:53.849 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:52.838+0000 7ff529e00700 1 --2- 172.21.15.47:0/613838233 >> [v2:172.21.15.47:6800/3836903972,v1:172.21.15.47:6801/3836903972] conn(0x7ff518039290 0x7ff51803b750 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7ff51c006fd0 tx=0x7ff51c006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:11:53.849 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:52.843+0000 7ff52be00700 1 -- 172.21.15.47:0/613838233 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7ff52402a020 con 0x7ff534109ba0 2024-03-21T10:11:53.849 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:53.089+0000 7ff539400700 1 -- 172.21.15.47:0/613838233 --> [v2:172.21.15.47:6800/3836903972,v1:172.21.15.47:6801/3836903972] -- mgr_command(tid 0: {"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}) v1 -- 0x7ff534107590 con 0x7ff518039290 2024-03-21T10:11:53.849 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:53.103+0000 7ff52be00700 1 -- 172.21.15.47:0/613838233 <== mgr.14120 v2:172.21.15.47:6800/3836903972 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7ff534107590 con 0x7ff518039290 2024-03-21T10:11:53.849 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:53.106+0000 7ff539400700 1 -- 172.21.15.47:0/613838233 >> [v2:172.21.15.47:6800/3836903972,v1:172.21.15.47:6801/3836903972] conn(0x7ff518039290 msgr2=0x7ff51803b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:11:53.849 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:53.106+0000 7ff539400700 1 --2- 172.21.15.47:0/613838233 >> [v2:172.21.15.47:6800/3836903972,v1:172.21.15.47:6801/3836903972] conn(0x7ff518039290 0x7ff51803b750 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7ff51c006fd0 tx=0x7ff51c006e40 comp rx=0 tx=0).stop 2024-03-21T10:11:53.849 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:53.106+0000 7ff539400700 1 -- 172.21.15.47:0/613838233 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff534109ba0 msgr2=0x7ff53419d900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:11:53.849 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:53.106+0000 7ff539400700 1 --2- 172.21.15.47:0/613838233 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff534109ba0 0x7ff53419d900 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7ff524003f60 tx=0x7ff524004670 comp rx=0 tx=0).stop 2024-03-21T10:11:53.849 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:53.106+0000 7ff539400700 1 -- 172.21.15.47:0/613838233 shutdown_connections 2024-03-21T10:11:53.850 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:53.106+0000 7ff539400700 1 --2- 172.21.15.47:0/613838233 >> [v2:172.21.15.47:6800/3836903972,v1:172.21.15.47:6801/3836903972] conn(0x7ff518039290 0x7ff51803b750 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:11:53.850 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:53.107+0000 7ff539400700 1 --2- 172.21.15.47:0/613838233 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff534109ba0 0x7ff53419d900 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:11:53.850 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:53.107+0000 7ff539400700 1 -- 172.21.15.47:0/613838233 >> 172.21.15.47:0/613838233 conn(0x7ff5341054a0 msgr2=0x7ff534106e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:11:53.850 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:53.107+0000 7ff539400700 1 -- 172.21.15.47:0/613838233 shutdown_connections 2024-03-21T10:11:53.850 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:53.107+0000 7ff539400700 1 -- 172.21.15.47:0/613838233 wait complete. 2024-03-21T10:11:55.307 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout value unchanged 2024-03-21T10:11:55.307 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:54.399+0000 7f5019200700 1 Processor -- start 2024-03-21T10:11:55.307 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:54.399+0000 7f5019200700 1 -- start start 2024-03-21T10:11:55.307 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:54.399+0000 7f5019200700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f501410e040 0x7f501410e460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:11:55.307 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:54.399+0000 7f5019200700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f501410ea30 con 0x7f501410e040 2024-03-21T10:11:55.309 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:54.400+0000 7f5013400700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f501410e040 0x7f501410e460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:11:55.309 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:54.400+0000 7f5013400700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f501410e040 0x7f501410e460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:40696/0 (socket says 172.21.15.47:40696) 2024-03-21T10:11:55.309 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:54.400+0000 7f5013400700 1 -- 172.21.15.47:0/2933542112 learned_addr learned my addr 172.21.15.47:0/2933542112 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:11:55.309 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:54.400+0000 7f5013400700 1 -- 172.21.15.47:0/2933542112 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f501410eec0 con 0x7f501410e040 2024-03-21T10:11:55.309 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:54.401+0000 7f5013400700 1 --2- 172.21.15.47:0/2933542112 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f501410e040 0x7f501410e460 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f50040098a0 tx=0x7f5004009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=64a24b75a3889fef server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:11:55.309 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:54.401+0000 7f5012000700 1 -- 172.21.15.47:0/2933542112 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f5004004160 con 0x7f501410e040 2024-03-21T10:11:55.309 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:54.401+0000 7f5012000700 1 -- 172.21.15.47:0/2933542112 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) v1 ==== 1071+0+0 (secure 0 0 0) 0x7f5004004300 con 0x7f501410e040 2024-03-21T10:11:55.310 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:54.401+0000 7f5012000700 1 -- 172.21.15.47:0/2933542112 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f5004011a80 con 0x7f501410e040 2024-03-21T10:11:55.310 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:54.401+0000 7f5019200700 1 -- 172.21.15.47:0/2933542112 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f501410e040 msgr2=0x7f501410e460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:11:55.310 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:54.401+0000 7f5019200700 1 --2- 172.21.15.47:0/2933542112 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f501410e040 0x7f501410e460 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f50040098a0 tx=0x7f5004009bb0 comp rx=0 tx=0).stop 2024-03-21T10:11:55.310 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:54.402+0000 7f5019200700 1 -- 172.21.15.47:0/2933542112 shutdown_connections 2024-03-21T10:11:55.310 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:54.402+0000 7f5019200700 1 --2- 172.21.15.47:0/2933542112 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f501410e040 0x7f501410e460 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:11:55.310 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:54.402+0000 7f5019200700 1 -- 172.21.15.47:0/2933542112 >> 172.21.15.47:0/2933542112 conn(0x7f501407ce50 msgr2=0x7f501407f270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:11:55.310 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:54.402+0000 7f5019200700 1 -- 172.21.15.47:0/2933542112 shutdown_connections 2024-03-21T10:11:55.313 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:54.402+0000 7f5019200700 1 -- 172.21.15.47:0/2933542112 wait complete. 2024-03-21T10:11:55.313 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:54.403+0000 7f5019200700 1 Processor -- start 2024-03-21T10:11:55.313 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:54.403+0000 7f5019200700 1 -- start start 2024-03-21T10:11:55.313 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:54.403+0000 7f5019200700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f501410e040 0x7f50141a1bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:11:55.313 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:54.403+0000 7f5019200700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5014112300 con 0x7f501410e040 2024-03-21T10:11:55.313 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:54.404+0000 7f5013400700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f501410e040 0x7f50141a1bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:11:55.313 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:54.404+0000 7f5013400700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f501410e040 0x7f50141a1bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:40706/0 (socket says 172.21.15.47:40706) 2024-03-21T10:11:55.313 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:54.404+0000 7f5013400700 1 -- 172.21.15.47:0/808266500 learned_addr learned my addr 172.21.15.47:0/808266500 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:11:55.313 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:54.404+0000 7f5013400700 1 -- 172.21.15.47:0/808266500 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50141a20f0 con 0x7f501410e040 2024-03-21T10:11:55.313 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:54.404+0000 7f5013400700 1 --2- 172.21.15.47:0/808266500 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f501410e040 0x7f50141a1bb0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f5004011a50 tx=0x7f5004004290 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:11:55.313 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:54.405+0000 7f500be00700 1 -- 172.21.15.47:0/808266500 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f50040039f0 con 0x7f501410e040 2024-03-21T10:11:55.314 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:54.405+0000 7f500be00700 1 -- 172.21.15.47:0/808266500 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) v1 ==== 1071+0+0 (secure 0 0 0) 0x7f5004003b90 con 0x7f501410e040 2024-03-21T10:11:55.314 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:54.405+0000 7f500be00700 1 -- 172.21.15.47:0/808266500 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f500401a600 con 0x7f501410e040 2024-03-21T10:11:55.314 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:54.405+0000 7f5019200700 1 -- 172.21.15.47:0/808266500 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f50141a2380 con 0x7f501410e040 2024-03-21T10:11:55.314 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:54.405+0000 7f5019200700 1 -- 172.21.15.47:0/808266500 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f50141a5070 con 0x7f501410e040 2024-03-21T10:11:55.314 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:54.406+0000 7f5019200700 1 -- 172.21.15.47:0/808266500 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f50140637c0 con 0x7f501410e040 2024-03-21T10:11:55.314 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:54.406+0000 7f500be00700 1 -- 172.21.15.47:0/808266500 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 8) v1 ==== 46294+0+0 (secure 0 0 0) 0x7f5004024a40 con 0x7f501410e040 2024-03-21T10:11:55.314 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:54.406+0000 7f500be00700 1 --2- 172.21.15.47:0/808266500 >> [v2:172.21.15.47:6800/3836903972,v1:172.21.15.47:6801/3836903972] conn(0x7f4ff8039290 0x7f4ff803b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:11:55.314 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:54.407+0000 7f500be00700 1 -- 172.21.15.47:0/808266500 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f500404d650 con 0x7f501410e040 2024-03-21T10:11:55.314 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:54.407+0000 7f5012a00700 1 --2- 172.21.15.47:0/808266500 >> [v2:172.21.15.47:6800/3836903972,v1:172.21.15.47:6801/3836903972] conn(0x7f4ff8039290 0x7f4ff803b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:11:55.314 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:54.408+0000 7f5012a00700 1 --2- 172.21.15.47:0/808266500 >> [v2:172.21.15.47:6800/3836903972,v1:172.21.15.47:6801/3836903972] conn(0x7f4ff8039290 0x7f4ff803b750 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f5000006fd0 tx=0x7f5000006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:11:55.314 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:54.413+0000 7f500be00700 1 -- 172.21.15.47:0/808266500 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f500404cfa0 con 0x7f501410e040 2024-03-21T10:11:55.314 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:54.639+0000 7f5019200700 1 -- 172.21.15.47:0/808266500 --> [v2:172.21.15.47:6800/3836903972,v1:172.21.15.47:6801/3836903972] -- mgr_command(tid 0: {"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}) v1 -- 0x7f501407df80 con 0x7f4ff8039290 2024-03-21T10:11:55.314 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:54.640+0000 7f500be00700 1 -- 172.21.15.47:0/808266500 <== mgr.14120 v2:172.21.15.47:6800/3836903972 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+16 (secure 0 0 0) 0x7f501407df80 con 0x7f4ff8039290 2024-03-21T10:11:55.314 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:54.642+0000 7f5019200700 1 -- 172.21.15.47:0/808266500 >> [v2:172.21.15.47:6800/3836903972,v1:172.21.15.47:6801/3836903972] conn(0x7f4ff8039290 msgr2=0x7f4ff803b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:11:55.315 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:54.642+0000 7f5019200700 1 --2- 172.21.15.47:0/808266500 >> [v2:172.21.15.47:6800/3836903972,v1:172.21.15.47:6801/3836903972] conn(0x7f4ff8039290 0x7f4ff803b750 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f5000006fd0 tx=0x7f5000006e40 comp rx=0 tx=0).stop 2024-03-21T10:11:55.316 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:54.642+0000 7f5019200700 1 -- 172.21.15.47:0/808266500 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f501410e040 msgr2=0x7f50141a1bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:11:55.316 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:54.642+0000 7f5019200700 1 --2- 172.21.15.47:0/808266500 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f501410e040 0x7f50141a1bb0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f5004011a50 tx=0x7f5004004290 comp rx=0 tx=0).stop 2024-03-21T10:11:55.316 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:54.643+0000 7f5019200700 1 -- 172.21.15.47:0/808266500 shutdown_connections 2024-03-21T10:11:55.316 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:54.643+0000 7f5019200700 1 --2- 172.21.15.47:0/808266500 >> [v2:172.21.15.47:6800/3836903972,v1:172.21.15.47:6801/3836903972] conn(0x7f4ff8039290 0x7f4ff803b750 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:11:55.316 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:54.643+0000 7f5019200700 1 --2- 172.21.15.47:0/808266500 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f501410e040 0x7f50141a1bb0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:11:55.316 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:54.643+0000 7f5019200700 1 -- 172.21.15.47:0/808266500 >> 172.21.15.47:0/808266500 conn(0x7f501407ce50 msgr2=0x7f501407d870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:11:55.316 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:54.643+0000 7f5019200700 1 -- 172.21.15.47:0/808266500 shutdown_connections 2024-03-21T10:11:55.316 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:54.643+0000 7f5019200700 1 -- 172.21.15.47:0/808266500 wait complete. 2024-03-21T10:11:55.316 INFO:teuthology.orchestra.run.smithi047.stdout:Generating ssh key... 2024-03-21T10:11:55.610 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:11:55 smithi047 ceph-mon[31547]: from='client.14134 -' entity='client.admin' cmd=[{"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:11:56.647 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:11:56 smithi047 ceph-mon[31547]: from='client.14136 -' entity='client.admin' cmd=[{"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:11:56.647 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:11:56 smithi047 ceph-mon[31547]: Generating ssh key... 2024-03-21T10:11:56.647 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:11:56 smithi047 ceph-mon[31547]: from='mgr.14120 172.21.15.47:0/2543639975' entity='mgr.x' 2024-03-21T10:11:56.647 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:11:56 smithi047 ceph-mon[31547]: from='mgr.14120 172.21.15.47:0/2543639975' entity='mgr.x' 2024-03-21T10:11:56.648 INFO:journalctl@ceph.mgr.x.smithi047.stdout:Mar 21 10:11:56 smithi047 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-mgr-x[31773]: Generating public/private rsa key pair. 2024-03-21T10:11:56.648 INFO:journalctl@ceph.mgr.x.smithi047.stdout:Mar 21 10:11:56 smithi047 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-mgr-x[31773]: Your identification has been saved in /tmp/tmpf4wrjdix/key. 2024-03-21T10:11:56.648 INFO:journalctl@ceph.mgr.x.smithi047.stdout:Mar 21 10:11:56 smithi047 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-mgr-x[31773]: Your public key has been saved in /tmp/tmpf4wrjdix/key.pub. 2024-03-21T10:11:56.648 INFO:journalctl@ceph.mgr.x.smithi047.stdout:Mar 21 10:11:56 smithi047 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-mgr-x[31773]: The key fingerprint is: 2024-03-21T10:11:56.648 INFO:journalctl@ceph.mgr.x.smithi047.stdout:Mar 21 10:11:56 smithi047 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-mgr-x[31773]: SHA256:pWNV970bn4M/fuHW6N1qqrh6mMx6TkWo9EvajJ4Crrw ceph-3056d51a-e76b-11ee-95cd-87774f69a715 2024-03-21T10:11:56.648 INFO:journalctl@ceph.mgr.x.smithi047.stdout:Mar 21 10:11:56 smithi047 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-mgr-x[31773]: The key's randomart image is: 2024-03-21T10:11:56.648 INFO:journalctl@ceph.mgr.x.smithi047.stdout:Mar 21 10:11:56 smithi047 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-mgr-x[31773]: +---[RSA 3072]----+ 2024-03-21T10:11:56.648 INFO:journalctl@ceph.mgr.x.smithi047.stdout:Mar 21 10:11:56 smithi047 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-mgr-x[31773]: | . . | 2024-03-21T10:11:56.648 INFO:journalctl@ceph.mgr.x.smithi047.stdout:Mar 21 10:11:56 smithi047 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-mgr-x[31773]: | . . . ..| 2024-03-21T10:11:56.648 INFO:journalctl@ceph.mgr.x.smithi047.stdout:Mar 21 10:11:56 smithi047 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-mgr-x[31773]: | . . . o o| 2024-03-21T10:11:56.648 INFO:journalctl@ceph.mgr.x.smithi047.stdout:Mar 21 10:11:56 smithi047 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-mgr-x[31773]: | . o . + .| 2024-03-21T10:11:56.649 INFO:journalctl@ceph.mgr.x.smithi047.stdout:Mar 21 10:11:56 smithi047 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-mgr-x[31773]: | . o S o | 2024-03-21T10:11:56.649 INFO:journalctl@ceph.mgr.x.smithi047.stdout:Mar 21 10:11:56 smithi047 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-mgr-x[31773]: |. * + . ..=| 2024-03-21T10:11:56.649 INFO:journalctl@ceph.mgr.x.smithi047.stdout:Mar 21 10:11:56 smithi047 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-mgr-x[31773]: |.. oo=o ..=+| 2024-03-21T10:11:56.649 INFO:journalctl@ceph.mgr.x.smithi047.stdout:Mar 21 10:11:56 smithi047 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-mgr-x[31773]: |.... o* .. +==| 2024-03-21T10:11:56.649 INFO:journalctl@ceph.mgr.x.smithi047.stdout:Mar 21 10:11:56 smithi047 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-mgr-x[31773]: |oE..++ooo....+=+=| 2024-03-21T10:11:56.649 INFO:journalctl@ceph.mgr.x.smithi047.stdout:Mar 21 10:11:56 smithi047 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-mgr-x[31773]: +----[SHA256]-----+ 2024-03-21T10:11:56.915 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:55.916+0000 7f2146c00700 1 Processor -- start 2024-03-21T10:11:56.916 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:55.917+0000 7f2146c00700 1 -- start start 2024-03-21T10:11:56.916 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:55.917+0000 7f2146c00700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2140109990 0x7f2140109db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:11:56.916 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:55.917+0000 7f2146c00700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f214010a380 con 0x7f2140109990 2024-03-21T10:11:56.916 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:55.917+0000 7f2145800700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2140109990 0x7f2140109db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:11:56.916 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:55.918+0000 7f2145800700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2140109990 0x7f2140109db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:40718/0 (socket says 172.21.15.47:40718) 2024-03-21T10:11:56.916 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:55.918+0000 7f2145800700 1 -- 172.21.15.47:0/3839692140 learned_addr learned my addr 172.21.15.47:0/3839692140 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:11:56.917 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:55.918+0000 7f2145800700 1 -- 172.21.15.47:0/3839692140 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f214010ab40 con 0x7f2140109990 2024-03-21T10:11:56.917 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:55.918+0000 7f2145800700 1 --2- 172.21.15.47:0/3839692140 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2140109990 0x7f2140109db0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f21340098a0 tx=0x7f2134009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2f8625e3622b124f server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:11:56.917 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:55.919+0000 7f213fe00700 1 -- 172.21.15.47:0/3839692140 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f2134004160 con 0x7f2140109990 2024-03-21T10:11:56.922 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:55.919+0000 7f213fe00700 1 -- 172.21.15.47:0/3839692140 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) v1 ==== 1071+0+0 (secure 0 0 0) 0x7f2134004300 con 0x7f2140109990 2024-03-21T10:11:56.922 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:55.919+0000 7f213fe00700 1 -- 172.21.15.47:0/3839692140 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f2134011a80 con 0x7f2140109990 2024-03-21T10:11:56.922 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:55.920+0000 7f2146c00700 1 -- 172.21.15.47:0/3839692140 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2140109990 msgr2=0x7f2140109db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:11:56.922 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:55.920+0000 7f2146c00700 1 --2- 172.21.15.47:0/3839692140 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2140109990 0x7f2140109db0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f21340098a0 tx=0x7f2134009bb0 comp rx=0 tx=0).stop 2024-03-21T10:11:56.922 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:55.920+0000 7f2146c00700 1 -- 172.21.15.47:0/3839692140 shutdown_connections 2024-03-21T10:11:56.922 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:55.920+0000 7f2146c00700 1 --2- 172.21.15.47:0/3839692140 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2140109990 0x7f2140109db0 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:11:56.922 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:55.920+0000 7f2146c00700 1 -- 172.21.15.47:0/3839692140 >> 172.21.15.47:0/3839692140 conn(0x7f21401054a0 msgr2=0x7f2140107890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:11:56.923 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:55.920+0000 7f2146c00700 1 -- 172.21.15.47:0/3839692140 shutdown_connections 2024-03-21T10:11:56.923 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:55.920+0000 7f2146c00700 1 -- 172.21.15.47:0/3839692140 wait complete. 2024-03-21T10:11:56.923 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:55.921+0000 7f2146c00700 1 Processor -- start 2024-03-21T10:11:56.923 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:55.921+0000 7f2146c00700 1 -- start start 2024-03-21T10:11:56.923 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:55.921+0000 7f2146c00700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2140109990 0x7f214019d7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:11:56.923 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:55.921+0000 7f2146c00700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f214010def0 con 0x7f2140109990 2024-03-21T10:11:56.923 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:55.922+0000 7f2145800700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2140109990 0x7f214019d7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:11:56.923 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:55.922+0000 7f2145800700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2140109990 0x7f214019d7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:40726/0 (socket says 172.21.15.47:40726) 2024-03-21T10:11:56.923 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:55.922+0000 7f2145800700 1 -- 172.21.15.47:0/454513714 learned_addr learned my addr 172.21.15.47:0/454513714 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:11:56.923 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:55.922+0000 7f2145800700 1 -- 172.21.15.47:0/454513714 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f214019dd20 con 0x7f2140109990 2024-03-21T10:11:56.923 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:55.922+0000 7f2145800700 1 --2- 172.21.15.47:0/454513714 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2140109990 0x7f214019d7e0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f2134006af0 tx=0x7f2134011b70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:11:56.923 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:55.923+0000 7f213e000700 1 -- 172.21.15.47:0/454513714 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f2134003930 con 0x7f2140109990 2024-03-21T10:11:56.923 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:55.923+0000 7f213e000700 1 -- 172.21.15.47:0/454513714 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) v1 ==== 1071+0+0 (secure 0 0 0) 0x7f2134003ad0 con 0x7f2140109990 2024-03-21T10:11:56.923 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:55.923+0000 7f2146c00700 1 -- 172.21.15.47:0/454513714 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f214019dfb0 con 0x7f2140109990 2024-03-21T10:11:56.924 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:55.923+0000 7f213e000700 1 -- 172.21.15.47:0/454513714 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f21340195e0 con 0x7f2140109990 2024-03-21T10:11:56.924 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:55.923+0000 7f2146c00700 1 -- 172.21.15.47:0/454513714 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f21401a0ca0 con 0x7f2140109990 2024-03-21T10:11:56.924 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:55.924+0000 7f213e000700 1 -- 172.21.15.47:0/454513714 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 8) v1 ==== 46294+0+0 (secure 0 0 0) 0x7f2134019890 con 0x7f2140109990 2024-03-21T10:11:56.924 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:55.924+0000 7f2146c00700 1 -- 172.21.15.47:0/454513714 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f21400637c0 con 0x7f2140109990 2024-03-21T10:11:56.924 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:55.925+0000 7f213e000700 1 --2- 172.21.15.47:0/454513714 >> [v2:172.21.15.47:6800/3836903972,v1:172.21.15.47:6801/3836903972] conn(0x7f2128039240 0x7f212803b700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:11:56.924 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:55.925+0000 7f213e000700 1 -- 172.21.15.47:0/454513714 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f213404d720 con 0x7f2140109990 2024-03-21T10:11:56.924 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:55.925+0000 7f2144e00700 1 --2- 172.21.15.47:0/454513714 >> [v2:172.21.15.47:6800/3836903972,v1:172.21.15.47:6801/3836903972] conn(0x7f2128039240 0x7f212803b700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:11:56.924 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:55.925+0000 7f2144e00700 1 --2- 172.21.15.47:0/454513714 >> [v2:172.21.15.47:6800/3836903972,v1:172.21.15.47:6801/3836903972] conn(0x7f2128039240 0x7f212803b700 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f212c006fd0 tx=0x7f212c006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:11:56.924 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:55.930+0000 7f213e000700 1 -- 172.21.15.47:0/454513714 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f2134016dd0 con 0x7f2140109990 2024-03-21T10:11:56.924 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:56.159+0000 7f2146c00700 1 -- 172.21.15.47:0/454513714 --> [v2:172.21.15.47:6800/3836903972,v1:172.21.15.47:6801/3836903972] -- mgr_command(tid 0: {"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}) v1 -- 0x7f21401075b0 con 0x7f2128039240 2024-03-21T10:11:56.924 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:56.339+0000 7f213e000700 1 -- 172.21.15.47:0/454513714 <== mgr.14120 v2:172.21.15.47:6800/3836903972 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f21401075b0 con 0x7f2128039240 2024-03-21T10:11:56.924 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:56.343+0000 7f2146c00700 1 -- 172.21.15.47:0/454513714 >> [v2:172.21.15.47:6800/3836903972,v1:172.21.15.47:6801/3836903972] conn(0x7f2128039240 msgr2=0x7f212803b700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:11:56.924 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:56.343+0000 7f2146c00700 1 --2- 172.21.15.47:0/454513714 >> [v2:172.21.15.47:6800/3836903972,v1:172.21.15.47:6801/3836903972] conn(0x7f2128039240 0x7f212803b700 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f212c006fd0 tx=0x7f212c006e40 comp rx=0 tx=0).stop 2024-03-21T10:11:56.924 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:56.343+0000 7f2146c00700 1 -- 172.21.15.47:0/454513714 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2140109990 msgr2=0x7f214019d7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:11:56.925 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:56.343+0000 7f2146c00700 1 --2- 172.21.15.47:0/454513714 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2140109990 0x7f214019d7e0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f2134006af0 tx=0x7f2134011b70 comp rx=0 tx=0).stop 2024-03-21T10:11:56.925 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:56.343+0000 7f2146c00700 1 -- 172.21.15.47:0/454513714 shutdown_connections 2024-03-21T10:11:56.925 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:56.343+0000 7f2146c00700 1 --2- 172.21.15.47:0/454513714 >> [v2:172.21.15.47:6800/3836903972,v1:172.21.15.47:6801/3836903972] conn(0x7f2128039240 0x7f212803b700 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:11:56.925 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:56.343+0000 7f2146c00700 1 --2- 172.21.15.47:0/454513714 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2140109990 0x7f214019d7e0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:11:56.925 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:56.343+0000 7f2146c00700 1 -- 172.21.15.47:0/454513714 >> 172.21.15.47:0/454513714 conn(0x7f21401054a0 msgr2=0x7f2140106ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:11:56.925 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:56.343+0000 7f2146c00700 1 -- 172.21.15.47:0/454513714 shutdown_connections 2024-03-21T10:11:56.925 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:56.343+0000 7f2146c00700 1 -- 172.21.15.47:0/454513714 wait complete. 2024-03-21T10:11:58.455 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:11:58 smithi047 ceph-mon[31547]: from='client.14138 -' entity='client.admin' cmd=[{"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:11:58.476 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDP4DxDtnGzZr7AHeST1cVeSRy4amZI9R0nPxcpwwJ+yi161x2vaXk0bFVj4r4gDYkIyagkDCnVYuW2fu1CMxsTDvvJ/yzCnoeo9xDQ9+7A9GVD2USTEpXSetKbBuvipAAY+zdMGmgnayGjkv1lHkeSrK0NEkgbmfpkVCoh8LWtj95aha1g4MNJxFOW8U0x2+ADjbCCJ+6AM5+SZVr9yEAc8WtrUMHyU/OZCDbbdNn6BzFkPg78c96pLvBuNHhJ1Ld7MW9CQ6SkdQDFe6GnPezGKF8C+FjJkMiv7rkfm95E4Hp6W5fnGw8kkeFd98BdbyN1mYaGwyH3Nuwn5yhN2t4VBwWmxJo4WUyRVIiq8h+Mq2CVPRLFfZk/F/EqeZYaKZho+u/ycMmArdO/15+9sZEMrG0ltkF6t0GwiBd/gm8jRZ3cbJvYwzEaRpUf0SCxqvEovS62KZTP7xhSxCdkf+T0NbTXyEE8Nh/qy1d5qVg8SwvNTZI6plxxjp/VT/Ek2iU= ceph-3056d51a-e76b-11ee-95cd-87774f69a715 2024-03-21T10:11:58.476 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:57.525+0000 7fe3fc000700 1 Processor -- start 2024-03-21T10:11:58.477 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:57.526+0000 7fe3fc000700 1 -- start start 2024-03-21T10:11:58.477 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:57.526+0000 7fe3fc000700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe3f4109b80 0x7fe3f4109fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:11:58.477 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:57.526+0000 7fe3fc000700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3f410a570 con 0x7fe3f4109b80 2024-03-21T10:11:58.477 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:57.527+0000 7fe3f9600700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe3f4109b80 0x7fe3f4109fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:11:58.477 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:57.527+0000 7fe3f9600700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe3f4109b80 0x7fe3f4109fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:40738/0 (socket says 172.21.15.47:40738) 2024-03-21T10:11:58.477 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:57.527+0000 7fe3f9600700 1 -- 172.21.15.47:0/1490714269 learned_addr learned my addr 172.21.15.47:0/1490714269 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:11:58.477 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:57.527+0000 7fe3f9600700 1 -- 172.21.15.47:0/1490714269 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3f410ace0 con 0x7fe3f4109b80 2024-03-21T10:11:58.477 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:57.527+0000 7fe3f9600700 1 --2- 172.21.15.47:0/1490714269 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe3f4109b80 0x7fe3f4109fa0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fe3e80098a0 tx=0x7fe3e8009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b779a494c8649b76 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:11:58.477 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:57.528+0000 7fe3f3e00700 1 -- 172.21.15.47:0/1490714269 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fe3e8004160 con 0x7fe3f4109b80 2024-03-21T10:11:58.477 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:57.528+0000 7fe3f3e00700 1 -- 172.21.15.47:0/1490714269 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) v1 ==== 1071+0+0 (secure 0 0 0) 0x7fe3e8004300 con 0x7fe3f4109b80 2024-03-21T10:11:58.477 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:57.528+0000 7fe3f3e00700 1 -- 172.21.15.47:0/1490714269 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fe3e8011a80 con 0x7fe3f4109b80 2024-03-21T10:11:58.477 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:57.528+0000 7fe3fc000700 1 -- 172.21.15.47:0/1490714269 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe3f4109b80 msgr2=0x7fe3f4109fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:11:58.478 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:57.528+0000 7fe3fc000700 1 --2- 172.21.15.47:0/1490714269 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe3f4109b80 0x7fe3f4109fa0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fe3e80098a0 tx=0x7fe3e8009bb0 comp rx=0 tx=0).stop 2024-03-21T10:11:58.478 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:57.529+0000 7fe3fc000700 1 -- 172.21.15.47:0/1490714269 shutdown_connections 2024-03-21T10:11:58.478 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:57.529+0000 7fe3fc000700 1 --2- 172.21.15.47:0/1490714269 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe3f4109b80 0x7fe3f4109fa0 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:11:58.478 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:57.529+0000 7fe3fc000700 1 -- 172.21.15.47:0/1490714269 >> 172.21.15.47:0/1490714269 conn(0x7fe3f41054a0 msgr2=0x7fe3f4107890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:11:58.478 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:57.529+0000 7fe3fc000700 1 -- 172.21.15.47:0/1490714269 shutdown_connections 2024-03-21T10:11:58.478 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:57.529+0000 7fe3fc000700 1 -- 172.21.15.47:0/1490714269 wait complete. 2024-03-21T10:11:58.478 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:57.530+0000 7fe3fc000700 1 Processor -- start 2024-03-21T10:11:58.478 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:57.530+0000 7fe3fc000700 1 -- start start 2024-03-21T10:11:58.478 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:57.530+0000 7fe3fc000700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe3f4109b80 0x7fe3f419da00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:11:58.478 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:57.530+0000 7fe3fc000700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3f410e090 con 0x7fe3f4109b80 2024-03-21T10:11:58.478 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:57.531+0000 7fe3f9600700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe3f4109b80 0x7fe3f419da00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:11:58.478 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:57.531+0000 7fe3f9600700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe3f4109b80 0x7fe3f419da00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:40746/0 (socket says 172.21.15.47:40746) 2024-03-21T10:11:58.478 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:57.531+0000 7fe3f9600700 1 -- 172.21.15.47:0/37082302 learned_addr learned my addr 172.21.15.47:0/37082302 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:11:58.479 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:57.531+0000 7fe3f9600700 1 -- 172.21.15.47:0/37082302 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3f419df40 con 0x7fe3f4109b80 2024-03-21T10:11:58.479 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:57.532+0000 7fe3f9600700 1 --2- 172.21.15.47:0/37082302 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe3f4109b80 0x7fe3f419da00 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fe3e8000c00 tx=0x7fe3e8004290 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:11:58.479 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:57.532+0000 7fe3f2000700 1 -- 172.21.15.47:0/37082302 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fe3e80039c0 con 0x7fe3f4109b80 2024-03-21T10:11:58.479 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:57.532+0000 7fe3f2000700 1 -- 172.21.15.47:0/37082302 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) v1 ==== 1071+0+0 (secure 0 0 0) 0x7fe3e8003b60 con 0x7fe3f4109b80 2024-03-21T10:11:58.479 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:57.532+0000 7fe3f2000700 1 -- 172.21.15.47:0/37082302 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fe3e801a6f0 con 0x7fe3f4109b80 2024-03-21T10:11:58.479 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:57.532+0000 7fe3fc000700 1 -- 172.21.15.47:0/37082302 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe3f419e1d0 con 0x7fe3f4109b80 2024-03-21T10:11:58.479 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:57.532+0000 7fe3fc000700 1 -- 172.21.15.47:0/37082302 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe3f41a0ec0 con 0x7fe3f4109b80 2024-03-21T10:11:58.479 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:57.533+0000 7fe3f2000700 1 -- 172.21.15.47:0/37082302 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 8) v1 ==== 46294+0+0 (secure 0 0 0) 0x7fe3e8024a40 con 0x7fe3f4109b80 2024-03-21T10:11:58.479 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:57.533+0000 7fe3fc000700 1 -- 172.21.15.47:0/37082302 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe3f40637c0 con 0x7fe3f4109b80 2024-03-21T10:11:58.479 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:57.534+0000 7fe3f2000700 1 --2- 172.21.15.47:0/37082302 >> [v2:172.21.15.47:6800/3836903972,v1:172.21.15.47:6801/3836903972] conn(0x7fe3dc039290 0x7fe3dc03b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:11:58.479 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:57.534+0000 7fe3f2000700 1 -- 172.21.15.47:0/37082302 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7fe3e804d520 con 0x7fe3f4109b80 2024-03-21T10:11:58.479 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:57.534+0000 7fe3f8c00700 1 --2- 172.21.15.47:0/37082302 >> [v2:172.21.15.47:6800/3836903972,v1:172.21.15.47:6801/3836903972] conn(0x7fe3dc039290 0x7fe3dc03b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:11:58.479 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:57.535+0000 7fe3f8c00700 1 --2- 172.21.15.47:0/37082302 >> [v2:172.21.15.47:6800/3836903972,v1:172.21.15.47:6801/3836903972] conn(0x7fe3dc039290 0x7fe3dc03b750 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fe3e0006fd0 tx=0x7fe3e0006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:11:58.479 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:57.539+0000 7fe3f2000700 1 -- 172.21.15.47:0/37082302 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fe3e800e260 con 0x7fe3f4109b80 2024-03-21T10:11:58.480 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:57.765+0000 7fe3fc000700 1 -- 172.21.15.47:0/37082302 --> [v2:172.21.15.47:6800/3836903972,v1:172.21.15.47:6801/3836903972] -- mgr_command(tid 0: {"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}) v1 -- 0x7fe3f41076b0 con 0x7fe3dc039290 2024-03-21T10:11:58.480 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:57.767+0000 7fe3f2000700 1 -- 172.21.15.47:0/37082302 <== mgr.14120 v2:172.21.15.47:6800/3836903972 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+595 (secure 0 0 0) 0x7fe3f41076b0 con 0x7fe3dc039290 2024-03-21T10:11:58.480 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:57.769+0000 7fe3fc000700 1 -- 172.21.15.47:0/37082302 >> [v2:172.21.15.47:6800/3836903972,v1:172.21.15.47:6801/3836903972] conn(0x7fe3dc039290 msgr2=0x7fe3dc03b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:11:58.480 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:57.769+0000 7fe3fc000700 1 --2- 172.21.15.47:0/37082302 >> [v2:172.21.15.47:6800/3836903972,v1:172.21.15.47:6801/3836903972] conn(0x7fe3dc039290 0x7fe3dc03b750 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fe3e0006fd0 tx=0x7fe3e0006e40 comp rx=0 tx=0).stop 2024-03-21T10:11:58.480 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:57.769+0000 7fe3fc000700 1 -- 172.21.15.47:0/37082302 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe3f4109b80 msgr2=0x7fe3f419da00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:11:58.480 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:57.770+0000 7fe3fc000700 1 --2- 172.21.15.47:0/37082302 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe3f4109b80 0x7fe3f419da00 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fe3e8000c00 tx=0x7fe3e8004290 comp rx=0 tx=0).stop 2024-03-21T10:11:58.480 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:57.770+0000 7fe3fc000700 1 -- 172.21.15.47:0/37082302 shutdown_connections 2024-03-21T10:11:58.480 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:57.770+0000 7fe3fc000700 1 --2- 172.21.15.47:0/37082302 >> [v2:172.21.15.47:6800/3836903972,v1:172.21.15.47:6801/3836903972] conn(0x7fe3dc039290 0x7fe3dc03b750 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:11:58.480 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:57.770+0000 7fe3fc000700 1 --2- 172.21.15.47:0/37082302 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe3f4109b80 0x7fe3f419da00 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:11:58.480 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:57.770+0000 7fe3fc000700 1 -- 172.21.15.47:0/37082302 >> 172.21.15.47:0/37082302 conn(0x7fe3f41054a0 msgr2=0x7fe3f4106fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:11:58.480 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:57.770+0000 7fe3fc000700 1 -- 172.21.15.47:0/37082302 shutdown_connections 2024-03-21T10:11:58.480 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:57.770+0000 7fe3fc000700 1 -- 172.21.15.47:0/37082302 wait complete. 2024-03-21T10:11:58.480 INFO:teuthology.orchestra.run.smithi047.stdout:Wrote public SSH key to /home/ubuntu/cephtest/ceph.pub 2024-03-21T10:11:58.480 INFO:teuthology.orchestra.run.smithi047.stdout:Adding key to root@localhost authorized_keys... 2024-03-21T10:11:58.481 INFO:teuthology.orchestra.run.smithi047.stdout:Adding host smithi047... 2024-03-21T10:12:00.463 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:12:00 smithi047 ceph-mon[31547]: from='client.14140 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi047", "addr": "172.21.15.47", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:12:01.687 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:12:01 smithi047 ceph-mon[31547]: Deploying cephadm binary to smithi047 2024-03-21T10:12:07.214 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout Added host 'smithi047' with addr '172.21.15.47' 2024-03-21T10:12:07.214 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:59.146+0000 7fa423200700 1 Processor -- start 2024-03-21T10:12:07.214 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:59.146+0000 7fa423200700 1 -- start start 2024-03-21T10:12:07.214 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:59.147+0000 7fa423200700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa424108260 0x7fa42410a650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:12:07.214 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:59.147+0000 7fa423200700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa42407add0 con 0x7fa424108260 2024-03-21T10:12:07.214 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:59.147+0000 7fa421e00700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa424108260 0x7fa42410a650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:12:07.214 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:59.148+0000 7fa421e00700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa424108260 0x7fa42410a650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:40758/0 (socket says 172.21.15.47:40758) 2024-03-21T10:12:07.214 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:59.148+0000 7fa421e00700 1 -- 172.21.15.47:0/2841404210 learned_addr learned my addr 172.21.15.47:0/2841404210 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:12:07.215 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:59.148+0000 7fa421e00700 1 -- 172.21.15.47:0/2841404210 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa42410ab90 con 0x7fa424108260 2024-03-21T10:12:07.215 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:59.148+0000 7fa421e00700 1 --2- 172.21.15.47:0/2841404210 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa424108260 0x7fa42410a650 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fa4140098a0 tx=0x7fa414009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=698f796cee9d9508 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:12:07.220 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:59.148+0000 7fa420a00700 1 -- 172.21.15.47:0/2841404210 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa414004160 con 0x7fa424108260 2024-03-21T10:12:07.220 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:59.149+0000 7fa420a00700 1 -- 172.21.15.47:0/2841404210 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) v1 ==== 1071+0+0 (secure 0 0 0) 0x7fa414004300 con 0x7fa424108260 2024-03-21T10:12:07.220 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:59.149+0000 7fa420a00700 1 -- 172.21.15.47:0/2841404210 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa414011a80 con 0x7fa424108260 2024-03-21T10:12:07.220 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:59.149+0000 7fa423200700 1 -- 172.21.15.47:0/2841404210 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa424108260 msgr2=0x7fa42410a650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:12:07.220 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:59.149+0000 7fa423200700 1 --2- 172.21.15.47:0/2841404210 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa424108260 0x7fa42410a650 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fa4140098a0 tx=0x7fa414009bb0 comp rx=0 tx=0).stop 2024-03-21T10:12:07.220 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:59.149+0000 7fa423200700 1 -- 172.21.15.47:0/2841404210 shutdown_connections 2024-03-21T10:12:07.220 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:59.149+0000 7fa423200700 1 --2- 172.21.15.47:0/2841404210 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa424108260 0x7fa42410a650 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:12:07.220 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:59.149+0000 7fa423200700 1 -- 172.21.15.47:0/2841404210 >> 172.21.15.47:0/2841404210 conn(0x7fa424101c50 msgr2=0x7fa424104070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:12:07.220 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:59.150+0000 7fa423200700 1 -- 172.21.15.47:0/2841404210 shutdown_connections 2024-03-21T10:12:07.220 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:59.150+0000 7fa423200700 1 -- 172.21.15.47:0/2841404210 wait complete. 2024-03-21T10:12:07.220 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:59.151+0000 7fa423200700 1 Processor -- start 2024-03-21T10:12:07.220 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:59.151+0000 7fa423200700 1 -- start start 2024-03-21T10:12:07.221 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:59.151+0000 7fa423200700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa424108260 0x7fa42419d810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:12:07.221 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:59.151+0000 7fa423200700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa42410baa0 con 0x7fa424108260 2024-03-21T10:12:07.221 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:59.151+0000 7fa421e00700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa424108260 0x7fa42419d810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:12:07.221 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:59.151+0000 7fa421e00700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa424108260 0x7fa42419d810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:40768/0 (socket says 172.21.15.47:40768) 2024-03-21T10:12:07.221 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:59.151+0000 7fa421e00700 1 -- 172.21.15.47:0/2158004918 learned_addr learned my addr 172.21.15.47:0/2158004918 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:12:07.221 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:59.152+0000 7fa421e00700 1 -- 172.21.15.47:0/2158004918 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa42419dd50 con 0x7fa424108260 2024-03-21T10:12:07.221 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:59.152+0000 7fa421e00700 1 --2- 172.21.15.47:0/2158004918 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa424108260 0x7fa42419d810 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fa414011a50 tx=0x7fa414004290 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:12:07.221 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:59.152+0000 7fa41aa00700 1 -- 172.21.15.47:0/2158004918 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa4140039f0 con 0x7fa424108260 2024-03-21T10:12:07.221 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:59.152+0000 7fa41aa00700 1 -- 172.21.15.47:0/2158004918 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) v1 ==== 1071+0+0 (secure 0 0 0) 0x7fa414003b90 con 0x7fa424108260 2024-03-21T10:12:07.221 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:59.152+0000 7fa41aa00700 1 -- 172.21.15.47:0/2158004918 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa41401a600 con 0x7fa424108260 2024-03-21T10:12:07.221 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:59.152+0000 7fa423200700 1 -- 172.21.15.47:0/2158004918 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa42419dfe0 con 0x7fa424108260 2024-03-21T10:12:07.221 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:59.153+0000 7fa423200700 1 -- 172.21.15.47:0/2158004918 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa4241a0cd0 con 0x7fa424108260 2024-03-21T10:12:07.222 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:59.154+0000 7fa41aa00700 1 -- 172.21.15.47:0/2158004918 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 8) v1 ==== 46294+0+0 (secure 0 0 0) 0x7fa414024a40 con 0x7fa424108260 2024-03-21T10:12:07.222 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:59.154+0000 7fa423200700 1 -- 172.21.15.47:0/2158004918 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa4240637c0 con 0x7fa424108260 2024-03-21T10:12:07.222 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:59.154+0000 7fa41aa00700 1 --2- 172.21.15.47:0/2158004918 >> [v2:172.21.15.47:6800/3836903972,v1:172.21.15.47:6801/3836903972] conn(0x7fa408039290 0x7fa40803b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:12:07.222 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:59.154+0000 7fa41aa00700 1 -- 172.21.15.47:0/2158004918 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7fa41404d690 con 0x7fa424108260 2024-03-21T10:12:07.222 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:59.155+0000 7fa421400700 1 --2- 172.21.15.47:0/2158004918 >> [v2:172.21.15.47:6800/3836903972,v1:172.21.15.47:6801/3836903972] conn(0x7fa408039290 0x7fa40803b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:12:07.222 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:59.155+0000 7fa421400700 1 --2- 172.21.15.47:0/2158004918 >> [v2:172.21.15.47:6800/3836903972,v1:172.21.15.47:6801/3836903972] conn(0x7fa408039290 0x7fa40803b750 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fa410006fd0 tx=0x7fa410006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:12:07.222 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:59.160+0000 7fa41aa00700 1 -- 172.21.15.47:0/2158004918 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fa41404cfe0 con 0x7fa424108260 2024-03-21T10:12:07.222 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:11:59.392+0000 7fa423200700 1 -- 172.21.15.47:0/2158004918 --> [v2:172.21.15.47:6800/3836903972,v1:172.21.15.47:6801/3836903972] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi047", "addr": "172.21.15.47", "target": ["mon-mgr", ""]}) v1 -- 0x7fa424102d50 con 0x7fa408039290 2024-03-21T10:12:07.222 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:12:06.382+0000 7fa41aa00700 1 -- 172.21.15.47:0/2158004918 <== mgr.14120 v2:172.21.15.47:6800/3836903972 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+48 (secure 0 0 0) 0x7fa424102d50 con 0x7fa408039290 2024-03-21T10:12:07.222 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:12:06.386+0000 7fa423200700 1 -- 172.21.15.47:0/2158004918 >> [v2:172.21.15.47:6800/3836903972,v1:172.21.15.47:6801/3836903972] conn(0x7fa408039290 msgr2=0x7fa40803b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:12:07.222 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:12:06.386+0000 7fa423200700 1 --2- 172.21.15.47:0/2158004918 >> [v2:172.21.15.47:6800/3836903972,v1:172.21.15.47:6801/3836903972] conn(0x7fa408039290 0x7fa40803b750 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fa410006fd0 tx=0x7fa410006e40 comp rx=0 tx=0).stop 2024-03-21T10:12:07.222 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:12:06.386+0000 7fa423200700 1 -- 172.21.15.47:0/2158004918 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa424108260 msgr2=0x7fa42419d810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:12:07.222 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:12:06.386+0000 7fa423200700 1 --2- 172.21.15.47:0/2158004918 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa424108260 0x7fa42419d810 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fa414011a50 tx=0x7fa414004290 comp rx=0 tx=0).stop 2024-03-21T10:12:07.223 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:12:06.386+0000 7fa423200700 1 -- 172.21.15.47:0/2158004918 shutdown_connections 2024-03-21T10:12:07.223 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:12:06.386+0000 7fa423200700 1 --2- 172.21.15.47:0/2158004918 >> [v2:172.21.15.47:6800/3836903972,v1:172.21.15.47:6801/3836903972] conn(0x7fa408039290 0x7fa40803b750 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:12:07.223 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:12:06.386+0000 7fa423200700 1 --2- 172.21.15.47:0/2158004918 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa424108260 0x7fa42419d810 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:12:07.223 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:12:06.386+0000 7fa423200700 1 -- 172.21.15.47:0/2158004918 >> 172.21.15.47:0/2158004918 conn(0x7fa424101c50 msgr2=0x7fa424102640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:12:07.223 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:12:06.387+0000 7fa423200700 1 -- 172.21.15.47:0/2158004918 shutdown_connections 2024-03-21T10:12:07.223 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:12:06.387+0000 7fa423200700 1 -- 172.21.15.47:0/2158004918 wait complete. 2024-03-21T10:12:07.223 INFO:teuthology.orchestra.run.smithi047.stdout:Deploying unmanaged mon service... 2024-03-21T10:12:07.464 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:12:07 smithi047 ceph-mon[31547]: from='mgr.14120 172.21.15.47:0/2543639975' entity='mgr.x' 2024-03-21T10:12:07.464 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:12:07 smithi047 ceph-mon[31547]: Added host smithi047 2024-03-21T10:12:07.464 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:12:07 smithi047 ceph-mon[31547]: from='mgr.14120 172.21.15.47:0/2543639975' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T10:12:08.808 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout Scheduled mon update... 2024-03-21T10:12:08.808 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:12:07.838+0000 7f1da4400700 1 Processor -- start 2024-03-21T10:12:08.808 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:12:07.838+0000 7f1da4400700 1 -- start start 2024-03-21T10:12:08.808 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:12:07.838+0000 7f1da4400700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1d9c109ae0 0x7f1d9c109f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:12:08.808 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:12:07.838+0000 7f1da4400700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d9c10a4d0 con 0x7f1d9c109ae0 2024-03-21T10:12:08.809 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:12:07.839+0000 7f1da1a00700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1d9c109ae0 0x7f1d9c109f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:12:08.809 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:12:07.839+0000 7f1da1a00700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1d9c109ae0 0x7f1d9c109f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:60484/0 (socket says 172.21.15.47:60484) 2024-03-21T10:12:08.809 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:12:07.839+0000 7f1da1a00700 1 -- 172.21.15.47:0/3910793934 learned_addr learned my addr 172.21.15.47:0/3910793934 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:12:08.809 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:12:07.839+0000 7f1da1a00700 1 -- 172.21.15.47:0/3910793934 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d9c10ac40 con 0x7f1d9c109ae0 2024-03-21T10:12:08.809 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:12:07.840+0000 7f1da1a00700 1 --2- 172.21.15.47:0/3910793934 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1d9c109ae0 0x7f1d9c109f00 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f1d900098a0 tx=0x7f1d90009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d0a6dbed32c94005 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:12:08.809 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:12:07.840+0000 7f1d9be00700 1 -- 172.21.15.47:0/3910793934 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f1d90004160 con 0x7f1d9c109ae0 2024-03-21T10:12:08.809 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:12:07.840+0000 7f1d9be00700 1 -- 172.21.15.47:0/3910793934 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) v1 ==== 1071+0+0 (secure 0 0 0) 0x7f1d90004300 con 0x7f1d9c109ae0 2024-03-21T10:12:08.809 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:12:07.840+0000 7f1d9be00700 1 -- 172.21.15.47:0/3910793934 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f1d90011a80 con 0x7f1d9c109ae0 2024-03-21T10:12:08.809 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:12:07.841+0000 7f1da4400700 1 -- 172.21.15.47:0/3910793934 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1d9c109ae0 msgr2=0x7f1d9c109f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:12:08.809 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:12:07.841+0000 7f1da4400700 1 --2- 172.21.15.47:0/3910793934 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1d9c109ae0 0x7f1d9c109f00 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f1d900098a0 tx=0x7f1d90009bb0 comp rx=0 tx=0).stop 2024-03-21T10:12:08.809 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:12:07.841+0000 7f1da4400700 1 -- 172.21.15.47:0/3910793934 shutdown_connections 2024-03-21T10:12:08.809 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:12:07.841+0000 7f1da4400700 1 --2- 172.21.15.47:0/3910793934 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1d9c109ae0 0x7f1d9c109f00 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:12:08.809 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:12:07.841+0000 7f1da4400700 1 -- 172.21.15.47:0/3910793934 >> 172.21.15.47:0/3910793934 conn(0x7f1d9c1054a0 msgr2=0x7f1d9c107890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:12:08.809 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:12:07.841+0000 7f1da4400700 1 -- 172.21.15.47:0/3910793934 shutdown_connections 2024-03-21T10:12:08.813 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:12:07.841+0000 7f1da4400700 1 -- 172.21.15.47:0/3910793934 wait complete. 2024-03-21T10:12:08.813 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:12:07.842+0000 7f1da4400700 1 Processor -- start 2024-03-21T10:12:08.813 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:12:07.843+0000 7f1da4400700 1 -- start start 2024-03-21T10:12:08.813 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:12:07.843+0000 7f1da4400700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1d9c109ae0 0x7f1d9c19d9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:12:08.814 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:12:07.843+0000 7f1da4400700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d9c10dff0 con 0x7f1d9c109ae0 2024-03-21T10:12:08.814 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:12:07.843+0000 7f1da1a00700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1d9c109ae0 0x7f1d9c19d9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:12:08.814 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:12:07.843+0000 7f1da1a00700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1d9c109ae0 0x7f1d9c19d9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:60490/0 (socket says 172.21.15.47:60490) 2024-03-21T10:12:08.814 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:12:07.843+0000 7f1da1a00700 1 -- 172.21.15.47:0/2586368488 learned_addr learned my addr 172.21.15.47:0/2586368488 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:12:08.814 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:12:07.844+0000 7f1da1a00700 1 -- 172.21.15.47:0/2586368488 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d9c19df30 con 0x7f1d9c109ae0 2024-03-21T10:12:08.814 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:12:07.844+0000 7f1da1a00700 1 --2- 172.21.15.47:0/2586368488 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1d9c109ae0 0x7f1d9c19d9f0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f1d90000c00 tx=0x7f1d90004290 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:12:08.814 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:12:07.844+0000 7f1d9a000700 1 -- 172.21.15.47:0/2586368488 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f1d900039c0 con 0x7f1d9c109ae0 2024-03-21T10:12:08.814 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:12:07.844+0000 7f1d9a000700 1 -- 172.21.15.47:0/2586368488 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) v1 ==== 1071+0+0 (secure 0 0 0) 0x7f1d90003b60 con 0x7f1d9c109ae0 2024-03-21T10:12:08.814 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:12:07.844+0000 7f1da4400700 1 -- 172.21.15.47:0/2586368488 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1d9c19e1c0 con 0x7f1d9c109ae0 2024-03-21T10:12:08.814 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:12:07.844+0000 7f1d9a000700 1 -- 172.21.15.47:0/2586368488 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f1d9001a6f0 con 0x7f1d9c109ae0 2024-03-21T10:12:08.814 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:12:07.844+0000 7f1da4400700 1 -- 172.21.15.47:0/2586368488 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1d9c1a0eb0 con 0x7f1d9c109ae0 2024-03-21T10:12:08.814 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:12:07.845+0000 7f1d9a000700 1 -- 172.21.15.47:0/2586368488 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 8) v1 ==== 46294+0+0 (secure 0 0 0) 0x7f1d90024a40 con 0x7f1d9c109ae0 2024-03-21T10:12:08.814 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:12:07.846+0000 7f1da4400700 1 -- 172.21.15.47:0/2586368488 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1d9c0637c0 con 0x7f1d9c109ae0 2024-03-21T10:12:08.815 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:12:07.846+0000 7f1d9a000700 1 --2- 172.21.15.47:0/2586368488 >> [v2:172.21.15.47:6800/3836903972,v1:172.21.15.47:6801/3836903972] conn(0x7f1d840391d0 0x7f1d8403b690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:12:08.815 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:12:07.846+0000 7f1d9a000700 1 -- 172.21.15.47:0/2586368488 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f1d9004d460 con 0x7f1d9c109ae0 2024-03-21T10:12:08.815 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:12:07.846+0000 7f1da1000700 1 --2- 172.21.15.47:0/2586368488 >> [v2:172.21.15.47:6800/3836903972,v1:172.21.15.47:6801/3836903972] conn(0x7f1d840391d0 0x7f1d8403b690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:12:08.815 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:12:07.847+0000 7f1da1000700 1 --2- 172.21.15.47:0/2586368488 >> [v2:172.21.15.47:6800/3836903972,v1:172.21.15.47:6801/3836903972] conn(0x7f1d840391d0 0x7f1d8403b690 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f1d88006fd0 tx=0x7f1d88006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:12:08.815 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:12:07.852+0000 7f1d9a000700 1 -- 172.21.15.47:0/2586368488 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f1d9000e260 con 0x7f1d9c109ae0 2024-03-21T10:12:08.815 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:12:08.095+0000 7f1da4400700 1 -- 172.21.15.47:0/2586368488 --> [v2:172.21.15.47:6800/3836903972,v1:172.21.15.47:6801/3836903972] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}) v1 -- 0x7f1d9c107750 con 0x7f1d840391d0 2024-03-21T10:12:08.815 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:12:08.102+0000 7f1d9a000700 1 -- 172.21.15.47:0/2586368488 <== mgr.14120 v2:172.21.15.47:6800/3836903972 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+24 (secure 0 0 0) 0x7f1d9c107750 con 0x7f1d840391d0 2024-03-21T10:12:08.815 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:12:08.105+0000 7f1da4400700 1 -- 172.21.15.47:0/2586368488 >> [v2:172.21.15.47:6800/3836903972,v1:172.21.15.47:6801/3836903972] conn(0x7f1d840391d0 msgr2=0x7f1d8403b690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:12:08.815 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:12:08.105+0000 7f1da4400700 1 --2- 172.21.15.47:0/2586368488 >> [v2:172.21.15.47:6800/3836903972,v1:172.21.15.47:6801/3836903972] conn(0x7f1d840391d0 0x7f1d8403b690 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f1d88006fd0 tx=0x7f1d88006e40 comp rx=0 tx=0).stop 2024-03-21T10:12:08.815 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:12:08.105+0000 7f1da4400700 1 -- 172.21.15.47:0/2586368488 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1d9c109ae0 msgr2=0x7f1d9c19d9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:12:08.815 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:12:08.105+0000 7f1da4400700 1 --2- 172.21.15.47:0/2586368488 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1d9c109ae0 0x7f1d9c19d9f0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f1d90000c00 tx=0x7f1d90004290 comp rx=0 tx=0).stop 2024-03-21T10:12:08.815 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:12:08.105+0000 7f1da4400700 1 -- 172.21.15.47:0/2586368488 shutdown_connections 2024-03-21T10:12:08.815 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:12:08.105+0000 7f1da4400700 1 --2- 172.21.15.47:0/2586368488 >> [v2:172.21.15.47:6800/3836903972,v1:172.21.15.47:6801/3836903972] conn(0x7f1d840391d0 0x7f1d8403b690 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:12:08.816 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:12:08.105+0000 7f1da4400700 1 --2- 172.21.15.47:0/2586368488 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1d9c109ae0 0x7f1d9c19d9f0 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:12:08.816 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:12:08.105+0000 7f1da4400700 1 -- 172.21.15.47:0/2586368488 >> 172.21.15.47:0/2586368488 conn(0x7f1d9c1054a0 msgr2=0x7f1d9c107040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:12:08.816 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:12:08.106+0000 7f1da4400700 1 -- 172.21.15.47:0/2586368488 shutdown_connections 2024-03-21T10:12:08.816 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:12:08.106+0000 7f1da4400700 1 -- 172.21.15.47:0/2586368488 wait complete. 2024-03-21T10:12:08.816 INFO:teuthology.orchestra.run.smithi047.stdout:Deploying unmanaged mgr service... 2024-03-21T10:12:09.314 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:12:09 smithi047 ceph-mon[31547]: from='client.14142 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:12:09.365 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:12:09 smithi047 ceph-mon[31547]: Saving service mon spec with placement count:5 2024-03-21T10:12:09.365 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:12:09 smithi047 ceph-mon[31547]: from='mgr.14120 172.21.15.47:0/2543639975' entity='mgr.x' 2024-03-21T10:12:11.029 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout Scheduled mgr update... 2024-03-21T10:12:11.029 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:12:09.649+0000 7f2fa6a00700 1 Processor -- start 2024-03-21T10:12:11.029 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:12:09.649+0000 7f2fa6a00700 1 -- start start 2024-03-21T10:12:11.029 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:12:09.649+0000 7f2fa6a00700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2fa0105740 0x7f2fa0105b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:12:11.029 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:12:09.649+0000 7f2fa6a00700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2fa0106130 con 0x7f2fa0105740 2024-03-21T10:12:11.029 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:12:09.649+0000 7f2f9fe00700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2fa0105740 0x7f2fa0105b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:12:11.029 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:12:09.649+0000 7f2f9fe00700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2fa0105740 0x7f2fa0105b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:60494/0 (socket says 172.21.15.47:60494) 2024-03-21T10:12:11.029 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:12:09.649+0000 7f2f9fe00700 1 -- 172.21.15.47:0/1252369865 learned_addr learned my addr 172.21.15.47:0/1252369865 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:12:11.030 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:12:09.650+0000 7f2f9fe00700 1 -- 172.21.15.47:0/1252369865 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2fa0106960 con 0x7f2fa0105740 2024-03-21T10:12:11.030 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:12:09.650+0000 7f2f9fe00700 1 --2- 172.21.15.47:0/1252369865 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2fa0105740 0x7f2fa0105b60 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f2f900098a0 tx=0x7f2f90009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=241ec146adcb9a3a server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:12:11.030 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:12:09.650+0000 7f2f9ea00700 1 -- 172.21.15.47:0/1252369865 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f2f90004160 con 0x7f2fa0105740 2024-03-21T10:12:11.030 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:12:09.650+0000 7f2f9ea00700 1 -- 172.21.15.47:0/1252369865 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) v1 ==== 1071+0+0 (secure 0 0 0) 0x7f2f90004300 con 0x7f2fa0105740 2024-03-21T10:12:11.030 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:12:09.650+0000 7f2f9ea00700 1 -- 172.21.15.47:0/1252369865 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f2f90011a80 con 0x7f2fa0105740 2024-03-21T10:12:11.030 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:12:09.651+0000 7f2fa6a00700 1 -- 172.21.15.47:0/1252369865 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2fa0105740 msgr2=0x7f2fa0105b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:12:11.030 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:12:09.651+0000 7f2fa6a00700 1 --2- 172.21.15.47:0/1252369865 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2fa0105740 0x7f2fa0105b60 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f2f900098a0 tx=0x7f2f90009bb0 comp rx=0 tx=0).stop 2024-03-21T10:12:11.030 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:12:09.651+0000 7f2fa6a00700 1 -- 172.21.15.47:0/1252369865 shutdown_connections 2024-03-21T10:12:11.030 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:12:09.651+0000 7f2fa6a00700 1 --2- 172.21.15.47:0/1252369865 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2fa0105740 0x7f2fa0105b60 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:12:11.030 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:12:09.651+0000 7f2fa6a00700 1 -- 172.21.15.47:0/1252369865 >> 172.21.15.47:0/1252369865 conn(0x7f2fa01011a0 msgr2=0x7f2fa01035c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:12:11.030 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:12:09.651+0000 7f2fa6a00700 1 -- 172.21.15.47:0/1252369865 shutdown_connections 2024-03-21T10:12:11.030 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:12:09.651+0000 7f2fa6a00700 1 -- 172.21.15.47:0/1252369865 wait complete. 2024-03-21T10:12:11.030 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:12:09.652+0000 7f2fa6a00700 1 Processor -- start 2024-03-21T10:12:11.031 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:12:09.652+0000 7f2fa6a00700 1 -- start start 2024-03-21T10:12:11.031 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:12:09.653+0000 7f2fa6a00700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2fa0105740 0x7f2fa01a1da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:12:11.031 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:12:09.653+0000 7f2fa6a00700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2fa0109bb0 con 0x7f2fa0105740 2024-03-21T10:12:11.031 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:12:09.653+0000 7f2f9fe00700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2fa0105740 0x7f2fa01a1da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:12:11.031 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:12:09.653+0000 7f2f9fe00700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2fa0105740 0x7f2fa01a1da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:60506/0 (socket says 172.21.15.47:60506) 2024-03-21T10:12:11.031 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:12:09.653+0000 7f2f9fe00700 1 -- 172.21.15.47:0/3950220199 learned_addr learned my addr 172.21.15.47:0/3950220199 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:12:11.031 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:12:09.653+0000 7f2f9fe00700 1 -- 172.21.15.47:0/3950220199 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2fa01a22e0 con 0x7f2fa0105740 2024-03-21T10:12:11.031 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:12:09.653+0000 7f2f9fe00700 1 --2- 172.21.15.47:0/3950220199 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2fa0105740 0x7f2fa01a1da0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f2f90011a50 tx=0x7f2f90011b30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:12:11.031 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:12:09.654+0000 7f2f9cc00700 1 -- 172.21.15.47:0/3950220199 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f2f90003a80 con 0x7f2fa0105740 2024-03-21T10:12:11.031 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:12:09.654+0000 7f2f9cc00700 1 -- 172.21.15.47:0/3950220199 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) v1 ==== 1071+0+0 (secure 0 0 0) 0x7f2f90003c20 con 0x7f2fa0105740 2024-03-21T10:12:11.031 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:12:09.654+0000 7f2fa6a00700 1 -- 172.21.15.47:0/3950220199 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2fa01a2570 con 0x7f2fa0105740 2024-03-21T10:12:11.031 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:12:09.654+0000 7f2f9cc00700 1 -- 172.21.15.47:0/3950220199 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f2f9001a650 con 0x7f2fa0105740 2024-03-21T10:12:11.031 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:12:09.654+0000 7f2fa6a00700 1 -- 172.21.15.47:0/3950220199 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2fa01a5260 con 0x7f2fa0105740 2024-03-21T10:12:11.031 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:12:09.655+0000 7f2f9cc00700 1 -- 172.21.15.47:0/3950220199 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 8) v1 ==== 46294+0+0 (secure 0 0 0) 0x7f2f90024ad0 con 0x7f2fa0105740 2024-03-21T10:12:11.032 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:12:09.655+0000 7f2fa6a00700 1 -- 172.21.15.47:0/3950220199 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2fa00637c0 con 0x7f2fa0105740 2024-03-21T10:12:11.032 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:12:09.655+0000 7f2f9cc00700 1 --2- 172.21.15.47:0/3950220199 >> [v2:172.21.15.47:6800/3836903972,v1:172.21.15.47:6801/3836903972] conn(0x7f2f840392f0 0x7f2f8403b7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:12:11.032 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:12:09.656+0000 7f2f9cc00700 1 -- 172.21.15.47:0/3950220199 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f2f9004d720 con 0x7f2fa0105740 2024-03-21T10:12:11.032 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:12:09.656+0000 7f2f9f400700 1 --2- 172.21.15.47:0/3950220199 >> [v2:172.21.15.47:6800/3836903972,v1:172.21.15.47:6801/3836903972] conn(0x7f2f840392f0 0x7f2f8403b7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:12:11.032 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:12:09.657+0000 7f2f9f400700 1 --2- 172.21.15.47:0/3950220199 >> [v2:172.21.15.47:6800/3836903972,v1:172.21.15.47:6801/3836903972] conn(0x7f2f840392f0 0x7f2f8403b7b0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f2f8c006fd0 tx=0x7f2f8c006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:12:11.032 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:12:09.662+0000 7f2f9cc00700 1 -- 172.21.15.47:0/3950220199 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f2f9002d770 con 0x7f2fa0105740 2024-03-21T10:12:11.032 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:12:09.904+0000 7f2fa6a00700 1 -- 172.21.15.47:0/3950220199 --> [v2:172.21.15.47:6800/3836903972,v1:172.21.15.47:6801/3836903972] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}) v1 -- 0x7f2fa01021c0 con 0x7f2f840392f0 2024-03-21T10:12:11.032 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:12:09.913+0000 7f2f9cc00700 1 -- 172.21.15.47:0/3950220199 <== mgr.14120 v2:172.21.15.47:6800/3836903972 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+24 (secure 0 0 0) 0x7f2fa01021c0 con 0x7f2f840392f0 2024-03-21T10:12:11.032 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:12:09.917+0000 7f2fa6a00700 1 -- 172.21.15.47:0/3950220199 >> [v2:172.21.15.47:6800/3836903972,v1:172.21.15.47:6801/3836903972] conn(0x7f2f840392f0 msgr2=0x7f2f8403b7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:12:11.032 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:12:09.917+0000 7f2fa6a00700 1 --2- 172.21.15.47:0/3950220199 >> [v2:172.21.15.47:6800/3836903972,v1:172.21.15.47:6801/3836903972] conn(0x7f2f840392f0 0x7f2f8403b7b0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f2f8c006fd0 tx=0x7f2f8c006e40 comp rx=0 tx=0).stop 2024-03-21T10:12:11.032 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:12:09.917+0000 7f2fa6a00700 1 -- 172.21.15.47:0/3950220199 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2fa0105740 msgr2=0x7f2fa01a1da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:12:11.032 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:12:09.917+0000 7f2fa6a00700 1 --2- 172.21.15.47:0/3950220199 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2fa0105740 0x7f2fa01a1da0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f2f90011a50 tx=0x7f2f90011b30 comp rx=0 tx=0).stop 2024-03-21T10:12:11.032 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:12:09.918+0000 7f2fa6a00700 1 -- 172.21.15.47:0/3950220199 shutdown_connections 2024-03-21T10:12:11.033 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:12:09.918+0000 7f2fa6a00700 1 --2- 172.21.15.47:0/3950220199 >> [v2:172.21.15.47:6800/3836903972,v1:172.21.15.47:6801/3836903972] conn(0x7f2f840392f0 0x7f2f8403b7b0 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:12:11.033 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:12:09.918+0000 7f2fa6a00700 1 --2- 172.21.15.47:0/3950220199 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2fa0105740 0x7f2fa01a1da0 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:12:11.033 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:12:09.918+0000 7f2fa6a00700 1 -- 172.21.15.47:0/3950220199 >> 172.21.15.47:0/3950220199 conn(0x7f2fa01011a0 msgr2=0x7f2fa0101ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:12:11.033 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:12:09.918+0000 7f2fa6a00700 1 -- 172.21.15.47:0/3950220199 shutdown_connections 2024-03-21T10:12:11.033 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:12:09.918+0000 7f2fa6a00700 1 -- 172.21.15.47:0/3950220199 wait complete. 2024-03-21T10:12:11.186 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:12:10 smithi047 ceph-mon[31547]: from='client.14144 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:12:11.186 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:12:10 smithi047 ceph-mon[31547]: Saving service mgr spec with placement count:2 2024-03-21T10:12:11.186 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:12:10 smithi047 ceph-mon[31547]: from='mgr.14120 172.21.15.47:0/2543639975' entity='mgr.x' 2024-03-21T10:12:11.186 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:12:10 smithi047 ceph-mon[31547]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T10:12:11.186 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:12:10 smithi047 ceph-mon[31547]: from='mgr.14120 172.21.15.47:0/2543639975' entity='mgr.x' 2024-03-21T10:12:12.658 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:12:12 smithi047 ceph-mon[31547]: from='mgr.14120 172.21.15.47:0/2543639975' entity='mgr.x' 2024-03-21T10:12:12.658 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:12:12 smithi047 ceph-mon[31547]: from='client.? 172.21.15.47:0/3607740136' entity='client.admin' 2024-03-21T10:12:12.658 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:12:12 smithi047 ceph-mon[31547]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T10:12:12.674 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:12:11.675+0000 7fc74f600700 1 Processor -- start 2024-03-21T10:12:12.674 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:12:11.675+0000 7fc74f600700 1 -- start start 2024-03-21T10:12:12.674 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:12:11.676+0000 7fc74f600700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc74807ce70 0x7fc74807b320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:12:12.674 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:12:11.676+0000 7fc74f600700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc74807b8f0 con 0x7fc74807ce70 2024-03-21T10:12:12.674 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:12:11.676+0000 7fc74cc00700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc74807ce70 0x7fc74807b320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:12:12.674 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:12:11.676+0000 7fc74cc00700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc74807ce70 0x7fc74807b320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:60522/0 (socket says 172.21.15.47:60522) 2024-03-21T10:12:12.674 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:12:11.676+0000 7fc74cc00700 1 -- 172.21.15.47:0/2411721619 learned_addr learned my addr 172.21.15.47:0/2411721619 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:12:12.675 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:12:11.677+0000 7fc74cc00700 1 -- 172.21.15.47:0/2411721619 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc74807ba70 con 0x7fc74807ce70 2024-03-21T10:12:12.675 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:12:11.677+0000 7fc74cc00700 1 --2- 172.21.15.47:0/2411721619 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc74807ce70 0x7fc74807b320 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fc73c0098a0 tx=0x7fc73c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8b2558bb5535f2f9 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:12:12.675 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:12:11.677+0000 7fc747400700 1 -- 172.21.15.47:0/2411721619 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc73c004160 con 0x7fc74807ce70 2024-03-21T10:12:12.675 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:12:11.678+0000 7fc747400700 1 -- 172.21.15.47:0/2411721619 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fc73c004300 con 0x7fc74807ce70 2024-03-21T10:12:12.675 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:12:11.678+0000 7fc747400700 1 -- 172.21.15.47:0/2411721619 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc73c011a00 con 0x7fc74807ce70 2024-03-21T10:12:12.675 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:12:11.678+0000 7fc74f600700 1 -- 172.21.15.47:0/2411721619 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc74807ce70 msgr2=0x7fc74807b320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:12:12.675 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:12:11.678+0000 7fc74f600700 1 --2- 172.21.15.47:0/2411721619 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc74807ce70 0x7fc74807b320 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fc73c0098a0 tx=0x7fc73c009bb0 comp rx=0 tx=0).stop 2024-03-21T10:12:12.675 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:12:11.678+0000 7fc74f600700 1 -- 172.21.15.47:0/2411721619 shutdown_connections 2024-03-21T10:12:12.675 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:12:11.678+0000 7fc74f600700 1 --2- 172.21.15.47:0/2411721619 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc74807ce70 0x7fc74807b320 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:12:12.675 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:12:11.678+0000 7fc74f600700 1 -- 172.21.15.47:0/2411721619 >> 172.21.15.47:0/2411721619 conn(0x7fc748102f50 msgr2=0x7fc748105370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:12:12.675 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:12:11.679+0000 7fc74f600700 1 -- 172.21.15.47:0/2411721619 shutdown_connections 2024-03-21T10:12:12.676 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:12:11.679+0000 7fc74f600700 1 -- 172.21.15.47:0/2411721619 wait complete. 2024-03-21T10:12:12.676 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:12:11.680+0000 7fc74f600700 1 Processor -- start 2024-03-21T10:12:12.676 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:12:11.680+0000 7fc74f600700 1 -- start start 2024-03-21T10:12:12.676 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:12:11.681+0000 7fc74f600700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc74807ce70 0x7fc74819d960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:12:12.676 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:12:11.681+0000 7fc74f600700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc74810de70 con 0x7fc74807ce70 2024-03-21T10:12:12.676 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:12:11.681+0000 7fc74cc00700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc74807ce70 0x7fc74819d960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:12:12.676 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:12:11.681+0000 7fc74cc00700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc74807ce70 0x7fc74819d960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:60534/0 (socket says 172.21.15.47:60534) 2024-03-21T10:12:12.676 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:12:11.681+0000 7fc74cc00700 1 -- 172.21.15.47:0/3607740136 learned_addr learned my addr 172.21.15.47:0/3607740136 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:12:12.676 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:12:11.681+0000 7fc74cc00700 1 -- 172.21.15.47:0/3607740136 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc74819dea0 con 0x7fc74807ce70 2024-03-21T10:12:12.676 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:12:11.682+0000 7fc74cc00700 1 --2- 172.21.15.47:0/3607740136 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc74807ce70 0x7fc74819d960 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fc73c004800 tx=0x7fc73c011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:12:12.676 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:12:11.682+0000 7fc745600700 1 -- 172.21.15.47:0/3607740136 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc73c0039c0 con 0x7fc74807ce70 2024-03-21T10:12:12.676 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:12:11.682+0000 7fc745600700 1 -- 172.21.15.47:0/3607740136 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fc73c003b60 con 0x7fc74807ce70 2024-03-21T10:12:12.677 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:12:11.682+0000 7fc745600700 1 -- 172.21.15.47:0/3607740136 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc73c0197a0 con 0x7fc74807ce70 2024-03-21T10:12:12.677 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:12:11.682+0000 7fc74f600700 1 -- 172.21.15.47:0/3607740136 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc74819e130 con 0x7fc74807ce70 2024-03-21T10:12:12.677 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:12:11.682+0000 7fc74f600700 1 -- 172.21.15.47:0/3607740136 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc7481a0e20 con 0x7fc74807ce70 2024-03-21T10:12:12.677 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:12:11.684+0000 7fc745600700 1 -- 172.21.15.47:0/3607740136 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 8) v1 ==== 46294+0+0 (secure 0 0 0) 0x7fc73c01ab90 con 0x7fc74807ce70 2024-03-21T10:12:12.677 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:12:11.684+0000 7fc74f600700 1 -- 172.21.15.47:0/3607740136 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc7480637c0 con 0x7fc74807ce70 2024-03-21T10:12:12.677 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:12:11.684+0000 7fc745600700 1 --2- 172.21.15.47:0/3607740136 >> [v2:172.21.15.47:6800/3836903972,v1:172.21.15.47:6801/3836903972] conn(0x7fc7300391d0 0x7fc73003b690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:12:12.677 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:12:11.684+0000 7fc745600700 1 -- 172.21.15.47:0/3607740136 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7fc73c04d770 con 0x7fc74807ce70 2024-03-21T10:12:12.677 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:12:11.685+0000 7fc747e00700 1 --2- 172.21.15.47:0/3607740136 >> [v2:172.21.15.47:6800/3836903972,v1:172.21.15.47:6801/3836903972] conn(0x7fc7300391d0 0x7fc73003b690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:12:12.677 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:12:11.685+0000 7fc747e00700 1 --2- 172.21.15.47:0/3607740136 >> [v2:172.21.15.47:6800/3836903972,v1:172.21.15.47:6801/3836903972] conn(0x7fc7300391d0 0x7fc73003b690 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fc734006fd0 tx=0x7fc734006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:12:12.677 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:12:11.690+0000 7fc745600700 1 -- 172.21.15.47:0/3607740136 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fc73c00b5a0 con 0x7fc74807ce70 2024-03-21T10:12:12.677 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:12:11.915+0000 7fc74f600700 1 -- 172.21.15.47:0/3607740136 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/container_init}] v 0) v1 -- 0x7fc7481a13d0 con 0x7fc74807ce70 2024-03-21T10:12:12.678 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:12:11.919+0000 7fc745600700 1 -- 172.21.15.47:0/3607740136 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/container_init}]=0 v7)=0 v7) v1 ==== 142+0+0 (secure 0 0 0) 0x7fc73c04fc70 con 0x7fc74807ce70 2024-03-21T10:12:12.678 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:12:11.924+0000 7fc74f600700 1 -- 172.21.15.47:0/3607740136 >> [v2:172.21.15.47:6800/3836903972,v1:172.21.15.47:6801/3836903972] conn(0x7fc7300391d0 msgr2=0x7fc73003b690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:12:12.678 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:12:11.924+0000 7fc74f600700 1 --2- 172.21.15.47:0/3607740136 >> [v2:172.21.15.47:6800/3836903972,v1:172.21.15.47:6801/3836903972] conn(0x7fc7300391d0 0x7fc73003b690 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fc734006fd0 tx=0x7fc734006e40 comp rx=0 tx=0).stop 2024-03-21T10:12:12.678 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:12:11.924+0000 7fc74f600700 1 -- 172.21.15.47:0/3607740136 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc74807ce70 msgr2=0x7fc74819d960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:12:12.678 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:12:11.924+0000 7fc74f600700 1 --2- 172.21.15.47:0/3607740136 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc74807ce70 0x7fc74819d960 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fc73c004800 tx=0x7fc73c011ad0 comp rx=0 tx=0).stop 2024-03-21T10:12:12.678 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:12:11.924+0000 7fc74f600700 1 -- 172.21.15.47:0/3607740136 shutdown_connections 2024-03-21T10:12:12.678 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:12:11.924+0000 7fc74f600700 1 --2- 172.21.15.47:0/3607740136 >> [v2:172.21.15.47:6800/3836903972,v1:172.21.15.47:6801/3836903972] conn(0x7fc7300391d0 0x7fc73003b690 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:12:12.678 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:12:11.924+0000 7fc74f600700 1 --2- 172.21.15.47:0/3607740136 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc74807ce70 0x7fc74819d960 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:12:12.678 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:12:11.924+0000 7fc74f600700 1 -- 172.21.15.47:0/3607740136 >> 172.21.15.47:0/3607740136 conn(0x7fc748102f50 msgr2=0x7fc748103a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:12:12.678 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:12:11.924+0000 7fc74f600700 1 -- 172.21.15.47:0/3607740136 shutdown_connections 2024-03-21T10:12:12.678 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:12:11.924+0000 7fc74f600700 1 -- 172.21.15.47:0/3607740136 wait complete. 2024-03-21T10:12:14.713 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:12:14 smithi047 ceph-mon[31547]: from='client.? 172.21.15.47:0/763298035' entity='client.admin' 2024-03-21T10:12:14.713 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:12:14 smithi047 ceph-mon[31547]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T10:12:14.785 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:12:13.305+0000 7f336e600700 1 Processor -- start 2024-03-21T10:12:14.786 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:12:13.306+0000 7f336e600700 1 -- start start 2024-03-21T10:12:14.786 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:12:13.306+0000 7f336e600700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3368106050 0x7f3368106470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:12:14.786 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:12:13.306+0000 7f336e600700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f336807ae10 con 0x7f3368106050 2024-03-21T10:12:14.786 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:12:13.307+0000 7f336d200700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3368106050 0x7f3368106470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:12:14.787 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:12:13.307+0000 7f336d200700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3368106050 0x7f3368106470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:42110/0 (socket says 172.21.15.47:42110) 2024-03-21T10:12:14.787 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:12:13.307+0000 7f336d200700 1 -- 172.21.15.47:0/55377126 learned_addr learned my addr 172.21.15.47:0/55377126 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:12:14.788 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:12:13.307+0000 7f336d200700 1 -- 172.21.15.47:0/55377126 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f33681069b0 con 0x7f3368106050 2024-03-21T10:12:14.788 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:12:13.308+0000 7f336d200700 1 --2- 172.21.15.47:0/55377126 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3368106050 0x7f3368106470 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f335c0098a0 tx=0x7f335c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=81d8b4466ce63ef7 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:12:14.788 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:12:13.308+0000 7f3367400700 1 -- 172.21.15.47:0/55377126 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f335c004160 con 0x7f3368106050 2024-03-21T10:12:14.788 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:12:13.308+0000 7f3367400700 1 -- 172.21.15.47:0/55377126 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f335c004300 con 0x7f3368106050 2024-03-21T10:12:14.788 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:12:13.308+0000 7f3367400700 1 -- 172.21.15.47:0/55377126 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f335c011a50 con 0x7f3368106050 2024-03-21T10:12:14.788 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:12:13.309+0000 7f336e600700 1 -- 172.21.15.47:0/55377126 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3368106050 msgr2=0x7f3368106470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:12:14.788 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:12:13.309+0000 7f336e600700 1 --2- 172.21.15.47:0/55377126 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3368106050 0x7f3368106470 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f335c0098a0 tx=0x7f335c009bb0 comp rx=0 tx=0).stop 2024-03-21T10:12:14.789 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:12:13.309+0000 7f336e600700 1 -- 172.21.15.47:0/55377126 shutdown_connections 2024-03-21T10:12:14.789 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:12:13.309+0000 7f336e600700 1 --2- 172.21.15.47:0/55377126 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3368106050 0x7f3368106470 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:12:14.789 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:12:13.309+0000 7f336e600700 1 -- 172.21.15.47:0/55377126 >> 172.21.15.47:0/55377126 conn(0x7f3368101cb0 msgr2=0x7f33681040d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:12:14.791 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:12:13.309+0000 7f336e600700 1 -- 172.21.15.47:0/55377126 shutdown_connections 2024-03-21T10:12:14.791 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:12:13.309+0000 7f336e600700 1 -- 172.21.15.47:0/55377126 wait complete. 2024-03-21T10:12:14.791 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:12:13.310+0000 7f336e600700 1 Processor -- start 2024-03-21T10:12:14.792 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:12:13.310+0000 7f336e600700 1 -- start start 2024-03-21T10:12:14.792 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:12:13.311+0000 7f336e600700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3368106050 0x7f336819d7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:12:14.792 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:12:13.311+0000 7f336e600700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3368107a20 con 0x7f3368106050 2024-03-21T10:12:14.792 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:12:13.311+0000 7f336d200700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3368106050 0x7f336819d7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:12:14.792 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:12:13.311+0000 7f336d200700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3368106050 0x7f336819d7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:42120/0 (socket says 172.21.15.47:42120) 2024-03-21T10:12:14.792 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:12:13.311+0000 7f336d200700 1 -- 172.21.15.47:0/763298035 learned_addr learned my addr 172.21.15.47:0/763298035 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:12:14.792 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:12:13.311+0000 7f336d200700 1 -- 172.21.15.47:0/763298035 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f336819dd10 con 0x7f3368106050 2024-03-21T10:12:14.792 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:12:13.312+0000 7f336d200700 1 --2- 172.21.15.47:0/763298035 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3368106050 0x7f336819d7d0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f335c011a20 tx=0x7f335c011b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:12:14.792 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:12:13.312+0000 7f3365600700 1 -- 172.21.15.47:0/763298035 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f335c0039c0 con 0x7f3368106050 2024-03-21T10:12:14.792 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:12:13.312+0000 7f3365600700 1 -- 172.21.15.47:0/763298035 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f335c003b60 con 0x7f3368106050 2024-03-21T10:12:14.792 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:12:13.312+0000 7f3365600700 1 -- 172.21.15.47:0/763298035 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f335c01a6b0 con 0x7f3368106050 2024-03-21T10:12:14.792 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:12:13.312+0000 7f336e600700 1 -- 172.21.15.47:0/763298035 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f336819dfa0 con 0x7f3368106050 2024-03-21T10:12:14.793 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:12:13.312+0000 7f336e600700 1 -- 172.21.15.47:0/763298035 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f33681a0c90 con 0x7f3368106050 2024-03-21T10:12:14.793 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:12:13.314+0000 7f3365600700 1 -- 172.21.15.47:0/763298035 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 8) v1 ==== 46294+0+0 (secure 0 0 0) 0x7f335c024430 con 0x7f3368106050 2024-03-21T10:12:14.793 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:12:13.314+0000 7f336e600700 1 -- 172.21.15.47:0/763298035 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f33680637c0 con 0x7f3368106050 2024-03-21T10:12:14.793 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:12:13.314+0000 7f3365600700 1 --2- 172.21.15.47:0/763298035 >> [v2:172.21.15.47:6800/3836903972,v1:172.21.15.47:6801/3836903972] conn(0x7f3350039240 0x7f335003b700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:12:14.793 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:12:13.314+0000 7f3365600700 1 -- 172.21.15.47:0/763298035 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f335c04d860 con 0x7f3368106050 2024-03-21T10:12:14.793 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:12:13.315+0000 7f3367e00700 1 --2- 172.21.15.47:0/763298035 >> [v2:172.21.15.47:6800/3836903972,v1:172.21.15.47:6801/3836903972] conn(0x7f3350039240 0x7f335003b700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:12:14.793 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:12:13.315+0000 7f3367e00700 1 --2- 172.21.15.47:0/763298035 >> [v2:172.21.15.47:6800/3836903972,v1:172.21.15.47:6801/3836903972] conn(0x7f3350039240 0x7f335003b700 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f3354006fd0 tx=0x7f3354006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:12:14.793 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:12:13.321+0000 7f3365600700 1 -- 172.21.15.47:0/763298035 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f335c024720 con 0x7f3368106050 2024-03-21T10:12:14.795 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:12:13.549+0000 7f336e600700 1 -- 172.21.15.47:0/763298035 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command([{prefix=config set, name=mgr/dashboard/ssl_server_port}] v 0) v1 -- 0x7f336802dc30 con 0x7f3368106050 2024-03-21T10:12:14.795 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:12:13.555+0000 7f3365600700 1 -- 172.21.15.47:0/763298035 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/dashboard/ssl_server_port}]=0 v8)=0 v8) v1 ==== 130+0+0 (secure 0 0 0) 0x7f335c04d1b0 con 0x7f3368106050 2024-03-21T10:12:14.795 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:12:13.558+0000 7f336e600700 1 -- 172.21.15.47:0/763298035 >> [v2:172.21.15.47:6800/3836903972,v1:172.21.15.47:6801/3836903972] conn(0x7f3350039240 msgr2=0x7f335003b700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:12:14.795 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:12:13.558+0000 7f336e600700 1 --2- 172.21.15.47:0/763298035 >> [v2:172.21.15.47:6800/3836903972,v1:172.21.15.47:6801/3836903972] conn(0x7f3350039240 0x7f335003b700 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f3354006fd0 tx=0x7f3354006e40 comp rx=0 tx=0).stop 2024-03-21T10:12:14.795 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:12:13.558+0000 7f336e600700 1 -- 172.21.15.47:0/763298035 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3368106050 msgr2=0x7f336819d7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:12:14.795 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:12:13.558+0000 7f336e600700 1 --2- 172.21.15.47:0/763298035 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3368106050 0x7f336819d7d0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f335c011a20 tx=0x7f335c011b00 comp rx=0 tx=0).stop 2024-03-21T10:12:14.795 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:12:13.558+0000 7f336e600700 1 -- 172.21.15.47:0/763298035 shutdown_connections 2024-03-21T10:12:14.795 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:12:13.558+0000 7f336e600700 1 --2- 172.21.15.47:0/763298035 >> [v2:172.21.15.47:6800/3836903972,v1:172.21.15.47:6801/3836903972] conn(0x7f3350039240 0x7f335003b700 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:12:14.795 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:12:13.559+0000 7f336e600700 1 --2- 172.21.15.47:0/763298035 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3368106050 0x7f336819d7d0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:12:14.795 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:12:13.559+0000 7f336e600700 1 -- 172.21.15.47:0/763298035 >> 172.21.15.47:0/763298035 conn(0x7f3368101cb0 msgr2=0x7f3368194470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:12:14.795 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:12:13.559+0000 7f336e600700 1 -- 172.21.15.47:0/763298035 shutdown_connections 2024-03-21T10:12:14.796 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:12:13.559+0000 7f336e600700 1 -- 172.21.15.47:0/763298035 wait complete. 2024-03-21T10:12:14.796 INFO:teuthology.orchestra.run.smithi047.stdout:Enabling the dashboard module... 2024-03-21T10:12:15.691 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:12:15 smithi047 ceph-mon[31547]: from='mgr.14120 172.21.15.47:0/2543639975' entity='mgr.x' 2024-03-21T10:12:16.904 INFO:journalctl@ceph.mgr.x.smithi047.stdout:Mar 21 10:12:16 smithi047 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-mgr-x[31773]: ignoring --setuser ceph since I am not root 2024-03-21T10:12:16.904 INFO:journalctl@ceph.mgr.x.smithi047.stdout:Mar 21 10:12:16 smithi047 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-mgr-x[31773]: ignoring --setgroup ceph since I am not root 2024-03-21T10:12:16.904 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:12:16 smithi047 ceph-mon[31547]: from='client.? 172.21.15.47:0/3062997849' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "dashboard"}]: dispatch 2024-03-21T10:12:16.904 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:12:16 smithi047 ceph-mon[31547]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T10:12:17.333 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:12:15.359+0000 7fdcd0600700 1 Processor -- start 2024-03-21T10:12:17.334 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:12:15.359+0000 7fdcd0600700 1 -- start start 2024-03-21T10:12:17.334 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:12:15.359+0000 7fdcd0600700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdcc8108320 0x7fdcc810a710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:12:17.334 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:12:15.359+0000 7fdcd0600700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdcc807ae50 con 0x7fdcc8108320 2024-03-21T10:12:17.334 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:12:15.360+0000 7fdccdc00700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdcc8108320 0x7fdcc810a710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:12:17.334 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:12:15.360+0000 7fdccdc00700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdcc8108320 0x7fdcc810a710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:42136/0 (socket says 172.21.15.47:42136) 2024-03-21T10:12:17.334 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:12:15.360+0000 7fdccdc00700 1 -- 172.21.15.47:0/3446135484 learned_addr learned my addr 172.21.15.47:0/3446135484 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:12:17.334 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:12:15.360+0000 7fdccdc00700 1 -- 172.21.15.47:0/3446135484 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdcc810ac50 con 0x7fdcc8108320 2024-03-21T10:12:17.334 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:12:15.361+0000 7fdccdc00700 1 --2- 172.21.15.47:0/3446135484 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdcc8108320 0x7fdcc810a710 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fdcbc0098a0 tx=0x7fdcbc009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4a0c8907d1bfc538 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:12:17.334 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:12:15.361+0000 7fdcc7e00700 1 -- 172.21.15.47:0/3446135484 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fdcbc004160 con 0x7fdcc8108320 2024-03-21T10:12:17.334 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:12:15.361+0000 7fdcc7e00700 1 -- 172.21.15.47:0/3446135484 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fdcbc004300 con 0x7fdcc8108320 2024-03-21T10:12:17.334 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:12:15.361+0000 7fdcc7e00700 1 -- 172.21.15.47:0/3446135484 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fdcbc011a50 con 0x7fdcc8108320 2024-03-21T10:12:17.334 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:12:15.362+0000 7fdcd0600700 1 -- 172.21.15.47:0/3446135484 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdcc8108320 msgr2=0x7fdcc810a710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:12:17.335 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:12:15.362+0000 7fdcd0600700 1 --2- 172.21.15.47:0/3446135484 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdcc8108320 0x7fdcc810a710 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fdcbc0098a0 tx=0x7fdcbc009bb0 comp rx=0 tx=0).stop 2024-03-21T10:12:17.335 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:12:15.362+0000 7fdcd0600700 1 -- 172.21.15.47:0/3446135484 shutdown_connections 2024-03-21T10:12:17.335 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:12:15.362+0000 7fdcd0600700 1 --2- 172.21.15.47:0/3446135484 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdcc8108320 0x7fdcc810a710 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:12:17.335 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:12:15.362+0000 7fdcd0600700 1 -- 172.21.15.47:0/3446135484 >> 172.21.15.47:0/3446135484 conn(0x7fdcc8101e30 msgr2=0x7fdcc8104250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:12:17.335 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:12:15.362+0000 7fdcd0600700 1 -- 172.21.15.47:0/3446135484 shutdown_connections 2024-03-21T10:12:17.335 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:12:15.362+0000 7fdcd0600700 1 -- 172.21.15.47:0/3446135484 wait complete. 2024-03-21T10:12:17.335 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:12:15.363+0000 7fdcd0600700 1 Processor -- start 2024-03-21T10:12:17.335 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:12:15.363+0000 7fdcd0600700 1 -- start start 2024-03-21T10:12:17.335 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:12:15.364+0000 7fdcd0600700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdcc8108320 0x7fdcc8107ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:12:17.335 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:12:15.364+0000 7fdcd0600700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdcc810bc50 con 0x7fdcc8108320 2024-03-21T10:12:17.335 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:12:15.364+0000 7fdccdc00700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdcc8108320 0x7fdcc8107ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:12:17.335 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:12:15.364+0000 7fdccdc00700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdcc8108320 0x7fdcc8107ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:42150/0 (socket says 172.21.15.47:42150) 2024-03-21T10:12:17.335 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:12:15.364+0000 7fdccdc00700 1 -- 172.21.15.47:0/3062997849 learned_addr learned my addr 172.21.15.47:0/3062997849 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:12:17.336 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:12:15.364+0000 7fdccdc00700 1 -- 172.21.15.47:0/3062997849 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdcc81080e0 con 0x7fdcc8108320 2024-03-21T10:12:17.336 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:12:15.365+0000 7fdccdc00700 1 --2- 172.21.15.47:0/3062997849 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdcc8108320 0x7fdcc8107ba0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fdcbc011a00 tx=0x7fdcbc011ae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:12:17.336 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:12:15.365+0000 7fdcc6000700 1 -- 172.21.15.47:0/3062997849 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fdcbc003af0 con 0x7fdcc8108320 2024-03-21T10:12:17.336 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:12:15.365+0000 7fdcc6000700 1 -- 172.21.15.47:0/3062997849 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fdcbc01a8e0 con 0x7fdcc8108320 2024-03-21T10:12:17.336 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:12:15.365+0000 7fdcc6000700 1 -- 172.21.15.47:0/3062997849 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fdcbc024840 con 0x7fdcc8108320 2024-03-21T10:12:17.336 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:12:15.365+0000 7fdcd0600700 1 -- 172.21.15.47:0/3062997849 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdcc81062d0 con 0x7fdcc8108320 2024-03-21T10:12:17.336 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:12:15.365+0000 7fdcd0600700 1 -- 172.21.15.47:0/3062997849 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdcc8106810 con 0x7fdcc8108320 2024-03-21T10:12:17.336 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:12:15.366+0000 7fdcc6000700 1 -- 172.21.15.47:0/3062997849 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 8) v1 ==== 46294+0+0 (secure 0 0 0) 0x7fdcbc022020 con 0x7fdcc8108320 2024-03-21T10:12:17.336 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:12:15.366+0000 7fdcd0600700 1 -- 172.21.15.47:0/3062997849 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdcc80637c0 con 0x7fdcc8108320 2024-03-21T10:12:17.336 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:12:15.367+0000 7fdcc6000700 1 --2- 172.21.15.47:0/3062997849 >> [v2:172.21.15.47:6800/3836903972,v1:172.21.15.47:6801/3836903972] conn(0x7fdcb0039180 0x7fdcb003b640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:12:17.336 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:12:15.367+0000 7fdcc6000700 1 -- 172.21.15.47:0/3062997849 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7fdcbc04d480 con 0x7fdcc8108320 2024-03-21T10:12:17.336 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:12:15.367+0000 7fdccd200700 1 --2- 172.21.15.47:0/3062997849 >> [v2:172.21.15.47:6800/3836903972,v1:172.21.15.47:6801/3836903972] conn(0x7fdcb0039180 0x7fdcb003b640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:12:17.336 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:12:15.368+0000 7fdccd200700 1 --2- 172.21.15.47:0/3062997849 >> [v2:172.21.15.47:6800/3836903972,v1:172.21.15.47:6801/3836903972] conn(0x7fdcb0039180 0x7fdcb003b640 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fdcb4006fd0 tx=0x7fdcb4006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:12:17.336 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:12:15.372+0000 7fdcc6000700 1 -- 172.21.15.47:0/3062997849 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fdcbc016260 con 0x7fdcc8108320 2024-03-21T10:12:17.336 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:12:15.689+0000 7fdcd0600700 1 -- 172.21.15.47:0/3062997849 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "dashboard"} v 0) v1 -- 0x7fdcc8106a90 con 0x7fdcc8108320 2024-03-21T10:12:17.337 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:12:16.570+0000 7fdcc6000700 1 -- 172.21.15.47:0/3062997849 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "dashboard"}]=0 v9) v1 ==== 88+0+0 (secure 0 0 0) 0x7fdcbc02ad90 con 0x7fdcc8108320 2024-03-21T10:12:17.337 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:12:16.574+0000 7fdcc6000700 1 -- 172.21.15.47:0/3062997849 <== mon.0 v2:172.21.15.47:3300/0 8 ==== mgrmap(e 9) v1 ==== 46307+0+0 (secure 0 0 0) 0x7fdcbc02c350 con 0x7fdcc8108320 2024-03-21T10:12:17.337 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:12:16.575+0000 7fdcd0600700 1 -- 172.21.15.47:0/3062997849 >> [v2:172.21.15.47:6800/3836903972,v1:172.21.15.47:6801/3836903972] conn(0x7fdcb0039180 msgr2=0x7fdcb003b640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:12:17.337 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:12:16.575+0000 7fdcd0600700 1 --2- 172.21.15.47:0/3062997849 >> [v2:172.21.15.47:6800/3836903972,v1:172.21.15.47:6801/3836903972] conn(0x7fdcb0039180 0x7fdcb003b640 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fdcb4006fd0 tx=0x7fdcb4006e40 comp rx=0 tx=0).stop 2024-03-21T10:12:17.337 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:12:16.575+0000 7fdcd0600700 1 -- 172.21.15.47:0/3062997849 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdcc8108320 msgr2=0x7fdcc8107ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:12:17.337 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:12:16.575+0000 7fdcd0600700 1 --2- 172.21.15.47:0/3062997849 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdcc8108320 0x7fdcc8107ba0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fdcbc011a00 tx=0x7fdcbc011ae0 comp rx=0 tx=0).stop 2024-03-21T10:12:17.337 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:12:16.575+0000 7fdcd0600700 1 -- 172.21.15.47:0/3062997849 shutdown_connections 2024-03-21T10:12:17.337 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:12:16.575+0000 7fdcd0600700 1 --2- 172.21.15.47:0/3062997849 >> [v2:172.21.15.47:6800/3836903972,v1:172.21.15.47:6801/3836903972] conn(0x7fdcb0039180 0x7fdcb003b640 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:12:17.337 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:12:16.575+0000 7fdcd0600700 1 --2- 172.21.15.47:0/3062997849 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdcc8108320 0x7fdcc8107ba0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:12:17.337 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:12:16.575+0000 7fdcd0600700 1 -- 172.21.15.47:0/3062997849 >> 172.21.15.47:0/3062997849 conn(0x7fdcc8101e30 msgr2=0x7fdcc8102910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:12:17.337 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:12:16.576+0000 7fdcd0600700 1 -- 172.21.15.47:0/3062997849 shutdown_connections 2024-03-21T10:12:17.337 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:12:16.576+0000 7fdcd0600700 1 -- 172.21.15.47:0/3062997849 wait complete. 2024-03-21T10:12:17.571 INFO:journalctl@ceph.mgr.x.smithi047.stdout:Mar 21 10:12:17 smithi047 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-mgr-x[31773]: 2024-03-21T10:12:17.484+0000 7fc45f223200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-03-21T10:12:17.879 INFO:journalctl@ceph.mgr.x.smithi047.stdout:Mar 21 10:12:17 smithi047 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-mgr-x[31773]: 2024-03-21T10:12:17.609+0000 7fc45f223200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-03-21T10:12:17.879 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:12:17 smithi047 ceph-mon[31547]: from='client.? 172.21.15.47:0/3062997849' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "dashboard"}]': finished 2024-03-21T10:12:17.879 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:12:17 smithi047 ceph-mon[31547]: mgrmap e9: x(active, since 26s) 2024-03-21T10:12:18.572 INFO:journalctl@ceph.mgr.x.smithi047.stdout:Mar 21 10:12:18 smithi047 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-mgr-x[31773]: 2024-03-21T10:12:18.448+0000 7fc45f223200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-03-21T10:12:18.910 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:12:18 smithi047 ceph-mon[31547]: from='client.? 172.21.15.47:0/1924186141' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-03-21T10:12:18.910 INFO:journalctl@ceph.mgr.x.smithi047.stdout:Mar 21 10:12:18 smithi047 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-mgr-x[31773]: 2024-03-21T10:12:18.578+0000 7fc45f223200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-03-21T10:12:18.910 INFO:journalctl@ceph.mgr.x.smithi047.stdout:Mar 21 10:12:18 smithi047 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-mgr-x[31773]: 2024-03-21T10:12:18.773+0000 7fc45f223200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-03-21T10:12:18.910 INFO:journalctl@ceph.mgr.x.smithi047.stdout:Mar 21 10:12:18 smithi047 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-mgr-x[31773]: 2024-03-21T10:12:18.908+0000 7fc45f223200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-03-21T10:12:19.159 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout { 2024-03-21T10:12:19.160 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "epoch": 9, 2024-03-21T10:12:19.160 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "available": true, 2024-03-21T10:12:19.160 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "active_name": "x", 2024-03-21T10:12:19.160 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-03-21T10:12:19.160 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout } 2024-03-21T10:12:19.166 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:12:18.032+0000 7f1efee00700 1 Processor -- start 2024-03-21T10:12:19.166 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:12:18.032+0000 7f1efee00700 1 -- start start 2024-03-21T10:12:19.166 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:12:18.033+0000 7f1efee00700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1ef807ce30 0x7f1ef807b2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:12:19.167 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:12:18.033+0000 7f1efee00700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ef807b8b0 con 0x7f1ef807ce30 2024-03-21T10:12:19.167 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:12:18.033+0000 7f1ef7e00700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1ef807ce30 0x7f1ef807b2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:12:19.167 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:12:18.033+0000 7f1ef7e00700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1ef807ce30 0x7f1ef807b2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:42180/0 (socket says 172.21.15.47:42180) 2024-03-21T10:12:19.167 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:12:18.033+0000 7f1ef7e00700 1 -- 172.21.15.47:0/707325388 learned_addr learned my addr 172.21.15.47:0/707325388 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:12:19.167 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:12:18.034+0000 7f1ef7e00700 1 -- 172.21.15.47:0/707325388 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ef807ba30 con 0x7f1ef807ce30 2024-03-21T10:12:19.167 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:12:18.034+0000 7f1ef7e00700 1 --2- 172.21.15.47:0/707325388 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1ef807ce30 0x7f1ef807b2e0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f1ee80098a0 tx=0x7f1ee8009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fcb4edb9a1f6fcd server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:12:19.167 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:12:18.034+0000 7f1ef6a00700 1 -- 172.21.15.47:0/707325388 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f1ee8004160 con 0x7f1ef807ce30 2024-03-21T10:12:19.167 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:12:18.034+0000 7f1ef6a00700 1 -- 172.21.15.47:0/707325388 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f1ee8004300 con 0x7f1ef807ce30 2024-03-21T10:12:19.167 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:12:18.034+0000 7f1ef6a00700 1 -- 172.21.15.47:0/707325388 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f1ee8011a50 con 0x7f1ef807ce30 2024-03-21T10:12:19.167 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:12:18.035+0000 7f1efee00700 1 -- 172.21.15.47:0/707325388 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1ef807ce30 msgr2=0x7f1ef807b2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:12:19.167 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:12:18.035+0000 7f1efee00700 1 --2- 172.21.15.47:0/707325388 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1ef807ce30 0x7f1ef807b2e0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f1ee80098a0 tx=0x7f1ee8009bb0 comp rx=0 tx=0).stop 2024-03-21T10:12:19.167 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:12:18.035+0000 7f1efee00700 1 -- 172.21.15.47:0/707325388 shutdown_connections 2024-03-21T10:12:19.167 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:12:18.035+0000 7f1efee00700 1 --2- 172.21.15.47:0/707325388 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1ef807ce30 0x7f1ef807b2e0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:12:19.167 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:12:18.035+0000 7f1efee00700 1 -- 172.21.15.47:0/707325388 >> 172.21.15.47:0/707325388 conn(0x7f1ef8102f50 msgr2=0x7f1ef8105370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:12:19.167 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:12:18.035+0000 7f1efee00700 1 -- 172.21.15.47:0/707325388 shutdown_connections 2024-03-21T10:12:19.168 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:12:18.035+0000 7f1efee00700 1 -- 172.21.15.47:0/707325388 wait complete. 2024-03-21T10:12:19.168 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:12:18.036+0000 7f1efee00700 1 Processor -- start 2024-03-21T10:12:19.168 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:12:18.036+0000 7f1efee00700 1 -- start start 2024-03-21T10:12:19.168 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:12:18.037+0000 7f1efee00700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1ef807ce30 0x7f1ef819d950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:12:19.168 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:12:18.037+0000 7f1efee00700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ef810de50 con 0x7f1ef807ce30 2024-03-21T10:12:19.168 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:12:18.037+0000 7f1ef7e00700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1ef807ce30 0x7f1ef819d950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:12:19.168 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:12:18.037+0000 7f1ef7e00700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1ef807ce30 0x7f1ef819d950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:42190/0 (socket says 172.21.15.47:42190) 2024-03-21T10:12:19.168 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:12:18.037+0000 7f1ef7e00700 1 -- 172.21.15.47:0/1924186141 learned_addr learned my addr 172.21.15.47:0/1924186141 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:12:19.168 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:12:18.037+0000 7f1ef7e00700 1 -- 172.21.15.47:0/1924186141 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ef819de90 con 0x7f1ef807ce30 2024-03-21T10:12:19.168 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:12:18.038+0000 7f1ef7e00700 1 --2- 172.21.15.47:0/1924186141 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1ef807ce30 0x7f1ef819d950 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f1ee8011a00 tx=0x7f1ee8011ae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:12:19.168 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:12:18.038+0000 7f1ef4c00700 1 -- 172.21.15.47:0/1924186141 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f1ee8003af0 con 0x7f1ef807ce30 2024-03-21T10:12:19.168 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:12:18.038+0000 7f1ef4c00700 1 -- 172.21.15.47:0/1924186141 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f1ee801a8e0 con 0x7f1ef807ce30 2024-03-21T10:12:19.168 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:12:18.038+0000 7f1efee00700 1 -- 172.21.15.47:0/1924186141 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1ef819e120 con 0x7f1ef807ce30 2024-03-21T10:12:19.168 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:12:18.038+0000 7f1ef4c00700 1 -- 172.21.15.47:0/1924186141 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f1ee8024840 con 0x7f1ef807ce30 2024-03-21T10:12:19.168 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:12:18.038+0000 7f1efee00700 1 -- 172.21.15.47:0/1924186141 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1ef81a0e10 con 0x7f1ef807ce30 2024-03-21T10:12:19.168 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:12:18.039+0000 7f1ef4c00700 1 -- 172.21.15.47:0/1924186141 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 9) v1 ==== 46307+0+0 (secure 0 0 0) 0x7f1ee8022020 con 0x7f1ef807ce30 2024-03-21T10:12:19.169 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:12:18.040+0000 7f1efee00700 1 -- 172.21.15.47:0/1924186141 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1ef80637c0 con 0x7f1ef807ce30 2024-03-21T10:12:19.169 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:12:18.040+0000 7f1ef4c00700 1 --2- 172.21.15.47:0/1924186141 >> [v2:172.21.15.47:6800/3836903972,v1:172.21.15.47:6801/3836903972] conn(0x7f1edc039230 0x7f1edc03b6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:12:19.169 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:12:18.040+0000 7f1ef4c00700 1 -- 172.21.15.47:0/1924186141 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f1ee804d630 con 0x7f1ef807ce30 2024-03-21T10:12:19.169 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:12:18.040+0000 7f1ef7400700 1 -- 172.21.15.47:0/1924186141 >> [v2:172.21.15.47:6800/3836903972,v1:172.21.15.47:6801/3836903972] conn(0x7f1edc039230 msgr2=0x7f1edc03b6f0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.47:6800/3836903972 2024-03-21T10:12:19.169 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:12:18.040+0000 7f1ef7400700 1 --2- 172.21.15.47:0/1924186141 >> [v2:172.21.15.47:6800/3836903972,v1:172.21.15.47:6801/3836903972] conn(0x7f1edc039230 0x7f1edc03b6f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-21T10:12:19.169 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:12:18.045+0000 7f1ef4c00700 1 -- 172.21.15.47:0/1924186141 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f1ee800b3e0 con 0x7f1ef807ce30 2024-03-21T10:12:19.169 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:12:18.241+0000 7f1ef7400700 1 -- 172.21.15.47:0/1924186141 >> [v2:172.21.15.47:6800/3836903972,v1:172.21.15.47:6801/3836903972] conn(0x7f1edc039230 msgr2=0x7f1edc03b6f0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.47:6800/3836903972 2024-03-21T10:12:19.169 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:12:18.241+0000 7f1ef7400700 1 --2- 172.21.15.47:0/1924186141 >> [v2:172.21.15.47:6800/3836903972,v1:172.21.15.47:6801/3836903972] conn(0x7f1edc039230 0x7f1edc03b6f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-21T10:12:19.169 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:12:18.346+0000 7f1efee00700 1 -- 172.21.15.47:0/1924186141 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) v1 -- 0x7f1ef81a13c0 con 0x7f1ef807ce30 2024-03-21T10:12:19.169 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:12:18.347+0000 7f1ef4c00700 1 -- 172.21.15.47:0/1924186141 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v9) v1 ==== 56+0+88 (secure 0 0 0) 0x7f1ee801a430 con 0x7f1ef807ce30 2024-03-21T10:12:19.169 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:12:18.350+0000 7f1efee00700 1 -- 172.21.15.47:0/1924186141 >> [v2:172.21.15.47:6800/3836903972,v1:172.21.15.47:6801/3836903972] conn(0x7f1edc039230 msgr2=0x7f1edc03b6f0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-21T10:12:19.169 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:12:18.350+0000 7f1efee00700 1 --2- 172.21.15.47:0/1924186141 >> [v2:172.21.15.47:6800/3836903972,v1:172.21.15.47:6801/3836903972] conn(0x7f1edc039230 0x7f1edc03b6f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:12:19.169 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:12:18.350+0000 7f1efee00700 1 -- 172.21.15.47:0/1924186141 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1ef807ce30 msgr2=0x7f1ef819d950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:12:19.169 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:12:18.350+0000 7f1efee00700 1 --2- 172.21.15.47:0/1924186141 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1ef807ce30 0x7f1ef819d950 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f1ee8011a00 tx=0x7f1ee8011ae0 comp rx=0 tx=0).stop 2024-03-21T10:12:19.169 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:12:18.350+0000 7f1efee00700 1 -- 172.21.15.47:0/1924186141 shutdown_connections 2024-03-21T10:12:19.170 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:12:18.350+0000 7f1efee00700 1 --2- 172.21.15.47:0/1924186141 >> [v2:172.21.15.47:6800/3836903972,v1:172.21.15.47:6801/3836903972] conn(0x7f1edc039230 0x7f1edc03b6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:12:19.170 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:12:18.350+0000 7f1efee00700 1 --2- 172.21.15.47:0/1924186141 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1ef807ce30 0x7f1ef819d950 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:12:19.170 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:12:18.350+0000 7f1efee00700 1 -- 172.21.15.47:0/1924186141 >> 172.21.15.47:0/1924186141 conn(0x7f1ef8102f50 msgr2=0x7f1ef81039e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:12:19.170 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:12:18.351+0000 7f1efee00700 1 -- 172.21.15.47:0/1924186141 shutdown_connections 2024-03-21T10:12:19.170 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:12:18.351+0000 7f1efee00700 1 -- 172.21.15.47:0/1924186141 wait complete. 2024-03-21T10:12:19.170 INFO:teuthology.orchestra.run.smithi047.stdout:Waiting for the mgr to restart... 2024-03-21T10:12:19.170 INFO:teuthology.orchestra.run.smithi047.stdout:Waiting for mgr epoch 9... 2024-03-21T10:12:19.213 INFO:journalctl@ceph.mgr.x.smithi047.stdout:Mar 21 10:12:19 smithi047 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-mgr-x[31773]: 2024-03-21T10:12:19.024+0000 7fc45f223200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-03-21T10:12:19.586 INFO:journalctl@ceph.mgr.x.smithi047.stdout:Mar 21 10:12:19 smithi047 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-mgr-x[31773]: 2024-03-21T10:12:19.386+0000 7fc45f223200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-03-21T10:12:19.586 INFO:journalctl@ceph.mgr.x.smithi047.stdout:Mar 21 10:12:19 smithi047 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-mgr-x[31773]: 2024-03-21T10:12:19.535+0000 7fc45f223200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-03-21T10:12:19.954 INFO:journalctl@ceph.mgr.x.smithi047.stdout:Mar 21 10:12:19 smithi047 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-mgr-x[31773]: 2024-03-21T10:12:19.655+0000 7fc45f223200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-03-21T10:12:19.954 INFO:journalctl@ceph.mgr.x.smithi047.stdout:Mar 21 10:12:19 smithi047 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-mgr-x[31773]: 2024-03-21T10:12:19.800+0000 7fc45f223200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-03-21T10:12:20.214 INFO:journalctl@ceph.mgr.x.smithi047.stdout:Mar 21 10:12:20 smithi047 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-mgr-x[31773]: 2024-03-21T10:12:20.041+0000 7fc45f223200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-03-21T10:12:21.463 INFO:journalctl@ceph.mgr.x.smithi047.stdout:Mar 21 10:12:21 smithi047 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-mgr-x[31773]: 2024-03-21T10:12:21.356+0000 7fc45f223200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-03-21T10:12:21.964 INFO:journalctl@ceph.mgr.x.smithi047.stdout:Mar 21 10:12:21 smithi047 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-mgr-x[31773]: 2024-03-21T10:12:21.534+0000 7fc45f223200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-03-21T10:12:22.349 INFO:journalctl@ceph.mgr.x.smithi047.stdout:Mar 21 10:12:21 smithi047 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-mgr-x[31773]: 2024-03-21T10:12:21.963+0000 7fc45f223200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-03-21T10:12:22.349 INFO:journalctl@ceph.mgr.x.smithi047.stdout:Mar 21 10:12:22 smithi047 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-mgr-x[31773]: 2024-03-21T10:12:22.085+0000 7fc45f223200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-03-21T10:12:22.349 INFO:journalctl@ceph.mgr.x.smithi047.stdout:Mar 21 10:12:22 smithi047 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-mgr-x[31773]: 2024-03-21T10:12:22.347+0000 7fc45f223200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-03-21T10:12:22.963 INFO:journalctl@ceph.mgr.x.smithi047.stdout:Mar 21 10:12:22 smithi047 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-mgr-x[31773]: 2024-03-21T10:12:22.598+0000 7fc45f223200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-03-21T10:12:23.310 INFO:journalctl@ceph.mgr.x.smithi047.stdout:Mar 21 10:12:22 smithi047 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-mgr-x[31773]: 2024-03-21T10:12:22.986+0000 7fc45f223200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-03-21T10:12:23.624 INFO:journalctl@ceph.mgr.x.smithi047.stdout:Mar 21 10:12:23 smithi047 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-mgr-x[31773]: 2024-03-21T10:12:23.308+0000 7fc45f223200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-03-21T10:12:23.963 INFO:journalctl@ceph.mgr.x.smithi047.stdout:Mar 21 10:12:23 smithi047 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-mgr-x[31773]: 2024-03-21T10:12:23.622+0000 7fc45f223200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-03-21T10:12:24.963 INFO:journalctl@ceph.mgr.x.smithi047.stdout:Mar 21 10:12:24 smithi047 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-mgr-x[31773]: 2024-03-21T10:12:24.690+0000 7fc45f223200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-03-21T10:12:25.714 INFO:journalctl@ceph.mgr.x.smithi047.stdout:Mar 21 10:12:25 smithi047 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-mgr-x[31773]: 2024-03-21T10:12:25.410+0000 7fc45f223200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-03-21T10:12:25.714 INFO:journalctl@ceph.mgr.x.smithi047.stdout:Mar 21 10:12:25 smithi047 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-mgr-x[31773]: 2024-03-21T10:12:25.545+0000 7fc45f223200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-03-21T10:12:26.214 INFO:journalctl@ceph.mgr.x.smithi047.stdout:Mar 21 10:12:25 smithi047 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-mgr-x[31773]: 2024-03-21T10:12:25.841+0000 7fc45f223200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-03-21T10:12:26.214 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:12:25 smithi047 ceph-mon[31547]: Active manager daemon x restarted 2024-03-21T10:12:26.214 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:12:25 smithi047 ceph-mon[31547]: Activating manager daemon x 2024-03-21T10:12:26.215 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:12:25 smithi047 ceph-mon[31547]: osdmap e3: 0 total, 0 up, 0 in 2024-03-21T10:12:26.215 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:12:25 smithi047 ceph-mon[31547]: mgrmap e10: x(active, starting, since 0.00480662s) 2024-03-21T10:12:26.215 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:12:25 smithi047 ceph-mon[31547]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-21T10:12:26.215 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:12:25 smithi047 ceph-mon[31547]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-03-21T10:12:26.215 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:12:25 smithi047 ceph-mon[31547]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-03-21T10:12:26.215 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:12:25 smithi047 ceph-mon[31547]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-03-21T10:12:26.216 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:12:25 smithi047 ceph-mon[31547]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-03-21T10:12:26.216 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:12:25 smithi047 ceph-mon[31547]: Manager daemon x is now available 2024-03-21T10:12:26.216 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:12:25 smithi047 ceph-mon[31547]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T10:12:27.214 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:12:26 smithi047 ceph-mon[31547]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-03-21T10:12:27.214 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:12:26 smithi047 ceph-mon[31547]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-03-21T10:12:27.214 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:12:26 smithi047 ceph-mon[31547]: mgrmap e11: x(active, since 1.00899s) 2024-03-21T10:12:27.554 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout { 2024-03-21T10:12:27.554 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 11, 2024-03-21T10:12:27.554 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "initialized": true 2024-03-21T10:12:27.554 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout } 2024-03-21T10:12:27.554 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:12:19.743+0000 7fdf96600700 1 Processor -- start 2024-03-21T10:12:27.554 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:12:19.743+0000 7fdf96600700 1 -- start start 2024-03-21T10:12:27.554 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:12:19.743+0000 7fdf96600700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdf90106160 0x7fdf90106580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:12:27.555 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:12:19.743+0000 7fdf96600700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf9007ae10 con 0x7fdf90106160 2024-03-21T10:12:27.555 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:12:19.744+0000 7fdf8f400700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdf90106160 0x7fdf90106580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:12:27.555 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:12:19.744+0000 7fdf8f400700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdf90106160 0x7fdf90106580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:42192/0 (socket says 172.21.15.47:42192) 2024-03-21T10:12:27.555 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:12:19.744+0000 7fdf8f400700 1 -- 172.21.15.47:0/3729670788 learned_addr learned my addr 172.21.15.47:0/3729670788 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:12:27.555 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:12:19.744+0000 7fdf8f400700 1 -- 172.21.15.47:0/3729670788 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf90106ac0 con 0x7fdf90106160 2024-03-21T10:12:27.555 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:12:19.745+0000 7fdf8f400700 1 --2- 172.21.15.47:0/3729670788 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdf90106160 0x7fdf90106580 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fdf800098a0 tx=0x7fdf80009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=89725af0bf99b38c server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:12:27.555 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:12:19.745+0000 7fdf8e000700 1 -- 172.21.15.47:0/3729670788 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fdf80004160 con 0x7fdf90106160 2024-03-21T10:12:27.555 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:12:19.745+0000 7fdf8e000700 1 -- 172.21.15.47:0/3729670788 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fdf80004300 con 0x7fdf90106160 2024-03-21T10:12:27.555 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:12:19.745+0000 7fdf8e000700 1 -- 172.21.15.47:0/3729670788 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fdf80011a50 con 0x7fdf90106160 2024-03-21T10:12:27.555 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:12:19.745+0000 7fdf96600700 1 -- 172.21.15.47:0/3729670788 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdf90106160 msgr2=0x7fdf90106580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:12:27.555 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:12:19.745+0000 7fdf96600700 1 --2- 172.21.15.47:0/3729670788 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdf90106160 0x7fdf90106580 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fdf800098a0 tx=0x7fdf80009bb0 comp rx=0 tx=0).stop 2024-03-21T10:12:27.555 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:12:19.746+0000 7fdf96600700 1 -- 172.21.15.47:0/3729670788 shutdown_connections 2024-03-21T10:12:27.555 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:12:19.746+0000 7fdf96600700 1 --2- 172.21.15.47:0/3729670788 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdf90106160 0x7fdf90106580 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:12:27.555 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:12:19.746+0000 7fdf96600700 1 -- 172.21.15.47:0/3729670788 >> 172.21.15.47:0/3729670788 conn(0x7fdf90101de0 msgr2=0x7fdf90104200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:12:27.555 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:12:19.746+0000 7fdf96600700 1 -- 172.21.15.47:0/3729670788 shutdown_connections 2024-03-21T10:12:27.555 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:12:19.746+0000 7fdf96600700 1 -- 172.21.15.47:0/3729670788 wait complete. 2024-03-21T10:12:27.556 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:12:19.747+0000 7fdf96600700 1 Processor -- start 2024-03-21T10:12:27.556 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:12:19.747+0000 7fdf96600700 1 -- start start 2024-03-21T10:12:27.556 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:12:19.747+0000 7fdf96600700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdf90106160 0x7fdf9019d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:12:27.556 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:12:19.747+0000 7fdf96600700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf90107b30 con 0x7fdf90106160 2024-03-21T10:12:27.556 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:12:19.747+0000 7fdf8f400700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdf90106160 0x7fdf9019d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:12:27.556 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:12:19.748+0000 7fdf8f400700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdf90106160 0x7fdf9019d990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:42204/0 (socket says 172.21.15.47:42204) 2024-03-21T10:12:27.556 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:12:19.748+0000 7fdf8f400700 1 -- 172.21.15.47:0/4111382225 learned_addr learned my addr 172.21.15.47:0/4111382225 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:12:27.556 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:12:19.748+0000 7fdf8f400700 1 -- 172.21.15.47:0/4111382225 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf9019ded0 con 0x7fdf90106160 2024-03-21T10:12:27.556 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:12:19.748+0000 7fdf8f400700 1 --2- 172.21.15.47:0/4111382225 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdf90106160 0x7fdf9019d990 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fdf80011a00 tx=0x7fdf80011ae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:12:27.556 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:12:19.748+0000 7fdf87e00700 1 -- 172.21.15.47:0/4111382225 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fdf80003af0 con 0x7fdf90106160 2024-03-21T10:12:27.556 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:12:19.748+0000 7fdf87e00700 1 -- 172.21.15.47:0/4111382225 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fdf8001a8e0 con 0x7fdf90106160 2024-03-21T10:12:27.556 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:12:19.748+0000 7fdf96600700 1 -- 172.21.15.47:0/4111382225 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdf9019e160 con 0x7fdf90106160 2024-03-21T10:12:27.556 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:12:19.749+0000 7fdf87e00700 1 -- 172.21.15.47:0/4111382225 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fdf80024840 con 0x7fdf90106160 2024-03-21T10:12:27.556 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:12:19.749+0000 7fdf96600700 1 -- 172.21.15.47:0/4111382225 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdf901a0e50 con 0x7fdf90106160 2024-03-21T10:12:27.556 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:12:19.750+0000 7fdf87e00700 1 -- 172.21.15.47:0/4111382225 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 9) v1 ==== 46307+0+0 (secure 0 0 0) 0x7fdf80022020 con 0x7fdf90106160 2024-03-21T10:12:27.557 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:12:19.750+0000 7fdf87e00700 1 --2- 172.21.15.47:0/4111382225 >> [v2:172.21.15.47:6800/3836903972,v1:172.21.15.47:6801/3836903972] conn(0x7fdf74039230 0x7fdf7403b6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:12:27.557 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:12:19.750+0000 7fdf87e00700 1 -- 172.21.15.47:0/4111382225 --> [v2:172.21.15.47:6800/3836903972,v1:172.21.15.47:6801/3836903972] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7fdf7403be00 con 0x7fdf74039230 2024-03-21T10:12:27.557 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:12:19.750+0000 7fdf87e00700 1 -- 172.21.15.47:0/4111382225 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7fdf8004d5a0 con 0x7fdf90106160 2024-03-21T10:12:27.557 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:12:19.751+0000 7fdf8ea00700 1 -- 172.21.15.47:0/4111382225 >> [v2:172.21.15.47:6800/3836903972,v1:172.21.15.47:6801/3836903972] conn(0x7fdf74039230 msgr2=0x7fdf7403b6f0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.47:6800/3836903972 2024-03-21T10:12:27.557 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:12:19.751+0000 7fdf8ea00700 1 --2- 172.21.15.47:0/4111382225 >> [v2:172.21.15.47:6800/3836903972,v1:172.21.15.47:6801/3836903972] conn(0x7fdf74039230 0x7fdf7403b6f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-21T10:12:27.557 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:12:19.952+0000 7fdf8ea00700 1 -- 172.21.15.47:0/4111382225 >> [v2:172.21.15.47:6800/3836903972,v1:172.21.15.47:6801/3836903972] conn(0x7fdf74039230 msgr2=0x7fdf7403b6f0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.47:6800/3836903972 2024-03-21T10:12:27.557 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:12:19.952+0000 7fdf8ea00700 1 --2- 172.21.15.47:0/4111382225 >> [v2:172.21.15.47:6800/3836903972,v1:172.21.15.47:6801/3836903972] conn(0x7fdf74039230 0x7fdf7403b6f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-21T10:12:27.557 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:12:20.353+0000 7fdf8ea00700 1 -- 172.21.15.47:0/4111382225 >> [v2:172.21.15.47:6800/3836903972,v1:172.21.15.47:6801/3836903972] conn(0x7fdf74039230 msgr2=0x7fdf7403b6f0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.47:6800/3836903972 2024-03-21T10:12:27.557 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:12:20.353+0000 7fdf8ea00700 1 --2- 172.21.15.47:0/4111382225 >> [v2:172.21.15.47:6800/3836903972,v1:172.21.15.47:6801/3836903972] conn(0x7fdf74039230 0x7fdf7403b6f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-21T10:12:27.557 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:12:21.154+0000 7fdf8ea00700 1 -- 172.21.15.47:0/4111382225 >> [v2:172.21.15.47:6800/3836903972,v1:172.21.15.47:6801/3836903972] conn(0x7fdf74039230 msgr2=0x7fdf7403b6f0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.47:6800/3836903972 2024-03-21T10:12:27.557 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:12:21.154+0000 7fdf8ea00700 1 --2- 172.21.15.47:0/4111382225 >> [v2:172.21.15.47:6800/3836903972,v1:172.21.15.47:6801/3836903972] conn(0x7fdf74039230 0x7fdf7403b6f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-21T10:12:27.557 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:12:22.756+0000 7fdf8ea00700 1 -- 172.21.15.47:0/4111382225 >> [v2:172.21.15.47:6800/3836903972,v1:172.21.15.47:6801/3836903972] conn(0x7fdf74039230 msgr2=0x7fdf7403b6f0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.47:6800/3836903972 2024-03-21T10:12:27.557 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:12:22.756+0000 7fdf8ea00700 1 --2- 172.21.15.47:0/4111382225 >> [v2:172.21.15.47:6800/3836903972,v1:172.21.15.47:6801/3836903972] conn(0x7fdf74039230 0x7fdf7403b6f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2024-03-21T10:12:27.557 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:12:25.848+0000 7fdf87e00700 1 -- 172.21.15.47:0/4111382225 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mgrmap(e 10) v1 ==== 46074+0+0 (secure 0 0 0) 0x7fdf80024af0 con 0x7fdf90106160 2024-03-21T10:12:27.558 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:12:25.849+0000 7fdf87e00700 1 -- 172.21.15.47:0/4111382225 >> [v2:172.21.15.47:6800/3836903972,v1:172.21.15.47:6801/3836903972] conn(0x7fdf74039230 msgr2=0x7fdf7403b6f0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-21T10:12:27.558 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:12:25.849+0000 7fdf87e00700 1 --2- 172.21.15.47:0/4111382225 >> [v2:172.21.15.47:6800/3836903972,v1:172.21.15.47:6801/3836903972] conn(0x7fdf74039230 0x7fdf7403b6f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:12:27.558 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:12:26.850+0000 7fdf87e00700 1 -- 172.21.15.47:0/4111382225 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mgrmap(e 11) v1 ==== 46201+0+0 (secure 0 0 0) 0x7fdf8004ecb0 con 0x7fdf90106160 2024-03-21T10:12:27.558 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:12:26.850+0000 7fdf87e00700 1 --2- 172.21.15.47:0/4111382225 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7fdf7403cce0 0x7fdf7403f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:12:27.558 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:12:26.850+0000 7fdf87e00700 1 -- 172.21.15.47:0/4111382225 --> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7fdf7403be00 con 0x7fdf7403cce0 2024-03-21T10:12:27.558 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:12:26.851+0000 7fdf8ea00700 1 --2- 172.21.15.47:0/4111382225 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7fdf7403cce0 0x7fdf7403f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:12:27.558 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:12:26.853+0000 7fdf8ea00700 1 --2- 172.21.15.47:0/4111382225 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7fdf7403cce0 0x7fdf7403f0d0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7fdf7c003a10 tx=0x7fdf7c009270 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:12:27.558 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:12:26.854+0000 7fdf87e00700 1 -- 172.21.15.47:0/4111382225 <== mgr.14152 v2:172.21.15.47:6800/2823885994 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+7774 (secure 0 0 0) 0x7fdf7403be00 con 0x7fdf7403cce0 2024-03-21T10:12:27.558 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:12:26.861+0000 7fdf96600700 1 -- 172.21.15.47:0/4111382225 --> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] -- command(tid 1: {"prefix": "mgr_status"}) v1 -- 0x7fdf900637c0 con 0x7fdf7403cce0 2024-03-21T10:12:27.558 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:12:26.862+0000 7fdf87e00700 1 -- 172.21.15.47:0/4111382225 <== mgr.14152 v2:172.21.15.47:6800/2823885994 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+52 (secure 0 0 0) 0x7fdf900637c0 con 0x7fdf7403cce0 2024-03-21T10:12:27.558 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:12:26.862+0000 7fdf96600700 1 -- 172.21.15.47:0/4111382225 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7fdf7403cce0 msgr2=0x7fdf7403f0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:12:27.558 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:12:26.862+0000 7fdf96600700 1 --2- 172.21.15.47:0/4111382225 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7fdf7403cce0 0x7fdf7403f0d0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7fdf7c003a10 tx=0x7fdf7c009270 comp rx=0 tx=0).stop 2024-03-21T10:12:27.558 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:12:26.862+0000 7fdf96600700 1 -- 172.21.15.47:0/4111382225 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdf90106160 msgr2=0x7fdf9019d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:12:27.558 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:12:26.862+0000 7fdf96600700 1 --2- 172.21.15.47:0/4111382225 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdf90106160 0x7fdf9019d990 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fdf80011a00 tx=0x7fdf80011ae0 comp rx=0 tx=0).stop 2024-03-21T10:12:27.559 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:12:26.862+0000 7fdf96600700 1 -- 172.21.15.47:0/4111382225 shutdown_connections 2024-03-21T10:12:27.559 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:12:26.862+0000 7fdf96600700 1 --2- 172.21.15.47:0/4111382225 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7fdf7403cce0 0x7fdf7403f0d0 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:12:27.559 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:12:26.862+0000 7fdf96600700 1 --2- 172.21.15.47:0/4111382225 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdf90106160 0x7fdf9019d990 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:12:27.559 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:12:26.862+0000 7fdf96600700 1 --2- 172.21.15.47:0/4111382225 >> [v2:172.21.15.47:6800/3836903972,v1:172.21.15.47:6801/3836903972] conn(0x7fdf74039230 0x7fdf7403b6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:12:27.559 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:12:26.862+0000 7fdf96600700 1 -- 172.21.15.47:0/4111382225 >> 172.21.15.47:0/4111382225 conn(0x7fdf90101de0 msgr2=0x7fdf90194570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:12:27.559 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:12:26.862+0000 7fdf96600700 1 -- 172.21.15.47:0/4111382225 shutdown_connections 2024-03-21T10:12:27.559 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:12:26.863+0000 7fdf96600700 1 -- 172.21.15.47:0/4111382225 wait complete. 2024-03-21T10:12:27.559 INFO:teuthology.orchestra.run.smithi047.stdout:mgr epoch 9 is available 2024-03-21T10:12:27.559 INFO:teuthology.orchestra.run.smithi047.stdout:Generating a dashboard self-signed certificate... 2024-03-21T10:12:28.214 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:12:27 smithi047 ceph-mon[31547]: [21/Mar/2024:10:12:26] ENGINE Bus STARTING 2024-03-21T10:12:28.214 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:12:27 smithi047 ceph-mon[31547]: from='client.14156 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-03-21T10:12:28.214 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:12:27 smithi047 ceph-mon[31547]: from='client.14156 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-03-21T10:12:28.214 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:12:27 smithi047 ceph-mon[31547]: [21/Mar/2024:10:12:26] ENGINE Serving on http://172.21.15.47:8765 2024-03-21T10:12:28.214 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:12:27 smithi047 ceph-mon[31547]: [21/Mar/2024:10:12:27] ENGINE Serving on https://172.21.15.47:7150 2024-03-21T10:12:28.215 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:12:27 smithi047 ceph-mon[31547]: [21/Mar/2024:10:12:27] ENGINE Bus STARTED 2024-03-21T10:12:29.214 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:12:28 smithi047 ceph-mon[31547]: mgrmap e12: x(active, since 2s) 2024-03-21T10:12:29.214 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:12:28 smithi047 ceph-mon[31547]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:12:29.214 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:12:28 smithi047 ceph-mon[31547]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:12:29.591 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout Self-signed certificate created 2024-03-21T10:12:29.591 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:12:28.525+0000 7fba35800700 1 Processor -- start 2024-03-21T10:12:29.591 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:12:28.525+0000 7fba35800700 1 -- start start 2024-03-21T10:12:29.591 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:12:28.526+0000 7fba35800700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fba301078b0 0x7fba30107cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:12:29.591 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:12:28.526+0000 7fba35800700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba301082a0 con 0x7fba301078b0 2024-03-21T10:12:29.591 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:12:28.526+0000 7fba2ea00700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fba301078b0 0x7fba30107cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:12:29.591 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:12:28.526+0000 7fba2ea00700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fba301078b0 0x7fba30107cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:39412/0 (socket says 172.21.15.47:39412) 2024-03-21T10:12:29.591 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:12:28.526+0000 7fba2ea00700 1 -- 172.21.15.47:0/3147392302 learned_addr learned my addr 172.21.15.47:0/3147392302 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:12:29.591 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:12:28.527+0000 7fba2ea00700 1 -- 172.21.15.47:0/3147392302 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba30108a60 con 0x7fba301078b0 2024-03-21T10:12:29.592 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:12:28.527+0000 7fba2ea00700 1 --2- 172.21.15.47:0/3147392302 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fba301078b0 0x7fba30107cd0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fba200098a0 tx=0x7fba20009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ab4604afd6a627c0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:12:29.592 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:12:28.528+0000 7fba2e000700 1 -- 172.21.15.47:0/3147392302 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fba20004160 con 0x7fba301078b0 2024-03-21T10:12:29.592 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:12:28.528+0000 7fba2e000700 1 -- 172.21.15.47:0/3147392302 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fba20004300 con 0x7fba301078b0 2024-03-21T10:12:29.592 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:12:28.528+0000 7fba2e000700 1 -- 172.21.15.47:0/3147392302 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fba20011a50 con 0x7fba301078b0 2024-03-21T10:12:29.592 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:12:28.528+0000 7fba35800700 1 -- 172.21.15.47:0/3147392302 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fba301078b0 msgr2=0x7fba30107cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:12:29.592 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:12:28.528+0000 7fba35800700 1 --2- 172.21.15.47:0/3147392302 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fba301078b0 0x7fba30107cd0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fba200098a0 tx=0x7fba20009bb0 comp rx=0 tx=0).stop 2024-03-21T10:12:29.592 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:12:28.529+0000 7fba35800700 1 -- 172.21.15.47:0/3147392302 shutdown_connections 2024-03-21T10:12:29.592 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:12:28.529+0000 7fba35800700 1 --2- 172.21.15.47:0/3147392302 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fba301078b0 0x7fba30107cd0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:12:29.592 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:12:28.529+0000 7fba35800700 1 -- 172.21.15.47:0/3147392302 >> 172.21.15.47:0/3147392302 conn(0x7fba30102f50 msgr2=0x7fba30105370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:12:29.592 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:12:28.529+0000 7fba35800700 1 -- 172.21.15.47:0/3147392302 shutdown_connections 2024-03-21T10:12:29.592 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:12:28.529+0000 7fba35800700 1 -- 172.21.15.47:0/3147392302 wait complete. 2024-03-21T10:12:29.592 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:12:28.530+0000 7fba35800700 1 Processor -- start 2024-03-21T10:12:29.592 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:12:28.530+0000 7fba35800700 1 -- start start 2024-03-21T10:12:29.592 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:12:28.530+0000 7fba35800700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fba301078b0 0x7fba3019d9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:12:29.592 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:12:28.530+0000 7fba35800700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba3010be10 con 0x7fba301078b0 2024-03-21T10:12:29.592 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:12:28.531+0000 7fba2ea00700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fba301078b0 0x7fba3019d9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:12:29.593 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:12:28.531+0000 7fba2ea00700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fba301078b0 0x7fba3019d9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:39420/0 (socket says 172.21.15.47:39420) 2024-03-21T10:12:29.593 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:12:28.531+0000 7fba2ea00700 1 -- 172.21.15.47:0/1918229872 learned_addr learned my addr 172.21.15.47:0/1918229872 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:12:29.593 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:12:28.531+0000 7fba2ea00700 1 -- 172.21.15.47:0/1918229872 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba3019def0 con 0x7fba301078b0 2024-03-21T10:12:29.593 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:12:28.531+0000 7fba2ea00700 1 --2- 172.21.15.47:0/1918229872 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fba301078b0 0x7fba3019d9b0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fba20011710 tx=0x7fba20011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:12:29.593 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:12:28.531+0000 7fba27400700 1 -- 172.21.15.47:0/1918229872 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fba200039c0 con 0x7fba301078b0 2024-03-21T10:12:29.593 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:12:28.531+0000 7fba27400700 1 -- 172.21.15.47:0/1918229872 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fba20003b60 con 0x7fba301078b0 2024-03-21T10:12:29.593 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:12:28.532+0000 7fba35800700 1 -- 172.21.15.47:0/1918229872 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fba3019e180 con 0x7fba301078b0 2024-03-21T10:12:29.593 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:12:28.532+0000 7fba27400700 1 -- 172.21.15.47:0/1918229872 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fba2001a690 con 0x7fba301078b0 2024-03-21T10:12:29.593 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:12:28.532+0000 7fba35800700 1 -- 172.21.15.47:0/1918229872 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fba301a0e70 con 0x7fba301078b0 2024-03-21T10:12:29.593 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:12:28.533+0000 7fba35800700 1 -- 172.21.15.47:0/1918229872 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fba300637c0 con 0x7fba301078b0 2024-03-21T10:12:29.593 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:12:28.533+0000 7fba27400700 1 -- 172.21.15.47:0/1918229872 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 12) v1 ==== 46307+0+0 (secure 0 0 0) 0x7fba20024430 con 0x7fba301078b0 2024-03-21T10:12:29.593 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:12:28.534+0000 7fba27400700 1 --2- 172.21.15.47:0/1918229872 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7fba14039180 0x7fba1403b640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:12:29.593 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:12:28.534+0000 7fba27400700 1 -- 172.21.15.47:0/1918229872 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fba2004d9f0 con 0x7fba301078b0 2024-03-21T10:12:29.593 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:12:28.534+0000 7fba27e00700 1 --2- 172.21.15.47:0/1918229872 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7fba14039180 0x7fba1403b640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:12:29.593 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:12:28.534+0000 7fba27e00700 1 --2- 172.21.15.47:0/1918229872 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7fba14039180 0x7fba1403b640 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fba18006fd0 tx=0x7fba18006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:12:29.594 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:12:28.539+0000 7fba27400700 1 -- 172.21.15.47:0/1918229872 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fba20024a70 con 0x7fba301078b0 2024-03-21T10:12:29.594 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:12:28.777+0000 7fba35800700 1 -- 172.21.15.47:0/1918229872 --> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] -- mgr_command(tid 0: {"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}) v1 -- 0x7fba301040c0 con 0x7fba14039180 2024-03-21T10:12:29.594 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:12:28.900+0000 7fba27400700 1 -- 172.21.15.47:0/1918229872 <== mgr.14152 v2:172.21.15.47:6800/2823885994 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+32 (secure 0 0 0) 0x7fba301040c0 con 0x7fba14039180 2024-03-21T10:12:29.594 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:12:28.903+0000 7fba35800700 1 -- 172.21.15.47:0/1918229872 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7fba14039180 msgr2=0x7fba1403b640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:12:29.594 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:12:28.903+0000 7fba35800700 1 --2- 172.21.15.47:0/1918229872 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7fba14039180 0x7fba1403b640 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fba18006fd0 tx=0x7fba18006e40 comp rx=0 tx=0).stop 2024-03-21T10:12:29.594 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:12:28.903+0000 7fba35800700 1 -- 172.21.15.47:0/1918229872 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fba301078b0 msgr2=0x7fba3019d9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:12:29.594 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:12:28.903+0000 7fba35800700 1 --2- 172.21.15.47:0/1918229872 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fba301078b0 0x7fba3019d9b0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fba20011710 tx=0x7fba20011ad0 comp rx=0 tx=0).stop 2024-03-21T10:12:29.594 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:12:28.904+0000 7fba35800700 1 -- 172.21.15.47:0/1918229872 shutdown_connections 2024-03-21T10:12:29.594 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:12:28.904+0000 7fba35800700 1 --2- 172.21.15.47:0/1918229872 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7fba14039180 0x7fba1403b640 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:12:29.594 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:12:28.904+0000 7fba35800700 1 --2- 172.21.15.47:0/1918229872 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fba301078b0 0x7fba3019d9b0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:12:29.594 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:12:28.904+0000 7fba35800700 1 -- 172.21.15.47:0/1918229872 >> 172.21.15.47:0/1918229872 conn(0x7fba30102f50 msgr2=0x7fba301039b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:12:29.594 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:12:28.904+0000 7fba35800700 1 -- 172.21.15.47:0/1918229872 shutdown_connections 2024-03-21T10:12:29.594 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:12:28.905+0000 7fba35800700 1 -- 172.21.15.47:0/1918229872 wait complete. 2024-03-21T10:12:29.594 INFO:teuthology.orchestra.run.smithi047.stdout:Creating initial admin user... 2024-03-21T10:12:30.181 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:12:29 smithi047 ceph-mon[31547]: from='client.14164 -' entity='client.admin' cmd=[{"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:12:30.182 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:12:29 smithi047 ceph-mon[31547]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:12:31.477 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout {"username": "admin", "password": "$2b$12$fAnJzLhxi2/mFeZ3mRp6sOOi2riA4trrs9rbpfxqD9XvRLljHZpRK", "roles": ["administrator"], "name": null, "email": null, "lastUpdate": 1711015950, "enabled": true, "pwdExpirationDate": null, "pwdUpdateRequired": true} 2024-03-21T10:12:31.478 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:12:30.180+0000 7fbec1000700 1 Processor -- start 2024-03-21T10:12:31.478 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:12:30.180+0000 7fbec1000700 1 -- start start 2024-03-21T10:12:31.478 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:12:30.181+0000 7fbec1000700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbebc07ce10 0x7fbebc07b2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:12:31.478 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:12:30.181+0000 7fbec1000700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbebc07b800 con 0x7fbebc07ce10 2024-03-21T10:12:31.478 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:12:30.181+0000 7fbeb9e00700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbebc07ce10 0x7fbebc07b2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:12:31.478 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:12:30.181+0000 7fbeb9e00700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbebc07ce10 0x7fbebc07b2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:39432/0 (socket says 172.21.15.47:39432) 2024-03-21T10:12:31.478 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:12:30.181+0000 7fbeb9e00700 1 -- 172.21.15.47:0/2458917646 learned_addr learned my addr 172.21.15.47:0/2458917646 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:12:31.478 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:12:30.182+0000 7fbeb9e00700 1 -- 172.21.15.47:0/2458917646 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbebc07b980 con 0x7fbebc07ce10 2024-03-21T10:12:31.478 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:12:30.182+0000 7fbeb9e00700 1 --2- 172.21.15.47:0/2458917646 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbebc07ce10 0x7fbebc07b2c0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fbeac0098a0 tx=0x7fbeac009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dedd19250fcaea6f server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:12:31.478 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:12:30.182+0000 7fbeb8a00700 1 -- 172.21.15.47:0/2458917646 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fbeac004160 con 0x7fbebc07ce10 2024-03-21T10:12:31.478 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:12:30.183+0000 7fbeb8a00700 1 -- 172.21.15.47:0/2458917646 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fbeac004300 con 0x7fbebc07ce10 2024-03-21T10:12:31.479 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:12:30.183+0000 7fbeb8a00700 1 -- 172.21.15.47:0/2458917646 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fbeac011a50 con 0x7fbebc07ce10 2024-03-21T10:12:31.479 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:12:30.183+0000 7fbec1000700 1 -- 172.21.15.47:0/2458917646 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbebc07ce10 msgr2=0x7fbebc07b2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:12:31.479 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:12:30.183+0000 7fbec1000700 1 --2- 172.21.15.47:0/2458917646 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbebc07ce10 0x7fbebc07b2c0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fbeac0098a0 tx=0x7fbeac009bb0 comp rx=0 tx=0).stop 2024-03-21T10:12:31.479 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:12:30.183+0000 7fbec1000700 1 -- 172.21.15.47:0/2458917646 shutdown_connections 2024-03-21T10:12:31.479 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:12:30.183+0000 7fbec1000700 1 --2- 172.21.15.47:0/2458917646 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbebc07ce10 0x7fbebc07b2c0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:12:31.479 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:12:30.183+0000 7fbec1000700 1 -- 172.21.15.47:0/2458917646 >> 172.21.15.47:0/2458917646 conn(0x7fbebc102f50 msgr2=0x7fbebc105370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:12:31.479 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:12:30.184+0000 7fbec1000700 1 -- 172.21.15.47:0/2458917646 shutdown_connections 2024-03-21T10:12:31.479 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:12:30.184+0000 7fbec1000700 1 -- 172.21.15.47:0/2458917646 wait complete. 2024-03-21T10:12:31.479 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:12:30.185+0000 7fbec1000700 1 Processor -- start 2024-03-21T10:12:31.479 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:12:30.185+0000 7fbec1000700 1 -- start start 2024-03-21T10:12:31.479 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:12:30.185+0000 7fbec1000700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbebc07ce10 0x7fbebc19d9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:12:31.479 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:12:30.185+0000 7fbec1000700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbebc10de10 con 0x7fbebc07ce10 2024-03-21T10:12:31.479 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:12:30.185+0000 7fbeb9e00700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbebc07ce10 0x7fbebc19d9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:12:31.479 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:12:30.185+0000 7fbeb9e00700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbebc07ce10 0x7fbebc19d9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:39434/0 (socket says 172.21.15.47:39434) 2024-03-21T10:12:31.479 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:12:30.185+0000 7fbeb9e00700 1 -- 172.21.15.47:0/2615462862 learned_addr learned my addr 172.21.15.47:0/2615462862 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:12:31.480 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:12:30.186+0000 7fbeb9e00700 1 -- 172.21.15.47:0/2615462862 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbebc19dee0 con 0x7fbebc07ce10 2024-03-21T10:12:31.480 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:12:30.186+0000 7fbeb9e00700 1 --2- 172.21.15.47:0/2615462862 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbebc07ce10 0x7fbebc19d9a0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fbeac009130 tx=0x7fbeac004620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:12:31.480 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:12:30.186+0000 7fbeb2a00700 1 -- 172.21.15.47:0/2615462862 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fbeac003af0 con 0x7fbebc07ce10 2024-03-21T10:12:31.480 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:12:30.186+0000 7fbeb2a00700 1 -- 172.21.15.47:0/2615462862 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fbeac01a8e0 con 0x7fbebc07ce10 2024-03-21T10:12:31.480 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:12:30.186+0000 7fbeb2a00700 1 -- 172.21.15.47:0/2615462862 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fbeac0248b0 con 0x7fbebc07ce10 2024-03-21T10:12:31.480 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:12:30.186+0000 7fbec1000700 1 -- 172.21.15.47:0/2615462862 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbebc19e170 con 0x7fbebc07ce10 2024-03-21T10:12:31.480 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:12:30.187+0000 7fbec1000700 1 -- 172.21.15.47:0/2615462862 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbebc1a0e60 con 0x7fbebc07ce10 2024-03-21T10:12:31.480 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:12:30.188+0000 7fbeb2a00700 1 -- 172.21.15.47:0/2615462862 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 12) v1 ==== 46307+0+0 (secure 0 0 0) 0x7fbeac022020 con 0x7fbebc07ce10 2024-03-21T10:12:31.480 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:12:30.188+0000 7fbec1000700 1 -- 172.21.15.47:0/2615462862 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbebc0637c0 con 0x7fbebc07ce10 2024-03-21T10:12:31.480 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:12:30.188+0000 7fbeb2a00700 1 --2- 172.21.15.47:0/2615462862 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7fbea0039280 0x7fbea003b740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:12:31.480 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:12:30.189+0000 7fbeb2a00700 1 -- 172.21.15.47:0/2615462862 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fbeac04d680 con 0x7fbebc07ce10 2024-03-21T10:12:31.480 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:12:30.189+0000 7fbeb9400700 1 --2- 172.21.15.47:0/2615462862 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7fbea0039280 0x7fbea003b740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:12:31.480 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:12:30.190+0000 7fbeb9400700 1 --2- 172.21.15.47:0/2615462862 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7fbea0039280 0x7fbea003b740 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fbea8006fd0 tx=0x7fbea8006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:12:31.480 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:12:30.194+0000 7fbeb2a00700 1 -- 172.21.15.47:0/2615462862 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fbeac00edf0 con 0x7fbebc07ce10 2024-03-21T10:12:31.480 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:12:30.440+0000 7fbec1000700 1 -- 172.21.15.47:0/2615462862 --> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] -- mgr_command(tid 0: {"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}) v1 -- 0x7fbebc1a10f0 con 0x7fbea0039280 2024-03-21T10:12:31.481 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:12:30.711+0000 7fbeb2a00700 1 -- 172.21.15.47:0/2615462862 <== mgr.14152 v2:172.21.15.47:6800/2823885994 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+252 (secure 0 0 0) 0x7fbebc1a10f0 con 0x7fbea0039280 2024-03-21T10:12:31.481 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:12:30.715+0000 7fbec1000700 1 -- 172.21.15.47:0/2615462862 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7fbea0039280 msgr2=0x7fbea003b740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:12:31.481 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:12:30.715+0000 7fbec1000700 1 --2- 172.21.15.47:0/2615462862 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7fbea0039280 0x7fbea003b740 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fbea8006fd0 tx=0x7fbea8006e40 comp rx=0 tx=0).stop 2024-03-21T10:12:31.481 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:12:30.715+0000 7fbec1000700 1 -- 172.21.15.47:0/2615462862 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbebc07ce10 msgr2=0x7fbebc19d9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:12:31.481 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:12:30.715+0000 7fbec1000700 1 --2- 172.21.15.47:0/2615462862 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbebc07ce10 0x7fbebc19d9a0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fbeac009130 tx=0x7fbeac004620 comp rx=0 tx=0).stop 2024-03-21T10:12:31.481 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:12:30.715+0000 7fbec1000700 1 -- 172.21.15.47:0/2615462862 shutdown_connections 2024-03-21T10:12:31.481 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:12:30.715+0000 7fbec1000700 1 --2- 172.21.15.47:0/2615462862 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7fbea0039280 0x7fbea003b740 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:12:31.481 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:12:30.715+0000 7fbec1000700 1 --2- 172.21.15.47:0/2615462862 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbebc07ce10 0x7fbebc19d9a0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:12:31.481 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:12:30.715+0000 7fbec1000700 1 -- 172.21.15.47:0/2615462862 >> 172.21.15.47:0/2615462862 conn(0x7fbebc102f50 msgr2=0x7fbebc103a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:12:31.481 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:12:30.715+0000 7fbec1000700 1 -- 172.21.15.47:0/2615462862 shutdown_connections 2024-03-21T10:12:31.481 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:12:30.715+0000 7fbec1000700 1 -- 172.21.15.47:0/2615462862 wait complete. 2024-03-21T10:12:31.481 INFO:teuthology.orchestra.run.smithi047.stdout:Fetching dashboard port number... 2024-03-21T10:12:31.713 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:12:31 smithi047 ceph-mon[31547]: from='client.14166 -' entity='client.admin' cmd=[{"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:12:31.713 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:12:31 smithi047 ceph-mon[31547]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:12:31.713 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:12:31 smithi047 ceph-mon[31547]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:12:32.963 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:12:32 smithi047 ceph-mon[31547]: from='client.? 172.21.15.47:0/2382940453' entity='client.admin' cmd=[{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]: dispatch 2024-03-21T10:12:33.471 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout 8443 2024-03-21T10:12:33.472 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:12:32.090+0000 7f7d56400700 1 Processor -- start 2024-03-21T10:12:33.472 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:12:32.090+0000 7f7d56400700 1 -- start start 2024-03-21T10:12:33.472 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:12:32.090+0000 7f7d56400700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7d5007ce10 0x7f7d5007b2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:12:33.472 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:12:32.090+0000 7f7d56400700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d5007b800 con 0x7f7d5007ce10 2024-03-21T10:12:33.472 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:12:32.091+0000 7f7d4f400700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7d5007ce10 0x7f7d5007b2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:12:33.472 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:12:32.091+0000 7f7d4f400700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7d5007ce10 0x7f7d5007b2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:60244/0 (socket says 172.21.15.47:60244) 2024-03-21T10:12:33.472 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:12:32.091+0000 7f7d4f400700 1 -- 172.21.15.47:0/1116800060 learned_addr learned my addr 172.21.15.47:0/1116800060 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:12:33.472 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:12:32.091+0000 7f7d4f400700 1 -- 172.21.15.47:0/1116800060 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d5007b980 con 0x7f7d5007ce10 2024-03-21T10:12:33.472 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:12:32.092+0000 7f7d4f400700 1 --2- 172.21.15.47:0/1116800060 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7d5007ce10 0x7f7d5007b2c0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f7d400098a0 tx=0x7f7d40009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4fcafb2bef367dd server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:12:33.472 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:12:32.092+0000 7f7d4e000700 1 -- 172.21.15.47:0/1116800060 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f7d40004160 con 0x7f7d5007ce10 2024-03-21T10:12:33.472 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:12:32.092+0000 7f7d4e000700 1 -- 172.21.15.47:0/1116800060 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f7d40004300 con 0x7f7d5007ce10 2024-03-21T10:12:33.473 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:12:32.092+0000 7f7d4e000700 1 -- 172.21.15.47:0/1116800060 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f7d40011a50 con 0x7f7d5007ce10 2024-03-21T10:12:33.473 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:12:32.093+0000 7f7d56400700 1 -- 172.21.15.47:0/1116800060 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7d5007ce10 msgr2=0x7f7d5007b2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:12:33.473 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:12:32.093+0000 7f7d56400700 1 --2- 172.21.15.47:0/1116800060 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7d5007ce10 0x7f7d5007b2c0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f7d400098a0 tx=0x7f7d40009bb0 comp rx=0 tx=0).stop 2024-03-21T10:12:33.473 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:12:32.093+0000 7f7d56400700 1 -- 172.21.15.47:0/1116800060 shutdown_connections 2024-03-21T10:12:33.473 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:12:32.093+0000 7f7d56400700 1 --2- 172.21.15.47:0/1116800060 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7d5007ce10 0x7f7d5007b2c0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:12:33.473 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:12:32.093+0000 7f7d56400700 1 -- 172.21.15.47:0/1116800060 >> 172.21.15.47:0/1116800060 conn(0x7f7d50102f50 msgr2=0x7f7d50105370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:12:33.473 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:12:32.093+0000 7f7d56400700 1 -- 172.21.15.47:0/1116800060 shutdown_connections 2024-03-21T10:12:33.473 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:12:32.094+0000 7f7d56400700 1 -- 172.21.15.47:0/1116800060 wait complete. 2024-03-21T10:12:33.473 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:12:32.095+0000 7f7d56400700 1 Processor -- start 2024-03-21T10:12:33.473 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:12:32.095+0000 7f7d56400700 1 -- start start 2024-03-21T10:12:33.473 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:12:32.095+0000 7f7d56400700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7d5007ce10 0x7f7d5019d920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:12:33.473 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:12:32.095+0000 7f7d56400700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d5010ddf0 con 0x7f7d5007ce10 2024-03-21T10:12:33.473 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:12:32.095+0000 7f7d4f400700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7d5007ce10 0x7f7d5019d920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:12:33.474 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:12:32.095+0000 7f7d4f400700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7d5007ce10 0x7f7d5019d920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:60260/0 (socket says 172.21.15.47:60260) 2024-03-21T10:12:33.474 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:12:32.095+0000 7f7d4f400700 1 -- 172.21.15.47:0/2382940453 learned_addr learned my addr 172.21.15.47:0/2382940453 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:12:33.474 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:12:32.096+0000 7f7d4f400700 1 -- 172.21.15.47:0/2382940453 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d5019de60 con 0x7f7d5007ce10 2024-03-21T10:12:33.474 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:12:32.096+0000 7f7d4f400700 1 --2- 172.21.15.47:0/2382940453 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7d5007ce10 0x7f7d5019d920 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f7d40006b20 tx=0x7f7d40011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:12:33.474 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:12:32.096+0000 7f7d47e00700 1 -- 172.21.15.47:0/2382940453 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f7d4001a430 con 0x7f7d5007ce10 2024-03-21T10:12:33.474 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:12:32.096+0000 7f7d47e00700 1 -- 172.21.15.47:0/2382940453 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f7d4001aa80 con 0x7f7d5007ce10 2024-03-21T10:12:33.474 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:12:32.096+0000 7f7d47e00700 1 -- 172.21.15.47:0/2382940453 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f7d40024a20 con 0x7f7d5007ce10 2024-03-21T10:12:33.474 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:12:32.096+0000 7f7d56400700 1 -- 172.21.15.47:0/2382940453 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7d5019e0f0 con 0x7f7d5007ce10 2024-03-21T10:12:33.474 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:12:32.097+0000 7f7d56400700 1 -- 172.21.15.47:0/2382940453 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7d501a0de0 con 0x7f7d5007ce10 2024-03-21T10:12:33.474 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:12:32.098+0000 7f7d47e00700 1 -- 172.21.15.47:0/2382940453 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 12) v1 ==== 46307+0+0 (secure 0 0 0) 0x7f7d40022020 con 0x7f7d5007ce10 2024-03-21T10:12:33.474 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:12:32.098+0000 7f7d56400700 1 -- 172.21.15.47:0/2382940453 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7d500637c0 con 0x7f7d5007ce10 2024-03-21T10:12:33.474 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:12:32.098+0000 7f7d47e00700 1 --2- 172.21.15.47:0/2382940453 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f7d34039230 0x7f7d3403b6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:12:33.475 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:12:32.098+0000 7f7d47e00700 1 -- 172.21.15.47:0/2382940453 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f7d4004d9e0 con 0x7f7d5007ce10 2024-03-21T10:12:33.475 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:12:32.099+0000 7f7d4ea00700 1 --2- 172.21.15.47:0/2382940453 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f7d34039230 0x7f7d3403b6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:12:33.475 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:12:32.100+0000 7f7d4ea00700 1 --2- 172.21.15.47:0/2382940453 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f7d34039230 0x7f7d3403b6f0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f7d3c006fd0 tx=0x7f7d3c006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:12:33.475 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:12:32.104+0000 7f7d47e00700 1 -- 172.21.15.47:0/2382940453 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f7d4002c3a0 con 0x7f7d5007ce10 2024-03-21T10:12:33.475 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:12:32.339+0000 7f7d56400700 1 -- 172.21.15.47:0/2382940453 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"} v 0) v1 -- 0x7f7d501a1390 con 0x7f7d5007ce10 2024-03-21T10:12:33.475 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:12:32.340+0000 7f7d47e00700 1 -- 172.21.15.47:0/2382940453 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]=0 v8) v1 ==== 112+0+5 (secure 0 0 0) 0x7f7d400094f0 con 0x7f7d5007ce10 2024-03-21T10:12:33.475 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:12:32.343+0000 7f7d56400700 1 -- 172.21.15.47:0/2382940453 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f7d34039230 msgr2=0x7f7d3403b6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:12:33.475 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:12:32.343+0000 7f7d56400700 1 --2- 172.21.15.47:0/2382940453 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f7d34039230 0x7f7d3403b6f0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f7d3c006fd0 tx=0x7f7d3c006e40 comp rx=0 tx=0).stop 2024-03-21T10:12:33.475 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:12:32.343+0000 7f7d56400700 1 -- 172.21.15.47:0/2382940453 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7d5007ce10 msgr2=0x7f7d5019d920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:12:33.475 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:12:32.343+0000 7f7d56400700 1 --2- 172.21.15.47:0/2382940453 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7d5007ce10 0x7f7d5019d920 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f7d40006b20 tx=0x7f7d40011ad0 comp rx=0 tx=0).stop 2024-03-21T10:12:33.475 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:12:32.343+0000 7f7d56400700 1 -- 172.21.15.47:0/2382940453 shutdown_connections 2024-03-21T10:12:33.475 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:12:32.343+0000 7f7d56400700 1 --2- 172.21.15.47:0/2382940453 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f7d34039230 0x7f7d3403b6f0 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:12:33.475 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:12:32.343+0000 7f7d56400700 1 --2- 172.21.15.47:0/2382940453 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7d5007ce10 0x7f7d5019d920 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:12:33.476 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:12:32.343+0000 7f7d56400700 1 -- 172.21.15.47:0/2382940453 >> 172.21.15.47:0/2382940453 conn(0x7f7d50102f50 msgr2=0x7f7d501039c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:12:33.476 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:12:32.344+0000 7f7d56400700 1 -- 172.21.15.47:0/2382940453 shutdown_connections 2024-03-21T10:12:33.476 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:12:32.344+0000 7f7d56400700 1 -- 172.21.15.47:0/2382940453 wait complete. 2024-03-21T10:12:33.502 INFO:teuthology.orchestra.run.smithi047.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-03-21T10:12:33.502 INFO:teuthology.orchestra.run.smithi047.stdout:systemctl: stdout disabled 2024-03-21T10:12:33.521 INFO:teuthology.orchestra.run.smithi047.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-03-21T10:12:33.521 INFO:teuthology.orchestra.run.smithi047.stdout:systemctl: stdout inactive 2024-03-21T10:12:33.522 INFO:teuthology.orchestra.run.smithi047.stdout:firewalld.service is not enabled 2024-03-21T10:12:33.522 INFO:teuthology.orchestra.run.smithi047.stdout:Not possible to open ports <[8443]>. firewalld.service is not available 2024-03-21T10:12:33.526 INFO:teuthology.orchestra.run.smithi047.stdout:Ceph Dashboard is now available at: 2024-03-21T10:12:33.526 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-21T10:12:33.526 INFO:teuthology.orchestra.run.smithi047.stdout: URL: https://smithi047.front.sepia.ceph.com:8443/ 2024-03-21T10:12:33.526 INFO:teuthology.orchestra.run.smithi047.stdout: User: admin 2024-03-21T10:12:33.526 INFO:teuthology.orchestra.run.smithi047.stdout: Password: 2bnsy51xr5 2024-03-21T10:12:33.526 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-21T10:12:33.527 INFO:teuthology.orchestra.run.smithi047.stdout:Saving cluster configuration to /var/lib/ceph/3056d51a-e76b-11ee-95cd-87774f69a715/config directory 2024-03-21T10:12:33.528 INFO:teuthology.orchestra.run.smithi047.stdout:Enabling autotune for osd_memory_target 2024-03-21T10:12:34.118 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:12:33 smithi047 ceph-mon[31547]: mgrmap e13: x(active, since 6s) 2024-03-21T10:12:35.473 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:12:34.524+0000 7f0e64000700 1 Processor -- start 2024-03-21T10:12:35.473 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:12:34.525+0000 7f0e64000700 1 -- start start 2024-03-21T10:12:35.473 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:12:34.525+0000 7f0e64000700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0e5c107890 0x7f0e5c107cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:12:35.474 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:12:34.525+0000 7f0e64000700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e5c108280 con 0x7f0e5c107890 2024-03-21T10:12:35.475 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:12:34.525+0000 7f0e61600700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0e5c107890 0x7f0e5c107cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:12:35.475 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:12:34.525+0000 7f0e61600700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0e5c107890 0x7f0e5c107cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:60270/0 (socket says 172.21.15.47:60270) 2024-03-21T10:12:35.475 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:12:34.526+0000 7f0e61600700 1 -- 172.21.15.47:0/452973865 learned_addr learned my addr 172.21.15.47:0/452973865 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:12:35.475 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:12:34.526+0000 7f0e61600700 1 -- 172.21.15.47:0/452973865 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e5c108a40 con 0x7f0e5c107890 2024-03-21T10:12:35.475 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:12:34.526+0000 7f0e61600700 1 --2- 172.21.15.47:0/452973865 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0e5c107890 0x7f0e5c107cb0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f0e50009e80 tx=0x7f0e5000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1577a999f0612c30 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:12:35.475 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:12:34.527+0000 7f0e5be00700 1 -- 172.21.15.47:0/452973865 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f0e50017070 con 0x7f0e5c107890 2024-03-21T10:12:35.475 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:12:34.527+0000 7f0e5be00700 1 -- 172.21.15.47:0/452973865 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f0e5000b8c0 con 0x7f0e5c107890 2024-03-21T10:12:35.475 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:12:34.527+0000 7f0e5be00700 1 -- 172.21.15.47:0/452973865 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f0e50004490 con 0x7f0e5c107890 2024-03-21T10:12:35.476 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:12:34.527+0000 7f0e64000700 1 -- 172.21.15.47:0/452973865 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0e5c107890 msgr2=0x7f0e5c107cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:12:35.476 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:12:34.527+0000 7f0e64000700 1 --2- 172.21.15.47:0/452973865 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0e5c107890 0x7f0e5c107cb0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f0e50009e80 tx=0x7f0e5000b330 comp rx=0 tx=0).stop 2024-03-21T10:12:35.476 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:12:34.528+0000 7f0e64000700 1 -- 172.21.15.47:0/452973865 shutdown_connections 2024-03-21T10:12:35.476 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:12:34.528+0000 7f0e64000700 1 --2- 172.21.15.47:0/452973865 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0e5c107890 0x7f0e5c107cb0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:12:35.477 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:12:34.528+0000 7f0e64000700 1 -- 172.21.15.47:0/452973865 >> 172.21.15.47:0/452973865 conn(0x7f0e5c102f50 msgr2=0x7f0e5c105370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:12:35.477 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:12:34.528+0000 7f0e64000700 1 -- 172.21.15.47:0/452973865 shutdown_connections 2024-03-21T10:12:35.477 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:12:34.528+0000 7f0e64000700 1 -- 172.21.15.47:0/452973865 wait complete. 2024-03-21T10:12:35.478 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:12:34.529+0000 7f0e64000700 1 Processor -- start 2024-03-21T10:12:35.478 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:12:34.529+0000 7f0e64000700 1 -- start start 2024-03-21T10:12:35.478 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:12:34.529+0000 7f0e64000700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0e5c107890 0x7f0e5c19d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:12:35.478 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:12:34.529+0000 7f0e64000700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e5c10bdf0 con 0x7f0e5c107890 2024-03-21T10:12:35.478 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:12:34.529+0000 7f0e61600700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0e5c107890 0x7f0e5c19d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:12:35.478 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:12:34.529+0000 7f0e61600700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0e5c107890 0x7f0e5c19d990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:60282/0 (socket says 172.21.15.47:60282) 2024-03-21T10:12:35.479 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:12:34.529+0000 7f0e61600700 1 -- 172.21.15.47:0/829756107 learned_addr learned my addr 172.21.15.47:0/829756107 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:12:35.479 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:12:34.530+0000 7f0e61600700 1 -- 172.21.15.47:0/829756107 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e5c19ded0 con 0x7f0e5c107890 2024-03-21T10:12:35.479 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:12:34.530+0000 7f0e61600700 1 --2- 172.21.15.47:0/829756107 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0e5c107890 0x7f0e5c19d990 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f0e50009e50 tx=0x7f0e50003d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:12:35.479 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:12:34.530+0000 7f0e5a000700 1 -- 172.21.15.47:0/829756107 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f0e50020020 con 0x7f0e5c107890 2024-03-21T10:12:35.479 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:12:34.530+0000 7f0e5a000700 1 -- 172.21.15.47:0/829756107 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f0e5000bba0 con 0x7f0e5c107890 2024-03-21T10:12:35.483 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:12:34.530+0000 7f0e5a000700 1 -- 172.21.15.47:0/829756107 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f0e50017040 con 0x7f0e5c107890 2024-03-21T10:12:35.483 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:12:34.530+0000 7f0e64000700 1 -- 172.21.15.47:0/829756107 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0e5c19e160 con 0x7f0e5c107890 2024-03-21T10:12:35.483 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:12:34.531+0000 7f0e64000700 1 -- 172.21.15.47:0/829756107 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0e5c1a0e50 con 0x7f0e5c107890 2024-03-21T10:12:35.483 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:12:34.532+0000 7f0e5a000700 1 -- 172.21.15.47:0/829756107 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 13) v1 ==== 46350+0+0 (secure 0 0 0) 0x7f0e50023020 con 0x7f0e5c107890 2024-03-21T10:12:35.483 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:12:34.532+0000 7f0e64000700 1 -- 172.21.15.47:0/829756107 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0e5c0637c0 con 0x7f0e5c107890 2024-03-21T10:12:35.483 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:12:34.532+0000 7f0e5a000700 1 --2- 172.21.15.47:0/829756107 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f0e44039280 0x7f0e4403b740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:12:35.483 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:12:34.532+0000 7f0e5a000700 1 -- 172.21.15.47:0/829756107 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f0e5000f050 con 0x7f0e5c107890 2024-03-21T10:12:35.483 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:12:34.533+0000 7f0e60c00700 1 --2- 172.21.15.47:0/829756107 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f0e44039280 0x7f0e4403b740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:12:35.483 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:12:34.533+0000 7f0e60c00700 1 --2- 172.21.15.47:0/829756107 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f0e44039280 0x7f0e4403b740 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f0e48006fd0 tx=0x7f0e48006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:12:35.483 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:12:34.538+0000 7f0e5a000700 1 -- 172.21.15.47:0/829756107 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f0e5002a020 con 0x7f0e5c107890 2024-03-21T10:12:35.483 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:12:34.763+0000 7f0e64000700 1 -- 172.21.15.47:0/829756107 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command([{prefix=config set, name=osd_memory_target_autotune}] v 0) v1 -- 0x7f0e5c1a1490 con 0x7f0e5c107890 2024-03-21T10:12:35.483 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:12:34.764+0000 7f0e5a000700 1 -- 172.21.15.47:0/829756107 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{prefix=config set, name=osd_memory_target_autotune}]=0 v8)=0 v8) v1 ==== 127+0+0 (secure 0 0 0) 0x7f0e5002a200 con 0x7f0e5c107890 2024-03-21T10:12:35.483 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:12:34.767+0000 7f0e64000700 1 -- 172.21.15.47:0/829756107 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f0e44039280 msgr2=0x7f0e4403b740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:12:35.484 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:12:34.767+0000 7f0e64000700 1 --2- 172.21.15.47:0/829756107 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f0e44039280 0x7f0e4403b740 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f0e48006fd0 tx=0x7f0e48006e40 comp rx=0 tx=0).stop 2024-03-21T10:12:35.484 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:12:34.767+0000 7f0e64000700 1 -- 172.21.15.47:0/829756107 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0e5c107890 msgr2=0x7f0e5c19d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:12:35.484 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:12:34.767+0000 7f0e64000700 1 --2- 172.21.15.47:0/829756107 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0e5c107890 0x7f0e5c19d990 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f0e50009e50 tx=0x7f0e50003d00 comp rx=0 tx=0).stop 2024-03-21T10:12:35.484 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:12:34.767+0000 7f0e64000700 1 -- 172.21.15.47:0/829756107 shutdown_connections 2024-03-21T10:12:35.484 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:12:34.767+0000 7f0e64000700 1 --2- 172.21.15.47:0/829756107 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f0e44039280 0x7f0e4403b740 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:12:35.484 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:12:34.767+0000 7f0e64000700 1 --2- 172.21.15.47:0/829756107 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0e5c107890 0x7f0e5c19d990 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:12:35.484 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:12:34.767+0000 7f0e64000700 1 -- 172.21.15.47:0/829756107 >> 172.21.15.47:0/829756107 conn(0x7f0e5c102f50 msgr2=0x7f0e5c103a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:12:35.484 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:12:34.767+0000 7f0e64000700 1 -- 172.21.15.47:0/829756107 shutdown_connections 2024-03-21T10:12:35.484 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:12:34.768+0000 7f0e64000700 1 -- 172.21.15.47:0/829756107 wait complete. 2024-03-21T10:12:37.103 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:12:36.044+0000 7fd19b400700 1 Processor -- start 2024-03-21T10:12:37.103 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:12:36.044+0000 7fd19b400700 1 -- start start 2024-03-21T10:12:37.104 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:12:36.044+0000 7fd19b400700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd19410e040 0x7fd19410e460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:12:37.104 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:12:36.044+0000 7fd19b400700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd19410e9a0 con 0x7fd19410e040 2024-03-21T10:12:37.104 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:12:36.045+0000 7fd198a00700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd19410e040 0x7fd19410e460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:12:37.105 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:12:36.045+0000 7fd198a00700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd19410e040 0x7fd19410e460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:60298/0 (socket says 172.21.15.47:60298) 2024-03-21T10:12:37.105 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:12:36.045+0000 7fd198a00700 1 -- 172.21.15.47:0/3541483293 learned_addr learned my addr 172.21.15.47:0/3541483293 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:12:37.105 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:12:36.045+0000 7fd198a00700 1 -- 172.21.15.47:0/3541483293 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd19410f070 con 0x7fd19410e040 2024-03-21T10:12:37.106 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:12:36.046+0000 7fd198a00700 1 --2- 172.21.15.47:0/3541483293 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd19410e040 0x7fd19410e460 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fd1880098a0 tx=0x7fd188009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6134175ec105c8c4 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:12:37.106 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:12:36.046+0000 7fd193400700 1 -- 172.21.15.47:0/3541483293 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd188004160 con 0x7fd19410e040 2024-03-21T10:12:37.106 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:12:36.046+0000 7fd193400700 1 -- 172.21.15.47:0/3541483293 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fd188004300 con 0x7fd19410e040 2024-03-21T10:12:37.106 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:12:36.046+0000 7fd193400700 1 -- 172.21.15.47:0/3541483293 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd188011a50 con 0x7fd19410e040 2024-03-21T10:12:37.106 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:12:36.047+0000 7fd19b400700 1 -- 172.21.15.47:0/3541483293 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd19410e040 msgr2=0x7fd19410e460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:12:37.107 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:12:36.047+0000 7fd19b400700 1 --2- 172.21.15.47:0/3541483293 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd19410e040 0x7fd19410e460 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fd1880098a0 tx=0x7fd188009bb0 comp rx=0 tx=0).stop 2024-03-21T10:12:37.107 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:12:36.047+0000 7fd19b400700 1 -- 172.21.15.47:0/3541483293 shutdown_connections 2024-03-21T10:12:37.107 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:12:36.047+0000 7fd19b400700 1 --2- 172.21.15.47:0/3541483293 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd19410e040 0x7fd19410e460 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:12:37.107 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:12:36.047+0000 7fd19b400700 1 -- 172.21.15.47:0/3541483293 >> 172.21.15.47:0/3541483293 conn(0x7fd19407cf30 msgr2=0x7fd19407f350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:12:37.108 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:12:36.047+0000 7fd19b400700 1 -- 172.21.15.47:0/3541483293 shutdown_connections 2024-03-21T10:12:37.108 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:12:36.047+0000 7fd19b400700 1 -- 172.21.15.47:0/3541483293 wait complete. 2024-03-21T10:12:37.108 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:12:36.048+0000 7fd19b400700 1 Processor -- start 2024-03-21T10:12:37.108 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:12:36.048+0000 7fd19b400700 1 -- start start 2024-03-21T10:12:37.109 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:12:36.049+0000 7fd19b400700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd19410e040 0x7fd1941a1cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:12:37.109 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:12:36.049+0000 7fd19b400700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd194112330 con 0x7fd19410e040 2024-03-21T10:12:37.109 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:12:36.049+0000 7fd198a00700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd19410e040 0x7fd1941a1cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:12:37.109 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:12:36.049+0000 7fd198a00700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd19410e040 0x7fd1941a1cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:60310/0 (socket says 172.21.15.47:60310) 2024-03-21T10:12:37.109 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:12:36.049+0000 7fd198a00700 1 -- 172.21.15.47:0/54308736 learned_addr learned my addr 172.21.15.47:0/54308736 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:12:37.110 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:12:36.049+0000 7fd198a00700 1 -- 172.21.15.47:0/54308736 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1941a2230 con 0x7fd19410e040 2024-03-21T10:12:37.110 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:12:36.050+0000 7fd198a00700 1 --2- 172.21.15.47:0/54308736 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd19410e040 0x7fd1941a1cf0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fd188006b20 tx=0x7fd188011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:12:37.110 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:12:36.050+0000 7fd191600700 1 -- 172.21.15.47:0/54308736 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd1880039e0 con 0x7fd19410e040 2024-03-21T10:12:37.111 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:12:36.050+0000 7fd191600700 1 -- 172.21.15.47:0/54308736 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fd18801a8e0 con 0x7fd19410e040 2024-03-21T10:12:37.111 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:12:36.050+0000 7fd191600700 1 -- 172.21.15.47:0/54308736 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd1880247e0 con 0x7fd19410e040 2024-03-21T10:12:37.111 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:12:36.050+0000 7fd19b400700 1 -- 172.21.15.47:0/54308736 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd1941a24c0 con 0x7fd19410e040 2024-03-21T10:12:37.112 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:12:36.050+0000 7fd19b400700 1 -- 172.21.15.47:0/54308736 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd1941a51a0 con 0x7fd19410e040 2024-03-21T10:12:37.112 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:12:36.051+0000 7fd19b400700 1 -- 172.21.15.47:0/54308736 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd1940637c0 con 0x7fd19410e040 2024-03-21T10:12:37.112 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:12:36.052+0000 7fd191600700 1 -- 172.21.15.47:0/54308736 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 13) v1 ==== 46350+0+0 (secure 0 0 0) 0x7fd188022020 con 0x7fd19410e040 2024-03-21T10:12:37.112 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:12:36.052+0000 7fd191600700 1 --2- 172.21.15.47:0/54308736 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7fd17c039380 0x7fd17c03b840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:12:37.113 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:12:36.052+0000 7fd191600700 1 -- 172.21.15.47:0/54308736 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fd18804d7d0 con 0x7fd19410e040 2024-03-21T10:12:37.113 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:12:36.052+0000 7fd193e00700 1 --2- 172.21.15.47:0/54308736 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7fd17c039380 0x7fd17c03b840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:12:37.113 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:12:36.053+0000 7fd193e00700 1 --2- 172.21.15.47:0/54308736 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7fd17c039380 0x7fd17c03b840 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fd180006fd0 tx=0x7fd180006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:12:37.113 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:12:36.058+0000 7fd191600700 1 -- 172.21.15.47:0/54308736 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fd18802c3a0 con 0x7fd19410e040 2024-03-21T10:12:37.113 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:12:36.406+0000 7fd19b400700 1 -- 172.21.15.47:0/54308736 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command([{prefix=config-key set, key=mgr/dashboard/cluster/status}] v 0) v1 -- 0x7fd1941a5370 con 0x7fd19410e040 2024-03-21T10:12:37.113 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:12:36.410+0000 7fd191600700 1 -- 172.21.15.47:0/54308736 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{prefix=config-key set, key=mgr/dashboard/cluster/status}]=0 set mgr/dashboard/cluster/status v27)=0 set mgr/dashboard/cluster/status v27) v1 ==== 153+0+0 (secure 0 0 0) 0x7fd1880172b0 con 0x7fd19410e040 2024-03-21T10:12:37.114 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:12:36.413+0000 7fd19b400700 1 -- 172.21.15.47:0/54308736 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7fd17c039380 msgr2=0x7fd17c03b840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:12:37.115 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:12:36.413+0000 7fd19b400700 1 --2- 172.21.15.47:0/54308736 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7fd17c039380 0x7fd17c03b840 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fd180006fd0 tx=0x7fd180006e40 comp rx=0 tx=0).stop 2024-03-21T10:12:37.115 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:12:36.413+0000 7fd19b400700 1 -- 172.21.15.47:0/54308736 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd19410e040 msgr2=0x7fd1941a1cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:12:37.115 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:12:36.413+0000 7fd19b400700 1 --2- 172.21.15.47:0/54308736 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd19410e040 0x7fd1941a1cf0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fd188006b20 tx=0x7fd188011ad0 comp rx=0 tx=0).stop 2024-03-21T10:12:37.115 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:12:36.413+0000 7fd19b400700 1 -- 172.21.15.47:0/54308736 shutdown_connections 2024-03-21T10:12:37.115 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:12:36.413+0000 7fd19b400700 1 --2- 172.21.15.47:0/54308736 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7fd17c039380 0x7fd17c03b840 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:12:37.115 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:12:36.413+0000 7fd19b400700 1 --2- 172.21.15.47:0/54308736 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd19410e040 0x7fd1941a1cf0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:12:37.115 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:12:36.413+0000 7fd19b400700 1 -- 172.21.15.47:0/54308736 >> 172.21.15.47:0/54308736 conn(0x7fd19407cf30 msgr2=0x7fd19407d900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:12:37.116 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:12:36.413+0000 7fd19b400700 1 -- 172.21.15.47:0/54308736 shutdown_connections 2024-03-21T10:12:37.116 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-03-21T10:12:36.413+0000 7fd19b400700 1 -- 172.21.15.47:0/54308736 wait complete. 2024-03-21T10:12:37.116 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr set mgr/dashboard/cluster/status 2024-03-21T10:12:37.116 INFO:teuthology.orchestra.run.smithi047.stdout:You can access the Ceph CLI as following in case of multi-cluster or non-default config: 2024-03-21T10:12:37.116 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-21T10:12:37.116 INFO:teuthology.orchestra.run.smithi047.stdout: sudo /home/ubuntu/cephtest/cephadm shell --fsid 3056d51a-e76b-11ee-95cd-87774f69a715 -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring 2024-03-21T10:12:37.117 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-21T10:12:37.117 INFO:teuthology.orchestra.run.smithi047.stdout:Or, if you are only running a single cluster on this host: 2024-03-21T10:12:37.117 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-21T10:12:37.117 INFO:teuthology.orchestra.run.smithi047.stdout: sudo /home/ubuntu/cephtest/cephadm shell 2024-03-21T10:12:37.117 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-21T10:12:37.117 INFO:teuthology.orchestra.run.smithi047.stdout:Please consider enabling telemetry to help improve Ceph: 2024-03-21T10:12:37.117 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-21T10:12:37.118 INFO:teuthology.orchestra.run.smithi047.stdout: ceph telemetry on 2024-03-21T10:12:37.118 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-21T10:12:37.118 INFO:teuthology.orchestra.run.smithi047.stdout:For more information see: 2024-03-21T10:12:37.118 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-21T10:12:37.118 INFO:teuthology.orchestra.run.smithi047.stdout: https://docs.ceph.com/en/latest/mgr/telemetry/ 2024-03-21T10:12:37.118 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-21T10:12:37.118 INFO:teuthology.orchestra.run.smithi047.stdout:Bootstrap complete. 2024-03-21T10:12:37.242 INFO:tasks.cephadm:Fetching config... 2024-03-21T10:12:37.242 DEBUG:teuthology.orchestra.run.smithi047:> set -ex 2024-03-21T10:12:37.242 DEBUG:teuthology.orchestra.run.smithi047:> dd if=/etc/ceph/ceph.conf of=/dev/stdout 2024-03-21T10:12:37.295 INFO:tasks.cephadm:Fetching client.admin keyring... 2024-03-21T10:12:37.295 DEBUG:teuthology.orchestra.run.smithi047:> set -ex 2024-03-21T10:12:37.295 DEBUG:teuthology.orchestra.run.smithi047:> dd if=/etc/ceph/ceph.client.admin.keyring of=/dev/stdout 2024-03-21T10:12:37.335 INFO:tasks.cephadm:Fetching mon keyring... 2024-03-21T10:12:37.336 DEBUG:teuthology.orchestra.run.smithi047:> set -ex 2024-03-21T10:12:37.336 DEBUG:teuthology.orchestra.run.smithi047:> sudo dd if=/var/lib/ceph/3056d51a-e76b-11ee-95cd-87774f69a715/mon.a/keyring of=/dev/stdout 2024-03-21T10:12:37.471 INFO:tasks.cephadm:Fetching pub ssh key... 2024-03-21T10:12:37.472 DEBUG:teuthology.orchestra.run.smithi047:> set -ex 2024-03-21T10:12:37.472 DEBUG:teuthology.orchestra.run.smithi047:> dd if=/home/ubuntu/cephtest/ceph.pub of=/dev/stdout 2024-03-21T10:12:37.527 INFO:tasks.cephadm:Installing pub ssh key for root users... 2024-03-21T10:12:37.527 DEBUG:teuthology.orchestra.run.smithi047:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDP4DxDtnGzZr7AHeST1cVeSRy4amZI9R0nPxcpwwJ+yi161x2vaXk0bFVj4r4gDYkIyagkDCnVYuW2fu1CMxsTDvvJ/yzCnoeo9xDQ9+7A9GVD2USTEpXSetKbBuvipAAY+zdMGmgnayGjkv1lHkeSrK0NEkgbmfpkVCoh8LWtj95aha1g4MNJxFOW8U0x2+ADjbCCJ+6AM5+SZVr9yEAc8WtrUMHyU/OZCDbbdNn6BzFkPg78c96pLvBuNHhJ1Ld7MW9CQ6SkdQDFe6GnPezGKF8C+FjJkMiv7rkfm95E4Hp6W5fnGw8kkeFd98BdbyN1mYaGwyH3Nuwn5yhN2t4VBwWmxJo4WUyRVIiq8h+Mq2CVPRLFfZk/F/EqeZYaKZho+u/ycMmArdO/15+9sZEMrG0ltkF6t0GwiBd/gm8jRZ3cbJvYwzEaRpUf0SCxqvEovS62KZTP7xhSxCdkf+T0NbTXyEE8Nh/qy1d5qVg8SwvNTZI6plxxjp/VT/Ek2iU= ceph-3056d51a-e76b-11ee-95cd-87774f69a715' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-03-21T10:12:37.664 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:12:37 smithi047 ceph-mon[31547]: from='client.? 172.21.15.47:0/54308736' entity='client.admin' 2024-03-21T10:12:37.669 INFO:teuthology.orchestra.run.smithi047.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDP4DxDtnGzZr7AHeST1cVeSRy4amZI9R0nPxcpwwJ+yi161x2vaXk0bFVj4r4gDYkIyagkDCnVYuW2fu1CMxsTDvvJ/yzCnoeo9xDQ9+7A9GVD2USTEpXSetKbBuvipAAY+zdMGmgnayGjkv1lHkeSrK0NEkgbmfpkVCoh8LWtj95aha1g4MNJxFOW8U0x2+ADjbCCJ+6AM5+SZVr9yEAc8WtrUMHyU/OZCDbbdNn6BzFkPg78c96pLvBuNHhJ1Ld7MW9CQ6SkdQDFe6GnPezGKF8C+FjJkMiv7rkfm95E4Hp6W5fnGw8kkeFd98BdbyN1mYaGwyH3Nuwn5yhN2t4VBwWmxJo4WUyRVIiq8h+Mq2CVPRLFfZk/F/EqeZYaKZho+u/ycMmArdO/15+9sZEMrG0ltkF6t0GwiBd/gm8jRZ3cbJvYwzEaRpUf0SCxqvEovS62KZTP7xhSxCdkf+T0NbTXyEE8Nh/qy1d5qVg8SwvNTZI6plxxjp/VT/Ek2iU= ceph-3056d51a-e76b-11ee-95cd-87774f69a715 2024-03-21T10:12:37.716 DEBUG:teuthology.orchestra.run.smithi100:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDP4DxDtnGzZr7AHeST1cVeSRy4amZI9R0nPxcpwwJ+yi161x2vaXk0bFVj4r4gDYkIyagkDCnVYuW2fu1CMxsTDvvJ/yzCnoeo9xDQ9+7A9GVD2USTEpXSetKbBuvipAAY+zdMGmgnayGjkv1lHkeSrK0NEkgbmfpkVCoh8LWtj95aha1g4MNJxFOW8U0x2+ADjbCCJ+6AM5+SZVr9yEAc8WtrUMHyU/OZCDbbdNn6BzFkPg78c96pLvBuNHhJ1Ld7MW9CQ6SkdQDFe6GnPezGKF8C+FjJkMiv7rkfm95E4Hp6W5fnGw8kkeFd98BdbyN1mYaGwyH3Nuwn5yhN2t4VBwWmxJo4WUyRVIiq8h+Mq2CVPRLFfZk/F/EqeZYaKZho+u/ycMmArdO/15+9sZEMrG0ltkF6t0GwiBd/gm8jRZ3cbJvYwzEaRpUf0SCxqvEovS62KZTP7xhSxCdkf+T0NbTXyEE8Nh/qy1d5qVg8SwvNTZI6plxxjp/VT/Ek2iU= ceph-3056d51a-e76b-11ee-95cd-87774f69a715' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-03-21T10:12:37.848 INFO:teuthology.orchestra.run.smithi100.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDP4DxDtnGzZr7AHeST1cVeSRy4amZI9R0nPxcpwwJ+yi161x2vaXk0bFVj4r4gDYkIyagkDCnVYuW2fu1CMxsTDvvJ/yzCnoeo9xDQ9+7A9GVD2USTEpXSetKbBuvipAAY+zdMGmgnayGjkv1lHkeSrK0NEkgbmfpkVCoh8LWtj95aha1g4MNJxFOW8U0x2+ADjbCCJ+6AM5+SZVr9yEAc8WtrUMHyU/OZCDbbdNn6BzFkPg78c96pLvBuNHhJ1Ld7MW9CQ6SkdQDFe6GnPezGKF8C+FjJkMiv7rkfm95E4Hp6W5fnGw8kkeFd98BdbyN1mYaGwyH3Nuwn5yhN2t4VBwWmxJo4WUyRVIiq8h+Mq2CVPRLFfZk/F/EqeZYaKZho+u/ycMmArdO/15+9sZEMrG0ltkF6t0GwiBd/gm8jRZ3cbJvYwzEaRpUf0SCxqvEovS62KZTP7xhSxCdkf+T0NbTXyEE8Nh/qy1d5qVg8SwvNTZI6plxxjp/VT/Ek2iU= ceph-3056d51a-e76b-11ee-95cd-87774f69a715 2024-03-21T10:12:37.909 DEBUG:teuthology.orchestra.run.smithi157:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDP4DxDtnGzZr7AHeST1cVeSRy4amZI9R0nPxcpwwJ+yi161x2vaXk0bFVj4r4gDYkIyagkDCnVYuW2fu1CMxsTDvvJ/yzCnoeo9xDQ9+7A9GVD2USTEpXSetKbBuvipAAY+zdMGmgnayGjkv1lHkeSrK0NEkgbmfpkVCoh8LWtj95aha1g4MNJxFOW8U0x2+ADjbCCJ+6AM5+SZVr9yEAc8WtrUMHyU/OZCDbbdNn6BzFkPg78c96pLvBuNHhJ1Ld7MW9CQ6SkdQDFe6GnPezGKF8C+FjJkMiv7rkfm95E4Hp6W5fnGw8kkeFd98BdbyN1mYaGwyH3Nuwn5yhN2t4VBwWmxJo4WUyRVIiq8h+Mq2CVPRLFfZk/F/EqeZYaKZho+u/ycMmArdO/15+9sZEMrG0ltkF6t0GwiBd/gm8jRZ3cbJvYwzEaRpUf0SCxqvEovS62KZTP7xhSxCdkf+T0NbTXyEE8Nh/qy1d5qVg8SwvNTZI6plxxjp/VT/Ek2iU= ceph-3056d51a-e76b-11ee-95cd-87774f69a715' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-03-21T10:12:38.047 INFO:teuthology.orchestra.run.smithi157.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDP4DxDtnGzZr7AHeST1cVeSRy4amZI9R0nPxcpwwJ+yi161x2vaXk0bFVj4r4gDYkIyagkDCnVYuW2fu1CMxsTDvvJ/yzCnoeo9xDQ9+7A9GVD2USTEpXSetKbBuvipAAY+zdMGmgnayGjkv1lHkeSrK0NEkgbmfpkVCoh8LWtj95aha1g4MNJxFOW8U0x2+ADjbCCJ+6AM5+SZVr9yEAc8WtrUMHyU/OZCDbbdNn6BzFkPg78c96pLvBuNHhJ1Ld7MW9CQ6SkdQDFe6GnPezGKF8C+FjJkMiv7rkfm95E4Hp6W5fnGw8kkeFd98BdbyN1mYaGwyH3Nuwn5yhN2t4VBwWmxJo4WUyRVIiq8h+Mq2CVPRLFfZk/F/EqeZYaKZho+u/ycMmArdO/15+9sZEMrG0ltkF6t0GwiBd/gm8jRZ3cbJvYwzEaRpUf0SCxqvEovS62KZTP7xhSxCdkf+T0NbTXyEE8Nh/qy1d5qVg8SwvNTZI6plxxjp/VT/Ek2iU= ceph-3056d51a-e76b-11ee-95cd-87774f69a715 2024-03-21T10:12:38.116 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2edfee68ad7243bed422a049b6f5f58619300f11 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3056d51a-e76b-11ee-95cd-87774f69a715 -- ceph config set mgr mgr/cephadm/allow_ptrace true 2024-03-21T10:12:38.548 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/3056d51a-e76b-11ee-95cd-87774f69a715/mon.a/config 2024-03-21T10:12:41.035 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:12:41.033+0000 7f54a8e00700 1 -- 172.21.15.47:0/3288320262 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f54a4102650 msgr2=0x7f54a4104a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:12:41.035 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:12:41.033+0000 7f54a8e00700 1 --2- 172.21.15.47:0/3288320262 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f54a4102650 0x7f54a4104a40 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f5494009910 tx=0x7f5494009c20 comp rx=0 tx=0).stop 2024-03-21T10:12:41.035 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:12:41.034+0000 7f54a8e00700 1 -- 172.21.15.47:0/3288320262 shutdown_connections 2024-03-21T10:12:41.035 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:12:41.034+0000 7f54a8e00700 1 --2- 172.21.15.47:0/3288320262 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f54a4102650 0x7f54a4104a40 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:12:41.035 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:12:41.034+0000 7f54a8e00700 1 -- 172.21.15.47:0/3288320262 >> 172.21.15.47:0/3288320262 conn(0x7f54a40fc060 msgr2=0x7f54a40fe4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:12:41.035 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:12:41.034+0000 7f54a8e00700 1 -- 172.21.15.47:0/3288320262 shutdown_connections 2024-03-21T10:12:41.036 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:12:41.034+0000 7f54a8e00700 1 -- 172.21.15.47:0/3288320262 wait complete. 2024-03-21T10:12:41.036 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:12:41.035+0000 7f54a8e00700 1 Processor -- start 2024-03-21T10:12:41.036 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:12:41.035+0000 7f54a8e00700 1 -- start start 2024-03-21T10:12:41.037 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:12:41.035+0000 7f54a8e00700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f54a4102650 0x7f54a4197150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:12:41.037 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:12:41.035+0000 7f54a8e00700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54a4107aa0 con 0x7f54a4102650 2024-03-21T10:12:41.037 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:12:41.036+0000 7f54a1e00700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f54a4102650 0x7f54a4197150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:12:41.037 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:12:41.036+0000 7f54a1e00700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f54a4102650 0x7f54a4197150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:60324/0 (socket says 172.21.15.47:60324) 2024-03-21T10:12:41.037 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:12:41.036+0000 7f54a1e00700 1 -- 172.21.15.47:0/685033640 learned_addr learned my addr 172.21.15.47:0/685033640 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:12:41.037 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:12:41.036+0000 7f54a1e00700 1 -- 172.21.15.47:0/685033640 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f54a4197690 con 0x7f54a4102650 2024-03-21T10:12:41.038 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:12:41.036+0000 7f54a1e00700 1 --2- 172.21.15.47:0/685033640 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f54a4102650 0x7f54a4197150 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f5494005190 tx=0x7f5494004f80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:12:41.038 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:12:41.037+0000 7f549aa00700 1 -- 172.21.15.47:0/685033640 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f5494027070 con 0x7f54a4102650 2024-03-21T10:12:41.038 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:12:41.037+0000 7f549aa00700 1 -- 172.21.15.47:0/685033640 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f5494022470 con 0x7f54a4102650 2024-03-21T10:12:41.038 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:12:41.037+0000 7f54a8e00700 1 -- 172.21.15.47:0/685033640 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f54a4197980 con 0x7f54a4102650 2024-03-21T10:12:41.038 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:12:41.037+0000 7f549aa00700 1 -- 172.21.15.47:0/685033640 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f549400f5f0 con 0x7f54a4102650 2024-03-21T10:12:41.038 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:12:41.037+0000 7f54a8e00700 1 -- 172.21.15.47:0/685033640 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f54a419a610 con 0x7f54a4102650 2024-03-21T10:12:41.039 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:12:41.038+0000 7f549aa00700 1 -- 172.21.15.47:0/685033640 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 13) v1 ==== 46350+0+0 (secure 0 0 0) 0x7f549402a070 con 0x7f54a4102650 2024-03-21T10:12:41.039 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:12:41.038+0000 7f54a8e00700 1 -- 172.21.15.47:0/685033640 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f54a4108220 con 0x7f54a4102650 2024-03-21T10:12:41.039 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:12:41.039+0000 7f549aa00700 1 --2- 172.21.15.47:0/685033640 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f5488039280 0x7f548803b740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:12:41.039 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:12:41.039+0000 7f549aa00700 1 -- 172.21.15.47:0/685033640 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f549401e030 con 0x7f54a4102650 2024-03-21T10:12:41.040 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:12:41.039+0000 7f54a1400700 1 --2- 172.21.15.47:0/685033640 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f5488039280 0x7f548803b740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:12:41.041 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:12:41.040+0000 7f54a1400700 1 --2- 172.21.15.47:0/685033640 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f5488039280 0x7f548803b740 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f5490006fd0 tx=0x7f5490006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:12:41.045 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:12:41.045+0000 7f549aa00700 1 -- 172.21.15.47:0/685033640 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f5494013070 con 0x7f54a4102650 2024-03-21T10:12:41.277 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:12:41.276+0000 7f54a8e00700 1 -- 172.21.15.47:0/685033640 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/allow_ptrace}] v 0) v1 -- 0x7f54a402dc20 con 0x7f54a4102650 2024-03-21T10:12:41.281 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:12:41.280+0000 7f549aa00700 1 -- 172.21.15.47:0/685033640 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/allow_ptrace}]=0 v9)=0 v9) v1 ==== 125+0+0 (secure 0 0 0) 0x7f5494031020 con 0x7f54a4102650 2024-03-21T10:12:41.285 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:12:41.284+0000 7f54a8e00700 1 -- 172.21.15.47:0/685033640 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f5488039280 msgr2=0x7f548803b740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:12:41.285 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:12:41.284+0000 7f54a8e00700 1 --2- 172.21.15.47:0/685033640 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f5488039280 0x7f548803b740 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f5490006fd0 tx=0x7f5490006e40 comp rx=0 tx=0).stop 2024-03-21T10:12:41.286 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:12:41.284+0000 7f54a8e00700 1 -- 172.21.15.47:0/685033640 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f54a4102650 msgr2=0x7f54a4197150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:12:41.286 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:12:41.284+0000 7f54a8e00700 1 --2- 172.21.15.47:0/685033640 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f54a4102650 0x7f54a4197150 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f5494005190 tx=0x7f5494004f80 comp rx=0 tx=0).stop 2024-03-21T10:12:41.286 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:12:41.284+0000 7f54a8e00700 1 -- 172.21.15.47:0/685033640 shutdown_connections 2024-03-21T10:12:41.286 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:12:41.284+0000 7f54a8e00700 1 --2- 172.21.15.47:0/685033640 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f5488039280 0x7f548803b740 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:12:41.286 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:12:41.284+0000 7f54a8e00700 1 --2- 172.21.15.47:0/685033640 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f54a4102650 0x7f54a4197150 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:12:41.286 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:12:41.284+0000 7f54a8e00700 1 -- 172.21.15.47:0/685033640 >> 172.21.15.47:0/685033640 conn(0x7f54a40fc060 msgr2=0x7f54a40fcb10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:12:41.286 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:12:41.285+0000 7f54a8e00700 1 -- 172.21.15.47:0/685033640 shutdown_connections 2024-03-21T10:12:41.287 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:12:41.285+0000 7f54a8e00700 1 -- 172.21.15.47:0/685033640 wait complete. 2024-03-21T10:12:41.964 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:12:41 smithi047 ceph-mon[31547]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:12:41.964 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:12:41 smithi047 ceph-mon[31547]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:12:41.964 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:12:41 smithi047 ceph-mon[31547]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi047", "name": "osd_memory_target"}]: dispatch 2024-03-21T10:12:41.964 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:12:41 smithi047 ceph-mon[31547]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:12:41.964 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:12:41 smithi047 ceph-mon[31547]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T10:12:41.964 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:12:41 smithi047 ceph-mon[31547]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:12:41.964 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:12:41 smithi047 ceph-mon[31547]: from='client.? 172.21.15.47:0/685033640' entity='client.admin' 2024-03-21T10:12:41.965 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:12:41 smithi047 ceph-mon[31547]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T10:12:41.965 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:12:41 smithi047 ceph-mon[31547]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:12:42.043 INFO:tasks.cephadm:Distributing conf and client.admin keyring to all hosts + 0755 2024-03-21T10:12:42.043 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2edfee68ad7243bed422a049b6f5f58619300f11 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3056d51a-e76b-11ee-95cd-87774f69a715 -- ceph orch client-keyring set client.admin '*' --mode 0755 2024-03-21T10:12:42.502 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/3056d51a-e76b-11ee-95cd-87774f69a715/mon.a/config 2024-03-21T10:12:44.538 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:12:44.536+0000 7f5e8b400700 1 -- 172.21.15.47:0/451596560 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5e84103df0 msgr2=0x7f5e84104210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:12:44.538 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:12:44.536+0000 7f5e8b400700 1 --2- 172.21.15.47:0/451596560 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5e84103df0 0x7f5e84104210 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f5e78009910 tx=0x7f5e78009c20 comp rx=0 tx=0).stop 2024-03-21T10:12:44.538 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:12:44.536+0000 7f5e8b400700 1 -- 172.21.15.47:0/451596560 shutdown_connections 2024-03-21T10:12:44.538 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:12:44.536+0000 7f5e8b400700 1 --2- 172.21.15.47:0/451596560 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5e84103df0 0x7f5e84104210 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:12:44.538 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:12:44.536+0000 7f5e8b400700 1 -- 172.21.15.47:0/451596560 >> 172.21.15.47:0/451596560 conn(0x7f5e840ff3b0 msgr2=0x7f5e841017d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:12:44.539 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:12:44.536+0000 7f5e8b400700 1 -- 172.21.15.47:0/451596560 shutdown_connections 2024-03-21T10:12:44.539 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:12:44.536+0000 7f5e8b400700 1 -- 172.21.15.47:0/451596560 wait complete. 2024-03-21T10:12:44.539 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:12:44.537+0000 7f5e8b400700 1 Processor -- start 2024-03-21T10:12:44.539 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:12:44.537+0000 7f5e8b400700 1 -- start start 2024-03-21T10:12:44.540 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:12:44.538+0000 7f5e8b400700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5e84103df0 0x7f5e84199440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:12:44.540 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:12:44.538+0000 7f5e8b400700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e84109d30 con 0x7f5e84103df0 2024-03-21T10:12:44.540 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:12:44.538+0000 7f5e88a00700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5e84103df0 0x7f5e84199440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:12:44.540 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:12:44.538+0000 7f5e88a00700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5e84103df0 0x7f5e84199440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:36488/0 (socket says 172.21.15.47:36488) 2024-03-21T10:12:44.541 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:12:44.538+0000 7f5e88a00700 1 -- 172.21.15.47:0/3050545095 learned_addr learned my addr 172.21.15.47:0/3050545095 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:12:44.541 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:12:44.538+0000 7f5e88a00700 1 -- 172.21.15.47:0/3050545095 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e84199980 con 0x7f5e84103df0 2024-03-21T10:12:44.541 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:12:44.539+0000 7f5e88a00700 1 --2- 172.21.15.47:0/3050545095 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5e84103df0 0x7f5e84199440 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f5e78005950 tx=0x7f5e78004fa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:12:44.541 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:12:44.539+0000 7f5e81600700 1 -- 172.21.15.47:0/3050545095 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f5e78026070 con 0x7f5e84103df0 2024-03-21T10:12:44.541 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:12:44.539+0000 7f5e81600700 1 -- 172.21.15.47:0/3050545095 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f5e7800bc00 con 0x7f5e84103df0 2024-03-21T10:12:44.541 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:12:44.539+0000 7f5e8b400700 1 -- 172.21.15.47:0/3050545095 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5e84199c10 con 0x7f5e84103df0 2024-03-21T10:12:44.541 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:12:44.539+0000 7f5e81600700 1 -- 172.21.15.47:0/3050545095 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f5e780039a0 con 0x7f5e84103df0 2024-03-21T10:12:44.541 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:12:44.539+0000 7f5e8b400700 1 -- 172.21.15.47:0/3050545095 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5e8419c8b0 con 0x7f5e84103df0 2024-03-21T10:12:44.542 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:12:44.540+0000 7f5e8b400700 1 -- 172.21.15.47:0/3050545095 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5e8410a4b0 con 0x7f5e84103df0 2024-03-21T10:12:44.542 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:12:44.540+0000 7f5e81600700 1 -- 172.21.15.47:0/3050545095 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 13) v1 ==== 46350+0+0 (secure 0 0 0) 0x7f5e78029070 con 0x7f5e84103df0 2024-03-21T10:12:44.542 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:12:44.541+0000 7f5e81600700 1 --2- 172.21.15.47:0/3050545095 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f5e6c039280 0x7f5e6c03b740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:12:44.542 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:12:44.541+0000 7f5e81600700 1 -- 172.21.15.47:0/3050545095 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f5e7801e070 con 0x7f5e84103df0 2024-03-21T10:12:44.542 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:12:44.541+0000 7f5e83e00700 1 --2- 172.21.15.47:0/3050545095 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f5e6c039280 0x7f5e6c03b740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:12:44.543 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:12:44.542+0000 7f5e83e00700 1 --2- 172.21.15.47:0/3050545095 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f5e6c039280 0x7f5e6c03b740 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f5e70006fd0 tx=0x7f5e70006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:12:44.547 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:12:44.546+0000 7f5e81600700 1 -- 172.21.15.47:0/3050545095 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f5e78013070 con 0x7f5e84103df0 2024-03-21T10:12:44.779 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:12:44.778+0000 7f5e8b400700 1 -- 172.21.15.47:0/3050545095 --> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] -- mgr_command(tid 0: {"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}) v1 -- 0x7f5e8402db40 con 0x7f5e6c039280 2024-03-21T10:12:44.784 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:12:44.783+0000 7f5e81600700 1 -- 172.21.15.47:0/3050545095 <== mgr.14152 v2:172.21.15.47:6800/2823885994 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f5e8402db40 con 0x7f5e6c039280 2024-03-21T10:12:44.788 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:12:44.786+0000 7f5e8b400700 1 -- 172.21.15.47:0/3050545095 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f5e6c039280 msgr2=0x7f5e6c03b740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:12:44.788 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:12:44.786+0000 7f5e8b400700 1 --2- 172.21.15.47:0/3050545095 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f5e6c039280 0x7f5e6c03b740 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f5e70006fd0 tx=0x7f5e70006e40 comp rx=0 tx=0).stop 2024-03-21T10:12:44.788 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:12:44.786+0000 7f5e8b400700 1 -- 172.21.15.47:0/3050545095 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5e84103df0 msgr2=0x7f5e84199440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:12:44.788 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:12:44.786+0000 7f5e8b400700 1 --2- 172.21.15.47:0/3050545095 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5e84103df0 0x7f5e84199440 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f5e78005950 tx=0x7f5e78004fa0 comp rx=0 tx=0).stop 2024-03-21T10:12:44.788 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:12:44.786+0000 7f5e8b400700 1 -- 172.21.15.47:0/3050545095 shutdown_connections 2024-03-21T10:12:44.788 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:12:44.786+0000 7f5e8b400700 1 --2- 172.21.15.47:0/3050545095 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f5e6c039280 0x7f5e6c03b740 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:12:44.788 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:12:44.786+0000 7f5e8b400700 1 --2- 172.21.15.47:0/3050545095 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5e84103df0 0x7f5e84199440 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:12:44.788 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:12:44.786+0000 7f5e8b400700 1 -- 172.21.15.47:0/3050545095 >> 172.21.15.47:0/3050545095 conn(0x7f5e840ff3b0 msgr2=0x7f5e840ffe50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:12:44.789 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:12:44.786+0000 7f5e8b400700 1 -- 172.21.15.47:0/3050545095 shutdown_connections 2024-03-21T10:12:44.789 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:12:44.787+0000 7f5e8b400700 1 -- 172.21.15.47:0/3050545095 wait complete. 2024-03-21T10:12:45.559 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi100 2024-03-21T10:12:45.559 DEBUG:teuthology.orchestra.run.smithi100:> set -ex 2024-03-21T10:12:45.559 DEBUG:teuthology.orchestra.run.smithi100:> dd of=/etc/ceph/ceph.conf 2024-03-21T10:12:45.616 DEBUG:teuthology.orchestra.run.smithi100:> set -ex 2024-03-21T10:12:45.616 DEBUG:teuthology.orchestra.run.smithi100:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-03-21T10:12:45.708 INFO:tasks.cephadm:Adding host smithi100 to orchestrator... 2024-03-21T10:12:45.708 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2edfee68ad7243bed422a049b6f5f58619300f11 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3056d51a-e76b-11ee-95cd-87774f69a715 -- ceph orch host add smithi100 2024-03-21T10:12:46.125 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/3056d51a-e76b-11ee-95cd-87774f69a715/mon.a/config 2024-03-21T10:12:46.213 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:12:45 smithi047 ceph-mon[31547]: from='client.14176 -' entity='client.admin' cmd=[{"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:12:46.213 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:12:45 smithi047 ceph-mon[31547]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:12:46.214 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:12:45 smithi047 ceph-mon[31547]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T10:12:46.214 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:12:45 smithi047 ceph-mon[31547]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T10:12:46.214 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:12:45 smithi047 ceph-mon[31547]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T10:12:46.214 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:12:45 smithi047 ceph-mon[31547]: Updating smithi047:/etc/ceph/ceph.conf 2024-03-21T10:12:46.214 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:12:45 smithi047 ceph-mon[31547]: Updating smithi047:/var/lib/ceph/3056d51a-e76b-11ee-95cd-87774f69a715/config/ceph.conf 2024-03-21T10:12:46.963 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:12:46 smithi047 ceph-mon[31547]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T10:12:46.963 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:12:46 smithi047 ceph-mon[31547]: Updating smithi047:/etc/ceph/ceph.client.admin.keyring 2024-03-21T10:12:46.964 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:12:46 smithi047 ceph-mon[31547]: Updating smithi047:/var/lib/ceph/3056d51a-e76b-11ee-95cd-87774f69a715/config/ceph.client.admin.keyring 2024-03-21T10:12:48.267 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:12:48.264+0000 7f3943a00700 1 -- 172.21.15.47:0/813465961 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f393c101dc0 msgr2=0x7f393c1021a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:12:48.267 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:12:48.264+0000 7f3943a00700 1 --2- 172.21.15.47:0/813465961 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f393c101dc0 0x7f393c1021a0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f3930009910 tx=0x7f3930009c20 comp rx=0 tx=0).stop 2024-03-21T10:12:48.267 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:12:48.264+0000 7f3943a00700 1 -- 172.21.15.47:0/813465961 shutdown_connections 2024-03-21T10:12:48.267 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:12:48.264+0000 7f3943a00700 1 --2- 172.21.15.47:0/813465961 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f393c101dc0 0x7f393c1021a0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:12:48.267 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:12:48.264+0000 7f3943a00700 1 -- 172.21.15.47:0/813465961 >> 172.21.15.47:0/813465961 conn(0x7f393c0fd670 msgr2=0x7f393c0ffa90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:12:48.267 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:12:48.265+0000 7f3943a00700 1 -- 172.21.15.47:0/813465961 shutdown_connections 2024-03-21T10:12:48.267 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:12:48.265+0000 7f3943a00700 1 -- 172.21.15.47:0/813465961 wait complete. 2024-03-21T10:12:48.268 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:12:48.266+0000 7f3943a00700 1 Processor -- start 2024-03-21T10:12:48.268 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:12:48.266+0000 7f3943a00700 1 -- start start 2024-03-21T10:12:48.268 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:12:48.266+0000 7f3943a00700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f393c101dc0 0x7f393c19f420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:12:48.269 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:12:48.266+0000 7f3943a00700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f393c10c040 con 0x7f393c101dc0 2024-03-21T10:12:48.269 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:12:48.267+0000 7f3941000700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f393c101dc0 0x7f393c19f420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:12:48.269 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:12:48.267+0000 7f3941000700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f393c101dc0 0x7f393c19f420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:36500/0 (socket says 172.21.15.47:36500) 2024-03-21T10:12:48.269 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:12:48.267+0000 7f3941000700 1 -- 172.21.15.47:0/3722079110 learned_addr learned my addr 172.21.15.47:0/3722079110 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:12:48.270 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:12:48.267+0000 7f3941000700 1 -- 172.21.15.47:0/3722079110 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f393c19f960 con 0x7f393c101dc0 2024-03-21T10:12:48.270 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:12:48.267+0000 7f3941000700 1 --2- 172.21.15.47:0/3722079110 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f393c101dc0 0x7f393c19f420 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f39300056d0 tx=0x7f39300057b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:12:48.270 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:12:48.268+0000 7f3939600700 1 -- 172.21.15.47:0/3722079110 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3930026070 con 0x7f393c101dc0 2024-03-21T10:12:48.270 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:12:48.268+0000 7f3943a00700 1 -- 172.21.15.47:0/3722079110 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f393c19fc50 con 0x7f393c101dc0 2024-03-21T10:12:48.270 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:12:48.268+0000 7f3939600700 1 -- 172.21.15.47:0/3722079110 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f393000bba0 con 0x7f393c101dc0 2024-03-21T10:12:48.270 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:12:48.268+0000 7f3939600700 1 -- 172.21.15.47:0/3722079110 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f39300037f0 con 0x7f393c101dc0 2024-03-21T10:12:48.270 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:12:48.268+0000 7f3943a00700 1 -- 172.21.15.47:0/3722079110 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f393c1995e0 con 0x7f393c101dc0 2024-03-21T10:12:48.271 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:12:48.269+0000 7f3943a00700 1 -- 172.21.15.47:0/3722079110 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f393c1032a0 con 0x7f393c101dc0 2024-03-21T10:12:48.271 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:12:48.269+0000 7f3939600700 1 -- 172.21.15.47:0/3722079110 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 13) v1 ==== 46350+0+0 (secure 0 0 0) 0x7f3930029070 con 0x7f393c101dc0 2024-03-21T10:12:48.271 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:12:48.270+0000 7f3939600700 1 --2- 172.21.15.47:0/3722079110 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f3924039220 0x7f392403b6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:12:48.271 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:12:48.270+0000 7f3939600700 1 -- 172.21.15.47:0/3722079110 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f393001e070 con 0x7f393c101dc0 2024-03-21T10:12:48.272 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:12:48.270+0000 7f393be00700 1 --2- 172.21.15.47:0/3722079110 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f3924039220 0x7f392403b6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:12:48.272 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:12:48.271+0000 7f393be00700 1 --2- 172.21.15.47:0/3722079110 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f3924039220 0x7f392403b6e0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f3928006fd0 tx=0x7f3928006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:12:48.277 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:12:48.275+0000 7f3939600700 1 -- 172.21.15.47:0/3722079110 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f3930013070 con 0x7f393c101dc0 2024-03-21T10:12:48.513 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:12:48.509+0000 7f3943a00700 1 -- 172.21.15.47:0/3722079110 --> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi100", "target": ["mon-mgr", ""]}) v1 -- 0x7f393c19ab10 con 0x7f3924039220 2024-03-21T10:12:48.714 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:12:48 smithi047 ceph-mon[31547]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:12:48.714 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:12:48 smithi047 ceph-mon[31547]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:12:48.714 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:12:48 smithi047 ceph-mon[31547]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:12:49.713 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:12:49 smithi047 ceph-mon[31547]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T10:12:49.713 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:12:49 smithi047 ceph-mon[31547]: from='client.14178 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi100", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:12:51.713 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:12:51 smithi047 ceph-mon[31547]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T10:12:51.714 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:12:51 smithi047 ceph-mon[31547]: Deploying cephadm binary to smithi100 2024-03-21T10:12:53.713 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:12:53 smithi047 ceph-mon[31547]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T10:12:55.224 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:12:55.220+0000 7f3939600700 1 -- 172.21.15.47:0/3722079110 <== mgr.14152 v2:172.21.15.47:6800/2823885994 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+49 (secure 0 0 0) 0x7f393c19ab10 con 0x7f3924039220 2024-03-21T10:12:55.224 INFO:teuthology.orchestra.run.smithi047.stdout:Added host 'smithi100' with addr '172.21.15.100' 2024-03-21T10:12:55.227 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:12:55.224+0000 7f3943a00700 1 -- 172.21.15.47:0/3722079110 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f3924039220 msgr2=0x7f392403b6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:12:55.227 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:12:55.224+0000 7f3943a00700 1 --2- 172.21.15.47:0/3722079110 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f3924039220 0x7f392403b6e0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f3928006fd0 tx=0x7f3928006e40 comp rx=0 tx=0).stop 2024-03-21T10:12:55.227 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:12:55.224+0000 7f3943a00700 1 -- 172.21.15.47:0/3722079110 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f393c101dc0 msgr2=0x7f393c19f420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:12:55.227 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:12:55.224+0000 7f3943a00700 1 --2- 172.21.15.47:0/3722079110 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f393c101dc0 0x7f393c19f420 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f39300056d0 tx=0x7f39300057b0 comp rx=0 tx=0).stop 2024-03-21T10:12:55.227 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:12:55.225+0000 7f3943a00700 1 -- 172.21.15.47:0/3722079110 shutdown_connections 2024-03-21T10:12:55.227 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:12:55.225+0000 7f3943a00700 1 --2- 172.21.15.47:0/3722079110 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f3924039220 0x7f392403b6e0 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:12:55.227 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:12:55.225+0000 7f3943a00700 1 --2- 172.21.15.47:0/3722079110 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f393c101dc0 0x7f393c19f420 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:12:55.227 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:12:55.225+0000 7f3943a00700 1 -- 172.21.15.47:0/3722079110 >> 172.21.15.47:0/3722079110 conn(0x7f393c0fd670 msgr2=0x7f393c0743c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:12:55.227 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:12:55.225+0000 7f3943a00700 1 -- 172.21.15.47:0/3722079110 shutdown_connections 2024-03-21T10:12:55.228 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:12:55.225+0000 7f3943a00700 1 -- 172.21.15.47:0/3722079110 wait complete. 2024-03-21T10:12:55.614 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:12:55 smithi047 ceph-mon[31547]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T10:12:55.614 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:12:55 smithi047 ceph-mon[31547]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:12:55.614 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:12:55 smithi047 ceph-mon[31547]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T10:12:56.021 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2edfee68ad7243bed422a049b6f5f58619300f11 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3056d51a-e76b-11ee-95cd-87774f69a715 -- ceph orch host ls --format=json 2024-03-21T10:12:56.492 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/3056d51a-e76b-11ee-95cd-87774f69a715/mon.a/config 2024-03-21T10:12:56.714 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:12:56 smithi047 ceph-mon[31547]: Added host smithi100 2024-03-21T10:12:56.714 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:12:56 smithi047 ceph-mon[31547]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:12:57.641 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:12:57 smithi047 ceph-mon[31547]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T10:12:57.641 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:12:57 smithi047 ceph-mon[31547]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:12:58.470 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:12:58.467+0000 7f33db200700 1 -- 172.21.15.47:0/457359824 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f33dc103e20 msgr2=0x7f33dc104200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:12:58.470 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:12:58.467+0000 7f33db200700 1 --2- 172.21.15.47:0/457359824 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f33dc103e20 0x7f33dc104200 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f33cc009910 tx=0x7f33cc009c20 comp rx=0 tx=0).stop 2024-03-21T10:12:58.470 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:12:58.467+0000 7f33db200700 1 -- 172.21.15.47:0/457359824 shutdown_connections 2024-03-21T10:12:58.470 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:12:58.467+0000 7f33db200700 1 --2- 172.21.15.47:0/457359824 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f33dc103e20 0x7f33dc104200 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:12:58.470 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:12:58.467+0000 7f33db200700 1 -- 172.21.15.47:0/457359824 >> 172.21.15.47:0/457359824 conn(0x7f33dc0ff690 msgr2=0x7f33dc101ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:12:58.470 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:12:58.467+0000 7f33db200700 1 -- 172.21.15.47:0/457359824 shutdown_connections 2024-03-21T10:12:58.470 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:12:58.467+0000 7f33db200700 1 -- 172.21.15.47:0/457359824 wait complete. 2024-03-21T10:12:58.471 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:12:58.468+0000 7f33db200700 1 Processor -- start 2024-03-21T10:12:58.471 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:12:58.469+0000 7f33db200700 1 -- start start 2024-03-21T10:12:58.471 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:12:58.469+0000 7f33db200700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f33dc103e20 0x7f33dc199680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:12:58.472 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:12:58.469+0000 7f33db200700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33dc10e0a0 con 0x7f33dc103e20 2024-03-21T10:12:58.472 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:12:58.469+0000 7f33d9e00700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f33dc103e20 0x7f33dc199680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:12:58.472 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:12:58.469+0000 7f33d9e00700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f33dc103e20 0x7f33dc199680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:46234/0 (socket says 172.21.15.47:46234) 2024-03-21T10:12:58.472 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:12:58.469+0000 7f33d9e00700 1 -- 172.21.15.47:0/1372138461 learned_addr learned my addr 172.21.15.47:0/1372138461 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:12:58.472 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:12:58.470+0000 7f33d9e00700 1 -- 172.21.15.47:0/1372138461 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f33dc199bc0 con 0x7f33dc103e20 2024-03-21T10:12:58.473 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:12:58.470+0000 7f33d9e00700 1 --2- 172.21.15.47:0/1372138461 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f33dc103e20 0x7f33dc199680 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f33cc006010 tx=0x7f33cc005780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:12:58.473 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:12:58.470+0000 7f33d2a00700 1 -- 172.21.15.47:0/1372138461 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f33cc026070 con 0x7f33dc103e20 2024-03-21T10:12:58.473 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:12:58.470+0000 7f33d2a00700 1 -- 172.21.15.47:0/1372138461 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f33cc00bba0 con 0x7f33dc103e20 2024-03-21T10:12:58.473 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:12:58.470+0000 7f33db200700 1 -- 172.21.15.47:0/1372138461 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f33dc199eb0 con 0x7f33dc103e20 2024-03-21T10:12:58.473 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:12:58.470+0000 7f33d2a00700 1 -- 172.21.15.47:0/1372138461 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f33cc0038a0 con 0x7f33dc103e20 2024-03-21T10:12:58.473 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:12:58.471+0000 7f33db200700 1 -- 172.21.15.47:0/1372138461 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f33dc19df00 con 0x7f33dc103e20 2024-03-21T10:12:58.473 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:12:58.472+0000 7f33d2a00700 1 -- 172.21.15.47:0/1372138461 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 13) v1 ==== 46350+0+0 (secure 0 0 0) 0x7f33cc029070 con 0x7f33dc103e20 2024-03-21T10:12:58.474 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:12:58.472+0000 7f33db200700 1 -- 172.21.15.47:0/1372138461 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f33dc105300 con 0x7f33dc103e20 2024-03-21T10:12:58.474 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:12:58.472+0000 7f33d2a00700 1 --2- 172.21.15.47:0/1372138461 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f33c0039280 0x7f33c003b740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:12:58.474 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:12:58.472+0000 7f33d2a00700 1 -- 172.21.15.47:0/1372138461 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f33cc01e070 con 0x7f33dc103e20 2024-03-21T10:12:58.474 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:12:58.472+0000 7f33d9400700 1 --2- 172.21.15.47:0/1372138461 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f33c0039280 0x7f33c003b740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:12:58.475 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:12:58.473+0000 7f33d9400700 1 --2- 172.21.15.47:0/1372138461 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f33c0039280 0x7f33c003b740 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f33c8006fd0 tx=0x7f33c8006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:12:58.480 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:12:58.478+0000 7f33d2a00700 1 -- 172.21.15.47:0/1372138461 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f33cc013070 con 0x7f33dc103e20 2024-03-21T10:12:58.713 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:12:58.711+0000 7f33db200700 1 -- 172.21.15.47:0/1372138461 --> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f33dc19a990 con 0x7f33c0039280 2024-03-21T10:12:58.715 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:12:58.713+0000 7f33d2a00700 1 -- 172.21.15.47:0/1372138461 <== mgr.14152 v2:172.21.15.47:6800/2823885994 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+160 (secure 0 0 0) 0x7f33dc19a990 con 0x7f33c0039280 2024-03-21T10:12:58.715 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-21T10:12:58.715 INFO:teuthology.orchestra.run.smithi047.stdout:[{"addr": "172.21.15.47", "hostname": "smithi047", "labels": [], "status": ""}, {"addr": "172.21.15.100", "hostname": "smithi100", "labels": [], "status": ""}] 2024-03-21T10:12:58.718 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:12:58.716+0000 7f33db200700 1 -- 172.21.15.47:0/1372138461 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f33c0039280 msgr2=0x7f33c003b740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:12:58.718 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:12:58.716+0000 7f33db200700 1 --2- 172.21.15.47:0/1372138461 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f33c0039280 0x7f33c003b740 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f33c8006fd0 tx=0x7f33c8006e40 comp rx=0 tx=0).stop 2024-03-21T10:12:58.719 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:12:58.716+0000 7f33db200700 1 -- 172.21.15.47:0/1372138461 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f33dc103e20 msgr2=0x7f33dc199680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:12:58.719 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:12:58.716+0000 7f33db200700 1 --2- 172.21.15.47:0/1372138461 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f33dc103e20 0x7f33dc199680 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f33cc006010 tx=0x7f33cc005780 comp rx=0 tx=0).stop 2024-03-21T10:12:58.719 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:12:58.716+0000 7f33db200700 1 -- 172.21.15.47:0/1372138461 shutdown_connections 2024-03-21T10:12:58.719 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:12:58.716+0000 7f33db200700 1 --2- 172.21.15.47:0/1372138461 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f33c0039280 0x7f33c003b740 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:12:58.719 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:12:58.716+0000 7f33db200700 1 --2- 172.21.15.47:0/1372138461 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f33dc103e20 0x7f33dc199680 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:12:58.719 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:12:58.716+0000 7f33db200700 1 -- 172.21.15.47:0/1372138461 >> 172.21.15.47:0/1372138461 conn(0x7f33dc0ff690 msgr2=0x7f33dc1057f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:12:58.719 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:12:58.717+0000 7f33db200700 1 -- 172.21.15.47:0/1372138461 shutdown_connections 2024-03-21T10:12:58.719 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:12:58.717+0000 7f33db200700 1 -- 172.21.15.47:0/1372138461 wait complete. 2024-03-21T10:12:59.469 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi157 2024-03-21T10:12:59.469 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2024-03-21T10:12:59.470 DEBUG:teuthology.orchestra.run.smithi157:> dd of=/etc/ceph/ceph.conf 2024-03-21T10:12:59.526 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2024-03-21T10:12:59.527 DEBUG:teuthology.orchestra.run.smithi157:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-03-21T10:12:59.621 INFO:tasks.cephadm:Adding host smithi157 to orchestrator... 2024-03-21T10:12:59.621 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2edfee68ad7243bed422a049b6f5f58619300f11 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3056d51a-e76b-11ee-95cd-87774f69a715 -- ceph orch host add smithi157 2024-03-21T10:12:59.714 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:12:59 smithi047 ceph-mon[31547]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T10:12:59.714 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:12:59 smithi047 ceph-mon[31547]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-21T10:12:59.714 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:12:59 smithi047 ceph-mon[31547]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:13:00.080 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/3056d51a-e76b-11ee-95cd-87774f69a715/mon.a/config 2024-03-21T10:13:01.604 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:13:01 smithi047 ceph-mon[31547]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T10:13:02.034 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:13:02.033+0000 7f0d11000700 1 -- 172.21.15.47:0/3543911049 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0d0c1083f0 msgr2=0x7f0d0c1087d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:13:02.034 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:13:02.033+0000 7f0d11000700 1 --2- 172.21.15.47:0/3543911049 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0d0c1083f0 0x7f0d0c1087d0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f0cfc009910 tx=0x7f0cfc009c20 comp rx=0 tx=0).stop 2024-03-21T10:13:02.034 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:13:02.033+0000 7f0d11000700 1 -- 172.21.15.47:0/3543911049 shutdown_connections 2024-03-21T10:13:02.034 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:13:02.033+0000 7f0d11000700 1 --2- 172.21.15.47:0/3543911049 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0d0c1083f0 0x7f0d0c1087d0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:13:02.034 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:13:02.033+0000 7f0d11000700 1 -- 172.21.15.47:0/3543911049 >> 172.21.15.47:0/3543911049 conn(0x7f0d0c077930 msgr2=0x7f0d0c079d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:13:02.035 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:13:02.033+0000 7f0d11000700 1 -- 172.21.15.47:0/3543911049 shutdown_connections 2024-03-21T10:13:02.035 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:13:02.033+0000 7f0d11000700 1 -- 172.21.15.47:0/3543911049 wait complete. 2024-03-21T10:13:02.035 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:13:02.034+0000 7f0d11000700 1 Processor -- start 2024-03-21T10:13:02.035 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:13:02.034+0000 7f0d11000700 1 -- start start 2024-03-21T10:13:02.035 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:13:02.035+0000 7f0d11000700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0d0c1083f0 0x7f0d0c1a3820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:13:02.036 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:13:02.035+0000 7f0d11000700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d0c112670 con 0x7f0d0c1083f0 2024-03-21T10:13:02.036 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:13:02.035+0000 7f0d09e00700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0d0c1083f0 0x7f0d0c1a3820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:13:02.036 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:13:02.035+0000 7f0d09e00700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0d0c1083f0 0x7f0d0c1a3820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:37858/0 (socket says 172.21.15.47:37858) 2024-03-21T10:13:02.036 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:13:02.035+0000 7f0d09e00700 1 -- 172.21.15.47:0/4220536826 learned_addr learned my addr 172.21.15.47:0/4220536826 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:13:02.037 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:13:02.035+0000 7f0d09e00700 1 -- 172.21.15.47:0/4220536826 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d0c1a3d60 con 0x7f0d0c1083f0 2024-03-21T10:13:02.037 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:13:02.036+0000 7f0d09e00700 1 --2- 172.21.15.47:0/4220536826 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0d0c1083f0 0x7f0d0c1a3820 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f0cfc006010 tx=0x7f0cfc005780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:13:02.037 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:13:02.036+0000 7f0d02a00700 1 -- 172.21.15.47:0/4220536826 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f0cfc026070 con 0x7f0d0c1083f0 2024-03-21T10:13:02.037 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:13:02.036+0000 7f0d02a00700 1 -- 172.21.15.47:0/4220536826 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f0cfc00bba0 con 0x7f0d0c1083f0 2024-03-21T10:13:02.037 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:13:02.036+0000 7f0d11000700 1 -- 172.21.15.47:0/4220536826 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0d0c1a4050 con 0x7f0d0c1083f0 2024-03-21T10:13:02.037 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:13:02.036+0000 7f0d02a00700 1 -- 172.21.15.47:0/4220536826 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f0cfc0038a0 con 0x7f0d0c1083f0 2024-03-21T10:13:02.037 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:13:02.036+0000 7f0d11000700 1 -- 172.21.15.47:0/4220536826 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0d0c19d9e0 con 0x7f0d0c1083f0 2024-03-21T10:13:02.038 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:13:02.037+0000 7f0d11000700 1 -- 172.21.15.47:0/4220536826 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0d0c1098d0 con 0x7f0d0c1083f0 2024-03-21T10:13:02.038 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:13:02.037+0000 7f0d02a00700 1 -- 172.21.15.47:0/4220536826 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 13) v1 ==== 46350+0+0 (secure 0 0 0) 0x7f0cfc029070 con 0x7f0d0c1083f0 2024-03-21T10:13:02.039 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:13:02.038+0000 7f0d02a00700 1 --2- 172.21.15.47:0/4220536826 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f0cf0039220 0x7f0cf003b6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:13:02.039 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:13:02.038+0000 7f0d02a00700 1 -- 172.21.15.47:0/4220536826 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f0cfc01e070 con 0x7f0d0c1083f0 2024-03-21T10:13:02.039 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:13:02.038+0000 7f0d09400700 1 --2- 172.21.15.47:0/4220536826 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f0cf0039220 0x7f0cf003b6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:13:02.040 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:13:02.039+0000 7f0d09400700 1 --2- 172.21.15.47:0/4220536826 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f0cf0039220 0x7f0cf003b6e0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f0cf4006fd0 tx=0x7f0cf4006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:13:02.044 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:13:02.043+0000 7f0d02a00700 1 -- 172.21.15.47:0/4220536826 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f0cfc013070 con 0x7f0d0c1083f0 2024-03-21T10:13:02.275 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:13:02.274+0000 7f0d11000700 1 -- 172.21.15.47:0/4220536826 --> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi157", "target": ["mon-mgr", ""]}) v1 -- 0x7f0d0c19ef80 con 0x7f0cf0039220 2024-03-21T10:13:03.714 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:13:03 smithi047 ceph-mon[31547]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T10:13:03.714 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:13:03 smithi047 ceph-mon[31547]: from='client.14182 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi157", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:13:04.714 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:13:04 smithi047 ceph-mon[31547]: Deploying cephadm binary to smithi157 2024-03-21T10:13:04.714 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:13:04 smithi047 ceph-mon[31547]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:13:04.714 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:13:04 smithi047 ceph-mon[31547]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:13:04.714 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:13:04 smithi047 ceph-mon[31547]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:13:04.714 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:13:04 smithi047 ceph-mon[31547]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:13:04.714 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:13:04 smithi047 ceph-mon[31547]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi100", "name": "osd_memory_target"}]: dispatch 2024-03-21T10:13:04.715 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:13:04 smithi047 ceph-mon[31547]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T10:13:04.715 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:13:04 smithi047 ceph-mon[31547]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T10:13:05.713 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:13:05 smithi047 ceph-mon[31547]: Updating smithi100:/etc/ceph/ceph.conf 2024-03-21T10:13:05.714 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:13:05 smithi047 ceph-mon[31547]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T10:13:06.713 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:13:06 smithi047 ceph-mon[31547]: Updating smithi100:/var/lib/ceph/3056d51a-e76b-11ee-95cd-87774f69a715/config/ceph.conf 2024-03-21T10:13:07.713 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:13:07 smithi047 ceph-mon[31547]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T10:13:09.041 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:13:09.039+0000 7f0d02a00700 1 -- 172.21.15.47:0/4220536826 <== mgr.14152 v2:172.21.15.47:6800/2823885994 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+49 (secure 0 0 0) 0x7f0d0c19ef80 con 0x7f0cf0039220 2024-03-21T10:13:09.041 INFO:teuthology.orchestra.run.smithi047.stdout:Added host 'smithi157' with addr '172.21.15.157' 2024-03-21T10:13:09.044 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:13:09.043+0000 7f0d11000700 1 -- 172.21.15.47:0/4220536826 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f0cf0039220 msgr2=0x7f0cf003b6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:13:09.044 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:13:09.043+0000 7f0d11000700 1 --2- 172.21.15.47:0/4220536826 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f0cf0039220 0x7f0cf003b6e0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f0cf4006fd0 tx=0x7f0cf4006e40 comp rx=0 tx=0).stop 2024-03-21T10:13:09.044 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:13:09.043+0000 7f0d11000700 1 -- 172.21.15.47:0/4220536826 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0d0c1083f0 msgr2=0x7f0d0c1a3820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:13:09.044 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:13:09.043+0000 7f0d11000700 1 --2- 172.21.15.47:0/4220536826 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0d0c1083f0 0x7f0d0c1a3820 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f0cfc006010 tx=0x7f0cfc005780 comp rx=0 tx=0).stop 2024-03-21T10:13:09.045 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:13:09.043+0000 7f0d11000700 1 -- 172.21.15.47:0/4220536826 shutdown_connections 2024-03-21T10:13:09.045 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:13:09.043+0000 7f0d11000700 1 --2- 172.21.15.47:0/4220536826 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f0cf0039220 0x7f0cf003b6e0 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:13:09.045 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:13:09.043+0000 7f0d11000700 1 --2- 172.21.15.47:0/4220536826 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0d0c1083f0 0x7f0d0c1a3820 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:13:09.045 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:13:09.043+0000 7f0d11000700 1 -- 172.21.15.47:0/4220536826 >> 172.21.15.47:0/4220536826 conn(0x7f0d0c077930 msgr2=0x7f0d0c109dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:13:09.045 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:13:09.044+0000 7f0d11000700 1 -- 172.21.15.47:0/4220536826 shutdown_connections 2024-03-21T10:13:09.045 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:13:09.044+0000 7f0d11000700 1 -- 172.21.15.47:0/4220536826 wait complete. 2024-03-21T10:13:09.713 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:13:09 smithi047 ceph-mon[31547]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T10:13:09.714 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:13:09 smithi047 ceph-mon[31547]: Updating smithi100:/etc/ceph/ceph.client.admin.keyring 2024-03-21T10:13:09.714 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:13:09 smithi047 ceph-mon[31547]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:13:09.854 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2edfee68ad7243bed422a049b6f5f58619300f11 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3056d51a-e76b-11ee-95cd-87774f69a715 -- ceph orch host ls --format=json 2024-03-21T10:13:10.310 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/3056d51a-e76b-11ee-95cd-87774f69a715/mon.a/config 2024-03-21T10:13:11.188 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:13:10 smithi047 ceph-mon[31547]: Updating smithi100:/var/lib/ceph/3056d51a-e76b-11ee-95cd-87774f69a715/config/ceph.client.admin.keyring 2024-03-21T10:13:11.188 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:13:10 smithi047 ceph-mon[31547]: Added host smithi157 2024-03-21T10:13:11.188 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:13:10 smithi047 ceph-mon[31547]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:13:11.189 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:13:10 smithi047 ceph-mon[31547]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:13:11.189 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:13:10 smithi047 ceph-mon[31547]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:13:11.189 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:13:10 smithi047 ceph-mon[31547]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T10:13:12.027 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:13:11 smithi047 ceph-mon[31547]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T10:13:12.027 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:13:11 smithi047 ceph-mon[31547]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:13:12.325 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:13:12.323+0000 7fcc7c200700 1 -- 172.21.15.47:0/3487580072 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fcc74103ff0 msgr2=0x7fcc741043d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:13:12.325 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:13:12.323+0000 7fcc7c200700 1 --2- 172.21.15.47:0/3487580072 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fcc74103ff0 0x7fcc741043d0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fcc68009910 tx=0x7fcc68009c20 comp rx=0 tx=0).stop 2024-03-21T10:13:12.325 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:13:12.324+0000 7fcc7c200700 1 -- 172.21.15.47:0/3487580072 shutdown_connections 2024-03-21T10:13:12.325 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:13:12.324+0000 7fcc7c200700 1 --2- 172.21.15.47:0/3487580072 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fcc74103ff0 0x7fcc741043d0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:13:12.325 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:13:12.324+0000 7fcc7c200700 1 -- 172.21.15.47:0/3487580072 >> 172.21.15.47:0/3487580072 conn(0x7fcc740ff8a0 msgr2=0x7fcc74101cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:13:12.325 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:13:12.324+0000 7fcc7c200700 1 -- 172.21.15.47:0/3487580072 shutdown_connections 2024-03-21T10:13:12.326 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:13:12.324+0000 7fcc7c200700 1 -- 172.21.15.47:0/3487580072 wait complete. 2024-03-21T10:13:12.326 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:13:12.325+0000 7fcc7c200700 1 Processor -- start 2024-03-21T10:13:12.326 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:13:12.325+0000 7fcc7c200700 1 -- start start 2024-03-21T10:13:12.326 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:13:12.325+0000 7fcc7c200700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fcc74103ff0 0x7fcc7419f400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:13:12.326 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:13:12.325+0000 7fcc7c200700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc7410e270 con 0x7fcc74103ff0 2024-03-21T10:13:12.327 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:13:12.326+0000 7fcc79800700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fcc74103ff0 0x7fcc7419f400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:13:12.327 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:13:12.326+0000 7fcc79800700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fcc74103ff0 0x7fcc7419f400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:41640/0 (socket says 172.21.15.47:41640) 2024-03-21T10:13:12.327 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:13:12.326+0000 7fcc79800700 1 -- 172.21.15.47:0/3640628870 learned_addr learned my addr 172.21.15.47:0/3640628870 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:13:12.327 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:13:12.326+0000 7fcc79800700 1 -- 172.21.15.47:0/3640628870 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc7419f940 con 0x7fcc74103ff0 2024-03-21T10:13:12.327 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:13:12.326+0000 7fcc79800700 1 --2- 172.21.15.47:0/3640628870 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fcc74103ff0 0x7fcc7419f400 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fcc680056d0 tx=0x7fcc680057b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:13:12.328 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:13:12.327+0000 7fcc72000700 1 -- 172.21.15.47:0/3640628870 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fcc68026070 con 0x7fcc74103ff0 2024-03-21T10:13:12.328 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:13:12.327+0000 7fcc72000700 1 -- 172.21.15.47:0/3640628870 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fcc6800bba0 con 0x7fcc74103ff0 2024-03-21T10:13:12.328 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:13:12.327+0000 7fcc7c200700 1 -- 172.21.15.47:0/3640628870 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcc7419fc30 con 0x7fcc74103ff0 2024-03-21T10:13:12.328 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:13:12.327+0000 7fcc72000700 1 -- 172.21.15.47:0/3640628870 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fcc680037f0 con 0x7fcc74103ff0 2024-03-21T10:13:12.328 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:13:12.327+0000 7fcc7c200700 1 -- 172.21.15.47:0/3640628870 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcc741995c0 con 0x7fcc74103ff0 2024-03-21T10:13:12.329 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:13:12.328+0000 7fcc7c200700 1 -- 172.21.15.47:0/3640628870 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcc741054d0 con 0x7fcc74103ff0 2024-03-21T10:13:12.329 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:13:12.328+0000 7fcc72000700 1 -- 172.21.15.47:0/3640628870 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 13) v1 ==== 46350+0+0 (secure 0 0 0) 0x7fcc68029070 con 0x7fcc74103ff0 2024-03-21T10:13:12.330 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:13:12.329+0000 7fcc72000700 1 --2- 172.21.15.47:0/3640628870 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7fcc5c039220 0x7fcc5c03b6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:13:12.330 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:13:12.329+0000 7fcc72000700 1 -- 172.21.15.47:0/3640628870 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fcc6801e070 con 0x7fcc74103ff0 2024-03-21T10:13:12.330 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:13:12.329+0000 7fcc78e00700 1 --2- 172.21.15.47:0/3640628870 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7fcc5c039220 0x7fcc5c03b6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:13:12.330 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:13:12.330+0000 7fcc78e00700 1 --2- 172.21.15.47:0/3640628870 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7fcc5c039220 0x7fcc5c03b6e0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fcc60006fd0 tx=0x7fcc60006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:13:12.335 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:13:12.335+0000 7fcc72000700 1 -- 172.21.15.47:0/3640628870 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fcc68013070 con 0x7fcc74103ff0 2024-03-21T10:13:12.570 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:13:12.568+0000 7fcc7c200700 1 -- 172.21.15.47:0/3640628870 --> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fcc7419aaf0 con 0x7fcc5c039220 2024-03-21T10:13:12.572 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:13:12.571+0000 7fcc72000700 1 -- 172.21.15.47:0/3640628870 <== mgr.14152 v2:172.21.15.47:6800/2823885994 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+240 (secure 0 0 0) 0x7fcc7419aaf0 con 0x7fcc5c039220 2024-03-21T10:13:12.572 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-21T10:13:12.573 INFO:teuthology.orchestra.run.smithi047.stdout:[{"addr": "172.21.15.47", "hostname": "smithi047", "labels": [], "status": ""}, {"addr": "172.21.15.100", "hostname": "smithi100", "labels": [], "status": ""}, {"addr": "172.21.15.157", "hostname": "smithi157", "labels": [], "status": ""}] 2024-03-21T10:13:12.576 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:13:12.574+0000 7fcc7c200700 1 -- 172.21.15.47:0/3640628870 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7fcc5c039220 msgr2=0x7fcc5c03b6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:13:12.576 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:13:12.574+0000 7fcc7c200700 1 --2- 172.21.15.47:0/3640628870 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7fcc5c039220 0x7fcc5c03b6e0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fcc60006fd0 tx=0x7fcc60006e40 comp rx=0 tx=0).stop 2024-03-21T10:13:12.576 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:13:12.574+0000 7fcc7c200700 1 -- 172.21.15.47:0/3640628870 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fcc74103ff0 msgr2=0x7fcc7419f400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:13:12.576 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:13:12.574+0000 7fcc7c200700 1 --2- 172.21.15.47:0/3640628870 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fcc74103ff0 0x7fcc7419f400 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fcc680056d0 tx=0x7fcc680057b0 comp rx=0 tx=0).stop 2024-03-21T10:13:12.576 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:13:12.574+0000 7fcc7c200700 1 -- 172.21.15.47:0/3640628870 shutdown_connections 2024-03-21T10:13:12.576 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:13:12.574+0000 7fcc7c200700 1 --2- 172.21.15.47:0/3640628870 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7fcc5c039220 0x7fcc5c03b6e0 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:13:12.577 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:13:12.574+0000 7fcc7c200700 1 --2- 172.21.15.47:0/3640628870 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fcc74103ff0 0x7fcc7419f400 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:13:12.577 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:13:12.574+0000 7fcc7c200700 1 -- 172.21.15.47:0/3640628870 >> 172.21.15.47:0/3640628870 conn(0x7fcc740ff8a0 msgr2=0x7fcc741059c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:13:12.577 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:13:12.574+0000 7fcc7c200700 1 -- 172.21.15.47:0/3640628870 shutdown_connections 2024-03-21T10:13:12.577 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:13:12.574+0000 7fcc7c200700 1 -- 172.21.15.47:0/3640628870 wait complete. 2024-03-21T10:13:13.189 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:13:12 smithi047 ceph-mon[31547]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T10:13:13.189 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:13:12 smithi047 ceph-mon[31547]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:13:13.189 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:13:12 smithi047 ceph-mon[31547]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-21T10:13:13.243 INFO:tasks.cephadm:Setting crush tunables to default 2024-03-21T10:13:13.244 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2edfee68ad7243bed422a049b6f5f58619300f11 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3056d51a-e76b-11ee-95cd-87774f69a715 -- ceph osd crush tunables default 2024-03-21T10:13:13.710 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/3056d51a-e76b-11ee-95cd-87774f69a715/mon.a/config 2024-03-21T10:13:14.964 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:13:14 smithi047 ceph-mon[31547]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T10:13:14.964 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:13:14 smithi047 ceph-mon[31547]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:13:15.822 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:13:15.820+0000 7f2ef1000700 1 -- 172.21.15.47:0/1432633727 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2eec101dd0 msgr2=0x7f2eec1021b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:13:15.822 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:13:15.820+0000 7f2ef1000700 1 --2- 172.21.15.47:0/1432633727 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2eec101dd0 0x7f2eec1021b0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f2edc009910 tx=0x7f2edc009c20 comp rx=0 tx=0).stop 2024-03-21T10:13:15.822 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:13:15.820+0000 7f2ef1000700 1 -- 172.21.15.47:0/1432633727 shutdown_connections 2024-03-21T10:13:15.822 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:13:15.820+0000 7f2ef1000700 1 --2- 172.21.15.47:0/1432633727 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2eec101dd0 0x7f2eec1021b0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:13:15.822 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:13:15.820+0000 7f2ef1000700 1 -- 172.21.15.47:0/1432633727 >> 172.21.15.47:0/1432633727 conn(0x7f2eec0fd6a0 msgr2=0x7f2eec0ffac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:13:15.822 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:13:15.821+0000 7f2ef1000700 1 -- 172.21.15.47:0/1432633727 shutdown_connections 2024-03-21T10:13:15.823 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:13:15.821+0000 7f2ef1000700 1 -- 172.21.15.47:0/1432633727 wait complete. 2024-03-21T10:13:15.823 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:13:15.822+0000 7f2ef1000700 1 Processor -- start 2024-03-21T10:13:15.823 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:13:15.822+0000 7f2ef1000700 1 -- start start 2024-03-21T10:13:15.824 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:13:15.823+0000 7f2ef1000700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2eec101dd0 0x7f2eec19f410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:13:15.824 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:13:15.823+0000 7f2ef1000700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2eec10c050 con 0x7f2eec101dd0 2024-03-21T10:13:15.824 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:13:15.823+0000 7f2ee9e00700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2eec101dd0 0x7f2eec19f410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:13:15.824 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:13:15.823+0000 7f2ee9e00700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2eec101dd0 0x7f2eec19f410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:41646/0 (socket says 172.21.15.47:41646) 2024-03-21T10:13:15.824 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:13:15.823+0000 7f2ee9e00700 1 -- 172.21.15.47:0/1166569903 learned_addr learned my addr 172.21.15.47:0/1166569903 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:13:15.825 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:13:15.823+0000 7f2ee9e00700 1 -- 172.21.15.47:0/1166569903 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2eec19f950 con 0x7f2eec101dd0 2024-03-21T10:13:15.825 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:13:15.824+0000 7f2ee9e00700 1 --2- 172.21.15.47:0/1166569903 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2eec101dd0 0x7f2eec19f410 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f2edc0056d0 tx=0x7f2edc0057b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:13:15.825 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:13:15.824+0000 7f2ee2a00700 1 -- 172.21.15.47:0/1166569903 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f2edc026070 con 0x7f2eec101dd0 2024-03-21T10:13:15.825 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:13:15.824+0000 7f2ee2a00700 1 -- 172.21.15.47:0/1166569903 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f2edc00bba0 con 0x7f2eec101dd0 2024-03-21T10:13:15.825 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:13:15.824+0000 7f2ee2a00700 1 -- 172.21.15.47:0/1166569903 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f2edc0037f0 con 0x7f2eec101dd0 2024-03-21T10:13:15.825 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:13:15.824+0000 7f2ef1000700 1 -- 172.21.15.47:0/1166569903 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2eec19fc40 con 0x7f2eec101dd0 2024-03-21T10:13:15.825 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:13:15.824+0000 7f2ef1000700 1 -- 172.21.15.47:0/1166569903 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2eec1995d0 con 0x7f2eec101dd0 2024-03-21T10:13:15.826 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:13:15.825+0000 7f2ee2a00700 1 -- 172.21.15.47:0/1166569903 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 13) v1 ==== 46350+0+0 (secure 0 0 0) 0x7f2edc029070 con 0x7f2eec101dd0 2024-03-21T10:13:15.826 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:13:15.825+0000 7f2ef1000700 1 -- 172.21.15.47:0/1166569903 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2eec1032b0 con 0x7f2eec101dd0 2024-03-21T10:13:15.827 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:13:15.826+0000 7f2ee2a00700 1 --2- 172.21.15.47:0/1166569903 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f2ed0039280 0x7f2ed003b740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:13:15.827 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:13:15.826+0000 7f2ee2a00700 1 -- 172.21.15.47:0/1166569903 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f2edc01e070 con 0x7f2eec101dd0 2024-03-21T10:13:15.827 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:13:15.826+0000 7f2ee9400700 1 --2- 172.21.15.47:0/1166569903 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f2ed0039280 0x7f2ed003b740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:13:15.828 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:13:15.827+0000 7f2ee9400700 1 --2- 172.21.15.47:0/1166569903 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f2ed0039280 0x7f2ed003b740 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f2ed8006fd0 tx=0x7f2ed8006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:13:15.833 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:13:15.832+0000 7f2ee2a00700 1 -- 172.21.15.47:0/1166569903 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f2edc013070 con 0x7f2eec101dd0 2024-03-21T10:13:16.067 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:13:16.066+0000 7f2ef1000700 1 -- 172.21.15.47:0/1166569903 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) v1 -- 0x7f2eec0637c0 con 0x7f2eec101dd0 2024-03-21T10:13:16.957 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:13:16.955+0000 7f2ee2a00700 1 -- 172.21.15.47:0/1166569903 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v4) v1 ==== 124+0+0 (secure 0 0 0) 0x7f2edc030090 con 0x7f2eec101dd0 2024-03-21T10:13:16.960 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:13:16.959+0000 7f2ef1000700 1 -- 172.21.15.47:0/1166569903 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f2ed0039280 msgr2=0x7f2ed003b740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:13:16.960 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:13:16.959+0000 7f2ef1000700 1 --2- 172.21.15.47:0/1166569903 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f2ed0039280 0x7f2ed003b740 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f2ed8006fd0 tx=0x7f2ed8006e40 comp rx=0 tx=0).stop 2024-03-21T10:13:16.961 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:13:16.959+0000 7f2ef1000700 1 -- 172.21.15.47:0/1166569903 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2eec101dd0 msgr2=0x7f2eec19f410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:13:16.961 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:13:16.959+0000 7f2ef1000700 1 --2- 172.21.15.47:0/1166569903 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2eec101dd0 0x7f2eec19f410 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f2edc0056d0 tx=0x7f2edc0057b0 comp rx=0 tx=0).stop 2024-03-21T10:13:16.961 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:13:16.959+0000 7f2ef1000700 1 -- 172.21.15.47:0/1166569903 shutdown_connections 2024-03-21T10:13:16.961 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:13:16.959+0000 7f2ef1000700 1 --2- 172.21.15.47:0/1166569903 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f2ed0039280 0x7f2ed003b740 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:13:16.961 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:13:16.959+0000 7f2ef1000700 1 --2- 172.21.15.47:0/1166569903 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2eec101dd0 0x7f2eec19f410 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:13:16.962 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:13:16.959+0000 7f2ef1000700 1 -- 172.21.15.47:0/1166569903 >> 172.21.15.47:0/1166569903 conn(0x7f2eec0fd6a0 msgr2=0x7f2eec0743c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:13:16.962 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:13:16.960+0000 7f2ef1000700 1 -- 172.21.15.47:0/1166569903 shutdown_connections 2024-03-21T10:13:16.962 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:13:16.960+0000 7f2ef1000700 1 -- 172.21.15.47:0/1166569903 wait complete. 2024-03-21T10:13:16.962 INFO:teuthology.orchestra.run.smithi047.stderr:adjusted tunables profile to default 2024-03-21T10:13:17.213 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:13:16 smithi047 ceph-mon[31547]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T10:13:17.214 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:13:16 smithi047 ceph-mon[31547]: from='client.? 172.21.15.47:0/1166569903' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-03-21T10:13:17.750 INFO:tasks.cephadm:Adding mon.a on smithi047 2024-03-21T10:13:17.751 INFO:tasks.cephadm:Adding mon.b on smithi100 2024-03-21T10:13:17.751 INFO:tasks.cephadm:Adding mon.c on smithi157 2024-03-21T10:13:17.751 DEBUG:teuthology.orchestra.run.smithi157:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2edfee68ad7243bed422a049b6f5f58619300f11 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3056d51a-e76b-11ee-95cd-87774f69a715 -- ceph orch apply mon '3;smithi047:172.21.15.47=a;smithi100:172.21.15.100=b;smithi157:172.21.15.157=c' 2024-03-21T10:13:18.214 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:13:17 smithi047 ceph-mon[31547]: from='client.? 172.21.15.47:0/1166569903' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-03-21T10:13:18.214 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:13:17 smithi047 ceph-mon[31547]: osdmap e4: 0 total, 0 up, 0 in 2024-03-21T10:13:18.697 INFO:teuthology.orchestra.run.smithi157.stderr:Inferring config /etc/ceph/ceph.conf 2024-03-21T10:13:18.803 INFO:teuthology.orchestra.run.smithi157.stderr:Inferring config /etc/ceph/ceph.conf 2024-03-21T10:13:19.464 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:13:18 smithi047 ceph-mon[31547]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T10:13:19.464 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:13:18 smithi047 ceph-mon[31547]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:13:19.464 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:13:18 smithi047 ceph-mon[31547]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:13:19.464 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:13:18 smithi047 ceph-mon[31547]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:13:19.464 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:13:18 smithi047 ceph-mon[31547]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:13:19.464 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:13:18 smithi047 ceph-mon[31547]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi157", "name": "osd_memory_target"}]: dispatch 2024-03-21T10:13:19.464 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:13:18 smithi047 ceph-mon[31547]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T10:13:19.464 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:13:18 smithi047 ceph-mon[31547]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T10:13:20.213 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:13:19 smithi047 ceph-mon[31547]: Updating smithi157:/etc/ceph/ceph.conf 2024-03-21T10:13:20.214 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:13:19 smithi047 ceph-mon[31547]: Updating smithi157:/var/lib/ceph/3056d51a-e76b-11ee-95cd-87774f69a715/config/ceph.conf 2024-03-21T10:13:20.745 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-21T10:13:20.743+0000 7fa3b5e00700 1 -- 172.21.15.157:0/2543781631 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa3b0102650 msgr2=0x7fa3b0104a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:13:20.745 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-21T10:13:20.743+0000 7fa3b5e00700 1 --2- 172.21.15.157:0/2543781631 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa3b0102650 0x7fa3b0104a40 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fa3a0009910 tx=0x7fa3a0009c20 comp rx=0 tx=0).stop 2024-03-21T10:13:20.746 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-21T10:13:20.743+0000 7fa3b5e00700 1 -- 172.21.15.157:0/2543781631 shutdown_connections 2024-03-21T10:13:20.746 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-21T10:13:20.743+0000 7fa3b5e00700 1 --2- 172.21.15.157:0/2543781631 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa3b0102650 0x7fa3b0104a40 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:13:20.746 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-21T10:13:20.743+0000 7fa3b5e00700 1 -- 172.21.15.157:0/2543781631 >> 172.21.15.157:0/2543781631 conn(0x7fa3b00fc060 msgr2=0x7fa3b00fe4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:13:20.746 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-21T10:13:20.743+0000 7fa3b5e00700 1 -- 172.21.15.157:0/2543781631 shutdown_connections 2024-03-21T10:13:20.746 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-21T10:13:20.744+0000 7fa3b5e00700 1 -- 172.21.15.157:0/2543781631 wait complete. 2024-03-21T10:13:20.747 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-21T10:13:20.745+0000 7fa3b5e00700 1 Processor -- start 2024-03-21T10:13:20.747 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-21T10:13:20.745+0000 7fa3b5e00700 1 -- start start 2024-03-21T10:13:20.747 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-21T10:13:20.745+0000 7fa3b5e00700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa3b0102650 0x7fa3b01971b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:13:20.747 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-21T10:13:20.745+0000 7fa3b5e00700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3b0107aa0 con 0x7fa3b0102650 2024-03-21T10:13:20.747 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-21T10:13:20.746+0000 7fa3aea00700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa3b0102650 0x7fa3b01971b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:13:20.747 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-21T10:13:20.746+0000 7fa3aea00700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa3b0102650 0x7fa3b01971b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.157:57094/0 (socket says 172.21.15.157:57094) 2024-03-21T10:13:20.747 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-21T10:13:20.746+0000 7fa3aea00700 1 -- 172.21.15.157:0/3379932209 learned_addr learned my addr 172.21.15.157:0/3379932209 (peer_addr_for_me v2:172.21.15.157:0/0) 2024-03-21T10:13:20.748 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-21T10:13:20.746+0000 7fa3aea00700 1 -- 172.21.15.157:0/3379932209 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3b01976f0 con 0x7fa3b0102650 2024-03-21T10:13:20.748 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-21T10:13:20.746+0000 7fa3aea00700 1 --2- 172.21.15.157:0/3379932209 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa3b0102650 0x7fa3b01971b0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fa3a0005b40 tx=0x7fa3a0004f80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:13:20.748 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-21T10:13:20.747+0000 7fa3b4a00700 1 -- 172.21.15.157:0/3379932209 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa3a0027070 con 0x7fa3b0102650 2024-03-21T10:13:20.748 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-21T10:13:20.747+0000 7fa3b5e00700 1 -- 172.21.15.157:0/3379932209 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa3b01979e0 con 0x7fa3b0102650 2024-03-21T10:13:20.748 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-21T10:13:20.747+0000 7fa3b5e00700 1 -- 172.21.15.157:0/3379932209 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa3b019a620 con 0x7fa3b0102650 2024-03-21T10:13:20.749 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-21T10:13:20.747+0000 7fa3b4a00700 1 -- 172.21.15.157:0/3379932209 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fa3a0004800 con 0x7fa3b0102650 2024-03-21T10:13:20.749 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-21T10:13:20.747+0000 7fa3b4a00700 1 -- 172.21.15.157:0/3379932209 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa3a0031060 con 0x7fa3b0102650 2024-03-21T10:13:20.749 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-21T10:13:20.748+0000 7fa3b5e00700 1 -- 172.21.15.157:0/3379932209 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa390005360 con 0x7fa3b0102650 2024-03-21T10:13:20.749 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-21T10:13:20.748+0000 7fa3b4a00700 1 -- 172.21.15.157:0/3379932209 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 13) v1 ==== 46350+0+0 (secure 0 0 0) 0x7fa3a002a070 con 0x7fa3b0102650 2024-03-21T10:13:20.750 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-21T10:13:20.749+0000 7fa3b4a00700 1 --2- 172.21.15.157:0/3379932209 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7fa394039280 0x7fa39403b740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:13:20.750 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-21T10:13:20.749+0000 7fa3b4a00700 1 -- 172.21.15.157:0/3379932209 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fa3a001e030 con 0x7fa3b0102650 2024-03-21T10:13:20.751 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-21T10:13:20.749+0000 7fa3ae000700 1 --2- 172.21.15.157:0/3379932209 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7fa394039280 0x7fa39403b740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:13:20.752 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-21T10:13:20.750+0000 7fa3ae000700 1 --2- 172.21.15.157:0/3379932209 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7fa394039280 0x7fa39403b740 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fa39c006fd0 tx=0x7fa39c006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:13:20.766 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-21T10:13:20.765+0000 7fa3b4a00700 1 -- 172.21.15.157:0/3379932209 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fa3a0013070 con 0x7fa3b0102650 2024-03-21T10:13:21.008 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-21T10:13:21.007+0000 7fa3b5e00700 1 -- 172.21.15.157:0/3379932209 --> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi047:172.21.15.47=a;smithi100:172.21.15.100=b;smithi157:172.21.15.157=c", "target": ["mon-mgr", ""]}) v1 -- 0x7fa390000cb0 con 0x7fa394039280 2024-03-21T10:13:21.022 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-21T10:13:21.021+0000 7fa3b4a00700 1 -- 172.21.15.157:0/3379932209 <== mgr.14152 v2:172.21.15.47:6800/2823885994 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+24 (secure 0 0 0) 0x7fa390000cb0 con 0x7fa394039280 2024-03-21T10:13:21.023 INFO:teuthology.orchestra.run.smithi157.stdout:Scheduled mon update... 2024-03-21T10:13:21.025 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-21T10:13:21.024+0000 7fa3b5e00700 1 -- 172.21.15.157:0/3379932209 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7fa394039280 msgr2=0x7fa39403b740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:13:21.025 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-21T10:13:21.024+0000 7fa3b5e00700 1 --2- 172.21.15.157:0/3379932209 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7fa394039280 0x7fa39403b740 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fa39c006fd0 tx=0x7fa39c006e40 comp rx=0 tx=0).stop 2024-03-21T10:13:21.025 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-21T10:13:21.024+0000 7fa3b5e00700 1 -- 172.21.15.157:0/3379932209 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa3b0102650 msgr2=0x7fa3b01971b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:13:21.025 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-21T10:13:21.024+0000 7fa3b5e00700 1 --2- 172.21.15.157:0/3379932209 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa3b0102650 0x7fa3b01971b0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fa3a0005b40 tx=0x7fa3a0004f80 comp rx=0 tx=0).stop 2024-03-21T10:13:21.026 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-21T10:13:21.025+0000 7fa3b5e00700 1 -- 172.21.15.157:0/3379932209 shutdown_connections 2024-03-21T10:13:21.026 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-21T10:13:21.025+0000 7fa3b5e00700 1 --2- 172.21.15.157:0/3379932209 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7fa394039280 0x7fa39403b740 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:13:21.026 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-21T10:13:21.025+0000 7fa3b5e00700 1 --2- 172.21.15.157:0/3379932209 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa3b0102650 0x7fa3b01971b0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:13:21.026 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-21T10:13:21.025+0000 7fa3b5e00700 1 -- 172.21.15.157:0/3379932209 >> 172.21.15.157:0/3379932209 conn(0x7fa3b00fc060 msgr2=0x7fa3b00fcb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:13:21.026 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-21T10:13:21.025+0000 7fa3b5e00700 1 -- 172.21.15.157:0/3379932209 shutdown_connections 2024-03-21T10:13:21.026 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-21T10:13:21.025+0000 7fa3b5e00700 1 -- 172.21.15.157:0/3379932209 wait complete. 2024-03-21T10:13:21.214 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:13:20 smithi047 ceph-mon[31547]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T10:13:21.214 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:13:20 smithi047 ceph-mon[31547]: Updating smithi157:/etc/ceph/ceph.client.admin.keyring 2024-03-21T10:13:21.742 DEBUG:teuthology.orchestra.run.smithi100:mon.b> sudo journalctl -f -n 0 -u ceph-3056d51a-e76b-11ee-95cd-87774f69a715@mon.b.service 2024-03-21T10:13:21.750 DEBUG:teuthology.orchestra.run.smithi157:mon.c> sudo journalctl -f -n 0 -u ceph-3056d51a-e76b-11ee-95cd-87774f69a715@mon.c.service 2024-03-21T10:13:21.753 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-03-21T10:13:21.754 DEBUG:teuthology.orchestra.run.smithi157:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2edfee68ad7243bed422a049b6f5f58619300f11 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3056d51a-e76b-11ee-95cd-87774f69a715 -- ceph mon dump -f json 2024-03-21T10:13:22.193 INFO:teuthology.orchestra.run.smithi157.stderr:Inferring config /var/lib/ceph/3056d51a-e76b-11ee-95cd-87774f69a715/config/ceph.conf 2024-03-21T10:13:22.464 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:13:22 smithi047 ceph-mon[31547]: Updating smithi157:/var/lib/ceph/3056d51a-e76b-11ee-95cd-87774f69a715/config/ceph.client.admin.keyring 2024-03-21T10:13:22.464 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:13:22 smithi047 ceph-mon[31547]: from='client.14188 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi047:172.21.15.47=a;smithi100:172.21.15.100=b;smithi157:172.21.15.157=c", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:13:22.464 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:13:22 smithi047 ceph-mon[31547]: Saving service mon spec with placement smithi047:172.21.15.47=a;smithi100:172.21.15.100=b;smithi157:172.21.15.157=c;count:3 2024-03-21T10:13:22.464 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:13:22 smithi047 ceph-mon[31547]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:13:22.464 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:13:22 smithi047 ceph-mon[31547]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:13:22.464 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:13:22 smithi047 ceph-mon[31547]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:13:22.464 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:13:22 smithi047 ceph-mon[31547]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:13:22.464 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:13:22 smithi047 ceph-mon[31547]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-21T10:13:22.465 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:13:22 smithi047 ceph-mon[31547]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T10:13:22.465 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:13:22 smithi047 ceph-mon[31547]: Deploying daemon mon.c on smithi157 2024-03-21T10:13:23.964 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:13:23 smithi047 ceph-mon[31547]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T10:13:24.191 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-21T10:13:24.189+0000 7f1f35600700 1 -- 172.21.15.157:0/488171566 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1f301079f0 msgr2=0x7f1f30107dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:13:24.192 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-21T10:13:24.189+0000 7f1f35600700 1 --2- 172.21.15.157:0/488171566 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1f301079f0 0x7f1f30107dd0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f1f20009910 tx=0x7f1f20009c20 comp rx=0 tx=0).stop 2024-03-21T10:13:24.192 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-21T10:13:24.190+0000 7f1f35600700 1 -- 172.21.15.157:0/488171566 shutdown_connections 2024-03-21T10:13:24.192 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-21T10:13:24.190+0000 7f1f35600700 1 --2- 172.21.15.157:0/488171566 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1f301079f0 0x7f1f30107dd0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:13:24.192 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-21T10:13:24.190+0000 7f1f35600700 1 -- 172.21.15.157:0/488171566 >> 172.21.15.157:0/488171566 conn(0x7f1f30076420 msgr2=0x7f1f30076830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:13:24.192 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-21T10:13:24.190+0000 7f1f35600700 1 -- 172.21.15.157:0/488171566 shutdown_connections 2024-03-21T10:13:24.192 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-21T10:13:24.190+0000 7f1f35600700 1 -- 172.21.15.157:0/488171566 wait complete. 2024-03-21T10:13:24.193 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-21T10:13:24.191+0000 7f1f35600700 1 Processor -- start 2024-03-21T10:13:24.193 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-21T10:13:24.191+0000 7f1f35600700 1 -- start start 2024-03-21T10:13:24.193 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-21T10:13:24.191+0000 7f1f35600700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1f301079f0 0x7f1f301995f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:13:24.193 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-21T10:13:24.191+0000 7f1f35600700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f3010bec0 con 0x7f1f301079f0 2024-03-21T10:13:24.193 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-21T10:13:24.192+0000 7f1f2fe00700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1f301079f0 0x7f1f301995f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:13:24.193 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-21T10:13:24.192+0000 7f1f2fe00700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1f301079f0 0x7f1f301995f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.157:57112/0 (socket says 172.21.15.157:57112) 2024-03-21T10:13:24.194 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-21T10:13:24.192+0000 7f1f2fe00700 1 -- 172.21.15.157:0/146512500 learned_addr learned my addr 172.21.15.157:0/146512500 (peer_addr_for_me v2:172.21.15.157:0/0) 2024-03-21T10:13:24.194 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-21T10:13:24.193+0000 7f1f2fe00700 1 -- 172.21.15.157:0/146512500 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f30199b30 con 0x7f1f301079f0 2024-03-21T10:13:24.194 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-21T10:13:24.193+0000 7f1f2fe00700 1 --2- 172.21.15.157:0/146512500 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1f301079f0 0x7f1f301995f0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f1f200056d0 tx=0x7f1f200057b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:13:24.195 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-21T10:13:24.194+0000 7f1f2cc00700 1 -- 172.21.15.157:0/146512500 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f1f20026070 con 0x7f1f301079f0 2024-03-21T10:13:24.195 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-21T10:13:24.194+0000 7f1f2cc00700 1 -- 172.21.15.157:0/146512500 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f1f20004370 con 0x7f1f301079f0 2024-03-21T10:13:24.195 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-21T10:13:24.194+0000 7f1f35600700 1 -- 172.21.15.157:0/146512500 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1f30199e20 con 0x7f1f301079f0 2024-03-21T10:13:24.195 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-21T10:13:24.194+0000 7f1f2cc00700 1 -- 172.21.15.157:0/146512500 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f1f200039b0 con 0x7f1f301079f0 2024-03-21T10:13:24.195 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-21T10:13:24.194+0000 7f1f35600700 1 -- 172.21.15.157:0/146512500 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1f3019dec0 con 0x7f1f301079f0 2024-03-21T10:13:24.196 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-21T10:13:24.195+0000 7f1f35600700 1 -- 172.21.15.157:0/146512500 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1f30103120 con 0x7f1f301079f0 2024-03-21T10:13:24.196 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-21T10:13:24.195+0000 7f1f2cc00700 1 -- 172.21.15.157:0/146512500 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 13) v1 ==== 46350+0+0 (secure 0 0 0) 0x7f1f20029070 con 0x7f1f301079f0 2024-03-21T10:13:24.197 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-21T10:13:24.196+0000 7f1f2cc00700 1 --2- 172.21.15.157:0/146512500 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f1f14039220 0x7f1f1403b6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:13:24.197 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-21T10:13:24.196+0000 7f1f2cc00700 1 -- 172.21.15.157:0/146512500 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f1f2001e070 con 0x7f1f301079f0 2024-03-21T10:13:24.198 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-21T10:13:24.197+0000 7f1f2f400700 1 --2- 172.21.15.157:0/146512500 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f1f14039220 0x7f1f1403b6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:13:24.199 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-21T10:13:24.198+0000 7f1f2f400700 1 --2- 172.21.15.157:0/146512500 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f1f14039220 0x7f1f1403b6e0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f1f1c006fd0 tx=0x7f1f1c006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:13:24.214 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-21T10:13:24.212+0000 7f1f2cc00700 1 -- 172.21.15.157:0/146512500 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f1f20013070 con 0x7f1f301079f0 2024-03-21T10:13:24.527 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-21T10:13:24.525+0000 7f1f35600700 1 -- 172.21.15.157:0/146512500 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f1f3019a950 con 0x7f1f301079f0 2024-03-21T10:13:24.528 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-21T10:13:24.526+0000 7f1f2cc00700 1 -- 172.21.15.157:0/146512500 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+745 (secure 0 0 0) 0x7f1f20030090 con 0x7f1f301079f0 2024-03-21T10:13:24.528 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-03-21T10:13:24.528 INFO:teuthology.orchestra.run.smithi157.stdout:{"epoch":1,"fsid":"3056d51a-e76b-11ee-95cd-87774f69a715","modified":"2024-03-21T10:11:12.872646Z","created":"2024-03-21T10:11:12.872646Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:3300","nonce":0},{"type":"v1","addr":"172.21.15.47:6789","nonce":0}]},"addr":"172.21.15.47:6789/0","public_addr":"172.21.15.47:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-03-21T10:13:24.531 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-21T10:13:24.529+0000 7f1f35600700 1 -- 172.21.15.157:0/146512500 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f1f14039220 msgr2=0x7f1f1403b6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:13:24.531 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-21T10:13:24.529+0000 7f1f35600700 1 --2- 172.21.15.157:0/146512500 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f1f14039220 0x7f1f1403b6e0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f1f1c006fd0 tx=0x7f1f1c006e40 comp rx=0 tx=0).stop 2024-03-21T10:13:24.531 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-21T10:13:24.529+0000 7f1f35600700 1 -- 172.21.15.157:0/146512500 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1f301079f0 msgr2=0x7f1f301995f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:13:24.531 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-21T10:13:24.529+0000 7f1f35600700 1 --2- 172.21.15.157:0/146512500 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1f301079f0 0x7f1f301995f0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f1f200056d0 tx=0x7f1f200057b0 comp rx=0 tx=0).stop 2024-03-21T10:13:24.531 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-21T10:13:24.529+0000 7f1f35600700 1 -- 172.21.15.157:0/146512500 shutdown_connections 2024-03-21T10:13:24.531 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-21T10:13:24.529+0000 7f1f35600700 1 --2- 172.21.15.157:0/146512500 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f1f14039220 0x7f1f1403b6e0 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:13:24.532 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-21T10:13:24.529+0000 7f1f35600700 1 --2- 172.21.15.157:0/146512500 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1f301079f0 0x7f1f301995f0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:13:24.532 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-21T10:13:24.529+0000 7f1f35600700 1 -- 172.21.15.157:0/146512500 >> 172.21.15.157:0/146512500 conn(0x7f1f30076420 msgr2=0x7f1f3010a050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:13:24.532 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-21T10:13:24.529+0000 7f1f35600700 1 -- 172.21.15.157:0/146512500 shutdown_connections 2024-03-21T10:13:24.532 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-21T10:13:24.530+0000 7f1f35600700 1 -- 172.21.15.157:0/146512500 wait complete. 2024-03-21T10:13:24.532 INFO:teuthology.orchestra.run.smithi157.stderr:dumped monmap epoch 1 2024-03-21T10:13:25.964 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:13:25 smithi047 ceph-mon[31547]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T10:13:25.964 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:13:25 smithi047 ceph-mon[31547]: from='client.? 172.21.15.157:0/146512500' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-03-21T10:13:27.077 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-03-21T10:13:27.078 DEBUG:teuthology.orchestra.run.smithi157:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2edfee68ad7243bed422a049b6f5f58619300f11 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3056d51a-e76b-11ee-95cd-87774f69a715 -- ceph mon dump -f json 2024-03-21T10:13:27.736 INFO:teuthology.orchestra.run.smithi157.stderr:Inferring config /var/lib/ceph/3056d51a-e76b-11ee-95cd-87774f69a715/mon.c/config 2024-03-21T10:13:27.964 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:13:27 smithi047 ceph-mon[31547]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T10:13:29.111 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:13:28 smithi157 podman[35543]: 2024-03-21 10:13:28.840824974 +0000 UTC m=+0.034782164 image pull 237e43c7a65ff48534d94b97c1bfdbe4b7be5672e3d9a613745083ea3702aa54 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232 2024-03-21T10:13:29.406 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:13:29 smithi157 podman[35543]: 2024-03-21 10:13:29.295432903 +0000 UTC m=+0.489390137 container init d9e77d928b1edb6585269826c3f06d8b28c3edaf426be89e4bff287c4b2551af (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-3056d51a-e76b-11ee-95cd-87774f69a715-mon-c, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240319, GIT_CLEAN=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, maintainer=Guillaume Abrioux , RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, org.label-schema.license=GPLv2) 2024-03-21T10:13:29.407 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:13:29 smithi157 podman[35543]: 2024-03-21 10:13:29.305785142 +0000 UTC m=+0.499742349 container start d9e77d928b1edb6585269826c3f06d8b28c3edaf426be89e4bff287c4b2551af (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-3056d51a-e76b-11ee-95cd-87774f69a715-mon-c, org.label-schema.license=GPLv2, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240319, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , ceph=True) 2024-03-21T10:13:29.407 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:13:29 smithi157 ceph-mon[35581]: set uid:gid to 167:167 (ceph:ceph) 2024-03-21T10:13:29.407 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:13:29 smithi157 ceph-mon[35581]: ceph version 19.0.0-2231-g2edfee68 (2edfee68ad7243bed422a049b6f5f58619300f11) squid (dev), process ceph-mon, pid 2 2024-03-21T10:13:29.407 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:13:29 smithi157 ceph-mon[35581]: pidfile_write: ignore empty --pid-file 2024-03-21T10:13:29.407 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:13:29 smithi157 ceph-mon[35581]: load: jerasure load: lrc 2024-03-21T10:13:29.407 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:13:29 smithi157 ceph-mon[35581]: rocksdb: RocksDB version: 7.9.2 2024-03-21T10:13:29.407 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:13:29 smithi157 ceph-mon[35581]: rocksdb: Git sha 0 2024-03-21T10:13:29.407 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:13:29 smithi157 ceph-mon[35581]: rocksdb: Compile date 2024-03-20 06:23:48 2024-03-21T10:13:29.407 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:13:29 smithi157 ceph-mon[35581]: rocksdb: DB SUMMARY 2024-03-21T10:13:29.407 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:13:29 smithi157 ceph-mon[35581]: rocksdb: DB Session ID: 6TW3GYW9FOUTXZFL45WV 2024-03-21T10:13:29.407 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:13:29 smithi157 ceph-mon[35581]: rocksdb: CURRENT file: CURRENT 2024-03-21T10:13:29.408 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:13:29 smithi157 ceph-mon[35581]: rocksdb: IDENTITY file: IDENTITY 2024-03-21T10:13:29.408 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:13:29 smithi157 ceph-mon[35581]: rocksdb: MANIFEST file: MANIFEST-000005 size: 59 Bytes 2024-03-21T10:13:29.408 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:13:29 smithi157 ceph-mon[35581]: rocksdb: SST files in /var/lib/ceph/mon/ceph-c/store.db dir, Total Num: 0, files: 2024-03-21T10:13:29.408 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:13:29 smithi157 ceph-mon[35581]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-c/store.db: 000004.log size: 511 ; 2024-03-21T10:13:29.408 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:13:29 smithi157 ceph-mon[35581]: rocksdb: Options.error_if_exists: 0 2024-03-21T10:13:29.408 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:13:29 smithi157 ceph-mon[35581]: rocksdb: Options.create_if_missing: 0 2024-03-21T10:13:29.408 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:13:29 smithi157 ceph-mon[35581]: rocksdb: Options.paranoid_checks: 1 2024-03-21T10:13:29.408 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:13:29 smithi157 ceph-mon[35581]: rocksdb: Options.flush_verify_memtable_count: 1 2024-03-21T10:13:29.408 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:13:29 smithi157 ceph-mon[35581]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-03-21T10:13:29.408 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:13:29 smithi157 ceph-mon[35581]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-03-21T10:13:29.408 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:13:29 smithi157 ceph-mon[35581]: rocksdb: Options.env: 0x5623a6b25b20 2024-03-21T10:13:29.408 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:13:29 smithi157 ceph-mon[35581]: rocksdb: Options.fs: PosixFileSystem 2024-03-21T10:13:29.408 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:13:29 smithi157 ceph-mon[35581]: rocksdb: Options.info_log: 0x5623a81e5960 2024-03-21T10:13:29.408 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:13:29 smithi157 ceph-mon[35581]: rocksdb: Options.max_file_opening_threads: 16 2024-03-21T10:13:29.410 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:13:29 smithi157 ceph-mon[35581]: rocksdb: Options.statistics: (nil) 2024-03-21T10:13:29.410 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:13:29 smithi157 ceph-mon[35581]: rocksdb: Options.use_fsync: 0 2024-03-21T10:13:29.410 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:13:29 smithi157 ceph-mon[35581]: rocksdb: Options.max_log_file_size: 0 2024-03-21T10:13:29.410 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:13:29 smithi157 ceph-mon[35581]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-03-21T10:13:29.410 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:13:29 smithi157 ceph-mon[35581]: rocksdb: Options.log_file_time_to_roll: 0 2024-03-21T10:13:29.410 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:13:29 smithi157 ceph-mon[35581]: rocksdb: Options.keep_log_file_num: 1000 2024-03-21T10:13:29.411 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:13:29 smithi157 ceph-mon[35581]: rocksdb: Options.recycle_log_file_num: 0 2024-03-21T10:13:29.411 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:13:29 smithi157 ceph-mon[35581]: rocksdb: Options.allow_fallocate: 1 2024-03-21T10:13:29.411 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:13:29 smithi157 ceph-mon[35581]: rocksdb: Options.allow_mmap_reads: 0 2024-03-21T10:13:29.411 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:13:29 smithi157 ceph-mon[35581]: rocksdb: Options.allow_mmap_writes: 0 2024-03-21T10:13:29.411 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:13:29 smithi157 ceph-mon[35581]: rocksdb: Options.use_direct_reads: 0 2024-03-21T10:13:29.411 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:13:29 smithi157 ceph-mon[35581]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-03-21T10:13:29.411 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:13:29 smithi157 ceph-mon[35581]: rocksdb: Options.create_missing_column_families: 0 2024-03-21T10:13:29.411 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:13:29 smithi157 ceph-mon[35581]: rocksdb: Options.db_log_dir: 2024-03-21T10:13:29.411 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:13:29 smithi157 ceph-mon[35581]: rocksdb: Options.wal_dir: 2024-03-21T10:13:29.411 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:13:29 smithi157 ceph-mon[35581]: rocksdb: Options.table_cache_numshardbits: 6 2024-03-21T10:13:29.411 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:13:29 smithi157 ceph-mon[35581]: rocksdb: Options.WAL_ttl_seconds: 0 2024-03-21T10:13:29.411 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:13:29 smithi157 ceph-mon[35581]: rocksdb: Options.WAL_size_limit_MB: 0 2024-03-21T10:13:29.411 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:13:29 smithi157 ceph-mon[35581]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-03-21T10:13:29.411 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:13:29 smithi157 ceph-mon[35581]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-03-21T10:13:29.412 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:13:29 smithi157 ceph-mon[35581]: rocksdb: Options.is_fd_close_on_exec: 1 2024-03-21T10:13:29.412 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:13:29 smithi157 ceph-mon[35581]: rocksdb: Options.advise_random_on_open: 1 2024-03-21T10:13:29.412 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:13:29 smithi157 ceph-mon[35581]: rocksdb: Options.db_write_buffer_size: 0 2024-03-21T10:13:29.412 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:13:29 smithi157 ceph-mon[35581]: rocksdb: Options.write_buffer_manager: 0x5623a81b81e0 2024-03-21T10:13:29.412 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:13:29 smithi157 ceph-mon[35581]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-03-21T10:13:29.412 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:13:29 smithi157 ceph-mon[35581]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-03-21T10:13:29.412 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:13:29 smithi157 ceph-mon[35581]: rocksdb: Options.use_adaptive_mutex: 0 2024-03-21T10:13:29.412 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:13:29 smithi157 ceph-mon[35581]: rocksdb: Options.rate_limiter: (nil) 2024-03-21T10:13:29.412 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:13:29 smithi157 ceph-mon[35581]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-03-21T10:13:29.412 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:13:29 smithi157 ceph-mon[35581]: rocksdb: Options.wal_recovery_mode: 2 2024-03-21T10:13:29.412 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:13:29 smithi157 ceph-mon[35581]: rocksdb: Options.enable_thread_tracking: 0 2024-03-21T10:13:29.412 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:13:29 smithi157 ceph-mon[35581]: rocksdb: Options.enable_pipelined_write: 0 2024-03-21T10:13:29.412 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:13:29 smithi157 ceph-mon[35581]: rocksdb: Options.unordered_write: 0 2024-03-21T10:13:29.413 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:13:29 smithi157 ceph-mon[35581]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-03-21T10:13:29.413 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:13:29 smithi157 ceph-mon[35581]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-03-21T10:13:29.413 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:13:29 smithi157 ceph-mon[35581]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-03-21T10:13:29.413 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:13:29 smithi157 ceph-mon[35581]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-03-21T10:13:29.413 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:13:29 smithi157 ceph-mon[35581]: rocksdb: Options.row_cache: None 2024-03-21T10:13:29.413 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:13:29 smithi157 ceph-mon[35581]: rocksdb: Options.wal_filter: None 2024-03-21T10:13:29.413 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:13:29 smithi157 ceph-mon[35581]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-03-21T10:13:29.413 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:13:29 smithi157 ceph-mon[35581]: rocksdb: Options.allow_ingest_behind: 0 2024-03-21T10:13:29.413 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:13:29 smithi157 ceph-mon[35581]: rocksdb: Options.two_write_queues: 0 2024-03-21T10:13:29.413 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:13:29 smithi157 ceph-mon[35581]: rocksdb: Options.manual_wal_flush: 0 2024-03-21T10:13:29.413 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:13:29 smithi157 ceph-mon[35581]: rocksdb: Options.wal_compression: 0 2024-03-21T10:13:29.413 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:13:29 smithi157 ceph-mon[35581]: rocksdb: Options.atomic_flush: 0 2024-03-21T10:13:29.413 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:13:29 smithi157 ceph-mon[35581]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-03-21T10:13:29.413 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:13:29 smithi157 ceph-mon[35581]: rocksdb: Options.persist_stats_to_disk: 0 2024-03-21T10:13:29.414 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:13:29 smithi157 ceph-mon[35581]: rocksdb: Options.write_dbid_to_manifest: 0 2024-03-21T10:13:29.414 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:13:29 smithi157 ceph-mon[35581]: rocksdb: Options.log_readahead_size: 0 2024-03-21T10:13:29.414 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:13:29 smithi157 ceph-mon[35581]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-03-21T10:13:29.414 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:13:29 smithi157 ceph-mon[35581]: rocksdb: Options.best_efforts_recovery: 0 2024-03-21T10:13:29.414 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:13:29 smithi157 ceph-mon[35581]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-03-21T10:13:29.414 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:13:29 smithi157 ceph-mon[35581]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-03-21T10:13:29.414 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:13:29 smithi157 ceph-mon[35581]: rocksdb: Options.allow_data_in_errors: 0 2024-03-21T10:13:29.414 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:13:29 smithi157 ceph-mon[35581]: rocksdb: Options.db_host_id: __hostname__ 2024-03-21T10:13:29.414 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:13:29 smithi157 ceph-mon[35581]: rocksdb: Options.enforce_single_del_contracts: true 2024-03-21T10:13:29.414 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:13:29 smithi157 ceph-mon[35581]: rocksdb: Options.max_background_jobs: 2 2024-03-21T10:13:29.414 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:13:29 smithi157 ceph-mon[35581]: rocksdb: Options.max_background_compactions: -1 2024-03-21T10:13:29.414 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:13:29 smithi157 ceph-mon[35581]: rocksdb: Options.max_subcompactions: 1 2024-03-21T10:13:29.414 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:13:29 smithi157 ceph-mon[35581]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-03-21T10:13:29.415 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:13:29 smithi157 ceph-mon[35581]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-03-21T10:13:29.415 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:13:29 smithi157 ceph-mon[35581]: rocksdb: Options.delayed_write_rate : 16777216 2024-03-21T10:13:29.415 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:13:29 smithi157 ceph-mon[35581]: rocksdb: Options.max_total_wal_size: 0 2024-03-21T10:13:29.415 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:13:29 smithi157 ceph-mon[35581]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-03-21T10:13:29.415 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:13:29 smithi157 ceph-mon[35581]: rocksdb: Options.stats_dump_period_sec: 600 2024-03-21T10:13:29.415 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:13:29 smithi157 ceph-mon[35581]: rocksdb: Options.stats_persist_period_sec: 600 2024-03-21T10:13:29.415 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:13:29 smithi157 ceph-mon[35581]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-03-21T10:13:29.415 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:13:29 smithi157 ceph-mon[35581]: rocksdb: Options.max_open_files: -1 2024-03-21T10:13:29.415 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:13:29 smithi157 ceph-mon[35581]: rocksdb: Options.bytes_per_sync: 0 2024-03-21T10:13:29.415 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:13:29 smithi157 ceph-mon[35581]: rocksdb: Options.wal_bytes_per_sync: 0 2024-03-21T10:13:29.415 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:13:29 smithi157 ceph-mon[35581]: rocksdb: Options.strict_bytes_per_sync: 0 2024-03-21T10:13:29.415 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:13:29 smithi157 ceph-mon[35581]: rocksdb: Options.compaction_readahead_size: 0 2024-03-21T10:13:29.415 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:13:29 smithi157 ceph-mon[35581]: rocksdb: Options.max_background_flushes: -1 2024-03-21T10:13:29.415 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:13:29 smithi157 ceph-mon[35581]: rocksdb: Compression algorithms supported: 2024-03-21T10:13:29.415 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:13:29 smithi157 ceph-mon[35581]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-03-21T10:13:29.416 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:13:29 smithi157 ceph-mon[35581]: rocksdb: kZSTD supported: 0 2024-03-21T10:13:29.416 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:13:29 smithi157 ceph-mon[35581]: rocksdb: kXpressCompression supported: 0 2024-03-21T10:13:29.416 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:13:29 smithi157 ceph-mon[35581]: rocksdb: kLZ4HCCompression supported: 1 2024-03-21T10:13:29.416 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:13:29 smithi157 ceph-mon[35581]: rocksdb: kZlibCompression supported: 1 2024-03-21T10:13:29.416 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:13:29 smithi157 ceph-mon[35581]: rocksdb: kSnappyCompression supported: 1 2024-03-21T10:13:29.416 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:13:29 smithi157 ceph-mon[35581]: rocksdb: kLZ4Compression supported: 1 2024-03-21T10:13:29.416 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:13:29 smithi157 ceph-mon[35581]: rocksdb: kBZip2Compression supported: 0 2024-03-21T10:13:29.416 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:13:29 smithi157 ceph-mon[35581]: rocksdb: Fast CRC32 supported: Supported on x86 2024-03-21T10:13:29.416 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:13:29 smithi157 ceph-mon[35581]: rocksdb: DMutex implementation: pthread_mutex_t 2024-03-21T10:13:29.416 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:13:29 smithi157 ceph-mon[35581]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-c/store.db/MANIFEST-000005 2024-03-21T10:13:29.416 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:13:29 smithi157 ceph-mon[35581]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-03-21T10:13:29.416 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:13:29 smithi157 ceph-mon[35581]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-03-21T10:13:29.416 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:13:29 smithi157 ceph-mon[35581]: rocksdb: Options.merge_operator: 2024-03-21T10:13:29.416 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:13:29 smithi157 ceph-mon[35581]: rocksdb: Options.compaction_filter: None 2024-03-21T10:13:29.416 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:13:29 smithi157 ceph-mon[35581]: rocksdb: Options.compaction_filter_factory: None 2024-03-21T10:13:29.417 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:13:29 smithi157 ceph-mon[35581]: rocksdb: Options.sst_partitioner_factory: None 2024-03-21T10:13:29.417 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:13:29 smithi157 ceph-mon[35581]: rocksdb: Options.memtable_factory: SkipListFactory 2024-03-21T10:13:29.417 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:13:29 smithi157 ceph-mon[35581]: rocksdb: Options.table_factory: BlockBasedTable 2024-03-21T10:13:29.417 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:13:29 smithi157 ceph-mon[35581]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x5623a81e5a60) 2024-03-21T10:13:29.417 INFO:journalctl@ceph.mon.c.smithi157.stdout: cache_index_and_filter_blocks: 1 2024-03-21T10:13:29.417 INFO:journalctl@ceph.mon.c.smithi157.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-03-21T10:13:29.417 INFO:journalctl@ceph.mon.c.smithi157.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-03-21T10:13:29.417 INFO:journalctl@ceph.mon.c.smithi157.stdout: pin_top_level_index_and_filter: 1 2024-03-21T10:13:29.417 INFO:journalctl@ceph.mon.c.smithi157.stdout: index_type: 0 2024-03-21T10:13:29.417 INFO:journalctl@ceph.mon.c.smithi157.stdout: data_block_index_type: 0 2024-03-21T10:13:29.417 INFO:journalctl@ceph.mon.c.smithi157.stdout: index_shortening: 1 2024-03-21T10:13:29.417 INFO:journalctl@ceph.mon.c.smithi157.stdout: data_block_hash_table_util_ratio: 0.750000 2024-03-21T10:13:29.417 INFO:journalctl@ceph.mon.c.smithi157.stdout: checksum: 4 2024-03-21T10:13:29.417 INFO:journalctl@ceph.mon.c.smithi157.stdout: no_block_cache: 0 2024-03-21T10:13:29.418 INFO:journalctl@ceph.mon.c.smithi157.stdout: block_cache: 0x5623a81eb090 2024-03-21T10:13:29.418 INFO:journalctl@ceph.mon.c.smithi157.stdout: block_cache_name: BinnedLRUCache 2024-03-21T10:13:29.418 INFO:journalctl@ceph.mon.c.smithi157.stdout: block_cache_options: 2024-03-21T10:13:29.418 INFO:journalctl@ceph.mon.c.smithi157.stdout: capacity : 536870912 2024-03-21T10:13:29.418 INFO:journalctl@ceph.mon.c.smithi157.stdout: num_shard_bits : 4 2024-03-21T10:13:29.418 INFO:journalctl@ceph.mon.c.smithi157.stdout: strict_capacity_limit : 0 2024-03-21T10:13:29.418 INFO:journalctl@ceph.mon.c.smithi157.stdout: high_pri_pool_ratio: 0.000 2024-03-21T10:13:29.418 INFO:journalctl@ceph.mon.c.smithi157.stdout: block_cache_compressed: (nil) 2024-03-21T10:13:29.418 INFO:journalctl@ceph.mon.c.smithi157.stdout: persistent_cache: (nil) 2024-03-21T10:13:29.418 INFO:journalctl@ceph.mon.c.smithi157.stdout: block_size: 4096 2024-03-21T10:13:29.418 INFO:journalctl@ceph.mon.c.smithi157.stdout: block_size_deviation: 10 2024-03-21T10:13:29.418 INFO:journalctl@ceph.mon.c.smithi157.stdout: block_restart_interval: 16 2024-03-21T10:13:29.418 INFO:journalctl@ceph.mon.c.smithi157.stdout: index_block_restart_interval: 1 2024-03-21T10:13:29.419 INFO:journalctl@ceph.mon.c.smithi157.stdout: metadata_block_size: 4096 2024-03-21T10:13:29.419 INFO:journalctl@ceph.mon.c.smithi157.stdout: partition_filters: 0 2024-03-21T10:13:29.419 INFO:journalctl@ceph.mon.c.smithi157.stdout: use_delta_encoding: 1 2024-03-21T10:13:29.419 INFO:journalctl@ceph.mon.c.smithi157.stdout: filter_policy: bloomfilter 2024-03-21T10:13:29.419 INFO:journalctl@ceph.mon.c.smithi157.stdout: whole_key_filtering: 1 2024-03-21T10:13:29.419 INFO:journalctl@ceph.mon.c.smithi157.stdout: verify_compression: 0 2024-03-21T10:13:29.419 INFO:journalctl@ceph.mon.c.smithi157.stdout: read_amp_bytes_per_bit: 0 2024-03-21T10:13:29.419 INFO:journalctl@ceph.mon.c.smithi157.stdout: format_version: 5 2024-03-21T10:13:29.419 INFO:journalctl@ceph.mon.c.smithi157.stdout: enable_index_compression: 1 2024-03-21T10:13:29.419 INFO:journalctl@ceph.mon.c.smithi157.stdout: block_align: 0 2024-03-21T10:13:29.419 INFO:journalctl@ceph.mon.c.smithi157.stdout: max_auto_readahead_size: 262144 2024-03-21T10:13:29.419 INFO:journalctl@ceph.mon.c.smithi157.stdout: prepopulate_block_cache: 0 2024-03-21T10:13:29.419 INFO:journalctl@ceph.mon.c.smithi157.stdout: initial_auto_readahead_size: 8192 2024-03-21T10:13:29.419 INFO:journalctl@ceph.mon.c.smithi157.stdout: num_file_reads_for_auto_readahead: 2 2024-03-21T10:13:29.419 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:13:29 smithi157 ceph-mon[35581]: rocksdb: Options.write_buffer_size: 33554432 2024-03-21T10:13:29.420 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:13:29 smithi157 ceph-mon[35581]: rocksdb: Options.max_write_buffer_number: 2 2024-03-21T10:13:29.420 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:13:29 smithi157 ceph-mon[35581]: rocksdb: Options.compression: NoCompression 2024-03-21T10:13:29.420 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:13:29 smithi157 ceph-mon[35581]: rocksdb: Options.bottommost_compression: Disabled 2024-03-21T10:13:29.420 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:13:29 smithi157 ceph-mon[35581]: rocksdb: Options.prefix_extractor: nullptr 2024-03-21T10:13:29.420 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:13:29 smithi157 ceph-mon[35581]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-03-21T10:13:29.420 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:13:29 smithi157 ceph-mon[35581]: rocksdb: Options.num_levels: 7 2024-03-21T10:13:29.420 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:13:29 smithi157 ceph-mon[35581]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-03-21T10:13:29.420 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:13:29 smithi157 ceph-mon[35581]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-03-21T10:13:29.420 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:13:29 smithi157 ceph-mon[35581]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-03-21T10:13:29.420 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:13:29 smithi157 ceph-mon[35581]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-03-21T10:13:29.420 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:13:29 smithi157 ceph-mon[35581]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-03-21T10:13:29.420 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:13:29 smithi157 ceph-mon[35581]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-03-21T10:13:29.420 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:13:29 smithi157 ceph-mon[35581]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-03-21T10:13:29.420 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:13:29 smithi157 ceph-mon[35581]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-03-21T10:13:29.420 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:13:29 smithi157 ceph-mon[35581]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-03-21T10:13:29.420 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:13:29 smithi157 ceph-mon[35581]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-03-21T10:13:29.421 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:13:29 smithi157 ceph-mon[35581]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-03-21T10:13:29.421 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:13:29 smithi157 ceph-mon[35581]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-03-21T10:13:29.421 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:13:29 smithi157 ceph-mon[35581]: rocksdb: Options.compression_opts.window_bits: -14 2024-03-21T10:13:29.421 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:13:29 smithi157 ceph-mon[35581]: rocksdb: Options.compression_opts.level: 32767 2024-03-21T10:13:29.421 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:13:29 smithi157 ceph-mon[35581]: rocksdb: Options.compression_opts.strategy: 0 2024-03-21T10:13:29.421 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:13:29 smithi157 ceph-mon[35581]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-03-21T10:13:29.421 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:13:29 smithi157 ceph-mon[35581]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-03-21T10:13:29.421 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:13:29 smithi157 ceph-mon[35581]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-03-21T10:13:29.421 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:13:29 smithi157 ceph-mon[35581]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-03-21T10:13:29.421 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:13:29 smithi157 ceph-mon[35581]: rocksdb: Options.compression_opts.enabled: false 2024-03-21T10:13:29.421 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:13:29 smithi157 ceph-mon[35581]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-03-21T10:13:29.421 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:13:29 smithi157 ceph-mon[35581]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-03-21T10:13:29.421 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:13:29 smithi157 ceph-mon[35581]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-03-21T10:13:29.421 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:13:29 smithi157 ceph-mon[35581]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-03-21T10:13:29.421 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:13:29 smithi157 ceph-mon[35581]: rocksdb: Options.target_file_size_base: 67108864 2024-03-21T10:13:29.422 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:13:29 smithi157 ceph-mon[35581]: rocksdb: Options.target_file_size_multiplier: 1 2024-03-21T10:13:29.422 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:13:29 smithi157 ceph-mon[35581]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-03-21T10:13:29.422 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:13:29 smithi157 ceph-mon[35581]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-03-21T10:13:29.422 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:13:29 smithi157 ceph-mon[35581]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-03-21T10:13:29.422 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:13:29 smithi157 ceph-mon[35581]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-03-21T10:13:29.422 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:13:29 smithi157 ceph-mon[35581]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-03-21T10:13:29.422 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:13:29 smithi157 ceph-mon[35581]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-03-21T10:13:29.422 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:13:29 smithi157 ceph-mon[35581]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-03-21T10:13:29.422 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:13:29 smithi157 ceph-mon[35581]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-03-21T10:13:29.422 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:13:29 smithi157 ceph-mon[35581]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-03-21T10:13:29.422 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:13:29 smithi157 ceph-mon[35581]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-03-21T10:13:29.422 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:13:29 smithi157 ceph-mon[35581]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-03-21T10:13:29.422 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:13:29 smithi157 ceph-mon[35581]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-03-21T10:13:29.422 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:13:29 smithi157 ceph-mon[35581]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-03-21T10:13:29.423 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:13:29 smithi157 ceph-mon[35581]: rocksdb: Options.arena_block_size: 1048576 2024-03-21T10:13:29.423 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:13:29 smithi157 ceph-mon[35581]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-03-21T10:13:29.423 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:13:29 smithi157 ceph-mon[35581]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-03-21T10:13:29.423 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:13:29 smithi157 ceph-mon[35581]: rocksdb: Options.disable_auto_compactions: 0 2024-03-21T10:13:29.423 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:13:29 smithi157 ceph-mon[35581]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-03-21T10:13:29.423 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:13:29 smithi157 ceph-mon[35581]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-03-21T10:13:29.423 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:13:29 smithi157 ceph-mon[35581]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-03-21T10:13:29.423 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:13:29 smithi157 ceph-mon[35581]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-03-21T10:13:29.423 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:13:29 smithi157 ceph-mon[35581]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-03-21T10:13:29.423 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:13:29 smithi157 ceph-mon[35581]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-03-21T10:13:29.423 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:13:29 smithi157 ceph-mon[35581]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-03-21T10:13:29.423 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:13:29 smithi157 ceph-mon[35581]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-03-21T10:13:29.423 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:13:29 smithi157 ceph-mon[35581]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-03-21T10:13:29.423 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:13:29 smithi157 ceph-mon[35581]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-03-21T10:13:29.423 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:13:29 smithi157 ceph-mon[35581]: rocksdb: Options.table_properties_collectors: 2024-03-21T10:13:29.424 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:13:29 smithi157 ceph-mon[35581]: rocksdb: Options.inplace_update_support: 0 2024-03-21T10:13:29.424 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:13:29 smithi157 ceph-mon[35581]: rocksdb: Options.inplace_update_num_locks: 10000 2024-03-21T10:13:29.424 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:13:29 smithi157 ceph-mon[35581]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-03-21T10:13:29.424 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:13:29 smithi157 ceph-mon[35581]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-03-21T10:13:29.424 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:13:29 smithi157 ceph-mon[35581]: rocksdb: Options.memtable_huge_page_size: 0 2024-03-21T10:13:29.424 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:13:29 smithi157 ceph-mon[35581]: rocksdb: Options.bloom_locality: 0 2024-03-21T10:13:29.424 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:13:29 smithi157 ceph-mon[35581]: rocksdb: Options.max_successive_merges: 0 2024-03-21T10:13:29.424 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:13:29 smithi157 ceph-mon[35581]: rocksdb: Options.optimize_filters_for_hits: 0 2024-03-21T10:13:29.424 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:13:29 smithi157 ceph-mon[35581]: rocksdb: Options.paranoid_file_checks: 0 2024-03-21T10:13:29.424 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:13:29 smithi157 ceph-mon[35581]: rocksdb: Options.force_consistency_checks: 1 2024-03-21T10:13:29.424 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:13:29 smithi157 ceph-mon[35581]: rocksdb: Options.report_bg_io_stats: 0 2024-03-21T10:13:29.424 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:13:29 smithi157 ceph-mon[35581]: rocksdb: Options.ttl: 2592000 2024-03-21T10:13:29.424 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:13:29 smithi157 ceph-mon[35581]: rocksdb: Options.periodic_compaction_seconds: 0 2024-03-21T10:13:29.424 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:13:29 smithi157 ceph-mon[35581]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-03-21T10:13:29.425 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:13:29 smithi157 ceph-mon[35581]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-03-21T10:13:29.425 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:13:29 smithi157 ceph-mon[35581]: rocksdb: Options.enable_blob_files: false 2024-03-21T10:13:29.425 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:13:29 smithi157 ceph-mon[35581]: rocksdb: Options.min_blob_size: 0 2024-03-21T10:13:29.425 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:13:29 smithi157 ceph-mon[35581]: rocksdb: Options.blob_file_size: 268435456 2024-03-21T10:13:29.425 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:13:29 smithi157 ceph-mon[35581]: rocksdb: Options.blob_compression_type: NoCompression 2024-03-21T10:13:29.425 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:13:29 smithi157 ceph-mon[35581]: rocksdb: Options.enable_blob_garbage_collection: false 2024-03-21T10:13:29.425 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:13:29 smithi157 ceph-mon[35581]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-03-21T10:13:29.425 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:13:29 smithi157 ceph-mon[35581]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-03-21T10:13:29.425 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:13:29 smithi157 ceph-mon[35581]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-03-21T10:13:29.425 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:13:29 smithi157 ceph-mon[35581]: rocksdb: Options.blob_file_starting_level: 0 2024-03-21T10:13:29.425 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:13:29 smithi157 ceph-mon[35581]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-03-21T10:13:29.425 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:13:29 smithi157 ceph-mon[35581]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-c/store.db/MANIFEST-000005 succeeded,manifest_file_number is 5, next_file_number is 7, last_sequence is 0, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 0 2024-03-21T10:13:29.425 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:13:29 smithi157 ceph-mon[35581]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 0 2024-03-21T10:13:29.425 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:13:29 smithi157 ceph-mon[35581]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: 145bb5d8-c2c6-44db-a4de-78242c33e1f0 2024-03-21T10:13:29.425 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:13:29 smithi157 ceph-mon[35581]: rocksdb: EVENT_LOG_v1 {"time_micros": 1711016009363317, "job": 1, "event": "recovery_started", "wal_files": [4]} 2024-03-21T10:13:29.426 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:13:29 smithi157 ceph-mon[35581]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #4 mode 2 2024-03-21T10:13:29.426 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:13:29 smithi157 ceph-mon[35581]: rocksdb: EVENT_LOG_v1 {"time_micros": 1711016009364231, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 8, "file_size": 1622, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 1, "largest_seqno": 5, "table_properties": {"data_size": 523, "index_size": 31, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 69, "raw_key_size": 115, "raw_average_key_size": 23, "raw_value_size": 401, "raw_average_value_size": 80, "num_data_blocks": 1, "num_entries": 5, "num_filter_entries": 5, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1711016009, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "145bb5d8-c2c6-44db-a4de-78242c33e1f0", "db_session_id": "6TW3GYW9FOUTXZFL45WV", "orig_file_number": 8, "seqno_to_time_mapping": "N/A"}} 2024-03-21T10:13:29.426 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:13:29 smithi157 ceph-mon[35581]: rocksdb: EVENT_LOG_v1 {"time_micros": 1711016009364466, "job": 1, "event": "recovery_finished"} 2024-03-21T10:13:29.426 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:13:29 smithi157 ceph-mon[35581]: rocksdb: [db/version_set.cc:5047] Creating manifest 10 2024-03-21T10:13:29.426 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:13:29 smithi157 ceph-mon[35581]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-c/store.db/000004.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-03-21T10:13:29.426 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:13:29 smithi157 ceph-mon[35581]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x5623a82de000 2024-03-21T10:13:29.426 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:13:29 smithi157 ceph-mon[35581]: rocksdb: DB pointer 0x5623a82c8000 2024-03-21T10:13:29.426 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:13:29 smithi157 ceph-mon[35581]: mon.c does not exist in monmap, will attempt to join an existing cluster 2024-03-21T10:13:29.426 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:13:29 smithi157 ceph-mon[35581]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-03-21T10:13:29.426 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:13:29 smithi157 ceph-mon[35581]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-03-21T10:13:29.426 INFO:journalctl@ceph.mon.c.smithi157.stdout: ** DB Stats ** 2024-03-21T10:13:29.426 INFO:journalctl@ceph.mon.c.smithi157.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-03-21T10:13:29.426 INFO:journalctl@ceph.mon.c.smithi157.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-03-21T10:13:29.427 INFO:journalctl@ceph.mon.c.smithi157.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-03-21T10:13:29.427 INFO:journalctl@ceph.mon.c.smithi157.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-03-21T10:13:29.427 INFO:journalctl@ceph.mon.c.smithi157.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-03-21T10:13:29.427 INFO:journalctl@ceph.mon.c.smithi157.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-03-21T10:13:29.427 INFO:journalctl@ceph.mon.c.smithi157.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-03-21T10:13:29.427 INFO:journalctl@ceph.mon.c.smithi157.stdout: 2024-03-21T10:13:29.427 INFO:journalctl@ceph.mon.c.smithi157.stdout: ** Compaction Stats [default] ** 2024-03-21T10:13:29.427 INFO:journalctl@ceph.mon.c.smithi157.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-03-21T10:13:29.427 INFO:journalctl@ceph.mon.c.smithi157.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-03-21T10:13:29.427 INFO:journalctl@ceph.mon.c.smithi157.stdout: L0 1/0 1.58 KB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 1.8 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-03-21T10:13:29.427 INFO:journalctl@ceph.mon.c.smithi157.stdout: Sum 1/0 1.58 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 1.8 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-03-21T10:13:29.427 INFO:journalctl@ceph.mon.c.smithi157.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 1.8 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-03-21T10:13:29.427 INFO:journalctl@ceph.mon.c.smithi157.stdout: 2024-03-21T10:13:29.427 INFO:journalctl@ceph.mon.c.smithi157.stdout: ** Compaction Stats [default] ** 2024-03-21T10:13:29.427 INFO:journalctl@ceph.mon.c.smithi157.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-03-21T10:13:29.428 INFO:journalctl@ceph.mon.c.smithi157.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-03-21T10:13:29.428 INFO:journalctl@ceph.mon.c.smithi157.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.8 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-03-21T10:13:29.428 INFO:journalctl@ceph.mon.c.smithi157.stdout: 2024-03-21T10:13:29.428 INFO:journalctl@ceph.mon.c.smithi157.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-03-21T10:13:29.428 INFO:journalctl@ceph.mon.c.smithi157.stdout: 2024-03-21T10:13:29.428 INFO:journalctl@ceph.mon.c.smithi157.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-03-21T10:13:29.428 INFO:journalctl@ceph.mon.c.smithi157.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-03-21T10:13:29.428 INFO:journalctl@ceph.mon.c.smithi157.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-03-21T10:13:29.428 INFO:journalctl@ceph.mon.c.smithi157.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-03-21T10:13:29.428 INFO:journalctl@ceph.mon.c.smithi157.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-03-21T10:13:29.428 INFO:journalctl@ceph.mon.c.smithi157.stdout: AddFile(Keys): cumulative 0, interval 0 2024-03-21T10:13:29.428 INFO:journalctl@ceph.mon.c.smithi157.stdout: Cumulative compaction: 0.00 GB write, 0.25 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-03-21T10:13:29.428 INFO:journalctl@ceph.mon.c.smithi157.stdout: Interval compaction: 0.00 GB write, 0.25 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-03-21T10:13:29.428 INFO:journalctl@ceph.mon.c.smithi157.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-03-21T10:13:29.428 INFO:journalctl@ceph.mon.c.smithi157.stdout: Block cache BinnedLRUCache@0x5623a81eb090#2 capacity: 512.00 MB usage: 0.86 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 2.5e-05 secs_since: 0 2024-03-21T10:13:29.429 INFO:journalctl@ceph.mon.c.smithi157.stdout: Block cache entry stats(count,size,portion): DataBlock(1,0.64 KB,0.00012219%) FilterBlock(1,0.11 KB,2.08616e-05%) IndexBlock(1,0.11 KB,2.08616e-05%) Misc(1,0.00 KB,0%) 2024-03-21T10:13:29.429 INFO:journalctl@ceph.mon.c.smithi157.stdout: 2024-03-21T10:13:29.429 INFO:journalctl@ceph.mon.c.smithi157.stdout: ** File Read Latency Histogram By Level [default] ** 2024-03-21T10:13:29.429 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:13:29 smithi157 ceph-mon[35581]: using public_addr v2:172.21.15.157:0/0 -> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] 2024-03-21T10:13:29.429 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:13:29 smithi157 ceph-mon[35581]: starting mon.c rank -1 at public addrs [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] at bind addrs [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] mon_data /var/lib/ceph/mon/ceph-c fsid 3056d51a-e76b-11ee-95cd-87774f69a715 2024-03-21T10:13:29.429 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:13:29 smithi157 ceph-mon[35581]: mon.c@-1(???) e0 preinit fsid 3056d51a-e76b-11ee-95cd-87774f69a715 2024-03-21T10:13:29.429 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:13:29 smithi157 ceph-mon[35581]: mon.c@-1(synchronizing).mds e1 new map 2024-03-21T10:13:29.429 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:13:29 smithi157 ceph-mon[35581]: mon.c@-1(synchronizing).mds e1 print_map 2024-03-21T10:13:29.429 INFO:journalctl@ceph.mon.c.smithi157.stdout: e1 2024-03-21T10:13:29.429 INFO:journalctl@ceph.mon.c.smithi157.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-03-21T10:13:29.429 INFO:journalctl@ceph.mon.c.smithi157.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments} 2024-03-21T10:13:29.429 INFO:journalctl@ceph.mon.c.smithi157.stdout: legacy client fscid: -1 2024-03-21T10:13:29.429 INFO:journalctl@ceph.mon.c.smithi157.stdout: 2024-03-21T10:13:29.430 INFO:journalctl@ceph.mon.c.smithi157.stdout: No filesystems configured 2024-03-21T10:13:29.430 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:13:29 smithi157 ceph-mon[35581]: mon.c@-1(synchronizing).osd e0 _set_cache_ratios kv ratio 0.25 inc ratio 0.375 full ratio 0.375 2024-03-21T10:13:29.430 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:13:29 smithi157 ceph-mon[35581]: mon.c@-1(synchronizing).osd e0 register_cache_with_pcm pcm target: 2147483648 pcm max: 1020054732 pcm min: 134217728 inc_osd_cache size: 1 2024-03-21T10:13:29.430 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:13:29 smithi157 ceph-mon[35581]: mon.c@-1(synchronizing).osd e1 e1: 0 total, 0 up, 0 in 2024-03-21T10:13:29.430 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:13:29 smithi157 ceph-mon[35581]: mon.c@-1(synchronizing).osd e2 e2: 0 total, 0 up, 0 in 2024-03-21T10:13:29.430 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:13:29 smithi157 ceph-mon[35581]: mon.c@-1(synchronizing).osd e3 e3: 0 total, 0 up, 0 in 2024-03-21T10:13:29.430 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:13:29 smithi157 ceph-mon[35581]: mon.c@-1(synchronizing).osd e4 e4: 0 total, 0 up, 0 in 2024-03-21T10:13:29.430 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:13:29 smithi157 ceph-mon[35581]: mon.c@-1(synchronizing).osd e4 crush map has features 3314932999778484224, adjusting msgr requires 2024-03-21T10:13:29.430 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:13:29 smithi157 ceph-mon[35581]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-03-21T10:13:29.430 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:13:29 smithi157 ceph-mon[35581]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-03-21T10:13:29.430 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:13:29 smithi157 ceph-mon[35581]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-03-21T10:13:29.430 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:13:29 smithi157 ceph-mon[35581]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T10:13:29.430 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:13:29 smithi157 ceph-mon[35581]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:13:29.430 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:13:29 smithi157 ceph-mon[35581]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T10:13:29.431 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:13:29 smithi157 ceph-mon[35581]: Added host smithi100 2024-03-21T10:13:29.431 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:13:29 smithi157 ceph-mon[35581]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:13:29.431 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:13:29 smithi157 ceph-mon[35581]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T10:13:29.431 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:13:29 smithi157 ceph-mon[35581]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:13:29.431 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:13:29 smithi157 ceph-mon[35581]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T10:13:29.431 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:13:29 smithi157 ceph-mon[35581]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-21T10:13:29.432 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:13:29 smithi157 ceph-mon[35581]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:13:29.432 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:13:29 smithi157 ceph-mon[35581]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T10:13:29.432 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:13:29 smithi157 ceph-mon[35581]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T10:13:29.432 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:13:29 smithi157 ceph-mon[35581]: from='client.14182 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi157", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:13:29.432 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:13:29 smithi157 ceph-mon[35581]: Deploying cephadm binary to smithi157 2024-03-21T10:13:29.432 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:13:29 smithi157 ceph-mon[35581]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:13:29.432 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:13:29 smithi157 ceph-mon[35581]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:13:29.432 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:13:29 smithi157 ceph-mon[35581]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:13:29.432 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:13:29 smithi157 ceph-mon[35581]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:13:29.432 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:13:29 smithi157 ceph-mon[35581]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi100", "name": "osd_memory_target"}]: dispatch 2024-03-21T10:13:29.432 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:13:29 smithi157 ceph-mon[35581]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T10:13:29.432 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:13:29 smithi157 ceph-mon[35581]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T10:13:29.432 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:13:29 smithi157 ceph-mon[35581]: Updating smithi100:/etc/ceph/ceph.conf 2024-03-21T10:13:29.432 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:13:29 smithi157 ceph-mon[35581]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T10:13:29.433 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:13:29 smithi157 ceph-mon[35581]: Updating smithi100:/var/lib/ceph/3056d51a-e76b-11ee-95cd-87774f69a715/config/ceph.conf 2024-03-21T10:13:29.433 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:13:29 smithi157 ceph-mon[35581]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T10:13:29.433 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:13:29 smithi157 ceph-mon[35581]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T10:13:29.433 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:13:29 smithi157 ceph-mon[35581]: Updating smithi100:/etc/ceph/ceph.client.admin.keyring 2024-03-21T10:13:29.433 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:13:29 smithi157 ceph-mon[35581]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:13:29.433 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:13:29 smithi157 ceph-mon[35581]: Updating smithi100:/var/lib/ceph/3056d51a-e76b-11ee-95cd-87774f69a715/config/ceph.client.admin.keyring 2024-03-21T10:13:29.433 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:13:29 smithi157 ceph-mon[35581]: Added host smithi157 2024-03-21T10:13:29.433 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:13:29 smithi157 ceph-mon[35581]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:13:29.433 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:13:29 smithi157 ceph-mon[35581]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:13:29.433 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:13:29 smithi157 ceph-mon[35581]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:13:29.433 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:13:29 smithi157 ceph-mon[35581]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T10:13:29.433 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:13:29 smithi157 ceph-mon[35581]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T10:13:29.433 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:13:29 smithi157 ceph-mon[35581]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:13:29.433 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:13:29 smithi157 ceph-mon[35581]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T10:13:29.434 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:13:29 smithi157 ceph-mon[35581]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:13:29.434 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:13:29 smithi157 ceph-mon[35581]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-21T10:13:29.434 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:13:29 smithi157 ceph-mon[35581]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T10:13:29.434 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:13:29 smithi157 ceph-mon[35581]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:13:29.434 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:13:29 smithi157 ceph-mon[35581]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T10:13:29.434 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:13:29 smithi157 ceph-mon[35581]: from='client.? 172.21.15.47:0/1166569903' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-03-21T10:13:29.434 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:13:29 smithi157 ceph-mon[35581]: from='client.? 172.21.15.47:0/1166569903' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-03-21T10:13:29.434 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:13:29 smithi157 ceph-mon[35581]: osdmap e4: 0 total, 0 up, 0 in 2024-03-21T10:13:29.434 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:13:29 smithi157 ceph-mon[35581]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T10:13:29.434 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:13:29 smithi157 ceph-mon[35581]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:13:29.434 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:13:29 smithi157 ceph-mon[35581]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:13:29.434 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:13:29 smithi157 ceph-mon[35581]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:13:29.434 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:13:29 smithi157 ceph-mon[35581]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:13:29.434 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:13:29 smithi157 ceph-mon[35581]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi157", "name": "osd_memory_target"}]: dispatch 2024-03-21T10:13:29.435 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:13:29 smithi157 ceph-mon[35581]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T10:13:29.435 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:13:29 smithi157 ceph-mon[35581]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T10:13:29.435 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:13:29 smithi157 ceph-mon[35581]: Updating smithi157:/etc/ceph/ceph.conf 2024-03-21T10:13:29.435 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:13:29 smithi157 ceph-mon[35581]: Updating smithi157:/var/lib/ceph/3056d51a-e76b-11ee-95cd-87774f69a715/config/ceph.conf 2024-03-21T10:13:29.435 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:13:29 smithi157 ceph-mon[35581]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T10:13:29.435 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:13:29 smithi157 ceph-mon[35581]: Updating smithi157:/etc/ceph/ceph.client.admin.keyring 2024-03-21T10:13:29.435 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:13:29 smithi157 ceph-mon[35581]: Updating smithi157:/var/lib/ceph/3056d51a-e76b-11ee-95cd-87774f69a715/config/ceph.client.admin.keyring 2024-03-21T10:13:29.435 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:13:29 smithi157 ceph-mon[35581]: from='client.14188 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi047:172.21.15.47=a;smithi100:172.21.15.100=b;smithi157:172.21.15.157=c", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:13:29.435 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:13:29 smithi157 ceph-mon[35581]: Saving service mon spec with placement smithi047:172.21.15.47=a;smithi100:172.21.15.100=b;smithi157:172.21.15.157=c;count:3 2024-03-21T10:13:29.435 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:13:29 smithi157 ceph-mon[35581]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:13:29.435 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:13:29 smithi157 ceph-mon[35581]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:13:29.435 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:13:29 smithi157 ceph-mon[35581]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:13:29.435 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:13:29 smithi157 ceph-mon[35581]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:13:29.435 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:13:29 smithi157 ceph-mon[35581]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-21T10:13:29.436 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:13:29 smithi157 ceph-mon[35581]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T10:13:29.436 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:13:29 smithi157 ceph-mon[35581]: Deploying daemon mon.c on smithi157 2024-03-21T10:13:29.436 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:13:29 smithi157 ceph-mon[35581]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T10:13:29.436 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:13:29 smithi157 ceph-mon[35581]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T10:13:29.436 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:13:29 smithi157 ceph-mon[35581]: from='client.? 172.21.15.157:0/146512500' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-03-21T10:13:29.436 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:13:29 smithi157 ceph-mon[35581]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T10:13:29.436 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:13:29 smithi157 ceph-mon[35581]: mon.c@-1(synchronizing).paxosservice(auth 1..3) refresh upgraded, format 0 -> 3 2024-03-21T10:13:29.717 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:13:29 smithi157 bash[35543]: d9e77d928b1edb6585269826c3f06d8b28c3edaf426be89e4bff287c4b2551af 2024-03-21T10:13:29.718 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:13:29 smithi157 systemd[1]: Started Ceph mon.c for 3056d51a-e76b-11ee-95cd-87774f69a715. 2024-03-21T10:13:34.964 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:13:34 smithi047 ceph-mon[31547]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-21T10:13:34.964 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:13:34 smithi047 ceph-mon[31547]: mon.a calling monitor election 2024-03-21T10:13:34.964 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:13:34 smithi047 ceph-mon[31547]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-21T10:13:34.965 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:13:34 smithi047 ceph-mon[31547]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T10:13:34.965 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:13:34 smithi047 ceph-mon[31547]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-21T10:13:34.965 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:13:34 smithi047 ceph-mon[31547]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-21T10:13:34.965 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:13:34 smithi047 ceph-mon[31547]: mon.c calling monitor election 2024-03-21T10:13:34.965 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:13:34 smithi047 ceph-mon[31547]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T10:13:34.965 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:13:34 smithi047 ceph-mon[31547]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-21T10:13:34.965 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:13:34 smithi047 ceph-mon[31547]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-21T10:13:34.965 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:13:34 smithi047 ceph-mon[31547]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-21T10:13:34.965 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:13:34 smithi047 ceph-mon[31547]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-03-21T10:13:34.965 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:13:34 smithi047 ceph-mon[31547]: monmap e2: 2 mons at {a=[v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0],c=[v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-03-21T10:13:34.965 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:13:34 smithi047 ceph-mon[31547]: fsmap 2024-03-21T10:13:34.965 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:13:34 smithi047 ceph-mon[31547]: osdmap e4: 0 total, 0 up, 0 in 2024-03-21T10:13:34.965 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:13:34 smithi047 ceph-mon[31547]: mgrmap e13: x(active, since 68s) 2024-03-21T10:13:34.965 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:13:34 smithi047 ceph-mon[31547]: overall HEALTH_OK 2024-03-21T10:13:34.965 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:13:34 smithi047 ceph-mon[31547]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:13:34.966 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:13:34 smithi047 ceph-mon[31547]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:13:34.990 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:13:34 smithi157 ceph-mon[35581]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-21T10:13:34.990 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:13:34 smithi157 ceph-mon[35581]: mon.a calling monitor election 2024-03-21T10:13:34.990 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:13:34 smithi157 ceph-mon[35581]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-21T10:13:34.990 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:13:34 smithi157 ceph-mon[35581]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T10:13:34.991 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:13:34 smithi157 ceph-mon[35581]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-21T10:13:34.991 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:13:34 smithi157 ceph-mon[35581]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-21T10:13:34.991 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:13:34 smithi157 ceph-mon[35581]: mon.c calling monitor election 2024-03-21T10:13:34.991 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:13:34 smithi157 ceph-mon[35581]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T10:13:34.991 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:13:34 smithi157 ceph-mon[35581]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-21T10:13:34.991 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:13:34 smithi157 ceph-mon[35581]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-21T10:13:34.991 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:13:34 smithi157 ceph-mon[35581]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-21T10:13:34.991 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:13:34 smithi157 ceph-mon[35581]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-03-21T10:13:34.991 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:13:34 smithi157 ceph-mon[35581]: monmap e2: 2 mons at {a=[v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0],c=[v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-03-21T10:13:34.991 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:13:34 smithi157 ceph-mon[35581]: fsmap 2024-03-21T10:13:34.991 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:13:34 smithi157 ceph-mon[35581]: osdmap e4: 0 total, 0 up, 0 in 2024-03-21T10:13:34.991 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:13:34 smithi157 ceph-mon[35581]: mgrmap e13: x(active, since 68s) 2024-03-21T10:13:34.991 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:13:34 smithi157 ceph-mon[35581]: overall HEALTH_OK 2024-03-21T10:13:34.991 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:13:34 smithi157 ceph-mon[35581]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:13:34.992 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:13:34 smithi157 ceph-mon[35581]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:13:35.886 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-21T10:13:35.883+0000 7f4ad6000700 1 -- 172.21.15.157:0/3196642561 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4ab8006230 msgr2=0x7f4ab80086a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:13:35.886 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-21T10:13:35.883+0000 7f4ad6000700 1 --2- 172.21.15.157:0/3196642561 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4ab8006230 0x7f4ab80086a0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f4ad0075590 tx=0x7f4ac400fe90 comp rx=0 tx=0).stop 2024-03-21T10:13:35.887 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-21T10:13:35.883+0000 7f4ad6000700 1 -- 172.21.15.157:0/3196642561 shutdown_connections 2024-03-21T10:13:35.887 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-21T10:13:35.883+0000 7f4ad6000700 1 --2- 172.21.15.157:0/3196642561 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4ab8006230 0x7f4ab80086a0 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:13:35.887 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-21T10:13:35.883+0000 7f4ad6000700 1 -- 172.21.15.157:0/3196642561 >> 172.21.15.157:0/3196642561 conn(0x7f4ad00fd5f0 msgr2=0x7f4ad00ffa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:13:35.887 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-21T10:13:35.884+0000 7f4ad6000700 1 -- 172.21.15.157:0/3196642561 shutdown_connections 2024-03-21T10:13:35.887 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-21T10:13:35.884+0000 7f4ad6000700 1 -- 172.21.15.157:0/3196642561 wait complete. 2024-03-21T10:13:35.887 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:13:35 smithi157 ceph-mon[35581]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T10:13:35.887 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:13:35 smithi157 ceph-mon[35581]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:13:35.887 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:13:35 smithi157 ceph-mon[35581]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-21T10:13:35.887 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:13:35 smithi157 ceph-mon[35581]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T10:13:35.887 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:13:35 smithi157 ceph-mon[35581]: Deploying daemon mon.b on smithi100 2024-03-21T10:13:35.887 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:13:35 smithi157 ceph-mon[35581]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-21T10:13:35.888 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-21T10:13:35.887+0000 7f4ad6000700 1 Processor -- start 2024-03-21T10:13:35.888 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-21T10:13:35.887+0000 7f4ad6000700 1 -- start start 2024-03-21T10:13:35.890 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-21T10:13:35.888+0000 7f4ad6000700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4ab8006230 0x7f4ad019f600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:13:35.890 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-21T10:13:35.889+0000 7f4ad6000700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4ad019fb40 0x7f4ad01996d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:13:35.890 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-21T10:13:35.889+0000 7f4ad6000700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ad010bc30 con 0x7f4ab8006230 2024-03-21T10:13:35.890 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-21T10:13:35.889+0000 7f4ad6000700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ad010bab0 con 0x7f4ad019fb40 2024-03-21T10:13:35.891 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-21T10:13:35.889+0000 7f4ad4c00700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4ab8006230 0x7f4ad019f600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:13:35.891 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-21T10:13:35.889+0000 7f4ad4c00700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4ab8006230 0x7f4ad019f600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.157:44664/0 (socket says 172.21.15.157:44664) 2024-03-21T10:13:35.891 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-21T10:13:35.889+0000 7f4ad4c00700 1 -- 172.21.15.157:0/4107451876 learned_addr learned my addr 172.21.15.157:0/4107451876 (peer_addr_for_me v2:172.21.15.157:0/0) 2024-03-21T10:13:35.891 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-21T10:13:35.890+0000 7f4acfe00700 1 --2- 172.21.15.157:0/4107451876 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4ad019fb40 0x7f4ad01996d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:13:35.891 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-21T10:13:35.890+0000 7f4acfe00700 1 -- 172.21.15.157:0/4107451876 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4ad019fb40 msgr2=0x7f4ad01996d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 13 2024-03-21T10:13:35.891 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-21T10:13:35.890+0000 7f4acfe00700 1 -- 172.21.15.157:0/4107451876 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4ad019fb40 msgr2=0x7f4ad01996d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-03-21T10:13:35.892 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-21T10:13:35.890+0000 7f4acfe00700 1 --2- 172.21.15.157:0/4107451876 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4ad019fb40 0x7f4ad01996d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-21T10:13:35.892 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-21T10:13:35.890+0000 7f4acfe00700 1 --2- 172.21.15.157:0/4107451876 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4ad019fb40 0x7f4ad01996d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-21T10:13:35.892 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-21T10:13:35.890+0000 7f4ad4c00700 1 -- 172.21.15.157:0/4107451876 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4ad019fb40 msgr2=0x7f4ad01996d0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-21T10:13:35.892 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-21T10:13:35.890+0000 7f4ad4c00700 1 --2- 172.21.15.157:0/4107451876 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4ad019fb40 0x7f4ad01996d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:13:35.892 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-21T10:13:35.890+0000 7f4ad4c00700 1 -- 172.21.15.157:0/4107451876 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ad01a00c0 con 0x7f4ab8006230 2024-03-21T10:13:35.892 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-21T10:13:35.891+0000 7f4ad4c00700 1 --2- 172.21.15.157:0/4107451876 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4ab8006230 0x7f4ad019f600 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f4ac401d9b0 tx=0x7f4ac401da90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:13:35.893 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-21T10:13:35.891+0000 7f4acd600700 1 -- 172.21.15.157:0/4107451876 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 309+0+0 (secure 0 0 0) 0x7f4ac4005400 con 0x7f4ab8006230 2024-03-21T10:13:35.893 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-21T10:13:35.891+0000 7f4acd600700 1 -- 172.21.15.157:0/4107451876 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f4ac40055a0 con 0x7f4ab8006230 2024-03-21T10:13:35.893 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-21T10:13:35.891+0000 7f4ad6000700 1 -- 172.21.15.157:0/4107451876 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4ad0199c70 con 0x7f4ab8006230 2024-03-21T10:13:35.893 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-21T10:13:35.891+0000 7f4acd600700 1 -- 172.21.15.157:0/4107451876 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 309+0+0 (secure 0 0 0) 0x7f4ac4011880 con 0x7f4ab8006230 2024-03-21T10:13:35.893 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-21T10:13:35.891+0000 7f4ad6000700 1 -- 172.21.15.157:0/4107451876 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4ad019a200 con 0x7f4ab8006230 2024-03-21T10:13:35.894 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-21T10:13:35.893+0000 7f4ad6000700 1 -- 172.21.15.157:0/4107451876 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4ad0075c10 con 0x7f4ab8006230 2024-03-21T10:13:35.894 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-21T10:13:35.893+0000 7f4acd600700 1 -- 172.21.15.157:0/4107451876 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 13) v1 ==== 46350+0+0 (secure 0 0 0) 0x7f4ac4011a20 con 0x7f4ab8006230 2024-03-21T10:13:35.895 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-21T10:13:35.894+0000 7f4acd600700 1 --2- 172.21.15.157:0/4107451876 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f4ab4039010 0x7f4ab403b4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:13:35.895 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-21T10:13:35.894+0000 7f4acd600700 1 -- 172.21.15.157:0/4107451876 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f4ac404f2e0 con 0x7f4ab8006230 2024-03-21T10:13:35.896 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-21T10:13:35.894+0000 7f4acfe00700 1 --2- 172.21.15.157:0/4107451876 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f4ab4039010 0x7f4ab403b4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:13:35.897 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-21T10:13:35.896+0000 7f4acfe00700 1 --2- 172.21.15.157:0/4107451876 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f4ab4039010 0x7f4ab403b4d0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f4ac0009850 tx=0x7f4ac0008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:13:35.911 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-21T10:13:35.910+0000 7f4acd600700 1 -- 172.21.15.157:0/4107451876 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f4ac402dd70 con 0x7f4ab8006230 2024-03-21T10:13:35.964 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:13:35 smithi047 ceph-mon[31547]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T10:13:35.964 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:13:35 smithi047 ceph-mon[31547]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:13:35.964 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:13:35 smithi047 ceph-mon[31547]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-21T10:13:35.964 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:13:35 smithi047 ceph-mon[31547]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T10:13:35.964 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:13:35 smithi047 ceph-mon[31547]: Deploying daemon mon.b on smithi100 2024-03-21T10:13:35.964 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:13:35 smithi047 ceph-mon[31547]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-21T10:13:36.228 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-21T10:13:36.227+0000 7f4ad6000700 1 -- 172.21.15.157:0/4107451876 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f4ad019a510 con 0x7f4ab8006230 2024-03-21T10:13:36.229 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-21T10:13:36.228+0000 7f4acd600700 1 -- 172.21.15.157:0/4107451876 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 2 v2) v1 ==== 95+0+1015 (secure 0 0 0) 0x7f4ac4011d10 con 0x7f4ab8006230 2024-03-21T10:13:36.230 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-03-21T10:13:36.230 INFO:teuthology.orchestra.run.smithi157.stdout:{"epoch":2,"fsid":"3056d51a-e76b-11ee-95cd-87774f69a715","modified":"2024-03-21T10:13:29.416624Z","created":"2024-03-21T10:11:12.872646Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:3300","nonce":0},{"type":"v1","addr":"172.21.15.47:6789","nonce":0}]},"addr":"172.21.15.47:6789/0","public_addr":"172.21.15.47:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:3300","nonce":0},{"type":"v1","addr":"172.21.15.157:6789","nonce":0}]},"addr":"172.21.15.157:6789/0","public_addr":"172.21.15.157:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-03-21T10:13:36.233 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-21T10:13:36.232+0000 7f4ad6000700 1 -- 172.21.15.157:0/4107451876 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f4ab4039010 msgr2=0x7f4ab403b4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:13:36.234 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-21T10:13:36.232+0000 7f4ad6000700 1 --2- 172.21.15.157:0/4107451876 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f4ab4039010 0x7f4ab403b4d0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f4ac0009850 tx=0x7f4ac0008040 comp rx=0 tx=0).stop 2024-03-21T10:13:36.234 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-21T10:13:36.232+0000 7f4ad6000700 1 -- 172.21.15.157:0/4107451876 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4ab8006230 msgr2=0x7f4ad019f600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:13:36.234 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-21T10:13:36.232+0000 7f4ad6000700 1 --2- 172.21.15.157:0/4107451876 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4ab8006230 0x7f4ad019f600 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f4ac401d9b0 tx=0x7f4ac401da90 comp rx=0 tx=0).stop 2024-03-21T10:13:36.234 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-21T10:13:36.232+0000 7f4ad6000700 1 -- 172.21.15.157:0/4107451876 shutdown_connections 2024-03-21T10:13:36.234 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-21T10:13:36.232+0000 7f4ad6000700 1 --2- 172.21.15.157:0/4107451876 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f4ab4039010 0x7f4ab403b4d0 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:13:36.234 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-21T10:13:36.232+0000 7f4ad6000700 1 --2- 172.21.15.157:0/4107451876 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4ab8006230 0x7f4ad019f600 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:13:36.235 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-21T10:13:36.232+0000 7f4ad6000700 1 --2- 172.21.15.157:0/4107451876 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4ad019fb40 0x7f4ad01996d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:13:36.235 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-21T10:13:36.232+0000 7f4ad6000700 1 -- 172.21.15.157:0/4107451876 >> 172.21.15.157:0/4107451876 conn(0x7f4ad00fd5f0 msgr2=0x7f4ad01921f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:13:36.235 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-21T10:13:36.232+0000 7f4ad6000700 1 -- 172.21.15.157:0/4107451876 shutdown_connections 2024-03-21T10:13:36.235 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-21T10:13:36.232+0000 7f4ad6000700 1 -- 172.21.15.157:0/4107451876 wait complete. 2024-03-21T10:13:36.235 INFO:teuthology.orchestra.run.smithi157.stderr:dumped monmap epoch 2 2024-03-21T10:13:36.884 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:13:36 smithi157 ceph-mon[35581]: from='client.? 172.21.15.157:0/4107451876' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-03-21T10:13:36.964 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:13:36 smithi047 ceph-mon[31547]: from='client.? 172.21.15.157:0/4107451876' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-03-21T10:13:37.952 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-03-21T10:13:37.953 DEBUG:teuthology.orchestra.run.smithi157:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2edfee68ad7243bed422a049b6f5f58619300f11 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3056d51a-e76b-11ee-95cd-87774f69a715 -- ceph mon dump -f json 2024-03-21T10:13:37.963 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:13:37 smithi047 ceph-mon[31547]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T10:13:37.989 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:13:37 smithi157 ceph-mon[35581]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T10:13:38.391 INFO:teuthology.orchestra.run.smithi157.stderr:Inferring config /var/lib/ceph/3056d51a-e76b-11ee-95cd-87774f69a715/mon.c/config 2024-03-21T10:13:39.740 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:13:39 smithi157 ceph-mon[35581]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T10:13:39.772 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:13:39 smithi100 podman[35175]: 2024-03-21 10:13:39.754832901 +0000 UTC m=+0.341187604 container init 5aa332c64f454ab905946166832a26da9695527b9633cf3a9379bdabea331d23 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-3056d51a-e76b-11ee-95cd-87774f69a715-mon-b, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.schema-version=1.0, ceph=True, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240319, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-21T10:13:39.772 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:13:39 smithi100 podman[35175]: 2024-03-21 10:13:39.771268829 +0000 UTC m=+0.357623511 container start 5aa332c64f454ab905946166832a26da9695527b9633cf3a9379bdabea331d23 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-3056d51a-e76b-11ee-95cd-87774f69a715-mon-b, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.build-date=20240319, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_CLEAN=True) 2024-03-21T10:13:39.963 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:13:39 smithi047 ceph-mon[31547]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T10:13:40.231 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:13:39 smithi100 ceph-mon[35193]: set uid:gid to 167:167 (ceph:ceph) 2024-03-21T10:13:40.231 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:13:39 smithi100 ceph-mon[35193]: ceph version 19.0.0-2231-g2edfee68 (2edfee68ad7243bed422a049b6f5f58619300f11) squid (dev), process ceph-mon, pid 2 2024-03-21T10:13:40.231 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:13:39 smithi100 ceph-mon[35193]: pidfile_write: ignore empty --pid-file 2024-03-21T10:13:40.231 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:13:39 smithi100 ceph-mon[35193]: load: jerasure load: lrc 2024-03-21T10:13:40.231 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:13:39 smithi100 ceph-mon[35193]: rocksdb: RocksDB version: 7.9.2 2024-03-21T10:13:40.231 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:13:39 smithi100 ceph-mon[35193]: rocksdb: Git sha 0 2024-03-21T10:13:40.231 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:13:39 smithi100 ceph-mon[35193]: rocksdb: Compile date 2024-03-20 06:23:48 2024-03-21T10:13:40.231 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:13:39 smithi100 ceph-mon[35193]: rocksdb: DB SUMMARY 2024-03-21T10:13:40.231 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:13:39 smithi100 ceph-mon[35193]: rocksdb: DB Session ID: W4BBA9E2HJ4IO7A2RR8W 2024-03-21T10:13:40.231 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:13:39 smithi100 ceph-mon[35193]: rocksdb: CURRENT file: CURRENT 2024-03-21T10:13:40.231 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:13:39 smithi100 ceph-mon[35193]: rocksdb: IDENTITY file: IDENTITY 2024-03-21T10:13:40.232 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:13:39 smithi100 ceph-mon[35193]: rocksdb: MANIFEST file: MANIFEST-000005 size: 59 Bytes 2024-03-21T10:13:40.232 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:13:39 smithi100 ceph-mon[35193]: rocksdb: SST files in /var/lib/ceph/mon/ceph-b/store.db dir, Total Num: 0, files: 2024-03-21T10:13:40.232 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:13:39 smithi100 ceph-mon[35193]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-b/store.db: 000004.log size: 636 ; 2024-03-21T10:13:40.232 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:13:39 smithi100 ceph-mon[35193]: rocksdb: Options.error_if_exists: 0 2024-03-21T10:13:40.232 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:13:39 smithi100 ceph-mon[35193]: rocksdb: Options.create_if_missing: 0 2024-03-21T10:13:40.232 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:13:39 smithi100 ceph-mon[35193]: rocksdb: Options.paranoid_checks: 1 2024-03-21T10:13:40.232 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:13:39 smithi100 ceph-mon[35193]: rocksdb: Options.flush_verify_memtable_count: 1 2024-03-21T10:13:40.232 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:13:39 smithi100 ceph-mon[35193]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-03-21T10:13:40.232 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:13:39 smithi100 ceph-mon[35193]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-03-21T10:13:40.232 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:13:39 smithi100 ceph-mon[35193]: rocksdb: Options.env: 0x55e69e4e8b20 2024-03-21T10:13:40.232 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:13:39 smithi100 ceph-mon[35193]: rocksdb: Options.fs: PosixFileSystem 2024-03-21T10:13:40.232 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:13:39 smithi100 ceph-mon[35193]: rocksdb: Options.info_log: 0x55e69f65f960 2024-03-21T10:13:40.232 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:13:39 smithi100 ceph-mon[35193]: rocksdb: Options.max_file_opening_threads: 16 2024-03-21T10:13:40.232 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:13:39 smithi100 ceph-mon[35193]: rocksdb: Options.statistics: (nil) 2024-03-21T10:13:40.233 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:13:39 smithi100 ceph-mon[35193]: rocksdb: Options.use_fsync: 0 2024-03-21T10:13:40.233 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:13:39 smithi100 ceph-mon[35193]: rocksdb: Options.max_log_file_size: 0 2024-03-21T10:13:40.233 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:13:39 smithi100 ceph-mon[35193]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-03-21T10:13:40.233 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:13:39 smithi100 ceph-mon[35193]: rocksdb: Options.log_file_time_to_roll: 0 2024-03-21T10:13:40.233 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:13:39 smithi100 ceph-mon[35193]: rocksdb: Options.keep_log_file_num: 1000 2024-03-21T10:13:40.233 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:13:39 smithi100 ceph-mon[35193]: rocksdb: Options.recycle_log_file_num: 0 2024-03-21T10:13:40.233 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:13:39 smithi100 ceph-mon[35193]: rocksdb: Options.allow_fallocate: 1 2024-03-21T10:13:40.233 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:13:39 smithi100 ceph-mon[35193]: rocksdb: Options.allow_mmap_reads: 0 2024-03-21T10:13:40.233 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:13:39 smithi100 ceph-mon[35193]: rocksdb: Options.allow_mmap_writes: 0 2024-03-21T10:13:40.233 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:13:39 smithi100 ceph-mon[35193]: rocksdb: Options.use_direct_reads: 0 2024-03-21T10:13:40.233 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:13:39 smithi100 ceph-mon[35193]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-03-21T10:13:40.233 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:13:39 smithi100 ceph-mon[35193]: rocksdb: Options.create_missing_column_families: 0 2024-03-21T10:13:40.233 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:13:39 smithi100 ceph-mon[35193]: rocksdb: Options.db_log_dir: 2024-03-21T10:13:40.233 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:13:39 smithi100 ceph-mon[35193]: rocksdb: Options.wal_dir: 2024-03-21T10:13:40.233 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:13:39 smithi100 ceph-mon[35193]: rocksdb: Options.table_cache_numshardbits: 6 2024-03-21T10:13:40.234 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:13:39 smithi100 ceph-mon[35193]: rocksdb: Options.WAL_ttl_seconds: 0 2024-03-21T10:13:40.234 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:13:39 smithi100 ceph-mon[35193]: rocksdb: Options.WAL_size_limit_MB: 0 2024-03-21T10:13:40.234 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:13:39 smithi100 ceph-mon[35193]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-03-21T10:13:40.234 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:13:39 smithi100 ceph-mon[35193]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-03-21T10:13:40.234 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:13:39 smithi100 ceph-mon[35193]: rocksdb: Options.is_fd_close_on_exec: 1 2024-03-21T10:13:40.234 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:13:39 smithi100 ceph-mon[35193]: rocksdb: Options.advise_random_on_open: 1 2024-03-21T10:13:40.235 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:13:39 smithi100 ceph-mon[35193]: rocksdb: Options.db_write_buffer_size: 0 2024-03-21T10:13:40.235 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:13:39 smithi100 ceph-mon[35193]: rocksdb: Options.write_buffer_manager: 0x55e69f6321e0 2024-03-21T10:13:40.235 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:13:39 smithi100 ceph-mon[35193]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-03-21T10:13:40.235 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:13:39 smithi100 ceph-mon[35193]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-03-21T10:13:40.235 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:13:39 smithi100 ceph-mon[35193]: rocksdb: Options.use_adaptive_mutex: 0 2024-03-21T10:13:40.237 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:13:39 smithi100 ceph-mon[35193]: rocksdb: Options.rate_limiter: (nil) 2024-03-21T10:13:40.237 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:13:39 smithi100 ceph-mon[35193]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-03-21T10:13:40.237 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:13:39 smithi100 ceph-mon[35193]: rocksdb: Options.wal_recovery_mode: 2 2024-03-21T10:13:40.237 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:13:39 smithi100 ceph-mon[35193]: rocksdb: Options.enable_thread_tracking: 0 2024-03-21T10:13:40.237 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:13:39 smithi100 ceph-mon[35193]: rocksdb: Options.enable_pipelined_write: 0 2024-03-21T10:13:40.238 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:13:39 smithi100 ceph-mon[35193]: rocksdb: Options.unordered_write: 0 2024-03-21T10:13:40.238 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:13:39 smithi100 ceph-mon[35193]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-03-21T10:13:40.238 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:13:39 smithi100 ceph-mon[35193]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-03-21T10:13:40.238 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:13:39 smithi100 ceph-mon[35193]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-03-21T10:13:40.238 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:13:39 smithi100 ceph-mon[35193]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-03-21T10:13:40.238 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:13:39 smithi100 ceph-mon[35193]: rocksdb: Options.row_cache: None 2024-03-21T10:13:40.238 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:13:39 smithi100 ceph-mon[35193]: rocksdb: Options.wal_filter: None 2024-03-21T10:13:40.238 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:13:39 smithi100 ceph-mon[35193]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-03-21T10:13:40.238 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:13:39 smithi100 ceph-mon[35193]: rocksdb: Options.allow_ingest_behind: 0 2024-03-21T10:13:40.238 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:13:39 smithi100 ceph-mon[35193]: rocksdb: Options.two_write_queues: 0 2024-03-21T10:13:40.238 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:13:39 smithi100 ceph-mon[35193]: rocksdb: Options.manual_wal_flush: 0 2024-03-21T10:13:40.238 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:13:39 smithi100 ceph-mon[35193]: rocksdb: Options.wal_compression: 0 2024-03-21T10:13:40.238 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:13:39 smithi100 ceph-mon[35193]: rocksdb: Options.atomic_flush: 0 2024-03-21T10:13:40.239 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:13:39 smithi100 ceph-mon[35193]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-03-21T10:13:40.239 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:13:39 smithi100 ceph-mon[35193]: rocksdb: Options.persist_stats_to_disk: 0 2024-03-21T10:13:40.239 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:13:39 smithi100 ceph-mon[35193]: rocksdb: Options.write_dbid_to_manifest: 0 2024-03-21T10:13:40.239 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:13:39 smithi100 ceph-mon[35193]: rocksdb: Options.log_readahead_size: 0 2024-03-21T10:13:40.239 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:13:39 smithi100 ceph-mon[35193]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-03-21T10:13:40.239 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:13:39 smithi100 ceph-mon[35193]: rocksdb: Options.best_efforts_recovery: 0 2024-03-21T10:13:40.239 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:13:39 smithi100 ceph-mon[35193]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-03-21T10:13:40.239 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:13:39 smithi100 ceph-mon[35193]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-03-21T10:13:40.239 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:13:39 smithi100 ceph-mon[35193]: rocksdb: Options.allow_data_in_errors: 0 2024-03-21T10:13:40.239 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:13:39 smithi100 ceph-mon[35193]: rocksdb: Options.db_host_id: __hostname__ 2024-03-21T10:13:40.239 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:13:39 smithi100 ceph-mon[35193]: rocksdb: Options.enforce_single_del_contracts: true 2024-03-21T10:13:40.239 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:13:39 smithi100 ceph-mon[35193]: rocksdb: Options.max_background_jobs: 2 2024-03-21T10:13:40.239 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:13:39 smithi100 ceph-mon[35193]: rocksdb: Options.max_background_compactions: -1 2024-03-21T10:13:40.239 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:13:39 smithi100 ceph-mon[35193]: rocksdb: Options.max_subcompactions: 1 2024-03-21T10:13:40.239 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:13:39 smithi100 ceph-mon[35193]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-03-21T10:13:40.240 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:13:39 smithi100 ceph-mon[35193]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-03-21T10:13:40.240 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:13:39 smithi100 ceph-mon[35193]: rocksdb: Options.delayed_write_rate : 16777216 2024-03-21T10:13:40.240 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:13:39 smithi100 ceph-mon[35193]: rocksdb: Options.max_total_wal_size: 0 2024-03-21T10:13:40.240 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:13:39 smithi100 ceph-mon[35193]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-03-21T10:13:40.240 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:13:39 smithi100 ceph-mon[35193]: rocksdb: Options.stats_dump_period_sec: 600 2024-03-21T10:13:40.240 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:13:39 smithi100 ceph-mon[35193]: rocksdb: Options.stats_persist_period_sec: 600 2024-03-21T10:13:40.240 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:13:39 smithi100 ceph-mon[35193]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-03-21T10:13:40.240 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:13:39 smithi100 ceph-mon[35193]: rocksdb: Options.max_open_files: -1 2024-03-21T10:13:40.240 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:13:39 smithi100 ceph-mon[35193]: rocksdb: Options.bytes_per_sync: 0 2024-03-21T10:13:40.240 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:13:39 smithi100 ceph-mon[35193]: rocksdb: Options.wal_bytes_per_sync: 0 2024-03-21T10:13:40.240 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:13:39 smithi100 ceph-mon[35193]: rocksdb: Options.strict_bytes_per_sync: 0 2024-03-21T10:13:40.240 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:13:39 smithi100 ceph-mon[35193]: rocksdb: Options.compaction_readahead_size: 0 2024-03-21T10:13:40.240 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:13:39 smithi100 ceph-mon[35193]: rocksdb: Options.max_background_flushes: -1 2024-03-21T10:13:40.240 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:13:39 smithi100 ceph-mon[35193]: rocksdb: Compression algorithms supported: 2024-03-21T10:13:40.240 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:13:39 smithi100 ceph-mon[35193]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-03-21T10:13:40.241 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:13:39 smithi100 ceph-mon[35193]: rocksdb: kZSTD supported: 0 2024-03-21T10:13:40.241 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:13:39 smithi100 ceph-mon[35193]: rocksdb: kXpressCompression supported: 0 2024-03-21T10:13:40.241 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:13:39 smithi100 ceph-mon[35193]: rocksdb: kLZ4HCCompression supported: 1 2024-03-21T10:13:40.241 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:13:39 smithi100 ceph-mon[35193]: rocksdb: kZlibCompression supported: 1 2024-03-21T10:13:40.241 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:13:39 smithi100 ceph-mon[35193]: rocksdb: kSnappyCompression supported: 1 2024-03-21T10:13:40.241 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:13:39 smithi100 ceph-mon[35193]: rocksdb: kLZ4Compression supported: 1 2024-03-21T10:13:40.241 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:13:39 smithi100 ceph-mon[35193]: rocksdb: kBZip2Compression supported: 0 2024-03-21T10:13:40.241 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:13:39 smithi100 ceph-mon[35193]: rocksdb: Fast CRC32 supported: Supported on x86 2024-03-21T10:13:40.241 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:13:39 smithi100 ceph-mon[35193]: rocksdb: DMutex implementation: pthread_mutex_t 2024-03-21T10:13:40.241 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:13:39 smithi100 ceph-mon[35193]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-b/store.db/MANIFEST-000005 2024-03-21T10:13:40.241 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:13:39 smithi100 ceph-mon[35193]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-03-21T10:13:40.241 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:13:39 smithi100 ceph-mon[35193]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-03-21T10:13:40.241 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:13:39 smithi100 ceph-mon[35193]: rocksdb: Options.merge_operator: 2024-03-21T10:13:40.241 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:13:39 smithi100 ceph-mon[35193]: rocksdb: Options.compaction_filter: None 2024-03-21T10:13:40.242 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:13:39 smithi100 ceph-mon[35193]: rocksdb: Options.compaction_filter_factory: None 2024-03-21T10:13:40.242 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:13:39 smithi100 ceph-mon[35193]: rocksdb: Options.sst_partitioner_factory: None 2024-03-21T10:13:40.242 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:13:39 smithi100 ceph-mon[35193]: rocksdb: Options.memtable_factory: SkipListFactory 2024-03-21T10:13:40.242 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:13:39 smithi100 ceph-mon[35193]: rocksdb: Options.table_factory: BlockBasedTable 2024-03-21T10:13:40.242 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:13:39 smithi100 ceph-mon[35193]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x55e69f65fa60) 2024-03-21T10:13:40.242 INFO:journalctl@ceph.mon.b.smithi100.stdout: cache_index_and_filter_blocks: 1 2024-03-21T10:13:40.242 INFO:journalctl@ceph.mon.b.smithi100.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-03-21T10:13:40.242 INFO:journalctl@ceph.mon.b.smithi100.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-03-21T10:13:40.242 INFO:journalctl@ceph.mon.b.smithi100.stdout: pin_top_level_index_and_filter: 1 2024-03-21T10:13:40.242 INFO:journalctl@ceph.mon.b.smithi100.stdout: index_type: 0 2024-03-21T10:13:40.242 INFO:journalctl@ceph.mon.b.smithi100.stdout: data_block_index_type: 0 2024-03-21T10:13:40.242 INFO:journalctl@ceph.mon.b.smithi100.stdout: index_shortening: 1 2024-03-21T10:13:40.242 INFO:journalctl@ceph.mon.b.smithi100.stdout: data_block_hash_table_util_ratio: 0.750000 2024-03-21T10:13:40.243 INFO:journalctl@ceph.mon.b.smithi100.stdout: checksum: 4 2024-03-21T10:13:40.243 INFO:journalctl@ceph.mon.b.smithi100.stdout: no_block_cache: 0 2024-03-21T10:13:40.243 INFO:journalctl@ceph.mon.b.smithi100.stdout: block_cache: 0x55e69f665090 2024-03-21T10:13:40.243 INFO:journalctl@ceph.mon.b.smithi100.stdout: block_cache_name: BinnedLRUCache 2024-03-21T10:13:40.243 INFO:journalctl@ceph.mon.b.smithi100.stdout: block_cache_options: 2024-03-21T10:13:40.243 INFO:journalctl@ceph.mon.b.smithi100.stdout: capacity : 536870912 2024-03-21T10:13:40.243 INFO:journalctl@ceph.mon.b.smithi100.stdout: num_shard_bits : 4 2024-03-21T10:13:40.243 INFO:journalctl@ceph.mon.b.smithi100.stdout: strict_capacity_limit : 0 2024-03-21T10:13:40.243 INFO:journalctl@ceph.mon.b.smithi100.stdout: high_pri_pool_ratio: 0.000 2024-03-21T10:13:40.243 INFO:journalctl@ceph.mon.b.smithi100.stdout: block_cache_compressed: (nil) 2024-03-21T10:13:40.243 INFO:journalctl@ceph.mon.b.smithi100.stdout: persistent_cache: (nil) 2024-03-21T10:13:40.243 INFO:journalctl@ceph.mon.b.smithi100.stdout: block_size: 4096 2024-03-21T10:13:40.243 INFO:journalctl@ceph.mon.b.smithi100.stdout: block_size_deviation: 10 2024-03-21T10:13:40.243 INFO:journalctl@ceph.mon.b.smithi100.stdout: block_restart_interval: 16 2024-03-21T10:13:40.243 INFO:journalctl@ceph.mon.b.smithi100.stdout: index_block_restart_interval: 1 2024-03-21T10:13:40.243 INFO:journalctl@ceph.mon.b.smithi100.stdout: metadata_block_size: 4096 2024-03-21T10:13:40.244 INFO:journalctl@ceph.mon.b.smithi100.stdout: partition_filters: 0 2024-03-21T10:13:40.244 INFO:journalctl@ceph.mon.b.smithi100.stdout: use_delta_encoding: 1 2024-03-21T10:13:40.244 INFO:journalctl@ceph.mon.b.smithi100.stdout: filter_policy: bloomfilter 2024-03-21T10:13:40.244 INFO:journalctl@ceph.mon.b.smithi100.stdout: whole_key_filtering: 1 2024-03-21T10:13:40.244 INFO:journalctl@ceph.mon.b.smithi100.stdout: verify_compression: 0 2024-03-21T10:13:40.244 INFO:journalctl@ceph.mon.b.smithi100.stdout: read_amp_bytes_per_bit: 0 2024-03-21T10:13:40.244 INFO:journalctl@ceph.mon.b.smithi100.stdout: format_version: 5 2024-03-21T10:13:40.244 INFO:journalctl@ceph.mon.b.smithi100.stdout: enable_index_compression: 1 2024-03-21T10:13:40.244 INFO:journalctl@ceph.mon.b.smithi100.stdout: block_align: 0 2024-03-21T10:13:40.244 INFO:journalctl@ceph.mon.b.smithi100.stdout: max_auto_readahead_size: 262144 2024-03-21T10:13:40.244 INFO:journalctl@ceph.mon.b.smithi100.stdout: prepopulate_block_cache: 0 2024-03-21T10:13:40.244 INFO:journalctl@ceph.mon.b.smithi100.stdout: initial_auto_readahead_size: 8192 2024-03-21T10:13:40.244 INFO:journalctl@ceph.mon.b.smithi100.stdout: num_file_reads_for_auto_readahead: 2 2024-03-21T10:13:40.244 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:13:39 smithi100 ceph-mon[35193]: rocksdb: Options.write_buffer_size: 33554432 2024-03-21T10:13:40.244 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:13:39 smithi100 ceph-mon[35193]: rocksdb: Options.max_write_buffer_number: 2 2024-03-21T10:13:40.245 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:13:39 smithi100 ceph-mon[35193]: rocksdb: Options.compression: NoCompression 2024-03-21T10:13:40.245 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:13:39 smithi100 ceph-mon[35193]: rocksdb: Options.bottommost_compression: Disabled 2024-03-21T10:13:40.245 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:13:39 smithi100 ceph-mon[35193]: rocksdb: Options.prefix_extractor: nullptr 2024-03-21T10:13:40.245 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:13:39 smithi100 ceph-mon[35193]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-03-21T10:13:40.245 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:13:39 smithi100 ceph-mon[35193]: rocksdb: Options.num_levels: 7 2024-03-21T10:13:40.245 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:13:39 smithi100 ceph-mon[35193]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-03-21T10:13:40.245 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:13:39 smithi100 ceph-mon[35193]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-03-21T10:13:40.245 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:13:39 smithi100 ceph-mon[35193]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-03-21T10:13:40.245 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:13:39 smithi100 ceph-mon[35193]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-03-21T10:13:40.245 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:13:39 smithi100 ceph-mon[35193]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-03-21T10:13:40.245 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:13:39 smithi100 ceph-mon[35193]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-03-21T10:13:40.245 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:13:39 smithi100 ceph-mon[35193]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-03-21T10:13:40.245 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:13:39 smithi100 ceph-mon[35193]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-03-21T10:13:40.247 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:13:39 smithi100 ceph-mon[35193]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-03-21T10:13:40.247 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:13:39 smithi100 ceph-mon[35193]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-03-21T10:13:40.247 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:13:39 smithi100 ceph-mon[35193]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-03-21T10:13:40.247 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:13:39 smithi100 ceph-mon[35193]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-03-21T10:13:40.247 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:13:39 smithi100 ceph-mon[35193]: rocksdb: Options.compression_opts.window_bits: -14 2024-03-21T10:13:40.247 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:13:39 smithi100 ceph-mon[35193]: rocksdb: Options.compression_opts.level: 32767 2024-03-21T10:13:40.247 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:13:39 smithi100 ceph-mon[35193]: rocksdb: Options.compression_opts.strategy: 0 2024-03-21T10:13:40.247 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:13:39 smithi100 ceph-mon[35193]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-03-21T10:13:40.247 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:13:39 smithi100 ceph-mon[35193]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-03-21T10:13:40.247 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:13:39 smithi100 ceph-mon[35193]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-03-21T10:13:40.247 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:13:39 smithi100 ceph-mon[35193]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-03-21T10:13:40.247 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:13:39 smithi100 ceph-mon[35193]: rocksdb: Options.compression_opts.enabled: false 2024-03-21T10:13:40.248 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:13:39 smithi100 ceph-mon[35193]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-03-21T10:13:40.248 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:13:39 smithi100 ceph-mon[35193]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-03-21T10:13:40.248 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:13:39 smithi100 ceph-mon[35193]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-03-21T10:13:40.248 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:13:39 smithi100 ceph-mon[35193]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-03-21T10:13:40.248 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:13:39 smithi100 ceph-mon[35193]: rocksdb: Options.target_file_size_base: 67108864 2024-03-21T10:13:40.248 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:13:39 smithi100 ceph-mon[35193]: rocksdb: Options.target_file_size_multiplier: 1 2024-03-21T10:13:40.248 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:13:39 smithi100 ceph-mon[35193]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-03-21T10:13:40.248 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:13:39 smithi100 ceph-mon[35193]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-03-21T10:13:40.248 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:13:39 smithi100 ceph-mon[35193]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-03-21T10:13:40.248 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:13:39 smithi100 ceph-mon[35193]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-03-21T10:13:40.248 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:13:39 smithi100 ceph-mon[35193]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-03-21T10:13:40.248 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:13:39 smithi100 ceph-mon[35193]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-03-21T10:13:40.248 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:13:39 smithi100 ceph-mon[35193]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-03-21T10:13:40.248 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:13:39 smithi100 ceph-mon[35193]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-03-21T10:13:40.249 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:13:39 smithi100 ceph-mon[35193]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-03-21T10:13:40.249 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:13:39 smithi100 ceph-mon[35193]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-03-21T10:13:40.249 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:13:39 smithi100 ceph-mon[35193]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-03-21T10:13:40.249 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:13:39 smithi100 ceph-mon[35193]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-03-21T10:13:40.249 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:13:39 smithi100 ceph-mon[35193]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-03-21T10:13:40.249 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:13:39 smithi100 ceph-mon[35193]: rocksdb: Options.arena_block_size: 1048576 2024-03-21T10:13:40.249 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:13:39 smithi100 ceph-mon[35193]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-03-21T10:13:40.249 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:13:39 smithi100 ceph-mon[35193]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-03-21T10:13:40.249 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:13:39 smithi100 ceph-mon[35193]: rocksdb: Options.disable_auto_compactions: 0 2024-03-21T10:13:40.249 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:13:39 smithi100 ceph-mon[35193]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-03-21T10:13:40.249 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:13:39 smithi100 ceph-mon[35193]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-03-21T10:13:40.249 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:13:39 smithi100 ceph-mon[35193]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-03-21T10:13:40.249 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:13:39 smithi100 ceph-mon[35193]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-03-21T10:13:40.249 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:13:39 smithi100 ceph-mon[35193]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-03-21T10:13:40.250 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:13:39 smithi100 ceph-mon[35193]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-03-21T10:13:40.250 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:13:39 smithi100 ceph-mon[35193]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-03-21T10:13:40.250 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:13:39 smithi100 ceph-mon[35193]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-03-21T10:13:40.250 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:13:39 smithi100 ceph-mon[35193]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-03-21T10:13:40.250 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:13:39 smithi100 ceph-mon[35193]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-03-21T10:13:40.250 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:13:39 smithi100 ceph-mon[35193]: rocksdb: Options.table_properties_collectors: 2024-03-21T10:13:40.250 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:13:39 smithi100 ceph-mon[35193]: rocksdb: Options.inplace_update_support: 0 2024-03-21T10:13:40.250 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:13:39 smithi100 ceph-mon[35193]: rocksdb: Options.inplace_update_num_locks: 10000 2024-03-21T10:13:40.250 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:13:39 smithi100 ceph-mon[35193]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-03-21T10:13:40.250 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:13:39 smithi100 ceph-mon[35193]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-03-21T10:13:40.250 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:13:39 smithi100 ceph-mon[35193]: rocksdb: Options.memtable_huge_page_size: 0 2024-03-21T10:13:40.250 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:13:39 smithi100 ceph-mon[35193]: rocksdb: Options.bloom_locality: 0 2024-03-21T10:13:40.250 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:13:39 smithi100 ceph-mon[35193]: rocksdb: Options.max_successive_merges: 0 2024-03-21T10:13:40.251 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:13:39 smithi100 ceph-mon[35193]: rocksdb: Options.optimize_filters_for_hits: 0 2024-03-21T10:13:40.251 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:13:39 smithi100 ceph-mon[35193]: rocksdb: Options.paranoid_file_checks: 0 2024-03-21T10:13:40.251 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:13:39 smithi100 ceph-mon[35193]: rocksdb: Options.force_consistency_checks: 1 2024-03-21T10:13:40.251 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:13:39 smithi100 ceph-mon[35193]: rocksdb: Options.report_bg_io_stats: 0 2024-03-21T10:13:40.251 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:13:39 smithi100 ceph-mon[35193]: rocksdb: Options.ttl: 2592000 2024-03-21T10:13:40.251 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:13:39 smithi100 ceph-mon[35193]: rocksdb: Options.periodic_compaction_seconds: 0 2024-03-21T10:13:40.251 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:13:39 smithi100 ceph-mon[35193]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-03-21T10:13:40.251 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:13:39 smithi100 ceph-mon[35193]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-03-21T10:13:40.251 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:13:39 smithi100 ceph-mon[35193]: rocksdb: Options.enable_blob_files: false 2024-03-21T10:13:40.251 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:13:39 smithi100 ceph-mon[35193]: rocksdb: Options.min_blob_size: 0 2024-03-21T10:13:40.251 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:13:39 smithi100 ceph-mon[35193]: rocksdb: Options.blob_file_size: 268435456 2024-03-21T10:13:40.251 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:13:39 smithi100 ceph-mon[35193]: rocksdb: Options.blob_compression_type: NoCompression 2024-03-21T10:13:40.251 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:13:39 smithi100 ceph-mon[35193]: rocksdb: Options.enable_blob_garbage_collection: false 2024-03-21T10:13:40.251 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:13:39 smithi100 ceph-mon[35193]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-03-21T10:13:40.251 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:13:39 smithi100 ceph-mon[35193]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-03-21T10:13:40.251 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:13:39 smithi100 ceph-mon[35193]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-03-21T10:13:40.252 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:13:39 smithi100 ceph-mon[35193]: rocksdb: Options.blob_file_starting_level: 0 2024-03-21T10:13:40.252 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:13:39 smithi100 ceph-mon[35193]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-03-21T10:13:40.252 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:13:39 smithi100 ceph-mon[35193]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-b/store.db/MANIFEST-000005 succeeded,manifest_file_number is 5, next_file_number is 7, last_sequence is 0, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 0 2024-03-21T10:13:40.252 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:13:39 smithi100 ceph-mon[35193]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 0 2024-03-21T10:13:40.252 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:13:39 smithi100 ceph-mon[35193]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: be3da077-1932-4828-b928-c245b4657703 2024-03-21T10:13:40.252 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:13:39 smithi100 ceph-mon[35193]: rocksdb: EVENT_LOG_v1 {"time_micros": 1711016019826969, "job": 1, "event": "recovery_started", "wal_files": [4]} 2024-03-21T10:13:40.252 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:13:39 smithi100 ceph-mon[35193]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #4 mode 2 2024-03-21T10:13:40.252 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:13:39 smithi100 ceph-mon[35193]: rocksdb: EVENT_LOG_v1 {"time_micros": 1711016019827890, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 8, "file_size": 1747, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 1, "largest_seqno": 5, "table_properties": {"data_size": 648, "index_size": 31, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 69, "raw_key_size": 115, "raw_average_key_size": 23, "raw_value_size": 526, "raw_average_value_size": 105, "num_data_blocks": 1, "num_entries": 5, "num_filter_entries": 5, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1711016019, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "be3da077-1932-4828-b928-c245b4657703", "db_session_id": "W4BBA9E2HJ4IO7A2RR8W", "orig_file_number": 8, "seqno_to_time_mapping": "N/A"}} 2024-03-21T10:13:40.252 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:13:39 smithi100 ceph-mon[35193]: rocksdb: EVENT_LOG_v1 {"time_micros": 1711016019828151, "job": 1, "event": "recovery_finished"} 2024-03-21T10:13:40.252 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:13:39 smithi100 ceph-mon[35193]: rocksdb: [db/version_set.cc:5047] Creating manifest 10 2024-03-21T10:13:40.252 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:13:39 smithi100 ceph-mon[35193]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-b/store.db/000004.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-03-21T10:13:40.252 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:13:39 smithi100 ceph-mon[35193]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x55e69f758000 2024-03-21T10:13:40.252 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:13:39 smithi100 ceph-mon[35193]: rocksdb: DB pointer 0x55e69f742000 2024-03-21T10:13:40.252 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:13:39 smithi100 ceph-mon[35193]: mon.b does not exist in monmap, will attempt to join an existing cluster 2024-03-21T10:13:40.253 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:13:39 smithi100 ceph-mon[35193]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-03-21T10:13:40.253 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:13:39 smithi100 ceph-mon[35193]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-03-21T10:13:40.253 INFO:journalctl@ceph.mon.b.smithi100.stdout: ** DB Stats ** 2024-03-21T10:13:40.253 INFO:journalctl@ceph.mon.b.smithi100.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-03-21T10:13:40.253 INFO:journalctl@ceph.mon.b.smithi100.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-03-21T10:13:40.253 INFO:journalctl@ceph.mon.b.smithi100.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-03-21T10:13:40.253 INFO:journalctl@ceph.mon.b.smithi100.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-03-21T10:13:40.253 INFO:journalctl@ceph.mon.b.smithi100.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-03-21T10:13:40.253 INFO:journalctl@ceph.mon.b.smithi100.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-03-21T10:13:40.253 INFO:journalctl@ceph.mon.b.smithi100.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-03-21T10:13:40.253 INFO:journalctl@ceph.mon.b.smithi100.stdout: 2024-03-21T10:13:40.253 INFO:journalctl@ceph.mon.b.smithi100.stdout: ** Compaction Stats [default] ** 2024-03-21T10:13:40.253 INFO:journalctl@ceph.mon.b.smithi100.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-03-21T10:13:40.254 INFO:journalctl@ceph.mon.b.smithi100.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-03-21T10:13:40.254 INFO:journalctl@ceph.mon.b.smithi100.stdout: L0 1/0 1.71 KB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 1.9 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-03-21T10:13:40.254 INFO:journalctl@ceph.mon.b.smithi100.stdout: Sum 1/0 1.71 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 1.9 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-03-21T10:13:40.254 INFO:journalctl@ceph.mon.b.smithi100.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 1.9 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-03-21T10:13:40.254 INFO:journalctl@ceph.mon.b.smithi100.stdout: 2024-03-21T10:13:40.254 INFO:journalctl@ceph.mon.b.smithi100.stdout: ** Compaction Stats [default] ** 2024-03-21T10:13:40.254 INFO:journalctl@ceph.mon.b.smithi100.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-03-21T10:13:40.254 INFO:journalctl@ceph.mon.b.smithi100.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-03-21T10:13:40.254 INFO:journalctl@ceph.mon.b.smithi100.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.9 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-03-21T10:13:40.254 INFO:journalctl@ceph.mon.b.smithi100.stdout: 2024-03-21T10:13:40.254 INFO:journalctl@ceph.mon.b.smithi100.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-03-21T10:13:40.254 INFO:journalctl@ceph.mon.b.smithi100.stdout: 2024-03-21T10:13:40.254 INFO:journalctl@ceph.mon.b.smithi100.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-03-21T10:13:40.254 INFO:journalctl@ceph.mon.b.smithi100.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-03-21T10:13:40.255 INFO:journalctl@ceph.mon.b.smithi100.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-03-21T10:13:40.255 INFO:journalctl@ceph.mon.b.smithi100.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-03-21T10:13:40.255 INFO:journalctl@ceph.mon.b.smithi100.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-03-21T10:13:40.255 INFO:journalctl@ceph.mon.b.smithi100.stdout: AddFile(Keys): cumulative 0, interval 0 2024-03-21T10:13:40.255 INFO:journalctl@ceph.mon.b.smithi100.stdout: Cumulative compaction: 0.00 GB write, 0.28 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-03-21T10:13:40.255 INFO:journalctl@ceph.mon.b.smithi100.stdout: Interval compaction: 0.00 GB write, 0.28 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-03-21T10:13:40.255 INFO:journalctl@ceph.mon.b.smithi100.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-03-21T10:13:40.255 INFO:journalctl@ceph.mon.b.smithi100.stdout: Block cache BinnedLRUCache@0x55e69f665090#2 capacity: 512.00 MB usage: 0.98 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 8.4e-05 secs_since: 0 2024-03-21T10:13:40.255 INFO:journalctl@ceph.mon.b.smithi100.stdout: Block cache entry stats(count,size,portion): FilterBlock(1,0.11 KB,2.08616e-05%) IndexBlock(1,0.11 KB,2.08616e-05%) Misc(2,0.77 KB,0.000146031%) 2024-03-21T10:13:40.255 INFO:journalctl@ceph.mon.b.smithi100.stdout: 2024-03-21T10:13:40.255 INFO:journalctl@ceph.mon.b.smithi100.stdout: ** File Read Latency Histogram By Level [default] ** 2024-03-21T10:13:40.255 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:13:39 smithi100 ceph-mon[35193]: using public_addr v2:172.21.15.100:0/0 -> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] 2024-03-21T10:13:40.255 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:13:39 smithi100 ceph-mon[35193]: starting mon.b rank -1 at public addrs [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] at bind addrs [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] mon_data /var/lib/ceph/mon/ceph-b fsid 3056d51a-e76b-11ee-95cd-87774f69a715 2024-03-21T10:13:40.255 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:13:39 smithi100 ceph-mon[35193]: mon.b@-1(???) e0 preinit fsid 3056d51a-e76b-11ee-95cd-87774f69a715 2024-03-21T10:13:40.255 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:13:39 smithi100 ceph-mon[35193]: mon.b@-1(synchronizing).mds e1 new map 2024-03-21T10:13:40.256 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:13:39 smithi100 ceph-mon[35193]: mon.b@-1(synchronizing).mds e1 print_map 2024-03-21T10:13:40.256 INFO:journalctl@ceph.mon.b.smithi100.stdout: e1 2024-03-21T10:13:40.256 INFO:journalctl@ceph.mon.b.smithi100.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-03-21T10:13:40.256 INFO:journalctl@ceph.mon.b.smithi100.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments} 2024-03-21T10:13:40.256 INFO:journalctl@ceph.mon.b.smithi100.stdout: legacy client fscid: -1 2024-03-21T10:13:40.256 INFO:journalctl@ceph.mon.b.smithi100.stdout: 2024-03-21T10:13:40.256 INFO:journalctl@ceph.mon.b.smithi100.stdout: No filesystems configured 2024-03-21T10:13:40.256 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:13:39 smithi100 ceph-mon[35193]: mon.b@-1(synchronizing).osd e0 _set_cache_ratios kv ratio 0.25 inc ratio 0.375 full ratio 0.375 2024-03-21T10:13:40.256 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:13:39 smithi100 ceph-mon[35193]: mon.b@-1(synchronizing).osd e0 register_cache_with_pcm pcm target: 2147483648 pcm max: 1020054732 pcm min: 134217728 inc_osd_cache size: 1 2024-03-21T10:13:40.256 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:13:39 smithi100 ceph-mon[35193]: mon.b@-1(synchronizing).osd e1 e1: 0 total, 0 up, 0 in 2024-03-21T10:13:40.256 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:13:39 smithi100 ceph-mon[35193]: mon.b@-1(synchronizing).osd e2 e2: 0 total, 0 up, 0 in 2024-03-21T10:13:40.256 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:13:39 smithi100 ceph-mon[35193]: mon.b@-1(synchronizing).osd e3 e3: 0 total, 0 up, 0 in 2024-03-21T10:13:40.256 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:13:39 smithi100 ceph-mon[35193]: mon.b@-1(synchronizing).osd e4 e4: 0 total, 0 up, 0 in 2024-03-21T10:13:40.256 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:13:39 smithi100 ceph-mon[35193]: mon.b@-1(synchronizing).osd e4 crush map has features 3314932999778484224, adjusting msgr requires 2024-03-21T10:13:40.256 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:13:39 smithi100 ceph-mon[35193]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-03-21T10:13:40.257 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:13:39 smithi100 ceph-mon[35193]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-03-21T10:13:40.257 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:13:39 smithi100 ceph-mon[35193]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-03-21T10:13:40.257 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:13:39 smithi100 ceph-mon[35193]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T10:13:40.257 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:13:39 smithi100 ceph-mon[35193]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:13:40.257 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:13:39 smithi100 ceph-mon[35193]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T10:13:40.257 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:13:39 smithi100 ceph-mon[35193]: Added host smithi100 2024-03-21T10:13:40.257 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:13:39 smithi100 ceph-mon[35193]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:13:40.257 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:13:39 smithi100 ceph-mon[35193]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T10:13:40.257 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:13:39 smithi100 ceph-mon[35193]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:13:40.257 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:13:39 smithi100 ceph-mon[35193]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T10:13:40.257 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:13:39 smithi100 ceph-mon[35193]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-21T10:13:40.257 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:13:39 smithi100 ceph-mon[35193]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:13:40.257 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:13:39 smithi100 ceph-mon[35193]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T10:13:40.257 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:13:39 smithi100 ceph-mon[35193]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T10:13:40.257 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:13:39 smithi100 ceph-mon[35193]: from='client.14182 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi157", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:13:40.258 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:13:39 smithi100 ceph-mon[35193]: Deploying cephadm binary to smithi157 2024-03-21T10:13:40.258 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:13:39 smithi100 ceph-mon[35193]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:13:40.258 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:13:39 smithi100 ceph-mon[35193]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:13:40.258 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:13:39 smithi100 ceph-mon[35193]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:13:40.258 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:13:39 smithi100 ceph-mon[35193]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:13:40.258 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:13:39 smithi100 ceph-mon[35193]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi100", "name": "osd_memory_target"}]: dispatch 2024-03-21T10:13:40.258 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:13:39 smithi100 ceph-mon[35193]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T10:13:40.258 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:13:39 smithi100 ceph-mon[35193]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T10:13:40.259 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:13:39 smithi100 ceph-mon[35193]: Updating smithi100:/etc/ceph/ceph.conf 2024-03-21T10:13:40.259 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:13:39 smithi100 ceph-mon[35193]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T10:13:40.259 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:13:39 smithi100 ceph-mon[35193]: Updating smithi100:/var/lib/ceph/3056d51a-e76b-11ee-95cd-87774f69a715/config/ceph.conf 2024-03-21T10:13:40.259 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:13:39 smithi100 ceph-mon[35193]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T10:13:40.259 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:13:39 smithi100 ceph-mon[35193]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T10:13:40.259 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:13:39 smithi100 ceph-mon[35193]: Updating smithi100:/etc/ceph/ceph.client.admin.keyring 2024-03-21T10:13:40.259 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:13:39 smithi100 ceph-mon[35193]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:13:40.259 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:13:39 smithi100 ceph-mon[35193]: Updating smithi100:/var/lib/ceph/3056d51a-e76b-11ee-95cd-87774f69a715/config/ceph.client.admin.keyring 2024-03-21T10:13:40.259 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:13:39 smithi100 ceph-mon[35193]: Added host smithi157 2024-03-21T10:13:40.259 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:13:39 smithi100 ceph-mon[35193]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:13:40.259 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:13:39 smithi100 ceph-mon[35193]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:13:40.259 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:13:39 smithi100 ceph-mon[35193]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:13:40.259 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:13:39 smithi100 ceph-mon[35193]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T10:13:40.259 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:13:39 smithi100 ceph-mon[35193]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T10:13:40.259 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:13:39 smithi100 ceph-mon[35193]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:13:40.260 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:13:39 smithi100 ceph-mon[35193]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T10:13:40.260 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:13:39 smithi100 ceph-mon[35193]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:13:40.260 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:13:39 smithi100 ceph-mon[35193]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-21T10:13:40.260 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:13:39 smithi100 ceph-mon[35193]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T10:13:40.260 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:13:39 smithi100 ceph-mon[35193]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:13:40.260 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:13:39 smithi100 ceph-mon[35193]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T10:13:40.260 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:13:39 smithi100 ceph-mon[35193]: from='client.? 172.21.15.47:0/1166569903' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-03-21T10:13:40.260 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:13:39 smithi100 ceph-mon[35193]: from='client.? 172.21.15.47:0/1166569903' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-03-21T10:13:40.260 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:13:39 smithi100 ceph-mon[35193]: osdmap e4: 0 total, 0 up, 0 in 2024-03-21T10:13:40.260 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:13:39 smithi100 ceph-mon[35193]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T10:13:40.260 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:13:39 smithi100 ceph-mon[35193]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:13:40.260 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:13:39 smithi100 ceph-mon[35193]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:13:40.260 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:13:39 smithi100 ceph-mon[35193]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:13:40.260 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:13:39 smithi100 ceph-mon[35193]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:13:40.260 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:13:39 smithi100 ceph-mon[35193]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi157", "name": "osd_memory_target"}]: dispatch 2024-03-21T10:13:40.261 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:13:39 smithi100 ceph-mon[35193]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T10:13:40.261 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:13:39 smithi100 ceph-mon[35193]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T10:13:40.261 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:13:39 smithi100 ceph-mon[35193]: Updating smithi157:/etc/ceph/ceph.conf 2024-03-21T10:13:40.261 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:13:39 smithi100 ceph-mon[35193]: Updating smithi157:/var/lib/ceph/3056d51a-e76b-11ee-95cd-87774f69a715/config/ceph.conf 2024-03-21T10:13:40.261 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:13:39 smithi100 ceph-mon[35193]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T10:13:40.261 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:13:39 smithi100 ceph-mon[35193]: Updating smithi157:/etc/ceph/ceph.client.admin.keyring 2024-03-21T10:13:40.261 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:13:39 smithi100 ceph-mon[35193]: Updating smithi157:/var/lib/ceph/3056d51a-e76b-11ee-95cd-87774f69a715/config/ceph.client.admin.keyring 2024-03-21T10:13:40.261 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:13:39 smithi100 ceph-mon[35193]: from='client.14188 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi047:172.21.15.47=a;smithi100:172.21.15.100=b;smithi157:172.21.15.157=c", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:13:40.261 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:13:39 smithi100 ceph-mon[35193]: Saving service mon spec with placement smithi047:172.21.15.47=a;smithi100:172.21.15.100=b;smithi157:172.21.15.157=c;count:3 2024-03-21T10:13:40.261 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:13:39 smithi100 ceph-mon[35193]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:13:40.261 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:13:39 smithi100 ceph-mon[35193]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:13:40.261 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:13:39 smithi100 ceph-mon[35193]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:13:40.261 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:13:39 smithi100 ceph-mon[35193]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:13:40.261 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:13:39 smithi100 ceph-mon[35193]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-21T10:13:40.261 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:13:39 smithi100 ceph-mon[35193]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T10:13:40.262 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:13:39 smithi100 ceph-mon[35193]: Deploying daemon mon.c on smithi157 2024-03-21T10:13:40.262 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:13:39 smithi100 ceph-mon[35193]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T10:13:40.262 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:13:39 smithi100 ceph-mon[35193]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T10:13:40.262 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:13:39 smithi100 ceph-mon[35193]: from='client.? 172.21.15.157:0/146512500' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-03-21T10:13:40.262 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:13:39 smithi100 ceph-mon[35193]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T10:13:40.262 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:13:39 smithi100 ceph-mon[35193]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-21T10:13:40.262 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:13:39 smithi100 ceph-mon[35193]: mon.a calling monitor election 2024-03-21T10:13:40.262 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:13:39 smithi100 ceph-mon[35193]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-21T10:13:40.262 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:13:39 smithi100 ceph-mon[35193]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T10:13:40.262 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:13:39 smithi100 ceph-mon[35193]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-21T10:13:40.262 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:13:39 smithi100 ceph-mon[35193]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-21T10:13:40.263 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:13:39 smithi100 ceph-mon[35193]: mon.c calling monitor election 2024-03-21T10:13:40.263 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:13:39 smithi100 ceph-mon[35193]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T10:13:40.263 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:13:39 smithi100 ceph-mon[35193]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-21T10:13:40.263 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:13:39 smithi100 ceph-mon[35193]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-21T10:13:40.263 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:13:39 smithi100 ceph-mon[35193]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-21T10:13:40.263 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:13:39 smithi100 ceph-mon[35193]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-03-21T10:13:40.263 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:13:39 smithi100 ceph-mon[35193]: monmap e2: 2 mons at {a=[v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0],c=[v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-03-21T10:13:40.263 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:13:39 smithi100 ceph-mon[35193]: fsmap 2024-03-21T10:13:40.263 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:13:39 smithi100 ceph-mon[35193]: osdmap e4: 0 total, 0 up, 0 in 2024-03-21T10:13:40.263 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:13:39 smithi100 ceph-mon[35193]: mgrmap e13: x(active, since 68s) 2024-03-21T10:13:40.263 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:13:39 smithi100 ceph-mon[35193]: overall HEALTH_OK 2024-03-21T10:13:40.263 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:13:39 smithi100 ceph-mon[35193]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:13:40.263 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:13:39 smithi100 ceph-mon[35193]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:13:40.264 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:13:39 smithi100 ceph-mon[35193]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T10:13:40.264 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:13:39 smithi100 ceph-mon[35193]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:13:40.264 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:13:39 smithi100 ceph-mon[35193]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-21T10:13:40.264 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:13:39 smithi100 ceph-mon[35193]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T10:13:40.264 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:13:39 smithi100 ceph-mon[35193]: Deploying daemon mon.b on smithi100 2024-03-21T10:13:40.264 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:13:39 smithi100 ceph-mon[35193]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-21T10:13:40.264 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:13:39 smithi100 ceph-mon[35193]: from='client.? 172.21.15.157:0/4107451876' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-03-21T10:13:40.264 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:13:39 smithi100 ceph-mon[35193]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T10:13:40.264 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:13:39 smithi100 ceph-mon[35193]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T10:13:40.264 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:13:39 smithi100 ceph-mon[35193]: mon.b@-1(synchronizing).paxosservice(auth 1..4) refresh upgraded, format 0 -> 3 2024-03-21T10:13:40.264 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:13:39 smithi100 bash[35175]: 5aa332c64f454ab905946166832a26da9695527b9633cf3a9379bdabea331d23 2024-03-21T10:13:40.264 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:13:39 smithi100 systemd[1]: Started Ceph mon.b for 3056d51a-e76b-11ee-95cd-87774f69a715. 2024-03-21T10:13:45.215 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:13:44 smithi047 ceph-mon[31547]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T10:13:45.215 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:13:44 smithi047 ceph-mon[31547]: mon.c calling monitor election 2024-03-21T10:13:45.215 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:13:44 smithi047 ceph-mon[31547]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-21T10:13:45.215 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:13:44 smithi047 ceph-mon[31547]: mon.a calling monitor election 2024-03-21T10:13:45.215 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:13:44 smithi047 ceph-mon[31547]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-21T10:13:45.215 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:13:44 smithi047 ceph-mon[31547]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-21T10:13:45.215 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:13:44 smithi047 ceph-mon[31547]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-21T10:13:45.215 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:13:44 smithi047 ceph-mon[31547]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T10:13:45.215 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:13:44 smithi047 ceph-mon[31547]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-21T10:13:45.216 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:13:44 smithi047 ceph-mon[31547]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-21T10:13:45.216 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:13:44 smithi047 ceph-mon[31547]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T10:13:45.216 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:13:44 smithi047 ceph-mon[31547]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-21T10:13:45.216 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:13:44 smithi047 ceph-mon[31547]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-21T10:13:45.216 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:13:44 smithi047 ceph-mon[31547]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-03-21T10:13:45.216 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:13:44 smithi047 ceph-mon[31547]: monmap e3: 3 mons at {a=[v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0],b=[v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0],c=[v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-03-21T10:13:45.216 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:13:44 smithi047 ceph-mon[31547]: fsmap 2024-03-21T10:13:45.216 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:13:44 smithi047 ceph-mon[31547]: osdmap e4: 0 total, 0 up, 0 in 2024-03-21T10:13:45.216 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:13:44 smithi047 ceph-mon[31547]: mgrmap e13: x(active, since 79s) 2024-03-21T10:13:45.216 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:13:44 smithi047 ceph-mon[31547]: Health check failed: 1/3 mons down, quorum a,c (MON_DOWN) 2024-03-21T10:13:45.216 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:13:44 smithi047 ceph-mon[31547]: Health detail: HEALTH_WARN 1/3 mons down, quorum a,c 2024-03-21T10:13:45.216 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:13:44 smithi047 ceph-mon[31547]: [WRN] MON_DOWN: 1/3 mons down, quorum a,c 2024-03-21T10:13:45.216 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:13:44 smithi047 ceph-mon[31547]: mon.b (rank 2) addr [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] is down (out of quorum) 2024-03-21T10:13:45.216 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:13:44 smithi047 ceph-mon[31547]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:13:45.216 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:13:44 smithi047 ceph-mon[31547]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:13:45.217 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:13:44 smithi047 ceph-mon[31547]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:13:45.217 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:13:44 smithi047 ceph-mon[31547]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:13:45.240 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:13:44 smithi157 ceph-mon[35581]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T10:13:45.240 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:13:44 smithi157 ceph-mon[35581]: mon.c calling monitor election 2024-03-21T10:13:45.240 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:13:44 smithi157 ceph-mon[35581]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-21T10:13:45.240 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:13:44 smithi157 ceph-mon[35581]: mon.a calling monitor election 2024-03-21T10:13:45.241 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:13:44 smithi157 ceph-mon[35581]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-21T10:13:45.241 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:13:44 smithi157 ceph-mon[35581]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-21T10:13:45.241 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:13:44 smithi157 ceph-mon[35581]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-21T10:13:45.241 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:13:44 smithi157 ceph-mon[35581]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T10:13:45.241 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:13:44 smithi157 ceph-mon[35581]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-21T10:13:45.241 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:13:44 smithi157 ceph-mon[35581]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-21T10:13:45.241 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:13:44 smithi157 ceph-mon[35581]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T10:13:45.241 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:13:44 smithi157 ceph-mon[35581]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-21T10:13:45.241 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:13:44 smithi157 ceph-mon[35581]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-21T10:13:45.241 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:13:44 smithi157 ceph-mon[35581]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-03-21T10:13:45.241 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:13:44 smithi157 ceph-mon[35581]: monmap e3: 3 mons at {a=[v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0],b=[v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0],c=[v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-03-21T10:13:45.241 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:13:44 smithi157 ceph-mon[35581]: fsmap 2024-03-21T10:13:45.241 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:13:44 smithi157 ceph-mon[35581]: osdmap e4: 0 total, 0 up, 0 in 2024-03-21T10:13:45.242 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:13:44 smithi157 ceph-mon[35581]: mgrmap e13: x(active, since 79s) 2024-03-21T10:13:45.242 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:13:44 smithi157 ceph-mon[35581]: Health check failed: 1/3 mons down, quorum a,c (MON_DOWN) 2024-03-21T10:13:45.242 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:13:44 smithi157 ceph-mon[35581]: Health detail: HEALTH_WARN 1/3 mons down, quorum a,c 2024-03-21T10:13:45.242 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:13:44 smithi157 ceph-mon[35581]: [WRN] MON_DOWN: 1/3 mons down, quorum a,c 2024-03-21T10:13:45.242 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:13:44 smithi157 ceph-mon[35581]: mon.b (rank 2) addr [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] is down (out of quorum) 2024-03-21T10:13:45.242 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:13:44 smithi157 ceph-mon[35581]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:13:45.242 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:13:44 smithi157 ceph-mon[35581]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:13:45.242 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:13:44 smithi157 ceph-mon[35581]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:13:45.242 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:13:44 smithi157 ceph-mon[35581]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:13:46.171 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-21T10:13:46.169+0000 7f6af0000700 1 -- 172.21.15.157:0/1720652437 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6ad0006280 msgr2=0x7f6ad0008710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:13:46.171 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-21T10:13:46.169+0000 7f6af0000700 1 --2- 172.21.15.157:0/1720652437 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6ad0006280 0x7f6ad0008710 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f6ae8104f40 tx=0x7f6adc00fe90 comp rx=0 tx=0).stop 2024-03-21T10:13:46.171 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-21T10:13:46.169+0000 7f6af0000700 1 -- 172.21.15.157:0/1720652437 shutdown_connections 2024-03-21T10:13:46.171 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-21T10:13:46.169+0000 7f6af0000700 1 --2- 172.21.15.157:0/1720652437 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6ad0006280 0x7f6ad0008710 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:13:46.171 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-21T10:13:46.169+0000 7f6af0000700 1 -- 172.21.15.157:0/1720652437 >> 172.21.15.157:0/1720652437 conn(0x7f6ae80ff990 msgr2=0x7f6ae8101db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:13:46.171 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-21T10:13:46.170+0000 7f6af0000700 1 -- 172.21.15.157:0/1720652437 shutdown_connections 2024-03-21T10:13:46.172 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-21T10:13:46.170+0000 7f6af0000700 1 -- 172.21.15.157:0/1720652437 wait complete. 2024-03-21T10:13:46.173 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-21T10:13:46.171+0000 7f6af0000700 1 Processor -- start 2024-03-21T10:13:46.173 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-21T10:13:46.171+0000 7f6af0000700 1 -- start start 2024-03-21T10:13:46.173 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-21T10:13:46.172+0000 7f6af0000700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6ae819f8c0 0x7f6ae819fca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:13:46.173 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-21T10:13:46.172+0000 7f6af0000700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6ae81a01e0 0x7f6ae8199a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:13:46.173 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-21T10:13:46.172+0000 7f6af0000700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6ae819a0d0 0x7f6ae81a6790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:13:46.173 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-21T10:13:46.172+0000 7f6af0000700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ae810bda0 con 0x7f6ae819a0d0 2024-03-21T10:13:46.173 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-21T10:13:46.172+0000 7f6af0000700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ae810bc20 con 0x7f6ae819f8c0 2024-03-21T10:13:46.173 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-21T10:13:46.172+0000 7f6af0000700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ae810bf20 con 0x7f6ae81a01e0 2024-03-21T10:13:46.173 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-21T10:13:46.172+0000 7f6aecc00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6ae81a01e0 0x7f6ae8199a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:13:46.173 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-21T10:13:46.172+0000 7f6aecc00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6ae81a01e0 0x7f6ae8199a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:50580/0 (socket says 172.21.15.157:50580) 2024-03-21T10:13:46.173 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-21T10:13:46.172+0000 7f6aecc00700 1 -- 172.21.15.157:0/3602695635 learned_addr learned my addr 172.21.15.157:0/3602695635 (peer_addr_for_me v2:172.21.15.157:0/0) 2024-03-21T10:13:46.173 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-21T10:13:46.172+0000 7f6aee000700 1 --2- 172.21.15.157:0/3602695635 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6ae819a0d0 0x7f6ae81a6790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:13:46.174 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-21T10:13:46.173+0000 7f6aecc00700 1 -- 172.21.15.157:0/3602695635 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6ae819f8c0 msgr2=0x7f6ae819fca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:13:46.174 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-21T10:13:46.173+0000 7f6aecc00700 1 --2- 172.21.15.157:0/3602695635 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6ae819f8c0 0x7f6ae819fca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:13:46.174 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-21T10:13:46.173+0000 7f6aecc00700 1 -- 172.21.15.157:0/3602695635 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6ae819a0d0 msgr2=0x7f6ae81a6790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:13:46.175 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-21T10:13:46.173+0000 7f6aecc00700 1 --2- 172.21.15.157:0/3602695635 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6ae819a0d0 0x7f6ae81a6790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:13:46.175 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-21T10:13:46.173+0000 7f6aecc00700 1 -- 172.21.15.157:0/3602695635 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ae81a6cd0 con 0x7f6ae81a01e0 2024-03-21T10:13:46.175 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-21T10:13:46.173+0000 7f6aecc00700 1 --2- 172.21.15.157:0/3602695635 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6ae81a01e0 0x7f6ae8199a80 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f6ad800b780 tx=0x7f6ad800ba90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:13:46.175 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-21T10:13:46.174+0000 7f6ae6000700 1 -- 172.21.15.157:0/3602695635 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ad8004230 con 0x7f6ae81a01e0 2024-03-21T10:13:46.175 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-21T10:13:46.174+0000 7f6ae6000700 1 -- 172.21.15.157:0/3602695635 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f6ad80043d0 con 0x7f6ae81a01e0 2024-03-21T10:13:46.175 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-21T10:13:46.174+0000 7f6ae6000700 1 -- 172.21.15.157:0/3602695635 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ad8011b60 con 0x7f6ae81a01e0 2024-03-21T10:13:46.175 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-21T10:13:46.174+0000 7f6af0000700 1 -- 172.21.15.157:0/3602695635 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6ae81a6ea0 con 0x7f6ae81a01e0 2024-03-21T10:13:46.175 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-21T10:13:46.174+0000 7f6af0000700 1 -- 172.21.15.157:0/3602695635 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6ae81a7240 con 0x7f6ae81a01e0 2024-03-21T10:13:46.176 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-21T10:13:46.175+0000 7f6ae6000700 1 -- 172.21.15.157:0/3602695635 <== mon.1 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 13) v1 ==== 46350+0+0 (secure 0 0 0) 0x7f6ad80036a0 con 0x7f6ae81a01e0 2024-03-21T10:13:46.176 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-21T10:13:46.175+0000 7f6af0000700 1 -- 172.21.15.157:0/3602695635 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6ae80637c0 con 0x7f6ae81a01e0 2024-03-21T10:13:46.176 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-21T10:13:46.176+0000 7f6ae6000700 1 --2- 172.21.15.157:0/3602695635 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f6acc0396b0 0x7f6acc03bb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:13:46.176 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-21T10:13:46.176+0000 7f6ae6000700 1 -- 172.21.15.157:0/3602695635 <== mon.1 v2:172.21.15.157:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f6ad804bee0 con 0x7f6ae81a01e0 2024-03-21T10:13:46.177 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-21T10:13:46.177+0000 7f6aed600700 1 --2- 172.21.15.157:0/3602695635 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f6acc0396b0 0x7f6acc03bb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:13:46.178 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-21T10:13:46.178+0000 7f6aed600700 1 --2- 172.21.15.157:0/3602695635 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f6acc0396b0 0x7f6acc03bb70 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f6adc015040 tx=0x7f6adc01dd30 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:13:46.183 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-21T10:13:46.183+0000 7f6ae6000700 1 -- 172.21.15.157:0/3602695635 <== mon.1 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f6ad804f030 con 0x7f6ae81a01e0 2024-03-21T10:13:46.239 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:13:45 smithi157 ceph-mon[35581]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T10:13:46.239 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:13:45 smithi157 ceph-mon[35581]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-21T10:13:46.239 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:13:45 smithi157 ceph-mon[35581]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:13:46.463 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:13:45 smithi047 ceph-mon[31547]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T10:13:46.464 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:13:45 smithi047 ceph-mon[31547]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-21T10:13:46.464 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:13:45 smithi047 ceph-mon[31547]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:13:46.514 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-21T10:13:46.513+0000 7f6af0000700 1 -- 172.21.15.157:0/3602695635 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f6ae802db50 con 0x7f6ae81a01e0 2024-03-21T10:13:46.515 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-21T10:13:46.513+0000 7f6ae6000700 1 -- 172.21.15.157:0/3602695635 <== mon.1 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 3 v3) v1 ==== 95+0+1283 (secure 0 0 0) 0x7f6ad8014160 con 0x7f6ae81a01e0 2024-03-21T10:13:46.515 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-03-21T10:13:46.515 INFO:teuthology.orchestra.run.smithi157.stdout:{"epoch":3,"fsid":"3056d51a-e76b-11ee-95cd-87774f69a715","modified":"2024-03-21T10:13:39.873022Z","created":"2024-03-21T10:11:12.872646Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:3300","nonce":0},{"type":"v1","addr":"172.21.15.47:6789","nonce":0}]},"addr":"172.21.15.47:6789/0","public_addr":"172.21.15.47:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:3300","nonce":0},{"type":"v1","addr":"172.21.15.157:6789","nonce":0}]},"addr":"172.21.15.157:6789/0","public_addr":"172.21.15.157:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:3300","nonce":0},{"type":"v1","addr":"172.21.15.100:6789","nonce":0}]},"addr":"172.21.15.100:6789/0","public_addr":"172.21.15.100:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-03-21T10:13:46.517 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-21T10:13:46.516+0000 7f6af0000700 1 -- 172.21.15.157:0/3602695635 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f6acc0396b0 msgr2=0x7f6acc03bb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:13:46.518 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-21T10:13:46.516+0000 7f6af0000700 1 --2- 172.21.15.157:0/3602695635 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f6acc0396b0 0x7f6acc03bb70 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f6adc015040 tx=0x7f6adc01dd30 comp rx=0 tx=0).stop 2024-03-21T10:13:46.518 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-21T10:13:46.516+0000 7f6af0000700 1 -- 172.21.15.157:0/3602695635 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6ae81a01e0 msgr2=0x7f6ae8199a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:13:46.518 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-21T10:13:46.516+0000 7f6af0000700 1 --2- 172.21.15.157:0/3602695635 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6ae81a01e0 0x7f6ae8199a80 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f6ad800b780 tx=0x7f6ad800ba90 comp rx=0 tx=0).stop 2024-03-21T10:13:46.518 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-21T10:13:46.516+0000 7f6af0000700 1 -- 172.21.15.157:0/3602695635 shutdown_connections 2024-03-21T10:13:46.518 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-21T10:13:46.516+0000 7f6af0000700 1 --2- 172.21.15.157:0/3602695635 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f6acc0396b0 0x7f6acc03bb70 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:13:46.518 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-21T10:13:46.516+0000 7f6af0000700 1 --2- 172.21.15.157:0/3602695635 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6ae819a0d0 0x7f6ae81a6790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:13:46.518 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-21T10:13:46.516+0000 7f6af0000700 1 --2- 172.21.15.157:0/3602695635 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6ae819f8c0 0x7f6ae819fca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:13:46.518 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-21T10:13:46.516+0000 7f6af0000700 1 --2- 172.21.15.157:0/3602695635 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6ae81a01e0 0x7f6ae8199a80 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:13:46.518 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-21T10:13:46.516+0000 7f6af0000700 1 -- 172.21.15.157:0/3602695635 >> 172.21.15.157:0/3602695635 conn(0x7f6ae80ff990 msgr2=0x7f6ae8101130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:13:46.519 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-21T10:13:46.516+0000 7f6af0000700 1 -- 172.21.15.157:0/3602695635 shutdown_connections 2024-03-21T10:13:46.519 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-21T10:13:46.516+0000 7f6af0000700 1 -- 172.21.15.157:0/3602695635 wait complete. 2024-03-21T10:13:46.519 INFO:teuthology.orchestra.run.smithi157.stderr:dumped monmap epoch 3 2024-03-21T10:13:47.230 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:13:46 smithi100 ceph-mon[35193]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T10:13:47.230 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:13:46 smithi100 ceph-mon[35193]: mon.c calling monitor election 2024-03-21T10:13:47.230 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:13:46 smithi100 ceph-mon[35193]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-21T10:13:47.231 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:13:46 smithi100 ceph-mon[35193]: mon.a calling monitor election 2024-03-21T10:13:47.231 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:13:46 smithi100 ceph-mon[35193]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-21T10:13:47.231 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:13:46 smithi100 ceph-mon[35193]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-21T10:13:47.231 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:13:46 smithi100 ceph-mon[35193]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-21T10:13:47.231 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:13:46 smithi100 ceph-mon[35193]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T10:13:47.231 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:13:46 smithi100 ceph-mon[35193]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-21T10:13:47.231 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:13:46 smithi100 ceph-mon[35193]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-21T10:13:47.231 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:13:46 smithi100 ceph-mon[35193]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T10:13:47.231 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:13:46 smithi100 ceph-mon[35193]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-21T10:13:47.231 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:13:46 smithi100 ceph-mon[35193]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-21T10:13:47.231 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:13:46 smithi100 ceph-mon[35193]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-03-21T10:13:47.231 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:13:46 smithi100 ceph-mon[35193]: monmap e3: 3 mons at {a=[v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0],b=[v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0],c=[v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-03-21T10:13:47.231 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:13:46 smithi100 ceph-mon[35193]: fsmap 2024-03-21T10:13:47.231 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:13:46 smithi100 ceph-mon[35193]: osdmap e4: 0 total, 0 up, 0 in 2024-03-21T10:13:47.232 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:13:46 smithi100 ceph-mon[35193]: mgrmap e13: x(active, since 79s) 2024-03-21T10:13:47.232 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:13:46 smithi100 ceph-mon[35193]: Health check failed: 1/3 mons down, quorum a,c (MON_DOWN) 2024-03-21T10:13:47.232 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:13:46 smithi100 ceph-mon[35193]: Health detail: HEALTH_WARN 1/3 mons down, quorum a,c 2024-03-21T10:13:47.232 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:13:46 smithi100 ceph-mon[35193]: [WRN] MON_DOWN: 1/3 mons down, quorum a,c 2024-03-21T10:13:47.232 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:13:46 smithi100 ceph-mon[35193]: mon.b (rank 2) addr [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] is down (out of quorum) 2024-03-21T10:13:47.232 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:13:46 smithi100 ceph-mon[35193]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:13:47.232 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:13:46 smithi100 ceph-mon[35193]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:13:47.232 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:13:46 smithi100 ceph-mon[35193]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:13:47.232 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:13:46 smithi100 ceph-mon[35193]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:13:47.232 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:13:46 smithi100 ceph-mon[35193]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T10:13:47.232 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:13:46 smithi100 ceph-mon[35193]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-21T10:13:47.232 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:13:46 smithi100 ceph-mon[35193]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:13:47.232 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:13:46 smithi100 ceph-mon[35193]: mon.b calling monitor election 2024-03-21T10:13:47.232 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:13:46 smithi100 ceph-mon[35193]: mon.b calling monitor election 2024-03-21T10:13:47.233 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:13:46 smithi100 ceph-mon[35193]: mon.c calling monitor election 2024-03-21T10:13:47.233 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:13:46 smithi100 ceph-mon[35193]: mon.a calling monitor election 2024-03-21T10:13:47.233 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:13:46 smithi100 ceph-mon[35193]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-03-21T10:13:47.233 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:13:46 smithi100 ceph-mon[35193]: monmap e3: 3 mons at {a=[v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0],b=[v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0],c=[v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-03-21T10:13:47.233 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:13:46 smithi100 ceph-mon[35193]: fsmap 2024-03-21T10:13:47.233 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:13:46 smithi100 ceph-mon[35193]: osdmap e4: 0 total, 0 up, 0 in 2024-03-21T10:13:47.233 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:13:46 smithi100 ceph-mon[35193]: mgrmap e13: x(active, since 81s) 2024-03-21T10:13:47.233 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:13:46 smithi100 ceph-mon[35193]: Health check cleared: MON_DOWN (was: 1/3 mons down, quorum a,c) 2024-03-21T10:13:47.233 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:13:46 smithi100 ceph-mon[35193]: Cluster is now healthy 2024-03-21T10:13:47.233 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:13:46 smithi100 ceph-mon[35193]: overall HEALTH_OK 2024-03-21T10:13:47.240 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:13:46 smithi157 ceph-mon[35581]: mon.b calling monitor election 2024-03-21T10:13:47.240 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:13:46 smithi157 ceph-mon[35581]: mon.b calling monitor election 2024-03-21T10:13:47.240 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:13:46 smithi157 ceph-mon[35581]: mon.c calling monitor election 2024-03-21T10:13:47.240 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:13:46 smithi157 ceph-mon[35581]: mon.a calling monitor election 2024-03-21T10:13:47.240 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:13:46 smithi157 ceph-mon[35581]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-03-21T10:13:47.240 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:13:46 smithi157 ceph-mon[35581]: monmap e3: 3 mons at {a=[v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0],b=[v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0],c=[v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-03-21T10:13:47.240 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:13:46 smithi157 ceph-mon[35581]: fsmap 2024-03-21T10:13:47.240 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:13:46 smithi157 ceph-mon[35581]: osdmap e4: 0 total, 0 up, 0 in 2024-03-21T10:13:47.240 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:13:46 smithi157 ceph-mon[35581]: mgrmap e13: x(active, since 81s) 2024-03-21T10:13:47.240 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:13:46 smithi157 ceph-mon[35581]: Health check cleared: MON_DOWN (was: 1/3 mons down, quorum a,c) 2024-03-21T10:13:47.241 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:13:46 smithi157 ceph-mon[35581]: Cluster is now healthy 2024-03-21T10:13:47.241 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:13:46 smithi157 ceph-mon[35581]: overall HEALTH_OK 2024-03-21T10:13:47.383 INFO:tasks.cephadm:Generating final ceph.conf file... 2024-03-21T10:13:47.383 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2edfee68ad7243bed422a049b6f5f58619300f11 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3056d51a-e76b-11ee-95cd-87774f69a715 -- ceph config generate-minimal-conf 2024-03-21T10:13:47.464 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:13:46 smithi047 ceph-mon[31547]: mon.b calling monitor election 2024-03-21T10:13:47.464 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:13:46 smithi047 ceph-mon[31547]: mon.b calling monitor election 2024-03-21T10:13:47.464 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:13:46 smithi047 ceph-mon[31547]: mon.c calling monitor election 2024-03-21T10:13:47.464 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:13:46 smithi047 ceph-mon[31547]: mon.a calling monitor election 2024-03-21T10:13:47.464 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:13:46 smithi047 ceph-mon[31547]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-03-21T10:13:47.464 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:13:46 smithi047 ceph-mon[31547]: monmap e3: 3 mons at {a=[v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0],b=[v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0],c=[v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-03-21T10:13:47.465 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:13:46 smithi047 ceph-mon[31547]: fsmap 2024-03-21T10:13:47.465 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:13:46 smithi047 ceph-mon[31547]: osdmap e4: 0 total, 0 up, 0 in 2024-03-21T10:13:47.465 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:13:46 smithi047 ceph-mon[31547]: mgrmap e13: x(active, since 81s) 2024-03-21T10:13:47.465 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:13:46 smithi047 ceph-mon[31547]: Health check cleared: MON_DOWN (was: 1/3 mons down, quorum a,c) 2024-03-21T10:13:47.465 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:13:46 smithi047 ceph-mon[31547]: Cluster is now healthy 2024-03-21T10:13:47.465 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:13:46 smithi047 ceph-mon[31547]: overall HEALTH_OK 2024-03-21T10:13:47.851 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/3056d51a-e76b-11ee-95cd-87774f69a715/mon.a/config 2024-03-21T10:13:48.489 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:13:48 smithi157 ceph-mon[35581]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:13:48.489 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:13:48 smithi157 ceph-mon[35581]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:13:48.490 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:13:48 smithi157 ceph-mon[35581]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:13:48.490 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:13:48 smithi157 ceph-mon[35581]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:13:48.490 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:13:48 smithi157 ceph-mon[35581]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T10:13:48.490 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:13:48 smithi157 ceph-mon[35581]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T10:13:48.490 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:13:48 smithi157 ceph-mon[35581]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-21T10:13:48.714 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:13:48 smithi047 ceph-mon[31547]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:13:48.714 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:13:48 smithi047 ceph-mon[31547]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:13:48.714 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:13:48 smithi047 ceph-mon[31547]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:13:48.714 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:13:48 smithi047 ceph-mon[31547]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:13:48.714 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:13:48 smithi047 ceph-mon[31547]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T10:13:48.714 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:13:48 smithi047 ceph-mon[31547]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T10:13:48.714 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:13:48 smithi047 ceph-mon[31547]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-21T10:13:48.729 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:13:48 smithi100 ceph-mon[35193]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:13:48.729 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:13:48 smithi100 ceph-mon[35193]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:13:48.729 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:13:48 smithi100 ceph-mon[35193]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:13:48.729 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:13:48 smithi100 ceph-mon[35193]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:13:48.729 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:13:48 smithi100 ceph-mon[35193]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T10:13:48.729 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:13:48 smithi100 ceph-mon[35193]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T10:13:48.729 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:13:48 smithi100 ceph-mon[35193]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-21T10:13:49.213 INFO:journalctl@ceph.mgr.x.smithi047.stdout:Mar 21 10:13:48 smithi047 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-mgr-x[31773]: 2024-03-21T10:13:48.874+0000 7fc414400700 -1 mgr.server handle_report got status from non-daemon mon.b 2024-03-21T10:13:49.614 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:13:49 smithi047 ceph-mon[31547]: Updating smithi047:/etc/ceph/ceph.conf 2024-03-21T10:13:49.615 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:13:49 smithi047 ceph-mon[31547]: Updating smithi100:/etc/ceph/ceph.conf 2024-03-21T10:13:49.615 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:13:49 smithi047 ceph-mon[31547]: Updating smithi157:/etc/ceph/ceph.conf 2024-03-21T10:13:49.615 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:13:49 smithi047 ceph-mon[31547]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T10:13:49.615 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:13:49 smithi047 ceph-mon[31547]: Updating smithi047:/var/lib/ceph/3056d51a-e76b-11ee-95cd-87774f69a715/config/ceph.conf 2024-03-21T10:13:49.615 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:13:49 smithi047 ceph-mon[31547]: Updating smithi100:/var/lib/ceph/3056d51a-e76b-11ee-95cd-87774f69a715/config/ceph.conf 2024-03-21T10:13:49.615 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:13:49 smithi047 ceph-mon[31547]: Updating smithi157:/var/lib/ceph/3056d51a-e76b-11ee-95cd-87774f69a715/config/ceph.conf 2024-03-21T10:13:49.615 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:13:49 smithi047 ceph-mon[31547]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:13:49.615 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:13:49 smithi047 ceph-mon[31547]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:13:49.729 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:13:49 smithi100 ceph-mon[35193]: Updating smithi047:/etc/ceph/ceph.conf 2024-03-21T10:13:49.729 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:13:49 smithi100 ceph-mon[35193]: Updating smithi100:/etc/ceph/ceph.conf 2024-03-21T10:13:49.730 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:13:49 smithi100 ceph-mon[35193]: Updating smithi157:/etc/ceph/ceph.conf 2024-03-21T10:13:49.730 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:13:49 smithi100 ceph-mon[35193]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T10:13:49.730 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:13:49 smithi100 ceph-mon[35193]: Updating smithi047:/var/lib/ceph/3056d51a-e76b-11ee-95cd-87774f69a715/config/ceph.conf 2024-03-21T10:13:49.730 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:13:49 smithi100 ceph-mon[35193]: Updating smithi100:/var/lib/ceph/3056d51a-e76b-11ee-95cd-87774f69a715/config/ceph.conf 2024-03-21T10:13:49.730 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:13:49 smithi100 ceph-mon[35193]: Updating smithi157:/var/lib/ceph/3056d51a-e76b-11ee-95cd-87774f69a715/config/ceph.conf 2024-03-21T10:13:49.730 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:13:49 smithi100 ceph-mon[35193]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:13:49.730 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:13:49 smithi100 ceph-mon[35193]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:13:49.739 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:13:49 smithi157 ceph-mon[35581]: Updating smithi047:/etc/ceph/ceph.conf 2024-03-21T10:13:49.739 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:13:49 smithi157 ceph-mon[35581]: Updating smithi100:/etc/ceph/ceph.conf 2024-03-21T10:13:49.739 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:13:49 smithi157 ceph-mon[35581]: Updating smithi157:/etc/ceph/ceph.conf 2024-03-21T10:13:49.739 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:13:49 smithi157 ceph-mon[35581]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T10:13:49.740 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:13:49 smithi157 ceph-mon[35581]: Updating smithi047:/var/lib/ceph/3056d51a-e76b-11ee-95cd-87774f69a715/config/ceph.conf 2024-03-21T10:13:49.740 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:13:49 smithi157 ceph-mon[35581]: Updating smithi100:/var/lib/ceph/3056d51a-e76b-11ee-95cd-87774f69a715/config/ceph.conf 2024-03-21T10:13:49.740 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:13:49 smithi157 ceph-mon[35581]: Updating smithi157:/var/lib/ceph/3056d51a-e76b-11ee-95cd-87774f69a715/config/ceph.conf 2024-03-21T10:13:49.740 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:13:49 smithi157 ceph-mon[35581]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:13:49.740 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:13:49 smithi157 ceph-mon[35581]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:13:50.093 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:13:50.091+0000 7fbcc7200700 1 -- 172.21.15.47:0/1457655378 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbcc00745c0 msgr2=0x7fbcc00749a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:13:50.094 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:13:50.091+0000 7fbcc7200700 1 --2- 172.21.15.47:0/1457655378 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbcc00745c0 0x7fbcc00749a0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fbcb0009910 tx=0x7fbcb0009c20 comp rx=0 tx=0).stop 2024-03-21T10:13:50.094 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:13:50.092+0000 7fbcc7200700 1 -- 172.21.15.47:0/1457655378 shutdown_connections 2024-03-21T10:13:50.094 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:13:50.092+0000 7fbcc7200700 1 --2- 172.21.15.47:0/1457655378 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbcc00745c0 0x7fbcc00749a0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:13:50.094 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:13:50.092+0000 7fbcc7200700 1 -- 172.21.15.47:0/1457655378 >> 172.21.15.47:0/1457655378 conn(0x7fbcc00fd670 msgr2=0x7fbcc00ffa90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:13:50.094 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:13:50.092+0000 7fbcc7200700 1 -- 172.21.15.47:0/1457655378 shutdown_connections 2024-03-21T10:13:50.094 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:13:50.092+0000 7fbcc7200700 1 -- 172.21.15.47:0/1457655378 wait complete. 2024-03-21T10:13:50.095 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:13:50.093+0000 7fbcc7200700 1 Processor -- start 2024-03-21T10:13:50.095 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:13:50.093+0000 7fbcc7200700 1 -- start start 2024-03-21T10:13:50.095 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:13:50.094+0000 7fbcc7200700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbcc00745c0 0x7fbcc019f700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:13:50.095 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:13:50.094+0000 7fbcc7200700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbcc019fc40 0x7fbcc0199960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:13:50.095 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:13:50.094+0000 7fbcc7200700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbcc0199ea0 0x7fbcc019a350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:13:50.095 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:13:50.094+0000 7fbcc7200700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbcc010e2c0 con 0x7fbcc00745c0 2024-03-21T10:13:50.095 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:13:50.094+0000 7fbcc7200700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbcc010e140 con 0x7fbcc0199ea0 2024-03-21T10:13:50.096 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:13:50.094+0000 7fbcc7200700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbcc010e440 con 0x7fbcc019fc40 2024-03-21T10:13:50.096 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:13:50.094+0000 7fbcbfe00700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbcc00745c0 0x7fbcc019f700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:13:50.096 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:13:50.094+0000 7fbcbfe00700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbcc00745c0 0x7fbcc019f700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:43350/0 (socket says 172.21.15.47:43350) 2024-03-21T10:13:50.096 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:13:50.094+0000 7fbcbfe00700 1 -- 172.21.15.47:0/1178375838 learned_addr learned my addr 172.21.15.47:0/1178375838 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:13:50.097 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:13:50.094+0000 7fbcbf400700 1 --2- 172.21.15.47:0/1178375838 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbcc019fc40 0x7fbcc0199960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:13:50.097 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:13:50.094+0000 7fbcc5200700 1 --2- 172.21.15.47:0/1178375838 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbcc0199ea0 0x7fbcc019a350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:13:50.097 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:13:50.095+0000 7fbcbfe00700 1 -- 172.21.15.47:0/1178375838 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbcc019fc40 msgr2=0x7fbcc0199960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:13:50.097 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:13:50.095+0000 7fbcbfe00700 1 --2- 172.21.15.47:0/1178375838 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbcc019fc40 0x7fbcc0199960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:13:50.097 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:13:50.095+0000 7fbcbfe00700 1 -- 172.21.15.47:0/1178375838 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbcc0199ea0 msgr2=0x7fbcc019a350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:13:50.097 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:13:50.095+0000 7fbcbfe00700 1 --2- 172.21.15.47:0/1178375838 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbcc0199ea0 0x7fbcc019a350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:13:50.097 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:13:50.095+0000 7fbcbfe00700 1 -- 172.21.15.47:0/1178375838 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbcc01a65d0 con 0x7fbcc00745c0 2024-03-21T10:13:50.097 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:13:50.095+0000 7fbcc5200700 1 --2- 172.21.15.47:0/1178375838 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbcc0199ea0 0x7fbcc019a350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T10:13:50.098 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:13:50.095+0000 7fbcbfe00700 1 --2- 172.21.15.47:0/1178375838 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbcc00745c0 0x7fbcc019f700 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fbcb0004f80 tx=0x7fbcb0005060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:13:50.098 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:13:50.095+0000 7fbcbcc00700 1 -- 172.21.15.47:0/1178375838 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbcb0027070 con 0x7fbcc00745c0 2024-03-21T10:13:50.098 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:13:50.095+0000 7fbcbcc00700 1 -- 172.21.15.47:0/1178375838 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fbcb000be10 con 0x7fbcc00745c0 2024-03-21T10:13:50.098 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:13:50.095+0000 7fbcc7200700 1 -- 172.21.15.47:0/1178375838 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbcc01a67a0 con 0x7fbcc00745c0 2024-03-21T10:13:50.098 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:13:50.096+0000 7fbcbcc00700 1 -- 172.21.15.47:0/1178375838 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbcb000f460 con 0x7fbcc00745c0 2024-03-21T10:13:50.098 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:13:50.096+0000 7fbcc7200700 1 -- 172.21.15.47:0/1178375838 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbcc01a6cd0 con 0x7fbcc00745c0 2024-03-21T10:13:50.099 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:13:50.097+0000 7fbcbcc00700 1 -- 172.21.15.47:0/1178375838 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 13) v1 ==== 46350+0+0 (secure 0 0 0) 0x7fbcb002a070 con 0x7fbcc00745c0 2024-03-21T10:13:50.099 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:13:50.097+0000 7fbcb6000700 1 -- 172.21.15.47:0/1178375838 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbcc00637c0 con 0x7fbcc00745c0 2024-03-21T10:13:50.099 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:13:50.097+0000 7fbcbcc00700 1 --2- 172.21.15.47:0/1178375838 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7fbca40396b0 0x7fbca403bb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:13:50.099 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:13:50.097+0000 7fbcbcc00700 1 -- 172.21.15.47:0/1178375838 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fbcb004d620 con 0x7fbcc00745c0 2024-03-21T10:13:50.099 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:13:50.098+0000 7fbcbf400700 1 --2- 172.21.15.47:0/1178375838 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7fbca40396b0 0x7fbca403bb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:13:50.099 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:13:50.099+0000 7fbcbf400700 1 --2- 172.21.15.47:0/1178375838 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7fbca40396b0 0x7fbca403bb70 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fbcac009850 tx=0x7fbcac008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:13:50.105 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:13:50.104+0000 7fbcbcc00700 1 -- 172.21.15.47:0/1178375838 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fbcb0013070 con 0x7fbcc00745c0 2024-03-21T10:13:50.338 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:13:50.337+0000 7fbcb6000700 1 -- 172.21.15.47:0/1178375838 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) v1 -- 0x7fbcc0062da0 con 0x7fbcc00745c0 2024-03-21T10:13:50.339 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:13:50.338+0000 7fbcbcc00700 1 -- 172.21.15.47:0/1178375838 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v9) v1 ==== 76+0+275 (secure 0 0 0) 0x7fbcb002a360 con 0x7fbcc00745c0 2024-03-21T10:13:50.339 INFO:teuthology.orchestra.run.smithi047.stdout:# minimal ceph.conf for 3056d51a-e76b-11ee-95cd-87774f69a715 2024-03-21T10:13:50.339 INFO:teuthology.orchestra.run.smithi047.stdout:[global] 2024-03-21T10:13:50.340 INFO:teuthology.orchestra.run.smithi047.stdout: fsid = 3056d51a-e76b-11ee-95cd-87774f69a715 2024-03-21T10:13:50.340 INFO:teuthology.orchestra.run.smithi047.stdout: mon_host = [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] 2024-03-21T10:13:50.343 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:13:50.342+0000 7fbcb6000700 1 -- 172.21.15.47:0/1178375838 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7fbca40396b0 msgr2=0x7fbca403bb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:13:50.343 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:13:50.342+0000 7fbcb6000700 1 --2- 172.21.15.47:0/1178375838 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7fbca40396b0 0x7fbca403bb70 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fbcac009850 tx=0x7fbcac008040 comp rx=0 tx=0).stop 2024-03-21T10:13:50.343 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:13:50.342+0000 7fbcb6000700 1 -- 172.21.15.47:0/1178375838 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbcc00745c0 msgr2=0x7fbcc019f700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:13:50.344 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:13:50.342+0000 7fbcb6000700 1 --2- 172.21.15.47:0/1178375838 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbcc00745c0 0x7fbcc019f700 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fbcb0004f80 tx=0x7fbcb0005060 comp rx=0 tx=0).stop 2024-03-21T10:13:50.344 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:13:50.342+0000 7fbcb6000700 1 -- 172.21.15.47:0/1178375838 shutdown_connections 2024-03-21T10:13:50.344 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:13:50.342+0000 7fbcb6000700 1 --2- 172.21.15.47:0/1178375838 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7fbca40396b0 0x7fbca403bb70 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:13:50.344 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:13:50.342+0000 7fbcb6000700 1 --2- 172.21.15.47:0/1178375838 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbcc0199ea0 0x7fbcc019a350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:13:50.344 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:13:50.342+0000 7fbcb6000700 1 --2- 172.21.15.47:0/1178375838 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbcc00745c0 0x7fbcc019f700 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:13:50.344 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:13:50.342+0000 7fbcb6000700 1 --2- 172.21.15.47:0/1178375838 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbcc019fc40 0x7fbcc0199960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:13:50.344 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:13:50.342+0000 7fbcb6000700 1 -- 172.21.15.47:0/1178375838 >> 172.21.15.47:0/1178375838 conn(0x7fbcc00fd670 msgr2=0x7fbcc0107e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:13:50.344 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:13:50.342+0000 7fbcb6000700 1 -- 172.21.15.47:0/1178375838 shutdown_connections 2024-03-21T10:13:50.344 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:13:50.342+0000 7fbcb6000700 1 -- 172.21.15.47:0/1178375838 wait complete. 2024-03-21T10:13:50.698 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:13:50 smithi047 ceph-mon[31547]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:13:50.698 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:13:50 smithi047 ceph-mon[31547]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:13:50.698 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:13:50 smithi047 ceph-mon[31547]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:13:50.698 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:13:50 smithi047 ceph-mon[31547]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:13:50.698 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:13:50 smithi047 ceph-mon[31547]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:13:50.698 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:13:50 smithi047 ceph-mon[31547]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:13:50.698 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:13:50 smithi047 ceph-mon[31547]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:13:50.698 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:13:50 smithi047 ceph-mon[31547]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:13:50.699 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:13:50 smithi047 ceph-mon[31547]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:13:50.699 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:13:50 smithi047 ceph-mon[31547]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-21T10:13:50.699 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:13:50 smithi047 ceph-mon[31547]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-03-21T10:13:50.699 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:13:50 smithi047 ceph-mon[31547]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T10:13:50.729 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:13:50 smithi100 ceph-mon[35193]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:13:50.730 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:13:50 smithi100 ceph-mon[35193]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:13:50.730 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:13:50 smithi100 ceph-mon[35193]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:13:50.730 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:13:50 smithi100 ceph-mon[35193]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:13:50.730 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:13:50 smithi100 ceph-mon[35193]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:13:50.730 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:13:50 smithi100 ceph-mon[35193]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:13:50.730 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:13:50 smithi100 ceph-mon[35193]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:13:50.730 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:13:50 smithi100 ceph-mon[35193]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:13:50.730 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:13:50 smithi100 ceph-mon[35193]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:13:50.730 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:13:50 smithi100 ceph-mon[35193]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-21T10:13:50.730 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:13:50 smithi100 ceph-mon[35193]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-03-21T10:13:50.730 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:13:50 smithi100 ceph-mon[35193]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T10:13:50.740 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:13:50 smithi157 ceph-mon[35581]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:13:50.740 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:13:50 smithi157 ceph-mon[35581]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:13:50.740 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:13:50 smithi157 ceph-mon[35581]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:13:50.740 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:13:50 smithi157 ceph-mon[35581]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:13:50.740 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:13:50 smithi157 ceph-mon[35581]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:13:50.740 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:13:50 smithi157 ceph-mon[35581]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:13:50.740 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:13:50 smithi157 ceph-mon[35581]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:13:50.740 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:13:50 smithi157 ceph-mon[35581]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:13:50.740 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:13:50 smithi157 ceph-mon[35581]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:13:50.740 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:13:50 smithi157 ceph-mon[35581]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-21T10:13:50.740 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:13:50 smithi157 ceph-mon[35581]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-03-21T10:13:50.741 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:13:50 smithi157 ceph-mon[35581]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T10:13:51.114 INFO:tasks.cephadm:Distributing (final) config and client.admin keyring... 2024-03-21T10:13:51.114 DEBUG:teuthology.orchestra.run.smithi047:> set -ex 2024-03-21T10:13:51.114 DEBUG:teuthology.orchestra.run.smithi047:> sudo dd of=/etc/ceph/ceph.conf 2024-03-21T10:13:51.208 DEBUG:teuthology.orchestra.run.smithi047:> set -ex 2024-03-21T10:13:51.208 DEBUG:teuthology.orchestra.run.smithi047:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-03-21T10:13:51.292 DEBUG:teuthology.orchestra.run.smithi100:> set -ex 2024-03-21T10:13:51.293 DEBUG:teuthology.orchestra.run.smithi100:> sudo dd of=/etc/ceph/ceph.conf 2024-03-21T10:13:51.396 DEBUG:teuthology.orchestra.run.smithi100:> set -ex 2024-03-21T10:13:51.396 DEBUG:teuthology.orchestra.run.smithi100:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-03-21T10:13:51.534 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2024-03-21T10:13:51.534 DEBUG:teuthology.orchestra.run.smithi157:> sudo dd of=/etc/ceph/ceph.conf 2024-03-21T10:13:51.567 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:13:51 smithi047 ceph-mon[31547]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T10:13:51.567 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:13:51 smithi047 ceph-mon[31547]: Reconfiguring mon.a (unknown last config time)... 2024-03-21T10:13:51.567 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:13:51 smithi047 ceph-mon[31547]: Reconfiguring daemon mon.a on smithi047 2024-03-21T10:13:51.567 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:13:51 smithi047 ceph-mon[31547]: from='client.? 172.21.15.47:0/1178375838' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T10:13:51.620 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:13:51 smithi157 ceph-mon[35581]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T10:13:51.621 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:13:51 smithi157 ceph-mon[35581]: Reconfiguring mon.a (unknown last config time)... 2024-03-21T10:13:51.621 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:13:51 smithi157 ceph-mon[35581]: Reconfiguring daemon mon.a on smithi047 2024-03-21T10:13:51.621 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:13:51 smithi157 ceph-mon[35581]: from='client.? 172.21.15.47:0/1178375838' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T10:13:51.637 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2024-03-21T10:13:51.637 DEBUG:teuthology.orchestra.run.smithi157:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-03-21T10:13:51.729 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:13:51 smithi100 ceph-mon[35193]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T10:13:51.730 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:13:51 smithi100 ceph-mon[35193]: Reconfiguring mon.a (unknown last config time)... 2024-03-21T10:13:51.730 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:13:51 smithi100 ceph-mon[35193]: Reconfiguring daemon mon.a on smithi047 2024-03-21T10:13:51.730 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:13:51 smithi100 ceph-mon[35193]: from='client.? 172.21.15.47:0/1178375838' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T10:13:51.776 INFO:tasks.cephadm:Adding mgr.x on smithi047 2024-03-21T10:13:51.776 INFO:tasks.cephadm:Adding mgr.y on smithi100 2024-03-21T10:13:51.776 INFO:tasks.cephadm:Adding mgr.z on smithi157 2024-03-21T10:13:51.776 DEBUG:teuthology.orchestra.run.smithi157:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2edfee68ad7243bed422a049b6f5f58619300f11 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3056d51a-e76b-11ee-95cd-87774f69a715 -- ceph orch apply mgr '3;smithi047=x;smithi100=y;smithi157=z' 2024-03-21T10:13:52.253 INFO:teuthology.orchestra.run.smithi157.stderr:Inferring config /var/lib/ceph/3056d51a-e76b-11ee-95cd-87774f69a715/mon.c/config 2024-03-21T10:13:53.653 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:13:53 smithi100 ceph-mon[35193]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T10:13:53.653 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:13:53 smithi100 ceph-mon[35193]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:13:53.654 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:13:53 smithi100 ceph-mon[35193]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:13:53.654 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:13:53 smithi100 ceph-mon[35193]: Reconfiguring mon.b (monmap changed)... 2024-03-21T10:13:53.654 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:13:53 smithi100 ceph-mon[35193]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-21T10:13:53.654 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:13:53 smithi100 ceph-mon[35193]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-03-21T10:13:53.654 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:13:53 smithi100 ceph-mon[35193]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T10:13:53.654 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:13:53 smithi100 ceph-mon[35193]: Reconfiguring daemon mon.b on smithi100 2024-03-21T10:13:53.691 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:13:53 smithi157 ceph-mon[35581]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T10:13:53.691 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:13:53 smithi157 ceph-mon[35581]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:13:53.691 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:13:53 smithi157 ceph-mon[35581]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:13:53.691 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:13:53 smithi157 ceph-mon[35581]: Reconfiguring mon.b (monmap changed)... 2024-03-21T10:13:53.691 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:13:53 smithi157 ceph-mon[35581]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-21T10:13:53.691 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:13:53 smithi157 ceph-mon[35581]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-03-21T10:13:53.691 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:13:53 smithi157 ceph-mon[35581]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T10:13:53.691 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:13:53 smithi157 ceph-mon[35581]: Reconfiguring daemon mon.b on smithi100 2024-03-21T10:13:53.964 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:13:53 smithi047 ceph-mon[31547]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T10:13:53.964 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:13:53 smithi047 ceph-mon[31547]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:13:53.964 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:13:53 smithi047 ceph-mon[31547]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:13:53.964 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:13:53 smithi047 ceph-mon[31547]: Reconfiguring mon.b (monmap changed)... 2024-03-21T10:13:53.965 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:13:53 smithi047 ceph-mon[31547]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-21T10:13:53.965 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:13:53 smithi047 ceph-mon[31547]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-03-21T10:13:53.965 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:13:53 smithi047 ceph-mon[31547]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T10:13:53.965 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:13:53 smithi047 ceph-mon[31547]: Reconfiguring daemon mon.b on smithi100 2024-03-21T10:13:54.138 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-21T10:13:54.136+0000 7f3317400700 1 -- 172.21.15.157:0/1241474604 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f33101099d0 msgr2=0x7f3310109df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:13:54.138 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-21T10:13:54.136+0000 7f3317400700 1 --2- 172.21.15.157:0/1241474604 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f33101099d0 0x7f3310109df0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f3304009910 tx=0x7f3304009c20 comp rx=0 tx=0).stop 2024-03-21T10:13:54.139 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-21T10:13:54.136+0000 7f3317400700 1 -- 172.21.15.157:0/1241474604 shutdown_connections 2024-03-21T10:13:54.139 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-21T10:13:54.136+0000 7f3317400700 1 --2- 172.21.15.157:0/1241474604 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f33101099d0 0x7f3310109df0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:13:54.139 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-21T10:13:54.136+0000 7f3317400700 1 -- 172.21.15.157:0/1241474604 >> 172.21.15.157:0/1241474604 conn(0x7f3310077480 msgr2=0x7f33100798c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:13:54.139 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-21T10:13:54.137+0000 7f3317400700 1 -- 172.21.15.157:0/1241474604 shutdown_connections 2024-03-21T10:13:54.139 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-21T10:13:54.137+0000 7f3317400700 1 -- 172.21.15.157:0/1241474604 wait complete. 2024-03-21T10:13:54.141 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-21T10:13:54.138+0000 7f3317400700 1 Processor -- start 2024-03-21T10:13:54.141 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-21T10:13:54.138+0000 7f3317400700 1 -- start start 2024-03-21T10:13:54.142 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-21T10:13:54.138+0000 7f3317400700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f33101099d0 0x7f331019db00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:13:54.142 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-21T10:13:54.138+0000 7f3317400700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f331019e040 0x7f33101a30b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:13:54.142 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-21T10:13:54.138+0000 7f3317400700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f331019e4c0 0x7f33101a5640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:13:54.142 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-21T10:13:54.138+0000 7f3317400700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f331010e360 con 0x7f331019e4c0 2024-03-21T10:13:54.142 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-21T10:13:54.138+0000 7f3317400700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f331010e1e0 con 0x7f331019e040 2024-03-21T10:13:54.145 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-21T10:13:54.138+0000 7f3317400700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f331010e4e0 con 0x7f33101099d0 2024-03-21T10:13:54.145 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-21T10:13:54.139+0000 7f3314a00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f33101099d0 0x7f331019db00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:13:54.145 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-21T10:13:54.139+0000 7f3314a00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f33101099d0 0x7f331019db00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:56550/0 (socket says 172.21.15.157:56550) 2024-03-21T10:13:54.145 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-21T10:13:54.139+0000 7f3314a00700 1 -- 172.21.15.157:0/2359951834 learned_addr learned my addr 172.21.15.157:0/2359951834 (peer_addr_for_me v2:172.21.15.157:0/0) 2024-03-21T10:13:54.146 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-21T10:13:54.139+0000 7f3315400700 1 --2- 172.21.15.157:0/2359951834 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f331019e4c0 0x7f33101a5640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:13:54.146 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-21T10:13:54.139+0000 7f3314a00700 1 -- 172.21.15.157:0/2359951834 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f331019e040 msgr2=0x7f33101a30b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:13:54.146 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-21T10:13:54.139+0000 7f330fe00700 1 --2- 172.21.15.157:0/2359951834 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f331019e040 0x7f33101a30b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:13:54.146 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-21T10:13:54.139+0000 7f3314a00700 1 --2- 172.21.15.157:0/2359951834 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f331019e040 0x7f33101a30b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:13:54.146 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-21T10:13:54.139+0000 7f3314a00700 1 -- 172.21.15.157:0/2359951834 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f331019e4c0 msgr2=0x7f33101a5640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:13:54.146 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-21T10:13:54.139+0000 7f3314a00700 1 --2- 172.21.15.157:0/2359951834 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f331019e4c0 0x7f33101a5640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:13:54.146 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-21T10:13:54.139+0000 7f3314a00700 1 -- 172.21.15.157:0/2359951834 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f33101a5f10 con 0x7f33101099d0 2024-03-21T10:13:54.146 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-21T10:13:54.139+0000 7f3315400700 1 --2- 172.21.15.157:0/2359951834 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f331019e4c0 0x7f33101a5640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T10:13:54.146 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-21T10:13:54.140+0000 7f3314a00700 1 --2- 172.21.15.157:0/2359951834 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f33101099d0 0x7f331019db00 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f3304005e00 tx=0x7f3304005050 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:13:54.146 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-21T10:13:54.140+0000 7f330d600700 1 -- 172.21.15.157:0/2359951834 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3304026070 con 0x7f33101099d0 2024-03-21T10:13:54.146 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-21T10:13:54.140+0000 7f330d600700 1 -- 172.21.15.157:0/2359951834 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f330400be10 con 0x7f33101099d0 2024-03-21T10:13:54.146 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-21T10:13:54.140+0000 7f3317400700 1 -- 172.21.15.157:0/2359951834 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f33101a61a0 con 0x7f33101099d0 2024-03-21T10:13:54.147 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-21T10:13:54.140+0000 7f330d600700 1 -- 172.21.15.157:0/2359951834 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3304017440 con 0x7f33101099d0 2024-03-21T10:13:54.147 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-21T10:13:54.140+0000 7f3317400700 1 -- 172.21.15.157:0/2359951834 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f33101a6700 con 0x7f33101099d0 2024-03-21T10:13:54.147 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-21T10:13:54.141+0000 7f3317400700 1 -- 172.21.15.157:0/2359951834 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f33100637c0 con 0x7f33101099d0 2024-03-21T10:13:54.148 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-21T10:13:54.142+0000 7f330d600700 1 -- 172.21.15.157:0/2359951834 <== mon.1 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 13) v1 ==== 46350+0+0 (secure 0 0 0) 0x7f3304029070 con 0x7f33101099d0 2024-03-21T10:13:54.148 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-21T10:13:54.142+0000 7f330d600700 1 --2- 172.21.15.157:0/2359951834 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f32f80396b0 0x7f32f803bb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:13:54.148 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-21T10:13:54.142+0000 7f330d600700 1 -- 172.21.15.157:0/2359951834 <== mon.1 v2:172.21.15.157:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f330404d1a0 con 0x7f33101099d0 2024-03-21T10:13:54.148 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-21T10:13:54.143+0000 7f330fe00700 1 --2- 172.21.15.157:0/2359951834 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f32f80396b0 0x7f32f803bb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:13:54.148 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-21T10:13:54.144+0000 7f330fe00700 1 --2- 172.21.15.157:0/2359951834 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f32f80396b0 0x7f32f803bb70 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f32fc0078e0 tx=0x7f32fc008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:13:54.150 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-21T10:13:54.148+0000 7f330d600700 1 -- 172.21.15.157:0/2359951834 <== mon.1 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f3304013070 con 0x7f33101099d0 2024-03-21T10:13:54.385 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-21T10:13:54.383+0000 7f3317400700 1 -- 172.21.15.157:0/2359951834 --> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi047=x;smithi100=y;smithi157=z", "target": ["mon-mgr", ""]}) v1 -- 0x7f33100625d0 con 0x7f32f80396b0 2024-03-21T10:13:54.399 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-21T10:13:54.398+0000 7f330d600700 1 -- 172.21.15.157:0/2359951834 <== mgr.14152 v2:172.21.15.47:6800/2823885994 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+24 (secure 0 0 0) 0x7f33100625d0 con 0x7f32f80396b0 2024-03-21T10:13:54.399 INFO:teuthology.orchestra.run.smithi157.stdout:Scheduled mgr update... 2024-03-21T10:13:54.402 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-21T10:13:54.401+0000 7f3317400700 1 -- 172.21.15.157:0/2359951834 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f32f80396b0 msgr2=0x7f32f803bb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:13:54.402 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-21T10:13:54.401+0000 7f3317400700 1 --2- 172.21.15.157:0/2359951834 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f32f80396b0 0x7f32f803bb70 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f32fc0078e0 tx=0x7f32fc008040 comp rx=0 tx=0).stop 2024-03-21T10:13:54.402 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-21T10:13:54.401+0000 7f3317400700 1 -- 172.21.15.157:0/2359951834 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f33101099d0 msgr2=0x7f331019db00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:13:54.402 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-21T10:13:54.401+0000 7f3317400700 1 --2- 172.21.15.157:0/2359951834 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f33101099d0 0x7f331019db00 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f3304005e00 tx=0x7f3304005050 comp rx=0 tx=0).stop 2024-03-21T10:13:54.403 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-21T10:13:54.401+0000 7f3317400700 1 -- 172.21.15.157:0/2359951834 shutdown_connections 2024-03-21T10:13:54.403 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-21T10:13:54.401+0000 7f3317400700 1 --2- 172.21.15.157:0/2359951834 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f32f80396b0 0x7f32f803bb70 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:13:54.403 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-21T10:13:54.401+0000 7f3317400700 1 --2- 172.21.15.157:0/2359951834 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f331019e4c0 0x7f33101a5640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:13:54.403 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-21T10:13:54.401+0000 7f3317400700 1 --2- 172.21.15.157:0/2359951834 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f33101099d0 0x7f331019db00 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:13:54.403 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-21T10:13:54.401+0000 7f3317400700 1 --2- 172.21.15.157:0/2359951834 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f331019e040 0x7f33101a30b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:13:54.403 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-21T10:13:54.401+0000 7f3317400700 1 -- 172.21.15.157:0/2359951834 >> 172.21.15.157:0/2359951834 conn(0x7f3310077480 msgr2=0x7f3310078090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:13:54.403 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-21T10:13:54.401+0000 7f3317400700 1 -- 172.21.15.157:0/2359951834 shutdown_connections 2024-03-21T10:13:54.403 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-21T10:13:54.401+0000 7f3317400700 1 -- 172.21.15.157:0/2359951834 wait complete. 2024-03-21T10:13:55.134 DEBUG:teuthology.orchestra.run.smithi100:mgr.y> sudo journalctl -f -n 0 -u ceph-3056d51a-e76b-11ee-95cd-87774f69a715@mgr.y.service 2024-03-21T10:13:55.142 DEBUG:teuthology.orchestra.run.smithi157:mgr.z> sudo journalctl -f -n 0 -u ceph-3056d51a-e76b-11ee-95cd-87774f69a715@mgr.z.service 2024-03-21T10:13:55.148 INFO:tasks.cephadm:Deploying OSDs... 2024-03-21T10:13:55.148 DEBUG:teuthology.orchestra.run.smithi047:> set -ex 2024-03-21T10:13:55.148 DEBUG:teuthology.orchestra.run.smithi047:> dd if=/scratch_devs of=/dev/stdout 2024-03-21T10:13:55.208 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-03-21T10:13:55.208 DEBUG:teuthology.orchestra.run.smithi047:> stat /dev/vg_nvme/lv_1 2024-03-21T10:13:55.267 INFO:teuthology.orchestra.run.smithi047.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-03-21T10:13:55.267 INFO:teuthology.orchestra.run.smithi047.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-21T10:13:55.267 INFO:teuthology.orchestra.run.smithi047.stdout:Device: 5h/5d Inode: 632 Links: 1 2024-03-21T10:13:55.267 INFO:teuthology.orchestra.run.smithi047.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-21T10:13:55.267 INFO:teuthology.orchestra.run.smithi047.stdout:Context: system_u:object_r:device_t:s0 2024-03-21T10:13:55.268 INFO:teuthology.orchestra.run.smithi047.stdout:Access: 2024-03-21 10:12:38.826179789 +0000 2024-03-21T10:13:55.268 INFO:teuthology.orchestra.run.smithi047.stdout:Modify: 2024-03-21 10:07:01.094427574 +0000 2024-03-21T10:13:55.268 INFO:teuthology.orchestra.run.smithi047.stdout:Change: 2024-03-21 10:07:01.094427574 +0000 2024-03-21T10:13:55.268 INFO:teuthology.orchestra.run.smithi047.stdout: Birth: 2024-03-21 10:07:01.094427574 +0000 2024-03-21T10:13:55.268 DEBUG:teuthology.orchestra.run.smithi047:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-03-21T10:13:55.364 INFO:teuthology.orchestra.run.smithi047.stderr:1+0 records in 2024-03-21T10:13:55.364 INFO:teuthology.orchestra.run.smithi047.stderr:1+0 records out 2024-03-21T10:13:55.364 INFO:teuthology.orchestra.run.smithi047.stderr:512 bytes copied, 0.000276547 s, 1.9 MB/s 2024-03-21T10:13:55.368 DEBUG:teuthology.orchestra.run.smithi047:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-03-21T10:13:55.428 DEBUG:teuthology.orchestra.run.smithi047:> stat /dev/vg_nvme/lv_2 2024-03-21T10:13:55.480 INFO:teuthology.orchestra.run.smithi047.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-03-21T10:13:55.480 INFO:teuthology.orchestra.run.smithi047.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-21T10:13:55.481 INFO:teuthology.orchestra.run.smithi047.stdout:Device: 5h/5d Inode: 650 Links: 1 2024-03-21T10:13:55.481 INFO:teuthology.orchestra.run.smithi047.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-21T10:13:55.481 INFO:teuthology.orchestra.run.smithi047.stdout:Context: system_u:object_r:device_t:s0 2024-03-21T10:13:55.481 INFO:teuthology.orchestra.run.smithi047.stdout:Access: 2024-03-21 10:12:38.924177624 +0000 2024-03-21T10:13:55.481 INFO:teuthology.orchestra.run.smithi047.stdout:Modify: 2024-03-21 10:07:01.661414952 +0000 2024-03-21T10:13:55.481 INFO:teuthology.orchestra.run.smithi047.stdout:Change: 2024-03-21 10:07:01.661414952 +0000 2024-03-21T10:13:55.481 INFO:teuthology.orchestra.run.smithi047.stdout: Birth: 2024-03-21 10:07:01.661414952 +0000 2024-03-21T10:13:55.481 DEBUG:teuthology.orchestra.run.smithi047:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-03-21T10:13:55.482 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:13:55 smithi100 ceph-mon[35193]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T10:13:55.482 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:13:55 smithi100 ceph-mon[35193]: from='client.24106 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi047=x;smithi100=y;smithi157=z", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:13:55.482 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:13:55 smithi100 ceph-mon[35193]: Saving service mgr spec with placement smithi047=x;smithi100=y;smithi157=z;count:3 2024-03-21T10:13:55.482 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:13:55 smithi100 ceph-mon[35193]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:13:55.482 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:13:55 smithi100 ceph-mon[35193]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:13:55.483 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:13:55 smithi100 ceph-mon[35193]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:13:55.483 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:13:55 smithi100 ceph-mon[35193]: Reconfiguring mon.c (monmap changed)... 2024-03-21T10:13:55.483 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:13:55 smithi100 ceph-mon[35193]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-21T10:13:55.483 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:13:55 smithi100 ceph-mon[35193]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-03-21T10:13:55.483 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:13:55 smithi100 ceph-mon[35193]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T10:13:55.483 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:13:55 smithi100 ceph-mon[35193]: Reconfiguring daemon mon.c on smithi157 2024-03-21T10:13:55.572 INFO:teuthology.orchestra.run.smithi047.stderr:1+0 records in 2024-03-21T10:13:55.572 INFO:teuthology.orchestra.run.smithi047.stderr:1+0 records out 2024-03-21T10:13:55.572 INFO:teuthology.orchestra.run.smithi047.stderr:512 bytes copied, 0.000274215 s, 1.9 MB/s 2024-03-21T10:13:55.576 DEBUG:teuthology.orchestra.run.smithi047:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-03-21T10:13:55.633 DEBUG:teuthology.orchestra.run.smithi047:> stat /dev/vg_nvme/lv_3 2024-03-21T10:13:55.686 INFO:teuthology.orchestra.run.smithi047.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-03-21T10:13:55.686 INFO:teuthology.orchestra.run.smithi047.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-21T10:13:55.686 INFO:teuthology.orchestra.run.smithi047.stdout:Device: 5h/5d Inode: 669 Links: 1 2024-03-21T10:13:55.686 INFO:teuthology.orchestra.run.smithi047.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-21T10:13:55.687 INFO:teuthology.orchestra.run.smithi047.stdout:Context: system_u:object_r:device_t:s0 2024-03-21T10:13:55.687 INFO:teuthology.orchestra.run.smithi047.stdout:Access: 2024-03-21 10:12:39.006175813 +0000 2024-03-21T10:13:55.687 INFO:teuthology.orchestra.run.smithi047.stdout:Modify: 2024-03-21 10:07:02.244401974 +0000 2024-03-21T10:13:55.687 INFO:teuthology.orchestra.run.smithi047.stdout:Change: 2024-03-21 10:07:02.244401974 +0000 2024-03-21T10:13:55.687 INFO:teuthology.orchestra.run.smithi047.stdout: Birth: 2024-03-21 10:07:02.244401974 +0000 2024-03-21T10:13:55.687 DEBUG:teuthology.orchestra.run.smithi047:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-03-21T10:13:55.707 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:13:55 smithi157 ceph-mon[35581]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T10:13:55.707 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:13:55 smithi157 ceph-mon[35581]: from='client.24106 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi047=x;smithi100=y;smithi157=z", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:13:55.708 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:13:55 smithi157 ceph-mon[35581]: Saving service mgr spec with placement smithi047=x;smithi100=y;smithi157=z;count:3 2024-03-21T10:13:55.708 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:13:55 smithi157 ceph-mon[35581]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:13:55.708 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:13:55 smithi157 ceph-mon[35581]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:13:55.708 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:13:55 smithi157 ceph-mon[35581]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:13:55.708 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:13:55 smithi157 ceph-mon[35581]: Reconfiguring mon.c (monmap changed)... 2024-03-21T10:13:55.708 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:13:55 smithi157 ceph-mon[35581]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-21T10:13:55.708 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:13:55 smithi157 ceph-mon[35581]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-03-21T10:13:55.708 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:13:55 smithi157 ceph-mon[35581]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T10:13:55.708 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:13:55 smithi157 ceph-mon[35581]: Reconfiguring daemon mon.c on smithi157 2024-03-21T10:13:55.713 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:13:55 smithi047 ceph-mon[31547]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T10:13:55.713 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:13:55 smithi047 ceph-mon[31547]: from='client.24106 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi047=x;smithi100=y;smithi157=z", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:13:55.714 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:13:55 smithi047 ceph-mon[31547]: Saving service mgr spec with placement smithi047=x;smithi100=y;smithi157=z;count:3 2024-03-21T10:13:55.714 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:13:55 smithi047 ceph-mon[31547]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:13:55.714 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:13:55 smithi047 ceph-mon[31547]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:13:55.714 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:13:55 smithi047 ceph-mon[31547]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:13:55.714 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:13:55 smithi047 ceph-mon[31547]: Reconfiguring mon.c (monmap changed)... 2024-03-21T10:13:55.714 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:13:55 smithi047 ceph-mon[31547]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-21T10:13:55.714 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:13:55 smithi047 ceph-mon[31547]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-03-21T10:13:55.714 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:13:55 smithi047 ceph-mon[31547]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T10:13:55.714 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:13:55 smithi047 ceph-mon[31547]: Reconfiguring daemon mon.c on smithi157 2024-03-21T10:13:55.782 INFO:teuthology.orchestra.run.smithi047.stderr:1+0 records in 2024-03-21T10:13:55.782 INFO:teuthology.orchestra.run.smithi047.stderr:1+0 records out 2024-03-21T10:13:55.782 INFO:teuthology.orchestra.run.smithi047.stderr:512 bytes copied, 0.00024146 s, 2.1 MB/s 2024-03-21T10:13:55.786 DEBUG:teuthology.orchestra.run.smithi047:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-03-21T10:13:55.890 DEBUG:teuthology.orchestra.run.smithi047:> stat /dev/vg_nvme/lv_4 2024-03-21T10:13:55.943 INFO:teuthology.orchestra.run.smithi047.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-03-21T10:13:55.943 INFO:teuthology.orchestra.run.smithi047.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-21T10:13:55.943 INFO:teuthology.orchestra.run.smithi047.stdout:Device: 5h/5d Inode: 689 Links: 1 2024-03-21T10:13:55.943 INFO:teuthology.orchestra.run.smithi047.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-21T10:13:55.943 INFO:teuthology.orchestra.run.smithi047.stdout:Context: system_u:object_r:device_t:s0 2024-03-21T10:13:55.944 INFO:teuthology.orchestra.run.smithi047.stdout:Access: 2024-03-21 10:12:39.070174400 +0000 2024-03-21T10:13:55.944 INFO:teuthology.orchestra.run.smithi047.stdout:Modify: 2024-03-21 10:07:02.825389041 +0000 2024-03-21T10:13:55.944 INFO:teuthology.orchestra.run.smithi047.stdout:Change: 2024-03-21 10:07:02.825389041 +0000 2024-03-21T10:13:55.944 INFO:teuthology.orchestra.run.smithi047.stdout: Birth: 2024-03-21 10:07:02.825389041 +0000 2024-03-21T10:13:55.944 DEBUG:teuthology.orchestra.run.smithi047:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-03-21T10:13:56.035 INFO:teuthology.orchestra.run.smithi047.stderr:1+0 records in 2024-03-21T10:13:56.036 INFO:teuthology.orchestra.run.smithi047.stderr:1+0 records out 2024-03-21T10:13:56.036 INFO:teuthology.orchestra.run.smithi047.stderr:512 bytes copied, 0.000245989 s, 2.1 MB/s 2024-03-21T10:13:56.040 DEBUG:teuthology.orchestra.run.smithi047:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-03-21T10:13:56.098 DEBUG:teuthology.orchestra.run.smithi100:> set -ex 2024-03-21T10:13:56.098 DEBUG:teuthology.orchestra.run.smithi100:> dd if=/scratch_devs of=/dev/stdout 2024-03-21T10:13:56.156 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-03-21T10:13:56.157 DEBUG:teuthology.orchestra.run.smithi100:> stat /dev/vg_nvme/lv_1 2024-03-21T10:13:56.208 INFO:teuthology.orchestra.run.smithi100.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-03-21T10:13:56.208 INFO:teuthology.orchestra.run.smithi100.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-21T10:13:56.208 INFO:teuthology.orchestra.run.smithi100.stdout:Device: 5h/5d Inode: 632 Links: 1 2024-03-21T10:13:56.208 INFO:teuthology.orchestra.run.smithi100.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-21T10:13:56.208 INFO:teuthology.orchestra.run.smithi100.stdout:Context: system_u:object_r:device_t:s0 2024-03-21T10:13:56.208 INFO:teuthology.orchestra.run.smithi100.stdout:Access: 2024-03-21 10:13:39.053718301 +0000 2024-03-21T10:13:56.208 INFO:teuthology.orchestra.run.smithi100.stdout:Modify: 2024-03-21 10:06:49.943953198 +0000 2024-03-21T10:13:56.208 INFO:teuthology.orchestra.run.smithi100.stdout:Change: 2024-03-21 10:06:49.943953198 +0000 2024-03-21T10:13:56.208 INFO:teuthology.orchestra.run.smithi100.stdout: Birth: 2024-03-21 10:06:49.943953198 +0000 2024-03-21T10:13:56.209 DEBUG:teuthology.orchestra.run.smithi100:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-03-21T10:13:56.295 INFO:teuthology.orchestra.run.smithi100.stderr:1+0 records in 2024-03-21T10:13:56.295 INFO:teuthology.orchestra.run.smithi100.stderr:1+0 records out 2024-03-21T10:13:56.295 INFO:teuthology.orchestra.run.smithi100.stderr:512 bytes copied, 0.000345765 s, 1.5 MB/s 2024-03-21T10:13:56.299 DEBUG:teuthology.orchestra.run.smithi100:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-03-21T10:13:56.353 DEBUG:teuthology.orchestra.run.smithi100:> stat /dev/vg_nvme/lv_2 2024-03-21T10:13:56.406 INFO:teuthology.orchestra.run.smithi100.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-03-21T10:13:56.406 INFO:teuthology.orchestra.run.smithi100.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-21T10:13:56.406 INFO:teuthology.orchestra.run.smithi100.stdout:Device: 5h/5d Inode: 652 Links: 1 2024-03-21T10:13:56.406 INFO:teuthology.orchestra.run.smithi100.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-21T10:13:56.406 INFO:teuthology.orchestra.run.smithi100.stdout:Context: system_u:object_r:device_t:s0 2024-03-21T10:13:56.406 INFO:teuthology.orchestra.run.smithi100.stdout:Access: 2024-03-21 10:13:39.054718275 +0000 2024-03-21T10:13:56.406 INFO:teuthology.orchestra.run.smithi100.stdout:Modify: 2024-03-21 10:06:50.506938667 +0000 2024-03-21T10:13:56.406 INFO:teuthology.orchestra.run.smithi100.stdout:Change: 2024-03-21 10:06:50.506938667 +0000 2024-03-21T10:13:56.406 INFO:teuthology.orchestra.run.smithi100.stdout: Birth: 2024-03-21 10:06:50.506938667 +0000 2024-03-21T10:13:56.407 DEBUG:teuthology.orchestra.run.smithi100:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-03-21T10:13:56.484 INFO:teuthology.orchestra.run.smithi100.stderr:1+0 records in 2024-03-21T10:13:56.484 INFO:teuthology.orchestra.run.smithi100.stderr:1+0 records out 2024-03-21T10:13:56.484 INFO:teuthology.orchestra.run.smithi100.stderr:512 bytes copied, 0.000503043 s, 1.0 MB/s 2024-03-21T10:13:56.486 DEBUG:teuthology.orchestra.run.smithi100:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-03-21T10:13:56.536 DEBUG:teuthology.orchestra.run.smithi100:> stat /dev/vg_nvme/lv_3 2024-03-21T10:13:56.581 INFO:teuthology.orchestra.run.smithi100.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-03-21T10:13:56.581 INFO:teuthology.orchestra.run.smithi100.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-21T10:13:56.581 INFO:teuthology.orchestra.run.smithi100.stdout:Device: 5h/5d Inode: 672 Links: 1 2024-03-21T10:13:56.581 INFO:teuthology.orchestra.run.smithi100.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-21T10:13:56.581 INFO:teuthology.orchestra.run.smithi100.stdout:Context: system_u:object_r:device_t:s0 2024-03-21T10:13:56.581 INFO:teuthology.orchestra.run.smithi100.stdout:Access: 2024-03-21 10:13:39.055718249 +0000 2024-03-21T10:13:56.581 INFO:teuthology.orchestra.run.smithi100.stdout:Modify: 2024-03-21 10:06:51.082923801 +0000 2024-03-21T10:13:56.581 INFO:teuthology.orchestra.run.smithi100.stdout:Change: 2024-03-21 10:06:51.082923801 +0000 2024-03-21T10:13:56.581 INFO:teuthology.orchestra.run.smithi100.stdout: Birth: 2024-03-21 10:06:51.082923801 +0000 2024-03-21T10:13:56.582 DEBUG:teuthology.orchestra.run.smithi100:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-03-21T10:13:56.713 INFO:teuthology.orchestra.run.smithi100.stderr:1+0 records in 2024-03-21T10:13:56.713 INFO:teuthology.orchestra.run.smithi100.stderr:1+0 records out 2024-03-21T10:13:56.713 INFO:teuthology.orchestra.run.smithi100.stderr:512 bytes copied, 0.000359153 s, 1.4 MB/s 2024-03-21T10:13:56.718 DEBUG:teuthology.orchestra.run.smithi100:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-03-21T10:13:56.772 DEBUG:teuthology.orchestra.run.smithi100:> stat /dev/vg_nvme/lv_4 2024-03-21T10:13:56.823 INFO:teuthology.orchestra.run.smithi100.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-03-21T10:13:56.823 INFO:teuthology.orchestra.run.smithi100.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-21T10:13:56.823 INFO:teuthology.orchestra.run.smithi100.stdout:Device: 5h/5d Inode: 690 Links: 1 2024-03-21T10:13:56.823 INFO:teuthology.orchestra.run.smithi100.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-21T10:13:56.823 INFO:teuthology.orchestra.run.smithi100.stdout:Context: system_u:object_r:device_t:s0 2024-03-21T10:13:56.823 INFO:teuthology.orchestra.run.smithi100.stdout:Access: 2024-03-21 10:13:39.055718249 +0000 2024-03-21T10:13:56.824 INFO:teuthology.orchestra.run.smithi100.stdout:Modify: 2024-03-21 10:06:51.637909477 +0000 2024-03-21T10:13:56.824 INFO:teuthology.orchestra.run.smithi100.stdout:Change: 2024-03-21 10:06:51.637909477 +0000 2024-03-21T10:13:56.824 INFO:teuthology.orchestra.run.smithi100.stdout: Birth: 2024-03-21 10:06:51.637909477 +0000 2024-03-21T10:13:56.824 DEBUG:teuthology.orchestra.run.smithi100:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-03-21T10:13:56.905 INFO:teuthology.orchestra.run.smithi100.stderr:1+0 records in 2024-03-21T10:13:56.905 INFO:teuthology.orchestra.run.smithi100.stderr:1+0 records out 2024-03-21T10:13:56.905 INFO:teuthology.orchestra.run.smithi100.stderr:512 bytes copied, 0.000303099 s, 1.7 MB/s 2024-03-21T10:13:56.908 DEBUG:teuthology.orchestra.run.smithi100:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-03-21T10:13:56.961 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2024-03-21T10:13:56.961 DEBUG:teuthology.orchestra.run.smithi157:> dd if=/scratch_devs of=/dev/stdout 2024-03-21T10:13:57.014 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-03-21T10:13:57.014 DEBUG:teuthology.orchestra.run.smithi157:> stat /dev/vg_nvme/lv_1 2024-03-21T10:13:57.060 INFO:teuthology.orchestra.run.smithi157.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-03-21T10:13:57.060 INFO:teuthology.orchestra.run.smithi157.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-21T10:13:57.060 INFO:teuthology.orchestra.run.smithi157.stdout:Device: 5h/5d Inode: 632 Links: 1 2024-03-21T10:13:57.061 INFO:teuthology.orchestra.run.smithi157.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-21T10:13:57.061 INFO:teuthology.orchestra.run.smithi157.stdout:Context: system_u:object_r:device_t:s0 2024-03-21T10:13:57.061 INFO:teuthology.orchestra.run.smithi157.stdout:Access: 2024-03-21 10:13:27.939329278 +0000 2024-03-21T10:13:57.061 INFO:teuthology.orchestra.run.smithi157.stdout:Modify: 2024-03-21 10:06:47.630308588 +0000 2024-03-21T10:13:57.061 INFO:teuthology.orchestra.run.smithi157.stdout:Change: 2024-03-21 10:06:47.630308588 +0000 2024-03-21T10:13:57.061 INFO:teuthology.orchestra.run.smithi157.stdout: Birth: 2024-03-21 10:06:47.630308588 +0000 2024-03-21T10:13:57.061 DEBUG:teuthology.orchestra.run.smithi157:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-03-21T10:13:57.188 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2024-03-21T10:13:57.189 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2024-03-21T10:13:57.189 INFO:teuthology.orchestra.run.smithi157.stderr:512 bytes copied, 0.000304001 s, 1.7 MB/s 2024-03-21T10:13:57.193 DEBUG:teuthology.orchestra.run.smithi157:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-03-21T10:13:57.238 DEBUG:teuthology.orchestra.run.smithi157:> stat /dev/vg_nvme/lv_2 2024-03-21T10:13:57.320 INFO:teuthology.orchestra.run.smithi157.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-03-21T10:13:57.320 INFO:teuthology.orchestra.run.smithi157.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-21T10:13:57.320 INFO:teuthology.orchestra.run.smithi157.stdout:Device: 5h/5d Inode: 653 Links: 1 2024-03-21T10:13:57.320 INFO:teuthology.orchestra.run.smithi157.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-21T10:13:57.320 INFO:teuthology.orchestra.run.smithi157.stdout:Context: system_u:object_r:device_t:s0 2024-03-21T10:13:57.320 INFO:teuthology.orchestra.run.smithi157.stdout:Access: 2024-03-21 10:13:27.939329278 +0000 2024-03-21T10:13:57.320 INFO:teuthology.orchestra.run.smithi157.stdout:Modify: 2024-03-21 10:06:48.190298623 +0000 2024-03-21T10:13:57.321 INFO:teuthology.orchestra.run.smithi157.stdout:Change: 2024-03-21 10:06:48.190298623 +0000 2024-03-21T10:13:57.321 INFO:teuthology.orchestra.run.smithi157.stdout: Birth: 2024-03-21 10:06:48.190298623 +0000 2024-03-21T10:13:57.321 DEBUG:teuthology.orchestra.run.smithi157:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-03-21T10:13:57.427 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2024-03-21T10:13:57.427 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2024-03-21T10:13:57.427 INFO:teuthology.orchestra.run.smithi157.stderr:512 bytes copied, 0.000179536 s, 2.9 MB/s 2024-03-21T10:13:57.430 DEBUG:teuthology.orchestra.run.smithi157:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-03-21T10:13:57.472 DEBUG:teuthology.orchestra.run.smithi157:> stat /dev/vg_nvme/lv_3 2024-03-21T10:13:57.562 INFO:teuthology.orchestra.run.smithi157.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-03-21T10:13:57.562 INFO:teuthology.orchestra.run.smithi157.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-21T10:13:57.563 INFO:teuthology.orchestra.run.smithi157.stdout:Device: 5h/5d Inode: 672 Links: 1 2024-03-21T10:13:57.563 INFO:teuthology.orchestra.run.smithi157.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-21T10:13:57.563 INFO:teuthology.orchestra.run.smithi157.stdout:Context: system_u:object_r:device_t:s0 2024-03-21T10:13:57.563 INFO:teuthology.orchestra.run.smithi157.stdout:Access: 2024-03-21 10:13:27.940329260 +0000 2024-03-21T10:13:57.563 INFO:teuthology.orchestra.run.smithi157.stdout:Modify: 2024-03-21 10:06:48.744288764 +0000 2024-03-21T10:13:57.563 INFO:teuthology.orchestra.run.smithi157.stdout:Change: 2024-03-21 10:06:48.744288764 +0000 2024-03-21T10:13:57.563 INFO:teuthology.orchestra.run.smithi157.stdout: Birth: 2024-03-21 10:06:48.744288764 +0000 2024-03-21T10:13:57.563 DEBUG:teuthology.orchestra.run.smithi157:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-03-21T10:13:57.629 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2024-03-21T10:13:57.629 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2024-03-21T10:13:57.630 INFO:teuthology.orchestra.run.smithi157.stderr:512 bytes copied, 0.000205212 s, 2.5 MB/s 2024-03-21T10:13:57.632 DEBUG:teuthology.orchestra.run.smithi157:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-03-21T10:13:57.678 DEBUG:teuthology.orchestra.run.smithi157:> stat /dev/vg_nvme/lv_4 2024-03-21T10:13:57.730 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:13:57 smithi100 ceph-mon[35193]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T10:13:57.730 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:13:57 smithi100 ceph-mon[35193]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:13:57.730 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:13:57 smithi100 ceph-mon[35193]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:13:57.730 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:13:57 smithi100 ceph-mon[35193]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T10:13:57.730 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:13:57 smithi100 ceph-mon[35193]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T10:13:57.730 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:13:57 smithi100 ceph-mon[35193]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T10:13:57.730 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:13:57 smithi100 ceph-mon[35193]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:13:57.730 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:13:57 smithi100 ceph-mon[35193]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-21T10:13:57.730 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:13:57 smithi100 ceph-mon[35193]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-03-21T10:13:57.730 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:13:57 smithi100 ceph-mon[35193]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-03-21T10:13:57.730 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:13:57 smithi100 ceph-mon[35193]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T10:13:57.730 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:13:57 smithi100 ceph-mon[35193]: Deploying daemon mgr.z on smithi157 2024-03-21T10:13:57.740 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:13:57 smithi157 ceph-mon[35581]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T10:13:57.740 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:13:57 smithi157 ceph-mon[35581]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:13:57.740 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:13:57 smithi157 ceph-mon[35581]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:13:57.740 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:13:57 smithi157 ceph-mon[35581]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T10:13:57.740 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:13:57 smithi157 ceph-mon[35581]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T10:13:57.741 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:13:57 smithi157 ceph-mon[35581]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T10:13:57.741 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:13:57 smithi157 ceph-mon[35581]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:13:57.741 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:13:57 smithi157 ceph-mon[35581]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-21T10:13:57.741 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:13:57 smithi157 ceph-mon[35581]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-03-21T10:13:57.741 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:13:57 smithi157 ceph-mon[35581]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-03-21T10:13:57.741 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:13:57 smithi157 ceph-mon[35581]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T10:13:57.741 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:13:57 smithi157 ceph-mon[35581]: Deploying daemon mgr.z on smithi157 2024-03-21T10:13:57.768 INFO:teuthology.orchestra.run.smithi157.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-03-21T10:13:57.768 INFO:teuthology.orchestra.run.smithi157.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-21T10:13:57.768 INFO:teuthology.orchestra.run.smithi157.stdout:Device: 5h/5d Inode: 691 Links: 1 2024-03-21T10:13:57.769 INFO:teuthology.orchestra.run.smithi157.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-21T10:13:57.769 INFO:teuthology.orchestra.run.smithi157.stdout:Context: system_u:object_r:device_t:s0 2024-03-21T10:13:57.769 INFO:teuthology.orchestra.run.smithi157.stdout:Access: 2024-03-21 10:13:27.941329242 +0000 2024-03-21T10:13:57.769 INFO:teuthology.orchestra.run.smithi157.stdout:Modify: 2024-03-21 10:06:49.309278710 +0000 2024-03-21T10:13:57.769 INFO:teuthology.orchestra.run.smithi157.stdout:Change: 2024-03-21 10:06:49.309278710 +0000 2024-03-21T10:13:57.769 INFO:teuthology.orchestra.run.smithi157.stdout: Birth: 2024-03-21 10:06:49.309278710 +0000 2024-03-21T10:13:57.769 DEBUG:teuthology.orchestra.run.smithi157:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-03-21T10:13:57.888 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2024-03-21T10:13:57.888 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2024-03-21T10:13:57.888 INFO:teuthology.orchestra.run.smithi157.stderr:512 bytes copied, 0.000277415 s, 1.8 MB/s 2024-03-21T10:13:57.891 DEBUG:teuthology.orchestra.run.smithi157:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-03-21T10:13:57.934 INFO:tasks.cephadm:Deploying osd.0 on smithi047 with /dev/vg_nvme/lv_4... 2024-03-21T10:13:57.934 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2edfee68ad7243bed422a049b6f5f58619300f11 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3056d51a-e76b-11ee-95cd-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-03-21T10:13:57.963 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:13:57 smithi047 ceph-mon[31547]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T10:13:57.964 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:13:57 smithi047 ceph-mon[31547]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:13:57.964 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:13:57 smithi047 ceph-mon[31547]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:13:57.964 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:13:57 smithi047 ceph-mon[31547]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T10:13:57.964 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:13:57 smithi047 ceph-mon[31547]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T10:13:57.964 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:13:57 smithi047 ceph-mon[31547]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T10:13:57.964 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:13:57 smithi047 ceph-mon[31547]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:13:57.964 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:13:57 smithi047 ceph-mon[31547]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-21T10:13:57.964 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:13:57 smithi047 ceph-mon[31547]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-03-21T10:13:57.964 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:13:57 smithi047 ceph-mon[31547]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-03-21T10:13:57.964 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:13:57 smithi047 ceph-mon[31547]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T10:13:57.964 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:13:57 smithi047 ceph-mon[31547]: Deploying daemon mgr.z on smithi157 2024-03-21T10:13:58.391 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/3056d51a-e76b-11ee-95cd-87774f69a715/mon.a/config 2024-03-21T10:13:59.592 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:13:59 smithi047 ceph-mon[31547]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T10:13:59.677 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:13:59 smithi157 ceph-mon[35581]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T10:13:59.729 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:13:59 smithi100 ceph-mon[35193]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T10:13:59.957 INFO:journalctl@ceph.mgr.z.smithi157.stdout:Mar 21 10:13:59 smithi157 podman[37303]: 2024-03-21 10:13:59.907165007 +0000 UTC m=+0.359088868 container start 7392304e277dddc346cec26193ccb280eea73c374221ec762f6f5d9fbe395f53 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-3056d51a-e76b-11ee-95cd-87774f69a715-mgr-z, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.build-date=20240319, org.label-schema.schema-version=1.0, ceph=True, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30) 2024-03-21T10:13:59.958 INFO:journalctl@ceph.mgr.z.smithi157.stdout:Mar 21 10:13:59 smithi157 ceph-mgr[37322]: -- 172.21.15.157:0/2058511309 <== mon.1 v2:172.21.15.157:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55a5d8acc5a0 con 0x55a5d8aa0c00 2024-03-21T10:14:00.240 INFO:journalctl@ceph.mgr.z.smithi157.stdout:Mar 21 10:13:59 smithi157 bash[37303]: 7392304e277dddc346cec26193ccb280eea73c374221ec762f6f5d9fbe395f53 2024-03-21T10:14:00.240 INFO:journalctl@ceph.mgr.z.smithi157.stdout:Mar 21 10:13:59 smithi157 systemd[1]: Started Ceph mgr.z for 3056d51a-e76b-11ee-95cd-87774f69a715. 2024-03-21T10:14:00.990 INFO:journalctl@ceph.mgr.z.smithi157.stdout:Mar 21 10:14:00 smithi157 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-mgr-z[37318]: 2024-03-21T10:14:00.692+0000 7f9cbd3f6200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-03-21T10:14:00.990 INFO:journalctl@ceph.mgr.z.smithi157.stdout:Mar 21 10:14:00 smithi157 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-mgr-z[37318]: 2024-03-21T10:14:00.811+0000 7f9cbd3f6200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-03-21T10:14:01.181 INFO:teuthology.orchestra.run.smithi047.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-03-21T10:14:01.181 INFO:teuthology.orchestra.run.smithi047.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-03-21T10:14:01.181 INFO:teuthology.orchestra.run.smithi047.stderr: stderr: 10+0 records in 2024-03-21T10:14:01.181 INFO:teuthology.orchestra.run.smithi047.stderr:10+0 records out 2024-03-21T10:14:01.181 INFO:teuthology.orchestra.run.smithi047.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0336533 s, 312 MB/s 2024-03-21T10:14:01.181 INFO:teuthology.orchestra.run.smithi047.stderr:--> Zapping successful for: 2024-03-21T10:14:01.442 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:14:01 smithi100 ceph-mon[35193]: pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T10:14:01.442 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:14:01 smithi100 ceph-mon[35193]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:14:01.442 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:14:01 smithi100 ceph-mon[35193]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:14:01.442 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:14:01 smithi100 ceph-mon[35193]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:14:01.442 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:14:01 smithi100 ceph-mon[35193]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-21T10:14:01.442 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:14:01 smithi100 ceph-mon[35193]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-03-21T10:14:01.442 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:14:01 smithi100 ceph-mon[35193]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-03-21T10:14:01.442 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:14:01 smithi100 ceph-mon[35193]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T10:14:01.443 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:14:01 smithi100 ceph-mon[35193]: Deploying daemon mgr.y on smithi100 2024-03-21T10:14:01.464 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:14:01 smithi047 ceph-mon[31547]: pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T10:14:01.464 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:14:01 smithi047 ceph-mon[31547]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:14:01.464 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:14:01 smithi047 ceph-mon[31547]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:14:01.464 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:14:01 smithi047 ceph-mon[31547]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:14:01.464 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:14:01 smithi047 ceph-mon[31547]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-21T10:14:01.464 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:14:01 smithi047 ceph-mon[31547]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-03-21T10:14:01.464 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:14:01 smithi047 ceph-mon[31547]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-03-21T10:14:01.465 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:14:01 smithi047 ceph-mon[31547]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T10:14:01.465 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:14:01 smithi047 ceph-mon[31547]: Deploying daemon mgr.y on smithi100 2024-03-21T10:14:01.489 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:14:01 smithi157 ceph-mon[35581]: pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T10:14:01.490 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:14:01 smithi157 ceph-mon[35581]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:14:01.490 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:14:01 smithi157 ceph-mon[35581]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:14:01.490 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:14:01 smithi157 ceph-mon[35581]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:14:01.490 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:14:01 smithi157 ceph-mon[35581]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-21T10:14:01.490 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:14:01 smithi157 ceph-mon[35581]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-03-21T10:14:01.490 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:14:01 smithi157 ceph-mon[35581]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-03-21T10:14:01.490 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:14:01 smithi157 ceph-mon[35581]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T10:14:01.490 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:14:01 smithi157 ceph-mon[35581]: Deploying daemon mgr.y on smithi100 2024-03-21T10:14:01.899 INFO:journalctl@ceph.mgr.z.smithi157.stdout:Mar 21 10:14:01 smithi157 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-mgr-z[37318]: 2024-03-21T10:14:01.601+0000 7f9cbd3f6200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-03-21T10:14:01.899 INFO:journalctl@ceph.mgr.z.smithi157.stdout:Mar 21 10:14:01 smithi157 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-mgr-z[37318]: 2024-03-21T10:14:01.720+0000 7f9cbd3f6200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-03-21T10:14:01.899 INFO:journalctl@ceph.mgr.z.smithi157.stdout:Mar 21 10:14:01 smithi157 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-mgr-z[37318]: 2024-03-21T10:14:01.896+0000 7f9cbd3f6200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-03-21T10:14:01.957 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2edfee68ad7243bed422a049b6f5f58619300f11 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3056d51a-e76b-11ee-95cd-87774f69a715 -- ceph orch daemon add osd smithi047:vg_nvme/lv_4 2024-03-21T10:14:02.240 INFO:journalctl@ceph.mgr.z.smithi157.stdout:Mar 21 10:14:02 smithi157 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-mgr-z[37318]: 2024-03-21T10:14:02.019+0000 7f9cbd3f6200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-03-21T10:14:02.240 INFO:journalctl@ceph.mgr.z.smithi157.stdout:Mar 21 10:14:02 smithi157 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-mgr-z[37318]: 2024-03-21T10:14:02.129+0000 7f9cbd3f6200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-03-21T10:14:02.476 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/3056d51a-e76b-11ee-95cd-87774f69a715/mon.a/config 2024-03-21T10:14:02.740 INFO:journalctl@ceph.mgr.z.smithi157.stdout:Mar 21 10:14:02 smithi157 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-mgr-z[37318]: 2024-03-21T10:14:02.459+0000 7f9cbd3f6200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-03-21T10:14:02.740 INFO:journalctl@ceph.mgr.z.smithi157.stdout:Mar 21 10:14:02 smithi157 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-mgr-z[37318]: 2024-03-21T10:14:02.598+0000 7f9cbd3f6200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-03-21T10:14:02.740 INFO:journalctl@ceph.mgr.z.smithi157.stdout:Mar 21 10:14:02 smithi157 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-mgr-z[37318]: 2024-03-21T10:14:02.707+0000 7f9cbd3f6200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-03-21T10:14:03.135 INFO:journalctl@ceph.mgr.z.smithi157.stdout:Mar 21 10:14:02 smithi157 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-mgr-z[37318]: 2024-03-21T10:14:02.829+0000 7f9cbd3f6200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-03-21T10:14:03.135 INFO:journalctl@ceph.mgr.z.smithi157.stdout:Mar 21 10:14:03 smithi157 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-mgr-z[37318]: 2024-03-21T10:14:03.046+0000 7f9cbd3f6200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-03-21T10:14:03.213 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:14:03 smithi047 ceph-mon[31547]: pgmap v42: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T10:14:03.229 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:14:03 smithi100 ceph-mon[35193]: pgmap v42: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T10:14:03.489 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:14:03 smithi157 ceph-mon[35581]: pgmap v42: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T10:14:03.557 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Mar 21 10:14:03 smithi100 podman[36563]: 2024-03-21 10:14:03.318858482 +0000 UTC m=+0.143419289 container create f904fc9002ac42b49e0c43d8dea04a7a8bede22cdde5ec4ded2c9c1e81800c5d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-3056d51a-e76b-11ee-95cd-87774f69a715-mgr-y, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, io.buildah.version=1.33.5, org.label-schema.build-date=20240319, GIT_CLEAN=True, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS) 2024-03-21T10:14:03.557 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Mar 21 10:14:03 smithi100 podman[36563]: 2024-03-21 10:14:03.555596734 +0000 UTC m=+0.380157547 container init f904fc9002ac42b49e0c43d8dea04a7a8bede22cdde5ec4ded2c9c1e81800c5d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-3056d51a-e76b-11ee-95cd-87774f69a715-mgr-y, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.build-date=20240319, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, ceph=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD) 2024-03-21T10:14:03.979 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Mar 21 10:14:03 smithi100 podman[36563]: 2024-03-21 10:14:03.566376409 +0000 UTC m=+0.390937186 container start f904fc9002ac42b49e0c43d8dea04a7a8bede22cdde5ec4ded2c9c1e81800c5d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-3056d51a-e76b-11ee-95cd-87774f69a715-mgr-y, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.build-date=20240319, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.schema-version=1.0, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, CEPH_POINT_RELEASE=) 2024-03-21T10:14:03.980 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Mar 21 10:14:03 smithi100 ceph-mgr[36582]: -- 172.21.15.100:0/1020191936 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x557a947024e0 con 0x557a946d8800 2024-03-21T10:14:03.980 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Mar 21 10:14:03 smithi100 ceph-mgr[36582]: -- 172.21.15.100:0/1020191936 <== mon.2 v2:172.21.15.100:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x557a947045a0 con 0x557a946d8800 2024-03-21T10:14:03.980 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Mar 21 10:14:03 smithi100 bash[36563]: f904fc9002ac42b49e0c43d8dea04a7a8bede22cdde5ec4ded2c9c1e81800c5d 2024-03-21T10:14:03.980 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Mar 21 10:14:03 smithi100 systemd[1]: Started Ceph mgr.y for 3056d51a-e76b-11ee-95cd-87774f69a715. 2024-03-21T10:14:04.468 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:14:04.467+0000 7f5ce7600700 1 -- 172.21.15.47:0/1380851861 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5ce0103d70 msgr2=0x7f5ce010a620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:14:04.468 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:14:04.467+0000 7f5ce7600700 1 --2- 172.21.15.47:0/1380851861 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5ce0103d70 0x7f5ce010a620 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f5cd800b0d0 tx=0x7f5cd800b3e0 comp rx=0 tx=0).stop 2024-03-21T10:14:04.469 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:14:04.467+0000 7f5ce7600700 1 -- 172.21.15.47:0/1380851861 shutdown_connections 2024-03-21T10:14:04.469 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:14:04.467+0000 7f5ce7600700 1 --2- 172.21.15.47:0/1380851861 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5ce0103d70 0x7f5ce010a620 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:14:04.469 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:14:04.467+0000 7f5ce7600700 1 --2- 172.21.15.47:0/1380851861 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5ce01021d0 0x7f5ce01025f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:14:04.469 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:14:04.467+0000 7f5ce7600700 1 --2- 172.21.15.47:0/1380851861 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5ce01033d0 0x7f5ce0103830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:14:04.469 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:14:04.467+0000 7f5ce7600700 1 -- 172.21.15.47:0/1380851861 >> 172.21.15.47:0/1380851861 conn(0x7f5ce00fd730 msgr2=0x7f5ce00ffbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:14:04.469 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:14:04.467+0000 7f5ce7600700 1 -- 172.21.15.47:0/1380851861 shutdown_connections 2024-03-21T10:14:04.469 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:14:04.467+0000 7f5ce7600700 1 -- 172.21.15.47:0/1380851861 wait complete. 2024-03-21T10:14:04.469 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:14:04.469+0000 7f5ce7600700 1 Processor -- start 2024-03-21T10:14:04.470 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:14:04.469+0000 7f5ce7600700 1 -- start start 2024-03-21T10:14:04.470 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:14:04.469+0000 7f5ce7600700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5ce01021d0 0x7f5ce019a470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:14:04.470 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:14:04.469+0000 7f5ce7600700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5ce01033d0 0x7f5ce019a9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:14:04.471 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:14:04.469+0000 7f5ce7600700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5ce0103d70 0x7f5ce01a1a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:14:04.471 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:14:04.469+0000 7f5ce7600700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ce010dcb0 con 0x7f5ce01021d0 2024-03-21T10:14:04.471 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:14:04.469+0000 7f5ce7600700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ce010db30 con 0x7f5ce0103d70 2024-03-21T10:14:04.471 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:14:04.469+0000 7f5ce7600700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ce010de30 con 0x7f5ce01033d0 2024-03-21T10:14:04.471 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:14:04.470+0000 7f5ce4c00700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5ce01021d0 0x7f5ce019a470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:14:04.471 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:14:04.470+0000 7f5ce4c00700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5ce01021d0 0x7f5ce019a470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:58354/0 (socket says 172.21.15.47:58354) 2024-03-21T10:14:04.471 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:14:04.470+0000 7f5ce4c00700 1 -- 172.21.15.47:0/1768035311 learned_addr learned my addr 172.21.15.47:0/1768035311 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:14:04.472 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:14:04.470+0000 7f5cd7e00700 1 --2- 172.21.15.47:0/1768035311 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5ce01033d0 0x7f5ce019a9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:14:04.472 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:14:04.470+0000 7f5ce5600700 1 --2- 172.21.15.47:0/1768035311 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5ce0103d70 0x7f5ce01a1a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:14:04.472 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:14:04.470+0000 7f5ce4c00700 1 -- 172.21.15.47:0/1768035311 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5ce01033d0 msgr2=0x7f5ce019a9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:14:04.472 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:14:04.470+0000 7f5ce4c00700 1 --2- 172.21.15.47:0/1768035311 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5ce01033d0 0x7f5ce019a9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:14:04.472 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:14:04.470+0000 7f5ce4c00700 1 -- 172.21.15.47:0/1768035311 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5ce0103d70 msgr2=0x7f5ce01a1a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:14:04.472 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:14:04.470+0000 7f5ce4c00700 1 --2- 172.21.15.47:0/1768035311 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5ce0103d70 0x7f5ce01a1a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:14:04.472 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:14:04.470+0000 7f5ce4c00700 1 -- 172.21.15.47:0/1768035311 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ce01a20d0 con 0x7f5ce01021d0 2024-03-21T10:14:04.473 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:14:04.471+0000 7f5ce4c00700 1 --2- 172.21.15.47:0/1768035311 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5ce01021d0 0x7f5ce019a470 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f5cd000e9a0 tx=0x7f5cd000ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:14:04.473 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:14:04.471+0000 7f5cde000700 1 -- 172.21.15.47:0/1768035311 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5cd000cbd0 con 0x7f5ce01021d0 2024-03-21T10:14:04.473 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:14:04.471+0000 7f5cde000700 1 -- 172.21.15.47:0/1768035311 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f5cd00044d0 con 0x7f5ce01021d0 2024-03-21T10:14:04.473 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:14:04.471+0000 7f5ce7600700 1 -- 172.21.15.47:0/1768035311 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5ce01a23c0 con 0x7f5ce01021d0 2024-03-21T10:14:04.473 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:14:04.471+0000 7f5cde000700 1 -- 172.21.15.47:0/1768035311 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5cd0003e30 con 0x7f5ce01021d0 2024-03-21T10:14:04.473 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:14:04.471+0000 7f5ce7600700 1 -- 172.21.15.47:0/1768035311 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5ce01a2950 con 0x7f5ce01021d0 2024-03-21T10:14:04.474 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:14:04.472+0000 7f5ce7600700 1 -- 172.21.15.47:0/1768035311 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5ce0068e50 con 0x7f5ce01021d0 2024-03-21T10:14:04.474 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:14:04.472+0000 7f5cde000700 1 -- 172.21.15.47:0/1768035311 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 13) v1 ==== 46350+0+0 (secure 0 0 0) 0x7f5cd0004000 con 0x7f5ce01021d0 2024-03-21T10:14:04.474 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:14:04.473+0000 7f5cde000700 1 --2- 172.21.15.47:0/1768035311 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f5cc4039460 0x7f5cc403b920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:14:04.474 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:14:04.473+0000 7f5cde000700 1 -- 172.21.15.47:0/1768035311 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f5cd004c7e0 con 0x7f5ce01021d0 2024-03-21T10:14:04.474 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:14:04.473+0000 7f5cd7e00700 1 --2- 172.21.15.47:0/1768035311 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f5cc4039460 0x7f5cc403b920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:14:04.475 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:14:04.474+0000 7f5cd7e00700 1 --2- 172.21.15.47:0/1768035311 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f5cc4039460 0x7f5cc403b920 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f5cc800ac50 tx=0x7f5cc800a380 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:14:04.479 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:14:04.479+0000 7f5cde000700 1 -- 172.21.15.47:0/1768035311 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f5cd0016330 con 0x7f5ce01021d0 2024-03-21T10:14:04.721 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:14:04.719+0000 7f5ce7600700 1 -- 172.21.15.47:0/1768035311 --> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi047:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) v1 -- 0x7f5ce0075ae0 con 0x7f5cc4039460 2024-03-21T10:14:04.729 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Mar 21 10:14:04 smithi100 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-mgr-y[36578]: 2024-03-21T10:14:04.336+0000 7f22da443200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-03-21T10:14:04.729 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Mar 21 10:14:04 smithi100 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-mgr-y[36578]: 2024-03-21T10:14:04.459+0000 7f22da443200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-03-21T10:14:04.739 INFO:journalctl@ceph.mgr.z.smithi157.stdout:Mar 21 10:14:04 smithi157 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-mgr-z[37318]: 2024-03-21T10:14:04.349+0000 7f9cbd3f6200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-03-21T10:14:04.740 INFO:journalctl@ceph.mgr.z.smithi157.stdout:Mar 21 10:14:04 smithi157 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-mgr-z[37318]: 2024-03-21T10:14:04.525+0000 7f9cbd3f6200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-03-21T10:14:05.086 INFO:journalctl@ceph.mgr.z.smithi157.stdout:Mar 21 10:14:04 smithi157 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-mgr-z[37318]: 2024-03-21T10:14:04.960+0000 7f9cbd3f6200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-03-21T10:14:05.087 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:14:04 smithi157 ceph-mon[35581]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:14:05.087 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:14:04 smithi157 ceph-mon[35581]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:14:05.087 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:14:04 smithi157 ceph-mon[35581]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:14:05.087 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:14:04 smithi157 ceph-mon[35581]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:14:05.087 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:14:04 smithi157 ceph-mon[35581]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T10:14:05.087 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:14:04 smithi157 ceph-mon[35581]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-21T10:14:05.087 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:14:04 smithi157 ceph-mon[35581]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-21T10:14:05.087 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:14:04 smithi157 ceph-mon[35581]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T10:14:05.214 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:14:04 smithi047 ceph-mon[31547]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:14:05.214 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:14:04 smithi047 ceph-mon[31547]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:14:05.214 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:14:04 smithi047 ceph-mon[31547]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:14:05.214 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:14:04 smithi047 ceph-mon[31547]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:14:05.214 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:14:04 smithi047 ceph-mon[31547]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T10:14:05.215 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:14:04 smithi047 ceph-mon[31547]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-21T10:14:05.215 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:14:04 smithi047 ceph-mon[31547]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-21T10:14:05.215 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:14:04 smithi047 ceph-mon[31547]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T10:14:05.229 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:14:04 smithi100 ceph-mon[35193]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:14:05.229 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:14:04 smithi100 ceph-mon[35193]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:14:05.229 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:14:04 smithi100 ceph-mon[35193]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:14:05.229 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:14:04 smithi100 ceph-mon[35193]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:14:05.229 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:14:04 smithi100 ceph-mon[35193]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T10:14:05.230 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:14:04 smithi100 ceph-mon[35193]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-21T10:14:05.230 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:14:04 smithi100 ceph-mon[35193]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-21T10:14:05.230 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:14:04 smithi100 ceph-mon[35193]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T10:14:05.363 INFO:journalctl@ceph.mgr.z.smithi157.stdout:Mar 21 10:14:05 smithi157 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-mgr-z[37318]: 2024-03-21T10:14:05.084+0000 7f9cbd3f6200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-03-21T10:14:05.595 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Mar 21 10:14:05 smithi100 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-mgr-y[36578]: 2024-03-21T10:14:05.282+0000 7f22da443200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-03-21T10:14:05.595 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Mar 21 10:14:05 smithi100 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-mgr-y[36578]: 2024-03-21T10:14:05.405+0000 7f22da443200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-03-21T10:14:05.595 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Mar 21 10:14:05 smithi100 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-mgr-y[36578]: 2024-03-21T10:14:05.593+0000 7f22da443200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-03-21T10:14:05.618 INFO:journalctl@ceph.mgr.z.smithi157.stdout:Mar 21 10:14:05 smithi157 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-mgr-z[37318]: 2024-03-21T10:14:05.361+0000 7f9cbd3f6200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-03-21T10:14:05.619 INFO:journalctl@ceph.mgr.z.smithi157.stdout:Mar 21 10:14:05 smithi157 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-mgr-z[37318]: 2024-03-21T10:14:05.616+0000 7f9cbd3f6200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-03-21T10:14:05.963 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:14:05 smithi047 ceph-mon[31547]: pgmap v43: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T10:14:05.964 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:14:05 smithi047 ceph-mon[31547]: from='client.14220 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi047:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:14:05.979 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:14:05 smithi100 ceph-mon[35193]: pgmap v43: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T10:14:05.979 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:14:05 smithi100 ceph-mon[35193]: from='client.14220 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi047:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:14:05.979 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Mar 21 10:14:05 smithi100 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-mgr-y[36578]: 2024-03-21T10:14:05.719+0000 7f22da443200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-03-21T10:14:05.980 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Mar 21 10:14:05 smithi100 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-mgr-y[36578]: 2024-03-21T10:14:05.835+0000 7f22da443200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-03-21T10:14:05.989 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:14:05 smithi157 ceph-mon[35581]: pgmap v43: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T10:14:05.989 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:14:05 smithi157 ceph-mon[35581]: from='client.14220 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi047:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:14:06.320 INFO:journalctl@ceph.mgr.z.smithi157.stdout:Mar 21 10:14:06 smithi157 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-mgr-z[37318]: 2024-03-21T10:14:05.999+0000 7f9cbd3f6200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-03-21T10:14:06.320 INFO:journalctl@ceph.mgr.z.smithi157.stdout:Mar 21 10:14:06 smithi157 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-mgr-z[37318]: 2024-03-21T10:14:06.318+0000 7f9cbd3f6200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-03-21T10:14:06.445 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Mar 21 10:14:06 smithi100 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-mgr-y[36578]: 2024-03-21T10:14:06.180+0000 7f22da443200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-03-21T10:14:06.445 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Mar 21 10:14:06 smithi100 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-mgr-y[36578]: 2024-03-21T10:14:06.327+0000 7f22da443200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-03-21T10:14:06.729 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Mar 21 10:14:06 smithi100 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-mgr-y[36578]: 2024-03-21T10:14:06.443+0000 7f22da443200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-03-21T10:14:06.729 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Mar 21 10:14:06 smithi100 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-mgr-y[36578]: 2024-03-21T10:14:06.569+0000 7f22da443200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-03-21T10:14:06.931 INFO:journalctl@ceph.mgr.z.smithi157.stdout:Mar 21 10:14:06 smithi157 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-mgr-z[37318]: 2024-03-21T10:14:06.620+0000 7f9cbd3f6200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-03-21T10:14:07.173 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:14:06 smithi047 ceph-mon[31547]: pgmap v44: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T10:14:07.173 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:14:06 smithi047 ceph-mon[31547]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:14:07.174 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:14:06 smithi047 ceph-mon[31547]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:14:07.174 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:14:06 smithi047 ceph-mon[31547]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:14:07.174 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:14:06 smithi047 ceph-mon[31547]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:14:07.174 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:14:06 smithi047 ceph-mon[31547]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:14:07.229 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:14:06 smithi100 ceph-mon[35193]: pgmap v44: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T10:14:07.229 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:14:06 smithi100 ceph-mon[35193]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:14:07.229 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:14:06 smithi100 ceph-mon[35193]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:14:07.229 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:14:06 smithi100 ceph-mon[35193]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:14:07.229 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:14:06 smithi100 ceph-mon[35193]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:14:07.229 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:14:06 smithi100 ceph-mon[35193]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:14:07.230 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Mar 21 10:14:06 smithi100 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-mgr-y[36578]: 2024-03-21T10:14:06.796+0000 7f22da443200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-03-21T10:14:07.240 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:14:06 smithi157 ceph-mon[35581]: pgmap v44: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T10:14:07.240 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:14:06 smithi157 ceph-mon[35581]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:14:07.240 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:14:06 smithi157 ceph-mon[35581]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:14:07.240 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:14:06 smithi157 ceph-mon[35581]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:14:07.240 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:14:06 smithi157 ceph-mon[35581]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:14:07.240 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:14:06 smithi157 ceph-mon[35581]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:14:07.934 INFO:journalctl@ceph.mgr.z.smithi157.stdout:Mar 21 10:14:07 smithi157 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-mgr-z[37318]: 2024-03-21T10:14:07.640+0000 7f9cbd3f6200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-03-21T10:14:08.213 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:14:07 smithi047 ceph-mon[31547]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T10:14:08.214 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:14:07 smithi047 ceph-mon[31547]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T10:14:08.214 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:14:07 smithi047 ceph-mon[31547]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:14:08.214 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:14:07 smithi047 ceph-mon[31547]: Reconfiguring mgr.x (unknown last config time)... 2024-03-21T10:14:08.214 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:14:07 smithi047 ceph-mon[31547]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-21T10:14:08.214 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:14:07 smithi047 ceph-mon[31547]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-03-21T10:14:08.214 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:14:07 smithi047 ceph-mon[31547]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T10:14:08.214 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:14:07 smithi047 ceph-mon[31547]: Reconfiguring daemon mgr.x on smithi047 2024-03-21T10:14:08.229 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Mar 21 10:14:08 smithi100 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-mgr-y[36578]: 2024-03-21T10:14:08.151+0000 7f22da443200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-03-21T10:14:08.229 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:14:07 smithi100 ceph-mon[35193]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T10:14:08.229 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:14:07 smithi100 ceph-mon[35193]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T10:14:08.229 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:14:07 smithi100 ceph-mon[35193]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:14:08.230 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:14:07 smithi100 ceph-mon[35193]: Reconfiguring mgr.x (unknown last config time)... 2024-03-21T10:14:08.230 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:14:07 smithi100 ceph-mon[35193]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-21T10:14:08.230 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:14:07 smithi100 ceph-mon[35193]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-03-21T10:14:08.230 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:14:07 smithi100 ceph-mon[35193]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T10:14:08.230 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:14:07 smithi100 ceph-mon[35193]: Reconfiguring daemon mgr.x on smithi047 2024-03-21T10:14:08.239 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:14:07 smithi157 ceph-mon[35581]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T10:14:08.239 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:14:07 smithi157 ceph-mon[35581]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T10:14:08.240 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:14:07 smithi157 ceph-mon[35581]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:14:08.240 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:14:07 smithi157 ceph-mon[35581]: Reconfiguring mgr.x (unknown last config time)... 2024-03-21T10:14:08.240 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:14:07 smithi157 ceph-mon[35581]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-21T10:14:08.240 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:14:07 smithi157 ceph-mon[35581]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-03-21T10:14:08.240 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:14:07 smithi157 ceph-mon[35581]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T10:14:08.240 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:14:07 smithi157 ceph-mon[35581]: Reconfiguring daemon mgr.x on smithi047 2024-03-21T10:14:08.729 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Mar 21 10:14:08 smithi100 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-mgr-y[36578]: 2024-03-21T10:14:08.321+0000 7f22da443200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-03-21T10:14:08.739 INFO:journalctl@ceph.mgr.z.smithi157.stdout:Mar 21 10:14:08 smithi157 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-mgr-z[37318]: 2024-03-21T10:14:08.335+0000 7f9cbd3f6200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-03-21T10:14:08.740 INFO:journalctl@ceph.mgr.z.smithi157.stdout:Mar 21 10:14:08 smithi157 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-mgr-z[37318]: 2024-03-21T10:14:08.464+0000 7f9cbd3f6200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-03-21T10:14:09.072 INFO:journalctl@ceph.mgr.z.smithi157.stdout:Mar 21 10:14:08 smithi157 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-mgr-z[37318]: 2024-03-21T10:14:08.745+0000 7f9cbd3f6200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-03-21T10:14:09.105 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Mar 21 10:14:08 smithi100 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-mgr-y[36578]: 2024-03-21T10:14:08.733+0000 7f22da443200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-03-21T10:14:09.106 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Mar 21 10:14:08 smithi100 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-mgr-y[36578]: 2024-03-21T10:14:08.849+0000 7f22da443200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-03-21T10:14:09.363 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:14:09 smithi157 ceph-mon[35581]: pgmap v45: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T10:14:09.363 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:14:09 smithi157 ceph-mon[35581]: from='client.? 172.21.15.47:0/2014670212' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b0c7a102-8d89-428b-b793-cbbe67151f50"}]: dispatch 2024-03-21T10:14:09.363 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:14:09 smithi157 ceph-mon[35581]: from='client.? 172.21.15.47:0/2014670212' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "b0c7a102-8d89-428b-b793-cbbe67151f50"}]': finished 2024-03-21T10:14:09.364 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:14:09 smithi157 ceph-mon[35581]: osdmap e5: 1 total, 0 up, 1 in 2024-03-21T10:14:09.364 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:14:09 smithi157 ceph-mon[35581]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-21T10:14:09.364 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:14:09 smithi157 ceph-mon[35581]: Standby manager daemon z started 2024-03-21T10:14:09.364 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:14:09 smithi157 ceph-mon[35581]: from='mgr.? 172.21.15.157:0/2889294658' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-03-21T10:14:09.364 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:14:09 smithi157 ceph-mon[35581]: from='mgr.? 172.21.15.157:0/2889294658' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-03-21T10:14:09.364 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:14:09 smithi157 ceph-mon[35581]: from='mgr.? 172.21.15.157:0/2889294658' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-03-21T10:14:09.364 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:14:09 smithi157 ceph-mon[35581]: from='mgr.? 172.21.15.157:0/2889294658' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-03-21T10:14:09.464 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:14:09 smithi047 ceph-mon[31547]: pgmap v45: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T10:14:09.464 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:14:09 smithi047 ceph-mon[31547]: from='client.? 172.21.15.47:0/2014670212' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b0c7a102-8d89-428b-b793-cbbe67151f50"}]: dispatch 2024-03-21T10:14:09.464 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:14:09 smithi047 ceph-mon[31547]: from='client.? 172.21.15.47:0/2014670212' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "b0c7a102-8d89-428b-b793-cbbe67151f50"}]': finished 2024-03-21T10:14:09.464 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:14:09 smithi047 ceph-mon[31547]: osdmap e5: 1 total, 0 up, 1 in 2024-03-21T10:14:09.464 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:14:09 smithi047 ceph-mon[31547]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-21T10:14:09.464 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:14:09 smithi047 ceph-mon[31547]: Standby manager daemon z started 2024-03-21T10:14:09.464 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:14:09 smithi047 ceph-mon[31547]: from='mgr.? 172.21.15.157:0/2889294658' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-03-21T10:14:09.464 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:14:09 smithi047 ceph-mon[31547]: from='mgr.? 172.21.15.157:0/2889294658' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-03-21T10:14:09.464 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:14:09 smithi047 ceph-mon[31547]: from='mgr.? 172.21.15.157:0/2889294658' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-03-21T10:14:09.464 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:14:09 smithi047 ceph-mon[31547]: from='mgr.? 172.21.15.157:0/2889294658' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-03-21T10:14:09.479 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Mar 21 10:14:09 smithi100 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-mgr-y[36578]: 2024-03-21T10:14:09.103+0000 7f22da443200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-03-21T10:14:09.479 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Mar 21 10:14:09 smithi100 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-mgr-y[36578]: 2024-03-21T10:14:09.345+0000 7f22da443200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-03-21T10:14:09.479 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:14:09 smithi100 ceph-mon[35193]: pgmap v45: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T10:14:09.480 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:14:09 smithi100 ceph-mon[35193]: from='client.? 172.21.15.47:0/2014670212' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b0c7a102-8d89-428b-b793-cbbe67151f50"}]: dispatch 2024-03-21T10:14:09.480 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:14:09 smithi100 ceph-mon[35193]: from='client.? 172.21.15.47:0/2014670212' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "b0c7a102-8d89-428b-b793-cbbe67151f50"}]': finished 2024-03-21T10:14:09.480 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:14:09 smithi100 ceph-mon[35193]: osdmap e5: 1 total, 0 up, 1 in 2024-03-21T10:14:09.480 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:14:09 smithi100 ceph-mon[35193]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-21T10:14:09.480 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:14:09 smithi100 ceph-mon[35193]: Standby manager daemon z started 2024-03-21T10:14:09.480 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:14:09 smithi100 ceph-mon[35193]: from='mgr.? 172.21.15.157:0/2889294658' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-03-21T10:14:09.480 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:14:09 smithi100 ceph-mon[35193]: from='mgr.? 172.21.15.157:0/2889294658' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-03-21T10:14:09.480 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:14:09 smithi100 ceph-mon[35193]: from='mgr.? 172.21.15.157:0/2889294658' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-03-21T10:14:09.480 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:14:09 smithi100 ceph-mon[35193]: from='mgr.? 172.21.15.157:0/2889294658' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-03-21T10:14:09.728 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:14:09.727+0000 7f5cde000700 1 -- 172.21.15.47:0/1768035311 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mgrmap(e 14) v1 ==== 92074+0+0 (secure 0 0 0) 0x7f5cd0021cd0 con 0x7f5ce01021d0 2024-03-21T10:14:09.979 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Mar 21 10:14:09 smithi100 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-mgr-y[36578]: 2024-03-21T10:14:09.714+0000 7f22da443200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-03-21T10:14:10.334 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Mar 21 10:14:10 smithi100 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-mgr-y[36578]: 2024-03-21T10:14:10.028+0000 7f22da443200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-03-21T10:14:10.334 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:14:10 smithi100 ceph-mon[35193]: from='client.? 172.21.15.47:0/92595736' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-21T10:14:10.334 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:14:10 smithi100 ceph-mon[35193]: mgrmap e14: x(active, since 103s), standbys: z 2024-03-21T10:14:10.334 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:14:10 smithi100 ceph-mon[35193]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-03-21T10:14:10.464 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:14:10 smithi047 ceph-mon[31547]: from='client.? 172.21.15.47:0/92595736' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-21T10:14:10.464 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:14:10 smithi047 ceph-mon[31547]: mgrmap e14: x(active, since 103s), standbys: z 2024-03-21T10:14:10.464 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:14:10 smithi047 ceph-mon[31547]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-03-21T10:14:10.490 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:14:10 smithi157 ceph-mon[35581]: from='client.? 172.21.15.47:0/92595736' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-21T10:14:10.490 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:14:10 smithi157 ceph-mon[35581]: mgrmap e14: x(active, since 103s), standbys: z 2024-03-21T10:14:10.490 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:14:10 smithi157 ceph-mon[35581]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-03-21T10:14:10.729 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Mar 21 10:14:10 smithi100 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-mgr-y[36578]: 2024-03-21T10:14:10.331+0000 7f22da443200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-03-21T10:14:11.723 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Mar 21 10:14:11 smithi100 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-mgr-y[36578]: 2024-03-21T10:14:11.349+0000 7f22da443200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-03-21T10:14:11.979 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:14:11 smithi100 ceph-mon[35193]: pgmap v47: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T10:14:11.989 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:14:11 smithi157 ceph-mon[35581]: pgmap v47: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T10:14:12.213 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:14:11 smithi047 ceph-mon[31547]: pgmap v47: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T10:14:12.473 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Mar 21 10:14:12 smithi100 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-mgr-y[36578]: 2024-03-21T10:14:12.062+0000 7f22da443200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-03-21T10:14:12.473 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Mar 21 10:14:12 smithi100 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-mgr-y[36578]: 2024-03-21T10:14:12.189+0000 7f22da443200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-03-21T10:14:12.729 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Mar 21 10:14:12 smithi100 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-mgr-y[36578]: 2024-03-21T10:14:12.471+0000 7f22da443200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-03-21T10:14:12.755 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:14:12.752+0000 7f5cde000700 1 -- 172.21.15.47:0/1768035311 <== mon.0 v2:172.21.15.47:3300/0 8 ==== mgrmap(e 15) v1 ==== 137798+0+0 (secure 0 0 0) 0x7f5cd004ebd0 con 0x7f5ce01021d0 2024-03-21T10:14:13.172 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:14:12 smithi047 ceph-mon[31547]: Standby manager daemon y started 2024-03-21T10:14:13.172 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:14:12 smithi047 ceph-mon[31547]: from='mgr.? 172.21.15.100:0/2031522755' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-03-21T10:14:13.172 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:14:12 smithi047 ceph-mon[31547]: from='mgr.? 172.21.15.100:0/2031522755' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-03-21T10:14:13.172 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:14:12 smithi047 ceph-mon[31547]: from='mgr.? 172.21.15.100:0/2031522755' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-03-21T10:14:13.172 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:14:12 smithi047 ceph-mon[31547]: from='mgr.? 172.21.15.100:0/2031522755' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-03-21T10:14:13.229 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:14:12 smithi100 ceph-mon[35193]: Standby manager daemon y started 2024-03-21T10:14:13.229 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:14:12 smithi100 ceph-mon[35193]: from='mgr.? 172.21.15.100:0/2031522755' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-03-21T10:14:13.230 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:14:12 smithi100 ceph-mon[35193]: from='mgr.? 172.21.15.100:0/2031522755' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-03-21T10:14:13.230 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:14:12 smithi100 ceph-mon[35193]: from='mgr.? 172.21.15.100:0/2031522755' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-03-21T10:14:13.230 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:14:12 smithi100 ceph-mon[35193]: from='mgr.? 172.21.15.100:0/2031522755' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-03-21T10:14:13.239 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:14:12 smithi157 ceph-mon[35581]: Standby manager daemon y started 2024-03-21T10:14:13.240 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:14:12 smithi157 ceph-mon[35581]: from='mgr.? 172.21.15.100:0/2031522755' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-03-21T10:14:13.240 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:14:12 smithi157 ceph-mon[35581]: from='mgr.? 172.21.15.100:0/2031522755' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-03-21T10:14:13.240 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:14:12 smithi157 ceph-mon[35581]: from='mgr.? 172.21.15.100:0/2031522755' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-03-21T10:14:13.240 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:14:12 smithi157 ceph-mon[35581]: from='mgr.? 172.21.15.100:0/2031522755' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-03-21T10:14:13.881 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:14:13 smithi047 ceph-mon[31547]: pgmap v48: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T10:14:13.881 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:14:13 smithi047 ceph-mon[31547]: mgrmap e15: x(active, since 106s), standbys: z, y 2024-03-21T10:14:13.882 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:14:13 smithi047 ceph-mon[31547]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-03-21T10:14:14.229 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:14:13 smithi100 ceph-mon[35193]: pgmap v48: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T10:14:14.229 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:14:13 smithi100 ceph-mon[35193]: mgrmap e15: x(active, since 106s), standbys: z, y 2024-03-21T10:14:14.229 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:14:13 smithi100 ceph-mon[35193]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-03-21T10:14:14.239 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:14:13 smithi157 ceph-mon[35581]: pgmap v48: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T10:14:14.240 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:14:13 smithi157 ceph-mon[35581]: mgrmap e15: x(active, since 106s), standbys: z, y 2024-03-21T10:14:14.240 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:14:13 smithi157 ceph-mon[35581]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-03-21T10:14:15.964 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:14:15 smithi047 ceph-mon[31547]: pgmap v49: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T10:14:15.964 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:14:15 smithi047 ceph-mon[31547]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:14:15.964 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:14:15 smithi047 ceph-mon[31547]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:14:15.964 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:14:15 smithi047 ceph-mon[31547]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T10:14:16.032 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:14:15 smithi100 ceph-mon[35193]: pgmap v49: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T10:14:16.032 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:14:15 smithi100 ceph-mon[35193]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:14:16.032 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:14:15 smithi100 ceph-mon[35193]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:14:16.033 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:14:15 smithi100 ceph-mon[35193]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T10:14:16.239 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:14:15 smithi157 ceph-mon[35581]: pgmap v49: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T10:14:16.239 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:14:15 smithi157 ceph-mon[35581]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:14:16.239 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:14:15 smithi157 ceph-mon[35581]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:14:16.239 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:14:15 smithi157 ceph-mon[35581]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T10:14:17.979 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:14:17 smithi100 ceph-mon[35193]: pgmap v50: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T10:14:17.979 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:14:17 smithi100 ceph-mon[35193]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T10:14:17.980 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:14:17 smithi100 ceph-mon[35193]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T10:14:17.980 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:14:17 smithi100 ceph-mon[35193]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:14:18.214 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:14:17 smithi047 ceph-mon[31547]: pgmap v50: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T10:14:18.214 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:14:17 smithi047 ceph-mon[31547]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T10:14:18.214 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:14:17 smithi047 ceph-mon[31547]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T10:14:18.214 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:14:17 smithi047 ceph-mon[31547]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:14:18.239 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:14:17 smithi157 ceph-mon[35581]: pgmap v50: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T10:14:18.240 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:14:17 smithi157 ceph-mon[35581]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T10:14:18.240 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:14:17 smithi157 ceph-mon[35581]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T10:14:18.240 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:14:17 smithi157 ceph-mon[35581]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:14:18.963 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:14:18 smithi047 ceph-mon[31547]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-03-21T10:14:18.964 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:14:18 smithi047 ceph-mon[31547]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T10:14:19.229 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:14:18 smithi100 ceph-mon[35193]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-03-21T10:14:19.230 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:14:18 smithi100 ceph-mon[35193]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T10:14:19.239 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:14:18 smithi157 ceph-mon[35581]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-03-21T10:14:19.239 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:14:18 smithi157 ceph-mon[35581]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T10:14:19.962 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:14:19 smithi047 ceph-mon[31547]: pgmap v51: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T10:14:19.962 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:14:19 smithi047 ceph-mon[31547]: Deploying daemon osd.0 on smithi047 2024-03-21T10:14:19.989 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:14:19 smithi157 ceph-mon[35581]: pgmap v51: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T10:14:19.989 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:14:19 smithi157 ceph-mon[35581]: Deploying daemon osd.0 on smithi047 2024-03-21T10:14:20.229 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:14:19 smithi100 ceph-mon[35193]: pgmap v51: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T10:14:20.229 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:14:19 smithi100 ceph-mon[35193]: Deploying daemon osd.0 on smithi047 2024-03-21T10:14:22.186 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:14:21 smithi047 ceph-mon[31547]: pgmap v52: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T10:14:22.229 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:14:21 smithi100 ceph-mon[35193]: pgmap v52: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T10:14:22.239 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:14:21 smithi157 ceph-mon[35581]: pgmap v52: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T10:14:23.964 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:14:23 smithi047 ceph-mon[31547]: pgmap v53: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T10:14:24.229 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:14:23 smithi100 ceph-mon[35193]: pgmap v53: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T10:14:24.239 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:14:23 smithi157 ceph-mon[35581]: pgmap v53: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T10:14:25.964 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:14:25 smithi047 ceph-mon[31547]: pgmap v54: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T10:14:26.229 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:14:25 smithi100 ceph-mon[35193]: pgmap v54: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T10:14:26.239 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:14:25 smithi157 ceph-mon[35581]: pgmap v54: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T10:14:26.963 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:14:26 smithi047 ceph-mon[31547]: pgmap v55: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T10:14:27.229 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:14:26 smithi100 ceph-mon[35193]: pgmap v55: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T10:14:27.239 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:14:26 smithi157 ceph-mon[35581]: pgmap v55: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T10:14:28.053 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:14:27 smithi047 ceph-mon[31547]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:14:28.053 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:14:27 smithi047 ceph-mon[31547]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T10:14:28.053 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:14:27 smithi047 ceph-mon[31547]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:14:28.229 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:14:27 smithi100 ceph-mon[35193]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:14:28.229 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:14:27 smithi100 ceph-mon[35193]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T10:14:28.229 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:14:27 smithi100 ceph-mon[35193]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:14:28.239 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:14:27 smithi157 ceph-mon[35581]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:14:28.239 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:14:27 smithi157 ceph-mon[35581]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T10:14:28.239 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:14:27 smithi157 ceph-mon[35581]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:14:28.868 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:14:28 smithi047 ceph-mon[31547]: pgmap v56: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T10:14:29.229 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:14:28 smithi100 ceph-mon[35193]: pgmap v56: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T10:14:29.239 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:14:28 smithi157 ceph-mon[35581]: pgmap v56: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T10:14:31.052 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:14:30 smithi047 ceph-mon[31547]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:14:31.052 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:14:30 smithi047 ceph-mon[31547]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:14:31.052 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:14:30 smithi047 ceph-mon[31547]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T10:14:31.052 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:14:30 smithi047 ceph-mon[31547]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T10:14:31.052 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:14:30 smithi047 ceph-mon[31547]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:14:31.053 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:14:30 smithi047 ceph-mon[31547]: pgmap v57: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T10:14:31.230 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:14:30 smithi100 ceph-mon[35193]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:14:31.230 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:14:30 smithi100 ceph-mon[35193]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:14:31.230 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:14:30 smithi100 ceph-mon[35193]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T10:14:31.230 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:14:30 smithi100 ceph-mon[35193]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T10:14:31.230 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:14:30 smithi100 ceph-mon[35193]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:14:31.230 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:14:30 smithi100 ceph-mon[35193]: pgmap v57: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T10:14:31.239 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:14:30 smithi157 ceph-mon[35581]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:14:31.240 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:14:30 smithi157 ceph-mon[35581]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:14:31.240 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:14:30 smithi157 ceph-mon[35581]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T10:14:31.240 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:14:30 smithi157 ceph-mon[35581]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T10:14:31.240 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:14:30 smithi157 ceph-mon[35581]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:14:31.240 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:14:30 smithi157 ceph-mon[35581]: pgmap v57: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T10:14:31.778 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:14:31.776+0000 7f5cde000700 1 -- 172.21.15.47:0/1768035311 <== mgr.14152 v2:172.21.15.47:6800/2823885994 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f5ce0075ae0 con 0x7f5cc4039460 2024-03-21T10:14:31.779 INFO:teuthology.orchestra.run.smithi047.stdout:Created osd(s) 0 on host 'smithi047' 2024-03-21T10:14:31.781 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:14:31.780+0000 7f5ce7600700 1 -- 172.21.15.47:0/1768035311 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f5cc4039460 msgr2=0x7f5cc403b920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:14:31.781 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:14:31.780+0000 7f5ce7600700 1 --2- 172.21.15.47:0/1768035311 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f5cc4039460 0x7f5cc403b920 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f5cc800ac50 tx=0x7f5cc800a380 comp rx=0 tx=0).stop 2024-03-21T10:14:31.781 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:14:31.780+0000 7f5ce7600700 1 -- 172.21.15.47:0/1768035311 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5ce01021d0 msgr2=0x7f5ce019a470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:14:31.781 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:14:31.780+0000 7f5ce7600700 1 --2- 172.21.15.47:0/1768035311 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5ce01021d0 0x7f5ce019a470 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f5cd000e9a0 tx=0x7f5cd000ecb0 comp rx=0 tx=0).stop 2024-03-21T10:14:31.782 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:14:31.780+0000 7f5ce7600700 1 -- 172.21.15.47:0/1768035311 shutdown_connections 2024-03-21T10:14:31.782 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:14:31.780+0000 7f5ce7600700 1 --2- 172.21.15.47:0/1768035311 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f5cc4039460 0x7f5cc403b920 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:14:31.782 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:14:31.780+0000 7f5ce7600700 1 --2- 172.21.15.47:0/1768035311 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5ce0103d70 0x7f5ce01a1a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:14:31.782 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:14:31.780+0000 7f5ce7600700 1 --2- 172.21.15.47:0/1768035311 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5ce01021d0 0x7f5ce019a470 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:14:31.782 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:14:31.780+0000 7f5ce7600700 1 --2- 172.21.15.47:0/1768035311 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5ce01033d0 0x7f5ce019a9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:14:31.782 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:14:31.780+0000 7f5ce7600700 1 -- 172.21.15.47:0/1768035311 >> 172.21.15.47:0/1768035311 conn(0x7f5ce00fd730 msgr2=0x7f5ce00743c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:14:31.782 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:14:31.781+0000 7f5ce7600700 1 -- 172.21.15.47:0/1768035311 shutdown_connections 2024-03-21T10:14:31.782 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:14:31.781+0000 7f5ce7600700 1 -- 172.21.15.47:0/1768035311 wait complete. 2024-03-21T10:14:31.964 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:14:31 smithi047 ceph-mon[31547]: from='osd.0 [v2:172.21.15.47:6802/775837313,v1:172.21.15.47:6803/775837313]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-03-21T10:14:31.964 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:14:31 smithi047 ceph-mon[31547]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T10:14:31.964 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:14:31 smithi047 ceph-mon[31547]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:14:31.964 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:14:31 smithi047 ceph-mon[31547]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:14:32.229 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:14:31 smithi100 ceph-mon[35193]: from='osd.0 [v2:172.21.15.47:6802/775837313,v1:172.21.15.47:6803/775837313]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-03-21T10:14:32.229 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:14:31 smithi100 ceph-mon[35193]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T10:14:32.230 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:14:31 smithi100 ceph-mon[35193]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:14:32.230 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:14:31 smithi100 ceph-mon[35193]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:14:32.239 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:14:31 smithi157 ceph-mon[35581]: from='osd.0 [v2:172.21.15.47:6802/775837313,v1:172.21.15.47:6803/775837313]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-03-21T10:14:32.240 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:14:31 smithi157 ceph-mon[35581]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T10:14:32.240 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:14:31 smithi157 ceph-mon[35581]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:14:32.240 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:14:31 smithi157 ceph-mon[35581]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:14:32.520 DEBUG:teuthology.orchestra.run.smithi047:osd.0> sudo journalctl -f -n 0 -u ceph-3056d51a-e76b-11ee-95cd-87774f69a715@osd.0.service 2024-03-21T10:14:32.525 INFO:tasks.cephadm:Deploying osd.1 on smithi100 with /dev/vg_nvme/lv_4... 2024-03-21T10:14:32.525 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2edfee68ad7243bed422a049b6f5f58619300f11 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3056d51a-e76b-11ee-95cd-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-03-21T10:14:32.973 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/3056d51a-e76b-11ee-95cd-87774f69a715/mon.b/config 2024-03-21T10:14:32.979 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:14:32 smithi100 ceph-mon[35193]: from='osd.0 [v2:172.21.15.47:6802/775837313,v1:172.21.15.47:6803/775837313]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-03-21T10:14:32.979 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:14:32 smithi100 ceph-mon[35193]: osdmap e6: 1 total, 0 up, 1 in 2024-03-21T10:14:32.979 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:14:32 smithi100 ceph-mon[35193]: from='osd.0 [v2:172.21.15.47:6802/775837313,v1:172.21.15.47:6803/775837313]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi047", "root=default"]}]: dispatch 2024-03-21T10:14:32.979 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:14:32 smithi100 ceph-mon[35193]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-21T10:14:32.979 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:14:32 smithi100 ceph-mon[35193]: pgmap v59: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T10:14:33.067 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:14:32 smithi047 ceph-mon[31547]: from='osd.0 [v2:172.21.15.47:6802/775837313,v1:172.21.15.47:6803/775837313]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-03-21T10:14:33.067 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:14:32 smithi047 ceph-mon[31547]: osdmap e6: 1 total, 0 up, 1 in 2024-03-21T10:14:33.067 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:14:32 smithi047 ceph-mon[31547]: from='osd.0 [v2:172.21.15.47:6802/775837313,v1:172.21.15.47:6803/775837313]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi047", "root=default"]}]: dispatch 2024-03-21T10:14:33.068 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:14:32 smithi047 ceph-mon[31547]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-21T10:14:33.068 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:14:32 smithi047 ceph-mon[31547]: pgmap v59: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T10:14:33.068 INFO:journalctl@ceph.osd.0.smithi047.stdout:Mar 21 10:14:32 smithi047 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-0[44057]: 2024-03-21T10:14:32.861+0000 7f04d9400700 -1 osd.0 0 waiting for initial osdmap 2024-03-21T10:14:33.068 INFO:journalctl@ceph.osd.0.smithi047.stdout:Mar 21 10:14:32 smithi047 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-0[44057]: 2024-03-21T10:14:32.869+0000 7f04d4200700 -1 osd.0 7 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-21T10:14:33.240 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:14:32 smithi157 ceph-mon[35581]: from='osd.0 [v2:172.21.15.47:6802/775837313,v1:172.21.15.47:6803/775837313]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-03-21T10:14:33.240 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:14:32 smithi157 ceph-mon[35581]: osdmap e6: 1 total, 0 up, 1 in 2024-03-21T10:14:33.241 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:14:32 smithi157 ceph-mon[35581]: from='osd.0 [v2:172.21.15.47:6802/775837313,v1:172.21.15.47:6803/775837313]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi047", "root=default"]}]: dispatch 2024-03-21T10:14:33.241 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:14:32 smithi157 ceph-mon[35581]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-21T10:14:33.241 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:14:32 smithi157 ceph-mon[35581]: pgmap v59: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T10:14:33.896 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:14:33 smithi047 ceph-mon[31547]: purged_snaps scrub starts 2024-03-21T10:14:33.896 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:14:33 smithi047 ceph-mon[31547]: purged_snaps scrub ok 2024-03-21T10:14:33.896 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:14:33 smithi047 ceph-mon[31547]: from='osd.0 [v2:172.21.15.47:6802/775837313,v1:172.21.15.47:6803/775837313]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi047", "root=default"]}]': finished 2024-03-21T10:14:33.896 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:14:33 smithi047 ceph-mon[31547]: osdmap e7: 1 total, 0 up, 1 in 2024-03-21T10:14:33.896 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:14:33 smithi047 ceph-mon[31547]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-21T10:14:33.896 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:14:33 smithi047 ceph-mon[31547]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-21T10:14:34.159 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:14:33 smithi100 ceph-mon[35193]: purged_snaps scrub starts 2024-03-21T10:14:34.160 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:14:33 smithi100 ceph-mon[35193]: purged_snaps scrub ok 2024-03-21T10:14:34.160 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:14:33 smithi100 ceph-mon[35193]: from='osd.0 [v2:172.21.15.47:6802/775837313,v1:172.21.15.47:6803/775837313]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi047", "root=default"]}]': finished 2024-03-21T10:14:34.160 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:14:33 smithi100 ceph-mon[35193]: osdmap e7: 1 total, 0 up, 1 in 2024-03-21T10:14:34.160 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:14:33 smithi100 ceph-mon[35193]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-21T10:14:34.160 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:14:33 smithi100 ceph-mon[35193]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-21T10:14:34.240 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:14:33 smithi157 ceph-mon[35581]: purged_snaps scrub starts 2024-03-21T10:14:34.240 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:14:33 smithi157 ceph-mon[35581]: purged_snaps scrub ok 2024-03-21T10:14:34.240 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:14:33 smithi157 ceph-mon[35581]: from='osd.0 [v2:172.21.15.47:6802/775837313,v1:172.21.15.47:6803/775837313]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi047", "root=default"]}]': finished 2024-03-21T10:14:34.240 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:14:33 smithi157 ceph-mon[35581]: osdmap e7: 1 total, 0 up, 1 in 2024-03-21T10:14:34.240 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:14:33 smithi157 ceph-mon[35581]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-21T10:14:34.240 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:14:33 smithi157 ceph-mon[35581]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-21T10:14:35.214 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:14:34 smithi047 ceph-mon[31547]: osd.0 [v2:172.21.15.47:6802/775837313,v1:172.21.15.47:6803/775837313] boot 2024-03-21T10:14:35.215 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:14:34 smithi047 ceph-mon[31547]: osdmap e8: 1 total, 1 up, 1 in 2024-03-21T10:14:35.215 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:14:34 smithi047 ceph-mon[31547]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-21T10:14:35.215 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:14:34 smithi047 ceph-mon[31547]: pgmap v62: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T10:14:35.229 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:14:34 smithi100 ceph-mon[35193]: osd.0 [v2:172.21.15.47:6802/775837313,v1:172.21.15.47:6803/775837313] boot 2024-03-21T10:14:35.229 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:14:34 smithi100 ceph-mon[35193]: osdmap e8: 1 total, 1 up, 1 in 2024-03-21T10:14:35.229 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:14:34 smithi100 ceph-mon[35193]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-21T10:14:35.229 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:14:34 smithi100 ceph-mon[35193]: pgmap v62: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T10:14:35.239 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:14:34 smithi157 ceph-mon[35581]: osd.0 [v2:172.21.15.47:6802/775837313,v1:172.21.15.47:6803/775837313] boot 2024-03-21T10:14:35.239 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:14:34 smithi157 ceph-mon[35581]: osdmap e8: 1 total, 1 up, 1 in 2024-03-21T10:14:35.239 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:14:34 smithi157 ceph-mon[35581]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-21T10:14:35.239 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:14:34 smithi157 ceph-mon[35581]: pgmap v62: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T10:14:35.654 INFO:teuthology.orchestra.run.smithi100.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-03-21T10:14:35.654 INFO:teuthology.orchestra.run.smithi100.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-03-21T10:14:35.655 INFO:teuthology.orchestra.run.smithi100.stderr: stderr: 10+0 records in 2024-03-21T10:14:35.655 INFO:teuthology.orchestra.run.smithi100.stderr:10+0 records out 2024-03-21T10:14:35.655 INFO:teuthology.orchestra.run.smithi100.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0348713 s, 301 MB/s 2024-03-21T10:14:35.655 INFO:teuthology.orchestra.run.smithi100.stderr:--> Zapping successful for: 2024-03-21T10:14:35.979 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:14:35 smithi100 ceph-mon[35193]: osdmap e9: 1 total, 1 up, 1 in 2024-03-21T10:14:36.213 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:14:35 smithi047 ceph-mon[31547]: osdmap e9: 1 total, 1 up, 1 in 2024-03-21T10:14:36.239 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:14:35 smithi157 ceph-mon[35581]: osdmap e9: 1 total, 1 up, 1 in 2024-03-21T10:14:36.407 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2edfee68ad7243bed422a049b6f5f58619300f11 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3056d51a-e76b-11ee-95cd-87774f69a715 -- ceph orch daemon add osd smithi100:vg_nvme/lv_4 2024-03-21T10:14:36.845 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/3056d51a-e76b-11ee-95cd-87774f69a715/mon.b/config 2024-03-21T10:14:37.149 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:14:36 smithi100 ceph-mon[35193]: pgmap v64: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-21T10:14:37.160 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:14:36 smithi047 ceph-mon[31547]: pgmap v64: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-21T10:14:37.240 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:14:36 smithi157 ceph-mon[35581]: pgmap v64: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-21T10:14:38.480 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:14:38 smithi100 ceph-mon[35193]: Detected new or changed devices on smithi047 2024-03-21T10:14:38.480 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:14:38 smithi100 ceph-mon[35193]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:14:38.480 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:14:38 smithi100 ceph-mon[35193]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:14:38.481 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:14:38 smithi100 ceph-mon[35193]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-21T10:14:38.481 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:14:38 smithi100 ceph-mon[35193]: Adjusting osd_memory_target on smithi047 to 17095M 2024-03-21T10:14:38.481 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:14:38 smithi100 ceph-mon[35193]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:14:38.481 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:14:38 smithi100 ceph-mon[35193]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T10:14:38.481 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:14:38 smithi100 ceph-mon[35193]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T10:14:38.481 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:14:38 smithi100 ceph-mon[35193]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:14:38.678 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-21T10:14:38.676+0000 7f172f200700 1 -- 172.21.15.100:0/780210938 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1728109a10 msgr2=0x7f1728109e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:14:38.678 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-21T10:14:38.676+0000 7f172f200700 1 --2- 172.21.15.100:0/780210938 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1728109a10 0x7f1728109e70 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f17140099c0 tx=0x7f1714009cd0 comp rx=0 tx=0).stop 2024-03-21T10:14:38.678 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-21T10:14:38.676+0000 7f172f200700 1 -- 172.21.15.100:0/780210938 shutdown_connections 2024-03-21T10:14:38.678 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-21T10:14:38.676+0000 7f172f200700 1 --2- 172.21.15.100:0/780210938 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f172810a3b0 0x7f1728110c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:14:38.678 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-21T10:14:38.676+0000 7f172f200700 1 --2- 172.21.15.100:0/780210938 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1728108810 0x7f1728108c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:14:38.678 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-21T10:14:38.676+0000 7f172f200700 1 --2- 172.21.15.100:0/780210938 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1728109a10 0x7f1728109e70 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:14:38.678 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-21T10:14:38.676+0000 7f172f200700 1 -- 172.21.15.100:0/780210938 >> 172.21.15.100:0/780210938 conn(0x7f17280777e0 msgr2=0x7f1728079c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:14:38.678 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-21T10:14:38.676+0000 7f172f200700 1 -- 172.21.15.100:0/780210938 shutdown_connections 2024-03-21T10:14:38.678 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-21T10:14:38.677+0000 7f172f200700 1 -- 172.21.15.100:0/780210938 wait complete. 2024-03-21T10:14:38.680 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-21T10:14:38.678+0000 7f172f200700 1 Processor -- start 2024-03-21T10:14:38.680 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-21T10:14:38.678+0000 7f172f200700 1 -- start start 2024-03-21T10:14:38.680 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-21T10:14:38.678+0000 7f172f200700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1728108810 0x7f172819ea70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:14:38.680 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-21T10:14:38.678+0000 7f172f200700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1728109a10 0x7f172819efb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:14:38.680 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-21T10:14:38.678+0000 7f172f200700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f172810a3b0 0x7f17281a6030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:14:38.681 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-21T10:14:38.678+0000 7f172f200700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17281143a0 con 0x7f1728109a10 2024-03-21T10:14:38.681 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-21T10:14:38.678+0000 7f172f200700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1728114220 con 0x7f1728108810 2024-03-21T10:14:38.681 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-21T10:14:38.678+0000 7f172f200700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1728114520 con 0x7f172810a3b0 2024-03-21T10:14:38.681 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-21T10:14:38.678+0000 7f1727e00700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1728108810 0x7f172819ea70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:14:38.681 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-21T10:14:38.678+0000 7f1727e00700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1728108810 0x7f172819ea70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:43520/0 (socket says 172.21.15.100:43520) 2024-03-21T10:14:38.681 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-21T10:14:38.678+0000 7f1727e00700 1 -- 172.21.15.100:0/3599973837 learned_addr learned my addr 172.21.15.100:0/3599973837 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-21T10:14:38.681 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-21T10:14:38.679+0000 7f172d200700 1 --2- 172.21.15.100:0/3599973837 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f172810a3b0 0x7f17281a6030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:14:38.681 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-21T10:14:38.679+0000 7f1727400700 1 --2- 172.21.15.100:0/3599973837 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1728109a10 0x7f172819efb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:14:38.681 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-21T10:14:38.679+0000 7f1727e00700 1 -- 172.21.15.100:0/3599973837 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f172810a3b0 msgr2=0x7f17281a6030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:14:38.681 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-21T10:14:38.679+0000 7f1727e00700 1 --2- 172.21.15.100:0/3599973837 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f172810a3b0 0x7f17281a6030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:14:38.681 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-21T10:14:38.679+0000 7f1727e00700 1 -- 172.21.15.100:0/3599973837 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1728109a10 msgr2=0x7f172819efb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:14:38.681 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-21T10:14:38.679+0000 7f1727e00700 1 --2- 172.21.15.100:0/3599973837 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1728109a10 0x7f172819efb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:14:38.682 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-21T10:14:38.679+0000 7f1727e00700 1 -- 172.21.15.100:0/3599973837 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f17281a66d0 con 0x7f1728108810 2024-03-21T10:14:38.682 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-21T10:14:38.679+0000 7f1727400700 1 --2- 172.21.15.100:0/3599973837 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1728109a10 0x7f172819efb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T10:14:38.682 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-21T10:14:38.679+0000 7f1727e00700 1 --2- 172.21.15.100:0/3599973837 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1728108810 0x7f172819ea70 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f171800b500 tx=0x7f171800b8c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:14:38.682 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-21T10:14:38.679+0000 7f1724c00700 1 -- 172.21.15.100:0/3599973837 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1718015020 con 0x7f1728108810 2024-03-21T10:14:38.682 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-21T10:14:38.680+0000 7f1724c00700 1 -- 172.21.15.100:0/3599973837 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f1718011ce0 con 0x7f1728108810 2024-03-21T10:14:38.682 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-21T10:14:38.680+0000 7f172f200700 1 -- 172.21.15.100:0/3599973837 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f17281a69c0 con 0x7f1728108810 2024-03-21T10:14:38.682 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-21T10:14:38.680+0000 7f1724c00700 1 -- 172.21.15.100:0/3599973837 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1718010d90 con 0x7f1728108810 2024-03-21T10:14:38.683 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-21T10:14:38.680+0000 7f172f200700 1 -- 172.21.15.100:0/3599973837 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f17281a6f80 con 0x7f1728108810 2024-03-21T10:14:38.683 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-21T10:14:38.681+0000 7f172f200700 1 -- 172.21.15.100:0/3599973837 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1728068e50 con 0x7f1728108810 2024-03-21T10:14:38.684 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-21T10:14:38.683+0000 7f1724c00700 1 -- 172.21.15.100:0/3599973837 <== mon.2 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 15) v1 ==== 137798+0+0 (secure 0 0 0) 0x7f1718018020 con 0x7f1728108810 2024-03-21T10:14:38.685 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-21T10:14:38.684+0000 7f1724c00700 1 --2- 172.21.15.100:0/3599973837 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f170c0a3020 0x7f170c0a54e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:14:38.685 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-21T10:14:38.685+0000 7f1724c00700 1 -- 172.21.15.100:0/3599973837 <== mon.2 v2:172.21.15.100:3300/0 5 ==== osd_map(9..9 src has 1..9) v4 ==== 1767+0+0 (secure 0 0 0) 0x7f17180cd720 con 0x7f1728108810 2024-03-21T10:14:38.686 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-21T10:14:38.685+0000 7f1727400700 1 --2- 172.21.15.100:0/3599973837 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f170c0a3020 0x7f170c0a54e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:14:38.687 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-21T10:14:38.686+0000 7f1727400700 1 --2- 172.21.15.100:0/3599973837 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f170c0a3020 0x7f170c0a54e0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f1714006010 tx=0x7f1714005c40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:14:38.689 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-21T10:14:38.688+0000 7f1724c00700 1 -- 172.21.15.100:0/3599973837 <== mon.2 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f171801f020 con 0x7f1728108810 2024-03-21T10:14:38.714 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:14:38 smithi047 ceph-mon[31547]: Detected new or changed devices on smithi047 2024-03-21T10:14:38.714 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:14:38 smithi047 ceph-mon[31547]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:14:38.714 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:14:38 smithi047 ceph-mon[31547]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:14:38.714 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:14:38 smithi047 ceph-mon[31547]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-21T10:14:38.714 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:14:38 smithi047 ceph-mon[31547]: Adjusting osd_memory_target on smithi047 to 17095M 2024-03-21T10:14:38.715 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:14:38 smithi047 ceph-mon[31547]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:14:38.715 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:14:38 smithi047 ceph-mon[31547]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T10:14:38.715 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:14:38 smithi047 ceph-mon[31547]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T10:14:38.715 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:14:38 smithi047 ceph-mon[31547]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:14:38.740 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:14:38 smithi157 ceph-mon[35581]: Detected new or changed devices on smithi047 2024-03-21T10:14:38.740 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:14:38 smithi157 ceph-mon[35581]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:14:38.740 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:14:38 smithi157 ceph-mon[35581]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:14:38.740 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:14:38 smithi157 ceph-mon[35581]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-21T10:14:38.740 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:14:38 smithi157 ceph-mon[35581]: Adjusting osd_memory_target on smithi047 to 17095M 2024-03-21T10:14:38.740 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:14:38 smithi157 ceph-mon[35581]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:14:38.740 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:14:38 smithi157 ceph-mon[35581]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T10:14:38.740 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:14:38 smithi157 ceph-mon[35581]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T10:14:38.740 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:14:38 smithi157 ceph-mon[35581]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:14:38.920 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-21T10:14:38.919+0000 7f172f200700 1 -- 172.21.15.100:0/3599973837 --> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi100:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) v1 -- 0x7f17280625d0 con 0x7f170c0a3020 2024-03-21T10:14:39.407 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:14:39 smithi157 ceph-mon[35581]: pgmap v65: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-21T10:14:39.407 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:14:39 smithi157 ceph-mon[35581]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-21T10:14:39.407 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:14:39 smithi157 ceph-mon[35581]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-21T10:14:39.407 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:14:39 smithi157 ceph-mon[35581]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T10:14:39.714 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:14:39 smithi047 ceph-mon[31547]: pgmap v65: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-21T10:14:39.714 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:14:39 smithi047 ceph-mon[31547]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-21T10:14:39.714 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:14:39 smithi047 ceph-mon[31547]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-21T10:14:39.714 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:14:39 smithi047 ceph-mon[31547]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T10:14:39.729 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:14:39 smithi100 ceph-mon[35193]: pgmap v65: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-21T10:14:39.729 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:14:39 smithi100 ceph-mon[35193]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-21T10:14:39.729 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:14:39 smithi100 ceph-mon[35193]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-21T10:14:39.729 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:14:39 smithi100 ceph-mon[35193]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T10:14:40.479 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:14:40 smithi100 ceph-mon[35193]: from='client.34136 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi100:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:14:40.714 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:14:40 smithi047 ceph-mon[31547]: from='client.34136 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi100:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:14:40.739 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:14:40 smithi157 ceph-mon[35581]: from='client.34136 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi100:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:14:41.585 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:14:41 smithi100 ceph-mon[35193]: pgmap v66: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-21T10:14:41.714 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:14:41 smithi047 ceph-mon[31547]: pgmap v66: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-21T10:14:41.739 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:14:41 smithi157 ceph-mon[35581]: pgmap v66: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-21T10:14:43.482 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:14:43 smithi100 ceph-mon[35193]: pgmap v67: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-21T10:14:43.482 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:14:43 smithi100 ceph-mon[35193]: from='client.? 172.21.15.100:0/3022408630' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "07bb81bb-7cf3-4bbc-ac9e-802f95185d11"}]: dispatch 2024-03-21T10:14:43.482 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:14:43 smithi100 ceph-mon[35193]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "07bb81bb-7cf3-4bbc-ac9e-802f95185d11"}]: dispatch 2024-03-21T10:14:43.482 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:14:43 smithi100 ceph-mon[35193]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "07bb81bb-7cf3-4bbc-ac9e-802f95185d11"}]': finished 2024-03-21T10:14:43.482 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:14:43 smithi100 ceph-mon[35193]: osdmap e10: 2 total, 1 up, 2 in 2024-03-21T10:14:43.482 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:14:43 smithi100 ceph-mon[35193]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-21T10:14:43.714 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:14:43 smithi047 ceph-mon[31547]: pgmap v67: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-21T10:14:43.714 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:14:43 smithi047 ceph-mon[31547]: from='client.? 172.21.15.100:0/3022408630' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "07bb81bb-7cf3-4bbc-ac9e-802f95185d11"}]: dispatch 2024-03-21T10:14:43.714 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:14:43 smithi047 ceph-mon[31547]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "07bb81bb-7cf3-4bbc-ac9e-802f95185d11"}]: dispatch 2024-03-21T10:14:43.714 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:14:43 smithi047 ceph-mon[31547]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "07bb81bb-7cf3-4bbc-ac9e-802f95185d11"}]': finished 2024-03-21T10:14:43.714 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:14:43 smithi047 ceph-mon[31547]: osdmap e10: 2 total, 1 up, 2 in 2024-03-21T10:14:43.714 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:14:43 smithi047 ceph-mon[31547]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-21T10:14:43.740 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:14:43 smithi157 ceph-mon[35581]: pgmap v67: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-21T10:14:43.740 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:14:43 smithi157 ceph-mon[35581]: from='client.? 172.21.15.100:0/3022408630' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "07bb81bb-7cf3-4bbc-ac9e-802f95185d11"}]: dispatch 2024-03-21T10:14:43.740 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:14:43 smithi157 ceph-mon[35581]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "07bb81bb-7cf3-4bbc-ac9e-802f95185d11"}]: dispatch 2024-03-21T10:14:43.740 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:14:43 smithi157 ceph-mon[35581]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "07bb81bb-7cf3-4bbc-ac9e-802f95185d11"}]': finished 2024-03-21T10:14:43.740 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:14:43 smithi157 ceph-mon[35581]: osdmap e10: 2 total, 1 up, 2 in 2024-03-21T10:14:43.740 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:14:43 smithi157 ceph-mon[35581]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-21T10:14:44.713 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:14:44 smithi047 ceph-mon[31547]: from='client.? 172.21.15.100:0/1978323836' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-21T10:14:44.729 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:14:44 smithi100 ceph-mon[35193]: from='client.? 172.21.15.100:0/1978323836' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-21T10:14:44.739 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:14:44 smithi157 ceph-mon[35581]: from='client.? 172.21.15.100:0/1978323836' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-21T10:14:45.713 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:14:45 smithi047 ceph-mon[31547]: pgmap v69: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-21T10:14:45.730 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:14:45 smithi100 ceph-mon[35193]: pgmap v69: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-21T10:14:45.739 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:14:45 smithi157 ceph-mon[35581]: pgmap v69: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-21T10:14:47.714 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:14:47 smithi047 ceph-mon[31547]: pgmap v70: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-21T10:14:47.729 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:14:47 smithi100 ceph-mon[35193]: pgmap v70: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-21T10:14:47.739 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:14:47 smithi157 ceph-mon[35581]: pgmap v70: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-21T10:14:49.595 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:14:49 smithi100 ceph-mon[35193]: pgmap v71: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-21T10:14:49.714 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:14:49 smithi047 ceph-mon[31547]: pgmap v71: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-21T10:14:49.739 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:14:49 smithi157 ceph-mon[35581]: pgmap v71: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-21T10:14:51.464 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:14:51 smithi047 ceph-mon[31547]: pgmap v72: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-21T10:14:51.464 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:14:51 smithi047 ceph-mon[31547]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-03-21T10:14:51.464 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:14:51 smithi047 ceph-mon[31547]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T10:14:51.729 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:14:51 smithi100 ceph-mon[35193]: pgmap v72: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-21T10:14:51.730 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:14:51 smithi100 ceph-mon[35193]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-03-21T10:14:51.730 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:14:51 smithi100 ceph-mon[35193]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T10:14:51.739 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:14:51 smithi157 ceph-mon[35581]: pgmap v72: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-21T10:14:51.739 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:14:51 smithi157 ceph-mon[35581]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-03-21T10:14:51.740 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:14:51 smithi157 ceph-mon[35581]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T10:14:52.621 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:14:52 smithi100 ceph-mon[35193]: Deploying daemon osd.1 on smithi100 2024-03-21T10:14:52.714 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:14:52 smithi047 ceph-mon[31547]: Deploying daemon osd.1 on smithi100 2024-03-21T10:14:52.739 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:14:52 smithi157 ceph-mon[35581]: Deploying daemon osd.1 on smithi100 2024-03-21T10:14:53.713 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:14:53 smithi047 ceph-mon[31547]: pgmap v73: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-21T10:14:53.729 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:14:53 smithi100 ceph-mon[35193]: pgmap v73: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-21T10:14:53.739 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:14:53 smithi157 ceph-mon[35581]: pgmap v73: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-21T10:14:55.479 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:14:55 smithi100 ceph-mon[35193]: pgmap v74: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-21T10:14:55.713 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:14:55 smithi047 ceph-mon[31547]: pgmap v74: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-21T10:14:55.739 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:14:55 smithi157 ceph-mon[35581]: pgmap v74: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-21T10:14:57.707 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:14:57 smithi100 ceph-mon[35193]: pgmap v75: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-21T10:14:57.713 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:14:57 smithi047 ceph-mon[31547]: pgmap v75: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-21T10:14:57.739 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:14:57 smithi157 ceph-mon[35581]: pgmap v75: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-21T10:14:59.675 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:14:59 smithi100 ceph-mon[35193]: pgmap v76: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-21T10:14:59.675 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:14:59 smithi100 ceph-mon[35193]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T10:14:59.675 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:14:59 smithi100 ceph-mon[35193]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:14:59.675 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:14:59 smithi100 ceph-mon[35193]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:14:59.714 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:14:59 smithi047 ceph-mon[31547]: pgmap v76: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-21T10:14:59.714 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:14:59 smithi047 ceph-mon[31547]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T10:14:59.714 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:14:59 smithi047 ceph-mon[31547]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:14:59.714 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:14:59 smithi047 ceph-mon[31547]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:14:59.739 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:14:59 smithi157 ceph-mon[35581]: pgmap v76: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-21T10:14:59.740 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:14:59 smithi157 ceph-mon[35581]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T10:14:59.740 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:14:59 smithi157 ceph-mon[35581]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:14:59.740 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:14:59 smithi157 ceph-mon[35581]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:15:01.479 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:15:01 smithi100 ceph-mon[35193]: pgmap v77: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-21T10:15:01.713 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:15:01 smithi047 ceph-mon[31547]: pgmap v77: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-21T10:15:01.739 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:15:01 smithi157 ceph-mon[35581]: pgmap v77: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-21T10:15:03.378 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-21T10:15:03.376+0000 7f1724c00700 1 -- 172.21.15.100:0/3599973837 <== mgr.14152 v2:172.21.15.47:6800/2823885994 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f17280625d0 con 0x7f170c0a3020 2024-03-21T10:15:03.379 INFO:teuthology.orchestra.run.smithi100.stdout:Created osd(s) 1 on host 'smithi100' 2024-03-21T10:15:03.381 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-21T10:15:03.380+0000 7f172f200700 1 -- 172.21.15.100:0/3599973837 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f170c0a3020 msgr2=0x7f170c0a54e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:15:03.381 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-21T10:15:03.380+0000 7f172f200700 1 --2- 172.21.15.100:0/3599973837 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f170c0a3020 0x7f170c0a54e0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f1714006010 tx=0x7f1714005c40 comp rx=0 tx=0).stop 2024-03-21T10:15:03.381 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-21T10:15:03.380+0000 7f172f200700 1 -- 172.21.15.100:0/3599973837 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1728108810 msgr2=0x7f172819ea70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:15:03.381 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-21T10:15:03.380+0000 7f172f200700 1 --2- 172.21.15.100:0/3599973837 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1728108810 0x7f172819ea70 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f171800b500 tx=0x7f171800b8c0 comp rx=0 tx=0).stop 2024-03-21T10:15:03.381 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-21T10:15:03.380+0000 7f172f200700 1 -- 172.21.15.100:0/3599973837 shutdown_connections 2024-03-21T10:15:03.382 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-21T10:15:03.380+0000 7f172f200700 1 --2- 172.21.15.100:0/3599973837 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f170c0a3020 0x7f170c0a54e0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:15:03.382 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-21T10:15:03.380+0000 7f172f200700 1 --2- 172.21.15.100:0/3599973837 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f172810a3b0 0x7f17281a6030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:15:03.382 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-21T10:15:03.380+0000 7f172f200700 1 --2- 172.21.15.100:0/3599973837 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1728108810 0x7f172819ea70 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:15:03.382 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-21T10:15:03.380+0000 7f172f200700 1 --2- 172.21.15.100:0/3599973837 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1728109a10 0x7f172819efb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:15:03.382 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-21T10:15:03.380+0000 7f172f200700 1 -- 172.21.15.100:0/3599973837 >> 172.21.15.100:0/3599973837 conn(0x7f17280777e0 msgr2=0x7f172810cd80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:15:03.382 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-21T10:15:03.380+0000 7f172f200700 1 -- 172.21.15.100:0/3599973837 shutdown_connections 2024-03-21T10:15:03.382 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-21T10:15:03.380+0000 7f172f200700 1 -- 172.21.15.100:0/3599973837 wait complete. 2024-03-21T10:15:03.730 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:15:03 smithi100 ceph-mon[35193]: pgmap v78: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-21T10:15:03.730 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:15:03 smithi100 ceph-mon[35193]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:15:03.730 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:15:03 smithi100 ceph-mon[35193]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:15:03.730 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:15:03 smithi100 ceph-mon[35193]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T10:15:03.730 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:15:03 smithi100 ceph-mon[35193]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T10:15:03.730 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:15:03 smithi100 ceph-mon[35193]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:15:03.730 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:15:03 smithi100 ceph-mon[35193]: from='osd.1 [v2:172.21.15.100:6800/2923304710,v1:172.21.15.100:6801/2923304710]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-03-21T10:15:03.731 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:15:03 smithi100 ceph-mon[35193]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-03-21T10:15:03.731 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:15:03 smithi100 ceph-mon[35193]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:15:03.731 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:15:03 smithi100 ceph-mon[35193]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T10:15:03.731 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:15:03 smithi100 ceph-mon[35193]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:15:03.964 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:15:03 smithi047 ceph-mon[31547]: pgmap v78: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-21T10:15:03.964 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:15:03 smithi047 ceph-mon[31547]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:15:03.964 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:15:03 smithi047 ceph-mon[31547]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:15:03.964 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:15:03 smithi047 ceph-mon[31547]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T10:15:03.964 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:15:03 smithi047 ceph-mon[31547]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T10:15:03.964 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:15:03 smithi047 ceph-mon[31547]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:15:03.964 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:15:03 smithi047 ceph-mon[31547]: from='osd.1 [v2:172.21.15.100:6800/2923304710,v1:172.21.15.100:6801/2923304710]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-03-21T10:15:03.965 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:15:03 smithi047 ceph-mon[31547]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-03-21T10:15:03.965 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:15:03 smithi047 ceph-mon[31547]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:15:03.965 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:15:03 smithi047 ceph-mon[31547]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T10:15:03.965 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:15:03 smithi047 ceph-mon[31547]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:15:03.990 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:15:03 smithi157 ceph-mon[35581]: pgmap v78: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-21T10:15:03.990 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:15:03 smithi157 ceph-mon[35581]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:15:03.990 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:15:03 smithi157 ceph-mon[35581]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:15:03.990 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:15:03 smithi157 ceph-mon[35581]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T10:15:03.990 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:15:03 smithi157 ceph-mon[35581]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T10:15:03.990 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:15:03 smithi157 ceph-mon[35581]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:15:03.990 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:15:03 smithi157 ceph-mon[35581]: from='osd.1 [v2:172.21.15.100:6800/2923304710,v1:172.21.15.100:6801/2923304710]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-03-21T10:15:03.990 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:15:03 smithi157 ceph-mon[35581]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-03-21T10:15:03.990 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:15:03 smithi157 ceph-mon[35581]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:15:03.991 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:15:03 smithi157 ceph-mon[35581]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T10:15:03.991 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:15:03 smithi157 ceph-mon[35581]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:15:04.289 DEBUG:teuthology.orchestra.run.smithi100:osd.1> sudo journalctl -f -n 0 -u ceph-3056d51a-e76b-11ee-95cd-87774f69a715@osd.1.service 2024-03-21T10:15:04.294 INFO:tasks.cephadm:Deploying osd.2 on smithi157 with /dev/vg_nvme/lv_4... 2024-03-21T10:15:04.295 DEBUG:teuthology.orchestra.run.smithi157:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2edfee68ad7243bed422a049b6f5f58619300f11 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3056d51a-e76b-11ee-95cd-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-03-21T10:15:04.739 INFO:teuthology.orchestra.run.smithi157.stderr:Inferring config /var/lib/ceph/3056d51a-e76b-11ee-95cd-87774f69a715/mon.c/config 2024-03-21T10:15:04.740 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:15:04 smithi157 ceph-mon[35581]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-03-21T10:15:04.740 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:15:04 smithi157 ceph-mon[35581]: from='osd.1 [v2:172.21.15.100:6800/2923304710,v1:172.21.15.100:6801/2923304710]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi100", "root=default"]}]: dispatch 2024-03-21T10:15:04.740 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:15:04 smithi157 ceph-mon[35581]: osdmap e11: 2 total, 1 up, 2 in 2024-03-21T10:15:04.740 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:15:04 smithi157 ceph-mon[35581]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-21T10:15:04.740 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:15:04 smithi157 ceph-mon[35581]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi100", "root=default"]}]: dispatch 2024-03-21T10:15:04.832 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:15:04 smithi100 ceph-mon[35193]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-03-21T10:15:04.832 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:15:04 smithi100 ceph-mon[35193]: from='osd.1 [v2:172.21.15.100:6800/2923304710,v1:172.21.15.100:6801/2923304710]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi100", "root=default"]}]: dispatch 2024-03-21T10:15:04.833 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:15:04 smithi100 ceph-mon[35193]: osdmap e11: 2 total, 1 up, 2 in 2024-03-21T10:15:04.833 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:15:04 smithi100 ceph-mon[35193]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-21T10:15:04.833 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:15:04 smithi100 ceph-mon[35193]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi100", "root=default"]}]: dispatch 2024-03-21T10:15:04.964 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:15:04 smithi047 ceph-mon[31547]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-03-21T10:15:04.964 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:15:04 smithi047 ceph-mon[31547]: from='osd.1 [v2:172.21.15.100:6800/2923304710,v1:172.21.15.100:6801/2923304710]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi100", "root=default"]}]: dispatch 2024-03-21T10:15:04.964 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:15:04 smithi047 ceph-mon[31547]: osdmap e11: 2 total, 1 up, 2 in 2024-03-21T10:15:04.964 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:15:04 smithi047 ceph-mon[31547]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-21T10:15:04.964 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:15:04 smithi047 ceph-mon[31547]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi100", "root=default"]}]: dispatch 2024-03-21T10:15:05.674 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:15:05 smithi100 ceph-mon[35193]: pgmap v80: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-21T10:15:05.674 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:15:05 smithi100 ceph-mon[35193]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi100", "root=default"]}]': finished 2024-03-21T10:15:05.674 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:15:05 smithi100 ceph-mon[35193]: osdmap e12: 2 total, 1 up, 2 in 2024-03-21T10:15:05.674 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:15:05 smithi100 ceph-mon[35193]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-21T10:15:05.674 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:15:05 smithi100 ceph-mon[35193]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-21T10:15:05.674 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:15:05 smithi100 ceph-mon[35193]: from='osd.1 ' entity='osd.1' 2024-03-21T10:15:05.675 INFO:journalctl@ceph.osd.1.smithi100.stdout:Mar 21 10:15:05 smithi100 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-1[39797]: 2024-03-21T10:15:05.348+0000 7fa35f000700 -1 osd.1 0 waiting for initial osdmap 2024-03-21T10:15:05.675 INFO:journalctl@ceph.osd.1.smithi100.stdout:Mar 21 10:15:05 smithi100 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-1[39797]: 2024-03-21T10:15:05.401+0000 7fa354400700 -1 osd.1 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-21T10:15:05.875 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:15:05 smithi157 ceph-mon[35581]: pgmap v80: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-21T10:15:05.876 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:15:05 smithi157 ceph-mon[35581]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi100", "root=default"]}]': finished 2024-03-21T10:15:05.876 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:15:05 smithi157 ceph-mon[35581]: osdmap e12: 2 total, 1 up, 2 in 2024-03-21T10:15:05.876 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:15:05 smithi157 ceph-mon[35581]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-21T10:15:05.876 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:15:05 smithi157 ceph-mon[35581]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-21T10:15:05.876 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:15:05 smithi157 ceph-mon[35581]: from='osd.1 ' entity='osd.1' 2024-03-21T10:15:05.964 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:15:05 smithi047 ceph-mon[31547]: pgmap v80: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-21T10:15:05.964 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:15:05 smithi047 ceph-mon[31547]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi100", "root=default"]}]': finished 2024-03-21T10:15:05.964 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:15:05 smithi047 ceph-mon[31547]: osdmap e12: 2 total, 1 up, 2 in 2024-03-21T10:15:05.964 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:15:05 smithi047 ceph-mon[31547]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-21T10:15:05.964 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:15:05 smithi047 ceph-mon[31547]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-21T10:15:05.964 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:15:05 smithi047 ceph-mon[31547]: from='osd.1 ' entity='osd.1' 2024-03-21T10:15:06.740 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:15:06 smithi157 ceph-mon[35581]: purged_snaps scrub starts 2024-03-21T10:15:06.740 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:15:06 smithi157 ceph-mon[35581]: purged_snaps scrub ok 2024-03-21T10:15:06.740 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:15:06 smithi157 ceph-mon[35581]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-21T10:15:06.740 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:15:06 smithi157 ceph-mon[35581]: osd.1 [v2:172.21.15.100:6800/2923304710,v1:172.21.15.100:6801/2923304710] boot 2024-03-21T10:15:06.740 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:15:06 smithi157 ceph-mon[35581]: osdmap e13: 2 total, 2 up, 2 in 2024-03-21T10:15:06.741 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:15:06 smithi157 ceph-mon[35581]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-21T10:15:06.964 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:15:06 smithi047 ceph-mon[31547]: purged_snaps scrub starts 2024-03-21T10:15:06.964 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:15:06 smithi047 ceph-mon[31547]: purged_snaps scrub ok 2024-03-21T10:15:06.964 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:15:06 smithi047 ceph-mon[31547]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-21T10:15:06.964 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:15:06 smithi047 ceph-mon[31547]: osd.1 [v2:172.21.15.100:6800/2923304710,v1:172.21.15.100:6801/2923304710] boot 2024-03-21T10:15:06.964 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:15:06 smithi047 ceph-mon[31547]: osdmap e13: 2 total, 2 up, 2 in 2024-03-21T10:15:06.964 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:15:06 smithi047 ceph-mon[31547]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-21T10:15:06.980 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:15:06 smithi100 ceph-mon[35193]: purged_snaps scrub starts 2024-03-21T10:15:06.980 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:15:06 smithi100 ceph-mon[35193]: purged_snaps scrub ok 2024-03-21T10:15:06.980 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:15:06 smithi100 ceph-mon[35193]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-21T10:15:06.980 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:15:06 smithi100 ceph-mon[35193]: osd.1 [v2:172.21.15.100:6800/2923304710,v1:172.21.15.100:6801/2923304710] boot 2024-03-21T10:15:06.980 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:15:06 smithi100 ceph-mon[35193]: osdmap e13: 2 total, 2 up, 2 in 2024-03-21T10:15:06.980 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:15:06 smithi100 ceph-mon[35193]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-21T10:15:07.274 INFO:teuthology.orchestra.run.smithi157.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-03-21T10:15:07.274 INFO:teuthology.orchestra.run.smithi157.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-03-21T10:15:07.274 INFO:teuthology.orchestra.run.smithi157.stderr: stderr: 10+0 records in 2024-03-21T10:15:07.274 INFO:teuthology.orchestra.run.smithi157.stderr:10+0 records out 2024-03-21T10:15:07.274 INFO:teuthology.orchestra.run.smithi157.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0317775 s, 330 MB/s 2024-03-21T10:15:07.274 INFO:teuthology.orchestra.run.smithi157.stderr:--> Zapping successful for: 2024-03-21T10:15:07.729 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:15:07 smithi100 ceph-mon[35193]: pgmap v82: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-21T10:15:07.940 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:15:07 smithi157 ceph-mon[35581]: pgmap v82: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-21T10:15:07.963 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:15:07 smithi047 ceph-mon[31547]: pgmap v82: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-21T10:15:08.007 DEBUG:teuthology.orchestra.run.smithi157:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2edfee68ad7243bed422a049b6f5f58619300f11 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3056d51a-e76b-11ee-95cd-87774f69a715 -- ceph orch daemon add osd smithi157:vg_nvme/lv_4 2024-03-21T10:15:08.421 INFO:teuthology.orchestra.run.smithi157.stderr:Inferring config /var/lib/ceph/3056d51a-e76b-11ee-95cd-87774f69a715/mon.c/config 2024-03-21T10:15:08.729 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:15:08 smithi100 ceph-mon[35193]: osdmap e14: 2 total, 2 up, 2 in 2024-03-21T10:15:08.730 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:15:08 smithi100 ceph-mon[35193]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:15:08.730 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:15:08 smithi100 ceph-mon[35193]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:15:08.730 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:15:08 smithi100 ceph-mon[35193]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-21T10:15:08.853 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:15:08 smithi157 ceph-mon[35581]: osdmap e14: 2 total, 2 up, 2 in 2024-03-21T10:15:08.853 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:15:08 smithi157 ceph-mon[35581]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:15:08.853 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:15:08 smithi157 ceph-mon[35581]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:15:08.853 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:15:08 smithi157 ceph-mon[35581]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-21T10:15:08.964 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:15:08 smithi047 ceph-mon[31547]: osdmap e14: 2 total, 2 up, 2 in 2024-03-21T10:15:08.964 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:15:08 smithi047 ceph-mon[31547]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:15:08.964 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:15:08 smithi047 ceph-mon[31547]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:15:08.964 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:15:08 smithi047 ceph-mon[31547]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-21T10:15:09.680 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:15:09 smithi157 ceph-mon[35581]: pgmap v85: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-21T10:15:09.680 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:15:09 smithi157 ceph-mon[35581]: Detected new or changed devices on smithi100 2024-03-21T10:15:09.680 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:15:09 smithi157 ceph-mon[35581]: Adjusting osd_memory_target on smithi100 to 17090M 2024-03-21T10:15:09.680 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:15:09 smithi157 ceph-mon[35581]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:15:09.680 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:15:09 smithi157 ceph-mon[35581]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T10:15:09.680 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:15:09 smithi157 ceph-mon[35581]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T10:15:09.680 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:15:09 smithi157 ceph-mon[35581]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:15:09.964 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:15:09 smithi047 ceph-mon[31547]: pgmap v85: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-21T10:15:09.964 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:15:09 smithi047 ceph-mon[31547]: Detected new or changed devices on smithi100 2024-03-21T10:15:09.964 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:15:09 smithi047 ceph-mon[31547]: Adjusting osd_memory_target on smithi100 to 17090M 2024-03-21T10:15:09.964 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:15:09 smithi047 ceph-mon[31547]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:15:09.964 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:15:09 smithi047 ceph-mon[31547]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T10:15:09.964 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:15:09 smithi047 ceph-mon[31547]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T10:15:09.964 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:15:09 smithi047 ceph-mon[31547]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:15:09.979 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:15:09 smithi100 ceph-mon[35193]: pgmap v85: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-21T10:15:09.979 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:15:09 smithi100 ceph-mon[35193]: Detected new or changed devices on smithi100 2024-03-21T10:15:09.979 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:15:09 smithi100 ceph-mon[35193]: Adjusting osd_memory_target on smithi100 to 17090M 2024-03-21T10:15:09.980 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:15:09 smithi100 ceph-mon[35193]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:15:09.980 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:15:09 smithi100 ceph-mon[35193]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T10:15:09.980 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:15:09 smithi100 ceph-mon[35193]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T10:15:09.980 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:15:09 smithi100 ceph-mon[35193]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:15:10.420 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-21T10:15:10.418+0000 7f878c600700 1 -- 172.21.15.157:0/2510955942 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8784105610 msgr2=0x7f8784105a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:15:10.420 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-21T10:15:10.418+0000 7f878c600700 1 --2- 172.21.15.157:0/2510955942 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8784105610 0x7f8784105a70 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f8770009990 tx=0x7f8770009ca0 comp rx=0 tx=0).stop 2024-03-21T10:15:10.420 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-21T10:15:10.418+0000 7f878c600700 1 -- 172.21.15.157:0/2510955942 shutdown_connections 2024-03-21T10:15:10.420 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-21T10:15:10.418+0000 7f878c600700 1 --2- 172.21.15.157:0/2510955942 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8784105fb0 0x7f878410c860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:15:10.420 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-21T10:15:10.418+0000 7f878c600700 1 --2- 172.21.15.157:0/2510955942 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8784104410 0x7f8784104830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:15:10.420 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-21T10:15:10.418+0000 7f878c600700 1 --2- 172.21.15.157:0/2510955942 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8784105610 0x7f8784105a70 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:15:10.420 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-21T10:15:10.418+0000 7f878c600700 1 -- 172.21.15.157:0/2510955942 >> 172.21.15.157:0/2510955942 conn(0x7f87840ff9b0 msgr2=0x7f8784101df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:15:10.420 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-21T10:15:10.418+0000 7f878c600700 1 -- 172.21.15.157:0/2510955942 shutdown_connections 2024-03-21T10:15:10.421 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-21T10:15:10.418+0000 7f878c600700 1 -- 172.21.15.157:0/2510955942 wait complete. 2024-03-21T10:15:10.422 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-21T10:15:10.419+0000 7f878c600700 1 Processor -- start 2024-03-21T10:15:10.422 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-21T10:15:10.419+0000 7f878c600700 1 -- start start 2024-03-21T10:15:10.422 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-21T10:15:10.420+0000 7f878c600700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8784104410 0x7f878419a4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:15:10.422 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-21T10:15:10.420+0000 7f878c600700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8784105610 0x7f878419aa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:15:10.422 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-21T10:15:10.420+0000 7f878c600700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8784105fb0 0x7f87841a1a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:15:10.422 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-21T10:15:10.420+0000 7f878c600700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f878410fee0 con 0x7f8784105fb0 2024-03-21T10:15:10.422 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-21T10:15:10.420+0000 7f878c600700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f878410fd60 con 0x7f8784105610 2024-03-21T10:15:10.422 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-21T10:15:10.420+0000 7f878c600700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8784110060 con 0x7f8784104410 2024-03-21T10:15:10.423 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-21T10:15:10.420+0000 7f8789c00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8784104410 0x7f878419a4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:15:10.423 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-21T10:15:10.420+0000 7f8789c00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8784104410 0x7f878419a4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:46060/0 (socket says 172.21.15.157:46060) 2024-03-21T10:15:10.423 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-21T10:15:10.420+0000 7f8789c00700 1 -- 172.21.15.157:0/2707514366 learned_addr learned my addr 172.21.15.157:0/2707514366 (peer_addr_for_me v2:172.21.15.157:0/0) 2024-03-21T10:15:10.423 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-21T10:15:10.421+0000 7f878a600700 1 --2- 172.21.15.157:0/2707514366 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8784105fb0 0x7f87841a1a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:15:10.423 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-21T10:15:10.421+0000 7f8789200700 1 --2- 172.21.15.157:0/2707514366 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8784105610 0x7f878419aa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:15:10.423 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-21T10:15:10.421+0000 7f8789c00700 1 -- 172.21.15.157:0/2707514366 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8784105610 msgr2=0x7f878419aa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:15:10.423 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-21T10:15:10.421+0000 7f8789c00700 1 --2- 172.21.15.157:0/2707514366 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8784105610 0x7f878419aa00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:15:10.423 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-21T10:15:10.421+0000 7f8789c00700 1 -- 172.21.15.157:0/2707514366 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8784105fb0 msgr2=0x7f87841a1a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:15:10.423 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-21T10:15:10.421+0000 7f8789c00700 1 --2- 172.21.15.157:0/2707514366 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8784105fb0 0x7f87841a1a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:15:10.423 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-21T10:15:10.421+0000 7f8789c00700 1 -- 172.21.15.157:0/2707514366 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87841a2120 con 0x7f8784104410 2024-03-21T10:15:10.423 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-21T10:15:10.421+0000 7f8789200700 1 --2- 172.21.15.157:0/2707514366 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8784105610 0x7f878419aa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T10:15:10.424 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-21T10:15:10.421+0000 7f8789c00700 1 --2- 172.21.15.157:0/2707514366 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8784104410 0x7f878419a4c0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f877800c9c0 tx=0x7f877800cd80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:15:10.424 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-21T10:15:10.421+0000 7f8782000700 1 -- 172.21.15.157:0/2707514366 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8778007ac0 con 0x7f8784104410 2024-03-21T10:15:10.424 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-21T10:15:10.421+0000 7f8782000700 1 -- 172.21.15.157:0/2707514366 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f8778007c60 con 0x7f8784104410 2024-03-21T10:15:10.424 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-21T10:15:10.421+0000 7f878c600700 1 -- 172.21.15.157:0/2707514366 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f87841a2410 con 0x7f8784104410 2024-03-21T10:15:10.424 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-21T10:15:10.421+0000 7f8782000700 1 -- 172.21.15.157:0/2707514366 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8778018660 con 0x7f8784104410 2024-03-21T10:15:10.425 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-21T10:15:10.422+0000 7f878c600700 1 -- 172.21.15.157:0/2707514366 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f87841a2950 con 0x7f8784104410 2024-03-21T10:15:10.425 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-21T10:15:10.423+0000 7f878c600700 1 -- 172.21.15.157:0/2707514366 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8784068e50 con 0x7f8784104410 2024-03-21T10:15:10.426 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-21T10:15:10.425+0000 7f8782000700 1 -- 172.21.15.157:0/2707514366 <== mon.1 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 15) v1 ==== 137798+0+0 (secure 0 0 0) 0x7f877800f450 con 0x7f8784104410 2024-03-21T10:15:10.428 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-21T10:15:10.426+0000 7f8782000700 1 --2- 172.21.15.157:0/2707514366 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f876c0a2f20 0x7f876c0a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:15:10.428 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-21T10:15:10.426+0000 7f8782000700 1 -- 172.21.15.157:0/2707514366 <== mon.1 v2:172.21.15.157:3300/0 5 ==== osd_map(14..14 src has 1..14) v4 ==== 2309+0+0 (secure 0 0 0) 0x7f87780cddf0 con 0x7f8784104410 2024-03-21T10:15:10.428 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-21T10:15:10.427+0000 7f8789200700 1 --2- 172.21.15.157:0/2707514366 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f876c0a2f20 0x7f876c0a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:15:10.429 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-21T10:15:10.428+0000 7f8789200700 1 --2- 172.21.15.157:0/2707514366 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f876c0a2f20 0x7f876c0a53e0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f877000b5c0 tx=0x7f8770005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:15:10.431 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-21T10:15:10.430+0000 7f8782000700 1 -- 172.21.15.157:0/2707514366 <== mon.1 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f8778014030 con 0x7f8784104410 2024-03-21T10:15:10.666 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-21T10:15:10.664+0000 7f878c600700 1 -- 172.21.15.157:0/2707514366 --> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi157:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) v1 -- 0x7f87840625d0 con 0x7f876c0a2f20 2024-03-21T10:15:11.915 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:15:11 smithi157 ceph-mon[35581]: pgmap v86: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-21T10:15:11.915 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:15:11 smithi157 ceph-mon[35581]: from='client.24154 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi157:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:15:11.915 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:15:11 smithi157 ceph-mon[35581]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-21T10:15:11.915 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:15:11 smithi157 ceph-mon[35581]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-21T10:15:11.915 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:15:11 smithi157 ceph-mon[35581]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T10:15:11.964 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:15:11 smithi047 ceph-mon[31547]: pgmap v86: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-21T10:15:11.964 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:15:11 smithi047 ceph-mon[31547]: from='client.24154 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi157:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:15:11.964 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:15:11 smithi047 ceph-mon[31547]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-21T10:15:11.964 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:15:11 smithi047 ceph-mon[31547]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-21T10:15:11.964 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:15:11 smithi047 ceph-mon[31547]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T10:15:11.979 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:15:11 smithi100 ceph-mon[35193]: pgmap v86: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-21T10:15:11.979 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:15:11 smithi100 ceph-mon[35193]: from='client.24154 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi157:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:15:11.979 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:15:11 smithi100 ceph-mon[35193]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-21T10:15:11.980 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:15:11 smithi100 ceph-mon[35193]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-21T10:15:11.980 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:15:11 smithi100 ceph-mon[35193]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T10:15:13.964 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:15:13 smithi047 ceph-mon[31547]: pgmap v87: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-21T10:15:13.979 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:15:13 smithi100 ceph-mon[35193]: pgmap v87: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-21T10:15:13.990 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:15:13 smithi157 ceph-mon[35581]: pgmap v87: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-21T10:15:14.867 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:15:14 smithi157 ceph-mon[35581]: from='client.? 172.21.15.157:0/1991379465' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "88ce76b9-6081-4836-aad0-8c249c56e8a1"}]: dispatch 2024-03-21T10:15:14.867 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:15:14 smithi157 ceph-mon[35581]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "88ce76b9-6081-4836-aad0-8c249c56e8a1"}]: dispatch 2024-03-21T10:15:14.868 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:15:14 smithi157 ceph-mon[35581]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "88ce76b9-6081-4836-aad0-8c249c56e8a1"}]': finished 2024-03-21T10:15:14.868 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:15:14 smithi157 ceph-mon[35581]: osdmap e15: 3 total, 2 up, 3 in 2024-03-21T10:15:14.868 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:15:14 smithi157 ceph-mon[35581]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-21T10:15:14.964 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:15:14 smithi047 ceph-mon[31547]: from='client.? 172.21.15.157:0/1991379465' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "88ce76b9-6081-4836-aad0-8c249c56e8a1"}]: dispatch 2024-03-21T10:15:14.964 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:15:14 smithi047 ceph-mon[31547]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "88ce76b9-6081-4836-aad0-8c249c56e8a1"}]: dispatch 2024-03-21T10:15:14.964 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:15:14 smithi047 ceph-mon[31547]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "88ce76b9-6081-4836-aad0-8c249c56e8a1"}]': finished 2024-03-21T10:15:14.964 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:15:14 smithi047 ceph-mon[31547]: osdmap e15: 3 total, 2 up, 3 in 2024-03-21T10:15:14.964 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:15:14 smithi047 ceph-mon[31547]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-21T10:15:14.979 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:15:14 smithi100 ceph-mon[35193]: from='client.? 172.21.15.157:0/1991379465' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "88ce76b9-6081-4836-aad0-8c249c56e8a1"}]: dispatch 2024-03-21T10:15:14.979 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:15:14 smithi100 ceph-mon[35193]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "88ce76b9-6081-4836-aad0-8c249c56e8a1"}]: dispatch 2024-03-21T10:15:14.979 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:15:14 smithi100 ceph-mon[35193]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "88ce76b9-6081-4836-aad0-8c249c56e8a1"}]': finished 2024-03-21T10:15:14.980 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:15:14 smithi100 ceph-mon[35193]: osdmap e15: 3 total, 2 up, 3 in 2024-03-21T10:15:14.980 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:15:14 smithi100 ceph-mon[35193]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-21T10:15:15.964 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:15:15 smithi047 ceph-mon[31547]: pgmap v88: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-21T10:15:15.964 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:15:15 smithi047 ceph-mon[31547]: from='client.? 172.21.15.157:0/1453824526' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-21T10:15:15.979 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:15:15 smithi100 ceph-mon[35193]: pgmap v88: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-21T10:15:15.979 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:15:15 smithi100 ceph-mon[35193]: from='client.? 172.21.15.157:0/1453824526' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-21T10:15:15.990 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:15:15 smithi157 ceph-mon[35581]: pgmap v88: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-21T10:15:15.990 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:15:15 smithi157 ceph-mon[35581]: from='client.? 172.21.15.157:0/1453824526' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-21T10:15:17.963 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:15:17 smithi047 ceph-mon[31547]: pgmap v90: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-21T10:15:17.979 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:15:17 smithi100 ceph-mon[35193]: pgmap v90: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-21T10:15:17.989 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:15:17 smithi157 ceph-mon[35581]: pgmap v90: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-21T10:15:19.964 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:15:19 smithi047 ceph-mon[31547]: pgmap v91: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-21T10:15:19.979 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:15:19 smithi100 ceph-mon[35193]: pgmap v91: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-21T10:15:19.989 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:15:19 smithi157 ceph-mon[35581]: pgmap v91: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-21T10:15:21.813 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:15:21 smithi157 ceph-mon[35581]: pgmap v92: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-21T10:15:21.963 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:15:21 smithi047 ceph-mon[31547]: pgmap v92: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-21T10:15:21.979 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:15:21 smithi100 ceph-mon[35193]: pgmap v92: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-21T10:15:23.964 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:15:23 smithi047 ceph-mon[31547]: pgmap v93: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-21T10:15:23.964 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:15:23 smithi047 ceph-mon[31547]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-03-21T10:15:23.964 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:15:23 smithi047 ceph-mon[31547]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T10:15:23.979 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:15:23 smithi100 ceph-mon[35193]: pgmap v93: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-21T10:15:23.979 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:15:23 smithi100 ceph-mon[35193]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-03-21T10:15:23.979 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:15:23 smithi100 ceph-mon[35193]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T10:15:23.989 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:15:23 smithi157 ceph-mon[35581]: pgmap v93: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-21T10:15:23.989 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:15:23 smithi157 ceph-mon[35581]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-03-21T10:15:23.989 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:15:23 smithi157 ceph-mon[35581]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T10:15:24.963 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:15:24 smithi047 ceph-mon[31547]: Deploying daemon osd.2 on smithi157 2024-03-21T10:15:24.979 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:15:24 smithi100 ceph-mon[35193]: Deploying daemon osd.2 on smithi157 2024-03-21T10:15:24.989 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:15:24 smithi157 ceph-mon[35581]: Deploying daemon osd.2 on smithi157 2024-03-21T10:15:25.828 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:15:25 smithi157 ceph-mon[35581]: pgmap v94: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-21T10:15:25.963 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:15:25 smithi047 ceph-mon[31547]: pgmap v94: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-21T10:15:25.979 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:15:25 smithi100 ceph-mon[35193]: pgmap v94: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-21T10:15:27.963 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:15:27 smithi047 ceph-mon[31547]: pgmap v95: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-21T10:15:27.972 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:15:27 smithi157 ceph-mon[35581]: pgmap v95: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-21T10:15:27.979 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:15:27 smithi100 ceph-mon[35193]: pgmap v95: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-21T10:15:29.740 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:15:29 smithi157 ceph-mon[35581]: pgmap v96: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-21T10:15:29.964 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:15:29 smithi047 ceph-mon[31547]: pgmap v96: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-21T10:15:29.979 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:15:29 smithi100 ceph-mon[35193]: pgmap v96: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-21T10:15:31.655 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:15:31 smithi157 ceph-mon[35581]: pgmap v97: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-21T10:15:31.655 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:15:31 smithi157 ceph-mon[35581]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T10:15:31.655 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:15:31 smithi157 ceph-mon[35581]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:15:31.656 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:15:31 smithi157 ceph-mon[35581]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:15:31.964 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:15:31 smithi047 ceph-mon[31547]: pgmap v97: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-21T10:15:31.964 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:15:31 smithi047 ceph-mon[31547]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T10:15:31.964 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:15:31 smithi047 ceph-mon[31547]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:15:31.964 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:15:31 smithi047 ceph-mon[31547]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:15:31.979 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:15:31 smithi100 ceph-mon[35193]: pgmap v97: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-21T10:15:31.979 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:15:31 smithi100 ceph-mon[35193]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T10:15:31.979 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:15:31 smithi100 ceph-mon[35193]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:15:31.979 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:15:31 smithi100 ceph-mon[35193]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:15:33.947 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:15:33 smithi157 ceph-mon[35581]: pgmap v98: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-21T10:15:33.963 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:15:33 smithi047 ceph-mon[31547]: pgmap v98: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-21T10:15:33.979 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:15:33 smithi100 ceph-mon[35193]: pgmap v98: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-21T10:15:35.399 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:15:35 smithi157 ceph-mon[35581]: pgmap v99: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-21T10:15:35.399 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:15:35 smithi157 ceph-mon[35581]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:15:35.399 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:15:35 smithi157 ceph-mon[35581]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:15:35.399 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:15:35 smithi157 ceph-mon[35581]: from='osd.2 [v2:172.21.15.157:6800/171377005,v1:172.21.15.157:6801/171377005]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-03-21T10:15:35.399 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:15:35 smithi157 ceph-mon[35581]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-03-21T10:15:35.464 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:15:35 smithi047 ceph-mon[31547]: pgmap v99: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-21T10:15:35.464 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:15:35 smithi047 ceph-mon[31547]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:15:35.464 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:15:35 smithi047 ceph-mon[31547]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:15:35.464 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:15:35 smithi047 ceph-mon[31547]: from='osd.2 [v2:172.21.15.157:6800/171377005,v1:172.21.15.157:6801/171377005]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-03-21T10:15:35.464 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:15:35 smithi047 ceph-mon[31547]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-03-21T10:15:35.479 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:15:35 smithi100 ceph-mon[35193]: pgmap v99: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-21T10:15:35.480 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:15:35 smithi100 ceph-mon[35193]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:15:35.480 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:15:35 smithi100 ceph-mon[35193]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:15:35.480 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:15:35 smithi100 ceph-mon[35193]: from='osd.2 [v2:172.21.15.157:6800/171377005,v1:172.21.15.157:6801/171377005]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-03-21T10:15:35.480 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:15:35 smithi100 ceph-mon[35193]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-03-21T10:15:35.529 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-21T10:15:35.528+0000 7f8782000700 1 -- 172.21.15.157:0/2707514366 <== mgr.14152 v2:172.21.15.47:6800/2823885994 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f87840625d0 con 0x7f876c0a2f20 2024-03-21T10:15:35.530 INFO:teuthology.orchestra.run.smithi157.stdout:Created osd(s) 2 on host 'smithi157' 2024-03-21T10:15:35.533 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-21T10:15:35.532+0000 7f878c600700 1 -- 172.21.15.157:0/2707514366 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f876c0a2f20 msgr2=0x7f876c0a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:15:35.533 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-21T10:15:35.532+0000 7f878c600700 1 --2- 172.21.15.157:0/2707514366 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f876c0a2f20 0x7f876c0a53e0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f877000b5c0 tx=0x7f8770005fb0 comp rx=0 tx=0).stop 2024-03-21T10:15:35.533 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-21T10:15:35.532+0000 7f878c600700 1 -- 172.21.15.157:0/2707514366 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8784104410 msgr2=0x7f878419a4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:15:35.533 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-21T10:15:35.532+0000 7f878c600700 1 --2- 172.21.15.157:0/2707514366 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8784104410 0x7f878419a4c0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f877800c9c0 tx=0x7f877800cd80 comp rx=0 tx=0).stop 2024-03-21T10:15:35.533 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-21T10:15:35.532+0000 7f878c600700 1 -- 172.21.15.157:0/2707514366 shutdown_connections 2024-03-21T10:15:35.533 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-21T10:15:35.532+0000 7f878c600700 1 --2- 172.21.15.157:0/2707514366 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f876c0a2f20 0x7f876c0a53e0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:15:35.533 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-21T10:15:35.532+0000 7f878c600700 1 --2- 172.21.15.157:0/2707514366 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8784105fb0 0x7f87841a1a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:15:35.533 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-21T10:15:35.532+0000 7f878c600700 1 --2- 172.21.15.157:0/2707514366 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8784104410 0x7f878419a4c0 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:15:35.534 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-21T10:15:35.532+0000 7f878c600700 1 --2- 172.21.15.157:0/2707514366 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8784105610 0x7f878419aa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:15:35.534 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-21T10:15:35.532+0000 7f878c600700 1 -- 172.21.15.157:0/2707514366 >> 172.21.15.157:0/2707514366 conn(0x7f87840ff9b0 msgr2=0x7f87841088b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:15:35.534 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-21T10:15:35.533+0000 7f878c600700 1 -- 172.21.15.157:0/2707514366 shutdown_connections 2024-03-21T10:15:35.534 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-21T10:15:35.533+0000 7f878c600700 1 -- 172.21.15.157:0/2707514366 wait complete. 2024-03-21T10:15:36.240 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:15:36 smithi157 ceph-mon[35581]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T10:15:36.240 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:15:36 smithi157 ceph-mon[35581]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T10:15:36.240 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:15:36 smithi157 ceph-mon[35581]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:15:36.240 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:15:36 smithi157 ceph-mon[35581]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:15:36.240 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:15:36 smithi157 ceph-mon[35581]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:15:36.240 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:15:36 smithi157 ceph-mon[35581]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-03-21T10:15:36.240 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:15:36 smithi157 ceph-mon[35581]: osdmap e16: 3 total, 2 up, 3 in 2024-03-21T10:15:36.241 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:15:36 smithi157 ceph-mon[35581]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-21T10:15:36.241 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:15:36 smithi157 ceph-mon[35581]: from='osd.2 [v2:172.21.15.157:6800/171377005,v1:172.21.15.157:6801/171377005]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi157", "root=default"]}]: dispatch 2024-03-21T10:15:36.241 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:15:36 smithi157 ceph-mon[35581]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi157", "root=default"]}]: dispatch 2024-03-21T10:15:36.241 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:15:36 smithi157 ceph-mon[35581]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T10:15:36.348 DEBUG:teuthology.orchestra.run.smithi157:osd.2> sudo journalctl -f -n 0 -u ceph-3056d51a-e76b-11ee-95cd-87774f69a715@osd.2.service 2024-03-21T10:15:36.353 INFO:tasks.cephadm:Deploying osd.3 on smithi047 with /dev/vg_nvme/lv_3... 2024-03-21T10:15:36.353 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2edfee68ad7243bed422a049b6f5f58619300f11 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3056d51a-e76b-11ee-95cd-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-03-21T10:15:36.449 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:15:36 smithi047 ceph-mon[31547]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T10:15:36.449 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:15:36 smithi047 ceph-mon[31547]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T10:15:36.449 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:15:36 smithi047 ceph-mon[31547]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:15:36.449 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:15:36 smithi047 ceph-mon[31547]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:15:36.449 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:15:36 smithi047 ceph-mon[31547]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:15:36.449 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:15:36 smithi047 ceph-mon[31547]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-03-21T10:15:36.449 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:15:36 smithi047 ceph-mon[31547]: osdmap e16: 3 total, 2 up, 3 in 2024-03-21T10:15:36.449 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:15:36 smithi047 ceph-mon[31547]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-21T10:15:36.449 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:15:36 smithi047 ceph-mon[31547]: from='osd.2 [v2:172.21.15.157:6800/171377005,v1:172.21.15.157:6801/171377005]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi157", "root=default"]}]: dispatch 2024-03-21T10:15:36.449 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:15:36 smithi047 ceph-mon[31547]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi157", "root=default"]}]: dispatch 2024-03-21T10:15:36.449 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:15:36 smithi047 ceph-mon[31547]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T10:15:36.480 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:15:36 smithi100 ceph-mon[35193]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T10:15:36.480 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:15:36 smithi100 ceph-mon[35193]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T10:15:36.480 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:15:36 smithi100 ceph-mon[35193]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:15:36.480 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:15:36 smithi100 ceph-mon[35193]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:15:36.481 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:15:36 smithi100 ceph-mon[35193]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:15:36.481 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:15:36 smithi100 ceph-mon[35193]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-03-21T10:15:36.481 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:15:36 smithi100 ceph-mon[35193]: osdmap e16: 3 total, 2 up, 3 in 2024-03-21T10:15:36.481 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:15:36 smithi100 ceph-mon[35193]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-21T10:15:36.481 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:15:36 smithi100 ceph-mon[35193]: from='osd.2 [v2:172.21.15.157:6800/171377005,v1:172.21.15.157:6801/171377005]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi157", "root=default"]}]: dispatch 2024-03-21T10:15:36.481 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:15:36 smithi100 ceph-mon[35193]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi157", "root=default"]}]: dispatch 2024-03-21T10:15:36.481 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:15:36 smithi100 ceph-mon[35193]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T10:15:36.836 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/3056d51a-e76b-11ee-95cd-87774f69a715/mon.a/config 2024-03-21T10:15:37.674 INFO:journalctl@ceph.osd.2.smithi157.stdout:Mar 21 10:15:37 smithi157 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-2[40607]: 2024-03-21T10:15:37.482+0000 7f4f76e00700 -1 osd.2 0 waiting for initial osdmap 2024-03-21T10:15:37.674 INFO:journalctl@ceph.osd.2.smithi157.stdout:Mar 21 10:15:37 smithi157 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-2[40607]: 2024-03-21T10:15:37.503+0000 7f4f6cc00700 -1 osd.2 17 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-21T10:15:37.964 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:15:37 smithi047 ceph-mon[31547]: pgmap v101: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-21T10:15:37.964 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:15:37 smithi047 ceph-mon[31547]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi157", "root=default"]}]': finished 2024-03-21T10:15:37.964 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:15:37 smithi047 ceph-mon[31547]: osdmap e17: 3 total, 2 up, 3 in 2024-03-21T10:15:37.964 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:15:37 smithi047 ceph-mon[31547]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-21T10:15:37.965 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:15:37 smithi047 ceph-mon[31547]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-21T10:15:37.965 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:15:37 smithi047 ceph-mon[31547]: from='osd.2 ' entity='osd.2' 2024-03-21T10:15:37.979 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:15:37 smithi100 ceph-mon[35193]: pgmap v101: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-21T10:15:37.979 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:15:37 smithi100 ceph-mon[35193]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi157", "root=default"]}]': finished 2024-03-21T10:15:37.979 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:15:37 smithi100 ceph-mon[35193]: osdmap e17: 3 total, 2 up, 3 in 2024-03-21T10:15:37.979 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:15:37 smithi100 ceph-mon[35193]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-21T10:15:37.979 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:15:37 smithi100 ceph-mon[35193]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-21T10:15:37.980 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:15:37 smithi100 ceph-mon[35193]: from='osd.2 ' entity='osd.2' 2024-03-21T10:15:37.990 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:15:37 smithi157 ceph-mon[35581]: pgmap v101: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-21T10:15:37.990 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:15:37 smithi157 ceph-mon[35581]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi157", "root=default"]}]': finished 2024-03-21T10:15:37.990 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:15:37 smithi157 ceph-mon[35581]: osdmap e17: 3 total, 2 up, 3 in 2024-03-21T10:15:37.990 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:15:37 smithi157 ceph-mon[35581]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-21T10:15:37.990 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:15:37 smithi157 ceph-mon[35581]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-21T10:15:37.990 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:15:37 smithi157 ceph-mon[35581]: from='osd.2 ' entity='osd.2' 2024-03-21T10:15:38.740 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:15:38 smithi157 ceph-mon[35581]: purged_snaps scrub starts 2024-03-21T10:15:38.741 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:15:38 smithi157 ceph-mon[35581]: purged_snaps scrub ok 2024-03-21T10:15:38.741 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:15:38 smithi157 ceph-mon[35581]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-21T10:15:38.741 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:15:38 smithi157 ceph-mon[35581]: osd.2 [v2:172.21.15.157:6800/171377005,v1:172.21.15.157:6801/171377005] boot 2024-03-21T10:15:38.741 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:15:38 smithi157 ceph-mon[35581]: osdmap e18: 3 total, 3 up, 3 in 2024-03-21T10:15:38.741 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:15:38 smithi157 ceph-mon[35581]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-21T10:15:38.784 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:15:38 smithi047 ceph-mon[31547]: purged_snaps scrub starts 2024-03-21T10:15:38.785 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:15:38 smithi047 ceph-mon[31547]: purged_snaps scrub ok 2024-03-21T10:15:38.785 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:15:38 smithi047 ceph-mon[31547]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-21T10:15:38.785 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:15:38 smithi047 ceph-mon[31547]: osd.2 [v2:172.21.15.157:6800/171377005,v1:172.21.15.157:6801/171377005] boot 2024-03-21T10:15:38.785 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:15:38 smithi047 ceph-mon[31547]: osdmap e18: 3 total, 3 up, 3 in 2024-03-21T10:15:38.785 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:15:38 smithi047 ceph-mon[31547]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-21T10:15:38.979 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:15:38 smithi100 ceph-mon[35193]: purged_snaps scrub starts 2024-03-21T10:15:38.980 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:15:38 smithi100 ceph-mon[35193]: purged_snaps scrub ok 2024-03-21T10:15:38.980 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:15:38 smithi100 ceph-mon[35193]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-21T10:15:38.980 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:15:38 smithi100 ceph-mon[35193]: osd.2 [v2:172.21.15.157:6800/171377005,v1:172.21.15.157:6801/171377005] boot 2024-03-21T10:15:38.980 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:15:38 smithi100 ceph-mon[35193]: osdmap e18: 3 total, 3 up, 3 in 2024-03-21T10:15:38.980 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:15:38 smithi100 ceph-mon[35193]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-21T10:15:39.806 INFO:teuthology.orchestra.run.smithi047.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-03-21T10:15:39.806 INFO:teuthology.orchestra.run.smithi047.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-03-21T10:15:39.806 INFO:teuthology.orchestra.run.smithi047.stderr: stderr: 10+0 records in 2024-03-21T10:15:39.806 INFO:teuthology.orchestra.run.smithi047.stderr:10+0 records out 2024-03-21T10:15:39.806 INFO:teuthology.orchestra.run.smithi047.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0386425 s, 271 MB/s 2024-03-21T10:15:39.806 INFO:teuthology.orchestra.run.smithi047.stderr:--> Zapping successful for: 2024-03-21T10:15:39.964 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:15:39 smithi047 ceph-mon[31547]: pgmap v103: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-21T10:15:39.979 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:15:39 smithi100 ceph-mon[35193]: pgmap v103: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-21T10:15:39.989 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:15:39 smithi157 ceph-mon[35581]: pgmap v103: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-21T10:15:40.707 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2edfee68ad7243bed422a049b6f5f58619300f11 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3056d51a-e76b-11ee-95cd-87774f69a715 -- ceph orch daemon add osd smithi047:vg_nvme/lv_3 2024-03-21T10:15:40.963 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:15:40 smithi047 ceph-mon[31547]: osdmap e19: 3 total, 3 up, 3 in 2024-03-21T10:15:40.963 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:15:40 smithi047 ceph-mon[31547]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-03-21T10:15:40.979 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:15:40 smithi100 ceph-mon[35193]: osdmap e19: 3 total, 3 up, 3 in 2024-03-21T10:15:40.979 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:15:40 smithi100 ceph-mon[35193]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-03-21T10:15:40.990 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:15:40 smithi157 ceph-mon[35581]: osdmap e19: 3 total, 3 up, 3 in 2024-03-21T10:15:40.990 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:15:40 smithi157 ceph-mon[35581]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-03-21T10:15:41.132 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/3056d51a-e76b-11ee-95cd-87774f69a715/mon.a/config 2024-03-21T10:15:41.964 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:15:41 smithi047 ceph-mon[31547]: pgmap v106: 0 pgs: ; 0 B data, 92 MiB used, 268 GiB / 268 GiB avail 2024-03-21T10:15:41.964 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:15:41 smithi047 ceph-mon[31547]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-03-21T10:15:41.964 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:15:41 smithi047 ceph-mon[31547]: osdmap e20: 3 total, 3 up, 3 in 2024-03-21T10:15:41.964 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:15:41 smithi047 ceph-mon[31547]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-03-21T10:15:41.964 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:15:41 smithi047 ceph-mon[31547]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:15:41.964 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:15:41 smithi047 ceph-mon[31547]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:15:41.964 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:15:41 smithi047 ceph-mon[31547]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-21T10:15:41.964 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:15:41 smithi047 ceph-mon[31547]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:15:41.964 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:15:41 smithi047 ceph-mon[31547]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T10:15:41.964 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:15:41 smithi047 ceph-mon[31547]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T10:15:41.965 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:15:41 smithi047 ceph-mon[31547]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:15:41.979 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:15:41 smithi100 ceph-mon[35193]: pgmap v106: 0 pgs: ; 0 B data, 92 MiB used, 268 GiB / 268 GiB avail 2024-03-21T10:15:41.980 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:15:41 smithi100 ceph-mon[35193]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-03-21T10:15:41.980 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:15:41 smithi100 ceph-mon[35193]: osdmap e20: 3 total, 3 up, 3 in 2024-03-21T10:15:41.980 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:15:41 smithi100 ceph-mon[35193]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-03-21T10:15:41.980 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:15:41 smithi100 ceph-mon[35193]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:15:41.980 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:15:41 smithi100 ceph-mon[35193]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:15:41.980 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:15:41 smithi100 ceph-mon[35193]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-21T10:15:41.980 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:15:41 smithi100 ceph-mon[35193]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:15:41.980 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:15:41 smithi100 ceph-mon[35193]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T10:15:41.980 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:15:41 smithi100 ceph-mon[35193]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T10:15:41.980 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:15:41 smithi100 ceph-mon[35193]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:15:41.990 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:15:41 smithi157 ceph-mon[35581]: pgmap v106: 0 pgs: ; 0 B data, 92 MiB used, 268 GiB / 268 GiB avail 2024-03-21T10:15:41.990 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:15:41 smithi157 ceph-mon[35581]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-03-21T10:15:41.990 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:15:41 smithi157 ceph-mon[35581]: osdmap e20: 3 total, 3 up, 3 in 2024-03-21T10:15:41.990 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:15:41 smithi157 ceph-mon[35581]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-03-21T10:15:41.990 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:15:41 smithi157 ceph-mon[35581]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:15:41.990 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:15:41 smithi157 ceph-mon[35581]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:15:41.990 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:15:41 smithi157 ceph-mon[35581]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-21T10:15:41.990 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:15:41 smithi157 ceph-mon[35581]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:15:41.990 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:15:41 smithi157 ceph-mon[35581]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T10:15:41.991 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:15:41 smithi157 ceph-mon[35581]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T10:15:41.991 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:15:41 smithi157 ceph-mon[35581]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:15:42.858 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:15:42 smithi047 ceph-mon[31547]: Detected new or changed devices on smithi157 2024-03-21T10:15:42.858 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:15:42 smithi047 ceph-mon[31547]: Adjusting osd_memory_target on smithi157 to 17091M 2024-03-21T10:15:42.859 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:15:42 smithi047 ceph-mon[31547]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-03-21T10:15:42.859 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:15:42 smithi047 ceph-mon[31547]: osdmap e21: 3 total, 3 up, 3 in 2024-03-21T10:15:42.859 INFO:journalctl@ceph.osd.0.smithi047.stdout:Mar 21 10:15:42 smithi047 sudo[48439]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-03-21T10:15:42.859 INFO:journalctl@ceph.osd.0.smithi047.stdout:Mar 21 10:15:42 smithi047 sudo[48439]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-21T10:15:42.859 INFO:journalctl@ceph.osd.0.smithi047.stdout:Mar 21 10:15:42 smithi047 sudo[48439]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-03-21T10:15:42.859 INFO:journalctl@ceph.osd.0.smithi047.stdout:Mar 21 10:15:42 smithi047 sudo[48439]: pam_unix(sudo:session): session closed for user root 2024-03-21T10:15:42.979 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:15:42 smithi100 ceph-mon[35193]: Detected new or changed devices on smithi157 2024-03-21T10:15:42.979 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:15:42 smithi100 ceph-mon[35193]: Adjusting osd_memory_target on smithi157 to 17091M 2024-03-21T10:15:42.980 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:15:42 smithi100 ceph-mon[35193]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-03-21T10:15:42.980 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:15:42 smithi100 ceph-mon[35193]: osdmap e21: 3 total, 3 up, 3 in 2024-03-21T10:15:42.989 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:15:42 smithi157 ceph-mon[35581]: Detected new or changed devices on smithi157 2024-03-21T10:15:42.990 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:15:42 smithi157 ceph-mon[35581]: Adjusting osd_memory_target on smithi157 to 17091M 2024-03-21T10:15:42.990 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:15:42 smithi157 ceph-mon[35581]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-03-21T10:15:42.990 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:15:42 smithi157 ceph-mon[35581]: osdmap e21: 3 total, 3 up, 3 in 2024-03-21T10:15:43.140 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:15:43.138+0000 7f5a94800700 1 -- 172.21.15.47:0/3917481774 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5a8c101420 msgr2=0x7f5a8c1018a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:15:43.140 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:15:43.138+0000 7f5a94800700 1 --2- 172.21.15.47:0/3917481774 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5a8c101420 0x7f5a8c1018a0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f5a7c009990 tx=0x7f5a7c009ca0 comp rx=0 tx=0).stop 2024-03-21T10:15:43.140 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:15:43.138+0000 7f5a94800700 1 -- 172.21.15.47:0/3917481774 shutdown_connections 2024-03-21T10:15:43.140 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:15:43.138+0000 7f5a94800700 1 --2- 172.21.15.47:0/3917481774 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5a8c101de0 0x7f5a8c102260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:15:43.140 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:15:43.138+0000 7f5a94800700 1 --2- 172.21.15.47:0/3917481774 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5a8c100ac0 0x7f5a8c100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:15:43.140 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:15:43.138+0000 7f5a94800700 1 --2- 172.21.15.47:0/3917481774 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5a8c101420 0x7f5a8c1018a0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:15:43.140 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:15:43.138+0000 7f5a94800700 1 -- 172.21.15.47:0/3917481774 >> 172.21.15.47:0/3917481774 conn(0x7f5a8c0fc660 msgr2=0x7f5a8c0feaa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:15:43.140 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:15:43.138+0000 7f5a94800700 1 -- 172.21.15.47:0/3917481774 shutdown_connections 2024-03-21T10:15:43.140 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:15:43.138+0000 7f5a94800700 1 -- 172.21.15.47:0/3917481774 wait complete. 2024-03-21T10:15:43.142 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:15:43.140+0000 7f5a94800700 1 Processor -- start 2024-03-21T10:15:43.142 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:15:43.140+0000 7f5a94800700 1 -- start start 2024-03-21T10:15:43.142 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:15:43.140+0000 7f5a94800700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5a8c100ac0 0x7f5a8c19a4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:15:43.142 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:15:43.140+0000 7f5a94800700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5a8c101420 0x7f5a8c19aa20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:15:43.142 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:15:43.140+0000 7f5a94800700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5a8c101de0 0x7f5a8c1a1aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:15:43.142 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:15:43.140+0000 7f5a94800700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a8c10fe90 con 0x7f5a8c101420 2024-03-21T10:15:43.142 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:15:43.140+0000 7f5a94800700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a8c10fd10 con 0x7f5a8c100ac0 2024-03-21T10:15:43.142 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:15:43.140+0000 7f5a94800700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a8c110010 con 0x7f5a8c101de0 2024-03-21T10:15:43.142 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:15:43.140+0000 7f5a91400700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5a8c101420 0x7f5a8c19aa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:15:43.143 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:15:43.141+0000 7f5a91400700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5a8c101420 0x7f5a8c19aa20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:38582/0 (socket says 172.21.15.47:38582) 2024-03-21T10:15:43.143 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:15:43.141+0000 7f5a91400700 1 -- 172.21.15.47:0/1604606417 learned_addr learned my addr 172.21.15.47:0/1604606417 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:15:43.143 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:15:43.141+0000 7f5a91e00700 1 --2- 172.21.15.47:0/1604606417 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5a8c100ac0 0x7f5a8c19a4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:15:43.143 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:15:43.141+0000 7f5a92800700 1 --2- 172.21.15.47:0/1604606417 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5a8c101de0 0x7f5a8c1a1aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:15:43.144 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:15:43.141+0000 7f5a91400700 1 -- 172.21.15.47:0/1604606417 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5a8c101de0 msgr2=0x7f5a8c1a1aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:15:43.144 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:15:43.141+0000 7f5a91400700 1 --2- 172.21.15.47:0/1604606417 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5a8c101de0 0x7f5a8c1a1aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:15:43.144 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:15:43.141+0000 7f5a91400700 1 -- 172.21.15.47:0/1604606417 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5a8c100ac0 msgr2=0x7f5a8c19a4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:15:43.144 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:15:43.141+0000 7f5a91400700 1 --2- 172.21.15.47:0/1604606417 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5a8c100ac0 0x7f5a8c19a4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:15:43.144 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:15:43.141+0000 7f5a91400700 1 -- 172.21.15.47:0/1604606417 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a8c1a2140 con 0x7f5a8c101420 2024-03-21T10:15:43.144 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:15:43.142+0000 7f5a91400700 1 --2- 172.21.15.47:0/1604606417 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5a8c101420 0x7f5a8c19aa20 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f5a7c000c00 tx=0x7f5a7c0057c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:15:43.144 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:15:43.142+0000 7f5a86a00700 1 -- 172.21.15.47:0/1604606417 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a7c027070 con 0x7f5a8c101420 2024-03-21T10:15:43.144 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:15:43.142+0000 7f5a86a00700 1 -- 172.21.15.47:0/1604606417 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f5a7c004010 con 0x7f5a8c101420 2024-03-21T10:15:43.144 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:15:43.142+0000 7f5a94800700 1 -- 172.21.15.47:0/1604606417 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5a8c1a23d0 con 0x7f5a8c101420 2024-03-21T10:15:43.144 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:15:43.142+0000 7f5a86a00700 1 -- 172.21.15.47:0/1604606417 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a7c022d60 con 0x7f5a8c101420 2024-03-21T10:15:43.145 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:15:43.142+0000 7f5a94800700 1 -- 172.21.15.47:0/1604606417 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5a8c1a2820 con 0x7f5a8c101420 2024-03-21T10:15:43.145 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:15:43.143+0000 7f5a94800700 1 -- 172.21.15.47:0/1604606417 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5a8c068e50 con 0x7f5a8c101420 2024-03-21T10:15:43.148 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:15:43.147+0000 7f5a86a00700 1 -- 172.21.15.47:0/1604606417 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 15) v1 ==== 137798+0+0 (secure 0 0 0) 0x7f5a7c02a070 con 0x7f5a8c101420 2024-03-21T10:15:43.150 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:15:43.149+0000 7f5a86a00700 1 --2- 172.21.15.47:0/1604606417 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f5a740a2f20 0x7f5a740a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:15:43.150 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:15:43.149+0000 7f5a86a00700 1 -- 172.21.15.47:0/1604606417 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 3237+0+0 (secure 0 0 0) 0x7f5a7c0cf870 con 0x7f5a8c101420 2024-03-21T10:15:43.150 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:15:43.149+0000 7f5a91e00700 1 --2- 172.21.15.47:0/1604606417 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f5a740a2f20 0x7f5a740a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:15:43.151 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:15:43.150+0000 7f5a91e00700 1 --2- 172.21.15.47:0/1604606417 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f5a740a2f20 0x7f5a740a53e0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f5a80005950 tx=0x7f5a8000a300 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:15:43.153 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:15:43.152+0000 7f5a86a00700 1 -- 172.21.15.47:0/1604606417 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f5a7c013070 con 0x7f5a8c101420 2024-03-21T10:15:43.398 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:15:43.397+0000 7f5a94800700 1 -- 172.21.15.47:0/1604606417 --> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi047:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) v1 -- 0x7f5a8c062690 con 0x7f5a740a2f20 2024-03-21T10:15:43.399 INFO:journalctl@ceph.osd.0.smithi047.stdout:Mar 21 10:15:43 smithi047 sudo[48469]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-03-21T10:15:43.399 INFO:journalctl@ceph.osd.0.smithi047.stdout:Mar 21 10:15:43 smithi047 sudo[48469]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-21T10:15:43.399 INFO:journalctl@ceph.osd.0.smithi047.stdout:Mar 21 10:15:43 smithi047 sudo[48469]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-03-21T10:15:43.399 INFO:journalctl@ceph.osd.0.smithi047.stdout:Mar 21 10:15:43 smithi047 sudo[48469]: pam_unix(sudo:session): session closed for user root 2024-03-21T10:15:43.990 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:15:43 smithi157 ceph-mon[35581]: pgmap v109: 1 pgs: 1 creating+peering; 0 B data, 92 MiB used, 268 GiB / 268 GiB avail 2024-03-21T10:15:43.990 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:15:43 smithi157 ceph-mon[35581]: osdmap e22: 3 total, 3 up, 3 in 2024-03-21T10:15:43.990 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:15:43 smithi157 ceph-mon[35581]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:15:43.990 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:15:43 smithi157 ceph-mon[35581]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-21T10:15:43.990 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:15:43 smithi157 ceph-mon[35581]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-21T10:15:43.990 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:15:43 smithi157 ceph-mon[35581]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T10:15:44.213 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:15:43 smithi047 ceph-mon[31547]: pgmap v109: 1 pgs: 1 creating+peering; 0 B data, 92 MiB used, 268 GiB / 268 GiB avail 2024-03-21T10:15:44.213 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:15:43 smithi047 ceph-mon[31547]: osdmap e22: 3 total, 3 up, 3 in 2024-03-21T10:15:44.213 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:15:43 smithi047 ceph-mon[31547]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:15:44.214 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:15:43 smithi047 ceph-mon[31547]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-21T10:15:44.214 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:15:43 smithi047 ceph-mon[31547]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-21T10:15:44.214 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:15:43 smithi047 ceph-mon[31547]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T10:15:44.229 INFO:journalctl@ceph.osd.1.smithi100.stdout:Mar 21 10:15:43 smithi100 sudo[43739]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-03-21T10:15:44.229 INFO:journalctl@ceph.osd.1.smithi100.stdout:Mar 21 10:15:43 smithi100 sudo[43739]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-21T10:15:44.229 INFO:journalctl@ceph.osd.1.smithi100.stdout:Mar 21 10:15:43 smithi100 sudo[43739]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-03-21T10:15:44.230 INFO:journalctl@ceph.osd.1.smithi100.stdout:Mar 21 10:15:43 smithi100 sudo[43739]: pam_unix(sudo:session): session closed for user root 2024-03-21T10:15:44.230 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:15:43 smithi100 ceph-mon[35193]: pgmap v109: 1 pgs: 1 creating+peering; 0 B data, 92 MiB used, 268 GiB / 268 GiB avail 2024-03-21T10:15:44.230 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:15:43 smithi100 ceph-mon[35193]: osdmap e22: 3 total, 3 up, 3 in 2024-03-21T10:15:44.230 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:15:43 smithi100 ceph-mon[35193]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:15:44.230 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:15:43 smithi100 ceph-mon[35193]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-21T10:15:44.230 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:15:43 smithi100 ceph-mon[35193]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-21T10:15:44.230 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:15:43 smithi100 ceph-mon[35193]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T10:15:44.729 INFO:journalctl@ceph.osd.1.smithi100.stdout:Mar 21 10:15:44 smithi100 sudo[43742]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-03-21T10:15:44.729 INFO:journalctl@ceph.osd.1.smithi100.stdout:Mar 21 10:15:44 smithi100 sudo[43742]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-21T10:15:44.730 INFO:journalctl@ceph.osd.1.smithi100.stdout:Mar 21 10:15:44 smithi100 sudo[43742]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-03-21T10:15:44.730 INFO:journalctl@ceph.osd.1.smithi100.stdout:Mar 21 10:15:44 smithi100 sudo[43742]: pam_unix(sudo:session): session closed for user root 2024-03-21T10:15:44.776 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:15:44.775+0000 7f5a86a00700 1 -- 172.21.15.47:0/1604606417 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mgrmap(e 16) v1 ==== 137854+0+0 (secure 0 0 0) 0x7f5a7c099ab0 con 0x7f5a8c101420 2024-03-21T10:15:44.963 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:15:44 smithi047 ceph-mon[31547]: from='client.14280 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi047:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:15:44.964 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:15:44 smithi047 ceph-mon[31547]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:15:45.016 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:15:44 smithi157 ceph-mon[35581]: from='client.14280 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi047:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:15:45.016 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:15:44 smithi157 ceph-mon[35581]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:15:45.017 INFO:journalctl@ceph.osd.2.smithi157.stdout:Mar 21 10:15:44 smithi157 sudo[44781]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-03-21T10:15:45.017 INFO:journalctl@ceph.osd.2.smithi157.stdout:Mar 21 10:15:44 smithi157 sudo[44781]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-21T10:15:45.017 INFO:journalctl@ceph.osd.2.smithi157.stdout:Mar 21 10:15:44 smithi157 sudo[44781]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-03-21T10:15:45.017 INFO:journalctl@ceph.osd.2.smithi157.stdout:Mar 21 10:15:45 smithi157 sudo[44781]: pam_unix(sudo:session): session closed for user root 2024-03-21T10:15:45.229 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:15:44 smithi100 ceph-mon[35193]: from='client.14280 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi047:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:15:45.229 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:15:44 smithi100 ceph-mon[35193]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:15:45.990 INFO:journalctl@ceph.osd.2.smithi157.stdout:Mar 21 10:15:45 smithi157 sudo[44784]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-03-21T10:15:45.990 INFO:journalctl@ceph.osd.2.smithi157.stdout:Mar 21 10:15:45 smithi157 sudo[44784]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-21T10:15:45.990 INFO:journalctl@ceph.osd.2.smithi157.stdout:Mar 21 10:15:45 smithi157 sudo[44784]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-03-21T10:15:45.990 INFO:journalctl@ceph.osd.2.smithi157.stdout:Mar 21 10:15:45 smithi157 sudo[44784]: pam_unix(sudo:session): session closed for user root 2024-03-21T10:15:46.463 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:15:46 smithi047 sudo[48640]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-03-21T10:15:46.464 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:15:46 smithi047 sudo[48640]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-21T10:15:46.464 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:15:46 smithi047 sudo[48640]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-03-21T10:15:46.464 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:15:46 smithi047 sudo[48640]: pam_unix(sudo:session): session closed for user root 2024-03-21T10:15:47.214 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:15:46 smithi047 sudo[48659]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-03-21T10:15:47.215 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:15:46 smithi047 sudo[48659]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-21T10:15:47.215 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:15:46 smithi047 sudo[48659]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-03-21T10:15:47.215 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:15:46 smithi047 sudo[48659]: pam_unix(sudo:session): session closed for user root 2024-03-21T10:15:47.215 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:15:46 smithi047 ceph-mon[31547]: pgmap v111: 1 pgs: 1 creating+peering; 0 B data, 93 MiB used, 268 GiB / 268 GiB avail 2024-03-21T10:15:47.215 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:15:46 smithi047 ceph-mon[31547]: mgrmap e16: x(active, since 3m), standbys: z, y 2024-03-21T10:15:47.215 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:15:46 smithi047 ceph-mon[31547]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:15:47.215 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:15:46 smithi047 ceph-mon[31547]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-03-21T10:15:47.229 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:15:46 smithi100 ceph-mon[35193]: pgmap v111: 1 pgs: 1 creating+peering; 0 B data, 93 MiB used, 268 GiB / 268 GiB avail 2024-03-21T10:15:47.229 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:15:46 smithi100 ceph-mon[35193]: mgrmap e16: x(active, since 3m), standbys: z, y 2024-03-21T10:15:47.229 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:15:46 smithi100 ceph-mon[35193]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:15:47.230 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:15:46 smithi100 ceph-mon[35193]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-03-21T10:15:47.239 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:15:46 smithi157 ceph-mon[35581]: pgmap v111: 1 pgs: 1 creating+peering; 0 B data, 93 MiB used, 268 GiB / 268 GiB avail 2024-03-21T10:15:47.239 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:15:46 smithi157 ceph-mon[35581]: mgrmap e16: x(active, since 3m), standbys: z, y 2024-03-21T10:15:47.240 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:15:46 smithi157 ceph-mon[35581]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:15:47.240 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:15:46 smithi157 ceph-mon[35581]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-03-21T10:15:47.739 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:15:47 smithi157 sudo[44787]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-03-21T10:15:47.740 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:15:47 smithi157 sudo[44787]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-21T10:15:47.740 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:15:47 smithi157 sudo[44787]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-03-21T10:15:47.740 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:15:47 smithi157 sudo[44787]: pam_unix(sudo:session): session closed for user root 2024-03-21T10:15:48.240 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:15:47 smithi157 sudo[44790]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-03-21T10:15:48.240 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:15:47 smithi157 sudo[44790]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-21T10:15:48.240 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:15:47 smithi157 sudo[44790]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-03-21T10:15:48.240 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:15:47 smithi157 sudo[44790]: pam_unix(sudo:session): session closed for user root 2024-03-21T10:15:48.729 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:15:48 smithi100 sudo[43745]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-03-21T10:15:48.729 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:15:48 smithi100 sudo[43745]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-21T10:15:48.729 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:15:48 smithi100 sudo[43745]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-03-21T10:15:48.729 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:15:48 smithi100 sudo[43745]: pam_unix(sudo:session): session closed for user root 2024-03-21T10:15:49.464 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:15:49 smithi047 ceph-mon[31547]: pgmap v112: 1 pgs: 1 creating+peering; 0 B data, 94 MiB used, 268 GiB / 268 GiB avail 2024-03-21T10:15:49.464 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:15:49 smithi047 ceph-mon[31547]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-03-21T10:15:49.464 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:15:49 smithi047 ceph-mon[31547]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-21T10:15:49.464 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:15:49 smithi047 ceph-mon[31547]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-21T10:15:49.464 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:15:49 smithi047 ceph-mon[31547]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-21T10:15:49.464 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:15:49 smithi047 ceph-mon[31547]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-21T10:15:49.465 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:15:49 smithi047 ceph-mon[31547]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-21T10:15:49.465 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:15:49 smithi047 ceph-mon[31547]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-21T10:15:49.465 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:15:49 smithi047 ceph-mon[31547]: from='client.? 172.21.15.47:0/592807835' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "518aa33d-263f-45e6-aab5-2201ceb40e74"}]: dispatch 2024-03-21T10:15:49.480 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:15:49 smithi100 sudo[43748]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-03-21T10:15:49.480 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:15:49 smithi100 sudo[43748]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-21T10:15:49.480 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:15:49 smithi100 sudo[43748]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-03-21T10:15:49.480 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:15:49 smithi100 sudo[43748]: pam_unix(sudo:session): session closed for user root 2024-03-21T10:15:49.480 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:15:49 smithi100 ceph-mon[35193]: pgmap v112: 1 pgs: 1 creating+peering; 0 B data, 94 MiB used, 268 GiB / 268 GiB avail 2024-03-21T10:15:49.480 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:15:49 smithi100 ceph-mon[35193]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-03-21T10:15:49.480 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:15:49 smithi100 ceph-mon[35193]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-21T10:15:49.481 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:15:49 smithi100 ceph-mon[35193]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-21T10:15:49.481 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:15:49 smithi100 ceph-mon[35193]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-21T10:15:49.481 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:15:49 smithi100 ceph-mon[35193]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-21T10:15:49.481 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:15:49 smithi100 ceph-mon[35193]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-21T10:15:49.481 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:15:49 smithi100 ceph-mon[35193]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-21T10:15:49.481 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:15:49 smithi100 ceph-mon[35193]: from='client.? 172.21.15.47:0/592807835' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "518aa33d-263f-45e6-aab5-2201ceb40e74"}]: dispatch 2024-03-21T10:15:49.490 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:15:49 smithi157 ceph-mon[35581]: pgmap v112: 1 pgs: 1 creating+peering; 0 B data, 94 MiB used, 268 GiB / 268 GiB avail 2024-03-21T10:15:49.490 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:15:49 smithi157 ceph-mon[35581]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-03-21T10:15:49.490 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:15:49 smithi157 ceph-mon[35581]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-21T10:15:49.490 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:15:49 smithi157 ceph-mon[35581]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-21T10:15:49.490 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:15:49 smithi157 ceph-mon[35581]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-21T10:15:49.490 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:15:49 smithi157 ceph-mon[35581]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-21T10:15:49.491 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:15:49 smithi157 ceph-mon[35581]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-21T10:15:49.491 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:15:49 smithi157 ceph-mon[35581]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-21T10:15:49.491 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:15:49 smithi157 ceph-mon[35581]: from='client.? 172.21.15.47:0/592807835' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "518aa33d-263f-45e6-aab5-2201ceb40e74"}]: dispatch 2024-03-21T10:15:50.464 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:15:50 smithi047 ceph-mon[31547]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-03-21T10:15:50.464 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:15:50 smithi047 ceph-mon[31547]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 95 MiB used, 268 GiB / 268 GiB avail 2024-03-21T10:15:50.464 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:15:50 smithi047 ceph-mon[31547]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-03-21T10:15:50.464 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:15:50 smithi047 ceph-mon[31547]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-03-21T10:15:50.465 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:15:50 smithi047 ceph-mon[31547]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-21T10:15:50.465 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:15:50 smithi047 ceph-mon[31547]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-21T10:15:50.465 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:15:50 smithi047 ceph-mon[31547]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-21T10:15:50.465 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:15:50 smithi047 ceph-mon[31547]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-03-21T10:15:50.465 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:15:50 smithi047 ceph-mon[31547]: from='client.? 172.21.15.47:0/592807835' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "518aa33d-263f-45e6-aab5-2201ceb40e74"}]': finished 2024-03-21T10:15:50.465 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:15:50 smithi047 ceph-mon[31547]: osdmap e23: 4 total, 3 up, 4 in 2024-03-21T10:15:50.465 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:15:50 smithi047 ceph-mon[31547]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-21T10:15:50.465 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:15:50 smithi047 ceph-mon[31547]: from='client.? 172.21.15.47:0/3299554905' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-21T10:15:50.479 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:15:50 smithi100 ceph-mon[35193]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-03-21T10:15:50.479 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:15:50 smithi100 ceph-mon[35193]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 95 MiB used, 268 GiB / 268 GiB avail 2024-03-21T10:15:50.480 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:15:50 smithi100 ceph-mon[35193]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-03-21T10:15:50.480 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:15:50 smithi100 ceph-mon[35193]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-03-21T10:15:50.480 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:15:50 smithi100 ceph-mon[35193]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-21T10:15:50.480 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:15:50 smithi100 ceph-mon[35193]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-21T10:15:50.480 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:15:50 smithi100 ceph-mon[35193]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-21T10:15:50.480 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:15:50 smithi100 ceph-mon[35193]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-03-21T10:15:50.480 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:15:50 smithi100 ceph-mon[35193]: from='client.? 172.21.15.47:0/592807835' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "518aa33d-263f-45e6-aab5-2201ceb40e74"}]': finished 2024-03-21T10:15:50.480 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:15:50 smithi100 ceph-mon[35193]: osdmap e23: 4 total, 3 up, 4 in 2024-03-21T10:15:50.480 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:15:50 smithi100 ceph-mon[35193]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-21T10:15:50.480 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:15:50 smithi100 ceph-mon[35193]: from='client.? 172.21.15.47:0/3299554905' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-21T10:15:50.490 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:15:50 smithi157 ceph-mon[35581]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-03-21T10:15:50.490 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:15:50 smithi157 ceph-mon[35581]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 95 MiB used, 268 GiB / 268 GiB avail 2024-03-21T10:15:50.490 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:15:50 smithi157 ceph-mon[35581]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-03-21T10:15:50.490 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:15:50 smithi157 ceph-mon[35581]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-03-21T10:15:50.491 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:15:50 smithi157 ceph-mon[35581]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-21T10:15:50.491 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:15:50 smithi157 ceph-mon[35581]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-21T10:15:50.491 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:15:50 smithi157 ceph-mon[35581]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-21T10:15:50.491 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:15:50 smithi157 ceph-mon[35581]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-03-21T10:15:50.491 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:15:50 smithi157 ceph-mon[35581]: from='client.? 172.21.15.47:0/592807835' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "518aa33d-263f-45e6-aab5-2201ceb40e74"}]': finished 2024-03-21T10:15:50.491 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:15:50 smithi157 ceph-mon[35581]: osdmap e23: 4 total, 3 up, 4 in 2024-03-21T10:15:50.491 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:15:50 smithi157 ceph-mon[35581]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-21T10:15:50.491 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:15:50 smithi157 ceph-mon[35581]: from='client.? 172.21.15.47:0/3299554905' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-21T10:15:51.464 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:15:51 smithi047 ceph-mon[31547]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-03-21T10:15:51.479 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:15:51 smithi100 ceph-mon[35193]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-03-21T10:15:51.489 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:15:51 smithi157 ceph-mon[35581]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-03-21T10:15:53.464 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:15:53 smithi047 ceph-mon[31547]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 95 MiB used, 268 GiB / 268 GiB avail; 2.5 KiB/s rd, 65 KiB/s wr, 6 op/s 2024-03-21T10:15:53.479 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:15:53 smithi100 ceph-mon[35193]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 95 MiB used, 268 GiB / 268 GiB avail; 2.5 KiB/s rd, 65 KiB/s wr, 6 op/s 2024-03-21T10:15:53.489 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:15:53 smithi157 ceph-mon[35581]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 95 MiB used, 268 GiB / 268 GiB avail; 2.5 KiB/s rd, 65 KiB/s wr, 6 op/s 2024-03-21T10:15:55.429 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:15:55 smithi047 ceph-mon[31547]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 94 MiB used, 268 GiB / 268 GiB avail; 1.7 KiB/s rd, 44 KiB/s wr, 4 op/s 2024-03-21T10:15:55.479 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:15:55 smithi100 ceph-mon[35193]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 94 MiB used, 268 GiB / 268 GiB avail; 1.7 KiB/s rd, 44 KiB/s wr, 4 op/s 2024-03-21T10:15:55.489 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:15:55 smithi157 ceph-mon[35581]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 94 MiB used, 268 GiB / 268 GiB avail; 1.7 KiB/s rd, 44 KiB/s wr, 4 op/s 2024-03-21T10:15:57.464 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:15:57 smithi047 ceph-mon[31547]: pgmap v118: 1 pgs: 1 active+clean; 577 KiB data, 94 MiB used, 268 GiB / 268 GiB avail; 1.2 KiB/s rd, 33 KiB/s wr, 3 op/s 2024-03-21T10:15:57.479 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:15:57 smithi100 ceph-mon[35193]: pgmap v118: 1 pgs: 1 active+clean; 577 KiB data, 94 MiB used, 268 GiB / 268 GiB avail; 1.2 KiB/s rd, 33 KiB/s wr, 3 op/s 2024-03-21T10:15:57.489 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:15:57 smithi157 ceph-mon[35581]: pgmap v118: 1 pgs: 1 active+clean; 577 KiB data, 94 MiB used, 268 GiB / 268 GiB avail; 1.2 KiB/s rd, 33 KiB/s wr, 3 op/s 2024-03-21T10:15:59.464 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:15:59 smithi047 ceph-mon[31547]: pgmap v119: 1 pgs: 1 active+clean; 577 KiB data, 94 MiB used, 268 GiB / 268 GiB avail; 1023 B/s rd, 26 KiB/s wr, 2 op/s 2024-03-21T10:15:59.464 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:15:59 smithi047 ceph-mon[31547]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-03-21T10:15:59.464 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:15:59 smithi047 ceph-mon[31547]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T10:15:59.464 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:15:59 smithi047 ceph-mon[31547]: Deploying daemon osd.3 on smithi047 2024-03-21T10:15:59.479 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:15:59 smithi100 ceph-mon[35193]: pgmap v119: 1 pgs: 1 active+clean; 577 KiB data, 94 MiB used, 268 GiB / 268 GiB avail; 1023 B/s rd, 26 KiB/s wr, 2 op/s 2024-03-21T10:15:59.479 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:15:59 smithi100 ceph-mon[35193]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-03-21T10:15:59.479 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:15:59 smithi100 ceph-mon[35193]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T10:15:59.479 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:15:59 smithi100 ceph-mon[35193]: Deploying daemon osd.3 on smithi047 2024-03-21T10:15:59.489 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:15:59 smithi157 ceph-mon[35581]: pgmap v119: 1 pgs: 1 active+clean; 577 KiB data, 94 MiB used, 268 GiB / 268 GiB avail; 1023 B/s rd, 26 KiB/s wr, 2 op/s 2024-03-21T10:15:59.490 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:15:59 smithi157 ceph-mon[35581]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-03-21T10:15:59.490 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:15:59 smithi157 ceph-mon[35581]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T10:15:59.490 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:15:59 smithi157 ceph-mon[35581]: Deploying daemon osd.3 on smithi047 2024-03-21T10:16:01.411 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:16:01 smithi047 ceph-mon[31547]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 94 MiB used, 268 GiB / 268 GiB avail; 941 B/s rd, 24 KiB/s wr, 2 op/s 2024-03-21T10:16:01.479 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:16:01 smithi100 ceph-mon[35193]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 94 MiB used, 268 GiB / 268 GiB avail; 941 B/s rd, 24 KiB/s wr, 2 op/s 2024-03-21T10:16:01.489 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:16:01 smithi157 ceph-mon[35581]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 94 MiB used, 268 GiB / 268 GiB avail; 941 B/s rd, 24 KiB/s wr, 2 op/s 2024-03-21T10:16:03.464 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:16:03 smithi047 ceph-mon[31547]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 94 MiB used, 268 GiB / 268 GiB avail; 853 B/s rd, 22 KiB/s wr, 2 op/s 2024-03-21T10:16:03.479 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:16:03 smithi100 ceph-mon[35193]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 94 MiB used, 268 GiB / 268 GiB avail; 853 B/s rd, 22 KiB/s wr, 2 op/s 2024-03-21T10:16:03.489 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:16:03 smithi157 ceph-mon[35581]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 94 MiB used, 268 GiB / 268 GiB avail; 853 B/s rd, 22 KiB/s wr, 2 op/s 2024-03-21T10:16:05.443 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:16:05 smithi047 ceph-mon[31547]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-03-21T10:16:05.479 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:16:05 smithi100 ceph-mon[35193]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-03-21T10:16:05.489 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:16:05 smithi157 ceph-mon[35581]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-03-21T10:16:07.213 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:16:07 smithi047 ceph-mon[31547]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-03-21T10:16:07.479 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:16:07 smithi100 ceph-mon[35193]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-03-21T10:16:07.489 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:16:07 smithi157 ceph-mon[35581]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-03-21T10:16:08.408 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:16:08 smithi047 ceph-mon[31547]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:16:08.408 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:16:08 smithi047 ceph-mon[31547]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T10:16:08.408 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:16:08 smithi047 ceph-mon[31547]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:16:08.479 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:16:08 smithi100 ceph-mon[35193]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:16:08.479 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:16:08 smithi100 ceph-mon[35193]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T10:16:08.479 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:16:08 smithi100 ceph-mon[35193]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:16:08.489 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:16:08 smithi157 ceph-mon[35581]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:16:08.489 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:16:08 smithi157 ceph-mon[35581]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T10:16:08.489 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:16:08 smithi157 ceph-mon[35581]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:16:09.464 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:16:09 smithi047 ceph-mon[31547]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-03-21T10:16:09.479 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:16:09 smithi100 ceph-mon[35193]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-03-21T10:16:09.489 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:16:09 smithi157 ceph-mon[35581]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-03-21T10:16:11.463 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:16:11 smithi047 ceph-mon[31547]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-03-21T10:16:11.464 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:16:11 smithi047 ceph-mon[31547]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:16:11.464 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:16:11 smithi047 ceph-mon[31547]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:16:11.464 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:16:11 smithi047 ceph-mon[31547]: from='osd.3 [v2:172.21.15.47:6810/640760101,v1:172.21.15.47:6811/640760101]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-03-21T10:16:11.479 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:16:11 smithi100 ceph-mon[35193]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-03-21T10:16:11.479 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:16:11 smithi100 ceph-mon[35193]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:16:11.479 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:16:11 smithi100 ceph-mon[35193]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:16:11.479 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:16:11 smithi100 ceph-mon[35193]: from='osd.3 [v2:172.21.15.47:6810/640760101,v1:172.21.15.47:6811/640760101]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-03-21T10:16:11.489 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:16:11 smithi157 ceph-mon[35581]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-03-21T10:16:11.489 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:16:11 smithi157 ceph-mon[35581]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:16:11.490 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:16:11 smithi157 ceph-mon[35581]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:16:11.490 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:16:11 smithi157 ceph-mon[35581]: from='osd.3 [v2:172.21.15.47:6810/640760101,v1:172.21.15.47:6811/640760101]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-03-21T10:16:12.362 INFO:teuthology.orchestra.run.smithi047.stdout:Created osd(s) 3 on host 'smithi047' 2024-03-21T10:16:12.362 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:16:12.360+0000 7f5a86a00700 1 -- 172.21.15.47:0/1604606417 <== mgr.14152 v2:172.21.15.47:6800/2823885994 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f5a8c062690 con 0x7f5a740a2f20 2024-03-21T10:16:12.364 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:16:12.363+0000 7f5a94800700 1 -- 172.21.15.47:0/1604606417 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f5a740a2f20 msgr2=0x7f5a740a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:16:12.365 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:16:12.363+0000 7f5a94800700 1 --2- 172.21.15.47:0/1604606417 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f5a740a2f20 0x7f5a740a53e0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f5a80005950 tx=0x7f5a8000a300 comp rx=0 tx=0).stop 2024-03-21T10:16:12.365 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:16:12.363+0000 7f5a94800700 1 -- 172.21.15.47:0/1604606417 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5a8c101420 msgr2=0x7f5a8c19aa20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:16:12.365 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:16:12.363+0000 7f5a94800700 1 --2- 172.21.15.47:0/1604606417 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5a8c101420 0x7f5a8c19aa20 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f5a7c000c00 tx=0x7f5a7c0057c0 comp rx=0 tx=0).stop 2024-03-21T10:16:12.365 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:16:12.364+0000 7f5a94800700 1 -- 172.21.15.47:0/1604606417 shutdown_connections 2024-03-21T10:16:12.365 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:16:12.364+0000 7f5a94800700 1 --2- 172.21.15.47:0/1604606417 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f5a740a2f20 0x7f5a740a53e0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:16:12.365 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:16:12.364+0000 7f5a94800700 1 --2- 172.21.15.47:0/1604606417 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5a8c101de0 0x7f5a8c1a1aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:16:12.365 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:16:12.364+0000 7f5a94800700 1 --2- 172.21.15.47:0/1604606417 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5a8c100ac0 0x7f5a8c19a4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:16:12.365 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:16:12.364+0000 7f5a94800700 1 --2- 172.21.15.47:0/1604606417 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5a8c101420 0x7f5a8c19aa20 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:16:12.365 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:16:12.364+0000 7f5a94800700 1 -- 172.21.15.47:0/1604606417 >> 172.21.15.47:0/1604606417 conn(0x7f5a8c0fc660 msgr2=0x7f5a8c0fe160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:16:12.366 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:16:12.364+0000 7f5a94800700 1 -- 172.21.15.47:0/1604606417 shutdown_connections 2024-03-21T10:16:12.366 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:16:12.364+0000 7f5a94800700 1 -- 172.21.15.47:0/1604606417 wait complete. 2024-03-21T10:16:12.961 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:16:12 smithi047 ceph-mon[31547]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-03-21T10:16:13.213 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:16:12 smithi047 ceph-mon[31547]: from='osd.3 [v2:172.21.15.47:6810/640760101,v1:172.21.15.47:6811/640760101]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-03-21T10:16:13.214 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:16:12 smithi047 ceph-mon[31547]: osdmap e24: 4 total, 3 up, 4 in 2024-03-21T10:16:13.214 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:16:12 smithi047 ceph-mon[31547]: from='osd.3 [v2:172.21.15.47:6810/640760101,v1:172.21.15.47:6811/640760101]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi047", "root=default"]}]: dispatch 2024-03-21T10:16:13.214 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:16:12 smithi047 ceph-mon[31547]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-21T10:16:13.214 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:16:12 smithi047 ceph-mon[31547]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T10:16:13.214 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:16:12 smithi047 ceph-mon[31547]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T10:16:13.214 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:16:12 smithi047 ceph-mon[31547]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:16:13.214 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:16:12 smithi047 ceph-mon[31547]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:16:13.214 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:16:12 smithi047 ceph-mon[31547]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:16:13.214 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:16:12 smithi047 ceph-mon[31547]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T10:16:13.230 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:16:12 smithi100 ceph-mon[35193]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-03-21T10:16:13.230 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:16:12 smithi100 ceph-mon[35193]: from='osd.3 [v2:172.21.15.47:6810/640760101,v1:172.21.15.47:6811/640760101]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-03-21T10:16:13.230 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:16:12 smithi100 ceph-mon[35193]: osdmap e24: 4 total, 3 up, 4 in 2024-03-21T10:16:13.230 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:16:12 smithi100 ceph-mon[35193]: from='osd.3 [v2:172.21.15.47:6810/640760101,v1:172.21.15.47:6811/640760101]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi047", "root=default"]}]: dispatch 2024-03-21T10:16:13.230 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:16:12 smithi100 ceph-mon[35193]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-21T10:16:13.230 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:16:12 smithi100 ceph-mon[35193]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T10:16:13.231 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:16:12 smithi100 ceph-mon[35193]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T10:16:13.231 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:16:12 smithi100 ceph-mon[35193]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:16:13.231 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:16:12 smithi100 ceph-mon[35193]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:16:13.231 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:16:12 smithi100 ceph-mon[35193]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:16:13.231 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:16:12 smithi100 ceph-mon[35193]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T10:16:13.240 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:16:12 smithi157 ceph-mon[35581]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-03-21T10:16:13.240 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:16:12 smithi157 ceph-mon[35581]: from='osd.3 [v2:172.21.15.47:6810/640760101,v1:172.21.15.47:6811/640760101]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-03-21T10:16:13.240 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:16:12 smithi157 ceph-mon[35581]: osdmap e24: 4 total, 3 up, 4 in 2024-03-21T10:16:13.240 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:16:12 smithi157 ceph-mon[35581]: from='osd.3 [v2:172.21.15.47:6810/640760101,v1:172.21.15.47:6811/640760101]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi047", "root=default"]}]: dispatch 2024-03-21T10:16:13.240 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:16:12 smithi157 ceph-mon[35581]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-21T10:16:13.240 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:16:12 smithi157 ceph-mon[35581]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T10:16:13.240 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:16:12 smithi157 ceph-mon[35581]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T10:16:13.240 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:16:12 smithi157 ceph-mon[35581]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:16:13.240 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:16:12 smithi157 ceph-mon[35581]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:16:13.240 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:16:12 smithi157 ceph-mon[35581]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:16:13.241 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:16:12 smithi157 ceph-mon[35581]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T10:16:13.252 DEBUG:teuthology.orchestra.run.smithi047:osd.3> sudo journalctl -f -n 0 -u ceph-3056d51a-e76b-11ee-95cd-87774f69a715@osd.3.service 2024-03-21T10:16:13.257 INFO:tasks.cephadm:Deploying osd.4 on smithi100 with /dev/vg_nvme/lv_3... 2024-03-21T10:16:13.257 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2edfee68ad7243bed422a049b6f5f58619300f11 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3056d51a-e76b-11ee-95cd-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-03-21T10:16:13.699 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/3056d51a-e76b-11ee-95cd-87774f69a715/mon.b/config 2024-03-21T10:16:13.867 INFO:journalctl@ceph.osd.3.smithi047.stdout:Mar 21 10:16:13 smithi047 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-3[50720]: 2024-03-21T10:16:13.632+0000 7f7841e00700 -1 osd.3 0 waiting for initial osdmap 2024-03-21T10:16:13.867 INFO:journalctl@ceph.osd.3.smithi047.stdout:Mar 21 10:16:13 smithi047 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-3[50720]: 2024-03-21T10:16:13.654+0000 7f7837c00700 -1 osd.3 26 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-21T10:16:14.175 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:16:13 smithi100 ceph-mon[35193]: from='osd.3 [v2:172.21.15.47:6810/640760101,v1:172.21.15.47:6811/640760101]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi047", "root=default"]}]': finished 2024-03-21T10:16:14.175 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:16:13 smithi100 ceph-mon[35193]: osdmap e25: 4 total, 3 up, 4 in 2024-03-21T10:16:14.175 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:16:13 smithi100 ceph-mon[35193]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-21T10:16:14.176 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:16:13 smithi100 ceph-mon[35193]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-21T10:16:14.176 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:16:13 smithi100 ceph-mon[35193]: osdmap e26: 4 total, 3 up, 4 in 2024-03-21T10:16:14.176 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:16:13 smithi100 ceph-mon[35193]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-21T10:16:14.176 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:16:13 smithi100 ceph-mon[35193]: from='osd.3 [v2:172.21.15.47:6810/640760101,v1:172.21.15.47:6811/640760101]' entity='osd.3' 2024-03-21T10:16:14.214 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:16:13 smithi047 ceph-mon[31547]: from='osd.3 [v2:172.21.15.47:6810/640760101,v1:172.21.15.47:6811/640760101]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi047", "root=default"]}]': finished 2024-03-21T10:16:14.214 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:16:13 smithi047 ceph-mon[31547]: osdmap e25: 4 total, 3 up, 4 in 2024-03-21T10:16:14.214 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:16:13 smithi047 ceph-mon[31547]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-21T10:16:14.215 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:16:13 smithi047 ceph-mon[31547]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-21T10:16:14.215 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:16:13 smithi047 ceph-mon[31547]: osdmap e26: 4 total, 3 up, 4 in 2024-03-21T10:16:14.215 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:16:13 smithi047 ceph-mon[31547]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-21T10:16:14.215 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:16:13 smithi047 ceph-mon[31547]: from='osd.3 [v2:172.21.15.47:6810/640760101,v1:172.21.15.47:6811/640760101]' entity='osd.3' 2024-03-21T10:16:14.240 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:16:13 smithi157 ceph-mon[35581]: from='osd.3 [v2:172.21.15.47:6810/640760101,v1:172.21.15.47:6811/640760101]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi047", "root=default"]}]': finished 2024-03-21T10:16:14.240 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:16:13 smithi157 ceph-mon[35581]: osdmap e25: 4 total, 3 up, 4 in 2024-03-21T10:16:14.240 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:16:13 smithi157 ceph-mon[35581]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-21T10:16:14.240 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:16:13 smithi157 ceph-mon[35581]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-21T10:16:14.240 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:16:13 smithi157 ceph-mon[35581]: osdmap e26: 4 total, 3 up, 4 in 2024-03-21T10:16:14.240 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:16:13 smithi157 ceph-mon[35581]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-21T10:16:14.240 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:16:13 smithi157 ceph-mon[35581]: from='osd.3 [v2:172.21.15.47:6810/640760101,v1:172.21.15.47:6811/640760101]' entity='osd.3' 2024-03-21T10:16:14.980 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:16:14 smithi100 ceph-mon[35193]: purged_snaps scrub starts 2024-03-21T10:16:14.980 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:16:14 smithi100 ceph-mon[35193]: purged_snaps scrub ok 2024-03-21T10:16:14.980 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:16:14 smithi100 ceph-mon[35193]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-03-21T10:16:14.980 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:16:14 smithi100 ceph-mon[35193]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-21T10:16:14.980 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:16:14 smithi100 ceph-mon[35193]: osd.3 [v2:172.21.15.47:6810/640760101,v1:172.21.15.47:6811/640760101] boot 2024-03-21T10:16:14.980 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:16:14 smithi100 ceph-mon[35193]: osdmap e27: 4 total, 4 up, 4 in 2024-03-21T10:16:14.980 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:16:14 smithi100 ceph-mon[35193]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-21T10:16:15.210 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:16:14 smithi047 ceph-mon[31547]: purged_snaps scrub starts 2024-03-21T10:16:15.210 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:16:14 smithi047 ceph-mon[31547]: purged_snaps scrub ok 2024-03-21T10:16:15.210 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:16:14 smithi047 ceph-mon[31547]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-03-21T10:16:15.210 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:16:14 smithi047 ceph-mon[31547]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-21T10:16:15.210 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:16:14 smithi047 ceph-mon[31547]: osd.3 [v2:172.21.15.47:6810/640760101,v1:172.21.15.47:6811/640760101] boot 2024-03-21T10:16:15.210 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:16:14 smithi047 ceph-mon[31547]: osdmap e27: 4 total, 4 up, 4 in 2024-03-21T10:16:15.210 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:16:14 smithi047 ceph-mon[31547]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-21T10:16:15.240 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:16:14 smithi157 ceph-mon[35581]: purged_snaps scrub starts 2024-03-21T10:16:15.240 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:16:14 smithi157 ceph-mon[35581]: purged_snaps scrub ok 2024-03-21T10:16:15.240 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:16:14 smithi157 ceph-mon[35581]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-03-21T10:16:15.240 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:16:14 smithi157 ceph-mon[35581]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-21T10:16:15.240 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:16:14 smithi157 ceph-mon[35581]: osd.3 [v2:172.21.15.47:6810/640760101,v1:172.21.15.47:6811/640760101] boot 2024-03-21T10:16:15.240 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:16:14 smithi157 ceph-mon[35581]: osdmap e27: 4 total, 4 up, 4 in 2024-03-21T10:16:15.240 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:16:14 smithi157 ceph-mon[35581]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-21T10:16:16.655 INFO:teuthology.orchestra.run.smithi100.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-03-21T10:16:16.656 INFO:teuthology.orchestra.run.smithi100.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-03-21T10:16:16.656 INFO:teuthology.orchestra.run.smithi100.stderr: stderr: 10+0 records in 2024-03-21T10:16:16.656 INFO:teuthology.orchestra.run.smithi100.stderr:10+0 records out 2024-03-21T10:16:16.656 INFO:teuthology.orchestra.run.smithi100.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0336749 s, 311 MB/s 2024-03-21T10:16:16.656 INFO:teuthology.orchestra.run.smithi100.stderr:--> Zapping successful for: 2024-03-21T10:16:16.713 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:16:16 smithi047 ceph-mon[31547]: osdmap e28: 4 total, 4 up, 4 in 2024-03-21T10:16:16.724 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:16:16 smithi100 ceph-mon[35193]: osdmap e28: 4 total, 4 up, 4 in 2024-03-21T10:16:16.739 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:16:16 smithi157 ceph-mon[35581]: osdmap e28: 4 total, 4 up, 4 in 2024-03-21T10:16:17.507 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2edfee68ad7243bed422a049b6f5f58619300f11 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3056d51a-e76b-11ee-95cd-87774f69a715 -- ceph orch daemon add osd smithi100:vg_nvme/lv_3 2024-03-21T10:16:17.714 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:16:17 smithi047 ceph-mon[31547]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 125 MiB used, 357 GiB / 358 GiB avail 2024-03-21T10:16:17.714 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:16:17 smithi047 ceph-mon[31547]: osdmap e29: 4 total, 4 up, 4 in 2024-03-21T10:16:17.729 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:16:17 smithi100 ceph-mon[35193]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 125 MiB used, 357 GiB / 358 GiB avail 2024-03-21T10:16:17.729 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:16:17 smithi100 ceph-mon[35193]: osdmap e29: 4 total, 4 up, 4 in 2024-03-21T10:16:17.739 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:16:17 smithi157 ceph-mon[35581]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 125 MiB used, 357 GiB / 358 GiB avail 2024-03-21T10:16:17.740 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:16:17 smithi157 ceph-mon[35581]: osdmap e29: 4 total, 4 up, 4 in 2024-03-21T10:16:17.927 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/3056d51a-e76b-11ee-95cd-87774f69a715/mon.b/config 2024-03-21T10:16:18.730 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:16:18 smithi100 ceph-mon[35193]: osdmap e30: 4 total, 4 up, 4 in 2024-03-21T10:16:18.739 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:16:18 smithi157 ceph-mon[35581]: osdmap e30: 4 total, 4 up, 4 in 2024-03-21T10:16:18.964 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:16:18 smithi047 ceph-mon[31547]: osdmap e30: 4 total, 4 up, 4 in 2024-03-21T10:16:19.624 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:16:19 smithi100 ceph-mon[35193]: pgmap v136: 1 pgs: 1 activating+undersized+degraded+remapped; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail; 2/6 objects degraded (33.333%) 2024-03-21T10:16:19.624 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:16:19 smithi100 ceph-mon[35193]: osdmap e31: 4 total, 4 up, 4 in 2024-03-21T10:16:19.624 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:16:19 smithi100 ceph-mon[35193]: Health check failed: Degraded data redundancy: 2/6 objects degraded (33.333%), 1 pg degraded (PG_DEGRADED) 2024-03-21T10:16:19.624 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:16:19 smithi100 ceph-mon[35193]: Detected new or changed devices on smithi047 2024-03-21T10:16:19.624 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:16:19 smithi100 ceph-mon[35193]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:16:19.624 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:16:19 smithi100 ceph-mon[35193]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:16:19.624 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:16:19 smithi100 ceph-mon[35193]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-21T10:16:19.625 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:16:19 smithi100 ceph-mon[35193]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-21T10:16:19.625 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:16:19 smithi100 ceph-mon[35193]: Adjusting osd_memory_target on smithi047 to 8547M 2024-03-21T10:16:19.625 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:16:19 smithi100 ceph-mon[35193]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:16:19.625 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:16:19 smithi100 ceph-mon[35193]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T10:16:19.625 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:16:19 smithi100 ceph-mon[35193]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T10:16:19.625 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:16:19 smithi100 ceph-mon[35193]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:16:19.740 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:16:19 smithi157 ceph-mon[35581]: pgmap v136: 1 pgs: 1 activating+undersized+degraded+remapped; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail; 2/6 objects degraded (33.333%) 2024-03-21T10:16:19.740 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:16:19 smithi157 ceph-mon[35581]: osdmap e31: 4 total, 4 up, 4 in 2024-03-21T10:16:19.740 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:16:19 smithi157 ceph-mon[35581]: Health check failed: Degraded data redundancy: 2/6 objects degraded (33.333%), 1 pg degraded (PG_DEGRADED) 2024-03-21T10:16:19.740 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:16:19 smithi157 ceph-mon[35581]: Detected new or changed devices on smithi047 2024-03-21T10:16:19.740 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:16:19 smithi157 ceph-mon[35581]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:16:19.740 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:16:19 smithi157 ceph-mon[35581]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:16:19.740 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:16:19 smithi157 ceph-mon[35581]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-21T10:16:19.741 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:16:19 smithi157 ceph-mon[35581]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-21T10:16:19.741 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:16:19 smithi157 ceph-mon[35581]: Adjusting osd_memory_target on smithi047 to 8547M 2024-03-21T10:16:19.741 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:16:19 smithi157 ceph-mon[35581]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:16:19.741 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:16:19 smithi157 ceph-mon[35581]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T10:16:19.741 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:16:19 smithi157 ceph-mon[35581]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T10:16:19.741 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:16:19 smithi157 ceph-mon[35581]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:16:19.867 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:16:19 smithi047 ceph-mon[31547]: pgmap v136: 1 pgs: 1 activating+undersized+degraded+remapped; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail; 2/6 objects degraded (33.333%) 2024-03-21T10:16:19.867 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:16:19 smithi047 ceph-mon[31547]: osdmap e31: 4 total, 4 up, 4 in 2024-03-21T10:16:19.867 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:16:19 smithi047 ceph-mon[31547]: Health check failed: Degraded data redundancy: 2/6 objects degraded (33.333%), 1 pg degraded (PG_DEGRADED) 2024-03-21T10:16:19.867 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:16:19 smithi047 ceph-mon[31547]: Detected new or changed devices on smithi047 2024-03-21T10:16:19.867 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:16:19 smithi047 ceph-mon[31547]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:16:19.867 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:16:19 smithi047 ceph-mon[31547]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:16:19.867 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:16:19 smithi047 ceph-mon[31547]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-21T10:16:19.867 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:16:19 smithi047 ceph-mon[31547]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-21T10:16:19.867 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:16:19 smithi047 ceph-mon[31547]: Adjusting osd_memory_target on smithi047 to 8547M 2024-03-21T10:16:19.868 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:16:19 smithi047 ceph-mon[31547]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:16:19.868 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:16:19 smithi047 ceph-mon[31547]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T10:16:19.868 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:16:19 smithi047 ceph-mon[31547]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T10:16:19.868 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:16:19 smithi047 ceph-mon[31547]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:16:20.096 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-21T10:16:20.094+0000 7f9871600700 1 -- 172.21.15.100:0/2808123351 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f986c1055f0 msgr2=0x7f986c1079e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:16:20.096 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-21T10:16:20.094+0000 7f9871600700 1 --2- 172.21.15.100:0/2808123351 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f986c1055f0 0x7f986c1079e0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f9858009990 tx=0x7f9858009ca0 comp rx=0 tx=0).stop 2024-03-21T10:16:20.096 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-21T10:16:20.094+0000 7f9871600700 1 -- 172.21.15.100:0/2808123351 shutdown_connections 2024-03-21T10:16:20.096 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-21T10:16:20.094+0000 7f9871600700 1 --2- 172.21.15.100:0/2808123351 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f986c107f20 0x7f986c10a390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:16:20.096 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-21T10:16:20.094+0000 7f9871600700 1 --2- 172.21.15.100:0/2808123351 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f986c102cc0 0x7f986c1050b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:16:20.096 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-21T10:16:20.094+0000 7f9871600700 1 --2- 172.21.15.100:0/2808123351 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f986c1055f0 0x7f986c1079e0 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:16:20.096 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-21T10:16:20.094+0000 7f9871600700 1 -- 172.21.15.100:0/2808123351 >> 172.21.15.100:0/2808123351 conn(0x7f986c0fc6f0 msgr2=0x7f986c0feb10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:16:20.096 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-21T10:16:20.094+0000 7f9871600700 1 -- 172.21.15.100:0/2808123351 shutdown_connections 2024-03-21T10:16:20.097 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-21T10:16:20.094+0000 7f9871600700 1 -- 172.21.15.100:0/2808123351 wait complete. 2024-03-21T10:16:20.098 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-21T10:16:20.095+0000 7f9871600700 1 Processor -- start 2024-03-21T10:16:20.098 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-21T10:16:20.096+0000 7f9871600700 1 -- start start 2024-03-21T10:16:20.099 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-21T10:16:20.096+0000 7f9871600700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f986c102cc0 0x7f986c19a4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:16:20.099 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-21T10:16:20.096+0000 7f9871600700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f986c1055f0 0x7f986c19aa10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:16:20.099 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-21T10:16:20.096+0000 7f9871600700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f986c107f20 0x7f986c1a1a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:16:20.099 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-21T10:16:20.096+0000 7f9871600700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f986c10dcd0 con 0x7f986c102cc0 2024-03-21T10:16:20.099 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-21T10:16:20.096+0000 7f9871600700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f986c10db50 con 0x7f986c107f20 2024-03-21T10:16:20.099 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-21T10:16:20.096+0000 7f9871600700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f986c10de50 con 0x7f986c1055f0 2024-03-21T10:16:20.099 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-21T10:16:20.096+0000 7f986b400700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f986c107f20 0x7f986c1a1a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:16:20.099 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-21T10:16:20.096+0000 7f986b400700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f986c107f20 0x7f986c1a1a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:59152/0 (socket says 172.21.15.100:59152) 2024-03-21T10:16:20.099 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-21T10:16:20.096+0000 7f986b400700 1 -- 172.21.15.100:0/2219133674 learned_addr learned my addr 172.21.15.100:0/2219133674 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-21T10:16:20.099 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-21T10:16:20.097+0000 7f986aa00700 1 --2- 172.21.15.100:0/2219133674 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f986c102cc0 0x7f986c19a4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:16:20.099 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-21T10:16:20.097+0000 7f986a000700 1 --2- 172.21.15.100:0/2219133674 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f986c1055f0 0x7f986c19aa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:16:20.099 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-21T10:16:20.097+0000 7f986b400700 1 -- 172.21.15.100:0/2219133674 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f986c1055f0 msgr2=0x7f986c19aa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:16:20.099 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-21T10:16:20.097+0000 7f986b400700 1 --2- 172.21.15.100:0/2219133674 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f986c1055f0 0x7f986c19aa10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:16:20.100 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-21T10:16:20.097+0000 7f986b400700 1 -- 172.21.15.100:0/2219133674 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f986c102cc0 msgr2=0x7f986c19a4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:16:20.100 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-21T10:16:20.097+0000 7f986b400700 1 --2- 172.21.15.100:0/2219133674 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f986c102cc0 0x7f986c19a4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:16:20.100 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-21T10:16:20.097+0000 7f986b400700 1 -- 172.21.15.100:0/2219133674 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f986c1a2130 con 0x7f986c107f20 2024-03-21T10:16:20.100 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-21T10:16:20.097+0000 7f986b400700 1 --2- 172.21.15.100:0/2219133674 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f986c107f20 0x7f986c1a1a90 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f9864007d40 tx=0x7f986400a340 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:16:20.100 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-21T10:16:20.097+0000 7f9863400700 1 -- 172.21.15.100:0/2219133674 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9864016020 con 0x7f986c107f20 2024-03-21T10:16:20.100 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-21T10:16:20.097+0000 7f9863400700 1 -- 172.21.15.100:0/2219133674 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f986400ad40 con 0x7f986c107f20 2024-03-21T10:16:20.100 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-21T10:16:20.097+0000 7f9871600700 1 -- 172.21.15.100:0/2219133674 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f986c1a2420 con 0x7f986c107f20 2024-03-21T10:16:20.100 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-21T10:16:20.097+0000 7f9863400700 1 -- 172.21.15.100:0/2219133674 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9864011660 con 0x7f986c107f20 2024-03-21T10:16:20.100 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-21T10:16:20.098+0000 7f9871600700 1 -- 172.21.15.100:0/2219133674 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f986c1a2910 con 0x7f986c107f20 2024-03-21T10:16:20.101 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-21T10:16:20.099+0000 7f9871600700 1 -- 172.21.15.100:0/2219133674 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f986c068e50 con 0x7f986c107f20 2024-03-21T10:16:20.102 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-21T10:16:20.101+0000 7f9863400700 1 -- 172.21.15.100:0/2219133674 <== mon.2 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 16) v1 ==== 137854+0+0 (secure 0 0 0) 0x7f9864019020 con 0x7f986c107f20 2024-03-21T10:16:20.104 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-21T10:16:20.102+0000 7f9863400700 1 --2- 172.21.15.100:0/2219133674 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f98500a2fb0 0x7f98500a5470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:16:20.104 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-21T10:16:20.102+0000 7f9863400700 1 -- 172.21.15.100:0/2219133674 <== mon.2 v2:172.21.15.100:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 3669+0+0 (secure 0 0 0) 0x7f98640ce690 con 0x7f986c107f20 2024-03-21T10:16:20.104 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-21T10:16:20.103+0000 7f986aa00700 1 --2- 172.21.15.100:0/2219133674 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f98500a2fb0 0x7f98500a5470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:16:20.105 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-21T10:16:20.104+0000 7f986aa00700 1 --2- 172.21.15.100:0/2219133674 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f98500a2fb0 0x7f98500a5470 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f985c005fd0 tx=0x7f985c00a560 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:16:20.108 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-21T10:16:20.107+0000 7f9863400700 1 -- 172.21.15.100:0/2219133674 <== mon.2 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f9864020020 con 0x7f986c107f20 2024-03-21T10:16:20.339 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-21T10:16:20.337+0000 7f9871600700 1 -- 172.21.15.100:0/2219133674 --> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi100:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) v1 -- 0x7f986c0625d0 con 0x7f98500a2fb0 2024-03-21T10:16:20.729 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:16:20 smithi100 ceph-mon[35193]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-21T10:16:20.729 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:16:20 smithi100 ceph-mon[35193]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-21T10:16:20.730 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:16:20 smithi100 ceph-mon[35193]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T10:16:20.964 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:16:20 smithi047 ceph-mon[31547]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-21T10:16:20.964 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:16:20 smithi047 ceph-mon[31547]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-21T10:16:20.964 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:16:20 smithi047 ceph-mon[31547]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T10:16:20.989 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:16:20 smithi157 ceph-mon[35581]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-21T10:16:20.990 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:16:20 smithi157 ceph-mon[35581]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-21T10:16:20.990 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:16:20 smithi157 ceph-mon[35581]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T10:16:21.643 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:16:21 smithi100 ceph-mon[35193]: pgmap v138: 1 pgs: 1 activating+undersized+degraded+remapped; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail; 2/6 objects degraded (33.333%) 2024-03-21T10:16:21.643 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:16:21 smithi100 ceph-mon[35193]: from='client.34214 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi100:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:16:21.963 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:16:21 smithi047 ceph-mon[31547]: pgmap v138: 1 pgs: 1 activating+undersized+degraded+remapped; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail; 2/6 objects degraded (33.333%) 2024-03-21T10:16:21.964 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:16:21 smithi047 ceph-mon[31547]: from='client.34214 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi100:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:16:21.989 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:16:21 smithi157 ceph-mon[35581]: pgmap v138: 1 pgs: 1 activating+undersized+degraded+remapped; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail; 2/6 objects degraded (33.333%) 2024-03-21T10:16:21.989 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:16:21 smithi157 ceph-mon[35581]: from='client.34214 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi100:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:16:22.834 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:16:22 smithi100 ceph-mon[35193]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 2/6 objects degraded (33.333%), 1 pg degraded) 2024-03-21T10:16:22.834 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:16:22 smithi100 ceph-mon[35193]: Cluster is now healthy 2024-03-21T10:16:22.963 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:16:22 smithi047 ceph-mon[31547]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 2/6 objects degraded (33.333%), 1 pg degraded) 2024-03-21T10:16:22.964 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:16:22 smithi047 ceph-mon[31547]: Cluster is now healthy 2024-03-21T10:16:22.989 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:16:22 smithi157 ceph-mon[35581]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 2/6 objects degraded (33.333%), 1 pg degraded) 2024-03-21T10:16:22.989 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:16:22 smithi157 ceph-mon[35581]: Cluster is now healthy 2024-03-21T10:16:23.963 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:16:23 smithi047 ceph-mon[31547]: pgmap v139: 1 pgs: 1 active+clean; 577 KiB data, 125 MiB used, 357 GiB / 358 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-03-21T10:16:23.979 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:16:23 smithi100 ceph-mon[35193]: pgmap v139: 1 pgs: 1 active+clean; 577 KiB data, 125 MiB used, 357 GiB / 358 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-03-21T10:16:23.989 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:16:23 smithi157 ceph-mon[35581]: pgmap v139: 1 pgs: 1 active+clean; 577 KiB data, 125 MiB used, 357 GiB / 358 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-03-21T10:16:24.908 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:16:24 smithi100 ceph-mon[35193]: from='client.? 172.21.15.100:0/4224793563' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "987bde83-ecc6-42e5-bd56-c143b3de4226"}]: dispatch 2024-03-21T10:16:24.908 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:16:24 smithi100 ceph-mon[35193]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "987bde83-ecc6-42e5-bd56-c143b3de4226"}]: dispatch 2024-03-21T10:16:24.908 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:16:24 smithi100 ceph-mon[35193]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "987bde83-ecc6-42e5-bd56-c143b3de4226"}]': finished 2024-03-21T10:16:24.908 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:16:24 smithi100 ceph-mon[35193]: osdmap e32: 5 total, 4 up, 5 in 2024-03-21T10:16:24.909 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:16:24 smithi100 ceph-mon[35193]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-21T10:16:24.964 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:16:24 smithi047 ceph-mon[31547]: from='client.? 172.21.15.100:0/4224793563' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "987bde83-ecc6-42e5-bd56-c143b3de4226"}]: dispatch 2024-03-21T10:16:24.964 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:16:24 smithi047 ceph-mon[31547]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "987bde83-ecc6-42e5-bd56-c143b3de4226"}]: dispatch 2024-03-21T10:16:24.964 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:16:24 smithi047 ceph-mon[31547]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "987bde83-ecc6-42e5-bd56-c143b3de4226"}]': finished 2024-03-21T10:16:24.965 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:16:24 smithi047 ceph-mon[31547]: osdmap e32: 5 total, 4 up, 5 in 2024-03-21T10:16:24.965 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:16:24 smithi047 ceph-mon[31547]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-21T10:16:24.990 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:16:24 smithi157 ceph-mon[35581]: from='client.? 172.21.15.100:0/4224793563' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "987bde83-ecc6-42e5-bd56-c143b3de4226"}]: dispatch 2024-03-21T10:16:24.990 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:16:24 smithi157 ceph-mon[35581]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "987bde83-ecc6-42e5-bd56-c143b3de4226"}]: dispatch 2024-03-21T10:16:24.990 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:16:24 smithi157 ceph-mon[35581]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "987bde83-ecc6-42e5-bd56-c143b3de4226"}]': finished 2024-03-21T10:16:24.990 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:16:24 smithi157 ceph-mon[35581]: osdmap e32: 5 total, 4 up, 5 in 2024-03-21T10:16:24.990 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:16:24 smithi157 ceph-mon[35581]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-21T10:16:25.964 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:16:25 smithi047 ceph-mon[31547]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 125 MiB used, 357 GiB / 358 GiB avail; 77 KiB/s, 0 objects/s recovering 2024-03-21T10:16:25.964 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:16:25 smithi047 ceph-mon[31547]: from='client.? 172.21.15.100:0/1947743959' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-21T10:16:25.979 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:16:25 smithi100 ceph-mon[35193]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 125 MiB used, 357 GiB / 358 GiB avail; 77 KiB/s, 0 objects/s recovering 2024-03-21T10:16:25.979 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:16:25 smithi100 ceph-mon[35193]: from='client.? 172.21.15.100:0/1947743959' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-21T10:16:25.989 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:16:25 smithi157 ceph-mon[35581]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 125 MiB used, 357 GiB / 358 GiB avail; 77 KiB/s, 0 objects/s recovering 2024-03-21T10:16:25.990 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:16:25 smithi157 ceph-mon[35581]: from='client.? 172.21.15.100:0/1947743959' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-21T10:16:27.963 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:16:27 smithi047 ceph-mon[31547]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 125 MiB used, 357 GiB / 358 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-03-21T10:16:27.979 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:16:27 smithi100 ceph-mon[35193]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 125 MiB used, 357 GiB / 358 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-03-21T10:16:27.989 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:16:27 smithi157 ceph-mon[35581]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 125 MiB used, 357 GiB / 358 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-03-21T10:16:29.943 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:16:29 smithi047 ceph-mon[31547]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 125 MiB used, 357 GiB / 358 GiB avail; 61 KiB/s, 0 objects/s recovering 2024-03-21T10:16:29.979 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:16:29 smithi100 ceph-mon[35193]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 125 MiB used, 357 GiB / 358 GiB avail; 61 KiB/s, 0 objects/s recovering 2024-03-21T10:16:29.989 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:16:29 smithi157 ceph-mon[35581]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 125 MiB used, 357 GiB / 358 GiB avail; 61 KiB/s, 0 objects/s recovering 2024-03-21T10:16:31.883 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:16:31 smithi100 ceph-mon[35193]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 125 MiB used, 357 GiB / 358 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-03-21T10:16:31.964 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:16:31 smithi047 ceph-mon[31547]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 125 MiB used, 357 GiB / 358 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-03-21T10:16:31.989 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:16:31 smithi157 ceph-mon[35581]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 125 MiB used, 357 GiB / 358 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-03-21T10:16:33.730 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:16:33 smithi100 ceph-mon[35193]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 125 MiB used, 357 GiB / 358 GiB avail 2024-03-21T10:16:33.730 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:16:33 smithi100 ceph-mon[35193]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-03-21T10:16:33.730 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:16:33 smithi100 ceph-mon[35193]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T10:16:33.964 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:16:33 smithi047 ceph-mon[31547]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 125 MiB used, 357 GiB / 358 GiB avail 2024-03-21T10:16:33.964 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:16:33 smithi047 ceph-mon[31547]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-03-21T10:16:33.964 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:16:33 smithi047 ceph-mon[31547]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T10:16:33.989 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:16:33 smithi157 ceph-mon[35581]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 125 MiB used, 357 GiB / 358 GiB avail 2024-03-21T10:16:33.990 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:16:33 smithi157 ceph-mon[35581]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-03-21T10:16:33.990 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:16:33 smithi157 ceph-mon[35581]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T10:16:34.720 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:16:34 smithi100 ceph-mon[35193]: Deploying daemon osd.4 on smithi100 2024-03-21T10:16:34.964 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:16:34 smithi047 ceph-mon[31547]: Deploying daemon osd.4 on smithi100 2024-03-21T10:16:34.989 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:16:34 smithi157 ceph-mon[35581]: Deploying daemon osd.4 on smithi100 2024-03-21T10:16:35.932 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:16:35 smithi100 ceph-mon[35193]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 125 MiB used, 357 GiB / 358 GiB avail 2024-03-21T10:16:35.963 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:16:35 smithi047 ceph-mon[31547]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 125 MiB used, 357 GiB / 358 GiB avail 2024-03-21T10:16:35.989 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:16:35 smithi157 ceph-mon[35581]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 125 MiB used, 357 GiB / 358 GiB avail 2024-03-21T10:16:37.964 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:16:37 smithi047 ceph-mon[31547]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 125 MiB used, 357 GiB / 358 GiB avail 2024-03-21T10:16:37.980 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:16:37 smithi100 ceph-mon[35193]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 125 MiB used, 357 GiB / 358 GiB avail 2024-03-21T10:16:37.989 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:16:37 smithi157 ceph-mon[35581]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 125 MiB used, 357 GiB / 358 GiB avail 2024-03-21T10:16:39.899 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:16:39 smithi100 ceph-mon[35193]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 125 MiB used, 357 GiB / 358 GiB avail 2024-03-21T10:16:39.963 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:16:39 smithi047 ceph-mon[31547]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 125 MiB used, 357 GiB / 358 GiB avail 2024-03-21T10:16:39.989 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:16:39 smithi157 ceph-mon[35581]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 125 MiB used, 357 GiB / 358 GiB avail 2024-03-21T10:16:41.964 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:16:41 smithi047 ceph-mon[31547]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 125 MiB used, 357 GiB / 358 GiB avail 2024-03-21T10:16:41.964 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:16:41 smithi047 ceph-mon[31547]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:16:41.964 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:16:41 smithi047 ceph-mon[31547]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:16:41.964 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:16:41 smithi047 ceph-mon[31547]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T10:16:41.979 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:16:41 smithi100 ceph-mon[35193]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 125 MiB used, 357 GiB / 358 GiB avail 2024-03-21T10:16:41.979 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:16:41 smithi100 ceph-mon[35193]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:16:41.979 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:16:41 smithi100 ceph-mon[35193]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:16:41.980 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:16:41 smithi100 ceph-mon[35193]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T10:16:41.989 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:16:41 smithi157 ceph-mon[35581]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 125 MiB used, 357 GiB / 358 GiB avail 2024-03-21T10:16:41.989 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:16:41 smithi157 ceph-mon[35581]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:16:41.990 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:16:41 smithi157 ceph-mon[35581]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:16:41.990 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:16:41 smithi157 ceph-mon[35581]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T10:16:43.909 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:16:43 smithi100 ceph-mon[35193]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 125 MiB used, 357 GiB / 358 GiB avail 2024-03-21T10:16:43.963 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:16:43 smithi047 ceph-mon[31547]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 125 MiB used, 357 GiB / 358 GiB avail 2024-03-21T10:16:43.989 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:16:43 smithi157 ceph-mon[35581]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 125 MiB used, 357 GiB / 358 GiB avail 2024-03-21T10:16:45.729 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:16:45 smithi100 ceph-mon[35193]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 125 MiB used, 357 GiB / 358 GiB avail 2024-03-21T10:16:45.729 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:16:45 smithi100 ceph-mon[35193]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:16:45.729 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:16:45 smithi100 ceph-mon[35193]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:16:45.729 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:16:45 smithi100 ceph-mon[35193]: from='osd.4 [v2:172.21.15.100:6808/2785478639,v1:172.21.15.100:6809/2785478639]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-03-21T10:16:45.730 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:16:45 smithi100 ceph-mon[35193]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-03-21T10:16:45.964 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:16:45 smithi047 ceph-mon[31547]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 125 MiB used, 357 GiB / 358 GiB avail 2024-03-21T10:16:45.964 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:16:45 smithi047 ceph-mon[31547]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:16:45.964 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:16:45 smithi047 ceph-mon[31547]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:16:45.964 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:16:45 smithi047 ceph-mon[31547]: from='osd.4 [v2:172.21.15.100:6808/2785478639,v1:172.21.15.100:6809/2785478639]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-03-21T10:16:45.964 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:16:45 smithi047 ceph-mon[31547]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-03-21T10:16:45.990 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:16:45 smithi157 ceph-mon[35581]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 125 MiB used, 357 GiB / 358 GiB avail 2024-03-21T10:16:45.990 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:16:45 smithi157 ceph-mon[35581]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:16:45.990 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:16:45 smithi157 ceph-mon[35581]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:16:45.990 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:16:45 smithi157 ceph-mon[35581]: from='osd.4 [v2:172.21.15.100:6808/2785478639,v1:172.21.15.100:6809/2785478639]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-03-21T10:16:45.990 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:16:45 smithi157 ceph-mon[35581]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-03-21T10:16:46.416 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-21T10:16:46.413+0000 7f9863400700 1 -- 172.21.15.100:0/2219133674 <== mgr.14152 v2:172.21.15.47:6800/2823885994 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f986c0625d0 con 0x7f98500a2fb0 2024-03-21T10:16:46.417 INFO:teuthology.orchestra.run.smithi100.stdout:Created osd(s) 4 on host 'smithi100' 2024-03-21T10:16:46.418 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-21T10:16:46.417+0000 7f9871600700 1 -- 172.21.15.100:0/2219133674 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f98500a2fb0 msgr2=0x7f98500a5470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:16:46.419 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-21T10:16:46.417+0000 7f9871600700 1 --2- 172.21.15.100:0/2219133674 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f98500a2fb0 0x7f98500a5470 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f985c005fd0 tx=0x7f985c00a560 comp rx=0 tx=0).stop 2024-03-21T10:16:46.419 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-21T10:16:46.417+0000 7f9871600700 1 -- 172.21.15.100:0/2219133674 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f986c107f20 msgr2=0x7f986c1a1a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:16:46.419 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-21T10:16:46.417+0000 7f9871600700 1 --2- 172.21.15.100:0/2219133674 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f986c107f20 0x7f986c1a1a90 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f9864007d40 tx=0x7f986400a340 comp rx=0 tx=0).stop 2024-03-21T10:16:46.419 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-21T10:16:46.417+0000 7f9871600700 1 -- 172.21.15.100:0/2219133674 shutdown_connections 2024-03-21T10:16:46.419 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-21T10:16:46.417+0000 7f9871600700 1 --2- 172.21.15.100:0/2219133674 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f98500a2fb0 0x7f98500a5470 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:16:46.419 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-21T10:16:46.417+0000 7f9871600700 1 --2- 172.21.15.100:0/2219133674 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f986c107f20 0x7f986c1a1a90 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:16:46.419 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-21T10:16:46.417+0000 7f9871600700 1 --2- 172.21.15.100:0/2219133674 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f986c102cc0 0x7f986c19a4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:16:46.419 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-21T10:16:46.417+0000 7f9871600700 1 --2- 172.21.15.100:0/2219133674 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f986c1055f0 0x7f986c19aa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:16:46.420 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-21T10:16:46.417+0000 7f9871600700 1 -- 172.21.15.100:0/2219133674 >> 172.21.15.100:0/2219133674 conn(0x7f986c0fc6f0 msgr2=0x7f986c1022d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:16:46.420 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-21T10:16:46.417+0000 7f9871600700 1 -- 172.21.15.100:0/2219133674 shutdown_connections 2024-03-21T10:16:46.420 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-21T10:16:46.417+0000 7f9871600700 1 -- 172.21.15.100:0/2219133674 wait complete. 2024-03-21T10:16:47.193 DEBUG:teuthology.orchestra.run.smithi100:osd.4> sudo journalctl -f -n 0 -u ceph-3056d51a-e76b-11ee-95cd-87774f69a715@osd.4.service 2024-03-21T10:16:47.198 INFO:tasks.cephadm:Deploying osd.5 on smithi157 with /dev/vg_nvme/lv_3... 2024-03-21T10:16:47.199 DEBUG:teuthology.orchestra.run.smithi157:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2edfee68ad7243bed422a049b6f5f58619300f11 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3056d51a-e76b-11ee-95cd-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-03-21T10:16:47.479 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:16:47 smithi100 ceph-mon[35193]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 125 MiB used, 357 GiB / 358 GiB avail 2024-03-21T10:16:47.479 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:16:47 smithi100 ceph-mon[35193]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-03-21T10:16:47.480 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:16:47 smithi100 ceph-mon[35193]: osdmap e33: 5 total, 4 up, 5 in 2024-03-21T10:16:47.480 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:16:47 smithi100 ceph-mon[35193]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-21T10:16:47.480 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:16:47 smithi100 ceph-mon[35193]: from='osd.4 [v2:172.21.15.100:6808/2785478639,v1:172.21.15.100:6809/2785478639]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi100", "root=default"]}]: dispatch 2024-03-21T10:16:47.480 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:16:47 smithi100 ceph-mon[35193]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi100", "root=default"]}]: dispatch 2024-03-21T10:16:47.480 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:16:47 smithi100 ceph-mon[35193]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:16:47.480 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:16:47 smithi100 ceph-mon[35193]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:16:47.490 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:16:47 smithi157 ceph-mon[35581]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 125 MiB used, 357 GiB / 358 GiB avail 2024-03-21T10:16:47.490 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:16:47 smithi157 ceph-mon[35581]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-03-21T10:16:47.490 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:16:47 smithi157 ceph-mon[35581]: osdmap e33: 5 total, 4 up, 5 in 2024-03-21T10:16:47.490 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:16:47 smithi157 ceph-mon[35581]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-21T10:16:47.490 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:16:47 smithi157 ceph-mon[35581]: from='osd.4 [v2:172.21.15.100:6808/2785478639,v1:172.21.15.100:6809/2785478639]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi100", "root=default"]}]: dispatch 2024-03-21T10:16:47.490 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:16:47 smithi157 ceph-mon[35581]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi100", "root=default"]}]: dispatch 2024-03-21T10:16:47.491 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:16:47 smithi157 ceph-mon[35581]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:16:47.491 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:16:47 smithi157 ceph-mon[35581]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:16:47.634 INFO:teuthology.orchestra.run.smithi157.stderr:Inferring config /var/lib/ceph/3056d51a-e76b-11ee-95cd-87774f69a715/mon.c/config 2024-03-21T10:16:47.714 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:16:47 smithi047 ceph-mon[31547]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 125 MiB used, 357 GiB / 358 GiB avail 2024-03-21T10:16:47.714 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:16:47 smithi047 ceph-mon[31547]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-03-21T10:16:47.714 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:16:47 smithi047 ceph-mon[31547]: osdmap e33: 5 total, 4 up, 5 in 2024-03-21T10:16:47.715 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:16:47 smithi047 ceph-mon[31547]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-21T10:16:47.715 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:16:47 smithi047 ceph-mon[31547]: from='osd.4 [v2:172.21.15.100:6808/2785478639,v1:172.21.15.100:6809/2785478639]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi100", "root=default"]}]: dispatch 2024-03-21T10:16:47.715 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:16:47 smithi047 ceph-mon[31547]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi100", "root=default"]}]: dispatch 2024-03-21T10:16:47.715 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:16:47 smithi047 ceph-mon[31547]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:16:47.715 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:16:47 smithi047 ceph-mon[31547]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:16:48.490 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:16:48 smithi157 ceph-mon[35581]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi100", "root=default"]}]': finished 2024-03-21T10:16:48.490 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:16:48 smithi157 ceph-mon[35581]: osdmap e34: 5 total, 4 up, 5 in 2024-03-21T10:16:48.490 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:16:48 smithi157 ceph-mon[35581]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-21T10:16:48.490 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:16:48 smithi157 ceph-mon[35581]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-21T10:16:48.646 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:16:48 smithi100 ceph-mon[35193]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi100", "root=default"]}]': finished 2024-03-21T10:16:48.646 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:16:48 smithi100 ceph-mon[35193]: osdmap e34: 5 total, 4 up, 5 in 2024-03-21T10:16:48.646 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:16:48 smithi100 ceph-mon[35193]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-21T10:16:48.646 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:16:48 smithi100 ceph-mon[35193]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-21T10:16:48.714 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:16:48 smithi047 ceph-mon[31547]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi100", "root=default"]}]': finished 2024-03-21T10:16:48.714 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:16:48 smithi047 ceph-mon[31547]: osdmap e34: 5 total, 4 up, 5 in 2024-03-21T10:16:48.714 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:16:48 smithi047 ceph-mon[31547]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-21T10:16:48.714 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:16:48 smithi047 ceph-mon[31547]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-21T10:16:49.480 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:16:49 smithi100 ceph-mon[35193]: purged_snaps scrub starts 2024-03-21T10:16:49.480 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:16:49 smithi100 ceph-mon[35193]: purged_snaps scrub ok 2024-03-21T10:16:49.480 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:16:49 smithi100 ceph-mon[35193]: pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 125 MiB used, 357 GiB / 358 GiB avail 2024-03-21T10:16:49.480 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:16:49 smithi100 ceph-mon[35193]: osd.4 [v2:172.21.15.100:6808/2785478639,v1:172.21.15.100:6809/2785478639] boot 2024-03-21T10:16:49.480 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:16:49 smithi100 ceph-mon[35193]: osdmap e35: 5 total, 5 up, 5 in 2024-03-21T10:16:49.480 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:16:49 smithi100 ceph-mon[35193]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-21T10:16:49.513 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:16:49 smithi157 ceph-mon[35581]: purged_snaps scrub starts 2024-03-21T10:16:49.513 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:16:49 smithi157 ceph-mon[35581]: purged_snaps scrub ok 2024-03-21T10:16:49.513 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:16:49 smithi157 ceph-mon[35581]: pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 125 MiB used, 357 GiB / 358 GiB avail 2024-03-21T10:16:49.513 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:16:49 smithi157 ceph-mon[35581]: osd.4 [v2:172.21.15.100:6808/2785478639,v1:172.21.15.100:6809/2785478639] boot 2024-03-21T10:16:49.514 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:16:49 smithi157 ceph-mon[35581]: osdmap e35: 5 total, 5 up, 5 in 2024-03-21T10:16:49.514 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:16:49 smithi157 ceph-mon[35581]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-21T10:16:49.615 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:16:49 smithi047 ceph-mon[31547]: purged_snaps scrub starts 2024-03-21T10:16:49.615 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:16:49 smithi047 ceph-mon[31547]: purged_snaps scrub ok 2024-03-21T10:16:49.615 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:16:49 smithi047 ceph-mon[31547]: pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 125 MiB used, 357 GiB / 358 GiB avail 2024-03-21T10:16:49.615 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:16:49 smithi047 ceph-mon[31547]: osd.4 [v2:172.21.15.100:6808/2785478639,v1:172.21.15.100:6809/2785478639] boot 2024-03-21T10:16:49.615 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:16:49 smithi047 ceph-mon[31547]: osdmap e35: 5 total, 5 up, 5 in 2024-03-21T10:16:49.615 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:16:49 smithi047 ceph-mon[31547]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-21T10:16:50.512 INFO:teuthology.orchestra.run.smithi157.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-03-21T10:16:50.512 INFO:teuthology.orchestra.run.smithi157.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-03-21T10:16:50.512 INFO:teuthology.orchestra.run.smithi157.stderr: stderr: 10+0 records in 2024-03-21T10:16:50.512 INFO:teuthology.orchestra.run.smithi157.stderr:10+0 records out 2024-03-21T10:16:50.512 INFO:teuthology.orchestra.run.smithi157.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0373242 s, 281 MB/s 2024-03-21T10:16:50.513 INFO:teuthology.orchestra.run.smithi157.stderr:--> Zapping successful for: 2024-03-21T10:16:50.549 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:16:50 smithi157 ceph-mon[35581]: osdmap e36: 5 total, 5 up, 5 in 2024-03-21T10:16:50.714 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:16:50 smithi047 ceph-mon[31547]: osdmap e36: 5 total, 5 up, 5 in 2024-03-21T10:16:50.729 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:16:50 smithi100 ceph-mon[35193]: osdmap e36: 5 total, 5 up, 5 in 2024-03-21T10:16:51.287 DEBUG:teuthology.orchestra.run.smithi157:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2edfee68ad7243bed422a049b6f5f58619300f11 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3056d51a-e76b-11ee-95cd-87774f69a715 -- ceph orch daemon add osd smithi157:vg_nvme/lv_3 2024-03-21T10:16:51.490 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:16:51 smithi157 ceph-mon[35581]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-03-21T10:16:51.661 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:16:51 smithi100 ceph-mon[35193]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-03-21T10:16:51.713 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:16:51 smithi047 ceph-mon[31547]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-03-21T10:16:51.729 INFO:teuthology.orchestra.run.smithi157.stderr:Inferring config /var/lib/ceph/3056d51a-e76b-11ee-95cd-87774f69a715/mon.c/config 2024-03-21T10:16:52.990 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:16:52 smithi157 ceph-mon[35581]: Detected new or changed devices on smithi100 2024-03-21T10:16:52.990 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:16:52 smithi157 ceph-mon[35581]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:16:52.991 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:16:52 smithi157 ceph-mon[35581]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:16:52.991 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:16:52 smithi157 ceph-mon[35581]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-21T10:16:52.991 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:16:52 smithi157 ceph-mon[35581]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-21T10:16:52.991 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:16:52 smithi157 ceph-mon[35581]: Adjusting osd_memory_target on smithi100 to 8545M 2024-03-21T10:16:52.991 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:16:52 smithi157 ceph-mon[35581]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:16:52.991 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:16:52 smithi157 ceph-mon[35581]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T10:16:52.991 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:16:52 smithi157 ceph-mon[35581]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T10:16:52.991 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:16:52 smithi157 ceph-mon[35581]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:16:52.991 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:16:52 smithi157 ceph-mon[35581]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T10:16:52.991 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:16:52 smithi157 ceph-mon[35581]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T10:16:52.991 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:16:52 smithi157 ceph-mon[35581]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T10:16:52.991 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:16:52 smithi157 ceph-mon[35581]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:16:53.214 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:16:52 smithi047 ceph-mon[31547]: Detected new or changed devices on smithi100 2024-03-21T10:16:53.214 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:16:52 smithi047 ceph-mon[31547]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:16:53.214 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:16:52 smithi047 ceph-mon[31547]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:16:53.214 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:16:52 smithi047 ceph-mon[31547]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-21T10:16:53.215 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:16:52 smithi047 ceph-mon[31547]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-21T10:16:53.215 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:16:52 smithi047 ceph-mon[31547]: Adjusting osd_memory_target on smithi100 to 8545M 2024-03-21T10:16:53.215 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:16:52 smithi047 ceph-mon[31547]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:16:53.215 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:16:52 smithi047 ceph-mon[31547]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T10:16:53.215 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:16:52 smithi047 ceph-mon[31547]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T10:16:53.215 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:16:52 smithi047 ceph-mon[31547]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:16:53.215 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:16:52 smithi047 ceph-mon[31547]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T10:16:53.215 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:16:52 smithi047 ceph-mon[31547]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T10:16:53.215 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:16:52 smithi047 ceph-mon[31547]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T10:16:53.215 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:16:52 smithi047 ceph-mon[31547]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:16:53.230 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:16:52 smithi100 ceph-mon[35193]: Detected new or changed devices on smithi100 2024-03-21T10:16:53.230 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:16:52 smithi100 ceph-mon[35193]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:16:53.230 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:16:52 smithi100 ceph-mon[35193]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:16:53.230 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:16:52 smithi100 ceph-mon[35193]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-21T10:16:53.230 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:16:52 smithi100 ceph-mon[35193]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-21T10:16:53.230 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:16:52 smithi100 ceph-mon[35193]: Adjusting osd_memory_target on smithi100 to 8545M 2024-03-21T10:16:53.230 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:16:52 smithi100 ceph-mon[35193]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:16:53.230 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:16:52 smithi100 ceph-mon[35193]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T10:16:53.231 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:16:52 smithi100 ceph-mon[35193]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T10:16:53.231 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:16:52 smithi100 ceph-mon[35193]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:16:53.231 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:16:52 smithi100 ceph-mon[35193]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T10:16:53.231 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:16:52 smithi100 ceph-mon[35193]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T10:16:53.231 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:16:52 smithi100 ceph-mon[35193]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T10:16:53.231 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:16:52 smithi100 ceph-mon[35193]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:16:53.859 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-21T10:16:53.858+0000 7fdfd0000700 1 -- 172.21.15.157:0/3385344539 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdfc8105610 msgr2=0x7fdfc8105a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:16:53.860 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-21T10:16:53.858+0000 7fdfd0000700 1 --2- 172.21.15.157:0/3385344539 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdfc8105610 0x7fdfc8105a70 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fdfb4009990 tx=0x7fdfb4009ca0 comp rx=0 tx=0).stop 2024-03-21T10:16:53.860 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-21T10:16:53.858+0000 7fdfd0000700 1 -- 172.21.15.157:0/3385344539 shutdown_connections 2024-03-21T10:16:53.860 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-21T10:16:53.858+0000 7fdfd0000700 1 --2- 172.21.15.157:0/3385344539 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdfc8105fb0 0x7fdfc810c860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:16:53.860 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-21T10:16:53.858+0000 7fdfd0000700 1 --2- 172.21.15.157:0/3385344539 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdfc8104410 0x7fdfc8104830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:16:53.860 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-21T10:16:53.858+0000 7fdfd0000700 1 --2- 172.21.15.157:0/3385344539 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdfc8105610 0x7fdfc8105a70 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:16:53.860 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-21T10:16:53.858+0000 7fdfd0000700 1 -- 172.21.15.157:0/3385344539 >> 172.21.15.157:0/3385344539 conn(0x7fdfc80ff9b0 msgr2=0x7fdfc8101df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:16:53.860 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-21T10:16:53.858+0000 7fdfd0000700 1 -- 172.21.15.157:0/3385344539 shutdown_connections 2024-03-21T10:16:53.860 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-21T10:16:53.858+0000 7fdfd0000700 1 -- 172.21.15.157:0/3385344539 wait complete. 2024-03-21T10:16:53.862 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-21T10:16:53.859+0000 7fdfd0000700 1 Processor -- start 2024-03-21T10:16:53.862 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-21T10:16:53.859+0000 7fdfd0000700 1 -- start start 2024-03-21T10:16:53.862 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-21T10:16:53.859+0000 7fdfd0000700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdfc8104410 0x7fdfc819a4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:16:53.863 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-21T10:16:53.859+0000 7fdfd0000700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdfc8105610 0x7fdfc819aa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:16:53.863 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-21T10:16:53.859+0000 7fdfd0000700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdfc8105fb0 0x7fdfc81a1a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:16:53.863 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-21T10:16:53.859+0000 7fdfd0000700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdfc810fee0 con 0x7fdfc8105fb0 2024-03-21T10:16:53.863 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-21T10:16:53.859+0000 7fdfd0000700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdfc810fd60 con 0x7fdfc8104410 2024-03-21T10:16:53.863 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-21T10:16:53.859+0000 7fdfd0000700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdfc8110060 con 0x7fdfc8105610 2024-03-21T10:16:53.863 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-21T10:16:53.860+0000 7fdfccc00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdfc8105610 0x7fdfc819aa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:16:53.863 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-21T10:16:53.860+0000 7fdfccc00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdfc8105610 0x7fdfc819aa00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:42024/0 (socket says 172.21.15.157:42024) 2024-03-21T10:16:53.863 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-21T10:16:53.860+0000 7fdfccc00700 1 -- 172.21.15.157:0/3336796004 learned_addr learned my addr 172.21.15.157:0/3336796004 (peer_addr_for_me v2:172.21.15.157:0/0) 2024-03-21T10:16:53.864 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-21T10:16:53.860+0000 7fdfcd600700 1 --2- 172.21.15.157:0/3336796004 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdfc8104410 0x7fdfc819a4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:16:53.864 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-21T10:16:53.860+0000 7fdfce000700 1 --2- 172.21.15.157:0/3336796004 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdfc8105fb0 0x7fdfc81a1a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:16:53.864 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-21T10:16:53.860+0000 7fdfccc00700 1 -- 172.21.15.157:0/3336796004 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdfc8104410 msgr2=0x7fdfc819a4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:16:53.864 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-21T10:16:53.860+0000 7fdfccc00700 1 --2- 172.21.15.157:0/3336796004 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdfc8104410 0x7fdfc819a4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:16:53.864 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-21T10:16:53.860+0000 7fdfccc00700 1 -- 172.21.15.157:0/3336796004 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdfc8105fb0 msgr2=0x7fdfc81a1a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:16:53.864 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-21T10:16:53.860+0000 7fdfccc00700 1 --2- 172.21.15.157:0/3336796004 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdfc8105fb0 0x7fdfc81a1a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:16:53.864 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-21T10:16:53.860+0000 7fdfccc00700 1 -- 172.21.15.157:0/3336796004 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdfc81a2120 con 0x7fdfc8105610 2024-03-21T10:16:53.864 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-21T10:16:53.861+0000 7fdfccc00700 1 --2- 172.21.15.157:0/3336796004 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdfc8105610 0x7fdfc819aa00 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fdfb40057c0 tx=0x7fdfb4003710 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:16:53.864 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-21T10:16:53.861+0000 7fdfc6000700 1 -- 172.21.15.157:0/3336796004 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdfb4027070 con 0x7fdfc8105610 2024-03-21T10:16:53.864 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-21T10:16:53.861+0000 7fdfc6000700 1 -- 172.21.15.157:0/3336796004 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fdfb4004180 con 0x7fdfc8105610 2024-03-21T10:16:53.864 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-21T10:16:53.861+0000 7fdfc6000700 1 -- 172.21.15.157:0/3336796004 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdfb400f460 con 0x7fdfc8105610 2024-03-21T10:16:53.865 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-21T10:16:53.861+0000 7fdfd0000700 1 -- 172.21.15.157:0/3336796004 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdfc81a23b0 con 0x7fdfc8105610 2024-03-21T10:16:53.865 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-21T10:16:53.861+0000 7fdfd0000700 1 -- 172.21.15.157:0/3336796004 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdfc81a27b0 con 0x7fdfc8105610 2024-03-21T10:16:53.865 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-21T10:16:53.862+0000 7fdfd0000700 1 -- 172.21.15.157:0/3336796004 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdfc8068e50 con 0x7fdfc8105610 2024-03-21T10:16:53.865 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-21T10:16:53.864+0000 7fdfc6000700 1 -- 172.21.15.157:0/3336796004 <== mon.1 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 16) v1 ==== 137854+0+0 (secure 0 0 0) 0x7fdfb402a070 con 0x7fdfc8105610 2024-03-21T10:16:53.867 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-21T10:16:53.865+0000 7fdfc6000700 1 --2- 172.21.15.157:0/3336796004 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7fdfb00a3030 0x7fdfb00a54f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:16:53.867 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-21T10:16:53.866+0000 7fdfc6000700 1 -- 172.21.15.157:0/3336796004 <== mon.1 v2:172.21.15.157:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 4101+0+0 (secure 0 0 0) 0x7fdfb40cfd00 con 0x7fdfc8105610 2024-03-21T10:16:53.867 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-21T10:16:53.866+0000 7fdfcd600700 1 --2- 172.21.15.157:0/3336796004 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7fdfb00a3030 0x7fdfb00a54f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:16:53.868 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-21T10:16:53.867+0000 7fdfcd600700 1 --2- 172.21.15.157:0/3336796004 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7fdfb00a3030 0x7fdfb00a54f0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fdfbc005fd0 tx=0x7fdfbc005e50 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:16:53.870 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-21T10:16:53.869+0000 7fdfc6000700 1 -- 172.21.15.157:0/3336796004 <== mon.1 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fdfb4031020 con 0x7fdfc8105610 2024-03-21T10:16:53.989 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:16:53 smithi157 ceph-mon[35581]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-03-21T10:16:54.102 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-21T10:16:54.100+0000 7fdfd0000700 1 -- 172.21.15.157:0/3336796004 --> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi157:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) v1 -- 0x7fdfc80625d0 con 0x7fdfb00a3030 2024-03-21T10:16:54.214 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:16:53 smithi047 ceph-mon[31547]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-03-21T10:16:54.229 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:16:53 smithi100 ceph-mon[35193]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-03-21T10:16:55.060 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:16:54 smithi157 ceph-mon[35581]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-21T10:16:55.060 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:16:54 smithi157 ceph-mon[35581]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-21T10:16:55.060 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:16:54 smithi157 ceph-mon[35581]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T10:16:55.214 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:16:54 smithi047 ceph-mon[31547]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-21T10:16:55.214 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:16:54 smithi047 ceph-mon[31547]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-21T10:16:55.214 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:16:54 smithi047 ceph-mon[31547]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T10:16:55.229 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:16:54 smithi100 ceph-mon[35193]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-21T10:16:55.230 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:16:54 smithi100 ceph-mon[35193]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-21T10:16:55.230 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:16:54 smithi100 ceph-mon[35193]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T10:16:56.214 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:16:55 smithi047 ceph-mon[31547]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-03-21T10:16:56.214 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:16:55 smithi047 ceph-mon[31547]: from='client.24217 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi157:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:16:56.229 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:16:55 smithi100 ceph-mon[35193]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-03-21T10:16:56.229 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:16:55 smithi100 ceph-mon[35193]: from='client.24217 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi157:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:16:56.239 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:16:55 smithi157 ceph-mon[35581]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-03-21T10:16:56.239 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:16:55 smithi157 ceph-mon[35581]: from='client.24217 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi157:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:16:58.191 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:16:57 smithi100 ceph-mon[35193]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-03-21T10:16:58.214 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:16:57 smithi047 ceph-mon[31547]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-03-21T10:16:58.239 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:16:57 smithi157 ceph-mon[35581]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-03-21T10:16:58.990 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:16:58 smithi157 ceph-mon[35581]: from='client.? 172.21.15.157:0/3995459035' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "012d2188-3a61-4a5f-8d01-2001c15ffbe2"}]: dispatch 2024-03-21T10:16:58.990 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:16:58 smithi157 ceph-mon[35581]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "012d2188-3a61-4a5f-8d01-2001c15ffbe2"}]: dispatch 2024-03-21T10:16:58.990 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:16:58 smithi157 ceph-mon[35581]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "012d2188-3a61-4a5f-8d01-2001c15ffbe2"}]': finished 2024-03-21T10:16:58.990 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:16:58 smithi157 ceph-mon[35581]: osdmap e37: 6 total, 5 up, 6 in 2024-03-21T10:16:58.990 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:16:58 smithi157 ceph-mon[35581]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-21T10:16:59.214 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:16:58 smithi047 ceph-mon[31547]: from='client.? 172.21.15.157:0/3995459035' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "012d2188-3a61-4a5f-8d01-2001c15ffbe2"}]: dispatch 2024-03-21T10:16:59.214 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:16:58 smithi047 ceph-mon[31547]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "012d2188-3a61-4a5f-8d01-2001c15ffbe2"}]: dispatch 2024-03-21T10:16:59.214 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:16:58 smithi047 ceph-mon[31547]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "012d2188-3a61-4a5f-8d01-2001c15ffbe2"}]': finished 2024-03-21T10:16:59.214 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:16:58 smithi047 ceph-mon[31547]: osdmap e37: 6 total, 5 up, 6 in 2024-03-21T10:16:59.214 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:16:58 smithi047 ceph-mon[31547]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-21T10:16:59.229 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:16:58 smithi100 ceph-mon[35193]: from='client.? 172.21.15.157:0/3995459035' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "012d2188-3a61-4a5f-8d01-2001c15ffbe2"}]: dispatch 2024-03-21T10:16:59.229 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:16:58 smithi100 ceph-mon[35193]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "012d2188-3a61-4a5f-8d01-2001c15ffbe2"}]: dispatch 2024-03-21T10:16:59.230 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:16:58 smithi100 ceph-mon[35193]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "012d2188-3a61-4a5f-8d01-2001c15ffbe2"}]': finished 2024-03-21T10:16:59.230 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:16:58 smithi100 ceph-mon[35193]: osdmap e37: 6 total, 5 up, 6 in 2024-03-21T10:16:59.230 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:16:58 smithi100 ceph-mon[35193]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-21T10:17:00.214 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:16:59 smithi047 ceph-mon[31547]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-03-21T10:17:00.214 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:16:59 smithi047 ceph-mon[31547]: from='client.? 172.21.15.157:0/4051235837' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-21T10:17:00.229 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:16:59 smithi100 ceph-mon[35193]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-03-21T10:17:00.229 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:16:59 smithi100 ceph-mon[35193]: from='client.? 172.21.15.157:0/4051235837' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-21T10:17:00.239 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:16:59 smithi157 ceph-mon[35581]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-03-21T10:17:00.239 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:16:59 smithi157 ceph-mon[35581]: from='client.? 172.21.15.157:0/4051235837' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-21T10:17:01.215 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:17:00 smithi047 ceph-mon[31547]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-03-21T10:17:01.229 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:17:00 smithi100 ceph-mon[35193]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-03-21T10:17:01.239 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:17:00 smithi157 ceph-mon[35581]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-03-21T10:17:03.239 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:17:02 smithi157 ceph-mon[35581]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-03-21T10:17:03.463 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:17:02 smithi047 ceph-mon[31547]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-03-21T10:17:03.479 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:17:02 smithi100 ceph-mon[35193]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-03-21T10:17:05.034 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:17:04 smithi157 ceph-mon[35581]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-03-21T10:17:05.464 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:17:04 smithi047 ceph-mon[31547]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-03-21T10:17:05.479 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:17:04 smithi100 ceph-mon[35193]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-03-21T10:17:07.240 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:17:06 smithi157 ceph-mon[35581]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-03-21T10:17:07.240 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:17:06 smithi157 ceph-mon[35581]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-03-21T10:17:07.240 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:17:06 smithi157 ceph-mon[35581]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T10:17:07.464 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:17:06 smithi047 ceph-mon[31547]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-03-21T10:17:07.464 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:17:06 smithi047 ceph-mon[31547]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-03-21T10:17:07.464 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:17:06 smithi047 ceph-mon[31547]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T10:17:07.479 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:17:06 smithi100 ceph-mon[35193]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-03-21T10:17:07.479 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:17:06 smithi100 ceph-mon[35193]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-03-21T10:17:07.479 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:17:06 smithi100 ceph-mon[35193]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T10:17:08.198 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:17:08 smithi157 ceph-mon[35581]: Deploying daemon osd.5 on smithi157 2024-03-21T10:17:08.464 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:17:08 smithi047 ceph-mon[31547]: Deploying daemon osd.5 on smithi157 2024-03-21T10:17:08.479 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:17:08 smithi100 ceph-mon[35193]: Deploying daemon osd.5 on smithi157 2024-03-21T10:17:09.464 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:17:09 smithi047 ceph-mon[31547]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-03-21T10:17:09.479 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:17:09 smithi100 ceph-mon[35193]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-03-21T10:17:09.489 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:17:09 smithi157 ceph-mon[35581]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-03-21T10:17:11.239 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:17:11 smithi157 ceph-mon[35581]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-03-21T10:17:11.464 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:17:11 smithi047 ceph-mon[31547]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-03-21T10:17:11.479 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:17:11 smithi100 ceph-mon[35193]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-03-21T10:17:13.464 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:17:13 smithi047 ceph-mon[31547]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-03-21T10:17:13.479 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:17:13 smithi100 ceph-mon[35193]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-03-21T10:17:13.490 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:17:13 smithi157 ceph-mon[35581]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-03-21T10:17:15.239 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:17:15 smithi157 ceph-mon[35581]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-03-21T10:17:15.464 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:17:15 smithi047 ceph-mon[31547]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-03-21T10:17:15.479 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:17:15 smithi100 ceph-mon[35193]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-03-21T10:17:16.308 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:17:16 smithi157 ceph-mon[35581]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T10:17:16.308 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:17:16 smithi157 ceph-mon[35581]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:17:16.309 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:17:16 smithi157 ceph-mon[35581]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:17:16.464 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:17:16 smithi047 ceph-mon[31547]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T10:17:16.464 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:17:16 smithi047 ceph-mon[31547]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:17:16.464 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:17:16 smithi047 ceph-mon[31547]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:17:16.479 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:17:16 smithi100 ceph-mon[35193]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T10:17:16.479 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:17:16 smithi100 ceph-mon[35193]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:17:16.479 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:17:16 smithi100 ceph-mon[35193]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:17:17.240 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:17:17 smithi157 ceph-mon[35581]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-03-21T10:17:17.408 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:17:17 smithi100 ceph-mon[35193]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-03-21T10:17:17.463 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:17:17 smithi047 ceph-mon[31547]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-03-21T10:17:19.919 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-21T10:17:19.917+0000 7fdfc6000700 1 -- 172.21.15.157:0/3336796004 <== mgr.14152 v2:172.21.15.47:6800/2823885994 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7fdfc80625d0 con 0x7fdfb00a3030 2024-03-21T10:17:19.920 INFO:teuthology.orchestra.run.smithi157.stdout:Created osd(s) 5 on host 'smithi157' 2024-03-21T10:17:19.922 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-21T10:17:19.920+0000 7fdfd0000700 1 -- 172.21.15.157:0/3336796004 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7fdfb00a3030 msgr2=0x7fdfb00a54f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:17:19.922 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-21T10:17:19.920+0000 7fdfd0000700 1 --2- 172.21.15.157:0/3336796004 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7fdfb00a3030 0x7fdfb00a54f0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fdfbc005fd0 tx=0x7fdfbc005e50 comp rx=0 tx=0).stop 2024-03-21T10:17:19.923 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-21T10:17:19.920+0000 7fdfd0000700 1 -- 172.21.15.157:0/3336796004 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdfc8105610 msgr2=0x7fdfc819aa00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:17:19.923 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-21T10:17:19.920+0000 7fdfd0000700 1 --2- 172.21.15.157:0/3336796004 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdfc8105610 0x7fdfc819aa00 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fdfb40057c0 tx=0x7fdfb4003710 comp rx=0 tx=0).stop 2024-03-21T10:17:19.923 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-21T10:17:19.921+0000 7fdfd0000700 1 -- 172.21.15.157:0/3336796004 shutdown_connections 2024-03-21T10:17:19.923 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-21T10:17:19.921+0000 7fdfd0000700 1 --2- 172.21.15.157:0/3336796004 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7fdfb00a3030 0x7fdfb00a54f0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:17:19.923 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-21T10:17:19.921+0000 7fdfd0000700 1 --2- 172.21.15.157:0/3336796004 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdfc8105fb0 0x7fdfc81a1a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:17:19.923 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-21T10:17:19.921+0000 7fdfd0000700 1 --2- 172.21.15.157:0/3336796004 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdfc8104410 0x7fdfc819a4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:17:19.923 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-21T10:17:19.921+0000 7fdfd0000700 1 --2- 172.21.15.157:0/3336796004 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdfc8105610 0x7fdfc819aa00 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:17:19.923 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-21T10:17:19.921+0000 7fdfd0000700 1 -- 172.21.15.157:0/3336796004 >> 172.21.15.157:0/3336796004 conn(0x7fdfc80ff9b0 msgr2=0x7fdfc81088b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:17:19.923 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-21T10:17:19.921+0000 7fdfd0000700 1 -- 172.21.15.157:0/3336796004 shutdown_connections 2024-03-21T10:17:19.923 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-21T10:17:19.921+0000 7fdfd0000700 1 -- 172.21.15.157:0/3336796004 wait complete. 2024-03-21T10:17:20.168 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:17:19 smithi047 ceph-mon[31547]: pgmap v173: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-03-21T10:17:20.168 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:17:19 smithi047 ceph-mon[31547]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:17:20.168 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:17:19 smithi047 ceph-mon[31547]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:17:20.168 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:17:19 smithi047 ceph-mon[31547]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T10:17:20.168 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:17:19 smithi047 ceph-mon[31547]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T10:17:20.169 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:17:19 smithi047 ceph-mon[31547]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:17:20.169 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:17:19 smithi047 ceph-mon[31547]: from='osd.5 [v2:172.21.15.157:6808/1525645410,v1:172.21.15.157:6809/1525645410]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-03-21T10:17:20.169 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:17:19 smithi047 ceph-mon[31547]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-03-21T10:17:20.229 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:17:19 smithi100 ceph-mon[35193]: pgmap v173: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-03-21T10:17:20.230 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:17:19 smithi100 ceph-mon[35193]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:17:20.230 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:17:19 smithi100 ceph-mon[35193]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:17:20.230 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:17:19 smithi100 ceph-mon[35193]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T10:17:20.230 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:17:19 smithi100 ceph-mon[35193]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T10:17:20.230 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:17:19 smithi100 ceph-mon[35193]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:17:20.230 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:17:19 smithi100 ceph-mon[35193]: from='osd.5 [v2:172.21.15.157:6808/1525645410,v1:172.21.15.157:6809/1525645410]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-03-21T10:17:20.230 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:17:19 smithi100 ceph-mon[35193]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-03-21T10:17:20.239 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:17:19 smithi157 ceph-mon[35581]: pgmap v173: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-03-21T10:17:20.240 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:17:19 smithi157 ceph-mon[35581]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:17:20.240 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:17:19 smithi157 ceph-mon[35581]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:17:20.240 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:17:19 smithi157 ceph-mon[35581]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T10:17:20.240 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:17:19 smithi157 ceph-mon[35581]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T10:17:20.240 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:17:19 smithi157 ceph-mon[35581]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:17:20.240 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:17:19 smithi157 ceph-mon[35581]: from='osd.5 [v2:172.21.15.157:6808/1525645410,v1:172.21.15.157:6809/1525645410]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-03-21T10:17:20.240 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:17:19 smithi157 ceph-mon[35581]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-03-21T10:17:20.831 DEBUG:teuthology.orchestra.run.smithi157:osd.5> sudo journalctl -f -n 0 -u ceph-3056d51a-e76b-11ee-95cd-87774f69a715@osd.5.service 2024-03-21T10:17:20.836 INFO:tasks.cephadm:Deploying osd.6 on smithi047 with /dev/vg_nvme/lv_2... 2024-03-21T10:17:20.836 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2edfee68ad7243bed422a049b6f5f58619300f11 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3056d51a-e76b-11ee-95cd-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-03-21T10:17:21.091 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:17:20 smithi157 ceph-mon[35581]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-03-21T10:17:21.091 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:17:20 smithi157 ceph-mon[35581]: osdmap e38: 6 total, 5 up, 6 in 2024-03-21T10:17:21.092 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:17:20 smithi157 ceph-mon[35581]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-21T10:17:21.092 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:17:20 smithi157 ceph-mon[35581]: from='osd.5 [v2:172.21.15.157:6808/1525645410,v1:172.21.15.157:6809/1525645410]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi157", "root=default"]}]: dispatch 2024-03-21T10:17:21.092 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:17:20 smithi157 ceph-mon[35581]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi157", "root=default"]}]: dispatch 2024-03-21T10:17:21.092 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:17:20 smithi157 ceph-mon[35581]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:17:21.092 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:17:20 smithi157 ceph-mon[35581]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T10:17:21.092 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:17:20 smithi157 ceph-mon[35581]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:17:21.213 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:17:20 smithi047 ceph-mon[31547]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-03-21T10:17:21.214 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:17:20 smithi047 ceph-mon[31547]: osdmap e38: 6 total, 5 up, 6 in 2024-03-21T10:17:21.214 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:17:20 smithi047 ceph-mon[31547]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-21T10:17:21.214 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:17:20 smithi047 ceph-mon[31547]: from='osd.5 [v2:172.21.15.157:6808/1525645410,v1:172.21.15.157:6809/1525645410]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi157", "root=default"]}]: dispatch 2024-03-21T10:17:21.214 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:17:20 smithi047 ceph-mon[31547]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi157", "root=default"]}]: dispatch 2024-03-21T10:17:21.214 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:17:20 smithi047 ceph-mon[31547]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:17:21.214 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:17:20 smithi047 ceph-mon[31547]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T10:17:21.214 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:17:20 smithi047 ceph-mon[31547]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:17:21.229 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:17:20 smithi100 ceph-mon[35193]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-03-21T10:17:21.229 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:17:20 smithi100 ceph-mon[35193]: osdmap e38: 6 total, 5 up, 6 in 2024-03-21T10:17:21.230 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:17:20 smithi100 ceph-mon[35193]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-21T10:17:21.230 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:17:20 smithi100 ceph-mon[35193]: from='osd.5 [v2:172.21.15.157:6808/1525645410,v1:172.21.15.157:6809/1525645410]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi157", "root=default"]}]: dispatch 2024-03-21T10:17:21.230 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:17:20 smithi100 ceph-mon[35193]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi157", "root=default"]}]: dispatch 2024-03-21T10:17:21.230 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:17:20 smithi100 ceph-mon[35193]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:17:21.230 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:17:20 smithi100 ceph-mon[35193]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T10:17:21.230 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:17:20 smithi100 ceph-mon[35193]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:17:21.291 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/3056d51a-e76b-11ee-95cd-87774f69a715/mon.a/config 2024-03-21T10:17:21.741 INFO:journalctl@ceph.osd.5.smithi157.stdout:Mar 21 10:17:21 smithi157 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-5[47667]: 2024-03-21T10:17:21.490+0000 7f2437e00700 -1 osd.5 0 waiting for initial osdmap 2024-03-21T10:17:21.741 INFO:journalctl@ceph.osd.5.smithi157.stdout:Mar 21 10:17:21 smithi157 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-5[47667]: 2024-03-21T10:17:21.509+0000 7f242dc00700 -1 osd.5 39 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-21T10:17:22.188 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:17:21 smithi157 ceph-mon[35581]: pgmap v175: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-03-21T10:17:22.188 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:17:21 smithi157 ceph-mon[35581]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi157", "root=default"]}]': finished 2024-03-21T10:17:22.188 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:17:21 smithi157 ceph-mon[35581]: osdmap e39: 6 total, 5 up, 6 in 2024-03-21T10:17:22.188 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:17:21 smithi157 ceph-mon[35581]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-21T10:17:22.188 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:17:21 smithi157 ceph-mon[35581]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-21T10:17:22.188 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:17:21 smithi157 ceph-mon[35581]: from='osd.5 ' entity='osd.5' 2024-03-21T10:17:22.214 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:17:21 smithi047 ceph-mon[31547]: pgmap v175: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-03-21T10:17:22.214 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:17:21 smithi047 ceph-mon[31547]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi157", "root=default"]}]': finished 2024-03-21T10:17:22.214 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:17:21 smithi047 ceph-mon[31547]: osdmap e39: 6 total, 5 up, 6 in 2024-03-21T10:17:22.214 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:17:21 smithi047 ceph-mon[31547]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-21T10:17:22.214 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:17:21 smithi047 ceph-mon[31547]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-21T10:17:22.214 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:17:21 smithi047 ceph-mon[31547]: from='osd.5 ' entity='osd.5' 2024-03-21T10:17:22.229 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:17:21 smithi100 ceph-mon[35193]: pgmap v175: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-03-21T10:17:22.229 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:17:21 smithi100 ceph-mon[35193]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi157", "root=default"]}]': finished 2024-03-21T10:17:22.229 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:17:21 smithi100 ceph-mon[35193]: osdmap e39: 6 total, 5 up, 6 in 2024-03-21T10:17:22.230 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:17:21 smithi100 ceph-mon[35193]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-21T10:17:22.230 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:17:21 smithi100 ceph-mon[35193]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-21T10:17:22.230 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:17:21 smithi100 ceph-mon[35193]: from='osd.5 ' entity='osd.5' 2024-03-21T10:17:22.964 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:17:22 smithi047 ceph-mon[31547]: purged_snaps scrub starts 2024-03-21T10:17:22.964 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:17:22 smithi047 ceph-mon[31547]: purged_snaps scrub ok 2024-03-21T10:17:22.964 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:17:22 smithi047 ceph-mon[31547]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-21T10:17:22.964 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:17:22 smithi047 ceph-mon[31547]: osd.5 [v2:172.21.15.157:6808/1525645410,v1:172.21.15.157:6809/1525645410] boot 2024-03-21T10:17:22.964 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:17:22 smithi047 ceph-mon[31547]: osdmap e40: 6 total, 6 up, 6 in 2024-03-21T10:17:22.965 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:17:22 smithi047 ceph-mon[31547]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-21T10:17:22.965 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:17:22 smithi047 ceph-mon[31547]: pgmap v178: 1 pgs: 1 peering; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-03-21T10:17:23.229 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:17:22 smithi100 ceph-mon[35193]: purged_snaps scrub starts 2024-03-21T10:17:23.230 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:17:22 smithi100 ceph-mon[35193]: purged_snaps scrub ok 2024-03-21T10:17:23.230 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:17:22 smithi100 ceph-mon[35193]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-21T10:17:23.230 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:17:22 smithi100 ceph-mon[35193]: osd.5 [v2:172.21.15.157:6808/1525645410,v1:172.21.15.157:6809/1525645410] boot 2024-03-21T10:17:23.230 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:17:22 smithi100 ceph-mon[35193]: osdmap e40: 6 total, 6 up, 6 in 2024-03-21T10:17:23.230 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:17:22 smithi100 ceph-mon[35193]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-21T10:17:23.230 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:17:22 smithi100 ceph-mon[35193]: pgmap v178: 1 pgs: 1 peering; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-03-21T10:17:23.240 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:17:22 smithi157 ceph-mon[35581]: purged_snaps scrub starts 2024-03-21T10:17:23.240 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:17:22 smithi157 ceph-mon[35581]: purged_snaps scrub ok 2024-03-21T10:17:23.240 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:17:22 smithi157 ceph-mon[35581]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-21T10:17:23.240 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:17:22 smithi157 ceph-mon[35581]: osd.5 [v2:172.21.15.157:6808/1525645410,v1:172.21.15.157:6809/1525645410] boot 2024-03-21T10:17:23.240 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:17:22 smithi157 ceph-mon[35581]: osdmap e40: 6 total, 6 up, 6 in 2024-03-21T10:17:23.240 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:17:22 smithi157 ceph-mon[35581]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-21T10:17:23.240 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:17:22 smithi157 ceph-mon[35581]: pgmap v178: 1 pgs: 1 peering; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-03-21T10:17:23.925 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:17:23 smithi047 ceph-mon[31547]: osdmap e41: 6 total, 6 up, 6 in 2024-03-21T10:17:24.229 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:17:23 smithi100 ceph-mon[35193]: osdmap e41: 6 total, 6 up, 6 in 2024-03-21T10:17:24.232 INFO:teuthology.orchestra.run.smithi047.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-03-21T10:17:24.232 INFO:teuthology.orchestra.run.smithi047.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-03-21T10:17:24.232 INFO:teuthology.orchestra.run.smithi047.stderr: stderr: 10+0 records in 2024-03-21T10:17:24.232 INFO:teuthology.orchestra.run.smithi047.stderr:10+0 records out 2024-03-21T10:17:24.232 INFO:teuthology.orchestra.run.smithi047.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0378637 s, 277 MB/s 2024-03-21T10:17:24.232 INFO:teuthology.orchestra.run.smithi047.stderr:--> Zapping successful for: 2024-03-21T10:17:24.239 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:17:23 smithi157 ceph-mon[35581]: osdmap e41: 6 total, 6 up, 6 in 2024-03-21T10:17:24.899 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:17:24 smithi047 ceph-mon[31547]: osdmap e42: 6 total, 6 up, 6 in 2024-03-21T10:17:24.899 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:17:24 smithi047 ceph-mon[31547]: pgmap v181: 1 pgs: 1 peering; 577 KiB data, 182 MiB used, 536 GiB / 536 GiB avail 2024-03-21T10:17:25.141 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2edfee68ad7243bed422a049b6f5f58619300f11 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3056d51a-e76b-11ee-95cd-87774f69a715 -- ceph orch daemon add osd smithi047:vg_nvme/lv_2 2024-03-21T10:17:25.229 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:17:24 smithi100 ceph-mon[35193]: osdmap e42: 6 total, 6 up, 6 in 2024-03-21T10:17:25.229 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:17:24 smithi100 ceph-mon[35193]: pgmap v181: 1 pgs: 1 peering; 577 KiB data, 182 MiB used, 536 GiB / 536 GiB avail 2024-03-21T10:17:25.239 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:17:24 smithi157 ceph-mon[35581]: osdmap e42: 6 total, 6 up, 6 in 2024-03-21T10:17:25.240 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:17:24 smithi157 ceph-mon[35581]: pgmap v181: 1 pgs: 1 peering; 577 KiB data, 182 MiB used, 536 GiB / 536 GiB avail 2024-03-21T10:17:25.585 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/3056d51a-e76b-11ee-95cd-87774f69a715/mon.a/config 2024-03-21T10:17:26.047 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:17:25 smithi047 ceph-mon[31547]: osdmap e43: 6 total, 6 up, 6 in 2024-03-21T10:17:26.047 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:17:25 smithi047 ceph-mon[31547]: Detected new or changed devices on smithi157 2024-03-21T10:17:26.047 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:17:25 smithi047 ceph-mon[31547]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:17:26.048 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:17:25 smithi047 ceph-mon[31547]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:17:26.048 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:17:25 smithi047 ceph-mon[31547]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-21T10:17:26.048 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:17:25 smithi047 ceph-mon[31547]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-21T10:17:26.048 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:17:25 smithi047 ceph-mon[31547]: Adjusting osd_memory_target on smithi157 to 8545M 2024-03-21T10:17:26.048 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:17:25 smithi047 ceph-mon[31547]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:17:26.048 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:17:25 smithi047 ceph-mon[31547]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T10:17:26.048 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:17:25 smithi047 ceph-mon[31547]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T10:17:26.048 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:17:25 smithi047 ceph-mon[31547]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:17:26.230 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:17:25 smithi100 ceph-mon[35193]: osdmap e43: 6 total, 6 up, 6 in 2024-03-21T10:17:26.230 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:17:25 smithi100 ceph-mon[35193]: Detected new or changed devices on smithi157 2024-03-21T10:17:26.230 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:17:25 smithi100 ceph-mon[35193]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:17:26.230 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:17:25 smithi100 ceph-mon[35193]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:17:26.230 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:17:25 smithi100 ceph-mon[35193]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-21T10:17:26.230 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:17:25 smithi100 ceph-mon[35193]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-21T10:17:26.230 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:17:25 smithi100 ceph-mon[35193]: Adjusting osd_memory_target on smithi157 to 8545M 2024-03-21T10:17:26.230 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:17:25 smithi100 ceph-mon[35193]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:17:26.230 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:17:25 smithi100 ceph-mon[35193]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T10:17:26.230 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:17:25 smithi100 ceph-mon[35193]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T10:17:26.231 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:17:25 smithi100 ceph-mon[35193]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:17:26.240 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:17:25 smithi157 ceph-mon[35581]: osdmap e43: 6 total, 6 up, 6 in 2024-03-21T10:17:26.240 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:17:25 smithi157 ceph-mon[35581]: Detected new or changed devices on smithi157 2024-03-21T10:17:26.240 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:17:25 smithi157 ceph-mon[35581]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:17:26.240 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:17:25 smithi157 ceph-mon[35581]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:17:26.240 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:17:25 smithi157 ceph-mon[35581]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-21T10:17:26.240 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:17:25 smithi157 ceph-mon[35581]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-21T10:17:26.240 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:17:25 smithi157 ceph-mon[35581]: Adjusting osd_memory_target on smithi157 to 8545M 2024-03-21T10:17:26.241 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:17:25 smithi157 ceph-mon[35581]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:17:26.241 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:17:25 smithi157 ceph-mon[35581]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T10:17:26.241 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:17:25 smithi157 ceph-mon[35581]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T10:17:26.241 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:17:25 smithi157 ceph-mon[35581]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:17:27.214 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:17:26 smithi047 ceph-mon[31547]: osdmap e44: 6 total, 6 up, 6 in 2024-03-21T10:17:27.214 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:17:26 smithi047 ceph-mon[31547]: pgmap v184: 1 pgs: 1 peering; 577 KiB data, 182 MiB used, 536 GiB / 536 GiB avail 2024-03-21T10:17:27.229 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:17:26 smithi100 ceph-mon[35193]: osdmap e44: 6 total, 6 up, 6 in 2024-03-21T10:17:27.229 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:17:26 smithi100 ceph-mon[35193]: pgmap v184: 1 pgs: 1 peering; 577 KiB data, 182 MiB used, 536 GiB / 536 GiB avail 2024-03-21T10:17:27.240 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:17:26 smithi157 ceph-mon[35581]: osdmap e44: 6 total, 6 up, 6 in 2024-03-21T10:17:27.240 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:17:26 smithi157 ceph-mon[35581]: pgmap v184: 1 pgs: 1 peering; 577 KiB data, 182 MiB used, 536 GiB / 536 GiB avail 2024-03-21T10:17:27.943 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:17:27.940+0000 7f3e43200700 1 -- 172.21.15.47:0/277971835 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3e3c102c50 msgr2=0x7f3e3c105040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:17:27.943 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:17:27.940+0000 7f3e43200700 1 --2- 172.21.15.47:0/277971835 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3e3c102c50 0x7f3e3c105040 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f3e2c009990 tx=0x7f3e2c009ca0 comp rx=0 tx=0).stop 2024-03-21T10:17:27.943 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:17:27.940+0000 7f3e43200700 1 -- 172.21.15.47:0/277971835 shutdown_connections 2024-03-21T10:17:27.943 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:17:27.940+0000 7f3e43200700 1 --2- 172.21.15.47:0/277971835 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3e3c107eb0 0x7f3e3c10a320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:17:27.943 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:17:27.940+0000 7f3e43200700 1 --2- 172.21.15.47:0/277971835 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3e3c102c50 0x7f3e3c105040 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:17:27.943 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:17:27.940+0000 7f3e43200700 1 --2- 172.21.15.47:0/277971835 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3e3c105580 0x7f3e3c107970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:17:27.943 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:17:27.940+0000 7f3e43200700 1 -- 172.21.15.47:0/277971835 >> 172.21.15.47:0/277971835 conn(0x7f3e3c0fc660 msgr2=0x7f3e3c0feaa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:17:27.943 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:17:27.940+0000 7f3e43200700 1 -- 172.21.15.47:0/277971835 shutdown_connections 2024-03-21T10:17:27.943 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:17:27.940+0000 7f3e43200700 1 -- 172.21.15.47:0/277971835 wait complete. 2024-03-21T10:17:27.945 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:17:27.941+0000 7f3e43200700 1 Processor -- start 2024-03-21T10:17:27.945 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:17:27.942+0000 7f3e43200700 1 -- start start 2024-03-21T10:17:27.945 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:17:27.942+0000 7f3e43200700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3e3c102c50 0x7f3e3c19a570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:17:27.945 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:17:27.942+0000 7f3e43200700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3e3c105580 0x7f3e3c19aab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:17:27.945 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:17:27.942+0000 7f3e43200700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3e3c107eb0 0x7f3e3c1a1b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:17:27.946 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:17:27.942+0000 7f3e43200700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e3c10dcd0 con 0x7f3e3c107eb0 2024-03-21T10:17:27.946 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:17:27.942+0000 7f3e43200700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e3c10db50 con 0x7f3e3c105580 2024-03-21T10:17:27.946 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:17:27.942+0000 7f3e43200700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e3c10de50 con 0x7f3e3c102c50 2024-03-21T10:17:27.946 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:17:27.942+0000 7f3e41200700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3e3c107eb0 0x7f3e3c1a1b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:17:27.946 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:17:27.943+0000 7f3e41200700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3e3c107eb0 0x7f3e3c1a1b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:57954/0 (socket says 172.21.15.47:57954) 2024-03-21T10:17:27.946 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:17:27.943+0000 7f3e41200700 1 -- 172.21.15.47:0/313257122 learned_addr learned my addr 172.21.15.47:0/313257122 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:17:27.946 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:17:27.943+0000 7f3e3be00700 1 --2- 172.21.15.47:0/313257122 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3e3c102c50 0x7f3e3c19a570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:17:27.946 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:17:27.943+0000 7f3e3b400700 1 --2- 172.21.15.47:0/313257122 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3e3c105580 0x7f3e3c19aab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:17:27.946 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:17:27.943+0000 7f3e41200700 1 -- 172.21.15.47:0/313257122 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3e3c102c50 msgr2=0x7f3e3c19a570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:17:27.946 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:17:27.943+0000 7f3e41200700 1 --2- 172.21.15.47:0/313257122 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3e3c102c50 0x7f3e3c19a570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:17:27.946 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:17:27.943+0000 7f3e41200700 1 -- 172.21.15.47:0/313257122 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3e3c105580 msgr2=0x7f3e3c19aab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:17:27.946 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:17:27.943+0000 7f3e41200700 1 --2- 172.21.15.47:0/313257122 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3e3c105580 0x7f3e3c19aab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:17:27.946 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:17:27.943+0000 7f3e41200700 1 -- 172.21.15.47:0/313257122 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e3c1a21d0 con 0x7f3e3c107eb0 2024-03-21T10:17:27.947 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:17:27.943+0000 7f3e41200700 1 --2- 172.21.15.47:0/313257122 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3e3c107eb0 0x7f3e3c1a1b30 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f3e34007ee0 tx=0x7f3e3400d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:17:27.947 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:17:27.944+0000 7f3e38c00700 1 -- 172.21.15.47:0/313257122 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e34004db0 con 0x7f3e3c107eb0 2024-03-21T10:17:27.947 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:17:27.944+0000 7f3e38c00700 1 -- 172.21.15.47:0/313257122 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f3e3400dbd0 con 0x7f3e3c107eb0 2024-03-21T10:17:27.948 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:17:27.944+0000 7f3e43200700 1 -- 172.21.15.47:0/313257122 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3e3c1a24c0 con 0x7f3e3c107eb0 2024-03-21T10:17:27.948 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:17:27.944+0000 7f3e38c00700 1 -- 172.21.15.47:0/313257122 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e34015400 con 0x7f3e3c107eb0 2024-03-21T10:17:27.948 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:17:27.944+0000 7f3e43200700 1 -- 172.21.15.47:0/313257122 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3e3c1a2a00 con 0x7f3e3c107eb0 2024-03-21T10:17:27.948 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:17:27.945+0000 7f3e43200700 1 -- 172.21.15.47:0/313257122 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3e3c068e50 con 0x7f3e3c107eb0 2024-03-21T10:17:27.949 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:17:27.947+0000 7f3e38c00700 1 -- 172.21.15.47:0/313257122 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 16) v1 ==== 137854+0+0 (secure 0 0 0) 0x7f3e34005000 con 0x7f3e3c107eb0 2024-03-21T10:17:27.952 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:17:27.950+0000 7f3e38c00700 1 --2- 172.21.15.47:0/313257122 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f3e200a2fb0 0x7f3e200a5470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:17:27.952 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:17:27.950+0000 7f3e38c00700 1 -- 172.21.15.47:0/313257122 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 4533+0+0 (secure 0 0 0) 0x7f3e340ce880 con 0x7f3e3c107eb0 2024-03-21T10:17:27.952 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:17:27.950+0000 7f3e3be00700 1 --2- 172.21.15.47:0/313257122 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f3e200a2fb0 0x7f3e200a5470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:17:27.953 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:17:27.951+0000 7f3e3be00700 1 --2- 172.21.15.47:0/313257122 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f3e200a2fb0 0x7f3e200a5470 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f3e2c005950 tx=0x7f3e2c0058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:17:27.955 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:17:27.954+0000 7f3e38c00700 1 -- 172.21.15.47:0/313257122 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f3e34010070 con 0x7f3e3c107eb0 2024-03-21T10:17:28.195 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:17:28.192+0000 7f3e43200700 1 -- 172.21.15.47:0/313257122 --> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi047:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) v1 -- 0x7f3e3c0625d0 con 0x7f3e200a2fb0 2024-03-21T10:17:29.240 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:17:28 smithi157 ceph-mon[35581]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail; 113 KiB/s, 0 objects/s recovering 2024-03-21T10:17:29.240 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:17:28 smithi157 ceph-mon[35581]: from='client.14340 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi047:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:17:29.240 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:17:28 smithi157 ceph-mon[35581]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-21T10:17:29.240 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:17:28 smithi157 ceph-mon[35581]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-21T10:17:29.240 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:17:28 smithi157 ceph-mon[35581]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T10:17:29.311 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:17:28 smithi047 ceph-mon[31547]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail; 113 KiB/s, 0 objects/s recovering 2024-03-21T10:17:29.311 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:17:28 smithi047 ceph-mon[31547]: from='client.14340 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi047:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:17:29.311 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:17:28 smithi047 ceph-mon[31547]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-21T10:17:29.311 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:17:28 smithi047 ceph-mon[31547]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-21T10:17:29.311 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:17:28 smithi047 ceph-mon[31547]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T10:17:29.479 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:17:28 smithi100 ceph-mon[35193]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail; 113 KiB/s, 0 objects/s recovering 2024-03-21T10:17:29.479 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:17:28 smithi100 ceph-mon[35193]: from='client.14340 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi047:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:17:29.480 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:17:28 smithi100 ceph-mon[35193]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-21T10:17:29.480 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:17:28 smithi100 ceph-mon[35193]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-21T10:17:29.480 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:17:28 smithi100 ceph-mon[35193]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T10:17:31.175 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:17:30 smithi047 ceph-mon[31547]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail; 95 KiB/s, 0 objects/s recovering 2024-03-21T10:17:31.479 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:17:30 smithi100 ceph-mon[35193]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail; 95 KiB/s, 0 objects/s recovering 2024-03-21T10:17:31.489 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:17:30 smithi157 ceph-mon[35581]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail; 95 KiB/s, 0 objects/s recovering 2024-03-21T10:17:33.214 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:17:32 smithi047 ceph-mon[31547]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-03-21T10:17:33.214 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:17:32 smithi047 ceph-mon[31547]: from='client.? 172.21.15.47:0/3547868604' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c12c14b4-85f0-4d88-937c-de4f556437de"}]: dispatch 2024-03-21T10:17:33.214 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:17:32 smithi047 ceph-mon[31547]: from='client.? 172.21.15.47:0/3547868604' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "c12c14b4-85f0-4d88-937c-de4f556437de"}]': finished 2024-03-21T10:17:33.214 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:17:32 smithi047 ceph-mon[31547]: osdmap e45: 7 total, 6 up, 7 in 2024-03-21T10:17:33.214 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:17:32 smithi047 ceph-mon[31547]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-21T10:17:33.479 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:17:32 smithi100 ceph-mon[35193]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-03-21T10:17:33.480 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:17:32 smithi100 ceph-mon[35193]: from='client.? 172.21.15.47:0/3547868604' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c12c14b4-85f0-4d88-937c-de4f556437de"}]: dispatch 2024-03-21T10:17:33.480 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:17:32 smithi100 ceph-mon[35193]: from='client.? 172.21.15.47:0/3547868604' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "c12c14b4-85f0-4d88-937c-de4f556437de"}]': finished 2024-03-21T10:17:33.480 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:17:32 smithi100 ceph-mon[35193]: osdmap e45: 7 total, 6 up, 7 in 2024-03-21T10:17:33.480 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:17:32 smithi100 ceph-mon[35193]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-21T10:17:33.490 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:17:32 smithi157 ceph-mon[35581]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-03-21T10:17:33.490 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:17:32 smithi157 ceph-mon[35581]: from='client.? 172.21.15.47:0/3547868604' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c12c14b4-85f0-4d88-937c-de4f556437de"}]: dispatch 2024-03-21T10:17:33.490 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:17:32 smithi157 ceph-mon[35581]: from='client.? 172.21.15.47:0/3547868604' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "c12c14b4-85f0-4d88-937c-de4f556437de"}]': finished 2024-03-21T10:17:33.490 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:17:32 smithi157 ceph-mon[35581]: osdmap e45: 7 total, 6 up, 7 in 2024-03-21T10:17:33.490 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:17:32 smithi157 ceph-mon[35581]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-21T10:17:34.314 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:17:34 smithi047 ceph-mon[31547]: from='client.? 172.21.15.47:0/702596389' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-21T10:17:34.479 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:17:34 smithi100 ceph-mon[35193]: from='client.? 172.21.15.47:0/702596389' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-21T10:17:34.489 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:17:34 smithi157 ceph-mon[35581]: from='client.? 172.21.15.47:0/702596389' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-21T10:17:35.464 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:17:35 smithi047 ceph-mon[31547]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-03-21T10:17:35.479 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:17:35 smithi100 ceph-mon[35193]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-03-21T10:17:35.489 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:17:35 smithi157 ceph-mon[35581]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-03-21T10:17:37.465 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:17:37 smithi047 ceph-mon[31547]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-03-21T10:17:37.479 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:17:37 smithi100 ceph-mon[35193]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-03-21T10:17:37.490 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:17:37 smithi157 ceph-mon[35581]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-03-21T10:17:39.213 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:17:39 smithi047 ceph-mon[31547]: pgmap v191: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail 2024-03-21T10:17:39.292 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:17:39 smithi157 ceph-mon[35581]: pgmap v191: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail 2024-03-21T10:17:39.479 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:17:39 smithi100 ceph-mon[35193]: pgmap v191: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail 2024-03-21T10:17:41.239 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:17:41 smithi157 ceph-mon[35581]: pgmap v192: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail 2024-03-21T10:17:41.442 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:17:41 smithi047 ceph-mon[31547]: pgmap v192: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail 2024-03-21T10:17:41.479 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:17:41 smithi100 ceph-mon[35193]: pgmap v192: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail 2024-03-21T10:17:42.213 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:17:42 smithi047 ceph-mon[31547]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-03-21T10:17:42.214 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:17:42 smithi047 ceph-mon[31547]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T10:17:42.479 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:17:42 smithi100 ceph-mon[35193]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-03-21T10:17:42.479 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:17:42 smithi100 ceph-mon[35193]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T10:17:42.489 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:17:42 smithi157 ceph-mon[35581]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-03-21T10:17:42.489 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:17:42 smithi157 ceph-mon[35581]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T10:17:43.335 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:17:43 smithi047 ceph-mon[31547]: pgmap v193: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail 2024-03-21T10:17:43.335 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:17:43 smithi047 ceph-mon[31547]: Deploying daemon osd.6 on smithi047 2024-03-21T10:17:43.479 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:17:43 smithi100 ceph-mon[35193]: pgmap v193: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail 2024-03-21T10:17:43.479 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:17:43 smithi100 ceph-mon[35193]: Deploying daemon osd.6 on smithi047 2024-03-21T10:17:43.489 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:17:43 smithi157 ceph-mon[35581]: pgmap v193: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail 2024-03-21T10:17:43.489 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:17:43 smithi157 ceph-mon[35581]: Deploying daemon osd.6 on smithi047 2024-03-21T10:17:45.464 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:17:45 smithi047 ceph-mon[31547]: pgmap v194: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail 2024-03-21T10:17:45.479 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:17:45 smithi100 ceph-mon[35193]: pgmap v194: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail 2024-03-21T10:17:45.489 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:17:45 smithi157 ceph-mon[35581]: pgmap v194: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail 2024-03-21T10:17:47.464 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:17:47 smithi047 ceph-mon[31547]: pgmap v195: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail 2024-03-21T10:17:47.479 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:17:47 smithi100 ceph-mon[35193]: pgmap v195: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail 2024-03-21T10:17:47.489 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:17:47 smithi157 ceph-mon[35581]: pgmap v195: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail 2024-03-21T10:17:49.464 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:17:49 smithi047 ceph-mon[31547]: pgmap v196: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail 2024-03-21T10:17:49.479 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:17:49 smithi100 ceph-mon[35193]: pgmap v196: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail 2024-03-21T10:17:49.489 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:17:49 smithi157 ceph-mon[35581]: pgmap v196: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail 2024-03-21T10:17:51.240 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:17:51 smithi157 ceph-mon[35581]: pgmap v197: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail 2024-03-21T10:17:51.240 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:17:51 smithi157 ceph-mon[35581]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:17:51.240 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:17:51 smithi157 ceph-mon[35581]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T10:17:51.240 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:17:51 smithi157 ceph-mon[35581]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:17:51.383 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:17:51 smithi047 ceph-mon[31547]: pgmap v197: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail 2024-03-21T10:17:51.383 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:17:51 smithi047 ceph-mon[31547]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:17:51.383 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:17:51 smithi047 ceph-mon[31547]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T10:17:51.384 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:17:51 smithi047 ceph-mon[31547]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:17:51.479 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:17:51 smithi100 ceph-mon[35193]: pgmap v197: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail 2024-03-21T10:17:51.480 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:17:51 smithi100 ceph-mon[35193]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:17:51.480 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:17:51 smithi100 ceph-mon[35193]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T10:17:51.480 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:17:51 smithi100 ceph-mon[35193]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:17:53.464 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:17:53 smithi047 ceph-mon[31547]: pgmap v198: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail 2024-03-21T10:17:53.479 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:17:53 smithi100 ceph-mon[35193]: pgmap v198: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail 2024-03-21T10:17:53.489 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:17:53 smithi157 ceph-mon[35581]: pgmap v198: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail 2024-03-21T10:17:55.964 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:17:55 smithi047 ceph-mon[31547]: pgmap v199: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail 2024-03-21T10:17:55.964 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:17:55 smithi047 ceph-mon[31547]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:17:55.964 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:17:55 smithi047 ceph-mon[31547]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:17:55.964 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:17:55 smithi047 ceph-mon[31547]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T10:17:55.964 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:17:55 smithi047 ceph-mon[31547]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T10:17:55.964 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:17:55 smithi047 ceph-mon[31547]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:17:55.964 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:17:55 smithi047 ceph-mon[31547]: from='osd.6 [v2:172.21.15.47:6818/2321642790,v1:172.21.15.47:6819/2321642790]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-03-21T10:17:56.229 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:17:55 smithi100 ceph-mon[35193]: pgmap v199: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail 2024-03-21T10:17:56.230 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:17:55 smithi100 ceph-mon[35193]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:17:56.230 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:17:55 smithi100 ceph-mon[35193]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:17:56.230 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:17:55 smithi100 ceph-mon[35193]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T10:17:56.230 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:17:55 smithi100 ceph-mon[35193]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T10:17:56.230 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:17:55 smithi100 ceph-mon[35193]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:17:56.230 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:17:55 smithi100 ceph-mon[35193]: from='osd.6 [v2:172.21.15.47:6818/2321642790,v1:172.21.15.47:6819/2321642790]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-03-21T10:17:56.240 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:17:55 smithi157 ceph-mon[35581]: pgmap v199: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail 2024-03-21T10:17:56.240 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:17:55 smithi157 ceph-mon[35581]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:17:56.240 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:17:55 smithi157 ceph-mon[35581]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:17:56.240 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:17:55 smithi157 ceph-mon[35581]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T10:17:56.240 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:17:55 smithi157 ceph-mon[35581]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T10:17:56.240 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:17:55 smithi157 ceph-mon[35581]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:17:56.240 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:17:55 smithi157 ceph-mon[35581]: from='osd.6 [v2:172.21.15.47:6818/2321642790,v1:172.21.15.47:6819/2321642790]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-03-21T10:17:56.257 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:17:56.256+0000 7f3e38c00700 1 -- 172.21.15.47:0/313257122 <== mgr.14152 v2:172.21.15.47:6800/2823885994 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f3e3c0625d0 con 0x7f3e200a2fb0 2024-03-21T10:17:56.258 INFO:teuthology.orchestra.run.smithi047.stdout:Created osd(s) 6 on host 'smithi047' 2024-03-21T10:17:56.262 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:17:56.260+0000 7f3e43200700 1 -- 172.21.15.47:0/313257122 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f3e200a2fb0 msgr2=0x7f3e200a5470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:17:56.262 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:17:56.260+0000 7f3e43200700 1 --2- 172.21.15.47:0/313257122 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f3e200a2fb0 0x7f3e200a5470 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f3e2c005950 tx=0x7f3e2c0058e0 comp rx=0 tx=0).stop 2024-03-21T10:17:56.262 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:17:56.260+0000 7f3e43200700 1 -- 172.21.15.47:0/313257122 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3e3c107eb0 msgr2=0x7f3e3c1a1b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:17:56.262 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:17:56.260+0000 7f3e43200700 1 --2- 172.21.15.47:0/313257122 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3e3c107eb0 0x7f3e3c1a1b30 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f3e34007ee0 tx=0x7f3e3400d3b0 comp rx=0 tx=0).stop 2024-03-21T10:17:56.262 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:17:56.260+0000 7f3e43200700 1 -- 172.21.15.47:0/313257122 shutdown_connections 2024-03-21T10:17:56.262 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:17:56.260+0000 7f3e43200700 1 --2- 172.21.15.47:0/313257122 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f3e200a2fb0 0x7f3e200a5470 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:17:56.262 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:17:56.260+0000 7f3e43200700 1 --2- 172.21.15.47:0/313257122 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3e3c107eb0 0x7f3e3c1a1b30 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:17:56.262 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:17:56.260+0000 7f3e43200700 1 --2- 172.21.15.47:0/313257122 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3e3c102c50 0x7f3e3c19a570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:17:56.262 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:17:56.260+0000 7f3e43200700 1 --2- 172.21.15.47:0/313257122 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3e3c105580 0x7f3e3c19aab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:17:56.262 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:17:56.260+0000 7f3e43200700 1 -- 172.21.15.47:0/313257122 >> 172.21.15.47:0/313257122 conn(0x7f3e3c0fc660 msgr2=0x7f3e3c1023c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:17:56.263 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:17:56.260+0000 7f3e43200700 1 -- 172.21.15.47:0/313257122 shutdown_connections 2024-03-21T10:17:56.263 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:17:56.260+0000 7f3e43200700 1 -- 172.21.15.47:0/313257122 wait complete. 2024-03-21T10:17:56.964 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:17:56 smithi047 ceph-mon[31547]: from='osd.6 [v2:172.21.15.47:6818/2321642790,v1:172.21.15.47:6819/2321642790]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-03-21T10:17:56.964 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:17:56 smithi047 ceph-mon[31547]: osdmap e46: 7 total, 6 up, 7 in 2024-03-21T10:17:56.964 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:17:56 smithi047 ceph-mon[31547]: from='osd.6 [v2:172.21.15.47:6818/2321642790,v1:172.21.15.47:6819/2321642790]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi047", "root=default"]}]: dispatch 2024-03-21T10:17:56.964 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:17:56 smithi047 ceph-mon[31547]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-21T10:17:56.964 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:17:56 smithi047 ceph-mon[31547]: pgmap v201: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail 2024-03-21T10:17:56.964 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:17:56 smithi047 ceph-mon[31547]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:17:56.964 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:17:56 smithi047 ceph-mon[31547]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:17:56.964 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:17:56 smithi047 ceph-mon[31547]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T10:17:57.036 DEBUG:teuthology.orchestra.run.smithi047:osd.6> sudo journalctl -f -n 0 -u ceph-3056d51a-e76b-11ee-95cd-87774f69a715@osd.6.service 2024-03-21T10:17:57.041 INFO:tasks.cephadm:Deploying osd.7 on smithi100 with /dev/vg_nvme/lv_2... 2024-03-21T10:17:57.041 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2edfee68ad7243bed422a049b6f5f58619300f11 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3056d51a-e76b-11ee-95cd-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-03-21T10:17:57.130 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:17:56 smithi100 ceph-mon[35193]: from='osd.6 [v2:172.21.15.47:6818/2321642790,v1:172.21.15.47:6819/2321642790]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-03-21T10:17:57.130 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:17:56 smithi100 ceph-mon[35193]: osdmap e46: 7 total, 6 up, 7 in 2024-03-21T10:17:57.130 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:17:56 smithi100 ceph-mon[35193]: from='osd.6 [v2:172.21.15.47:6818/2321642790,v1:172.21.15.47:6819/2321642790]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi047", "root=default"]}]: dispatch 2024-03-21T10:17:57.130 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:17:56 smithi100 ceph-mon[35193]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-21T10:17:57.131 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:17:56 smithi100 ceph-mon[35193]: pgmap v201: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail 2024-03-21T10:17:57.131 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:17:56 smithi100 ceph-mon[35193]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:17:57.131 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:17:56 smithi100 ceph-mon[35193]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:17:57.131 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:17:56 smithi100 ceph-mon[35193]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T10:17:57.240 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:17:56 smithi157 ceph-mon[35581]: from='osd.6 [v2:172.21.15.47:6818/2321642790,v1:172.21.15.47:6819/2321642790]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-03-21T10:17:57.240 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:17:56 smithi157 ceph-mon[35581]: osdmap e46: 7 total, 6 up, 7 in 2024-03-21T10:17:57.240 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:17:56 smithi157 ceph-mon[35581]: from='osd.6 [v2:172.21.15.47:6818/2321642790,v1:172.21.15.47:6819/2321642790]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi047", "root=default"]}]: dispatch 2024-03-21T10:17:57.240 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:17:56 smithi157 ceph-mon[35581]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-21T10:17:57.240 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:17:56 smithi157 ceph-mon[35581]: pgmap v201: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail 2024-03-21T10:17:57.240 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:17:56 smithi157 ceph-mon[35581]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:17:57.240 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:17:56 smithi157 ceph-mon[35581]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:17:57.241 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:17:56 smithi157 ceph-mon[35581]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T10:17:57.488 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/3056d51a-e76b-11ee-95cd-87774f69a715/mon.b/config 2024-03-21T10:17:57.960 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:17:57 smithi100 ceph-mon[35193]: from='osd.6 [v2:172.21.15.47:6818/2321642790,v1:172.21.15.47:6819/2321642790]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi047", "root=default"]}]': finished 2024-03-21T10:17:57.961 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:17:57 smithi100 ceph-mon[35193]: osdmap e47: 7 total, 6 up, 7 in 2024-03-21T10:17:57.961 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:17:57 smithi100 ceph-mon[35193]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-21T10:17:57.961 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:17:57 smithi100 ceph-mon[35193]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-21T10:17:57.961 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:17:57 smithi100 ceph-mon[35193]: from='osd.6 [v2:172.21.15.47:6818/2321642790,v1:172.21.15.47:6819/2321642790]' entity='osd.6' 2024-03-21T10:17:57.964 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:17:57 smithi047 ceph-mon[31547]: from='osd.6 [v2:172.21.15.47:6818/2321642790,v1:172.21.15.47:6819/2321642790]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi047", "root=default"]}]': finished 2024-03-21T10:17:57.964 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:17:57 smithi047 ceph-mon[31547]: osdmap e47: 7 total, 6 up, 7 in 2024-03-21T10:17:57.964 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:17:57 smithi047 ceph-mon[31547]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-21T10:17:57.964 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:17:57 smithi047 ceph-mon[31547]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-21T10:17:57.964 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:17:57 smithi047 ceph-mon[31547]: from='osd.6 [v2:172.21.15.47:6818/2321642790,v1:172.21.15.47:6819/2321642790]' entity='osd.6' 2024-03-21T10:17:57.965 INFO:journalctl@ceph.osd.6.smithi047.stdout:Mar 21 10:17:57 smithi047 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-6[57372]: 2024-03-21T10:17:57.651+0000 7f1587000700 -1 osd.6 0 waiting for initial osdmap 2024-03-21T10:17:57.965 INFO:journalctl@ceph.osd.6.smithi047.stdout:Mar 21 10:17:57 smithi047 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-6[57372]: 2024-03-21T10:17:57.677+0000 7f157c400700 -1 osd.6 47 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-21T10:17:58.240 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:17:57 smithi157 ceph-mon[35581]: from='osd.6 [v2:172.21.15.47:6818/2321642790,v1:172.21.15.47:6819/2321642790]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi047", "root=default"]}]': finished 2024-03-21T10:17:58.240 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:17:57 smithi157 ceph-mon[35581]: osdmap e47: 7 total, 6 up, 7 in 2024-03-21T10:17:58.240 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:17:57 smithi157 ceph-mon[35581]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-21T10:17:58.240 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:17:57 smithi157 ceph-mon[35581]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-21T10:17:58.240 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:17:57 smithi157 ceph-mon[35581]: from='osd.6 [v2:172.21.15.47:6818/2321642790,v1:172.21.15.47:6819/2321642790]' entity='osd.6' 2024-03-21T10:17:59.166 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:17:58 smithi100 ceph-mon[35193]: purged_snaps scrub starts 2024-03-21T10:17:59.166 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:17:58 smithi100 ceph-mon[35193]: purged_snaps scrub ok 2024-03-21T10:17:59.166 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:17:58 smithi100 ceph-mon[35193]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-21T10:17:59.166 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:17:58 smithi100 ceph-mon[35193]: pgmap v203: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail 2024-03-21T10:17:59.166 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:17:58 smithi100 ceph-mon[35193]: osd.6 [v2:172.21.15.47:6818/2321642790,v1:172.21.15.47:6819/2321642790] boot 2024-03-21T10:17:59.166 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:17:58 smithi100 ceph-mon[35193]: osdmap e48: 7 total, 7 up, 7 in 2024-03-21T10:17:59.166 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:17:58 smithi100 ceph-mon[35193]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-21T10:17:59.182 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:17:58 smithi047 ceph-mon[31547]: purged_snaps scrub starts 2024-03-21T10:17:59.182 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:17:58 smithi047 ceph-mon[31547]: purged_snaps scrub ok 2024-03-21T10:17:59.182 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:17:58 smithi047 ceph-mon[31547]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-21T10:17:59.183 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:17:58 smithi047 ceph-mon[31547]: pgmap v203: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail 2024-03-21T10:17:59.183 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:17:58 smithi047 ceph-mon[31547]: osd.6 [v2:172.21.15.47:6818/2321642790,v1:172.21.15.47:6819/2321642790] boot 2024-03-21T10:17:59.183 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:17:58 smithi047 ceph-mon[31547]: osdmap e48: 7 total, 7 up, 7 in 2024-03-21T10:17:59.183 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:17:58 smithi047 ceph-mon[31547]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-21T10:17:59.240 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:17:58 smithi157 ceph-mon[35581]: purged_snaps scrub starts 2024-03-21T10:17:59.240 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:17:58 smithi157 ceph-mon[35581]: purged_snaps scrub ok 2024-03-21T10:17:59.240 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:17:58 smithi157 ceph-mon[35581]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-21T10:17:59.240 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:17:58 smithi157 ceph-mon[35581]: pgmap v203: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail 2024-03-21T10:17:59.240 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:17:58 smithi157 ceph-mon[35581]: osd.6 [v2:172.21.15.47:6818/2321642790,v1:172.21.15.47:6819/2321642790] boot 2024-03-21T10:17:59.240 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:17:58 smithi157 ceph-mon[35581]: osdmap e48: 7 total, 7 up, 7 in 2024-03-21T10:17:59.240 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:17:58 smithi157 ceph-mon[35581]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-21T10:18:00.380 INFO:teuthology.orchestra.run.smithi100.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-03-21T10:18:00.380 INFO:teuthology.orchestra.run.smithi100.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-03-21T10:18:00.380 INFO:teuthology.orchestra.run.smithi100.stderr: stderr: 10+0 records in 2024-03-21T10:18:00.381 INFO:teuthology.orchestra.run.smithi100.stderr:10+0 records out 2024-03-21T10:18:00.381 INFO:teuthology.orchestra.run.smithi100.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0335611 s, 312 MB/s 2024-03-21T10:18:00.381 INFO:teuthology.orchestra.run.smithi100.stderr:--> Zapping successful for: 2024-03-21T10:18:00.964 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:18:00 smithi047 ceph-mon[31547]: osdmap e49: 7 total, 7 up, 7 in 2024-03-21T10:18:00.979 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:18:00 smithi100 ceph-mon[35193]: osdmap e49: 7 total, 7 up, 7 in 2024-03-21T10:18:00.989 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:18:00 smithi157 ceph-mon[35581]: osdmap e49: 7 total, 7 up, 7 in 2024-03-21T10:18:01.299 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2edfee68ad7243bed422a049b6f5f58619300f11 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3056d51a-e76b-11ee-95cd-87774f69a715 -- ceph orch daemon add osd smithi100:vg_nvme/lv_2 2024-03-21T10:18:01.724 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/3056d51a-e76b-11ee-95cd-87774f69a715/mon.b/config 2024-03-21T10:18:01.963 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:18:01 smithi047 ceph-mon[31547]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 218 MiB used, 626 GiB / 626 GiB avail 2024-03-21T10:18:01.979 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:18:01 smithi100 ceph-mon[35193]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 218 MiB used, 626 GiB / 626 GiB avail 2024-03-21T10:18:01.989 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:18:01 smithi157 ceph-mon[35581]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 218 MiB used, 626 GiB / 626 GiB avail 2024-03-21T10:18:03.364 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:18:03 smithi100 ceph-mon[35193]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 218 MiB used, 626 GiB / 626 GiB avail 2024-03-21T10:18:03.364 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:18:03 smithi100 ceph-mon[35193]: Detected new or changed devices on smithi047 2024-03-21T10:18:03.364 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:18:03 smithi100 ceph-mon[35193]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:18:03.365 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:18:03 smithi100 ceph-mon[35193]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:18:03.365 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:18:03 smithi100 ceph-mon[35193]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-21T10:18:03.365 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:18:03 smithi100 ceph-mon[35193]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-21T10:18:03.365 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:18:03 smithi100 ceph-mon[35193]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-21T10:18:03.365 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:18:03 smithi100 ceph-mon[35193]: Adjusting osd_memory_target on smithi047 to 5698M 2024-03-21T10:18:03.365 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:18:03 smithi100 ceph-mon[35193]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:18:03.365 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:18:03 smithi100 ceph-mon[35193]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T10:18:03.365 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:18:03 smithi100 ceph-mon[35193]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T10:18:03.365 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:18:03 smithi100 ceph-mon[35193]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:18:03.714 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:18:03 smithi047 ceph-mon[31547]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 218 MiB used, 626 GiB / 626 GiB avail 2024-03-21T10:18:03.714 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:18:03 smithi047 ceph-mon[31547]: Detected new or changed devices on smithi047 2024-03-21T10:18:03.714 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:18:03 smithi047 ceph-mon[31547]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:18:03.714 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:18:03 smithi047 ceph-mon[31547]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:18:03.714 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:18:03 smithi047 ceph-mon[31547]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-21T10:18:03.714 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:18:03 smithi047 ceph-mon[31547]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-21T10:18:03.715 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:18:03 smithi047 ceph-mon[31547]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-21T10:18:03.715 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:18:03 smithi047 ceph-mon[31547]: Adjusting osd_memory_target on smithi047 to 5698M 2024-03-21T10:18:03.715 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:18:03 smithi047 ceph-mon[31547]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:18:03.715 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:18:03 smithi047 ceph-mon[31547]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T10:18:03.715 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:18:03 smithi047 ceph-mon[31547]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T10:18:03.715 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:18:03 smithi047 ceph-mon[31547]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:18:03.740 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:18:03 smithi157 ceph-mon[35581]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 218 MiB used, 626 GiB / 626 GiB avail 2024-03-21T10:18:03.740 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:18:03 smithi157 ceph-mon[35581]: Detected new or changed devices on smithi047 2024-03-21T10:18:03.740 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:18:03 smithi157 ceph-mon[35581]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:18:03.740 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:18:03 smithi157 ceph-mon[35581]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:18:03.740 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:18:03 smithi157 ceph-mon[35581]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-21T10:18:03.740 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:18:03 smithi157 ceph-mon[35581]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-21T10:18:03.741 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:18:03 smithi157 ceph-mon[35581]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-21T10:18:03.741 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:18:03 smithi157 ceph-mon[35581]: Adjusting osd_memory_target on smithi047 to 5698M 2024-03-21T10:18:03.741 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:18:03 smithi157 ceph-mon[35581]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:18:03.741 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:18:03 smithi157 ceph-mon[35581]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T10:18:03.741 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:18:03 smithi157 ceph-mon[35581]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T10:18:03.741 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:18:03 smithi157 ceph-mon[35581]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:18:03.861 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-21T10:18:03.858+0000 7faa95a00700 1 -- 172.21.15.100:0/4290803260 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7faa90074bf0 msgr2=0x7faa90075070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:18:03.861 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-21T10:18:03.858+0000 7faa95a00700 1 --2- 172.21.15.100:0/4290803260 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7faa90074bf0 0x7faa90075070 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7faa7c009990 tx=0x7faa7c009ca0 comp rx=0 tx=0).stop 2024-03-21T10:18:03.861 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-21T10:18:03.859+0000 7faa95a00700 1 -- 172.21.15.100:0/4290803260 shutdown_connections 2024-03-21T10:18:03.861 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-21T10:18:03.859+0000 7faa95a00700 1 --2- 172.21.15.100:0/4290803260 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7faa9010a090 0x7faa9010c520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:18:03.861 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-21T10:18:03.859+0000 7faa95a00700 1 --2- 172.21.15.100:0/4290803260 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7faa900761c0 0x7faa90074620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:18:03.861 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-21T10:18:03.859+0000 7faa95a00700 1 --2- 172.21.15.100:0/4290803260 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7faa90074bf0 0x7faa90075070 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:18:03.862 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-21T10:18:03.859+0000 7faa95a00700 1 -- 172.21.15.100:0/4290803260 >> 172.21.15.100:0/4290803260 conn(0x7faa900fd740 msgr2=0x7faa900ffba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:18:03.862 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-21T10:18:03.859+0000 7faa95a00700 1 -- 172.21.15.100:0/4290803260 shutdown_connections 2024-03-21T10:18:03.862 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-21T10:18:03.859+0000 7faa95a00700 1 -- 172.21.15.100:0/4290803260 wait complete. 2024-03-21T10:18:03.863 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-21T10:18:03.860+0000 7faa95a00700 1 Processor -- start 2024-03-21T10:18:03.864 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-21T10:18:03.860+0000 7faa95a00700 1 -- start start 2024-03-21T10:18:03.864 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-21T10:18:03.860+0000 7faa95a00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7faa90074bf0 0x7faa9019a5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:18:03.864 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-21T10:18:03.860+0000 7faa95a00700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7faa900761c0 0x7faa9019aae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:18:03.864 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-21T10:18:03.860+0000 7faa95a00700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7faa9010a090 0x7faa901a1b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:18:03.864 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-21T10:18:03.860+0000 7faa95a00700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa9010fe50 con 0x7faa900761c0 2024-03-21T10:18:03.864 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-21T10:18:03.860+0000 7faa95a00700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa9010fcd0 con 0x7faa9010a090 2024-03-21T10:18:03.864 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-21T10:18:03.860+0000 7faa95a00700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa9010ffd0 con 0x7faa90074bf0 2024-03-21T10:18:03.864 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-21T10:18:03.861+0000 7faa8f400700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7faa9010a090 0x7faa901a1b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:18:03.864 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-21T10:18:03.861+0000 7faa8f400700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7faa9010a090 0x7faa901a1b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:35026/0 (socket says 172.21.15.100:35026) 2024-03-21T10:18:03.864 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-21T10:18:03.861+0000 7faa8f400700 1 -- 172.21.15.100:0/3705220278 learned_addr learned my addr 172.21.15.100:0/3705220278 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-21T10:18:03.864 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-21T10:18:03.861+0000 7faa8ea00700 1 --2- 172.21.15.100:0/3705220278 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7faa90074bf0 0x7faa9019a5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:18:03.864 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-21T10:18:03.861+0000 7faa8e000700 1 --2- 172.21.15.100:0/3705220278 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7faa900761c0 0x7faa9019aae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:18:03.865 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-21T10:18:03.861+0000 7faa8f400700 1 -- 172.21.15.100:0/3705220278 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7faa90074bf0 msgr2=0x7faa9019a5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:18:03.865 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-21T10:18:03.861+0000 7faa8f400700 1 --2- 172.21.15.100:0/3705220278 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7faa90074bf0 0x7faa9019a5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:18:03.865 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-21T10:18:03.861+0000 7faa8f400700 1 -- 172.21.15.100:0/3705220278 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7faa900761c0 msgr2=0x7faa9019aae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:18:03.865 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-21T10:18:03.861+0000 7faa8f400700 1 --2- 172.21.15.100:0/3705220278 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7faa900761c0 0x7faa9019aae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:18:03.865 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-21T10:18:03.861+0000 7faa8f400700 1 -- 172.21.15.100:0/3705220278 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa901a21b0 con 0x7faa9010a090 2024-03-21T10:18:03.865 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-21T10:18:03.861+0000 7faa8f400700 1 --2- 172.21.15.100:0/3705220278 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7faa9010a090 0x7faa901a1b10 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7faa88007ee0 tx=0x7faa8800d3b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:18:03.865 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-21T10:18:03.862+0000 7faa87400700 1 -- 172.21.15.100:0/3705220278 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa8800dbd0 con 0x7faa9010a090 2024-03-21T10:18:03.865 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-21T10:18:03.862+0000 7faa87400700 1 -- 172.21.15.100:0/3705220278 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7faa8800dd70 con 0x7faa9010a090 2024-03-21T10:18:03.865 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-21T10:18:03.862+0000 7faa87400700 1 -- 172.21.15.100:0/3705220278 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa88005a00 con 0x7faa9010a090 2024-03-21T10:18:03.865 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-21T10:18:03.862+0000 7faa95a00700 1 -- 172.21.15.100:0/3705220278 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faa901a24a0 con 0x7faa9010a090 2024-03-21T10:18:03.865 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-21T10:18:03.862+0000 7faa95a00700 1 -- 172.21.15.100:0/3705220278 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faa901a2990 con 0x7faa9010a090 2024-03-21T10:18:03.866 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-21T10:18:03.863+0000 7faa95a00700 1 -- 172.21.15.100:0/3705220278 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faa90068e50 con 0x7faa9010a090 2024-03-21T10:18:03.870 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-21T10:18:03.865+0000 7faa87400700 1 -- 172.21.15.100:0/3705220278 <== mon.2 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 16) v1 ==== 137854+0+0 (secure 0 0 0) 0x7faa8801a020 con 0x7faa9010a090 2024-03-21T10:18:03.870 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-21T10:18:03.866+0000 7faa87400700 1 --2- 172.21.15.100:0/3705220278 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7faa740a3030 0x7faa740a54f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:18:03.870 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-21T10:18:03.867+0000 7faa87400700 1 -- 172.21.15.100:0/3705220278 <== mon.2 v2:172.21.15.100:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 4965+0+0 (secure 0 0 0) 0x7faa880ce600 con 0x7faa9010a090 2024-03-21T10:18:03.870 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-21T10:18:03.867+0000 7faa8ea00700 1 --2- 172.21.15.100:0/3705220278 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7faa740a3030 0x7faa740a54f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:18:03.871 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-21T10:18:03.868+0000 7faa8ea00700 1 --2- 172.21.15.100:0/3705220278 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7faa740a3030 0x7faa740a54f0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7faa80005950 tx=0x7faa800058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:18:03.872 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-21T10:18:03.870+0000 7faa87400700 1 -- 172.21.15.100:0/3705220278 <== mon.2 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7faa88010070 con 0x7faa9010a090 2024-03-21T10:18:04.105 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-21T10:18:04.102+0000 7faa95a00700 1 -- 172.21.15.100:0/3705220278 --> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi100:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) v1 -- 0x7faa90062690 con 0x7faa740a3030 2024-03-21T10:18:04.479 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:18:04 smithi100 ceph-mon[35193]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-21T10:18:04.480 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:18:04 smithi100 ceph-mon[35193]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-21T10:18:04.480 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:18:04 smithi100 ceph-mon[35193]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T10:18:04.714 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:18:04 smithi047 ceph-mon[31547]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-21T10:18:04.714 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:18:04 smithi047 ceph-mon[31547]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-21T10:18:04.714 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:18:04 smithi047 ceph-mon[31547]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T10:18:04.740 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:18:04 smithi157 ceph-mon[35581]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-21T10:18:04.740 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:18:04 smithi157 ceph-mon[35581]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-21T10:18:04.740 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:18:04 smithi157 ceph-mon[35581]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T10:18:05.410 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:18:05 smithi100 ceph-mon[35193]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 218 MiB used, 626 GiB / 626 GiB avail 2024-03-21T10:18:05.411 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:18:05 smithi100 ceph-mon[35193]: from='client.34274 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi100:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:18:05.715 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:18:05 smithi047 ceph-mon[31547]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 218 MiB used, 626 GiB / 626 GiB avail 2024-03-21T10:18:05.715 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:18:05 smithi047 ceph-mon[31547]: from='client.34274 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi100:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:18:05.740 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:18:05 smithi157 ceph-mon[35581]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 218 MiB used, 626 GiB / 626 GiB avail 2024-03-21T10:18:05.740 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:18:05 smithi157 ceph-mon[35581]: from='client.34274 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi100:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:18:07.714 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:18:07 smithi047 ceph-mon[31547]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 218 MiB used, 626 GiB / 626 GiB avail 2024-03-21T10:18:07.729 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:18:07 smithi100 ceph-mon[35193]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 218 MiB used, 626 GiB / 626 GiB avail 2024-03-21T10:18:07.739 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:18:07 smithi157 ceph-mon[35581]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 218 MiB used, 626 GiB / 626 GiB avail 2024-03-21T10:18:08.656 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:18:08 smithi100 ceph-mon[35193]: from='client.? 172.21.15.100:0/1910032294' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3c87c6d0-1d1f-4294-9917-aaaad7dda824"}]: dispatch 2024-03-21T10:18:08.656 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:18:08 smithi100 ceph-mon[35193]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3c87c6d0-1d1f-4294-9917-aaaad7dda824"}]: dispatch 2024-03-21T10:18:08.656 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:18:08 smithi100 ceph-mon[35193]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "3c87c6d0-1d1f-4294-9917-aaaad7dda824"}]': finished 2024-03-21T10:18:08.656 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:18:08 smithi100 ceph-mon[35193]: osdmap e50: 8 total, 7 up, 8 in 2024-03-21T10:18:08.656 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:18:08 smithi100 ceph-mon[35193]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-21T10:18:08.714 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:18:08 smithi047 ceph-mon[31547]: from='client.? 172.21.15.100:0/1910032294' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3c87c6d0-1d1f-4294-9917-aaaad7dda824"}]: dispatch 2024-03-21T10:18:08.714 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:18:08 smithi047 ceph-mon[31547]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3c87c6d0-1d1f-4294-9917-aaaad7dda824"}]: dispatch 2024-03-21T10:18:08.714 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:18:08 smithi047 ceph-mon[31547]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "3c87c6d0-1d1f-4294-9917-aaaad7dda824"}]': finished 2024-03-21T10:18:08.714 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:18:08 smithi047 ceph-mon[31547]: osdmap e50: 8 total, 7 up, 8 in 2024-03-21T10:18:08.714 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:18:08 smithi047 ceph-mon[31547]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-21T10:18:08.740 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:18:08 smithi157 ceph-mon[35581]: from='client.? 172.21.15.100:0/1910032294' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3c87c6d0-1d1f-4294-9917-aaaad7dda824"}]: dispatch 2024-03-21T10:18:08.740 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:18:08 smithi157 ceph-mon[35581]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3c87c6d0-1d1f-4294-9917-aaaad7dda824"}]: dispatch 2024-03-21T10:18:08.740 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:18:08 smithi157 ceph-mon[35581]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "3c87c6d0-1d1f-4294-9917-aaaad7dda824"}]': finished 2024-03-21T10:18:08.740 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:18:08 smithi157 ceph-mon[35581]: osdmap e50: 8 total, 7 up, 8 in 2024-03-21T10:18:08.740 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:18:08 smithi157 ceph-mon[35581]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-21T10:18:09.641 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:18:09 smithi157 ceph-mon[35581]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 218 MiB used, 626 GiB / 626 GiB avail 2024-03-21T10:18:09.641 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:18:09 smithi157 ceph-mon[35581]: from='client.? 172.21.15.100:0/1076839254' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-21T10:18:09.714 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:18:09 smithi047 ceph-mon[31547]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 218 MiB used, 626 GiB / 626 GiB avail 2024-03-21T10:18:09.714 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:18:09 smithi047 ceph-mon[31547]: from='client.? 172.21.15.100:0/1076839254' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-21T10:18:09.729 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:18:09 smithi100 ceph-mon[35193]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 218 MiB used, 626 GiB / 626 GiB avail 2024-03-21T10:18:09.729 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:18:09 smithi100 ceph-mon[35193]: from='client.? 172.21.15.100:0/1076839254' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-21T10:18:11.479 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:18:11 smithi100 ceph-mon[35193]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 218 MiB used, 626 GiB / 626 GiB avail 2024-03-21T10:18:11.713 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:18:11 smithi047 ceph-mon[31547]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 218 MiB used, 626 GiB / 626 GiB avail 2024-03-21T10:18:11.739 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:18:11 smithi157 ceph-mon[35581]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 218 MiB used, 626 GiB / 626 GiB avail 2024-03-21T10:18:13.713 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:18:13 smithi047 ceph-mon[31547]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 218 MiB used, 626 GiB / 626 GiB avail 2024-03-21T10:18:13.729 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:18:13 smithi100 ceph-mon[35193]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 218 MiB used, 626 GiB / 626 GiB avail 2024-03-21T10:18:13.739 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:18:13 smithi157 ceph-mon[35581]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 218 MiB used, 626 GiB / 626 GiB avail 2024-03-21T10:18:15.619 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:18:15 smithi100 ceph-mon[35193]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 218 MiB used, 626 GiB / 626 GiB avail 2024-03-21T10:18:15.714 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:18:15 smithi047 ceph-mon[31547]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 218 MiB used, 626 GiB / 626 GiB avail 2024-03-21T10:18:15.739 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:18:15 smithi157 ceph-mon[35581]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 218 MiB used, 626 GiB / 626 GiB avail 2024-03-21T10:18:17.714 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:18:17 smithi047 ceph-mon[31547]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 218 MiB used, 626 GiB / 626 GiB avail 2024-03-21T10:18:17.714 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:18:17 smithi047 ceph-mon[31547]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-03-21T10:18:17.714 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:18:17 smithi047 ceph-mon[31547]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T10:18:17.729 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:18:17 smithi100 ceph-mon[35193]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 218 MiB used, 626 GiB / 626 GiB avail 2024-03-21T10:18:17.729 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:18:17 smithi100 ceph-mon[35193]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-03-21T10:18:17.729 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:18:17 smithi100 ceph-mon[35193]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T10:18:17.739 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:18:17 smithi157 ceph-mon[35581]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 218 MiB used, 626 GiB / 626 GiB avail 2024-03-21T10:18:17.740 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:18:17 smithi157 ceph-mon[35581]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-03-21T10:18:17.740 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:18:17 smithi157 ceph-mon[35581]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T10:18:18.714 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:18:18 smithi047 ceph-mon[31547]: Deploying daemon osd.7 on smithi100 2024-03-21T10:18:18.729 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:18:18 smithi100 ceph-mon[35193]: Deploying daemon osd.7 on smithi100 2024-03-21T10:18:18.739 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:18:18 smithi157 ceph-mon[35581]: Deploying daemon osd.7 on smithi100 2024-03-21T10:18:19.480 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:18:19 smithi100 ceph-mon[35193]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 218 MiB used, 626 GiB / 626 GiB avail 2024-03-21T10:18:19.714 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:18:19 smithi047 ceph-mon[31547]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 218 MiB used, 626 GiB / 626 GiB avail 2024-03-21T10:18:19.739 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:18:19 smithi157 ceph-mon[35581]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 218 MiB used, 626 GiB / 626 GiB avail 2024-03-21T10:18:21.714 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:18:21 smithi047 ceph-mon[31547]: pgmap v217: 1 pgs: 1 active+clean; 577 KiB data, 218 MiB used, 626 GiB / 626 GiB avail 2024-03-21T10:18:21.714 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:18:21 smithi100 ceph-mon[35193]: pgmap v217: 1 pgs: 1 active+clean; 577 KiB data, 218 MiB used, 626 GiB / 626 GiB avail 2024-03-21T10:18:21.739 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:18:21 smithi157 ceph-mon[35581]: pgmap v217: 1 pgs: 1 active+clean; 577 KiB data, 218 MiB used, 626 GiB / 626 GiB avail 2024-03-21T10:18:23.464 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:18:23 smithi047 ceph-mon[31547]: pgmap v218: 1 pgs: 1 active+clean; 577 KiB data, 218 MiB used, 626 GiB / 626 GiB avail 2024-03-21T10:18:23.730 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:18:23 smithi100 ceph-mon[35193]: pgmap v218: 1 pgs: 1 active+clean; 577 KiB data, 218 MiB used, 626 GiB / 626 GiB avail 2024-03-21T10:18:23.740 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:18:23 smithi157 ceph-mon[35581]: pgmap v218: 1 pgs: 1 active+clean; 577 KiB data, 218 MiB used, 626 GiB / 626 GiB avail 2024-03-21T10:18:25.714 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:18:25 smithi047 ceph-mon[31547]: pgmap v219: 1 pgs: 1 active+clean; 577 KiB data, 218 MiB used, 626 GiB / 626 GiB avail 2024-03-21T10:18:25.714 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:18:25 smithi047 ceph-mon[31547]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T10:18:25.714 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:18:25 smithi047 ceph-mon[31547]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:18:25.714 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:18:25 smithi047 ceph-mon[31547]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:18:25.729 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:18:25 smithi100 ceph-mon[35193]: pgmap v219: 1 pgs: 1 active+clean; 577 KiB data, 218 MiB used, 626 GiB / 626 GiB avail 2024-03-21T10:18:25.729 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:18:25 smithi100 ceph-mon[35193]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T10:18:25.729 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:18:25 smithi100 ceph-mon[35193]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:18:25.729 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:18:25 smithi100 ceph-mon[35193]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:18:25.740 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:18:25 smithi157 ceph-mon[35581]: pgmap v219: 1 pgs: 1 active+clean; 577 KiB data, 218 MiB used, 626 GiB / 626 GiB avail 2024-03-21T10:18:25.740 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:18:25 smithi157 ceph-mon[35581]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T10:18:25.740 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:18:25 smithi157 ceph-mon[35581]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:18:25.740 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:18:25 smithi157 ceph-mon[35581]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:18:27.678 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:18:27 smithi100 ceph-mon[35193]: pgmap v220: 1 pgs: 1 active+clean; 577 KiB data, 218 MiB used, 626 GiB / 626 GiB avail 2024-03-21T10:18:27.714 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:18:27 smithi047 ceph-mon[31547]: pgmap v220: 1 pgs: 1 active+clean; 577 KiB data, 218 MiB used, 626 GiB / 626 GiB avail 2024-03-21T10:18:27.739 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:18:27 smithi157 ceph-mon[35581]: pgmap v220: 1 pgs: 1 active+clean; 577 KiB data, 218 MiB used, 626 GiB / 626 GiB avail 2024-03-21T10:18:29.916 INFO:teuthology.orchestra.run.smithi100.stdout:Created osd(s) 7 on host 'smithi100' 2024-03-21T10:18:29.916 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-21T10:18:29.914+0000 7faa87400700 1 -- 172.21.15.100:0/3705220278 <== mgr.14152 v2:172.21.15.47:6800/2823885994 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7faa90062690 con 0x7faa740a3030 2024-03-21T10:18:29.918 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-21T10:18:29.917+0000 7faa95a00700 1 -- 172.21.15.100:0/3705220278 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7faa740a3030 msgr2=0x7faa740a54f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:18:29.918 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-21T10:18:29.917+0000 7faa95a00700 1 --2- 172.21.15.100:0/3705220278 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7faa740a3030 0x7faa740a54f0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7faa80005950 tx=0x7faa800058e0 comp rx=0 tx=0).stop 2024-03-21T10:18:29.918 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-21T10:18:29.917+0000 7faa95a00700 1 -- 172.21.15.100:0/3705220278 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7faa9010a090 msgr2=0x7faa901a1b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:18:29.919 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-21T10:18:29.917+0000 7faa95a00700 1 --2- 172.21.15.100:0/3705220278 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7faa9010a090 0x7faa901a1b10 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7faa88007ee0 tx=0x7faa8800d3b0 comp rx=0 tx=0).stop 2024-03-21T10:18:29.919 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-21T10:18:29.917+0000 7faa95a00700 1 -- 172.21.15.100:0/3705220278 shutdown_connections 2024-03-21T10:18:29.919 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-21T10:18:29.917+0000 7faa95a00700 1 --2- 172.21.15.100:0/3705220278 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7faa740a3030 0x7faa740a54f0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:18:29.919 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-21T10:18:29.917+0000 7faa95a00700 1 --2- 172.21.15.100:0/3705220278 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7faa9010a090 0x7faa901a1b10 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:18:29.919 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-21T10:18:29.917+0000 7faa95a00700 1 --2- 172.21.15.100:0/3705220278 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7faa90074bf0 0x7faa9019a5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:18:29.919 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-21T10:18:29.917+0000 7faa95a00700 1 --2- 172.21.15.100:0/3705220278 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7faa900761c0 0x7faa9019aae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:18:29.919 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-21T10:18:29.918+0000 7faa95a00700 1 -- 172.21.15.100:0/3705220278 >> 172.21.15.100:0/3705220278 conn(0x7faa900fd740 msgr2=0x7faa90105f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:18:29.919 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-21T10:18:29.918+0000 7faa95a00700 1 -- 172.21.15.100:0/3705220278 shutdown_connections 2024-03-21T10:18:29.919 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-21T10:18:29.918+0000 7faa95a00700 1 -- 172.21.15.100:0/3705220278 wait complete. 2024-03-21T10:18:30.214 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:18:29 smithi047 ceph-mon[31547]: pgmap v221: 1 pgs: 1 active+clean; 577 KiB data, 218 MiB used, 626 GiB / 626 GiB avail 2024-03-21T10:18:30.214 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:18:29 smithi047 ceph-mon[31547]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:18:30.214 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:18:29 smithi047 ceph-mon[31547]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:18:30.214 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:18:29 smithi047 ceph-mon[31547]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T10:18:30.214 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:18:29 smithi047 ceph-mon[31547]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T10:18:30.214 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:18:29 smithi047 ceph-mon[31547]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:18:30.214 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:18:29 smithi047 ceph-mon[31547]: from='osd.7 [v2:172.21.15.100:6816/4104727330,v1:172.21.15.100:6817/4104727330]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-03-21T10:18:30.215 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:18:29 smithi047 ceph-mon[31547]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-03-21T10:18:30.229 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:18:29 smithi100 ceph-mon[35193]: pgmap v221: 1 pgs: 1 active+clean; 577 KiB data, 218 MiB used, 626 GiB / 626 GiB avail 2024-03-21T10:18:30.229 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:18:29 smithi100 ceph-mon[35193]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:18:30.230 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:18:29 smithi100 ceph-mon[35193]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:18:30.230 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:18:29 smithi100 ceph-mon[35193]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T10:18:30.230 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:18:29 smithi100 ceph-mon[35193]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T10:18:30.230 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:18:29 smithi100 ceph-mon[35193]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:18:30.230 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:18:29 smithi100 ceph-mon[35193]: from='osd.7 [v2:172.21.15.100:6816/4104727330,v1:172.21.15.100:6817/4104727330]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-03-21T10:18:30.230 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:18:29 smithi100 ceph-mon[35193]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-03-21T10:18:30.240 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:18:29 smithi157 ceph-mon[35581]: pgmap v221: 1 pgs: 1 active+clean; 577 KiB data, 218 MiB used, 626 GiB / 626 GiB avail 2024-03-21T10:18:30.240 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:18:29 smithi157 ceph-mon[35581]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:18:30.240 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:18:29 smithi157 ceph-mon[35581]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:18:30.240 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:18:29 smithi157 ceph-mon[35581]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T10:18:30.240 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:18:29 smithi157 ceph-mon[35581]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T10:18:30.240 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:18:29 smithi157 ceph-mon[35581]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:18:30.240 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:18:29 smithi157 ceph-mon[35581]: from='osd.7 [v2:172.21.15.100:6816/4104727330,v1:172.21.15.100:6817/4104727330]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-03-21T10:18:30.240 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:18:29 smithi157 ceph-mon[35581]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-03-21T10:18:30.656 DEBUG:teuthology.orchestra.run.smithi100:osd.7> sudo journalctl -f -n 0 -u ceph-3056d51a-e76b-11ee-95cd-87774f69a715@osd.7.service 2024-03-21T10:18:30.660 INFO:tasks.cephadm:Deploying osd.8 on smithi157 with /dev/vg_nvme/lv_2... 2024-03-21T10:18:30.660 DEBUG:teuthology.orchestra.run.smithi157:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2edfee68ad7243bed422a049b6f5f58619300f11 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3056d51a-e76b-11ee-95cd-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-03-21T10:18:30.877 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:18:30 smithi100 ceph-mon[35193]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-03-21T10:18:30.877 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:18:30 smithi100 ceph-mon[35193]: osdmap e51: 8 total, 7 up, 8 in 2024-03-21T10:18:30.877 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:18:30 smithi100 ceph-mon[35193]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-21T10:18:30.877 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:18:30 smithi100 ceph-mon[35193]: from='osd.7 [v2:172.21.15.100:6816/4104727330,v1:172.21.15.100:6817/4104727330]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi100", "root=default"]}]: dispatch 2024-03-21T10:18:30.878 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:18:30 smithi100 ceph-mon[35193]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi100", "root=default"]}]: dispatch 2024-03-21T10:18:30.878 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:18:30 smithi100 ceph-mon[35193]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:18:30.878 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:18:30 smithi100 ceph-mon[35193]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:18:30.878 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:18:30 smithi100 ceph-mon[35193]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T10:18:30.878 INFO:journalctl@ceph.osd.7.smithi100.stdout:Mar 21 10:18:30 smithi100 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-7[53119]: 2024-03-21T10:18:30.846+0000 7fb6f0800700 -1 osd.7 0 waiting for initial osdmap 2024-03-21T10:18:30.878 INFO:journalctl@ceph.osd.7.smithi100.stdout:Mar 21 10:18:30 smithi100 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-7[53119]: 2024-03-21T10:18:30.875+0000 7fb6eb600700 -1 osd.7 52 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-21T10:18:31.108 INFO:teuthology.orchestra.run.smithi157.stderr:Inferring config /var/lib/ceph/3056d51a-e76b-11ee-95cd-87774f69a715/mon.c/config 2024-03-21T10:18:31.214 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:18:30 smithi047 ceph-mon[31547]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-03-21T10:18:31.214 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:18:30 smithi047 ceph-mon[31547]: osdmap e51: 8 total, 7 up, 8 in 2024-03-21T10:18:31.214 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:18:30 smithi047 ceph-mon[31547]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-21T10:18:31.214 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:18:30 smithi047 ceph-mon[31547]: from='osd.7 [v2:172.21.15.100:6816/4104727330,v1:172.21.15.100:6817/4104727330]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi100", "root=default"]}]: dispatch 2024-03-21T10:18:31.214 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:18:30 smithi047 ceph-mon[31547]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi100", "root=default"]}]: dispatch 2024-03-21T10:18:31.214 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:18:30 smithi047 ceph-mon[31547]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:18:31.214 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:18:30 smithi047 ceph-mon[31547]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:18:31.214 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:18:30 smithi047 ceph-mon[31547]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T10:18:31.239 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:18:30 smithi157 ceph-mon[35581]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-03-21T10:18:31.240 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:18:30 smithi157 ceph-mon[35581]: osdmap e51: 8 total, 7 up, 8 in 2024-03-21T10:18:31.240 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:18:30 smithi157 ceph-mon[35581]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-21T10:18:31.240 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:18:30 smithi157 ceph-mon[35581]: from='osd.7 [v2:172.21.15.100:6816/4104727330,v1:172.21.15.100:6817/4104727330]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi100", "root=default"]}]: dispatch 2024-03-21T10:18:31.240 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:18:30 smithi157 ceph-mon[35581]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi100", "root=default"]}]: dispatch 2024-03-21T10:18:31.240 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:18:30 smithi157 ceph-mon[35581]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:18:31.240 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:18:30 smithi157 ceph-mon[35581]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:18:31.240 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:18:30 smithi157 ceph-mon[35581]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T10:18:32.101 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:18:31 smithi100 ceph-mon[35193]: pgmap v223: 1 pgs: 1 active+clean; 577 KiB data, 218 MiB used, 626 GiB / 626 GiB avail 2024-03-21T10:18:32.101 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:18:31 smithi100 ceph-mon[35193]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi100", "root=default"]}]': finished 2024-03-21T10:18:32.102 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:18:31 smithi100 ceph-mon[35193]: osdmap e52: 8 total, 7 up, 8 in 2024-03-21T10:18:32.102 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:18:31 smithi100 ceph-mon[35193]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-21T10:18:32.102 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:18:31 smithi100 ceph-mon[35193]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-21T10:18:32.214 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:18:31 smithi047 ceph-mon[31547]: pgmap v223: 1 pgs: 1 active+clean; 577 KiB data, 218 MiB used, 626 GiB / 626 GiB avail 2024-03-21T10:18:32.214 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:18:31 smithi047 ceph-mon[31547]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi100", "root=default"]}]': finished 2024-03-21T10:18:32.214 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:18:31 smithi047 ceph-mon[31547]: osdmap e52: 8 total, 7 up, 8 in 2024-03-21T10:18:32.214 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:18:31 smithi047 ceph-mon[31547]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-21T10:18:32.214 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:18:31 smithi047 ceph-mon[31547]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-21T10:18:32.240 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:18:31 smithi157 ceph-mon[35581]: pgmap v223: 1 pgs: 1 active+clean; 577 KiB data, 218 MiB used, 626 GiB / 626 GiB avail 2024-03-21T10:18:32.240 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:18:31 smithi157 ceph-mon[35581]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi100", "root=default"]}]': finished 2024-03-21T10:18:32.240 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:18:31 smithi157 ceph-mon[35581]: osdmap e52: 8 total, 7 up, 8 in 2024-03-21T10:18:32.240 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:18:31 smithi157 ceph-mon[35581]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-21T10:18:32.240 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:18:31 smithi157 ceph-mon[35581]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-21T10:18:32.964 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:18:32 smithi047 ceph-mon[31547]: purged_snaps scrub starts 2024-03-21T10:18:32.964 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:18:32 smithi047 ceph-mon[31547]: purged_snaps scrub ok 2024-03-21T10:18:32.964 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:18:32 smithi047 ceph-mon[31547]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-21T10:18:32.964 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:18:32 smithi047 ceph-mon[31547]: osd.7 [v2:172.21.15.100:6816/4104727330,v1:172.21.15.100:6817/4104727330] boot 2024-03-21T10:18:32.964 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:18:32 smithi047 ceph-mon[31547]: osdmap e53: 8 total, 8 up, 8 in 2024-03-21T10:18:32.964 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:18:32 smithi047 ceph-mon[31547]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-21T10:18:32.979 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:18:32 smithi100 ceph-mon[35193]: purged_snaps scrub starts 2024-03-21T10:18:32.979 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:18:32 smithi100 ceph-mon[35193]: purged_snaps scrub ok 2024-03-21T10:18:32.979 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:18:32 smithi100 ceph-mon[35193]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-21T10:18:32.980 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:18:32 smithi100 ceph-mon[35193]: osd.7 [v2:172.21.15.100:6816/4104727330,v1:172.21.15.100:6817/4104727330] boot 2024-03-21T10:18:32.980 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:18:32 smithi100 ceph-mon[35193]: osdmap e53: 8 total, 8 up, 8 in 2024-03-21T10:18:32.980 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:18:32 smithi100 ceph-mon[35193]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-21T10:18:33.034 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:18:32 smithi157 ceph-mon[35581]: purged_snaps scrub starts 2024-03-21T10:18:33.034 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:18:32 smithi157 ceph-mon[35581]: purged_snaps scrub ok 2024-03-21T10:18:33.034 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:18:32 smithi157 ceph-mon[35581]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-21T10:18:33.034 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:18:32 smithi157 ceph-mon[35581]: osd.7 [v2:172.21.15.100:6816/4104727330,v1:172.21.15.100:6817/4104727330] boot 2024-03-21T10:18:33.034 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:18:32 smithi157 ceph-mon[35581]: osdmap e53: 8 total, 8 up, 8 in 2024-03-21T10:18:33.034 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:18:32 smithi157 ceph-mon[35581]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-21T10:18:33.963 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:18:33 smithi157 ceph-mon[35581]: pgmap v226: 1 pgs: 1 active+clean; 577 KiB data, 218 MiB used, 626 GiB / 626 GiB avail 2024-03-21T10:18:33.993 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:18:33 smithi157 ceph-mon[35581]: osdmap e54: 8 total, 8 up, 8 in 2024-03-21T10:18:33.993 INFO:teuthology.orchestra.run.smithi157.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-03-21T10:18:33.993 INFO:teuthology.orchestra.run.smithi157.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-03-21T10:18:33.993 INFO:teuthology.orchestra.run.smithi157.stderr: stderr: 10+0 records in 2024-03-21T10:18:33.993 INFO:teuthology.orchestra.run.smithi157.stderr:10+0 records out 2024-03-21T10:18:33.993 INFO:teuthology.orchestra.run.smithi157.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0308285 s, 340 MB/s 2024-03-21T10:18:33.993 INFO:teuthology.orchestra.run.smithi157.stderr:--> Zapping successful for: 2024-03-21T10:18:34.214 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:18:33 smithi047 ceph-mon[31547]: pgmap v226: 1 pgs: 1 active+clean; 577 KiB data, 218 MiB used, 626 GiB / 626 GiB avail 2024-03-21T10:18:34.214 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:18:33 smithi047 ceph-mon[31547]: osdmap e54: 8 total, 8 up, 8 in 2024-03-21T10:18:34.230 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:18:33 smithi100 ceph-mon[35193]: pgmap v226: 1 pgs: 1 active+clean; 577 KiB data, 218 MiB used, 626 GiB / 626 GiB avail 2024-03-21T10:18:34.230 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:18:33 smithi100 ceph-mon[35193]: osdmap e54: 8 total, 8 up, 8 in 2024-03-21T10:18:34.611 DEBUG:teuthology.orchestra.run.smithi157:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2edfee68ad7243bed422a049b6f5f58619300f11 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3056d51a-e76b-11ee-95cd-87774f69a715 -- ceph orch daemon add osd smithi157:vg_nvme/lv_2 2024-03-21T10:18:35.026 INFO:teuthology.orchestra.run.smithi157.stderr:Inferring config /var/lib/ceph/3056d51a-e76b-11ee-95cd-87774f69a715/mon.c/config 2024-03-21T10:18:35.170 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:18:34 smithi157 ceph-mon[35581]: osdmap e55: 8 total, 8 up, 8 in 2024-03-21T10:18:35.170 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:18:34 smithi157 ceph-mon[35581]: pgmap v229: 1 pgs: 1 remapped; 577 KiB data, 245 MiB used, 715 GiB / 715 GiB avail 2024-03-21T10:18:35.189 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:18:34 smithi100 ceph-mon[35193]: osdmap e55: 8 total, 8 up, 8 in 2024-03-21T10:18:35.189 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:18:34 smithi100 ceph-mon[35193]: pgmap v229: 1 pgs: 1 remapped; 577 KiB data, 245 MiB used, 715 GiB / 715 GiB avail 2024-03-21T10:18:35.213 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:18:34 smithi047 ceph-mon[31547]: osdmap e55: 8 total, 8 up, 8 in 2024-03-21T10:18:35.214 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:18:34 smithi047 ceph-mon[31547]: pgmap v229: 1 pgs: 1 remapped; 577 KiB data, 245 MiB used, 715 GiB / 715 GiB avail 2024-03-21T10:18:36.214 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:18:35 smithi047 ceph-mon[31547]: Health check failed: Reduced data availability: 1 pg inactive (PG_AVAILABILITY) 2024-03-21T10:18:36.214 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:18:35 smithi047 ceph-mon[31547]: osdmap e56: 8 total, 8 up, 8 in 2024-03-21T10:18:36.214 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:18:35 smithi047 ceph-mon[31547]: Detected new or changed devices on smithi100 2024-03-21T10:18:36.214 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:18:35 smithi047 ceph-mon[31547]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:18:36.214 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:18:35 smithi047 ceph-mon[31547]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:18:36.215 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:18:35 smithi047 ceph-mon[31547]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-21T10:18:36.215 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:18:35 smithi047 ceph-mon[31547]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-21T10:18:36.215 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:18:35 smithi047 ceph-mon[31547]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-21T10:18:36.215 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:18:35 smithi047 ceph-mon[31547]: Adjusting osd_memory_target on smithi100 to 5696M 2024-03-21T10:18:36.215 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:18:35 smithi047 ceph-mon[31547]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:18:36.215 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:18:35 smithi047 ceph-mon[31547]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T10:18:36.215 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:18:35 smithi047 ceph-mon[31547]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T10:18:36.215 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:18:35 smithi047 ceph-mon[31547]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:18:36.226 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:18:35 smithi157 ceph-mon[35581]: Health check failed: Reduced data availability: 1 pg inactive (PG_AVAILABILITY) 2024-03-21T10:18:36.226 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:18:35 smithi157 ceph-mon[35581]: osdmap e56: 8 total, 8 up, 8 in 2024-03-21T10:18:36.226 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:18:35 smithi157 ceph-mon[35581]: Detected new or changed devices on smithi100 2024-03-21T10:18:36.226 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:18:35 smithi157 ceph-mon[35581]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:18:36.226 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:18:35 smithi157 ceph-mon[35581]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:18:36.226 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:18:35 smithi157 ceph-mon[35581]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-21T10:18:36.226 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:18:35 smithi157 ceph-mon[35581]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-21T10:18:36.226 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:18:35 smithi157 ceph-mon[35581]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-21T10:18:36.227 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:18:35 smithi157 ceph-mon[35581]: Adjusting osd_memory_target on smithi100 to 5696M 2024-03-21T10:18:36.227 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:18:35 smithi157 ceph-mon[35581]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:18:36.227 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:18:35 smithi157 ceph-mon[35581]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T10:18:36.227 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:18:35 smithi157 ceph-mon[35581]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T10:18:36.227 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:18:35 smithi157 ceph-mon[35581]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:18:36.230 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:18:35 smithi100 ceph-mon[35193]: Health check failed: Reduced data availability: 1 pg inactive (PG_AVAILABILITY) 2024-03-21T10:18:36.230 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:18:35 smithi100 ceph-mon[35193]: osdmap e56: 8 total, 8 up, 8 in 2024-03-21T10:18:36.230 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:18:35 smithi100 ceph-mon[35193]: Detected new or changed devices on smithi100 2024-03-21T10:18:36.230 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:18:35 smithi100 ceph-mon[35193]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:18:36.230 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:18:35 smithi100 ceph-mon[35193]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:18:36.230 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:18:35 smithi100 ceph-mon[35193]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-21T10:18:36.230 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:18:35 smithi100 ceph-mon[35193]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-21T10:18:36.230 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:18:35 smithi100 ceph-mon[35193]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-21T10:18:36.230 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:18:35 smithi100 ceph-mon[35193]: Adjusting osd_memory_target on smithi100 to 5696M 2024-03-21T10:18:36.230 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:18:35 smithi100 ceph-mon[35193]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:18:36.230 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:18:35 smithi100 ceph-mon[35193]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T10:18:36.230 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:18:35 smithi100 ceph-mon[35193]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T10:18:36.231 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:18:35 smithi100 ceph-mon[35193]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:18:36.990 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:18:36 smithi157 ceph-mon[35581]: osdmap e57: 8 total, 8 up, 8 in 2024-03-21T10:18:36.990 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:18:36 smithi157 ceph-mon[35581]: pgmap v232: 1 pgs: 1 remapped; 577 KiB data, 245 MiB used, 715 GiB / 715 GiB avail 2024-03-21T10:18:37.214 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:18:36 smithi047 ceph-mon[31547]: osdmap e57: 8 total, 8 up, 8 in 2024-03-21T10:18:37.214 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:18:36 smithi047 ceph-mon[31547]: pgmap v232: 1 pgs: 1 remapped; 577 KiB data, 245 MiB used, 715 GiB / 715 GiB avail 2024-03-21T10:18:37.229 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:18:36 smithi100 ceph-mon[35193]: osdmap e57: 8 total, 8 up, 8 in 2024-03-21T10:18:37.229 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:18:36 smithi100 ceph-mon[35193]: pgmap v232: 1 pgs: 1 remapped; 577 KiB data, 245 MiB used, 715 GiB / 715 GiB avail 2024-03-21T10:18:37.254 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-21T10:18:37.252+0000 7f4845a00700 1 -- 172.21.15.157:0/1319730509 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4840105510 msgr2=0x7f4840105970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:18:37.254 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-21T10:18:37.252+0000 7f4845a00700 1 --2- 172.21.15.157:0/1319730509 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4840105510 0x7f4840105970 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f482c009990 tx=0x7f482c009ca0 comp rx=0 tx=0).stop 2024-03-21T10:18:37.254 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-21T10:18:37.252+0000 7f4845a00700 1 -- 172.21.15.157:0/1319730509 shutdown_connections 2024-03-21T10:18:37.254 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-21T10:18:37.252+0000 7f4845a00700 1 --2- 172.21.15.157:0/1319730509 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4840105eb0 0x7f484010c760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:18:37.254 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-21T10:18:37.252+0000 7f4845a00700 1 --2- 172.21.15.157:0/1319730509 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4840104310 0x7f4840104730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:18:37.254 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-21T10:18:37.252+0000 7f4845a00700 1 --2- 172.21.15.157:0/1319730509 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4840105510 0x7f4840105970 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:18:37.255 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-21T10:18:37.252+0000 7f4845a00700 1 -- 172.21.15.157:0/1319730509 >> 172.21.15.157:0/1319730509 conn(0x7f48400ff8b0 msgr2=0x7f4840101cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:18:37.255 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-21T10:18:37.253+0000 7f4845a00700 1 -- 172.21.15.157:0/1319730509 shutdown_connections 2024-03-21T10:18:37.255 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-21T10:18:37.253+0000 7f4845a00700 1 -- 172.21.15.157:0/1319730509 wait complete. 2024-03-21T10:18:37.256 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-21T10:18:37.254+0000 7f4845a00700 1 Processor -- start 2024-03-21T10:18:37.257 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-21T10:18:37.254+0000 7f4845a00700 1 -- start start 2024-03-21T10:18:37.257 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-21T10:18:37.254+0000 7f4845a00700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4840104310 0x7f484019a4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:18:37.257 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-21T10:18:37.254+0000 7f4845a00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4840105510 0x7f484019aa10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:18:37.257 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-21T10:18:37.254+0000 7f4845a00700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4840105eb0 0x7f48401a1a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:18:37.257 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-21T10:18:37.254+0000 7f4845a00700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f484010fe00 con 0x7f4840104310 2024-03-21T10:18:37.257 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-21T10:18:37.254+0000 7f4845a00700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f484010fc80 con 0x7f4840105eb0 2024-03-21T10:18:37.257 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-21T10:18:37.254+0000 7f4845a00700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f484010ff80 con 0x7f4840105510 2024-03-21T10:18:37.257 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-21T10:18:37.254+0000 7f483f400700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4840105510 0x7f484019aa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:18:37.257 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-21T10:18:37.254+0000 7f483f400700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4840105510 0x7f484019aa10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:37554/0 (socket says 172.21.15.157:37554) 2024-03-21T10:18:37.257 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-21T10:18:37.254+0000 7f483f400700 1 -- 172.21.15.157:0/4192638743 learned_addr learned my addr 172.21.15.157:0/4192638743 (peer_addr_for_me v2:172.21.15.157:0/0) 2024-03-21T10:18:37.257 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-21T10:18:37.255+0000 7f483fe00700 1 --2- 172.21.15.157:0/4192638743 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4840104310 0x7f484019a4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:18:37.258 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-21T10:18:37.255+0000 7f4845000700 1 --2- 172.21.15.157:0/4192638743 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4840105eb0 0x7f48401a1a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:18:37.258 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-21T10:18:37.255+0000 7f483f400700 1 -- 172.21.15.157:0/4192638743 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4840105eb0 msgr2=0x7f48401a1a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:18:37.258 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-21T10:18:37.255+0000 7f483f400700 1 --2- 172.21.15.157:0/4192638743 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4840105eb0 0x7f48401a1a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:18:37.258 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-21T10:18:37.255+0000 7f483f400700 1 -- 172.21.15.157:0/4192638743 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4840104310 msgr2=0x7f484019a4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:18:37.258 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-21T10:18:37.255+0000 7f483f400700 1 --2- 172.21.15.157:0/4192638743 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4840104310 0x7f484019a4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:18:37.258 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-21T10:18:37.255+0000 7f483f400700 1 -- 172.21.15.157:0/4192638743 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48401a20e0 con 0x7f4840105510 2024-03-21T10:18:37.258 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-21T10:18:37.255+0000 7f4845000700 1 --2- 172.21.15.157:0/4192638743 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4840105eb0 0x7f48401a1a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T10:18:37.258 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-21T10:18:37.255+0000 7f483f400700 1 --2- 172.21.15.157:0/4192638743 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4840105510 0x7f484019aa10 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f482c005910 tx=0x7f482c0047b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:18:37.258 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-21T10:18:37.255+0000 7f483cc00700 1 -- 172.21.15.157:0/4192638743 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f482c027070 con 0x7f4840105510 2024-03-21T10:18:37.259 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-21T10:18:37.256+0000 7f483cc00700 1 -- 172.21.15.157:0/4192638743 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f482c004100 con 0x7f4840105510 2024-03-21T10:18:37.259 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-21T10:18:37.256+0000 7f4845a00700 1 -- 172.21.15.157:0/4192638743 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f48401a2370 con 0x7f4840105510 2024-03-21T10:18:37.259 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-21T10:18:37.256+0000 7f483cc00700 1 -- 172.21.15.157:0/4192638743 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f482c00f460 con 0x7f4840105510 2024-03-21T10:18:37.259 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-21T10:18:37.256+0000 7f4845a00700 1 -- 172.21.15.157:0/4192638743 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f48401a2820 con 0x7f4840105510 2024-03-21T10:18:37.259 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-21T10:18:37.257+0000 7f4845a00700 1 -- 172.21.15.157:0/4192638743 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4840068e50 con 0x7f4840105510 2024-03-21T10:18:37.260 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-21T10:18:37.259+0000 7f483cc00700 1 -- 172.21.15.157:0/4192638743 <== mon.1 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 16) v1 ==== 137854+0+0 (secure 0 0 0) 0x7f482c02a070 con 0x7f4840105510 2024-03-21T10:18:37.262 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-21T10:18:37.260+0000 7f483cc00700 1 --2- 172.21.15.157:0/4192638743 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f48240a3030 0x7f48240a54f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:18:37.262 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-21T10:18:37.260+0000 7f483cc00700 1 -- 172.21.15.157:0/4192638743 <== mon.1 v2:172.21.15.157:3300/0 5 ==== osd_map(57..57 src has 1..57) v4 ==== 5397+0+0 (secure 0 0 0) 0x7f482c0cffa0 con 0x7f4840105510 2024-03-21T10:18:37.262 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-21T10:18:37.261+0000 7f483fe00700 1 --2- 172.21.15.157:0/4192638743 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f48240a3030 0x7f48240a54f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:18:37.263 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-21T10:18:37.262+0000 7f483fe00700 1 --2- 172.21.15.157:0/4192638743 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f48240a3030 0x7f48240a54f0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f4830005950 tx=0x7f48300058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:18:37.264 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-21T10:18:37.263+0000 7f483cc00700 1 -- 172.21.15.157:0/4192638743 <== mon.1 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f482c031070 con 0x7f4840105510 2024-03-21T10:18:37.495 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-21T10:18:37.493+0000 7f4845a00700 1 -- 172.21.15.157:0/4192638743 --> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi157:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) v1 -- 0x7f48400625d0 con 0x7f48240a3030 2024-03-21T10:18:38.214 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:18:37 smithi047 ceph-mon[31547]: from='client.24283 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi157:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:18:38.214 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:18:37 smithi047 ceph-mon[31547]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-21T10:18:38.214 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:18:37 smithi047 ceph-mon[31547]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-21T10:18:38.214 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:18:37 smithi047 ceph-mon[31547]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T10:18:38.229 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:18:37 smithi100 ceph-mon[35193]: from='client.24283 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi157:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:18:38.229 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:18:37 smithi100 ceph-mon[35193]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-21T10:18:38.229 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:18:37 smithi100 ceph-mon[35193]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-21T10:18:38.230 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:18:37 smithi100 ceph-mon[35193]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T10:18:38.239 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:18:37 smithi157 ceph-mon[35581]: from='client.24283 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi157:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:18:38.239 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:18:37 smithi157 ceph-mon[35581]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-21T10:18:38.239 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:18:37 smithi157 ceph-mon[35581]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-21T10:18:38.239 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:18:37 smithi157 ceph-mon[35581]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T10:18:39.213 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:18:38 smithi047 ceph-mon[31547]: pgmap v233: 1 pgs: 1 active+clean; 577 KiB data, 246 MiB used, 715 GiB / 715 GiB avail 2024-03-21T10:18:39.229 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:18:38 smithi100 ceph-mon[35193]: pgmap v233: 1 pgs: 1 active+clean; 577 KiB data, 246 MiB used, 715 GiB / 715 GiB avail 2024-03-21T10:18:39.239 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:18:38 smithi157 ceph-mon[35581]: pgmap v233: 1 pgs: 1 active+clean; 577 KiB data, 246 MiB used, 715 GiB / 715 GiB avail 2024-03-21T10:18:40.093 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:18:39 smithi157 ceph-mon[35581]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg inactive) 2024-03-21T10:18:40.093 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:18:39 smithi157 ceph-mon[35581]: Cluster is now healthy 2024-03-21T10:18:40.214 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:18:39 smithi047 ceph-mon[31547]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg inactive) 2024-03-21T10:18:40.214 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:18:39 smithi047 ceph-mon[31547]: Cluster is now healthy 2024-03-21T10:18:40.229 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:18:39 smithi100 ceph-mon[35193]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg inactive) 2024-03-21T10:18:40.229 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:18:39 smithi100 ceph-mon[35193]: Cluster is now healthy 2024-03-21T10:18:41.213 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:18:40 smithi047 ceph-mon[31547]: pgmap v234: 1 pgs: 1 active+clean; 577 KiB data, 246 MiB used, 715 GiB / 715 GiB avail 2024-03-21T10:18:41.229 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:18:40 smithi100 ceph-mon[35193]: pgmap v234: 1 pgs: 1 active+clean; 577 KiB data, 246 MiB used, 715 GiB / 715 GiB avail 2024-03-21T10:18:41.240 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:18:40 smithi157 ceph-mon[35581]: pgmap v234: 1 pgs: 1 active+clean; 577 KiB data, 246 MiB used, 715 GiB / 715 GiB avail 2024-03-21T10:18:42.187 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:18:41 smithi157 ceph-mon[35581]: from='client.? 172.21.15.157:0/1228329503' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "502d65cf-53c8-4936-8de4-e11b2a6e1e18"}]: dispatch 2024-03-21T10:18:42.188 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:18:41 smithi157 ceph-mon[35581]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "502d65cf-53c8-4936-8de4-e11b2a6e1e18"}]: dispatch 2024-03-21T10:18:42.188 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:18:41 smithi157 ceph-mon[35581]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "502d65cf-53c8-4936-8de4-e11b2a6e1e18"}]': finished 2024-03-21T10:18:42.188 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:18:41 smithi157 ceph-mon[35581]: osdmap e58: 9 total, 8 up, 9 in 2024-03-21T10:18:42.188 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:18:41 smithi157 ceph-mon[35581]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-21T10:18:42.214 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:18:41 smithi047 ceph-mon[31547]: from='client.? 172.21.15.157:0/1228329503' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "502d65cf-53c8-4936-8de4-e11b2a6e1e18"}]: dispatch 2024-03-21T10:18:42.214 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:18:41 smithi047 ceph-mon[31547]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "502d65cf-53c8-4936-8de4-e11b2a6e1e18"}]: dispatch 2024-03-21T10:18:42.214 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:18:41 smithi047 ceph-mon[31547]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "502d65cf-53c8-4936-8de4-e11b2a6e1e18"}]': finished 2024-03-21T10:18:42.214 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:18:41 smithi047 ceph-mon[31547]: osdmap e58: 9 total, 8 up, 9 in 2024-03-21T10:18:42.214 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:18:41 smithi047 ceph-mon[31547]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-21T10:18:42.229 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:18:41 smithi100 ceph-mon[35193]: from='client.? 172.21.15.157:0/1228329503' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "502d65cf-53c8-4936-8de4-e11b2a6e1e18"}]: dispatch 2024-03-21T10:18:42.229 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:18:41 smithi100 ceph-mon[35193]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "502d65cf-53c8-4936-8de4-e11b2a6e1e18"}]: dispatch 2024-03-21T10:18:42.230 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:18:41 smithi100 ceph-mon[35193]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "502d65cf-53c8-4936-8de4-e11b2a6e1e18"}]': finished 2024-03-21T10:18:42.230 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:18:41 smithi100 ceph-mon[35193]: osdmap e58: 9 total, 8 up, 9 in 2024-03-21T10:18:42.230 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:18:41 smithi100 ceph-mon[35193]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-21T10:18:43.214 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:18:42 smithi047 ceph-mon[31547]: pgmap v236: 1 pgs: 1 active+clean; 577 KiB data, 246 MiB used, 715 GiB / 715 GiB avail 2024-03-21T10:18:43.214 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:18:42 smithi047 ceph-mon[31547]: from='client.? 172.21.15.157:0/4038264238' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-21T10:18:43.229 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:18:42 smithi100 ceph-mon[35193]: pgmap v236: 1 pgs: 1 active+clean; 577 KiB data, 246 MiB used, 715 GiB / 715 GiB avail 2024-03-21T10:18:43.229 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:18:42 smithi100 ceph-mon[35193]: from='client.? 172.21.15.157:0/4038264238' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-21T10:18:43.239 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:18:42 smithi157 ceph-mon[35581]: pgmap v236: 1 pgs: 1 active+clean; 577 KiB data, 246 MiB used, 715 GiB / 715 GiB avail 2024-03-21T10:18:43.239 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:18:42 smithi157 ceph-mon[35581]: from='client.? 172.21.15.157:0/4038264238' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-21T10:18:45.463 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:18:45 smithi047 ceph-mon[31547]: pgmap v237: 1 pgs: 1 active+clean; 577 KiB data, 246 MiB used, 715 GiB / 715 GiB avail 2024-03-21T10:18:45.479 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:18:45 smithi100 ceph-mon[35193]: pgmap v237: 1 pgs: 1 active+clean; 577 KiB data, 246 MiB used, 715 GiB / 715 GiB avail 2024-03-21T10:18:45.489 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:18:45 smithi157 ceph-mon[35581]: pgmap v237: 1 pgs: 1 active+clean; 577 KiB data, 246 MiB used, 715 GiB / 715 GiB avail 2024-03-21T10:18:47.281 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:18:47 smithi100 ceph-mon[35193]: pgmap v238: 1 pgs: 1 active+clean; 577 KiB data, 246 MiB used, 715 GiB / 715 GiB avail 2024-03-21T10:18:47.464 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:18:47 smithi047 ceph-mon[31547]: pgmap v238: 1 pgs: 1 active+clean; 577 KiB data, 246 MiB used, 715 GiB / 715 GiB avail 2024-03-21T10:18:47.489 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:18:47 smithi157 ceph-mon[35581]: pgmap v238: 1 pgs: 1 active+clean; 577 KiB data, 246 MiB used, 715 GiB / 715 GiB avail 2024-03-21T10:18:49.186 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:18:49 smithi157 ceph-mon[35581]: pgmap v239: 1 pgs: 1 active+clean; 577 KiB data, 246 MiB used, 715 GiB / 715 GiB avail 2024-03-21T10:18:49.229 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:18:49 smithi100 ceph-mon[35193]: pgmap v239: 1 pgs: 1 active+clean; 577 KiB data, 246 MiB used, 715 GiB / 715 GiB avail 2024-03-21T10:18:49.463 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:18:49 smithi047 ceph-mon[31547]: pgmap v239: 1 pgs: 1 active+clean; 577 KiB data, 246 MiB used, 715 GiB / 715 GiB avail 2024-03-21T10:18:51.289 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:18:51 smithi047 ceph-mon[31547]: pgmap v240: 1 pgs: 1 active+clean; 577 KiB data, 246 MiB used, 715 GiB / 715 GiB avail 2024-03-21T10:18:51.289 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:18:51 smithi047 ceph-mon[31547]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-03-21T10:18:51.289 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:18:51 smithi047 ceph-mon[31547]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T10:18:51.479 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:18:51 smithi100 ceph-mon[35193]: pgmap v240: 1 pgs: 1 active+clean; 577 KiB data, 246 MiB used, 715 GiB / 715 GiB avail 2024-03-21T10:18:51.479 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:18:51 smithi100 ceph-mon[35193]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-03-21T10:18:51.480 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:18:51 smithi100 ceph-mon[35193]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T10:18:51.489 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:18:51 smithi157 ceph-mon[35581]: pgmap v240: 1 pgs: 1 active+clean; 577 KiB data, 246 MiB used, 715 GiB / 715 GiB avail 2024-03-21T10:18:51.489 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:18:51 smithi157 ceph-mon[35581]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-03-21T10:18:51.489 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:18:51 smithi157 ceph-mon[35581]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T10:18:52.121 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:18:52 smithi157 ceph-mon[35581]: Deploying daemon osd.8 on smithi157 2024-03-21T10:18:52.214 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:18:52 smithi047 ceph-mon[31547]: Deploying daemon osd.8 on smithi157 2024-03-21T10:18:52.479 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:18:52 smithi100 ceph-mon[35193]: Deploying daemon osd.8 on smithi157 2024-03-21T10:18:53.214 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:18:53 smithi047 ceph-mon[31547]: pgmap v241: 1 pgs: 1 active+clean; 577 KiB data, 246 MiB used, 715 GiB / 715 GiB avail 2024-03-21T10:18:53.479 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:18:53 smithi100 ceph-mon[35193]: pgmap v241: 1 pgs: 1 active+clean; 577 KiB data, 246 MiB used, 715 GiB / 715 GiB avail 2024-03-21T10:18:53.489 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:18:53 smithi157 ceph-mon[35581]: pgmap v241: 1 pgs: 1 active+clean; 577 KiB data, 246 MiB used, 715 GiB / 715 GiB avail 2024-03-21T10:18:55.414 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:18:55 smithi157 ceph-mon[35581]: pgmap v242: 1 pgs: 1 active+clean; 577 KiB data, 246 MiB used, 715 GiB / 715 GiB avail 2024-03-21T10:18:55.463 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:18:55 smithi047 ceph-mon[31547]: pgmap v242: 1 pgs: 1 active+clean; 577 KiB data, 246 MiB used, 715 GiB / 715 GiB avail 2024-03-21T10:18:55.479 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:18:55 smithi100 ceph-mon[35193]: pgmap v242: 1 pgs: 1 active+clean; 577 KiB data, 246 MiB used, 715 GiB / 715 GiB avail 2024-03-21T10:18:57.240 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:18:57 smithi157 ceph-mon[35581]: pgmap v243: 1 pgs: 1 active+clean; 577 KiB data, 246 MiB used, 715 GiB / 715 GiB avail 2024-03-21T10:18:57.463 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:18:57 smithi047 ceph-mon[31547]: pgmap v243: 1 pgs: 1 active+clean; 577 KiB data, 246 MiB used, 715 GiB / 715 GiB avail 2024-03-21T10:18:57.479 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:18:57 smithi100 ceph-mon[35193]: pgmap v243: 1 pgs: 1 active+clean; 577 KiB data, 246 MiB used, 715 GiB / 715 GiB avail 2024-03-21T10:18:59.240 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:18:59 smithi157 ceph-mon[35581]: pgmap v244: 1 pgs: 1 active+clean; 577 KiB data, 246 MiB used, 715 GiB / 715 GiB avail 2024-03-21T10:18:59.464 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:18:59 smithi047 ceph-mon[31547]: pgmap v244: 1 pgs: 1 active+clean; 577 KiB data, 246 MiB used, 715 GiB / 715 GiB avail 2024-03-21T10:18:59.479 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:18:59 smithi100 ceph-mon[35193]: pgmap v244: 1 pgs: 1 active+clean; 577 KiB data, 246 MiB used, 715 GiB / 715 GiB avail 2024-03-21T10:19:00.740 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:19:00 smithi157 ceph-mon[35581]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:19:00.740 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:19:00 smithi157 ceph-mon[35581]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:19:00.740 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:19:00 smithi157 ceph-mon[35581]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T10:19:00.964 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:19:00 smithi047 ceph-mon[31547]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:19:00.964 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:19:00 smithi047 ceph-mon[31547]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:19:00.964 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:19:00 smithi047 ceph-mon[31547]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T10:19:00.980 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:19:00 smithi100 ceph-mon[35193]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:19:00.981 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:19:00 smithi100 ceph-mon[35193]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:19:00.981 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:19:00 smithi100 ceph-mon[35193]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T10:19:01.923 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:19:01 smithi157 ceph-mon[35581]: pgmap v245: 1 pgs: 1 active+clean; 577 KiB data, 246 MiB used, 715 GiB / 715 GiB avail 2024-03-21T10:19:01.963 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:19:01 smithi047 ceph-mon[31547]: pgmap v245: 1 pgs: 1 active+clean; 577 KiB data, 246 MiB used, 715 GiB / 715 GiB avail 2024-03-21T10:19:01.979 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:19:01 smithi100 ceph-mon[35193]: pgmap v245: 1 pgs: 1 active+clean; 577 KiB data, 246 MiB used, 715 GiB / 715 GiB avail 2024-03-21T10:19:03.740 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:19:03 smithi157 ceph-mon[35581]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 246 MiB used, 715 GiB / 715 GiB avail 2024-03-21T10:19:03.740 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:19:03 smithi157 ceph-mon[35581]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:19:03.740 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:19:03 smithi157 ceph-mon[35581]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:19:03.740 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:19:03 smithi157 ceph-mon[35581]: from='osd.8 [v2:172.21.15.157:6816/4067715347,v1:172.21.15.157:6817/4067715347]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-03-21T10:19:03.740 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:19:03 smithi157 ceph-mon[35581]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-03-21T10:19:03.964 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:19:03 smithi047 ceph-mon[31547]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 246 MiB used, 715 GiB / 715 GiB avail 2024-03-21T10:19:03.964 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:19:03 smithi047 ceph-mon[31547]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:19:03.964 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:19:03 smithi047 ceph-mon[31547]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:19:03.964 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:19:03 smithi047 ceph-mon[31547]: from='osd.8 [v2:172.21.15.157:6816/4067715347,v1:172.21.15.157:6817/4067715347]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-03-21T10:19:03.964 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:19:03 smithi047 ceph-mon[31547]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-03-21T10:19:03.979 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:19:03 smithi100 ceph-mon[35193]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 246 MiB used, 715 GiB / 715 GiB avail 2024-03-21T10:19:03.979 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:19:03 smithi100 ceph-mon[35193]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:19:03.980 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:19:03 smithi100 ceph-mon[35193]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:19:03.980 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:19:03 smithi100 ceph-mon[35193]: from='osd.8 [v2:172.21.15.157:6816/4067715347,v1:172.21.15.157:6817/4067715347]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-03-21T10:19:03.980 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:19:03 smithi100 ceph-mon[35193]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-03-21T10:19:04.388 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-21T10:19:04.386+0000 7f483cc00700 1 -- 172.21.15.157:0/4192638743 <== mgr.14152 v2:172.21.15.47:6800/2823885994 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f48400625d0 con 0x7f48240a3030 2024-03-21T10:19:04.388 INFO:teuthology.orchestra.run.smithi157.stdout:Created osd(s) 8 on host 'smithi157' 2024-03-21T10:19:04.391 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-21T10:19:04.389+0000 7f4845a00700 1 -- 172.21.15.157:0/4192638743 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f48240a3030 msgr2=0x7f48240a54f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:19:04.391 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-21T10:19:04.389+0000 7f4845a00700 1 --2- 172.21.15.157:0/4192638743 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f48240a3030 0x7f48240a54f0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f4830005950 tx=0x7f48300058e0 comp rx=0 tx=0).stop 2024-03-21T10:19:04.391 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-21T10:19:04.389+0000 7f4845a00700 1 -- 172.21.15.157:0/4192638743 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4840105510 msgr2=0x7f484019aa10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:19:04.391 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-21T10:19:04.389+0000 7f4845a00700 1 --2- 172.21.15.157:0/4192638743 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4840105510 0x7f484019aa10 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f482c005910 tx=0x7f482c0047b0 comp rx=0 tx=0).stop 2024-03-21T10:19:04.391 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-21T10:19:04.389+0000 7f4845a00700 1 -- 172.21.15.157:0/4192638743 shutdown_connections 2024-03-21T10:19:04.392 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-21T10:19:04.389+0000 7f4845a00700 1 --2- 172.21.15.157:0/4192638743 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f48240a3030 0x7f48240a54f0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:19:04.392 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-21T10:19:04.389+0000 7f4845a00700 1 --2- 172.21.15.157:0/4192638743 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4840105eb0 0x7f48401a1a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:19:04.392 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-21T10:19:04.389+0000 7f4845a00700 1 --2- 172.21.15.157:0/4192638743 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4840104310 0x7f484019a4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:19:04.392 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-21T10:19:04.389+0000 7f4845a00700 1 --2- 172.21.15.157:0/4192638743 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4840105510 0x7f484019aa10 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:19:04.392 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-21T10:19:04.389+0000 7f4845a00700 1 -- 172.21.15.157:0/4192638743 >> 172.21.15.157:0/4192638743 conn(0x7f48400ff8b0 msgr2=0x7f4840108830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:19:04.392 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-21T10:19:04.390+0000 7f4845a00700 1 -- 172.21.15.157:0/4192638743 shutdown_connections 2024-03-21T10:19:04.392 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-21T10:19:04.390+0000 7f4845a00700 1 -- 172.21.15.157:0/4192638743 wait complete. 2024-03-21T10:19:05.167 DEBUG:teuthology.orchestra.run.smithi157:osd.8> sudo journalctl -f -n 0 -u ceph-3056d51a-e76b-11ee-95cd-87774f69a715@osd.8.service 2024-03-21T10:19:05.172 INFO:tasks.cephadm:Deploying osd.9 on smithi047 with /dev/vg_nvme/lv_1... 2024-03-21T10:19:05.172 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2edfee68ad7243bed422a049b6f5f58619300f11 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3056d51a-e76b-11ee-95cd-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-03-21T10:19:05.458 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:19:05 smithi157 ceph-mon[35581]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 246 MiB used, 715 GiB / 715 GiB avail 2024-03-21T10:19:05.458 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:19:05 smithi157 ceph-mon[35581]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-03-21T10:19:05.458 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:19:05 smithi157 ceph-mon[35581]: from='osd.8 [v2:172.21.15.157:6816/4067715347,v1:172.21.15.157:6817/4067715347]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi157", "root=default"]}]: dispatch 2024-03-21T10:19:05.458 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:19:05 smithi157 ceph-mon[35581]: osdmap e59: 9 total, 8 up, 9 in 2024-03-21T10:19:05.458 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:19:05 smithi157 ceph-mon[35581]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-21T10:19:05.459 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:19:05 smithi157 ceph-mon[35581]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi157", "root=default"]}]: dispatch 2024-03-21T10:19:05.459 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:19:05 smithi157 ceph-mon[35581]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:19:05.459 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:19:05 smithi157 ceph-mon[35581]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:19:05.463 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:19:05 smithi047 ceph-mon[31547]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 246 MiB used, 715 GiB / 715 GiB avail 2024-03-21T10:19:05.464 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:19:05 smithi047 ceph-mon[31547]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-03-21T10:19:05.464 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:19:05 smithi047 ceph-mon[31547]: from='osd.8 [v2:172.21.15.157:6816/4067715347,v1:172.21.15.157:6817/4067715347]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi157", "root=default"]}]: dispatch 2024-03-21T10:19:05.464 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:19:05 smithi047 ceph-mon[31547]: osdmap e59: 9 total, 8 up, 9 in 2024-03-21T10:19:05.464 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:19:05 smithi047 ceph-mon[31547]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-21T10:19:05.464 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:19:05 smithi047 ceph-mon[31547]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi157", "root=default"]}]: dispatch 2024-03-21T10:19:05.464 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:19:05 smithi047 ceph-mon[31547]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:19:05.464 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:19:05 smithi047 ceph-mon[31547]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:19:05.480 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:19:05 smithi100 ceph-mon[35193]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 246 MiB used, 715 GiB / 715 GiB avail 2024-03-21T10:19:05.480 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:19:05 smithi100 ceph-mon[35193]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-03-21T10:19:05.480 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:19:05 smithi100 ceph-mon[35193]: from='osd.8 [v2:172.21.15.157:6816/4067715347,v1:172.21.15.157:6817/4067715347]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi157", "root=default"]}]: dispatch 2024-03-21T10:19:05.480 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:19:05 smithi100 ceph-mon[35193]: osdmap e59: 9 total, 8 up, 9 in 2024-03-21T10:19:05.480 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:19:05 smithi100 ceph-mon[35193]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-21T10:19:05.480 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:19:05 smithi100 ceph-mon[35193]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi157", "root=default"]}]: dispatch 2024-03-21T10:19:05.480 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:19:05 smithi100 ceph-mon[35193]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:19:05.480 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:19:05 smithi100 ceph-mon[35193]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:19:05.632 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/3056d51a-e76b-11ee-95cd-87774f69a715/mon.a/config 2024-03-21T10:19:06.228 INFO:journalctl@ceph.osd.8.smithi157.stdout:Mar 21 10:19:06 smithi157 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-8[54391]: 2024-03-21T10:19:06.005+0000 7fca8e200700 -1 osd.8 0 waiting for initial osdmap 2024-03-21T10:19:06.228 INFO:journalctl@ceph.osd.8.smithi157.stdout:Mar 21 10:19:06 smithi157 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-8[54391]: 2024-03-21T10:19:06.032+0000 7fca84000700 -1 osd.8 60 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-21T10:19:06.464 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:19:06 smithi047 ceph-mon[31547]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi157", "root=default"]}]': finished 2024-03-21T10:19:06.464 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:19:06 smithi047 ceph-mon[31547]: osdmap e60: 9 total, 8 up, 9 in 2024-03-21T10:19:06.464 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:19:06 smithi047 ceph-mon[31547]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-21T10:19:06.464 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:19:06 smithi047 ceph-mon[31547]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-21T10:19:06.464 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:19:06 smithi047 ceph-mon[31547]: from='osd.8 ' entity='osd.8' 2024-03-21T10:19:06.479 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:19:06 smithi100 ceph-mon[35193]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi157", "root=default"]}]': finished 2024-03-21T10:19:06.479 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:19:06 smithi100 ceph-mon[35193]: osdmap e60: 9 total, 8 up, 9 in 2024-03-21T10:19:06.479 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:19:06 smithi100 ceph-mon[35193]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-21T10:19:06.479 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:19:06 smithi100 ceph-mon[35193]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-21T10:19:06.480 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:19:06 smithi100 ceph-mon[35193]: from='osd.8 ' entity='osd.8' 2024-03-21T10:19:06.489 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:19:06 smithi157 ceph-mon[35581]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi157", "root=default"]}]': finished 2024-03-21T10:19:06.490 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:19:06 smithi157 ceph-mon[35581]: osdmap e60: 9 total, 8 up, 9 in 2024-03-21T10:19:06.490 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:19:06 smithi157 ceph-mon[35581]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-21T10:19:06.490 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:19:06 smithi157 ceph-mon[35581]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-21T10:19:06.490 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:19:06 smithi157 ceph-mon[35581]: from='osd.8 ' entity='osd.8' 2024-03-21T10:19:07.479 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:19:07 smithi100 ceph-mon[35193]: purged_snaps scrub starts 2024-03-21T10:19:07.480 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:19:07 smithi100 ceph-mon[35193]: purged_snaps scrub ok 2024-03-21T10:19:07.480 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:19:07 smithi100 ceph-mon[35193]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 246 MiB used, 715 GiB / 715 GiB avail 2024-03-21T10:19:07.480 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:19:07 smithi100 ceph-mon[35193]: osd.8 [v2:172.21.15.157:6816/4067715347,v1:172.21.15.157:6817/4067715347] boot 2024-03-21T10:19:07.480 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:19:07 smithi100 ceph-mon[35193]: osdmap e61: 9 total, 9 up, 9 in 2024-03-21T10:19:07.480 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:19:07 smithi100 ceph-mon[35193]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-21T10:19:07.490 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:19:07 smithi157 ceph-mon[35581]: purged_snaps scrub starts 2024-03-21T10:19:07.490 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:19:07 smithi157 ceph-mon[35581]: purged_snaps scrub ok 2024-03-21T10:19:07.490 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:19:07 smithi157 ceph-mon[35581]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 246 MiB used, 715 GiB / 715 GiB avail 2024-03-21T10:19:07.490 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:19:07 smithi157 ceph-mon[35581]: osd.8 [v2:172.21.15.157:6816/4067715347,v1:172.21.15.157:6817/4067715347] boot 2024-03-21T10:19:07.490 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:19:07 smithi157 ceph-mon[35581]: osdmap e61: 9 total, 9 up, 9 in 2024-03-21T10:19:07.490 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:19:07 smithi157 ceph-mon[35581]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-21T10:19:07.528 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:19:07 smithi047 ceph-mon[31547]: purged_snaps scrub starts 2024-03-21T10:19:07.528 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:19:07 smithi047 ceph-mon[31547]: purged_snaps scrub ok 2024-03-21T10:19:07.528 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:19:07 smithi047 ceph-mon[31547]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 246 MiB used, 715 GiB / 715 GiB avail 2024-03-21T10:19:07.528 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:19:07 smithi047 ceph-mon[31547]: osd.8 [v2:172.21.15.157:6816/4067715347,v1:172.21.15.157:6817/4067715347] boot 2024-03-21T10:19:07.528 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:19:07 smithi047 ceph-mon[31547]: osdmap e61: 9 total, 9 up, 9 in 2024-03-21T10:19:07.528 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:19:07 smithi047 ceph-mon[31547]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-21T10:19:08.648 INFO:teuthology.orchestra.run.smithi047.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-03-21T10:19:08.649 INFO:teuthology.orchestra.run.smithi047.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-03-21T10:19:08.649 INFO:teuthology.orchestra.run.smithi047.stderr: stderr: 10+0 records in 2024-03-21T10:19:08.649 INFO:teuthology.orchestra.run.smithi047.stderr:10+0 records out 2024-03-21T10:19:08.649 INFO:teuthology.orchestra.run.smithi047.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0387173 s, 271 MB/s 2024-03-21T10:19:08.649 INFO:teuthology.orchestra.run.smithi047.stderr:--> Zapping successful for: 2024-03-21T10:19:08.649 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:19:08 smithi047 ceph-mon[31547]: osdmap e62: 9 total, 9 up, 9 in 2024-03-21T10:19:08.729 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:19:08 smithi100 ceph-mon[35193]: osdmap e62: 9 total, 9 up, 9 in 2024-03-21T10:19:08.739 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:19:08 smithi157 ceph-mon[35581]: osdmap e62: 9 total, 9 up, 9 in 2024-03-21T10:19:09.440 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:19:09 smithi047 ceph-mon[31547]: pgmap v253: 1 pgs: 1 remapped+peering; 577 KiB data, 285 MiB used, 804 GiB / 805 GiB avail 2024-03-21T10:19:09.440 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:19:09 smithi047 ceph-mon[31547]: osdmap e63: 9 total, 9 up, 9 in 2024-03-21T10:19:09.440 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:19:09 smithi047 ceph-mon[31547]: osdmap e64: 9 total, 9 up, 9 in 2024-03-21T10:19:09.490 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:19:09 smithi157 ceph-mon[35581]: pgmap v253: 1 pgs: 1 remapped+peering; 577 KiB data, 285 MiB used, 804 GiB / 805 GiB avail 2024-03-21T10:19:09.490 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:19:09 smithi157 ceph-mon[35581]: osdmap e63: 9 total, 9 up, 9 in 2024-03-21T10:19:09.490 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:19:09 smithi157 ceph-mon[35581]: osdmap e64: 9 total, 9 up, 9 in 2024-03-21T10:19:09.522 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2edfee68ad7243bed422a049b6f5f58619300f11 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3056d51a-e76b-11ee-95cd-87774f69a715 -- ceph orch daemon add osd smithi047:vg_nvme/lv_1 2024-03-21T10:19:09.729 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:19:09 smithi100 ceph-mon[35193]: pgmap v253: 1 pgs: 1 remapped+peering; 577 KiB data, 285 MiB used, 804 GiB / 805 GiB avail 2024-03-21T10:19:09.729 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:19:09 smithi100 ceph-mon[35193]: osdmap e63: 9 total, 9 up, 9 in 2024-03-21T10:19:09.730 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:19:09 smithi100 ceph-mon[35193]: osdmap e64: 9 total, 9 up, 9 in 2024-03-21T10:19:09.997 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/3056d51a-e76b-11ee-95cd-87774f69a715/mon.a/config 2024-03-21T10:19:10.964 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:19:10 smithi047 ceph-mon[31547]: osdmap e65: 9 total, 9 up, 9 in 2024-03-21T10:19:10.964 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:19:10 smithi047 ceph-mon[31547]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:19:10.964 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:19:10 smithi047 ceph-mon[31547]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:19:10.964 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:19:10 smithi047 ceph-mon[31547]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-21T10:19:10.964 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:19:10 smithi047 ceph-mon[31547]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-21T10:19:10.965 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:19:10 smithi047 ceph-mon[31547]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-03-21T10:19:10.965 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:19:10 smithi047 ceph-mon[31547]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:19:10.965 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:19:10 smithi047 ceph-mon[31547]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T10:19:10.965 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:19:10 smithi047 ceph-mon[31547]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T10:19:10.965 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:19:10 smithi047 ceph-mon[31547]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:19:10.979 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:19:10 smithi100 ceph-mon[35193]: osdmap e65: 9 total, 9 up, 9 in 2024-03-21T10:19:10.979 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:19:10 smithi100 ceph-mon[35193]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:19:10.979 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:19:10 smithi100 ceph-mon[35193]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:19:10.979 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:19:10 smithi100 ceph-mon[35193]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-21T10:19:10.979 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:19:10 smithi100 ceph-mon[35193]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-21T10:19:10.979 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:19:10 smithi100 ceph-mon[35193]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-03-21T10:19:10.979 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:19:10 smithi100 ceph-mon[35193]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:19:10.980 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:19:10 smithi100 ceph-mon[35193]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T10:19:10.980 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:19:10 smithi100 ceph-mon[35193]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T10:19:10.980 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:19:10 smithi100 ceph-mon[35193]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:19:10.990 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:19:10 smithi157 ceph-mon[35581]: osdmap e65: 9 total, 9 up, 9 in 2024-03-21T10:19:10.990 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:19:10 smithi157 ceph-mon[35581]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:19:10.990 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:19:10 smithi157 ceph-mon[35581]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:19:10.990 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:19:10 smithi157 ceph-mon[35581]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-21T10:19:10.990 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:19:10 smithi157 ceph-mon[35581]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-21T10:19:10.990 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:19:10 smithi157 ceph-mon[35581]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-03-21T10:19:10.990 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:19:10 smithi157 ceph-mon[35581]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:19:10.991 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:19:10 smithi157 ceph-mon[35581]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T10:19:10.991 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:19:10 smithi157 ceph-mon[35581]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T10:19:10.991 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:19:10 smithi157 ceph-mon[35581]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:19:11.693 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:19:11 smithi047 ceph-mon[31547]: pgmap v257: 1 pgs: 1 remapped+peering; 577 KiB data, 285 MiB used, 804 GiB / 805 GiB avail 2024-03-21T10:19:11.693 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:19:11 smithi047 ceph-mon[31547]: Detected new or changed devices on smithi157 2024-03-21T10:19:11.693 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:19:11 smithi047 ceph-mon[31547]: Adjusting osd_memory_target on smithi157 to 5697M 2024-03-21T10:19:11.693 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:19:11 smithi047 ceph-mon[31547]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T10:19:11.693 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:19:11 smithi047 ceph-mon[31547]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T10:19:11.693 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:19:11 smithi047 ceph-mon[31547]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T10:19:11.693 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:19:11 smithi047 ceph-mon[31547]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:19:11.980 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:19:11 smithi100 ceph-mon[35193]: pgmap v257: 1 pgs: 1 remapped+peering; 577 KiB data, 285 MiB used, 804 GiB / 805 GiB avail 2024-03-21T10:19:11.980 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:19:11 smithi100 ceph-mon[35193]: Detected new or changed devices on smithi157 2024-03-21T10:19:11.980 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:19:11 smithi100 ceph-mon[35193]: Adjusting osd_memory_target on smithi157 to 5697M 2024-03-21T10:19:11.980 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:19:11 smithi100 ceph-mon[35193]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T10:19:11.980 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:19:11 smithi100 ceph-mon[35193]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T10:19:11.980 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:19:11 smithi100 ceph-mon[35193]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T10:19:11.980 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:19:11 smithi100 ceph-mon[35193]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:19:11.990 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:19:11 smithi157 ceph-mon[35581]: pgmap v257: 1 pgs: 1 remapped+peering; 577 KiB data, 285 MiB used, 804 GiB / 805 GiB avail 2024-03-21T10:19:11.990 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:19:11 smithi157 ceph-mon[35581]: Detected new or changed devices on smithi157 2024-03-21T10:19:11.990 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:19:11 smithi157 ceph-mon[35581]: Adjusting osd_memory_target on smithi157 to 5697M 2024-03-21T10:19:11.990 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:19:11 smithi157 ceph-mon[35581]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T10:19:11.990 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:19:11 smithi157 ceph-mon[35581]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T10:19:11.990 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:19:11 smithi157 ceph-mon[35581]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T10:19:11.990 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:19:11 smithi157 ceph-mon[35581]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:19:12.365 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:19:12.363+0000 7ffbb0200700 1 -- 172.21.15.47:0/4009907881 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ffba8103410 msgr2=0x7ffba8103870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:19:12.365 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:19:12.363+0000 7ffbb0200700 1 --2- 172.21.15.47:0/4009907881 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ffba8103410 0x7ffba8103870 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7ffb940099c0 tx=0x7ffb94009cd0 comp rx=0 tx=0).stop 2024-03-21T10:19:12.365 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:19:12.363+0000 7ffbb0200700 1 -- 172.21.15.47:0/4009907881 shutdown_connections 2024-03-21T10:19:12.365 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:19:12.363+0000 7ffbb0200700 1 --2- 172.21.15.47:0/4009907881 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ffba8103db0 0x7ffba810a660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:19:12.365 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:19:12.363+0000 7ffbb0200700 1 --2- 172.21.15.47:0/4009907881 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ffba8102210 0x7ffba8102630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:19:12.365 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:19:12.363+0000 7ffbb0200700 1 --2- 172.21.15.47:0/4009907881 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ffba8103410 0x7ffba8103870 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:19:12.365 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:19:12.363+0000 7ffbb0200700 1 -- 172.21.15.47:0/4009907881 >> 172.21.15.47:0/4009907881 conn(0x7ffba80fd7b0 msgr2=0x7ffba80ffbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:19:12.365 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:19:12.363+0000 7ffbb0200700 1 -- 172.21.15.47:0/4009907881 shutdown_connections 2024-03-21T10:19:12.365 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:19:12.363+0000 7ffbb0200700 1 -- 172.21.15.47:0/4009907881 wait complete. 2024-03-21T10:19:12.366 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:19:12.364+0000 7ffbb0200700 1 Processor -- start 2024-03-21T10:19:12.366 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:19:12.365+0000 7ffbb0200700 1 -- start start 2024-03-21T10:19:12.367 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:19:12.365+0000 7ffbb0200700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ffba8102210 0x7ffba8075f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:19:12.367 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:19:12.365+0000 7ffbb0200700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ffba8103410 0x7ffba8074580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:19:12.367 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:19:12.365+0000 7ffbb0200700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ffba8103db0 0x7ffba8074ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:19:12.367 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:19:12.365+0000 7ffbb0200700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffba810dca0 con 0x7ffba8103db0 2024-03-21T10:19:12.367 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:19:12.365+0000 7ffbb0200700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffba810db20 con 0x7ffba8102210 2024-03-21T10:19:12.367 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:19:12.365+0000 7ffbb0200700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffba810de20 con 0x7ffba8103410 2024-03-21T10:19:12.367 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:19:12.366+0000 7ffbae200700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ffba8103db0 0x7ffba8074ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:19:12.368 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:19:12.366+0000 7ffbae200700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ffba8103db0 0x7ffba8074ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:34844/0 (socket says 172.21.15.47:34844) 2024-03-21T10:19:12.368 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:19:12.366+0000 7ffbae200700 1 -- 172.21.15.47:0/1102474319 learned_addr learned my addr 172.21.15.47:0/1102474319 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:19:12.368 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:19:12.366+0000 7ffbad800700 1 --2- 172.21.15.47:0/1102474319 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ffba8102210 0x7ffba8075f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:19:12.368 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:19:12.366+0000 7ffbace00700 1 --2- 172.21.15.47:0/1102474319 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ffba8103410 0x7ffba8074580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:19:12.369 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:19:12.366+0000 7ffbae200700 1 -- 172.21.15.47:0/1102474319 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ffba8103410 msgr2=0x7ffba8074580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:19:12.369 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:19:12.366+0000 7ffbae200700 1 --2- 172.21.15.47:0/1102474319 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ffba8103410 0x7ffba8074580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:19:12.369 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:19:12.366+0000 7ffbae200700 1 -- 172.21.15.47:0/1102474319 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ffba8102210 msgr2=0x7ffba8075f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:19:12.369 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:19:12.366+0000 7ffbae200700 1 --2- 172.21.15.47:0/1102474319 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ffba8102210 0x7ffba8075f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:19:12.369 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:19:12.366+0000 7ffbae200700 1 -- 172.21.15.47:0/1102474319 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffba8075370 con 0x7ffba8103db0 2024-03-21T10:19:12.369 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:19:12.367+0000 7ffbae200700 1 --2- 172.21.15.47:0/1102474319 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ffba8103db0 0x7ffba8074ac0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7ffba000ef30 tx=0x7ffba000c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:19:12.369 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:19:12.367+0000 7ffba6000700 1 -- 172.21.15.47:0/1102474319 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffba00092e0 con 0x7ffba8103db0 2024-03-21T10:19:12.369 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:19:12.367+0000 7ffba6000700 1 -- 172.21.15.47:0/1102474319 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7ffba0004bb0 con 0x7ffba8103db0 2024-03-21T10:19:12.369 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:19:12.367+0000 7ffba6000700 1 -- 172.21.15.47:0/1102474319 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffba0004060 con 0x7ffba8103db0 2024-03-21T10:19:12.369 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:19:12.367+0000 7ffbb0200700 1 -- 172.21.15.47:0/1102474319 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffba81a2410 con 0x7ffba8103db0 2024-03-21T10:19:12.369 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:19:12.367+0000 7ffbb0200700 1 -- 172.21.15.47:0/1102474319 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffba81a28d0 con 0x7ffba8103db0 2024-03-21T10:19:12.370 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:19:12.368+0000 7ffbb0200700 1 -- 172.21.15.47:0/1102474319 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffba8068e50 con 0x7ffba8103db0 2024-03-21T10:19:12.371 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:19:12.370+0000 7ffba6000700 1 -- 172.21.15.47:0/1102474319 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 16) v1 ==== 137854+0+0 (secure 0 0 0) 0x7ffba001b030 con 0x7ffba8103db0 2024-03-21T10:19:12.373 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:19:12.372+0000 7ffba6000700 1 --2- 172.21.15.47:0/1102474319 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7ffb900a2fb0 0x7ffb900a5470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:19:12.373 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:19:12.372+0000 7ffba6000700 1 -- 172.21.15.47:0/1102474319 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 5829+0+0 (secure 0 0 0) 0x7ffba00ce660 con 0x7ffba8103db0 2024-03-21T10:19:12.373 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:19:12.372+0000 7ffbad800700 1 --2- 172.21.15.47:0/1102474319 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7ffb900a2fb0 0x7ffb900a5470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:19:12.374 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:19:12.373+0000 7ffbad800700 1 --2- 172.21.15.47:0/1102474319 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7ffb900a2fb0 0x7ffb900a5470 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7ffba8103270 tx=0x7ffb9c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:19:12.378 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:19:12.377+0000 7ffba6000700 1 -- 172.21.15.47:0/1102474319 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7ffba0022020 con 0x7ffba8103db0 2024-03-21T10:19:12.615 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:19:12.614+0000 7ffbb0200700 1 -- 172.21.15.47:0/1102474319 --> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi047:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) v1 -- 0x7ffba80625d0 con 0x7ffb900a2fb0 2024-03-21T10:19:13.537 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:19:13 smithi047 ceph-mon[31547]: pgmap v258: 1 pgs: 1 remapped+peering; 577 KiB data, 285 MiB used, 804 GiB / 805 GiB avail 2024-03-21T10:19:13.537 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:19:13 smithi047 ceph-mon[31547]: from='client.14394 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi047:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:19:13.802 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:19:13 smithi047 ceph-mon[31547]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-21T10:19:13.802 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:19:13 smithi047 ceph-mon[31547]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-21T10:19:13.802 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:19:13 smithi047 ceph-mon[31547]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T10:19:13.979 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:19:13 smithi100 ceph-mon[35193]: pgmap v258: 1 pgs: 1 remapped+peering; 577 KiB data, 285 MiB used, 804 GiB / 805 GiB avail 2024-03-21T10:19:13.980 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:19:13 smithi100 ceph-mon[35193]: from='client.14394 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi047:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:19:13.980 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:19:13 smithi100 ceph-mon[35193]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-21T10:19:13.980 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:19:13 smithi100 ceph-mon[35193]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-21T10:19:13.980 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:19:13 smithi100 ceph-mon[35193]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T10:19:13.989 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:19:13 smithi157 ceph-mon[35581]: pgmap v258: 1 pgs: 1 remapped+peering; 577 KiB data, 285 MiB used, 804 GiB / 805 GiB avail 2024-03-21T10:19:13.990 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:19:13 smithi157 ceph-mon[35581]: from='client.14394 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi047:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:19:13.990 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:19:13 smithi157 ceph-mon[35581]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-21T10:19:13.990 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:19:13 smithi157 ceph-mon[35581]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-21T10:19:13.990 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:19:13 smithi157 ceph-mon[35581]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T10:19:15.646 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:19:15 smithi047 ceph-mon[31547]: pgmap v259: 1 pgs: 1 active+clean; 577 KiB data, 285 MiB used, 804 GiB / 805 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-03-21T10:19:15.979 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:19:15 smithi100 ceph-mon[35193]: pgmap v259: 1 pgs: 1 active+clean; 577 KiB data, 285 MiB used, 804 GiB / 805 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-03-21T10:19:15.989 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:19:15 smithi157 ceph-mon[35581]: pgmap v259: 1 pgs: 1 active+clean; 577 KiB data, 285 MiB used, 804 GiB / 805 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-03-21T10:19:17.714 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:19:17 smithi047 ceph-mon[31547]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 286 MiB used, 804 GiB / 805 GiB avail; 75 KiB/s, 0 objects/s recovering 2024-03-21T10:19:17.714 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:19:17 smithi047 ceph-mon[31547]: from='client.? 172.21.15.47:0/1718160723' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5ea1f5be-8458-4c93-9e61-c2e867a514f3"}]: dispatch 2024-03-21T10:19:17.714 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:19:17 smithi047 ceph-mon[31547]: from='client.? 172.21.15.47:0/1718160723' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "5ea1f5be-8458-4c93-9e61-c2e867a514f3"}]': finished 2024-03-21T10:19:17.714 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:19:17 smithi047 ceph-mon[31547]: osdmap e66: 10 total, 9 up, 10 in 2024-03-21T10:19:17.714 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:19:17 smithi047 ceph-mon[31547]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-21T10:19:17.782 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:19:17 smithi100 ceph-mon[35193]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 286 MiB used, 804 GiB / 805 GiB avail; 75 KiB/s, 0 objects/s recovering 2024-03-21T10:19:17.782 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:19:17 smithi100 ceph-mon[35193]: from='client.? 172.21.15.47:0/1718160723' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5ea1f5be-8458-4c93-9e61-c2e867a514f3"}]: dispatch 2024-03-21T10:19:17.782 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:19:17 smithi100 ceph-mon[35193]: from='client.? 172.21.15.47:0/1718160723' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "5ea1f5be-8458-4c93-9e61-c2e867a514f3"}]': finished 2024-03-21T10:19:17.782 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:19:17 smithi100 ceph-mon[35193]: osdmap e66: 10 total, 9 up, 10 in 2024-03-21T10:19:17.782 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:19:17 smithi100 ceph-mon[35193]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-21T10:19:17.990 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:19:17 smithi157 ceph-mon[35581]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 286 MiB used, 804 GiB / 805 GiB avail; 75 KiB/s, 0 objects/s recovering 2024-03-21T10:19:17.990 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:19:17 smithi157 ceph-mon[35581]: from='client.? 172.21.15.47:0/1718160723' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5ea1f5be-8458-4c93-9e61-c2e867a514f3"}]: dispatch 2024-03-21T10:19:17.990 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:19:17 smithi157 ceph-mon[35581]: from='client.? 172.21.15.47:0/1718160723' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "5ea1f5be-8458-4c93-9e61-c2e867a514f3"}]': finished 2024-03-21T10:19:17.990 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:19:17 smithi157 ceph-mon[35581]: osdmap e66: 10 total, 9 up, 10 in 2024-03-21T10:19:17.990 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:19:17 smithi157 ceph-mon[35581]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-21T10:19:18.729 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:19:18 smithi100 ceph-mon[35193]: from='client.? 172.21.15.47:0/243090474' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-21T10:19:18.964 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:19:18 smithi047 ceph-mon[31547]: from='client.? 172.21.15.47:0/243090474' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-21T10:19:18.989 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:19:18 smithi157 ceph-mon[35581]: from='client.? 172.21.15.47:0/243090474' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-21T10:19:19.729 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:19:19 smithi100 ceph-mon[35193]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 286 MiB used, 804 GiB / 805 GiB avail; 68 KiB/s, 0 objects/s recovering 2024-03-21T10:19:19.964 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:19:19 smithi047 ceph-mon[31547]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 286 MiB used, 804 GiB / 805 GiB avail; 68 KiB/s, 0 objects/s recovering 2024-03-21T10:19:19.990 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:19:19 smithi157 ceph-mon[35581]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 286 MiB used, 804 GiB / 805 GiB avail; 68 KiB/s, 0 objects/s recovering 2024-03-21T10:19:21.964 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:19:21 smithi047 ceph-mon[31547]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 286 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-03-21T10:19:21.979 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:19:21 smithi100 ceph-mon[35193]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 286 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-03-21T10:19:21.989 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:19:21 smithi157 ceph-mon[35581]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 286 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-03-21T10:19:23.964 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:19:23 smithi047 ceph-mon[31547]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 286 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-03-21T10:19:23.979 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:19:23 smithi100 ceph-mon[35193]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 286 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-03-21T10:19:23.989 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:19:23 smithi157 ceph-mon[35581]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 286 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-03-21T10:19:25.964 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:19:25 smithi047 ceph-mon[31547]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 286 MiB used, 804 GiB / 805 GiB avail 2024-03-21T10:19:25.979 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:19:25 smithi100 ceph-mon[35193]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 286 MiB used, 804 GiB / 805 GiB avail 2024-03-21T10:19:25.989 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:19:25 smithi157 ceph-mon[35581]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 286 MiB used, 804 GiB / 805 GiB avail 2024-03-21T10:19:27.924 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:19:27 smithi047 ceph-mon[31547]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 286 MiB used, 804 GiB / 805 GiB avail 2024-03-21T10:19:27.924 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:19:27 smithi047 ceph-mon[31547]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-03-21T10:19:27.924 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:19:27 smithi047 ceph-mon[31547]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T10:19:27.924 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:19:27 smithi047 ceph-mon[31547]: Deploying daemon osd.9 on smithi047 2024-03-21T10:19:27.979 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:19:27 smithi100 ceph-mon[35193]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 286 MiB used, 804 GiB / 805 GiB avail 2024-03-21T10:19:27.980 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:19:27 smithi100 ceph-mon[35193]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-03-21T10:19:27.980 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:19:27 smithi100 ceph-mon[35193]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T10:19:27.980 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:19:27 smithi100 ceph-mon[35193]: Deploying daemon osd.9 on smithi047 2024-03-21T10:19:27.990 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:19:27 smithi157 ceph-mon[35581]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 286 MiB used, 804 GiB / 805 GiB avail 2024-03-21T10:19:27.990 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:19:27 smithi157 ceph-mon[35581]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-03-21T10:19:27.990 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:19:27 smithi157 ceph-mon[35581]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T10:19:27.990 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:19:27 smithi157 ceph-mon[35581]: Deploying daemon osd.9 on smithi047 2024-03-21T10:19:29.729 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:19:29 smithi100 ceph-mon[35193]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 286 MiB used, 804 GiB / 805 GiB avail 2024-03-21T10:19:29.850 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:19:29 smithi047 ceph-mon[31547]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 286 MiB used, 804 GiB / 805 GiB avail 2024-03-21T10:19:29.990 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:19:29 smithi157 ceph-mon[35581]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 286 MiB used, 804 GiB / 805 GiB avail 2024-03-21T10:19:31.893 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:19:31 smithi047 ceph-mon[31547]: pgmap v268: 1 pgs: 1 active+clean; 577 KiB data, 286 MiB used, 804 GiB / 805 GiB avail 2024-03-21T10:19:31.979 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:19:31 smithi100 ceph-mon[35193]: pgmap v268: 1 pgs: 1 active+clean; 577 KiB data, 286 MiB used, 804 GiB / 805 GiB avail 2024-03-21T10:19:31.989 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:19:31 smithi157 ceph-mon[35581]: pgmap v268: 1 pgs: 1 active+clean; 577 KiB data, 286 MiB used, 804 GiB / 805 GiB avail 2024-03-21T10:19:33.855 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:19:33 smithi047 ceph-mon[31547]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 286 MiB used, 804 GiB / 805 GiB avail 2024-03-21T10:19:33.979 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:19:33 smithi100 ceph-mon[35193]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 286 MiB used, 804 GiB / 805 GiB avail 2024-03-21T10:19:33.989 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:19:33 smithi157 ceph-mon[35581]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 286 MiB used, 804 GiB / 805 GiB avail 2024-03-21T10:19:35.714 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:19:35 smithi047 ceph-mon[31547]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 286 MiB used, 804 GiB / 805 GiB avail 2024-03-21T10:19:35.979 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:19:35 smithi100 ceph-mon[35193]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 286 MiB used, 804 GiB / 805 GiB avail 2024-03-21T10:19:35.989 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:19:35 smithi157 ceph-mon[35581]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 286 MiB used, 804 GiB / 805 GiB avail 2024-03-21T10:19:36.922 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:19:36 smithi047 ceph-mon[31547]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:19:36.923 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:19:36 smithi047 ceph-mon[31547]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:19:36.923 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:19:36 smithi047 ceph-mon[31547]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T10:19:36.979 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:19:36 smithi100 ceph-mon[35193]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:19:36.979 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:19:36 smithi100 ceph-mon[35193]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:19:36.980 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:19:36 smithi100 ceph-mon[35193]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T10:19:36.989 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:19:36 smithi157 ceph-mon[35581]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:19:36.990 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:19:36 smithi157 ceph-mon[35581]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:19:36.990 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:19:36 smithi157 ceph-mon[35581]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T10:19:37.964 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:19:37 smithi047 ceph-mon[31547]: pgmap v271: 1 pgs: 1 active+clean; 577 KiB data, 286 MiB used, 804 GiB / 805 GiB avail 2024-03-21T10:19:37.979 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:19:37 smithi100 ceph-mon[35193]: pgmap v271: 1 pgs: 1 active+clean; 577 KiB data, 286 MiB used, 804 GiB / 805 GiB avail 2024-03-21T10:19:37.989 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:19:37 smithi157 ceph-mon[35581]: pgmap v271: 1 pgs: 1 active+clean; 577 KiB data, 286 MiB used, 804 GiB / 805 GiB avail 2024-03-21T10:19:39.964 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:19:39 smithi047 ceph-mon[31547]: pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 286 MiB used, 804 GiB / 805 GiB avail 2024-03-21T10:19:39.964 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:19:39 smithi047 ceph-mon[31547]: from='osd.9 [v2:172.21.15.47:6826/1982108738,v1:172.21.15.47:6827/1982108738]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-03-21T10:19:39.965 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:19:39 smithi047 ceph-mon[31547]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:19:39.965 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:19:39 smithi047 ceph-mon[31547]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:19:39.979 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:19:39 smithi100 ceph-mon[35193]: pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 286 MiB used, 804 GiB / 805 GiB avail 2024-03-21T10:19:39.980 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:19:39 smithi100 ceph-mon[35193]: from='osd.9 [v2:172.21.15.47:6826/1982108738,v1:172.21.15.47:6827/1982108738]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-03-21T10:19:39.980 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:19:39 smithi100 ceph-mon[35193]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:19:39.980 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:19:39 smithi100 ceph-mon[35193]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:19:39.989 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:19:39 smithi157 ceph-mon[35581]: pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 286 MiB used, 804 GiB / 805 GiB avail 2024-03-21T10:19:39.990 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:19:39 smithi157 ceph-mon[35581]: from='osd.9 [v2:172.21.15.47:6826/1982108738,v1:172.21.15.47:6827/1982108738]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-03-21T10:19:39.990 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:19:39 smithi157 ceph-mon[35581]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:19:39.990 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:19:39 smithi157 ceph-mon[35581]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:19:40.758 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:19:40.755+0000 7ffba6000700 1 -- 172.21.15.47:0/1102474319 <== mgr.14152 v2:172.21.15.47:6800/2823885994 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7ffba80625d0 con 0x7ffb900a2fb0 2024-03-21T10:19:40.758 INFO:teuthology.orchestra.run.smithi047.stdout:Created osd(s) 9 on host 'smithi047' 2024-03-21T10:19:40.761 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:19:40.759+0000 7ffbb0200700 1 -- 172.21.15.47:0/1102474319 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7ffb900a2fb0 msgr2=0x7ffb900a5470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:19:40.761 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:19:40.759+0000 7ffbb0200700 1 --2- 172.21.15.47:0/1102474319 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7ffb900a2fb0 0x7ffb900a5470 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7ffba8103270 tx=0x7ffb9c009260 comp rx=0 tx=0).stop 2024-03-21T10:19:40.762 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:19:40.759+0000 7ffbb0200700 1 -- 172.21.15.47:0/1102474319 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ffba8103db0 msgr2=0x7ffba8074ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:19:40.762 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:19:40.759+0000 7ffbb0200700 1 --2- 172.21.15.47:0/1102474319 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ffba8103db0 0x7ffba8074ac0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7ffba000ef30 tx=0x7ffba000c3a0 comp rx=0 tx=0).stop 2024-03-21T10:19:40.762 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:19:40.759+0000 7ffbb0200700 1 -- 172.21.15.47:0/1102474319 shutdown_connections 2024-03-21T10:19:40.762 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:19:40.759+0000 7ffbb0200700 1 --2- 172.21.15.47:0/1102474319 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7ffb900a2fb0 0x7ffb900a5470 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:19:40.762 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:19:40.759+0000 7ffbb0200700 1 --2- 172.21.15.47:0/1102474319 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ffba8103db0 0x7ffba8074ac0 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:19:40.762 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:19:40.759+0000 7ffbb0200700 1 --2- 172.21.15.47:0/1102474319 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ffba8102210 0x7ffba8075f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:19:40.762 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:19:40.759+0000 7ffbb0200700 1 --2- 172.21.15.47:0/1102474319 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ffba8103410 0x7ffba8074580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:19:40.762 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:19:40.759+0000 7ffbb0200700 1 -- 172.21.15.47:0/1102474319 >> 172.21.15.47:0/1102474319 conn(0x7ffba80fd7b0 msgr2=0x7ffba8106710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:19:40.763 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:19:40.759+0000 7ffbb0200700 1 -- 172.21.15.47:0/1102474319 shutdown_connections 2024-03-21T10:19:40.763 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:19:40.759+0000 7ffbb0200700 1 -- 172.21.15.47:0/1102474319 wait complete. 2024-03-21T10:19:40.964 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:19:40 smithi047 ceph-mon[31547]: from='osd.9 [v2:172.21.15.47:6826/1982108738,v1:172.21.15.47:6827/1982108738]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-03-21T10:19:40.964 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:19:40 smithi047 ceph-mon[31547]: osdmap e67: 10 total, 9 up, 10 in 2024-03-21T10:19:40.964 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:19:40 smithi047 ceph-mon[31547]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-21T10:19:40.964 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:19:40 smithi047 ceph-mon[31547]: from='osd.9 [v2:172.21.15.47:6826/1982108738,v1:172.21.15.47:6827/1982108738]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi047", "root=default"]}]: dispatch 2024-03-21T10:19:40.964 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:19:40 smithi047 ceph-mon[31547]: from='osd.9 [v2:172.21.15.47:6826/1982108738,v1:172.21.15.47:6827/1982108738]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi047", "root=default"]}]': finished 2024-03-21T10:19:40.979 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:19:40 smithi100 ceph-mon[35193]: from='osd.9 [v2:172.21.15.47:6826/1982108738,v1:172.21.15.47:6827/1982108738]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-03-21T10:19:40.979 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:19:40 smithi100 ceph-mon[35193]: osdmap e67: 10 total, 9 up, 10 in 2024-03-21T10:19:40.979 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:19:40 smithi100 ceph-mon[35193]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-21T10:19:40.979 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:19:40 smithi100 ceph-mon[35193]: from='osd.9 [v2:172.21.15.47:6826/1982108738,v1:172.21.15.47:6827/1982108738]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi047", "root=default"]}]: dispatch 2024-03-21T10:19:40.980 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:19:40 smithi100 ceph-mon[35193]: from='osd.9 [v2:172.21.15.47:6826/1982108738,v1:172.21.15.47:6827/1982108738]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi047", "root=default"]}]': finished 2024-03-21T10:19:40.989 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:19:40 smithi157 ceph-mon[35581]: from='osd.9 [v2:172.21.15.47:6826/1982108738,v1:172.21.15.47:6827/1982108738]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-03-21T10:19:40.990 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:19:40 smithi157 ceph-mon[35581]: osdmap e67: 10 total, 9 up, 10 in 2024-03-21T10:19:40.990 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:19:40 smithi157 ceph-mon[35581]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-21T10:19:40.990 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:19:40 smithi157 ceph-mon[35581]: from='osd.9 [v2:172.21.15.47:6826/1982108738,v1:172.21.15.47:6827/1982108738]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi047", "root=default"]}]: dispatch 2024-03-21T10:19:40.990 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:19:40 smithi157 ceph-mon[35581]: from='osd.9 [v2:172.21.15.47:6826/1982108738,v1:172.21.15.47:6827/1982108738]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi047", "root=default"]}]': finished 2024-03-21T10:19:41.583 DEBUG:teuthology.orchestra.run.smithi047:osd.9> sudo journalctl -f -n 0 -u ceph-3056d51a-e76b-11ee-95cd-87774f69a715@osd.9.service 2024-03-21T10:19:41.591 INFO:tasks.cephadm:Deploying osd.10 on smithi100 with /dev/vg_nvme/lv_1... 2024-03-21T10:19:41.591 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2edfee68ad7243bed422a049b6f5f58619300f11 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3056d51a-e76b-11ee-95cd-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-03-21T10:19:41.963 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:19:41 smithi047 ceph-mon[31547]: pgmap v274: 1 pgs: 1 active+clean; 577 KiB data, 286 MiB used, 804 GiB / 805 GiB avail 2024-03-21T10:19:41.964 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:19:41 smithi047 ceph-mon[31547]: osdmap e68: 10 total, 9 up, 10 in 2024-03-21T10:19:41.964 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:19:41 smithi047 ceph-mon[31547]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-21T10:19:41.964 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:19:41 smithi047 ceph-mon[31547]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-21T10:19:41.964 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:19:41 smithi047 ceph-mon[31547]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:19:41.964 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:19:41 smithi047 ceph-mon[31547]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:19:41.979 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:19:41 smithi100 ceph-mon[35193]: pgmap v274: 1 pgs: 1 active+clean; 577 KiB data, 286 MiB used, 804 GiB / 805 GiB avail 2024-03-21T10:19:41.979 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:19:41 smithi100 ceph-mon[35193]: osdmap e68: 10 total, 9 up, 10 in 2024-03-21T10:19:41.979 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:19:41 smithi100 ceph-mon[35193]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-21T10:19:41.979 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:19:41 smithi100 ceph-mon[35193]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-21T10:19:41.980 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:19:41 smithi100 ceph-mon[35193]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:19:41.980 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:19:41 smithi100 ceph-mon[35193]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:19:41.990 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:19:41 smithi157 ceph-mon[35581]: pgmap v274: 1 pgs: 1 active+clean; 577 KiB data, 286 MiB used, 804 GiB / 805 GiB avail 2024-03-21T10:19:41.990 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:19:41 smithi157 ceph-mon[35581]: osdmap e68: 10 total, 9 up, 10 in 2024-03-21T10:19:41.990 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:19:41 smithi157 ceph-mon[35581]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-21T10:19:41.990 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:19:41 smithi157 ceph-mon[35581]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-21T10:19:41.990 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:19:41 smithi157 ceph-mon[35581]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:19:41.990 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:19:41 smithi157 ceph-mon[35581]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:19:42.033 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/3056d51a-e76b-11ee-95cd-87774f69a715/mon.b/config 2024-03-21T10:19:42.964 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:19:42 smithi047 ceph-mon[31547]: purged_snaps scrub starts 2024-03-21T10:19:42.964 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:19:42 smithi047 ceph-mon[31547]: purged_snaps scrub ok 2024-03-21T10:19:42.964 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:19:42 smithi047 ceph-mon[31547]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-21T10:19:42.964 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:19:42 smithi047 ceph-mon[31547]: osd.9 [v2:172.21.15.47:6826/1982108738,v1:172.21.15.47:6827/1982108738] boot 2024-03-21T10:19:42.964 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:19:42 smithi047 ceph-mon[31547]: osdmap e69: 10 total, 10 up, 10 in 2024-03-21T10:19:42.965 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:19:42 smithi047 ceph-mon[31547]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-21T10:19:42.979 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:19:42 smithi100 ceph-mon[35193]: purged_snaps scrub starts 2024-03-21T10:19:42.980 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:19:42 smithi100 ceph-mon[35193]: purged_snaps scrub ok 2024-03-21T10:19:42.980 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:19:42 smithi100 ceph-mon[35193]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-21T10:19:42.980 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:19:42 smithi100 ceph-mon[35193]: osd.9 [v2:172.21.15.47:6826/1982108738,v1:172.21.15.47:6827/1982108738] boot 2024-03-21T10:19:42.980 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:19:42 smithi100 ceph-mon[35193]: osdmap e69: 10 total, 10 up, 10 in 2024-03-21T10:19:42.980 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:19:42 smithi100 ceph-mon[35193]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-21T10:19:42.990 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:19:42 smithi157 ceph-mon[35581]: purged_snaps scrub starts 2024-03-21T10:19:42.990 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:19:42 smithi157 ceph-mon[35581]: purged_snaps scrub ok 2024-03-21T10:19:42.990 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:19:42 smithi157 ceph-mon[35581]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-21T10:19:42.990 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:19:42 smithi157 ceph-mon[35581]: osd.9 [v2:172.21.15.47:6826/1982108738,v1:172.21.15.47:6827/1982108738] boot 2024-03-21T10:19:42.990 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:19:42 smithi157 ceph-mon[35581]: osdmap e69: 10 total, 10 up, 10 in 2024-03-21T10:19:42.990 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:19:42 smithi157 ceph-mon[35581]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-21T10:19:43.946 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:19:43 smithi100 ceph-mon[35193]: pgmap v277: 1 pgs: 1 active+clean; 577 KiB data, 286 MiB used, 804 GiB / 805 GiB avail 2024-03-21T10:19:43.947 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:19:43 smithi100 ceph-mon[35193]: osdmap e70: 10 total, 10 up, 10 in 2024-03-21T10:19:43.982 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:19:43 smithi047 ceph-mon[31547]: pgmap v277: 1 pgs: 1 active+clean; 577 KiB data, 286 MiB used, 804 GiB / 805 GiB avail 2024-03-21T10:19:43.983 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:19:43 smithi047 ceph-mon[31547]: osdmap e70: 10 total, 10 up, 10 in 2024-03-21T10:19:43.989 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:19:43 smithi157 ceph-mon[35581]: pgmap v277: 1 pgs: 1 active+clean; 577 KiB data, 286 MiB used, 804 GiB / 805 GiB avail 2024-03-21T10:19:43.989 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:19:43 smithi157 ceph-mon[35581]: osdmap e70: 10 total, 10 up, 10 in 2024-03-21T10:19:44.904 INFO:teuthology.orchestra.run.smithi100.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-03-21T10:19:44.905 INFO:teuthology.orchestra.run.smithi100.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-03-21T10:19:44.905 INFO:teuthology.orchestra.run.smithi100.stderr: stderr: 10+0 records in 2024-03-21T10:19:44.905 INFO:teuthology.orchestra.run.smithi100.stderr:10+0 records out 2024-03-21T10:19:44.905 INFO:teuthology.orchestra.run.smithi100.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0342594 s, 306 MB/s 2024-03-21T10:19:44.905 INFO:teuthology.orchestra.run.smithi100.stderr:--> Zapping successful for: 2024-03-21T10:19:45.763 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2edfee68ad7243bed422a049b6f5f58619300f11 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3056d51a-e76b-11ee-95cd-87774f69a715 -- ceph orch daemon add osd smithi100:vg_nvme/lv_1 2024-03-21T10:19:45.979 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:19:45 smithi100 ceph-mon[35193]: pgmap v279: 1 pgs: 1 active+clean; 577 KiB data, 313 MiB used, 894 GiB / 894 GiB avail 2024-03-21T10:19:45.989 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:19:45 smithi157 ceph-mon[35581]: pgmap v279: 1 pgs: 1 active+clean; 577 KiB data, 313 MiB used, 894 GiB / 894 GiB avail 2024-03-21T10:19:46.055 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:19:45 smithi047 ceph-mon[31547]: pgmap v279: 1 pgs: 1 active+clean; 577 KiB data, 313 MiB used, 894 GiB / 894 GiB avail 2024-03-21T10:19:46.200 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/3056d51a-e76b-11ee-95cd-87774f69a715/mon.b/config 2024-03-21T10:19:47.980 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:19:47 smithi100 ceph-mon[35193]: pgmap v280: 1 pgs: 1 active+clean; 577 KiB data, 313 MiB used, 894 GiB / 894 GiB avail 2024-03-21T10:19:47.981 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:19:47 smithi100 ceph-mon[35193]: Detected new or changed devices on smithi047 2024-03-21T10:19:47.981 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:19:47 smithi100 ceph-mon[35193]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:19:47.981 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:19:47 smithi100 ceph-mon[35193]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:19:47.981 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:19:47 smithi100 ceph-mon[35193]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-21T10:19:47.981 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:19:47 smithi100 ceph-mon[35193]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-21T10:19:47.981 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:19:47 smithi100 ceph-mon[35193]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-21T10:19:47.981 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:19:47 smithi100 ceph-mon[35193]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-03-21T10:19:47.981 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:19:47 smithi100 ceph-mon[35193]: Adjusting osd_memory_target on smithi047 to 4273M 2024-03-21T10:19:47.981 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:19:47 smithi100 ceph-mon[35193]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:19:47.981 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:19:47 smithi100 ceph-mon[35193]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T10:19:47.981 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:19:47 smithi100 ceph-mon[35193]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T10:19:47.981 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:19:47 smithi100 ceph-mon[35193]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:19:47.982 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:19:47 smithi100 ceph-mon[35193]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T10:19:47.982 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:19:47 smithi100 ceph-mon[35193]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T10:19:47.982 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:19:47 smithi100 ceph-mon[35193]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T10:19:47.982 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:19:47 smithi100 ceph-mon[35193]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:19:48.214 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:19:47 smithi047 ceph-mon[31547]: pgmap v280: 1 pgs: 1 active+clean; 577 KiB data, 313 MiB used, 894 GiB / 894 GiB avail 2024-03-21T10:19:48.214 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:19:47 smithi047 ceph-mon[31547]: Detected new or changed devices on smithi047 2024-03-21T10:19:48.215 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:19:47 smithi047 ceph-mon[31547]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:19:48.215 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:19:47 smithi047 ceph-mon[31547]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:19:48.215 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:19:47 smithi047 ceph-mon[31547]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-21T10:19:48.215 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:19:47 smithi047 ceph-mon[31547]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-21T10:19:48.215 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:19:47 smithi047 ceph-mon[31547]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-21T10:19:48.215 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:19:47 smithi047 ceph-mon[31547]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-03-21T10:19:48.215 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:19:47 smithi047 ceph-mon[31547]: Adjusting osd_memory_target on smithi047 to 4273M 2024-03-21T10:19:48.215 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:19:47 smithi047 ceph-mon[31547]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:19:48.215 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:19:47 smithi047 ceph-mon[31547]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T10:19:48.215 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:19:47 smithi047 ceph-mon[31547]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T10:19:48.215 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:19:47 smithi047 ceph-mon[31547]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:19:48.215 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:19:47 smithi047 ceph-mon[31547]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T10:19:48.215 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:19:47 smithi047 ceph-mon[31547]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T10:19:48.216 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:19:47 smithi047 ceph-mon[31547]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T10:19:48.216 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:19:47 smithi047 ceph-mon[31547]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:19:48.240 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:19:47 smithi157 ceph-mon[35581]: pgmap v280: 1 pgs: 1 active+clean; 577 KiB data, 313 MiB used, 894 GiB / 894 GiB avail 2024-03-21T10:19:48.240 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:19:47 smithi157 ceph-mon[35581]: Detected new or changed devices on smithi047 2024-03-21T10:19:48.240 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:19:47 smithi157 ceph-mon[35581]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:19:48.240 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:19:47 smithi157 ceph-mon[35581]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:19:48.240 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:19:47 smithi157 ceph-mon[35581]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-21T10:19:48.240 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:19:47 smithi157 ceph-mon[35581]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-21T10:19:48.241 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:19:47 smithi157 ceph-mon[35581]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-21T10:19:48.241 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:19:47 smithi157 ceph-mon[35581]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-03-21T10:19:48.241 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:19:47 smithi157 ceph-mon[35581]: Adjusting osd_memory_target on smithi047 to 4273M 2024-03-21T10:19:48.241 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:19:47 smithi157 ceph-mon[35581]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:19:48.241 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:19:47 smithi157 ceph-mon[35581]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T10:19:48.241 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:19:47 smithi157 ceph-mon[35581]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T10:19:48.241 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:19:47 smithi157 ceph-mon[35581]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:19:48.241 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:19:47 smithi157 ceph-mon[35581]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T10:19:48.241 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:19:47 smithi157 ceph-mon[35581]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T10:19:48.241 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:19:47 smithi157 ceph-mon[35581]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T10:19:48.241 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:19:47 smithi157 ceph-mon[35581]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:19:48.541 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-21T10:19:48.539+0000 7f25ff800700 1 -- 172.21.15.100:0/391884818 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f25f81033d0 msgr2=0x7f25f8103830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:19:48.541 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-21T10:19:48.539+0000 7f25ff800700 1 --2- 172.21.15.100:0/391884818 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f25f81033d0 0x7f25f8103830 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f25e40099c0 tx=0x7f25e4009cd0 comp rx=0 tx=0).stop 2024-03-21T10:19:48.541 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-21T10:19:48.539+0000 7f25ff800700 1 -- 172.21.15.100:0/391884818 shutdown_connections 2024-03-21T10:19:48.541 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-21T10:19:48.539+0000 7f25ff800700 1 --2- 172.21.15.100:0/391884818 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f25f8103d70 0x7f25f810a620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:19:48.541 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-21T10:19:48.539+0000 7f25ff800700 1 --2- 172.21.15.100:0/391884818 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f25f81021d0 0x7f25f81025f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:19:48.541 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-21T10:19:48.539+0000 7f25ff800700 1 --2- 172.21.15.100:0/391884818 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f25f81033d0 0x7f25f8103830 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:19:48.541 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-21T10:19:48.539+0000 7f25ff800700 1 -- 172.21.15.100:0/391884818 >> 172.21.15.100:0/391884818 conn(0x7f25f80fd730 msgr2=0x7f25f80ffbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:19:48.541 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-21T10:19:48.539+0000 7f25ff800700 1 -- 172.21.15.100:0/391884818 shutdown_connections 2024-03-21T10:19:48.541 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-21T10:19:48.539+0000 7f25ff800700 1 -- 172.21.15.100:0/391884818 wait complete. 2024-03-21T10:19:48.544 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-21T10:19:48.540+0000 7f25ff800700 1 Processor -- start 2024-03-21T10:19:48.544 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-21T10:19:48.540+0000 7f25ff800700 1 -- start start 2024-03-21T10:19:48.544 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-21T10:19:48.541+0000 7f25ff800700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f25f81021d0 0x7f25f819a470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:19:48.544 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-21T10:19:48.541+0000 7f25ff800700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f25f81033d0 0x7f25f819a9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:19:48.544 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-21T10:19:48.541+0000 7f25ff800700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f25f8103d70 0x7f25f81a1a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:19:48.544 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-21T10:19:48.541+0000 7f25ff800700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25f810dc40 con 0x7f25f81033d0 2024-03-21T10:19:48.544 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-21T10:19:48.541+0000 7f25ff800700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25f810dac0 con 0x7f25f8103d70 2024-03-21T10:19:48.544 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-21T10:19:48.541+0000 7f25ff800700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25f810ddc0 con 0x7f25f81021d0 2024-03-21T10:19:48.545 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-21T10:19:48.541+0000 7f25fd800700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f25f8103d70 0x7f25f81a1a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:19:48.545 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-21T10:19:48.541+0000 7f25fd800700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f25f8103d70 0x7f25f81a1a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:37344/0 (socket says 172.21.15.100:37344) 2024-03-21T10:19:48.545 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-21T10:19:48.541+0000 7f25fd800700 1 -- 172.21.15.100:0/741660907 learned_addr learned my addr 172.21.15.100:0/741660907 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-21T10:19:48.545 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-21T10:19:48.542+0000 7f25f7e00700 1 --2- 172.21.15.100:0/741660907 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f25f81033d0 0x7f25f819a9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:19:48.545 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-21T10:19:48.542+0000 7f25fce00700 1 --2- 172.21.15.100:0/741660907 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f25f81021d0 0x7f25f819a470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:19:48.545 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-21T10:19:48.542+0000 7f25fd800700 1 -- 172.21.15.100:0/741660907 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f25f81021d0 msgr2=0x7f25f819a470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:19:48.545 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-21T10:19:48.542+0000 7f25fd800700 1 --2- 172.21.15.100:0/741660907 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f25f81021d0 0x7f25f819a470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:19:48.545 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-21T10:19:48.542+0000 7f25fd800700 1 -- 172.21.15.100:0/741660907 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f25f81033d0 msgr2=0x7f25f819a9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:19:48.545 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-21T10:19:48.542+0000 7f25fd800700 1 --2- 172.21.15.100:0/741660907 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f25f81033d0 0x7f25f819a9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:19:48.545 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-21T10:19:48.542+0000 7f25fd800700 1 -- 172.21.15.100:0/741660907 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25f81a20d0 con 0x7f25f8103d70 2024-03-21T10:19:48.545 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-21T10:19:48.542+0000 7f25fd800700 1 --2- 172.21.15.100:0/741660907 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f25f8103d70 0x7f25f81a1a30 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f25f000ada0 tx=0x7f25f000c330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:19:48.545 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-21T10:19:48.543+0000 7f25f5600700 1 -- 172.21.15.100:0/741660907 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25f0017020 con 0x7f25f8103d70 2024-03-21T10:19:48.546 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-21T10:19:48.543+0000 7f25f5600700 1 -- 172.21.15.100:0/741660907 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f25f000cd80 con 0x7f25f8103d70 2024-03-21T10:19:48.546 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-21T10:19:48.543+0000 7f25f5600700 1 -- 172.21.15.100:0/741660907 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25f0005240 con 0x7f25f8103d70 2024-03-21T10:19:48.546 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-21T10:19:48.543+0000 7f25ff800700 1 -- 172.21.15.100:0/741660907 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f25f81a23c0 con 0x7f25f8103d70 2024-03-21T10:19:48.546 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-21T10:19:48.543+0000 7f25ff800700 1 -- 172.21.15.100:0/741660907 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f25f81a28b0 con 0x7f25f8103d70 2024-03-21T10:19:48.546 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-21T10:19:48.544+0000 7f25ff800700 1 -- 172.21.15.100:0/741660907 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f25f8068e50 con 0x7f25f8103d70 2024-03-21T10:19:48.549 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-21T10:19:48.548+0000 7f25f5600700 1 -- 172.21.15.100:0/741660907 <== mon.2 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 16) v1 ==== 137854+0+0 (secure 0 0 0) 0x7f25f000c8d0 con 0x7f25f8103d70 2024-03-21T10:19:48.550 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-21T10:19:48.549+0000 7f25f5600700 1 --2- 172.21.15.100:0/741660907 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f25e00a3030 0x7f25e00a54f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:19:48.550 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-21T10:19:48.549+0000 7f25f5600700 1 -- 172.21.15.100:0/741660907 <== mon.2 v2:172.21.15.100:3300/0 5 ==== osd_map(70..70 src has 1..70) v4 ==== 6261+0+0 (secure 0 0 0) 0x7f25f00ce830 con 0x7f25f8103d70 2024-03-21T10:19:48.551 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-21T10:19:48.550+0000 7f25fce00700 1 --2- 172.21.15.100:0/741660907 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f25e00a3030 0x7f25e00a54f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:19:48.551 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-21T10:19:48.551+0000 7f25fce00700 1 --2- 172.21.15.100:0/741660907 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f25e00a3030 0x7f25e00a54f0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f25f8103230 tx=0x7f25ec00a3b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:19:48.554 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-21T10:19:48.553+0000 7f25f5600700 1 -- 172.21.15.100:0/741660907 <== mon.2 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f25f0021020 con 0x7f25f8103d70 2024-03-21T10:19:48.787 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-21T10:19:48.786+0000 7f25ff800700 1 -- 172.21.15.100:0/741660907 --> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi100:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) v1 -- 0x7f25f8075ae0 con 0x7f25e00a3030 2024-03-21T10:19:49.980 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:19:49 smithi100 ceph-mon[35193]: pgmap v281: 1 pgs: 1 active+clean; 577 KiB data, 313 MiB used, 894 GiB / 894 GiB avail 2024-03-21T10:19:49.980 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:19:49 smithi100 ceph-mon[35193]: from='client.34337 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi100:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:19:49.980 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:19:49 smithi100 ceph-mon[35193]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-21T10:19:49.980 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:19:49 smithi100 ceph-mon[35193]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-21T10:19:49.980 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:19:49 smithi100 ceph-mon[35193]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T10:19:50.214 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:19:49 smithi047 ceph-mon[31547]: pgmap v281: 1 pgs: 1 active+clean; 577 KiB data, 313 MiB used, 894 GiB / 894 GiB avail 2024-03-21T10:19:50.214 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:19:49 smithi047 ceph-mon[31547]: from='client.34337 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi100:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:19:50.214 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:19:49 smithi047 ceph-mon[31547]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-21T10:19:50.214 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:19:49 smithi047 ceph-mon[31547]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-21T10:19:50.214 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:19:49 smithi047 ceph-mon[31547]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T10:19:50.240 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:19:49 smithi157 ceph-mon[35581]: pgmap v281: 1 pgs: 1 active+clean; 577 KiB data, 313 MiB used, 894 GiB / 894 GiB avail 2024-03-21T10:19:50.240 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:19:49 smithi157 ceph-mon[35581]: from='client.34337 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi100:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:19:50.240 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:19:49 smithi157 ceph-mon[35581]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-21T10:19:50.240 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:19:49 smithi157 ceph-mon[35581]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-21T10:19:50.240 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:19:49 smithi157 ceph-mon[35581]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T10:19:51.979 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:19:51 smithi100 ceph-mon[35193]: pgmap v282: 1 pgs: 1 active+clean; 577 KiB data, 313 MiB used, 894 GiB / 894 GiB avail 2024-03-21T10:19:52.214 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:19:51 smithi047 ceph-mon[31547]: pgmap v282: 1 pgs: 1 active+clean; 577 KiB data, 313 MiB used, 894 GiB / 894 GiB avail 2024-03-21T10:19:52.239 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:19:51 smithi157 ceph-mon[35581]: pgmap v282: 1 pgs: 1 active+clean; 577 KiB data, 313 MiB used, 894 GiB / 894 GiB avail 2024-03-21T10:19:52.921 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:19:52 smithi047 ceph-mon[31547]: from='client.? 172.21.15.100:0/1067960648' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2da3da90-a0be-4a54-a9ff-b2d6c9b36870"}]: dispatch 2024-03-21T10:19:52.921 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:19:52 smithi047 ceph-mon[31547]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2da3da90-a0be-4a54-a9ff-b2d6c9b36870"}]: dispatch 2024-03-21T10:19:52.921 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:19:52 smithi047 ceph-mon[31547]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "2da3da90-a0be-4a54-a9ff-b2d6c9b36870"}]': finished 2024-03-21T10:19:52.921 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:19:52 smithi047 ceph-mon[31547]: osdmap e71: 11 total, 10 up, 11 in 2024-03-21T10:19:52.921 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:19:52 smithi047 ceph-mon[31547]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-21T10:19:53.229 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:19:52 smithi100 ceph-mon[35193]: from='client.? 172.21.15.100:0/1067960648' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2da3da90-a0be-4a54-a9ff-b2d6c9b36870"}]: dispatch 2024-03-21T10:19:53.229 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:19:52 smithi100 ceph-mon[35193]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2da3da90-a0be-4a54-a9ff-b2d6c9b36870"}]: dispatch 2024-03-21T10:19:53.230 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:19:52 smithi100 ceph-mon[35193]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "2da3da90-a0be-4a54-a9ff-b2d6c9b36870"}]': finished 2024-03-21T10:19:53.230 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:19:52 smithi100 ceph-mon[35193]: osdmap e71: 11 total, 10 up, 11 in 2024-03-21T10:19:53.230 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:19:52 smithi100 ceph-mon[35193]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-21T10:19:53.240 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:19:52 smithi157 ceph-mon[35581]: from='client.? 172.21.15.100:0/1067960648' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2da3da90-a0be-4a54-a9ff-b2d6c9b36870"}]: dispatch 2024-03-21T10:19:53.240 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:19:52 smithi157 ceph-mon[35581]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2da3da90-a0be-4a54-a9ff-b2d6c9b36870"}]: dispatch 2024-03-21T10:19:53.240 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:19:52 smithi157 ceph-mon[35581]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "2da3da90-a0be-4a54-a9ff-b2d6c9b36870"}]': finished 2024-03-21T10:19:53.240 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:19:52 smithi157 ceph-mon[35581]: osdmap e71: 11 total, 10 up, 11 in 2024-03-21T10:19:53.240 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:19:52 smithi157 ceph-mon[35581]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-21T10:19:54.214 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:19:53 smithi047 ceph-mon[31547]: pgmap v283: 1 pgs: 1 active+clean; 577 KiB data, 313 MiB used, 894 GiB / 894 GiB avail 2024-03-21T10:19:54.214 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:19:53 smithi047 ceph-mon[31547]: from='client.? 172.21.15.100:0/2088629667' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-21T10:19:54.230 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:19:53 smithi100 ceph-mon[35193]: pgmap v283: 1 pgs: 1 active+clean; 577 KiB data, 313 MiB used, 894 GiB / 894 GiB avail 2024-03-21T10:19:54.230 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:19:53 smithi100 ceph-mon[35193]: from='client.? 172.21.15.100:0/2088629667' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-21T10:19:54.240 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:19:53 smithi157 ceph-mon[35581]: pgmap v283: 1 pgs: 1 active+clean; 577 KiB data, 313 MiB used, 894 GiB / 894 GiB avail 2024-03-21T10:19:54.240 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:19:53 smithi157 ceph-mon[35581]: from='client.? 172.21.15.100:0/2088629667' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-21T10:19:56.214 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:19:55 smithi047 ceph-mon[31547]: pgmap v285: 1 pgs: 1 active+clean; 577 KiB data, 313 MiB used, 894 GiB / 894 GiB avail 2024-03-21T10:19:56.229 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:19:55 smithi100 ceph-mon[35193]: pgmap v285: 1 pgs: 1 active+clean; 577 KiB data, 313 MiB used, 894 GiB / 894 GiB avail 2024-03-21T10:19:56.239 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:19:55 smithi157 ceph-mon[35581]: pgmap v285: 1 pgs: 1 active+clean; 577 KiB data, 313 MiB used, 894 GiB / 894 GiB avail 2024-03-21T10:19:58.214 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:19:57 smithi047 ceph-mon[31547]: pgmap v286: 1 pgs: 1 active+clean; 577 KiB data, 313 MiB used, 894 GiB / 894 GiB avail 2024-03-21T10:19:58.229 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:19:57 smithi100 ceph-mon[35193]: pgmap v286: 1 pgs: 1 active+clean; 577 KiB data, 313 MiB used, 894 GiB / 894 GiB avail 2024-03-21T10:19:58.239 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:19:57 smithi157 ceph-mon[35581]: pgmap v286: 1 pgs: 1 active+clean; 577 KiB data, 313 MiB used, 894 GiB / 894 GiB avail 2024-03-21T10:20:00.117 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:19:59 smithi100 ceph-mon[35193]: pgmap v287: 1 pgs: 1 active+clean; 577 KiB data, 313 MiB used, 894 GiB / 894 GiB avail 2024-03-21T10:20:00.214 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:19:59 smithi047 ceph-mon[31547]: pgmap v287: 1 pgs: 1 active+clean; 577 KiB data, 313 MiB used, 894 GiB / 894 GiB avail 2024-03-21T10:20:00.239 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:19:59 smithi157 ceph-mon[35581]: pgmap v287: 1 pgs: 1 active+clean; 577 KiB data, 313 MiB used, 894 GiB / 894 GiB avail 2024-03-21T10:20:01.214 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:20:00 smithi047 ceph-mon[31547]: overall HEALTH_OK 2024-03-21T10:20:01.230 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:20:00 smithi100 ceph-mon[35193]: overall HEALTH_OK 2024-03-21T10:20:01.240 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:20:00 smithi157 ceph-mon[35581]: overall HEALTH_OK 2024-03-21T10:20:01.980 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:20:01 smithi100 ceph-mon[35193]: pgmap v288: 1 pgs: 1 active+clean; 577 KiB data, 313 MiB used, 894 GiB / 894 GiB avail 2024-03-21T10:20:02.214 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:20:01 smithi047 ceph-mon[31547]: pgmap v288: 1 pgs: 1 active+clean; 577 KiB data, 313 MiB used, 894 GiB / 894 GiB avail 2024-03-21T10:20:02.240 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:20:01 smithi157 ceph-mon[35581]: pgmap v288: 1 pgs: 1 active+clean; 577 KiB data, 313 MiB used, 894 GiB / 894 GiB avail 2024-03-21T10:20:03.214 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:20:02 smithi047 ceph-mon[31547]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-03-21T10:20:03.214 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:20:02 smithi047 ceph-mon[31547]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T10:20:03.220 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:20:02 smithi100 ceph-mon[35193]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-03-21T10:20:03.220 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:20:02 smithi100 ceph-mon[35193]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T10:20:03.239 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:20:02 smithi157 ceph-mon[35581]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-03-21T10:20:03.239 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:20:02 smithi157 ceph-mon[35581]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T10:20:04.214 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:20:03 smithi047 ceph-mon[31547]: pgmap v289: 1 pgs: 1 active+clean; 577 KiB data, 313 MiB used, 894 GiB / 894 GiB avail 2024-03-21T10:20:04.214 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:20:03 smithi047 ceph-mon[31547]: Deploying daemon osd.10 on smithi100 2024-03-21T10:20:04.230 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:20:03 smithi100 ceph-mon[35193]: pgmap v289: 1 pgs: 1 active+clean; 577 KiB data, 313 MiB used, 894 GiB / 894 GiB avail 2024-03-21T10:20:04.230 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:20:03 smithi100 ceph-mon[35193]: Deploying daemon osd.10 on smithi100 2024-03-21T10:20:04.239 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:20:03 smithi157 ceph-mon[35581]: pgmap v289: 1 pgs: 1 active+clean; 577 KiB data, 313 MiB used, 894 GiB / 894 GiB avail 2024-03-21T10:20:04.239 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:20:03 smithi157 ceph-mon[35581]: Deploying daemon osd.10 on smithi100 2024-03-21T10:20:05.127 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:20:04 smithi100 ceph-mon[35193]: pgmap v290: 1 pgs: 1 active+clean; 577 KiB data, 313 MiB used, 894 GiB / 894 GiB avail 2024-03-21T10:20:05.214 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:20:04 smithi047 ceph-mon[31547]: pgmap v290: 1 pgs: 1 active+clean; 577 KiB data, 313 MiB used, 894 GiB / 894 GiB avail 2024-03-21T10:20:05.239 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:20:04 smithi157 ceph-mon[35581]: pgmap v290: 1 pgs: 1 active+clean; 577 KiB data, 313 MiB used, 894 GiB / 894 GiB avail 2024-03-21T10:20:07.372 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:20:07 smithi100 ceph-mon[35193]: pgmap v291: 1 pgs: 1 active+clean; 577 KiB data, 313 MiB used, 894 GiB / 894 GiB avail 2024-03-21T10:20:07.464 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:20:07 smithi047 ceph-mon[31547]: pgmap v291: 1 pgs: 1 active+clean; 577 KiB data, 313 MiB used, 894 GiB / 894 GiB avail 2024-03-21T10:20:07.490 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:20:07 smithi157 ceph-mon[35581]: pgmap v291: 1 pgs: 1 active+clean; 577 KiB data, 313 MiB used, 894 GiB / 894 GiB avail 2024-03-21T10:20:09.230 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:20:09 smithi100 ceph-mon[35193]: pgmap v292: 1 pgs: 1 active+clean; 577 KiB data, 313 MiB used, 894 GiB / 894 GiB avail 2024-03-21T10:20:09.293 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:20:09 smithi157 ceph-mon[35581]: pgmap v292: 1 pgs: 1 active+clean; 577 KiB data, 313 MiB used, 894 GiB / 894 GiB avail 2024-03-21T10:20:09.464 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:20:09 smithi047 ceph-mon[31547]: pgmap v292: 1 pgs: 1 active+clean; 577 KiB data, 313 MiB used, 894 GiB / 894 GiB avail 2024-03-21T10:20:11.979 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:20:11 smithi100 ceph-mon[35193]: pgmap v293: 1 pgs: 1 active+clean; 577 KiB data, 313 MiB used, 894 GiB / 894 GiB avail 2024-03-21T10:20:11.979 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:20:11 smithi100 ceph-mon[35193]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:20:11.980 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:20:11 smithi100 ceph-mon[35193]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T10:20:11.980 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:20:11 smithi100 ceph-mon[35193]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:20:12.214 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:20:11 smithi047 ceph-mon[31547]: pgmap v293: 1 pgs: 1 active+clean; 577 KiB data, 313 MiB used, 894 GiB / 894 GiB avail 2024-03-21T10:20:12.214 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:20:11 smithi047 ceph-mon[31547]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:20:12.214 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:20:11 smithi047 ceph-mon[31547]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T10:20:12.214 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:20:11 smithi047 ceph-mon[31547]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:20:12.240 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:20:11 smithi157 ceph-mon[35581]: pgmap v293: 1 pgs: 1 active+clean; 577 KiB data, 313 MiB used, 894 GiB / 894 GiB avail 2024-03-21T10:20:12.240 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:20:11 smithi157 ceph-mon[35581]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:20:12.240 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:20:11 smithi157 ceph-mon[35581]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T10:20:12.240 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:20:11 smithi157 ceph-mon[35581]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:20:13.888 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:20:13 smithi100 ceph-mon[35193]: pgmap v294: 1 pgs: 1 active+clean; 577 KiB data, 313 MiB used, 894 GiB / 894 GiB avail 2024-03-21T10:20:14.214 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:20:13 smithi047 ceph-mon[31547]: pgmap v294: 1 pgs: 1 active+clean; 577 KiB data, 313 MiB used, 894 GiB / 894 GiB avail 2024-03-21T10:20:14.240 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:20:13 smithi157 ceph-mon[35581]: pgmap v294: 1 pgs: 1 active+clean; 577 KiB data, 313 MiB used, 894 GiB / 894 GiB avail 2024-03-21T10:20:15.730 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:20:15 smithi100 ceph-mon[35193]: pgmap v295: 1 pgs: 1 active+clean; 577 KiB data, 313 MiB used, 894 GiB / 894 GiB avail 2024-03-21T10:20:15.730 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:20:15 smithi100 ceph-mon[35193]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:20:15.730 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:20:15 smithi100 ceph-mon[35193]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:20:15.730 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:20:15 smithi100 ceph-mon[35193]: from='osd.10 [v2:172.21.15.100:6824/396391028,v1:172.21.15.100:6825/396391028]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-03-21T10:20:15.730 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:20:15 smithi100 ceph-mon[35193]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-03-21T10:20:15.851 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-21T10:20:15.850+0000 7f25f5600700 1 -- 172.21.15.100:0/741660907 <== mgr.14152 v2:172.21.15.47:6800/2823885994 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+38 (secure 0 0 0) 0x7f25f8075ae0 con 0x7f25e00a3030 2024-03-21T10:20:15.852 INFO:teuthology.orchestra.run.smithi100.stdout:Created osd(s) 10 on host 'smithi100' 2024-03-21T10:20:15.854 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-21T10:20:15.853+0000 7f25ff800700 1 -- 172.21.15.100:0/741660907 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f25e00a3030 msgr2=0x7f25e00a54f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:20:15.854 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-21T10:20:15.853+0000 7f25ff800700 1 --2- 172.21.15.100:0/741660907 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f25e00a3030 0x7f25e00a54f0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f25f8103230 tx=0x7f25ec00a3b0 comp rx=0 tx=0).stop 2024-03-21T10:20:15.854 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-21T10:20:15.853+0000 7f25ff800700 1 -- 172.21.15.100:0/741660907 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f25f8103d70 msgr2=0x7f25f81a1a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:20:15.854 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-21T10:20:15.853+0000 7f25ff800700 1 --2- 172.21.15.100:0/741660907 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f25f8103d70 0x7f25f81a1a30 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f25f000ada0 tx=0x7f25f000c330 comp rx=0 tx=0).stop 2024-03-21T10:20:15.855 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-21T10:20:15.853+0000 7f25ff800700 1 -- 172.21.15.100:0/741660907 shutdown_connections 2024-03-21T10:20:15.855 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-21T10:20:15.853+0000 7f25ff800700 1 --2- 172.21.15.100:0/741660907 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f25e00a3030 0x7f25e00a54f0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:20:15.855 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-21T10:20:15.853+0000 7f25ff800700 1 --2- 172.21.15.100:0/741660907 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f25f8103d70 0x7f25f81a1a30 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:20:15.855 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-21T10:20:15.853+0000 7f25ff800700 1 --2- 172.21.15.100:0/741660907 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f25f81021d0 0x7f25f819a470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:20:15.855 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-21T10:20:15.853+0000 7f25ff800700 1 --2- 172.21.15.100:0/741660907 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f25f81033d0 0x7f25f819a9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:20:15.855 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-21T10:20:15.853+0000 7f25ff800700 1 -- 172.21.15.100:0/741660907 >> 172.21.15.100:0/741660907 conn(0x7f25f80fd730 msgr2=0x7f25f80743c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:20:15.855 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-21T10:20:15.853+0000 7f25ff800700 1 -- 172.21.15.100:0/741660907 shutdown_connections 2024-03-21T10:20:15.855 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-21T10:20:15.853+0000 7f25ff800700 1 -- 172.21.15.100:0/741660907 wait complete. 2024-03-21T10:20:15.964 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:20:15 smithi047 ceph-mon[31547]: pgmap v295: 1 pgs: 1 active+clean; 577 KiB data, 313 MiB used, 894 GiB / 894 GiB avail 2024-03-21T10:20:15.964 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:20:15 smithi047 ceph-mon[31547]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:20:15.964 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:20:15 smithi047 ceph-mon[31547]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:20:15.964 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:20:15 smithi047 ceph-mon[31547]: from='osd.10 [v2:172.21.15.100:6824/396391028,v1:172.21.15.100:6825/396391028]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-03-21T10:20:15.965 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:20:15 smithi047 ceph-mon[31547]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-03-21T10:20:15.990 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:20:15 smithi157 ceph-mon[35581]: pgmap v295: 1 pgs: 1 active+clean; 577 KiB data, 313 MiB used, 894 GiB / 894 GiB avail 2024-03-21T10:20:15.990 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:20:15 smithi157 ceph-mon[35581]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:20:15.990 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:20:15 smithi157 ceph-mon[35581]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:20:15.990 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:20:15 smithi157 ceph-mon[35581]: from='osd.10 [v2:172.21.15.100:6824/396391028,v1:172.21.15.100:6825/396391028]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-03-21T10:20:15.990 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:20:15 smithi157 ceph-mon[35581]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-03-21T10:20:16.680 DEBUG:teuthology.orchestra.run.smithi100:osd.10> sudo journalctl -f -n 0 -u ceph-3056d51a-e76b-11ee-95cd-87774f69a715@osd.10.service 2024-03-21T10:20:16.688 INFO:tasks.cephadm:Deploying osd.11 on smithi157 with /dev/vg_nvme/lv_1... 2024-03-21T10:20:16.688 DEBUG:teuthology.orchestra.run.smithi157:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2edfee68ad7243bed422a049b6f5f58619300f11 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3056d51a-e76b-11ee-95cd-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-03-21T10:20:16.777 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:20:16 smithi157 ceph-mon[35581]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-03-21T10:20:16.777 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:20:16 smithi157 ceph-mon[35581]: osdmap e72: 11 total, 10 up, 11 in 2024-03-21T10:20:16.777 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:20:16 smithi157 ceph-mon[35581]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-21T10:20:16.777 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:20:16 smithi157 ceph-mon[35581]: from='osd.10 [v2:172.21.15.100:6824/396391028,v1:172.21.15.100:6825/396391028]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi100", "root=default"]}]: dispatch 2024-03-21T10:20:16.777 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:20:16 smithi157 ceph-mon[35581]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi100", "root=default"]}]: dispatch 2024-03-21T10:20:16.777 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:20:16 smithi157 ceph-mon[35581]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:20:16.777 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:20:16 smithi157 ceph-mon[35581]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:20:16.964 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:20:16 smithi047 ceph-mon[31547]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-03-21T10:20:16.964 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:20:16 smithi047 ceph-mon[31547]: osdmap e72: 11 total, 10 up, 11 in 2024-03-21T10:20:16.964 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:20:16 smithi047 ceph-mon[31547]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-21T10:20:16.964 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:20:16 smithi047 ceph-mon[31547]: from='osd.10 [v2:172.21.15.100:6824/396391028,v1:172.21.15.100:6825/396391028]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi100", "root=default"]}]: dispatch 2024-03-21T10:20:16.964 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:20:16 smithi047 ceph-mon[31547]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi100", "root=default"]}]: dispatch 2024-03-21T10:20:16.964 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:20:16 smithi047 ceph-mon[31547]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:20:16.965 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:20:16 smithi047 ceph-mon[31547]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:20:16.980 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:20:16 smithi100 ceph-mon[35193]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-03-21T10:20:16.980 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:20:16 smithi100 ceph-mon[35193]: osdmap e72: 11 total, 10 up, 11 in 2024-03-21T10:20:16.980 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:20:16 smithi100 ceph-mon[35193]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-21T10:20:16.980 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:20:16 smithi100 ceph-mon[35193]: from='osd.10 [v2:172.21.15.100:6824/396391028,v1:172.21.15.100:6825/396391028]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi100", "root=default"]}]: dispatch 2024-03-21T10:20:16.980 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:20:16 smithi100 ceph-mon[35193]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi100", "root=default"]}]: dispatch 2024-03-21T10:20:16.980 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:20:16 smithi100 ceph-mon[35193]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:20:16.981 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:20:16 smithi100 ceph-mon[35193]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:20:17.142 INFO:teuthology.orchestra.run.smithi157.stderr:Inferring config /var/lib/ceph/3056d51a-e76b-11ee-95cd-87774f69a715/mon.c/config 2024-03-21T10:20:17.604 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:20:17 smithi157 ceph-mon[35581]: pgmap v297: 1 pgs: 1 active+clean; 577 KiB data, 313 MiB used, 894 GiB / 894 GiB avail 2024-03-21T10:20:17.604 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:20:17 smithi157 ceph-mon[35581]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi100", "root=default"]}]': finished 2024-03-21T10:20:17.604 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:20:17 smithi157 ceph-mon[35581]: osdmap e73: 11 total, 10 up, 11 in 2024-03-21T10:20:17.604 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:20:17 smithi157 ceph-mon[35581]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-21T10:20:17.604 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:20:17 smithi157 ceph-mon[35581]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-21T10:20:17.604 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:20:17 smithi157 ceph-mon[35581]: from='osd.10 ' entity='osd.10' 2024-03-21T10:20:17.729 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:20:17 smithi100 ceph-mon[35193]: pgmap v297: 1 pgs: 1 active+clean; 577 KiB data, 313 MiB used, 894 GiB / 894 GiB avail 2024-03-21T10:20:17.730 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:20:17 smithi100 ceph-mon[35193]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi100", "root=default"]}]': finished 2024-03-21T10:20:17.730 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:20:17 smithi100 ceph-mon[35193]: osdmap e73: 11 total, 10 up, 11 in 2024-03-21T10:20:17.730 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:20:17 smithi100 ceph-mon[35193]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-21T10:20:17.730 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:20:17 smithi100 ceph-mon[35193]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-21T10:20:17.730 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:20:17 smithi100 ceph-mon[35193]: from='osd.10 ' entity='osd.10' 2024-03-21T10:20:17.730 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 10:20:17 smithi100 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-10[59767]: 2024-03-21T10:20:17.408+0000 7f3165e00700 -1 osd.10 0 waiting for initial osdmap 2024-03-21T10:20:17.730 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 10:20:17 smithi100 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-10[59767]: 2024-03-21T10:20:17.443+0000 7f315bc00700 -1 osd.10 73 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-21T10:20:17.964 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:20:17 smithi047 ceph-mon[31547]: pgmap v297: 1 pgs: 1 active+clean; 577 KiB data, 313 MiB used, 894 GiB / 894 GiB avail 2024-03-21T10:20:17.964 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:20:17 smithi047 ceph-mon[31547]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi100", "root=default"]}]': finished 2024-03-21T10:20:17.964 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:20:17 smithi047 ceph-mon[31547]: osdmap e73: 11 total, 10 up, 11 in 2024-03-21T10:20:17.964 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:20:17 smithi047 ceph-mon[31547]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-21T10:20:17.964 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:20:17 smithi047 ceph-mon[31547]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-21T10:20:17.964 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:20:17 smithi047 ceph-mon[31547]: from='osd.10 ' entity='osd.10' 2024-03-21T10:20:18.730 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:20:18 smithi100 ceph-mon[35193]: purged_snaps scrub starts 2024-03-21T10:20:18.730 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:20:18 smithi100 ceph-mon[35193]: purged_snaps scrub ok 2024-03-21T10:20:18.730 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:20:18 smithi100 ceph-mon[35193]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-21T10:20:18.730 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:20:18 smithi100 ceph-mon[35193]: osd.10 [v2:172.21.15.100:6824/396391028,v1:172.21.15.100:6825/396391028] boot 2024-03-21T10:20:18.730 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:20:18 smithi100 ceph-mon[35193]: osdmap e74: 11 total, 11 up, 11 in 2024-03-21T10:20:18.730 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:20:18 smithi100 ceph-mon[35193]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-21T10:20:18.731 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:20:18 smithi100 ceph-mon[35193]: osdmap e75: 11 total, 11 up, 11 in 2024-03-21T10:20:18.845 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:20:18 smithi157 ceph-mon[35581]: purged_snaps scrub starts 2024-03-21T10:20:18.846 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:20:18 smithi157 ceph-mon[35581]: purged_snaps scrub ok 2024-03-21T10:20:18.846 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:20:18 smithi157 ceph-mon[35581]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-21T10:20:18.846 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:20:18 smithi157 ceph-mon[35581]: osd.10 [v2:172.21.15.100:6824/396391028,v1:172.21.15.100:6825/396391028] boot 2024-03-21T10:20:18.846 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:20:18 smithi157 ceph-mon[35581]: osdmap e74: 11 total, 11 up, 11 in 2024-03-21T10:20:18.846 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:20:18 smithi157 ceph-mon[35581]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-21T10:20:18.846 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:20:18 smithi157 ceph-mon[35581]: osdmap e75: 11 total, 11 up, 11 in 2024-03-21T10:20:18.964 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:20:18 smithi047 ceph-mon[31547]: purged_snaps scrub starts 2024-03-21T10:20:18.964 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:20:18 smithi047 ceph-mon[31547]: purged_snaps scrub ok 2024-03-21T10:20:18.964 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:20:18 smithi047 ceph-mon[31547]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-21T10:20:18.964 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:20:18 smithi047 ceph-mon[31547]: osd.10 [v2:172.21.15.100:6824/396391028,v1:172.21.15.100:6825/396391028] boot 2024-03-21T10:20:18.964 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:20:18 smithi047 ceph-mon[31547]: osdmap e74: 11 total, 11 up, 11 in 2024-03-21T10:20:18.964 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:20:18 smithi047 ceph-mon[31547]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-21T10:20:18.964 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:20:18 smithi047 ceph-mon[31547]: osdmap e75: 11 total, 11 up, 11 in 2024-03-21T10:20:19.964 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:20:19 smithi047 ceph-mon[31547]: pgmap v300: 1 pgs: 1 remapped+peering; 577 KiB data, 313 MiB used, 894 GiB / 894 GiB avail 2024-03-21T10:20:19.964 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:20:19 smithi047 ceph-mon[31547]: osdmap e76: 11 total, 11 up, 11 in 2024-03-21T10:20:19.979 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:20:19 smithi100 ceph-mon[35193]: pgmap v300: 1 pgs: 1 remapped+peering; 577 KiB data, 313 MiB used, 894 GiB / 894 GiB avail 2024-03-21T10:20:19.979 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:20:19 smithi100 ceph-mon[35193]: osdmap e76: 11 total, 11 up, 11 in 2024-03-21T10:20:19.990 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:20:19 smithi157 ceph-mon[35581]: pgmap v300: 1 pgs: 1 remapped+peering; 577 KiB data, 313 MiB used, 894 GiB / 894 GiB avail 2024-03-21T10:20:19.990 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:20:19 smithi157 ceph-mon[35581]: osdmap e76: 11 total, 11 up, 11 in 2024-03-21T10:20:20.009 INFO:teuthology.orchestra.run.smithi157.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-03-21T10:20:20.009 INFO:teuthology.orchestra.run.smithi157.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-03-21T10:20:20.009 INFO:teuthology.orchestra.run.smithi157.stderr: stderr: 10+0 records in 2024-03-21T10:20:20.009 INFO:teuthology.orchestra.run.smithi157.stderr:10+0 records out 2024-03-21T10:20:20.009 INFO:teuthology.orchestra.run.smithi157.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0324947 s, 323 MB/s 2024-03-21T10:20:20.009 INFO:teuthology.orchestra.run.smithi157.stderr:--> Zapping successful for: 2024-03-21T10:20:20.929 DEBUG:teuthology.orchestra.run.smithi157:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2edfee68ad7243bed422a049b6f5f58619300f11 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3056d51a-e76b-11ee-95cd-87774f69a715 -- ceph orch daemon add osd smithi157:vg_nvme/lv_1 2024-03-21T10:20:21.368 INFO:teuthology.orchestra.run.smithi157.stderr:Inferring config /var/lib/ceph/3056d51a-e76b-11ee-95cd-87774f69a715/mon.c/config 2024-03-21T10:20:21.826 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:20:21 smithi157 ceph-mon[35581]: pgmap v303: 1 pgs: 1 remapped+peering; 577 KiB data, 345 MiB used, 983 GiB / 983 GiB avail 2024-03-21T10:20:21.826 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:20:21 smithi157 ceph-mon[35581]: osdmap e77: 11 total, 11 up, 11 in 2024-03-21T10:20:21.964 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:20:21 smithi047 ceph-mon[31547]: pgmap v303: 1 pgs: 1 remapped+peering; 577 KiB data, 345 MiB used, 983 GiB / 983 GiB avail 2024-03-21T10:20:21.964 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:20:21 smithi047 ceph-mon[31547]: osdmap e77: 11 total, 11 up, 11 in 2024-03-21T10:20:21.979 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:20:21 smithi100 ceph-mon[35193]: pgmap v303: 1 pgs: 1 remapped+peering; 577 KiB data, 345 MiB used, 983 GiB / 983 GiB avail 2024-03-21T10:20:21.980 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:20:21 smithi100 ceph-mon[35193]: osdmap e77: 11 total, 11 up, 11 in 2024-03-21T10:20:22.724 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:20:22 smithi157 ceph-mon[35581]: osdmap e78: 11 total, 11 up, 11 in 2024-03-21T10:20:22.724 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:20:22 smithi157 ceph-mon[35581]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:20:22.724 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:20:22 smithi157 ceph-mon[35581]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:20:22.724 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:20:22 smithi157 ceph-mon[35581]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-21T10:20:22.724 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:20:22 smithi157 ceph-mon[35581]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-21T10:20:22.724 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:20:22 smithi157 ceph-mon[35581]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-21T10:20:22.724 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:20:22 smithi157 ceph-mon[35581]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-03-21T10:20:22.724 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:20:22 smithi157 ceph-mon[35581]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:20:22.724 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:20:22 smithi157 ceph-mon[35581]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T10:20:22.725 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:20:22 smithi157 ceph-mon[35581]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T10:20:22.725 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:20:22 smithi157 ceph-mon[35581]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:20:22.866 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:20:22 smithi047 ceph-mon[31547]: osdmap e78: 11 total, 11 up, 11 in 2024-03-21T10:20:22.867 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:20:22 smithi047 ceph-mon[31547]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:20:22.867 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:20:22 smithi047 ceph-mon[31547]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:20:22.867 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:20:22 smithi047 ceph-mon[31547]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-21T10:20:22.867 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:20:22 smithi047 ceph-mon[31547]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-21T10:20:22.867 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:20:22 smithi047 ceph-mon[31547]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-21T10:20:22.867 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:20:22 smithi047 ceph-mon[31547]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-03-21T10:20:22.867 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:20:22 smithi047 ceph-mon[31547]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:20:22.867 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:20:22 smithi047 ceph-mon[31547]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T10:20:22.867 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:20:22 smithi047 ceph-mon[31547]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T10:20:22.867 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:20:22 smithi047 ceph-mon[31547]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:20:22.980 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:20:22 smithi100 ceph-mon[35193]: osdmap e78: 11 total, 11 up, 11 in 2024-03-21T10:20:22.980 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:20:22 smithi100 ceph-mon[35193]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:20:22.980 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:20:22 smithi100 ceph-mon[35193]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:20:22.980 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:20:22 smithi100 ceph-mon[35193]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-21T10:20:22.980 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:20:22 smithi100 ceph-mon[35193]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-21T10:20:22.980 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:20:22 smithi100 ceph-mon[35193]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-21T10:20:22.980 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:20:22 smithi100 ceph-mon[35193]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-03-21T10:20:22.980 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:20:22 smithi100 ceph-mon[35193]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:20:22.980 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:20:22 smithi100 ceph-mon[35193]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T10:20:22.980 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:20:22 smithi100 ceph-mon[35193]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T10:20:22.981 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:20:22 smithi100 ceph-mon[35193]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:20:23.567 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-21T10:20:23.565+0000 7f4f44600700 1 -- 172.21.15.157:0/1095506296 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4f3c105fb0 msgr2=0x7f4f3c10c860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:20:23.567 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-21T10:20:23.565+0000 7f4f44600700 1 --2- 172.21.15.157:0/1095506296 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4f3c105fb0 0x7f4f3c10c860 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f4f3400b0a0 tx=0x7f4f3400b3b0 comp rx=0 tx=0).stop 2024-03-21T10:20:23.568 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-21T10:20:23.565+0000 7f4f44600700 1 -- 172.21.15.157:0/1095506296 shutdown_connections 2024-03-21T10:20:23.568 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-21T10:20:23.565+0000 7f4f44600700 1 --2- 172.21.15.157:0/1095506296 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4f3c105fb0 0x7f4f3c10c860 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:20:23.568 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-21T10:20:23.566+0000 7f4f44600700 1 --2- 172.21.15.157:0/1095506296 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4f3c104410 0x7f4f3c104830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:20:23.568 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-21T10:20:23.566+0000 7f4f44600700 1 --2- 172.21.15.157:0/1095506296 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4f3c105610 0x7f4f3c105a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:20:23.568 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-21T10:20:23.566+0000 7f4f44600700 1 -- 172.21.15.157:0/1095506296 >> 172.21.15.157:0/1095506296 conn(0x7f4f3c0ff9b0 msgr2=0x7f4f3c101df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:20:23.568 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-21T10:20:23.566+0000 7f4f44600700 1 -- 172.21.15.157:0/1095506296 shutdown_connections 2024-03-21T10:20:23.569 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-21T10:20:23.566+0000 7f4f44600700 1 -- 172.21.15.157:0/1095506296 wait complete. 2024-03-21T10:20:23.572 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-21T10:20:23.567+0000 7f4f44600700 1 Processor -- start 2024-03-21T10:20:23.573 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-21T10:20:23.567+0000 7f4f44600700 1 -- start start 2024-03-21T10:20:23.573 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-21T10:20:23.567+0000 7f4f44600700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4f3c104410 0x7f4f3c19a4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:20:23.573 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-21T10:20:23.567+0000 7f4f44600700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4f3c105610 0x7f4f3c19aa20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:20:23.573 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-21T10:20:23.567+0000 7f4f44600700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4f3c105fb0 0x7f4f3c1a1aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:20:23.573 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-21T10:20:23.567+0000 7f4f44600700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f3c10ff00 con 0x7f4f3c105610 2024-03-21T10:20:23.573 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-21T10:20:23.567+0000 7f4f44600700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f3c10fd80 con 0x7f4f3c105fb0 2024-03-21T10:20:23.574 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-21T10:20:23.567+0000 7f4f44600700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f3c110080 con 0x7f4f3c104410 2024-03-21T10:20:23.574 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-21T10:20:23.568+0000 7f4f41c00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4f3c104410 0x7f4f3c19a4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:20:23.574 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-21T10:20:23.568+0000 7f4f41c00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4f3c104410 0x7f4f3c19a4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:38906/0 (socket says 172.21.15.157:38906) 2024-03-21T10:20:23.574 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-21T10:20:23.568+0000 7f4f41c00700 1 -- 172.21.15.157:0/2528217324 learned_addr learned my addr 172.21.15.157:0/2528217324 (peer_addr_for_me v2:172.21.15.157:0/0) 2024-03-21T10:20:23.574 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-21T10:20:23.568+0000 7f4f41200700 1 --2- 172.21.15.157:0/2528217324 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4f3c105610 0x7f4f3c19aa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:20:23.574 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-21T10:20:23.568+0000 7f4f42600700 1 --2- 172.21.15.157:0/2528217324 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4f3c105fb0 0x7f4f3c1a1aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:20:23.575 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-21T10:20:23.568+0000 7f4f41c00700 1 -- 172.21.15.157:0/2528217324 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4f3c105fb0 msgr2=0x7f4f3c1a1aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:20:23.575 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-21T10:20:23.568+0000 7f4f41c00700 1 --2- 172.21.15.157:0/2528217324 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4f3c105fb0 0x7f4f3c1a1aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:20:23.575 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-21T10:20:23.568+0000 7f4f41c00700 1 -- 172.21.15.157:0/2528217324 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4f3c105610 msgr2=0x7f4f3c19aa20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:20:23.575 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-21T10:20:23.568+0000 7f4f41c00700 1 --2- 172.21.15.157:0/2528217324 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4f3c105610 0x7f4f3c19aa20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:20:23.575 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-21T10:20:23.568+0000 7f4f41c00700 1 -- 172.21.15.157:0/2528217324 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4f3c1a2140 con 0x7f4f3c104410 2024-03-21T10:20:23.575 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-21T10:20:23.568+0000 7f4f41200700 1 --2- 172.21.15.157:0/2528217324 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4f3c105610 0x7f4f3c19aa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T10:20:23.575 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-21T10:20:23.568+0000 7f4f42600700 1 --2- 172.21.15.157:0/2528217324 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4f3c105fb0 0x7f4f3c1a1aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T10:20:23.576 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-21T10:20:23.568+0000 7f4f41c00700 1 --2- 172.21.15.157:0/2528217324 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4f3c104410 0x7f4f3c19a4e0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f4f3000d9b0 tx=0x7f4f3000dd70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:20:23.576 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-21T10:20:23.569+0000 7f4f3a000700 1 -- 172.21.15.157:0/2528217324 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f30017030 con 0x7f4f3c104410 2024-03-21T10:20:23.576 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-21T10:20:23.569+0000 7f4f3a000700 1 -- 172.21.15.157:0/2528217324 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f4f300044d0 con 0x7f4f3c104410 2024-03-21T10:20:23.576 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-21T10:20:23.569+0000 7f4f44600700 1 -- 172.21.15.157:0/2528217324 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4f3c1a2430 con 0x7f4f3c104410 2024-03-21T10:20:23.576 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-21T10:20:23.569+0000 7f4f3a000700 1 -- 172.21.15.157:0/2528217324 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f300039e0 con 0x7f4f3c104410 2024-03-21T10:20:23.576 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-21T10:20:23.569+0000 7f4f44600700 1 -- 172.21.15.157:0/2528217324 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4f3c1a29c0 con 0x7f4f3c104410 2024-03-21T10:20:23.577 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-21T10:20:23.570+0000 7f4f44600700 1 -- 172.21.15.157:0/2528217324 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4f3c068e50 con 0x7f4f3c104410 2024-03-21T10:20:23.578 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-21T10:20:23.572+0000 7f4f3a000700 1 -- 172.21.15.157:0/2528217324 <== mon.1 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 16) v1 ==== 137854+0+0 (secure 0 0 0) 0x7f4f3001a070 con 0x7f4f3c104410 2024-03-21T10:20:23.578 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-21T10:20:23.575+0000 7f4f3a000700 1 --2- 172.21.15.157:0/2528217324 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f4f240a3030 0x7f4f240a54f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:20:23.579 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-21T10:20:23.575+0000 7f4f3a000700 1 -- 172.21.15.157:0/2528217324 <== mon.1 v2:172.21.15.157:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 6694+0+0 (secure 0 0 0) 0x7f4f300ceb50 con 0x7f4f3c104410 2024-03-21T10:20:23.579 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-21T10:20:23.575+0000 7f4f41200700 1 --2- 172.21.15.157:0/2528217324 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f4f240a3030 0x7f4f240a54f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:20:23.579 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-21T10:20:23.576+0000 7f4f41200700 1 --2- 172.21.15.157:0/2528217324 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f4f240a3030 0x7f4f240a54f0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f4f28005fd0 tx=0x7f4f28005ee0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:20:23.579 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-21T10:20:23.578+0000 7f4f3a000700 1 -- 172.21.15.157:0/2528217324 <== mon.1 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f4f3009bfb0 con 0x7f4f3c104410 2024-03-21T10:20:23.740 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:20:23 smithi157 ceph-mon[35581]: pgmap v306: 1 pgs: 1 remapped+peering; 577 KiB data, 345 MiB used, 983 GiB / 983 GiB avail 2024-03-21T10:20:23.741 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:20:23 smithi157 ceph-mon[35581]: Detected new or changed devices on smithi100 2024-03-21T10:20:23.741 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:20:23 smithi157 ceph-mon[35581]: Adjusting osd_memory_target on smithi100 to 4272M 2024-03-21T10:20:23.741 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:20:23 smithi157 ceph-mon[35581]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T10:20:23.741 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:20:23 smithi157 ceph-mon[35581]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T10:20:23.741 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:20:23 smithi157 ceph-mon[35581]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T10:20:23.741 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:20:23 smithi157 ceph-mon[35581]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:20:23.816 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-21T10:20:23.814+0000 7f4f44600700 1 -- 172.21.15.157:0/2528217324 --> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi157:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) v1 -- 0x7f4f3c0625d0 con 0x7f4f240a3030 2024-03-21T10:20:23.964 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:20:23 smithi047 ceph-mon[31547]: pgmap v306: 1 pgs: 1 remapped+peering; 577 KiB data, 345 MiB used, 983 GiB / 983 GiB avail 2024-03-21T10:20:23.964 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:20:23 smithi047 ceph-mon[31547]: Detected new or changed devices on smithi100 2024-03-21T10:20:23.964 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:20:23 smithi047 ceph-mon[31547]: Adjusting osd_memory_target on smithi100 to 4272M 2024-03-21T10:20:23.964 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:20:23 smithi047 ceph-mon[31547]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T10:20:23.964 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:20:23 smithi047 ceph-mon[31547]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T10:20:23.965 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:20:23 smithi047 ceph-mon[31547]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T10:20:23.965 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:20:23 smithi047 ceph-mon[31547]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:20:23.979 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:20:23 smithi100 ceph-mon[35193]: pgmap v306: 1 pgs: 1 remapped+peering; 577 KiB data, 345 MiB used, 983 GiB / 983 GiB avail 2024-03-21T10:20:23.980 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:20:23 smithi100 ceph-mon[35193]: Detected new or changed devices on smithi100 2024-03-21T10:20:23.980 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:20:23 smithi100 ceph-mon[35193]: Adjusting osd_memory_target on smithi100 to 4272M 2024-03-21T10:20:23.980 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:20:23 smithi100 ceph-mon[35193]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T10:20:23.980 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:20:23 smithi100 ceph-mon[35193]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T10:20:23.980 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:20:23 smithi100 ceph-mon[35193]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T10:20:23.980 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:20:23 smithi100 ceph-mon[35193]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:20:24.917 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:20:24 smithi157 ceph-mon[35581]: from='client.24343 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi157:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:20:24.917 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:20:24 smithi157 ceph-mon[35581]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-21T10:20:24.917 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:20:24 smithi157 ceph-mon[35581]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-21T10:20:24.918 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:20:24 smithi157 ceph-mon[35581]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T10:20:24.964 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:20:24 smithi047 ceph-mon[31547]: from='client.24343 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi157:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:20:24.964 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:20:24 smithi047 ceph-mon[31547]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-21T10:20:24.964 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:20:24 smithi047 ceph-mon[31547]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-21T10:20:24.964 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:20:24 smithi047 ceph-mon[31547]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T10:20:24.979 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:20:24 smithi100 ceph-mon[35193]: from='client.24343 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi157:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:20:24.979 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:20:24 smithi100 ceph-mon[35193]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-21T10:20:24.980 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:20:24 smithi100 ceph-mon[35193]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-21T10:20:24.980 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:20:24 smithi100 ceph-mon[35193]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T10:20:25.787 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:20:25 smithi157 ceph-mon[35581]: pgmap v307: 1 pgs: 1 active+clean; 577 KiB data, 346 MiB used, 983 GiB / 983 GiB avail; 106 KiB/s, 0 objects/s recovering 2024-03-21T10:20:25.964 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:20:25 smithi047 ceph-mon[31547]: pgmap v307: 1 pgs: 1 active+clean; 577 KiB data, 346 MiB used, 983 GiB / 983 GiB avail; 106 KiB/s, 0 objects/s recovering 2024-03-21T10:20:25.979 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:20:25 smithi100 ceph-mon[35193]: pgmap v307: 1 pgs: 1 active+clean; 577 KiB data, 346 MiB used, 983 GiB / 983 GiB avail; 106 KiB/s, 0 objects/s recovering 2024-03-21T10:20:27.883 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:20:27 smithi100 ceph-mon[35193]: pgmap v308: 1 pgs: 1 active+clean; 577 KiB data, 346 MiB used, 983 GiB / 983 GiB avail; 89 KiB/s, 0 objects/s recovering 2024-03-21T10:20:27.964 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:20:27 smithi047 ceph-mon[31547]: pgmap v308: 1 pgs: 1 active+clean; 577 KiB data, 346 MiB used, 983 GiB / 983 GiB avail; 89 KiB/s, 0 objects/s recovering 2024-03-21T10:20:27.990 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:20:27 smithi157 ceph-mon[35581]: pgmap v308: 1 pgs: 1 active+clean; 577 KiB data, 346 MiB used, 983 GiB / 983 GiB avail; 89 KiB/s, 0 objects/s recovering 2024-03-21T10:20:28.964 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:20:28 smithi047 ceph-mon[31547]: from='client.? 172.21.15.157:0/892594336' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "11798d41-7f7e-4319-95b8-8d3d0946bde9"}]: dispatch 2024-03-21T10:20:28.964 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:20:28 smithi047 ceph-mon[31547]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "11798d41-7f7e-4319-95b8-8d3d0946bde9"}]: dispatch 2024-03-21T10:20:28.964 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:20:28 smithi047 ceph-mon[31547]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "11798d41-7f7e-4319-95b8-8d3d0946bde9"}]': finished 2024-03-21T10:20:28.964 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:20:28 smithi047 ceph-mon[31547]: osdmap e79: 12 total, 11 up, 12 in 2024-03-21T10:20:28.964 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:20:28 smithi047 ceph-mon[31547]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-21T10:20:28.964 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:20:28 smithi047 ceph-mon[31547]: from='client.? 172.21.15.157:0/125616015' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-21T10:20:28.979 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:20:28 smithi100 ceph-mon[35193]: from='client.? 172.21.15.157:0/892594336' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "11798d41-7f7e-4319-95b8-8d3d0946bde9"}]: dispatch 2024-03-21T10:20:28.980 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:20:28 smithi100 ceph-mon[35193]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "11798d41-7f7e-4319-95b8-8d3d0946bde9"}]: dispatch 2024-03-21T10:20:28.980 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:20:28 smithi100 ceph-mon[35193]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "11798d41-7f7e-4319-95b8-8d3d0946bde9"}]': finished 2024-03-21T10:20:28.980 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:20:28 smithi100 ceph-mon[35193]: osdmap e79: 12 total, 11 up, 12 in 2024-03-21T10:20:28.980 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:20:28 smithi100 ceph-mon[35193]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-21T10:20:28.980 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:20:28 smithi100 ceph-mon[35193]: from='client.? 172.21.15.157:0/125616015' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-21T10:20:28.989 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:20:28 smithi157 ceph-mon[35581]: from='client.? 172.21.15.157:0/892594336' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "11798d41-7f7e-4319-95b8-8d3d0946bde9"}]: dispatch 2024-03-21T10:20:28.989 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:20:28 smithi157 ceph-mon[35581]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "11798d41-7f7e-4319-95b8-8d3d0946bde9"}]: dispatch 2024-03-21T10:20:28.990 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:20:28 smithi157 ceph-mon[35581]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "11798d41-7f7e-4319-95b8-8d3d0946bde9"}]': finished 2024-03-21T10:20:28.990 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:20:28 smithi157 ceph-mon[35581]: osdmap e79: 12 total, 11 up, 12 in 2024-03-21T10:20:28.990 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:20:28 smithi157 ceph-mon[35581]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-21T10:20:28.990 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:20:28 smithi157 ceph-mon[35581]: from='client.? 172.21.15.157:0/125616015' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-21T10:20:29.964 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:20:29 smithi047 ceph-mon[31547]: pgmap v310: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 983 GiB / 983 GiB avail; 78 KiB/s, 0 objects/s recovering 2024-03-21T10:20:29.979 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:20:29 smithi100 ceph-mon[35193]: pgmap v310: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 983 GiB / 983 GiB avail; 78 KiB/s, 0 objects/s recovering 2024-03-21T10:20:29.989 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:20:29 smithi157 ceph-mon[35581]: pgmap v310: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 983 GiB / 983 GiB avail; 78 KiB/s, 0 objects/s recovering 2024-03-21T10:20:31.964 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:20:31 smithi047 ceph-mon[31547]: pgmap v311: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 983 GiB / 983 GiB avail; 69 KiB/s, 0 objects/s recovering 2024-03-21T10:20:31.979 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:20:31 smithi100 ceph-mon[35193]: pgmap v311: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 983 GiB / 983 GiB avail; 69 KiB/s, 0 objects/s recovering 2024-03-21T10:20:31.989 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:20:31 smithi157 ceph-mon[35581]: pgmap v311: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 983 GiB / 983 GiB avail; 69 KiB/s, 0 objects/s recovering 2024-03-21T10:20:33.945 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:20:33 smithi157 ceph-mon[35581]: pgmap v312: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 983 GiB / 983 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-03-21T10:20:33.964 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:20:33 smithi047 ceph-mon[31547]: pgmap v312: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 983 GiB / 983 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-03-21T10:20:33.979 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:20:33 smithi100 ceph-mon[35193]: pgmap v312: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 983 GiB / 983 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-03-21T10:20:35.964 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:20:35 smithi047 ceph-mon[31547]: pgmap v313: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 983 GiB / 983 GiB avail 2024-03-21T10:20:35.979 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:20:35 smithi100 ceph-mon[35193]: pgmap v313: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 983 GiB / 983 GiB avail 2024-03-21T10:20:35.990 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:20:35 smithi157 ceph-mon[35581]: pgmap v313: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 983 GiB / 983 GiB avail 2024-03-21T10:20:37.944 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:20:37 smithi157 ceph-mon[35581]: pgmap v314: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 983 GiB / 983 GiB avail 2024-03-21T10:20:37.945 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:20:37 smithi157 ceph-mon[35581]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-03-21T10:20:37.945 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:20:37 smithi157 ceph-mon[35581]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T10:20:37.964 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:20:37 smithi047 ceph-mon[31547]: pgmap v314: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 983 GiB / 983 GiB avail 2024-03-21T10:20:37.964 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:20:37 smithi047 ceph-mon[31547]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-03-21T10:20:37.964 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:20:37 smithi047 ceph-mon[31547]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T10:20:37.979 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:20:37 smithi100 ceph-mon[35193]: pgmap v314: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 983 GiB / 983 GiB avail 2024-03-21T10:20:37.979 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:20:37 smithi100 ceph-mon[35193]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-03-21T10:20:37.980 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:20:37 smithi100 ceph-mon[35193]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T10:20:38.955 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:20:38 smithi157 ceph-mon[35581]: Deploying daemon osd.11 on smithi157 2024-03-21T10:20:38.963 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:20:38 smithi047 ceph-mon[31547]: Deploying daemon osd.11 on smithi157 2024-03-21T10:20:38.979 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:20:38 smithi100 ceph-mon[35193]: Deploying daemon osd.11 on smithi157 2024-03-21T10:20:39.881 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:20:39 smithi157 ceph-mon[35581]: pgmap v315: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 983 GiB / 983 GiB avail 2024-03-21T10:20:39.964 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:20:39 smithi047 ceph-mon[31547]: pgmap v315: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 983 GiB / 983 GiB avail 2024-03-21T10:20:39.979 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:20:39 smithi100 ceph-mon[35193]: pgmap v315: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 983 GiB / 983 GiB avail 2024-03-21T10:20:41.964 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:20:41 smithi047 ceph-mon[31547]: pgmap v316: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 983 GiB / 983 GiB avail 2024-03-21T10:20:41.979 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:20:41 smithi100 ceph-mon[35193]: pgmap v316: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 983 GiB / 983 GiB avail 2024-03-21T10:20:41.989 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:20:41 smithi157 ceph-mon[35581]: pgmap v316: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 983 GiB / 983 GiB avail 2024-03-21T10:20:43.964 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:20:43 smithi047 ceph-mon[31547]: pgmap v317: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 983 GiB / 983 GiB avail 2024-03-21T10:20:43.975 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:20:43 smithi157 ceph-mon[35581]: pgmap v317: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 983 GiB / 983 GiB avail 2024-03-21T10:20:43.979 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:20:43 smithi100 ceph-mon[35193]: pgmap v317: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 983 GiB / 983 GiB avail 2024-03-21T10:20:45.979 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:20:45 smithi100 ceph-mon[35193]: pgmap v318: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 983 GiB / 983 GiB avail 2024-03-21T10:20:45.979 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:20:45 smithi100 ceph-mon[35193]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:20:45.980 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:20:45 smithi100 ceph-mon[35193]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:20:45.980 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:20:45 smithi100 ceph-mon[35193]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T10:20:45.989 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:20:45 smithi157 ceph-mon[35581]: pgmap v318: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 983 GiB / 983 GiB avail 2024-03-21T10:20:45.990 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:20:45 smithi157 ceph-mon[35581]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:20:45.990 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:20:45 smithi157 ceph-mon[35581]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:20:45.990 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:20:45 smithi157 ceph-mon[35581]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T10:20:46.214 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:20:45 smithi047 ceph-mon[31547]: pgmap v318: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 983 GiB / 983 GiB avail 2024-03-21T10:20:46.214 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:20:45 smithi047 ceph-mon[31547]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:20:46.214 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:20:45 smithi047 ceph-mon[31547]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:20:46.214 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:20:45 smithi047 ceph-mon[31547]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T10:20:47.979 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:20:47 smithi100 ceph-mon[35193]: pgmap v319: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 983 GiB / 983 GiB avail 2024-03-21T10:20:47.990 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:20:47 smithi157 ceph-mon[35581]: pgmap v319: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 983 GiB / 983 GiB avail 2024-03-21T10:20:48.214 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:20:47 smithi047 ceph-mon[31547]: pgmap v319: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 983 GiB / 983 GiB avail 2024-03-21T10:20:49.926 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:20:49 smithi157 ceph-mon[35581]: pgmap v320: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 983 GiB / 983 GiB avail 2024-03-21T10:20:49.926 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:20:49 smithi157 ceph-mon[35581]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:20:49.926 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:20:49 smithi157 ceph-mon[35581]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:20:49.926 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:20:49 smithi157 ceph-mon[35581]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T10:20:49.926 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:20:49 smithi157 ceph-mon[35581]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T10:20:49.926 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:20:49 smithi157 ceph-mon[35581]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:20:49.927 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:20:49 smithi157 ceph-mon[35581]: from='osd.11 [v2:172.21.15.157:6824/2671029483,v1:172.21.15.157:6825/2671029483]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-03-21T10:20:49.927 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:20:49 smithi157 ceph-mon[35581]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-03-21T10:20:49.980 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:20:49 smithi100 ceph-mon[35193]: pgmap v320: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 983 GiB / 983 GiB avail 2024-03-21T10:20:49.980 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:20:49 smithi100 ceph-mon[35193]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:20:49.980 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:20:49 smithi100 ceph-mon[35193]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:20:49.980 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:20:49 smithi100 ceph-mon[35193]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T10:20:49.980 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:20:49 smithi100 ceph-mon[35193]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T10:20:49.980 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:20:49 smithi100 ceph-mon[35193]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:20:49.980 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:20:49 smithi100 ceph-mon[35193]: from='osd.11 [v2:172.21.15.157:6824/2671029483,v1:172.21.15.157:6825/2671029483]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-03-21T10:20:49.980 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:20:49 smithi100 ceph-mon[35193]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-03-21T10:20:50.028 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-21T10:20:50.026+0000 7f4f3a000700 1 -- 172.21.15.157:0/2528217324 <== mgr.14152 v2:172.21.15.47:6800/2823885994 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+38 (secure 0 0 0) 0x7f4f3c0625d0 con 0x7f4f240a3030 2024-03-21T10:20:50.029 INFO:teuthology.orchestra.run.smithi157.stdout:Created osd(s) 11 on host 'smithi157' 2024-03-21T10:20:50.031 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-21T10:20:50.030+0000 7f4f44600700 1 -- 172.21.15.157:0/2528217324 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f4f240a3030 msgr2=0x7f4f240a54f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:20:50.031 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-21T10:20:50.030+0000 7f4f44600700 1 --2- 172.21.15.157:0/2528217324 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f4f240a3030 0x7f4f240a54f0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f4f28005fd0 tx=0x7f4f28005ee0 comp rx=0 tx=0).stop 2024-03-21T10:20:50.032 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-21T10:20:50.030+0000 7f4f44600700 1 -- 172.21.15.157:0/2528217324 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4f3c104410 msgr2=0x7f4f3c19a4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:20:50.032 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-21T10:20:50.030+0000 7f4f44600700 1 --2- 172.21.15.157:0/2528217324 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4f3c104410 0x7f4f3c19a4e0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f4f3000d9b0 tx=0x7f4f3000dd70 comp rx=0 tx=0).stop 2024-03-21T10:20:50.032 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-21T10:20:50.030+0000 7f4f44600700 1 -- 172.21.15.157:0/2528217324 shutdown_connections 2024-03-21T10:20:50.032 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-21T10:20:50.030+0000 7f4f44600700 1 --2- 172.21.15.157:0/2528217324 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f4f240a3030 0x7f4f240a54f0 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:20:50.032 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-21T10:20:50.030+0000 7f4f44600700 1 --2- 172.21.15.157:0/2528217324 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4f3c105fb0 0x7f4f3c1a1aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:20:50.032 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-21T10:20:50.030+0000 7f4f44600700 1 --2- 172.21.15.157:0/2528217324 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4f3c104410 0x7f4f3c19a4e0 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:20:50.032 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-21T10:20:50.030+0000 7f4f44600700 1 --2- 172.21.15.157:0/2528217324 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4f3c105610 0x7f4f3c19aa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:20:50.033 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-21T10:20:50.030+0000 7f4f44600700 1 -- 172.21.15.157:0/2528217324 >> 172.21.15.157:0/2528217324 conn(0x7f4f3c0ff9b0 msgr2=0x7f4f3c108910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:20:50.033 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-21T10:20:50.031+0000 7f4f44600700 1 -- 172.21.15.157:0/2528217324 shutdown_connections 2024-03-21T10:20:50.033 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-21T10:20:50.031+0000 7f4f44600700 1 -- 172.21.15.157:0/2528217324 wait complete. 2024-03-21T10:20:50.214 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:20:49 smithi047 ceph-mon[31547]: pgmap v320: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 983 GiB / 983 GiB avail 2024-03-21T10:20:50.214 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:20:49 smithi047 ceph-mon[31547]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:20:50.214 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:20:49 smithi047 ceph-mon[31547]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:20:50.214 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:20:49 smithi047 ceph-mon[31547]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T10:20:50.215 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:20:49 smithi047 ceph-mon[31547]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T10:20:50.215 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:20:49 smithi047 ceph-mon[31547]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:20:50.215 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:20:49 smithi047 ceph-mon[31547]: from='osd.11 [v2:172.21.15.157:6824/2671029483,v1:172.21.15.157:6825/2671029483]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-03-21T10:20:50.215 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:20:49 smithi047 ceph-mon[31547]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-03-21T10:20:50.856 DEBUG:teuthology.orchestra.run.smithi157:osd.11> sudo journalctl -f -n 0 -u ceph-3056d51a-e76b-11ee-95cd-87774f69a715@osd.11.service 2024-03-21T10:20:50.862 INFO:tasks.cephadm:Waiting for 12 OSDs to come up... 2024-03-21T10:20:50.862 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2edfee68ad7243bed422a049b6f5f58619300f11 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3056d51a-e76b-11ee-95cd-87774f69a715 -- ceph osd stat -f json 2024-03-21T10:20:51.039 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:20:50 smithi157 ceph-mon[35581]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-03-21T10:20:51.039 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:20:50 smithi157 ceph-mon[35581]: osdmap e80: 12 total, 11 up, 12 in 2024-03-21T10:20:51.039 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:20:50 smithi157 ceph-mon[35581]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-21T10:20:51.040 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:20:50 smithi157 ceph-mon[35581]: from='osd.11 [v2:172.21.15.157:6824/2671029483,v1:172.21.15.157:6825/2671029483]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi157", "root=default"]}]: dispatch 2024-03-21T10:20:51.040 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:20:50 smithi157 ceph-mon[35581]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi157", "root=default"]}]: dispatch 2024-03-21T10:20:51.040 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:20:50 smithi157 ceph-mon[35581]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:20:51.040 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:20:50 smithi157 ceph-mon[35581]: pgmap v322: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 983 GiB / 983 GiB avail 2024-03-21T10:20:51.040 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:20:50 smithi157 ceph-mon[35581]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:20:51.040 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:20:50 smithi157 ceph-mon[35581]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T10:20:51.214 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:20:50 smithi047 ceph-mon[31547]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-03-21T10:20:51.214 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:20:50 smithi047 ceph-mon[31547]: osdmap e80: 12 total, 11 up, 12 in 2024-03-21T10:20:51.214 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:20:50 smithi047 ceph-mon[31547]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-21T10:20:51.214 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:20:50 smithi047 ceph-mon[31547]: from='osd.11 [v2:172.21.15.157:6824/2671029483,v1:172.21.15.157:6825/2671029483]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi157", "root=default"]}]: dispatch 2024-03-21T10:20:51.214 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:20:50 smithi047 ceph-mon[31547]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi157", "root=default"]}]: dispatch 2024-03-21T10:20:51.214 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:20:50 smithi047 ceph-mon[31547]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:20:51.214 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:20:50 smithi047 ceph-mon[31547]: pgmap v322: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 983 GiB / 983 GiB avail 2024-03-21T10:20:51.214 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:20:50 smithi047 ceph-mon[31547]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:20:51.215 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:20:50 smithi047 ceph-mon[31547]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T10:20:51.230 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:20:50 smithi100 ceph-mon[35193]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-03-21T10:20:51.230 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:20:50 smithi100 ceph-mon[35193]: osdmap e80: 12 total, 11 up, 12 in 2024-03-21T10:20:51.230 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:20:50 smithi100 ceph-mon[35193]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-21T10:20:51.230 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:20:50 smithi100 ceph-mon[35193]: from='osd.11 [v2:172.21.15.157:6824/2671029483,v1:172.21.15.157:6825/2671029483]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi157", "root=default"]}]: dispatch 2024-03-21T10:20:51.230 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:20:50 smithi100 ceph-mon[35193]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi157", "root=default"]}]: dispatch 2024-03-21T10:20:51.230 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:20:50 smithi100 ceph-mon[35193]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:20:51.230 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:20:50 smithi100 ceph-mon[35193]: pgmap v322: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 983 GiB / 983 GiB avail 2024-03-21T10:20:51.230 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:20:50 smithi100 ceph-mon[35193]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:20:51.230 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:20:50 smithi100 ceph-mon[35193]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T10:20:51.349 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/3056d51a-e76b-11ee-95cd-87774f69a715/mon.a/config 2024-03-21T10:20:51.928 INFO:journalctl@ceph.osd.11.smithi157.stdout:Mar 21 10:20:51 smithi157 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-11[61259]: 2024-03-21T10:20:51.670+0000 7fd100800700 -1 osd.11 0 waiting for initial osdmap 2024-03-21T10:20:51.928 INFO:journalctl@ceph.osd.11.smithi157.stdout:Mar 21 10:20:51 smithi157 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-11[61259]: 2024-03-21T10:20:51.706+0000 7fd0f6600700 -1 osd.11 81 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-21T10:20:52.214 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:20:51 smithi047 ceph-mon[31547]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi157", "root=default"]}]': finished 2024-03-21T10:20:52.214 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:20:51 smithi047 ceph-mon[31547]: osdmap e81: 12 total, 11 up, 12 in 2024-03-21T10:20:52.214 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:20:51 smithi047 ceph-mon[31547]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-21T10:20:52.214 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:20:51 smithi047 ceph-mon[31547]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-21T10:20:52.214 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:20:51 smithi047 ceph-mon[31547]: from='osd.11 ' entity='osd.11' 2024-03-21T10:20:52.229 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:20:51 smithi100 ceph-mon[35193]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi157", "root=default"]}]': finished 2024-03-21T10:20:52.230 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:20:51 smithi100 ceph-mon[35193]: osdmap e81: 12 total, 11 up, 12 in 2024-03-21T10:20:52.230 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:20:51 smithi100 ceph-mon[35193]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-21T10:20:52.230 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:20:51 smithi100 ceph-mon[35193]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-21T10:20:52.230 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:20:51 smithi100 ceph-mon[35193]: from='osd.11 ' entity='osd.11' 2024-03-21T10:20:52.239 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:20:51 smithi157 ceph-mon[35581]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi157", "root=default"]}]': finished 2024-03-21T10:20:52.240 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:20:51 smithi157 ceph-mon[35581]: osdmap e81: 12 total, 11 up, 12 in 2024-03-21T10:20:52.240 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:20:51 smithi157 ceph-mon[35581]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-21T10:20:52.240 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:20:51 smithi157 ceph-mon[35581]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-21T10:20:52.240 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:20:51 smithi157 ceph-mon[35581]: from='osd.11 ' entity='osd.11' 2024-03-21T10:20:53.214 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:20:52 smithi047 ceph-mon[31547]: purged_snaps scrub starts 2024-03-21T10:20:53.214 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:20:52 smithi047 ceph-mon[31547]: purged_snaps scrub ok 2024-03-21T10:20:53.214 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:20:52 smithi047 ceph-mon[31547]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-21T10:20:53.214 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:20:52 smithi047 ceph-mon[31547]: pgmap v324: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 983 GiB / 983 GiB avail 2024-03-21T10:20:53.214 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:20:52 smithi047 ceph-mon[31547]: osd.11 [v2:172.21.15.157:6824/2671029483,v1:172.21.15.157:6825/2671029483] boot 2024-03-21T10:20:53.215 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:20:52 smithi047 ceph-mon[31547]: osdmap e82: 12 total, 12 up, 12 in 2024-03-21T10:20:53.215 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:20:52 smithi047 ceph-mon[31547]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-21T10:20:53.229 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:20:52 smithi100 ceph-mon[35193]: purged_snaps scrub starts 2024-03-21T10:20:53.230 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:20:52 smithi100 ceph-mon[35193]: purged_snaps scrub ok 2024-03-21T10:20:53.230 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:20:52 smithi100 ceph-mon[35193]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-21T10:20:53.230 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:20:52 smithi100 ceph-mon[35193]: pgmap v324: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 983 GiB / 983 GiB avail 2024-03-21T10:20:53.230 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:20:52 smithi100 ceph-mon[35193]: osd.11 [v2:172.21.15.157:6824/2671029483,v1:172.21.15.157:6825/2671029483] boot 2024-03-21T10:20:53.230 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:20:52 smithi100 ceph-mon[35193]: osdmap e82: 12 total, 12 up, 12 in 2024-03-21T10:20:53.230 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:20:52 smithi100 ceph-mon[35193]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-21T10:20:53.240 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:20:52 smithi157 ceph-mon[35581]: purged_snaps scrub starts 2024-03-21T10:20:53.240 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:20:52 smithi157 ceph-mon[35581]: purged_snaps scrub ok 2024-03-21T10:20:53.240 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:20:52 smithi157 ceph-mon[35581]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-21T10:20:53.240 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:20:52 smithi157 ceph-mon[35581]: pgmap v324: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 983 GiB / 983 GiB avail 2024-03-21T10:20:53.240 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:20:52 smithi157 ceph-mon[35581]: osd.11 [v2:172.21.15.157:6824/2671029483,v1:172.21.15.157:6825/2671029483] boot 2024-03-21T10:20:53.240 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:20:52 smithi157 ceph-mon[35581]: osdmap e82: 12 total, 12 up, 12 in 2024-03-21T10:20:53.240 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:20:52 smithi157 ceph-mon[35581]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-21T10:20:53.797 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:20:53.794+0000 7f6e14400700 1 -- 172.21.15.47:0/1050851693 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6e0c1033d0 msgr2=0x7f6e0c103830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:20:53.797 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:20:53.794+0000 7f6e14400700 1 --2- 172.21.15.47:0/1050851693 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6e0c1033d0 0x7f6e0c103830 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f6df80099c0 tx=0x7f6df8009cd0 comp rx=0 tx=0).stop 2024-03-21T10:20:53.797 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:20:53.794+0000 7f6e14400700 1 -- 172.21.15.47:0/1050851693 shutdown_connections 2024-03-21T10:20:53.798 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:20:53.795+0000 7f6e14400700 1 --2- 172.21.15.47:0/1050851693 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6e0c103d70 0x7f6e0c10a620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:20:53.798 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:20:53.795+0000 7f6e14400700 1 --2- 172.21.15.47:0/1050851693 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6e0c1021d0 0x7f6e0c1025f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:20:53.798 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:20:53.795+0000 7f6e14400700 1 --2- 172.21.15.47:0/1050851693 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6e0c1033d0 0x7f6e0c103830 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:20:53.798 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:20:53.795+0000 7f6e14400700 1 -- 172.21.15.47:0/1050851693 >> 172.21.15.47:0/1050851693 conn(0x7f6e0c0fd730 msgr2=0x7f6e0c0ffbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:20:53.798 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:20:53.795+0000 7f6e14400700 1 -- 172.21.15.47:0/1050851693 shutdown_connections 2024-03-21T10:20:53.798 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:20:53.795+0000 7f6e14400700 1 -- 172.21.15.47:0/1050851693 wait complete. 2024-03-21T10:20:53.798 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:20:53.796+0000 7f6e14400700 1 Processor -- start 2024-03-21T10:20:53.798 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:20:53.796+0000 7f6e14400700 1 -- start start 2024-03-21T10:20:53.800 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:20:53.797+0000 7f6e14400700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6e0c1021d0 0x7f6e0c075f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:20:53.800 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:20:53.797+0000 7f6e14400700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6e0c1033d0 0x7f6e0c074580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:20:53.800 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:20:53.797+0000 7f6e14400700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6e0c103d70 0x7f6e0c074ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:20:53.800 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:20:53.797+0000 7f6e14400700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e0c10dc60 con 0x7f6e0c1033d0 2024-03-21T10:20:53.800 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:20:53.797+0000 7f6e14400700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e0c10dae0 con 0x7f6e0c1021d0 2024-03-21T10:20:53.800 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:20:53.797+0000 7f6e14400700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e0c10dde0 con 0x7f6e0c103d70 2024-03-21T10:20:53.800 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:20:53.797+0000 7f6e11000700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6e0c1033d0 0x7f6e0c074580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:20:53.801 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:20:53.797+0000 7f6e11000700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6e0c1033d0 0x7f6e0c074580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:35090/0 (socket says 172.21.15.47:35090) 2024-03-21T10:20:53.801 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:20:53.797+0000 7f6e11000700 1 -- 172.21.15.47:0/1673370558 learned_addr learned my addr 172.21.15.47:0/1673370558 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:20:53.801 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:20:53.797+0000 7f6e12400700 1 --2- 172.21.15.47:0/1673370558 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6e0c103d70 0x7f6e0c074ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:20:53.801 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:20:53.797+0000 7f6e11a00700 1 --2- 172.21.15.47:0/1673370558 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6e0c1021d0 0x7f6e0c075f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:20:53.801 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:20:53.798+0000 7f6e11000700 1 -- 172.21.15.47:0/1673370558 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6e0c103d70 msgr2=0x7f6e0c074ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:20:53.801 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:20:53.798+0000 7f6e11000700 1 --2- 172.21.15.47:0/1673370558 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6e0c103d70 0x7f6e0c074ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:20:53.801 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:20:53.798+0000 7f6e11000700 1 -- 172.21.15.47:0/1673370558 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6e0c1021d0 msgr2=0x7f6e0c075f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:20:53.801 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:20:53.798+0000 7f6e11000700 1 --2- 172.21.15.47:0/1673370558 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6e0c1021d0 0x7f6e0c075f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:20:53.801 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:20:53.798+0000 7f6e11000700 1 -- 172.21.15.47:0/1673370558 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e0c075370 con 0x7f6e0c1033d0 2024-03-21T10:20:53.801 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:20:53.798+0000 7f6e12400700 1 --2- 172.21.15.47:0/1673370558 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6e0c103d70 0x7f6e0c074ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T10:20:53.801 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:20:53.798+0000 7f6e11000700 1 --2- 172.21.15.47:0/1673370558 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6e0c1033d0 0x7f6e0c074580 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f6df8006010 tx=0x7f6df80047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:20:53.801 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:20:53.798+0000 7f6e0a000700 1 -- 172.21.15.47:0/1673370558 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6df8027070 con 0x7f6e0c1033d0 2024-03-21T10:20:53.802 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:20:53.798+0000 7f6e0a000700 1 -- 172.21.15.47:0/1673370558 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f6df80040d0 con 0x7f6e0c1033d0 2024-03-21T10:20:53.802 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:20:53.798+0000 7f6e14400700 1 -- 172.21.15.47:0/1673370558 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6e0c1a2400 con 0x7f6e0c1033d0 2024-03-21T10:20:53.802 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:20:53.798+0000 7f6e0a000700 1 -- 172.21.15.47:0/1673370558 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6df800f460 con 0x7f6e0c1033d0 2024-03-21T10:20:53.802 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:20:53.798+0000 7f6e14400700 1 -- 172.21.15.47:0/1673370558 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6e0c1a2850 con 0x7f6e0c1033d0 2024-03-21T10:20:53.803 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:20:53.800+0000 7f6e14400700 1 -- 172.21.15.47:0/1673370558 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6e0c068e50 con 0x7f6e0c1033d0 2024-03-21T10:20:53.803 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:20:53.802+0000 7f6e0a000700 1 -- 172.21.15.47:0/1673370558 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 16) v1 ==== 137854+0+0 (secure 0 0 0) 0x7f6df802a070 con 0x7f6e0c1033d0 2024-03-21T10:20:53.805 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:20:53.803+0000 7f6e0a000700 1 --2- 172.21.15.47:0/1673370558 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f6df40a2fb0 0x7f6df40a5470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:20:53.805 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:20:53.803+0000 7f6e0a000700 1 -- 172.21.15.47:0/1673370558 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f6df80d07f0 con 0x7f6e0c1033d0 2024-03-21T10:20:53.805 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:20:53.804+0000 7f6e11a00700 1 --2- 172.21.15.47:0/1673370558 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f6df40a2fb0 0x7f6df40a5470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:20:53.805 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:20:53.805+0000 7f6e11a00700 1 --2- 172.21.15.47:0/1673370558 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f6df40a2fb0 0x7f6df40a5470 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f6e00005950 tx=0x7f6e000058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:20:53.807 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:20:53.807+0000 7f6e0a000700 1 -- 172.21.15.47:0/1673370558 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f6df8031070 con 0x7f6e0c1033d0 2024-03-21T10:20:54.043 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:20:54.042+0000 7f6e14400700 1 -- 172.21.15.47:0/1673370558 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) v1 -- 0x7f6e0c02da30 con 0x7f6e0c1033d0 2024-03-21T10:20:54.044 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:20:54.043+0000 7f6e0a000700 1 -- 172.21.15.47:0/1673370558 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v83) v1 ==== 74+0+133 (secure 0 0 0) 0x7f6df8013070 con 0x7f6e0c1033d0 2024-03-21T10:20:54.044 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-21T10:20:54.047 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:20:54.046+0000 7f6e14400700 1 -- 172.21.15.47:0/1673370558 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f6df40a2fb0 msgr2=0x7f6df40a5470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:20:54.047 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:20:54.046+0000 7f6e14400700 1 --2- 172.21.15.47:0/1673370558 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f6df40a2fb0 0x7f6df40a5470 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f6e00005950 tx=0x7f6e000058e0 comp rx=0 tx=0).stop 2024-03-21T10:20:54.047 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:20:54.046+0000 7f6e14400700 1 -- 172.21.15.47:0/1673370558 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6e0c1033d0 msgr2=0x7f6e0c074580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:20:54.047 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:20:54.046+0000 7f6e14400700 1 --2- 172.21.15.47:0/1673370558 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6e0c1033d0 0x7f6e0c074580 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f6df8006010 tx=0x7f6df80047b0 comp rx=0 tx=0).stop 2024-03-21T10:20:54.048 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:20:54.046+0000 7f6e14400700 1 -- 172.21.15.47:0/1673370558 shutdown_connections 2024-03-21T10:20:54.048 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:20:54.046+0000 7f6e14400700 1 --2- 172.21.15.47:0/1673370558 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f6df40a2fb0 0x7f6df40a5470 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:20:54.048 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:20:54.046+0000 7f6e14400700 1 --2- 172.21.15.47:0/1673370558 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6e0c103d70 0x7f6e0c074ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:20:54.048 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:20:54.046+0000 7f6e14400700 1 --2- 172.21.15.47:0/1673370558 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6e0c1021d0 0x7f6e0c075f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:20:54.048 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:20:54.046+0000 7f6e14400700 1 --2- 172.21.15.47:0/1673370558 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6e0c1033d0 0x7f6e0c074580 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:20:54.048 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:20:54.046+0000 7f6e14400700 1 -- 172.21.15.47:0/1673370558 >> 172.21.15.47:0/1673370558 conn(0x7f6e0c0fd730 msgr2=0x7f6e0c1067f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:20:54.048 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:20:54.046+0000 7f6e14400700 1 -- 172.21.15.47:0/1673370558 shutdown_connections 2024-03-21T10:20:54.048 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:20:54.046+0000 7f6e14400700 1 -- 172.21.15.47:0/1673370558 wait complete. 2024-03-21T10:20:54.915 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":83,"num_osds":12,"num_up_osds":12,"osd_up_since":1711016452,"num_in_osds":12,"osd_in_since":1711016427,"num_remapped_pgs":0} 2024-03-21T10:20:54.916 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2edfee68ad7243bed422a049b6f5f58619300f11 shell --fsid 3056d51a-e76b-11ee-95cd-87774f69a715 -- ceph osd dump --format=json 2024-03-21T10:20:55.213 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:20:54 smithi047 ceph-mon[31547]: osdmap e83: 12 total, 12 up, 12 in 2024-03-21T10:20:55.214 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:20:54 smithi047 ceph-mon[31547]: pgmap v327: 1 pgs: 1 active+clean; 577 KiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-21T10:20:55.214 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:20:54 smithi047 ceph-mon[31547]: from='client.? 172.21.15.47:0/1673370558' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-03-21T10:20:55.240 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:20:54 smithi157 ceph-mon[35581]: osdmap e83: 12 total, 12 up, 12 in 2024-03-21T10:20:55.241 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:20:54 smithi157 ceph-mon[35581]: pgmap v327: 1 pgs: 1 active+clean; 577 KiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-21T10:20:55.241 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:20:54 smithi157 ceph-mon[35581]: from='client.? 172.21.15.47:0/1673370558' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-03-21T10:20:55.381 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/3056d51a-e76b-11ee-95cd-87774f69a715/mon.a/config 2024-03-21T10:20:55.479 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:20:54 smithi100 ceph-mon[35193]: osdmap e83: 12 total, 12 up, 12 in 2024-03-21T10:20:55.479 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:20:54 smithi100 ceph-mon[35193]: pgmap v327: 1 pgs: 1 active+clean; 577 KiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-21T10:20:55.480 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:20:54 smithi100 ceph-mon[35193]: from='client.? 172.21.15.47:0/1673370558' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-03-21T10:20:56.715 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:20:56 smithi047 ceph-mon[31547]: Detected new or changed devices on smithi157 2024-03-21T10:20:56.715 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:20:56 smithi047 ceph-mon[31547]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:20:56.715 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:20:56 smithi047 ceph-mon[31547]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:20:56.715 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:20:56 smithi047 ceph-mon[31547]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-21T10:20:56.715 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:20:56 smithi047 ceph-mon[31547]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-21T10:20:56.715 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:20:56 smithi047 ceph-mon[31547]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-03-21T10:20:56.715 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:20:56 smithi047 ceph-mon[31547]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-03-21T10:20:56.715 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:20:56 smithi047 ceph-mon[31547]: Adjusting osd_memory_target on smithi157 to 4272M 2024-03-21T10:20:56.715 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:20:56 smithi047 ceph-mon[31547]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:20:56.715 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:20:56 smithi047 ceph-mon[31547]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T10:20:56.716 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:20:56 smithi047 ceph-mon[31547]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T10:20:56.716 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:20:56 smithi047 ceph-mon[31547]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:20:56.980 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:20:56 smithi100 ceph-mon[35193]: Detected new or changed devices on smithi157 2024-03-21T10:20:56.980 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:20:56 smithi100 ceph-mon[35193]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:20:56.980 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:20:56 smithi100 ceph-mon[35193]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:20:56.981 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:20:56 smithi100 ceph-mon[35193]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-21T10:20:56.981 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:20:56 smithi100 ceph-mon[35193]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-21T10:20:56.981 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:20:56 smithi100 ceph-mon[35193]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-03-21T10:20:56.981 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:20:56 smithi100 ceph-mon[35193]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-03-21T10:20:56.981 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:20:56 smithi100 ceph-mon[35193]: Adjusting osd_memory_target on smithi157 to 4272M 2024-03-21T10:20:56.981 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:20:56 smithi100 ceph-mon[35193]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:20:56.981 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:20:56 smithi100 ceph-mon[35193]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T10:20:56.981 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:20:56 smithi100 ceph-mon[35193]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T10:20:56.981 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:20:56 smithi100 ceph-mon[35193]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:20:56.990 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:20:56 smithi157 ceph-mon[35581]: Detected new or changed devices on smithi157 2024-03-21T10:20:56.990 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:20:56 smithi157 ceph-mon[35581]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:20:56.991 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:20:56 smithi157 ceph-mon[35581]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:20:56.991 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:20:56 smithi157 ceph-mon[35581]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-21T10:20:56.991 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:20:56 smithi157 ceph-mon[35581]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-21T10:20:56.991 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:20:56 smithi157 ceph-mon[35581]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-03-21T10:20:56.991 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:20:56 smithi157 ceph-mon[35581]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-03-21T10:20:56.991 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:20:56 smithi157 ceph-mon[35581]: Adjusting osd_memory_target on smithi157 to 4272M 2024-03-21T10:20:56.991 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:20:56 smithi157 ceph-mon[35581]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:20:56.991 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:20:56 smithi157 ceph-mon[35581]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T10:20:56.991 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:20:56 smithi157 ceph-mon[35581]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T10:20:56.991 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:20:56 smithi157 ceph-mon[35581]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:20:57.709 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:20:57.706+0000 7f06e1000700 1 -- 172.21.15.47:0/811778576 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f06dc102410 msgr2=0x7f06dc1027f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:20:57.709 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:20:57.706+0000 7f06e1000700 1 --2- 172.21.15.47:0/811778576 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f06dc102410 0x7f06dc1027f0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f06cc0099c0 tx=0x7f06cc009cd0 comp rx=0 tx=0).stop 2024-03-21T10:20:57.709 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:20:57.707+0000 7f06e1000700 1 -- 172.21.15.47:0/811778576 shutdown_connections 2024-03-21T10:20:57.709 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:20:57.707+0000 7f06e1000700 1 --2- 172.21.15.47:0/811778576 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f06dc1074f0 0x7f06dc10f0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:20:57.709 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:20:57.707+0000 7f06e1000700 1 --2- 172.21.15.47:0/811778576 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f06dc102410 0x7f06dc1027f0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:20:57.710 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:20:57.707+0000 7f06e1000700 1 --2- 172.21.15.47:0/811778576 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f06dc102dc0 0x7f06dc106e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:20:57.710 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:20:57.707+0000 7f06e1000700 1 -- 172.21.15.47:0/811778576 >> 172.21.15.47:0/811778576 conn(0x7f06dc0fdc60 msgr2=0x7f06dc100080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:20:57.710 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:20:57.707+0000 7f06e1000700 1 -- 172.21.15.47:0/811778576 shutdown_connections 2024-03-21T10:20:57.710 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:20:57.707+0000 7f06e1000700 1 -- 172.21.15.47:0/811778576 wait complete. 2024-03-21T10:20:57.712 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:20:57.708+0000 7f06e1000700 1 Processor -- start 2024-03-21T10:20:57.712 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:20:57.708+0000 7f06e1000700 1 -- start start 2024-03-21T10:20:57.712 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:20:57.709+0000 7f06e1000700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f06dc102410 0x7f06dc19e600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:20:57.712 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:20:57.709+0000 7f06e1000700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f06dc102dc0 0x7f06dc19eb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:20:57.713 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:20:57.709+0000 7f06e1000700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f06dc1074f0 0x7f06dc1986d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:20:57.713 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:20:57.709+0000 7f06e1000700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06dc112450 con 0x7f06dc102410 2024-03-21T10:20:57.713 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:20:57.709+0000 7f06e1000700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06dc1122d0 con 0x7f06dc1074f0 2024-03-21T10:20:57.713 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:20:57.709+0000 7f06e1000700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06dc1125d0 con 0x7f06dc102dc0 2024-03-21T10:20:57.713 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:20:57.709+0000 7f06d9e00700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f06dc102410 0x7f06dc19e600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:20:57.713 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:20:57.709+0000 7f06d9e00700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f06dc102410 0x7f06dc19e600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:35112/0 (socket says 172.21.15.47:35112) 2024-03-21T10:20:57.713 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:20:57.709+0000 7f06d9e00700 1 -- 172.21.15.47:0/999578582 learned_addr learned my addr 172.21.15.47:0/999578582 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:20:57.713 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:20:57.709+0000 7f06d9400700 1 --2- 172.21.15.47:0/999578582 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f06dc102dc0 0x7f06dc19eb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:20:57.713 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:20:57.710+0000 7f06da800700 1 --2- 172.21.15.47:0/999578582 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f06dc1074f0 0x7f06dc1986d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:20:57.713 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:20:57.710+0000 7f06d9e00700 1 -- 172.21.15.47:0/999578582 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f06dc102dc0 msgr2=0x7f06dc19eb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:20:57.713 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:20:57.710+0000 7f06d9e00700 1 --2- 172.21.15.47:0/999578582 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f06dc102dc0 0x7f06dc19eb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:20:57.713 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:20:57.710+0000 7f06d9e00700 1 -- 172.21.15.47:0/999578582 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f06dc1074f0 msgr2=0x7f06dc1986d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:20:57.713 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:20:57.710+0000 7f06d9e00700 1 --2- 172.21.15.47:0/999578582 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f06dc1074f0 0x7f06dc1986d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:20:57.713 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:20:57.710+0000 7f06d9e00700 1 -- 172.21.15.47:0/999578582 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f06dc198fb0 con 0x7f06dc102410 2024-03-21T10:20:57.714 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:20:57.710+0000 7f06d9e00700 1 --2- 172.21.15.47:0/999578582 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f06dc102410 0x7f06dc19e600 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f06cc009990 tx=0x7f06cc003730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:20:57.714 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:20:57.710+0000 7f06d2a00700 1 -- 172.21.15.47:0/999578582 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06cc026070 con 0x7f06dc102410 2024-03-21T10:20:57.714 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:20:57.710+0000 7f06d2a00700 1 -- 172.21.15.47:0/999578582 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f06cc0040d0 con 0x7f06dc102410 2024-03-21T10:20:57.714 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:20:57.710+0000 7f06e1000700 1 -- 172.21.15.47:0/999578582 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f06dc199240 con 0x7f06dc102410 2024-03-21T10:20:57.714 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:20:57.711+0000 7f06d2a00700 1 -- 172.21.15.47:0/999578582 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06cc005290 con 0x7f06dc102410 2024-03-21T10:20:57.714 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:20:57.711+0000 7f06e1000700 1 -- 172.21.15.47:0/999578582 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f06dc1a5580 con 0x7f06dc102410 2024-03-21T10:20:57.714 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:20:57.712+0000 7f06e1000700 1 -- 172.21.15.47:0/999578582 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f06dc1995a0 con 0x7f06dc102410 2024-03-21T10:20:57.715 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:20:57.714+0000 7f06d2a00700 1 -- 172.21.15.47:0/999578582 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 16) v1 ==== 137854+0+0 (secure 0 0 0) 0x7f06cc029070 con 0x7f06dc102410 2024-03-21T10:20:57.717 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:20:57.716+0000 7f06d2a00700 1 --2- 172.21.15.47:0/999578582 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f06c00a3030 0x7f06c00a54f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:20:57.717 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:20:57.716+0000 7f06d2a00700 1 -- 172.21.15.47:0/999578582 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f06cc0cfb90 con 0x7f06dc102410 2024-03-21T10:20:57.717 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:20:57.716+0000 7f06d9400700 1 --2- 172.21.15.47:0/999578582 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f06c00a3030 0x7f06c00a54f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:20:57.717 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:20:57.717+0000 7f06d9400700 1 --2- 172.21.15.47:0/999578582 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f06c00a3030 0x7f06c00a54f0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f06c8006fd0 tx=0x7f06c8008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:20:57.721 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:20:57.720+0000 7f06d2a00700 1 -- 172.21.15.47:0/999578582 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f06cc05d740 con 0x7f06dc102410 2024-03-21T10:20:57.946 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:20:57.945+0000 7f06e1000700 1 -- 172.21.15.47:0/999578582 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f06dc199c00 con 0x7f06dc102410 2024-03-21T10:20:57.947 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:20:57 smithi047 ceph-mon[31547]: pgmap v328: 1 pgs: 1 active+clean; 577 KiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-21T10:20:57.947 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:20:57.946+0000 7f06d2a00700 1 -- 172.21.15.47:0/999578582 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v83) v1 ==== 74+0+18444 (secure 0 0 0) 0x7f06cc013070 con 0x7f06dc102410 2024-03-21T10:20:57.948 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-21T10:20:57.948 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":83,"fsid":"3056d51a-e76b-11ee-95cd-87774f69a715","created":"2024-03-21T10:11:17.019965+0000","modified":"2024-03-21T10:20:53.966317+0000","last_up_change":"2024-03-21T10:20:52.675328+0000","last_in_change":"2024-03-21T10:20:27.648212+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-21T10:15:40.553114+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b0c7a102-8d89-428b-b793-cbbe67151f50","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6802","nonce":775837313},{"type":"v1","addr":"172.21.15.47:6803","nonce":775837313}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6804","nonce":775837313},{"type":"v1","addr":"172.21.15.47:6805","nonce":775837313}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6808","nonce":775837313},{"type":"v1","addr":"172.21.15.47:6809","nonce":775837313}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6806","nonce":775837313},{"type":"v1","addr":"172.21.15.47:6807","nonce":775837313}]},"public_addr":"172.21.15.47:6803/775837313","cluster_addr":"172.21.15.47:6805/775837313","heartbeat_back_addr":"172.21.15.47:6809/775837313","heartbeat_front_addr":"172.21.15.47:6807/775837313","state":["exists","up"]},{"osd":1,"uuid":"07bb81bb-7cf3-4bbc-ac9e-802f95185d11","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6800","nonce":2923304710},{"type":"v1","addr":"172.21.15.100:6801","nonce":2923304710}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":2923304710},{"type":"v1","addr":"172.21.15.100:6803","nonce":2923304710}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":2923304710},{"type":"v1","addr":"172.21.15.100:6807","nonce":2923304710}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6804","nonce":2923304710},{"type":"v1","addr":"172.21.15.100:6805","nonce":2923304710}]},"public_addr":"172.21.15.100:6801/2923304710","cluster_addr":"172.21.15.100:6803/2923304710","heartbeat_back_addr":"172.21.15.100:6807/2923304710","heartbeat_front_addr":"172.21.15.100:6805/2923304710","state":["exists","up"]},{"osd":2,"uuid":"88ce76b9-6081-4836-aad0-8c249c56e8a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":171377005},{"type":"v1","addr":"172.21.15.157:6801","nonce":171377005}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":171377005},{"type":"v1","addr":"172.21.15.157:6803","nonce":171377005}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":171377005},{"type":"v1","addr":"172.21.15.157:6807","nonce":171377005}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":171377005},{"type":"v1","addr":"172.21.15.157:6805","nonce":171377005}]},"public_addr":"172.21.15.157:6801/171377005","cluster_addr":"172.21.15.157:6803/171377005","heartbeat_back_addr":"172.21.15.157:6807/171377005","heartbeat_front_addr":"172.21.15.157:6805/171377005","state":["exists","up"]},{"osd":3,"uuid":"518aa33d-263f-45e6-aab5-2201ceb40e74","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":27,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6810","nonce":640760101},{"type":"v1","addr":"172.21.15.47:6811","nonce":640760101}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6812","nonce":640760101},{"type":"v1","addr":"172.21.15.47:6813","nonce":640760101}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6816","nonce":640760101},{"type":"v1","addr":"172.21.15.47:6817","nonce":640760101}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6814","nonce":640760101},{"type":"v1","addr":"172.21.15.47:6815","nonce":640760101}]},"public_addr":"172.21.15.47:6811/640760101","cluster_addr":"172.21.15.47:6813/640760101","heartbeat_back_addr":"172.21.15.47:6817/640760101","heartbeat_front_addr":"172.21.15.47:6815/640760101","state":["exists","up"]},{"osd":4,"uuid":"987bde83-ecc6-42e5-bd56-c143b3de4226","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":35,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6808","nonce":2785478639},{"type":"v1","addr":"172.21.15.100:6809","nonce":2785478639}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6810","nonce":2785478639},{"type":"v1","addr":"172.21.15.100:6811","nonce":2785478639}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6814","nonce":2785478639},{"type":"v1","addr":"172.21.15.100:6815","nonce":2785478639}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":2785478639},{"type":"v1","addr":"172.21.15.100:6813","nonce":2785478639}]},"public_addr":"172.21.15.100:6809/2785478639","cluster_addr":"172.21.15.100:6811/2785478639","heartbeat_back_addr":"172.21.15.100:6815/2785478639","heartbeat_front_addr":"172.21.15.100:6813/2785478639","state":["exists","up"]},{"osd":5,"uuid":"012d2188-3a61-4a5f-8d01-2001c15ffbe2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":40,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":1525645410},{"type":"v1","addr":"172.21.15.157:6809","nonce":1525645410}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":1525645410},{"type":"v1","addr":"172.21.15.157:6811","nonce":1525645410}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":1525645410},{"type":"v1","addr":"172.21.15.157:6815","nonce":1525645410}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":1525645410},{"type":"v1","addr":"172.21.15.157:6813","nonce":1525645410}]},"public_addr":"172.21.15.157:6809/1525645410","cluster_addr":"172.21.15.157:6811/1525645410","heartbeat_back_addr":"172.21.15.157:6815/1525645410","heartbeat_front_addr":"172.21.15.157:6813/1525645410","state":["exists","up"]},{"osd":6,"uuid":"c12c14b4-85f0-4d88-937c-de4f556437de","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6818","nonce":2321642790},{"type":"v1","addr":"172.21.15.47:6819","nonce":2321642790}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6820","nonce":2321642790},{"type":"v1","addr":"172.21.15.47:6821","nonce":2321642790}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6824","nonce":2321642790},{"type":"v1","addr":"172.21.15.47:6825","nonce":2321642790}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6822","nonce":2321642790},{"type":"v1","addr":"172.21.15.47:6823","nonce":2321642790}]},"public_addr":"172.21.15.47:6819/2321642790","cluster_addr":"172.21.15.47:6821/2321642790","heartbeat_back_addr":"172.21.15.47:6825/2321642790","heartbeat_front_addr":"172.21.15.47:6823/2321642790","state":["exists","up"]},{"osd":7,"uuid":"3c87c6d0-1d1f-4294-9917-aaaad7dda824","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6816","nonce":4104727330},{"type":"v1","addr":"172.21.15.100:6817","nonce":4104727330}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":4104727330},{"type":"v1","addr":"172.21.15.100:6819","nonce":4104727330}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":4104727330},{"type":"v1","addr":"172.21.15.100:6823","nonce":4104727330}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6820","nonce":4104727330},{"type":"v1","addr":"172.21.15.100:6821","nonce":4104727330}]},"public_addr":"172.21.15.100:6817/4104727330","cluster_addr":"172.21.15.100:6819/4104727330","heartbeat_back_addr":"172.21.15.100:6823/4104727330","heartbeat_front_addr":"172.21.15.100:6821/4104727330","state":["exists","up"]},{"osd":8,"uuid":"502d65cf-53c8-4936-8de4-e11b2a6e1e18","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":61,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":4067715347},{"type":"v1","addr":"172.21.15.157:6817","nonce":4067715347}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":4067715347},{"type":"v1","addr":"172.21.15.157:6819","nonce":4067715347}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":4067715347},{"type":"v1","addr":"172.21.15.157:6823","nonce":4067715347}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":4067715347},{"type":"v1","addr":"172.21.15.157:6821","nonce":4067715347}]},"public_addr":"172.21.15.157:6817/4067715347","cluster_addr":"172.21.15.157:6819/4067715347","heartbeat_back_addr":"172.21.15.157:6823/4067715347","heartbeat_front_addr":"172.21.15.157:6821/4067715347","state":["exists","up"]},{"osd":9,"uuid":"5ea1f5be-8458-4c93-9e61-c2e867a514f3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6826","nonce":1982108738},{"type":"v1","addr":"172.21.15.47:6827","nonce":1982108738}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6828","nonce":1982108738},{"type":"v1","addr":"172.21.15.47:6829","nonce":1982108738}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6832","nonce":1982108738},{"type":"v1","addr":"172.21.15.47:6833","nonce":1982108738}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6830","nonce":1982108738},{"type":"v1","addr":"172.21.15.47:6831","nonce":1982108738}]},"public_addr":"172.21.15.47:6827/1982108738","cluster_addr":"172.21.15.47:6829/1982108738","heartbeat_back_addr":"172.21.15.47:6833/1982108738","heartbeat_front_addr":"172.21.15.47:6831/1982108738","state":["exists","up"]},{"osd":10,"uuid":"2da3da90-a0be-4a54-a9ff-b2d6c9b36870","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":74,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":396391028},{"type":"v1","addr":"172.21.15.100:6825","nonce":396391028}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":396391028},{"type":"v1","addr":"172.21.15.100:6827","nonce":396391028}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6830","nonce":396391028},{"type":"v1","addr":"172.21.15.100:6831","nonce":396391028}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":396391028},{"type":"v1","addr":"172.21.15.100:6829","nonce":396391028}]},"public_addr":"172.21.15.100:6825/396391028","cluster_addr":"172.21.15.100:6827/396391028","heartbeat_back_addr":"172.21.15.100:6831/396391028","heartbeat_front_addr":"172.21.15.100:6829/396391028","state":["exists","up"]},{"osd":11,"uuid":"11798d41-7f7e-4319-95b8-8d3d0946bde9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":82,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":2671029483},{"type":"v1","addr":"172.21.15.157:6825","nonce":2671029483}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":2671029483},{"type":"v1","addr":"172.21.15.157:6827","nonce":2671029483}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":2671029483},{"type":"v1","addr":"172.21.15.157:6831","nonce":2671029483}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":2671029483},{"type":"v1","addr":"172.21.15.157:6829","nonce":2671029483}]},"public_addr":"172.21.15.157:6825/2671029483","cluster_addr":"172.21.15.157:6827/2671029483","heartbeat_back_addr":"172.21.15.157:6831/2671029483","heartbeat_front_addr":"172.21.15.157:6829/2671029483","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T10:14:31.822399+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T10:15:04.175721+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T10:15:35.902595+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T10:16:12.088891+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T10:16:46.428704+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T10:17:20.158270+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T10:17:55.915253+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T10:18:30.064641+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T10:19:04.410004+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T10:19:40.605181+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T10:20:15.636605+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T10:20:50.151365+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.47:0/356822431":"2024-03-22T10:12:25.844530+0000","172.21.15.47:0/1167490879":"2024-03-22T10:12:25.844530+0000","172.21.15.47:0/1540718138":"2024-03-22T10:12:25.844530+0000","172.21.15.47:6801/3836903972":"2024-03-22T10:12:25.844530+0000","172.21.15.47:0/235436602":"2024-03-22T10:11:50.301838+0000","172.21.15.47:0/2052815572":"2024-03-22T10:11:50.301838+0000","172.21.15.47:6800/3942658432":"2024-03-22T10:11:50.301838+0000","172.21.15.47:6801/3942658432":"2024-03-22T10:11:50.301838+0000","172.21.15.47:0/1419516257":"2024-03-22T10:11:50.301838+0000","172.21.15.47:6800/3836903972":"2024-03-22T10:12:25.844530+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-21T10:20:57.950 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:20:57.949+0000 7f06e1000700 1 -- 172.21.15.47:0/999578582 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f06c00a3030 msgr2=0x7f06c00a54f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:20:57.950 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:20:57.949+0000 7f06e1000700 1 --2- 172.21.15.47:0/999578582 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f06c00a3030 0x7f06c00a54f0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f06c8006fd0 tx=0x7f06c8008040 comp rx=0 tx=0).stop 2024-03-21T10:20:57.950 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:20:57.949+0000 7f06e1000700 1 -- 172.21.15.47:0/999578582 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f06dc102410 msgr2=0x7f06dc19e600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:20:57.951 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:20:57.949+0000 7f06e1000700 1 --2- 172.21.15.47:0/999578582 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f06dc102410 0x7f06dc19e600 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f06cc009990 tx=0x7f06cc003730 comp rx=0 tx=0).stop 2024-03-21T10:20:57.951 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:20:57.950+0000 7f06e1000700 1 -- 172.21.15.47:0/999578582 shutdown_connections 2024-03-21T10:20:57.951 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:20:57.950+0000 7f06e1000700 1 --2- 172.21.15.47:0/999578582 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f06c00a3030 0x7f06c00a54f0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:20:57.951 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:20:57.950+0000 7f06e1000700 1 --2- 172.21.15.47:0/999578582 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f06dc1074f0 0x7f06dc1986d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:20:57.952 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:20:57.950+0000 7f06e1000700 1 --2- 172.21.15.47:0/999578582 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f06dc102410 0x7f06dc19e600 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:20:57.952 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:20:57.950+0000 7f06e1000700 1 --2- 172.21.15.47:0/999578582 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f06dc102dc0 0x7f06dc19eb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:20:57.952 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:20:57.950+0000 7f06e1000700 1 -- 172.21.15.47:0/999578582 >> 172.21.15.47:0/999578582 conn(0x7f06dc0fdc60 msgr2=0x7f06dc0ff020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:20:57.952 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:20:57.950+0000 7f06e1000700 1 -- 172.21.15.47:0/999578582 shutdown_connections 2024-03-21T10:20:57.952 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:20:57.950+0000 7f06e1000700 1 -- 172.21.15.47:0/999578582 wait complete. 2024-03-21T10:20:57.979 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:20:57 smithi100 ceph-mon[35193]: pgmap v328: 1 pgs: 1 active+clean; 577 KiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-21T10:20:57.989 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:20:57 smithi157 ceph-mon[35581]: pgmap v328: 1 pgs: 1 active+clean; 577 KiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-21T10:20:58.861 INFO:tasks.cephadm.ceph_manager.ceph:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-21T10:15:40.553114+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-21T10:20:58.861 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2edfee68ad7243bed422a049b6f5f58619300f11 shell --fsid 3056d51a-e76b-11ee-95cd-87774f69a715 -- ceph osd pool get .mgr pg_num 2024-03-21T10:20:58.957 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:20:58 smithi047 ceph-mon[31547]: from='client.? 172.21.15.47:0/999578582' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-21T10:20:58.979 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:20:58 smithi100 ceph-mon[35193]: from='client.? 172.21.15.47:0/999578582' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-21T10:20:58.989 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:20:58 smithi157 ceph-mon[35581]: from='client.? 172.21.15.47:0/999578582' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-21T10:20:59.328 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/3056d51a-e76b-11ee-95cd-87774f69a715/mon.a/config 2024-03-21T10:20:59.824 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:20:59 smithi047 ceph-mon[31547]: pgmap v329: 1 pgs: 1 active+clean; 577 KiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-21T10:20:59.979 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:20:59 smithi100 ceph-mon[35193]: pgmap v329: 1 pgs: 1 active+clean; 577 KiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-21T10:20:59.989 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:20:59 smithi157 ceph-mon[35581]: pgmap v329: 1 pgs: 1 active+clean; 577 KiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-21T10:21:01.493 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:01.491+0000 7f4c8ee00700 1 -- 172.21.15.47:0/3284942776 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4c8810e7a0 msgr2=0x7f4c8810eb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:21:01.493 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:01.491+0000 7f4c8ee00700 1 --2- 172.21.15.47:0/3284942776 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4c8810e7a0 0x7f4c8810eb80 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f4c78009990 tx=0x7f4c78009ca0 comp rx=0 tx=0).stop 2024-03-21T10:21:01.493 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:01.492+0000 7f4c8ee00700 1 -- 172.21.15.47:0/3284942776 shutdown_connections 2024-03-21T10:21:01.493 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:01.492+0000 7f4c8ee00700 1 --2- 172.21.15.47:0/3284942776 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4c88109160 0x7f4c88115670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:21:01.493 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:01.492+0000 7f4c8ee00700 1 --2- 172.21.15.47:0/3284942776 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4c8810e7a0 0x7f4c8810eb80 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:21:01.494 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:01.492+0000 7f4c8ee00700 1 --2- 172.21.15.47:0/3284942776 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4c881087a0 0x7f4c88108c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:21:01.494 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:01.492+0000 7f4c8ee00700 1 -- 172.21.15.47:0/3284942776 >> 172.21.15.47:0/3284942776 conn(0x7f4c88075960 msgr2=0x7f4c88077d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:21:01.494 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:01.492+0000 7f4c8ee00700 1 -- 172.21.15.47:0/3284942776 shutdown_connections 2024-03-21T10:21:01.494 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:01.492+0000 7f4c8ee00700 1 -- 172.21.15.47:0/3284942776 wait complete. 2024-03-21T10:21:01.495 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:01.493+0000 7f4c8ee00700 1 Processor -- start 2024-03-21T10:21:01.495 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:01.493+0000 7f4c8ee00700 1 -- start start 2024-03-21T10:21:01.495 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:01.493+0000 7f4c8ee00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4c881087a0 0x7f4c8819ee60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:21:01.495 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:01.494+0000 7f4c8ee00700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4c88109160 0x7f4c8819f3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:21:01.495 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:01.494+0000 7f4c8ee00700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4c8810e7a0 0x7f4c881a3730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:21:01.495 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:01.494+0000 7f4c8ee00700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c881189d0 con 0x7f4c88109160 2024-03-21T10:21:01.495 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:01.494+0000 7f4c8ee00700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c88118850 con 0x7f4c8810e7a0 2024-03-21T10:21:01.495 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:01.494+0000 7f4c8ee00700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c88118b50 con 0x7f4c881087a0 2024-03-21T10:21:01.495 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:01.494+0000 7f4c87400700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4c88109160 0x7f4c8819f3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:21:01.495 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:01.494+0000 7f4c87400700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4c88109160 0x7f4c8819f3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:35122/0 (socket says 172.21.15.47:35122) 2024-03-21T10:21:01.496 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:01.494+0000 7f4c87400700 1 -- 172.21.15.47:0/2401090592 learned_addr learned my addr 172.21.15.47:0/2401090592 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:21:01.497 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:01.494+0000 7f4c87e00700 1 --2- 172.21.15.47:0/2401090592 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4c881087a0 0x7f4c8819ee60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:21:01.497 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:01.494+0000 7f4c8ce00700 1 --2- 172.21.15.47:0/2401090592 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4c8810e7a0 0x7f4c881a3730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:21:01.497 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:01.494+0000 7f4c87400700 1 -- 172.21.15.47:0/2401090592 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4c881087a0 msgr2=0x7f4c8819ee60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:21:01.497 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:01.494+0000 7f4c87400700 1 --2- 172.21.15.47:0/2401090592 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4c881087a0 0x7f4c8819ee60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:21:01.497 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:01.494+0000 7f4c87400700 1 -- 172.21.15.47:0/2401090592 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4c8810e7a0 msgr2=0x7f4c881a3730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:21:01.497 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:01.494+0000 7f4c87400700 1 --2- 172.21.15.47:0/2401090592 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4c8810e7a0 0x7f4c881a3730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:21:01.497 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:01.494+0000 7f4c87400700 1 -- 172.21.15.47:0/2401090592 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c881a3e70 con 0x7f4c88109160 2024-03-21T10:21:01.497 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:01.495+0000 7f4c87400700 1 --2- 172.21.15.47:0/2401090592 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4c88109160 0x7f4c8819f3a0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f4c7400d950 tx=0x7f4c7400dd10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:21:01.497 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:01.495+0000 7f4c84c00700 1 -- 172.21.15.47:0/2401090592 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c74016070 con 0x7f4c88109160 2024-03-21T10:21:01.497 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:01.495+0000 7f4c84c00700 1 -- 172.21.15.47:0/2401090592 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f4c74010460 con 0x7f4c88109160 2024-03-21T10:21:01.497 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:01.495+0000 7f4c8ee00700 1 -- 172.21.15.47:0/2401090592 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4c881a4160 con 0x7f4c88109160 2024-03-21T10:21:01.498 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:01.495+0000 7f4c84c00700 1 -- 172.21.15.47:0/2401090592 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c7400f620 con 0x7f4c88109160 2024-03-21T10:21:01.498 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:01.495+0000 7f4c8ee00700 1 -- 172.21.15.47:0/2401090592 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4c881ab9e0 con 0x7f4c88109160 2024-03-21T10:21:01.498 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:01.497+0000 7f4c8ee00700 1 -- 172.21.15.47:0/2401090592 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4c88068e50 con 0x7f4c88109160 2024-03-21T10:21:01.499 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:01.499+0000 7f4c84c00700 1 -- 172.21.15.47:0/2401090592 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 16) v1 ==== 137854+0+0 (secure 0 0 0) 0x7f4c74019070 con 0x7f4c88109160 2024-03-21T10:21:01.502 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:01.501+0000 7f4c84c00700 1 --2- 172.21.15.47:0/2401090592 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f4c6c0a2fb0 0x7f4c6c0a5470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:21:01.502 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:01.501+0000 7f4c84c00700 1 -- 172.21.15.47:0/2401090592 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f4c740cec60 con 0x7f4c88109160 2024-03-21T10:21:01.502 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:01.501+0000 7f4c87e00700 1 --2- 172.21.15.47:0/2401090592 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f4c6c0a2fb0 0x7f4c6c0a5470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:21:01.503 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:01.502+0000 7f4c87e00700 1 --2- 172.21.15.47:0/2401090592 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f4c6c0a2fb0 0x7f4c6c0a5470 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f4c7800b5c0 tx=0x7f4c7801a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:21:01.507 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:01.506+0000 7f4c84c00700 1 -- 172.21.15.47:0/2401090592 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f4c7409be90 con 0x7f4c88109160 2024-03-21T10:21:01.740 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:01.739+0000 7f4c8ee00700 1 -- 172.21.15.47:0/2401090592 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f4c8802da30 con 0x7f4c88109160 2024-03-21T10:21:01.741 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:01.740+0000 7f4c84c00700 1 -- 172.21.15.47:0/2401090592 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v83) v1 ==== 93+0+10 (secure 0 0 0) 0x7f4c7409b790 con 0x7f4c88109160 2024-03-21T10:21:01.741 INFO:teuthology.orchestra.run.smithi047.stdout:pg_num: 1 2024-03-21T10:21:01.743 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:01.742+0000 7f4c8ee00700 1 -- 172.21.15.47:0/2401090592 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f4c6c0a2fb0 msgr2=0x7f4c6c0a5470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:21:01.744 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:01.743+0000 7f4c8ee00700 1 --2- 172.21.15.47:0/2401090592 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f4c6c0a2fb0 0x7f4c6c0a5470 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f4c7800b5c0 tx=0x7f4c7801a040 comp rx=0 tx=0).stop 2024-03-21T10:21:01.744 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:01.743+0000 7f4c8ee00700 1 -- 172.21.15.47:0/2401090592 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4c88109160 msgr2=0x7f4c8819f3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:21:01.744 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:01.743+0000 7f4c8ee00700 1 --2- 172.21.15.47:0/2401090592 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4c88109160 0x7f4c8819f3a0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f4c7400d950 tx=0x7f4c7400dd10 comp rx=0 tx=0).stop 2024-03-21T10:21:01.744 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:01.743+0000 7f4c8ee00700 1 -- 172.21.15.47:0/2401090592 shutdown_connections 2024-03-21T10:21:01.744 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:01.743+0000 7f4c8ee00700 1 --2- 172.21.15.47:0/2401090592 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f4c6c0a2fb0 0x7f4c6c0a5470 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:21:01.744 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:01.743+0000 7f4c8ee00700 1 --2- 172.21.15.47:0/2401090592 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4c8810e7a0 0x7f4c881a3730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:21:01.745 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:01.743+0000 7f4c8ee00700 1 --2- 172.21.15.47:0/2401090592 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4c881087a0 0x7f4c8819ee60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:21:01.745 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:01.743+0000 7f4c8ee00700 1 --2- 172.21.15.47:0/2401090592 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4c88109160 0x7f4c8819f3a0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:21:01.745 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:01.743+0000 7f4c8ee00700 1 -- 172.21.15.47:0/2401090592 >> 172.21.15.47:0/2401090592 conn(0x7f4c88075960 msgr2=0x7f4c880760d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:21:01.745 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:01.743+0000 7f4c8ee00700 1 -- 172.21.15.47:0/2401090592 shutdown_connections 2024-03-21T10:21:01.745 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:01.743+0000 7f4c8ee00700 1 -- 172.21.15.47:0/2401090592 wait complete. 2024-03-21T10:21:01.979 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:21:01 smithi100 ceph-mon[35193]: pgmap v330: 1 pgs: 1 active+clean; 577 KiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-21T10:21:01.989 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:21:01 smithi157 ceph-mon[35581]: pgmap v330: 1 pgs: 1 active+clean; 577 KiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-21T10:21:02.099 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:21:01 smithi047 ceph-mon[31547]: pgmap v330: 1 pgs: 1 active+clean; 577 KiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-21T10:21:02.588 INFO:tasks.cephadm:Adding mds.a on smithi047 2024-03-21T10:21:02.589 INFO:tasks.cephadm:Adding mds.d on smithi047 2024-03-21T10:21:02.589 INFO:tasks.cephadm:Adding mds.g on smithi047 2024-03-21T10:21:02.589 INFO:tasks.cephadm:Adding mds.j on smithi047 2024-03-21T10:21:02.589 INFO:tasks.cephadm:Adding mds.b on smithi100 2024-03-21T10:21:02.589 INFO:tasks.cephadm:Adding mds.e on smithi100 2024-03-21T10:21:02.589 INFO:tasks.cephadm:Adding mds.h on smithi100 2024-03-21T10:21:02.589 INFO:tasks.cephadm:Adding mds.k on smithi100 2024-03-21T10:21:02.589 INFO:tasks.cephadm:Adding mds.c on smithi157 2024-03-21T10:21:02.589 INFO:tasks.cephadm:Adding mds.f on smithi157 2024-03-21T10:21:02.589 INFO:tasks.cephadm:Adding mds.i on smithi157 2024-03-21T10:21:02.589 INFO:tasks.cephadm:Adding mds.l on smithi157 2024-03-21T10:21:02.589 DEBUG:teuthology.orchestra.run.smithi157:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2edfee68ad7243bed422a049b6f5f58619300f11 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3056d51a-e76b-11ee-95cd-87774f69a715 -- ceph orch apply mds all '12;smithi047=a;smithi047=d;smithi047=g;smithi047=j;smithi100=b;smithi100=e;smithi100=h;smithi100=k;smithi157=c;smithi157=f;smithi157=i;smithi157=l' 2024-03-21T10:21:02.964 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:21:02 smithi047 ceph-mon[31547]: from='client.? 172.21.15.47:0/2401090592' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-21T10:21:02.990 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:21:02 smithi157 ceph-mon[35581]: from='client.? 172.21.15.47:0/2401090592' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-21T10:21:03.043 INFO:teuthology.orchestra.run.smithi157.stderr:Inferring config /var/lib/ceph/3056d51a-e76b-11ee-95cd-87774f69a715/mon.c/config 2024-03-21T10:21:03.229 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:21:02 smithi100 ceph-mon[35193]: from='client.? 172.21.15.47:0/2401090592' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-21T10:21:03.990 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:21:03 smithi157 ceph-mon[35581]: pgmap v331: 1 pgs: 1 active+clean; 577 KiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-21T10:21:04.214 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:21:03 smithi047 ceph-mon[31547]: pgmap v331: 1 pgs: 1 active+clean; 577 KiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-21T10:21:04.229 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:21:03 smithi100 ceph-mon[35193]: pgmap v331: 1 pgs: 1 active+clean; 577 KiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-21T10:21:05.241 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-21T10:21:05.240+0000 7f92b1200700 1 -- 172.21.15.157:0/3989043051 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f92ac105610 msgr2=0x7f92ac105a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:21:05.241 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-21T10:21:05.240+0000 7f92b1200700 1 --2- 172.21.15.157:0/3989043051 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f92ac105610 0x7f92ac105a70 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f9298009990 tx=0x7f9298009ca0 comp rx=0 tx=0).stop 2024-03-21T10:21:05.241 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-21T10:21:05.240+0000 7f92b1200700 1 -- 172.21.15.157:0/3989043051 shutdown_connections 2024-03-21T10:21:05.242 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-21T10:21:05.240+0000 7f92b1200700 1 --2- 172.21.15.157:0/3989043051 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f92ac105fb0 0x7f92ac10c860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:21:05.242 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-21T10:21:05.240+0000 7f92b1200700 1 --2- 172.21.15.157:0/3989043051 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f92ac104410 0x7f92ac104830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:21:05.242 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-21T10:21:05.240+0000 7f92b1200700 1 --2- 172.21.15.157:0/3989043051 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f92ac105610 0x7f92ac105a70 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:21:05.242 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-21T10:21:05.240+0000 7f92b1200700 1 -- 172.21.15.157:0/3989043051 >> 172.21.15.157:0/3989043051 conn(0x7f92ac0ff9b0 msgr2=0x7f92ac101df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:21:05.242 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-21T10:21:05.240+0000 7f92b1200700 1 -- 172.21.15.157:0/3989043051 shutdown_connections 2024-03-21T10:21:05.242 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-21T10:21:05.240+0000 7f92b1200700 1 -- 172.21.15.157:0/3989043051 wait complete. 2024-03-21T10:21:05.243 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-21T10:21:05.241+0000 7f92b1200700 1 Processor -- start 2024-03-21T10:21:05.243 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-21T10:21:05.242+0000 7f92b1200700 1 -- start start 2024-03-21T10:21:05.243 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-21T10:21:05.242+0000 7f92b1200700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f92ac104410 0x7f92ac19a550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:21:05.243 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-21T10:21:05.242+0000 7f92b1200700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f92ac105610 0x7f92ac19aa90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:21:05.244 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-21T10:21:05.242+0000 7f92b1200700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f92ac105fb0 0x7f92ac1a1b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:21:05.244 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-21T10:21:05.242+0000 7f92b1200700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92ac10ff70 con 0x7f92ac105fb0 2024-03-21T10:21:05.244 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-21T10:21:05.242+0000 7f92b1200700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92ac10fdf0 con 0x7f92ac105610 2024-03-21T10:21:05.244 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-21T10:21:05.242+0000 7f92b1200700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92ac1100f0 con 0x7f92ac104410 2024-03-21T10:21:05.244 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-21T10:21:05.242+0000 7f92a9e00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f92ac104410 0x7f92ac19a550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:21:05.244 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-21T10:21:05.242+0000 7f92a9e00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f92ac104410 0x7f92ac19a550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:54100/0 (socket says 172.21.15.157:54100) 2024-03-21T10:21:05.244 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-21T10:21:05.242+0000 7f92a9e00700 1 -- 172.21.15.157:0/3770703810 learned_addr learned my addr 172.21.15.157:0/3770703810 (peer_addr_for_me v2:172.21.15.157:0/0) 2024-03-21T10:21:05.244 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-21T10:21:05.243+0000 7f92a9400700 1 --2- 172.21.15.157:0/3770703810 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f92ac105610 0x7f92ac19aa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:21:05.244 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-21T10:21:05.243+0000 7f92aa800700 1 --2- 172.21.15.157:0/3770703810 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f92ac105fb0 0x7f92ac1a1b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:21:05.244 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-21T10:21:05.243+0000 7f92a9e00700 1 -- 172.21.15.157:0/3770703810 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f92ac105610 msgr2=0x7f92ac19aa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:21:05.244 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-21T10:21:05.243+0000 7f92a9e00700 1 --2- 172.21.15.157:0/3770703810 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f92ac105610 0x7f92ac19aa90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:21:05.244 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-21T10:21:05.243+0000 7f92a9e00700 1 -- 172.21.15.157:0/3770703810 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f92ac105fb0 msgr2=0x7f92ac1a1b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:21:05.244 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-21T10:21:05.243+0000 7f92a9e00700 1 --2- 172.21.15.157:0/3770703810 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f92ac105fb0 0x7f92ac1a1b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:21:05.245 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-21T10:21:05.243+0000 7f92a9e00700 1 -- 172.21.15.157:0/3770703810 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f92ac1a21b0 con 0x7f92ac104410 2024-03-21T10:21:05.245 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-21T10:21:05.243+0000 7f92aa800700 1 --2- 172.21.15.157:0/3770703810 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f92ac105fb0 0x7f92ac1a1b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T10:21:05.245 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-21T10:21:05.243+0000 7f92a9e00700 1 --2- 172.21.15.157:0/3770703810 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f92ac104410 0x7f92ac19a550 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f929c00b530 tx=0x7f929c00b8f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:21:05.245 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-21T10:21:05.243+0000 7f92a2a00700 1 -- 172.21.15.157:0/3770703810 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f929c015020 con 0x7f92ac104410 2024-03-21T10:21:05.245 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-21T10:21:05.243+0000 7f92a2a00700 1 -- 172.21.15.157:0/3770703810 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f929c011ce0 con 0x7f92ac104410 2024-03-21T10:21:05.245 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-21T10:21:05.243+0000 7f92a2a00700 1 -- 172.21.15.157:0/3770703810 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f929c010d60 con 0x7f92ac104410 2024-03-21T10:21:05.246 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-21T10:21:05.243+0000 7f92b1200700 1 -- 172.21.15.157:0/3770703810 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f92ac1a24a0 con 0x7f92ac104410 2024-03-21T10:21:05.247 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-21T10:21:05.244+0000 7f92b1200700 1 -- 172.21.15.157:0/3770703810 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f92ac1a2a60 con 0x7f92ac104410 2024-03-21T10:21:05.247 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-21T10:21:05.245+0000 7f92b1200700 1 -- 172.21.15.157:0/3770703810 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f92ac068e50 con 0x7f92ac104410 2024-03-21T10:21:05.247 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-21T10:21:05.247+0000 7f92a2a00700 1 -- 172.21.15.157:0/3770703810 <== mon.1 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 16) v1 ==== 137854+0+0 (secure 0 0 0) 0x7f929c018020 con 0x7f92ac104410 2024-03-21T10:21:05.248 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-21T10:21:05.248+0000 7f92a2a00700 1 --2- 172.21.15.157:0/3770703810 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f92900a2fb0 0x7f92900a5470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:21:05.248 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-21T10:21:05.248+0000 7f92a2a00700 1 -- 172.21.15.157:0/3770703810 <== mon.1 v2:172.21.15.157:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f929c0ce7e0 con 0x7f92ac104410 2024-03-21T10:21:05.249 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-21T10:21:05.249+0000 7f92a9400700 1 --2- 172.21.15.157:0/3770703810 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f92900a2fb0 0x7f92900a5470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:21:05.250 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-21T10:21:05.250+0000 7f92a9400700 1 --2- 172.21.15.157:0/3770703810 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f92900a2fb0 0x7f92900a5470 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f9298006010 tx=0x7f92980038e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:21:05.254 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-21T10:21:05.253+0000 7f92a2a00700 1 -- 172.21.15.157:0/3770703810 <== mon.1 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f929c01f020 con 0x7f92ac104410 2024-03-21T10:21:05.484 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-21T10:21:05.483+0000 7f92b1200700 1 -- 172.21.15.157:0/3770703810 --> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] -- mgr_command(tid 0: {"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi047=a;smithi047=d;smithi047=g;smithi047=j;smithi100=b;smithi100=e;smithi100=h;smithi100=k;smithi157=c;smithi157=f;smithi157=i;smithi157=l", "target": ["mon-mgr", ""]}) v1 -- 0x7f92ac0626d0 con 0x7f92900a2fb0 2024-03-21T10:21:05.502 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-21T10:21:05.502+0000 7f92a2a00700 1 -- 172.21.15.157:0/3770703810 <== mgr.14152 v2:172.21.15.47:6800/2823885994 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+28 (secure 0 0 0) 0x7f92ac0626d0 con 0x7f92900a2fb0 2024-03-21T10:21:05.503 INFO:teuthology.orchestra.run.smithi157.stdout:Scheduled mds.all update... 2024-03-21T10:21:05.505 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-21T10:21:05.505+0000 7f92b1200700 1 -- 172.21.15.157:0/3770703810 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f92900a2fb0 msgr2=0x7f92900a5470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:21:05.506 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-21T10:21:05.505+0000 7f92b1200700 1 --2- 172.21.15.157:0/3770703810 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f92900a2fb0 0x7f92900a5470 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f9298006010 tx=0x7f92980038e0 comp rx=0 tx=0).stop 2024-03-21T10:21:05.506 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-21T10:21:05.505+0000 7f92b1200700 1 -- 172.21.15.157:0/3770703810 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f92ac104410 msgr2=0x7f92ac19a550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:21:05.506 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-21T10:21:05.505+0000 7f92b1200700 1 --2- 172.21.15.157:0/3770703810 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f92ac104410 0x7f92ac19a550 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f929c00b530 tx=0x7f929c00b8f0 comp rx=0 tx=0).stop 2024-03-21T10:21:05.506 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-21T10:21:05.505+0000 7f92b1200700 1 -- 172.21.15.157:0/3770703810 shutdown_connections 2024-03-21T10:21:05.506 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-21T10:21:05.505+0000 7f92b1200700 1 --2- 172.21.15.157:0/3770703810 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f92900a2fb0 0x7f92900a5470 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:21:05.506 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-21T10:21:05.505+0000 7f92b1200700 1 --2- 172.21.15.157:0/3770703810 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f92ac105fb0 0x7f92ac1a1b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:21:05.506 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-21T10:21:05.505+0000 7f92b1200700 1 --2- 172.21.15.157:0/3770703810 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f92ac104410 0x7f92ac19a550 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:21:05.507 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-21T10:21:05.505+0000 7f92b1200700 1 --2- 172.21.15.157:0/3770703810 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f92ac105610 0x7f92ac19aa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:21:05.507 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-21T10:21:05.505+0000 7f92b1200700 1 -- 172.21.15.157:0/3770703810 >> 172.21.15.157:0/3770703810 conn(0x7f92ac0ff9b0 msgr2=0x7f92ac1088b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:21:05.507 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-21T10:21:05.505+0000 7f92b1200700 1 -- 172.21.15.157:0/3770703810 shutdown_connections 2024-03-21T10:21:05.507 INFO:teuthology.orchestra.run.smithi157.stderr:2024-03-21T10:21:05.505+0000 7f92b1200700 1 -- 172.21.15.157:0/3770703810 wait complete. 2024-03-21T10:21:06.205 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:21:05 smithi157 ceph-mon[35581]: pgmap v332: 1 pgs: 1 active+clean; 577 KiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-21T10:21:06.205 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:21:05 smithi157 ceph-mon[35581]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:21:06.205 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:21:05 smithi157 ceph-mon[35581]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T10:21:06.205 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:21:05 smithi157 ceph-mon[35581]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T10:21:06.205 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:21:05 smithi157 ceph-mon[35581]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T10:21:06.205 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:21:05 smithi157 ceph-mon[35581]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:21:06.205 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:21:05 smithi157 ceph-mon[35581]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-21T10:21:06.206 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:21:05 smithi157 ceph-mon[35581]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-21T10:21:06.206 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:21:05 smithi157 ceph-mon[35581]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T10:21:06.214 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:21:05 smithi047 ceph-mon[31547]: pgmap v332: 1 pgs: 1 active+clean; 577 KiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-21T10:21:06.214 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:21:05 smithi047 ceph-mon[31547]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:21:06.214 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:21:05 smithi047 ceph-mon[31547]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T10:21:06.214 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:21:05 smithi047 ceph-mon[31547]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T10:21:06.214 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:21:05 smithi047 ceph-mon[31547]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T10:21:06.215 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:21:05 smithi047 ceph-mon[31547]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:21:06.215 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:21:05 smithi047 ceph-mon[31547]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-21T10:21:06.215 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:21:05 smithi047 ceph-mon[31547]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-21T10:21:06.215 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:21:05 smithi047 ceph-mon[31547]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T10:21:06.229 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:21:05 smithi100 ceph-mon[35193]: pgmap v332: 1 pgs: 1 active+clean; 577 KiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-21T10:21:06.230 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:21:05 smithi100 ceph-mon[35193]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:21:06.230 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:21:05 smithi100 ceph-mon[35193]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T10:21:06.230 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:21:05 smithi100 ceph-mon[35193]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T10:21:06.230 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:21:05 smithi100 ceph-mon[35193]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T10:21:06.230 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:21:05 smithi100 ceph-mon[35193]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:21:06.230 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:21:05 smithi100 ceph-mon[35193]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-21T10:21:06.230 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:21:05 smithi100 ceph-mon[35193]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-21T10:21:06.230 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:21:05 smithi100 ceph-mon[35193]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T10:21:06.256 DEBUG:teuthology.orchestra.run.smithi047:mds.a> sudo journalctl -f -n 0 -u ceph-3056d51a-e76b-11ee-95cd-87774f69a715@mds.a.service 2024-03-21T10:21:06.264 DEBUG:teuthology.orchestra.run.smithi047:mds.d> sudo journalctl -f -n 0 -u ceph-3056d51a-e76b-11ee-95cd-87774f69a715@mds.d.service 2024-03-21T10:21:06.310 DEBUG:teuthology.orchestra.run.smithi047:mds.g> sudo journalctl -f -n 0 -u ceph-3056d51a-e76b-11ee-95cd-87774f69a715@mds.g.service 2024-03-21T10:21:06.357 DEBUG:teuthology.orchestra.run.smithi047:mds.j> sudo journalctl -f -n 0 -u ceph-3056d51a-e76b-11ee-95cd-87774f69a715@mds.j.service 2024-03-21T10:21:06.402 DEBUG:teuthology.orchestra.run.smithi100:mds.b> sudo journalctl -f -n 0 -u ceph-3056d51a-e76b-11ee-95cd-87774f69a715@mds.b.service 2024-03-21T10:21:06.410 DEBUG:teuthology.orchestra.run.smithi100:mds.e> sudo journalctl -f -n 0 -u ceph-3056d51a-e76b-11ee-95cd-87774f69a715@mds.e.service 2024-03-21T10:21:06.456 DEBUG:teuthology.orchestra.run.smithi100:mds.h> sudo journalctl -f -n 0 -u ceph-3056d51a-e76b-11ee-95cd-87774f69a715@mds.h.service 2024-03-21T10:21:06.502 DEBUG:teuthology.orchestra.run.smithi100:mds.k> sudo journalctl -f -n 0 -u ceph-3056d51a-e76b-11ee-95cd-87774f69a715@mds.k.service 2024-03-21T10:21:06.548 DEBUG:teuthology.orchestra.run.smithi157:mds.c> sudo journalctl -f -n 0 -u ceph-3056d51a-e76b-11ee-95cd-87774f69a715@mds.c.service 2024-03-21T10:21:06.554 DEBUG:teuthology.orchestra.run.smithi157:mds.f> sudo journalctl -f -n 0 -u ceph-3056d51a-e76b-11ee-95cd-87774f69a715@mds.f.service 2024-03-21T10:21:06.600 DEBUG:teuthology.orchestra.run.smithi157:mds.i> sudo journalctl -f -n 0 -u ceph-3056d51a-e76b-11ee-95cd-87774f69a715@mds.i.service 2024-03-21T10:21:06.646 DEBUG:teuthology.orchestra.run.smithi157:mds.l> sudo journalctl -f -n 0 -u ceph-3056d51a-e76b-11ee-95cd-87774f69a715@mds.l.service 2024-03-21T10:21:06.692 INFO:tasks.cephadm:Setting up CephFS filesystem(s)... 2024-03-21T10:21:06.692 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-21T10:21:06.834 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:21:06 smithi157 ceph-mon[35581]: from='client.24385 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi047=a;smithi047=d;smithi047=g;smithi047=j;smithi100=b;smithi100=e;smithi100=h;smithi100=k;smithi157=c;smithi157=f;smithi157=i;smithi157=l", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:21:06.834 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:21:06 smithi157 ceph-mon[35581]: Saving service mds.all spec with placement smithi047=a;smithi047=d;smithi047=g;smithi047=j;smithi100=b;smithi100=e;smithi100=h;smithi100=k;smithi157=c;smithi157=f;smithi157=i;smithi157=l;count:12 2024-03-21T10:21:06.834 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:21:06 smithi157 ceph-mon[35581]: Deploying daemon mds.l on smithi157 2024-03-21T10:21:06.944 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:06.941+0000 7f7b4e400640 1 -- 172.21.15.47:0/379101404 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7b481026f0 msgr2=0x7f7b48102b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:21:06.944 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:06.941+0000 7f7b4e400640 1 --2- 172.21.15.47:0/379101404 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7b481026f0 0x7f7b48102b70 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f7b34009900 tx=0x7f7b3402bc30 comp rx=0 tx=0).stop 2024-03-21T10:21:06.944 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:06.941+0000 7f7b4e400640 1 -- 172.21.15.47:0/379101404 shutdown_connections 2024-03-21T10:21:06.944 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:06.941+0000 7f7b4e400640 1 --2- 172.21.15.47:0/379101404 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7b481030b0 0x7f7b48109940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:21:06.944 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:06.941+0000 7f7b4e400640 1 --2- 172.21.15.47:0/379101404 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7b481026f0 0x7f7b48102b70 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:21:06.944 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:06.941+0000 7f7b4e400640 1 --2- 172.21.15.47:0/379101404 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7b48101500 0x7f7b48101920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:21:06.944 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:06.941+0000 7f7b4e400640 1 -- 172.21.15.47:0/379101404 >> 172.21.15.47:0/379101404 conn(0x7f7b480fca70 msgr2=0x7f7b480feeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:21:06.944 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:06.942+0000 7f7b4e400640 1 -- 172.21.15.47:0/379101404 shutdown_connections 2024-03-21T10:21:06.944 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:06.942+0000 7f7b4e400640 1 -- 172.21.15.47:0/379101404 wait complete. 2024-03-21T10:21:06.946 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:06.942+0000 7f7b4e400640 1 Processor -- start 2024-03-21T10:21:06.947 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:06.943+0000 7f7b4e400640 1 -- start start 2024-03-21T10:21:06.947 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:06.943+0000 7f7b4e400640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7b48101500 0x7f7b48197750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:21:06.947 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:06.943+0000 7f7b4e400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7b481026f0 0x7f7b48197c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:21:06.947 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:06.943+0000 7f7b4e400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7b481030b0 0x7f7b4819ed00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:21:06.947 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:06.943+0000 7f7b4e400640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b4810d0d0 con 0x7f7b481030b0 2024-03-21T10:21:06.947 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:06.943+0000 7f7b4e400640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b4810cf50 con 0x7f7b481026f0 2024-03-21T10:21:06.947 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:06.943+0000 7f7b4e400640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b4810d250 con 0x7f7b48101500 2024-03-21T10:21:06.947 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:06.943+0000 7f7b4da00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7b481030b0 0x7f7b4819ed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:21:06.947 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:06.943+0000 7f7b4da00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7b481030b0 0x7f7b4819ed00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:53452/0 (socket says 172.21.15.47:53452) 2024-03-21T10:21:06.947 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:06.943+0000 7f7b4da00640 1 -- 172.21.15.47:0/2153494057 learned_addr learned my addr 172.21.15.47:0/2153494057 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:21:06.947 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:06.944+0000 7f7b4d000640 1 --2- 172.21.15.47:0/2153494057 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7b48101500 0x7f7b48197750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:21:06.948 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:06.944+0000 7f7b47e00640 1 --2- 172.21.15.47:0/2153494057 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7b481026f0 0x7f7b48197c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:21:06.948 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:06.944+0000 7f7b4da00640 1 -- 172.21.15.47:0/2153494057 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7b48101500 msgr2=0x7f7b48197750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:21:06.948 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:06.944+0000 7f7b4da00640 1 --2- 172.21.15.47:0/2153494057 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7b48101500 0x7f7b48197750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:21:06.948 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:06.944+0000 7f7b4da00640 1 -- 172.21.15.47:0/2153494057 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7b481026f0 msgr2=0x7f7b48197c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:21:06.948 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:06.944+0000 7f7b4da00640 1 --2- 172.21.15.47:0/2153494057 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7b481026f0 0x7f7b48197c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:21:06.948 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:06.944+0000 7f7b4da00640 1 -- 172.21.15.47:0/2153494057 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b4819f370 con 0x7f7b481030b0 2024-03-21T10:21:06.948 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:06.944+0000 7f7b4da00640 1 --2- 172.21.15.47:0/2153494057 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7b481030b0 0x7f7b4819ed00 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f7b40009fd0 tx=0x7f7b4000b8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:21:06.948 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:06.944+0000 7f7b45600640 1 -- 172.21.15.47:0/2153494057 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b40011830 con 0x7f7b481030b0 2024-03-21T10:21:06.948 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:06.945+0000 7f7b4e400640 1 -- 172.21.15.47:0/2153494057 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7b4819f660 con 0x7f7b481030b0 2024-03-21T10:21:06.948 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:06.944+0000 7f7b45600640 1 -- 172.21.15.47:0/2153494057 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f7b40016070 con 0x7f7b481030b0 2024-03-21T10:21:06.948 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:06.945+0000 7f7b45600640 1 -- 172.21.15.47:0/2153494057 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b40010de0 con 0x7f7b481030b0 2024-03-21T10:21:06.948 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:06.945+0000 7f7b4e400640 1 -- 172.21.15.47:0/2153494057 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7b4819fc30 con 0x7f7b481030b0 2024-03-21T10:21:06.949 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:06.946+0000 7f7b4e400640 1 -- 172.21.15.47:0/2153494057 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7b48108820 con 0x7f7b481030b0 2024-03-21T10:21:06.950 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:06.948+0000 7f7b45600640 1 -- 172.21.15.47:0/2153494057 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 16) v1 ==== 137854+0+0 (secure 0 0 0) 0x7f7b400119d0 con 0x7f7b481030b0 2024-03-21T10:21:06.951 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:06.949+0000 7f7b45600640 1 --2- 172.21.15.47:0/2153494057 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f7b2c0a2fa0 0x7f7b2c0a5460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:21:06.951 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:06.950+0000 7f7b45600640 1 -- 172.21.15.47:0/2153494057 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f7b400ce780 con 0x7f7b481030b0 2024-03-21T10:21:06.951 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:06.950+0000 7f7b4d000640 1 --2- 172.21.15.47:0/2153494057 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f7b2c0a2fa0 0x7f7b2c0a5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:21:06.952 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:06.950+0000 7f7b4d000640 1 --2- 172.21.15.47:0/2153494057 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f7b2c0a2fa0 0x7f7b2c0a5460 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f7b3800a300 tx=0x7f7b3800a290 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:21:06.956 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:06.954+0000 7f7b45600640 1 -- 172.21.15.47:0/2153494057 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f7b4009b9b0 con 0x7f7b481030b0 2024-03-21T10:21:07.160 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:07.158+0000 7f7b4e400640 1 -- 172.21.15.47:0/2153494057 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f7b480622a0 con 0x7f7b481030b0 2024-03-21T10:21:07.162 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:07.160+0000 7f7b45600640 1 -- 172.21.15.47:0/2153494057 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v83) v1 ==== 74+0+18444 (secure 0 0 0) 0x7f7b40095e20 con 0x7f7b481030b0 2024-03-21T10:21:07.163 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-21T10:21:07.164 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":83,"fsid":"3056d51a-e76b-11ee-95cd-87774f69a715","created":"2024-03-21T10:11:17.019965+0000","modified":"2024-03-21T10:20:53.966317+0000","last_up_change":"2024-03-21T10:20:52.675328+0000","last_in_change":"2024-03-21T10:20:27.648212+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-21T10:15:40.553114+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b0c7a102-8d89-428b-b793-cbbe67151f50","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6802","nonce":775837313},{"type":"v1","addr":"172.21.15.47:6803","nonce":775837313}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6804","nonce":775837313},{"type":"v1","addr":"172.21.15.47:6805","nonce":775837313}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6808","nonce":775837313},{"type":"v1","addr":"172.21.15.47:6809","nonce":775837313}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6806","nonce":775837313},{"type":"v1","addr":"172.21.15.47:6807","nonce":775837313}]},"public_addr":"172.21.15.47:6803/775837313","cluster_addr":"172.21.15.47:6805/775837313","heartbeat_back_addr":"172.21.15.47:6809/775837313","heartbeat_front_addr":"172.21.15.47:6807/775837313","state":["exists","up"]},{"osd":1,"uuid":"07bb81bb-7cf3-4bbc-ac9e-802f95185d11","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6800","nonce":2923304710},{"type":"v1","addr":"172.21.15.100:6801","nonce":2923304710}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":2923304710},{"type":"v1","addr":"172.21.15.100:6803","nonce":2923304710}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":2923304710},{"type":"v1","addr":"172.21.15.100:6807","nonce":2923304710}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6804","nonce":2923304710},{"type":"v1","addr":"172.21.15.100:6805","nonce":2923304710}]},"public_addr":"172.21.15.100:6801/2923304710","cluster_addr":"172.21.15.100:6803/2923304710","heartbeat_back_addr":"172.21.15.100:6807/2923304710","heartbeat_front_addr":"172.21.15.100:6805/2923304710","state":["exists","up"]},{"osd":2,"uuid":"88ce76b9-6081-4836-aad0-8c249c56e8a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":171377005},{"type":"v1","addr":"172.21.15.157:6801","nonce":171377005}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":171377005},{"type":"v1","addr":"172.21.15.157:6803","nonce":171377005}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":171377005},{"type":"v1","addr":"172.21.15.157:6807","nonce":171377005}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":171377005},{"type":"v1","addr":"172.21.15.157:6805","nonce":171377005}]},"public_addr":"172.21.15.157:6801/171377005","cluster_addr":"172.21.15.157:6803/171377005","heartbeat_back_addr":"172.21.15.157:6807/171377005","heartbeat_front_addr":"172.21.15.157:6805/171377005","state":["exists","up"]},{"osd":3,"uuid":"518aa33d-263f-45e6-aab5-2201ceb40e74","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":27,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6810","nonce":640760101},{"type":"v1","addr":"172.21.15.47:6811","nonce":640760101}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6812","nonce":640760101},{"type":"v1","addr":"172.21.15.47:6813","nonce":640760101}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6816","nonce":640760101},{"type":"v1","addr":"172.21.15.47:6817","nonce":640760101}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6814","nonce":640760101},{"type":"v1","addr":"172.21.15.47:6815","nonce":640760101}]},"public_addr":"172.21.15.47:6811/640760101","cluster_addr":"172.21.15.47:6813/640760101","heartbeat_back_addr":"172.21.15.47:6817/640760101","heartbeat_front_addr":"172.21.15.47:6815/640760101","state":["exists","up"]},{"osd":4,"uuid":"987bde83-ecc6-42e5-bd56-c143b3de4226","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":35,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6808","nonce":2785478639},{"type":"v1","addr":"172.21.15.100:6809","nonce":2785478639}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6810","nonce":2785478639},{"type":"v1","addr":"172.21.15.100:6811","nonce":2785478639}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6814","nonce":2785478639},{"type":"v1","addr":"172.21.15.100:6815","nonce":2785478639}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":2785478639},{"type":"v1","addr":"172.21.15.100:6813","nonce":2785478639}]},"public_addr":"172.21.15.100:6809/2785478639","cluster_addr":"172.21.15.100:6811/2785478639","heartbeat_back_addr":"172.21.15.100:6815/2785478639","heartbeat_front_addr":"172.21.15.100:6813/2785478639","state":["exists","up"]},{"osd":5,"uuid":"012d2188-3a61-4a5f-8d01-2001c15ffbe2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":40,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":1525645410},{"type":"v1","addr":"172.21.15.157:6809","nonce":1525645410}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":1525645410},{"type":"v1","addr":"172.21.15.157:6811","nonce":1525645410}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":1525645410},{"type":"v1","addr":"172.21.15.157:6815","nonce":1525645410}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":1525645410},{"type":"v1","addr":"172.21.15.157:6813","nonce":1525645410}]},"public_addr":"172.21.15.157:6809/1525645410","cluster_addr":"172.21.15.157:6811/1525645410","heartbeat_back_addr":"172.21.15.157:6815/1525645410","heartbeat_front_addr":"172.21.15.157:6813/1525645410","state":["exists","up"]},{"osd":6,"uuid":"c12c14b4-85f0-4d88-937c-de4f556437de","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6818","nonce":2321642790},{"type":"v1","addr":"172.21.15.47:6819","nonce":2321642790}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6820","nonce":2321642790},{"type":"v1","addr":"172.21.15.47:6821","nonce":2321642790}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6824","nonce":2321642790},{"type":"v1","addr":"172.21.15.47:6825","nonce":2321642790}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6822","nonce":2321642790},{"type":"v1","addr":"172.21.15.47:6823","nonce":2321642790}]},"public_addr":"172.21.15.47:6819/2321642790","cluster_addr":"172.21.15.47:6821/2321642790","heartbeat_back_addr":"172.21.15.47:6825/2321642790","heartbeat_front_addr":"172.21.15.47:6823/2321642790","state":["exists","up"]},{"osd":7,"uuid":"3c87c6d0-1d1f-4294-9917-aaaad7dda824","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6816","nonce":4104727330},{"type":"v1","addr":"172.21.15.100:6817","nonce":4104727330}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":4104727330},{"type":"v1","addr":"172.21.15.100:6819","nonce":4104727330}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":4104727330},{"type":"v1","addr":"172.21.15.100:6823","nonce":4104727330}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6820","nonce":4104727330},{"type":"v1","addr":"172.21.15.100:6821","nonce":4104727330}]},"public_addr":"172.21.15.100:6817/4104727330","cluster_addr":"172.21.15.100:6819/4104727330","heartbeat_back_addr":"172.21.15.100:6823/4104727330","heartbeat_front_addr":"172.21.15.100:6821/4104727330","state":["exists","up"]},{"osd":8,"uuid":"502d65cf-53c8-4936-8de4-e11b2a6e1e18","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":61,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":4067715347},{"type":"v1","addr":"172.21.15.157:6817","nonce":4067715347}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":4067715347},{"type":"v1","addr":"172.21.15.157:6819","nonce":4067715347}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":4067715347},{"type":"v1","addr":"172.21.15.157:6823","nonce":4067715347}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":4067715347},{"type":"v1","addr":"172.21.15.157:6821","nonce":4067715347}]},"public_addr":"172.21.15.157:6817/4067715347","cluster_addr":"172.21.15.157:6819/4067715347","heartbeat_back_addr":"172.21.15.157:6823/4067715347","heartbeat_front_addr":"172.21.15.157:6821/4067715347","state":["exists","up"]},{"osd":9,"uuid":"5ea1f5be-8458-4c93-9e61-c2e867a514f3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6826","nonce":1982108738},{"type":"v1","addr":"172.21.15.47:6827","nonce":1982108738}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6828","nonce":1982108738},{"type":"v1","addr":"172.21.15.47:6829","nonce":1982108738}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6832","nonce":1982108738},{"type":"v1","addr":"172.21.15.47:6833","nonce":1982108738}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6830","nonce":1982108738},{"type":"v1","addr":"172.21.15.47:6831","nonce":1982108738}]},"public_addr":"172.21.15.47:6827/1982108738","cluster_addr":"172.21.15.47:6829/1982108738","heartbeat_back_addr":"172.21.15.47:6833/1982108738","heartbeat_front_addr":"172.21.15.47:6831/1982108738","state":["exists","up"]},{"osd":10,"uuid":"2da3da90-a0be-4a54-a9ff-b2d6c9b36870","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":74,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":396391028},{"type":"v1","addr":"172.21.15.100:6825","nonce":396391028}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":396391028},{"type":"v1","addr":"172.21.15.100:6827","nonce":396391028}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6830","nonce":396391028},{"type":"v1","addr":"172.21.15.100:6831","nonce":396391028}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":396391028},{"type":"v1","addr":"172.21.15.100:6829","nonce":396391028}]},"public_addr":"172.21.15.100:6825/396391028","cluster_addr":"172.21.15.100:6827/396391028","heartbeat_back_addr":"172.21.15.100:6831/396391028","heartbeat_front_addr":"172.21.15.100:6829/396391028","state":["exists","up"]},{"osd":11,"uuid":"11798d41-7f7e-4319-95b8-8d3d0946bde9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":82,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":2671029483},{"type":"v1","addr":"172.21.15.157:6825","nonce":2671029483}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":2671029483},{"type":"v1","addr":"172.21.15.157:6827","nonce":2671029483}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":2671029483},{"type":"v1","addr":"172.21.15.157:6831","nonce":2671029483}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":2671029483},{"type":"v1","addr":"172.21.15.157:6829","nonce":2671029483}]},"public_addr":"172.21.15.157:6825/2671029483","cluster_addr":"172.21.15.157:6827/2671029483","heartbeat_back_addr":"172.21.15.157:6831/2671029483","heartbeat_front_addr":"172.21.15.157:6829/2671029483","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T10:14:31.822399+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T10:15:04.175721+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T10:15:35.902595+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T10:16:12.088891+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T10:16:46.428704+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T10:17:20.158270+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T10:17:55.915253+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T10:18:30.064641+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T10:19:04.410004+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T10:19:40.605181+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T10:20:15.636605+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T10:20:50.151365+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.47:0/356822431":"2024-03-22T10:12:25.844530+0000","172.21.15.47:0/1167490879":"2024-03-22T10:12:25.844530+0000","172.21.15.47:0/1540718138":"2024-03-22T10:12:25.844530+0000","172.21.15.47:6801/3836903972":"2024-03-22T10:12:25.844530+0000","172.21.15.47:0/235436602":"2024-03-22T10:11:50.301838+0000","172.21.15.47:0/2052815572":"2024-03-22T10:11:50.301838+0000","172.21.15.47:6800/3942658432":"2024-03-22T10:11:50.301838+0000","172.21.15.47:6801/3942658432":"2024-03-22T10:11:50.301838+0000","172.21.15.47:0/1419516257":"2024-03-22T10:11:50.301838+0000","172.21.15.47:6800/3836903972":"2024-03-22T10:12:25.844530+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-21T10:21:07.166 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:07.163+0000 7f7b4e400640 1 -- 172.21.15.47:0/2153494057 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f7b2c0a2fa0 msgr2=0x7f7b2c0a5460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:21:07.166 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:07.163+0000 7f7b4e400640 1 --2- 172.21.15.47:0/2153494057 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f7b2c0a2fa0 0x7f7b2c0a5460 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f7b3800a300 tx=0x7f7b3800a290 comp rx=0 tx=0).stop 2024-03-21T10:21:07.166 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:07.163+0000 7f7b4e400640 1 -- 172.21.15.47:0/2153494057 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7b481030b0 msgr2=0x7f7b4819ed00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:21:07.166 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:07.163+0000 7f7b4e400640 1 --2- 172.21.15.47:0/2153494057 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7b481030b0 0x7f7b4819ed00 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f7b40009fd0 tx=0x7f7b4000b8e0 comp rx=0 tx=0).stop 2024-03-21T10:21:07.166 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:07.164+0000 7f7b4e400640 1 -- 172.21.15.47:0/2153494057 shutdown_connections 2024-03-21T10:21:07.166 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:07.164+0000 7f7b4e400640 1 --2- 172.21.15.47:0/2153494057 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f7b2c0a2fa0 0x7f7b2c0a5460 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:21:07.166 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:07.164+0000 7f7b4e400640 1 --2- 172.21.15.47:0/2153494057 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7b481030b0 0x7f7b4819ed00 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:21:07.166 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:07.164+0000 7f7b4e400640 1 --2- 172.21.15.47:0/2153494057 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7b481026f0 0x7f7b48197c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:21:07.167 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:07.164+0000 7f7b4e400640 1 --2- 172.21.15.47:0/2153494057 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7b48101500 0x7f7b48197750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:21:07.167 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:07.164+0000 7f7b4e400640 1 -- 172.21.15.47:0/2153494057 >> 172.21.15.47:0/2153494057 conn(0x7f7b480fca70 msgr2=0x7f7b481059f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:21:07.167 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:07.164+0000 7f7b4e400640 1 -- 172.21.15.47:0/2153494057 shutdown_connections 2024-03-21T10:21:07.167 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:07.164+0000 7f7b4e400640 1 -- 172.21.15.47:0/2153494057 wait complete. 2024-03-21T10:21:07.183 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:21:06 smithi047 ceph-mon[31547]: from='client.24385 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi047=a;smithi047=d;smithi047=g;smithi047=j;smithi100=b;smithi100=e;smithi100=h;smithi100=k;smithi157=c;smithi157=f;smithi157=i;smithi157=l", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:21:07.183 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:21:06 smithi047 ceph-mon[31547]: Saving service mds.all spec with placement smithi047=a;smithi047=d;smithi047=g;smithi047=j;smithi100=b;smithi100=e;smithi100=h;smithi100=k;smithi157=c;smithi157=f;smithi157=i;smithi157=l;count:12 2024-03-21T10:21:07.183 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:21:06 smithi047 ceph-mon[31547]: Deploying daemon mds.l on smithi157 2024-03-21T10:21:07.185 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-21T10:15:40.553114+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-21T10:21:07.186 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-21T10:21:07.229 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:21:06 smithi100 ceph-mon[35193]: from='client.24385 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi047=a;smithi047=d;smithi047=g;smithi047=j;smithi100=b;smithi100=e;smithi100=h;smithi100=k;smithi157=c;smithi157=f;smithi157=i;smithi157=l", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:21:07.230 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:21:06 smithi100 ceph-mon[35193]: Saving service mds.all spec with placement smithi047=a;smithi047=d;smithi047=g;smithi047=j;smithi100=b;smithi100=e;smithi100=h;smithi100=k;smithi157=c;smithi157=f;smithi157=i;smithi157=l;count:12 2024-03-21T10:21:07.230 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:21:06 smithi100 ceph-mon[35193]: Deploying daemon mds.l on smithi157 2024-03-21T10:21:07.414 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:07.411+0000 7f26fee00640 1 -- 172.21.15.47:0/2744892935 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f26f8102000 msgr2=0x7f26f81043f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:21:07.414 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:07.411+0000 7f26fee00640 1 --2- 172.21.15.47:0/2744892935 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f26f8102000 0x7f26f81043f0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f26ec009900 tx=0x7f26ec02bc30 comp rx=0 tx=0).stop 2024-03-21T10:21:07.414 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:07.412+0000 7f26fee00640 1 -- 172.21.15.47:0/2744892935 shutdown_connections 2024-03-21T10:21:07.415 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:07.412+0000 7f26fee00640 1 --2- 172.21.15.47:0/2744892935 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f26f8107260 0x7f26f81096b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:21:07.415 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:07.412+0000 7f26fee00640 1 --2- 172.21.15.47:0/2744892935 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f26f8104930 0x7f26f8106d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:21:07.415 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:07.412+0000 7f26fee00640 1 --2- 172.21.15.47:0/2744892935 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f26f8102000 0x7f26f81043f0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:21:07.415 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:07.412+0000 7f26fee00640 1 -- 172.21.15.47:0/2744892935 >> 172.21.15.47:0/2744892935 conn(0x7f26f80fba10 msgr2=0x7f26f80fde30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:21:07.415 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:07.412+0000 7f26fee00640 1 -- 172.21.15.47:0/2744892935 shutdown_connections 2024-03-21T10:21:07.415 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:07.412+0000 7f26fee00640 1 -- 172.21.15.47:0/2744892935 wait complete. 2024-03-21T10:21:07.417 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:07.413+0000 7f26fee00640 1 Processor -- start 2024-03-21T10:21:07.417 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:07.413+0000 7f26fee00640 1 -- start start 2024-03-21T10:21:07.417 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:07.413+0000 7f26fee00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f26f8102000 0x7f26f81977c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:21:07.417 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:07.413+0000 7f26fee00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f26f8104930 0x7f26f8197d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:21:07.417 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:07.413+0000 7f26fee00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f26f8107260 0x7f26f819ed70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:21:07.417 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:07.413+0000 7f26fee00640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26f810d110 con 0x7f26f8104930 2024-03-21T10:21:07.417 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:07.413+0000 7f26fee00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26f810cf90 con 0x7f26f8107260 2024-03-21T10:21:07.417 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:07.413+0000 7f26fee00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26f810d290 con 0x7f26f8102000 2024-03-21T10:21:07.417 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:07.414+0000 7f26fd000640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f26f8104930 0x7f26f8197d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:21:07.417 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:07.414+0000 7f26fd000640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f26f8104930 0x7f26f8197d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:53470/0 (socket says 172.21.15.47:53470) 2024-03-21T10:21:07.417 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:07.414+0000 7f26fd000640 1 -- 172.21.15.47:0/802792246 learned_addr learned my addr 172.21.15.47:0/802792246 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:21:07.418 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:07.414+0000 7f26fda00640 1 --2- 172.21.15.47:0/802792246 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f26f8102000 0x7f26f81977c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:21:07.418 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:07.414+0000 7f26fe400640 1 --2- 172.21.15.47:0/802792246 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f26f8107260 0x7f26f819ed70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:21:07.418 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:07.414+0000 7f26fd000640 1 -- 172.21.15.47:0/802792246 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f26f8102000 msgr2=0x7f26f81977c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:21:07.418 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:07.414+0000 7f26fd000640 1 --2- 172.21.15.47:0/802792246 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f26f8102000 0x7f26f81977c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:21:07.418 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:07.414+0000 7f26fd000640 1 -- 172.21.15.47:0/802792246 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f26f8107260 msgr2=0x7f26f819ed70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:21:07.418 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:07.414+0000 7f26fd000640 1 --2- 172.21.15.47:0/802792246 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f26f8107260 0x7f26f819ed70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:21:07.418 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:07.414+0000 7f26fd000640 1 -- 172.21.15.47:0/802792246 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f26f819f3e0 con 0x7f26f8104930 2024-03-21T10:21:07.418 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:07.414+0000 7f26fd000640 1 --2- 172.21.15.47:0/802792246 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f26f8104930 0x7f26f8197d00 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f26e400d770 tx=0x7f26e400dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:21:07.418 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:07.415+0000 7f26f6000640 1 -- 172.21.15.47:0/802792246 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26e4017070 con 0x7f26f8104930 2024-03-21T10:21:07.418 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:07.415+0000 7f26fee00640 1 -- 172.21.15.47:0/802792246 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f26f819f6d0 con 0x7f26f8104930 2024-03-21T10:21:07.418 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:07.415+0000 7f26f6000640 1 -- 172.21.15.47:0/802792246 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f26e4009c30 con 0x7f26f8104930 2024-03-21T10:21:07.418 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:07.415+0000 7f26f6000640 1 -- 172.21.15.47:0/802792246 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26e400f620 con 0x7f26f8104930 2024-03-21T10:21:07.418 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:07.415+0000 7f26fee00640 1 -- 172.21.15.47:0/802792246 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f26f819fc20 con 0x7f26f8104930 2024-03-21T10:21:07.419 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:07.416+0000 7f26fee00640 1 -- 172.21.15.47:0/802792246 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f26f80fd500 con 0x7f26f8104930 2024-03-21T10:21:07.420 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:07.418+0000 7f26f6000640 1 -- 172.21.15.47:0/802792246 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 16) v1 ==== 137854+0+0 (secure 0 0 0) 0x7f26e401a070 con 0x7f26f8104930 2024-03-21T10:21:07.421 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:07.419+0000 7f26f6000640 1 --2- 172.21.15.47:0/802792246 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f26e00a2f20 0x7f26e00a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:21:07.422 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:07.420+0000 7f26f6000640 1 -- 172.21.15.47:0/802792246 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f26e40ceb90 con 0x7f26f8104930 2024-03-21T10:21:07.422 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:07.420+0000 7f26fda00640 1 --2- 172.21.15.47:0/802792246 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f26e00a2f20 0x7f26e00a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:21:07.423 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:07.421+0000 7f26fda00640 1 --2- 172.21.15.47:0/802792246 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f26e00a2f20 0x7f26e00a53e0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f26ec006010 tx=0x7f26ec000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:21:07.427 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:07.425+0000 7f26f6000640 1 -- 172.21.15.47:0/802792246 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f26e4009de0 con 0x7f26f8104930 2024-03-21T10:21:07.628 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:07.626+0000 7f26fee00640 1 -- 172.21.15.47:0/802792246 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f26f80622a0 con 0x7f26f8104930 2024-03-21T10:21:07.629 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:07.627+0000 7f26f6000640 1 -- 172.21.15.47:0/802792246 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v83) v1 ==== 93+0+10 (secure 0 0 0) 0x7f26e409bdc0 con 0x7f26f8104930 2024-03-21T10:21:07.629 INFO:teuthology.orchestra.run.smithi047.stdout:pg_num: 1 2024-03-21T10:21:07.632 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:07.629+0000 7f26fee00640 1 -- 172.21.15.47:0/802792246 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f26e00a2f20 msgr2=0x7f26e00a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:21:07.632 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:07.629+0000 7f26fee00640 1 --2- 172.21.15.47:0/802792246 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f26e00a2f20 0x7f26e00a53e0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f26ec006010 tx=0x7f26ec000bc0 comp rx=0 tx=0).stop 2024-03-21T10:21:07.632 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:07.629+0000 7f26fee00640 1 -- 172.21.15.47:0/802792246 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f26f8104930 msgr2=0x7f26f8197d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:21:07.632 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:07.629+0000 7f26fee00640 1 --2- 172.21.15.47:0/802792246 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f26f8104930 0x7f26f8197d00 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f26e400d770 tx=0x7f26e400dc30 comp rx=0 tx=0).stop 2024-03-21T10:21:07.632 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:07.630+0000 7f26fee00640 1 -- 172.21.15.47:0/802792246 shutdown_connections 2024-03-21T10:21:07.632 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:07.630+0000 7f26fee00640 1 --2- 172.21.15.47:0/802792246 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f26e00a2f20 0x7f26e00a53e0 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:21:07.632 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:07.630+0000 7f26fee00640 1 --2- 172.21.15.47:0/802792246 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f26f8107260 0x7f26f819ed70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:21:07.632 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:07.630+0000 7f26fee00640 1 --2- 172.21.15.47:0/802792246 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f26f8104930 0x7f26f8197d00 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:21:07.632 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:07.630+0000 7f26fee00640 1 --2- 172.21.15.47:0/802792246 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f26f8102000 0x7f26f81977c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:21:07.632 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:07.630+0000 7f26fee00640 1 -- 172.21.15.47:0/802792246 >> 172.21.15.47:0/802792246 conn(0x7f26f80fba10 msgr2=0x7f26f8102a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:21:07.633 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:07.630+0000 7f26fee00640 1 -- 172.21.15.47:0/802792246 shutdown_connections 2024-03-21T10:21:07.633 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:07.630+0000 7f26fee00640 1 -- 172.21.15.47:0/802792246 wait complete. 2024-03-21T10:21:07.650 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-21T10:21:07.862 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:07.860+0000 7f01b7000640 1 -- 172.21.15.47:0/3760052199 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f01b01038e0 msgr2=0x7f01b0103d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:21:07.862 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:07.860+0000 7f01b7000640 1 --2- 172.21.15.47:0/3760052199 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f01b01038e0 0x7f01b0103d00 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f01a0009900 tx=0x7f01a002bc30 comp rx=0 tx=0).stop 2024-03-21T10:21:07.863 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:07.860+0000 7f01b7000640 1 -- 172.21.15.47:0/3760052199 shutdown_connections 2024-03-21T10:21:07.863 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:07.860+0000 7f01b7000640 1 --2- 172.21.15.47:0/3760052199 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f01b0105490 0x7f01b010bd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:21:07.863 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:07.860+0000 7f01b7000640 1 --2- 172.21.15.47:0/3760052199 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f01b0104ad0 0x7f01b0104f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:21:07.863 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:07.860+0000 7f01b7000640 1 --2- 172.21.15.47:0/3760052199 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f01b01038e0 0x7f01b0103d00 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:21:07.863 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:07.860+0000 7f01b7000640 1 -- 172.21.15.47:0/3760052199 >> 172.21.15.47:0/3760052199 conn(0x7f01b00fee90 msgr2=0x7f01b01012b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:21:07.863 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:07.860+0000 7f01b7000640 1 -- 172.21.15.47:0/3760052199 shutdown_connections 2024-03-21T10:21:07.863 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:07.860+0000 7f01b7000640 1 -- 172.21.15.47:0/3760052199 wait complete. 2024-03-21T10:21:07.864 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:07.861+0000 7f01b7000640 1 Processor -- start 2024-03-21T10:21:07.864 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:07.861+0000 7f01b7000640 1 -- start start 2024-03-21T10:21:07.864 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:07.862+0000 7f01b7000640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f01b01038e0 0x7f01b0199b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:21:07.864 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:07.862+0000 7f01b7000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f01b0104ad0 0x7f01b019a090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:21:07.864 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:07.862+0000 7f01b7000640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f01b0105490 0x7f01b01a1100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:21:07.864 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:07.862+0000 7f01b7000640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01b010f450 con 0x7f01b0105490 2024-03-21T10:21:07.864 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:07.862+0000 7f01b7000640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01b010f2d0 con 0x7f01b0104ad0 2024-03-21T10:21:07.865 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:07.862+0000 7f01b7000640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01b010f5d0 con 0x7f01b01038e0 2024-03-21T10:21:07.865 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:07.862+0000 7f01b5000640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f01b0105490 0x7f01b01a1100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:21:07.865 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:07.862+0000 7f01b5000640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f01b0105490 0x7f01b01a1100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:53494/0 (socket says 172.21.15.47:53494) 2024-03-21T10:21:07.865 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:07.862+0000 7f01b5000640 1 -- 172.21.15.47:0/2161014818 learned_addr learned my addr 172.21.15.47:0/2161014818 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:21:07.865 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:07.862+0000 7f01afe00640 1 --2- 172.21.15.47:0/2161014818 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f01b01038e0 0x7f01b0199b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:21:07.866 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:07.863+0000 7f01af400640 1 --2- 172.21.15.47:0/2161014818 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f01b0104ad0 0x7f01b019a090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:21:07.866 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:07.863+0000 7f01b5000640 1 -- 172.21.15.47:0/2161014818 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f01b01038e0 msgr2=0x7f01b0199b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:21:07.866 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:07.863+0000 7f01b5000640 1 --2- 172.21.15.47:0/2161014818 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f01b01038e0 0x7f01b0199b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:21:07.866 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:07.863+0000 7f01b5000640 1 -- 172.21.15.47:0/2161014818 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f01b0104ad0 msgr2=0x7f01b019a090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:21:07.866 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:07.863+0000 7f01b5000640 1 --2- 172.21.15.47:0/2161014818 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f01b0104ad0 0x7f01b019a090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:21:07.866 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:07.863+0000 7f01b5000640 1 -- 172.21.15.47:0/2161014818 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01b01a1770 con 0x7f01b0105490 2024-03-21T10:21:07.866 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:07.863+0000 7f01af400640 1 --2- 172.21.15.47:0/2161014818 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f01b0104ad0 0x7f01b019a090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T10:21:07.866 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:07.863+0000 7f01b5000640 1 --2- 172.21.15.47:0/2161014818 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f01b0105490 0x7f01b01a1100 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f01a800eb10 tx=0x7f01a800c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:21:07.867 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:07.863+0000 7f01acc00640 1 -- 172.21.15.47:0/2161014818 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01a8004b20 con 0x7f01b0105490 2024-03-21T10:21:07.867 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:07.863+0000 7f01acc00640 1 -- 172.21.15.47:0/2161014818 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f01a8004cc0 con 0x7f01b0105490 2024-03-21T10:21:07.867 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:07.863+0000 7f01b7000640 1 -- 172.21.15.47:0/2161014818 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f01b01a1a60 con 0x7f01b0105490 2024-03-21T10:21:07.867 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:07.863+0000 7f01acc00640 1 -- 172.21.15.47:0/2161014818 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01a8010690 con 0x7f01b0105490 2024-03-21T10:21:07.867 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:07.863+0000 7f01b7000640 1 -- 172.21.15.47:0/2161014818 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f01b01a1f60 con 0x7f01b0105490 2024-03-21T10:21:07.867 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:07.865+0000 7f01b7000640 1 -- 172.21.15.47:0/2161014818 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f01b010ac70 con 0x7f01b0105490 2024-03-21T10:21:07.869 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:07.867+0000 7f01acc00640 1 -- 172.21.15.47:0/2161014818 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 16) v1 ==== 137854+0+0 (secure 0 0 0) 0x7f01a8005000 con 0x7f01b0105490 2024-03-21T10:21:07.870 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:07.868+0000 7f01acc00640 1 --2- 172.21.15.47:0/2161014818 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f01940a2f20 0x7f01940a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:21:07.870 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:07.868+0000 7f01acc00640 1 -- 172.21.15.47:0/2161014818 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f01a80cf8e0 con 0x7f01b0105490 2024-03-21T10:21:07.870 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:07.868+0000 7f01afe00640 1 --2- 172.21.15.47:0/2161014818 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f01940a2f20 0x7f01940a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:21:07.872 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:07.869+0000 7f01afe00640 1 --2- 172.21.15.47:0/2161014818 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f01940a2f20 0x7f01940a53e0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f01a002c240 tx=0x7f01a0005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:21:07.873 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:07.871+0000 7f01acc00640 1 -- 172.21.15.47:0/2161014818 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f01a8053740 con 0x7f01b0105490 2024-03-21T10:21:07.964 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:21:07 smithi047 ceph-mon[31547]: pgmap v333: 1 pgs: 1 active+clean; 577 KiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-21T10:21:07.964 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:21:07 smithi047 ceph-mon[31547]: from='client.? 172.21.15.47:0/2153494057' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-21T10:21:07.964 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:21:07 smithi047 ceph-mon[31547]: from='client.? 172.21.15.47:0/802792246' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-21T10:21:07.990 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:21:07 smithi157 ceph-mon[35581]: pgmap v333: 1 pgs: 1 active+clean; 577 KiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-21T10:21:07.990 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:21:07 smithi157 ceph-mon[35581]: from='client.? 172.21.15.47:0/2153494057' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-21T10:21:07.990 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:21:07 smithi157 ceph-mon[35581]: from='client.? 172.21.15.47:0/802792246' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-21T10:21:08.116 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:08.114+0000 7f01b7000640 1 -- 172.21.15.47:0/2161014818 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f01b002d070 con 0x7f01b0105490 2024-03-21T10:21:08.117 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:08.115+0000 7f01acc00640 1 -- 172.21.15.47:0/2161014818 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 1 v1) v1 ==== 93+0+535 (secure 0 0 0) 0x7f01a809cb10 con 0x7f01b0105490 2024-03-21T10:21:08.117 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-21T10:21:08.118 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":1,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[]} 2024-03-21T10:21:08.118 INFO:teuthology.orchestra.run.smithi047.stderr:dumped fsmap epoch 1 2024-03-21T10:21:08.120 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:08.118+0000 7f01b7000640 1 -- 172.21.15.47:0/2161014818 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f01940a2f20 msgr2=0x7f01940a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:21:08.120 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:08.118+0000 7f01b7000640 1 --2- 172.21.15.47:0/2161014818 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f01940a2f20 0x7f01940a53e0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f01a002c240 tx=0x7f01a0005fd0 comp rx=0 tx=0).stop 2024-03-21T10:21:08.120 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:08.118+0000 7f01b7000640 1 -- 172.21.15.47:0/2161014818 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f01b0105490 msgr2=0x7f01b01a1100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:21:08.120 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:08.118+0000 7f01b7000640 1 --2- 172.21.15.47:0/2161014818 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f01b0105490 0x7f01b01a1100 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f01a800eb10 tx=0x7f01a800c2d0 comp rx=0 tx=0).stop 2024-03-21T10:21:08.120 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:08.118+0000 7f01b7000640 1 -- 172.21.15.47:0/2161014818 shutdown_connections 2024-03-21T10:21:08.121 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:08.118+0000 7f01b7000640 1 --2- 172.21.15.47:0/2161014818 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f01940a2f20 0x7f01940a53e0 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:21:08.121 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:08.118+0000 7f01b7000640 1 --2- 172.21.15.47:0/2161014818 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f01b0105490 0x7f01b01a1100 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:21:08.121 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:08.118+0000 7f01b7000640 1 --2- 172.21.15.47:0/2161014818 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f01b0104ad0 0x7f01b019a090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:21:08.121 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:08.118+0000 7f01b7000640 1 --2- 172.21.15.47:0/2161014818 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f01b01038e0 0x7f01b0199b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:21:08.121 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:08.118+0000 7f01b7000640 1 -- 172.21.15.47:0/2161014818 >> 172.21.15.47:0/2161014818 conn(0x7f01b00fee90 msgr2=0x7f01b0107e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:21:08.121 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:08.118+0000 7f01b7000640 1 -- 172.21.15.47:0/2161014818 shutdown_connections 2024-03-21T10:21:08.121 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:08.118+0000 7f01b7000640 1 -- 172.21.15.47:0/2161014818 wait complete. 2024-03-21T10:21:08.229 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:21:07 smithi100 ceph-mon[35193]: pgmap v333: 1 pgs: 1 active+clean; 577 KiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-21T10:21:08.229 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:21:07 smithi100 ceph-mon[35193]: from='client.? 172.21.15.47:0/2153494057' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-21T10:21:08.230 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:21:07 smithi100 ceph-mon[35193]: from='client.? 172.21.15.47:0/802792246' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-21T10:21:08.867 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:21:08 smithi157 ceph-mon[35581]: from='client.? 172.21.15.47:0/2161014818' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-21T10:21:09.170 INFO:journalctl@ceph.mds.l.smithi157.stdout:Mar 21 10:21:08 smithi157 podman[65644]: 2024-03-21 10:21:08.889191636 +0000 UTC m=+0.152917262 container create 49f83b9ee10028688a9908fe705d11794e6453616d82420c5ce2eb932178549c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-3056d51a-e76b-11ee-95cd-87774f69a715-mds-l, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240319, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.license=GPLv2, GIT_CLEAN=True, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, GIT_BRANCH=HEAD, ceph=True) 2024-03-21T10:21:09.171 INFO:journalctl@ceph.mds.l.smithi157.stdout:Mar 21 10:21:09 smithi157 podman[65644]: 2024-03-21 10:21:09.168280636 +0000 UTC m=+0.432006268 container init 49f83b9ee10028688a9908fe705d11794e6453616d82420c5ce2eb932178549c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-3056d51a-e76b-11ee-95cd-87774f69a715-mds-l, org.label-schema.license=GPLv2, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, org.label-schema.build-date=20240319) 2024-03-21T10:21:09.214 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:21:08 smithi047 ceph-mon[31547]: from='client.? 172.21.15.47:0/2161014818' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-21T10:21:09.229 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:21:08 smithi100 ceph-mon[35193]: from='client.? 172.21.15.47:0/2161014818' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-21T10:21:09.490 INFO:journalctl@ceph.mds.l.smithi157.stdout:Mar 21 10:21:09 smithi157 podman[65644]: 2024-03-21 10:21:09.178696694 +0000 UTC m=+0.442422292 container start 49f83b9ee10028688a9908fe705d11794e6453616d82420c5ce2eb932178549c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-3056d51a-e76b-11ee-95cd-87774f69a715-mds-l, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.license=GPLv2, org.label-schema.build-date=20240319, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_CLEAN=True, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5) 2024-03-21T10:21:09.490 INFO:journalctl@ceph.mds.l.smithi157.stdout:Mar 21 10:21:09 smithi157 ceph-mds[65663]: -- 172.21.15.157:0/225400369 <== mon.1 v2:172.21.15.157:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x5652c6397a40 con 0x5652c636ac00 2024-03-21T10:21:09.491 INFO:journalctl@ceph.mds.l.smithi157.stdout:Mar 21 10:21:09 smithi157 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-mds-l[65659]: starting mds.l at 2024-03-21T10:21:09.491 INFO:journalctl@ceph.mds.l.smithi157.stdout:Mar 21 10:21:09 smithi157 bash[65644]: 49f83b9ee10028688a9908fe705d11794e6453616d82420c5ce2eb932178549c 2024-03-21T10:21:09.491 INFO:journalctl@ceph.mds.l.smithi157.stdout:Mar 21 10:21:09 smithi157 systemd[1]: Started Ceph mds.l for 3056d51a-e76b-11ee-95cd-87774f69a715. 2024-03-21T10:21:10.140 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-21T10:21:10.214 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:21:09 smithi047 ceph-mon[31547]: pgmap v334: 1 pgs: 1 active+clean; 577 KiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-21T10:21:10.214 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:21:09 smithi047 ceph-mon[31547]: mds.? [v2:172.21.15.157:6832/4000010136,v1:172.21.15.157:6833/4000010136] up:boot 2024-03-21T10:21:10.214 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:21:09 smithi047 ceph-mon[31547]: fsmap 1 up:standby 2024-03-21T10:21:10.214 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:21:09 smithi047 ceph-mon[31547]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-03-21T10:21:10.214 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:21:09 smithi047 ceph-mon[31547]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:21:10.215 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:21:09 smithi047 ceph-mon[31547]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:21:10.215 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:21:09 smithi047 ceph-mon[31547]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:21:10.215 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:21:09 smithi047 ceph-mon[31547]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-21T10:21:10.215 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:21:09 smithi047 ceph-mon[31547]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-21T10:21:10.215 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:21:09 smithi047 ceph-mon[31547]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T10:21:10.229 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:21:09 smithi100 ceph-mon[35193]: pgmap v334: 1 pgs: 1 active+clean; 577 KiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-21T10:21:10.229 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:21:09 smithi100 ceph-mon[35193]: mds.? [v2:172.21.15.157:6832/4000010136,v1:172.21.15.157:6833/4000010136] up:boot 2024-03-21T10:21:10.230 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:21:09 smithi100 ceph-mon[35193]: fsmap 1 up:standby 2024-03-21T10:21:10.230 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:21:09 smithi100 ceph-mon[35193]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-03-21T10:21:10.230 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:21:09 smithi100 ceph-mon[35193]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:21:10.230 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:21:09 smithi100 ceph-mon[35193]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:21:10.230 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:21:09 smithi100 ceph-mon[35193]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:21:10.230 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:21:09 smithi100 ceph-mon[35193]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-21T10:21:10.230 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:21:09 smithi100 ceph-mon[35193]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-21T10:21:10.230 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:21:09 smithi100 ceph-mon[35193]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T10:21:10.240 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:21:09 smithi157 ceph-mon[35581]: pgmap v334: 1 pgs: 1 active+clean; 577 KiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-21T10:21:10.240 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:21:09 smithi157 ceph-mon[35581]: mds.? [v2:172.21.15.157:6832/4000010136,v1:172.21.15.157:6833/4000010136] up:boot 2024-03-21T10:21:10.240 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:21:09 smithi157 ceph-mon[35581]: fsmap 1 up:standby 2024-03-21T10:21:10.240 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:21:09 smithi157 ceph-mon[35581]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-03-21T10:21:10.240 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:21:09 smithi157 ceph-mon[35581]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:21:10.240 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:21:09 smithi157 ceph-mon[35581]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:21:10.240 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:21:09 smithi157 ceph-mon[35581]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:21:10.241 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:21:09 smithi157 ceph-mon[35581]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-21T10:21:10.241 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:21:09 smithi157 ceph-mon[35581]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-21T10:21:10.241 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:21:09 smithi157 ceph-mon[35581]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T10:21:10.377 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:10.375+0000 7f3c4d200640 1 -- 172.21.15.47:0/338127507 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3c481014e0 msgr2=0x7f3c48101900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:21:10.377 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:10.375+0000 7f3c4d200640 1 --2- 172.21.15.47:0/338127507 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3c481014e0 0x7f3c48101900 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f3c38009900 tx=0x7f3c3802bc30 comp rx=0 tx=0).stop 2024-03-21T10:21:10.377 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:10.375+0000 7f3c4d200640 1 -- 172.21.15.47:0/338127507 shutdown_connections 2024-03-21T10:21:10.378 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:10.375+0000 7f3c4d200640 1 --2- 172.21.15.47:0/338127507 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3c48103090 0x7f3c48109920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:21:10.378 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:10.375+0000 7f3c4d200640 1 --2- 172.21.15.47:0/338127507 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3c481026d0 0x7f3c48102b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:21:10.378 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:10.375+0000 7f3c4d200640 1 --2- 172.21.15.47:0/338127507 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3c481014e0 0x7f3c48101900 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:21:10.378 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:10.375+0000 7f3c4d200640 1 -- 172.21.15.47:0/338127507 >> 172.21.15.47:0/338127507 conn(0x7f3c480fca30 msgr2=0x7f3c480fee90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:21:10.378 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:10.375+0000 7f3c4d200640 1 -- 172.21.15.47:0/338127507 shutdown_connections 2024-03-21T10:21:10.378 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:10.375+0000 7f3c4d200640 1 -- 172.21.15.47:0/338127507 wait complete. 2024-03-21T10:21:10.380 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:10.376+0000 7f3c4d200640 1 Processor -- start 2024-03-21T10:21:10.380 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:10.376+0000 7f3c4d200640 1 -- start start 2024-03-21T10:21:10.380 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:10.376+0000 7f3c4d200640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3c481014e0 0x7f3c481999c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:21:10.380 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:10.376+0000 7f3c4d200640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3c481026d0 0x7f3c48199f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:21:10.380 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:10.377+0000 7f3c4d200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3c48103090 0x7f3c481a0f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:21:10.380 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:10.377+0000 7f3c4d200640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c4810d0b0 con 0x7f3c481014e0 2024-03-21T10:21:10.381 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:10.377+0000 7f3c4d200640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c4810cf30 con 0x7f3c48103090 2024-03-21T10:21:10.381 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:10.377+0000 7f3c4d200640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c4810d230 con 0x7f3c481026d0 2024-03-21T10:21:10.381 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:10.377+0000 7f3c47400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3c481014e0 0x7f3c481999c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:21:10.381 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:10.377+0000 7f3c47400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3c481014e0 0x7f3c481999c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:53500/0 (socket says 172.21.15.47:53500) 2024-03-21T10:21:10.381 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:10.377+0000 7f3c47400640 1 -- 172.21.15.47:0/100650765 learned_addr learned my addr 172.21.15.47:0/100650765 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:21:10.381 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:10.377+0000 7f3c3ea00640 1 --2- 172.21.15.47:0/100650765 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3c481026d0 0x7f3c48199f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:21:10.381 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:10.377+0000 7f3c47e00640 1 --2- 172.21.15.47:0/100650765 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3c48103090 0x7f3c481a0f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:21:10.381 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:10.377+0000 7f3c47400640 1 -- 172.21.15.47:0/100650765 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3c481026d0 msgr2=0x7f3c48199f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:21:10.381 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:10.377+0000 7f3c47400640 1 --2- 172.21.15.47:0/100650765 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3c481026d0 0x7f3c48199f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:21:10.381 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:10.377+0000 7f3c47400640 1 -- 172.21.15.47:0/100650765 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3c48103090 msgr2=0x7f3c481a0f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:21:10.381 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:10.377+0000 7f3c47400640 1 --2- 172.21.15.47:0/100650765 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3c48103090 0x7f3c481a0f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:21:10.381 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:10.377+0000 7f3c47400640 1 -- 172.21.15.47:0/100650765 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3c481a15e0 con 0x7f3c481014e0 2024-03-21T10:21:10.381 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:10.377+0000 7f3c47e00640 1 --2- 172.21.15.47:0/100650765 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3c48103090 0x7f3c481a0f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T10:21:10.381 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:10.378+0000 7f3c47400640 1 --2- 172.21.15.47:0/100650765 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3c481014e0 0x7f3c481999c0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f3c38005ab0 tx=0x7f3c380042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:21:10.382 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:10.378+0000 7f3c44c00640 1 -- 172.21.15.47:0/100650765 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c38043020 con 0x7f3c481014e0 2024-03-21T10:21:10.382 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:10.378+0000 7f3c44c00640 1 -- 172.21.15.47:0/100650765 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f3c380054c0 con 0x7f3c481014e0 2024-03-21T10:21:10.382 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:10.378+0000 7f3c44c00640 1 -- 172.21.15.47:0/100650765 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c38003d10 con 0x7f3c481014e0 2024-03-21T10:21:10.382 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:10.378+0000 7f3c4d200640 1 -- 172.21.15.47:0/100650765 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3c481a1870 con 0x7f3c481014e0 2024-03-21T10:21:10.382 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:10.378+0000 7f3c4d200640 1 -- 172.21.15.47:0/100650765 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3c481a1e90 con 0x7f3c481014e0 2024-03-21T10:21:10.382 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:10.379+0000 7f3c4d200640 1 -- 172.21.15.47:0/100650765 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3c481a2060 con 0x7f3c481014e0 2024-03-21T10:21:10.383 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:10.381+0000 7f3c44c00640 1 -- 172.21.15.47:0/100650765 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 16) v1 ==== 137854+0+0 (secure 0 0 0) 0x7f3c38046020 con 0x7f3c481014e0 2024-03-21T10:21:10.384 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:10.383+0000 7f3c44c00640 1 --2- 172.21.15.47:0/100650765 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f3c2c0a2fa0 0x7f3c2c0a5460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:21:10.385 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:10.383+0000 7f3c44c00640 1 -- 172.21.15.47:0/100650765 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f3c380f1440 con 0x7f3c481014e0 2024-03-21T10:21:10.385 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:10.383+0000 7f3c3ea00640 1 --2- 172.21.15.47:0/100650765 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f3c2c0a2fa0 0x7f3c2c0a5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:21:10.386 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:10.384+0000 7f3c3ea00640 1 --2- 172.21.15.47:0/100650765 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f3c2c0a2fa0 0x7f3c2c0a5460 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f3c4819aed0 tx=0x7f3c300072f0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:21:10.389 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:10.387+0000 7f3c44c00640 1 -- 172.21.15.47:0/100650765 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f3c3804d020 con 0x7f3c481014e0 2024-03-21T10:21:10.636 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:10.633+0000 7f3c4d200640 1 -- 172.21.15.47:0/100650765 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f3c481a2230 con 0x7f3c481014e0 2024-03-21T10:21:10.636 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:10.634+0000 7f3c44c00640 1 -- 172.21.15.47:0/100650765 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 2 v2) v1 ==== 93+0+1328 (secure 0 0 0) 0x7f3c380be790 con 0x7f3c481014e0 2024-03-21T10:21:10.637 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-21T10:21:10.637 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":2,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24406,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6833/4000010136","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":4000010136},{"type":"v1","addr":"172.21.15.157:6833","nonce":4000010136}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2}],"filesystems":[]} 2024-03-21T10:21:10.637 INFO:teuthology.orchestra.run.smithi047.stderr:dumped fsmap epoch 2 2024-03-21T10:21:10.639 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:10.637+0000 7f3c4d200640 1 -- 172.21.15.47:0/100650765 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f3c2c0a2fa0 msgr2=0x7f3c2c0a5460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:21:10.640 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:10.637+0000 7f3c4d200640 1 --2- 172.21.15.47:0/100650765 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f3c2c0a2fa0 0x7f3c2c0a5460 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f3c4819aed0 tx=0x7f3c300072f0 comp rx=0 tx=0).stop 2024-03-21T10:21:10.640 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:10.637+0000 7f3c4d200640 1 -- 172.21.15.47:0/100650765 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3c481014e0 msgr2=0x7f3c481999c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:21:10.640 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:10.637+0000 7f3c4d200640 1 --2- 172.21.15.47:0/100650765 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3c481014e0 0x7f3c481999c0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f3c38005ab0 tx=0x7f3c380042f0 comp rx=0 tx=0).stop 2024-03-21T10:21:10.640 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:10.637+0000 7f3c4d200640 1 -- 172.21.15.47:0/100650765 shutdown_connections 2024-03-21T10:21:10.640 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:10.637+0000 7f3c4d200640 1 --2- 172.21.15.47:0/100650765 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f3c2c0a2fa0 0x7f3c2c0a5460 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:21:10.640 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:10.637+0000 7f3c4d200640 1 --2- 172.21.15.47:0/100650765 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3c48103090 0x7f3c481a0f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:21:10.640 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:10.637+0000 7f3c4d200640 1 --2- 172.21.15.47:0/100650765 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3c481026d0 0x7f3c48199f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:21:10.640 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:10.637+0000 7f3c4d200640 1 --2- 172.21.15.47:0/100650765 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3c481014e0 0x7f3c481999c0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:21:10.640 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:10.637+0000 7f3c4d200640 1 -- 172.21.15.47:0/100650765 >> 172.21.15.47:0/100650765 conn(0x7f3c480fca30 msgr2=0x7f3c48107e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:21:10.640 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:10.637+0000 7f3c4d200640 1 -- 172.21.15.47:0/100650765 shutdown_connections 2024-03-21T10:21:10.640 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:10.638+0000 7f3c4d200640 1 -- 172.21.15.47:0/100650765 wait complete. 2024-03-21T10:21:10.964 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:21:10 smithi047 ceph-mon[31547]: Deploying daemon mds.k on smithi100 2024-03-21T10:21:10.964 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:21:10 smithi047 ceph-mon[31547]: from='client.? 172.21.15.47:0/100650765' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-21T10:21:10.989 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:21:10 smithi157 ceph-mon[35581]: Deploying daemon mds.k on smithi100 2024-03-21T10:21:10.990 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:21:10 smithi157 ceph-mon[35581]: from='client.? 172.21.15.47:0/100650765' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-21T10:21:11.229 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:21:10 smithi100 ceph-mon[35193]: Deploying daemon mds.k on smithi100 2024-03-21T10:21:11.230 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:21:10 smithi100 ceph-mon[35193]: from='client.? 172.21.15.47:0/100650765' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-21T10:21:12.191 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:21:11 smithi100 ceph-mon[35193]: pgmap v335: 1 pgs: 1 active+clean; 577 KiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-21T10:21:12.214 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:21:11 smithi047 ceph-mon[31547]: pgmap v335: 1 pgs: 1 active+clean; 577 KiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-21T10:21:12.240 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:21:11 smithi157 ceph-mon[35581]: pgmap v335: 1 pgs: 1 active+clean; 577 KiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-21T10:21:12.659 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-21T10:21:12.892 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:12.889+0000 7f470d800640 1 -- 172.21.15.47:0/2193448956 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f47080ff980 msgr2=0x7f47080ffda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:21:12.892 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:12.889+0000 7f470d800640 1 --2- 172.21.15.47:0/2193448956 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f47080ff980 0x7f47080ffda0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f46f8009900 tx=0x7f46f802bc30 comp rx=0 tx=0).stop 2024-03-21T10:21:12.892 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:12.889+0000 7f470d800640 1 -- 172.21.15.47:0/2193448956 shutdown_connections 2024-03-21T10:21:12.892 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:12.889+0000 7f470d800640 1 --2- 172.21.15.47:0/2193448956 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4708103310 0x7f4708109ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:21:12.892 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:12.889+0000 7f470d800640 1 --2- 172.21.15.47:0/2193448956 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4708102950 0x7f4708102dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:21:12.892 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:12.889+0000 7f470d800640 1 --2- 172.21.15.47:0/2193448956 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f47080ff980 0x7f47080ffda0 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:21:12.892 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:12.889+0000 7f470d800640 1 -- 172.21.15.47:0/2193448956 >> 172.21.15.47:0/2193448956 conn(0x7f47080fb2d0 msgr2=0x7f47080fd6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:21:12.892 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:12.889+0000 7f470d800640 1 -- 172.21.15.47:0/2193448956 shutdown_connections 2024-03-21T10:21:12.892 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:12.890+0000 7f470d800640 1 -- 172.21.15.47:0/2193448956 wait complete. 2024-03-21T10:21:12.894 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:12.890+0000 7f470d800640 1 Processor -- start 2024-03-21T10:21:12.894 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:12.891+0000 7f470d800640 1 -- start start 2024-03-21T10:21:12.895 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:12.891+0000 7f470d800640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f47080ff980 0x7f4708197980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:21:12.895 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:12.891+0000 7f470d800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4708102950 0x7f4708197ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:21:12.895 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:12.891+0000 7f470d800640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4708103310 0x7f470819eee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:21:12.895 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:12.891+0000 7f470d800640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f470810d250 con 0x7f47080ff980 2024-03-21T10:21:12.895 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:12.891+0000 7f470d800640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f470810d0d0 con 0x7f4708102950 2024-03-21T10:21:12.895 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:12.891+0000 7f470d800640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f470810d3d0 con 0x7f4708103310 2024-03-21T10:21:12.895 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:12.891+0000 7f4706a00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f47080ff980 0x7f4708197980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:21:12.895 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:12.891+0000 7f4706a00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f47080ff980 0x7f4708197980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:48044/0 (socket says 172.21.15.47:48044) 2024-03-21T10:21:12.895 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:12.891+0000 7f4706a00640 1 -- 172.21.15.47:0/3924530679 learned_addr learned my addr 172.21.15.47:0/3924530679 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:21:12.895 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:12.891+0000 7f46ffe00640 1 --2- 172.21.15.47:0/3924530679 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4708102950 0x7f4708197ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:21:12.895 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:12.892+0000 7f4707400640 1 --2- 172.21.15.47:0/3924530679 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4708103310 0x7f470819eee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:21:12.895 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:12.892+0000 7f4706a00640 1 -- 172.21.15.47:0/3924530679 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4708103310 msgr2=0x7f470819eee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:21:12.896 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:12.892+0000 7f4706a00640 1 --2- 172.21.15.47:0/3924530679 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4708103310 0x7f470819eee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:21:12.896 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:12.892+0000 7f4706a00640 1 -- 172.21.15.47:0/3924530679 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4708102950 msgr2=0x7f4708197ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:21:12.896 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:12.892+0000 7f4706a00640 1 --2- 172.21.15.47:0/3924530679 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4708102950 0x7f4708197ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:21:12.896 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:12.892+0000 7f4706a00640 1 -- 172.21.15.47:0/3924530679 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f470819f550 con 0x7f47080ff980 2024-03-21T10:21:12.896 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:12.892+0000 7f46ffe00640 1 --2- 172.21.15.47:0/3924530679 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4708102950 0x7f4708197ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T10:21:12.896 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:12.892+0000 7f4706a00640 1 --2- 172.21.15.47:0/3924530679 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f47080ff980 0x7f4708197980 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f46f8000c00 tx=0x7f46f80042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:21:12.896 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:12.892+0000 7f46ff400640 1 -- 172.21.15.47:0/3924530679 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46f8043020 con 0x7f47080ff980 2024-03-21T10:21:12.896 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:12.892+0000 7f46ff400640 1 -- 172.21.15.47:0/3924530679 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f46f80054c0 con 0x7f47080ff980 2024-03-21T10:21:12.896 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:12.892+0000 7f470d800640 1 -- 172.21.15.47:0/3924530679 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f470819f7e0 con 0x7f47080ff980 2024-03-21T10:21:12.896 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:12.892+0000 7f46ff400640 1 -- 172.21.15.47:0/3924530679 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46f8003d40 con 0x7f47080ff980 2024-03-21T10:21:12.896 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:12.893+0000 7f470d800640 1 -- 172.21.15.47:0/3924530679 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f470819fd00 con 0x7f47080ff980 2024-03-21T10:21:12.897 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:12.894+0000 7f470d800640 1 -- 172.21.15.47:0/3924530679 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4708108ba0 con 0x7f47080ff980 2024-03-21T10:21:12.898 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:12.896+0000 7f46ff400640 1 -- 172.21.15.47:0/3924530679 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 16) v1 ==== 137854+0+0 (secure 0 0 0) 0x7f46f8046020 con 0x7f47080ff980 2024-03-21T10:21:12.899 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:12.897+0000 7f46ff400640 1 --2- 172.21.15.47:0/3924530679 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f46ec0a2f20 0x7f46ec0a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:21:12.899 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:12.897+0000 7f46ff400640 1 -- 172.21.15.47:0/3924530679 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f46f80f1430 con 0x7f47080ff980 2024-03-21T10:21:12.899 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:12.897+0000 7f46ffe00640 1 --2- 172.21.15.47:0/3924530679 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f46ec0a2f20 0x7f46ec0a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:21:12.900 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:12.898+0000 7f46ffe00640 1 --2- 172.21.15.47:0/3924530679 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f46ec0a2f20 0x7f46ec0a53e0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f4708198e40 tx=0x7f46f000b390 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:21:12.903 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:12.902+0000 7f46ff400640 1 -- 172.21.15.47:0/3924530679 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f46f804d020 con 0x7f47080ff980 2024-03-21T10:21:13.128 INFO:journalctl@ceph.mds.k.smithi100.stdout:Mar 21 10:21:12 smithi100 podman[64050]: 2024-03-21 10:21:12.966711425 +0000 UTC m=+0.155246150 container create 6867ea9076c0ab6dc16b0e288d2479cb09856d735987273937c2a3c421815794 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-3056d51a-e76b-11ee-95cd-87774f69a715-mds-k, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.build-date=20240319, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, GIT_BRANCH=HEAD) 2024-03-21T10:21:13.149 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:13.147+0000 7f470d800640 1 -- 172.21.15.47:0/3924530679 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f470802d290 con 0x7f47080ff980 2024-03-21T10:21:13.150 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:13.148+0000 7f46ff400640 1 -- 172.21.15.47:0/3924530679 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 2 v2) v1 ==== 93+0+1328 (secure 0 0 0) 0x7f46f80be780 con 0x7f47080ff980 2024-03-21T10:21:13.150 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-21T10:21:13.150 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":2,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24406,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6833/4000010136","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":4000010136},{"type":"v1","addr":"172.21.15.157:6833","nonce":4000010136}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2}],"filesystems":[]} 2024-03-21T10:21:13.151 INFO:teuthology.orchestra.run.smithi047.stderr:dumped fsmap epoch 2 2024-03-21T10:21:13.153 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:13.151+0000 7f470d800640 1 -- 172.21.15.47:0/3924530679 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f46ec0a2f20 msgr2=0x7f46ec0a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:21:13.153 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:13.151+0000 7f470d800640 1 --2- 172.21.15.47:0/3924530679 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f46ec0a2f20 0x7f46ec0a53e0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f4708198e40 tx=0x7f46f000b390 comp rx=0 tx=0).stop 2024-03-21T10:21:13.153 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:13.151+0000 7f470d800640 1 -- 172.21.15.47:0/3924530679 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f47080ff980 msgr2=0x7f4708197980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:21:13.153 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:13.151+0000 7f470d800640 1 --2- 172.21.15.47:0/3924530679 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f47080ff980 0x7f4708197980 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f46f8000c00 tx=0x7f46f80042f0 comp rx=0 tx=0).stop 2024-03-21T10:21:13.154 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:13.151+0000 7f470d800640 1 -- 172.21.15.47:0/3924530679 shutdown_connections 2024-03-21T10:21:13.154 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:13.151+0000 7f470d800640 1 --2- 172.21.15.47:0/3924530679 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f46ec0a2f20 0x7f46ec0a53e0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:21:13.154 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:13.151+0000 7f470d800640 1 --2- 172.21.15.47:0/3924530679 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4708103310 0x7f470819eee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:21:13.154 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:13.151+0000 7f470d800640 1 --2- 172.21.15.47:0/3924530679 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4708102950 0x7f4708197ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:21:13.154 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:13.151+0000 7f470d800640 1 --2- 172.21.15.47:0/3924530679 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f47080ff980 0x7f4708197980 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:21:13.154 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:13.151+0000 7f470d800640 1 -- 172.21.15.47:0/3924530679 >> 172.21.15.47:0/3924530679 conn(0x7f47080fb2d0 msgr2=0x7f4708105d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:21:13.154 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:13.151+0000 7f470d800640 1 -- 172.21.15.47:0/3924530679 shutdown_connections 2024-03-21T10:21:13.154 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:13.151+0000 7f470d800640 1 -- 172.21.15.47:0/3924530679 wait complete. 2024-03-21T10:21:13.731 INFO:journalctl@ceph.mds.k.smithi100.stdout:Mar 21 10:21:13 smithi100 podman[64050]: 2024-03-21 10:21:13.237555112 +0000 UTC m=+0.426089824 container init 6867ea9076c0ab6dc16b0e288d2479cb09856d735987273937c2a3c421815794 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-3056d51a-e76b-11ee-95cd-87774f69a715-mds-k, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240319, maintainer=Guillaume Abrioux , ceph=True, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, GIT_CLEAN=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30) 2024-03-21T10:21:13.731 INFO:journalctl@ceph.mds.k.smithi100.stdout:Mar 21 10:21:13 smithi100 podman[64050]: 2024-03-21 10:21:13.247663894 +0000 UTC m=+0.436198567 container start 6867ea9076c0ab6dc16b0e288d2479cb09856d735987273937c2a3c421815794 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-3056d51a-e76b-11ee-95cd-87774f69a715-mds-k, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_BRANCH=HEAD, org.label-schema.build-date=20240319, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.vendor=CentOS) 2024-03-21T10:21:13.731 INFO:journalctl@ceph.mds.k.smithi100.stdout:Mar 21 10:21:13 smithi100 ceph-mds[64076]: -- 172.21.15.100:0/1550450085 <== mon.2 v2:172.21.15.100:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x564757203a40 con 0x5647571d6c00 2024-03-21T10:21:13.731 INFO:journalctl@ceph.mds.k.smithi100.stdout:Mar 21 10:21:13 smithi100 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-mds-k[64070]: starting mds.k at 2024-03-21T10:21:13.731 INFO:journalctl@ceph.mds.k.smithi100.stdout:Mar 21 10:21:13 smithi100 bash[64050]: 6867ea9076c0ab6dc16b0e288d2479cb09856d735987273937c2a3c421815794 2024-03-21T10:21:13.731 INFO:journalctl@ceph.mds.k.smithi100.stdout:Mar 21 10:21:13 smithi100 systemd[1]: Started Ceph mds.k for 3056d51a-e76b-11ee-95cd-87774f69a715. 2024-03-21T10:21:14.214 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:21:13 smithi047 ceph-mon[31547]: pgmap v336: 1 pgs: 1 active+clean; 577 KiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-21T10:21:14.214 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:21:13 smithi047 ceph-mon[31547]: from='client.? 172.21.15.47:0/3924530679' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-21T10:21:14.214 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:21:13 smithi047 ceph-mon[31547]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:21:14.214 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:21:13 smithi047 ceph-mon[31547]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:21:14.214 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:21:13 smithi047 ceph-mon[31547]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:21:14.214 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:21:13 smithi047 ceph-mon[31547]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-21T10:21:14.214 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:21:13 smithi047 ceph-mon[31547]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-21T10:21:14.214 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:21:13 smithi047 ceph-mon[31547]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T10:21:14.230 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:21:13 smithi100 ceph-mon[35193]: pgmap v336: 1 pgs: 1 active+clean; 577 KiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-21T10:21:14.230 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:21:13 smithi100 ceph-mon[35193]: from='client.? 172.21.15.47:0/3924530679' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-21T10:21:14.230 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:21:13 smithi100 ceph-mon[35193]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:21:14.230 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:21:13 smithi100 ceph-mon[35193]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:21:14.230 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:21:13 smithi100 ceph-mon[35193]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:21:14.230 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:21:13 smithi100 ceph-mon[35193]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-21T10:21:14.231 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:21:13 smithi100 ceph-mon[35193]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-21T10:21:14.231 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:21:13 smithi100 ceph-mon[35193]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T10:21:14.240 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:21:13 smithi157 ceph-mon[35581]: pgmap v336: 1 pgs: 1 active+clean; 577 KiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-21T10:21:14.240 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:21:13 smithi157 ceph-mon[35581]: from='client.? 172.21.15.47:0/3924530679' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-21T10:21:14.240 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:21:13 smithi157 ceph-mon[35581]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:21:14.240 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:21:13 smithi157 ceph-mon[35581]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:21:14.240 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:21:13 smithi157 ceph-mon[35581]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:21:14.240 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:21:13 smithi157 ceph-mon[35581]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-21T10:21:14.240 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:21:13 smithi157 ceph-mon[35581]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-21T10:21:14.240 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:21:13 smithi157 ceph-mon[35581]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T10:21:14.894 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:21:14 smithi047 ceph-mon[31547]: Deploying daemon mds.j on smithi047 2024-03-21T10:21:14.894 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:21:14 smithi047 ceph-mon[31547]: mds.? [v2:172.21.15.100:6832/2723302038,v1:172.21.15.100:6833/2723302038] up:boot 2024-03-21T10:21:14.894 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:21:14 smithi047 ceph-mon[31547]: fsmap 2 up:standby 2024-03-21T10:21:14.894 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:21:14 smithi047 ceph-mon[31547]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-03-21T10:21:15.175 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-21T10:21:15.229 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:21:14 smithi100 ceph-mon[35193]: Deploying daemon mds.j on smithi047 2024-03-21T10:21:15.230 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:21:14 smithi100 ceph-mon[35193]: mds.? [v2:172.21.15.100:6832/2723302038,v1:172.21.15.100:6833/2723302038] up:boot 2024-03-21T10:21:15.230 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:21:14 smithi100 ceph-mon[35193]: fsmap 2 up:standby 2024-03-21T10:21:15.230 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:21:14 smithi100 ceph-mon[35193]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-03-21T10:21:15.239 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:21:14 smithi157 ceph-mon[35581]: Deploying daemon mds.j on smithi047 2024-03-21T10:21:15.240 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:21:14 smithi157 ceph-mon[35581]: mds.? [v2:172.21.15.100:6832/2723302038,v1:172.21.15.100:6833/2723302038] up:boot 2024-03-21T10:21:15.240 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:21:14 smithi157 ceph-mon[35581]: fsmap 2 up:standby 2024-03-21T10:21:15.240 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:21:14 smithi157 ceph-mon[35581]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-03-21T10:21:15.393 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:15.390+0000 7f22d3e00640 1 -- 172.21.15.47:0/1956040252 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f22d4075680 msgr2=0x7f22d4073ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:21:15.393 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:15.390+0000 7f22d3e00640 1 --2- 172.21.15.47:0/1956040252 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f22d4075680 0x7f22d4073ad0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f22c4009ca0 tx=0x7f22c402c190 comp rx=0 tx=0).stop 2024-03-21T10:21:15.393 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:15.391+0000 7f22d3e00640 1 -- 172.21.15.47:0/1956040252 shutdown_connections 2024-03-21T10:21:15.393 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:15.391+0000 7f22d3e00640 1 --2- 172.21.15.47:0/1956040252 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f22d4109370 0x7f22d410b810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:21:15.394 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:15.391+0000 7f22d3e00640 1 --2- 172.21.15.47:0/1956040252 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f22d40740a0 0x7f22d4074500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:21:15.394 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:15.391+0000 7f22d3e00640 1 --2- 172.21.15.47:0/1956040252 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f22d4075680 0x7f22d4073ad0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:21:15.394 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:15.391+0000 7f22d3e00640 1 -- 172.21.15.47:0/1956040252 >> 172.21.15.47:0/1956040252 conn(0x7f22d40fca50 msgr2=0x7f22d40fee90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:21:15.394 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:15.391+0000 7f22d3e00640 1 -- 172.21.15.47:0/1956040252 shutdown_connections 2024-03-21T10:21:15.394 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:15.391+0000 7f22d3e00640 1 -- 172.21.15.47:0/1956040252 wait complete. 2024-03-21T10:21:15.397 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:15.392+0000 7f22d3e00640 1 Processor -- start 2024-03-21T10:21:15.397 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:15.392+0000 7f22d3e00640 1 -- start start 2024-03-21T10:21:15.397 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:15.392+0000 7f22d3e00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f22d40740a0 0x7f22d41998d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:21:15.397 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:15.392+0000 7f22d3e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f22d4075680 0x7f22d4199e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:21:15.397 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:15.392+0000 7f22d3e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f22d4109370 0x7f22d41a0e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:21:15.397 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:15.392+0000 7f22d3e00640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22d410f260 con 0x7f22d4109370 2024-03-21T10:21:15.397 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:15.392+0000 7f22d3e00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22d410f0e0 con 0x7f22d4075680 2024-03-21T10:21:15.397 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:15.392+0000 7f22d3e00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22d410f3e0 con 0x7f22d40740a0 2024-03-21T10:21:15.398 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:15.392+0000 7f22d3400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f22d4109370 0x7f22d41a0e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:21:15.398 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:15.392+0000 7f22d3400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f22d4109370 0x7f22d41a0e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:48072/0 (socket says 172.21.15.47:48072) 2024-03-21T10:21:15.398 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:15.392+0000 7f22d3400640 1 -- 172.21.15.47:0/1180421794 learned_addr learned my addr 172.21.15.47:0/1180421794 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:21:15.398 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:15.393+0000 7f22d2a00640 1 --2- 172.21.15.47:0/1180421794 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f22d40740a0 0x7f22d41998d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:21:15.398 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:15.393+0000 7f22d2000640 1 --2- 172.21.15.47:0/1180421794 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f22d4075680 0x7f22d4199e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:21:15.398 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:15.393+0000 7f22d3400640 1 -- 172.21.15.47:0/1180421794 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f22d40740a0 msgr2=0x7f22d41998d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:21:15.398 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:15.393+0000 7f22d3400640 1 --2- 172.21.15.47:0/1180421794 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f22d40740a0 0x7f22d41998d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:21:15.398 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:15.393+0000 7f22d3400640 1 -- 172.21.15.47:0/1180421794 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f22d4075680 msgr2=0x7f22d4199e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:21:15.398 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:15.393+0000 7f22d3400640 1 --2- 172.21.15.47:0/1180421794 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f22d4075680 0x7f22d4199e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:21:15.398 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:15.393+0000 7f22d3400640 1 -- 172.21.15.47:0/1180421794 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22d41a14f0 con 0x7f22d4109370 2024-03-21T10:21:15.398 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:15.393+0000 7f22d3400640 1 --2- 172.21.15.47:0/1180421794 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f22d4109370 0x7f22d41a0e80 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f22cc009fd0 tx=0x7f22cc00b8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:21:15.399 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:15.394+0000 7f22cb400640 1 -- 172.21.15.47:0/1180421794 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22cc011830 con 0x7f22d4109370 2024-03-21T10:21:15.399 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:15.394+0000 7f22cb400640 1 -- 172.21.15.47:0/1180421794 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f22cc016070 con 0x7f22d4109370 2024-03-21T10:21:15.399 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:15.394+0000 7f22d3e00640 1 -- 172.21.15.47:0/1180421794 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f22d41a17e0 con 0x7f22d4109370 2024-03-21T10:21:15.399 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:15.394+0000 7f22cb400640 1 -- 172.21.15.47:0/1180421794 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22cc010de0 con 0x7f22d4109370 2024-03-21T10:21:15.399 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:15.394+0000 7f22d3e00640 1 -- 172.21.15.47:0/1180421794 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f22d41a1d60 con 0x7f22d4109370 2024-03-21T10:21:15.399 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:15.395+0000 7f22d3e00640 1 -- 172.21.15.47:0/1180421794 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f22d4108130 con 0x7f22d4109370 2024-03-21T10:21:15.399 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:15.397+0000 7f22cb400640 1 -- 172.21.15.47:0/1180421794 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 16) v1 ==== 137854+0+0 (secure 0 0 0) 0x7f22cc0119d0 con 0x7f22d4109370 2024-03-21T10:21:15.400 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:15.398+0000 7f22cb400640 1 --2- 172.21.15.47:0/1180421794 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f22b80a2f20 0x7f22b80a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:21:15.400 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:15.398+0000 7f22cb400640 1 -- 172.21.15.47:0/1180421794 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f22cc0ce6d0 con 0x7f22d4109370 2024-03-21T10:21:15.400 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:15.398+0000 7f22d2a00640 1 --2- 172.21.15.47:0/1180421794 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f22b80a2f20 0x7f22b80a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:21:15.401 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:15.399+0000 7f22d2a00640 1 --2- 172.21.15.47:0/1180421794 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f22b80a2f20 0x7f22b80a53e0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f22c402c7a0 tx=0x7f22c4005ef0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:21:15.406 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:15.404+0000 7f22cb400640 1 -- 172.21.15.47:0/1180421794 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f22cc09b900 con 0x7f22d4109370 2024-03-21T10:21:15.654 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:15.650+0000 7f22d3e00640 1 -- 172.21.15.47:0/1180421794 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f22d402d070 con 0x7f22d4109370 2024-03-21T10:21:15.655 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:15.651+0000 7f22cb400640 1 -- 172.21.15.47:0/1180421794 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) v1 ==== 93+0+2122 (secure 0 0 0) 0x7f22cc09b200 con 0x7f22d4109370 2024-03-21T10:21:15.655 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-21T10:21:15.655 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":3,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24406,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6833/4000010136","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":4000010136},{"type":"v1","addr":"172.21.15.157:6833","nonce":4000010136}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":34427,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6833/2723302038","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6832","nonce":2723302038},{"type":"v1","addr":"172.21.15.100:6833","nonce":2723302038}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[]} 2024-03-21T10:21:15.657 INFO:teuthology.orchestra.run.smithi047.stderr:dumped fsmap epoch 3 2024-03-21T10:21:15.657 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:15.654+0000 7f22d3e00640 1 -- 172.21.15.47:0/1180421794 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f22b80a2f20 msgr2=0x7f22b80a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:21:15.657 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:15.654+0000 7f22d3e00640 1 --2- 172.21.15.47:0/1180421794 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f22b80a2f20 0x7f22b80a53e0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f22c402c7a0 tx=0x7f22c4005ef0 comp rx=0 tx=0).stop 2024-03-21T10:21:15.658 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:15.654+0000 7f22d3e00640 1 -- 172.21.15.47:0/1180421794 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f22d4109370 msgr2=0x7f22d41a0e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:21:15.658 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:15.654+0000 7f22d3e00640 1 --2- 172.21.15.47:0/1180421794 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f22d4109370 0x7f22d41a0e80 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f22cc009fd0 tx=0x7f22cc00b8e0 comp rx=0 tx=0).stop 2024-03-21T10:21:15.658 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:15.654+0000 7f22d3e00640 1 -- 172.21.15.47:0/1180421794 shutdown_connections 2024-03-21T10:21:15.658 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:15.654+0000 7f22d3e00640 1 --2- 172.21.15.47:0/1180421794 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f22b80a2f20 0x7f22b80a53e0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:21:15.658 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:15.654+0000 7f22d3e00640 1 --2- 172.21.15.47:0/1180421794 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f22d4109370 0x7f22d41a0e80 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:21:15.658 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:15.654+0000 7f22d3e00640 1 --2- 172.21.15.47:0/1180421794 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f22d4075680 0x7f22d4199e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:21:15.659 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:15.654+0000 7f22d3e00640 1 --2- 172.21.15.47:0/1180421794 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f22d40740a0 0x7f22d41998d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:21:15.659 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:15.654+0000 7f22d3e00640 1 -- 172.21.15.47:0/1180421794 >> 172.21.15.47:0/1180421794 conn(0x7f22d40fca50 msgr2=0x7f22d4105300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:21:15.659 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:15.654+0000 7f22d3e00640 1 -- 172.21.15.47:0/1180421794 shutdown_connections 2024-03-21T10:21:15.659 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:15.654+0000 7f22d3e00640 1 -- 172.21.15.47:0/1180421794 wait complete. 2024-03-21T10:21:15.946 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:21:15 smithi047 ceph-mon[31547]: pgmap v337: 1 pgs: 1 active+clean; 577 KiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-21T10:21:15.946 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:21:15 smithi047 ceph-mon[31547]: from='client.? 172.21.15.47:0/1180421794' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-21T10:21:16.229 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:21:15 smithi100 ceph-mon[35193]: pgmap v337: 1 pgs: 1 active+clean; 577 KiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-21T10:21:16.230 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:21:15 smithi100 ceph-mon[35193]: from='client.? 172.21.15.47:0/1180421794' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-21T10:21:16.240 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:21:15 smithi157 ceph-mon[35581]: pgmap v337: 1 pgs: 1 active+clean; 577 KiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-21T10:21:16.240 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:21:15 smithi157 ceph-mon[35581]: from='client.? 172.21.15.47:0/1180421794' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-21T10:21:16.964 INFO:journalctl@ceph.mds.j.smithi047.stdout:Mar 21 10:21:16 smithi047 systemd[1]: Starting Ceph mds.j for 3056d51a-e76b-11ee-95cd-87774f69a715... 2024-03-21T10:21:17.348 INFO:journalctl@ceph.mds.j.smithi047.stdout:Mar 21 10:21:17 smithi047 podman[69702]: 2024-03-21 10:21:16.904605143 +0000 UTC m=+0.035189791 image pull 237e43c7a65ff48534d94b97c1bfdbe4b7be5672e3d9a613745083ea3702aa54 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232 2024-03-21T10:21:17.348 INFO:journalctl@ceph.mds.j.smithi047.stdout:Mar 21 10:21:17 smithi047 podman[69702]: 2024-03-21 10:21:17.057950616 +0000 UTC m=+0.188535288 container create 2488f82e2f505c307e52eb483a7879dd4fe69698767cb9fd3c2f7479ff5b920d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-3056d51a-e76b-11ee-95cd-87774f69a715-mds-j, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_BRANCH=HEAD, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.build-date=20240319, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS) 2024-03-21T10:21:17.349 INFO:journalctl@ceph.mds.j.smithi047.stdout:Mar 21 10:21:17 smithi047 podman[69702]: 2024-03-21 10:21:17.345320488 +0000 UTC m=+0.475905204 container init 2488f82e2f505c307e52eb483a7879dd4fe69698767cb9fd3c2f7479ff5b920d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-3056d51a-e76b-11ee-95cd-87774f69a715-mds-j, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240319, org.label-schema.license=GPLv2, io.buildah.version=1.33.5) 2024-03-21T10:21:17.677 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-21T10:21:17.714 INFO:journalctl@ceph.mds.j.smithi047.stdout:Mar 21 10:21:17 smithi047 podman[69702]: 2024-03-21 10:21:17.356668839 +0000 UTC m=+0.487253489 container start 2488f82e2f505c307e52eb483a7879dd4fe69698767cb9fd3c2f7479ff5b920d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-3056d51a-e76b-11ee-95cd-87774f69a715-mds-j, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, org.label-schema.build-date=20240319, org.label-schema.license=GPLv2, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30) 2024-03-21T10:21:17.714 INFO:journalctl@ceph.mds.j.smithi047.stdout:Mar 21 10:21:17 smithi047 ceph-mds[69721]: -- 172.21.15.47:0/2086733200 <== mon.0 v2:172.21.15.47:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55fe55bd1a40 con 0x55fe55ba5000 2024-03-21T10:21:17.714 INFO:journalctl@ceph.mds.j.smithi047.stdout:Mar 21 10:21:17 smithi047 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-mds-j[69717]: starting mds.j at 2024-03-21T10:21:17.714 INFO:journalctl@ceph.mds.j.smithi047.stdout:Mar 21 10:21:17 smithi047 bash[69702]: 2488f82e2f505c307e52eb483a7879dd4fe69698767cb9fd3c2f7479ff5b920d 2024-03-21T10:21:17.714 INFO:journalctl@ceph.mds.j.smithi047.stdout:Mar 21 10:21:17 smithi047 systemd[1]: Started Ceph mds.j for 3056d51a-e76b-11ee-95cd-87774f69a715. 2024-03-21T10:21:17.894 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:17.892+0000 7fd23e400640 1 -- 172.21.15.47:0/2700565000 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd238104ad0 msgr2=0x7fd238104f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:21:17.894 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:17.892+0000 7fd23e400640 1 --2- 172.21.15.47:0/2700565000 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd238104ad0 0x7fd238104f50 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fd220009900 tx=0x7fd22002bc50 comp rx=0 tx=0).stop 2024-03-21T10:21:17.894 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:17.892+0000 7fd23e400640 1 -- 172.21.15.47:0/2700565000 shutdown_connections 2024-03-21T10:21:17.894 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:17.892+0000 7fd23e400640 1 --2- 172.21.15.47:0/2700565000 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd238105490 0x7fd23810bd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:21:17.894 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:17.892+0000 7fd23e400640 1 --2- 172.21.15.47:0/2700565000 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd238104ad0 0x7fd238104f50 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:21:17.894 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:17.892+0000 7fd23e400640 1 --2- 172.21.15.47:0/2700565000 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd2381038e0 0x7fd238103d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:21:17.894 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:17.892+0000 7fd23e400640 1 -- 172.21.15.47:0/2700565000 >> 172.21.15.47:0/2700565000 conn(0x7fd2380fee90 msgr2=0x7fd2381012b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:21:17.894 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:17.893+0000 7fd23e400640 1 -- 172.21.15.47:0/2700565000 shutdown_connections 2024-03-21T10:21:17.895 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:17.893+0000 7fd23e400640 1 -- 172.21.15.47:0/2700565000 wait complete. 2024-03-21T10:21:17.897 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:17.894+0000 7fd23e400640 1 Processor -- start 2024-03-21T10:21:17.897 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:17.894+0000 7fd23e400640 1 -- start start 2024-03-21T10:21:17.897 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:17.894+0000 7fd23e400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd2381038e0 0x7fd238199b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:21:17.897 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:17.894+0000 7fd23e400640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd238104ad0 0x7fd23819a060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:21:17.897 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:17.894+0000 7fd23e400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd238105490 0x7fd2381a10d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:21:17.897 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:17.894+0000 7fd23e400640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd23810f410 con 0x7fd238105490 2024-03-21T10:21:17.897 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:17.894+0000 7fd23e400640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd23810f290 con 0x7fd2381038e0 2024-03-21T10:21:17.897 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:17.894+0000 7fd23e400640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd23810f590 con 0x7fd238104ad0 2024-03-21T10:21:17.897 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:17.894+0000 7fd237e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd238105490 0x7fd2381a10d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:21:17.897 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:17.895+0000 7fd237e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd238105490 0x7fd2381a10d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:48114/0 (socket says 172.21.15.47:48114) 2024-03-21T10:21:17.897 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:17.895+0000 7fd237e00640 1 -- 172.21.15.47:0/3006472767 learned_addr learned my addr 172.21.15.47:0/3006472767 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:21:17.897 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:17.895+0000 7fd237400640 1 --2- 172.21.15.47:0/3006472767 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd2381038e0 0x7fd238199b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:21:17.898 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:17.895+0000 7fd22fe00640 1 --2- 172.21.15.47:0/3006472767 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd238104ad0 0x7fd23819a060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:21:17.898 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:17.895+0000 7fd237e00640 1 -- 172.21.15.47:0/3006472767 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd238104ad0 msgr2=0x7fd23819a060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:21:17.898 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:17.895+0000 7fd237e00640 1 --2- 172.21.15.47:0/3006472767 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd238104ad0 0x7fd23819a060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:21:17.898 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:17.895+0000 7fd237e00640 1 -- 172.21.15.47:0/3006472767 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd2381038e0 msgr2=0x7fd238199b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:21:17.898 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:17.895+0000 7fd237e00640 1 --2- 172.21.15.47:0/3006472767 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd2381038e0 0x7fd238199b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:21:17.898 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:17.895+0000 7fd237e00640 1 -- 172.21.15.47:0/3006472767 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2381a1740 con 0x7fd238105490 2024-03-21T10:21:17.898 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:17.895+0000 7fd237400640 1 --2- 172.21.15.47:0/3006472767 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd2381038e0 0x7fd238199b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T10:21:17.898 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:17.895+0000 7fd237e00640 1 --2- 172.21.15.47:0/3006472767 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd238105490 0x7fd2381a10d0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fd23000eaa0 tx=0x7fd23000ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:21:17.898 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:17.896+0000 7fd234c00640 1 -- 172.21.15.47:0/3006472767 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd23000cdb0 con 0x7fd238105490 2024-03-21T10:21:17.898 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:17.896+0000 7fd234c00640 1 -- 172.21.15.47:0/3006472767 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fd230005020 con 0x7fd238105490 2024-03-21T10:21:17.898 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:17.896+0000 7fd23e400640 1 -- 172.21.15.47:0/3006472767 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd2381a1a30 con 0x7fd238105490 2024-03-21T10:21:17.898 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:17.896+0000 7fd234c00640 1 -- 172.21.15.47:0/3006472767 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2300116d0 con 0x7fd238105490 2024-03-21T10:21:17.898 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:17.896+0000 7fd23e400640 1 -- 172.21.15.47:0/3006472767 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd2381a1f80 con 0x7fd238105490 2024-03-21T10:21:17.899 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:17.897+0000 7fd23e400640 1 -- 172.21.15.47:0/3006472767 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd23810ac50 con 0x7fd238105490 2024-03-21T10:21:17.901 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:17.901+0000 7fd234c00640 1 -- 172.21.15.47:0/3006472767 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 16) v1 ==== 137854+0+0 (secure 0 0 0) 0x7fd2300051d0 con 0x7fd238105490 2024-03-21T10:21:17.905 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:17.902+0000 7fd234c00640 1 --2- 172.21.15.47:0/3006472767 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7fd21c0a2f20 0x7fd21c0a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:21:17.905 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:17.902+0000 7fd234c00640 1 -- 172.21.15.47:0/3006472767 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 7127+0+0 (secure 0 0 0) 0x7fd2300cf220 con 0x7fd238105490 2024-03-21T10:21:17.905 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:17.902+0000 7fd237400640 1 --2- 172.21.15.47:0/3006472767 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7fd21c0a2f20 0x7fd21c0a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:21:17.905 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:17.903+0000 7fd237400640 1 --2- 172.21.15.47:0/3006472767 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7fd21c0a2f20 0x7fd21c0a53e0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fd238104930 tx=0x7fd228005e70 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:21:17.908 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:17.907+0000 7fd234c00640 1 -- 172.21.15.47:0/3006472767 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fd230016030 con 0x7fd238105490 2024-03-21T10:21:18.155 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:18.154+0000 7fd23e400640 1 -- 172.21.15.47:0/3006472767 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd23802d070 con 0x7fd238105490 2024-03-21T10:21:18.156 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:18.155+0000 7fd234c00640 1 -- 172.21.15.47:0/3006472767 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 4 v4) v1 ==== 93+0+2910 (secure 0 0 0) 0x7fd23009c450 con 0x7fd238105490 2024-03-21T10:21:18.156 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-21T10:21:18.156 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":4,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14511,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6835/735724037","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6834","nonce":735724037},{"type":"v1","addr":"172.21.15.47:6835","nonce":735724037}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":24406,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6833/4000010136","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":4000010136},{"type":"v1","addr":"172.21.15.157:6833","nonce":4000010136}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":34427,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6833/2723302038","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6832","nonce":2723302038},{"type":"v1","addr":"172.21.15.100:6833","nonce":2723302038}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[]} 2024-03-21T10:21:18.156 INFO:teuthology.orchestra.run.smithi047.stderr:dumped fsmap epoch 4 2024-03-21T10:21:18.158 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:18.157+0000 7fd23e400640 1 -- 172.21.15.47:0/3006472767 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7fd21c0a2f20 msgr2=0x7fd21c0a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:21:18.158 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:18.157+0000 7fd23e400640 1 --2- 172.21.15.47:0/3006472767 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7fd21c0a2f20 0x7fd21c0a53e0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fd238104930 tx=0x7fd228005e70 comp rx=0 tx=0).stop 2024-03-21T10:21:18.159 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:18.158+0000 7fd23e400640 1 -- 172.21.15.47:0/3006472767 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd238105490 msgr2=0x7fd2381a10d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:21:18.159 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:18.158+0000 7fd23e400640 1 --2- 172.21.15.47:0/3006472767 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd238105490 0x7fd2381a10d0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fd23000eaa0 tx=0x7fd23000ef60 comp rx=0 tx=0).stop 2024-03-21T10:21:18.159 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:18.158+0000 7fd23e400640 1 -- 172.21.15.47:0/3006472767 shutdown_connections 2024-03-21T10:21:18.159 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:18.158+0000 7fd23e400640 1 --2- 172.21.15.47:0/3006472767 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7fd21c0a2f20 0x7fd21c0a53e0 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:21:18.159 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:18.158+0000 7fd23e400640 1 --2- 172.21.15.47:0/3006472767 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd238105490 0x7fd2381a10d0 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:21:18.159 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:18.158+0000 7fd23e400640 1 --2- 172.21.15.47:0/3006472767 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd238104ad0 0x7fd23819a060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:21:18.159 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:18.158+0000 7fd23e400640 1 --2- 172.21.15.47:0/3006472767 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd2381038e0 0x7fd238199b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:21:18.159 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:18.158+0000 7fd23e400640 1 -- 172.21.15.47:0/3006472767 >> 172.21.15.47:0/3006472767 conn(0x7fd2380fee90 msgr2=0x7fd238107e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:21:18.160 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:18.158+0000 7fd23e400640 1 -- 172.21.15.47:0/3006472767 shutdown_connections 2024-03-21T10:21:18.160 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:18.158+0000 7fd23e400640 1 -- 172.21.15.47:0/3006472767 wait complete. 2024-03-21T10:21:18.177 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:21:17 smithi047 ceph-mon[31547]: pgmap v338: 1 pgs: 1 active+clean; 577 KiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-21T10:21:18.177 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:21:17 smithi047 ceph-mon[31547]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:21:18.177 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:21:17 smithi047 ceph-mon[31547]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:21:18.177 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:21:17 smithi047 ceph-mon[31547]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:21:18.177 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:21:17 smithi047 ceph-mon[31547]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-21T10:21:18.177 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:21:17 smithi047 ceph-mon[31547]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-21T10:21:18.177 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:21:17 smithi047 ceph-mon[31547]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T10:21:18.230 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:21:17 smithi100 ceph-mon[35193]: pgmap v338: 1 pgs: 1 active+clean; 577 KiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-21T10:21:18.230 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:21:17 smithi100 ceph-mon[35193]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:21:18.230 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:21:17 smithi100 ceph-mon[35193]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:21:18.230 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:21:17 smithi100 ceph-mon[35193]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:21:18.230 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:21:17 smithi100 ceph-mon[35193]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-21T10:21:18.230 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:21:17 smithi100 ceph-mon[35193]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-21T10:21:18.230 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:21:17 smithi100 ceph-mon[35193]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T10:21:18.240 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:21:17 smithi157 ceph-mon[35581]: pgmap v338: 1 pgs: 1 active+clean; 577 KiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-21T10:21:18.240 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:21:17 smithi157 ceph-mon[35581]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:21:18.240 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:21:17 smithi157 ceph-mon[35581]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:21:18.240 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:21:17 smithi157 ceph-mon[35581]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:21:18.240 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:21:17 smithi157 ceph-mon[35581]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-21T10:21:18.240 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:21:17 smithi157 ceph-mon[35581]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-21T10:21:18.240 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:21:17 smithi157 ceph-mon[35581]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T10:21:18.964 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:21:18 smithi047 ceph-mon[31547]: Deploying daemon mds.d on smithi047 2024-03-21T10:21:18.964 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:21:18 smithi047 ceph-mon[31547]: mds.? [v2:172.21.15.47:6834/735724037,v1:172.21.15.47:6835/735724037] up:boot 2024-03-21T10:21:18.964 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:21:18 smithi047 ceph-mon[31547]: fsmap 3 up:standby 2024-03-21T10:21:18.964 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:21:18 smithi047 ceph-mon[31547]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-03-21T10:21:18.964 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:21:18 smithi047 ceph-mon[31547]: from='client.? 172.21.15.47:0/3006472767' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-21T10:21:19.230 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:21:18 smithi100 ceph-mon[35193]: Deploying daemon mds.d on smithi047 2024-03-21T10:21:19.230 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:21:18 smithi100 ceph-mon[35193]: mds.? [v2:172.21.15.47:6834/735724037,v1:172.21.15.47:6835/735724037] up:boot 2024-03-21T10:21:19.230 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:21:18 smithi100 ceph-mon[35193]: fsmap 3 up:standby 2024-03-21T10:21:19.230 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:21:18 smithi100 ceph-mon[35193]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-03-21T10:21:19.230 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:21:18 smithi100 ceph-mon[35193]: from='client.? 172.21.15.47:0/3006472767' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-21T10:21:19.240 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:21:18 smithi157 ceph-mon[35581]: Deploying daemon mds.d on smithi047 2024-03-21T10:21:19.240 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:21:18 smithi157 ceph-mon[35581]: mds.? [v2:172.21.15.47:6834/735724037,v1:172.21.15.47:6835/735724037] up:boot 2024-03-21T10:21:19.240 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:21:18 smithi157 ceph-mon[35581]: fsmap 3 up:standby 2024-03-21T10:21:19.240 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:21:18 smithi157 ceph-mon[35581]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-03-21T10:21:19.240 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:21:18 smithi157 ceph-mon[35581]: from='client.? 172.21.15.47:0/3006472767' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-21T10:21:19.964 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:21:19 smithi047 ceph-mon[31547]: pgmap v339: 1 pgs: 1 active+clean; 577 KiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-21T10:21:20.182 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-21T10:21:20.229 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:21:19 smithi100 ceph-mon[35193]: pgmap v339: 1 pgs: 1 active+clean; 577 KiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-21T10:21:20.239 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:21:19 smithi157 ceph-mon[35581]: pgmap v339: 1 pgs: 1 active+clean; 577 KiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-21T10:21:20.620 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:20.618+0000 7f8ca9200640 1 -- 172.21.15.47:0/29422416 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8ca4076060 msgr2=0x7f8ca4076480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:21:20.620 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:20.618+0000 7f8ca9200640 1 --2- 172.21.15.47:0/29422416 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8ca4076060 0x7f8ca4076480 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f8c94009900 tx=0x7f8c9402bc40 comp rx=0 tx=0).stop 2024-03-21T10:21:20.620 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:20.618+0000 7f8ca9200640 1 -- 172.21.15.47:0/29422416 shutdown_connections 2024-03-21T10:21:20.620 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:20.618+0000 7f8ca9200640 1 --2- 172.21.15.47:0/29422416 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8ca4077c40 0x7f8ca40784e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:21:20.620 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:20.618+0000 7f8ca9200640 1 --2- 172.21.15.47:0/29422416 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8ca40772a0 0x7f8ca4077700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:21:20.620 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:20.618+0000 7f8ca9200640 1 --2- 172.21.15.47:0/29422416 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8ca4076060 0x7f8ca4076480 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:21:20.620 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:20.618+0000 7f8ca9200640 1 -- 172.21.15.47:0/29422416 >> 172.21.15.47:0/29422416 conn(0x7f8ca40fee50 msgr2=0x7f8ca4101290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:21:20.620 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:20.619+0000 7f8ca9200640 1 -- 172.21.15.47:0/29422416 shutdown_connections 2024-03-21T10:21:20.622 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:20.619+0000 7f8ca9200640 1 -- 172.21.15.47:0/29422416 wait complete. 2024-03-21T10:21:20.622 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:20.621+0000 7f8ca9200640 1 Processor -- start 2024-03-21T10:21:20.622 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:20.621+0000 7f8ca9200640 1 -- start start 2024-03-21T10:21:20.623 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:20.622+0000 7f8ca9200640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8ca4076060 0x7f8ca4106b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:21:20.623 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:20.622+0000 7f8ca9200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8ca40772a0 0x7f8ca41070b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:21:20.623 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:20.622+0000 7f8ca9200640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8ca4077c40 0x7f8ca410e120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:21:20.623 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:20.622+0000 7f8ca9200640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ca4103a30 con 0x7f8ca4077c40 2024-03-21T10:21:20.623 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:20.622+0000 7f8ca9200640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ca41038b0 con 0x7f8ca40772a0 2024-03-21T10:21:20.623 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:20.622+0000 7f8ca9200640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ca4103bb0 con 0x7f8ca4076060 2024-03-21T10:21:20.624 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:20.623+0000 7f8ca1e00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8ca4076060 0x7f8ca4106b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:21:20.624 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:20.623+0000 7f8ca1e00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8ca4076060 0x7f8ca4106b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.47:36018/0 (socket says 172.21.15.47:36018) 2024-03-21T10:21:20.624 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:20.623+0000 7f8ca1e00640 1 -- 172.21.15.47:0/689317356 learned_addr learned my addr 172.21.15.47:0/689317356 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:21:20.624 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:20.623+0000 7f8ca2800640 1 --2- 172.21.15.47:0/689317356 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8ca4077c40 0x7f8ca410e120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:21:20.625 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:20.623+0000 7f8ca1400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8ca40772a0 0x7f8ca41070b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:21:20.625 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:20.624+0000 7f8ca1e00640 1 -- 172.21.15.47:0/689317356 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8ca40772a0 msgr2=0x7f8ca41070b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:21:20.625 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:20.624+0000 7f8ca1e00640 1 --2- 172.21.15.47:0/689317356 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8ca40772a0 0x7f8ca41070b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:21:20.625 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:20.624+0000 7f8ca1e00640 1 -- 172.21.15.47:0/689317356 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8ca4077c40 msgr2=0x7f8ca410e120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:21:20.625 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:20.624+0000 7f8ca1e00640 1 --2- 172.21.15.47:0/689317356 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8ca4077c40 0x7f8ca410e120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:21:20.626 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:20.624+0000 7f8ca1e00640 1 -- 172.21.15.47:0/689317356 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ca410e790 con 0x7f8ca4076060 2024-03-21T10:21:20.626 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:20.624+0000 7f8ca2800640 1 --2- 172.21.15.47:0/689317356 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8ca4077c40 0x7f8ca410e120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T10:21:20.626 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:20.625+0000 7f8ca1e00640 1 --2- 172.21.15.47:0/689317356 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8ca4076060 0x7f8ca4106b70 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f8c94000c00 tx=0x7f8c94004270 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:21:20.627 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:20.626+0000 7f8c9aa00640 1 -- 172.21.15.47:0/689317356 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c94043020 con 0x7f8ca4076060 2024-03-21T10:21:20.627 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:20.626+0000 7f8c9aa00640 1 -- 172.21.15.47:0/689317356 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f8c940052b0 con 0x7f8ca4076060 2024-03-21T10:21:20.627 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:20.626+0000 7f8c9aa00640 1 -- 172.21.15.47:0/689317356 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c94043020 con 0x7f8ca4076060 2024-03-21T10:21:20.627 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:20.626+0000 7f8ca9200640 1 -- 172.21.15.47:0/689317356 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8ca410ea20 con 0x7f8ca4076060 2024-03-21T10:21:20.629 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:20.627+0000 7f8ca9200640 1 -- 172.21.15.47:0/689317356 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8ca410ee60 con 0x7f8ca4076060 2024-03-21T10:21:20.630 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:20.628+0000 7f8ca9200640 1 -- 172.21.15.47:0/689317356 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8ca406f960 con 0x7f8ca4076060 2024-03-21T10:21:20.634 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:20.633+0000 7f8c9aa00640 1 -- 172.21.15.47:0/689317356 <== mon.1 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 16) v1 ==== 137854+0+0 (secure 0 0 0) 0x7f8c94046020 con 0x7f8ca4076060 2024-03-21T10:21:20.635 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:20.634+0000 7f8c9aa00640 1 --2- 172.21.15.47:0/689317356 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f8c880a31e0 0x7f8c880a56a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:21:20.635 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:20.635+0000 7f8c9aa00640 1 -- 172.21.15.47:0/689317356 <== mon.1 v2:172.21.15.157:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f8c940f1170 con 0x7f8ca4076060 2024-03-21T10:21:20.636 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:20.635+0000 7f8ca1400640 1 --2- 172.21.15.47:0/689317356 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f8c880a31e0 0x7f8c880a56a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:21:20.637 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:20.635+0000 7f8ca1400640 1 --2- 172.21.15.47:0/689317356 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f8c880a31e0 0x7f8c880a56a0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f8ca4108080 tx=0x7f8c90005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:21:20.647 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:20.646+0000 7f8c9aa00640 1 -- 172.21.15.47:0/689317356 <== mon.1 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f8c9404d020 con 0x7f8ca4076060 2024-03-21T10:21:20.896 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:20.895+0000 7f8ca9200640 1 -- 172.21.15.47:0/689317356 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f8ca402d070 con 0x7f8ca4076060 2024-03-21T10:21:20.898 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:20.897+0000 7f8c9aa00640 1 -- 172.21.15.47:0/689317356 <== mon.1 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 4 v4) v1 ==== 93+0+2910 (secure 0 0 0) 0x7f8c940be3a0 con 0x7f8ca4076060 2024-03-21T10:21:20.898 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-21T10:21:20.898 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":4,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14511,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6835/735724037","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6834","nonce":735724037},{"type":"v1","addr":"172.21.15.47:6835","nonce":735724037}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":24406,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6833/4000010136","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":4000010136},{"type":"v1","addr":"172.21.15.157:6833","nonce":4000010136}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":34427,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6833/2723302038","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6832","nonce":2723302038},{"type":"v1","addr":"172.21.15.100:6833","nonce":2723302038}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[]} 2024-03-21T10:21:20.898 INFO:teuthology.orchestra.run.smithi047.stderr:dumped fsmap epoch 4 2024-03-21T10:21:20.901 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:20.900+0000 7f8ca9200640 1 -- 172.21.15.47:0/689317356 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f8c880a31e0 msgr2=0x7f8c880a56a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:21:20.901 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:20.900+0000 7f8ca9200640 1 --2- 172.21.15.47:0/689317356 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f8c880a31e0 0x7f8c880a56a0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f8ca4108080 tx=0x7f8c90005f90 comp rx=0 tx=0).stop 2024-03-21T10:21:20.901 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:20.900+0000 7f8ca9200640 1 -- 172.21.15.47:0/689317356 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8ca4076060 msgr2=0x7f8ca4106b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:21:20.901 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:20.900+0000 7f8ca9200640 1 --2- 172.21.15.47:0/689317356 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8ca4076060 0x7f8ca4106b70 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f8c94000c00 tx=0x7f8c94004270 comp rx=0 tx=0).stop 2024-03-21T10:21:20.901 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:20.900+0000 7f8ca9200640 1 -- 172.21.15.47:0/689317356 shutdown_connections 2024-03-21T10:21:20.902 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:20.900+0000 7f8ca9200640 1 --2- 172.21.15.47:0/689317356 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f8c880a31e0 0x7f8c880a56a0 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:21:20.902 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:20.900+0000 7f8ca9200640 1 --2- 172.21.15.47:0/689317356 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8ca4077c40 0x7f8ca410e120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:21:20.902 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:20.900+0000 7f8ca9200640 1 --2- 172.21.15.47:0/689317356 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8ca40772a0 0x7f8ca41070b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:21:20.902 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:20.900+0000 7f8ca9200640 1 --2- 172.21.15.47:0/689317356 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8ca4076060 0x7f8ca4106b70 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:21:20.902 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:20.900+0000 7f8ca9200640 1 -- 172.21.15.47:0/689317356 >> 172.21.15.47:0/689317356 conn(0x7f8ca40fee50 msgr2=0x7f8ca4100560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:21:20.902 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:20.900+0000 7f8ca9200640 1 -- 172.21.15.47:0/689317356 shutdown_connections 2024-03-21T10:21:20.902 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:20.900+0000 7f8ca9200640 1 -- 172.21.15.47:0/689317356 wait complete. 2024-03-21T10:21:21.465 INFO:journalctl@ceph.mds.d.smithi047.stdout:Mar 21 10:21:21 smithi047 podman[70087]: 2024-03-21 10:21:21.232449892 +0000 UTC m=+0.154005334 container create 4ae97ca5479a46279de4f2018d9df8327e903391c120cbc5d5fbb89dc48ae6e8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-3056d51a-e76b-11ee-95cd-87774f69a715-mds-d, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, ceph=True, GIT_BRANCH=HEAD, org.label-schema.build-date=20240319, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_CLEAN=True) 2024-03-21T10:21:21.847 INFO:journalctl@ceph.mds.d.smithi047.stdout:Mar 21 10:21:21 smithi047 podman[70087]: 2024-03-21 10:21:21.493893422 +0000 UTC m=+0.415448924 container init 4ae97ca5479a46279de4f2018d9df8327e903391c120cbc5d5fbb89dc48ae6e8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-3056d51a-e76b-11ee-95cd-87774f69a715-mds-d, org.label-schema.build-date=20240319, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_CLEAN=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6) 2024-03-21T10:21:21.847 INFO:journalctl@ceph.mds.d.smithi047.stdout:Mar 21 10:21:21 smithi047 podman[70087]: 2024-03-21 10:21:21.504360015 +0000 UTC m=+0.425915453 container start 4ae97ca5479a46279de4f2018d9df8327e903391c120cbc5d5fbb89dc48ae6e8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-3056d51a-e76b-11ee-95cd-87774f69a715-mds-d, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.build-date=20240319, CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, ceph=True, org.label-schema.vendor=CentOS) 2024-03-21T10:21:21.847 INFO:journalctl@ceph.mds.d.smithi047.stdout:Mar 21 10:21:21 smithi047 ceph-mds[70106]: -- 172.21.15.47:0/3590282888 <== mon.0 v2:172.21.15.47:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55a40c40fa40 con 0x55a40c3e2800 2024-03-21T10:21:21.847 INFO:journalctl@ceph.mds.d.smithi047.stdout:Mar 21 10:21:21 smithi047 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-mds-d[70102]: starting mds.d at 2024-03-21T10:21:21.848 INFO:journalctl@ceph.mds.d.smithi047.stdout:Mar 21 10:21:21 smithi047 bash[70087]: 4ae97ca5479a46279de4f2018d9df8327e903391c120cbc5d5fbb89dc48ae6e8 2024-03-21T10:21:21.848 INFO:journalctl@ceph.mds.d.smithi047.stdout:Mar 21 10:21:21 smithi047 systemd[1]: Started Ceph mds.d for 3056d51a-e76b-11ee-95cd-87774f69a715. 2024-03-21T10:21:22.214 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:21:21 smithi047 ceph-mon[31547]: pgmap v340: 1 pgs: 1 active+clean; 577 KiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-21T10:21:22.214 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:21:21 smithi047 ceph-mon[31547]: from='client.? 172.21.15.47:0/689317356' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-21T10:21:22.214 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:21:21 smithi047 ceph-mon[31547]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:21:22.214 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:21:21 smithi047 ceph-mon[31547]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:21:22.214 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:21:21 smithi047 ceph-mon[31547]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:21:22.215 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:21:21 smithi047 ceph-mon[31547]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-21T10:21:22.215 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:21:21 smithi047 ceph-mon[31547]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-21T10:21:22.215 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:21:21 smithi047 ceph-mon[31547]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T10:21:22.229 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:21:21 smithi100 ceph-mon[35193]: pgmap v340: 1 pgs: 1 active+clean; 577 KiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-21T10:21:22.230 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:21:21 smithi100 ceph-mon[35193]: from='client.? 172.21.15.47:0/689317356' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-21T10:21:22.230 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:21:21 smithi100 ceph-mon[35193]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:21:22.230 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:21:21 smithi100 ceph-mon[35193]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:21:22.230 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:21:21 smithi100 ceph-mon[35193]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:21:22.230 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:21:21 smithi100 ceph-mon[35193]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-21T10:21:22.230 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:21:21 smithi100 ceph-mon[35193]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-21T10:21:22.230 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:21:21 smithi100 ceph-mon[35193]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T10:21:22.242 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:21:21 smithi157 ceph-mon[35581]: pgmap v340: 1 pgs: 1 active+clean; 577 KiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-21T10:21:22.242 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:21:21 smithi157 ceph-mon[35581]: from='client.? 172.21.15.47:0/689317356' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-21T10:21:22.242 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:21:21 smithi157 ceph-mon[35581]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:21:22.242 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:21:21 smithi157 ceph-mon[35581]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:21:22.242 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:21:21 smithi157 ceph-mon[35581]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:21:22.242 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:21:21 smithi157 ceph-mon[35581]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-21T10:21:22.242 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:21:21 smithi157 ceph-mon[35581]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-21T10:21:22.243 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:21:21 smithi157 ceph-mon[35581]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T10:21:22.922 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-21T10:21:23.042 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:21:22 smithi100 ceph-mon[35193]: Deploying daemon mds.e on smithi100 2024-03-21T10:21:23.042 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:21:22 smithi100 ceph-mon[35193]: mds.? [v2:172.21.15.47:6836/2248191724,v1:172.21.15.47:6837/2248191724] up:boot 2024-03-21T10:21:23.042 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:21:22 smithi100 ceph-mon[35193]: fsmap 4 up:standby 2024-03-21T10:21:23.042 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:21:22 smithi100 ceph-mon[35193]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-03-21T10:21:23.116 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:21:22 smithi047 ceph-mon[31547]: Deploying daemon mds.e on smithi100 2024-03-21T10:21:23.116 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:21:22 smithi047 ceph-mon[31547]: mds.? [v2:172.21.15.47:6836/2248191724,v1:172.21.15.47:6837/2248191724] up:boot 2024-03-21T10:21:23.116 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:21:22 smithi047 ceph-mon[31547]: fsmap 4 up:standby 2024-03-21T10:21:23.116 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:21:22 smithi047 ceph-mon[31547]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-03-21T10:21:23.147 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:23.146+0000 7f550ee00640 1 -- 172.21.15.47:0/2844998900 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f55081038e0 msgr2=0x7f5508103d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:21:23.148 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:23.146+0000 7f550ee00640 1 --2- 172.21.15.47:0/2844998900 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f55081038e0 0x7f5508103d00 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f54f8009900 tx=0x7f54f802bc30 comp rx=0 tx=0).stop 2024-03-21T10:21:23.148 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:23.146+0000 7f550ee00640 1 -- 172.21.15.47:0/2844998900 shutdown_connections 2024-03-21T10:21:23.148 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:23.146+0000 7f550ee00640 1 --2- 172.21.15.47:0/2844998900 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5508105490 0x7f550810bd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:21:23.148 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:23.146+0000 7f550ee00640 1 --2- 172.21.15.47:0/2844998900 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5508104ad0 0x7f5508104f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:21:23.148 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:23.146+0000 7f550ee00640 1 --2- 172.21.15.47:0/2844998900 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f55081038e0 0x7f5508103d00 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:21:23.148 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:23.146+0000 7f550ee00640 1 -- 172.21.15.47:0/2844998900 >> 172.21.15.47:0/2844998900 conn(0x7f55080fee50 msgr2=0x7f5508101290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:21:23.148 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:23.147+0000 7f550ee00640 1 -- 172.21.15.47:0/2844998900 shutdown_connections 2024-03-21T10:21:23.148 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:23.147+0000 7f550ee00640 1 -- 172.21.15.47:0/2844998900 wait complete. 2024-03-21T10:21:23.150 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:23.148+0000 7f550ee00640 1 Processor -- start 2024-03-21T10:21:23.150 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:23.148+0000 7f550ee00640 1 -- start start 2024-03-21T10:21:23.150 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:23.148+0000 7f550ee00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f55081038e0 0x7f5508199b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:21:23.150 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:23.148+0000 7f550ee00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5508104ad0 0x7f550819a050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:21:23.151 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:23.148+0000 7f550ee00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5508105490 0x7f55081a10c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:21:23.151 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:23.148+0000 7f550ee00640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f550810f3e0 con 0x7f5508104ad0 2024-03-21T10:21:23.151 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:23.148+0000 7f550ee00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f550810f260 con 0x7f55081038e0 2024-03-21T10:21:23.151 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:23.148+0000 7f550ee00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f550810f560 con 0x7f5508105490 2024-03-21T10:21:23.151 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:23.149+0000 7f5507400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5508104ad0 0x7f550819a050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:21:23.151 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:23.149+0000 7f5507400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5508104ad0 0x7f550819a050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:44958/0 (socket says 172.21.15.47:44958) 2024-03-21T10:21:23.151 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:23.149+0000 7f5507400640 1 -- 172.21.15.47:0/2555028505 learned_addr learned my addr 172.21.15.47:0/2555028505 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:21:23.151 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:23.149+0000 7f5507e00640 1 --2- 172.21.15.47:0/2555028505 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f55081038e0 0x7f5508199b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:21:23.151 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:23.149+0000 7f550ce00640 1 --2- 172.21.15.47:0/2555028505 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5508105490 0x7f55081a10c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:21:23.151 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:23.149+0000 7f5507400640 1 -- 172.21.15.47:0/2555028505 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5508105490 msgr2=0x7f55081a10c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:21:23.151 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:23.149+0000 7f5507400640 1 --2- 172.21.15.47:0/2555028505 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5508105490 0x7f55081a10c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:21:23.151 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:23.149+0000 7f5507400640 1 -- 172.21.15.47:0/2555028505 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f55081038e0 msgr2=0x7f5508199b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:21:23.151 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:23.149+0000 7f5507400640 1 --2- 172.21.15.47:0/2555028505 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f55081038e0 0x7f5508199b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:21:23.152 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:23.149+0000 7f5507400640 1 -- 172.21.15.47:0/2555028505 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f55081a1730 con 0x7f5508104ad0 2024-03-21T10:21:23.152 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:23.149+0000 7f550ce00640 1 --2- 172.21.15.47:0/2555028505 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5508105490 0x7f55081a10c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T10:21:23.152 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:23.149+0000 7f5507400640 1 --2- 172.21.15.47:0/2555028505 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5508104ad0 0x7f550819a050 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f54f400e9f0 tx=0x7f54f400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:21:23.152 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:23.150+0000 7f5504c00640 1 -- 172.21.15.47:0/2555028505 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54f400cd80 con 0x7f5508104ad0 2024-03-21T10:21:23.152 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:23.150+0000 7f5504c00640 1 -- 172.21.15.47:0/2555028505 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f54f40108e0 con 0x7f5508104ad0 2024-03-21T10:21:23.153 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:23.150+0000 7f550ee00640 1 -- 172.21.15.47:0/2555028505 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f55081a1a20 con 0x7f5508104ad0 2024-03-21T10:21:23.153 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:23.150+0000 7f5504c00640 1 -- 172.21.15.47:0/2555028505 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54f4018920 con 0x7f5508104ad0 2024-03-21T10:21:23.153 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:23.150+0000 7f550ee00640 1 -- 172.21.15.47:0/2555028505 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f55081a1fc0 con 0x7f5508104ad0 2024-03-21T10:21:23.153 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:23.151+0000 7f550ee00640 1 -- 172.21.15.47:0/2555028505 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f550810ac80 con 0x7f5508104ad0 2024-03-21T10:21:23.155 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:23.154+0000 7f5504c00640 1 -- 172.21.15.47:0/2555028505 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 16) v1 ==== 137854+0+0 (secure 0 0 0) 0x7f54f4010a90 con 0x7f5508104ad0 2024-03-21T10:21:23.157 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:23.156+0000 7f5504c00640 1 --2- 172.21.15.47:0/2555028505 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f54ec0a2fa0 0x7f54ec0a5460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:21:23.158 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:23.157+0000 7f5504c00640 1 -- 172.21.15.47:0/2555028505 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f54f40cf390 con 0x7f5508104ad0 2024-03-21T10:21:23.158 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:23.157+0000 7f5507e00640 1 --2- 172.21.15.47:0/2555028505 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f54ec0a2fa0 0x7f54ec0a5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:21:23.158 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:23.158+0000 7f5507e00640 1 --2- 172.21.15.47:0/2555028505 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f54ec0a2fa0 0x7f54ec0a5460 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f54f8006010 tx=0x7f54f8005be0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:21:23.162 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:23.162+0000 7f5504c00640 1 -- 172.21.15.47:0/2555028505 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f54f409c5c0 con 0x7f5508104ad0 2024-03-21T10:21:23.240 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:21:22 smithi157 ceph-mon[35581]: Deploying daemon mds.e on smithi100 2024-03-21T10:21:23.240 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:21:22 smithi157 ceph-mon[35581]: mds.? [v2:172.21.15.47:6836/2248191724,v1:172.21.15.47:6837/2248191724] up:boot 2024-03-21T10:21:23.240 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:21:22 smithi157 ceph-mon[35581]: fsmap 4 up:standby 2024-03-21T10:21:23.240 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:21:22 smithi157 ceph-mon[35581]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-03-21T10:21:23.409 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:23.408+0000 7f550ee00640 1 -- 172.21.15.47:0/2555028505 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f550802d070 con 0x7f5508104ad0 2024-03-21T10:21:23.410 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:23.409+0000 7f5504c00640 1 -- 172.21.15.47:0/2555028505 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 5 v5) v1 ==== 93+0+3701 (secure 0 0 0) 0x7f54f409bec0 con 0x7f5508104ad0 2024-03-21T10:21:23.411 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-21T10:21:23.411 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":5,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14511,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6835/735724037","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6834","nonce":735724037},{"type":"v1","addr":"172.21.15.47:6835","nonce":735724037}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14529,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6837/2248191724","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6836","nonce":2248191724},{"type":"v1","addr":"172.21.15.47:6837","nonce":2248191724}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":24406,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6833/4000010136","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":4000010136},{"type":"v1","addr":"172.21.15.157:6833","nonce":4000010136}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":34427,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6833/2723302038","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6832","nonce":2723302038},{"type":"v1","addr":"172.21.15.100:6833","nonce":2723302038}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[]} 2024-03-21T10:21:23.411 INFO:teuthology.orchestra.run.smithi047.stderr:dumped fsmap epoch 5 2024-03-21T10:21:23.413 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:23.412+0000 7f550ee00640 1 -- 172.21.15.47:0/2555028505 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f54ec0a2fa0 msgr2=0x7f54ec0a5460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:21:23.413 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:23.412+0000 7f550ee00640 1 --2- 172.21.15.47:0/2555028505 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f54ec0a2fa0 0x7f54ec0a5460 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f54f8006010 tx=0x7f54f8005be0 comp rx=0 tx=0).stop 2024-03-21T10:21:23.413 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:23.412+0000 7f550ee00640 1 -- 172.21.15.47:0/2555028505 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5508104ad0 msgr2=0x7f550819a050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:21:23.413 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:23.412+0000 7f550ee00640 1 --2- 172.21.15.47:0/2555028505 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5508104ad0 0x7f550819a050 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f54f400e9f0 tx=0x7f54f400eeb0 comp rx=0 tx=0).stop 2024-03-21T10:21:23.414 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:23.412+0000 7f550ee00640 1 -- 172.21.15.47:0/2555028505 shutdown_connections 2024-03-21T10:21:23.414 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:23.412+0000 7f550ee00640 1 --2- 172.21.15.47:0/2555028505 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f54ec0a2fa0 0x7f54ec0a5460 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:21:23.414 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:23.412+0000 7f550ee00640 1 --2- 172.21.15.47:0/2555028505 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5508105490 0x7f55081a10c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:21:23.414 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:23.412+0000 7f550ee00640 1 --2- 172.21.15.47:0/2555028505 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5508104ad0 0x7f550819a050 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:21:23.414 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:23.412+0000 7f550ee00640 1 --2- 172.21.15.47:0/2555028505 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f55081038e0 0x7f5508199b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:21:23.414 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:23.412+0000 7f550ee00640 1 -- 172.21.15.47:0/2555028505 >> 172.21.15.47:0/2555028505 conn(0x7f55080fee50 msgr2=0x7f5508107e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:21:23.414 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:23.412+0000 7f550ee00640 1 -- 172.21.15.47:0/2555028505 shutdown_connections 2024-03-21T10:21:23.414 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:23.412+0000 7f550ee00640 1 -- 172.21.15.47:0/2555028505 wait complete. 2024-03-21T10:21:24.214 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:21:23 smithi047 ceph-mon[31547]: pgmap v341: 1 pgs: 1 active+clean; 577 KiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-21T10:21:24.214 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:21:23 smithi047 ceph-mon[31547]: from='client.? 172.21.15.47:0/2555028505' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-21T10:21:24.226 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:21:23 smithi100 ceph-mon[35193]: pgmap v341: 1 pgs: 1 active+clean; 577 KiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-21T10:21:24.226 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:21:23 smithi100 ceph-mon[35193]: from='client.? 172.21.15.47:0/2555028505' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-21T10:21:24.239 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:21:23 smithi157 ceph-mon[35581]: pgmap v341: 1 pgs: 1 active+clean; 577 KiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-21T10:21:24.240 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:21:23 smithi157 ceph-mon[35581]: from='client.? 172.21.15.47:0/2555028505' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-21T10:21:25.230 INFO:journalctl@ceph.mds.e.smithi100.stdout:Mar 21 10:21:24 smithi100 systemd[1]: Starting Ceph mds.e for 3056d51a-e76b-11ee-95cd-87774f69a715... 2024-03-21T10:21:25.432 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-21T10:21:25.587 INFO:journalctl@ceph.mds.e.smithi100.stdout:Mar 21 10:21:25 smithi100 podman[64364]: 2024-03-21 10:21:25.193160955 +0000 UTC m=+0.032989133 image pull 237e43c7a65ff48534d94b97c1bfdbe4b7be5672e3d9a613745083ea3702aa54 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232 2024-03-21T10:21:25.587 INFO:journalctl@ceph.mds.e.smithi100.stdout:Mar 21 10:21:25 smithi100 podman[64364]: 2024-03-21 10:21:25.314007663 +0000 UTC m=+0.153835867 container create bce3049fc80580804e3e6d25b54faf5cb48c904d63c538381b47310e4d1ff714 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-3056d51a-e76b-11ee-95cd-87774f69a715-mds-e, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, io.buildah.version=1.33.5, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_BRANCH=HEAD, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240319, GIT_CLEAN=True) 2024-03-21T10:21:25.587 INFO:journalctl@ceph.mds.e.smithi100.stdout:Mar 21 10:21:25 smithi100 podman[64364]: 2024-03-21 10:21:25.584935738 +0000 UTC m=+0.424763945 container init bce3049fc80580804e3e6d25b54faf5cb48c904d63c538381b47310e4d1ff714 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-3056d51a-e76b-11ee-95cd-87774f69a715-mds-e, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240319, GIT_BRANCH=HEAD, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, ceph=True) 2024-03-21T10:21:25.666 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:25.665+0000 7f64fba00640 1 -- 172.21.15.47:0/3927962380 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f64f4108ed0 msgr2=0x7f64f4109350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:21:25.667 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:25.665+0000 7f64fba00640 1 --2- 172.21.15.47:0/3927962380 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f64f4108ed0 0x7f64f4109350 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f64e0009900 tx=0x7f64e002bc30 comp rx=0 tx=0).stop 2024-03-21T10:21:25.667 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:25.665+0000 7f64fba00640 1 -- 172.21.15.47:0/3927962380 shutdown_connections 2024-03-21T10:21:25.667 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:25.665+0000 7f64fba00640 1 --2- 172.21.15.47:0/3927962380 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f64f4109890 0x7f64f4110120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:21:25.667 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:25.665+0000 7f64fba00640 1 --2- 172.21.15.47:0/3927962380 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f64f4108ed0 0x7f64f4109350 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:21:25.667 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:25.665+0000 7f64fba00640 1 --2- 172.21.15.47:0/3927962380 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f64f4107ce0 0x7f64f4108100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:21:25.667 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:25.665+0000 7f64fba00640 1 -- 172.21.15.47:0/3927962380 >> 172.21.15.47:0/3927962380 conn(0x7f64f4076cb0 msgr2=0x7f64f40790d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:21:25.667 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:25.665+0000 7f64fba00640 1 -- 172.21.15.47:0/3927962380 shutdown_connections 2024-03-21T10:21:25.667 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:25.665+0000 7f64fba00640 1 -- 172.21.15.47:0/3927962380 wait complete. 2024-03-21T10:21:25.669 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:25.666+0000 7f64fba00640 1 Processor -- start 2024-03-21T10:21:25.670 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:25.667+0000 7f64fba00640 1 -- start start 2024-03-21T10:21:25.670 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:25.667+0000 7f64fba00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f64f4107ce0 0x7f64f419df10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:21:25.670 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:25.667+0000 7f64fba00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f64f4108ed0 0x7f64f419e450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:21:25.670 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:25.667+0000 7f64fba00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f64f4109890 0x7f64f41a54c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:21:25.670 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:25.667+0000 7f64fba00640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64f4113810 con 0x7f64f4107ce0 2024-03-21T10:21:25.670 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:25.667+0000 7f64fba00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64f4113690 con 0x7f64f4109890 2024-03-21T10:21:25.670 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:25.667+0000 7f64fba00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64f4113990 con 0x7f64f4108ed0 2024-03-21T10:21:25.670 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:25.667+0000 7f64f9000640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f64f4107ce0 0x7f64f419df10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:21:25.670 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:25.667+0000 7f64f9000640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f64f4107ce0 0x7f64f419df10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:44986/0 (socket says 172.21.15.47:44986) 2024-03-21T10:21:25.670 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:25.667+0000 7f64f9000640 1 -- 172.21.15.47:0/2429652328 learned_addr learned my addr 172.21.15.47:0/2429652328 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:21:25.670 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:25.667+0000 7f64f9a00640 1 --2- 172.21.15.47:0/2429652328 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f64f4109890 0x7f64f41a54c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:21:25.670 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:25.667+0000 7f64f3e00640 1 --2- 172.21.15.47:0/2429652328 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f64f4108ed0 0x7f64f419e450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:21:25.671 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:25.668+0000 7f64f9000640 1 -- 172.21.15.47:0/2429652328 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f64f4108ed0 msgr2=0x7f64f419e450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:21:25.671 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:25.668+0000 7f64f9000640 1 --2- 172.21.15.47:0/2429652328 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f64f4108ed0 0x7f64f419e450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:21:25.671 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:25.668+0000 7f64f9000640 1 -- 172.21.15.47:0/2429652328 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f64f4109890 msgr2=0x7f64f41a54c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:21:25.671 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:25.668+0000 7f64f9000640 1 --2- 172.21.15.47:0/2429652328 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f64f4109890 0x7f64f41a54c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:21:25.671 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:25.668+0000 7f64f9000640 1 -- 172.21.15.47:0/2429652328 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64f41a5b30 con 0x7f64f4107ce0 2024-03-21T10:21:25.671 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:25.668+0000 7f64f9000640 1 --2- 172.21.15.47:0/2429652328 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f64f4107ce0 0x7f64f419df10 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f64e800e9c0 tx=0x7f64e800ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:21:25.671 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:25.668+0000 7f64f1600640 1 -- 172.21.15.47:0/2429652328 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64e800cde0 con 0x7f64f4107ce0 2024-03-21T10:21:25.671 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:25.668+0000 7f64f1600640 1 -- 172.21.15.47:0/2429652328 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f64e80108e0 con 0x7f64f4107ce0 2024-03-21T10:21:25.671 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:25.668+0000 7f64fba00640 1 -- 172.21.15.47:0/2429652328 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f64f41a5e20 con 0x7f64f4107ce0 2024-03-21T10:21:25.671 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:25.668+0000 7f64f1600640 1 -- 172.21.15.47:0/2429652328 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64e8018990 con 0x7f64f4107ce0 2024-03-21T10:21:25.671 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:25.669+0000 7f64fba00640 1 -- 172.21.15.47:0/2429652328 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f64f41a6280 con 0x7f64f4107ce0 2024-03-21T10:21:25.672 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:25.670+0000 7f64fba00640 1 -- 172.21.15.47:0/2429652328 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f64f410f020 con 0x7f64f4107ce0 2024-03-21T10:21:25.673 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:25.672+0000 7f64f1600640 1 -- 172.21.15.47:0/2429652328 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 16) v1 ==== 137854+0+0 (secure 0 0 0) 0x7f64e801f020 con 0x7f64f4107ce0 2024-03-21T10:21:25.674 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:25.673+0000 7f64f1600640 1 --2- 172.21.15.47:0/2429652328 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f64dc0a2f20 0x7f64dc0a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:21:25.674 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:25.673+0000 7f64f1600640 1 -- 172.21.15.47:0/2429652328 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f64e80cf220 con 0x7f64f4107ce0 2024-03-21T10:21:25.674 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:25.673+0000 7f64f3e00640 1 --2- 172.21.15.47:0/2429652328 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f64dc0a2f20 0x7f64dc0a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:21:25.675 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:25.674+0000 7f64f3e00640 1 --2- 172.21.15.47:0/2429652328 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f64dc0a2f20 0x7f64dc0a53e0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f64f419f420 tx=0x7f64e0005be0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:21:25.679 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:25.678+0000 7f64f1600640 1 -- 172.21.15.47:0/2429652328 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f64e809c450 con 0x7f64f4107ce0 2024-03-21T10:21:25.873 INFO:journalctl@ceph.mds.e.smithi100.stdout:Mar 21 10:21:25 smithi100 podman[64364]: 2024-03-21 10:21:25.596353112 +0000 UTC m=+0.436181284 container start bce3049fc80580804e3e6d25b54faf5cb48c904d63c538381b47310e4d1ff714 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-3056d51a-e76b-11ee-95cd-87774f69a715-mds-e, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240319, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, ceph=True, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30) 2024-03-21T10:21:25.873 INFO:journalctl@ceph.mds.e.smithi100.stdout:Mar 21 10:21:25 smithi100 ceph-mds[64383]: -- 172.21.15.100:0/704629293 <== mon.2 v2:172.21.15.100:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x5586275cfa40 con 0x5586275a2c00 2024-03-21T10:21:25.873 INFO:journalctl@ceph.mds.e.smithi100.stdout:Mar 21 10:21:25 smithi100 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-mds-e[64379]: starting mds.e at 2024-03-21T10:21:25.873 INFO:journalctl@ceph.mds.e.smithi100.stdout:Mar 21 10:21:25 smithi100 bash[64364]: bce3049fc80580804e3e6d25b54faf5cb48c904d63c538381b47310e4d1ff714 2024-03-21T10:21:25.874 INFO:journalctl@ceph.mds.e.smithi100.stdout:Mar 21 10:21:25 smithi100 systemd[1]: Started Ceph mds.e for 3056d51a-e76b-11ee-95cd-87774f69a715. 2024-03-21T10:21:25.932 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:25.931+0000 7f64fba00640 1 -- 172.21.15.47:0/2429652328 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f64f4061ae0 con 0x7f64f4107ce0 2024-03-21T10:21:25.933 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:25.932+0000 7f64f1600640 1 -- 172.21.15.47:0/2429652328 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 6 v6) v1 ==== 93+0+4495 (secure 0 0 0) 0x7f64e809bd50 con 0x7f64f4107ce0 2024-03-21T10:21:25.933 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-21T10:21:25.933 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":6,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14511,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6835/735724037","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6834","nonce":735724037},{"type":"v1","addr":"172.21.15.47:6835","nonce":735724037}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14529,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6837/2248191724","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6836","nonce":2248191724},{"type":"v1","addr":"172.21.15.47:6837","nonce":2248191724}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":24406,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6833/4000010136","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":4000010136},{"type":"v1","addr":"172.21.15.157:6833","nonce":4000010136}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":34427,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6833/2723302038","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6832","nonce":2723302038},{"type":"v1","addr":"172.21.15.100:6833","nonce":2723302038}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":34454,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6835/2597849957","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":2597849957},{"type":"v1","addr":"172.21.15.100:6835","nonce":2597849957}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6}],"filesystems":[]} 2024-03-21T10:21:25.933 INFO:teuthology.orchestra.run.smithi047.stderr:dumped fsmap epoch 6 2024-03-21T10:21:25.936 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:25.935+0000 7f64fba00640 1 -- 172.21.15.47:0/2429652328 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f64dc0a2f20 msgr2=0x7f64dc0a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:21:25.936 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:25.935+0000 7f64fba00640 1 --2- 172.21.15.47:0/2429652328 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f64dc0a2f20 0x7f64dc0a53e0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f64f419f420 tx=0x7f64e0005be0 comp rx=0 tx=0).stop 2024-03-21T10:21:25.936 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:25.935+0000 7f64fba00640 1 -- 172.21.15.47:0/2429652328 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f64f4107ce0 msgr2=0x7f64f419df10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:21:25.936 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:25.935+0000 7f64fba00640 1 --2- 172.21.15.47:0/2429652328 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f64f4107ce0 0x7f64f419df10 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f64e800e9c0 tx=0x7f64e800ee80 comp rx=0 tx=0).stop 2024-03-21T10:21:25.936 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:25.935+0000 7f64fba00640 1 -- 172.21.15.47:0/2429652328 shutdown_connections 2024-03-21T10:21:25.936 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:25.935+0000 7f64fba00640 1 --2- 172.21.15.47:0/2429652328 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f64dc0a2f20 0x7f64dc0a53e0 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:21:25.936 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:25.935+0000 7f64fba00640 1 --2- 172.21.15.47:0/2429652328 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f64f4109890 0x7f64f41a54c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:21:25.937 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:25.935+0000 7f64fba00640 1 --2- 172.21.15.47:0/2429652328 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f64f4108ed0 0x7f64f419e450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:21:25.937 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:25.935+0000 7f64fba00640 1 --2- 172.21.15.47:0/2429652328 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f64f4107ce0 0x7f64f419df10 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:21:25.937 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:25.935+0000 7f64fba00640 1 -- 172.21.15.47:0/2429652328 >> 172.21.15.47:0/2429652328 conn(0x7f64f4076cb0 msgr2=0x7f64f410c1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:21:25.937 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:25.935+0000 7f64fba00640 1 -- 172.21.15.47:0/2429652328 shutdown_connections 2024-03-21T10:21:25.937 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:25.935+0000 7f64fba00640 1 -- 172.21.15.47:0/2429652328 wait complete. 2024-03-21T10:21:26.214 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:21:25 smithi047 ceph-mon[31547]: pgmap v342: 1 pgs: 1 active+clean; 577 KiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-21T10:21:26.214 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:21:25 smithi047 ceph-mon[31547]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:21:26.214 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:21:25 smithi047 ceph-mon[31547]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:21:26.230 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:21:25 smithi100 ceph-mon[35193]: pgmap v342: 1 pgs: 1 active+clean; 577 KiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-21T10:21:26.230 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:21:25 smithi100 ceph-mon[35193]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:21:26.230 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:21:25 smithi100 ceph-mon[35193]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:21:26.240 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:21:25 smithi157 ceph-mon[35581]: pgmap v342: 1 pgs: 1 active+clean; 577 KiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-21T10:21:26.240 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:21:25 smithi157 ceph-mon[35581]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:21:26.240 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:21:25 smithi157 ceph-mon[35581]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:21:27.134 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:21:26 smithi157 ceph-mon[35581]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:21:27.134 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:21:26 smithi157 ceph-mon[35581]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-21T10:21:27.134 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:21:26 smithi157 ceph-mon[35581]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-21T10:21:27.134 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:21:26 smithi157 ceph-mon[35581]: mds.? [v2:172.21.15.100:6834/2597849957,v1:172.21.15.100:6835/2597849957] up:boot 2024-03-21T10:21:27.134 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:21:26 smithi157 ceph-mon[35581]: fsmap 5 up:standby 2024-03-21T10:21:27.134 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:21:26 smithi157 ceph-mon[35581]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-03-21T10:21:27.134 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:21:26 smithi157 ceph-mon[35581]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T10:21:27.134 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:21:26 smithi157 ceph-mon[35581]: Deploying daemon mds.f on smithi157 2024-03-21T10:21:27.134 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:21:26 smithi157 ceph-mon[35581]: from='client.? 172.21.15.47:0/2429652328' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-21T10:21:27.134 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:21:26 smithi157 ceph-mon[35581]: pgmap v343: 1 pgs: 1 active+clean; 577 KiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-21T10:21:27.214 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:21:26 smithi047 ceph-mon[31547]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:21:27.214 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:21:26 smithi047 ceph-mon[31547]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-21T10:21:27.214 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:21:26 smithi047 ceph-mon[31547]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-21T10:21:27.214 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:21:26 smithi047 ceph-mon[31547]: mds.? [v2:172.21.15.100:6834/2597849957,v1:172.21.15.100:6835/2597849957] up:boot 2024-03-21T10:21:27.215 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:21:26 smithi047 ceph-mon[31547]: fsmap 5 up:standby 2024-03-21T10:21:27.215 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:21:26 smithi047 ceph-mon[31547]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-03-21T10:21:27.215 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:21:26 smithi047 ceph-mon[31547]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T10:21:27.215 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:21:26 smithi047 ceph-mon[31547]: Deploying daemon mds.f on smithi157 2024-03-21T10:21:27.215 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:21:26 smithi047 ceph-mon[31547]: from='client.? 172.21.15.47:0/2429652328' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-21T10:21:27.215 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:21:26 smithi047 ceph-mon[31547]: pgmap v343: 1 pgs: 1 active+clean; 577 KiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-21T10:21:27.230 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:21:26 smithi100 ceph-mon[35193]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:21:27.230 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:21:26 smithi100 ceph-mon[35193]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-21T10:21:27.230 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:21:26 smithi100 ceph-mon[35193]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-21T10:21:27.230 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:21:26 smithi100 ceph-mon[35193]: mds.? [v2:172.21.15.100:6834/2597849957,v1:172.21.15.100:6835/2597849957] up:boot 2024-03-21T10:21:27.230 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:21:26 smithi100 ceph-mon[35193]: fsmap 5 up:standby 2024-03-21T10:21:27.230 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:21:26 smithi100 ceph-mon[35193]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-03-21T10:21:27.230 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:21:26 smithi100 ceph-mon[35193]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T10:21:27.230 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:21:26 smithi100 ceph-mon[35193]: Deploying daemon mds.f on smithi157 2024-03-21T10:21:27.230 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:21:26 smithi100 ceph-mon[35193]: from='client.? 172.21.15.47:0/2429652328' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-21T10:21:27.230 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:21:26 smithi100 ceph-mon[35193]: pgmap v343: 1 pgs: 1 active+clean; 577 KiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-21T10:21:27.956 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-21T10:21:28.187 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:28.186+0000 7f7a81200640 1 -- 172.21.15.47:0/239531137 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7a7c109890 msgr2=0x7f7a7c110120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:21:28.217 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:28.186+0000 7f7a81200640 1 --2- 172.21.15.47:0/239531137 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7a7c109890 0x7f7a7c110120 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f7a74009ca0 tx=0x7f7a7402c0f0 comp rx=0 tx=0).stop 2024-03-21T10:21:28.218 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:28.186+0000 7f7a81200640 1 -- 172.21.15.47:0/239531137 shutdown_connections 2024-03-21T10:21:28.218 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:28.186+0000 7f7a81200640 1 --2- 172.21.15.47:0/239531137 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7a7c109890 0x7f7a7c110120 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:21:28.218 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:28.186+0000 7f7a81200640 1 --2- 172.21.15.47:0/239531137 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7a7c108ed0 0x7f7a7c109350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:21:28.218 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:28.186+0000 7f7a81200640 1 --2- 172.21.15.47:0/239531137 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7a7c107ce0 0x7f7a7c108100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:21:28.218 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:28.186+0000 7f7a81200640 1 -- 172.21.15.47:0/239531137 >> 172.21.15.47:0/239531137 conn(0x7f7a7c076cb0 msgr2=0x7f7a7c0790d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:21:28.219 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:28.186+0000 7f7a81200640 1 -- 172.21.15.47:0/239531137 shutdown_connections 2024-03-21T10:21:28.219 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:28.186+0000 7f7a81200640 1 -- 172.21.15.47:0/239531137 wait complete. 2024-03-21T10:21:28.223 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:28.187+0000 7f7a81200640 1 Processor -- start 2024-03-21T10:21:28.223 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:28.187+0000 7f7a81200640 1 -- start start 2024-03-21T10:21:28.223 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:28.187+0000 7f7a81200640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7a7c107ce0 0x7f7a7c19de90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:21:28.223 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:28.187+0000 7f7a81200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7a7c108ed0 0x7f7a7c19e3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:21:28.223 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:28.187+0000 7f7a81200640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7a7c109890 0x7f7a7c1a5440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:21:28.224 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:28.188+0000 7f7a81200640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a7c113810 con 0x7f7a7c107ce0 2024-03-21T10:21:28.224 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:28.188+0000 7f7a81200640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a7c113690 con 0x7f7a7c108ed0 2024-03-21T10:21:28.224 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:28.188+0000 7f7a81200640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a7c113990 con 0x7f7a7c109890 2024-03-21T10:21:28.224 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:28.188+0000 7f7a79e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7a7c107ce0 0x7f7a7c19de90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:21:28.224 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:28.188+0000 7f7a79e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7a7c107ce0 0x7f7a7c19de90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:44996/0 (socket says 172.21.15.47:44996) 2024-03-21T10:21:28.224 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:28.188+0000 7f7a79e00640 1 -- 172.21.15.47:0/270064237 learned_addr learned my addr 172.21.15.47:0/270064237 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:21:28.224 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:28.188+0000 7f7a79400640 1 --2- 172.21.15.47:0/270064237 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7a7c108ed0 0x7f7a7c19e3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:21:28.224 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:28.188+0000 7f7a7a800640 1 --2- 172.21.15.47:0/270064237 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7a7c109890 0x7f7a7c1a5440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:21:28.224 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:28.188+0000 7f7a79e00640 1 -- 172.21.15.47:0/270064237 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7a7c109890 msgr2=0x7f7a7c1a5440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:21:28.224 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:28.188+0000 7f7a79e00640 1 --2- 172.21.15.47:0/270064237 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7a7c109890 0x7f7a7c1a5440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:21:28.224 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:28.188+0000 7f7a79e00640 1 -- 172.21.15.47:0/270064237 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7a7c108ed0 msgr2=0x7f7a7c19e3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:21:28.225 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:28.188+0000 7f7a79e00640 1 --2- 172.21.15.47:0/270064237 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7a7c108ed0 0x7f7a7c19e3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:21:28.225 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:28.188+0000 7f7a79e00640 1 -- 172.21.15.47:0/270064237 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a7c1a5ab0 con 0x7f7a7c107ce0 2024-03-21T10:21:28.225 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:28.188+0000 7f7a79400640 1 --2- 172.21.15.47:0/270064237 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7a7c108ed0 0x7f7a7c19e3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T10:21:28.225 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:28.188+0000 7f7a79e00640 1 --2- 172.21.15.47:0/270064237 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7a7c107ce0 0x7f7a7c19de90 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f7a6c00d770 tx=0x7f7a6c00dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:21:28.225 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:28.189+0000 7f7a72a00640 1 -- 172.21.15.47:0/270064237 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a6c017070 con 0x7f7a7c107ce0 2024-03-21T10:21:28.225 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:28.189+0000 7f7a72a00640 1 -- 172.21.15.47:0/270064237 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f7a6c009c30 con 0x7f7a7c107ce0 2024-03-21T10:21:28.225 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:28.189+0000 7f7a81200640 1 -- 172.21.15.47:0/270064237 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7a7c1a5da0 con 0x7f7a7c107ce0 2024-03-21T10:21:28.225 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:28.189+0000 7f7a72a00640 1 -- 172.21.15.47:0/270064237 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a6c003d00 con 0x7f7a7c107ce0 2024-03-21T10:21:28.225 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:28.189+0000 7f7a81200640 1 -- 172.21.15.47:0/270064237 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7a7c1a6260 con 0x7f7a7c107ce0 2024-03-21T10:21:28.225 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:28.190+0000 7f7a81200640 1 -- 172.21.15.47:0/270064237 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7a7c10f000 con 0x7f7a7c107ce0 2024-03-21T10:21:28.226 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:28.192+0000 7f7a72a00640 1 -- 172.21.15.47:0/270064237 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 16) v1 ==== 137854+0+0 (secure 0 0 0) 0x7f7a6c003ea0 con 0x7f7a7c107ce0 2024-03-21T10:21:28.226 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:28.194+0000 7f7a72a00640 1 --2- 172.21.15.47:0/270064237 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f7a600a2fa0 0x7f7a600a5460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:21:28.226 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:28.194+0000 7f7a72a00640 1 -- 172.21.15.47:0/270064237 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f7a6c0ced40 con 0x7f7a7c107ce0 2024-03-21T10:21:28.226 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:28.195+0000 7f7a79400640 1 --2- 172.21.15.47:0/270064237 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f7a600a2fa0 0x7f7a600a5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:21:28.226 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:28.195+0000 7f7a79400640 1 --2- 172.21.15.47:0/270064237 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f7a600a2fa0 0x7f7a600a5460 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f7a680058c0 tx=0x7f7a68005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:21:28.226 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:28.199+0000 7f7a72a00640 1 -- 172.21.15.47:0/270064237 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f7a6c09bf70 con 0x7f7a7c107ce0 2024-03-21T10:21:28.446 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:28.445+0000 7f7a81200640 1 -- 172.21.15.47:0/270064237 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f7a7c061ae0 con 0x7f7a7c107ce0 2024-03-21T10:21:28.448 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:28.447+0000 7f7a72a00640 1 -- 172.21.15.47:0/270064237 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 6 v6) v1 ==== 93+0+4495 (secure 0 0 0) 0x7f7a6c009de0 con 0x7f7a7c107ce0 2024-03-21T10:21:28.448 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-21T10:21:28.448 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":6,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14511,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6835/735724037","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6834","nonce":735724037},{"type":"v1","addr":"172.21.15.47:6835","nonce":735724037}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14529,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6837/2248191724","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6836","nonce":2248191724},{"type":"v1","addr":"172.21.15.47:6837","nonce":2248191724}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":24406,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6833/4000010136","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":4000010136},{"type":"v1","addr":"172.21.15.157:6833","nonce":4000010136}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":34427,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6833/2723302038","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6832","nonce":2723302038},{"type":"v1","addr":"172.21.15.100:6833","nonce":2723302038}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":34454,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6835/2597849957","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":2597849957},{"type":"v1","addr":"172.21.15.100:6835","nonce":2597849957}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6}],"filesystems":[]} 2024-03-21T10:21:28.448 INFO:teuthology.orchestra.run.smithi047.stderr:dumped fsmap epoch 6 2024-03-21T10:21:28.451 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:28.450+0000 7f7a81200640 1 -- 172.21.15.47:0/270064237 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f7a600a2fa0 msgr2=0x7f7a600a5460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:21:28.451 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:28.450+0000 7f7a81200640 1 --2- 172.21.15.47:0/270064237 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f7a600a2fa0 0x7f7a600a5460 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f7a680058c0 tx=0x7f7a68005850 comp rx=0 tx=0).stop 2024-03-21T10:21:28.451 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:28.450+0000 7f7a81200640 1 -- 172.21.15.47:0/270064237 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7a7c107ce0 msgr2=0x7f7a7c19de90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:21:28.451 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:28.450+0000 7f7a81200640 1 --2- 172.21.15.47:0/270064237 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7a7c107ce0 0x7f7a7c19de90 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f7a6c00d770 tx=0x7f7a6c00dc30 comp rx=0 tx=0).stop 2024-03-21T10:21:28.452 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:28.450+0000 7f7a81200640 1 -- 172.21.15.47:0/270064237 shutdown_connections 2024-03-21T10:21:28.452 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:28.450+0000 7f7a81200640 1 --2- 172.21.15.47:0/270064237 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f7a600a2fa0 0x7f7a600a5460 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:21:28.452 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:28.450+0000 7f7a81200640 1 --2- 172.21.15.47:0/270064237 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7a7c109890 0x7f7a7c1a5440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:21:28.452 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:28.450+0000 7f7a81200640 1 --2- 172.21.15.47:0/270064237 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7a7c108ed0 0x7f7a7c19e3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:21:28.452 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:28.450+0000 7f7a81200640 1 --2- 172.21.15.47:0/270064237 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7a7c107ce0 0x7f7a7c19de90 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:21:28.452 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:28.450+0000 7f7a81200640 1 -- 172.21.15.47:0/270064237 >> 172.21.15.47:0/270064237 conn(0x7f7a7c076cb0 msgr2=0x7f7a7c10c1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:21:28.452 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:28.450+0000 7f7a81200640 1 -- 172.21.15.47:0/270064237 shutdown_connections 2024-03-21T10:21:28.452 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:28.450+0000 7f7a81200640 1 -- 172.21.15.47:0/270064237 wait complete. 2024-03-21T10:21:29.201 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:21:29 smithi157 ceph-mon[35581]: pgmap v344: 1 pgs: 1 active+clean; 577 KiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-21T10:21:29.201 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:21:29 smithi157 ceph-mon[35581]: from='client.? 172.21.15.47:0/270064237' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-21T10:21:29.464 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:21:29 smithi047 ceph-mon[31547]: pgmap v344: 1 pgs: 1 active+clean; 577 KiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-21T10:21:29.464 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:21:29 smithi047 ceph-mon[31547]: from='client.? 172.21.15.47:0/270064237' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-21T10:21:29.479 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:21:29 smithi100 ceph-mon[35193]: pgmap v344: 1 pgs: 1 active+clean; 577 KiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-21T10:21:29.480 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:21:29 smithi100 ceph-mon[35193]: from='client.? 172.21.15.47:0/270064237' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-21T10:21:29.766 INFO:journalctl@ceph.mds.f.smithi157.stdout:Mar 21 10:21:29 smithi157 systemd[1]: Started Ceph mds.f for 3056d51a-e76b-11ee-95cd-87774f69a715. 2024-03-21T10:21:30.472 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-21T10:21:30.709 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:30.708+0000 7f5666e00640 1 -- 172.21.15.47:0/3372500598 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f56601028b0 msgr2=0x7f5660102d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:21:30.710 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:30.708+0000 7f5666e00640 1 --2- 172.21.15.47:0/3372500598 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f56601028b0 0x7f5660102d30 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f564c009900 tx=0x7f564c02bc30 comp rx=0 tx=0).stop 2024-03-21T10:21:30.710 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:30.708+0000 7f5666e00640 1 -- 172.21.15.47:0/3372500598 shutdown_connections 2024-03-21T10:21:30.710 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:30.708+0000 7f5666e00640 1 --2- 172.21.15.47:0/3372500598 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5660103270 0x7f5660109b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:21:30.710 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:30.708+0000 7f5666e00640 1 --2- 172.21.15.47:0/3372500598 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f56601028b0 0x7f5660102d30 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:21:30.710 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:30.708+0000 7f5666e00640 1 --2- 172.21.15.47:0/3372500598 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f56601016c0 0x7f5660101ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:21:30.710 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:30.708+0000 7f5666e00640 1 -- 172.21.15.47:0/3372500598 >> 172.21.15.47:0/3372500598 conn(0x7f56600fcc50 msgr2=0x7f56600ff070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:21:30.710 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:30.708+0000 7f5666e00640 1 -- 172.21.15.47:0/3372500598 shutdown_connections 2024-03-21T10:21:30.710 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:30.708+0000 7f5666e00640 1 -- 172.21.15.47:0/3372500598 wait complete. 2024-03-21T10:21:30.713 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:30.709+0000 7f5666e00640 1 Processor -- start 2024-03-21T10:21:30.713 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:30.709+0000 7f5666e00640 1 -- start start 2024-03-21T10:21:30.713 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:30.710+0000 7f5666e00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f56601016c0 0x7f5660075450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:21:30.713 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:30.710+0000 7f5666e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f56601028b0 0x7f5660073a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:21:30.713 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:30.710+0000 7f5666e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5660103270 0x7f5660073fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:21:30.713 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:30.710+0000 7f5666e00640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f566010d1f0 con 0x7f5660103270 2024-03-21T10:21:30.713 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:30.710+0000 7f5666e00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f566010d070 con 0x7f56601028b0 2024-03-21T10:21:30.713 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:30.710+0000 7f5666e00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f566010d370 con 0x7f56601016c0 2024-03-21T10:21:30.713 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:30.710+0000 7f5664e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5660103270 0x7f5660073fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:21:30.714 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:30.710+0000 7f5664e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5660103270 0x7f5660073fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:45014/0 (socket says 172.21.15.47:45014) 2024-03-21T10:21:30.714 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:30.710+0000 7f5664e00640 1 -- 172.21.15.47:0/3373149742 learned_addr learned my addr 172.21.15.47:0/3373149742 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:21:30.714 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:30.710+0000 7f565fe00640 1 --2- 172.21.15.47:0/3373149742 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f56601016c0 0x7f5660075450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:21:30.714 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:30.710+0000 7f565f400640 1 --2- 172.21.15.47:0/3373149742 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f56601028b0 0x7f5660073a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:21:30.714 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:30.711+0000 7f5664e00640 1 -- 172.21.15.47:0/3373149742 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f56601016c0 msgr2=0x7f5660075450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:21:30.714 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:30.711+0000 7f5664e00640 1 --2- 172.21.15.47:0/3373149742 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f56601016c0 0x7f5660075450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:21:30.714 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:30.711+0000 7f5664e00640 1 -- 172.21.15.47:0/3373149742 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f56601028b0 msgr2=0x7f5660073a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:21:30.714 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:30.711+0000 7f5664e00640 1 --2- 172.21.15.47:0/3373149742 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f56601028b0 0x7f5660073a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:21:30.714 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:30.711+0000 7f5664e00640 1 -- 172.21.15.47:0/3373149742 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5660074860 con 0x7f5660103270 2024-03-21T10:21:30.714 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:30.711+0000 7f5664e00640 1 --2- 172.21.15.47:0/3373149742 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5660103270 0x7f5660073fd0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f565800eaa0 tx=0x7f565800ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:21:30.714 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:30.711+0000 7f565cc00640 1 -- 172.21.15.47:0/3373149742 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f565800cdb0 con 0x7f5660103270 2024-03-21T10:21:30.715 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:30.711+0000 7f565cc00640 1 -- 172.21.15.47:0/3373149742 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f5658005020 con 0x7f5660103270 2024-03-21T10:21:30.715 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:30.711+0000 7f5666e00640 1 -- 172.21.15.47:0/3373149742 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f56601a1a10 con 0x7f5660103270 2024-03-21T10:21:30.715 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:30.712+0000 7f565cc00640 1 -- 172.21.15.47:0/3373149742 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56580116d0 con 0x7f5660103270 2024-03-21T10:21:30.715 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:30.712+0000 7f5666e00640 1 -- 172.21.15.47:0/3373149742 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f56601a1f30 con 0x7f5660103270 2024-03-21T10:21:30.715 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:30.713+0000 7f5666e00640 1 -- 172.21.15.47:0/3373149742 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f56600688e0 con 0x7f5660103270 2024-03-21T10:21:30.716 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:30.715+0000 7f565cc00640 1 -- 172.21.15.47:0/3373149742 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 16) v1 ==== 137854+0+0 (secure 0 0 0) 0x7f56580051d0 con 0x7f5660103270 2024-03-21T10:21:30.717 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:30.716+0000 7f565cc00640 1 --2- 172.21.15.47:0/3373149742 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f56440a2f20 0x7f56440a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:21:30.717 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:30.716+0000 7f565cc00640 1 -- 172.21.15.47:0/3373149742 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f56580cf140 con 0x7f5660103270 2024-03-21T10:21:30.717 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:30.716+0000 7f565fe00640 1 --2- 172.21.15.47:0/3373149742 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f56440a2f20 0x7f56440a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:21:30.718 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:30.717+0000 7f565fe00640 1 --2- 172.21.15.47:0/3373149742 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f56440a2f20 0x7f56440a53e0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f5660102710 tx=0x7f565000b390 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:21:30.722 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:30.721+0000 7f565cc00640 1 -- 172.21.15.47:0/3373149742 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f5658016030 con 0x7f5660103270 2024-03-21T10:21:30.964 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:21:30 smithi047 ceph-mon[31547]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:21:30.964 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:21:30 smithi047 ceph-mon[31547]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:21:30.964 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:21:30 smithi047 ceph-mon[31547]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:21:30.964 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:21:30 smithi047 ceph-mon[31547]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-21T10:21:30.964 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:21:30 smithi047 ceph-mon[31547]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-21T10:21:30.965 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:21:30 smithi047 ceph-mon[31547]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T10:21:30.965 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:21:30 smithi047 ceph-mon[31547]: Deploying daemon mds.c on smithi157 2024-03-21T10:21:30.965 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:21:30 smithi047 ceph-mon[31547]: mds.? [v2:172.21.15.157:6834/3902868651,v1:172.21.15.157:6835/3902868651] up:boot 2024-03-21T10:21:30.965 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:21:30 smithi047 ceph-mon[31547]: fsmap 6 up:standby 2024-03-21T10:21:30.965 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:21:30 smithi047 ceph-mon[31547]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-03-21T10:21:30.970 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:30.969+0000 7f5666e00640 1 -- 172.21.15.47:0/3373149742 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f5660108a30 con 0x7f5660103270 2024-03-21T10:21:30.971 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:30.970+0000 7f565cc00640 1 -- 172.21.15.47:0/3373149742 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) v1 ==== 93+0+5289 (secure 0 0 0) 0x7f565809bd20 con 0x7f5660103270 2024-03-21T10:21:30.971 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-21T10:21:30.971 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":7,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14511,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6835/735724037","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6834","nonce":735724037},{"type":"v1","addr":"172.21.15.47:6835","nonce":735724037}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14529,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6837/2248191724","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6836","nonce":2248191724},{"type":"v1","addr":"172.21.15.47:6837","nonce":2248191724}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":24406,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6833/4000010136","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":4000010136},{"type":"v1","addr":"172.21.15.157:6833","nonce":4000010136}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24457,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6835/3902868651","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":3902868651},{"type":"v1","addr":"172.21.15.157:6835","nonce":3902868651}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":7},{"gid":34427,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6833/2723302038","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6832","nonce":2723302038},{"type":"v1","addr":"172.21.15.100:6833","nonce":2723302038}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":34454,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6835/2597849957","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":2597849957},{"type":"v1","addr":"172.21.15.100:6835","nonce":2597849957}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6}],"filesystems":[]} 2024-03-21T10:21:30.972 INFO:teuthology.orchestra.run.smithi047.stderr:dumped fsmap epoch 7 2024-03-21T10:21:30.974 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:30.973+0000 7f5666e00640 1 -- 172.21.15.47:0/3373149742 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f56440a2f20 msgr2=0x7f56440a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:21:30.975 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:30.973+0000 7f5666e00640 1 --2- 172.21.15.47:0/3373149742 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f56440a2f20 0x7f56440a53e0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f5660102710 tx=0x7f565000b390 comp rx=0 tx=0).stop 2024-03-21T10:21:30.975 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:30.973+0000 7f5666e00640 1 -- 172.21.15.47:0/3373149742 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5660103270 msgr2=0x7f5660073fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:21:30.975 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:30.973+0000 7f5666e00640 1 --2- 172.21.15.47:0/3373149742 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5660103270 0x7f5660073fd0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f565800eaa0 tx=0x7f565800ef60 comp rx=0 tx=0).stop 2024-03-21T10:21:30.975 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:30.973+0000 7f5666e00640 1 -- 172.21.15.47:0/3373149742 shutdown_connections 2024-03-21T10:21:30.975 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:30.973+0000 7f5666e00640 1 --2- 172.21.15.47:0/3373149742 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f56440a2f20 0x7f56440a53e0 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:21:30.975 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:30.973+0000 7f5666e00640 1 --2- 172.21.15.47:0/3373149742 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5660103270 0x7f5660073fd0 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:21:30.975 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:30.973+0000 7f5666e00640 1 --2- 172.21.15.47:0/3373149742 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f56601028b0 0x7f5660073a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:21:30.975 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:30.973+0000 7f5666e00640 1 --2- 172.21.15.47:0/3373149742 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f56601016c0 0x7f5660075450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:21:30.975 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:30.973+0000 7f5666e00640 1 -- 172.21.15.47:0/3373149742 >> 172.21.15.47:0/3373149742 conn(0x7f56600fcc50 msgr2=0x7f5660105c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:21:30.975 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:30.973+0000 7f5666e00640 1 -- 172.21.15.47:0/3373149742 shutdown_connections 2024-03-21T10:21:30.975 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:30.973+0000 7f5666e00640 1 -- 172.21.15.47:0/3373149742 wait complete. 2024-03-21T10:21:31.006 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:21:30 smithi157 ceph-mon[35581]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:21:31.006 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:21:30 smithi157 ceph-mon[35581]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:21:31.006 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:21:30 smithi157 ceph-mon[35581]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:21:31.006 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:21:30 smithi157 ceph-mon[35581]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-21T10:21:31.006 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:21:30 smithi157 ceph-mon[35581]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-21T10:21:31.006 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:21:30 smithi157 ceph-mon[35581]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T10:21:31.006 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:21:30 smithi157 ceph-mon[35581]: Deploying daemon mds.c on smithi157 2024-03-21T10:21:31.006 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:21:30 smithi157 ceph-mon[35581]: mds.? [v2:172.21.15.157:6834/3902868651,v1:172.21.15.157:6835/3902868651] up:boot 2024-03-21T10:21:31.006 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:21:30 smithi157 ceph-mon[35581]: fsmap 6 up:standby 2024-03-21T10:21:31.007 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:21:30 smithi157 ceph-mon[35581]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-03-21T10:21:31.230 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:21:30 smithi100 ceph-mon[35193]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:21:31.230 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:21:30 smithi100 ceph-mon[35193]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:21:31.230 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:21:30 smithi100 ceph-mon[35193]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:21:31.230 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:21:30 smithi100 ceph-mon[35193]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-21T10:21:31.230 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:21:30 smithi100 ceph-mon[35193]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-21T10:21:31.230 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:21:30 smithi100 ceph-mon[35193]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T10:21:31.230 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:21:30 smithi100 ceph-mon[35193]: Deploying daemon mds.c on smithi157 2024-03-21T10:21:31.230 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:21:30 smithi100 ceph-mon[35193]: mds.? [v2:172.21.15.157:6834/3902868651,v1:172.21.15.157:6835/3902868651] up:boot 2024-03-21T10:21:31.231 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:21:30 smithi100 ceph-mon[35193]: fsmap 6 up:standby 2024-03-21T10:21:31.231 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:21:30 smithi100 ceph-mon[35193]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-03-21T10:21:31.956 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:21:31 smithi157 ceph-mon[35581]: pgmap v345: 1 pgs: 1 active+clean; 577 KiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-21T10:21:31.956 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:21:31 smithi157 ceph-mon[35581]: from='client.? 172.21.15.47:0/3373149742' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-21T10:21:32.214 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:21:31 smithi047 ceph-mon[31547]: pgmap v345: 1 pgs: 1 active+clean; 577 KiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-21T10:21:32.214 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:21:31 smithi047 ceph-mon[31547]: from='client.? 172.21.15.47:0/3373149742' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-21T10:21:32.229 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:21:31 smithi100 ceph-mon[35193]: pgmap v345: 1 pgs: 1 active+clean; 577 KiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-21T10:21:32.230 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:21:31 smithi100 ceph-mon[35193]: from='client.? 172.21.15.47:0/3373149742' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-21T10:21:32.997 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-21T10:21:33.129 INFO:journalctl@ceph.mds.c.smithi157.stdout:Mar 21 10:21:32 smithi157 systemd[1]: Starting Ceph mds.c for 3056d51a-e76b-11ee-95cd-87774f69a715... 2024-03-21T10:21:33.234 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:33.233+0000 7ff673200640 1 -- 172.21.15.47:0/3991826945 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff674101500 msgr2=0x7ff674101920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:21:33.235 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:33.233+0000 7ff673200640 1 --2- 172.21.15.47:0/3991826945 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff674101500 0x7ff674101920 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7ff664009900 tx=0x7ff66402bc30 comp rx=0 tx=0).stop 2024-03-21T10:21:33.235 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:33.233+0000 7ff673200640 1 -- 172.21.15.47:0/3991826945 shutdown_connections 2024-03-21T10:21:33.235 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:33.233+0000 7ff673200640 1 --2- 172.21.15.47:0/3991826945 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff6741030b0 0x7ff674109940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:21:33.235 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:33.233+0000 7ff673200640 1 --2- 172.21.15.47:0/3991826945 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff6741026f0 0x7ff674102b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:21:33.235 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:33.233+0000 7ff673200640 1 --2- 172.21.15.47:0/3991826945 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff674101500 0x7ff674101920 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:21:33.235 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:33.233+0000 7ff673200640 1 -- 172.21.15.47:0/3991826945 >> 172.21.15.47:0/3991826945 conn(0x7ff6740fca70 msgr2=0x7ff6740feeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:21:33.235 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:33.233+0000 7ff673200640 1 -- 172.21.15.47:0/3991826945 shutdown_connections 2024-03-21T10:21:33.235 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:33.233+0000 7ff673200640 1 -- 172.21.15.47:0/3991826945 wait complete. 2024-03-21T10:21:33.237 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:33.234+0000 7ff673200640 1 Processor -- start 2024-03-21T10:21:33.237 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:33.234+0000 7ff673200640 1 -- start start 2024-03-21T10:21:33.237 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:33.235+0000 7ff673200640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff674101500 0x7ff674075450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:21:33.238 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:33.235+0000 7ff673200640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff6741026f0 0x7ff674073a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:21:33.238 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:33.235+0000 7ff673200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff6741030b0 0x7ff674073fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:21:33.238 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:33.235+0000 7ff673200640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff67410d090 con 0x7ff6741026f0 2024-03-21T10:21:33.238 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:33.235+0000 7ff673200640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff67410cf10 con 0x7ff6741030b0 2024-03-21T10:21:33.238 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:33.235+0000 7ff673200640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff67410d210 con 0x7ff674101500 2024-03-21T10:21:33.238 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:33.235+0000 7ff669400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff6741026f0 0x7ff674073a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:21:33.238 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:33.235+0000 7ff669400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff6741026f0 0x7ff674073a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:56074/0 (socket says 172.21.15.47:56074) 2024-03-21T10:21:33.238 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:33.235+0000 7ff669400640 1 -- 172.21.15.47:0/1867142023 learned_addr learned my addr 172.21.15.47:0/1867142023 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:21:33.238 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:33.235+0000 7ff672800640 1 --2- 172.21.15.47:0/1867142023 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff6741030b0 0x7ff674073fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:21:33.238 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:33.235+0000 7ff671e00640 1 --2- 172.21.15.47:0/1867142023 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff674101500 0x7ff674075450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:21:33.238 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:33.236+0000 7ff669400640 1 -- 172.21.15.47:0/1867142023 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff674101500 msgr2=0x7ff674075450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:21:33.238 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:33.236+0000 7ff669400640 1 --2- 172.21.15.47:0/1867142023 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff674101500 0x7ff674075450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:21:33.238 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:33.236+0000 7ff669400640 1 -- 172.21.15.47:0/1867142023 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff6741030b0 msgr2=0x7ff674073fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:21:33.239 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:33.236+0000 7ff669400640 1 --2- 172.21.15.47:0/1867142023 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff6741030b0 0x7ff674073fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:21:33.239 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:33.236+0000 7ff669400640 1 -- 172.21.15.47:0/1867142023 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff674074860 con 0x7ff6741026f0 2024-03-21T10:21:33.239 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:33.236+0000 7ff672800640 1 --2- 172.21.15.47:0/1867142023 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff6741030b0 0x7ff674073fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T10:21:33.239 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:33.236+0000 7ff669400640 1 --2- 172.21.15.47:0/1867142023 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff6741026f0 0x7ff674073a90 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7ff65c00e9f0 tx=0x7ff65c00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:21:33.239 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:33.236+0000 7ff66b400640 1 -- 172.21.15.47:0/1867142023 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff65c00cd80 con 0x7ff6741026f0 2024-03-21T10:21:33.239 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:33.236+0000 7ff66b400640 1 -- 172.21.15.47:0/1867142023 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7ff65c0108e0 con 0x7ff6741026f0 2024-03-21T10:21:33.239 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:33.236+0000 7ff673200640 1 -- 172.21.15.47:0/1867142023 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff6741a18c0 con 0x7ff6741026f0 2024-03-21T10:21:33.239 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:33.236+0000 7ff66b400640 1 -- 172.21.15.47:0/1867142023 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff65c018920 con 0x7ff6741026f0 2024-03-21T10:21:33.239 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:33.236+0000 7ff673200640 1 -- 172.21.15.47:0/1867142023 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff6741a1e10 con 0x7ff6741026f0 2024-03-21T10:21:33.239 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:33.238+0000 7ff673200640 1 -- 172.21.15.47:0/1867142023 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff6741088e0 con 0x7ff6741026f0 2024-03-21T10:21:33.241 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:33.240+0000 7ff66b400640 1 -- 172.21.15.47:0/1867142023 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 16) v1 ==== 137854+0+0 (secure 0 0 0) 0x7ff65c010a90 con 0x7ff6741026f0 2024-03-21T10:21:33.242 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:33.241+0000 7ff66b400640 1 --2- 172.21.15.47:0/1867142023 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7ff6580a2f20 0x7ff6580a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:21:33.243 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:33.242+0000 7ff66b400640 1 -- 172.21.15.47:0/1867142023 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 7127+0+0 (secure 0 0 0) 0x7ff65c0cf420 con 0x7ff6741026f0 2024-03-21T10:21:33.243 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:33.242+0000 7ff671e00640 1 --2- 172.21.15.47:0/1867142023 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7ff6580a2f20 0x7ff6580a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:21:33.243 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:33.243+0000 7ff671e00640 1 --2- 172.21.15.47:0/1867142023 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7ff6580a2f20 0x7ff6580a53e0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7ff66402c240 tx=0x7ff664005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:21:33.247 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:33.246+0000 7ff66b400640 1 -- 172.21.15.47:0/1867142023 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7ff65c09c650 con 0x7ff6741026f0 2024-03-21T10:21:33.438 INFO:journalctl@ceph.mds.c.smithi157.stdout:Mar 21 10:21:33 smithi157 podman[66278]: 2024-03-21 10:21:33.032885386 +0000 UTC m=+0.035243551 image pull 237e43c7a65ff48534d94b97c1bfdbe4b7be5672e3d9a613745083ea3702aa54 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232 2024-03-21T10:21:33.438 INFO:journalctl@ceph.mds.c.smithi157.stdout:Mar 21 10:21:33 smithi157 podman[66278]: 2024-03-21 10:21:33.165188564 +0000 UTC m=+0.167546742 container create 55d8b6b1a13b2ba9ec9abc3ec6ed4dc7eb96680b2a2fe637a89a64f1cbd2542a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-3056d51a-e76b-11ee-95cd-87774f69a715-mds-c, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, ceph=True, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240319) 2024-03-21T10:21:33.438 INFO:journalctl@ceph.mds.c.smithi157.stdout:Mar 21 10:21:33 smithi157 podman[66278]: 2024-03-21 10:21:33.436048133 +0000 UTC m=+0.438406318 container init 55d8b6b1a13b2ba9ec9abc3ec6ed4dc7eb96680b2a2fe637a89a64f1cbd2542a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-3056d51a-e76b-11ee-95cd-87774f69a715-mds-c, ceph=True, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.build-date=20240319, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30) 2024-03-21T10:21:33.493 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:33.492+0000 7ff673200640 1 -- 172.21.15.47:0/1867142023 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff6740622a0 con 0x7ff6741026f0 2024-03-21T10:21:33.494 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:33.493+0000 7ff66b400640 1 -- 172.21.15.47:0/1867142023 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) v1 ==== 93+0+5289 (secure 0 0 0) 0x7ff65c09bf50 con 0x7ff6741026f0 2024-03-21T10:21:33.494 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-21T10:21:33.495 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":7,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14511,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6835/735724037","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6834","nonce":735724037},{"type":"v1","addr":"172.21.15.47:6835","nonce":735724037}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14529,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6837/2248191724","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6836","nonce":2248191724},{"type":"v1","addr":"172.21.15.47:6837","nonce":2248191724}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":24406,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6833/4000010136","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":4000010136},{"type":"v1","addr":"172.21.15.157:6833","nonce":4000010136}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24457,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6835/3902868651","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":3902868651},{"type":"v1","addr":"172.21.15.157:6835","nonce":3902868651}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":7},{"gid":34427,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6833/2723302038","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6832","nonce":2723302038},{"type":"v1","addr":"172.21.15.100:6833","nonce":2723302038}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":34454,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6835/2597849957","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":2597849957},{"type":"v1","addr":"172.21.15.100:6835","nonce":2597849957}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6}],"filesystems":[]} 2024-03-21T10:21:33.495 INFO:teuthology.orchestra.run.smithi047.stderr:dumped fsmap epoch 7 2024-03-21T10:21:33.497 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:33.496+0000 7ff673200640 1 -- 172.21.15.47:0/1867142023 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7ff6580a2f20 msgr2=0x7ff6580a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:21:33.497 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:33.496+0000 7ff673200640 1 --2- 172.21.15.47:0/1867142023 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7ff6580a2f20 0x7ff6580a53e0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7ff66402c240 tx=0x7ff664005fd0 comp rx=0 tx=0).stop 2024-03-21T10:21:33.498 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:33.496+0000 7ff673200640 1 -- 172.21.15.47:0/1867142023 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff6741026f0 msgr2=0x7ff674073a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:21:33.498 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:33.496+0000 7ff673200640 1 --2- 172.21.15.47:0/1867142023 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff6741026f0 0x7ff674073a90 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7ff65c00e9f0 tx=0x7ff65c00eeb0 comp rx=0 tx=0).stop 2024-03-21T10:21:33.498 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:33.496+0000 7ff673200640 1 -- 172.21.15.47:0/1867142023 shutdown_connections 2024-03-21T10:21:33.498 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:33.496+0000 7ff673200640 1 --2- 172.21.15.47:0/1867142023 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7ff6580a2f20 0x7ff6580a53e0 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:21:33.498 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:33.496+0000 7ff673200640 1 --2- 172.21.15.47:0/1867142023 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff6741030b0 0x7ff674073fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:21:33.498 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:33.496+0000 7ff673200640 1 --2- 172.21.15.47:0/1867142023 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff6741026f0 0x7ff674073a90 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:21:33.498 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:33.496+0000 7ff673200640 1 --2- 172.21.15.47:0/1867142023 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff674101500 0x7ff674075450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:21:33.498 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:33.496+0000 7ff673200640 1 -- 172.21.15.47:0/1867142023 >> 172.21.15.47:0/1867142023 conn(0x7ff6740fca70 msgr2=0x7ff674105ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:21:33.498 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:33.496+0000 7ff673200640 1 -- 172.21.15.47:0/1867142023 shutdown_connections 2024-03-21T10:21:33.498 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:33.496+0000 7ff673200640 1 -- 172.21.15.47:0/1867142023 wait complete. 2024-03-21T10:21:33.740 INFO:journalctl@ceph.mds.c.smithi157.stdout:Mar 21 10:21:33 smithi157 podman[66278]: 2024-03-21 10:21:33.446017313 +0000 UTC m=+0.448375461 container start 55d8b6b1a13b2ba9ec9abc3ec6ed4dc7eb96680b2a2fe637a89a64f1cbd2542a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-3056d51a-e76b-11ee-95cd-87774f69a715-mds-c, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.schema-version=1.0, ceph=True, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, io.buildah.version=1.33.5, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240319, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2) 2024-03-21T10:21:33.740 INFO:journalctl@ceph.mds.c.smithi157.stdout:Mar 21 10:21:33 smithi157 ceph-mds[66298]: -- 172.21.15.157:0/579746495 <== mon.1 v2:172.21.15.157:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x562a05495a40 con 0x562a05468800 2024-03-21T10:21:33.741 INFO:journalctl@ceph.mds.c.smithi157.stdout:Mar 21 10:21:33 smithi157 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-mds-c[66294]: starting mds.c at 2024-03-21T10:21:33.741 INFO:journalctl@ceph.mds.c.smithi157.stdout:Mar 21 10:21:33 smithi157 bash[66278]: 55d8b6b1a13b2ba9ec9abc3ec6ed4dc7eb96680b2a2fe637a89a64f1cbd2542a 2024-03-21T10:21:33.741 INFO:journalctl@ceph.mds.c.smithi157.stdout:Mar 21 10:21:33 smithi157 systemd[1]: Started Ceph mds.c for 3056d51a-e76b-11ee-95cd-87774f69a715. 2024-03-21T10:21:34.176 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:21:33 smithi047 ceph-mon[31547]: pgmap v346: 1 pgs: 1 active+clean; 577 KiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-21T10:21:34.176 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:21:33 smithi047 ceph-mon[31547]: from='client.? 172.21.15.47:0/1867142023' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-21T10:21:34.176 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:21:33 smithi047 ceph-mon[31547]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:21:34.176 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:21:33 smithi047 ceph-mon[31547]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:21:34.176 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:21:33 smithi047 ceph-mon[31547]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:21:34.176 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:21:33 smithi047 ceph-mon[31547]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-21T10:21:34.176 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:21:33 smithi047 ceph-mon[31547]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-21T10:21:34.176 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:21:33 smithi047 ceph-mon[31547]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T10:21:34.230 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:21:33 smithi100 ceph-mon[35193]: pgmap v346: 1 pgs: 1 active+clean; 577 KiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-21T10:21:34.230 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:21:33 smithi100 ceph-mon[35193]: from='client.? 172.21.15.47:0/1867142023' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-21T10:21:34.230 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:21:33 smithi100 ceph-mon[35193]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:21:34.230 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:21:33 smithi100 ceph-mon[35193]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:21:34.230 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:21:33 smithi100 ceph-mon[35193]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:21:34.230 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:21:33 smithi100 ceph-mon[35193]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-21T10:21:34.230 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:21:33 smithi100 ceph-mon[35193]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-21T10:21:34.230 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:21:33 smithi100 ceph-mon[35193]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T10:21:34.240 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:21:33 smithi157 ceph-mon[35581]: pgmap v346: 1 pgs: 1 active+clean; 577 KiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-21T10:21:34.240 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:21:33 smithi157 ceph-mon[35581]: from='client.? 172.21.15.47:0/1867142023' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-21T10:21:34.240 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:21:33 smithi157 ceph-mon[35581]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:21:34.240 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:21:33 smithi157 ceph-mon[35581]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:21:34.240 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:21:33 smithi157 ceph-mon[35581]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:21:34.240 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:21:33 smithi157 ceph-mon[35581]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-21T10:21:34.240 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:21:33 smithi157 ceph-mon[35581]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-21T10:21:34.241 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:21:33 smithi157 ceph-mon[35581]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T10:21:35.057 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:21:34 smithi100 ceph-mon[35193]: Deploying daemon mds.h on smithi100 2024-03-21T10:21:35.057 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:21:34 smithi100 ceph-mon[35193]: mds.? [v2:172.21.15.157:6836/1908087204,v1:172.21.15.157:6837/1908087204] up:boot 2024-03-21T10:21:35.057 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:21:34 smithi100 ceph-mon[35193]: fsmap 7 up:standby 2024-03-21T10:21:35.057 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:21:34 smithi100 ceph-mon[35193]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-03-21T10:21:35.214 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:21:34 smithi047 ceph-mon[31547]: Deploying daemon mds.h on smithi100 2024-03-21T10:21:35.214 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:21:34 smithi047 ceph-mon[31547]: mds.? [v2:172.21.15.157:6836/1908087204,v1:172.21.15.157:6837/1908087204] up:boot 2024-03-21T10:21:35.214 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:21:34 smithi047 ceph-mon[31547]: fsmap 7 up:standby 2024-03-21T10:21:35.214 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:21:34 smithi047 ceph-mon[31547]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-03-21T10:21:35.240 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:21:34 smithi157 ceph-mon[35581]: Deploying daemon mds.h on smithi100 2024-03-21T10:21:35.240 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:21:34 smithi157 ceph-mon[35581]: mds.? [v2:172.21.15.157:6836/1908087204,v1:172.21.15.157:6837/1908087204] up:boot 2024-03-21T10:21:35.240 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:21:34 smithi157 ceph-mon[35581]: fsmap 7 up:standby 2024-03-21T10:21:35.240 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:21:34 smithi157 ceph-mon[35581]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-03-21T10:21:35.516 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-21T10:21:35.748 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:35.747+0000 7f692f400640 1 -- 172.21.15.47:0/1822531065 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6928076060 msgr2=0x7f6928076480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:21:35.749 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:35.747+0000 7f692f400640 1 --2- 172.21.15.47:0/1822531065 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6928076060 0x7f6928076480 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f691c009900 tx=0x7f691c02bc30 comp rx=0 tx=0).stop 2024-03-21T10:21:35.749 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:35.747+0000 7f692f400640 1 -- 172.21.15.47:0/1822531065 shutdown_connections 2024-03-21T10:21:35.749 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:35.747+0000 7f692f400640 1 --2- 172.21.15.47:0/1822531065 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6928077c40 0x7f69280784e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:21:35.749 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:35.747+0000 7f692f400640 1 --2- 172.21.15.47:0/1822531065 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f69280772a0 0x7f6928077700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:21:35.749 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:35.747+0000 7f692f400640 1 --2- 172.21.15.47:0/1822531065 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6928076060 0x7f6928076480 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:21:35.749 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:35.747+0000 7f692f400640 1 -- 172.21.15.47:0/1822531065 >> 172.21.15.47:0/1822531065 conn(0x7f69280fee50 msgr2=0x7f6928101290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:21:35.749 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:35.747+0000 7f692f400640 1 -- 172.21.15.47:0/1822531065 shutdown_connections 2024-03-21T10:21:35.749 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:35.747+0000 7f692f400640 1 -- 172.21.15.47:0/1822531065 wait complete. 2024-03-21T10:21:35.752 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:35.748+0000 7f692f400640 1 Processor -- start 2024-03-21T10:21:35.752 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:35.748+0000 7f692f400640 1 -- start start 2024-03-21T10:21:35.752 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:35.749+0000 7f692f400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6928076060 0x7f692819dfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:21:35.752 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:35.749+0000 7f692f400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f69280772a0 0x7f692819e4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:21:35.752 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:35.749+0000 7f692f400640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6928077c40 0x7f69281a5500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:21:35.752 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:35.749+0000 7f692f400640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6928113860 con 0x7f6928076060 2024-03-21T10:21:35.752 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:35.749+0000 7f692f400640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69281136e0 con 0x7f69280772a0 2024-03-21T10:21:35.752 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:35.749+0000 7f692f400640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69281139e0 con 0x7f6928077c40 2024-03-21T10:21:35.752 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:35.749+0000 7f692ca00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6928076060 0x7f692819dfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:21:35.752 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:35.749+0000 7f692ca00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6928076060 0x7f692819dfa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:56090/0 (socket says 172.21.15.47:56090) 2024-03-21T10:21:35.752 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:35.749+0000 7f692ca00640 1 -- 172.21.15.47:0/2675125140 learned_addr learned my addr 172.21.15.47:0/2675125140 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:21:35.752 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:35.749+0000 7f692d400640 1 --2- 172.21.15.47:0/2675125140 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6928077c40 0x7f69281a5500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:21:35.753 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:35.749+0000 7f6927e00640 1 --2- 172.21.15.47:0/2675125140 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f69280772a0 0x7f692819e4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:21:35.753 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:35.749+0000 7f692ca00640 1 -- 172.21.15.47:0/2675125140 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6928077c40 msgr2=0x7f69281a5500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:21:35.753 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:35.749+0000 7f692ca00640 1 --2- 172.21.15.47:0/2675125140 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6928077c40 0x7f69281a5500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:21:35.753 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:35.749+0000 7f692ca00640 1 -- 172.21.15.47:0/2675125140 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f69280772a0 msgr2=0x7f692819e4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:21:35.753 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:35.749+0000 7f692ca00640 1 --2- 172.21.15.47:0/2675125140 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f69280772a0 0x7f692819e4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:21:35.754 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:35.749+0000 7f692ca00640 1 -- 172.21.15.47:0/2675125140 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f69281a5b70 con 0x7f6928076060 2024-03-21T10:21:35.754 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:35.749+0000 7f6927e00640 1 --2- 172.21.15.47:0/2675125140 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f69280772a0 0x7f692819e4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T10:21:35.754 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:35.750+0000 7f692ca00640 1 --2- 172.21.15.47:0/2675125140 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6928076060 0x7f692819dfa0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f691c02c840 tx=0x7f691c0042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:21:35.754 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:35.750+0000 7f6925600640 1 -- 172.21.15.47:0/2675125140 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f691c043020 con 0x7f6928076060 2024-03-21T10:21:35.754 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:35.750+0000 7f692f400640 1 -- 172.21.15.47:0/2675125140 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f69281a5e00 con 0x7f6928076060 2024-03-21T10:21:35.754 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:35.750+0000 7f6925600640 1 -- 172.21.15.47:0/2675125140 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f691c004aa0 con 0x7f6928076060 2024-03-21T10:21:35.754 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:35.750+0000 7f6925600640 1 -- 172.21.15.47:0/2675125140 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f691c003cd0 con 0x7f6928076060 2024-03-21T10:21:35.754 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:35.750+0000 7f692f400640 1 -- 172.21.15.47:0/2675125140 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f69281a6260 con 0x7f6928076060 2024-03-21T10:21:35.755 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:35.751+0000 7f692f400640 1 -- 172.21.15.47:0/2675125140 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6928110250 con 0x7f6928076060 2024-03-21T10:21:35.755 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:35.753+0000 7f6925600640 1 -- 172.21.15.47:0/2675125140 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 16) v1 ==== 137854+0+0 (secure 0 0 0) 0x7f691c046020 con 0x7f6928076060 2024-03-21T10:21:35.756 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:35.755+0000 7f6925600640 1 --2- 172.21.15.47:0/2675125140 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f69100a2f20 0x7f69100a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:21:35.756 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:35.755+0000 7f6925600640 1 -- 172.21.15.47:0/2675125140 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f691c0f1300 con 0x7f6928076060 2024-03-21T10:21:35.756 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:35.755+0000 7f6927e00640 1 --2- 172.21.15.47:0/2675125140 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f69100a2f20 0x7f69100a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:21:35.757 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:35.756+0000 7f6927e00640 1 --2- 172.21.15.47:0/2675125140 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f69100a2f20 0x7f69100a53e0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f692819f460 tx=0x7f6914005e50 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:21:35.761 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:35.759+0000 7f6925600640 1 -- 172.21.15.47:0/2675125140 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f691c04d020 con 0x7f6928076060 2024-03-21T10:21:35.963 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:21:35 smithi047 ceph-mon[31547]: pgmap v347: 1 pgs: 1 active+clean; 577 KiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-21T10:21:36.004 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:36.003+0000 7f692f400640 1 -- 172.21.15.47:0/2675125140 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f692802d2c0 con 0x7f6928076060 2024-03-21T10:21:36.005 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:36.004+0000 7f6925600640 1 -- 172.21.15.47:0/2675125140 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+6083 (secure 0 0 0) 0x7f691c0be5d0 con 0x7f6928076060 2024-03-21T10:21:36.005 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-21T10:21:36.006 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":8,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14511,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6835/735724037","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6834","nonce":735724037},{"type":"v1","addr":"172.21.15.47:6835","nonce":735724037}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14529,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6837/2248191724","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6836","nonce":2248191724},{"type":"v1","addr":"172.21.15.47:6837","nonce":2248191724}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":24406,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6833/4000010136","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":4000010136},{"type":"v1","addr":"172.21.15.157:6833","nonce":4000010136}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24457,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6835/3902868651","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":3902868651},{"type":"v1","addr":"172.21.15.157:6835","nonce":3902868651}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":7},{"gid":24475,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6837/1908087204","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6836","nonce":1908087204},{"type":"v1","addr":"172.21.15.157:6837","nonce":1908087204}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":8},{"gid":34427,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6833/2723302038","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6832","nonce":2723302038},{"type":"v1","addr":"172.21.15.100:6833","nonce":2723302038}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":34454,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6835/2597849957","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":2597849957},{"type":"v1","addr":"172.21.15.100:6835","nonce":2597849957}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6}],"filesystems":[]} 2024-03-21T10:21:36.006 INFO:teuthology.orchestra.run.smithi047.stderr:dumped fsmap epoch 8 2024-03-21T10:21:36.008 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:36.007+0000 7f692f400640 1 -- 172.21.15.47:0/2675125140 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f69100a2f20 msgr2=0x7f69100a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:21:36.008 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:36.007+0000 7f692f400640 1 --2- 172.21.15.47:0/2675125140 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f69100a2f20 0x7f69100a53e0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f692819f460 tx=0x7f6914005e50 comp rx=0 tx=0).stop 2024-03-21T10:21:36.008 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:36.007+0000 7f692f400640 1 -- 172.21.15.47:0/2675125140 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6928076060 msgr2=0x7f692819dfa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:21:36.008 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:36.007+0000 7f692f400640 1 --2- 172.21.15.47:0/2675125140 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6928076060 0x7f692819dfa0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f691c02c840 tx=0x7f691c0042f0 comp rx=0 tx=0).stop 2024-03-21T10:21:36.008 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:36.007+0000 7f692f400640 1 -- 172.21.15.47:0/2675125140 shutdown_connections 2024-03-21T10:21:36.008 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:36.007+0000 7f692f400640 1 --2- 172.21.15.47:0/2675125140 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f69100a2f20 0x7f69100a53e0 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:21:36.009 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:36.007+0000 7f692f400640 1 --2- 172.21.15.47:0/2675125140 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6928077c40 0x7f69281a5500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:21:36.009 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:36.007+0000 7f692f400640 1 --2- 172.21.15.47:0/2675125140 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f69280772a0 0x7f692819e4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:21:36.009 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:36.007+0000 7f692f400640 1 --2- 172.21.15.47:0/2675125140 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6928076060 0x7f692819dfa0 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:21:36.009 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:36.007+0000 7f692f400640 1 -- 172.21.15.47:0/2675125140 >> 172.21.15.47:0/2675125140 conn(0x7f69280fee50 msgr2=0x7f6928100950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:21:36.009 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:36.007+0000 7f692f400640 1 -- 172.21.15.47:0/2675125140 shutdown_connections 2024-03-21T10:21:36.009 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:36.008+0000 7f692f400640 1 -- 172.21.15.47:0/2675125140 wait complete. 2024-03-21T10:21:36.109 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:21:35 smithi100 ceph-mon[35193]: pgmap v347: 1 pgs: 1 active+clean; 577 KiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-21T10:21:36.240 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:21:35 smithi157 ceph-mon[35581]: pgmap v347: 1 pgs: 1 active+clean; 577 KiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-21T10:21:37.214 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:21:36 smithi047 ceph-mon[31547]: from='client.? 172.21.15.47:0/2675125140' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-21T10:21:37.230 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:21:36 smithi100 ceph-mon[35193]: from='client.? 172.21.15.47:0/2675125140' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-21T10:21:37.230 INFO:journalctl@ceph.mds.h.smithi100.stdout:Mar 21 10:21:36 smithi100 systemd[1]: Starting Ceph mds.h for 3056d51a-e76b-11ee-95cd-87774f69a715... 2024-03-21T10:21:37.239 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:21:36 smithi157 ceph-mon[35581]: from='client.? 172.21.15.47:0/2675125140' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-21T10:21:37.517 INFO:journalctl@ceph.mds.h.smithi100.stdout:Mar 21 10:21:37 smithi100 podman[64667]: 2024-03-21 10:21:37.134745944 +0000 UTC m=+0.032900335 image pull 237e43c7a65ff48534d94b97c1bfdbe4b7be5672e3d9a613745083ea3702aa54 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232 2024-03-21T10:21:37.517 INFO:journalctl@ceph.mds.h.smithi100.stdout:Mar 21 10:21:37 smithi100 podman[64667]: 2024-03-21 10:21:37.237003926 +0000 UTC m=+0.135158362 container create 8a41fac6864248e992b6e5b139cdeac025974f3418e722dbcb048f7eda8e8524 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-3056d51a-e76b-11ee-95cd-87774f69a715-mds-h, org.label-schema.build-date=20240319, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, GIT_CLEAN=True, GIT_BRANCH=HEAD, io.buildah.version=1.33.5) 2024-03-21T10:21:37.517 INFO:journalctl@ceph.mds.h.smithi100.stdout:Mar 21 10:21:37 smithi100 podman[64667]: 2024-03-21 10:21:37.515975713 +0000 UTC m=+0.414130135 container init 8a41fac6864248e992b6e5b139cdeac025974f3418e722dbcb048f7eda8e8524 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-3056d51a-e76b-11ee-95cd-87774f69a715-mds-h, ceph=True, io.buildah.version=1.33.5, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240319, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30) 2024-03-21T10:21:37.832 INFO:journalctl@ceph.mds.h.smithi100.stdout:Mar 21 10:21:37 smithi100 podman[64667]: 2024-03-21 10:21:37.526587903 +0000 UTC m=+0.424742292 container start 8a41fac6864248e992b6e5b139cdeac025974f3418e722dbcb048f7eda8e8524 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-3056d51a-e76b-11ee-95cd-87774f69a715-mds-h, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_CLEAN=True, ceph=True, org.label-schema.schema-version=1.0, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, org.label-schema.build-date=20240319, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30) 2024-03-21T10:21:37.832 INFO:journalctl@ceph.mds.h.smithi100.stdout:Mar 21 10:21:37 smithi100 ceph-mds[64686]: -- 172.21.15.100:0/3823214346 <== mon.2 v2:172.21.15.100:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55eb07a0ba40 con 0x55eb079de800 2024-03-21T10:21:37.832 INFO:journalctl@ceph.mds.h.smithi100.stdout:Mar 21 10:21:37 smithi100 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-mds-h[64682]: starting mds.h at 2024-03-21T10:21:37.832 INFO:journalctl@ceph.mds.h.smithi100.stdout:Mar 21 10:21:37 smithi100 bash[64667]: 8a41fac6864248e992b6e5b139cdeac025974f3418e722dbcb048f7eda8e8524 2024-03-21T10:21:37.832 INFO:journalctl@ceph.mds.h.smithi100.stdout:Mar 21 10:21:37 smithi100 systemd[1]: Started Ceph mds.h for 3056d51a-e76b-11ee-95cd-87774f69a715. 2024-03-21T10:21:38.027 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-21T10:21:38.114 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:21:37 smithi047 ceph-mon[31547]: pgmap v348: 1 pgs: 1 active+clean; 577 KiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-21T10:21:38.114 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:21:37 smithi047 ceph-mon[31547]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:21:38.114 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:21:37 smithi047 ceph-mon[31547]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:21:38.114 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:21:37 smithi047 ceph-mon[31547]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:21:38.114 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:21:37 smithi047 ceph-mon[31547]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-21T10:21:38.114 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:21:37 smithi047 ceph-mon[31547]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-21T10:21:38.114 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:21:37 smithi047 ceph-mon[31547]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T10:21:38.230 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:21:37 smithi100 ceph-mon[35193]: pgmap v348: 1 pgs: 1 active+clean; 577 KiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-21T10:21:38.230 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:21:37 smithi100 ceph-mon[35193]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:21:38.230 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:21:37 smithi100 ceph-mon[35193]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:21:38.230 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:21:37 smithi100 ceph-mon[35193]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:21:38.230 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:21:37 smithi100 ceph-mon[35193]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-21T10:21:38.230 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:21:37 smithi100 ceph-mon[35193]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-21T10:21:38.230 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:21:37 smithi100 ceph-mon[35193]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T10:21:38.240 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:21:37 smithi157 ceph-mon[35581]: pgmap v348: 1 pgs: 1 active+clean; 577 KiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-21T10:21:38.240 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:21:37 smithi157 ceph-mon[35581]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:21:38.240 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:21:37 smithi157 ceph-mon[35581]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:21:38.241 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:21:37 smithi157 ceph-mon[35581]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:21:38.241 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:21:37 smithi157 ceph-mon[35581]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-21T10:21:38.241 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:21:37 smithi157 ceph-mon[35581]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-21T10:21:38.241 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:21:37 smithi157 ceph-mon[35581]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T10:21:38.256 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:38.255+0000 7fb6e5000640 1 -- 172.21.15.47:0/3484444929 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb6e0107b00 msgr2=0x7fb6e0107f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:21:38.257 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:38.255+0000 7fb6e5000640 1 --2- 172.21.15.47:0/3484444929 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb6e0107b00 0x7fb6e0107f20 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fb6d0009900 tx=0x7fb6d002bc30 comp rx=0 tx=0).stop 2024-03-21T10:21:38.257 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:38.255+0000 7fb6e5000640 1 -- 172.21.15.47:0/3484444929 shutdown_connections 2024-03-21T10:21:38.257 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:38.255+0000 7fb6e5000640 1 --2- 172.21.15.47:0/3484444929 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb6e01096b0 0x7fb6e010ff40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:21:38.257 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:38.255+0000 7fb6e5000640 1 --2- 172.21.15.47:0/3484444929 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb6e0108cf0 0x7fb6e0109170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:21:38.257 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:38.255+0000 7fb6e5000640 1 --2- 172.21.15.47:0/3484444929 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb6e0107b00 0x7fb6e0107f20 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:21:38.257 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:38.255+0000 7fb6e5000640 1 -- 172.21.15.47:0/3484444929 >> 172.21.15.47:0/3484444929 conn(0x7fb6e0076ab0 msgr2=0x7fb6e0078ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:21:38.257 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:38.255+0000 7fb6e5000640 1 -- 172.21.15.47:0/3484444929 shutdown_connections 2024-03-21T10:21:38.257 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:38.255+0000 7fb6e5000640 1 -- 172.21.15.47:0/3484444929 wait complete. 2024-03-21T10:21:38.259 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:38.256+0000 7fb6e5000640 1 Processor -- start 2024-03-21T10:21:38.259 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:38.256+0000 7fb6e5000640 1 -- start start 2024-03-21T10:21:38.260 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:38.256+0000 7fb6e5000640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb6e0107b00 0x7fb6e019dcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:21:38.260 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:38.257+0000 7fb6e5000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb6e0108cf0 0x7fb6e019e1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:21:38.260 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:38.257+0000 7fb6e5000640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb6e01096b0 0x7fb6e01a5250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:21:38.260 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:38.257+0000 7fb6e5000640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6e0113670 con 0x7fb6e0107b00 2024-03-21T10:21:38.260 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:38.257+0000 7fb6e5000640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6e01134f0 con 0x7fb6e0108cf0 2024-03-21T10:21:38.260 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:38.257+0000 7fb6e5000640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6e01137f0 con 0x7fb6e01096b0 2024-03-21T10:21:38.260 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:38.257+0000 7fb6df400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb6e0107b00 0x7fb6e019dcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:21:38.260 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:38.257+0000 7fb6df400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb6e0107b00 0x7fb6e019dcb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:56106/0 (socket says 172.21.15.47:56106) 2024-03-21T10:21:38.261 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:38.257+0000 7fb6df400640 1 -- 172.21.15.47:0/1303744143 learned_addr learned my addr 172.21.15.47:0/1303744143 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:21:38.261 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:38.257+0000 7fb6dea00640 1 --2- 172.21.15.47:0/1303744143 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb6e0108cf0 0x7fb6e019e1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:21:38.261 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:38.257+0000 7fb6df400640 1 -- 172.21.15.47:0/1303744143 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb6e01096b0 msgr2=0x7fb6e01a5250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:21:38.261 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:38.257+0000 7fb6dfe00640 1 --2- 172.21.15.47:0/1303744143 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb6e01096b0 0x7fb6e01a5250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:21:38.261 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:38.257+0000 7fb6df400640 1 --2- 172.21.15.47:0/1303744143 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb6e01096b0 0x7fb6e01a5250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:21:38.261 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:38.257+0000 7fb6df400640 1 -- 172.21.15.47:0/1303744143 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb6e0108cf0 msgr2=0x7fb6e019e1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:21:38.261 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:38.257+0000 7fb6df400640 1 --2- 172.21.15.47:0/1303744143 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb6e0108cf0 0x7fb6e019e1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:21:38.261 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:38.257+0000 7fb6df400640 1 -- 172.21.15.47:0/1303744143 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb6e01a58f0 con 0x7fb6e0107b00 2024-03-21T10:21:38.261 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:38.257+0000 7fb6dea00640 1 --2- 172.21.15.47:0/1303744143 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb6e0108cf0 0x7fb6e019e1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T10:21:38.261 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:38.258+0000 7fb6df400640 1 --2- 172.21.15.47:0/1303744143 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb6e0107b00 0x7fb6e019dcb0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fb6d0005ab0 tx=0x7fb6d00042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:21:38.262 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:38.258+0000 7fb6d7e00640 1 -- 172.21.15.47:0/1303744143 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6d0043020 con 0x7fb6e0107b00 2024-03-21T10:21:38.262 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:38.258+0000 7fb6d7e00640 1 -- 172.21.15.47:0/1303744143 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fb6d00054c0 con 0x7fb6e0107b00 2024-03-21T10:21:38.262 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:38.258+0000 7fb6d7e00640 1 -- 172.21.15.47:0/1303744143 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6d0003d10 con 0x7fb6e0107b00 2024-03-21T10:21:38.262 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:38.258+0000 7fb6e5000640 1 -- 172.21.15.47:0/1303744143 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb6e01a5b80 con 0x7fb6e0107b00 2024-03-21T10:21:38.262 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:38.258+0000 7fb6e5000640 1 -- 172.21.15.47:0/1303744143 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb6e01a6120 con 0x7fb6e0107b00 2024-03-21T10:21:38.263 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:38.259+0000 7fb6e5000640 1 -- 172.21.15.47:0/1303744143 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb6e010ee20 con 0x7fb6e0107b00 2024-03-21T10:21:38.263 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:38.261+0000 7fb6d7e00640 1 -- 172.21.15.47:0/1303744143 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 16) v1 ==== 137854+0+0 (secure 0 0 0) 0x7fb6d0046020 con 0x7fb6e0107b00 2024-03-21T10:21:38.263 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:38.262+0000 7fb6d7e00640 1 --2- 172.21.15.47:0/1303744143 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7fb6c40a2fa0 0x7fb6c40a5460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:21:38.264 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:38.263+0000 7fb6d7e00640 1 -- 172.21.15.47:0/1303744143 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 7127+0+0 (secure 0 0 0) 0x7fb6d00f1380 con 0x7fb6e0107b00 2024-03-21T10:21:38.264 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:38.263+0000 7fb6dea00640 1 --2- 172.21.15.47:0/1303744143 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7fb6c40a2fa0 0x7fb6c40a5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:21:38.264 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:38.263+0000 7fb6dea00640 1 --2- 172.21.15.47:0/1303744143 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7fb6c40a2fa0 0x7fb6c40a5460 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fb6e019f1b0 tx=0x7fb6c8005e50 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:21:38.268 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:38.267+0000 7fb6d7e00640 1 -- 172.21.15.47:0/1303744143 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fb6d004d020 con 0x7fb6e0107b00 2024-03-21T10:21:38.516 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:38.515+0000 7fb6e5000640 1 -- 172.21.15.47:0/1303744143 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb6e00622a0 con 0x7fb6e0107b00 2024-03-21T10:21:38.518 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:38.517+0000 7fb6d7e00640 1 -- 172.21.15.47:0/1303744143 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) v1 ==== 93+0+6877 (secure 0 0 0) 0x7fb6d00be6d0 con 0x7fb6e0107b00 2024-03-21T10:21:38.518 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-21T10:21:38.518 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":9,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14511,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6835/735724037","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6834","nonce":735724037},{"type":"v1","addr":"172.21.15.47:6835","nonce":735724037}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14529,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6837/2248191724","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6836","nonce":2248191724},{"type":"v1","addr":"172.21.15.47:6837","nonce":2248191724}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":24406,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6833/4000010136","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":4000010136},{"type":"v1","addr":"172.21.15.157:6833","nonce":4000010136}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24457,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6835/3902868651","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":3902868651},{"type":"v1","addr":"172.21.15.157:6835","nonce":3902868651}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":7},{"gid":24475,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6837/1908087204","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6836","nonce":1908087204},{"type":"v1","addr":"172.21.15.157:6837","nonce":1908087204}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":8},{"gid":34427,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6833/2723302038","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6832","nonce":2723302038},{"type":"v1","addr":"172.21.15.100:6833","nonce":2723302038}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":34454,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6835/2597849957","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":2597849957},{"type":"v1","addr":"172.21.15.100:6835","nonce":2597849957}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6},{"gid":34493,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6837/3497644162","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6836","nonce":3497644162},{"type":"v1","addr":"172.21.15.100:6837","nonce":3497644162}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":9}],"filesystems":[]} 2024-03-21T10:21:38.518 INFO:teuthology.orchestra.run.smithi047.stderr:dumped fsmap epoch 9 2024-03-21T10:21:38.521 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:38.520+0000 7fb6e5000640 1 -- 172.21.15.47:0/1303744143 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7fb6c40a2fa0 msgr2=0x7fb6c40a5460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:21:38.521 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:38.520+0000 7fb6e5000640 1 --2- 172.21.15.47:0/1303744143 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7fb6c40a2fa0 0x7fb6c40a5460 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fb6e019f1b0 tx=0x7fb6c8005e50 comp rx=0 tx=0).stop 2024-03-21T10:21:38.521 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:38.520+0000 7fb6e5000640 1 -- 172.21.15.47:0/1303744143 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb6e0107b00 msgr2=0x7fb6e019dcb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:21:38.521 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:38.520+0000 7fb6e5000640 1 --2- 172.21.15.47:0/1303744143 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb6e0107b00 0x7fb6e019dcb0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fb6d0005ab0 tx=0x7fb6d00042f0 comp rx=0 tx=0).stop 2024-03-21T10:21:38.521 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:38.520+0000 7fb6e5000640 1 -- 172.21.15.47:0/1303744143 shutdown_connections 2024-03-21T10:21:38.521 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:38.520+0000 7fb6e5000640 1 --2- 172.21.15.47:0/1303744143 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7fb6c40a2fa0 0x7fb6c40a5460 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:21:38.521 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:38.520+0000 7fb6e5000640 1 --2- 172.21.15.47:0/1303744143 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb6e01096b0 0x7fb6e01a5250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:21:38.521 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:38.520+0000 7fb6e5000640 1 --2- 172.21.15.47:0/1303744143 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb6e0108cf0 0x7fb6e019e1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:21:38.521 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:38.520+0000 7fb6e5000640 1 --2- 172.21.15.47:0/1303744143 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb6e0107b00 0x7fb6e019dcb0 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:21:38.522 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:38.520+0000 7fb6e5000640 1 -- 172.21.15.47:0/1303744143 >> 172.21.15.47:0/1303744143 conn(0x7fb6e0076ab0 msgr2=0x7fb6e010bff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:21:38.522 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:38.520+0000 7fb6e5000640 1 -- 172.21.15.47:0/1303744143 shutdown_connections 2024-03-21T10:21:38.522 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:38.520+0000 7fb6e5000640 1 -- 172.21.15.47:0/1303744143 wait complete. 2024-03-21T10:21:39.124 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:21:38 smithi157 ceph-mon[35581]: Deploying daemon mds.i on smithi157 2024-03-21T10:21:39.124 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:21:38 smithi157 ceph-mon[35581]: mds.? [v2:172.21.15.100:6836/3497644162,v1:172.21.15.100:6837/3497644162] up:boot 2024-03-21T10:21:39.124 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:21:38 smithi157 ceph-mon[35581]: fsmap 8 up:standby 2024-03-21T10:21:39.124 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:21:38 smithi157 ceph-mon[35581]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-03-21T10:21:39.124 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:21:38 smithi157 ceph-mon[35581]: from='client.? 172.21.15.47:0/1303744143' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-21T10:21:39.214 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:21:38 smithi047 ceph-mon[31547]: Deploying daemon mds.i on smithi157 2024-03-21T10:21:39.214 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:21:38 smithi047 ceph-mon[31547]: mds.? [v2:172.21.15.100:6836/3497644162,v1:172.21.15.100:6837/3497644162] up:boot 2024-03-21T10:21:39.214 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:21:38 smithi047 ceph-mon[31547]: fsmap 8 up:standby 2024-03-21T10:21:39.214 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:21:38 smithi047 ceph-mon[31547]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-03-21T10:21:39.215 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:21:38 smithi047 ceph-mon[31547]: from='client.? 172.21.15.47:0/1303744143' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-21T10:21:39.229 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:21:38 smithi100 ceph-mon[35193]: Deploying daemon mds.i on smithi157 2024-03-21T10:21:39.230 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:21:38 smithi100 ceph-mon[35193]: mds.? [v2:172.21.15.100:6836/3497644162,v1:172.21.15.100:6837/3497644162] up:boot 2024-03-21T10:21:39.230 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:21:38 smithi100 ceph-mon[35193]: fsmap 8 up:standby 2024-03-21T10:21:39.230 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:21:38 smithi100 ceph-mon[35193]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-03-21T10:21:39.230 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:21:38 smithi100 ceph-mon[35193]: from='client.? 172.21.15.47:0/1303744143' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-21T10:21:39.989 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:21:39 smithi157 ceph-mon[35581]: pgmap v349: 1 pgs: 1 active+clean; 577 KiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-21T10:21:40.214 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:21:39 smithi047 ceph-mon[31547]: pgmap v349: 1 pgs: 1 active+clean; 577 KiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-21T10:21:40.229 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:21:39 smithi100 ceph-mon[35193]: pgmap v349: 1 pgs: 1 active+clean; 577 KiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-21T10:21:40.540 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-21T10:21:40.777 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:40.775+0000 7f2f4dc00640 1 -- 172.21.15.47:0/2430973572 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2f48107ce0 msgr2=0x7f2f48108100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:21:40.777 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:40.775+0000 7f2f4dc00640 1 --2- 172.21.15.47:0/2430973572 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2f48107ce0 0x7f2f48108100 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f2f38009900 tx=0x7f2f3802bc30 comp rx=0 tx=0).stop 2024-03-21T10:21:40.777 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:40.776+0000 7f2f4dc00640 1 -- 172.21.15.47:0/2430973572 shutdown_connections 2024-03-21T10:21:40.777 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:40.776+0000 7f2f4dc00640 1 --2- 172.21.15.47:0/2430973572 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2f48109890 0x7f2f48110120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:21:40.777 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:40.776+0000 7f2f4dc00640 1 --2- 172.21.15.47:0/2430973572 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2f48108ed0 0x7f2f48109350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:21:40.777 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:40.776+0000 7f2f4dc00640 1 --2- 172.21.15.47:0/2430973572 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2f48107ce0 0x7f2f48108100 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:21:40.777 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:40.776+0000 7f2f4dc00640 1 -- 172.21.15.47:0/2430973572 >> 172.21.15.47:0/2430973572 conn(0x7f2f48076cb0 msgr2=0x7f2f480790d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:21:40.777 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:40.776+0000 7f2f4dc00640 1 -- 172.21.15.47:0/2430973572 shutdown_connections 2024-03-21T10:21:40.778 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:40.776+0000 7f2f4dc00640 1 -- 172.21.15.47:0/2430973572 wait complete. 2024-03-21T10:21:40.778 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:40.777+0000 7f2f4dc00640 1 Processor -- start 2024-03-21T10:21:40.778 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:40.777+0000 7f2f4dc00640 1 -- start start 2024-03-21T10:21:40.780 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:40.778+0000 7f2f4dc00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2f48107ce0 0x7f2f4819def0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:21:40.780 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:40.778+0000 7f2f4dc00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2f48108ed0 0x7f2f4819e430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:21:40.780 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:40.778+0000 7f2f4dc00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2f48109890 0x7f2f481a54a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:21:40.780 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:40.778+0000 7f2f4dc00640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f48113810 con 0x7f2f48109890 2024-03-21T10:21:40.780 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:40.778+0000 7f2f4dc00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f48113690 con 0x7f2f48107ce0 2024-03-21T10:21:40.780 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:40.778+0000 7f2f4dc00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f48113990 con 0x7f2f48108ed0 2024-03-21T10:21:40.780 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:40.778+0000 7f2f47400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2f48109890 0x7f2f481a54a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:21:40.780 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:40.778+0000 7f2f47400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2f48109890 0x7f2f481a54a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:56116/0 (socket says 172.21.15.47:56116) 2024-03-21T10:21:40.780 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:40.778+0000 7f2f47400640 1 -- 172.21.15.47:0/1733045635 learned_addr learned my addr 172.21.15.47:0/1733045635 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:21:40.781 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:40.778+0000 7f2f3e000640 1 --2- 172.21.15.47:0/1733045635 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2f48108ed0 0x7f2f4819e430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:21:40.781 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:40.778+0000 7f2f46a00640 1 --2- 172.21.15.47:0/1733045635 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2f48107ce0 0x7f2f4819def0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:21:40.781 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:40.779+0000 7f2f47400640 1 -- 172.21.15.47:0/1733045635 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2f48108ed0 msgr2=0x7f2f4819e430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:21:40.781 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:40.779+0000 7f2f47400640 1 --2- 172.21.15.47:0/1733045635 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2f48108ed0 0x7f2f4819e430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:21:40.781 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:40.779+0000 7f2f47400640 1 -- 172.21.15.47:0/1733045635 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2f48107ce0 msgr2=0x7f2f4819def0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:21:40.782 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:40.779+0000 7f2f47400640 1 --2- 172.21.15.47:0/1733045635 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2f48107ce0 0x7f2f4819def0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:21:40.782 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:40.779+0000 7f2f47400640 1 -- 172.21.15.47:0/1733045635 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f481a5b10 con 0x7f2f48109890 2024-03-21T10:21:40.783 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:40.779+0000 7f2f47400640 1 --2- 172.21.15.47:0/1733045635 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2f48109890 0x7f2f481a54a0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f2f4000eaa0 tx=0x7f2f4000ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:21:40.783 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:40.779+0000 7f2f3fe00640 1 -- 172.21.15.47:0/1733045635 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f4000cdb0 con 0x7f2f48109890 2024-03-21T10:21:40.783 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:40.779+0000 7f2f3fe00640 1 -- 172.21.15.47:0/1733045635 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f2f40005020 con 0x7f2f48109890 2024-03-21T10:21:40.783 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:40.779+0000 7f2f3fe00640 1 -- 172.21.15.47:0/1733045635 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f400116d0 con 0x7f2f48109890 2024-03-21T10:21:40.783 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:40.779+0000 7f2f4dc00640 1 -- 172.21.15.47:0/1733045635 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2f481a5e00 con 0x7f2f48109890 2024-03-21T10:21:40.783 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:40.780+0000 7f2f4dc00640 1 -- 172.21.15.47:0/1733045635 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2f481a62c0 con 0x7f2f48109890 2024-03-21T10:21:40.783 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:40.781+0000 7f2f4dc00640 1 -- 172.21.15.47:0/1733045635 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2f4810f080 con 0x7f2f48109890 2024-03-21T10:21:40.783 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:40.783+0000 7f2f3fe00640 1 -- 172.21.15.47:0/1733045635 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 16) v1 ==== 137854+0+0 (secure 0 0 0) 0x7f2f400051d0 con 0x7f2f48109890 2024-03-21T10:21:40.785 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:40.784+0000 7f2f3fe00640 1 --2- 172.21.15.47:0/1733045635 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f2f2c0a2f20 0x7f2f2c0a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:21:40.785 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:40.784+0000 7f2f3fe00640 1 -- 172.21.15.47:0/1733045635 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f2f400cf230 con 0x7f2f48109890 2024-03-21T10:21:40.785 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:40.784+0000 7f2f46a00640 1 --2- 172.21.15.47:0/1733045635 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f2f2c0a2f20 0x7f2f2c0a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:21:40.786 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:40.785+0000 7f2f46a00640 1 --2- 172.21.15.47:0/1733045635 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f2f2c0a2f20 0x7f2f2c0a53e0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f2f38006010 tx=0x7f2f38005be0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:21:40.789 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:40.789+0000 7f2f3fe00640 1 -- 172.21.15.47:0/1733045635 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f2f40016030 con 0x7f2f48109890 2024-03-21T10:21:41.030 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:41.029+0000 7f2f4dc00640 1 -- 172.21.15.47:0/1733045635 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f2f4802d070 con 0x7f2f48109890 2024-03-21T10:21:41.031 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:41.030+0000 7f2f3fe00640 1 -- 172.21.15.47:0/1733045635 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) v1 ==== 93+0+6877 (secure 0 0 0) 0x7f2f4009c460 con 0x7f2f48109890 2024-03-21T10:21:41.031 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-21T10:21:41.032 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":9,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14511,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6835/735724037","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6834","nonce":735724037},{"type":"v1","addr":"172.21.15.47:6835","nonce":735724037}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14529,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6837/2248191724","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6836","nonce":2248191724},{"type":"v1","addr":"172.21.15.47:6837","nonce":2248191724}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":24406,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6833/4000010136","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":4000010136},{"type":"v1","addr":"172.21.15.157:6833","nonce":4000010136}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24457,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6835/3902868651","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":3902868651},{"type":"v1","addr":"172.21.15.157:6835","nonce":3902868651}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":7},{"gid":24475,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6837/1908087204","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6836","nonce":1908087204},{"type":"v1","addr":"172.21.15.157:6837","nonce":1908087204}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":8},{"gid":34427,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6833/2723302038","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6832","nonce":2723302038},{"type":"v1","addr":"172.21.15.100:6833","nonce":2723302038}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":34454,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6835/2597849957","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":2597849957},{"type":"v1","addr":"172.21.15.100:6835","nonce":2597849957}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6},{"gid":34493,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6837/3497644162","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6836","nonce":3497644162},{"type":"v1","addr":"172.21.15.100:6837","nonce":3497644162}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":9}],"filesystems":[]} 2024-03-21T10:21:41.032 INFO:teuthology.orchestra.run.smithi047.stderr:dumped fsmap epoch 9 2024-03-21T10:21:41.034 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:41.033+0000 7f2f4dc00640 1 -- 172.21.15.47:0/1733045635 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f2f2c0a2f20 msgr2=0x7f2f2c0a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:21:41.034 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:41.033+0000 7f2f4dc00640 1 --2- 172.21.15.47:0/1733045635 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f2f2c0a2f20 0x7f2f2c0a53e0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f2f38006010 tx=0x7f2f38005be0 comp rx=0 tx=0).stop 2024-03-21T10:21:41.034 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:41.033+0000 7f2f4dc00640 1 -- 172.21.15.47:0/1733045635 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2f48109890 msgr2=0x7f2f481a54a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:21:41.034 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:41.033+0000 7f2f4dc00640 1 --2- 172.21.15.47:0/1733045635 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2f48109890 0x7f2f481a54a0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f2f4000eaa0 tx=0x7f2f4000ef60 comp rx=0 tx=0).stop 2024-03-21T10:21:41.034 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:41.033+0000 7f2f4dc00640 1 -- 172.21.15.47:0/1733045635 shutdown_connections 2024-03-21T10:21:41.034 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:41.033+0000 7f2f4dc00640 1 --2- 172.21.15.47:0/1733045635 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f2f2c0a2f20 0x7f2f2c0a53e0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:21:41.034 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:41.033+0000 7f2f4dc00640 1 --2- 172.21.15.47:0/1733045635 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2f48109890 0x7f2f481a54a0 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:21:41.034 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:41.033+0000 7f2f4dc00640 1 --2- 172.21.15.47:0/1733045635 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2f48108ed0 0x7f2f4819e430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:21:41.035 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:41.033+0000 7f2f4dc00640 1 --2- 172.21.15.47:0/1733045635 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2f48107ce0 0x7f2f4819def0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:21:41.035 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:41.033+0000 7f2f4dc00640 1 -- 172.21.15.47:0/1733045635 >> 172.21.15.47:0/1733045635 conn(0x7f2f48076cb0 msgr2=0x7f2f4810c250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:21:41.035 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:41.033+0000 7f2f4dc00640 1 -- 172.21.15.47:0/1733045635 shutdown_connections 2024-03-21T10:21:41.035 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:41.033+0000 7f2f4dc00640 1 -- 172.21.15.47:0/1733045635 wait complete. 2024-03-21T10:21:41.175 INFO:journalctl@ceph.mds.i.smithi157.stdout:Mar 21 10:21:40 smithi157 systemd[1]: Starting Ceph mds.i for 3056d51a-e76b-11ee-95cd-87774f69a715... 2024-03-21T10:21:41.468 INFO:journalctl@ceph.mds.i.smithi157.stdout:Mar 21 10:21:41 smithi157 podman[66584]: 2024-03-21 10:21:41.079304324 +0000 UTC m=+0.032222056 image pull 237e43c7a65ff48534d94b97c1bfdbe4b7be5672e3d9a613745083ea3702aa54 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232 2024-03-21T10:21:41.468 INFO:journalctl@ceph.mds.i.smithi157.stdout:Mar 21 10:21:41 smithi157 podman[66584]: 2024-03-21 10:21:41.187030701 +0000 UTC m=+0.139948497 container create 9a440ce57d234172d1c605e0aaa9e951fbf775c9f25ec84d0874e522ec3a2262 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-3056d51a-e76b-11ee-95cd-87774f69a715-mds-i, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240319, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, CEPH_POINT_RELEASE=, ceph=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_CLEAN=True) 2024-03-21T10:21:41.468 INFO:journalctl@ceph.mds.i.smithi157.stdout:Mar 21 10:21:41 smithi157 podman[66584]: 2024-03-21 10:21:41.466055122 +0000 UTC m=+0.418972897 container init 9a440ce57d234172d1c605e0aaa9e951fbf775c9f25ec84d0874e522ec3a2262 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-3056d51a-e76b-11ee-95cd-87774f69a715-mds-i, org.label-schema.build-date=20240319, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, ceph=True, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS) 2024-03-21T10:21:41.741 INFO:journalctl@ceph.mds.i.smithi157.stdout:Mar 21 10:21:41 smithi157 podman[66584]: 2024-03-21 10:21:41.476856181 +0000 UTC m=+0.429773918 container start 9a440ce57d234172d1c605e0aaa9e951fbf775c9f25ec84d0874e522ec3a2262 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-3056d51a-e76b-11ee-95cd-87774f69a715-mds-i, org.label-schema.build-date=20240319, GIT_CLEAN=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-21T10:21:41.741 INFO:journalctl@ceph.mds.i.smithi157.stdout:Mar 21 10:21:41 smithi157 ceph-mds[66603]: -- 172.21.15.157:0/3496021641 <== mon.1 v2:172.21.15.157:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x558fbbb17a40 con 0x558fbbaeac00 2024-03-21T10:21:41.741 INFO:journalctl@ceph.mds.i.smithi157.stdout:Mar 21 10:21:41 smithi157 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-mds-i[66599]: starting mds.i at 2024-03-21T10:21:41.741 INFO:journalctl@ceph.mds.i.smithi157.stdout:Mar 21 10:21:41 smithi157 bash[66584]: 9a440ce57d234172d1c605e0aaa9e951fbf775c9f25ec84d0874e522ec3a2262 2024-03-21T10:21:41.741 INFO:journalctl@ceph.mds.i.smithi157.stdout:Mar 21 10:21:41 smithi157 systemd[1]: Started Ceph mds.i for 3056d51a-e76b-11ee-95cd-87774f69a715. 2024-03-21T10:21:42.214 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:21:41 smithi047 ceph-mon[31547]: pgmap v350: 1 pgs: 1 active+clean; 577 KiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-21T10:21:42.215 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:21:41 smithi047 ceph-mon[31547]: from='client.? 172.21.15.47:0/1733045635' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-21T10:21:42.215 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:21:41 smithi047 ceph-mon[31547]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:21:42.215 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:21:41 smithi047 ceph-mon[31547]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:21:42.215 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:21:41 smithi047 ceph-mon[31547]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:21:42.215 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:21:41 smithi047 ceph-mon[31547]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-21T10:21:42.215 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:21:41 smithi047 ceph-mon[31547]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-21T10:21:42.215 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:21:41 smithi047 ceph-mon[31547]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T10:21:42.230 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:21:41 smithi100 ceph-mon[35193]: pgmap v350: 1 pgs: 1 active+clean; 577 KiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-21T10:21:42.230 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:21:41 smithi100 ceph-mon[35193]: from='client.? 172.21.15.47:0/1733045635' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-21T10:21:42.230 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:21:41 smithi100 ceph-mon[35193]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:21:42.230 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:21:41 smithi100 ceph-mon[35193]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:21:42.231 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:21:41 smithi100 ceph-mon[35193]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:21:42.231 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:21:41 smithi100 ceph-mon[35193]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-21T10:21:42.231 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:21:41 smithi100 ceph-mon[35193]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-21T10:21:42.231 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:21:41 smithi100 ceph-mon[35193]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T10:21:42.240 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:21:41 smithi157 ceph-mon[35581]: pgmap v350: 1 pgs: 1 active+clean; 577 KiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-21T10:21:42.240 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:21:41 smithi157 ceph-mon[35581]: from='client.? 172.21.15.47:0/1733045635' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-21T10:21:42.240 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:21:41 smithi157 ceph-mon[35581]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:21:42.240 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:21:41 smithi157 ceph-mon[35581]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:21:42.240 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:21:41 smithi157 ceph-mon[35581]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:21:42.240 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:21:41 smithi157 ceph-mon[35581]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-21T10:21:42.240 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:21:41 smithi157 ceph-mon[35581]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-21T10:21:42.241 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:21:41 smithi157 ceph-mon[35581]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T10:21:43.055 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-21T10:21:43.077 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:21:42 smithi047 ceph-mon[31547]: Deploying daemon mds.g on smithi047 2024-03-21T10:21:43.077 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:21:42 smithi047 ceph-mon[31547]: mds.? [v2:172.21.15.157:6838/4064242876,v1:172.21.15.157:6839/4064242876] up:boot 2024-03-21T10:21:43.077 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:21:42 smithi047 ceph-mon[31547]: fsmap 9 up:standby 2024-03-21T10:21:43.077 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:21:42 smithi047 ceph-mon[31547]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-03-21T10:21:43.230 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:21:42 smithi100 ceph-mon[35193]: Deploying daemon mds.g on smithi047 2024-03-21T10:21:43.230 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:21:42 smithi100 ceph-mon[35193]: mds.? [v2:172.21.15.157:6838/4064242876,v1:172.21.15.157:6839/4064242876] up:boot 2024-03-21T10:21:43.230 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:21:42 smithi100 ceph-mon[35193]: fsmap 9 up:standby 2024-03-21T10:21:43.230 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:21:42 smithi100 ceph-mon[35193]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-03-21T10:21:43.240 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:21:42 smithi157 ceph-mon[35581]: Deploying daemon mds.g on smithi047 2024-03-21T10:21:43.240 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:21:42 smithi157 ceph-mon[35581]: mds.? [v2:172.21.15.157:6838/4064242876,v1:172.21.15.157:6839/4064242876] up:boot 2024-03-21T10:21:43.240 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:21:42 smithi157 ceph-mon[35581]: fsmap 9 up:standby 2024-03-21T10:21:43.240 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:21:42 smithi157 ceph-mon[35581]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-03-21T10:21:43.280 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:43.278+0000 7ff369400640 1 -- 172.21.15.47:0/2893609067 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff364107ce0 msgr2=0x7ff364108100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:21:43.280 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:43.278+0000 7ff369400640 1 --2- 172.21.15.47:0/2893609067 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff364107ce0 0x7ff364108100 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7ff354009900 tx=0x7ff35402bc40 comp rx=0 tx=0).stop 2024-03-21T10:21:43.280 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:43.279+0000 7ff369400640 1 -- 172.21.15.47:0/2893609067 shutdown_connections 2024-03-21T10:21:43.280 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:43.279+0000 7ff369400640 1 --2- 172.21.15.47:0/2893609067 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff364109890 0x7ff364110120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:21:43.280 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:43.279+0000 7ff369400640 1 --2- 172.21.15.47:0/2893609067 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff364108ed0 0x7ff364109350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:21:43.280 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:43.279+0000 7ff369400640 1 --2- 172.21.15.47:0/2893609067 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff364107ce0 0x7ff364108100 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:21:43.280 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:43.279+0000 7ff369400640 1 -- 172.21.15.47:0/2893609067 >> 172.21.15.47:0/2893609067 conn(0x7ff364076cb0 msgr2=0x7ff3640790d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:21:43.280 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:43.279+0000 7ff369400640 1 -- 172.21.15.47:0/2893609067 shutdown_connections 2024-03-21T10:21:43.281 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:43.279+0000 7ff369400640 1 -- 172.21.15.47:0/2893609067 wait complete. 2024-03-21T10:21:43.283 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:43.280+0000 7ff369400640 1 Processor -- start 2024-03-21T10:21:43.283 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:43.280+0000 7ff369400640 1 -- start start 2024-03-21T10:21:43.283 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:43.280+0000 7ff369400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff364107ce0 0x7ff36419e060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:21:43.283 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:43.280+0000 7ff369400640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff364108ed0 0x7ff36419e5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:21:43.283 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:43.280+0000 7ff369400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff364109890 0x7ff3641a5610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:21:43.283 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:43.280+0000 7ff369400640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3641138e0 con 0x7ff364107ce0 2024-03-21T10:21:43.283 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:43.280+0000 7ff369400640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff364113760 con 0x7ff364109890 2024-03-21T10:21:43.283 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:43.280+0000 7ff369400640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff364113a60 con 0x7ff364108ed0 2024-03-21T10:21:43.283 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:43.280+0000 7ff362800640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff364107ce0 0x7ff36419e060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:21:43.283 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:43.280+0000 7ff362800640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff364107ce0 0x7ff36419e060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:60256/0 (socket says 172.21.15.47:60256) 2024-03-21T10:21:43.283 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:43.280+0000 7ff362800640 1 -- 172.21.15.47:0/236628055 learned_addr learned my addr 172.21.15.47:0/236628055 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:21:43.284 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:43.281+0000 7ff362800640 1 -- 172.21.15.47:0/236628055 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff364108ed0 msgr2=0x7ff36419e5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:21:43.284 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:43.281+0000 7ff35be00640 1 --2- 172.21.15.47:0/236628055 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff364108ed0 0x7ff36419e5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:21:43.284 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:43.281+0000 7ff363200640 1 --2- 172.21.15.47:0/236628055 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff364109890 0x7ff3641a5610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:21:43.284 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:43.281+0000 7ff362800640 1 --2- 172.21.15.47:0/236628055 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff364108ed0 0x7ff36419e5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:21:43.284 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:43.281+0000 7ff362800640 1 -- 172.21.15.47:0/236628055 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff364109890 msgr2=0x7ff3641a5610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:21:43.284 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:43.281+0000 7ff362800640 1 --2- 172.21.15.47:0/236628055 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff364109890 0x7ff3641a5610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:21:43.284 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:43.281+0000 7ff362800640 1 -- 172.21.15.47:0/236628055 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff3641a5c80 con 0x7ff364107ce0 2024-03-21T10:21:43.284 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:43.281+0000 7ff35be00640 1 --2- 172.21.15.47:0/236628055 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff364108ed0 0x7ff36419e5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T10:21:43.284 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:43.281+0000 7ff362800640 1 --2- 172.21.15.47:0/236628055 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff364107ce0 0x7ff36419e060 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7ff354036040 tx=0x7ff3540042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:21:43.284 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:43.281+0000 7ff35b400640 1 -- 172.21.15.47:0/236628055 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff354043020 con 0x7ff364107ce0 2024-03-21T10:21:43.284 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:43.281+0000 7ff35b400640 1 -- 172.21.15.47:0/236628055 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7ff354004aa0 con 0x7ff364107ce0 2024-03-21T10:21:43.284 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:43.281+0000 7ff35b400640 1 -- 172.21.15.47:0/236628055 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff354003de0 con 0x7ff364107ce0 2024-03-21T10:21:43.284 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:43.281+0000 7ff369400640 1 -- 172.21.15.47:0/236628055 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff3641a5f10 con 0x7ff364107ce0 2024-03-21T10:21:43.284 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:43.282+0000 7ff369400640 1 -- 172.21.15.47:0/236628055 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff3641a63f0 con 0x7ff364107ce0 2024-03-21T10:21:43.285 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:43.283+0000 7ff369400640 1 -- 172.21.15.47:0/236628055 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff3640688e0 con 0x7ff364107ce0 2024-03-21T10:21:43.285 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:43.285+0000 7ff35b400640 1 -- 172.21.15.47:0/236628055 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 16) v1 ==== 137854+0+0 (secure 0 0 0) 0x7ff354046020 con 0x7ff364107ce0 2024-03-21T10:21:43.288 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:43.287+0000 7ff35b400640 1 --2- 172.21.15.47:0/236628055 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7ff3480a2f20 0x7ff3480a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:21:43.288 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:43.287+0000 7ff35b400640 1 -- 172.21.15.47:0/236628055 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 7127+0+0 (secure 0 0 0) 0x7ff3540f1470 con 0x7ff364107ce0 2024-03-21T10:21:43.288 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:43.287+0000 7ff35be00640 1 --2- 172.21.15.47:0/236628055 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7ff3480a2f20 0x7ff3480a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:21:43.289 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:43.288+0000 7ff35be00640 1 --2- 172.21.15.47:0/236628055 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7ff3480a2f20 0x7ff3480a53e0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7ff34c006fd0 tx=0x7ff34c008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:21:43.292 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:43.291+0000 7ff35b400640 1 -- 172.21.15.47:0/236628055 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7ff35404d020 con 0x7ff364107ce0 2024-03-21T10:21:43.537 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:43.536+0000 7ff369400640 1 -- 172.21.15.47:0/236628055 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff3640622a0 con 0x7ff364107ce0 2024-03-21T10:21:43.538 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:43.537+0000 7ff35b400640 1 -- 172.21.15.47:0/236628055 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 10 v10) v1 ==== 94+0+7673 (secure 0 0 0) 0x7ff3540be6a0 con 0x7ff364107ce0 2024-03-21T10:21:43.538 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-21T10:21:43.538 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":10,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14511,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6835/735724037","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6834","nonce":735724037},{"type":"v1","addr":"172.21.15.47:6835","nonce":735724037}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14529,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6837/2248191724","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6836","nonce":2248191724},{"type":"v1","addr":"172.21.15.47:6837","nonce":2248191724}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":24406,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6833/4000010136","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":4000010136},{"type":"v1","addr":"172.21.15.157:6833","nonce":4000010136}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24457,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6835/3902868651","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":3902868651},{"type":"v1","addr":"172.21.15.157:6835","nonce":3902868651}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":7},{"gid":24475,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6837/1908087204","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6836","nonce":1908087204},{"type":"v1","addr":"172.21.15.157:6837","nonce":1908087204}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":8},{"gid":24499,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6839/4064242876","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6838","nonce":4064242876},{"type":"v1","addr":"172.21.15.157:6839","nonce":4064242876}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":10},{"gid":34427,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6833/2723302038","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6832","nonce":2723302038},{"type":"v1","addr":"172.21.15.100:6833","nonce":2723302038}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":34454,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6835/2597849957","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":2597849957},{"type":"v1","addr":"172.21.15.100:6835","nonce":2597849957}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6},{"gid":34493,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6837/3497644162","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6836","nonce":3497644162},{"type":"v1","addr":"172.21.15.100:6837","nonce":3497644162}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":9}],"filesystems":[]} 2024-03-21T10:21:43.539 INFO:teuthology.orchestra.run.smithi047.stderr:dumped fsmap epoch 10 2024-03-21T10:21:43.541 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:43.539+0000 7ff369400640 1 -- 172.21.15.47:0/236628055 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7ff3480a2f20 msgr2=0x7ff3480a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:21:43.541 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:43.539+0000 7ff369400640 1 --2- 172.21.15.47:0/236628055 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7ff3480a2f20 0x7ff3480a53e0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7ff34c006fd0 tx=0x7ff34c008040 comp rx=0 tx=0).stop 2024-03-21T10:21:43.541 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:43.539+0000 7ff369400640 1 -- 172.21.15.47:0/236628055 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff364107ce0 msgr2=0x7ff36419e060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:21:43.541 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:43.539+0000 7ff369400640 1 --2- 172.21.15.47:0/236628055 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff364107ce0 0x7ff36419e060 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7ff354036040 tx=0x7ff3540042f0 comp rx=0 tx=0).stop 2024-03-21T10:21:43.541 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:43.540+0000 7ff369400640 1 -- 172.21.15.47:0/236628055 shutdown_connections 2024-03-21T10:21:43.541 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:43.540+0000 7ff369400640 1 --2- 172.21.15.47:0/236628055 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7ff3480a2f20 0x7ff3480a53e0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:21:43.541 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:43.540+0000 7ff369400640 1 --2- 172.21.15.47:0/236628055 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff364109890 0x7ff3641a5610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:21:43.541 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:43.540+0000 7ff369400640 1 --2- 172.21.15.47:0/236628055 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff364108ed0 0x7ff36419e5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:21:43.542 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:43.540+0000 7ff369400640 1 --2- 172.21.15.47:0/236628055 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff364107ce0 0x7ff36419e060 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:21:43.542 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:43.540+0000 7ff369400640 1 -- 172.21.15.47:0/236628055 >> 172.21.15.47:0/236628055 conn(0x7ff364076cb0 msgr2=0x7ff36410c2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:21:43.542 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:43.540+0000 7ff369400640 1 -- 172.21.15.47:0/236628055 shutdown_connections 2024-03-21T10:21:43.542 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:43.540+0000 7ff369400640 1 -- 172.21.15.47:0/236628055 wait complete. 2024-03-21T10:21:44.214 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:21:43 smithi047 ceph-mon[31547]: pgmap v351: 1 pgs: 1 active+clean; 577 KiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-21T10:21:44.214 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:21:43 smithi047 ceph-mon[31547]: from='client.? 172.21.15.47:0/236628055' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-21T10:21:44.229 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:21:43 smithi100 ceph-mon[35193]: pgmap v351: 1 pgs: 1 active+clean; 577 KiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-21T10:21:44.229 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:21:43 smithi100 ceph-mon[35193]: from='client.? 172.21.15.47:0/236628055' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-21T10:21:44.240 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:21:43 smithi157 ceph-mon[35581]: pgmap v351: 1 pgs: 1 active+clean; 577 KiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-21T10:21:44.240 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:21:43 smithi157 ceph-mon[35581]: from='client.? 172.21.15.47:0/236628055' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-21T10:21:45.214 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:21:44 smithi047 ceph-mon[31547]: pgmap v352: 1 pgs: 1 active+clean; 577 KiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-21T10:21:45.214 INFO:journalctl@ceph.mds.g.smithi047.stdout:Mar 21 10:21:44 smithi047 systemd[1]: Starting Ceph mds.g for 3056d51a-e76b-11ee-95cd-87774f69a715... 2024-03-21T10:21:45.229 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:21:44 smithi100 ceph-mon[35193]: pgmap v352: 1 pgs: 1 active+clean; 577 KiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-21T10:21:45.239 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:21:44 smithi157 ceph-mon[35581]: pgmap v352: 1 pgs: 1 active+clean; 577 KiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-21T10:21:45.540 INFO:journalctl@ceph.mds.g.smithi047.stdout:Mar 21 10:21:45 smithi047 podman[70769]: 2024-03-21 10:21:45.19130633 +0000 UTC m=+0.043398366 image pull 237e43c7a65ff48534d94b97c1bfdbe4b7be5672e3d9a613745083ea3702aa54 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232 2024-03-21T10:21:45.540 INFO:journalctl@ceph.mds.g.smithi047.stdout:Mar 21 10:21:45 smithi047 podman[70769]: 2024-03-21 10:21:45.321970759 +0000 UTC m=+0.174062825 container create b018904203b9a1d608f6ca4316601ca6e5a1896b34b908959c9eebd0f12f0dcc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-3056d51a-e76b-11ee-95cd-87774f69a715-mds-g, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240319, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, ceph=True) 2024-03-21T10:21:45.563 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-21T10:21:45.787 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:45.785+0000 7f1687800640 1 -- 172.21.15.47:0/264214235 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1680105490 msgr2=0x7f168010bd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:21:45.787 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:45.785+0000 7f1687800640 1 --2- 172.21.15.47:0/264214235 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1680105490 0x7f168010bd20 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f1678009ca0 tx=0x7f167802c0f0 comp rx=0 tx=0).stop 2024-03-21T10:21:45.787 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:45.786+0000 7f1687800640 1 -- 172.21.15.47:0/264214235 shutdown_connections 2024-03-21T10:21:45.787 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:45.786+0000 7f1687800640 1 --2- 172.21.15.47:0/264214235 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1680105490 0x7f168010bd20 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:21:45.787 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:45.786+0000 7f1687800640 1 --2- 172.21.15.47:0/264214235 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1680104ad0 0x7f1680104f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:21:45.787 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:45.786+0000 7f1687800640 1 --2- 172.21.15.47:0/264214235 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f16801038e0 0x7f1680103d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:21:45.787 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:45.786+0000 7f1687800640 1 -- 172.21.15.47:0/264214235 >> 172.21.15.47:0/264214235 conn(0x7f16800fee90 msgr2=0x7f16801012b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:21:45.787 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:45.786+0000 7f1687800640 1 -- 172.21.15.47:0/264214235 shutdown_connections 2024-03-21T10:21:45.788 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:45.786+0000 7f1687800640 1 -- 172.21.15.47:0/264214235 wait complete. 2024-03-21T10:21:45.788 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:45.787+0000 7f1687800640 1 Processor -- start 2024-03-21T10:21:45.788 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:45.788+0000 7f1687800640 1 -- start start 2024-03-21T10:21:45.790 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:45.788+0000 7f1687800640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f16801038e0 0x7f1680199ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:21:45.790 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:45.788+0000 7f1687800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1680104ad0 0x7f168019a010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:21:45.790 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:45.788+0000 7f1687800640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1680105490 0x7f16801a1080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:21:45.790 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:45.788+0000 7f1687800640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f168010f410 con 0x7f16801038e0 2024-03-21T10:21:45.790 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:45.788+0000 7f1687800640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f168010f290 con 0x7f1680104ad0 2024-03-21T10:21:45.791 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:45.788+0000 7f1687800640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f168010f590 con 0x7f1680105490 2024-03-21T10:21:45.791 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:45.788+0000 7f1684e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f16801038e0 0x7f1680199ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:21:45.791 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:45.788+0000 7f1684e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f16801038e0 0x7f1680199ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:60298/0 (socket says 172.21.15.47:60298) 2024-03-21T10:21:45.791 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:45.788+0000 7f1684e00640 1 -- 172.21.15.47:0/2979982774 learned_addr learned my addr 172.21.15.47:0/2979982774 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:21:45.791 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:45.789+0000 7f167fe00640 1 --2- 172.21.15.47:0/2979982774 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1680104ad0 0x7f168019a010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:21:45.791 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:45.789+0000 7f1685800640 1 --2- 172.21.15.47:0/2979982774 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1680105490 0x7f16801a1080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:21:45.791 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:45.789+0000 7f1684e00640 1 -- 172.21.15.47:0/2979982774 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1680105490 msgr2=0x7f16801a1080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:21:45.791 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:45.789+0000 7f1684e00640 1 --2- 172.21.15.47:0/2979982774 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1680105490 0x7f16801a1080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:21:45.791 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:45.789+0000 7f1684e00640 1 -- 172.21.15.47:0/2979982774 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1680104ad0 msgr2=0x7f168019a010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:21:45.791 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:45.789+0000 7f1684e00640 1 --2- 172.21.15.47:0/2979982774 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1680104ad0 0x7f168019a010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:21:45.791 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:45.789+0000 7f1684e00640 1 -- 172.21.15.47:0/2979982774 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f16801a16f0 con 0x7f16801038e0 2024-03-21T10:21:45.791 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:45.789+0000 7f1684e00640 1 --2- 172.21.15.47:0/2979982774 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f16801038e0 0x7f1680199ad0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f167400e9c0 tx=0x7f167400ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:21:45.792 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:45.790+0000 7f167d600640 1 -- 172.21.15.47:0/2979982774 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f167400cd80 con 0x7f16801038e0 2024-03-21T10:21:45.792 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:45.790+0000 7f167d600640 1 -- 172.21.15.47:0/2979982774 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f16740035f0 con 0x7f16801038e0 2024-03-21T10:21:45.792 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:45.790+0000 7f1687800640 1 -- 172.21.15.47:0/2979982774 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f16801a19e0 con 0x7f16801038e0 2024-03-21T10:21:45.792 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:45.790+0000 7f167d600640 1 -- 172.21.15.47:0/2979982774 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1674010690 con 0x7f16801038e0 2024-03-21T10:21:45.792 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:45.790+0000 7f1687800640 1 -- 172.21.15.47:0/2979982774 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f16801a1f30 con 0x7f16801038e0 2024-03-21T10:21:45.792 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:45.791+0000 7f1687800640 1 -- 172.21.15.47:0/2979982774 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f168010acd0 con 0x7f16801038e0 2024-03-21T10:21:45.794 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:45.794+0000 7f167d600640 1 -- 172.21.15.47:0/2979982774 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 16) v1 ==== 137854+0+0 (secure 0 0 0) 0x7f16740037a0 con 0x7f16801038e0 2024-03-21T10:21:45.796 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:45.795+0000 7f167d600640 1 --2- 172.21.15.47:0/2979982774 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f16680a2fa0 0x7f16680a5460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:21:45.796 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:45.795+0000 7f167d600640 1 -- 172.21.15.47:0/2979982774 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f16740cf6d0 con 0x7f16801038e0 2024-03-21T10:21:45.796 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:45.795+0000 7f167fe00640 1 --2- 172.21.15.47:0/2979982774 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f16680a2fa0 0x7f16680a5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:21:45.797 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:45.796+0000 7f167fe00640 1 --2- 172.21.15.47:0/2979982774 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f16680a2fa0 0x7f16680a5460 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f168019afe0 tx=0x7f166c005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:21:45.804 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:45.803+0000 7f167d600640 1 -- 172.21.15.47:0/2979982774 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f1674016370 con 0x7f16801038e0 2024-03-21T10:21:45.935 INFO:journalctl@ceph.mds.g.smithi047.stdout:Mar 21 10:21:45 smithi047 podman[70769]: 2024-03-21 10:21:45.650944184 +0000 UTC m=+0.503036252 container init b018904203b9a1d608f6ca4316601ca6e5a1896b34b908959c9eebd0f12f0dcc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-3056d51a-e76b-11ee-95cd-87774f69a715-mds-g, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_CLEAN=True, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.build-date=20240319, CEPH_POINT_RELEASE=) 2024-03-21T10:21:45.935 INFO:journalctl@ceph.mds.g.smithi047.stdout:Mar 21 10:21:45 smithi047 podman[70769]: 2024-03-21 10:21:45.660549646 +0000 UTC m=+0.512641694 container start b018904203b9a1d608f6ca4316601ca6e5a1896b34b908959c9eebd0f12f0dcc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-3056d51a-e76b-11ee-95cd-87774f69a715-mds-g, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, GIT_CLEAN=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240319, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS) 2024-03-21T10:21:45.936 INFO:journalctl@ceph.mds.g.smithi047.stdout:Mar 21 10:21:45 smithi047 ceph-mds[70808]: -- 172.21.15.47:0/624979305 <== mon.0 v2:172.21.15.47:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x56390f025a40 con 0x56390eff8c00 2024-03-21T10:21:45.936 INFO:journalctl@ceph.mds.g.smithi047.stdout:Mar 21 10:21:45 smithi047 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-mds-g[70784]: starting mds.g at 2024-03-21T10:21:45.936 INFO:journalctl@ceph.mds.g.smithi047.stdout:Mar 21 10:21:45 smithi047 bash[70769]: b018904203b9a1d608f6ca4316601ca6e5a1896b34b908959c9eebd0f12f0dcc 2024-03-21T10:21:45.936 INFO:journalctl@ceph.mds.g.smithi047.stdout:Mar 21 10:21:45 smithi047 systemd[1]: Started Ceph mds.g for 3056d51a-e76b-11ee-95cd-87774f69a715. 2024-03-21T10:21:46.075 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:46.074+0000 7f1687800640 1 -- 172.21.15.47:0/2979982774 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f168002d070 con 0x7f16801038e0 2024-03-21T10:21:46.076 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:46.075+0000 7f167d600640 1 -- 172.21.15.47:0/2979982774 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 11 v11) v1 ==== 94+0+8465 (secure 0 0 0) 0x7f167409c9a0 con 0x7f16801038e0 2024-03-21T10:21:46.076 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-21T10:21:46.077 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":11,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14511,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6835/735724037","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6834","nonce":735724037},{"type":"v1","addr":"172.21.15.47:6835","nonce":735724037}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14529,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6837/2248191724","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6836","nonce":2248191724},{"type":"v1","addr":"172.21.15.47:6837","nonce":2248191724}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14604,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6839/2881335356","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6838","nonce":2881335356},{"type":"v1","addr":"172.21.15.47:6839","nonce":2881335356}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":24406,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6833/4000010136","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":4000010136},{"type":"v1","addr":"172.21.15.157:6833","nonce":4000010136}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24457,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6835/3902868651","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":3902868651},{"type":"v1","addr":"172.21.15.157:6835","nonce":3902868651}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":7},{"gid":24475,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6837/1908087204","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6836","nonce":1908087204},{"type":"v1","addr":"172.21.15.157:6837","nonce":1908087204}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":8},{"gid":24499,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6839/4064242876","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6838","nonce":4064242876},{"type":"v1","addr":"172.21.15.157:6839","nonce":4064242876}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":10},{"gid":34427,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6833/2723302038","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6832","nonce":2723302038},{"type":"v1","addr":"172.21.15.100:6833","nonce":2723302038}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":34454,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6835/2597849957","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":2597849957},{"type":"v1","addr":"172.21.15.100:6835","nonce":2597849957}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6},{"gid":34493,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6837/3497644162","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6836","nonce":3497644162},{"type":"v1","addr":"172.21.15.100:6837","nonce":3497644162}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":9}],"filesystems":[]} 2024-03-21T10:21:46.077 INFO:teuthology.orchestra.run.smithi047.stderr:dumped fsmap epoch 11 2024-03-21T10:21:46.080 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:46.079+0000 7f1687800640 1 -- 172.21.15.47:0/2979982774 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f16680a2fa0 msgr2=0x7f16680a5460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:21:46.080 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:46.079+0000 7f1687800640 1 --2- 172.21.15.47:0/2979982774 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f16680a2fa0 0x7f16680a5460 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f168019afe0 tx=0x7f166c005f90 comp rx=0 tx=0).stop 2024-03-21T10:21:46.080 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:46.079+0000 7f1687800640 1 -- 172.21.15.47:0/2979982774 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f16801038e0 msgr2=0x7f1680199ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:21:46.081 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:46.079+0000 7f1687800640 1 --2- 172.21.15.47:0/2979982774 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f16801038e0 0x7f1680199ad0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f167400e9c0 tx=0x7f167400ee80 comp rx=0 tx=0).stop 2024-03-21T10:21:46.081 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:46.079+0000 7f1687800640 1 -- 172.21.15.47:0/2979982774 shutdown_connections 2024-03-21T10:21:46.081 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:46.079+0000 7f1687800640 1 --2- 172.21.15.47:0/2979982774 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f16680a2fa0 0x7f16680a5460 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:21:46.081 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:46.079+0000 7f1687800640 1 --2- 172.21.15.47:0/2979982774 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1680105490 0x7f16801a1080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:21:46.081 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:46.079+0000 7f1687800640 1 --2- 172.21.15.47:0/2979982774 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1680104ad0 0x7f168019a010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:21:46.081 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:46.079+0000 7f1687800640 1 --2- 172.21.15.47:0/2979982774 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f16801038e0 0x7f1680199ad0 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:21:46.081 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:46.079+0000 7f1687800640 1 -- 172.21.15.47:0/2979982774 >> 172.21.15.47:0/2979982774 conn(0x7f16800fee90 msgr2=0x7f1680107ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:21:46.081 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:46.079+0000 7f1687800640 1 -- 172.21.15.47:0/2979982774 shutdown_connections 2024-03-21T10:21:46.081 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:46.079+0000 7f1687800640 1 -- 172.21.15.47:0/2979982774 wait complete. 2024-03-21T10:21:47.113 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:21:46 smithi047 ceph-mon[31547]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:21:47.113 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:21:46 smithi047 ceph-mon[31547]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:21:47.113 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:21:46 smithi047 ceph-mon[31547]: mds.? [v2:172.21.15.47:6838/2881335356,v1:172.21.15.47:6839/2881335356] up:boot 2024-03-21T10:21:47.113 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:21:46 smithi047 ceph-mon[31547]: fsmap 10 up:standby 2024-03-21T10:21:47.113 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:21:46 smithi047 ceph-mon[31547]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-03-21T10:21:47.114 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:21:46 smithi047 ceph-mon[31547]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:21:47.114 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:21:46 smithi047 ceph-mon[31547]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-21T10:21:47.114 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:21:46 smithi047 ceph-mon[31547]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-21T10:21:47.114 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:21:46 smithi047 ceph-mon[31547]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T10:21:47.114 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:21:46 smithi047 ceph-mon[31547]: from='client.? 172.21.15.47:0/2979982774' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-21T10:21:47.230 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:21:46 smithi100 ceph-mon[35193]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:21:47.230 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:21:46 smithi100 ceph-mon[35193]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:21:47.230 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:21:46 smithi100 ceph-mon[35193]: mds.? [v2:172.21.15.47:6838/2881335356,v1:172.21.15.47:6839/2881335356] up:boot 2024-03-21T10:21:47.230 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:21:46 smithi100 ceph-mon[35193]: fsmap 10 up:standby 2024-03-21T10:21:47.230 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:21:46 smithi100 ceph-mon[35193]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-03-21T10:21:47.230 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:21:46 smithi100 ceph-mon[35193]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:21:47.230 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:21:46 smithi100 ceph-mon[35193]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-21T10:21:47.230 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:21:46 smithi100 ceph-mon[35193]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-21T10:21:47.231 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:21:46 smithi100 ceph-mon[35193]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T10:21:47.231 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:21:46 smithi100 ceph-mon[35193]: from='client.? 172.21.15.47:0/2979982774' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-21T10:21:47.240 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:21:46 smithi157 ceph-mon[35581]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:21:47.240 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:21:46 smithi157 ceph-mon[35581]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:21:47.240 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:21:46 smithi157 ceph-mon[35581]: mds.? [v2:172.21.15.47:6838/2881335356,v1:172.21.15.47:6839/2881335356] up:boot 2024-03-21T10:21:47.240 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:21:46 smithi157 ceph-mon[35581]: fsmap 10 up:standby 2024-03-21T10:21:47.240 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:21:46 smithi157 ceph-mon[35581]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-03-21T10:21:47.240 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:21:46 smithi157 ceph-mon[35581]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:21:47.240 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:21:46 smithi157 ceph-mon[35581]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-21T10:21:47.240 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:21:46 smithi157 ceph-mon[35581]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-21T10:21:47.240 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:21:46 smithi157 ceph-mon[35581]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T10:21:47.241 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:21:46 smithi157 ceph-mon[35581]: from='client.? 172.21.15.47:0/2979982774' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-21T10:21:47.940 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:21:47 smithi047 ceph-mon[31547]: Deploying daemon mds.a on smithi047 2024-03-21T10:21:47.940 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:21:47 smithi047 ceph-mon[31547]: pgmap v353: 1 pgs: 1 active+clean; 577 KiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-21T10:21:48.106 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-21T10:21:48.132 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:21:47 smithi100 ceph-mon[35193]: Deploying daemon mds.a on smithi047 2024-03-21T10:21:48.132 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:21:47 smithi100 ceph-mon[35193]: pgmap v353: 1 pgs: 1 active+clean; 577 KiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-21T10:21:48.240 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:21:47 smithi157 ceph-mon[35581]: Deploying daemon mds.a on smithi047 2024-03-21T10:21:48.240 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:21:47 smithi157 ceph-mon[35581]: pgmap v353: 1 pgs: 1 active+clean; 577 KiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-21T10:21:48.629 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:48.628+0000 7f6bbb200640 1 -- 172.21.15.47:0/3788491004 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6bb410b780 msgr2=0x7f6bb410db70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:21:48.630 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:48.628+0000 7f6bbb200640 1 --2- 172.21.15.47:0/3788491004 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6bb410b780 0x7f6bb410db70 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f6ba0009900 tx=0x7f6ba002bc40 comp rx=0 tx=0).stop 2024-03-21T10:21:48.630 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:48.628+0000 7f6bbb200640 1 -- 172.21.15.47:0/3788491004 shutdown_connections 2024-03-21T10:21:48.630 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:48.628+0000 7f6bbb200640 1 --2- 172.21.15.47:0/3788491004 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6bb410e1e0 0x7f6bb4110680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:21:48.630 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:48.628+0000 7f6bbb200640 1 --2- 172.21.15.47:0/3788491004 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6bb410b780 0x7f6bb410db70 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:21:48.630 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:48.628+0000 7f6bbb200640 1 --2- 172.21.15.47:0/3788491004 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6bb40732f0 0x7f6bb410b1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:21:48.630 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:48.628+0000 7f6bbb200640 1 -- 172.21.15.47:0/3788491004 >> 172.21.15.47:0/3788491004 conn(0x7f6bb406e510 msgr2=0x7f6bb4070950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:21:48.630 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:48.628+0000 7f6bbb200640 1 -- 172.21.15.47:0/3788491004 shutdown_connections 2024-03-21T10:21:48.630 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:48.628+0000 7f6bbb200640 1 -- 172.21.15.47:0/3788491004 wait complete. 2024-03-21T10:21:48.631 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:48.630+0000 7f6bbb200640 1 Processor -- start 2024-03-21T10:21:48.631 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:48.630+0000 7f6bbb200640 1 -- start start 2024-03-21T10:21:48.631 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:48.630+0000 7f6bbb200640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6bb40732f0 0x7f6bb4113b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:21:48.632 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:48.631+0000 7f6bbb200640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6bb410b780 0x7f6bb41121f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:21:48.632 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:48.631+0000 7f6bbb200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6bb410e1e0 0x7f6bb4112730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:21:48.632 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:48.631+0000 7f6bbb200640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6bb4115a60 con 0x7f6bb40732f0 2024-03-21T10:21:48.632 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:48.631+0000 7f6bbb200640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6bb41158e0 con 0x7f6bb410e1e0 2024-03-21T10:21:48.632 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:48.631+0000 7f6bbb200640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6bb4115be0 con 0x7f6bb410b780 2024-03-21T10:21:48.632 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:48.632+0000 7f6bb3400640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6bb410b780 0x7f6bb41121f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:21:48.633 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:48.632+0000 7f6bb3400640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6bb410b780 0x7f6bb41121f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.47:50106/0 (socket says 172.21.15.47:50106) 2024-03-21T10:21:48.633 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:48.632+0000 7f6bb3400640 1 -- 172.21.15.47:0/2213768710 learned_addr learned my addr 172.21.15.47:0/2213768710 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:21:48.633 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:48.632+0000 7f6bb9200640 1 --2- 172.21.15.47:0/2213768710 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6bb410e1e0 0x7f6bb4112730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:21:48.633 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:48.632+0000 7f6bb3400640 1 -- 172.21.15.47:0/2213768710 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6bb410e1e0 msgr2=0x7f6bb4112730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:21:48.633 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:48.632+0000 7f6bb3400640 1 --2- 172.21.15.47:0/2213768710 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6bb410e1e0 0x7f6bb4112730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:21:48.633 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:48.632+0000 7f6bb3400640 1 -- 172.21.15.47:0/2213768710 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6bb40732f0 msgr2=0x7f6bb4113b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:21:48.633 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:48.632+0000 7f6bb3400640 1 --2- 172.21.15.47:0/2213768710 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6bb40732f0 0x7f6bb4113b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:21:48.633 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:48.632+0000 7f6bb3400640 1 -- 172.21.15.47:0/2213768710 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6bb4112ff0 con 0x7f6bb410b780 2024-03-21T10:21:48.634 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:48.633+0000 7f6bb3400640 1 --2- 172.21.15.47:0/2213768710 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6bb410b780 0x7f6bb41121f0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f6ba0009850 tx=0x7f6ba00042f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:21:48.634 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:48.634+0000 7f6bb0c00640 1 -- 172.21.15.47:0/2213768710 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ba0043020 con 0x7f6bb410b780 2024-03-21T10:21:48.639 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:48.634+0000 7f6bb0c00640 1 -- 172.21.15.47:0/2213768710 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f6ba00054c0 con 0x7f6bb410b780 2024-03-21T10:21:48.639 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:48.634+0000 7f6bb0c00640 1 -- 172.21.15.47:0/2213768710 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ba0003d80 con 0x7f6bb410b780 2024-03-21T10:21:48.639 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:48.634+0000 7f6bbb200640 1 -- 172.21.15.47:0/2213768710 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6bb41aa020 con 0x7f6bb410b780 2024-03-21T10:21:48.639 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:48.634+0000 7f6bbb200640 1 -- 172.21.15.47:0/2213768710 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6bb41aa4f0 con 0x7f6bb410b780 2024-03-21T10:21:48.639 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:48.636+0000 7f6bbb200640 1 -- 172.21.15.47:0/2213768710 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6bb40684c0 con 0x7f6bb410b780 2024-03-21T10:21:48.645 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:48.644+0000 7f6bb0c00640 1 -- 172.21.15.47:0/2213768710 <== mon.1 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 16) v1 ==== 137854+0+0 (secure 0 0 0) 0x7f6ba0046020 con 0x7f6bb410b780 2024-03-21T10:21:48.647 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:48.646+0000 7f6bb0c00640 1 --2- 172.21.15.47:0/2213768710 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f6b980a2fa0 0x7f6b980a5460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:21:48.647 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:48.646+0000 7f6bb0c00640 1 -- 172.21.15.47:0/2213768710 <== mon.1 v2:172.21.15.157:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f6ba00f1820 con 0x7f6bb410b780 2024-03-21T10:21:48.648 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:48.647+0000 7f6bb3e00640 1 --2- 172.21.15.47:0/2213768710 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f6b980a2fa0 0x7f6b980a5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:21:48.648 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:48.648+0000 7f6bb3e00640 1 --2- 172.21.15.47:0/2213768710 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f6b980a2fa0 0x7f6b980a5460 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f6ba4005fd0 tx=0x7f6ba400b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:21:48.653 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:48.653+0000 7f6bb0c00640 1 -- 172.21.15.47:0/2213768710 <== mon.1 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f6bb40684c0 con 0x7f6bb410b780 2024-03-21T10:21:48.905 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:48.904+0000 7f6bbb200640 1 -- 172.21.15.47:0/2213768710 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f6bb4071060 con 0x7f6bb410b780 2024-03-21T10:21:48.906 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:48.905+0000 7f6bb0c00640 1 -- 172.21.15.47:0/2213768710 <== mon.1 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 11 v11) v1 ==== 94+0+8465 (secure 0 0 0) 0x7f6bb4071060 con 0x7f6bb410b780 2024-03-21T10:21:48.907 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-21T10:21:48.907 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":11,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14511,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6835/735724037","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6834","nonce":735724037},{"type":"v1","addr":"172.21.15.47:6835","nonce":735724037}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14529,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6837/2248191724","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6836","nonce":2248191724},{"type":"v1","addr":"172.21.15.47:6837","nonce":2248191724}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14604,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6839/2881335356","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6838","nonce":2881335356},{"type":"v1","addr":"172.21.15.47:6839","nonce":2881335356}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":24406,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6833/4000010136","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":4000010136},{"type":"v1","addr":"172.21.15.157:6833","nonce":4000010136}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24457,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6835/3902868651","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":3902868651},{"type":"v1","addr":"172.21.15.157:6835","nonce":3902868651}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":7},{"gid":24475,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6837/1908087204","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6836","nonce":1908087204},{"type":"v1","addr":"172.21.15.157:6837","nonce":1908087204}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":8},{"gid":24499,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6839/4064242876","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6838","nonce":4064242876},{"type":"v1","addr":"172.21.15.157:6839","nonce":4064242876}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":10},{"gid":34427,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6833/2723302038","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6832","nonce":2723302038},{"type":"v1","addr":"172.21.15.100:6833","nonce":2723302038}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":34454,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6835/2597849957","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":2597849957},{"type":"v1","addr":"172.21.15.100:6835","nonce":2597849957}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6},{"gid":34493,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6837/3497644162","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6836","nonce":3497644162},{"type":"v1","addr":"172.21.15.100:6837","nonce":3497644162}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":9}],"filesystems":[]} 2024-03-21T10:21:48.907 INFO:teuthology.orchestra.run.smithi047.stderr:dumped fsmap epoch 11 2024-03-21T10:21:48.909 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:48.908+0000 7f6bbb200640 1 -- 172.21.15.47:0/2213768710 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f6b980a2fa0 msgr2=0x7f6b980a5460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:21:48.909 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:48.908+0000 7f6bbb200640 1 --2- 172.21.15.47:0/2213768710 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f6b980a2fa0 0x7f6b980a5460 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f6ba4005fd0 tx=0x7f6ba400b410 comp rx=0 tx=0).stop 2024-03-21T10:21:48.909 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:48.908+0000 7f6bbb200640 1 -- 172.21.15.47:0/2213768710 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6bb410b780 msgr2=0x7f6bb41121f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:21:48.909 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:48.908+0000 7f6bbb200640 1 --2- 172.21.15.47:0/2213768710 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6bb410b780 0x7f6bb41121f0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f6ba0009850 tx=0x7f6ba00042f0 comp rx=0 tx=0).stop 2024-03-21T10:21:48.909 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:48.908+0000 7f6bbb200640 1 -- 172.21.15.47:0/2213768710 shutdown_connections 2024-03-21T10:21:48.909 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:48.908+0000 7f6bbb200640 1 --2- 172.21.15.47:0/2213768710 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f6b980a2fa0 0x7f6b980a5460 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:21:48.910 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:48.908+0000 7f6bbb200640 1 --2- 172.21.15.47:0/2213768710 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6bb410e1e0 0x7f6bb4112730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:21:48.910 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:48.908+0000 7f6bbb200640 1 --2- 172.21.15.47:0/2213768710 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6bb410b780 0x7f6bb41121f0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:21:48.910 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:48.908+0000 7f6bbb200640 1 --2- 172.21.15.47:0/2213768710 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6bb40732f0 0x7f6bb4113b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:21:48.910 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:48.908+0000 7f6bbb200640 1 -- 172.21.15.47:0/2213768710 >> 172.21.15.47:0/2213768710 conn(0x7f6bb406e510 msgr2=0x7f6bb410d930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:21:48.910 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:48.908+0000 7f6bbb200640 1 -- 172.21.15.47:0/2213768710 shutdown_connections 2024-03-21T10:21:48.910 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:48.908+0000 7f6bbb200640 1 -- 172.21.15.47:0/2213768710 wait complete. 2024-03-21T10:21:49.513 INFO:journalctl@ceph.mds.a.smithi047.stdout:Mar 21 10:21:49 smithi047 podman[71171]: 2024-03-21 10:21:49.279219709 +0000 UTC m=+0.166684610 container create 4ecabcd2ea3456885d498ce3e1b521700a3d4ad37ce5f34b4ed1b814f05dc46c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-3056d51a-e76b-11ee-95cd-87774f69a715-mds-a, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, CEPH_POINT_RELEASE=, ceph=True, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.build-date=20240319, org.label-schema.license=GPLv2, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6) 2024-03-21T10:21:49.513 INFO:journalctl@ceph.mds.a.smithi047.stdout:Mar 21 10:21:49 smithi047 podman[71171]: 2024-03-21 10:21:49.510670977 +0000 UTC m=+0.398135872 container init 4ecabcd2ea3456885d498ce3e1b521700a3d4ad37ce5f34b4ed1b814f05dc46c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-3056d51a-e76b-11ee-95cd-87774f69a715-mds-a, org.label-schema.build-date=20240319, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_CLEAN=True) 2024-03-21T10:21:49.878 INFO:journalctl@ceph.mds.a.smithi047.stdout:Mar 21 10:21:49 smithi047 podman[71171]: 2024-03-21 10:21:49.521201631 +0000 UTC m=+0.408666502 container start 4ecabcd2ea3456885d498ce3e1b521700a3d4ad37ce5f34b4ed1b814f05dc46c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-3056d51a-e76b-11ee-95cd-87774f69a715-mds-a, ceph=True, org.label-schema.build-date=20240319, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True) 2024-03-21T10:21:49.878 INFO:journalctl@ceph.mds.a.smithi047.stdout:Mar 21 10:21:49 smithi047 ceph-mds[71189]: -- 172.21.15.47:0/3192368917 <== mon.0 v2:172.21.15.47:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x5629fd017a40 con 0x5629fcfeac00 2024-03-21T10:21:49.878 INFO:journalctl@ceph.mds.a.smithi047.stdout:Mar 21 10:21:49 smithi047 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-mds-a[71185]: starting mds.a at 2024-03-21T10:21:49.878 INFO:journalctl@ceph.mds.a.smithi047.stdout:Mar 21 10:21:49 smithi047 bash[71171]: 4ecabcd2ea3456885d498ce3e1b521700a3d4ad37ce5f34b4ed1b814f05dc46c 2024-03-21T10:21:49.878 INFO:journalctl@ceph.mds.a.smithi047.stdout:Mar 21 10:21:49 smithi047 systemd[1]: Started Ceph mds.a for 3056d51a-e76b-11ee-95cd-87774f69a715. 2024-03-21T10:21:50.214 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:21:49 smithi047 ceph-mon[31547]: pgmap v354: 1 pgs: 1 active+clean; 577 KiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-21T10:21:50.214 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:21:49 smithi047 ceph-mon[31547]: from='client.? 172.21.15.47:0/2213768710' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-21T10:21:50.214 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:21:49 smithi047 ceph-mon[31547]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:21:50.214 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:21:49 smithi047 ceph-mon[31547]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:21:50.215 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:21:49 smithi047 ceph-mon[31547]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:21:50.215 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:21:49 smithi047 ceph-mon[31547]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-21T10:21:50.215 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:21:49 smithi047 ceph-mon[31547]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-21T10:21:50.215 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:21:49 smithi047 ceph-mon[31547]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T10:21:50.230 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:21:49 smithi100 ceph-mon[35193]: pgmap v354: 1 pgs: 1 active+clean; 577 KiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-21T10:21:50.230 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:21:49 smithi100 ceph-mon[35193]: from='client.? 172.21.15.47:0/2213768710' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-21T10:21:50.230 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:21:49 smithi100 ceph-mon[35193]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:21:50.230 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:21:49 smithi100 ceph-mon[35193]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:21:50.230 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:21:49 smithi100 ceph-mon[35193]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:21:50.230 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:21:49 smithi100 ceph-mon[35193]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-21T10:21:50.230 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:21:49 smithi100 ceph-mon[35193]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-21T10:21:50.230 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:21:49 smithi100 ceph-mon[35193]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T10:21:50.240 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:21:49 smithi157 ceph-mon[35581]: pgmap v354: 1 pgs: 1 active+clean; 577 KiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-21T10:21:50.240 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:21:49 smithi157 ceph-mon[35581]: from='client.? 172.21.15.47:0/2213768710' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-21T10:21:50.240 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:21:49 smithi157 ceph-mon[35581]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:21:50.240 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:21:49 smithi157 ceph-mon[35581]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:21:50.240 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:21:49 smithi157 ceph-mon[35581]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:21:50.241 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:21:49 smithi157 ceph-mon[35581]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-21T10:21:50.241 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:21:49 smithi157 ceph-mon[35581]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-21T10:21:50.241 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:21:49 smithi157 ceph-mon[35581]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T10:21:50.928 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-21T10:21:50.980 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:21:50 smithi100 ceph-mon[35193]: Deploying daemon mds.b on smithi100 2024-03-21T10:21:50.980 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:21:50 smithi100 ceph-mon[35193]: mds.? [v2:172.21.15.47:6840/3401578978,v1:172.21.15.47:6841/3401578978] up:boot 2024-03-21T10:21:50.980 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:21:50 smithi100 ceph-mon[35193]: fsmap 11 up:standby 2024-03-21T10:21:50.980 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:21:50 smithi100 ceph-mon[35193]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-03-21T10:21:50.980 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:21:50 smithi100 ceph-mon[35193]: pgmap v355: 1 pgs: 1 active+clean; 577 KiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-21T10:21:51.163 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:51.161+0000 7f5ee7200640 1 -- 172.21.15.47:0/4160896251 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5ee8101500 msgr2=0x7f5ee8101920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:21:51.163 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:51.161+0000 7f5ee7200640 1 --2- 172.21.15.47:0/4160896251 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5ee8101500 0x7f5ee8101920 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f5ed8009900 tx=0x7f5ed802bc30 comp rx=0 tx=0).stop 2024-03-21T10:21:51.163 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:51.161+0000 7f5ee7200640 1 -- 172.21.15.47:0/4160896251 shutdown_connections 2024-03-21T10:21:51.163 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:51.161+0000 7f5ee7200640 1 --2- 172.21.15.47:0/4160896251 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5ee81030b0 0x7f5ee8109940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:21:51.164 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:51.161+0000 7f5ee7200640 1 --2- 172.21.15.47:0/4160896251 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5ee81026f0 0x7f5ee8102b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:21:51.164 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:51.161+0000 7f5ee7200640 1 --2- 172.21.15.47:0/4160896251 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5ee8101500 0x7f5ee8101920 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:21:51.164 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:51.161+0000 7f5ee7200640 1 -- 172.21.15.47:0/4160896251 >> 172.21.15.47:0/4160896251 conn(0x7f5ee80fca70 msgr2=0x7f5ee80feeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:21:51.164 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:51.161+0000 7f5ee7200640 1 -- 172.21.15.47:0/4160896251 shutdown_connections 2024-03-21T10:21:51.164 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:51.161+0000 7f5ee7200640 1 -- 172.21.15.47:0/4160896251 wait complete. 2024-03-21T10:21:51.166 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:51.162+0000 7f5ee7200640 1 Processor -- start 2024-03-21T10:21:51.167 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:51.162+0000 7f5ee7200640 1 -- start start 2024-03-21T10:21:51.167 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:51.162+0000 7f5ee7200640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5ee8101500 0x7f5ee8075450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:21:51.167 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:51.162+0000 7f5ee7200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5ee81026f0 0x7f5ee8073a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:21:51.167 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:51.162+0000 7f5ee7200640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5ee81030b0 0x7f5ee8073fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:21:51.167 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:51.162+0000 7f5ee7200640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ee810d090 con 0x7f5ee81030b0 2024-03-21T10:21:51.167 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:51.162+0000 7f5ee7200640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ee810cf10 con 0x7f5ee81026f0 2024-03-21T10:21:51.167 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:51.162+0000 7f5ee7200640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ee810d210 con 0x7f5ee8101500 2024-03-21T10:21:51.167 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:51.163+0000 7f5ee6800640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5ee81030b0 0x7f5ee8073fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:21:51.167 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:51.163+0000 7f5ee6800640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5ee81030b0 0x7f5ee8073fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:60370/0 (socket says 172.21.15.47:60370) 2024-03-21T10:21:51.167 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:51.163+0000 7f5ee6800640 1 -- 172.21.15.47:0/573970177 learned_addr learned my addr 172.21.15.47:0/573970177 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:21:51.167 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:51.163+0000 7f5edd400640 1 --2- 172.21.15.47:0/573970177 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5ee81026f0 0x7f5ee8073a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:21:51.167 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:51.163+0000 7f5ee5e00640 1 --2- 172.21.15.47:0/573970177 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5ee8101500 0x7f5ee8075450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:21:51.168 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:51.163+0000 7f5ee6800640 1 -- 172.21.15.47:0/573970177 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5ee8101500 msgr2=0x7f5ee8075450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:21:51.168 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:51.163+0000 7f5ee6800640 1 --2- 172.21.15.47:0/573970177 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5ee8101500 0x7f5ee8075450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:21:51.168 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:51.163+0000 7f5ee6800640 1 -- 172.21.15.47:0/573970177 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5ee81026f0 msgr2=0x7f5ee8073a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:21:51.168 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:51.163+0000 7f5ee6800640 1 --2- 172.21.15.47:0/573970177 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5ee81026f0 0x7f5ee8073a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:21:51.168 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:51.163+0000 7f5ee6800640 1 -- 172.21.15.47:0/573970177 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ee8074860 con 0x7f5ee81030b0 2024-03-21T10:21:51.168 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:51.163+0000 7f5edd400640 1 --2- 172.21.15.47:0/573970177 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5ee81026f0 0x7f5ee8073a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T10:21:51.168 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:51.164+0000 7f5ee6800640 1 --2- 172.21.15.47:0/573970177 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5ee81030b0 0x7f5ee8073fd0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f5ee000eaa0 tx=0x7f5ee000ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:21:51.168 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:51.164+0000 7f5edf400640 1 -- 172.21.15.47:0/573970177 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ee000cdb0 con 0x7f5ee81030b0 2024-03-21T10:21:51.168 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:51.164+0000 7f5edf400640 1 -- 172.21.15.47:0/573970177 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f5ee0005020 con 0x7f5ee81030b0 2024-03-21T10:21:51.168 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:51.164+0000 7f5ee7200640 1 -- 172.21.15.47:0/573970177 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5ee81a1890 con 0x7f5ee81030b0 2024-03-21T10:21:51.168 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:51.164+0000 7f5edf400640 1 -- 172.21.15.47:0/573970177 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ee00116d0 con 0x7f5ee81030b0 2024-03-21T10:21:51.168 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:51.164+0000 7f5ee7200640 1 -- 172.21.15.47:0/573970177 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5ee81a1de0 con 0x7f5ee81030b0 2024-03-21T10:21:51.169 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:51.165+0000 7f5ee7200640 1 -- 172.21.15.47:0/573970177 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5ee8108890 con 0x7f5ee81030b0 2024-03-21T10:21:51.169 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:51.167+0000 7f5edf400640 1 -- 172.21.15.47:0/573970177 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 16) v1 ==== 137854+0+0 (secure 0 0 0) 0x7f5ee00051d0 con 0x7f5ee81030b0 2024-03-21T10:21:51.170 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:51.168+0000 7f5edf400640 1 --2- 172.21.15.47:0/573970177 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f5ecc0a2fa0 0x7f5ecc0a5460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:21:51.170 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:51.169+0000 7f5edf400640 1 -- 172.21.15.47:0/573970177 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f5ee00cf250 con 0x7f5ee81030b0 2024-03-21T10:21:51.171 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:51.169+0000 7f5ee5e00640 1 --2- 172.21.15.47:0/573970177 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f5ecc0a2fa0 0x7f5ecc0a5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:21:51.171 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:51.169+0000 7f5ee5e00640 1 --2- 172.21.15.47:0/573970177 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f5ecc0a2fa0 0x7f5ecc0a5460 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f5ed802c240 tx=0x7f5ed8005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:21:51.175 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:51.173+0000 7f5edf400640 1 -- 172.21.15.47:0/573970177 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f5ee0016030 con 0x7f5ee81030b0 2024-03-21T10:21:51.214 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:21:50 smithi047 ceph-mon[31547]: Deploying daemon mds.b on smithi100 2024-03-21T10:21:51.214 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:21:50 smithi047 ceph-mon[31547]: mds.? [v2:172.21.15.47:6840/3401578978,v1:172.21.15.47:6841/3401578978] up:boot 2024-03-21T10:21:51.214 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:21:50 smithi047 ceph-mon[31547]: fsmap 11 up:standby 2024-03-21T10:21:51.214 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:21:50 smithi047 ceph-mon[31547]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-03-21T10:21:51.214 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:21:50 smithi047 ceph-mon[31547]: pgmap v355: 1 pgs: 1 active+clean; 577 KiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-21T10:21:51.240 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:21:50 smithi157 ceph-mon[35581]: Deploying daemon mds.b on smithi100 2024-03-21T10:21:51.240 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:21:50 smithi157 ceph-mon[35581]: mds.? [v2:172.21.15.47:6840/3401578978,v1:172.21.15.47:6841/3401578978] up:boot 2024-03-21T10:21:51.240 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:21:50 smithi157 ceph-mon[35581]: fsmap 11 up:standby 2024-03-21T10:21:51.240 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:21:50 smithi157 ceph-mon[35581]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-03-21T10:21:51.240 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:21:50 smithi157 ceph-mon[35581]: pgmap v355: 1 pgs: 1 active+clean; 577 KiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-21T10:21:51.424 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:51.422+0000 7f5ee7200640 1 -- 172.21.15.47:0/573970177 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f5ee80622a0 con 0x7f5ee81030b0 2024-03-21T10:21:51.426 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:51.424+0000 7f5edf400640 1 -- 172.21.15.47:0/573970177 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 12 v12) v1 ==== 94+0+9257 (secure 0 0 0) 0x7f5ee009c480 con 0x7f5ee81030b0 2024-03-21T10:21:51.426 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-21T10:21:51.427 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":12,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14511,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6835/735724037","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6834","nonce":735724037},{"type":"v1","addr":"172.21.15.47:6835","nonce":735724037}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14529,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6837/2248191724","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6836","nonce":2248191724},{"type":"v1","addr":"172.21.15.47:6837","nonce":2248191724}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14604,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6839/2881335356","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6838","nonce":2881335356},{"type":"v1","addr":"172.21.15.47:6839","nonce":2881335356}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":14619,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6841/3401578978","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6840","nonce":3401578978},{"type":"v1","addr":"172.21.15.47:6841","nonce":3401578978}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":24406,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6833/4000010136","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":4000010136},{"type":"v1","addr":"172.21.15.157:6833","nonce":4000010136}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24457,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6835/3902868651","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":3902868651},{"type":"v1","addr":"172.21.15.157:6835","nonce":3902868651}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":7},{"gid":24475,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6837/1908087204","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6836","nonce":1908087204},{"type":"v1","addr":"172.21.15.157:6837","nonce":1908087204}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":8},{"gid":24499,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6839/4064242876","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6838","nonce":4064242876},{"type":"v1","addr":"172.21.15.157:6839","nonce":4064242876}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":10},{"gid":34427,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6833/2723302038","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6832","nonce":2723302038},{"type":"v1","addr":"172.21.15.100:6833","nonce":2723302038}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":34454,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6835/2597849957","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":2597849957},{"type":"v1","addr":"172.21.15.100:6835","nonce":2597849957}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6},{"gid":34493,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6837/3497644162","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6836","nonce":3497644162},{"type":"v1","addr":"172.21.15.100:6837","nonce":3497644162}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":9}],"filesystems":[]} 2024-03-21T10:21:51.427 INFO:teuthology.orchestra.run.smithi047.stderr:dumped fsmap epoch 12 2024-03-21T10:21:51.429 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:51.427+0000 7f5ee7200640 1 -- 172.21.15.47:0/573970177 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f5ecc0a2fa0 msgr2=0x7f5ecc0a5460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:21:51.429 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:51.427+0000 7f5ee7200640 1 --2- 172.21.15.47:0/573970177 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f5ecc0a2fa0 0x7f5ecc0a5460 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f5ed802c240 tx=0x7f5ed8005fd0 comp rx=0 tx=0).stop 2024-03-21T10:21:51.429 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:51.427+0000 7f5ee7200640 1 -- 172.21.15.47:0/573970177 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5ee81030b0 msgr2=0x7f5ee8073fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:21:51.429 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:51.427+0000 7f5ee7200640 1 --2- 172.21.15.47:0/573970177 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5ee81030b0 0x7f5ee8073fd0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f5ee000eaa0 tx=0x7f5ee000ef60 comp rx=0 tx=0).stop 2024-03-21T10:21:51.429 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:51.427+0000 7f5ee7200640 1 -- 172.21.15.47:0/573970177 shutdown_connections 2024-03-21T10:21:51.429 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:51.427+0000 7f5ee7200640 1 --2- 172.21.15.47:0/573970177 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f5ecc0a2fa0 0x7f5ecc0a5460 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:21:51.430 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:51.427+0000 7f5ee7200640 1 --2- 172.21.15.47:0/573970177 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5ee81030b0 0x7f5ee8073fd0 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:21:51.430 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:51.427+0000 7f5ee7200640 1 --2- 172.21.15.47:0/573970177 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5ee81026f0 0x7f5ee8073a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:21:51.430 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:51.427+0000 7f5ee7200640 1 --2- 172.21.15.47:0/573970177 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5ee8101500 0x7f5ee8075450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:21:51.430 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:51.427+0000 7f5ee7200640 1 -- 172.21.15.47:0/573970177 >> 172.21.15.47:0/573970177 conn(0x7f5ee80fca70 msgr2=0x7f5ee8105a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:21:51.430 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:51.427+0000 7f5ee7200640 1 -- 172.21.15.47:0/573970177 shutdown_connections 2024-03-21T10:21:51.430 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:51.427+0000 7f5ee7200640 1 -- 172.21.15.47:0/573970177 wait complete. 2024-03-21T10:21:52.179 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:21:51 smithi100 ceph-mon[35193]: from='client.? 172.21.15.47:0/573970177' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-21T10:21:52.214 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:21:51 smithi047 ceph-mon[31547]: from='client.? 172.21.15.47:0/573970177' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-21T10:21:52.239 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:21:51 smithi157 ceph-mon[35581]: from='client.? 172.21.15.47:0/573970177' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-21T10:21:53.214 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:21:52 smithi047 ceph-mon[31547]: pgmap v356: 1 pgs: 1 active+clean; 577 KiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-21T10:21:53.230 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:21:52 smithi100 ceph-mon[35193]: pgmap v356: 1 pgs: 1 active+clean; 577 KiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-21T10:21:53.240 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:21:52 smithi157 ceph-mon[35581]: pgmap v356: 1 pgs: 1 active+clean; 577 KiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-21T10:21:53.450 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-21T10:21:53.697 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:53.691+0000 7fa623e00640 1 -- 172.21.15.47:0/1633450456 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa6241003f0 msgr2=0x7fa624100810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:21:53.697 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:53.691+0000 7fa623e00640 1 --2- 172.21.15.47:0/1633450456 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa6241003f0 0x7fa624100810 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fa614009900 tx=0x7fa61402bc30 comp rx=0 tx=0).stop 2024-03-21T10:21:53.697 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:53.691+0000 7fa623e00640 1 -- 172.21.15.47:0/1633450456 shutdown_connections 2024-03-21T10:21:53.697 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:53.691+0000 7fa623e00640 1 --2- 172.21.15.47:0/1633450456 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa624101fa0 0x7fa624108830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:21:53.697 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:53.691+0000 7fa623e00640 1 --2- 172.21.15.47:0/1633450456 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa6241015e0 0x7fa624101a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:21:53.698 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:53.691+0000 7fa623e00640 1 --2- 172.21.15.47:0/1633450456 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa6241003f0 0x7fa624100810 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:21:53.698 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:53.691+0000 7fa623e00640 1 -- 172.21.15.47:0/1633450456 >> 172.21.15.47:0/1633450456 conn(0x7fa6240fb980 msgr2=0x7fa6240fdda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:21:53.698 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:53.692+0000 7fa623e00640 1 -- 172.21.15.47:0/1633450456 shutdown_connections 2024-03-21T10:21:53.698 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:53.692+0000 7fa623e00640 1 -- 172.21.15.47:0/1633450456 wait complete. 2024-03-21T10:21:53.698 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:53.693+0000 7fa623e00640 1 Processor -- start 2024-03-21T10:21:53.698 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:53.693+0000 7fa623e00640 1 -- start start 2024-03-21T10:21:53.698 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:53.693+0000 7fa623e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa6241003f0 0x7fa624104450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:21:53.699 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:53.693+0000 7fa623e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa6241015e0 0x7fa624102a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:21:53.699 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:53.693+0000 7fa623e00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa624101fa0 0x7fa624102fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:21:53.699 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:53.693+0000 7fa623e00640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa62410bf50 con 0x7fa6241015e0 2024-03-21T10:21:53.699 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:53.693+0000 7fa623e00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa62410bdd0 con 0x7fa6241003f0 2024-03-21T10:21:53.699 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:53.693+0000 7fa623e00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa62410c0d0 con 0x7fa624101fa0 2024-03-21T10:21:53.699 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:53.694+0000 7fa622000640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa6241015e0 0x7fa624102a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:21:53.699 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:53.694+0000 7fa622a00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa6241003f0 0x7fa624104450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:21:53.700 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:53.694+0000 7fa622000640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa6241015e0 0x7fa624102a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:57734/0 (socket says 172.21.15.47:57734) 2024-03-21T10:21:53.700 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:53.694+0000 7fa622000640 1 -- 172.21.15.47:0/1206610973 learned_addr learned my addr 172.21.15.47:0/1206610973 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:21:53.700 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:53.694+0000 7fa623400640 1 --2- 172.21.15.47:0/1206610973 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa624101fa0 0x7fa624102fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:21:53.700 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:53.694+0000 7fa622000640 1 -- 172.21.15.47:0/1206610973 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa624101fa0 msgr2=0x7fa624102fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:21:53.700 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:53.694+0000 7fa622000640 1 --2- 172.21.15.47:0/1206610973 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa624101fa0 0x7fa624102fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:21:53.700 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:53.694+0000 7fa622000640 1 -- 172.21.15.47:0/1206610973 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa6241003f0 msgr2=0x7fa624104450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:21:53.700 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:53.694+0000 7fa622000640 1 --2- 172.21.15.47:0/1206610973 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa6241003f0 0x7fa624104450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:21:53.701 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:53.694+0000 7fa622000640 1 -- 172.21.15.47:0/1206610973 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa624103860 con 0x7fa6241015e0 2024-03-21T10:21:53.701 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:53.694+0000 7fa622000640 1 --2- 172.21.15.47:0/1206610973 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa6241015e0 0x7fa624102a90 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fa60c00cce0 tx=0x7fa60c00a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:21:53.701 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:53.695+0000 7fa61b400640 1 -- 172.21.15.47:0/1206610973 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa60c004b20 con 0x7fa6241015e0 2024-03-21T10:21:53.701 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:53.695+0000 7fa61b400640 1 -- 172.21.15.47:0/1206610973 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fa60c004cc0 con 0x7fa6241015e0 2024-03-21T10:21:53.701 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:53.695+0000 7fa61b400640 1 -- 172.21.15.47:0/1206610973 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa60c00f6b0 con 0x7fa6241015e0 2024-03-21T10:21:53.701 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:53.695+0000 7fa623e00640 1 -- 172.21.15.47:0/1206610973 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa6241a1840 con 0x7fa6241015e0 2024-03-21T10:21:53.701 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:53.695+0000 7fa623e00640 1 -- 172.21.15.47:0/1206610973 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa6241a1d40 con 0x7fa6241015e0 2024-03-21T10:21:53.702 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:53.696+0000 7fa623e00640 1 -- 172.21.15.47:0/1206610973 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa6241a1fa0 con 0x7fa6241015e0 2024-03-21T10:21:53.702 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:53.698+0000 7fa61b400640 1 -- 172.21.15.47:0/1206610973 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 16) v1 ==== 137854+0+0 (secure 0 0 0) 0x7fa60c00f850 con 0x7fa6241015e0 2024-03-21T10:21:53.703 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:53.699+0000 7fa61b400640 1 --2- 172.21.15.47:0/1206610973 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7fa6040a2f20 0x7fa6040a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:21:53.703 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:53.699+0000 7fa61b400640 1 -- 172.21.15.47:0/1206610973 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 7127+0+0 (secure 0 0 0) 0x7fa60c0cf0e0 con 0x7fa6241015e0 2024-03-21T10:21:53.703 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:53.700+0000 7fa622a00640 1 --2- 172.21.15.47:0/1206610973 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7fa6040a2f20 0x7fa6040a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:21:53.703 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:53.700+0000 7fa622a00640 1 --2- 172.21.15.47:0/1206610973 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7fa6040a2f20 0x7fa6040a53e0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fa61402c240 tx=0x7fa614000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:21:53.706 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:53.704+0000 7fa61b400640 1 -- 172.21.15.47:0/1206610973 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fa60c014030 con 0x7fa6241015e0 2024-03-21T10:21:53.961 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:53.959+0000 7fa623e00640 1 -- 172.21.15.47:0/1206610973 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa6241a2330 con 0x7fa6241015e0 2024-03-21T10:21:53.962 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:53.960+0000 7fa61b400640 1 -- 172.21.15.47:0/1206610973 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 13 v13) v1 ==== 94+0+10052 (secure 0 0 0) 0x7fa60c09c310 con 0x7fa6241015e0 2024-03-21T10:21:53.962 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-21T10:21:53.962 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":13,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14511,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6835/735724037","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6834","nonce":735724037},{"type":"v1","addr":"172.21.15.47:6835","nonce":735724037}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14529,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6837/2248191724","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6836","nonce":2248191724},{"type":"v1","addr":"172.21.15.47:6837","nonce":2248191724}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14604,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6839/2881335356","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6838","nonce":2881335356},{"type":"v1","addr":"172.21.15.47:6839","nonce":2881335356}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":14619,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6841/3401578978","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6840","nonce":3401578978},{"type":"v1","addr":"172.21.15.47:6841","nonce":3401578978}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":24406,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6833/4000010136","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":4000010136},{"type":"v1","addr":"172.21.15.157:6833","nonce":4000010136}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24457,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6835/3902868651","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":3902868651},{"type":"v1","addr":"172.21.15.157:6835","nonce":3902868651}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":7},{"gid":24475,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6837/1908087204","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6836","nonce":1908087204},{"type":"v1","addr":"172.21.15.157:6837","nonce":1908087204}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":8},{"gid":24499,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6839/4064242876","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6838","nonce":4064242876},{"type":"v1","addr":"172.21.15.157:6839","nonce":4064242876}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":10},{"gid":34427,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6833/2723302038","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6832","nonce":2723302038},{"type":"v1","addr":"172.21.15.100:6833","nonce":2723302038}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":34454,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6835/2597849957","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":2597849957},{"type":"v1","addr":"172.21.15.100:6835","nonce":2597849957}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6},{"gid":34493,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6837/3497644162","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6836","nonce":3497644162},{"type":"v1","addr":"172.21.15.100:6837","nonce":3497644162}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":9},{"gid":34535,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6839/4173168440","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6838","nonce":4173168440},{"type":"v1","addr":"172.21.15.100:6839","nonce":4173168440}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":13}],"filesystems":[]} 2024-03-21T10:21:53.963 INFO:teuthology.orchestra.run.smithi047.stderr:dumped fsmap epoch 13 2024-03-21T10:21:53.965 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:53.963+0000 7fa623e00640 1 -- 172.21.15.47:0/1206610973 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7fa6040a2f20 msgr2=0x7fa6040a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:21:53.965 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:53.963+0000 7fa623e00640 1 --2- 172.21.15.47:0/1206610973 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7fa6040a2f20 0x7fa6040a53e0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fa61402c240 tx=0x7fa614000bc0 comp rx=0 tx=0).stop 2024-03-21T10:21:53.965 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:53.963+0000 7fa623e00640 1 -- 172.21.15.47:0/1206610973 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa6241015e0 msgr2=0x7fa624102a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:21:53.965 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:53.963+0000 7fa623e00640 1 --2- 172.21.15.47:0/1206610973 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa6241015e0 0x7fa624102a90 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fa60c00cce0 tx=0x7fa60c00a620 comp rx=0 tx=0).stop 2024-03-21T10:21:53.966 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:53.963+0000 7fa623e00640 1 -- 172.21.15.47:0/1206610973 shutdown_connections 2024-03-21T10:21:53.966 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:53.963+0000 7fa623e00640 1 --2- 172.21.15.47:0/1206610973 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7fa6040a2f20 0x7fa6040a53e0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:21:53.966 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:53.963+0000 7fa623e00640 1 --2- 172.21.15.47:0/1206610973 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa624101fa0 0x7fa624102fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:21:53.966 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:53.963+0000 7fa623e00640 1 --2- 172.21.15.47:0/1206610973 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa6241015e0 0x7fa624102a90 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:21:53.966 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:53.963+0000 7fa623e00640 1 --2- 172.21.15.47:0/1206610973 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa6241003f0 0x7fa624104450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:21:53.966 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:53.963+0000 7fa623e00640 1 -- 172.21.15.47:0/1206610973 >> 172.21.15.47:0/1206610973 conn(0x7fa6240fb980 msgr2=0x7fa624068800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:21:53.966 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:53.963+0000 7fa623e00640 1 -- 172.21.15.47:0/1206610973 shutdown_connections 2024-03-21T10:21:53.966 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:53.963+0000 7fa623e00640 1 -- 172.21.15.47:0/1206610973 wait complete. 2024-03-21T10:21:53.979 INFO:journalctl@ceph.mds.b.smithi100.stdout:Mar 21 10:21:53 smithi100 systemd[1]: Started Ceph mds.b for 3056d51a-e76b-11ee-95cd-87774f69a715. 2024-03-21T10:21:53.986 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-21T10:21:54.207 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:54.204+0000 7f01b6200640 1 -- 172.21.15.47:0/3307214194 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f01b0105490 msgr2=0x7f01b010bd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:21:54.207 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:54.204+0000 7f01b6200640 1 --2- 172.21.15.47:0/3307214194 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f01b0105490 0x7f01b010bd20 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f01a8009900 tx=0x7f01a802bc30 comp rx=0 tx=0).stop 2024-03-21T10:21:54.207 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:54.204+0000 7f01b6200640 1 -- 172.21.15.47:0/3307214194 shutdown_connections 2024-03-21T10:21:54.207 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:54.204+0000 7f01b6200640 1 --2- 172.21.15.47:0/3307214194 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f01b0105490 0x7f01b010bd20 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:21:54.207 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:54.204+0000 7f01b6200640 1 --2- 172.21.15.47:0/3307214194 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f01b0104ad0 0x7f01b0104f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:21:54.207 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:54.204+0000 7f01b6200640 1 --2- 172.21.15.47:0/3307214194 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f01b01038e0 0x7f01b0103d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:21:54.207 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:54.204+0000 7f01b6200640 1 -- 172.21.15.47:0/3307214194 >> 172.21.15.47:0/3307214194 conn(0x7f01b00fee90 msgr2=0x7f01b01012b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:21:54.207 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:54.205+0000 7f01b6200640 1 -- 172.21.15.47:0/3307214194 shutdown_connections 2024-03-21T10:21:54.207 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:54.205+0000 7f01b6200640 1 -- 172.21.15.47:0/3307214194 wait complete. 2024-03-21T10:21:54.209 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:54.206+0000 7f01b6200640 1 Processor -- start 2024-03-21T10:21:54.210 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:54.206+0000 7f01b6200640 1 -- start start 2024-03-21T10:21:54.210 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:54.206+0000 7f01b6200640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f01b01038e0 0x7f01b0199aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:21:54.210 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:54.206+0000 7f01b6200640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f01b0104ad0 0x7f01b0199fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:21:54.210 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:54.206+0000 7f01b6200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f01b0105490 0x7f01b01a1050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:21:54.210 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:54.206+0000 7f01b6200640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01b010f460 con 0x7f01b01038e0 2024-03-21T10:21:54.210 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:54.206+0000 7f01b6200640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01b010f2e0 con 0x7f01b0105490 2024-03-21T10:21:54.210 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:54.206+0000 7f01b6200640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01b010f5e0 con 0x7f01b0104ad0 2024-03-21T10:21:54.210 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:54.206+0000 7f01af400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f01b01038e0 0x7f01b0199aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:21:54.210 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:54.206+0000 7f01af400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f01b01038e0 0x7f01b0199aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:57760/0 (socket says 172.21.15.47:57760) 2024-03-21T10:21:54.210 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:54.206+0000 7f01af400640 1 -- 172.21.15.47:0/2917444933 learned_addr learned my addr 172.21.15.47:0/2917444933 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:21:54.210 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:54.207+0000 7f01aea00640 1 --2- 172.21.15.47:0/2917444933 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f01b0104ad0 0x7f01b0199fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:21:54.210 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:54.207+0000 7f01af400640 1 -- 172.21.15.47:0/2917444933 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f01b0104ad0 msgr2=0x7f01b0199fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:21:54.211 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:54.207+0000 7f01afe00640 1 --2- 172.21.15.47:0/2917444933 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f01b0105490 0x7f01b01a1050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:21:54.211 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:54.207+0000 7f01af400640 1 --2- 172.21.15.47:0/2917444933 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f01b0104ad0 0x7f01b0199fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:21:54.211 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:54.207+0000 7f01af400640 1 -- 172.21.15.47:0/2917444933 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f01b0105490 msgr2=0x7f01b01a1050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:21:54.211 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:54.207+0000 7f01af400640 1 --2- 172.21.15.47:0/2917444933 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f01b0105490 0x7f01b01a1050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:21:54.211 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:54.207+0000 7f01af400640 1 -- 172.21.15.47:0/2917444933 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01b01a16c0 con 0x7f01b01038e0 2024-03-21T10:21:54.211 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:54.207+0000 7f01aea00640 1 --2- 172.21.15.47:0/2917444933 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f01b0104ad0 0x7f01b0199fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T10:21:54.211 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:54.207+0000 7f01af400640 1 --2- 172.21.15.47:0/2917444933 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f01b01038e0 0x7f01b0199aa0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f01a000eaa0 tx=0x7f01a000ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:21:54.211 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:54.207+0000 7f01a7e00640 1 -- 172.21.15.47:0/2917444933 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01a0016070 con 0x7f01b01038e0 2024-03-21T10:21:54.211 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:54.207+0000 7f01a7e00640 1 -- 172.21.15.47:0/2917444933 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f01a0003e20 con 0x7f01b01038e0 2024-03-21T10:21:54.211 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:54.207+0000 7f01a7e00640 1 -- 172.21.15.47:0/2917444933 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01a00116d0 con 0x7f01b01038e0 2024-03-21T10:21:54.211 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:54.208+0000 7f01b6200640 1 -- 172.21.15.47:0/2917444933 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f01b01a19b0 con 0x7f01b01038e0 2024-03-21T10:21:54.211 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:54.208+0000 7f01b6200640 1 -- 172.21.15.47:0/2917444933 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f01b01a1f00 con 0x7f01b01038e0 2024-03-21T10:21:54.212 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:54.209+0000 7f01b6200640 1 -- 172.21.15.47:0/2917444933 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f01b010ac00 con 0x7f01b01038e0 2024-03-21T10:21:54.213 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:54.211+0000 7f01a7e00640 1 -- 172.21.15.47:0/2917444933 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 16) v1 ==== 137854+0+0 (secure 0 0 0) 0x7f01a0019070 con 0x7f01b01038e0 2024-03-21T10:21:54.214 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:54.212+0000 7f01a7e00640 1 --2- 172.21.15.47:0/2917444933 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f01940a2fa0 0x7f01940a5460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:21:54.214 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:54.212+0000 7f01a7e00640 1 -- 172.21.15.47:0/2917444933 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f01a00cf520 con 0x7f01b01038e0 2024-03-21T10:21:54.214 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:54.213+0000 7f01aea00640 1 --2- 172.21.15.47:0/2917444933 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f01940a2fa0 0x7f01940a5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:21:54.215 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:54.213+0000 7f01aea00640 1 --2- 172.21.15.47:0/2917444933 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f01940a2fa0 0x7f01940a5460 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f01b019afb0 tx=0x7f0198005e50 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:21:54.217 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:54.215+0000 7f01a7e00640 1 -- 172.21.15.47:0/2917444933 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f01a009c7d0 con 0x7f01b01038e0 2024-03-21T10:21:54.443 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:54.441+0000 7f01b6200640 1 -- 172.21.15.47:0/2917444933 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f01b002d070 con 0x7f01b01038e0 2024-03-21T10:21:54.446 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:54.444+0000 7f01a7e00640 1 -- 172.21.15.47:0/2917444933 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v83) v1 ==== 74+0+18444 (secure 0 0 0) 0x7f01a009c0d0 con 0x7f01b01038e0 2024-03-21T10:21:54.447 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-21T10:21:54.447 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":83,"fsid":"3056d51a-e76b-11ee-95cd-87774f69a715","created":"2024-03-21T10:11:17.019965+0000","modified":"2024-03-21T10:20:53.966317+0000","last_up_change":"2024-03-21T10:20:52.675328+0000","last_in_change":"2024-03-21T10:20:27.648212+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-21T10:15:40.553114+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b0c7a102-8d89-428b-b793-cbbe67151f50","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6802","nonce":775837313},{"type":"v1","addr":"172.21.15.47:6803","nonce":775837313}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6804","nonce":775837313},{"type":"v1","addr":"172.21.15.47:6805","nonce":775837313}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6808","nonce":775837313},{"type":"v1","addr":"172.21.15.47:6809","nonce":775837313}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6806","nonce":775837313},{"type":"v1","addr":"172.21.15.47:6807","nonce":775837313}]},"public_addr":"172.21.15.47:6803/775837313","cluster_addr":"172.21.15.47:6805/775837313","heartbeat_back_addr":"172.21.15.47:6809/775837313","heartbeat_front_addr":"172.21.15.47:6807/775837313","state":["exists","up"]},{"osd":1,"uuid":"07bb81bb-7cf3-4bbc-ac9e-802f95185d11","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6800","nonce":2923304710},{"type":"v1","addr":"172.21.15.100:6801","nonce":2923304710}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":2923304710},{"type":"v1","addr":"172.21.15.100:6803","nonce":2923304710}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":2923304710},{"type":"v1","addr":"172.21.15.100:6807","nonce":2923304710}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6804","nonce":2923304710},{"type":"v1","addr":"172.21.15.100:6805","nonce":2923304710}]},"public_addr":"172.21.15.100:6801/2923304710","cluster_addr":"172.21.15.100:6803/2923304710","heartbeat_back_addr":"172.21.15.100:6807/2923304710","heartbeat_front_addr":"172.21.15.100:6805/2923304710","state":["exists","up"]},{"osd":2,"uuid":"88ce76b9-6081-4836-aad0-8c249c56e8a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":171377005},{"type":"v1","addr":"172.21.15.157:6801","nonce":171377005}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":171377005},{"type":"v1","addr":"172.21.15.157:6803","nonce":171377005}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":171377005},{"type":"v1","addr":"172.21.15.157:6807","nonce":171377005}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":171377005},{"type":"v1","addr":"172.21.15.157:6805","nonce":171377005}]},"public_addr":"172.21.15.157:6801/171377005","cluster_addr":"172.21.15.157:6803/171377005","heartbeat_back_addr":"172.21.15.157:6807/171377005","heartbeat_front_addr":"172.21.15.157:6805/171377005","state":["exists","up"]},{"osd":3,"uuid":"518aa33d-263f-45e6-aab5-2201ceb40e74","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":27,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6810","nonce":640760101},{"type":"v1","addr":"172.21.15.47:6811","nonce":640760101}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6812","nonce":640760101},{"type":"v1","addr":"172.21.15.47:6813","nonce":640760101}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6816","nonce":640760101},{"type":"v1","addr":"172.21.15.47:6817","nonce":640760101}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6814","nonce":640760101},{"type":"v1","addr":"172.21.15.47:6815","nonce":640760101}]},"public_addr":"172.21.15.47:6811/640760101","cluster_addr":"172.21.15.47:6813/640760101","heartbeat_back_addr":"172.21.15.47:6817/640760101","heartbeat_front_addr":"172.21.15.47:6815/640760101","state":["exists","up"]},{"osd":4,"uuid":"987bde83-ecc6-42e5-bd56-c143b3de4226","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":35,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6808","nonce":2785478639},{"type":"v1","addr":"172.21.15.100:6809","nonce":2785478639}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6810","nonce":2785478639},{"type":"v1","addr":"172.21.15.100:6811","nonce":2785478639}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6814","nonce":2785478639},{"type":"v1","addr":"172.21.15.100:6815","nonce":2785478639}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":2785478639},{"type":"v1","addr":"172.21.15.100:6813","nonce":2785478639}]},"public_addr":"172.21.15.100:6809/2785478639","cluster_addr":"172.21.15.100:6811/2785478639","heartbeat_back_addr":"172.21.15.100:6815/2785478639","heartbeat_front_addr":"172.21.15.100:6813/2785478639","state":["exists","up"]},{"osd":5,"uuid":"012d2188-3a61-4a5f-8d01-2001c15ffbe2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":40,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":1525645410},{"type":"v1","addr":"172.21.15.157:6809","nonce":1525645410}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":1525645410},{"type":"v1","addr":"172.21.15.157:6811","nonce":1525645410}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":1525645410},{"type":"v1","addr":"172.21.15.157:6815","nonce":1525645410}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":1525645410},{"type":"v1","addr":"172.21.15.157:6813","nonce":1525645410}]},"public_addr":"172.21.15.157:6809/1525645410","cluster_addr":"172.21.15.157:6811/1525645410","heartbeat_back_addr":"172.21.15.157:6815/1525645410","heartbeat_front_addr":"172.21.15.157:6813/1525645410","state":["exists","up"]},{"osd":6,"uuid":"c12c14b4-85f0-4d88-937c-de4f556437de","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6818","nonce":2321642790},{"type":"v1","addr":"172.21.15.47:6819","nonce":2321642790}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6820","nonce":2321642790},{"type":"v1","addr":"172.21.15.47:6821","nonce":2321642790}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6824","nonce":2321642790},{"type":"v1","addr":"172.21.15.47:6825","nonce":2321642790}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6822","nonce":2321642790},{"type":"v1","addr":"172.21.15.47:6823","nonce":2321642790}]},"public_addr":"172.21.15.47:6819/2321642790","cluster_addr":"172.21.15.47:6821/2321642790","heartbeat_back_addr":"172.21.15.47:6825/2321642790","heartbeat_front_addr":"172.21.15.47:6823/2321642790","state":["exists","up"]},{"osd":7,"uuid":"3c87c6d0-1d1f-4294-9917-aaaad7dda824","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6816","nonce":4104727330},{"type":"v1","addr":"172.21.15.100:6817","nonce":4104727330}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":4104727330},{"type":"v1","addr":"172.21.15.100:6819","nonce":4104727330}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":4104727330},{"type":"v1","addr":"172.21.15.100:6823","nonce":4104727330}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6820","nonce":4104727330},{"type":"v1","addr":"172.21.15.100:6821","nonce":4104727330}]},"public_addr":"172.21.15.100:6817/4104727330","cluster_addr":"172.21.15.100:6819/4104727330","heartbeat_back_addr":"172.21.15.100:6823/4104727330","heartbeat_front_addr":"172.21.15.100:6821/4104727330","state":["exists","up"]},{"osd":8,"uuid":"502d65cf-53c8-4936-8de4-e11b2a6e1e18","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":61,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":4067715347},{"type":"v1","addr":"172.21.15.157:6817","nonce":4067715347}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":4067715347},{"type":"v1","addr":"172.21.15.157:6819","nonce":4067715347}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":4067715347},{"type":"v1","addr":"172.21.15.157:6823","nonce":4067715347}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":4067715347},{"type":"v1","addr":"172.21.15.157:6821","nonce":4067715347}]},"public_addr":"172.21.15.157:6817/4067715347","cluster_addr":"172.21.15.157:6819/4067715347","heartbeat_back_addr":"172.21.15.157:6823/4067715347","heartbeat_front_addr":"172.21.15.157:6821/4067715347","state":["exists","up"]},{"osd":9,"uuid":"5ea1f5be-8458-4c93-9e61-c2e867a514f3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6826","nonce":1982108738},{"type":"v1","addr":"172.21.15.47:6827","nonce":1982108738}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6828","nonce":1982108738},{"type":"v1","addr":"172.21.15.47:6829","nonce":1982108738}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6832","nonce":1982108738},{"type":"v1","addr":"172.21.15.47:6833","nonce":1982108738}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6830","nonce":1982108738},{"type":"v1","addr":"172.21.15.47:6831","nonce":1982108738}]},"public_addr":"172.21.15.47:6827/1982108738","cluster_addr":"172.21.15.47:6829/1982108738","heartbeat_back_addr":"172.21.15.47:6833/1982108738","heartbeat_front_addr":"172.21.15.47:6831/1982108738","state":["exists","up"]},{"osd":10,"uuid":"2da3da90-a0be-4a54-a9ff-b2d6c9b36870","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":74,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":396391028},{"type":"v1","addr":"172.21.15.100:6825","nonce":396391028}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":396391028},{"type":"v1","addr":"172.21.15.100:6827","nonce":396391028}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6830","nonce":396391028},{"type":"v1","addr":"172.21.15.100:6831","nonce":396391028}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":396391028},{"type":"v1","addr":"172.21.15.100:6829","nonce":396391028}]},"public_addr":"172.21.15.100:6825/396391028","cluster_addr":"172.21.15.100:6827/396391028","heartbeat_back_addr":"172.21.15.100:6831/396391028","heartbeat_front_addr":"172.21.15.100:6829/396391028","state":["exists","up"]},{"osd":11,"uuid":"11798d41-7f7e-4319-95b8-8d3d0946bde9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":82,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":2671029483},{"type":"v1","addr":"172.21.15.157:6825","nonce":2671029483}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":2671029483},{"type":"v1","addr":"172.21.15.157:6827","nonce":2671029483}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":2671029483},{"type":"v1","addr":"172.21.15.157:6831","nonce":2671029483}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":2671029483},{"type":"v1","addr":"172.21.15.157:6829","nonce":2671029483}]},"public_addr":"172.21.15.157:6825/2671029483","cluster_addr":"172.21.15.157:6827/2671029483","heartbeat_back_addr":"172.21.15.157:6831/2671029483","heartbeat_front_addr":"172.21.15.157:6829/2671029483","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T10:14:31.822399+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T10:15:04.175721+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T10:15:35.902595+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T10:16:12.088891+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T10:16:46.428704+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T10:17:20.158270+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T10:17:55.915253+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T10:18:30.064641+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T10:19:04.410004+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T10:19:40.605181+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T10:20:15.636605+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T10:20:50.151365+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.47:0/356822431":"2024-03-22T10:12:25.844530+0000","172.21.15.47:0/1167490879":"2024-03-22T10:12:25.844530+0000","172.21.15.47:0/1540718138":"2024-03-22T10:12:25.844530+0000","172.21.15.47:6801/3836903972":"2024-03-22T10:12:25.844530+0000","172.21.15.47:0/235436602":"2024-03-22T10:11:50.301838+0000","172.21.15.47:0/2052815572":"2024-03-22T10:11:50.301838+0000","172.21.15.47:6800/3942658432":"2024-03-22T10:11:50.301838+0000","172.21.15.47:6801/3942658432":"2024-03-22T10:11:50.301838+0000","172.21.15.47:0/1419516257":"2024-03-22T10:11:50.301838+0000","172.21.15.47:6800/3836903972":"2024-03-22T10:12:25.844530+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-21T10:21:54.450 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:54.447+0000 7f01b6200640 1 -- 172.21.15.47:0/2917444933 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f01940a2fa0 msgr2=0x7f01940a5460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:21:54.450 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:54.447+0000 7f01b6200640 1 --2- 172.21.15.47:0/2917444933 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f01940a2fa0 0x7f01940a5460 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f01b019afb0 tx=0x7f0198005e50 comp rx=0 tx=0).stop 2024-03-21T10:21:54.450 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:54.447+0000 7f01b6200640 1 -- 172.21.15.47:0/2917444933 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f01b01038e0 msgr2=0x7f01b0199aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:21:54.450 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:54.447+0000 7f01b6200640 1 --2- 172.21.15.47:0/2917444933 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f01b01038e0 0x7f01b0199aa0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f01a000eaa0 tx=0x7f01a000ef60 comp rx=0 tx=0).stop 2024-03-21T10:21:54.450 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:54.447+0000 7f01b6200640 1 -- 172.21.15.47:0/2917444933 shutdown_connections 2024-03-21T10:21:54.450 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:54.447+0000 7f01b6200640 1 --2- 172.21.15.47:0/2917444933 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f01940a2fa0 0x7f01940a5460 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:21:54.450 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:54.447+0000 7f01b6200640 1 --2- 172.21.15.47:0/2917444933 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f01b0105490 0x7f01b01a1050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:21:54.450 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:54.447+0000 7f01b6200640 1 --2- 172.21.15.47:0/2917444933 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f01b0104ad0 0x7f01b0199fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:21:54.450 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:54.447+0000 7f01b6200640 1 --2- 172.21.15.47:0/2917444933 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f01b01038e0 0x7f01b0199aa0 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:21:54.450 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:54.447+0000 7f01b6200640 1 -- 172.21.15.47:0/2917444933 >> 172.21.15.47:0/2917444933 conn(0x7f01b00fee90 msgr2=0x7f01b0107dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:21:54.450 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:54.447+0000 7f01b6200640 1 -- 172.21.15.47:0/2917444933 shutdown_connections 2024-03-21T10:21:54.450 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:54.448+0000 7f01b6200640 1 -- 172.21.15.47:0/2917444933 wait complete. 2024-03-21T10:21:54.468 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-21T10:15:40.553114+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-21T10:21:54.468 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-21T10:21:54.692 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:54.690+0000 7fa807e00640 1 -- 172.21.15.47:0/2558929805 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa808104930 msgr2=0x7fa808106d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:21:54.692 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:54.690+0000 7fa807e00640 1 --2- 172.21.15.47:0/2558929805 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa808104930 0x7fa808106d20 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fa7f0009900 tx=0x7fa7f002bc30 comp rx=0 tx=0).stop 2024-03-21T10:21:54.692 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:54.690+0000 7fa807e00640 1 -- 172.21.15.47:0/2558929805 shutdown_connections 2024-03-21T10:21:54.692 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:54.690+0000 7fa807e00640 1 --2- 172.21.15.47:0/2558929805 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa808107260 0x7fa8081096b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:21:54.692 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:54.690+0000 7fa807e00640 1 --2- 172.21.15.47:0/2558929805 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa808104930 0x7fa808106d20 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:21:54.692 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:54.690+0000 7fa807e00640 1 --2- 172.21.15.47:0/2558929805 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa808102000 0x7fa8081043f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:21:54.692 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:54.690+0000 7fa807e00640 1 -- 172.21.15.47:0/2558929805 >> 172.21.15.47:0/2558929805 conn(0x7fa8080fba10 msgr2=0x7fa8080fde30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:21:54.693 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:54.690+0000 7fa807e00640 1 -- 172.21.15.47:0/2558929805 shutdown_connections 2024-03-21T10:21:54.693 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:54.690+0000 7fa807e00640 1 -- 172.21.15.47:0/2558929805 wait complete. 2024-03-21T10:21:54.695 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:54.691+0000 7fa807e00640 1 Processor -- start 2024-03-21T10:21:54.695 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:54.691+0000 7fa807e00640 1 -- start start 2024-03-21T10:21:54.695 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:54.691+0000 7fa807e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa808102000 0x7fa8081977d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:21:54.695 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:54.691+0000 7fa807e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa808104930 0x7fa808197d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:21:54.695 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:54.691+0000 7fa807e00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa808107260 0x7fa80819ed80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:21:54.695 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:54.691+0000 7fa807e00640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa80810d0c0 con 0x7fa808102000 2024-03-21T10:21:54.695 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:54.691+0000 7fa807e00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa80810cf40 con 0x7fa808104930 2024-03-21T10:21:54.695 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:54.691+0000 7fa807e00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa80810d240 con 0x7fa808107260 2024-03-21T10:21:54.695 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:54.692+0000 7fa806a00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa808102000 0x7fa8081977d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:21:54.695 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:54.692+0000 7fa806a00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa808102000 0x7fa8081977d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:57776/0 (socket says 172.21.15.47:57776) 2024-03-21T10:21:54.695 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:54.692+0000 7fa806a00640 1 -- 172.21.15.47:0/3108308029 learned_addr learned my addr 172.21.15.47:0/3108308029 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:21:54.696 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:54.692+0000 7fa7fe000640 1 --2- 172.21.15.47:0/3108308029 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa808104930 0x7fa808197d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:21:54.696 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:54.692+0000 7fa807400640 1 --2- 172.21.15.47:0/3108308029 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa808107260 0x7fa80819ed80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:21:54.696 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:54.692+0000 7fa806a00640 1 -- 172.21.15.47:0/3108308029 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa808107260 msgr2=0x7fa80819ed80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:21:54.696 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:54.692+0000 7fa806a00640 1 --2- 172.21.15.47:0/3108308029 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa808107260 0x7fa80819ed80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:21:54.696 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:54.692+0000 7fa806a00640 1 -- 172.21.15.47:0/3108308029 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa808104930 msgr2=0x7fa808197d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:21:54.696 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:54.692+0000 7fa806a00640 1 --2- 172.21.15.47:0/3108308029 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa808104930 0x7fa808197d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:21:54.696 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:54.692+0000 7fa806a00640 1 -- 172.21.15.47:0/3108308029 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa80819f3f0 con 0x7fa808102000 2024-03-21T10:21:54.696 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:54.692+0000 7fa806a00640 1 --2- 172.21.15.47:0/3108308029 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa808102000 0x7fa8081977d0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fa7f800ea80 tx=0x7fa7f800ef40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:21:54.696 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:54.693+0000 7fa7ffe00640 1 -- 172.21.15.47:0/3108308029 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7f800ce40 con 0x7fa808102000 2024-03-21T10:21:54.696 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:54.693+0000 7fa807e00640 1 -- 172.21.15.47:0/3108308029 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa80819f6e0 con 0x7fa808102000 2024-03-21T10:21:54.696 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:54.693+0000 7fa7ffe00640 1 -- 172.21.15.47:0/3108308029 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fa7f8005020 con 0x7fa808102000 2024-03-21T10:21:54.696 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:54.693+0000 7fa7ffe00640 1 -- 172.21.15.47:0/3108308029 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7f80116d0 con 0x7fa808102000 2024-03-21T10:21:54.696 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:54.693+0000 7fa807e00640 1 -- 172.21.15.47:0/3108308029 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa80819fbb0 con 0x7fa808102000 2024-03-21T10:21:54.697 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:54.694+0000 7fa807e00640 1 -- 172.21.15.47:0/3108308029 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa8080fd500 con 0x7fa808102000 2024-03-21T10:21:54.698 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:54.696+0000 7fa7ffe00640 1 -- 172.21.15.47:0/3108308029 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 16) v1 ==== 137854+0+0 (secure 0 0 0) 0x7fa7f80051d0 con 0x7fa808102000 2024-03-21T10:21:54.699 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:54.697+0000 7fa7ffe00640 1 --2- 172.21.15.47:0/3108308029 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7fa7ec0a2f20 0x7fa7ec0a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:21:54.699 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:54.697+0000 7fa7ffe00640 1 -- 172.21.15.47:0/3108308029 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 7127+0+0 (secure 0 0 0) 0x7fa7f80cf3b0 con 0x7fa808102000 2024-03-21T10:21:54.700 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:54.698+0000 7fa7fe000640 1 --2- 172.21.15.47:0/3108308029 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7fa7ec0a2f20 0x7fa7ec0a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:21:54.700 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:54.698+0000 7fa7fe000640 1 --2- 172.21.15.47:0/3108308029 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7fa7ec0a2f20 0x7fa7ec0a53e0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fa7f0006010 tx=0x7fa7f0005be0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:21:54.704 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:54.702+0000 7fa7ffe00640 1 -- 172.21.15.47:0/3108308029 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fa7f8016030 con 0x7fa808102000 2024-03-21T10:21:54.917 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:54.914+0000 7fa807e00640 1 -- 172.21.15.47:0/3108308029 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fa8080622a0 con 0x7fa808102000 2024-03-21T10:21:54.917 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:54.915+0000 7fa7ffe00640 1 -- 172.21.15.47:0/3108308029 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v83) v1 ==== 93+0+10 (secure 0 0 0) 0x7fa7f809c5e0 con 0x7fa808102000 2024-03-21T10:21:54.917 INFO:teuthology.orchestra.run.smithi047.stdout:pg_num: 1 2024-03-21T10:21:54.920 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:54.918+0000 7fa807e00640 1 -- 172.21.15.47:0/3108308029 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7fa7ec0a2f20 msgr2=0x7fa7ec0a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:21:54.920 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:54.918+0000 7fa807e00640 1 --2- 172.21.15.47:0/3108308029 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7fa7ec0a2f20 0x7fa7ec0a53e0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fa7f0006010 tx=0x7fa7f0005be0 comp rx=0 tx=0).stop 2024-03-21T10:21:54.920 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:54.918+0000 7fa807e00640 1 -- 172.21.15.47:0/3108308029 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa808102000 msgr2=0x7fa8081977d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:21:54.920 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:54.918+0000 7fa807e00640 1 --2- 172.21.15.47:0/3108308029 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa808102000 0x7fa8081977d0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fa7f800ea80 tx=0x7fa7f800ef40 comp rx=0 tx=0).stop 2024-03-21T10:21:54.920 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:54.918+0000 7fa807e00640 1 -- 172.21.15.47:0/3108308029 shutdown_connections 2024-03-21T10:21:54.921 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:54.918+0000 7fa807e00640 1 --2- 172.21.15.47:0/3108308029 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7fa7ec0a2f20 0x7fa7ec0a53e0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:21:54.921 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:54.918+0000 7fa807e00640 1 --2- 172.21.15.47:0/3108308029 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa808107260 0x7fa80819ed80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:21:54.921 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:54.918+0000 7fa807e00640 1 --2- 172.21.15.47:0/3108308029 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa808104930 0x7fa808197d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:21:54.921 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:54.918+0000 7fa807e00640 1 --2- 172.21.15.47:0/3108308029 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa808102000 0x7fa8081977d0 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:21:54.921 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:54.918+0000 7fa807e00640 1 -- 172.21.15.47:0/3108308029 >> 172.21.15.47:0/3108308029 conn(0x7fa8080fba10 msgr2=0x7fa808102ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:21:54.921 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:54.918+0000 7fa807e00640 1 -- 172.21.15.47:0/3108308029 shutdown_connections 2024-03-21T10:21:54.921 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:54.918+0000 7fa807e00640 1 -- 172.21.15.47:0/3108308029 wait complete. 2024-03-21T10:21:54.940 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-03-21T10:21:55.136 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:55.133+0000 7f48e9400640 1 -- 172.21.15.47:0/3310443503 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f48e41038e0 msgr2=0x7f48e4103d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:21:55.136 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:55.133+0000 7f48e9400640 1 --2- 172.21.15.47:0/3310443503 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f48e41038e0 0x7f48e4103d00 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f48d4009900 tx=0x7f48d402bc30 comp rx=0 tx=0).stop 2024-03-21T10:21:55.136 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:55.133+0000 7f48e9400640 1 -- 172.21.15.47:0/3310443503 shutdown_connections 2024-03-21T10:21:55.136 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:55.133+0000 7f48e9400640 1 --2- 172.21.15.47:0/3310443503 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f48e4105490 0x7f48e410bd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:21:55.136 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:55.133+0000 7f48e9400640 1 --2- 172.21.15.47:0/3310443503 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f48e4104ad0 0x7f48e4104f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:21:55.136 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:55.133+0000 7f48e9400640 1 --2- 172.21.15.47:0/3310443503 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f48e41038e0 0x7f48e4103d00 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:21:55.136 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:55.133+0000 7f48e9400640 1 -- 172.21.15.47:0/3310443503 >> 172.21.15.47:0/3310443503 conn(0x7f48e40fee50 msgr2=0x7f48e4101290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:21:55.136 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:55.133+0000 7f48e9400640 1 -- 172.21.15.47:0/3310443503 shutdown_connections 2024-03-21T10:21:55.136 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:55.133+0000 7f48e9400640 1 -- 172.21.15.47:0/3310443503 wait complete. 2024-03-21T10:21:55.137 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:55.135+0000 7f48e9400640 1 Processor -- start 2024-03-21T10:21:55.138 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:55.135+0000 7f48e9400640 1 -- start start 2024-03-21T10:21:55.138 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:55.135+0000 7f48e9400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f48e41038e0 0x7f48e4199b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:21:55.138 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:55.135+0000 7f48e9400640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f48e4104ad0 0x7f48e419a050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:21:55.138 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:55.135+0000 7f48e9400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f48e4105490 0x7f48e41a10c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:21:55.138 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:55.135+0000 7f48e9400640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48e410f3e0 con 0x7f48e41038e0 2024-03-21T10:21:55.138 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:55.135+0000 7f48e9400640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48e410f260 con 0x7f48e4105490 2024-03-21T10:21:55.138 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:55.135+0000 7f48e9400640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48e410f560 con 0x7f48e4104ad0 2024-03-21T10:21:55.138 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:55.136+0000 7f48e2800640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f48e41038e0 0x7f48e4199b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:21:55.138 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:55.136+0000 7f48e2800640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f48e41038e0 0x7f48e4199b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:57796/0 (socket says 172.21.15.47:57796) 2024-03-21T10:21:55.138 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:55.136+0000 7f48e2800640 1 -- 172.21.15.47:0/3014062434 learned_addr learned my addr 172.21.15.47:0/3014062434 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:21:55.138 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:55.136+0000 7f48dbe00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f48e4104ad0 0x7f48e419a050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:21:55.138 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:55.136+0000 7f48e3200640 1 --2- 172.21.15.47:0/3014062434 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f48e4105490 0x7f48e41a10c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:21:55.139 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:55.136+0000 7f48e2800640 1 -- 172.21.15.47:0/3014062434 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f48e4104ad0 msgr2=0x7f48e419a050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:21:55.140 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:55.136+0000 7f48e2800640 1 --2- 172.21.15.47:0/3014062434 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f48e4104ad0 0x7f48e419a050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:21:55.140 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:55.136+0000 7f48e2800640 1 -- 172.21.15.47:0/3014062434 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f48e4105490 msgr2=0x7f48e41a10c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:21:55.140 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:55.136+0000 7f48e2800640 1 --2- 172.21.15.47:0/3014062434 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f48e4105490 0x7f48e41a10c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:21:55.140 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:55.136+0000 7f48e2800640 1 -- 172.21.15.47:0/3014062434 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48e41a1730 con 0x7f48e41038e0 2024-03-21T10:21:55.140 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:55.137+0000 7f48e2800640 1 --2- 172.21.15.47:0/3014062434 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f48e41038e0 0x7f48e4199b10 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f48d4000c00 tx=0x7f48d4004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:21:55.140 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:55.137+0000 7f48db400640 1 -- 172.21.15.47:0/3014062434 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48d4043020 con 0x7f48e41038e0 2024-03-21T10:21:55.140 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:55.137+0000 7f48db400640 1 -- 172.21.15.47:0/3014062434 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f48d4004aa0 con 0x7f48e41038e0 2024-03-21T10:21:55.140 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:55.137+0000 7f48e9400640 1 -- 172.21.15.47:0/3014062434 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f48e41a19c0 con 0x7f48e41038e0 2024-03-21T10:21:55.140 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:55.137+0000 7f48db400640 1 -- 172.21.15.47:0/3014062434 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48d4003d00 con 0x7f48e41038e0 2024-03-21T10:21:55.140 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:55.137+0000 7f48e9400640 1 -- 172.21.15.47:0/3014062434 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f48e41a1f10 con 0x7f48e41038e0 2024-03-21T10:21:55.141 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:55.139+0000 7f48e9400640 1 -- 172.21.15.47:0/3014062434 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f48e40688e0 con 0x7f48e41038e0 2024-03-21T10:21:55.142 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:55.140+0000 7f48db400640 1 -- 172.21.15.47:0/3014062434 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 16) v1 ==== 137854+0+0 (secure 0 0 0) 0x7f48d4046020 con 0x7f48e41038e0 2024-03-21T10:21:55.143 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:55.142+0000 7f48db400640 1 --2- 172.21.15.47:0/3014062434 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f48c80a2f20 0x7f48c80a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:21:55.144 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:55.142+0000 7f48db400640 1 -- 172.21.15.47:0/3014062434 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f48d40f1500 con 0x7f48e41038e0 2024-03-21T10:21:55.144 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:55.142+0000 7f48dbe00640 1 --2- 172.21.15.47:0/3014062434 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f48c80a2f20 0x7f48c80a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:21:55.144 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:55.143+0000 7f48dbe00640 1 --2- 172.21.15.47:0/3014062434 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f48c80a2f20 0x7f48c80a53e0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f48e419b020 tx=0x7f48cc00b390 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:21:55.148 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:55.146+0000 7f48db400640 1 -- 172.21.15.47:0/3014062434 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f48d404d020 con 0x7f48e41038e0 2024-03-21T10:21:55.214 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:21:54 smithi047 ceph-mon[31547]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:21:55.214 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:21:54 smithi047 ceph-mon[31547]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:21:55.214 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:21:54 smithi047 ceph-mon[31547]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:21:55.214 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:21:54 smithi047 ceph-mon[31547]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:21:55.214 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:21:54 smithi047 ceph-mon[31547]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:21:55.214 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:21:54 smithi047 ceph-mon[31547]: mds.? [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] up:boot 2024-03-21T10:21:55.214 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:21:54 smithi047 ceph-mon[31547]: fsmap 12 up:standby 2024-03-21T10:21:55.214 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:21:54 smithi047 ceph-mon[31547]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-03-21T10:21:55.215 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:21:54 smithi047 ceph-mon[31547]: from='client.? 172.21.15.47:0/1206610973' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-21T10:21:55.215 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:21:54 smithi047 ceph-mon[31547]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T10:21:55.215 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:21:54 smithi047 ceph-mon[31547]: from='client.? 172.21.15.47:0/2917444933' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-21T10:21:55.229 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:21:54 smithi100 ceph-mon[35193]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:21:55.229 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:21:54 smithi100 ceph-mon[35193]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:21:55.229 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:21:54 smithi100 ceph-mon[35193]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:21:55.230 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:21:54 smithi100 ceph-mon[35193]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:21:55.230 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:21:54 smithi100 ceph-mon[35193]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:21:55.230 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:21:54 smithi100 ceph-mon[35193]: mds.? [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] up:boot 2024-03-21T10:21:55.230 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:21:54 smithi100 ceph-mon[35193]: fsmap 12 up:standby 2024-03-21T10:21:55.230 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:21:54 smithi100 ceph-mon[35193]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-03-21T10:21:55.230 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:21:54 smithi100 ceph-mon[35193]: from='client.? 172.21.15.47:0/1206610973' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-21T10:21:55.230 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:21:54 smithi100 ceph-mon[35193]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T10:21:55.230 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:21:54 smithi100 ceph-mon[35193]: from='client.? 172.21.15.47:0/2917444933' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-21T10:21:55.239 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:21:54 smithi157 ceph-mon[35581]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:21:55.240 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:21:54 smithi157 ceph-mon[35581]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:21:55.240 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:21:54 smithi157 ceph-mon[35581]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:21:55.240 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:21:54 smithi157 ceph-mon[35581]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:21:55.240 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:21:54 smithi157 ceph-mon[35581]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:21:55.240 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:21:54 smithi157 ceph-mon[35581]: mds.? [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] up:boot 2024-03-21T10:21:55.240 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:21:54 smithi157 ceph-mon[35581]: fsmap 12 up:standby 2024-03-21T10:21:55.240 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:21:54 smithi157 ceph-mon[35581]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-03-21T10:21:55.240 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:21:54 smithi157 ceph-mon[35581]: from='client.? 172.21.15.47:0/1206610973' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-21T10:21:55.240 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:21:54 smithi157 ceph-mon[35581]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T10:21:55.240 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:21:54 smithi157 ceph-mon[35581]: from='client.? 172.21.15.47:0/2917444933' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-21T10:21:55.359 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:55.357+0000 7f48e9400640 1 -- 172.21.15.47:0/3014062434 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f48e40622a0 con 0x7f48e41038e0 2024-03-21T10:21:55.360 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:55.358+0000 7f48db400640 1 -- 172.21.15.47:0/3014062434 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v83) v1 ==== 84+0+65 (secure 0 0 0) 0x7f48d40be730 con 0x7f48e41038e0 2024-03-21T10:21:55.360 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-21T10:21:55.360 INFO:teuthology.orchestra.run.smithi047.stdout:[ 2024-03-21T10:21:55.360 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-03-21T10:21:55.360 INFO:teuthology.orchestra.run.smithi047.stdout: "poolnum": 1, 2024-03-21T10:21:55.360 INFO:teuthology.orchestra.run.smithi047.stdout: "poolname": ".mgr" 2024-03-21T10:21:55.360 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-21T10:21:55.361 INFO:teuthology.orchestra.run.smithi047.stdout:] 2024-03-21T10:21:55.362 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:55.361+0000 7f48e9400640 1 -- 172.21.15.47:0/3014062434 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f48c80a2f20 msgr2=0x7f48c80a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:21:55.363 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:55.361+0000 7f48e9400640 1 --2- 172.21.15.47:0/3014062434 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f48c80a2f20 0x7f48c80a53e0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f48e419b020 tx=0x7f48cc00b390 comp rx=0 tx=0).stop 2024-03-21T10:21:55.363 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:55.361+0000 7f48e9400640 1 -- 172.21.15.47:0/3014062434 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f48e41038e0 msgr2=0x7f48e4199b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:21:55.363 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:55.361+0000 7f48e9400640 1 --2- 172.21.15.47:0/3014062434 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f48e41038e0 0x7f48e4199b10 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f48d4000c00 tx=0x7f48d4004270 comp rx=0 tx=0).stop 2024-03-21T10:21:55.363 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:55.361+0000 7f48e9400640 1 -- 172.21.15.47:0/3014062434 shutdown_connections 2024-03-21T10:21:55.363 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:55.361+0000 7f48e9400640 1 --2- 172.21.15.47:0/3014062434 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f48c80a2f20 0x7f48c80a53e0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:21:55.363 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:55.361+0000 7f48e9400640 1 --2- 172.21.15.47:0/3014062434 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f48e4105490 0x7f48e41a10c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:21:55.363 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:55.361+0000 7f48e9400640 1 --2- 172.21.15.47:0/3014062434 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f48e4104ad0 0x7f48e419a050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:21:55.363 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:55.361+0000 7f48e9400640 1 --2- 172.21.15.47:0/3014062434 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f48e41038e0 0x7f48e4199b10 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:21:55.363 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:55.361+0000 7f48e9400640 1 -- 172.21.15.47:0/3014062434 >> 172.21.15.47:0/3014062434 conn(0x7f48e40fee50 msgr2=0x7f48e4107e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:21:55.364 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:55.361+0000 7f48e9400640 1 -- 172.21.15.47:0/3014062434 shutdown_connections 2024-03-21T10:21:55.364 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:55.361+0000 7f48e9400640 1 -- 172.21.15.47:0/3014062434 wait complete. 2024-03-21T10:21:55.383 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-03-21T10:21:55.384 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-03-21T10:21:55.608 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:55.606+0000 7f35fee00640 1 -- 172.21.15.47:0/3470233176 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f35f8102130 msgr2=0x7f35f8104520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:21:55.608 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:55.606+0000 7f35fee00640 1 --2- 172.21.15.47:0/3470233176 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f35f8102130 0x7f35f8104520 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f35e8009900 tx=0x7f35e802bc30 comp rx=0 tx=0).stop 2024-03-21T10:21:55.608 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:55.606+0000 7f35fee00640 1 -- 172.21.15.47:0/3470233176 shutdown_connections 2024-03-21T10:21:55.608 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:55.606+0000 7f35fee00640 1 --2- 172.21.15.47:0/3470233176 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f35f8107390 0x7f35f81097e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:21:55.608 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:55.606+0000 7f35fee00640 1 --2- 172.21.15.47:0/3470233176 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f35f8104a60 0x7f35f8106e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:21:55.609 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:55.606+0000 7f35fee00640 1 --2- 172.21.15.47:0/3470233176 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f35f8102130 0x7f35f8104520 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:21:55.609 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:55.606+0000 7f35fee00640 1 -- 172.21.15.47:0/3470233176 >> 172.21.15.47:0/3470233176 conn(0x7f35f80fbb40 msgr2=0x7f35f80fdf60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:21:55.609 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:55.606+0000 7f35fee00640 1 -- 172.21.15.47:0/3470233176 shutdown_connections 2024-03-21T10:21:55.609 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:55.606+0000 7f35fee00640 1 -- 172.21.15.47:0/3470233176 wait complete. 2024-03-21T10:21:55.611 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:55.607+0000 7f35fee00640 1 Processor -- start 2024-03-21T10:21:55.611 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:55.607+0000 7f35fee00640 1 -- start start 2024-03-21T10:21:55.611 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:55.607+0000 7f35fee00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f35f8102130 0x7f35f8101a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:21:55.611 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:55.607+0000 7f35fee00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f35f8104a60 0x7f35f81000e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:21:55.611 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:55.607+0000 7f35fee00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f35f8107390 0x7f35f8100620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:21:55.611 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:55.607+0000 7f35fee00640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35f810d210 con 0x7f35f8104a60 2024-03-21T10:21:55.611 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:55.607+0000 7f35fee00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35f810d090 con 0x7f35f8102130 2024-03-21T10:21:55.611 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:55.607+0000 7f35fee00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35f810d390 con 0x7f35f8107390 2024-03-21T10:21:55.612 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:55.608+0000 7f35efe00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f35f8104a60 0x7f35f81000e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:21:55.612 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:55.608+0000 7f35efe00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f35f8104a60 0x7f35f81000e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:57802/0 (socket says 172.21.15.47:57802) 2024-03-21T10:21:55.612 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:55.608+0000 7f35efe00640 1 -- 172.21.15.47:0/2766052447 learned_addr learned my addr 172.21.15.47:0/2766052447 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:21:55.612 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:55.608+0000 7f35f7e00640 1 --2- 172.21.15.47:0/2766052447 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f35f8102130 0x7f35f8101a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:21:55.612 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:55.608+0000 7f35fce00640 1 --2- 172.21.15.47:0/2766052447 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f35f8107390 0x7f35f8100620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:21:55.612 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:55.608+0000 7f35efe00640 1 -- 172.21.15.47:0/2766052447 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f35f8107390 msgr2=0x7f35f8100620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:21:55.612 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:55.608+0000 7f35efe00640 1 --2- 172.21.15.47:0/2766052447 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f35f8107390 0x7f35f8100620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:21:55.612 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:55.608+0000 7f35efe00640 1 -- 172.21.15.47:0/2766052447 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f35f8102130 msgr2=0x7f35f8101a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:21:55.612 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:55.608+0000 7f35efe00640 1 --2- 172.21.15.47:0/2766052447 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f35f8102130 0x7f35f8101a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:21:55.612 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:55.608+0000 7f35efe00640 1 -- 172.21.15.47:0/2766052447 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35f8100ee0 con 0x7f35f8104a60 2024-03-21T10:21:55.612 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:55.608+0000 7f35f7e00640 1 --2- 172.21.15.47:0/2766052447 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f35f8102130 0x7f35f8101a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T10:21:55.612 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:55.609+0000 7f35efe00640 1 --2- 172.21.15.47:0/2766052447 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f35f8104a60 0x7f35f81000e0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f35e000d770 tx=0x7f35e000dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:21:55.612 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:55.609+0000 7f35f5600640 1 -- 172.21.15.47:0/2766052447 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35e0017070 con 0x7f35f8104a60 2024-03-21T10:21:55.612 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:55.609+0000 7f35f5600640 1 -- 172.21.15.47:0/2766052447 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f35e0009c30 con 0x7f35f8104a60 2024-03-21T10:21:55.613 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:55.609+0000 7f35f5600640 1 -- 172.21.15.47:0/2766052447 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35e000f620 con 0x7f35f8104a60 2024-03-21T10:21:55.613 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:55.609+0000 7f35fee00640 1 -- 172.21.15.47:0/2766052447 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f35f81a1ad0 con 0x7f35f8104a60 2024-03-21T10:21:55.613 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:55.609+0000 7f35fee00640 1 -- 172.21.15.47:0/2766052447 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f35f81a2040 con 0x7f35f8104a60 2024-03-21T10:21:55.613 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:55.610+0000 7f35fee00640 1 -- 172.21.15.47:0/2766052447 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f35f80fd630 con 0x7f35f8104a60 2024-03-21T10:21:55.614 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:55.612+0000 7f35f5600640 1 -- 172.21.15.47:0/2766052447 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 16) v1 ==== 137854+0+0 (secure 0 0 0) 0x7f35e001a070 con 0x7f35f8104a60 2024-03-21T10:21:55.615 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:55.613+0000 7f35f5600640 1 --2- 172.21.15.47:0/2766052447 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f35dc0a2fa0 0x7f35dc0a5460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:21:55.615 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:55.614+0000 7f35f5600640 1 -- 172.21.15.47:0/2766052447 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f35e00ceb30 con 0x7f35f8104a60 2024-03-21T10:21:55.616 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:55.614+0000 7f35f7e00640 1 --2- 172.21.15.47:0/2766052447 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f35dc0a2fa0 0x7f35dc0a5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:21:55.617 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:55.615+0000 7f35f7e00640 1 --2- 172.21.15.47:0/2766052447 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f35dc0a2fa0 0x7f35dc0a5460 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f35e8006010 tx=0x7f35e802c7e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:21:55.620 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:55.618+0000 7f35f5600640 1 -- 172.21.15.47:0/2766052447 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f35e0009de0 con 0x7f35f8104a60 2024-03-21T10:21:55.846 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:55.843+0000 7f35fee00640 1 -- 172.21.15.47:0/2766052447 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f35f80622a0 con 0x7f35f8104a60 2024-03-21T10:21:55.963 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:21:55 smithi047 ceph-mon[31547]: pgmap v357: 1 pgs: 1 active+clean; 577 KiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-21T10:21:55.963 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:21:55 smithi047 ceph-mon[31547]: from='client.? 172.21.15.47:0/3108308029' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-21T10:21:55.964 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:21:55 smithi047 ceph-mon[31547]: from='client.? 172.21.15.47:0/3014062434' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-03-21T10:21:56.229 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:21:55 smithi100 ceph-mon[35193]: pgmap v357: 1 pgs: 1 active+clean; 577 KiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-21T10:21:56.229 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:21:55 smithi100 ceph-mon[35193]: from='client.? 172.21.15.47:0/3108308029' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-21T10:21:56.229 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:21:55 smithi100 ceph-mon[35193]: from='client.? 172.21.15.47:0/3014062434' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-03-21T10:21:56.239 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:21:55 smithi157 ceph-mon[35581]: pgmap v357: 1 pgs: 1 active+clean; 577 KiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-21T10:21:56.240 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:21:55 smithi157 ceph-mon[35581]: from='client.? 172.21.15.47:0/3108308029' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-21T10:21:56.240 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:21:55 smithi157 ceph-mon[35581]: from='client.? 172.21.15.47:0/3014062434' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-03-21T10:21:56.797 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:56.794+0000 7f35f5600640 1 -- 172.21.15.47:0/2766052447 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v84) v1 ==== 138+0+0 (secure 0 0 0) 0x7f35e009bd60 con 0x7f35f8104a60 2024-03-21T10:21:56.797 INFO:teuthology.orchestra.run.smithi047.stderr:pool 'cephfs_metadata' created 2024-03-21T10:21:56.800 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:56.798+0000 7f35fee00640 1 -- 172.21.15.47:0/2766052447 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f35dc0a2fa0 msgr2=0x7f35dc0a5460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:21:56.800 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:56.798+0000 7f35fee00640 1 --2- 172.21.15.47:0/2766052447 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f35dc0a2fa0 0x7f35dc0a5460 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f35e8006010 tx=0x7f35e802c7e0 comp rx=0 tx=0).stop 2024-03-21T10:21:56.800 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:56.798+0000 7f35fee00640 1 -- 172.21.15.47:0/2766052447 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f35f8104a60 msgr2=0x7f35f81000e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:21:56.800 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:56.798+0000 7f35fee00640 1 --2- 172.21.15.47:0/2766052447 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f35f8104a60 0x7f35f81000e0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f35e000d770 tx=0x7f35e000dc30 comp rx=0 tx=0).stop 2024-03-21T10:21:56.801 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:56.798+0000 7f35fee00640 1 -- 172.21.15.47:0/2766052447 shutdown_connections 2024-03-21T10:21:56.801 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:56.798+0000 7f35fee00640 1 --2- 172.21.15.47:0/2766052447 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f35dc0a2fa0 0x7f35dc0a5460 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:21:56.801 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:56.798+0000 7f35fee00640 1 --2- 172.21.15.47:0/2766052447 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f35f8107390 0x7f35f8100620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:21:56.802 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:56.798+0000 7f35fee00640 1 --2- 172.21.15.47:0/2766052447 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f35f8104a60 0x7f35f81000e0 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:21:56.802 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:56.798+0000 7f35fee00640 1 --2- 172.21.15.47:0/2766052447 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f35f8102130 0x7f35f8101a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:21:56.802 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:56.798+0000 7f35fee00640 1 -- 172.21.15.47:0/2766052447 >> 172.21.15.47:0/2766052447 conn(0x7f35f80fbb40 msgr2=0x7f35f8102c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:21:56.802 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:56.799+0000 7f35fee00640 1 -- 172.21.15.47:0/2766052447 shutdown_connections 2024-03-21T10:21:56.802 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:56.799+0000 7f35fee00640 1 -- 172.21.15.47:0/2766052447 wait complete. 2024-03-21T10:21:56.831 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.05 2024-03-21T10:21:57.040 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:57.038+0000 7fc303c00640 1 -- 172.21.15.47:0/2533156953 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc2fc109550 msgr2=0x7fc2fc10b9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:21:57.040 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:57.038+0000 7fc303c00640 1 --2- 172.21.15.47:0/2533156953 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc2fc109550 0x7fc2fc10b9f0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fc2f4009ca0 tx=0x7fc2f402c190 comp rx=0 tx=0).stop 2024-03-21T10:21:57.041 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:57.038+0000 7fc303c00640 1 -- 172.21.15.47:0/2533156953 shutdown_connections 2024-03-21T10:21:57.041 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:57.038+0000 7fc303c00640 1 --2- 172.21.15.47:0/2533156953 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc2fc109550 0x7fc2fc10b9f0 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:21:57.041 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:57.038+0000 7fc303c00640 1 --2- 172.21.15.47:0/2533156953 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc2fc0740a0 0x7fc2fc074500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:21:57.041 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:57.038+0000 7fc303c00640 1 --2- 172.21.15.47:0/2533156953 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc2fc075680 0x7fc2fc073ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:21:57.041 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:57.038+0000 7fc303c00640 1 -- 172.21.15.47:0/2533156953 >> 172.21.15.47:0/2533156953 conn(0x7fc2fc0fcc50 msgr2=0x7fc2fc0ff070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:21:57.041 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:57.038+0000 7fc303c00640 1 -- 172.21.15.47:0/2533156953 shutdown_connections 2024-03-21T10:21:57.041 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:57.038+0000 7fc303c00640 1 -- 172.21.15.47:0/2533156953 wait complete. 2024-03-21T10:21:57.043 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:57.039+0000 7fc303c00640 1 Processor -- start 2024-03-21T10:21:57.043 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:57.040+0000 7fc303c00640 1 -- start start 2024-03-21T10:21:57.043 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:57.040+0000 7fc303c00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc2fc0740a0 0x7fc2fc199a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:21:57.043 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:57.040+0000 7fc303c00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc2fc075680 0x7fc2fc199f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:21:57.043 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:57.040+0000 7fc303c00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc2fc109550 0x7fc2fc1a0fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:21:57.043 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:57.040+0000 7fc303c00640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2fc10f3d0 con 0x7fc2fc109550 2024-03-21T10:21:57.043 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:57.040+0000 7fc303c00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2fc10f250 con 0x7fc2fc075680 2024-03-21T10:21:57.043 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:57.040+0000 7fc303c00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2fc10f550 con 0x7fc2fc0740a0 2024-03-21T10:21:57.043 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:57.040+0000 7fc301c00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc2fc109550 0x7fc2fc1a0fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:21:57.044 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:57.040+0000 7fc301c00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc2fc109550 0x7fc2fc1a0fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:57828/0 (socket says 172.21.15.47:57828) 2024-03-21T10:21:57.044 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:57.040+0000 7fc301c00640 1 -- 172.21.15.47:0/4124883544 learned_addr learned my addr 172.21.15.47:0/4124883544 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:21:57.044 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:57.041+0000 7fc2fbe00640 1 --2- 172.21.15.47:0/4124883544 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc2fc075680 0x7fc2fc199f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:21:57.044 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:57.041+0000 7fc301200640 1 --2- 172.21.15.47:0/4124883544 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc2fc0740a0 0x7fc2fc199a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:21:57.044 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:57.041+0000 7fc301c00640 1 -- 172.21.15.47:0/4124883544 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc2fc0740a0 msgr2=0x7fc2fc199a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:21:57.044 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:57.041+0000 7fc301c00640 1 --2- 172.21.15.47:0/4124883544 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc2fc0740a0 0x7fc2fc199a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:21:57.044 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:57.041+0000 7fc301c00640 1 -- 172.21.15.47:0/4124883544 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc2fc075680 msgr2=0x7fc2fc199f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:21:57.044 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:57.041+0000 7fc301c00640 1 --2- 172.21.15.47:0/4124883544 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc2fc075680 0x7fc2fc199f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:21:57.044 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:57.041+0000 7fc301c00640 1 -- 172.21.15.47:0/4124883544 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc2fc1a1650 con 0x7fc2fc109550 2024-03-21T10:21:57.045 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:57.041+0000 7fc301c00640 1 --2- 172.21.15.47:0/4124883544 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc2fc109550 0x7fc2fc1a0fe0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fc2f402c6d0 tx=0x7fc2f4003ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:21:57.045 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:57.042+0000 7fc2f9600640 1 -- 172.21.15.47:0/4124883544 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2f4039040 con 0x7fc2fc109550 2024-03-21T10:21:57.045 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:57.042+0000 7fc2f9600640 1 -- 172.21.15.47:0/4124883544 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fc2f4044070 con 0x7fc2fc109550 2024-03-21T10:21:57.046 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:57.042+0000 7fc303c00640 1 -- 172.21.15.47:0/4124883544 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc2fc1a18e0 con 0x7fc2fc109550 2024-03-21T10:21:57.046 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:57.042+0000 7fc2f9600640 1 -- 172.21.15.47:0/4124883544 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2f4004330 con 0x7fc2fc109550 2024-03-21T10:21:57.046 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:57.042+0000 7fc303c00640 1 -- 172.21.15.47:0/4124883544 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc2fc1a1dd0 con 0x7fc2fc109550 2024-03-21T10:21:57.046 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:57.043+0000 7fc303c00640 1 -- 172.21.15.47:0/4124883544 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc2fc108310 con 0x7fc2fc109550 2024-03-21T10:21:57.047 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:57.045+0000 7fc2f9600640 1 -- 172.21.15.47:0/4124883544 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 16) v1 ==== 137854+0+0 (secure 0 0 0) 0x7fc2f40044d0 con 0x7fc2fc109550 2024-03-21T10:21:57.049 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:57.047+0000 7fc2f9600640 1 --2- 172.21.15.47:0/4124883544 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7fc2e40a3070 0x7fc2e40a5530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:21:57.049 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:57.047+0000 7fc2f9600640 1 -- 172.21.15.47:0/4124883544 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 7497+0+0 (secure 0 0 0) 0x7fc2f40f1790 con 0x7fc2fc109550 2024-03-21T10:21:57.049 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:57.047+0000 7fc301200640 1 --2- 172.21.15.47:0/4124883544 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7fc2e40a3070 0x7fc2e40a5530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:21:57.050 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:57.048+0000 7fc301200640 1 --2- 172.21.15.47:0/4124883544 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7fc2e40a3070 0x7fc2e40a5530 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fc2f0005fd0 tx=0x7fc2f0005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:21:57.056 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:57.053+0000 7fc2f9600640 1 -- 172.21.15.47:0/4124883544 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fc2f40be850 con 0x7fc2fc109550 2024-03-21T10:21:57.214 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:21:56 smithi047 ceph-mon[31547]: from='client.? 172.21.15.47:0/2766052447' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-03-21T10:21:57.214 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:21:56 smithi047 ceph-mon[31547]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:21:57.229 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:21:56 smithi100 ceph-mon[35193]: from='client.? 172.21.15.47:0/2766052447' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-03-21T10:21:57.229 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:21:56 smithi100 ceph-mon[35193]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:21:57.239 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:21:56 smithi157 ceph-mon[35581]: from='client.? 172.21.15.47:0/2766052447' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-03-21T10:21:57.239 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:21:56 smithi157 ceph-mon[35581]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:21:57.270 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:57.268+0000 7fc303c00640 1 -- 172.21.15.47:0/4124883544 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05} v 0) v1 -- 0x7fc2fc02d070 con 0x7fc2fc109550 2024-03-21T10:21:57.828 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:57.826+0000 7fc2f9600640 1 -- 172.21.15.47:0/4124883544 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]=0 pool 'cephfs_data' created v85) v1 ==== 171+0+0 (secure 0 0 0) 0x7fc2f40be150 con 0x7fc2fc109550 2024-03-21T10:21:57.829 INFO:teuthology.orchestra.run.smithi047.stderr:pool 'cephfs_data' created 2024-03-21T10:21:57.833 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:57.830+0000 7fc303c00640 1 -- 172.21.15.47:0/4124883544 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7fc2e40a3070 msgr2=0x7fc2e40a5530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:21:57.833 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:57.830+0000 7fc303c00640 1 --2- 172.21.15.47:0/4124883544 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7fc2e40a3070 0x7fc2e40a5530 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fc2f0005fd0 tx=0x7fc2f0005850 comp rx=0 tx=0).stop 2024-03-21T10:21:57.833 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:57.830+0000 7fc303c00640 1 -- 172.21.15.47:0/4124883544 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc2fc109550 msgr2=0x7fc2fc1a0fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:21:57.833 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:57.830+0000 7fc303c00640 1 --2- 172.21.15.47:0/4124883544 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc2fc109550 0x7fc2fc1a0fe0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fc2f402c6d0 tx=0x7fc2f4003ba0 comp rx=0 tx=0).stop 2024-03-21T10:21:57.833 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:57.830+0000 7fc303c00640 1 -- 172.21.15.47:0/4124883544 shutdown_connections 2024-03-21T10:21:57.834 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:57.830+0000 7fc303c00640 1 --2- 172.21.15.47:0/4124883544 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7fc2e40a3070 0x7fc2e40a5530 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:21:57.834 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:57.830+0000 7fc303c00640 1 --2- 172.21.15.47:0/4124883544 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc2fc109550 0x7fc2fc1a0fe0 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:21:57.834 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:57.830+0000 7fc303c00640 1 --2- 172.21.15.47:0/4124883544 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc2fc075680 0x7fc2fc199f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:21:57.834 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:57.830+0000 7fc303c00640 1 --2- 172.21.15.47:0/4124883544 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc2fc0740a0 0x7fc2fc199a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:21:57.834 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:57.830+0000 7fc303c00640 1 -- 172.21.15.47:0/4124883544 >> 172.21.15.47:0/4124883544 conn(0x7fc2fc0fcc50 msgr2=0x7fc2fc1054e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:21:57.834 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:57.831+0000 7fc303c00640 1 -- 172.21.15.47:0/4124883544 shutdown_connections 2024-03-21T10:21:57.834 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:57.831+0000 7fc303c00640 1 -- 172.21.15.47:0/4124883544 wait complete. 2024-03-21T10:21:57.876 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-03-21T10:21:58.060 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:58.057+0000 7f697ce00640 1 -- 172.21.15.47:0/2942511436 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f69781038e0 msgr2=0x7f6978103d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:21:58.060 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:58.057+0000 7f697ce00640 1 --2- 172.21.15.47:0/2942511436 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f69781038e0 0x7f6978103d00 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f6968009900 tx=0x7f696802bc30 comp rx=0 tx=0).stop 2024-03-21T10:21:58.060 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:58.057+0000 7f697ce00640 1 -- 172.21.15.47:0/2942511436 shutdown_connections 2024-03-21T10:21:58.060 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:58.057+0000 7f697ce00640 1 --2- 172.21.15.47:0/2942511436 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6978105490 0x7f697810bd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:21:58.060 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:58.057+0000 7f697ce00640 1 --2- 172.21.15.47:0/2942511436 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6978104ad0 0x7f6978104f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:21:58.060 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:58.057+0000 7f697ce00640 1 --2- 172.21.15.47:0/2942511436 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f69781038e0 0x7f6978103d00 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:21:58.060 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:58.058+0000 7f697ce00640 1 -- 172.21.15.47:0/2942511436 >> 172.21.15.47:0/2942511436 conn(0x7f69780fee50 msgr2=0x7f6978101290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:21:58.060 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:58.058+0000 7f697ce00640 1 -- 172.21.15.47:0/2942511436 shutdown_connections 2024-03-21T10:21:58.060 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:58.058+0000 7f697ce00640 1 -- 172.21.15.47:0/2942511436 wait complete. 2024-03-21T10:21:58.061 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:58.059+0000 7f697ce00640 1 Processor -- start 2024-03-21T10:21:58.061 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:58.059+0000 7f697ce00640 1 -- start start 2024-03-21T10:21:58.063 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:58.060+0000 7f697ce00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f69781038e0 0x7f6978199b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:21:58.063 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:58.060+0000 7f697ce00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6978104ad0 0x7f697819a050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:21:58.063 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:58.060+0000 7f697ce00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6978105490 0x7f69781a10c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:21:58.063 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:58.060+0000 7f697ce00640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f697810f410 con 0x7f69781038e0 2024-03-21T10:21:58.063 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:58.060+0000 7f697ce00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f697810f290 con 0x7f6978104ad0 2024-03-21T10:21:58.064 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:58.060+0000 7f697ce00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f697810f590 con 0x7f6978105490 2024-03-21T10:21:58.064 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:58.060+0000 7f6975e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f69781038e0 0x7f6978199b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:21:58.064 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:58.060+0000 7f6975e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f69781038e0 0x7f6978199b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:57844/0 (socket says 172.21.15.47:57844) 2024-03-21T10:21:58.064 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:58.060+0000 7f6975e00640 1 -- 172.21.15.47:0/1275790043 learned_addr learned my addr 172.21.15.47:0/1275790043 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:21:58.064 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:58.061+0000 7f6975400640 1 --2- 172.21.15.47:0/1275790043 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6978104ad0 0x7f697819a050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:21:58.064 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:58.061+0000 7f6976800640 1 --2- 172.21.15.47:0/1275790043 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6978105490 0x7f69781a10c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:21:58.064 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:58.061+0000 7f6975e00640 1 -- 172.21.15.47:0/1275790043 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6978105490 msgr2=0x7f69781a10c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:21:58.064 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:58.061+0000 7f6975e00640 1 --2- 172.21.15.47:0/1275790043 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6978105490 0x7f69781a10c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:21:58.064 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:58.061+0000 7f6975e00640 1 -- 172.21.15.47:0/1275790043 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6978104ad0 msgr2=0x7f697819a050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:21:58.064 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:58.061+0000 7f6975e00640 1 --2- 172.21.15.47:0/1275790043 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6978104ad0 0x7f697819a050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:21:58.064 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:58.061+0000 7f6975e00640 1 -- 172.21.15.47:0/1275790043 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f69781a1730 con 0x7f69781038e0 2024-03-21T10:21:58.064 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:58.061+0000 7f6975e00640 1 --2- 172.21.15.47:0/1275790043 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f69781038e0 0x7f6978199b10 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f6968000c00 tx=0x7f6968004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:21:58.065 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:58.062+0000 7f696ea00640 1 -- 172.21.15.47:0/1275790043 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6968043020 con 0x7f69781038e0 2024-03-21T10:21:58.065 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:58.062+0000 7f697ce00640 1 -- 172.21.15.47:0/1275790043 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f69781a19c0 con 0x7f69781038e0 2024-03-21T10:21:58.065 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:58.062+0000 7f696ea00640 1 -- 172.21.15.47:0/1275790043 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f6968004aa0 con 0x7f69781038e0 2024-03-21T10:21:58.065 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:58.062+0000 7f696ea00640 1 -- 172.21.15.47:0/1275790043 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6968003d00 con 0x7f69781038e0 2024-03-21T10:21:58.065 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:58.062+0000 7f697ce00640 1 -- 172.21.15.47:0/1275790043 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f69781a1f10 con 0x7f69781038e0 2024-03-21T10:21:58.066 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:58.064+0000 7f697ce00640 1 -- 172.21.15.47:0/1275790043 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f697810ac80 con 0x7f69781038e0 2024-03-21T10:21:58.068 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:58.066+0000 7f696ea00640 1 -- 172.21.15.47:0/1275790043 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 16) v1 ==== 137854+0+0 (secure 0 0 0) 0x7f6968046020 con 0x7f69781038e0 2024-03-21T10:21:58.069 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:58.067+0000 7f696ea00640 1 --2- 172.21.15.47:0/1275790043 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f695c0a2f20 0x7f695c0a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:21:58.069 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:58.067+0000 7f696ea00640 1 -- 172.21.15.47:0/1275790043 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 7879+0+0 (secure 0 0 0) 0x7f69680f1be0 con 0x7f69781038e0 2024-03-21T10:21:58.069 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:58.067+0000 7f6975400640 1 --2- 172.21.15.47:0/1275790043 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f695c0a2f20 0x7f695c0a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:21:58.070 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:58.068+0000 7f6975400640 1 --2- 172.21.15.47:0/1275790043 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f695c0a2f20 0x7f695c0a53e0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f697819b020 tx=0x7f696400b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:21:58.076 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:58.074+0000 7f696ea00640 1 -- 172.21.15.47:0/1275790043 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f696804d020 con 0x7f69781038e0 2024-03-21T10:21:58.214 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:21:57 smithi047 ceph-mon[31547]: pgmap v358: 1 pgs: 1 active+clean; 577 KiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-21T10:21:58.214 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:21:57 smithi047 ceph-mon[31547]: from='client.? 172.21.15.47:0/2766052447' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-03-21T10:21:58.214 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:21:57 smithi047 ceph-mon[31547]: osdmap e84: 12 total, 12 up, 12 in 2024-03-21T10:21:58.214 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:21:57 smithi047 ceph-mon[31547]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:21:58.214 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:21:57 smithi047 ceph-mon[31547]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:21:58.214 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:21:57 smithi047 ceph-mon[31547]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:21:58.214 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:21:57 smithi047 ceph-mon[31547]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:21:58.214 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:21:57 smithi047 ceph-mon[31547]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:21:58.214 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:21:57 smithi047 ceph-mon[31547]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:21:58.230 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:21:57 smithi100 ceph-mon[35193]: pgmap v358: 1 pgs: 1 active+clean; 577 KiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-21T10:21:58.230 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:21:57 smithi100 ceph-mon[35193]: from='client.? 172.21.15.47:0/2766052447' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-03-21T10:21:58.230 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:21:57 smithi100 ceph-mon[35193]: osdmap e84: 12 total, 12 up, 12 in 2024-03-21T10:21:58.230 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:21:57 smithi100 ceph-mon[35193]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:21:58.230 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:21:57 smithi100 ceph-mon[35193]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:21:58.230 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:21:57 smithi100 ceph-mon[35193]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:21:58.230 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:21:57 smithi100 ceph-mon[35193]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:21:58.230 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:21:57 smithi100 ceph-mon[35193]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:21:58.230 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:21:57 smithi100 ceph-mon[35193]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:21:58.240 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:21:57 smithi157 ceph-mon[35581]: pgmap v358: 1 pgs: 1 active+clean; 577 KiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-21T10:21:58.240 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:21:57 smithi157 ceph-mon[35581]: from='client.? 172.21.15.47:0/2766052447' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-03-21T10:21:58.240 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:21:57 smithi157 ceph-mon[35581]: osdmap e84: 12 total, 12 up, 12 in 2024-03-21T10:21:58.240 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:21:57 smithi157 ceph-mon[35581]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:21:58.240 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:21:57 smithi157 ceph-mon[35581]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:21:58.240 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:21:57 smithi157 ceph-mon[35581]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:21:58.240 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:21:57 smithi157 ceph-mon[35581]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:21:58.240 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:21:57 smithi157 ceph-mon[35581]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:21:58.240 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:21:57 smithi157 ceph-mon[35581]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:21:58.328 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:58.326+0000 7f697ce00640 1 -- 172.21.15.47:0/1275790043 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f6978062310 con 0x7f69781038e0 2024-03-21T10:21:59.214 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:21:58 smithi047 ceph-mon[31547]: from='client.? 172.21.15.47:0/4124883544' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]: dispatch 2024-03-21T10:21:59.214 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:21:58 smithi047 ceph-mon[31547]: from='client.? 172.21.15.47:0/4124883544' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]': finished 2024-03-21T10:21:59.214 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:21:58 smithi047 ceph-mon[31547]: osdmap e85: 12 total, 12 up, 12 in 2024-03-21T10:21:59.215 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:21:58 smithi047 ceph-mon[31547]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T10:21:59.215 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:21:58 smithi047 ceph-mon[31547]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T10:21:59.215 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:21:58 smithi047 ceph-mon[31547]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:21:59.215 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:21:58 smithi047 ceph-mon[31547]: from='client.? 172.21.15.47:0/1275790043' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-03-21T10:21:59.215 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:21:58 smithi047 ceph-mon[31547]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-03-21T10:21:59.215 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:21:58 smithi047 ceph-mon[31547]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T10:21:59.230 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:21:58 smithi100 ceph-mon[35193]: from='client.? 172.21.15.47:0/4124883544' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]: dispatch 2024-03-21T10:21:59.230 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:21:58 smithi100 ceph-mon[35193]: from='client.? 172.21.15.47:0/4124883544' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]': finished 2024-03-21T10:21:59.230 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:21:58 smithi100 ceph-mon[35193]: osdmap e85: 12 total, 12 up, 12 in 2024-03-21T10:21:59.230 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:21:58 smithi100 ceph-mon[35193]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T10:21:59.230 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:21:58 smithi100 ceph-mon[35193]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T10:21:59.230 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:21:58 smithi100 ceph-mon[35193]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:21:59.231 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:21:58 smithi100 ceph-mon[35193]: from='client.? 172.21.15.47:0/1275790043' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-03-21T10:21:59.231 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:21:58 smithi100 ceph-mon[35193]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-03-21T10:21:59.231 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:21:58 smithi100 ceph-mon[35193]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T10:21:59.240 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:21:58 smithi157 ceph-mon[35581]: from='client.? 172.21.15.47:0/4124883544' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]: dispatch 2024-03-21T10:21:59.240 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:21:58 smithi157 ceph-mon[35581]: from='client.? 172.21.15.47:0/4124883544' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]': finished 2024-03-21T10:21:59.240 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:21:58 smithi157 ceph-mon[35581]: osdmap e85: 12 total, 12 up, 12 in 2024-03-21T10:21:59.240 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:21:58 smithi157 ceph-mon[35581]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T10:21:59.240 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:21:58 smithi157 ceph-mon[35581]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T10:21:59.241 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:21:58 smithi157 ceph-mon[35581]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:21:59.241 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:21:58 smithi157 ceph-mon[35581]: from='client.? 172.21.15.47:0/1275790043' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-03-21T10:21:59.241 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:21:58 smithi157 ceph-mon[35581]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-03-21T10:21:59.241 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:21:58 smithi157 ceph-mon[35581]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T10:21:59.255 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:59.252+0000 7f696ea00640 1 -- 172.21.15.47:0/1275790043 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 2 and data pool 3 v14) v1 ==== 172+0+0 (secure 0 0 0) 0x7f69680beba0 con 0x7f69781038e0 2024-03-21T10:21:59.255 INFO:teuthology.orchestra.run.smithi047.stderr:new fs with metadata pool 2 and data pool 3 2024-03-21T10:21:59.259 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:59.256+0000 7f697ce00640 1 -- 172.21.15.47:0/1275790043 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f695c0a2f20 msgr2=0x7f695c0a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:21:59.259 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:59.256+0000 7f697ce00640 1 --2- 172.21.15.47:0/1275790043 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f695c0a2f20 0x7f695c0a53e0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f697819b020 tx=0x7f696400b410 comp rx=0 tx=0).stop 2024-03-21T10:21:59.259 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:59.256+0000 7f697ce00640 1 -- 172.21.15.47:0/1275790043 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f69781038e0 msgr2=0x7f6978199b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:21:59.259 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:59.256+0000 7f697ce00640 1 --2- 172.21.15.47:0/1275790043 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f69781038e0 0x7f6978199b10 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f6968000c00 tx=0x7f6968004270 comp rx=0 tx=0).stop 2024-03-21T10:21:59.259 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:59.257+0000 7f697ce00640 1 -- 172.21.15.47:0/1275790043 shutdown_connections 2024-03-21T10:21:59.259 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:59.257+0000 7f697ce00640 1 --2- 172.21.15.47:0/1275790043 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f695c0a2f20 0x7f695c0a53e0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:21:59.260 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:59.257+0000 7f697ce00640 1 --2- 172.21.15.47:0/1275790043 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6978105490 0x7f69781a10c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:21:59.260 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:59.257+0000 7f697ce00640 1 --2- 172.21.15.47:0/1275790043 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6978104ad0 0x7f697819a050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:21:59.260 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:59.257+0000 7f697ce00640 1 --2- 172.21.15.47:0/1275790043 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f69781038e0 0x7f6978199b10 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:21:59.260 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:59.257+0000 7f697ce00640 1 -- 172.21.15.47:0/1275790043 >> 172.21.15.47:0/1275790043 conn(0x7f69780fee50 msgr2=0x7f6978107e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:21:59.261 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:59.257+0000 7f697ce00640 1 -- 172.21.15.47:0/1275790043 shutdown_connections 2024-03-21T10:21:59.261 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:59.257+0000 7f697ce00640 1 -- 172.21.15.47:0/1275790043 wait complete. 2024-03-21T10:21:59.284 DEBUG:tasks.cephfs.filesystem:EC profile is ['m=2', 'k=2', 'crush-failure-domain=osd'] 2024-03-21T10:21:59.284 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd erasure-code-profile set cephfs_data_ec m=2 k=2 crush-failure-domain=osd 2024-03-21T10:21:59.483 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:59.480+0000 7f5c36200640 1 -- 172.21.15.47:0/3980038337 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5c30107320 msgr2=0x7f5c301097e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:21:59.483 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:59.480+0000 7f5c36200640 1 --2- 172.21.15.47:0/3980038337 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5c30107320 0x7f5c301097e0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f5c28009ca0 tx=0x7f5c2802c0f0 comp rx=0 tx=0).stop 2024-03-21T10:21:59.483 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:59.481+0000 7f5c36200640 1 -- 172.21.15.47:0/3980038337 shutdown_connections 2024-03-21T10:21:59.483 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:59.481+0000 7f5c36200640 1 --2- 172.21.15.47:0/3980038337 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5c30107320 0x7f5c301097e0 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:21:59.483 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:59.481+0000 7f5c36200640 1 --2- 172.21.15.47:0/3980038337 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5c300ffd90 0x7f5c30106d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:21:59.484 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:59.481+0000 7f5c36200640 1 --2- 172.21.15.47:0/3980038337 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5c300ff430 0x7f5c300ff850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:21:59.484 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:59.481+0000 7f5c36200640 1 -- 172.21.15.47:0/3980038337 >> 172.21.15.47:0/3980038337 conn(0x7f5c300fae60 msgr2=0x7f5c300fd2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:21:59.484 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:59.481+0000 7f5c36200640 1 -- 172.21.15.47:0/3980038337 shutdown_connections 2024-03-21T10:21:59.484 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:59.481+0000 7f5c36200640 1 -- 172.21.15.47:0/3980038337 wait complete. 2024-03-21T10:21:59.486 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:59.482+0000 7f5c36200640 1 Processor -- start 2024-03-21T10:21:59.486 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:59.482+0000 7f5c36200640 1 -- start start 2024-03-21T10:21:59.486 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:59.482+0000 7f5c36200640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5c300ff430 0x7f5c301026e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:21:59.486 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:59.482+0000 7f5c36200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5c300ffd90 0x7f5c30100d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:21:59.486 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:59.482+0000 7f5c36200640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5c30107320 0x7f5c30101260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:21:59.486 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:59.482+0000 7f5c36200640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c3010d1e0 con 0x7f5c30107320 2024-03-21T10:21:59.486 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:59.482+0000 7f5c36200640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c3010d060 con 0x7f5c300ffd90 2024-03-21T10:21:59.486 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:59.482+0000 7f5c36200640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c3010d360 con 0x7f5c300ff430 2024-03-21T10:21:59.486 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:59.483+0000 7f5c2fe00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5c30107320 0x7f5c30101260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:21:59.487 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:59.483+0000 7f5c2f400640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5c300ff430 0x7f5c301026e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:21:59.487 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:59.483+0000 7f5c2ea00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5c300ffd90 0x7f5c30100d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:21:59.487 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:59.483+0000 7f5c2fe00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5c30107320 0x7f5c30101260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:57856/0 (socket says 172.21.15.47:57856) 2024-03-21T10:21:59.487 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:59.483+0000 7f5c2f400640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5c300ff430 0x7f5c301026e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.47:42318/0 (socket says 172.21.15.47:42318) 2024-03-21T10:21:59.487 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:59.483+0000 7f5c2fe00640 1 -- 172.21.15.47:0/2050949862 learned_addr learned my addr 172.21.15.47:0/2050949862 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:21:59.487 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:59.483+0000 7f5c2fe00640 1 -- 172.21.15.47:0/2050949862 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5c300ff430 msgr2=0x7f5c301026e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:21:59.487 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:59.483+0000 7f5c2fe00640 1 --2- 172.21.15.47:0/2050949862 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5c300ff430 0x7f5c301026e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:21:59.487 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:59.483+0000 7f5c2fe00640 1 -- 172.21.15.47:0/2050949862 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5c300ffd90 msgr2=0x7f5c30100d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:21:59.487 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:59.483+0000 7f5c2fe00640 1 --2- 172.21.15.47:0/2050949862 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5c300ffd90 0x7f5c30100d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:21:59.487 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:59.483+0000 7f5c2fe00640 1 -- 172.21.15.47:0/2050949862 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c30101af0 con 0x7f5c30107320 2024-03-21T10:21:59.487 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:59.483+0000 7f5c2fe00640 1 --2- 172.21.15.47:0/2050949862 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5c30107320 0x7f5c30101260 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f5c28032380 tx=0x7f5c28032820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:21:59.487 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:59.484+0000 7f5c27e00640 1 -- 172.21.15.47:0/2050949862 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c28041020 con 0x7f5c30107320 2024-03-21T10:21:59.487 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:59.484+0000 7f5c27e00640 1 -- 172.21.15.47:0/2050949862 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f5c28003e20 con 0x7f5c30107320 2024-03-21T10:21:59.487 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:59.484+0000 7f5c36200640 1 -- 172.21.15.47:0/2050949862 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5c301a19a0 con 0x7f5c30107320 2024-03-21T10:21:59.488 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:59.484+0000 7f5c27e00640 1 -- 172.21.15.47:0/2050949862 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c28039040 con 0x7f5c30107320 2024-03-21T10:21:59.488 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:59.484+0000 7f5c36200640 1 -- 172.21.15.47:0/2050949862 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5c301a1ef0 con 0x7f5c30107320 2024-03-21T10:21:59.488 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:59.485+0000 7f5c36200640 1 -- 172.21.15.47:0/2050949862 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5c30105ba0 con 0x7f5c30107320 2024-03-21T10:21:59.489 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:59.487+0000 7f5c27e00640 1 -- 172.21.15.47:0/2050949862 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 16) v1 ==== 137854+0+0 (secure 0 0 0) 0x7f5c28044020 con 0x7f5c30107320 2024-03-21T10:21:59.490 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:59.488+0000 7f5c27e00640 1 --2- 172.21.15.47:0/2050949862 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f5c140a2f20 0x7f5c140a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:21:59.493 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:59.488+0000 7f5c27e00640 1 -- 172.21.15.47:0/2050949862 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f5c280f1430 con 0x7f5c30107320 2024-03-21T10:21:59.493 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:59.489+0000 7f5c2f400640 1 --2- 172.21.15.47:0/2050949862 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f5c140a2f20 0x7f5c140a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:21:59.494 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:59.489+0000 7f5c2f400640 1 --2- 172.21.15.47:0/2050949862 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f5c140a2f20 0x7f5c140a53e0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f5c20009bd0 tx=0x7f5c20005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:21:59.495 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:59.493+0000 7f5c27e00640 1 -- 172.21.15.47:0/2050949862 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f5c2804b020 con 0x7f5c30107320 2024-03-21T10:21:59.730 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:59.728+0000 7f5c36200640 1 -- 172.21.15.47:0/2050949862 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]} v 0) v1 -- 0x7f5c3002d070 con 0x7f5c30107320 2024-03-21T10:21:59.844 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:59.842+0000 7f5c27e00640 1 -- 172.21.15.47:0/2050949862 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]=0 v87) v1 ==== 157+0+0 (secure 0 0 0) 0x7f5c280be310 con 0x7f5c30107320 2024-03-21T10:21:59.849 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:59.846+0000 7f5c36200640 1 -- 172.21.15.47:0/2050949862 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f5c140a2f20 msgr2=0x7f5c140a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:21:59.849 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:59.846+0000 7f5c36200640 1 --2- 172.21.15.47:0/2050949862 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f5c140a2f20 0x7f5c140a53e0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f5c20009bd0 tx=0x7f5c20005f90 comp rx=0 tx=0).stop 2024-03-21T10:21:59.849 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:59.846+0000 7f5c36200640 1 -- 172.21.15.47:0/2050949862 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5c30107320 msgr2=0x7f5c30101260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:21:59.849 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:59.846+0000 7f5c36200640 1 --2- 172.21.15.47:0/2050949862 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5c30107320 0x7f5c30101260 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f5c28032380 tx=0x7f5c28032820 comp rx=0 tx=0).stop 2024-03-21T10:21:59.849 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:59.847+0000 7f5c36200640 1 -- 172.21.15.47:0/2050949862 shutdown_connections 2024-03-21T10:21:59.849 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:59.847+0000 7f5c36200640 1 --2- 172.21.15.47:0/2050949862 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f5c140a2f20 0x7f5c140a53e0 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:21:59.849 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:59.847+0000 7f5c36200640 1 --2- 172.21.15.47:0/2050949862 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5c30107320 0x7f5c30101260 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:21:59.849 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:59.847+0000 7f5c36200640 1 --2- 172.21.15.47:0/2050949862 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5c300ffd90 0x7f5c30100d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:21:59.849 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:59.847+0000 7f5c36200640 1 --2- 172.21.15.47:0/2050949862 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5c300ff430 0x7f5c301026e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:21:59.850 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:59.847+0000 7f5c36200640 1 -- 172.21.15.47:0/2050949862 >> 172.21.15.47:0/2050949862 conn(0x7f5c300fae60 msgr2=0x7f5c30102d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:21:59.850 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:59.847+0000 7f5c36200640 1 -- 172.21.15.47:0/2050949862 shutdown_connections 2024-03-21T10:21:59.850 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:21:59.847+0000 7f5c36200640 1 -- 172.21.15.47:0/2050949862 wait complete. 2024-03-21T10:21:59.866 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data_ec erasure cephfs_data_ec --pg_num_min 64 --target_size_ratio 0.9 2024-03-21T10:22:00.092 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:00.089+0000 7fc780200640 1 -- 172.21.15.47:0/2992815038 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc778104ab0 msgr2=0x7fc778104f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:22:00.092 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:00.089+0000 7fc780200640 1 --2- 172.21.15.47:0/2992815038 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc778104ab0 0x7fc778104f30 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fc764009900 tx=0x7fc76402bc30 comp rx=0 tx=0).stop 2024-03-21T10:22:00.092 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:00.090+0000 7fc780200640 1 -- 172.21.15.47:0/2992815038 shutdown_connections 2024-03-21T10:22:00.092 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:00.090+0000 7fc780200640 1 --2- 172.21.15.47:0/2992815038 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc778105470 0x7fc77810bd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:22:00.092 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:00.090+0000 7fc780200640 1 --2- 172.21.15.47:0/2992815038 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc778104ab0 0x7fc778104f30 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:22:00.093 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:00.090+0000 7fc780200640 1 --2- 172.21.15.47:0/2992815038 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc7781038c0 0x7fc778103ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:22:00.093 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:00.090+0000 7fc780200640 1 -- 172.21.15.47:0/2992815038 >> 172.21.15.47:0/2992815038 conn(0x7fc7780fee50 msgr2=0x7fc778101270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:22:00.093 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:00.090+0000 7fc780200640 1 -- 172.21.15.47:0/2992815038 shutdown_connections 2024-03-21T10:22:00.093 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:00.090+0000 7fc780200640 1 -- 172.21.15.47:0/2992815038 wait complete. 2024-03-21T10:22:00.095 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:00.091+0000 7fc780200640 1 Processor -- start 2024-03-21T10:22:00.095 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:00.091+0000 7fc780200640 1 -- start start 2024-03-21T10:22:00.095 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:00.091+0000 7fc780200640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc7781038c0 0x7fc778199b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:22:00.095 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:00.091+0000 7fc780200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc778104ab0 0x7fc77819a070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:22:00.095 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:00.091+0000 7fc780200640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc778105470 0x7fc7781a10e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:22:00.095 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:00.091+0000 7fc780200640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc77810f3d0 con 0x7fc778105470 2024-03-21T10:22:00.095 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:00.091+0000 7fc780200640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc77810f250 con 0x7fc778104ab0 2024-03-21T10:22:00.095 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:00.091+0000 7fc780200640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc77810f550 con 0x7fc7781038c0 2024-03-21T10:22:00.095 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:00.091+0000 7fc77e200640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc778105470 0x7fc7781a10e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:22:00.095 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:00.091+0000 7fc77e200640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc778105470 0x7fc7781a10e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:57874/0 (socket says 172.21.15.47:57874) 2024-03-21T10:22:00.095 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:00.091+0000 7fc77e200640 1 -- 172.21.15.47:0/1469421755 learned_addr learned my addr 172.21.15.47:0/1469421755 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:22:00.096 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:00.092+0000 7fc77d800640 1 --2- 172.21.15.47:0/1469421755 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc7781038c0 0x7fc778199b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:22:00.096 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:00.092+0000 7fc77ce00640 1 --2- 172.21.15.47:0/1469421755 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc778104ab0 0x7fc77819a070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:22:00.096 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:00.092+0000 7fc77e200640 1 -- 172.21.15.47:0/1469421755 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc7781038c0 msgr2=0x7fc778199b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:22:00.096 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:00.092+0000 7fc77e200640 1 --2- 172.21.15.47:0/1469421755 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc7781038c0 0x7fc778199b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:22:00.096 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:00.092+0000 7fc77e200640 1 -- 172.21.15.47:0/1469421755 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc778104ab0 msgr2=0x7fc77819a070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:22:00.096 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:00.092+0000 7fc77e200640 1 --2- 172.21.15.47:0/1469421755 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc778104ab0 0x7fc77819a070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:22:00.096 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:00.092+0000 7fc77e200640 1 -- 172.21.15.47:0/1469421755 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc7781a1750 con 0x7fc778105470 2024-03-21T10:22:00.096 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:00.092+0000 7fc77e200640 1 --2- 172.21.15.47:0/1469421755 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc778105470 0x7fc7781a10e0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fc77000cad0 tx=0x7fc77000cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:22:00.096 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:00.092+0000 7fc776000640 1 -- 172.21.15.47:0/1469421755 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc77000ae20 con 0x7fc778105470 2024-03-21T10:22:00.096 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:00.092+0000 7fc776000640 1 -- 172.21.15.47:0/1469421755 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fc770005020 con 0x7fc778105470 2024-03-21T10:22:00.096 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:00.092+0000 7fc780200640 1 -- 172.21.15.47:0/1469421755 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc7781a1a40 con 0x7fc778105470 2024-03-21T10:22:00.096 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:00.092+0000 7fc776000640 1 -- 172.21.15.47:0/1469421755 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc77000f6b0 con 0x7fc778105470 2024-03-21T10:22:00.096 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:00.093+0000 7fc780200640 1 -- 172.21.15.47:0/1469421755 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc7781a1f40 con 0x7fc778105470 2024-03-21T10:22:00.097 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:00.094+0000 7fc780200640 1 -- 172.21.15.47:0/1469421755 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc77810acd0 con 0x7fc778105470 2024-03-21T10:22:00.097 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:00.096+0000 7fc776000640 1 -- 172.21.15.47:0/1469421755 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 16) v1 ==== 137854+0+0 (secure 0 0 0) 0x7fc7700051d0 con 0x7fc778105470 2024-03-21T10:22:00.100 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:00.098+0000 7fc776000640 1 --2- 172.21.15.47:0/1469421755 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7fc7600a2f20 0x7fc7600a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:22:00.100 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:00.098+0000 7fc776000640 1 -- 172.21.15.47:0/1469421755 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8277+0+0 (secure 0 0 0) 0x7fc7700cf590 con 0x7fc778105470 2024-03-21T10:22:00.100 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:00.098+0000 7fc77d800640 1 --2- 172.21.15.47:0/1469421755 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7fc7600a2f20 0x7fc7600a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:22:00.101 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:00.099+0000 7fc77d800640 1 --2- 172.21.15.47:0/1469421755 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7fc7600a2f20 0x7fc7600a53e0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fc76c005db0 tx=0x7fc76c005d40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:22:00.105 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:00.102+0000 7fc776000640 1 -- 172.21.15.47:0/1469421755 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fc770014030 con 0x7fc778105470 2024-03-21T10:22:00.148 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:21:59 smithi100 ceph-mon[35193]: pgmap v361: 97 pgs: 9 active+clean, 29 creating+peering, 59 unknown; 577 KiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-21T10:22:00.148 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:21:59 smithi100 ceph-mon[35193]: osdmap e86: 12 total, 12 up, 12 in 2024-03-21T10:22:00.148 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:21:59 smithi100 ceph-mon[35193]: Health check failed: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-21T10:22:00.148 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:21:59 smithi100 ceph-mon[35193]: from='client.? 172.21.15.47:0/1275790043' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-03-21T10:22:00.149 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:21:59 smithi100 ceph-mon[35193]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-03-21T10:22:00.149 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:21:59 smithi100 ceph-mon[35193]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-03-21T10:22:00.149 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:21:59 smithi100 ceph-mon[35193]: from='client.? 172.21.15.47:0/2050949862' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]: dispatch 2024-03-21T10:22:00.178 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:21:59 smithi157 ceph-mon[35581]: pgmap v361: 97 pgs: 9 active+clean, 29 creating+peering, 59 unknown; 577 KiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-21T10:22:00.178 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:21:59 smithi157 ceph-mon[35581]: osdmap e86: 12 total, 12 up, 12 in 2024-03-21T10:22:00.178 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:21:59 smithi157 ceph-mon[35581]: Health check failed: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-21T10:22:00.179 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:21:59 smithi157 ceph-mon[35581]: from='client.? 172.21.15.47:0/1275790043' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-03-21T10:22:00.179 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:21:59 smithi157 ceph-mon[35581]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-03-21T10:22:00.179 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:21:59 smithi157 ceph-mon[35581]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-03-21T10:22:00.179 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:21:59 smithi157 ceph-mon[35581]: from='client.? 172.21.15.47:0/2050949862' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]: dispatch 2024-03-21T10:22:00.209 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:21:59 smithi047 ceph-mon[31547]: pgmap v361: 97 pgs: 9 active+clean, 29 creating+peering, 59 unknown; 577 KiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-21T10:22:00.209 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:21:59 smithi047 ceph-mon[31547]: osdmap e86: 12 total, 12 up, 12 in 2024-03-21T10:22:00.209 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:21:59 smithi047 ceph-mon[31547]: Health check failed: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-21T10:22:00.209 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:21:59 smithi047 ceph-mon[31547]: from='client.? 172.21.15.47:0/1275790043' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-03-21T10:22:00.209 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:21:59 smithi047 ceph-mon[31547]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-03-21T10:22:00.209 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:21:59 smithi047 ceph-mon[31547]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-03-21T10:22:00.209 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:21:59 smithi047 ceph-mon[31547]: from='client.? 172.21.15.47:0/2050949862' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]: dispatch 2024-03-21T10:22:00.304 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:00.301+0000 7fc780200640 1 -- 172.21.15.47:0/1469421755 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7fc77802d070 con 0x7fc778105470 2024-03-21T10:22:01.214 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:22:00 smithi047 ceph-mon[31547]: from='client.? 172.21.15.47:0/2050949862' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]': finished 2024-03-21T10:22:01.214 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:22:00 smithi047 ceph-mon[31547]: osdmap e87: 12 total, 12 up, 12 in 2024-03-21T10:22:01.214 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:22:00 smithi047 ceph-mon[31547]: mds.? [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] up:active 2024-03-21T10:22:01.214 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:22:00 smithi047 ceph-mon[31547]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-03-21T10:22:01.214 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:22:00 smithi047 ceph-mon[31547]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-03-21T10:22:01.214 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:22:00 smithi047 ceph-mon[31547]: from='client.? 172.21.15.47:0/1469421755' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-03-21T10:22:01.230 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:22:00 smithi100 ceph-mon[35193]: from='client.? 172.21.15.47:0/2050949862' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]': finished 2024-03-21T10:22:01.230 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:22:00 smithi100 ceph-mon[35193]: osdmap e87: 12 total, 12 up, 12 in 2024-03-21T10:22:01.230 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:22:00 smithi100 ceph-mon[35193]: mds.? [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] up:active 2024-03-21T10:22:01.230 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:22:00 smithi100 ceph-mon[35193]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-03-21T10:22:01.230 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:22:00 smithi100 ceph-mon[35193]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-03-21T10:22:01.230 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:22:00 smithi100 ceph-mon[35193]: from='client.? 172.21.15.47:0/1469421755' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-03-21T10:22:01.239 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:22:00 smithi157 ceph-mon[35581]: from='client.? 172.21.15.47:0/2050949862' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]': finished 2024-03-21T10:22:01.239 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:22:00 smithi157 ceph-mon[35581]: osdmap e87: 12 total, 12 up, 12 in 2024-03-21T10:22:01.239 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:22:00 smithi157 ceph-mon[35581]: mds.? [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] up:active 2024-03-21T10:22:01.240 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:22:00 smithi157 ceph-mon[35581]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-03-21T10:22:01.240 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:22:00 smithi157 ceph-mon[35581]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-03-21T10:22:01.240 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:22:00 smithi157 ceph-mon[35581]: from='client.? 172.21.15.47:0/1469421755' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-03-21T10:22:02.214 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:22:01 smithi047 ceph-mon[31547]: pgmap v364: 97 pgs: 37 active+clean, 37 creating+peering, 23 unknown; 577 KiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 0 op/s 2024-03-21T10:22:02.214 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:22:01 smithi047 ceph-mon[31547]: Health check cleared: POOL_APP_NOT_ENABLED (was: 2 pool(s) do not have an application enabled) 2024-03-21T10:22:02.214 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:22:01 smithi047 ceph-mon[31547]: Cluster is now healthy 2024-03-21T10:22:02.214 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:22:01 smithi047 ceph-mon[31547]: osdmap e88: 12 total, 12 up, 12 in 2024-03-21T10:22:02.214 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:22:01 smithi047 ceph-mon[31547]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:22:02.229 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:22:01 smithi100 ceph-mon[35193]: pgmap v364: 97 pgs: 37 active+clean, 37 creating+peering, 23 unknown; 577 KiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 0 op/s 2024-03-21T10:22:02.229 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:22:01 smithi100 ceph-mon[35193]: Health check cleared: POOL_APP_NOT_ENABLED (was: 2 pool(s) do not have an application enabled) 2024-03-21T10:22:02.230 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:22:01 smithi100 ceph-mon[35193]: Cluster is now healthy 2024-03-21T10:22:02.230 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:22:01 smithi100 ceph-mon[35193]: osdmap e88: 12 total, 12 up, 12 in 2024-03-21T10:22:02.230 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:22:01 smithi100 ceph-mon[35193]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:22:02.240 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:22:01 smithi157 ceph-mon[35581]: pgmap v364: 97 pgs: 37 active+clean, 37 creating+peering, 23 unknown; 577 KiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 0 op/s 2024-03-21T10:22:02.240 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:22:01 smithi157 ceph-mon[35581]: Health check cleared: POOL_APP_NOT_ENABLED (was: 2 pool(s) do not have an application enabled) 2024-03-21T10:22:02.240 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:22:01 smithi157 ceph-mon[35581]: Cluster is now healthy 2024-03-21T10:22:02.240 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:22:01 smithi157 ceph-mon[35581]: osdmap e88: 12 total, 12 up, 12 in 2024-03-21T10:22:02.240 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:22:01 smithi157 ceph-mon[35581]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:22:02.275 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:02.274+0000 7fc776000640 1 -- 172.21.15.47:0/1469421755 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data_ec' created v89) v1 ==== 228+0+0 (secure 0 0 0) 0x7fc77009c330 con 0x7fc778105470 2024-03-21T10:22:02.276 INFO:teuthology.orchestra.run.smithi047.stderr:pool 'cephfs_data_ec' created 2024-03-21T10:22:02.280 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:02.279+0000 7fc780200640 1 -- 172.21.15.47:0/1469421755 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7fc7600a2f20 msgr2=0x7fc7600a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:22:02.280 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:02.279+0000 7fc780200640 1 --2- 172.21.15.47:0/1469421755 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7fc7600a2f20 0x7fc7600a53e0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fc76c005db0 tx=0x7fc76c005d40 comp rx=0 tx=0).stop 2024-03-21T10:22:02.280 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:02.279+0000 7fc780200640 1 -- 172.21.15.47:0/1469421755 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc778105470 msgr2=0x7fc7781a10e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:22:02.280 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:02.279+0000 7fc780200640 1 --2- 172.21.15.47:0/1469421755 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc778105470 0x7fc7781a10e0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fc77000cad0 tx=0x7fc77000cf90 comp rx=0 tx=0).stop 2024-03-21T10:22:02.281 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:02.279+0000 7fc780200640 1 -- 172.21.15.47:0/1469421755 shutdown_connections 2024-03-21T10:22:02.281 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:02.279+0000 7fc780200640 1 --2- 172.21.15.47:0/1469421755 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7fc7600a2f20 0x7fc7600a53e0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:22:02.281 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:02.279+0000 7fc780200640 1 --2- 172.21.15.47:0/1469421755 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc778105470 0x7fc7781a10e0 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:22:02.281 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:02.279+0000 7fc780200640 1 --2- 172.21.15.47:0/1469421755 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc778104ab0 0x7fc77819a070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:22:02.281 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:02.279+0000 7fc780200640 1 --2- 172.21.15.47:0/1469421755 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc7781038c0 0x7fc778199b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:22:02.281 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:02.279+0000 7fc780200640 1 -- 172.21.15.47:0/1469421755 >> 172.21.15.47:0/1469421755 conn(0x7fc7780fee50 msgr2=0x7fc778107ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:22:02.281 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:02.280+0000 7fc780200640 1 -- 172.21.15.47:0/1469421755 shutdown_connections 2024-03-21T10:22:02.281 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:02.280+0000 7fc780200640 1 -- 172.21.15.47:0/1469421755 wait complete. 2024-03-21T10:22:02.310 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool set cephfs_data_ec allow_ec_overwrites true 2024-03-21T10:22:02.504 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:02.502+0000 7fe7f7200640 1 -- 172.21.15.47:0/1446260604 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe7f0108db0 msgr2=0x7fe7f0109230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:22:02.504 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:02.502+0000 7fe7f7200640 1 --2- 172.21.15.47:0/1446260604 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe7f0108db0 0x7fe7f0109230 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fe7e0009900 tx=0x7fe7e002bc30 comp rx=0 tx=0).stop 2024-03-21T10:22:02.504 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:02.502+0000 7fe7f7200640 1 -- 172.21.15.47:0/1446260604 shutdown_connections 2024-03-21T10:22:02.504 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:02.502+0000 7fe7f7200640 1 --2- 172.21.15.47:0/1446260604 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe7f0109770 0x7fe7f0110000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:22:02.504 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:02.502+0000 7fe7f7200640 1 --2- 172.21.15.47:0/1446260604 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe7f0108db0 0x7fe7f0109230 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:22:02.504 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:02.502+0000 7fe7f7200640 1 --2- 172.21.15.47:0/1446260604 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe7f0107bc0 0x7fe7f0107fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:22:02.504 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:02.502+0000 7fe7f7200640 1 -- 172.21.15.47:0/1446260604 >> 172.21.15.47:0/1446260604 conn(0x7fe7f0076b70 msgr2=0x7fe7f0078f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:22:02.504 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:02.502+0000 7fe7f7200640 1 -- 172.21.15.47:0/1446260604 shutdown_connections 2024-03-21T10:22:02.504 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:02.502+0000 7fe7f7200640 1 -- 172.21.15.47:0/1446260604 wait complete. 2024-03-21T10:22:02.506 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:02.503+0000 7fe7f7200640 1 Processor -- start 2024-03-21T10:22:02.506 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:02.503+0000 7fe7f7200640 1 -- start start 2024-03-21T10:22:02.507 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:02.504+0000 7fe7f7200640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe7f0107bc0 0x7fe7f019de10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:22:02.507 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:02.504+0000 7fe7f7200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe7f0108db0 0x7fe7f019e350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:22:02.507 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:02.504+0000 7fe7f7200640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe7f0109770 0x7fe7f01a53c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:22:02.507 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:02.504+0000 7fe7f7200640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7f01136f0 con 0x7fe7f0109770 2024-03-21T10:22:02.507 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:02.504+0000 7fe7f7200640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7f0113570 con 0x7fe7f0108db0 2024-03-21T10:22:02.507 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:02.504+0000 7fe7f7200640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7f0113870 con 0x7fe7f0107bc0 2024-03-21T10:22:02.507 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:02.504+0000 7fe7f6800640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe7f0109770 0x7fe7f01a53c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:22:02.507 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:02.504+0000 7fe7f6800640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe7f0109770 0x7fe7f01a53c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:46936/0 (socket says 172.21.15.47:46936) 2024-03-21T10:22:02.507 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:02.504+0000 7fe7f6800640 1 -- 172.21.15.47:0/2951784641 learned_addr learned my addr 172.21.15.47:0/2951784641 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:22:02.507 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:02.504+0000 7fe7f5400640 1 --2- 172.21.15.47:0/2951784641 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe7f0108db0 0x7fe7f019e350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:22:02.507 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:02.504+0000 7fe7f5e00640 1 --2- 172.21.15.47:0/2951784641 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe7f0107bc0 0x7fe7f019de10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:22:02.507 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:02.504+0000 7fe7f6800640 1 -- 172.21.15.47:0/2951784641 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe7f0107bc0 msgr2=0x7fe7f019de10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:22:02.507 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:02.504+0000 7fe7f6800640 1 --2- 172.21.15.47:0/2951784641 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe7f0107bc0 0x7fe7f019de10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:22:02.508 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:02.504+0000 7fe7f6800640 1 -- 172.21.15.47:0/2951784641 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe7f0108db0 msgr2=0x7fe7f019e350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:22:02.508 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:02.504+0000 7fe7f6800640 1 --2- 172.21.15.47:0/2951784641 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe7f0108db0 0x7fe7f019e350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:22:02.508 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:02.504+0000 7fe7f6800640 1 -- 172.21.15.47:0/2951784641 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe7f01a5a30 con 0x7fe7f0109770 2024-03-21T10:22:02.508 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:02.505+0000 7fe7f6800640 1 --2- 172.21.15.47:0/2951784641 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe7f0109770 0x7fe7f01a53c0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fe7ec00eb10 tx=0x7fe7ec00c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:22:02.508 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:02.505+0000 7fe7eaa00640 1 -- 172.21.15.47:0/2951784641 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7ec004b20 con 0x7fe7f0109770 2024-03-21T10:22:02.508 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:02.505+0000 7fe7eaa00640 1 -- 172.21.15.47:0/2951784641 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fe7ec004cc0 con 0x7fe7f0109770 2024-03-21T10:22:02.508 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:02.505+0000 7fe7eaa00640 1 -- 172.21.15.47:0/2951784641 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7ec010690 con 0x7fe7f0109770 2024-03-21T10:22:02.508 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:02.505+0000 7fe7f7200640 1 -- 172.21.15.47:0/2951784641 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe7f01a5d20 con 0x7fe7f0109770 2024-03-21T10:22:02.508 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:02.505+0000 7fe7f7200640 1 -- 172.21.15.47:0/2951784641 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe7f01a6190 con 0x7fe7f0109770 2024-03-21T10:22:02.508 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:02.506+0000 7fe7f7200640 1 -- 172.21.15.47:0/2951784641 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe7f010efb0 con 0x7fe7f0109770 2024-03-21T10:22:02.509 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:02.508+0000 7fe7eaa00640 1 -- 172.21.15.47:0/2951784641 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 16) v1 ==== 137854+0+0 (secure 0 0 0) 0x7fe7ec005020 con 0x7fe7f0109770 2024-03-21T10:22:02.511 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:02.510+0000 7fe7eaa00640 1 --2- 172.21.15.47:0/2951784641 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7fe7d80a2f20 0x7fe7d80a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:22:02.511 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:02.510+0000 7fe7eaa00640 1 -- 172.21.15.47:0/2951784641 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8770+0+0 (secure 0 0 0) 0x7fe7ec0cfc70 con 0x7fe7f0109770 2024-03-21T10:22:02.511 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:02.510+0000 7fe7f5e00640 1 --2- 172.21.15.47:0/2951784641 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7fe7d80a2f20 0x7fe7d80a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:22:02.511 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:02.510+0000 7fe7f5e00640 1 --2- 172.21.15.47:0/2951784641 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7fe7d80a2f20 0x7fe7d80a53e0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fe7e4005fd0 tx=0x7fe7e4005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:22:02.517 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:02.515+0000 7fe7eaa00640 1 -- 172.21.15.47:0/2951784641 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fe7ec09c820 con 0x7fe7f0109770 2024-03-21T10:22:02.709 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:02.708+0000 7fe7f7200640 1 -- 172.21.15.47:0/2951784641 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"} v 0) v1 -- 0x7fe7f002d070 con 0x7fe7f0109770 2024-03-21T10:22:03.215 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:22:02 smithi047 ceph-mon[31547]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:22:03.215 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:22:02 smithi047 ceph-mon[31547]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:22:03.215 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:22:02 smithi047 ceph-mon[31547]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:22:03.215 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:22:02 smithi047 ceph-mon[31547]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:22:03.215 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:22:02 smithi047 ceph-mon[31547]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:22:03.215 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:22:02 smithi047 ceph-mon[31547]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T10:22:03.215 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:22:02 smithi047 ceph-mon[31547]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T10:22:03.215 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:22:02 smithi047 ceph-mon[31547]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:22:03.215 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:22:02 smithi047 ceph-mon[31547]: from='client.? 172.21.15.47:0/1469421755' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-03-21T10:22:03.215 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:22:02 smithi047 ceph-mon[31547]: osdmap e89: 12 total, 12 up, 12 in 2024-03-21T10:22:03.215 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:22:02 smithi047 ceph-mon[31547]: from='client.? 172.21.15.47:0/2951784641' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]: dispatch 2024-03-21T10:22:03.230 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:22:02 smithi100 ceph-mon[35193]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:22:03.230 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:22:02 smithi100 ceph-mon[35193]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:22:03.230 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:22:02 smithi100 ceph-mon[35193]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:22:03.230 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:22:02 smithi100 ceph-mon[35193]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:22:03.230 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:22:02 smithi100 ceph-mon[35193]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:22:03.230 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:22:02 smithi100 ceph-mon[35193]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T10:22:03.230 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:22:02 smithi100 ceph-mon[35193]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T10:22:03.230 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:22:02 smithi100 ceph-mon[35193]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:22:03.230 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:22:02 smithi100 ceph-mon[35193]: from='client.? 172.21.15.47:0/1469421755' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-03-21T10:22:03.231 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:22:02 smithi100 ceph-mon[35193]: osdmap e89: 12 total, 12 up, 12 in 2024-03-21T10:22:03.231 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:22:02 smithi100 ceph-mon[35193]: from='client.? 172.21.15.47:0/2951784641' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]: dispatch 2024-03-21T10:22:03.241 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:22:02 smithi157 ceph-mon[35581]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:22:03.241 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:22:02 smithi157 ceph-mon[35581]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:22:03.241 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:22:02 smithi157 ceph-mon[35581]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:22:03.241 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:22:02 smithi157 ceph-mon[35581]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:22:03.241 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:22:02 smithi157 ceph-mon[35581]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:22:03.241 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:22:02 smithi157 ceph-mon[35581]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T10:22:03.241 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:22:02 smithi157 ceph-mon[35581]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T10:22:03.241 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:22:02 smithi157 ceph-mon[35581]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:22:03.241 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:22:02 smithi157 ceph-mon[35581]: from='client.? 172.21.15.47:0/1469421755' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-03-21T10:22:03.241 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:22:02 smithi157 ceph-mon[35581]: osdmap e89: 12 total, 12 up, 12 in 2024-03-21T10:22:03.241 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:22:02 smithi157 ceph-mon[35581]: from='client.? 172.21.15.47:0/2951784641' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]: dispatch 2024-03-21T10:22:03.280 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:03.278+0000 7fe7eaa00640 1 -- 172.21.15.47:0/2951784641 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]=0 set pool 4 allow_ec_overwrites to true v90) v1 ==== 169+0+0 (secure 0 0 0) 0x7fe7ec09c120 con 0x7fe7f0109770 2024-03-21T10:22:03.280 INFO:teuthology.orchestra.run.smithi047.stderr:set pool 4 allow_ec_overwrites to true 2024-03-21T10:22:03.284 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:03.282+0000 7fe7f7200640 1 -- 172.21.15.47:0/2951784641 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7fe7d80a2f20 msgr2=0x7fe7d80a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:22:03.284 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:03.282+0000 7fe7f7200640 1 --2- 172.21.15.47:0/2951784641 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7fe7d80a2f20 0x7fe7d80a53e0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fe7e4005fd0 tx=0x7fe7e4005850 comp rx=0 tx=0).stop 2024-03-21T10:22:03.284 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:03.282+0000 7fe7f7200640 1 -- 172.21.15.47:0/2951784641 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe7f0109770 msgr2=0x7fe7f01a53c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:22:03.284 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:03.282+0000 7fe7f7200640 1 --2- 172.21.15.47:0/2951784641 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe7f0109770 0x7fe7f01a53c0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fe7ec00eb10 tx=0x7fe7ec00c2d0 comp rx=0 tx=0).stop 2024-03-21T10:22:03.284 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:03.282+0000 7fe7f7200640 1 -- 172.21.15.47:0/2951784641 shutdown_connections 2024-03-21T10:22:03.284 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:03.282+0000 7fe7f7200640 1 --2- 172.21.15.47:0/2951784641 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7fe7d80a2f20 0x7fe7d80a53e0 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:22:03.284 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:03.282+0000 7fe7f7200640 1 --2- 172.21.15.47:0/2951784641 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe7f0109770 0x7fe7f01a53c0 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:22:03.284 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:03.282+0000 7fe7f7200640 1 --2- 172.21.15.47:0/2951784641 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe7f0108db0 0x7fe7f019e350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:22:03.284 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:03.282+0000 7fe7f7200640 1 --2- 172.21.15.47:0/2951784641 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe7f0107bc0 0x7fe7f019de10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:22:03.285 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:03.282+0000 7fe7f7200640 1 -- 172.21.15.47:0/2951784641 >> 172.21.15.47:0/2951784641 conn(0x7fe7f0076b70 msgr2=0x7fe7f010c180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:22:03.285 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:03.283+0000 7fe7f7200640 1 -- 172.21.15.47:0/2951784641 shutdown_connections 2024-03-21T10:22:03.285 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:03.284+0000 7fe7f7200640 1 -- 172.21.15.47:0/2951784641 wait complete. 2024-03-21T10:22:03.320 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs add_data_pool cephfs cephfs_data_ec 2024-03-21T10:22:03.501 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:03.499+0000 7f42c8400640 1 -- 172.21.15.47:0/4185083517 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f42c0104a70 msgr2=0x7f42c0106e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:22:03.501 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:03.499+0000 7f42c8400640 1 --2- 172.21.15.47:0/4185083517 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f42c0104a70 0x7f42c0106e60 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f42ac009900 tx=0x7f42ac02bc30 comp rx=0 tx=0).stop 2024-03-21T10:22:03.501 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:03.500+0000 7f42c8400640 1 -- 172.21.15.47:0/4185083517 shutdown_connections 2024-03-21T10:22:03.502 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:03.500+0000 7f42c8400640 1 --2- 172.21.15.47:0/4185083517 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f42c01073a0 0x7f42c01097f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:22:03.502 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:03.500+0000 7f42c8400640 1 --2- 172.21.15.47:0/4185083517 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f42c0104a70 0x7f42c0106e60 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:22:03.502 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:03.500+0000 7f42c8400640 1 --2- 172.21.15.47:0/4185083517 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f42c0102140 0x7f42c0104530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:22:03.502 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:03.500+0000 7f42c8400640 1 -- 172.21.15.47:0/4185083517 >> 172.21.15.47:0/4185083517 conn(0x7f42c00fbb50 msgr2=0x7f42c00fdf70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:22:03.502 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:03.500+0000 7f42c8400640 1 -- 172.21.15.47:0/4185083517 shutdown_connections 2024-03-21T10:22:03.502 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:03.500+0000 7f42c8400640 1 -- 172.21.15.47:0/4185083517 wait complete. 2024-03-21T10:22:03.504 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:03.501+0000 7f42c8400640 1 Processor -- start 2024-03-21T10:22:03.505 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:03.501+0000 7f42c8400640 1 -- start start 2024-03-21T10:22:03.505 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:03.502+0000 7f42c8400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f42c0102140 0x7f42c0101a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:22:03.505 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:03.502+0000 7f42c8400640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f42c0104a70 0x7f42c01000f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:22:03.505 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:03.502+0000 7f42c8400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f42c01073a0 0x7f42c0100630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:22:03.505 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:03.502+0000 7f42c8400640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42c010d200 con 0x7f42c0102140 2024-03-21T10:22:03.505 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:03.502+0000 7f42c8400640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42c010d080 con 0x7f42c01073a0 2024-03-21T10:22:03.505 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:03.502+0000 7f42c8400640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42c010d380 con 0x7f42c0104a70 2024-03-21T10:22:03.505 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:03.502+0000 7f42c5a00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f42c0102140 0x7f42c0101a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:22:03.505 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:03.502+0000 7f42c5a00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f42c0102140 0x7f42c0101a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:46948/0 (socket says 172.21.15.47:46948) 2024-03-21T10:22:03.505 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:03.502+0000 7f42c5a00640 1 -- 172.21.15.47:0/4266342554 learned_addr learned my addr 172.21.15.47:0/4266342554 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:22:03.506 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:03.502+0000 7f42c6400640 1 --2- 172.21.15.47:0/4266342554 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f42c01073a0 0x7f42c0100630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:22:03.506 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:03.502+0000 7f42c5000640 1 --2- 172.21.15.47:0/4266342554 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f42c0104a70 0x7f42c01000f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:22:03.506 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:03.503+0000 7f42c5a00640 1 -- 172.21.15.47:0/4266342554 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f42c0104a70 msgr2=0x7f42c01000f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:22:03.506 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:03.503+0000 7f42c5a00640 1 --2- 172.21.15.47:0/4266342554 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f42c0104a70 0x7f42c01000f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:22:03.506 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:03.503+0000 7f42c5a00640 1 -- 172.21.15.47:0/4266342554 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f42c01073a0 msgr2=0x7f42c0100630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:22:03.506 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:03.503+0000 7f42c5a00640 1 --2- 172.21.15.47:0/4266342554 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f42c01073a0 0x7f42c0100630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:22:03.506 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:03.503+0000 7f42c5a00640 1 -- 172.21.15.47:0/4266342554 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f42c0100ef0 con 0x7f42c0102140 2024-03-21T10:22:03.506 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:03.503+0000 7f42c5000640 1 --2- 172.21.15.47:0/4266342554 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f42c0104a70 0x7f42c01000f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T10:22:03.506 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:03.503+0000 7f42c5a00640 1 --2- 172.21.15.47:0/4266342554 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f42c0102140 0x7f42c0101a60 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f42b400d770 tx=0x7f42b400dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:22:03.506 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:03.503+0000 7f42be000640 1 -- 172.21.15.47:0/4266342554 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42b4017070 con 0x7f42c0102140 2024-03-21T10:22:03.507 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:03.503+0000 7f42be000640 1 -- 172.21.15.47:0/4266342554 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f42b40054d0 con 0x7f42c0102140 2024-03-21T10:22:03.507 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:03.503+0000 7f42be000640 1 -- 172.21.15.47:0/4266342554 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42b4004cf0 con 0x7f42c0102140 2024-03-21T10:22:03.507 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:03.503+0000 7f42c8400640 1 -- 172.21.15.47:0/4266342554 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f42c01a1ac0 con 0x7f42c0102140 2024-03-21T10:22:03.507 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:03.503+0000 7f42c8400640 1 -- 172.21.15.47:0/4266342554 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f42c01a1f90 con 0x7f42c0102140 2024-03-21T10:22:03.507 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:03.505+0000 7f42c8400640 1 -- 172.21.15.47:0/4266342554 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f42c00fd640 con 0x7f42c0102140 2024-03-21T10:22:03.509 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:03.508+0000 7f42be000640 1 -- 172.21.15.47:0/4266342554 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 16) v1 ==== 137854+0+0 (secure 0 0 0) 0x7f42b4004e90 con 0x7f42c0102140 2024-03-21T10:22:03.511 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:03.510+0000 7f42be000640 1 --2- 172.21.15.47:0/4266342554 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f42a80a2fa0 0x7f42a80a5460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:22:03.512 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:03.511+0000 7f42be000640 1 -- 172.21.15.47:0/4266342554 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 8770+0+0 (secure 0 0 0) 0x7f42b40cf370 con 0x7f42c0102140 2024-03-21T10:22:03.512 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:03.511+0000 7f42c5000640 1 --2- 172.21.15.47:0/4266342554 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f42a80a2fa0 0x7f42a80a5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:22:03.512 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:03.512+0000 7f42c5000640 1 --2- 172.21.15.47:0/4266342554 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f42a80a2fa0 0x7f42a80a5460 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f42ac006010 tx=0x7f42ac02c7e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:22:03.513 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:03.513+0000 7f42be000640 1 -- 172.21.15.47:0/4266342554 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f42b409c000 con 0x7f42c0102140 2024-03-21T10:22:03.758 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:03.757+0000 7f42c8400640 1 -- 172.21.15.47:0/4266342554 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"} v 0) v1 -- 0x7f42c002d070 con 0x7f42c0102140 2024-03-21T10:22:04.214 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:22:03 smithi047 ceph-mon[31547]: pgmap v366: 97 pgs: 54 active+clean, 37 creating+peering, 6 unknown; 577 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 0 op/s 2024-03-21T10:22:04.214 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:22:03 smithi047 ceph-mon[31547]: from='client.? 172.21.15.47:0/2951784641' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]': finished 2024-03-21T10:22:04.214 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:22:03 smithi047 ceph-mon[31547]: osdmap e90: 12 total, 12 up, 12 in 2024-03-21T10:22:04.214 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:22:03 smithi047 ceph-mon[31547]: from='client.? 172.21.15.47:0/4266342554' entity='client.admin' cmd=[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]: dispatch 2024-03-21T10:22:04.229 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:22:03 smithi100 ceph-mon[35193]: pgmap v366: 97 pgs: 54 active+clean, 37 creating+peering, 6 unknown; 577 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 0 op/s 2024-03-21T10:22:04.229 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:22:03 smithi100 ceph-mon[35193]: from='client.? 172.21.15.47:0/2951784641' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]': finished 2024-03-21T10:22:04.229 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:22:03 smithi100 ceph-mon[35193]: osdmap e90: 12 total, 12 up, 12 in 2024-03-21T10:22:04.230 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:22:03 smithi100 ceph-mon[35193]: from='client.? 172.21.15.47:0/4266342554' entity='client.admin' cmd=[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]: dispatch 2024-03-21T10:22:04.230 INFO:journalctl@ceph.mds.b.smithi100.stdout:Mar 21 10:22:03 smithi100 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-mds-b[64996]: 2024-03-21T10:22:03.926+0000 7f0aad200700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-03-21T10:22:04.240 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:22:03 smithi157 ceph-mon[35581]: pgmap v366: 97 pgs: 54 active+clean, 37 creating+peering, 6 unknown; 577 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 0 op/s 2024-03-21T10:22:04.240 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:22:03 smithi157 ceph-mon[35581]: from='client.? 172.21.15.47:0/2951784641' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]': finished 2024-03-21T10:22:04.240 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:22:03 smithi157 ceph-mon[35581]: osdmap e90: 12 total, 12 up, 12 in 2024-03-21T10:22:04.240 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:22:03 smithi157 ceph-mon[35581]: from='client.? 172.21.15.47:0/4266342554' entity='client.admin' cmd=[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]: dispatch 2024-03-21T10:22:04.565 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:04.563+0000 7f42be000640 1 -- 172.21.15.47:0/4266342554 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]=0 added data pool 4 to fsmap v17) v1 ==== 137+0+0 (secure 0 0 0) 0x7f42b40cf020 con 0x7f42c0102140 2024-03-21T10:22:04.565 INFO:teuthology.orchestra.run.smithi047.stderr:added data pool 4 to fsmap 2024-03-21T10:22:04.568 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:04.567+0000 7f42c8400640 1 -- 172.21.15.47:0/4266342554 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f42a80a2fa0 msgr2=0x7f42a80a5460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:22:04.568 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:04.567+0000 7f42c8400640 1 --2- 172.21.15.47:0/4266342554 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f42a80a2fa0 0x7f42a80a5460 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f42ac006010 tx=0x7f42ac02c7e0 comp rx=0 tx=0).stop 2024-03-21T10:22:04.568 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:04.567+0000 7f42c8400640 1 -- 172.21.15.47:0/4266342554 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f42c0102140 msgr2=0x7f42c0101a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:22:04.568 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:04.567+0000 7f42c8400640 1 --2- 172.21.15.47:0/4266342554 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f42c0102140 0x7f42c0101a60 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f42b400d770 tx=0x7f42b400dc30 comp rx=0 tx=0).stop 2024-03-21T10:22:04.568 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:04.567+0000 7f42c8400640 1 -- 172.21.15.47:0/4266342554 shutdown_connections 2024-03-21T10:22:04.569 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:04.567+0000 7f42c8400640 1 --2- 172.21.15.47:0/4266342554 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f42a80a2fa0 0x7f42a80a5460 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:22:04.569 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:04.567+0000 7f42c8400640 1 --2- 172.21.15.47:0/4266342554 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f42c01073a0 0x7f42c0100630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:22:04.569 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:04.567+0000 7f42c8400640 1 --2- 172.21.15.47:0/4266342554 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f42c0104a70 0x7f42c01000f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:22:04.569 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:04.567+0000 7f42c8400640 1 --2- 172.21.15.47:0/4266342554 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f42c0102140 0x7f42c0101a60 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:22:04.569 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:04.567+0000 7f42c8400640 1 -- 172.21.15.47:0/4266342554 >> 172.21.15.47:0/4266342554 conn(0x7f42c00fbb50 msgr2=0x7f42c0102be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:22:04.569 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:04.567+0000 7f42c8400640 1 -- 172.21.15.47:0/4266342554 shutdown_connections 2024-03-21T10:22:04.569 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:04.567+0000 7f42c8400640 1 -- 172.21.15.47:0/4266342554 wait complete. 2024-03-21T10:22:04.590 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-21T10:22:04.812 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:04.807+0000 7faa0ea00640 1 -- 172.21.15.47:0/2816336843 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7faa08108ed0 msgr2=0x7faa08109350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:22:04.813 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:04.807+0000 7faa0ea00640 1 --2- 172.21.15.47:0/2816336843 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7faa08108ed0 0x7faa08109350 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fa9f0009900 tx=0x7fa9f002bc30 comp rx=0 tx=0).stop 2024-03-21T10:22:04.813 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:04.807+0000 7faa0ea00640 1 -- 172.21.15.47:0/2816336843 shutdown_connections 2024-03-21T10:22:04.813 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:04.807+0000 7faa0ea00640 1 --2- 172.21.15.47:0/2816336843 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7faa08109890 0x7faa08110120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:22:04.813 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:04.807+0000 7faa0ea00640 1 --2- 172.21.15.47:0/2816336843 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7faa08108ed0 0x7faa08109350 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:22:04.813 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:04.807+0000 7faa0ea00640 1 --2- 172.21.15.47:0/2816336843 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7faa08107ce0 0x7faa08108100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:22:04.813 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:04.808+0000 7faa0ea00640 1 -- 172.21.15.47:0/2816336843 >> 172.21.15.47:0/2816336843 conn(0x7faa08076cb0 msgr2=0x7faa080790d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:22:04.813 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:04.808+0000 7faa0ea00640 1 -- 172.21.15.47:0/2816336843 shutdown_connections 2024-03-21T10:22:04.813 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:04.808+0000 7faa0ea00640 1 -- 172.21.15.47:0/2816336843 wait complete. 2024-03-21T10:22:04.813 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:04.809+0000 7faa0ea00640 1 Processor -- start 2024-03-21T10:22:04.813 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:04.809+0000 7faa0ea00640 1 -- start start 2024-03-21T10:22:04.813 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:04.809+0000 7faa0ea00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7faa08107ce0 0x7faa0819df10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:22:04.813 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:04.809+0000 7faa0ea00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7faa08108ed0 0x7faa0819e450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:22:04.813 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:04.809+0000 7faa0ea00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7faa08109890 0x7faa081a54c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:22:04.814 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:04.809+0000 7faa0ea00640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa08113840 con 0x7faa08109890 2024-03-21T10:22:04.814 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:04.809+0000 7faa0ea00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa081136c0 con 0x7faa08108ed0 2024-03-21T10:22:04.814 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:04.809+0000 7faa0ea00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa081139c0 con 0x7faa08107ce0 2024-03-21T10:22:04.814 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:04.810+0000 7faa0ca00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7faa08109890 0x7faa081a54c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:22:04.814 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:04.810+0000 7faa0ca00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7faa08109890 0x7faa081a54c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:46974/0 (socket says 172.21.15.47:46974) 2024-03-21T10:22:04.814 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:04.810+0000 7faa0ca00640 1 -- 172.21.15.47:0/3478741258 learned_addr learned my addr 172.21.15.47:0/3478741258 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:22:04.814 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:04.810+0000 7fa9ff400640 1 --2- 172.21.15.47:0/3478741258 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7faa08108ed0 0x7faa0819e450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:22:04.814 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:04.810+0000 7faa07e00640 1 --2- 172.21.15.47:0/3478741258 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7faa08107ce0 0x7faa0819df10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:22:04.814 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:04.810+0000 7faa0ca00640 1 -- 172.21.15.47:0/3478741258 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7faa08107ce0 msgr2=0x7faa0819df10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:22:04.814 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:04.810+0000 7faa0ca00640 1 --2- 172.21.15.47:0/3478741258 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7faa08107ce0 0x7faa0819df10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:22:04.814 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:04.810+0000 7faa0ca00640 1 -- 172.21.15.47:0/3478741258 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7faa08108ed0 msgr2=0x7faa0819e450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:22:04.814 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:04.810+0000 7faa0ca00640 1 --2- 172.21.15.47:0/3478741258 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7faa08108ed0 0x7faa0819e450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:22:04.814 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:04.810+0000 7faa0ca00640 1 -- 172.21.15.47:0/3478741258 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa081a5b30 con 0x7faa08109890 2024-03-21T10:22:04.815 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:04.810+0000 7faa0ca00640 1 --2- 172.21.15.47:0/3478741258 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7faa08109890 0x7faa081a54c0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7faa0000d750 tx=0x7faa0000dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:22:04.815 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:04.811+0000 7faa05600640 1 -- 172.21.15.47:0/3478741258 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa00017070 con 0x7faa08109890 2024-03-21T10:22:04.815 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:04.811+0000 7faa0ea00640 1 -- 172.21.15.47:0/3478741258 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faa081a5e20 con 0x7faa08109890 2024-03-21T10:22:04.815 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:04.811+0000 7faa05600640 1 -- 172.21.15.47:0/3478741258 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7faa000054d0 con 0x7faa08109890 2024-03-21T10:22:04.815 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:04.811+0000 7faa05600640 1 -- 172.21.15.47:0/3478741258 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa00004c60 con 0x7faa08109890 2024-03-21T10:22:04.815 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:04.811+0000 7faa0ea00640 1 -- 172.21.15.47:0/3478741258 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faa081a62e0 con 0x7faa08109890 2024-03-21T10:22:04.815 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:04.812+0000 7faa0ea00640 1 -- 172.21.15.47:0/3478741258 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faa0810f050 con 0x7faa08109890 2024-03-21T10:22:04.815 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:04.814+0000 7faa05600640 1 -- 172.21.15.47:0/3478741258 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 16) v1 ==== 137854+0+0 (secure 0 0 0) 0x7faa00004e00 con 0x7faa08109890 2024-03-21T10:22:04.816 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:04.815+0000 7faa05600640 1 --2- 172.21.15.47:0/3478741258 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7fa9ec0a2fa0 0x7fa9ec0a5460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:22:04.816 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:04.815+0000 7faa05600640 1 -- 172.21.15.47:0/3478741258 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 8802+0+0 (secure 0 0 0) 0x7faa000cf060 con 0x7faa08109890 2024-03-21T10:22:04.817 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:04.816+0000 7faa07e00640 1 --2- 172.21.15.47:0/3478741258 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7fa9ec0a2fa0 0x7fa9ec0a5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:22:04.817 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:04.816+0000 7faa07e00640 1 --2- 172.21.15.47:0/3478741258 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7fa9ec0a2fa0 0x7fa9ec0a5460 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7faa08108d30 tx=0x7fa9f8009870 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:22:04.821 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:04.820+0000 7faa05600640 1 -- 172.21.15.47:0/3478741258 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7faa0009bbf0 con 0x7faa08109890 2024-03-21T10:22:05.058 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:05.057+0000 7faa0ea00640 1 -- 172.21.15.47:0/3478741258 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7faa0802d470 con 0x7faa08109890 2024-03-21T10:22:05.060 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:05.059+0000 7faa05600640 1 -- 172.21.15.47:0/3478741258 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) v1 ==== 94+0+11336 (secure 0 0 0) 0x7faa0009b4f0 con 0x7faa08109890 2024-03-21T10:22:05.061 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-21T10:22:05.061 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":17,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14511,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6835/735724037","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6834","nonce":735724037},{"type":"v1","addr":"172.21.15.47:6835","nonce":735724037}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14529,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6837/2248191724","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6836","nonce":2248191724},{"type":"v1","addr":"172.21.15.47:6837","nonce":2248191724}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14604,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6839/2881335356","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6838","nonce":2881335356},{"type":"v1","addr":"172.21.15.47:6839","nonce":2881335356}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":14619,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6841/3401578978","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6840","nonce":3401578978},{"type":"v1","addr":"172.21.15.47:6841","nonce":3401578978}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":24406,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6833/4000010136","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":4000010136},{"type":"v1","addr":"172.21.15.157:6833","nonce":4000010136}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24457,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6835/3902868651","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":3902868651},{"type":"v1","addr":"172.21.15.157:6835","nonce":3902868651}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":7},{"gid":24475,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6837/1908087204","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6836","nonce":1908087204},{"type":"v1","addr":"172.21.15.157:6837","nonce":1908087204}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":8},{"gid":24499,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6839/4064242876","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6838","nonce":4064242876},{"type":"v1","addr":"172.21.15.157:6839","nonce":4064242876}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":10},{"gid":34427,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6833/2723302038","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6832","nonce":2723302038},{"type":"v1","addr":"172.21.15.100:6833","nonce":2723302038}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":34454,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6835/2597849957","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":2597849957},{"type":"v1","addr":"172.21.15.100:6835","nonce":2597849957}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6},{"gid":34493,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6837/3497644162","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6836","nonce":3497644162},{"type":"v1","addr":"172.21.15.100:6837","nonce":3497644162}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":9}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-21T10:21:58.329063+0000","modified":"2024-03-21T10:22:03.759800+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":34535},"failed":[],"damaged":[],"stopped":[],"info":{"gid_34535":{"gid":34535,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.100:6839/4173168440","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6838","nonce":4173168440},{"type":"v1","addr":"172.21.15.100:6839","nonce":4173168440}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":34535,"qdb_cluster":[34535]},"id":1}]} 2024-03-21T10:22:05.061 INFO:teuthology.orchestra.run.smithi047.stderr:dumped fsmap epoch 17 2024-03-21T10:22:05.063 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:05.062+0000 7faa0ea00640 1 -- 172.21.15.47:0/3478741258 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7fa9ec0a2fa0 msgr2=0x7fa9ec0a5460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:22:05.063 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:05.062+0000 7faa0ea00640 1 --2- 172.21.15.47:0/3478741258 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7fa9ec0a2fa0 0x7fa9ec0a5460 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7faa08108d30 tx=0x7fa9f8009870 comp rx=0 tx=0).stop 2024-03-21T10:22:05.063 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:05.062+0000 7faa0ea00640 1 -- 172.21.15.47:0/3478741258 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7faa08109890 msgr2=0x7faa081a54c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:22:05.063 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:05.062+0000 7faa0ea00640 1 --2- 172.21.15.47:0/3478741258 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7faa08109890 0x7faa081a54c0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7faa0000d750 tx=0x7faa0000dc10 comp rx=0 tx=0).stop 2024-03-21T10:22:05.064 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:05.062+0000 7faa0ea00640 1 -- 172.21.15.47:0/3478741258 shutdown_connections 2024-03-21T10:22:05.064 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:05.062+0000 7faa0ea00640 1 --2- 172.21.15.47:0/3478741258 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7fa9ec0a2fa0 0x7fa9ec0a5460 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:22:05.064 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:05.062+0000 7faa0ea00640 1 --2- 172.21.15.47:0/3478741258 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7faa08109890 0x7faa081a54c0 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:22:05.064 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:05.062+0000 7faa0ea00640 1 --2- 172.21.15.47:0/3478741258 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7faa08108ed0 0x7faa0819e450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:22:05.064 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:05.062+0000 7faa0ea00640 1 --2- 172.21.15.47:0/3478741258 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7faa08107ce0 0x7faa0819df10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:22:05.064 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:05.062+0000 7faa0ea00640 1 -- 172.21.15.47:0/3478741258 >> 172.21.15.47:0/3478741258 conn(0x7faa08076cb0 msgr2=0x7faa0810c220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:22:05.064 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:05.063+0000 7faa0ea00640 1 -- 172.21.15.47:0/3478741258 shutdown_connections 2024-03-21T10:22:05.065 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:05.063+0000 7faa0ea00640 1 -- 172.21.15.47:0/3478741258 wait complete. 2024-03-21T10:22:05.080 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-21T10:22:05.158 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:22:04 smithi047 ceph-mon[31547]: osdmap e91: 12 total, 12 up, 12 in 2024-03-21T10:22:05.158 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:22:04 smithi047 ceph-mon[31547]: from='client.? 172.21.15.47:0/4266342554' entity='client.admin' cmd='[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]': finished 2024-03-21T10:22:05.158 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:22:04 smithi047 ceph-mon[31547]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-03-21T10:22:05.229 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:22:04 smithi100 ceph-mon[35193]: osdmap e91: 12 total, 12 up, 12 in 2024-03-21T10:22:05.230 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:22:04 smithi100 ceph-mon[35193]: from='client.? 172.21.15.47:0/4266342554' entity='client.admin' cmd='[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]': finished 2024-03-21T10:22:05.230 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:22:04 smithi100 ceph-mon[35193]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-03-21T10:22:05.240 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:22:04 smithi157 ceph-mon[35581]: osdmap e91: 12 total, 12 up, 12 in 2024-03-21T10:22:05.240 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:22:04 smithi157 ceph-mon[35581]: from='client.? 172.21.15.47:0/4266342554' entity='client.admin' cmd='[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]': finished 2024-03-21T10:22:05.240 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:22:04 smithi157 ceph-mon[35581]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-03-21T10:22:05.292 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:05.290+0000 7fba73e00640 1 -- 172.21.15.47:0/3421041606 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fba741052a0 msgr2=0x7fba7410bb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:22:05.292 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:05.290+0000 7fba73e00640 1 --2- 172.21.15.47:0/3421041606 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fba741052a0 0x7fba7410bb30 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fba6c009ca0 tx=0x7fba6c02c100 comp rx=0 tx=0).stop 2024-03-21T10:22:05.292 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:05.291+0000 7fba73e00640 1 -- 172.21.15.47:0/3421041606 shutdown_connections 2024-03-21T10:22:05.292 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:05.291+0000 7fba73e00640 1 --2- 172.21.15.47:0/3421041606 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fba741052a0 0x7fba7410bb30 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:22:05.292 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:05.291+0000 7fba73e00640 1 --2- 172.21.15.47:0/3421041606 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fba741048e0 0x7fba74104d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:22:05.292 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:05.291+0000 7fba73e00640 1 --2- 172.21.15.47:0/3421041606 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fba741036f0 0x7fba74103b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:22:05.292 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:05.291+0000 7fba73e00640 1 -- 172.21.15.47:0/3421041606 >> 172.21.15.47:0/3421041606 conn(0x7fba740fec80 msgr2=0x7fba741010a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:22:05.293 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:05.291+0000 7fba73e00640 1 -- 172.21.15.47:0/3421041606 shutdown_connections 2024-03-21T10:22:05.293 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:05.291+0000 7fba73e00640 1 -- 172.21.15.47:0/3421041606 wait complete. 2024-03-21T10:22:05.294 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:05.291+0000 7fba73e00640 1 Processor -- start 2024-03-21T10:22:05.294 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:05.292+0000 7fba73e00640 1 -- start start 2024-03-21T10:22:05.295 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:05.292+0000 7fba73e00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fba741036f0 0x7fba741998c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:22:05.295 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:05.292+0000 7fba73e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fba741048e0 0x7fba74199e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:22:05.295 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:05.292+0000 7fba73e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fba741052a0 0x7fba741a0e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:22:05.295 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:05.292+0000 7fba73e00640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba7410f220 con 0x7fba741052a0 2024-03-21T10:22:05.295 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:05.292+0000 7fba73e00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba7410f0a0 con 0x7fba741048e0 2024-03-21T10:22:05.295 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:05.292+0000 7fba73e00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba7410f3a0 con 0x7fba741036f0 2024-03-21T10:22:05.295 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:05.292+0000 7fba73400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fba741052a0 0x7fba741a0e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:22:05.295 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:05.292+0000 7fba73400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fba741052a0 0x7fba741a0e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:46992/0 (socket says 172.21.15.47:46992) 2024-03-21T10:22:05.295 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:05.292+0000 7fba73400640 1 -- 172.21.15.47:0/4078933318 learned_addr learned my addr 172.21.15.47:0/4078933318 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:22:05.295 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:05.292+0000 7fba72a00640 1 --2- 172.21.15.47:0/4078933318 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fba741036f0 0x7fba741998c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:22:05.295 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:05.293+0000 7fba73400640 1 -- 172.21.15.47:0/4078933318 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fba741036f0 msgr2=0x7fba741998c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:22:05.295 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:05.293+0000 7fba73400640 1 --2- 172.21.15.47:0/4078933318 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fba741036f0 0x7fba741998c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:22:05.296 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:05.293+0000 7fba73400640 1 -- 172.21.15.47:0/4078933318 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fba741048e0 msgr2=0x7fba74199e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:22:05.296 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:05.293+0000 7fba73400640 1 --2- 172.21.15.47:0/4078933318 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fba741048e0 0x7fba74199e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:22:05.296 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:05.293+0000 7fba73400640 1 -- 172.21.15.47:0/4078933318 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba741a14e0 con 0x7fba741052a0 2024-03-21T10:22:05.296 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:05.293+0000 7fba72a00640 1 --2- 172.21.15.47:0/4078933318 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fba741036f0 0x7fba741998c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T10:22:05.296 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:05.293+0000 7fba73400640 1 --2- 172.21.15.47:0/4078933318 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fba741052a0 0x7fba741a0e70 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fba6c032520 tx=0x7fba6c034880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:22:05.296 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:05.293+0000 7fba6b400640 1 -- 172.21.15.47:0/4078933318 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba6c039040 con 0x7fba741052a0 2024-03-21T10:22:05.296 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:05.293+0000 7fba6b400640 1 -- 172.21.15.47:0/4078933318 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fba6c044070 con 0x7fba741052a0 2024-03-21T10:22:05.296 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:05.293+0000 7fba6b400640 1 -- 172.21.15.47:0/4078933318 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba6c0043c0 con 0x7fba741052a0 2024-03-21T10:22:05.296 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:05.293+0000 7fba73e00640 1 -- 172.21.15.47:0/4078933318 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fba741a1770 con 0x7fba741052a0 2024-03-21T10:22:05.296 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:05.293+0000 7fba73e00640 1 -- 172.21.15.47:0/4078933318 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fba741a1cb0 con 0x7fba741052a0 2024-03-21T10:22:05.297 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:05.294+0000 7fba73e00640 1 -- 172.21.15.47:0/4078933318 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fba7410aa60 con 0x7fba741052a0 2024-03-21T10:22:05.297 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:05.296+0000 7fba6b400640 1 -- 172.21.15.47:0/4078933318 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 16) v1 ==== 137854+0+0 (secure 0 0 0) 0x7fba6c004560 con 0x7fba741052a0 2024-03-21T10:22:05.300 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:05.299+0000 7fba6b400640 1 --2- 172.21.15.47:0/4078933318 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7fba580a2fa0 0x7fba580a5460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:22:05.300 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:05.299+0000 7fba6b400640 1 -- 172.21.15.47:0/4078933318 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 8802+0+0 (secure 0 0 0) 0x7fba6c0f1d70 con 0x7fba741052a0 2024-03-21T10:22:05.300 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:05.299+0000 7fba72a00640 1 --2- 172.21.15.47:0/4078933318 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7fba580a2fa0 0x7fba580a5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:22:05.300 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:05.300+0000 7fba72a00640 1 --2- 172.21.15.47:0/4078933318 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7fba580a2fa0 0x7fba580a5460 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fba74104740 tx=0x7fba64005e70 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:22:05.304 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:05.303+0000 7fba6b400640 1 -- 172.21.15.47:0/4078933318 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fba6c0333a0 con 0x7fba741052a0 2024-03-21T10:22:05.500 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:05.498+0000 7fba73e00640 1 -- 172.21.15.47:0/4078933318 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fba740622a0 con 0x7fba741052a0 2024-03-21T10:22:05.501 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:05.500+0000 7fba6b400640 1 -- 172.21.15.47:0/4078933318 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v91) v1 ==== 74+0+24116 (secure 0 0 0) 0x7fba6c0be900 con 0x7fba741052a0 2024-03-21T10:22:05.502 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-21T10:22:05.502 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":91,"fsid":"3056d51a-e76b-11ee-95cd-87774f69a715","created":"2024-03-21T10:11:17.019965+0000","modified":"2024-03-21T10:22:03.759214+0000","last_up_change":"2024-03-21T10:20:52.675328+0000","last_in_change":"2024-03-21T10:20:27.648212+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-21T10:15:40.553114+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-21T10:21:56.399518+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-21T10:21:57.812940+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-21T10:22:01.822568+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"91","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"b0c7a102-8d89-428b-b793-cbbe67151f50","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6802","nonce":775837313},{"type":"v1","addr":"172.21.15.47:6803","nonce":775837313}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6804","nonce":775837313},{"type":"v1","addr":"172.21.15.47:6805","nonce":775837313}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6808","nonce":775837313},{"type":"v1","addr":"172.21.15.47:6809","nonce":775837313}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6806","nonce":775837313},{"type":"v1","addr":"172.21.15.47:6807","nonce":775837313}]},"public_addr":"172.21.15.47:6803/775837313","cluster_addr":"172.21.15.47:6805/775837313","heartbeat_back_addr":"172.21.15.47:6809/775837313","heartbeat_front_addr":"172.21.15.47:6807/775837313","state":["exists","up"]},{"osd":1,"uuid":"07bb81bb-7cf3-4bbc-ac9e-802f95185d11","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6800","nonce":2923304710},{"type":"v1","addr":"172.21.15.100:6801","nonce":2923304710}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":2923304710},{"type":"v1","addr":"172.21.15.100:6803","nonce":2923304710}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":2923304710},{"type":"v1","addr":"172.21.15.100:6807","nonce":2923304710}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6804","nonce":2923304710},{"type":"v1","addr":"172.21.15.100:6805","nonce":2923304710}]},"public_addr":"172.21.15.100:6801/2923304710","cluster_addr":"172.21.15.100:6803/2923304710","heartbeat_back_addr":"172.21.15.100:6807/2923304710","heartbeat_front_addr":"172.21.15.100:6805/2923304710","state":["exists","up"]},{"osd":2,"uuid":"88ce76b9-6081-4836-aad0-8c249c56e8a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":171377005},{"type":"v1","addr":"172.21.15.157:6801","nonce":171377005}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":171377005},{"type":"v1","addr":"172.21.15.157:6803","nonce":171377005}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":171377005},{"type":"v1","addr":"172.21.15.157:6807","nonce":171377005}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":171377005},{"type":"v1","addr":"172.21.15.157:6805","nonce":171377005}]},"public_addr":"172.21.15.157:6801/171377005","cluster_addr":"172.21.15.157:6803/171377005","heartbeat_back_addr":"172.21.15.157:6807/171377005","heartbeat_front_addr":"172.21.15.157:6805/171377005","state":["exists","up"]},{"osd":3,"uuid":"518aa33d-263f-45e6-aab5-2201ceb40e74","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":27,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6810","nonce":640760101},{"type":"v1","addr":"172.21.15.47:6811","nonce":640760101}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6812","nonce":640760101},{"type":"v1","addr":"172.21.15.47:6813","nonce":640760101}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6816","nonce":640760101},{"type":"v1","addr":"172.21.15.47:6817","nonce":640760101}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6814","nonce":640760101},{"type":"v1","addr":"172.21.15.47:6815","nonce":640760101}]},"public_addr":"172.21.15.47:6811/640760101","cluster_addr":"172.21.15.47:6813/640760101","heartbeat_back_addr":"172.21.15.47:6817/640760101","heartbeat_front_addr":"172.21.15.47:6815/640760101","state":["exists","up"]},{"osd":4,"uuid":"987bde83-ecc6-42e5-bd56-c143b3de4226","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":35,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6808","nonce":2785478639},{"type":"v1","addr":"172.21.15.100:6809","nonce":2785478639}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6810","nonce":2785478639},{"type":"v1","addr":"172.21.15.100:6811","nonce":2785478639}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6814","nonce":2785478639},{"type":"v1","addr":"172.21.15.100:6815","nonce":2785478639}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":2785478639},{"type":"v1","addr":"172.21.15.100:6813","nonce":2785478639}]},"public_addr":"172.21.15.100:6809/2785478639","cluster_addr":"172.21.15.100:6811/2785478639","heartbeat_back_addr":"172.21.15.100:6815/2785478639","heartbeat_front_addr":"172.21.15.100:6813/2785478639","state":["exists","up"]},{"osd":5,"uuid":"012d2188-3a61-4a5f-8d01-2001c15ffbe2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":40,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":1525645410},{"type":"v1","addr":"172.21.15.157:6809","nonce":1525645410}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":1525645410},{"type":"v1","addr":"172.21.15.157:6811","nonce":1525645410}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":1525645410},{"type":"v1","addr":"172.21.15.157:6815","nonce":1525645410}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":1525645410},{"type":"v1","addr":"172.21.15.157:6813","nonce":1525645410}]},"public_addr":"172.21.15.157:6809/1525645410","cluster_addr":"172.21.15.157:6811/1525645410","heartbeat_back_addr":"172.21.15.157:6815/1525645410","heartbeat_front_addr":"172.21.15.157:6813/1525645410","state":["exists","up"]},{"osd":6,"uuid":"c12c14b4-85f0-4d88-937c-de4f556437de","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6818","nonce":2321642790},{"type":"v1","addr":"172.21.15.47:6819","nonce":2321642790}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6820","nonce":2321642790},{"type":"v1","addr":"172.21.15.47:6821","nonce":2321642790}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6824","nonce":2321642790},{"type":"v1","addr":"172.21.15.47:6825","nonce":2321642790}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6822","nonce":2321642790},{"type":"v1","addr":"172.21.15.47:6823","nonce":2321642790}]},"public_addr":"172.21.15.47:6819/2321642790","cluster_addr":"172.21.15.47:6821/2321642790","heartbeat_back_addr":"172.21.15.47:6825/2321642790","heartbeat_front_addr":"172.21.15.47:6823/2321642790","state":["exists","up"]},{"osd":7,"uuid":"3c87c6d0-1d1f-4294-9917-aaaad7dda824","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6816","nonce":4104727330},{"type":"v1","addr":"172.21.15.100:6817","nonce":4104727330}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":4104727330},{"type":"v1","addr":"172.21.15.100:6819","nonce":4104727330}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":4104727330},{"type":"v1","addr":"172.21.15.100:6823","nonce":4104727330}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6820","nonce":4104727330},{"type":"v1","addr":"172.21.15.100:6821","nonce":4104727330}]},"public_addr":"172.21.15.100:6817/4104727330","cluster_addr":"172.21.15.100:6819/4104727330","heartbeat_back_addr":"172.21.15.100:6823/4104727330","heartbeat_front_addr":"172.21.15.100:6821/4104727330","state":["exists","up"]},{"osd":8,"uuid":"502d65cf-53c8-4936-8de4-e11b2a6e1e18","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":61,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":4067715347},{"type":"v1","addr":"172.21.15.157:6817","nonce":4067715347}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":4067715347},{"type":"v1","addr":"172.21.15.157:6819","nonce":4067715347}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":4067715347},{"type":"v1","addr":"172.21.15.157:6823","nonce":4067715347}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":4067715347},{"type":"v1","addr":"172.21.15.157:6821","nonce":4067715347}]},"public_addr":"172.21.15.157:6817/4067715347","cluster_addr":"172.21.15.157:6819/4067715347","heartbeat_back_addr":"172.21.15.157:6823/4067715347","heartbeat_front_addr":"172.21.15.157:6821/4067715347","state":["exists","up"]},{"osd":9,"uuid":"5ea1f5be-8458-4c93-9e61-c2e867a514f3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6826","nonce":1982108738},{"type":"v1","addr":"172.21.15.47:6827","nonce":1982108738}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6828","nonce":1982108738},{"type":"v1","addr":"172.21.15.47:6829","nonce":1982108738}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6832","nonce":1982108738},{"type":"v1","addr":"172.21.15.47:6833","nonce":1982108738}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6830","nonce":1982108738},{"type":"v1","addr":"172.21.15.47:6831","nonce":1982108738}]},"public_addr":"172.21.15.47:6827/1982108738","cluster_addr":"172.21.15.47:6829/1982108738","heartbeat_back_addr":"172.21.15.47:6833/1982108738","heartbeat_front_addr":"172.21.15.47:6831/1982108738","state":["exists","up"]},{"osd":10,"uuid":"2da3da90-a0be-4a54-a9ff-b2d6c9b36870","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":74,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":396391028},{"type":"v1","addr":"172.21.15.100:6825","nonce":396391028}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":396391028},{"type":"v1","addr":"172.21.15.100:6827","nonce":396391028}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6830","nonce":396391028},{"type":"v1","addr":"172.21.15.100:6831","nonce":396391028}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":396391028},{"type":"v1","addr":"172.21.15.100:6829","nonce":396391028}]},"public_addr":"172.21.15.100:6825/396391028","cluster_addr":"172.21.15.100:6827/396391028","heartbeat_back_addr":"172.21.15.100:6831/396391028","heartbeat_front_addr":"172.21.15.100:6829/396391028","state":["exists","up"]},{"osd":11,"uuid":"11798d41-7f7e-4319-95b8-8d3d0946bde9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":82,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":2671029483},{"type":"v1","addr":"172.21.15.157:6825","nonce":2671029483}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":2671029483},{"type":"v1","addr":"172.21.15.157:6827","nonce":2671029483}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":2671029483},{"type":"v1","addr":"172.21.15.157:6831","nonce":2671029483}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":2671029483},{"type":"v1","addr":"172.21.15.157:6829","nonce":2671029483}]},"public_addr":"172.21.15.157:6825/2671029483","cluster_addr":"172.21.15.157:6827/2671029483","heartbeat_back_addr":"172.21.15.157:6831/2671029483","heartbeat_front_addr":"172.21.15.157:6829/2671029483","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T10:14:31.822399+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T10:15:04.175721+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T10:15:35.902595+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T10:16:12.088891+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T10:16:46.428704+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T10:17:20.158270+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T10:17:55.915253+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T10:18:30.064641+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T10:19:04.410004+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T10:19:40.605181+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T10:20:15.636605+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T10:20:50.151365+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.47:0/356822431":"2024-03-22T10:12:25.844530+0000","172.21.15.47:0/1167490879":"2024-03-22T10:12:25.844530+0000","172.21.15.47:0/1540718138":"2024-03-22T10:12:25.844530+0000","172.21.15.47:6801/3836903972":"2024-03-22T10:12:25.844530+0000","172.21.15.47:0/235436602":"2024-03-22T10:11:50.301838+0000","172.21.15.47:0/2052815572":"2024-03-22T10:11:50.301838+0000","172.21.15.47:6800/3942658432":"2024-03-22T10:11:50.301838+0000","172.21.15.47:6801/3942658432":"2024-03-22T10:11:50.301838+0000","172.21.15.47:0/1419516257":"2024-03-22T10:11:50.301838+0000","172.21.15.47:6800/3836903972":"2024-03-22T10:12:25.844530+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-21T10:22:05.505 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:05.504+0000 7fba73e00640 1 -- 172.21.15.47:0/4078933318 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7fba580a2fa0 msgr2=0x7fba580a5460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:22:05.505 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:05.504+0000 7fba73e00640 1 --2- 172.21.15.47:0/4078933318 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7fba580a2fa0 0x7fba580a5460 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fba74104740 tx=0x7fba64005e70 comp rx=0 tx=0).stop 2024-03-21T10:22:05.505 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:05.504+0000 7fba73e00640 1 -- 172.21.15.47:0/4078933318 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fba741052a0 msgr2=0x7fba741a0e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:22:05.505 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:05.504+0000 7fba73e00640 1 --2- 172.21.15.47:0/4078933318 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fba741052a0 0x7fba741a0e70 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fba6c032520 tx=0x7fba6c034880 comp rx=0 tx=0).stop 2024-03-21T10:22:05.505 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:05.504+0000 7fba73e00640 1 -- 172.21.15.47:0/4078933318 shutdown_connections 2024-03-21T10:22:05.505 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:05.504+0000 7fba73e00640 1 --2- 172.21.15.47:0/4078933318 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7fba580a2fa0 0x7fba580a5460 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:22:05.506 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:05.504+0000 7fba73e00640 1 --2- 172.21.15.47:0/4078933318 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fba741052a0 0x7fba741a0e70 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:22:05.506 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:05.504+0000 7fba73e00640 1 --2- 172.21.15.47:0/4078933318 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fba741048e0 0x7fba74199e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:22:05.506 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:05.504+0000 7fba73e00640 1 --2- 172.21.15.47:0/4078933318 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fba741036f0 0x7fba741998c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:22:05.506 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:05.504+0000 7fba73e00640 1 -- 172.21.15.47:0/4078933318 >> 172.21.15.47:0/4078933318 conn(0x7fba740fec80 msgr2=0x7fba74107c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:22:05.506 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:05.504+0000 7fba73e00640 1 -- 172.21.15.47:0/4078933318 shutdown_connections 2024-03-21T10:22:05.506 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:05.504+0000 7fba73e00640 1 -- 172.21.15.47:0/4078933318 wait complete. 2024-03-21T10:22:05.527 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-21T10:22:05.742 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:05.740+0000 7f486ac00640 1 -- 172.21.15.47:0/1193972029 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4864076060 msgr2=0x7f4864076480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:22:05.742 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:05.740+0000 7f486ac00640 1 --2- 172.21.15.47:0/1193972029 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4864076060 0x7f4864076480 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f4854009ca0 tx=0x7f485402c100 comp rx=0 tx=0).stop 2024-03-21T10:22:05.742 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:05.740+0000 7f486ac00640 1 -- 172.21.15.47:0/1193972029 shutdown_connections 2024-03-21T10:22:05.742 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:05.740+0000 7f486ac00640 1 --2- 172.21.15.47:0/1193972029 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4864077c40 0x7f48640784e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:22:05.742 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:05.740+0000 7f486ac00640 1 --2- 172.21.15.47:0/1193972029 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f48640772a0 0x7f4864077700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:22:05.742 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:05.740+0000 7f486ac00640 1 --2- 172.21.15.47:0/1193972029 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4864076060 0x7f4864076480 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:22:05.742 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:05.740+0000 7f486ac00640 1 -- 172.21.15.47:0/1193972029 >> 172.21.15.47:0/1193972029 conn(0x7f48640fee50 msgr2=0x7f4864101290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:22:05.743 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:05.741+0000 7f486ac00640 1 -- 172.21.15.47:0/1193972029 shutdown_connections 2024-03-21T10:22:05.743 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:05.741+0000 7f486ac00640 1 -- 172.21.15.47:0/1193972029 wait complete. 2024-03-21T10:22:05.745 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:05.741+0000 7f486ac00640 1 Processor -- start 2024-03-21T10:22:05.745 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:05.742+0000 7f486ac00640 1 -- start start 2024-03-21T10:22:05.746 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:05.742+0000 7f486ac00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4864076060 0x7f486419df90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:22:05.746 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:05.742+0000 7f486ac00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f48640772a0 0x7f486419e4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:22:05.746 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:05.742+0000 7f486ac00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4864077c40 0x7f48641a5540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:22:05.746 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:05.742+0000 7f486ac00640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4864113860 con 0x7f4864077c40 2024-03-21T10:22:05.746 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:05.742+0000 7f486ac00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48641136e0 con 0x7f48640772a0 2024-03-21T10:22:05.746 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:05.742+0000 7f486ac00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48641139e0 con 0x7f4864076060 2024-03-21T10:22:05.746 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:05.742+0000 7f4868c00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4864077c40 0x7f48641a5540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:22:05.746 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:05.742+0000 7f4868c00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4864077c40 0x7f48641a5540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:47022/0 (socket says 172.21.15.47:47022) 2024-03-21T10:22:05.746 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:05.742+0000 7f4868c00640 1 -- 172.21.15.47:0/232678433 learned_addr learned my addr 172.21.15.47:0/232678433 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:22:05.747 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:05.743+0000 7f4863e00640 1 --2- 172.21.15.47:0/232678433 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4864076060 0x7f486419df90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:22:05.747 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:05.743+0000 7f485b400640 1 --2- 172.21.15.47:0/232678433 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f48640772a0 0x7f486419e4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:22:05.747 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:05.743+0000 7f4868c00640 1 -- 172.21.15.47:0/232678433 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4864076060 msgr2=0x7f486419df90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:22:05.747 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:05.743+0000 7f4868c00640 1 --2- 172.21.15.47:0/232678433 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4864076060 0x7f486419df90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:22:05.747 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:05.743+0000 7f4868c00640 1 -- 172.21.15.47:0/232678433 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f48640772a0 msgr2=0x7f486419e4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:22:05.747 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:05.743+0000 7f4868c00640 1 --2- 172.21.15.47:0/232678433 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f48640772a0 0x7f486419e4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:22:05.747 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:05.743+0000 7f4868c00640 1 -- 172.21.15.47:0/232678433 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48641a5bb0 con 0x7f4864077c40 2024-03-21T10:22:05.747 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:05.743+0000 7f4863e00640 1 --2- 172.21.15.47:0/232678433 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4864076060 0x7f486419df90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T10:22:05.747 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:05.743+0000 7f485b400640 1 --2- 172.21.15.47:0/232678433 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f48640772a0 0x7f486419e4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T10:22:05.747 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:05.743+0000 7f4868c00640 1 --2- 172.21.15.47:0/232678433 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4864077c40 0x7f48641a5540 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f485c009fd0 tx=0x7f485c00b8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:22:05.747 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:05.744+0000 7f4861600640 1 -- 172.21.15.47:0/232678433 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f485c011830 con 0x7f4864077c40 2024-03-21T10:22:05.748 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:05.744+0000 7f4861600640 1 -- 172.21.15.47:0/232678433 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f485c016070 con 0x7f4864077c40 2024-03-21T10:22:05.748 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:05.744+0000 7f486ac00640 1 -- 172.21.15.47:0/232678433 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f48641a5ea0 con 0x7f4864077c40 2024-03-21T10:22:05.748 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:05.744+0000 7f4861600640 1 -- 172.21.15.47:0/232678433 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f485c010de0 con 0x7f4864077c40 2024-03-21T10:22:05.748 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:05.744+0000 7f486ac00640 1 -- 172.21.15.47:0/232678433 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f48641a63e0 con 0x7f4864077c40 2024-03-21T10:22:05.748 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:05.745+0000 7f486ac00640 1 -- 172.21.15.47:0/232678433 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f48641104d0 con 0x7f4864077c40 2024-03-21T10:22:05.748 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:05.747+0000 7f4861600640 1 -- 172.21.15.47:0/232678433 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 16) v1 ==== 137854+0+0 (secure 0 0 0) 0x7f485c0119d0 con 0x7f4864077c40 2024-03-21T10:22:05.749 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:05.748+0000 7f4861600640 1 --2- 172.21.15.47:0/232678433 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f48480a2fa0 0x7f48480a5460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:22:05.749 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:05.748+0000 7f4861600640 1 -- 172.21.15.47:0/232678433 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 8802+0+0 (secure 0 0 0) 0x7f485c0ced70 con 0x7f4864077c40 2024-03-21T10:22:05.749 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:05.748+0000 7f4863e00640 1 --2- 172.21.15.47:0/232678433 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f48480a2fa0 0x7f48480a5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:22:05.750 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:05.749+0000 7f4863e00640 1 --2- 172.21.15.47:0/232678433 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f48480a2fa0 0x7f48480a5460 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f4854006010 tx=0x7f4854009890 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:22:05.754 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:05.753+0000 7f4861600640 1 -- 172.21.15.47:0/232678433 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f485c09b960 con 0x7f4864077c40 2024-03-21T10:22:05.964 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:05.963+0000 7f486ac00640 1 -- 172.21.15.47:0/232678433 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f4864062340 con 0x7f4864077c40 2024-03-21T10:22:05.967 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:05.966+0000 7f4861600640 1 -- 172.21.15.47:0/232678433 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v91) v1 ==== 74+0+24116 (secure 0 0 0) 0x7f485c09b260 con 0x7f4864077c40 2024-03-21T10:22:05.967 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-21T10:22:05.968 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":91,"fsid":"3056d51a-e76b-11ee-95cd-87774f69a715","created":"2024-03-21T10:11:17.019965+0000","modified":"2024-03-21T10:22:03.759214+0000","last_up_change":"2024-03-21T10:20:52.675328+0000","last_in_change":"2024-03-21T10:20:27.648212+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-21T10:15:40.553114+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-21T10:21:56.399518+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-21T10:21:57.812940+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-21T10:22:01.822568+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"91","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"b0c7a102-8d89-428b-b793-cbbe67151f50","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6802","nonce":775837313},{"type":"v1","addr":"172.21.15.47:6803","nonce":775837313}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6804","nonce":775837313},{"type":"v1","addr":"172.21.15.47:6805","nonce":775837313}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6808","nonce":775837313},{"type":"v1","addr":"172.21.15.47:6809","nonce":775837313}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6806","nonce":775837313},{"type":"v1","addr":"172.21.15.47:6807","nonce":775837313}]},"public_addr":"172.21.15.47:6803/775837313","cluster_addr":"172.21.15.47:6805/775837313","heartbeat_back_addr":"172.21.15.47:6809/775837313","heartbeat_front_addr":"172.21.15.47:6807/775837313","state":["exists","up"]},{"osd":1,"uuid":"07bb81bb-7cf3-4bbc-ac9e-802f95185d11","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6800","nonce":2923304710},{"type":"v1","addr":"172.21.15.100:6801","nonce":2923304710}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":2923304710},{"type":"v1","addr":"172.21.15.100:6803","nonce":2923304710}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":2923304710},{"type":"v1","addr":"172.21.15.100:6807","nonce":2923304710}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6804","nonce":2923304710},{"type":"v1","addr":"172.21.15.100:6805","nonce":2923304710}]},"public_addr":"172.21.15.100:6801/2923304710","cluster_addr":"172.21.15.100:6803/2923304710","heartbeat_back_addr":"172.21.15.100:6807/2923304710","heartbeat_front_addr":"172.21.15.100:6805/2923304710","state":["exists","up"]},{"osd":2,"uuid":"88ce76b9-6081-4836-aad0-8c249c56e8a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":171377005},{"type":"v1","addr":"172.21.15.157:6801","nonce":171377005}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":171377005},{"type":"v1","addr":"172.21.15.157:6803","nonce":171377005}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":171377005},{"type":"v1","addr":"172.21.15.157:6807","nonce":171377005}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":171377005},{"type":"v1","addr":"172.21.15.157:6805","nonce":171377005}]},"public_addr":"172.21.15.157:6801/171377005","cluster_addr":"172.21.15.157:6803/171377005","heartbeat_back_addr":"172.21.15.157:6807/171377005","heartbeat_front_addr":"172.21.15.157:6805/171377005","state":["exists","up"]},{"osd":3,"uuid":"518aa33d-263f-45e6-aab5-2201ceb40e74","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":27,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6810","nonce":640760101},{"type":"v1","addr":"172.21.15.47:6811","nonce":640760101}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6812","nonce":640760101},{"type":"v1","addr":"172.21.15.47:6813","nonce":640760101}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6816","nonce":640760101},{"type":"v1","addr":"172.21.15.47:6817","nonce":640760101}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6814","nonce":640760101},{"type":"v1","addr":"172.21.15.47:6815","nonce":640760101}]},"public_addr":"172.21.15.47:6811/640760101","cluster_addr":"172.21.15.47:6813/640760101","heartbeat_back_addr":"172.21.15.47:6817/640760101","heartbeat_front_addr":"172.21.15.47:6815/640760101","state":["exists","up"]},{"osd":4,"uuid":"987bde83-ecc6-42e5-bd56-c143b3de4226","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":35,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6808","nonce":2785478639},{"type":"v1","addr":"172.21.15.100:6809","nonce":2785478639}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6810","nonce":2785478639},{"type":"v1","addr":"172.21.15.100:6811","nonce":2785478639}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6814","nonce":2785478639},{"type":"v1","addr":"172.21.15.100:6815","nonce":2785478639}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":2785478639},{"type":"v1","addr":"172.21.15.100:6813","nonce":2785478639}]},"public_addr":"172.21.15.100:6809/2785478639","cluster_addr":"172.21.15.100:6811/2785478639","heartbeat_back_addr":"172.21.15.100:6815/2785478639","heartbeat_front_addr":"172.21.15.100:6813/2785478639","state":["exists","up"]},{"osd":5,"uuid":"012d2188-3a61-4a5f-8d01-2001c15ffbe2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":40,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":1525645410},{"type":"v1","addr":"172.21.15.157:6809","nonce":1525645410}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":1525645410},{"type":"v1","addr":"172.21.15.157:6811","nonce":1525645410}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":1525645410},{"type":"v1","addr":"172.21.15.157:6815","nonce":1525645410}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":1525645410},{"type":"v1","addr":"172.21.15.157:6813","nonce":1525645410}]},"public_addr":"172.21.15.157:6809/1525645410","cluster_addr":"172.21.15.157:6811/1525645410","heartbeat_back_addr":"172.21.15.157:6815/1525645410","heartbeat_front_addr":"172.21.15.157:6813/1525645410","state":["exists","up"]},{"osd":6,"uuid":"c12c14b4-85f0-4d88-937c-de4f556437de","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6818","nonce":2321642790},{"type":"v1","addr":"172.21.15.47:6819","nonce":2321642790}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6820","nonce":2321642790},{"type":"v1","addr":"172.21.15.47:6821","nonce":2321642790}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6824","nonce":2321642790},{"type":"v1","addr":"172.21.15.47:6825","nonce":2321642790}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6822","nonce":2321642790},{"type":"v1","addr":"172.21.15.47:6823","nonce":2321642790}]},"public_addr":"172.21.15.47:6819/2321642790","cluster_addr":"172.21.15.47:6821/2321642790","heartbeat_back_addr":"172.21.15.47:6825/2321642790","heartbeat_front_addr":"172.21.15.47:6823/2321642790","state":["exists","up"]},{"osd":7,"uuid":"3c87c6d0-1d1f-4294-9917-aaaad7dda824","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6816","nonce":4104727330},{"type":"v1","addr":"172.21.15.100:6817","nonce":4104727330}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":4104727330},{"type":"v1","addr":"172.21.15.100:6819","nonce":4104727330}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":4104727330},{"type":"v1","addr":"172.21.15.100:6823","nonce":4104727330}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6820","nonce":4104727330},{"type":"v1","addr":"172.21.15.100:6821","nonce":4104727330}]},"public_addr":"172.21.15.100:6817/4104727330","cluster_addr":"172.21.15.100:6819/4104727330","heartbeat_back_addr":"172.21.15.100:6823/4104727330","heartbeat_front_addr":"172.21.15.100:6821/4104727330","state":["exists","up"]},{"osd":8,"uuid":"502d65cf-53c8-4936-8de4-e11b2a6e1e18","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":61,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":4067715347},{"type":"v1","addr":"172.21.15.157:6817","nonce":4067715347}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":4067715347},{"type":"v1","addr":"172.21.15.157:6819","nonce":4067715347}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":4067715347},{"type":"v1","addr":"172.21.15.157:6823","nonce":4067715347}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":4067715347},{"type":"v1","addr":"172.21.15.157:6821","nonce":4067715347}]},"public_addr":"172.21.15.157:6817/4067715347","cluster_addr":"172.21.15.157:6819/4067715347","heartbeat_back_addr":"172.21.15.157:6823/4067715347","heartbeat_front_addr":"172.21.15.157:6821/4067715347","state":["exists","up"]},{"osd":9,"uuid":"5ea1f5be-8458-4c93-9e61-c2e867a514f3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6826","nonce":1982108738},{"type":"v1","addr":"172.21.15.47:6827","nonce":1982108738}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6828","nonce":1982108738},{"type":"v1","addr":"172.21.15.47:6829","nonce":1982108738}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6832","nonce":1982108738},{"type":"v1","addr":"172.21.15.47:6833","nonce":1982108738}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6830","nonce":1982108738},{"type":"v1","addr":"172.21.15.47:6831","nonce":1982108738}]},"public_addr":"172.21.15.47:6827/1982108738","cluster_addr":"172.21.15.47:6829/1982108738","heartbeat_back_addr":"172.21.15.47:6833/1982108738","heartbeat_front_addr":"172.21.15.47:6831/1982108738","state":["exists","up"]},{"osd":10,"uuid":"2da3da90-a0be-4a54-a9ff-b2d6c9b36870","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":74,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":396391028},{"type":"v1","addr":"172.21.15.100:6825","nonce":396391028}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":396391028},{"type":"v1","addr":"172.21.15.100:6827","nonce":396391028}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6830","nonce":396391028},{"type":"v1","addr":"172.21.15.100:6831","nonce":396391028}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":396391028},{"type":"v1","addr":"172.21.15.100:6829","nonce":396391028}]},"public_addr":"172.21.15.100:6825/396391028","cluster_addr":"172.21.15.100:6827/396391028","heartbeat_back_addr":"172.21.15.100:6831/396391028","heartbeat_front_addr":"172.21.15.100:6829/396391028","state":["exists","up"]},{"osd":11,"uuid":"11798d41-7f7e-4319-95b8-8d3d0946bde9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":82,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":2671029483},{"type":"v1","addr":"172.21.15.157:6825","nonce":2671029483}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":2671029483},{"type":"v1","addr":"172.21.15.157:6827","nonce":2671029483}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":2671029483},{"type":"v1","addr":"172.21.15.157:6831","nonce":2671029483}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":2671029483},{"type":"v1","addr":"172.21.15.157:6829","nonce":2671029483}]},"public_addr":"172.21.15.157:6825/2671029483","cluster_addr":"172.21.15.157:6827/2671029483","heartbeat_back_addr":"172.21.15.157:6831/2671029483","heartbeat_front_addr":"172.21.15.157:6829/2671029483","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T10:14:31.822399+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T10:15:04.175721+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T10:15:35.902595+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T10:16:12.088891+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T10:16:46.428704+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T10:17:20.158270+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T10:17:55.915253+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T10:18:30.064641+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T10:19:04.410004+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T10:19:40.605181+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T10:20:15.636605+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T10:20:50.151365+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.47:0/356822431":"2024-03-22T10:12:25.844530+0000","172.21.15.47:0/1167490879":"2024-03-22T10:12:25.844530+0000","172.21.15.47:0/1540718138":"2024-03-22T10:12:25.844530+0000","172.21.15.47:6801/3836903972":"2024-03-22T10:12:25.844530+0000","172.21.15.47:0/235436602":"2024-03-22T10:11:50.301838+0000","172.21.15.47:0/2052815572":"2024-03-22T10:11:50.301838+0000","172.21.15.47:6800/3942658432":"2024-03-22T10:11:50.301838+0000","172.21.15.47:6801/3942658432":"2024-03-22T10:11:50.301838+0000","172.21.15.47:0/1419516257":"2024-03-22T10:11:50.301838+0000","172.21.15.47:6800/3836903972":"2024-03-22T10:12:25.844530+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-21T10:22:05.969 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:05.968+0000 7f486ac00640 1 -- 172.21.15.47:0/232678433 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f48480a2fa0 msgr2=0x7f48480a5460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:22:05.969 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:05.968+0000 7f486ac00640 1 --2- 172.21.15.47:0/232678433 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f48480a2fa0 0x7f48480a5460 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f4854006010 tx=0x7f4854009890 comp rx=0 tx=0).stop 2024-03-21T10:22:05.969 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:05.968+0000 7f486ac00640 1 -- 172.21.15.47:0/232678433 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4864077c40 msgr2=0x7f48641a5540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:22:05.969 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:05.968+0000 7f486ac00640 1 --2- 172.21.15.47:0/232678433 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4864077c40 0x7f48641a5540 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f485c009fd0 tx=0x7f485c00b8e0 comp rx=0 tx=0).stop 2024-03-21T10:22:05.970 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:05.968+0000 7f486ac00640 1 -- 172.21.15.47:0/232678433 shutdown_connections 2024-03-21T10:22:05.970 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:05.968+0000 7f486ac00640 1 --2- 172.21.15.47:0/232678433 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f48480a2fa0 0x7f48480a5460 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:22:05.970 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:05.968+0000 7f486ac00640 1 --2- 172.21.15.47:0/232678433 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4864077c40 0x7f48641a5540 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:22:05.970 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:05.968+0000 7f486ac00640 1 --2- 172.21.15.47:0/232678433 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f48640772a0 0x7f486419e4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:22:05.970 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:05.968+0000 7f486ac00640 1 --2- 172.21.15.47:0/232678433 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4864076060 0x7f486419df90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:22:05.970 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:05.969+0000 7f486ac00640 1 -- 172.21.15.47:0/232678433 >> 172.21.15.47:0/232678433 conn(0x7f48640fee50 msgr2=0x7f4864100640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:22:05.971 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:05.969+0000 7f486ac00640 1 -- 172.21.15.47:0/232678433 shutdown_connections 2024-03-21T10:22:05.971 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:05.969+0000 7f486ac00640 1 -- 172.21.15.47:0/232678433 wait complete. 2024-03-21T10:22:06.006 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-21T10:22:06.214 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:22:05 smithi047 ceph-mon[31547]: pgmap v370: 129 pgs: 104 active+clean, 12 creating+peering, 13 unknown; 579 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s wr, 10 op/s 2024-03-21T10:22:06.214 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:22:05 smithi047 ceph-mon[31547]: from='client.? 172.21.15.47:0/3478741258' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-21T10:22:06.214 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:22:05 smithi047 ceph-mon[31547]: from='client.? 172.21.15.47:0/4078933318' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-21T10:22:06.228 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:06.227+0000 7f420f200640 1 -- 172.21.15.47:0/2587208966 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4208105490 msgr2=0x7f420810bd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:22:06.228 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:06.227+0000 7f420f200640 1 --2- 172.21.15.47:0/2587208966 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4208105490 0x7f420810bd20 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f4200009ca0 tx=0x7f420002c0f0 comp rx=0 tx=0).stop 2024-03-21T10:22:06.229 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:06.227+0000 7f420f200640 1 -- 172.21.15.47:0/2587208966 shutdown_connections 2024-03-21T10:22:06.229 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:06.227+0000 7f420f200640 1 --2- 172.21.15.47:0/2587208966 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4208105490 0x7f420810bd20 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:22:06.229 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:06.227+0000 7f420f200640 1 --2- 172.21.15.47:0/2587208966 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4208104ad0 0x7f4208104f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:22:06.229 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:06.227+0000 7f420f200640 1 --2- 172.21.15.47:0/2587208966 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f42081038e0 0x7f4208103d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:22:06.229 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:06.227+0000 7f420f200640 1 -- 172.21.15.47:0/2587208966 >> 172.21.15.47:0/2587208966 conn(0x7f42080fee50 msgr2=0x7f4208101290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:22:06.229 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:06.227+0000 7f420f200640 1 -- 172.21.15.47:0/2587208966 shutdown_connections 2024-03-21T10:22:06.229 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:06.227+0000 7f420f200640 1 -- 172.21.15.47:0/2587208966 wait complete. 2024-03-21T10:22:06.230 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:22:05 smithi100 ceph-mon[35193]: pgmap v370: 129 pgs: 104 active+clean, 12 creating+peering, 13 unknown; 579 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s wr, 10 op/s 2024-03-21T10:22:06.230 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:22:05 smithi100 ceph-mon[35193]: from='client.? 172.21.15.47:0/3478741258' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-21T10:22:06.230 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:22:05 smithi100 ceph-mon[35193]: from='client.? 172.21.15.47:0/4078933318' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-21T10:22:06.232 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:06.228+0000 7f420f200640 1 Processor -- start 2024-03-21T10:22:06.233 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:06.228+0000 7f420f200640 1 -- start start 2024-03-21T10:22:06.233 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:06.228+0000 7f420f200640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f42081038e0 0x7f4208199a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:22:06.233 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:06.229+0000 7f420f200640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4208104ad0 0x7f4208199f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:22:06.233 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:06.229+0000 7f420f200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4208105490 0x7f42081a0fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:22:06.233 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:06.229+0000 7f420f200640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f420810f3f0 con 0x7f42081038e0 2024-03-21T10:22:06.233 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:06.229+0000 7f420f200640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f420810f270 con 0x7f4208105490 2024-03-21T10:22:06.233 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:06.229+0000 7f420f200640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f420810f570 con 0x7f4208104ad0 2024-03-21T10:22:06.233 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:06.229+0000 7f4207e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f42081038e0 0x7f4208199a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:22:06.233 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:06.229+0000 7f4207e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f42081038e0 0x7f4208199a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:47038/0 (socket says 172.21.15.47:47038) 2024-03-21T10:22:06.233 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:06.229+0000 7f4207e00640 1 -- 172.21.15.47:0/2305308242 learned_addr learned my addr 172.21.15.47:0/2305308242 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:22:06.233 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:06.229+0000 7f420d200640 1 --2- 172.21.15.47:0/2305308242 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4208105490 0x7f42081a0fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:22:06.233 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:06.229+0000 7f4207e00640 1 -- 172.21.15.47:0/2305308242 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4208104ad0 msgr2=0x7f4208199f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:22:06.233 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:06.229+0000 7f41ff400640 1 --2- 172.21.15.47:0/2305308242 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4208104ad0 0x7f4208199f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:22:06.234 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:06.229+0000 7f4207e00640 1 --2- 172.21.15.47:0/2305308242 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4208104ad0 0x7f4208199f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:22:06.234 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:06.229+0000 7f4207e00640 1 -- 172.21.15.47:0/2305308242 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4208105490 msgr2=0x7f42081a0fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:22:06.234 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:06.229+0000 7f4207e00640 1 --2- 172.21.15.47:0/2305308242 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4208105490 0x7f42081a0fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:22:06.234 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:06.229+0000 7f4207e00640 1 -- 172.21.15.47:0/2305308242 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f42081a1630 con 0x7f42081038e0 2024-03-21T10:22:06.234 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:06.229+0000 7f420d200640 1 --2- 172.21.15.47:0/2305308242 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4208105490 0x7f42081a0fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T10:22:06.234 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:06.230+0000 7f4207e00640 1 --2- 172.21.15.47:0/2305308242 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f42081038e0 0x7f4208199a10 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f41f800c970 tx=0x7f41f800ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:22:06.234 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:06.230+0000 7f4205600640 1 -- 172.21.15.47:0/2305308242 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41f8007c10 con 0x7f42081038e0 2024-03-21T10:22:06.234 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:06.230+0000 7f4205600640 1 -- 172.21.15.47:0/2305308242 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f41f8007db0 con 0x7f42081038e0 2024-03-21T10:22:06.234 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:06.230+0000 7f4205600640 1 -- 172.21.15.47:0/2305308242 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41f800f870 con 0x7f42081038e0 2024-03-21T10:22:06.234 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:06.230+0000 7f420f200640 1 -- 172.21.15.47:0/2305308242 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f42081a1920 con 0x7f42081038e0 2024-03-21T10:22:06.234 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:06.230+0000 7f420f200640 1 -- 172.21.15.47:0/2305308242 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f42081a1e20 con 0x7f42081038e0 2024-03-21T10:22:06.234 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:06.231+0000 7f420f200640 1 -- 172.21.15.47:0/2305308242 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f420810ac00 con 0x7f42081038e0 2024-03-21T10:22:06.237 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:06.233+0000 7f4205600640 1 -- 172.21.15.47:0/2305308242 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 16) v1 ==== 137854+0+0 (secure 0 0 0) 0x7f41f800fa10 con 0x7f42081038e0 2024-03-21T10:22:06.238 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:06.234+0000 7f4205600640 1 --2- 172.21.15.47:0/2305308242 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f41ec0a2fa0 0x7f41ec0a5460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:22:06.238 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:06.235+0000 7f4205600640 1 -- 172.21.15.47:0/2305308242 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 8802+0+0 (secure 0 0 0) 0x7f41f80cfa80 con 0x7f42081038e0 2024-03-21T10:22:06.238 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:06.235+0000 7f41ff400640 1 --2- 172.21.15.47:0/2305308242 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f41ec0a2fa0 0x7f41ec0a5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:22:06.238 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:06.236+0000 7f41ff400640 1 --2- 172.21.15.47:0/2305308242 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f41ec0a2fa0 0x7f41ec0a5460 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f420819af20 tx=0x7f41f0005c60 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:22:06.240 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:22:05 smithi157 ceph-mon[35581]: pgmap v370: 129 pgs: 104 active+clean, 12 creating+peering, 13 unknown; 579 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s wr, 10 op/s 2024-03-21T10:22:06.240 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:22:05 smithi157 ceph-mon[35581]: from='client.? 172.21.15.47:0/3478741258' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-21T10:22:06.240 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:22:05 smithi157 ceph-mon[35581]: from='client.? 172.21.15.47:0/4078933318' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-21T10:22:06.240 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:06.239+0000 7f4205600640 1 -- 172.21.15.47:0/2305308242 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f41f80d1e20 con 0x7f42081038e0 2024-03-21T10:22:06.487 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:06.485+0000 7f420f200640 1 -- 172.21.15.47:0/2305308242 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f420802d320 con 0x7f42081038e0 2024-03-21T10:22:06.488 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:06.487+0000 7f4205600640 1 -- 172.21.15.47:0/2305308242 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) v1 ==== 94+0+11336 (secure 0 0 0) 0x7f41f809c6a0 con 0x7f42081038e0 2024-03-21T10:22:06.488 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-21T10:22:06.488 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":17,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14511,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6835/735724037","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6834","nonce":735724037},{"type":"v1","addr":"172.21.15.47:6835","nonce":735724037}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14529,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6837/2248191724","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6836","nonce":2248191724},{"type":"v1","addr":"172.21.15.47:6837","nonce":2248191724}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14604,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6839/2881335356","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6838","nonce":2881335356},{"type":"v1","addr":"172.21.15.47:6839","nonce":2881335356}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":14619,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6841/3401578978","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6840","nonce":3401578978},{"type":"v1","addr":"172.21.15.47:6841","nonce":3401578978}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":24406,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6833/4000010136","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":4000010136},{"type":"v1","addr":"172.21.15.157:6833","nonce":4000010136}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24457,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6835/3902868651","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":3902868651},{"type":"v1","addr":"172.21.15.157:6835","nonce":3902868651}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":7},{"gid":24475,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6837/1908087204","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6836","nonce":1908087204},{"type":"v1","addr":"172.21.15.157:6837","nonce":1908087204}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":8},{"gid":24499,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6839/4064242876","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6838","nonce":4064242876},{"type":"v1","addr":"172.21.15.157:6839","nonce":4064242876}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":10},{"gid":34427,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6833/2723302038","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6832","nonce":2723302038},{"type":"v1","addr":"172.21.15.100:6833","nonce":2723302038}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":34454,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6835/2597849957","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":2597849957},{"type":"v1","addr":"172.21.15.100:6835","nonce":2597849957}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6},{"gid":34493,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6837/3497644162","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6836","nonce":3497644162},{"type":"v1","addr":"172.21.15.100:6837","nonce":3497644162}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":9}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-21T10:21:58.329063+0000","modified":"2024-03-21T10:22:03.759800+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":34535},"failed":[],"damaged":[],"stopped":[],"info":{"gid_34535":{"gid":34535,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.100:6839/4173168440","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6838","nonce":4173168440},{"type":"v1","addr":"172.21.15.100:6839","nonce":4173168440}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":34535,"qdb_cluster":[34535]},"id":1}]} 2024-03-21T10:22:06.489 INFO:teuthology.orchestra.run.smithi047.stderr:dumped fsmap epoch 17 2024-03-21T10:22:06.491 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:06.489+0000 7f420f200640 1 -- 172.21.15.47:0/2305308242 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f41ec0a2fa0 msgr2=0x7f41ec0a5460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:22:06.491 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:06.489+0000 7f420f200640 1 --2- 172.21.15.47:0/2305308242 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f41ec0a2fa0 0x7f41ec0a5460 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f420819af20 tx=0x7f41f0005c60 comp rx=0 tx=0).stop 2024-03-21T10:22:06.491 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:06.490+0000 7f420f200640 1 -- 172.21.15.47:0/2305308242 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f42081038e0 msgr2=0x7f4208199a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:22:06.491 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:06.490+0000 7f420f200640 1 --2- 172.21.15.47:0/2305308242 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f42081038e0 0x7f4208199a10 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f41f800c970 tx=0x7f41f800ce30 comp rx=0 tx=0).stop 2024-03-21T10:22:06.491 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:06.490+0000 7f420f200640 1 -- 172.21.15.47:0/2305308242 shutdown_connections 2024-03-21T10:22:06.491 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:06.490+0000 7f420f200640 1 --2- 172.21.15.47:0/2305308242 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f41ec0a2fa0 0x7f41ec0a5460 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:22:06.491 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:06.490+0000 7f420f200640 1 --2- 172.21.15.47:0/2305308242 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4208105490 0x7f42081a0fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:22:06.492 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:06.490+0000 7f420f200640 1 --2- 172.21.15.47:0/2305308242 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4208104ad0 0x7f4208199f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:22:06.492 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:06.490+0000 7f420f200640 1 --2- 172.21.15.47:0/2305308242 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f42081038e0 0x7f4208199a10 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:22:06.492 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:06.490+0000 7f420f200640 1 -- 172.21.15.47:0/2305308242 >> 172.21.15.47:0/2305308242 conn(0x7f42080fee50 msgr2=0x7f4208107dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:22:06.492 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:06.490+0000 7f420f200640 1 -- 172.21.15.47:0/2305308242 shutdown_connections 2024-03-21T10:22:06.492 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:06.490+0000 7f420f200640 1 -- 172.21.15.47:0/2305308242 wait complete. 2024-03-21T10:22:06.511 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-21T10:22:06.739 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:06.738+0000 7f25e8c00640 1 -- 172.21.15.47:0/3417821543 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f25e4077c40 msgr2=0x7f25e40784e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:22:06.739 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:06.738+0000 7f25e8c00640 1 --2- 172.21.15.47:0/3417821543 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f25e4077c40 0x7f25e40784e0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f25dc009900 tx=0x7f25dc02bc30 comp rx=0 tx=0).stop 2024-03-21T10:22:06.739 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:06.738+0000 7f25e8c00640 1 -- 172.21.15.47:0/3417821543 shutdown_connections 2024-03-21T10:22:06.739 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:06.738+0000 7f25e8c00640 1 --2- 172.21.15.47:0/3417821543 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f25e4077c40 0x7f25e40784e0 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:22:06.740 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:06.738+0000 7f25e8c00640 1 --2- 172.21.15.47:0/3417821543 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f25e40772a0 0x7f25e4077700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:22:06.740 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:06.738+0000 7f25e8c00640 1 --2- 172.21.15.47:0/3417821543 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f25e4076060 0x7f25e4076480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:22:06.740 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:06.738+0000 7f25e8c00640 1 -- 172.21.15.47:0/3417821543 >> 172.21.15.47:0/3417821543 conn(0x7f25e40fee50 msgr2=0x7f25e4101290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:22:06.740 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:06.738+0000 7f25e8c00640 1 -- 172.21.15.47:0/3417821543 shutdown_connections 2024-03-21T10:22:06.740 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:06.738+0000 7f25e8c00640 1 -- 172.21.15.47:0/3417821543 wait complete. 2024-03-21T10:22:06.742 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:06.739+0000 7f25e8c00640 1 Processor -- start 2024-03-21T10:22:06.742 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:06.739+0000 7f25e8c00640 1 -- start start 2024-03-21T10:22:06.743 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:06.739+0000 7f25e8c00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f25e4076060 0x7f25e419de60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:22:06.743 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:06.739+0000 7f25e8c00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f25e40772a0 0x7f25e419e3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:22:06.743 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:06.739+0000 7f25e8c00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f25e4077c40 0x7f25e41a5410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:22:06.743 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:06.739+0000 7f25e8c00640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25e4113860 con 0x7f25e4077c40 2024-03-21T10:22:06.743 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:06.739+0000 7f25e8c00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25e41136e0 con 0x7f25e4076060 2024-03-21T10:22:06.743 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:06.739+0000 7f25e8c00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25e41139e0 con 0x7f25e40772a0 2024-03-21T10:22:06.743 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:06.740+0000 7f25e2800640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f25e4077c40 0x7f25e41a5410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:22:06.743 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:06.740+0000 7f25e2800640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f25e4077c40 0x7f25e41a5410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:47054/0 (socket says 172.21.15.47:47054) 2024-03-21T10:22:06.743 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:06.740+0000 7f25e2800640 1 -- 172.21.15.47:0/2089858526 learned_addr learned my addr 172.21.15.47:0/2089858526 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:22:06.743 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:06.740+0000 7f25e1e00640 1 --2- 172.21.15.47:0/2089858526 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f25e4076060 0x7f25e419de60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:22:06.744 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:06.740+0000 7f25e2800640 1 -- 172.21.15.47:0/2089858526 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f25e40772a0 msgr2=0x7f25e419e3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:22:06.744 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:06.740+0000 7f25e2800640 1 --2- 172.21.15.47:0/2089858526 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f25e40772a0 0x7f25e419e3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:22:06.744 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:06.740+0000 7f25e2800640 1 -- 172.21.15.47:0/2089858526 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f25e4076060 msgr2=0x7f25e419de60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:22:06.744 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:06.740+0000 7f25e2800640 1 --2- 172.21.15.47:0/2089858526 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f25e4076060 0x7f25e419de60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:22:06.744 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:06.740+0000 7f25e2800640 1 -- 172.21.15.47:0/2089858526 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25e41a5a80 con 0x7f25e4077c40 2024-03-21T10:22:06.744 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:06.740+0000 7f25e1e00640 1 --2- 172.21.15.47:0/2089858526 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f25e4076060 0x7f25e419de60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T10:22:06.744 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:06.740+0000 7f25e2800640 1 --2- 172.21.15.47:0/2089858526 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f25e4077c40 0x7f25e41a5410 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f25dc036040 tx=0x7f25dc02cdd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:22:06.745 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:06.741+0000 7f25db400640 1 -- 172.21.15.47:0/2089858526 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25dc043020 con 0x7f25e4077c40 2024-03-21T10:22:06.745 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:06.741+0000 7f25db400640 1 -- 172.21.15.47:0/2089858526 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f25dc0044c0 con 0x7f25e4077c40 2024-03-21T10:22:06.745 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:06.741+0000 7f25db400640 1 -- 172.21.15.47:0/2089858526 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25dc003bf0 con 0x7f25e4077c40 2024-03-21T10:22:06.745 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:06.741+0000 7f25e8c00640 1 -- 172.21.15.47:0/2089858526 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f25e41a5d10 con 0x7f25e4077c40 2024-03-21T10:22:06.745 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:06.741+0000 7f25e8c00640 1 -- 172.21.15.47:0/2089858526 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f25e41a6210 con 0x7f25e4077c40 2024-03-21T10:22:06.745 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:06.742+0000 7f25e8c00640 1 -- 172.21.15.47:0/2089858526 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f25e41103b0 con 0x7f25e4077c40 2024-03-21T10:22:06.745 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:06.744+0000 7f25db400640 1 -- 172.21.15.47:0/2089858526 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 16) v1 ==== 137854+0+0 (secure 0 0 0) 0x7f25dc046020 con 0x7f25e4077c40 2024-03-21T10:22:06.746 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:06.745+0000 7f25db400640 1 --2- 172.21.15.47:0/2089858526 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f25c80a2fa0 0x7f25c80a5460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:22:06.747 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:06.746+0000 7f25db400640 1 -- 172.21.15.47:0/2089858526 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 8802+0+0 (secure 0 0 0) 0x7f25dc0f19a0 con 0x7f25e4077c40 2024-03-21T10:22:06.747 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:06.746+0000 7f25e1e00640 1 --2- 172.21.15.47:0/2089858526 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f25c80a2fa0 0x7f25c80a5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:22:06.747 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:06.746+0000 7f25e1e00640 1 --2- 172.21.15.47:0/2089858526 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f25c80a2fa0 0x7f25c80a5460 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f25d4005fd0 tx=0x7f25d4009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:22:06.750 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:06.750+0000 7f25db400640 1 -- 172.21.15.47:0/2089858526 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f25dc04d020 con 0x7f25e4077c40 2024-03-21T10:22:06.965 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:06.964+0000 7f25e8c00640 1 -- 172.21.15.47:0/2089858526 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f25e40623a0 con 0x7f25e4077c40 2024-03-21T10:22:06.967 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:06.966+0000 7f25db400640 1 -- 172.21.15.47:0/2089858526 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v91) v1 ==== 74+0+24116 (secure 0 0 0) 0x7f25dc03c350 con 0x7f25e4077c40 2024-03-21T10:22:06.968 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-21T10:22:06.968 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":91,"fsid":"3056d51a-e76b-11ee-95cd-87774f69a715","created":"2024-03-21T10:11:17.019965+0000","modified":"2024-03-21T10:22:03.759214+0000","last_up_change":"2024-03-21T10:20:52.675328+0000","last_in_change":"2024-03-21T10:20:27.648212+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-21T10:15:40.553114+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-21T10:21:56.399518+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-21T10:21:57.812940+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-21T10:22:01.822568+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"91","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"b0c7a102-8d89-428b-b793-cbbe67151f50","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6802","nonce":775837313},{"type":"v1","addr":"172.21.15.47:6803","nonce":775837313}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6804","nonce":775837313},{"type":"v1","addr":"172.21.15.47:6805","nonce":775837313}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6808","nonce":775837313},{"type":"v1","addr":"172.21.15.47:6809","nonce":775837313}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6806","nonce":775837313},{"type":"v1","addr":"172.21.15.47:6807","nonce":775837313}]},"public_addr":"172.21.15.47:6803/775837313","cluster_addr":"172.21.15.47:6805/775837313","heartbeat_back_addr":"172.21.15.47:6809/775837313","heartbeat_front_addr":"172.21.15.47:6807/775837313","state":["exists","up"]},{"osd":1,"uuid":"07bb81bb-7cf3-4bbc-ac9e-802f95185d11","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6800","nonce":2923304710},{"type":"v1","addr":"172.21.15.100:6801","nonce":2923304710}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":2923304710},{"type":"v1","addr":"172.21.15.100:6803","nonce":2923304710}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":2923304710},{"type":"v1","addr":"172.21.15.100:6807","nonce":2923304710}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6804","nonce":2923304710},{"type":"v1","addr":"172.21.15.100:6805","nonce":2923304710}]},"public_addr":"172.21.15.100:6801/2923304710","cluster_addr":"172.21.15.100:6803/2923304710","heartbeat_back_addr":"172.21.15.100:6807/2923304710","heartbeat_front_addr":"172.21.15.100:6805/2923304710","state":["exists","up"]},{"osd":2,"uuid":"88ce76b9-6081-4836-aad0-8c249c56e8a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":171377005},{"type":"v1","addr":"172.21.15.157:6801","nonce":171377005}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":171377005},{"type":"v1","addr":"172.21.15.157:6803","nonce":171377005}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":171377005},{"type":"v1","addr":"172.21.15.157:6807","nonce":171377005}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":171377005},{"type":"v1","addr":"172.21.15.157:6805","nonce":171377005}]},"public_addr":"172.21.15.157:6801/171377005","cluster_addr":"172.21.15.157:6803/171377005","heartbeat_back_addr":"172.21.15.157:6807/171377005","heartbeat_front_addr":"172.21.15.157:6805/171377005","state":["exists","up"]},{"osd":3,"uuid":"518aa33d-263f-45e6-aab5-2201ceb40e74","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":27,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6810","nonce":640760101},{"type":"v1","addr":"172.21.15.47:6811","nonce":640760101}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6812","nonce":640760101},{"type":"v1","addr":"172.21.15.47:6813","nonce":640760101}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6816","nonce":640760101},{"type":"v1","addr":"172.21.15.47:6817","nonce":640760101}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6814","nonce":640760101},{"type":"v1","addr":"172.21.15.47:6815","nonce":640760101}]},"public_addr":"172.21.15.47:6811/640760101","cluster_addr":"172.21.15.47:6813/640760101","heartbeat_back_addr":"172.21.15.47:6817/640760101","heartbeat_front_addr":"172.21.15.47:6815/640760101","state":["exists","up"]},{"osd":4,"uuid":"987bde83-ecc6-42e5-bd56-c143b3de4226","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":35,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6808","nonce":2785478639},{"type":"v1","addr":"172.21.15.100:6809","nonce":2785478639}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6810","nonce":2785478639},{"type":"v1","addr":"172.21.15.100:6811","nonce":2785478639}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6814","nonce":2785478639},{"type":"v1","addr":"172.21.15.100:6815","nonce":2785478639}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":2785478639},{"type":"v1","addr":"172.21.15.100:6813","nonce":2785478639}]},"public_addr":"172.21.15.100:6809/2785478639","cluster_addr":"172.21.15.100:6811/2785478639","heartbeat_back_addr":"172.21.15.100:6815/2785478639","heartbeat_front_addr":"172.21.15.100:6813/2785478639","state":["exists","up"]},{"osd":5,"uuid":"012d2188-3a61-4a5f-8d01-2001c15ffbe2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":40,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":1525645410},{"type":"v1","addr":"172.21.15.157:6809","nonce":1525645410}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":1525645410},{"type":"v1","addr":"172.21.15.157:6811","nonce":1525645410}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":1525645410},{"type":"v1","addr":"172.21.15.157:6815","nonce":1525645410}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":1525645410},{"type":"v1","addr":"172.21.15.157:6813","nonce":1525645410}]},"public_addr":"172.21.15.157:6809/1525645410","cluster_addr":"172.21.15.157:6811/1525645410","heartbeat_back_addr":"172.21.15.157:6815/1525645410","heartbeat_front_addr":"172.21.15.157:6813/1525645410","state":["exists","up"]},{"osd":6,"uuid":"c12c14b4-85f0-4d88-937c-de4f556437de","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6818","nonce":2321642790},{"type":"v1","addr":"172.21.15.47:6819","nonce":2321642790}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6820","nonce":2321642790},{"type":"v1","addr":"172.21.15.47:6821","nonce":2321642790}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6824","nonce":2321642790},{"type":"v1","addr":"172.21.15.47:6825","nonce":2321642790}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6822","nonce":2321642790},{"type":"v1","addr":"172.21.15.47:6823","nonce":2321642790}]},"public_addr":"172.21.15.47:6819/2321642790","cluster_addr":"172.21.15.47:6821/2321642790","heartbeat_back_addr":"172.21.15.47:6825/2321642790","heartbeat_front_addr":"172.21.15.47:6823/2321642790","state":["exists","up"]},{"osd":7,"uuid":"3c87c6d0-1d1f-4294-9917-aaaad7dda824","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6816","nonce":4104727330},{"type":"v1","addr":"172.21.15.100:6817","nonce":4104727330}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":4104727330},{"type":"v1","addr":"172.21.15.100:6819","nonce":4104727330}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":4104727330},{"type":"v1","addr":"172.21.15.100:6823","nonce":4104727330}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6820","nonce":4104727330},{"type":"v1","addr":"172.21.15.100:6821","nonce":4104727330}]},"public_addr":"172.21.15.100:6817/4104727330","cluster_addr":"172.21.15.100:6819/4104727330","heartbeat_back_addr":"172.21.15.100:6823/4104727330","heartbeat_front_addr":"172.21.15.100:6821/4104727330","state":["exists","up"]},{"osd":8,"uuid":"502d65cf-53c8-4936-8de4-e11b2a6e1e18","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":61,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":4067715347},{"type":"v1","addr":"172.21.15.157:6817","nonce":4067715347}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":4067715347},{"type":"v1","addr":"172.21.15.157:6819","nonce":4067715347}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":4067715347},{"type":"v1","addr":"172.21.15.157:6823","nonce":4067715347}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":4067715347},{"type":"v1","addr":"172.21.15.157:6821","nonce":4067715347}]},"public_addr":"172.21.15.157:6817/4067715347","cluster_addr":"172.21.15.157:6819/4067715347","heartbeat_back_addr":"172.21.15.157:6823/4067715347","heartbeat_front_addr":"172.21.15.157:6821/4067715347","state":["exists","up"]},{"osd":9,"uuid":"5ea1f5be-8458-4c93-9e61-c2e867a514f3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6826","nonce":1982108738},{"type":"v1","addr":"172.21.15.47:6827","nonce":1982108738}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6828","nonce":1982108738},{"type":"v1","addr":"172.21.15.47:6829","nonce":1982108738}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6832","nonce":1982108738},{"type":"v1","addr":"172.21.15.47:6833","nonce":1982108738}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6830","nonce":1982108738},{"type":"v1","addr":"172.21.15.47:6831","nonce":1982108738}]},"public_addr":"172.21.15.47:6827/1982108738","cluster_addr":"172.21.15.47:6829/1982108738","heartbeat_back_addr":"172.21.15.47:6833/1982108738","heartbeat_front_addr":"172.21.15.47:6831/1982108738","state":["exists","up"]},{"osd":10,"uuid":"2da3da90-a0be-4a54-a9ff-b2d6c9b36870","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":74,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":396391028},{"type":"v1","addr":"172.21.15.100:6825","nonce":396391028}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":396391028},{"type":"v1","addr":"172.21.15.100:6827","nonce":396391028}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6830","nonce":396391028},{"type":"v1","addr":"172.21.15.100:6831","nonce":396391028}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":396391028},{"type":"v1","addr":"172.21.15.100:6829","nonce":396391028}]},"public_addr":"172.21.15.100:6825/396391028","cluster_addr":"172.21.15.100:6827/396391028","heartbeat_back_addr":"172.21.15.100:6831/396391028","heartbeat_front_addr":"172.21.15.100:6829/396391028","state":["exists","up"]},{"osd":11,"uuid":"11798d41-7f7e-4319-95b8-8d3d0946bde9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":82,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":2671029483},{"type":"v1","addr":"172.21.15.157:6825","nonce":2671029483}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":2671029483},{"type":"v1","addr":"172.21.15.157:6827","nonce":2671029483}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":2671029483},{"type":"v1","addr":"172.21.15.157:6831","nonce":2671029483}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":2671029483},{"type":"v1","addr":"172.21.15.157:6829","nonce":2671029483}]},"public_addr":"172.21.15.157:6825/2671029483","cluster_addr":"172.21.15.157:6827/2671029483","heartbeat_back_addr":"172.21.15.157:6831/2671029483","heartbeat_front_addr":"172.21.15.157:6829/2671029483","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T10:14:31.822399+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T10:15:04.175721+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T10:15:35.902595+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T10:16:12.088891+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T10:16:46.428704+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T10:17:20.158270+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T10:17:55.915253+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T10:18:30.064641+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T10:19:04.410004+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T10:19:40.605181+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T10:20:15.636605+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T10:20:50.151365+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.47:0/356822431":"2024-03-22T10:12:25.844530+0000","172.21.15.47:0/1167490879":"2024-03-22T10:12:25.844530+0000","172.21.15.47:0/1540718138":"2024-03-22T10:12:25.844530+0000","172.21.15.47:6801/3836903972":"2024-03-22T10:12:25.844530+0000","172.21.15.47:0/235436602":"2024-03-22T10:11:50.301838+0000","172.21.15.47:0/2052815572":"2024-03-22T10:11:50.301838+0000","172.21.15.47:6800/3942658432":"2024-03-22T10:11:50.301838+0000","172.21.15.47:6801/3942658432":"2024-03-22T10:11:50.301838+0000","172.21.15.47:0/1419516257":"2024-03-22T10:11:50.301838+0000","172.21.15.47:6800/3836903972":"2024-03-22T10:12:25.844530+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-21T10:22:06.970 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:06.969+0000 7f25e8c00640 1 -- 172.21.15.47:0/2089858526 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f25c80a2fa0 msgr2=0x7f25c80a5460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:22:06.971 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:06.969+0000 7f25e8c00640 1 --2- 172.21.15.47:0/2089858526 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f25c80a2fa0 0x7f25c80a5460 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f25d4005fd0 tx=0x7f25d4009500 comp rx=0 tx=0).stop 2024-03-21T10:22:06.971 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:06.969+0000 7f25e8c00640 1 -- 172.21.15.47:0/2089858526 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f25e4077c40 msgr2=0x7f25e41a5410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:22:06.971 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:06.969+0000 7f25e8c00640 1 --2- 172.21.15.47:0/2089858526 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f25e4077c40 0x7f25e41a5410 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f25dc036040 tx=0x7f25dc02cdd0 comp rx=0 tx=0).stop 2024-03-21T10:22:06.971 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:06.969+0000 7f25e8c00640 1 -- 172.21.15.47:0/2089858526 shutdown_connections 2024-03-21T10:22:06.971 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:06.969+0000 7f25e8c00640 1 --2- 172.21.15.47:0/2089858526 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f25c80a2fa0 0x7f25c80a5460 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:22:06.971 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:06.969+0000 7f25e8c00640 1 --2- 172.21.15.47:0/2089858526 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f25e4077c40 0x7f25e41a5410 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:22:06.971 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:06.969+0000 7f25e8c00640 1 --2- 172.21.15.47:0/2089858526 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f25e40772a0 0x7f25e419e3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:22:06.971 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:06.969+0000 7f25e8c00640 1 --2- 172.21.15.47:0/2089858526 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f25e4076060 0x7f25e419de60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:22:06.971 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:06.969+0000 7f25e8c00640 1 -- 172.21.15.47:0/2089858526 >> 172.21.15.47:0/2089858526 conn(0x7f25e40fee50 msgr2=0x7f25e4100540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:22:06.971 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:06.970+0000 7f25e8c00640 1 -- 172.21.15.47:0/2089858526 shutdown_connections 2024-03-21T10:22:06.971 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:06.970+0000 7f25e8c00640 1 -- 172.21.15.47:0/2089858526 wait complete. 2024-03-21T10:22:06.993 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 17, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-03-21T10:21:58.329063+0000', 'modified': '2024-03-21T10:22:03.759800+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 34535}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_34535': {'gid': 34535, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.100:6839/4173168440', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.100:6838', 'nonce': 4173168440}, {'type': 'v1', 'addr': '172.21.15.100:6839', 'nonce': 4173168440}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 1, 'qdb_leader': 34535, 'qdb_cluster': [34535]} 2024-03-21T10:22:06.993 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-03-21T10:22:06.993 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2024-03-21T10:22:07.214 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:22:06 smithi047 ceph-mon[31547]: from='client.? 172.21.15.47:0/232678433' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-21T10:22:07.214 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:22:06 smithi047 ceph-mon[31547]: from='client.? 172.21.15.47:0/2305308242' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-21T10:22:07.224 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:07.223+0000 7f2567a00640 1 -- 172.21.15.47:0/2056116472 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f25601028a0 msgr2=0x7f2560102d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:22:07.225 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:07.223+0000 7f2567a00640 1 --2- 172.21.15.47:0/2056116472 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f25601028a0 0x7f2560102d20 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f254c009900 tx=0x7f254c02bc40 comp rx=0 tx=0).stop 2024-03-21T10:22:07.225 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:07.223+0000 7f2567a00640 1 -- 172.21.15.47:0/2056116472 shutdown_connections 2024-03-21T10:22:07.225 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:07.223+0000 7f2567a00640 1 --2- 172.21.15.47:0/2056116472 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2560103260 0x7f2560109af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:22:07.225 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:07.223+0000 7f2567a00640 1 --2- 172.21.15.47:0/2056116472 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f25601028a0 0x7f2560102d20 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:22:07.225 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:07.223+0000 7f2567a00640 1 --2- 172.21.15.47:0/2056116472 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f25601016b0 0x7f2560101ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:22:07.225 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:07.223+0000 7f2567a00640 1 -- 172.21.15.47:0/2056116472 >> 172.21.15.47:0/2056116472 conn(0x7f25600fcc40 msgr2=0x7f25600ff060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:22:07.225 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:07.223+0000 7f2567a00640 1 -- 172.21.15.47:0/2056116472 shutdown_connections 2024-03-21T10:22:07.225 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:07.223+0000 7f2567a00640 1 -- 172.21.15.47:0/2056116472 wait complete. 2024-03-21T10:22:07.227 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:07.224+0000 7f2567a00640 1 Processor -- start 2024-03-21T10:22:07.227 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:07.224+0000 7f2567a00640 1 -- start start 2024-03-21T10:22:07.228 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:07.224+0000 7f2567a00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f25601016b0 0x7f2560197880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:22:07.228 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:07.225+0000 7f2567a00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f25601028a0 0x7f2560197dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:22:07.228 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:07.225+0000 7f2567a00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2560103260 0x7f256019ee30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:22:07.228 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:07.225+0000 7f2567a00640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f256010d210 con 0x7f25601016b0 2024-03-21T10:22:07.228 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:07.225+0000 7f2567a00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f256010d090 con 0x7f2560103260 2024-03-21T10:22:07.228 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:07.225+0000 7f2567a00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f256010d390 con 0x7f25601028a0 2024-03-21T10:22:07.228 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:07.225+0000 7f2565000640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f25601016b0 0x7f2560197880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:22:07.229 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:07.225+0000 7f2565000640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f25601016b0 0x7f2560197880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:47060/0 (socket says 172.21.15.47:47060) 2024-03-21T10:22:07.229 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:07.225+0000 7f2565000640 1 -- 172.21.15.47:0/4213352584 learned_addr learned my addr 172.21.15.47:0/4213352584 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:22:07.229 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:07.225+0000 7f2565a00640 1 --2- 172.21.15.47:0/4213352584 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2560103260 0x7f256019ee30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:22:07.229 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:07.225+0000 7f255fe00640 1 --2- 172.21.15.47:0/4213352584 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f25601028a0 0x7f2560197dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:22:07.229 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:07.225+0000 7f2565000640 1 -- 172.21.15.47:0/4213352584 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f25601028a0 msgr2=0x7f2560197dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:22:07.229 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:07.225+0000 7f2565000640 1 --2- 172.21.15.47:0/4213352584 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f25601028a0 0x7f2560197dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:22:07.229 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:07.225+0000 7f2565000640 1 -- 172.21.15.47:0/4213352584 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2560103260 msgr2=0x7f256019ee30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:22:07.229 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:07.225+0000 7f2565000640 1 --2- 172.21.15.47:0/4213352584 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2560103260 0x7f256019ee30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:22:07.229 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:07.225+0000 7f2565000640 1 -- 172.21.15.47:0/4213352584 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f256019f4a0 con 0x7f25601016b0 2024-03-21T10:22:07.229 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:07.225+0000 7f255fe00640 1 --2- 172.21.15.47:0/4213352584 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f25601028a0 0x7f2560197dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T10:22:07.229 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:07.226+0000 7f2565000640 1 --2- 172.21.15.47:0/4213352584 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f25601016b0 0x7f2560197880 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f255400c970 tx=0x7f255400ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:22:07.229 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:07.226+0000 7f255d600640 1 -- 172.21.15.47:0/4213352584 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2554007c10 con 0x7f25601016b0 2024-03-21T10:22:07.229 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:07.226+0000 7f255d600640 1 -- 172.21.15.47:0/4213352584 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f2554007db0 con 0x7f25601016b0 2024-03-21T10:22:07.229 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:07.226+0000 7f2567a00640 1 -- 172.21.15.47:0/4213352584 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f256019f790 con 0x7f25601016b0 2024-03-21T10:22:07.230 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:07.226+0000 7f255d600640 1 -- 172.21.15.47:0/4213352584 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f255400f870 con 0x7f25601016b0 2024-03-21T10:22:07.230 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:07.226+0000 7f2567a00640 1 -- 172.21.15.47:0/4213352584 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f256019fce0 con 0x7f25601016b0 2024-03-21T10:22:07.230 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:22:06 smithi100 ceph-mon[35193]: from='client.? 172.21.15.47:0/232678433' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-21T10:22:07.230 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:22:06 smithi100 ceph-mon[35193]: from='client.? 172.21.15.47:0/2305308242' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-21T10:22:07.233 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:07.230+0000 7f256a7bb740 1 Processor -- start 2024-03-21T10:22:07.233 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:07.230+0000 7f255d600640 1 -- 172.21.15.47:0/4213352584 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 16) v1 ==== 137854+0+0 (secure 0 0 0) 0x7f255400fa10 con 0x7f25601016b0 2024-03-21T10:22:07.233 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:07.230+0000 7f256a7bb740 1 -- start start 2024-03-21T10:22:07.233 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:07.230+0000 7f256a7bb740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5594e6390fb0 0x5594e63913d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:22:07.233 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:07.230+0000 7f256a7bb740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5594e63919a0 0x5594e63964a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:22:07.233 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:07.230+0000 7f256a7bb740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5594e6396b10 0x5594e6398fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:22:07.233 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:07.230+0000 7f256a7bb740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x5594e62f1c90 con 0x5594e6390fb0 2024-03-21T10:22:07.233 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:07.230+0000 7f256a7bb740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x5594e629a980 con 0x5594e6396b10 2024-03-21T10:22:07.233 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:07.230+0000 7f256a7bb740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x5594e62edd30 con 0x5594e63919a0 2024-03-21T10:22:07.233 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:07.230+0000 7f2565a00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5594e6390fb0 0x5594e63913d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:22:07.233 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:07.230+0000 7f2565a00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5594e6390fb0 0x5594e63913d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:47062/0 (socket says 172.21.15.47:47062) 2024-03-21T10:22:07.233 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:07.230+0000 7f2565a00640 1 -- 172.21.15.47:0/19793416 learned_addr learned my addr 172.21.15.47:0/19793416 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:22:07.233 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:07.231+0000 7f2565000640 1 --2- 172.21.15.47:0/19793416 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5594e63919a0 0x5594e63964a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:22:07.233 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:07.231+0000 7f255fe00640 1 --2- 172.21.15.47:0/19793416 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5594e6396b10 0x5594e6398fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:22:07.234 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:07.231+0000 7f2565a00640 1 -- 172.21.15.47:0/19793416 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5594e63919a0 msgr2=0x5594e63964a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:22:07.234 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:07.231+0000 7f2565a00640 1 --2- 172.21.15.47:0/19793416 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5594e63919a0 0x5594e63964a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:22:07.234 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:07.231+0000 7f2565a00640 1 -- 172.21.15.47:0/19793416 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5594e6396b10 msgr2=0x5594e6398fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:22:07.234 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:07.231+0000 7f2565a00640 1 --2- 172.21.15.47:0/19793416 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5594e6396b10 0x5594e6398fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:22:07.234 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:07.231+0000 7f2565a00640 1 -- 172.21.15.47:0/19793416 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5594e62b2510 con 0x5594e6390fb0 2024-03-21T10:22:07.234 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:07.231+0000 7f2565a00640 1 --2- 172.21.15.47:0/19793416 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5594e6390fb0 0x5594e63913d0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f2558007e50 tx=0x7f255800e720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3296e22fd55ca911 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:22:07.234 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:07.231+0000 7f255d600640 1 --2- 172.21.15.47:0/4213352584 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f25480a2f20 0x7f25480a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:22:07.234 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:07.231+0000 7f2552000640 1 -- 172.21.15.47:0/19793416 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2558014070 con 0x5594e6390fb0 2024-03-21T10:22:07.234 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:07.232+0000 7f2552000640 1 -- 172.21.15.47:0/19793416 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f255800ee90 con 0x5594e6390fb0 2024-03-21T10:22:07.234 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:07.232+0000 7f2552000640 1 -- 172.21.15.47:0/19793416 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2558005380 con 0x5594e6390fb0 2024-03-21T10:22:07.234 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:07.232+0000 7f255d600640 1 -- 172.21.15.47:0/4213352584 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 8802+0+0 (secure 0 0 0) 0x7f25540cfaf0 con 0x7f25601016b0 2024-03-21T10:22:07.234 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:07.232+0000 7f2565000640 1 --2- 172.21.15.47:0/4213352584 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f25480a2f20 0x7f25480a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:22:07.238 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:07.232+0000 7f256a7bb740 1 -- 172.21.15.47:0/19793416 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5594e6390fb0 msgr2=0x5594e63913d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:22:07.238 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:07.232+0000 7f2565000640 1 --2- 172.21.15.47:0/4213352584 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f25480a2f20 0x7f25480a53e0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f25540989e0 tx=0x7f2554000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:22:07.239 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:07.232+0000 7f256a7bb740 1 --2- 172.21.15.47:0/19793416 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5594e6390fb0 0x5594e63913d0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f2558007e50 tx=0x7f255800e720 comp rx=0 tx=0).stop 2024-03-21T10:22:07.239 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:07.233+0000 7f256a7bb740 1 -- 172.21.15.47:0/19793416 shutdown_connections 2024-03-21T10:22:07.239 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:07.233+0000 7f256a7bb740 1 --2- 172.21.15.47:0/19793416 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5594e6396b10 0x5594e6398fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:22:07.239 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:07.233+0000 7f256a7bb740 1 --2- 172.21.15.47:0/19793416 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5594e63919a0 0x5594e63964a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:22:07.239 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:07.233+0000 7f256a7bb740 1 --2- 172.21.15.47:0/19793416 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5594e6390fb0 0x5594e63913d0 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:22:07.239 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:07.233+0000 7f256a7bb740 1 -- 172.21.15.47:0/19793416 >> 172.21.15.47:0/19793416 conn(0x5594e6388110 msgr2=0x5594e6388510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:22:07.239 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:07.233+0000 7f256a7bb740 1 -- 172.21.15.47:0/19793416 shutdown_connections 2024-03-21T10:22:07.239 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:07.233+0000 7f256a7bb740 1 -- 172.21.15.47:0/19793416 wait complete. 2024-03-21T10:22:07.239 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:07.233+0000 7f256a7bb740 1 -- start start 2024-03-21T10:22:07.239 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:07.233+0000 7f256a7bb740 1 Processor -- start 2024-03-21T10:22:07.239 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:07.234+0000 7f256a7bb740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5594e6390fb0 0x5594e639b4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:22:07.239 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:07.234+0000 7f256a7bb740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5594e63919a0 0x5594e639ba00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:22:07.239 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:07.234+0000 7f256a7bb740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5594e6396b10 0x5594e63a2a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:22:07.240 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:07.234+0000 7f256a7bb740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x5594e6399ac0 con 0x5594e6396b10 2024-03-21T10:22:07.240 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:07.234+0000 7f256a7bb740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x5594e62addd0 con 0x5594e63919a0 2024-03-21T10:22:07.240 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:07.234+0000 7f256a7bb740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x5594e6399c40 con 0x5594e6390fb0 2024-03-21T10:22:07.240 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:07.234+0000 7f256a7bb740 10 client.0 _ll_register_callbacks cb 0x5594e6005ac0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T10:22:07.240 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:07.234+0000 7f2565a00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5594e6396b10 0x5594e63a2a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:22:07.240 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:07.234+0000 7f2565a00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5594e6396b10 0x5594e63a2a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:47076/0 (socket says 172.21.15.47:47076) 2024-03-21T10:22:07.240 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:07.234+0000 7f2565a00640 1 -- 172.21.15.47:0/4215101704 learned_addr learned my addr 172.21.15.47:0/4215101704 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:22:07.240 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:07.234+0000 7f255fe00640 1 --2- 172.21.15.47:0/4215101704 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5594e63919a0 0x5594e639ba00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:22:07.240 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:07.234+0000 7f2565a00640 1 --2- 172.21.15.47:0/4215101704 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5594e6390fb0 0x5594e639b4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:22:07.240 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:07.235+0000 7f2565a00640 1 -- 172.21.15.47:0/4215101704 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5594e6390fb0 msgr2=0x5594e639b4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:22:07.240 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:07.235+0000 7f2565a00640 1 --2- 172.21.15.47:0/4215101704 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5594e6390fb0 0x5594e639b4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:22:07.240 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:07.235+0000 7f2565a00640 1 -- 172.21.15.47:0/4215101704 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5594e63919a0 msgr2=0x5594e639ba00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:22:07.240 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:07.235+0000 7f2565a00640 1 --2- 172.21.15.47:0/4215101704 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5594e63919a0 0x5594e639ba00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:22:07.240 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:07.235+0000 7f2565a00640 1 -- 172.21.15.47:0/4215101704 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5594e60062d0 con 0x5594e6396b10 2024-03-21T10:22:07.241 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:07.235+0000 7f255fe00640 1 --2- 172.21.15.47:0/4215101704 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5594e63919a0 0x5594e639ba00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T10:22:07.241 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:07.235+0000 7f2565a00640 1 --2- 172.21.15.47:0/4215101704 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5594e6396b10 0x5594e63a2a40 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f2558007710 tx=0x7f2558010d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=351f7ce87805ce7f server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:22:07.241 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:07.235+0000 7f2546a00640 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-03-21T10:22:07.241 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:07.235+0000 7f2546a00640 1 -- 172.21.15.47:0/4215101704 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f255801c030 con 0x5594e6396b10 2024-03-21T10:22:07.241 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:07.235+0000 7f256a7bb740 1 -- 172.21.15.47:0/4215101704 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f255800ee90 con 0x5594e6396b10 2024-03-21T10:22:07.241 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:07.235+0000 7f2546a00640 1 -- 172.21.15.47:0/4215101704 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 8802+0+0 (secure 0 0 0) 0x7f255801d070 con 0x5594e6396b10 2024-03-21T10:22:07.241 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:07.235+0000 7f2546a00640 1 -- 172.21.15.47:0/4215101704 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=17) v2 ==== 24+0+0 (secure 0 0 0) 0x7f255802c050 con 0x5594e6396b10 2024-03-21T10:22:07.241 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:07.235+0000 7f256a7bb740 10 client.14736 fetch_fsmap learned FSMap version 17 2024-03-21T10:22:07.241 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:07.235+0000 7f256a7bb740 1 -- 172.21.15.47:0/4215101704 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=17,monmap=4+}) v3 -- 0x5594e60d06b0 con 0x5594e6396b10 2024-03-21T10:22:07.241 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:07.236+0000 7f2546a00640 1 -- 172.21.15.47:0/4215101704 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f255800ee90 con 0x5594e6396b10 2024-03-21T10:22:07.241 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:07.236+0000 7f2546a00640 1 -- 172.21.15.47:0/4215101704 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 17) v1 ==== 7922+0+0 (secure 0 0 0) 0x7f2558003a00 con 0x5594e6396b10 2024-03-21T10:22:07.241 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:07.236+0000 7f256a7bb740 10 client.14736 fetch_fsmap finished waiting for FSMap version 17 2024-03-21T10:22:07.241 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:07.236+0000 7f256a7bb740 10 client.14736 resolve_mds: resolved name 'b' to daemon mds.b 2024-03-21T10:22:07.242 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:07.236+0000 7f256a7bb740 20 client.14736 populate_metadata read hostname 'smithi047' 2024-03-21T10:22:07.242 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:07.236+0000 7f256a7bb740 1 --2- 172.21.15.47:0/4215101704 >> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] conn(0x5594e63a7a90 0x5594e63a6cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:22:07.242 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:07.236+0000 7f256a7bb740 4 client.14736 mds_command: new command op to 34535 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T10:22:07.242 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:07.236+0000 7f256a7bb740 1 -- 172.21.15.47:0/4215101704 --> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5594e6293520 con 0x5594e63a7a90 2024-03-21T10:22:07.242 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:07.237+0000 7f255fe00640 1 --2- 172.21.15.47:0/4215101704 >> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] conn(0x5594e63a7a90 0x5594e63a6cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:22:07.242 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:07.238+0000 7f255fe00640 1 --2- 172.21.15.47:0/4215101704 >> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] conn(0x5594e63a7a90 0x5594e63a6cd0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=97493d4035861cdf server_cookie=d406592e897bfcea in_seq=0 out_seq=0 2024-03-21T10:22:07.242 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:07.238+0000 7f2546a00640 10 client.14736 ms_handle_connect on v2:172.21.15.100:6838/4173168440 2024-03-21T10:22:07.243 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:22:06 smithi157 ceph-mon[35581]: from='client.? 172.21.15.47:0/232678433' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-21T10:22:07.243 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:22:06 smithi157 ceph-mon[35581]: from='client.? 172.21.15.47:0/2305308242' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-21T10:22:07.243 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:07.240+0000 7f2546a00640 1 -- 172.21.15.47:0/4215101704 <== mds.0 v2:172.21.15.100:6838/4173168440 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x5594e6293520 con 0x5594e63a7a90 2024-03-21T10:22:07.244 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:07.240+0000 7f2546a00640 10 client.14736 handle_command_reply: tid=0 2024-03-21T10:22:07.244 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:07.240+0000 7f256a7bb740 1 client.14736 shutdown 2024-03-21T10:22:07.244 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:07.240+0000 7f256a7bb740 1 -- 172.21.15.47:0/4215101704 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5594e6396b10 msgr2=0x5594e63a2a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:22:07.244 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:07.240+0000 7f256a7bb740 1 --2- 172.21.15.47:0/4215101704 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5594e6396b10 0x5594e63a2a40 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f2558007710 tx=0x7f2558010d00 comp rx=0 tx=0).stop 2024-03-21T10:22:07.244 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:07.240+0000 7f256a7bb740 1 -- 172.21.15.47:0/4215101704 shutdown_connections 2024-03-21T10:22:07.244 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:07.240+0000 7f256a7bb740 1 --2- 172.21.15.47:0/4215101704 >> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] conn(0x5594e63a7a90 0x5594e63a6cd0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:22:07.244 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:07.240+0000 7f256a7bb740 1 --2- 172.21.15.47:0/4215101704 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5594e6396b10 0x5594e63a2a40 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:22:07.244 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:07.240+0000 7f256a7bb740 1 --2- 172.21.15.47:0/4215101704 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5594e63919a0 0x5594e639ba00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:22:07.244 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:07.240+0000 7f256a7bb740 1 --2- 172.21.15.47:0/4215101704 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5594e6390fb0 0x5594e639b4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:22:07.244 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:07.240+0000 7f2546a00640 0 client.14736 ms_handle_reset on v2:172.21.15.100:6838/4173168440 2024-03-21T10:22:07.244 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:07.240+0000 7f256a7bb740 1 -- 172.21.15.47:0/4215101704 >> 172.21.15.47:0/4215101704 conn(0x5594e6388110 msgr2=0x5594e6387310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:22:07.244 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:07.240+0000 7f256a7bb740 1 -- 172.21.15.47:0/4215101704 shutdown_connections 2024-03-21T10:22:07.244 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:07.241+0000 7f256a7bb740 1 -- 172.21.15.47:0/4215101704 wait complete. 2024-03-21T10:22:07.245 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:07.241+0000 7f256a7bb740 20 client.14736 trim_cache size 0 max 16384 2024-03-21T10:22:07.262 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:07.259+0000 7f256a7bb740 1 Processor -- start 2024-03-21T10:22:07.262 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:07.259+0000 7f256a7bb740 1 -- start start 2024-03-21T10:22:07.262 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:07.259+0000 7f256a7bb740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5594e6390fb0 0x5594e6382e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:22:07.262 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:07.259+0000 7f256a7bb740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5594e63919a0 0x5594e639b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:22:07.262 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:07.259+0000 7f256a7bb740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5594e6396b10 0x5594e639b610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:22:07.262 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:07.259+0000 7f256a7bb740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x5594e63a72b0 con 0x5594e63919a0 2024-03-21T10:22:07.262 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:07.259+0000 7f256a7bb740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x5594e63a7910 con 0x5594e6396b10 2024-03-21T10:22:07.262 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:07.259+0000 7f256a7bb740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x5594e63a7f70 con 0x5594e6390fb0 2024-03-21T10:22:07.262 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:07.260+0000 7f255fe00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5594e63919a0 0x5594e639b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:22:07.262 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:07.260+0000 7f255fe00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5594e63919a0 0x5594e639b0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:47080/0 (socket says 172.21.15.47:47080) 2024-03-21T10:22:07.262 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:07.260+0000 7f255fe00640 1 -- 172.21.15.47:0/3917214260 learned_addr learned my addr 172.21.15.47:0/3917214260 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:22:07.263 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:07.260+0000 7f2565a00640 1 --2- 172.21.15.47:0/3917214260 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5594e6390fb0 0x5594e6382e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:22:07.263 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:07.260+0000 7f255fe00640 1 -- 172.21.15.47:0/3917214260 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5594e6390fb0 msgr2=0x5594e6382e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:22:07.263 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:07.260+0000 7f255fe00640 1 --2- 172.21.15.47:0/3917214260 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5594e6390fb0 0x5594e6382e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:22:07.263 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:07.260+0000 7f2565a00640 1 --2- 172.21.15.47:0/3917214260 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5594e6396b10 0x5594e639b610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:22:07.263 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:07.260+0000 7f255fe00640 1 -- 172.21.15.47:0/3917214260 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5594e6396b10 msgr2=0x5594e639b610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:22:07.263 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:07.260+0000 7f255fe00640 1 --2- 172.21.15.47:0/3917214260 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5594e6396b10 0x5594e639b610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:22:07.263 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:07.260+0000 7f255fe00640 1 -- 172.21.15.47:0/3917214260 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5594e63a7040 con 0x5594e63919a0 2024-03-21T10:22:07.263 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:07.260+0000 7f255fe00640 1 --2- 172.21.15.47:0/3917214260 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5594e63919a0 0x5594e639b0d0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f254c005ab0 tx=0x7f254c009560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=101be48a6e0341b6 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:22:07.263 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:07.261+0000 7f2551600640 1 -- 172.21.15.47:0/3917214260 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f254c004240 con 0x5594e63919a0 2024-03-21T10:22:07.263 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:07.261+0000 7f2551600640 1 -- 172.21.15.47:0/3917214260 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f254c0043e0 con 0x5594e63919a0 2024-03-21T10:22:07.263 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:07.261+0000 7f2551600640 1 -- 172.21.15.47:0/3917214260 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f254c004d40 con 0x5594e63919a0 2024-03-21T10:22:07.266 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:07.261+0000 7f256a7bb740 1 -- 172.21.15.47:0/3917214260 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5594e63919a0 msgr2=0x5594e639b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:22:07.266 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:07.261+0000 7f256a7bb740 1 --2- 172.21.15.47:0/3917214260 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5594e63919a0 0x5594e639b0d0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f254c005ab0 tx=0x7f254c009560 comp rx=0 tx=0).stop 2024-03-21T10:22:07.267 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:07.261+0000 7f256a7bb740 1 -- 172.21.15.47:0/3917214260 shutdown_connections 2024-03-21T10:22:07.267 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:07.261+0000 7f256a7bb740 1 --2- 172.21.15.47:0/3917214260 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5594e6396b10 0x5594e639b610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:22:07.267 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:07.261+0000 7f256a7bb740 1 --2- 172.21.15.47:0/3917214260 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5594e63919a0 0x5594e639b0d0 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:22:07.267 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:07.261+0000 7f256a7bb740 1 --2- 172.21.15.47:0/3917214260 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5594e6390fb0 0x5594e6382e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:22:07.267 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:07.261+0000 7f256a7bb740 1 -- 172.21.15.47:0/3917214260 >> 172.21.15.47:0/3917214260 conn(0x5594e6388110 msgr2=0x5594e63884f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:22:07.267 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:07.262+0000 7f256a7bb740 1 -- 172.21.15.47:0/3917214260 shutdown_connections 2024-03-21T10:22:07.267 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:07.262+0000 7f256a7bb740 1 -- 172.21.15.47:0/3917214260 wait complete. 2024-03-21T10:22:07.267 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:07.262+0000 7f256a7bb740 1 -- start start 2024-03-21T10:22:07.267 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:07.262+0000 7f256a7bb740 1 Processor -- start 2024-03-21T10:22:07.267 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:07.262+0000 7f256a7bb740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5594e6390fb0 0x5594e63901d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:22:07.267 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:07.262+0000 7f256a7bb740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5594e63919a0 0x5594e6390710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:22:07.267 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:07.262+0000 7f256a7bb740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5594e6396b10 0x5594e63a2960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:22:07.267 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:07.263+0000 7f256a7bb740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x5594e63a8c30 con 0x5594e6396b10 2024-03-21T10:22:07.268 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:07.263+0000 7f256a7bb740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x5594e63a85d0 con 0x5594e6390fb0 2024-03-21T10:22:07.268 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:07.263+0000 7f256a7bb740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x5594e63a9290 con 0x5594e63919a0 2024-03-21T10:22:07.268 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:07.263+0000 7f256a7bb740 10 client.0 _ll_register_callbacks cb 0x7f2558007ca0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T10:22:07.268 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:07.263+0000 7f2565a00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5594e6396b10 0x5594e63a2960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:22:07.268 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:07.263+0000 7f2565a00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5594e6396b10 0x5594e63a2960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:47090/0 (socket says 172.21.15.47:47090) 2024-03-21T10:22:07.268 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:07.263+0000 7f2565a00640 1 -- 172.21.15.47:0/4272281459 learned_addr learned my addr 172.21.15.47:0/4272281459 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:22:07.268 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:07.263+0000 7f2565a00640 1 --2- 172.21.15.47:0/4272281459 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5594e6390fb0 0x5594e63901d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:22:07.268 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:07.263+0000 7f255fe00640 1 --2- 172.21.15.47:0/4272281459 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5594e63919a0 0x5594e6390710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:22:07.268 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:07.263+0000 7f2565a00640 1 -- 172.21.15.47:0/4272281459 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5594e63919a0 msgr2=0x5594e6390710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:22:07.268 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:07.263+0000 7f2565a00640 1 --2- 172.21.15.47:0/4272281459 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5594e63919a0 0x5594e6390710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:22:07.268 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:07.263+0000 7f2565a00640 1 -- 172.21.15.47:0/4272281459 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5594e6390fb0 msgr2=0x5594e63901d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:22:07.268 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:07.263+0000 7f2565a00640 1 --2- 172.21.15.47:0/4272281459 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5594e6390fb0 0x5594e63901d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:22:07.268 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:07.263+0000 7f2565a00640 1 -- 172.21.15.47:0/4272281459 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5594e638ea10 con 0x5594e6396b10 2024-03-21T10:22:07.268 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:07.264+0000 7f2565a00640 1 --2- 172.21.15.47:0/4272281459 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5594e6396b10 0x5594e63a2960 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f2558005920 tx=0x7f2558010d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=141329051b87c0b0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:22:07.269 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:07.264+0000 7f2546a00640 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-03-21T10:22:07.269 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:07.264+0000 7f2546a00640 1 -- 172.21.15.47:0/4272281459 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f255801c040 con 0x5594e6396b10 2024-03-21T10:22:07.269 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:07.264+0000 7f256a7bb740 1 -- 172.21.15.47:0/4272281459 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f254c0043e0 con 0x5594e6396b10 2024-03-21T10:22:07.269 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:07.264+0000 7f2546a00640 1 -- 172.21.15.47:0/4272281459 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 8802+0+0 (secure 0 0 0) 0x7f2558002030 con 0x5594e6396b10 2024-03-21T10:22:07.269 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:07.264+0000 7f2546a00640 1 -- 172.21.15.47:0/4272281459 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=17) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2558014070 con 0x5594e6396b10 2024-03-21T10:22:07.269 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:07.264+0000 7f256a7bb740 10 client.14742 fetch_fsmap learned FSMap version 17 2024-03-21T10:22:07.269 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:07.264+0000 7f256a7bb740 1 -- 172.21.15.47:0/4272281459 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=17,monmap=4+}) v3 -- 0x5594e64077d0 con 0x5594e6396b10 2024-03-21T10:22:07.269 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:07.264+0000 7f2546a00640 1 -- 172.21.15.47:0/4272281459 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f254c0043e0 con 0x5594e6396b10 2024-03-21T10:22:07.269 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:07.265+0000 7f2546a00640 1 -- 172.21.15.47:0/4272281459 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 17) v1 ==== 7922+0+0 (secure 0 0 0) 0x7f2558004cc0 con 0x5594e6396b10 2024-03-21T10:22:07.269 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:07.265+0000 7f256a7bb740 10 client.14742 fetch_fsmap finished waiting for FSMap version 17 2024-03-21T10:22:07.269 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:07.265+0000 7f256a7bb740 10 client.14742 resolve_mds: resolved name 'b' to daemon mds.b 2024-03-21T10:22:07.269 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:07.265+0000 7f256a7bb740 20 client.14742 populate_metadata read hostname 'smithi047' 2024-03-21T10:22:07.269 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:07.265+0000 7f256a7bb740 1 --2- 172.21.15.47:0/4272281459 >> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] conn(0x5594e640bdd0 0x5594e6407a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:22:07.269 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:07.265+0000 7f256a7bb740 4 client.14742 mds_command: new command op to 34535 tid=0 multi_id=0 [{"prefix": "status"}] 2024-03-21T10:22:07.270 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:07.265+0000 7f256a7bb740 1 -- 172.21.15.47:0/4272281459 --> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] -- command(tid 0: {"prefix": "status"}) v1 -- 0x5594e62ff1d0 con 0x5594e640bdd0 2024-03-21T10:22:07.270 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:07.265+0000 7f255fe00640 1 --2- 172.21.15.47:0/4272281459 >> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] conn(0x5594e640bdd0 0x5594e6407a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:22:07.271 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:07.266+0000 7f255fe00640 1 --2- 172.21.15.47:0/4272281459 >> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] conn(0x5594e640bdd0 0x5594e6407a00 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=51b97ec30b7dc7cb server_cookie=f155b8eba06bc042 in_seq=0 out_seq=0 2024-03-21T10:22:07.271 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:07.266+0000 7f2546a00640 10 client.14742 ms_handle_connect on v2:172.21.15.100:6838/4173168440 2024-03-21T10:22:07.271 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:07.267+0000 7f2546a00640 1 -- 172.21.15.47:0/4272281459 <== mds.0 v2:172.21.15.100:6838/4173168440 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+332 (crc 0 0 0) 0x5594e62ff1d0 con 0x5594e640bdd0 2024-03-21T10:22:07.271 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:07.267+0000 7f2546a00640 10 client.14742 handle_command_reply: tid=0 2024-03-21T10:22:07.271 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:07.267+0000 7f256a7bb740 1 client.14742 shutdown 2024-03-21T10:22:07.271 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:07.267+0000 7f256a7bb740 1 -- 172.21.15.47:0/4272281459 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5594e6396b10 msgr2=0x5594e63a2960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:22:07.271 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:07.267+0000 7f256a7bb740 1 --2- 172.21.15.47:0/4272281459 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5594e6396b10 0x5594e63a2960 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f2558005920 tx=0x7f2558010d00 comp rx=0 tx=0).stop 2024-03-21T10:22:07.271 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:07.267+0000 7f256a7bb740 1 -- 172.21.15.47:0/4272281459 shutdown_connections 2024-03-21T10:22:07.272 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:07.267+0000 7f256a7bb740 1 --2- 172.21.15.47:0/4272281459 >> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] conn(0x5594e640bdd0 0x5594e6407a00 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:22:07.272 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:07.267+0000 7f256a7bb740 1 --2- 172.21.15.47:0/4272281459 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5594e6396b10 0x5594e63a2960 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:22:07.272 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:07.267+0000 7f256a7bb740 1 --2- 172.21.15.47:0/4272281459 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5594e63919a0 0x5594e6390710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:22:07.272 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:07.267+0000 7f2546a00640 0 client.14742 ms_handle_reset on v2:172.21.15.100:6838/4173168440 2024-03-21T10:22:07.272 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:07.267+0000 7f256a7bb740 1 --2- 172.21.15.47:0/4272281459 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5594e6390fb0 0x5594e63901d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:22:07.272 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:07.267+0000 7f256a7bb740 1 -- 172.21.15.47:0/4272281459 >> 172.21.15.47:0/4272281459 conn(0x5594e6388110 msgr2=0x5594e6382e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:22:07.272 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:07.267+0000 7f256a7bb740 1 -- 172.21.15.47:0/4272281459 shutdown_connections 2024-03-21T10:22:07.272 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:07.267+0000 7f256a7bb740 1 -- 172.21.15.47:0/4272281459 wait complete. 2024-03-21T10:22:07.272 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:07.267+0000 7f256a7bb740 20 client.14742 trim_cache size 0 max 16384 2024-03-21T10:22:07.272 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-03-21T10:22:07.272 INFO:teuthology.orchestra.run.smithi047.stdout: "cluster_fsid": "3056d51a-e76b-11ee-95cd-87774f69a715", 2024-03-21T10:22:07.273 INFO:teuthology.orchestra.run.smithi047.stdout: "whoami": 0, 2024-03-21T10:22:07.273 INFO:teuthology.orchestra.run.smithi047.stdout: "id": 34535, 2024-03-21T10:22:07.273 INFO:teuthology.orchestra.run.smithi047.stdout: "want_state": "up:active", 2024-03-21T10:22:07.273 INFO:teuthology.orchestra.run.smithi047.stdout: "state": "up:active", 2024-03-21T10:22:07.273 INFO:teuthology.orchestra.run.smithi047.stdout: "fs_name": "cephfs", 2024-03-21T10:22:07.273 INFO:teuthology.orchestra.run.smithi047.stdout: "rank_uptime": 8.0103177559999992, 2024-03-21T10:22:07.273 INFO:teuthology.orchestra.run.smithi047.stdout: "mdsmap_epoch": 17, 2024-03-21T10:22:07.273 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap_epoch": 91, 2024-03-21T10:22:07.273 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap_epoch_barrier": 91, 2024-03-21T10:22:07.273 INFO:teuthology.orchestra.run.smithi047.stdout: "uptime": 13.672114839000001 2024-03-21T10:22:07.273 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-03-21T10:22:07.274 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:07.268+0000 7f2567a00640 1 -- 172.21.15.47:0/4213352584 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f25480a2f20 msgr2=0x7f25480a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:22:07.274 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:07.268+0000 7f2567a00640 1 --2- 172.21.15.47:0/4213352584 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f25480a2f20 0x7f25480a53e0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f25540989e0 tx=0x7f2554000bc0 comp rx=0 tx=0).stop 2024-03-21T10:22:07.274 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:07.268+0000 7f2567a00640 1 -- 172.21.15.47:0/4213352584 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f25601016b0 msgr2=0x7f2560197880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:22:07.274 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:07.268+0000 7f2567a00640 1 --2- 172.21.15.47:0/4213352584 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f25601016b0 0x7f2560197880 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f255400c970 tx=0x7f255400ce30 comp rx=0 tx=0).stop 2024-03-21T10:22:07.274 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:07.268+0000 7f2567a00640 1 -- 172.21.15.47:0/4213352584 shutdown_connections 2024-03-21T10:22:07.274 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:07.268+0000 7f2567a00640 1 --2- 172.21.15.47:0/4213352584 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f25480a2f20 0x7f25480a53e0 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:22:07.274 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:07.268+0000 7f2567a00640 1 --2- 172.21.15.47:0/4213352584 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2560103260 0x7f256019ee30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:22:07.274 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:07.268+0000 7f2567a00640 1 --2- 172.21.15.47:0/4213352584 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f25601028a0 0x7f2560197dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:22:07.275 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:07.268+0000 7f2567a00640 1 --2- 172.21.15.47:0/4213352584 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f25601016b0 0x7f2560197880 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:22:07.275 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:07.268+0000 7f2567a00640 1 -- 172.21.15.47:0/4213352584 >> 172.21.15.47:0/4213352584 conn(0x7f25600fcc40 msgr2=0x7f2560105ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:22:07.275 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:07.268+0000 7f2567a00640 1 -- 172.21.15.47:0/4213352584 shutdown_connections 2024-03-21T10:22:07.275 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:07.268+0000 7f2567a00640 1 -- 172.21.15.47:0/4213352584 wait complete. 2024-03-21T10:22:07.287 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-03-21T10:22:07.287 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.admin 2024-03-21T10:22:07.287 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2edfee68ad7243bed422a049b6f5f58619300f11 shell --fsid 3056d51a-e76b-11ee-95cd-87774f69a715 -- ceph fs ls 2024-03-21T10:22:07.743 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/3056d51a-e76b-11ee-95cd-87774f69a715/mon.a/config 2024-03-21T10:22:08.189 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:22:07 smithi047 ceph-mon[31547]: pgmap v371: 129 pgs: 115 active+clean, 12 creating+peering, 2 unknown; 579 KiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s wr, 8 op/s 2024-03-21T10:22:08.190 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:22:07 smithi047 ceph-mon[31547]: from='client.? 172.21.15.47:0/2089858526' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-21T10:22:08.229 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:22:07 smithi100 ceph-mon[35193]: pgmap v371: 129 pgs: 115 active+clean, 12 creating+peering, 2 unknown; 579 KiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s wr, 8 op/s 2024-03-21T10:22:08.230 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:22:07 smithi100 ceph-mon[35193]: from='client.? 172.21.15.47:0/2089858526' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-21T10:22:08.240 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:22:07 smithi157 ceph-mon[35581]: pgmap v371: 129 pgs: 115 active+clean, 12 creating+peering, 2 unknown; 579 KiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s wr, 8 op/s 2024-03-21T10:22:08.240 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:22:07 smithi157 ceph-mon[35581]: from='client.? 172.21.15.47:0/2089858526' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-21T10:22:09.077 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:22:08 smithi047 ceph-mon[31547]: pgmap v372: 129 pgs: 129 active+clean; 579 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s wr, 6 op/s 2024-03-21T10:22:09.143 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:22:08 smithi157 ceph-mon[35581]: pgmap v372: 129 pgs: 129 active+clean; 579 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s wr, 6 op/s 2024-03-21T10:22:09.229 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:22:08 smithi100 ceph-mon[35193]: pgmap v372: 129 pgs: 129 active+clean; 579 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s wr, 6 op/s 2024-03-21T10:22:09.897 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:09.894+0000 7f1333600700 1 -- 172.21.15.47:0/3048121104 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f132c1074f0 msgr2=0x7f132c10f0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:22:09.897 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:09.894+0000 7f1333600700 1 --2- 172.21.15.47:0/3048121104 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f132c1074f0 0x7f132c10f0c0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f132400b0d0 tx=0x7f132400b3e0 comp rx=0 tx=0).stop 2024-03-21T10:22:09.897 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:09.895+0000 7f1333600700 1 -- 172.21.15.47:0/3048121104 shutdown_connections 2024-03-21T10:22:09.897 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:09.895+0000 7f1333600700 1 --2- 172.21.15.47:0/3048121104 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f132c1074f0 0x7f132c10f0c0 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:22:09.897 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:09.895+0000 7f1333600700 1 --2- 172.21.15.47:0/3048121104 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f132c102410 0x7f132c1027f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:22:09.897 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:09.895+0000 7f1333600700 1 --2- 172.21.15.47:0/3048121104 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f132c102dc0 0x7f132c106e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:22:09.898 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:09.895+0000 7f1333600700 1 -- 172.21.15.47:0/3048121104 >> 172.21.15.47:0/3048121104 conn(0x7f132c0fdc60 msgr2=0x7f132c100080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:22:09.898 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:09.895+0000 7f1333600700 1 -- 172.21.15.47:0/3048121104 shutdown_connections 2024-03-21T10:22:09.898 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:09.895+0000 7f1333600700 1 -- 172.21.15.47:0/3048121104 wait complete. 2024-03-21T10:22:09.898 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:09.896+0000 7f1333600700 1 Processor -- start 2024-03-21T10:22:09.898 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:09.896+0000 7f1333600700 1 -- start start 2024-03-21T10:22:09.900 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:09.896+0000 7f1333600700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f132c102410 0x7f132c1a06b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:22:09.900 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:09.896+0000 7f1333600700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f132c102dc0 0x7f132c1a0bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:22:09.900 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:09.896+0000 7f1333600700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f132c1074f0 0x7f132c19a870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:22:09.900 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:09.896+0000 7f1333600700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f132c112450 con 0x7f132c102dc0 2024-03-21T10:22:09.900 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:09.896+0000 7f1333600700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f132c1122d0 con 0x7f132c102410 2024-03-21T10:22:09.900 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:09.896+0000 7f1333600700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f132c1125d0 con 0x7f132c1074f0 2024-03-21T10:22:09.900 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:09.897+0000 7f132be00700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f132c102dc0 0x7f132c1a0bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:22:09.900 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:09.897+0000 7f132be00700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f132c102dc0 0x7f132c1a0bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:47100/0 (socket says 172.21.15.47:47100) 2024-03-21T10:22:09.900 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:09.897+0000 7f132be00700 1 -- 172.21.15.47:0/1787904275 learned_addr learned my addr 172.21.15.47:0/1787904275 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:22:09.901 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:09.897+0000 7f1330c00700 1 --2- 172.21.15.47:0/1787904275 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f132c102410 0x7f132c1a06b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:22:09.901 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:09.897+0000 7f1331600700 1 --2- 172.21.15.47:0/1787904275 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f132c1074f0 0x7f132c19a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:22:09.901 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:09.897+0000 7f132be00700 1 -- 172.21.15.47:0/1787904275 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f132c1074f0 msgr2=0x7f132c19a870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:22:09.901 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:09.897+0000 7f132be00700 1 --2- 172.21.15.47:0/1787904275 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f132c1074f0 0x7f132c19a870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:22:09.901 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:09.897+0000 7f132be00700 1 -- 172.21.15.47:0/1787904275 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f132c102410 msgr2=0x7f132c1a06b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:22:09.901 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:09.897+0000 7f132be00700 1 --2- 172.21.15.47:0/1787904275 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f132c102410 0x7f132c1a06b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:22:09.901 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:09.897+0000 7f132be00700 1 -- 172.21.15.47:0/1787904275 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f132c19afd0 con 0x7f132c102dc0 2024-03-21T10:22:09.901 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:09.897+0000 7f1330c00700 1 --2- 172.21.15.47:0/1787904275 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f132c102410 0x7f132c1a06b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T10:22:09.901 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:09.898+0000 7f132be00700 1 --2- 172.21.15.47:0/1787904275 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f132c102dc0 0x7f132c1a0bf0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f131800c9e0 tx=0x7f131800cda0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:22:09.901 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:09.898+0000 7f1329600700 1 -- 172.21.15.47:0/1787904275 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1318007aa0 con 0x7f132c102dc0 2024-03-21T10:22:09.901 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:09.898+0000 7f1329600700 1 -- 172.21.15.47:0/1787904275 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f1318007c40 con 0x7f132c102dc0 2024-03-21T10:22:09.901 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:09.898+0000 7f1329600700 1 -- 172.21.15.47:0/1787904275 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1318018660 con 0x7f132c102dc0 2024-03-21T10:22:09.901 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:09.898+0000 7f1333600700 1 -- 172.21.15.47:0/1787904275 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f132c19b2c0 con 0x7f132c102dc0 2024-03-21T10:22:09.901 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:09.898+0000 7f1333600700 1 -- 172.21.15.47:0/1787904275 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f132c1a7580 con 0x7f132c102dc0 2024-03-21T10:22:09.902 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:09.899+0000 7f1333600700 1 -- 172.21.15.47:0/1787904275 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f132c068e50 con 0x7f132c102dc0 2024-03-21T10:22:09.902 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:09.901+0000 7f1329600700 1 -- 172.21.15.47:0/1787904275 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 16) v1 ==== 137854+0+0 (secure 0 0 0) 0x7f131800f450 con 0x7f132c102dc0 2024-03-21T10:22:09.904 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:09.904+0000 7f1329600700 1 --2- 172.21.15.47:0/1787904275 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f13140a3030 0x7f13140a54f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:22:09.905 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:09.904+0000 7f1329600700 1 -- 172.21.15.47:0/1787904275 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 8802+0+0 (secure 0 0 0) 0x7f13180cf5a0 con 0x7f132c102dc0 2024-03-21T10:22:09.905 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:09.904+0000 7f1330c00700 1 --2- 172.21.15.47:0/1787904275 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f13140a3030 0x7f13140a54f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:22:09.906 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:09.905+0000 7f1330c00700 1 --2- 172.21.15.47:0/1787904275 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f13140a3030 0x7f13140a54f0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f132c19b9b0 tx=0x7f1320008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:22:09.909 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:09.908+0000 7f1329600700 1 -- 172.21.15.47:0/1787904275 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f1318014030 con 0x7f132c102dc0 2024-03-21T10:22:10.194 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:10.192+0000 7f1333600700 1 -- 172.21.15.47:0/1787904275 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f132c1a7840 con 0x7f132c102dc0 2024-03-21T10:22:10.194 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:10.193+0000 7f1329600700 1 -- 172.21.15.47:0/1787904275 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v17) v1 ==== 53+0+88 (secure 0 0 0) 0x7f131809c130 con 0x7f132c102dc0 2024-03-21T10:22:10.195 INFO:teuthology.orchestra.run.smithi047.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2024-03-21T10:22:10.197 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:10.196+0000 7f1333600700 1 -- 172.21.15.47:0/1787904275 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f13140a3030 msgr2=0x7f13140a54f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:22:10.197 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:10.196+0000 7f1333600700 1 --2- 172.21.15.47:0/1787904275 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f13140a3030 0x7f13140a54f0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f132c19b9b0 tx=0x7f1320008040 comp rx=0 tx=0).stop 2024-03-21T10:22:10.197 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:10.196+0000 7f1333600700 1 -- 172.21.15.47:0/1787904275 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f132c102dc0 msgr2=0x7f132c1a0bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:22:10.197 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:10.196+0000 7f1333600700 1 --2- 172.21.15.47:0/1787904275 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f132c102dc0 0x7f132c1a0bf0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f131800c9e0 tx=0x7f131800cda0 comp rx=0 tx=0).stop 2024-03-21T10:22:10.198 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:10.196+0000 7f1333600700 1 -- 172.21.15.47:0/1787904275 shutdown_connections 2024-03-21T10:22:10.198 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:10.196+0000 7f1333600700 1 --2- 172.21.15.47:0/1787904275 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f13140a3030 0x7f13140a54f0 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:22:10.198 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:10.196+0000 7f1333600700 1 --2- 172.21.15.47:0/1787904275 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f132c1074f0 0x7f132c19a870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:22:10.198 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:10.196+0000 7f1333600700 1 --2- 172.21.15.47:0/1787904275 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f132c102410 0x7f132c1a06b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:22:10.198 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:10.196+0000 7f1333600700 1 --2- 172.21.15.47:0/1787904275 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f132c102dc0 0x7f132c1a0bf0 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:22:10.198 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:10.196+0000 7f1333600700 1 -- 172.21.15.47:0/1787904275 >> 172.21.15.47:0/1787904275 conn(0x7f132c0fdc60 msgr2=0x7f132c10d9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:22:10.198 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:10.196+0000 7f1333600700 1 -- 172.21.15.47:0/1787904275 shutdown_connections 2024-03-21T10:22:10.198 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:10.196+0000 7f1333600700 1 -- 172.21.15.47:0/1787904275 wait complete. 2024-03-21T10:22:11.206 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-03-21T10:22:11.206 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-03-21T10:22:11.206 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi047.front.sepia.ceph.com 2024-03-21T10:22:11.206 INFO:tasks.cephfs.mount:self.client.name = client.admin 2024-03-21T10:22:11.206 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.admin 2024-03-21T10:22:11.206 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-03-21T10:22:11.206 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-03-21T10:22:11.206 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-03-21T10:22:11.206 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.admin' 2024-03-21T10:22:11.207 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-21T10:22:11.207 DEBUG:teuthology.orchestra.run.smithi047:> ip addr 2024-03-21T10:22:11.267 INFO:teuthology.orchestra.run.smithi047.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-03-21T10:22:11.267 INFO:teuthology.orchestra.run.smithi047.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-03-21T10:22:11.267 INFO:teuthology.orchestra.run.smithi047.stdout: inet 127.0.0.1/8 scope host lo 2024-03-21T10:22:11.268 INFO:teuthology.orchestra.run.smithi047.stdout: valid_lft forever preferred_lft forever 2024-03-21T10:22:11.268 INFO:teuthology.orchestra.run.smithi047.stdout: inet6 ::1/128 scope host 2024-03-21T10:22:11.268 INFO:teuthology.orchestra.run.smithi047.stdout: valid_lft forever preferred_lft forever 2024-03-21T10:22:11.268 INFO:teuthology.orchestra.run.smithi047.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-21T10:22:11.268 INFO:teuthology.orchestra.run.smithi047.stdout: link/ether 0c:c4:7a:6b:ff:78 brd ff:ff:ff:ff:ff:ff 2024-03-21T10:22:11.268 INFO:teuthology.orchestra.run.smithi047.stdout: altname enp7s0f0 2024-03-21T10:22:11.268 INFO:teuthology.orchestra.run.smithi047.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-21T10:22:11.268 INFO:teuthology.orchestra.run.smithi047.stdout: link/ether 0c:c4:7a:6b:ff:79 brd ff:ff:ff:ff:ff:ff 2024-03-21T10:22:11.268 INFO:teuthology.orchestra.run.smithi047.stdout: altname enp7s0f1 2024-03-21T10:22:11.268 INFO:teuthology.orchestra.run.smithi047.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-03-21T10:22:11.268 INFO:teuthology.orchestra.run.smithi047.stdout: link/ether 0c:c4:7a:bd:14:8e brd ff:ff:ff:ff:ff:ff 2024-03-21T10:22:11.268 INFO:teuthology.orchestra.run.smithi047.stdout: altname enp1s0f0 2024-03-21T10:22:11.268 INFO:teuthology.orchestra.run.smithi047.stdout: inet 172.21.15.47/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-03-21T10:22:11.268 INFO:teuthology.orchestra.run.smithi047.stdout: valid_lft 41909sec preferred_lft 41909sec 2024-03-21T10:22:11.269 INFO:teuthology.orchestra.run.smithi047.stdout: inet6 fe80::ec4:7aff:febd:148e/64 scope link noprefixroute 2024-03-21T10:22:11.269 INFO:teuthology.orchestra.run.smithi047.stdout: valid_lft forever preferred_lft forever 2024-03-21T10:22:11.269 INFO:teuthology.orchestra.run.smithi047.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-21T10:22:11.269 INFO:teuthology.orchestra.run.smithi047.stdout: link/ether 0c:c4:7a:bd:14:8f brd ff:ff:ff:ff:ff:ff 2024-03-21T10:22:11.269 INFO:teuthology.orchestra.run.smithi047.stdout: altname enp1s0f1 2024-03-21T10:22:11.270 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-03-21T10:22:11.270 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-21T10:22:11.270 DEBUG:teuthology.orchestra.run.smithi047:> (cd / && exec bash -c ' 2024-03-21T10:22:11.270 DEBUG:teuthology.orchestra.run.smithi047:> set -e 2024-03-21T10:22:11.271 DEBUG:teuthology.orchestra.run.smithi047:> sudo ip link add name ceph-brx type bridge 2024-03-21T10:22:11.271 DEBUG:teuthology.orchestra.run.smithi047:> sudo ip addr flush dev ceph-brx 2024-03-21T10:22:11.271 DEBUG:teuthology.orchestra.run.smithi047:> sudo ip link set ceph-brx up 2024-03-21T10:22:11.271 DEBUG:teuthology.orchestra.run.smithi047:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-03-21T10:22:11.271 DEBUG:teuthology.orchestra.run.smithi047:> ') 2024-03-21T10:22:11.363 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:22:11 smithi047 ceph-mon[31547]: pgmap v373: 129 pgs: 129 active+clean; 579 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s wr, 2 op/s 2024-03-21T10:22:11.364 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:22:11 smithi047 ceph-mon[31547]: from='client.? 172.21.15.47:0/1787904275' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-03-21T10:22:11.479 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:22:11 smithi100 ceph-mon[35193]: pgmap v373: 129 pgs: 129 active+clean; 579 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s wr, 2 op/s 2024-03-21T10:22:11.480 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:22:11 smithi100 ceph-mon[35193]: from='client.? 172.21.15.47:0/1787904275' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-03-21T10:22:11.490 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:22:11 smithi157 ceph-mon[35581]: pgmap v373: 129 pgs: 129 active+clean; 579 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s wr, 2 op/s 2024-03-21T10:22:11.490 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:22:11 smithi157 ceph-mon[35581]: from='client.? 172.21.15.47:0/1787904275' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-03-21T10:22:11.712 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-21T10:22:11.713 DEBUG:teuthology.orchestra.run.smithi047:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-03-21T10:22:11.792 INFO:teuthology.orchestra.run.smithi047.stdout:1 2024-03-21T10:22:11.796 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-21T10:22:11.796 DEBUG:teuthology.orchestra.run.smithi047:> route 2024-03-21T10:22:11.857 INFO:teuthology.orchestra.run.smithi047.stdout:Kernel IP routing table 2024-03-21T10:22:11.857 INFO:teuthology.orchestra.run.smithi047.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-03-21T10:22:11.857 INFO:teuthology.orchestra.run.smithi047.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-03-21T10:22:11.857 INFO:teuthology.orchestra.run.smithi047.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-03-21T10:22:11.857 INFO:teuthology.orchestra.run.smithi047.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-03-21T10:22:11.857 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-21T10:22:11.858 DEBUG:teuthology.orchestra.run.smithi047:> (cd / && exec bash -c ' 2024-03-21T10:22:11.858 DEBUG:teuthology.orchestra.run.smithi047:> set -e 2024-03-21T10:22:11.858 DEBUG:teuthology.orchestra.run.smithi047:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-03-21T10:22:11.858 DEBUG:teuthology.orchestra.run.smithi047:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-03-21T10:22:11.858 DEBUG:teuthology.orchestra.run.smithi047:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-03-21T10:22:11.858 DEBUG:teuthology.orchestra.run.smithi047:> ') 2024-03-21T10:22:12.711 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-21T10:22:12.711 DEBUG:teuthology.orchestra.run.smithi047:> ip netns list 2024-03-21T10:22:12.767 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-21T10:22:12.767 DEBUG:teuthology.orchestra.run.smithi047:> ip netns list-id 2024-03-21T10:22:12.820 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-21T10:22:12.820 DEBUG:teuthology.orchestra.run.smithi047:> (cd / && exec bash -c ' 2024-03-21T10:22:12.820 DEBUG:teuthology.orchestra.run.smithi047:> set -e 2024-03-21T10:22:12.820 DEBUG:teuthology.orchestra.run.smithi047:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-03-21T10:22:12.820 DEBUG:teuthology.orchestra.run.smithi047:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.admin 0 2024-03-21T10:22:12.820 DEBUG:teuthology.orchestra.run.smithi047:> ') 2024-03-21T10:22:12.968 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.admin' with 192.168.0.1/16 2024-03-21T10:22:12.969 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-21T10:22:12.969 DEBUG:teuthology.orchestra.run.smithi047:> (cd / && exec bash -c ' 2024-03-21T10:22:12.969 DEBUG:teuthology.orchestra.run.smithi047:> set -e 2024-03-21T10:22:12.969 DEBUG:teuthology.orchestra.run.smithi047:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.admin type veth peer name brx.0 2024-03-21T10:22:12.969 DEBUG:teuthology.orchestra.run.smithi047:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-03-21T10:22:12.969 DEBUG:teuthology.orchestra.run.smithi047:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip link set veth0 up 2024-03-21T10:22:12.969 DEBUG:teuthology.orchestra.run.smithi047:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip link set lo up 2024-03-21T10:22:12.969 DEBUG:teuthology.orchestra.run.smithi047:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip route add default via 192.168.255.254 2024-03-21T10:22:12.969 DEBUG:teuthology.orchestra.run.smithi047:> ') 2024-03-21T10:22:13.206 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:22:13 smithi047 ceph-mon[31547]: pgmap v374: 129 pgs: 129 active+clean; 579 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s wr, 1 op/s 2024-03-21T10:22:13.263 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-21T10:22:13.263 DEBUG:teuthology.orchestra.run.smithi047:> (cd / && exec bash -c ' 2024-03-21T10:22:13.263 DEBUG:teuthology.orchestra.run.smithi047:> set -e 2024-03-21T10:22:13.263 DEBUG:teuthology.orchestra.run.smithi047:> sudo ip link set brx.0 up 2024-03-21T10:22:13.263 DEBUG:teuthology.orchestra.run.smithi047:> sudo ip link set dev brx.0 master ceph-brx 2024-03-21T10:22:13.263 DEBUG:teuthology.orchestra.run.smithi047:> ') 2024-03-21T10:22:13.438 INFO:tasks.cephfs.fuse_mount:Client client.admin config is {} 2024-03-21T10:22:13.438 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-03-21T10:22:13.438 DEBUG:teuthology.orchestra.run.smithi047:> mkdir -p -v /home/ubuntu/cephtest/mnt.admin 2024-03-21T10:22:13.479 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:22:13 smithi100 ceph-mon[35193]: pgmap v374: 129 pgs: 129 active+clean; 579 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s wr, 1 op/s 2024-03-21T10:22:13.490 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:22:13 smithi157 ceph-mon[35581]: pgmap v374: 129 pgs: 129 active+clean; 579 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s wr, 1 op/s 2024-03-21T10:22:13.493 INFO:teuthology.orchestra.run.smithi047.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.admin' 2024-03-21T10:22:13.493 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-03-21T10:22:13.493 DEBUG:teuthology.orchestra.run.smithi047:> chmod 0000 /home/ubuntu/cephtest/mnt.admin 2024-03-21T10:22:13.542 DEBUG:teuthology.orchestra.run.smithi047:> sudo modprobe fuse 2024-03-21T10:22:13.620 DEBUG:teuthology.orchestra.run.smithi047:> cat /proc/self/mounts | awk '{print $2}' 2024-03-21T10:22:13.666 INFO:teuthology.orchestra.run.smithi047.stdout:/proc 2024-03-21T10:22:13.666 INFO:teuthology.orchestra.run.smithi047.stdout:/sys 2024-03-21T10:22:13.666 INFO:teuthology.orchestra.run.smithi047.stdout:/dev 2024-03-21T10:22:13.666 INFO:teuthology.orchestra.run.smithi047.stdout:/sys/kernel/security 2024-03-21T10:22:13.666 INFO:teuthology.orchestra.run.smithi047.stdout:/dev/shm 2024-03-21T10:22:13.666 INFO:teuthology.orchestra.run.smithi047.stdout:/dev/pts 2024-03-21T10:22:13.666 INFO:teuthology.orchestra.run.smithi047.stdout:/run 2024-03-21T10:22:13.666 INFO:teuthology.orchestra.run.smithi047.stdout:/sys/fs/cgroup 2024-03-21T10:22:13.666 INFO:teuthology.orchestra.run.smithi047.stdout:/sys/fs/pstore 2024-03-21T10:22:13.666 INFO:teuthology.orchestra.run.smithi047.stdout:/sys/fs/bpf 2024-03-21T10:22:13.666 INFO:teuthology.orchestra.run.smithi047.stdout:/ 2024-03-21T10:22:13.667 INFO:teuthology.orchestra.run.smithi047.stdout:/sys/fs/selinux 2024-03-21T10:22:13.667 INFO:teuthology.orchestra.run.smithi047.stdout:/proc/sys/fs/binfmt_misc 2024-03-21T10:22:13.667 INFO:teuthology.orchestra.run.smithi047.stdout:/dev/mqueue 2024-03-21T10:22:13.667 INFO:teuthology.orchestra.run.smithi047.stdout:/dev/hugepages 2024-03-21T10:22:13.667 INFO:teuthology.orchestra.run.smithi047.stdout:/sys/kernel/debug 2024-03-21T10:22:13.667 INFO:teuthology.orchestra.run.smithi047.stdout:/sys/kernel/tracing 2024-03-21T10:22:13.667 INFO:teuthology.orchestra.run.smithi047.stdout:/sys/fs/fuse/connections 2024-03-21T10:22:13.667 INFO:teuthology.orchestra.run.smithi047.stdout:/sys/kernel/config 2024-03-21T10:22:13.667 INFO:teuthology.orchestra.run.smithi047.stdout:/run/credentials/systemd-sysctl.service 2024-03-21T10:22:13.667 INFO:teuthology.orchestra.run.smithi047.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-03-21T10:22:13.667 INFO:teuthology.orchestra.run.smithi047.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-03-21T10:22:13.667 INFO:teuthology.orchestra.run.smithi047.stdout:/var/lib/nfs/rpc_pipefs 2024-03-21T10:22:13.667 INFO:teuthology.orchestra.run.smithi047.stdout:/run/user/1000 2024-03-21T10:22:13.667 INFO:teuthology.orchestra.run.smithi047.stdout:/var/lib/ceph 2024-03-21T10:22:13.668 INFO:teuthology.orchestra.run.smithi047.stdout:/var/lib/containers/storage/overlay 2024-03-21T10:22:13.668 INFO:teuthology.orchestra.run.smithi047.stdout:/var/lib/containers/storage/overlay/10ab6f66d550c87573f945f803c4f92650c76c2142b200e308dd5734436cc6e8/merged 2024-03-21T10:22:13.668 INFO:teuthology.orchestra.run.smithi047.stdout:/var/lib/containers/storage/overlay/4193f3bc34f41e5fb90c176b8210c5c797208467c6f372b377075907f20e100c/merged 2024-03-21T10:22:13.668 INFO:teuthology.orchestra.run.smithi047.stdout:/run/user/0 2024-03-21T10:22:13.668 INFO:teuthology.orchestra.run.smithi047.stdout:/proc/sys/fs/binfmt_misc 2024-03-21T10:22:13.668 INFO:teuthology.orchestra.run.smithi047.stdout:/var/lib/containers/storage/overlay/fa63c9447d222c5abaf50fb70c1cc34ea356854abf3426eb0c8a7f015a95e543/merged 2024-03-21T10:22:13.668 INFO:teuthology.orchestra.run.smithi047.stdout:/var/lib/containers/storage/overlay/cab8c1c11ddc44b3f7251050e29e61c724a07774891943f3f169428cf41f19ce/merged 2024-03-21T10:22:13.668 INFO:teuthology.orchestra.run.smithi047.stdout:/var/lib/containers/storage/overlay/8bc14524d7f02db40a42ab65f069f5682514962833182f9de83d3508887ef2e7/merged 2024-03-21T10:22:13.668 INFO:teuthology.orchestra.run.smithi047.stdout:/var/lib/containers/storage/overlay/225ab22923163649720ee3c99e303986cfdc6df3ece1f25ae6bea1c7386ea563/merged 2024-03-21T10:22:13.668 INFO:teuthology.orchestra.run.smithi047.stdout:/var/lib/containers/storage/overlay/a670d6b1797f7b111d0422282a9f2fc87507a063f306170c368d80948f2e4034/merged 2024-03-21T10:22:13.668 INFO:teuthology.orchestra.run.smithi047.stdout:/var/lib/containers/storage/overlay/d41f794868e6e4c013a6f5180d396f28874ea4f7b53dc71022d9b672a7f62644/merged 2024-03-21T10:22:13.668 INFO:teuthology.orchestra.run.smithi047.stdout:/var/lib/containers/storage/overlay/d2f5e9cf3dce6353de30b11e940b05e7d1241191f5cbdf874907eb57db79323c/merged 2024-03-21T10:22:13.668 INFO:teuthology.orchestra.run.smithi047.stdout:/var/lib/containers/storage/overlay/decd4140254122fd73d80c489f7134415c0f4da62e417409ccd425e24cd125cd/merged 2024-03-21T10:22:13.668 INFO:teuthology.orchestra.run.smithi047.stdout:/run/netns 2024-03-21T10:22:13.668 INFO:teuthology.orchestra.run.smithi047.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-03-21T10:22:13.669 INFO:teuthology.orchestra.run.smithi047.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-03-21T10:22:13.669 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-21T10:22:13.670 DEBUG:teuthology.orchestra.run.smithi047:> ls /sys/fs/fuse/connections 2024-03-21T10:22:13.715 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-03-21T10:22:13.715 DEBUG:teuthology.orchestra.run.smithi047:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.admin --id admin --client_fs=cephfs) 2024-03-21T10:22:13.720 DEBUG:teuthology.orchestra.run.smithi047:> sudo modprobe fuse 2024-03-21T10:22:13.791 DEBUG:teuthology.orchestra.run.smithi047:> cat /proc/self/mounts | awk '{print $2}' 2024-03-21T10:22:13.834 INFO:teuthology.orchestra.run.smithi047.stdout:/proc 2024-03-21T10:22:13.835 INFO:teuthology.orchestra.run.smithi047.stdout:/sys 2024-03-21T10:22:13.835 INFO:teuthology.orchestra.run.smithi047.stdout:/dev 2024-03-21T10:22:13.835 INFO:teuthology.orchestra.run.smithi047.stdout:/sys/kernel/security 2024-03-21T10:22:13.835 INFO:teuthology.orchestra.run.smithi047.stdout:/dev/shm 2024-03-21T10:22:13.835 INFO:teuthology.orchestra.run.smithi047.stdout:/dev/pts 2024-03-21T10:22:13.835 INFO:teuthology.orchestra.run.smithi047.stdout:/run 2024-03-21T10:22:13.835 INFO:teuthology.orchestra.run.smithi047.stdout:/sys/fs/cgroup 2024-03-21T10:22:13.835 INFO:teuthology.orchestra.run.smithi047.stdout:/sys/fs/pstore 2024-03-21T10:22:13.835 INFO:teuthology.orchestra.run.smithi047.stdout:/sys/fs/bpf 2024-03-21T10:22:13.835 INFO:teuthology.orchestra.run.smithi047.stdout:/ 2024-03-21T10:22:13.835 INFO:teuthology.orchestra.run.smithi047.stdout:/sys/fs/selinux 2024-03-21T10:22:13.835 INFO:teuthology.orchestra.run.smithi047.stdout:/proc/sys/fs/binfmt_misc 2024-03-21T10:22:13.836 INFO:teuthology.orchestra.run.smithi047.stdout:/dev/mqueue 2024-03-21T10:22:13.836 INFO:teuthology.orchestra.run.smithi047.stdout:/dev/hugepages 2024-03-21T10:22:13.836 INFO:teuthology.orchestra.run.smithi047.stdout:/sys/kernel/debug 2024-03-21T10:22:13.836 INFO:teuthology.orchestra.run.smithi047.stdout:/sys/kernel/tracing 2024-03-21T10:22:13.836 INFO:teuthology.orchestra.run.smithi047.stdout:/sys/fs/fuse/connections 2024-03-21T10:22:13.836 INFO:teuthology.orchestra.run.smithi047.stdout:/sys/kernel/config 2024-03-21T10:22:13.836 INFO:teuthology.orchestra.run.smithi047.stdout:/run/credentials/systemd-sysctl.service 2024-03-21T10:22:13.836 INFO:teuthology.orchestra.run.smithi047.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-03-21T10:22:13.836 INFO:teuthology.orchestra.run.smithi047.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-03-21T10:22:13.836 INFO:teuthology.orchestra.run.smithi047.stdout:/var/lib/nfs/rpc_pipefs 2024-03-21T10:22:13.836 INFO:teuthology.orchestra.run.smithi047.stdout:/run/user/1000 2024-03-21T10:22:13.836 INFO:teuthology.orchestra.run.smithi047.stdout:/var/lib/ceph 2024-03-21T10:22:13.836 INFO:teuthology.orchestra.run.smithi047.stdout:/var/lib/containers/storage/overlay 2024-03-21T10:22:13.836 INFO:teuthology.orchestra.run.smithi047.stdout:/var/lib/containers/storage/overlay/10ab6f66d550c87573f945f803c4f92650c76c2142b200e308dd5734436cc6e8/merged 2024-03-21T10:22:13.837 INFO:teuthology.orchestra.run.smithi047.stdout:/var/lib/containers/storage/overlay/4193f3bc34f41e5fb90c176b8210c5c797208467c6f372b377075907f20e100c/merged 2024-03-21T10:22:13.837 INFO:teuthology.orchestra.run.smithi047.stdout:/run/user/0 2024-03-21T10:22:13.837 INFO:teuthology.orchestra.run.smithi047.stdout:/proc/sys/fs/binfmt_misc 2024-03-21T10:22:13.837 INFO:teuthology.orchestra.run.smithi047.stdout:/var/lib/containers/storage/overlay/fa63c9447d222c5abaf50fb70c1cc34ea356854abf3426eb0c8a7f015a95e543/merged 2024-03-21T10:22:13.837 INFO:teuthology.orchestra.run.smithi047.stdout:/var/lib/containers/storage/overlay/cab8c1c11ddc44b3f7251050e29e61c724a07774891943f3f169428cf41f19ce/merged 2024-03-21T10:22:13.837 INFO:teuthology.orchestra.run.smithi047.stdout:/var/lib/containers/storage/overlay/8bc14524d7f02db40a42ab65f069f5682514962833182f9de83d3508887ef2e7/merged 2024-03-21T10:22:13.837 INFO:teuthology.orchestra.run.smithi047.stdout:/var/lib/containers/storage/overlay/225ab22923163649720ee3c99e303986cfdc6df3ece1f25ae6bea1c7386ea563/merged 2024-03-21T10:22:13.837 INFO:teuthology.orchestra.run.smithi047.stdout:/var/lib/containers/storage/overlay/a670d6b1797f7b111d0422282a9f2fc87507a063f306170c368d80948f2e4034/merged 2024-03-21T10:22:13.837 INFO:teuthology.orchestra.run.smithi047.stdout:/var/lib/containers/storage/overlay/d41f794868e6e4c013a6f5180d396f28874ea4f7b53dc71022d9b672a7f62644/merged 2024-03-21T10:22:13.837 INFO:teuthology.orchestra.run.smithi047.stdout:/var/lib/containers/storage/overlay/d2f5e9cf3dce6353de30b11e940b05e7d1241191f5cbdf874907eb57db79323c/merged 2024-03-21T10:22:13.837 INFO:teuthology.orchestra.run.smithi047.stdout:/var/lib/containers/storage/overlay/decd4140254122fd73d80c489f7134415c0f4da62e417409ccd425e24cd125cd/merged 2024-03-21T10:22:13.837 INFO:teuthology.orchestra.run.smithi047.stdout:/run/netns 2024-03-21T10:22:13.837 INFO:teuthology.orchestra.run.smithi047.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-03-21T10:22:13.837 INFO:teuthology.orchestra.run.smithi047.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-03-21T10:22:13.838 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-21T10:22:13.838 DEBUG:teuthology.orchestra.run.smithi047:> ls /sys/fs/fuse/connections 2024-03-21T10:22:13.930 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi047.stderr:2024-03-21T10:22:13.929+0000 7f148820c480 -1 init, newargv = 0x561b5ec2d080 newargc=15 2024-03-21T10:22:13.930 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi047.stderr:ceph-fuse[73396]: starting ceph client 2024-03-21T10:22:13.946 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi047.stderr:ceph-fuse[73396]: starting fuse 2024-03-21T10:22:15.022 DEBUG:teuthology.orchestra.run.smithi047:> sudo modprobe fuse 2024-03-21T10:22:15.125 DEBUG:teuthology.orchestra.run.smithi047:> cat /proc/self/mounts | awk '{print $2}' 2024-03-21T10:22:15.170 INFO:teuthology.orchestra.run.smithi047.stdout:/proc 2024-03-21T10:22:15.170 INFO:teuthology.orchestra.run.smithi047.stdout:/sys 2024-03-21T10:22:15.170 INFO:teuthology.orchestra.run.smithi047.stdout:/dev 2024-03-21T10:22:15.170 INFO:teuthology.orchestra.run.smithi047.stdout:/sys/kernel/security 2024-03-21T10:22:15.170 INFO:teuthology.orchestra.run.smithi047.stdout:/dev/shm 2024-03-21T10:22:15.170 INFO:teuthology.orchestra.run.smithi047.stdout:/dev/pts 2024-03-21T10:22:15.170 INFO:teuthology.orchestra.run.smithi047.stdout:/run 2024-03-21T10:22:15.170 INFO:teuthology.orchestra.run.smithi047.stdout:/sys/fs/cgroup 2024-03-21T10:22:15.171 INFO:teuthology.orchestra.run.smithi047.stdout:/sys/fs/pstore 2024-03-21T10:22:15.171 INFO:teuthology.orchestra.run.smithi047.stdout:/sys/fs/bpf 2024-03-21T10:22:15.171 INFO:teuthology.orchestra.run.smithi047.stdout:/ 2024-03-21T10:22:15.171 INFO:teuthology.orchestra.run.smithi047.stdout:/sys/fs/selinux 2024-03-21T10:22:15.171 INFO:teuthology.orchestra.run.smithi047.stdout:/proc/sys/fs/binfmt_misc 2024-03-21T10:22:15.171 INFO:teuthology.orchestra.run.smithi047.stdout:/dev/mqueue 2024-03-21T10:22:15.171 INFO:teuthology.orchestra.run.smithi047.stdout:/dev/hugepages 2024-03-21T10:22:15.171 INFO:teuthology.orchestra.run.smithi047.stdout:/sys/kernel/debug 2024-03-21T10:22:15.171 INFO:teuthology.orchestra.run.smithi047.stdout:/sys/kernel/tracing 2024-03-21T10:22:15.171 INFO:teuthology.orchestra.run.smithi047.stdout:/sys/fs/fuse/connections 2024-03-21T10:22:15.171 INFO:teuthology.orchestra.run.smithi047.stdout:/sys/kernel/config 2024-03-21T10:22:15.171 INFO:teuthology.orchestra.run.smithi047.stdout:/run/credentials/systemd-sysctl.service 2024-03-21T10:22:15.171 INFO:teuthology.orchestra.run.smithi047.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-03-21T10:22:15.172 INFO:teuthology.orchestra.run.smithi047.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-03-21T10:22:15.172 INFO:teuthology.orchestra.run.smithi047.stdout:/var/lib/nfs/rpc_pipefs 2024-03-21T10:22:15.172 INFO:teuthology.orchestra.run.smithi047.stdout:/run/user/1000 2024-03-21T10:22:15.172 INFO:teuthology.orchestra.run.smithi047.stdout:/var/lib/ceph 2024-03-21T10:22:15.172 INFO:teuthology.orchestra.run.smithi047.stdout:/var/lib/containers/storage/overlay 2024-03-21T10:22:15.172 INFO:teuthology.orchestra.run.smithi047.stdout:/var/lib/containers/storage/overlay/10ab6f66d550c87573f945f803c4f92650c76c2142b200e308dd5734436cc6e8/merged 2024-03-21T10:22:15.172 INFO:teuthology.orchestra.run.smithi047.stdout:/var/lib/containers/storage/overlay/4193f3bc34f41e5fb90c176b8210c5c797208467c6f372b377075907f20e100c/merged 2024-03-21T10:22:15.172 INFO:teuthology.orchestra.run.smithi047.stdout:/run/user/0 2024-03-21T10:22:15.172 INFO:teuthology.orchestra.run.smithi047.stdout:/proc/sys/fs/binfmt_misc 2024-03-21T10:22:15.172 INFO:teuthology.orchestra.run.smithi047.stdout:/var/lib/containers/storage/overlay/fa63c9447d222c5abaf50fb70c1cc34ea356854abf3426eb0c8a7f015a95e543/merged 2024-03-21T10:22:15.172 INFO:teuthology.orchestra.run.smithi047.stdout:/var/lib/containers/storage/overlay/cab8c1c11ddc44b3f7251050e29e61c724a07774891943f3f169428cf41f19ce/merged 2024-03-21T10:22:15.172 INFO:teuthology.orchestra.run.smithi047.stdout:/var/lib/containers/storage/overlay/8bc14524d7f02db40a42ab65f069f5682514962833182f9de83d3508887ef2e7/merged 2024-03-21T10:22:15.172 INFO:teuthology.orchestra.run.smithi047.stdout:/var/lib/containers/storage/overlay/225ab22923163649720ee3c99e303986cfdc6df3ece1f25ae6bea1c7386ea563/merged 2024-03-21T10:22:15.172 INFO:teuthology.orchestra.run.smithi047.stdout:/var/lib/containers/storage/overlay/a670d6b1797f7b111d0422282a9f2fc87507a063f306170c368d80948f2e4034/merged 2024-03-21T10:22:15.172 INFO:teuthology.orchestra.run.smithi047.stdout:/var/lib/containers/storage/overlay/d41f794868e6e4c013a6f5180d396f28874ea4f7b53dc71022d9b672a7f62644/merged 2024-03-21T10:22:15.173 INFO:teuthology.orchestra.run.smithi047.stdout:/var/lib/containers/storage/overlay/d2f5e9cf3dce6353de30b11e940b05e7d1241191f5cbdf874907eb57db79323c/merged 2024-03-21T10:22:15.173 INFO:teuthology.orchestra.run.smithi047.stdout:/var/lib/containers/storage/overlay/decd4140254122fd73d80c489f7134415c0f4da62e417409ccd425e24cd125cd/merged 2024-03-21T10:22:15.173 INFO:teuthology.orchestra.run.smithi047.stdout:/run/netns 2024-03-21T10:22:15.173 INFO:teuthology.orchestra.run.smithi047.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-03-21T10:22:15.173 INFO:teuthology.orchestra.run.smithi047.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-03-21T10:22:15.173 INFO:teuthology.orchestra.run.smithi047.stdout:/home/ubuntu/cephtest/mnt.admin 2024-03-21T10:22:15.174 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-21T10:22:15.174 DEBUG:teuthology.orchestra.run.smithi047:> ls /sys/fs/fuse/connections 2024-03-21T10:22:15.226 INFO:teuthology.orchestra.run.smithi047.stdout:118 2024-03-21T10:22:15.226 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [118] 2024-03-21T10:22:15.226 DEBUG:teuthology.orchestra.run.smithi047:> sudo stdin-killer -- python3 -c ' 2024-03-21T10:22:15.226 DEBUG:teuthology.orchestra.run.smithi047:> import glob 2024-03-21T10:22:15.227 DEBUG:teuthology.orchestra.run.smithi047:> import re 2024-03-21T10:22:15.227 DEBUG:teuthology.orchestra.run.smithi047:> import os 2024-03-21T10:22:15.227 DEBUG:teuthology.orchestra.run.smithi047:> import subprocess 2024-03-21T10:22:15.227 DEBUG:teuthology.orchestra.run.smithi047:> 2024-03-21T10:22:15.227 DEBUG:teuthology.orchestra.run.smithi047:> def _find_admin_socket(client_name): 2024-03-21T10:22:15.227 DEBUG:teuthology.orchestra.run.smithi047:> asok_path = "/var/run/ceph/ceph-client.admin.*.asok" 2024-03-21T10:22:15.227 DEBUG:teuthology.orchestra.run.smithi047:> files = glob.glob(asok_path) 2024-03-21T10:22:15.227 DEBUG:teuthology.orchestra.run.smithi047:> mountpoint = "/home/ubuntu/cephtest/mnt.admin" 2024-03-21T10:22:15.227 DEBUG:teuthology.orchestra.run.smithi047:> 2024-03-21T10:22:15.227 DEBUG:teuthology.orchestra.run.smithi047:> # Given a non-glob path, it better be there 2024-03-21T10:22:15.227 DEBUG:teuthology.orchestra.run.smithi047:> if "*" not in asok_path: 2024-03-21T10:22:15.227 DEBUG:teuthology.orchestra.run.smithi047:> assert(len(files) == 1) 2024-03-21T10:22:15.227 DEBUG:teuthology.orchestra.run.smithi047:> return files[0] 2024-03-21T10:22:15.227 DEBUG:teuthology.orchestra.run.smithi047:> 2024-03-21T10:22:15.227 DEBUG:teuthology.orchestra.run.smithi047:> for f in files: 2024-03-21T10:22:15.228 DEBUG:teuthology.orchestra.run.smithi047:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-03-21T10:22:15.228 DEBUG:teuthology.orchestra.run.smithi047:> if os.path.exists("/proc/{0}".format(pid)): 2024-03-21T10:22:15.228 DEBUG:teuthology.orchestra.run.smithi047:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-03-21T10:22:15.228 DEBUG:teuthology.orchestra.run.smithi047:> contents = proc_f.read() 2024-03-21T10:22:15.228 DEBUG:teuthology.orchestra.run.smithi047:> if mountpoint in contents: 2024-03-21T10:22:15.228 DEBUG:teuthology.orchestra.run.smithi047:> return f 2024-03-21T10:22:15.228 DEBUG:teuthology.orchestra.run.smithi047:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-03-21T10:22:15.228 DEBUG:teuthology.orchestra.run.smithi047:> 2024-03-21T10:22:15.228 DEBUG:teuthology.orchestra.run.smithi047:> print(_find_admin_socket("client.admin")) 2024-03-21T10:22:15.228 DEBUG:teuthology.orchestra.run.smithi047:> ' 2024-03-21T10:22:15.410 INFO:teuthology.orchestra.run.smithi047.stdout:/var/run/ceph/ceph-client.admin.73396.asok 2024-03-21T10:22:15.414 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:15 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-21T10:22:15.425 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:22:15 smithi047 ceph-mon[31547]: pgmap v375: 129 pgs: 129 active+clean; 579 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 0 op/s 2024-03-21T10:22:15.429 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.admin.73396.asok 2024-03-21T10:22:15.429 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-21T10:22:15.429 DEBUG:teuthology.orchestra.run.smithi047:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.admin.73396.asok status 2024-03-21T10:22:15.479 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:22:15 smithi100 ceph-mon[35193]: pgmap v375: 129 pgs: 129 active+clean; 579 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 0 op/s 2024-03-21T10:22:15.489 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:22:15 smithi157 ceph-mon[35581]: pgmap v375: 129 pgs: 129 active+clean; 579 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 0 op/s 2024-03-21T10:22:15.679 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-03-21T10:22:15.679 INFO:teuthology.orchestra.run.smithi047.stdout: "metadata": { 2024-03-21T10:22:15.679 INFO:teuthology.orchestra.run.smithi047.stdout: "ceph_sha1": "2edfee68ad7243bed422a049b6f5f58619300f11", 2024-03-21T10:22:15.679 INFO:teuthology.orchestra.run.smithi047.stdout: "ceph_version": "ceph version 19.0.0-2231-g2edfee68 (2edfee68ad7243bed422a049b6f5f58619300f11) squid (dev)", 2024-03-21T10:22:15.680 INFO:teuthology.orchestra.run.smithi047.stdout: "entity_id": "admin", 2024-03-21T10:22:15.680 INFO:teuthology.orchestra.run.smithi047.stdout: "hostname": "smithi047", 2024-03-21T10:22:15.680 INFO:teuthology.orchestra.run.smithi047.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.admin", 2024-03-21T10:22:15.680 INFO:teuthology.orchestra.run.smithi047.stdout: "pid": "73396", 2024-03-21T10:22:15.680 INFO:teuthology.orchestra.run.smithi047.stdout: "root": "/" 2024-03-21T10:22:15.680 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:22:15.680 INFO:teuthology.orchestra.run.smithi047.stdout: "dentry_count": 0, 2024-03-21T10:22:15.680 INFO:teuthology.orchestra.run.smithi047.stdout: "dentry_pinned_count": 0, 2024-03-21T10:22:15.680 INFO:teuthology.orchestra.run.smithi047.stdout: "id": 14754, 2024-03-21T10:22:15.680 INFO:teuthology.orchestra.run.smithi047.stdout: "inst": { 2024-03-21T10:22:15.680 INFO:teuthology.orchestra.run.smithi047.stdout: "name": { 2024-03-21T10:22:15.680 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "client", 2024-03-21T10:22:15.680 INFO:teuthology.orchestra.run.smithi047.stdout: "num": 14754 2024-03-21T10:22:15.680 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:22:15.681 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": { 2024-03-21T10:22:15.681 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-03-21T10:22:15.681 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "192.168.0.1:0", 2024-03-21T10:22:15.681 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 2770422856 2024-03-21T10:22:15.681 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-21T10:22:15.681 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:22:15.681 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": { 2024-03-21T10:22:15.681 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-03-21T10:22:15.681 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "192.168.0.1:0", 2024-03-21T10:22:15.681 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 2770422856 2024-03-21T10:22:15.681 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:22:15.681 INFO:teuthology.orchestra.run.smithi047.stdout: "inst_str": "client.14754 192.168.0.1:0/2770422856", 2024-03-21T10:22:15.681 INFO:teuthology.orchestra.run.smithi047.stdout: "addr_str": "192.168.0.1:0/2770422856", 2024-03-21T10:22:15.681 INFO:teuthology.orchestra.run.smithi047.stdout: "inode_count": 1, 2024-03-21T10:22:15.682 INFO:teuthology.orchestra.run.smithi047.stdout: "mds_epoch": 17, 2024-03-21T10:22:15.682 INFO:teuthology.orchestra.run.smithi047.stdout: "osd_epoch": 91, 2024-03-21T10:22:15.682 INFO:teuthology.orchestra.run.smithi047.stdout: "osd_epoch_barrier": 0, 2024-03-21T10:22:15.682 INFO:teuthology.orchestra.run.smithi047.stdout: "blocklisted": false, 2024-03-21T10:22:15.682 INFO:teuthology.orchestra.run.smithi047.stdout: "fs_name": "cephfs" 2024-03-21T10:22:15.682 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-03-21T10:22:15.695 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-21T10:22:15.695 DEBUG:teuthology.orchestra.run.smithi047:> stat --file-system '--printf=%T 2024-03-21T10:22:15.695 DEBUG:teuthology.orchestra.run.smithi047:> ' -- /home/ubuntu/cephtest/mnt.admin 2024-03-21T10:22:15.750 INFO:teuthology.orchestra.run.smithi047.stdout:fuseblk 2024-03-21T10:22:15.751 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.admin 2024-03-21T10:22:15.751 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-21T10:22:15.751 DEBUG:teuthology.orchestra.run.smithi047:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.admin 2024-03-21T10:22:15.847 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-21T10:22:15.847 DEBUG:teuthology.orchestra.run.smithi047:> (cd /home/ubuntu/cephtest/mnt.admin && exec bash -c 'setfattr -n ceph.dir.layout.pool -v cephfs_data_ec . && getfattr -n ceph.dir.layout .') 2024-03-21T10:22:16.004 INFO:teuthology.orchestra.run.smithi047.stdout:# file: . 2024-03-21T10:22:16.004 INFO:teuthology.orchestra.run.smithi047.stdout:ceph.dir.layout="stripe_unit=4194304 stripe_count=1 object_size=4194304 pool=cephfs_data_ec" 2024-03-21T10:22:16.004 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-21T10:22:16.006 DEBUG:teuthology.orchestra.run.smithi047:> set -ex 2024-03-21T10:22:16.006 DEBUG:teuthology.orchestra.run.smithi047:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-21T10:22:16.063 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-21T10:22:16.289 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:16.288+0000 7f5165400640 1 -- 172.21.15.47:0/3477084186 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f51601036f0 msgr2=0x7f5160103b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:22:16.290 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:16.288+0000 7f5165400640 1 --2- 172.21.15.47:0/3477084186 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f51601036f0 0x7f5160103b10 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f5150009900 tx=0x7f515002bc30 comp rx=0 tx=0).stop 2024-03-21T10:22:16.290 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:16.288+0000 7f5165400640 1 -- 172.21.15.47:0/3477084186 shutdown_connections 2024-03-21T10:22:16.290 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:16.288+0000 7f5165400640 1 --2- 172.21.15.47:0/3477084186 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f51601052a0 0x7f516010bb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:22:16.290 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:16.288+0000 7f5165400640 1 --2- 172.21.15.47:0/3477084186 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f51601048e0 0x7f5160104d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:22:16.290 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:16.288+0000 7f5165400640 1 --2- 172.21.15.47:0/3477084186 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f51601036f0 0x7f5160103b10 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:22:16.290 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:16.288+0000 7f5165400640 1 -- 172.21.15.47:0/3477084186 >> 172.21.15.47:0/3477084186 conn(0x7f51600fec80 msgr2=0x7f51601010a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:22:16.290 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:16.288+0000 7f5165400640 1 -- 172.21.15.47:0/3477084186 shutdown_connections 2024-03-21T10:22:16.290 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:16.288+0000 7f5165400640 1 -- 172.21.15.47:0/3477084186 wait complete. 2024-03-21T10:22:16.292 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:16.289+0000 7f5165400640 1 Processor -- start 2024-03-21T10:22:16.292 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:16.289+0000 7f5165400640 1 -- start start 2024-03-21T10:22:16.293 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:16.290+0000 7f5165400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f51601036f0 0x7f5160199920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:22:16.293 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:16.290+0000 7f5165400640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f51601048e0 0x7f5160199e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:22:16.293 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:16.290+0000 7f5165400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f51601052a0 0x7f51601a0ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:22:16.293 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:16.290+0000 7f5165400640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f516010f240 con 0x7f51601036f0 2024-03-21T10:22:16.293 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:16.290+0000 7f5165400640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f516010f0c0 con 0x7f51601052a0 2024-03-21T10:22:16.293 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:16.290+0000 7f5165400640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f516010f3c0 con 0x7f51601048e0 2024-03-21T10:22:16.293 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:16.290+0000 7f515fe00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f51601036f0 0x7f5160199920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:22:16.293 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:16.290+0000 7f515fe00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f51601036f0 0x7f5160199920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:51602/0 (socket says 172.21.15.47:51602) 2024-03-21T10:22:16.293 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:16.290+0000 7f515fe00640 1 -- 172.21.15.47:0/848189204 learned_addr learned my addr 172.21.15.47:0/848189204 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:22:16.293 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:16.290+0000 7f515f400640 1 --2- 172.21.15.47:0/848189204 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f51601048e0 0x7f5160199e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:22:16.293 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:16.290+0000 7f5164a00640 1 --2- 172.21.15.47:0/848189204 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f51601052a0 0x7f51601a0ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:22:16.293 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:16.291+0000 7f515fe00640 1 -- 172.21.15.47:0/848189204 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f51601048e0 msgr2=0x7f5160199e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:22:16.293 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:16.291+0000 7f515fe00640 1 --2- 172.21.15.47:0/848189204 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f51601048e0 0x7f5160199e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:22:16.294 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:16.291+0000 7f515fe00640 1 -- 172.21.15.47:0/848189204 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f51601052a0 msgr2=0x7f51601a0ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:22:16.294 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:16.291+0000 7f515fe00640 1 --2- 172.21.15.47:0/848189204 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f51601052a0 0x7f51601a0ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:22:16.294 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:16.291+0000 7f515fe00640 1 -- 172.21.15.47:0/848189204 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51601a1540 con 0x7f51601036f0 2024-03-21T10:22:16.294 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:16.291+0000 7f5164a00640 1 --2- 172.21.15.47:0/848189204 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f51601052a0 0x7f51601a0ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T10:22:16.294 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:16.291+0000 7f515fe00640 1 --2- 172.21.15.47:0/848189204 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f51601036f0 0x7f5160199920 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f5150005ab0 tx=0x7f51500042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:22:16.294 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:16.291+0000 7f515cc00640 1 -- 172.21.15.47:0/848189204 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5150043020 con 0x7f51601036f0 2024-03-21T10:22:16.294 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:16.291+0000 7f515cc00640 1 -- 172.21.15.47:0/848189204 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f51500054c0 con 0x7f51601036f0 2024-03-21T10:22:16.294 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:16.291+0000 7f515cc00640 1 -- 172.21.15.47:0/848189204 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5150003d10 con 0x7f51601036f0 2024-03-21T10:22:16.294 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:16.291+0000 7f5165400640 1 -- 172.21.15.47:0/848189204 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f51601a17d0 con 0x7f51601036f0 2024-03-21T10:22:16.294 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:16.292+0000 7f5165400640 1 -- 172.21.15.47:0/848189204 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f51601a1d20 con 0x7f51601036f0 2024-03-21T10:22:16.294 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:16.293+0000 7f5165400640 1 -- 172.21.15.47:0/848189204 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f516010aa30 con 0x7f51601036f0 2024-03-21T10:22:16.295 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:16.295+0000 7f515cc00640 1 -- 172.21.15.47:0/848189204 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 16) v1 ==== 137854+0+0 (secure 0 0 0) 0x7f5150046020 con 0x7f51601036f0 2024-03-21T10:22:16.297 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:16.296+0000 7f515cc00640 1 --2- 172.21.15.47:0/848189204 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f51440a2f20 0x7f51440a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:22:16.297 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:16.296+0000 7f515cc00640 1 -- 172.21.15.47:0/848189204 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 8802+0+0 (secure 0 0 0) 0x7f51500f1e70 con 0x7f51601036f0 2024-03-21T10:22:16.297 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:16.296+0000 7f515f400640 1 --2- 172.21.15.47:0/848189204 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f51440a2f20 0x7f51440a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:22:16.298 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:16.297+0000 7f515f400640 1 --2- 172.21.15.47:0/848189204 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f51440a2f20 0x7f51440a53e0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f514c005fd0 tx=0x7f514c005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:22:16.300 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:16.299+0000 7f515cc00640 1 -- 172.21.15.47:0/848189204 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f515004d020 con 0x7f51601036f0 2024-03-21T10:22:16.506 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:16.505+0000 7f5165400640 1 -- 172.21.15.47:0/848189204 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f51600622a0 con 0x7f51601036f0 2024-03-21T10:22:16.510 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:16.509+0000 7f515cc00640 1 -- 172.21.15.47:0/848189204 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v91) v1 ==== 74+0+24116 (secure 0 0 0) 0x7f51500bea00 con 0x7f51601036f0 2024-03-21T10:22:16.511 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-21T10:22:16.511 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":91,"fsid":"3056d51a-e76b-11ee-95cd-87774f69a715","created":"2024-03-21T10:11:17.019965+0000","modified":"2024-03-21T10:22:03.759214+0000","last_up_change":"2024-03-21T10:20:52.675328+0000","last_in_change":"2024-03-21T10:20:27.648212+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-21T10:15:40.553114+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-21T10:21:56.399518+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-21T10:21:57.812940+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-21T10:22:01.822568+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"91","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"b0c7a102-8d89-428b-b793-cbbe67151f50","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6802","nonce":775837313},{"type":"v1","addr":"172.21.15.47:6803","nonce":775837313}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6804","nonce":775837313},{"type":"v1","addr":"172.21.15.47:6805","nonce":775837313}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6808","nonce":775837313},{"type":"v1","addr":"172.21.15.47:6809","nonce":775837313}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6806","nonce":775837313},{"type":"v1","addr":"172.21.15.47:6807","nonce":775837313}]},"public_addr":"172.21.15.47:6803/775837313","cluster_addr":"172.21.15.47:6805/775837313","heartbeat_back_addr":"172.21.15.47:6809/775837313","heartbeat_front_addr":"172.21.15.47:6807/775837313","state":["exists","up"]},{"osd":1,"uuid":"07bb81bb-7cf3-4bbc-ac9e-802f95185d11","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6800","nonce":2923304710},{"type":"v1","addr":"172.21.15.100:6801","nonce":2923304710}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":2923304710},{"type":"v1","addr":"172.21.15.100:6803","nonce":2923304710}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":2923304710},{"type":"v1","addr":"172.21.15.100:6807","nonce":2923304710}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6804","nonce":2923304710},{"type":"v1","addr":"172.21.15.100:6805","nonce":2923304710}]},"public_addr":"172.21.15.100:6801/2923304710","cluster_addr":"172.21.15.100:6803/2923304710","heartbeat_back_addr":"172.21.15.100:6807/2923304710","heartbeat_front_addr":"172.21.15.100:6805/2923304710","state":["exists","up"]},{"osd":2,"uuid":"88ce76b9-6081-4836-aad0-8c249c56e8a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":171377005},{"type":"v1","addr":"172.21.15.157:6801","nonce":171377005}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":171377005},{"type":"v1","addr":"172.21.15.157:6803","nonce":171377005}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":171377005},{"type":"v1","addr":"172.21.15.157:6807","nonce":171377005}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":171377005},{"type":"v1","addr":"172.21.15.157:6805","nonce":171377005}]},"public_addr":"172.21.15.157:6801/171377005","cluster_addr":"172.21.15.157:6803/171377005","heartbeat_back_addr":"172.21.15.157:6807/171377005","heartbeat_front_addr":"172.21.15.157:6805/171377005","state":["exists","up"]},{"osd":3,"uuid":"518aa33d-263f-45e6-aab5-2201ceb40e74","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":27,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6810","nonce":640760101},{"type":"v1","addr":"172.21.15.47:6811","nonce":640760101}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6812","nonce":640760101},{"type":"v1","addr":"172.21.15.47:6813","nonce":640760101}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6816","nonce":640760101},{"type":"v1","addr":"172.21.15.47:6817","nonce":640760101}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6814","nonce":640760101},{"type":"v1","addr":"172.21.15.47:6815","nonce":640760101}]},"public_addr":"172.21.15.47:6811/640760101","cluster_addr":"172.21.15.47:6813/640760101","heartbeat_back_addr":"172.21.15.47:6817/640760101","heartbeat_front_addr":"172.21.15.47:6815/640760101","state":["exists","up"]},{"osd":4,"uuid":"987bde83-ecc6-42e5-bd56-c143b3de4226","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":35,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6808","nonce":2785478639},{"type":"v1","addr":"172.21.15.100:6809","nonce":2785478639}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6810","nonce":2785478639},{"type":"v1","addr":"172.21.15.100:6811","nonce":2785478639}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6814","nonce":2785478639},{"type":"v1","addr":"172.21.15.100:6815","nonce":2785478639}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":2785478639},{"type":"v1","addr":"172.21.15.100:6813","nonce":2785478639}]},"public_addr":"172.21.15.100:6809/2785478639","cluster_addr":"172.21.15.100:6811/2785478639","heartbeat_back_addr":"172.21.15.100:6815/2785478639","heartbeat_front_addr":"172.21.15.100:6813/2785478639","state":["exists","up"]},{"osd":5,"uuid":"012d2188-3a61-4a5f-8d01-2001c15ffbe2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":40,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":1525645410},{"type":"v1","addr":"172.21.15.157:6809","nonce":1525645410}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":1525645410},{"type":"v1","addr":"172.21.15.157:6811","nonce":1525645410}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":1525645410},{"type":"v1","addr":"172.21.15.157:6815","nonce":1525645410}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":1525645410},{"type":"v1","addr":"172.21.15.157:6813","nonce":1525645410}]},"public_addr":"172.21.15.157:6809/1525645410","cluster_addr":"172.21.15.157:6811/1525645410","heartbeat_back_addr":"172.21.15.157:6815/1525645410","heartbeat_front_addr":"172.21.15.157:6813/1525645410","state":["exists","up"]},{"osd":6,"uuid":"c12c14b4-85f0-4d88-937c-de4f556437de","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6818","nonce":2321642790},{"type":"v1","addr":"172.21.15.47:6819","nonce":2321642790}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6820","nonce":2321642790},{"type":"v1","addr":"172.21.15.47:6821","nonce":2321642790}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6824","nonce":2321642790},{"type":"v1","addr":"172.21.15.47:6825","nonce":2321642790}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6822","nonce":2321642790},{"type":"v1","addr":"172.21.15.47:6823","nonce":2321642790}]},"public_addr":"172.21.15.47:6819/2321642790","cluster_addr":"172.21.15.47:6821/2321642790","heartbeat_back_addr":"172.21.15.47:6825/2321642790","heartbeat_front_addr":"172.21.15.47:6823/2321642790","state":["exists","up"]},{"osd":7,"uuid":"3c87c6d0-1d1f-4294-9917-aaaad7dda824","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6816","nonce":4104727330},{"type":"v1","addr":"172.21.15.100:6817","nonce":4104727330}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":4104727330},{"type":"v1","addr":"172.21.15.100:6819","nonce":4104727330}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":4104727330},{"type":"v1","addr":"172.21.15.100:6823","nonce":4104727330}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6820","nonce":4104727330},{"type":"v1","addr":"172.21.15.100:6821","nonce":4104727330}]},"public_addr":"172.21.15.100:6817/4104727330","cluster_addr":"172.21.15.100:6819/4104727330","heartbeat_back_addr":"172.21.15.100:6823/4104727330","heartbeat_front_addr":"172.21.15.100:6821/4104727330","state":["exists","up"]},{"osd":8,"uuid":"502d65cf-53c8-4936-8de4-e11b2a6e1e18","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":61,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":4067715347},{"type":"v1","addr":"172.21.15.157:6817","nonce":4067715347}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":4067715347},{"type":"v1","addr":"172.21.15.157:6819","nonce":4067715347}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":4067715347},{"type":"v1","addr":"172.21.15.157:6823","nonce":4067715347}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":4067715347},{"type":"v1","addr":"172.21.15.157:6821","nonce":4067715347}]},"public_addr":"172.21.15.157:6817/4067715347","cluster_addr":"172.21.15.157:6819/4067715347","heartbeat_back_addr":"172.21.15.157:6823/4067715347","heartbeat_front_addr":"172.21.15.157:6821/4067715347","state":["exists","up"]},{"osd":9,"uuid":"5ea1f5be-8458-4c93-9e61-c2e867a514f3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6826","nonce":1982108738},{"type":"v1","addr":"172.21.15.47:6827","nonce":1982108738}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6828","nonce":1982108738},{"type":"v1","addr":"172.21.15.47:6829","nonce":1982108738}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6832","nonce":1982108738},{"type":"v1","addr":"172.21.15.47:6833","nonce":1982108738}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6830","nonce":1982108738},{"type":"v1","addr":"172.21.15.47:6831","nonce":1982108738}]},"public_addr":"172.21.15.47:6827/1982108738","cluster_addr":"172.21.15.47:6829/1982108738","heartbeat_back_addr":"172.21.15.47:6833/1982108738","heartbeat_front_addr":"172.21.15.47:6831/1982108738","state":["exists","up"]},{"osd":10,"uuid":"2da3da90-a0be-4a54-a9ff-b2d6c9b36870","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":74,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":396391028},{"type":"v1","addr":"172.21.15.100:6825","nonce":396391028}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":396391028},{"type":"v1","addr":"172.21.15.100:6827","nonce":396391028}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6830","nonce":396391028},{"type":"v1","addr":"172.21.15.100:6831","nonce":396391028}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":396391028},{"type":"v1","addr":"172.21.15.100:6829","nonce":396391028}]},"public_addr":"172.21.15.100:6825/396391028","cluster_addr":"172.21.15.100:6827/396391028","heartbeat_back_addr":"172.21.15.100:6831/396391028","heartbeat_front_addr":"172.21.15.100:6829/396391028","state":["exists","up"]},{"osd":11,"uuid":"11798d41-7f7e-4319-95b8-8d3d0946bde9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":82,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":2671029483},{"type":"v1","addr":"172.21.15.157:6825","nonce":2671029483}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":2671029483},{"type":"v1","addr":"172.21.15.157:6827","nonce":2671029483}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":2671029483},{"type":"v1","addr":"172.21.15.157:6831","nonce":2671029483}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":2671029483},{"type":"v1","addr":"172.21.15.157:6829","nonce":2671029483}]},"public_addr":"172.21.15.157:6825/2671029483","cluster_addr":"172.21.15.157:6827/2671029483","heartbeat_back_addr":"172.21.15.157:6831/2671029483","heartbeat_front_addr":"172.21.15.157:6829/2671029483","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T10:14:31.822399+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T10:15:04.175721+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T10:15:35.902595+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T10:16:12.088891+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T10:16:46.428704+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T10:17:20.158270+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T10:17:55.915253+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T10:18:30.064641+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T10:19:04.410004+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T10:19:40.605181+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T10:20:15.636605+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T10:20:50.151365+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.47:0/356822431":"2024-03-22T10:12:25.844530+0000","172.21.15.47:0/1167490879":"2024-03-22T10:12:25.844530+0000","172.21.15.47:0/1540718138":"2024-03-22T10:12:25.844530+0000","172.21.15.47:6801/3836903972":"2024-03-22T10:12:25.844530+0000","172.21.15.47:0/235436602":"2024-03-22T10:11:50.301838+0000","172.21.15.47:0/2052815572":"2024-03-22T10:11:50.301838+0000","172.21.15.47:6800/3942658432":"2024-03-22T10:11:50.301838+0000","172.21.15.47:6801/3942658432":"2024-03-22T10:11:50.301838+0000","172.21.15.47:0/1419516257":"2024-03-22T10:11:50.301838+0000","172.21.15.47:6800/3836903972":"2024-03-22T10:12:25.844530+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-21T10:22:16.513 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:16.512+0000 7f5165400640 1 -- 172.21.15.47:0/848189204 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f51440a2f20 msgr2=0x7f51440a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:22:16.513 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:16.512+0000 7f5165400640 1 --2- 172.21.15.47:0/848189204 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f51440a2f20 0x7f51440a53e0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f514c005fd0 tx=0x7f514c005850 comp rx=0 tx=0).stop 2024-03-21T10:22:16.513 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:16.512+0000 7f5165400640 1 -- 172.21.15.47:0/848189204 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f51601036f0 msgr2=0x7f5160199920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:22:16.513 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:16.512+0000 7f5165400640 1 --2- 172.21.15.47:0/848189204 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f51601036f0 0x7f5160199920 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f5150005ab0 tx=0x7f51500042f0 comp rx=0 tx=0).stop 2024-03-21T10:22:16.513 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:16.512+0000 7f5165400640 1 -- 172.21.15.47:0/848189204 shutdown_connections 2024-03-21T10:22:16.513 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:16.512+0000 7f5165400640 1 --2- 172.21.15.47:0/848189204 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f51440a2f20 0x7f51440a53e0 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:22:16.513 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:16.512+0000 7f5165400640 1 --2- 172.21.15.47:0/848189204 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f51601052a0 0x7f51601a0ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:22:16.513 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:16.512+0000 7f5165400640 1 --2- 172.21.15.47:0/848189204 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f51601048e0 0x7f5160199e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:22:16.513 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:16.512+0000 7f5165400640 1 --2- 172.21.15.47:0/848189204 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f51601036f0 0x7f5160199920 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:22:16.513 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:16.512+0000 7f5165400640 1 -- 172.21.15.47:0/848189204 >> 172.21.15.47:0/848189204 conn(0x7f51600fec80 msgr2=0x7f5160107c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:22:16.514 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:16.512+0000 7f5165400640 1 -- 172.21.15.47:0/848189204 shutdown_connections 2024-03-21T10:22:16.514 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:16.512+0000 7f5165400640 1 -- 172.21.15.47:0/848189204 wait complete. 2024-03-21T10:22:16.533 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-21T10:15:40.553114+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-21T10:21:56.399518+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '86', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-21T10:21:57.812940+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '87', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-03-21T10:22:01.822568+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '91', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-03-21T10:22:16.533 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-21T10:22:16.759 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:16.758+0000 7fc2ed400640 1 -- 172.21.15.47:0/2887278887 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc2e8103150 msgr2=0x7fc2e81099e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:22:16.759 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:16.758+0000 7fc2ed400640 1 --2- 172.21.15.47:0/2887278887 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc2e8103150 0x7fc2e81099e0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fc2e0009ca0 tx=0x7fc2e002c0f0 comp rx=0 tx=0).stop 2024-03-21T10:22:16.760 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:16.758+0000 7fc2ed400640 1 -- 172.21.15.47:0/2887278887 shutdown_connections 2024-03-21T10:22:16.760 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:16.758+0000 7fc2ed400640 1 --2- 172.21.15.47:0/2887278887 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc2e8103150 0x7fc2e81099e0 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:22:16.760 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:16.758+0000 7fc2ed400640 1 --2- 172.21.15.47:0/2887278887 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc2e8102790 0x7fc2e8102c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:22:16.760 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:16.758+0000 7fc2ed400640 1 --2- 172.21.15.47:0/2887278887 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc2e81015a0 0x7fc2e81019c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:22:16.760 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:16.758+0000 7fc2ed400640 1 -- 172.21.15.47:0/2887278887 >> 172.21.15.47:0/2887278887 conn(0x7fc2e80fcb10 msgr2=0x7fc2e80fef50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:22:16.760 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:16.758+0000 7fc2ed400640 1 -- 172.21.15.47:0/2887278887 shutdown_connections 2024-03-21T10:22:16.760 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:16.758+0000 7fc2ed400640 1 -- 172.21.15.47:0/2887278887 wait complete. 2024-03-21T10:22:16.763 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:16.759+0000 7fc2ed400640 1 Processor -- start 2024-03-21T10:22:16.763 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:16.759+0000 7fc2ed400640 1 -- start start 2024-03-21T10:22:16.763 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:16.760+0000 7fc2ed400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc2e81015a0 0x7fc2e8197710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:22:16.763 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:16.760+0000 7fc2ed400640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc2e8102790 0x7fc2e8197c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:22:16.763 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:16.760+0000 7fc2ed400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc2e8103150 0x7fc2e819ecc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:22:16.764 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:16.760+0000 7fc2ed400640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2e810d0e0 con 0x7fc2e8103150 2024-03-21T10:22:16.764 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:16.760+0000 7fc2ed400640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2e810cf60 con 0x7fc2e81015a0 2024-03-21T10:22:16.764 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:16.760+0000 7fc2ed400640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2e810d260 con 0x7fc2e8102790 2024-03-21T10:22:16.764 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:16.760+0000 7fc2eca00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc2e8103150 0x7fc2e819ecc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:22:16.764 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:16.760+0000 7fc2eca00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc2e8103150 0x7fc2e819ecc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:51628/0 (socket says 172.21.15.47:51628) 2024-03-21T10:22:16.764 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:16.760+0000 7fc2eca00640 1 -- 172.21.15.47:0/2887231860 learned_addr learned my addr 172.21.15.47:0/2887231860 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:22:16.764 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:16.760+0000 7fc2df400640 1 --2- 172.21.15.47:0/2887231860 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc2e8102790 0x7fc2e8197c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:22:16.764 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:16.760+0000 7fc2e7e00640 1 --2- 172.21.15.47:0/2887231860 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc2e81015a0 0x7fc2e8197710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:22:16.764 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:16.761+0000 7fc2eca00640 1 -- 172.21.15.47:0/2887231860 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc2e8102790 msgr2=0x7fc2e8197c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:22:16.765 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:16.761+0000 7fc2eca00640 1 --2- 172.21.15.47:0/2887231860 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc2e8102790 0x7fc2e8197c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:22:16.765 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:16.761+0000 7fc2eca00640 1 -- 172.21.15.47:0/2887231860 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc2e81015a0 msgr2=0x7fc2e8197710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:22:16.765 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:16.761+0000 7fc2eca00640 1 --2- 172.21.15.47:0/2887231860 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc2e81015a0 0x7fc2e8197710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:22:16.765 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:16.761+0000 7fc2eca00640 1 -- 172.21.15.47:0/2887231860 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc2e819f330 con 0x7fc2e8103150 2024-03-21T10:22:16.765 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:16.761+0000 7fc2e7e00640 1 --2- 172.21.15.47:0/2887231860 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc2e81015a0 0x7fc2e8197710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T10:22:16.765 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:16.761+0000 7fc2eca00640 1 --2- 172.21.15.47:0/2887231860 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc2e8103150 0x7fc2e819ecc0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fc2e0032380 tx=0x7fc2e0032820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:22:16.765 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:16.761+0000 7fc2e5600640 1 -- 172.21.15.47:0/2887231860 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2e0041020 con 0x7fc2e8103150 2024-03-21T10:22:16.765 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:16.761+0000 7fc2e5600640 1 -- 172.21.15.47:0/2887231860 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fc2e0003e20 con 0x7fc2e8103150 2024-03-21T10:22:16.765 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:16.761+0000 7fc2e5600640 1 -- 172.21.15.47:0/2887231860 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2e0039040 con 0x7fc2e8103150 2024-03-21T10:22:16.766 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:16.762+0000 7fc2ed400640 1 -- 172.21.15.47:0/2887231860 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc2e819f5c0 con 0x7fc2e8103150 2024-03-21T10:22:16.766 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:16.762+0000 7fc2ed400640 1 -- 172.21.15.47:0/2887231860 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc2e819fa60 con 0x7fc2e8103150 2024-03-21T10:22:16.766 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:16.763+0000 7fc2ed400640 1 -- 172.21.15.47:0/2887231860 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc2e81088c0 con 0x7fc2e8103150 2024-03-21T10:22:16.766 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:16.765+0000 7fc2e5600640 1 -- 172.21.15.47:0/2887231860 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 16) v1 ==== 137854+0+0 (secure 0 0 0) 0x7fc2e0044020 con 0x7fc2e8103150 2024-03-21T10:22:16.768 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:16.766+0000 7fc2e5600640 1 --2- 172.21.15.47:0/2887231860 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7fc2cc0a2f20 0x7fc2cc0a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:22:16.768 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:16.766+0000 7fc2e5600640 1 -- 172.21.15.47:0/2887231860 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 8802+0+0 (secure 0 0 0) 0x7fc2e00f1950 con 0x7fc2e8103150 2024-03-21T10:22:16.768 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:16.767+0000 7fc2e7e00640 1 --2- 172.21.15.47:0/2887231860 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7fc2cc0a2f20 0x7fc2cc0a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:22:16.768 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:16.767+0000 7fc2e7e00640 1 --2- 172.21.15.47:0/2887231860 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7fc2cc0a2f20 0x7fc2cc0a53e0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fc2e81025f0 tx=0x7fc2d8005e70 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:22:16.772 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:16.771+0000 7fc2e5600640 1 -- 172.21.15.47:0/2887231860 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fc2e004b020 con 0x7fc2e8103150 2024-03-21T10:22:16.977 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:16.976+0000 7fc2ed400640 1 -- 172.21.15.47:0/2887231860 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fc2e80622a0 con 0x7fc2e8103150 2024-03-21T10:22:16.978 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:16.977+0000 7fc2e5600640 1 -- 172.21.15.47:0/2887231860 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v91) v1 ==== 93+0+10 (secure 0 0 0) 0x7fc2e00be4e0 con 0x7fc2e8103150 2024-03-21T10:22:16.978 INFO:teuthology.orchestra.run.smithi047.stdout:pg_num: 1 2024-03-21T10:22:16.982 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:16.980+0000 7fc2ed400640 1 -- 172.21.15.47:0/2887231860 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7fc2cc0a2f20 msgr2=0x7fc2cc0a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:22:16.982 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:16.980+0000 7fc2ed400640 1 --2- 172.21.15.47:0/2887231860 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7fc2cc0a2f20 0x7fc2cc0a53e0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fc2e81025f0 tx=0x7fc2d8005e70 comp rx=0 tx=0).stop 2024-03-21T10:22:16.982 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:16.980+0000 7fc2ed400640 1 -- 172.21.15.47:0/2887231860 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc2e8103150 msgr2=0x7fc2e819ecc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:22:16.982 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:16.980+0000 7fc2ed400640 1 --2- 172.21.15.47:0/2887231860 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc2e8103150 0x7fc2e819ecc0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fc2e0032380 tx=0x7fc2e0032820 comp rx=0 tx=0).stop 2024-03-21T10:22:16.982 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:16.980+0000 7fc2ed400640 1 -- 172.21.15.47:0/2887231860 shutdown_connections 2024-03-21T10:22:16.982 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:16.980+0000 7fc2ed400640 1 --2- 172.21.15.47:0/2887231860 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7fc2cc0a2f20 0x7fc2cc0a53e0 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:22:16.983 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:16.980+0000 7fc2ed400640 1 --2- 172.21.15.47:0/2887231860 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc2e8103150 0x7fc2e819ecc0 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:22:16.983 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:16.980+0000 7fc2ed400640 1 --2- 172.21.15.47:0/2887231860 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc2e8102790 0x7fc2e8197c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:22:16.983 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:16.980+0000 7fc2ed400640 1 --2- 172.21.15.47:0/2887231860 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc2e81015a0 0x7fc2e8197710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:22:16.983 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:16.980+0000 7fc2ed400640 1 -- 172.21.15.47:0/2887231860 >> 172.21.15.47:0/2887231860 conn(0x7fc2e80fcb10 msgr2=0x7fc2e8105a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:22:16.983 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:16.980+0000 7fc2ed400640 1 -- 172.21.15.47:0/2887231860 shutdown_connections 2024-03-21T10:22:16.983 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:16.980+0000 7fc2ed400640 1 -- 172.21.15.47:0/2887231860 wait complete. 2024-03-21T10:22:17.000 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-21T10:22:17.223 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:17.221+0000 7f2eaf000640 1 -- 172.21.15.47:0/1230099313 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2ea8107260 msgr2=0x7f2ea81096b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:22:17.223 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:17.221+0000 7f2eaf000640 1 --2- 172.21.15.47:0/1230099313 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2ea8107260 0x7f2ea81096b0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f2ea0009900 tx=0x7f2ea002bc30 comp rx=0 tx=0).stop 2024-03-21T10:22:17.223 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:17.221+0000 7f2eaf000640 1 -- 172.21.15.47:0/1230099313 shutdown_connections 2024-03-21T10:22:17.223 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:17.221+0000 7f2eaf000640 1 --2- 172.21.15.47:0/1230099313 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2ea8107260 0x7f2ea81096b0 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:22:17.223 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:17.221+0000 7f2eaf000640 1 --2- 172.21.15.47:0/1230099313 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2ea8104930 0x7f2ea8106d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:22:17.223 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:17.221+0000 7f2eaf000640 1 --2- 172.21.15.47:0/1230099313 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2ea8102000 0x7f2ea81043f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:22:17.223 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:17.221+0000 7f2eaf000640 1 -- 172.21.15.47:0/1230099313 >> 172.21.15.47:0/1230099313 conn(0x7f2ea80fba10 msgr2=0x7f2ea80fde30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:22:17.223 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:17.221+0000 7f2eaf000640 1 -- 172.21.15.47:0/1230099313 shutdown_connections 2024-03-21T10:22:17.223 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:17.221+0000 7f2eaf000640 1 -- 172.21.15.47:0/1230099313 wait complete. 2024-03-21T10:22:17.225 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:17.222+0000 7f2eaf000640 1 Processor -- start 2024-03-21T10:22:17.226 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:17.222+0000 7f2eaf000640 1 -- start start 2024-03-21T10:22:17.226 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:17.223+0000 7f2eaf000640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2ea8102000 0x7f2ea8199980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:22:17.226 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:17.223+0000 7f2eaf000640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2ea8104930 0x7f2ea8199ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:22:17.226 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:17.223+0000 7f2eaf000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2ea8107260 0x7f2ea81a0f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:22:17.226 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:17.223+0000 7f2eaf000640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ea810d110 con 0x7f2ea8102000 2024-03-21T10:22:17.226 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:17.223+0000 7f2eaf000640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ea810cf90 con 0x7f2ea8107260 2024-03-21T10:22:17.226 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:17.223+0000 7f2eaf000640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ea810d290 con 0x7f2ea8104930 2024-03-21T10:22:17.226 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:17.223+0000 7f2eadc00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2ea8102000 0x7f2ea8199980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:22:17.226 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:17.223+0000 7f2eadc00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2ea8102000 0x7f2ea8199980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:51654/0 (socket says 172.21.15.47:51654) 2024-03-21T10:22:17.226 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:17.223+0000 7f2eadc00640 1 -- 172.21.15.47:0/2620023211 learned_addr learned my addr 172.21.15.47:0/2620023211 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:22:17.226 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:17.223+0000 7f2ead200640 1 --2- 172.21.15.47:0/2620023211 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2ea8104930 0x7f2ea8199ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:22:17.226 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:17.223+0000 7f2eadc00640 1 -- 172.21.15.47:0/2620023211 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2ea8104930 msgr2=0x7f2ea8199ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:22:17.227 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:17.223+0000 7f2eadc00640 1 --2- 172.21.15.47:0/2620023211 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2ea8104930 0x7f2ea8199ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:22:17.227 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:17.223+0000 7f2eadc00640 1 -- 172.21.15.47:0/2620023211 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2ea8107260 msgr2=0x7f2ea81a0f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:22:17.227 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:17.223+0000 7f2eae600640 1 --2- 172.21.15.47:0/2620023211 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2ea8107260 0x7f2ea81a0f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:22:17.227 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:17.223+0000 7f2eadc00640 1 --2- 172.21.15.47:0/2620023211 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2ea8107260 0x7f2ea81a0f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:22:17.227 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:17.223+0000 7f2eadc00640 1 -- 172.21.15.47:0/2620023211 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ea81a15a0 con 0x7f2ea8102000 2024-03-21T10:22:17.227 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:17.224+0000 7f2eadc00640 1 --2- 172.21.15.47:0/2620023211 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2ea8102000 0x7f2ea8199980 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f2e9c00eab0 tx=0x7f2e9c00ef70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:22:17.227 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:17.224+0000 7f2ea6000640 1 -- 172.21.15.47:0/2620023211 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e9c00ce40 con 0x7f2ea8102000 2024-03-21T10:22:17.227 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:17.224+0000 7f2ea6000640 1 -- 172.21.15.47:0/2620023211 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f2e9c011920 con 0x7f2ea8102000 2024-03-21T10:22:17.227 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:17.224+0000 7f2ea6000640 1 -- 172.21.15.47:0/2620023211 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e9c010a00 con 0x7f2ea8102000 2024-03-21T10:22:17.227 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:17.224+0000 7f2eaf000640 1 -- 172.21.15.47:0/2620023211 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2ea81a1890 con 0x7f2ea8102000 2024-03-21T10:22:17.227 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:17.224+0000 7f2eaf000640 1 -- 172.21.15.47:0/2620023211 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2ea81a1d90 con 0x7f2ea8102000 2024-03-21T10:22:17.227 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:17.226+0000 7f2eaf000640 1 -- 172.21.15.47:0/2620023211 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2ea8108d50 con 0x7f2ea8102000 2024-03-21T10:22:17.228 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:17.227+0000 7f2ea6000640 1 -- 172.21.15.47:0/2620023211 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 16) v1 ==== 137854+0+0 (secure 0 0 0) 0x7f2e9c011ad0 con 0x7f2ea8102000 2024-03-21T10:22:17.230 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:17.230+0000 7f2ea6000640 1 --2- 172.21.15.47:0/2620023211 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f2e900a2fa0 0x7f2e900a5460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:22:17.231 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:17.230+0000 7f2ea6000640 1 -- 172.21.15.47:0/2620023211 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 8802+0+0 (secure 0 0 0) 0x7f2e9c0cfcc0 con 0x7f2ea8102000 2024-03-21T10:22:17.231 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:17.230+0000 7f2ead200640 1 --2- 172.21.15.47:0/2620023211 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f2e900a2fa0 0x7f2e900a5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:22:17.231 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:17.231+0000 7f2ead200640 1 --2- 172.21.15.47:0/2620023211 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f2e900a2fa0 0x7f2e900a5460 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f2e94005fd0 tx=0x7f2e94005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:22:17.235 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:17.235+0000 7f2ea6000640 1 -- 172.21.15.47:0/2620023211 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f2e9c016030 con 0x7f2ea8102000 2024-03-21T10:22:17.441 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:17.440+0000 7f2eaf000640 1 -- 172.21.15.47:0/2620023211 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f2ea80622d0 con 0x7f2ea8102000 2024-03-21T10:22:17.442 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:17.441+0000 7f2ea6000640 1 -- 172.21.15.47:0/2620023211 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v91) v1 ==== 104+0+11 (secure 0 0 0) 0x7f2e9c09c8b0 con 0x7f2ea8102000 2024-03-21T10:22:17.442 INFO:teuthology.orchestra.run.smithi047.stdout:pg_num: 32 2024-03-21T10:22:17.445 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:17.443+0000 7f2eaf000640 1 -- 172.21.15.47:0/2620023211 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f2e900a2fa0 msgr2=0x7f2e900a5460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:22:17.445 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:17.443+0000 7f2eaf000640 1 --2- 172.21.15.47:0/2620023211 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f2e900a2fa0 0x7f2e900a5460 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f2e94005fd0 tx=0x7f2e94005850 comp rx=0 tx=0).stop 2024-03-21T10:22:17.445 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:17.444+0000 7f2eaf000640 1 -- 172.21.15.47:0/2620023211 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2ea8102000 msgr2=0x7f2ea8199980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:22:17.445 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:17.444+0000 7f2eaf000640 1 --2- 172.21.15.47:0/2620023211 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2ea8102000 0x7f2ea8199980 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f2e9c00eab0 tx=0x7f2e9c00ef70 comp rx=0 tx=0).stop 2024-03-21T10:22:17.445 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:17.444+0000 7f2eaf000640 1 -- 172.21.15.47:0/2620023211 shutdown_connections 2024-03-21T10:22:17.445 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:17.444+0000 7f2eaf000640 1 --2- 172.21.15.47:0/2620023211 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f2e900a2fa0 0x7f2e900a5460 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:22:17.445 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:17.444+0000 7f2eaf000640 1 --2- 172.21.15.47:0/2620023211 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2ea8107260 0x7f2ea81a0f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:22:17.445 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:17.444+0000 7f2eaf000640 1 --2- 172.21.15.47:0/2620023211 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2ea8104930 0x7f2ea8199ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:22:17.445 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:17.444+0000 7f2eaf000640 1 --2- 172.21.15.47:0/2620023211 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2ea8102000 0x7f2ea8199980 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:22:17.445 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:17.444+0000 7f2eaf000640 1 -- 172.21.15.47:0/2620023211 >> 172.21.15.47:0/2620023211 conn(0x7f2ea80fba10 msgr2=0x7f2ea81016c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:22:17.445 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:17.444+0000 7f2eaf000640 1 -- 172.21.15.47:0/2620023211 shutdown_connections 2024-03-21T10:22:17.446 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:17.444+0000 7f2eaf000640 1 -- 172.21.15.47:0/2620023211 wait complete. 2024-03-21T10:22:17.462 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:22:17 smithi047 ceph-mon[31547]: pgmap v376: 129 pgs: 129 active+clean; 579 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-21T10:22:17.462 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:22:17 smithi047 ceph-mon[31547]: from='client.? 172.21.15.47:0/848189204' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-21T10:22:17.462 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:22:17 smithi047 ceph-mon[31547]: from='client.? 172.21.15.47:0/2887231860' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-21T10:22:17.466 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-21T10:22:17.480 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:22:17 smithi100 ceph-mon[35193]: pgmap v376: 129 pgs: 129 active+clean; 579 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-21T10:22:17.480 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:22:17 smithi100 ceph-mon[35193]: from='client.? 172.21.15.47:0/848189204' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-21T10:22:17.480 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:22:17 smithi100 ceph-mon[35193]: from='client.? 172.21.15.47:0/2887231860' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-21T10:22:17.490 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:22:17 smithi157 ceph-mon[35581]: pgmap v376: 129 pgs: 129 active+clean; 579 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-21T10:22:17.490 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:22:17 smithi157 ceph-mon[35581]: from='client.? 172.21.15.47:0/848189204' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-21T10:22:17.490 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:22:17 smithi157 ceph-mon[35581]: from='client.? 172.21.15.47:0/2887231860' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-21T10:22:17.696 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:17.694+0000 7fb4d0400640 1 -- 172.21.15.47:0/3996107398 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb4c81038c0 msgr2=0x7fb4c8103ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:22:17.696 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:17.694+0000 7fb4d0400640 1 --2- 172.21.15.47:0/3996107398 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb4c81038c0 0x7fb4c8103ce0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fb4bc009ca0 tx=0x7fb4bc02c0f0 comp rx=0 tx=0).stop 2024-03-21T10:22:17.696 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:17.694+0000 7fb4d0400640 1 -- 172.21.15.47:0/3996107398 shutdown_connections 2024-03-21T10:22:17.696 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:17.694+0000 7fb4d0400640 1 --2- 172.21.15.47:0/3996107398 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb4c8105470 0x7fb4c810bd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:22:17.697 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:17.694+0000 7fb4d0400640 1 --2- 172.21.15.47:0/3996107398 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb4c8104ab0 0x7fb4c8104f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:22:17.697 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:17.694+0000 7fb4d0400640 1 --2- 172.21.15.47:0/3996107398 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb4c81038c0 0x7fb4c8103ce0 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:22:17.697 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:17.694+0000 7fb4d0400640 1 -- 172.21.15.47:0/3996107398 >> 172.21.15.47:0/3996107398 conn(0x7fb4c80fee50 msgr2=0x7fb4c8101270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:22:17.697 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:17.694+0000 7fb4d0400640 1 -- 172.21.15.47:0/3996107398 shutdown_connections 2024-03-21T10:22:17.697 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:17.694+0000 7fb4d0400640 1 -- 172.21.15.47:0/3996107398 wait complete. 2024-03-21T10:22:17.697 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:17.695+0000 7fb4d0400640 1 Processor -- start 2024-03-21T10:22:17.697 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:17.695+0000 7fb4d0400640 1 -- start start 2024-03-21T10:22:17.701 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:17.695+0000 7fb4d0400640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb4c81038c0 0x7fb4c8199ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:22:17.701 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:17.696+0000 7fb4d0400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb4c8104ab0 0x7fb4c819a020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:22:17.701 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:17.696+0000 7fb4d0400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb4c8105470 0x7fb4c81a1090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:22:17.701 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:17.696+0000 7fb4d0400640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4c810f3f0 con 0x7fb4c8104ab0 2024-03-21T10:22:17.701 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:17.696+0000 7fb4d0400640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4c810f270 con 0x7fb4c8105470 2024-03-21T10:22:17.701 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:17.696+0000 7fb4d0400640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4c810f570 con 0x7fb4c81038c0 2024-03-21T10:22:17.701 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:17.696+0000 7fb4cd000640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb4c8104ab0 0x7fb4c819a020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:22:17.702 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:17.696+0000 7fb4cd000640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb4c8104ab0 0x7fb4c819a020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:51668/0 (socket says 172.21.15.47:51668) 2024-03-21T10:22:17.702 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:17.696+0000 7fb4cd000640 1 -- 172.21.15.47:0/1985310013 learned_addr learned my addr 172.21.15.47:0/1985310013 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:22:17.702 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:17.696+0000 7fb4ce400640 1 --2- 172.21.15.47:0/1985310013 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb4c8105470 0x7fb4c81a1090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:22:17.702 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:17.696+0000 7fb4cda00640 1 --2- 172.21.15.47:0/1985310013 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb4c81038c0 0x7fb4c8199ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:22:17.702 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:17.696+0000 7fb4cd000640 1 -- 172.21.15.47:0/1985310013 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb4c81038c0 msgr2=0x7fb4c8199ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:22:17.702 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:17.696+0000 7fb4cd000640 1 --2- 172.21.15.47:0/1985310013 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb4c81038c0 0x7fb4c8199ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:22:17.702 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:17.696+0000 7fb4cd000640 1 -- 172.21.15.47:0/1985310013 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb4c8105470 msgr2=0x7fb4c81a1090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:22:17.702 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:17.696+0000 7fb4cd000640 1 --2- 172.21.15.47:0/1985310013 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb4c8105470 0x7fb4c81a1090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:22:17.702 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:17.696+0000 7fb4cd000640 1 -- 172.21.15.47:0/1985310013 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb4c81a1700 con 0x7fb4c8104ab0 2024-03-21T10:22:17.702 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:17.697+0000 7fb4cd000640 1 --2- 172.21.15.47:0/1985310013 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb4c8104ab0 0x7fb4c819a020 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fb4b400d950 tx=0x7fb4b400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:22:17.702 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:17.697+0000 7fb4c6000640 1 -- 172.21.15.47:0/1985310013 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4b4017070 con 0x7fb4c8104ab0 2024-03-21T10:22:17.702 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:17.697+0000 7fb4c6000640 1 -- 172.21.15.47:0/1985310013 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fb4b4009c30 con 0x7fb4c8104ab0 2024-03-21T10:22:17.702 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:17.697+0000 7fb4c6000640 1 -- 172.21.15.47:0/1985310013 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4b400f620 con 0x7fb4c8104ab0 2024-03-21T10:22:17.702 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:17.697+0000 7fb4d0400640 1 -- 172.21.15.47:0/1985310013 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb4c81a19f0 con 0x7fb4c8104ab0 2024-03-21T10:22:17.703 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:17.697+0000 7fb4d0400640 1 -- 172.21.15.47:0/1985310013 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb4c81a1f90 con 0x7fb4c8104ab0 2024-03-21T10:22:17.703 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:17.698+0000 7fb4d0400640 1 -- 172.21.15.47:0/1985310013 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb4c810acb0 con 0x7fb4c8104ab0 2024-03-21T10:22:17.703 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:17.701+0000 7fb4c6000640 1 -- 172.21.15.47:0/1985310013 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 16) v1 ==== 137854+0+0 (secure 0 0 0) 0x7fb4b40109a0 con 0x7fb4c8104ab0 2024-03-21T10:22:17.704 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:17.703+0000 7fb4c6000640 1 --2- 172.21.15.47:0/1985310013 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7fb4b00a2fa0 0x7fb4b00a5460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:22:17.705 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:17.704+0000 7fb4c6000640 1 -- 172.21.15.47:0/1985310013 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 8802+0+0 (secure 0 0 0) 0x7fb4b40cf2f0 con 0x7fb4c8104ab0 2024-03-21T10:22:17.705 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:17.704+0000 7fb4cda00640 1 --2- 172.21.15.47:0/1985310013 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7fb4b00a2fa0 0x7fb4b00a5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:22:17.705 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:17.704+0000 7fb4cda00640 1 --2- 172.21.15.47:0/1985310013 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7fb4b00a2fa0 0x7fb4b00a5460 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fb4bc037040 tx=0x7fb4bc032340 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:22:17.709 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:17.708+0000 7fb4c6000640 1 -- 172.21.15.47:0/1985310013 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fb4b4009de0 con 0x7fb4c8104ab0 2024-03-21T10:22:17.915 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:17.914+0000 7fb4d0400640 1 -- 172.21.15.47:0/1985310013 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fb4c80622d0 con 0x7fb4c8104ab0 2024-03-21T10:22:17.916 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:17.915+0000 7fb4c6000640 1 -- 172.21.15.47:0/1985310013 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v91) v1 ==== 100+0+11 (secure 0 0 0) 0x7fb4b409be80 con 0x7fb4c8104ab0 2024-03-21T10:22:17.916 INFO:teuthology.orchestra.run.smithi047.stdout:pg_num: 64 2024-03-21T10:22:17.919 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:17.917+0000 7fb4d0400640 1 -- 172.21.15.47:0/1985310013 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7fb4b00a2fa0 msgr2=0x7fb4b00a5460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:22:17.919 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:17.917+0000 7fb4d0400640 1 --2- 172.21.15.47:0/1985310013 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7fb4b00a2fa0 0x7fb4b00a5460 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fb4bc037040 tx=0x7fb4bc032340 comp rx=0 tx=0).stop 2024-03-21T10:22:17.919 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:17.918+0000 7fb4d0400640 1 -- 172.21.15.47:0/1985310013 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb4c8104ab0 msgr2=0x7fb4c819a020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:22:17.919 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:17.918+0000 7fb4d0400640 1 --2- 172.21.15.47:0/1985310013 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb4c8104ab0 0x7fb4c819a020 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fb4b400d950 tx=0x7fb4b400de10 comp rx=0 tx=0).stop 2024-03-21T10:22:17.919 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:17.918+0000 7fb4d0400640 1 -- 172.21.15.47:0/1985310013 shutdown_connections 2024-03-21T10:22:17.919 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:17.918+0000 7fb4d0400640 1 --2- 172.21.15.47:0/1985310013 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7fb4b00a2fa0 0x7fb4b00a5460 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:22:17.920 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:17.918+0000 7fb4d0400640 1 --2- 172.21.15.47:0/1985310013 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb4c8105470 0x7fb4c81a1090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:22:17.920 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:17.918+0000 7fb4d0400640 1 --2- 172.21.15.47:0/1985310013 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb4c8104ab0 0x7fb4c819a020 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:22:17.920 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:17.918+0000 7fb4d0400640 1 --2- 172.21.15.47:0/1985310013 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb4c81038c0 0x7fb4c8199ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:22:17.920 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:17.918+0000 7fb4d0400640 1 -- 172.21.15.47:0/1985310013 >> 172.21.15.47:0/1985310013 conn(0x7fb4c80fee50 msgr2=0x7fb4c8107e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:22:17.920 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:17.918+0000 7fb4d0400640 1 -- 172.21.15.47:0/1985310013 shutdown_connections 2024-03-21T10:22:17.920 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:17.918+0000 7fb4d0400640 1 -- 172.21.15.47:0/1985310013 wait complete. 2024-03-21T10:22:17.938 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-03-21T10:22:18.169 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:18.164+0000 7f0dbf400640 1 -- 172.21.15.47:0/4046304609 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0db8104ab0 msgr2=0x7f0db8104f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:22:18.170 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:18.164+0000 7f0dbf400640 1 --2- 172.21.15.47:0/4046304609 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0db8104ab0 0x7f0db8104f30 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f0da4009900 tx=0x7f0da402bc30 comp rx=0 tx=0).stop 2024-03-21T10:22:18.170 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:18.164+0000 7f0dbf400640 1 -- 172.21.15.47:0/4046304609 shutdown_connections 2024-03-21T10:22:18.170 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:18.164+0000 7f0dbf400640 1 --2- 172.21.15.47:0/4046304609 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0db8105470 0x7f0db810bd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:22:18.170 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:18.164+0000 7f0dbf400640 1 --2- 172.21.15.47:0/4046304609 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0db8104ab0 0x7f0db8104f30 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:22:18.170 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:18.164+0000 7f0dbf400640 1 --2- 172.21.15.47:0/4046304609 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0db81038c0 0x7f0db8103ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:22:18.170 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:18.164+0000 7f0dbf400640 1 -- 172.21.15.47:0/4046304609 >> 172.21.15.47:0/4046304609 conn(0x7f0db80fee50 msgr2=0x7f0db8101270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:22:18.170 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:18.164+0000 7f0dbf400640 1 -- 172.21.15.47:0/4046304609 shutdown_connections 2024-03-21T10:22:18.171 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:18.165+0000 7f0dbf400640 1 -- 172.21.15.47:0/4046304609 wait complete. 2024-03-21T10:22:18.171 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:18.165+0000 7f0dbf400640 1 Processor -- start 2024-03-21T10:22:18.171 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:18.166+0000 7f0dbf400640 1 -- start start 2024-03-21T10:22:18.171 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:18.166+0000 7f0dbf400640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0db81038c0 0x7f0db8199a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:22:18.171 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:18.166+0000 7f0dbf400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0db8104ab0 0x7f0db8199f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:22:18.171 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:18.166+0000 7f0dbf400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0db8105470 0x7f0db81a0ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:22:18.171 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:18.166+0000 7f0dbf400640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0db810f3d0 con 0x7f0db8105470 2024-03-21T10:22:18.172 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:18.166+0000 7f0dbf400640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0db810f250 con 0x7f0db8104ab0 2024-03-21T10:22:18.172 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:18.166+0000 7f0dbf400640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0db810f550 con 0x7f0db81038c0 2024-03-21T10:22:18.172 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:18.166+0000 7f0dbd400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0db8105470 0x7f0db81a0ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:22:18.172 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:18.166+0000 7f0dbd400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0db8105470 0x7f0db81a0ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:51682/0 (socket says 172.21.15.47:51682) 2024-03-21T10:22:18.172 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:18.166+0000 7f0dbd400640 1 -- 172.21.15.47:0/1488222901 learned_addr learned my addr 172.21.15.47:0/1488222901 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:22:18.172 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:18.167+0000 7f0db7e00640 1 --2- 172.21.15.47:0/1488222901 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0db8104ab0 0x7f0db8199f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:22:18.172 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:18.167+0000 7f0dbca00640 1 --2- 172.21.15.47:0/1488222901 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0db81038c0 0x7f0db8199a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:22:18.172 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:18.167+0000 7f0dbd400640 1 -- 172.21.15.47:0/1488222901 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0db81038c0 msgr2=0x7f0db8199a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:22:18.173 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:18.167+0000 7f0dbd400640 1 --2- 172.21.15.47:0/1488222901 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0db81038c0 0x7f0db8199a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:22:18.173 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:18.167+0000 7f0dbd400640 1 -- 172.21.15.47:0/1488222901 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0db8104ab0 msgr2=0x7f0db8199f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:22:18.173 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:18.167+0000 7f0dbd400640 1 --2- 172.21.15.47:0/1488222901 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0db8104ab0 0x7f0db8199f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:22:18.173 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:18.167+0000 7f0dbd400640 1 -- 172.21.15.47:0/1488222901 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0db81a1660 con 0x7f0db8105470 2024-03-21T10:22:18.173 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:18.167+0000 7f0dbd400640 1 --2- 172.21.15.47:0/1488222901 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0db8105470 0x7f0db81a0ff0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f0db000eaa0 tx=0x7f0db000ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:22:18.173 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:18.167+0000 7f0db5600640 1 -- 172.21.15.47:0/1488222901 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0db000cdb0 con 0x7f0db8105470 2024-03-21T10:22:18.173 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:18.167+0000 7f0dbf400640 1 -- 172.21.15.47:0/1488222901 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0db81a1950 con 0x7f0db8105470 2024-03-21T10:22:18.173 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:18.167+0000 7f0db5600640 1 -- 172.21.15.47:0/1488222901 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f0db0005020 con 0x7f0db8105470 2024-03-21T10:22:18.174 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:18.167+0000 7f0db5600640 1 -- 172.21.15.47:0/1488222901 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0db00116d0 con 0x7f0db8105470 2024-03-21T10:22:18.174 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:18.168+0000 7f0dbf400640 1 -- 172.21.15.47:0/1488222901 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0db81a1e50 con 0x7f0db8105470 2024-03-21T10:22:18.176 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:18.169+0000 7f0dbf400640 1 -- 172.21.15.47:0/1488222901 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0db810abe0 con 0x7f0db8105470 2024-03-21T10:22:18.176 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:18.171+0000 7f0db5600640 1 -- 172.21.15.47:0/1488222901 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 16) v1 ==== 137854+0+0 (secure 0 0 0) 0x7f0db00051d0 con 0x7f0db8105470 2024-03-21T10:22:18.176 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:18.172+0000 7f0db5600640 1 --2- 172.21.15.47:0/1488222901 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f0da00a2f20 0x7f0da00a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:22:18.176 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:18.172+0000 7f0db5600640 1 -- 172.21.15.47:0/1488222901 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 8802+0+0 (secure 0 0 0) 0x7f0db00cf930 con 0x7f0db8105470 2024-03-21T10:22:18.176 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:18.173+0000 7f0dbca00640 1 --2- 172.21.15.47:0/1488222901 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f0da00a2f20 0x7f0da00a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:22:18.176 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:18.173+0000 7f0dbca00640 1 --2- 172.21.15.47:0/1488222901 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f0da00a2f20 0x7f0da00a53e0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f0dac0058c0 tx=0x7f0dac005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:22:18.177 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:18.177+0000 7f0db5600640 1 -- 172.21.15.47:0/1488222901 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f0db0016030 con 0x7f0db8105470 2024-03-21T10:22:18.213 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:22:18 smithi047 ceph-mon[31547]: from='client.? 172.21.15.47:0/2620023211' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-21T10:22:18.214 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:22:18 smithi047 ceph-mon[31547]: from='client.? 172.21.15.47:0/1985310013' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-21T10:22:18.380 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:18.378+0000 7f0dbf400640 1 -- 172.21.15.47:0/1488222901 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f0db80622a0 con 0x7f0db8105470 2024-03-21T10:22:18.381 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:18.380+0000 7f0db5600640 1 -- 172.21.15.47:0/1488222901 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v91) v1 ==== 103+0+11 (secure 0 0 0) 0x7f0db009c4c0 con 0x7f0db8105470 2024-03-21T10:22:18.381 INFO:teuthology.orchestra.run.smithi047.stdout:pg_num: 32 2024-03-21T10:22:18.381 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:22:18 smithi100 ceph-mon[35193]: from='client.? 172.21.15.47:0/2620023211' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-21T10:22:18.381 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:22:18 smithi100 ceph-mon[35193]: from='client.? 172.21.15.47:0/1985310013' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-21T10:22:18.384 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:18.382+0000 7f0dbf400640 1 -- 172.21.15.47:0/1488222901 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f0da00a2f20 msgr2=0x7f0da00a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:22:18.384 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:18.382+0000 7f0dbf400640 1 --2- 172.21.15.47:0/1488222901 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f0da00a2f20 0x7f0da00a53e0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f0dac0058c0 tx=0x7f0dac005850 comp rx=0 tx=0).stop 2024-03-21T10:22:18.384 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:18.382+0000 7f0dbf400640 1 -- 172.21.15.47:0/1488222901 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0db8105470 msgr2=0x7f0db81a0ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:22:18.384 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:18.382+0000 7f0dbf400640 1 --2- 172.21.15.47:0/1488222901 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0db8105470 0x7f0db81a0ff0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f0db000eaa0 tx=0x7f0db000ef60 comp rx=0 tx=0).stop 2024-03-21T10:22:18.384 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:18.382+0000 7f0dbf400640 1 -- 172.21.15.47:0/1488222901 shutdown_connections 2024-03-21T10:22:18.384 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:18.382+0000 7f0dbf400640 1 --2- 172.21.15.47:0/1488222901 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f0da00a2f20 0x7f0da00a53e0 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:22:18.384 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:18.382+0000 7f0dbf400640 1 --2- 172.21.15.47:0/1488222901 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0db8105470 0x7f0db81a0ff0 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:22:18.385 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:18.382+0000 7f0dbf400640 1 --2- 172.21.15.47:0/1488222901 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0db8104ab0 0x7f0db8199f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:22:18.385 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:18.382+0000 7f0dbf400640 1 --2- 172.21.15.47:0/1488222901 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0db81038c0 0x7f0db8199a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:22:18.385 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:18.382+0000 7f0dbf400640 1 -- 172.21.15.47:0/1488222901 >> 172.21.15.47:0/1488222901 conn(0x7f0db80fee50 msgr2=0x7f0db8107db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:22:18.385 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:18.383+0000 7f0dbf400640 1 -- 172.21.15.47:0/1488222901 shutdown_connections 2024-03-21T10:22:18.385 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:18.383+0000 7f0dbf400640 1 -- 172.21.15.47:0/1488222901 wait complete. 2024-03-21T10:22:18.404 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-03-21T10:22:18.490 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:22:18 smithi157 ceph-mon[35581]: from='client.? 172.21.15.47:0/2620023211' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-21T10:22:18.490 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:22:18 smithi157 ceph-mon[35581]: from='client.? 172.21.15.47:0/1985310013' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-21T10:22:18.628 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:18.627+0000 7f45a3e00640 1 -- 172.21.15.47:0/719436390 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f45a40756e0 msgr2=0x7f45a4073b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:22:18.629 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:18.627+0000 7f45a3e00640 1 --2- 172.21.15.47:0/719436390 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f45a40756e0 0x7f45a4073b30 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f4594009900 tx=0x7f459402bc30 comp rx=0 tx=0).stop 2024-03-21T10:22:18.629 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:18.627+0000 7f45a3e00640 1 -- 172.21.15.47:0/719436390 shutdown_connections 2024-03-21T10:22:18.629 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:18.627+0000 7f45a3e00640 1 --2- 172.21.15.47:0/719436390 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f45a4109460 0x7f45a410b8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:22:18.629 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:18.627+0000 7f45a3e00640 1 --2- 172.21.15.47:0/719436390 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f45a4074100 0x7f45a4074580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:22:18.629 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:18.627+0000 7f45a3e00640 1 --2- 172.21.15.47:0/719436390 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f45a40756e0 0x7f45a4073b30 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:22:18.629 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:18.627+0000 7f45a3e00640 1 -- 172.21.15.47:0/719436390 >> 172.21.15.47:0/719436390 conn(0x7f45a40fcb10 msgr2=0x7f45a40fef50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:22:18.629 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:18.627+0000 7f45a3e00640 1 -- 172.21.15.47:0/719436390 shutdown_connections 2024-03-21T10:22:18.629 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:18.627+0000 7f45a3e00640 1 -- 172.21.15.47:0/719436390 wait complete. 2024-03-21T10:22:18.630 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:18.629+0000 7f45a3e00640 1 Processor -- start 2024-03-21T10:22:18.630 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:18.629+0000 7f45a3e00640 1 -- start start 2024-03-21T10:22:18.632 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:18.629+0000 7f45a3e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f45a4074100 0x7f45a41999a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:22:18.632 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:18.629+0000 7f45a3e00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f45a40756e0 0x7f45a4199ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:22:18.632 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:18.629+0000 7f45a3e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f45a4109460 0x7f45a41a0f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:22:18.632 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:18.629+0000 7f45a3e00640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45a410f2b0 con 0x7f45a4109460 2024-03-21T10:22:18.632 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:18.629+0000 7f45a3e00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45a410f130 con 0x7f45a4074100 2024-03-21T10:22:18.632 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:18.629+0000 7f45a3e00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45a410f430 con 0x7f45a40756e0 2024-03-21T10:22:18.632 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:18.630+0000 7f45a3400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f45a4109460 0x7f45a41a0f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:22:18.632 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:18.630+0000 7f45a3400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f45a4109460 0x7f45a41a0f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:51696/0 (socket says 172.21.15.47:51696) 2024-03-21T10:22:18.632 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:18.630+0000 7f45a3400640 1 -- 172.21.15.47:0/427733293 learned_addr learned my addr 172.21.15.47:0/427733293 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:22:18.633 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:18.630+0000 7f45a2a00640 1 --2- 172.21.15.47:0/427733293 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f45a4074100 0x7f45a41999a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:22:18.633 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:18.630+0000 7f45a2000640 1 --2- 172.21.15.47:0/427733293 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f45a40756e0 0x7f45a4199ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:22:18.633 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:18.630+0000 7f45a3400640 1 -- 172.21.15.47:0/427733293 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f45a40756e0 msgr2=0x7f45a4199ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:22:18.633 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:18.630+0000 7f45a3400640 1 --2- 172.21.15.47:0/427733293 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f45a40756e0 0x7f45a4199ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:22:18.633 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:18.630+0000 7f45a3400640 1 -- 172.21.15.47:0/427733293 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f45a4074100 msgr2=0x7f45a41999a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:22:18.633 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:18.630+0000 7f45a3400640 1 --2- 172.21.15.47:0/427733293 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f45a4074100 0x7f45a41999a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:22:18.633 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:18.630+0000 7f45a3400640 1 -- 172.21.15.47:0/427733293 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f45a41a15c0 con 0x7f45a4109460 2024-03-21T10:22:18.633 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:18.631+0000 7f45a3400640 1 --2- 172.21.15.47:0/427733293 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f45a4109460 0x7f45a41a0f50 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f459c00c970 tx=0x7f459c00ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:22:18.633 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:18.631+0000 7f459b400640 1 -- 172.21.15.47:0/427733293 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f459c007bb0 con 0x7f45a4109460 2024-03-21T10:22:18.633 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:18.631+0000 7f459b400640 1 -- 172.21.15.47:0/427733293 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f459c007d50 con 0x7f45a4109460 2024-03-21T10:22:18.633 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:18.631+0000 7f459b400640 1 -- 172.21.15.47:0/427733293 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f459c018700 con 0x7f45a4109460 2024-03-21T10:22:18.633 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:18.631+0000 7f45a3e00640 1 -- 172.21.15.47:0/427733293 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f45a41a18b0 con 0x7f45a4109460 2024-03-21T10:22:18.633 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:18.631+0000 7f45a3e00640 1 -- 172.21.15.47:0/427733293 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f45a41a1e00 con 0x7f45a4109460 2024-03-21T10:22:18.634 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:18.632+0000 7f45a3e00640 1 -- 172.21.15.47:0/427733293 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f45a4108190 con 0x7f45a4109460 2024-03-21T10:22:18.635 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:18.634+0000 7f459b400640 1 -- 172.21.15.47:0/427733293 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 16) v1 ==== 137854+0+0 (secure 0 0 0) 0x7f459c00fa20 con 0x7f45a4109460 2024-03-21T10:22:18.637 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:18.636+0000 7f459b400640 1 --2- 172.21.15.47:0/427733293 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f45840a2fa0 0x7f45840a5460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:22:18.637 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:18.636+0000 7f459b400640 1 -- 172.21.15.47:0/427733293 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 8802+0+0 (secure 0 0 0) 0x7f459c0cf6f0 con 0x7f45a4109460 2024-03-21T10:22:18.637 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:18.636+0000 7f45a2a00640 1 --2- 172.21.15.47:0/427733293 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f45840a2fa0 0x7f45840a5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:22:18.638 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:18.637+0000 7f45a2a00640 1 --2- 172.21.15.47:0/427733293 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f45840a2fa0 0x7f45840a5460 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f4594009230 tx=0x7f4594005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:22:18.641 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:18.640+0000 7f459b400640 1 -- 172.21.15.47:0/427733293 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f459c014030 con 0x7f45a4109460 2024-03-21T10:22:18.859 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:18.857+0000 7f45a3e00640 1 -- 172.21.15.47:0/427733293 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f45a402d070 con 0x7f45a4109460 2024-03-21T10:22:18.859 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:18.858+0000 7f459b400640 1 -- 172.21.15.47:0/427733293 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 10 entries v91) v1 ==== 81+0+590 (secure 0 0 0) 0x7f459c09c280 con 0x7f45a4109460 2024-03-21T10:22:18.861 INFO:teuthology.orchestra.run.smithi047.stdout:172.21.15.47:0/356822431 2024-03-22T10:12:25.844530+0000 2024-03-21T10:22:18.861 INFO:teuthology.orchestra.run.smithi047.stdout:172.21.15.47:0/1167490879 2024-03-22T10:12:25.844530+0000 2024-03-21T10:22:18.861 INFO:teuthology.orchestra.run.smithi047.stdout:172.21.15.47:0/1540718138 2024-03-22T10:12:25.844530+0000 2024-03-21T10:22:18.861 INFO:teuthology.orchestra.run.smithi047.stdout:172.21.15.47:6801/3836903972 2024-03-22T10:12:25.844530+0000 2024-03-21T10:22:18.861 INFO:teuthology.orchestra.run.smithi047.stdout:172.21.15.47:0/235436602 2024-03-22T10:11:50.301838+0000 2024-03-21T10:22:18.861 INFO:teuthology.orchestra.run.smithi047.stdout:172.21.15.47:0/2052815572 2024-03-22T10:11:50.301838+0000 2024-03-21T10:22:18.862 INFO:teuthology.orchestra.run.smithi047.stdout:172.21.15.47:6800/3942658432 2024-03-22T10:11:50.301838+0000 2024-03-21T10:22:18.862 INFO:teuthology.orchestra.run.smithi047.stdout:172.21.15.47:6801/3942658432 2024-03-22T10:11:50.301838+0000 2024-03-21T10:22:18.862 INFO:teuthology.orchestra.run.smithi047.stdout:172.21.15.47:0/1419516257 2024-03-22T10:11:50.301838+0000 2024-03-21T10:22:18.862 INFO:teuthology.orchestra.run.smithi047.stdout:172.21.15.47:6800/3836903972 2024-03-22T10:12:25.844530+0000 2024-03-21T10:22:18.863 INFO:teuthology.orchestra.run.smithi047.stderr:listed 10 entries 2024-03-21T10:22:18.863 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:18.861+0000 7f45a3e00640 1 -- 172.21.15.47:0/427733293 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f45840a2fa0 msgr2=0x7f45840a5460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:22:18.863 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:18.861+0000 7f45a3e00640 1 --2- 172.21.15.47:0/427733293 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f45840a2fa0 0x7f45840a5460 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f4594009230 tx=0x7f4594005fd0 comp rx=0 tx=0).stop 2024-03-21T10:22:18.863 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:18.861+0000 7f45a3e00640 1 -- 172.21.15.47:0/427733293 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f45a4109460 msgr2=0x7f45a41a0f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:22:18.863 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:18.861+0000 7f45a3e00640 1 --2- 172.21.15.47:0/427733293 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f45a4109460 0x7f45a41a0f50 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f459c00c970 tx=0x7f459c00ce30 comp rx=0 tx=0).stop 2024-03-21T10:22:18.865 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:18.861+0000 7f45a3e00640 1 -- 172.21.15.47:0/427733293 shutdown_connections 2024-03-21T10:22:18.865 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:18.861+0000 7f45a3e00640 1 --2- 172.21.15.47:0/427733293 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f45840a2fa0 0x7f45840a5460 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:22:18.865 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:18.861+0000 7f45a3e00640 1 --2- 172.21.15.47:0/427733293 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f45a4109460 0x7f45a41a0f50 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:22:18.865 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:18.861+0000 7f45a3e00640 1 --2- 172.21.15.47:0/427733293 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f45a40756e0 0x7f45a4199ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:22:18.865 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:18.861+0000 7f45a3e00640 1 --2- 172.21.15.47:0/427733293 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f45a4074100 0x7f45a41999a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:22:18.865 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:18.861+0000 7f45a3e00640 1 -- 172.21.15.47:0/427733293 >> 172.21.15.47:0/427733293 conn(0x7f45a40fcb10 msgr2=0x7f45a4105360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:22:18.865 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:18.862+0000 7f45a3e00640 1 -- 172.21.15.47:0/427733293 shutdown_connections 2024-03-21T10:22:18.866 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:18.862+0000 7f45a3e00640 1 -- 172.21.15.47:0/427733293 wait complete. 2024-03-21T10:22:18.882 DEBUG:teuthology.orchestra.run.smithi047:> set -ex 2024-03-21T10:22:18.882 DEBUG:teuthology.orchestra.run.smithi047:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-21T10:22:18.938 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-21T10:22:19.156 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:19.154+0000 7ff7eaa00640 1 -- 172.21.15.47:0/3560202968 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff7e41048e0 msgr2=0x7ff7e4104d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:22:19.156 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:19.154+0000 7ff7eaa00640 1 --2- 172.21.15.47:0/3560202968 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff7e41048e0 0x7ff7e4104d60 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7ff7d4009900 tx=0x7ff7d402bc40 comp rx=0 tx=0).stop 2024-03-21T10:22:19.157 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:19.154+0000 7ff7eaa00640 1 -- 172.21.15.47:0/3560202968 shutdown_connections 2024-03-21T10:22:19.157 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:19.154+0000 7ff7eaa00640 1 --2- 172.21.15.47:0/3560202968 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff7e41052a0 0x7ff7e410bb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:22:19.157 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:19.154+0000 7ff7eaa00640 1 --2- 172.21.15.47:0/3560202968 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff7e41048e0 0x7ff7e4104d60 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:22:19.157 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:19.154+0000 7ff7eaa00640 1 --2- 172.21.15.47:0/3560202968 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff7e41036f0 0x7ff7e4103b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:22:19.157 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:19.154+0000 7ff7eaa00640 1 -- 172.21.15.47:0/3560202968 >> 172.21.15.47:0/3560202968 conn(0x7ff7e40fec80 msgr2=0x7ff7e41010a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:22:19.157 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:19.154+0000 7ff7eaa00640 1 -- 172.21.15.47:0/3560202968 shutdown_connections 2024-03-21T10:22:19.157 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:19.154+0000 7ff7eaa00640 1 -- 172.21.15.47:0/3560202968 wait complete. 2024-03-21T10:22:19.157 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:19.155+0000 7ff7eaa00640 1 Processor -- start 2024-03-21T10:22:19.157 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:19.155+0000 7ff7eaa00640 1 -- start start 2024-03-21T10:22:19.159 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:19.156+0000 7ff7eaa00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff7e41036f0 0x7ff7e4199800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:22:19.159 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:19.156+0000 7ff7eaa00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff7e41048e0 0x7ff7e4199d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:22:19.159 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:19.156+0000 7ff7eaa00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff7e41052a0 0x7ff7e41a0db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:22:19.159 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:19.156+0000 7ff7eaa00640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7e410f250 con 0x7ff7e41036f0 2024-03-21T10:22:19.159 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:19.156+0000 7ff7eaa00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7e410f0d0 con 0x7ff7e41052a0 2024-03-21T10:22:19.159 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:19.156+0000 7ff7eaa00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7e410f3d0 con 0x7ff7e41048e0 2024-03-21T10:22:19.159 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:19.156+0000 7ff7e9600640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff7e41036f0 0x7ff7e4199800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:22:19.160 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:19.156+0000 7ff7e9600640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff7e41036f0 0x7ff7e4199800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:51712/0 (socket says 172.21.15.47:51712) 2024-03-21T10:22:19.160 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:19.156+0000 7ff7e9600640 1 -- 172.21.15.47:0/3014412117 learned_addr learned my addr 172.21.15.47:0/3014412117 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:22:19.160 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:19.157+0000 7ff7ea000640 1 --2- 172.21.15.47:0/3014412117 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff7e41052a0 0x7ff7e41a0db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:22:19.160 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:19.157+0000 7ff7e8c00640 1 --2- 172.21.15.47:0/3014412117 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff7e41048e0 0x7ff7e4199d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:22:19.160 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:19.157+0000 7ff7e9600640 1 -- 172.21.15.47:0/3014412117 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff7e41048e0 msgr2=0x7ff7e4199d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:22:19.160 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:19.157+0000 7ff7e9600640 1 --2- 172.21.15.47:0/3014412117 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff7e41048e0 0x7ff7e4199d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:22:19.160 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:19.157+0000 7ff7e9600640 1 -- 172.21.15.47:0/3014412117 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff7e41052a0 msgr2=0x7ff7e41a0db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:22:19.160 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:19.157+0000 7ff7e9600640 1 --2- 172.21.15.47:0/3014412117 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff7e41052a0 0x7ff7e41a0db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:22:19.160 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:19.157+0000 7ff7e9600640 1 -- 172.21.15.47:0/3014412117 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7e41a1420 con 0x7ff7e41036f0 2024-03-21T10:22:19.160 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:19.157+0000 7ff7e9600640 1 --2- 172.21.15.47:0/3014412117 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff7e41036f0 0x7ff7e4199800 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7ff7d800c970 tx=0x7ff7d800ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:22:19.160 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:19.157+0000 7ff7e2000640 1 -- 172.21.15.47:0/3014412117 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7d8007c10 con 0x7ff7e41036f0 2024-03-21T10:22:19.160 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:19.157+0000 7ff7e2000640 1 -- 172.21.15.47:0/3014412117 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7ff7d8007db0 con 0x7ff7e41036f0 2024-03-21T10:22:19.160 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:19.157+0000 7ff7eaa00640 1 -- 172.21.15.47:0/3014412117 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff7e41a1710 con 0x7ff7e41036f0 2024-03-21T10:22:19.160 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:19.157+0000 7ff7e2000640 1 -- 172.21.15.47:0/3014412117 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7d800f870 con 0x7ff7e41036f0 2024-03-21T10:22:19.161 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:19.158+0000 7ff7eaa00640 1 -- 172.21.15.47:0/3014412117 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff7e41a1cb0 con 0x7ff7e41036f0 2024-03-21T10:22:19.161 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:19.159+0000 7ff7eaa00640 1 -- 172.21.15.47:0/3014412117 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff7e410a940 con 0x7ff7e41036f0 2024-03-21T10:22:19.161 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:19.161+0000 7ff7e2000640 1 -- 172.21.15.47:0/3014412117 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 16) v1 ==== 137854+0+0 (secure 0 0 0) 0x7ff7d800fa10 con 0x7ff7e41036f0 2024-03-21T10:22:19.163 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:19.162+0000 7ff7e2000640 1 --2- 172.21.15.47:0/3014412117 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7ff7cc0a2fa0 0x7ff7cc0a5460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:22:19.163 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:19.162+0000 7ff7e2000640 1 -- 172.21.15.47:0/3014412117 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 8802+0+0 (secure 0 0 0) 0x7ff7d80cfa70 con 0x7ff7e41036f0 2024-03-21T10:22:19.163 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:19.162+0000 7ff7e8c00640 1 --2- 172.21.15.47:0/3014412117 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7ff7cc0a2fa0 0x7ff7cc0a5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:22:19.164 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:19.163+0000 7ff7e8c00640 1 --2- 172.21.15.47:0/3014412117 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7ff7cc0a2fa0 0x7ff7cc0a5460 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7ff7e419ad10 tx=0x7ff7d4000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:22:19.167 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:19.166+0000 7ff7e2000640 1 -- 172.21.15.47:0/3014412117 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7ff7d809c690 con 0x7ff7e41036f0 2024-03-21T10:22:19.214 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:22:19 smithi047 ceph-mon[31547]: pgmap v377: 129 pgs: 129 active+clean; 579 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-21T10:22:19.214 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:22:19 smithi047 ceph-mon[31547]: from='client.? 172.21.15.47:0/1488222901' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-03-21T10:22:19.214 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:22:19 smithi047 ceph-mon[31547]: from='client.? 172.21.15.47:0/427733293' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-03-21T10:22:19.380 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:19.378+0000 7ff7eaa00640 1 -- 172.21.15.47:0/3014412117 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ff7e40622a0 con 0x7ff7e41036f0 2024-03-21T10:22:19.382 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:19.382+0000 7ff7e2000640 1 -- 172.21.15.47:0/3014412117 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v91) v1 ==== 74+0+24116 (secure 0 0 0) 0x7ff7d809bf90 con 0x7ff7e41036f0 2024-03-21T10:22:19.384 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-21T10:22:19.384 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":91,"fsid":"3056d51a-e76b-11ee-95cd-87774f69a715","created":"2024-03-21T10:11:17.019965+0000","modified":"2024-03-21T10:22:03.759214+0000","last_up_change":"2024-03-21T10:20:52.675328+0000","last_in_change":"2024-03-21T10:20:27.648212+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-21T10:15:40.553114+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-21T10:21:56.399518+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-21T10:21:57.812940+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-21T10:22:01.822568+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"91","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"b0c7a102-8d89-428b-b793-cbbe67151f50","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6802","nonce":775837313},{"type":"v1","addr":"172.21.15.47:6803","nonce":775837313}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6804","nonce":775837313},{"type":"v1","addr":"172.21.15.47:6805","nonce":775837313}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6808","nonce":775837313},{"type":"v1","addr":"172.21.15.47:6809","nonce":775837313}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6806","nonce":775837313},{"type":"v1","addr":"172.21.15.47:6807","nonce":775837313}]},"public_addr":"172.21.15.47:6803/775837313","cluster_addr":"172.21.15.47:6805/775837313","heartbeat_back_addr":"172.21.15.47:6809/775837313","heartbeat_front_addr":"172.21.15.47:6807/775837313","state":["exists","up"]},{"osd":1,"uuid":"07bb81bb-7cf3-4bbc-ac9e-802f95185d11","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6800","nonce":2923304710},{"type":"v1","addr":"172.21.15.100:6801","nonce":2923304710}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":2923304710},{"type":"v1","addr":"172.21.15.100:6803","nonce":2923304710}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":2923304710},{"type":"v1","addr":"172.21.15.100:6807","nonce":2923304710}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6804","nonce":2923304710},{"type":"v1","addr":"172.21.15.100:6805","nonce":2923304710}]},"public_addr":"172.21.15.100:6801/2923304710","cluster_addr":"172.21.15.100:6803/2923304710","heartbeat_back_addr":"172.21.15.100:6807/2923304710","heartbeat_front_addr":"172.21.15.100:6805/2923304710","state":["exists","up"]},{"osd":2,"uuid":"88ce76b9-6081-4836-aad0-8c249c56e8a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":171377005},{"type":"v1","addr":"172.21.15.157:6801","nonce":171377005}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":171377005},{"type":"v1","addr":"172.21.15.157:6803","nonce":171377005}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":171377005},{"type":"v1","addr":"172.21.15.157:6807","nonce":171377005}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":171377005},{"type":"v1","addr":"172.21.15.157:6805","nonce":171377005}]},"public_addr":"172.21.15.157:6801/171377005","cluster_addr":"172.21.15.157:6803/171377005","heartbeat_back_addr":"172.21.15.157:6807/171377005","heartbeat_front_addr":"172.21.15.157:6805/171377005","state":["exists","up"]},{"osd":3,"uuid":"518aa33d-263f-45e6-aab5-2201ceb40e74","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":27,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6810","nonce":640760101},{"type":"v1","addr":"172.21.15.47:6811","nonce":640760101}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6812","nonce":640760101},{"type":"v1","addr":"172.21.15.47:6813","nonce":640760101}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6816","nonce":640760101},{"type":"v1","addr":"172.21.15.47:6817","nonce":640760101}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6814","nonce":640760101},{"type":"v1","addr":"172.21.15.47:6815","nonce":640760101}]},"public_addr":"172.21.15.47:6811/640760101","cluster_addr":"172.21.15.47:6813/640760101","heartbeat_back_addr":"172.21.15.47:6817/640760101","heartbeat_front_addr":"172.21.15.47:6815/640760101","state":["exists","up"]},{"osd":4,"uuid":"987bde83-ecc6-42e5-bd56-c143b3de4226","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":35,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6808","nonce":2785478639},{"type":"v1","addr":"172.21.15.100:6809","nonce":2785478639}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6810","nonce":2785478639},{"type":"v1","addr":"172.21.15.100:6811","nonce":2785478639}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6814","nonce":2785478639},{"type":"v1","addr":"172.21.15.100:6815","nonce":2785478639}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":2785478639},{"type":"v1","addr":"172.21.15.100:6813","nonce":2785478639}]},"public_addr":"172.21.15.100:6809/2785478639","cluster_addr":"172.21.15.100:6811/2785478639","heartbeat_back_addr":"172.21.15.100:6815/2785478639","heartbeat_front_addr":"172.21.15.100:6813/2785478639","state":["exists","up"]},{"osd":5,"uuid":"012d2188-3a61-4a5f-8d01-2001c15ffbe2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":40,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":1525645410},{"type":"v1","addr":"172.21.15.157:6809","nonce":1525645410}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":1525645410},{"type":"v1","addr":"172.21.15.157:6811","nonce":1525645410}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":1525645410},{"type":"v1","addr":"172.21.15.157:6815","nonce":1525645410}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":1525645410},{"type":"v1","addr":"172.21.15.157:6813","nonce":1525645410}]},"public_addr":"172.21.15.157:6809/1525645410","cluster_addr":"172.21.15.157:6811/1525645410","heartbeat_back_addr":"172.21.15.157:6815/1525645410","heartbeat_front_addr":"172.21.15.157:6813/1525645410","state":["exists","up"]},{"osd":6,"uuid":"c12c14b4-85f0-4d88-937c-de4f556437de","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6818","nonce":2321642790},{"type":"v1","addr":"172.21.15.47:6819","nonce":2321642790}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6820","nonce":2321642790},{"type":"v1","addr":"172.21.15.47:6821","nonce":2321642790}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6824","nonce":2321642790},{"type":"v1","addr":"172.21.15.47:6825","nonce":2321642790}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6822","nonce":2321642790},{"type":"v1","addr":"172.21.15.47:6823","nonce":2321642790}]},"public_addr":"172.21.15.47:6819/2321642790","cluster_addr":"172.21.15.47:6821/2321642790","heartbeat_back_addr":"172.21.15.47:6825/2321642790","heartbeat_front_addr":"172.21.15.47:6823/2321642790","state":["exists","up"]},{"osd":7,"uuid":"3c87c6d0-1d1f-4294-9917-aaaad7dda824","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6816","nonce":4104727330},{"type":"v1","addr":"172.21.15.100:6817","nonce":4104727330}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":4104727330},{"type":"v1","addr":"172.21.15.100:6819","nonce":4104727330}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":4104727330},{"type":"v1","addr":"172.21.15.100:6823","nonce":4104727330}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6820","nonce":4104727330},{"type":"v1","addr":"172.21.15.100:6821","nonce":4104727330}]},"public_addr":"172.21.15.100:6817/4104727330","cluster_addr":"172.21.15.100:6819/4104727330","heartbeat_back_addr":"172.21.15.100:6823/4104727330","heartbeat_front_addr":"172.21.15.100:6821/4104727330","state":["exists","up"]},{"osd":8,"uuid":"502d65cf-53c8-4936-8de4-e11b2a6e1e18","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":61,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":4067715347},{"type":"v1","addr":"172.21.15.157:6817","nonce":4067715347}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":4067715347},{"type":"v1","addr":"172.21.15.157:6819","nonce":4067715347}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":4067715347},{"type":"v1","addr":"172.21.15.157:6823","nonce":4067715347}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":4067715347},{"type":"v1","addr":"172.21.15.157:6821","nonce":4067715347}]},"public_addr":"172.21.15.157:6817/4067715347","cluster_addr":"172.21.15.157:6819/4067715347","heartbeat_back_addr":"172.21.15.157:6823/4067715347","heartbeat_front_addr":"172.21.15.157:6821/4067715347","state":["exists","up"]},{"osd":9,"uuid":"5ea1f5be-8458-4c93-9e61-c2e867a514f3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6826","nonce":1982108738},{"type":"v1","addr":"172.21.15.47:6827","nonce":1982108738}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6828","nonce":1982108738},{"type":"v1","addr":"172.21.15.47:6829","nonce":1982108738}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6832","nonce":1982108738},{"type":"v1","addr":"172.21.15.47:6833","nonce":1982108738}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6830","nonce":1982108738},{"type":"v1","addr":"172.21.15.47:6831","nonce":1982108738}]},"public_addr":"172.21.15.47:6827/1982108738","cluster_addr":"172.21.15.47:6829/1982108738","heartbeat_back_addr":"172.21.15.47:6833/1982108738","heartbeat_front_addr":"172.21.15.47:6831/1982108738","state":["exists","up"]},{"osd":10,"uuid":"2da3da90-a0be-4a54-a9ff-b2d6c9b36870","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":74,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":396391028},{"type":"v1","addr":"172.21.15.100:6825","nonce":396391028}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":396391028},{"type":"v1","addr":"172.21.15.100:6827","nonce":396391028}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6830","nonce":396391028},{"type":"v1","addr":"172.21.15.100:6831","nonce":396391028}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":396391028},{"type":"v1","addr":"172.21.15.100:6829","nonce":396391028}]},"public_addr":"172.21.15.100:6825/396391028","cluster_addr":"172.21.15.100:6827/396391028","heartbeat_back_addr":"172.21.15.100:6831/396391028","heartbeat_front_addr":"172.21.15.100:6829/396391028","state":["exists","up"]},{"osd":11,"uuid":"11798d41-7f7e-4319-95b8-8d3d0946bde9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":82,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":2671029483},{"type":"v1","addr":"172.21.15.157:6825","nonce":2671029483}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":2671029483},{"type":"v1","addr":"172.21.15.157:6827","nonce":2671029483}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":2671029483},{"type":"v1","addr":"172.21.15.157:6831","nonce":2671029483}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":2671029483},{"type":"v1","addr":"172.21.15.157:6829","nonce":2671029483}]},"public_addr":"172.21.15.157:6825/2671029483","cluster_addr":"172.21.15.157:6827/2671029483","heartbeat_back_addr":"172.21.15.157:6831/2671029483","heartbeat_front_addr":"172.21.15.157:6829/2671029483","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T10:14:31.822399+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T10:15:04.175721+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T10:15:35.902595+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T10:16:12.088891+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T10:16:46.428704+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T10:17:20.158270+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T10:17:55.915253+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T10:18:30.064641+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T10:19:04.410004+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T10:19:40.605181+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T10:20:15.636605+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T10:20:50.151365+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.47:0/356822431":"2024-03-22T10:12:25.844530+0000","172.21.15.47:0/1167490879":"2024-03-22T10:12:25.844530+0000","172.21.15.47:0/1540718138":"2024-03-22T10:12:25.844530+0000","172.21.15.47:6801/3836903972":"2024-03-22T10:12:25.844530+0000","172.21.15.47:0/235436602":"2024-03-22T10:11:50.301838+0000","172.21.15.47:0/2052815572":"2024-03-22T10:11:50.301838+0000","172.21.15.47:6800/3942658432":"2024-03-22T10:11:50.301838+0000","172.21.15.47:6801/3942658432":"2024-03-22T10:11:50.301838+0000","172.21.15.47:0/1419516257":"2024-03-22T10:11:50.301838+0000","172.21.15.47:6800/3836903972":"2024-03-22T10:12:25.844530+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-21T10:22:19.386 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:19.384+0000 7ff7eaa00640 1 -- 172.21.15.47:0/3014412117 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7ff7cc0a2fa0 msgr2=0x7ff7cc0a5460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:22:19.386 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:19.384+0000 7ff7eaa00640 1 --2- 172.21.15.47:0/3014412117 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7ff7cc0a2fa0 0x7ff7cc0a5460 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7ff7e419ad10 tx=0x7ff7d4000bc0 comp rx=0 tx=0).stop 2024-03-21T10:22:19.386 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:19.384+0000 7ff7eaa00640 1 -- 172.21.15.47:0/3014412117 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff7e41036f0 msgr2=0x7ff7e4199800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:22:19.386 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:19.384+0000 7ff7eaa00640 1 --2- 172.21.15.47:0/3014412117 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff7e41036f0 0x7ff7e4199800 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7ff7d800c970 tx=0x7ff7d800ce30 comp rx=0 tx=0).stop 2024-03-21T10:22:19.387 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:19.385+0000 7ff7eaa00640 1 -- 172.21.15.47:0/3014412117 shutdown_connections 2024-03-21T10:22:19.387 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:19.385+0000 7ff7eaa00640 1 --2- 172.21.15.47:0/3014412117 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7ff7cc0a2fa0 0x7ff7cc0a5460 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:22:19.387 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:19.385+0000 7ff7eaa00640 1 --2- 172.21.15.47:0/3014412117 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff7e41052a0 0x7ff7e41a0db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:22:19.387 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:19.385+0000 7ff7eaa00640 1 --2- 172.21.15.47:0/3014412117 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff7e41048e0 0x7ff7e4199d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:22:19.387 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:19.385+0000 7ff7eaa00640 1 --2- 172.21.15.47:0/3014412117 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff7e41036f0 0x7ff7e4199800 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:22:19.387 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:19.385+0000 7ff7eaa00640 1 -- 172.21.15.47:0/3014412117 >> 172.21.15.47:0/3014412117 conn(0x7ff7e40fec80 msgr2=0x7ff7e4107b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:22:19.387 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:19.385+0000 7ff7eaa00640 1 -- 172.21.15.47:0/3014412117 shutdown_connections 2024-03-21T10:22:19.387 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:19.385+0000 7ff7eaa00640 1 -- 172.21.15.47:0/3014412117 wait complete. 2024-03-21T10:22:19.406 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-21T10:15:40.553114+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-21T10:21:56.399518+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '86', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-21T10:21:57.812940+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '87', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-03-21T10:22:01.822568+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '91', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-03-21T10:22:19.406 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-21T10:22:19.410 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:22:19 smithi100 ceph-mon[35193]: pgmap v377: 129 pgs: 129 active+clean; 579 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-21T10:22:19.411 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:22:19 smithi100 ceph-mon[35193]: from='client.? 172.21.15.47:0/1488222901' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-03-21T10:22:19.411 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:22:19 smithi100 ceph-mon[35193]: from='client.? 172.21.15.47:0/427733293' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-03-21T10:22:19.490 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:22:19 smithi157 ceph-mon[35581]: pgmap v377: 129 pgs: 129 active+clean; 579 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-21T10:22:19.491 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:22:19 smithi157 ceph-mon[35581]: from='client.? 172.21.15.47:0/1488222901' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-03-21T10:22:19.491 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:22:19 smithi157 ceph-mon[35581]: from='client.? 172.21.15.47:0/427733293' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-03-21T10:22:19.630 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:19.628+0000 7fde36600640 1 -- 172.21.15.47:0/2697854562 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fde301037c0 msgr2=0x7fde30103be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:22:19.630 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:19.628+0000 7fde36600640 1 --2- 172.21.15.47:0/2697854562 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fde301037c0 0x7fde30103be0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fde24009900 tx=0x7fde2402bc30 comp rx=0 tx=0).stop 2024-03-21T10:22:19.630 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:19.629+0000 7fde36600640 1 -- 172.21.15.47:0/2697854562 shutdown_connections 2024-03-21T10:22:19.630 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:19.629+0000 7fde36600640 1 --2- 172.21.15.47:0/2697854562 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fde30105370 0x7fde3010bc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:22:19.630 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:19.629+0000 7fde36600640 1 --2- 172.21.15.47:0/2697854562 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fde301049b0 0x7fde30104e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:22:19.630 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:19.629+0000 7fde36600640 1 --2- 172.21.15.47:0/2697854562 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fde301037c0 0x7fde30103be0 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:22:19.630 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:19.629+0000 7fde36600640 1 -- 172.21.15.47:0/2697854562 >> 172.21.15.47:0/2697854562 conn(0x7fde300fed50 msgr2=0x7fde30101170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:22:19.631 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:19.629+0000 7fde36600640 1 -- 172.21.15.47:0/2697854562 shutdown_connections 2024-03-21T10:22:19.631 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:19.629+0000 7fde36600640 1 -- 172.21.15.47:0/2697854562 wait complete. 2024-03-21T10:22:19.633 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:19.630+0000 7fde36600640 1 Processor -- start 2024-03-21T10:22:19.633 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:19.630+0000 7fde36600640 1 -- start start 2024-03-21T10:22:19.633 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:19.630+0000 7fde36600640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fde301037c0 0x7fde301999e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:22:19.633 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:19.630+0000 7fde36600640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fde301049b0 0x7fde30199f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:22:19.633 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:19.630+0000 7fde36600640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fde30105370 0x7fde301a0f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:22:19.633 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:19.630+0000 7fde36600640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde3010f340 con 0x7fde301037c0 2024-03-21T10:22:19.633 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:19.630+0000 7fde36600640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde3010f1c0 con 0x7fde30105370 2024-03-21T10:22:19.633 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:19.630+0000 7fde36600640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde3010f4c0 con 0x7fde301049b0 2024-03-21T10:22:19.633 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:19.630+0000 7fde35200640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fde301037c0 0x7fde301999e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:22:19.633 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:19.630+0000 7fde35200640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fde301037c0 0x7fde301999e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:51740/0 (socket says 172.21.15.47:51740) 2024-03-21T10:22:19.633 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:19.630+0000 7fde35200640 1 -- 172.21.15.47:0/571473305 learned_addr learned my addr 172.21.15.47:0/571473305 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:22:19.634 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:19.631+0000 7fde35c00640 1 --2- 172.21.15.47:0/571473305 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fde30105370 0x7fde301a0f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:22:19.634 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:19.631+0000 7fde2fe00640 1 --2- 172.21.15.47:0/571473305 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fde301049b0 0x7fde30199f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:22:19.634 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:19.631+0000 7fde35200640 1 -- 172.21.15.47:0/571473305 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fde301049b0 msgr2=0x7fde30199f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:22:19.634 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:19.631+0000 7fde35200640 1 --2- 172.21.15.47:0/571473305 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fde301049b0 0x7fde30199f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:22:19.634 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:19.631+0000 7fde35200640 1 -- 172.21.15.47:0/571473305 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fde30105370 msgr2=0x7fde301a0f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:22:19.634 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:19.631+0000 7fde35200640 1 --2- 172.21.15.47:0/571473305 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fde30105370 0x7fde301a0f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:22:19.634 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:19.631+0000 7fde35200640 1 -- 172.21.15.47:0/571473305 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde301a1600 con 0x7fde301037c0 2024-03-21T10:22:19.634 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:19.631+0000 7fde35c00640 1 --2- 172.21.15.47:0/571473305 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fde30105370 0x7fde301a0f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T10:22:19.634 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:19.631+0000 7fde35200640 1 --2- 172.21.15.47:0/571473305 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fde301037c0 0x7fde301999e0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fde24005ab0 tx=0x7fde240042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:22:19.634 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:19.632+0000 7fde2d600640 1 -- 172.21.15.47:0/571473305 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde24043020 con 0x7fde301037c0 2024-03-21T10:22:19.634 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:19.632+0000 7fde2d600640 1 -- 172.21.15.47:0/571473305 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fde240054c0 con 0x7fde301037c0 2024-03-21T10:22:19.634 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:19.632+0000 7fde36600640 1 -- 172.21.15.47:0/571473305 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fde301a1890 con 0x7fde301037c0 2024-03-21T10:22:19.634 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:19.632+0000 7fde2d600640 1 -- 172.21.15.47:0/571473305 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde24003d10 con 0x7fde301037c0 2024-03-21T10:22:19.634 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:19.632+0000 7fde36600640 1 -- 172.21.15.47:0/571473305 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fde301a1e30 con 0x7fde301037c0 2024-03-21T10:22:19.635 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:19.633+0000 7fde36600640 1 -- 172.21.15.47:0/571473305 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fde3010ab60 con 0x7fde301037c0 2024-03-21T10:22:19.638 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:19.638+0000 7fde2d600640 1 -- 172.21.15.47:0/571473305 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 16) v1 ==== 137854+0+0 (secure 0 0 0) 0x7fde24046020 con 0x7fde301037c0 2024-03-21T10:22:19.640 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:19.639+0000 7fde2d600640 1 --2- 172.21.15.47:0/571473305 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7fde180a2fa0 0x7fde180a5460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:22:19.640 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:19.639+0000 7fde2d600640 1 -- 172.21.15.47:0/571473305 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 8802+0+0 (secure 0 0 0) 0x7fde240f1dc0 con 0x7fde301037c0 2024-03-21T10:22:19.640 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:19.639+0000 7fde2fe00640 1 --2- 172.21.15.47:0/571473305 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7fde180a2fa0 0x7fde180a5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:22:19.642 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:19.641+0000 7fde2fe00640 1 --2- 172.21.15.47:0/571473305 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7fde180a2fa0 0x7fde180a5460 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fde1c005fd0 tx=0x7fde1c005c70 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:22:19.643 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:19.642+0000 7fde2d600640 1 -- 172.21.15.47:0/571473305 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fde2404d020 con 0x7fde301037c0 2024-03-21T10:22:19.845 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:19.844+0000 7fde36600640 1 -- 172.21.15.47:0/571473305 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fde3002d300 con 0x7fde301037c0 2024-03-21T10:22:19.846 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:19.845+0000 7fde2d600640 1 -- 172.21.15.47:0/571473305 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v91) v1 ==== 93+0+10 (secure 0 0 0) 0x7fde240b8e20 con 0x7fde301037c0 2024-03-21T10:22:19.846 INFO:teuthology.orchestra.run.smithi047.stdout:pg_num: 1 2024-03-21T10:22:19.849 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:19.847+0000 7fde36600640 1 -- 172.21.15.47:0/571473305 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7fde180a2fa0 msgr2=0x7fde180a5460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:22:19.850 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:19.847+0000 7fde36600640 1 --2- 172.21.15.47:0/571473305 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7fde180a2fa0 0x7fde180a5460 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fde1c005fd0 tx=0x7fde1c005c70 comp rx=0 tx=0).stop 2024-03-21T10:22:19.850 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:19.847+0000 7fde36600640 1 -- 172.21.15.47:0/571473305 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fde301037c0 msgr2=0x7fde301999e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:22:19.850 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:19.847+0000 7fde36600640 1 --2- 172.21.15.47:0/571473305 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fde301037c0 0x7fde301999e0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fde24005ab0 tx=0x7fde240042f0 comp rx=0 tx=0).stop 2024-03-21T10:22:19.850 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:19.848+0000 7fde36600640 1 -- 172.21.15.47:0/571473305 shutdown_connections 2024-03-21T10:22:19.851 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:19.848+0000 7fde36600640 1 --2- 172.21.15.47:0/571473305 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7fde180a2fa0 0x7fde180a5460 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:22:19.851 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:19.848+0000 7fde36600640 1 --2- 172.21.15.47:0/571473305 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fde30105370 0x7fde301a0f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:22:19.851 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:19.848+0000 7fde36600640 1 --2- 172.21.15.47:0/571473305 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fde301049b0 0x7fde30199f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:22:19.851 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:19.848+0000 7fde36600640 1 --2- 172.21.15.47:0/571473305 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fde301037c0 0x7fde301999e0 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:22:19.851 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:19.848+0000 7fde36600640 1 -- 172.21.15.47:0/571473305 >> 172.21.15.47:0/571473305 conn(0x7fde300fed50 msgr2=0x7fde30107d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:22:19.851 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:19.848+0000 7fde36600640 1 -- 172.21.15.47:0/571473305 shutdown_connections 2024-03-21T10:22:19.851 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:19.848+0000 7fde36600640 1 -- 172.21.15.47:0/571473305 wait complete. 2024-03-21T10:22:19.867 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-21T10:22:20.099 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:20.097+0000 7f308b000640 1 -- 172.21.15.47:0/839359190 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3084104ab0 msgr2=0x7f3084104f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:22:20.099 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:20.097+0000 7f308b000640 1 --2- 172.21.15.47:0/839359190 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3084104ab0 0x7f3084104f30 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f306c009900 tx=0x7f306c02bc30 comp rx=0 tx=0).stop 2024-03-21T10:22:20.099 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:20.097+0000 7f308b000640 1 -- 172.21.15.47:0/839359190 shutdown_connections 2024-03-21T10:22:20.099 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:20.097+0000 7f308b000640 1 --2- 172.21.15.47:0/839359190 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3084105470 0x7f308410bd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:22:20.100 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:20.097+0000 7f308b000640 1 --2- 172.21.15.47:0/839359190 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3084104ab0 0x7f3084104f30 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:22:20.100 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:20.097+0000 7f308b000640 1 --2- 172.21.15.47:0/839359190 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f30841038c0 0x7f3084103ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:22:20.100 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:20.097+0000 7f308b000640 1 -- 172.21.15.47:0/839359190 >> 172.21.15.47:0/839359190 conn(0x7f30840fee50 msgr2=0x7f3084101270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:22:20.100 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:20.097+0000 7f308b000640 1 -- 172.21.15.47:0/839359190 shutdown_connections 2024-03-21T10:22:20.100 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:20.097+0000 7f308b000640 1 -- 172.21.15.47:0/839359190 wait complete. 2024-03-21T10:22:20.103 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:20.098+0000 7f308b000640 1 Processor -- start 2024-03-21T10:22:20.103 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:20.099+0000 7f308b000640 1 -- start start 2024-03-21T10:22:20.103 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:20.099+0000 7f308b000640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f30841038c0 0x7f3084199b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:22:20.103 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:20.099+0000 7f308b000640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3084104ab0 0x7f308419a080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:22:20.103 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:20.099+0000 7f308b000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3084105470 0x7f30841a10f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:22:20.103 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:20.099+0000 7f308b000640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f308410f440 con 0x7f30841038c0 2024-03-21T10:22:20.104 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:20.099+0000 7f308b000640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f308410f2c0 con 0x7f3084105470 2024-03-21T10:22:20.104 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:20.099+0000 7f308b000640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f308410f5c0 con 0x7f3084104ab0 2024-03-21T10:22:20.104 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:20.099+0000 7f3083e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f30841038c0 0x7f3084199b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:22:20.104 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:20.099+0000 7f3083e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f30841038c0 0x7f3084199b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:51760/0 (socket says 172.21.15.47:51760) 2024-03-21T10:22:20.104 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:20.099+0000 7f3083e00640 1 -- 172.21.15.47:0/3096041762 learned_addr learned my addr 172.21.15.47:0/3096041762 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:22:20.104 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:20.100+0000 7f3089000640 1 --2- 172.21.15.47:0/3096041762 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3084105470 0x7f30841a10f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:22:20.104 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:20.100+0000 7f3083400640 1 --2- 172.21.15.47:0/3096041762 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3084104ab0 0x7f308419a080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:22:20.104 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:20.100+0000 7f3083e00640 1 -- 172.21.15.47:0/3096041762 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3084104ab0 msgr2=0x7f308419a080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:22:20.104 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:20.100+0000 7f3083e00640 1 --2- 172.21.15.47:0/3096041762 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3084104ab0 0x7f308419a080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:22:20.104 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:20.100+0000 7f3083e00640 1 -- 172.21.15.47:0/3096041762 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3084105470 msgr2=0x7f30841a10f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:22:20.105 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:20.100+0000 7f3083e00640 1 --2- 172.21.15.47:0/3096041762 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3084105470 0x7f30841a10f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:22:20.105 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:20.100+0000 7f3083e00640 1 -- 172.21.15.47:0/3096041762 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f30841a1760 con 0x7f30841038c0 2024-03-21T10:22:20.105 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:20.100+0000 7f3089000640 1 --2- 172.21.15.47:0/3096041762 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3084105470 0x7f30841a10f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T10:22:20.105 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:20.100+0000 7f3083e00640 1 --2- 172.21.15.47:0/3096041762 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f30841038c0 0x7f3084199b40 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f307400d770 tx=0x7f307400dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:22:20.105 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:20.100+0000 7f3080c00640 1 -- 172.21.15.47:0/3096041762 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3074017070 con 0x7f30841038c0 2024-03-21T10:22:20.105 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:20.101+0000 7f3080c00640 1 -- 172.21.15.47:0/3096041762 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f3074009c30 con 0x7f30841038c0 2024-03-21T10:22:20.105 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:20.101+0000 7f3080c00640 1 -- 172.21.15.47:0/3096041762 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3074003d00 con 0x7f30841038c0 2024-03-21T10:22:20.105 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:20.101+0000 7f308b000640 1 -- 172.21.15.47:0/3096041762 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f30841a1a50 con 0x7f30841038c0 2024-03-21T10:22:20.105 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:20.101+0000 7f308b000640 1 -- 172.21.15.47:0/3096041762 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f30841a1ff0 con 0x7f30841038c0 2024-03-21T10:22:20.105 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:20.102+0000 7f308b000640 1 -- 172.21.15.47:0/3096041762 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f308410ac50 con 0x7f30841038c0 2024-03-21T10:22:20.105 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:20.104+0000 7f3080c00640 1 -- 172.21.15.47:0/3096041762 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 16) v1 ==== 137854+0+0 (secure 0 0 0) 0x7f3074003ea0 con 0x7f30841038c0 2024-03-21T10:22:20.106 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:20.105+0000 7f3080c00640 1 --2- 172.21.15.47:0/3096041762 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f30640a2fa0 0x7f30640a5460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:22:20.106 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:20.105+0000 7f3080c00640 1 -- 172.21.15.47:0/3096041762 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 8802+0+0 (secure 0 0 0) 0x7f30740cf380 con 0x7f30841038c0 2024-03-21T10:22:20.106 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:20.106+0000 7f3083400640 1 --2- 172.21.15.47:0/3096041762 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f30640a2fa0 0x7f30640a5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:22:20.107 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:20.106+0000 7f3083400640 1 --2- 172.21.15.47:0/3096041762 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f30640a2fa0 0x7f30640a5460 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f306c0096c0 tx=0x7f306c005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:22:20.111 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:20.110+0000 7f3080c00640 1 -- 172.21.15.47:0/3096041762 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f307409bf70 con 0x7f30841038c0 2024-03-21T10:22:20.214 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:22:20 smithi047 ceph-mon[31547]: from='client.? 172.21.15.47:0/3014412117' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-21T10:22:20.214 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:22:20 smithi047 ceph-mon[31547]: from='client.? 172.21.15.47:0/571473305' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-21T10:22:20.315 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:20.313+0000 7f308b000640 1 -- 172.21.15.47:0/3096041762 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f30840622d0 con 0x7f30841038c0 2024-03-21T10:22:20.316 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:20.315+0000 7f3080c00640 1 -- 172.21.15.47:0/3096041762 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v91) v1 ==== 104+0+11 (secure 0 0 0) 0x7f3074009de0 con 0x7f30841038c0 2024-03-21T10:22:20.316 INFO:teuthology.orchestra.run.smithi047.stdout:pg_num: 32 2024-03-21T10:22:20.319 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:20.317+0000 7f308b000640 1 -- 172.21.15.47:0/3096041762 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f30640a2fa0 msgr2=0x7f30640a5460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:22:20.319 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:20.317+0000 7f308b000640 1 --2- 172.21.15.47:0/3096041762 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f30640a2fa0 0x7f30640a5460 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f306c0096c0 tx=0x7f306c005fd0 comp rx=0 tx=0).stop 2024-03-21T10:22:20.319 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:20.317+0000 7f308b000640 1 -- 172.21.15.47:0/3096041762 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f30841038c0 msgr2=0x7f3084199b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:22:20.319 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:20.317+0000 7f308b000640 1 --2- 172.21.15.47:0/3096041762 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f30841038c0 0x7f3084199b40 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f307400d770 tx=0x7f307400dc30 comp rx=0 tx=0).stop 2024-03-21T10:22:20.320 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:20.318+0000 7f308b000640 1 -- 172.21.15.47:0/3096041762 shutdown_connections 2024-03-21T10:22:20.320 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:20.318+0000 7f308b000640 1 --2- 172.21.15.47:0/3096041762 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f30640a2fa0 0x7f30640a5460 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:22:20.320 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:20.318+0000 7f308b000640 1 --2- 172.21.15.47:0/3096041762 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3084105470 0x7f30841a10f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:22:20.320 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:20.318+0000 7f308b000640 1 --2- 172.21.15.47:0/3096041762 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3084104ab0 0x7f308419a080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:22:20.320 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:20.318+0000 7f308b000640 1 --2- 172.21.15.47:0/3096041762 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f30841038c0 0x7f3084199b40 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:22:20.320 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:20.318+0000 7f308b000640 1 -- 172.21.15.47:0/3096041762 >> 172.21.15.47:0/3096041762 conn(0x7f30840fee50 msgr2=0x7f3084107e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:22:20.320 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:20.318+0000 7f308b000640 1 -- 172.21.15.47:0/3096041762 shutdown_connections 2024-03-21T10:22:20.320 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:20.318+0000 7f308b000640 1 -- 172.21.15.47:0/3096041762 wait complete. 2024-03-21T10:22:20.338 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-21T10:22:20.480 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:22:20 smithi100 ceph-mon[35193]: from='client.? 172.21.15.47:0/3014412117' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-21T10:22:20.480 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:22:20 smithi100 ceph-mon[35193]: from='client.? 172.21.15.47:0/571473305' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-21T10:22:20.490 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:22:20 smithi157 ceph-mon[35581]: from='client.? 172.21.15.47:0/3014412117' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-21T10:22:20.490 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:22:20 smithi157 ceph-mon[35581]: from='client.? 172.21.15.47:0/571473305' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-21T10:22:20.560 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:20.559+0000 7fc447800640 1 -- 172.21.15.47:0/1121159892 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc4400772a0 msgr2=0x7fc440077700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:22:20.560 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:20.559+0000 7fc447800640 1 --2- 172.21.15.47:0/1121159892 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc4400772a0 0x7fc440077700 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fc42c009900 tx=0x7fc42c02bc30 comp rx=0 tx=0).stop 2024-03-21T10:22:20.561 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:20.559+0000 7fc447800640 1 -- 172.21.15.47:0/1121159892 shutdown_connections 2024-03-21T10:22:20.561 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:20.559+0000 7fc447800640 1 --2- 172.21.15.47:0/1121159892 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc440077c40 0x7fc4400784e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:22:20.561 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:20.559+0000 7fc447800640 1 --2- 172.21.15.47:0/1121159892 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc4400772a0 0x7fc440077700 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:22:20.561 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:20.559+0000 7fc447800640 1 --2- 172.21.15.47:0/1121159892 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc440076060 0x7fc440076480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:22:20.561 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:20.559+0000 7fc447800640 1 -- 172.21.15.47:0/1121159892 >> 172.21.15.47:0/1121159892 conn(0x7fc4400fee50 msgr2=0x7fc440101290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:22:20.561 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:20.559+0000 7fc447800640 1 -- 172.21.15.47:0/1121159892 shutdown_connections 2024-03-21T10:22:20.561 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:20.559+0000 7fc447800640 1 -- 172.21.15.47:0/1121159892 wait complete. 2024-03-21T10:22:20.564 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:20.560+0000 7fc447800640 1 Processor -- start 2024-03-21T10:22:20.564 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:20.560+0000 7fc447800640 1 -- start start 2024-03-21T10:22:20.564 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:20.560+0000 7fc447800640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc440076060 0x7fc44019df50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:22:20.564 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:20.560+0000 7fc447800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc4400772a0 0x7fc44019e490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:22:20.564 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:20.560+0000 7fc447800640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc440077c40 0x7fc4401a5500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:22:20.564 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:20.560+0000 7fc447800640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc440113860 con 0x7fc440077c40 2024-03-21T10:22:20.564 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:20.560+0000 7fc447800640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4401136e0 con 0x7fc4400772a0 2024-03-21T10:22:20.564 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:20.560+0000 7fc447800640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4401139e0 con 0x7fc440076060 2024-03-21T10:22:20.564 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:20.561+0000 7fc445800640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc440077c40 0x7fc4401a5500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:22:20.564 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:20.561+0000 7fc445800640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc440077c40 0x7fc4401a5500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:51768/0 (socket says 172.21.15.47:51768) 2024-03-21T10:22:20.565 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:20.561+0000 7fc445800640 1 -- 172.21.15.47:0/3969267362 learned_addr learned my addr 172.21.15.47:0/3969267362 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:22:20.565 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:20.561+0000 7fc43fe00640 1 --2- 172.21.15.47:0/3969267362 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc4400772a0 0x7fc44019e490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:22:20.565 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:20.561+0000 7fc444e00640 1 --2- 172.21.15.47:0/3969267362 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc440076060 0x7fc44019df50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:22:20.565 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:20.561+0000 7fc445800640 1 -- 172.21.15.47:0/3969267362 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc440076060 msgr2=0x7fc44019df50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:22:20.565 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:20.561+0000 7fc445800640 1 --2- 172.21.15.47:0/3969267362 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc440076060 0x7fc44019df50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:22:20.565 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:20.561+0000 7fc445800640 1 -- 172.21.15.47:0/3969267362 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc4400772a0 msgr2=0x7fc44019e490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:22:20.565 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:20.561+0000 7fc445800640 1 --2- 172.21.15.47:0/3969267362 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc4400772a0 0x7fc44019e490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:22:20.565 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:20.561+0000 7fc445800640 1 -- 172.21.15.47:0/3969267362 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc4401a5b70 con 0x7fc440077c40 2024-03-21T10:22:20.565 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:20.561+0000 7fc444e00640 1 --2- 172.21.15.47:0/3969267362 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc440076060 0x7fc44019df50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T10:22:20.565 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:20.562+0000 7fc445800640 1 --2- 172.21.15.47:0/3969267362 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc440077c40 0x7fc4401a5500 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fc43800eaa0 tx=0x7fc43800ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:22:20.565 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:20.562+0000 7fc43d600640 1 -- 172.21.15.47:0/3969267362 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc43800cdb0 con 0x7fc440077c40 2024-03-21T10:22:20.565 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:20.562+0000 7fc43d600640 1 -- 172.21.15.47:0/3969267362 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fc438005020 con 0x7fc440077c40 2024-03-21T10:22:20.565 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:20.562+0000 7fc43d600640 1 -- 172.21.15.47:0/3969267362 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4380116d0 con 0x7fc440077c40 2024-03-21T10:22:20.565 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:20.562+0000 7fc447800640 1 -- 172.21.15.47:0/3969267362 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc4401a5e60 con 0x7fc440077c40 2024-03-21T10:22:20.566 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:20.562+0000 7fc447800640 1 -- 172.21.15.47:0/3969267362 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc4401a62d0 con 0x7fc440077c40 2024-03-21T10:22:20.566 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:20.563+0000 7fc447800640 1 -- 172.21.15.47:0/3969267362 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc440110250 con 0x7fc440077c40 2024-03-21T10:22:20.566 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:20.566+0000 7fc43d600640 1 -- 172.21.15.47:0/3969267362 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 16) v1 ==== 137854+0+0 (secure 0 0 0) 0x7fc4380051d0 con 0x7fc440077c40 2024-03-21T10:22:20.569 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:20.568+0000 7fc43d600640 1 --2- 172.21.15.47:0/3969267362 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7fc4280a2fa0 0x7fc4280a5460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:22:20.569 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:20.568+0000 7fc43d600640 1 -- 172.21.15.47:0/3969267362 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 8802+0+0 (secure 0 0 0) 0x7fc4380cf7c0 con 0x7fc440077c40 2024-03-21T10:22:20.569 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:20.568+0000 7fc444e00640 1 --2- 172.21.15.47:0/3969267362 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7fc4280a2fa0 0x7fc4280a5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:22:20.570 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:20.569+0000 7fc444e00640 1 --2- 172.21.15.47:0/3969267362 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7fc4280a2fa0 0x7fc4280a5460 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fc4340058c0 tx=0x7fc434005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:22:20.573 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:20.573+0000 7fc43d600640 1 -- 172.21.15.47:0/3969267362 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fc438016030 con 0x7fc440077c40 2024-03-21T10:22:20.777 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:20.776+0000 7fc447800640 1 -- 172.21.15.47:0/3969267362 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fc4400623a0 con 0x7fc440077c40 2024-03-21T10:22:20.778 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:20.777+0000 7fc43d600640 1 -- 172.21.15.47:0/3969267362 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v91) v1 ==== 100+0+11 (secure 0 0 0) 0x7fc43809c350 con 0x7fc440077c40 2024-03-21T10:22:20.778 INFO:teuthology.orchestra.run.smithi047.stdout:pg_num: 64 2024-03-21T10:22:20.782 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:20.780+0000 7fc447800640 1 -- 172.21.15.47:0/3969267362 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7fc4280a2fa0 msgr2=0x7fc4280a5460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:22:20.782 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:20.780+0000 7fc447800640 1 --2- 172.21.15.47:0/3969267362 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7fc4280a2fa0 0x7fc4280a5460 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fc4340058c0 tx=0x7fc434005850 comp rx=0 tx=0).stop 2024-03-21T10:22:20.782 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:20.780+0000 7fc447800640 1 -- 172.21.15.47:0/3969267362 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc440077c40 msgr2=0x7fc4401a5500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:22:20.782 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:20.780+0000 7fc447800640 1 --2- 172.21.15.47:0/3969267362 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc440077c40 0x7fc4401a5500 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fc43800eaa0 tx=0x7fc43800ef60 comp rx=0 tx=0).stop 2024-03-21T10:22:20.782 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:20.780+0000 7fc447800640 1 -- 172.21.15.47:0/3969267362 shutdown_connections 2024-03-21T10:22:20.782 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:20.780+0000 7fc447800640 1 --2- 172.21.15.47:0/3969267362 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7fc4280a2fa0 0x7fc4280a5460 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:22:20.782 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:20.780+0000 7fc447800640 1 --2- 172.21.15.47:0/3969267362 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc440077c40 0x7fc4401a5500 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:22:20.782 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:20.780+0000 7fc447800640 1 --2- 172.21.15.47:0/3969267362 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc4400772a0 0x7fc44019e490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:22:20.782 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:20.780+0000 7fc447800640 1 --2- 172.21.15.47:0/3969267362 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc440076060 0x7fc44019df50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:22:20.782 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:20.780+0000 7fc447800640 1 -- 172.21.15.47:0/3969267362 >> 172.21.15.47:0/3969267362 conn(0x7fc4400fee50 msgr2=0x7fc440100950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:22:20.782 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:20.780+0000 7fc447800640 1 -- 172.21.15.47:0/3969267362 shutdown_connections 2024-03-21T10:22:20.783 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:20.780+0000 7fc447800640 1 -- 172.21.15.47:0/3969267362 wait complete. 2024-03-21T10:22:20.801 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-03-21T10:22:21.027 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:21.025+0000 7fea90800640 1 -- 172.21.15.47:0/960969276 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fea8810c0b0 msgr2=0x7fea8810c4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:22:21.027 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:21.025+0000 7fea90800640 1 --2- 172.21.15.47:0/960969276 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fea8810c0b0 0x7fea8810c4d0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fea7c009900 tx=0x7fea7c02bc30 comp rx=0 tx=0).stop 2024-03-21T10:22:21.027 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:21.025+0000 7fea90800640 1 -- 172.21.15.47:0/960969276 shutdown_connections 2024-03-21T10:22:21.027 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:21.025+0000 7fea90800640 1 --2- 172.21.15.47:0/960969276 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fea8810dc60 0x7fea881144f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:22:21.027 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:21.025+0000 7fea90800640 1 --2- 172.21.15.47:0/960969276 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fea8810d2a0 0x7fea8810d720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:22:21.028 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:21.025+0000 7fea90800640 1 --2- 172.21.15.47:0/960969276 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fea8810c0b0 0x7fea8810c4d0 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:22:21.028 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:21.025+0000 7fea90800640 1 -- 172.21.15.47:0/960969276 >> 172.21.15.47:0/960969276 conn(0x7fea880729f0 msgr2=0x7fea88072e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:22:21.028 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:21.025+0000 7fea90800640 1 -- 172.21.15.47:0/960969276 shutdown_connections 2024-03-21T10:22:21.028 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:21.025+0000 7fea90800640 1 -- 172.21.15.47:0/960969276 wait complete. 2024-03-21T10:22:21.029 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:21.027+0000 7fea90800640 1 Processor -- start 2024-03-21T10:22:21.029 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:21.027+0000 7fea90800640 1 -- start start 2024-03-21T10:22:21.029 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:21.027+0000 7fea90800640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fea8810c0b0 0x7fea881a2370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:22:21.029 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:21.027+0000 7fea90800640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fea8810d2a0 0x7fea881a28b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:22:21.029 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:21.027+0000 7fea90800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fea8810dc60 0x7fea881a9920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:22:21.029 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:21.027+0000 7fea90800640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea88117bf0 con 0x7fea8810d2a0 2024-03-21T10:22:21.029 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:21.027+0000 7fea90800640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea88117a70 con 0x7fea8810dc60 2024-03-21T10:22:21.030 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:21.027+0000 7fea90800640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea88117d70 con 0x7fea8810c0b0 2024-03-21T10:22:21.030 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:21.028+0000 7fea8de00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fea8810c0b0 0x7fea881a2370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:22:21.030 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:21.028+0000 7fea8de00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fea8810c0b0 0x7fea881a2370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.47:41616/0 (socket says 172.21.15.47:41616) 2024-03-21T10:22:21.030 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:21.028+0000 7fea8de00640 1 -- 172.21.15.47:0/1933373454 learned_addr learned my addr 172.21.15.47:0/1933373454 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:22:21.030 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:21.028+0000 7fea8e800640 1 --2- 172.21.15.47:0/1933373454 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fea8810dc60 0x7fea881a9920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:22:21.030 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:21.028+0000 7fea8d400640 1 --2- 172.21.15.47:0/1933373454 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fea8810d2a0 0x7fea881a28b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:22:21.032 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:21.028+0000 7fea8d400640 1 -- 172.21.15.47:0/1933373454 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fea8810c0b0 msgr2=0x7fea881a2370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:22:21.032 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:21.028+0000 7fea8d400640 1 --2- 172.21.15.47:0/1933373454 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fea8810c0b0 0x7fea881a2370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:22:21.033 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:21.028+0000 7fea8d400640 1 -- 172.21.15.47:0/1933373454 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fea8810dc60 msgr2=0x7fea881a9920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:22:21.033 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:21.028+0000 7fea8d400640 1 --2- 172.21.15.47:0/1933373454 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fea8810dc60 0x7fea881a9920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:22:21.033 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:21.029+0000 7fea8d400640 1 -- 172.21.15.47:0/1933373454 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea881a9f90 con 0x7fea8810d2a0 2024-03-21T10:22:21.033 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:21.029+0000 7fea8d400640 1 --2- 172.21.15.47:0/1933373454 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fea8810d2a0 0x7fea881a28b0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fea7400d720 tx=0x7fea7400dbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:22:21.033 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:21.029+0000 7fea82a00640 1 -- 172.21.15.47:0/1933373454 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea74017070 con 0x7fea8810d2a0 2024-03-21T10:22:21.033 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:21.029+0000 7fea90800640 1 -- 172.21.15.47:0/1933373454 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fea881aa280 con 0x7fea8810d2a0 2024-03-21T10:22:21.033 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:21.029+0000 7fea82a00640 1 -- 172.21.15.47:0/1933373454 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fea7400de30 con 0x7fea8810d2a0 2024-03-21T10:22:21.033 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:21.029+0000 7fea82a00640 1 -- 172.21.15.47:0/1933373454 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea740047a0 con 0x7fea8810d2a0 2024-03-21T10:22:21.033 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:21.029+0000 7fea90800640 1 -- 172.21.15.47:0/1933373454 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fea881aa7d0 con 0x7fea8810d2a0 2024-03-21T10:22:21.034 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:21.031+0000 7fea90800640 1 -- 172.21.15.47:0/1933373454 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fea88113540 con 0x7fea8810d2a0 2024-03-21T10:22:21.034 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:21.033+0000 7fea82a00640 1 -- 172.21.15.47:0/1933373454 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 16) v1 ==== 137854+0+0 (secure 0 0 0) 0x7fea740099d0 con 0x7fea8810d2a0 2024-03-21T10:22:21.035 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:21.034+0000 7fea82a00640 1 --2- 172.21.15.47:0/1933373454 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7fea700a2fa0 0x7fea700a5460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:22:21.036 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:21.034+0000 7fea82a00640 1 -- 172.21.15.47:0/1933373454 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 8802+0+0 (secure 0 0 0) 0x7fea740cf5d0 con 0x7fea8810d2a0 2024-03-21T10:22:21.036 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:21.035+0000 7fea8de00640 1 --2- 172.21.15.47:0/1933373454 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7fea700a2fa0 0x7fea700a5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:22:21.036 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:21.035+0000 7fea8de00640 1 --2- 172.21.15.47:0/1933373454 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7fea700a2fa0 0x7fea700a5460 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fea7c02c840 tx=0x7fea7c0095a0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:22:21.038 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:21.037+0000 7fea82a00640 1 -- 172.21.15.47:0/1933373454 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fea74020020 con 0x7fea8810d2a0 2024-03-21T10:22:21.242 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:21.240+0000 7fea90800640 1 -- 172.21.15.47:0/1933373454 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7fea8802d2c0 con 0x7fea8810d2a0 2024-03-21T10:22:21.243 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:21.241+0000 7fea82a00640 1 -- 172.21.15.47:0/1933373454 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v91) v1 ==== 103+0+11 (secure 0 0 0) 0x7fea7409c1e0 con 0x7fea8810d2a0 2024-03-21T10:22:21.243 INFO:teuthology.orchestra.run.smithi047.stdout:pg_num: 32 2024-03-21T10:22:21.246 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:21.245+0000 7fea90800640 1 -- 172.21.15.47:0/1933373454 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7fea700a2fa0 msgr2=0x7fea700a5460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:22:21.246 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:21.245+0000 7fea90800640 1 --2- 172.21.15.47:0/1933373454 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7fea700a2fa0 0x7fea700a5460 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fea7c02c840 tx=0x7fea7c0095a0 comp rx=0 tx=0).stop 2024-03-21T10:22:21.246 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:21.245+0000 7fea90800640 1 -- 172.21.15.47:0/1933373454 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fea8810d2a0 msgr2=0x7fea881a28b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:22:21.246 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:21.245+0000 7fea90800640 1 --2- 172.21.15.47:0/1933373454 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fea8810d2a0 0x7fea881a28b0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fea7400d720 tx=0x7fea7400dbe0 comp rx=0 tx=0).stop 2024-03-21T10:22:21.246 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:21.245+0000 7fea90800640 1 -- 172.21.15.47:0/1933373454 shutdown_connections 2024-03-21T10:22:21.246 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:21.245+0000 7fea90800640 1 --2- 172.21.15.47:0/1933373454 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7fea700a2fa0 0x7fea700a5460 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:22:21.247 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:21.245+0000 7fea90800640 1 --2- 172.21.15.47:0/1933373454 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fea8810dc60 0x7fea881a9920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:22:21.247 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:21.245+0000 7fea90800640 1 --2- 172.21.15.47:0/1933373454 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fea8810d2a0 0x7fea881a28b0 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:22:21.247 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:21.245+0000 7fea90800640 1 --2- 172.21.15.47:0/1933373454 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fea8810c0b0 0x7fea881a2370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:22:21.247 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:21.245+0000 7fea90800640 1 -- 172.21.15.47:0/1933373454 >> 172.21.15.47:0/1933373454 conn(0x7fea880729f0 msgr2=0x7fea88110710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:22:21.247 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:21.245+0000 7fea90800640 1 -- 172.21.15.47:0/1933373454 shutdown_connections 2024-03-21T10:22:21.247 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:21.245+0000 7fea90800640 1 -- 172.21.15.47:0/1933373454 wait complete. 2024-03-21T10:22:21.266 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-03-21T10:22:21.464 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:22:21 smithi047 ceph-mon[31547]: pgmap v378: 129 pgs: 129 active+clean; 581 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 0 op/s 2024-03-21T10:22:21.464 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:22:21 smithi047 ceph-mon[31547]: from='client.? 172.21.15.47:0/3096041762' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-21T10:22:21.464 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:22:21 smithi047 ceph-mon[31547]: from='client.? 172.21.15.47:0/3969267362' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-21T10:22:21.480 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:22:21 smithi100 ceph-mon[35193]: pgmap v378: 129 pgs: 129 active+clean; 581 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 0 op/s 2024-03-21T10:22:21.480 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:22:21 smithi100 ceph-mon[35193]: from='client.? 172.21.15.47:0/3096041762' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-21T10:22:21.480 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:22:21 smithi100 ceph-mon[35193]: from='client.? 172.21.15.47:0/3969267362' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-21T10:22:21.489 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:22:21 smithi157 ceph-mon[35581]: pgmap v378: 129 pgs: 129 active+clean; 581 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 0 op/s 2024-03-21T10:22:21.489 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:22:21 smithi157 ceph-mon[35581]: from='client.? 172.21.15.47:0/3096041762' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-21T10:22:21.490 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:22:21 smithi157 ceph-mon[35581]: from='client.? 172.21.15.47:0/3969267362' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-21T10:22:21.490 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:21.489+0000 7efd83400640 1 -- 172.21.15.47:0/2935248011 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7efd7c104a60 msgr2=0x7efd7c106e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:22:21.491 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:21.489+0000 7efd83400640 1 --2- 172.21.15.47:0/2935248011 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7efd7c104a60 0x7efd7c106e50 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7efd68009900 tx=0x7efd6802bc30 comp rx=0 tx=0).stop 2024-03-21T10:22:21.491 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:21.489+0000 7efd83400640 1 -- 172.21.15.47:0/2935248011 shutdown_connections 2024-03-21T10:22:21.491 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:21.489+0000 7efd83400640 1 --2- 172.21.15.47:0/2935248011 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7efd7c107390 0x7efd7c1097e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:22:21.491 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:21.489+0000 7efd83400640 1 --2- 172.21.15.47:0/2935248011 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7efd7c104a60 0x7efd7c106e50 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:22:21.491 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:21.489+0000 7efd83400640 1 --2- 172.21.15.47:0/2935248011 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7efd7c102130 0x7efd7c104520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:22:21.491 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:21.489+0000 7efd83400640 1 -- 172.21.15.47:0/2935248011 >> 172.21.15.47:0/2935248011 conn(0x7efd7c0fbb40 msgr2=0x7efd7c0fdf60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:22:21.491 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:21.489+0000 7efd83400640 1 -- 172.21.15.47:0/2935248011 shutdown_connections 2024-03-21T10:22:21.491 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:21.489+0000 7efd83400640 1 -- 172.21.15.47:0/2935248011 wait complete. 2024-03-21T10:22:21.493 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:21.490+0000 7efd83400640 1 Processor -- start 2024-03-21T10:22:21.493 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:21.490+0000 7efd83400640 1 -- start start 2024-03-21T10:22:21.493 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:21.491+0000 7efd83400640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7efd7c102130 0x7efd7c197920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:22:21.493 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:21.491+0000 7efd83400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7efd7c104a60 0x7efd7c197e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:22:21.493 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:21.491+0000 7efd83400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7efd7c107390 0x7efd7c19eed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:22:21.493 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:21.491+0000 7efd83400640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd7c10d220 con 0x7efd7c107390 2024-03-21T10:22:21.493 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:21.491+0000 7efd83400640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd7c10d0a0 con 0x7efd7c104a60 2024-03-21T10:22:21.493 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:21.491+0000 7efd83400640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd7c10d3a0 con 0x7efd7c102130 2024-03-21T10:22:21.493 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:21.491+0000 7efd81400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7efd7c107390 0x7efd7c19eed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:22:21.493 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:21.491+0000 7efd81400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7efd7c107390 0x7efd7c19eed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:51796/0 (socket says 172.21.15.47:51796) 2024-03-21T10:22:21.493 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:21.491+0000 7efd81400640 1 -- 172.21.15.47:0/1112250971 learned_addr learned my addr 172.21.15.47:0/1112250971 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:22:21.494 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:21.491+0000 7efd7be00640 1 --2- 172.21.15.47:0/1112250971 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7efd7c104a60 0x7efd7c197e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:22:21.494 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:21.492+0000 7efd80a00640 1 --2- 172.21.15.47:0/1112250971 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7efd7c102130 0x7efd7c197920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:22:21.494 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:21.492+0000 7efd81400640 1 -- 172.21.15.47:0/1112250971 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7efd7c102130 msgr2=0x7efd7c197920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:22:21.494 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:21.492+0000 7efd81400640 1 --2- 172.21.15.47:0/1112250971 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7efd7c102130 0x7efd7c197920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:22:21.494 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:21.492+0000 7efd81400640 1 -- 172.21.15.47:0/1112250971 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7efd7c104a60 msgr2=0x7efd7c197e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:22:21.494 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:21.492+0000 7efd81400640 1 --2- 172.21.15.47:0/1112250971 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7efd7c104a60 0x7efd7c197e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:22:21.494 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:21.492+0000 7efd81400640 1 -- 172.21.15.47:0/1112250971 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd7c19f540 con 0x7efd7c107390 2024-03-21T10:22:21.494 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:21.492+0000 7efd80a00640 1 --2- 172.21.15.47:0/1112250971 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7efd7c102130 0x7efd7c197920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T10:22:21.494 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:21.492+0000 7efd81400640 1 --2- 172.21.15.47:0/1112250971 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7efd7c107390 0x7efd7c19eed0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7efd7400cad0 tx=0x7efd7400cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:22:21.495 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:21.492+0000 7efd79600640 1 -- 172.21.15.47:0/1112250971 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd7400ae20 con 0x7efd7c107390 2024-03-21T10:22:21.495 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:21.492+0000 7efd79600640 1 -- 172.21.15.47:0/1112250971 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7efd74005020 con 0x7efd7c107390 2024-03-21T10:22:21.495 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:21.492+0000 7efd83400640 1 -- 172.21.15.47:0/1112250971 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efd7c19f830 con 0x7efd7c107390 2024-03-21T10:22:21.495 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:21.492+0000 7efd79600640 1 -- 172.21.15.47:0/1112250971 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd7400f6b0 con 0x7efd7c107390 2024-03-21T10:22:21.495 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:21.492+0000 7efd83400640 1 -- 172.21.15.47:0/1112250971 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efd7c19fd00 con 0x7efd7c107390 2024-03-21T10:22:21.495 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:21.494+0000 7efd83400640 1 -- 172.21.15.47:0/1112250971 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efd7c0fd630 con 0x7efd7c107390 2024-03-21T10:22:21.496 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:21.495+0000 7efd79600640 1 -- 172.21.15.47:0/1112250971 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 16) v1 ==== 137854+0+0 (secure 0 0 0) 0x7efd740051d0 con 0x7efd7c107390 2024-03-21T10:22:21.498 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:21.497+0000 7efd79600640 1 --2- 172.21.15.47:0/1112250971 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7efd600a2f20 0x7efd600a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:22:21.498 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:21.497+0000 7efd79600640 1 -- 172.21.15.47:0/1112250971 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 8802+0+0 (secure 0 0 0) 0x7efd740cf760 con 0x7efd7c107390 2024-03-21T10:22:21.498 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:21.497+0000 7efd80a00640 1 --2- 172.21.15.47:0/1112250971 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7efd600a2f20 0x7efd600a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:22:21.498 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:21.498+0000 7efd80a00640 1 --2- 172.21.15.47:0/1112250971 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7efd600a2f20 0x7efd600a53e0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7efd6c009b40 tx=0x7efd6c005e50 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:22:21.502 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:21.501+0000 7efd79600640 1 -- 172.21.15.47:0/1112250971 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7efd74014030 con 0x7efd7c107390 2024-03-21T10:22:21.719 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:21.718+0000 7efd83400640 1 -- 172.21.15.47:0/1112250971 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7efd7c0622a0 con 0x7efd7c107390 2024-03-21T10:22:21.720 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:21.719+0000 7efd79600640 1 -- 172.21.15.47:0/1112250971 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 10 entries v91) v1 ==== 81+0+590 (secure 0 0 0) 0x7efd7409c350 con 0x7efd7c107390 2024-03-21T10:22:21.720 INFO:teuthology.orchestra.run.smithi047.stdout:172.21.15.47:0/356822431 2024-03-22T10:12:25.844530+0000 2024-03-21T10:22:21.721 INFO:teuthology.orchestra.run.smithi047.stdout:172.21.15.47:0/1167490879 2024-03-22T10:12:25.844530+0000 2024-03-21T10:22:21.721 INFO:teuthology.orchestra.run.smithi047.stdout:172.21.15.47:0/1540718138 2024-03-22T10:12:25.844530+0000 2024-03-21T10:22:21.721 INFO:teuthology.orchestra.run.smithi047.stdout:172.21.15.47:6801/3836903972 2024-03-22T10:12:25.844530+0000 2024-03-21T10:22:21.721 INFO:teuthology.orchestra.run.smithi047.stdout:172.21.15.47:0/235436602 2024-03-22T10:11:50.301838+0000 2024-03-21T10:22:21.721 INFO:teuthology.orchestra.run.smithi047.stdout:172.21.15.47:0/2052815572 2024-03-22T10:11:50.301838+0000 2024-03-21T10:22:21.721 INFO:teuthology.orchestra.run.smithi047.stdout:172.21.15.47:6800/3942658432 2024-03-22T10:11:50.301838+0000 2024-03-21T10:22:21.721 INFO:teuthology.orchestra.run.smithi047.stdout:172.21.15.47:6801/3942658432 2024-03-22T10:11:50.301838+0000 2024-03-21T10:22:21.721 INFO:teuthology.orchestra.run.smithi047.stdout:172.21.15.47:0/1419516257 2024-03-22T10:11:50.301838+0000 2024-03-21T10:22:21.721 INFO:teuthology.orchestra.run.smithi047.stdout:172.21.15.47:6800/3836903972 2024-03-22T10:12:25.844530+0000 2024-03-21T10:22:21.721 INFO:teuthology.orchestra.run.smithi047.stderr:listed 10 entries 2024-03-21T10:22:21.723 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:21.722+0000 7efd83400640 1 -- 172.21.15.47:0/1112250971 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7efd600a2f20 msgr2=0x7efd600a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:22:21.723 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:21.722+0000 7efd83400640 1 --2- 172.21.15.47:0/1112250971 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7efd600a2f20 0x7efd600a53e0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7efd6c009b40 tx=0x7efd6c005e50 comp rx=0 tx=0).stop 2024-03-21T10:22:21.723 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:21.722+0000 7efd83400640 1 -- 172.21.15.47:0/1112250971 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7efd7c107390 msgr2=0x7efd7c19eed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:22:21.723 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:21.722+0000 7efd83400640 1 --2- 172.21.15.47:0/1112250971 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7efd7c107390 0x7efd7c19eed0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7efd7400cad0 tx=0x7efd7400cf90 comp rx=0 tx=0).stop 2024-03-21T10:22:21.724 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:21.722+0000 7efd83400640 1 -- 172.21.15.47:0/1112250971 shutdown_connections 2024-03-21T10:22:21.724 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:21.722+0000 7efd83400640 1 --2- 172.21.15.47:0/1112250971 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7efd600a2f20 0x7efd600a53e0 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:22:21.724 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:21.722+0000 7efd83400640 1 --2- 172.21.15.47:0/1112250971 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7efd7c107390 0x7efd7c19eed0 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:22:21.724 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:21.722+0000 7efd83400640 1 --2- 172.21.15.47:0/1112250971 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7efd7c104a60 0x7efd7c197e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:22:21.724 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:21.722+0000 7efd83400640 1 --2- 172.21.15.47:0/1112250971 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7efd7c102130 0x7efd7c197920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:22:21.724 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:21.722+0000 7efd83400640 1 -- 172.21.15.47:0/1112250971 >> 172.21.15.47:0/1112250971 conn(0x7efd7c0fbb40 msgr2=0x7efd7c102b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:22:21.724 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:21.722+0000 7efd83400640 1 -- 172.21.15.47:0/1112250971 shutdown_connections 2024-03-21T10:22:21.724 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:21.722+0000 7efd83400640 1 -- 172.21.15.47:0/1112250971 wait complete. 2024-03-21T10:22:21.742 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi047.front.sepia.ceph.com... 2024-03-21T10:22:21.742 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-21T10:22:21.743 DEBUG:teuthology.orchestra.run.smithi047:> sudo fusermount -u /home/ubuntu/cephtest/mnt.admin 2024-03-21T10:22:21.839 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi047.stderr:ceph-fuse[73396]: fuse finished with error 0 and tester_r 0 2024-03-21T10:22:21.848 INFO:teuthology.orchestra.run:waiting for 300 2024-03-21T10:22:21.910 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi047.front.sepia.ceph.com 2024-03-21T10:22:21.911 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-21T10:22:21.911 DEBUG:teuthology.orchestra.run.smithi047:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.admin) 2024-03-21T10:22:21.979 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-21T10:22:22.212 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:22.211+0000 7fedc1800640 1 -- 172.21.15.47:0/1709754444 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fedbc108ed0 msgr2=0x7fedbc109350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:22:22.212 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:22.211+0000 7fedc1800640 1 --2- 172.21.15.47:0/1709754444 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fedbc108ed0 0x7fedbc109350 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7feda8009900 tx=0x7feda802bc30 comp rx=0 tx=0).stop 2024-03-21T10:22:22.212 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:22.211+0000 7fedc1800640 1 -- 172.21.15.47:0/1709754444 shutdown_connections 2024-03-21T10:22:22.212 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:22.211+0000 7fedc1800640 1 --2- 172.21.15.47:0/1709754444 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fedbc109890 0x7fedbc110120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:22:22.213 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:22.211+0000 7fedc1800640 1 --2- 172.21.15.47:0/1709754444 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fedbc108ed0 0x7fedbc109350 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:22:22.213 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:22.211+0000 7fedc1800640 1 --2- 172.21.15.47:0/1709754444 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fedbc107ce0 0x7fedbc108100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:22:22.213 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:22.211+0000 7fedc1800640 1 -- 172.21.15.47:0/1709754444 >> 172.21.15.47:0/1709754444 conn(0x7fedbc076cb0 msgr2=0x7fedbc0790d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:22:22.213 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:22.211+0000 7fedc1800640 1 -- 172.21.15.47:0/1709754444 shutdown_connections 2024-03-21T10:22:22.213 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:22.211+0000 7fedc1800640 1 -- 172.21.15.47:0/1709754444 wait complete. 2024-03-21T10:22:22.215 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:22.212+0000 7fedc1800640 1 Processor -- start 2024-03-21T10:22:22.215 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:22.212+0000 7fedc1800640 1 -- start start 2024-03-21T10:22:22.215 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:22.212+0000 7fedc1800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fedbc107ce0 0x7fedbc19df30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:22:22.215 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:22.212+0000 7fedc1800640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fedbc108ed0 0x7fedbc19e470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:22:22.216 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:22.212+0000 7fedc1800640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fedbc109890 0x7fedbc1a54e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:22:22.216 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:22.212+0000 7fedc1800640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fedbc113860 con 0x7fedbc109890 2024-03-21T10:22:22.216 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:22.212+0000 7fedc1800640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fedbc1136e0 con 0x7fedbc107ce0 2024-03-21T10:22:22.216 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:22.212+0000 7fedc1800640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fedbc1139e0 con 0x7fedbc108ed0 2024-03-21T10:22:22.216 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:22.213+0000 7fedbb400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fedbc109890 0x7fedbc1a54e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:22:22.216 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:22.213+0000 7fedbb400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fedbc109890 0x7fedbc1a54e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:56434/0 (socket says 172.21.15.47:56434) 2024-03-21T10:22:22.216 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:22.213+0000 7fedbb400640 1 -- 172.21.15.47:0/523497770 learned_addr learned my addr 172.21.15.47:0/523497770 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:22:22.216 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:22.213+0000 7fedbaa00640 1 --2- 172.21.15.47:0/523497770 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fedbc107ce0 0x7fedbc19df30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:22:22.216 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:22.213+0000 7fedba000640 1 --2- 172.21.15.47:0/523497770 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fedbc108ed0 0x7fedbc19e470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:22:22.216 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:22.213+0000 7fedbb400640 1 -- 172.21.15.47:0/523497770 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fedbc108ed0 msgr2=0x7fedbc19e470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:22:22.216 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:22.213+0000 7fedbb400640 1 --2- 172.21.15.47:0/523497770 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fedbc108ed0 0x7fedbc19e470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:22:22.216 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:22.213+0000 7fedbb400640 1 -- 172.21.15.47:0/523497770 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fedbc107ce0 msgr2=0x7fedbc19df30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:22:22.216 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:22.213+0000 7fedbb400640 1 --2- 172.21.15.47:0/523497770 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fedbc107ce0 0x7fedbc19df30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:22:22.217 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:22.213+0000 7fedbb400640 1 -- 172.21.15.47:0/523497770 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fedbc1a5b50 con 0x7fedbc109890 2024-03-21T10:22:22.217 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:22.213+0000 7fedbb400640 1 --2- 172.21.15.47:0/523497770 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fedbc109890 0x7fedbc1a54e0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fedb4009fd0 tx=0x7fedb400b260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:22:22.217 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:22.214+0000 7fedb3400640 1 -- 172.21.15.47:0/523497770 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fedb4017070 con 0x7fedbc109890 2024-03-21T10:22:22.217 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:22.214+0000 7fedb3400640 1 -- 172.21.15.47:0/523497770 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fedb4003e20 con 0x7fedbc109890 2024-03-21T10:22:22.217 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:22.214+0000 7fedb3400640 1 -- 172.21.15.47:0/523497770 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fedb4010460 con 0x7fedbc109890 2024-03-21T10:22:22.217 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:22.214+0000 7fedc1800640 1 -- 172.21.15.47:0/523497770 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fedbc1a5e40 con 0x7fedbc109890 2024-03-21T10:22:22.217 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:22.214+0000 7fedc1800640 1 -- 172.21.15.47:0/523497770 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fedbc1a62b0 con 0x7fedbc109890 2024-03-21T10:22:22.218 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:22.215+0000 7fedc1800640 1 -- 172.21.15.47:0/523497770 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fedbc10f0a0 con 0x7fedbc109890 2024-03-21T10:22:22.218 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:22.217+0000 7fedb3400640 1 -- 172.21.15.47:0/523497770 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 16) v1 ==== 137854+0+0 (secure 0 0 0) 0x7fedb4003970 con 0x7fedbc109890 2024-03-21T10:22:22.219 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:22.218+0000 7fedb3400640 1 --2- 172.21.15.47:0/523497770 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7feda00a2fa0 0x7feda00a5460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:22:22.219 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:22.218+0000 7fedb3400640 1 -- 172.21.15.47:0/523497770 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 8802+0+0 (secure 0 0 0) 0x7fedb40cf520 con 0x7fedbc109890 2024-03-21T10:22:22.219 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:22.219+0000 7fedbaa00640 1 --2- 172.21.15.47:0/523497770 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7feda00a2fa0 0x7feda00a5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:22:22.220 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:22.219+0000 7fedbaa00640 1 --2- 172.21.15.47:0/523497770 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7feda00a2fa0 0x7feda00a5460 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fedac005fd0 tx=0x7fedac005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:22:22.224 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:22.223+0000 7fedb3400640 1 -- 172.21.15.47:0/523497770 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fedb409c0b0 con 0x7fedbc109890 2024-03-21T10:22:22.432 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:22.431+0000 7fedc1800640 1 -- 172.21.15.47:0/523497770 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fedbc062340 con 0x7fedbc109890 2024-03-21T10:22:22.435 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:22.434+0000 7fedb3400640 1 -- 172.21.15.47:0/523497770 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v91) v1 ==== 74+0+24116 (secure 0 0 0) 0x7fedb409b9b0 con 0x7fedbc109890 2024-03-21T10:22:22.436 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-21T10:22:22.436 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":91,"fsid":"3056d51a-e76b-11ee-95cd-87774f69a715","created":"2024-03-21T10:11:17.019965+0000","modified":"2024-03-21T10:22:03.759214+0000","last_up_change":"2024-03-21T10:20:52.675328+0000","last_in_change":"2024-03-21T10:20:27.648212+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-21T10:15:40.553114+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-21T10:21:56.399518+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-21T10:21:57.812940+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-21T10:22:01.822568+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"91","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"b0c7a102-8d89-428b-b793-cbbe67151f50","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6802","nonce":775837313},{"type":"v1","addr":"172.21.15.47:6803","nonce":775837313}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6804","nonce":775837313},{"type":"v1","addr":"172.21.15.47:6805","nonce":775837313}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6808","nonce":775837313},{"type":"v1","addr":"172.21.15.47:6809","nonce":775837313}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6806","nonce":775837313},{"type":"v1","addr":"172.21.15.47:6807","nonce":775837313}]},"public_addr":"172.21.15.47:6803/775837313","cluster_addr":"172.21.15.47:6805/775837313","heartbeat_back_addr":"172.21.15.47:6809/775837313","heartbeat_front_addr":"172.21.15.47:6807/775837313","state":["exists","up"]},{"osd":1,"uuid":"07bb81bb-7cf3-4bbc-ac9e-802f95185d11","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6800","nonce":2923304710},{"type":"v1","addr":"172.21.15.100:6801","nonce":2923304710}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":2923304710},{"type":"v1","addr":"172.21.15.100:6803","nonce":2923304710}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":2923304710},{"type":"v1","addr":"172.21.15.100:6807","nonce":2923304710}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6804","nonce":2923304710},{"type":"v1","addr":"172.21.15.100:6805","nonce":2923304710}]},"public_addr":"172.21.15.100:6801/2923304710","cluster_addr":"172.21.15.100:6803/2923304710","heartbeat_back_addr":"172.21.15.100:6807/2923304710","heartbeat_front_addr":"172.21.15.100:6805/2923304710","state":["exists","up"]},{"osd":2,"uuid":"88ce76b9-6081-4836-aad0-8c249c56e8a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":171377005},{"type":"v1","addr":"172.21.15.157:6801","nonce":171377005}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":171377005},{"type":"v1","addr":"172.21.15.157:6803","nonce":171377005}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":171377005},{"type":"v1","addr":"172.21.15.157:6807","nonce":171377005}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":171377005},{"type":"v1","addr":"172.21.15.157:6805","nonce":171377005}]},"public_addr":"172.21.15.157:6801/171377005","cluster_addr":"172.21.15.157:6803/171377005","heartbeat_back_addr":"172.21.15.157:6807/171377005","heartbeat_front_addr":"172.21.15.157:6805/171377005","state":["exists","up"]},{"osd":3,"uuid":"518aa33d-263f-45e6-aab5-2201ceb40e74","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":27,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6810","nonce":640760101},{"type":"v1","addr":"172.21.15.47:6811","nonce":640760101}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6812","nonce":640760101},{"type":"v1","addr":"172.21.15.47:6813","nonce":640760101}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6816","nonce":640760101},{"type":"v1","addr":"172.21.15.47:6817","nonce":640760101}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6814","nonce":640760101},{"type":"v1","addr":"172.21.15.47:6815","nonce":640760101}]},"public_addr":"172.21.15.47:6811/640760101","cluster_addr":"172.21.15.47:6813/640760101","heartbeat_back_addr":"172.21.15.47:6817/640760101","heartbeat_front_addr":"172.21.15.47:6815/640760101","state":["exists","up"]},{"osd":4,"uuid":"987bde83-ecc6-42e5-bd56-c143b3de4226","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":35,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6808","nonce":2785478639},{"type":"v1","addr":"172.21.15.100:6809","nonce":2785478639}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6810","nonce":2785478639},{"type":"v1","addr":"172.21.15.100:6811","nonce":2785478639}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6814","nonce":2785478639},{"type":"v1","addr":"172.21.15.100:6815","nonce":2785478639}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":2785478639},{"type":"v1","addr":"172.21.15.100:6813","nonce":2785478639}]},"public_addr":"172.21.15.100:6809/2785478639","cluster_addr":"172.21.15.100:6811/2785478639","heartbeat_back_addr":"172.21.15.100:6815/2785478639","heartbeat_front_addr":"172.21.15.100:6813/2785478639","state":["exists","up"]},{"osd":5,"uuid":"012d2188-3a61-4a5f-8d01-2001c15ffbe2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":40,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":1525645410},{"type":"v1","addr":"172.21.15.157:6809","nonce":1525645410}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":1525645410},{"type":"v1","addr":"172.21.15.157:6811","nonce":1525645410}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":1525645410},{"type":"v1","addr":"172.21.15.157:6815","nonce":1525645410}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":1525645410},{"type":"v1","addr":"172.21.15.157:6813","nonce":1525645410}]},"public_addr":"172.21.15.157:6809/1525645410","cluster_addr":"172.21.15.157:6811/1525645410","heartbeat_back_addr":"172.21.15.157:6815/1525645410","heartbeat_front_addr":"172.21.15.157:6813/1525645410","state":["exists","up"]},{"osd":6,"uuid":"c12c14b4-85f0-4d88-937c-de4f556437de","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6818","nonce":2321642790},{"type":"v1","addr":"172.21.15.47:6819","nonce":2321642790}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6820","nonce":2321642790},{"type":"v1","addr":"172.21.15.47:6821","nonce":2321642790}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6824","nonce":2321642790},{"type":"v1","addr":"172.21.15.47:6825","nonce":2321642790}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6822","nonce":2321642790},{"type":"v1","addr":"172.21.15.47:6823","nonce":2321642790}]},"public_addr":"172.21.15.47:6819/2321642790","cluster_addr":"172.21.15.47:6821/2321642790","heartbeat_back_addr":"172.21.15.47:6825/2321642790","heartbeat_front_addr":"172.21.15.47:6823/2321642790","state":["exists","up"]},{"osd":7,"uuid":"3c87c6d0-1d1f-4294-9917-aaaad7dda824","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6816","nonce":4104727330},{"type":"v1","addr":"172.21.15.100:6817","nonce":4104727330}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":4104727330},{"type":"v1","addr":"172.21.15.100:6819","nonce":4104727330}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":4104727330},{"type":"v1","addr":"172.21.15.100:6823","nonce":4104727330}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6820","nonce":4104727330},{"type":"v1","addr":"172.21.15.100:6821","nonce":4104727330}]},"public_addr":"172.21.15.100:6817/4104727330","cluster_addr":"172.21.15.100:6819/4104727330","heartbeat_back_addr":"172.21.15.100:6823/4104727330","heartbeat_front_addr":"172.21.15.100:6821/4104727330","state":["exists","up"]},{"osd":8,"uuid":"502d65cf-53c8-4936-8de4-e11b2a6e1e18","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":61,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":4067715347},{"type":"v1","addr":"172.21.15.157:6817","nonce":4067715347}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":4067715347},{"type":"v1","addr":"172.21.15.157:6819","nonce":4067715347}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":4067715347},{"type":"v1","addr":"172.21.15.157:6823","nonce":4067715347}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":4067715347},{"type":"v1","addr":"172.21.15.157:6821","nonce":4067715347}]},"public_addr":"172.21.15.157:6817/4067715347","cluster_addr":"172.21.15.157:6819/4067715347","heartbeat_back_addr":"172.21.15.157:6823/4067715347","heartbeat_front_addr":"172.21.15.157:6821/4067715347","state":["exists","up"]},{"osd":9,"uuid":"5ea1f5be-8458-4c93-9e61-c2e867a514f3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6826","nonce":1982108738},{"type":"v1","addr":"172.21.15.47:6827","nonce":1982108738}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6828","nonce":1982108738},{"type":"v1","addr":"172.21.15.47:6829","nonce":1982108738}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6832","nonce":1982108738},{"type":"v1","addr":"172.21.15.47:6833","nonce":1982108738}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6830","nonce":1982108738},{"type":"v1","addr":"172.21.15.47:6831","nonce":1982108738}]},"public_addr":"172.21.15.47:6827/1982108738","cluster_addr":"172.21.15.47:6829/1982108738","heartbeat_back_addr":"172.21.15.47:6833/1982108738","heartbeat_front_addr":"172.21.15.47:6831/1982108738","state":["exists","up"]},{"osd":10,"uuid":"2da3da90-a0be-4a54-a9ff-b2d6c9b36870","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":74,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":396391028},{"type":"v1","addr":"172.21.15.100:6825","nonce":396391028}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":396391028},{"type":"v1","addr":"172.21.15.100:6827","nonce":396391028}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6830","nonce":396391028},{"type":"v1","addr":"172.21.15.100:6831","nonce":396391028}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":396391028},{"type":"v1","addr":"172.21.15.100:6829","nonce":396391028}]},"public_addr":"172.21.15.100:6825/396391028","cluster_addr":"172.21.15.100:6827/396391028","heartbeat_back_addr":"172.21.15.100:6831/396391028","heartbeat_front_addr":"172.21.15.100:6829/396391028","state":["exists","up"]},{"osd":11,"uuid":"11798d41-7f7e-4319-95b8-8d3d0946bde9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":82,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":2671029483},{"type":"v1","addr":"172.21.15.157:6825","nonce":2671029483}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":2671029483},{"type":"v1","addr":"172.21.15.157:6827","nonce":2671029483}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":2671029483},{"type":"v1","addr":"172.21.15.157:6831","nonce":2671029483}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":2671029483},{"type":"v1","addr":"172.21.15.157:6829","nonce":2671029483}]},"public_addr":"172.21.15.157:6825/2671029483","cluster_addr":"172.21.15.157:6827/2671029483","heartbeat_back_addr":"172.21.15.157:6831/2671029483","heartbeat_front_addr":"172.21.15.157:6829/2671029483","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T10:14:31.822399+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T10:15:04.175721+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T10:15:35.902595+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T10:16:12.088891+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T10:16:46.428704+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T10:17:20.158270+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T10:17:55.915253+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T10:18:30.064641+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T10:19:04.410004+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T10:19:40.605181+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T10:20:15.636605+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T10:20:50.151365+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.47:0/356822431":"2024-03-22T10:12:25.844530+0000","172.21.15.47:0/1167490879":"2024-03-22T10:12:25.844530+0000","172.21.15.47:0/1540718138":"2024-03-22T10:12:25.844530+0000","172.21.15.47:6801/3836903972":"2024-03-22T10:12:25.844530+0000","172.21.15.47:0/235436602":"2024-03-22T10:11:50.301838+0000","172.21.15.47:0/2052815572":"2024-03-22T10:11:50.301838+0000","172.21.15.47:6800/3942658432":"2024-03-22T10:11:50.301838+0000","172.21.15.47:6801/3942658432":"2024-03-22T10:11:50.301838+0000","172.21.15.47:0/1419516257":"2024-03-22T10:11:50.301838+0000","172.21.15.47:6800/3836903972":"2024-03-22T10:12:25.844530+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-21T10:22:22.438 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:22.437+0000 7fedc1800640 1 -- 172.21.15.47:0/523497770 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7feda00a2fa0 msgr2=0x7feda00a5460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:22:22.438 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:22.437+0000 7fedc1800640 1 --2- 172.21.15.47:0/523497770 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7feda00a2fa0 0x7feda00a5460 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fedac005fd0 tx=0x7fedac005850 comp rx=0 tx=0).stop 2024-03-21T10:22:22.438 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:22.437+0000 7fedc1800640 1 -- 172.21.15.47:0/523497770 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fedbc109890 msgr2=0x7fedbc1a54e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:22:22.439 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:22.437+0000 7fedc1800640 1 --2- 172.21.15.47:0/523497770 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fedbc109890 0x7fedbc1a54e0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fedb4009fd0 tx=0x7fedb400b260 comp rx=0 tx=0).stop 2024-03-21T10:22:22.439 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:22.438+0000 7fedc1800640 1 -- 172.21.15.47:0/523497770 shutdown_connections 2024-03-21T10:22:22.440 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:22.438+0000 7fedc1800640 1 --2- 172.21.15.47:0/523497770 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7feda00a2fa0 0x7feda00a5460 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:22:22.440 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:22.438+0000 7fedc1800640 1 --2- 172.21.15.47:0/523497770 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fedbc109890 0x7fedbc1a54e0 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:22:22.440 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:22.438+0000 7fedc1800640 1 --2- 172.21.15.47:0/523497770 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fedbc108ed0 0x7fedbc19e470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:22:22.440 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:22.438+0000 7fedc1800640 1 --2- 172.21.15.47:0/523497770 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fedbc107ce0 0x7fedbc19df30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:22:22.440 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:22.438+0000 7fedc1800640 1 -- 172.21.15.47:0/523497770 >> 172.21.15.47:0/523497770 conn(0x7fedbc076cb0 msgr2=0x7fedbc10c270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:22:22.440 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:22.438+0000 7fedc1800640 1 -- 172.21.15.47:0/523497770 shutdown_connections 2024-03-21T10:22:22.440 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:22.438+0000 7fedc1800640 1 -- 172.21.15.47:0/523497770 wait complete. 2024-03-21T10:22:22.455 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:22:22 smithi047 ceph-mon[31547]: from='client.? 172.21.15.47:0/1933373454' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-03-21T10:22:22.455 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:22:22 smithi047 ceph-mon[31547]: from='client.? 172.21.15.47:0/1112250971' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-03-21T10:22:22.460 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-21T10:22:22.480 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:22:22 smithi100 ceph-mon[35193]: from='client.? 172.21.15.47:0/1933373454' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-03-21T10:22:22.480 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:22:22 smithi100 ceph-mon[35193]: from='client.? 172.21.15.47:0/1112250971' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-03-21T10:22:22.490 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:22:22 smithi157 ceph-mon[35581]: from='client.? 172.21.15.47:0/1933373454' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-03-21T10:22:22.490 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:22:22 smithi157 ceph-mon[35581]: from='client.? 172.21.15.47:0/1112250971' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-03-21T10:22:22.688 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:22.687+0000 7f4464a00640 1 -- 172.21.15.47:0/3441544035 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f44601048e0 msgr2=0x7f4460104d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:22:22.689 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:22.687+0000 7f4464a00640 1 --2- 172.21.15.47:0/3441544035 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f44601048e0 0x7f4460104d60 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f4448009900 tx=0x7f444802bc30 comp rx=0 tx=0).stop 2024-03-21T10:22:22.689 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:22.687+0000 7f4464a00640 1 -- 172.21.15.47:0/3441544035 shutdown_connections 2024-03-21T10:22:22.689 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:22.687+0000 7f4464a00640 1 --2- 172.21.15.47:0/3441544035 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f44601052a0 0x7f446010bb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:22:22.689 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:22.687+0000 7f4464a00640 1 --2- 172.21.15.47:0/3441544035 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f44601048e0 0x7f4460104d60 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:22:22.689 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:22.687+0000 7f4464a00640 1 --2- 172.21.15.47:0/3441544035 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f44601036f0 0x7f4460103b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:22:22.689 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:22.687+0000 7f4464a00640 1 -- 172.21.15.47:0/3441544035 >> 172.21.15.47:0/3441544035 conn(0x7f44600fec80 msgr2=0x7f44601010a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:22:22.689 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:22.687+0000 7f4464a00640 1 -- 172.21.15.47:0/3441544035 shutdown_connections 2024-03-21T10:22:22.689 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:22.687+0000 7f4464a00640 1 -- 172.21.15.47:0/3441544035 wait complete. 2024-03-21T10:22:22.692 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:22.688+0000 7f4464a00640 1 Processor -- start 2024-03-21T10:22:22.692 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:22.688+0000 7f4464a00640 1 -- start start 2024-03-21T10:22:22.692 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:22.689+0000 7f4464a00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f44601036f0 0x7f44601998c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:22:22.692 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:22.689+0000 7f4464a00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f44601048e0 0x7f4460199e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:22:22.692 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:22.689+0000 7f4464a00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f44601052a0 0x7f44601a0e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:22:22.692 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:22.689+0000 7f4464a00640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f446010f270 con 0x7f44601052a0 2024-03-21T10:22:22.692 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:22.689+0000 7f4464a00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f446010f0f0 con 0x7f44601036f0 2024-03-21T10:22:22.692 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:22.689+0000 7f4464a00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f446010f3f0 con 0x7f44601048e0 2024-03-21T10:22:22.692 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:22.689+0000 7f445f200640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f44601052a0 0x7f44601a0e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:22:22.692 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:22.689+0000 7f445f200640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f44601052a0 0x7f44601a0e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:56450/0 (socket says 172.21.15.47:56450) 2024-03-21T10:22:22.692 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:22.689+0000 7f445f200640 1 -- 172.21.15.47:0/172850696 learned_addr learned my addr 172.21.15.47:0/172850696 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:22:22.692 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:22.689+0000 7f445e800640 1 --2- 172.21.15.47:0/172850696 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f44601036f0 0x7f44601998c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:22:22.693 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:22.689+0000 7f4457e00640 1 --2- 172.21.15.47:0/172850696 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f44601048e0 0x7f4460199e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:22:22.693 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:22.690+0000 7f445f200640 1 -- 172.21.15.47:0/172850696 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f44601048e0 msgr2=0x7f4460199e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:22:22.693 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:22.690+0000 7f445f200640 1 --2- 172.21.15.47:0/172850696 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f44601048e0 0x7f4460199e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:22:22.693 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:22.690+0000 7f445f200640 1 -- 172.21.15.47:0/172850696 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f44601036f0 msgr2=0x7f44601998c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:22:22.693 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:22.690+0000 7f445f200640 1 --2- 172.21.15.47:0/172850696 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f44601036f0 0x7f44601998c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:22:22.693 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:22.690+0000 7f445f200640 1 -- 172.21.15.47:0/172850696 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f44601a14e0 con 0x7f44601052a0 2024-03-21T10:22:22.693 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:22.690+0000 7f445e800640 1 --2- 172.21.15.47:0/172850696 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f44601036f0 0x7f44601998c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T10:22:22.693 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:22.690+0000 7f445f200640 1 --2- 172.21.15.47:0/172850696 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f44601052a0 0x7f44601a0e70 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f445800eaa0 tx=0x7f445800ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:22:22.693 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:22.690+0000 7f4457400640 1 -- 172.21.15.47:0/172850696 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f445800cdb0 con 0x7f44601052a0 2024-03-21T10:22:22.693 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:22.690+0000 7f4457400640 1 -- 172.21.15.47:0/172850696 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f4458005020 con 0x7f44601052a0 2024-03-21T10:22:22.693 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:22.690+0000 7f4464a00640 1 -- 172.21.15.47:0/172850696 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f44601a17d0 con 0x7f44601052a0 2024-03-21T10:22:22.693 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:22.690+0000 7f4457400640 1 -- 172.21.15.47:0/172850696 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44580116d0 con 0x7f44601052a0 2024-03-21T10:22:22.693 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:22.691+0000 7f4464a00640 1 -- 172.21.15.47:0/172850696 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f44601a1cf0 con 0x7f44601052a0 2024-03-21T10:22:22.694 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:22.692+0000 7f4464a00640 1 -- 172.21.15.47:0/172850696 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f446010aa10 con 0x7f44601052a0 2024-03-21T10:22:22.694 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:22.694+0000 7f4457400640 1 -- 172.21.15.47:0/172850696 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 16) v1 ==== 137854+0+0 (secure 0 0 0) 0x7f44580051d0 con 0x7f44601052a0 2024-03-21T10:22:22.696 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:22.695+0000 7f4457400640 1 --2- 172.21.15.47:0/172850696 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f44440a2f20 0x7f44440a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:22:22.696 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:22.695+0000 7f4457400640 1 -- 172.21.15.47:0/172850696 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 8802+0+0 (secure 0 0 0) 0x7f44580cf790 con 0x7f44601052a0 2024-03-21T10:22:22.696 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:22.695+0000 7f445e800640 1 --2- 172.21.15.47:0/172850696 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f44440a2f20 0x7f44440a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:22:22.697 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:22.696+0000 7f445e800640 1 --2- 172.21.15.47:0/172850696 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f44440a2f20 0x7f44440a53e0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f4450005d00 tx=0x7f4450005c90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:22:22.702 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:22.701+0000 7f4457400640 1 -- 172.21.15.47:0/172850696 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f4458016030 con 0x7f44601052a0 2024-03-21T10:22:22.916 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:22.915+0000 7f4464a00640 1 -- 172.21.15.47:0/172850696 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f446002d070 con 0x7f44601052a0 2024-03-21T10:22:22.920 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:22.920+0000 7f4457400640 1 -- 172.21.15.47:0/172850696 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v91) v1 ==== 74+0+24116 (secure 0 0 0) 0x7f4458086020 con 0x7f44601052a0 2024-03-21T10:22:22.921 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-21T10:22:22.922 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":91,"fsid":"3056d51a-e76b-11ee-95cd-87774f69a715","created":"2024-03-21T10:11:17.019965+0000","modified":"2024-03-21T10:22:03.759214+0000","last_up_change":"2024-03-21T10:20:52.675328+0000","last_in_change":"2024-03-21T10:20:27.648212+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-21T10:15:40.553114+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-21T10:21:56.399518+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-21T10:21:57.812940+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-21T10:22:01.822568+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"91","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"b0c7a102-8d89-428b-b793-cbbe67151f50","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6802","nonce":775837313},{"type":"v1","addr":"172.21.15.47:6803","nonce":775837313}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6804","nonce":775837313},{"type":"v1","addr":"172.21.15.47:6805","nonce":775837313}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6808","nonce":775837313},{"type":"v1","addr":"172.21.15.47:6809","nonce":775837313}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6806","nonce":775837313},{"type":"v1","addr":"172.21.15.47:6807","nonce":775837313}]},"public_addr":"172.21.15.47:6803/775837313","cluster_addr":"172.21.15.47:6805/775837313","heartbeat_back_addr":"172.21.15.47:6809/775837313","heartbeat_front_addr":"172.21.15.47:6807/775837313","state":["exists","up"]},{"osd":1,"uuid":"07bb81bb-7cf3-4bbc-ac9e-802f95185d11","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6800","nonce":2923304710},{"type":"v1","addr":"172.21.15.100:6801","nonce":2923304710}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":2923304710},{"type":"v1","addr":"172.21.15.100:6803","nonce":2923304710}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":2923304710},{"type":"v1","addr":"172.21.15.100:6807","nonce":2923304710}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6804","nonce":2923304710},{"type":"v1","addr":"172.21.15.100:6805","nonce":2923304710}]},"public_addr":"172.21.15.100:6801/2923304710","cluster_addr":"172.21.15.100:6803/2923304710","heartbeat_back_addr":"172.21.15.100:6807/2923304710","heartbeat_front_addr":"172.21.15.100:6805/2923304710","state":["exists","up"]},{"osd":2,"uuid":"88ce76b9-6081-4836-aad0-8c249c56e8a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":171377005},{"type":"v1","addr":"172.21.15.157:6801","nonce":171377005}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":171377005},{"type":"v1","addr":"172.21.15.157:6803","nonce":171377005}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":171377005},{"type":"v1","addr":"172.21.15.157:6807","nonce":171377005}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":171377005},{"type":"v1","addr":"172.21.15.157:6805","nonce":171377005}]},"public_addr":"172.21.15.157:6801/171377005","cluster_addr":"172.21.15.157:6803/171377005","heartbeat_back_addr":"172.21.15.157:6807/171377005","heartbeat_front_addr":"172.21.15.157:6805/171377005","state":["exists","up"]},{"osd":3,"uuid":"518aa33d-263f-45e6-aab5-2201ceb40e74","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":27,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6810","nonce":640760101},{"type":"v1","addr":"172.21.15.47:6811","nonce":640760101}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6812","nonce":640760101},{"type":"v1","addr":"172.21.15.47:6813","nonce":640760101}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6816","nonce":640760101},{"type":"v1","addr":"172.21.15.47:6817","nonce":640760101}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6814","nonce":640760101},{"type":"v1","addr":"172.21.15.47:6815","nonce":640760101}]},"public_addr":"172.21.15.47:6811/640760101","cluster_addr":"172.21.15.47:6813/640760101","heartbeat_back_addr":"172.21.15.47:6817/640760101","heartbeat_front_addr":"172.21.15.47:6815/640760101","state":["exists","up"]},{"osd":4,"uuid":"987bde83-ecc6-42e5-bd56-c143b3de4226","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":35,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6808","nonce":2785478639},{"type":"v1","addr":"172.21.15.100:6809","nonce":2785478639}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6810","nonce":2785478639},{"type":"v1","addr":"172.21.15.100:6811","nonce":2785478639}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6814","nonce":2785478639},{"type":"v1","addr":"172.21.15.100:6815","nonce":2785478639}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":2785478639},{"type":"v1","addr":"172.21.15.100:6813","nonce":2785478639}]},"public_addr":"172.21.15.100:6809/2785478639","cluster_addr":"172.21.15.100:6811/2785478639","heartbeat_back_addr":"172.21.15.100:6815/2785478639","heartbeat_front_addr":"172.21.15.100:6813/2785478639","state":["exists","up"]},{"osd":5,"uuid":"012d2188-3a61-4a5f-8d01-2001c15ffbe2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":40,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":1525645410},{"type":"v1","addr":"172.21.15.157:6809","nonce":1525645410}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":1525645410},{"type":"v1","addr":"172.21.15.157:6811","nonce":1525645410}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":1525645410},{"type":"v1","addr":"172.21.15.157:6815","nonce":1525645410}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":1525645410},{"type":"v1","addr":"172.21.15.157:6813","nonce":1525645410}]},"public_addr":"172.21.15.157:6809/1525645410","cluster_addr":"172.21.15.157:6811/1525645410","heartbeat_back_addr":"172.21.15.157:6815/1525645410","heartbeat_front_addr":"172.21.15.157:6813/1525645410","state":["exists","up"]},{"osd":6,"uuid":"c12c14b4-85f0-4d88-937c-de4f556437de","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6818","nonce":2321642790},{"type":"v1","addr":"172.21.15.47:6819","nonce":2321642790}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6820","nonce":2321642790},{"type":"v1","addr":"172.21.15.47:6821","nonce":2321642790}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6824","nonce":2321642790},{"type":"v1","addr":"172.21.15.47:6825","nonce":2321642790}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6822","nonce":2321642790},{"type":"v1","addr":"172.21.15.47:6823","nonce":2321642790}]},"public_addr":"172.21.15.47:6819/2321642790","cluster_addr":"172.21.15.47:6821/2321642790","heartbeat_back_addr":"172.21.15.47:6825/2321642790","heartbeat_front_addr":"172.21.15.47:6823/2321642790","state":["exists","up"]},{"osd":7,"uuid":"3c87c6d0-1d1f-4294-9917-aaaad7dda824","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6816","nonce":4104727330},{"type":"v1","addr":"172.21.15.100:6817","nonce":4104727330}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":4104727330},{"type":"v1","addr":"172.21.15.100:6819","nonce":4104727330}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":4104727330},{"type":"v1","addr":"172.21.15.100:6823","nonce":4104727330}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6820","nonce":4104727330},{"type":"v1","addr":"172.21.15.100:6821","nonce":4104727330}]},"public_addr":"172.21.15.100:6817/4104727330","cluster_addr":"172.21.15.100:6819/4104727330","heartbeat_back_addr":"172.21.15.100:6823/4104727330","heartbeat_front_addr":"172.21.15.100:6821/4104727330","state":["exists","up"]},{"osd":8,"uuid":"502d65cf-53c8-4936-8de4-e11b2a6e1e18","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":61,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":4067715347},{"type":"v1","addr":"172.21.15.157:6817","nonce":4067715347}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":4067715347},{"type":"v1","addr":"172.21.15.157:6819","nonce":4067715347}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":4067715347},{"type":"v1","addr":"172.21.15.157:6823","nonce":4067715347}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":4067715347},{"type":"v1","addr":"172.21.15.157:6821","nonce":4067715347}]},"public_addr":"172.21.15.157:6817/4067715347","cluster_addr":"172.21.15.157:6819/4067715347","heartbeat_back_addr":"172.21.15.157:6823/4067715347","heartbeat_front_addr":"172.21.15.157:6821/4067715347","state":["exists","up"]},{"osd":9,"uuid":"5ea1f5be-8458-4c93-9e61-c2e867a514f3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6826","nonce":1982108738},{"type":"v1","addr":"172.21.15.47:6827","nonce":1982108738}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6828","nonce":1982108738},{"type":"v1","addr":"172.21.15.47:6829","nonce":1982108738}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6832","nonce":1982108738},{"type":"v1","addr":"172.21.15.47:6833","nonce":1982108738}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6830","nonce":1982108738},{"type":"v1","addr":"172.21.15.47:6831","nonce":1982108738}]},"public_addr":"172.21.15.47:6827/1982108738","cluster_addr":"172.21.15.47:6829/1982108738","heartbeat_back_addr":"172.21.15.47:6833/1982108738","heartbeat_front_addr":"172.21.15.47:6831/1982108738","state":["exists","up"]},{"osd":10,"uuid":"2da3da90-a0be-4a54-a9ff-b2d6c9b36870","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":74,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":396391028},{"type":"v1","addr":"172.21.15.100:6825","nonce":396391028}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":396391028},{"type":"v1","addr":"172.21.15.100:6827","nonce":396391028}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6830","nonce":396391028},{"type":"v1","addr":"172.21.15.100:6831","nonce":396391028}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":396391028},{"type":"v1","addr":"172.21.15.100:6829","nonce":396391028}]},"public_addr":"172.21.15.100:6825/396391028","cluster_addr":"172.21.15.100:6827/396391028","heartbeat_back_addr":"172.21.15.100:6831/396391028","heartbeat_front_addr":"172.21.15.100:6829/396391028","state":["exists","up"]},{"osd":11,"uuid":"11798d41-7f7e-4319-95b8-8d3d0946bde9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":82,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":2671029483},{"type":"v1","addr":"172.21.15.157:6825","nonce":2671029483}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":2671029483},{"type":"v1","addr":"172.21.15.157:6827","nonce":2671029483}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":2671029483},{"type":"v1","addr":"172.21.15.157:6831","nonce":2671029483}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":2671029483},{"type":"v1","addr":"172.21.15.157:6829","nonce":2671029483}]},"public_addr":"172.21.15.157:6825/2671029483","cluster_addr":"172.21.15.157:6827/2671029483","heartbeat_back_addr":"172.21.15.157:6831/2671029483","heartbeat_front_addr":"172.21.15.157:6829/2671029483","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T10:14:31.822399+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T10:15:04.175721+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T10:15:35.902595+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T10:16:12.088891+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T10:16:46.428704+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T10:17:20.158270+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T10:17:55.915253+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T10:18:30.064641+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T10:19:04.410004+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T10:19:40.605181+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T10:20:15.636605+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T10:20:50.151365+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.47:0/356822431":"2024-03-22T10:12:25.844530+0000","172.21.15.47:0/1167490879":"2024-03-22T10:12:25.844530+0000","172.21.15.47:0/1540718138":"2024-03-22T10:12:25.844530+0000","172.21.15.47:6801/3836903972":"2024-03-22T10:12:25.844530+0000","172.21.15.47:0/235436602":"2024-03-22T10:11:50.301838+0000","172.21.15.47:0/2052815572":"2024-03-22T10:11:50.301838+0000","172.21.15.47:6800/3942658432":"2024-03-22T10:11:50.301838+0000","172.21.15.47:6801/3942658432":"2024-03-22T10:11:50.301838+0000","172.21.15.47:0/1419516257":"2024-03-22T10:11:50.301838+0000","172.21.15.47:6800/3836903972":"2024-03-22T10:12:25.844530+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-21T10:22:22.924 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:22.922+0000 7f4464a00640 1 -- 172.21.15.47:0/172850696 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f44440a2f20 msgr2=0x7f44440a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:22:22.924 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:22.922+0000 7f4464a00640 1 --2- 172.21.15.47:0/172850696 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f44440a2f20 0x7f44440a53e0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f4450005d00 tx=0x7f4450005c90 comp rx=0 tx=0).stop 2024-03-21T10:22:22.924 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:22.922+0000 7f4464a00640 1 -- 172.21.15.47:0/172850696 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f44601052a0 msgr2=0x7f44601a0e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:22:22.924 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:22.922+0000 7f4464a00640 1 --2- 172.21.15.47:0/172850696 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f44601052a0 0x7f44601a0e70 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f445800eaa0 tx=0x7f445800ef60 comp rx=0 tx=0).stop 2024-03-21T10:22:22.924 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:22.923+0000 7f4464a00640 1 -- 172.21.15.47:0/172850696 shutdown_connections 2024-03-21T10:22:22.924 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:22.923+0000 7f4464a00640 1 --2- 172.21.15.47:0/172850696 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f44440a2f20 0x7f44440a53e0 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:22:22.924 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:22.923+0000 7f4464a00640 1 --2- 172.21.15.47:0/172850696 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f44601052a0 0x7f44601a0e70 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:22:22.924 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:22.923+0000 7f4464a00640 1 --2- 172.21.15.47:0/172850696 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f44601048e0 0x7f4460199e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:22:22.924 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:22.923+0000 7f4464a00640 1 --2- 172.21.15.47:0/172850696 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f44601036f0 0x7f44601998c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:22:22.924 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:22.923+0000 7f4464a00640 1 -- 172.21.15.47:0/172850696 >> 172.21.15.47:0/172850696 conn(0x7f44600fec80 msgr2=0x7f4460107be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:22:22.925 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:22.923+0000 7f4464a00640 1 -- 172.21.15.47:0/172850696 shutdown_connections 2024-03-21T10:22:22.925 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:22.923+0000 7f4464a00640 1 -- 172.21.15.47:0/172850696 wait complete. 2024-03-21T10:22:22.944 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-03-21T10:22:23.165 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:23.163+0000 7f99bce00640 1 -- 172.21.15.47:0/1560524839 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f99b8105370 msgr2=0x7f99b810bc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:22:23.165 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:23.163+0000 7f99bce00640 1 --2- 172.21.15.47:0/1560524839 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f99b8105370 0x7f99b810bc00 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f99b0009900 tx=0x7f99b002bc30 comp rx=0 tx=0).stop 2024-03-21T10:22:23.165 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:23.163+0000 7f99bce00640 1 -- 172.21.15.47:0/1560524839 shutdown_connections 2024-03-21T10:22:23.165 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:23.163+0000 7f99bce00640 1 --2- 172.21.15.47:0/1560524839 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f99b8105370 0x7f99b810bc00 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:22:23.165 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:23.163+0000 7f99bce00640 1 --2- 172.21.15.47:0/1560524839 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f99b81049b0 0x7f99b8104e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:22:23.165 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:23.163+0000 7f99bce00640 1 --2- 172.21.15.47:0/1560524839 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f99b81037c0 0x7f99b8103be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:22:23.165 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:23.163+0000 7f99bce00640 1 -- 172.21.15.47:0/1560524839 >> 172.21.15.47:0/1560524839 conn(0x7f99b80fed50 msgr2=0x7f99b8101170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:22:23.165 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:23.164+0000 7f99bce00640 1 -- 172.21.15.47:0/1560524839 shutdown_connections 2024-03-21T10:22:23.165 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:23.164+0000 7f99bce00640 1 -- 172.21.15.47:0/1560524839 wait complete. 2024-03-21T10:22:23.167 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:23.164+0000 7f99bce00640 1 Processor -- start 2024-03-21T10:22:23.168 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:23.165+0000 7f99bce00640 1 -- start start 2024-03-21T10:22:23.168 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:23.165+0000 7f99bce00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f99b81037c0 0x7f99b8199940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:22:23.168 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:23.165+0000 7f99bce00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f99b81049b0 0x7f99b8199e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:22:23.168 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:23.165+0000 7f99bce00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f99b8105370 0x7f99b81a0ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:22:23.168 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:23.165+0000 7f99bce00640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99b810f320 con 0x7f99b8105370 2024-03-21T10:22:23.168 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:23.165+0000 7f99bce00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99b810f1a0 con 0x7f99b81037c0 2024-03-21T10:22:23.168 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:23.165+0000 7f99bce00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99b810f4a0 con 0x7f99b81049b0 2024-03-21T10:22:23.168 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:23.165+0000 7f99b7e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f99b8105370 0x7f99b81a0ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:22:23.168 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:23.165+0000 7f99b7e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f99b8105370 0x7f99b81a0ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:56466/0 (socket says 172.21.15.47:56466) 2024-03-21T10:22:23.168 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:23.165+0000 7f99b7e00640 1 -- 172.21.15.47:0/4269027289 learned_addr learned my addr 172.21.15.47:0/4269027289 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:22:23.168 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:23.166+0000 7f99b7400640 1 --2- 172.21.15.47:0/4269027289 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f99b81037c0 0x7f99b8199940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:22:23.168 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:23.166+0000 7f99b6a00640 1 --2- 172.21.15.47:0/4269027289 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f99b81049b0 0x7f99b8199e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:22:23.169 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:23.166+0000 7f99b7e00640 1 -- 172.21.15.47:0/4269027289 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f99b81049b0 msgr2=0x7f99b8199e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:22:23.169 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:23.166+0000 7f99b7e00640 1 --2- 172.21.15.47:0/4269027289 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f99b81049b0 0x7f99b8199e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:22:23.169 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:23.166+0000 7f99b7e00640 1 -- 172.21.15.47:0/4269027289 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f99b81037c0 msgr2=0x7f99b8199940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:22:23.169 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:23.166+0000 7f99b7e00640 1 --2- 172.21.15.47:0/4269027289 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f99b81037c0 0x7f99b8199940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:22:23.169 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:23.166+0000 7f99b7e00640 1 -- 172.21.15.47:0/4269027289 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99b81a1560 con 0x7f99b8105370 2024-03-21T10:22:23.169 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:23.166+0000 7f99b7e00640 1 --2- 172.21.15.47:0/4269027289 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f99b8105370 0x7f99b81a0ef0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f99b0036040 tx=0x7f99b0004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:22:23.169 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:23.166+0000 7f99afe00640 1 -- 172.21.15.47:0/4269027289 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99b0043020 con 0x7f99b8105370 2024-03-21T10:22:23.169 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:23.167+0000 7f99afe00640 1 -- 172.21.15.47:0/4269027289 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f99b0004490 con 0x7f99b8105370 2024-03-21T10:22:23.169 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:23.167+0000 7f99bce00640 1 -- 172.21.15.47:0/4269027289 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f99b81a17f0 con 0x7f99b8105370 2024-03-21T10:22:23.169 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:23.167+0000 7f99afe00640 1 -- 172.21.15.47:0/4269027289 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99b0003d10 con 0x7f99b8105370 2024-03-21T10:22:23.169 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:23.167+0000 7f99bce00640 1 -- 172.21.15.47:0/4269027289 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f99b81a1d40 con 0x7f99b8105370 2024-03-21T10:22:23.170 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:23.168+0000 7f99bce00640 1 -- 172.21.15.47:0/4269027289 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f99b810aae0 con 0x7f99b8105370 2024-03-21T10:22:23.171 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:23.170+0000 7f99afe00640 1 -- 172.21.15.47:0/4269027289 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 16) v1 ==== 137854+0+0 (secure 0 0 0) 0x7f99b0046020 con 0x7f99b8105370 2024-03-21T10:22:23.172 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:23.171+0000 7f99afe00640 1 --2- 172.21.15.47:0/4269027289 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f999c0a2f20 0x7f999c0a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:22:23.172 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:23.171+0000 7f99afe00640 1 -- 172.21.15.47:0/4269027289 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 8802+0+0 (secure 0 0 0) 0x7f99b00f1af0 con 0x7f99b8105370 2024-03-21T10:22:23.172 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:23.172+0000 7f99b7400640 1 --2- 172.21.15.47:0/4269027289 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f999c0a2f20 0x7f999c0a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:22:23.173 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:23.172+0000 7f99b7400640 1 --2- 172.21.15.47:0/4269027289 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f999c0a2f20 0x7f999c0a53e0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f99b8104810 tx=0x7f99a80098f0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:22:23.176 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:23.175+0000 7f99afe00640 1 -- 172.21.15.47:0/4269027289 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f99b004d020 con 0x7f99b8105370 2024-03-21T10:22:23.214 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:22:23 smithi047 ceph-mon[31547]: pgmap v379: 129 pgs: 129 active+clean; 581 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 0 op/s 2024-03-21T10:22:23.214 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:22:23 smithi047 ceph-mon[31547]: from='client.? 172.21.15.47:0/523497770' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-21T10:22:23.214 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:22:23 smithi047 ceph-mon[31547]: from='client.? 172.21.15.47:0/172850696' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-21T10:22:23.424 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:23.423+0000 7f99bce00640 1 -- 172.21.15.47:0/4269027289 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f99b80622a0 con 0x7f99b8105370 2024-03-21T10:22:23.479 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:22:23 smithi100 ceph-mon[35193]: pgmap v379: 129 pgs: 129 active+clean; 581 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 0 op/s 2024-03-21T10:22:23.480 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:22:23 smithi100 ceph-mon[35193]: from='client.? 172.21.15.47:0/523497770' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-21T10:22:23.480 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:22:23 smithi100 ceph-mon[35193]: from='client.? 172.21.15.47:0/172850696' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-21T10:22:23.490 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:22:23 smithi157 ceph-mon[35581]: pgmap v379: 129 pgs: 129 active+clean; 581 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 0 op/s 2024-03-21T10:22:23.490 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:22:23 smithi157 ceph-mon[35581]: from='client.? 172.21.15.47:0/523497770' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-21T10:22:23.490 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:22:23 smithi157 ceph-mon[35581]: from='client.? 172.21.15.47:0/172850696' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-21T10:22:24.154 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:24.152+0000 7f99afe00640 1 -- 172.21.15.47:0/4269027289 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v18) v1 ==== 118+0+0 (secure 0 0 0) 0x7f99b00be680 con 0x7f99b8105370 2024-03-21T10:22:24.157 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:24.156+0000 7f99bce00640 1 -- 172.21.15.47:0/4269027289 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f999c0a2f20 msgr2=0x7f999c0a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:22:24.157 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:24.156+0000 7f99bce00640 1 --2- 172.21.15.47:0/4269027289 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f999c0a2f20 0x7f999c0a53e0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f99b8104810 tx=0x7f99a80098f0 comp rx=0 tx=0).stop 2024-03-21T10:22:24.157 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:24.156+0000 7f99bce00640 1 -- 172.21.15.47:0/4269027289 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f99b8105370 msgr2=0x7f99b81a0ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:22:24.158 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:24.156+0000 7f99bce00640 1 --2- 172.21.15.47:0/4269027289 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f99b8105370 0x7f99b81a0ef0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f99b0036040 tx=0x7f99b0004270 comp rx=0 tx=0).stop 2024-03-21T10:22:24.158 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:24.156+0000 7f99bce00640 1 -- 172.21.15.47:0/4269027289 shutdown_connections 2024-03-21T10:22:24.158 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:24.156+0000 7f99bce00640 1 --2- 172.21.15.47:0/4269027289 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f999c0a2f20 0x7f999c0a53e0 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:22:24.158 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:24.156+0000 7f99bce00640 1 --2- 172.21.15.47:0/4269027289 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f99b8105370 0x7f99b81a0ef0 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:22:24.158 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:24.156+0000 7f99bce00640 1 --2- 172.21.15.47:0/4269027289 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f99b81049b0 0x7f99b8199e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:22:24.158 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:24.156+0000 7f99bce00640 1 --2- 172.21.15.47:0/4269027289 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f99b81037c0 0x7f99b8199940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:22:24.158 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:24.156+0000 7f99bce00640 1 -- 172.21.15.47:0/4269027289 >> 172.21.15.47:0/4269027289 conn(0x7f99b80fed50 msgr2=0x7f99b8107cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:22:24.158 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:24.156+0000 7f99bce00640 1 -- 172.21.15.47:0/4269027289 shutdown_connections 2024-03-21T10:22:24.158 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:24.156+0000 7f99bce00640 1 -- 172.21.15.47:0/4269027289 wait complete. 2024-03-21T10:22:24.180 DEBUG:tasks.cephfs.filesystem:fs_config: {'ec_profile': ['m=2', 'k=2', 'crush-failure-domain=osd'], 'max_mds': 5, 'session_timeout': 300, 'standby_replay': True, 'subvols': {'create': 2, 'subvol_options': '--namespace-isolated'}} 2024-03-21T10:22:24.181 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs max_mds 5 2024-03-21T10:22:24.413 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:24.411+0000 7ffa47a00640 1 -- 172.21.15.47:0/2444954158 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ffa401016b0 msgr2=0x7ffa40101ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:22:24.413 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:24.411+0000 7ffa47a00640 1 --2- 172.21.15.47:0/2444954158 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ffa401016b0 0x7ffa40101ad0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7ffa38009ca0 tx=0x7ffa3802c0f0 comp rx=0 tx=0).stop 2024-03-21T10:22:24.413 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:24.411+0000 7ffa47a00640 1 -- 172.21.15.47:0/2444954158 shutdown_connections 2024-03-21T10:22:24.413 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:24.411+0000 7ffa47a00640 1 --2- 172.21.15.47:0/2444954158 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ffa40103260 0x7ffa40109af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:22:24.413 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:24.411+0000 7ffa47a00640 1 --2- 172.21.15.47:0/2444954158 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ffa401028a0 0x7ffa40102d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:22:24.413 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:24.412+0000 7ffa47a00640 1 --2- 172.21.15.47:0/2444954158 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ffa401016b0 0x7ffa40101ad0 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:22:24.414 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:24.412+0000 7ffa47a00640 1 -- 172.21.15.47:0/2444954158 >> 172.21.15.47:0/2444954158 conn(0x7ffa400fcc40 msgr2=0x7ffa400ff060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:22:24.414 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:24.412+0000 7ffa47a00640 1 -- 172.21.15.47:0/2444954158 shutdown_connections 2024-03-21T10:22:24.414 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:24.412+0000 7ffa47a00640 1 -- 172.21.15.47:0/2444954158 wait complete. 2024-03-21T10:22:24.416 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:24.413+0000 7ffa47a00640 1 Processor -- start 2024-03-21T10:22:24.416 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:24.413+0000 7ffa47a00640 1 -- start start 2024-03-21T10:22:24.416 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:24.413+0000 7ffa47a00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ffa401016b0 0x7ffa40199b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:22:24.416 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:24.413+0000 7ffa47a00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ffa401028a0 0x7ffa4019a090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:22:24.416 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:24.413+0000 7ffa47a00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ffa40103260 0x7ffa401a1100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:22:24.416 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:24.413+0000 7ffa47a00640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa4010d1e0 con 0x7ffa401028a0 2024-03-21T10:22:24.417 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:24.413+0000 7ffa47a00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa4010d060 con 0x7ffa401016b0 2024-03-21T10:22:24.417 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:24.413+0000 7ffa47a00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa4010d360 con 0x7ffa40103260 2024-03-21T10:22:24.417 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:24.413+0000 7ffa3fe00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ffa401028a0 0x7ffa4019a090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:22:24.417 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:24.414+0000 7ffa3fe00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ffa401028a0 0x7ffa4019a090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:56484/0 (socket says 172.21.15.47:56484) 2024-03-21T10:22:24.417 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:24.414+0000 7ffa3fe00640 1 -- 172.21.15.47:0/87746632 learned_addr learned my addr 172.21.15.47:0/87746632 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:22:24.417 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:24.414+0000 7ffa45000640 1 --2- 172.21.15.47:0/87746632 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ffa401016b0 0x7ffa40199b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:22:24.417 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:24.414+0000 7ffa45a00640 1 --2- 172.21.15.47:0/87746632 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ffa40103260 0x7ffa401a1100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:22:24.417 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:24.414+0000 7ffa3fe00640 1 -- 172.21.15.47:0/87746632 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ffa40103260 msgr2=0x7ffa401a1100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:22:24.417 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:24.414+0000 7ffa3fe00640 1 --2- 172.21.15.47:0/87746632 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ffa40103260 0x7ffa401a1100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:22:24.417 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:24.414+0000 7ffa3fe00640 1 -- 172.21.15.47:0/87746632 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ffa401016b0 msgr2=0x7ffa40199b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:22:24.417 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:24.414+0000 7ffa3fe00640 1 --2- 172.21.15.47:0/87746632 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ffa401016b0 0x7ffa40199b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:22:24.417 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:24.414+0000 7ffa3fe00640 1 -- 172.21.15.47:0/87746632 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffa401a1770 con 0x7ffa401028a0 2024-03-21T10:22:24.417 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:24.414+0000 7ffa45000640 1 --2- 172.21.15.47:0/87746632 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ffa401016b0 0x7ffa40199b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T10:22:24.418 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:24.414+0000 7ffa3fe00640 1 --2- 172.21.15.47:0/87746632 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ffa401028a0 0x7ffa4019a090 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7ffa2800d770 tx=0x7ffa2800dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:22:24.418 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:24.415+0000 7ffa3d600640 1 -- 172.21.15.47:0/87746632 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa28017070 con 0x7ffa401028a0 2024-03-21T10:22:24.418 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:24.415+0000 7ffa3d600640 1 -- 172.21.15.47:0/87746632 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7ffa280054d0 con 0x7ffa401028a0 2024-03-21T10:22:24.418 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:24.415+0000 7ffa3d600640 1 -- 172.21.15.47:0/87746632 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa28004cf0 con 0x7ffa401028a0 2024-03-21T10:22:24.418 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:24.415+0000 7ffa47a00640 1 -- 172.21.15.47:0/87746632 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffa401a1a60 con 0x7ffa401028a0 2024-03-21T10:22:24.418 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:24.415+0000 7ffa47a00640 1 -- 172.21.15.47:0/87746632 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffa401a1f60 con 0x7ffa401028a0 2024-03-21T10:22:24.418 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:24.416+0000 7ffa47a00640 1 -- 172.21.15.47:0/87746632 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffa401a2190 con 0x7ffa401028a0 2024-03-21T10:22:24.419 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:24.419+0000 7ffa3d600640 1 -- 172.21.15.47:0/87746632 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 16) v1 ==== 137854+0+0 (secure 0 0 0) 0x7ffa28004e90 con 0x7ffa401028a0 2024-03-21T10:22:24.421 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:24.420+0000 7ffa3d600640 1 --2- 172.21.15.47:0/87746632 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7ffa240a2fa0 0x7ffa240a5460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:22:24.421 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:24.420+0000 7ffa3d600640 1 -- 172.21.15.47:0/87746632 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 8802+0+0 (secure 0 0 0) 0x7ffa280cf4e0 con 0x7ffa401028a0 2024-03-21T10:22:24.421 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:24.420+0000 7ffa45000640 1 --2- 172.21.15.47:0/87746632 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7ffa240a2fa0 0x7ffa240a5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:22:24.422 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:24.421+0000 7ffa45000640 1 --2- 172.21.15.47:0/87746632 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7ffa240a2fa0 0x7ffa240a5460 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7ffa3802c630 tx=0x7ffa38032510 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:22:24.424 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:24.423+0000 7ffa3d600640 1 -- 172.21.15.47:0/87746632 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7ffa2809c0f0 con 0x7ffa401028a0 2024-03-21T10:22:24.464 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:22:24 smithi047 ceph-mon[31547]: from='client.? 172.21.15.47:0/4269027289' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-03-21T10:22:24.479 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:22:24 smithi100 ceph-mon[35193]: from='client.? 172.21.15.47:0/4269027289' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-03-21T10:22:24.489 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:22:24 smithi157 ceph-mon[35581]: from='client.? 172.21.15.47:0/4269027289' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-03-21T10:22:24.672 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:24.670+0000 7ffa47a00640 1 -- 172.21.15.47:0/87746632 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"} v 0) v1 -- 0x7ffa401a2480 con 0x7ffa401028a0 2024-03-21T10:22:25.157 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:25.156+0000 7ffa3d600640 1 -- 172.21.15.47:0/87746632 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]=0 v19) v1 ==== 105+0+0 (secure 0 0 0) 0x7ffa2809b9f0 con 0x7ffa401028a0 2024-03-21T10:22:25.162 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:25.160+0000 7ffa47a00640 1 -- 172.21.15.47:0/87746632 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7ffa240a2fa0 msgr2=0x7ffa240a5460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:22:25.162 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:25.160+0000 7ffa47a00640 1 --2- 172.21.15.47:0/87746632 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7ffa240a2fa0 0x7ffa240a5460 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7ffa3802c630 tx=0x7ffa38032510 comp rx=0 tx=0).stop 2024-03-21T10:22:25.162 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:25.160+0000 7ffa47a00640 1 -- 172.21.15.47:0/87746632 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ffa401028a0 msgr2=0x7ffa4019a090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:22:25.162 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:25.160+0000 7ffa47a00640 1 --2- 172.21.15.47:0/87746632 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ffa401028a0 0x7ffa4019a090 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7ffa2800d770 tx=0x7ffa2800dc30 comp rx=0 tx=0).stop 2024-03-21T10:22:25.162 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:25.160+0000 7ffa47a00640 1 -- 172.21.15.47:0/87746632 shutdown_connections 2024-03-21T10:22:25.162 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:25.160+0000 7ffa47a00640 1 --2- 172.21.15.47:0/87746632 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7ffa240a2fa0 0x7ffa240a5460 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:22:25.162 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:25.160+0000 7ffa47a00640 1 --2- 172.21.15.47:0/87746632 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ffa40103260 0x7ffa401a1100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:22:25.162 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:25.160+0000 7ffa47a00640 1 --2- 172.21.15.47:0/87746632 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ffa401028a0 0x7ffa4019a090 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:22:25.162 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:25.160+0000 7ffa47a00640 1 --2- 172.21.15.47:0/87746632 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ffa401016b0 0x7ffa40199b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:22:25.162 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:25.160+0000 7ffa47a00640 1 -- 172.21.15.47:0/87746632 >> 172.21.15.47:0/87746632 conn(0x7ffa400fcc40 msgr2=0x7ffa40108070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:22:25.163 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:25.160+0000 7ffa47a00640 1 -- 172.21.15.47:0/87746632 shutdown_connections 2024-03-21T10:22:25.163 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:25.161+0000 7ffa47a00640 1 -- 172.21.15.47:0/87746632 wait complete. 2024-03-21T10:22:25.189 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay true 2024-03-21T10:22:25.398 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:25.396+0000 7f9a89800640 1 -- 172.21.15.47:0/3199682310 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9a84109870 msgr2=0x7f9a84110100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:22:25.399 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:25.396+0000 7f9a89800640 1 --2- 172.21.15.47:0/3199682310 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9a84109870 0x7f9a84110100 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f9a7c009900 tx=0x7f9a7c02bc40 comp rx=0 tx=0).stop 2024-03-21T10:22:25.399 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:25.396+0000 7f9a89800640 1 -- 172.21.15.47:0/3199682310 shutdown_connections 2024-03-21T10:22:25.399 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:25.396+0000 7f9a89800640 1 --2- 172.21.15.47:0/3199682310 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9a84109870 0x7f9a84110100 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:22:25.399 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:25.396+0000 7f9a89800640 1 --2- 172.21.15.47:0/3199682310 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9a84108eb0 0x7f9a84109330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:22:25.400 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:25.396+0000 7f9a89800640 1 --2- 172.21.15.47:0/3199682310 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9a84107cc0 0x7f9a841080e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:22:25.400 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:25.396+0000 7f9a89800640 1 -- 172.21.15.47:0/3199682310 >> 172.21.15.47:0/3199682310 conn(0x7f9a84076c60 msgr2=0x7f9a840790a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:22:25.400 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:25.396+0000 7f9a89800640 1 -- 172.21.15.47:0/3199682310 shutdown_connections 2024-03-21T10:22:25.400 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:25.397+0000 7f9a89800640 1 -- 172.21.15.47:0/3199682310 wait complete. 2024-03-21T10:22:25.402 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:25.397+0000 7f9a89800640 1 Processor -- start 2024-03-21T10:22:25.402 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:25.398+0000 7f9a89800640 1 -- start start 2024-03-21T10:22:25.402 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:25.398+0000 7f9a89800640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9a84107cc0 0x7f9a8419deb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:22:25.402 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:25.398+0000 7f9a89800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9a84108eb0 0x7f9a8419e3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:22:25.402 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:25.398+0000 7f9a89800640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9a84109870 0x7f9a841a5460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:22:25.402 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:25.398+0000 7f9a89800640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a84113840 con 0x7f9a84107cc0 2024-03-21T10:22:25.402 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:25.398+0000 7f9a89800640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a841136c0 con 0x7f9a84108eb0 2024-03-21T10:22:25.402 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:25.398+0000 7f9a89800640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a841139c0 con 0x7f9a84109870 2024-03-21T10:22:25.402 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:25.398+0000 7f9a82a00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9a84107cc0 0x7f9a8419deb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:22:25.403 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:25.398+0000 7f9a82a00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9a84107cc0 0x7f9a8419deb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:56512/0 (socket says 172.21.15.47:56512) 2024-03-21T10:22:25.403 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:25.398+0000 7f9a82a00640 1 -- 172.21.15.47:0/1030923006 learned_addr learned my addr 172.21.15.47:0/1030923006 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:22:25.403 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:25.399+0000 7f9a82000640 1 --2- 172.21.15.47:0/1030923006 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9a84108eb0 0x7f9a8419e3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:22:25.403 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:25.399+0000 7f9a82a00640 1 -- 172.21.15.47:0/1030923006 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9a84109870 msgr2=0x7f9a841a5460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:22:25.403 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:25.399+0000 7f9a82a00640 1 --2- 172.21.15.47:0/1030923006 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9a84109870 0x7f9a841a5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:22:25.403 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:25.399+0000 7f9a82a00640 1 -- 172.21.15.47:0/1030923006 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9a84108eb0 msgr2=0x7f9a8419e3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:22:25.403 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:25.399+0000 7f9a82a00640 1 --2- 172.21.15.47:0/1030923006 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9a84108eb0 0x7f9a8419e3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:22:25.403 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:25.399+0000 7f9a82a00640 1 -- 172.21.15.47:0/1030923006 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a841a5ad0 con 0x7f9a84107cc0 2024-03-21T10:22:25.403 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:25.399+0000 7f9a82000640 1 --2- 172.21.15.47:0/1030923006 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9a84108eb0 0x7f9a8419e3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T10:22:25.403 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:25.399+0000 7f9a82a00640 1 --2- 172.21.15.47:0/1030923006 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9a84107cc0 0x7f9a8419deb0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f9a7400d950 tx=0x7f9a7400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:22:25.403 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:25.399+0000 7f9a7b400640 1 -- 172.21.15.47:0/1030923006 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a74017070 con 0x7f9a84107cc0 2024-03-21T10:22:25.403 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:25.399+0000 7f9a89800640 1 -- 172.21.15.47:0/1030923006 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9a841a5dc0 con 0x7f9a84107cc0 2024-03-21T10:22:25.403 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:25.399+0000 7f9a7b400640 1 -- 172.21.15.47:0/1030923006 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f9a74009c30 con 0x7f9a84107cc0 2024-03-21T10:22:25.403 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:25.399+0000 7f9a7b400640 1 -- 172.21.15.47:0/1030923006 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a7400f620 con 0x7f9a84107cc0 2024-03-21T10:22:25.404 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:25.400+0000 7f9a89800640 1 -- 172.21.15.47:0/1030923006 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9a841a6230 con 0x7f9a84107cc0 2024-03-21T10:22:25.404 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:25.401+0000 7f9a89800640 1 -- 172.21.15.47:0/1030923006 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9a8410efe0 con 0x7f9a84107cc0 2024-03-21T10:22:25.404 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:25.403+0000 7f9a7b400640 1 -- 172.21.15.47:0/1030923006 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 16) v1 ==== 137854+0+0 (secure 0 0 0) 0x7f9a7401a070 con 0x7f9a84107cc0 2024-03-21T10:22:25.406 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:25.405+0000 7f9a7b400640 1 --2- 172.21.15.47:0/1030923006 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f9a680a2f20 0x7f9a680a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:22:25.406 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:25.405+0000 7f9a7b400640 1 -- 172.21.15.47:0/1030923006 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 8802+0+0 (secure 0 0 0) 0x7f9a740cf190 con 0x7f9a84107cc0 2024-03-21T10:22:25.406 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:25.405+0000 7f9a82000640 1 --2- 172.21.15.47:0/1030923006 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f9a680a2f20 0x7f9a680a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:22:25.407 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:25.406+0000 7f9a82000640 1 --2- 172.21.15.47:0/1030923006 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f9a680a2f20 0x7f9a680a53e0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f9a8419f3c0 tx=0x7f9a700093e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:22:25.410 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:25.410+0000 7f9a7b400640 1 -- 172.21.15.47:0/1030923006 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f9a74009de0 con 0x7f9a84107cc0 2024-03-21T10:22:25.464 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:22:25 smithi047 ceph-mon[31547]: pgmap v380: 129 pgs: 129 active+clean; 581 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2024-03-21T10:22:25.464 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:22:25 smithi047 ceph-mon[31547]: from='client.? 172.21.15.47:0/4269027289' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-03-21T10:22:25.464 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:22:25 smithi047 ceph-mon[31547]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-03-21T10:22:25.464 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:22:25 smithi047 ceph-mon[31547]: from='client.? 172.21.15.47:0/87746632' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]: dispatch 2024-03-21T10:22:25.480 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:22:25 smithi100 ceph-mon[35193]: pgmap v380: 129 pgs: 129 active+clean; 581 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2024-03-21T10:22:25.480 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:22:25 smithi100 ceph-mon[35193]: from='client.? 172.21.15.47:0/4269027289' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-03-21T10:22:25.480 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:22:25 smithi100 ceph-mon[35193]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-03-21T10:22:25.480 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:22:25 smithi100 ceph-mon[35193]: from='client.? 172.21.15.47:0/87746632' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]: dispatch 2024-03-21T10:22:25.490 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:22:25 smithi157 ceph-mon[35581]: pgmap v380: 129 pgs: 129 active+clean; 581 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2024-03-21T10:22:25.490 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:22:25 smithi157 ceph-mon[35581]: from='client.? 172.21.15.47:0/4269027289' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-03-21T10:22:25.490 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:22:25 smithi157 ceph-mon[35581]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-03-21T10:22:25.490 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:22:25 smithi157 ceph-mon[35581]: from='client.? 172.21.15.47:0/87746632' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]: dispatch 2024-03-21T10:22:25.655 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:25.654+0000 7f9a89800640 1 -- 172.21.15.47:0/1030923006 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"} v 0) v1 -- 0x7f9a840623f0 con 0x7f9a84107cc0 2024-03-21T10:22:26.172 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:26.171+0000 7f9a7b400640 1 -- 172.21.15.47:0/1030923006 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]=0 v21) v1 ==== 121+0+0 (secure 0 0 0) 0x7f9a7409bd20 con 0x7f9a84107cc0 2024-03-21T10:22:26.176 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:26.174+0000 7f9a89800640 1 -- 172.21.15.47:0/1030923006 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f9a680a2f20 msgr2=0x7f9a680a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:22:26.176 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:26.174+0000 7f9a89800640 1 --2- 172.21.15.47:0/1030923006 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f9a680a2f20 0x7f9a680a53e0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f9a8419f3c0 tx=0x7f9a700093e0 comp rx=0 tx=0).stop 2024-03-21T10:22:26.176 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:26.175+0000 7f9a89800640 1 -- 172.21.15.47:0/1030923006 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9a84107cc0 msgr2=0x7f9a8419deb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:22:26.176 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:26.175+0000 7f9a89800640 1 --2- 172.21.15.47:0/1030923006 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9a84107cc0 0x7f9a8419deb0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f9a7400d950 tx=0x7f9a7400de10 comp rx=0 tx=0).stop 2024-03-21T10:22:26.176 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:26.175+0000 7f9a89800640 1 -- 172.21.15.47:0/1030923006 shutdown_connections 2024-03-21T10:22:26.176 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:26.175+0000 7f9a89800640 1 --2- 172.21.15.47:0/1030923006 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f9a680a2f20 0x7f9a680a53e0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:22:26.176 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:26.175+0000 7f9a89800640 1 --2- 172.21.15.47:0/1030923006 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9a84109870 0x7f9a841a5460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:22:26.177 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:26.175+0000 7f9a89800640 1 --2- 172.21.15.47:0/1030923006 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9a84108eb0 0x7f9a8419e3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:22:26.177 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:26.175+0000 7f9a89800640 1 --2- 172.21.15.47:0/1030923006 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9a84107cc0 0x7f9a8419deb0 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:22:26.177 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:26.175+0000 7f9a89800640 1 -- 172.21.15.47:0/1030923006 >> 172.21.15.47:0/1030923006 conn(0x7f9a84076c60 msgr2=0x7f9a8410c1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:22:26.177 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:26.175+0000 7f9a89800640 1 -- 172.21.15.47:0/1030923006 shutdown_connections 2024-03-21T10:22:26.177 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:26.175+0000 7f9a89800640 1 -- 172.21.15.47:0/1030923006 wait complete. 2024-03-21T10:22:26.210 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs session_timeout 300 2024-03-21T10:22:26.403 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:26.401+0000 7fd64e400640 1 -- 172.21.15.47:0/3659326983 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd648074100 msgr2=0x7fd648074580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:22:26.403 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:26.401+0000 7fd64e400640 1 --2- 172.21.15.47:0/3659326983 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd648074100 0x7fd648074580 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fd630009900 tx=0x7fd63002bc30 comp rx=0 tx=0).stop 2024-03-21T10:22:26.403 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:26.401+0000 7fd64e400640 1 -- 172.21.15.47:0/3659326983 shutdown_connections 2024-03-21T10:22:26.403 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:26.401+0000 7fd64e400640 1 --2- 172.21.15.47:0/3659326983 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd648109460 0x7fd64810b8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:22:26.403 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:26.401+0000 7fd64e400640 1 --2- 172.21.15.47:0/3659326983 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd648074100 0x7fd648074580 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:22:26.403 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:26.401+0000 7fd64e400640 1 --2- 172.21.15.47:0/3659326983 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd6480756e0 0x7fd648073b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:22:26.403 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:26.401+0000 7fd64e400640 1 -- 172.21.15.47:0/3659326983 >> 172.21.15.47:0/3659326983 conn(0x7fd6480fcb10 msgr2=0x7fd6480fef50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:22:26.404 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:26.401+0000 7fd64e400640 1 -- 172.21.15.47:0/3659326983 shutdown_connections 2024-03-21T10:22:26.404 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:26.401+0000 7fd64e400640 1 -- 172.21.15.47:0/3659326983 wait complete. 2024-03-21T10:22:26.406 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:26.402+0000 7fd64e400640 1 Processor -- start 2024-03-21T10:22:26.406 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:26.402+0000 7fd64e400640 1 -- start start 2024-03-21T10:22:26.406 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:26.403+0000 7fd64e400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd648074100 0x7fd6481999a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:22:26.407 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:26.403+0000 7fd64e400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd6480756e0 0x7fd648199ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:22:26.407 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:26.403+0000 7fd64e400640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd648109460 0x7fd6481a0f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:22:26.407 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:26.403+0000 7fd64e400640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd64810f2e0 con 0x7fd648074100 2024-03-21T10:22:26.407 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:26.403+0000 7fd64e400640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd64810f160 con 0x7fd6480756e0 2024-03-21T10:22:26.407 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:26.403+0000 7fd64e400640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd64810f460 con 0x7fd648109460 2024-03-21T10:22:26.407 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:26.403+0000 7fd64d000640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd648074100 0x7fd6481999a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:22:26.407 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:26.403+0000 7fd64d000640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd648074100 0x7fd6481999a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:56532/0 (socket says 172.21.15.47:56532) 2024-03-21T10:22:26.407 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:26.403+0000 7fd64d000640 1 -- 172.21.15.47:0/1492577017 learned_addr learned my addr 172.21.15.47:0/1492577017 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:22:26.407 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:26.403+0000 7fd63fe00640 1 --2- 172.21.15.47:0/1492577017 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd6480756e0 0x7fd648199ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:22:26.407 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:26.404+0000 7fd64da00640 1 --2- 172.21.15.47:0/1492577017 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd648109460 0x7fd6481a0f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:22:26.407 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:26.404+0000 7fd64d000640 1 -- 172.21.15.47:0/1492577017 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd648109460 msgr2=0x7fd6481a0f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:22:26.407 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:26.404+0000 7fd64d000640 1 --2- 172.21.15.47:0/1492577017 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd648109460 0x7fd6481a0f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:22:26.408 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:26.404+0000 7fd64d000640 1 -- 172.21.15.47:0/1492577017 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd6480756e0 msgr2=0x7fd648199ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:22:26.408 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:26.404+0000 7fd64d000640 1 --2- 172.21.15.47:0/1492577017 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd6480756e0 0x7fd648199ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:22:26.408 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:26.404+0000 7fd64d000640 1 -- 172.21.15.47:0/1492577017 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6481a15c0 con 0x7fd648074100 2024-03-21T10:22:26.408 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:26.404+0000 7fd64da00640 1 --2- 172.21.15.47:0/1492577017 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd648109460 0x7fd6481a0f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T10:22:26.408 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:26.404+0000 7fd64d000640 1 --2- 172.21.15.47:0/1492577017 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd648074100 0x7fd6481999a0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fd63800d950 tx=0x7fd63800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:22:26.408 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:26.404+0000 7fd646000640 1 -- 172.21.15.47:0/1492577017 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd638017070 con 0x7fd648074100 2024-03-21T10:22:26.408 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:26.404+0000 7fd646000640 1 -- 172.21.15.47:0/1492577017 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fd638009c30 con 0x7fd648074100 2024-03-21T10:22:26.408 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:26.404+0000 7fd64e400640 1 -- 172.21.15.47:0/1492577017 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd6481a18b0 con 0x7fd648074100 2024-03-21T10:22:26.408 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:26.404+0000 7fd646000640 1 -- 172.21.15.47:0/1492577017 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd63800f620 con 0x7fd648074100 2024-03-21T10:22:26.408 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:26.405+0000 7fd64e400640 1 -- 172.21.15.47:0/1492577017 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd6481a1dd0 con 0x7fd648074100 2024-03-21T10:22:26.409 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:26.406+0000 7fd64e400640 1 -- 172.21.15.47:0/1492577017 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd648108190 con 0x7fd648074100 2024-03-21T10:22:26.409 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:26.408+0000 7fd646000640 1 -- 172.21.15.47:0/1492577017 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 16) v1 ==== 137854+0+0 (secure 0 0 0) 0x7fd6380109a0 con 0x7fd648074100 2024-03-21T10:22:26.410 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:26.409+0000 7fd646000640 1 --2- 172.21.15.47:0/1492577017 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7fd62c0a2f20 0x7fd62c0a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:22:26.410 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:26.409+0000 7fd646000640 1 -- 172.21.15.47:0/1492577017 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9092+0+0 (secure 0 0 0) 0x7fd6380cf3f0 con 0x7fd648074100 2024-03-21T10:22:26.410 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:26.409+0000 7fd63fe00640 1 --2- 172.21.15.47:0/1492577017 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7fd62c0a2f20 0x7fd62c0a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:22:26.411 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:26.410+0000 7fd63fe00640 1 --2- 172.21.15.47:0/1492577017 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7fd62c0a2f20 0x7fd62c0a53e0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fd630000c00 tx=0x7fd63002c7e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:22:26.414 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:26.414+0000 7fd646000640 1 -- 172.21.15.47:0/1492577017 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fd638009de0 con 0x7fd648074100 2024-03-21T10:22:26.465 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:22:26 smithi047 ceph-mon[31547]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-03-21T10:22:26.465 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:22:26 smithi047 ceph-mon[31547]: from='client.? 172.21.15.47:0/87746632' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]': finished 2024-03-21T10:22:26.465 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:22:26 smithi047 ceph-mon[31547]: daemon mds.h assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-03-21T10:22:26.465 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:22:26 smithi047 ceph-mon[31547]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-03-21T10:22:26.465 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:22:26 smithi047 ceph-mon[31547]: fsmap cephfs:2 {0=b=up:active,1=h=up:creating} 10 up:standby 2024-03-21T10:22:26.465 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:22:26 smithi047 ceph-mon[31547]: daemon mds.h is now active in filesystem cephfs as rank 1 2024-03-21T10:22:26.465 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:22:26 smithi047 ceph-mon[31547]: from='client.? 172.21.15.47:0/1030923006' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-03-21T10:22:26.465 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:22:26 smithi047 ceph-mon[31547]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.11", "id": [7, 11]}]: dispatch 2024-03-21T10:22:26.465 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:22:26 smithi047 ceph-mon[31547]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.14", "id": [2, 4]}]: dispatch 2024-03-21T10:22:26.465 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:22:26 smithi047 ceph-mon[31547]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1a", "id": [8, 1]}]: dispatch 2024-03-21T10:22:26.465 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:22:26 smithi047 ceph-mon[31547]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1b", "id": [2, 4]}]: dispatch 2024-03-21T10:22:26.466 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:22:26 smithi047 ceph-mon[31547]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1d", "id": [8, 11]}]: dispatch 2024-03-21T10:22:26.466 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:22:26 smithi047 ceph-mon[31547]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.0", "id": [4, 8]}]: dispatch 2024-03-21T10:22:26.466 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:22:26 smithi047 ceph-mon[31547]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.6", "id": [3, 7]}]: dispatch 2024-03-21T10:22:26.466 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:22:26 smithi047 ceph-mon[31547]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1b", "id": [4, 6]}]: dispatch 2024-03-21T10:22:26.466 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:22:26 smithi047 ceph-mon[31547]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.33", "id": [5, 9]}]: dispatch 2024-03-21T10:22:26.466 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:22:26 smithi047 ceph-mon[31547]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3c", "id": [4, 9]}]: dispatch 2024-03-21T10:22:26.480 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:22:26 smithi100 ceph-mon[35193]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-03-21T10:22:26.480 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:22:26 smithi100 ceph-mon[35193]: from='client.? 172.21.15.47:0/87746632' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]': finished 2024-03-21T10:22:26.480 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:22:26 smithi100 ceph-mon[35193]: daemon mds.h assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-03-21T10:22:26.480 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:22:26 smithi100 ceph-mon[35193]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-03-21T10:22:26.480 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:22:26 smithi100 ceph-mon[35193]: fsmap cephfs:2 {0=b=up:active,1=h=up:creating} 10 up:standby 2024-03-21T10:22:26.480 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:22:26 smithi100 ceph-mon[35193]: daemon mds.h is now active in filesystem cephfs as rank 1 2024-03-21T10:22:26.480 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:22:26 smithi100 ceph-mon[35193]: from='client.? 172.21.15.47:0/1030923006' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-03-21T10:22:26.480 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:22:26 smithi100 ceph-mon[35193]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.11", "id": [7, 11]}]: dispatch 2024-03-21T10:22:26.480 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:22:26 smithi100 ceph-mon[35193]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.14", "id": [2, 4]}]: dispatch 2024-03-21T10:22:26.481 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:22:26 smithi100 ceph-mon[35193]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1a", "id": [8, 1]}]: dispatch 2024-03-21T10:22:26.481 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:22:26 smithi100 ceph-mon[35193]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1b", "id": [2, 4]}]: dispatch 2024-03-21T10:22:26.481 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:22:26 smithi100 ceph-mon[35193]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1d", "id": [8, 11]}]: dispatch 2024-03-21T10:22:26.481 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:22:26 smithi100 ceph-mon[35193]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.0", "id": [4, 8]}]: dispatch 2024-03-21T10:22:26.481 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:22:26 smithi100 ceph-mon[35193]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.6", "id": [3, 7]}]: dispatch 2024-03-21T10:22:26.481 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:22:26 smithi100 ceph-mon[35193]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1b", "id": [4, 6]}]: dispatch 2024-03-21T10:22:26.481 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:22:26 smithi100 ceph-mon[35193]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.33", "id": [5, 9]}]: dispatch 2024-03-21T10:22:26.481 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:22:26 smithi100 ceph-mon[35193]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3c", "id": [4, 9]}]: dispatch 2024-03-21T10:22:26.490 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:22:26 smithi157 ceph-mon[35581]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-03-21T10:22:26.490 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:22:26 smithi157 ceph-mon[35581]: from='client.? 172.21.15.47:0/87746632' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]': finished 2024-03-21T10:22:26.490 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:22:26 smithi157 ceph-mon[35581]: daemon mds.h assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-03-21T10:22:26.491 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:22:26 smithi157 ceph-mon[35581]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-03-21T10:22:26.491 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:22:26 smithi157 ceph-mon[35581]: fsmap cephfs:2 {0=b=up:active,1=h=up:creating} 10 up:standby 2024-03-21T10:22:26.491 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:22:26 smithi157 ceph-mon[35581]: daemon mds.h is now active in filesystem cephfs as rank 1 2024-03-21T10:22:26.491 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:22:26 smithi157 ceph-mon[35581]: from='client.? 172.21.15.47:0/1030923006' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-03-21T10:22:26.491 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:22:26 smithi157 ceph-mon[35581]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.11", "id": [7, 11]}]: dispatch 2024-03-21T10:22:26.491 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:22:26 smithi157 ceph-mon[35581]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.14", "id": [2, 4]}]: dispatch 2024-03-21T10:22:26.491 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:22:26 smithi157 ceph-mon[35581]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1a", "id": [8, 1]}]: dispatch 2024-03-21T10:22:26.491 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:22:26 smithi157 ceph-mon[35581]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1b", "id": [2, 4]}]: dispatch 2024-03-21T10:22:26.491 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:22:26 smithi157 ceph-mon[35581]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1d", "id": [8, 11]}]: dispatch 2024-03-21T10:22:26.491 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:22:26 smithi157 ceph-mon[35581]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.0", "id": [4, 8]}]: dispatch 2024-03-21T10:22:26.491 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:22:26 smithi157 ceph-mon[35581]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.6", "id": [3, 7]}]: dispatch 2024-03-21T10:22:26.491 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:22:26 smithi157 ceph-mon[35581]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1b", "id": [4, 6]}]: dispatch 2024-03-21T10:22:26.491 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:22:26 smithi157 ceph-mon[35581]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.33", "id": [5, 9]}]: dispatch 2024-03-21T10:22:26.491 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:22:26 smithi157 ceph-mon[35581]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3c", "id": [4, 9]}]: dispatch 2024-03-21T10:22:26.662 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:26.661+0000 7fd64e400640 1 -- 172.21.15.47:0/1492577017 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"} v 0) v1 -- 0x7fd6480622a0 con 0x7fd648074100 2024-03-21T10:22:27.198 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:27.196+0000 7fd646000640 1 -- 172.21.15.47:0/1492577017 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]=0 v23) v1 ==== 115+0+0 (secure 0 0 0) 0x7fd63809bec0 con 0x7fd648074100 2024-03-21T10:22:27.201 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:27.200+0000 7fd64e400640 1 -- 172.21.15.47:0/1492577017 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7fd62c0a2f20 msgr2=0x7fd62c0a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:22:27.202 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:27.200+0000 7fd64e400640 1 --2- 172.21.15.47:0/1492577017 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7fd62c0a2f20 0x7fd62c0a53e0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fd630000c00 tx=0x7fd63002c7e0 comp rx=0 tx=0).stop 2024-03-21T10:22:27.202 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:27.201+0000 7fd64e400640 1 -- 172.21.15.47:0/1492577017 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd648074100 msgr2=0x7fd6481999a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:22:27.202 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:27.201+0000 7fd64e400640 1 --2- 172.21.15.47:0/1492577017 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd648074100 0x7fd6481999a0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fd63800d950 tx=0x7fd63800de10 comp rx=0 tx=0).stop 2024-03-21T10:22:27.204 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:27.201+0000 7fd64e400640 1 -- 172.21.15.47:0/1492577017 shutdown_connections 2024-03-21T10:22:27.204 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:27.201+0000 7fd64e400640 1 --2- 172.21.15.47:0/1492577017 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7fd62c0a2f20 0x7fd62c0a53e0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:22:27.204 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:27.201+0000 7fd64e400640 1 --2- 172.21.15.47:0/1492577017 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd648109460 0x7fd6481a0f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:22:27.204 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:27.201+0000 7fd64e400640 1 --2- 172.21.15.47:0/1492577017 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd6480756e0 0x7fd648199ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:22:27.204 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:27.201+0000 7fd64e400640 1 --2- 172.21.15.47:0/1492577017 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd648074100 0x7fd6481999a0 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:22:27.205 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:27.201+0000 7fd64e400640 1 -- 172.21.15.47:0/1492577017 >> 172.21.15.47:0/1492577017 conn(0x7fd6480fcb10 msgr2=0x7fd648105360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:22:27.205 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:27.202+0000 7fd64e400640 1 -- 172.21.15.47:0/1492577017 shutdown_connections 2024-03-21T10:22:27.205 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:27.202+0000 7fd64e400640 1 -- 172.21.15.47:0/1492577017 wait complete. 2024-03-21T10:22:27.231 DEBUG:tasks.cephfs.filesystem:Creating {'create': 2, 'subvol_options': '--namespace-isolated'} subvols for filesystem 'cephfs' 2024-03-21T10:22:27.232 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolumegroup create cephfs qa 2024-03-21T10:22:27.445 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:27.444+0000 7fd662400640 1 -- 172.21.15.47:0/3063652875 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd65c109890 msgr2=0x7fd65c110120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:22:27.446 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:27.444+0000 7fd662400640 1 --2- 172.21.15.47:0/3063652875 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd65c109890 0x7fd65c110120 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fd654009ca0 tx=0x7fd65402c100 comp rx=0 tx=0).stop 2024-03-21T10:22:27.446 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:27.444+0000 7fd662400640 1 -- 172.21.15.47:0/3063652875 shutdown_connections 2024-03-21T10:22:27.446 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:27.444+0000 7fd662400640 1 --2- 172.21.15.47:0/3063652875 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd65c109890 0x7fd65c110120 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:22:27.446 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:27.444+0000 7fd662400640 1 --2- 172.21.15.47:0/3063652875 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd65c108ed0 0x7fd65c109350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:22:27.446 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:27.444+0000 7fd662400640 1 --2- 172.21.15.47:0/3063652875 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd65c107ce0 0x7fd65c108100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:22:27.446 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:27.444+0000 7fd662400640 1 -- 172.21.15.47:0/3063652875 >> 172.21.15.47:0/3063652875 conn(0x7fd65c076c90 msgr2=0x7fd65c0790b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:22:27.446 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:27.444+0000 7fd662400640 1 -- 172.21.15.47:0/3063652875 shutdown_connections 2024-03-21T10:22:27.446 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:27.444+0000 7fd662400640 1 -- 172.21.15.47:0/3063652875 wait complete. 2024-03-21T10:22:27.448 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:27.445+0000 7fd662400640 1 Processor -- start 2024-03-21T10:22:27.448 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:27.445+0000 7fd662400640 1 -- start start 2024-03-21T10:22:27.448 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:27.446+0000 7fd662400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd65c107ce0 0x7fd65c19ddd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:22:27.448 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:27.446+0000 7fd662400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd65c108ed0 0x7fd65c19e310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:22:27.449 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:27.446+0000 7fd662400640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd65c109890 0x7fd65c1a5380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:22:27.449 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:27.446+0000 7fd662400640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd65c1137f0 con 0x7fd65c107ce0 2024-03-21T10:22:27.449 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:27.446+0000 7fd662400640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd65c113670 con 0x7fd65c108ed0 2024-03-21T10:22:27.449 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:27.446+0000 7fd662400640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd65c113970 con 0x7fd65c109890 2024-03-21T10:22:27.449 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:27.446+0000 7fd65b400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd65c107ce0 0x7fd65c19ddd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:22:27.449 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:27.446+0000 7fd65b400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd65c107ce0 0x7fd65c19ddd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:56548/0 (socket says 172.21.15.47:56548) 2024-03-21T10:22:27.449 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:27.446+0000 7fd65b400640 1 -- 172.21.15.47:0/2950717752 learned_addr learned my addr 172.21.15.47:0/2950717752 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:22:27.449 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:27.446+0000 7fd653e00640 1 --2- 172.21.15.47:0/2950717752 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd65c108ed0 0x7fd65c19e310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:22:27.449 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:27.446+0000 7fd65be00640 1 --2- 172.21.15.47:0/2950717752 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd65c109890 0x7fd65c1a5380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:22:27.449 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:27.446+0000 7fd65b400640 1 -- 172.21.15.47:0/2950717752 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd65c109890 msgr2=0x7fd65c1a5380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:22:27.449 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:27.446+0000 7fd65b400640 1 --2- 172.21.15.47:0/2950717752 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd65c109890 0x7fd65c1a5380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:22:27.449 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:27.446+0000 7fd65b400640 1 -- 172.21.15.47:0/2950717752 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd65c108ed0 msgr2=0x7fd65c19e310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:22:27.449 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:27.446+0000 7fd65b400640 1 --2- 172.21.15.47:0/2950717752 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd65c108ed0 0x7fd65c19e310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:22:27.449 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:27.446+0000 7fd65b400640 1 -- 172.21.15.47:0/2950717752 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd65c1a59f0 con 0x7fd65c107ce0 2024-03-21T10:22:27.449 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:27.447+0000 7fd65b400640 1 --2- 172.21.15.47:0/2950717752 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd65c107ce0 0x7fd65c19ddd0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fd64c00c970 tx=0x7fd64c00ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:22:27.450 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:27.447+0000 7fd658c00640 1 -- 172.21.15.47:0/2950717752 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd64c007c10 con 0x7fd65c107ce0 2024-03-21T10:22:27.450 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:27.447+0000 7fd658c00640 1 -- 172.21.15.47:0/2950717752 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fd64c007db0 con 0x7fd65c107ce0 2024-03-21T10:22:27.450 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:27.447+0000 7fd658c00640 1 -- 172.21.15.47:0/2950717752 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd64c00f7d0 con 0x7fd65c107ce0 2024-03-21T10:22:27.450 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:27.447+0000 7fd662400640 1 -- 172.21.15.47:0/2950717752 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd65c1a5ce0 con 0x7fd65c107ce0 2024-03-21T10:22:27.450 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:27.447+0000 7fd662400640 1 -- 172.21.15.47:0/2950717752 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd65c1a6180 con 0x7fd65c107ce0 2024-03-21T10:22:27.450 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:27.448+0000 7fd662400640 1 -- 172.21.15.47:0/2950717752 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd65c10f000 con 0x7fd65c107ce0 2024-03-21T10:22:27.451 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:27.450+0000 7fd658c00640 1 -- 172.21.15.47:0/2950717752 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 16) v1 ==== 137854+0+0 (secure 0 0 0) 0x7fd64c018480 con 0x7fd65c107ce0 2024-03-21T10:22:27.453 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:27.452+0000 7fd658c00640 1 --2- 172.21.15.47:0/2950717752 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7fd6400a2fa0 0x7fd6400a5460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:22:27.453 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:27.452+0000 7fd658c00640 1 -- 172.21.15.47:0/2950717752 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9092+0+0 (secure 0 0 0) 0x7fd64c0cfac0 con 0x7fd65c107ce0 2024-03-21T10:22:27.453 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:27.452+0000 7fd653e00640 1 --2- 172.21.15.47:0/2950717752 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7fd6400a2fa0 0x7fd6400a5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:22:27.454 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:27.453+0000 7fd653e00640 1 --2- 172.21.15.47:0/2950717752 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7fd6400a2fa0 0x7fd6400a5460 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fd65c19f2e0 tx=0x7fd644005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:22:27.457 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:27.456+0000 7fd658c00640 1 -- 172.21.15.47:0/2950717752 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fd64c09c620 con 0x7fd65c107ce0 2024-03-21T10:22:27.463 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:22:27 smithi047 ceph-mon[31547]: pgmap v381: 129 pgs: 129 active+clean; 581 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2024-03-21T10:22:27.464 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:22:27 smithi047 ceph-mon[31547]: from='client.? 172.21.15.47:0/1030923006' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-03-21T10:22:27.464 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:22:27 smithi047 ceph-mon[31547]: mds.? [v2:172.21.15.100:6836/3497644162,v1:172.21.15.100:6837/3497644162] up:active 2024-03-21T10:22:27.464 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:22:27 smithi047 ceph-mon[31547]: daemon mds.e assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-03-21T10:22:27.464 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:22:27 smithi047 ceph-mon[31547]: fsmap cephfs:2 {0=b=up:active,1=h=up:active} 10 up:standby 2024-03-21T10:22:27.464 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:22:27 smithi047 ceph-mon[31547]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.11", "id": [7, 11]}]': finished 2024-03-21T10:22:27.464 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:22:27 smithi047 ceph-mon[31547]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.14", "id": [2, 4]}]': finished 2024-03-21T10:22:27.464 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:22:27 smithi047 ceph-mon[31547]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1a", "id": [8, 1]}]': finished 2024-03-21T10:22:27.464 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:22:27 smithi047 ceph-mon[31547]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1b", "id": [2, 4]}]': finished 2024-03-21T10:22:27.464 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:22:27 smithi047 ceph-mon[31547]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1d", "id": [8, 11]}]': finished 2024-03-21T10:22:27.464 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:22:27 smithi047 ceph-mon[31547]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.0", "id": [4, 8]}]': finished 2024-03-21T10:22:27.464 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:22:27 smithi047 ceph-mon[31547]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.6", "id": [3, 7]}]': finished 2024-03-21T10:22:27.464 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:22:27 smithi047 ceph-mon[31547]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1b", "id": [4, 6]}]': finished 2024-03-21T10:22:27.464 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:22:27 smithi047 ceph-mon[31547]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.33", "id": [5, 9]}]': finished 2024-03-21T10:22:27.464 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:22:27 smithi047 ceph-mon[31547]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3c", "id": [4, 9]}]': finished 2024-03-21T10:22:27.465 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:22:27 smithi047 ceph-mon[31547]: osdmap e92: 12 total, 12 up, 12 in 2024-03-21T10:22:27.465 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:22:27 smithi047 ceph-mon[31547]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=e=up:creating} 9 up:standby 2024-03-21T10:22:27.465 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:22:27 smithi047 ceph-mon[31547]: from='client.? 172.21.15.47:0/1492577017' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-03-21T10:22:27.480 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:22:27 smithi100 ceph-mon[35193]: pgmap v381: 129 pgs: 129 active+clean; 581 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2024-03-21T10:22:27.480 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:22:27 smithi100 ceph-mon[35193]: from='client.? 172.21.15.47:0/1030923006' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-03-21T10:22:27.480 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:22:27 smithi100 ceph-mon[35193]: mds.? [v2:172.21.15.100:6836/3497644162,v1:172.21.15.100:6837/3497644162] up:active 2024-03-21T10:22:27.480 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:22:27 smithi100 ceph-mon[35193]: daemon mds.e assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-03-21T10:22:27.480 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:22:27 smithi100 ceph-mon[35193]: fsmap cephfs:2 {0=b=up:active,1=h=up:active} 10 up:standby 2024-03-21T10:22:27.480 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:22:27 smithi100 ceph-mon[35193]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.11", "id": [7, 11]}]': finished 2024-03-21T10:22:27.480 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:22:27 smithi100 ceph-mon[35193]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.14", "id": [2, 4]}]': finished 2024-03-21T10:22:27.480 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:22:27 smithi100 ceph-mon[35193]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1a", "id": [8, 1]}]': finished 2024-03-21T10:22:27.480 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:22:27 smithi100 ceph-mon[35193]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1b", "id": [2, 4]}]': finished 2024-03-21T10:22:27.480 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:22:27 smithi100 ceph-mon[35193]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1d", "id": [8, 11]}]': finished 2024-03-21T10:22:27.481 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:22:27 smithi100 ceph-mon[35193]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.0", "id": [4, 8]}]': finished 2024-03-21T10:22:27.481 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:22:27 smithi100 ceph-mon[35193]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.6", "id": [3, 7]}]': finished 2024-03-21T10:22:27.481 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:22:27 smithi100 ceph-mon[35193]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1b", "id": [4, 6]}]': finished 2024-03-21T10:22:27.481 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:22:27 smithi100 ceph-mon[35193]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.33", "id": [5, 9]}]': finished 2024-03-21T10:22:27.481 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:22:27 smithi100 ceph-mon[35193]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3c", "id": [4, 9]}]': finished 2024-03-21T10:22:27.481 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:22:27 smithi100 ceph-mon[35193]: osdmap e92: 12 total, 12 up, 12 in 2024-03-21T10:22:27.481 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:22:27 smithi100 ceph-mon[35193]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=e=up:creating} 9 up:standby 2024-03-21T10:22:27.481 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:22:27 smithi100 ceph-mon[35193]: from='client.? 172.21.15.47:0/1492577017' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-03-21T10:22:27.490 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:22:27 smithi157 ceph-mon[35581]: pgmap v381: 129 pgs: 129 active+clean; 581 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2024-03-21T10:22:27.491 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:22:27 smithi157 ceph-mon[35581]: from='client.? 172.21.15.47:0/1030923006' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-03-21T10:22:27.491 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:22:27 smithi157 ceph-mon[35581]: mds.? [v2:172.21.15.100:6836/3497644162,v1:172.21.15.100:6837/3497644162] up:active 2024-03-21T10:22:27.491 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:22:27 smithi157 ceph-mon[35581]: daemon mds.e assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-03-21T10:22:27.491 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:22:27 smithi157 ceph-mon[35581]: fsmap cephfs:2 {0=b=up:active,1=h=up:active} 10 up:standby 2024-03-21T10:22:27.491 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:22:27 smithi157 ceph-mon[35581]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.11", "id": [7, 11]}]': finished 2024-03-21T10:22:27.491 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:22:27 smithi157 ceph-mon[35581]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.14", "id": [2, 4]}]': finished 2024-03-21T10:22:27.491 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:22:27 smithi157 ceph-mon[35581]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1a", "id": [8, 1]}]': finished 2024-03-21T10:22:27.491 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:22:27 smithi157 ceph-mon[35581]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1b", "id": [2, 4]}]': finished 2024-03-21T10:22:27.491 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:22:27 smithi157 ceph-mon[35581]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1d", "id": [8, 11]}]': finished 2024-03-21T10:22:27.491 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:22:27 smithi157 ceph-mon[35581]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.0", "id": [4, 8]}]': finished 2024-03-21T10:22:27.491 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:22:27 smithi157 ceph-mon[35581]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.6", "id": [3, 7]}]': finished 2024-03-21T10:22:27.491 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:22:27 smithi157 ceph-mon[35581]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1b", "id": [4, 6]}]': finished 2024-03-21T10:22:27.491 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:22:27 smithi157 ceph-mon[35581]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.33", "id": [5, 9]}]': finished 2024-03-21T10:22:27.491 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:22:27 smithi157 ceph-mon[35581]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3c", "id": [4, 9]}]': finished 2024-03-21T10:22:27.492 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:22:27 smithi157 ceph-mon[35581]: osdmap e92: 12 total, 12 up, 12 in 2024-03-21T10:22:27.492 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:22:27 smithi157 ceph-mon[35581]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=e=up:creating} 9 up:standby 2024-03-21T10:22:27.492 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:22:27 smithi157 ceph-mon[35581]: from='client.? 172.21.15.47:0/1492577017' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-03-21T10:22:27.690 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:27.689+0000 7fd662400640 1 -- 172.21.15.47:0/2950717752 --> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}) v1 -- 0x7fd65c0624f0 con 0x7fd6400a2fa0 2024-03-21T10:22:27.965 INFO:journalctl@ceph.mgr.x.smithi047.stdout:Mar 21 10:22:27 smithi047 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-mgr-x[31773]: 2024-03-21T10:22:27.700+0000 7fc3b4c00700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-21T10:22:27.965 INFO:journalctl@ceph.mgr.x.smithi047.stdout:Mar 21 10:22:27 smithi047 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-mgr-x[31773]: 2024-03-21T10:22:27.700+0000 7fc3b4c00700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-21T10:22:27.965 INFO:journalctl@ceph.mgr.x.smithi047.stdout:Mar 21 10:22:27 smithi047 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-mgr-x[31773]: 2024-03-21T10:22:27.700+0000 7fc3b4c00700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-21T10:22:27.965 INFO:journalctl@ceph.mgr.x.smithi047.stdout:Mar 21 10:22:27 smithi047 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-mgr-x[31773]: 2024-03-21T10:22:27.700+0000 7fc3b4c00700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-21T10:22:27.965 INFO:journalctl@ceph.mgr.x.smithi047.stdout:Mar 21 10:22:27 smithi047 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-mgr-x[31773]: 2024-03-21T10:22:27.700+0000 7fc3b4c00700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-21T10:22:28.253 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:28.252+0000 7fd658c00640 1 -- 172.21.15.47:0/2950717752 <== mgr.14152 v2:172.21.15.47:6800/2823885994 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7fd65c0624f0 con 0x7fd6400a2fa0 2024-03-21T10:22:28.257 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:28.256+0000 7fd662400640 1 -- 172.21.15.47:0/2950717752 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7fd6400a2fa0 msgr2=0x7fd6400a5460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:22:28.257 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:28.256+0000 7fd662400640 1 --2- 172.21.15.47:0/2950717752 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7fd6400a2fa0 0x7fd6400a5460 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fd65c19f2e0 tx=0x7fd644005f90 comp rx=0 tx=0).stop 2024-03-21T10:22:28.258 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:28.256+0000 7fd662400640 1 -- 172.21.15.47:0/2950717752 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd65c107ce0 msgr2=0x7fd65c19ddd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:22:28.258 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:28.256+0000 7fd662400640 1 --2- 172.21.15.47:0/2950717752 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd65c107ce0 0x7fd65c19ddd0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fd64c00c970 tx=0x7fd64c00ce30 comp rx=0 tx=0).stop 2024-03-21T10:22:28.258 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:28.256+0000 7fd662400640 1 -- 172.21.15.47:0/2950717752 shutdown_connections 2024-03-21T10:22:28.258 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:28.256+0000 7fd662400640 1 --2- 172.21.15.47:0/2950717752 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7fd6400a2fa0 0x7fd6400a5460 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:22:28.258 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:28.256+0000 7fd662400640 1 --2- 172.21.15.47:0/2950717752 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd65c109890 0x7fd65c1a5380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:22:28.258 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:28.256+0000 7fd662400640 1 --2- 172.21.15.47:0/2950717752 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd65c108ed0 0x7fd65c19e310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:22:28.258 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:28.256+0000 7fd662400640 1 --2- 172.21.15.47:0/2950717752 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd65c107ce0 0x7fd65c19ddd0 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:22:28.258 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:28.256+0000 7fd662400640 1 -- 172.21.15.47:0/2950717752 >> 172.21.15.47:0/2950717752 conn(0x7fd65c076c90 msgr2=0x7fd65c10c1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:22:28.258 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:28.256+0000 7fd662400640 1 -- 172.21.15.47:0/2950717752 shutdown_connections 2024-03-21T10:22:28.258 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:28.256+0000 7fd662400640 1 -- 172.21.15.47:0/2950717752 wait complete. 2024-03-21T10:22:28.278 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume create cephfs sv_0 --group_name qa 2024-03-21T10:22:28.464 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:22:28 smithi047 ceph-mon[31547]: from='client.? 172.21.15.47:0/1492577017' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-03-21T10:22:28.464 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:22:28 smithi047 ceph-mon[31547]: osdmap e93: 12 total, 12 up, 12 in 2024-03-21T10:22:28.464 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:22:28 smithi047 ceph-mon[31547]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=e=up:creating} 9 up:standby 2024-03-21T10:22:28.464 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:22:28 smithi047 ceph-mon[31547]: daemon mds.e is now active in filesystem cephfs as rank 2 2024-03-21T10:22:28.464 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:22:28 smithi047 ceph-mon[31547]: from='client.14868 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:22:28.479 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:22:28 smithi100 ceph-mon[35193]: from='client.? 172.21.15.47:0/1492577017' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-03-21T10:22:28.479 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:22:28 smithi100 ceph-mon[35193]: osdmap e93: 12 total, 12 up, 12 in 2024-03-21T10:22:28.480 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:22:28 smithi100 ceph-mon[35193]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=e=up:creating} 9 up:standby 2024-03-21T10:22:28.480 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:22:28 smithi100 ceph-mon[35193]: daemon mds.e is now active in filesystem cephfs as rank 2 2024-03-21T10:22:28.480 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:22:28 smithi100 ceph-mon[35193]: from='client.14868 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:22:28.486 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:28.485+0000 7f85b2000640 1 -- 172.21.15.47:0/318216169 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f85ac109870 msgr2=0x7f85ac110100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:22:28.486 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:28.485+0000 7f85b2000640 1 --2- 172.21.15.47:0/318216169 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f85ac109870 0x7f85ac110100 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f85a4009ca0 tx=0x7f85a402c0f0 comp rx=0 tx=0).stop 2024-03-21T10:22:28.486 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:28.485+0000 7f85b2000640 1 -- 172.21.15.47:0/318216169 shutdown_connections 2024-03-21T10:22:28.486 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:28.485+0000 7f85b2000640 1 --2- 172.21.15.47:0/318216169 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f85ac109870 0x7f85ac110100 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:22:28.487 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:28.485+0000 7f85b2000640 1 --2- 172.21.15.47:0/318216169 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f85ac108eb0 0x7f85ac109330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:22:28.487 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:28.485+0000 7f85b2000640 1 --2- 172.21.15.47:0/318216169 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f85ac107cc0 0x7f85ac1080e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:22:28.487 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:28.485+0000 7f85b2000640 1 -- 172.21.15.47:0/318216169 >> 172.21.15.47:0/318216169 conn(0x7f85ac076c60 msgr2=0x7f85ac0790a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:22:28.487 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:28.485+0000 7f85b2000640 1 -- 172.21.15.47:0/318216169 shutdown_connections 2024-03-21T10:22:28.487 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:28.485+0000 7f85b2000640 1 -- 172.21.15.47:0/318216169 wait complete. 2024-03-21T10:22:28.489 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:28.486+0000 7f85b2000640 1 Processor -- start 2024-03-21T10:22:28.489 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:28.486+0000 7f85b2000640 1 -- start start 2024-03-21T10:22:28.489 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:28.486+0000 7f85b2000640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f85ac107cc0 0x7f85ac19de60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:22:28.489 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:28.486+0000 7f85b2000640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f85ac108eb0 0x7f85ac19e3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:22:28.489 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:28.486+0000 7f85b2000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f85ac109870 0x7f85ac1a5410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:22:28.489 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:28.486+0000 7f85b2000640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85ac1137f0 con 0x7f85ac107cc0 2024-03-21T10:22:28.490 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:28.486+0000 7f85b2000640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85ac113670 con 0x7f85ac109870 2024-03-21T10:22:28.490 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:28.486+0000 7f85b2000640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85ac113970 con 0x7f85ac108eb0 2024-03-21T10:22:28.490 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:28.487+0000 7f85ab400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f85ac107cc0 0x7f85ac19de60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:22:28.490 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:28.487+0000 7f85ab400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f85ac107cc0 0x7f85ac19de60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:56592/0 (socket says 172.21.15.47:56592) 2024-03-21T10:22:28.490 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:28.487+0000 7f85ab400640 1 -- 172.21.15.47:0/1716424933 learned_addr learned my addr 172.21.15.47:0/1716424933 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:22:28.490 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:28.487+0000 7f85abe00640 1 --2- 172.21.15.47:0/1716424933 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f85ac109870 0x7f85ac1a5410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:22:28.490 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:28.487+0000 7f85aaa00640 1 --2- 172.21.15.47:0/1716424933 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f85ac108eb0 0x7f85ac19e3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:22:28.490 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:28.487+0000 7f85ab400640 1 -- 172.21.15.47:0/1716424933 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f85ac108eb0 msgr2=0x7f85ac19e3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:22:28.490 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:28.487+0000 7f85ab400640 1 --2- 172.21.15.47:0/1716424933 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f85ac108eb0 0x7f85ac19e3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:22:28.490 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:28.487+0000 7f85ab400640 1 -- 172.21.15.47:0/1716424933 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f85ac109870 msgr2=0x7f85ac1a5410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:22:28.490 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:28.487+0000 7f85ab400640 1 --2- 172.21.15.47:0/1716424933 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f85ac109870 0x7f85ac1a5410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:22:28.490 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:28.487+0000 7f85ab400640 1 -- 172.21.15.47:0/1716424933 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f85ac1a5a80 con 0x7f85ac107cc0 2024-03-21T10:22:28.490 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:28.487+0000 7f85aaa00640 1 --2- 172.21.15.47:0/1716424933 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f85ac108eb0 0x7f85ac19e3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T10:22:28.491 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:28.487+0000 7f85abe00640 1 --2- 172.21.15.47:0/1716424933 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f85ac109870 0x7f85ac1a5410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T10:22:28.491 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:28.487+0000 7f85ab400640 1 --2- 172.21.15.47:0/1716424933 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f85ac107cc0 0x7f85ac19de60 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f859c00e9c0 tx=0x7f859c00ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:22:28.491 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:28.488+0000 7f85a3e00640 1 -- 172.21.15.47:0/1716424933 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f859c00cde0 con 0x7f85ac107cc0 2024-03-21T10:22:28.491 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:28.488+0000 7f85a3e00640 1 -- 172.21.15.47:0/1716424933 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f859c0108e0 con 0x7f85ac107cc0 2024-03-21T10:22:28.491 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:28.488+0000 7f85b2000640 1 -- 172.21.15.47:0/1716424933 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f85ac1a5d70 con 0x7f85ac107cc0 2024-03-21T10:22:28.491 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:28.488+0000 7f85a3e00640 1 -- 172.21.15.47:0/1716424933 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f859c018990 con 0x7f85ac107cc0 2024-03-21T10:22:28.491 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:28.488+0000 7f85b2000640 1 -- 172.21.15.47:0/1716424933 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f85ac1a6200 con 0x7f85ac107cc0 2024-03-21T10:22:28.492 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:22:28 smithi157 ceph-mon[35581]: from='client.? 172.21.15.47:0/1492577017' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-03-21T10:22:28.492 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:22:28 smithi157 ceph-mon[35581]: osdmap e93: 12 total, 12 up, 12 in 2024-03-21T10:22:28.492 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:22:28 smithi157 ceph-mon[35581]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=e=up:creating} 9 up:standby 2024-03-21T10:22:28.492 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:22:28 smithi157 ceph-mon[35581]: daemon mds.e is now active in filesystem cephfs as rank 2 2024-03-21T10:22:28.492 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:22:28 smithi157 ceph-mon[35581]: from='client.14868 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:22:28.492 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:28.489+0000 7f85b2000640 1 -- 172.21.15.47:0/1716424933 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f85ac10efe0 con 0x7f85ac107cc0 2024-03-21T10:22:28.492 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:28.491+0000 7f85a3e00640 1 -- 172.21.15.47:0/1716424933 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 16) v1 ==== 137854+0+0 (secure 0 0 0) 0x7f859c01f020 con 0x7f85ac107cc0 2024-03-21T10:22:28.493 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:28.492+0000 7f85a3e00640 1 --2- 172.21.15.47:0/1716424933 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f85900a2f20 0x7f85900a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:22:28.493 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:28.492+0000 7f85a3e00640 1 -- 172.21.15.47:0/1716424933 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9092+0+0 (secure 0 0 0) 0x7f859c0cf960 con 0x7f85ac107cc0 2024-03-21T10:22:28.494 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:28.493+0000 7f85aaa00640 1 --2- 172.21.15.47:0/1716424933 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f85900a2f20 0x7f85900a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:22:28.494 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:28.493+0000 7f85aaa00640 1 --2- 172.21.15.47:0/1716424933 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f85900a2f20 0x7f85900a53e0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f85ac19f370 tx=0x7f8598005e50 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:22:28.498 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:28.497+0000 7f85a3e00640 1 -- 172.21.15.47:0/1716424933 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f859c09c430 con 0x7f85ac107cc0 2024-03-21T10:22:28.743 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:28.742+0000 7f85b2000640 1 -- 172.21.15.47:0/1716424933 --> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}) v1 -- 0x7f85ac061b70 con 0x7f85900a2f20 2024-03-21T10:22:28.778 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:28.777+0000 7f85a3e00640 1 -- 172.21.15.47:0/1716424933 <== mgr.14152 v2:172.21.15.47:6800/2823885994 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f85ac061b70 con 0x7f85900a2f20 2024-03-21T10:22:28.781 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:28.780+0000 7f85b2000640 1 -- 172.21.15.47:0/1716424933 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f85900a2f20 msgr2=0x7f85900a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:22:28.781 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:28.780+0000 7f85b2000640 1 --2- 172.21.15.47:0/1716424933 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f85900a2f20 0x7f85900a53e0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f85ac19f370 tx=0x7f8598005e50 comp rx=0 tx=0).stop 2024-03-21T10:22:28.781 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:28.780+0000 7f85b2000640 1 -- 172.21.15.47:0/1716424933 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f85ac107cc0 msgr2=0x7f85ac19de60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:22:28.781 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:28.780+0000 7f85b2000640 1 --2- 172.21.15.47:0/1716424933 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f85ac107cc0 0x7f85ac19de60 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f859c00e9c0 tx=0x7f859c00ee80 comp rx=0 tx=0).stop 2024-03-21T10:22:28.782 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:28.780+0000 7f85b2000640 1 -- 172.21.15.47:0/1716424933 shutdown_connections 2024-03-21T10:22:28.782 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:28.780+0000 7f85b2000640 1 --2- 172.21.15.47:0/1716424933 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f85900a2f20 0x7f85900a53e0 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:22:28.782 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:28.780+0000 7f85b2000640 1 --2- 172.21.15.47:0/1716424933 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f85ac109870 0x7f85ac1a5410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:22:28.782 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:28.780+0000 7f85b2000640 1 --2- 172.21.15.47:0/1716424933 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f85ac108eb0 0x7f85ac19e3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:22:28.782 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:28.780+0000 7f85b2000640 1 --2- 172.21.15.47:0/1716424933 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f85ac107cc0 0x7f85ac19de60 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:22:28.782 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:28.780+0000 7f85b2000640 1 -- 172.21.15.47:0/1716424933 >> 172.21.15.47:0/1716424933 conn(0x7f85ac076c60 msgr2=0x7f85ac10c1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:22:28.782 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:28.780+0000 7f85b2000640 1 -- 172.21.15.47:0/1716424933 shutdown_connections 2024-03-21T10:22:28.782 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:28.780+0000 7f85b2000640 1 -- 172.21.15.47:0/1716424933 wait complete. 2024-03-21T10:22:28.800 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs --group_name qa sv_0 2024-03-21T10:22:29.026 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:29.024+0000 7f8365a00640 1 -- 172.21.15.47:0/1774210648 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f83601037c0 msgr2=0x7f8360103be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:22:29.026 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:29.024+0000 7f8365a00640 1 --2- 172.21.15.47:0/1774210648 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f83601037c0 0x7f8360103be0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f8350009900 tx=0x7f835002bc30 comp rx=0 tx=0).stop 2024-03-21T10:22:29.026 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:29.025+0000 7f8365a00640 1 -- 172.21.15.47:0/1774210648 shutdown_connections 2024-03-21T10:22:29.026 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:29.025+0000 7f8365a00640 1 --2- 172.21.15.47:0/1774210648 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8360105370 0x7f836010bc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:22:29.026 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:29.025+0000 7f8365a00640 1 --2- 172.21.15.47:0/1774210648 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f83601049b0 0x7f8360104e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:22:29.026 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:29.025+0000 7f8365a00640 1 --2- 172.21.15.47:0/1774210648 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f83601037c0 0x7f8360103be0 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:22:29.027 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:29.025+0000 7f8365a00640 1 -- 172.21.15.47:0/1774210648 >> 172.21.15.47:0/1774210648 conn(0x7f83600fed50 msgr2=0x7f8360101170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:22:29.027 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:29.025+0000 7f8365a00640 1 -- 172.21.15.47:0/1774210648 shutdown_connections 2024-03-21T10:22:29.027 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:29.025+0000 7f8365a00640 1 -- 172.21.15.47:0/1774210648 wait complete. 2024-03-21T10:22:29.028 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:29.026+0000 7f8365a00640 1 Processor -- start 2024-03-21T10:22:29.028 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:29.026+0000 7f8365a00640 1 -- start start 2024-03-21T10:22:29.028 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:29.026+0000 7f8365a00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f83601037c0 0x7f8360199a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:22:29.028 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:29.026+0000 7f8365a00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f83601049b0 0x7f8360199fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:22:29.028 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:29.026+0000 7f8365a00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8360105370 0x7f83601a1010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:22:29.028 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:29.026+0000 7f8365a00640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f836010f390 con 0x7f8360105370 2024-03-21T10:22:29.029 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:29.026+0000 7f8365a00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f836010f210 con 0x7f83601049b0 2024-03-21T10:22:29.029 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:29.026+0000 7f8365a00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f836010f510 con 0x7f83601037c0 2024-03-21T10:22:29.029 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:29.026+0000 7f8365000640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8360105370 0x7f83601a1010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:22:29.029 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:29.027+0000 7f8365000640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8360105370 0x7f83601a1010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:56602/0 (socket says 172.21.15.47:56602) 2024-03-21T10:22:29.029 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:29.027+0000 7f8357e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f83601049b0 0x7f8360199fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:22:29.029 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:29.027+0000 7f8365000640 1 -- 172.21.15.47:0/1697084928 learned_addr learned my addr 172.21.15.47:0/1697084928 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:22:29.029 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:29.027+0000 7f835fe00640 1 --2- 172.21.15.47:0/1697084928 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f83601037c0 0x7f8360199a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:22:29.029 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:29.027+0000 7f8365000640 1 -- 172.21.15.47:0/1697084928 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f83601037c0 msgr2=0x7f8360199a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:22:29.029 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:29.027+0000 7f8365000640 1 --2- 172.21.15.47:0/1697084928 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f83601037c0 0x7f8360199a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:22:29.029 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:29.027+0000 7f8365000640 1 -- 172.21.15.47:0/1697084928 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f83601049b0 msgr2=0x7f8360199fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:22:29.029 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:29.027+0000 7f8365000640 1 --2- 172.21.15.47:0/1697084928 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f83601049b0 0x7f8360199fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:22:29.029 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:29.027+0000 7f8365000640 1 -- 172.21.15.47:0/1697084928 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f83601a1680 con 0x7f8360105370 2024-03-21T10:22:29.029 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:29.027+0000 7f8365000640 1 --2- 172.21.15.47:0/1697084928 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8360105370 0x7f83601a1010 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f835800eb10 tx=0x7f835800ef10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:22:29.030 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:29.028+0000 7f835d600640 1 -- 172.21.15.47:0/1697084928 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f835800ce20 con 0x7f8360105370 2024-03-21T10:22:29.030 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:29.028+0000 7f835d600640 1 -- 172.21.15.47:0/1697084928 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f8358005020 con 0x7f8360105370 2024-03-21T10:22:29.030 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:29.028+0000 7f8365a00640 1 -- 172.21.15.47:0/1697084928 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f83601a1970 con 0x7f8360105370 2024-03-21T10:22:29.030 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:29.028+0000 7f835d600640 1 -- 172.21.15.47:0/1697084928 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8358010690 con 0x7f8360105370 2024-03-21T10:22:29.030 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:29.028+0000 7f8365a00640 1 -- 172.21.15.47:0/1697084928 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f83601a1ef0 con 0x7f8360105370 2024-03-21T10:22:29.030 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:29.029+0000 7f8365a00640 1 -- 172.21.15.47:0/1697084928 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f836010abb0 con 0x7f8360105370 2024-03-21T10:22:29.032 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:29.031+0000 7f835d600640 1 -- 172.21.15.47:0/1697084928 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 16) v1 ==== 137854+0+0 (secure 0 0 0) 0x7f83580051d0 con 0x7f8360105370 2024-03-21T10:22:29.033 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:29.032+0000 7f835d600640 1 --2- 172.21.15.47:0/1697084928 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f83440a2f20 0x7f83440a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:22:29.034 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:29.033+0000 7f835d600640 1 -- 172.21.15.47:0/1697084928 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9092+0+0 (secure 0 0 0) 0x7f83580cfd60 con 0x7f8360105370 2024-03-21T10:22:29.034 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:29.033+0000 7f835fe00640 1 --2- 172.21.15.47:0/1697084928 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f83440a2f20 0x7f83440a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:22:29.034 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:29.034+0000 7f835fe00640 1 --2- 172.21.15.47:0/1697084928 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f83440a2f20 0x7f83440a53e0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f835002c240 tx=0x7f8350005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:22:29.039 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:29.038+0000 7f835d600640 1 -- 172.21.15.47:0/1697084928 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f835809c830 con 0x7f8360105370 2024-03-21T10:22:29.213 INFO:journalctl@ceph.mds.b.smithi100.stdout:Mar 21 10:22:28 smithi100 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-mds-b[64996]: 2024-03-21T10:22:28.927+0000 7f0aad200700 -1 mds.pinger is_rank_lagging: rank=1 was never sent ping request. 2024-03-21T10:22:29.213 INFO:journalctl@ceph.mds.b.smithi100.stdout:Mar 21 10:22:28 smithi100 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-mds-b[64996]: 2024-03-21T10:22:28.927+0000 7f0aad200700 -1 mds.pinger is_rank_lagging: rank=2 was never sent ping request. 2024-03-21T10:22:29.213 INFO:journalctl@ceph.mds.b.smithi100.stdout:Mar 21 10:22:28 smithi100 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-mds-b[64996]: 2024-03-21T10:22:28.927+0000 7f0aad200700 -1 mds.pinger is_rank_lagging: rank=3 was never sent ping request. 2024-03-21T10:22:29.283 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:29.282+0000 7f8365a00640 1 -- 172.21.15.47:0/1697084928 --> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}) v1 -- 0x7f8360061ae0 con 0x7f83440a2f20 2024-03-21T10:22:29.293 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:29.291+0000 7f835d600640 1 -- 172.21.15.47:0/1697084928 <== mgr.14152 v2:172.21.15.47:6800/2823885994 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+54 (secure 0 0 0) 0x7f8360061ae0 con 0x7f83440a2f20 2024-03-21T10:22:29.293 INFO:teuthology.orchestra.run.smithi047.stdout:/volumes/qa/sv_0/2e602e66-0b27-4e4c-a210-4a1ed53947cf 2024-03-21T10:22:29.296 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:29.294+0000 7f8365a00640 1 -- 172.21.15.47:0/1697084928 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f83440a2f20 msgr2=0x7f83440a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:22:29.296 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:29.294+0000 7f8365a00640 1 --2- 172.21.15.47:0/1697084928 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f83440a2f20 0x7f83440a53e0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f835002c240 tx=0x7f8350005fd0 comp rx=0 tx=0).stop 2024-03-21T10:22:29.296 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:29.294+0000 7f8365a00640 1 -- 172.21.15.47:0/1697084928 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8360105370 msgr2=0x7f83601a1010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:22:29.296 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:29.294+0000 7f8365a00640 1 --2- 172.21.15.47:0/1697084928 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8360105370 0x7f83601a1010 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f835800eb10 tx=0x7f835800ef10 comp rx=0 tx=0).stop 2024-03-21T10:22:29.296 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:29.295+0000 7f8365a00640 1 -- 172.21.15.47:0/1697084928 shutdown_connections 2024-03-21T10:22:29.296 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:29.295+0000 7f8365a00640 1 --2- 172.21.15.47:0/1697084928 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f83440a2f20 0x7f83440a53e0 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:22:29.296 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:29.295+0000 7f8365a00640 1 --2- 172.21.15.47:0/1697084928 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8360105370 0x7f83601a1010 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:22:29.297 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:29.295+0000 7f8365a00640 1 --2- 172.21.15.47:0/1697084928 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f83601049b0 0x7f8360199fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:22:29.297 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:29.295+0000 7f8365a00640 1 --2- 172.21.15.47:0/1697084928 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f83601037c0 0x7f8360199a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:22:29.297 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:29.295+0000 7f8365a00640 1 -- 172.21.15.47:0/1697084928 >> 172.21.15.47:0/1697084928 conn(0x7f83600fed50 msgr2=0x7f8360107d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:22:29.297 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:29.295+0000 7f8365a00640 1 -- 172.21.15.47:0/1697084928 shutdown_connections 2024-03-21T10:22:29.297 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:29.295+0000 7f8365a00640 1 -- 172.21.15.47:0/1697084928 wait complete. 2024-03-21T10:22:29.315 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume create cephfs sv_1 --group_name qa 2024-03-21T10:22:29.464 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:22:29 smithi047 ceph-mon[31547]: pgmap v384: 129 pgs: 129 active+clean; 582 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 KiB/s wr, 4 op/s; 77 B/s, 0 objects/s recovering 2024-03-21T10:22:29.464 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:22:29 smithi047 ceph-mon[31547]: mds.? [v2:172.21.15.100:6834/2597849957,v1:172.21.15.100:6835/2597849957] up:active 2024-03-21T10:22:29.464 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:22:29 smithi047 ceph-mon[31547]: daemon mds.k assigned to filesystem cephfs as rank 3 (now has 4 ranks) 2024-03-21T10:22:29.465 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:22:29 smithi047 ceph-mon[31547]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=e=up:active} 9 up:standby 2024-03-21T10:22:29.465 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:22:29 smithi047 ceph-mon[31547]: fsmap cephfs:4 {0=b=up:active,1=h=up:active,2=e=up:active,3=k=up:creating} 8 up:standby 2024-03-21T10:22:29.465 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:22:29 smithi047 ceph-mon[31547]: daemon mds.k is now active in filesystem cephfs as rank 3 2024-03-21T10:22:29.465 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:22:29 smithi047 ceph-mon[31547]: mds.? [v2:172.21.15.100:6832/2723302038,v1:172.21.15.100:6833/2723302038] up:active 2024-03-21T10:22:29.465 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:22:29 smithi047 ceph-mon[31547]: daemon mds.a assigned to filesystem cephfs as rank 4 (now has 5 ranks) 2024-03-21T10:22:29.465 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:22:29 smithi047 ceph-mon[31547]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-03-21T10:22:29.465 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:22:29 smithi047 ceph-mon[31547]: Cluster is now healthy 2024-03-21T10:22:29.465 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:22:29 smithi047 ceph-mon[31547]: fsmap cephfs:4 {0=b=up:active,1=h=up:active,2=e=up:active,3=k=up:active} 4 up:standby-replay 4 up:standby 2024-03-21T10:22:29.465 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:22:29 smithi047 ceph-mon[31547]: fsmap cephfs:5 4 up:standby-replay 1 up:creating 3 up:standby 4 up:active 2024-03-21T10:22:29.465 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:22:29 smithi047 ceph-mon[31547]: daemon mds.a is now active in filesystem cephfs as rank 4 2024-03-21T10:22:29.465 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:22:29 smithi047 ceph-mon[31547]: from='client.14880 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:22:29.480 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:22:29 smithi100 ceph-mon[35193]: pgmap v384: 129 pgs: 129 active+clean; 582 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 KiB/s wr, 4 op/s; 77 B/s, 0 objects/s recovering 2024-03-21T10:22:29.480 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:22:29 smithi100 ceph-mon[35193]: mds.? [v2:172.21.15.100:6834/2597849957,v1:172.21.15.100:6835/2597849957] up:active 2024-03-21T10:22:29.480 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:22:29 smithi100 ceph-mon[35193]: daemon mds.k assigned to filesystem cephfs as rank 3 (now has 4 ranks) 2024-03-21T10:22:29.480 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:22:29 smithi100 ceph-mon[35193]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=e=up:active} 9 up:standby 2024-03-21T10:22:29.480 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:22:29 smithi100 ceph-mon[35193]: fsmap cephfs:4 {0=b=up:active,1=h=up:active,2=e=up:active,3=k=up:creating} 8 up:standby 2024-03-21T10:22:29.480 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:22:29 smithi100 ceph-mon[35193]: daemon mds.k is now active in filesystem cephfs as rank 3 2024-03-21T10:22:29.480 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:22:29 smithi100 ceph-mon[35193]: mds.? [v2:172.21.15.100:6832/2723302038,v1:172.21.15.100:6833/2723302038] up:active 2024-03-21T10:22:29.480 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:22:29 smithi100 ceph-mon[35193]: daemon mds.a assigned to filesystem cephfs as rank 4 (now has 5 ranks) 2024-03-21T10:22:29.480 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:22:29 smithi100 ceph-mon[35193]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-03-21T10:22:29.480 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:22:29 smithi100 ceph-mon[35193]: Cluster is now healthy 2024-03-21T10:22:29.481 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:22:29 smithi100 ceph-mon[35193]: fsmap cephfs:4 {0=b=up:active,1=h=up:active,2=e=up:active,3=k=up:active} 4 up:standby-replay 4 up:standby 2024-03-21T10:22:29.481 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:22:29 smithi100 ceph-mon[35193]: fsmap cephfs:5 4 up:standby-replay 1 up:creating 3 up:standby 4 up:active 2024-03-21T10:22:29.481 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:22:29 smithi100 ceph-mon[35193]: daemon mds.a is now active in filesystem cephfs as rank 4 2024-03-21T10:22:29.481 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:22:29 smithi100 ceph-mon[35193]: from='client.14880 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:22:29.490 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:22:29 smithi157 ceph-mon[35581]: pgmap v384: 129 pgs: 129 active+clean; 582 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 KiB/s wr, 4 op/s; 77 B/s, 0 objects/s recovering 2024-03-21T10:22:29.490 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:22:29 smithi157 ceph-mon[35581]: mds.? [v2:172.21.15.100:6834/2597849957,v1:172.21.15.100:6835/2597849957] up:active 2024-03-21T10:22:29.490 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:22:29 smithi157 ceph-mon[35581]: daemon mds.k assigned to filesystem cephfs as rank 3 (now has 4 ranks) 2024-03-21T10:22:29.490 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:22:29 smithi157 ceph-mon[35581]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=e=up:active} 9 up:standby 2024-03-21T10:22:29.490 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:22:29 smithi157 ceph-mon[35581]: fsmap cephfs:4 {0=b=up:active,1=h=up:active,2=e=up:active,3=k=up:creating} 8 up:standby 2024-03-21T10:22:29.491 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:22:29 smithi157 ceph-mon[35581]: daemon mds.k is now active in filesystem cephfs as rank 3 2024-03-21T10:22:29.491 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:22:29 smithi157 ceph-mon[35581]: mds.? [v2:172.21.15.100:6832/2723302038,v1:172.21.15.100:6833/2723302038] up:active 2024-03-21T10:22:29.491 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:22:29 smithi157 ceph-mon[35581]: daemon mds.a assigned to filesystem cephfs as rank 4 (now has 5 ranks) 2024-03-21T10:22:29.491 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:22:29 smithi157 ceph-mon[35581]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-03-21T10:22:29.491 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:22:29 smithi157 ceph-mon[35581]: Cluster is now healthy 2024-03-21T10:22:29.491 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:22:29 smithi157 ceph-mon[35581]: fsmap cephfs:4 {0=b=up:active,1=h=up:active,2=e=up:active,3=k=up:active} 4 up:standby-replay 4 up:standby 2024-03-21T10:22:29.491 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:22:29 smithi157 ceph-mon[35581]: fsmap cephfs:5 4 up:standby-replay 1 up:creating 3 up:standby 4 up:active 2024-03-21T10:22:29.491 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:22:29 smithi157 ceph-mon[35581]: daemon mds.a is now active in filesystem cephfs as rank 4 2024-03-21T10:22:29.491 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:22:29 smithi157 ceph-mon[35581]: from='client.14880 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:22:29.547 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:29.545+0000 7f4d8be00640 1 -- 172.21.15.47:0/634752760 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4d8c102000 msgr2=0x7f4d8c1043f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:22:29.547 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:29.545+0000 7f4d8be00640 1 --2- 172.21.15.47:0/634752760 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4d8c102000 0x7f4d8c1043f0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f4d7c009900 tx=0x7f4d7c02bc30 comp rx=0 tx=0).stop 2024-03-21T10:22:29.547 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:29.546+0000 7f4d8be00640 1 -- 172.21.15.47:0/634752760 shutdown_connections 2024-03-21T10:22:29.547 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:29.546+0000 7f4d8be00640 1 --2- 172.21.15.47:0/634752760 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4d8c107260 0x7f4d8c1096b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:22:29.547 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:29.546+0000 7f4d8be00640 1 --2- 172.21.15.47:0/634752760 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4d8c104930 0x7f4d8c106d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:22:29.547 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:29.546+0000 7f4d8be00640 1 --2- 172.21.15.47:0/634752760 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4d8c102000 0x7f4d8c1043f0 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:22:29.547 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:29.546+0000 7f4d8be00640 1 -- 172.21.15.47:0/634752760 >> 172.21.15.47:0/634752760 conn(0x7f4d8c0fba10 msgr2=0x7f4d8c0fde30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:22:29.547 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:29.546+0000 7f4d8be00640 1 -- 172.21.15.47:0/634752760 shutdown_connections 2024-03-21T10:22:29.547 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:29.546+0000 7f4d8be00640 1 -- 172.21.15.47:0/634752760 wait complete. 2024-03-21T10:22:29.549 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:29.547+0000 7f4d8be00640 1 Processor -- start 2024-03-21T10:22:29.549 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:29.547+0000 7f4d8be00640 1 -- start start 2024-03-21T10:22:29.549 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:29.547+0000 7f4d8be00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4d8c102000 0x7f4d8c199a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:22:29.549 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:29.547+0000 7f4d8be00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4d8c104930 0x7f4d8c199f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:22:29.549 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:29.547+0000 7f4d8be00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4d8c107260 0x7f4d8c1a0fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:22:29.549 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:29.547+0000 7f4d8be00640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d8c10d110 con 0x7f4d8c102000 2024-03-21T10:22:29.549 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:29.547+0000 7f4d8be00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d8c10cf90 con 0x7f4d8c107260 2024-03-21T10:22:29.549 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:29.547+0000 7f4d8be00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d8c10d290 con 0x7f4d8c104930 2024-03-21T10:22:29.549 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:29.548+0000 7f4d8aa00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4d8c102000 0x7f4d8c199a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:22:29.550 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:29.548+0000 7f4d8b400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4d8c107260 0x7f4d8c1a0fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:22:29.550 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:29.548+0000 7f4d8b400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4d8c107260 0x7f4d8c1a0fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:51758/0 (socket says 172.21.15.47:51758) 2024-03-21T10:22:29.550 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:29.548+0000 7f4d8b400640 1 -- 172.21.15.47:0/2481443316 learned_addr learned my addr 172.21.15.47:0/2481443316 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:22:29.550 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:29.548+0000 7f4d8aa00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4d8c102000 0x7f4d8c199a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:56630/0 (socket says 172.21.15.47:56630) 2024-03-21T10:22:29.550 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:29.548+0000 7f4d82000640 1 --2- 172.21.15.47:0/2481443316 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4d8c104930 0x7f4d8c199f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:22:29.550 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:29.548+0000 7f4d8aa00640 1 -- 172.21.15.47:0/2481443316 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4d8c104930 msgr2=0x7f4d8c199f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:22:29.550 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:29.548+0000 7f4d8aa00640 1 --2- 172.21.15.47:0/2481443316 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4d8c104930 0x7f4d8c199f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:22:29.550 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:29.548+0000 7f4d8aa00640 1 -- 172.21.15.47:0/2481443316 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4d8c107260 msgr2=0x7f4d8c1a0fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:22:29.550 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:29.548+0000 7f4d8aa00640 1 --2- 172.21.15.47:0/2481443316 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4d8c107260 0x7f4d8c1a0fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:22:29.550 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:29.548+0000 7f4d8aa00640 1 -- 172.21.15.47:0/2481443316 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d8c1a1620 con 0x7f4d8c102000 2024-03-21T10:22:29.551 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:29.548+0000 7f4d8aa00640 1 --2- 172.21.15.47:0/2481443316 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4d8c102000 0x7f4d8c199a00 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f4d7c02c170 tx=0x7f4d7c0040c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:22:29.551 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:29.549+0000 7f4d83e00640 1 -- 172.21.15.47:0/2481443316 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d7c043020 con 0x7f4d8c102000 2024-03-21T10:22:29.551 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:29.549+0000 7f4d83e00640 1 -- 172.21.15.47:0/2481443316 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f4d7c0044c0 con 0x7f4d8c102000 2024-03-21T10:22:29.551 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:29.549+0000 7f4d83e00640 1 -- 172.21.15.47:0/2481443316 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d7c003c30 con 0x7f4d8c102000 2024-03-21T10:22:29.551 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:29.549+0000 7f4d8be00640 1 -- 172.21.15.47:0/2481443316 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4d8c1a18b0 con 0x7f4d8c102000 2024-03-21T10:22:29.551 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:29.549+0000 7f4d8be00640 1 -- 172.21.15.47:0/2481443316 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4d8c1a1e20 con 0x7f4d8c102000 2024-03-21T10:22:29.552 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:29.550+0000 7f4d8be00640 1 -- 172.21.15.47:0/2481443316 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4d8c108db0 con 0x7f4d8c102000 2024-03-21T10:22:29.553 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:29.552+0000 7f4d83e00640 1 -- 172.21.15.47:0/2481443316 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 16) v1 ==== 137854+0+0 (secure 0 0 0) 0x7f4d7c046020 con 0x7f4d8c102000 2024-03-21T10:22:29.555 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:29.554+0000 7f4d83e00640 1 --2- 172.21.15.47:0/2481443316 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f4d700a2fa0 0x7f4d700a5460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:22:29.555 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:29.554+0000 7f4d83e00640 1 -- 172.21.15.47:0/2481443316 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9092+0+0 (secure 0 0 0) 0x7f4d7c0f1980 con 0x7f4d8c102000 2024-03-21T10:22:29.555 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:29.554+0000 7f4d82000640 1 --2- 172.21.15.47:0/2481443316 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f4d700a2fa0 0x7f4d700a5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:22:29.555 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:29.555+0000 7f4d82000640 1 --2- 172.21.15.47:0/2481443316 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f4d700a2fa0 0x7f4d700a5460 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f4d74005fd0 tx=0x7f4d74005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:22:29.559 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:29.558+0000 7f4d83e00640 1 -- 172.21.15.47:0/2481443316 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f4d7c04d020 con 0x7f4d8c102000 2024-03-21T10:22:29.800 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:29.799+0000 7f4d8be00640 1 -- 172.21.15.47:0/2481443316 --> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}) v1 -- 0x7f4d8c061ae0 con 0x7f4d700a2fa0 2024-03-21T10:22:29.838 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:29.837+0000 7f4d83e00640 1 -- 172.21.15.47:0/2481443316 <== mgr.14152 v2:172.21.15.47:6800/2823885994 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f4d8c061ae0 con 0x7f4d700a2fa0 2024-03-21T10:22:29.842 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:29.841+0000 7f4d8be00640 1 -- 172.21.15.47:0/2481443316 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f4d700a2fa0 msgr2=0x7f4d700a5460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:22:29.842 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:29.841+0000 7f4d8be00640 1 --2- 172.21.15.47:0/2481443316 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f4d700a2fa0 0x7f4d700a5460 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f4d74005fd0 tx=0x7f4d74005850 comp rx=0 tx=0).stop 2024-03-21T10:22:29.842 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:29.841+0000 7f4d8be00640 1 -- 172.21.15.47:0/2481443316 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4d8c102000 msgr2=0x7f4d8c199a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:22:29.842 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:29.841+0000 7f4d8be00640 1 --2- 172.21.15.47:0/2481443316 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4d8c102000 0x7f4d8c199a00 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f4d7c02c170 tx=0x7f4d7c0040c0 comp rx=0 tx=0).stop 2024-03-21T10:22:29.843 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:29.841+0000 7f4d8be00640 1 -- 172.21.15.47:0/2481443316 shutdown_connections 2024-03-21T10:22:29.843 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:29.841+0000 7f4d8be00640 1 --2- 172.21.15.47:0/2481443316 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f4d700a2fa0 0x7f4d700a5460 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:22:29.843 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:29.841+0000 7f4d8be00640 1 --2- 172.21.15.47:0/2481443316 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4d8c107260 0x7f4d8c1a0fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:22:29.843 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:29.841+0000 7f4d8be00640 1 --2- 172.21.15.47:0/2481443316 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4d8c104930 0x7f4d8c199f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:22:29.843 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:29.841+0000 7f4d8be00640 1 --2- 172.21.15.47:0/2481443316 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4d8c102000 0x7f4d8c199a00 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:22:29.843 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:29.841+0000 7f4d8be00640 1 -- 172.21.15.47:0/2481443316 >> 172.21.15.47:0/2481443316 conn(0x7f4d8c0fba10 msgr2=0x7f4d8c1016d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:22:29.843 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:29.841+0000 7f4d8be00640 1 -- 172.21.15.47:0/2481443316 shutdown_connections 2024-03-21T10:22:29.843 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:29.841+0000 7f4d8be00640 1 -- 172.21.15.47:0/2481443316 wait complete. 2024-03-21T10:22:29.863 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs --group_name qa sv_1 2024-03-21T10:22:30.079 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:30.078+0000 7fc271e00640 1 -- 172.21.15.47:0/1905306255 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc26c1049d0 msgr2=0x7fc26c106dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:22:30.079 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:30.078+0000 7fc271e00640 1 --2- 172.21.15.47:0/1905306255 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc26c1049d0 0x7fc26c106dc0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fc258009900 tx=0x7fc25802bc30 comp rx=0 tx=0).stop 2024-03-21T10:22:30.079 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:30.078+0000 7fc271e00640 1 -- 172.21.15.47:0/1905306255 shutdown_connections 2024-03-21T10:22:30.080 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:30.078+0000 7fc271e00640 1 --2- 172.21.15.47:0/1905306255 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc26c107300 0x7fc26c1096f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:22:30.080 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:30.078+0000 7fc271e00640 1 --2- 172.21.15.47:0/1905306255 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc26c1049d0 0x7fc26c106dc0 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:22:30.080 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:30.078+0000 7fc271e00640 1 --2- 172.21.15.47:0/1905306255 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc26c1020a0 0x7fc26c104490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:22:30.080 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:30.078+0000 7fc271e00640 1 -- 172.21.15.47:0/1905306255 >> 172.21.15.47:0/1905306255 conn(0x7fc26c0fba10 msgr2=0x7fc26c0fde30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:22:30.080 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:30.078+0000 7fc271e00640 1 -- 172.21.15.47:0/1905306255 shutdown_connections 2024-03-21T10:22:30.080 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:30.078+0000 7fc271e00640 1 -- 172.21.15.47:0/1905306255 wait complete. 2024-03-21T10:22:30.082 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:30.079+0000 7fc271e00640 1 Processor -- start 2024-03-21T10:22:30.082 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:30.079+0000 7fc271e00640 1 -- start start 2024-03-21T10:22:30.082 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:30.079+0000 7fc271e00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc26c1020a0 0x7fc26c199a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:22:30.082 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:30.079+0000 7fc271e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc26c1049d0 0x7fc26c199f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:22:30.082 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:30.079+0000 7fc271e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc26c107300 0x7fc26c1a0ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:22:30.082 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:30.080+0000 7fc271e00640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc26c10d0e0 con 0x7fc26c1049d0 2024-03-21T10:22:30.082 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:30.080+0000 7fc271e00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc26c10cf60 con 0x7fc26c107300 2024-03-21T10:22:30.082 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:30.080+0000 7fc271e00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc26c10d260 con 0x7fc26c1020a0 2024-03-21T10:22:30.082 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:30.080+0000 7fc26be00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc26c1049d0 0x7fc26c199f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:22:30.082 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:30.080+0000 7fc26be00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc26c1049d0 0x7fc26c199f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:56644/0 (socket says 172.21.15.47:56644) 2024-03-21T10:22:30.082 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:30.080+0000 7fc26be00640 1 -- 172.21.15.47:0/859530090 learned_addr learned my addr 172.21.15.47:0/859530090 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:22:30.082 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:30.080+0000 7fc271400640 1 --2- 172.21.15.47:0/859530090 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc26c107300 0x7fc26c1a0ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:22:30.083 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:30.080+0000 7fc270a00640 1 --2- 172.21.15.47:0/859530090 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc26c1020a0 0x7fc26c199a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:22:30.083 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:30.080+0000 7fc26be00640 1 -- 172.21.15.47:0/859530090 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc26c1020a0 msgr2=0x7fc26c199a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:22:30.083 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:30.080+0000 7fc26be00640 1 --2- 172.21.15.47:0/859530090 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc26c1020a0 0x7fc26c199a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:22:30.083 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:30.080+0000 7fc26be00640 1 -- 172.21.15.47:0/859530090 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc26c107300 msgr2=0x7fc26c1a0ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:22:30.083 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:30.080+0000 7fc26be00640 1 --2- 172.21.15.47:0/859530090 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc26c107300 0x7fc26c1a0ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:22:30.083 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:30.080+0000 7fc26be00640 1 -- 172.21.15.47:0/859530090 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc26c1a1660 con 0x7fc26c1049d0 2024-03-21T10:22:30.083 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:30.080+0000 7fc271400640 1 --2- 172.21.15.47:0/859530090 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc26c107300 0x7fc26c1a0ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T10:22:30.083 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:30.080+0000 7fc270a00640 1 --2- 172.21.15.47:0/859530090 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc26c1020a0 0x7fc26c199a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T10:22:30.083 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:30.081+0000 7fc26be00640 1 --2- 172.21.15.47:0/859530090 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc26c1049d0 0x7fc26c199f80 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fc258000c00 tx=0x7fc258005dc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:22:30.084 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:30.081+0000 7fc269600640 1 -- 172.21.15.47:0/859530090 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc258043020 con 0x7fc26c1049d0 2024-03-21T10:22:30.084 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:30.081+0000 7fc269600640 1 -- 172.21.15.47:0/859530090 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fc258004350 con 0x7fc26c1049d0 2024-03-21T10:22:30.084 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:30.081+0000 7fc269600640 1 -- 172.21.15.47:0/859530090 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc258003b10 con 0x7fc26c1049d0 2024-03-21T10:22:30.084 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:30.081+0000 7fc271e00640 1 -- 172.21.15.47:0/859530090 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc26c1a18f0 con 0x7fc26c1049d0 2024-03-21T10:22:30.084 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:30.081+0000 7fc271e00640 1 -- 172.21.15.47:0/859530090 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc26c1a1e90 con 0x7fc26c1049d0 2024-03-21T10:22:30.084 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:30.083+0000 7fc271e00640 1 -- 172.21.15.47:0/859530090 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc26c0683b0 con 0x7fc26c1049d0 2024-03-21T10:22:30.087 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:30.086+0000 7fc269600640 1 -- 172.21.15.47:0/859530090 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 16) v1 ==== 137854+0+0 (secure 0 0 0) 0x7fc258046020 con 0x7fc26c1049d0 2024-03-21T10:22:30.089 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:30.088+0000 7fc269600640 1 --2- 172.21.15.47:0/859530090 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7fc2540a2fa0 0x7fc2540a5460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:22:30.089 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:30.088+0000 7fc269600640 1 -- 172.21.15.47:0/859530090 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9092+0+0 (secure 0 0 0) 0x7fc2580f18e0 con 0x7fc26c1049d0 2024-03-21T10:22:30.089 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:30.089+0000 7fc270a00640 1 --2- 172.21.15.47:0/859530090 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7fc2540a2fa0 0x7fc2540a5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:22:30.090 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:30.089+0000 7fc270a00640 1 --2- 172.21.15.47:0/859530090 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7fc2540a2fa0 0x7fc2540a5460 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fc260009b50 tx=0x7fc260005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:22:30.093 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:30.092+0000 7fc269600640 1 -- 172.21.15.47:0/859530090 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fc25804d020 con 0x7fc26c1049d0 2024-03-21T10:22:30.323 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:30.322+0000 7fc271e00640 1 -- 172.21.15.47:0/859530090 --> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}) v1 -- 0x7fc26c061ba0 con 0x7fc2540a2fa0 2024-03-21T10:22:30.330 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:30.329+0000 7fc269600640 1 -- 172.21.15.47:0/859530090 <== mgr.14152 v2:172.21.15.47:6800/2823885994 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+54 (secure 0 0 0) 0x7fc26c061ba0 con 0x7fc2540a2fa0 2024-03-21T10:22:30.330 INFO:teuthology.orchestra.run.smithi047.stdout:/volumes/qa/sv_1/5b8e4fbe-43c5-4bda-83c2-1b6758ed9703 2024-03-21T10:22:30.333 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:30.332+0000 7fc271e00640 1 -- 172.21.15.47:0/859530090 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7fc2540a2fa0 msgr2=0x7fc2540a5460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:22:30.333 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:30.332+0000 7fc271e00640 1 --2- 172.21.15.47:0/859530090 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7fc2540a2fa0 0x7fc2540a5460 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fc260009b50 tx=0x7fc260005f90 comp rx=0 tx=0).stop 2024-03-21T10:22:30.333 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:30.332+0000 7fc271e00640 1 -- 172.21.15.47:0/859530090 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc26c1049d0 msgr2=0x7fc26c199f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:22:30.333 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:30.332+0000 7fc271e00640 1 --2- 172.21.15.47:0/859530090 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc26c1049d0 0x7fc26c199f80 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fc258000c00 tx=0x7fc258005dc0 comp rx=0 tx=0).stop 2024-03-21T10:22:30.333 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:30.332+0000 7fc271e00640 1 -- 172.21.15.47:0/859530090 shutdown_connections 2024-03-21T10:22:30.334 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:30.332+0000 7fc271e00640 1 --2- 172.21.15.47:0/859530090 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7fc2540a2fa0 0x7fc2540a5460 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:22:30.334 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:30.332+0000 7fc271e00640 1 --2- 172.21.15.47:0/859530090 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc26c107300 0x7fc26c1a0ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:22:30.334 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:30.332+0000 7fc271e00640 1 --2- 172.21.15.47:0/859530090 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc26c1049d0 0x7fc26c199f80 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:22:30.334 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:30.332+0000 7fc271e00640 1 --2- 172.21.15.47:0/859530090 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc26c1020a0 0x7fc26c199a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:22:30.334 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:30.332+0000 7fc271e00640 1 -- 172.21.15.47:0/859530090 >> 172.21.15.47:0/859530090 conn(0x7fc26c0fba10 msgr2=0x7fc26c101940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:22:30.334 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:30.332+0000 7fc271e00640 1 -- 172.21.15.47:0/859530090 shutdown_connections 2024-03-21T10:22:30.334 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:30.332+0000 7fc271e00640 1 -- 172.21.15.47:0/859530090 wait complete. 2024-03-21T10:22:30.354 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-21T10:22:30.554 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:30.553+0000 7f60cae00640 1 -- 172.21.15.47:0/4236904629 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f60c41016c0 msgr2=0x7f60c4101ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:22:30.555 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:30.553+0000 7f60cae00640 1 --2- 172.21.15.47:0/4236904629 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f60c41016c0 0x7f60c4101ae0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f60b4009900 tx=0x7f60b402bc30 comp rx=0 tx=0).stop 2024-03-21T10:22:30.555 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:30.553+0000 7f60cae00640 1 -- 172.21.15.47:0/4236904629 shutdown_connections 2024-03-21T10:22:30.555 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:30.553+0000 7f60cae00640 1 --2- 172.21.15.47:0/4236904629 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f60c4103270 0x7f60c4109b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:22:30.555 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:30.553+0000 7f60cae00640 1 --2- 172.21.15.47:0/4236904629 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f60c41028b0 0x7f60c4102d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:22:30.555 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:30.553+0000 7f60cae00640 1 --2- 172.21.15.47:0/4236904629 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f60c41016c0 0x7f60c4101ae0 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:22:30.555 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:30.553+0000 7f60cae00640 1 -- 172.21.15.47:0/4236904629 >> 172.21.15.47:0/4236904629 conn(0x7f60c40fcc50 msgr2=0x7f60c40ff070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:22:30.555 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:30.553+0000 7f60cae00640 1 -- 172.21.15.47:0/4236904629 shutdown_connections 2024-03-21T10:22:30.555 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:30.553+0000 7f60cae00640 1 -- 172.21.15.47:0/4236904629 wait complete. 2024-03-21T10:22:30.557 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:30.554+0000 7f60cae00640 1 Processor -- start 2024-03-21T10:22:30.557 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:30.554+0000 7f60cae00640 1 -- start start 2024-03-21T10:22:30.557 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:30.555+0000 7f60cae00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f60c41016c0 0x7f60c4199b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:22:30.558 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:30.555+0000 7f60cae00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f60c41028b0 0x7f60c419a050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:22:30.558 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:30.555+0000 7f60cae00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f60c4103270 0x7f60c41a10c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:22:30.558 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:30.555+0000 7f60cae00640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60c410d230 con 0x7f60c41028b0 2024-03-21T10:22:30.558 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:30.555+0000 7f60cae00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60c410d0b0 con 0x7f60c4103270 2024-03-21T10:22:30.558 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:30.555+0000 7f60cae00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60c410d3b0 con 0x7f60c41016c0 2024-03-21T10:22:30.558 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:30.555+0000 7f60c3400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f60c41028b0 0x7f60c419a050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:22:30.558 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:30.555+0000 7f60c3400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f60c41028b0 0x7f60c419a050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:56672/0 (socket says 172.21.15.47:56672) 2024-03-21T10:22:30.558 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:30.555+0000 7f60c3400640 1 -- 172.21.15.47:0/1096767453 learned_addr learned my addr 172.21.15.47:0/1096767453 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:22:30.558 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:30.555+0000 7f60c3e00640 1 --2- 172.21.15.47:0/1096767453 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f60c41016c0 0x7f60c4199b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:22:30.558 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:30.555+0000 7f60c8e00640 1 --2- 172.21.15.47:0/1096767453 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f60c4103270 0x7f60c41a10c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:22:30.558 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:30.555+0000 7f60c3400640 1 -- 172.21.15.47:0/1096767453 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f60c41016c0 msgr2=0x7f60c4199b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:22:30.558 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:30.555+0000 7f60c3400640 1 --2- 172.21.15.47:0/1096767453 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f60c41016c0 0x7f60c4199b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:22:30.559 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:30.555+0000 7f60c3400640 1 -- 172.21.15.47:0/1096767453 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f60c4103270 msgr2=0x7f60c41a10c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:22:30.559 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:30.555+0000 7f60c3400640 1 --2- 172.21.15.47:0/1096767453 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f60c4103270 0x7f60c41a10c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:22:30.559 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:30.555+0000 7f60c3400640 1 -- 172.21.15.47:0/1096767453 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f60c41a1730 con 0x7f60c41028b0 2024-03-21T10:22:30.559 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:30.556+0000 7f60c3400640 1 --2- 172.21.15.47:0/1096767453 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f60c41028b0 0x7f60c419a050 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f60b000d770 tx=0x7f60b000dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:22:30.559 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:30.556+0000 7f60c0c00640 1 -- 172.21.15.47:0/1096767453 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60b0017070 con 0x7f60c41028b0 2024-03-21T10:22:30.559 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:30.556+0000 7f60c0c00640 1 -- 172.21.15.47:0/1096767453 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f60b00054d0 con 0x7f60c41028b0 2024-03-21T10:22:30.559 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:30.556+0000 7f60cae00640 1 -- 172.21.15.47:0/1096767453 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f60c41a1a20 con 0x7f60c41028b0 2024-03-21T10:22:30.559 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:30.556+0000 7f60c0c00640 1 -- 172.21.15.47:0/1096767453 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60b0004cf0 con 0x7f60c41028b0 2024-03-21T10:22:30.559 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:30.556+0000 7f60cae00640 1 -- 172.21.15.47:0/1096767453 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f60c41a1f70 con 0x7f60c41028b0 2024-03-21T10:22:30.561 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:30.558+0000 7f60cae00640 1 -- 172.21.15.47:0/1096767453 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f60c41a21a0 con 0x7f60c41028b0 2024-03-21T10:22:30.561 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:30.560+0000 7f60c0c00640 1 -- 172.21.15.47:0/1096767453 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 16) v1 ==== 137854+0+0 (secure 0 0 0) 0x7f60b0004e90 con 0x7f60c41028b0 2024-03-21T10:22:30.563 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:30.562+0000 7f60c0c00640 1 --2- 172.21.15.47:0/1096767453 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f60a80a2fa0 0x7f60a80a5460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:22:30.564 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:30.563+0000 7f60c0c00640 1 -- 172.21.15.47:0/1096767453 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9092+0+0 (secure 0 0 0) 0x7f60b00cf260 con 0x7f60c41028b0 2024-03-21T10:22:30.564 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:30.563+0000 7f60c3e00640 1 --2- 172.21.15.47:0/1096767453 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f60a80a2fa0 0x7f60a80a5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:22:30.564 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:30.564+0000 7f60c3e00640 1 --2- 172.21.15.47:0/1096767453 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f60a80a2fa0 0x7f60a80a5460 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f60b402c240 tx=0x7f60b4005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:22:30.567 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:30.566+0000 7f60c0c00640 1 -- 172.21.15.47:0/1096767453 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f60b009bcd0 con 0x7f60c41028b0 2024-03-21T10:22:30.622 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:30.621+0000 7f60c0c00640 1 -- 172.21.15.47:0/1096767453 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mgrmap(e 17) v1 ==== 137905+0+0 (secure 0 0 0) 0x7f60b0097c40 con 0x7f60c41028b0 2024-03-21T10:22:30.714 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:22:30 smithi047 ceph-mon[31547]: from='client.14886 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:22:30.714 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:22:30 smithi047 ceph-mon[31547]: mds.? [v2:172.21.15.47:6840/3401578978,v1:172.21.15.47:6841/3401578978] up:active 2024-03-21T10:22:30.714 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:22:30 smithi047 ceph-mon[31547]: fsmap cephfs:5 4 up:standby-replay 3 up:standby 5 up:active 2024-03-21T10:22:30.714 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:22:30 smithi047 ceph-mon[31547]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-21T10:22:30.714 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:22:30 smithi047 ceph-mon[31547]: from='client.14892 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:22:30.729 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:22:30 smithi100 ceph-mon[35193]: from='client.14886 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:22:30.729 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:22:30 smithi100 ceph-mon[35193]: mds.? [v2:172.21.15.47:6840/3401578978,v1:172.21.15.47:6841/3401578978] up:active 2024-03-21T10:22:30.729 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:22:30 smithi100 ceph-mon[35193]: fsmap cephfs:5 4 up:standby-replay 3 up:standby 5 up:active 2024-03-21T10:22:30.729 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:22:30 smithi100 ceph-mon[35193]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-21T10:22:30.729 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:22:30 smithi100 ceph-mon[35193]: from='client.14892 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:22:30.878 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:30.877+0000 7f60cae00640 1 -- 172.21.15.47:0/1096767453 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f60c41a2370 con 0x7f60c41028b0 2024-03-21T10:22:30.880 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:30.879+0000 7f60c0c00640 1 -- 172.21.15.47:0/1096767453 <== mon.0 v2:172.21.15.47:3300/0 8 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) v1 ==== 94+0+11471 (secure 0 0 0) 0x7f60b00af980 con 0x7f60c41028b0 2024-03-21T10:22:30.880 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-21T10:22:30.881 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":29,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14511,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6835/735724037","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6834","nonce":735724037},{"type":"v1","addr":"172.21.15.47:6835","nonce":735724037}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14529,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6837/2248191724","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6836","nonce":2248191724},{"type":"v1","addr":"172.21.15.47:6837","nonce":2248191724}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":29,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-21T10:21:58.329063+0000","modified":"2024-03-21T10:22:29.588509+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":34535,"mds_1":34493,"mds_2":34454,"mds_3":34427,"mds_4":14619},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14604":{"gid":14604,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.47:6839/2881335356","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6838","nonce":2881335356},{"type":"v1","addr":"172.21.15.47:6839","nonce":2881335356}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_14619":{"gid":14619,"name":"a","rank":4,"incarnation":27,"state":"up:active","state_seq":11,"addr":"172.21.15.47:6841/3401578978","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6840","nonce":3401578978},{"type":"v1","addr":"172.21.15.47:6841","nonce":3401578978}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24406":{"gid":24406,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.157:6833/4000010136","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":4000010136},{"type":"v1","addr":"172.21.15.157:6833","nonce":4000010136}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24457":{"gid":24457,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.157:6835/3902868651","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":3902868651},{"type":"v1","addr":"172.21.15.157:6835","nonce":3902868651}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24475":{"gid":24475,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.157:6837/1908087204","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6836","nonce":1908087204},{"type":"v1","addr":"172.21.15.157:6837","nonce":1908087204}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24499":{"gid":24499,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.157:6839/4064242876","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6838","nonce":4064242876},{"type":"v1","addr":"172.21.15.157:6839","nonce":4064242876}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_34427":{"gid":34427,"name":"k","rank":3,"incarnation":25,"state":"up:active","state_seq":20,"addr":"172.21.15.100:6833/2723302038","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6832","nonce":2723302038},{"type":"v1","addr":"172.21.15.100:6833","nonce":2723302038}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_34454":{"gid":34454,"name":"e","rank":2,"incarnation":22,"state":"up:active","state_seq":17,"addr":"172.21.15.100:6835/2597849957","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":2597849957},{"type":"v1","addr":"172.21.15.100:6835","nonce":2597849957}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_34493":{"gid":34493,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":13,"addr":"172.21.15.100:6837/3497644162","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6836","nonce":3497644162},{"type":"v1","addr":"172.21.15.100:6837","nonce":3497644162}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_34535":{"gid":34535,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.100:6839/4173168440","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6838","nonce":4173168440},{"type":"v1","addr":"172.21.15.100:6839","nonce":4173168440}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":34535,"qdb_cluster":[34493,34535,34454,34427,14619]},"id":1}]} 2024-03-21T10:22:30.881 INFO:teuthology.orchestra.run.smithi047.stderr:dumped fsmap epoch 29 2024-03-21T10:22:30.883 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:30.882+0000 7f60cae00640 1 -- 172.21.15.47:0/1096767453 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f60a80a2fa0 msgr2=0x7f60a80a5460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:22:30.883 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:30.882+0000 7f60cae00640 1 --2- 172.21.15.47:0/1096767453 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f60a80a2fa0 0x7f60a80a5460 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f60b402c240 tx=0x7f60b4005fd0 comp rx=0 tx=0).stop 2024-03-21T10:22:30.883 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:30.882+0000 7f60cae00640 1 -- 172.21.15.47:0/1096767453 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f60c41028b0 msgr2=0x7f60c419a050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:22:30.883 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:30.882+0000 7f60cae00640 1 --2- 172.21.15.47:0/1096767453 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f60c41028b0 0x7f60c419a050 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f60b000d770 tx=0x7f60b000dc30 comp rx=0 tx=0).stop 2024-03-21T10:22:30.883 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:30.882+0000 7f60cae00640 1 -- 172.21.15.47:0/1096767453 shutdown_connections 2024-03-21T10:22:30.883 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:30.882+0000 7f60cae00640 1 --2- 172.21.15.47:0/1096767453 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f60a80a2fa0 0x7f60a80a5460 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:22:30.883 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:30.882+0000 7f60cae00640 1 --2- 172.21.15.47:0/1096767453 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f60c4103270 0x7f60c41a10c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:22:30.884 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:30.882+0000 7f60cae00640 1 --2- 172.21.15.47:0/1096767453 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f60c41028b0 0x7f60c419a050 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:22:30.884 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:30.882+0000 7f60cae00640 1 --2- 172.21.15.47:0/1096767453 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f60c41016c0 0x7f60c4199b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:22:30.884 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:30.882+0000 7f60cae00640 1 -- 172.21.15.47:0/1096767453 >> 172.21.15.47:0/1096767453 conn(0x7f60c40fcc50 msgr2=0x7f60c40738d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:22:30.884 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:30.882+0000 7f60cae00640 1 -- 172.21.15.47:0/1096767453 shutdown_connections 2024-03-21T10:22:30.884 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:30.882+0000 7f60cae00640 1 -- 172.21.15.47:0/1096767453 wait complete. 2024-03-21T10:22:30.901 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-21T10:22:30.990 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:22:30 smithi157 ceph-mon[35581]: from='client.14886 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:22:30.990 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:22:30 smithi157 ceph-mon[35581]: mds.? [v2:172.21.15.47:6840/3401578978,v1:172.21.15.47:6841/3401578978] up:active 2024-03-21T10:22:30.990 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:22:30 smithi157 ceph-mon[35581]: fsmap cephfs:5 4 up:standby-replay 3 up:standby 5 up:active 2024-03-21T10:22:30.990 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:22:30 smithi157 ceph-mon[35581]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-21T10:22:30.990 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:22:30 smithi157 ceph-mon[35581]: from='client.14892 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:22:31.116 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:31.114+0000 7f478ba00640 1 -- 172.21.15.47:0/4138082568 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4784102140 msgr2=0x7f4784104530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:22:31.116 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:31.114+0000 7f478ba00640 1 --2- 172.21.15.47:0/4138082568 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4784102140 0x7f4784104530 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f4778009900 tx=0x7f477802bc40 comp rx=0 tx=0).stop 2024-03-21T10:22:31.116 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:31.114+0000 7f478ba00640 1 -- 172.21.15.47:0/4138082568 shutdown_connections 2024-03-21T10:22:31.116 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:31.114+0000 7f478ba00640 1 --2- 172.21.15.47:0/4138082568 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f47841073a0 0x7f47841097f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:22:31.116 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:31.114+0000 7f478ba00640 1 --2- 172.21.15.47:0/4138082568 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4784104a70 0x7f4784106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:22:31.116 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:31.114+0000 7f478ba00640 1 --2- 172.21.15.47:0/4138082568 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4784102140 0x7f4784104530 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:22:31.116 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:31.114+0000 7f478ba00640 1 -- 172.21.15.47:0/4138082568 >> 172.21.15.47:0/4138082568 conn(0x7f47840fbb30 msgr2=0x7f47840fdf70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:22:31.116 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:31.114+0000 7f478ba00640 1 -- 172.21.15.47:0/4138082568 shutdown_connections 2024-03-21T10:22:31.116 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:31.114+0000 7f478ba00640 1 -- 172.21.15.47:0/4138082568 wait complete. 2024-03-21T10:22:31.119 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:31.115+0000 7f478ba00640 1 Processor -- start 2024-03-21T10:22:31.119 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:31.115+0000 7f478ba00640 1 -- start start 2024-03-21T10:22:31.119 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:31.116+0000 7f478ba00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4784102140 0x7f4784101ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:22:31.119 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:31.116+0000 7f478ba00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4784104a70 0x7f47841000f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:22:31.119 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:31.116+0000 7f478ba00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f47841073a0 0x7f4784100630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:22:31.119 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:31.116+0000 7f478ba00640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f478410d200 con 0x7f4784104a70 2024-03-21T10:22:31.119 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:31.116+0000 7f478ba00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f478410d080 con 0x7f4784102140 2024-03-21T10:22:31.119 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:31.116+0000 7f478ba00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f478410d380 con 0x7f47841073a0 2024-03-21T10:22:31.119 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:31.116+0000 7f4783e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4784104a70 0x7f47841000f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:22:31.119 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:31.116+0000 7f4783e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4784104a70 0x7f47841000f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:56696/0 (socket says 172.21.15.47:56696) 2024-03-21T10:22:31.119 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:31.116+0000 7f4783e00640 1 -- 172.21.15.47:0/3857899676 learned_addr learned my addr 172.21.15.47:0/3857899676 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:22:31.120 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:31.116+0000 7f4789000640 1 --2- 172.21.15.47:0/3857899676 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4784102140 0x7f4784101ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:22:31.120 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:31.116+0000 7f4789a00640 1 --2- 172.21.15.47:0/3857899676 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f47841073a0 0x7f4784100630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:22:31.120 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:31.117+0000 7f4783e00640 1 -- 172.21.15.47:0/3857899676 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f47841073a0 msgr2=0x7f4784100630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:22:31.120 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:31.117+0000 7f4783e00640 1 --2- 172.21.15.47:0/3857899676 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f47841073a0 0x7f4784100630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:22:31.120 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:31.117+0000 7f4783e00640 1 -- 172.21.15.47:0/3857899676 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4784102140 msgr2=0x7f4784101ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:22:31.120 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:31.117+0000 7f4783e00640 1 --2- 172.21.15.47:0/3857899676 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4784102140 0x7f4784101ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:22:31.120 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:31.117+0000 7f4783e00640 1 -- 172.21.15.47:0/3857899676 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4784100ec0 con 0x7f4784104a70 2024-03-21T10:22:31.120 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:31.117+0000 7f4783e00640 1 --2- 172.21.15.47:0/3857899676 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4784104a70 0x7f47841000f0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f4770005fd0 tx=0x7f477000dde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:22:31.120 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:31.117+0000 7f4781600640 1 -- 172.21.15.47:0/3857899676 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4770017070 con 0x7f4784104a70 2024-03-21T10:22:31.120 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:31.117+0000 7f4781600640 1 -- 172.21.15.47:0/3857899676 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f4770009c30 con 0x7f4784104a70 2024-03-21T10:22:31.120 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:31.117+0000 7f4781600640 1 -- 172.21.15.47:0/3857899676 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4770003d40 con 0x7f4784104a70 2024-03-21T10:22:31.120 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:31.117+0000 7f478ba00640 1 -- 172.21.15.47:0/3857899676 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f47841a19d0 con 0x7f4784104a70 2024-03-21T10:22:31.120 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:31.118+0000 7f478ba00640 1 -- 172.21.15.47:0/3857899676 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f47841a1ed0 con 0x7f4784104a70 2024-03-21T10:22:31.121 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:31.119+0000 7f478ba00640 1 -- 172.21.15.47:0/3857899676 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f47840fd620 con 0x7f4784104a70 2024-03-21T10:22:31.122 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:31.122+0000 7f4781600640 1 -- 172.21.15.47:0/3857899676 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 17) v1 ==== 137905+0+0 (secure 0 0 0) 0x7f4770003ee0 con 0x7f4784104a70 2024-03-21T10:22:31.125 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:31.124+0000 7f4781600640 1 --2- 172.21.15.47:0/3857899676 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f476c0a2fb0 0x7f476c0a5470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:22:31.125 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:31.124+0000 7f4781600640 1 -- 172.21.15.47:0/3857899676 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9092+0+0 (secure 0 0 0) 0x7f47700cf5e0 con 0x7f4784104a70 2024-03-21T10:22:31.125 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:31.124+0000 7f4789000640 1 --2- 172.21.15.47:0/3857899676 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f476c0a2fb0 0x7f476c0a5470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:22:31.126 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:31.125+0000 7f4789000640 1 --2- 172.21.15.47:0/3857899676 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f476c0a2fb0 0x7f476c0a5470 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f4778009230 tx=0x7f4778005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:22:31.130 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:31.129+0000 7f4781600640 1 -- 172.21.15.47:0/3857899676 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f4770020030 con 0x7f4784104a70 2024-03-21T10:22:31.343 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:31.342+0000 7f478ba00640 1 -- 172.21.15.47:0/3857899676 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f478402d2c0 con 0x7f4784104a70 2024-03-21T10:22:31.348 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:31.347+0000 7f4781600640 1 -- 172.21.15.47:0/3857899676 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v93) v1 ==== 74+0+24644 (secure 0 0 0) 0x7f477009bf50 con 0x7f4784104a70 2024-03-21T10:22:31.349 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-21T10:22:31.349 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":93,"fsid":"3056d51a-e76b-11ee-95cd-87774f69a715","created":"2024-03-21T10:11:17.019965+0000","modified":"2024-03-21T10:22:27.181362+0000","last_up_change":"2024-03-21T10:20:52.675328+0000","last_in_change":"2024-03-21T10:20:27.648212+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-21T10:15:40.553114+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-21T10:21:56.399518+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-21T10:21:57.812940+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-21T10:22:01.822568+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"91","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"b0c7a102-8d89-428b-b793-cbbe67151f50","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6802","nonce":775837313},{"type":"v1","addr":"172.21.15.47:6803","nonce":775837313}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6804","nonce":775837313},{"type":"v1","addr":"172.21.15.47:6805","nonce":775837313}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6808","nonce":775837313},{"type":"v1","addr":"172.21.15.47:6809","nonce":775837313}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6806","nonce":775837313},{"type":"v1","addr":"172.21.15.47:6807","nonce":775837313}]},"public_addr":"172.21.15.47:6803/775837313","cluster_addr":"172.21.15.47:6805/775837313","heartbeat_back_addr":"172.21.15.47:6809/775837313","heartbeat_front_addr":"172.21.15.47:6807/775837313","state":["exists","up"]},{"osd":1,"uuid":"07bb81bb-7cf3-4bbc-ac9e-802f95185d11","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6800","nonce":2923304710},{"type":"v1","addr":"172.21.15.100:6801","nonce":2923304710}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":2923304710},{"type":"v1","addr":"172.21.15.100:6803","nonce":2923304710}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":2923304710},{"type":"v1","addr":"172.21.15.100:6807","nonce":2923304710}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6804","nonce":2923304710},{"type":"v1","addr":"172.21.15.100:6805","nonce":2923304710}]},"public_addr":"172.21.15.100:6801/2923304710","cluster_addr":"172.21.15.100:6803/2923304710","heartbeat_back_addr":"172.21.15.100:6807/2923304710","heartbeat_front_addr":"172.21.15.100:6805/2923304710","state":["exists","up"]},{"osd":2,"uuid":"88ce76b9-6081-4836-aad0-8c249c56e8a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":171377005},{"type":"v1","addr":"172.21.15.157:6801","nonce":171377005}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":171377005},{"type":"v1","addr":"172.21.15.157:6803","nonce":171377005}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":171377005},{"type":"v1","addr":"172.21.15.157:6807","nonce":171377005}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":171377005},{"type":"v1","addr":"172.21.15.157:6805","nonce":171377005}]},"public_addr":"172.21.15.157:6801/171377005","cluster_addr":"172.21.15.157:6803/171377005","heartbeat_back_addr":"172.21.15.157:6807/171377005","heartbeat_front_addr":"172.21.15.157:6805/171377005","state":["exists","up"]},{"osd":3,"uuid":"518aa33d-263f-45e6-aab5-2201ceb40e74","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":27,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6810","nonce":640760101},{"type":"v1","addr":"172.21.15.47:6811","nonce":640760101}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6812","nonce":640760101},{"type":"v1","addr":"172.21.15.47:6813","nonce":640760101}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6816","nonce":640760101},{"type":"v1","addr":"172.21.15.47:6817","nonce":640760101}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6814","nonce":640760101},{"type":"v1","addr":"172.21.15.47:6815","nonce":640760101}]},"public_addr":"172.21.15.47:6811/640760101","cluster_addr":"172.21.15.47:6813/640760101","heartbeat_back_addr":"172.21.15.47:6817/640760101","heartbeat_front_addr":"172.21.15.47:6815/640760101","state":["exists","up"]},{"osd":4,"uuid":"987bde83-ecc6-42e5-bd56-c143b3de4226","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":35,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6808","nonce":2785478639},{"type":"v1","addr":"172.21.15.100:6809","nonce":2785478639}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6810","nonce":2785478639},{"type":"v1","addr":"172.21.15.100:6811","nonce":2785478639}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6814","nonce":2785478639},{"type":"v1","addr":"172.21.15.100:6815","nonce":2785478639}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":2785478639},{"type":"v1","addr":"172.21.15.100:6813","nonce":2785478639}]},"public_addr":"172.21.15.100:6809/2785478639","cluster_addr":"172.21.15.100:6811/2785478639","heartbeat_back_addr":"172.21.15.100:6815/2785478639","heartbeat_front_addr":"172.21.15.100:6813/2785478639","state":["exists","up"]},{"osd":5,"uuid":"012d2188-3a61-4a5f-8d01-2001c15ffbe2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":40,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":1525645410},{"type":"v1","addr":"172.21.15.157:6809","nonce":1525645410}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":1525645410},{"type":"v1","addr":"172.21.15.157:6811","nonce":1525645410}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":1525645410},{"type":"v1","addr":"172.21.15.157:6815","nonce":1525645410}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":1525645410},{"type":"v1","addr":"172.21.15.157:6813","nonce":1525645410}]},"public_addr":"172.21.15.157:6809/1525645410","cluster_addr":"172.21.15.157:6811/1525645410","heartbeat_back_addr":"172.21.15.157:6815/1525645410","heartbeat_front_addr":"172.21.15.157:6813/1525645410","state":["exists","up"]},{"osd":6,"uuid":"c12c14b4-85f0-4d88-937c-de4f556437de","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6818","nonce":2321642790},{"type":"v1","addr":"172.21.15.47:6819","nonce":2321642790}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6820","nonce":2321642790},{"type":"v1","addr":"172.21.15.47:6821","nonce":2321642790}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6824","nonce":2321642790},{"type":"v1","addr":"172.21.15.47:6825","nonce":2321642790}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6822","nonce":2321642790},{"type":"v1","addr":"172.21.15.47:6823","nonce":2321642790}]},"public_addr":"172.21.15.47:6819/2321642790","cluster_addr":"172.21.15.47:6821/2321642790","heartbeat_back_addr":"172.21.15.47:6825/2321642790","heartbeat_front_addr":"172.21.15.47:6823/2321642790","state":["exists","up"]},{"osd":7,"uuid":"3c87c6d0-1d1f-4294-9917-aaaad7dda824","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6816","nonce":4104727330},{"type":"v1","addr":"172.21.15.100:6817","nonce":4104727330}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":4104727330},{"type":"v1","addr":"172.21.15.100:6819","nonce":4104727330}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":4104727330},{"type":"v1","addr":"172.21.15.100:6823","nonce":4104727330}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6820","nonce":4104727330},{"type":"v1","addr":"172.21.15.100:6821","nonce":4104727330}]},"public_addr":"172.21.15.100:6817/4104727330","cluster_addr":"172.21.15.100:6819/4104727330","heartbeat_back_addr":"172.21.15.100:6823/4104727330","heartbeat_front_addr":"172.21.15.100:6821/4104727330","state":["exists","up"]},{"osd":8,"uuid":"502d65cf-53c8-4936-8de4-e11b2a6e1e18","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":61,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":4067715347},{"type":"v1","addr":"172.21.15.157:6817","nonce":4067715347}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":4067715347},{"type":"v1","addr":"172.21.15.157:6819","nonce":4067715347}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":4067715347},{"type":"v1","addr":"172.21.15.157:6823","nonce":4067715347}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":4067715347},{"type":"v1","addr":"172.21.15.157:6821","nonce":4067715347}]},"public_addr":"172.21.15.157:6817/4067715347","cluster_addr":"172.21.15.157:6819/4067715347","heartbeat_back_addr":"172.21.15.157:6823/4067715347","heartbeat_front_addr":"172.21.15.157:6821/4067715347","state":["exists","up"]},{"osd":9,"uuid":"5ea1f5be-8458-4c93-9e61-c2e867a514f3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6826","nonce":1982108738},{"type":"v1","addr":"172.21.15.47:6827","nonce":1982108738}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6828","nonce":1982108738},{"type":"v1","addr":"172.21.15.47:6829","nonce":1982108738}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6832","nonce":1982108738},{"type":"v1","addr":"172.21.15.47:6833","nonce":1982108738}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6830","nonce":1982108738},{"type":"v1","addr":"172.21.15.47:6831","nonce":1982108738}]},"public_addr":"172.21.15.47:6827/1982108738","cluster_addr":"172.21.15.47:6829/1982108738","heartbeat_back_addr":"172.21.15.47:6833/1982108738","heartbeat_front_addr":"172.21.15.47:6831/1982108738","state":["exists","up"]},{"osd":10,"uuid":"2da3da90-a0be-4a54-a9ff-b2d6c9b36870","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":74,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":396391028},{"type":"v1","addr":"172.21.15.100:6825","nonce":396391028}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":396391028},{"type":"v1","addr":"172.21.15.100:6827","nonce":396391028}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6830","nonce":396391028},{"type":"v1","addr":"172.21.15.100:6831","nonce":396391028}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":396391028},{"type":"v1","addr":"172.21.15.100:6829","nonce":396391028}]},"public_addr":"172.21.15.100:6825/396391028","cluster_addr":"172.21.15.100:6827/396391028","heartbeat_back_addr":"172.21.15.100:6831/396391028","heartbeat_front_addr":"172.21.15.100:6829/396391028","state":["exists","up"]},{"osd":11,"uuid":"11798d41-7f7e-4319-95b8-8d3d0946bde9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":82,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":2671029483},{"type":"v1","addr":"172.21.15.157:6825","nonce":2671029483}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":2671029483},{"type":"v1","addr":"172.21.15.157:6827","nonce":2671029483}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":2671029483},{"type":"v1","addr":"172.21.15.157:6831","nonce":2671029483}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":2671029483},{"type":"v1","addr":"172.21.15.157:6829","nonce":2671029483}]},"public_addr":"172.21.15.157:6825/2671029483","cluster_addr":"172.21.15.157:6827/2671029483","heartbeat_back_addr":"172.21.15.157:6831/2671029483","heartbeat_front_addr":"172.21.15.157:6829/2671029483","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T10:14:31.822399+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T10:15:04.175721+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T10:15:35.902595+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T10:16:12.088891+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T10:16:46.428704+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T10:17:20.158270+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T10:17:55.915253+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T10:18:30.064641+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T10:19:04.410004+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T10:19:40.605181+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T10:20:15.636605+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T10:20:50.151365+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.11","mappings":[{"from":7,"to":11}]},{"pgid":"2.14","mappings":[{"from":2,"to":4}]},{"pgid":"2.1a","mappings":[{"from":8,"to":1}]},{"pgid":"2.1b","mappings":[{"from":2,"to":4}]},{"pgid":"2.1d","mappings":[{"from":8,"to":11}]},{"pgid":"3.0","mappings":[{"from":4,"to":8}]},{"pgid":"3.6","mappings":[{"from":3,"to":7}]},{"pgid":"3.1b","mappings":[{"from":4,"to":6}]},{"pgid":"3.33","mappings":[{"from":5,"to":9}]},{"pgid":"3.3c","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.47:0/356822431":"2024-03-22T10:12:25.844530+0000","172.21.15.47:0/1167490879":"2024-03-22T10:12:25.844530+0000","172.21.15.47:0/1540718138":"2024-03-22T10:12:25.844530+0000","172.21.15.47:6801/3836903972":"2024-03-22T10:12:25.844530+0000","172.21.15.47:0/235436602":"2024-03-22T10:11:50.301838+0000","172.21.15.47:0/2052815572":"2024-03-22T10:11:50.301838+0000","172.21.15.47:6800/3942658432":"2024-03-22T10:11:50.301838+0000","172.21.15.47:6801/3942658432":"2024-03-22T10:11:50.301838+0000","172.21.15.47:0/1419516257":"2024-03-22T10:11:50.301838+0000","172.21.15.47:6800/3836903972":"2024-03-22T10:12:25.844530+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-21T10:22:31.352 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:31.350+0000 7f478ba00640 1 -- 172.21.15.47:0/3857899676 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f476c0a2fb0 msgr2=0x7f476c0a5470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:22:31.352 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:31.350+0000 7f478ba00640 1 --2- 172.21.15.47:0/3857899676 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f476c0a2fb0 0x7f476c0a5470 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f4778009230 tx=0x7f4778005fd0 comp rx=0 tx=0).stop 2024-03-21T10:22:31.352 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:31.350+0000 7f478ba00640 1 -- 172.21.15.47:0/3857899676 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4784104a70 msgr2=0x7f47841000f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:22:31.352 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:31.350+0000 7f478ba00640 1 --2- 172.21.15.47:0/3857899676 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4784104a70 0x7f47841000f0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f4770005fd0 tx=0x7f477000dde0 comp rx=0 tx=0).stop 2024-03-21T10:22:31.352 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:31.350+0000 7f478ba00640 1 -- 172.21.15.47:0/3857899676 shutdown_connections 2024-03-21T10:22:31.352 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:31.350+0000 7f478ba00640 1 --2- 172.21.15.47:0/3857899676 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f476c0a2fb0 0x7f476c0a5470 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:22:31.352 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:31.350+0000 7f478ba00640 1 --2- 172.21.15.47:0/3857899676 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f47841073a0 0x7f4784100630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:22:31.352 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:31.350+0000 7f478ba00640 1 --2- 172.21.15.47:0/3857899676 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4784104a70 0x7f47841000f0 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:22:31.352 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:31.350+0000 7f478ba00640 1 --2- 172.21.15.47:0/3857899676 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4784102140 0x7f4784101ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:22:31.352 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:31.350+0000 7f478ba00640 1 -- 172.21.15.47:0/3857899676 >> 172.21.15.47:0/3857899676 conn(0x7f47840fbb30 msgr2=0x7f4784102be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:22:31.352 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:31.351+0000 7f478ba00640 1 -- 172.21.15.47:0/3857899676 shutdown_connections 2024-03-21T10:22:31.352 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:31.351+0000 7f478ba00640 1 -- 172.21.15.47:0/3857899676 wait complete. 2024-03-21T10:22:31.371 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-03-21T10:22:31.372 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-21T10:22:31.635 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:31.632+0000 7f9a36000640 1 -- 172.21.15.47:0/749209299 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9a301096b0 msgr2=0x7f9a3010ff40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:22:31.635 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:31.632+0000 7f9a36000640 1 --2- 172.21.15.47:0/749209299 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9a301096b0 0x7f9a3010ff40 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f9a28009ca0 tx=0x7f9a2802c0f0 comp rx=0 tx=0).stop 2024-03-21T10:22:31.635 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:31.632+0000 7f9a36000640 1 -- 172.21.15.47:0/749209299 shutdown_connections 2024-03-21T10:22:31.635 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:31.632+0000 7f9a36000640 1 --2- 172.21.15.47:0/749209299 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9a301096b0 0x7f9a3010ff40 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:22:31.635 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:31.632+0000 7f9a36000640 1 --2- 172.21.15.47:0/749209299 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9a30108cf0 0x7f9a30109170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:22:31.635 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:31.633+0000 7f9a36000640 1 --2- 172.21.15.47:0/749209299 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9a30107b00 0x7f9a30107f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:22:31.635 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:31.633+0000 7f9a36000640 1 -- 172.21.15.47:0/749209299 >> 172.21.15.47:0/749209299 conn(0x7f9a30076ab0 msgr2=0x7f9a30078ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:22:31.636 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:31.633+0000 7f9a36000640 1 -- 172.21.15.47:0/749209299 shutdown_connections 2024-03-21T10:22:31.636 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:31.633+0000 7f9a36000640 1 -- 172.21.15.47:0/749209299 wait complete. 2024-03-21T10:22:31.639 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:31.634+0000 7f9a36000640 1 Processor -- start 2024-03-21T10:22:31.640 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:31.634+0000 7f9a36000640 1 -- start start 2024-03-21T10:22:31.640 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:31.635+0000 7f9a36000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9a30107b00 0x7f9a3019dcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:22:31.640 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:31.635+0000 7f9a36000640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9a30108cf0 0x7f9a3019e230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:22:31.640 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:31.635+0000 7f9a36000640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9a301096b0 0x7f9a301a5260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:22:31.640 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:31.635+0000 7f9a36000640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a30113630 con 0x7f9a30108cf0 2024-03-21T10:22:31.640 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:31.635+0000 7f9a36000640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a301134b0 con 0x7f9a30107b00 2024-03-21T10:22:31.640 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:31.635+0000 7f9a36000640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a301137b0 con 0x7f9a301096b0 2024-03-21T10:22:31.641 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:31.635+0000 7f9a2fe00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9a30108cf0 0x7f9a3019e230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:22:31.641 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:31.635+0000 7f9a34c00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9a30107b00 0x7f9a3019dcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:22:31.641 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:31.635+0000 7f9a34c00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9a30107b00 0x7f9a3019dcf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:51812/0 (socket says 172.21.15.47:51812) 2024-03-21T10:22:31.641 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:31.635+0000 7f9a34c00640 1 -- 172.21.15.47:0/1031284601 learned_addr learned my addr 172.21.15.47:0/1031284601 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:22:31.641 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:31.635+0000 7f9a35600640 1 --2- 172.21.15.47:0/1031284601 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9a301096b0 0x7f9a301a5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:22:31.641 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:31.636+0000 7f9a2fe00640 1 -- 172.21.15.47:0/1031284601 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9a301096b0 msgr2=0x7f9a301a5260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:22:31.641 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:31.636+0000 7f9a2fe00640 1 --2- 172.21.15.47:0/1031284601 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9a301096b0 0x7f9a301a5260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:22:31.642 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:31.636+0000 7f9a2fe00640 1 -- 172.21.15.47:0/1031284601 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9a30107b00 msgr2=0x7f9a3019dcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:22:31.642 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:31.636+0000 7f9a2fe00640 1 --2- 172.21.15.47:0/1031284601 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9a30107b00 0x7f9a3019dcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:22:31.642 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:31.636+0000 7f9a2fe00640 1 -- 172.21.15.47:0/1031284601 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a301a5900 con 0x7f9a30108cf0 2024-03-21T10:22:31.642 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:31.636+0000 7f9a2fe00640 1 --2- 172.21.15.47:0/1031284601 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9a30108cf0 0x7f9a3019e230 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f9a1c00e9f0 tx=0x7f9a1c00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:22:31.642 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:31.636+0000 7f9a2d600640 1 -- 172.21.15.47:0/1031284601 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a1c00ccc0 con 0x7f9a30108cf0 2024-03-21T10:22:31.642 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:31.636+0000 7f9a2d600640 1 -- 172.21.15.47:0/1031284601 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f9a1c0108e0 con 0x7f9a30108cf0 2024-03-21T10:22:31.642 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:31.636+0000 7f9a36000640 1 -- 172.21.15.47:0/1031284601 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9a301a5bf0 con 0x7f9a30108cf0 2024-03-21T10:22:31.643 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:31.636+0000 7f9a2d600640 1 -- 172.21.15.47:0/1031284601 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a1c018880 con 0x7f9a30108cf0 2024-03-21T10:22:31.643 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:31.636+0000 7f9a36000640 1 -- 172.21.15.47:0/1031284601 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9a301a6140 con 0x7f9a30108cf0 2024-03-21T10:22:31.643 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:31.638+0000 7f9a36000640 1 -- 172.21.15.47:0/1031284601 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9a3010ee20 con 0x7f9a30108cf0 2024-03-21T10:22:31.644 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:31.640+0000 7f9a2d600640 1 -- 172.21.15.47:0/1031284601 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 17) v1 ==== 137905+0+0 (secure 0 0 0) 0x7f9a1c010a90 con 0x7f9a30108cf0 2024-03-21T10:22:31.644 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:31.643+0000 7f9a2d600640 1 --2- 172.21.15.47:0/1031284601 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f9a180a2fb0 0x7f9a180a5470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:22:31.644 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:31.643+0000 7f9a2d600640 1 -- 172.21.15.47:0/1031284601 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9092+0+0 (secure 0 0 0) 0x7f9a1c0cfd60 con 0x7f9a30108cf0 2024-03-21T10:22:31.645 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:31.643+0000 7f9a34c00640 1 --2- 172.21.15.47:0/1031284601 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f9a180a2fb0 0x7f9a180a5470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:22:31.645 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:31.644+0000 7f9a34c00640 1 --2- 172.21.15.47:0/1031284601 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f9a180a2fb0 0x7f9a180a5470 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f9a3019f1c0 tx=0x7f9a2400b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:22:31.647 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:31.646+0000 7f9a2d600640 1 -- 172.21.15.47:0/1031284601 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f9a1c09c6d0 con 0x7f9a30108cf0 2024-03-21T10:22:31.714 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:22:31 smithi047 ceph-mon[31547]: pgmap v385: 129 pgs: 129 active+clean; 591 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 KiB/s rd, 5.7 KiB/s wr, 16 op/s; 557 B/s, 0 objects/s recovering 2024-03-21T10:22:31.714 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:22:31 smithi047 ceph-mon[31547]: from='client.14898 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:22:31.714 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:22:31 smithi047 ceph-mon[31547]: mgrmap e17: x(active, since 10m), standbys: z, y 2024-03-21T10:22:31.714 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:22:31 smithi047 ceph-mon[31547]: from='client.? 172.21.15.47:0/1096767453' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-21T10:22:31.714 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:22:31 smithi047 ceph-mon[31547]: from='client.? 172.21.15.47:0/3857899676' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-21T10:22:31.848 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:31.847+0000 7f9a36000640 1 -- 172.21.15.47:0/1031284601 --> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f9a30061ae0 con 0x7f9a180a2fb0 2024-03-21T10:22:31.881 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:31.880+0000 7f9a2d600640 1 -- 172.21.15.47:0/1031284601 <== mgr.14152 v2:172.21.15.47:6800/2823885994 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+340479 (secure 0 0 0) 0x7f9a30061ae0 con 0x7f9a180a2fb0 2024-03-21T10:22:31.887 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-21T10:22:31.889 INFO:teuthology.orchestra.run.smithi047.stderr:dumped all 2024-03-21T10:22:31.891 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:31.888+0000 7f9a36000640 1 -- 172.21.15.47:0/1031284601 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f9a180a2fb0 msgr2=0x7f9a180a5470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:22:31.891 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:31.888+0000 7f9a36000640 1 --2- 172.21.15.47:0/1031284601 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f9a180a2fb0 0x7f9a180a5470 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f9a3019f1c0 tx=0x7f9a2400b410 comp rx=0 tx=0).stop 2024-03-21T10:22:31.891 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:31.888+0000 7f9a36000640 1 -- 172.21.15.47:0/1031284601 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9a30108cf0 msgr2=0x7f9a3019e230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:22:31.891 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:31.888+0000 7f9a36000640 1 --2- 172.21.15.47:0/1031284601 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9a30108cf0 0x7f9a3019e230 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f9a1c00e9f0 tx=0x7f9a1c00eeb0 comp rx=0 tx=0).stop 2024-03-21T10:22:31.891 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:31.888+0000 7f9a36000640 1 -- 172.21.15.47:0/1031284601 shutdown_connections 2024-03-21T10:22:31.891 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:31.888+0000 7f9a36000640 1 --2- 172.21.15.47:0/1031284601 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f9a180a2fb0 0x7f9a180a5470 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:22:31.891 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:31.888+0000 7f9a36000640 1 --2- 172.21.15.47:0/1031284601 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9a301096b0 0x7f9a301a5260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:22:31.892 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:31.888+0000 7f9a36000640 1 --2- 172.21.15.47:0/1031284601 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9a30108cf0 0x7f9a3019e230 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:22:31.892 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:31.888+0000 7f9a36000640 1 --2- 172.21.15.47:0/1031284601 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9a30107b00 0x7f9a3019dcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:22:31.892 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:31.888+0000 7f9a36000640 1 -- 172.21.15.47:0/1031284601 >> 172.21.15.47:0/1031284601 conn(0x7f9a30076ab0 msgr2=0x7f9a3010bff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:22:31.892 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:31.888+0000 7f9a36000640 1 -- 172.21.15.47:0/1031284601 shutdown_connections 2024-03-21T10:22:31.892 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:31.888+0000 7f9a36000640 1 -- 172.21.15.47:0/1031284601 wait complete. 2024-03-21T10:22:31.908 INFO:teuthology.orchestra.run.smithi047.stdout:{"pg_ready":true,"pg_map":{"version":385,"stamp":"2024-03-21T10:22:30.027783+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":605601,"num_objects":77,"num_object_clones":0,"num_object_copies":231,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":77,"num_whiteouts":0,"num_read":150,"num_read_kb":131,"num_write":348,"num_write_kb":3002,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":1185200,"num_keys_recovered":0,"num_objects_omap":49,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":281,"ondisk_log_size":281,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":396524,"kb_used_data":14040,"kb_used_omap":18,"kb_used_meta":382317,"kb_avail":1124545300,"statfs":{"total":1151940427776,"available":1151534387200,"internally_reserved":0,"allocated":14376960,"data_stored":7791920,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19142,"internal_metadata":391492922},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":17,"apply_latency_ms":17,"commit_latency_ns":17000000,"apply_latency_ns":17000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":10734,"num_objects":53,"num_object_clones":0,"num_object_copies":159,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":53,"num_whiteouts":0,"num_read":14,"num_read_kb":12,"num_write":119,"num_write_kb":46,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":4464,"num_keys_recovered":0,"num_objects_omap":35,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.004267"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":25,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-21T10:22:27.219990+0000","last_change":"2024-03-21T10:21:58.872480+0000","last_active":"2024-03-21T10:22:27.219990+0000","last_peered":"2024-03-21T10:22:27.219990+0000","last_clean":"2024-03-21T10:22:27.219990+0000","last_became_active":"2024-03-21T10:21:58.872143+0000","last_became_peered":"2024-03-21T10:21:58.872143+0000","last_unstale":"2024-03-21T10:22:27.219990+0000","last_undegraded":"2024-03-21T10:22:27.219990+0000","last_fullsized":"2024-03-21T10:22:27.219990+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_clean_scrub_stamp":"2024-03-21T10:21:57.813712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T15:20:00.409201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":25,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-21T10:22:27.210964+0000","last_change":"2024-03-21T10:21:58.871130+0000","last_active":"2024-03-21T10:22:27.210964+0000","last_peered":"2024-03-21T10:22:27.210964+0000","last_clean":"2024-03-21T10:22:27.210964+0000","last_became_active":"2024-03-21T10:21:58.870750+0000","last_became_peered":"2024-03-21T10:21:58.870750+0000","last_unstale":"2024-03-21T10:22:27.210964+0000","last_undegraded":"2024-03-21T10:22:27.210964+0000","last_fullsized":"2024-03-21T10:22:27.210964+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_clean_scrub_stamp":"2024-03-21T10:21:57.813712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T17:38:04.830212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":25,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-21T10:22:27.210597+0000","last_change":"2024-03-21T10:21:58.868026+0000","last_active":"2024-03-21T10:22:27.210597+0000","last_peered":"2024-03-21T10:22:27.210597+0000","last_clean":"2024-03-21T10:22:27.210597+0000","last_became_active":"2024-03-21T10:21:58.859877+0000","last_became_peered":"2024-03-21T10:21:58.859877+0000","last_unstale":"2024-03-21T10:22:27.210597+0000","last_undegraded":"2024-03-21T10:22:27.210597+0000","last_fullsized":"2024-03-21T10:22:27.210597+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_clean_scrub_stamp":"2024-03-21T10:21:57.813712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T14:45:12.870111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":11,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-21T10:22:27.232499+0000","last_change":"2024-03-21T10:22:27.231695+0000","last_active":"2024-03-21T10:22:27.232499+0000","last_peered":"2024-03-21T10:22:27.232499+0000","last_clean":"2024-03-21T10:22:27.232499+0000","last_became_active":"2024-03-21T10:22:27.231149+0000","last_became_peered":"2024-03-21T10:22:27.231149+0000","last_unstale":"2024-03-21T10:22:27.232499+0000","last_undegraded":"2024-03-21T10:22:27.232499+0000","last_fullsized":"2024-03-21T10:22:27.232499+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_clean_scrub_stamp":"2024-03-21T10:21:57.813712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T18:19:19.105366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,2],"acting":[9,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":25,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-21T10:22:27.228786+0000","last_change":"2024-03-21T10:21:58.853998+0000","last_active":"2024-03-21T10:22:27.228786+0000","last_peered":"2024-03-21T10:22:27.228786+0000","last_clean":"2024-03-21T10:22:27.228786+0000","last_became_active":"2024-03-21T10:21:58.853059+0000","last_became_peered":"2024-03-21T10:21:58.853059+0000","last_unstale":"2024-03-21T10:22:27.228786+0000","last_undegraded":"2024-03-21T10:22:27.228786+0000","last_fullsized":"2024-03-21T10:22:27.228786+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_clean_scrub_stamp":"2024-03-21T10:21:57.813712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T17:06:36.859522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":25,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-21T10:22:27.208014+0000","last_change":"2024-03-21T10:21:58.855535+0000","last_active":"2024-03-21T10:22:27.208014+0000","last_peered":"2024-03-21T10:22:27.208014+0000","last_clean":"2024-03-21T10:22:27.208014+0000","last_became_active":"2024-03-21T10:21:58.855181+0000","last_became_peered":"2024-03-21T10:21:58.855181+0000","last_unstale":"2024-03-21T10:22:27.208014+0000","last_undegraded":"2024-03-21T10:22:27.208014+0000","last_fullsized":"2024-03-21T10:22:27.208014+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_clean_scrub_stamp":"2024-03-21T10:21:57.813712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T14:46:35.045646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":25,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-21T10:22:27.215800+0000","last_change":"2024-03-21T10:21:58.856527+0000","last_active":"2024-03-21T10:22:27.215800+0000","last_peered":"2024-03-21T10:22:27.215800+0000","last_clean":"2024-03-21T10:22:27.215800+0000","last_became_active":"2024-03-21T10:21:58.856176+0000","last_became_peered":"2024-03-21T10:21:58.856176+0000","last_unstale":"2024-03-21T10:22:27.215800+0000","last_undegraded":"2024-03-21T10:22:27.215800+0000","last_fullsized":"2024-03-21T10:22:27.215800+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_clean_scrub_stamp":"2024-03-21T10:21:57.813712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T21:34:07.804293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":25,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-21T10:22:27.223195+0000","last_change":"2024-03-21T10:21:58.846260+0000","last_active":"2024-03-21T10:22:27.223195+0000","last_peered":"2024-03-21T10:22:27.223195+0000","last_clean":"2024-03-21T10:22:27.223195+0000","last_became_active":"2024-03-21T10:21:58.844245+0000","last_became_peered":"2024-03-21T10:21:58.844245+0000","last_unstale":"2024-03-21T10:22:27.223195+0000","last_undegraded":"2024-03-21T10:22:27.223195+0000","last_fullsized":"2024-03-21T10:22:27.223195+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_clean_scrub_stamp":"2024-03-21T10:21:57.813712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T16:37:07.480867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":23,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-21T10:22:26.196665+0000","last_change":"2024-03-21T10:21:58.858413+0000","last_active":"2024-03-21T10:22:26.196665+0000","last_peered":"2024-03-21T10:22:26.196665+0000","last_clean":"2024-03-21T10:22:26.196665+0000","last_became_active":"2024-03-21T10:21:58.858007+0000","last_became_peered":"2024-03-21T10:21:58.858007+0000","last_unstale":"2024-03-21T10:22:26.196665+0000","last_undegraded":"2024-03-21T10:22:26.196665+0000","last_fullsized":"2024-03-21T10:22:26.196665+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_clean_scrub_stamp":"2024-03-21T10:21:57.813712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T21:49:12.428918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":25,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-21T10:22:27.209745+0000","last_change":"2024-03-21T10:21:58.864393+0000","last_active":"2024-03-21T10:22:27.209745+0000","last_peered":"2024-03-21T10:22:27.209745+0000","last_clean":"2024-03-21T10:22:27.209745+0000","last_became_active":"2024-03-21T10:21:58.864061+0000","last_became_peered":"2024-03-21T10:21:58.864061+0000","last_unstale":"2024-03-21T10:22:27.209745+0000","last_undegraded":"2024-03-21T10:22:27.209745+0000","last_fullsized":"2024-03-21T10:22:27.209745+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_clean_scrub_stamp":"2024-03-21T10:21:57.813712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T20:19:44.209612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":25,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-21T10:22:27.216667+0000","last_change":"2024-03-21T10:21:58.873885+0000","last_active":"2024-03-21T10:22:27.216667+0000","last_peered":"2024-03-21T10:22:27.216667+0000","last_clean":"2024-03-21T10:22:27.216667+0000","last_became_active":"2024-03-21T10:21:58.869738+0000","last_became_peered":"2024-03-21T10:21:58.869738+0000","last_unstale":"2024-03-21T10:22:27.216667+0000","last_undegraded":"2024-03-21T10:22:27.216667+0000","last_fullsized":"2024-03-21T10:22:27.216667+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_clean_scrub_stamp":"2024-03-21T10:21:57.813712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T13:33:26.816720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":25,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-21T10:22:27.203520+0000","last_change":"2024-03-21T10:21:58.867563+0000","last_active":"2024-03-21T10:22:27.203520+0000","last_peered":"2024-03-21T10:22:27.203520+0000","last_clean":"2024-03-21T10:22:27.203520+0000","last_became_active":"2024-03-21T10:21:58.863455+0000","last_became_peered":"2024-03-21T10:21:58.863455+0000","last_unstale":"2024-03-21T10:22:27.203520+0000","last_undegraded":"2024-03-21T10:22:27.203520+0000","last_fullsized":"2024-03-21T10:22:27.203520+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_clean_scrub_stamp":"2024-03-21T10:21:57.813712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T15:04:16.636126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":31,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-21T10:22:27.213460+0000","last_change":"2024-03-21T10:22:27.213460+0000","last_active":"2024-03-21T10:22:27.213460+0000","last_peered":"2024-03-21T10:22:27.213460+0000","last_clean":"2024-03-21T10:22:27.213460+0000","last_became_active":"2024-03-21T10:22:27.212961+0000","last_became_peered":"2024-03-21T10:22:27.212961+0000","last_unstale":"2024-03-21T10:22:27.213460+0000","last_undegraded":"2024-03-21T10:22:27.213460+0000","last_fullsized":"2024-03-21T10:22:27.213460+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_clean_scrub_stamp":"2024-03-21T10:21:57.813712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T18:26:03.881767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,9],"acting":[6,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":25,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-21T10:22:27.207773+0000","last_change":"2024-03-21T10:21:58.870851+0000","last_active":"2024-03-21T10:22:27.207773+0000","last_peered":"2024-03-21T10:22:27.207773+0000","last_clean":"2024-03-21T10:22:27.207773+0000","last_became_active":"2024-03-21T10:21:58.870132+0000","last_became_peered":"2024-03-21T10:21:58.870132+0000","last_unstale":"2024-03-21T10:22:27.207773+0000","last_undegraded":"2024-03-21T10:22:27.207773+0000","last_fullsized":"2024-03-21T10:22:27.207773+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_clean_scrub_stamp":"2024-03-21T10:21:57.813712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T20:20:04.207090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":25,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-21T10:22:27.208023+0000","last_change":"2024-03-21T10:21:58.865084+0000","last_active":"2024-03-21T10:22:27.208023+0000","last_peered":"2024-03-21T10:22:27.208023+0000","last_clean":"2024-03-21T10:22:27.208023+0000","last_became_active":"2024-03-21T10:21:58.864583+0000","last_became_peered":"2024-03-21T10:21:58.864583+0000","last_unstale":"2024-03-21T10:22:27.208023+0000","last_undegraded":"2024-03-21T10:22:27.208023+0000","last_fullsized":"2024-03-21T10:22:27.208023+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_clean_scrub_stamp":"2024-03-21T10:21:57.813712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T14:09:16.781462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":23,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-21T10:22:26.191520+0000","last_change":"2024-03-21T10:21:58.870370+0000","last_active":"2024-03-21T10:22:26.191520+0000","last_peered":"2024-03-21T10:22:26.191520+0000","last_clean":"2024-03-21T10:22:26.191520+0000","last_became_active":"2024-03-21T10:21:58.870044+0000","last_became_peered":"2024-03-21T10:21:58.870044+0000","last_unstale":"2024-03-21T10:22:26.191520+0000","last_undegraded":"2024-03-21T10:22:26.191520+0000","last_fullsized":"2024-03-21T10:22:26.191520+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_clean_scrub_stamp":"2024-03-21T10:21:57.813712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T16:02:00.342426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":25,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-21T10:22:27.219320+0000","last_change":"2024-03-21T10:21:58.872787+0000","last_active":"2024-03-21T10:22:27.219320+0000","last_peered":"2024-03-21T10:22:27.219320+0000","last_clean":"2024-03-21T10:22:27.219320+0000","last_became_active":"2024-03-21T10:21:58.872390+0000","last_became_peered":"2024-03-21T10:21:58.872390+0000","last_unstale":"2024-03-21T10:22:27.219320+0000","last_undegraded":"2024-03-21T10:22:27.219320+0000","last_fullsized":"2024-03-21T10:22:27.219320+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_clean_scrub_stamp":"2024-03-21T10:21:57.813712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T15:56:51.798173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":25,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-21T10:22:27.208635+0000","last_change":"2024-03-21T10:21:58.858474+0000","last_active":"2024-03-21T10:22:27.208635+0000","last_peered":"2024-03-21T10:22:27.208635+0000","last_clean":"2024-03-21T10:22:27.208635+0000","last_became_active":"2024-03-21T10:21:58.858076+0000","last_became_peered":"2024-03-21T10:21:58.858076+0000","last_unstale":"2024-03-21T10:22:27.208635+0000","last_undegraded":"2024-03-21T10:22:27.208635+0000","last_fullsized":"2024-03-21T10:22:27.208635+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_clean_scrub_stamp":"2024-03-21T10:21:57.813712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T12:25:17.160405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":25,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-21T10:22:27.213834+0000","last_change":"2024-03-21T10:21:58.854156+0000","last_active":"2024-03-21T10:22:27.213834+0000","last_peered":"2024-03-21T10:22:27.213834+0000","last_clean":"2024-03-21T10:22:27.213834+0000","last_became_active":"2024-03-21T10:21:58.853758+0000","last_became_peered":"2024-03-21T10:21:58.853758+0000","last_unstale":"2024-03-21T10:22:27.213834+0000","last_undegraded":"2024-03-21T10:22:27.213834+0000","last_fullsized":"2024-03-21T10:22:27.213834+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_clean_scrub_stamp":"2024-03-21T10:21:57.813712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T16:58:54.854826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":18,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-21T10:22:27.214010+0000","last_change":"2024-03-21T10:22:03.319388+0000","last_active":"2024-03-21T10:22:27.214010+0000","last_peered":"2024-03-21T10:22:27.214010+0000","last_clean":"2024-03-21T10:22:27.214010+0000","last_became_active":"2024-03-21T10:22:03.318846+0000","last_became_peered":"2024-03-21T10:22:03.318846+0000","last_unstale":"2024-03-21T10:22:27.214010+0000","last_undegraded":"2024-03-21T10:22:27.214010+0000","last_fullsized":"2024-03-21T10:22:27.214010+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:22:02.269614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:22:02.269614+0000","last_clean_scrub_stamp":"2024-03-21T10:22:02.269614+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T17:28:16.298825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"92'9","reported_seq":36,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-21T10:22:27.209988+0000","last_change":"2024-03-21T10:21:57.859554+0000","last_active":"2024-03-21T10:22:27.209988+0000","last_peered":"2024-03-21T10:22:27.209988+0000","last_clean":"2024-03-21T10:22:27.209988+0000","last_became_active":"2024-03-21T10:21:57.857997+0000","last_became_peered":"2024-03-21T10:21:57.857997+0000","last_unstale":"2024-03-21T10:22:27.209988+0000","last_undegraded":"2024-03-21T10:22:27.209988+0000","last_fullsized":"2024-03-21T10:22:27.209988+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:21:56.788908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:21:56.788908+0000","last_clean_scrub_stamp":"2024-03-21T10:21:56.788908+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T14:27:47.516832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":25,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-21T10:22:27.224817+0000","last_change":"2024-03-21T10:21:58.850197+0000","last_active":"2024-03-21T10:22:27.224817+0000","last_peered":"2024-03-21T10:22:27.224817+0000","last_clean":"2024-03-21T10:22:27.224817+0000","last_became_active":"2024-03-21T10:21:58.849029+0000","last_became_peered":"2024-03-21T10:21:58.849029+0000","last_unstale":"2024-03-21T10:22:27.224817+0000","last_undegraded":"2024-03-21T10:22:27.224817+0000","last_fullsized":"2024-03-21T10:22:27.224817+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_clean_scrub_stamp":"2024-03-21T10:21:57.813712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T15:46:49.077382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":18,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-21T10:22:27.202873+0000","last_change":"2024-03-21T10:22:03.320716+0000","last_active":"2024-03-21T10:22:27.202873+0000","last_peered":"2024-03-21T10:22:27.202873+0000","last_clean":"2024-03-21T10:22:27.202873+0000","last_became_active":"2024-03-21T10:22:03.320262+0000","last_became_peered":"2024-03-21T10:22:03.320262+0000","last_unstale":"2024-03-21T10:22:27.202873+0000","last_undegraded":"2024-03-21T10:22:27.202873+0000","last_fullsized":"2024-03-21T10:22:27.202873+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:22:02.269614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:22:02.269614+0000","last_clean_scrub_stamp":"2024-03-21T10:22:02.269614+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T16:16:07.423168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"92'2","reported_seq":29,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-21T10:22:27.225025+0000","last_change":"2024-03-21T10:21:57.863642+0000","last_active":"2024-03-21T10:22:27.225025+0000","last_peered":"2024-03-21T10:22:27.225025+0000","last_clean":"2024-03-21T10:22:27.225025+0000","last_became_active":"2024-03-21T10:21:57.863330+0000","last_became_peered":"2024-03-21T10:21:57.863330+0000","last_unstale":"2024-03-21T10:22:27.225025+0000","last_undegraded":"2024-03-21T10:22:27.225025+0000","last_fullsized":"2024-03-21T10:22:27.225025+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:21:56.788908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:21:56.788908+0000","last_clean_scrub_stamp":"2024-03-21T10:21:56.788908+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T14:45:02.131458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":25,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-21T10:22:27.211017+0000","last_change":"2024-03-21T10:21:58.862940+0000","last_active":"2024-03-21T10:22:27.211017+0000","last_peered":"2024-03-21T10:22:27.211017+0000","last_clean":"2024-03-21T10:22:27.211017+0000","last_became_active":"2024-03-21T10:21:58.862489+0000","last_became_peered":"2024-03-21T10:21:58.862489+0000","last_unstale":"2024-03-21T10:22:27.211017+0000","last_undegraded":"2024-03-21T10:22:27.211017+0000","last_fullsized":"2024-03-21T10:22:27.211017+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_clean_scrub_stamp":"2024-03-21T10:21:57.813712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T19:28:02.175642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":18,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-21T10:22:27.203272+0000","last_change":"2024-03-21T10:22:03.318660+0000","last_active":"2024-03-21T10:22:27.203272+0000","last_peered":"2024-03-21T10:22:27.203272+0000","last_clean":"2024-03-21T10:22:27.203272+0000","last_became_active":"2024-03-21T10:22:03.318216+0000","last_became_peered":"2024-03-21T10:22:03.318216+0000","last_unstale":"2024-03-21T10:22:27.203272+0000","last_undegraded":"2024-03-21T10:22:27.203272+0000","last_fullsized":"2024-03-21T10:22:27.203272+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:22:02.269614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:22:02.269614+0000","last_clean_scrub_stamp":"2024-03-21T10:22:02.269614+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T16:36:39.930726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"93'4","reported_seq":17,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-21T10:22:27.240020+0000","last_change":"2024-03-21T10:22:27.239342+0000","last_active":"2024-03-21T10:22:27.240020+0000","last_peered":"2024-03-21T10:22:27.240020+0000","last_clean":"2024-03-21T10:22:27.240020+0000","last_became_active":"2024-03-21T10:22:27.224086+0000","last_became_peered":"2024-03-21T10:22:27.224086+0000","last_unstale":"2024-03-21T10:22:27.240020+0000","last_undegraded":"2024-03-21T10:22:27.240020+0000","last_fullsized":"2024-03-21T10:22:27.240020+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:21:56.788908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:21:56.788908+0000","last_clean_scrub_stamp":"2024-03-21T10:21:56.788908+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T20:46:45.951485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,7],"acting":[11,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":23,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-21T10:22:26.192253+0000","last_change":"2024-03-21T10:21:58.873239+0000","last_active":"2024-03-21T10:22:26.192253+0000","last_peered":"2024-03-21T10:22:26.192253+0000","last_clean":"2024-03-21T10:22:26.192253+0000","last_became_active":"2024-03-21T10:21:58.872877+0000","last_became_peered":"2024-03-21T10:21:58.872877+0000","last_unstale":"2024-03-21T10:22:26.192253+0000","last_undegraded":"2024-03-21T10:22:26.192253+0000","last_fullsized":"2024-03-21T10:22:26.192253+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_clean_scrub_stamp":"2024-03-21T10:21:57.813712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T16:22:49.070697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":18,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-21T10:22:27.202999+0000","last_change":"2024-03-21T10:22:03.311589+0000","last_active":"2024-03-21T10:22:27.202999+0000","last_peered":"2024-03-21T10:22:27.202999+0000","last_clean":"2024-03-21T10:22:27.202999+0000","last_became_active":"2024-03-21T10:22:03.311134+0000","last_became_peered":"2024-03-21T10:22:03.311134+0000","last_unstale":"2024-03-21T10:22:27.202999+0000","last_undegraded":"2024-03-21T10:22:27.202999+0000","last_fullsized":"2024-03-21T10:22:27.202999+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:22:02.269614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:22:02.269614+0000","last_clean_scrub_stamp":"2024-03-21T10:22:02.269614+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T10:22:29.138348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"86'2","reported_seq":29,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-21T10:22:27.232300+0000","last_change":"2024-03-21T10:21:57.858981+0000","last_active":"2024-03-21T10:22:27.232300+0000","last_peered":"2024-03-21T10:22:27.232300+0000","last_clean":"2024-03-21T10:22:27.232300+0000","last_became_active":"2024-03-21T10:21:57.858564+0000","last_became_peered":"2024-03-21T10:21:57.858564+0000","last_unstale":"2024-03-21T10:22:27.232300+0000","last_undegraded":"2024-03-21T10:22:27.232300+0000","last_fullsized":"2024-03-21T10:22:27.232300+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:21:56.788908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:21:56.788908+0000","last_clean_scrub_stamp":"2024-03-21T10:21:56.788908+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T10:45:09.488194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":25,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-21T10:22:27.206512+0000","last_change":"2024-03-21T10:21:58.858053+0000","last_active":"2024-03-21T10:22:27.206512+0000","last_peered":"2024-03-21T10:22:27.206512+0000","last_clean":"2024-03-21T10:22:27.206512+0000","last_became_active":"2024-03-21T10:21:58.857688+0000","last_became_peered":"2024-03-21T10:21:58.857688+0000","last_unstale":"2024-03-21T10:22:27.206512+0000","last_undegraded":"2024-03-21T10:22:27.206512+0000","last_fullsized":"2024-03-21T10:22:27.206512+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_clean_scrub_stamp":"2024-03-21T10:21:57.813712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T19:45:13.043034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":18,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-21T10:22:27.213571+0000","last_change":"2024-03-21T10:22:03.315315+0000","last_active":"2024-03-21T10:22:27.213571+0000","last_peered":"2024-03-21T10:22:27.213571+0000","last_clean":"2024-03-21T10:22:27.213571+0000","last_became_active":"2024-03-21T10:22:03.314861+0000","last_became_peered":"2024-03-21T10:22:03.314861+0000","last_unstale":"2024-03-21T10:22:27.213571+0000","last_undegraded":"2024-03-21T10:22:27.213571+0000","last_fullsized":"2024-03-21T10:22:27.213571+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:22:02.269614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:22:02.269614+0000","last_clean_scrub_stamp":"2024-03-21T10:22:02.269614+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T22:13:09.417907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"91'5","reported_seq":44,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-21T10:22:27.235744+0000","last_change":"2024-03-21T10:22:27.235744+0000","last_active":"2024-03-21T10:22:27.235744+0000","last_peered":"2024-03-21T10:22:27.235744+0000","last_clean":"2024-03-21T10:22:27.235744+0000","last_became_active":"2024-03-21T10:22:27.219999+0000","last_became_peered":"2024-03-21T10:22:27.219999+0000","last_unstale":"2024-03-21T10:22:27.235744+0000","last_undegraded":"2024-03-21T10:22:27.235744+0000","last_fullsized":"2024-03-21T10:22:27.235744+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:21:56.788908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:21:56.788908+0000","last_clean_scrub_stamp":"2024-03-21T10:21:56.788908+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T18:15:41.288034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":622,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7],"acting":[5,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":25,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-21T10:22:27.229634+0000","last_change":"2024-03-21T10:21:58.854752+0000","last_active":"2024-03-21T10:22:27.229634+0000","last_peered":"2024-03-21T10:22:27.229634+0000","last_clean":"2024-03-21T10:22:27.229634+0000","last_became_active":"2024-03-21T10:21:58.854442+0000","last_became_peered":"2024-03-21T10:21:58.854442+0000","last_unstale":"2024-03-21T10:22:27.229634+0000","last_undegraded":"2024-03-21T10:22:27.229634+0000","last_fullsized":"2024-03-21T10:22:27.229634+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_clean_scrub_stamp":"2024-03-21T10:21:57.813712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T15:12:40.590585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":18,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-21T10:22:27.213953+0000","last_change":"2024-03-21T10:22:03.319247+0000","last_active":"2024-03-21T10:22:27.213953+0000","last_peered":"2024-03-21T10:22:27.213953+0000","last_clean":"2024-03-21T10:22:27.213953+0000","last_became_active":"2024-03-21T10:22:03.318731+0000","last_became_peered":"2024-03-21T10:22:03.318731+0000","last_unstale":"2024-03-21T10:22:27.213953+0000","last_undegraded":"2024-03-21T10:22:27.213953+0000","last_fullsized":"2024-03-21T10:22:27.213953+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:22:02.269614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:22:02.269614+0000","last_clean_scrub_stamp":"2024-03-21T10:22:02.269614+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T20:06:36.519559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"93'7","reported_seq":40,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-21T10:22:28.243943+0000","last_change":"2024-03-21T10:22:27.222981+0000","last_active":"2024-03-21T10:22:28.243943+0000","last_peered":"2024-03-21T10:22:28.243943+0000","last_clean":"2024-03-21T10:22:28.243943+0000","last_became_active":"2024-03-21T10:22:27.221088+0000","last_became_peered":"2024-03-21T10:22:27.221088+0000","last_unstale":"2024-03-21T10:22:28.243943+0000","last_undegraded":"2024-03-21T10:22:28.243943+0000","last_fullsized":"2024-03-21T10:22:28.243943+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:21:56.788908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:21:56.788908+0000","last_clean_scrub_stamp":"2024-03-21T10:21:56.788908+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T15:43:11.740353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,6],"acting":[10,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":31,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-21T10:22:27.226733+0000","last_change":"2024-03-21T10:22:27.226733+0000","last_active":"2024-03-21T10:22:27.226733+0000","last_peered":"2024-03-21T10:22:27.226733+0000","last_clean":"2024-03-21T10:22:27.226733+0000","last_became_active":"2024-03-21T10:22:27.225576+0000","last_became_peered":"2024-03-21T10:22:27.225576+0000","last_unstale":"2024-03-21T10:22:27.226733+0000","last_undegraded":"2024-03-21T10:22:27.226733+0000","last_fullsized":"2024-03-21T10:22:27.226733+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_clean_scrub_stamp":"2024-03-21T10:21:57.813712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T20:06:09.534268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,3],"acting":[5,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":18,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-21T10:22:27.218863+0000","last_change":"2024-03-21T10:22:03.321595+0000","last_active":"2024-03-21T10:22:27.218863+0000","last_peered":"2024-03-21T10:22:27.218863+0000","last_clean":"2024-03-21T10:22:27.218863+0000","last_became_active":"2024-03-21T10:22:03.320897+0000","last_became_peered":"2024-03-21T10:22:03.320897+0000","last_unstale":"2024-03-21T10:22:27.218863+0000","last_undegraded":"2024-03-21T10:22:27.218863+0000","last_fullsized":"2024-03-21T10:22:27.218863+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:22:02.269614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:22:02.269614+0000","last_clean_scrub_stamp":"2024-03-21T10:22:02.269614+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T17:37:02.633709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"93'3","reported_seq":30,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-21T10:22:27.202538+0000","last_change":"2024-03-21T10:21:57.854801+0000","last_active":"2024-03-21T10:22:27.202538+0000","last_peered":"2024-03-21T10:22:27.202538+0000","last_clean":"2024-03-21T10:22:27.202538+0000","last_became_active":"2024-03-21T10:21:57.854443+0000","last_became_peered":"2024-03-21T10:21:57.854443+0000","last_unstale":"2024-03-21T10:22:27.202538+0000","last_undegraded":"2024-03-21T10:22:27.202538+0000","last_fullsized":"2024-03-21T10:22:27.202538+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:21:56.788908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:21:56.788908+0000","last_clean_scrub_stamp":"2024-03-21T10:21:56.788908+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T15:49:23.573360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":25,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-21T10:22:27.222969+0000","last_change":"2024-03-21T10:21:58.857167+0000","last_active":"2024-03-21T10:22:27.222969+0000","last_peered":"2024-03-21T10:22:27.222969+0000","last_clean":"2024-03-21T10:22:27.222969+0000","last_became_active":"2024-03-21T10:21:58.856827+0000","last_became_peered":"2024-03-21T10:21:58.856827+0000","last_unstale":"2024-03-21T10:22:27.222969+0000","last_undegraded":"2024-03-21T10:22:27.222969+0000","last_fullsized":"2024-03-21T10:22:27.222969+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_clean_scrub_stamp":"2024-03-21T10:21:57.813712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T15:04:43.848209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":18,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-21T10:22:27.210859+0000","last_change":"2024-03-21T10:22:03.306554+0000","last_active":"2024-03-21T10:22:27.210859+0000","last_peered":"2024-03-21T10:22:27.210859+0000","last_clean":"2024-03-21T10:22:27.210859+0000","last_became_active":"2024-03-21T10:22:03.306087+0000","last_became_peered":"2024-03-21T10:22:03.306087+0000","last_unstale":"2024-03-21T10:22:27.210859+0000","last_undegraded":"2024-03-21T10:22:27.210859+0000","last_fullsized":"2024-03-21T10:22:27.210859+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:22:02.269614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:22:02.269614+0000","last_clean_scrub_stamp":"2024-03-21T10:22:02.269614+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T11:36:35.344770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"91'1","reported_seq":28,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-21T10:22:27.204688+0000","last_change":"2024-03-21T10:21:57.860343+0000","last_active":"2024-03-21T10:22:27.204688+0000","last_peered":"2024-03-21T10:22:27.204688+0000","last_clean":"2024-03-21T10:22:27.204688+0000","last_became_active":"2024-03-21T10:21:57.859808+0000","last_became_peered":"2024-03-21T10:21:57.859808+0000","last_unstale":"2024-03-21T10:22:27.204688+0000","last_undegraded":"2024-03-21T10:22:27.204688+0000","last_fullsized":"2024-03-21T10:22:27.204688+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:21:56.788908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:21:56.788908+0000","last_clean_scrub_stamp":"2024-03-21T10:21:56.788908+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T14:41:24.162795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":25,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-21T10:22:27.208314+0000","last_change":"2024-03-21T10:21:58.850991+0000","last_active":"2024-03-21T10:22:27.208314+0000","last_peered":"2024-03-21T10:22:27.208314+0000","last_clean":"2024-03-21T10:22:27.208314+0000","last_became_active":"2024-03-21T10:21:58.850581+0000","last_became_peered":"2024-03-21T10:21:58.850581+0000","last_unstale":"2024-03-21T10:22:27.208314+0000","last_undegraded":"2024-03-21T10:22:27.208314+0000","last_fullsized":"2024-03-21T10:22:27.208314+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_clean_scrub_stamp":"2024-03-21T10:21:57.813712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T17:07:11.350806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":18,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-21T10:22:27.223291+0000","last_change":"2024-03-21T10:22:03.315333+0000","last_active":"2024-03-21T10:22:27.223291+0000","last_peered":"2024-03-21T10:22:27.223291+0000","last_clean":"2024-03-21T10:22:27.223291+0000","last_became_active":"2024-03-21T10:22:03.313553+0000","last_became_peered":"2024-03-21T10:22:03.313553+0000","last_unstale":"2024-03-21T10:22:27.223291+0000","last_undegraded":"2024-03-21T10:22:27.223291+0000","last_fullsized":"2024-03-21T10:22:27.223291+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:22:02.269614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:22:02.269614+0000","last_clean_scrub_stamp":"2024-03-21T10:22:02.269614+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T15:25:24.290789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"93'4","reported_seq":31,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-21T10:22:27.207857+0000","last_change":"2024-03-21T10:21:57.864930+0000","last_active":"2024-03-21T10:22:27.207857+0000","last_peered":"2024-03-21T10:22:27.207857+0000","last_clean":"2024-03-21T10:22:27.207857+0000","last_became_active":"2024-03-21T10:21:57.863769+0000","last_became_peered":"2024-03-21T10:21:57.863769+0000","last_unstale":"2024-03-21T10:22:27.207857+0000","last_undegraded":"2024-03-21T10:22:27.207857+0000","last_fullsized":"2024-03-21T10:22:27.207857+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:21:56.788908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:21:56.788908+0000","last_clean_scrub_stamp":"2024-03-21T10:21:56.788908+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T21:35:39.144065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":23,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-21T10:22:26.193323+0000","last_change":"2024-03-21T10:21:58.872084+0000","last_active":"2024-03-21T10:22:26.193323+0000","last_peered":"2024-03-21T10:22:26.193323+0000","last_clean":"2024-03-21T10:22:26.193323+0000","last_became_active":"2024-03-21T10:21:58.871725+0000","last_became_peered":"2024-03-21T10:21:58.871725+0000","last_unstale":"2024-03-21T10:22:26.193323+0000","last_undegraded":"2024-03-21T10:22:26.193323+0000","last_fullsized":"2024-03-21T10:22:26.193323+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_clean_scrub_stamp":"2024-03-21T10:21:57.813712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T16:19:37.240053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":18,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-21T10:22:27.207450+0000","last_change":"2024-03-21T10:22:03.317808+0000","last_active":"2024-03-21T10:22:27.207450+0000","last_peered":"2024-03-21T10:22:27.207450+0000","last_clean":"2024-03-21T10:22:27.207450+0000","last_became_active":"2024-03-21T10:22:03.313751+0000","last_became_peered":"2024-03-21T10:22:03.313751+0000","last_unstale":"2024-03-21T10:22:27.207450+0000","last_undegraded":"2024-03-21T10:22:27.207450+0000","last_fullsized":"2024-03-21T10:22:27.207450+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:22:02.269614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:22:02.269614+0000","last_clean_scrub_stamp":"2024-03-21T10:22:02.269614+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T20:05:08.105795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"91'2","reported_seq":29,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-21T10:22:27.216923+0000","last_change":"2024-03-21T10:21:57.865462+0000","last_active":"2024-03-21T10:22:27.216923+0000","last_peered":"2024-03-21T10:22:27.216923+0000","last_clean":"2024-03-21T10:22:27.216923+0000","last_became_active":"2024-03-21T10:21:57.862922+0000","last_became_peered":"2024-03-21T10:21:57.862922+0000","last_unstale":"2024-03-21T10:22:27.216923+0000","last_undegraded":"2024-03-21T10:22:27.216923+0000","last_fullsized":"2024-03-21T10:22:27.216923+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:21:56.788908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:21:56.788908+0000","last_clean_scrub_stamp":"2024-03-21T10:21:56.788908+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T15:31:17.167806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":25,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-21T10:22:27.211329+0000","last_change":"2024-03-21T10:21:58.865459+0000","last_active":"2024-03-21T10:22:27.211329+0000","last_peered":"2024-03-21T10:22:27.211329+0000","last_clean":"2024-03-21T10:22:27.211329+0000","last_became_active":"2024-03-21T10:21:58.860909+0000","last_became_peered":"2024-03-21T10:21:58.860909+0000","last_unstale":"2024-03-21T10:22:27.211329+0000","last_undegraded":"2024-03-21T10:22:27.211329+0000","last_fullsized":"2024-03-21T10:22:27.211329+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_clean_scrub_stamp":"2024-03-21T10:21:57.813712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T15:53:35.110404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":18,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-21T10:22:27.203484+0000","last_change":"2024-03-21T10:22:03.318651+0000","last_active":"2024-03-21T10:22:27.203484+0000","last_peered":"2024-03-21T10:22:27.203484+0000","last_clean":"2024-03-21T10:22:27.203484+0000","last_became_active":"2024-03-21T10:22:03.318108+0000","last_became_peered":"2024-03-21T10:22:03.318108+0000","last_unstale":"2024-03-21T10:22:27.203484+0000","last_undegraded":"2024-03-21T10:22:27.203484+0000","last_fullsized":"2024-03-21T10:22:27.203484+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:22:02.269614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:22:02.269614+0000","last_clean_scrub_stamp":"2024-03-21T10:22:02.269614+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T19:00:04.216749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"91'4","reported_seq":31,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-21T10:22:27.206182+0000","last_change":"2024-03-21T10:21:57.865516+0000","last_active":"2024-03-21T10:22:27.206182+0000","last_peered":"2024-03-21T10:22:27.206182+0000","last_clean":"2024-03-21T10:22:27.206182+0000","last_became_active":"2024-03-21T10:21:57.865012+0000","last_became_peered":"2024-03-21T10:21:57.865012+0000","last_unstale":"2024-03-21T10:22:27.206182+0000","last_undegraded":"2024-03-21T10:22:27.206182+0000","last_fullsized":"2024-03-21T10:22:27.206182+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:21:56.788908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:21:56.788908+0000","last_clean_scrub_stamp":"2024-03-21T10:21:56.788908+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T21:38:02.813453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":23,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-21T10:22:26.192707+0000","last_change":"2024-03-21T10:21:58.870876+0000","last_active":"2024-03-21T10:22:26.192707+0000","last_peered":"2024-03-21T10:22:26.192707+0000","last_clean":"2024-03-21T10:22:26.192707+0000","last_became_active":"2024-03-21T10:21:58.868240+0000","last_became_peered":"2024-03-21T10:21:58.868240+0000","last_unstale":"2024-03-21T10:22:26.192707+0000","last_undegraded":"2024-03-21T10:22:26.192707+0000","last_fullsized":"2024-03-21T10:22:26.192707+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_clean_scrub_stamp":"2024-03-21T10:21:57.813712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T15:16:59.496020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":18,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-21T10:22:27.207905+0000","last_change":"2024-03-21T10:22:03.314854+0000","last_active":"2024-03-21T10:22:27.207905+0000","last_peered":"2024-03-21T10:22:27.207905+0000","last_clean":"2024-03-21T10:22:27.207905+0000","last_became_active":"2024-03-21T10:22:03.314422+0000","last_became_peered":"2024-03-21T10:22:03.314422+0000","last_unstale":"2024-03-21T10:22:27.207905+0000","last_undegraded":"2024-03-21T10:22:27.207905+0000","last_fullsized":"2024-03-21T10:22:27.207905+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:22:02.269614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:22:02.269614+0000","last_clean_scrub_stamp":"2024-03-21T10:22:02.269614+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T12:34:31.519118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.14","version":"93'15","reported_seq":60,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-21T10:22:28.233231+0000","last_change":"2024-03-21T10:22:27.243817+0000","last_active":"2024-03-21T10:22:28.233231+0000","last_peered":"2024-03-21T10:22:28.233231+0000","last_clean":"2024-03-21T10:22:28.233231+0000","last_became_active":"2024-03-21T10:22:27.218873+0000","last_became_peered":"2024-03-21T10:22:27.218873+0000","last_unstale":"2024-03-21T10:22:28.233231+0000","last_undegraded":"2024-03-21T10:22:28.233231+0000","last_fullsized":"2024-03-21T10:22:28.233231+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:21:56.788908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:21:56.788908+0000","last_clean_scrub_stamp":"2024-03-21T10:21:56.788908+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T12:27:43.574451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10442,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16,"num_write_kb":16,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":3842,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,4],"acting":[10,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":25,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-21T10:22:27.232409+0000","last_change":"2024-03-21T10:21:58.860386+0000","last_active":"2024-03-21T10:22:27.232409+0000","last_peered":"2024-03-21T10:22:27.232409+0000","last_clean":"2024-03-21T10:22:27.232409+0000","last_became_active":"2024-03-21T10:21:58.860013+0000","last_became_peered":"2024-03-21T10:21:58.860013+0000","last_unstale":"2024-03-21T10:22:27.232409+0000","last_undegraded":"2024-03-21T10:22:27.232409+0000","last_fullsized":"2024-03-21T10:22:27.232409+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_clean_scrub_stamp":"2024-03-21T10:21:57.813712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T18:02:33.920374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":18,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-21T10:22:27.207748+0000","last_change":"2024-03-21T10:22:03.314083+0000","last_active":"2024-03-21T10:22:27.207748+0000","last_peered":"2024-03-21T10:22:27.207748+0000","last_clean":"2024-03-21T10:22:27.207748+0000","last_became_active":"2024-03-21T10:22:03.313652+0000","last_became_peered":"2024-03-21T10:22:03.313652+0000","last_unstale":"2024-03-21T10:22:27.207748+0000","last_undegraded":"2024-03-21T10:22:27.207748+0000","last_fullsized":"2024-03-21T10:22:27.207748+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:22:02.269614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:22:02.269614+0000","last_clean_scrub_stamp":"2024-03-21T10:22:02.269614+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T13:23:51.256535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"93'9","reported_seq":38,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-21T10:22:28.601592+0000","last_change":"2024-03-21T10:21:57.866716+0000","last_active":"2024-03-21T10:22:28.601592+0000","last_peered":"2024-03-21T10:22:28.601592+0000","last_clean":"2024-03-21T10:22:28.601592+0000","last_became_active":"2024-03-21T10:21:57.860900+0000","last_became_peered":"2024-03-21T10:21:57.860900+0000","last_unstale":"2024-03-21T10:22:28.601592+0000","last_undegraded":"2024-03-21T10:22:28.601592+0000","last_fullsized":"2024-03-21T10:22:28.601592+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:21:56.788908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:21:56.788908+0000","last_clean_scrub_stamp":"2024-03-21T10:21:56.788908+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T13:13:04.937120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":25,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-21T10:22:27.202537+0000","last_change":"2024-03-21T10:21:58.858815+0000","last_active":"2024-03-21T10:22:27.202537+0000","last_peered":"2024-03-21T10:22:27.202537+0000","last_clean":"2024-03-21T10:22:27.202537+0000","last_became_active":"2024-03-21T10:21:58.856483+0000","last_became_peered":"2024-03-21T10:21:58.856483+0000","last_unstale":"2024-03-21T10:22:27.202537+0000","last_undegraded":"2024-03-21T10:22:27.202537+0000","last_fullsized":"2024-03-21T10:22:27.202537+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_clean_scrub_stamp":"2024-03-21T10:21:57.813712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T15:42:38.208093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":18,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-21T10:22:27.219768+0000","last_change":"2024-03-21T10:22:03.319719+0000","last_active":"2024-03-21T10:22:27.219768+0000","last_peered":"2024-03-21T10:22:27.219768+0000","last_clean":"2024-03-21T10:22:27.219768+0000","last_became_active":"2024-03-21T10:22:03.319234+0000","last_became_peered":"2024-03-21T10:22:03.319234+0000","last_unstale":"2024-03-21T10:22:27.219768+0000","last_undegraded":"2024-03-21T10:22:27.219768+0000","last_fullsized":"2024-03-21T10:22:27.219768+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:22:02.269614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:22:02.269614+0000","last_clean_scrub_stamp":"2024-03-21T10:22:02.269614+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T14:11:21.609594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"91'3","reported_seq":30,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-21T10:22:27.206316+0000","last_change":"2024-03-21T10:21:57.862744+0000","last_active":"2024-03-21T10:22:27.206316+0000","last_peered":"2024-03-21T10:22:27.206316+0000","last_clean":"2024-03-21T10:22:27.206316+0000","last_became_active":"2024-03-21T10:21:57.862352+0000","last_became_peered":"2024-03-21T10:21:57.862352+0000","last_unstale":"2024-03-21T10:22:27.206316+0000","last_undegraded":"2024-03-21T10:22:27.206316+0000","last_fullsized":"2024-03-21T10:22:27.206316+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:21:56.788908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:21:56.788908+0000","last_clean_scrub_stamp":"2024-03-21T10:21:56.788908+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T20:19:50.138904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":25,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-21T10:22:27.216267+0000","last_change":"2024-03-21T10:21:58.863514+0000","last_active":"2024-03-21T10:22:27.216267+0000","last_peered":"2024-03-21T10:22:27.216267+0000","last_clean":"2024-03-21T10:22:27.216267+0000","last_became_active":"2024-03-21T10:21:58.861981+0000","last_became_peered":"2024-03-21T10:21:58.861981+0000","last_unstale":"2024-03-21T10:22:27.216267+0000","last_undegraded":"2024-03-21T10:22:27.216267+0000","last_fullsized":"2024-03-21T10:22:27.216267+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_clean_scrub_stamp":"2024-03-21T10:21:57.813712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T15:00:33.813456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":18,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-21T10:22:27.209637+0000","last_change":"2024-03-21T10:22:03.309844+0000","last_active":"2024-03-21T10:22:27.209637+0000","last_peered":"2024-03-21T10:22:27.209637+0000","last_clean":"2024-03-21T10:22:27.209637+0000","last_became_active":"2024-03-21T10:22:03.309448+0000","last_became_peered":"2024-03-21T10:22:03.309448+0000","last_unstale":"2024-03-21T10:22:27.209637+0000","last_undegraded":"2024-03-21T10:22:27.209637+0000","last_fullsized":"2024-03-21T10:22:27.209637+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:22:02.269614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:22:02.269614+0000","last_clean_scrub_stamp":"2024-03-21T10:22:02.269614+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T18:38:51.167166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.11","version":"93'6","reported_seq":42,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-21T10:22:28.586323+0000","last_change":"2024-03-21T10:22:27.223337+0000","last_active":"2024-03-21T10:22:28.586323+0000","last_peered":"2024-03-21T10:22:28.586323+0000","last_clean":"2024-03-21T10:22:28.586323+0000","last_became_active":"2024-03-21T10:22:27.222999+0000","last_became_peered":"2024-03-21T10:22:27.222999+0000","last_unstale":"2024-03-21T10:22:28.586323+0000","last_undegraded":"2024-03-21T10:22:28.586323+0000","last_fullsized":"2024-03-21T10:22:28.586323+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:21:56.788908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:21:56.788908+0000","last_clean_scrub_stamp":"2024-03-21T10:21:56.788908+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T20:40:45.780696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":979,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":4,"num_read_kb":3,"num_write":6,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,10],"acting":[4,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":25,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-21T10:22:27.214464+0000","last_change":"2024-03-21T10:21:58.862225+0000","last_active":"2024-03-21T10:22:27.214464+0000","last_peered":"2024-03-21T10:22:27.214464+0000","last_clean":"2024-03-21T10:22:27.214464+0000","last_became_active":"2024-03-21T10:21:58.861816+0000","last_became_peered":"2024-03-21T10:21:58.861816+0000","last_unstale":"2024-03-21T10:22:27.214464+0000","last_undegraded":"2024-03-21T10:22:27.214464+0000","last_fullsized":"2024-03-21T10:22:27.214464+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_clean_scrub_stamp":"2024-03-21T10:21:57.813712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T18:36:13.420867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":18,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-21T10:22:27.203338+0000","last_change":"2024-03-21T10:22:03.315306+0000","last_active":"2024-03-21T10:22:27.203338+0000","last_peered":"2024-03-21T10:22:27.203338+0000","last_clean":"2024-03-21T10:22:27.203338+0000","last_became_active":"2024-03-21T10:22:03.314839+0000","last_became_peered":"2024-03-21T10:22:03.314839+0000","last_unstale":"2024-03-21T10:22:27.203338+0000","last_undegraded":"2024-03-21T10:22:27.203338+0000","last_fullsized":"2024-03-21T10:22:27.203338+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:22:02.269614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:22:02.269614+0000","last_clean_scrub_stamp":"2024-03-21T10:22:02.269614+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T15:47:45.690434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"92'15","reported_seq":42,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-21T10:22:27.227633+0000","last_change":"2024-03-21T10:21:57.870998+0000","last_active":"2024-03-21T10:22:27.227633+0000","last_peered":"2024-03-21T10:22:27.227633+0000","last_clean":"2024-03-21T10:22:27.227633+0000","last_became_active":"2024-03-21T10:21:57.870605+0000","last_became_peered":"2024-03-21T10:21:57.870605+0000","last_unstale":"2024-03-21T10:22:27.227633+0000","last_undegraded":"2024-03-21T10:22:27.227633+0000","last_fullsized":"2024-03-21T10:22:27.227633+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:21:56.788908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:21:56.788908+0000","last_clean_scrub_stamp":"2024-03-21T10:21:56.788908+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T13:33:54.775122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":25,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-21T10:22:27.204847+0000","last_change":"2024-03-21T10:21:58.858675+0000","last_active":"2024-03-21T10:22:27.204847+0000","last_peered":"2024-03-21T10:22:27.204847+0000","last_clean":"2024-03-21T10:22:27.204847+0000","last_became_active":"2024-03-21T10:21:58.856907+0000","last_became_peered":"2024-03-21T10:21:58.856907+0000","last_unstale":"2024-03-21T10:22:27.204847+0000","last_undegraded":"2024-03-21T10:22:27.204847+0000","last_fullsized":"2024-03-21T10:22:27.204847+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_clean_scrub_stamp":"2024-03-21T10:21:57.813712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T18:17:31.208728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":18,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-21T10:22:27.204692+0000","last_change":"2024-03-21T10:22:03.318257+0000","last_active":"2024-03-21T10:22:27.204692+0000","last_peered":"2024-03-21T10:22:27.204692+0000","last_clean":"2024-03-21T10:22:27.204692+0000","last_became_active":"2024-03-21T10:22:03.317622+0000","last_became_peered":"2024-03-21T10:22:03.317622+0000","last_unstale":"2024-03-21T10:22:27.204692+0000","last_undegraded":"2024-03-21T10:22:27.204692+0000","last_fullsized":"2024-03-21T10:22:27.204692+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:22:02.269614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:22:02.269614+0000","last_clean_scrub_stamp":"2024-03-21T10:22:02.269614+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T17:15:50.272193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"86'1","reported_seq":28,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-21T10:22:27.232910+0000","last_change":"2024-03-21T10:21:57.868014+0000","last_active":"2024-03-21T10:22:27.232910+0000","last_peered":"2024-03-21T10:22:27.232910+0000","last_clean":"2024-03-21T10:22:27.232910+0000","last_became_active":"2024-03-21T10:21:57.867189+0000","last_became_peered":"2024-03-21T10:21:57.867189+0000","last_unstale":"2024-03-21T10:22:27.232910+0000","last_undegraded":"2024-03-21T10:22:27.232910+0000","last_fullsized":"2024-03-21T10:22:27.232910+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:21:56.788908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:21:56.788908+0000","last_clean_scrub_stamp":"2024-03-21T10:21:56.788908+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T22:02:48.704932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":25,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-21T10:22:27.218969+0000","last_change":"2024-03-21T10:21:58.862205+0000","last_active":"2024-03-21T10:22:27.218969+0000","last_peered":"2024-03-21T10:22:27.218969+0000","last_clean":"2024-03-21T10:22:27.218969+0000","last_became_active":"2024-03-21T10:21:58.861186+0000","last_became_peered":"2024-03-21T10:21:58.861186+0000","last_unstale":"2024-03-21T10:22:27.218969+0000","last_undegraded":"2024-03-21T10:22:27.218969+0000","last_fullsized":"2024-03-21T10:22:27.218969+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_clean_scrub_stamp":"2024-03-21T10:21:57.813712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T19:44:49.719845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":18,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-21T10:22:27.229378+0000","last_change":"2024-03-21T10:22:03.636064+0000","last_active":"2024-03-21T10:22:27.229378+0000","last_peered":"2024-03-21T10:22:27.229378+0000","last_clean":"2024-03-21T10:22:27.229378+0000","last_became_active":"2024-03-21T10:22:03.633596+0000","last_became_peered":"2024-03-21T10:22:03.633596+0000","last_unstale":"2024-03-21T10:22:27.229378+0000","last_undegraded":"2024-03-21T10:22:27.229378+0000","last_fullsized":"2024-03-21T10:22:27.229378+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:22:02.269614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:22:02.269614+0000","last_clean_scrub_stamp":"2024-03-21T10:22:02.269614+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T19:32:31.844066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":27,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-21T10:22:27.208381+0000","last_change":"2024-03-21T10:21:57.851955+0000","last_active":"2024-03-21T10:22:27.208381+0000","last_peered":"2024-03-21T10:22:27.208381+0000","last_clean":"2024-03-21T10:22:27.208381+0000","last_became_active":"2024-03-21T10:21:57.851571+0000","last_became_peered":"2024-03-21T10:21:57.851571+0000","last_unstale":"2024-03-21T10:22:27.208381+0000","last_undegraded":"2024-03-21T10:22:27.208381+0000","last_fullsized":"2024-03-21T10:22:27.208381+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:21:56.788908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:21:56.788908+0000","last_clean_scrub_stamp":"2024-03-21T10:21:56.788908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T17:23:40.441895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":25,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-21T10:22:27.207574+0000","last_change":"2024-03-21T10:21:58.856393+0000","last_active":"2024-03-21T10:22:27.207574+0000","last_peered":"2024-03-21T10:22:27.207574+0000","last_clean":"2024-03-21T10:22:27.207574+0000","last_became_active":"2024-03-21T10:21:58.855139+0000","last_became_peered":"2024-03-21T10:21:58.855139+0000","last_unstale":"2024-03-21T10:22:27.207574+0000","last_undegraded":"2024-03-21T10:22:27.207574+0000","last_fullsized":"2024-03-21T10:22:27.207574+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_clean_scrub_stamp":"2024-03-21T10:21:57.813712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T14:24:55.663576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":18,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-21T10:22:27.202914+0000","last_change":"2024-03-21T10:22:03.315586+0000","last_active":"2024-03-21T10:22:27.202914+0000","last_peered":"2024-03-21T10:22:27.202914+0000","last_clean":"2024-03-21T10:22:27.202914+0000","last_became_active":"2024-03-21T10:22:03.315198+0000","last_became_peered":"2024-03-21T10:22:03.315198+0000","last_unstale":"2024-03-21T10:22:27.202914+0000","last_undegraded":"2024-03-21T10:22:27.202914+0000","last_fullsized":"2024-03-21T10:22:27.202914+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:22:02.269614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:22:02.269614+0000","last_clean_scrub_stamp":"2024-03-21T10:22:02.269614+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T14:25:13.534311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"92'1","reported_seq":28,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-21T10:22:27.216126+0000","last_change":"2024-03-21T10:21:57.862094+0000","last_active":"2024-03-21T10:22:27.216126+0000","last_peered":"2024-03-21T10:22:27.216126+0000","last_clean":"2024-03-21T10:22:27.216126+0000","last_became_active":"2024-03-21T10:21:57.860307+0000","last_became_peered":"2024-03-21T10:21:57.860307+0000","last_unstale":"2024-03-21T10:22:27.216126+0000","last_undegraded":"2024-03-21T10:22:27.216126+0000","last_fullsized":"2024-03-21T10:22:27.216126+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:21:56.788908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:21:56.788908+0000","last_clean_scrub_stamp":"2024-03-21T10:21:56.788908+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T12:58:50.701218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":25,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-21T10:22:27.211160+0000","last_change":"2024-03-21T10:21:58.870880+0000","last_active":"2024-03-21T10:22:27.211160+0000","last_peered":"2024-03-21T10:22:27.211160+0000","last_clean":"2024-03-21T10:22:27.211160+0000","last_became_active":"2024-03-21T10:21:58.869882+0000","last_became_peered":"2024-03-21T10:21:58.869882+0000","last_unstale":"2024-03-21T10:22:27.211160+0000","last_undegraded":"2024-03-21T10:22:27.211160+0000","last_fullsized":"2024-03-21T10:22:27.211160+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_clean_scrub_stamp":"2024-03-21T10:21:57.813712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T18:01:42.613732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":18,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-21T10:22:27.203242+0000","last_change":"2024-03-21T10:22:03.634425+0000","last_active":"2024-03-21T10:22:27.203242+0000","last_peered":"2024-03-21T10:22:27.203242+0000","last_clean":"2024-03-21T10:22:27.203242+0000","last_became_active":"2024-03-21T10:22:03.632464+0000","last_became_peered":"2024-03-21T10:22:03.632464+0000","last_unstale":"2024-03-21T10:22:27.203242+0000","last_undegraded":"2024-03-21T10:22:27.203242+0000","last_fullsized":"2024-03-21T10:22:27.203242+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:22:02.269614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:22:02.269614+0000","last_clean_scrub_stamp":"2024-03-21T10:22:02.269614+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T16:03:28.270757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":31,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-21T10:22:27.233910+0000","last_change":"2024-03-21T10:22:27.233910+0000","last_active":"2024-03-21T10:22:27.233910+0000","last_peered":"2024-03-21T10:22:27.233910+0000","last_clean":"2024-03-21T10:22:27.233910+0000","last_became_active":"2024-03-21T10:22:27.233342+0000","last_became_peered":"2024-03-21T10:22:27.233342+0000","last_unstale":"2024-03-21T10:22:27.233910+0000","last_undegraded":"2024-03-21T10:22:27.233910+0000","last_fullsized":"2024-03-21T10:22:27.233910+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_clean_scrub_stamp":"2024-03-21T10:21:57.813712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T19:14:41.147035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,8],"acting":[5,2,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"93'5","reported_seq":33,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-21T10:22:28.585282+0000","last_change":"2024-03-21T10:21:57.856111+0000","last_active":"2024-03-21T10:22:28.585282+0000","last_peered":"2024-03-21T10:22:28.585282+0000","last_clean":"2024-03-21T10:22:28.585282+0000","last_became_active":"2024-03-21T10:21:57.855584+0000","last_became_peered":"2024-03-21T10:21:57.855584+0000","last_unstale":"2024-03-21T10:22:28.585282+0000","last_undegraded":"2024-03-21T10:22:28.585282+0000","last_fullsized":"2024-03-21T10:22:28.585282+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:21:56.788908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:21:56.788908+0000","last_clean_scrub_stamp":"2024-03-21T10:21:56.788908+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T17:53:48.208554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":18,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-21T10:22:27.216392+0000","last_change":"2024-03-21T10:22:03.318831+0000","last_active":"2024-03-21T10:22:27.216392+0000","last_peered":"2024-03-21T10:22:27.216392+0000","last_clean":"2024-03-21T10:22:27.216392+0000","last_became_active":"2024-03-21T10:22:03.318404+0000","last_became_peered":"2024-03-21T10:22:03.318404+0000","last_unstale":"2024-03-21T10:22:27.216392+0000","last_undegraded":"2024-03-21T10:22:27.216392+0000","last_fullsized":"2024-03-21T10:22:27.216392+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:22:02.269614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:22:02.269614+0000","last_clean_scrub_stamp":"2024-03-21T10:22:02.269614+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T12:06:52.043286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":25,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-21T10:22:27.223893+0000","last_change":"2024-03-21T10:21:58.856469+0000","last_active":"2024-03-21T10:22:27.223893+0000","last_peered":"2024-03-21T10:22:27.223893+0000","last_clean":"2024-03-21T10:22:27.223893+0000","last_became_active":"2024-03-21T10:21:58.856032+0000","last_became_peered":"2024-03-21T10:21:58.856032+0000","last_unstale":"2024-03-21T10:22:27.223893+0000","last_undegraded":"2024-03-21T10:22:27.223893+0000","last_fullsized":"2024-03-21T10:22:27.223893+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_clean_scrub_stamp":"2024-03-21T10:21:57.813712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T21:19:59.302418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"93'6","reported_seq":35,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-21T10:22:28.583397+0000","last_change":"2024-03-21T10:21:57.854210+0000","last_active":"2024-03-21T10:22:28.583397+0000","last_peered":"2024-03-21T10:22:28.583397+0000","last_clean":"2024-03-21T10:22:28.583397+0000","last_became_active":"2024-03-21T10:21:57.853787+0000","last_became_peered":"2024-03-21T10:21:57.853787+0000","last_unstale":"2024-03-21T10:22:28.583397+0000","last_undegraded":"2024-03-21T10:22:28.583397+0000","last_fullsized":"2024-03-21T10:22:28.583397+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:21:56.788908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:21:56.788908+0000","last_clean_scrub_stamp":"2024-03-21T10:21:56.788908+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T12:10:42.926436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"23'137","reported_seq":361,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-21T10:22:27.214365+0000","last_change":"2024-03-21T10:20:21.585339+0000","last_active":"2024-03-21T10:22:27.214365+0000","last_peered":"2024-03-21T10:22:27.214365+0000","last_clean":"2024-03-21T10:22:27.214365+0000","last_became_active":"2024-03-21T10:20:21.583999+0000","last_became_peered":"2024-03-21T10:20:21.583999+0000","last_unstale":"2024-03-21T10:22:27.214365+0000","last_undegraded":"2024-03-21T10:22:27.214365+0000","last_fullsized":"2024-03-21T10:22:27.214365+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:15:40.696168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:15:40.696168+0000","last_clean_scrub_stamp":"2024-03-21T10:15:40.696168+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T15:31:54.603722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":18,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-21T10:22:27.208940+0000","last_change":"2024-03-21T10:22:03.634970+0000","last_active":"2024-03-21T10:22:27.208940+0000","last_peered":"2024-03-21T10:22:27.208940+0000","last_clean":"2024-03-21T10:22:27.208940+0000","last_became_active":"2024-03-21T10:22:03.633603+0000","last_became_peered":"2024-03-21T10:22:03.633603+0000","last_unstale":"2024-03-21T10:22:27.208940+0000","last_undegraded":"2024-03-21T10:22:27.208940+0000","last_fullsized":"2024-03-21T10:22:27.208940+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:22:02.269614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:22:02.269614+0000","last_clean_scrub_stamp":"2024-03-21T10:22:02.269614+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T12:13:02.041661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":25,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-21T10:22:27.202200+0000","last_change":"2024-03-21T10:21:58.861159+0000","last_active":"2024-03-21T10:22:27.202200+0000","last_peered":"2024-03-21T10:22:27.202200+0000","last_clean":"2024-03-21T10:22:27.202200+0000","last_became_active":"2024-03-21T10:21:58.860690+0000","last_became_peered":"2024-03-21T10:21:58.860690+0000","last_unstale":"2024-03-21T10:22:27.202200+0000","last_undegraded":"2024-03-21T10:22:27.202200+0000","last_fullsized":"2024-03-21T10:22:27.202200+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_clean_scrub_stamp":"2024-03-21T10:21:57.813712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T16:21:53.632997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"92'3","reported_seq":30,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-21T10:22:27.214557+0000","last_change":"2024-03-21T10:21:57.863581+0000","last_active":"2024-03-21T10:22:27.214557+0000","last_peered":"2024-03-21T10:22:27.214557+0000","last_clean":"2024-03-21T10:22:27.214557+0000","last_became_active":"2024-03-21T10:21:57.863269+0000","last_became_peered":"2024-03-21T10:21:57.863269+0000","last_unstale":"2024-03-21T10:22:27.214557+0000","last_undegraded":"2024-03-21T10:22:27.214557+0000","last_fullsized":"2024-03-21T10:22:27.214557+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:21:56.788908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:21:56.788908+0000","last_clean_scrub_stamp":"2024-03-21T10:21:56.788908+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T14:03:59.930570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":18,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-21T10:22:27.218249+0000","last_change":"2024-03-21T10:22:03.313515+0000","last_active":"2024-03-21T10:22:27.218249+0000","last_peered":"2024-03-21T10:22:27.218249+0000","last_clean":"2024-03-21T10:22:27.218249+0000","last_became_active":"2024-03-21T10:22:03.311862+0000","last_became_peered":"2024-03-21T10:22:03.311862+0000","last_unstale":"2024-03-21T10:22:27.218249+0000","last_undegraded":"2024-03-21T10:22:27.218249+0000","last_fullsized":"2024-03-21T10:22:27.218249+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:22:02.269614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:22:02.269614+0000","last_clean_scrub_stamp":"2024-03-21T10:22:02.269614+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T16:47:50.761205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"91'1","reported_seq":28,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-21T10:22:27.232498+0000","last_change":"2024-03-21T10:21:57.870913+0000","last_active":"2024-03-21T10:22:27.232498+0000","last_peered":"2024-03-21T10:22:27.232498+0000","last_clean":"2024-03-21T10:22:27.232498+0000","last_became_active":"2024-03-21T10:21:57.870578+0000","last_became_peered":"2024-03-21T10:21:57.870578+0000","last_unstale":"2024-03-21T10:22:27.232498+0000","last_undegraded":"2024-03-21T10:22:27.232498+0000","last_fullsized":"2024-03-21T10:22:27.232498+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:21:56.788908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:21:56.788908+0000","last_clean_scrub_stamp":"2024-03-21T10:21:56.788908+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T11:46:48.621501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":25,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-21T10:22:27.203308+0000","last_change":"2024-03-21T10:21:58.852247+0000","last_active":"2024-03-21T10:22:27.203308+0000","last_peered":"2024-03-21T10:22:27.203308+0000","last_clean":"2024-03-21T10:22:27.203308+0000","last_became_active":"2024-03-21T10:21:58.851976+0000","last_became_peered":"2024-03-21T10:21:58.851976+0000","last_unstale":"2024-03-21T10:22:27.203308+0000","last_undegraded":"2024-03-21T10:22:27.203308+0000","last_fullsized":"2024-03-21T10:22:27.203308+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_clean_scrub_stamp":"2024-03-21T10:21:57.813712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T14:27:11.383741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":18,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-21T10:22:27.222696+0000","last_change":"2024-03-21T10:22:03.637671+0000","last_active":"2024-03-21T10:22:27.222696+0000","last_peered":"2024-03-21T10:22:27.222696+0000","last_clean":"2024-03-21T10:22:27.222696+0000","last_became_active":"2024-03-21T10:22:03.634265+0000","last_became_peered":"2024-03-21T10:22:03.634265+0000","last_unstale":"2024-03-21T10:22:27.222696+0000","last_undegraded":"2024-03-21T10:22:27.222696+0000","last_fullsized":"2024-03-21T10:22:27.222696+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:22:02.269614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:22:02.269614+0000","last_clean_scrub_stamp":"2024-03-21T10:22:02.269614+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T15:38:02.214178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"92'3","reported_seq":30,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-21T10:22:27.208779+0000","last_change":"2024-03-21T10:21:57.862027+0000","last_active":"2024-03-21T10:22:27.208779+0000","last_peered":"2024-03-21T10:22:27.208779+0000","last_clean":"2024-03-21T10:22:27.208779+0000","last_became_active":"2024-03-21T10:21:57.860097+0000","last_became_peered":"2024-03-21T10:21:57.860097+0000","last_unstale":"2024-03-21T10:22:27.208779+0000","last_undegraded":"2024-03-21T10:22:27.208779+0000","last_fullsized":"2024-03-21T10:22:27.208779+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:21:56.788908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:21:56.788908+0000","last_clean_scrub_stamp":"2024-03-21T10:21:56.788908+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T13:46:19.562452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":25,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-21T10:22:27.225488+0000","last_change":"2024-03-21T10:21:58.856082+0000","last_active":"2024-03-21T10:22:27.225488+0000","last_peered":"2024-03-21T10:22:27.225488+0000","last_clean":"2024-03-21T10:22:27.225488+0000","last_became_active":"2024-03-21T10:21:58.854172+0000","last_became_peered":"2024-03-21T10:21:58.854172+0000","last_unstale":"2024-03-21T10:22:27.225488+0000","last_undegraded":"2024-03-21T10:22:27.225488+0000","last_fullsized":"2024-03-21T10:22:27.225488+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_clean_scrub_stamp":"2024-03-21T10:21:57.813712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T14:59:33.995716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":18,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-21T10:22:27.213780+0000","last_change":"2024-03-21T10:22:03.310205+0000","last_active":"2024-03-21T10:22:27.213780+0000","last_peered":"2024-03-21T10:22:27.213780+0000","last_clean":"2024-03-21T10:22:27.213780+0000","last_became_active":"2024-03-21T10:22:03.309039+0000","last_became_peered":"2024-03-21T10:22:03.309039+0000","last_unstale":"2024-03-21T10:22:27.213780+0000","last_undegraded":"2024-03-21T10:22:27.213780+0000","last_fullsized":"2024-03-21T10:22:27.213780+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:22:02.269614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:22:02.269614+0000","last_clean_scrub_stamp":"2024-03-21T10:22:02.269614+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T11:16:46.075260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"93'7","reported_seq":35,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-21T10:22:28.599712+0000","last_change":"2024-03-21T10:21:57.857890+0000","last_active":"2024-03-21T10:22:28.599712+0000","last_peered":"2024-03-21T10:22:28.599712+0000","last_clean":"2024-03-21T10:22:28.599712+0000","last_became_active":"2024-03-21T10:21:57.857492+0000","last_became_peered":"2024-03-21T10:21:57.857492+0000","last_unstale":"2024-03-21T10:22:28.599712+0000","last_undegraded":"2024-03-21T10:22:28.599712+0000","last_fullsized":"2024-03-21T10:22:28.599712+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:21:56.788908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:21:56.788908+0000","last_clean_scrub_stamp":"2024-03-21T10:21:56.788908+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T21:18:45.566843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":31,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-21T10:22:27.219339+0000","last_change":"2024-03-21T10:22:27.219339+0000","last_active":"2024-03-21T10:22:27.219339+0000","last_peered":"2024-03-21T10:22:27.219339+0000","last_clean":"2024-03-21T10:22:27.219339+0000","last_became_active":"2024-03-21T10:22:27.218604+0000","last_became_peered":"2024-03-21T10:22:27.218604+0000","last_unstale":"2024-03-21T10:22:27.219339+0000","last_undegraded":"2024-03-21T10:22:27.219339+0000","last_fullsized":"2024-03-21T10:22:27.219339+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_clean_scrub_stamp":"2024-03-21T10:21:57.813712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T15:58:53.161767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,4],"acting":[0,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":18,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-21T10:22:27.203123+0000","last_change":"2024-03-21T10:22:03.633789+0000","last_active":"2024-03-21T10:22:27.203123+0000","last_peered":"2024-03-21T10:22:27.203123+0000","last_clean":"2024-03-21T10:22:27.203123+0000","last_became_active":"2024-03-21T10:22:03.632716+0000","last_became_peered":"2024-03-21T10:22:03.632716+0000","last_unstale":"2024-03-21T10:22:27.203123+0000","last_undegraded":"2024-03-21T10:22:27.203123+0000","last_fullsized":"2024-03-21T10:22:27.203123+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:22:02.269614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:22:02.269614+0000","last_clean_scrub_stamp":"2024-03-21T10:22:02.269614+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T14:08:47.301881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"91'2","reported_seq":27,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-21T10:22:26.191650+0000","last_change":"2024-03-21T10:21:57.862413+0000","last_active":"2024-03-21T10:22:26.191650+0000","last_peered":"2024-03-21T10:22:26.191650+0000","last_clean":"2024-03-21T10:22:26.191650+0000","last_became_active":"2024-03-21T10:21:57.862104+0000","last_became_peered":"2024-03-21T10:21:57.862104+0000","last_unstale":"2024-03-21T10:22:26.191650+0000","last_undegraded":"2024-03-21T10:22:26.191650+0000","last_fullsized":"2024-03-21T10:22:26.191650+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:21:56.788908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:21:56.788908+0000","last_clean_scrub_stamp":"2024-03-21T10:21:56.788908+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T17:49:21.729492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":25,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-21T10:22:27.227390+0000","last_change":"2024-03-21T10:21:58.864688+0000","last_active":"2024-03-21T10:22:27.227390+0000","last_peered":"2024-03-21T10:22:27.227390+0000","last_clean":"2024-03-21T10:22:27.227390+0000","last_became_active":"2024-03-21T10:21:58.862821+0000","last_became_peered":"2024-03-21T10:21:58.862821+0000","last_unstale":"2024-03-21T10:22:27.227390+0000","last_undegraded":"2024-03-21T10:22:27.227390+0000","last_fullsized":"2024-03-21T10:22:27.227390+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_clean_scrub_stamp":"2024-03-21T10:21:57.813712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T11:17:09.650649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":18,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-21T10:22:27.209863+0000","last_change":"2024-03-21T10:22:03.315886+0000","last_active":"2024-03-21T10:22:27.209863+0000","last_peered":"2024-03-21T10:22:27.209863+0000","last_clean":"2024-03-21T10:22:27.209863+0000","last_became_active":"2024-03-21T10:22:03.315460+0000","last_became_peered":"2024-03-21T10:22:03.315460+0000","last_unstale":"2024-03-21T10:22:27.209863+0000","last_undegraded":"2024-03-21T10:22:27.209863+0000","last_fullsized":"2024-03-21T10:22:27.209863+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:22:02.269614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:22:02.269614+0000","last_clean_scrub_stamp":"2024-03-21T10:22:02.269614+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T14:56:12.869771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.9","version":"93'7","reported_seq":34,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-21T10:22:28.597888+0000","last_change":"2024-03-21T10:21:57.868033+0000","last_active":"2024-03-21T10:22:28.597888+0000","last_peered":"2024-03-21T10:22:28.597888+0000","last_clean":"2024-03-21T10:22:28.597888+0000","last_became_active":"2024-03-21T10:21:57.867279+0000","last_became_peered":"2024-03-21T10:21:57.867279+0000","last_unstale":"2024-03-21T10:22:28.597888+0000","last_undegraded":"2024-03-21T10:22:28.597888+0000","last_fullsized":"2024-03-21T10:22:28.597888+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:21:56.788908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:21:56.788908+0000","last_clean_scrub_stamp":"2024-03-21T10:21:56.788908+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T20:05:37.701443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":25,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-21T10:22:27.217015+0000","last_change":"2024-03-21T10:21:58.868455+0000","last_active":"2024-03-21T10:22:27.217015+0000","last_peered":"2024-03-21T10:22:27.217015+0000","last_clean":"2024-03-21T10:22:27.217015+0000","last_became_active":"2024-03-21T10:21:58.867786+0000","last_became_peered":"2024-03-21T10:21:58.867786+0000","last_unstale":"2024-03-21T10:22:27.217015+0000","last_undegraded":"2024-03-21T10:22:27.217015+0000","last_fullsized":"2024-03-21T10:22:27.217015+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_clean_scrub_stamp":"2024-03-21T10:21:57.813712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T19:16:12.258470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":18,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-21T10:22:27.204035+0000","last_change":"2024-03-21T10:22:03.315540+0000","last_active":"2024-03-21T10:22:27.204035+0000","last_peered":"2024-03-21T10:22:27.204035+0000","last_clean":"2024-03-21T10:22:27.204035+0000","last_became_active":"2024-03-21T10:22:03.315006+0000","last_became_peered":"2024-03-21T10:22:03.315006+0000","last_unstale":"2024-03-21T10:22:27.204035+0000","last_undegraded":"2024-03-21T10:22:27.204035+0000","last_fullsized":"2024-03-21T10:22:27.204035+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:22:02.269614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:22:02.269614+0000","last_clean_scrub_stamp":"2024-03-21T10:22:02.269614+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T12:24:46.792737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"93'2","reported_seq":29,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-21T10:22:27.213164+0000","last_change":"2024-03-21T10:21:57.862748+0000","last_active":"2024-03-21T10:22:27.213164+0000","last_peered":"2024-03-21T10:22:27.213164+0000","last_clean":"2024-03-21T10:22:27.213164+0000","last_became_active":"2024-03-21T10:21:57.862350+0000","last_became_peered":"2024-03-21T10:21:57.862350+0000","last_unstale":"2024-03-21T10:22:27.213164+0000","last_undegraded":"2024-03-21T10:22:27.213164+0000","last_fullsized":"2024-03-21T10:22:27.213164+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:21:56.788908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:21:56.788908+0000","last_clean_scrub_stamp":"2024-03-21T10:21:56.788908+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T17:08:47.301758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":25,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-21T10:22:27.225266+0000","last_change":"2024-03-21T10:21:58.866340+0000","last_active":"2024-03-21T10:22:27.225266+0000","last_peered":"2024-03-21T10:22:27.225266+0000","last_clean":"2024-03-21T10:22:27.225266+0000","last_became_active":"2024-03-21T10:21:58.865965+0000","last_became_peered":"2024-03-21T10:21:58.865965+0000","last_unstale":"2024-03-21T10:22:27.225266+0000","last_undegraded":"2024-03-21T10:22:27.225266+0000","last_fullsized":"2024-03-21T10:22:27.225266+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_clean_scrub_stamp":"2024-03-21T10:21:57.813712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T10:42:23.854176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":18,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-21T10:22:27.203089+0000","last_change":"2024-03-21T10:22:03.634108+0000","last_active":"2024-03-21T10:22:27.203089+0000","last_peered":"2024-03-21T10:22:27.203089+0000","last_clean":"2024-03-21T10:22:27.203089+0000","last_became_active":"2024-03-21T10:22:03.633344+0000","last_became_peered":"2024-03-21T10:22:03.633344+0000","last_unstale":"2024-03-21T10:22:27.203089+0000","last_undegraded":"2024-03-21T10:22:27.203089+0000","last_fullsized":"2024-03-21T10:22:27.203089+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:22:02.269614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:22:02.269614+0000","last_clean_scrub_stamp":"2024-03-21T10:22:02.269614+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T21:00:59.893994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"91'4","reported_seq":31,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-21T10:22:27.228877+0000","last_change":"2024-03-21T10:21:57.871520+0000","last_active":"2024-03-21T10:22:27.228877+0000","last_peered":"2024-03-21T10:22:27.228877+0000","last_clean":"2024-03-21T10:22:27.228877+0000","last_became_active":"2024-03-21T10:21:57.871127+0000","last_became_peered":"2024-03-21T10:21:57.871127+0000","last_unstale":"2024-03-21T10:22:27.228877+0000","last_undegraded":"2024-03-21T10:22:27.228877+0000","last_fullsized":"2024-03-21T10:22:27.228877+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:21:56.788908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:21:56.788908+0000","last_clean_scrub_stamp":"2024-03-21T10:21:56.788908+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T20:43:39.537307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":25,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-21T10:22:27.216094+0000","last_change":"2024-03-21T10:21:58.861786+0000","last_active":"2024-03-21T10:22:27.216094+0000","last_peered":"2024-03-21T10:22:27.216094+0000","last_clean":"2024-03-21T10:22:27.216094+0000","last_became_active":"2024-03-21T10:21:58.861428+0000","last_became_peered":"2024-03-21T10:21:58.861428+0000","last_unstale":"2024-03-21T10:22:27.216094+0000","last_undegraded":"2024-03-21T10:22:27.216094+0000","last_fullsized":"2024-03-21T10:22:27.216094+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_clean_scrub_stamp":"2024-03-21T10:21:57.813712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T15:56:30.256266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":16,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-21T10:22:26.194783+0000","last_change":"2024-03-21T10:22:03.305085+0000","last_active":"2024-03-21T10:22:26.194783+0000","last_peered":"2024-03-21T10:22:26.194783+0000","last_clean":"2024-03-21T10:22:26.194783+0000","last_became_active":"2024-03-21T10:22:03.304474+0000","last_became_peered":"2024-03-21T10:22:03.304474+0000","last_unstale":"2024-03-21T10:22:26.194783+0000","last_undegraded":"2024-03-21T10:22:26.194783+0000","last_fullsized":"2024-03-21T10:22:26.194783+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:22:02.269614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:22:02.269614+0000","last_clean_scrub_stamp":"2024-03-21T10:22:02.269614+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T18:28:10.480981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"93'5","reported_seq":35,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-21T10:22:28.589283+0000","last_change":"2024-03-21T10:21:57.860657+0000","last_active":"2024-03-21T10:22:28.589283+0000","last_peered":"2024-03-21T10:22:28.589283+0000","last_clean":"2024-03-21T10:22:28.589283+0000","last_became_active":"2024-03-21T10:21:57.860340+0000","last_became_peered":"2024-03-21T10:21:57.860340+0000","last_unstale":"2024-03-21T10:22:28.589283+0000","last_undegraded":"2024-03-21T10:22:28.589283+0000","last_fullsized":"2024-03-21T10:22:28.589283+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:21:56.788908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:21:56.788908+0000","last_clean_scrub_stamp":"2024-03-21T10:21:56.788908+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T21:58:08.706812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1170,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":25,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-21T10:22:27.216774+0000","last_change":"2024-03-21T10:21:58.874254+0000","last_active":"2024-03-21T10:22:27.216774+0000","last_peered":"2024-03-21T10:22:27.216774+0000","last_clean":"2024-03-21T10:22:27.216774+0000","last_became_active":"2024-03-21T10:21:58.869795+0000","last_became_peered":"2024-03-21T10:21:58.869795+0000","last_unstale":"2024-03-21T10:22:27.216774+0000","last_undegraded":"2024-03-21T10:22:27.216774+0000","last_fullsized":"2024-03-21T10:22:27.216774+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_clean_scrub_stamp":"2024-03-21T10:21:57.813712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T19:51:28.296199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":18,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-21T10:22:27.209002+0000","last_change":"2024-03-21T10:22:03.306761+0000","last_active":"2024-03-21T10:22:27.209002+0000","last_peered":"2024-03-21T10:22:27.209002+0000","last_clean":"2024-03-21T10:22:27.209002+0000","last_became_active":"2024-03-21T10:22:03.305106+0000","last_became_peered":"2024-03-21T10:22:03.305106+0000","last_unstale":"2024-03-21T10:22:27.209002+0000","last_undegraded":"2024-03-21T10:22:27.209002+0000","last_fullsized":"2024-03-21T10:22:27.209002+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:22:02.269614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:22:02.269614+0000","last_clean_scrub_stamp":"2024-03-21T10:22:02.269614+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T21:13:00.846169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"86'1","reported_seq":28,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-21T10:22:27.206783+0000","last_change":"2024-03-21T10:21:57.868012+0000","last_active":"2024-03-21T10:22:27.206783+0000","last_peered":"2024-03-21T10:22:27.206783+0000","last_clean":"2024-03-21T10:22:27.206783+0000","last_became_active":"2024-03-21T10:21:57.867682+0000","last_became_peered":"2024-03-21T10:21:57.867682+0000","last_unstale":"2024-03-21T10:22:27.206783+0000","last_undegraded":"2024-03-21T10:22:27.206783+0000","last_fullsized":"2024-03-21T10:22:27.206783+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:21:56.788908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:21:56.788908+0000","last_clean_scrub_stamp":"2024-03-21T10:21:56.788908+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T16:59:56.061167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":25,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-21T10:22:27.216386+0000","last_change":"2024-03-21T10:21:58.864949+0000","last_active":"2024-03-21T10:22:27.216386+0000","last_peered":"2024-03-21T10:22:27.216386+0000","last_clean":"2024-03-21T10:22:27.216386+0000","last_became_active":"2024-03-21T10:21:58.859796+0000","last_became_peered":"2024-03-21T10:21:58.859796+0000","last_unstale":"2024-03-21T10:22:27.216386+0000","last_undegraded":"2024-03-21T10:22:27.216386+0000","last_fullsized":"2024-03-21T10:22:27.216386+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_clean_scrub_stamp":"2024-03-21T10:21:57.813712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T18:02:51.873103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":18,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-21T10:22:27.204274+0000","last_change":"2024-03-21T10:22:03.308451+0000","last_active":"2024-03-21T10:22:27.204274+0000","last_peered":"2024-03-21T10:22:27.204274+0000","last_clean":"2024-03-21T10:22:27.204274+0000","last_became_active":"2024-03-21T10:22:03.307986+0000","last_became_peered":"2024-03-21T10:22:03.307986+0000","last_unstale":"2024-03-21T10:22:27.204274+0000","last_undegraded":"2024-03-21T10:22:27.204274+0000","last_fullsized":"2024-03-21T10:22:27.204274+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:22:02.269614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:22:02.269614+0000","last_clean_scrub_stamp":"2024-03-21T10:22:02.269614+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T21:08:42.241657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"93'5","reported_seq":32,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-21T10:22:28.592651+0000","last_change":"2024-03-21T10:21:57.851717+0000","last_active":"2024-03-21T10:22:28.592651+0000","last_peered":"2024-03-21T10:22:28.592651+0000","last_clean":"2024-03-21T10:22:28.592651+0000","last_became_active":"2024-03-21T10:21:57.849909+0000","last_became_peered":"2024-03-21T10:21:57.849909+0000","last_unstale":"2024-03-21T10:22:28.592651+0000","last_undegraded":"2024-03-21T10:22:28.592651+0000","last_fullsized":"2024-03-21T10:22:28.592651+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:21:56.788908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:21:56.788908+0000","last_clean_scrub_stamp":"2024-03-21T10:21:56.788908+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T11:41:37.611400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":25,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-21T10:22:27.217017+0000","last_change":"2024-03-21T10:21:58.860892+0000","last_active":"2024-03-21T10:22:27.217017+0000","last_peered":"2024-03-21T10:22:27.217017+0000","last_clean":"2024-03-21T10:22:27.217017+0000","last_became_active":"2024-03-21T10:21:58.860403+0000","last_became_peered":"2024-03-21T10:21:58.860403+0000","last_unstale":"2024-03-21T10:22:27.217017+0000","last_undegraded":"2024-03-21T10:22:27.217017+0000","last_fullsized":"2024-03-21T10:22:27.217017+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_clean_scrub_stamp":"2024-03-21T10:21:57.813712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T15:15:23.303864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":25,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-21T10:22:27.217341+0000","last_change":"2024-03-21T10:21:58.869275+0000","last_active":"2024-03-21T10:22:27.217341+0000","last_peered":"2024-03-21T10:22:27.217341+0000","last_clean":"2024-03-21T10:22:27.217341+0000","last_became_active":"2024-03-21T10:21:58.868834+0000","last_became_peered":"2024-03-21T10:21:58.868834+0000","last_unstale":"2024-03-21T10:22:27.217341+0000","last_undegraded":"2024-03-21T10:22:27.217341+0000","last_fullsized":"2024-03-21T10:22:27.217341+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_clean_scrub_stamp":"2024-03-21T10:21:57.813712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T14:01:11.074702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":25,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-21T10:22:27.214465+0000","last_change":"2024-03-21T10:21:58.851658+0000","last_active":"2024-03-21T10:22:27.214465+0000","last_peered":"2024-03-21T10:22:27.214465+0000","last_clean":"2024-03-21T10:22:27.214465+0000","last_became_active":"2024-03-21T10:21:58.851295+0000","last_became_peered":"2024-03-21T10:21:58.851295+0000","last_unstale":"2024-03-21T10:22:27.214465+0000","last_undegraded":"2024-03-21T10:22:27.214465+0000","last_fullsized":"2024-03-21T10:22:27.214465+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_clean_scrub_stamp":"2024-03-21T10:21:57.813712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T13:15:32.107783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":25,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-21T10:22:27.212804+0000","last_change":"2024-03-21T10:21:58.859275+0000","last_active":"2024-03-21T10:22:27.212804+0000","last_peered":"2024-03-21T10:22:27.212804+0000","last_clean":"2024-03-21T10:22:27.212804+0000","last_became_active":"2024-03-21T10:21:58.857054+0000","last_became_peered":"2024-03-21T10:21:58.857054+0000","last_unstale":"2024-03-21T10:22:27.212804+0000","last_undegraded":"2024-03-21T10:22:27.212804+0000","last_fullsized":"2024-03-21T10:22:27.212804+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_clean_scrub_stamp":"2024-03-21T10:21:57.813712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T20:05:13.819966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":25,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-21T10:22:27.209486+0000","last_change":"2024-03-21T10:21:58.869025+0000","last_active":"2024-03-21T10:22:27.209486+0000","last_peered":"2024-03-21T10:22:27.209486+0000","last_clean":"2024-03-21T10:22:27.209486+0000","last_became_active":"2024-03-21T10:21:58.868240+0000","last_became_peered":"2024-03-21T10:21:58.868240+0000","last_unstale":"2024-03-21T10:22:27.209486+0000","last_undegraded":"2024-03-21T10:22:27.209486+0000","last_fullsized":"2024-03-21T10:22:27.209486+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_clean_scrub_stamp":"2024-03-21T10:21:57.813712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T21:56:45.642916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":25,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-21T10:22:27.203071+0000","last_change":"2024-03-21T10:21:58.865024+0000","last_active":"2024-03-21T10:22:27.203071+0000","last_peered":"2024-03-21T10:22:27.203071+0000","last_clean":"2024-03-21T10:22:27.203071+0000","last_became_active":"2024-03-21T10:21:58.864649+0000","last_became_peered":"2024-03-21T10:21:58.864649+0000","last_unstale":"2024-03-21T10:22:27.203071+0000","last_undegraded":"2024-03-21T10:22:27.203071+0000","last_fullsized":"2024-03-21T10:22:27.203071+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_clean_scrub_stamp":"2024-03-21T10:21:57.813712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T22:09:03.561283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":25,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-21T10:22:27.221827+0000","last_change":"2024-03-21T10:21:58.865206+0000","last_active":"2024-03-21T10:22:27.221827+0000","last_peered":"2024-03-21T10:22:27.221827+0000","last_clean":"2024-03-21T10:22:27.221827+0000","last_became_active":"2024-03-21T10:21:58.864669+0000","last_became_peered":"2024-03-21T10:21:58.864669+0000","last_unstale":"2024-03-21T10:22:27.221827+0000","last_undegraded":"2024-03-21T10:22:27.221827+0000","last_fullsized":"2024-03-21T10:22:27.221827+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_clean_scrub_stamp":"2024-03-21T10:21:57.813712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T13:57:45.668192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":25,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-21T10:22:27.214598+0000","last_change":"2024-03-21T10:21:58.862664+0000","last_active":"2024-03-21T10:22:27.214598+0000","last_peered":"2024-03-21T10:22:27.214598+0000","last_clean":"2024-03-21T10:22:27.214598+0000","last_became_active":"2024-03-21T10:21:58.862144+0000","last_became_peered":"2024-03-21T10:21:58.862144+0000","last_unstale":"2024-03-21T10:22:27.214598+0000","last_undegraded":"2024-03-21T10:22:27.214598+0000","last_fullsized":"2024-03-21T10:22:27.214598+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_clean_scrub_stamp":"2024-03-21T10:21:57.813712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T11:10:10.225102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":25,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-21T10:22:27.225718+0000","last_change":"2024-03-21T10:21:58.851000+0000","last_active":"2024-03-21T10:22:27.225718+0000","last_peered":"2024-03-21T10:22:27.225718+0000","last_clean":"2024-03-21T10:22:27.225718+0000","last_became_active":"2024-03-21T10:21:58.850563+0000","last_became_peered":"2024-03-21T10:21:58.850563+0000","last_unstale":"2024-03-21T10:22:27.225718+0000","last_undegraded":"2024-03-21T10:22:27.225718+0000","last_fullsized":"2024-03-21T10:22:27.225718+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_clean_scrub_stamp":"2024-03-21T10:21:57.813712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T19:39:08.983713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":25,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-21T10:22:27.204769+0000","last_change":"2024-03-21T10:21:58.838426+0000","last_active":"2024-03-21T10:22:27.204769+0000","last_peered":"2024-03-21T10:22:27.204769+0000","last_clean":"2024-03-21T10:22:27.204769+0000","last_became_active":"2024-03-21T10:21:58.837625+0000","last_became_peered":"2024-03-21T10:21:58.837625+0000","last_unstale":"2024-03-21T10:22:27.204769+0000","last_undegraded":"2024-03-21T10:22:27.204769+0000","last_fullsized":"2024-03-21T10:22:27.204769+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_clean_scrub_stamp":"2024-03-21T10:21:57.813712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T16:22:26.523685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":25,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-21T10:22:27.223935+0000","last_change":"2024-03-21T10:21:58.854114+0000","last_active":"2024-03-21T10:22:27.223935+0000","last_peered":"2024-03-21T10:22:27.223935+0000","last_clean":"2024-03-21T10:22:27.223935+0000","last_became_active":"2024-03-21T10:21:58.853686+0000","last_became_peered":"2024-03-21T10:21:58.853686+0000","last_unstale":"2024-03-21T10:22:27.223935+0000","last_undegraded":"2024-03-21T10:22:27.223935+0000","last_fullsized":"2024-03-21T10:22:27.223935+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_clean_scrub_stamp":"2024-03-21T10:21:57.813712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T11:07:27.346766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":23,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-21T10:22:26.195025+0000","last_change":"2024-03-21T10:21:58.857778+0000","last_active":"2024-03-21T10:22:26.195025+0000","last_peered":"2024-03-21T10:22:26.195025+0000","last_clean":"2024-03-21T10:22:26.195025+0000","last_became_active":"2024-03-21T10:21:58.857336+0000","last_became_peered":"2024-03-21T10:21:58.857336+0000","last_unstale":"2024-03-21T10:22:26.195025+0000","last_undegraded":"2024-03-21T10:22:26.195025+0000","last_fullsized":"2024-03-21T10:22:26.195025+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_clean_scrub_stamp":"2024-03-21T10:21:57.813712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T15:22:57.760341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":25,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-21T10:22:27.202830+0000","last_change":"2024-03-21T10:21:58.869774+0000","last_active":"2024-03-21T10:22:27.202830+0000","last_peered":"2024-03-21T10:22:27.202830+0000","last_clean":"2024-03-21T10:22:27.202830+0000","last_became_active":"2024-03-21T10:21:58.868815+0000","last_became_peered":"2024-03-21T10:21:58.868815+0000","last_unstale":"2024-03-21T10:22:27.202830+0000","last_undegraded":"2024-03-21T10:22:27.202830+0000","last_fullsized":"2024-03-21T10:22:27.202830+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_clean_scrub_stamp":"2024-03-21T10:21:57.813712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T16:57:55.082127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":25,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-21T10:22:27.220667+0000","last_change":"2024-03-21T10:21:58.864746+0000","last_active":"2024-03-21T10:22:27.220667+0000","last_peered":"2024-03-21T10:22:27.220667+0000","last_clean":"2024-03-21T10:22:27.220667+0000","last_became_active":"2024-03-21T10:21:58.864366+0000","last_became_peered":"2024-03-21T10:21:58.864366+0000","last_unstale":"2024-03-21T10:22:27.220667+0000","last_undegraded":"2024-03-21T10:22:27.220667+0000","last_fullsized":"2024-03-21T10:22:27.220667+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_clean_scrub_stamp":"2024-03-21T10:21:57.813712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T13:10:27.303843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4751360,"data_stored":4722944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":15233,"num_objects":75,"num_object_clones":0,"num_object_copies":225,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":75,"num_whiteouts":0,"num_read":14,"num_read_kb":12,"num_write":167,"num_write_kb":64,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":4464,"num_keys_recovered":0,"num_objects_omap":49,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":335872,"data_stored":51317,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":144,"ondisk_log_size":144,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":82,"seq":352187318293,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36336,"kb_used_data":996,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708816,"statfs":{"total":95995035648,"available":95957827584,"internally_reserved":0,"allocated":1019904,"data_stored":494060,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":36174276},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":10,"up_from":74,"seq":317827579932,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32816,"kb_used_data":1632,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712336,"statfs":{"total":95995035648,"available":95961432064,"internally_reserved":0,"allocated":1671168,"data_stored":1098614,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":9,"up_from":69,"seq":296352743459,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28032,"kb_used_data":1012,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717120,"statfs":{"total":95995035648,"available":95966330880,"internally_reserved":0,"allocated":1036288,"data_stored":494296,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":8,"up_from":61,"seq":261993005098,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36352,"kb_used_data":1012,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708800,"statfs":{"total":95995035648,"available":95957811200,"internally_reserved":0,"allocated":1036288,"data_stored":494216,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":7,"up_from":53,"seq":227633266738,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28604,"kb_used_data":1584,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716548,"statfs":{"total":95995035648,"available":95965745152,"internally_reserved":0,"allocated":1622016,"data_stored":1085050,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":48,"seq":206158430265,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36368,"kb_used_data":1024,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708784,"statfs":{"total":95995035648,"available":95957794816,"internally_reserved":0,"allocated":1048576,"data_stored":503548,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574938,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32184,"kb_used_data":1004,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712968,"statfs":{"total":95995035648,"available":95962079232,"internally_reserved":0,"allocated":1028096,"data_stored":497320,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738465,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28068,"kb_used_data":1048,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717084,"statfs":{"total":95995035648,"available":95966294016,"internally_reserved":0,"allocated":1073152,"data_stored":505004,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411413,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36364,"kb_used_data":1024,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708788,"statfs":{"total":95995035648,"available":95957798912,"internally_reserved":0,"allocated":1048576,"data_stored":503554,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":3,"up_from":27,"seq":115964117070,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36376,"kb_used_data":1036,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708776,"statfs":{"total":95995035648,"available":95957786624,"internally_reserved":0,"allocated":1060864,"data_stored":503256,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":35,"seq":150323855431,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28084,"kb_used_data":1068,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717068,"statfs":{"total":95995035648,"available":95966277632,"internally_reserved":0,"allocated":1093632,"data_stored":527680,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":40,"seq":171798691903,"num_pgs":46,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36940,"kb_used_data":1600,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708212,"statfs":{"total":95995035648,"available":95957209088,"internally_reserved":0,"allocated":1638400,"data_stored":1085322,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2178,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3842,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1350,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":24854,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1476,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":722,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1204,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":12535,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":1170,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-21T10:22:31.914 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-21T10:22:31.979 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:22:31 smithi100 ceph-mon[35193]: pgmap v385: 129 pgs: 129 active+clean; 591 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 KiB/s rd, 5.7 KiB/s wr, 16 op/s; 557 B/s, 0 objects/s recovering 2024-03-21T10:22:31.979 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:22:31 smithi100 ceph-mon[35193]: from='client.14898 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:22:31.979 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:22:31 smithi100 ceph-mon[35193]: mgrmap e17: x(active, since 10m), standbys: z, y 2024-03-21T10:22:31.979 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:22:31 smithi100 ceph-mon[35193]: from='client.? 172.21.15.47:0/1096767453' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-21T10:22:31.979 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:22:31 smithi100 ceph-mon[35193]: from='client.? 172.21.15.47:0/3857899676' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-21T10:22:31.990 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:22:31 smithi157 ceph-mon[35581]: pgmap v385: 129 pgs: 129 active+clean; 591 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 KiB/s rd, 5.7 KiB/s wr, 16 op/s; 557 B/s, 0 objects/s recovering 2024-03-21T10:22:31.990 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:22:31 smithi157 ceph-mon[35581]: from='client.14898 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:22:31.990 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:22:31 smithi157 ceph-mon[35581]: mgrmap e17: x(active, since 10m), standbys: z, y 2024-03-21T10:22:31.990 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:22:31 smithi157 ceph-mon[35581]: from='client.? 172.21.15.47:0/1096767453' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-21T10:22:31.990 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:22:31 smithi157 ceph-mon[35581]: from='client.? 172.21.15.47:0/3857899676' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-21T10:22:32.166 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:32.165+0000 7f2fea200640 1 -- 172.21.15.47:0/1485476515 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2fe4105490 msgr2=0x7f2fe410bd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:22:32.166 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:32.165+0000 7f2fea200640 1 --2- 172.21.15.47:0/1485476515 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2fe4105490 0x7f2fe410bd20 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f2fdc009900 tx=0x7f2fdc02bc30 comp rx=0 tx=0).stop 2024-03-21T10:22:32.166 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:32.165+0000 7f2fea200640 1 -- 172.21.15.47:0/1485476515 shutdown_connections 2024-03-21T10:22:32.166 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:32.165+0000 7f2fea200640 1 --2- 172.21.15.47:0/1485476515 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2fe4105490 0x7f2fe410bd20 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:22:32.167 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:32.165+0000 7f2fea200640 1 --2- 172.21.15.47:0/1485476515 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2fe4104ad0 0x7f2fe4104f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:22:32.167 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:32.165+0000 7f2fea200640 1 --2- 172.21.15.47:0/1485476515 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2fe41038e0 0x7f2fe4103d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:22:32.167 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:32.165+0000 7f2fea200640 1 -- 172.21.15.47:0/1485476515 >> 172.21.15.47:0/1485476515 conn(0x7f2fe40fee50 msgr2=0x7f2fe4101290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:22:32.167 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:32.165+0000 7f2fea200640 1 -- 172.21.15.47:0/1485476515 shutdown_connections 2024-03-21T10:22:32.167 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:32.165+0000 7f2fea200640 1 -- 172.21.15.47:0/1485476515 wait complete. 2024-03-21T10:22:32.168 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:32.166+0000 7f2fea200640 1 Processor -- start 2024-03-21T10:22:32.168 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:32.167+0000 7f2fea200640 1 -- start start 2024-03-21T10:22:32.168 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:32.167+0000 7f2fea200640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2fe41038e0 0x7f2fe4199ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:22:32.168 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:32.167+0000 7f2fea200640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2fe4104ad0 0x7f2fe4199ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:22:32.169 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:32.167+0000 7f2fea200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2fe4105490 0x7f2fe41a1060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:22:32.169 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:32.167+0000 7f2fea200640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2fe410f440 con 0x7f2fe41038e0 2024-03-21T10:22:32.169 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:32.167+0000 7f2fea200640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2fe410f2c0 con 0x7f2fe4105490 2024-03-21T10:22:32.169 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:32.167+0000 7f2fea200640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2fe410f5c0 con 0x7f2fe4104ad0 2024-03-21T10:22:32.169 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:32.167+0000 7f2fe3400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2fe41038e0 0x7f2fe4199ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:22:32.169 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:32.167+0000 7f2fe3400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2fe41038e0 0x7f2fe4199ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:56854/0 (socket says 172.21.15.47:56854) 2024-03-21T10:22:32.169 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:32.167+0000 7f2fe3400640 1 -- 172.21.15.47:0/2305025597 learned_addr learned my addr 172.21.15.47:0/2305025597 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:22:32.169 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:32.167+0000 7f2fe3e00640 1 --2- 172.21.15.47:0/2305025597 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2fe4105490 0x7f2fe41a1060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:22:32.169 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:32.167+0000 7f2fdaa00640 1 --2- 172.21.15.47:0/2305025597 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2fe4104ad0 0x7f2fe4199ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:22:32.170 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:32.168+0000 7f2fe3400640 1 -- 172.21.15.47:0/2305025597 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2fe4104ad0 msgr2=0x7f2fe4199ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:22:32.170 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:32.168+0000 7f2fe3400640 1 --2- 172.21.15.47:0/2305025597 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2fe4104ad0 0x7f2fe4199ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:22:32.170 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:32.168+0000 7f2fe3400640 1 -- 172.21.15.47:0/2305025597 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2fe4105490 msgr2=0x7f2fe41a1060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:22:32.170 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:32.168+0000 7f2fe3400640 1 --2- 172.21.15.47:0/2305025597 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2fe4105490 0x7f2fe41a1060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:22:32.170 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:32.168+0000 7f2fe3400640 1 -- 172.21.15.47:0/2305025597 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2fe41a16d0 con 0x7f2fe41038e0 2024-03-21T10:22:32.171 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:32.168+0000 7f2fdaa00640 1 --2- 172.21.15.47:0/2305025597 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2fe4104ad0 0x7f2fe4199ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T10:22:32.171 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:32.168+0000 7f2fe3400640 1 --2- 172.21.15.47:0/2305025597 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2fe41038e0 0x7f2fe4199ab0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f2fd400ebc0 tx=0x7f2fd400c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:22:32.171 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:32.168+0000 7f2fe0c00640 1 -- 172.21.15.47:0/2305025597 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fd4003b70 con 0x7f2fe41038e0 2024-03-21T10:22:32.171 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:32.168+0000 7f2fe0c00640 1 -- 172.21.15.47:0/2305025597 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f2fd4003d10 con 0x7f2fe41038e0 2024-03-21T10:22:32.171 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:32.168+0000 7f2fea200640 1 -- 172.21.15.47:0/2305025597 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2fe41a19c0 con 0x7f2fe41038e0 2024-03-21T10:22:32.171 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:32.168+0000 7f2fe0c00640 1 -- 172.21.15.47:0/2305025597 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fd4019840 con 0x7f2fe41038e0 2024-03-21T10:22:32.171 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:32.168+0000 7f2fea200640 1 -- 172.21.15.47:0/2305025597 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2fe41a1f10 con 0x7f2fe41038e0 2024-03-21T10:22:32.171 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:32.170+0000 7f2fea200640 1 -- 172.21.15.47:0/2305025597 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2fe4068490 con 0x7f2fe41038e0 2024-03-21T10:22:32.172 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:32.172+0000 7f2fe0c00640 1 -- 172.21.15.47:0/2305025597 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 17) v1 ==== 137905+0+0 (secure 0 0 0) 0x7f2fd40199e0 con 0x7f2fe41038e0 2024-03-21T10:22:32.175 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:32.174+0000 7f2fe0c00640 1 --2- 172.21.15.47:0/2305025597 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f2fc80a2fb0 0x7f2fc80a5470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:22:32.175 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:32.174+0000 7f2fe0c00640 1 -- 172.21.15.47:0/2305025597 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9092+0+0 (secure 0 0 0) 0x7f2fd40d0320 con 0x7f2fe41038e0 2024-03-21T10:22:32.175 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:32.174+0000 7f2fdaa00640 1 --2- 172.21.15.47:0/2305025597 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f2fc80a2fb0 0x7f2fc80a5470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:22:32.176 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:32.175+0000 7f2fdaa00640 1 --2- 172.21.15.47:0/2305025597 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f2fc80a2fb0 0x7f2fc80a5470 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f2fcc0058c0 tx=0x7f2fcc005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:22:32.179 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:32.179+0000 7f2fe0c00640 1 -- 172.21.15.47:0/2305025597 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f2fd409cc30 con 0x7f2fe41038e0 2024-03-21T10:22:32.382 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:32.381+0000 7f2fea200640 1 -- 172.21.15.47:0/2305025597 --> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f2fe4061ae0 con 0x7f2fc80a2fb0 2024-03-21T10:22:32.393 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:32.392+0000 7f2fe0c00640 1 -- 172.21.15.47:0/2305025597 <== mgr.14152 v2:172.21.15.47:6800/2823885994 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+340480 (secure 0 0 0) 0x7f2fe4061ae0 con 0x7f2fc80a2fb0 2024-03-21T10:22:32.399 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-21T10:22:32.401 INFO:teuthology.orchestra.run.smithi047.stderr:dumped all 2024-03-21T10:22:32.403 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:32.399+0000 7f2fea200640 1 -- 172.21.15.47:0/2305025597 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f2fc80a2fb0 msgr2=0x7f2fc80a5470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:22:32.403 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:32.399+0000 7f2fea200640 1 --2- 172.21.15.47:0/2305025597 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f2fc80a2fb0 0x7f2fc80a5470 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f2fcc0058c0 tx=0x7f2fcc005850 comp rx=0 tx=0).stop 2024-03-21T10:22:32.403 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:32.399+0000 7f2fea200640 1 -- 172.21.15.47:0/2305025597 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2fe41038e0 msgr2=0x7f2fe4199ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:22:32.403 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:32.399+0000 7f2fea200640 1 --2- 172.21.15.47:0/2305025597 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2fe41038e0 0x7f2fe4199ab0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f2fd400ebc0 tx=0x7f2fd400c6a0 comp rx=0 tx=0).stop 2024-03-21T10:22:32.403 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:32.399+0000 7f2fea200640 1 -- 172.21.15.47:0/2305025597 shutdown_connections 2024-03-21T10:22:32.403 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:32.399+0000 7f2fea200640 1 --2- 172.21.15.47:0/2305025597 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f2fc80a2fb0 0x7f2fc80a5470 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:22:32.403 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:32.399+0000 7f2fea200640 1 --2- 172.21.15.47:0/2305025597 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2fe4105490 0x7f2fe41a1060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:22:32.403 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:32.399+0000 7f2fea200640 1 --2- 172.21.15.47:0/2305025597 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2fe4104ad0 0x7f2fe4199ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:22:32.403 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:32.399+0000 7f2fea200640 1 --2- 172.21.15.47:0/2305025597 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2fe41038e0 0x7f2fe4199ab0 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:22:32.404 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:32.399+0000 7f2fea200640 1 -- 172.21.15.47:0/2305025597 >> 172.21.15.47:0/2305025597 conn(0x7f2fe40fee50 msgr2=0x7f2fe4107dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:22:32.404 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:32.399+0000 7f2fea200640 1 -- 172.21.15.47:0/2305025597 shutdown_connections 2024-03-21T10:22:32.404 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:32.399+0000 7f2fea200640 1 -- 172.21.15.47:0/2305025597 wait complete. 2024-03-21T10:22:32.423 INFO:teuthology.orchestra.run.smithi047.stdout:{"pg_ready":true,"pg_map":{"version":386,"stamp":"2024-03-21T10:22:32.028987+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":605601,"num_objects":77,"num_object_clones":0,"num_object_copies":231,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":77,"num_whiteouts":0,"num_read":150,"num_read_kb":131,"num_write":348,"num_write_kb":3002,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":1185200,"num_keys_recovered":0,"num_objects_omap":49,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":281,"ondisk_log_size":281,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":420,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":396588,"kb_used_data":14108,"kb_used_omap":18,"kb_used_meta":382317,"kb_avail":1124545236,"statfs":{"total":1151940427776,"available":1151534321664,"internally_reserved":0,"allocated":14446592,"data_stored":7841084,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19142,"internal_metadata":391492922},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":17,"apply_latency_ms":17,"commit_latency_ns":17000000,"apply_latency_ns":17000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":10593,"num_objects":53,"num_object_clones":0,"num_object_copies":159,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":53,"num_whiteouts":0,"num_read":14,"num_read_kb":12,"num_write":118,"num_write_kb":45,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":4464,"num_keys_recovered":0,"num_objects_omap":35,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.004693"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":25,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-21T10:22:27.219990+0000","last_change":"2024-03-21T10:21:58.872480+0000","last_active":"2024-03-21T10:22:27.219990+0000","last_peered":"2024-03-21T10:22:27.219990+0000","last_clean":"2024-03-21T10:22:27.219990+0000","last_became_active":"2024-03-21T10:21:58.872143+0000","last_became_peered":"2024-03-21T10:21:58.872143+0000","last_unstale":"2024-03-21T10:22:27.219990+0000","last_undegraded":"2024-03-21T10:22:27.219990+0000","last_fullsized":"2024-03-21T10:22:27.219990+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_clean_scrub_stamp":"2024-03-21T10:21:57.813712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T15:20:00.409201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":25,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-21T10:22:27.210964+0000","last_change":"2024-03-21T10:21:58.871130+0000","last_active":"2024-03-21T10:22:27.210964+0000","last_peered":"2024-03-21T10:22:27.210964+0000","last_clean":"2024-03-21T10:22:27.210964+0000","last_became_active":"2024-03-21T10:21:58.870750+0000","last_became_peered":"2024-03-21T10:21:58.870750+0000","last_unstale":"2024-03-21T10:22:27.210964+0000","last_undegraded":"2024-03-21T10:22:27.210964+0000","last_fullsized":"2024-03-21T10:22:27.210964+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_clean_scrub_stamp":"2024-03-21T10:21:57.813712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T17:38:04.830212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":25,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-21T10:22:27.210597+0000","last_change":"2024-03-21T10:21:58.868026+0000","last_active":"2024-03-21T10:22:27.210597+0000","last_peered":"2024-03-21T10:22:27.210597+0000","last_clean":"2024-03-21T10:22:27.210597+0000","last_became_active":"2024-03-21T10:21:58.859877+0000","last_became_peered":"2024-03-21T10:21:58.859877+0000","last_unstale":"2024-03-21T10:22:27.210597+0000","last_undegraded":"2024-03-21T10:22:27.210597+0000","last_fullsized":"2024-03-21T10:22:27.210597+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_clean_scrub_stamp":"2024-03-21T10:21:57.813712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T14:45:12.870111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":11,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-21T10:22:27.232499+0000","last_change":"2024-03-21T10:22:27.231695+0000","last_active":"2024-03-21T10:22:27.232499+0000","last_peered":"2024-03-21T10:22:27.232499+0000","last_clean":"2024-03-21T10:22:27.232499+0000","last_became_active":"2024-03-21T10:22:27.231149+0000","last_became_peered":"2024-03-21T10:22:27.231149+0000","last_unstale":"2024-03-21T10:22:27.232499+0000","last_undegraded":"2024-03-21T10:22:27.232499+0000","last_fullsized":"2024-03-21T10:22:27.232499+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_clean_scrub_stamp":"2024-03-21T10:21:57.813712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T18:19:19.105366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,2],"acting":[9,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":25,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-21T10:22:27.228786+0000","last_change":"2024-03-21T10:21:58.853998+0000","last_active":"2024-03-21T10:22:27.228786+0000","last_peered":"2024-03-21T10:22:27.228786+0000","last_clean":"2024-03-21T10:22:27.228786+0000","last_became_active":"2024-03-21T10:21:58.853059+0000","last_became_peered":"2024-03-21T10:21:58.853059+0000","last_unstale":"2024-03-21T10:22:27.228786+0000","last_undegraded":"2024-03-21T10:22:27.228786+0000","last_fullsized":"2024-03-21T10:22:27.228786+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_clean_scrub_stamp":"2024-03-21T10:21:57.813712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T17:06:36.859522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":25,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-21T10:22:27.208014+0000","last_change":"2024-03-21T10:21:58.855535+0000","last_active":"2024-03-21T10:22:27.208014+0000","last_peered":"2024-03-21T10:22:27.208014+0000","last_clean":"2024-03-21T10:22:27.208014+0000","last_became_active":"2024-03-21T10:21:58.855181+0000","last_became_peered":"2024-03-21T10:21:58.855181+0000","last_unstale":"2024-03-21T10:22:27.208014+0000","last_undegraded":"2024-03-21T10:22:27.208014+0000","last_fullsized":"2024-03-21T10:22:27.208014+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_clean_scrub_stamp":"2024-03-21T10:21:57.813712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T14:46:35.045646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":25,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-21T10:22:27.215800+0000","last_change":"2024-03-21T10:21:58.856527+0000","last_active":"2024-03-21T10:22:27.215800+0000","last_peered":"2024-03-21T10:22:27.215800+0000","last_clean":"2024-03-21T10:22:27.215800+0000","last_became_active":"2024-03-21T10:21:58.856176+0000","last_became_peered":"2024-03-21T10:21:58.856176+0000","last_unstale":"2024-03-21T10:22:27.215800+0000","last_undegraded":"2024-03-21T10:22:27.215800+0000","last_fullsized":"2024-03-21T10:22:27.215800+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_clean_scrub_stamp":"2024-03-21T10:21:57.813712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T21:34:07.804293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":25,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-21T10:22:27.223195+0000","last_change":"2024-03-21T10:21:58.846260+0000","last_active":"2024-03-21T10:22:27.223195+0000","last_peered":"2024-03-21T10:22:27.223195+0000","last_clean":"2024-03-21T10:22:27.223195+0000","last_became_active":"2024-03-21T10:21:58.844245+0000","last_became_peered":"2024-03-21T10:21:58.844245+0000","last_unstale":"2024-03-21T10:22:27.223195+0000","last_undegraded":"2024-03-21T10:22:27.223195+0000","last_fullsized":"2024-03-21T10:22:27.223195+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_clean_scrub_stamp":"2024-03-21T10:21:57.813712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T16:37:07.480867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":25,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-21T10:22:27.217365+0000","last_change":"2024-03-21T10:21:58.858413+0000","last_active":"2024-03-21T10:22:27.217365+0000","last_peered":"2024-03-21T10:22:27.217365+0000","last_clean":"2024-03-21T10:22:27.217365+0000","last_became_active":"2024-03-21T10:21:58.858007+0000","last_became_peered":"2024-03-21T10:21:58.858007+0000","last_unstale":"2024-03-21T10:22:27.217365+0000","last_undegraded":"2024-03-21T10:22:27.217365+0000","last_fullsized":"2024-03-21T10:22:27.217365+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_clean_scrub_stamp":"2024-03-21T10:21:57.813712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T21:49:12.428918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":25,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-21T10:22:27.209745+0000","last_change":"2024-03-21T10:21:58.864393+0000","last_active":"2024-03-21T10:22:27.209745+0000","last_peered":"2024-03-21T10:22:27.209745+0000","last_clean":"2024-03-21T10:22:27.209745+0000","last_became_active":"2024-03-21T10:21:58.864061+0000","last_became_peered":"2024-03-21T10:21:58.864061+0000","last_unstale":"2024-03-21T10:22:27.209745+0000","last_undegraded":"2024-03-21T10:22:27.209745+0000","last_fullsized":"2024-03-21T10:22:27.209745+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_clean_scrub_stamp":"2024-03-21T10:21:57.813712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T20:19:44.209612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":25,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-21T10:22:27.216667+0000","last_change":"2024-03-21T10:21:58.873885+0000","last_active":"2024-03-21T10:22:27.216667+0000","last_peered":"2024-03-21T10:22:27.216667+0000","last_clean":"2024-03-21T10:22:27.216667+0000","last_became_active":"2024-03-21T10:21:58.869738+0000","last_became_peered":"2024-03-21T10:21:58.869738+0000","last_unstale":"2024-03-21T10:22:27.216667+0000","last_undegraded":"2024-03-21T10:22:27.216667+0000","last_fullsized":"2024-03-21T10:22:27.216667+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_clean_scrub_stamp":"2024-03-21T10:21:57.813712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T13:33:26.816720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":25,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-21T10:22:27.203520+0000","last_change":"2024-03-21T10:21:58.867563+0000","last_active":"2024-03-21T10:22:27.203520+0000","last_peered":"2024-03-21T10:22:27.203520+0000","last_clean":"2024-03-21T10:22:27.203520+0000","last_became_active":"2024-03-21T10:21:58.863455+0000","last_became_peered":"2024-03-21T10:21:58.863455+0000","last_unstale":"2024-03-21T10:22:27.203520+0000","last_undegraded":"2024-03-21T10:22:27.203520+0000","last_fullsized":"2024-03-21T10:22:27.203520+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_clean_scrub_stamp":"2024-03-21T10:21:57.813712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T15:04:16.636126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":31,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-21T10:22:27.213460+0000","last_change":"2024-03-21T10:22:27.213460+0000","last_active":"2024-03-21T10:22:27.213460+0000","last_peered":"2024-03-21T10:22:27.213460+0000","last_clean":"2024-03-21T10:22:27.213460+0000","last_became_active":"2024-03-21T10:22:27.212961+0000","last_became_peered":"2024-03-21T10:22:27.212961+0000","last_unstale":"2024-03-21T10:22:27.213460+0000","last_undegraded":"2024-03-21T10:22:27.213460+0000","last_fullsized":"2024-03-21T10:22:27.213460+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_clean_scrub_stamp":"2024-03-21T10:21:57.813712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T18:26:03.881767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,9],"acting":[6,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":25,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-21T10:22:27.207773+0000","last_change":"2024-03-21T10:21:58.870851+0000","last_active":"2024-03-21T10:22:27.207773+0000","last_peered":"2024-03-21T10:22:27.207773+0000","last_clean":"2024-03-21T10:22:27.207773+0000","last_became_active":"2024-03-21T10:21:58.870132+0000","last_became_peered":"2024-03-21T10:21:58.870132+0000","last_unstale":"2024-03-21T10:22:27.207773+0000","last_undegraded":"2024-03-21T10:22:27.207773+0000","last_fullsized":"2024-03-21T10:22:27.207773+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_clean_scrub_stamp":"2024-03-21T10:21:57.813712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T20:20:04.207090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":25,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-21T10:22:27.208023+0000","last_change":"2024-03-21T10:21:58.865084+0000","last_active":"2024-03-21T10:22:27.208023+0000","last_peered":"2024-03-21T10:22:27.208023+0000","last_clean":"2024-03-21T10:22:27.208023+0000","last_became_active":"2024-03-21T10:21:58.864583+0000","last_became_peered":"2024-03-21T10:21:58.864583+0000","last_unstale":"2024-03-21T10:22:27.208023+0000","last_undegraded":"2024-03-21T10:22:27.208023+0000","last_fullsized":"2024-03-21T10:22:27.208023+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_clean_scrub_stamp":"2024-03-21T10:21:57.813712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T14:09:16.781462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":25,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-21T10:22:27.215574+0000","last_change":"2024-03-21T10:21:58.870370+0000","last_active":"2024-03-21T10:22:27.215574+0000","last_peered":"2024-03-21T10:22:27.215574+0000","last_clean":"2024-03-21T10:22:27.215574+0000","last_became_active":"2024-03-21T10:21:58.870044+0000","last_became_peered":"2024-03-21T10:21:58.870044+0000","last_unstale":"2024-03-21T10:22:27.215574+0000","last_undegraded":"2024-03-21T10:22:27.215574+0000","last_fullsized":"2024-03-21T10:22:27.215574+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_clean_scrub_stamp":"2024-03-21T10:21:57.813712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T16:02:00.342426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":25,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-21T10:22:27.219320+0000","last_change":"2024-03-21T10:21:58.872787+0000","last_active":"2024-03-21T10:22:27.219320+0000","last_peered":"2024-03-21T10:22:27.219320+0000","last_clean":"2024-03-21T10:22:27.219320+0000","last_became_active":"2024-03-21T10:21:58.872390+0000","last_became_peered":"2024-03-21T10:21:58.872390+0000","last_unstale":"2024-03-21T10:22:27.219320+0000","last_undegraded":"2024-03-21T10:22:27.219320+0000","last_fullsized":"2024-03-21T10:22:27.219320+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_clean_scrub_stamp":"2024-03-21T10:21:57.813712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T15:56:51.798173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":25,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-21T10:22:27.208635+0000","last_change":"2024-03-21T10:21:58.858474+0000","last_active":"2024-03-21T10:22:27.208635+0000","last_peered":"2024-03-21T10:22:27.208635+0000","last_clean":"2024-03-21T10:22:27.208635+0000","last_became_active":"2024-03-21T10:21:58.858076+0000","last_became_peered":"2024-03-21T10:21:58.858076+0000","last_unstale":"2024-03-21T10:22:27.208635+0000","last_undegraded":"2024-03-21T10:22:27.208635+0000","last_fullsized":"2024-03-21T10:22:27.208635+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_clean_scrub_stamp":"2024-03-21T10:21:57.813712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T12:25:17.160405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":25,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-21T10:22:27.213834+0000","last_change":"2024-03-21T10:21:58.854156+0000","last_active":"2024-03-21T10:22:27.213834+0000","last_peered":"2024-03-21T10:22:27.213834+0000","last_clean":"2024-03-21T10:22:27.213834+0000","last_became_active":"2024-03-21T10:21:58.853758+0000","last_became_peered":"2024-03-21T10:21:58.853758+0000","last_unstale":"2024-03-21T10:22:27.213834+0000","last_undegraded":"2024-03-21T10:22:27.213834+0000","last_fullsized":"2024-03-21T10:22:27.213834+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_clean_scrub_stamp":"2024-03-21T10:21:57.813712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T16:58:54.854826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":18,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-21T10:22:27.214010+0000","last_change":"2024-03-21T10:22:03.319388+0000","last_active":"2024-03-21T10:22:27.214010+0000","last_peered":"2024-03-21T10:22:27.214010+0000","last_clean":"2024-03-21T10:22:27.214010+0000","last_became_active":"2024-03-21T10:22:03.318846+0000","last_became_peered":"2024-03-21T10:22:03.318846+0000","last_unstale":"2024-03-21T10:22:27.214010+0000","last_undegraded":"2024-03-21T10:22:27.214010+0000","last_fullsized":"2024-03-21T10:22:27.214010+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:22:02.269614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:22:02.269614+0000","last_clean_scrub_stamp":"2024-03-21T10:22:02.269614+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T17:28:16.298825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"92'9","reported_seq":36,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-21T10:22:27.209988+0000","last_change":"2024-03-21T10:21:57.859554+0000","last_active":"2024-03-21T10:22:27.209988+0000","last_peered":"2024-03-21T10:22:27.209988+0000","last_clean":"2024-03-21T10:22:27.209988+0000","last_became_active":"2024-03-21T10:21:57.857997+0000","last_became_peered":"2024-03-21T10:21:57.857997+0000","last_unstale":"2024-03-21T10:22:27.209988+0000","last_undegraded":"2024-03-21T10:22:27.209988+0000","last_fullsized":"2024-03-21T10:22:27.209988+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:21:56.788908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:21:56.788908+0000","last_clean_scrub_stamp":"2024-03-21T10:21:56.788908+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T14:27:47.516832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":25,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-21T10:22:27.224817+0000","last_change":"2024-03-21T10:21:58.850197+0000","last_active":"2024-03-21T10:22:27.224817+0000","last_peered":"2024-03-21T10:22:27.224817+0000","last_clean":"2024-03-21T10:22:27.224817+0000","last_became_active":"2024-03-21T10:21:58.849029+0000","last_became_peered":"2024-03-21T10:21:58.849029+0000","last_unstale":"2024-03-21T10:22:27.224817+0000","last_undegraded":"2024-03-21T10:22:27.224817+0000","last_fullsized":"2024-03-21T10:22:27.224817+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_clean_scrub_stamp":"2024-03-21T10:21:57.813712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T15:46:49.077382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":18,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-21T10:22:27.202873+0000","last_change":"2024-03-21T10:22:03.320716+0000","last_active":"2024-03-21T10:22:27.202873+0000","last_peered":"2024-03-21T10:22:27.202873+0000","last_clean":"2024-03-21T10:22:27.202873+0000","last_became_active":"2024-03-21T10:22:03.320262+0000","last_became_peered":"2024-03-21T10:22:03.320262+0000","last_unstale":"2024-03-21T10:22:27.202873+0000","last_undegraded":"2024-03-21T10:22:27.202873+0000","last_fullsized":"2024-03-21T10:22:27.202873+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:22:02.269614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:22:02.269614+0000","last_clean_scrub_stamp":"2024-03-21T10:22:02.269614+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T16:16:07.423168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"92'2","reported_seq":29,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-21T10:22:27.225025+0000","last_change":"2024-03-21T10:21:57.863642+0000","last_active":"2024-03-21T10:22:27.225025+0000","last_peered":"2024-03-21T10:22:27.225025+0000","last_clean":"2024-03-21T10:22:27.225025+0000","last_became_active":"2024-03-21T10:21:57.863330+0000","last_became_peered":"2024-03-21T10:21:57.863330+0000","last_unstale":"2024-03-21T10:22:27.225025+0000","last_undegraded":"2024-03-21T10:22:27.225025+0000","last_fullsized":"2024-03-21T10:22:27.225025+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:21:56.788908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:21:56.788908+0000","last_clean_scrub_stamp":"2024-03-21T10:21:56.788908+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T14:45:02.131458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":25,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-21T10:22:27.211017+0000","last_change":"2024-03-21T10:21:58.862940+0000","last_active":"2024-03-21T10:22:27.211017+0000","last_peered":"2024-03-21T10:22:27.211017+0000","last_clean":"2024-03-21T10:22:27.211017+0000","last_became_active":"2024-03-21T10:21:58.862489+0000","last_became_peered":"2024-03-21T10:21:58.862489+0000","last_unstale":"2024-03-21T10:22:27.211017+0000","last_undegraded":"2024-03-21T10:22:27.211017+0000","last_fullsized":"2024-03-21T10:22:27.211017+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_clean_scrub_stamp":"2024-03-21T10:21:57.813712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T19:28:02.175642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":18,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-21T10:22:27.203272+0000","last_change":"2024-03-21T10:22:03.318660+0000","last_active":"2024-03-21T10:22:27.203272+0000","last_peered":"2024-03-21T10:22:27.203272+0000","last_clean":"2024-03-21T10:22:27.203272+0000","last_became_active":"2024-03-21T10:22:03.318216+0000","last_became_peered":"2024-03-21T10:22:03.318216+0000","last_unstale":"2024-03-21T10:22:27.203272+0000","last_undegraded":"2024-03-21T10:22:27.203272+0000","last_fullsized":"2024-03-21T10:22:27.203272+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:22:02.269614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:22:02.269614+0000","last_clean_scrub_stamp":"2024-03-21T10:22:02.269614+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T16:36:39.930726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"93'4","reported_seq":17,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-21T10:22:27.240020+0000","last_change":"2024-03-21T10:22:27.239342+0000","last_active":"2024-03-21T10:22:27.240020+0000","last_peered":"2024-03-21T10:22:27.240020+0000","last_clean":"2024-03-21T10:22:27.240020+0000","last_became_active":"2024-03-21T10:22:27.224086+0000","last_became_peered":"2024-03-21T10:22:27.224086+0000","last_unstale":"2024-03-21T10:22:27.240020+0000","last_undegraded":"2024-03-21T10:22:27.240020+0000","last_fullsized":"2024-03-21T10:22:27.240020+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:21:56.788908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:21:56.788908+0000","last_clean_scrub_stamp":"2024-03-21T10:21:56.788908+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T20:46:45.951485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,7],"acting":[11,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":25,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-21T10:22:27.215278+0000","last_change":"2024-03-21T10:21:58.873239+0000","last_active":"2024-03-21T10:22:27.215278+0000","last_peered":"2024-03-21T10:22:27.215278+0000","last_clean":"2024-03-21T10:22:27.215278+0000","last_became_active":"2024-03-21T10:21:58.872877+0000","last_became_peered":"2024-03-21T10:21:58.872877+0000","last_unstale":"2024-03-21T10:22:27.215278+0000","last_undegraded":"2024-03-21T10:22:27.215278+0000","last_fullsized":"2024-03-21T10:22:27.215278+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_clean_scrub_stamp":"2024-03-21T10:21:57.813712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T16:22:49.070697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":18,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-21T10:22:27.202999+0000","last_change":"2024-03-21T10:22:03.311589+0000","last_active":"2024-03-21T10:22:27.202999+0000","last_peered":"2024-03-21T10:22:27.202999+0000","last_clean":"2024-03-21T10:22:27.202999+0000","last_became_active":"2024-03-21T10:22:03.311134+0000","last_became_peered":"2024-03-21T10:22:03.311134+0000","last_unstale":"2024-03-21T10:22:27.202999+0000","last_undegraded":"2024-03-21T10:22:27.202999+0000","last_fullsized":"2024-03-21T10:22:27.202999+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:22:02.269614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:22:02.269614+0000","last_clean_scrub_stamp":"2024-03-21T10:22:02.269614+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T10:22:29.138348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"86'2","reported_seq":29,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-21T10:22:27.232300+0000","last_change":"2024-03-21T10:21:57.858981+0000","last_active":"2024-03-21T10:22:27.232300+0000","last_peered":"2024-03-21T10:22:27.232300+0000","last_clean":"2024-03-21T10:22:27.232300+0000","last_became_active":"2024-03-21T10:21:57.858564+0000","last_became_peered":"2024-03-21T10:21:57.858564+0000","last_unstale":"2024-03-21T10:22:27.232300+0000","last_undegraded":"2024-03-21T10:22:27.232300+0000","last_fullsized":"2024-03-21T10:22:27.232300+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:21:56.788908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:21:56.788908+0000","last_clean_scrub_stamp":"2024-03-21T10:21:56.788908+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T10:45:09.488194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":25,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-21T10:22:27.206512+0000","last_change":"2024-03-21T10:21:58.858053+0000","last_active":"2024-03-21T10:22:27.206512+0000","last_peered":"2024-03-21T10:22:27.206512+0000","last_clean":"2024-03-21T10:22:27.206512+0000","last_became_active":"2024-03-21T10:21:58.857688+0000","last_became_peered":"2024-03-21T10:21:58.857688+0000","last_unstale":"2024-03-21T10:22:27.206512+0000","last_undegraded":"2024-03-21T10:22:27.206512+0000","last_fullsized":"2024-03-21T10:22:27.206512+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_clean_scrub_stamp":"2024-03-21T10:21:57.813712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T19:45:13.043034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":18,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-21T10:22:27.213571+0000","last_change":"2024-03-21T10:22:03.315315+0000","last_active":"2024-03-21T10:22:27.213571+0000","last_peered":"2024-03-21T10:22:27.213571+0000","last_clean":"2024-03-21T10:22:27.213571+0000","last_became_active":"2024-03-21T10:22:03.314861+0000","last_became_peered":"2024-03-21T10:22:03.314861+0000","last_unstale":"2024-03-21T10:22:27.213571+0000","last_undegraded":"2024-03-21T10:22:27.213571+0000","last_fullsized":"2024-03-21T10:22:27.213571+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:22:02.269614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:22:02.269614+0000","last_clean_scrub_stamp":"2024-03-21T10:22:02.269614+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T22:13:09.417907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"91'5","reported_seq":44,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-21T10:22:27.235744+0000","last_change":"2024-03-21T10:22:27.235744+0000","last_active":"2024-03-21T10:22:27.235744+0000","last_peered":"2024-03-21T10:22:27.235744+0000","last_clean":"2024-03-21T10:22:27.235744+0000","last_became_active":"2024-03-21T10:22:27.219999+0000","last_became_peered":"2024-03-21T10:22:27.219999+0000","last_unstale":"2024-03-21T10:22:27.235744+0000","last_undegraded":"2024-03-21T10:22:27.235744+0000","last_fullsized":"2024-03-21T10:22:27.235744+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:21:56.788908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:21:56.788908+0000","last_clean_scrub_stamp":"2024-03-21T10:21:56.788908+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T18:15:41.288034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":622,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7],"acting":[5,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":25,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-21T10:22:27.229634+0000","last_change":"2024-03-21T10:21:58.854752+0000","last_active":"2024-03-21T10:22:27.229634+0000","last_peered":"2024-03-21T10:22:27.229634+0000","last_clean":"2024-03-21T10:22:27.229634+0000","last_became_active":"2024-03-21T10:21:58.854442+0000","last_became_peered":"2024-03-21T10:21:58.854442+0000","last_unstale":"2024-03-21T10:22:27.229634+0000","last_undegraded":"2024-03-21T10:22:27.229634+0000","last_fullsized":"2024-03-21T10:22:27.229634+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_clean_scrub_stamp":"2024-03-21T10:21:57.813712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T15:12:40.590585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":18,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-21T10:22:27.213953+0000","last_change":"2024-03-21T10:22:03.319247+0000","last_active":"2024-03-21T10:22:27.213953+0000","last_peered":"2024-03-21T10:22:27.213953+0000","last_clean":"2024-03-21T10:22:27.213953+0000","last_became_active":"2024-03-21T10:22:03.318731+0000","last_became_peered":"2024-03-21T10:22:03.318731+0000","last_unstale":"2024-03-21T10:22:27.213953+0000","last_undegraded":"2024-03-21T10:22:27.213953+0000","last_fullsized":"2024-03-21T10:22:27.213953+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:22:02.269614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:22:02.269614+0000","last_clean_scrub_stamp":"2024-03-21T10:22:02.269614+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T20:06:36.519559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"93'7","reported_seq":40,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-21T10:22:28.243943+0000","last_change":"2024-03-21T10:22:27.222981+0000","last_active":"2024-03-21T10:22:28.243943+0000","last_peered":"2024-03-21T10:22:28.243943+0000","last_clean":"2024-03-21T10:22:28.243943+0000","last_became_active":"2024-03-21T10:22:27.221088+0000","last_became_peered":"2024-03-21T10:22:27.221088+0000","last_unstale":"2024-03-21T10:22:28.243943+0000","last_undegraded":"2024-03-21T10:22:28.243943+0000","last_fullsized":"2024-03-21T10:22:28.243943+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:21:56.788908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:21:56.788908+0000","last_clean_scrub_stamp":"2024-03-21T10:21:56.788908+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T15:43:11.740353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,6],"acting":[10,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":31,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-21T10:22:27.226733+0000","last_change":"2024-03-21T10:22:27.226733+0000","last_active":"2024-03-21T10:22:27.226733+0000","last_peered":"2024-03-21T10:22:27.226733+0000","last_clean":"2024-03-21T10:22:27.226733+0000","last_became_active":"2024-03-21T10:22:27.225576+0000","last_became_peered":"2024-03-21T10:22:27.225576+0000","last_unstale":"2024-03-21T10:22:27.226733+0000","last_undegraded":"2024-03-21T10:22:27.226733+0000","last_fullsized":"2024-03-21T10:22:27.226733+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_clean_scrub_stamp":"2024-03-21T10:21:57.813712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T20:06:09.534268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,3],"acting":[5,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":18,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-21T10:22:27.218863+0000","last_change":"2024-03-21T10:22:03.321595+0000","last_active":"2024-03-21T10:22:27.218863+0000","last_peered":"2024-03-21T10:22:27.218863+0000","last_clean":"2024-03-21T10:22:27.218863+0000","last_became_active":"2024-03-21T10:22:03.320897+0000","last_became_peered":"2024-03-21T10:22:03.320897+0000","last_unstale":"2024-03-21T10:22:27.218863+0000","last_undegraded":"2024-03-21T10:22:27.218863+0000","last_fullsized":"2024-03-21T10:22:27.218863+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:22:02.269614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:22:02.269614+0000","last_clean_scrub_stamp":"2024-03-21T10:22:02.269614+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T17:37:02.633709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"93'3","reported_seq":30,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-21T10:22:27.202538+0000","last_change":"2024-03-21T10:21:57.854801+0000","last_active":"2024-03-21T10:22:27.202538+0000","last_peered":"2024-03-21T10:22:27.202538+0000","last_clean":"2024-03-21T10:22:27.202538+0000","last_became_active":"2024-03-21T10:21:57.854443+0000","last_became_peered":"2024-03-21T10:21:57.854443+0000","last_unstale":"2024-03-21T10:22:27.202538+0000","last_undegraded":"2024-03-21T10:22:27.202538+0000","last_fullsized":"2024-03-21T10:22:27.202538+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:21:56.788908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:21:56.788908+0000","last_clean_scrub_stamp":"2024-03-21T10:21:56.788908+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T15:49:23.573360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":25,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-21T10:22:27.222969+0000","last_change":"2024-03-21T10:21:58.857167+0000","last_active":"2024-03-21T10:22:27.222969+0000","last_peered":"2024-03-21T10:22:27.222969+0000","last_clean":"2024-03-21T10:22:27.222969+0000","last_became_active":"2024-03-21T10:21:58.856827+0000","last_became_peered":"2024-03-21T10:21:58.856827+0000","last_unstale":"2024-03-21T10:22:27.222969+0000","last_undegraded":"2024-03-21T10:22:27.222969+0000","last_fullsized":"2024-03-21T10:22:27.222969+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_clean_scrub_stamp":"2024-03-21T10:21:57.813712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T15:04:43.848209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":18,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-21T10:22:27.210859+0000","last_change":"2024-03-21T10:22:03.306554+0000","last_active":"2024-03-21T10:22:27.210859+0000","last_peered":"2024-03-21T10:22:27.210859+0000","last_clean":"2024-03-21T10:22:27.210859+0000","last_became_active":"2024-03-21T10:22:03.306087+0000","last_became_peered":"2024-03-21T10:22:03.306087+0000","last_unstale":"2024-03-21T10:22:27.210859+0000","last_undegraded":"2024-03-21T10:22:27.210859+0000","last_fullsized":"2024-03-21T10:22:27.210859+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:22:02.269614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:22:02.269614+0000","last_clean_scrub_stamp":"2024-03-21T10:22:02.269614+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T11:36:35.344770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"91'1","reported_seq":28,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-21T10:22:27.204688+0000","last_change":"2024-03-21T10:21:57.860343+0000","last_active":"2024-03-21T10:22:27.204688+0000","last_peered":"2024-03-21T10:22:27.204688+0000","last_clean":"2024-03-21T10:22:27.204688+0000","last_became_active":"2024-03-21T10:21:57.859808+0000","last_became_peered":"2024-03-21T10:21:57.859808+0000","last_unstale":"2024-03-21T10:22:27.204688+0000","last_undegraded":"2024-03-21T10:22:27.204688+0000","last_fullsized":"2024-03-21T10:22:27.204688+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:21:56.788908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:21:56.788908+0000","last_clean_scrub_stamp":"2024-03-21T10:21:56.788908+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T14:41:24.162795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":25,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-21T10:22:27.208314+0000","last_change":"2024-03-21T10:21:58.850991+0000","last_active":"2024-03-21T10:22:27.208314+0000","last_peered":"2024-03-21T10:22:27.208314+0000","last_clean":"2024-03-21T10:22:27.208314+0000","last_became_active":"2024-03-21T10:21:58.850581+0000","last_became_peered":"2024-03-21T10:21:58.850581+0000","last_unstale":"2024-03-21T10:22:27.208314+0000","last_undegraded":"2024-03-21T10:22:27.208314+0000","last_fullsized":"2024-03-21T10:22:27.208314+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_clean_scrub_stamp":"2024-03-21T10:21:57.813712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T17:07:11.350806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":18,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-21T10:22:27.223291+0000","last_change":"2024-03-21T10:22:03.315333+0000","last_active":"2024-03-21T10:22:27.223291+0000","last_peered":"2024-03-21T10:22:27.223291+0000","last_clean":"2024-03-21T10:22:27.223291+0000","last_became_active":"2024-03-21T10:22:03.313553+0000","last_became_peered":"2024-03-21T10:22:03.313553+0000","last_unstale":"2024-03-21T10:22:27.223291+0000","last_undegraded":"2024-03-21T10:22:27.223291+0000","last_fullsized":"2024-03-21T10:22:27.223291+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:22:02.269614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:22:02.269614+0000","last_clean_scrub_stamp":"2024-03-21T10:22:02.269614+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T15:25:24.290789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"93'4","reported_seq":31,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-21T10:22:27.207857+0000","last_change":"2024-03-21T10:21:57.864930+0000","last_active":"2024-03-21T10:22:27.207857+0000","last_peered":"2024-03-21T10:22:27.207857+0000","last_clean":"2024-03-21T10:22:27.207857+0000","last_became_active":"2024-03-21T10:21:57.863769+0000","last_became_peered":"2024-03-21T10:21:57.863769+0000","last_unstale":"2024-03-21T10:22:27.207857+0000","last_undegraded":"2024-03-21T10:22:27.207857+0000","last_fullsized":"2024-03-21T10:22:27.207857+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:21:56.788908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:21:56.788908+0000","last_clean_scrub_stamp":"2024-03-21T10:21:56.788908+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T21:35:39.144065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":25,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-21T10:22:27.215600+0000","last_change":"2024-03-21T10:21:58.872084+0000","last_active":"2024-03-21T10:22:27.215600+0000","last_peered":"2024-03-21T10:22:27.215600+0000","last_clean":"2024-03-21T10:22:27.215600+0000","last_became_active":"2024-03-21T10:21:58.871725+0000","last_became_peered":"2024-03-21T10:21:58.871725+0000","last_unstale":"2024-03-21T10:22:27.215600+0000","last_undegraded":"2024-03-21T10:22:27.215600+0000","last_fullsized":"2024-03-21T10:22:27.215600+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_clean_scrub_stamp":"2024-03-21T10:21:57.813712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T16:19:37.240053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":18,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-21T10:22:27.207450+0000","last_change":"2024-03-21T10:22:03.317808+0000","last_active":"2024-03-21T10:22:27.207450+0000","last_peered":"2024-03-21T10:22:27.207450+0000","last_clean":"2024-03-21T10:22:27.207450+0000","last_became_active":"2024-03-21T10:22:03.313751+0000","last_became_peered":"2024-03-21T10:22:03.313751+0000","last_unstale":"2024-03-21T10:22:27.207450+0000","last_undegraded":"2024-03-21T10:22:27.207450+0000","last_fullsized":"2024-03-21T10:22:27.207450+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:22:02.269614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:22:02.269614+0000","last_clean_scrub_stamp":"2024-03-21T10:22:02.269614+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T20:05:08.105795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"91'2","reported_seq":29,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-21T10:22:27.216923+0000","last_change":"2024-03-21T10:21:57.865462+0000","last_active":"2024-03-21T10:22:27.216923+0000","last_peered":"2024-03-21T10:22:27.216923+0000","last_clean":"2024-03-21T10:22:27.216923+0000","last_became_active":"2024-03-21T10:21:57.862922+0000","last_became_peered":"2024-03-21T10:21:57.862922+0000","last_unstale":"2024-03-21T10:22:27.216923+0000","last_undegraded":"2024-03-21T10:22:27.216923+0000","last_fullsized":"2024-03-21T10:22:27.216923+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:21:56.788908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:21:56.788908+0000","last_clean_scrub_stamp":"2024-03-21T10:21:56.788908+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T15:31:17.167806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":25,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-21T10:22:27.211329+0000","last_change":"2024-03-21T10:21:58.865459+0000","last_active":"2024-03-21T10:22:27.211329+0000","last_peered":"2024-03-21T10:22:27.211329+0000","last_clean":"2024-03-21T10:22:27.211329+0000","last_became_active":"2024-03-21T10:21:58.860909+0000","last_became_peered":"2024-03-21T10:21:58.860909+0000","last_unstale":"2024-03-21T10:22:27.211329+0000","last_undegraded":"2024-03-21T10:22:27.211329+0000","last_fullsized":"2024-03-21T10:22:27.211329+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_clean_scrub_stamp":"2024-03-21T10:21:57.813712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T15:53:35.110404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":18,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-21T10:22:27.203484+0000","last_change":"2024-03-21T10:22:03.318651+0000","last_active":"2024-03-21T10:22:27.203484+0000","last_peered":"2024-03-21T10:22:27.203484+0000","last_clean":"2024-03-21T10:22:27.203484+0000","last_became_active":"2024-03-21T10:22:03.318108+0000","last_became_peered":"2024-03-21T10:22:03.318108+0000","last_unstale":"2024-03-21T10:22:27.203484+0000","last_undegraded":"2024-03-21T10:22:27.203484+0000","last_fullsized":"2024-03-21T10:22:27.203484+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:22:02.269614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:22:02.269614+0000","last_clean_scrub_stamp":"2024-03-21T10:22:02.269614+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T19:00:04.216749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"91'4","reported_seq":31,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-21T10:22:27.206182+0000","last_change":"2024-03-21T10:21:57.865516+0000","last_active":"2024-03-21T10:22:27.206182+0000","last_peered":"2024-03-21T10:22:27.206182+0000","last_clean":"2024-03-21T10:22:27.206182+0000","last_became_active":"2024-03-21T10:21:57.865012+0000","last_became_peered":"2024-03-21T10:21:57.865012+0000","last_unstale":"2024-03-21T10:22:27.206182+0000","last_undegraded":"2024-03-21T10:22:27.206182+0000","last_fullsized":"2024-03-21T10:22:27.206182+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:21:56.788908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:21:56.788908+0000","last_clean_scrub_stamp":"2024-03-21T10:21:56.788908+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T21:38:02.813453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":25,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-21T10:22:27.215725+0000","last_change":"2024-03-21T10:21:58.870876+0000","last_active":"2024-03-21T10:22:27.215725+0000","last_peered":"2024-03-21T10:22:27.215725+0000","last_clean":"2024-03-21T10:22:27.215725+0000","last_became_active":"2024-03-21T10:21:58.868240+0000","last_became_peered":"2024-03-21T10:21:58.868240+0000","last_unstale":"2024-03-21T10:22:27.215725+0000","last_undegraded":"2024-03-21T10:22:27.215725+0000","last_fullsized":"2024-03-21T10:22:27.215725+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_clean_scrub_stamp":"2024-03-21T10:21:57.813712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T15:16:59.496020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":18,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-21T10:22:27.207905+0000","last_change":"2024-03-21T10:22:03.314854+0000","last_active":"2024-03-21T10:22:27.207905+0000","last_peered":"2024-03-21T10:22:27.207905+0000","last_clean":"2024-03-21T10:22:27.207905+0000","last_became_active":"2024-03-21T10:22:03.314422+0000","last_became_peered":"2024-03-21T10:22:03.314422+0000","last_unstale":"2024-03-21T10:22:27.207905+0000","last_undegraded":"2024-03-21T10:22:27.207905+0000","last_fullsized":"2024-03-21T10:22:27.207905+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:22:02.269614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:22:02.269614+0000","last_clean_scrub_stamp":"2024-03-21T10:22:02.269614+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T12:34:31.519118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.14","version":"93'15","reported_seq":60,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-21T10:22:28.233231+0000","last_change":"2024-03-21T10:22:27.243817+0000","last_active":"2024-03-21T10:22:28.233231+0000","last_peered":"2024-03-21T10:22:28.233231+0000","last_clean":"2024-03-21T10:22:28.233231+0000","last_became_active":"2024-03-21T10:22:27.218873+0000","last_became_peered":"2024-03-21T10:22:27.218873+0000","last_unstale":"2024-03-21T10:22:28.233231+0000","last_undegraded":"2024-03-21T10:22:28.233231+0000","last_fullsized":"2024-03-21T10:22:28.233231+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:21:56.788908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:21:56.788908+0000","last_clean_scrub_stamp":"2024-03-21T10:21:56.788908+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T12:27:43.574451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10442,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16,"num_write_kb":16,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":3842,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,4],"acting":[10,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":25,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-21T10:22:27.232409+0000","last_change":"2024-03-21T10:21:58.860386+0000","last_active":"2024-03-21T10:22:27.232409+0000","last_peered":"2024-03-21T10:22:27.232409+0000","last_clean":"2024-03-21T10:22:27.232409+0000","last_became_active":"2024-03-21T10:21:58.860013+0000","last_became_peered":"2024-03-21T10:21:58.860013+0000","last_unstale":"2024-03-21T10:22:27.232409+0000","last_undegraded":"2024-03-21T10:22:27.232409+0000","last_fullsized":"2024-03-21T10:22:27.232409+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_clean_scrub_stamp":"2024-03-21T10:21:57.813712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T18:02:33.920374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":18,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-21T10:22:27.207748+0000","last_change":"2024-03-21T10:22:03.314083+0000","last_active":"2024-03-21T10:22:27.207748+0000","last_peered":"2024-03-21T10:22:27.207748+0000","last_clean":"2024-03-21T10:22:27.207748+0000","last_became_active":"2024-03-21T10:22:03.313652+0000","last_became_peered":"2024-03-21T10:22:03.313652+0000","last_unstale":"2024-03-21T10:22:27.207748+0000","last_undegraded":"2024-03-21T10:22:27.207748+0000","last_fullsized":"2024-03-21T10:22:27.207748+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:22:02.269614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:22:02.269614+0000","last_clean_scrub_stamp":"2024-03-21T10:22:02.269614+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T13:23:51.256535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"93'9","reported_seq":38,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-21T10:22:28.601592+0000","last_change":"2024-03-21T10:21:57.866716+0000","last_active":"2024-03-21T10:22:28.601592+0000","last_peered":"2024-03-21T10:22:28.601592+0000","last_clean":"2024-03-21T10:22:28.601592+0000","last_became_active":"2024-03-21T10:21:57.860900+0000","last_became_peered":"2024-03-21T10:21:57.860900+0000","last_unstale":"2024-03-21T10:22:28.601592+0000","last_undegraded":"2024-03-21T10:22:28.601592+0000","last_fullsized":"2024-03-21T10:22:28.601592+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:21:56.788908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:21:56.788908+0000","last_clean_scrub_stamp":"2024-03-21T10:21:56.788908+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T13:13:04.937120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":25,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-21T10:22:27.202537+0000","last_change":"2024-03-21T10:21:58.858815+0000","last_active":"2024-03-21T10:22:27.202537+0000","last_peered":"2024-03-21T10:22:27.202537+0000","last_clean":"2024-03-21T10:22:27.202537+0000","last_became_active":"2024-03-21T10:21:58.856483+0000","last_became_peered":"2024-03-21T10:21:58.856483+0000","last_unstale":"2024-03-21T10:22:27.202537+0000","last_undegraded":"2024-03-21T10:22:27.202537+0000","last_fullsized":"2024-03-21T10:22:27.202537+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_clean_scrub_stamp":"2024-03-21T10:21:57.813712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T15:42:38.208093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":18,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-21T10:22:27.219768+0000","last_change":"2024-03-21T10:22:03.319719+0000","last_active":"2024-03-21T10:22:27.219768+0000","last_peered":"2024-03-21T10:22:27.219768+0000","last_clean":"2024-03-21T10:22:27.219768+0000","last_became_active":"2024-03-21T10:22:03.319234+0000","last_became_peered":"2024-03-21T10:22:03.319234+0000","last_unstale":"2024-03-21T10:22:27.219768+0000","last_undegraded":"2024-03-21T10:22:27.219768+0000","last_fullsized":"2024-03-21T10:22:27.219768+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:22:02.269614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:22:02.269614+0000","last_clean_scrub_stamp":"2024-03-21T10:22:02.269614+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T14:11:21.609594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"91'3","reported_seq":30,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-21T10:22:27.206316+0000","last_change":"2024-03-21T10:21:57.862744+0000","last_active":"2024-03-21T10:22:27.206316+0000","last_peered":"2024-03-21T10:22:27.206316+0000","last_clean":"2024-03-21T10:22:27.206316+0000","last_became_active":"2024-03-21T10:21:57.862352+0000","last_became_peered":"2024-03-21T10:21:57.862352+0000","last_unstale":"2024-03-21T10:22:27.206316+0000","last_undegraded":"2024-03-21T10:22:27.206316+0000","last_fullsized":"2024-03-21T10:22:27.206316+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:21:56.788908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:21:56.788908+0000","last_clean_scrub_stamp":"2024-03-21T10:21:56.788908+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T20:19:50.138904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":25,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-21T10:22:27.216267+0000","last_change":"2024-03-21T10:21:58.863514+0000","last_active":"2024-03-21T10:22:27.216267+0000","last_peered":"2024-03-21T10:22:27.216267+0000","last_clean":"2024-03-21T10:22:27.216267+0000","last_became_active":"2024-03-21T10:21:58.861981+0000","last_became_peered":"2024-03-21T10:21:58.861981+0000","last_unstale":"2024-03-21T10:22:27.216267+0000","last_undegraded":"2024-03-21T10:22:27.216267+0000","last_fullsized":"2024-03-21T10:22:27.216267+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_clean_scrub_stamp":"2024-03-21T10:21:57.813712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T15:00:33.813456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":18,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-21T10:22:27.209637+0000","last_change":"2024-03-21T10:22:03.309844+0000","last_active":"2024-03-21T10:22:27.209637+0000","last_peered":"2024-03-21T10:22:27.209637+0000","last_clean":"2024-03-21T10:22:27.209637+0000","last_became_active":"2024-03-21T10:22:03.309448+0000","last_became_peered":"2024-03-21T10:22:03.309448+0000","last_unstale":"2024-03-21T10:22:27.209637+0000","last_undegraded":"2024-03-21T10:22:27.209637+0000","last_fullsized":"2024-03-21T10:22:27.209637+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:22:02.269614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:22:02.269614+0000","last_clean_scrub_stamp":"2024-03-21T10:22:02.269614+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T18:38:51.167166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.11","version":"93'6","reported_seq":42,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-21T10:22:28.586323+0000","last_change":"2024-03-21T10:22:27.223337+0000","last_active":"2024-03-21T10:22:28.586323+0000","last_peered":"2024-03-21T10:22:28.586323+0000","last_clean":"2024-03-21T10:22:28.586323+0000","last_became_active":"2024-03-21T10:22:27.222999+0000","last_became_peered":"2024-03-21T10:22:27.222999+0000","last_unstale":"2024-03-21T10:22:28.586323+0000","last_undegraded":"2024-03-21T10:22:28.586323+0000","last_fullsized":"2024-03-21T10:22:28.586323+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:21:56.788908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:21:56.788908+0000","last_clean_scrub_stamp":"2024-03-21T10:21:56.788908+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T20:40:45.780696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":979,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":4,"num_read_kb":3,"num_write":6,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,10],"acting":[4,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":25,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-21T10:22:27.214464+0000","last_change":"2024-03-21T10:21:58.862225+0000","last_active":"2024-03-21T10:22:27.214464+0000","last_peered":"2024-03-21T10:22:27.214464+0000","last_clean":"2024-03-21T10:22:27.214464+0000","last_became_active":"2024-03-21T10:21:58.861816+0000","last_became_peered":"2024-03-21T10:21:58.861816+0000","last_unstale":"2024-03-21T10:22:27.214464+0000","last_undegraded":"2024-03-21T10:22:27.214464+0000","last_fullsized":"2024-03-21T10:22:27.214464+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_clean_scrub_stamp":"2024-03-21T10:21:57.813712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T18:36:13.420867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":18,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-21T10:22:27.203338+0000","last_change":"2024-03-21T10:22:03.315306+0000","last_active":"2024-03-21T10:22:27.203338+0000","last_peered":"2024-03-21T10:22:27.203338+0000","last_clean":"2024-03-21T10:22:27.203338+0000","last_became_active":"2024-03-21T10:22:03.314839+0000","last_became_peered":"2024-03-21T10:22:03.314839+0000","last_unstale":"2024-03-21T10:22:27.203338+0000","last_undegraded":"2024-03-21T10:22:27.203338+0000","last_fullsized":"2024-03-21T10:22:27.203338+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:22:02.269614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:22:02.269614+0000","last_clean_scrub_stamp":"2024-03-21T10:22:02.269614+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T15:47:45.690434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"92'15","reported_seq":42,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-21T10:22:27.227633+0000","last_change":"2024-03-21T10:21:57.870998+0000","last_active":"2024-03-21T10:22:27.227633+0000","last_peered":"2024-03-21T10:22:27.227633+0000","last_clean":"2024-03-21T10:22:27.227633+0000","last_became_active":"2024-03-21T10:21:57.870605+0000","last_became_peered":"2024-03-21T10:21:57.870605+0000","last_unstale":"2024-03-21T10:22:27.227633+0000","last_undegraded":"2024-03-21T10:22:27.227633+0000","last_fullsized":"2024-03-21T10:22:27.227633+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:21:56.788908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:21:56.788908+0000","last_clean_scrub_stamp":"2024-03-21T10:21:56.788908+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T13:33:54.775122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":25,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-21T10:22:27.204847+0000","last_change":"2024-03-21T10:21:58.858675+0000","last_active":"2024-03-21T10:22:27.204847+0000","last_peered":"2024-03-21T10:22:27.204847+0000","last_clean":"2024-03-21T10:22:27.204847+0000","last_became_active":"2024-03-21T10:21:58.856907+0000","last_became_peered":"2024-03-21T10:21:58.856907+0000","last_unstale":"2024-03-21T10:22:27.204847+0000","last_undegraded":"2024-03-21T10:22:27.204847+0000","last_fullsized":"2024-03-21T10:22:27.204847+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_clean_scrub_stamp":"2024-03-21T10:21:57.813712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T18:17:31.208728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":18,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-21T10:22:27.204692+0000","last_change":"2024-03-21T10:22:03.318257+0000","last_active":"2024-03-21T10:22:27.204692+0000","last_peered":"2024-03-21T10:22:27.204692+0000","last_clean":"2024-03-21T10:22:27.204692+0000","last_became_active":"2024-03-21T10:22:03.317622+0000","last_became_peered":"2024-03-21T10:22:03.317622+0000","last_unstale":"2024-03-21T10:22:27.204692+0000","last_undegraded":"2024-03-21T10:22:27.204692+0000","last_fullsized":"2024-03-21T10:22:27.204692+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:22:02.269614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:22:02.269614+0000","last_clean_scrub_stamp":"2024-03-21T10:22:02.269614+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T17:15:50.272193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"86'1","reported_seq":28,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-21T10:22:27.232910+0000","last_change":"2024-03-21T10:21:57.868014+0000","last_active":"2024-03-21T10:22:27.232910+0000","last_peered":"2024-03-21T10:22:27.232910+0000","last_clean":"2024-03-21T10:22:27.232910+0000","last_became_active":"2024-03-21T10:21:57.867189+0000","last_became_peered":"2024-03-21T10:21:57.867189+0000","last_unstale":"2024-03-21T10:22:27.232910+0000","last_undegraded":"2024-03-21T10:22:27.232910+0000","last_fullsized":"2024-03-21T10:22:27.232910+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:21:56.788908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:21:56.788908+0000","last_clean_scrub_stamp":"2024-03-21T10:21:56.788908+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T22:02:48.704932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":25,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-21T10:22:27.218969+0000","last_change":"2024-03-21T10:21:58.862205+0000","last_active":"2024-03-21T10:22:27.218969+0000","last_peered":"2024-03-21T10:22:27.218969+0000","last_clean":"2024-03-21T10:22:27.218969+0000","last_became_active":"2024-03-21T10:21:58.861186+0000","last_became_peered":"2024-03-21T10:21:58.861186+0000","last_unstale":"2024-03-21T10:22:27.218969+0000","last_undegraded":"2024-03-21T10:22:27.218969+0000","last_fullsized":"2024-03-21T10:22:27.218969+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_clean_scrub_stamp":"2024-03-21T10:21:57.813712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T19:44:49.719845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":18,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-21T10:22:27.229378+0000","last_change":"2024-03-21T10:22:03.636064+0000","last_active":"2024-03-21T10:22:27.229378+0000","last_peered":"2024-03-21T10:22:27.229378+0000","last_clean":"2024-03-21T10:22:27.229378+0000","last_became_active":"2024-03-21T10:22:03.633596+0000","last_became_peered":"2024-03-21T10:22:03.633596+0000","last_unstale":"2024-03-21T10:22:27.229378+0000","last_undegraded":"2024-03-21T10:22:27.229378+0000","last_fullsized":"2024-03-21T10:22:27.229378+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:22:02.269614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:22:02.269614+0000","last_clean_scrub_stamp":"2024-03-21T10:22:02.269614+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T19:32:31.844066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":27,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-21T10:22:27.208381+0000","last_change":"2024-03-21T10:21:57.851955+0000","last_active":"2024-03-21T10:22:27.208381+0000","last_peered":"2024-03-21T10:22:27.208381+0000","last_clean":"2024-03-21T10:22:27.208381+0000","last_became_active":"2024-03-21T10:21:57.851571+0000","last_became_peered":"2024-03-21T10:21:57.851571+0000","last_unstale":"2024-03-21T10:22:27.208381+0000","last_undegraded":"2024-03-21T10:22:27.208381+0000","last_fullsized":"2024-03-21T10:22:27.208381+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:21:56.788908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:21:56.788908+0000","last_clean_scrub_stamp":"2024-03-21T10:21:56.788908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T17:23:40.441895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":25,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-21T10:22:27.207574+0000","last_change":"2024-03-21T10:21:58.856393+0000","last_active":"2024-03-21T10:22:27.207574+0000","last_peered":"2024-03-21T10:22:27.207574+0000","last_clean":"2024-03-21T10:22:27.207574+0000","last_became_active":"2024-03-21T10:21:58.855139+0000","last_became_peered":"2024-03-21T10:21:58.855139+0000","last_unstale":"2024-03-21T10:22:27.207574+0000","last_undegraded":"2024-03-21T10:22:27.207574+0000","last_fullsized":"2024-03-21T10:22:27.207574+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_clean_scrub_stamp":"2024-03-21T10:21:57.813712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T14:24:55.663576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":18,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-21T10:22:27.202914+0000","last_change":"2024-03-21T10:22:03.315586+0000","last_active":"2024-03-21T10:22:27.202914+0000","last_peered":"2024-03-21T10:22:27.202914+0000","last_clean":"2024-03-21T10:22:27.202914+0000","last_became_active":"2024-03-21T10:22:03.315198+0000","last_became_peered":"2024-03-21T10:22:03.315198+0000","last_unstale":"2024-03-21T10:22:27.202914+0000","last_undegraded":"2024-03-21T10:22:27.202914+0000","last_fullsized":"2024-03-21T10:22:27.202914+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:22:02.269614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:22:02.269614+0000","last_clean_scrub_stamp":"2024-03-21T10:22:02.269614+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T14:25:13.534311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"92'1","reported_seq":28,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-21T10:22:27.216126+0000","last_change":"2024-03-21T10:21:57.862094+0000","last_active":"2024-03-21T10:22:27.216126+0000","last_peered":"2024-03-21T10:22:27.216126+0000","last_clean":"2024-03-21T10:22:27.216126+0000","last_became_active":"2024-03-21T10:21:57.860307+0000","last_became_peered":"2024-03-21T10:21:57.860307+0000","last_unstale":"2024-03-21T10:22:27.216126+0000","last_undegraded":"2024-03-21T10:22:27.216126+0000","last_fullsized":"2024-03-21T10:22:27.216126+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:21:56.788908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:21:56.788908+0000","last_clean_scrub_stamp":"2024-03-21T10:21:56.788908+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T12:58:50.701218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":25,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-21T10:22:27.211160+0000","last_change":"2024-03-21T10:21:58.870880+0000","last_active":"2024-03-21T10:22:27.211160+0000","last_peered":"2024-03-21T10:22:27.211160+0000","last_clean":"2024-03-21T10:22:27.211160+0000","last_became_active":"2024-03-21T10:21:58.869882+0000","last_became_peered":"2024-03-21T10:21:58.869882+0000","last_unstale":"2024-03-21T10:22:27.211160+0000","last_undegraded":"2024-03-21T10:22:27.211160+0000","last_fullsized":"2024-03-21T10:22:27.211160+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_clean_scrub_stamp":"2024-03-21T10:21:57.813712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T18:01:42.613732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":18,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-21T10:22:27.203242+0000","last_change":"2024-03-21T10:22:03.634425+0000","last_active":"2024-03-21T10:22:27.203242+0000","last_peered":"2024-03-21T10:22:27.203242+0000","last_clean":"2024-03-21T10:22:27.203242+0000","last_became_active":"2024-03-21T10:22:03.632464+0000","last_became_peered":"2024-03-21T10:22:03.632464+0000","last_unstale":"2024-03-21T10:22:27.203242+0000","last_undegraded":"2024-03-21T10:22:27.203242+0000","last_fullsized":"2024-03-21T10:22:27.203242+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:22:02.269614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:22:02.269614+0000","last_clean_scrub_stamp":"2024-03-21T10:22:02.269614+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T16:03:28.270757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":31,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-21T10:22:27.233910+0000","last_change":"2024-03-21T10:22:27.233910+0000","last_active":"2024-03-21T10:22:27.233910+0000","last_peered":"2024-03-21T10:22:27.233910+0000","last_clean":"2024-03-21T10:22:27.233910+0000","last_became_active":"2024-03-21T10:22:27.233342+0000","last_became_peered":"2024-03-21T10:22:27.233342+0000","last_unstale":"2024-03-21T10:22:27.233910+0000","last_undegraded":"2024-03-21T10:22:27.233910+0000","last_fullsized":"2024-03-21T10:22:27.233910+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_clean_scrub_stamp":"2024-03-21T10:21:57.813712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T19:14:41.147035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,8],"acting":[5,2,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"93'5","reported_seq":33,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-21T10:22:28.585282+0000","last_change":"2024-03-21T10:21:57.856111+0000","last_active":"2024-03-21T10:22:28.585282+0000","last_peered":"2024-03-21T10:22:28.585282+0000","last_clean":"2024-03-21T10:22:28.585282+0000","last_became_active":"2024-03-21T10:21:57.855584+0000","last_became_peered":"2024-03-21T10:21:57.855584+0000","last_unstale":"2024-03-21T10:22:28.585282+0000","last_undegraded":"2024-03-21T10:22:28.585282+0000","last_fullsized":"2024-03-21T10:22:28.585282+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:21:56.788908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:21:56.788908+0000","last_clean_scrub_stamp":"2024-03-21T10:21:56.788908+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T17:53:48.208554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":18,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-21T10:22:27.216392+0000","last_change":"2024-03-21T10:22:03.318831+0000","last_active":"2024-03-21T10:22:27.216392+0000","last_peered":"2024-03-21T10:22:27.216392+0000","last_clean":"2024-03-21T10:22:27.216392+0000","last_became_active":"2024-03-21T10:22:03.318404+0000","last_became_peered":"2024-03-21T10:22:03.318404+0000","last_unstale":"2024-03-21T10:22:27.216392+0000","last_undegraded":"2024-03-21T10:22:27.216392+0000","last_fullsized":"2024-03-21T10:22:27.216392+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:22:02.269614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:22:02.269614+0000","last_clean_scrub_stamp":"2024-03-21T10:22:02.269614+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T12:06:52.043286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":25,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-21T10:22:27.223893+0000","last_change":"2024-03-21T10:21:58.856469+0000","last_active":"2024-03-21T10:22:27.223893+0000","last_peered":"2024-03-21T10:22:27.223893+0000","last_clean":"2024-03-21T10:22:27.223893+0000","last_became_active":"2024-03-21T10:21:58.856032+0000","last_became_peered":"2024-03-21T10:21:58.856032+0000","last_unstale":"2024-03-21T10:22:27.223893+0000","last_undegraded":"2024-03-21T10:22:27.223893+0000","last_fullsized":"2024-03-21T10:22:27.223893+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_clean_scrub_stamp":"2024-03-21T10:21:57.813712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T21:19:59.302418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"93'6","reported_seq":35,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-21T10:22:28.583397+0000","last_change":"2024-03-21T10:21:57.854210+0000","last_active":"2024-03-21T10:22:28.583397+0000","last_peered":"2024-03-21T10:22:28.583397+0000","last_clean":"2024-03-21T10:22:28.583397+0000","last_became_active":"2024-03-21T10:21:57.853787+0000","last_became_peered":"2024-03-21T10:21:57.853787+0000","last_unstale":"2024-03-21T10:22:28.583397+0000","last_undegraded":"2024-03-21T10:22:28.583397+0000","last_fullsized":"2024-03-21T10:22:28.583397+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:21:56.788908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:21:56.788908+0000","last_clean_scrub_stamp":"2024-03-21T10:21:56.788908+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T12:10:42.926436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"23'137","reported_seq":361,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-21T10:22:27.214365+0000","last_change":"2024-03-21T10:20:21.585339+0000","last_active":"2024-03-21T10:22:27.214365+0000","last_peered":"2024-03-21T10:22:27.214365+0000","last_clean":"2024-03-21T10:22:27.214365+0000","last_became_active":"2024-03-21T10:20:21.583999+0000","last_became_peered":"2024-03-21T10:20:21.583999+0000","last_unstale":"2024-03-21T10:22:27.214365+0000","last_undegraded":"2024-03-21T10:22:27.214365+0000","last_fullsized":"2024-03-21T10:22:27.214365+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:15:40.696168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:15:40.696168+0000","last_clean_scrub_stamp":"2024-03-21T10:15:40.696168+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T15:31:54.603722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":18,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-21T10:22:27.208940+0000","last_change":"2024-03-21T10:22:03.634970+0000","last_active":"2024-03-21T10:22:27.208940+0000","last_peered":"2024-03-21T10:22:27.208940+0000","last_clean":"2024-03-21T10:22:27.208940+0000","last_became_active":"2024-03-21T10:22:03.633603+0000","last_became_peered":"2024-03-21T10:22:03.633603+0000","last_unstale":"2024-03-21T10:22:27.208940+0000","last_undegraded":"2024-03-21T10:22:27.208940+0000","last_fullsized":"2024-03-21T10:22:27.208940+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:22:02.269614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:22:02.269614+0000","last_clean_scrub_stamp":"2024-03-21T10:22:02.269614+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T12:13:02.041661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":25,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-21T10:22:27.202200+0000","last_change":"2024-03-21T10:21:58.861159+0000","last_active":"2024-03-21T10:22:27.202200+0000","last_peered":"2024-03-21T10:22:27.202200+0000","last_clean":"2024-03-21T10:22:27.202200+0000","last_became_active":"2024-03-21T10:21:58.860690+0000","last_became_peered":"2024-03-21T10:21:58.860690+0000","last_unstale":"2024-03-21T10:22:27.202200+0000","last_undegraded":"2024-03-21T10:22:27.202200+0000","last_fullsized":"2024-03-21T10:22:27.202200+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_clean_scrub_stamp":"2024-03-21T10:21:57.813712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T16:21:53.632997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"92'3","reported_seq":30,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-21T10:22:27.214557+0000","last_change":"2024-03-21T10:21:57.863581+0000","last_active":"2024-03-21T10:22:27.214557+0000","last_peered":"2024-03-21T10:22:27.214557+0000","last_clean":"2024-03-21T10:22:27.214557+0000","last_became_active":"2024-03-21T10:21:57.863269+0000","last_became_peered":"2024-03-21T10:21:57.863269+0000","last_unstale":"2024-03-21T10:22:27.214557+0000","last_undegraded":"2024-03-21T10:22:27.214557+0000","last_fullsized":"2024-03-21T10:22:27.214557+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:21:56.788908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:21:56.788908+0000","last_clean_scrub_stamp":"2024-03-21T10:21:56.788908+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T14:03:59.930570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":18,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-21T10:22:27.218249+0000","last_change":"2024-03-21T10:22:03.313515+0000","last_active":"2024-03-21T10:22:27.218249+0000","last_peered":"2024-03-21T10:22:27.218249+0000","last_clean":"2024-03-21T10:22:27.218249+0000","last_became_active":"2024-03-21T10:22:03.311862+0000","last_became_peered":"2024-03-21T10:22:03.311862+0000","last_unstale":"2024-03-21T10:22:27.218249+0000","last_undegraded":"2024-03-21T10:22:27.218249+0000","last_fullsized":"2024-03-21T10:22:27.218249+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:22:02.269614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:22:02.269614+0000","last_clean_scrub_stamp":"2024-03-21T10:22:02.269614+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T16:47:50.761205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"91'1","reported_seq":28,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-21T10:22:27.232498+0000","last_change":"2024-03-21T10:21:57.870913+0000","last_active":"2024-03-21T10:22:27.232498+0000","last_peered":"2024-03-21T10:22:27.232498+0000","last_clean":"2024-03-21T10:22:27.232498+0000","last_became_active":"2024-03-21T10:21:57.870578+0000","last_became_peered":"2024-03-21T10:21:57.870578+0000","last_unstale":"2024-03-21T10:22:27.232498+0000","last_undegraded":"2024-03-21T10:22:27.232498+0000","last_fullsized":"2024-03-21T10:22:27.232498+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:21:56.788908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:21:56.788908+0000","last_clean_scrub_stamp":"2024-03-21T10:21:56.788908+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T11:46:48.621501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":25,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-21T10:22:27.203308+0000","last_change":"2024-03-21T10:21:58.852247+0000","last_active":"2024-03-21T10:22:27.203308+0000","last_peered":"2024-03-21T10:22:27.203308+0000","last_clean":"2024-03-21T10:22:27.203308+0000","last_became_active":"2024-03-21T10:21:58.851976+0000","last_became_peered":"2024-03-21T10:21:58.851976+0000","last_unstale":"2024-03-21T10:22:27.203308+0000","last_undegraded":"2024-03-21T10:22:27.203308+0000","last_fullsized":"2024-03-21T10:22:27.203308+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_clean_scrub_stamp":"2024-03-21T10:21:57.813712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T14:27:11.383741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":18,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-21T10:22:27.222696+0000","last_change":"2024-03-21T10:22:03.637671+0000","last_active":"2024-03-21T10:22:27.222696+0000","last_peered":"2024-03-21T10:22:27.222696+0000","last_clean":"2024-03-21T10:22:27.222696+0000","last_became_active":"2024-03-21T10:22:03.634265+0000","last_became_peered":"2024-03-21T10:22:03.634265+0000","last_unstale":"2024-03-21T10:22:27.222696+0000","last_undegraded":"2024-03-21T10:22:27.222696+0000","last_fullsized":"2024-03-21T10:22:27.222696+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:22:02.269614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:22:02.269614+0000","last_clean_scrub_stamp":"2024-03-21T10:22:02.269614+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T15:38:02.214178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"92'3","reported_seq":30,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-21T10:22:27.208779+0000","last_change":"2024-03-21T10:21:57.862027+0000","last_active":"2024-03-21T10:22:27.208779+0000","last_peered":"2024-03-21T10:22:27.208779+0000","last_clean":"2024-03-21T10:22:27.208779+0000","last_became_active":"2024-03-21T10:21:57.860097+0000","last_became_peered":"2024-03-21T10:21:57.860097+0000","last_unstale":"2024-03-21T10:22:27.208779+0000","last_undegraded":"2024-03-21T10:22:27.208779+0000","last_fullsized":"2024-03-21T10:22:27.208779+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:21:56.788908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:21:56.788908+0000","last_clean_scrub_stamp":"2024-03-21T10:21:56.788908+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T13:46:19.562452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":25,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-21T10:22:27.225488+0000","last_change":"2024-03-21T10:21:58.856082+0000","last_active":"2024-03-21T10:22:27.225488+0000","last_peered":"2024-03-21T10:22:27.225488+0000","last_clean":"2024-03-21T10:22:27.225488+0000","last_became_active":"2024-03-21T10:21:58.854172+0000","last_became_peered":"2024-03-21T10:21:58.854172+0000","last_unstale":"2024-03-21T10:22:27.225488+0000","last_undegraded":"2024-03-21T10:22:27.225488+0000","last_fullsized":"2024-03-21T10:22:27.225488+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_clean_scrub_stamp":"2024-03-21T10:21:57.813712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T14:59:33.995716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":18,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-21T10:22:27.213780+0000","last_change":"2024-03-21T10:22:03.310205+0000","last_active":"2024-03-21T10:22:27.213780+0000","last_peered":"2024-03-21T10:22:27.213780+0000","last_clean":"2024-03-21T10:22:27.213780+0000","last_became_active":"2024-03-21T10:22:03.309039+0000","last_became_peered":"2024-03-21T10:22:03.309039+0000","last_unstale":"2024-03-21T10:22:27.213780+0000","last_undegraded":"2024-03-21T10:22:27.213780+0000","last_fullsized":"2024-03-21T10:22:27.213780+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:22:02.269614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:22:02.269614+0000","last_clean_scrub_stamp":"2024-03-21T10:22:02.269614+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T11:16:46.075260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"93'7","reported_seq":35,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-21T10:22:28.599712+0000","last_change":"2024-03-21T10:21:57.857890+0000","last_active":"2024-03-21T10:22:28.599712+0000","last_peered":"2024-03-21T10:22:28.599712+0000","last_clean":"2024-03-21T10:22:28.599712+0000","last_became_active":"2024-03-21T10:21:57.857492+0000","last_became_peered":"2024-03-21T10:21:57.857492+0000","last_unstale":"2024-03-21T10:22:28.599712+0000","last_undegraded":"2024-03-21T10:22:28.599712+0000","last_fullsized":"2024-03-21T10:22:28.599712+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:21:56.788908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:21:56.788908+0000","last_clean_scrub_stamp":"2024-03-21T10:21:56.788908+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T21:18:45.566843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":31,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-21T10:22:27.219339+0000","last_change":"2024-03-21T10:22:27.219339+0000","last_active":"2024-03-21T10:22:27.219339+0000","last_peered":"2024-03-21T10:22:27.219339+0000","last_clean":"2024-03-21T10:22:27.219339+0000","last_became_active":"2024-03-21T10:22:27.218604+0000","last_became_peered":"2024-03-21T10:22:27.218604+0000","last_unstale":"2024-03-21T10:22:27.219339+0000","last_undegraded":"2024-03-21T10:22:27.219339+0000","last_fullsized":"2024-03-21T10:22:27.219339+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_clean_scrub_stamp":"2024-03-21T10:21:57.813712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T15:58:53.161767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,4],"acting":[0,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":18,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-21T10:22:27.203123+0000","last_change":"2024-03-21T10:22:03.633789+0000","last_active":"2024-03-21T10:22:27.203123+0000","last_peered":"2024-03-21T10:22:27.203123+0000","last_clean":"2024-03-21T10:22:27.203123+0000","last_became_active":"2024-03-21T10:22:03.632716+0000","last_became_peered":"2024-03-21T10:22:03.632716+0000","last_unstale":"2024-03-21T10:22:27.203123+0000","last_undegraded":"2024-03-21T10:22:27.203123+0000","last_fullsized":"2024-03-21T10:22:27.203123+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:22:02.269614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:22:02.269614+0000","last_clean_scrub_stamp":"2024-03-21T10:22:02.269614+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T14:08:47.301881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"91'2","reported_seq":29,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-21T10:22:27.215727+0000","last_change":"2024-03-21T10:21:57.862413+0000","last_active":"2024-03-21T10:22:27.215727+0000","last_peered":"2024-03-21T10:22:27.215727+0000","last_clean":"2024-03-21T10:22:27.215727+0000","last_became_active":"2024-03-21T10:21:57.862104+0000","last_became_peered":"2024-03-21T10:21:57.862104+0000","last_unstale":"2024-03-21T10:22:27.215727+0000","last_undegraded":"2024-03-21T10:22:27.215727+0000","last_fullsized":"2024-03-21T10:22:27.215727+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:21:56.788908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:21:56.788908+0000","last_clean_scrub_stamp":"2024-03-21T10:21:56.788908+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T17:49:21.729492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":25,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-21T10:22:27.227390+0000","last_change":"2024-03-21T10:21:58.864688+0000","last_active":"2024-03-21T10:22:27.227390+0000","last_peered":"2024-03-21T10:22:27.227390+0000","last_clean":"2024-03-21T10:22:27.227390+0000","last_became_active":"2024-03-21T10:21:58.862821+0000","last_became_peered":"2024-03-21T10:21:58.862821+0000","last_unstale":"2024-03-21T10:22:27.227390+0000","last_undegraded":"2024-03-21T10:22:27.227390+0000","last_fullsized":"2024-03-21T10:22:27.227390+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_clean_scrub_stamp":"2024-03-21T10:21:57.813712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T11:17:09.650649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":18,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-21T10:22:27.209863+0000","last_change":"2024-03-21T10:22:03.315886+0000","last_active":"2024-03-21T10:22:27.209863+0000","last_peered":"2024-03-21T10:22:27.209863+0000","last_clean":"2024-03-21T10:22:27.209863+0000","last_became_active":"2024-03-21T10:22:03.315460+0000","last_became_peered":"2024-03-21T10:22:03.315460+0000","last_unstale":"2024-03-21T10:22:27.209863+0000","last_undegraded":"2024-03-21T10:22:27.209863+0000","last_fullsized":"2024-03-21T10:22:27.209863+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:22:02.269614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:22:02.269614+0000","last_clean_scrub_stamp":"2024-03-21T10:22:02.269614+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T14:56:12.869771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.9","version":"93'7","reported_seq":34,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-21T10:22:28.597888+0000","last_change":"2024-03-21T10:21:57.868033+0000","last_active":"2024-03-21T10:22:28.597888+0000","last_peered":"2024-03-21T10:22:28.597888+0000","last_clean":"2024-03-21T10:22:28.597888+0000","last_became_active":"2024-03-21T10:21:57.867279+0000","last_became_peered":"2024-03-21T10:21:57.867279+0000","last_unstale":"2024-03-21T10:22:28.597888+0000","last_undegraded":"2024-03-21T10:22:28.597888+0000","last_fullsized":"2024-03-21T10:22:28.597888+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:21:56.788908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:21:56.788908+0000","last_clean_scrub_stamp":"2024-03-21T10:21:56.788908+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T20:05:37.701443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":25,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-21T10:22:27.217015+0000","last_change":"2024-03-21T10:21:58.868455+0000","last_active":"2024-03-21T10:22:27.217015+0000","last_peered":"2024-03-21T10:22:27.217015+0000","last_clean":"2024-03-21T10:22:27.217015+0000","last_became_active":"2024-03-21T10:21:58.867786+0000","last_became_peered":"2024-03-21T10:21:58.867786+0000","last_unstale":"2024-03-21T10:22:27.217015+0000","last_undegraded":"2024-03-21T10:22:27.217015+0000","last_fullsized":"2024-03-21T10:22:27.217015+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_clean_scrub_stamp":"2024-03-21T10:21:57.813712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T19:16:12.258470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":18,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-21T10:22:27.204035+0000","last_change":"2024-03-21T10:22:03.315540+0000","last_active":"2024-03-21T10:22:27.204035+0000","last_peered":"2024-03-21T10:22:27.204035+0000","last_clean":"2024-03-21T10:22:27.204035+0000","last_became_active":"2024-03-21T10:22:03.315006+0000","last_became_peered":"2024-03-21T10:22:03.315006+0000","last_unstale":"2024-03-21T10:22:27.204035+0000","last_undegraded":"2024-03-21T10:22:27.204035+0000","last_fullsized":"2024-03-21T10:22:27.204035+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:22:02.269614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:22:02.269614+0000","last_clean_scrub_stamp":"2024-03-21T10:22:02.269614+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T12:24:46.792737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"93'2","reported_seq":29,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-21T10:22:27.213164+0000","last_change":"2024-03-21T10:21:57.862748+0000","last_active":"2024-03-21T10:22:27.213164+0000","last_peered":"2024-03-21T10:22:27.213164+0000","last_clean":"2024-03-21T10:22:27.213164+0000","last_became_active":"2024-03-21T10:21:57.862350+0000","last_became_peered":"2024-03-21T10:21:57.862350+0000","last_unstale":"2024-03-21T10:22:27.213164+0000","last_undegraded":"2024-03-21T10:22:27.213164+0000","last_fullsized":"2024-03-21T10:22:27.213164+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:21:56.788908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:21:56.788908+0000","last_clean_scrub_stamp":"2024-03-21T10:21:56.788908+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T17:08:47.301758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":25,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-21T10:22:27.225266+0000","last_change":"2024-03-21T10:21:58.866340+0000","last_active":"2024-03-21T10:22:27.225266+0000","last_peered":"2024-03-21T10:22:27.225266+0000","last_clean":"2024-03-21T10:22:27.225266+0000","last_became_active":"2024-03-21T10:21:58.865965+0000","last_became_peered":"2024-03-21T10:21:58.865965+0000","last_unstale":"2024-03-21T10:22:27.225266+0000","last_undegraded":"2024-03-21T10:22:27.225266+0000","last_fullsized":"2024-03-21T10:22:27.225266+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_clean_scrub_stamp":"2024-03-21T10:21:57.813712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T10:42:23.854176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":18,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-21T10:22:27.203089+0000","last_change":"2024-03-21T10:22:03.634108+0000","last_active":"2024-03-21T10:22:27.203089+0000","last_peered":"2024-03-21T10:22:27.203089+0000","last_clean":"2024-03-21T10:22:27.203089+0000","last_became_active":"2024-03-21T10:22:03.633344+0000","last_became_peered":"2024-03-21T10:22:03.633344+0000","last_unstale":"2024-03-21T10:22:27.203089+0000","last_undegraded":"2024-03-21T10:22:27.203089+0000","last_fullsized":"2024-03-21T10:22:27.203089+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:22:02.269614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:22:02.269614+0000","last_clean_scrub_stamp":"2024-03-21T10:22:02.269614+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T21:00:59.893994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"91'4","reported_seq":31,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-21T10:22:27.228877+0000","last_change":"2024-03-21T10:21:57.871520+0000","last_active":"2024-03-21T10:22:27.228877+0000","last_peered":"2024-03-21T10:22:27.228877+0000","last_clean":"2024-03-21T10:22:27.228877+0000","last_became_active":"2024-03-21T10:21:57.871127+0000","last_became_peered":"2024-03-21T10:21:57.871127+0000","last_unstale":"2024-03-21T10:22:27.228877+0000","last_undegraded":"2024-03-21T10:22:27.228877+0000","last_fullsized":"2024-03-21T10:22:27.228877+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:21:56.788908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:21:56.788908+0000","last_clean_scrub_stamp":"2024-03-21T10:21:56.788908+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T20:43:39.537307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":25,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-21T10:22:27.216094+0000","last_change":"2024-03-21T10:21:58.861786+0000","last_active":"2024-03-21T10:22:27.216094+0000","last_peered":"2024-03-21T10:22:27.216094+0000","last_clean":"2024-03-21T10:22:27.216094+0000","last_became_active":"2024-03-21T10:21:58.861428+0000","last_became_peered":"2024-03-21T10:21:58.861428+0000","last_unstale":"2024-03-21T10:22:27.216094+0000","last_undegraded":"2024-03-21T10:22:27.216094+0000","last_fullsized":"2024-03-21T10:22:27.216094+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_clean_scrub_stamp":"2024-03-21T10:21:57.813712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T15:56:30.256266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":18,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-21T10:22:27.216773+0000","last_change":"2024-03-21T10:22:03.305085+0000","last_active":"2024-03-21T10:22:27.216773+0000","last_peered":"2024-03-21T10:22:27.216773+0000","last_clean":"2024-03-21T10:22:27.216773+0000","last_became_active":"2024-03-21T10:22:03.304474+0000","last_became_peered":"2024-03-21T10:22:03.304474+0000","last_unstale":"2024-03-21T10:22:27.216773+0000","last_undegraded":"2024-03-21T10:22:27.216773+0000","last_fullsized":"2024-03-21T10:22:27.216773+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:22:02.269614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:22:02.269614+0000","last_clean_scrub_stamp":"2024-03-21T10:22:02.269614+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T18:28:10.480981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"93'5","reported_seq":35,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-21T10:22:28.589283+0000","last_change":"2024-03-21T10:21:57.860657+0000","last_active":"2024-03-21T10:22:28.589283+0000","last_peered":"2024-03-21T10:22:28.589283+0000","last_clean":"2024-03-21T10:22:28.589283+0000","last_became_active":"2024-03-21T10:21:57.860340+0000","last_became_peered":"2024-03-21T10:21:57.860340+0000","last_unstale":"2024-03-21T10:22:28.589283+0000","last_undegraded":"2024-03-21T10:22:28.589283+0000","last_fullsized":"2024-03-21T10:22:28.589283+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:21:56.788908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:21:56.788908+0000","last_clean_scrub_stamp":"2024-03-21T10:21:56.788908+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T21:58:08.706812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1170,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":25,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-21T10:22:27.216774+0000","last_change":"2024-03-21T10:21:58.874254+0000","last_active":"2024-03-21T10:22:27.216774+0000","last_peered":"2024-03-21T10:22:27.216774+0000","last_clean":"2024-03-21T10:22:27.216774+0000","last_became_active":"2024-03-21T10:21:58.869795+0000","last_became_peered":"2024-03-21T10:21:58.869795+0000","last_unstale":"2024-03-21T10:22:27.216774+0000","last_undegraded":"2024-03-21T10:22:27.216774+0000","last_fullsized":"2024-03-21T10:22:27.216774+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_clean_scrub_stamp":"2024-03-21T10:21:57.813712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T19:51:28.296199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":18,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-21T10:22:27.209002+0000","last_change":"2024-03-21T10:22:03.306761+0000","last_active":"2024-03-21T10:22:27.209002+0000","last_peered":"2024-03-21T10:22:27.209002+0000","last_clean":"2024-03-21T10:22:27.209002+0000","last_became_active":"2024-03-21T10:22:03.305106+0000","last_became_peered":"2024-03-21T10:22:03.305106+0000","last_unstale":"2024-03-21T10:22:27.209002+0000","last_undegraded":"2024-03-21T10:22:27.209002+0000","last_fullsized":"2024-03-21T10:22:27.209002+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:22:02.269614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:22:02.269614+0000","last_clean_scrub_stamp":"2024-03-21T10:22:02.269614+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T21:13:00.846169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"86'1","reported_seq":28,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-21T10:22:27.206783+0000","last_change":"2024-03-21T10:21:57.868012+0000","last_active":"2024-03-21T10:22:27.206783+0000","last_peered":"2024-03-21T10:22:27.206783+0000","last_clean":"2024-03-21T10:22:27.206783+0000","last_became_active":"2024-03-21T10:21:57.867682+0000","last_became_peered":"2024-03-21T10:21:57.867682+0000","last_unstale":"2024-03-21T10:22:27.206783+0000","last_undegraded":"2024-03-21T10:22:27.206783+0000","last_fullsized":"2024-03-21T10:22:27.206783+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:21:56.788908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:21:56.788908+0000","last_clean_scrub_stamp":"2024-03-21T10:21:56.788908+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T16:59:56.061167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":25,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-21T10:22:27.216386+0000","last_change":"2024-03-21T10:21:58.864949+0000","last_active":"2024-03-21T10:22:27.216386+0000","last_peered":"2024-03-21T10:22:27.216386+0000","last_clean":"2024-03-21T10:22:27.216386+0000","last_became_active":"2024-03-21T10:21:58.859796+0000","last_became_peered":"2024-03-21T10:21:58.859796+0000","last_unstale":"2024-03-21T10:22:27.216386+0000","last_undegraded":"2024-03-21T10:22:27.216386+0000","last_fullsized":"2024-03-21T10:22:27.216386+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_clean_scrub_stamp":"2024-03-21T10:21:57.813712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T18:02:51.873103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":18,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-21T10:22:27.204274+0000","last_change":"2024-03-21T10:22:03.308451+0000","last_active":"2024-03-21T10:22:27.204274+0000","last_peered":"2024-03-21T10:22:27.204274+0000","last_clean":"2024-03-21T10:22:27.204274+0000","last_became_active":"2024-03-21T10:22:03.307986+0000","last_became_peered":"2024-03-21T10:22:03.307986+0000","last_unstale":"2024-03-21T10:22:27.204274+0000","last_undegraded":"2024-03-21T10:22:27.204274+0000","last_fullsized":"2024-03-21T10:22:27.204274+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:22:02.269614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:22:02.269614+0000","last_clean_scrub_stamp":"2024-03-21T10:22:02.269614+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T21:08:42.241657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"93'5","reported_seq":32,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-21T10:22:28.592651+0000","last_change":"2024-03-21T10:21:57.851717+0000","last_active":"2024-03-21T10:22:28.592651+0000","last_peered":"2024-03-21T10:22:28.592651+0000","last_clean":"2024-03-21T10:22:28.592651+0000","last_became_active":"2024-03-21T10:21:57.849909+0000","last_became_peered":"2024-03-21T10:21:57.849909+0000","last_unstale":"2024-03-21T10:22:28.592651+0000","last_undegraded":"2024-03-21T10:22:28.592651+0000","last_fullsized":"2024-03-21T10:22:28.592651+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:21:56.788908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:21:56.788908+0000","last_clean_scrub_stamp":"2024-03-21T10:21:56.788908+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T11:41:37.611400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":25,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-21T10:22:27.217017+0000","last_change":"2024-03-21T10:21:58.860892+0000","last_active":"2024-03-21T10:22:27.217017+0000","last_peered":"2024-03-21T10:22:27.217017+0000","last_clean":"2024-03-21T10:22:27.217017+0000","last_became_active":"2024-03-21T10:21:58.860403+0000","last_became_peered":"2024-03-21T10:21:58.860403+0000","last_unstale":"2024-03-21T10:22:27.217017+0000","last_undegraded":"2024-03-21T10:22:27.217017+0000","last_fullsized":"2024-03-21T10:22:27.217017+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_clean_scrub_stamp":"2024-03-21T10:21:57.813712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T15:15:23.303864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":25,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-21T10:22:27.217341+0000","last_change":"2024-03-21T10:21:58.869275+0000","last_active":"2024-03-21T10:22:27.217341+0000","last_peered":"2024-03-21T10:22:27.217341+0000","last_clean":"2024-03-21T10:22:27.217341+0000","last_became_active":"2024-03-21T10:21:58.868834+0000","last_became_peered":"2024-03-21T10:21:58.868834+0000","last_unstale":"2024-03-21T10:22:27.217341+0000","last_undegraded":"2024-03-21T10:22:27.217341+0000","last_fullsized":"2024-03-21T10:22:27.217341+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_clean_scrub_stamp":"2024-03-21T10:21:57.813712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T14:01:11.074702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":25,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-21T10:22:27.214465+0000","last_change":"2024-03-21T10:21:58.851658+0000","last_active":"2024-03-21T10:22:27.214465+0000","last_peered":"2024-03-21T10:22:27.214465+0000","last_clean":"2024-03-21T10:22:27.214465+0000","last_became_active":"2024-03-21T10:21:58.851295+0000","last_became_peered":"2024-03-21T10:21:58.851295+0000","last_unstale":"2024-03-21T10:22:27.214465+0000","last_undegraded":"2024-03-21T10:22:27.214465+0000","last_fullsized":"2024-03-21T10:22:27.214465+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_clean_scrub_stamp":"2024-03-21T10:21:57.813712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T13:15:32.107783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":25,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-21T10:22:27.212804+0000","last_change":"2024-03-21T10:21:58.859275+0000","last_active":"2024-03-21T10:22:27.212804+0000","last_peered":"2024-03-21T10:22:27.212804+0000","last_clean":"2024-03-21T10:22:27.212804+0000","last_became_active":"2024-03-21T10:21:58.857054+0000","last_became_peered":"2024-03-21T10:21:58.857054+0000","last_unstale":"2024-03-21T10:22:27.212804+0000","last_undegraded":"2024-03-21T10:22:27.212804+0000","last_fullsized":"2024-03-21T10:22:27.212804+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_clean_scrub_stamp":"2024-03-21T10:21:57.813712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T20:05:13.819966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":25,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-21T10:22:27.209486+0000","last_change":"2024-03-21T10:21:58.869025+0000","last_active":"2024-03-21T10:22:27.209486+0000","last_peered":"2024-03-21T10:22:27.209486+0000","last_clean":"2024-03-21T10:22:27.209486+0000","last_became_active":"2024-03-21T10:21:58.868240+0000","last_became_peered":"2024-03-21T10:21:58.868240+0000","last_unstale":"2024-03-21T10:22:27.209486+0000","last_undegraded":"2024-03-21T10:22:27.209486+0000","last_fullsized":"2024-03-21T10:22:27.209486+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_clean_scrub_stamp":"2024-03-21T10:21:57.813712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T21:56:45.642916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":25,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-21T10:22:27.203071+0000","last_change":"2024-03-21T10:21:58.865024+0000","last_active":"2024-03-21T10:22:27.203071+0000","last_peered":"2024-03-21T10:22:27.203071+0000","last_clean":"2024-03-21T10:22:27.203071+0000","last_became_active":"2024-03-21T10:21:58.864649+0000","last_became_peered":"2024-03-21T10:21:58.864649+0000","last_unstale":"2024-03-21T10:22:27.203071+0000","last_undegraded":"2024-03-21T10:22:27.203071+0000","last_fullsized":"2024-03-21T10:22:27.203071+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_clean_scrub_stamp":"2024-03-21T10:21:57.813712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T22:09:03.561283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":25,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-21T10:22:27.221827+0000","last_change":"2024-03-21T10:21:58.865206+0000","last_active":"2024-03-21T10:22:27.221827+0000","last_peered":"2024-03-21T10:22:27.221827+0000","last_clean":"2024-03-21T10:22:27.221827+0000","last_became_active":"2024-03-21T10:21:58.864669+0000","last_became_peered":"2024-03-21T10:21:58.864669+0000","last_unstale":"2024-03-21T10:22:27.221827+0000","last_undegraded":"2024-03-21T10:22:27.221827+0000","last_fullsized":"2024-03-21T10:22:27.221827+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_clean_scrub_stamp":"2024-03-21T10:21:57.813712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T13:57:45.668192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":25,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-21T10:22:27.214598+0000","last_change":"2024-03-21T10:21:58.862664+0000","last_active":"2024-03-21T10:22:27.214598+0000","last_peered":"2024-03-21T10:22:27.214598+0000","last_clean":"2024-03-21T10:22:27.214598+0000","last_became_active":"2024-03-21T10:21:58.862144+0000","last_became_peered":"2024-03-21T10:21:58.862144+0000","last_unstale":"2024-03-21T10:22:27.214598+0000","last_undegraded":"2024-03-21T10:22:27.214598+0000","last_fullsized":"2024-03-21T10:22:27.214598+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_clean_scrub_stamp":"2024-03-21T10:21:57.813712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T11:10:10.225102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":25,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-21T10:22:27.225718+0000","last_change":"2024-03-21T10:21:58.851000+0000","last_active":"2024-03-21T10:22:27.225718+0000","last_peered":"2024-03-21T10:22:27.225718+0000","last_clean":"2024-03-21T10:22:27.225718+0000","last_became_active":"2024-03-21T10:21:58.850563+0000","last_became_peered":"2024-03-21T10:21:58.850563+0000","last_unstale":"2024-03-21T10:22:27.225718+0000","last_undegraded":"2024-03-21T10:22:27.225718+0000","last_fullsized":"2024-03-21T10:22:27.225718+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_clean_scrub_stamp":"2024-03-21T10:21:57.813712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T19:39:08.983713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":25,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-21T10:22:27.204769+0000","last_change":"2024-03-21T10:21:58.838426+0000","last_active":"2024-03-21T10:22:27.204769+0000","last_peered":"2024-03-21T10:22:27.204769+0000","last_clean":"2024-03-21T10:22:27.204769+0000","last_became_active":"2024-03-21T10:21:58.837625+0000","last_became_peered":"2024-03-21T10:21:58.837625+0000","last_unstale":"2024-03-21T10:22:27.204769+0000","last_undegraded":"2024-03-21T10:22:27.204769+0000","last_fullsized":"2024-03-21T10:22:27.204769+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_clean_scrub_stamp":"2024-03-21T10:21:57.813712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T16:22:26.523685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":25,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-21T10:22:27.223935+0000","last_change":"2024-03-21T10:21:58.854114+0000","last_active":"2024-03-21T10:22:27.223935+0000","last_peered":"2024-03-21T10:22:27.223935+0000","last_clean":"2024-03-21T10:22:27.223935+0000","last_became_active":"2024-03-21T10:21:58.853686+0000","last_became_peered":"2024-03-21T10:21:58.853686+0000","last_unstale":"2024-03-21T10:22:27.223935+0000","last_undegraded":"2024-03-21T10:22:27.223935+0000","last_fullsized":"2024-03-21T10:22:27.223935+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_clean_scrub_stamp":"2024-03-21T10:21:57.813712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T11:07:27.346766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":25,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-21T10:22:27.216897+0000","last_change":"2024-03-21T10:21:58.857778+0000","last_active":"2024-03-21T10:22:27.216897+0000","last_peered":"2024-03-21T10:22:27.216897+0000","last_clean":"2024-03-21T10:22:27.216897+0000","last_became_active":"2024-03-21T10:21:58.857336+0000","last_became_peered":"2024-03-21T10:21:58.857336+0000","last_unstale":"2024-03-21T10:22:27.216897+0000","last_undegraded":"2024-03-21T10:22:27.216897+0000","last_fullsized":"2024-03-21T10:22:27.216897+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_clean_scrub_stamp":"2024-03-21T10:21:57.813712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T15:22:57.760341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":25,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-21T10:22:27.202830+0000","last_change":"2024-03-21T10:21:58.869774+0000","last_active":"2024-03-21T10:22:27.202830+0000","last_peered":"2024-03-21T10:22:27.202830+0000","last_clean":"2024-03-21T10:22:27.202830+0000","last_became_active":"2024-03-21T10:21:58.868815+0000","last_became_peered":"2024-03-21T10:21:58.868815+0000","last_unstale":"2024-03-21T10:22:27.202830+0000","last_undegraded":"2024-03-21T10:22:27.202830+0000","last_fullsized":"2024-03-21T10:22:27.202830+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_clean_scrub_stamp":"2024-03-21T10:21:57.813712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T16:57:55.082127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":25,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-21T10:22:27.220667+0000","last_change":"2024-03-21T10:21:58.864746+0000","last_active":"2024-03-21T10:22:27.220667+0000","last_peered":"2024-03-21T10:22:27.220667+0000","last_clean":"2024-03-21T10:22:27.220667+0000","last_became_active":"2024-03-21T10:21:58.864366+0000","last_became_peered":"2024-03-21T10:21:58.864366+0000","last_unstale":"2024-03-21T10:22:27.220667+0000","last_undegraded":"2024-03-21T10:22:27.220667+0000","last_fullsized":"2024-03-21T10:22:27.220667+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_clean_scrub_stamp":"2024-03-21T10:21:57.813712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T13:10:27.303843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4751360,"data_stored":4722944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":15233,"num_objects":75,"num_object_clones":0,"num_object_copies":225,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":75,"num_whiteouts":0,"num_read":14,"num_read_kb":12,"num_write":167,"num_write_kb":64,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":4464,"num_keys_recovered":0,"num_objects_omap":49,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":389120,"data_stored":91133,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":144,"ondisk_log_size":144,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":82,"seq":352187318293,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36336,"kb_used_data":996,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708816,"statfs":{"total":95995035648,"available":95957827584,"internally_reserved":0,"allocated":1019904,"data_stored":494060,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":36174276},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":10,"up_from":74,"seq":317827579932,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32816,"kb_used_data":1632,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712336,"statfs":{"total":95995035648,"available":95961432064,"internally_reserved":0,"allocated":1671168,"data_stored":1098614,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":9,"up_from":69,"seq":296352743459,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28032,"kb_used_data":1012,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717120,"statfs":{"total":95995035648,"available":95966330880,"internally_reserved":0,"allocated":1036288,"data_stored":494296,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":8,"up_from":61,"seq":261993005098,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36352,"kb_used_data":1012,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708800,"statfs":{"total":95995035648,"available":95957811200,"internally_reserved":0,"allocated":1036288,"data_stored":494216,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":7,"up_from":53,"seq":227633266738,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28604,"kb_used_data":1584,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716548,"statfs":{"total":95995035648,"available":95965745152,"internally_reserved":0,"allocated":1622016,"data_stored":1085050,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":48,"seq":206158430265,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36368,"kb_used_data":1024,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708784,"statfs":{"total":95995035648,"available":95957794816,"internally_reserved":0,"allocated":1048576,"data_stored":503548,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574939,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32248,"kb_used_data":1072,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712904,"statfs":{"total":95995035648,"available":95962013696,"internally_reserved":0,"allocated":1097728,"data_stored":546484,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738465,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28068,"kb_used_data":1048,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717084,"statfs":{"total":95995035648,"available":95966294016,"internally_reserved":0,"allocated":1073152,"data_stored":505004,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411413,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36364,"kb_used_data":1024,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708788,"statfs":{"total":95995035648,"available":95957798912,"internally_reserved":0,"allocated":1048576,"data_stored":503554,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":3,"up_from":27,"seq":115964117070,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36376,"kb_used_data":1036,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708776,"statfs":{"total":95995035648,"available":95957786624,"internally_reserved":0,"allocated":1060864,"data_stored":503256,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":35,"seq":150323855431,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28084,"kb_used_data":1068,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717068,"statfs":{"total":95995035648,"available":95966277632,"internally_reserved":0,"allocated":1093632,"data_stored":527680,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":40,"seq":171798691903,"num_pgs":46,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36940,"kb_used_data":1600,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708212,"statfs":{"total":95995035648,"available":95957209088,"internally_reserved":0,"allocated":1638400,"data_stored":1085322,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2178,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":43658,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1350,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":24854,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1476,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":722,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1204,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":12535,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":1170,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-21T10:22:32.428 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-03-21T10:22:32.429 INFO:tasks.cephadm:Setting up client nodes... 2024-03-21T10:22:32.429 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2edfee68ad7243bed422a049b6f5f58619300f11 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3056d51a-e76b-11ee-95cd-87774f69a715 -- ceph auth get-or-create client.0 mon 'allow *' osd 'allow *' mds 'allow *' mgr 'allow *' 2024-03-21T10:22:32.714 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:22:32 smithi047 ceph-mon[31547]: from='client.14916 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-21T10:22:32.956 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/3056d51a-e76b-11ee-95cd-87774f69a715/mon.a/config 2024-03-21T10:22:32.979 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:22:32 smithi100 ceph-mon[35193]: from='client.14916 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-21T10:22:32.989 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:22:32 smithi157 ceph-mon[35581]: from='client.14916 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-21T10:22:33.964 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:22:33 smithi047 ceph-mon[31547]: pgmap v386: 129 pgs: 129 active+clean; 591 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 KiB/s rd, 5.6 KiB/s wr, 16 op/s; 557 B/s, 0 objects/s recovering 2024-03-21T10:22:33.964 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:22:33 smithi047 ceph-mon[31547]: from='client.14922 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-21T10:22:33.979 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:22:33 smithi100 ceph-mon[35193]: pgmap v386: 129 pgs: 129 active+clean; 591 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 KiB/s rd, 5.6 KiB/s wr, 16 op/s; 557 B/s, 0 objects/s recovering 2024-03-21T10:22:33.979 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:22:33 smithi100 ceph-mon[35193]: from='client.14922 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-21T10:22:33.989 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:22:33 smithi157 ceph-mon[35581]: pgmap v386: 129 pgs: 129 active+clean; 591 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 KiB/s rd, 5.6 KiB/s wr, 16 op/s; 557 B/s, 0 objects/s recovering 2024-03-21T10:22:33.990 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:22:33 smithi157 ceph-mon[35581]: from='client.14922 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-21T10:22:35.377 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:35.373+0000 7f67f2c00700 1 -- 172.21.15.47:0/2476945316 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f67ec101340 msgr2=0x7f67ec1017c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:22:35.377 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:35.373+0000 7f67f2c00700 1 --2- 172.21.15.47:0/2476945316 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f67ec101340 0x7f67ec1017c0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f67d8009990 tx=0x7f67d8009ca0 comp rx=0 tx=0).stop 2024-03-21T10:22:35.377 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:35.374+0000 7f67f2c00700 1 -- 172.21.15.47:0/2476945316 shutdown_connections 2024-03-21T10:22:35.377 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:35.374+0000 7f67f2c00700 1 --2- 172.21.15.47:0/2476945316 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f67ec101d00 0x7f67ec1095a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:22:35.377 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:35.374+0000 7f67f2c00700 1 --2- 172.21.15.47:0/2476945316 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f67ec1026e0 0x7f67ec102b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:22:35.377 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:35.374+0000 7f67f2c00700 1 --2- 172.21.15.47:0/2476945316 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f67ec101340 0x7f67ec1017c0 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:22:35.377 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:35.374+0000 7f67f2c00700 1 -- 172.21.15.47:0/2476945316 >> 172.21.15.47:0/2476945316 conn(0x7f67ec0fbe30 msgr2=0x7f67ec0fe270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:22:35.377 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:35.374+0000 7f67f2c00700 1 -- 172.21.15.47:0/2476945316 shutdown_connections 2024-03-21T10:22:35.377 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:35.374+0000 7f67f2c00700 1 -- 172.21.15.47:0/2476945316 wait complete. 2024-03-21T10:22:35.379 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:35.375+0000 7f67f2c00700 1 Processor -- start 2024-03-21T10:22:35.379 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:35.375+0000 7f67f2c00700 1 -- start start 2024-03-21T10:22:35.379 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:35.376+0000 7f67f2c00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f67ec101340 0x7f67ec19a530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:22:35.379 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:35.376+0000 7f67f2c00700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f67ec101d00 0x7f67ec19aa70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:22:35.380 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:35.376+0000 7f67f2c00700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f67ec1026e0 0x7f67ec1a1af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:22:35.380 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:35.376+0000 7f67f2c00700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67ec10cc20 con 0x7f67ec101d00 2024-03-21T10:22:35.380 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:35.376+0000 7f67f2c00700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67ec10caa0 con 0x7f67ec1026e0 2024-03-21T10:22:35.380 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:35.376+0000 7f67f2c00700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67ec10cda0 con 0x7f67ec101340 2024-03-21T10:22:35.380 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:35.376+0000 7f67eb400700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f67ec101d00 0x7f67ec19aa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:22:35.380 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:35.376+0000 7f67eb400700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f67ec101d00 0x7f67ec19aa70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:56882/0 (socket says 172.21.15.47:56882) 2024-03-21T10:22:35.380 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:35.376+0000 7f67eb400700 1 -- 172.21.15.47:0/95450818 learned_addr learned my addr 172.21.15.47:0/95450818 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:22:35.380 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:35.376+0000 7f67ebe00700 1 --2- 172.21.15.47:0/95450818 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f67ec101340 0x7f67ec19a530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:22:35.380 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:35.377+0000 7f67f0c00700 1 --2- 172.21.15.47:0/95450818 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f67ec1026e0 0x7f67ec1a1af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:22:35.380 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:35.377+0000 7f67eb400700 1 -- 172.21.15.47:0/95450818 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f67ec101340 msgr2=0x7f67ec19a530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:22:35.380 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:35.377+0000 7f67eb400700 1 --2- 172.21.15.47:0/95450818 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f67ec101340 0x7f67ec19a530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:22:35.380 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:35.377+0000 7f67eb400700 1 -- 172.21.15.47:0/95450818 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f67ec1026e0 msgr2=0x7f67ec1a1af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:22:35.380 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:35.377+0000 7f67eb400700 1 --2- 172.21.15.47:0/95450818 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f67ec1026e0 0x7f67ec1a1af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:22:35.381 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:35.377+0000 7f67eb400700 1 -- 172.21.15.47:0/95450818 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67ec1a2190 con 0x7f67ec101d00 2024-03-21T10:22:35.381 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:35.377+0000 7f67f0c00700 1 --2- 172.21.15.47:0/95450818 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f67ec1026e0 0x7f67ec1a1af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T10:22:35.381 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:35.377+0000 7f67eb400700 1 --2- 172.21.15.47:0/95450818 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f67ec101d00 0x7f67ec19aa70 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f67d80098e0 tx=0x7f67d8009580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:22:35.381 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:35.378+0000 7f67e8c00700 1 -- 172.21.15.47:0/95450818 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67d8026070 con 0x7f67ec101d00 2024-03-21T10:22:35.382 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:35.378+0000 7f67e8c00700 1 -- 172.21.15.47:0/95450818 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f67d8021410 con 0x7f67ec101d00 2024-03-21T10:22:35.382 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:35.378+0000 7f67e8c00700 1 -- 172.21.15.47:0/95450818 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67d8017650 con 0x7f67ec101d00 2024-03-21T10:22:35.382 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:35.378+0000 7f67f2c00700 1 -- 172.21.15.47:0/95450818 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f67ec1a2420 con 0x7f67ec101d00 2024-03-21T10:22:35.382 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:35.378+0000 7f67f2c00700 1 -- 172.21.15.47:0/95450818 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f67ec1a28d0 con 0x7f67ec101d00 2024-03-21T10:22:35.382 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:35.379+0000 7f67f2c00700 1 -- 172.21.15.47:0/95450818 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f67ec0506a0 con 0x7f67ec101d00 2024-03-21T10:22:35.383 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:35.381+0000 7f67e8c00700 1 -- 172.21.15.47:0/95450818 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 17) v1 ==== 137905+0+0 (secure 0 0 0) 0x7f67d8029070 con 0x7f67ec101d00 2024-03-21T10:22:35.385 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:35.383+0000 7f67e8c00700 1 --2- 172.21.15.47:0/95450818 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f67d00a3170 0x7f67d00a5630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:22:35.385 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:35.383+0000 7f67e8c00700 1 -- 172.21.15.47:0/95450818 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9092+0+0 (secure 0 0 0) 0x7f67d80d0750 con 0x7f67ec101d00 2024-03-21T10:22:35.385 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:35.383+0000 7f67ebe00700 1 --2- 172.21.15.47:0/95450818 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f67d00a3170 0x7f67d00a5630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:22:35.386 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:35.384+0000 7f67ebe00700 1 --2- 172.21.15.47:0/95450818 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f67d00a3170 0x7f67d00a5630 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f67ec06a340 tx=0x7f67dc00b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:22:35.393 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:35.391+0000 7f67e8c00700 1 -- 172.21.15.47:0/95450818 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f67d805d740 con 0x7f67ec101d00 2024-03-21T10:22:35.732 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:35.729+0000 7f67f2c00700 1 -- 172.21.15.47:0/95450818 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} v 0) v1 -- 0x7f67ec1a2d30 con 0x7f67ec101d00 2024-03-21T10:22:35.736 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:35.734+0000 7f67e8c00700 1 -- 172.21.15.47:0/95450818 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]=0 v33) v1 ==== 170+0+59 (secure 0 0 0) 0x7f67d8013070 con 0x7f67ec101d00 2024-03-21T10:22:35.736 INFO:teuthology.orchestra.run.smithi047.stdout:[client.0] 2024-03-21T10:22:35.736 INFO:teuthology.orchestra.run.smithi047.stdout: key = AQBrCvxljuClKxAADu6MTqThHwjsi1UXGaMh2w== 2024-03-21T10:22:35.740 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:35.738+0000 7f67f2c00700 1 -- 172.21.15.47:0/95450818 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f67d00a3170 msgr2=0x7f67d00a5630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:22:35.740 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:35.738+0000 7f67f2c00700 1 --2- 172.21.15.47:0/95450818 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f67d00a3170 0x7f67d00a5630 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f67ec06a340 tx=0x7f67dc00b410 comp rx=0 tx=0).stop 2024-03-21T10:22:35.740 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:35.738+0000 7f67f2c00700 1 -- 172.21.15.47:0/95450818 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f67ec101d00 msgr2=0x7f67ec19aa70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:22:35.740 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:35.738+0000 7f67f2c00700 1 --2- 172.21.15.47:0/95450818 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f67ec101d00 0x7f67ec19aa70 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f67d80098e0 tx=0x7f67d8009580 comp rx=0 tx=0).stop 2024-03-21T10:22:35.741 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:35.738+0000 7f67f2c00700 1 -- 172.21.15.47:0/95450818 shutdown_connections 2024-03-21T10:22:35.741 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:35.738+0000 7f67f2c00700 1 --2- 172.21.15.47:0/95450818 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f67d00a3170 0x7f67d00a5630 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:22:35.741 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:35.738+0000 7f67f2c00700 1 --2- 172.21.15.47:0/95450818 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f67ec1026e0 0x7f67ec1a1af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:22:35.741 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:35.738+0000 7f67f2c00700 1 --2- 172.21.15.47:0/95450818 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f67ec101340 0x7f67ec19a530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:22:35.741 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:35.738+0000 7f67f2c00700 1 --2- 172.21.15.47:0/95450818 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f67ec101d00 0x7f67ec19aa70 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:22:35.741 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:35.738+0000 7f67f2c00700 1 -- 172.21.15.47:0/95450818 >> 172.21.15.47:0/95450818 conn(0x7f67ec0fbe30 msgr2=0x7f67ec0fd800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:22:35.741 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:35.738+0000 7f67f2c00700 1 -- 172.21.15.47:0/95450818 shutdown_connections 2024-03-21T10:22:35.741 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:35.739+0000 7f67f2c00700 1 -- 172.21.15.47:0/95450818 wait complete. 2024-03-21T10:22:35.964 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:22:35 smithi047 ceph-mon[31547]: pgmap v387: 129 pgs: 129 active+clean; 624 KiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 12 KiB/s wr, 32 op/s; 557 B/s, 0 objects/s recovering 2024-03-21T10:22:35.979 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:22:35 smithi100 ceph-mon[35193]: pgmap v387: 129 pgs: 129 active+clean; 624 KiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 12 KiB/s wr, 32 op/s; 557 B/s, 0 objects/s recovering 2024-03-21T10:22:35.989 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:22:35 smithi157 ceph-mon[35581]: pgmap v387: 129 pgs: 129 active+clean; 624 KiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 12 KiB/s wr, 32 op/s; 557 B/s, 0 objects/s recovering 2024-03-21T10:22:36.742 DEBUG:teuthology.orchestra.run.smithi047:> set -ex 2024-03-21T10:22:36.742 DEBUG:teuthology.orchestra.run.smithi047:> sudo dd of=/etc/ceph/ceph.client.0.keyring 2024-03-21T10:22:36.742 DEBUG:teuthology.orchestra.run.smithi047:> sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-03-21T10:22:36.890 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:22:36 smithi047 ceph-mon[31547]: from='client.? 172.21.15.47:0/95450818' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-03-21T10:22:36.890 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:22:36 smithi047 ceph-mon[31547]: from='client.? 172.21.15.47:0/95450818' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-03-21T10:22:36.911 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-03-21T10:22:36.911 INFO:tasks.cephadm.ceph_manager.ceph:waiting for mgr available 2024-03-21T10:22:36.911 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2edfee68ad7243bed422a049b6f5f58619300f11 shell --fsid 3056d51a-e76b-11ee-95cd-87774f69a715 -- ceph mgr dump --format=json 2024-03-21T10:22:36.979 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:22:36 smithi100 ceph-mon[35193]: from='client.? 172.21.15.47:0/95450818' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-03-21T10:22:36.980 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:22:36 smithi100 ceph-mon[35193]: from='client.? 172.21.15.47:0/95450818' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-03-21T10:22:36.990 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:22:36 smithi157 ceph-mon[35581]: from='client.? 172.21.15.47:0/95450818' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-03-21T10:22:36.990 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:22:36 smithi157 ceph-mon[35581]: from='client.? 172.21.15.47:0/95450818' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-03-21T10:22:37.437 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/3056d51a-e76b-11ee-95cd-87774f69a715/mon.a/config 2024-03-21T10:22:37.899 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:22:37 smithi047 ceph-mon[31547]: pgmap v388: 129 pgs: 129 active+clean; 624 KiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 9.7 KiB/s wr, 26 op/s; 453 B/s, 0 objects/s recovering 2024-03-21T10:22:38.229 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:22:37 smithi100 ceph-mon[35193]: pgmap v388: 129 pgs: 129 active+clean; 624 KiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 9.7 KiB/s wr, 26 op/s; 453 B/s, 0 objects/s recovering 2024-03-21T10:22:38.240 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:22:37 smithi157 ceph-mon[35581]: pgmap v388: 129 pgs: 129 active+clean; 624 KiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 9.7 KiB/s wr, 26 op/s; 453 B/s, 0 objects/s recovering 2024-03-21T10:22:39.734 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:39.731+0000 7fd2f1000700 1 -- 172.21.15.47:0/3293348489 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd2ec0766c0 msgr2=0x7fd2ec076aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:22:39.734 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:39.731+0000 7fd2f1000700 1 --2- 172.21.15.47:0/3293348489 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd2ec0766c0 0x7fd2ec076aa0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7fd2dc009990 tx=0x7fd2dc009ca0 comp rx=0 tx=0).stop 2024-03-21T10:22:39.734 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:39.731+0000 7fd2f1000700 1 -- 172.21.15.47:0/3293348489 shutdown_connections 2024-03-21T10:22:39.734 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:39.731+0000 7fd2f1000700 1 --2- 172.21.15.47:0/3293348489 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd2ec113020 0x7fd2ec115410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:22:39.734 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:39.731+0000 7fd2f1000700 1 --2- 172.21.15.47:0/3293348489 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd2ec0766c0 0x7fd2ec076aa0 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:22:39.734 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:39.731+0000 7fd2f1000700 1 --2- 172.21.15.47:0/3293348489 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd2ec076fe0 0x7fd2ec112ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:22:39.735 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:39.731+0000 7fd2f1000700 1 -- 172.21.15.47:0/3293348489 >> 172.21.15.47:0/3293348489 conn(0x7fd2ec0ffe90 msgr2=0x7fd2ec1022b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:22:39.735 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:39.731+0000 7fd2f1000700 1 -- 172.21.15.47:0/3293348489 shutdown_connections 2024-03-21T10:22:39.735 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:39.731+0000 7fd2f1000700 1 -- 172.21.15.47:0/3293348489 wait complete. 2024-03-21T10:22:39.736 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:39.733+0000 7fd2f1000700 1 Processor -- start 2024-03-21T10:22:39.736 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:39.733+0000 7fd2f1000700 1 -- start start 2024-03-21T10:22:39.737 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:39.733+0000 7fd2f1000700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd2ec0766c0 0x7fd2ec19eea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:22:39.737 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:39.733+0000 7fd2f1000700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd2ec076fe0 0x7fd2ec19f3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:22:39.737 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:39.733+0000 7fd2f1000700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd2ec113020 0x7fd2ec1a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:22:39.737 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:39.733+0000 7fd2f1000700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2ec118990 con 0x7fd2ec113020 2024-03-21T10:22:39.737 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:39.733+0000 7fd2f1000700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2ec118810 con 0x7fd2ec076fe0 2024-03-21T10:22:39.737 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:39.733+0000 7fd2f1000700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2ec118b10 con 0x7fd2ec0766c0 2024-03-21T10:22:39.737 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:39.734+0000 7fd2e9400700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd2ec076fe0 0x7fd2ec19f3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:22:39.737 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:39.734+0000 7fd2e9400700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd2ec076fe0 0x7fd2ec19f3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:58278/0 (socket says 172.21.15.47:58278) 2024-03-21T10:22:39.737 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:39.734+0000 7fd2e9400700 1 -- 172.21.15.47:0/2190204713 learned_addr learned my addr 172.21.15.47:0/2190204713 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:22:39.737 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:39.734+0000 7fd2ea800700 1 --2- 172.21.15.47:0/2190204713 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd2ec113020 0x7fd2ec1a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:22:39.737 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:39.734+0000 7fd2e9e00700 1 --2- 172.21.15.47:0/2190204713 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd2ec0766c0 0x7fd2ec19eea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:22:39.737 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:39.734+0000 7fd2e9400700 1 -- 172.21.15.47:0/2190204713 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd2ec0766c0 msgr2=0x7fd2ec19eea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:22:39.738 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:39.734+0000 7fd2e9400700 1 --2- 172.21.15.47:0/2190204713 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd2ec0766c0 0x7fd2ec19eea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:22:39.738 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:39.734+0000 7fd2e9400700 1 -- 172.21.15.47:0/2190204713 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd2ec113020 msgr2=0x7fd2ec1a3770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:22:39.738 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:39.734+0000 7fd2e9400700 1 --2- 172.21.15.47:0/2190204713 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd2ec113020 0x7fd2ec1a3770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:22:39.738 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:39.734+0000 7fd2e9400700 1 -- 172.21.15.47:0/2190204713 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2ec1a3eb0 con 0x7fd2ec076fe0 2024-03-21T10:22:39.739 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:39.735+0000 7fd2e9400700 1 --2- 172.21.15.47:0/2190204713 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd2ec076fe0 0x7fd2ec19f3e0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fd2d800e9a0 tx=0x7fd2d800ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:22:39.739 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:39.735+0000 7fd2e2a00700 1 -- 172.21.15.47:0/2190204713 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2d800cbd0 con 0x7fd2ec076fe0 2024-03-21T10:22:39.739 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:39.735+0000 7fd2f1000700 1 -- 172.21.15.47:0/2190204713 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd2ec1a41a0 con 0x7fd2ec076fe0 2024-03-21T10:22:39.739 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:39.735+0000 7fd2e2a00700 1 -- 172.21.15.47:0/2190204713 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fd2d800cd70 con 0x7fd2ec076fe0 2024-03-21T10:22:39.739 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:39.735+0000 7fd2e2a00700 1 -- 172.21.15.47:0/2190204713 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2d8018680 con 0x7fd2ec076fe0 2024-03-21T10:22:39.739 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:39.735+0000 7fd2f1000700 1 -- 172.21.15.47:0/2190204713 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd2ec1aba20 con 0x7fd2ec076fe0 2024-03-21T10:22:39.739 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:39.736+0000 7fd2f1000700 1 -- 172.21.15.47:0/2190204713 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd2ec068e50 con 0x7fd2ec076fe0 2024-03-21T10:22:39.741 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:39.739+0000 7fd2e2a00700 1 -- 172.21.15.47:0/2190204713 <== mon.2 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 17) v1 ==== 137905+0+0 (secure 0 0 0) 0x7fd2d8010430 con 0x7fd2ec076fe0 2024-03-21T10:22:39.742 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:39.740+0000 7fd2e2a00700 1 --2- 172.21.15.47:0/2190204713 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7fd2d00a3040 0x7fd2d00a5500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:22:39.743 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:39.741+0000 7fd2e2a00700 1 -- 172.21.15.47:0/2190204713 <== mon.2 v2:172.21.15.100:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9092+0+0 (secure 0 0 0) 0x7fd2d80cffe0 con 0x7fd2ec076fe0 2024-03-21T10:22:39.743 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:39.741+0000 7fd2e9e00700 1 --2- 172.21.15.47:0/2190204713 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7fd2d00a3040 0x7fd2d00a5500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:22:39.743 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:39.742+0000 7fd2e9e00700 1 --2- 172.21.15.47:0/2190204713 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7fd2d00a3040 0x7fd2d00a5500 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fd2dc006010 tx=0x7fd2dc01a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:22:39.754 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:39.752+0000 7fd2e2a00700 1 -- 172.21.15.47:0/2190204713 <== mon.2 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fd2d809c950 con 0x7fd2ec076fe0 2024-03-21T10:22:39.964 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:22:39 smithi047 ceph-mon[31547]: pgmap v389: 129 pgs: 129 active+clean; 624 KiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 9.5 KiB/s rd, 9.0 KiB/s wr, 26 op/s; 412 B/s, 0 objects/s recovering 2024-03-21T10:22:40.042 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:22:39 smithi157 ceph-mon[35581]: pgmap v389: 129 pgs: 129 active+clean; 624 KiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 9.5 KiB/s rd, 9.0 KiB/s wr, 26 op/s; 412 B/s, 0 objects/s recovering 2024-03-21T10:22:40.056 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:40.054+0000 7fd2f1000700 1 -- 172.21.15.47:0/2190204713 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) v1 -- 0x7fd2ec02da30 con 0x7fd2ec076fe0 2024-03-21T10:22:40.075 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:40.073+0000 7fd2e2a00700 1 -- 172.21.15.47:0/2190204713 <== mon.2 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v17) v1 ==== 74+0+263550 (secure 0 0 0) 0x7fd2d809c250 con 0x7fd2ec076fe0 2024-03-21T10:22:40.077 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-21T10:22:40.082 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:40.079+0000 7fd2f1000700 1 -- 172.21.15.47:0/2190204713 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7fd2d00a3040 msgr2=0x7fd2d00a5500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:22:40.083 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:40.079+0000 7fd2f1000700 1 --2- 172.21.15.47:0/2190204713 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7fd2d00a3040 0x7fd2d00a5500 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fd2dc006010 tx=0x7fd2dc01a040 comp rx=0 tx=0).stop 2024-03-21T10:22:40.083 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:40.079+0000 7fd2f1000700 1 -- 172.21.15.47:0/2190204713 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd2ec076fe0 msgr2=0x7fd2ec19f3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:22:40.083 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:40.079+0000 7fd2f1000700 1 --2- 172.21.15.47:0/2190204713 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd2ec076fe0 0x7fd2ec19f3e0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fd2d800e9a0 tx=0x7fd2d800ecb0 comp rx=0 tx=0).stop 2024-03-21T10:22:40.083 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:40.079+0000 7fd2f1000700 1 -- 172.21.15.47:0/2190204713 shutdown_connections 2024-03-21T10:22:40.083 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:40.079+0000 7fd2f1000700 1 --2- 172.21.15.47:0/2190204713 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7fd2d00a3040 0x7fd2d00a5500 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:22:40.083 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:40.079+0000 7fd2f1000700 1 --2- 172.21.15.47:0/2190204713 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd2ec113020 0x7fd2ec1a3770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:22:40.083 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:40.079+0000 7fd2f1000700 1 --2- 172.21.15.47:0/2190204713 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd2ec0766c0 0x7fd2ec19eea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:22:40.083 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:40.079+0000 7fd2f1000700 1 --2- 172.21.15.47:0/2190204713 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd2ec076fe0 0x7fd2ec19f3e0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:22:40.083 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:40.079+0000 7fd2f1000700 1 -- 172.21.15.47:0/2190204713 >> 172.21.15.47:0/2190204713 conn(0x7fd2ec0ffe90 msgr2=0x7fd2ec102280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:22:40.083 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:40.080+0000 7fd2f1000700 1 -- 172.21.15.47:0/2190204713 shutdown_connections 2024-03-21T10:22:40.083 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:40.080+0000 7fd2f1000700 1 -- 172.21.15.47:0/2190204713 wait complete. 2024-03-21T10:22:40.229 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:22:39 smithi100 ceph-mon[35193]: pgmap v389: 129 pgs: 129 active+clean; 624 KiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 9.5 KiB/s rd, 9.0 KiB/s wr, 26 op/s; 412 B/s, 0 objects/s recovering 2024-03-21T10:22:41.031 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:22:40 smithi047 ceph-mon[31547]: from='client.? 172.21.15.47:0/2190204713' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-03-21T10:22:41.036 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":17,"flags":0,"active_gid":14152,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6800","nonce":2823885994},{"type":"v1","addr":"172.21.15.47:6801","nonce":2823885994}]},"active_addr":"172.21.15.47:6801/2823885994","active_change":"2024-03-21T10:12:25.844772+0000","active_mgr_features":4540701547738038271,"available":true,"standbys":[{"gid":24112,"name":"z","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.0.0","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]},{"gid":34103,"name":"y","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.0.0","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["cephadm","dashboard","iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.0.0","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{"dashboard":"https://172.21.15.47:8443/"},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"squid":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":3,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.47:0","nonce":3101796817}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.47:0","nonce":1607664848}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.47:0","nonce":2983192238}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.47:0","nonce":936039355}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.47:0","nonce":204593275}]}]} 2024-03-21T10:22:41.042 INFO:tasks.cephadm.ceph_manager.ceph:mgr available! 2024-03-21T10:22:41.042 INFO:tasks.cephadm.ceph_manager.ceph:waiting for all up 2024-03-21T10:22:41.042 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2edfee68ad7243bed422a049b6f5f58619300f11 shell --fsid 3056d51a-e76b-11ee-95cd-87774f69a715 -- ceph osd dump --format=json 2024-03-21T10:22:41.229 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:22:40 smithi100 ceph-mon[35193]: from='client.? 172.21.15.47:0/2190204713' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-03-21T10:22:41.239 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:22:40 smithi157 ceph-mon[35581]: from='client.? 172.21.15.47:0/2190204713' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-03-21T10:22:41.541 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/3056d51a-e76b-11ee-95cd-87774f69a715/mon.a/config 2024-03-21T10:22:41.964 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:22:41 smithi047 ceph-mon[31547]: pgmap v390: 129 pgs: 129 active+clean; 630 KiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 7.7 KiB/s wr, 23 op/s; 319 B/s, 0 objects/s recovering 2024-03-21T10:22:42.229 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:22:41 smithi100 ceph-mon[35193]: pgmap v390: 129 pgs: 129 active+clean; 630 KiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 7.7 KiB/s wr, 23 op/s; 319 B/s, 0 objects/s recovering 2024-03-21T10:22:42.240 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:22:41 smithi157 ceph-mon[35581]: pgmap v390: 129 pgs: 129 active+clean; 630 KiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 7.7 KiB/s wr, 23 op/s; 319 B/s, 0 objects/s recovering 2024-03-21T10:22:43.929 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:43.926+0000 7f86b0200700 1 -- 172.21.15.47:0/2149247916 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f86a8113040 msgr2=0x7f86a8115430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:22:43.929 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:43.926+0000 7f86b0200700 1 --2- 172.21.15.47:0/2149247916 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f86a8113040 0x7f86a8115430 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f86a000b0a0 tx=0x7f86a000b3b0 comp rx=0 tx=0).stop 2024-03-21T10:22:43.929 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:43.926+0000 7f86b0200700 1 -- 172.21.15.47:0/2149247916 shutdown_connections 2024-03-21T10:22:43.929 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:43.926+0000 7f86b0200700 1 --2- 172.21.15.47:0/2149247916 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f86a8113040 0x7f86a8115430 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:22:43.929 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:43.926+0000 7f86b0200700 1 --2- 172.21.15.47:0/2149247916 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f86a80766c0 0x7f86a8076aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:22:43.929 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:43.926+0000 7f86b0200700 1 --2- 172.21.15.47:0/2149247916 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f86a8076fe0 0x7f86a8112b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:22:43.929 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:43.926+0000 7f86b0200700 1 -- 172.21.15.47:0/2149247916 >> 172.21.15.47:0/2149247916 conn(0x7f86a80ffe70 msgr2=0x7f86a8102290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:22:43.929 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:43.926+0000 7f86b0200700 1 -- 172.21.15.47:0/2149247916 shutdown_connections 2024-03-21T10:22:43.930 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:43.927+0000 7f86b0200700 1 -- 172.21.15.47:0/2149247916 wait complete. 2024-03-21T10:22:43.930 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:43.928+0000 7f86b0200700 1 Processor -- start 2024-03-21T10:22:43.930 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:43.928+0000 7f86b0200700 1 -- start start 2024-03-21T10:22:43.931 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:43.929+0000 7f86b0200700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f86a80766c0 0x7f86a819ee70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:22:43.931 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:43.929+0000 7f86b0200700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f86a8076fe0 0x7f86a819f3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:22:43.931 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:43.929+0000 7f86b0200700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f86a8113040 0x7f86a81a3740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:22:43.931 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:43.929+0000 7f86b0200700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86a81189e0 con 0x7f86a8113040 2024-03-21T10:22:43.931 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:43.929+0000 7f86b0200700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86a8118860 con 0x7f86a8076fe0 2024-03-21T10:22:43.931 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:43.929+0000 7f86b0200700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86a8118b60 con 0x7f86a80766c0 2024-03-21T10:22:43.932 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:43.929+0000 7f86ae200700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f86a8113040 0x7f86a81a3740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:22:43.932 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:43.929+0000 7f86ace00700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f86a8076fe0 0x7f86a819f3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:22:43.932 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:43.929+0000 7f86ae200700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f86a8113040 0x7f86a81a3740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:32792/0 (socket says 172.21.15.47:32792) 2024-03-21T10:22:43.933 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:43.929+0000 7f86ae200700 1 -- 172.21.15.47:0/3821163145 learned_addr learned my addr 172.21.15.47:0/3821163145 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:22:43.933 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:43.930+0000 7f86ad800700 1 --2- 172.21.15.47:0/3821163145 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f86a80766c0 0x7f86a819ee70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:22:43.933 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:43.930+0000 7f86ace00700 1 -- 172.21.15.47:0/3821163145 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f86a80766c0 msgr2=0x7f86a819ee70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:22:43.933 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:43.930+0000 7f86ace00700 1 --2- 172.21.15.47:0/3821163145 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f86a80766c0 0x7f86a819ee70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:22:43.933 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:43.930+0000 7f86ace00700 1 -- 172.21.15.47:0/3821163145 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f86a8113040 msgr2=0x7f86a81a3740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:22:43.933 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:43.930+0000 7f86ace00700 1 --2- 172.21.15.47:0/3821163145 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f86a8113040 0x7f86a81a3740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:22:43.933 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:43.930+0000 7f86ace00700 1 -- 172.21.15.47:0/3821163145 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86a81a3e80 con 0x7f86a8076fe0 2024-03-21T10:22:43.933 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:43.930+0000 7f86ae200700 1 --2- 172.21.15.47:0/3821163145 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f86a8113040 0x7f86a81a3740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T10:22:43.933 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:43.930+0000 7f86ace00700 1 --2- 172.21.15.47:0/3821163145 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f86a8076fe0 0x7f86a819f3b0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f869400e9f0 tx=0x7f869400ed00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:22:43.934 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:43.931+0000 7f86a6000700 1 -- 172.21.15.47:0/3821163145 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f869400cbd0 con 0x7f86a8076fe0 2024-03-21T10:22:43.934 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:43.931+0000 7f86b0200700 1 -- 172.21.15.47:0/3821163145 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f86a81a4110 con 0x7f86a8076fe0 2024-03-21T10:22:43.934 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:43.931+0000 7f86a6000700 1 -- 172.21.15.47:0/3821163145 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f869400ee90 con 0x7f86a8076fe0 2024-03-21T10:22:43.934 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:43.931+0000 7f86a6000700 1 -- 172.21.15.47:0/3821163145 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8694018680 con 0x7f86a8076fe0 2024-03-21T10:22:43.934 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:43.931+0000 7f86b0200700 1 -- 172.21.15.47:0/3821163145 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f86a81ab9f0 con 0x7f86a8076fe0 2024-03-21T10:22:43.934 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:43.932+0000 7f86b0200700 1 -- 172.21.15.47:0/3821163145 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f86a8068e50 con 0x7f86a8076fe0 2024-03-21T10:22:43.937 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:43.935+0000 7f86a6000700 1 -- 172.21.15.47:0/3821163145 <== mon.2 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 17) v1 ==== 137905+0+0 (secure 0 0 0) 0x7f8694010430 con 0x7f86a8076fe0 2024-03-21T10:22:43.938 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:43.936+0000 7f86a6000700 1 --2- 172.21.15.47:0/3821163145 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f86900a3040 0x7f86900a5500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:22:43.938 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:43.937+0000 7f86a6000700 1 -- 172.21.15.47:0/3821163145 <== mon.2 v2:172.21.15.100:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9092+0+0 (secure 0 0 0) 0x7f86940cfd60 con 0x7f86a8076fe0 2024-03-21T10:22:43.939 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:43.937+0000 7f86ad800700 1 --2- 172.21.15.47:0/3821163145 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f86900a3040 0x7f86900a5500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:22:43.940 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:43.938+0000 7f86ad800700 1 --2- 172.21.15.47:0/3821163145 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f86900a3040 0x7f86900a5500 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f869c004630 tx=0x7f869c00b3c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:22:43.949 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:43.947+0000 7f86a6000700 1 -- 172.21.15.47:0/3821163145 <== mon.2 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f869409c780 con 0x7f86a8076fe0 2024-03-21T10:22:44.214 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:22:43 smithi047 ceph-mon[31547]: pgmap v391: 129 pgs: 129 active+clean; 630 KiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 5.0 KiB/s wr, 16 op/s 2024-03-21T10:22:44.215 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:44.213+0000 7f86b0200700 1 -- 172.21.15.47:0/3821163145 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f86a802da30 con 0x7f86a8076fe0 2024-03-21T10:22:44.218 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:44.216+0000 7f86a6000700 1 -- 172.21.15.47:0/3821163145 <== mon.2 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v93) v1 ==== 74+0+24644 (secure 0 0 0) 0x7f869400cd70 con 0x7f86a8076fe0 2024-03-21T10:22:44.219 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-21T10:22:44.219 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":93,"fsid":"3056d51a-e76b-11ee-95cd-87774f69a715","created":"2024-03-21T10:11:17.019965+0000","modified":"2024-03-21T10:22:27.181362+0000","last_up_change":"2024-03-21T10:20:52.675328+0000","last_in_change":"2024-03-21T10:20:27.648212+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-21T10:15:40.553114+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-21T10:21:56.399518+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-21T10:21:57.812940+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-21T10:22:01.822568+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"91","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"b0c7a102-8d89-428b-b793-cbbe67151f50","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6802","nonce":775837313},{"type":"v1","addr":"172.21.15.47:6803","nonce":775837313}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6804","nonce":775837313},{"type":"v1","addr":"172.21.15.47:6805","nonce":775837313}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6808","nonce":775837313},{"type":"v1","addr":"172.21.15.47:6809","nonce":775837313}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6806","nonce":775837313},{"type":"v1","addr":"172.21.15.47:6807","nonce":775837313}]},"public_addr":"172.21.15.47:6803/775837313","cluster_addr":"172.21.15.47:6805/775837313","heartbeat_back_addr":"172.21.15.47:6809/775837313","heartbeat_front_addr":"172.21.15.47:6807/775837313","state":["exists","up"]},{"osd":1,"uuid":"07bb81bb-7cf3-4bbc-ac9e-802f95185d11","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6800","nonce":2923304710},{"type":"v1","addr":"172.21.15.100:6801","nonce":2923304710}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":2923304710},{"type":"v1","addr":"172.21.15.100:6803","nonce":2923304710}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":2923304710},{"type":"v1","addr":"172.21.15.100:6807","nonce":2923304710}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6804","nonce":2923304710},{"type":"v1","addr":"172.21.15.100:6805","nonce":2923304710}]},"public_addr":"172.21.15.100:6801/2923304710","cluster_addr":"172.21.15.100:6803/2923304710","heartbeat_back_addr":"172.21.15.100:6807/2923304710","heartbeat_front_addr":"172.21.15.100:6805/2923304710","state":["exists","up"]},{"osd":2,"uuid":"88ce76b9-6081-4836-aad0-8c249c56e8a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":171377005},{"type":"v1","addr":"172.21.15.157:6801","nonce":171377005}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":171377005},{"type":"v1","addr":"172.21.15.157:6803","nonce":171377005}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":171377005},{"type":"v1","addr":"172.21.15.157:6807","nonce":171377005}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":171377005},{"type":"v1","addr":"172.21.15.157:6805","nonce":171377005}]},"public_addr":"172.21.15.157:6801/171377005","cluster_addr":"172.21.15.157:6803/171377005","heartbeat_back_addr":"172.21.15.157:6807/171377005","heartbeat_front_addr":"172.21.15.157:6805/171377005","state":["exists","up"]},{"osd":3,"uuid":"518aa33d-263f-45e6-aab5-2201ceb40e74","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":27,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6810","nonce":640760101},{"type":"v1","addr":"172.21.15.47:6811","nonce":640760101}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6812","nonce":640760101},{"type":"v1","addr":"172.21.15.47:6813","nonce":640760101}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6816","nonce":640760101},{"type":"v1","addr":"172.21.15.47:6817","nonce":640760101}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6814","nonce":640760101},{"type":"v1","addr":"172.21.15.47:6815","nonce":640760101}]},"public_addr":"172.21.15.47:6811/640760101","cluster_addr":"172.21.15.47:6813/640760101","heartbeat_back_addr":"172.21.15.47:6817/640760101","heartbeat_front_addr":"172.21.15.47:6815/640760101","state":["exists","up"]},{"osd":4,"uuid":"987bde83-ecc6-42e5-bd56-c143b3de4226","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":35,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6808","nonce":2785478639},{"type":"v1","addr":"172.21.15.100:6809","nonce":2785478639}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6810","nonce":2785478639},{"type":"v1","addr":"172.21.15.100:6811","nonce":2785478639}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6814","nonce":2785478639},{"type":"v1","addr":"172.21.15.100:6815","nonce":2785478639}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":2785478639},{"type":"v1","addr":"172.21.15.100:6813","nonce":2785478639}]},"public_addr":"172.21.15.100:6809/2785478639","cluster_addr":"172.21.15.100:6811/2785478639","heartbeat_back_addr":"172.21.15.100:6815/2785478639","heartbeat_front_addr":"172.21.15.100:6813/2785478639","state":["exists","up"]},{"osd":5,"uuid":"012d2188-3a61-4a5f-8d01-2001c15ffbe2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":40,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":1525645410},{"type":"v1","addr":"172.21.15.157:6809","nonce":1525645410}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":1525645410},{"type":"v1","addr":"172.21.15.157:6811","nonce":1525645410}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":1525645410},{"type":"v1","addr":"172.21.15.157:6815","nonce":1525645410}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":1525645410},{"type":"v1","addr":"172.21.15.157:6813","nonce":1525645410}]},"public_addr":"172.21.15.157:6809/1525645410","cluster_addr":"172.21.15.157:6811/1525645410","heartbeat_back_addr":"172.21.15.157:6815/1525645410","heartbeat_front_addr":"172.21.15.157:6813/1525645410","state":["exists","up"]},{"osd":6,"uuid":"c12c14b4-85f0-4d88-937c-de4f556437de","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6818","nonce":2321642790},{"type":"v1","addr":"172.21.15.47:6819","nonce":2321642790}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6820","nonce":2321642790},{"type":"v1","addr":"172.21.15.47:6821","nonce":2321642790}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6824","nonce":2321642790},{"type":"v1","addr":"172.21.15.47:6825","nonce":2321642790}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6822","nonce":2321642790},{"type":"v1","addr":"172.21.15.47:6823","nonce":2321642790}]},"public_addr":"172.21.15.47:6819/2321642790","cluster_addr":"172.21.15.47:6821/2321642790","heartbeat_back_addr":"172.21.15.47:6825/2321642790","heartbeat_front_addr":"172.21.15.47:6823/2321642790","state":["exists","up"]},{"osd":7,"uuid":"3c87c6d0-1d1f-4294-9917-aaaad7dda824","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6816","nonce":4104727330},{"type":"v1","addr":"172.21.15.100:6817","nonce":4104727330}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":4104727330},{"type":"v1","addr":"172.21.15.100:6819","nonce":4104727330}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":4104727330},{"type":"v1","addr":"172.21.15.100:6823","nonce":4104727330}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6820","nonce":4104727330},{"type":"v1","addr":"172.21.15.100:6821","nonce":4104727330}]},"public_addr":"172.21.15.100:6817/4104727330","cluster_addr":"172.21.15.100:6819/4104727330","heartbeat_back_addr":"172.21.15.100:6823/4104727330","heartbeat_front_addr":"172.21.15.100:6821/4104727330","state":["exists","up"]},{"osd":8,"uuid":"502d65cf-53c8-4936-8de4-e11b2a6e1e18","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":61,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":4067715347},{"type":"v1","addr":"172.21.15.157:6817","nonce":4067715347}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":4067715347},{"type":"v1","addr":"172.21.15.157:6819","nonce":4067715347}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":4067715347},{"type":"v1","addr":"172.21.15.157:6823","nonce":4067715347}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":4067715347},{"type":"v1","addr":"172.21.15.157:6821","nonce":4067715347}]},"public_addr":"172.21.15.157:6817/4067715347","cluster_addr":"172.21.15.157:6819/4067715347","heartbeat_back_addr":"172.21.15.157:6823/4067715347","heartbeat_front_addr":"172.21.15.157:6821/4067715347","state":["exists","up"]},{"osd":9,"uuid":"5ea1f5be-8458-4c93-9e61-c2e867a514f3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6826","nonce":1982108738},{"type":"v1","addr":"172.21.15.47:6827","nonce":1982108738}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6828","nonce":1982108738},{"type":"v1","addr":"172.21.15.47:6829","nonce":1982108738}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6832","nonce":1982108738},{"type":"v1","addr":"172.21.15.47:6833","nonce":1982108738}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6830","nonce":1982108738},{"type":"v1","addr":"172.21.15.47:6831","nonce":1982108738}]},"public_addr":"172.21.15.47:6827/1982108738","cluster_addr":"172.21.15.47:6829/1982108738","heartbeat_back_addr":"172.21.15.47:6833/1982108738","heartbeat_front_addr":"172.21.15.47:6831/1982108738","state":["exists","up"]},{"osd":10,"uuid":"2da3da90-a0be-4a54-a9ff-b2d6c9b36870","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":74,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":396391028},{"type":"v1","addr":"172.21.15.100:6825","nonce":396391028}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":396391028},{"type":"v1","addr":"172.21.15.100:6827","nonce":396391028}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6830","nonce":396391028},{"type":"v1","addr":"172.21.15.100:6831","nonce":396391028}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":396391028},{"type":"v1","addr":"172.21.15.100:6829","nonce":396391028}]},"public_addr":"172.21.15.100:6825/396391028","cluster_addr":"172.21.15.100:6827/396391028","heartbeat_back_addr":"172.21.15.100:6831/396391028","heartbeat_front_addr":"172.21.15.100:6829/396391028","state":["exists","up"]},{"osd":11,"uuid":"11798d41-7f7e-4319-95b8-8d3d0946bde9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":82,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":2671029483},{"type":"v1","addr":"172.21.15.157:6825","nonce":2671029483}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":2671029483},{"type":"v1","addr":"172.21.15.157:6827","nonce":2671029483}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":2671029483},{"type":"v1","addr":"172.21.15.157:6831","nonce":2671029483}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":2671029483},{"type":"v1","addr":"172.21.15.157:6829","nonce":2671029483}]},"public_addr":"172.21.15.157:6825/2671029483","cluster_addr":"172.21.15.157:6827/2671029483","heartbeat_back_addr":"172.21.15.157:6831/2671029483","heartbeat_front_addr":"172.21.15.157:6829/2671029483","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T10:14:31.822399+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T10:15:04.175721+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T10:15:35.902595+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T10:16:12.088891+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T10:16:46.428704+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T10:17:20.158270+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T10:17:55.915253+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T10:18:30.064641+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T10:19:04.410004+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T10:19:40.605181+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T10:20:15.636605+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T10:20:50.151365+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.11","mappings":[{"from":7,"to":11}]},{"pgid":"2.14","mappings":[{"from":2,"to":4}]},{"pgid":"2.1a","mappings":[{"from":8,"to":1}]},{"pgid":"2.1b","mappings":[{"from":2,"to":4}]},{"pgid":"2.1d","mappings":[{"from":8,"to":11}]},{"pgid":"3.0","mappings":[{"from":4,"to":8}]},{"pgid":"3.6","mappings":[{"from":3,"to":7}]},{"pgid":"3.1b","mappings":[{"from":4,"to":6}]},{"pgid":"3.33","mappings":[{"from":5,"to":9}]},{"pgid":"3.3c","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.47:0/356822431":"2024-03-22T10:12:25.844530+0000","172.21.15.47:0/1167490879":"2024-03-22T10:12:25.844530+0000","172.21.15.47:0/1540718138":"2024-03-22T10:12:25.844530+0000","172.21.15.47:6801/3836903972":"2024-03-22T10:12:25.844530+0000","172.21.15.47:0/235436602":"2024-03-22T10:11:50.301838+0000","172.21.15.47:0/2052815572":"2024-03-22T10:11:50.301838+0000","172.21.15.47:6800/3942658432":"2024-03-22T10:11:50.301838+0000","172.21.15.47:6801/3942658432":"2024-03-22T10:11:50.301838+0000","172.21.15.47:0/1419516257":"2024-03-22T10:11:50.301838+0000","172.21.15.47:6800/3836903972":"2024-03-22T10:12:25.844530+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-21T10:22:44.221 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:44.219+0000 7f86b0200700 1 -- 172.21.15.47:0/3821163145 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f86900a3040 msgr2=0x7f86900a5500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:22:44.222 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:44.219+0000 7f86b0200700 1 --2- 172.21.15.47:0/3821163145 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f86900a3040 0x7f86900a5500 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f869c004630 tx=0x7f869c00b3c0 comp rx=0 tx=0).stop 2024-03-21T10:22:44.222 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:44.219+0000 7f86b0200700 1 -- 172.21.15.47:0/3821163145 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f86a8076fe0 msgr2=0x7f86a819f3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:22:44.222 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:44.219+0000 7f86b0200700 1 --2- 172.21.15.47:0/3821163145 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f86a8076fe0 0x7f86a819f3b0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f869400e9f0 tx=0x7f869400ed00 comp rx=0 tx=0).stop 2024-03-21T10:22:44.222 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:44.219+0000 7f86b0200700 1 -- 172.21.15.47:0/3821163145 shutdown_connections 2024-03-21T10:22:44.222 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:44.219+0000 7f86b0200700 1 --2- 172.21.15.47:0/3821163145 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f86900a3040 0x7f86900a5500 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:22:44.222 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:44.219+0000 7f86b0200700 1 --2- 172.21.15.47:0/3821163145 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f86a8113040 0x7f86a81a3740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:22:44.222 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:44.219+0000 7f86b0200700 1 --2- 172.21.15.47:0/3821163145 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f86a80766c0 0x7f86a819ee70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:22:44.222 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:44.219+0000 7f86b0200700 1 --2- 172.21.15.47:0/3821163145 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f86a8076fe0 0x7f86a819f3b0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:22:44.222 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:44.219+0000 7f86b0200700 1 -- 172.21.15.47:0/3821163145 >> 172.21.15.47:0/3821163145 conn(0x7f86a80ffe70 msgr2=0x7f86a8102260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:22:44.222 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:44.219+0000 7f86b0200700 1 -- 172.21.15.47:0/3821163145 shutdown_connections 2024-03-21T10:22:44.222 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:44.220+0000 7f86b0200700 1 -- 172.21.15.47:0/3821163145 wait complete. 2024-03-21T10:22:44.229 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:22:43 smithi100 ceph-mon[35193]: pgmap v391: 129 pgs: 129 active+clean; 630 KiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 5.0 KiB/s wr, 16 op/s 2024-03-21T10:22:44.239 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:22:43 smithi157 ceph-mon[35581]: pgmap v391: 129 pgs: 129 active+clean; 630 KiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 5.0 KiB/s wr, 16 op/s 2024-03-21T10:22:45.214 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:22:44 smithi047 ceph-mon[31547]: from='client.? 172.21.15.47:0/3821163145' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-21T10:22:45.229 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:22:44 smithi100 ceph-mon[35193]: from='client.? 172.21.15.47:0/3821163145' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-21T10:22:45.239 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:22:44 smithi157 ceph-mon[35581]: from='client.? 172.21.15.47:0/3821163145' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-21T10:22:45.316 INFO:tasks.cephadm.ceph_manager.ceph:all up! 2024-03-21T10:22:45.316 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2edfee68ad7243bed422a049b6f5f58619300f11 shell --fsid 3056d51a-e76b-11ee-95cd-87774f69a715 -- ceph osd dump --format=json 2024-03-21T10:22:45.775 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/3056d51a-e76b-11ee-95cd-87774f69a715/mon.a/config 2024-03-21T10:22:46.064 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:22:45 smithi047 ceph-mon[31547]: pgmap v392: 129 pgs: 129 active+clean; 642 KiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 6.2 KiB/s wr, 20 op/s 2024-03-21T10:22:46.229 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:22:45 smithi100 ceph-mon[35193]: pgmap v392: 129 pgs: 129 active+clean; 642 KiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 6.2 KiB/s wr, 20 op/s 2024-03-21T10:22:46.239 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:22:45 smithi157 ceph-mon[35581]: pgmap v392: 129 pgs: 129 active+clean; 642 KiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 6.2 KiB/s wr, 20 op/s 2024-03-21T10:22:47.964 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:22:47 smithi047 ceph-mon[31547]: pgmap v393: 129 pgs: 129 active+clean; 642 KiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 2.0 KiB/s wr, 10 op/s 2024-03-21T10:22:48.229 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:22:47 smithi100 ceph-mon[35193]: pgmap v393: 129 pgs: 129 active+clean; 642 KiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 2.0 KiB/s wr, 10 op/s 2024-03-21T10:22:48.239 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:22:47 smithi157 ceph-mon[35581]: pgmap v393: 129 pgs: 129 active+clean; 642 KiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 2.0 KiB/s wr, 10 op/s 2024-03-21T10:22:48.276 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:48.275+0000 7f594ae00700 1 -- 172.21.15.47:0/2430325256 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f59441081c0 msgr2=0x7f59441085a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:22:48.277 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:48.275+0000 7f594ae00700 1 --2- 172.21.15.47:0/2430325256 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f59441081c0 0x7f59441085a0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f59340099c0 tx=0x7f5934009cd0 comp rx=0 tx=0).stop 2024-03-21T10:22:48.277 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:48.275+0000 7f594ae00700 1 -- 172.21.15.47:0/2430325256 shutdown_connections 2024-03-21T10:22:48.277 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:48.275+0000 7f594ae00700 1 --2- 172.21.15.47:0/2430325256 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5944102ba0 0x7f594410efe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:22:48.277 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:48.275+0000 7f594ae00700 1 --2- 172.21.15.47:0/2430325256 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f59441081c0 0x7f59441085a0 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:22:48.277 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:48.275+0000 7f594ae00700 1 --2- 172.21.15.47:0/2430325256 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f59441021e0 0x7f5944102660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:22:48.277 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:48.275+0000 7f594ae00700 1 -- 172.21.15.47:0/2430325256 >> 172.21.15.47:0/2430325256 conn(0x7f59440792c0 msgr2=0x7f59440796d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:22:48.277 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:48.275+0000 7f594ae00700 1 -- 172.21.15.47:0/2430325256 shutdown_connections 2024-03-21T10:22:48.277 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:48.275+0000 7f594ae00700 1 -- 172.21.15.47:0/2430325256 wait complete. 2024-03-21T10:22:48.278 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:48.276+0000 7f594ae00700 1 Processor -- start 2024-03-21T10:22:48.278 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:48.277+0000 7f594ae00700 1 -- start start 2024-03-21T10:22:48.278 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:48.277+0000 7f594ae00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f59441021e0 0x7f59441a0690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:22:48.278 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:48.277+0000 7f594ae00700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5944102ba0 0x7f59441a0bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:22:48.279 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:48.277+0000 7f594ae00700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f59441081c0 0x7f594419a810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:22:48.279 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:48.277+0000 7f594ae00700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5944112340 con 0x7f5944102ba0 2024-03-21T10:22:48.279 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:48.277+0000 7f594ae00700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59441121c0 con 0x7f59441081c0 2024-03-21T10:22:48.279 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:48.277+0000 7f594ae00700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59441124c0 con 0x7f59441021e0 2024-03-21T10:22:48.279 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:48.278+0000 7f5948e00700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f59441081c0 0x7f594419a810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:22:48.279 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:48.278+0000 7f5948e00700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f59441081c0 0x7f594419a810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:35384/0 (socket says 172.21.15.47:35384) 2024-03-21T10:22:48.279 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:48.278+0000 7f5948e00700 1 -- 172.21.15.47:0/1641808757 learned_addr learned my addr 172.21.15.47:0/1641808757 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:22:48.280 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:48.278+0000 7f5943e00700 1 --2- 172.21.15.47:0/1641808757 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f59441021e0 0x7f59441a0690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:22:48.280 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:48.278+0000 7f5948e00700 1 -- 172.21.15.47:0/1641808757 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f59441021e0 msgr2=0x7f59441a0690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:22:48.280 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:48.278+0000 7f5948e00700 1 --2- 172.21.15.47:0/1641808757 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f59441021e0 0x7f59441a0690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:22:48.280 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:48.278+0000 7f5948e00700 1 -- 172.21.15.47:0/1641808757 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5944102ba0 msgr2=0x7f59441a0bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:22:48.280 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:48.278+0000 7f5948e00700 1 --2- 172.21.15.47:0/1641808757 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5944102ba0 0x7f59441a0bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:22:48.280 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:48.278+0000 7f5948e00700 1 -- 172.21.15.47:0/1641808757 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f594419aff0 con 0x7f59441081c0 2024-03-21T10:22:48.280 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:48.279+0000 7f5948e00700 1 --2- 172.21.15.47:0/1641808757 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f59441081c0 0x7f594419a810 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f593c007ee0 tx=0x7f593c00d3b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:22:48.281 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:48.279+0000 7f5941600700 1 -- 172.21.15.47:0/1641808757 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f593c00dbd0 con 0x7f59441081c0 2024-03-21T10:22:48.281 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:48.279+0000 7f594ae00700 1 -- 172.21.15.47:0/1641808757 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f594419b2e0 con 0x7f59441081c0 2024-03-21T10:22:48.281 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:48.279+0000 7f5941600700 1 -- 172.21.15.47:0/1641808757 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f593c00dd70 con 0x7f59441081c0 2024-03-21T10:22:48.281 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:48.279+0000 7f5941600700 1 -- 172.21.15.47:0/1641808757 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f593c005a20 con 0x7f59441081c0 2024-03-21T10:22:48.281 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:48.279+0000 7f594ae00700 1 -- 172.21.15.47:0/1641808757 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f59441a74f0 con 0x7f59441081c0 2024-03-21T10:22:48.282 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:48.281+0000 7f594ae00700 1 -- 172.21.15.47:0/1641808757 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5944103870 con 0x7f59441081c0 2024-03-21T10:22:48.284 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:48.283+0000 7f5941600700 1 -- 172.21.15.47:0/1641808757 <== mon.2 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 17) v1 ==== 137905+0+0 (secure 0 0 0) 0x7f593c01a020 con 0x7f59441081c0 2024-03-21T10:22:48.285 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:48.284+0000 7f5941600700 1 --2- 172.21.15.47:0/1641808757 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f59280a3040 0x7f59280a5500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:22:48.285 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:48.284+0000 7f5941600700 1 -- 172.21.15.47:0/1641808757 <== mon.2 v2:172.21.15.100:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9092+0+0 (secure 0 0 0) 0x7f593c0ceec0 con 0x7f59441081c0 2024-03-21T10:22:48.286 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:48.284+0000 7f5943e00700 1 --2- 172.21.15.47:0/1641808757 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f59280a3040 0x7f59280a5500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:22:48.286 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:48.285+0000 7f5943e00700 1 --2- 172.21.15.47:0/1641808757 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f59280a3040 0x7f59280a5500 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f5934003760 tx=0x7f5934005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:22:48.300 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:48.299+0000 7f5941600700 1 -- 172.21.15.47:0/1641808757 <== mon.2 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f593c010070 con 0x7f59441081c0 2024-03-21T10:22:48.531 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:48.529+0000 7f594ae00700 1 -- 172.21.15.47:0/1641808757 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f59441a7720 con 0x7f59441081c0 2024-03-21T10:22:48.534 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:48.533+0000 7f5941600700 1 -- 172.21.15.47:0/1641808757 <== mon.2 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v93) v1 ==== 74+0+24644 (secure 0 0 0) 0x7f593c09b7d0 con 0x7f59441081c0 2024-03-21T10:22:48.535 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-21T10:22:48.535 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":93,"fsid":"3056d51a-e76b-11ee-95cd-87774f69a715","created":"2024-03-21T10:11:17.019965+0000","modified":"2024-03-21T10:22:27.181362+0000","last_up_change":"2024-03-21T10:20:52.675328+0000","last_in_change":"2024-03-21T10:20:27.648212+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-21T10:15:40.553114+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-21T10:21:56.399518+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-21T10:21:57.812940+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-21T10:22:01.822568+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"91","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"b0c7a102-8d89-428b-b793-cbbe67151f50","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6802","nonce":775837313},{"type":"v1","addr":"172.21.15.47:6803","nonce":775837313}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6804","nonce":775837313},{"type":"v1","addr":"172.21.15.47:6805","nonce":775837313}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6808","nonce":775837313},{"type":"v1","addr":"172.21.15.47:6809","nonce":775837313}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6806","nonce":775837313},{"type":"v1","addr":"172.21.15.47:6807","nonce":775837313}]},"public_addr":"172.21.15.47:6803/775837313","cluster_addr":"172.21.15.47:6805/775837313","heartbeat_back_addr":"172.21.15.47:6809/775837313","heartbeat_front_addr":"172.21.15.47:6807/775837313","state":["exists","up"]},{"osd":1,"uuid":"07bb81bb-7cf3-4bbc-ac9e-802f95185d11","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6800","nonce":2923304710},{"type":"v1","addr":"172.21.15.100:6801","nonce":2923304710}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":2923304710},{"type":"v1","addr":"172.21.15.100:6803","nonce":2923304710}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":2923304710},{"type":"v1","addr":"172.21.15.100:6807","nonce":2923304710}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6804","nonce":2923304710},{"type":"v1","addr":"172.21.15.100:6805","nonce":2923304710}]},"public_addr":"172.21.15.100:6801/2923304710","cluster_addr":"172.21.15.100:6803/2923304710","heartbeat_back_addr":"172.21.15.100:6807/2923304710","heartbeat_front_addr":"172.21.15.100:6805/2923304710","state":["exists","up"]},{"osd":2,"uuid":"88ce76b9-6081-4836-aad0-8c249c56e8a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":171377005},{"type":"v1","addr":"172.21.15.157:6801","nonce":171377005}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":171377005},{"type":"v1","addr":"172.21.15.157:6803","nonce":171377005}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":171377005},{"type":"v1","addr":"172.21.15.157:6807","nonce":171377005}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":171377005},{"type":"v1","addr":"172.21.15.157:6805","nonce":171377005}]},"public_addr":"172.21.15.157:6801/171377005","cluster_addr":"172.21.15.157:6803/171377005","heartbeat_back_addr":"172.21.15.157:6807/171377005","heartbeat_front_addr":"172.21.15.157:6805/171377005","state":["exists","up"]},{"osd":3,"uuid":"518aa33d-263f-45e6-aab5-2201ceb40e74","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":27,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6810","nonce":640760101},{"type":"v1","addr":"172.21.15.47:6811","nonce":640760101}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6812","nonce":640760101},{"type":"v1","addr":"172.21.15.47:6813","nonce":640760101}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6816","nonce":640760101},{"type":"v1","addr":"172.21.15.47:6817","nonce":640760101}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6814","nonce":640760101},{"type":"v1","addr":"172.21.15.47:6815","nonce":640760101}]},"public_addr":"172.21.15.47:6811/640760101","cluster_addr":"172.21.15.47:6813/640760101","heartbeat_back_addr":"172.21.15.47:6817/640760101","heartbeat_front_addr":"172.21.15.47:6815/640760101","state":["exists","up"]},{"osd":4,"uuid":"987bde83-ecc6-42e5-bd56-c143b3de4226","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":35,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6808","nonce":2785478639},{"type":"v1","addr":"172.21.15.100:6809","nonce":2785478639}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6810","nonce":2785478639},{"type":"v1","addr":"172.21.15.100:6811","nonce":2785478639}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6814","nonce":2785478639},{"type":"v1","addr":"172.21.15.100:6815","nonce":2785478639}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":2785478639},{"type":"v1","addr":"172.21.15.100:6813","nonce":2785478639}]},"public_addr":"172.21.15.100:6809/2785478639","cluster_addr":"172.21.15.100:6811/2785478639","heartbeat_back_addr":"172.21.15.100:6815/2785478639","heartbeat_front_addr":"172.21.15.100:6813/2785478639","state":["exists","up"]},{"osd":5,"uuid":"012d2188-3a61-4a5f-8d01-2001c15ffbe2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":40,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":1525645410},{"type":"v1","addr":"172.21.15.157:6809","nonce":1525645410}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":1525645410},{"type":"v1","addr":"172.21.15.157:6811","nonce":1525645410}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":1525645410},{"type":"v1","addr":"172.21.15.157:6815","nonce":1525645410}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":1525645410},{"type":"v1","addr":"172.21.15.157:6813","nonce":1525645410}]},"public_addr":"172.21.15.157:6809/1525645410","cluster_addr":"172.21.15.157:6811/1525645410","heartbeat_back_addr":"172.21.15.157:6815/1525645410","heartbeat_front_addr":"172.21.15.157:6813/1525645410","state":["exists","up"]},{"osd":6,"uuid":"c12c14b4-85f0-4d88-937c-de4f556437de","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6818","nonce":2321642790},{"type":"v1","addr":"172.21.15.47:6819","nonce":2321642790}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6820","nonce":2321642790},{"type":"v1","addr":"172.21.15.47:6821","nonce":2321642790}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6824","nonce":2321642790},{"type":"v1","addr":"172.21.15.47:6825","nonce":2321642790}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6822","nonce":2321642790},{"type":"v1","addr":"172.21.15.47:6823","nonce":2321642790}]},"public_addr":"172.21.15.47:6819/2321642790","cluster_addr":"172.21.15.47:6821/2321642790","heartbeat_back_addr":"172.21.15.47:6825/2321642790","heartbeat_front_addr":"172.21.15.47:6823/2321642790","state":["exists","up"]},{"osd":7,"uuid":"3c87c6d0-1d1f-4294-9917-aaaad7dda824","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6816","nonce":4104727330},{"type":"v1","addr":"172.21.15.100:6817","nonce":4104727330}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":4104727330},{"type":"v1","addr":"172.21.15.100:6819","nonce":4104727330}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":4104727330},{"type":"v1","addr":"172.21.15.100:6823","nonce":4104727330}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6820","nonce":4104727330},{"type":"v1","addr":"172.21.15.100:6821","nonce":4104727330}]},"public_addr":"172.21.15.100:6817/4104727330","cluster_addr":"172.21.15.100:6819/4104727330","heartbeat_back_addr":"172.21.15.100:6823/4104727330","heartbeat_front_addr":"172.21.15.100:6821/4104727330","state":["exists","up"]},{"osd":8,"uuid":"502d65cf-53c8-4936-8de4-e11b2a6e1e18","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":61,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":4067715347},{"type":"v1","addr":"172.21.15.157:6817","nonce":4067715347}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":4067715347},{"type":"v1","addr":"172.21.15.157:6819","nonce":4067715347}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":4067715347},{"type":"v1","addr":"172.21.15.157:6823","nonce":4067715347}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":4067715347},{"type":"v1","addr":"172.21.15.157:6821","nonce":4067715347}]},"public_addr":"172.21.15.157:6817/4067715347","cluster_addr":"172.21.15.157:6819/4067715347","heartbeat_back_addr":"172.21.15.157:6823/4067715347","heartbeat_front_addr":"172.21.15.157:6821/4067715347","state":["exists","up"]},{"osd":9,"uuid":"5ea1f5be-8458-4c93-9e61-c2e867a514f3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6826","nonce":1982108738},{"type":"v1","addr":"172.21.15.47:6827","nonce":1982108738}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6828","nonce":1982108738},{"type":"v1","addr":"172.21.15.47:6829","nonce":1982108738}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6832","nonce":1982108738},{"type":"v1","addr":"172.21.15.47:6833","nonce":1982108738}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6830","nonce":1982108738},{"type":"v1","addr":"172.21.15.47:6831","nonce":1982108738}]},"public_addr":"172.21.15.47:6827/1982108738","cluster_addr":"172.21.15.47:6829/1982108738","heartbeat_back_addr":"172.21.15.47:6833/1982108738","heartbeat_front_addr":"172.21.15.47:6831/1982108738","state":["exists","up"]},{"osd":10,"uuid":"2da3da90-a0be-4a54-a9ff-b2d6c9b36870","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":74,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":396391028},{"type":"v1","addr":"172.21.15.100:6825","nonce":396391028}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":396391028},{"type":"v1","addr":"172.21.15.100:6827","nonce":396391028}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6830","nonce":396391028},{"type":"v1","addr":"172.21.15.100:6831","nonce":396391028}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":396391028},{"type":"v1","addr":"172.21.15.100:6829","nonce":396391028}]},"public_addr":"172.21.15.100:6825/396391028","cluster_addr":"172.21.15.100:6827/396391028","heartbeat_back_addr":"172.21.15.100:6831/396391028","heartbeat_front_addr":"172.21.15.100:6829/396391028","state":["exists","up"]},{"osd":11,"uuid":"11798d41-7f7e-4319-95b8-8d3d0946bde9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":82,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":2671029483},{"type":"v1","addr":"172.21.15.157:6825","nonce":2671029483}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":2671029483},{"type":"v1","addr":"172.21.15.157:6827","nonce":2671029483}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":2671029483},{"type":"v1","addr":"172.21.15.157:6831","nonce":2671029483}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":2671029483},{"type":"v1","addr":"172.21.15.157:6829","nonce":2671029483}]},"public_addr":"172.21.15.157:6825/2671029483","cluster_addr":"172.21.15.157:6827/2671029483","heartbeat_back_addr":"172.21.15.157:6831/2671029483","heartbeat_front_addr":"172.21.15.157:6829/2671029483","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T10:14:31.822399+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T10:15:04.175721+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T10:15:35.902595+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T10:16:12.088891+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T10:16:46.428704+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T10:17:20.158270+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T10:17:55.915253+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T10:18:30.064641+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T10:19:04.410004+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T10:19:40.605181+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T10:20:15.636605+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T10:20:50.151365+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.11","mappings":[{"from":7,"to":11}]},{"pgid":"2.14","mappings":[{"from":2,"to":4}]},{"pgid":"2.1a","mappings":[{"from":8,"to":1}]},{"pgid":"2.1b","mappings":[{"from":2,"to":4}]},{"pgid":"2.1d","mappings":[{"from":8,"to":11}]},{"pgid":"3.0","mappings":[{"from":4,"to":8}]},{"pgid":"3.6","mappings":[{"from":3,"to":7}]},{"pgid":"3.1b","mappings":[{"from":4,"to":6}]},{"pgid":"3.33","mappings":[{"from":5,"to":9}]},{"pgid":"3.3c","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.47:0/356822431":"2024-03-22T10:12:25.844530+0000","172.21.15.47:0/1167490879":"2024-03-22T10:12:25.844530+0000","172.21.15.47:0/1540718138":"2024-03-22T10:12:25.844530+0000","172.21.15.47:6801/3836903972":"2024-03-22T10:12:25.844530+0000","172.21.15.47:0/235436602":"2024-03-22T10:11:50.301838+0000","172.21.15.47:0/2052815572":"2024-03-22T10:11:50.301838+0000","172.21.15.47:6800/3942658432":"2024-03-22T10:11:50.301838+0000","172.21.15.47:6801/3942658432":"2024-03-22T10:11:50.301838+0000","172.21.15.47:0/1419516257":"2024-03-22T10:11:50.301838+0000","172.21.15.47:6800/3836903972":"2024-03-22T10:12:25.844530+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-21T10:22:48.537 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:48.536+0000 7f594ae00700 1 -- 172.21.15.47:0/1641808757 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f59280a3040 msgr2=0x7f59280a5500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:22:48.537 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:48.536+0000 7f594ae00700 1 --2- 172.21.15.47:0/1641808757 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f59280a3040 0x7f59280a5500 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f5934003760 tx=0x7f5934005fb0 comp rx=0 tx=0).stop 2024-03-21T10:22:48.537 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:48.536+0000 7f594ae00700 1 -- 172.21.15.47:0/1641808757 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f59441081c0 msgr2=0x7f594419a810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:22:48.538 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:48.536+0000 7f594ae00700 1 --2- 172.21.15.47:0/1641808757 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f59441081c0 0x7f594419a810 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f593c007ee0 tx=0x7f593c00d3b0 comp rx=0 tx=0).stop 2024-03-21T10:22:48.538 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:48.536+0000 7f594ae00700 1 -- 172.21.15.47:0/1641808757 shutdown_connections 2024-03-21T10:22:48.538 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:48.536+0000 7f594ae00700 1 --2- 172.21.15.47:0/1641808757 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f59280a3040 0x7f59280a5500 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:22:48.538 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:48.536+0000 7f594ae00700 1 --2- 172.21.15.47:0/1641808757 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f59441081c0 0x7f594419a810 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:22:48.538 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:48.536+0000 7f594ae00700 1 --2- 172.21.15.47:0/1641808757 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f59441021e0 0x7f59441a0690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:22:48.538 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:48.536+0000 7f594ae00700 1 --2- 172.21.15.47:0/1641808757 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5944102ba0 0x7f59441a0bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:22:48.538 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:48.536+0000 7f594ae00700 1 -- 172.21.15.47:0/1641808757 >> 172.21.15.47:0/1641808757 conn(0x7f59440792c0 msgr2=0x7f5944074650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:22:48.538 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:48.536+0000 7f594ae00700 1 -- 172.21.15.47:0/1641808757 shutdown_connections 2024-03-21T10:22:48.538 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:48.536+0000 7f594ae00700 1 -- 172.21.15.47:0/1641808757 wait complete. 2024-03-21T10:22:49.214 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:22:48 smithi047 ceph-mon[31547]: from='client.? 172.21.15.47:0/1641808757' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-21T10:22:49.229 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:22:48 smithi100 ceph-mon[35193]: from='client.? 172.21.15.47:0/1641808757' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-21T10:22:49.239 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:22:48 smithi157 ceph-mon[35581]: from='client.? 172.21.15.47:0/1641808757' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-21T10:22:49.443 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2edfee68ad7243bed422a049b6f5f58619300f11 shell --fsid 3056d51a-e76b-11ee-95cd-87774f69a715 -- ceph tell osd.0 flush_pg_stats 2024-03-21T10:22:49.443 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2edfee68ad7243bed422a049b6f5f58619300f11 shell --fsid 3056d51a-e76b-11ee-95cd-87774f69a715 -- ceph tell osd.1 flush_pg_stats 2024-03-21T10:22:49.444 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2edfee68ad7243bed422a049b6f5f58619300f11 shell --fsid 3056d51a-e76b-11ee-95cd-87774f69a715 -- ceph tell osd.2 flush_pg_stats 2024-03-21T10:22:49.444 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2edfee68ad7243bed422a049b6f5f58619300f11 shell --fsid 3056d51a-e76b-11ee-95cd-87774f69a715 -- ceph tell osd.3 flush_pg_stats 2024-03-21T10:22:49.444 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2edfee68ad7243bed422a049b6f5f58619300f11 shell --fsid 3056d51a-e76b-11ee-95cd-87774f69a715 -- ceph tell osd.4 flush_pg_stats 2024-03-21T10:22:49.445 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2edfee68ad7243bed422a049b6f5f58619300f11 shell --fsid 3056d51a-e76b-11ee-95cd-87774f69a715 -- ceph tell osd.5 flush_pg_stats 2024-03-21T10:22:49.445 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2edfee68ad7243bed422a049b6f5f58619300f11 shell --fsid 3056d51a-e76b-11ee-95cd-87774f69a715 -- ceph tell osd.6 flush_pg_stats 2024-03-21T10:22:49.446 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2edfee68ad7243bed422a049b6f5f58619300f11 shell --fsid 3056d51a-e76b-11ee-95cd-87774f69a715 -- ceph tell osd.7 flush_pg_stats 2024-03-21T10:22:49.446 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2edfee68ad7243bed422a049b6f5f58619300f11 shell --fsid 3056d51a-e76b-11ee-95cd-87774f69a715 -- ceph tell osd.8 flush_pg_stats 2024-03-21T10:22:49.447 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2edfee68ad7243bed422a049b6f5f58619300f11 shell --fsid 3056d51a-e76b-11ee-95cd-87774f69a715 -- ceph tell osd.9 flush_pg_stats 2024-03-21T10:22:49.447 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2edfee68ad7243bed422a049b6f5f58619300f11 shell --fsid 3056d51a-e76b-11ee-95cd-87774f69a715 -- ceph tell osd.10 flush_pg_stats 2024-03-21T10:22:49.447 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2edfee68ad7243bed422a049b6f5f58619300f11 shell --fsid 3056d51a-e76b-11ee-95cd-87774f69a715 -- ceph tell osd.11 flush_pg_stats 2024-03-21T10:22:49.968 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:22:49 smithi047 ceph-mon[31547]: pgmap v394: 129 pgs: 129 active+clean; 642 KiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 1.9 KiB/s wr, 10 op/s 2024-03-21T10:22:50.229 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:22:49 smithi100 ceph-mon[35193]: pgmap v394: 129 pgs: 129 active+clean; 642 KiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 1.9 KiB/s wr, 10 op/s 2024-03-21T10:22:50.239 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:22:49 smithi157 ceph-mon[35581]: pgmap v394: 129 pgs: 129 active+clean; 642 KiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 1.9 KiB/s wr, 10 op/s 2024-03-21T10:22:50.580 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/3056d51a-e76b-11ee-95cd-87774f69a715/mon.a/config 2024-03-21T10:22:50.585 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/3056d51a-e76b-11ee-95cd-87774f69a715/mon.a/config 2024-03-21T10:22:50.643 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/3056d51a-e76b-11ee-95cd-87774f69a715/mon.a/config 2024-03-21T10:22:50.668 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/3056d51a-e76b-11ee-95cd-87774f69a715/mon.a/config 2024-03-21T10:22:50.710 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/3056d51a-e76b-11ee-95cd-87774f69a715/mon.a/config 2024-03-21T10:22:50.757 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/3056d51a-e76b-11ee-95cd-87774f69a715/mon.a/config 2024-03-21T10:22:51.964 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:22:51 smithi047 ceph-mon[31547]: pgmap v395: 129 pgs: 129 active+clean; 642 KiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 1.8 KiB/s wr, 11 op/s 2024-03-21T10:22:52.229 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:22:51 smithi100 ceph-mon[35193]: pgmap v395: 129 pgs: 129 active+clean; 642 KiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 1.8 KiB/s wr, 11 op/s 2024-03-21T10:22:52.239 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:22:51 smithi157 ceph-mon[35581]: pgmap v395: 129 pgs: 129 active+clean; 642 KiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 1.8 KiB/s wr, 11 op/s 2024-03-21T10:22:54.161 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:22:53 smithi047 ceph-mon[31547]: pgmap v396: 129 pgs: 129 active+clean; 642 KiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 1.2 KiB/s wr, 8 op/s 2024-03-21T10:22:54.229 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:22:53 smithi100 ceph-mon[35193]: pgmap v396: 129 pgs: 129 active+clean; 642 KiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 1.2 KiB/s wr, 8 op/s 2024-03-21T10:22:54.239 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:22:53 smithi157 ceph-mon[35581]: pgmap v396: 129 pgs: 129 active+clean; 642 KiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 1.2 KiB/s wr, 8 op/s 2024-03-21T10:22:56.213 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:22:55 smithi047 ceph-mon[31547]: pgmap v397: 129 pgs: 129 active+clean; 642 KiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 1.2 KiB/s wr, 12 op/s 2024-03-21T10:22:56.229 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:22:55 smithi100 ceph-mon[35193]: pgmap v397: 129 pgs: 129 active+clean; 642 KiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 1.2 KiB/s wr, 12 op/s 2024-03-21T10:22:56.240 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:22:55 smithi157 ceph-mon[35581]: pgmap v397: 129 pgs: 129 active+clean; 642 KiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 1.2 KiB/s wr, 12 op/s 2024-03-21T10:22:56.647 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/3056d51a-e76b-11ee-95cd-87774f69a715/mon.a/config 2024-03-21T10:22:56.648 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/3056d51a-e76b-11ee-95cd-87774f69a715/mon.a/config 2024-03-21T10:22:56.650 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/3056d51a-e76b-11ee-95cd-87774f69a715/mon.a/config 2024-03-21T10:22:57.883 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/3056d51a-e76b-11ee-95cd-87774f69a715/mon.a/config 2024-03-21T10:22:58.087 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:22:57 smithi047 ceph-mon[31547]: pgmap v398: 129 pgs: 129 active+clean; 642 KiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-21T10:22:58.229 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:22:57 smithi100 ceph-mon[35193]: pgmap v398: 129 pgs: 129 active+clean; 642 KiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-21T10:22:58.240 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:22:57 smithi157 ceph-mon[35581]: pgmap v398: 129 pgs: 129 active+clean; 642 KiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-21T10:22:59.781 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:59.779+0000 7f983f000700 1 -- 172.21.15.47:0/3039119585 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9838102df0 msgr2=0x7f9838106e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:22:59.781 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:59.779+0000 7f983f000700 1 --2- 172.21.15.47:0/3039119585 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9838102df0 0x7f9838106e30 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f98240099c0 tx=0x7f9824009cd0 comp rx=0 tx=0).stop 2024-03-21T10:22:59.781 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:59.779+0000 7f983f000700 1 -- 172.21.15.47:0/3039119585 shutdown_connections 2024-03-21T10:22:59.781 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:59.779+0000 7f983f000700 1 --2- 172.21.15.47:0/3039119585 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9838107510 0x7f983810f0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:22:59.781 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:59.779+0000 7f983f000700 1 --2- 172.21.15.47:0/3039119585 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9838102440 0x7f9838102820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:22:59.781 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:59.779+0000 7f983f000700 1 --2- 172.21.15.47:0/3039119585 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9838102df0 0x7f9838106e30 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:22:59.781 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:59.779+0000 7f983f000700 1 -- 172.21.15.47:0/3039119585 >> 172.21.15.47:0/3039119585 conn(0x7f98380792f0 msgr2=0x7f9838079700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:22:59.782 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:59.780+0000 7f983f000700 1 -- 172.21.15.47:0/3039119585 shutdown_connections 2024-03-21T10:22:59.782 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:59.780+0000 7f983f000700 1 -- 172.21.15.47:0/3039119585 wait complete. 2024-03-21T10:22:59.782 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:59.781+0000 7f983f000700 1 Processor -- start 2024-03-21T10:22:59.782 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:59.781+0000 7f983f000700 1 -- start start 2024-03-21T10:22:59.783 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:59.782+0000 7f983f000700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9838102440 0x7f983819e690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:22:59.783 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:59.782+0000 7f983f000700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9838102df0 0x7f983819ebd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:22:59.784 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:59.782+0000 7f983f000700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9838107510 0x7f9838198810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:22:59.784 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:59.782+0000 7f983f000700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98381123f0 con 0x7f9838107510 2024-03-21T10:22:59.784 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:59.782+0000 7f983f000700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9838112270 con 0x7f9838102df0 2024-03-21T10:22:59.784 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:59.782+0000 7f983f000700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9838112570 con 0x7f9838102440 2024-03-21T10:22:59.784 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:59.782+0000 7f9837400700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9838102df0 0x7f983819ebd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:22:59.784 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:59.782+0000 7f983d000700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9838107510 0x7f9838198810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:22:59.784 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:59.782+0000 7f9837400700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9838102df0 0x7f983819ebd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:42374/0 (socket says 172.21.15.47:42374) 2024-03-21T10:22:59.784 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:59.782+0000 7f9837400700 1 -- 172.21.15.47:0/4179743435 learned_addr learned my addr 172.21.15.47:0/4179743435 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:22:59.784 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:59.782+0000 7f9837e00700 1 --2- 172.21.15.47:0/4179743435 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9838102440 0x7f983819e690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:22:59.785 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:59.783+0000 7f9837400700 1 -- 172.21.15.47:0/4179743435 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9838102440 msgr2=0x7f983819e690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:22:59.785 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:59.783+0000 7f9837400700 1 --2- 172.21.15.47:0/4179743435 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9838102440 0x7f983819e690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:22:59.785 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:59.783+0000 7f9837400700 1 -- 172.21.15.47:0/4179743435 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9838107510 msgr2=0x7f9838198810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:22:59.785 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:59.783+0000 7f9837400700 1 --2- 172.21.15.47:0/4179743435 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9838107510 0x7f9838198810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:22:59.785 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:59.783+0000 7f9837400700 1 -- 172.21.15.47:0/4179743435 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9838198ff0 con 0x7f9838102df0 2024-03-21T10:22:59.785 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:59.783+0000 7f9837e00700 1 --2- 172.21.15.47:0/4179743435 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9838102440 0x7f983819e690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T10:22:59.786 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:59.783+0000 7f9837400700 1 --2- 172.21.15.47:0/4179743435 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9838102df0 0x7f983819ebd0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f98240055f0 tx=0x7f9824005620 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:22:59.786 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:59.784+0000 7f9834c00700 1 -- 172.21.15.47:0/4179743435 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9824026070 con 0x7f9838102df0 2024-03-21T10:22:59.786 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:59.784+0000 7f983f000700 1 -- 172.21.15.47:0/4179743435 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9838199280 con 0x7f9838102df0 2024-03-21T10:22:59.786 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:59.784+0000 7f9834c00700 1 -- 172.21.15.47:0/4179743435 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f982400f460 con 0x7f9838102df0 2024-03-21T10:22:59.786 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:59.784+0000 7f9834c00700 1 -- 172.21.15.47:0/4179743435 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9824009e60 con 0x7f9838102df0 2024-03-21T10:22:59.786 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:59.784+0000 7f983f000700 1 -- 172.21.15.47:0/4179743435 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f98381a54f0 con 0x7f9838102df0 2024-03-21T10:22:59.786 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:59.785+0000 7f983f000700 1 -- 172.21.15.47:0/4179743435 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f98380628a0 con 0x7f9838102df0 2024-03-21T10:22:59.789 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:59.788+0000 7f9834c00700 1 -- 172.21.15.47:0/4179743435 <== mon.2 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 17) v1 ==== 137905+0+0 (secure 0 0 0) 0x7f9824029070 con 0x7f9838102df0 2024-03-21T10:22:59.792 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:59.791+0000 7f9834c00700 1 --2- 172.21.15.47:0/4179743435 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f981c0a3040 0x7f981c0a5500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:22:59.792 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:59.791+0000 7f9834c00700 1 -- 172.21.15.47:0/4179743435 <== mon.2 v2:172.21.15.100:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9092+0+0 (secure 0 0 0) 0x7f98240d05a0 con 0x7f9838102df0 2024-03-21T10:22:59.793 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:59.791+0000 7f9837e00700 1 --2- 172.21.15.47:0/4179743435 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f981c0a3040 0x7f981c0a5500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:22:59.793 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:59.791+0000 7f9834c00700 1 --2- 172.21.15.47:0/4179743435 >> [v2:172.21.15.100:6808/2785478639,v1:172.21.15.100:6809/2785478639] conn(0x7f981c0ae1a0 0x7f981c0b0640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:22:59.793 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:59.792+0000 7f9834c00700 1 -- 172.21.15.47:0/4179743435 --> [v2:172.21.15.100:6808/2785478639,v1:172.21.15.100:6809/2785478639] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f982402de50 con 0x7f981c0ae1a0 2024-03-21T10:22:59.793 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:59.792+0000 7f9834c00700 1 -- 172.21.15.47:0/4179743435 <== mon.2 v2:172.21.15.100:3300/0 6 ==== mon_get_version_reply(handle=1 version=93) v2 ==== 24+0+0 (secure 0 0 0) 0x7f98240d09f0 con 0x7f9838102df0 2024-03-21T10:22:59.793 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:59.792+0000 7f9837e00700 1 --2- 172.21.15.47:0/4179743435 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f981c0a3040 0x7f981c0a5500 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f982800bcf0 tx=0x7f982800b2c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:22:59.802 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:59.801+0000 7f983d000700 1 --2- 172.21.15.47:0/4179743435 >> [v2:172.21.15.100:6808/2785478639,v1:172.21.15.100:6809/2785478639] conn(0x7f981c0ae1a0 0x7f981c0b0640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:22:59.803 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:59.802+0000 7f983d000700 1 --2- 172.21.15.47:0/4179743435 >> [v2:172.21.15.100:6808/2785478639,v1:172.21.15.100:6809/2785478639] conn(0x7f981c0ae1a0 0x7f981c0b0640 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:22:59.805 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:59.804+0000 7f9834c00700 1 -- 172.21.15.47:0/4179743435 <== osd.4 v2:172.21.15.100:6808/2785478639 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7f982402de50 con 0x7f981c0ae1a0 2024-03-21T10:22:59.825 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:59.824+0000 7f983f000700 1 -- 172.21.15.47:0/4179743435 --> [v2:172.21.15.100:6808/2785478639,v1:172.21.15.100:6809/2785478639] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f9838062ca0 con 0x7f981c0ae1a0 2024-03-21T10:22:59.826 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:59.825+0000 7f9834c00700 1 -- 172.21.15.47:0/4179743435 <== osd.4 v2:172.21.15.100:6808/2785478639 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f9838062ca0 con 0x7f981c0ae1a0 2024-03-21T10:22:59.827 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:59.825+0000 7f983f000700 1 -- 172.21.15.47:0/4179743435 >> [v2:172.21.15.100:6808/2785478639,v1:172.21.15.100:6809/2785478639] conn(0x7f981c0ae1a0 msgr2=0x7f981c0b0640 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:22:59.827 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:59.825+0000 7f983f000700 1 --2- 172.21.15.47:0/4179743435 >> [v2:172.21.15.100:6808/2785478639,v1:172.21.15.100:6809/2785478639] conn(0x7f981c0ae1a0 0x7f981c0b0640 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:22:59.827 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:59.826+0000 7f983f000700 1 -- 172.21.15.47:0/4179743435 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f981c0a3040 msgr2=0x7f981c0a5500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:22:59.827 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:59.826+0000 7f983f000700 1 --2- 172.21.15.47:0/4179743435 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f981c0a3040 0x7f981c0a5500 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f982800bcf0 tx=0x7f982800b2c0 comp rx=0 tx=0).stop 2024-03-21T10:22:59.827 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:59.826+0000 7f983f000700 1 -- 172.21.15.47:0/4179743435 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9838102df0 msgr2=0x7f983819ebd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:22:59.827 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:59.826+0000 7f983f000700 1 --2- 172.21.15.47:0/4179743435 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9838102df0 0x7f983819ebd0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f98240055f0 tx=0x7f9824005620 comp rx=0 tx=0).stop 2024-03-21T10:22:59.827 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:59.826+0000 7f983d000700 1 -- 172.21.15.47:0/4179743435 reap_dead start 2024-03-21T10:22:59.827 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:59.826+0000 7f983f000700 1 -- 172.21.15.47:0/4179743435 shutdown_connections 2024-03-21T10:22:59.827 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:59.826+0000 7f983f000700 1 -- 172.21.15.47:0/4179743435 >> 172.21.15.47:0/4179743435 conn(0x7f98380792f0 msgr2=0x7f9838100500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:22:59.828 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:59.826+0000 7f983f000700 1 -- 172.21.15.47:0/4179743435 shutdown_connections 2024-03-21T10:22:59.828 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:22:59.826+0000 7f983f000700 1 -- 172.21.15.47:0/4179743435 wait complete. 2024-03-21T10:23:00.011 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:22:59 smithi047 ceph-mon[31547]: pgmap v399: 129 pgs: 129 active+clean; 642 KiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-03-21T10:23:00.229 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:22:59 smithi100 ceph-mon[35193]: pgmap v399: 129 pgs: 129 active+clean; 642 KiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-03-21T10:23:00.239 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:22:59 smithi157 ceph-mon[35581]: pgmap v399: 129 pgs: 129 active+clean; 642 KiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-03-21T10:23:02.207 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:23:01 smithi047 ceph-mon[31547]: pgmap v400: 129 pgs: 129 active+clean; 642 KiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-03-21T10:23:02.229 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:23:01 smithi100 ceph-mon[35193]: pgmap v400: 129 pgs: 129 active+clean; 642 KiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-03-21T10:23:02.240 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:23:01 smithi157 ceph-mon[35581]: pgmap v400: 129 pgs: 129 active+clean; 642 KiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-03-21T10:23:03.204 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:23:02 smithi047 ceph-mon[31547]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T10:23:03.229 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:23:02 smithi100 ceph-mon[35193]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T10:23:03.239 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:23:02 smithi157 ceph-mon[35581]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T10:23:03.314 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:03.219+0000 7fa331800700 1 -- 172.21.15.47:0/2071851799 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa32c1021e0 msgr2=0x7fa32c102660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:23:03.314 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:03.219+0000 7fa331800700 1 --2- 172.21.15.47:0/2071851799 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa32c1021e0 0x7fa32c102660 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fa3180099c0 tx=0x7fa318009cd0 comp rx=0 tx=0).stop 2024-03-21T10:23:03.314 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:03.219+0000 7fa331800700 1 -- 172.21.15.47:0/2071851799 shutdown_connections 2024-03-21T10:23:03.314 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:03.219+0000 7fa331800700 1 --2- 172.21.15.47:0/2071851799 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa32c102ba0 0x7fa32c10efe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:23:03.314 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:03.219+0000 7fa331800700 1 --2- 172.21.15.47:0/2071851799 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa32c1081c0 0x7fa32c1085a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:23:03.314 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:03.219+0000 7fa331800700 1 --2- 172.21.15.47:0/2071851799 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa32c1021e0 0x7fa32c102660 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:23:03.314 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:03.219+0000 7fa331800700 1 -- 172.21.15.47:0/2071851799 >> 172.21.15.47:0/2071851799 conn(0x7fa32c0792c0 msgr2=0x7fa32c0796d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:23:03.314 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:03.219+0000 7fa331800700 1 -- 172.21.15.47:0/2071851799 shutdown_connections 2024-03-21T10:23:03.315 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:03.220+0000 7fa331800700 1 -- 172.21.15.47:0/2071851799 wait complete. 2024-03-21T10:23:03.315 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:03.221+0000 7fa331800700 1 Processor -- start 2024-03-21T10:23:03.315 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:03.221+0000 7fa331800700 1 -- start start 2024-03-21T10:23:03.315 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:03.221+0000 7fa331800700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa32c1021e0 0x7fa32c19e5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:23:03.315 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:03.221+0000 7fa331800700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa32c102ba0 0x7fa32c19eb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:23:03.315 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:03.221+0000 7fa331800700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa32c1081c0 0x7fa32c198690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:23:03.315 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:03.221+0000 7fa331800700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa32c112310 con 0x7fa32c1021e0 2024-03-21T10:23:03.315 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:03.221+0000 7fa331800700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa32c112190 con 0x7fa32c102ba0 2024-03-21T10:23:03.315 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:03.221+0000 7fa331800700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa32c112490 con 0x7fa32c1081c0 2024-03-21T10:23:03.315 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:03.221+0000 7fa32aa00700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa32c1021e0 0x7fa32c19e5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:23:03.315 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:03.221+0000 7fa32aa00700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa32c1021e0 0x7fa32c19e5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:54854/0 (socket says 172.21.15.47:54854) 2024-03-21T10:23:03.315 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:03.222+0000 7fa32aa00700 1 -- 172.21.15.47:0/3318462909 learned_addr learned my addr 172.21.15.47:0/3318462909 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:23:03.315 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:03.222+0000 7fa32a000700 1 --2- 172.21.15.47:0/3318462909 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa32c102ba0 0x7fa32c19eb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:23:03.315 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:03.222+0000 7fa32b400700 1 --2- 172.21.15.47:0/3318462909 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa32c1081c0 0x7fa32c198690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:23:03.316 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:03.222+0000 7fa32a000700 1 -- 172.21.15.47:0/3318462909 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa32c1081c0 msgr2=0x7fa32c198690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:23:03.316 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:03.222+0000 7fa32a000700 1 --2- 172.21.15.47:0/3318462909 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa32c1081c0 0x7fa32c198690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:23:03.316 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:03.222+0000 7fa32a000700 1 -- 172.21.15.47:0/3318462909 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa32c1021e0 msgr2=0x7fa32c19e5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:23:03.316 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:03.222+0000 7fa32a000700 1 --2- 172.21.15.47:0/3318462909 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa32c1021e0 0x7fa32c19e5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:23:03.316 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:03.222+0000 7fa32a000700 1 -- 172.21.15.47:0/3318462909 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa32c198f70 con 0x7fa32c102ba0 2024-03-21T10:23:03.316 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:03.222+0000 7fa32aa00700 1 --2- 172.21.15.47:0/3318462909 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa32c1021e0 0x7fa32c19e5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T10:23:03.316 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:03.222+0000 7fa32a000700 1 --2- 172.21.15.47:0/3318462909 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa32c102ba0 0x7fa32c19eb00 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fa3180098e0 tx=0x7fa318004270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:23:03.316 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:03.223+0000 7fa323400700 1 -- 172.21.15.47:0/3318462909 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa318026070 con 0x7fa32c102ba0 2024-03-21T10:23:03.316 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:03.223+0000 7fa331800700 1 -- 172.21.15.47:0/3318462909 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa32c199200 con 0x7fa32c102ba0 2024-03-21T10:23:03.316 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:03.223+0000 7fa323400700 1 -- 172.21.15.47:0/3318462909 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fa318003c70 con 0x7fa32c102ba0 2024-03-21T10:23:03.316 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:03.223+0000 7fa323400700 1 -- 172.21.15.47:0/3318462909 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa31800f9c0 con 0x7fa32c102ba0 2024-03-21T10:23:03.316 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:03.223+0000 7fa331800700 1 -- 172.21.15.47:0/3318462909 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa32c1a5370 con 0x7fa32c102ba0 2024-03-21T10:23:03.316 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:03.224+0000 7fa331800700 1 -- 172.21.15.47:0/3318462909 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fa32c0628e0 con 0x7fa32c102ba0 2024-03-21T10:23:03.316 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:03.228+0000 7fa323400700 1 -- 172.21.15.47:0/3318462909 <== mon.2 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 17) v1 ==== 137905+0+0 (secure 0 0 0) 0x7fa318029070 con 0x7fa32c102ba0 2024-03-21T10:23:03.317 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:03.229+0000 7fa323400700 1 --2- 172.21.15.47:0/3318462909 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7fa3100a3040 0x7fa3100a5500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:23:03.317 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:03.229+0000 7fa323400700 1 -- 172.21.15.47:0/3318462909 <== mon.2 v2:172.21.15.100:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9092+0+0 (secure 0 0 0) 0x7fa3180d0410 con 0x7fa32c102ba0 2024-03-21T10:23:03.317 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:03.230+0000 7fa32aa00700 1 --2- 172.21.15.47:0/3318462909 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7fa3100a3040 0x7fa3100a5500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:23:03.317 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:03.230+0000 7fa323400700 1 --2- 172.21.15.47:0/3318462909 >> [v2:172.21.15.100:6824/396391028,v1:172.21.15.100:6825/396391028] conn(0x7fa3100ae1a0 0x7fa3100b0640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:23:03.317 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:03.230+0000 7fa323400700 1 -- 172.21.15.47:0/3318462909 --> [v2:172.21.15.100:6824/396391028,v1:172.21.15.100:6825/396391028] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fa31800b0b0 con 0x7fa3100ae1a0 2024-03-21T10:23:03.317 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:03.230+0000 7fa323400700 1 -- 172.21.15.47:0/3318462909 <== mon.2 v2:172.21.15.100:3300/0 6 ==== mon_get_version_reply(handle=1 version=93) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa3180d0860 con 0x7fa32c102ba0 2024-03-21T10:23:03.317 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:03.230+0000 7fa32b400700 1 --2- 172.21.15.47:0/3318462909 >> [v2:172.21.15.100:6824/396391028,v1:172.21.15.100:6825/396391028] conn(0x7fa3100ae1a0 0x7fa3100b0640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:23:03.317 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:03.231+0000 7fa32aa00700 1 --2- 172.21.15.47:0/3318462909 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7fa3100a3040 0x7fa3100a5500 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fa31c011550 tx=0x7fa31c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:23:03.317 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:03.231+0000 7fa32b400700 1 --2- 172.21.15.47:0/3318462909 >> [v2:172.21.15.100:6824/396391028,v1:172.21.15.100:6825/396391028] conn(0x7fa3100ae1a0 0x7fa3100b0640 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.10 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:23:03.317 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:03.232+0000 7fa323400700 1 -- 172.21.15.47:0/3318462909 <== osd.10 v2:172.21.15.100:6824/396391028 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7fa31800b0b0 con 0x7fa3100ae1a0 2024-03-21T10:23:03.317 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:03.252+0000 7fa331800700 1 -- 172.21.15.47:0/3318462909 --> [v2:172.21.15.100:6824/396391028,v1:172.21.15.100:6825/396391028] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fa32c062ce0 con 0x7fa3100ae1a0 2024-03-21T10:23:03.318 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:03.252+0000 7fa323400700 1 -- 172.21.15.47:0/3318462909 <== osd.10 v2:172.21.15.100:6824/396391028 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7fa32c062ce0 con 0x7fa3100ae1a0 2024-03-21T10:23:03.318 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:03.253+0000 7fa331800700 1 -- 172.21.15.47:0/3318462909 >> [v2:172.21.15.100:6824/396391028,v1:172.21.15.100:6825/396391028] conn(0x7fa3100ae1a0 msgr2=0x7fa3100b0640 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:23:03.318 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:03.253+0000 7fa331800700 1 --2- 172.21.15.47:0/3318462909 >> [v2:172.21.15.100:6824/396391028,v1:172.21.15.100:6825/396391028] conn(0x7fa3100ae1a0 0x7fa3100b0640 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:23:03.318 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:03.253+0000 7fa331800700 1 -- 172.21.15.47:0/3318462909 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7fa3100a3040 msgr2=0x7fa3100a5500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:23:03.318 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:03.253+0000 7fa331800700 1 --2- 172.21.15.47:0/3318462909 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7fa3100a3040 0x7fa3100a5500 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fa31c011550 tx=0x7fa31c009260 comp rx=0 tx=0).stop 2024-03-21T10:23:03.318 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:03.253+0000 7fa331800700 1 -- 172.21.15.47:0/3318462909 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa32c102ba0 msgr2=0x7fa32c19eb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:23:03.318 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:03.253+0000 7fa331800700 1 --2- 172.21.15.47:0/3318462909 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa32c102ba0 0x7fa32c19eb00 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fa3180098e0 tx=0x7fa318004270 comp rx=0 tx=0).stop 2024-03-21T10:23:03.318 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:03.253+0000 7fa32b400700 1 -- 172.21.15.47:0/3318462909 reap_dead start 2024-03-21T10:23:03.319 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:03.253+0000 7fa331800700 1 -- 172.21.15.47:0/3318462909 shutdown_connections 2024-03-21T10:23:03.319 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:03.253+0000 7fa331800700 1 -- 172.21.15.47:0/3318462909 >> 172.21.15.47:0/3318462909 conn(0x7fa32c0792c0 msgr2=0x7fa32c100f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:23:03.319 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:03.253+0000 7fa331800700 1 -- 172.21.15.47:0/3318462909 shutdown_connections 2024-03-21T10:23:03.319 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:03.253+0000 7fa331800700 1 -- 172.21.15.47:0/3318462909 wait complete. 2024-03-21T10:23:03.904 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:23:03 smithi047 ceph-mon[31547]: pgmap v401: 129 pgs: 129 active+clean; 642 KiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-21T10:23:03.904 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:23:03 smithi047 ceph-mon[31547]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:23:03.904 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:23:03 smithi047 ceph-mon[31547]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:23:03.990 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:23:03 smithi157 ceph-mon[35581]: pgmap v401: 129 pgs: 129 active+clean; 642 KiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-21T10:23:03.990 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:23:03 smithi157 ceph-mon[35581]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:23:03.990 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:23:03 smithi157 ceph-mon[35581]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:23:04.229 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:23:03 smithi100 ceph-mon[35193]: pgmap v401: 129 pgs: 129 active+clean; 642 KiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-21T10:23:04.230 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:23:03 smithi100 ceph-mon[35193]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:23:04.230 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:23:03 smithi100 ceph-mon[35193]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:23:04.405 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:04.334+0000 7f585f800700 1 -- 172.21.15.47:0/3335297644 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5858102440 msgr2=0x7f5858102820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:23:04.406 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:04.334+0000 7f585f800700 1 --2- 172.21.15.47:0/3335297644 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5858102440 0x7f5858102820 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f584c00b0a0 tx=0x7f584c00b3b0 comp rx=0 tx=0).stop 2024-03-21T10:23:04.406 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:04.335+0000 7f585f800700 1 -- 172.21.15.47:0/3335297644 shutdown_connections 2024-03-21T10:23:04.406 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:04.335+0000 7f585f800700 1 --2- 172.21.15.47:0/3335297644 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5858107510 0x7f585810f0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:23:04.406 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:04.335+0000 7f585f800700 1 --2- 172.21.15.47:0/3335297644 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5858102440 0x7f5858102820 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:23:04.406 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:04.335+0000 7f585f800700 1 --2- 172.21.15.47:0/3335297644 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5858102df0 0x7f5858106e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:23:04.406 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:04.335+0000 7f585f800700 1 -- 172.21.15.47:0/3335297644 >> 172.21.15.47:0/3335297644 conn(0x7f58580792f0 msgr2=0x7f5858079700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:23:04.406 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:04.335+0000 7f585f800700 1 -- 172.21.15.47:0/3335297644 shutdown_connections 2024-03-21T10:23:04.406 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:04.335+0000 7f585f800700 1 -- 172.21.15.47:0/3335297644 wait complete. 2024-03-21T10:23:04.406 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:04.336+0000 7f585f800700 1 Processor -- start 2024-03-21T10:23:04.406 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:04.336+0000 7f585f800700 1 -- start start 2024-03-21T10:23:04.406 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:04.336+0000 7f585f800700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5858102440 0x7f58581090b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:23:04.407 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:04.336+0000 7f585f800700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5858102df0 0x7f58581095f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:23:04.407 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:04.336+0000 7f585f800700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5858107510 0x7f5858109b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:23:04.407 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:04.336+0000 7f585f800700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58581123f0 con 0x7f5858102440 2024-03-21T10:23:04.407 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:04.336+0000 7f585f800700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5858112270 con 0x7f5858102df0 2024-03-21T10:23:04.407 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:04.336+0000 7f585f800700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5858112570 con 0x7f5858107510 2024-03-21T10:23:04.407 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:04.337+0000 7f585ce00700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5858102440 0x7f58581090b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:23:04.407 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:04.337+0000 7f5857e00700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5858102df0 0x7f58581095f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:23:04.407 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:04.337+0000 7f585ce00700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5858102440 0x7f58581090b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:54866/0 (socket says 172.21.15.47:54866) 2024-03-21T10:23:04.407 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:04.337+0000 7f585ce00700 1 -- 172.21.15.47:0/125680163 learned_addr learned my addr 172.21.15.47:0/125680163 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:23:04.407 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:04.337+0000 7f585d800700 1 --2- 172.21.15.47:0/125680163 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5858107510 0x7f5858109b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:23:04.407 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:04.337+0000 7f585ce00700 1 -- 172.21.15.47:0/125680163 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5858107510 msgr2=0x7f5858109b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:23:04.407 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:04.337+0000 7f585ce00700 1 --2- 172.21.15.47:0/125680163 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5858107510 0x7f5858109b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:23:04.407 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:04.337+0000 7f585ce00700 1 -- 172.21.15.47:0/125680163 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5858102df0 msgr2=0x7f58581095f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:23:04.408 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:04.337+0000 7f585ce00700 1 --2- 172.21.15.47:0/125680163 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5858102df0 0x7f58581095f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:23:04.408 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:04.337+0000 7f585ce00700 1 -- 172.21.15.47:0/125680163 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58581ab1f0 con 0x7f5858102440 2024-03-21T10:23:04.408 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:04.337+0000 7f5857e00700 1 --2- 172.21.15.47:0/125680163 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5858102df0 0x7f58581095f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T10:23:04.408 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:04.338+0000 7f585ce00700 1 --2- 172.21.15.47:0/125680163 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5858102440 0x7f58581090b0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f584c004c60 tx=0x7f584c004c90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:23:04.408 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:04.338+0000 7f5855600700 1 -- 172.21.15.47:0/125680163 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f584c021070 con 0x7f5858102440 2024-03-21T10:23:04.408 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:04.338+0000 7f5855600700 1 -- 172.21.15.47:0/125680163 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f584c003c00 con 0x7f5858102440 2024-03-21T10:23:04.408 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:04.338+0000 7f5855600700 1 -- 172.21.15.47:0/125680163 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f584c004780 con 0x7f5858102440 2024-03-21T10:23:04.408 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:04.338+0000 7f585f800700 1 -- 172.21.15.47:0/125680163 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f58581ab480 con 0x7f5858102440 2024-03-21T10:23:04.408 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:04.338+0000 7f585f800700 1 -- 172.21.15.47:0/125680163 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f58581ab8e0 con 0x7f5858102440 2024-03-21T10:23:04.408 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:04.339+0000 7f585f800700 1 -- 172.21.15.47:0/125680163 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f58581089b0 con 0x7f5858102440 2024-03-21T10:23:04.408 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:04.341+0000 7f5855600700 1 -- 172.21.15.47:0/125680163 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 17) v1 ==== 137905+0+0 (secure 0 0 0) 0x7f584c007980 con 0x7f5858102440 2024-03-21T10:23:04.408 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:04.343+0000 7f5855600700 1 --2- 172.21.15.47:0/125680163 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f58400a30f0 0x7f58400a55b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:23:04.408 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:04.343+0000 7f5855600700 1 -- 172.21.15.47:0/125680163 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9092+0+0 (secure 0 0 0) 0x7f584c00f020 con 0x7f5858102440 2024-03-21T10:23:04.408 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:04.343+0000 7f5857e00700 1 --2- 172.21.15.47:0/125680163 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f58400a30f0 0x7f58400a55b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:23:04.408 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:04.343+0000 7f5855600700 1 --2- 172.21.15.47:0/125680163 >> [v2:172.21.15.157:6800/171377005,v1:172.21.15.157:6801/171377005] conn(0x7f58400ae250 0x7f58400b06f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:23:04.409 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:04.343+0000 7f5855600700 1 -- 172.21.15.47:0/125680163 --> [v2:172.21.15.157:6800/171377005,v1:172.21.15.157:6801/171377005] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f584c0020b0 con 0x7f58400ae250 2024-03-21T10:23:04.409 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:04.343+0000 7f5855600700 1 -- 172.21.15.47:0/125680163 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_get_version_reply(handle=1 version=93) v2 ==== 24+0+0 (secure 0 0 0) 0x7f584c0d52a0 con 0x7f5858102440 2024-03-21T10:23:04.409 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:04.344+0000 7f585d800700 1 --2- 172.21.15.47:0/125680163 >> [v2:172.21.15.157:6800/171377005,v1:172.21.15.157:6801/171377005] conn(0x7f58400ae250 0x7f58400b06f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:23:04.409 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:04.344+0000 7f5857e00700 1 --2- 172.21.15.47:0/125680163 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f58400a30f0 0x7f58400a55b0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f5844009680 tx=0x7f5844009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:23:04.409 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:04.344+0000 7f585d800700 1 --2- 172.21.15.47:0/125680163 >> [v2:172.21.15.157:6800/171377005,v1:172.21.15.157:6801/171377005] conn(0x7f58400ae250 0x7f58400b06f0 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:23:04.409 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:04.347+0000 7f5855600700 1 -- 172.21.15.47:0/125680163 <== osd.2 v2:172.21.15.157:6800/171377005 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7f584c0020b0 con 0x7f58400ae250 2024-03-21T10:23:04.409 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:04.368+0000 7f585f800700 1 -- 172.21.15.47:0/125680163 --> [v2:172.21.15.157:6800/171377005,v1:172.21.15.157:6801/171377005] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f5858103910 con 0x7f58400ae250 2024-03-21T10:23:04.409 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:04.369+0000 7f5855600700 1 -- 172.21.15.47:0/125680163 <== osd.2 v2:172.21.15.157:6800/171377005 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f5858103910 con 0x7f58400ae250 2024-03-21T10:23:04.410 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:04.370+0000 7f585f800700 1 -- 172.21.15.47:0/125680163 >> [v2:172.21.15.157:6800/171377005,v1:172.21.15.157:6801/171377005] conn(0x7f58400ae250 msgr2=0x7f58400b06f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:23:04.410 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:04.370+0000 7f585f800700 1 --2- 172.21.15.47:0/125680163 >> [v2:172.21.15.157:6800/171377005,v1:172.21.15.157:6801/171377005] conn(0x7f58400ae250 0x7f58400b06f0 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:23:04.410 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:04.370+0000 7f585f800700 1 -- 172.21.15.47:0/125680163 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f58400a30f0 msgr2=0x7f58400a55b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:23:04.410 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:04.370+0000 7f585f800700 1 --2- 172.21.15.47:0/125680163 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f58400a30f0 0x7f58400a55b0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f5844009680 tx=0x7f5844009260 comp rx=0 tx=0).stop 2024-03-21T10:23:04.410 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:04.370+0000 7f585f800700 1 -- 172.21.15.47:0/125680163 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5858102440 msgr2=0x7f58581090b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:23:04.410 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:04.370+0000 7f585f800700 1 --2- 172.21.15.47:0/125680163 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5858102440 0x7f58581090b0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f584c004c60 tx=0x7f584c004c90 comp rx=0 tx=0).stop 2024-03-21T10:23:04.410 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:04.370+0000 7f585d800700 1 -- 172.21.15.47:0/125680163 reap_dead start 2024-03-21T10:23:04.411 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:04.370+0000 7f585f800700 1 -- 172.21.15.47:0/125680163 shutdown_connections 2024-03-21T10:23:04.411 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:04.370+0000 7f585f800700 1 -- 172.21.15.47:0/125680163 >> 172.21.15.47:0/125680163 conn(0x7f58580792f0 msgr2=0x7f5858100ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:23:04.411 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:04.370+0000 7f585f800700 1 -- 172.21.15.47:0/125680163 shutdown_connections 2024-03-21T10:23:04.411 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:04.370+0000 7f585f800700 1 -- 172.21.15.47:0/125680163 wait complete. 2024-03-21T10:23:05.963 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:23:05 smithi047 ceph-mon[31547]: pgmap v402: 129 pgs: 129 active+clean; 642 KiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-03-21T10:23:05.964 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:23:05 smithi047 ceph-mon[31547]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:23:05.964 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:23:05 smithi047 ceph-mon[31547]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:23:05.964 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:23:05 smithi047 ceph-mon[31547]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:23:05.964 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:23:05 smithi047 ceph-mon[31547]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:23:06.140 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:06.139+0000 7f5467800700 1 -- 172.21.15.47:0/100939140 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5460102df0 msgr2=0x7f5460106e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:23:06.140 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:06.139+0000 7f5467800700 1 --2- 172.21.15.47:0/100939140 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5460102df0 0x7f5460106e30 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f544c0099c0 tx=0x7f544c009cd0 comp rx=0 tx=0).stop 2024-03-21T10:23:06.141 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:06.140+0000 7f5467800700 1 -- 172.21.15.47:0/100939140 shutdown_connections 2024-03-21T10:23:06.141 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:06.140+0000 7f5467800700 1 --2- 172.21.15.47:0/100939140 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5460107510 0x7f546010f0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:23:06.141 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:06.140+0000 7f5467800700 1 --2- 172.21.15.47:0/100939140 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5460102440 0x7f5460102820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:23:06.141 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:06.140+0000 7f5467800700 1 --2- 172.21.15.47:0/100939140 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5460102df0 0x7f5460106e30 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:23:06.141 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:06.140+0000 7f5467800700 1 -- 172.21.15.47:0/100939140 >> 172.21.15.47:0/100939140 conn(0x7f54600792f0 msgr2=0x7f5460079700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:23:06.143 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/3056d51a-e76b-11ee-95cd-87774f69a715/mon.a/config 2024-03-21T10:23:06.144 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:06.142+0000 7f5467800700 1 -- 172.21.15.47:0/100939140 shutdown_connections 2024-03-21T10:23:06.144 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:06.143+0000 7f5467800700 1 -- 172.21.15.47:0/100939140 wait complete. 2024-03-21T10:23:06.145 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:06.144+0000 7f5467800700 1 Processor -- start 2024-03-21T10:23:06.146 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:06.145+0000 7f5467800700 1 -- start start 2024-03-21T10:23:06.146 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:06.145+0000 7f5467800700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5460102440 0x7f546019f570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:23:06.146 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:06.145+0000 7f5467800700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5460102df0 0x7f546019fab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:23:06.147 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:06.145+0000 7f5467800700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5460107510 0x7f546010e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:23:06.147 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:06.145+0000 7f5467800700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54601123f0 con 0x7f5460107510 2024-03-21T10:23:06.147 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:06.145+0000 7f5467800700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5460112270 con 0x7f5460102df0 2024-03-21T10:23:06.147 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:06.145+0000 7f5467800700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5460112570 con 0x7f5460102440 2024-03-21T10:23:06.147 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:06.146+0000 7f5464e00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5460102440 0x7f546019f570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:23:06.147 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:06.146+0000 7f5464e00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5460102440 0x7f546019f570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.47:48856/0 (socket says 172.21.15.47:48856) 2024-03-21T10:23:06.148 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:06.146+0000 7f5464e00700 1 -- 172.21.15.47:0/1387670103 learned_addr learned my addr 172.21.15.47:0/1387670103 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:23:06.148 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/3056d51a-e76b-11ee-95cd-87774f69a715/mon.a/config 2024-03-21T10:23:06.148 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:06.146+0000 7f5465800700 1 --2- 172.21.15.47:0/1387670103 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5460107510 0x7f546010e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:23:06.149 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:06.147+0000 7f5465800700 1 -- 172.21.15.47:0/1387670103 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5460102440 msgr2=0x7f546019f570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:23:06.149 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:06.147+0000 7f5465800700 1 --2- 172.21.15.47:0/1387670103 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5460102440 0x7f546019f570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:23:06.149 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:06.147+0000 7f5465800700 1 -- 172.21.15.47:0/1387670103 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5460102df0 msgr2=0x7f546019fab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:23:06.149 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:06.147+0000 7f5465800700 1 --2- 172.21.15.47:0/1387670103 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5460102df0 0x7f546019fab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:23:06.149 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:06.147+0000 7f5465800700 1 -- 172.21.15.47:0/1387670103 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f546010ebc0 con 0x7f5460107510 2024-03-21T10:23:06.149 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:06.147+0000 7f5465800700 1 --2- 172.21.15.47:0/1387670103 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5460107510 0x7f546010e2f0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f545800efc0 tx=0x7f545800c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:23:06.149 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:06.148+0000 7f545d600700 1 -- 172.21.15.47:0/1387670103 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54580092e0 con 0x7f5460107510 2024-03-21T10:23:06.149 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:06.148+0000 7f545d600700 1 -- 172.21.15.47:0/1387670103 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f5458004bb0 con 0x7f5460107510 2024-03-21T10:23:06.149 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:06.148+0000 7f545d600700 1 -- 172.21.15.47:0/1387670103 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5458004280 con 0x7f5460107510 2024-03-21T10:23:06.150 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:06.148+0000 7f5467800700 1 -- 172.21.15.47:0/1387670103 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f546010ee50 con 0x7f5460107510 2024-03-21T10:23:06.150 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:06.148+0000 7f5467800700 1 -- 172.21.15.47:0/1387670103 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f546019b1f0 con 0x7f5460107510 2024-03-21T10:23:06.151 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:06.150+0000 7f5467800700 1 -- 172.21.15.47:0/1387670103 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f546019b720 con 0x7f5460107510 2024-03-21T10:23:06.157 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:06.156+0000 7f545d600700 1 -- 172.21.15.47:0/1387670103 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 17) v1 ==== 137905+0+0 (secure 0 0 0) 0x7f545801b030 con 0x7f5460107510 2024-03-21T10:23:06.160 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:06.158+0000 7f545d600700 1 --2- 172.21.15.47:0/1387670103 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f54480a3140 0x7f54480a5600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:23:06.161 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:06.158+0000 7f545d600700 1 -- 172.21.15.47:0/1387670103 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9092+0+0 (secure 0 0 0) 0x7f54580cf9f0 con 0x7f5460107510 2024-03-21T10:23:06.161 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:06.159+0000 7f5464e00700 1 --2- 172.21.15.47:0/1387670103 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f54480a3140 0x7f54480a5600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:23:06.161 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:06.160+0000 7f545d600700 1 --2- 172.21.15.47:0/1387670103 >> [v2:172.21.15.157:6824/2671029483,v1:172.21.15.157:6825/2671029483] conn(0x7f54480ae2a0 0x7f54480b0740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:23:06.161 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:06.160+0000 7f545d600700 1 -- 172.21.15.47:0/1387670103 --> [v2:172.21.15.157:6824/2671029483,v1:172.21.15.157:6825/2671029483] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f545801fe50 con 0x7f54480ae2a0 2024-03-21T10:23:06.162 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:06.160+0000 7f545d600700 1 -- 172.21.15.47:0/1387670103 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_get_version_reply(handle=1 version=93) v2 ==== 24+0+0 (secure 0 0 0) 0x7f54580cfde0 con 0x7f5460107510 2024-03-21T10:23:06.164 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:06.163+0000 7f5464e00700 1 --2- 172.21.15.47:0/1387670103 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f54480a3140 0x7f54480a5600 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f5460102120 tx=0x7f5454009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:23:06.165 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:06.163+0000 7f545fe00700 1 --2- 172.21.15.47:0/1387670103 >> [v2:172.21.15.157:6824/2671029483,v1:172.21.15.157:6825/2671029483] conn(0x7f54480ae2a0 0x7f54480b0740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:23:06.165 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:06.165+0000 7f545fe00700 1 --2- 172.21.15.47:0/1387670103 >> [v2:172.21.15.157:6824/2671029483,v1:172.21.15.157:6825/2671029483] conn(0x7f54480ae2a0 0x7f54480b0740 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.11 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:23:06.167 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:06.166+0000 7f545d600700 1 -- 172.21.15.47:0/1387670103 <== osd.11 v2:172.21.15.157:6824/2671029483 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7f545801fe50 con 0x7f54480ae2a0 2024-03-21T10:23:06.200 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:06.198+0000 7f5467800700 1 -- 172.21.15.47:0/1387670103 --> [v2:172.21.15.157:6824/2671029483,v1:172.21.15.157:6825/2671029483] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f546019bbf0 con 0x7f54480ae2a0 2024-03-21T10:23:06.200 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:06.199+0000 7f545d600700 1 -- 172.21.15.47:0/1387670103 <== osd.11 v2:172.21.15.157:6824/2671029483 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f546019bbf0 con 0x7f54480ae2a0 2024-03-21T10:23:06.202 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:06.200+0000 7f5467800700 1 -- 172.21.15.47:0/1387670103 >> [v2:172.21.15.157:6824/2671029483,v1:172.21.15.157:6825/2671029483] conn(0x7f54480ae2a0 msgr2=0x7f54480b0740 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:23:06.202 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:06.200+0000 7f5467800700 1 --2- 172.21.15.47:0/1387670103 >> [v2:172.21.15.157:6824/2671029483,v1:172.21.15.157:6825/2671029483] conn(0x7f54480ae2a0 0x7f54480b0740 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:23:06.202 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:06.200+0000 7f5467800700 1 -- 172.21.15.47:0/1387670103 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f54480a3140 msgr2=0x7f54480a5600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:23:06.202 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:06.200+0000 7f5467800700 1 --2- 172.21.15.47:0/1387670103 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f54480a3140 0x7f54480a5600 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f5460102120 tx=0x7f5454009260 comp rx=0 tx=0).stop 2024-03-21T10:23:06.203 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:06.200+0000 7f5467800700 1 -- 172.21.15.47:0/1387670103 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5460107510 msgr2=0x7f546010e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:23:06.203 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:06.200+0000 7f5467800700 1 --2- 172.21.15.47:0/1387670103 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5460107510 0x7f546010e2f0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f545800efc0 tx=0x7f545800c3a0 comp rx=0 tx=0).stop 2024-03-21T10:23:06.203 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:06.200+0000 7f5465800700 1 -- 172.21.15.47:0/1387670103 reap_dead start 2024-03-21T10:23:06.203 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:06.200+0000 7f5467800700 1 -- 172.21.15.47:0/1387670103 shutdown_connections 2024-03-21T10:23:06.204 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:06.200+0000 7f5467800700 1 -- 172.21.15.47:0/1387670103 >> 172.21.15.47:0/1387670103 conn(0x7f54600792f0 msgr2=0x7f5460100620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:23:06.204 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:06.201+0000 7f5467800700 1 -- 172.21.15.47:0/1387670103 shutdown_connections 2024-03-21T10:23:06.204 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:06.201+0000 7f5467800700 1 -- 172.21.15.47:0/1387670103 wait complete. 2024-03-21T10:23:06.229 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:23:05 smithi100 ceph-mon[35193]: pgmap v402: 129 pgs: 129 active+clean; 642 KiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-03-21T10:23:06.230 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:23:05 smithi100 ceph-mon[35193]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:23:06.230 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:23:05 smithi100 ceph-mon[35193]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:23:06.230 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:23:05 smithi100 ceph-mon[35193]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:23:06.230 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:23:05 smithi100 ceph-mon[35193]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:23:06.239 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:23:05 smithi157 ceph-mon[35581]: pgmap v402: 129 pgs: 129 active+clean; 642 KiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-03-21T10:23:06.240 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:23:05 smithi157 ceph-mon[35581]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:23:06.240 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:23:05 smithi157 ceph-mon[35581]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:23:06.240 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:23:05 smithi157 ceph-mon[35581]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:23:06.240 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:23:05 smithi157 ceph-mon[35581]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:23:06.462 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:06.461+0000 7f44d6000700 1 -- 172.21.15.47:0/3994733901 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f44d01081c0 msgr2=0x7f44d01085a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:23:06.462 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:06.461+0000 7f44d6000700 1 --2- 172.21.15.47:0/3994733901 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f44d01081c0 0x7f44d01085a0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f44c00099c0 tx=0x7f44c0009cd0 comp rx=0 tx=0).stop 2024-03-21T10:23:06.463 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:06.461+0000 7f44d6000700 1 -- 172.21.15.47:0/3994733901 shutdown_connections 2024-03-21T10:23:06.463 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:06.461+0000 7f44d6000700 1 --2- 172.21.15.47:0/3994733901 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f44d0102ba0 0x7f44d010efe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:23:06.463 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:06.461+0000 7f44d6000700 1 --2- 172.21.15.47:0/3994733901 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f44d01081c0 0x7f44d01085a0 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:23:06.463 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:06.461+0000 7f44d6000700 1 --2- 172.21.15.47:0/3994733901 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f44d01021e0 0x7f44d0102660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:23:06.463 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:06.461+0000 7f44d6000700 1 -- 172.21.15.47:0/3994733901 >> 172.21.15.47:0/3994733901 conn(0x7f44d00792c0 msgr2=0x7f44d00796d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:23:06.463 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:06.461+0000 7f44d6000700 1 -- 172.21.15.47:0/3994733901 shutdown_connections 2024-03-21T10:23:06.463 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:06.461+0000 7f44d6000700 1 -- 172.21.15.47:0/3994733901 wait complete. 2024-03-21T10:23:06.463 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:06.463+0000 7f44d6000700 1 Processor -- start 2024-03-21T10:23:06.464 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:06.463+0000 7f44d6000700 1 -- start start 2024-03-21T10:23:06.464 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:06.463+0000 7f44d6000700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f44d01021e0 0x7f44d0074490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:23:06.465 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:06.463+0000 7f44d6000700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f44d0102ba0 0x7f44d00749d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:23:06.465 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:06.464+0000 7f44d6000700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f44d01081c0 0x7f44d0074f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:23:06.465 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:06.464+0000 7f44d6000700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44d0112340 con 0x7f44d0102ba0 2024-03-21T10:23:06.465 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:06.464+0000 7f44d6000700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44d01121c0 con 0x7f44d01021e0 2024-03-21T10:23:06.465 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:06.464+0000 7f44d6000700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44d01124c0 con 0x7f44d01081c0 2024-03-21T10:23:06.466 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:06.464+0000 7f44cf400700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f44d01021e0 0x7f44d0074490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:23:06.466 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:06.464+0000 7f44cf400700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f44d01021e0 0x7f44d0074490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:50816/0 (socket says 172.21.15.47:50816) 2024-03-21T10:23:06.466 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:06.464+0000 7f44cf400700 1 -- 172.21.15.47:0/652157262 learned_addr learned my addr 172.21.15.47:0/652157262 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:23:06.466 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:06.464+0000 7f44cea00700 1 --2- 172.21.15.47:0/652157262 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f44d0102ba0 0x7f44d00749d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:23:06.466 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:06.464+0000 7f44cfe00700 1 --2- 172.21.15.47:0/652157262 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f44d01081c0 0x7f44d0074f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:23:06.466 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:06.464+0000 7f44cf400700 1 -- 172.21.15.47:0/652157262 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f44d01081c0 msgr2=0x7f44d0074f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:23:06.466 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:06.464+0000 7f44cf400700 1 --2- 172.21.15.47:0/652157262 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f44d01081c0 0x7f44d0074f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:23:06.466 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:06.464+0000 7f44cf400700 1 -- 172.21.15.47:0/652157262 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f44d0102ba0 msgr2=0x7f44d00749d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:23:06.466 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:06.464+0000 7f44cf400700 1 --2- 172.21.15.47:0/652157262 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f44d0102ba0 0x7f44d00749d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:23:06.467 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:06.464+0000 7f44cf400700 1 -- 172.21.15.47:0/652157262 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f44d01a6cc0 con 0x7f44d01021e0 2024-03-21T10:23:06.467 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:06.465+0000 7f44cea00700 1 --2- 172.21.15.47:0/652157262 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f44d0102ba0 0x7f44d00749d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T10:23:06.467 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:06.465+0000 7f44cf400700 1 --2- 172.21.15.47:0/652157262 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f44d01021e0 0x7f44d0074490 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f44c0009990 tx=0x7f44c0009f80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:23:06.467 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:06.465+0000 7f44c7e00700 1 -- 172.21.15.47:0/652157262 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44c0027070 con 0x7f44d01021e0 2024-03-21T10:23:06.467 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:06.465+0000 7f44d6000700 1 -- 172.21.15.47:0/652157262 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f44d01a6f50 con 0x7f44d01021e0 2024-03-21T10:23:06.467 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:06.465+0000 7f44c7e00700 1 -- 172.21.15.47:0/652157262 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f44c0003fd0 con 0x7f44d01021e0 2024-03-21T10:23:06.468 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:06.465+0000 7f44c7e00700 1 -- 172.21.15.47:0/652157262 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44c0022c70 con 0x7f44d01021e0 2024-03-21T10:23:06.468 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:06.466+0000 7f44d6000700 1 -- 172.21.15.47:0/652157262 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f44d01a73a0 con 0x7f44d01021e0 2024-03-21T10:23:06.468 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:06.467+0000 7f44d6000700 1 -- 172.21.15.47:0/652157262 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f44d00628e0 con 0x7f44d01021e0 2024-03-21T10:23:06.470 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:06.469+0000 7f44c7e00700 1 -- 172.21.15.47:0/652157262 <== mon.2 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 17) v1 ==== 137905+0+0 (secure 0 0 0) 0x7f44c002a070 con 0x7f44d01021e0 2024-03-21T10:23:06.471 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:06.470+0000 7f44c7e00700 1 --2- 172.21.15.47:0/652157262 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f44b40a3040 0x7f44b40a5500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:23:06.472 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:06.470+0000 7f44c7e00700 1 -- 172.21.15.47:0/652157262 <== mon.2 v2:172.21.15.100:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9092+0+0 (secure 0 0 0) 0x7f44c00d1280 con 0x7f44d01021e0 2024-03-21T10:23:06.472 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:06.471+0000 7f44cea00700 1 --2- 172.21.15.47:0/652157262 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f44b40a3040 0x7f44b40a5500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:23:06.472 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:06.471+0000 7f44c7e00700 1 --2- 172.21.15.47:0/652157262 >> [v2:172.21.15.157:6808/1525645410,v1:172.21.15.157:6809/1525645410] conn(0x7f44b40ae1a0 0x7f44b40b0640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:23:06.472 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:06.471+0000 7f44c7e00700 1 -- 172.21.15.47:0/652157262 --> [v2:172.21.15.157:6808/1525645410,v1:172.21.15.157:6809/1525645410] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f44c0024ec0 con 0x7f44b40ae1a0 2024-03-21T10:23:06.472 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:06.471+0000 7f44c7e00700 1 -- 172.21.15.47:0/652157262 <== mon.2 v2:172.21.15.100:3300/0 6 ==== mon_get_version_reply(handle=1 version=93) v2 ==== 24+0+0 (secure 0 0 0) 0x7f44c00d1630 con 0x7f44d01021e0 2024-03-21T10:23:06.473 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:06.471+0000 7f44cfe00700 1 --2- 172.21.15.47:0/652157262 >> [v2:172.21.15.157:6808/1525645410,v1:172.21.15.157:6809/1525645410] conn(0x7f44b40ae1a0 0x7f44b40b0640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:23:06.473 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:06.472+0000 7f44cea00700 1 --2- 172.21.15.47:0/652157262 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f44b40a3040 0x7f44b40a5500 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f44bc005fd0 tx=0x7f44bc005e30 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:23:06.473 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:06.472+0000 7f44cfe00700 1 --2- 172.21.15.47:0/652157262 >> [v2:172.21.15.157:6808/1525645410,v1:172.21.15.157:6809/1525645410] conn(0x7f44b40ae1a0 0x7f44b40b0640 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:23:06.475 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:06.474+0000 7f44c7e00700 1 -- 172.21.15.47:0/652157262 <== osd.5 v2:172.21.15.157:6808/1525645410 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7f44c0024ec0 con 0x7f44b40ae1a0 2024-03-21T10:23:06.494 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:06.493+0000 7f44d6000700 1 -- 172.21.15.47:0/652157262 --> [v2:172.21.15.157:6808/1525645410,v1:172.21.15.157:6809/1525645410] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f44d0062d60 con 0x7f44b40ae1a0 2024-03-21T10:23:06.495 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:06.494+0000 7f44c7e00700 1 -- 172.21.15.47:0/652157262 <== osd.5 v2:172.21.15.157:6808/1525645410 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f44d0062d60 con 0x7f44b40ae1a0 2024-03-21T10:23:06.496 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:06.494+0000 7f44d6000700 1 -- 172.21.15.47:0/652157262 >> [v2:172.21.15.157:6808/1525645410,v1:172.21.15.157:6809/1525645410] conn(0x7f44b40ae1a0 msgr2=0x7f44b40b0640 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:23:06.496 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:06.494+0000 7f44d6000700 1 --2- 172.21.15.47:0/652157262 >> [v2:172.21.15.157:6808/1525645410,v1:172.21.15.157:6809/1525645410] conn(0x7f44b40ae1a0 0x7f44b40b0640 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:23:06.496 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:06.495+0000 7f44d6000700 1 -- 172.21.15.47:0/652157262 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f44b40a3040 msgr2=0x7f44b40a5500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:23:06.496 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:06.495+0000 7f44d6000700 1 --2- 172.21.15.47:0/652157262 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f44b40a3040 0x7f44b40a5500 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f44bc005fd0 tx=0x7f44bc005e30 comp rx=0 tx=0).stop 2024-03-21T10:23:06.496 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:06.495+0000 7f44d6000700 1 -- 172.21.15.47:0/652157262 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f44d01021e0 msgr2=0x7f44d0074490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:23:06.496 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:06.495+0000 7f44d6000700 1 --2- 172.21.15.47:0/652157262 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f44d01021e0 0x7f44d0074490 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f44c0009990 tx=0x7f44c0009f80 comp rx=0 tx=0).stop 2024-03-21T10:23:06.496 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:06.495+0000 7f44cfe00700 1 -- 172.21.15.47:0/652157262 reap_dead start 2024-03-21T10:23:06.496 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:06.495+0000 7f44d6000700 1 -- 172.21.15.47:0/652157262 shutdown_connections 2024-03-21T10:23:06.496 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:06.495+0000 7f44d6000700 1 -- 172.21.15.47:0/652157262 >> 172.21.15.47:0/652157262 conn(0x7f44d00792c0 msgr2=0x7f44d0100e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:23:06.497 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:06.495+0000 7f44d6000700 1 -- 172.21.15.47:0/652157262 shutdown_connections 2024-03-21T10:23:06.497 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:06.495+0000 7f44d6000700 1 -- 172.21.15.47:0/652157262 wait complete. 2024-03-21T10:23:07.153 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:05.318+0000 7f8f2dc00700 1 -- 172.21.15.47:0/1450464923 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8f28102d80 msgr2=0x7f8f28106dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:23:07.154 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:05.318+0000 7f8f2dc00700 1 --2- 172.21.15.47:0/1450464923 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8f28102d80 0x7f8f28106dd0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f8f140098f0 tx=0x7f8f14009c00 comp rx=0 tx=0).stop 2024-03-21T10:23:07.154 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:05.318+0000 7f8f2dc00700 1 -- 172.21.15.47:0/1450464923 shutdown_connections 2024-03-21T10:23:07.154 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:05.318+0000 7f8f2dc00700 1 --2- 172.21.15.47:0/1450464923 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8f281074b0 0x7f8f2810f080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:23:07.154 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:05.318+0000 7f8f2dc00700 1 --2- 172.21.15.47:0/1450464923 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8f281023d0 0x7f8f281027b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:23:07.154 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:05.318+0000 7f8f2dc00700 1 --2- 172.21.15.47:0/1450464923 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8f28102d80 0x7f8f28106dd0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:23:07.154 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:05.318+0000 7f8f2dc00700 1 -- 172.21.15.47:0/1450464923 >> 172.21.15.47:0/1450464923 conn(0x7f8f280fdc40 msgr2=0x7f8f28100060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:23:07.154 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:05.318+0000 7f8f2dc00700 1 -- 172.21.15.47:0/1450464923 shutdown_connections 2024-03-21T10:23:07.154 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:05.318+0000 7f8f2dc00700 1 -- 172.21.15.47:0/1450464923 wait complete. 2024-03-21T10:23:07.154 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:05.319+0000 7f8f2dc00700 1 Processor -- start 2024-03-21T10:23:07.154 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:05.320+0000 7f8f2dc00700 1 -- start start 2024-03-21T10:23:07.154 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:05.320+0000 7f8f2dc00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8f281023d0 0x7f8f281a0750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:23:07.154 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:05.320+0000 7f8f2dc00700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8f28102d80 0x7f8f281a0c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:23:07.155 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:05.320+0000 7f8f2dc00700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8f281074b0 0x7f8f2819a910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:23:07.155 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:05.320+0000 7f8f2dc00700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f28112370 con 0x7f8f281074b0 2024-03-21T10:23:07.155 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:05.320+0000 7f8f2dc00700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f281121f0 con 0x7f8f28102d80 2024-03-21T10:23:07.155 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:05.320+0000 7f8f2dc00700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f281124f0 con 0x7f8f281023d0 2024-03-21T10:23:07.155 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:05.320+0000 7f8f26000700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8f28102d80 0x7f8f281a0c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:23:07.155 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:05.321+0000 7f8f26000700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8f28102d80 0x7f8f281a0c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:50796/0 (socket says 172.21.15.47:50796) 2024-03-21T10:23:07.155 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:05.321+0000 7f8f27400700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8f281074b0 0x7f8f2819a910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:23:07.155 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:05.321+0000 7f8f26000700 1 -- 172.21.15.47:0/4219372473 learned_addr learned my addr 172.21.15.47:0/4219372473 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:23:07.155 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:05.321+0000 7f8f26a00700 1 --2- 172.21.15.47:0/4219372473 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8f281023d0 0x7f8f281a0750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:23:07.155 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:05.321+0000 7f8f26000700 1 -- 172.21.15.47:0/4219372473 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8f281023d0 msgr2=0x7f8f281a0750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:23:07.155 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:05.321+0000 7f8f26000700 1 --2- 172.21.15.47:0/4219372473 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8f281023d0 0x7f8f281a0750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:23:07.155 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:05.321+0000 7f8f26000700 1 -- 172.21.15.47:0/4219372473 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8f281074b0 msgr2=0x7f8f2819a910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:23:07.156 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:05.321+0000 7f8f26000700 1 --2- 172.21.15.47:0/4219372473 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8f281074b0 0x7f8f2819a910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:23:07.156 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:05.321+0000 7f8f26000700 1 -- 172.21.15.47:0/4219372473 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f2819b070 con 0x7f8f28102d80 2024-03-21T10:23:07.156 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:05.322+0000 7f8f26000700 1 --2- 172.21.15.47:0/4219372473 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8f28102d80 0x7f8f281a0c90 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f8f1400c010 tx=0x7f8f1400f9b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:23:07.156 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:05.322+0000 7f8f1f400700 1 -- 172.21.15.47:0/4219372473 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f14027070 con 0x7f8f28102d80 2024-03-21T10:23:07.156 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:05.322+0000 7f8f2dc00700 1 -- 172.21.15.47:0/4219372473 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8f2819b300 con 0x7f8f28102d80 2024-03-21T10:23:07.156 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:05.322+0000 7f8f1f400700 1 -- 172.21.15.47:0/4219372473 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f8f140054f0 con 0x7f8f28102d80 2024-03-21T10:23:07.156 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:05.322+0000 7f8f1f400700 1 -- 172.21.15.47:0/4219372473 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f14004930 con 0x7f8f28102d80 2024-03-21T10:23:07.156 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:05.322+0000 7f8f2dc00700 1 -- 172.21.15.47:0/4219372473 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8f281a7620 con 0x7f8f28102d80 2024-03-21T10:23:07.156 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:05.323+0000 7f8f2dc00700 1 -- 172.21.15.47:0/4219372473 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f8f28075720 con 0x7f8f28102d80 2024-03-21T10:23:07.156 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:05.325+0000 7f8f1f400700 1 -- 172.21.15.47:0/4219372473 <== mon.2 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 17) v1 ==== 137905+0+0 (secure 0 0 0) 0x7f8f1402a070 con 0x7f8f28102d80 2024-03-21T10:23:07.156 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:05.327+0000 7f8f1f400700 1 --2- 172.21.15.47:0/4219372473 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f8f0c0a3040 0x7f8f0c0a5500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:23:07.156 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:05.327+0000 7f8f1f400700 1 -- 172.21.15.47:0/4219372473 <== mon.2 v2:172.21.15.100:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9092+0+0 (secure 0 0 0) 0x7f8f140d0960 con 0x7f8f28102d80 2024-03-21T10:23:07.156 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:05.327+0000 7f8f26a00700 1 --2- 172.21.15.47:0/4219372473 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f8f0c0a3040 0x7f8f0c0a5500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:23:07.156 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:05.327+0000 7f8f1f400700 1 --2- 172.21.15.47:0/4219372473 >> [v2:172.21.15.47:6802/775837313,v1:172.21.15.47:6803/775837313] conn(0x7f8f0c0ae1a0 0x7f8f0c0b0640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:23:07.157 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:05.327+0000 7f8f1f400700 1 -- 172.21.15.47:0/4219372473 --> [v2:172.21.15.47:6802/775837313,v1:172.21.15.47:6803/775837313] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f8f0c0b0d10 con 0x7f8f0c0ae1a0 2024-03-21T10:23:07.157 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:05.327+0000 7f8f1f400700 1 -- 172.21.15.47:0/4219372473 <== mon.2 v2:172.21.15.100:3300/0 6 ==== mon_get_version_reply(handle=1 version=93) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8f140d0d10 con 0x7f8f28102d80 2024-03-21T10:23:07.157 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:05.328+0000 7f8f27400700 1 --2- 172.21.15.47:0/4219372473 >> [v2:172.21.15.47:6802/775837313,v1:172.21.15.47:6803/775837313] conn(0x7f8f0c0ae1a0 0x7f8f0c0b0640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:23:07.157 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:05.328+0000 7f8f26a00700 1 --2- 172.21.15.47:0/4219372473 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f8f0c0a3040 0x7f8f0c0a5500 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f8f2819ba50 tx=0x7f8f18009310 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:23:07.157 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:05.328+0000 7f8f27400700 1 --2- 172.21.15.47:0/4219372473 >> [v2:172.21.15.47:6802/775837313,v1:172.21.15.47:6803/775837313] conn(0x7f8f0c0ae1a0 0x7f8f0c0b0640 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:23:07.157 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:05.329+0000 7f8f1f400700 1 -- 172.21.15.47:0/4219372473 <== osd.0 v2:172.21.15.47:6802/775837313 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7f8f0c0b0d10 con 0x7f8f0c0ae1a0 2024-03-21T10:23:07.157 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:05.356+0000 7f8f2dc00700 1 -- 172.21.15.47:0/4219372473 --> [v2:172.21.15.47:6802/775837313,v1:172.21.15.47:6803/775837313] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f8f281038b0 con 0x7f8f0c0ae1a0 2024-03-21T10:23:07.158 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:05.356+0000 7f8f1f400700 1 -- 172.21.15.47:0/4219372473 <== osd.0 v2:172.21.15.47:6802/775837313 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f8f281038b0 con 0x7f8f0c0ae1a0 2024-03-21T10:23:07.158 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:05.357+0000 7f8f2dc00700 1 -- 172.21.15.47:0/4219372473 >> [v2:172.21.15.47:6802/775837313,v1:172.21.15.47:6803/775837313] conn(0x7f8f0c0ae1a0 msgr2=0x7f8f0c0b0640 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:23:07.158 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:05.357+0000 7f8f2dc00700 1 --2- 172.21.15.47:0/4219372473 >> [v2:172.21.15.47:6802/775837313,v1:172.21.15.47:6803/775837313] conn(0x7f8f0c0ae1a0 0x7f8f0c0b0640 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:23:07.158 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:05.357+0000 7f8f2dc00700 1 -- 172.21.15.47:0/4219372473 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f8f0c0a3040 msgr2=0x7f8f0c0a5500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:23:07.158 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:05.357+0000 7f8f2dc00700 1 --2- 172.21.15.47:0/4219372473 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f8f0c0a3040 0x7f8f0c0a5500 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f8f2819ba50 tx=0x7f8f18009310 comp rx=0 tx=0).stop 2024-03-21T10:23:07.158 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:05.357+0000 7f8f2dc00700 1 -- 172.21.15.47:0/4219372473 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8f28102d80 msgr2=0x7f8f281a0c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:23:07.158 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:05.357+0000 7f8f2dc00700 1 --2- 172.21.15.47:0/4219372473 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8f28102d80 0x7f8f281a0c90 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f8f1400c010 tx=0x7f8f1400f9b0 comp rx=0 tx=0).stop 2024-03-21T10:23:07.158 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:05.357+0000 7f8f27400700 1 -- 172.21.15.47:0/4219372473 reap_dead start 2024-03-21T10:23:07.158 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:05.357+0000 7f8f2dc00700 1 -- 172.21.15.47:0/4219372473 shutdown_connections 2024-03-21T10:23:07.158 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:05.357+0000 7f8f2dc00700 1 -- 172.21.15.47:0/4219372473 >> 172.21.15.47:0/4219372473 conn(0x7f8f280fdc40 msgr2=0x7f8f280fff80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:23:07.158 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:05.358+0000 7f8f2dc00700 1 -- 172.21.15.47:0/4219372473 shutdown_connections 2024-03-21T10:23:07.158 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:05.358+0000 7f8f2dc00700 1 -- 172.21.15.47:0/4219372473 wait complete. 2024-03-21T10:23:07.463 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:23:07 smithi047 ceph-mon[31547]: pgmap v403: 129 pgs: 129 active+clean; 642 KiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-21T10:23:07.464 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:23:07 smithi047 ceph-mon[31547]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:23:07.464 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:23:07 smithi047 ceph-mon[31547]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:23:07.729 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:23:07 smithi100 ceph-mon[35193]: pgmap v403: 129 pgs: 129 active+clean; 642 KiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-21T10:23:07.729 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:23:07 smithi100 ceph-mon[35193]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:23:07.729 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:23:07 smithi100 ceph-mon[35193]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:23:07.740 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:23:07 smithi157 ceph-mon[35581]: pgmap v403: 129 pgs: 129 active+clean; 642 KiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-21T10:23:07.740 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:23:07 smithi157 ceph-mon[35581]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:23:07.740 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:23:07 smithi157 ceph-mon[35581]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:23:08.839 INFO:teuthology.orchestra.run.smithi047.stdout:34359738473 2024-03-21T10:23:08.839 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2edfee68ad7243bed422a049b6f5f58619300f11 shell --fsid 3056d51a-e76b-11ee-95cd-87774f69a715 -- ceph osd last-stat-seq osd.0 2024-03-21T10:23:09.398 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:23:09 smithi047 ceph-mon[31547]: pgmap v404: 129 pgs: 129 active+clean; 642 KiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 9 op/s 2024-03-21T10:23:09.463 INFO:teuthology.orchestra.run.smithi047.stdout:77309411421 2024-03-21T10:23:09.463 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2edfee68ad7243bed422a049b6f5f58619300f11 shell --fsid 3056d51a-e76b-11ee-95cd-87774f69a715 -- ceph osd last-stat-seq osd.2 2024-03-21T10:23:09.729 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:23:09 smithi100 ceph-mon[35193]: pgmap v404: 129 pgs: 129 active+clean; 642 KiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 9 op/s 2024-03-21T10:23:09.739 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:23:09 smithi157 ceph-mon[35581]: pgmap v404: 129 pgs: 129 active+clean; 642 KiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 9 op/s 2024-03-21T10:23:11.238 INFO:teuthology.orchestra.run.smithi047.stdout:352187318301 2024-03-21T10:23:11.239 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2edfee68ad7243bed422a049b6f5f58619300f11 shell --fsid 3056d51a-e76b-11ee-95cd-87774f69a715 -- ceph osd last-stat-seq osd.11 2024-03-21T10:23:11.330 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:23:11 smithi047 ceph-mon[31547]: pgmap v405: 129 pgs: 129 active+clean; 642 KiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 11 op/s 2024-03-21T10:23:11.564 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:23:11 smithi157 ceph-mon[35581]: pgmap v405: 129 pgs: 129 active+clean; 642 KiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 11 op/s 2024-03-21T10:23:11.729 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:23:11 smithi100 ceph-mon[35193]: pgmap v405: 129 pgs: 129 active+clean; 642 KiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 11 op/s 2024-03-21T10:23:12.081 INFO:teuthology.orchestra.run.smithi047.stdout:150323855438 2024-03-21T10:23:12.081 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2edfee68ad7243bed422a049b6f5f58619300f11 shell --fsid 3056d51a-e76b-11ee-95cd-87774f69a715 -- ceph osd last-stat-seq osd.4 2024-03-21T10:23:12.235 INFO:teuthology.orchestra.run.smithi047.stdout:171798691911 2024-03-21T10:23:12.235 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2edfee68ad7243bed422a049b6f5f58619300f11 shell --fsid 3056d51a-e76b-11ee-95cd-87774f69a715 -- ceph osd last-stat-seq osd.5 2024-03-21T10:23:13.490 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:23:13 smithi157 ceph-mon[35581]: pgmap v406: 129 pgs: 129 active+clean; 642 KiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-03-21T10:23:13.570 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:23:13 smithi047 ceph-mon[31547]: pgmap v406: 129 pgs: 129 active+clean; 642 KiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-03-21T10:23:13.729 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:23:13 smithi100 ceph-mon[35193]: pgmap v406: 129 pgs: 129 active+clean; 642 KiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-03-21T10:23:13.843 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:09.710+0000 7f98c3c00700 1 -- 172.21.15.47:0/941760075 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f98bc102ba0 msgr2=0x7f98bc10efe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:23:13.843 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:09.710+0000 7f98c3c00700 1 --2- 172.21.15.47:0/941760075 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f98bc102ba0 0x7f98bc10efe0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f98b400b0d0 tx=0x7f98b400b3e0 comp rx=0 tx=0).stop 2024-03-21T10:23:13.843 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:09.710+0000 7f98c3c00700 1 -- 172.21.15.47:0/941760075 shutdown_connections 2024-03-21T10:23:13.843 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:09.710+0000 7f98c3c00700 1 --2- 172.21.15.47:0/941760075 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f98bc102ba0 0x7f98bc10efe0 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:23:13.843 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:09.710+0000 7f98c3c00700 1 --2- 172.21.15.47:0/941760075 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f98bc1081c0 0x7f98bc1085a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:23:13.843 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:09.710+0000 7f98c3c00700 1 --2- 172.21.15.47:0/941760075 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f98bc1021e0 0x7f98bc102660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:23:13.844 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:09.710+0000 7f98c3c00700 1 -- 172.21.15.47:0/941760075 >> 172.21.15.47:0/941760075 conn(0x7f98bc0792c0 msgr2=0x7f98bc0796d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:23:13.844 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:09.710+0000 7f98c3c00700 1 -- 172.21.15.47:0/941760075 shutdown_connections 2024-03-21T10:23:13.844 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:09.710+0000 7f98c3c00700 1 -- 172.21.15.47:0/941760075 wait complete. 2024-03-21T10:23:13.844 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:09.711+0000 7f98c3c00700 1 Processor -- start 2024-03-21T10:23:13.844 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:09.711+0000 7f98c3c00700 1 -- start start 2024-03-21T10:23:13.844 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:09.712+0000 7f98c3c00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f98bc1021e0 0x7f98bc1a0660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:23:13.844 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:09.712+0000 7f98c3c00700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f98bc102ba0 0x7f98bc1a0ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:23:13.844 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:09.712+0000 7f98c3c00700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f98bc1081c0 0x7f98bc19a7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:23:13.844 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:09.712+0000 7f98c3c00700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98bc112370 con 0x7f98bc1081c0 2024-03-21T10:23:13.844 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:09.712+0000 7f98c3c00700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98bc1121f0 con 0x7f98bc102ba0 2024-03-21T10:23:13.844 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:09.712+0000 7f98c3c00700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98bc1124f0 con 0x7f98bc1021e0 2024-03-21T10:23:13.844 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:09.712+0000 7f98c1c00700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f98bc1081c0 0x7f98bc19a7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:23:13.844 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:09.712+0000 7f98bbe00700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f98bc102ba0 0x7f98bc1a0ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:23:13.845 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:09.712+0000 7f98c1c00700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f98bc1081c0 0x7f98bc19a7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:54946/0 (socket says 172.21.15.47:54946) 2024-03-21T10:23:13.845 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:09.712+0000 7f98c1c00700 1 -- 172.21.15.47:0/836623154 learned_addr learned my addr 172.21.15.47:0/836623154 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:23:13.845 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:09.712+0000 7f98c1200700 1 --2- 172.21.15.47:0/836623154 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f98bc1021e0 0x7f98bc1a0660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:23:13.845 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:09.713+0000 7f98c1c00700 1 -- 172.21.15.47:0/836623154 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f98bc1021e0 msgr2=0x7f98bc1a0660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:23:13.845 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:09.713+0000 7f98c1c00700 1 --2- 172.21.15.47:0/836623154 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f98bc1021e0 0x7f98bc1a0660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:23:13.845 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:09.713+0000 7f98c1c00700 1 -- 172.21.15.47:0/836623154 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f98bc102ba0 msgr2=0x7f98bc1a0ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:23:13.845 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:09.713+0000 7f98c1c00700 1 --2- 172.21.15.47:0/836623154 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f98bc102ba0 0x7f98bc1a0ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:23:13.845 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:09.713+0000 7f98c1c00700 1 -- 172.21.15.47:0/836623154 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98bc19afc0 con 0x7f98bc1081c0 2024-03-21T10:23:13.845 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:09.713+0000 7f98c1200700 1 --2- 172.21.15.47:0/836623154 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f98bc1021e0 0x7f98bc1a0660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T10:23:13.845 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:09.713+0000 7f98bbe00700 1 --2- 172.21.15.47:0/836623154 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f98bc102ba0 0x7f98bc1a0ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T10:23:13.845 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:09.713+0000 7f98c1c00700 1 --2- 172.21.15.47:0/836623154 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f98bc1081c0 0x7f98bc19a7e0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f98b40079a0 tx=0x7f98b40079d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:23:13.845 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:09.713+0000 7f98b9600700 1 -- 172.21.15.47:0/836623154 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98b401f070 con 0x7f98bc1081c0 2024-03-21T10:23:13.845 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:09.713+0000 7f98b9600700 1 -- 172.21.15.47:0/836623154 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f98b4003c00 con 0x7f98bc1081c0 2024-03-21T10:23:13.846 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:09.713+0000 7f98b9600700 1 -- 172.21.15.47:0/836623154 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98b401b690 con 0x7f98bc1081c0 2024-03-21T10:23:13.846 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:09.713+0000 7f98c3c00700 1 -- 172.21.15.47:0/836623154 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f98bc19b250 con 0x7f98bc1081c0 2024-03-21T10:23:13.846 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:09.714+0000 7f98c3c00700 1 -- 172.21.15.47:0/836623154 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f98bc1a74c0 con 0x7f98bc1081c0 2024-03-21T10:23:13.846 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:09.715+0000 7f98c3c00700 1 -- 172.21.15.47:0/836623154 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f98bc076080 con 0x7f98bc1081c0 2024-03-21T10:23:13.846 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:09.718+0000 7f98b9600700 1 -- 172.21.15.47:0/836623154 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 17) v1 ==== 137905+0+0 (secure 0 0 0) 0x7f98b4022070 con 0x7f98bc1081c0 2024-03-21T10:23:13.846 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:09.720+0000 7f98b9600700 1 --2- 172.21.15.47:0/836623154 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f98a40a30c0 0x7f98a40a5580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:23:13.846 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:09.720+0000 7f98b9600700 1 -- 172.21.15.47:0/836623154 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9092+0+0 (secure 0 0 0) 0x7f98b400f020 con 0x7f98bc1081c0 2024-03-21T10:23:13.846 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:09.720+0000 7f98c1200700 1 --2- 172.21.15.47:0/836623154 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f98a40a30c0 0x7f98a40a5580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:23:13.846 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:09.721+0000 7f98b9600700 1 --2- 172.21.15.47:0/836623154 >> [v2:172.21.15.47:6810/640760101,v1:172.21.15.47:6811/640760101] conn(0x7f98a40ae0f0 0x7f98a40b0590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:23:13.846 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:09.721+0000 7f98b9600700 1 -- 172.21.15.47:0/836623154 --> [v2:172.21.15.47:6810/640760101,v1:172.21.15.47:6811/640760101] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f98b40020b0 con 0x7f98a40ae0f0 2024-03-21T10:23:13.846 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:09.721+0000 7f98b9600700 1 -- 172.21.15.47:0/836623154 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_get_version_reply(handle=1 version=93) v2 ==== 24+0+0 (secure 0 0 0) 0x7f98b40d1040 con 0x7f98bc1081c0 2024-03-21T10:23:13.846 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:09.721+0000 7f98bbe00700 1 --2- 172.21.15.47:0/836623154 >> [v2:172.21.15.47:6810/640760101,v1:172.21.15.47:6811/640760101] conn(0x7f98a40ae0f0 0x7f98a40b0590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:23:13.846 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:09.721+0000 7f98c1200700 1 --2- 172.21.15.47:0/836623154 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f98a40a30c0 0x7f98a40a5580 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f98b0009cb0 tx=0x7f98b0009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:23:13.847 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:09.722+0000 7f98bbe00700 1 --2- 172.21.15.47:0/836623154 >> [v2:172.21.15.47:6810/640760101,v1:172.21.15.47:6811/640760101] conn(0x7f98a40ae0f0 0x7f98a40b0590 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:23:13.847 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:09.723+0000 7f98b9600700 1 -- 172.21.15.47:0/836623154 <== osd.3 v2:172.21.15.47:6810/640760101 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7f98b40020b0 con 0x7f98a40ae0f0 2024-03-21T10:23:13.847 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:09.746+0000 7f98c3c00700 1 -- 172.21.15.47:0/836623154 --> [v2:172.21.15.47:6810/640760101,v1:172.21.15.47:6811/640760101] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f98bc103870 con 0x7f98a40ae0f0 2024-03-21T10:23:13.847 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:09.746+0000 7f98b9600700 1 -- 172.21.15.47:0/836623154 <== osd.3 v2:172.21.15.47:6810/640760101 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f98bc103870 con 0x7f98a40ae0f0 2024-03-21T10:23:13.847 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:09.747+0000 7f98c3c00700 1 -- 172.21.15.47:0/836623154 >> [v2:172.21.15.47:6810/640760101,v1:172.21.15.47:6811/640760101] conn(0x7f98a40ae0f0 msgr2=0x7f98a40b0590 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:23:13.848 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:09.747+0000 7f98c3c00700 1 --2- 172.21.15.47:0/836623154 >> [v2:172.21.15.47:6810/640760101,v1:172.21.15.47:6811/640760101] conn(0x7f98a40ae0f0 0x7f98a40b0590 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:23:13.848 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:09.747+0000 7f98c3c00700 1 -- 172.21.15.47:0/836623154 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f98a40a30c0 msgr2=0x7f98a40a5580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:23:13.848 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:09.747+0000 7f98c3c00700 1 --2- 172.21.15.47:0/836623154 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f98a40a30c0 0x7f98a40a5580 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f98b0009cb0 tx=0x7f98b0009260 comp rx=0 tx=0).stop 2024-03-21T10:23:13.848 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:09.747+0000 7f98c3c00700 1 -- 172.21.15.47:0/836623154 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f98bc1081c0 msgr2=0x7f98bc19a7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:23:13.848 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:09.747+0000 7f98c3c00700 1 --2- 172.21.15.47:0/836623154 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f98bc1081c0 0x7f98bc19a7e0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f98b40079a0 tx=0x7f98b40079d0 comp rx=0 tx=0).stop 2024-03-21T10:23:13.848 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:09.747+0000 7f98c1c00700 1 -- 172.21.15.47:0/836623154 reap_dead start 2024-03-21T10:23:13.848 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:09.748+0000 7f98c3c00700 1 -- 172.21.15.47:0/836623154 shutdown_connections 2024-03-21T10:23:13.848 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:09.748+0000 7f98c3c00700 1 -- 172.21.15.47:0/836623154 >> 172.21.15.47:0/836623154 conn(0x7f98bc0792c0 msgr2=0x7f98bc074650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:23:13.848 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:09.748+0000 7f98c3c00700 1 -- 172.21.15.47:0/836623154 shutdown_connections 2024-03-21T10:23:13.848 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:09.748+0000 7f98c3c00700 1 -- 172.21.15.47:0/836623154 wait complete. 2024-03-21T10:23:14.622 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:11.268+0000 7f23e8c00700 1 -- 172.21.15.47:0/2243247257 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f23e4104600 msgr2=0x7f23e41049e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:23:14.622 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:11.268+0000 7f23e8c00700 1 --2- 172.21.15.47:0/2243247257 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f23e4104600 0x7f23e41049e0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f23d40099c0 tx=0x7f23d4009cd0 comp rx=0 tx=0).stop 2024-03-21T10:23:14.622 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:11.268+0000 7f23e8c00700 1 -- 172.21.15.47:0/2243247257 shutdown_connections 2024-03-21T10:23:14.622 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:11.268+0000 7f23e8c00700 1 --2- 172.21.15.47:0/2243247257 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f23e41096e0 0x7f23e41112b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:23:14.622 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:11.268+0000 7f23e8c00700 1 --2- 172.21.15.47:0/2243247257 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f23e4104600 0x7f23e41049e0 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:23:14.622 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:11.268+0000 7f23e8c00700 1 --2- 172.21.15.47:0/2243247257 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f23e4104fb0 0x7f23e4109000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:23:14.622 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:11.268+0000 7f23e8c00700 1 -- 172.21.15.47:0/2243247257 >> 172.21.15.47:0/2243247257 conn(0x7f23e40ffe90 msgr2=0x7f23e41022b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:23:14.622 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:11.268+0000 7f23e8c00700 1 -- 172.21.15.47:0/2243247257 shutdown_connections 2024-03-21T10:23:14.622 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:11.268+0000 7f23e8c00700 1 -- 172.21.15.47:0/2243247257 wait complete. 2024-03-21T10:23:14.622 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:11.270+0000 7f23e8c00700 1 Processor -- start 2024-03-21T10:23:14.622 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:11.270+0000 7f23e8c00700 1 -- start start 2024-03-21T10:23:14.623 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:11.270+0000 7f23e8c00700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f23e4104600 0x7f23e4073920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:23:14.623 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:11.270+0000 7f23e8c00700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f23e4104fb0 0x7f23e4073e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:23:14.623 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:11.270+0000 7f23e8c00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f23e41096e0 0x7f23e406cb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:23:14.623 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:11.270+0000 7f23e8c00700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23e4114570 con 0x7f23e4104600 2024-03-21T10:23:14.623 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:11.270+0000 7f23e8c00700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23e41143f0 con 0x7f23e4104fb0 2024-03-21T10:23:14.623 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:11.270+0000 7f23e8c00700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23e41146f0 con 0x7f23e41096e0 2024-03-21T10:23:14.623 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:11.271+0000 7f23e1e00700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f23e4104600 0x7f23e4073920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:23:14.623 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:11.271+0000 7f23e1400700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f23e4104fb0 0x7f23e4073e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:23:14.623 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:11.271+0000 7f23e1e00700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f23e4104600 0x7f23e4073920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:54962/0 (socket says 172.21.15.47:54962) 2024-03-21T10:23:14.623 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:11.271+0000 7f23e1e00700 1 -- 172.21.15.47:0/261549640 learned_addr learned my addr 172.21.15.47:0/261549640 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:23:14.623 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:11.271+0000 7f23e1400700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f23e4104fb0 0x7f23e4073e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:50848/0 (socket says 172.21.15.47:50848) 2024-03-21T10:23:14.623 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:11.271+0000 7f23e2800700 1 --2- 172.21.15.47:0/261549640 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f23e41096e0 0x7f23e406cb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:23:14.623 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:11.272+0000 7f23e1e00700 1 -- 172.21.15.47:0/261549640 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f23e41096e0 msgr2=0x7f23e406cb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:23:14.623 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:11.272+0000 7f23e1e00700 1 --2- 172.21.15.47:0/261549640 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f23e41096e0 0x7f23e406cb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:23:14.624 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:11.272+0000 7f23e1e00700 1 -- 172.21.15.47:0/261549640 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f23e4104fb0 msgr2=0x7f23e4073e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:23:14.624 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:11.272+0000 7f23e1e00700 1 --2- 172.21.15.47:0/261549640 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f23e4104fb0 0x7f23e4073e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:23:14.624 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:11.272+0000 7f23e1e00700 1 -- 172.21.15.47:0/261549640 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23e406d490 con 0x7f23e4104600 2024-03-21T10:23:14.624 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:11.272+0000 7f23e1e00700 1 --2- 172.21.15.47:0/261549640 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f23e4104600 0x7f23e4073920 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f23d4006010 tx=0x7f23d4009f80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:23:14.624 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:11.272+0000 7f23daa00700 1 -- 172.21.15.47:0/261549640 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23d4026070 con 0x7f23e4104600 2024-03-21T10:23:14.624 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:11.272+0000 7f23daa00700 1 -- 172.21.15.47:0/261549640 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f23d4004ca0 con 0x7f23e4104600 2024-03-21T10:23:14.624 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:11.272+0000 7f23e8c00700 1 -- 172.21.15.47:0/261549640 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f23e406d720 con 0x7f23e4104600 2024-03-21T10:23:14.624 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:11.272+0000 7f23daa00700 1 -- 172.21.15.47:0/261549640 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23d4003d20 con 0x7f23e4104600 2024-03-21T10:23:14.624 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:11.272+0000 7f23e8c00700 1 -- 172.21.15.47:0/261549640 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f23e4071670 con 0x7f23e4104600 2024-03-21T10:23:14.624 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:11.273+0000 7f23e8c00700 1 -- 172.21.15.47:0/261549640 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f23e4071bf0 con 0x7f23e4104600 2024-03-21T10:23:14.624 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:11.278+0000 7f23daa00700 1 -- 172.21.15.47:0/261549640 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 17) v1 ==== 137905+0+0 (secure 0 0 0) 0x7f23d4029070 con 0x7f23e4104600 2024-03-21T10:23:14.624 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:11.279+0000 7f23daa00700 1 --2- 172.21.15.47:0/261549640 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f23c80a3040 0x7f23c80a5500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:23:14.624 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:11.279+0000 7f23daa00700 1 -- 172.21.15.47:0/261549640 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9092+0+0 (secure 0 0 0) 0x7f23d40d0640 con 0x7f23e4104600 2024-03-21T10:23:14.625 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:11.279+0000 7f23e1400700 1 --2- 172.21.15.47:0/261549640 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f23c80a3040 0x7f23c80a5500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:23:14.625 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:11.280+0000 7f23daa00700 1 --2- 172.21.15.47:0/261549640 >> [v2:172.21.15.100:6816/4104727330,v1:172.21.15.100:6817/4104727330] conn(0x7f23c80ae1a0 0x7f23c80b0640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:23:14.625 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:11.280+0000 7f23daa00700 1 -- 172.21.15.47:0/261549640 --> [v2:172.21.15.100:6816/4104727330,v1:172.21.15.100:6817/4104727330] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f23d4023ec0 con 0x7f23c80ae1a0 2024-03-21T10:23:14.625 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:11.280+0000 7f23daa00700 1 -- 172.21.15.47:0/261549640 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_get_version_reply(handle=1 version=93) v2 ==== 24+0+0 (secure 0 0 0) 0x7f23d40d09f0 con 0x7f23e4104600 2024-03-21T10:23:14.625 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:11.280+0000 7f23e2800700 1 --2- 172.21.15.47:0/261549640 >> [v2:172.21.15.100:6816/4104727330,v1:172.21.15.100:6817/4104727330] conn(0x7f23c80ae1a0 0x7f23c80b0640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:23:14.625 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:11.280+0000 7f23e1400700 1 --2- 172.21.15.47:0/261549640 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f23c80a3040 0x7f23c80a5500 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f23e406e170 tx=0x7f23d00073d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:23:14.626 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:11.281+0000 7f23e2800700 1 --2- 172.21.15.47:0/261549640 >> [v2:172.21.15.100:6816/4104727330,v1:172.21.15.100:6817/4104727330] conn(0x7f23c80ae1a0 0x7f23c80b0640 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:23:14.626 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:11.284+0000 7f23daa00700 1 -- 172.21.15.47:0/261549640 <== osd.7 v2:172.21.15.100:6816/4104727330 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7f23d4023ec0 con 0x7f23c80ae1a0 2024-03-21T10:23:14.626 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:11.303+0000 7f23e8c00700 1 -- 172.21.15.47:0/261549640 --> [v2:172.21.15.100:6816/4104727330,v1:172.21.15.100:6817/4104727330] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f23e4072070 con 0x7f23c80ae1a0 2024-03-21T10:23:14.626 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:11.304+0000 7f23daa00700 1 -- 172.21.15.47:0/261549640 <== osd.7 v2:172.21.15.100:6816/4104727330 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f23e4072070 con 0x7f23c80ae1a0 2024-03-21T10:23:14.626 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:11.305+0000 7f23e8c00700 1 -- 172.21.15.47:0/261549640 >> [v2:172.21.15.100:6816/4104727330,v1:172.21.15.100:6817/4104727330] conn(0x7f23c80ae1a0 msgr2=0x7f23c80b0640 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:23:14.626 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:11.305+0000 7f23e8c00700 1 --2- 172.21.15.47:0/261549640 >> [v2:172.21.15.100:6816/4104727330,v1:172.21.15.100:6817/4104727330] conn(0x7f23c80ae1a0 0x7f23c80b0640 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:23:14.626 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:11.305+0000 7f23e8c00700 1 -- 172.21.15.47:0/261549640 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f23c80a3040 msgr2=0x7f23c80a5500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:23:14.626 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:11.305+0000 7f23e8c00700 1 --2- 172.21.15.47:0/261549640 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f23c80a3040 0x7f23c80a5500 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f23e406e170 tx=0x7f23d00073d0 comp rx=0 tx=0).stop 2024-03-21T10:23:14.626 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:11.305+0000 7f23e8c00700 1 -- 172.21.15.47:0/261549640 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f23e4104600 msgr2=0x7f23e4073920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:23:14.626 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:11.305+0000 7f23e8c00700 1 --2- 172.21.15.47:0/261549640 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f23e4104600 0x7f23e4073920 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f23d4006010 tx=0x7f23d4009f80 comp rx=0 tx=0).stop 2024-03-21T10:23:14.626 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:11.305+0000 7f23e2800700 1 -- 172.21.15.47:0/261549640 reap_dead start 2024-03-21T10:23:14.626 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:11.305+0000 7f23e8c00700 1 -- 172.21.15.47:0/261549640 shutdown_connections 2024-03-21T10:23:14.626 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:11.305+0000 7f23e8c00700 1 -- 172.21.15.47:0/261549640 >> 172.21.15.47:0/261549640 conn(0x7f23e40ffe90 msgr2=0x7f23e41021d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:23:14.626 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:11.305+0000 7f23e8c00700 1 -- 172.21.15.47:0/261549640 shutdown_connections 2024-03-21T10:23:14.626 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:11.305+0000 7f23e8c00700 1 -- 172.21.15.47:0/261549640 wait complete. 2024-03-21T10:23:15.564 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:23:15 smithi047 ceph-mon[31547]: pgmap v407: 129 pgs: 129 active+clean; 642 KiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-03-21T10:23:15.695 INFO:teuthology.orchestra.run.smithi047.stdout:317827579939 2024-03-21T10:23:15.695 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2edfee68ad7243bed422a049b6f5f58619300f11 shell --fsid 3056d51a-e76b-11ee-95cd-87774f69a715 -- ceph osd last-stat-seq osd.10 2024-03-21T10:23:15.729 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:23:15 smithi100 ceph-mon[35193]: pgmap v407: 129 pgs: 129 active+clean; 642 KiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-03-21T10:23:15.740 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:23:15 smithi157 ceph-mon[35581]: pgmap v407: 129 pgs: 129 active+clean; 642 KiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-03-21T10:23:16.007 INFO:teuthology.orchestra.run.smithi047.stdout:115964117078 2024-03-21T10:23:16.007 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2edfee68ad7243bed422a049b6f5f58619300f11 shell --fsid 3056d51a-e76b-11ee-95cd-87774f69a715 -- ceph osd last-stat-seq osd.3 2024-03-21T10:23:16.485 INFO:teuthology.orchestra.run.smithi047.stdout:227633266747 2024-03-21T10:23:16.485 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2edfee68ad7243bed422a049b6f5f58619300f11 shell --fsid 3056d51a-e76b-11ee-95cd-87774f69a715 -- ceph osd last-stat-seq osd.7 2024-03-21T10:23:17.360 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:17.306+0000 7f31a3000700 1 -- 172.21.15.47:0/1663555405 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f319c102c70 msgr2=0x7f319c106cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:23:17.360 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:17.306+0000 7f31a3000700 1 --2- 172.21.15.47:0/1663555405 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f319c102c70 0x7f319c106cc0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f31880099c0 tx=0x7f3188009cd0 comp rx=0 tx=0).stop 2024-03-21T10:23:17.360 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:17.306+0000 7f31a3000700 1 -- 172.21.15.47:0/1663555405 shutdown_connections 2024-03-21T10:23:17.360 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:17.306+0000 7f31a3000700 1 --2- 172.21.15.47:0/1663555405 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f319c1073a0 0x7f319c10ef70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:23:17.361 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:17.306+0000 7f31a3000700 1 --2- 172.21.15.47:0/1663555405 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f319c1022c0 0x7f319c1026a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:23:17.361 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:17.306+0000 7f31a3000700 1 --2- 172.21.15.47:0/1663555405 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f319c102c70 0x7f319c106cc0 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:23:17.361 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:17.306+0000 7f31a3000700 1 -- 172.21.15.47:0/1663555405 >> 172.21.15.47:0/1663555405 conn(0x7f319c0fdbb0 msgr2=0x7f319c0fffd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:23:17.361 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:17.307+0000 7f31a3000700 1 -- 172.21.15.47:0/1663555405 shutdown_connections 2024-03-21T10:23:17.361 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:17.307+0000 7f31a3000700 1 -- 172.21.15.47:0/1663555405 wait complete. 2024-03-21T10:23:17.361 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:17.308+0000 7f31a3000700 1 Processor -- start 2024-03-21T10:23:17.361 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:17.309+0000 7f31a3000700 1 -- start start 2024-03-21T10:23:17.361 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:17.309+0000 7f31a3000700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f319c1022c0 0x7f319c1a0630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:23:17.362 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:17.309+0000 7f31a3000700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f319c102c70 0x7f319c1a0b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:23:17.362 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:17.309+0000 7f31a3000700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f319c1073a0 0x7f319c19a7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:23:17.362 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:17.309+0000 7f31a3000700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f319c112260 con 0x7f319c1022c0 2024-03-21T10:23:17.362 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:17.309+0000 7f31a3000700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f319c1120e0 con 0x7f319c102c70 2024-03-21T10:23:17.362 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:17.309+0000 7f31a3000700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f319c1123e0 con 0x7f319c1073a0 2024-03-21T10:23:17.362 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:17.310+0000 7f31a1c00700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f319c1022c0 0x7f319c1a0630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:23:17.362 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:17.310+0000 7f31a1c00700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f319c1022c0 0x7f319c1a0630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:58742/0 (socket says 172.21.15.47:58742) 2024-03-21T10:23:17.363 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:17.310+0000 7f31a1c00700 1 -- 172.21.15.47:0/3114354146 learned_addr learned my addr 172.21.15.47:0/3114354146 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:23:17.363 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:17.310+0000 7f31a1200700 1 --2- 172.21.15.47:0/3114354146 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f319c102c70 0x7f319c1a0b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:23:17.363 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:17.310+0000 7f31a2600700 1 --2- 172.21.15.47:0/3114354146 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f319c1073a0 0x7f319c19a7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:23:17.363 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:17.310+0000 7f31a1c00700 1 -- 172.21.15.47:0/3114354146 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f319c1073a0 msgr2=0x7f319c19a7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:23:17.363 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:17.310+0000 7f31a1c00700 1 --2- 172.21.15.47:0/3114354146 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f319c1073a0 0x7f319c19a7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:23:17.363 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:17.310+0000 7f31a1c00700 1 -- 172.21.15.47:0/3114354146 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f319c102c70 msgr2=0x7f319c1a0b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:23:17.364 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:17.310+0000 7f31a1c00700 1 --2- 172.21.15.47:0/3114354146 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f319c102c70 0x7f319c1a0b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:23:17.364 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:17.310+0000 7f31a1c00700 1 -- 172.21.15.47:0/3114354146 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f319c19af90 con 0x7f319c1022c0 2024-03-21T10:23:17.364 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:17.311+0000 7f31a1c00700 1 --2- 172.21.15.47:0/3114354146 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f319c1022c0 0x7f319c1a0630 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f319000d950 tx=0x7f319000dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:23:17.364 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:17.311+0000 7f319a000700 1 -- 172.21.15.47:0/3114354146 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3190016070 con 0x7f319c1022c0 2024-03-21T10:23:17.364 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:17.311+0000 7f319a000700 1 -- 172.21.15.47:0/3114354146 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f3190009df0 con 0x7f319c1022c0 2024-03-21T10:23:17.364 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:17.311+0000 7f319a000700 1 -- 172.21.15.47:0/3114354146 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f319000f3c0 con 0x7f319c1022c0 2024-03-21T10:23:17.365 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:17.311+0000 7f31a3000700 1 -- 172.21.15.47:0/3114354146 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f319c19b280 con 0x7f319c1022c0 2024-03-21T10:23:17.365 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:17.311+0000 7f31a3000700 1 -- 172.21.15.47:0/3114354146 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f319c108f20 con 0x7f319c1022c0 2024-03-21T10:23:17.365 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:17.313+0000 7f31a3000700 1 -- 172.21.15.47:0/3114354146 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f319c075720 con 0x7f319c1022c0 2024-03-21T10:23:17.365 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:17.321+0000 7f319a000700 1 -- 172.21.15.47:0/3114354146 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 17) v1 ==== 137905+0+0 (secure 0 0 0) 0x7f3190019070 con 0x7f319c1022c0 2024-03-21T10:23:17.365 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:17.322+0000 7f319a000700 1 --2- 172.21.15.47:0/3114354146 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f31840a30c0 0x7f31840a5580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:23:17.365 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:17.322+0000 7f319a000700 1 -- 172.21.15.47:0/3114354146 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9092+0+0 (secure 0 0 0) 0x7f31900cf9c0 con 0x7f319c1022c0 2024-03-21T10:23:17.365 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:17.323+0000 7f31a1200700 1 --2- 172.21.15.47:0/3114354146 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f31840a30c0 0x7f31840a5580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:23:17.366 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:17.323+0000 7f319a000700 1 --2- 172.21.15.47:0/3114354146 >> [v2:172.21.15.47:6826/1982108738,v1:172.21.15.47:6827/1982108738] conn(0x7f31840ae220 0x7f31840b06c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:23:17.366 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:17.323+0000 7f319a000700 1 -- 172.21.15.47:0/3114354146 --> [v2:172.21.15.47:6826/1982108738,v1:172.21.15.47:6827/1982108738] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f3190024ea0 con 0x7f31840ae220 2024-03-21T10:23:17.366 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:17.323+0000 7f319a000700 1 -- 172.21.15.47:0/3114354146 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_get_version_reply(handle=1 version=93) v2 ==== 24+0+0 (secure 0 0 0) 0x7f31900cfd70 con 0x7f319c1022c0 2024-03-21T10:23:17.366 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:17.323+0000 7f31a2600700 1 --2- 172.21.15.47:0/3114354146 >> [v2:172.21.15.47:6826/1982108738,v1:172.21.15.47:6827/1982108738] conn(0x7f31840ae220 0x7f31840b06c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:23:17.366 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:17.324+0000 7f31a1200700 1 --2- 172.21.15.47:0/3114354146 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f31840a30c0 0x7f31840a5580 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f3188000c00 tx=0x7f3188005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:23:17.366 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:17.324+0000 7f31a2600700 1 --2- 172.21.15.47:0/3114354146 >> [v2:172.21.15.47:6826/1982108738,v1:172.21.15.47:6827/1982108738] conn(0x7f31840ae220 0x7f31840b06c0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.9 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:23:17.367 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:17.326+0000 7f319a000700 1 -- 172.21.15.47:0/3114354146 <== osd.9 v2:172.21.15.47:6826/1982108738 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7f3190024ea0 con 0x7f31840ae220 2024-03-21T10:23:17.368 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:17.365+0000 7f31a3000700 1 -- 172.21.15.47:0/3114354146 --> [v2:172.21.15.47:6826/1982108738,v1:172.21.15.47:6827/1982108738] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f319c1037a0 con 0x7f31840ae220 2024-03-21T10:23:17.368 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:17.366+0000 7f319a000700 1 -- 172.21.15.47:0/3114354146 <== osd.9 v2:172.21.15.47:6826/1982108738 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f319c1037a0 con 0x7f31840ae220 2024-03-21T10:23:17.369 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:17.366+0000 7f31a3000700 1 -- 172.21.15.47:0/3114354146 >> [v2:172.21.15.47:6826/1982108738,v1:172.21.15.47:6827/1982108738] conn(0x7f31840ae220 msgr2=0x7f31840b06c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:23:17.370 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:17.366+0000 7f31a3000700 1 --2- 172.21.15.47:0/3114354146 >> [v2:172.21.15.47:6826/1982108738,v1:172.21.15.47:6827/1982108738] conn(0x7f31840ae220 0x7f31840b06c0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:23:17.370 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:17.366+0000 7f31a3000700 1 -- 172.21.15.47:0/3114354146 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f31840a30c0 msgr2=0x7f31840a5580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:23:17.370 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:17.366+0000 7f31a3000700 1 --2- 172.21.15.47:0/3114354146 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f31840a30c0 0x7f31840a5580 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f3188000c00 tx=0x7f3188005fb0 comp rx=0 tx=0).stop 2024-03-21T10:23:17.370 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:17.366+0000 7f31a3000700 1 -- 172.21.15.47:0/3114354146 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f319c1022c0 msgr2=0x7f319c1a0630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:23:17.370 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:17.366+0000 7f31a3000700 1 --2- 172.21.15.47:0/3114354146 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f319c1022c0 0x7f319c1a0630 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f319000d950 tx=0x7f319000dc60 comp rx=0 tx=0).stop 2024-03-21T10:23:17.370 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:17.367+0000 7f31a2600700 1 -- 172.21.15.47:0/3114354146 reap_dead start 2024-03-21T10:23:17.370 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:17.367+0000 7f31a3000700 1 -- 172.21.15.47:0/3114354146 shutdown_connections 2024-03-21T10:23:17.370 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:17.367+0000 7f31a3000700 1 -- 172.21.15.47:0/3114354146 >> 172.21.15.47:0/3114354146 conn(0x7f319c0fdbb0 msgr2=0x7f319c0feef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:23:17.370 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:17.367+0000 7f31a3000700 1 -- 172.21.15.47:0/3114354146 shutdown_connections 2024-03-21T10:23:17.370 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:17.367+0000 7f31a3000700 1 -- 172.21.15.47:0/3114354146 wait complete. 2024-03-21T10:23:17.394 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:23:17 smithi047 ceph-mon[31547]: pgmap v408: 129 pgs: 129 active+clean; 642 KiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-03-21T10:23:17.729 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:23:17 smithi100 ceph-mon[35193]: pgmap v408: 129 pgs: 129 active+clean; 642 KiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-03-21T10:23:17.739 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:23:17 smithi157 ceph-mon[35581]: pgmap v408: 129 pgs: 129 active+clean; 642 KiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-03-21T10:23:17.812 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:13.965+0000 7fa74ac00700 1 -- 172.21.15.47:0/3662277796 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa74406a760 msgr2=0x7fa74410e700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:23:17.812 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:13.965+0000 7fa74ac00700 1 --2- 172.21.15.47:0/3662277796 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa74406a760 0x7fa74410e700 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7fa730009990 tx=0x7fa730009ca0 comp rx=0 tx=0).stop 2024-03-21T10:23:17.812 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:13.965+0000 7fa74ac00700 1 -- 172.21.15.47:0/3662277796 shutdown_connections 2024-03-21T10:23:17.812 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:13.965+0000 7fa74ac00700 1 --2- 172.21.15.47:0/3662277796 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa74410ec40 0x7fa744111030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:23:17.812 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:13.965+0000 7fa74ac00700 1 --2- 172.21.15.47:0/3662277796 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa744069e40 0x7fa74406a220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:23:17.812 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:13.965+0000 7fa74ac00700 1 --2- 172.21.15.47:0/3662277796 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa74406a760 0x7fa74410e700 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:23:17.813 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:13.965+0000 7fa74ac00700 1 -- 172.21.15.47:0/3662277796 >> 172.21.15.47:0/3662277796 conn(0x7fa744077fc0 msgr2=0x7fa7440783d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:23:17.813 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:13.965+0000 7fa74ac00700 1 -- 172.21.15.47:0/3662277796 shutdown_connections 2024-03-21T10:23:17.813 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:13.966+0000 7fa74ac00700 1 -- 172.21.15.47:0/3662277796 wait complete. 2024-03-21T10:23:17.813 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:13.967+0000 7fa74ac00700 1 Processor -- start 2024-03-21T10:23:17.813 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:13.967+0000 7fa74ac00700 1 -- start start 2024-03-21T10:23:17.813 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:13.968+0000 7fa74ac00700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa744069e40 0x7fa7441a07e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:23:17.813 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:13.968+0000 7fa74ac00700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa74406a760 0x7fa7441a0d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:23:17.813 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:13.968+0000 7fa74ac00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa74410ec40 0x7fa74419a9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:23:17.813 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:13.968+0000 7fa74ac00700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa744114560 con 0x7fa744069e40 2024-03-21T10:23:17.813 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:13.968+0000 7fa74ac00700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7441143e0 con 0x7fa74406a760 2024-03-21T10:23:17.813 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:13.968+0000 7fa74ac00700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7441146e0 con 0x7fa74410ec40 2024-03-21T10:23:17.814 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:13.968+0000 7fa743e00700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa744069e40 0x7fa7441a07e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:23:17.814 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:13.968+0000 7fa743e00700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa744069e40 0x7fa7441a07e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:58722/0 (socket says 172.21.15.47:58722) 2024-03-21T10:23:17.814 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:13.968+0000 7fa743e00700 1 -- 172.21.15.47:0/116698127 learned_addr learned my addr 172.21.15.47:0/116698127 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:23:17.814 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:13.968+0000 7fa743400700 1 --2- 172.21.15.47:0/116698127 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa74406a760 0x7fa7441a0d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:23:17.814 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:13.969+0000 7fa748c00700 1 --2- 172.21.15.47:0/116698127 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa74410ec40 0x7fa74419a9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:23:17.814 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:13.969+0000 7fa743e00700 1 -- 172.21.15.47:0/116698127 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa74410ec40 msgr2=0x7fa74419a9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:23:17.815 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:13.969+0000 7fa743e00700 1 --2- 172.21.15.47:0/116698127 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa74410ec40 0x7fa74419a9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:23:17.815 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:13.969+0000 7fa743e00700 1 -- 172.21.15.47:0/116698127 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa74406a760 msgr2=0x7fa7441a0d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:23:17.815 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:13.969+0000 7fa743e00700 1 --2- 172.21.15.47:0/116698127 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa74406a760 0x7fa7441a0d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:23:17.815 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:13.969+0000 7fa743e00700 1 -- 172.21.15.47:0/116698127 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa74419b100 con 0x7fa744069e40 2024-03-21T10:23:17.815 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:13.969+0000 7fa748c00700 1 --2- 172.21.15.47:0/116698127 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa74410ec40 0x7fa74419a9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T10:23:17.815 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:13.969+0000 7fa743e00700 1 --2- 172.21.15.47:0/116698127 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa744069e40 0x7fa7441a07e0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7fa73400e9a0 tx=0x7fa73400ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:23:17.815 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:13.970+0000 7fa740c00700 1 -- 172.21.15.47:0/116698127 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa73400cbd0 con 0x7fa744069e40 2024-03-21T10:23:17.815 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:13.970+0000 7fa740c00700 1 -- 172.21.15.47:0/116698127 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fa73400cd70 con 0x7fa744069e40 2024-03-21T10:23:17.815 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:13.970+0000 7fa74ac00700 1 -- 172.21.15.47:0/116698127 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa74419b3f0 con 0x7fa744069e40 2024-03-21T10:23:17.815 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:13.970+0000 7fa740c00700 1 -- 172.21.15.47:0/116698127 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa734018680 con 0x7fa744069e40 2024-03-21T10:23:17.815 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:13.970+0000 7fa74ac00700 1 -- 172.21.15.47:0/116698127 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa7441a76b0 con 0x7fa744069e40 2024-03-21T10:23:17.815 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:13.971+0000 7fa74ac00700 1 -- 172.21.15.47:0/116698127 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fa7440628a0 con 0x7fa744069e40 2024-03-21T10:23:17.815 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:13.973+0000 7fa740c00700 1 -- 172.21.15.47:0/116698127 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 17) v1 ==== 137905+0+0 (secure 0 0 0) 0x7fa734010430 con 0x7fa744069e40 2024-03-21T10:23:17.816 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:13.974+0000 7fa740c00700 1 --2- 172.21.15.47:0/116698127 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7fa7280a3040 0x7fa7280a5500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:23:17.816 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:13.975+0000 7fa740c00700 1 -- 172.21.15.47:0/116698127 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9092+0+0 (secure 0 0 0) 0x7fa7340cfb90 con 0x7fa744069e40 2024-03-21T10:23:17.816 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:13.975+0000 7fa743400700 1 --2- 172.21.15.47:0/116698127 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7fa7280a3040 0x7fa7280a5500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:23:17.816 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:13.975+0000 7fa740c00700 1 --2- 172.21.15.47:0/116698127 >> [v2:172.21.15.157:6816/4067715347,v1:172.21.15.157:6817/4067715347] conn(0x7fa7280ae1a0 0x7fa7280b0640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:23:17.816 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:13.975+0000 7fa740c00700 1 -- 172.21.15.47:0/116698127 --> [v2:172.21.15.157:6816/4067715347,v1:172.21.15.157:6817/4067715347] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fa734012ec0 con 0x7fa7280ae1a0 2024-03-21T10:23:17.816 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:13.975+0000 7fa740c00700 1 -- 172.21.15.47:0/116698127 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_get_version_reply(handle=1 version=93) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa7340d42a0 con 0x7fa744069e40 2024-03-21T10:23:17.816 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:13.975+0000 7fa743400700 1 --2- 172.21.15.47:0/116698127 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7fa7280a3040 0x7fa7280a5500 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fa744102740 tx=0x7fa73000b260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:23:17.816 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:13.975+0000 7fa748c00700 1 --2- 172.21.15.47:0/116698127 >> [v2:172.21.15.157:6816/4067715347,v1:172.21.15.157:6817/4067715347] conn(0x7fa7280ae1a0 0x7fa7280b0640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:23:17.816 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:13.976+0000 7fa748c00700 1 --2- 172.21.15.47:0/116698127 >> [v2:172.21.15.157:6816/4067715347,v1:172.21.15.157:6817/4067715347] conn(0x7fa7280ae1a0 0x7fa7280b0640 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:23:17.816 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:13.978+0000 7fa740c00700 1 -- 172.21.15.47:0/116698127 <== osd.8 v2:172.21.15.157:6816/4067715347 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7fa734012ec0 con 0x7fa7280ae1a0 2024-03-21T10:23:17.816 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:13.996+0000 7fa74ac00700 1 -- 172.21.15.47:0/116698127 --> [v2:172.21.15.157:6816/4067715347,v1:172.21.15.157:6817/4067715347] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fa744062cd0 con 0x7fa7280ae1a0 2024-03-21T10:23:17.816 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:13.997+0000 7fa740c00700 1 -- 172.21.15.47:0/116698127 <== osd.8 v2:172.21.15.157:6816/4067715347 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7fa744062cd0 con 0x7fa7280ae1a0 2024-03-21T10:23:17.816 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:13.998+0000 7fa74ac00700 1 -- 172.21.15.47:0/116698127 >> [v2:172.21.15.157:6816/4067715347,v1:172.21.15.157:6817/4067715347] conn(0x7fa7280ae1a0 msgr2=0x7fa7280b0640 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:23:17.816 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:13.998+0000 7fa74ac00700 1 --2- 172.21.15.47:0/116698127 >> [v2:172.21.15.157:6816/4067715347,v1:172.21.15.157:6817/4067715347] conn(0x7fa7280ae1a0 0x7fa7280b0640 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:23:17.816 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:13.998+0000 7fa74ac00700 1 -- 172.21.15.47:0/116698127 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7fa7280a3040 msgr2=0x7fa7280a5500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:23:17.817 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:13.998+0000 7fa74ac00700 1 --2- 172.21.15.47:0/116698127 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7fa7280a3040 0x7fa7280a5500 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fa744102740 tx=0x7fa73000b260 comp rx=0 tx=0).stop 2024-03-21T10:23:17.817 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:13.998+0000 7fa74ac00700 1 -- 172.21.15.47:0/116698127 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa744069e40 msgr2=0x7fa7441a07e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:23:17.817 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:13.998+0000 7fa74ac00700 1 --2- 172.21.15.47:0/116698127 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa744069e40 0x7fa7441a07e0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7fa73400e9a0 tx=0x7fa73400ecb0 comp rx=0 tx=0).stop 2024-03-21T10:23:17.817 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:13.998+0000 7fa748c00700 1 -- 172.21.15.47:0/116698127 reap_dead start 2024-03-21T10:23:17.817 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:13.998+0000 7fa74ac00700 1 -- 172.21.15.47:0/116698127 shutdown_connections 2024-03-21T10:23:17.817 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:13.998+0000 7fa74ac00700 1 -- 172.21.15.47:0/116698127 >> 172.21.15.47:0/116698127 conn(0x7fa744077fc0 msgr2=0x7fa7441102d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:23:17.817 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:13.998+0000 7fa74ac00700 1 -- 172.21.15.47:0/116698127 shutdown_connections 2024-03-21T10:23:17.817 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:13.998+0000 7fa74ac00700 1 -- 172.21.15.47:0/116698127 wait complete. 2024-03-21T10:23:18.934 INFO:teuthology.orchestra.run.smithi047.stdout:261993005108 2024-03-21T10:23:18.935 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2edfee68ad7243bed422a049b6f5f58619300f11 shell --fsid 3056d51a-e76b-11ee-95cd-87774f69a715 -- ceph osd last-stat-seq osd.8 2024-03-21T10:23:19.297 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:18.494+0000 7f2f5b200700 1 -- 172.21.15.47:0/1542013190 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2f5c102c80 msgr2=0x7f2f5c106cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:23:19.298 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:18.494+0000 7f2f5b200700 1 --2- 172.21.15.47:0/1542013190 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2f5c102c80 0x7f2f5c106cd0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f2f480099c0 tx=0x7f2f48009cd0 comp rx=0 tx=0).stop 2024-03-21T10:23:19.298 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:18.495+0000 7f2f5b200700 1 -- 172.21.15.47:0/1542013190 shutdown_connections 2024-03-21T10:23:19.298 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:18.495+0000 7f2f5b200700 1 --2- 172.21.15.47:0/1542013190 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2f5c1073b0 0x7f2f5c10ef80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:23:19.298 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:18.495+0000 7f2f5b200700 1 --2- 172.21.15.47:0/1542013190 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2f5c1022d0 0x7f2f5c1026b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:23:19.298 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:18.495+0000 7f2f5b200700 1 --2- 172.21.15.47:0/1542013190 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2f5c102c80 0x7f2f5c106cd0 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:23:19.298 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:18.495+0000 7f2f5b200700 1 -- 172.21.15.47:0/1542013190 >> 172.21.15.47:0/1542013190 conn(0x7f2f5c0fdb80 msgr2=0x7f2f5c0fffa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:23:19.298 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:18.495+0000 7f2f5b200700 1 -- 172.21.15.47:0/1542013190 shutdown_connections 2024-03-21T10:23:19.298 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:18.495+0000 7f2f5b200700 1 -- 172.21.15.47:0/1542013190 wait complete. 2024-03-21T10:23:19.298 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:18.496+0000 7f2f5b200700 1 Processor -- start 2024-03-21T10:23:19.298 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:18.496+0000 7f2f5b200700 1 -- start start 2024-03-21T10:23:19.298 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:18.496+0000 7f2f5b200700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2f5c1022d0 0x7f2f5c1a0640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:23:19.298 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:18.496+0000 7f2f5b200700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2f5c102c80 0x7f2f5c1a0b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:23:19.299 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:18.496+0000 7f2f5b200700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2f5c1073b0 0x7f2f5c19a810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:23:19.299 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:18.496+0000 7f2f5b200700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f5c112270 con 0x7f2f5c1022d0 2024-03-21T10:23:19.299 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:18.496+0000 7f2f5b200700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f5c1120f0 con 0x7f2f5c1073b0 2024-03-21T10:23:19.299 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:18.496+0000 7f2f5b200700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f5c1123f0 con 0x7f2f5c102c80 2024-03-21T10:23:19.299 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:18.497+0000 7f2f59e00700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2f5c1022d0 0x7f2f5c1a0640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:23:19.299 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:18.497+0000 7f2f59e00700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2f5c1022d0 0x7f2f5c1a0640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:58754/0 (socket says 172.21.15.47:58754) 2024-03-21T10:23:19.299 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:18.497+0000 7f2f59e00700 1 -- 172.21.15.47:0/3753222346 learned_addr learned my addr 172.21.15.47:0/3753222346 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:23:19.299 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:18.497+0000 7f2f5a800700 1 --2- 172.21.15.47:0/3753222346 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2f5c1073b0 0x7f2f5c19a810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:23:19.299 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:18.497+0000 7f2f59400700 1 --2- 172.21.15.47:0/3753222346 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2f5c102c80 0x7f2f5c1a0b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:23:19.299 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:18.497+0000 7f2f59e00700 1 -- 172.21.15.47:0/3753222346 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2f5c102c80 msgr2=0x7f2f5c1a0b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:23:19.299 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:18.497+0000 7f2f59e00700 1 --2- 172.21.15.47:0/3753222346 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2f5c102c80 0x7f2f5c1a0b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:23:19.299 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:18.497+0000 7f2f59e00700 1 -- 172.21.15.47:0/3753222346 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2f5c1073b0 msgr2=0x7f2f5c19a810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:23:19.299 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:18.497+0000 7f2f59e00700 1 --2- 172.21.15.47:0/3753222346 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2f5c1073b0 0x7f2f5c19a810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:23:19.299 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:18.497+0000 7f2f59e00700 1 -- 172.21.15.47:0/3753222346 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f5c19aff0 con 0x7f2f5c1022d0 2024-03-21T10:23:19.300 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:18.497+0000 7f2f59e00700 1 --2- 172.21.15.47:0/3753222346 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2f5c1022d0 0x7f2f5c1a0640 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f2f4c00d950 tx=0x7f2f4c00dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:23:19.300 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:18.498+0000 7f2f52a00700 1 -- 172.21.15.47:0/3753222346 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f4c016070 con 0x7f2f5c1022d0 2024-03-21T10:23:19.300 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:18.498+0000 7f2f52a00700 1 -- 172.21.15.47:0/3753222346 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f2f4c009df0 con 0x7f2f5c1022d0 2024-03-21T10:23:19.300 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:18.498+0000 7f2f52a00700 1 -- 172.21.15.47:0/3753222346 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f4c00f3c0 con 0x7f2f5c1022d0 2024-03-21T10:23:19.300 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:18.498+0000 7f2f5b200700 1 -- 172.21.15.47:0/3753222346 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2f5c19b2e0 con 0x7f2f5c1022d0 2024-03-21T10:23:19.300 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:18.498+0000 7f2f5b200700 1 -- 172.21.15.47:0/3753222346 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2f5c1a74a0 con 0x7f2f5c1022d0 2024-03-21T10:23:19.300 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:18.499+0000 7f2f5b200700 1 -- 172.21.15.47:0/3753222346 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f2f5c10e6a0 con 0x7f2f5c1022d0 2024-03-21T10:23:19.300 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:18.501+0000 7f2f52a00700 1 -- 172.21.15.47:0/3753222346 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 17) v1 ==== 137905+0+0 (secure 0 0 0) 0x7f2f4c019070 con 0x7f2f5c1022d0 2024-03-21T10:23:19.300 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:18.503+0000 7f2f52a00700 1 --2- 172.21.15.47:0/3753222346 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f2f400a30c0 0x7f2f400a5580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:23:19.301 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:18.503+0000 7f2f52a00700 1 -- 172.21.15.47:0/3753222346 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9092+0+0 (secure 0 0 0) 0x7f2f4c0cf9a0 con 0x7f2f5c1022d0 2024-03-21T10:23:19.301 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:18.503+0000 7f2f59400700 1 --2- 172.21.15.47:0/3753222346 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f2f400a30c0 0x7f2f400a5580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:23:19.301 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:18.503+0000 7f2f52a00700 1 --2- 172.21.15.47:0/3753222346 >> [v2:172.21.15.100:6800/2923304710,v1:172.21.15.100:6801/2923304710] conn(0x7f2f400ae220 0x7f2f400b06c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:23:19.301 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:18.503+0000 7f2f52a00700 1 -- 172.21.15.47:0/3753222346 --> [v2:172.21.15.100:6800/2923304710,v1:172.21.15.100:6801/2923304710] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f2f4c024ea0 con 0x7f2f400ae220 2024-03-21T10:23:19.301 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:18.503+0000 7f2f52a00700 1 -- 172.21.15.47:0/3753222346 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_get_version_reply(handle=1 version=93) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2f4c0cfd50 con 0x7f2f5c1022d0 2024-03-21T10:23:19.301 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:18.504+0000 7f2f5a800700 1 --2- 172.21.15.47:0/3753222346 >> [v2:172.21.15.100:6800/2923304710,v1:172.21.15.100:6801/2923304710] conn(0x7f2f400ae220 0x7f2f400b06c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:23:19.301 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:18.504+0000 7f2f59400700 1 --2- 172.21.15.47:0/3753222346 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f2f400a30c0 0x7f2f400a5580 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f2f48000c00 tx=0x7f2f48005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:23:19.301 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:18.505+0000 7f2f5a800700 1 --2- 172.21.15.47:0/3753222346 >> [v2:172.21.15.100:6800/2923304710,v1:172.21.15.100:6801/2923304710] conn(0x7f2f400ae220 0x7f2f400b06c0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:23:19.301 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:18.506+0000 7f2f52a00700 1 -- 172.21.15.47:0/3753222346 <== osd.1 v2:172.21.15.100:6800/2923304710 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7f2f4c024ea0 con 0x7f2f400ae220 2024-03-21T10:23:19.301 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:18.524+0000 7f2f5b200700 1 -- 172.21.15.47:0/3753222346 --> [v2:172.21.15.100:6800/2923304710,v1:172.21.15.100:6801/2923304710] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f2f5c1037b0 con 0x7f2f400ae220 2024-03-21T10:23:19.301 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:18.525+0000 7f2f52a00700 1 -- 172.21.15.47:0/3753222346 <== osd.1 v2:172.21.15.100:6800/2923304710 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f2f5c1037b0 con 0x7f2f400ae220 2024-03-21T10:23:19.301 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:18.526+0000 7f2f5b200700 1 -- 172.21.15.47:0/3753222346 >> [v2:172.21.15.100:6800/2923304710,v1:172.21.15.100:6801/2923304710] conn(0x7f2f400ae220 msgr2=0x7f2f400b06c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:23:19.301 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:18.526+0000 7f2f5b200700 1 --2- 172.21.15.47:0/3753222346 >> [v2:172.21.15.100:6800/2923304710,v1:172.21.15.100:6801/2923304710] conn(0x7f2f400ae220 0x7f2f400b06c0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:23:19.302 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:18.526+0000 7f2f5b200700 1 -- 172.21.15.47:0/3753222346 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f2f400a30c0 msgr2=0x7f2f400a5580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:23:19.302 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:18.526+0000 7f2f5b200700 1 --2- 172.21.15.47:0/3753222346 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f2f400a30c0 0x7f2f400a5580 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f2f48000c00 tx=0x7f2f48005fb0 comp rx=0 tx=0).stop 2024-03-21T10:23:19.302 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:18.526+0000 7f2f5b200700 1 -- 172.21.15.47:0/3753222346 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2f5c1022d0 msgr2=0x7f2f5c1a0640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:23:19.302 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:18.526+0000 7f2f5b200700 1 --2- 172.21.15.47:0/3753222346 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2f5c1022d0 0x7f2f5c1a0640 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f2f4c00d950 tx=0x7f2f4c00dc60 comp rx=0 tx=0).stop 2024-03-21T10:23:19.302 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:18.526+0000 7f2f5a800700 1 -- 172.21.15.47:0/3753222346 reap_dead start 2024-03-21T10:23:19.302 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:18.526+0000 7f2f5b200700 1 -- 172.21.15.47:0/3753222346 shutdown_connections 2024-03-21T10:23:19.302 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:18.526+0000 7f2f5b200700 1 -- 172.21.15.47:0/3753222346 >> 172.21.15.47:0/3753222346 conn(0x7f2f5c0fdb80 msgr2=0x7f2f5c0ffe60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:23:19.302 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:18.526+0000 7f2f5b200700 1 -- 172.21.15.47:0/3753222346 shutdown_connections 2024-03-21T10:23:19.302 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:18.526+0000 7f2f5b200700 1 -- 172.21.15.47:0/3753222346 wait complete. 2024-03-21T10:23:19.855 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/3056d51a-e76b-11ee-95cd-87774f69a715/mon.a/config 2024-03-21T10:23:19.891 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:23:19 smithi047 ceph-mon[31547]: pgmap v409: 129 pgs: 129 active+clean; 642 KiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 9 op/s 2024-03-21T10:23:19.894 INFO:teuthology.orchestra.run.smithi047.stdout:296352743469 2024-03-21T10:23:19.895 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2edfee68ad7243bed422a049b6f5f58619300f11 shell --fsid 3056d51a-e76b-11ee-95cd-87774f69a715 -- ceph osd last-stat-seq osd.9 2024-03-21T10:23:19.979 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:23:19 smithi100 ceph-mon[35193]: pgmap v409: 129 pgs: 129 active+clean; 642 KiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 9 op/s 2024-03-21T10:23:19.989 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:23:19 smithi157 ceph-mon[35581]: pgmap v409: 129 pgs: 129 active+clean; 642 KiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 9 op/s 2024-03-21T10:23:20.483 INFO:teuthology.orchestra.run.smithi047.stdout:55834574949 2024-03-21T10:23:20.483 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2edfee68ad7243bed422a049b6f5f58619300f11 shell --fsid 3056d51a-e76b-11ee-95cd-87774f69a715 -- ceph osd last-stat-seq osd.1 2024-03-21T10:23:20.642 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:20.240+0000 7fe7a1800700 1 -- 172.21.15.47:0/2484129172 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe79c113040 msgr2=0x7fe79c115430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:23:20.642 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:20.240+0000 7fe7a1800700 1 --2- 172.21.15.47:0/2484129172 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe79c113040 0x7fe79c115430 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7fe79400b0a0 tx=0x7fe79400b3b0 comp rx=0 tx=0).stop 2024-03-21T10:23:20.642 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:20.240+0000 7fe7a1800700 1 -- 172.21.15.47:0/2484129172 shutdown_connections 2024-03-21T10:23:20.642 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:20.240+0000 7fe7a1800700 1 --2- 172.21.15.47:0/2484129172 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe79c113040 0x7fe79c115430 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:23:20.643 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:20.240+0000 7fe7a1800700 1 --2- 172.21.15.47:0/2484129172 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe79c0766c0 0x7fe79c076aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:23:20.643 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:20.240+0000 7fe7a1800700 1 --2- 172.21.15.47:0/2484129172 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe79c076fe0 0x7fe79c112b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:23:20.643 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:20.240+0000 7fe7a1800700 1 -- 172.21.15.47:0/2484129172 >> 172.21.15.47:0/2484129172 conn(0x7fe79c0ffe90 msgr2=0x7fe79c1022b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:23:20.643 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:20.241+0000 7fe7a1800700 1 -- 172.21.15.47:0/2484129172 shutdown_connections 2024-03-21T10:23:20.643 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:20.241+0000 7fe7a1800700 1 -- 172.21.15.47:0/2484129172 wait complete. 2024-03-21T10:23:20.643 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:20.242+0000 7fe7a1800700 1 Processor -- start 2024-03-21T10:23:20.643 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:20.242+0000 7fe7a1800700 1 -- start start 2024-03-21T10:23:20.643 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:20.242+0000 7fe7a1800700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe79c0766c0 0x7fe79c19eee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:23:20.643 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:20.242+0000 7fe7a1800700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe79c076fe0 0x7fe79c19f420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:23:20.643 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:20.242+0000 7fe7a1800700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe79c113040 0x7fe79c1a37b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:23:20.643 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:20.242+0000 7fe7a1800700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe79c118990 con 0x7fe79c113040 2024-03-21T10:23:20.643 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:20.242+0000 7fe7a1800700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe79c118810 con 0x7fe79c076fe0 2024-03-21T10:23:20.643 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:20.242+0000 7fe7a1800700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe79c118b10 con 0x7fe79c0766c0 2024-03-21T10:23:20.644 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:20.243+0000 7fe79b400700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe79c113040 0x7fe79c1a37b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:23:20.644 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:20.243+0000 7fe79b400700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe79c113040 0x7fe79c1a37b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:58776/0 (socket says 172.21.15.47:58776) 2024-03-21T10:23:20.644 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:20.243+0000 7fe79b400700 1 -- 172.21.15.47:0/1032896127 learned_addr learned my addr 172.21.15.47:0/1032896127 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:23:20.644 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:20.243+0000 7fe793e00700 1 --2- 172.21.15.47:0/1032896127 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe79c076fe0 0x7fe79c19f420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:23:20.644 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:20.243+0000 7fe79aa00700 1 --2- 172.21.15.47:0/1032896127 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe79c0766c0 0x7fe79c19eee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:23:20.644 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:20.243+0000 7fe79b400700 1 -- 172.21.15.47:0/1032896127 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe79c0766c0 msgr2=0x7fe79c19eee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:23:20.644 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:20.243+0000 7fe79b400700 1 --2- 172.21.15.47:0/1032896127 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe79c0766c0 0x7fe79c19eee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:23:20.644 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:20.243+0000 7fe79b400700 1 -- 172.21.15.47:0/1032896127 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe79c076fe0 msgr2=0x7fe79c19f420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:23:20.644 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:20.243+0000 7fe79b400700 1 --2- 172.21.15.47:0/1032896127 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe79c076fe0 0x7fe79c19f420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:23:20.644 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:20.243+0000 7fe79b400700 1 -- 172.21.15.47:0/1032896127 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe79c1a3ef0 con 0x7fe79c113040 2024-03-21T10:23:20.644 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:20.243+0000 7fe79aa00700 1 --2- 172.21.15.47:0/1032896127 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe79c0766c0 0x7fe79c19eee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T10:23:20.644 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:20.244+0000 7fe79b400700 1 --2- 172.21.15.47:0/1032896127 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe79c113040 0x7fe79c1a37b0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7fe79400b070 tx=0x7fe794004ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:23:20.644 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:20.244+0000 7fe793400700 1 -- 172.21.15.47:0/1032896127 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe794021070 con 0x7fe79c113040 2024-03-21T10:23:20.644 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:20.244+0000 7fe7a1800700 1 -- 172.21.15.47:0/1032896127 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe79c1a4180 con 0x7fe79c113040 2024-03-21T10:23:20.645 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:20.244+0000 7fe793400700 1 -- 172.21.15.47:0/1032896127 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fe794007b80 con 0x7fe79c113040 2024-03-21T10:23:20.645 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:20.244+0000 7fe793400700 1 -- 172.21.15.47:0/1032896127 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7940052a0 con 0x7fe79c113040 2024-03-21T10:23:20.645 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:20.244+0000 7fe7a1800700 1 -- 172.21.15.47:0/1032896127 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe79c1aba60 con 0x7fe79c113040 2024-03-21T10:23:20.645 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:20.245+0000 7fe7a1800700 1 -- 172.21.15.47:0/1032896127 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fe79c062860 con 0x7fe79c113040 2024-03-21T10:23:20.645 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:20.247+0000 7fe793400700 1 -- 172.21.15.47:0/1032896127 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 17) v1 ==== 137905+0+0 (secure 0 0 0) 0x7fe794024070 con 0x7fe79c113040 2024-03-21T10:23:20.645 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:20.250+0000 7fe793400700 1 --2- 172.21.15.47:0/1032896127 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7fe7800a3040 0x7fe7800a5500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:23:20.645 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:20.250+0000 7fe793400700 1 -- 172.21.15.47:0/1032896127 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9092+0+0 (secure 0 0 0) 0x7fe79400f020 con 0x7fe79c113040 2024-03-21T10:23:20.645 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:20.250+0000 7fe79aa00700 1 --2- 172.21.15.47:0/1032896127 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7fe7800a3040 0x7fe7800a5500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:23:20.645 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:20.250+0000 7fe793400700 1 --2- 172.21.15.47:0/1032896127 >> [v2:172.21.15.47:6818/2321642790,v1:172.21.15.47:6819/2321642790] conn(0x7fe7800ae1a0 0x7fe7800b0640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:23:20.645 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:20.250+0000 7fe793400700 1 -- 172.21.15.47:0/1032896127 --> [v2:172.21.15.47:6818/2321642790,v1:172.21.15.47:6819/2321642790] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fe794026e50 con 0x7fe7800ae1a0 2024-03-21T10:23:20.645 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:20.250+0000 7fe793400700 1 -- 172.21.15.47:0/1032896127 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_get_version_reply(handle=1 version=93) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe7940d1670 con 0x7fe79c113040 2024-03-21T10:23:20.645 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:20.251+0000 7fe793e00700 1 --2- 172.21.15.47:0/1032896127 >> [v2:172.21.15.47:6818/2321642790,v1:172.21.15.47:6819/2321642790] conn(0x7fe7800ae1a0 0x7fe7800b0640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:23:20.645 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:20.251+0000 7fe79aa00700 1 --2- 172.21.15.47:0/1032896127 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7fe7800a3040 0x7fe7800a5500 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fe78c00bcf0 tx=0x7fe78c00b2c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:23:20.645 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:20.251+0000 7fe793e00700 1 --2- 172.21.15.47:0/1032896127 >> [v2:172.21.15.47:6818/2321642790,v1:172.21.15.47:6819/2321642790] conn(0x7fe7800ae1a0 0x7fe7800b0640 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:23:20.646 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:20.252+0000 7fe793400700 1 -- 172.21.15.47:0/1032896127 <== osd.6 v2:172.21.15.47:6818/2321642790 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7fe794026e50 con 0x7fe7800ae1a0 2024-03-21T10:23:20.646 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:20.270+0000 7fe7a1800700 1 -- 172.21.15.47:0/1032896127 --> [v2:172.21.15.47:6818/2321642790,v1:172.21.15.47:6819/2321642790] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fe79c062ce0 con 0x7fe7800ae1a0 2024-03-21T10:23:20.646 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:20.271+0000 7fe793400700 1 -- 172.21.15.47:0/1032896127 <== osd.6 v2:172.21.15.47:6818/2321642790 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7fe79c062ce0 con 0x7fe7800ae1a0 2024-03-21T10:23:20.646 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:20.271+0000 7fe7a1800700 1 -- 172.21.15.47:0/1032896127 >> [v2:172.21.15.47:6818/2321642790,v1:172.21.15.47:6819/2321642790] conn(0x7fe7800ae1a0 msgr2=0x7fe7800b0640 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:23:20.646 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:20.271+0000 7fe7a1800700 1 --2- 172.21.15.47:0/1032896127 >> [v2:172.21.15.47:6818/2321642790,v1:172.21.15.47:6819/2321642790] conn(0x7fe7800ae1a0 0x7fe7800b0640 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:23:20.646 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:20.271+0000 7fe7a1800700 1 -- 172.21.15.47:0/1032896127 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7fe7800a3040 msgr2=0x7fe7800a5500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:23:20.646 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:20.271+0000 7fe7a1800700 1 --2- 172.21.15.47:0/1032896127 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7fe7800a3040 0x7fe7800a5500 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fe78c00bcf0 tx=0x7fe78c00b2c0 comp rx=0 tx=0).stop 2024-03-21T10:23:20.647 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:20.271+0000 7fe7a1800700 1 -- 172.21.15.47:0/1032896127 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe79c113040 msgr2=0x7fe79c1a37b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:23:20.647 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:20.271+0000 7fe7a1800700 1 --2- 172.21.15.47:0/1032896127 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe79c113040 0x7fe79c1a37b0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7fe79400b070 tx=0x7fe794004ce0 comp rx=0 tx=0).stop 2024-03-21T10:23:20.647 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:20.271+0000 7fe79b400700 1 -- 172.21.15.47:0/1032896127 reap_dead start 2024-03-21T10:23:20.647 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:20.272+0000 7fe7a1800700 1 -- 172.21.15.47:0/1032896127 shutdown_connections 2024-03-21T10:23:20.647 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:20.272+0000 7fe7a1800700 1 -- 172.21.15.47:0/1032896127 >> 172.21.15.47:0/1032896127 conn(0x7fe79c0ffe90 msgr2=0x7fe79c102280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:23:20.647 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:20.272+0000 7fe7a1800700 1 -- 172.21.15.47:0/1032896127 shutdown_connections 2024-03-21T10:23:20.647 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:20.272+0000 7fe7a1800700 1 -- 172.21.15.47:0/1032896127 wait complete. 2024-03-21T10:23:21.076 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/3056d51a-e76b-11ee-95cd-87774f69a715/mon.a/config 2024-03-21T10:23:21.613 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/3056d51a-e76b-11ee-95cd-87774f69a715/mon.a/config 2024-03-21T10:23:21.631 INFO:teuthology.orchestra.run.smithi047.stdout:206158430276 2024-03-21T10:23:21.632 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2edfee68ad7243bed422a049b6f5f58619300f11 shell --fsid 3056d51a-e76b-11ee-95cd-87774f69a715 -- ceph osd last-stat-seq osd.6 2024-03-21T10:23:21.867 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:23:21 smithi047 ceph-mon[31547]: pgmap v410: 129 pgs: 129 active+clean; 642 KiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-03-21T10:23:21.869 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:23:21 smithi100 ceph-mon[35193]: pgmap v410: 129 pgs: 129 active+clean; 642 KiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-03-21T10:23:21.990 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:23:21 smithi157 ceph-mon[35581]: pgmap v410: 129 pgs: 129 active+clean; 642 KiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-03-21T10:23:22.459 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/3056d51a-e76b-11ee-95cd-87774f69a715/mon.a/config 2024-03-21T10:23:22.466 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/3056d51a-e76b-11ee-95cd-87774f69a715/mon.a/config 2024-03-21T10:23:22.467 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/3056d51a-e76b-11ee-95cd-87774f69a715/mon.a/config 2024-03-21T10:23:22.475 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/3056d51a-e76b-11ee-95cd-87774f69a715/mon.a/config 2024-03-21T10:23:23.906 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:23:23 smithi047 ceph-mon[31547]: pgmap v411: 129 pgs: 129 active+clean; 642 KiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-21T10:23:23.979 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:23:23 smithi100 ceph-mon[35193]: pgmap v411: 129 pgs: 129 active+clean; 642 KiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-21T10:23:23.990 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:23:23 smithi157 ceph-mon[35581]: pgmap v411: 129 pgs: 129 active+clean; 642 KiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-21T10:23:24.550 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/3056d51a-e76b-11ee-95cd-87774f69a715/mon.a/config 2024-03-21T10:23:25.878 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:23:25 smithi047 ceph-mon[31547]: pgmap v412: 129 pgs: 129 active+clean; 642 KiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-03-21T10:23:25.979 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:23:25 smithi100 ceph-mon[35193]: pgmap v412: 129 pgs: 129 active+clean; 642 KiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-03-21T10:23:25.989 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:23:25 smithi157 ceph-mon[35581]: pgmap v412: 129 pgs: 129 active+clean; 642 KiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-03-21T10:23:26.860 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:23:26 smithi047 ceph-mon[31547]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.0", "id": [4, 8, 5, 9]}]: dispatch 2024-03-21T10:23:26.860 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:23:26 smithi047 ceph-mon[31547]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.c", "id": [9, 10]}]: dispatch 2024-03-21T10:23:26.860 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:23:26 smithi047 ceph-mon[31547]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.f", "id": [9, 8]}]: dispatch 2024-03-21T10:23:26.860 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:23:26 smithi047 ceph-mon[31547]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.19", "id": [9, 8]}]: dispatch 2024-03-21T10:23:26.860 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:23:26 smithi047 ceph-mon[31547]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1d", "id": [5, 8]}]: dispatch 2024-03-21T10:23:26.979 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:23:26 smithi100 ceph-mon[35193]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.0", "id": [4, 8, 5, 9]}]: dispatch 2024-03-21T10:23:26.979 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:23:26 smithi100 ceph-mon[35193]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.c", "id": [9, 10]}]: dispatch 2024-03-21T10:23:26.979 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:23:26 smithi100 ceph-mon[35193]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.f", "id": [9, 8]}]: dispatch 2024-03-21T10:23:26.979 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:23:26 smithi100 ceph-mon[35193]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.19", "id": [9, 8]}]: dispatch 2024-03-21T10:23:26.980 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:23:26 smithi100 ceph-mon[35193]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1d", "id": [5, 8]}]: dispatch 2024-03-21T10:23:26.990 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:23:26 smithi157 ceph-mon[35581]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.0", "id": [4, 8, 5, 9]}]: dispatch 2024-03-21T10:23:26.990 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:23:26 smithi157 ceph-mon[35581]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.c", "id": [9, 10]}]: dispatch 2024-03-21T10:23:26.990 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:23:26 smithi157 ceph-mon[35581]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.f", "id": [9, 8]}]: dispatch 2024-03-21T10:23:26.990 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:23:26 smithi157 ceph-mon[35581]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.19", "id": [9, 8]}]: dispatch 2024-03-21T10:23:26.990 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:23:26 smithi157 ceph-mon[35581]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1d", "id": [5, 8]}]: dispatch 2024-03-21T10:23:27.734 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:23:27 smithi047 ceph-mon[31547]: pgmap v413: 129 pgs: 129 active+clean; 642 KiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-21T10:23:27.734 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:23:27 smithi047 ceph-mon[31547]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.0", "id": [4, 8, 5, 9]}]': finished 2024-03-21T10:23:27.735 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:23:27 smithi047 ceph-mon[31547]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.c", "id": [9, 10]}]': finished 2024-03-21T10:23:27.735 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:23:27 smithi047 ceph-mon[31547]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.f", "id": [9, 8]}]': finished 2024-03-21T10:23:27.735 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:23:27 smithi047 ceph-mon[31547]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.19", "id": [9, 8]}]': finished 2024-03-21T10:23:27.735 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:23:27 smithi047 ceph-mon[31547]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1d", "id": [5, 8]}]': finished 2024-03-21T10:23:27.735 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:23:27 smithi047 ceph-mon[31547]: osdmap e94: 12 total, 12 up, 12 in 2024-03-21T10:23:27.735 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:23:27 smithi047 ceph-mon[31547]: from='client.? 172.21.15.47:0/206373765' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-03-21T10:23:27.979 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:23:27 smithi100 ceph-mon[35193]: pgmap v413: 129 pgs: 129 active+clean; 642 KiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-21T10:23:27.980 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:23:27 smithi100 ceph-mon[35193]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.0", "id": [4, 8, 5, 9]}]': finished 2024-03-21T10:23:27.980 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:23:27 smithi100 ceph-mon[35193]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.c", "id": [9, 10]}]': finished 2024-03-21T10:23:27.980 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:23:27 smithi100 ceph-mon[35193]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.f", "id": [9, 8]}]': finished 2024-03-21T10:23:27.980 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:23:27 smithi100 ceph-mon[35193]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.19", "id": [9, 8]}]': finished 2024-03-21T10:23:27.980 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:23:27 smithi100 ceph-mon[35193]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1d", "id": [5, 8]}]': finished 2024-03-21T10:23:27.980 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:23:27 smithi100 ceph-mon[35193]: osdmap e94: 12 total, 12 up, 12 in 2024-03-21T10:23:27.980 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:23:27 smithi100 ceph-mon[35193]: from='client.? 172.21.15.47:0/206373765' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-03-21T10:23:27.990 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:23:27 smithi157 ceph-mon[35581]: pgmap v413: 129 pgs: 129 active+clean; 642 KiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-21T10:23:27.990 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:23:27 smithi157 ceph-mon[35581]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.0", "id": [4, 8, 5, 9]}]': finished 2024-03-21T10:23:27.990 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:23:27 smithi157 ceph-mon[35581]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.c", "id": [9, 10]}]': finished 2024-03-21T10:23:27.990 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:23:27 smithi157 ceph-mon[35581]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.f", "id": [9, 8]}]': finished 2024-03-21T10:23:27.990 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:23:27 smithi157 ceph-mon[35581]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.19", "id": [9, 8]}]': finished 2024-03-21T10:23:27.990 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:23:27 smithi157 ceph-mon[35581]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1d", "id": [5, 8]}]': finished 2024-03-21T10:23:27.990 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:23:27 smithi157 ceph-mon[35581]: osdmap e94: 12 total, 12 up, 12 in 2024-03-21T10:23:27.990 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:23:27 smithi157 ceph-mon[35581]: from='client.? 172.21.15.47:0/206373765' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-03-21T10:23:28.009 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:27.281+0000 7f95e8c00700 1 -- 172.21.15.47:0/904152918 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f95e4102df0 msgr2=0x7f95e4106e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:23:28.009 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:27.281+0000 7f95e8c00700 1 --2- 172.21.15.47:0/904152918 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f95e4102df0 0x7f95e4106e30 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f95d00099c0 tx=0x7f95d0009cd0 comp rx=0 tx=0).stop 2024-03-21T10:23:28.009 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:27.281+0000 7f95e8c00700 1 -- 172.21.15.47:0/904152918 shutdown_connections 2024-03-21T10:23:28.010 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:27.281+0000 7f95e8c00700 1 --2- 172.21.15.47:0/904152918 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f95e4107510 0x7f95e410f0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:23:28.010 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:27.281+0000 7f95e8c00700 1 --2- 172.21.15.47:0/904152918 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f95e4102440 0x7f95e4102820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:23:28.010 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:27.281+0000 7f95e8c00700 1 --2- 172.21.15.47:0/904152918 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f95e4102df0 0x7f95e4106e30 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:23:28.010 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:27.281+0000 7f95e8c00700 1 -- 172.21.15.47:0/904152918 >> 172.21.15.47:0/904152918 conn(0x7f95e40792f0 msgr2=0x7f95e4079700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:23:28.010 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:27.282+0000 7f95e8c00700 1 -- 172.21.15.47:0/904152918 shutdown_connections 2024-03-21T10:23:28.010 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:27.282+0000 7f95e8c00700 1 -- 172.21.15.47:0/904152918 wait complete. 2024-03-21T10:23:28.010 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:27.283+0000 7f95e8c00700 1 Processor -- start 2024-03-21T10:23:28.010 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:27.283+0000 7f95e8c00700 1 -- start start 2024-03-21T10:23:28.010 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:27.284+0000 7f95e8c00700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f95e4102440 0x7f95e419e690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:23:28.010 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:27.284+0000 7f95e8c00700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f95e4102df0 0x7f95e419ebd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:23:28.010 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:27.284+0000 7f95e8c00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f95e4107510 0x7f95e4198810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:23:28.010 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:27.284+0000 7f95e8c00700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95e41123f0 con 0x7f95e4102df0 2024-03-21T10:23:28.011 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:27.284+0000 7f95e8c00700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95e4112270 con 0x7f95e4102440 2024-03-21T10:23:28.011 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:27.284+0000 7f95e8c00700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95e4112570 con 0x7f95e4107510 2024-03-21T10:23:28.011 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:27.284+0000 7f95e1400700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f95e4102df0 0x7f95e419ebd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:23:28.011 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:27.284+0000 7f95e1e00700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f95e4102440 0x7f95e419e690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:23:28.011 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:27.284+0000 7f95e1400700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f95e4102df0 0x7f95e419ebd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:48918/0 (socket says 172.21.15.47:48918) 2024-03-21T10:23:28.011 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:27.284+0000 7f95e1400700 1 -- 172.21.15.47:0/206373765 learned_addr learned my addr 172.21.15.47:0/206373765 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:23:28.011 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:27.285+0000 7f95e2800700 1 --2- 172.21.15.47:0/206373765 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f95e4107510 0x7f95e4198810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:23:28.011 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:27.285+0000 7f95e1400700 1 -- 172.21.15.47:0/206373765 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f95e4107510 msgr2=0x7f95e4198810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:23:28.011 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:27.285+0000 7f95e1400700 1 --2- 172.21.15.47:0/206373765 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f95e4107510 0x7f95e4198810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:23:28.011 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:27.285+0000 7f95e1400700 1 -- 172.21.15.47:0/206373765 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f95e4102440 msgr2=0x7f95e419e690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:23:28.011 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:27.285+0000 7f95e1400700 1 --2- 172.21.15.47:0/206373765 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f95e4102440 0x7f95e419e690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:23:28.011 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:27.285+0000 7f95e1400700 1 -- 172.21.15.47:0/206373765 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95e4198ff0 con 0x7f95e4102df0 2024-03-21T10:23:28.012 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:27.285+0000 7f95e1400700 1 --2- 172.21.15.47:0/206373765 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f95e4102df0 0x7f95e419ebd0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f95d000b580 tx=0x7f95d00045e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:23:28.012 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:27.286+0000 7f95daa00700 1 -- 172.21.15.47:0/206373765 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95d0009e60 con 0x7f95e4102df0 2024-03-21T10:23:28.012 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:27.286+0000 7f95daa00700 1 -- 172.21.15.47:0/206373765 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f95d00047f0 con 0x7f95e4102df0 2024-03-21T10:23:28.012 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:27.286+0000 7f95daa00700 1 -- 172.21.15.47:0/206373765 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95d0005250 con 0x7f95e4102df0 2024-03-21T10:23:28.012 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:27.286+0000 7f95e8c00700 1 -- 172.21.15.47:0/206373765 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f95e4199280 con 0x7f95e4102df0 2024-03-21T10:23:28.012 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:27.286+0000 7f95e8c00700 1 -- 172.21.15.47:0/206373765 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f95e41a54f0 con 0x7f95e4102df0 2024-03-21T10:23:28.012 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:27.287+0000 7f95e8c00700 1 -- 172.21.15.47:0/206373765 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f95e4103910 con 0x7f95e4102df0 2024-03-21T10:23:28.012 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:27.289+0000 7f95daa00700 1 -- 172.21.15.47:0/206373765 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 17) v1 ==== 137905+0+0 (secure 0 0 0) 0x7f95d00049a0 con 0x7f95e4102df0 2024-03-21T10:23:28.012 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:27.291+0000 7f95daa00700 1 --2- 172.21.15.47:0/206373765 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f95c80a30c0 0x7f95c80a5580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:23:28.012 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:27.291+0000 7f95daa00700 1 -- 172.21.15.47:0/206373765 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9216+0+0 (secure 0 0 0) 0x7f95d00d0860 con 0x7f95e4102df0 2024-03-21T10:23:28.013 INFO:teuthology.orchestra.run.smithi047.stdout:34359738477 2024-03-21T10:23:28.013 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:27.291+0000 7f95e1e00700 1 --2- 172.21.15.47:0/206373765 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f95c80a30c0 0x7f95c80a5580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:23:28.013 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:27.292+0000 7f95e1e00700 1 --2- 172.21.15.47:0/206373765 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f95c80a30c0 0x7f95c80a5580 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f95d4004630 tx=0x7f95d400b3c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:23:28.013 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:27.293+0000 7f95daa00700 1 -- 172.21.15.47:0/206373765 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f95d0013070 con 0x7f95e4102df0 2024-03-21T10:23:28.013 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:27.564+0000 7f95e8c00700 1 -- 172.21.15.47:0/206373765 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f95e4199d40 con 0x7f95e4102df0 2024-03-21T10:23:28.013 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:27.565+0000 7f95daa00700 1 -- 172.21.15.47:0/206373765 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f95d009d2d0 con 0x7f95e4102df0 2024-03-21T10:23:28.013 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:27.568+0000 7f95e8c00700 1 -- 172.21.15.47:0/206373765 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f95c80a30c0 msgr2=0x7f95c80a5580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:23:28.013 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:27.568+0000 7f95e8c00700 1 --2- 172.21.15.47:0/206373765 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f95c80a30c0 0x7f95c80a5580 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f95d4004630 tx=0x7f95d400b3c0 comp rx=0 tx=0).stop 2024-03-21T10:23:28.013 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:27.568+0000 7f95e8c00700 1 -- 172.21.15.47:0/206373765 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f95e4102df0 msgr2=0x7f95e419ebd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:23:28.013 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:27.568+0000 7f95e8c00700 1 --2- 172.21.15.47:0/206373765 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f95e4102df0 0x7f95e419ebd0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f95d000b580 tx=0x7f95d00045e0 comp rx=0 tx=0).stop 2024-03-21T10:23:28.014 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:27.568+0000 7f95e8c00700 1 -- 172.21.15.47:0/206373765 shutdown_connections 2024-03-21T10:23:28.014 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:27.568+0000 7f95e8c00700 1 --2- 172.21.15.47:0/206373765 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f95c80a30c0 0x7f95c80a5580 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:23:28.014 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:27.568+0000 7f95e8c00700 1 --2- 172.21.15.47:0/206373765 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f95e4107510 0x7f95e4198810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:23:28.014 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:27.568+0000 7f95e8c00700 1 --2- 172.21.15.47:0/206373765 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f95e4102440 0x7f95e419e690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:23:28.014 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:27.568+0000 7f95e8c00700 1 --2- 172.21.15.47:0/206373765 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f95e4102df0 0x7f95e419ebd0 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:23:28.014 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:27.568+0000 7f95e8c00700 1 -- 172.21.15.47:0/206373765 >> 172.21.15.47:0/206373765 conn(0x7f95e40792f0 msgr2=0x7f95e4100500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:23:28.014 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:27.568+0000 7f95e8c00700 1 -- 172.21.15.47:0/206373765 shutdown_connections 2024-03-21T10:23:28.014 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:27.568+0000 7f95e8c00700 1 -- 172.21.15.47:0/206373765 wait complete. 2024-03-21T10:23:28.755 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:23:28 smithi047 ceph-mon[31547]: osdmap e95: 12 total, 12 up, 12 in 2024-03-21T10:23:28.979 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:23:28 smithi100 ceph-mon[35193]: osdmap e95: 12 total, 12 up, 12 in 2024-03-21T10:23:28.989 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:23:28 smithi157 ceph-mon[35581]: osdmap e95: 12 total, 12 up, 12 in 2024-03-21T10:23:29.901 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:23:29 smithi047 ceph-mon[31547]: pgmap v416: 129 pgs: 1 peering, 128 active+clean; 642 KiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 8 op/s 2024-03-21T10:23:29.901 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:23:29 smithi047 ceph-mon[31547]: Health check failed: Reduced data availability: 1 pg inactive, 1 pg peering (PG_AVAILABILITY) 2024-03-21T10:23:29.979 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:23:29 smithi100 ceph-mon[35193]: pgmap v416: 129 pgs: 1 peering, 128 active+clean; 642 KiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 8 op/s 2024-03-21T10:23:29.979 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:23:29 smithi100 ceph-mon[35193]: Health check failed: Reduced data availability: 1 pg inactive, 1 pg peering (PG_AVAILABILITY) 2024-03-21T10:23:29.990 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:23:29 smithi157 ceph-mon[35581]: pgmap v416: 129 pgs: 1 peering, 128 active+clean; 642 KiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 8 op/s 2024-03-21T10:23:29.990 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:23:29 smithi157 ceph-mon[35581]: Health check failed: Reduced data availability: 1 pg inactive, 1 pg peering (PG_AVAILABILITY) 2024-03-21T10:23:31.647 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:23:31 smithi047 ceph-mon[31547]: pgmap v417: 129 pgs: 1 peering, 128 active+clean; 642 KiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 12 op/s; 14 B/s, 0 objects/s recovering 2024-03-21T10:23:31.979 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:23:31 smithi100 ceph-mon[35193]: pgmap v417: 129 pgs: 1 peering, 128 active+clean; 642 KiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 12 op/s; 14 B/s, 0 objects/s recovering 2024-03-21T10:23:31.989 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:23:31 smithi157 ceph-mon[35581]: pgmap v417: 129 pgs: 1 peering, 128 active+clean; 642 KiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 12 op/s; 14 B/s, 0 objects/s recovering 2024-03-21T10:23:32.189 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:32.187+0000 7f7c71800700 1 -- 172.21.15.47:0/1439071099 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7c6c069e40 msgr2=0x7f7c6c06a2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:23:32.189 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:32.187+0000 7f7c71800700 1 --2- 172.21.15.47:0/1439071099 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7c6c069e40 0x7f7c6c06a2a0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f7c580099c0 tx=0x7f7c58009cd0 comp rx=0 tx=0).stop 2024-03-21T10:23:32.190 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:32.187+0000 7f7c71800700 1 -- 172.21.15.47:0/1439071099 shutdown_connections 2024-03-21T10:23:32.190 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:32.187+0000 7f7c71800700 1 --2- 172.21.15.47:0/1439071099 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7c6c06a7e0 0x7f7c6c10f020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:23:32.190 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:32.187+0000 7f7c71800700 1 --2- 172.21.15.47:0/1439071099 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7c6c1081e0 0x7f7c6c1085c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:23:32.191 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:32.187+0000 7f7c71800700 1 --2- 172.21.15.47:0/1439071099 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7c6c069e40 0x7f7c6c06a2a0 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:23:32.191 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:32.187+0000 7f7c71800700 1 -- 172.21.15.47:0/1439071099 >> 172.21.15.47:0/1439071099 conn(0x7f7c6c077fc0 msgr2=0x7f7c6c0783d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:23:32.191 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:32.188+0000 7f7c71800700 1 -- 172.21.15.47:0/1439071099 shutdown_connections 2024-03-21T10:23:32.191 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:32.189+0000 7f7c71800700 1 -- 172.21.15.47:0/1439071099 wait complete. 2024-03-21T10:23:32.191 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:32.190+0000 7f7c71800700 1 Processor -- start 2024-03-21T10:23:32.191 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:32.190+0000 7f7c71800700 1 -- start start 2024-03-21T10:23:32.192 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:32.191+0000 7f7c71800700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7c6c069e40 0x7f7c6c1a06b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:23:32.192 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:32.191+0000 7f7c71800700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7c6c06a7e0 0x7f7c6c1a0bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:23:32.192 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:32.191+0000 7f7c71800700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7c6c1081e0 0x7f7c6c19a830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:23:32.192 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:32.191+0000 7f7c71800700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c6c112310 con 0x7f7c6c1081e0 2024-03-21T10:23:32.192 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:32.191+0000 7f7c71800700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c6c112190 con 0x7f7c6c069e40 2024-03-21T10:23:32.192 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:32.191+0000 7f7c71800700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c6c112490 con 0x7f7c6c06a7e0 2024-03-21T10:23:32.193 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:32.191+0000 7f7c6b400700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7c6c1081e0 0x7f7c6c19a830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:23:32.193 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:32.191+0000 7f7c6aa00700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7c6c069e40 0x7f7c6c1a06b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:23:32.193 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:32.191+0000 7f7c6aa00700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7c6c069e40 0x7f7c6c1a06b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:38860/0 (socket says 172.21.15.47:38860) 2024-03-21T10:23:32.193 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:32.191+0000 7f7c6aa00700 1 -- 172.21.15.47:0/1858437380 learned_addr learned my addr 172.21.15.47:0/1858437380 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:23:32.193 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:32.192+0000 7f7c6a000700 1 --2- 172.21.15.47:0/1858437380 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7c6c06a7e0 0x7f7c6c1a0bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:23:32.194 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:32.192+0000 7f7c6b400700 1 -- 172.21.15.47:0/1858437380 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7c6c06a7e0 msgr2=0x7f7c6c1a0bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:23:32.194 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:32.192+0000 7f7c6b400700 1 --2- 172.21.15.47:0/1858437380 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7c6c06a7e0 0x7f7c6c1a0bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:23:32.194 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:32.192+0000 7f7c6b400700 1 -- 172.21.15.47:0/1858437380 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7c6c069e40 msgr2=0x7f7c6c1a06b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:23:32.194 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:32.192+0000 7f7c6b400700 1 --2- 172.21.15.47:0/1858437380 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7c6c069e40 0x7f7c6c1a06b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:23:32.194 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:32.192+0000 7f7c6b400700 1 -- 172.21.15.47:0/1858437380 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7c6c19b010 con 0x7f7c6c1081e0 2024-03-21T10:23:32.194 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:32.192+0000 7f7c6aa00700 1 --2- 172.21.15.47:0/1858437380 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7c6c069e40 0x7f7c6c1a06b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T10:23:32.194 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:32.192+0000 7f7c6b400700 1 --2- 172.21.15.47:0/1858437380 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7c6c1081e0 0x7f7c6c19a830 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f7c64007f10 tx=0x7f7c6400d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:23:32.195 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:32.193+0000 7f7c63400700 1 -- 172.21.15.47:0/1858437380 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c64004db0 con 0x7f7c6c1081e0 2024-03-21T10:23:32.195 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:32.193+0000 7f7c63400700 1 -- 172.21.15.47:0/1858437380 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f7c64009e20 con 0x7f7c6c1081e0 2024-03-21T10:23:32.195 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:32.193+0000 7f7c71800700 1 -- 172.21.15.47:0/1858437380 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7c6c19b300 con 0x7f7c6c1081e0 2024-03-21T10:23:32.195 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:32.193+0000 7f7c63400700 1 -- 172.21.15.47:0/1858437380 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c64015400 con 0x7f7c6c1081e0 2024-03-21T10:23:32.195 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:32.193+0000 7f7c71800700 1 -- 172.21.15.47:0/1858437380 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7c6c1a7510 con 0x7f7c6c1081e0 2024-03-21T10:23:32.195 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:32.195+0000 7f7c71800700 1 -- 172.21.15.47:0/1858437380 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7c6c103890 con 0x7f7c6c1081e0 2024-03-21T10:23:32.197 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:32.196+0000 7f7c63400700 1 -- 172.21.15.47:0/1858437380 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 17) v1 ==== 137905+0+0 (secure 0 0 0) 0x7f7c6400db20 con 0x7f7c6c1081e0 2024-03-21T10:23:32.200 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:32.199+0000 7f7c63400700 1 --2- 172.21.15.47:0/1858437380 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f7c500a3040 0x7f7c500a5500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:23:32.200 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:32.199+0000 7f7c63400700 1 -- 172.21.15.47:0/1858437380 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9216+0+0 (secure 0 0 0) 0x7f7c640cf2a0 con 0x7f7c6c1081e0 2024-03-21T10:23:32.200 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:32.199+0000 7f7c6aa00700 1 --2- 172.21.15.47:0/1858437380 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f7c500a3040 0x7f7c500a5500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:23:32.201 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:32.200+0000 7f7c6aa00700 1 --2- 172.21.15.47:0/1858437380 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f7c500a3040 0x7f7c500a5500 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f7c5c011550 tx=0x7f7c5c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:23:32.205 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:32.204+0000 7f7c63400700 1 -- 172.21.15.47:0/1858437380 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f7c64010070 con 0x7f7c6c1081e0 2024-03-21T10:23:32.379 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/3056d51a-e76b-11ee-95cd-87774f69a715/mon.a/config 2024-03-21T10:23:32.456 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:32.455+0000 7f7c71800700 1 -- 172.21.15.47:0/1858437380 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) v1 -- 0x7f7c6c1a7740 con 0x7f7c6c1081e0 2024-03-21T10:23:32.457 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:32.456+0000 7f7c63400700 1 -- 172.21.15.47:0/1858437380 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f7c6409bb60 con 0x7f7c6c1081e0 2024-03-21T10:23:32.457 INFO:teuthology.orchestra.run.smithi047.stdout:150323855444 2024-03-21T10:23:32.460 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:32.459+0000 7f7c71800700 1 -- 172.21.15.47:0/1858437380 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f7c500a3040 msgr2=0x7f7c500a5500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:23:32.460 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:32.459+0000 7f7c71800700 1 --2- 172.21.15.47:0/1858437380 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f7c500a3040 0x7f7c500a5500 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f7c5c011550 tx=0x7f7c5c009260 comp rx=0 tx=0).stop 2024-03-21T10:23:32.460 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:32.459+0000 7f7c71800700 1 -- 172.21.15.47:0/1858437380 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7c6c1081e0 msgr2=0x7f7c6c19a830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:23:32.460 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:32.459+0000 7f7c71800700 1 --2- 172.21.15.47:0/1858437380 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7c6c1081e0 0x7f7c6c19a830 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f7c64007f10 tx=0x7f7c6400d3b0 comp rx=0 tx=0).stop 2024-03-21T10:23:32.460 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:32.459+0000 7f7c71800700 1 -- 172.21.15.47:0/1858437380 shutdown_connections 2024-03-21T10:23:32.460 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:32.459+0000 7f7c71800700 1 --2- 172.21.15.47:0/1858437380 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f7c500a3040 0x7f7c500a5500 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:23:32.461 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:32.459+0000 7f7c71800700 1 --2- 172.21.15.47:0/1858437380 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7c6c1081e0 0x7f7c6c19a830 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:23:32.461 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:32.459+0000 7f7c71800700 1 --2- 172.21.15.47:0/1858437380 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7c6c069e40 0x7f7c6c1a06b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:23:32.461 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:32.459+0000 7f7c71800700 1 --2- 172.21.15.47:0/1858437380 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7c6c06a7e0 0x7f7c6c1a0bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:23:32.461 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:32.459+0000 7f7c71800700 1 -- 172.21.15.47:0/1858437380 >> 172.21.15.47:0/1858437380 conn(0x7f7c6c077fc0 msgr2=0x7f7c6c10d370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:23:32.461 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:32.459+0000 7f7c71800700 1 -- 172.21.15.47:0/1858437380 shutdown_connections 2024-03-21T10:23:32.461 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:32.459+0000 7f7c71800700 1 -- 172.21.15.47:0/1858437380 wait complete. 2024-03-21T10:23:32.713 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:23:32 smithi047 ceph-mon[31547]: from='client.? 172.21.15.47:0/2736068741' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-03-21T10:23:32.714 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:23:32 smithi047 ceph-mon[31547]: from='client.? 172.21.15.47:0/1858437380' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-03-21T10:23:32.930 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:32.059+0000 7fdde9400700 1 -- 172.21.15.47:0/3930941796 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdde41081e0 msgr2=0x7fdde41085c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:23:32.930 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:32.059+0000 7fdde9400700 1 --2- 172.21.15.47:0/3930941796 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdde41081e0 0x7fdde41085c0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7fddd40099c0 tx=0x7fddd4009cd0 comp rx=0 tx=0).stop 2024-03-21T10:23:32.930 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:32.059+0000 7fdde9400700 1 -- 172.21.15.47:0/3930941796 shutdown_connections 2024-03-21T10:23:32.930 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:32.059+0000 7fdde9400700 1 --2- 172.21.15.47:0/3930941796 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdde406a7e0 0x7fdde410f020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:23:32.930 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:32.059+0000 7fdde9400700 1 --2- 172.21.15.47:0/3930941796 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdde41081e0 0x7fdde41085c0 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:23:32.930 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:32.059+0000 7fdde9400700 1 --2- 172.21.15.47:0/3930941796 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdde4069e40 0x7fdde406a2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:23:32.930 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:32.059+0000 7fdde9400700 1 -- 172.21.15.47:0/3930941796 >> 172.21.15.47:0/3930941796 conn(0x7fdde4077fc0 msgr2=0x7fdde40783d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:23:32.930 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:32.059+0000 7fdde9400700 1 -- 172.21.15.47:0/3930941796 shutdown_connections 2024-03-21T10:23:32.931 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:32.059+0000 7fdde9400700 1 -- 172.21.15.47:0/3930941796 wait complete. 2024-03-21T10:23:32.931 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:32.060+0000 7fdde9400700 1 Processor -- start 2024-03-21T10:23:32.931 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:32.061+0000 7fdde9400700 1 -- start start 2024-03-21T10:23:32.931 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:32.061+0000 7fdde9400700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdde4069e40 0x7fdde41a06b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:23:32.931 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:32.061+0000 7fdde9400700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdde406a7e0 0x7fdde41a0bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:23:32.931 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:32.061+0000 7fdde9400700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdde41081e0 0x7fdde419a870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:23:32.931 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:32.061+0000 7fdde9400700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdde41122f0 con 0x7fdde41081e0 2024-03-21T10:23:32.931 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:32.061+0000 7fdde9400700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdde4112170 con 0x7fdde4069e40 2024-03-21T10:23:32.931 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:32.061+0000 7fdde9400700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdde4112470 con 0x7fdde406a7e0 2024-03-21T10:23:32.931 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:32.061+0000 7fdde3200700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdde41081e0 0x7fdde419a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:23:32.931 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:32.061+0000 7fdde3200700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdde41081e0 0x7fdde419a870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:48086/0 (socket says 172.21.15.47:48086) 2024-03-21T10:23:32.931 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:32.061+0000 7fdde3200700 1 -- 172.21.15.47:0/2736068741 learned_addr learned my addr 172.21.15.47:0/2736068741 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:23:32.931 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:32.062+0000 7fdde2800700 1 --2- 172.21.15.47:0/2736068741 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdde4069e40 0x7fdde41a06b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:23:32.932 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:32.062+0000 7fdde1e00700 1 --2- 172.21.15.47:0/2736068741 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdde406a7e0 0x7fdde41a0bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:23:32.932 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:32.062+0000 7fdde3200700 1 -- 172.21.15.47:0/2736068741 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdde406a7e0 msgr2=0x7fdde41a0bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:23:32.932 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:32.062+0000 7fdde3200700 1 --2- 172.21.15.47:0/2736068741 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdde406a7e0 0x7fdde41a0bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:23:32.932 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:32.062+0000 7fdde3200700 1 -- 172.21.15.47:0/2736068741 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdde4069e40 msgr2=0x7fdde41a06b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:23:32.932 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:32.062+0000 7fdde3200700 1 --2- 172.21.15.47:0/2736068741 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdde4069e40 0x7fdde41a06b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:23:32.932 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:32.062+0000 7fdde3200700 1 -- 172.21.15.47:0/2736068741 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdde419afd0 con 0x7fdde41081e0 2024-03-21T10:23:32.932 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:32.062+0000 7fdde2800700 1 --2- 172.21.15.47:0/2736068741 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdde4069e40 0x7fdde41a06b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T10:23:32.932 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:32.062+0000 7fdde1e00700 1 --2- 172.21.15.47:0/2736068741 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdde406a7e0 0x7fdde41a0bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T10:23:32.932 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:32.062+0000 7fdde3200700 1 --2- 172.21.15.47:0/2736068741 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdde41081e0 0x7fdde419a870 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7fdddc007ee0 tx=0x7fdddc00d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:23:32.932 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:32.063+0000 7fdddb400700 1 -- 172.21.15.47:0/2736068741 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdddc00dbd0 con 0x7fdde41081e0 2024-03-21T10:23:32.932 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:32.063+0000 7fdddb400700 1 -- 172.21.15.47:0/2736068741 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fdddc00dd70 con 0x7fdde41081e0 2024-03-21T10:23:32.932 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:32.063+0000 7fdde9400700 1 -- 172.21.15.47:0/2736068741 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdde419b2c0 con 0x7fdde41081e0 2024-03-21T10:23:32.932 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:32.063+0000 7fdde9400700 1 -- 172.21.15.47:0/2736068741 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdde41a7580 con 0x7fdde41081e0 2024-03-21T10:23:32.932 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:32.063+0000 7fdddb400700 1 -- 172.21.15.47:0/2736068741 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdddc005a20 con 0x7fdde41081e0 2024-03-21T10:23:32.933 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:32.064+0000 7fdde9400700 1 -- 172.21.15.47:0/2736068741 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdde4068e50 con 0x7fdde41081e0 2024-03-21T10:23:32.933 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:32.066+0000 7fdddb400700 1 -- 172.21.15.47:0/2736068741 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 17) v1 ==== 137905+0+0 (secure 0 0 0) 0x7fdddc01a020 con 0x7fdde41081e0 2024-03-21T10:23:32.933 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:32.068+0000 7fdddb400700 1 --2- 172.21.15.47:0/2736068741 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7fddc80a3040 0x7fddc80a5500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:23:32.933 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:32.068+0000 7fdddb400700 1 -- 172.21.15.47:0/2736068741 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9216+0+0 (secure 0 0 0) 0x7fdddc0cf130 con 0x7fdde41081e0 2024-03-21T10:23:32.933 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:32.068+0000 7fdde2800700 1 --2- 172.21.15.47:0/2736068741 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7fddc80a3040 0x7fddc80a5500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:23:32.933 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:32.069+0000 7fdde2800700 1 --2- 172.21.15.47:0/2736068741 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7fddc80a3040 0x7fddc80a5500 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fddd4000c00 tx=0x7fddd4005ab0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:23:32.933 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:32.077+0000 7fdddb400700 1 -- 172.21.15.47:0/2736068741 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fdddc010070 con 0x7fdde41081e0 2024-03-21T10:23:32.933 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:32.434+0000 7fdde9400700 1 -- 172.21.15.47:0/2736068741 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7fdde41a7810 con 0x7fdde41081e0 2024-03-21T10:23:32.933 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:32.434+0000 7fdddb400700 1 -- 172.21.15.47:0/2736068741 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fdddc09b9f0 con 0x7fdde41081e0 2024-03-21T10:23:32.933 INFO:teuthology.orchestra.run.smithi047.stdout:77309411426 2024-03-21T10:23:32.934 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:32.437+0000 7fdde9400700 1 -- 172.21.15.47:0/2736068741 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7fddc80a3040 msgr2=0x7fddc80a5500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:23:32.934 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:32.437+0000 7fdde9400700 1 --2- 172.21.15.47:0/2736068741 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7fddc80a3040 0x7fddc80a5500 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fddd4000c00 tx=0x7fddd4005ab0 comp rx=0 tx=0).stop 2024-03-21T10:23:32.934 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:32.437+0000 7fdde9400700 1 -- 172.21.15.47:0/2736068741 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdde41081e0 msgr2=0x7fdde419a870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:23:32.934 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:32.437+0000 7fdde9400700 1 --2- 172.21.15.47:0/2736068741 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdde41081e0 0x7fdde419a870 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7fdddc007ee0 tx=0x7fdddc00d3b0 comp rx=0 tx=0).stop 2024-03-21T10:23:32.934 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:32.438+0000 7fdde9400700 1 -- 172.21.15.47:0/2736068741 shutdown_connections 2024-03-21T10:23:32.934 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:32.438+0000 7fdde9400700 1 --2- 172.21.15.47:0/2736068741 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7fddc80a3040 0x7fddc80a5500 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:23:32.934 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:32.438+0000 7fdde9400700 1 --2- 172.21.15.47:0/2736068741 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdde41081e0 0x7fdde419a870 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:23:32.934 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:32.438+0000 7fdde9400700 1 --2- 172.21.15.47:0/2736068741 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdde4069e40 0x7fdde41a06b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:23:32.935 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:32.438+0000 7fdde9400700 1 --2- 172.21.15.47:0/2736068741 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdde406a7e0 0x7fdde41a0bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:23:32.935 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:32.438+0000 7fdde9400700 1 -- 172.21.15.47:0/2736068741 >> 172.21.15.47:0/2736068741 conn(0x7fdde4077fc0 msgr2=0x7fdde410d370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:23:32.935 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:32.438+0000 7fdde9400700 1 -- 172.21.15.47:0/2736068741 shutdown_connections 2024-03-21T10:23:32.935 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:32.438+0000 7fdde9400700 1 -- 172.21.15.47:0/2736068741 wait complete. 2024-03-21T10:23:32.979 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:23:32 smithi100 ceph-mon[35193]: from='client.? 172.21.15.47:0/2736068741' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-03-21T10:23:32.980 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:23:32 smithi100 ceph-mon[35193]: from='client.? 172.21.15.47:0/1858437380' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-03-21T10:23:32.989 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:23:32 smithi157 ceph-mon[35581]: from='client.? 172.21.15.47:0/2736068741' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-03-21T10:23:32.990 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:23:32 smithi157 ceph-mon[35581]: from='client.? 172.21.15.47:0/1858437380' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-03-21T10:23:33.874 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:23:33 smithi047 ceph-mon[31547]: pgmap v418: 129 pgs: 1 peering, 128 active+clean; 642 KiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s; 14 B/s, 0 objects/s recovering 2024-03-21T10:23:33.979 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:23:33 smithi100 ceph-mon[35193]: pgmap v418: 129 pgs: 1 peering, 128 active+clean; 642 KiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s; 14 B/s, 0 objects/s recovering 2024-03-21T10:23:33.989 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:23:33 smithi157 ceph-mon[35581]: pgmap v418: 129 pgs: 1 peering, 128 active+clean; 642 KiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s; 14 B/s, 0 objects/s recovering 2024-03-21T10:23:34.624 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:34.548+0000 7f9d0de00700 1 -- 172.21.15.47:0/3911503608 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9d0810eb90 msgr2=0x7f9d08110f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:23:34.625 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:34.548+0000 7f9d0de00700 1 --2- 172.21.15.47:0/3911503608 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9d0810eb90 0x7f9d08110f80 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f9d0000b0a0 tx=0x7f9d0000b3b0 comp rx=0 tx=0).stop 2024-03-21T10:23:34.625 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:34.548+0000 7f9d0de00700 1 -- 172.21.15.47:0/3911503608 shutdown_connections 2024-03-21T10:23:34.625 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:34.548+0000 7f9d0de00700 1 --2- 172.21.15.47:0/3911503608 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9d0810eb90 0x7f9d08110f80 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:23:34.625 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:34.548+0000 7f9d0de00700 1 --2- 172.21.15.47:0/3911503608 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9d080743c0 0x7f9d080747a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:23:34.625 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:34.548+0000 7f9d0de00700 1 --2- 172.21.15.47:0/3911503608 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9d08074ce0 0x7f9d0810e650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:23:34.625 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:34.548+0000 7f9d0de00700 1 -- 172.21.15.47:0/3911503608 >> 172.21.15.47:0/3911503608 conn(0x7f9d080792c0 msgr2=0x7f9d080796d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:23:34.625 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:34.548+0000 7f9d0de00700 1 -- 172.21.15.47:0/3911503608 shutdown_connections 2024-03-21T10:23:34.625 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:34.549+0000 7f9d0de00700 1 -- 172.21.15.47:0/3911503608 wait complete. 2024-03-21T10:23:34.625 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:34.550+0000 7f9d0de00700 1 Processor -- start 2024-03-21T10:23:34.625 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:34.550+0000 7f9d0de00700 1 -- start start 2024-03-21T10:23:34.625 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:34.551+0000 7f9d0de00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9d080743c0 0x7f9d081a0740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:23:34.626 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:34.551+0000 7f9d0de00700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9d08074ce0 0x7f9d081a0c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:23:34.626 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:34.551+0000 7f9d0de00700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9d0810eb90 0x7f9d0819a900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:23:34.626 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:34.551+0000 7f9d0de00700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d08114580 con 0x7f9d08074ce0 2024-03-21T10:23:34.626 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:34.551+0000 7f9d0de00700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d08114400 con 0x7f9d0810eb90 2024-03-21T10:23:34.626 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:34.551+0000 7f9d0de00700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d08114700 con 0x7f9d080743c0 2024-03-21T10:23:34.626 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:34.551+0000 7f9d06000700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9d08074ce0 0x7f9d081a0c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:23:34.626 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:34.552+0000 7f9d06000700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9d08074ce0 0x7f9d081a0c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:48122/0 (socket says 172.21.15.47:48122) 2024-03-21T10:23:34.626 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:34.552+0000 7f9d06000700 1 -- 172.21.15.47:0/4176925223 learned_addr learned my addr 172.21.15.47:0/4176925223 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:23:34.626 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:34.552+0000 7f9d06a00700 1 --2- 172.21.15.47:0/4176925223 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9d080743c0 0x7f9d081a0740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:23:34.626 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:34.552+0000 7f9d07400700 1 --2- 172.21.15.47:0/4176925223 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9d0810eb90 0x7f9d0819a900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:23:34.626 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:34.552+0000 7f9d06000700 1 -- 172.21.15.47:0/4176925223 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9d080743c0 msgr2=0x7f9d081a0740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:23:34.626 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:34.552+0000 7f9d06000700 1 --2- 172.21.15.47:0/4176925223 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9d080743c0 0x7f9d081a0740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:23:34.626 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:34.552+0000 7f9d06000700 1 -- 172.21.15.47:0/4176925223 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9d0810eb90 msgr2=0x7f9d0819a900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:23:34.627 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:34.552+0000 7f9d06000700 1 --2- 172.21.15.47:0/4176925223 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9d0810eb90 0x7f9d0819a900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:23:34.627 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:34.552+0000 7f9d06000700 1 -- 172.21.15.47:0/4176925223 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d0819b060 con 0x7f9d08074ce0 2024-03-21T10:23:34.627 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:34.553+0000 7f9d06000700 1 --2- 172.21.15.47:0/4176925223 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9d08074ce0 0x7f9d081a0c80 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f9cf400e9c0 tx=0x7f9cf400ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:23:34.627 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:34.553+0000 7f9d0ca00700 1 -- 172.21.15.47:0/4176925223 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9cf400cbd0 con 0x7f9d08074ce0 2024-03-21T10:23:34.627 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:34.553+0000 7f9d0ca00700 1 -- 172.21.15.47:0/4176925223 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f9cf400cd70 con 0x7f9d08074ce0 2024-03-21T10:23:34.627 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:34.553+0000 7f9d0de00700 1 -- 172.21.15.47:0/4176925223 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9d0819b350 con 0x7f9d08074ce0 2024-03-21T10:23:34.628 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:34.554+0000 7f9d0de00700 1 -- 172.21.15.47:0/4176925223 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9d081a7610 con 0x7f9d08074ce0 2024-03-21T10:23:34.628 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:34.554+0000 7f9d0ca00700 1 -- 172.21.15.47:0/4176925223 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9cf40056c0 con 0x7f9d08074ce0 2024-03-21T10:23:34.628 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:34.556+0000 7f9d0de00700 1 -- 172.21.15.47:0/4176925223 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9d08068e50 con 0x7f9d08074ce0 2024-03-21T10:23:34.628 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:34.557+0000 7f9d0ca00700 1 -- 172.21.15.47:0/4176925223 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 17) v1 ==== 137905+0+0 (secure 0 0 0) 0x7f9cf4004830 con 0x7f9d08074ce0 2024-03-21T10:23:34.628 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:34.559+0000 7f9d0ca00700 1 --2- 172.21.15.47:0/4176925223 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f9cec0a30c0 0x7f9cec0a5580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:23:34.628 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:34.559+0000 7f9d0ca00700 1 -- 172.21.15.47:0/4176925223 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9216+0+0 (secure 0 0 0) 0x7f9cf40cfb20 con 0x7f9d08074ce0 2024-03-21T10:23:34.628 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:34.560+0000 7f9d06a00700 1 --2- 172.21.15.47:0/4176925223 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f9cec0a30c0 0x7f9cec0a5580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:23:34.628 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:34.560+0000 7f9d06a00700 1 --2- 172.21.15.47:0/4176925223 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f9cec0a30c0 0x7f9cec0a5580 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f9d0819ba40 tx=0x7f9cf8009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:23:34.628 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:34.564+0000 7f9d0ca00700 1 -- 172.21.15.47:0/4176925223 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f9cf409c420 con 0x7f9d08074ce0 2024-03-21T10:23:34.802 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:34.801+0000 7f9d0de00700 1 -- 172.21.15.47:0/4176925223 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 11} v 0) v1 -- 0x7f9d0819be30 con 0x7f9d08074ce0 2024-03-21T10:23:34.803 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:23:34 smithi047 ceph-mon[31547]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg inactive, 1 pg peering) 2024-03-21T10:23:34.803 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:23:34 smithi047 ceph-mon[31547]: Cluster is now healthy 2024-03-21T10:23:34.804 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:34.802+0000 7f9d0ca00700 1 -- 172.21.15.47:0/4176925223 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 11}]=0 v0) v1 ==== 75+0+13 (secure 0 0 0) 0x7f9cf409bd20 con 0x7f9d08074ce0 2024-03-21T10:23:34.804 INFO:teuthology.orchestra.run.smithi047.stdout:352187318307 2024-03-21T10:23:34.806 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:34.805+0000 7f9d0de00700 1 -- 172.21.15.47:0/4176925223 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f9cec0a30c0 msgr2=0x7f9cec0a5580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:23:34.806 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:34.805+0000 7f9d0de00700 1 --2- 172.21.15.47:0/4176925223 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f9cec0a30c0 0x7f9cec0a5580 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f9d0819ba40 tx=0x7f9cf8009260 comp rx=0 tx=0).stop 2024-03-21T10:23:34.806 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:34.805+0000 7f9d0de00700 1 -- 172.21.15.47:0/4176925223 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9d08074ce0 msgr2=0x7f9d081a0c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:23:34.806 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:34.805+0000 7f9d0de00700 1 --2- 172.21.15.47:0/4176925223 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9d08074ce0 0x7f9d081a0c80 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f9cf400e9c0 tx=0x7f9cf400ecd0 comp rx=0 tx=0).stop 2024-03-21T10:23:34.806 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:34.805+0000 7f9d0de00700 1 -- 172.21.15.47:0/4176925223 shutdown_connections 2024-03-21T10:23:34.807 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:34.805+0000 7f9d0de00700 1 --2- 172.21.15.47:0/4176925223 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f9cec0a30c0 0x7f9cec0a5580 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:23:34.807 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:34.805+0000 7f9d0de00700 1 --2- 172.21.15.47:0/4176925223 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9d0810eb90 0x7f9d0819a900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:23:34.807 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:34.805+0000 7f9d0de00700 1 --2- 172.21.15.47:0/4176925223 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9d080743c0 0x7f9d081a0740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:23:34.807 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:34.805+0000 7f9d0de00700 1 --2- 172.21.15.47:0/4176925223 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9d08074ce0 0x7f9d081a0c80 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:23:34.807 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:34.805+0000 7f9d0de00700 1 -- 172.21.15.47:0/4176925223 >> 172.21.15.47:0/4176925223 conn(0x7f9d080792c0 msgr2=0x7f9d080ff440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:23:34.807 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:34.805+0000 7f9d0de00700 1 -- 172.21.15.47:0/4176925223 shutdown_connections 2024-03-21T10:23:34.807 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:34.805+0000 7f9d0de00700 1 -- 172.21.15.47:0/4176925223 wait complete. 2024-03-21T10:23:34.979 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:23:34 smithi100 ceph-mon[35193]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg inactive, 1 pg peering) 2024-03-21T10:23:34.980 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:23:34 smithi100 ceph-mon[35193]: Cluster is now healthy 2024-03-21T10:23:34.990 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:23:34 smithi157 ceph-mon[35581]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg inactive, 1 pg peering) 2024-03-21T10:23:34.990 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:23:34 smithi157 ceph-mon[35581]: Cluster is now healthy 2024-03-21T10:23:35.014 INFO:tasks.cephadm.ceph_manager.ceph:need seq 150323855438 got 150323855444 for osd.4 2024-03-21T10:23:35.014 DEBUG:teuthology.parallel:result is None 2024-03-21T10:23:35.814 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:23:35 smithi047 ceph-mon[31547]: pgmap v419: 129 pgs: 129 active+clean; 642 KiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s; 14 B/s, 0 objects/s recovering 2024-03-21T10:23:35.814 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:23:35 smithi047 ceph-mon[31547]: from='client.? 172.21.15.47:0/4176925223' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-03-21T10:23:35.979 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:23:35 smithi100 ceph-mon[35193]: pgmap v419: 129 pgs: 129 active+clean; 642 KiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s; 14 B/s, 0 objects/s recovering 2024-03-21T10:23:35.996 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:23:35 smithi100 ceph-mon[35193]: from='client.? 172.21.15.47:0/4176925223' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-03-21T10:23:35.996 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:23:35 smithi157 ceph-mon[35581]: pgmap v419: 129 pgs: 129 active+clean; 642 KiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s; 14 B/s, 0 objects/s recovering 2024-03-21T10:23:35.996 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:23:35 smithi157 ceph-mon[35581]: from='client.? 172.21.15.47:0/4176925223' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-03-21T10:23:36.046 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:35.527+0000 7fec1f000700 1 -- 172.21.15.47:0/2673980001 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fec181086c0 msgr2=0x7fec18108b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:23:36.046 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:35.527+0000 7fec1f000700 1 --2- 172.21.15.47:0/2673980001 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fec181086c0 0x7fec18108b40 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7fec04009990 tx=0x7fec04009ca0 comp rx=0 tx=0).stop 2024-03-21T10:23:36.046 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:35.527+0000 7fec1f000700 1 -- 172.21.15.47:0/2673980001 shutdown_connections 2024-03-21T10:23:36.046 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:35.527+0000 7fec1f000700 1 --2- 172.21.15.47:0/2673980001 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fec18109080 0x7fec18115540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:23:36.046 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:35.527+0000 7fec1f000700 1 --2- 172.21.15.47:0/2673980001 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fec1810e670 0x7fec1810ea50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:23:36.046 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:35.527+0000 7fec1f000700 1 --2- 172.21.15.47:0/2673980001 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fec181086c0 0x7fec18108b40 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:23:36.047 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:35.527+0000 7fec1f000700 1 -- 172.21.15.47:0/2673980001 >> 172.21.15.47:0/2673980001 conn(0x7fec18077e30 msgr2=0x7fec1807a250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:23:36.047 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:35.527+0000 7fec1f000700 1 -- 172.21.15.47:0/2673980001 shutdown_connections 2024-03-21T10:23:36.047 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:35.527+0000 7fec1f000700 1 -- 172.21.15.47:0/2673980001 wait complete. 2024-03-21T10:23:36.047 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:35.528+0000 7fec1f000700 1 Processor -- start 2024-03-21T10:23:36.047 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:35.529+0000 7fec1f000700 1 -- start start 2024-03-21T10:23:36.047 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:35.529+0000 7fec1f000700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fec181086c0 0x7fec1819ed10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:23:36.047 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:35.529+0000 7fec1f000700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fec18109080 0x7fec1819f250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:23:36.047 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:35.529+0000 7fec1f000700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fec1810e670 0x7fec181a35e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:23:36.047 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:35.529+0000 7fec1f000700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec181188c0 con 0x7fec1810e670 2024-03-21T10:23:36.047 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:35.529+0000 7fec1f000700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec18118740 con 0x7fec181086c0 2024-03-21T10:23:36.047 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:35.529+0000 7fec1f000700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec18118a40 con 0x7fec18109080 2024-03-21T10:23:36.047 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:35.529+0000 7fec1e600700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fec1810e670 0x7fec181a35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:23:36.048 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:35.530+0000 7fec1e600700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fec1810e670 0x7fec181a35e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:48142/0 (socket says 172.21.15.47:48142) 2024-03-21T10:23:36.048 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:35.530+0000 7fec1dc00700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fec181086c0 0x7fec1819ed10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:23:36.048 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:35.530+0000 7fec1e600700 1 -- 172.21.15.47:0/1856715316 learned_addr learned my addr 172.21.15.47:0/1856715316 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:23:36.048 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:35.530+0000 7fec1d200700 1 --2- 172.21.15.47:0/1856715316 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fec18109080 0x7fec1819f250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:23:36.048 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:35.530+0000 7fec1e600700 1 -- 172.21.15.47:0/1856715316 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fec18109080 msgr2=0x7fec1819f250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:23:36.048 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:35.530+0000 7fec1e600700 1 --2- 172.21.15.47:0/1856715316 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fec18109080 0x7fec1819f250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:23:36.048 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:35.530+0000 7fec1e600700 1 -- 172.21.15.47:0/1856715316 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fec181086c0 msgr2=0x7fec1819ed10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:23:36.048 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:35.530+0000 7fec1e600700 1 --2- 172.21.15.47:0/1856715316 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fec181086c0 0x7fec1819ed10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:23:36.048 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:35.530+0000 7fec1e600700 1 -- 172.21.15.47:0/1856715316 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec181a3d20 con 0x7fec1810e670 2024-03-21T10:23:36.048 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:35.530+0000 7fec1d200700 1 --2- 172.21.15.47:0/1856715316 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fec18109080 0x7fec1819f250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T10:23:36.048 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:35.530+0000 7fec1e600700 1 --2- 172.21.15.47:0/1856715316 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fec1810e670 0x7fec181a35e0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7fec10007ee0 tx=0x7fec1000d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:23:36.048 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:35.531+0000 7fec16000700 1 -- 172.21.15.47:0/1856715316 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec1000dbd0 con 0x7fec1810e670 2024-03-21T10:23:36.049 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:35.531+0000 7fec16000700 1 -- 172.21.15.47:0/1856715316 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fec1000dd70 con 0x7fec1810e670 2024-03-21T10:23:36.049 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:35.531+0000 7fec1f000700 1 -- 172.21.15.47:0/1856715316 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fec181a4010 con 0x7fec1810e670 2024-03-21T10:23:36.049 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:35.531+0000 7fec16000700 1 -- 172.21.15.47:0/1856715316 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec10005990 con 0x7fec1810e670 2024-03-21T10:23:36.049 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:35.531+0000 7fec1f000700 1 -- 172.21.15.47:0/1856715316 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fec181ab890 con 0x7fec1810e670 2024-03-21T10:23:36.049 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:35.532+0000 7fec1f000700 1 -- 172.21.15.47:0/1856715316 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fec18068e50 con 0x7fec1810e670 2024-03-21T10:23:36.049 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:35.535+0000 7fec16000700 1 -- 172.21.15.47:0/1856715316 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 17) v1 ==== 137905+0+0 (secure 0 0 0) 0x7fec1001a020 con 0x7fec1810e670 2024-03-21T10:23:36.049 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:35.537+0000 7fec16000700 1 --2- 172.21.15.47:0/1856715316 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7fec000a30c0 0x7fec000a5580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:23:36.049 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:35.537+0000 7fec16000700 1 -- 172.21.15.47:0/1856715316 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9216+0+0 (secure 0 0 0) 0x7fec100cf040 con 0x7fec1810e670 2024-03-21T10:23:36.049 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:35.537+0000 7fec1dc00700 1 --2- 172.21.15.47:0/1856715316 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7fec000a30c0 0x7fec000a5580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:23:36.049 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:35.538+0000 7fec1dc00700 1 --2- 172.21.15.47:0/1856715316 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7fec000a30c0 0x7fec000a5580 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fec0c007b60 tx=0x7fec0c0073d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:23:36.049 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:35.541+0000 7fec16000700 1 -- 172.21.15.47:0/1856715316 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fec10010070 con 0x7fec1810e670 2024-03-21T10:23:36.050 INFO:teuthology.orchestra.run.smithi047.stdout:317827579946 2024-03-21T10:23:36.050 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:35.812+0000 7fec1f000700 1 -- 172.21.15.47:0/1856715316 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 10} v 0) v1 -- 0x7fec1802da30 con 0x7fec1810e670 2024-03-21T10:23:36.050 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:35.813+0000 7fec16000700 1 -- 172.21.15.47:0/1856715316 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 10}]=0 v0) v1 ==== 75+0+13 (secure 0 0 0) 0x7fec1009b900 con 0x7fec1810e670 2024-03-21T10:23:36.050 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:35.817+0000 7fec1f000700 1 -- 172.21.15.47:0/1856715316 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7fec000a30c0 msgr2=0x7fec000a5580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:23:36.050 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:35.817+0000 7fec1f000700 1 --2- 172.21.15.47:0/1856715316 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7fec000a30c0 0x7fec000a5580 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fec0c007b60 tx=0x7fec0c0073d0 comp rx=0 tx=0).stop 2024-03-21T10:23:36.051 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:35.817+0000 7fec1f000700 1 -- 172.21.15.47:0/1856715316 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fec1810e670 msgr2=0x7fec181a35e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:23:36.051 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:35.817+0000 7fec1f000700 1 --2- 172.21.15.47:0/1856715316 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fec1810e670 0x7fec181a35e0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7fec10007ee0 tx=0x7fec1000d3b0 comp rx=0 tx=0).stop 2024-03-21T10:23:36.051 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:35.817+0000 7fec1f000700 1 -- 172.21.15.47:0/1856715316 shutdown_connections 2024-03-21T10:23:36.051 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:35.817+0000 7fec1f000700 1 --2- 172.21.15.47:0/1856715316 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7fec000a30c0 0x7fec000a5580 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:23:36.051 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:35.817+0000 7fec1f000700 1 --2- 172.21.15.47:0/1856715316 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fec1810e670 0x7fec181a35e0 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:23:36.051 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:35.817+0000 7fec1f000700 1 --2- 172.21.15.47:0/1856715316 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fec181086c0 0x7fec1819ed10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:23:36.051 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:35.817+0000 7fec1f000700 1 --2- 172.21.15.47:0/1856715316 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fec18109080 0x7fec1819f250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:23:36.051 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:35.817+0000 7fec1f000700 1 -- 172.21.15.47:0/1856715316 >> 172.21.15.47:0/1856715316 conn(0x7fec18077e30 msgr2=0x7fec18078510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:23:36.051 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:35.820+0000 7fec1f000700 1 -- 172.21.15.47:0/1856715316 shutdown_connections 2024-03-21T10:23:36.051 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:35.820+0000 7fec1f000700 1 -- 172.21.15.47:0/1856715316 wait complete. 2024-03-21T10:23:36.615 INFO:tasks.cephadm.ceph_manager.ceph:need seq 77309411421 got 77309411426 for osd.2 2024-03-21T10:23:36.615 DEBUG:teuthology.parallel:result is None 2024-03-21T10:23:36.862 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:23:36 smithi047 ceph-mon[31547]: from='client.? 172.21.15.47:0/1856715316' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-03-21T10:23:36.958 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:36.880+0000 7fa9e6200700 1 -- 172.21.15.47:0/1584968073 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa9e0107510 msgr2=0x7fa9e010f0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:23:36.959 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:36.880+0000 7fa9e6200700 1 --2- 172.21.15.47:0/1584968073 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa9e0107510 0x7fa9e010f0e0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7fa9d800b0d0 tx=0x7fa9d800b3e0 comp rx=0 tx=0).stop 2024-03-21T10:23:36.959 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:36.880+0000 7fa9e6200700 1 -- 172.21.15.47:0/1584968073 shutdown_connections 2024-03-21T10:23:36.959 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:36.880+0000 7fa9e6200700 1 --2- 172.21.15.47:0/1584968073 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa9e0107510 0x7fa9e010f0e0 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:23:36.959 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:36.880+0000 7fa9e6200700 1 --2- 172.21.15.47:0/1584968073 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa9e0102440 0x7fa9e0102820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:23:36.959 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:36.880+0000 7fa9e6200700 1 --2- 172.21.15.47:0/1584968073 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa9e0102df0 0x7fa9e0106e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:23:36.959 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:36.880+0000 7fa9e6200700 1 -- 172.21.15.47:0/1584968073 >> 172.21.15.47:0/1584968073 conn(0x7fa9e00792f0 msgr2=0x7fa9e0079700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:23:36.959 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:36.880+0000 7fa9e6200700 1 -- 172.21.15.47:0/1584968073 shutdown_connections 2024-03-21T10:23:36.959 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:36.880+0000 7fa9e6200700 1 -- 172.21.15.47:0/1584968073 wait complete. 2024-03-21T10:23:36.959 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:36.881+0000 7fa9e6200700 1 Processor -- start 2024-03-21T10:23:36.959 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:36.882+0000 7fa9e6200700 1 -- start start 2024-03-21T10:23:36.959 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:36.882+0000 7fa9e6200700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa9e0102440 0x7fa9e01a06e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:23:36.959 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:36.882+0000 7fa9e6200700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa9e0102df0 0x7fa9e01a0c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:23:36.960 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:36.882+0000 7fa9e6200700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa9e0107510 0x7fa9e019a8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:23:36.960 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:36.882+0000 7fa9e6200700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9e01123e0 con 0x7fa9e0102440 2024-03-21T10:23:36.960 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:36.882+0000 7fa9e6200700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9e0112260 con 0x7fa9e0107510 2024-03-21T10:23:36.960 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:36.882+0000 7fa9e6200700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9e0112560 con 0x7fa9e0102df0 2024-03-21T10:23:36.960 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:36.882+0000 7fa9df400700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa9e0102440 0x7fa9e01a06e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:23:36.960 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:36.882+0000 7fa9df400700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa9e0102440 0x7fa9e01a06e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:48160/0 (socket says 172.21.15.47:48160) 2024-03-21T10:23:36.960 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:36.882+0000 7fa9df400700 1 -- 172.21.15.47:0/287604139 learned_addr learned my addr 172.21.15.47:0/287604139 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:23:36.960 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:36.882+0000 7fa9dfe00700 1 --2- 172.21.15.47:0/287604139 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa9e0107510 0x7fa9e019a8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:23:36.960 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:36.883+0000 7fa9dea00700 1 --2- 172.21.15.47:0/287604139 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa9e0102df0 0x7fa9e01a0c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:23:36.960 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:36.883+0000 7fa9df400700 1 -- 172.21.15.47:0/287604139 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa9e0102df0 msgr2=0x7fa9e01a0c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:23:36.960 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:36.883+0000 7fa9df400700 1 --2- 172.21.15.47:0/287604139 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa9e0102df0 0x7fa9e01a0c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:23:36.961 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:36.883+0000 7fa9df400700 1 -- 172.21.15.47:0/287604139 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa9e0107510 msgr2=0x7fa9e019a8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:23:36.961 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:36.883+0000 7fa9df400700 1 --2- 172.21.15.47:0/287604139 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa9e0107510 0x7fa9e019a8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:23:36.961 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:36.883+0000 7fa9df400700 1 -- 172.21.15.47:0/287604139 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa9e019b050 con 0x7fa9e0102440 2024-03-21T10:23:36.961 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:36.883+0000 7fa9df400700 1 --2- 172.21.15.47:0/287604139 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa9e0102440 0x7fa9e01a06e0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7fa9d000e9a0 tx=0x7fa9d000ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:23:36.961 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:36.884+0000 7fa9d7e00700 1 -- 172.21.15.47:0/287604139 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9d000ca10 con 0x7fa9e0102440 2024-03-21T10:23:36.961 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:36.884+0000 7fa9d7e00700 1 -- 172.21.15.47:0/287604139 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fa9d000cbb0 con 0x7fa9e0102440 2024-03-21T10:23:36.961 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:36.884+0000 7fa9d7e00700 1 -- 172.21.15.47:0/287604139 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9d0018680 con 0x7fa9e0102440 2024-03-21T10:23:36.961 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:36.884+0000 7fa9e6200700 1 -- 172.21.15.47:0/287604139 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa9e019b340 con 0x7fa9e0102440 2024-03-21T10:23:36.961 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:36.884+0000 7fa9e6200700 1 -- 172.21.15.47:0/287604139 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa9e01a75b0 con 0x7fa9e0102440 2024-03-21T10:23:36.961 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:36.886+0000 7fa9e6200700 1 -- 172.21.15.47:0/287604139 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa9e0068e50 con 0x7fa9e0102440 2024-03-21T10:23:36.961 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:36.889+0000 7fa9d7e00700 1 -- 172.21.15.47:0/287604139 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 17) v1 ==== 137905+0+0 (secure 0 0 0) 0x7fa9d0010430 con 0x7fa9e0102440 2024-03-21T10:23:36.961 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:36.892+0000 7fa9d7e00700 1 --2- 172.21.15.47:0/287604139 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7fa9c40a30c0 0x7fa9c40a5580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:23:36.961 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:36.892+0000 7fa9d7e00700 1 -- 172.21.15.47:0/287604139 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9216+0+0 (secure 0 0 0) 0x7fa9d00cfac0 con 0x7fa9e0102440 2024-03-21T10:23:36.962 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:36.892+0000 7fa9dea00700 1 --2- 172.21.15.47:0/287604139 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7fa9c40a30c0 0x7fa9c40a5580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:23:36.962 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:36.893+0000 7fa9dea00700 1 --2- 172.21.15.47:0/287604139 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7fa9c40a30c0 0x7fa9c40a5580 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fa9e0076420 tx=0x7fa9cc00b3c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:23:36.962 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:36.896+0000 7fa9d7e00700 1 -- 172.21.15.47:0/287604139 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fa9d009c380 con 0x7fa9e0102440 2024-03-21T10:23:36.979 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:23:36 smithi100 ceph-mon[35193]: from='client.? 172.21.15.47:0/1856715316' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-03-21T10:23:36.989 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:23:36 smithi157 ceph-mon[35581]: from='client.? 172.21.15.47:0/1856715316' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-03-21T10:23:37.139 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:37.138+0000 7fa9e6200700 1 -- 172.21.15.47:0/287604139 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) v1 -- 0x7fa9e01a7780 con 0x7fa9e0102440 2024-03-21T10:23:37.140 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:37.139+0000 7fa9d7e00700 1 -- 172.21.15.47:0/287604139 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fa9d009bc80 con 0x7fa9e0102440 2024-03-21T10:23:37.140 INFO:teuthology.orchestra.run.smithi047.stdout:227633266752 2024-03-21T10:23:37.143 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:37.142+0000 7fa9e6200700 1 -- 172.21.15.47:0/287604139 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7fa9c40a30c0 msgr2=0x7fa9c40a5580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:23:37.144 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:37.142+0000 7fa9e6200700 1 --2- 172.21.15.47:0/287604139 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7fa9c40a30c0 0x7fa9c40a5580 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fa9e0076420 tx=0x7fa9cc00b3c0 comp rx=0 tx=0).stop 2024-03-21T10:23:37.144 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:37.142+0000 7fa9e6200700 1 -- 172.21.15.47:0/287604139 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa9e0102440 msgr2=0x7fa9e01a06e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:23:37.144 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:37.142+0000 7fa9e6200700 1 --2- 172.21.15.47:0/287604139 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa9e0102440 0x7fa9e01a06e0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7fa9d000e9a0 tx=0x7fa9d000ecb0 comp rx=0 tx=0).stop 2024-03-21T10:23:37.144 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:37.143+0000 7fa9e6200700 1 -- 172.21.15.47:0/287604139 shutdown_connections 2024-03-21T10:23:37.144 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:37.143+0000 7fa9e6200700 1 --2- 172.21.15.47:0/287604139 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7fa9c40a30c0 0x7fa9c40a5580 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:23:37.144 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:37.143+0000 7fa9e6200700 1 --2- 172.21.15.47:0/287604139 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa9e0107510 0x7fa9e019a8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:23:37.144 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:37.143+0000 7fa9e6200700 1 --2- 172.21.15.47:0/287604139 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa9e0102440 0x7fa9e01a06e0 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:23:37.145 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:37.143+0000 7fa9e6200700 1 --2- 172.21.15.47:0/287604139 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa9e0102df0 0x7fa9e01a0c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:23:37.145 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:37.143+0000 7fa9e6200700 1 -- 172.21.15.47:0/287604139 >> 172.21.15.47:0/287604139 conn(0x7fa9e00792f0 msgr2=0x7fa9e0100420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:23:37.145 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:37.144+0000 7fa9e6200700 1 -- 172.21.15.47:0/287604139 shutdown_connections 2024-03-21T10:23:37.145 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:37.144+0000 7fa9e6200700 1 -- 172.21.15.47:0/287604139 wait complete. 2024-03-21T10:23:37.964 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:23:37 smithi047 ceph-mon[31547]: pgmap v420: 129 pgs: 129 active+clean; 642 KiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 10 op/s; 12 B/s, 0 objects/s recovering 2024-03-21T10:23:37.964 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:23:37 smithi047 ceph-mon[31547]: from='client.? 172.21.15.47:0/287604139' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-03-21T10:23:37.990 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:23:37 smithi157 ceph-mon[35581]: pgmap v420: 129 pgs: 129 active+clean; 642 KiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 10 op/s; 12 B/s, 0 objects/s recovering 2024-03-21T10:23:37.990 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:23:37 smithi157 ceph-mon[35581]: from='client.? 172.21.15.47:0/287604139' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-03-21T10:23:38.229 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:23:37 smithi100 ceph-mon[35193]: pgmap v420: 129 pgs: 129 active+clean; 642 KiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 10 op/s; 12 B/s, 0 objects/s recovering 2024-03-21T10:23:38.229 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:23:37 smithi100 ceph-mon[35193]: from='client.? 172.21.15.47:0/287604139' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-03-21T10:23:38.231 INFO:tasks.cephadm.ceph_manager.ceph:need seq 34359738473 got 34359738477 for osd.0 2024-03-21T10:23:38.231 DEBUG:teuthology.parallel:result is None 2024-03-21T10:23:39.990 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:23:39 smithi157 ceph-mon[35581]: pgmap v421: 129 pgs: 129 active+clean; 642 KiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 10 op/s; 11 B/s, 0 objects/s recovering 2024-03-21T10:23:39.990 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:23:39 smithi157 ceph-mon[35581]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:23:39.990 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:23:39 smithi157 ceph-mon[35581]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:23:40.213 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:23:39 smithi047 ceph-mon[31547]: pgmap v421: 129 pgs: 129 active+clean; 642 KiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 10 op/s; 11 B/s, 0 objects/s recovering 2024-03-21T10:23:40.214 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:23:39 smithi047 ceph-mon[31547]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:23:40.214 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:23:39 smithi047 ceph-mon[31547]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:23:40.229 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:23:39 smithi100 ceph-mon[35193]: pgmap v421: 129 pgs: 129 active+clean; 642 KiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 10 op/s; 11 B/s, 0 objects/s recovering 2024-03-21T10:23:40.229 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:23:39 smithi100 ceph-mon[35193]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:23:40.229 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:23:39 smithi100 ceph-mon[35193]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:23:40.261 INFO:tasks.cephadm.ceph_manager.ceph:need seq 227633266747 got 227633266752 for osd.7 2024-03-21T10:23:40.262 DEBUG:teuthology.parallel:result is None 2024-03-21T10:23:40.393 INFO:tasks.cephadm.ceph_manager.ceph:need seq 317827579939 got 317827579946 for osd.10 2024-03-21T10:23:40.394 DEBUG:teuthology.parallel:result is None 2024-03-21T10:23:41.627 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:41.625+0000 7f4e7f200700 1 -- 172.21.15.47:0/3062623198 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4e781022d0 msgr2=0x7f4e781026b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:23:41.627 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:41.625+0000 7f4e7f200700 1 --2- 172.21.15.47:0/3062623198 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4e781022d0 0x7f4e781026b0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f4e6c009990 tx=0x7f4e6c009ca0 comp rx=0 tx=0).stop 2024-03-21T10:23:41.627 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:41.626+0000 7f4e7f200700 1 -- 172.21.15.47:0/3062623198 shutdown_connections 2024-03-21T10:23:41.628 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:41.626+0000 7f4e7f200700 1 --2- 172.21.15.47:0/3062623198 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4e781073b0 0x7f4e7810ef80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:23:41.628 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:41.626+0000 7f4e7f200700 1 --2- 172.21.15.47:0/3062623198 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4e781022d0 0x7f4e781026b0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:23:41.628 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:41.626+0000 7f4e7f200700 1 --2- 172.21.15.47:0/3062623198 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4e78102c80 0x7f4e78106cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:23:41.628 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:41.626+0000 7f4e7f200700 1 -- 172.21.15.47:0/3062623198 >> 172.21.15.47:0/3062623198 conn(0x7f4e780fdb60 msgr2=0x7f4e780fff80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:23:41.628 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:41.626+0000 7f4e7f200700 1 -- 172.21.15.47:0/3062623198 shutdown_connections 2024-03-21T10:23:41.628 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:41.626+0000 7f4e7f200700 1 -- 172.21.15.47:0/3062623198 wait complete. 2024-03-21T10:23:41.628 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:41.627+0000 7f4e7f200700 1 Processor -- start 2024-03-21T10:23:41.629 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:41.627+0000 7f4e7f200700 1 -- start start 2024-03-21T10:23:41.629 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:41.628+0000 7f4e7f200700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4e781022d0 0x7f4e781a0610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:23:41.630 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:41.628+0000 7f4e7f200700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4e78102c80 0x7f4e781a0b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:23:41.630 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:41.628+0000 7f4e7f200700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4e781073b0 0x7f4e7819a790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:23:41.630 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:41.628+0000 7f4e7f200700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e78112240 con 0x7f4e781022d0 2024-03-21T10:23:41.630 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:41.628+0000 7f4e7f200700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e781120c0 con 0x7f4e781073b0 2024-03-21T10:23:41.630 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:41.628+0000 7f4e7f200700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e781123c0 con 0x7f4e78102c80 2024-03-21T10:23:41.630 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:41.628+0000 7f4e7de00700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4e781022d0 0x7f4e781a0610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:23:41.630 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:41.628+0000 7f4e7de00700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4e781022d0 0x7f4e781a0610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:48166/0 (socket says 172.21.15.47:48166) 2024-03-21T10:23:41.630 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:41.628+0000 7f4e7de00700 1 -- 172.21.15.47:0/3668543176 learned_addr learned my addr 172.21.15.47:0/3668543176 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:23:41.630 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:41.628+0000 7f4e7e800700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4e781073b0 0x7f4e7819a790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:23:41.630 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:41.628+0000 7f4e7d400700 1 --2- 172.21.15.47:0/3668543176 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4e78102c80 0x7f4e781a0b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:23:41.631 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:41.629+0000 7f4e7de00700 1 -- 172.21.15.47:0/3668543176 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4e78102c80 msgr2=0x7f4e781a0b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:23:41.631 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:41.629+0000 7f4e7de00700 1 --2- 172.21.15.47:0/3668543176 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4e78102c80 0x7f4e781a0b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:23:41.631 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:41.629+0000 7f4e7de00700 1 -- 172.21.15.47:0/3668543176 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4e781073b0 msgr2=0x7f4e7819a790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:23:41.631 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:41.629+0000 7f4e7de00700 1 --2- 172.21.15.47:0/3668543176 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4e781073b0 0x7f4e7819a790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:23:41.632 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:41.629+0000 7f4e7de00700 1 -- 172.21.15.47:0/3668543176 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e7819af70 con 0x7f4e781022d0 2024-03-21T10:23:41.632 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:41.629+0000 7f4e7e800700 1 --2- 172.21.15.47:0/3668543176 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4e781073b0 0x7f4e7819a790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T10:23:41.632 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:41.629+0000 7f4e7de00700 1 --2- 172.21.15.47:0/3668543176 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4e781022d0 0x7f4e781a0610 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f4e6c00ba40 tx=0x7f4e6c00ba70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:23:41.632 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:41.630+0000 7f4e72a00700 1 -- 172.21.15.47:0/3668543176 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e6c026070 con 0x7f4e781022d0 2024-03-21T10:23:41.632 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:41.630+0000 7f4e72a00700 1 -- 172.21.15.47:0/3668543176 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f4e6c003d40 con 0x7f4e781022d0 2024-03-21T10:23:41.632 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:41.630+0000 7f4e7f200700 1 -- 172.21.15.47:0/3668543176 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4e7819b200 con 0x7f4e781022d0 2024-03-21T10:23:41.632 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:41.630+0000 7f4e72a00700 1 -- 172.21.15.47:0/3668543176 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e6c00fe50 con 0x7f4e781022d0 2024-03-21T10:23:41.632 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:41.630+0000 7f4e7f200700 1 -- 172.21.15.47:0/3668543176 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4e781a7470 con 0x7f4e781022d0 2024-03-21T10:23:41.633 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:41.631+0000 7f4e7f200700 1 -- 172.21.15.47:0/3668543176 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4e781037b0 con 0x7f4e781022d0 2024-03-21T10:23:41.634 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:41.634+0000 7f4e72a00700 1 -- 172.21.15.47:0/3668543176 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 17) v1 ==== 137905+0+0 (secure 0 0 0) 0x7f4e6c009d10 con 0x7f4e781022d0 2024-03-21T10:23:41.637 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:41.636+0000 7f4e72a00700 1 --2- 172.21.15.47:0/3668543176 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f4e600a31e0 0x7f4e600a56a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:23:41.637 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:41.636+0000 7f4e72a00700 1 -- 172.21.15.47:0/3668543176 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9216+0+0 (secure 0 0 0) 0x7f4e6c0d0b60 con 0x7f4e781022d0 2024-03-21T10:23:41.637 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:41.636+0000 7f4e7d400700 1 --2- 172.21.15.47:0/3668543176 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f4e600a31e0 0x7f4e600a56a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:23:41.638 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:41.637+0000 7f4e7d400700 1 --2- 172.21.15.47:0/3668543176 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f4e600a31e0 0x7f4e600a56a0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f4e780ff2f0 tx=0x7f4e68009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:23:41.640 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:41.639+0000 7f4e72a00700 1 -- 172.21.15.47:0/3668543176 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f4e6c02f020 con 0x7f4e781022d0 2024-03-21T10:23:41.889 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:41.887+0000 7f7334600700 1 -- 172.21.15.47:0/957028835 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f732c106370 msgr2=0x7f732c10df40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:23:41.889 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:41.887+0000 7f7334600700 1 --2- 172.21.15.47:0/957028835 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f732c106370 0x7f732c10df40 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f7324009fc0 tx=0x7f732400b330 comp rx=0 tx=0).stop 2024-03-21T10:23:41.889 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:41.887+0000 7f7334600700 1 -- 172.21.15.47:0/957028835 shutdown_connections 2024-03-21T10:23:41.889 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:41.887+0000 7f7334600700 1 --2- 172.21.15.47:0/957028835 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f732c106370 0x7f732c10df40 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:23:41.889 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:41.887+0000 7f7334600700 1 --2- 172.21.15.47:0/957028835 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f732c101320 0x7f732c101700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:23:41.889 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:41.887+0000 7f7334600700 1 --2- 172.21.15.47:0/957028835 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f732c101cd0 0x7f732c105c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:23:41.889 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:41.887+0000 7f7334600700 1 -- 172.21.15.47:0/957028835 >> 172.21.15.47:0/957028835 conn(0x7f732c0770e0 msgr2=0x7f732c0774f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:23:41.889 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:41.887+0000 7f7334600700 1 -- 172.21.15.47:0/957028835 shutdown_connections 2024-03-21T10:23:41.890 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:23:41 smithi047 ceph-mon[31547]: pgmap v422: 129 pgs: 129 active+clean; 642 KiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s; 9 B/s, 0 objects/s recovering 2024-03-21T10:23:41.890 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:23:41 smithi047 ceph-mon[31547]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T10:23:41.890 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:23:41 smithi047 ceph-mon[31547]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T10:23:41.890 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:23:41 smithi047 ceph-mon[31547]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:23:41.891 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:41.887+0000 7f7334600700 1 -- 172.21.15.47:0/957028835 wait complete. 2024-03-21T10:23:41.891 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:41.889+0000 7f7334600700 1 Processor -- start 2024-03-21T10:23:41.891 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:41.889+0000 7f7334600700 1 -- start start 2024-03-21T10:23:41.891 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:41.889+0000 7f7334600700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f732c101320 0x7f732c1a06e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:23:41.891 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:41.889+0000 7f7334600700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f732c101cd0 0x7f732c1a0c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:23:41.891 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:41.889+0000 7f7334600700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f732c106370 0x7f732c19a8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:23:41.891 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:41.889+0000 7f7334600700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f732c111280 con 0x7f732c101320 2024-03-21T10:23:41.891 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:41.889+0000 7f7334600700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f732c111100 con 0x7f732c106370 2024-03-21T10:23:41.892 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:41.889+0000 7f7334600700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f732c111400 con 0x7f732c101cd0 2024-03-21T10:23:41.892 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:41.890+0000 7f7331c00700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f732c101320 0x7f732c1a06e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:23:41.892 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:41.890+0000 7f7331c00700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f732c101320 0x7f732c1a06e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:46786/0 (socket says 172.21.15.47:46786) 2024-03-21T10:23:41.892 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:41.890+0000 7f7331c00700 1 -- 172.21.15.47:0/901222434 learned_addr learned my addr 172.21.15.47:0/901222434 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:23:41.893 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:41.890+0000 7f7332600700 1 --2- 172.21.15.47:0/901222434 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f732c106370 0x7f732c19a8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:23:41.893 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:41.890+0000 7f7331c00700 1 -- 172.21.15.47:0/901222434 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f732c101cd0 msgr2=0x7f732c1a0c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:23:41.893 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:41.890+0000 7f7331c00700 1 --2- 172.21.15.47:0/901222434 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f732c101cd0 0x7f732c1a0c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:23:41.893 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:41.890+0000 7f7331c00700 1 -- 172.21.15.47:0/901222434 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f732c106370 msgr2=0x7f732c19a8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:23:41.893 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:41.890+0000 7f7331c00700 1 --2- 172.21.15.47:0/901222434 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f732c106370 0x7f732c19a8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:23:41.893 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:41.890+0000 7f7331c00700 1 -- 172.21.15.47:0/901222434 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f732c19b000 con 0x7f732c101320 2024-03-21T10:23:41.893 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:41.891+0000 7f7331c00700 1 --2- 172.21.15.47:0/901222434 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f732c101320 0x7f732c1a06e0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f732000e9c0 tx=0x7f732000ed80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:23:41.893 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:41.891+0000 7f732a000700 1 -- 172.21.15.47:0/901222434 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f732000cbd0 con 0x7f732c101320 2024-03-21T10:23:41.893 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:41.891+0000 7f7334600700 1 -- 172.21.15.47:0/901222434 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f732c19b2f0 con 0x7f732c101320 2024-03-21T10:23:41.894 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:41.891+0000 7f732a000700 1 -- 172.21.15.47:0/901222434 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f732000cd70 con 0x7f732c101320 2024-03-21T10:23:41.894 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:41.891+0000 7f732a000700 1 -- 172.21.15.47:0/901222434 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7320005650 con 0x7f732c101320 2024-03-21T10:23:41.894 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:41.891+0000 7f7334600700 1 -- 172.21.15.47:0/901222434 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f732c1a75b0 con 0x7f732c101320 2024-03-21T10:23:41.894 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:41.893+0000 7f7334600700 1 -- 172.21.15.47:0/901222434 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f732c068e50 con 0x7f732c101320 2024-03-21T10:23:41.896 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:41.895+0000 7f732a000700 1 -- 172.21.15.47:0/901222434 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 17) v1 ==== 137905+0+0 (secure 0 0 0) 0x7f7320004750 con 0x7f732c101320 2024-03-21T10:23:41.897 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:41.896+0000 7f732a000700 1 --2- 172.21.15.47:0/901222434 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f73140a3040 0x7f73140a5500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:23:41.898 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:41.896+0000 7f732a000700 1 -- 172.21.15.47:0/901222434 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9216+0+0 (secure 0 0 0) 0x7f73200cfbb0 con 0x7f732c101320 2024-03-21T10:23:41.898 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:41.897+0000 7f7331200700 1 --2- 172.21.15.47:0/901222434 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f73140a3040 0x7f73140a5500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:23:41.899 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:41.898+0000 7f7331200700 1 --2- 172.21.15.47:0/901222434 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f73140a3040 0x7f73140a5500 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f7318005fd0 tx=0x7f7318005ee0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:23:41.906 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:41.905+0000 7f732a000700 1 -- 172.21.15.47:0/901222434 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f732009c470 con 0x7f732c101320 2024-03-21T10:23:41.911 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:41.910+0000 7f4e7f200700 1 -- 172.21.15.47:0/3668543176 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) v1 -- 0x7f4e7819bcc0 con 0x7f4e781022d0 2024-03-21T10:23:41.911 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:41.910+0000 7f4e72a00700 1 -- 172.21.15.47:0/3668543176 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f4e6c013070 con 0x7f4e781022d0 2024-03-21T10:23:41.912 INFO:teuthology.orchestra.run.smithi047.stdout:171798691918 2024-03-21T10:23:41.914 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:41.913+0000 7f4e7f200700 1 -- 172.21.15.47:0/3668543176 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f4e600a31e0 msgr2=0x7f4e600a56a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:23:41.914 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:41.913+0000 7f4e7f200700 1 --2- 172.21.15.47:0/3668543176 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f4e600a31e0 0x7f4e600a56a0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f4e780ff2f0 tx=0x7f4e68009260 comp rx=0 tx=0).stop 2024-03-21T10:23:41.914 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:41.913+0000 7f4e7f200700 1 -- 172.21.15.47:0/3668543176 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4e781022d0 msgr2=0x7f4e781a0610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:23:41.914 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:41.913+0000 7f4e7f200700 1 --2- 172.21.15.47:0/3668543176 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4e781022d0 0x7f4e781a0610 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f4e6c00ba40 tx=0x7f4e6c00ba70 comp rx=0 tx=0).stop 2024-03-21T10:23:41.915 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:41.913+0000 7f4e7f200700 1 -- 172.21.15.47:0/3668543176 shutdown_connections 2024-03-21T10:23:41.915 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:41.913+0000 7f4e7f200700 1 --2- 172.21.15.47:0/3668543176 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f4e600a31e0 0x7f4e600a56a0 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:23:41.915 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:41.913+0000 7f4e7f200700 1 --2- 172.21.15.47:0/3668543176 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4e781073b0 0x7f4e7819a790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:23:41.915 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:41.913+0000 7f4e7f200700 1 --2- 172.21.15.47:0/3668543176 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4e781022d0 0x7f4e781a0610 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:23:41.915 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:41.913+0000 7f4e7f200700 1 --2- 172.21.15.47:0/3668543176 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4e78102c80 0x7f4e781a0b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:23:41.915 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:41.913+0000 7f4e7f200700 1 -- 172.21.15.47:0/3668543176 >> 172.21.15.47:0/3668543176 conn(0x7f4e780fdb60 msgr2=0x7f4e7810e250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:23:41.915 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:41.913+0000 7f4e7f200700 1 -- 172.21.15.47:0/3668543176 shutdown_connections 2024-03-21T10:23:41.915 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:41.913+0000 7f4e7f200700 1 -- 172.21.15.47:0/3668543176 wait complete. 2024-03-21T10:23:42.147 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:42.146+0000 7f7334600700 1 -- 172.21.15.47:0/901222434 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f732c02da30 con 0x7f732c101320 2024-03-21T10:23:42.148 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:42.147+0000 7f732a000700 1 -- 172.21.15.47:0/901222434 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f732009bd70 con 0x7f732c101320 2024-03-21T10:23:42.149 INFO:teuthology.orchestra.run.smithi047.stdout:115964117085 2024-03-21T10:23:42.151 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:42.150+0000 7f7334600700 1 -- 172.21.15.47:0/901222434 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f73140a3040 msgr2=0x7f73140a5500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:23:42.152 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:42.150+0000 7f7334600700 1 --2- 172.21.15.47:0/901222434 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f73140a3040 0x7f73140a5500 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f7318005fd0 tx=0x7f7318005ee0 comp rx=0 tx=0).stop 2024-03-21T10:23:42.152 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:42.150+0000 7f7334600700 1 -- 172.21.15.47:0/901222434 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f732c101320 msgr2=0x7f732c1a06e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:23:42.152 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:42.150+0000 7f7334600700 1 --2- 172.21.15.47:0/901222434 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f732c101320 0x7f732c1a06e0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f732000e9c0 tx=0x7f732000ed80 comp rx=0 tx=0).stop 2024-03-21T10:23:42.152 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:42.150+0000 7f7334600700 1 -- 172.21.15.47:0/901222434 shutdown_connections 2024-03-21T10:23:42.152 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:42.150+0000 7f7334600700 1 --2- 172.21.15.47:0/901222434 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f73140a3040 0x7f73140a5500 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:23:42.152 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:42.150+0000 7f7334600700 1 --2- 172.21.15.47:0/901222434 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f732c106370 0x7f732c19a8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:23:42.152 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:42.150+0000 7f7334600700 1 --2- 172.21.15.47:0/901222434 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f732c101320 0x7f732c1a06e0 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:23:42.152 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:42.150+0000 7f7334600700 1 --2- 172.21.15.47:0/901222434 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f732c101cd0 0x7f732c1a0c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:23:42.152 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:42.150+0000 7f7334600700 1 -- 172.21.15.47:0/901222434 >> 172.21.15.47:0/901222434 conn(0x7f732c0770e0 msgr2=0x7f732c10cad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:23:42.152 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:42.150+0000 7f7334600700 1 -- 172.21.15.47:0/901222434 shutdown_connections 2024-03-21T10:23:42.152 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:42.151+0000 7f7334600700 1 -- 172.21.15.47:0/901222434 wait complete. 2024-03-21T10:23:42.229 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:23:41 smithi100 ceph-mon[35193]: pgmap v422: 129 pgs: 129 active+clean; 642 KiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s; 9 B/s, 0 objects/s recovering 2024-03-21T10:23:42.230 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:23:41 smithi100 ceph-mon[35193]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T10:23:42.230 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:23:41 smithi100 ceph-mon[35193]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T10:23:42.230 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:23:41 smithi100 ceph-mon[35193]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:23:42.240 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:23:41 smithi157 ceph-mon[35581]: pgmap v422: 129 pgs: 129 active+clean; 642 KiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s; 9 B/s, 0 objects/s recovering 2024-03-21T10:23:42.240 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:23:41 smithi157 ceph-mon[35581]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T10:23:42.240 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:23:41 smithi157 ceph-mon[35581]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T10:23:42.240 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:23:41 smithi157 ceph-mon[35581]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:23:42.373 INFO:tasks.cephadm.ceph_manager.ceph:need seq 352187318301 got 352187318307 for osd.11 2024-03-21T10:23:42.373 DEBUG:teuthology.parallel:result is None 2024-03-21T10:23:42.610 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/3056d51a-e76b-11ee-95cd-87774f69a715/mon.a/config 2024-03-21T10:23:42.964 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:23:42 smithi047 ceph-mon[31547]: from='client.? 172.21.15.47:0/3668543176' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-03-21T10:23:42.964 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:23:42 smithi047 ceph-mon[31547]: from='client.? 172.21.15.47:0/901222434' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-03-21T10:23:43.229 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:23:42 smithi100 ceph-mon[35193]: from='client.? 172.21.15.47:0/3668543176' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-03-21T10:23:43.229 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:23:42 smithi100 ceph-mon[35193]: from='client.? 172.21.15.47:0/901222434' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-03-21T10:23:43.240 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:23:42 smithi157 ceph-mon[35581]: from='client.? 172.21.15.47:0/3668543176' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-03-21T10:23:43.240 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:23:42 smithi157 ceph-mon[35581]: from='client.? 172.21.15.47:0/901222434' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-03-21T10:23:43.727 INFO:tasks.cephadm.ceph_manager.ceph:need seq 171798691911 got 171798691918 for osd.5 2024-03-21T10:23:43.727 DEBUG:teuthology.parallel:result is None 2024-03-21T10:23:43.938 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:23:43 smithi047 ceph-mon[31547]: pgmap v423: 129 pgs: 129 active+clean; 642 KiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-21T10:23:44.229 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:23:43 smithi100 ceph-mon[35193]: pgmap v423: 129 pgs: 129 active+clean; 642 KiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-21T10:23:44.240 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:23:43 smithi157 ceph-mon[35581]: pgmap v423: 129 pgs: 129 active+clean; 642 KiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-21T10:23:44.995 INFO:tasks.cephadm.ceph_manager.ceph:need seq 115964117078 got 115964117085 for osd.3 2024-03-21T10:23:44.996 DEBUG:teuthology.parallel:result is None 2024-03-21T10:23:45.274 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/3056d51a-e76b-11ee-95cd-87774f69a715/mon.a/config 2024-03-21T10:23:45.275 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/3056d51a-e76b-11ee-95cd-87774f69a715/mon.a/config 2024-03-21T10:23:45.317 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:45.315+0000 7fbfa7a00700 1 -- 172.21.15.47:0/2591299718 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbfa0069e40 msgr2=0x7fbfa006a2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:23:45.317 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:45.315+0000 7fbfa7a00700 1 --2- 172.21.15.47:0/2591299718 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbfa0069e40 0x7fbfa006a2a0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fbf8c009990 tx=0x7fbf8c009ca0 comp rx=0 tx=0).stop 2024-03-21T10:23:45.317 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:45.316+0000 7fbfa7a00700 1 -- 172.21.15.47:0/2591299718 shutdown_connections 2024-03-21T10:23:45.317 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:45.316+0000 7fbfa7a00700 1 --2- 172.21.15.47:0/2591299718 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbfa006a7e0 0x7fbfa010f020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:23:45.317 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:45.316+0000 7fbfa7a00700 1 --2- 172.21.15.47:0/2591299718 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbfa01081e0 0x7fbfa01085c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:23:45.317 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:45.316+0000 7fbfa7a00700 1 --2- 172.21.15.47:0/2591299718 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbfa0069e40 0x7fbfa006a2a0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:23:45.318 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:45.316+0000 7fbfa7a00700 1 -- 172.21.15.47:0/2591299718 >> 172.21.15.47:0/2591299718 conn(0x7fbfa0077fc0 msgr2=0x7fbfa00783d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:23:45.318 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:45.317+0000 7fbfa7a00700 1 -- 172.21.15.47:0/2591299718 shutdown_connections 2024-03-21T10:23:45.318 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:45.317+0000 7fbfa7a00700 1 -- 172.21.15.47:0/2591299718 wait complete. 2024-03-21T10:23:45.319 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:45.318+0000 7fbfa7a00700 1 Processor -- start 2024-03-21T10:23:45.323 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:45.318+0000 7fbfa7a00700 1 -- start start 2024-03-21T10:23:45.323 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:45.319+0000 7fbfa7a00700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbfa0069e40 0x7fbfa019e5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:23:45.323 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:45.319+0000 7fbfa7a00700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbfa006a7e0 0x7fbfa019eb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:23:45.323 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:45.319+0000 7fbfa7a00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbfa01081e0 0x7fbfa01986c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:23:45.323 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:45.319+0000 7fbfa7a00700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbfa0112340 con 0x7fbfa006a7e0 2024-03-21T10:23:45.323 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:45.319+0000 7fbfa7a00700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbfa01121c0 con 0x7fbfa0069e40 2024-03-21T10:23:45.323 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:45.319+0000 7fbfa7a00700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbfa01124c0 con 0x7fbfa01081e0 2024-03-21T10:23:45.323 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:45.319+0000 7fbfa5000700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbfa0069e40 0x7fbfa019e5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:23:45.324 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:45.319+0000 7fbfa5000700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbfa0069e40 0x7fbfa019e5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:60876/0 (socket says 172.21.15.47:60876) 2024-03-21T10:23:45.324 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:45.319+0000 7fbfa5000700 1 -- 172.21.15.47:0/501659898 learned_addr learned my addr 172.21.15.47:0/501659898 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:23:45.324 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:45.320+0000 7fbfa5a00700 1 --2- 172.21.15.47:0/501659898 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbfa01081e0 0x7fbfa01986c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:23:45.324 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:45.320+0000 7fbfa5000700 1 -- 172.21.15.47:0/501659898 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbfa01081e0 msgr2=0x7fbfa01986c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:23:45.324 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:45.320+0000 7fbfa5000700 1 --2- 172.21.15.47:0/501659898 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbfa01081e0 0x7fbfa01986c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:23:45.324 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:45.320+0000 7fbfa5000700 1 -- 172.21.15.47:0/501659898 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbfa006a7e0 msgr2=0x7fbfa019eb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:23:45.324 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:45.320+0000 7fbfa5000700 1 --2- 172.21.15.47:0/501659898 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbfa006a7e0 0x7fbfa019eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:23:45.324 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:45.320+0000 7fbfa5000700 1 -- 172.21.15.47:0/501659898 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbfa0198fa0 con 0x7fbfa0069e40 2024-03-21T10:23:45.324 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:45.320+0000 7fbfa5000700 1 --2- 172.21.15.47:0/501659898 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbfa0069e40 0x7fbfa019e5f0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fbf9400d950 tx=0x7fbf9400dd10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:23:45.324 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:45.321+0000 7fbf9d600700 1 -- 172.21.15.47:0/501659898 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf94016070 con 0x7fbfa0069e40 2024-03-21T10:23:45.324 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:45.321+0000 7fbfa7a00700 1 -- 172.21.15.47:0/501659898 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbfa0199290 con 0x7fbfa0069e40 2024-03-21T10:23:45.324 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:45.321+0000 7fbfa7a00700 1 -- 172.21.15.47:0/501659898 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbfa0101260 con 0x7fbfa0069e40 2024-03-21T10:23:45.324 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:45.321+0000 7fbf9d600700 1 -- 172.21.15.47:0/501659898 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fbf94010460 con 0x7fbfa0069e40 2024-03-21T10:23:45.324 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:45.321+0000 7fbf9d600700 1 -- 172.21.15.47:0/501659898 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf9400f620 con 0x7fbfa0069e40 2024-03-21T10:23:45.325 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:45.323+0000 7fbfa7a00700 1 -- 172.21.15.47:0/501659898 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbf80005360 con 0x7fbfa0069e40 2024-03-21T10:23:45.328 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:45.326+0000 7fbf9d600700 1 -- 172.21.15.47:0/501659898 <== mon.2 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 17) v1 ==== 137905+0+0 (secure 0 0 0) 0x7fbf94019070 con 0x7fbfa0069e40 2024-03-21T10:23:45.329 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:45.327+0000 7fbf9d600700 1 --2- 172.21.15.47:0/501659898 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7fbf880a32c0 0x7fbf880a5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:23:45.329 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:45.328+0000 7fbf9d600700 1 -- 172.21.15.47:0/501659898 <== mon.2 v2:172.21.15.100:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9216+0+0 (secure 0 0 0) 0x7fbf9400f3c0 con 0x7fbfa0069e40 2024-03-21T10:23:45.329 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:45.328+0000 7fbf9fe00700 1 --2- 172.21.15.47:0/501659898 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7fbf880a32c0 0x7fbf880a5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:23:45.330 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:45.329+0000 7fbf9fe00700 1 --2- 172.21.15.47:0/501659898 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7fbf880a32c0 0x7fbf880a5780 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fbf8c0098e0 tx=0x7fbf8c005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:23:45.341 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:45.339+0000 7fbf9d600700 1 -- 172.21.15.47:0/501659898 <== mon.2 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fbf9409bb00 con 0x7fbfa0069e40 2024-03-21T10:23:45.586 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:45.585+0000 7fbfa7a00700 1 -- 172.21.15.47:0/501659898 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 9} v 0) v1 -- 0x7fbf80005920 con 0x7fbfa0069e40 2024-03-21T10:23:45.587 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:45.586+0000 7fbf9d600700 1 -- 172.21.15.47:0/501659898 <== mon.2 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 9}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fbf9409b400 con 0x7fbfa0069e40 2024-03-21T10:23:45.587 INFO:teuthology.orchestra.run.smithi047.stdout:296352743475 2024-03-21T10:23:45.590 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:45.589+0000 7fbfa7a00700 1 -- 172.21.15.47:0/501659898 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7fbf880a32c0 msgr2=0x7fbf880a5780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:23:45.590 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:45.589+0000 7fbfa7a00700 1 --2- 172.21.15.47:0/501659898 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7fbf880a32c0 0x7fbf880a5780 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fbf8c0098e0 tx=0x7fbf8c005fb0 comp rx=0 tx=0).stop 2024-03-21T10:23:45.590 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:45.589+0000 7fbfa7a00700 1 -- 172.21.15.47:0/501659898 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbfa0069e40 msgr2=0x7fbfa019e5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:23:45.590 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:45.589+0000 7fbfa7a00700 1 --2- 172.21.15.47:0/501659898 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbfa0069e40 0x7fbfa019e5f0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fbf9400d950 tx=0x7fbf9400dd10 comp rx=0 tx=0).stop 2024-03-21T10:23:45.590 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:45.589+0000 7fbfa7a00700 1 -- 172.21.15.47:0/501659898 shutdown_connections 2024-03-21T10:23:45.590 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:45.589+0000 7fbfa7a00700 1 --2- 172.21.15.47:0/501659898 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7fbf880a32c0 0x7fbf880a5780 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:23:45.590 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:45.589+0000 7fbfa7a00700 1 --2- 172.21.15.47:0/501659898 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbfa01081e0 0x7fbfa01986c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:23:45.591 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:45.589+0000 7fbfa7a00700 1 --2- 172.21.15.47:0/501659898 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbfa0069e40 0x7fbfa019e5f0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:23:45.591 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:45.589+0000 7fbfa7a00700 1 --2- 172.21.15.47:0/501659898 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbfa006a7e0 0x7fbfa019eb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:23:45.591 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:45.589+0000 7fbfa7a00700 1 -- 172.21.15.47:0/501659898 >> 172.21.15.47:0/501659898 conn(0x7fbfa0077fc0 msgr2=0x7fbfa0100060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:23:45.591 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:45.589+0000 7fbfa7a00700 1 -- 172.21.15.47:0/501659898 shutdown_connections 2024-03-21T10:23:45.591 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:45.589+0000 7fbfa7a00700 1 -- 172.21.15.47:0/501659898 wait complete. 2024-03-21T10:23:45.964 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:23:45 smithi047 ceph-mon[31547]: pgmap v424: 129 pgs: 129 active+clean; 642 KiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-03-21T10:23:45.964 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:23:45 smithi047 ceph-mon[31547]: from='client.? 172.21.15.47:0/501659898' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-03-21T10:23:46.229 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:23:45 smithi100 ceph-mon[35193]: pgmap v424: 129 pgs: 129 active+clean; 642 KiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-03-21T10:23:46.229 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:23:45 smithi100 ceph-mon[35193]: from='client.? 172.21.15.47:0/501659898' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-03-21T10:23:46.240 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:23:45 smithi157 ceph-mon[35581]: pgmap v424: 129 pgs: 129 active+clean; 642 KiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-03-21T10:23:46.240 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:23:45 smithi157 ceph-mon[35581]: from='client.? 172.21.15.47:0/501659898' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-03-21T10:23:46.755 INFO:tasks.cephadm.ceph_manager.ceph:need seq 296352743469 got 296352743475 for osd.9 2024-03-21T10:23:46.755 DEBUG:teuthology.parallel:result is None 2024-03-21T10:23:47.826 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:23:47 smithi047 ceph-mon[31547]: pgmap v425: 129 pgs: 129 active+clean; 642 KiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-21T10:23:48.064 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:48.062+0000 7f0bd1200700 1 -- 172.21.15.47:0/3997607430 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0bcc074dd0 msgr2=0x7f0bcc10e3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:23:48.064 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:48.062+0000 7f0bd1200700 1 --2- 172.21.15.47:0/3997607430 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0bcc074dd0 0x7f0bcc10e3f0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f0bb80098f0 tx=0x7f0bb8009c00 comp rx=0 tx=0).stop 2024-03-21T10:23:48.064 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:48.063+0000 7f0bd1200700 1 -- 172.21.15.47:0/3997607430 shutdown_connections 2024-03-21T10:23:48.064 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:48.063+0000 7f0bd1200700 1 --2- 172.21.15.47:0/3997607430 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0bcc10e930 0x7f0bcc110d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:23:48.064 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:48.063+0000 7f0bd1200700 1 --2- 172.21.15.47:0/3997607430 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0bcc0744b0 0x7f0bcc074890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:23:48.065 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:48.063+0000 7f0bd1200700 1 --2- 172.21.15.47:0/3997607430 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0bcc074dd0 0x7f0bcc10e3f0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:23:48.065 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:48.063+0000 7f0bd1200700 1 -- 172.21.15.47:0/3997607430 >> 172.21.15.47:0/3997607430 conn(0x7f0bcc0fdb60 msgr2=0x7f0bcc0fff80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:23:48.065 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:48.063+0000 7f0bd1200700 1 -- 172.21.15.47:0/3997607430 shutdown_connections 2024-03-21T10:23:48.065 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:48.063+0000 7f0bd1200700 1 -- 172.21.15.47:0/3997607430 wait complete. 2024-03-21T10:23:48.065 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:48.064+0000 7f0bd1200700 1 Processor -- start 2024-03-21T10:23:48.065 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:48.064+0000 7f0bd1200700 1 -- start start 2024-03-21T10:23:48.066 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:48.064+0000 7f0bd1200700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0bcc0744b0 0x7f0bcc1a06a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:23:48.066 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:48.065+0000 7f0bd1200700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0bcc074dd0 0x7f0bcc1a0be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:23:48.066 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:48.065+0000 7f0bd1200700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0bcc10e930 0x7f0bcc19a820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:23:48.066 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:48.065+0000 7f0bd1200700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0bcc114480 con 0x7f0bcc074dd0 2024-03-21T10:23:48.067 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:48.065+0000 7f0bd1200700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0bcc114300 con 0x7f0bcc10e930 2024-03-21T10:23:48.067 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:48.065+0000 7f0bd1200700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0bcc114600 con 0x7f0bcc0744b0 2024-03-21T10:23:48.067 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:48.065+0000 7f0bcaa00700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0bcc074dd0 0x7f0bcc1a0be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:23:48.067 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:48.065+0000 7f0bcaa00700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0bcc074dd0 0x7f0bcc1a0be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:46824/0 (socket says 172.21.15.47:46824) 2024-03-21T10:23:48.067 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:48.065+0000 7f0bcaa00700 1 -- 172.21.15.47:0/3957275914 learned_addr learned my addr 172.21.15.47:0/3957275914 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:23:48.067 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:48.065+0000 7f0bcb400700 1 --2- 172.21.15.47:0/3957275914 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0bcc0744b0 0x7f0bcc1a06a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:23:48.067 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:48.065+0000 7f0bcbe00700 1 --2- 172.21.15.47:0/3957275914 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0bcc10e930 0x7f0bcc19a820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:23:48.068 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:48.066+0000 7f0bcaa00700 1 -- 172.21.15.47:0/3957275914 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0bcc0744b0 msgr2=0x7f0bcc1a06a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:23:48.068 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:48.066+0000 7f0bcaa00700 1 --2- 172.21.15.47:0/3957275914 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0bcc0744b0 0x7f0bcc1a06a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:23:48.068 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:48.066+0000 7f0bcaa00700 1 -- 172.21.15.47:0/3957275914 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0bcc10e930 msgr2=0x7f0bcc19a820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:23:48.068 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:48.066+0000 7f0bcaa00700 1 --2- 172.21.15.47:0/3957275914 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0bcc10e930 0x7f0bcc19a820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:23:48.068 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:48.066+0000 7f0bcaa00700 1 -- 172.21.15.47:0/3957275914 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0bcc19b000 con 0x7f0bcc074dd0 2024-03-21T10:23:48.068 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:48.066+0000 7f0bcbe00700 1 --2- 172.21.15.47:0/3957275914 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0bcc10e930 0x7f0bcc19a820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T10:23:48.068 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:48.066+0000 7f0bcaa00700 1 --2- 172.21.15.47:0/3957275914 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0bcc074dd0 0x7f0bcc1a0be0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f0bb8009230 tx=0x7f0bb800fa70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:23:48.068 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:48.066+0000 7f0bc3e00700 1 -- 172.21.15.47:0/3957275914 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0bb80050c0 con 0x7f0bcc074dd0 2024-03-21T10:23:48.068 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:48.067+0000 7f0bc3e00700 1 -- 172.21.15.47:0/3957275914 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f0bb800b560 con 0x7f0bcc074dd0 2024-03-21T10:23:48.068 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:48.067+0000 7f0bc3e00700 1 -- 172.21.15.47:0/3957275914 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0bb8003a00 con 0x7f0bcc074dd0 2024-03-21T10:23:48.069 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:48.067+0000 7f0bd1200700 1 -- 172.21.15.47:0/3957275914 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0bcc19b290 con 0x7f0bcc074dd0 2024-03-21T10:23:48.069 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:48.067+0000 7f0bd1200700 1 -- 172.21.15.47:0/3957275914 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0bcc1a7500 con 0x7f0bcc074dd0 2024-03-21T10:23:48.069 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:48.068+0000 7f0bd1200700 1 -- 172.21.15.47:0/3957275914 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0bcc075bf0 con 0x7f0bcc074dd0 2024-03-21T10:23:48.073 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:48.072+0000 7f0bc3e00700 1 -- 172.21.15.47:0/3957275914 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 17) v1 ==== 137905+0+0 (secure 0 0 0) 0x7f0bb800b710 con 0x7f0bcc074dd0 2024-03-21T10:23:48.074 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:48.073+0000 7f0bc3e00700 1 --2- 172.21.15.47:0/3957275914 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f0bb00a30c0 0x7f0bb00a5580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:23:48.075 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:48.074+0000 7f0bc3e00700 1 -- 172.21.15.47:0/3957275914 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9216+0+0 (secure 0 0 0) 0x7f0bb80d09d0 con 0x7f0bcc074dd0 2024-03-21T10:23:48.075 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:48.074+0000 7f0bcb400700 1 --2- 172.21.15.47:0/3957275914 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f0bb00a30c0 0x7f0bb00a5580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:23:48.075 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:48.074+0000 7f0bcb400700 1 --2- 172.21.15.47:0/3957275914 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f0bb00a30c0 0x7f0bb00a5580 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f0bbc009c40 tx=0x7f0bbc009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:23:48.076 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:48.075+0000 7f0bc3e00700 1 -- 172.21.15.47:0/3957275914 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f0bb805e740 con 0x7f0bcc074dd0 2024-03-21T10:23:48.229 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:23:47 smithi100 ceph-mon[35193]: pgmap v425: 129 pgs: 129 active+clean; 642 KiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-21T10:23:48.240 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:23:47 smithi157 ceph-mon[35581]: pgmap v425: 129 pgs: 129 active+clean; 642 KiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-21T10:23:48.367 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:48.366+0000 7f0bd1200700 1 -- 172.21.15.47:0/3957275914 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 8} v 0) v1 -- 0x7f0bcc19bd50 con 0x7f0bcc074dd0 2024-03-21T10:23:48.368 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:48.366+0000 7f0bc3e00700 1 -- 172.21.15.47:0/3957275914 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 8}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f0bb8013030 con 0x7f0bcc074dd0 2024-03-21T10:23:48.368 INFO:teuthology.orchestra.run.smithi047.stdout:261993005114 2024-03-21T10:23:48.370 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:48.369+0000 7f0bd1200700 1 -- 172.21.15.47:0/3957275914 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f0bb00a30c0 msgr2=0x7f0bb00a5580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:23:48.371 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:48.369+0000 7f0bd1200700 1 --2- 172.21.15.47:0/3957275914 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f0bb00a30c0 0x7f0bb00a5580 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f0bbc009c40 tx=0x7f0bbc009260 comp rx=0 tx=0).stop 2024-03-21T10:23:48.371 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:48.369+0000 7f0bd1200700 1 -- 172.21.15.47:0/3957275914 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0bcc074dd0 msgr2=0x7f0bcc1a0be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:23:48.371 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:48.369+0000 7f0bd1200700 1 --2- 172.21.15.47:0/3957275914 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0bcc074dd0 0x7f0bcc1a0be0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f0bb8009230 tx=0x7f0bb800fa70 comp rx=0 tx=0).stop 2024-03-21T10:23:48.371 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:48.369+0000 7f0bd1200700 1 -- 172.21.15.47:0/3957275914 shutdown_connections 2024-03-21T10:23:48.371 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:48.369+0000 7f0bd1200700 1 --2- 172.21.15.47:0/3957275914 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f0bb00a30c0 0x7f0bb00a5580 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:23:48.371 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:48.369+0000 7f0bd1200700 1 --2- 172.21.15.47:0/3957275914 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0bcc10e930 0x7f0bcc19a820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:23:48.371 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:48.369+0000 7f0bd1200700 1 --2- 172.21.15.47:0/3957275914 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0bcc0744b0 0x7f0bcc1a06a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:23:48.371 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:48.369+0000 7f0bd1200700 1 --2- 172.21.15.47:0/3957275914 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0bcc074dd0 0x7f0bcc1a0be0 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:23:48.371 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:48.369+0000 7f0bd1200700 1 -- 172.21.15.47:0/3957275914 >> 172.21.15.47:0/3957275914 conn(0x7f0bcc0fdb60 msgr2=0x7f0bcc0fe8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:23:48.371 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:48.370+0000 7f0bd1200700 1 -- 172.21.15.47:0/3957275914 shutdown_connections 2024-03-21T10:23:48.372 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:48.370+0000 7f0bd1200700 1 -- 172.21.15.47:0/3957275914 wait complete. 2024-03-21T10:23:49.131 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:23:48 smithi100 ceph-mon[35193]: from='client.? 172.21.15.47:0/3957275914' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-03-21T10:23:49.214 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:23:48 smithi047 ceph-mon[31547]: from='client.? 172.21.15.47:0/3957275914' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-03-21T10:23:49.240 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:23:48 smithi157 ceph-mon[35581]: from='client.? 172.21.15.47:0/3957275914' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-03-21T10:23:50.034 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:23:49 smithi047 ceph-mon[31547]: pgmap v426: 129 pgs: 129 active+clean; 642 KiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-03-21T10:23:50.229 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:23:49 smithi100 ceph-mon[35193]: pgmap v426: 129 pgs: 129 active+clean; 642 KiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-03-21T10:23:50.239 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:23:49 smithi157 ceph-mon[35581]: pgmap v426: 129 pgs: 129 active+clean; 642 KiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-03-21T10:23:50.919 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:50.718+0000 7f643be00700 1 -- 172.21.15.47:0/2398674220 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f643c100e30 msgr2=0x7f643c101210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:23:50.919 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:50.718+0000 7f643be00700 1 --2- 172.21.15.47:0/2398674220 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f643c100e30 0x7f643c101210 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f642c009990 tx=0x7f642c009ca0 comp rx=0 tx=0).stop 2024-03-21T10:23:50.919 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:50.718+0000 7f643be00700 1 -- 172.21.15.47:0/2398674220 shutdown_connections 2024-03-21T10:23:50.919 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:50.718+0000 7f643be00700 1 --2- 172.21.15.47:0/2398674220 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f643c10e9e0 0x7f643c110dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:23:50.919 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:50.718+0000 7f643be00700 1 --2- 172.21.15.47:0/2398674220 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f643c100e30 0x7f643c101210 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:23:50.919 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:50.718+0000 7f643be00700 1 --2- 172.21.15.47:0/2398674220 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f643c1017e0 0x7f643c10e4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:23:50.919 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:50.718+0000 7f643be00700 1 -- 172.21.15.47:0/2398674220 >> 172.21.15.47:0/2398674220 conn(0x7f643c0fca50 msgr2=0x7f643c0fee70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:23:50.919 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:50.718+0000 7f643be00700 1 -- 172.21.15.47:0/2398674220 shutdown_connections 2024-03-21T10:23:50.919 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:50.718+0000 7f643be00700 1 -- 172.21.15.47:0/2398674220 wait complete. 2024-03-21T10:23:50.919 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:50.720+0000 7f643be00700 1 Processor -- start 2024-03-21T10:23:50.920 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:50.720+0000 7f643be00700 1 -- start start 2024-03-21T10:23:50.920 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:50.721+0000 7f643be00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f643c100e30 0x7f643c1a0620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:23:50.920 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:50.721+0000 7f643be00700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f643c1017e0 0x7f643c1a0b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:23:50.920 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:50.721+0000 7f643be00700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f643c10e9e0 0x7f643c19a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:23:50.920 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:50.721+0000 7f643be00700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f643c1144a0 con 0x7f643c1017e0 2024-03-21T10:23:50.920 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:50.721+0000 7f643be00700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f643c114320 con 0x7f643c10e9e0 2024-03-21T10:23:50.920 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:50.721+0000 7f643be00700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f643c114620 con 0x7f643c100e30 2024-03-21T10:23:50.920 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:50.721+0000 7f643a000700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f643c1017e0 0x7f643c1a0b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:23:50.920 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:50.721+0000 7f643b400700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f643c10e9e0 0x7f643c19a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:23:50.920 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:50.721+0000 7f643a000700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f643c1017e0 0x7f643c1a0b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:46844/0 (socket says 172.21.15.47:46844) 2024-03-21T10:23:50.920 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:50.721+0000 7f643a000700 1 -- 172.21.15.47:0/3242571862 learned_addr learned my addr 172.21.15.47:0/3242571862 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:23:50.921 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:50.721+0000 7f643aa00700 1 --2- 172.21.15.47:0/3242571862 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f643c100e30 0x7f643c1a0620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:23:50.921 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:50.722+0000 7f643a000700 1 -- 172.21.15.47:0/3242571862 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f643c100e30 msgr2=0x7f643c1a0620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:23:50.921 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:50.722+0000 7f643a000700 1 --2- 172.21.15.47:0/3242571862 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f643c100e30 0x7f643c1a0620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:23:50.921 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:50.722+0000 7f643a000700 1 -- 172.21.15.47:0/3242571862 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f643c10e9e0 msgr2=0x7f643c19a7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:23:50.921 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:50.722+0000 7f643a000700 1 --2- 172.21.15.47:0/3242571862 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f643c10e9e0 0x7f643c19a7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:23:50.921 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:50.722+0000 7f643a000700 1 -- 172.21.15.47:0/3242571862 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f643c19af80 con 0x7f643c1017e0 2024-03-21T10:23:50.921 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:50.722+0000 7f643a000700 1 --2- 172.21.15.47:0/3242571862 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f643c1017e0 0x7f643c1a0b60 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f642800d950 tx=0x7f642800dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:23:50.921 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:50.723+0000 7f6440a00700 1 -- 172.21.15.47:0/3242571862 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6428016070 con 0x7f643c1017e0 2024-03-21T10:23:50.921 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:50.723+0000 7f6440a00700 1 -- 172.21.15.47:0/3242571862 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f642800fd40 con 0x7f643c1017e0 2024-03-21T10:23:50.921 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:50.723+0000 7f643be00700 1 -- 172.21.15.47:0/3242571862 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f643c19b270 con 0x7f643c1017e0 2024-03-21T10:23:50.921 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:50.723+0000 7f6440a00700 1 -- 172.21.15.47:0/3242571862 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f642800e3b0 con 0x7f643c1017e0 2024-03-21T10:23:50.921 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:50.723+0000 7f643be00700 1 -- 172.21.15.47:0/3242571862 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f643c1a7480 con 0x7f643c1017e0 2024-03-21T10:23:50.921 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:50.724+0000 7f643be00700 1 -- 172.21.15.47:0/3242571862 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f643c06af50 con 0x7f643c1017e0 2024-03-21T10:23:50.921 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:50.726+0000 7f6440a00700 1 -- 172.21.15.47:0/3242571862 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 17) v1 ==== 137905+0+0 (secure 0 0 0) 0x7f6428019070 con 0x7f643c1017e0 2024-03-21T10:23:50.921 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:50.728+0000 7f6440a00700 1 --2- 172.21.15.47:0/3242571862 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f64200a30c0 0x7f64200a5580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:23:50.922 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:50.728+0000 7f6440a00700 1 -- 172.21.15.47:0/3242571862 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9216+0+0 (secure 0 0 0) 0x7f64280cf3e0 con 0x7f643c1017e0 2024-03-21T10:23:50.922 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:50.728+0000 7f643aa00700 1 --2- 172.21.15.47:0/3242571862 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f64200a30c0 0x7f64200a5580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:23:50.922 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:50.729+0000 7f643aa00700 1 --2- 172.21.15.47:0/3242571862 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f64200a30c0 0x7f64200a5580 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f642c006010 tx=0x7f642c00b540 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:23:50.922 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:50.730+0000 7f6440a00700 1 -- 172.21.15.47:0/3242571862 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f642809bca0 con 0x7f643c1017e0 2024-03-21T10:23:51.007 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:51.006+0000 7f643be00700 1 -- 172.21.15.47:0/3242571862 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f643c02da30 con 0x7f643c1017e0 2024-03-21T10:23:51.007 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:51.006+0000 7f6440a00700 1 -- 172.21.15.47:0/3242571862 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f642809b5a0 con 0x7f643c1017e0 2024-03-21T10:23:51.008 INFO:teuthology.orchestra.run.smithi047.stdout:55834574955 2024-03-21T10:23:51.011 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:51.009+0000 7f643be00700 1 -- 172.21.15.47:0/3242571862 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f64200a30c0 msgr2=0x7f64200a5580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:23:51.011 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:51.009+0000 7f643be00700 1 --2- 172.21.15.47:0/3242571862 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f64200a30c0 0x7f64200a5580 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f642c006010 tx=0x7f642c00b540 comp rx=0 tx=0).stop 2024-03-21T10:23:51.011 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:51.009+0000 7f643be00700 1 -- 172.21.15.47:0/3242571862 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f643c1017e0 msgr2=0x7f643c1a0b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:23:51.011 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:51.009+0000 7f643be00700 1 --2- 172.21.15.47:0/3242571862 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f643c1017e0 0x7f643c1a0b60 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f642800d950 tx=0x7f642800dc60 comp rx=0 tx=0).stop 2024-03-21T10:23:51.011 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:51.009+0000 7f643be00700 1 -- 172.21.15.47:0/3242571862 shutdown_connections 2024-03-21T10:23:51.011 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:51.009+0000 7f643be00700 1 --2- 172.21.15.47:0/3242571862 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f64200a30c0 0x7f64200a5580 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:23:51.011 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:51.009+0000 7f643be00700 1 --2- 172.21.15.47:0/3242571862 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f643c10e9e0 0x7f643c19a7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:23:51.011 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:51.009+0000 7f643be00700 1 --2- 172.21.15.47:0/3242571862 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f643c100e30 0x7f643c1a0620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:23:51.011 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:51.009+0000 7f643be00700 1 --2- 172.21.15.47:0/3242571862 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f643c1017e0 0x7f643c1a0b60 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:23:51.012 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:51.009+0000 7f643be00700 1 -- 172.21.15.47:0/3242571862 >> 172.21.15.47:0/3242571862 conn(0x7f643c0fca50 msgr2=0x7f643c0fee40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:23:51.012 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:51.010+0000 7f643be00700 1 -- 172.21.15.47:0/3242571862 shutdown_connections 2024-03-21T10:23:51.012 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:51.010+0000 7f643be00700 1 -- 172.21.15.47:0/3242571862 wait complete. 2024-03-21T10:23:51.250 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:51.249+0000 7fd386a00700 1 -- 172.21.15.47:0/2349778682 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd38010ec40 msgr2=0x7fd380111030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:23:51.251 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:51.249+0000 7fd386a00700 1 --2- 172.21.15.47:0/2349778682 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd38010ec40 0x7fd380111030 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7fd378009990 tx=0x7fd378009ca0 comp rx=0 tx=0).stop 2024-03-21T10:23:51.251 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:51.249+0000 7fd386a00700 1 -- 172.21.15.47:0/2349778682 shutdown_connections 2024-03-21T10:23:51.251 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:51.249+0000 7fd386a00700 1 --2- 172.21.15.47:0/2349778682 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd38010ec40 0x7fd380111030 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:23:51.251 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:51.249+0000 7fd386a00700 1 --2- 172.21.15.47:0/2349778682 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd380069e40 0x7fd38006a220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:23:51.251 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:51.249+0000 7fd386a00700 1 --2- 172.21.15.47:0/2349778682 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd38006a760 0x7fd38010e700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:23:51.251 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:51.249+0000 7fd386a00700 1 -- 172.21.15.47:0/2349778682 >> 172.21.15.47:0/2349778682 conn(0x7fd380077fc0 msgr2=0x7fd3800783d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:23:51.251 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:51.249+0000 7fd386a00700 1 -- 172.21.15.47:0/2349778682 shutdown_connections 2024-03-21T10:23:51.251 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:51.251+0000 7fd386a00700 1 -- 172.21.15.47:0/2349778682 wait complete. 2024-03-21T10:23:51.253 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:51.252+0000 7fd386a00700 1 Processor -- start 2024-03-21T10:23:51.253 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:51.252+0000 7fd386a00700 1 -- start start 2024-03-21T10:23:51.253 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:51.252+0000 7fd386a00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd380069e40 0x7fd3801a06d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:23:51.253 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:51.252+0000 7fd386a00700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd38006a760 0x7fd3801a0c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:23:51.253 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:51.252+0000 7fd386a00700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd38010ec40 0x7fd38019a890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:23:51.253 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:51.252+0000 7fd386a00700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3801144f0 con 0x7fd38010ec40 2024-03-21T10:23:51.253 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:51.252+0000 7fd386a00700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd380114370 con 0x7fd38006a760 2024-03-21T10:23:51.254 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:51.252+0000 7fd386a00700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd380114670 con 0x7fd380069e40 2024-03-21T10:23:51.254 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:51.253+0000 7fd377e00700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd38006a760 0x7fd3801a0c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:23:51.254 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:51.253+0000 7fd384a00700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd38010ec40 0x7fd38019a890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:23:51.254 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:51.253+0000 7fd377e00700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd38006a760 0x7fd3801a0c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:60930/0 (socket says 172.21.15.47:60930) 2024-03-21T10:23:51.254 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:51.253+0000 7fd377e00700 1 -- 172.21.15.47:0/2597977229 learned_addr learned my addr 172.21.15.47:0/2597977229 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:23:51.254 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:51.253+0000 7fd37fe00700 1 --2- 172.21.15.47:0/2597977229 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd380069e40 0x7fd3801a06d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:23:51.255 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:51.253+0000 7fd384a00700 1 -- 172.21.15.47:0/2597977229 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd380069e40 msgr2=0x7fd3801a06d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:23:51.255 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:51.253+0000 7fd384a00700 1 --2- 172.21.15.47:0/2597977229 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd380069e40 0x7fd3801a06d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:23:51.255 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:51.253+0000 7fd384a00700 1 -- 172.21.15.47:0/2597977229 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd38006a760 msgr2=0x7fd3801a0c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:23:51.255 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:51.253+0000 7fd384a00700 1 --2- 172.21.15.47:0/2597977229 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd38006a760 0x7fd3801a0c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:23:51.256 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:51.253+0000 7fd384a00700 1 -- 172.21.15.47:0/2597977229 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd38019aff0 con 0x7fd38010ec40 2024-03-21T10:23:51.256 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:51.253+0000 7fd377e00700 1 --2- 172.21.15.47:0/2597977229 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd38006a760 0x7fd3801a0c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T10:23:51.256 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:51.254+0000 7fd384a00700 1 --2- 172.21.15.47:0/2597977229 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd38010ec40 0x7fd38019a890 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7fd3780098e0 tx=0x7fd378009580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:23:51.256 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:51.254+0000 7fd37d600700 1 -- 172.21.15.47:0/2597977229 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd378027070 con 0x7fd38010ec40 2024-03-21T10:23:51.256 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:51.254+0000 7fd37d600700 1 -- 172.21.15.47:0/2597977229 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fd378022470 con 0x7fd38010ec40 2024-03-21T10:23:51.256 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:51.254+0000 7fd37d600700 1 -- 172.21.15.47:0/2597977229 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd37800b610 con 0x7fd38010ec40 2024-03-21T10:23:51.256 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:51.254+0000 7fd386a00700 1 -- 172.21.15.47:0/2597977229 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd38019b280 con 0x7fd38010ec40 2024-03-21T10:23:51.256 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:51.254+0000 7fd386a00700 1 -- 172.21.15.47:0/2597977229 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd3801a75a0 con 0x7fd38010ec40 2024-03-21T10:23:51.257 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:51.256+0000 7fd386a00700 1 -- 172.21.15.47:0/2597977229 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd380068e50 con 0x7fd38010ec40 2024-03-21T10:23:51.260 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:51.259+0000 7fd37d600700 1 -- 172.21.15.47:0/2597977229 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 17) v1 ==== 137905+0+0 (secure 0 0 0) 0x7fd37802a070 con 0x7fd38010ec40 2024-03-21T10:23:51.262 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:51.261+0000 7fd37d600700 1 --2- 172.21.15.47:0/2597977229 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7fd3640a30c0 0x7fd3640a5580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:23:51.262 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:51.261+0000 7fd37d600700 1 -- 172.21.15.47:0/2597977229 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9216+0+0 (secure 0 0 0) 0x7fd3780d0e10 con 0x7fd38010ec40 2024-03-21T10:23:51.262 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:51.261+0000 7fd37fe00700 1 --2- 172.21.15.47:0/2597977229 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7fd3640a30c0 0x7fd3640a5580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:23:51.263 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:51.262+0000 7fd37fe00700 1 --2- 172.21.15.47:0/2597977229 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7fd3640a30c0 0x7fd3640a5580 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fd38019b9d0 tx=0x7fd370009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:23:51.264 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:51.264+0000 7fd37d600700 1 -- 172.21.15.47:0/2597977229 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fd378031070 con 0x7fd38010ec40 2024-03-21T10:23:51.507 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:51.506+0000 7fd386a00700 1 -- 172.21.15.47:0/2597977229 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) v1 -- 0x7fd38002da30 con 0x7fd38010ec40 2024-03-21T10:23:51.508 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:51.506+0000 7fd37d600700 1 -- 172.21.15.47:0/2597977229 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fd378013070 con 0x7fd38010ec40 2024-03-21T10:23:51.508 INFO:teuthology.orchestra.run.smithi047.stdout:206158430282 2024-03-21T10:23:51.512 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:51.510+0000 7fd386a00700 1 -- 172.21.15.47:0/2597977229 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7fd3640a30c0 msgr2=0x7fd3640a5580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:23:51.512 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:51.510+0000 7fd386a00700 1 --2- 172.21.15.47:0/2597977229 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7fd3640a30c0 0x7fd3640a5580 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fd38019b9d0 tx=0x7fd370009260 comp rx=0 tx=0).stop 2024-03-21T10:23:51.512 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:51.510+0000 7fd386a00700 1 -- 172.21.15.47:0/2597977229 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd38010ec40 msgr2=0x7fd38019a890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:23:51.512 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:51.510+0000 7fd386a00700 1 --2- 172.21.15.47:0/2597977229 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd38010ec40 0x7fd38019a890 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7fd3780098e0 tx=0x7fd378009580 comp rx=0 tx=0).stop 2024-03-21T10:23:51.512 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:51.510+0000 7fd386a00700 1 -- 172.21.15.47:0/2597977229 shutdown_connections 2024-03-21T10:23:51.512 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:51.510+0000 7fd386a00700 1 --2- 172.21.15.47:0/2597977229 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7fd3640a30c0 0x7fd3640a5580 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:23:51.512 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:51.510+0000 7fd386a00700 1 --2- 172.21.15.47:0/2597977229 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd38010ec40 0x7fd38019a890 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:23:51.512 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:51.510+0000 7fd386a00700 1 --2- 172.21.15.47:0/2597977229 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd380069e40 0x7fd3801a06d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:23:51.512 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:51.510+0000 7fd386a00700 1 --2- 172.21.15.47:0/2597977229 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd38006a760 0x7fd3801a0c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:23:51.512 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:51.510+0000 7fd386a00700 1 -- 172.21.15.47:0/2597977229 >> 172.21.15.47:0/2597977229 conn(0x7fd380077fc0 msgr2=0x7fd3801102d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:23:51.513 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:51.511+0000 7fd386a00700 1 -- 172.21.15.47:0/2597977229 shutdown_connections 2024-03-21T10:23:51.513 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:51.511+0000 7fd386a00700 1 -- 172.21.15.47:0/2597977229 wait complete. 2024-03-21T10:23:51.693 INFO:tasks.cephadm.ceph_manager.ceph:need seq 261993005108 got 261993005114 for osd.8 2024-03-21T10:23:51.693 DEBUG:teuthology.parallel:result is None 2024-03-21T10:23:52.066 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:23:51 smithi047 ceph-mon[31547]: pgmap v427: 129 pgs: 129 active+clean; 642 KiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-03-21T10:23:52.066 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:23:51 smithi047 ceph-mon[31547]: from='client.? 172.21.15.47:0/3242571862' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-03-21T10:23:52.066 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:23:51 smithi047 ceph-mon[31547]: from='client.? 172.21.15.47:0/2597977229' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-03-21T10:23:52.071 INFO:tasks.cephadm.ceph_manager.ceph:need seq 55834574949 got 55834574955 for osd.1 2024-03-21T10:23:52.071 DEBUG:teuthology.parallel:result is None 2024-03-21T10:23:52.229 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:23:51 smithi100 ceph-mon[35193]: pgmap v427: 129 pgs: 129 active+clean; 642 KiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-03-21T10:23:52.230 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:23:51 smithi100 ceph-mon[35193]: from='client.? 172.21.15.47:0/3242571862' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-03-21T10:23:52.230 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:23:51 smithi100 ceph-mon[35193]: from='client.? 172.21.15.47:0/2597977229' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-03-21T10:23:52.240 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:23:51 smithi157 ceph-mon[35581]: pgmap v427: 129 pgs: 129 active+clean; 642 KiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-03-21T10:23:52.240 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:23:51 smithi157 ceph-mon[35581]: from='client.? 172.21.15.47:0/3242571862' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-03-21T10:23:52.240 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:23:51 smithi157 ceph-mon[35581]: from='client.? 172.21.15.47:0/2597977229' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-03-21T10:23:52.973 INFO:tasks.cephadm.ceph_manager.ceph:need seq 206158430276 got 206158430282 for osd.6 2024-03-21T10:23:52.973 DEBUG:teuthology.parallel:result is None 2024-03-21T10:23:52.973 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-03-21T10:23:52.974 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2edfee68ad7243bed422a049b6f5f58619300f11 shell --fsid 3056d51a-e76b-11ee-95cd-87774f69a715 -- ceph pg dump --format=json 2024-03-21T10:23:53.504 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/3056d51a-e76b-11ee-95cd-87774f69a715/mon.a/config 2024-03-21T10:23:53.907 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:23:53 smithi047 ceph-mon[31547]: pgmap v428: 129 pgs: 129 active+clean; 642 KiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-21T10:23:54.229 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:23:53 smithi100 ceph-mon[35193]: pgmap v428: 129 pgs: 129 active+clean; 642 KiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-21T10:23:54.242 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:23:53 smithi157 ceph-mon[35581]: pgmap v428: 129 pgs: 129 active+clean; 642 KiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-21T10:23:55.956 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:55.954+0000 7f070b800700 1 -- 172.21.15.47:0/3561632996 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f07041081c0 msgr2=0x7f07041085a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:23:55.956 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:55.954+0000 7f070b800700 1 --2- 172.21.15.47:0/3561632996 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f07041081c0 0x7f07041085a0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f06f40099c0 tx=0x7f06f4009cd0 comp rx=0 tx=0).stop 2024-03-21T10:23:55.956 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:55.954+0000 7f070b800700 1 -- 172.21.15.47:0/3561632996 shutdown_connections 2024-03-21T10:23:55.956 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:55.954+0000 7f070b800700 1 --2- 172.21.15.47:0/3561632996 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0704102ba0 0x7f070410efe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:23:55.956 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:55.954+0000 7f070b800700 1 --2- 172.21.15.47:0/3561632996 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f07041081c0 0x7f07041085a0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:23:55.957 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:55.954+0000 7f070b800700 1 --2- 172.21.15.47:0/3561632996 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f07041021e0 0x7f0704102660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:23:55.957 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:55.954+0000 7f070b800700 1 -- 172.21.15.47:0/3561632996 >> 172.21.15.47:0/3561632996 conn(0x7f07040792c0 msgr2=0x7f07040796d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:23:55.957 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:55.955+0000 7f070b800700 1 -- 172.21.15.47:0/3561632996 shutdown_connections 2024-03-21T10:23:55.957 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:23:55 smithi047 ceph-mon[31547]: pgmap v429: 129 pgs: 129 active+clean; 642 KiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-03-21T10:23:55.957 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:55.955+0000 7f070b800700 1 -- 172.21.15.47:0/3561632996 wait complete. 2024-03-21T10:23:55.958 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:55.957+0000 7f070b800700 1 Processor -- start 2024-03-21T10:23:55.960 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:55.958+0000 7f070b800700 1 -- start start 2024-03-21T10:23:55.961 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:55.959+0000 7f070b800700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f07041021e0 0x7f07041a0700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:23:55.962 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:55.960+0000 7f070b800700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0704102ba0 0x7f07041a0c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:23:55.962 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:55.960+0000 7f070b800700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f07041081c0 0x7f070419a8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:23:55.962 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:55.960+0000 7f070b800700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0704112340 con 0x7f07041081c0 2024-03-21T10:23:55.962 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:55.960+0000 7f070b800700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07041121c0 con 0x7f0704102ba0 2024-03-21T10:23:55.962 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:55.960+0000 7f070b800700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07041124c0 con 0x7f07041021e0 2024-03-21T10:23:55.963 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:55.961+0000 7f06fbe00700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0704102ba0 0x7f07041a0c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:23:55.964 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:55.961+0000 7f06fbe00700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0704102ba0 0x7f07041a0c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:39014/0 (socket says 172.21.15.47:39014) 2024-03-21T10:23:55.964 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:55.961+0000 7f06fbe00700 1 -- 172.21.15.47:0/319892728 learned_addr learned my addr 172.21.15.47:0/319892728 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:23:55.964 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:55.962+0000 7f0709800700 1 --2- 172.21.15.47:0/319892728 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f07041081c0 0x7f070419a8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:23:55.964 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:55.962+0000 7f0708e00700 1 --2- 172.21.15.47:0/319892728 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f07041021e0 0x7f07041a0700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:23:55.965 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:55.964+0000 7f0709800700 1 -- 172.21.15.47:0/319892728 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f07041021e0 msgr2=0x7f07041a0700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:23:55.965 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:55.964+0000 7f0709800700 1 --2- 172.21.15.47:0/319892728 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f07041021e0 0x7f07041a0700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:23:55.966 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:55.964+0000 7f0709800700 1 -- 172.21.15.47:0/319892728 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0704102ba0 msgr2=0x7f07041a0c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:23:55.966 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:55.964+0000 7f0709800700 1 --2- 172.21.15.47:0/319892728 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0704102ba0 0x7f07041a0c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:23:55.966 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:55.964+0000 7f0709800700 1 -- 172.21.15.47:0/319892728 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f070419b020 con 0x7f07041081c0 2024-03-21T10:23:55.966 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:55.964+0000 7f0708e00700 1 --2- 172.21.15.47:0/319892728 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f07041021e0 0x7f07041a0700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T10:23:55.966 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:55.965+0000 7f0709800700 1 --2- 172.21.15.47:0/319892728 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f07041081c0 0x7f070419a8c0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f06fc00efc0 tx=0x7f06fc00c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:23:55.967 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:55.966+0000 7f0702000700 1 -- 172.21.15.47:0/319892728 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06fc0092e0 con 0x7f07041081c0 2024-03-21T10:23:55.968 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:55.967+0000 7f0702000700 1 -- 172.21.15.47:0/319892728 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f06fc00c9f0 con 0x7f07041081c0 2024-03-21T10:23:55.968 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:55.967+0000 7f0702000700 1 -- 172.21.15.47:0/319892728 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06fc0092e0 con 0x7f07041081c0 2024-03-21T10:23:55.969 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:55.967+0000 7f070b800700 1 -- 172.21.15.47:0/319892728 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f070419b2b0 con 0x7f07041081c0 2024-03-21T10:23:55.969 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:55.968+0000 7f070b800700 1 -- 172.21.15.47:0/319892728 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f07041a75d0 con 0x7f07041081c0 2024-03-21T10:23:55.973 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:55.972+0000 7f070b800700 1 -- 172.21.15.47:0/319892728 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0704068e50 con 0x7f07041081c0 2024-03-21T10:23:55.989 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:55.984+0000 7f0702000700 1 -- 172.21.15.47:0/319892728 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 17) v1 ==== 137905+0+0 (secure 0 0 0) 0x7f06fc012050 con 0x7f07041081c0 2024-03-21T10:23:55.989 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:55.987+0000 7f0702000700 1 --2- 172.21.15.47:0/319892728 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f06e80a32c0 0x7f06e80a5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:23:55.989 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:55.987+0000 7f0702000700 1 -- 172.21.15.47:0/319892728 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9216+0+0 (secure 0 0 0) 0x7f06fc0cf320 con 0x7f07041081c0 2024-03-21T10:23:55.989 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:55.987+0000 7f0708e00700 1 --2- 172.21.15.47:0/319892728 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f06e80a32c0 0x7f06e80a5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:23:55.990 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:55.988+0000 7f0708e00700 1 --2- 172.21.15.47:0/319892728 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f06e80a32c0 0x7f06e80a5780 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f06f4000c00 tx=0x7f06f4005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:23:55.992 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:55.991+0000 7f0702000700 1 -- 172.21.15.47:0/319892728 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f06fc010040 con 0x7f07041081c0 2024-03-21T10:23:56.229 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:23:55 smithi100 ceph-mon[35193]: pgmap v429: 129 pgs: 129 active+clean; 642 KiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-03-21T10:23:56.239 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:23:55 smithi157 ceph-mon[35581]: pgmap v429: 129 pgs: 129 active+clean; 642 KiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-03-21T10:23:56.240 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:56.239+0000 7f070b800700 1 -- 172.21.15.47:0/319892728 --> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f0704075de0 con 0x7f06e80a32c0 2024-03-21T10:23:56.253 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:56.251+0000 7f0702000700 1 -- 172.21.15.47:0/319892728 <== mgr.14152 v2:172.21.15.47:6800/2823885994 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+340428 (secure 0 0 0) 0x7f06f407b1e0 con 0x7f06e80a32c0 2024-03-21T10:23:56.257 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-21T10:23:56.260 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:56.258+0000 7f070b800700 1 -- 172.21.15.47:0/319892728 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f06e80a32c0 msgr2=0x7f06e80a5780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:23:56.260 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:56.258+0000 7f070b800700 1 --2- 172.21.15.47:0/319892728 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f06e80a32c0 0x7f06e80a5780 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f06f4000c00 tx=0x7f06f4005fd0 comp rx=0 tx=0).stop 2024-03-21T10:23:56.260 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:56.258+0000 7f070b800700 1 -- 172.21.15.47:0/319892728 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f07041081c0 msgr2=0x7f070419a8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:23:56.260 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:56.258+0000 7f070b800700 1 --2- 172.21.15.47:0/319892728 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f07041081c0 0x7f070419a8c0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f06fc00efc0 tx=0x7f06fc00c3a0 comp rx=0 tx=0).stop 2024-03-21T10:23:56.260 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:56.258+0000 7f070b800700 1 -- 172.21.15.47:0/319892728 shutdown_connections 2024-03-21T10:23:56.260 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:56.259+0000 7f070b800700 1 --2- 172.21.15.47:0/319892728 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f06e80a32c0 0x7f06e80a5780 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:23:56.261 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:56.259+0000 7f070b800700 1 --2- 172.21.15.47:0/319892728 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f07041081c0 0x7f070419a8c0 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:23:56.261 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:56.259+0000 7f070b800700 1 --2- 172.21.15.47:0/319892728 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f07041021e0 0x7f07041a0700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:23:56.261 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:56.259+0000 7f070b800700 1 --2- 172.21.15.47:0/319892728 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0704102ba0 0x7f07041a0c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:23:56.261 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:56.259+0000 7f070b800700 1 -- 172.21.15.47:0/319892728 >> 172.21.15.47:0/319892728 conn(0x7f07040792c0 msgr2=0x7f07040746c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:23:56.261 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:56.259+0000 7f070b800700 1 -- 172.21.15.47:0/319892728 shutdown_connections 2024-03-21T10:23:56.261 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:56.259+0000 7f070b800700 1 -- 172.21.15.47:0/319892728 wait complete. 2024-03-21T10:23:56.261 INFO:teuthology.orchestra.run.smithi047.stderr:dumped all 2024-03-21T10:23:57.110 INFO:teuthology.orchestra.run.smithi047.stdout:{"pg_ready":true,"pg_map":{"version":430,"stamp":"2024-03-21T10:23:56.066508+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":657207,"num_objects":102,"num_object_clones":0,"num_object_copies":308,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":1021,"num_read_kb":634,"num_write":423,"num_write_kb":3076,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13,"num_bytes_recovered":1185317,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":347,"ondisk_log_size":347,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":397320,"kb_used_data":14804,"kb_used_omap":18,"kb_used_meta":382317,"kb_avail":1124544504,"statfs":{"total":1151940427776,"available":1151533572096,"internally_reserved":0,"allocated":15159296,"data_stored":8265803,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19142,"internal_metadata":391492922},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":105,"num_read_kb":50,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004699"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":29,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T10:23:27.837224+0000","last_change":"2024-03-21T10:21:58.872480+0000","last_active":"2024-03-21T10:23:27.837224+0000","last_peered":"2024-03-21T10:23:27.837224+0000","last_clean":"2024-03-21T10:23:27.837224+0000","last_became_active":"2024-03-21T10:21:58.872143+0000","last_became_peered":"2024-03-21T10:21:58.872143+0000","last_unstale":"2024-03-21T10:23:27.837224+0000","last_undegraded":"2024-03-21T10:23:27.837224+0000","last_fullsized":"2024-03-21T10:23:27.837224+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_clean_scrub_stamp":"2024-03-21T10:21:57.813712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T15:20:00.409201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":29,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T10:23:27.834972+0000","last_change":"2024-03-21T10:21:58.871130+0000","last_active":"2024-03-21T10:23:27.834972+0000","last_peered":"2024-03-21T10:23:27.834972+0000","last_clean":"2024-03-21T10:23:27.834972+0000","last_became_active":"2024-03-21T10:21:58.870750+0000","last_became_peered":"2024-03-21T10:21:58.870750+0000","last_unstale":"2024-03-21T10:23:27.834972+0000","last_undegraded":"2024-03-21T10:23:27.834972+0000","last_fullsized":"2024-03-21T10:23:27.834972+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_clean_scrub_stamp":"2024-03-21T10:21:57.813712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T17:38:04.830212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":29,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T10:23:27.834692+0000","last_change":"2024-03-21T10:21:58.868026+0000","last_active":"2024-03-21T10:23:27.834692+0000","last_peered":"2024-03-21T10:23:27.834692+0000","last_clean":"2024-03-21T10:23:27.834692+0000","last_became_active":"2024-03-21T10:21:58.859877+0000","last_became_peered":"2024-03-21T10:21:58.859877+0000","last_unstale":"2024-03-21T10:23:27.834692+0000","last_undegraded":"2024-03-21T10:23:27.834692+0000","last_fullsized":"2024-03-21T10:23:27.834692+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_clean_scrub_stamp":"2024-03-21T10:21:57.813712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T14:45:12.870111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":15,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T10:23:27.642260+0000","last_change":"2024-03-21T10:22:27.231695+0000","last_active":"2024-03-21T10:23:27.642260+0000","last_peered":"2024-03-21T10:23:27.642260+0000","last_clean":"2024-03-21T10:23:27.642260+0000","last_became_active":"2024-03-21T10:22:27.231149+0000","last_became_peered":"2024-03-21T10:22:27.231149+0000","last_unstale":"2024-03-21T10:23:27.642260+0000","last_undegraded":"2024-03-21T10:23:27.642260+0000","last_fullsized":"2024-03-21T10:23:27.642260+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_clean_scrub_stamp":"2024-03-21T10:21:57.813712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T18:19:19.105366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,2],"acting":[9,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":29,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T10:23:27.643768+0000","last_change":"2024-03-21T10:21:58.853998+0000","last_active":"2024-03-21T10:23:27.643768+0000","last_peered":"2024-03-21T10:23:27.643768+0000","last_clean":"2024-03-21T10:23:27.643768+0000","last_became_active":"2024-03-21T10:21:58.853059+0000","last_became_peered":"2024-03-21T10:21:58.853059+0000","last_unstale":"2024-03-21T10:23:27.643768+0000","last_undegraded":"2024-03-21T10:23:27.643768+0000","last_fullsized":"2024-03-21T10:23:27.643768+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_clean_scrub_stamp":"2024-03-21T10:21:57.813712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T17:06:36.859522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":29,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T10:23:27.744443+0000","last_change":"2024-03-21T10:21:58.855535+0000","last_active":"2024-03-21T10:23:27.744443+0000","last_peered":"2024-03-21T10:23:27.744443+0000","last_clean":"2024-03-21T10:23:27.744443+0000","last_became_active":"2024-03-21T10:21:58.855181+0000","last_became_peered":"2024-03-21T10:21:58.855181+0000","last_unstale":"2024-03-21T10:23:27.744443+0000","last_undegraded":"2024-03-21T10:23:27.744443+0000","last_fullsized":"2024-03-21T10:23:27.744443+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_clean_scrub_stamp":"2024-03-21T10:21:57.813712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T14:46:35.045646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":29,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T10:23:27.643598+0000","last_change":"2024-03-21T10:21:58.856527+0000","last_active":"2024-03-21T10:23:27.643598+0000","last_peered":"2024-03-21T10:23:27.643598+0000","last_clean":"2024-03-21T10:23:27.643598+0000","last_became_active":"2024-03-21T10:21:58.856176+0000","last_became_peered":"2024-03-21T10:21:58.856176+0000","last_unstale":"2024-03-21T10:23:27.643598+0000","last_undegraded":"2024-03-21T10:23:27.643598+0000","last_fullsized":"2024-03-21T10:23:27.643598+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_clean_scrub_stamp":"2024-03-21T10:21:57.813712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T21:34:07.804293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":29,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T10:23:27.638323+0000","last_change":"2024-03-21T10:21:58.846260+0000","last_active":"2024-03-21T10:23:27.638323+0000","last_peered":"2024-03-21T10:23:27.638323+0000","last_clean":"2024-03-21T10:23:27.638323+0000","last_became_active":"2024-03-21T10:21:58.844245+0000","last_became_peered":"2024-03-21T10:21:58.844245+0000","last_unstale":"2024-03-21T10:23:27.638323+0000","last_undegraded":"2024-03-21T10:23:27.638323+0000","last_fullsized":"2024-03-21T10:23:27.638323+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_clean_scrub_stamp":"2024-03-21T10:21:57.813712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T16:37:07.480867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":29,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T10:23:27.833470+0000","last_change":"2024-03-21T10:21:58.858413+0000","last_active":"2024-03-21T10:23:27.833470+0000","last_peered":"2024-03-21T10:23:27.833470+0000","last_clean":"2024-03-21T10:23:27.833470+0000","last_became_active":"2024-03-21T10:21:58.858007+0000","last_became_peered":"2024-03-21T10:21:58.858007+0000","last_unstale":"2024-03-21T10:23:27.833470+0000","last_undegraded":"2024-03-21T10:23:27.833470+0000","last_fullsized":"2024-03-21T10:23:27.833470+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_clean_scrub_stamp":"2024-03-21T10:21:57.813712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T21:49:12.428918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":29,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T10:23:27.745816+0000","last_change":"2024-03-21T10:21:58.864393+0000","last_active":"2024-03-21T10:23:27.745816+0000","last_peered":"2024-03-21T10:23:27.745816+0000","last_clean":"2024-03-21T10:23:27.745816+0000","last_became_active":"2024-03-21T10:21:58.864061+0000","last_became_peered":"2024-03-21T10:21:58.864061+0000","last_unstale":"2024-03-21T10:23:27.745816+0000","last_undegraded":"2024-03-21T10:23:27.745816+0000","last_fullsized":"2024-03-21T10:23:27.745816+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_clean_scrub_stamp":"2024-03-21T10:21:57.813712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T20:19:44.209612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":29,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T10:23:27.645363+0000","last_change":"2024-03-21T10:21:58.873885+0000","last_active":"2024-03-21T10:23:27.645363+0000","last_peered":"2024-03-21T10:23:27.645363+0000","last_clean":"2024-03-21T10:23:27.645363+0000","last_became_active":"2024-03-21T10:21:58.869738+0000","last_became_peered":"2024-03-21T10:21:58.869738+0000","last_unstale":"2024-03-21T10:23:27.645363+0000","last_undegraded":"2024-03-21T10:23:27.645363+0000","last_fullsized":"2024-03-21T10:23:27.645363+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_clean_scrub_stamp":"2024-03-21T10:21:57.813712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T13:33:26.816720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":29,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T10:23:27.640767+0000","last_change":"2024-03-21T10:21:58.867563+0000","last_active":"2024-03-21T10:23:27.640767+0000","last_peered":"2024-03-21T10:23:27.640767+0000","last_clean":"2024-03-21T10:23:27.640767+0000","last_became_active":"2024-03-21T10:21:58.863455+0000","last_became_peered":"2024-03-21T10:21:58.863455+0000","last_unstale":"2024-03-21T10:23:27.640767+0000","last_undegraded":"2024-03-21T10:23:27.640767+0000","last_fullsized":"2024-03-21T10:23:27.640767+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_clean_scrub_stamp":"2024-03-21T10:21:57.813712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T15:04:16.636126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":35,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T10:23:27.742185+0000","last_change":"2024-03-21T10:22:27.213460+0000","last_active":"2024-03-21T10:23:27.742185+0000","last_peered":"2024-03-21T10:23:27.742185+0000","last_clean":"2024-03-21T10:23:27.742185+0000","last_became_active":"2024-03-21T10:22:27.212961+0000","last_became_peered":"2024-03-21T10:22:27.212961+0000","last_unstale":"2024-03-21T10:23:27.742185+0000","last_undegraded":"2024-03-21T10:23:27.742185+0000","last_fullsized":"2024-03-21T10:23:27.742185+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_clean_scrub_stamp":"2024-03-21T10:21:57.813712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T18:26:03.881767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,9],"acting":[6,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":29,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T10:23:27.834277+0000","last_change":"2024-03-21T10:21:58.870851+0000","last_active":"2024-03-21T10:23:27.834277+0000","last_peered":"2024-03-21T10:23:27.834277+0000","last_clean":"2024-03-21T10:23:27.834277+0000","last_became_active":"2024-03-21T10:21:58.870132+0000","last_became_peered":"2024-03-21T10:21:58.870132+0000","last_unstale":"2024-03-21T10:23:27.834277+0000","last_undegraded":"2024-03-21T10:23:27.834277+0000","last_fullsized":"2024-03-21T10:23:27.834277+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_clean_scrub_stamp":"2024-03-21T10:21:57.813712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T20:20:04.207090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":29,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T10:23:27.834360+0000","last_change":"2024-03-21T10:21:58.865084+0000","last_active":"2024-03-21T10:23:27.834360+0000","last_peered":"2024-03-21T10:23:27.834360+0000","last_clean":"2024-03-21T10:23:27.834360+0000","last_became_active":"2024-03-21T10:21:58.864583+0000","last_became_peered":"2024-03-21T10:21:58.864583+0000","last_unstale":"2024-03-21T10:23:27.834360+0000","last_undegraded":"2024-03-21T10:23:27.834360+0000","last_fullsized":"2024-03-21T10:23:27.834360+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_clean_scrub_stamp":"2024-03-21T10:21:57.813712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T14:09:16.781462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":29,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T10:23:27.832948+0000","last_change":"2024-03-21T10:21:58.870370+0000","last_active":"2024-03-21T10:23:27.832948+0000","last_peered":"2024-03-21T10:23:27.832948+0000","last_clean":"2024-03-21T10:23:27.832948+0000","last_became_active":"2024-03-21T10:21:58.870044+0000","last_became_peered":"2024-03-21T10:21:58.870044+0000","last_unstale":"2024-03-21T10:23:27.832948+0000","last_undegraded":"2024-03-21T10:23:27.832948+0000","last_fullsized":"2024-03-21T10:23:27.832948+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_clean_scrub_stamp":"2024-03-21T10:21:57.813712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T16:02:00.342426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":29,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T10:23:27.836442+0000","last_change":"2024-03-21T10:21:58.872787+0000","last_active":"2024-03-21T10:23:27.836442+0000","last_peered":"2024-03-21T10:23:27.836442+0000","last_clean":"2024-03-21T10:23:27.836442+0000","last_became_active":"2024-03-21T10:21:58.872390+0000","last_became_peered":"2024-03-21T10:21:58.872390+0000","last_unstale":"2024-03-21T10:23:27.836442+0000","last_undegraded":"2024-03-21T10:23:27.836442+0000","last_fullsized":"2024-03-21T10:23:27.836442+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_clean_scrub_stamp":"2024-03-21T10:21:57.813712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T15:56:51.798173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":29,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T10:23:27.642294+0000","last_change":"2024-03-21T10:21:58.858474+0000","last_active":"2024-03-21T10:23:27.642294+0000","last_peered":"2024-03-21T10:23:27.642294+0000","last_clean":"2024-03-21T10:23:27.642294+0000","last_became_active":"2024-03-21T10:21:58.858076+0000","last_became_peered":"2024-03-21T10:21:58.858076+0000","last_unstale":"2024-03-21T10:23:27.642294+0000","last_undegraded":"2024-03-21T10:23:27.642294+0000","last_fullsized":"2024-03-21T10:23:27.642294+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_clean_scrub_stamp":"2024-03-21T10:21:57.813712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T12:25:17.160405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":29,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T10:23:27.648925+0000","last_change":"2024-03-21T10:21:58.854156+0000","last_active":"2024-03-21T10:23:27.648925+0000","last_peered":"2024-03-21T10:23:27.648925+0000","last_clean":"2024-03-21T10:23:27.648925+0000","last_became_active":"2024-03-21T10:21:58.853758+0000","last_became_peered":"2024-03-21T10:21:58.853758+0000","last_unstale":"2024-03-21T10:23:27.648925+0000","last_undegraded":"2024-03-21T10:23:27.648925+0000","last_fullsized":"2024-03-21T10:23:27.648925+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_clean_scrub_stamp":"2024-03-21T10:21:57.813712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T16:58:54.854826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":29,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T10:23:27.838445+0000","last_change":"2024-03-21T10:23:27.838445+0000","last_active":"2024-03-21T10:23:27.838445+0000","last_peered":"2024-03-21T10:23:27.838445+0000","last_clean":"2024-03-21T10:23:27.838445+0000","last_became_active":"2024-03-21T10:23:27.836992+0000","last_became_peered":"2024-03-21T10:23:27.836992+0000","last_unstale":"2024-03-21T10:23:27.838445+0000","last_undegraded":"2024-03-21T10:23:27.838445+0000","last_fullsized":"2024-03-21T10:23:27.838445+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:22:02.269614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:22:02.269614+0000","last_clean_scrub_stamp":"2024-03-21T10:22:02.269614+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T15:34:51.615332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8,0],"acting":[2,3,8,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"93'13","reported_seq":128,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T10:23:50.782165+0000","last_change":"2024-03-21T10:21:57.859554+0000","last_active":"2024-03-21T10:23:50.782165+0000","last_peered":"2024-03-21T10:23:50.782165+0000","last_clean":"2024-03-21T10:23:50.782165+0000","last_became_active":"2024-03-21T10:21:57.857997+0000","last_became_peered":"2024-03-21T10:21:57.857997+0000","last_unstale":"2024-03-21T10:23:50.782165+0000","last_undegraded":"2024-03-21T10:23:50.782165+0000","last_fullsized":"2024-03-21T10:23:50.782165+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:21:56.788908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:21:56.788908+0000","last_clean_scrub_stamp":"2024-03-21T10:21:56.788908+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T14:27:47.516832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":84,"num_read_kb":84,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":29,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T10:23:27.638918+0000","last_change":"2024-03-21T10:21:58.850197+0000","last_active":"2024-03-21T10:23:27.638918+0000","last_peered":"2024-03-21T10:23:27.638918+0000","last_clean":"2024-03-21T10:23:27.638918+0000","last_became_active":"2024-03-21T10:21:58.849029+0000","last_became_peered":"2024-03-21T10:21:58.849029+0000","last_unstale":"2024-03-21T10:23:27.638918+0000","last_undegraded":"2024-03-21T10:23:27.638918+0000","last_fullsized":"2024-03-21T10:23:27.638918+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_clean_scrub_stamp":"2024-03-21T10:21:57.813712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T15:46:49.077382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":22,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T10:23:27.639994+0000","last_change":"2024-03-21T10:22:03.320716+0000","last_active":"2024-03-21T10:23:27.639994+0000","last_peered":"2024-03-21T10:23:27.639994+0000","last_clean":"2024-03-21T10:23:27.639994+0000","last_became_active":"2024-03-21T10:22:03.320262+0000","last_became_peered":"2024-03-21T10:22:03.320262+0000","last_unstale":"2024-03-21T10:23:27.639994+0000","last_undegraded":"2024-03-21T10:23:27.639994+0000","last_fullsized":"2024-03-21T10:23:27.639994+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:22:02.269614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:22:02.269614+0000","last_clean_scrub_stamp":"2024-03-21T10:22:02.269614+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T16:16:07.423168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"92'2","reported_seq":33,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T10:23:27.638962+0000","last_change":"2024-03-21T10:21:57.863642+0000","last_active":"2024-03-21T10:23:27.638962+0000","last_peered":"2024-03-21T10:23:27.638962+0000","last_clean":"2024-03-21T10:23:27.638962+0000","last_became_active":"2024-03-21T10:21:57.863330+0000","last_became_peered":"2024-03-21T10:21:57.863330+0000","last_unstale":"2024-03-21T10:23:27.638962+0000","last_undegraded":"2024-03-21T10:23:27.638962+0000","last_fullsized":"2024-03-21T10:23:27.638962+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:21:56.788908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:21:56.788908+0000","last_clean_scrub_stamp":"2024-03-21T10:21:56.788908+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T14:45:02.131458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":29,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T10:23:27.834954+0000","last_change":"2024-03-21T10:21:58.862940+0000","last_active":"2024-03-21T10:23:27.834954+0000","last_peered":"2024-03-21T10:23:27.834954+0000","last_clean":"2024-03-21T10:23:27.834954+0000","last_became_active":"2024-03-21T10:21:58.862489+0000","last_became_peered":"2024-03-21T10:21:58.862489+0000","last_unstale":"2024-03-21T10:23:27.834954+0000","last_undegraded":"2024-03-21T10:23:27.834954+0000","last_fullsized":"2024-03-21T10:23:27.834954+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_clean_scrub_stamp":"2024-03-21T10:21:57.813712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T19:28:02.175642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":22,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T10:23:27.640475+0000","last_change":"2024-03-21T10:22:03.318660+0000","last_active":"2024-03-21T10:23:27.640475+0000","last_peered":"2024-03-21T10:23:27.640475+0000","last_clean":"2024-03-21T10:23:27.640475+0000","last_became_active":"2024-03-21T10:22:03.318216+0000","last_became_peered":"2024-03-21T10:22:03.318216+0000","last_unstale":"2024-03-21T10:23:27.640475+0000","last_undegraded":"2024-03-21T10:23:27.640475+0000","last_fullsized":"2024-03-21T10:23:27.640475+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:22:02.269614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:22:02.269614+0000","last_clean_scrub_stamp":"2024-03-21T10:22:02.269614+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T16:36:39.930726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"93'12","reported_seq":30,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T10:23:27.837073+0000","last_change":"2024-03-21T10:22:27.239342+0000","last_active":"2024-03-21T10:23:27.837073+0000","last_peered":"2024-03-21T10:23:27.837073+0000","last_clean":"2024-03-21T10:23:27.837073+0000","last_became_active":"2024-03-21T10:22:27.224086+0000","last_became_peered":"2024-03-21T10:22:27.224086+0000","last_unstale":"2024-03-21T10:23:27.837073+0000","last_undegraded":"2024-03-21T10:23:27.837073+0000","last_fullsized":"2024-03-21T10:23:27.837073+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:21:56.788908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:21:56.788908+0000","last_clean_scrub_stamp":"2024-03-21T10:21:56.788908+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T20:46:45.951485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,7],"acting":[11,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":29,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T10:23:27.833191+0000","last_change":"2024-03-21T10:21:58.873239+0000","last_active":"2024-03-21T10:23:27.833191+0000","last_peered":"2024-03-21T10:23:27.833191+0000","last_clean":"2024-03-21T10:23:27.833191+0000","last_became_active":"2024-03-21T10:21:58.872877+0000","last_became_peered":"2024-03-21T10:21:58.872877+0000","last_unstale":"2024-03-21T10:23:27.833191+0000","last_undegraded":"2024-03-21T10:23:27.833191+0000","last_fullsized":"2024-03-21T10:23:27.833191+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_clean_scrub_stamp":"2024-03-21T10:21:57.813712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T16:22:49.070697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":22,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T10:23:27.640146+0000","last_change":"2024-03-21T10:22:03.311589+0000","last_active":"2024-03-21T10:23:27.640146+0000","last_peered":"2024-03-21T10:23:27.640146+0000","last_clean":"2024-03-21T10:23:27.640146+0000","last_became_active":"2024-03-21T10:22:03.311134+0000","last_became_peered":"2024-03-21T10:22:03.311134+0000","last_unstale":"2024-03-21T10:23:27.640146+0000","last_undegraded":"2024-03-21T10:23:27.640146+0000","last_fullsized":"2024-03-21T10:23:27.640146+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:22:02.269614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:22:02.269614+0000","last_clean_scrub_stamp":"2024-03-21T10:22:02.269614+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T10:22:29.138348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"93'5","reported_seq":37,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T10:23:27.646116+0000","last_change":"2024-03-21T10:21:57.858981+0000","last_active":"2024-03-21T10:23:27.646116+0000","last_peered":"2024-03-21T10:23:27.646116+0000","last_clean":"2024-03-21T10:23:27.646116+0000","last_became_active":"2024-03-21T10:21:57.858564+0000","last_became_peered":"2024-03-21T10:21:57.858564+0000","last_unstale":"2024-03-21T10:23:27.646116+0000","last_undegraded":"2024-03-21T10:23:27.646116+0000","last_fullsized":"2024-03-21T10:23:27.646116+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:21:56.788908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:21:56.788908+0000","last_clean_scrub_stamp":"2024-03-21T10:21:56.788908+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T10:45:09.488194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":29,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T10:23:27.751873+0000","last_change":"2024-03-21T10:21:58.858053+0000","last_active":"2024-03-21T10:23:27.751873+0000","last_peered":"2024-03-21T10:23:27.751873+0000","last_clean":"2024-03-21T10:23:27.751873+0000","last_became_active":"2024-03-21T10:21:58.857688+0000","last_became_peered":"2024-03-21T10:21:58.857688+0000","last_unstale":"2024-03-21T10:23:27.751873+0000","last_undegraded":"2024-03-21T10:23:27.751873+0000","last_fullsized":"2024-03-21T10:23:27.751873+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_clean_scrub_stamp":"2024-03-21T10:21:57.813712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T19:45:13.043034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":29,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T10:23:27.842200+0000","last_change":"2024-03-21T10:23:27.842200+0000","last_active":"2024-03-21T10:23:27.842200+0000","last_peered":"2024-03-21T10:23:27.842200+0000","last_clean":"2024-03-21T10:23:27.842200+0000","last_became_active":"2024-03-21T10:23:27.840749+0000","last_became_peered":"2024-03-21T10:23:27.840749+0000","last_unstale":"2024-03-21T10:23:27.842200+0000","last_undegraded":"2024-03-21T10:23:27.842200+0000","last_fullsized":"2024-03-21T10:23:27.842200+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:22:02.269614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:22:02.269614+0000","last_clean_scrub_stamp":"2024-03-21T10:22:02.269614+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T22:01:31.714771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,7,11],"acting":[2,8,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"91'5","reported_seq":135,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T10:23:51.810623+0000","last_change":"2024-03-21T10:22:27.235744+0000","last_active":"2024-03-21T10:23:51.810623+0000","last_peered":"2024-03-21T10:23:51.810623+0000","last_clean":"2024-03-21T10:23:51.810623+0000","last_became_active":"2024-03-21T10:22:27.219999+0000","last_became_peered":"2024-03-21T10:22:27.219999+0000","last_unstale":"2024-03-21T10:23:51.810623+0000","last_undegraded":"2024-03-21T10:23:51.810623+0000","last_fullsized":"2024-03-21T10:23:51.810623+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:21:56.788908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:21:56.788908+0000","last_clean_scrub_stamp":"2024-03-21T10:21:56.788908+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T18:15:41.288034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":87,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":622,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7],"acting":[5,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":29,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T10:23:27.644194+0000","last_change":"2024-03-21T10:21:58.854752+0000","last_active":"2024-03-21T10:23:27.644194+0000","last_peered":"2024-03-21T10:23:27.644194+0000","last_clean":"2024-03-21T10:23:27.644194+0000","last_became_active":"2024-03-21T10:21:58.854442+0000","last_became_peered":"2024-03-21T10:21:58.854442+0000","last_unstale":"2024-03-21T10:23:27.644194+0000","last_undegraded":"2024-03-21T10:23:27.644194+0000","last_fullsized":"2024-03-21T10:23:27.644194+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_clean_scrub_stamp":"2024-03-21T10:21:57.813712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T15:12:40.590585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":22,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T10:23:27.648186+0000","last_change":"2024-03-21T10:22:03.319247+0000","last_active":"2024-03-21T10:23:27.648186+0000","last_peered":"2024-03-21T10:23:27.648186+0000","last_clean":"2024-03-21T10:23:27.648186+0000","last_became_active":"2024-03-21T10:22:03.318731+0000","last_became_peered":"2024-03-21T10:22:03.318731+0000","last_unstale":"2024-03-21T10:23:27.648186+0000","last_undegraded":"2024-03-21T10:23:27.648186+0000","last_fullsized":"2024-03-21T10:23:27.648186+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:22:02.269614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:22:02.269614+0000","last_clean_scrub_stamp":"2024-03-21T10:22:02.269614+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T20:06:36.519559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"93'7","reported_seq":44,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T10:23:27.744356+0000","last_change":"2024-03-21T10:22:27.222981+0000","last_active":"2024-03-21T10:23:27.744356+0000","last_peered":"2024-03-21T10:23:27.744356+0000","last_clean":"2024-03-21T10:23:27.744356+0000","last_became_active":"2024-03-21T10:22:27.221088+0000","last_became_peered":"2024-03-21T10:22:27.221088+0000","last_unstale":"2024-03-21T10:23:27.744356+0000","last_undegraded":"2024-03-21T10:23:27.744356+0000","last_fullsized":"2024-03-21T10:23:27.744356+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:21:56.788908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:21:56.788908+0000","last_clean_scrub_stamp":"2024-03-21T10:21:56.788908+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T15:43:11.740353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,6],"acting":[10,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":35,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T10:23:27.751308+0000","last_change":"2024-03-21T10:22:27.226733+0000","last_active":"2024-03-21T10:23:27.751308+0000","last_peered":"2024-03-21T10:23:27.751308+0000","last_clean":"2024-03-21T10:23:27.751308+0000","last_became_active":"2024-03-21T10:22:27.225576+0000","last_became_peered":"2024-03-21T10:22:27.225576+0000","last_unstale":"2024-03-21T10:23:27.751308+0000","last_undegraded":"2024-03-21T10:23:27.751308+0000","last_fullsized":"2024-03-21T10:23:27.751308+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_clean_scrub_stamp":"2024-03-21T10:21:57.813712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T20:06:09.534268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,3],"acting":[5,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":22,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T10:23:27.836108+0000","last_change":"2024-03-21T10:22:03.321595+0000","last_active":"2024-03-21T10:23:27.836108+0000","last_peered":"2024-03-21T10:23:27.836108+0000","last_clean":"2024-03-21T10:23:27.836108+0000","last_became_active":"2024-03-21T10:22:03.320897+0000","last_became_peered":"2024-03-21T10:22:03.320897+0000","last_unstale":"2024-03-21T10:23:27.836108+0000","last_undegraded":"2024-03-21T10:23:27.836108+0000","last_fullsized":"2024-03-21T10:23:27.836108+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:22:02.269614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:22:02.269614+0000","last_clean_scrub_stamp":"2024-03-21T10:22:02.269614+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T17:37:02.633709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"93'3","reported_seq":34,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T10:23:27.742152+0000","last_change":"2024-03-21T10:21:57.854801+0000","last_active":"2024-03-21T10:23:27.742152+0000","last_peered":"2024-03-21T10:23:27.742152+0000","last_clean":"2024-03-21T10:23:27.742152+0000","last_became_active":"2024-03-21T10:21:57.854443+0000","last_became_peered":"2024-03-21T10:21:57.854443+0000","last_unstale":"2024-03-21T10:23:27.742152+0000","last_undegraded":"2024-03-21T10:23:27.742152+0000","last_fullsized":"2024-03-21T10:23:27.742152+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:21:56.788908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:21:56.788908+0000","last_clean_scrub_stamp":"2024-03-21T10:21:56.788908+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T15:49:23.573360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":29,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T10:23:27.638208+0000","last_change":"2024-03-21T10:21:58.857167+0000","last_active":"2024-03-21T10:23:27.638208+0000","last_peered":"2024-03-21T10:23:27.638208+0000","last_clean":"2024-03-21T10:23:27.638208+0000","last_became_active":"2024-03-21T10:21:58.856827+0000","last_became_peered":"2024-03-21T10:21:58.856827+0000","last_unstale":"2024-03-21T10:23:27.638208+0000","last_undegraded":"2024-03-21T10:23:27.638208+0000","last_fullsized":"2024-03-21T10:23:27.638208+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_clean_scrub_stamp":"2024-03-21T10:21:57.813712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T15:04:43.848209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":22,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T10:23:27.834872+0000","last_change":"2024-03-21T10:22:03.306554+0000","last_active":"2024-03-21T10:23:27.834872+0000","last_peered":"2024-03-21T10:23:27.834872+0000","last_clean":"2024-03-21T10:23:27.834872+0000","last_became_active":"2024-03-21T10:22:03.306087+0000","last_became_peered":"2024-03-21T10:22:03.306087+0000","last_unstale":"2024-03-21T10:23:27.834872+0000","last_undegraded":"2024-03-21T10:23:27.834872+0000","last_fullsized":"2024-03-21T10:23:27.834872+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:22:02.269614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:22:02.269614+0000","last_clean_scrub_stamp":"2024-03-21T10:22:02.269614+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T11:36:35.344770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"93'3","reported_seq":35,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T10:23:27.750933+0000","last_change":"2024-03-21T10:21:57.860343+0000","last_active":"2024-03-21T10:23:27.750933+0000","last_peered":"2024-03-21T10:23:27.750933+0000","last_clean":"2024-03-21T10:23:27.750933+0000","last_became_active":"2024-03-21T10:21:57.859808+0000","last_became_peered":"2024-03-21T10:21:57.859808+0000","last_unstale":"2024-03-21T10:23:27.750933+0000","last_undegraded":"2024-03-21T10:23:27.750933+0000","last_fullsized":"2024-03-21T10:23:27.750933+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:21:56.788908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:21:56.788908+0000","last_clean_scrub_stamp":"2024-03-21T10:21:56.788908+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T14:41:24.162795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":29,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T10:23:27.641267+0000","last_change":"2024-03-21T10:21:58.850991+0000","last_active":"2024-03-21T10:23:27.641267+0000","last_peered":"2024-03-21T10:23:27.641267+0000","last_clean":"2024-03-21T10:23:27.641267+0000","last_became_active":"2024-03-21T10:21:58.850581+0000","last_became_peered":"2024-03-21T10:21:58.850581+0000","last_unstale":"2024-03-21T10:23:27.641267+0000","last_undegraded":"2024-03-21T10:23:27.641267+0000","last_fullsized":"2024-03-21T10:23:27.641267+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_clean_scrub_stamp":"2024-03-21T10:21:57.813712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T17:07:11.350806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":22,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T10:23:27.638473+0000","last_change":"2024-03-21T10:22:03.315333+0000","last_active":"2024-03-21T10:23:27.638473+0000","last_peered":"2024-03-21T10:23:27.638473+0000","last_clean":"2024-03-21T10:23:27.638473+0000","last_became_active":"2024-03-21T10:22:03.313553+0000","last_became_peered":"2024-03-21T10:22:03.313553+0000","last_unstale":"2024-03-21T10:23:27.638473+0000","last_undegraded":"2024-03-21T10:23:27.638473+0000","last_fullsized":"2024-03-21T10:23:27.638473+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:22:02.269614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:22:02.269614+0000","last_clean_scrub_stamp":"2024-03-21T10:22:02.269614+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T15:25:24.290789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"93'4","reported_seq":205,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T10:23:52.810881+0000","last_change":"2024-03-21T10:21:57.864930+0000","last_active":"2024-03-21T10:23:52.810881+0000","last_peered":"2024-03-21T10:23:52.810881+0000","last_clean":"2024-03-21T10:23:52.810881+0000","last_became_active":"2024-03-21T10:21:57.863769+0000","last_became_peered":"2024-03-21T10:21:57.863769+0000","last_unstale":"2024-03-21T10:23:52.810881+0000","last_undegraded":"2024-03-21T10:23:52.810881+0000","last_fullsized":"2024-03-21T10:23:52.810881+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:21:56.788908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:21:56.788908+0000","last_clean_scrub_stamp":"2024-03-21T10:21:56.788908+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T21:35:39.144065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":170,"num_read_kb":170,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":29,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T10:23:27.833466+0000","last_change":"2024-03-21T10:21:58.872084+0000","last_active":"2024-03-21T10:23:27.833466+0000","last_peered":"2024-03-21T10:23:27.833466+0000","last_clean":"2024-03-21T10:23:27.833466+0000","last_became_active":"2024-03-21T10:21:58.871725+0000","last_became_peered":"2024-03-21T10:21:58.871725+0000","last_unstale":"2024-03-21T10:23:27.833466+0000","last_undegraded":"2024-03-21T10:23:27.833466+0000","last_fullsized":"2024-03-21T10:23:27.833466+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_clean_scrub_stamp":"2024-03-21T10:21:57.813712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T16:19:37.240053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":22,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T10:23:27.640980+0000","last_change":"2024-03-21T10:22:03.317808+0000","last_active":"2024-03-21T10:23:27.640980+0000","last_peered":"2024-03-21T10:23:27.640980+0000","last_clean":"2024-03-21T10:23:27.640980+0000","last_became_active":"2024-03-21T10:22:03.313751+0000","last_became_peered":"2024-03-21T10:22:03.313751+0000","last_unstale":"2024-03-21T10:23:27.640980+0000","last_undegraded":"2024-03-21T10:23:27.640980+0000","last_fullsized":"2024-03-21T10:23:27.640980+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:22:02.269614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:22:02.269614+0000","last_clean_scrub_stamp":"2024-03-21T10:22:02.269614+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T20:05:08.105795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"93'4","reported_seq":35,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T10:23:27.646569+0000","last_change":"2024-03-21T10:21:57.865462+0000","last_active":"2024-03-21T10:23:27.646569+0000","last_peered":"2024-03-21T10:23:27.646569+0000","last_clean":"2024-03-21T10:23:27.646569+0000","last_became_active":"2024-03-21T10:21:57.862922+0000","last_became_peered":"2024-03-21T10:21:57.862922+0000","last_unstale":"2024-03-21T10:23:27.646569+0000","last_undegraded":"2024-03-21T10:23:27.646569+0000","last_fullsized":"2024-03-21T10:23:27.646569+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:21:56.788908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:21:56.788908+0000","last_clean_scrub_stamp":"2024-03-21T10:21:56.788908+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T15:31:17.167806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":29,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T10:23:27.835183+0000","last_change":"2024-03-21T10:21:58.865459+0000","last_active":"2024-03-21T10:23:27.835183+0000","last_peered":"2024-03-21T10:23:27.835183+0000","last_clean":"2024-03-21T10:23:27.835183+0000","last_became_active":"2024-03-21T10:21:58.860909+0000","last_became_peered":"2024-03-21T10:21:58.860909+0000","last_unstale":"2024-03-21T10:23:27.835183+0000","last_undegraded":"2024-03-21T10:23:27.835183+0000","last_fullsized":"2024-03-21T10:23:27.835183+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_clean_scrub_stamp":"2024-03-21T10:21:57.813712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T15:53:35.110404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":22,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T10:23:27.640600+0000","last_change":"2024-03-21T10:22:03.318651+0000","last_active":"2024-03-21T10:23:27.640600+0000","last_peered":"2024-03-21T10:23:27.640600+0000","last_clean":"2024-03-21T10:23:27.640600+0000","last_became_active":"2024-03-21T10:22:03.318108+0000","last_became_peered":"2024-03-21T10:22:03.318108+0000","last_unstale":"2024-03-21T10:23:27.640600+0000","last_undegraded":"2024-03-21T10:23:27.640600+0000","last_fullsized":"2024-03-21T10:23:27.640600+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:22:02.269614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:22:02.269614+0000","last_clean_scrub_stamp":"2024-03-21T10:22:02.269614+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T19:00:04.216749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"93'5","reported_seq":37,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T10:23:27.751637+0000","last_change":"2024-03-21T10:21:57.865516+0000","last_active":"2024-03-21T10:23:27.751637+0000","last_peered":"2024-03-21T10:23:27.751637+0000","last_clean":"2024-03-21T10:23:27.751637+0000","last_became_active":"2024-03-21T10:21:57.865012+0000","last_became_peered":"2024-03-21T10:21:57.865012+0000","last_unstale":"2024-03-21T10:23:27.751637+0000","last_undegraded":"2024-03-21T10:23:27.751637+0000","last_fullsized":"2024-03-21T10:23:27.751637+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:21:56.788908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:21:56.788908+0000","last_clean_scrub_stamp":"2024-03-21T10:21:56.788908+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T21:38:02.813453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":29,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T10:23:27.833293+0000","last_change":"2024-03-21T10:21:58.870876+0000","last_active":"2024-03-21T10:23:27.833293+0000","last_peered":"2024-03-21T10:23:27.833293+0000","last_clean":"2024-03-21T10:23:27.833293+0000","last_became_active":"2024-03-21T10:21:58.868240+0000","last_became_peered":"2024-03-21T10:21:58.868240+0000","last_unstale":"2024-03-21T10:23:27.833293+0000","last_undegraded":"2024-03-21T10:23:27.833293+0000","last_fullsized":"2024-03-21T10:23:27.833293+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_clean_scrub_stamp":"2024-03-21T10:21:57.813712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T15:16:59.496020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":22,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T10:23:27.641156+0000","last_change":"2024-03-21T10:22:03.314854+0000","last_active":"2024-03-21T10:23:27.641156+0000","last_peered":"2024-03-21T10:23:27.641156+0000","last_clean":"2024-03-21T10:23:27.641156+0000","last_became_active":"2024-03-21T10:22:03.314422+0000","last_became_peered":"2024-03-21T10:22:03.314422+0000","last_unstale":"2024-03-21T10:23:27.641156+0000","last_undegraded":"2024-03-21T10:23:27.641156+0000","last_fullsized":"2024-03-21T10:23:27.641156+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:22:02.269614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:22:02.269614+0000","last_clean_scrub_stamp":"2024-03-21T10:22:02.269614+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T12:34:31.519118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.14","version":"93'32","reported_seq":257,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T10:23:52.811042+0000","last_change":"2024-03-21T10:22:27.243817+0000","last_active":"2024-03-21T10:23:52.811042+0000","last_peered":"2024-03-21T10:23:52.811042+0000","last_clean":"2024-03-21T10:23:52.811042+0000","last_became_active":"2024-03-21T10:22:27.218873+0000","last_became_peered":"2024-03-21T10:22:27.218873+0000","last_unstale":"2024-03-21T10:23:52.811042+0000","last_undegraded":"2024-03-21T10:23:52.811042+0000","last_fullsized":"2024-03-21T10:23:52.811042+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:21:56.788908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:21:56.788908+0000","last_clean_scrub_stamp":"2024-03-21T10:21:56.788908+0000","objects_scrubbed":0,"log_size":32,"log_dups_size":0,"ondisk_log_size":32,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T12:27:43.574451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":59792,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":176,"num_read_kb":146,"num_write":34,"num_write_kb":72,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":3842,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,4],"acting":[10,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":29,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T10:23:27.645957+0000","last_change":"2024-03-21T10:21:58.860386+0000","last_active":"2024-03-21T10:23:27.645957+0000","last_peered":"2024-03-21T10:23:27.645957+0000","last_clean":"2024-03-21T10:23:27.645957+0000","last_became_active":"2024-03-21T10:21:58.860013+0000","last_became_peered":"2024-03-21T10:21:58.860013+0000","last_unstale":"2024-03-21T10:23:27.645957+0000","last_undegraded":"2024-03-21T10:23:27.645957+0000","last_fullsized":"2024-03-21T10:23:27.645957+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_clean_scrub_stamp":"2024-03-21T10:21:57.813712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T18:02:33.920374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":22,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T10:23:27.752375+0000","last_change":"2024-03-21T10:22:03.314083+0000","last_active":"2024-03-21T10:23:27.752375+0000","last_peered":"2024-03-21T10:23:27.752375+0000","last_clean":"2024-03-21T10:23:27.752375+0000","last_became_active":"2024-03-21T10:22:03.313652+0000","last_became_peered":"2024-03-21T10:22:03.313652+0000","last_unstale":"2024-03-21T10:23:27.752375+0000","last_undegraded":"2024-03-21T10:23:27.752375+0000","last_fullsized":"2024-03-21T10:23:27.752375+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:22:02.269614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:22:02.269614+0000","last_clean_scrub_stamp":"2024-03-21T10:22:02.269614+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T13:23:51.256535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"93'9","reported_seq":42,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T10:23:27.647998+0000","last_change":"2024-03-21T10:21:57.866716+0000","last_active":"2024-03-21T10:23:27.647998+0000","last_peered":"2024-03-21T10:23:27.647998+0000","last_clean":"2024-03-21T10:23:27.647998+0000","last_became_active":"2024-03-21T10:21:57.860900+0000","last_became_peered":"2024-03-21T10:21:57.860900+0000","last_unstale":"2024-03-21T10:23:27.647998+0000","last_undegraded":"2024-03-21T10:23:27.647998+0000","last_fullsized":"2024-03-21T10:23:27.647998+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:21:56.788908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:21:56.788908+0000","last_clean_scrub_stamp":"2024-03-21T10:21:56.788908+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T13:13:04.937120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":29,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T10:23:27.742321+0000","last_change":"2024-03-21T10:21:58.858815+0000","last_active":"2024-03-21T10:23:27.742321+0000","last_peered":"2024-03-21T10:23:27.742321+0000","last_clean":"2024-03-21T10:23:27.742321+0000","last_became_active":"2024-03-21T10:21:58.856483+0000","last_became_peered":"2024-03-21T10:21:58.856483+0000","last_unstale":"2024-03-21T10:23:27.742321+0000","last_undegraded":"2024-03-21T10:23:27.742321+0000","last_fullsized":"2024-03-21T10:23:27.742321+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_clean_scrub_stamp":"2024-03-21T10:21:57.813712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T15:42:38.208093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":22,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T10:23:27.836414+0000","last_change":"2024-03-21T10:22:03.319719+0000","last_active":"2024-03-21T10:23:27.836414+0000","last_peered":"2024-03-21T10:23:27.836414+0000","last_clean":"2024-03-21T10:23:27.836414+0000","last_became_active":"2024-03-21T10:22:03.319234+0000","last_became_peered":"2024-03-21T10:22:03.319234+0000","last_unstale":"2024-03-21T10:23:27.836414+0000","last_undegraded":"2024-03-21T10:23:27.836414+0000","last_fullsized":"2024-03-21T10:23:27.836414+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:22:02.269614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:22:02.269614+0000","last_clean_scrub_stamp":"2024-03-21T10:22:02.269614+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T14:11:21.609594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"93'5","reported_seq":37,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T10:23:27.751003+0000","last_change":"2024-03-21T10:21:57.862744+0000","last_active":"2024-03-21T10:23:27.751003+0000","last_peered":"2024-03-21T10:23:27.751003+0000","last_clean":"2024-03-21T10:23:27.751003+0000","last_became_active":"2024-03-21T10:21:57.862352+0000","last_became_peered":"2024-03-21T10:21:57.862352+0000","last_unstale":"2024-03-21T10:23:27.751003+0000","last_undegraded":"2024-03-21T10:23:27.751003+0000","last_fullsized":"2024-03-21T10:23:27.751003+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:21:56.788908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:21:56.788908+0000","last_clean_scrub_stamp":"2024-03-21T10:21:56.788908+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T20:19:50.138904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":29,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T10:23:27.645648+0000","last_change":"2024-03-21T10:21:58.863514+0000","last_active":"2024-03-21T10:23:27.645648+0000","last_peered":"2024-03-21T10:23:27.645648+0000","last_clean":"2024-03-21T10:23:27.645648+0000","last_became_active":"2024-03-21T10:21:58.861981+0000","last_became_peered":"2024-03-21T10:21:58.861981+0000","last_unstale":"2024-03-21T10:23:27.645648+0000","last_undegraded":"2024-03-21T10:23:27.645648+0000","last_fullsized":"2024-03-21T10:23:27.645648+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_clean_scrub_stamp":"2024-03-21T10:21:57.813712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T15:00:33.813456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":22,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T10:23:27.642269+0000","last_change":"2024-03-21T10:22:03.309844+0000","last_active":"2024-03-21T10:23:27.642269+0000","last_peered":"2024-03-21T10:23:27.642269+0000","last_clean":"2024-03-21T10:23:27.642269+0000","last_became_active":"2024-03-21T10:22:03.309448+0000","last_became_peered":"2024-03-21T10:22:03.309448+0000","last_unstale":"2024-03-21T10:23:27.642269+0000","last_undegraded":"2024-03-21T10:23:27.642269+0000","last_fullsized":"2024-03-21T10:23:27.642269+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:22:02.269614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:22:02.269614+0000","last_clean_scrub_stamp":"2024-03-21T10:22:02.269614+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T18:38:51.167166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.11","version":"93'6","reported_seq":131,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T10:23:52.807571+0000","last_change":"2024-03-21T10:22:27.223337+0000","last_active":"2024-03-21T10:23:52.807571+0000","last_peered":"2024-03-21T10:23:52.807571+0000","last_clean":"2024-03-21T10:23:52.807571+0000","last_became_active":"2024-03-21T10:22:27.222999+0000","last_became_peered":"2024-03-21T10:22:27.222999+0000","last_unstale":"2024-03-21T10:23:52.807571+0000","last_undegraded":"2024-03-21T10:23:52.807571+0000","last_fullsized":"2024-03-21T10:23:52.807571+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:21:56.788908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:21:56.788908+0000","last_clean_scrub_stamp":"2024-03-21T10:21:56.788908+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T20:40:45.780696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":979,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":89,"num_read_kb":3,"num_write":6,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,10],"acting":[4,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":29,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T10:23:27.643710+0000","last_change":"2024-03-21T10:21:58.862225+0000","last_active":"2024-03-21T10:23:27.643710+0000","last_peered":"2024-03-21T10:23:27.643710+0000","last_clean":"2024-03-21T10:23:27.643710+0000","last_became_active":"2024-03-21T10:21:58.861816+0000","last_became_peered":"2024-03-21T10:21:58.861816+0000","last_unstale":"2024-03-21T10:23:27.643710+0000","last_undegraded":"2024-03-21T10:23:27.643710+0000","last_fullsized":"2024-03-21T10:23:27.643710+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_clean_scrub_stamp":"2024-03-21T10:21:57.813712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T18:36:13.420867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":22,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T10:23:27.742892+0000","last_change":"2024-03-21T10:22:03.315306+0000","last_active":"2024-03-21T10:23:27.742892+0000","last_peered":"2024-03-21T10:23:27.742892+0000","last_clean":"2024-03-21T10:23:27.742892+0000","last_became_active":"2024-03-21T10:22:03.314839+0000","last_became_peered":"2024-03-21T10:22:03.314839+0000","last_unstale":"2024-03-21T10:23:27.742892+0000","last_undegraded":"2024-03-21T10:23:27.742892+0000","last_fullsized":"2024-03-21T10:23:27.742892+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:22:02.269614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:22:02.269614+0000","last_clean_scrub_stamp":"2024-03-21T10:22:02.269614+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T15:47:45.690434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"93'17","reported_seq":49,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T10:23:27.644391+0000","last_change":"2024-03-21T10:21:57.870998+0000","last_active":"2024-03-21T10:23:27.644391+0000","last_peered":"2024-03-21T10:23:27.644391+0000","last_clean":"2024-03-21T10:23:27.644391+0000","last_became_active":"2024-03-21T10:21:57.870605+0000","last_became_peered":"2024-03-21T10:21:57.870605+0000","last_unstale":"2024-03-21T10:23:27.644391+0000","last_undegraded":"2024-03-21T10:23:27.644391+0000","last_fullsized":"2024-03-21T10:23:27.644391+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:21:56.788908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:21:56.788908+0000","last_clean_scrub_stamp":"2024-03-21T10:21:56.788908+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T13:33:54.775122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":29,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T10:23:27.750626+0000","last_change":"2024-03-21T10:21:58.858675+0000","last_active":"2024-03-21T10:23:27.750626+0000","last_peered":"2024-03-21T10:23:27.750626+0000","last_clean":"2024-03-21T10:23:27.750626+0000","last_became_active":"2024-03-21T10:21:58.856907+0000","last_became_peered":"2024-03-21T10:21:58.856907+0000","last_unstale":"2024-03-21T10:23:27.750626+0000","last_undegraded":"2024-03-21T10:23:27.750626+0000","last_fullsized":"2024-03-21T10:23:27.750626+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_clean_scrub_stamp":"2024-03-21T10:21:57.813712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T18:17:31.208728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":22,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T10:23:27.750514+0000","last_change":"2024-03-21T10:22:03.318257+0000","last_active":"2024-03-21T10:23:27.750514+0000","last_peered":"2024-03-21T10:23:27.750514+0000","last_clean":"2024-03-21T10:23:27.750514+0000","last_became_active":"2024-03-21T10:22:03.317622+0000","last_became_peered":"2024-03-21T10:22:03.317622+0000","last_unstale":"2024-03-21T10:23:27.750514+0000","last_undegraded":"2024-03-21T10:23:27.750514+0000","last_fullsized":"2024-03-21T10:23:27.750514+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:22:02.269614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:22:02.269614+0000","last_clean_scrub_stamp":"2024-03-21T10:22:02.269614+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T17:15:50.272193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"93'2","reported_seq":34,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T10:23:27.647207+0000","last_change":"2024-03-21T10:21:57.868014+0000","last_active":"2024-03-21T10:23:27.647207+0000","last_peered":"2024-03-21T10:23:27.647207+0000","last_clean":"2024-03-21T10:23:27.647207+0000","last_became_active":"2024-03-21T10:21:57.867189+0000","last_became_peered":"2024-03-21T10:21:57.867189+0000","last_unstale":"2024-03-21T10:23:27.647207+0000","last_undegraded":"2024-03-21T10:23:27.647207+0000","last_fullsized":"2024-03-21T10:23:27.647207+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:21:56.788908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:21:56.788908+0000","last_clean_scrub_stamp":"2024-03-21T10:21:56.788908+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T22:02:48.704932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":29,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T10:23:27.836072+0000","last_change":"2024-03-21T10:21:58.862205+0000","last_active":"2024-03-21T10:23:27.836072+0000","last_peered":"2024-03-21T10:23:27.836072+0000","last_clean":"2024-03-21T10:23:27.836072+0000","last_became_active":"2024-03-21T10:21:58.861186+0000","last_became_peered":"2024-03-21T10:21:58.861186+0000","last_unstale":"2024-03-21T10:23:27.836072+0000","last_undegraded":"2024-03-21T10:23:27.836072+0000","last_fullsized":"2024-03-21T10:23:27.836072+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_clean_scrub_stamp":"2024-03-21T10:21:57.813712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T19:44:49.719845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":22,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T10:23:27.643412+0000","last_change":"2024-03-21T10:22:03.636064+0000","last_active":"2024-03-21T10:23:27.643412+0000","last_peered":"2024-03-21T10:23:27.643412+0000","last_clean":"2024-03-21T10:23:27.643412+0000","last_became_active":"2024-03-21T10:22:03.633596+0000","last_became_peered":"2024-03-21T10:22:03.633596+0000","last_unstale":"2024-03-21T10:23:27.643412+0000","last_undegraded":"2024-03-21T10:23:27.643412+0000","last_fullsized":"2024-03-21T10:23:27.643412+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:22:02.269614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:22:02.269614+0000","last_clean_scrub_stamp":"2024-03-21T10:22:02.269614+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T19:32:31.844066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"93'1","reported_seq":32,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T10:23:27.642130+0000","last_change":"2024-03-21T10:21:57.851955+0000","last_active":"2024-03-21T10:23:27.642130+0000","last_peered":"2024-03-21T10:23:27.642130+0000","last_clean":"2024-03-21T10:23:27.642130+0000","last_became_active":"2024-03-21T10:21:57.851571+0000","last_became_peered":"2024-03-21T10:21:57.851571+0000","last_unstale":"2024-03-21T10:23:27.642130+0000","last_undegraded":"2024-03-21T10:23:27.642130+0000","last_fullsized":"2024-03-21T10:23:27.642130+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:21:56.788908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:21:56.788908+0000","last_clean_scrub_stamp":"2024-03-21T10:21:56.788908+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T17:23:40.441895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":29,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T10:23:27.745199+0000","last_change":"2024-03-21T10:21:58.856393+0000","last_active":"2024-03-21T10:23:27.745199+0000","last_peered":"2024-03-21T10:23:27.745199+0000","last_clean":"2024-03-21T10:23:27.745199+0000","last_became_active":"2024-03-21T10:21:58.855139+0000","last_became_peered":"2024-03-21T10:21:58.855139+0000","last_unstale":"2024-03-21T10:23:27.745199+0000","last_undegraded":"2024-03-21T10:23:27.745199+0000","last_fullsized":"2024-03-21T10:23:27.745199+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_clean_scrub_stamp":"2024-03-21T10:21:57.813712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T14:24:55.663576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":22,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T10:23:27.742362+0000","last_change":"2024-03-21T10:22:03.315586+0000","last_active":"2024-03-21T10:23:27.742362+0000","last_peered":"2024-03-21T10:23:27.742362+0000","last_clean":"2024-03-21T10:23:27.742362+0000","last_became_active":"2024-03-21T10:22:03.315198+0000","last_became_peered":"2024-03-21T10:22:03.315198+0000","last_unstale":"2024-03-21T10:23:27.742362+0000","last_undegraded":"2024-03-21T10:23:27.742362+0000","last_fullsized":"2024-03-21T10:23:27.742362+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:22:02.269614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:22:02.269614+0000","last_clean_scrub_stamp":"2024-03-21T10:22:02.269614+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T14:25:13.534311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"93'5","reported_seq":121,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T10:23:51.785755+0000","last_change":"2024-03-21T10:21:57.862094+0000","last_active":"2024-03-21T10:23:51.785755+0000","last_peered":"2024-03-21T10:23:51.785755+0000","last_clean":"2024-03-21T10:23:51.785755+0000","last_became_active":"2024-03-21T10:21:57.860307+0000","last_became_peered":"2024-03-21T10:21:57.860307+0000","last_unstale":"2024-03-21T10:23:51.785755+0000","last_undegraded":"2024-03-21T10:23:51.785755+0000","last_fullsized":"2024-03-21T10:23:51.785755+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:21:56.788908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:21:56.788908+0000","last_clean_scrub_stamp":"2024-03-21T10:21:56.788908+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T12:58:50.701218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":85,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":29,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T10:23:27.744712+0000","last_change":"2024-03-21T10:21:58.870880+0000","last_active":"2024-03-21T10:23:27.744712+0000","last_peered":"2024-03-21T10:23:27.744712+0000","last_clean":"2024-03-21T10:23:27.744712+0000","last_became_active":"2024-03-21T10:21:58.869882+0000","last_became_peered":"2024-03-21T10:21:58.869882+0000","last_unstale":"2024-03-21T10:23:27.744712+0000","last_undegraded":"2024-03-21T10:23:27.744712+0000","last_fullsized":"2024-03-21T10:23:27.744712+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_clean_scrub_stamp":"2024-03-21T10:21:57.813712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T18:01:42.613732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":22,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T10:23:27.742825+0000","last_change":"2024-03-21T10:22:03.634425+0000","last_active":"2024-03-21T10:23:27.742825+0000","last_peered":"2024-03-21T10:23:27.742825+0000","last_clean":"2024-03-21T10:23:27.742825+0000","last_became_active":"2024-03-21T10:22:03.632464+0000","last_became_peered":"2024-03-21T10:22:03.632464+0000","last_unstale":"2024-03-21T10:23:27.742825+0000","last_undegraded":"2024-03-21T10:23:27.742825+0000","last_fullsized":"2024-03-21T10:23:27.742825+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:22:02.269614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:22:02.269614+0000","last_clean_scrub_stamp":"2024-03-21T10:22:02.269614+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T16:03:28.270757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":11,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T10:23:27.649854+0000","last_change":"2024-03-21T10:23:27.649397+0000","last_active":"2024-03-21T10:23:27.649854+0000","last_peered":"2024-03-21T10:23:27.649854+0000","last_clean":"2024-03-21T10:23:27.649854+0000","last_became_active":"2024-03-21T10:23:27.649148+0000","last_became_peered":"2024-03-21T10:23:27.649148+0000","last_unstale":"2024-03-21T10:23:27.649854+0000","last_undegraded":"2024-03-21T10:23:27.649854+0000","last_fullsized":"2024-03-21T10:23:27.649854+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_clean_scrub_stamp":"2024-03-21T10:21:57.813712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T19:28:03.830551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,8],"acting":[9,2,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.1","version":"93'5","reported_seq":123,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T10:23:52.806393+0000","last_change":"2024-03-21T10:21:57.856111+0000","last_active":"2024-03-21T10:23:52.806393+0000","last_peered":"2024-03-21T10:23:52.806393+0000","last_clean":"2024-03-21T10:23:52.806393+0000","last_became_active":"2024-03-21T10:21:57.855584+0000","last_became_peered":"2024-03-21T10:21:57.855584+0000","last_unstale":"2024-03-21T10:23:52.806393+0000","last_undegraded":"2024-03-21T10:23:52.806393+0000","last_fullsized":"2024-03-21T10:23:52.806393+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:21:56.788908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:21:56.788908+0000","last_clean_scrub_stamp":"2024-03-21T10:21:56.788908+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T17:53:48.208554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":87,"num_read_kb":87,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"93'1","reported_seq":23,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T10:23:27.645122+0000","last_change":"2024-03-21T10:22:03.318831+0000","last_active":"2024-03-21T10:23:27.645122+0000","last_peered":"2024-03-21T10:23:27.645122+0000","last_clean":"2024-03-21T10:23:27.645122+0000","last_became_active":"2024-03-21T10:22:03.318404+0000","last_became_peered":"2024-03-21T10:22:03.318404+0000","last_unstale":"2024-03-21T10:23:27.645122+0000","last_undegraded":"2024-03-21T10:23:27.645122+0000","last_fullsized":"2024-03-21T10:23:27.645122+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:22:02.269614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:22:02.269614+0000","last_clean_scrub_stamp":"2024-03-21T10:22:02.269614+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T12:06:52.043286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":29,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T10:23:27.638312+0000","last_change":"2024-03-21T10:21:58.856469+0000","last_active":"2024-03-21T10:23:27.638312+0000","last_peered":"2024-03-21T10:23:27.638312+0000","last_clean":"2024-03-21T10:23:27.638312+0000","last_became_active":"2024-03-21T10:21:58.856032+0000","last_became_peered":"2024-03-21T10:21:58.856032+0000","last_unstale":"2024-03-21T10:23:27.638312+0000","last_undegraded":"2024-03-21T10:23:27.638312+0000","last_fullsized":"2024-03-21T10:23:27.638312+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_clean_scrub_stamp":"2024-03-21T10:21:57.813712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T21:19:59.302418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"93'6","reported_seq":40,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T10:23:27.640543+0000","last_change":"2024-03-21T10:21:57.854210+0000","last_active":"2024-03-21T10:23:27.640543+0000","last_peered":"2024-03-21T10:23:27.640543+0000","last_clean":"2024-03-21T10:23:27.640543+0000","last_became_active":"2024-03-21T10:21:57.853787+0000","last_became_peered":"2024-03-21T10:21:57.853787+0000","last_unstale":"2024-03-21T10:23:27.640543+0000","last_undegraded":"2024-03-21T10:23:27.640543+0000","last_fullsized":"2024-03-21T10:23:27.640543+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:21:56.788908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:21:56.788908+0000","last_clean_scrub_stamp":"2024-03-21T10:21:56.788908+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T12:10:42.926436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"23'137","reported_seq":366,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T10:23:27.643575+0000","last_change":"2024-03-21T10:20:21.585339+0000","last_active":"2024-03-21T10:23:27.643575+0000","last_peered":"2024-03-21T10:23:27.643575+0000","last_clean":"2024-03-21T10:23:27.643575+0000","last_became_active":"2024-03-21T10:20:21.583999+0000","last_became_peered":"2024-03-21T10:20:21.583999+0000","last_unstale":"2024-03-21T10:23:27.643575+0000","last_undegraded":"2024-03-21T10:23:27.643575+0000","last_fullsized":"2024-03-21T10:23:27.643575+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:15:40.696168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:15:40.696168+0000","last_clean_scrub_stamp":"2024-03-21T10:15:40.696168+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T15:31:54.603722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":22,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T10:23:27.752196+0000","last_change":"2024-03-21T10:22:03.634970+0000","last_active":"2024-03-21T10:23:27.752196+0000","last_peered":"2024-03-21T10:23:27.752196+0000","last_clean":"2024-03-21T10:23:27.752196+0000","last_became_active":"2024-03-21T10:22:03.633603+0000","last_became_peered":"2024-03-21T10:22:03.633603+0000","last_unstale":"2024-03-21T10:23:27.752196+0000","last_undegraded":"2024-03-21T10:23:27.752196+0000","last_fullsized":"2024-03-21T10:23:27.752196+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:22:02.269614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:22:02.269614+0000","last_clean_scrub_stamp":"2024-03-21T10:22:02.269614+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T12:13:02.041661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":29,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T10:23:27.742015+0000","last_change":"2024-03-21T10:21:58.861159+0000","last_active":"2024-03-21T10:23:27.742015+0000","last_peered":"2024-03-21T10:23:27.742015+0000","last_clean":"2024-03-21T10:23:27.742015+0000","last_became_active":"2024-03-21T10:21:58.860690+0000","last_became_peered":"2024-03-21T10:21:58.860690+0000","last_unstale":"2024-03-21T10:23:27.742015+0000","last_undegraded":"2024-03-21T10:23:27.742015+0000","last_fullsized":"2024-03-21T10:23:27.742015+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_clean_scrub_stamp":"2024-03-21T10:21:57.813712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T16:21:53.632997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"93'4","reported_seq":35,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T10:23:27.643797+0000","last_change":"2024-03-21T10:21:57.863581+0000","last_active":"2024-03-21T10:23:27.643797+0000","last_peered":"2024-03-21T10:23:27.643797+0000","last_clean":"2024-03-21T10:23:27.643797+0000","last_became_active":"2024-03-21T10:21:57.863269+0000","last_became_peered":"2024-03-21T10:21:57.863269+0000","last_unstale":"2024-03-21T10:23:27.643797+0000","last_undegraded":"2024-03-21T10:23:27.643797+0000","last_fullsized":"2024-03-21T10:23:27.643797+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:21:56.788908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:21:56.788908+0000","last_clean_scrub_stamp":"2024-03-21T10:21:56.788908+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T14:03:59.930570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":22,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T10:23:27.835628+0000","last_change":"2024-03-21T10:22:03.313515+0000","last_active":"2024-03-21T10:23:27.835628+0000","last_peered":"2024-03-21T10:23:27.835628+0000","last_clean":"2024-03-21T10:23:27.835628+0000","last_became_active":"2024-03-21T10:22:03.311862+0000","last_became_peered":"2024-03-21T10:22:03.311862+0000","last_unstale":"2024-03-21T10:23:27.835628+0000","last_undegraded":"2024-03-21T10:23:27.835628+0000","last_fullsized":"2024-03-21T10:23:27.835628+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:22:02.269614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:22:02.269614+0000","last_clean_scrub_stamp":"2024-03-21T10:22:02.269614+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T16:47:50.761205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"93'4","reported_seq":36,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T10:23:27.646295+0000","last_change":"2024-03-21T10:21:57.870913+0000","last_active":"2024-03-21T10:23:27.646295+0000","last_peered":"2024-03-21T10:23:27.646295+0000","last_clean":"2024-03-21T10:23:27.646295+0000","last_became_active":"2024-03-21T10:21:57.870578+0000","last_became_peered":"2024-03-21T10:21:57.870578+0000","last_unstale":"2024-03-21T10:23:27.646295+0000","last_undegraded":"2024-03-21T10:23:27.646295+0000","last_fullsized":"2024-03-21T10:23:27.646295+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:21:56.788908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:21:56.788908+0000","last_clean_scrub_stamp":"2024-03-21T10:21:56.788908+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T11:46:48.621501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":29,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T10:23:27.742642+0000","last_change":"2024-03-21T10:21:58.852247+0000","last_active":"2024-03-21T10:23:27.742642+0000","last_peered":"2024-03-21T10:23:27.742642+0000","last_clean":"2024-03-21T10:23:27.742642+0000","last_became_active":"2024-03-21T10:21:58.851976+0000","last_became_peered":"2024-03-21T10:21:58.851976+0000","last_unstale":"2024-03-21T10:23:27.742642+0000","last_undegraded":"2024-03-21T10:23:27.742642+0000","last_fullsized":"2024-03-21T10:23:27.742642+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_clean_scrub_stamp":"2024-03-21T10:21:57.813712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T14:27:11.383741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":22,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T10:23:27.638208+0000","last_change":"2024-03-21T10:22:03.637671+0000","last_active":"2024-03-21T10:23:27.638208+0000","last_peered":"2024-03-21T10:23:27.638208+0000","last_clean":"2024-03-21T10:23:27.638208+0000","last_became_active":"2024-03-21T10:22:03.634265+0000","last_became_peered":"2024-03-21T10:22:03.634265+0000","last_unstale":"2024-03-21T10:23:27.638208+0000","last_undegraded":"2024-03-21T10:23:27.638208+0000","last_fullsized":"2024-03-21T10:23:27.638208+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:22:02.269614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:22:02.269614+0000","last_clean_scrub_stamp":"2024-03-21T10:22:02.269614+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T15:38:02.214178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"93'13","reported_seq":44,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T10:23:27.642398+0000","last_change":"2024-03-21T10:21:57.862027+0000","last_active":"2024-03-21T10:23:27.642398+0000","last_peered":"2024-03-21T10:23:27.642398+0000","last_clean":"2024-03-21T10:23:27.642398+0000","last_became_active":"2024-03-21T10:21:57.860097+0000","last_became_peered":"2024-03-21T10:21:57.860097+0000","last_unstale":"2024-03-21T10:23:27.642398+0000","last_undegraded":"2024-03-21T10:23:27.642398+0000","last_fullsized":"2024-03-21T10:23:27.642398+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:21:56.788908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:21:56.788908+0000","last_clean_scrub_stamp":"2024-03-21T10:21:56.788908+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T13:46:19.562452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":22,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":29,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T10:23:27.639272+0000","last_change":"2024-03-21T10:21:58.856082+0000","last_active":"2024-03-21T10:23:27.639272+0000","last_peered":"2024-03-21T10:23:27.639272+0000","last_clean":"2024-03-21T10:23:27.639272+0000","last_became_active":"2024-03-21T10:21:58.854172+0000","last_became_peered":"2024-03-21T10:21:58.854172+0000","last_unstale":"2024-03-21T10:23:27.639272+0000","last_undegraded":"2024-03-21T10:23:27.639272+0000","last_fullsized":"2024-03-21T10:23:27.639272+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_clean_scrub_stamp":"2024-03-21T10:21:57.813712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T14:59:33.995716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":22,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T10:23:27.646875+0000","last_change":"2024-03-21T10:22:03.310205+0000","last_active":"2024-03-21T10:23:27.646875+0000","last_peered":"2024-03-21T10:23:27.646875+0000","last_clean":"2024-03-21T10:23:27.646875+0000","last_became_active":"2024-03-21T10:22:03.309039+0000","last_became_peered":"2024-03-21T10:22:03.309039+0000","last_unstale":"2024-03-21T10:23:27.646875+0000","last_undegraded":"2024-03-21T10:23:27.646875+0000","last_fullsized":"2024-03-21T10:23:27.646875+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:22:02.269614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:22:02.269614+0000","last_clean_scrub_stamp":"2024-03-21T10:22:02.269614+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T11:16:46.075260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"93'7","reported_seq":39,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T10:23:27.641734+0000","last_change":"2024-03-21T10:21:57.857890+0000","last_active":"2024-03-21T10:23:27.641734+0000","last_peered":"2024-03-21T10:23:27.641734+0000","last_clean":"2024-03-21T10:23:27.641734+0000","last_became_active":"2024-03-21T10:21:57.857492+0000","last_became_peered":"2024-03-21T10:21:57.857492+0000","last_unstale":"2024-03-21T10:23:27.641734+0000","last_undegraded":"2024-03-21T10:23:27.641734+0000","last_fullsized":"2024-03-21T10:23:27.641734+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:21:56.788908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:21:56.788908+0000","last_clean_scrub_stamp":"2024-03-21T10:21:56.788908+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T21:18:45.566843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":35,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T10:23:27.835109+0000","last_change":"2024-03-21T10:22:27.219339+0000","last_active":"2024-03-21T10:23:27.835109+0000","last_peered":"2024-03-21T10:23:27.835109+0000","last_clean":"2024-03-21T10:23:27.835109+0000","last_became_active":"2024-03-21T10:22:27.218604+0000","last_became_peered":"2024-03-21T10:22:27.218604+0000","last_unstale":"2024-03-21T10:23:27.835109+0000","last_undegraded":"2024-03-21T10:23:27.835109+0000","last_fullsized":"2024-03-21T10:23:27.835109+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_clean_scrub_stamp":"2024-03-21T10:21:57.813712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T15:58:53.161767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,4],"acting":[0,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":22,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T10:23:27.641245+0000","last_change":"2024-03-21T10:22:03.633789+0000","last_active":"2024-03-21T10:23:27.641245+0000","last_peered":"2024-03-21T10:23:27.641245+0000","last_clean":"2024-03-21T10:23:27.641245+0000","last_became_active":"2024-03-21T10:22:03.632716+0000","last_became_peered":"2024-03-21T10:22:03.632716+0000","last_unstale":"2024-03-21T10:23:27.641245+0000","last_undegraded":"2024-03-21T10:23:27.641245+0000","last_fullsized":"2024-03-21T10:23:27.641245+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:22:02.269614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:22:02.269614+0000","last_clean_scrub_stamp":"2024-03-21T10:22:02.269614+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T14:08:47.301881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"91'2","reported_seq":33,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T10:23:27.833086+0000","last_change":"2024-03-21T10:21:57.862413+0000","last_active":"2024-03-21T10:23:27.833086+0000","last_peered":"2024-03-21T10:23:27.833086+0000","last_clean":"2024-03-21T10:23:27.833086+0000","last_became_active":"2024-03-21T10:21:57.862104+0000","last_became_peered":"2024-03-21T10:21:57.862104+0000","last_unstale":"2024-03-21T10:23:27.833086+0000","last_undegraded":"2024-03-21T10:23:27.833086+0000","last_fullsized":"2024-03-21T10:23:27.833086+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:21:56.788908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:21:56.788908+0000","last_clean_scrub_stamp":"2024-03-21T10:21:56.788908+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T17:49:21.729492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":29,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T10:23:27.643080+0000","last_change":"2024-03-21T10:21:58.864688+0000","last_active":"2024-03-21T10:23:27.643080+0000","last_peered":"2024-03-21T10:23:27.643080+0000","last_clean":"2024-03-21T10:23:27.643080+0000","last_became_active":"2024-03-21T10:21:58.862821+0000","last_became_peered":"2024-03-21T10:21:58.862821+0000","last_unstale":"2024-03-21T10:23:27.643080+0000","last_undegraded":"2024-03-21T10:23:27.643080+0000","last_fullsized":"2024-03-21T10:23:27.643080+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_clean_scrub_stamp":"2024-03-21T10:21:57.813712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T11:17:09.650649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T10:23:27.756711+0000","last_change":"2024-03-21T10:23:27.756711+0000","last_active":"2024-03-21T10:23:27.756711+0000","last_peered":"2024-03-21T10:23:27.756711+0000","last_clean":"2024-03-21T10:23:27.756711+0000","last_became_active":"2024-03-21T10:23:27.754993+0000","last_became_peered":"2024-03-21T10:23:27.754993+0000","last_unstale":"2024-03-21T10:23:27.756711+0000","last_undegraded":"2024-03-21T10:23:27.756711+0000","last_fullsized":"2024-03-21T10:23:27.756711+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:22:02.269614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:22:02.269614+0000","last_clean_scrub_stamp":"2024-03-21T10:22:02.269614+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T11:22:51.851047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,2,5],"acting":[8,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"93'7","reported_seq":38,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T10:23:27.638581+0000","last_change":"2024-03-21T10:21:57.868033+0000","last_active":"2024-03-21T10:23:27.638581+0000","last_peered":"2024-03-21T10:23:27.638581+0000","last_clean":"2024-03-21T10:23:27.638581+0000","last_became_active":"2024-03-21T10:21:57.867279+0000","last_became_peered":"2024-03-21T10:21:57.867279+0000","last_unstale":"2024-03-21T10:23:27.638581+0000","last_undegraded":"2024-03-21T10:23:27.638581+0000","last_fullsized":"2024-03-21T10:23:27.638581+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:21:56.788908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:21:56.788908+0000","last_clean_scrub_stamp":"2024-03-21T10:21:56.788908+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T20:05:37.701443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":29,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T10:23:27.835198+0000","last_change":"2024-03-21T10:21:58.868455+0000","last_active":"2024-03-21T10:23:27.835198+0000","last_peered":"2024-03-21T10:23:27.835198+0000","last_clean":"2024-03-21T10:23:27.835198+0000","last_became_active":"2024-03-21T10:21:58.867786+0000","last_became_peered":"2024-03-21T10:21:58.867786+0000","last_unstale":"2024-03-21T10:23:27.835198+0000","last_undegraded":"2024-03-21T10:23:27.835198+0000","last_fullsized":"2024-03-21T10:23:27.835198+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_clean_scrub_stamp":"2024-03-21T10:21:57.813712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T19:16:12.258470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"93'1","reported_seq":34,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T10:23:27.753222+0000","last_change":"2024-03-21T10:23:27.753222+0000","last_active":"2024-03-21T10:23:27.753222+0000","last_peered":"2024-03-21T10:23:27.753222+0000","last_clean":"2024-03-21T10:23:27.753222+0000","last_became_active":"2024-03-21T10:23:27.746251+0000","last_became_peered":"2024-03-21T10:23:27.746251+0000","last_unstale":"2024-03-21T10:23:27.753222+0000","last_undegraded":"2024-03-21T10:23:27.753222+0000","last_fullsized":"2024-03-21T10:23:27.753222+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:22:02.269614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:22:02.269614+0000","last_clean_scrub_stamp":"2024-03-21T10:22:02.269614+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T13:42:00.362336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":117,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,7,8],"acting":[4,10,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"93'3","reported_seq":34,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T10:23:27.647528+0000","last_change":"2024-03-21T10:21:57.862748+0000","last_active":"2024-03-21T10:23:27.647528+0000","last_peered":"2024-03-21T10:23:27.647528+0000","last_clean":"2024-03-21T10:23:27.647528+0000","last_became_active":"2024-03-21T10:21:57.862350+0000","last_became_peered":"2024-03-21T10:21:57.862350+0000","last_unstale":"2024-03-21T10:23:27.647528+0000","last_undegraded":"2024-03-21T10:23:27.647528+0000","last_fullsized":"2024-03-21T10:23:27.647528+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:21:56.788908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:21:56.788908+0000","last_clean_scrub_stamp":"2024-03-21T10:21:56.788908+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T17:08:47.301758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":29,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T10:23:27.638690+0000","last_change":"2024-03-21T10:21:58.866340+0000","last_active":"2024-03-21T10:23:27.638690+0000","last_peered":"2024-03-21T10:23:27.638690+0000","last_clean":"2024-03-21T10:23:27.638690+0000","last_became_active":"2024-03-21T10:21:58.865965+0000","last_became_peered":"2024-03-21T10:21:58.865965+0000","last_unstale":"2024-03-21T10:23:27.638690+0000","last_undegraded":"2024-03-21T10:23:27.638690+0000","last_fullsized":"2024-03-21T10:23:27.638690+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_clean_scrub_stamp":"2024-03-21T10:21:57.813712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T10:42:23.854176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":22,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T10:23:27.640253+0000","last_change":"2024-03-21T10:22:03.634108+0000","last_active":"2024-03-21T10:23:27.640253+0000","last_peered":"2024-03-21T10:23:27.640253+0000","last_clean":"2024-03-21T10:23:27.640253+0000","last_became_active":"2024-03-21T10:22:03.633344+0000","last_became_peered":"2024-03-21T10:22:03.633344+0000","last_unstale":"2024-03-21T10:23:27.640253+0000","last_undegraded":"2024-03-21T10:23:27.640253+0000","last_fullsized":"2024-03-21T10:23:27.640253+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:22:02.269614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:22:02.269614+0000","last_clean_scrub_stamp":"2024-03-21T10:22:02.269614+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T21:00:59.893994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"91'4","reported_seq":36,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T10:23:27.644057+0000","last_change":"2024-03-21T10:21:57.871520+0000","last_active":"2024-03-21T10:23:27.644057+0000","last_peered":"2024-03-21T10:23:27.644057+0000","last_clean":"2024-03-21T10:23:27.644057+0000","last_became_active":"2024-03-21T10:21:57.871127+0000","last_became_peered":"2024-03-21T10:21:57.871127+0000","last_unstale":"2024-03-21T10:23:27.644057+0000","last_undegraded":"2024-03-21T10:23:27.644057+0000","last_fullsized":"2024-03-21T10:23:27.644057+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:21:56.788908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:21:56.788908+0000","last_clean_scrub_stamp":"2024-03-21T10:21:56.788908+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T20:43:39.537307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":29,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T10:23:27.644307+0000","last_change":"2024-03-21T10:21:58.861786+0000","last_active":"2024-03-21T10:23:27.644307+0000","last_peered":"2024-03-21T10:23:27.644307+0000","last_clean":"2024-03-21T10:23:27.644307+0000","last_became_active":"2024-03-21T10:21:58.861428+0000","last_became_peered":"2024-03-21T10:21:58.861428+0000","last_unstale":"2024-03-21T10:23:27.644307+0000","last_undegraded":"2024-03-21T10:23:27.644307+0000","last_fullsized":"2024-03-21T10:23:27.644307+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_clean_scrub_stamp":"2024-03-21T10:21:57.813712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T15:56:30.256266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":22,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T10:23:27.833676+0000","last_change":"2024-03-21T10:22:03.305085+0000","last_active":"2024-03-21T10:23:27.833676+0000","last_peered":"2024-03-21T10:23:27.833676+0000","last_clean":"2024-03-21T10:23:27.833676+0000","last_became_active":"2024-03-21T10:22:03.304474+0000","last_became_peered":"2024-03-21T10:22:03.304474+0000","last_unstale":"2024-03-21T10:23:27.833676+0000","last_undegraded":"2024-03-21T10:23:27.833676+0000","last_fullsized":"2024-03-21T10:23:27.833676+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:22:02.269614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:22:02.269614+0000","last_clean_scrub_stamp":"2024-03-21T10:22:02.269614+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T18:28:10.480981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"93'5","reported_seq":123,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T10:23:51.809722+0000","last_change":"2024-03-21T10:21:57.860657+0000","last_active":"2024-03-21T10:23:51.809722+0000","last_peered":"2024-03-21T10:23:51.809722+0000","last_clean":"2024-03-21T10:23:51.809722+0000","last_became_active":"2024-03-21T10:21:57.860340+0000","last_became_peered":"2024-03-21T10:21:57.860340+0000","last_unstale":"2024-03-21T10:23:51.809722+0000","last_undegraded":"2024-03-21T10:23:51.809722+0000","last_fullsized":"2024-03-21T10:23:51.809722+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:21:56.788908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:21:56.788908+0000","last_clean_scrub_stamp":"2024-03-21T10:21:56.788908+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T21:58:08.706812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1170,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":87,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":29,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T10:23:27.645454+0000","last_change":"2024-03-21T10:21:58.874254+0000","last_active":"2024-03-21T10:23:27.645454+0000","last_peered":"2024-03-21T10:23:27.645454+0000","last_clean":"2024-03-21T10:23:27.645454+0000","last_became_active":"2024-03-21T10:21:58.869795+0000","last_became_peered":"2024-03-21T10:21:58.869795+0000","last_unstale":"2024-03-21T10:23:27.645454+0000","last_undegraded":"2024-03-21T10:23:27.645454+0000","last_fullsized":"2024-03-21T10:23:27.645454+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_clean_scrub_stamp":"2024-03-21T10:21:57.813712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T19:51:28.296199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":22,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T10:23:27.834249+0000","last_change":"2024-03-21T10:22:03.306761+0000","last_active":"2024-03-21T10:23:27.834249+0000","last_peered":"2024-03-21T10:23:27.834249+0000","last_clean":"2024-03-21T10:23:27.834249+0000","last_became_active":"2024-03-21T10:22:03.305106+0000","last_became_peered":"2024-03-21T10:22:03.305106+0000","last_unstale":"2024-03-21T10:23:27.834249+0000","last_undegraded":"2024-03-21T10:23:27.834249+0000","last_fullsized":"2024-03-21T10:23:27.834249+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:22:02.269614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:22:02.269614+0000","last_clean_scrub_stamp":"2024-03-21T10:22:02.269614+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T21:13:00.846169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"93'3","reported_seq":35,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T10:23:27.751970+0000","last_change":"2024-03-21T10:21:57.868012+0000","last_active":"2024-03-21T10:23:27.751970+0000","last_peered":"2024-03-21T10:23:27.751970+0000","last_clean":"2024-03-21T10:23:27.751970+0000","last_became_active":"2024-03-21T10:21:57.867682+0000","last_became_peered":"2024-03-21T10:21:57.867682+0000","last_unstale":"2024-03-21T10:23:27.751970+0000","last_undegraded":"2024-03-21T10:23:27.751970+0000","last_fullsized":"2024-03-21T10:23:27.751970+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:21:56.788908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:21:56.788908+0000","last_clean_scrub_stamp":"2024-03-21T10:21:56.788908+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T16:59:56.061167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":29,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T10:23:27.645341+0000","last_change":"2024-03-21T10:21:58.864949+0000","last_active":"2024-03-21T10:23:27.645341+0000","last_peered":"2024-03-21T10:23:27.645341+0000","last_clean":"2024-03-21T10:23:27.645341+0000","last_became_active":"2024-03-21T10:21:58.859796+0000","last_became_peered":"2024-03-21T10:21:58.859796+0000","last_unstale":"2024-03-21T10:23:27.645341+0000","last_undegraded":"2024-03-21T10:23:27.645341+0000","last_fullsized":"2024-03-21T10:23:27.645341+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_clean_scrub_stamp":"2024-03-21T10:21:57.813712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T18:02:51.873103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":22,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T10:23:27.749977+0000","last_change":"2024-03-21T10:22:03.308451+0000","last_active":"2024-03-21T10:23:27.749977+0000","last_peered":"2024-03-21T10:23:27.749977+0000","last_clean":"2024-03-21T10:23:27.749977+0000","last_became_active":"2024-03-21T10:22:03.307986+0000","last_became_peered":"2024-03-21T10:22:03.307986+0000","last_unstale":"2024-03-21T10:23:27.749977+0000","last_undegraded":"2024-03-21T10:23:27.749977+0000","last_fullsized":"2024-03-21T10:23:27.749977+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:22:02.269614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:22:02.269614+0000","last_clean_scrub_stamp":"2024-03-21T10:22:02.269614+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T21:08:42.241657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"93'5","reported_seq":36,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T10:23:27.742859+0000","last_change":"2024-03-21T10:21:57.851717+0000","last_active":"2024-03-21T10:23:27.742859+0000","last_peered":"2024-03-21T10:23:27.742859+0000","last_clean":"2024-03-21T10:23:27.742859+0000","last_became_active":"2024-03-21T10:21:57.849909+0000","last_became_peered":"2024-03-21T10:21:57.849909+0000","last_unstale":"2024-03-21T10:23:27.742859+0000","last_undegraded":"2024-03-21T10:23:27.742859+0000","last_fullsized":"2024-03-21T10:23:27.742859+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:21:56.788908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:21:56.788908+0000","last_clean_scrub_stamp":"2024-03-21T10:21:56.788908+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T11:41:37.611400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":29,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T10:23:27.645246+0000","last_change":"2024-03-21T10:21:58.860892+0000","last_active":"2024-03-21T10:23:27.645246+0000","last_peered":"2024-03-21T10:23:27.645246+0000","last_clean":"2024-03-21T10:23:27.645246+0000","last_became_active":"2024-03-21T10:21:58.860403+0000","last_became_peered":"2024-03-21T10:21:58.860403+0000","last_unstale":"2024-03-21T10:23:27.645246+0000","last_undegraded":"2024-03-21T10:23:27.645246+0000","last_fullsized":"2024-03-21T10:23:27.645246+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_clean_scrub_stamp":"2024-03-21T10:21:57.813712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T15:15:23.303864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":29,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T10:23:27.835488+0000","last_change":"2024-03-21T10:21:58.869275+0000","last_active":"2024-03-21T10:23:27.835488+0000","last_peered":"2024-03-21T10:23:27.835488+0000","last_clean":"2024-03-21T10:23:27.835488+0000","last_became_active":"2024-03-21T10:21:58.868834+0000","last_became_peered":"2024-03-21T10:21:58.868834+0000","last_unstale":"2024-03-21T10:23:27.835488+0000","last_undegraded":"2024-03-21T10:23:27.835488+0000","last_fullsized":"2024-03-21T10:23:27.835488+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_clean_scrub_stamp":"2024-03-21T10:21:57.813712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T14:01:11.074702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":29,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T10:23:27.646021+0000","last_change":"2024-03-21T10:21:58.851658+0000","last_active":"2024-03-21T10:23:27.646021+0000","last_peered":"2024-03-21T10:23:27.646021+0000","last_clean":"2024-03-21T10:23:27.646021+0000","last_became_active":"2024-03-21T10:21:58.851295+0000","last_became_peered":"2024-03-21T10:21:58.851295+0000","last_unstale":"2024-03-21T10:23:27.646021+0000","last_undegraded":"2024-03-21T10:23:27.646021+0000","last_fullsized":"2024-03-21T10:23:27.646021+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_clean_scrub_stamp":"2024-03-21T10:21:57.813712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T13:15:32.107783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":29,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T10:23:27.647250+0000","last_change":"2024-03-21T10:21:58.859275+0000","last_active":"2024-03-21T10:23:27.647250+0000","last_peered":"2024-03-21T10:23:27.647250+0000","last_clean":"2024-03-21T10:23:27.647250+0000","last_became_active":"2024-03-21T10:21:58.857054+0000","last_became_peered":"2024-03-21T10:21:58.857054+0000","last_unstale":"2024-03-21T10:23:27.647250+0000","last_undegraded":"2024-03-21T10:23:27.647250+0000","last_fullsized":"2024-03-21T10:23:27.647250+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_clean_scrub_stamp":"2024-03-21T10:21:57.813712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T20:05:13.819966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":29,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T10:23:27.834579+0000","last_change":"2024-03-21T10:21:58.869025+0000","last_active":"2024-03-21T10:23:27.834579+0000","last_peered":"2024-03-21T10:23:27.834579+0000","last_clean":"2024-03-21T10:23:27.834579+0000","last_became_active":"2024-03-21T10:21:58.868240+0000","last_became_peered":"2024-03-21T10:21:58.868240+0000","last_unstale":"2024-03-21T10:23:27.834579+0000","last_undegraded":"2024-03-21T10:23:27.834579+0000","last_fullsized":"2024-03-21T10:23:27.834579+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_clean_scrub_stamp":"2024-03-21T10:21:57.813712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T21:56:45.642916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":29,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T10:23:27.742692+0000","last_change":"2024-03-21T10:21:58.865024+0000","last_active":"2024-03-21T10:23:27.742692+0000","last_peered":"2024-03-21T10:23:27.742692+0000","last_clean":"2024-03-21T10:23:27.742692+0000","last_became_active":"2024-03-21T10:21:58.864649+0000","last_became_peered":"2024-03-21T10:21:58.864649+0000","last_unstale":"2024-03-21T10:23:27.742692+0000","last_undegraded":"2024-03-21T10:23:27.742692+0000","last_fullsized":"2024-03-21T10:23:27.742692+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_clean_scrub_stamp":"2024-03-21T10:21:57.813712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T22:09:03.561283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":29,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T10:23:27.837451+0000","last_change":"2024-03-21T10:21:58.865206+0000","last_active":"2024-03-21T10:23:27.837451+0000","last_peered":"2024-03-21T10:23:27.837451+0000","last_clean":"2024-03-21T10:23:27.837451+0000","last_became_active":"2024-03-21T10:21:58.864669+0000","last_became_peered":"2024-03-21T10:21:58.864669+0000","last_unstale":"2024-03-21T10:23:27.837451+0000","last_undegraded":"2024-03-21T10:23:27.837451+0000","last_fullsized":"2024-03-21T10:23:27.837451+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_clean_scrub_stamp":"2024-03-21T10:21:57.813712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T13:57:45.668192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":29,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T10:23:27.647110+0000","last_change":"2024-03-21T10:21:58.862664+0000","last_active":"2024-03-21T10:23:27.647110+0000","last_peered":"2024-03-21T10:23:27.647110+0000","last_clean":"2024-03-21T10:23:27.647110+0000","last_became_active":"2024-03-21T10:21:58.862144+0000","last_became_peered":"2024-03-21T10:21:58.862144+0000","last_unstale":"2024-03-21T10:23:27.647110+0000","last_undegraded":"2024-03-21T10:23:27.647110+0000","last_fullsized":"2024-03-21T10:23:27.647110+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_clean_scrub_stamp":"2024-03-21T10:21:57.813712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T11:10:10.225102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":29,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T10:23:27.639451+0000","last_change":"2024-03-21T10:21:58.851000+0000","last_active":"2024-03-21T10:23:27.639451+0000","last_peered":"2024-03-21T10:23:27.639451+0000","last_clean":"2024-03-21T10:23:27.639451+0000","last_became_active":"2024-03-21T10:21:58.850563+0000","last_became_peered":"2024-03-21T10:21:58.850563+0000","last_unstale":"2024-03-21T10:23:27.639451+0000","last_undegraded":"2024-03-21T10:23:27.639451+0000","last_fullsized":"2024-03-21T10:23:27.639451+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_clean_scrub_stamp":"2024-03-21T10:21:57.813712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T19:39:08.983713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":29,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T10:23:27.751071+0000","last_change":"2024-03-21T10:21:58.838426+0000","last_active":"2024-03-21T10:23:27.751071+0000","last_peered":"2024-03-21T10:23:27.751071+0000","last_clean":"2024-03-21T10:23:27.751071+0000","last_became_active":"2024-03-21T10:21:58.837625+0000","last_became_peered":"2024-03-21T10:21:58.837625+0000","last_unstale":"2024-03-21T10:23:27.751071+0000","last_undegraded":"2024-03-21T10:23:27.751071+0000","last_fullsized":"2024-03-21T10:23:27.751071+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_clean_scrub_stamp":"2024-03-21T10:21:57.813712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T16:22:26.523685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":29,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T10:23:27.638672+0000","last_change":"2024-03-21T10:21:58.854114+0000","last_active":"2024-03-21T10:23:27.638672+0000","last_peered":"2024-03-21T10:23:27.638672+0000","last_clean":"2024-03-21T10:23:27.638672+0000","last_became_active":"2024-03-21T10:21:58.853686+0000","last_became_peered":"2024-03-21T10:21:58.853686+0000","last_unstale":"2024-03-21T10:23:27.638672+0000","last_undegraded":"2024-03-21T10:23:27.638672+0000","last_fullsized":"2024-03-21T10:23:27.638672+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_clean_scrub_stamp":"2024-03-21T10:21:57.813712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T11:07:27.346766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":29,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T10:23:27.833445+0000","last_change":"2024-03-21T10:21:58.857778+0000","last_active":"2024-03-21T10:23:27.833445+0000","last_peered":"2024-03-21T10:23:27.833445+0000","last_clean":"2024-03-21T10:23:27.833445+0000","last_became_active":"2024-03-21T10:21:58.857336+0000","last_became_peered":"2024-03-21T10:21:58.857336+0000","last_unstale":"2024-03-21T10:23:27.833445+0000","last_undegraded":"2024-03-21T10:23:27.833445+0000","last_fullsized":"2024-03-21T10:23:27.833445+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_clean_scrub_stamp":"2024-03-21T10:21:57.813712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T15:22:57.760341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":29,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T10:23:27.742336+0000","last_change":"2024-03-21T10:21:58.869774+0000","last_active":"2024-03-21T10:23:27.742336+0000","last_peered":"2024-03-21T10:23:27.742336+0000","last_clean":"2024-03-21T10:23:27.742336+0000","last_became_active":"2024-03-21T10:21:58.868815+0000","last_became_peered":"2024-03-21T10:21:58.868815+0000","last_unstale":"2024-03-21T10:23:27.742336+0000","last_undegraded":"2024-03-21T10:23:27.742336+0000","last_fullsized":"2024-03-21T10:23:27.742336+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_clean_scrub_stamp":"2024-03-21T10:21:57.813712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T16:57:55.082127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":29,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T10:23:27.836099+0000","last_change":"2024-03-21T10:21:58.864746+0000","last_active":"2024-03-21T10:23:27.836099+0000","last_peered":"2024-03-21T10:23:27.836099+0000","last_clean":"2024-03-21T10:23:27.836099+0000","last_became_active":"2024-03-21T10:21:58.864366+0000","last_became_peered":"2024-03-21T10:21:58.864366+0000","last_unstale":"2024-03-21T10:23:27.836099+0000","last_undegraded":"2024-03-21T10:23:27.836099+0000","last_fullsized":"2024-03-21T10:23:27.836099+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_clean_scrub_stamp":"2024-03-21T10:21:57.813712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T13:10:27.303843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":117,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4751360,"data_stored":4722944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":66605,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":885,"num_read_kb":515,"num_write":240,"num_write_kb":136,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":4464,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":565248,"data_stored":199815,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":208,"ondisk_log_size":208,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":82,"seq":352187318311,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36400,"kb_used_data":1060,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708752,"statfs":{"total":95995035648,"available":95957762048,"internally_reserved":0,"allocated":1085440,"data_stored":528110,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":36174276},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":74,"seq":317827579950,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32920,"kb_used_data":1736,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712232,"statfs":{"total":95995035648,"available":95961325568,"internally_reserved":0,"allocated":1777664,"data_stored":1178226,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":69,"seq":296352743477,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28104,"kb_used_data":1080,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717048,"statfs":{"total":95995035648,"available":95966257152,"internally_reserved":0,"allocated":1105920,"data_stored":527061,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":61,"seq":261993005116,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36416,"kb_used_data":1076,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708736,"statfs":{"total":95995035648,"available":95957745664,"internally_reserved":0,"allocated":1101824,"data_stored":527439,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":53,"seq":227633266756,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28676,"kb_used_data":1656,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716476,"statfs":{"total":95995035648,"available":95965671424,"internally_reserved":0,"allocated":1695744,"data_stored":1122957,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":48,"seq":206158430283,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36400,"kb_used_data":1056,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708752,"statfs":{"total":95995035648,"available":95957762048,"internally_reserved":0,"allocated":1081344,"data_stored":522565,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574956,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32304,"kb_used_data":1120,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712848,"statfs":{"total":95995035648,"available":95961956352,"internally_reserved":0,"allocated":1146880,"data_stored":583387,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738483,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28104,"kb_used_data":1080,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717048,"statfs":{"total":95995035648,"available":95966257152,"internally_reserved":0,"allocated":1105920,"data_stored":524021,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411431,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36408,"kb_used_data":1064,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708744,"statfs":{"total":95995035648,"available":95957753856,"internally_reserved":0,"allocated":1089536,"data_stored":523193,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":27,"seq":115964117088,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36412,"kb_used_data":1068,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708740,"statfs":{"total":95995035648,"available":95957749760,"internally_reserved":0,"allocated":1093632,"data_stored":522273,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":35,"seq":150323855449,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28164,"kb_used_data":1140,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716988,"statfs":{"total":95995035648,"available":95966195712,"internally_reserved":0,"allocated":1167360,"data_stored":587444,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":40,"seq":171798691921,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":37012,"kb_used_data":1668,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708140,"statfs":{"total":95995035648,"available":95957135360,"internally_reserved":0,"allocated":1708032,"data_stored":1119127,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2178,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":61544,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1350,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":98304,"data_stored":61505,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":2820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":722,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":2554,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":1500,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":61919,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2171,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-21T10:23:57.116 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2edfee68ad7243bed422a049b6f5f58619300f11 shell --fsid 3056d51a-e76b-11ee-95cd-87774f69a715 -- ceph pg dump --format=json 2024-03-21T10:23:57.637 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/3056d51a-e76b-11ee-95cd-87774f69a715/mon.a/config 2024-03-21T10:23:58.121 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:23:57 smithi047 ceph-mon[31547]: pgmap v430: 129 pgs: 129 active+clean; 642 KiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-21T10:23:58.122 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:23:57 smithi047 ceph-mon[31547]: from='client.15090 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-21T10:23:58.122 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:23:57 smithi047 ceph-mon[31547]: mgrmap e18: x(active, since 11m), standbys: z, y 2024-03-21T10:23:58.229 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:23:57 smithi100 ceph-mon[35193]: pgmap v430: 129 pgs: 129 active+clean; 642 KiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-21T10:23:58.229 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:23:57 smithi100 ceph-mon[35193]: from='client.15090 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-21T10:23:58.229 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:23:57 smithi100 ceph-mon[35193]: mgrmap e18: x(active, since 11m), standbys: z, y 2024-03-21T10:23:58.240 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:23:57 smithi157 ceph-mon[35581]: pgmap v430: 129 pgs: 129 active+clean; 642 KiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-21T10:23:58.240 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:23:57 smithi157 ceph-mon[35581]: from='client.15090 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-21T10:23:58.240 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:23:57 smithi157 ceph-mon[35581]: mgrmap e18: x(active, since 11m), standbys: z, y 2024-03-21T10:23:59.853 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:59.851+0000 7f7c79200700 1 -- 172.21.15.47:0/1783310269 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7c740745c0 msgr2=0x7f7c740749a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:23:59.853 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:59.851+0000 7f7c79200700 1 --2- 172.21.15.47:0/1783310269 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7c740745c0 0x7f7c740749a0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f7c64009990 tx=0x7f7c64009ca0 comp rx=0 tx=0).stop 2024-03-21T10:23:59.854 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:59.852+0000 7f7c79200700 1 -- 172.21.15.47:0/1783310269 shutdown_connections 2024-03-21T10:23:59.854 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:59.852+0000 7f7c79200700 1 --2- 172.21.15.47:0/1783310269 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7c7410e8b0 0x7f7c74110d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:23:59.854 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:59.852+0000 7f7c79200700 1 --2- 172.21.15.47:0/1783310269 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7c740745c0 0x7f7c740749a0 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:23:59.854 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:59.852+0000 7f7c79200700 1 --2- 172.21.15.47:0/1783310269 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7c74074ee0 0x7f7c7410e370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:23:59.854 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:59.852+0000 7f7c79200700 1 -- 172.21.15.47:0/1783310269 >> 172.21.15.47:0/1783310269 conn(0x7f7c740fdb60 msgr2=0x7f7c740fff80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:23:59.854 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:59.852+0000 7f7c79200700 1 -- 172.21.15.47:0/1783310269 shutdown_connections 2024-03-21T10:23:59.854 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:59.852+0000 7f7c79200700 1 -- 172.21.15.47:0/1783310269 wait complete. 2024-03-21T10:23:59.856 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:59.853+0000 7f7c79200700 1 Processor -- start 2024-03-21T10:23:59.856 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:59.853+0000 7f7c79200700 1 -- start start 2024-03-21T10:23:59.856 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:59.853+0000 7f7c79200700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7c740745c0 0x7f7c741a0690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:23:59.856 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:59.853+0000 7f7c79200700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7c74074ee0 0x7f7c741a0bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:23:59.856 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:59.853+0000 7f7c79200700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7c7410e8b0 0x7f7c7419a7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:23:59.856 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:59.853+0000 7f7c79200700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c74114430 con 0x7f7c74074ee0 2024-03-21T10:23:59.856 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:59.853+0000 7f7c79200700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c741142b0 con 0x7f7c740745c0 2024-03-21T10:23:59.856 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:59.853+0000 7f7c79200700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c741145b0 con 0x7f7c7410e8b0 2024-03-21T10:23:59.856 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:59.854+0000 7f7c72a00700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7c74074ee0 0x7f7c741a0bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:23:59.856 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:59.854+0000 7f7c72a00700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7c74074ee0 0x7f7c741a0bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:39586/0 (socket says 172.21.15.47:39586) 2024-03-21T10:23:59.856 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:59.854+0000 7f7c72a00700 1 -- 172.21.15.47:0/246188408 learned_addr learned my addr 172.21.15.47:0/246188408 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:23:59.857 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:59.854+0000 7f7c73400700 1 --2- 172.21.15.47:0/246188408 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7c740745c0 0x7f7c741a0690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:23:59.857 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:59.854+0000 7f7c73e00700 1 --2- 172.21.15.47:0/246188408 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7c7410e8b0 0x7f7c7419a7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:23:59.857 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:59.854+0000 7f7c72a00700 1 -- 172.21.15.47:0/246188408 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7c7410e8b0 msgr2=0x7f7c7419a7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:23:59.857 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:59.854+0000 7f7c72a00700 1 --2- 172.21.15.47:0/246188408 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7c7410e8b0 0x7f7c7419a7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:23:59.857 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:59.854+0000 7f7c72a00700 1 -- 172.21.15.47:0/246188408 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7c740745c0 msgr2=0x7f7c741a0690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:23:59.857 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:59.854+0000 7f7c72a00700 1 --2- 172.21.15.47:0/246188408 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7c740745c0 0x7f7c741a0690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:23:59.857 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:59.854+0000 7f7c72a00700 1 -- 172.21.15.47:0/246188408 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7c7419b090 con 0x7f7c74074ee0 2024-03-21T10:23:59.857 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:59.854+0000 7f7c73e00700 1 --2- 172.21.15.47:0/246188408 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7c7410e8b0 0x7f7c7419a7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T10:23:59.857 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:59.855+0000 7f7c72a00700 1 --2- 172.21.15.47:0/246188408 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7c74074ee0 0x7f7c741a0bd0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f7c6000d950 tx=0x7f7c6000dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:23:59.858 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:59.855+0000 7f7c6be00700 1 -- 172.21.15.47:0/246188408 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c60016070 con 0x7f7c74074ee0 2024-03-21T10:23:59.858 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:59.855+0000 7f7c6be00700 1 -- 172.21.15.47:0/246188408 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f7c60009df0 con 0x7f7c74074ee0 2024-03-21T10:23:59.858 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:59.855+0000 7f7c6be00700 1 -- 172.21.15.47:0/246188408 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c6000f3c0 con 0x7f7c74074ee0 2024-03-21T10:23:59.858 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:59.855+0000 7f7c79200700 1 -- 172.21.15.47:0/246188408 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7c7419b380 con 0x7f7c74074ee0 2024-03-21T10:23:59.858 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:59.855+0000 7f7c79200700 1 -- 172.21.15.47:0/246188408 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7c741a7610 con 0x7f7c74074ee0 2024-03-21T10:23:59.858 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:59.857+0000 7f7c79200700 1 -- 172.21.15.47:0/246188408 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7c74075c70 con 0x7f7c74074ee0 2024-03-21T10:23:59.860 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:59.859+0000 7f7c6be00700 1 -- 172.21.15.47:0/246188408 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 18) v1 ==== 137854+0+0 (secure 0 0 0) 0x7f7c60019070 con 0x7f7c74074ee0 2024-03-21T10:23:59.861 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:59.860+0000 7f7c6be00700 1 --2- 172.21.15.47:0/246188408 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f7c580a2fb0 0x7f7c580a5470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:23:59.862 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:59.860+0000 7f7c6be00700 1 -- 172.21.15.47:0/246188408 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9216+0+0 (secure 0 0 0) 0x7f7c600cf3f0 con 0x7f7c74074ee0 2024-03-21T10:23:59.862 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:59.861+0000 7f7c73400700 1 --2- 172.21.15.47:0/246188408 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f7c580a2fb0 0x7f7c580a5470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:23:59.862 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:59.862+0000 7f7c73400700 1 --2- 172.21.15.47:0/246188408 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f7c580a2fb0 0x7f7c580a5470 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f7c640098e0 tx=0x7f7c64005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:23:59.864 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:23:59.863+0000 7f7c6be00700 1 -- 172.21.15.47:0/246188408 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f7c6009be10 con 0x7f7c74074ee0 2024-03-21T10:23:59.966 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:23:59 smithi047 ceph-mon[31547]: pgmap v431: 129 pgs: 129 active+clean; 642 KiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 0 B/s wr, 9 op/s 2024-03-21T10:24:00.140 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:00.139+0000 7f7c79200700 1 -- 172.21.15.47:0/246188408 --> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f7c7419bc90 con 0x7f7c580a2fb0 2024-03-21T10:24:00.162 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:00.160+0000 7f7c6be00700 1 -- 172.21.15.47:0/246188408 <== mgr.14152 v2:172.21.15.47:6800/2823885994 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+340714 (secure 0 0 0) 0x7f7c7419bc90 con 0x7f7c580a2fb0 2024-03-21T10:24:00.163 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-21T10:24:00.174 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:00.167+0000 7f7c79200700 1 -- 172.21.15.47:0/246188408 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f7c580a2fb0 msgr2=0x7f7c580a5470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:24:00.174 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:00.167+0000 7f7c79200700 1 --2- 172.21.15.47:0/246188408 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f7c580a2fb0 0x7f7c580a5470 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f7c640098e0 tx=0x7f7c64005960 comp rx=0 tx=0).stop 2024-03-21T10:24:00.174 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:00.167+0000 7f7c79200700 1 -- 172.21.15.47:0/246188408 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7c74074ee0 msgr2=0x7f7c741a0bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:24:00.174 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:00.167+0000 7f7c79200700 1 --2- 172.21.15.47:0/246188408 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7c74074ee0 0x7f7c741a0bd0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f7c6000d950 tx=0x7f7c6000dc60 comp rx=0 tx=0).stop 2024-03-21T10:24:00.175 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:00.167+0000 7f7c79200700 1 -- 172.21.15.47:0/246188408 shutdown_connections 2024-03-21T10:24:00.175 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:00.167+0000 7f7c79200700 1 --2- 172.21.15.47:0/246188408 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f7c580a2fb0 0x7f7c580a5470 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:24:00.175 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:00.167+0000 7f7c79200700 1 --2- 172.21.15.47:0/246188408 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7c7410e8b0 0x7f7c7419a7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:24:00.175 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:00.167+0000 7f7c79200700 1 --2- 172.21.15.47:0/246188408 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7c740745c0 0x7f7c741a0690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:24:00.175 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:00.167+0000 7f7c79200700 1 --2- 172.21.15.47:0/246188408 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7c74074ee0 0x7f7c741a0bd0 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:24:00.175 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:00.167+0000 7f7c79200700 1 -- 172.21.15.47:0/246188408 >> 172.21.15.47:0/246188408 conn(0x7f7c740fdb60 msgr2=0x7f7c740fe870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:24:00.175 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:00.168+0000 7f7c79200700 1 -- 172.21.15.47:0/246188408 shutdown_connections 2024-03-21T10:24:00.175 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:00.168+0000 7f7c79200700 1 -- 172.21.15.47:0/246188408 wait complete. 2024-03-21T10:24:00.175 INFO:teuthology.orchestra.run.smithi047.stderr:dumped all 2024-03-21T10:24:00.229 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:23:59 smithi100 ceph-mon[35193]: pgmap v431: 129 pgs: 129 active+clean; 642 KiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 0 B/s wr, 9 op/s 2024-03-21T10:24:00.240 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:23:59 smithi157 ceph-mon[35581]: pgmap v431: 129 pgs: 129 active+clean; 642 KiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 0 B/s wr, 9 op/s 2024-03-21T10:24:01.093 INFO:teuthology.orchestra.run.smithi047.stdout:{"pg_ready":true,"pg_map":{"version":432,"stamp":"2024-03-21T10:24:00.068231+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":657597,"num_objects":102,"num_object_clones":0,"num_object_copies":308,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":1073,"num_read_kb":661,"num_write":430,"num_write_kb":3081,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13,"num_bytes_recovered":1185317,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":853,"ondisk_log_size":853,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":397236,"kb_used_data":14804,"kb_used_omap":18,"kb_used_meta":382317,"kb_avail":1124544588,"statfs":{"total":1151940427776,"available":1151533658112,"internally_reserved":0,"allocated":15159296,"data_stored":8266897,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19142,"internal_metadata":391492922},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":87,"apply_latency_ms":87,"commit_latency_ns":87000000,"apply_latency_ns":87000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":390,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":137,"num_read_kb":72,"num_write":7,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004904"},"pg_stats":[{"pgid":"3.3f","version":"95'8","reported_seq":37,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T10:23:56.048900+0000","last_change":"2024-03-21T10:21:58.872480+0000","last_active":"2024-03-21T10:23:56.048900+0000","last_peered":"2024-03-21T10:23:56.048900+0000","last_clean":"2024-03-21T10:23:56.048900+0000","last_became_active":"2024-03-21T10:21:58.872143+0000","last_became_peered":"2024-03-21T10:21:58.872143+0000","last_unstale":"2024-03-21T10:23:56.048900+0000","last_undegraded":"2024-03-21T10:23:56.048900+0000","last_fullsized":"2024-03-21T10:23:56.048900+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_clean_scrub_stamp":"2024-03-21T10:21:57.813712+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T15:20:00.409201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"95'4","reported_seq":33,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T10:23:56.020060+0000","last_change":"2024-03-21T10:21:58.871130+0000","last_active":"2024-03-21T10:23:56.020060+0000","last_peered":"2024-03-21T10:23:56.020060+0000","last_clean":"2024-03-21T10:23:56.020060+0000","last_became_active":"2024-03-21T10:21:58.870750+0000","last_became_peered":"2024-03-21T10:21:58.870750+0000","last_unstale":"2024-03-21T10:23:56.020060+0000","last_undegraded":"2024-03-21T10:23:56.020060+0000","last_fullsized":"2024-03-21T10:23:56.020060+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_clean_scrub_stamp":"2024-03-21T10:21:57.813712+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T17:38:04.830212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"95'7","reported_seq":36,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T10:23:56.048268+0000","last_change":"2024-03-21T10:21:58.868026+0000","last_active":"2024-03-21T10:23:56.048268+0000","last_peered":"2024-03-21T10:23:56.048268+0000","last_clean":"2024-03-21T10:23:56.048268+0000","last_became_active":"2024-03-21T10:21:58.859877+0000","last_became_peered":"2024-03-21T10:21:58.859877+0000","last_unstale":"2024-03-21T10:23:56.048268+0000","last_undegraded":"2024-03-21T10:23:56.048268+0000","last_fullsized":"2024-03-21T10:23:56.048268+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_clean_scrub_stamp":"2024-03-21T10:21:57.813712+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T14:45:12.870111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"95'8","reported_seq":23,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T10:23:56.009540+0000","last_change":"2024-03-21T10:22:27.231695+0000","last_active":"2024-03-21T10:23:56.009540+0000","last_peered":"2024-03-21T10:23:56.009540+0000","last_clean":"2024-03-21T10:23:56.009540+0000","last_became_active":"2024-03-21T10:22:27.231149+0000","last_became_peered":"2024-03-21T10:22:27.231149+0000","last_unstale":"2024-03-21T10:23:56.009540+0000","last_undegraded":"2024-03-21T10:23:56.009540+0000","last_fullsized":"2024-03-21T10:23:56.009540+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_clean_scrub_stamp":"2024-03-21T10:21:57.813712+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T18:19:19.105366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,2],"acting":[9,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.3b","version":"95'4","reported_seq":33,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T10:23:56.012750+0000","last_change":"2024-03-21T10:21:58.853998+0000","last_active":"2024-03-21T10:23:56.012750+0000","last_peered":"2024-03-21T10:23:56.012750+0000","last_clean":"2024-03-21T10:23:56.012750+0000","last_became_active":"2024-03-21T10:21:58.853059+0000","last_became_peered":"2024-03-21T10:21:58.853059+0000","last_unstale":"2024-03-21T10:23:56.012750+0000","last_undegraded":"2024-03-21T10:23:56.012750+0000","last_fullsized":"2024-03-21T10:23:56.012750+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_clean_scrub_stamp":"2024-03-21T10:21:57.813712+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T17:06:36.859522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"95'6","reported_seq":35,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T10:23:56.024443+0000","last_change":"2024-03-21T10:21:58.855535+0000","last_active":"2024-03-21T10:23:56.024443+0000","last_peered":"2024-03-21T10:23:56.024443+0000","last_clean":"2024-03-21T10:23:56.024443+0000","last_became_active":"2024-03-21T10:21:58.855181+0000","last_became_peered":"2024-03-21T10:21:58.855181+0000","last_unstale":"2024-03-21T10:23:56.024443+0000","last_undegraded":"2024-03-21T10:23:56.024443+0000","last_fullsized":"2024-03-21T10:23:56.024443+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_clean_scrub_stamp":"2024-03-21T10:21:57.813712+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T14:46:35.045646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"95'5","reported_seq":34,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T10:23:56.040913+0000","last_change":"2024-03-21T10:21:58.856527+0000","last_active":"2024-03-21T10:23:56.040913+0000","last_peered":"2024-03-21T10:23:56.040913+0000","last_clean":"2024-03-21T10:23:56.040913+0000","last_became_active":"2024-03-21T10:21:58.856176+0000","last_became_peered":"2024-03-21T10:21:58.856176+0000","last_unstale":"2024-03-21T10:23:56.040913+0000","last_undegraded":"2024-03-21T10:23:56.040913+0000","last_fullsized":"2024-03-21T10:23:56.040913+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_clean_scrub_stamp":"2024-03-21T10:21:57.813712+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T21:34:07.804293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"95'9","reported_seq":38,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T10:23:56.026875+0000","last_change":"2024-03-21T10:21:58.846260+0000","last_active":"2024-03-21T10:23:56.026875+0000","last_peered":"2024-03-21T10:23:56.026875+0000","last_clean":"2024-03-21T10:23:56.026875+0000","last_became_active":"2024-03-21T10:21:58.844245+0000","last_became_peered":"2024-03-21T10:21:58.844245+0000","last_unstale":"2024-03-21T10:23:56.026875+0000","last_undegraded":"2024-03-21T10:23:56.026875+0000","last_fullsized":"2024-03-21T10:23:56.026875+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_clean_scrub_stamp":"2024-03-21T10:21:57.813712+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T16:37:07.480867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"95'11","reported_seq":40,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T10:23:56.047079+0000","last_change":"2024-03-21T10:21:58.858413+0000","last_active":"2024-03-21T10:23:56.047079+0000","last_peered":"2024-03-21T10:23:56.047079+0000","last_clean":"2024-03-21T10:23:56.047079+0000","last_became_active":"2024-03-21T10:21:58.858007+0000","last_became_peered":"2024-03-21T10:21:58.858007+0000","last_unstale":"2024-03-21T10:23:56.047079+0000","last_undegraded":"2024-03-21T10:23:56.047079+0000","last_fullsized":"2024-03-21T10:23:56.047079+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_clean_scrub_stamp":"2024-03-21T10:21:57.813712+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T21:49:12.428918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"95'16","reported_seq":45,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T10:23:56.026581+0000","last_change":"2024-03-21T10:21:58.864393+0000","last_active":"2024-03-21T10:23:56.026581+0000","last_peered":"2024-03-21T10:23:56.026581+0000","last_clean":"2024-03-21T10:23:56.026581+0000","last_became_active":"2024-03-21T10:21:58.864061+0000","last_became_peered":"2024-03-21T10:21:58.864061+0000","last_unstale":"2024-03-21T10:23:56.026581+0000","last_undegraded":"2024-03-21T10:23:56.026581+0000","last_fullsized":"2024-03-21T10:23:56.026581+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_clean_scrub_stamp":"2024-03-21T10:21:57.813712+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T20:19:44.209612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"95'6","reported_seq":35,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T10:23:56.048550+0000","last_change":"2024-03-21T10:21:58.873885+0000","last_active":"2024-03-21T10:23:56.048550+0000","last_peered":"2024-03-21T10:23:56.048550+0000","last_clean":"2024-03-21T10:23:56.048550+0000","last_became_active":"2024-03-21T10:21:58.869738+0000","last_became_peered":"2024-03-21T10:21:58.869738+0000","last_unstale":"2024-03-21T10:23:56.048550+0000","last_undegraded":"2024-03-21T10:23:56.048550+0000","last_fullsized":"2024-03-21T10:23:56.048550+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_clean_scrub_stamp":"2024-03-21T10:21:57.813712+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T13:33:26.816720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"95'7","reported_seq":36,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T10:23:56.007249+0000","last_change":"2024-03-21T10:21:58.867563+0000","last_active":"2024-03-21T10:23:56.007249+0000","last_peered":"2024-03-21T10:23:56.007249+0000","last_clean":"2024-03-21T10:23:56.007249+0000","last_became_active":"2024-03-21T10:21:58.863455+0000","last_became_peered":"2024-03-21T10:21:58.863455+0000","last_unstale":"2024-03-21T10:23:56.007249+0000","last_undegraded":"2024-03-21T10:23:56.007249+0000","last_fullsized":"2024-03-21T10:23:56.007249+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_clean_scrub_stamp":"2024-03-21T10:21:57.813712+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T15:04:16.636126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"95'10","reported_seq":45,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T10:23:56.019213+0000","last_change":"2024-03-21T10:22:27.213460+0000","last_active":"2024-03-21T10:23:56.019213+0000","last_peered":"2024-03-21T10:23:56.019213+0000","last_clean":"2024-03-21T10:23:56.019213+0000","last_became_active":"2024-03-21T10:22:27.212961+0000","last_became_peered":"2024-03-21T10:22:27.212961+0000","last_unstale":"2024-03-21T10:23:56.019213+0000","last_undegraded":"2024-03-21T10:23:56.019213+0000","last_fullsized":"2024-03-21T10:23:56.019213+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_clean_scrub_stamp":"2024-03-21T10:21:57.813712+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T18:26:03.881767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,9],"acting":[6,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"95'7","reported_seq":36,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T10:23:56.024543+0000","last_change":"2024-03-21T10:21:58.870851+0000","last_active":"2024-03-21T10:23:56.024543+0000","last_peered":"2024-03-21T10:23:56.024543+0000","last_clean":"2024-03-21T10:23:56.024543+0000","last_became_active":"2024-03-21T10:21:58.870132+0000","last_became_peered":"2024-03-21T10:21:58.870132+0000","last_unstale":"2024-03-21T10:23:56.024543+0000","last_undegraded":"2024-03-21T10:23:56.024543+0000","last_fullsized":"2024-03-21T10:23:56.024543+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_clean_scrub_stamp":"2024-03-21T10:21:57.813712+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T20:20:04.207090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"95'7","reported_seq":36,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T10:23:56.020144+0000","last_change":"2024-03-21T10:21:58.865084+0000","last_active":"2024-03-21T10:23:56.020144+0000","last_peered":"2024-03-21T10:23:56.020144+0000","last_clean":"2024-03-21T10:23:56.020144+0000","last_became_active":"2024-03-21T10:21:58.864583+0000","last_became_peered":"2024-03-21T10:21:58.864583+0000","last_unstale":"2024-03-21T10:23:56.020144+0000","last_undegraded":"2024-03-21T10:23:56.020144+0000","last_fullsized":"2024-03-21T10:23:56.020144+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_clean_scrub_stamp":"2024-03-21T10:21:57.813712+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T14:09:16.781462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"95'9","reported_seq":38,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T10:23:56.045187+0000","last_change":"2024-03-21T10:21:58.870370+0000","last_active":"2024-03-21T10:23:56.045187+0000","last_peered":"2024-03-21T10:23:56.045187+0000","last_clean":"2024-03-21T10:23:56.045187+0000","last_became_active":"2024-03-21T10:21:58.870044+0000","last_became_peered":"2024-03-21T10:21:58.870044+0000","last_unstale":"2024-03-21T10:23:56.045187+0000","last_undegraded":"2024-03-21T10:23:56.045187+0000","last_fullsized":"2024-03-21T10:23:56.045187+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_clean_scrub_stamp":"2024-03-21T10:21:57.813712+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T16:02:00.342426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"95'8","reported_seq":37,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T10:23:56.048977+0000","last_change":"2024-03-21T10:21:58.872787+0000","last_active":"2024-03-21T10:23:56.048977+0000","last_peered":"2024-03-21T10:23:56.048977+0000","last_clean":"2024-03-21T10:23:56.048977+0000","last_became_active":"2024-03-21T10:21:58.872390+0000","last_became_peered":"2024-03-21T10:21:58.872390+0000","last_unstale":"2024-03-21T10:23:56.048977+0000","last_undegraded":"2024-03-21T10:23:56.048977+0000","last_fullsized":"2024-03-21T10:23:56.048977+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_clean_scrub_stamp":"2024-03-21T10:21:57.813712+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T15:56:51.798173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"95'10","reported_seq":39,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T10:23:56.042099+0000","last_change":"2024-03-21T10:21:58.858474+0000","last_active":"2024-03-21T10:23:56.042099+0000","last_peered":"2024-03-21T10:23:56.042099+0000","last_clean":"2024-03-21T10:23:56.042099+0000","last_became_active":"2024-03-21T10:21:58.858076+0000","last_became_peered":"2024-03-21T10:21:58.858076+0000","last_unstale":"2024-03-21T10:23:56.042099+0000","last_undegraded":"2024-03-21T10:23:56.042099+0000","last_fullsized":"2024-03-21T10:23:56.042099+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_clean_scrub_stamp":"2024-03-21T10:21:57.813712+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T12:25:17.160405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"95'7","reported_seq":36,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T10:23:56.026836+0000","last_change":"2024-03-21T10:21:58.854156+0000","last_active":"2024-03-21T10:23:56.026836+0000","last_peered":"2024-03-21T10:23:56.026836+0000","last_clean":"2024-03-21T10:23:56.026836+0000","last_became_active":"2024-03-21T10:21:58.853758+0000","last_became_peered":"2024-03-21T10:21:58.853758+0000","last_unstale":"2024-03-21T10:23:56.026836+0000","last_undegraded":"2024-03-21T10:23:56.026836+0000","last_fullsized":"2024-03-21T10:23:56.026836+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_clean_scrub_stamp":"2024-03-21T10:21:57.813712+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T16:58:54.854826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":29,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T10:23:27.838445+0000","last_change":"2024-03-21T10:23:27.838445+0000","last_active":"2024-03-21T10:23:27.838445+0000","last_peered":"2024-03-21T10:23:27.838445+0000","last_clean":"2024-03-21T10:23:27.838445+0000","last_became_active":"2024-03-21T10:23:27.836992+0000","last_became_peered":"2024-03-21T10:23:27.836992+0000","last_unstale":"2024-03-21T10:23:27.838445+0000","last_undegraded":"2024-03-21T10:23:27.838445+0000","last_fullsized":"2024-03-21T10:23:27.838445+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:22:02.269614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:22:02.269614+0000","last_clean_scrub_stamp":"2024-03-21T10:22:02.269614+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T15:34:51.615332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8,0],"acting":[2,3,8,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"93'13","reported_seq":133,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T10:23:55.795133+0000","last_change":"2024-03-21T10:21:57.859554+0000","last_active":"2024-03-21T10:23:55.795133+0000","last_peered":"2024-03-21T10:23:55.795133+0000","last_clean":"2024-03-21T10:23:55.795133+0000","last_became_active":"2024-03-21T10:21:57.857997+0000","last_became_peered":"2024-03-21T10:21:57.857997+0000","last_unstale":"2024-03-21T10:23:55.795133+0000","last_undegraded":"2024-03-21T10:23:55.795133+0000","last_fullsized":"2024-03-21T10:23:55.795133+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:21:56.788908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:21:56.788908+0000","last_clean_scrub_stamp":"2024-03-21T10:21:56.788908+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T14:27:47.516832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":89,"num_read_kb":89,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"95'5","reported_seq":35,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T10:23:56.021709+0000","last_change":"2024-03-21T10:21:58.850197+0000","last_active":"2024-03-21T10:23:56.021709+0000","last_peered":"2024-03-21T10:23:56.021709+0000","last_clean":"2024-03-21T10:23:56.021709+0000","last_became_active":"2024-03-21T10:21:58.849029+0000","last_became_peered":"2024-03-21T10:21:58.849029+0000","last_unstale":"2024-03-21T10:23:56.021709+0000","last_undegraded":"2024-03-21T10:23:56.021709+0000","last_fullsized":"2024-03-21T10:23:56.021709+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_clean_scrub_stamp":"2024-03-21T10:21:57.813712+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T15:46:49.077382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":22,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T10:23:27.639994+0000","last_change":"2024-03-21T10:22:03.320716+0000","last_active":"2024-03-21T10:23:27.639994+0000","last_peered":"2024-03-21T10:23:27.639994+0000","last_clean":"2024-03-21T10:23:27.639994+0000","last_became_active":"2024-03-21T10:22:03.320262+0000","last_became_peered":"2024-03-21T10:22:03.320262+0000","last_unstale":"2024-03-21T10:23:27.639994+0000","last_undegraded":"2024-03-21T10:23:27.639994+0000","last_fullsized":"2024-03-21T10:23:27.639994+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:22:02.269614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:22:02.269614+0000","last_clean_scrub_stamp":"2024-03-21T10:22:02.269614+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T16:16:07.423168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"92'2","reported_seq":34,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T10:23:27.639016+0000","last_change":"2024-03-21T10:21:57.863642+0000","last_active":"2024-03-21T10:23:27.639016+0000","last_peered":"2024-03-21T10:23:27.639016+0000","last_clean":"2024-03-21T10:23:27.639016+0000","last_became_active":"2024-03-21T10:21:57.863330+0000","last_became_peered":"2024-03-21T10:21:57.863330+0000","last_unstale":"2024-03-21T10:23:27.639016+0000","last_undegraded":"2024-03-21T10:23:27.639016+0000","last_fullsized":"2024-03-21T10:23:27.639016+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:21:56.788908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:21:56.788908+0000","last_clean_scrub_stamp":"2024-03-21T10:21:56.788908+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T14:45:02.131458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"95'6","reported_seq":35,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T10:23:56.048551+0000","last_change":"2024-03-21T10:21:58.862940+0000","last_active":"2024-03-21T10:23:56.048551+0000","last_peered":"2024-03-21T10:23:56.048551+0000","last_clean":"2024-03-21T10:23:56.048551+0000","last_became_active":"2024-03-21T10:21:58.862489+0000","last_became_peered":"2024-03-21T10:21:58.862489+0000","last_unstale":"2024-03-21T10:23:56.048551+0000","last_undegraded":"2024-03-21T10:23:56.048551+0000","last_fullsized":"2024-03-21T10:23:56.048551+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_clean_scrub_stamp":"2024-03-21T10:21:57.813712+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T19:28:02.175642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":22,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T10:23:27.640475+0000","last_change":"2024-03-21T10:22:03.318660+0000","last_active":"2024-03-21T10:23:27.640475+0000","last_peered":"2024-03-21T10:23:27.640475+0000","last_clean":"2024-03-21T10:23:27.640475+0000","last_became_active":"2024-03-21T10:22:03.318216+0000","last_became_peered":"2024-03-21T10:22:03.318216+0000","last_unstale":"2024-03-21T10:23:27.640475+0000","last_undegraded":"2024-03-21T10:23:27.640475+0000","last_fullsized":"2024-03-21T10:23:27.640475+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:22:02.269614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:22:02.269614+0000","last_clean_scrub_stamp":"2024-03-21T10:22:02.269614+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T16:36:39.930726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"93'12","reported_seq":30,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T10:23:27.837073+0000","last_change":"2024-03-21T10:22:27.239342+0000","last_active":"2024-03-21T10:23:27.837073+0000","last_peered":"2024-03-21T10:23:27.837073+0000","last_clean":"2024-03-21T10:23:27.837073+0000","last_became_active":"2024-03-21T10:22:27.224086+0000","last_became_peered":"2024-03-21T10:22:27.224086+0000","last_unstale":"2024-03-21T10:23:27.837073+0000","last_undegraded":"2024-03-21T10:23:27.837073+0000","last_fullsized":"2024-03-21T10:23:27.837073+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:21:56.788908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:21:56.788908+0000","last_clean_scrub_stamp":"2024-03-21T10:21:56.788908+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T20:46:45.951485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,7],"acting":[11,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1c","version":"95'9","reported_seq":38,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T10:23:56.043031+0000","last_change":"2024-03-21T10:21:58.873239+0000","last_active":"2024-03-21T10:23:56.043031+0000","last_peered":"2024-03-21T10:23:56.043031+0000","last_clean":"2024-03-21T10:23:56.043031+0000","last_became_active":"2024-03-21T10:21:58.872877+0000","last_became_peered":"2024-03-21T10:21:58.872877+0000","last_unstale":"2024-03-21T10:23:56.043031+0000","last_undegraded":"2024-03-21T10:23:56.043031+0000","last_fullsized":"2024-03-21T10:23:56.043031+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_clean_scrub_stamp":"2024-03-21T10:21:57.813712+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T16:22:49.070697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":22,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T10:23:27.640146+0000","last_change":"2024-03-21T10:22:03.311589+0000","last_active":"2024-03-21T10:23:27.640146+0000","last_peered":"2024-03-21T10:23:27.640146+0000","last_clean":"2024-03-21T10:23:27.640146+0000","last_became_active":"2024-03-21T10:22:03.311134+0000","last_became_peered":"2024-03-21T10:22:03.311134+0000","last_unstale":"2024-03-21T10:23:27.640146+0000","last_undegraded":"2024-03-21T10:23:27.640146+0000","last_fullsized":"2024-03-21T10:23:27.640146+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:22:02.269614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:22:02.269614+0000","last_clean_scrub_stamp":"2024-03-21T10:22:02.269614+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T10:22:29.138348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"93'5","reported_seq":37,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T10:23:27.646116+0000","last_change":"2024-03-21T10:21:57.858981+0000","last_active":"2024-03-21T10:23:27.646116+0000","last_peered":"2024-03-21T10:23:27.646116+0000","last_clean":"2024-03-21T10:23:27.646116+0000","last_became_active":"2024-03-21T10:21:57.858564+0000","last_became_peered":"2024-03-21T10:21:57.858564+0000","last_unstale":"2024-03-21T10:23:27.646116+0000","last_undegraded":"2024-03-21T10:23:27.646116+0000","last_fullsized":"2024-03-21T10:23:27.646116+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:21:56.788908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:21:56.788908+0000","last_clean_scrub_stamp":"2024-03-21T10:21:56.788908+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T10:45:09.488194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"95'15","reported_seq":44,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T10:23:56.039595+0000","last_change":"2024-03-21T10:21:58.858053+0000","last_active":"2024-03-21T10:23:56.039595+0000","last_peered":"2024-03-21T10:23:56.039595+0000","last_clean":"2024-03-21T10:23:56.039595+0000","last_became_active":"2024-03-21T10:21:58.857688+0000","last_became_peered":"2024-03-21T10:21:58.857688+0000","last_unstale":"2024-03-21T10:23:56.039595+0000","last_undegraded":"2024-03-21T10:23:56.039595+0000","last_fullsized":"2024-03-21T10:23:56.039595+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_clean_scrub_stamp":"2024-03-21T10:21:57.813712+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T19:45:13.043034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":29,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T10:23:27.842200+0000","last_change":"2024-03-21T10:23:27.842200+0000","last_active":"2024-03-21T10:23:27.842200+0000","last_peered":"2024-03-21T10:23:27.842200+0000","last_clean":"2024-03-21T10:23:27.842200+0000","last_became_active":"2024-03-21T10:23:27.840749+0000","last_became_peered":"2024-03-21T10:23:27.840749+0000","last_unstale":"2024-03-21T10:23:27.842200+0000","last_undegraded":"2024-03-21T10:23:27.842200+0000","last_fullsized":"2024-03-21T10:23:27.842200+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:22:02.269614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:22:02.269614+0000","last_clean_scrub_stamp":"2024-03-21T10:22:02.269614+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T22:01:31.714771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,7,11],"acting":[2,8,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"91'5","reported_seq":140,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T10:23:56.824925+0000","last_change":"2024-03-21T10:22:27.235744+0000","last_active":"2024-03-21T10:23:56.824925+0000","last_peered":"2024-03-21T10:23:56.824925+0000","last_clean":"2024-03-21T10:23:56.824925+0000","last_became_active":"2024-03-21T10:22:27.219999+0000","last_became_peered":"2024-03-21T10:22:27.219999+0000","last_unstale":"2024-03-21T10:23:56.824925+0000","last_undegraded":"2024-03-21T10:23:56.824925+0000","last_fullsized":"2024-03-21T10:23:56.824925+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:21:56.788908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:21:56.788908+0000","last_clean_scrub_stamp":"2024-03-21T10:21:56.788908+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T18:15:41.288034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":92,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":622,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7],"acting":[5,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"95'7","reported_seq":36,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T10:23:55.996757+0000","last_change":"2024-03-21T10:21:58.854752+0000","last_active":"2024-03-21T10:23:55.996757+0000","last_peered":"2024-03-21T10:23:55.996757+0000","last_clean":"2024-03-21T10:23:55.996757+0000","last_became_active":"2024-03-21T10:21:58.854442+0000","last_became_peered":"2024-03-21T10:21:58.854442+0000","last_unstale":"2024-03-21T10:23:55.996757+0000","last_undegraded":"2024-03-21T10:23:55.996757+0000","last_fullsized":"2024-03-21T10:23:55.996757+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_clean_scrub_stamp":"2024-03-21T10:21:57.813712+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T15:12:40.590585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":22,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T10:23:27.648186+0000","last_change":"2024-03-21T10:22:03.319247+0000","last_active":"2024-03-21T10:23:27.648186+0000","last_peered":"2024-03-21T10:23:27.648186+0000","last_clean":"2024-03-21T10:23:27.648186+0000","last_became_active":"2024-03-21T10:22:03.318731+0000","last_became_peered":"2024-03-21T10:22:03.318731+0000","last_unstale":"2024-03-21T10:23:27.648186+0000","last_undegraded":"2024-03-21T10:23:27.648186+0000","last_fullsized":"2024-03-21T10:23:27.648186+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:22:02.269614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:22:02.269614+0000","last_clean_scrub_stamp":"2024-03-21T10:22:02.269614+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T20:06:36.519559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"93'7","reported_seq":44,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T10:23:27.744356+0000","last_change":"2024-03-21T10:22:27.222981+0000","last_active":"2024-03-21T10:23:27.744356+0000","last_peered":"2024-03-21T10:23:27.744356+0000","last_clean":"2024-03-21T10:23:27.744356+0000","last_became_active":"2024-03-21T10:22:27.221088+0000","last_became_peered":"2024-03-21T10:22:27.221088+0000","last_unstale":"2024-03-21T10:23:27.744356+0000","last_undegraded":"2024-03-21T10:23:27.744356+0000","last_fullsized":"2024-03-21T10:23:27.744356+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:21:56.788908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:21:56.788908+0000","last_clean_scrub_stamp":"2024-03-21T10:21:56.788908+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T15:43:11.740353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,6],"acting":[10,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"95'8","reported_seq":43,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T10:23:56.027278+0000","last_change":"2024-03-21T10:22:27.226733+0000","last_active":"2024-03-21T10:23:56.027278+0000","last_peered":"2024-03-21T10:23:56.027278+0000","last_clean":"2024-03-21T10:23:56.027278+0000","last_became_active":"2024-03-21T10:22:27.225576+0000","last_became_peered":"2024-03-21T10:22:27.225576+0000","last_unstale":"2024-03-21T10:23:56.027278+0000","last_undegraded":"2024-03-21T10:23:56.027278+0000","last_fullsized":"2024-03-21T10:23:56.027278+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_clean_scrub_stamp":"2024-03-21T10:21:57.813712+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T20:06:09.534268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,3],"acting":[5,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":22,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T10:23:27.836108+0000","last_change":"2024-03-21T10:22:03.321595+0000","last_active":"2024-03-21T10:23:27.836108+0000","last_peered":"2024-03-21T10:23:27.836108+0000","last_clean":"2024-03-21T10:23:27.836108+0000","last_became_active":"2024-03-21T10:22:03.320897+0000","last_became_peered":"2024-03-21T10:22:03.320897+0000","last_unstale":"2024-03-21T10:23:27.836108+0000","last_undegraded":"2024-03-21T10:23:27.836108+0000","last_fullsized":"2024-03-21T10:23:27.836108+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:22:02.269614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:22:02.269614+0000","last_clean_scrub_stamp":"2024-03-21T10:22:02.269614+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T17:37:02.633709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"93'3","reported_seq":34,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T10:23:27.742152+0000","last_change":"2024-03-21T10:21:57.854801+0000","last_active":"2024-03-21T10:23:27.742152+0000","last_peered":"2024-03-21T10:23:27.742152+0000","last_clean":"2024-03-21T10:23:27.742152+0000","last_became_active":"2024-03-21T10:21:57.854443+0000","last_became_peered":"2024-03-21T10:21:57.854443+0000","last_unstale":"2024-03-21T10:23:27.742152+0000","last_undegraded":"2024-03-21T10:23:27.742152+0000","last_fullsized":"2024-03-21T10:23:27.742152+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:21:56.788908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:21:56.788908+0000","last_clean_scrub_stamp":"2024-03-21T10:21:56.788908+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T15:49:23.573360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"95'8","reported_seq":37,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T10:23:56.022963+0000","last_change":"2024-03-21T10:21:58.857167+0000","last_active":"2024-03-21T10:23:56.022963+0000","last_peered":"2024-03-21T10:23:56.022963+0000","last_clean":"2024-03-21T10:23:56.022963+0000","last_became_active":"2024-03-21T10:21:58.856827+0000","last_became_peered":"2024-03-21T10:21:58.856827+0000","last_unstale":"2024-03-21T10:23:56.022963+0000","last_undegraded":"2024-03-21T10:23:56.022963+0000","last_fullsized":"2024-03-21T10:23:56.022963+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_clean_scrub_stamp":"2024-03-21T10:21:57.813712+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T15:04:43.848209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":22,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T10:23:27.834872+0000","last_change":"2024-03-21T10:22:03.306554+0000","last_active":"2024-03-21T10:23:27.834872+0000","last_peered":"2024-03-21T10:23:27.834872+0000","last_clean":"2024-03-21T10:23:27.834872+0000","last_became_active":"2024-03-21T10:22:03.306087+0000","last_became_peered":"2024-03-21T10:22:03.306087+0000","last_unstale":"2024-03-21T10:23:27.834872+0000","last_undegraded":"2024-03-21T10:23:27.834872+0000","last_fullsized":"2024-03-21T10:23:27.834872+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:22:02.269614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:22:02.269614+0000","last_clean_scrub_stamp":"2024-03-21T10:22:02.269614+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T11:36:35.344770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"93'3","reported_seq":35,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T10:23:27.750933+0000","last_change":"2024-03-21T10:21:57.860343+0000","last_active":"2024-03-21T10:23:27.750933+0000","last_peered":"2024-03-21T10:23:27.750933+0000","last_clean":"2024-03-21T10:23:27.750933+0000","last_became_active":"2024-03-21T10:21:57.859808+0000","last_became_peered":"2024-03-21T10:21:57.859808+0000","last_unstale":"2024-03-21T10:23:27.750933+0000","last_undegraded":"2024-03-21T10:23:27.750933+0000","last_fullsized":"2024-03-21T10:23:27.750933+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:21:56.788908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:21:56.788908+0000","last_clean_scrub_stamp":"2024-03-21T10:21:56.788908+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T14:41:24.162795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"95'11","reported_seq":40,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T10:23:56.017971+0000","last_change":"2024-03-21T10:21:58.850991+0000","last_active":"2024-03-21T10:23:56.017971+0000","last_peered":"2024-03-21T10:23:56.017971+0000","last_clean":"2024-03-21T10:23:56.017971+0000","last_became_active":"2024-03-21T10:21:58.850581+0000","last_became_peered":"2024-03-21T10:21:58.850581+0000","last_unstale":"2024-03-21T10:23:56.017971+0000","last_undegraded":"2024-03-21T10:23:56.017971+0000","last_fullsized":"2024-03-21T10:23:56.017971+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_clean_scrub_stamp":"2024-03-21T10:21:57.813712+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T17:07:11.350806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":22,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T10:23:27.638473+0000","last_change":"2024-03-21T10:22:03.315333+0000","last_active":"2024-03-21T10:23:27.638473+0000","last_peered":"2024-03-21T10:23:27.638473+0000","last_clean":"2024-03-21T10:23:27.638473+0000","last_became_active":"2024-03-21T10:22:03.313553+0000","last_became_peered":"2024-03-21T10:22:03.313553+0000","last_unstale":"2024-03-21T10:23:27.638473+0000","last_undegraded":"2024-03-21T10:23:27.638473+0000","last_fullsized":"2024-03-21T10:23:27.638473+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:22:02.269614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:22:02.269614+0000","last_clean_scrub_stamp":"2024-03-21T10:22:02.269614+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T15:25:24.290789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"93'4","reported_seq":215,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T10:23:57.825016+0000","last_change":"2024-03-21T10:21:57.864930+0000","last_active":"2024-03-21T10:23:57.825016+0000","last_peered":"2024-03-21T10:23:57.825016+0000","last_clean":"2024-03-21T10:23:57.825016+0000","last_became_active":"2024-03-21T10:21:57.863769+0000","last_became_peered":"2024-03-21T10:21:57.863769+0000","last_unstale":"2024-03-21T10:23:57.825016+0000","last_undegraded":"2024-03-21T10:23:57.825016+0000","last_fullsized":"2024-03-21T10:23:57.825016+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:21:56.788908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:21:56.788908+0000","last_clean_scrub_stamp":"2024-03-21T10:21:56.788908+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T21:35:39.144065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":180,"num_read_kb":180,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"95'10","reported_seq":39,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T10:23:56.044272+0000","last_change":"2024-03-21T10:21:58.872084+0000","last_active":"2024-03-21T10:23:56.044272+0000","last_peered":"2024-03-21T10:23:56.044272+0000","last_clean":"2024-03-21T10:23:56.044272+0000","last_became_active":"2024-03-21T10:21:58.871725+0000","last_became_peered":"2024-03-21T10:21:58.871725+0000","last_unstale":"2024-03-21T10:23:56.044272+0000","last_undegraded":"2024-03-21T10:23:56.044272+0000","last_fullsized":"2024-03-21T10:23:56.044272+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_clean_scrub_stamp":"2024-03-21T10:21:57.813712+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T16:19:37.240053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":22,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T10:23:27.640980+0000","last_change":"2024-03-21T10:22:03.317808+0000","last_active":"2024-03-21T10:23:27.640980+0000","last_peered":"2024-03-21T10:23:27.640980+0000","last_clean":"2024-03-21T10:23:27.640980+0000","last_became_active":"2024-03-21T10:22:03.313751+0000","last_became_peered":"2024-03-21T10:22:03.313751+0000","last_unstale":"2024-03-21T10:23:27.640980+0000","last_undegraded":"2024-03-21T10:23:27.640980+0000","last_fullsized":"2024-03-21T10:23:27.640980+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:22:02.269614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:22:02.269614+0000","last_clean_scrub_stamp":"2024-03-21T10:22:02.269614+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T20:05:08.105795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"93'4","reported_seq":35,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T10:23:27.646569+0000","last_change":"2024-03-21T10:21:57.865462+0000","last_active":"2024-03-21T10:23:27.646569+0000","last_peered":"2024-03-21T10:23:27.646569+0000","last_clean":"2024-03-21T10:23:27.646569+0000","last_became_active":"2024-03-21T10:21:57.862922+0000","last_became_peered":"2024-03-21T10:21:57.862922+0000","last_unstale":"2024-03-21T10:23:27.646569+0000","last_undegraded":"2024-03-21T10:23:27.646569+0000","last_fullsized":"2024-03-21T10:23:27.646569+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:21:56.788908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:21:56.788908+0000","last_clean_scrub_stamp":"2024-03-21T10:21:56.788908+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T15:31:17.167806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"95'9","reported_seq":38,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T10:23:56.022798+0000","last_change":"2024-03-21T10:21:58.865459+0000","last_active":"2024-03-21T10:23:56.022798+0000","last_peered":"2024-03-21T10:23:56.022798+0000","last_clean":"2024-03-21T10:23:56.022798+0000","last_became_active":"2024-03-21T10:21:58.860909+0000","last_became_peered":"2024-03-21T10:21:58.860909+0000","last_unstale":"2024-03-21T10:23:56.022798+0000","last_undegraded":"2024-03-21T10:23:56.022798+0000","last_fullsized":"2024-03-21T10:23:56.022798+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_clean_scrub_stamp":"2024-03-21T10:21:57.813712+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T15:53:35.110404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":22,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T10:23:27.640600+0000","last_change":"2024-03-21T10:22:03.318651+0000","last_active":"2024-03-21T10:23:27.640600+0000","last_peered":"2024-03-21T10:23:27.640600+0000","last_clean":"2024-03-21T10:23:27.640600+0000","last_became_active":"2024-03-21T10:22:03.318108+0000","last_became_peered":"2024-03-21T10:22:03.318108+0000","last_unstale":"2024-03-21T10:23:27.640600+0000","last_undegraded":"2024-03-21T10:23:27.640600+0000","last_fullsized":"2024-03-21T10:23:27.640600+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:22:02.269614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:22:02.269614+0000","last_clean_scrub_stamp":"2024-03-21T10:22:02.269614+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T19:00:04.216749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"93'5","reported_seq":37,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T10:23:27.751637+0000","last_change":"2024-03-21T10:21:57.865516+0000","last_active":"2024-03-21T10:23:27.751637+0000","last_peered":"2024-03-21T10:23:27.751637+0000","last_clean":"2024-03-21T10:23:27.751637+0000","last_became_active":"2024-03-21T10:21:57.865012+0000","last_became_peered":"2024-03-21T10:21:57.865012+0000","last_unstale":"2024-03-21T10:23:27.751637+0000","last_undegraded":"2024-03-21T10:23:27.751637+0000","last_fullsized":"2024-03-21T10:23:27.751637+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:21:56.788908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:21:56.788908+0000","last_clean_scrub_stamp":"2024-03-21T10:21:56.788908+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T21:38:02.813453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"95'11","reported_seq":40,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T10:23:56.044642+0000","last_change":"2024-03-21T10:21:58.870876+0000","last_active":"2024-03-21T10:23:56.044642+0000","last_peered":"2024-03-21T10:23:56.044642+0000","last_clean":"2024-03-21T10:23:56.044642+0000","last_became_active":"2024-03-21T10:21:58.868240+0000","last_became_peered":"2024-03-21T10:21:58.868240+0000","last_unstale":"2024-03-21T10:23:56.044642+0000","last_undegraded":"2024-03-21T10:23:56.044642+0000","last_fullsized":"2024-03-21T10:23:56.044642+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_clean_scrub_stamp":"2024-03-21T10:21:57.813712+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T15:16:59.496020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":22,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T10:23:27.641156+0000","last_change":"2024-03-21T10:22:03.314854+0000","last_active":"2024-03-21T10:23:27.641156+0000","last_peered":"2024-03-21T10:23:27.641156+0000","last_clean":"2024-03-21T10:23:27.641156+0000","last_became_active":"2024-03-21T10:22:03.314422+0000","last_became_peered":"2024-03-21T10:22:03.314422+0000","last_unstale":"2024-03-21T10:23:27.641156+0000","last_undegraded":"2024-03-21T10:23:27.641156+0000","last_fullsized":"2024-03-21T10:23:27.641156+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:22:02.269614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:22:02.269614+0000","last_clean_scrub_stamp":"2024-03-21T10:22:02.269614+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T12:34:31.519118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.14","version":"95'35","reported_seq":271,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T10:23:57.825713+0000","last_change":"2024-03-21T10:22:27.243817+0000","last_active":"2024-03-21T10:23:57.825713+0000","last_peered":"2024-03-21T10:23:57.825713+0000","last_clean":"2024-03-21T10:23:57.825713+0000","last_became_active":"2024-03-21T10:22:27.218873+0000","last_became_peered":"2024-03-21T10:22:27.218873+0000","last_unstale":"2024-03-21T10:23:57.825713+0000","last_undegraded":"2024-03-21T10:23:57.825713+0000","last_fullsized":"2024-03-21T10:23:57.825713+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:21:56.788908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:21:56.788908+0000","last_clean_scrub_stamp":"2024-03-21T10:21:56.788908+0000","objects_scrubbed":0,"log_size":35,"log_dups_size":0,"ondisk_log_size":35,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T12:27:43.574451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":60041,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":187,"num_read_kb":152,"num_write":37,"num_write_kb":75,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":3842,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,4],"acting":[10,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"95'6","reported_seq":35,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T10:23:56.011874+0000","last_change":"2024-03-21T10:21:58.860386+0000","last_active":"2024-03-21T10:23:56.011874+0000","last_peered":"2024-03-21T10:23:56.011874+0000","last_clean":"2024-03-21T10:23:56.011874+0000","last_became_active":"2024-03-21T10:21:58.860013+0000","last_became_peered":"2024-03-21T10:21:58.860013+0000","last_unstale":"2024-03-21T10:23:56.011874+0000","last_undegraded":"2024-03-21T10:23:56.011874+0000","last_fullsized":"2024-03-21T10:23:56.011874+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_clean_scrub_stamp":"2024-03-21T10:21:57.813712+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T18:02:33.920374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":22,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T10:23:27.752375+0000","last_change":"2024-03-21T10:22:03.314083+0000","last_active":"2024-03-21T10:23:27.752375+0000","last_peered":"2024-03-21T10:23:27.752375+0000","last_clean":"2024-03-21T10:23:27.752375+0000","last_became_active":"2024-03-21T10:22:03.313652+0000","last_became_peered":"2024-03-21T10:22:03.313652+0000","last_unstale":"2024-03-21T10:23:27.752375+0000","last_undegraded":"2024-03-21T10:23:27.752375+0000","last_fullsized":"2024-03-21T10:23:27.752375+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:22:02.269614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:22:02.269614+0000","last_clean_scrub_stamp":"2024-03-21T10:22:02.269614+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T13:23:51.256535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"93'9","reported_seq":42,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T10:23:27.647998+0000","last_change":"2024-03-21T10:21:57.866716+0000","last_active":"2024-03-21T10:23:27.647998+0000","last_peered":"2024-03-21T10:23:27.647998+0000","last_clean":"2024-03-21T10:23:27.647998+0000","last_became_active":"2024-03-21T10:21:57.860900+0000","last_became_peered":"2024-03-21T10:21:57.860900+0000","last_unstale":"2024-03-21T10:23:27.647998+0000","last_undegraded":"2024-03-21T10:23:27.647998+0000","last_fullsized":"2024-03-21T10:23:27.647998+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:21:56.788908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:21:56.788908+0000","last_clean_scrub_stamp":"2024-03-21T10:21:56.788908+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T13:13:04.937120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"95'9","reported_seq":38,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T10:23:56.016388+0000","last_change":"2024-03-21T10:21:58.858815+0000","last_active":"2024-03-21T10:23:56.016388+0000","last_peered":"2024-03-21T10:23:56.016388+0000","last_clean":"2024-03-21T10:23:56.016388+0000","last_became_active":"2024-03-21T10:21:58.856483+0000","last_became_peered":"2024-03-21T10:21:58.856483+0000","last_unstale":"2024-03-21T10:23:56.016388+0000","last_undegraded":"2024-03-21T10:23:56.016388+0000","last_fullsized":"2024-03-21T10:23:56.016388+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_clean_scrub_stamp":"2024-03-21T10:21:57.813712+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T15:42:38.208093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":22,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T10:23:27.836414+0000","last_change":"2024-03-21T10:22:03.319719+0000","last_active":"2024-03-21T10:23:27.836414+0000","last_peered":"2024-03-21T10:23:27.836414+0000","last_clean":"2024-03-21T10:23:27.836414+0000","last_became_active":"2024-03-21T10:22:03.319234+0000","last_became_peered":"2024-03-21T10:22:03.319234+0000","last_unstale":"2024-03-21T10:23:27.836414+0000","last_undegraded":"2024-03-21T10:23:27.836414+0000","last_fullsized":"2024-03-21T10:23:27.836414+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:22:02.269614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:22:02.269614+0000","last_clean_scrub_stamp":"2024-03-21T10:22:02.269614+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T14:11:21.609594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"93'5","reported_seq":37,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T10:23:27.751003+0000","last_change":"2024-03-21T10:21:57.862744+0000","last_active":"2024-03-21T10:23:27.751003+0000","last_peered":"2024-03-21T10:23:27.751003+0000","last_clean":"2024-03-21T10:23:27.751003+0000","last_became_active":"2024-03-21T10:21:57.862352+0000","last_became_peered":"2024-03-21T10:21:57.862352+0000","last_unstale":"2024-03-21T10:23:27.751003+0000","last_undegraded":"2024-03-21T10:23:27.751003+0000","last_fullsized":"2024-03-21T10:23:27.751003+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:21:56.788908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:21:56.788908+0000","last_clean_scrub_stamp":"2024-03-21T10:21:56.788908+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T20:19:50.138904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"95'15","reported_seq":44,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T10:23:56.046249+0000","last_change":"2024-03-21T10:21:58.863514+0000","last_active":"2024-03-21T10:23:56.046249+0000","last_peered":"2024-03-21T10:23:56.046249+0000","last_clean":"2024-03-21T10:23:56.046249+0000","last_became_active":"2024-03-21T10:21:58.861981+0000","last_became_peered":"2024-03-21T10:21:58.861981+0000","last_unstale":"2024-03-21T10:23:56.046249+0000","last_undegraded":"2024-03-21T10:23:56.046249+0000","last_fullsized":"2024-03-21T10:23:56.046249+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_clean_scrub_stamp":"2024-03-21T10:21:57.813712+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T15:00:33.813456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":22,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T10:23:27.642269+0000","last_change":"2024-03-21T10:22:03.309844+0000","last_active":"2024-03-21T10:23:27.642269+0000","last_peered":"2024-03-21T10:23:27.642269+0000","last_clean":"2024-03-21T10:23:27.642269+0000","last_became_active":"2024-03-21T10:22:03.309448+0000","last_became_peered":"2024-03-21T10:22:03.309448+0000","last_unstale":"2024-03-21T10:23:27.642269+0000","last_undegraded":"2024-03-21T10:23:27.642269+0000","last_fullsized":"2024-03-21T10:23:27.642269+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:22:02.269614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:22:02.269614+0000","last_clean_scrub_stamp":"2024-03-21T10:22:02.269614+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T18:38:51.167166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.11","version":"95'7","reported_seq":138,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T10:23:57.826401+0000","last_change":"2024-03-21T10:22:27.223337+0000","last_active":"2024-03-21T10:23:57.826401+0000","last_peered":"2024-03-21T10:23:57.826401+0000","last_clean":"2024-03-21T10:23:57.826401+0000","last_became_active":"2024-03-21T10:22:27.222999+0000","last_became_peered":"2024-03-21T10:22:27.222999+0000","last_unstale":"2024-03-21T10:23:57.826401+0000","last_undegraded":"2024-03-21T10:23:57.826401+0000","last_fullsized":"2024-03-21T10:23:57.826401+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:21:56.788908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:21:56.788908+0000","last_clean_scrub_stamp":"2024-03-21T10:21:56.788908+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T20:40:45.780696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1120,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":95,"num_read_kb":4,"num_write":7,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,10],"acting":[4,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"95'6","reported_seq":35,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T10:23:56.037051+0000","last_change":"2024-03-21T10:21:58.862225+0000","last_active":"2024-03-21T10:23:56.037051+0000","last_peered":"2024-03-21T10:23:56.037051+0000","last_clean":"2024-03-21T10:23:56.037051+0000","last_became_active":"2024-03-21T10:21:58.861816+0000","last_became_peered":"2024-03-21T10:21:58.861816+0000","last_unstale":"2024-03-21T10:23:56.037051+0000","last_undegraded":"2024-03-21T10:23:56.037051+0000","last_fullsized":"2024-03-21T10:23:56.037051+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_clean_scrub_stamp":"2024-03-21T10:21:57.813712+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T18:36:13.420867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":22,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T10:23:27.742892+0000","last_change":"2024-03-21T10:22:03.315306+0000","last_active":"2024-03-21T10:23:27.742892+0000","last_peered":"2024-03-21T10:23:27.742892+0000","last_clean":"2024-03-21T10:23:27.742892+0000","last_became_active":"2024-03-21T10:22:03.314839+0000","last_became_peered":"2024-03-21T10:22:03.314839+0000","last_unstale":"2024-03-21T10:23:27.742892+0000","last_undegraded":"2024-03-21T10:23:27.742892+0000","last_fullsized":"2024-03-21T10:23:27.742892+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:22:02.269614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:22:02.269614+0000","last_clean_scrub_stamp":"2024-03-21T10:22:02.269614+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T15:47:45.690434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"93'17","reported_seq":49,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T10:23:27.644391+0000","last_change":"2024-03-21T10:21:57.870998+0000","last_active":"2024-03-21T10:23:27.644391+0000","last_peered":"2024-03-21T10:23:27.644391+0000","last_clean":"2024-03-21T10:23:27.644391+0000","last_became_active":"2024-03-21T10:21:57.870605+0000","last_became_peered":"2024-03-21T10:21:57.870605+0000","last_unstale":"2024-03-21T10:23:27.644391+0000","last_undegraded":"2024-03-21T10:23:27.644391+0000","last_fullsized":"2024-03-21T10:23:27.644391+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:21:56.788908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:21:56.788908+0000","last_clean_scrub_stamp":"2024-03-21T10:21:56.788908+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T13:33:54.775122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"95'6","reported_seq":35,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T10:23:56.030355+0000","last_change":"2024-03-21T10:21:58.858675+0000","last_active":"2024-03-21T10:23:56.030355+0000","last_peered":"2024-03-21T10:23:56.030355+0000","last_clean":"2024-03-21T10:23:56.030355+0000","last_became_active":"2024-03-21T10:21:58.856907+0000","last_became_peered":"2024-03-21T10:21:58.856907+0000","last_unstale":"2024-03-21T10:23:56.030355+0000","last_undegraded":"2024-03-21T10:23:56.030355+0000","last_fullsized":"2024-03-21T10:23:56.030355+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_clean_scrub_stamp":"2024-03-21T10:21:57.813712+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T18:17:31.208728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":22,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T10:23:27.750514+0000","last_change":"2024-03-21T10:22:03.318257+0000","last_active":"2024-03-21T10:23:27.750514+0000","last_peered":"2024-03-21T10:23:27.750514+0000","last_clean":"2024-03-21T10:23:27.750514+0000","last_became_active":"2024-03-21T10:22:03.317622+0000","last_became_peered":"2024-03-21T10:22:03.317622+0000","last_unstale":"2024-03-21T10:23:27.750514+0000","last_undegraded":"2024-03-21T10:23:27.750514+0000","last_fullsized":"2024-03-21T10:23:27.750514+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:22:02.269614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:22:02.269614+0000","last_clean_scrub_stamp":"2024-03-21T10:22:02.269614+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T17:15:50.272193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"93'2","reported_seq":34,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T10:23:27.647207+0000","last_change":"2024-03-21T10:21:57.868014+0000","last_active":"2024-03-21T10:23:27.647207+0000","last_peered":"2024-03-21T10:23:27.647207+0000","last_clean":"2024-03-21T10:23:27.647207+0000","last_became_active":"2024-03-21T10:21:57.867189+0000","last_became_peered":"2024-03-21T10:21:57.867189+0000","last_unstale":"2024-03-21T10:23:27.647207+0000","last_undegraded":"2024-03-21T10:23:27.647207+0000","last_fullsized":"2024-03-21T10:23:27.647207+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:21:56.788908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:21:56.788908+0000","last_clean_scrub_stamp":"2024-03-21T10:21:56.788908+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T22:02:48.704932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"95'5","reported_seq":34,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T10:23:56.003661+0000","last_change":"2024-03-21T10:21:58.862205+0000","last_active":"2024-03-21T10:23:56.003661+0000","last_peered":"2024-03-21T10:23:56.003661+0000","last_clean":"2024-03-21T10:23:56.003661+0000","last_became_active":"2024-03-21T10:21:58.861186+0000","last_became_peered":"2024-03-21T10:21:58.861186+0000","last_unstale":"2024-03-21T10:23:56.003661+0000","last_undegraded":"2024-03-21T10:23:56.003661+0000","last_fullsized":"2024-03-21T10:23:56.003661+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_clean_scrub_stamp":"2024-03-21T10:21:57.813712+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T19:44:49.719845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":22,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T10:23:27.643412+0000","last_change":"2024-03-21T10:22:03.636064+0000","last_active":"2024-03-21T10:23:27.643412+0000","last_peered":"2024-03-21T10:23:27.643412+0000","last_clean":"2024-03-21T10:23:27.643412+0000","last_became_active":"2024-03-21T10:22:03.633596+0000","last_became_peered":"2024-03-21T10:22:03.633596+0000","last_unstale":"2024-03-21T10:23:27.643412+0000","last_undegraded":"2024-03-21T10:23:27.643412+0000","last_fullsized":"2024-03-21T10:23:27.643412+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:22:02.269614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:22:02.269614+0000","last_clean_scrub_stamp":"2024-03-21T10:22:02.269614+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T19:32:31.844066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"93'1","reported_seq":32,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T10:23:27.642130+0000","last_change":"2024-03-21T10:21:57.851955+0000","last_active":"2024-03-21T10:23:27.642130+0000","last_peered":"2024-03-21T10:23:27.642130+0000","last_clean":"2024-03-21T10:23:27.642130+0000","last_became_active":"2024-03-21T10:21:57.851571+0000","last_became_peered":"2024-03-21T10:21:57.851571+0000","last_unstale":"2024-03-21T10:23:27.642130+0000","last_undegraded":"2024-03-21T10:23:27.642130+0000","last_fullsized":"2024-03-21T10:23:27.642130+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:21:56.788908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:21:56.788908+0000","last_clean_scrub_stamp":"2024-03-21T10:21:56.788908+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T17:23:40.441895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"95'4","reported_seq":33,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T10:23:56.014917+0000","last_change":"2024-03-21T10:21:58.856393+0000","last_active":"2024-03-21T10:23:56.014917+0000","last_peered":"2024-03-21T10:23:56.014917+0000","last_clean":"2024-03-21T10:23:56.014917+0000","last_became_active":"2024-03-21T10:21:58.855139+0000","last_became_peered":"2024-03-21T10:21:58.855139+0000","last_unstale":"2024-03-21T10:23:56.014917+0000","last_undegraded":"2024-03-21T10:23:56.014917+0000","last_fullsized":"2024-03-21T10:23:56.014917+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_clean_scrub_stamp":"2024-03-21T10:21:57.813712+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T14:24:55.663576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":22,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T10:23:27.742362+0000","last_change":"2024-03-21T10:22:03.315586+0000","last_active":"2024-03-21T10:23:27.742362+0000","last_peered":"2024-03-21T10:23:27.742362+0000","last_clean":"2024-03-21T10:23:27.742362+0000","last_became_active":"2024-03-21T10:22:03.315198+0000","last_became_peered":"2024-03-21T10:22:03.315198+0000","last_unstale":"2024-03-21T10:23:27.742362+0000","last_undegraded":"2024-03-21T10:23:27.742362+0000","last_fullsized":"2024-03-21T10:23:27.742362+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:22:02.269614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:22:02.269614+0000","last_clean_scrub_stamp":"2024-03-21T10:22:02.269614+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T14:25:13.534311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"93'5","reported_seq":126,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T10:23:56.799198+0000","last_change":"2024-03-21T10:21:57.862094+0000","last_active":"2024-03-21T10:23:56.799198+0000","last_peered":"2024-03-21T10:23:56.799198+0000","last_clean":"2024-03-21T10:23:56.799198+0000","last_became_active":"2024-03-21T10:21:57.860307+0000","last_became_peered":"2024-03-21T10:21:57.860307+0000","last_unstale":"2024-03-21T10:23:56.799198+0000","last_undegraded":"2024-03-21T10:23:56.799198+0000","last_fullsized":"2024-03-21T10:23:56.799198+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:21:56.788908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:21:56.788908+0000","last_clean_scrub_stamp":"2024-03-21T10:21:56.788908+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T12:58:50.701218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":90,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"95'7","reported_seq":36,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T10:23:56.024204+0000","last_change":"2024-03-21T10:21:58.870880+0000","last_active":"2024-03-21T10:23:56.024204+0000","last_peered":"2024-03-21T10:23:56.024204+0000","last_clean":"2024-03-21T10:23:56.024204+0000","last_became_active":"2024-03-21T10:21:58.869882+0000","last_became_peered":"2024-03-21T10:21:58.869882+0000","last_unstale":"2024-03-21T10:23:56.024204+0000","last_undegraded":"2024-03-21T10:23:56.024204+0000","last_fullsized":"2024-03-21T10:23:56.024204+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_clean_scrub_stamp":"2024-03-21T10:21:57.813712+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T18:01:42.613732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":22,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T10:23:27.742825+0000","last_change":"2024-03-21T10:22:03.634425+0000","last_active":"2024-03-21T10:23:27.742825+0000","last_peered":"2024-03-21T10:23:27.742825+0000","last_clean":"2024-03-21T10:23:27.742825+0000","last_became_active":"2024-03-21T10:22:03.632464+0000","last_became_peered":"2024-03-21T10:22:03.632464+0000","last_unstale":"2024-03-21T10:23:27.742825+0000","last_undegraded":"2024-03-21T10:23:27.742825+0000","last_fullsized":"2024-03-21T10:23:27.742825+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:22:02.269614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:22:02.269614+0000","last_clean_scrub_stamp":"2024-03-21T10:22:02.269614+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T16:03:28.270757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"95'8","reported_seq":19,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T10:23:56.006280+0000","last_change":"2024-03-21T10:23:27.649397+0000","last_active":"2024-03-21T10:23:56.006280+0000","last_peered":"2024-03-21T10:23:56.006280+0000","last_clean":"2024-03-21T10:23:56.006280+0000","last_became_active":"2024-03-21T10:23:27.649148+0000","last_became_peered":"2024-03-21T10:23:27.649148+0000","last_unstale":"2024-03-21T10:23:56.006280+0000","last_undegraded":"2024-03-21T10:23:56.006280+0000","last_fullsized":"2024-03-21T10:23:56.006280+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_clean_scrub_stamp":"2024-03-21T10:21:57.813712+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T19:28:03.830551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,8],"acting":[9,2,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.1","version":"95'6","reported_seq":129,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T10:23:57.825488+0000","last_change":"2024-03-21T10:21:57.856111+0000","last_active":"2024-03-21T10:23:57.825488+0000","last_peered":"2024-03-21T10:23:57.825488+0000","last_clean":"2024-03-21T10:23:57.825488+0000","last_became_active":"2024-03-21T10:21:57.855584+0000","last_became_peered":"2024-03-21T10:21:57.855584+0000","last_unstale":"2024-03-21T10:23:57.825488+0000","last_undegraded":"2024-03-21T10:23:57.825488+0000","last_fullsized":"2024-03-21T10:23:57.825488+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:21:56.788908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:21:56.788908+0000","last_clean_scrub_stamp":"2024-03-21T10:21:56.788908+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T17:53:48.208554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":92,"num_read_kb":92,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"93'1","reported_seq":23,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T10:23:27.645122+0000","last_change":"2024-03-21T10:22:03.318831+0000","last_active":"2024-03-21T10:23:27.645122+0000","last_peered":"2024-03-21T10:23:27.645122+0000","last_clean":"2024-03-21T10:23:27.645122+0000","last_became_active":"2024-03-21T10:22:03.318404+0000","last_became_peered":"2024-03-21T10:22:03.318404+0000","last_unstale":"2024-03-21T10:23:27.645122+0000","last_undegraded":"2024-03-21T10:23:27.645122+0000","last_fullsized":"2024-03-21T10:23:27.645122+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:22:02.269614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:22:02.269614+0000","last_clean_scrub_stamp":"2024-03-21T10:22:02.269614+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T12:06:52.043286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"95'8","reported_seq":37,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T10:23:56.038216+0000","last_change":"2024-03-21T10:21:58.856469+0000","last_active":"2024-03-21T10:23:56.038216+0000","last_peered":"2024-03-21T10:23:56.038216+0000","last_clean":"2024-03-21T10:23:56.038216+0000","last_became_active":"2024-03-21T10:21:58.856032+0000","last_became_peered":"2024-03-21T10:21:58.856032+0000","last_unstale":"2024-03-21T10:23:56.038216+0000","last_undegraded":"2024-03-21T10:23:56.038216+0000","last_fullsized":"2024-03-21T10:23:56.038216+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_clean_scrub_stamp":"2024-03-21T10:21:57.813712+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T21:19:59.302418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"93'6","reported_seq":40,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T10:23:27.640543+0000","last_change":"2024-03-21T10:21:57.854210+0000","last_active":"2024-03-21T10:23:27.640543+0000","last_peered":"2024-03-21T10:23:27.640543+0000","last_clean":"2024-03-21T10:23:27.640543+0000","last_became_active":"2024-03-21T10:21:57.853787+0000","last_became_peered":"2024-03-21T10:21:57.853787+0000","last_unstale":"2024-03-21T10:23:27.640543+0000","last_undegraded":"2024-03-21T10:23:27.640543+0000","last_fullsized":"2024-03-21T10:23:27.640543+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:21:56.788908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:21:56.788908+0000","last_clean_scrub_stamp":"2024-03-21T10:21:56.788908+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T12:10:42.926436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"23'137","reported_seq":366,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T10:23:27.643575+0000","last_change":"2024-03-21T10:20:21.585339+0000","last_active":"2024-03-21T10:23:27.643575+0000","last_peered":"2024-03-21T10:23:27.643575+0000","last_clean":"2024-03-21T10:23:27.643575+0000","last_became_active":"2024-03-21T10:20:21.583999+0000","last_became_peered":"2024-03-21T10:20:21.583999+0000","last_unstale":"2024-03-21T10:23:27.643575+0000","last_undegraded":"2024-03-21T10:23:27.643575+0000","last_fullsized":"2024-03-21T10:23:27.643575+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:15:40.696168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:15:40.696168+0000","last_clean_scrub_stamp":"2024-03-21T10:15:40.696168+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T15:31:54.603722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":22,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T10:23:27.752196+0000","last_change":"2024-03-21T10:22:03.634970+0000","last_active":"2024-03-21T10:23:27.752196+0000","last_peered":"2024-03-21T10:23:27.752196+0000","last_clean":"2024-03-21T10:23:27.752196+0000","last_became_active":"2024-03-21T10:22:03.633603+0000","last_became_peered":"2024-03-21T10:22:03.633603+0000","last_unstale":"2024-03-21T10:23:27.752196+0000","last_undegraded":"2024-03-21T10:23:27.752196+0000","last_fullsized":"2024-03-21T10:23:27.752196+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:22:02.269614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:22:02.269614+0000","last_clean_scrub_stamp":"2024-03-21T10:22:02.269614+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T12:13:02.041661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"95'5","reported_seq":34,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T10:23:55.994406+0000","last_change":"2024-03-21T10:21:58.861159+0000","last_active":"2024-03-21T10:23:55.994406+0000","last_peered":"2024-03-21T10:23:55.994406+0000","last_clean":"2024-03-21T10:23:55.994406+0000","last_became_active":"2024-03-21T10:21:58.860690+0000","last_became_peered":"2024-03-21T10:21:58.860690+0000","last_unstale":"2024-03-21T10:23:55.994406+0000","last_undegraded":"2024-03-21T10:23:55.994406+0000","last_fullsized":"2024-03-21T10:23:55.994406+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_clean_scrub_stamp":"2024-03-21T10:21:57.813712+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T16:21:53.632997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"93'4","reported_seq":36,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T10:23:27.643848+0000","last_change":"2024-03-21T10:21:57.863581+0000","last_active":"2024-03-21T10:23:27.643848+0000","last_peered":"2024-03-21T10:23:27.643848+0000","last_clean":"2024-03-21T10:23:27.643848+0000","last_became_active":"2024-03-21T10:21:57.863269+0000","last_became_peered":"2024-03-21T10:21:57.863269+0000","last_unstale":"2024-03-21T10:23:27.643848+0000","last_undegraded":"2024-03-21T10:23:27.643848+0000","last_fullsized":"2024-03-21T10:23:27.643848+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:21:56.788908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:21:56.788908+0000","last_clean_scrub_stamp":"2024-03-21T10:21:56.788908+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T14:03:59.930570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":22,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T10:23:27.835628+0000","last_change":"2024-03-21T10:22:03.313515+0000","last_active":"2024-03-21T10:23:27.835628+0000","last_peered":"2024-03-21T10:23:27.835628+0000","last_clean":"2024-03-21T10:23:27.835628+0000","last_became_active":"2024-03-21T10:22:03.311862+0000","last_became_peered":"2024-03-21T10:22:03.311862+0000","last_unstale":"2024-03-21T10:23:27.835628+0000","last_undegraded":"2024-03-21T10:23:27.835628+0000","last_fullsized":"2024-03-21T10:23:27.835628+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:22:02.269614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:22:02.269614+0000","last_clean_scrub_stamp":"2024-03-21T10:22:02.269614+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T16:47:50.761205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"93'4","reported_seq":36,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T10:23:27.646295+0000","last_change":"2024-03-21T10:21:57.870913+0000","last_active":"2024-03-21T10:23:27.646295+0000","last_peered":"2024-03-21T10:23:27.646295+0000","last_clean":"2024-03-21T10:23:27.646295+0000","last_became_active":"2024-03-21T10:21:57.870578+0000","last_became_peered":"2024-03-21T10:21:57.870578+0000","last_unstale":"2024-03-21T10:23:27.646295+0000","last_undegraded":"2024-03-21T10:23:27.646295+0000","last_fullsized":"2024-03-21T10:23:27.646295+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:21:56.788908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:21:56.788908+0000","last_clean_scrub_stamp":"2024-03-21T10:21:56.788908+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T11:46:48.621501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"95'9","reported_seq":38,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T10:23:56.005540+0000","last_change":"2024-03-21T10:21:58.852247+0000","last_active":"2024-03-21T10:23:56.005540+0000","last_peered":"2024-03-21T10:23:56.005540+0000","last_clean":"2024-03-21T10:23:56.005540+0000","last_became_active":"2024-03-21T10:21:58.851976+0000","last_became_peered":"2024-03-21T10:21:58.851976+0000","last_unstale":"2024-03-21T10:23:56.005540+0000","last_undegraded":"2024-03-21T10:23:56.005540+0000","last_fullsized":"2024-03-21T10:23:56.005540+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_clean_scrub_stamp":"2024-03-21T10:21:57.813712+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T14:27:11.383741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":22,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T10:23:27.638208+0000","last_change":"2024-03-21T10:22:03.637671+0000","last_active":"2024-03-21T10:23:27.638208+0000","last_peered":"2024-03-21T10:23:27.638208+0000","last_clean":"2024-03-21T10:23:27.638208+0000","last_became_active":"2024-03-21T10:22:03.634265+0000","last_became_peered":"2024-03-21T10:22:03.634265+0000","last_unstale":"2024-03-21T10:23:27.638208+0000","last_undegraded":"2024-03-21T10:23:27.638208+0000","last_fullsized":"2024-03-21T10:23:27.638208+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:22:02.269614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:22:02.269614+0000","last_clean_scrub_stamp":"2024-03-21T10:22:02.269614+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T15:38:02.214178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"95'14","reported_seq":45,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T10:23:27.642433+0000","last_change":"2024-03-21T10:21:57.862027+0000","last_active":"2024-03-21T10:23:27.642433+0000","last_peered":"2024-03-21T10:23:27.642433+0000","last_clean":"2024-03-21T10:23:27.642433+0000","last_became_active":"2024-03-21T10:21:57.860097+0000","last_became_peered":"2024-03-21T10:21:57.860097+0000","last_unstale":"2024-03-21T10:23:27.642433+0000","last_undegraded":"2024-03-21T10:23:27.642433+0000","last_fullsized":"2024-03-21T10:23:27.642433+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:21:56.788908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:21:56.788908+0000","last_clean_scrub_stamp":"2024-03-21T10:21:56.788908+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T13:46:19.562452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":24,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"95'5","reported_seq":34,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T10:23:56.040652+0000","last_change":"2024-03-21T10:21:58.856082+0000","last_active":"2024-03-21T10:23:56.040652+0000","last_peered":"2024-03-21T10:23:56.040652+0000","last_clean":"2024-03-21T10:23:56.040652+0000","last_became_active":"2024-03-21T10:21:58.854172+0000","last_became_peered":"2024-03-21T10:21:58.854172+0000","last_unstale":"2024-03-21T10:23:56.040652+0000","last_undegraded":"2024-03-21T10:23:56.040652+0000","last_fullsized":"2024-03-21T10:23:56.040652+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_clean_scrub_stamp":"2024-03-21T10:21:57.813712+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T14:59:33.995716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":22,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T10:23:27.646875+0000","last_change":"2024-03-21T10:22:03.310205+0000","last_active":"2024-03-21T10:23:27.646875+0000","last_peered":"2024-03-21T10:23:27.646875+0000","last_clean":"2024-03-21T10:23:27.646875+0000","last_became_active":"2024-03-21T10:22:03.309039+0000","last_became_peered":"2024-03-21T10:22:03.309039+0000","last_unstale":"2024-03-21T10:23:27.646875+0000","last_undegraded":"2024-03-21T10:23:27.646875+0000","last_fullsized":"2024-03-21T10:23:27.646875+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:22:02.269614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:22:02.269614+0000","last_clean_scrub_stamp":"2024-03-21T10:22:02.269614+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T11:16:46.075260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"93'7","reported_seq":39,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T10:23:27.641734+0000","last_change":"2024-03-21T10:21:57.857890+0000","last_active":"2024-03-21T10:23:27.641734+0000","last_peered":"2024-03-21T10:23:27.641734+0000","last_clean":"2024-03-21T10:23:27.641734+0000","last_became_active":"2024-03-21T10:21:57.857492+0000","last_became_peered":"2024-03-21T10:21:57.857492+0000","last_unstale":"2024-03-21T10:23:27.641734+0000","last_undegraded":"2024-03-21T10:23:27.641734+0000","last_fullsized":"2024-03-21T10:23:27.641734+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:21:56.788908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:21:56.788908+0000","last_clean_scrub_stamp":"2024-03-21T10:21:56.788908+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T21:18:45.566843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"95'7","reported_seq":42,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T10:23:56.040857+0000","last_change":"2024-03-21T10:22:27.219339+0000","last_active":"2024-03-21T10:23:56.040857+0000","last_peered":"2024-03-21T10:23:56.040857+0000","last_clean":"2024-03-21T10:23:56.040857+0000","last_became_active":"2024-03-21T10:22:27.218604+0000","last_became_peered":"2024-03-21T10:22:27.218604+0000","last_unstale":"2024-03-21T10:23:56.040857+0000","last_undegraded":"2024-03-21T10:23:56.040857+0000","last_fullsized":"2024-03-21T10:23:56.040857+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_clean_scrub_stamp":"2024-03-21T10:21:57.813712+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T15:58:53.161767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,4],"acting":[0,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":22,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T10:23:27.641245+0000","last_change":"2024-03-21T10:22:03.633789+0000","last_active":"2024-03-21T10:23:27.641245+0000","last_peered":"2024-03-21T10:23:27.641245+0000","last_clean":"2024-03-21T10:23:27.641245+0000","last_became_active":"2024-03-21T10:22:03.632716+0000","last_became_peered":"2024-03-21T10:22:03.632716+0000","last_unstale":"2024-03-21T10:23:27.641245+0000","last_undegraded":"2024-03-21T10:23:27.641245+0000","last_fullsized":"2024-03-21T10:23:27.641245+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:22:02.269614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:22:02.269614+0000","last_clean_scrub_stamp":"2024-03-21T10:22:02.269614+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T14:08:47.301881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"91'2","reported_seq":33,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T10:23:27.833086+0000","last_change":"2024-03-21T10:21:57.862413+0000","last_active":"2024-03-21T10:23:27.833086+0000","last_peered":"2024-03-21T10:23:27.833086+0000","last_clean":"2024-03-21T10:23:27.833086+0000","last_became_active":"2024-03-21T10:21:57.862104+0000","last_became_peered":"2024-03-21T10:21:57.862104+0000","last_unstale":"2024-03-21T10:23:27.833086+0000","last_undegraded":"2024-03-21T10:23:27.833086+0000","last_fullsized":"2024-03-21T10:23:27.833086+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:21:56.788908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:21:56.788908+0000","last_clean_scrub_stamp":"2024-03-21T10:21:56.788908+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T17:49:21.729492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"95'2","reported_seq":31,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T10:23:56.004636+0000","last_change":"2024-03-21T10:21:58.864688+0000","last_active":"2024-03-21T10:23:56.004636+0000","last_peered":"2024-03-21T10:23:56.004636+0000","last_clean":"2024-03-21T10:23:56.004636+0000","last_became_active":"2024-03-21T10:21:58.862821+0000","last_became_peered":"2024-03-21T10:21:58.862821+0000","last_unstale":"2024-03-21T10:23:56.004636+0000","last_undegraded":"2024-03-21T10:23:56.004636+0000","last_fullsized":"2024-03-21T10:23:56.004636+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_clean_scrub_stamp":"2024-03-21T10:21:57.813712+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T11:17:09.650649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T10:23:27.756711+0000","last_change":"2024-03-21T10:23:27.756711+0000","last_active":"2024-03-21T10:23:27.756711+0000","last_peered":"2024-03-21T10:23:27.756711+0000","last_clean":"2024-03-21T10:23:27.756711+0000","last_became_active":"2024-03-21T10:23:27.754993+0000","last_became_peered":"2024-03-21T10:23:27.754993+0000","last_unstale":"2024-03-21T10:23:27.756711+0000","last_undegraded":"2024-03-21T10:23:27.756711+0000","last_fullsized":"2024-03-21T10:23:27.756711+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:22:02.269614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:22:02.269614+0000","last_clean_scrub_stamp":"2024-03-21T10:22:02.269614+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T11:22:51.851047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,2,5],"acting":[8,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"93'7","reported_seq":38,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T10:23:27.638581+0000","last_change":"2024-03-21T10:21:57.868033+0000","last_active":"2024-03-21T10:23:27.638581+0000","last_peered":"2024-03-21T10:23:27.638581+0000","last_clean":"2024-03-21T10:23:27.638581+0000","last_became_active":"2024-03-21T10:21:57.867279+0000","last_became_peered":"2024-03-21T10:21:57.867279+0000","last_unstale":"2024-03-21T10:23:27.638581+0000","last_undegraded":"2024-03-21T10:23:27.638581+0000","last_fullsized":"2024-03-21T10:23:27.638581+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:21:56.788908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:21:56.788908+0000","last_clean_scrub_stamp":"2024-03-21T10:21:56.788908+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T20:05:37.701443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"95'9","reported_seq":38,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T10:23:56.022480+0000","last_change":"2024-03-21T10:21:58.868455+0000","last_active":"2024-03-21T10:23:56.022480+0000","last_peered":"2024-03-21T10:23:56.022480+0000","last_clean":"2024-03-21T10:23:56.022480+0000","last_became_active":"2024-03-21T10:21:58.867786+0000","last_became_peered":"2024-03-21T10:21:58.867786+0000","last_unstale":"2024-03-21T10:23:56.022480+0000","last_undegraded":"2024-03-21T10:23:56.022480+0000","last_fullsized":"2024-03-21T10:23:56.022480+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_clean_scrub_stamp":"2024-03-21T10:21:57.813712+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T19:16:12.258470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"93'1","reported_seq":34,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T10:23:27.753222+0000","last_change":"2024-03-21T10:23:27.753222+0000","last_active":"2024-03-21T10:23:27.753222+0000","last_peered":"2024-03-21T10:23:27.753222+0000","last_clean":"2024-03-21T10:23:27.753222+0000","last_became_active":"2024-03-21T10:23:27.746251+0000","last_became_peered":"2024-03-21T10:23:27.746251+0000","last_unstale":"2024-03-21T10:23:27.753222+0000","last_undegraded":"2024-03-21T10:23:27.753222+0000","last_fullsized":"2024-03-21T10:23:27.753222+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:22:02.269614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:22:02.269614+0000","last_clean_scrub_stamp":"2024-03-21T10:22:02.269614+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T13:42:00.362336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":117,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,7,8],"acting":[4,10,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"93'3","reported_seq":34,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T10:23:27.647528+0000","last_change":"2024-03-21T10:21:57.862748+0000","last_active":"2024-03-21T10:23:27.647528+0000","last_peered":"2024-03-21T10:23:27.647528+0000","last_clean":"2024-03-21T10:23:27.647528+0000","last_became_active":"2024-03-21T10:21:57.862350+0000","last_became_peered":"2024-03-21T10:21:57.862350+0000","last_unstale":"2024-03-21T10:23:27.647528+0000","last_undegraded":"2024-03-21T10:23:27.647528+0000","last_fullsized":"2024-03-21T10:23:27.647528+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:21:56.788908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:21:56.788908+0000","last_clean_scrub_stamp":"2024-03-21T10:21:56.788908+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T17:08:47.301758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"95'8","reported_seq":37,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T10:23:56.025683+0000","last_change":"2024-03-21T10:21:58.866340+0000","last_active":"2024-03-21T10:23:56.025683+0000","last_peered":"2024-03-21T10:23:56.025683+0000","last_clean":"2024-03-21T10:23:56.025683+0000","last_became_active":"2024-03-21T10:21:58.865965+0000","last_became_peered":"2024-03-21T10:21:58.865965+0000","last_unstale":"2024-03-21T10:23:56.025683+0000","last_undegraded":"2024-03-21T10:23:56.025683+0000","last_fullsized":"2024-03-21T10:23:56.025683+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_clean_scrub_stamp":"2024-03-21T10:21:57.813712+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T10:42:23.854176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":22,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T10:23:27.640253+0000","last_change":"2024-03-21T10:22:03.634108+0000","last_active":"2024-03-21T10:23:27.640253+0000","last_peered":"2024-03-21T10:23:27.640253+0000","last_clean":"2024-03-21T10:23:27.640253+0000","last_became_active":"2024-03-21T10:22:03.633344+0000","last_became_peered":"2024-03-21T10:22:03.633344+0000","last_unstale":"2024-03-21T10:23:27.640253+0000","last_undegraded":"2024-03-21T10:23:27.640253+0000","last_fullsized":"2024-03-21T10:23:27.640253+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:22:02.269614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:22:02.269614+0000","last_clean_scrub_stamp":"2024-03-21T10:22:02.269614+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T21:00:59.893994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"91'4","reported_seq":36,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T10:23:27.644057+0000","last_change":"2024-03-21T10:21:57.871520+0000","last_active":"2024-03-21T10:23:27.644057+0000","last_peered":"2024-03-21T10:23:27.644057+0000","last_clean":"2024-03-21T10:23:27.644057+0000","last_became_active":"2024-03-21T10:21:57.871127+0000","last_became_peered":"2024-03-21T10:21:57.871127+0000","last_unstale":"2024-03-21T10:23:27.644057+0000","last_undegraded":"2024-03-21T10:23:27.644057+0000","last_fullsized":"2024-03-21T10:23:27.644057+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:21:56.788908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:21:56.788908+0000","last_clean_scrub_stamp":"2024-03-21T10:21:56.788908+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T20:43:39.537307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"95'5","reported_seq":34,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T10:23:56.044600+0000","last_change":"2024-03-21T10:21:58.861786+0000","last_active":"2024-03-21T10:23:56.044600+0000","last_peered":"2024-03-21T10:23:56.044600+0000","last_clean":"2024-03-21T10:23:56.044600+0000","last_became_active":"2024-03-21T10:21:58.861428+0000","last_became_peered":"2024-03-21T10:21:58.861428+0000","last_unstale":"2024-03-21T10:23:56.044600+0000","last_undegraded":"2024-03-21T10:23:56.044600+0000","last_fullsized":"2024-03-21T10:23:56.044600+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_clean_scrub_stamp":"2024-03-21T10:21:57.813712+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T15:56:30.256266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":22,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T10:23:27.833676+0000","last_change":"2024-03-21T10:22:03.305085+0000","last_active":"2024-03-21T10:23:27.833676+0000","last_peered":"2024-03-21T10:23:27.833676+0000","last_clean":"2024-03-21T10:23:27.833676+0000","last_became_active":"2024-03-21T10:22:03.304474+0000","last_became_peered":"2024-03-21T10:22:03.304474+0000","last_unstale":"2024-03-21T10:23:27.833676+0000","last_undegraded":"2024-03-21T10:23:27.833676+0000","last_fullsized":"2024-03-21T10:23:27.833676+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:22:02.269614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:22:02.269614+0000","last_clean_scrub_stamp":"2024-03-21T10:22:02.269614+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T18:28:10.480981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"93'5","reported_seq":128,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T10:23:56.823374+0000","last_change":"2024-03-21T10:21:57.860657+0000","last_active":"2024-03-21T10:23:56.823374+0000","last_peered":"2024-03-21T10:23:56.823374+0000","last_clean":"2024-03-21T10:23:56.823374+0000","last_became_active":"2024-03-21T10:21:57.860340+0000","last_became_peered":"2024-03-21T10:21:57.860340+0000","last_unstale":"2024-03-21T10:23:56.823374+0000","last_undegraded":"2024-03-21T10:23:56.823374+0000","last_fullsized":"2024-03-21T10:23:56.823374+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:21:56.788908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:21:56.788908+0000","last_clean_scrub_stamp":"2024-03-21T10:21:56.788908+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T21:58:08.706812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1170,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":92,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"95'7","reported_seq":36,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T10:23:56.048680+0000","last_change":"2024-03-21T10:21:58.874254+0000","last_active":"2024-03-21T10:23:56.048680+0000","last_peered":"2024-03-21T10:23:56.048680+0000","last_clean":"2024-03-21T10:23:56.048680+0000","last_became_active":"2024-03-21T10:21:58.869795+0000","last_became_peered":"2024-03-21T10:21:58.869795+0000","last_unstale":"2024-03-21T10:23:56.048680+0000","last_undegraded":"2024-03-21T10:23:56.048680+0000","last_fullsized":"2024-03-21T10:23:56.048680+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_clean_scrub_stamp":"2024-03-21T10:21:57.813712+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T19:51:28.296199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":22,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T10:23:27.834249+0000","last_change":"2024-03-21T10:22:03.306761+0000","last_active":"2024-03-21T10:23:27.834249+0000","last_peered":"2024-03-21T10:23:27.834249+0000","last_clean":"2024-03-21T10:23:27.834249+0000","last_became_active":"2024-03-21T10:22:03.305106+0000","last_became_peered":"2024-03-21T10:22:03.305106+0000","last_unstale":"2024-03-21T10:23:27.834249+0000","last_undegraded":"2024-03-21T10:23:27.834249+0000","last_fullsized":"2024-03-21T10:23:27.834249+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:22:02.269614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:22:02.269614+0000","last_clean_scrub_stamp":"2024-03-21T10:22:02.269614+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T21:13:00.846169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"93'3","reported_seq":35,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T10:23:27.751970+0000","last_change":"2024-03-21T10:21:57.868012+0000","last_active":"2024-03-21T10:23:27.751970+0000","last_peered":"2024-03-21T10:23:27.751970+0000","last_clean":"2024-03-21T10:23:27.751970+0000","last_became_active":"2024-03-21T10:21:57.867682+0000","last_became_peered":"2024-03-21T10:21:57.867682+0000","last_unstale":"2024-03-21T10:23:27.751970+0000","last_undegraded":"2024-03-21T10:23:27.751970+0000","last_fullsized":"2024-03-21T10:23:27.751970+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:21:56.788908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:21:56.788908+0000","last_clean_scrub_stamp":"2024-03-21T10:21:56.788908+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T16:59:56.061167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"95'6","reported_seq":35,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T10:23:56.033828+0000","last_change":"2024-03-21T10:21:58.864949+0000","last_active":"2024-03-21T10:23:56.033828+0000","last_peered":"2024-03-21T10:23:56.033828+0000","last_clean":"2024-03-21T10:23:56.033828+0000","last_became_active":"2024-03-21T10:21:58.859796+0000","last_became_peered":"2024-03-21T10:21:58.859796+0000","last_unstale":"2024-03-21T10:23:56.033828+0000","last_undegraded":"2024-03-21T10:23:56.033828+0000","last_fullsized":"2024-03-21T10:23:56.033828+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_clean_scrub_stamp":"2024-03-21T10:21:57.813712+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T18:02:51.873103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":22,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T10:23:27.749977+0000","last_change":"2024-03-21T10:22:03.308451+0000","last_active":"2024-03-21T10:23:27.749977+0000","last_peered":"2024-03-21T10:23:27.749977+0000","last_clean":"2024-03-21T10:23:27.749977+0000","last_became_active":"2024-03-21T10:22:03.307986+0000","last_became_peered":"2024-03-21T10:22:03.307986+0000","last_unstale":"2024-03-21T10:23:27.749977+0000","last_undegraded":"2024-03-21T10:23:27.749977+0000","last_fullsized":"2024-03-21T10:23:27.749977+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:22:02.269614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:22:02.269614+0000","last_clean_scrub_stamp":"2024-03-21T10:22:02.269614+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T21:08:42.241657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"93'5","reported_seq":36,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T10:23:27.742859+0000","last_change":"2024-03-21T10:21:57.851717+0000","last_active":"2024-03-21T10:23:27.742859+0000","last_peered":"2024-03-21T10:23:27.742859+0000","last_clean":"2024-03-21T10:23:27.742859+0000","last_became_active":"2024-03-21T10:21:57.849909+0000","last_became_peered":"2024-03-21T10:21:57.849909+0000","last_unstale":"2024-03-21T10:23:27.742859+0000","last_undegraded":"2024-03-21T10:23:27.742859+0000","last_fullsized":"2024-03-21T10:23:27.742859+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:21:56.788908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:21:56.788908+0000","last_clean_scrub_stamp":"2024-03-21T10:21:56.788908+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T11:41:37.611400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"95'9","reported_seq":38,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T10:23:56.030074+0000","last_change":"2024-03-21T10:21:58.860892+0000","last_active":"2024-03-21T10:23:56.030074+0000","last_peered":"2024-03-21T10:23:56.030074+0000","last_clean":"2024-03-21T10:23:56.030074+0000","last_became_active":"2024-03-21T10:21:58.860403+0000","last_became_peered":"2024-03-21T10:21:58.860403+0000","last_unstale":"2024-03-21T10:23:56.030074+0000","last_undegraded":"2024-03-21T10:23:56.030074+0000","last_fullsized":"2024-03-21T10:23:56.030074+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_clean_scrub_stamp":"2024-03-21T10:21:57.813712+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T15:15:23.303864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"95'9","reported_seq":38,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T10:23:56.009937+0000","last_change":"2024-03-21T10:21:58.869275+0000","last_active":"2024-03-21T10:23:56.009937+0000","last_peered":"2024-03-21T10:23:56.009937+0000","last_clean":"2024-03-21T10:23:56.009937+0000","last_became_active":"2024-03-21T10:21:58.868834+0000","last_became_peered":"2024-03-21T10:21:58.868834+0000","last_unstale":"2024-03-21T10:23:56.009937+0000","last_undegraded":"2024-03-21T10:23:56.009937+0000","last_fullsized":"2024-03-21T10:23:56.009937+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_clean_scrub_stamp":"2024-03-21T10:21:57.813712+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T14:01:11.074702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"95'12","reported_seq":41,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T10:23:56.002885+0000","last_change":"2024-03-21T10:21:58.851658+0000","last_active":"2024-03-21T10:23:56.002885+0000","last_peered":"2024-03-21T10:23:56.002885+0000","last_clean":"2024-03-21T10:23:56.002885+0000","last_became_active":"2024-03-21T10:21:58.851295+0000","last_became_peered":"2024-03-21T10:21:58.851295+0000","last_unstale":"2024-03-21T10:23:56.002885+0000","last_undegraded":"2024-03-21T10:23:56.002885+0000","last_fullsized":"2024-03-21T10:23:56.002885+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_clean_scrub_stamp":"2024-03-21T10:21:57.813712+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T13:15:32.107783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"95'8","reported_seq":37,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T10:23:56.017364+0000","last_change":"2024-03-21T10:21:58.859275+0000","last_active":"2024-03-21T10:23:56.017364+0000","last_peered":"2024-03-21T10:23:56.017364+0000","last_clean":"2024-03-21T10:23:56.017364+0000","last_became_active":"2024-03-21T10:21:58.857054+0000","last_became_peered":"2024-03-21T10:21:58.857054+0000","last_unstale":"2024-03-21T10:23:56.017364+0000","last_undegraded":"2024-03-21T10:23:56.017364+0000","last_fullsized":"2024-03-21T10:23:56.017364+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_clean_scrub_stamp":"2024-03-21T10:21:57.813712+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T20:05:13.819966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"95'7","reported_seq":36,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T10:23:56.022147+0000","last_change":"2024-03-21T10:21:58.869025+0000","last_active":"2024-03-21T10:23:56.022147+0000","last_peered":"2024-03-21T10:23:56.022147+0000","last_clean":"2024-03-21T10:23:56.022147+0000","last_became_active":"2024-03-21T10:21:58.868240+0000","last_became_peered":"2024-03-21T10:21:58.868240+0000","last_unstale":"2024-03-21T10:23:56.022147+0000","last_undegraded":"2024-03-21T10:23:56.022147+0000","last_fullsized":"2024-03-21T10:23:56.022147+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_clean_scrub_stamp":"2024-03-21T10:21:57.813712+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T21:56:45.642916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"95'4","reported_seq":33,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T10:23:56.021559+0000","last_change":"2024-03-21T10:21:58.865024+0000","last_active":"2024-03-21T10:23:56.021559+0000","last_peered":"2024-03-21T10:23:56.021559+0000","last_clean":"2024-03-21T10:23:56.021559+0000","last_became_active":"2024-03-21T10:21:58.864649+0000","last_became_peered":"2024-03-21T10:21:58.864649+0000","last_unstale":"2024-03-21T10:23:56.021559+0000","last_undegraded":"2024-03-21T10:23:56.021559+0000","last_fullsized":"2024-03-21T10:23:56.021559+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_clean_scrub_stamp":"2024-03-21T10:21:57.813712+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T22:09:03.561283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"95'6","reported_seq":35,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T10:23:56.004522+0000","last_change":"2024-03-21T10:21:58.865206+0000","last_active":"2024-03-21T10:23:56.004522+0000","last_peered":"2024-03-21T10:23:56.004522+0000","last_clean":"2024-03-21T10:23:56.004522+0000","last_became_active":"2024-03-21T10:21:58.864669+0000","last_became_peered":"2024-03-21T10:21:58.864669+0000","last_unstale":"2024-03-21T10:23:56.004522+0000","last_undegraded":"2024-03-21T10:23:56.004522+0000","last_fullsized":"2024-03-21T10:23:56.004522+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_clean_scrub_stamp":"2024-03-21T10:21:57.813712+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T13:57:45.668192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"95'15","reported_seq":44,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T10:23:56.044446+0000","last_change":"2024-03-21T10:21:58.862664+0000","last_active":"2024-03-21T10:23:56.044446+0000","last_peered":"2024-03-21T10:23:56.044446+0000","last_clean":"2024-03-21T10:23:56.044446+0000","last_became_active":"2024-03-21T10:21:58.862144+0000","last_became_peered":"2024-03-21T10:21:58.862144+0000","last_unstale":"2024-03-21T10:23:56.044446+0000","last_undegraded":"2024-03-21T10:23:56.044446+0000","last_fullsized":"2024-03-21T10:23:56.044446+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_clean_scrub_stamp":"2024-03-21T10:21:57.813712+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T11:10:10.225102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"95'2","reported_seq":31,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T10:23:56.017561+0000","last_change":"2024-03-21T10:21:58.851000+0000","last_active":"2024-03-21T10:23:56.017561+0000","last_peered":"2024-03-21T10:23:56.017561+0000","last_clean":"2024-03-21T10:23:56.017561+0000","last_became_active":"2024-03-21T10:21:58.850563+0000","last_became_peered":"2024-03-21T10:21:58.850563+0000","last_unstale":"2024-03-21T10:23:56.017561+0000","last_undegraded":"2024-03-21T10:23:56.017561+0000","last_fullsized":"2024-03-21T10:23:56.017561+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_clean_scrub_stamp":"2024-03-21T10:21:57.813712+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T19:39:08.983713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"95'6","reported_seq":35,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T10:23:56.027350+0000","last_change":"2024-03-21T10:21:58.838426+0000","last_active":"2024-03-21T10:23:56.027350+0000","last_peered":"2024-03-21T10:23:56.027350+0000","last_clean":"2024-03-21T10:23:56.027350+0000","last_became_active":"2024-03-21T10:21:58.837625+0000","last_became_peered":"2024-03-21T10:21:58.837625+0000","last_unstale":"2024-03-21T10:23:56.027350+0000","last_undegraded":"2024-03-21T10:23:56.027350+0000","last_fullsized":"2024-03-21T10:23:56.027350+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_clean_scrub_stamp":"2024-03-21T10:21:57.813712+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T16:22:26.523685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"95'8","reported_seq":37,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T10:23:56.022370+0000","last_change":"2024-03-21T10:21:58.854114+0000","last_active":"2024-03-21T10:23:56.022370+0000","last_peered":"2024-03-21T10:23:56.022370+0000","last_clean":"2024-03-21T10:23:56.022370+0000","last_became_active":"2024-03-21T10:21:58.853686+0000","last_became_peered":"2024-03-21T10:21:58.853686+0000","last_unstale":"2024-03-21T10:23:56.022370+0000","last_undegraded":"2024-03-21T10:23:56.022370+0000","last_fullsized":"2024-03-21T10:23:56.022370+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_clean_scrub_stamp":"2024-03-21T10:21:57.813712+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T11:07:27.346766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"95'10","reported_seq":39,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T10:23:56.049452+0000","last_change":"2024-03-21T10:21:58.857778+0000","last_active":"2024-03-21T10:23:56.049452+0000","last_peered":"2024-03-21T10:23:56.049452+0000","last_clean":"2024-03-21T10:23:56.049452+0000","last_became_active":"2024-03-21T10:21:58.857336+0000","last_became_peered":"2024-03-21T10:21:58.857336+0000","last_unstale":"2024-03-21T10:23:56.049452+0000","last_undegraded":"2024-03-21T10:23:56.049452+0000","last_fullsized":"2024-03-21T10:23:56.049452+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_clean_scrub_stamp":"2024-03-21T10:21:57.813712+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T15:22:57.760341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"95'11","reported_seq":40,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T10:23:56.022623+0000","last_change":"2024-03-21T10:21:58.869774+0000","last_active":"2024-03-21T10:23:56.022623+0000","last_peered":"2024-03-21T10:23:56.022623+0000","last_clean":"2024-03-21T10:23:56.022623+0000","last_became_active":"2024-03-21T10:21:58.868815+0000","last_became_peered":"2024-03-21T10:21:58.868815+0000","last_unstale":"2024-03-21T10:23:56.022623+0000","last_undegraded":"2024-03-21T10:23:56.022623+0000","last_fullsized":"2024-03-21T10:23:56.022623+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_clean_scrub_stamp":"2024-03-21T10:21:57.813712+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T16:57:55.082127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"95'8","reported_seq":37,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T10:23:56.004175+0000","last_change":"2024-03-21T10:21:58.864746+0000","last_active":"2024-03-21T10:23:56.004175+0000","last_peered":"2024-03-21T10:23:56.004175+0000","last_clean":"2024-03-21T10:23:56.004175+0000","last_became_active":"2024-03-21T10:21:58.864366+0000","last_became_peered":"2024-03-21T10:21:58.864366+0000","last_unstale":"2024-03-21T10:23:56.004175+0000","last_undegraded":"2024-03-21T10:23:56.004175+0000","last_fullsized":"2024-03-21T10:23:56.004175+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_clean_scrub_stamp":"2024-03-21T10:21:57.813712+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T13:10:27.303843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":117,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":500,"ondisk_log_size":500,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4751360,"data_stored":4722944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":66995,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":937,"num_read_kb":542,"num_write":247,"num_write_kb":141,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":4464,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":565248,"data_stored":200985,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":214,"ondisk_log_size":214,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":82,"seq":352187318312,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36396,"kb_used_data":1060,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708756,"statfs":{"total":95995035648,"available":95957766144,"internally_reserved":0,"allocated":1085440,"data_stored":528251,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":36174276},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":10,"up_from":74,"seq":317827579951,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32912,"kb_used_data":1736,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712240,"statfs":{"total":95995035648,"available":95961333760,"internally_reserved":0,"allocated":1777664,"data_stored":1178616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[]},{"osd":9,"up_from":69,"seq":296352743478,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28096,"kb_used_data":1080,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717056,"statfs":{"total":95995035648,"available":95966265344,"internally_reserved":0,"allocated":1105920,"data_stored":527061,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":8,"up_from":61,"seq":261993005117,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36412,"kb_used_data":1076,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708740,"statfs":{"total":95995035648,"available":95957749760,"internally_reserved":0,"allocated":1101824,"data_stored":527439,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":7,"up_from":53,"seq":227633266757,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28672,"kb_used_data":1656,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716480,"statfs":{"total":95995035648,"available":95965675520,"internally_reserved":0,"allocated":1695744,"data_stored":1122957,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":10,"apply_latency_ms":10,"commit_latency_ns":10000000,"apply_latency_ns":10000000},"alerts":[]},{"osd":6,"up_from":48,"seq":206158430284,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36392,"kb_used_data":1056,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708760,"statfs":{"total":95995035648,"available":95957770240,"internally_reserved":0,"allocated":1081344,"data_stored":522565,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574957,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32296,"kb_used_data":1120,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712856,"statfs":{"total":95995035648,"available":95961964544,"internally_reserved":0,"allocated":1146880,"data_stored":583560,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":11,"apply_latency_ms":11,"commit_latency_ns":11000000,"apply_latency_ns":11000000},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738484,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28096,"kb_used_data":1080,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717056,"statfs":{"total":95995035648,"available":95966265344,"internally_reserved":0,"allocated":1105920,"data_stored":524021,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":14,"apply_latency_ms":14,"commit_latency_ns":14000000,"apply_latency_ns":14000000},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411432,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36400,"kb_used_data":1064,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708752,"statfs":{"total":95995035648,"available":95957762048,"internally_reserved":0,"allocated":1089536,"data_stored":523193,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":3,"up_from":27,"seq":115964117089,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36404,"kb_used_data":1068,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708748,"statfs":{"total":95995035648,"available":95957757952,"internally_reserved":0,"allocated":1093632,"data_stored":522273,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":17,"apply_latency_ms":17,"commit_latency_ns":17000000,"apply_latency_ns":17000000},"alerts":[]},{"osd":4,"up_from":35,"seq":150323855450,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28156,"kb_used_data":1140,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716996,"statfs":{"total":95995035648,"available":95966203904,"internally_reserved":0,"allocated":1167360,"data_stored":587834,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":40,"seq":171798691922,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":37004,"kb_used_data":1668,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708148,"statfs":{"total":95995035648,"available":95957143552,"internally_reserved":0,"allocated":1708032,"data_stored":1119127,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2178,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":61793,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1350,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":98304,"data_stored":61895,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":2820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":722,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":2554,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":1500,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":62309,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2312,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-21T10:24:01.100 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-03-21T10:24:01.100 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-03-21T10:24:01.100 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy 2024-03-21T10:24:01.101 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2edfee68ad7243bed422a049b6f5f58619300f11 shell --fsid 3056d51a-e76b-11ee-95cd-87774f69a715 -- ceph health --format=json 2024-03-21T10:24:01.590 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/3056d51a-e76b-11ee-95cd-87774f69a715/mon.a/config 2024-03-21T10:24:02.011 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:24:01 smithi047 ceph-mon[31547]: pgmap v432: 129 pgs: 129 active+clean; 642 KiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 426 B/s wr, 11 op/s 2024-03-21T10:24:02.012 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:24:01 smithi047 ceph-mon[31547]: from='client.15096 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-21T10:24:02.229 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:24:01 smithi100 ceph-mon[35193]: pgmap v432: 129 pgs: 129 active+clean; 642 KiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 426 B/s wr, 11 op/s 2024-03-21T10:24:02.230 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:24:01 smithi100 ceph-mon[35193]: from='client.15096 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-21T10:24:02.240 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:24:01 smithi157 ceph-mon[35581]: pgmap v432: 129 pgs: 129 active+clean; 642 KiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 426 B/s wr, 11 op/s 2024-03-21T10:24:02.240 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:24:01 smithi157 ceph-mon[35581]: from='client.15096 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-21T10:24:03.905 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:24:03 smithi047 ceph-mon[31547]: pgmap v433: 129 pgs: 129 active+clean; 642 KiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 426 B/s wr, 9 op/s 2024-03-21T10:24:03.944 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:03.942+0000 7f1861c00700 1 -- 172.21.15.47:0/1296457765 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f185c06a7e0 msgr2=0x7f185c10f020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:24:03.944 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:03.942+0000 7f1861c00700 1 --2- 172.21.15.47:0/1296457765 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f185c06a7e0 0x7f185c10f020 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f185400b0d0 tx=0x7f185400b3e0 comp rx=0 tx=0).stop 2024-03-21T10:24:03.944 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:03.942+0000 7f1861c00700 1 -- 172.21.15.47:0/1296457765 shutdown_connections 2024-03-21T10:24:03.944 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:03.942+0000 7f1861c00700 1 --2- 172.21.15.47:0/1296457765 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f185c06a7e0 0x7f185c10f020 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:24:03.944 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:03.942+0000 7f1861c00700 1 --2- 172.21.15.47:0/1296457765 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f185c1081e0 0x7f185c1085c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:24:03.944 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:03.942+0000 7f1861c00700 1 --2- 172.21.15.47:0/1296457765 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f185c069e40 0x7f185c06a2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:24:03.944 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:03.942+0000 7f1861c00700 1 -- 172.21.15.47:0/1296457765 >> 172.21.15.47:0/1296457765 conn(0x7f185c077fc0 msgr2=0x7f185c0783d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:24:03.944 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:03.942+0000 7f1861c00700 1 -- 172.21.15.47:0/1296457765 shutdown_connections 2024-03-21T10:24:03.944 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:03.942+0000 7f1861c00700 1 -- 172.21.15.47:0/1296457765 wait complete. 2024-03-21T10:24:03.945 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:03.944+0000 7f1861c00700 1 Processor -- start 2024-03-21T10:24:03.945 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:03.944+0000 7f1861c00700 1 -- start start 2024-03-21T10:24:03.946 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:03.945+0000 7f1861c00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f185c069e40 0x7f185c19e5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:24:03.946 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:03.945+0000 7f1861c00700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f185c06a7e0 0x7f185c19eb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:24:03.946 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:03.945+0000 7f1861c00700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f185c1081e0 0x7f185c1986e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:24:03.946 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:03.945+0000 7f1861c00700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f185c112360 con 0x7f185c06a7e0 2024-03-21T10:24:03.946 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:03.945+0000 7f1861c00700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f185c1121e0 con 0x7f185c1081e0 2024-03-21T10:24:03.946 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:03.945+0000 7f1861c00700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f185c1124e0 con 0x7f185c069e40 2024-03-21T10:24:03.947 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:03.945+0000 7f185a000700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f185c06a7e0 0x7f185c19eb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:24:03.947 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:03.945+0000 7f185a000700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f185c06a7e0 0x7f185c19eb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:42160/0 (socket says 172.21.15.47:42160) 2024-03-21T10:24:03.947 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:03.945+0000 7f185a000700 1 -- 172.21.15.47:0/1934878139 learned_addr learned my addr 172.21.15.47:0/1934878139 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:24:03.947 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:03.945+0000 7f185b400700 1 --2- 172.21.15.47:0/1934878139 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f185c1081e0 0x7f185c1986e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:24:03.947 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:03.945+0000 7f185aa00700 1 --2- 172.21.15.47:0/1934878139 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f185c069e40 0x7f185c19e5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:24:03.947 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:03.946+0000 7f185b400700 1 -- 172.21.15.47:0/1934878139 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f185c069e40 msgr2=0x7f185c19e5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:24:03.947 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:03.946+0000 7f185b400700 1 --2- 172.21.15.47:0/1934878139 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f185c069e40 0x7f185c19e5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:24:03.947 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:03.946+0000 7f185b400700 1 -- 172.21.15.47:0/1934878139 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f185c06a7e0 msgr2=0x7f185c19eb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:24:03.948 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:03.946+0000 7f185b400700 1 --2- 172.21.15.47:0/1934878139 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f185c06a7e0 0x7f185c19eb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:24:03.948 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:03.946+0000 7f185b400700 1 -- 172.21.15.47:0/1934878139 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f185c198fc0 con 0x7f185c1081e0 2024-03-21T10:24:03.948 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:03.946+0000 7f185b400700 1 --2- 172.21.15.47:0/1934878139 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f185c1081e0 0x7f185c1986e0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f185400b770 tx=0x7f18540079a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:24:03.948 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:03.946+0000 7f1853400700 1 -- 172.21.15.47:0/1934878139 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f185401f070 con 0x7f185c1081e0 2024-03-21T10:24:03.948 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:03.946+0000 7f1861c00700 1 -- 172.21.15.47:0/1934878139 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f185c199250 con 0x7f185c1081e0 2024-03-21T10:24:03.948 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:03.946+0000 7f1853400700 1 -- 172.21.15.47:0/1934878139 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f1854004c50 con 0x7f185c1081e0 2024-03-21T10:24:03.948 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:03.946+0000 7f1853400700 1 -- 172.21.15.47:0/1934878139 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1854005a60 con 0x7f185c1081e0 2024-03-21T10:24:03.948 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:03.947+0000 7f1861c00700 1 -- 172.21.15.47:0/1934878139 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f185c1a5370 con 0x7f185c1081e0 2024-03-21T10:24:03.949 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:03.948+0000 7f1861c00700 1 -- 172.21.15.47:0/1934878139 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f185c103890 con 0x7f185c1081e0 2024-03-21T10:24:03.953 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:03.952+0000 7f1853400700 1 -- 172.21.15.47:0/1934878139 <== mon.2 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 18) v1 ==== 137854+0+0 (secure 0 0 0) 0x7f1854022070 con 0x7f185c1081e0 2024-03-21T10:24:03.954 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:03.953+0000 7f1853400700 1 --2- 172.21.15.47:0/1934878139 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f18400a2fb0 0x7f18400a5470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:24:03.954 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:03.953+0000 7f1853400700 1 -- 172.21.15.47:0/1934878139 <== mon.2 v2:172.21.15.100:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9216+0+0 (secure 0 0 0) 0x7f18540d0650 con 0x7f185c1081e0 2024-03-21T10:24:03.954 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:03.953+0000 7f185aa00700 1 --2- 172.21.15.47:0/1934878139 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f18400a2fb0 0x7f18400a5470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:24:03.955 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:03.954+0000 7f185aa00700 1 --2- 172.21.15.47:0/1934878139 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f18400a2fb0 0x7f18400a5470 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f184c009be0 tx=0x7f184c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:24:03.956 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:03.955+0000 7f1853400700 1 -- 172.21.15.47:0/1934878139 <== mon.2 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f185409d100 con 0x7f185c1081e0 2024-03-21T10:24:04.229 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:24:03 smithi100 ceph-mon[35193]: pgmap v433: 129 pgs: 129 active+clean; 642 KiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 426 B/s wr, 9 op/s 2024-03-21T10:24:04.240 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:24:03 smithi157 ceph-mon[35581]: pgmap v433: 129 pgs: 129 active+clean; 642 KiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 426 B/s wr, 9 op/s 2024-03-21T10:24:04.266 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:04.264+0000 7f1861c00700 1 -- 172.21.15.47:0/1934878139 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7f185c02da30 con 0x7f185c1081e0 2024-03-21T10:24:04.266 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:04.265+0000 7f1853400700 1 -- 172.21.15.47:0/1934878139 <== mon.2 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+46 (secure 0 0 0) 0x7f185409ca00 con 0x7f185c1081e0 2024-03-21T10:24:04.266 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-21T10:24:04.266 INFO:teuthology.orchestra.run.smithi047.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-03-21T10:24:04.269 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:04.268+0000 7f1861c00700 1 -- 172.21.15.47:0/1934878139 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f18400a2fb0 msgr2=0x7f18400a5470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:24:04.269 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:04.268+0000 7f1861c00700 1 --2- 172.21.15.47:0/1934878139 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f18400a2fb0 0x7f18400a5470 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f184c009be0 tx=0x7f184c009260 comp rx=0 tx=0).stop 2024-03-21T10:24:04.269 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:04.268+0000 7f1861c00700 1 -- 172.21.15.47:0/1934878139 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f185c1081e0 msgr2=0x7f185c1986e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:24:04.270 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:04.268+0000 7f1861c00700 1 --2- 172.21.15.47:0/1934878139 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f185c1081e0 0x7f185c1986e0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f185400b770 tx=0x7f18540079a0 comp rx=0 tx=0).stop 2024-03-21T10:24:04.270 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:04.268+0000 7f1861c00700 1 -- 172.21.15.47:0/1934878139 shutdown_connections 2024-03-21T10:24:04.270 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:04.268+0000 7f1861c00700 1 --2- 172.21.15.47:0/1934878139 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f18400a2fb0 0x7f18400a5470 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:24:04.270 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:04.268+0000 7f1861c00700 1 --2- 172.21.15.47:0/1934878139 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f185c1081e0 0x7f185c1986e0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:24:04.270 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:04.268+0000 7f1861c00700 1 --2- 172.21.15.47:0/1934878139 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f185c069e40 0x7f185c19e5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:24:04.270 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:04.268+0000 7f1861c00700 1 --2- 172.21.15.47:0/1934878139 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f185c06a7e0 0x7f185c19eb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:24:04.270 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:04.268+0000 7f1861c00700 1 -- 172.21.15.47:0/1934878139 >> 172.21.15.47:0/1934878139 conn(0x7f185c077fc0 msgr2=0x7f185c0ffd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:24:04.270 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:04.268+0000 7f1861c00700 1 -- 172.21.15.47:0/1934878139 shutdown_connections 2024-03-21T10:24:04.270 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:04.268+0000 7f1861c00700 1 -- 172.21.15.47:0/1934878139 wait complete. 2024-03-21T10:24:04.964 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:24:04 smithi047 ceph-mon[31547]: from='client.? 172.21.15.47:0/1934878139' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-03-21T10:24:05.187 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy done 2024-03-21T10:24:05.187 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-21T10:24:05.229 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:24:04 smithi100 ceph-mon[35193]: from='client.? 172.21.15.47:0/1934878139' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-03-21T10:24:05.240 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:24:04 smithi157 ceph-mon[35581]: from='client.? 172.21.15.47:0/1934878139' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-03-21T10:24:05.427 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:05.425+0000 7fe3b5400640 1 -- 172.21.15.47:0/3541231051 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe3b01016c0 msgr2=0x7fe3b0101ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:24:05.428 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:05.425+0000 7fe3b5400640 1 --2- 172.21.15.47:0/3541231051 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe3b01016c0 0x7fe3b0101ae0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7fe3a0009900 tx=0x7fe3a002bc30 comp rx=0 tx=0).stop 2024-03-21T10:24:05.428 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:05.425+0000 7fe3b5400640 1 -- 172.21.15.47:0/3541231051 shutdown_connections 2024-03-21T10:24:05.428 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:05.425+0000 7fe3b5400640 1 --2- 172.21.15.47:0/3541231051 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe3b0103270 0x7fe3b0109b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:24:05.428 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:05.425+0000 7fe3b5400640 1 --2- 172.21.15.47:0/3541231051 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe3b01028b0 0x7fe3b0102d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:24:05.428 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:05.425+0000 7fe3b5400640 1 --2- 172.21.15.47:0/3541231051 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe3b01016c0 0x7fe3b0101ae0 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:24:05.428 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:05.425+0000 7fe3b5400640 1 -- 172.21.15.47:0/3541231051 >> 172.21.15.47:0/3541231051 conn(0x7fe3b00fcc50 msgr2=0x7fe3b00ff070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:24:05.428 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:05.425+0000 7fe3b5400640 1 -- 172.21.15.47:0/3541231051 shutdown_connections 2024-03-21T10:24:05.428 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:05.425+0000 7fe3b5400640 1 -- 172.21.15.47:0/3541231051 wait complete. 2024-03-21T10:24:05.430 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:05.426+0000 7fe3b5400640 1 Processor -- start 2024-03-21T10:24:05.430 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:05.426+0000 7fe3b5400640 1 -- start start 2024-03-21T10:24:05.430 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:05.426+0000 7fe3b5400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe3b01016c0 0x7fe3b0199b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:24:05.430 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:05.426+0000 7fe3b5400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe3b01028b0 0x7fe3b019a0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:24:05.431 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:05.426+0000 7fe3b5400640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe3b0103270 0x7fe3b01a1120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:24:05.431 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:05.426+0000 7fe3b5400640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3b010d1f0 con 0x7fe3b01016c0 2024-03-21T10:24:05.431 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:05.426+0000 7fe3b5400640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3b010d070 con 0x7fe3b01028b0 2024-03-21T10:24:05.431 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:05.426+0000 7fe3b5400640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3b010d370 con 0x7fe3b0103270 2024-03-21T10:24:05.431 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:05.427+0000 7fe3ae800640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe3b01016c0 0x7fe3b0199b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:24:05.431 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:05.427+0000 7fe3ae800640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe3b01016c0 0x7fe3b0199b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:42176/0 (socket says 172.21.15.47:42176) 2024-03-21T10:24:05.431 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:05.427+0000 7fe3ae800640 1 -- 172.21.15.47:0/3180768636 learned_addr learned my addr 172.21.15.47:0/3180768636 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:24:05.431 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:05.427+0000 7fe3a7e00640 1 --2- 172.21.15.47:0/3180768636 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe3b01028b0 0x7fe3b019a0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:24:05.431 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:05.427+0000 7fe3af200640 1 --2- 172.21.15.47:0/3180768636 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe3b0103270 0x7fe3b01a1120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:24:05.431 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:05.427+0000 7fe3ae800640 1 -- 172.21.15.47:0/3180768636 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe3b0103270 msgr2=0x7fe3b01a1120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:24:05.431 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:05.427+0000 7fe3ae800640 1 --2- 172.21.15.47:0/3180768636 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe3b0103270 0x7fe3b01a1120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:24:05.431 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:05.427+0000 7fe3ae800640 1 -- 172.21.15.47:0/3180768636 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe3b01028b0 msgr2=0x7fe3b019a0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:24:05.431 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:05.427+0000 7fe3ae800640 1 --2- 172.21.15.47:0/3180768636 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe3b01028b0 0x7fe3b019a0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:24:05.432 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:05.427+0000 7fe3ae800640 1 -- 172.21.15.47:0/3180768636 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3b01a1790 con 0x7fe3b01016c0 2024-03-21T10:24:05.432 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:05.427+0000 7fe3ae800640 1 --2- 172.21.15.47:0/3180768636 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe3b01016c0 0x7fe3b0199b70 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7fe3a0000c00 tx=0x7fe3a0004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:24:05.432 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:05.428+0000 7fe3a7400640 1 -- 172.21.15.47:0/3180768636 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3a0043020 con 0x7fe3b01016c0 2024-03-21T10:24:05.432 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:05.428+0000 7fe3a7400640 1 -- 172.21.15.47:0/3180768636 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fe3a0004440 con 0x7fe3b01016c0 2024-03-21T10:24:05.432 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:05.428+0000 7fe3a7400640 1 -- 172.21.15.47:0/3180768636 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3a0003c90 con 0x7fe3b01016c0 2024-03-21T10:24:05.432 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:05.428+0000 7fe3b5400640 1 -- 172.21.15.47:0/3180768636 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe3b01a1a20 con 0x7fe3b01016c0 2024-03-21T10:24:05.432 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:05.428+0000 7fe3b5400640 1 -- 172.21.15.47:0/3180768636 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe3b01a1f20 con 0x7fe3b01016c0 2024-03-21T10:24:05.432 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:05.429+0000 7fe3b5400640 1 -- 172.21.15.47:0/3180768636 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe3b01a20f0 con 0x7fe3b01016c0 2024-03-21T10:24:05.433 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:05.431+0000 7fe3a7400640 1 -- 172.21.15.47:0/3180768636 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 18) v1 ==== 137854+0+0 (secure 0 0 0) 0x7fe3a0046020 con 0x7fe3b01016c0 2024-03-21T10:24:05.435 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:05.432+0000 7fe3a7400640 1 --2- 172.21.15.47:0/3180768636 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7fe3940a2f20 0x7fe3940a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:24:05.435 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:05.433+0000 7fe3a7400640 1 -- 172.21.15.47:0/3180768636 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9216+0+0 (secure 0 0 0) 0x7fe3a00f1a80 con 0x7fe3b01016c0 2024-03-21T10:24:05.435 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:05.433+0000 7fe3a7e00640 1 --2- 172.21.15.47:0/3180768636 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7fe3940a2f20 0x7fe3940a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:24:05.435 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:05.434+0000 7fe3a7e00640 1 --2- 172.21.15.47:0/3180768636 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7fe3940a2f20 0x7fe3940a53e0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fe3b019b080 tx=0x7fe39800b390 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:24:05.439 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:05.438+0000 7fe3a7400640 1 -- 172.21.15.47:0/3180768636 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fe3a00f3e20 con 0x7fe3b01016c0 2024-03-21T10:24:05.715 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:05.713+0000 7fe3b5400640 1 -- 172.21.15.47:0/3180768636 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fe3b01a2480 con 0x7fe3b01016c0 2024-03-21T10:24:05.717 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:05.715+0000 7fe3a7400640 1 -- 172.21.15.47:0/3180768636 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v95) v1 ==== 74+0+24849 (secure 0 0 0) 0x7fe3a004d020 con 0x7fe3b01016c0 2024-03-21T10:24:05.717 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-21T10:24:05.718 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":95,"fsid":"3056d51a-e76b-11ee-95cd-87774f69a715","created":"2024-03-21T10:11:17.019965+0000","modified":"2024-03-21T10:23:27.619804+0000","last_up_change":"2024-03-21T10:20:52.675328+0000","last_in_change":"2024-03-21T10:20:27.648212+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-21T10:15:40.553114+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-21T10:21:56.399518+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-21T10:21:57.812940+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-21T10:22:01.822568+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"91","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"b0c7a102-8d89-428b-b793-cbbe67151f50","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6802","nonce":775837313},{"type":"v1","addr":"172.21.15.47:6803","nonce":775837313}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6804","nonce":775837313},{"type":"v1","addr":"172.21.15.47:6805","nonce":775837313}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6808","nonce":775837313},{"type":"v1","addr":"172.21.15.47:6809","nonce":775837313}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6806","nonce":775837313},{"type":"v1","addr":"172.21.15.47:6807","nonce":775837313}]},"public_addr":"172.21.15.47:6803/775837313","cluster_addr":"172.21.15.47:6805/775837313","heartbeat_back_addr":"172.21.15.47:6809/775837313","heartbeat_front_addr":"172.21.15.47:6807/775837313","state":["exists","up"]},{"osd":1,"uuid":"07bb81bb-7cf3-4bbc-ac9e-802f95185d11","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6800","nonce":2923304710},{"type":"v1","addr":"172.21.15.100:6801","nonce":2923304710}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":2923304710},{"type":"v1","addr":"172.21.15.100:6803","nonce":2923304710}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":2923304710},{"type":"v1","addr":"172.21.15.100:6807","nonce":2923304710}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6804","nonce":2923304710},{"type":"v1","addr":"172.21.15.100:6805","nonce":2923304710}]},"public_addr":"172.21.15.100:6801/2923304710","cluster_addr":"172.21.15.100:6803/2923304710","heartbeat_back_addr":"172.21.15.100:6807/2923304710","heartbeat_front_addr":"172.21.15.100:6805/2923304710","state":["exists","up"]},{"osd":2,"uuid":"88ce76b9-6081-4836-aad0-8c249c56e8a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":171377005},{"type":"v1","addr":"172.21.15.157:6801","nonce":171377005}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":171377005},{"type":"v1","addr":"172.21.15.157:6803","nonce":171377005}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":171377005},{"type":"v1","addr":"172.21.15.157:6807","nonce":171377005}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":171377005},{"type":"v1","addr":"172.21.15.157:6805","nonce":171377005}]},"public_addr":"172.21.15.157:6801/171377005","cluster_addr":"172.21.15.157:6803/171377005","heartbeat_back_addr":"172.21.15.157:6807/171377005","heartbeat_front_addr":"172.21.15.157:6805/171377005","state":["exists","up"]},{"osd":3,"uuid":"518aa33d-263f-45e6-aab5-2201ceb40e74","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":27,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6810","nonce":640760101},{"type":"v1","addr":"172.21.15.47:6811","nonce":640760101}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6812","nonce":640760101},{"type":"v1","addr":"172.21.15.47:6813","nonce":640760101}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6816","nonce":640760101},{"type":"v1","addr":"172.21.15.47:6817","nonce":640760101}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6814","nonce":640760101},{"type":"v1","addr":"172.21.15.47:6815","nonce":640760101}]},"public_addr":"172.21.15.47:6811/640760101","cluster_addr":"172.21.15.47:6813/640760101","heartbeat_back_addr":"172.21.15.47:6817/640760101","heartbeat_front_addr":"172.21.15.47:6815/640760101","state":["exists","up"]},{"osd":4,"uuid":"987bde83-ecc6-42e5-bd56-c143b3de4226","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":35,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6808","nonce":2785478639},{"type":"v1","addr":"172.21.15.100:6809","nonce":2785478639}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6810","nonce":2785478639},{"type":"v1","addr":"172.21.15.100:6811","nonce":2785478639}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6814","nonce":2785478639},{"type":"v1","addr":"172.21.15.100:6815","nonce":2785478639}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":2785478639},{"type":"v1","addr":"172.21.15.100:6813","nonce":2785478639}]},"public_addr":"172.21.15.100:6809/2785478639","cluster_addr":"172.21.15.100:6811/2785478639","heartbeat_back_addr":"172.21.15.100:6815/2785478639","heartbeat_front_addr":"172.21.15.100:6813/2785478639","state":["exists","up"]},{"osd":5,"uuid":"012d2188-3a61-4a5f-8d01-2001c15ffbe2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":40,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":1525645410},{"type":"v1","addr":"172.21.15.157:6809","nonce":1525645410}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":1525645410},{"type":"v1","addr":"172.21.15.157:6811","nonce":1525645410}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":1525645410},{"type":"v1","addr":"172.21.15.157:6815","nonce":1525645410}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":1525645410},{"type":"v1","addr":"172.21.15.157:6813","nonce":1525645410}]},"public_addr":"172.21.15.157:6809/1525645410","cluster_addr":"172.21.15.157:6811/1525645410","heartbeat_back_addr":"172.21.15.157:6815/1525645410","heartbeat_front_addr":"172.21.15.157:6813/1525645410","state":["exists","up"]},{"osd":6,"uuid":"c12c14b4-85f0-4d88-937c-de4f556437de","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6818","nonce":2321642790},{"type":"v1","addr":"172.21.15.47:6819","nonce":2321642790}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6820","nonce":2321642790},{"type":"v1","addr":"172.21.15.47:6821","nonce":2321642790}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6824","nonce":2321642790},{"type":"v1","addr":"172.21.15.47:6825","nonce":2321642790}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6822","nonce":2321642790},{"type":"v1","addr":"172.21.15.47:6823","nonce":2321642790}]},"public_addr":"172.21.15.47:6819/2321642790","cluster_addr":"172.21.15.47:6821/2321642790","heartbeat_back_addr":"172.21.15.47:6825/2321642790","heartbeat_front_addr":"172.21.15.47:6823/2321642790","state":["exists","up"]},{"osd":7,"uuid":"3c87c6d0-1d1f-4294-9917-aaaad7dda824","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6816","nonce":4104727330},{"type":"v1","addr":"172.21.15.100:6817","nonce":4104727330}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":4104727330},{"type":"v1","addr":"172.21.15.100:6819","nonce":4104727330}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":4104727330},{"type":"v1","addr":"172.21.15.100:6823","nonce":4104727330}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6820","nonce":4104727330},{"type":"v1","addr":"172.21.15.100:6821","nonce":4104727330}]},"public_addr":"172.21.15.100:6817/4104727330","cluster_addr":"172.21.15.100:6819/4104727330","heartbeat_back_addr":"172.21.15.100:6823/4104727330","heartbeat_front_addr":"172.21.15.100:6821/4104727330","state":["exists","up"]},{"osd":8,"uuid":"502d65cf-53c8-4936-8de4-e11b2a6e1e18","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":61,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":4067715347},{"type":"v1","addr":"172.21.15.157:6817","nonce":4067715347}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":4067715347},{"type":"v1","addr":"172.21.15.157:6819","nonce":4067715347}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":4067715347},{"type":"v1","addr":"172.21.15.157:6823","nonce":4067715347}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":4067715347},{"type":"v1","addr":"172.21.15.157:6821","nonce":4067715347}]},"public_addr":"172.21.15.157:6817/4067715347","cluster_addr":"172.21.15.157:6819/4067715347","heartbeat_back_addr":"172.21.15.157:6823/4067715347","heartbeat_front_addr":"172.21.15.157:6821/4067715347","state":["exists","up"]},{"osd":9,"uuid":"5ea1f5be-8458-4c93-9e61-c2e867a514f3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6826","nonce":1982108738},{"type":"v1","addr":"172.21.15.47:6827","nonce":1982108738}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6828","nonce":1982108738},{"type":"v1","addr":"172.21.15.47:6829","nonce":1982108738}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6832","nonce":1982108738},{"type":"v1","addr":"172.21.15.47:6833","nonce":1982108738}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6830","nonce":1982108738},{"type":"v1","addr":"172.21.15.47:6831","nonce":1982108738}]},"public_addr":"172.21.15.47:6827/1982108738","cluster_addr":"172.21.15.47:6829/1982108738","heartbeat_back_addr":"172.21.15.47:6833/1982108738","heartbeat_front_addr":"172.21.15.47:6831/1982108738","state":["exists","up"]},{"osd":10,"uuid":"2da3da90-a0be-4a54-a9ff-b2d6c9b36870","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":74,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":396391028},{"type":"v1","addr":"172.21.15.100:6825","nonce":396391028}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":396391028},{"type":"v1","addr":"172.21.15.100:6827","nonce":396391028}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6830","nonce":396391028},{"type":"v1","addr":"172.21.15.100:6831","nonce":396391028}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":396391028},{"type":"v1","addr":"172.21.15.100:6829","nonce":396391028}]},"public_addr":"172.21.15.100:6825/396391028","cluster_addr":"172.21.15.100:6827/396391028","heartbeat_back_addr":"172.21.15.100:6831/396391028","heartbeat_front_addr":"172.21.15.100:6829/396391028","state":["exists","up"]},{"osd":11,"uuid":"11798d41-7f7e-4319-95b8-8d3d0946bde9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":82,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":2671029483},{"type":"v1","addr":"172.21.15.157:6825","nonce":2671029483}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":2671029483},{"type":"v1","addr":"172.21.15.157:6827","nonce":2671029483}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":2671029483},{"type":"v1","addr":"172.21.15.157:6831","nonce":2671029483}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":2671029483},{"type":"v1","addr":"172.21.15.157:6829","nonce":2671029483}]},"public_addr":"172.21.15.157:6825/2671029483","cluster_addr":"172.21.15.157:6827/2671029483","heartbeat_back_addr":"172.21.15.157:6831/2671029483","heartbeat_front_addr":"172.21.15.157:6829/2671029483","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T10:14:31.822399+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T10:15:04.175721+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T10:15:35.902595+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T10:16:12.088891+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T10:16:46.428704+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T10:17:20.158270+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T10:17:55.915253+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T10:18:30.064641+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T10:19:04.410004+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T10:19:40.605181+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T10:20:15.636605+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T10:20:50.151365+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.11","mappings":[{"from":7,"to":11}]},{"pgid":"2.14","mappings":[{"from":2,"to":4}]},{"pgid":"2.1a","mappings":[{"from":8,"to":1}]},{"pgid":"2.1b","mappings":[{"from":2,"to":4}]},{"pgid":"2.1d","mappings":[{"from":8,"to":11}]},{"pgid":"3.0","mappings":[{"from":4,"to":8},{"from":5,"to":9}]},{"pgid":"3.6","mappings":[{"from":3,"to":7}]},{"pgid":"3.1b","mappings":[{"from":4,"to":6}]},{"pgid":"3.33","mappings":[{"from":5,"to":9}]},{"pgid":"3.3c","mappings":[{"from":4,"to":9}]},{"pgid":"4.c","mappings":[{"from":9,"to":10}]},{"pgid":"4.f","mappings":[{"from":9,"to":8}]},{"pgid":"4.19","mappings":[{"from":9,"to":8}]},{"pgid":"4.1d","mappings":[{"from":5,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.47:0/356822431":"2024-03-22T10:12:25.844530+0000","172.21.15.47:0/1167490879":"2024-03-22T10:12:25.844530+0000","172.21.15.47:0/1540718138":"2024-03-22T10:12:25.844530+0000","172.21.15.47:6801/3836903972":"2024-03-22T10:12:25.844530+0000","172.21.15.47:0/235436602":"2024-03-22T10:11:50.301838+0000","172.21.15.47:0/2052815572":"2024-03-22T10:11:50.301838+0000","172.21.15.47:6800/3942658432":"2024-03-22T10:11:50.301838+0000","172.21.15.47:6801/3942658432":"2024-03-22T10:11:50.301838+0000","172.21.15.47:0/1419516257":"2024-03-22T10:11:50.301838+0000","172.21.15.47:6800/3836903972":"2024-03-22T10:12:25.844530+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-21T10:24:05.720 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:05.717+0000 7fe3b5400640 1 -- 172.21.15.47:0/3180768636 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7fe3940a2f20 msgr2=0x7fe3940a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:24:05.720 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:05.718+0000 7fe3b5400640 1 --2- 172.21.15.47:0/3180768636 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7fe3940a2f20 0x7fe3940a53e0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fe3b019b080 tx=0x7fe39800b390 comp rx=0 tx=0).stop 2024-03-21T10:24:05.720 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:05.718+0000 7fe3b5400640 1 -- 172.21.15.47:0/3180768636 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe3b01016c0 msgr2=0x7fe3b0199b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:24:05.720 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:05.718+0000 7fe3b5400640 1 --2- 172.21.15.47:0/3180768636 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe3b01016c0 0x7fe3b0199b70 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7fe3a0000c00 tx=0x7fe3a0004040 comp rx=0 tx=0).stop 2024-03-21T10:24:05.720 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:05.718+0000 7fe3b5400640 1 -- 172.21.15.47:0/3180768636 shutdown_connections 2024-03-21T10:24:05.720 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:05.718+0000 7fe3b5400640 1 --2- 172.21.15.47:0/3180768636 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7fe3940a2f20 0x7fe3940a53e0 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:24:05.720 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:05.718+0000 7fe3b5400640 1 --2- 172.21.15.47:0/3180768636 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe3b0103270 0x7fe3b01a1120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:24:05.721 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:05.718+0000 7fe3b5400640 1 --2- 172.21.15.47:0/3180768636 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe3b01028b0 0x7fe3b019a0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:24:05.721 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:05.718+0000 7fe3b5400640 1 --2- 172.21.15.47:0/3180768636 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe3b01016c0 0x7fe3b0199b70 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:24:05.721 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:05.718+0000 7fe3b5400640 1 -- 172.21.15.47:0/3180768636 >> 172.21.15.47:0/3180768636 conn(0x7fe3b00fcc50 msgr2=0x7fe3b00738d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:24:05.721 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:05.718+0000 7fe3b5400640 1 -- 172.21.15.47:0/3180768636 shutdown_connections 2024-03-21T10:24:05.721 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:05.718+0000 7fe3b5400640 1 -- 172.21.15.47:0/3180768636 wait complete. 2024-03-21T10:24:05.739 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-21T10:15:40.553114+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-21T10:21:56.399518+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '86', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-21T10:21:57.812940+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '87', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-03-21T10:22:01.822568+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '91', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-03-21T10:24:05.739 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-21T10:24:05.998 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:05.996+0000 7f390ca00640 1 -- 172.21.15.47:0/2414500222 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3908076060 msgr2=0x7f3908076480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:24:05.998 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:05.996+0000 7f390ca00640 1 --2- 172.21.15.47:0/2414500222 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3908076060 0x7f3908076480 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f38f8009ca0 tx=0x7f38f802c0f0 comp rx=0 tx=0).stop 2024-03-21T10:24:05.998 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:05.996+0000 7f390ca00640 1 -- 172.21.15.47:0/2414500222 shutdown_connections 2024-03-21T10:24:05.999 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:05.996+0000 7f390ca00640 1 --2- 172.21.15.47:0/2414500222 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3908077c40 0x7f39080784e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:24:05.999 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:05.996+0000 7f390ca00640 1 --2- 172.21.15.47:0/2414500222 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f39080772a0 0x7f3908077700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:24:05.999 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:05.996+0000 7f390ca00640 1 --2- 172.21.15.47:0/2414500222 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3908076060 0x7f3908076480 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:24:05.999 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:05.996+0000 7f390ca00640 1 -- 172.21.15.47:0/2414500222 >> 172.21.15.47:0/2414500222 conn(0x7f39080fed10 msgr2=0x7f3908101150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:24:05.999 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:05.996+0000 7f390ca00640 1 -- 172.21.15.47:0/2414500222 shutdown_connections 2024-03-21T10:24:05.999 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:05.996+0000 7f390ca00640 1 -- 172.21.15.47:0/2414500222 wait complete. 2024-03-21T10:24:05.999 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:05.997+0000 7f390ca00640 1 Processor -- start 2024-03-21T10:24:06.000 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:05.998+0000 7f390ca00640 1 -- start start 2024-03-21T10:24:06.001 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:05.998+0000 7f390ca00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3908076060 0x7f390819de60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:24:06.001 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:05.998+0000 7f390ca00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f39080772a0 0x7f390819e3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:24:06.001 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:05.998+0000 7f390ca00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3908077c40 0x7f39081a53c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:24:06.001 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:05.998+0000 7f390ca00640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39081136e0 con 0x7f3908076060 2024-03-21T10:24:06.001 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:05.998+0000 7f390ca00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3908113560 con 0x7f3908077c40 2024-03-21T10:24:06.001 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:05.998+0000 7f390ca00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3908113860 con 0x7f39080772a0 2024-03-21T10:24:06.001 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:05.998+0000 7f3907400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3908076060 0x7f390819de60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:24:06.001 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:05.998+0000 7f3907400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3908076060 0x7f390819de60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:42194/0 (socket says 172.21.15.47:42194) 2024-03-21T10:24:06.002 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:05.999+0000 7f3907400640 1 -- 172.21.15.47:0/1349126613 learned_addr learned my addr 172.21.15.47:0/1349126613 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:24:06.002 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:05.999+0000 7f3907e00640 1 --2- 172.21.15.47:0/1349126613 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3908077c40 0x7f39081a53c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:24:06.002 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:05.999+0000 7f3906a00640 1 --2- 172.21.15.47:0/1349126613 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f39080772a0 0x7f390819e3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:24:06.002 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:05.999+0000 7f3907400640 1 -- 172.21.15.47:0/1349126613 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f39080772a0 msgr2=0x7f390819e3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:24:06.002 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:05.999+0000 7f3907400640 1 --2- 172.21.15.47:0/1349126613 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f39080772a0 0x7f390819e3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:24:06.002 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:05.999+0000 7f3907400640 1 -- 172.21.15.47:0/1349126613 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3908077c40 msgr2=0x7f39081a53c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:24:06.002 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:05.999+0000 7f3907400640 1 --2- 172.21.15.47:0/1349126613 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3908077c40 0x7f39081a53c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:24:06.002 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:05.999+0000 7f3907400640 1 -- 172.21.15.47:0/1349126613 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f39081a5a30 con 0x7f3908076060 2024-03-21T10:24:06.002 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:05.999+0000 7f3907400640 1 --2- 172.21.15.47:0/1349126613 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3908076060 0x7f390819de60 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f38f8009dc0 tx=0x7f38f80328a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:24:06.003 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:06.000+0000 7f38ffe00640 1 -- 172.21.15.47:0/1349126613 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38f8041020 con 0x7f3908076060 2024-03-21T10:24:06.003 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:06.000+0000 7f390ca00640 1 -- 172.21.15.47:0/1349126613 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f39081a5cc0 con 0x7f3908076060 2024-03-21T10:24:06.003 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:06.000+0000 7f38ffe00640 1 -- 172.21.15.47:0/1349126613 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f38f8003e20 con 0x7f3908076060 2024-03-21T10:24:06.003 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:06.000+0000 7f38ffe00640 1 -- 172.21.15.47:0/1349126613 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38f8039040 con 0x7f3908076060 2024-03-21T10:24:06.003 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:06.000+0000 7f390ca00640 1 -- 172.21.15.47:0/1349126613 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f39081a61b0 con 0x7f3908076060 2024-03-21T10:24:06.003 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:06.001+0000 7f390ca00640 1 -- 172.21.15.47:0/1349126613 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3908110110 con 0x7f3908076060 2024-03-21T10:24:06.006 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:06.004+0000 7f38ffe00640 1 -- 172.21.15.47:0/1349126613 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 18) v1 ==== 137854+0+0 (secure 0 0 0) 0x7f38f8044020 con 0x7f3908076060 2024-03-21T10:24:06.007 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:06.005+0000 7f38ffe00640 1 --2- 172.21.15.47:0/1349126613 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f38ec0a2f20 0x7f38ec0a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:24:06.007 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:06.005+0000 7f38ffe00640 1 -- 172.21.15.47:0/1349126613 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9216+0+0 (secure 0 0 0) 0x7f38f80f1f10 con 0x7f3908076060 2024-03-21T10:24:06.008 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:06.006+0000 7f3906a00640 1 --2- 172.21.15.47:0/1349126613 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f38ec0a2f20 0x7f38ec0a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:24:06.008 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:06.007+0000 7f3906a00640 1 --2- 172.21.15.47:0/1349126613 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f38ec0a2f20 0x7f38ec0a53e0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f38f4005fd0 tx=0x7f38f4005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:24:06.010 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:06.008+0000 7f38ffe00640 1 -- 172.21.15.47:0/1349126613 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f38f804b020 con 0x7f3908076060 2024-03-21T10:24:06.214 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:24:05 smithi047 ceph-mon[31547]: pgmap v434: 129 pgs: 129 active+clean; 642 KiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 426 B/s wr, 12 op/s 2024-03-21T10:24:06.214 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:24:05 smithi047 ceph-mon[31547]: from='client.? 172.21.15.47:0/3180768636' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-21T10:24:06.218 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:06.216+0000 7f390ca00640 1 -- 172.21.15.47:0/1349126613 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f390802d2e0 con 0x7f3908076060 2024-03-21T10:24:06.219 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:06.217+0000 7f38ffe00640 1 -- 172.21.15.47:0/1349126613 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v95) v1 ==== 93+0+10 (secure 0 0 0) 0x7f38f80be930 con 0x7f3908076060 2024-03-21T10:24:06.219 INFO:teuthology.orchestra.run.smithi047.stdout:pg_num: 1 2024-03-21T10:24:06.222 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:06.220+0000 7f390ca00640 1 -- 172.21.15.47:0/1349126613 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f38ec0a2f20 msgr2=0x7f38ec0a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:24:06.222 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:06.220+0000 7f390ca00640 1 --2- 172.21.15.47:0/1349126613 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f38ec0a2f20 0x7f38ec0a53e0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f38f4005fd0 tx=0x7f38f4005850 comp rx=0 tx=0).stop 2024-03-21T10:24:06.222 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:06.220+0000 7f390ca00640 1 -- 172.21.15.47:0/1349126613 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3908076060 msgr2=0x7f390819de60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:24:06.223 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:06.220+0000 7f390ca00640 1 --2- 172.21.15.47:0/1349126613 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3908076060 0x7f390819de60 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f38f8009dc0 tx=0x7f38f80328a0 comp rx=0 tx=0).stop 2024-03-21T10:24:06.223 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:06.220+0000 7f390ca00640 1 -- 172.21.15.47:0/1349126613 shutdown_connections 2024-03-21T10:24:06.223 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:06.220+0000 7f390ca00640 1 --2- 172.21.15.47:0/1349126613 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f38ec0a2f20 0x7f38ec0a53e0 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:24:06.223 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:06.220+0000 7f390ca00640 1 --2- 172.21.15.47:0/1349126613 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3908077c40 0x7f39081a53c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:24:06.223 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:06.220+0000 7f390ca00640 1 --2- 172.21.15.47:0/1349126613 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f39080772a0 0x7f390819e3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:24:06.223 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:06.220+0000 7f390ca00640 1 --2- 172.21.15.47:0/1349126613 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3908076060 0x7f390819de60 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:24:06.223 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:06.220+0000 7f390ca00640 1 -- 172.21.15.47:0/1349126613 >> 172.21.15.47:0/1349126613 conn(0x7f39080fed10 msgr2=0x7f3908102070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:24:06.223 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:06.220+0000 7f390ca00640 1 -- 172.21.15.47:0/1349126613 shutdown_connections 2024-03-21T10:24:06.223 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:06.220+0000 7f390ca00640 1 -- 172.21.15.47:0/1349126613 wait complete. 2024-03-21T10:24:06.229 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:24:05 smithi100 ceph-mon[35193]: pgmap v434: 129 pgs: 129 active+clean; 642 KiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 426 B/s wr, 12 op/s 2024-03-21T10:24:06.229 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:24:05 smithi100 ceph-mon[35193]: from='client.? 172.21.15.47:0/3180768636' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-21T10:24:06.239 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:24:05 smithi157 ceph-mon[35581]: pgmap v434: 129 pgs: 129 active+clean; 642 KiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 426 B/s wr, 12 op/s 2024-03-21T10:24:06.240 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:24:05 smithi157 ceph-mon[35581]: from='client.? 172.21.15.47:0/3180768636' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-21T10:24:06.242 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-21T10:24:06.460 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:06.458+0000 7f149b200640 1 -- 172.21.15.47:0/4267027304 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f149c1037a0 msgr2=0x7f149c103bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:24:06.461 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:06.458+0000 7f149b200640 1 --2- 172.21.15.47:0/4267027304 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f149c1037a0 0x7f149c103bc0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f148c009900 tx=0x7f148c02bc40 comp rx=0 tx=0).stop 2024-03-21T10:24:06.461 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:06.458+0000 7f149b200640 1 -- 172.21.15.47:0/4267027304 shutdown_connections 2024-03-21T10:24:06.461 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:06.458+0000 7f149b200640 1 --2- 172.21.15.47:0/4267027304 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f149c105350 0x7f149c10bbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:24:06.461 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:06.458+0000 7f149b200640 1 --2- 172.21.15.47:0/4267027304 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f149c104990 0x7f149c104e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:24:06.461 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:06.458+0000 7f149b200640 1 --2- 172.21.15.47:0/4267027304 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f149c1037a0 0x7f149c103bc0 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:24:06.461 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:06.458+0000 7f149b200640 1 -- 172.21.15.47:0/4267027304 >> 172.21.15.47:0/4267027304 conn(0x7f149c0fed10 msgr2=0x7f149c101150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:24:06.461 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:06.458+0000 7f149b200640 1 -- 172.21.15.47:0/4267027304 shutdown_connections 2024-03-21T10:24:06.461 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:06.458+0000 7f149b200640 1 -- 172.21.15.47:0/4267027304 wait complete. 2024-03-21T10:24:06.464 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:06.459+0000 7f149b200640 1 Processor -- start 2024-03-21T10:24:06.464 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:06.459+0000 7f149b200640 1 -- start start 2024-03-21T10:24:06.464 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:06.460+0000 7f149b200640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f149c1037a0 0x7f149c199a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:24:06.464 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:06.460+0000 7f149b200640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f149c104990 0x7f149c199f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:24:06.464 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:06.460+0000 7f149b200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f149c105350 0x7f149c1a0fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:24:06.464 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:06.460+0000 7f149b200640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f149c10f2a0 con 0x7f149c104990 2024-03-21T10:24:06.464 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:06.460+0000 7f149b200640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f149c10f120 con 0x7f149c105350 2024-03-21T10:24:06.464 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:06.460+0000 7f149b200640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f149c10f420 con 0x7f149c1037a0 2024-03-21T10:24:06.464 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:06.460+0000 7f1493e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f149c104990 0x7f149c199f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:24:06.464 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:06.460+0000 7f1493e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f149c104990 0x7f149c199f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:42224/0 (socket says 172.21.15.47:42224) 2024-03-21T10:24:06.464 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:06.460+0000 7f1493e00640 1 -- 172.21.15.47:0/3788674061 learned_addr learned my addr 172.21.15.47:0/3788674061 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:24:06.464 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:06.460+0000 7f149a800640 1 --2- 172.21.15.47:0/3788674061 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f149c105350 0x7f149c1a0fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:24:06.464 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:06.460+0000 7f1499e00640 1 --2- 172.21.15.47:0/3788674061 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f149c1037a0 0x7f149c199a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:24:06.465 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:06.461+0000 7f1493e00640 1 -- 172.21.15.47:0/3788674061 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f149c1037a0 msgr2=0x7f149c199a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:24:06.465 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:06.461+0000 7f1493e00640 1 --2- 172.21.15.47:0/3788674061 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f149c1037a0 0x7f149c199a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:24:06.465 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:06.461+0000 7f1493e00640 1 -- 172.21.15.47:0/3788674061 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f149c105350 msgr2=0x7f149c1a0fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:24:06.465 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:06.461+0000 7f1493e00640 1 --2- 172.21.15.47:0/3788674061 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f149c105350 0x7f149c1a0fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:24:06.465 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:06.461+0000 7f1493e00640 1 -- 172.21.15.47:0/3788674061 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f149c1a1630 con 0x7f149c104990 2024-03-21T10:24:06.465 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:06.461+0000 7f149a800640 1 --2- 172.21.15.47:0/3788674061 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f149c105350 0x7f149c1a0fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T10:24:06.465 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:06.461+0000 7f1493e00640 1 --2- 172.21.15.47:0/3788674061 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f149c104990 0x7f149c199f50 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f148400d950 tx=0x7f148400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:24:06.465 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:06.461+0000 7f1492a00640 1 -- 172.21.15.47:0/3788674061 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1484017070 con 0x7f149c104990 2024-03-21T10:24:06.465 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:06.461+0000 7f1492a00640 1 -- 172.21.15.47:0/3788674061 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f1484009c30 con 0x7f149c104990 2024-03-21T10:24:06.465 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:06.461+0000 7f149b200640 1 -- 172.21.15.47:0/3788674061 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f149c1a1920 con 0x7f149c104990 2024-03-21T10:24:06.465 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:06.461+0000 7f1492a00640 1 -- 172.21.15.47:0/3788674061 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f148400f620 con 0x7f149c104990 2024-03-21T10:24:06.465 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:06.462+0000 7f149b200640 1 -- 172.21.15.47:0/3788674061 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f149c1a1e70 con 0x7f149c104990 2024-03-21T10:24:06.466 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:06.463+0000 7f149b200640 1 -- 172.21.15.47:0/3788674061 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f149c10ab30 con 0x7f149c104990 2024-03-21T10:24:06.468 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:06.467+0000 7f1492a00640 1 -- 172.21.15.47:0/3788674061 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 18) v1 ==== 137854+0+0 (secure 0 0 0) 0x7f14840109a0 con 0x7f149c104990 2024-03-21T10:24:06.470 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:06.468+0000 7f1492a00640 1 --2- 172.21.15.47:0/3788674061 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f14800a2f20 0x7f14800a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:24:06.470 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:06.468+0000 7f1492a00640 1 -- 172.21.15.47:0/3788674061 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9216+0+0 (secure 0 0 0) 0x7f14840cf4e0 con 0x7f149c104990 2024-03-21T10:24:06.470 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:06.468+0000 7f1499e00640 1 --2- 172.21.15.47:0/3788674061 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f14800a2f20 0x7f14800a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:24:06.471 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:06.469+0000 7f1492a00640 1 -- 172.21.15.47:0/3788674061 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f1484009de0 con 0x7f149c104990 2024-03-21T10:24:06.471 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:06.469+0000 7f1499e00640 1 --2- 172.21.15.47:0/3788674061 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f14800a2f20 0x7f14800a53e0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f148c006010 tx=0x7f148c005be0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:24:06.676 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:06.673+0000 7f149b200640 1 -- 172.21.15.47:0/3788674061 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f149c02d2c0 con 0x7f149c104990 2024-03-21T10:24:06.676 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:06.674+0000 7f1492a00640 1 -- 172.21.15.47:0/3788674061 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v95) v1 ==== 104+0+11 (secure 0 0 0) 0x7f148409bf00 con 0x7f149c104990 2024-03-21T10:24:06.677 INFO:teuthology.orchestra.run.smithi047.stdout:pg_num: 32 2024-03-21T10:24:06.680 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:06.677+0000 7f149b200640 1 -- 172.21.15.47:0/3788674061 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f14800a2f20 msgr2=0x7f14800a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:24:06.680 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:06.677+0000 7f149b200640 1 --2- 172.21.15.47:0/3788674061 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f14800a2f20 0x7f14800a53e0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f148c006010 tx=0x7f148c005be0 comp rx=0 tx=0).stop 2024-03-21T10:24:06.680 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:06.677+0000 7f149b200640 1 -- 172.21.15.47:0/3788674061 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f149c104990 msgr2=0x7f149c199f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:24:06.680 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:06.677+0000 7f149b200640 1 --2- 172.21.15.47:0/3788674061 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f149c104990 0x7f149c199f50 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f148400d950 tx=0x7f148400de10 comp rx=0 tx=0).stop 2024-03-21T10:24:06.680 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:06.677+0000 7f149b200640 1 -- 172.21.15.47:0/3788674061 shutdown_connections 2024-03-21T10:24:06.680 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:06.677+0000 7f149b200640 1 --2- 172.21.15.47:0/3788674061 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f14800a2f20 0x7f14800a53e0 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:24:06.680 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:06.677+0000 7f149b200640 1 --2- 172.21.15.47:0/3788674061 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f149c105350 0x7f149c1a0fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:24:06.680 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:06.677+0000 7f149b200640 1 --2- 172.21.15.47:0/3788674061 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f149c104990 0x7f149c199f50 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:24:06.680 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:06.677+0000 7f149b200640 1 --2- 172.21.15.47:0/3788674061 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f149c1037a0 0x7f149c199a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:24:06.681 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:06.677+0000 7f149b200640 1 -- 172.21.15.47:0/3788674061 >> 172.21.15.47:0/3788674061 conn(0x7f149c0fed10 msgr2=0x7f149c107d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:24:06.681 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:06.678+0000 7f149b200640 1 -- 172.21.15.47:0/3788674061 shutdown_connections 2024-03-21T10:24:06.681 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:06.678+0000 7f149b200640 1 -- 172.21.15.47:0/3788674061 wait complete. 2024-03-21T10:24:06.701 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-21T10:24:06.943 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:06.940+0000 7f313d200640 1 -- 172.21.15.47:0/4023000767 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f31381016b0 msgr2=0x7f3138101ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:24:06.943 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:06.940+0000 7f313d200640 1 --2- 172.21.15.47:0/4023000767 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f31381016b0 0x7f3138101ad0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f3128009900 tx=0x7f312802bc30 comp rx=0 tx=0).stop 2024-03-21T10:24:06.944 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:06.940+0000 7f313d200640 1 -- 172.21.15.47:0/4023000767 shutdown_connections 2024-03-21T10:24:06.944 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:06.940+0000 7f313d200640 1 --2- 172.21.15.47:0/4023000767 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3138103260 0x7f3138109af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:24:06.944 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:06.940+0000 7f313d200640 1 --2- 172.21.15.47:0/4023000767 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f31381028a0 0x7f3138102d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:24:06.944 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:06.940+0000 7f313d200640 1 --2- 172.21.15.47:0/4023000767 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f31381016b0 0x7f3138101ad0 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:24:06.944 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:06.940+0000 7f313d200640 1 -- 172.21.15.47:0/4023000767 >> 172.21.15.47:0/4023000767 conn(0x7f31380fcc40 msgr2=0x7f31380ff060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:24:06.944 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:06.940+0000 7f313d200640 1 -- 172.21.15.47:0/4023000767 shutdown_connections 2024-03-21T10:24:06.944 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:06.940+0000 7f313d200640 1 -- 172.21.15.47:0/4023000767 wait complete. 2024-03-21T10:24:06.944 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:06.941+0000 7f313d200640 1 Processor -- start 2024-03-21T10:24:06.948 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:06.942+0000 7f313d200640 1 -- start start 2024-03-21T10:24:06.948 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:06.942+0000 7f313d200640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f31381016b0 0x7f3138199b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:24:06.948 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:06.942+0000 7f313d200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f31381028a0 0x7f313819a080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:24:06.948 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:06.942+0000 7f313d200640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3138103260 0x7f31381a10f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:24:06.948 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:06.942+0000 7f313d200640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f313810d200 con 0x7f31381016b0 2024-03-21T10:24:06.949 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:06.942+0000 7f313d200640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f313810d080 con 0x7f31381028a0 2024-03-21T10:24:06.949 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:06.942+0000 7f313d200640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f313810d380 con 0x7f3138103260 2024-03-21T10:24:06.949 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:06.942+0000 7f3135e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f31381016b0 0x7f3138199b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:24:06.949 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:06.942+0000 7f3135e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f31381016b0 0x7f3138199b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:42246/0 (socket says 172.21.15.47:42246) 2024-03-21T10:24:06.949 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:06.942+0000 7f3135e00640 1 -- 172.21.15.47:0/1444476810 learned_addr learned my addr 172.21.15.47:0/1444476810 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:24:06.949 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:06.942+0000 7f3135400640 1 --2- 172.21.15.47:0/1444476810 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f31381028a0 0x7f313819a080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:24:06.949 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:06.943+0000 7f3136800640 1 --2- 172.21.15.47:0/1444476810 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3138103260 0x7f31381a10f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:24:06.950 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:06.943+0000 7f3135e00640 1 -- 172.21.15.47:0/1444476810 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3138103260 msgr2=0x7f31381a10f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:24:06.950 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:06.943+0000 7f3135e00640 1 --2- 172.21.15.47:0/1444476810 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3138103260 0x7f31381a10f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:24:06.950 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:06.943+0000 7f3135e00640 1 -- 172.21.15.47:0/1444476810 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f31381028a0 msgr2=0x7f313819a080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:24:06.950 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:06.943+0000 7f3135e00640 1 --2- 172.21.15.47:0/1444476810 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f31381028a0 0x7f313819a080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:24:06.950 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:06.943+0000 7f3135e00640 1 -- 172.21.15.47:0/1444476810 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f31381a1760 con 0x7f31381016b0 2024-03-21T10:24:06.950 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:06.943+0000 7f3135e00640 1 --2- 172.21.15.47:0/1444476810 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f31381016b0 0x7f3138199b40 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f3128000c00 tx=0x7f31280040c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:24:06.950 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:06.943+0000 7f312ea00640 1 -- 172.21.15.47:0/1444476810 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3128043020 con 0x7f31381016b0 2024-03-21T10:24:06.950 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:06.944+0000 7f312ea00640 1 -- 172.21.15.47:0/1444476810 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f31280044c0 con 0x7f31381016b0 2024-03-21T10:24:06.951 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:06.944+0000 7f313d200640 1 -- 172.21.15.47:0/1444476810 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f31381a19f0 con 0x7f31381016b0 2024-03-21T10:24:06.951 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:06.944+0000 7f312ea00640 1 -- 172.21.15.47:0/1444476810 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3128003c90 con 0x7f31381016b0 2024-03-21T10:24:06.951 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:06.944+0000 7f313d200640 1 -- 172.21.15.47:0/1444476810 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f31381a1f90 con 0x7f31381016b0 2024-03-21T10:24:06.951 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:06.945+0000 7f313d200640 1 -- 172.21.15.47:0/1444476810 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f31381a2160 con 0x7f31381016b0 2024-03-21T10:24:06.951 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:06.948+0000 7f312ea00640 1 -- 172.21.15.47:0/1444476810 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 18) v1 ==== 137854+0+0 (secure 0 0 0) 0x7f3128046020 con 0x7f31381016b0 2024-03-21T10:24:06.952 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:06.950+0000 7f312ea00640 1 --2- 172.21.15.47:0/1444476810 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f311c0a2fa0 0x7f311c0a5460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:24:06.952 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:06.950+0000 7f312ea00640 1 -- 172.21.15.47:0/1444476810 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9216+0+0 (secure 0 0 0) 0x7f31280f1a30 con 0x7f31381016b0 2024-03-21T10:24:06.952 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:06.950+0000 7f3135400640 1 --2- 172.21.15.47:0/1444476810 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f311c0a2fa0 0x7f311c0a5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:24:06.953 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:06.951+0000 7f3135400640 1 --2- 172.21.15.47:0/1444476810 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f311c0a2fa0 0x7f311c0a5460 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f313819b050 tx=0x7f312400b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:24:06.956 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:06.954+0000 7f312ea00640 1 -- 172.21.15.47:0/1444476810 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f312804d020 con 0x7f31381016b0 2024-03-21T10:24:06.963 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:24:06 smithi047 ceph-mon[31547]: from='client.? 172.21.15.47:0/1349126613' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-21T10:24:06.963 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:24:06 smithi047 ceph-mon[31547]: from='client.? 172.21.15.47:0/3788674061' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-21T10:24:07.161 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:07.159+0000 7f313d200640 1 -- 172.21.15.47:0/1444476810 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f31381a2330 con 0x7f31381016b0 2024-03-21T10:24:07.162 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:07.159+0000 7f312ea00640 1 -- 172.21.15.47:0/1444476810 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v95) v1 ==== 100+0+11 (secure 0 0 0) 0x7f31280be570 con 0x7f31381016b0 2024-03-21T10:24:07.162 INFO:teuthology.orchestra.run.smithi047.stdout:pg_num: 64 2024-03-21T10:24:07.165 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:07.162+0000 7f313d200640 1 -- 172.21.15.47:0/1444476810 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f311c0a2fa0 msgr2=0x7f311c0a5460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:24:07.165 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:07.162+0000 7f313d200640 1 --2- 172.21.15.47:0/1444476810 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f311c0a2fa0 0x7f311c0a5460 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f313819b050 tx=0x7f312400b410 comp rx=0 tx=0).stop 2024-03-21T10:24:07.165 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:07.162+0000 7f313d200640 1 -- 172.21.15.47:0/1444476810 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f31381016b0 msgr2=0x7f3138199b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:24:07.165 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:07.162+0000 7f313d200640 1 --2- 172.21.15.47:0/1444476810 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f31381016b0 0x7f3138199b40 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f3128000c00 tx=0x7f31280040c0 comp rx=0 tx=0).stop 2024-03-21T10:24:07.166 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:07.162+0000 7f313d200640 1 -- 172.21.15.47:0/1444476810 shutdown_connections 2024-03-21T10:24:07.166 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:07.162+0000 7f313d200640 1 --2- 172.21.15.47:0/1444476810 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f311c0a2fa0 0x7f311c0a5460 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:24:07.166 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:07.162+0000 7f313d200640 1 --2- 172.21.15.47:0/1444476810 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3138103260 0x7f31381a10f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:24:07.166 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:07.162+0000 7f313d200640 1 --2- 172.21.15.47:0/1444476810 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f31381028a0 0x7f313819a080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:24:07.166 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:07.162+0000 7f313d200640 1 --2- 172.21.15.47:0/1444476810 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f31381016b0 0x7f3138199b40 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:24:07.166 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:07.162+0000 7f313d200640 1 -- 172.21.15.47:0/1444476810 >> 172.21.15.47:0/1444476810 conn(0x7f31380fcc40 msgr2=0x7f31380738d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:24:07.167 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:07.163+0000 7f313d200640 1 -- 172.21.15.47:0/1444476810 shutdown_connections 2024-03-21T10:24:07.167 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:07.163+0000 7f313d200640 1 -- 172.21.15.47:0/1444476810 wait complete. 2024-03-21T10:24:07.184 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-03-21T10:24:07.229 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:24:06 smithi100 ceph-mon[35193]: from='client.? 172.21.15.47:0/1349126613' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-21T10:24:07.229 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:24:06 smithi100 ceph-mon[35193]: from='client.? 172.21.15.47:0/3788674061' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-21T10:24:07.240 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:24:06 smithi157 ceph-mon[35581]: from='client.? 172.21.15.47:0/1349126613' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-21T10:24:07.240 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:24:06 smithi157 ceph-mon[35581]: from='client.? 172.21.15.47:0/3788674061' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-21T10:24:07.413 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:07.411+0000 7f620d000640 1 -- 172.21.15.47:0/367205915 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f62081049d0 msgr2=0x7f6208106dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:24:07.414 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:07.411+0000 7f620d000640 1 --2- 172.21.15.47:0/367205915 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f62081049d0 0x7f6208106dc0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f61f0009900 tx=0x7f61f002bc30 comp rx=0 tx=0).stop 2024-03-21T10:24:07.414 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:07.411+0000 7f620d000640 1 -- 172.21.15.47:0/367205915 shutdown_connections 2024-03-21T10:24:07.414 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:07.411+0000 7f620d000640 1 --2- 172.21.15.47:0/367205915 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6208107300 0x7f62081096f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:24:07.414 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:07.411+0000 7f620d000640 1 --2- 172.21.15.47:0/367205915 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f62081049d0 0x7f6208106dc0 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:24:07.414 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:07.411+0000 7f620d000640 1 --2- 172.21.15.47:0/367205915 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f62081020a0 0x7f6208104490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:24:07.414 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:07.411+0000 7f620d000640 1 -- 172.21.15.47:0/367205915 >> 172.21.15.47:0/367205915 conn(0x7f62080fba10 msgr2=0x7f62080fde30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:24:07.414 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:07.411+0000 7f620d000640 1 -- 172.21.15.47:0/367205915 shutdown_connections 2024-03-21T10:24:07.414 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:07.411+0000 7f620d000640 1 -- 172.21.15.47:0/367205915 wait complete. 2024-03-21T10:24:07.416 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:07.412+0000 7f620d000640 1 Processor -- start 2024-03-21T10:24:07.416 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:07.413+0000 7f620d000640 1 -- start start 2024-03-21T10:24:07.416 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:07.413+0000 7f620d000640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f62081020a0 0x7f62081999d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:24:07.416 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:07.413+0000 7f620d000640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f62081049d0 0x7f6208199f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:24:07.416 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:07.413+0000 7f620d000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6208107300 0x7f62081a0f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:24:07.416 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:07.413+0000 7f620d000640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f620810d0e0 con 0x7f62081020a0 2024-03-21T10:24:07.416 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:07.413+0000 7f620d000640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f620810cf60 con 0x7f6208107300 2024-03-21T10:24:07.416 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:07.413+0000 7f620d000640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f620810d260 con 0x7f62081049d0 2024-03-21T10:24:07.416 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:07.413+0000 7f6207400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f62081020a0 0x7f62081999d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:24:07.416 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:07.413+0000 7f6207400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f62081020a0 0x7f62081999d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:42266/0 (socket says 172.21.15.47:42266) 2024-03-21T10:24:07.417 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:07.413+0000 7f6207400640 1 -- 172.21.15.47:0/2441737183 learned_addr learned my addr 172.21.15.47:0/2441737183 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:24:07.417 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:07.413+0000 7f6207e00640 1 --2- 172.21.15.47:0/2441737183 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6208107300 0x7f62081a0f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:24:07.417 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:07.413+0000 7f6206a00640 1 --2- 172.21.15.47:0/2441737183 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f62081049d0 0x7f6208199f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:24:07.417 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:07.414+0000 7f6207400640 1 -- 172.21.15.47:0/2441737183 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f62081049d0 msgr2=0x7f6208199f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:24:07.417 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:07.414+0000 7f6207400640 1 --2- 172.21.15.47:0/2441737183 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f62081049d0 0x7f6208199f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:24:07.417 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:07.414+0000 7f6207400640 1 -- 172.21.15.47:0/2441737183 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6208107300 msgr2=0x7f62081a0f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:24:07.417 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:07.414+0000 7f6207400640 1 --2- 172.21.15.47:0/2441737183 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6208107300 0x7f62081a0f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:24:07.417 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:07.414+0000 7f6207400640 1 -- 172.21.15.47:0/2441737183 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f62081a15f0 con 0x7f62081020a0 2024-03-21T10:24:07.417 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:07.414+0000 7f6207e00640 1 --2- 172.21.15.47:0/2441737183 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6208107300 0x7f62081a0f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T10:24:07.418 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:07.414+0000 7f6207400640 1 --2- 172.21.15.47:0/2441737183 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f62081020a0 0x7f62081999d0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f61f800ebc0 tx=0x7f61f800c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:24:07.418 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:07.415+0000 7f61ffe00640 1 -- 172.21.15.47:0/2441737183 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61f8010630 con 0x7f62081020a0 2024-03-21T10:24:07.418 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:07.415+0000 7f61ffe00640 1 -- 172.21.15.47:0/2441737183 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f61f8010c80 con 0x7f62081020a0 2024-03-21T10:24:07.418 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:07.415+0000 7f620d000640 1 -- 172.21.15.47:0/2441737183 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f62081a18e0 con 0x7f62081020a0 2024-03-21T10:24:07.418 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:07.415+0000 7f61ffe00640 1 -- 172.21.15.47:0/2441737183 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61f8018b30 con 0x7f62081020a0 2024-03-21T10:24:07.418 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:07.415+0000 7f620d000640 1 -- 172.21.15.47:0/2441737183 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f62081a1e00 con 0x7f62081020a0 2024-03-21T10:24:07.418 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:07.416+0000 7f620d000640 1 -- 172.21.15.47:0/2441737183 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f62080683b0 con 0x7f62081020a0 2024-03-21T10:24:07.420 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:07.418+0000 7f61ffe00640 1 -- 172.21.15.47:0/2441737183 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 18) v1 ==== 137854+0+0 (secure 0 0 0) 0x7f61f8018cd0 con 0x7f62081020a0 2024-03-21T10:24:07.422 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:07.420+0000 7f61ffe00640 1 --2- 172.21.15.47:0/2441737183 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f61ec0a2f20 0x7f61ec0a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:24:07.423 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:07.421+0000 7f61ffe00640 1 -- 172.21.15.47:0/2441737183 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9216+0+0 (secure 0 0 0) 0x7f61f80cfe40 con 0x7f62081020a0 2024-03-21T10:24:07.423 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:07.421+0000 7f6206a00640 1 --2- 172.21.15.47:0/2441737183 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f61ec0a2f20 0x7f61ec0a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:24:07.423 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:07.422+0000 7f6206a00640 1 --2- 172.21.15.47:0/2441737183 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f61ec0a2f20 0x7f61ec0a53e0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f61f0005ab0 tx=0x7f61f0005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:24:07.426 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:07.424+0000 7f61ffe00640 1 -- 172.21.15.47:0/2441737183 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f61f809c8a0 con 0x7f62081020a0 2024-03-21T10:24:07.630 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:07.628+0000 7f620d000640 1 -- 172.21.15.47:0/2441737183 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f62080622a0 con 0x7f62081020a0 2024-03-21T10:24:07.631 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:07.629+0000 7f61ffe00640 1 -- 172.21.15.47:0/2441737183 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v95) v1 ==== 103+0+11 (secure 0 0 0) 0x7f61f809c1a0 con 0x7f62081020a0 2024-03-21T10:24:07.632 INFO:teuthology.orchestra.run.smithi047.stdout:pg_num: 32 2024-03-21T10:24:07.635 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:07.631+0000 7f620d000640 1 -- 172.21.15.47:0/2441737183 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f61ec0a2f20 msgr2=0x7f61ec0a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:24:07.635 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:07.631+0000 7f620d000640 1 --2- 172.21.15.47:0/2441737183 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f61ec0a2f20 0x7f61ec0a53e0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f61f0005ab0 tx=0x7f61f0005fd0 comp rx=0 tx=0).stop 2024-03-21T10:24:07.635 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:07.631+0000 7f620d000640 1 -- 172.21.15.47:0/2441737183 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f62081020a0 msgr2=0x7f62081999d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:24:07.635 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:07.631+0000 7f620d000640 1 --2- 172.21.15.47:0/2441737183 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f62081020a0 0x7f62081999d0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f61f800ebc0 tx=0x7f61f800c6a0 comp rx=0 tx=0).stop 2024-03-21T10:24:07.635 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:07.632+0000 7f620d000640 1 -- 172.21.15.47:0/2441737183 shutdown_connections 2024-03-21T10:24:07.635 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:07.632+0000 7f620d000640 1 --2- 172.21.15.47:0/2441737183 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f61ec0a2f20 0x7f61ec0a53e0 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:24:07.635 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:07.632+0000 7f620d000640 1 --2- 172.21.15.47:0/2441737183 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6208107300 0x7f62081a0f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:24:07.635 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:07.632+0000 7f620d000640 1 --2- 172.21.15.47:0/2441737183 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f62081049d0 0x7f6208199f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:24:07.636 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:07.632+0000 7f620d000640 1 --2- 172.21.15.47:0/2441737183 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f62081020a0 0x7f62081999d0 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:24:07.636 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:07.632+0000 7f620d000640 1 -- 172.21.15.47:0/2441737183 >> 172.21.15.47:0/2441737183 conn(0x7f62080fba10 msgr2=0x7f62081017b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:24:07.636 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:07.632+0000 7f620d000640 1 -- 172.21.15.47:0/2441737183 shutdown_connections 2024-03-21T10:24:07.636 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:07.632+0000 7f620d000640 1 -- 172.21.15.47:0/2441737183 wait complete. 2024-03-21T10:24:07.653 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-21T10:24:07.879 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:07.876+0000 7fc68b200640 1 -- 172.21.15.47:0/2548430389 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc684103260 msgr2=0x7fc684109af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:24:07.879 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:07.876+0000 7fc68b200640 1 --2- 172.21.15.47:0/2548430389 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc684103260 0x7fc684109af0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7fc67c009ca0 tx=0x7fc67c02c0f0 comp rx=0 tx=0).stop 2024-03-21T10:24:07.879 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:07.876+0000 7fc68b200640 1 -- 172.21.15.47:0/2548430389 shutdown_connections 2024-03-21T10:24:07.879 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:07.876+0000 7fc68b200640 1 --2- 172.21.15.47:0/2548430389 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc684103260 0x7fc684109af0 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:24:07.879 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:07.876+0000 7fc68b200640 1 --2- 172.21.15.47:0/2548430389 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc6841028a0 0x7fc684102d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:24:07.879 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:07.876+0000 7fc68b200640 1 --2- 172.21.15.47:0/2548430389 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc6841016b0 0x7fc684101ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:24:07.879 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:07.876+0000 7fc68b200640 1 -- 172.21.15.47:0/2548430389 >> 172.21.15.47:0/2548430389 conn(0x7fc6840fcc80 msgr2=0x7fc6840ff0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:24:07.879 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:07.876+0000 7fc68b200640 1 -- 172.21.15.47:0/2548430389 shutdown_connections 2024-03-21T10:24:07.879 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:07.876+0000 7fc68b200640 1 -- 172.21.15.47:0/2548430389 wait complete. 2024-03-21T10:24:07.880 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:07.878+0000 7fc68b200640 1 Processor -- start 2024-03-21T10:24:07.880 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:07.878+0000 7fc68b200640 1 -- start start 2024-03-21T10:24:07.881 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:07.878+0000 7fc68b200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc6841016b0 0x7fc684197890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:24:07.881 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:07.878+0000 7fc68b200640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc6841028a0 0x7fc684197dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:24:07.882 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:07.878+0000 7fc68b200640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc684103260 0x7fc68419ee40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:24:07.882 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:07.878+0000 7fc68b200640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc68410d1c0 con 0x7fc6841028a0 2024-03-21T10:24:07.882 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:07.878+0000 7fc68b200640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc68410d040 con 0x7fc6841016b0 2024-03-21T10:24:07.882 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:07.878+0000 7fc68b200640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc68410d340 con 0x7fc684103260 2024-03-21T10:24:07.882 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:07.879+0000 7fc683400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc6841028a0 0x7fc684197dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:24:07.882 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:07.879+0000 7fc683400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc6841028a0 0x7fc684197dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:42282/0 (socket says 172.21.15.47:42282) 2024-03-21T10:24:07.882 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:07.879+0000 7fc683400640 1 -- 172.21.15.47:0/3518280300 learned_addr learned my addr 172.21.15.47:0/3518280300 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:24:07.882 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:07.879+0000 7fc683e00640 1 --2- 172.21.15.47:0/3518280300 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc6841016b0 0x7fc684197890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:24:07.882 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:07.879+0000 7fc689200640 1 --2- 172.21.15.47:0/3518280300 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc684103260 0x7fc68419ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:24:07.882 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:07.879+0000 7fc683400640 1 -- 172.21.15.47:0/3518280300 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc684103260 msgr2=0x7fc68419ee40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:24:07.882 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:07.879+0000 7fc683400640 1 --2- 172.21.15.47:0/3518280300 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc684103260 0x7fc68419ee40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:24:07.882 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:07.879+0000 7fc683400640 1 -- 172.21.15.47:0/3518280300 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc6841016b0 msgr2=0x7fc684197890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:24:07.883 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:07.879+0000 7fc683400640 1 --2- 172.21.15.47:0/3518280300 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc6841016b0 0x7fc684197890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:24:07.883 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:07.879+0000 7fc683400640 1 -- 172.21.15.47:0/3518280300 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc68419f4b0 con 0x7fc6841028a0 2024-03-21T10:24:07.883 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:07.879+0000 7fc683e00640 1 --2- 172.21.15.47:0/3518280300 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc6841016b0 0x7fc684197890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T10:24:07.883 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:07.880+0000 7fc683400640 1 --2- 172.21.15.47:0/3518280300 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc6841028a0 0x7fc684197dd0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7fc67000d950 tx=0x7fc67000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:24:07.883 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:07.880+0000 7fc680c00640 1 -- 172.21.15.47:0/3518280300 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc670017070 con 0x7fc6841028a0 2024-03-21T10:24:07.884 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:07.880+0000 7fc680c00640 1 -- 172.21.15.47:0/3518280300 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fc670009c30 con 0x7fc6841028a0 2024-03-21T10:24:07.884 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:07.880+0000 7fc68b200640 1 -- 172.21.15.47:0/3518280300 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc68419f7a0 con 0x7fc6841028a0 2024-03-21T10:24:07.884 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:07.880+0000 7fc680c00640 1 -- 172.21.15.47:0/3518280300 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc67000f620 con 0x7fc6841028a0 2024-03-21T10:24:07.884 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:07.880+0000 7fc68b200640 1 -- 172.21.15.47:0/3518280300 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc68419fcf0 con 0x7fc6841028a0 2024-03-21T10:24:07.884 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:07.881+0000 7fc68b200640 1 -- 172.21.15.47:0/3518280300 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc684108a20 con 0x7fc6841028a0 2024-03-21T10:24:07.886 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:07.884+0000 7fc680c00640 1 -- 172.21.15.47:0/3518280300 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 18) v1 ==== 137854+0+0 (secure 0 0 0) 0x7fc6700109a0 con 0x7fc6841028a0 2024-03-21T10:24:07.888 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:07.886+0000 7fc680c00640 1 --2- 172.21.15.47:0/3518280300 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7fc6680a2f20 0x7fc6680a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:24:07.889 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:07.887+0000 7fc680c00640 1 -- 172.21.15.47:0/3518280300 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9216+0+0 (secure 0 0 0) 0x7fc6700cf550 con 0x7fc6841028a0 2024-03-21T10:24:07.889 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:07.887+0000 7fc683e00640 1 --2- 172.21.15.47:0/3518280300 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7fc6680a2f20 0x7fc6680a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:24:07.889 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:07.888+0000 7fc683e00640 1 --2- 172.21.15.47:0/3518280300 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7fc6680a2f20 0x7fc6680a53e0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fc6740058c0 tx=0x7fc674005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:24:07.893 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:07.891+0000 7fc680c00640 1 -- 172.21.15.47:0/3518280300 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fc670009de0 con 0x7fc6841028a0 2024-03-21T10:24:07.964 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:24:07 smithi047 ceph-mon[31547]: pgmap v435: 129 pgs: 129 active+clean; 642 KiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 426 B/s wr, 9 op/s 2024-03-21T10:24:07.964 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:24:07 smithi047 ceph-mon[31547]: from='client.? 172.21.15.47:0/1444476810' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-21T10:24:07.964 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:24:07 smithi047 ceph-mon[31547]: from='client.? 172.21.15.47:0/2441737183' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-03-21T10:24:08.139 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:08.137+0000 7fc68b200640 1 -- 172.21.15.47:0/3518280300 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc6840622a0 con 0x7fc6841028a0 2024-03-21T10:24:08.140 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:08.138+0000 7fc680c00640 1 -- 172.21.15.47:0/3518280300 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) v1 ==== 94+0+11471 (secure 0 0 0) 0x7fc67009bf70 con 0x7fc6841028a0 2024-03-21T10:24:08.141 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-21T10:24:08.141 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":29,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14511,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6835/735724037","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6834","nonce":735724037},{"type":"v1","addr":"172.21.15.47:6835","nonce":735724037}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14529,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6837/2248191724","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6836","nonce":2248191724},{"type":"v1","addr":"172.21.15.47:6837","nonce":2248191724}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":29,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-21T10:21:58.329063+0000","modified":"2024-03-21T10:22:29.588509+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":34535,"mds_1":34493,"mds_2":34454,"mds_3":34427,"mds_4":14619},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14604":{"gid":14604,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.47:6839/2881335356","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6838","nonce":2881335356},{"type":"v1","addr":"172.21.15.47:6839","nonce":2881335356}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_14619":{"gid":14619,"name":"a","rank":4,"incarnation":27,"state":"up:active","state_seq":11,"addr":"172.21.15.47:6841/3401578978","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6840","nonce":3401578978},{"type":"v1","addr":"172.21.15.47:6841","nonce":3401578978}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24406":{"gid":24406,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.157:6833/4000010136","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":4000010136},{"type":"v1","addr":"172.21.15.157:6833","nonce":4000010136}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24457":{"gid":24457,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.157:6835/3902868651","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":3902868651},{"type":"v1","addr":"172.21.15.157:6835","nonce":3902868651}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24475":{"gid":24475,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.157:6837/1908087204","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6836","nonce":1908087204},{"type":"v1","addr":"172.21.15.157:6837","nonce":1908087204}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24499":{"gid":24499,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.157:6839/4064242876","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6838","nonce":4064242876},{"type":"v1","addr":"172.21.15.157:6839","nonce":4064242876}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_34427":{"gid":34427,"name":"k","rank":3,"incarnation":25,"state":"up:active","state_seq":20,"addr":"172.21.15.100:6833/2723302038","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6832","nonce":2723302038},{"type":"v1","addr":"172.21.15.100:6833","nonce":2723302038}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_34454":{"gid":34454,"name":"e","rank":2,"incarnation":22,"state":"up:active","state_seq":17,"addr":"172.21.15.100:6835/2597849957","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":2597849957},{"type":"v1","addr":"172.21.15.100:6835","nonce":2597849957}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_34493":{"gid":34493,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":13,"addr":"172.21.15.100:6837/3497644162","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6836","nonce":3497644162},{"type":"v1","addr":"172.21.15.100:6837","nonce":3497644162}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_34535":{"gid":34535,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.100:6839/4173168440","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6838","nonce":4173168440},{"type":"v1","addr":"172.21.15.100:6839","nonce":4173168440}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":34535,"qdb_cluster":[34493,34535,34454,34427,14619]},"id":1}]} 2024-03-21T10:24:08.141 INFO:teuthology.orchestra.run.smithi047.stderr:dumped fsmap epoch 29 2024-03-21T10:24:08.143 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:08.141+0000 7fc68b200640 1 -- 172.21.15.47:0/3518280300 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7fc6680a2f20 msgr2=0x7fc6680a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:24:08.143 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:08.141+0000 7fc68b200640 1 --2- 172.21.15.47:0/3518280300 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7fc6680a2f20 0x7fc6680a53e0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fc6740058c0 tx=0x7fc674005850 comp rx=0 tx=0).stop 2024-03-21T10:24:08.143 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:08.141+0000 7fc68b200640 1 -- 172.21.15.47:0/3518280300 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc6841028a0 msgr2=0x7fc684197dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:24:08.143 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:08.141+0000 7fc68b200640 1 --2- 172.21.15.47:0/3518280300 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc6841028a0 0x7fc684197dd0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7fc67000d950 tx=0x7fc67000de10 comp rx=0 tx=0).stop 2024-03-21T10:24:08.145 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:08.141+0000 7fc68b200640 1 -- 172.21.15.47:0/3518280300 shutdown_connections 2024-03-21T10:24:08.145 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:08.141+0000 7fc68b200640 1 --2- 172.21.15.47:0/3518280300 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7fc6680a2f20 0x7fc6680a53e0 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:24:08.145 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:08.141+0000 7fc68b200640 1 --2- 172.21.15.47:0/3518280300 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc684103260 0x7fc68419ee40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:24:08.145 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:08.141+0000 7fc68b200640 1 --2- 172.21.15.47:0/3518280300 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc6841028a0 0x7fc684197dd0 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:24:08.145 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:08.141+0000 7fc68b200640 1 --2- 172.21.15.47:0/3518280300 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc6841016b0 0x7fc684197890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:24:08.145 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:08.141+0000 7fc68b200640 1 -- 172.21.15.47:0/3518280300 >> 172.21.15.47:0/3518280300 conn(0x7fc6840fcc80 msgr2=0x7fc684105bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:24:08.145 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:08.141+0000 7fc68b200640 1 -- 172.21.15.47:0/3518280300 shutdown_connections 2024-03-21T10:24:08.146 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:08.141+0000 7fc68b200640 1 -- 172.21.15.47:0/3518280300 wait complete. 2024-03-21T10:24:08.165 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-21T10:24:08.229 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:24:07 smithi100 ceph-mon[35193]: pgmap v435: 129 pgs: 129 active+clean; 642 KiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 426 B/s wr, 9 op/s 2024-03-21T10:24:08.229 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:24:07 smithi100 ceph-mon[35193]: from='client.? 172.21.15.47:0/1444476810' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-21T10:24:08.229 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:24:07 smithi100 ceph-mon[35193]: from='client.? 172.21.15.47:0/2441737183' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-03-21T10:24:08.240 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:24:07 smithi157 ceph-mon[35581]: pgmap v435: 129 pgs: 129 active+clean; 642 KiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 426 B/s wr, 9 op/s 2024-03-21T10:24:08.240 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:24:07 smithi157 ceph-mon[35581]: from='client.? 172.21.15.47:0/1444476810' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-21T10:24:08.240 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:24:07 smithi157 ceph-mon[35581]: from='client.? 172.21.15.47:0/2441737183' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-03-21T10:24:08.390 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:08.387+0000 7f3871600640 1 -- 172.21.15.47:0/2710043502 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f386c0740a0 msgr2=0x7f386c074500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:24:08.390 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:08.387+0000 7f3871600640 1 --2- 172.21.15.47:0/2710043502 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f386c0740a0 0x7f386c074500 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f3858009900 tx=0x7f385802bcd0 comp rx=0 tx=0).stop 2024-03-21T10:24:08.390 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:08.387+0000 7f3871600640 1 -- 172.21.15.47:0/2710043502 shutdown_connections 2024-03-21T10:24:08.390 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:08.388+0000 7f3871600640 1 --2- 172.21.15.47:0/2710043502 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f386c109550 0x7f386c10b9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:24:08.390 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:08.388+0000 7f3871600640 1 --2- 172.21.15.47:0/2710043502 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f386c0740a0 0x7f386c074500 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:24:08.390 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:08.388+0000 7f3871600640 1 --2- 172.21.15.47:0/2710043502 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f386c075680 0x7f386c073ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:24:08.390 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:08.388+0000 7f3871600640 1 -- 172.21.15.47:0/2710043502 >> 172.21.15.47:0/2710043502 conn(0x7f386c0fcc50 msgr2=0x7f386c0ff070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:24:08.391 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:08.388+0000 7f3871600640 1 -- 172.21.15.47:0/2710043502 shutdown_connections 2024-03-21T10:24:08.391 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:08.388+0000 7f3871600640 1 -- 172.21.15.47:0/2710043502 wait complete. 2024-03-21T10:24:08.392 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:08.389+0000 7f3871600640 1 Processor -- start 2024-03-21T10:24:08.392 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:08.389+0000 7f3871600640 1 -- start start 2024-03-21T10:24:08.393 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:08.389+0000 7f3871600640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f386c0740a0 0x7f386c199b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:24:08.393 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:08.389+0000 7f3871600640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f386c075680 0x7f386c19a080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:24:08.393 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:08.389+0000 7f3871600640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f386c109550 0x7f386c1a10f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:24:08.393 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:08.389+0000 7f3871600640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f386c10f440 con 0x7f386c075680 2024-03-21T10:24:08.393 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:08.389+0000 7f3871600640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f386c10f2c0 con 0x7f386c109550 2024-03-21T10:24:08.393 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:08.389+0000 7f3871600640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f386c10f5c0 con 0x7f386c0740a0 2024-03-21T10:24:08.393 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:08.389+0000 7f386a000640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f386c075680 0x7f386c19a080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:24:08.393 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:08.389+0000 7f386a000640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f386c075680 0x7f386c19a080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:42294/0 (socket says 172.21.15.47:42294) 2024-03-21T10:24:08.393 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:08.389+0000 7f386a000640 1 -- 172.21.15.47:0/4155838175 learned_addr learned my addr 172.21.15.47:0/4155838175 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:24:08.393 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:08.390+0000 7f386b400640 1 --2- 172.21.15.47:0/4155838175 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f386c109550 0x7f386c1a10f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:24:08.393 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:08.390+0000 7f386aa00640 1 --2- 172.21.15.47:0/4155838175 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f386c0740a0 0x7f386c199b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:24:08.393 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:08.390+0000 7f386a000640 1 -- 172.21.15.47:0/4155838175 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f386c0740a0 msgr2=0x7f386c199b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:24:08.394 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:08.390+0000 7f386a000640 1 --2- 172.21.15.47:0/4155838175 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f386c0740a0 0x7f386c199b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:24:08.394 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:08.390+0000 7f386a000640 1 -- 172.21.15.47:0/4155838175 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f386c109550 msgr2=0x7f386c1a10f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:24:08.394 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:08.390+0000 7f386a000640 1 --2- 172.21.15.47:0/4155838175 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f386c109550 0x7f386c1a10f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:24:08.394 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:08.390+0000 7f386a000640 1 -- 172.21.15.47:0/4155838175 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f386c1a1760 con 0x7f386c075680 2024-03-21T10:24:08.394 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:08.390+0000 7f386b400640 1 --2- 172.21.15.47:0/4155838175 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f386c109550 0x7f386c1a10f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T10:24:08.394 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:08.390+0000 7f386a000640 1 --2- 172.21.15.47:0/4155838175 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f386c075680 0x7f386c19a080 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f3858000c00 tx=0x7f385802d990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:24:08.394 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:08.390+0000 7f3863400640 1 -- 172.21.15.47:0/4155838175 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3858043020 con 0x7f386c075680 2024-03-21T10:24:08.394 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:08.390+0000 7f3863400640 1 -- 172.21.15.47:0/4155838175 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f385802dd90 con 0x7f386c075680 2024-03-21T10:24:08.394 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:08.390+0000 7f3871600640 1 -- 172.21.15.47:0/4155838175 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f386c1a19f0 con 0x7f386c075680 2024-03-21T10:24:08.395 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:08.391+0000 7f3863400640 1 -- 172.21.15.47:0/4155838175 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3858003c20 con 0x7f386c075680 2024-03-21T10:24:08.395 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:08.391+0000 7f3871600640 1 -- 172.21.15.47:0/4155838175 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f386c1a1f40 con 0x7f386c075680 2024-03-21T10:24:08.395 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:08.392+0000 7f3871600640 1 -- 172.21.15.47:0/4155838175 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f386c108310 con 0x7f386c075680 2024-03-21T10:24:08.397 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:08.395+0000 7f3863400640 1 -- 172.21.15.47:0/4155838175 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 18) v1 ==== 137854+0+0 (secure 0 0 0) 0x7f3858046020 con 0x7f386c075680 2024-03-21T10:24:08.399 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:08.398+0000 7f3863400640 1 --2- 172.21.15.47:0/4155838175 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f38500a2f20 0x7f38500a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:24:08.400 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:08.398+0000 7f3863400640 1 -- 172.21.15.47:0/4155838175 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9216+0+0 (secure 0 0 0) 0x7f38580f1ed0 con 0x7f386c075680 2024-03-21T10:24:08.400 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:08.398+0000 7f386aa00640 1 --2- 172.21.15.47:0/4155838175 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f38500a2f20 0x7f38500a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:24:08.400 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:08.398+0000 7f3863400640 1 -- 172.21.15.47:0/4155838175 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f385804d020 con 0x7f386c075680 2024-03-21T10:24:08.401 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:08.399+0000 7f386aa00640 1 --2- 172.21.15.47:0/4155838175 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f38500a2f20 0x7f38500a53e0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f385c005fd0 tx=0x7f385c005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:24:08.608 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:08.606+0000 7f3871600640 1 -- 172.21.15.47:0/4155838175 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f386c02d240 con 0x7f386c075680 2024-03-21T10:24:08.611 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:08.609+0000 7f3863400640 1 -- 172.21.15.47:0/4155838175 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v95) v1 ==== 74+0+24849 (secure 0 0 0) 0x7f38580473b0 con 0x7f386c075680 2024-03-21T10:24:08.612 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-21T10:24:08.612 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":95,"fsid":"3056d51a-e76b-11ee-95cd-87774f69a715","created":"2024-03-21T10:11:17.019965+0000","modified":"2024-03-21T10:23:27.619804+0000","last_up_change":"2024-03-21T10:20:52.675328+0000","last_in_change":"2024-03-21T10:20:27.648212+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-21T10:15:40.553114+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-21T10:21:56.399518+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-21T10:21:57.812940+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-21T10:22:01.822568+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"91","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"b0c7a102-8d89-428b-b793-cbbe67151f50","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6802","nonce":775837313},{"type":"v1","addr":"172.21.15.47:6803","nonce":775837313}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6804","nonce":775837313},{"type":"v1","addr":"172.21.15.47:6805","nonce":775837313}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6808","nonce":775837313},{"type":"v1","addr":"172.21.15.47:6809","nonce":775837313}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6806","nonce":775837313},{"type":"v1","addr":"172.21.15.47:6807","nonce":775837313}]},"public_addr":"172.21.15.47:6803/775837313","cluster_addr":"172.21.15.47:6805/775837313","heartbeat_back_addr":"172.21.15.47:6809/775837313","heartbeat_front_addr":"172.21.15.47:6807/775837313","state":["exists","up"]},{"osd":1,"uuid":"07bb81bb-7cf3-4bbc-ac9e-802f95185d11","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6800","nonce":2923304710},{"type":"v1","addr":"172.21.15.100:6801","nonce":2923304710}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":2923304710},{"type":"v1","addr":"172.21.15.100:6803","nonce":2923304710}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":2923304710},{"type":"v1","addr":"172.21.15.100:6807","nonce":2923304710}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6804","nonce":2923304710},{"type":"v1","addr":"172.21.15.100:6805","nonce":2923304710}]},"public_addr":"172.21.15.100:6801/2923304710","cluster_addr":"172.21.15.100:6803/2923304710","heartbeat_back_addr":"172.21.15.100:6807/2923304710","heartbeat_front_addr":"172.21.15.100:6805/2923304710","state":["exists","up"]},{"osd":2,"uuid":"88ce76b9-6081-4836-aad0-8c249c56e8a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":171377005},{"type":"v1","addr":"172.21.15.157:6801","nonce":171377005}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":171377005},{"type":"v1","addr":"172.21.15.157:6803","nonce":171377005}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":171377005},{"type":"v1","addr":"172.21.15.157:6807","nonce":171377005}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":171377005},{"type":"v1","addr":"172.21.15.157:6805","nonce":171377005}]},"public_addr":"172.21.15.157:6801/171377005","cluster_addr":"172.21.15.157:6803/171377005","heartbeat_back_addr":"172.21.15.157:6807/171377005","heartbeat_front_addr":"172.21.15.157:6805/171377005","state":["exists","up"]},{"osd":3,"uuid":"518aa33d-263f-45e6-aab5-2201ceb40e74","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":27,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6810","nonce":640760101},{"type":"v1","addr":"172.21.15.47:6811","nonce":640760101}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6812","nonce":640760101},{"type":"v1","addr":"172.21.15.47:6813","nonce":640760101}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6816","nonce":640760101},{"type":"v1","addr":"172.21.15.47:6817","nonce":640760101}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6814","nonce":640760101},{"type":"v1","addr":"172.21.15.47:6815","nonce":640760101}]},"public_addr":"172.21.15.47:6811/640760101","cluster_addr":"172.21.15.47:6813/640760101","heartbeat_back_addr":"172.21.15.47:6817/640760101","heartbeat_front_addr":"172.21.15.47:6815/640760101","state":["exists","up"]},{"osd":4,"uuid":"987bde83-ecc6-42e5-bd56-c143b3de4226","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":35,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6808","nonce":2785478639},{"type":"v1","addr":"172.21.15.100:6809","nonce":2785478639}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6810","nonce":2785478639},{"type":"v1","addr":"172.21.15.100:6811","nonce":2785478639}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6814","nonce":2785478639},{"type":"v1","addr":"172.21.15.100:6815","nonce":2785478639}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":2785478639},{"type":"v1","addr":"172.21.15.100:6813","nonce":2785478639}]},"public_addr":"172.21.15.100:6809/2785478639","cluster_addr":"172.21.15.100:6811/2785478639","heartbeat_back_addr":"172.21.15.100:6815/2785478639","heartbeat_front_addr":"172.21.15.100:6813/2785478639","state":["exists","up"]},{"osd":5,"uuid":"012d2188-3a61-4a5f-8d01-2001c15ffbe2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":40,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":1525645410},{"type":"v1","addr":"172.21.15.157:6809","nonce":1525645410}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":1525645410},{"type":"v1","addr":"172.21.15.157:6811","nonce":1525645410}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":1525645410},{"type":"v1","addr":"172.21.15.157:6815","nonce":1525645410}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":1525645410},{"type":"v1","addr":"172.21.15.157:6813","nonce":1525645410}]},"public_addr":"172.21.15.157:6809/1525645410","cluster_addr":"172.21.15.157:6811/1525645410","heartbeat_back_addr":"172.21.15.157:6815/1525645410","heartbeat_front_addr":"172.21.15.157:6813/1525645410","state":["exists","up"]},{"osd":6,"uuid":"c12c14b4-85f0-4d88-937c-de4f556437de","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6818","nonce":2321642790},{"type":"v1","addr":"172.21.15.47:6819","nonce":2321642790}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6820","nonce":2321642790},{"type":"v1","addr":"172.21.15.47:6821","nonce":2321642790}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6824","nonce":2321642790},{"type":"v1","addr":"172.21.15.47:6825","nonce":2321642790}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6822","nonce":2321642790},{"type":"v1","addr":"172.21.15.47:6823","nonce":2321642790}]},"public_addr":"172.21.15.47:6819/2321642790","cluster_addr":"172.21.15.47:6821/2321642790","heartbeat_back_addr":"172.21.15.47:6825/2321642790","heartbeat_front_addr":"172.21.15.47:6823/2321642790","state":["exists","up"]},{"osd":7,"uuid":"3c87c6d0-1d1f-4294-9917-aaaad7dda824","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6816","nonce":4104727330},{"type":"v1","addr":"172.21.15.100:6817","nonce":4104727330}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":4104727330},{"type":"v1","addr":"172.21.15.100:6819","nonce":4104727330}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":4104727330},{"type":"v1","addr":"172.21.15.100:6823","nonce":4104727330}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6820","nonce":4104727330},{"type":"v1","addr":"172.21.15.100:6821","nonce":4104727330}]},"public_addr":"172.21.15.100:6817/4104727330","cluster_addr":"172.21.15.100:6819/4104727330","heartbeat_back_addr":"172.21.15.100:6823/4104727330","heartbeat_front_addr":"172.21.15.100:6821/4104727330","state":["exists","up"]},{"osd":8,"uuid":"502d65cf-53c8-4936-8de4-e11b2a6e1e18","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":61,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":4067715347},{"type":"v1","addr":"172.21.15.157:6817","nonce":4067715347}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":4067715347},{"type":"v1","addr":"172.21.15.157:6819","nonce":4067715347}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":4067715347},{"type":"v1","addr":"172.21.15.157:6823","nonce":4067715347}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":4067715347},{"type":"v1","addr":"172.21.15.157:6821","nonce":4067715347}]},"public_addr":"172.21.15.157:6817/4067715347","cluster_addr":"172.21.15.157:6819/4067715347","heartbeat_back_addr":"172.21.15.157:6823/4067715347","heartbeat_front_addr":"172.21.15.157:6821/4067715347","state":["exists","up"]},{"osd":9,"uuid":"5ea1f5be-8458-4c93-9e61-c2e867a514f3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6826","nonce":1982108738},{"type":"v1","addr":"172.21.15.47:6827","nonce":1982108738}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6828","nonce":1982108738},{"type":"v1","addr":"172.21.15.47:6829","nonce":1982108738}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6832","nonce":1982108738},{"type":"v1","addr":"172.21.15.47:6833","nonce":1982108738}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6830","nonce":1982108738},{"type":"v1","addr":"172.21.15.47:6831","nonce":1982108738}]},"public_addr":"172.21.15.47:6827/1982108738","cluster_addr":"172.21.15.47:6829/1982108738","heartbeat_back_addr":"172.21.15.47:6833/1982108738","heartbeat_front_addr":"172.21.15.47:6831/1982108738","state":["exists","up"]},{"osd":10,"uuid":"2da3da90-a0be-4a54-a9ff-b2d6c9b36870","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":74,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":396391028},{"type":"v1","addr":"172.21.15.100:6825","nonce":396391028}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":396391028},{"type":"v1","addr":"172.21.15.100:6827","nonce":396391028}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6830","nonce":396391028},{"type":"v1","addr":"172.21.15.100:6831","nonce":396391028}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":396391028},{"type":"v1","addr":"172.21.15.100:6829","nonce":396391028}]},"public_addr":"172.21.15.100:6825/396391028","cluster_addr":"172.21.15.100:6827/396391028","heartbeat_back_addr":"172.21.15.100:6831/396391028","heartbeat_front_addr":"172.21.15.100:6829/396391028","state":["exists","up"]},{"osd":11,"uuid":"11798d41-7f7e-4319-95b8-8d3d0946bde9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":82,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":2671029483},{"type":"v1","addr":"172.21.15.157:6825","nonce":2671029483}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":2671029483},{"type":"v1","addr":"172.21.15.157:6827","nonce":2671029483}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":2671029483},{"type":"v1","addr":"172.21.15.157:6831","nonce":2671029483}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":2671029483},{"type":"v1","addr":"172.21.15.157:6829","nonce":2671029483}]},"public_addr":"172.21.15.157:6825/2671029483","cluster_addr":"172.21.15.157:6827/2671029483","heartbeat_back_addr":"172.21.15.157:6831/2671029483","heartbeat_front_addr":"172.21.15.157:6829/2671029483","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T10:14:31.822399+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T10:15:04.175721+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T10:15:35.902595+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T10:16:12.088891+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T10:16:46.428704+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T10:17:20.158270+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T10:17:55.915253+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T10:18:30.064641+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T10:19:04.410004+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T10:19:40.605181+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T10:20:15.636605+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T10:20:50.151365+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.11","mappings":[{"from":7,"to":11}]},{"pgid":"2.14","mappings":[{"from":2,"to":4}]},{"pgid":"2.1a","mappings":[{"from":8,"to":1}]},{"pgid":"2.1b","mappings":[{"from":2,"to":4}]},{"pgid":"2.1d","mappings":[{"from":8,"to":11}]},{"pgid":"3.0","mappings":[{"from":4,"to":8},{"from":5,"to":9}]},{"pgid":"3.6","mappings":[{"from":3,"to":7}]},{"pgid":"3.1b","mappings":[{"from":4,"to":6}]},{"pgid":"3.33","mappings":[{"from":5,"to":9}]},{"pgid":"3.3c","mappings":[{"from":4,"to":9}]},{"pgid":"4.c","mappings":[{"from":9,"to":10}]},{"pgid":"4.f","mappings":[{"from":9,"to":8}]},{"pgid":"4.19","mappings":[{"from":9,"to":8}]},{"pgid":"4.1d","mappings":[{"from":5,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.47:0/356822431":"2024-03-22T10:12:25.844530+0000","172.21.15.47:0/1167490879":"2024-03-22T10:12:25.844530+0000","172.21.15.47:0/1540718138":"2024-03-22T10:12:25.844530+0000","172.21.15.47:6801/3836903972":"2024-03-22T10:12:25.844530+0000","172.21.15.47:0/235436602":"2024-03-22T10:11:50.301838+0000","172.21.15.47:0/2052815572":"2024-03-22T10:11:50.301838+0000","172.21.15.47:6800/3942658432":"2024-03-22T10:11:50.301838+0000","172.21.15.47:6801/3942658432":"2024-03-22T10:11:50.301838+0000","172.21.15.47:0/1419516257":"2024-03-22T10:11:50.301838+0000","172.21.15.47:6800/3836903972":"2024-03-22T10:12:25.844530+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-21T10:24:08.615 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:08.613+0000 7f3871600640 1 -- 172.21.15.47:0/4155838175 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f38500a2f20 msgr2=0x7f38500a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:24:08.615 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:08.613+0000 7f3871600640 1 --2- 172.21.15.47:0/4155838175 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f38500a2f20 0x7f38500a53e0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f385c005fd0 tx=0x7f385c005850 comp rx=0 tx=0).stop 2024-03-21T10:24:08.615 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:08.613+0000 7f3871600640 1 -- 172.21.15.47:0/4155838175 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f386c075680 msgr2=0x7f386c19a080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:24:08.615 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:08.613+0000 7f3871600640 1 --2- 172.21.15.47:0/4155838175 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f386c075680 0x7f386c19a080 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f3858000c00 tx=0x7f385802d990 comp rx=0 tx=0).stop 2024-03-21T10:24:08.615 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:08.613+0000 7f3871600640 1 -- 172.21.15.47:0/4155838175 shutdown_connections 2024-03-21T10:24:08.615 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:08.613+0000 7f3871600640 1 --2- 172.21.15.47:0/4155838175 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f38500a2f20 0x7f38500a53e0 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:24:08.615 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:08.613+0000 7f3871600640 1 --2- 172.21.15.47:0/4155838175 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f386c109550 0x7f386c1a10f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:24:08.615 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:08.613+0000 7f3871600640 1 --2- 172.21.15.47:0/4155838175 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f386c075680 0x7f386c19a080 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:24:08.616 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:08.613+0000 7f3871600640 1 --2- 172.21.15.47:0/4155838175 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f386c0740a0 0x7f386c199b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:24:08.616 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:08.613+0000 7f3871600640 1 -- 172.21.15.47:0/4155838175 >> 172.21.15.47:0/4155838175 conn(0x7f386c0fcc50 msgr2=0x7f386c1054e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:24:08.616 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:08.613+0000 7f3871600640 1 -- 172.21.15.47:0/4155838175 shutdown_connections 2024-03-21T10:24:08.616 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:08.613+0000 7f3871600640 1 -- 172.21.15.47:0/4155838175 wait complete. 2024-03-21T10:24:08.634 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-21T10:15:40.553114+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-21T10:21:56.399518+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '86', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-21T10:21:57.812940+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '87', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-03-21T10:22:01.822568+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '91', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-03-21T10:24:08.635 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-21T10:24:08.875 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:08.872+0000 7f7bb6400640 1 -- 172.21.15.47:0/2307545255 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7bb0101150 msgr2=0x7f7bb01015d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:24:08.875 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:08.872+0000 7f7bb6400640 1 --2- 172.21.15.47:0/2307545255 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7bb0101150 0x7f7bb01015d0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f7ba8009be0 tx=0x7f7ba802bef0 comp rx=0 tx=0).stop 2024-03-21T10:24:08.875 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:08.872+0000 7f7bb6400640 1 -- 172.21.15.47:0/2307545255 shutdown_connections 2024-03-21T10:24:08.875 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:08.872+0000 7f7bb6400640 1 --2- 172.21.15.47:0/2307545255 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7bb0101150 0x7f7bb01015d0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:24:08.875 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:08.872+0000 7f7bb6400640 1 --2- 172.21.15.47:0/2307545255 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7bb01007b0 0x7f7bb0100c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:24:08.875 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:08.872+0000 7f7bb6400640 1 --2- 172.21.15.47:0/2307545255 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7bb00ffe50 0x7f7bb0100270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:24:08.875 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:08.872+0000 7f7bb6400640 1 -- 172.21.15.47:0/2307545255 >> 172.21.15.47:0/2307545255 conn(0x7f7bb00fba10 msgr2=0x7f7bb00fde30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:24:08.875 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:08.872+0000 7f7bb6400640 1 -- 172.21.15.47:0/2307545255 shutdown_connections 2024-03-21T10:24:08.875 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:08.872+0000 7f7bb6400640 1 -- 172.21.15.47:0/2307545255 wait complete. 2024-03-21T10:24:08.876 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:08.874+0000 7f7bb6400640 1 Processor -- start 2024-03-21T10:24:08.876 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:08.874+0000 7f7bb6400640 1 -- start start 2024-03-21T10:24:08.877 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:08.875+0000 7f7bb6400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7bb00ffe50 0x7f7bb0199930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:24:08.877 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:08.875+0000 7f7bb6400640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7bb01007b0 0x7f7bb0199e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:24:08.877 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:08.875+0000 7f7bb6400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7bb0101150 0x7f7bb01a0ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:24:08.878 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:08.875+0000 7f7bb6400640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7bb010f2f0 con 0x7f7bb0101150 2024-03-21T10:24:08.878 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:08.875+0000 7f7bb6400640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7bb010f170 con 0x7f7bb00ffe50 2024-03-21T10:24:08.878 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:08.875+0000 7f7bb6400640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7bb010f470 con 0x7f7bb01007b0 2024-03-21T10:24:08.878 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:08.875+0000 7f7bb5000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7bb00ffe50 0x7f7bb0199930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:24:08.878 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:08.875+0000 7f7bb5000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7bb00ffe50 0x7f7bb0199930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:53862/0 (socket says 172.21.15.47:53862) 2024-03-21T10:24:08.878 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:08.875+0000 7f7bb5000640 1 -- 172.21.15.47:0/752184441 learned_addr learned my addr 172.21.15.47:0/752184441 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:24:08.878 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:08.875+0000 7f7bb5a00640 1 --2- 172.21.15.47:0/752184441 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7bb0101150 0x7f7bb01a0ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:24:08.878 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:08.875+0000 7f7bafe00640 1 --2- 172.21.15.47:0/752184441 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7bb01007b0 0x7f7bb0199e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:24:08.879 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:08.876+0000 7f7bb5000640 1 -- 172.21.15.47:0/752184441 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7bb01007b0 msgr2=0x7f7bb0199e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:24:08.879 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:08.876+0000 7f7bb5000640 1 --2- 172.21.15.47:0/752184441 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7bb01007b0 0x7f7bb0199e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:24:08.879 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:08.876+0000 7f7bb5000640 1 -- 172.21.15.47:0/752184441 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7bb0101150 msgr2=0x7f7bb01a0ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:24:08.879 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:08.876+0000 7f7bb5000640 1 --2- 172.21.15.47:0/752184441 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7bb0101150 0x7f7bb01a0ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:24:08.879 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:08.876+0000 7f7bb5000640 1 -- 172.21.15.47:0/752184441 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7bb01a1550 con 0x7f7bb00ffe50 2024-03-21T10:24:08.879 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:08.876+0000 7f7bb5a00640 1 --2- 172.21.15.47:0/752184441 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7bb0101150 0x7f7bb01a0ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T10:24:08.880 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:08.876+0000 7f7bb5000640 1 --2- 172.21.15.47:0/752184441 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7bb00ffe50 0x7f7bb0199930 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f7ba000e9f0 tx=0x7f7ba000edf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:24:08.881 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:08.878+0000 7f7bad600640 1 -- 172.21.15.47:0/752184441 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ba000cd80 con 0x7f7bb00ffe50 2024-03-21T10:24:08.881 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:08.878+0000 7f7bb6400640 1 -- 172.21.15.47:0/752184441 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7bb01a1840 con 0x7f7bb00ffe50 2024-03-21T10:24:08.881 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:08.878+0000 7f7bad600640 1 -- 172.21.15.47:0/752184441 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f7ba00054d0 con 0x7f7bb00ffe50 2024-03-21T10:24:08.881 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:08.878+0000 7f7bad600640 1 -- 172.21.15.47:0/752184441 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ba0010490 con 0x7f7bb00ffe50 2024-03-21T10:24:08.881 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:08.878+0000 7f7bb6400640 1 -- 172.21.15.47:0/752184441 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7bb01a1d90 con 0x7f7bb00ffe50 2024-03-21T10:24:08.882 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:08.879+0000 7f7bb6400640 1 -- 172.21.15.47:0/752184441 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7bb010ad30 con 0x7f7bb00ffe50 2024-03-21T10:24:08.884 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:08.882+0000 7f7bad600640 1 -- 172.21.15.47:0/752184441 <== mon.2 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 18) v1 ==== 137854+0+0 (secure 0 0 0) 0x7f7ba0005000 con 0x7f7bb00ffe50 2024-03-21T10:24:08.886 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:08.883+0000 7f7bad600640 1 --2- 172.21.15.47:0/752184441 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f7b940a2fa0 0x7f7b940a5460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:24:08.886 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:08.884+0000 7f7bad600640 1 -- 172.21.15.47:0/752184441 <== mon.2 v2:172.21.15.100:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9216+0+0 (secure 0 0 0) 0x7f7ba00cfd80 con 0x7f7bb00ffe50 2024-03-21T10:24:08.886 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:08.884+0000 7f7bafe00640 1 --2- 172.21.15.47:0/752184441 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f7b940a2fa0 0x7f7b940a5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:24:08.887 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:08.885+0000 7f7bafe00640 1 --2- 172.21.15.47:0/752184441 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f7b940a2fa0 0x7f7b940a5460 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f7bb019ae40 tx=0x7f7b9c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:24:08.894 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:08.892+0000 7f7bad600640 1 -- 172.21.15.47:0/752184441 <== mon.2 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f7ba0053740 con 0x7f7bb00ffe50 2024-03-21T10:24:08.964 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:24:08 smithi047 ceph-mon[31547]: from='client.? 172.21.15.47:0/3518280300' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-21T10:24:08.964 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:24:08 smithi047 ceph-mon[31547]: from='client.? 172.21.15.47:0/4155838175' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-21T10:24:09.101 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:09.098+0000 7f7bb6400640 1 -- 172.21.15.47:0/752184441 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f7bb002d200 con 0x7f7bb00ffe50 2024-03-21T10:24:09.101 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:09.099+0000 7f7bad600640 1 -- 172.21.15.47:0/752184441 <== mon.2 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v95) v1 ==== 93+0+10 (secure 0 0 0) 0x7f7ba00052f0 con 0x7f7bb00ffe50 2024-03-21T10:24:09.102 INFO:teuthology.orchestra.run.smithi047.stdout:pg_num: 1 2024-03-21T10:24:09.105 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:09.102+0000 7f7bb6400640 1 -- 172.21.15.47:0/752184441 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f7b940a2fa0 msgr2=0x7f7b940a5460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:24:09.105 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:09.102+0000 7f7bb6400640 1 --2- 172.21.15.47:0/752184441 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f7b940a2fa0 0x7f7b940a5460 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f7bb019ae40 tx=0x7f7b9c009260 comp rx=0 tx=0).stop 2024-03-21T10:24:09.105 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:09.102+0000 7f7bb6400640 1 -- 172.21.15.47:0/752184441 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7bb00ffe50 msgr2=0x7f7bb0199930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:24:09.105 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:09.102+0000 7f7bb6400640 1 --2- 172.21.15.47:0/752184441 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7bb00ffe50 0x7f7bb0199930 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f7ba000e9f0 tx=0x7f7ba000edf0 comp rx=0 tx=0).stop 2024-03-21T10:24:09.105 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:09.102+0000 7f7bb6400640 1 -- 172.21.15.47:0/752184441 shutdown_connections 2024-03-21T10:24:09.105 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:09.102+0000 7f7bb6400640 1 --2- 172.21.15.47:0/752184441 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f7b940a2fa0 0x7f7b940a5460 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:24:09.105 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:09.102+0000 7f7bb6400640 1 --2- 172.21.15.47:0/752184441 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7bb0101150 0x7f7bb01a0ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:24:09.105 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:09.102+0000 7f7bb6400640 1 --2- 172.21.15.47:0/752184441 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7bb01007b0 0x7f7bb0199e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:24:09.105 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:09.102+0000 7f7bb6400640 1 --2- 172.21.15.47:0/752184441 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7bb00ffe50 0x7f7bb0199930 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:24:09.105 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:09.102+0000 7f7bb6400640 1 -- 172.21.15.47:0/752184441 >> 172.21.15.47:0/752184441 conn(0x7f7bb00fba10 msgr2=0x7f7bb00fd0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:24:09.105 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:09.103+0000 7f7bb6400640 1 -- 172.21.15.47:0/752184441 shutdown_connections 2024-03-21T10:24:09.105 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:09.103+0000 7f7bb6400640 1 -- 172.21.15.47:0/752184441 wait complete. 2024-03-21T10:24:09.126 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-21T10:24:09.230 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:24:08 smithi100 ceph-mon[35193]: from='client.? 172.21.15.47:0/3518280300' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-21T10:24:09.230 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:24:08 smithi100 ceph-mon[35193]: from='client.? 172.21.15.47:0/4155838175' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-21T10:24:09.240 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:24:08 smithi157 ceph-mon[35581]: from='client.? 172.21.15.47:0/3518280300' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-21T10:24:09.240 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:24:08 smithi157 ceph-mon[35581]: from='client.? 172.21.15.47:0/4155838175' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-21T10:24:09.352 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:09.350+0000 7f4823200640 1 -- 172.21.15.47:0/3033451963 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4824105350 msgr2=0x7f482410bbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:24:09.352 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:09.350+0000 7f4823200640 1 --2- 172.21.15.47:0/3033451963 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4824105350 0x7f482410bbe0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f481c009ca0 tx=0x7f481c02c100 comp rx=0 tx=0).stop 2024-03-21T10:24:09.353 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:09.350+0000 7f4823200640 1 -- 172.21.15.47:0/3033451963 shutdown_connections 2024-03-21T10:24:09.353 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:09.350+0000 7f4823200640 1 --2- 172.21.15.47:0/3033451963 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4824105350 0x7f482410bbe0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:24:09.353 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:09.350+0000 7f4823200640 1 --2- 172.21.15.47:0/3033451963 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4824104990 0x7f4824104e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:24:09.353 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:09.350+0000 7f4823200640 1 --2- 172.21.15.47:0/3033451963 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f48241037a0 0x7f4824103bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:24:09.353 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:09.350+0000 7f4823200640 1 -- 172.21.15.47:0/3033451963 >> 172.21.15.47:0/3033451963 conn(0x7f48240fed10 msgr2=0x7f4824101150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:24:09.353 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:09.350+0000 7f4823200640 1 -- 172.21.15.47:0/3033451963 shutdown_connections 2024-03-21T10:24:09.353 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:09.350+0000 7f4823200640 1 -- 172.21.15.47:0/3033451963 wait complete. 2024-03-21T10:24:09.355 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:09.351+0000 7f4823200640 1 Processor -- start 2024-03-21T10:24:09.355 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:09.351+0000 7f4823200640 1 -- start start 2024-03-21T10:24:09.356 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:09.351+0000 7f4823200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f48241037a0 0x7f4824199920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:24:09.356 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:09.352+0000 7f4823200640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4824104990 0x7f4824199e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:24:09.356 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:09.352+0000 7f4823200640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4824105350 0x7f48241a0ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:24:09.356 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:09.352+0000 7f4823200640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f482410f2d0 con 0x7f4824104990 2024-03-21T10:24:09.356 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:09.352+0000 7f4823200640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f482410f150 con 0x7f48241037a0 2024-03-21T10:24:09.356 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:09.352+0000 7f4823200640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f482410f450 con 0x7f4824105350 2024-03-21T10:24:09.356 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:09.352+0000 7f4821400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4824104990 0x7f4824199e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:24:09.356 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:09.352+0000 7f4821e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f48241037a0 0x7f4824199920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:24:09.356 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:09.352+0000 7f4821400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4824104990 0x7f4824199e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:42320/0 (socket says 172.21.15.47:42320) 2024-03-21T10:24:09.356 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:09.352+0000 7f4821400640 1 -- 172.21.15.47:0/3385580472 learned_addr learned my addr 172.21.15.47:0/3385580472 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:24:09.356 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:09.352+0000 7f4822800640 1 --2- 172.21.15.47:0/3385580472 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4824105350 0x7f48241a0ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:24:09.356 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:09.352+0000 7f4821400640 1 -- 172.21.15.47:0/3385580472 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4824105350 msgr2=0x7f48241a0ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:24:09.356 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:09.352+0000 7f4821400640 1 --2- 172.21.15.47:0/3385580472 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4824105350 0x7f48241a0ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:24:09.356 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:09.352+0000 7f4821400640 1 -- 172.21.15.47:0/3385580472 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f48241037a0 msgr2=0x7f4824199920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:24:09.357 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:09.352+0000 7f4821400640 1 --2- 172.21.15.47:0/3385580472 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f48241037a0 0x7f4824199920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:24:09.357 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:09.353+0000 7f4821400640 1 -- 172.21.15.47:0/3385580472 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48241a1540 con 0x7f4824104990 2024-03-21T10:24:09.357 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:09.353+0000 7f4821400640 1 --2- 172.21.15.47:0/3385580472 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4824104990 0x7f4824199e60 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f481000d950 tx=0x7f481000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:24:09.357 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:09.353+0000 7f481aa00640 1 -- 172.21.15.47:0/3385580472 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4810017030 con 0x7f4824104990 2024-03-21T10:24:09.357 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:09.353+0000 7f481aa00640 1 -- 172.21.15.47:0/3385580472 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f4810004020 con 0x7f4824104990 2024-03-21T10:24:09.357 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:09.353+0000 7f481aa00640 1 -- 172.21.15.47:0/3385580472 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48100047a0 con 0x7f4824104990 2024-03-21T10:24:09.357 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:09.353+0000 7f4823200640 1 -- 172.21.15.47:0/3385580472 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f48241a1830 con 0x7f4824104990 2024-03-21T10:24:09.357 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:09.353+0000 7f4823200640 1 -- 172.21.15.47:0/3385580472 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f48241a1d30 con 0x7f4824104990 2024-03-21T10:24:09.357 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:09.355+0000 7f4823200640 1 -- 172.21.15.47:0/3385580472 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f482410aac0 con 0x7f4824104990 2024-03-21T10:24:09.361 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:09.359+0000 7f481aa00640 1 -- 172.21.15.47:0/3385580472 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 18) v1 ==== 137854+0+0 (secure 0 0 0) 0x7f48100041d0 con 0x7f4824104990 2024-03-21T10:24:09.363 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:09.361+0000 7f481aa00640 1 --2- 172.21.15.47:0/3385580472 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f48080a2fa0 0x7f48080a5460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:24:09.363 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:09.361+0000 7f481aa00640 1 -- 172.21.15.47:0/3385580472 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9216+0+0 (secure 0 0 0) 0x7f48100cf830 con 0x7f4824104990 2024-03-21T10:24:09.363 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:09.361+0000 7f4821e00640 1 --2- 172.21.15.47:0/3385580472 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f48080a2fa0 0x7f48080a5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:24:09.364 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:09.362+0000 7f4821e00640 1 --2- 172.21.15.47:0/3385580472 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f48080a2fa0 0x7f48080a5460 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f48241047f0 tx=0x7f4814005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:24:09.365 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:09.363+0000 7f481aa00640 1 -- 172.21.15.47:0/3385580472 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f481009c310 con 0x7f4824104990 2024-03-21T10:24:09.568 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:09.566+0000 7f4823200640 1 -- 172.21.15.47:0/3385580472 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f482402d200 con 0x7f4824104990 2024-03-21T10:24:09.569 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:09.567+0000 7f481aa00640 1 -- 172.21.15.47:0/3385580472 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v95) v1 ==== 104+0+11 (secure 0 0 0) 0x7f481009bc10 con 0x7f4824104990 2024-03-21T10:24:09.569 INFO:teuthology.orchestra.run.smithi047.stdout:pg_num: 32 2024-03-21T10:24:09.572 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:09.570+0000 7f4823200640 1 -- 172.21.15.47:0/3385580472 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f48080a2fa0 msgr2=0x7f48080a5460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:24:09.573 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:09.570+0000 7f4823200640 1 --2- 172.21.15.47:0/3385580472 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f48080a2fa0 0x7f48080a5460 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f48241047f0 tx=0x7f4814005f90 comp rx=0 tx=0).stop 2024-03-21T10:24:09.573 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:09.570+0000 7f4823200640 1 -- 172.21.15.47:0/3385580472 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4824104990 msgr2=0x7f4824199e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:24:09.573 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:09.570+0000 7f4823200640 1 --2- 172.21.15.47:0/3385580472 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4824104990 0x7f4824199e60 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f481000d950 tx=0x7f481000de10 comp rx=0 tx=0).stop 2024-03-21T10:24:09.573 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:09.570+0000 7f4823200640 1 -- 172.21.15.47:0/3385580472 shutdown_connections 2024-03-21T10:24:09.573 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:09.570+0000 7f4823200640 1 --2- 172.21.15.47:0/3385580472 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f48080a2fa0 0x7f48080a5460 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:24:09.573 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:09.570+0000 7f4823200640 1 --2- 172.21.15.47:0/3385580472 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4824105350 0x7f48241a0ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:24:09.573 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:09.570+0000 7f4823200640 1 --2- 172.21.15.47:0/3385580472 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4824104990 0x7f4824199e60 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:24:09.573 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:09.570+0000 7f4823200640 1 --2- 172.21.15.47:0/3385580472 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f48241037a0 0x7f4824199920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:24:09.573 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:09.570+0000 7f4823200640 1 -- 172.21.15.47:0/3385580472 >> 172.21.15.47:0/3385580472 conn(0x7f48240fed10 msgr2=0x7f4824107c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:24:09.574 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:09.570+0000 7f4823200640 1 -- 172.21.15.47:0/3385580472 shutdown_connections 2024-03-21T10:24:09.574 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:09.570+0000 7f4823200640 1 -- 172.21.15.47:0/3385580472 wait complete. 2024-03-21T10:24:09.590 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-21T10:24:09.812 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:09.810+0000 7fc189400640 1 -- 172.21.15.47:0/182298416 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc184102150 msgr2=0x7fc1841089e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:24:09.812 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:09.810+0000 7fc189400640 1 --2- 172.21.15.47:0/182298416 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc184102150 0x7fc1841089e0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7fc17c009ca0 tx=0x7fc17c02c100 comp rx=0 tx=0).stop 2024-03-21T10:24:09.813 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:09.810+0000 7fc189400640 1 -- 172.21.15.47:0/182298416 shutdown_connections 2024-03-21T10:24:09.813 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:09.810+0000 7fc189400640 1 --2- 172.21.15.47:0/182298416 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc184102150 0x7fc1841089e0 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:24:09.813 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:09.810+0000 7fc189400640 1 --2- 172.21.15.47:0/182298416 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc184101790 0x7fc184101c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:24:09.813 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:09.810+0000 7fc189400640 1 --2- 172.21.15.47:0/182298416 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc1841005a0 0x7fc1841009c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:24:09.813 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:09.810+0000 7fc189400640 1 -- 172.21.15.47:0/182298416 >> 172.21.15.47:0/182298416 conn(0x7fc1840fbb30 msgr2=0x7fc1840fdf50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:24:09.813 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:09.810+0000 7fc189400640 1 -- 172.21.15.47:0/182298416 shutdown_connections 2024-03-21T10:24:09.813 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:09.810+0000 7fc189400640 1 -- 172.21.15.47:0/182298416 wait complete. 2024-03-21T10:24:09.815 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:09.811+0000 7fc189400640 1 Processor -- start 2024-03-21T10:24:09.815 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:09.811+0000 7fc189400640 1 -- start start 2024-03-21T10:24:09.815 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:09.812+0000 7fc189400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc1841005a0 0x7fc184199af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:24:09.815 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:09.812+0000 7fc189400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc184101790 0x7fc18419a030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:24:09.815 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:09.812+0000 7fc189400640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc184102150 0x7fc1841a10a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:24:09.815 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:09.812+0000 7fc189400640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc18410c0b0 con 0x7fc1841005a0 2024-03-21T10:24:09.816 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:09.812+0000 7fc189400640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc18410bf30 con 0x7fc184101790 2024-03-21T10:24:09.816 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:09.812+0000 7fc189400640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc18410c230 con 0x7fc184102150 2024-03-21T10:24:09.816 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:09.812+0000 7fc182800640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc1841005a0 0x7fc184199af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:24:09.816 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:09.812+0000 7fc182800640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc1841005a0 0x7fc184199af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:42334/0 (socket says 172.21.15.47:42334) 2024-03-21T10:24:09.816 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:09.812+0000 7fc182800640 1 -- 172.21.15.47:0/3868199973 learned_addr learned my addr 172.21.15.47:0/3868199973 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:24:09.816 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:09.812+0000 7fc181e00640 1 --2- 172.21.15.47:0/3868199973 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc184101790 0x7fc18419a030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:24:09.816 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:09.812+0000 7fc183200640 1 --2- 172.21.15.47:0/3868199973 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc184102150 0x7fc1841a10a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:24:09.816 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:09.812+0000 7fc182800640 1 -- 172.21.15.47:0/3868199973 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc184102150 msgr2=0x7fc1841a10a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:24:09.816 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:09.812+0000 7fc182800640 1 --2- 172.21.15.47:0/3868199973 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc184102150 0x7fc1841a10a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:24:09.816 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:09.812+0000 7fc182800640 1 -- 172.21.15.47:0/3868199973 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc184101790 msgr2=0x7fc18419a030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:24:09.816 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:09.812+0000 7fc182800640 1 --2- 172.21.15.47:0/3868199973 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc184101790 0x7fc18419a030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:24:09.816 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:09.812+0000 7fc182800640 1 -- 172.21.15.47:0/3868199973 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc1841a1710 con 0x7fc1841005a0 2024-03-21T10:24:09.816 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:09.813+0000 7fc182800640 1 --2- 172.21.15.47:0/3868199973 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc1841005a0 0x7fc184199af0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7fc17400e9c0 tx=0x7fc17400ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:24:09.816 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:09.813+0000 7fc17b400640 1 -- 172.21.15.47:0/3868199973 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc17400cd80 con 0x7fc1841005a0 2024-03-21T10:24:09.817 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:09.813+0000 7fc17b400640 1 -- 172.21.15.47:0/3868199973 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fc1740035f0 con 0x7fc1841005a0 2024-03-21T10:24:09.817 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:09.813+0000 7fc17b400640 1 -- 172.21.15.47:0/3868199973 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc174010690 con 0x7fc1841005a0 2024-03-21T10:24:09.817 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:09.813+0000 7fc189400640 1 -- 172.21.15.47:0/3868199973 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc1841a1a00 con 0x7fc1841005a0 2024-03-21T10:24:09.817 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:09.813+0000 7fc189400640 1 -- 172.21.15.47:0/3868199973 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc1841a1f00 con 0x7fc1841005a0 2024-03-21T10:24:09.817 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:09.815+0000 7fc189400640 1 -- 172.21.15.47:0/3868199973 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc1841a2190 con 0x7fc1841005a0 2024-03-21T10:24:09.818 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:09.816+0000 7fc17b400640 1 -- 172.21.15.47:0/3868199973 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 18) v1 ==== 137854+0+0 (secure 0 0 0) 0x7fc1740037a0 con 0x7fc1841005a0 2024-03-21T10:24:09.819 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:09.818+0000 7fc17b400640 1 --2- 172.21.15.47:0/3868199973 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7fc1680a2f20 0x7fc1680a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:24:09.820 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:09.818+0000 7fc17b400640 1 -- 172.21.15.47:0/3868199973 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9216+0+0 (secure 0 0 0) 0x7fc1740cfce0 con 0x7fc1841005a0 2024-03-21T10:24:09.820 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:09.818+0000 7fc181e00640 1 --2- 172.21.15.47:0/3868199973 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7fc1680a2f20 0x7fc1680a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:24:09.820 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:09.819+0000 7fc181e00640 1 --2- 172.21.15.47:0/3868199973 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7fc1680a2f20 0x7fc1680a53e0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fc170005fd0 tx=0x7fc170005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:24:09.828 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:09.826+0000 7fc17b400640 1 -- 172.21.15.47:0/3868199973 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fc17409c700 con 0x7fc1841005a0 2024-03-21T10:24:10.049 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:10.047+0000 7fc189400640 1 -- 172.21.15.47:0/3868199973 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fc1841a2900 con 0x7fc1841005a0 2024-03-21T10:24:10.050 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:10.048+0000 7fc17b400640 1 -- 172.21.15.47:0/3868199973 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v95) v1 ==== 100+0+11 (secure 0 0 0) 0x7fc17409c000 con 0x7fc1841005a0 2024-03-21T10:24:10.050 INFO:teuthology.orchestra.run.smithi047.stdout:pg_num: 64 2024-03-21T10:24:10.054 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:10.052+0000 7fc189400640 1 -- 172.21.15.47:0/3868199973 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7fc1680a2f20 msgr2=0x7fc1680a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:24:10.054 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:10.052+0000 7fc189400640 1 --2- 172.21.15.47:0/3868199973 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7fc1680a2f20 0x7fc1680a53e0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fc170005fd0 tx=0x7fc170005850 comp rx=0 tx=0).stop 2024-03-21T10:24:10.054 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:10.052+0000 7fc189400640 1 -- 172.21.15.47:0/3868199973 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc1841005a0 msgr2=0x7fc184199af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:24:10.054 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:10.052+0000 7fc189400640 1 --2- 172.21.15.47:0/3868199973 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc1841005a0 0x7fc184199af0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7fc17400e9c0 tx=0x7fc17400ee80 comp rx=0 tx=0).stop 2024-03-21T10:24:10.054 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:10.052+0000 7fc189400640 1 -- 172.21.15.47:0/3868199973 shutdown_connections 2024-03-21T10:24:10.054 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:10.052+0000 7fc189400640 1 --2- 172.21.15.47:0/3868199973 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7fc1680a2f20 0x7fc1680a53e0 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:24:10.055 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:10.052+0000 7fc189400640 1 --2- 172.21.15.47:0/3868199973 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc184102150 0x7fc1841a10a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:24:10.055 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:10.052+0000 7fc189400640 1 --2- 172.21.15.47:0/3868199973 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc184101790 0x7fc18419a030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:24:10.055 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:10.052+0000 7fc189400640 1 --2- 172.21.15.47:0/3868199973 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc1841005a0 0x7fc184199af0 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:24:10.055 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:10.052+0000 7fc189400640 1 -- 172.21.15.47:0/3868199973 >> 172.21.15.47:0/3868199973 conn(0x7fc1840fbb30 msgr2=0x7fc1840688f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:24:10.055 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:10.052+0000 7fc189400640 1 -- 172.21.15.47:0/3868199973 shutdown_connections 2024-03-21T10:24:10.055 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:10.052+0000 7fc189400640 1 -- 172.21.15.47:0/3868199973 wait complete. 2024-03-21T10:24:10.080 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-03-21T10:24:10.157 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:24:09 smithi047 ceph-mon[31547]: pgmap v436: 129 pgs: 129 active+clean; 642 KiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 426 B/s wr, 10 op/s 2024-03-21T10:24:10.157 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:24:09 smithi047 ceph-mon[31547]: from='client.? 172.21.15.47:0/752184441' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-21T10:24:10.157 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:24:09 smithi047 ceph-mon[31547]: from='client.? 172.21.15.47:0/3385580472' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-21T10:24:10.229 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:24:09 smithi100 ceph-mon[35193]: pgmap v436: 129 pgs: 129 active+clean; 642 KiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 426 B/s wr, 10 op/s 2024-03-21T10:24:10.230 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:24:09 smithi100 ceph-mon[35193]: from='client.? 172.21.15.47:0/752184441' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-21T10:24:10.230 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:24:09 smithi100 ceph-mon[35193]: from='client.? 172.21.15.47:0/3385580472' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-21T10:24:10.240 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:24:09 smithi157 ceph-mon[35581]: pgmap v436: 129 pgs: 129 active+clean; 642 KiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 426 B/s wr, 10 op/s 2024-03-21T10:24:10.240 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:24:09 smithi157 ceph-mon[35581]: from='client.? 172.21.15.47:0/752184441' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-21T10:24:10.240 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:24:09 smithi157 ceph-mon[35581]: from='client.? 172.21.15.47:0/3385580472' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-21T10:24:10.299 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:10.296+0000 7fbef5400640 1 -- 172.21.15.47:0/4051215972 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbef010db70 msgr2=0x7fbef010ff60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:24:10.299 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:10.296+0000 7fbef5400640 1 --2- 172.21.15.47:0/4051215972 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbef010db70 0x7fbef010ff60 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7fbee8009cd0 tx=0x7fbee802c0e0 comp rx=0 tx=0).stop 2024-03-21T10:24:10.299 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:10.296+0000 7fbef5400640 1 -- 172.21.15.47:0/4051215972 shutdown_connections 2024-03-21T10:24:10.299 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:10.296+0000 7fbef5400640 1 --2- 172.21.15.47:0/4051215972 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbef010db70 0x7fbef010ff60 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:24:10.299 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:10.296+0000 7fbef5400640 1 --2- 172.21.15.47:0/4051215972 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbef0074100 0x7fbef0074580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:24:10.299 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:10.296+0000 7fbef5400640 1 --2- 172.21.15.47:0/4051215972 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbef00756e0 0x7fbef0073b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:24:10.300 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:10.296+0000 7fbef5400640 1 -- 172.21.15.47:0/4051215972 >> 172.21.15.47:0/4051215972 conn(0x7fbef00fcb10 msgr2=0x7fbef00fef50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:24:10.300 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:10.296+0000 7fbef5400640 1 -- 172.21.15.47:0/4051215972 shutdown_connections 2024-03-21T10:24:10.300 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:10.296+0000 7fbef5400640 1 -- 172.21.15.47:0/4051215972 wait complete. 2024-03-21T10:24:10.302 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:10.297+0000 7fbef5400640 1 Processor -- start 2024-03-21T10:24:10.302 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:10.298+0000 7fbef5400640 1 -- start start 2024-03-21T10:24:10.302 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:10.298+0000 7fbef5400640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbef0074100 0x7fbef019ddc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:24:10.302 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:10.298+0000 7fbef5400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbef00756e0 0x7fbef019e300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:24:10.302 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:10.298+0000 7fbef5400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbef010db70 0x7fbef01a5330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:24:10.302 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:10.298+0000 7fbef5400640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbef01136c0 con 0x7fbef00756e0 2024-03-21T10:24:10.302 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:10.298+0000 7fbef5400640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbef0113540 con 0x7fbef010db70 2024-03-21T10:24:10.303 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:10.298+0000 7fbef5400640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbef0113840 con 0x7fbef0074100 2024-03-21T10:24:10.303 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:10.298+0000 7fbee7e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbef00756e0 0x7fbef019e300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:24:10.303 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:10.298+0000 7fbee7e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbef00756e0 0x7fbef019e300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:42350/0 (socket says 172.21.15.47:42350) 2024-03-21T10:24:10.303 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:10.298+0000 7fbee7e00640 1 -- 172.21.15.47:0/3206335175 learned_addr learned my addr 172.21.15.47:0/3206335175 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:24:10.303 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:10.298+0000 7fbef4a00640 1 --2- 172.21.15.47:0/3206335175 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbef010db70 0x7fbef01a5330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:24:10.303 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:10.299+0000 7fbeefe00640 1 --2- 172.21.15.47:0/3206335175 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbef0074100 0x7fbef019ddc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:24:10.303 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:10.299+0000 7fbee7e00640 1 -- 172.21.15.47:0/3206335175 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbef0074100 msgr2=0x7fbef019ddc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:24:10.303 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:10.299+0000 7fbee7e00640 1 --2- 172.21.15.47:0/3206335175 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbef0074100 0x7fbef019ddc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:24:10.303 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:10.299+0000 7fbee7e00640 1 -- 172.21.15.47:0/3206335175 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbef010db70 msgr2=0x7fbef01a5330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:24:10.303 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:10.299+0000 7fbee7e00640 1 --2- 172.21.15.47:0/3206335175 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbef010db70 0x7fbef01a5330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:24:10.303 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:10.299+0000 7fbee7e00640 1 -- 172.21.15.47:0/3206335175 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbef01a59d0 con 0x7fbef00756e0 2024-03-21T10:24:10.303 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:10.299+0000 7fbeefe00640 1 --2- 172.21.15.47:0/3206335175 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbef0074100 0x7fbef019ddc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T10:24:10.304 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:10.299+0000 7fbee7e00640 1 --2- 172.21.15.47:0/3206335175 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbef00756e0 0x7fbef019e300 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7fbed800e9f0 tx=0x7fbed800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:24:10.304 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:10.299+0000 7fbeed600640 1 -- 172.21.15.47:0/3206335175 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbed800cd80 con 0x7fbef00756e0 2024-03-21T10:24:10.304 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:10.299+0000 7fbeed600640 1 -- 172.21.15.47:0/3206335175 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fbed80108e0 con 0x7fbef00756e0 2024-03-21T10:24:10.304 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:10.299+0000 7fbeed600640 1 -- 172.21.15.47:0/3206335175 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbed8018920 con 0x7fbef00756e0 2024-03-21T10:24:10.304 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:10.299+0000 7fbef5400640 1 -- 172.21.15.47:0/3206335175 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbef01a5cc0 con 0x7fbef00756e0 2024-03-21T10:24:10.304 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:10.300+0000 7fbef5400640 1 -- 172.21.15.47:0/3206335175 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbef01a61c0 con 0x7fbef00756e0 2024-03-21T10:24:10.304 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:10.301+0000 7fbef5400640 1 -- 172.21.15.47:0/3206335175 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbef0103c30 con 0x7fbef00756e0 2024-03-21T10:24:10.305 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:10.303+0000 7fbeed600640 1 -- 172.21.15.47:0/3206335175 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 18) v1 ==== 137854+0+0 (secure 0 0 0) 0x7fbed8010a90 con 0x7fbef00756e0 2024-03-21T10:24:10.306 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:10.304+0000 7fbeed600640 1 --2- 172.21.15.47:0/3206335175 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7fbed40a2fa0 0x7fbed40a5460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:24:10.307 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:10.304+0000 7fbeed600640 1 -- 172.21.15.47:0/3206335175 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9216+0+0 (secure 0 0 0) 0x7fbed80cfb70 con 0x7fbef00756e0 2024-03-21T10:24:10.307 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:10.304+0000 7fbeefe00640 1 --2- 172.21.15.47:0/3206335175 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7fbed40a2fa0 0x7fbed40a5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:24:10.307 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:10.305+0000 7fbeefe00640 1 --2- 172.21.15.47:0/3206335175 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7fbed40a2fa0 0x7fbed40a5460 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fbef019f290 tx=0x7fbee0005e70 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:24:10.312 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:10.309+0000 7fbeed600640 1 -- 172.21.15.47:0/3206335175 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fbed809c5d0 con 0x7fbef00756e0 2024-03-21T10:24:10.515 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:10.513+0000 7fbef5400640 1 -- 172.21.15.47:0/3206335175 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7fbef00622a0 con 0x7fbef00756e0 2024-03-21T10:24:10.516 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:10.514+0000 7fbeed600640 1 -- 172.21.15.47:0/3206335175 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v95) v1 ==== 103+0+11 (secure 0 0 0) 0x7fbed80bc0c0 con 0x7fbef00756e0 2024-03-21T10:24:10.516 INFO:teuthology.orchestra.run.smithi047.stdout:pg_num: 32 2024-03-21T10:24:10.519 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:10.517+0000 7fbef5400640 1 -- 172.21.15.47:0/3206335175 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7fbed40a2fa0 msgr2=0x7fbed40a5460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:24:10.519 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:10.517+0000 7fbef5400640 1 --2- 172.21.15.47:0/3206335175 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7fbed40a2fa0 0x7fbed40a5460 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fbef019f290 tx=0x7fbee0005e70 comp rx=0 tx=0).stop 2024-03-21T10:24:10.519 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:10.517+0000 7fbef5400640 1 -- 172.21.15.47:0/3206335175 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbef00756e0 msgr2=0x7fbef019e300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:24:10.520 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:10.517+0000 7fbef5400640 1 --2- 172.21.15.47:0/3206335175 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbef00756e0 0x7fbef019e300 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7fbed800e9f0 tx=0x7fbed800eeb0 comp rx=0 tx=0).stop 2024-03-21T10:24:10.520 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:10.517+0000 7fbef5400640 1 -- 172.21.15.47:0/3206335175 shutdown_connections 2024-03-21T10:24:10.520 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:10.517+0000 7fbef5400640 1 --2- 172.21.15.47:0/3206335175 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7fbed40a2fa0 0x7fbed40a5460 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:24:10.520 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:10.517+0000 7fbef5400640 1 --2- 172.21.15.47:0/3206335175 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbef010db70 0x7fbef01a5330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:24:10.520 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:10.517+0000 7fbef5400640 1 --2- 172.21.15.47:0/3206335175 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbef00756e0 0x7fbef019e300 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:24:10.520 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:10.517+0000 7fbef5400640 1 --2- 172.21.15.47:0/3206335175 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbef0074100 0x7fbef019ddc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:24:10.520 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:10.517+0000 7fbef5400640 1 -- 172.21.15.47:0/3206335175 >> 172.21.15.47:0/3206335175 conn(0x7fbef00fcb10 msgr2=0x7fbef0100e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:24:10.520 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:10.517+0000 7fbef5400640 1 -- 172.21.15.47:0/3206335175 shutdown_connections 2024-03-21T10:24:10.520 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:10.517+0000 7fbef5400640 1 -- 172.21.15.47:0/3206335175 wait complete. 2024-03-21T10:24:10.538 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-21T10:24:10.752 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:10.750+0000 7fd6d6800640 1 -- 172.21.15.47:0/2997905425 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd6d0103260 msgr2=0x7fd6d0109af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:24:10.753 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:10.750+0000 7fd6d6800640 1 --2- 172.21.15.47:0/2997905425 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd6d0103260 0x7fd6d0109af0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7fd6c8009900 tx=0x7fd6c802bc30 comp rx=0 tx=0).stop 2024-03-21T10:24:10.753 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:10.750+0000 7fd6d6800640 1 -- 172.21.15.47:0/2997905425 shutdown_connections 2024-03-21T10:24:10.753 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:10.750+0000 7fd6d6800640 1 --2- 172.21.15.47:0/2997905425 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd6d0103260 0x7fd6d0109af0 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:24:10.753 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:10.750+0000 7fd6d6800640 1 --2- 172.21.15.47:0/2997905425 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd6d01028a0 0x7fd6d0102d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:24:10.753 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:10.750+0000 7fd6d6800640 1 --2- 172.21.15.47:0/2997905425 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd6d01016b0 0x7fd6d0101ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:24:10.753 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:10.750+0000 7fd6d6800640 1 -- 172.21.15.47:0/2997905425 >> 172.21.15.47:0/2997905425 conn(0x7fd6d00fcc80 msgr2=0x7fd6d00ff0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:24:10.753 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:10.750+0000 7fd6d6800640 1 -- 172.21.15.47:0/2997905425 shutdown_connections 2024-03-21T10:24:10.753 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:10.750+0000 7fd6d6800640 1 -- 172.21.15.47:0/2997905425 wait complete. 2024-03-21T10:24:10.756 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:10.751+0000 7fd6d6800640 1 Processor -- start 2024-03-21T10:24:10.756 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:10.751+0000 7fd6d6800640 1 -- start start 2024-03-21T10:24:10.756 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:10.752+0000 7fd6d6800640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd6d01016b0 0x7fd6d0199ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:24:10.756 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:10.752+0000 7fd6d6800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd6d01028a0 0x7fd6d019a010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:24:10.756 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:10.752+0000 7fd6d6800640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd6d0103260 0x7fd6d01a1080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:24:10.756 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:10.752+0000 7fd6d6800640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6d010d1e0 con 0x7fd6d0103260 2024-03-21T10:24:10.756 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:10.752+0000 7fd6d6800640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6d010d060 con 0x7fd6d01028a0 2024-03-21T10:24:10.757 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:10.752+0000 7fd6d6800640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6d010d360 con 0x7fd6d01016b0 2024-03-21T10:24:10.757 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:10.752+0000 7fd6cfe00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd6d0103260 0x7fd6d01a1080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:24:10.757 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:10.752+0000 7fd6cfe00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd6d0103260 0x7fd6d01a1080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:42366/0 (socket says 172.21.15.47:42366) 2024-03-21T10:24:10.757 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:10.752+0000 7fd6cfe00640 1 -- 172.21.15.47:0/1225857554 learned_addr learned my addr 172.21.15.47:0/1225857554 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:24:10.757 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:10.752+0000 7fd6c6a00640 1 --2- 172.21.15.47:0/1225857554 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd6d01028a0 0x7fd6d019a010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:24:10.757 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:10.752+0000 7fd6cf400640 1 --2- 172.21.15.47:0/1225857554 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd6d01016b0 0x7fd6d0199ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:24:10.757 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:10.753+0000 7fd6cfe00640 1 -- 172.21.15.47:0/1225857554 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd6d01016b0 msgr2=0x7fd6d0199ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:24:10.757 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:10.753+0000 7fd6cfe00640 1 --2- 172.21.15.47:0/1225857554 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd6d01016b0 0x7fd6d0199ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:24:10.757 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:10.753+0000 7fd6cfe00640 1 -- 172.21.15.47:0/1225857554 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd6d01028a0 msgr2=0x7fd6d019a010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:24:10.757 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:10.753+0000 7fd6cfe00640 1 --2- 172.21.15.47:0/1225857554 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd6d01028a0 0x7fd6d019a010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:24:10.757 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:10.753+0000 7fd6cfe00640 1 -- 172.21.15.47:0/1225857554 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6d01a16f0 con 0x7fd6d0103260 2024-03-21T10:24:10.758 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:10.753+0000 7fd6cf400640 1 --2- 172.21.15.47:0/1225857554 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd6d01016b0 0x7fd6d0199ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T10:24:10.758 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:10.753+0000 7fd6cfe00640 1 --2- 172.21.15.47:0/1225857554 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd6d0103260 0x7fd6d01a1080 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7fd6c8009230 tx=0x7fd6c8005e40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:24:10.758 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:10.753+0000 7fd6ccc00640 1 -- 172.21.15.47:0/1225857554 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6c8043020 con 0x7fd6d0103260 2024-03-21T10:24:10.758 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:10.753+0000 7fd6ccc00640 1 -- 172.21.15.47:0/1225857554 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fd6c80043d0 con 0x7fd6d0103260 2024-03-21T10:24:10.758 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:10.753+0000 7fd6d6800640 1 -- 172.21.15.47:0/1225857554 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd6d01a1980 con 0x7fd6d0103260 2024-03-21T10:24:10.758 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:10.753+0000 7fd6ccc00640 1 -- 172.21.15.47:0/1225857554 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6c8003b40 con 0x7fd6d0103260 2024-03-21T10:24:10.758 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:10.754+0000 7fd6d6800640 1 -- 172.21.15.47:0/1225857554 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd6d01a1ed0 con 0x7fd6d0103260 2024-03-21T10:24:10.758 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:10.755+0000 7fd6d6800640 1 -- 172.21.15.47:0/1225857554 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd6d01a20a0 con 0x7fd6d0103260 2024-03-21T10:24:10.759 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:10.757+0000 7fd6ccc00640 1 -- 172.21.15.47:0/1225857554 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 18) v1 ==== 137854+0+0 (secure 0 0 0) 0x7fd6c8046020 con 0x7fd6d0103260 2024-03-21T10:24:10.760 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:10.758+0000 7fd6ccc00640 1 --2- 172.21.15.47:0/1225857554 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7fd6b40a2f20 0x7fd6b40a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:24:10.760 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:10.758+0000 7fd6ccc00640 1 -- 172.21.15.47:0/1225857554 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9216+0+0 (secure 0 0 0) 0x7fd6c80f1e60 con 0x7fd6d0103260 2024-03-21T10:24:10.760 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:10.758+0000 7fd6cf400640 1 --2- 172.21.15.47:0/1225857554 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7fd6b40a2f20 0x7fd6b40a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:24:10.761 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:10.759+0000 7fd6cf400640 1 --2- 172.21.15.47:0/1225857554 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7fd6b40a2f20 0x7fd6b40a53e0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fd6d0102700 tx=0x7fd6c0009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:24:10.767 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:10.765+0000 7fd6ccc00640 1 -- 172.21.15.47:0/1225857554 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fd6c804d020 con 0x7fd6d0103260 2024-03-21T10:24:11.014 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:11.012+0000 7fd6d6800640 1 -- 172.21.15.47:0/1225857554 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd6d01a2390 con 0x7fd6d0103260 2024-03-21T10:24:11.015 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:11.013+0000 7fd6ccc00640 1 -- 172.21.15.47:0/1225857554 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) v1 ==== 94+0+11471 (secure 0 0 0) 0x7fd6c80be880 con 0x7fd6d0103260 2024-03-21T10:24:11.016 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-21T10:24:11.016 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":29,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14511,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6835/735724037","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6834","nonce":735724037},{"type":"v1","addr":"172.21.15.47:6835","nonce":735724037}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14529,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6837/2248191724","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6836","nonce":2248191724},{"type":"v1","addr":"172.21.15.47:6837","nonce":2248191724}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":29,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-21T10:21:58.329063+0000","modified":"2024-03-21T10:22:29.588509+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":34535,"mds_1":34493,"mds_2":34454,"mds_3":34427,"mds_4":14619},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14604":{"gid":14604,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.47:6839/2881335356","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6838","nonce":2881335356},{"type":"v1","addr":"172.21.15.47:6839","nonce":2881335356}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_14619":{"gid":14619,"name":"a","rank":4,"incarnation":27,"state":"up:active","state_seq":11,"addr":"172.21.15.47:6841/3401578978","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6840","nonce":3401578978},{"type":"v1","addr":"172.21.15.47:6841","nonce":3401578978}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24406":{"gid":24406,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.157:6833/4000010136","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":4000010136},{"type":"v1","addr":"172.21.15.157:6833","nonce":4000010136}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24457":{"gid":24457,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.157:6835/3902868651","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":3902868651},{"type":"v1","addr":"172.21.15.157:6835","nonce":3902868651}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24475":{"gid":24475,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.157:6837/1908087204","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6836","nonce":1908087204},{"type":"v1","addr":"172.21.15.157:6837","nonce":1908087204}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24499":{"gid":24499,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.157:6839/4064242876","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6838","nonce":4064242876},{"type":"v1","addr":"172.21.15.157:6839","nonce":4064242876}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_34427":{"gid":34427,"name":"k","rank":3,"incarnation":25,"state":"up:active","state_seq":20,"addr":"172.21.15.100:6833/2723302038","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6832","nonce":2723302038},{"type":"v1","addr":"172.21.15.100:6833","nonce":2723302038}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_34454":{"gid":34454,"name":"e","rank":2,"incarnation":22,"state":"up:active","state_seq":17,"addr":"172.21.15.100:6835/2597849957","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":2597849957},{"type":"v1","addr":"172.21.15.100:6835","nonce":2597849957}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_34493":{"gid":34493,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":13,"addr":"172.21.15.100:6837/3497644162","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6836","nonce":3497644162},{"type":"v1","addr":"172.21.15.100:6837","nonce":3497644162}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_34535":{"gid":34535,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.100:6839/4173168440","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6838","nonce":4173168440},{"type":"v1","addr":"172.21.15.100:6839","nonce":4173168440}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":34535,"qdb_cluster":[34493,34535,34454,34427,14619]},"id":1}]} 2024-03-21T10:24:11.016 INFO:teuthology.orchestra.run.smithi047.stderr:dumped fsmap epoch 29 2024-03-21T10:24:11.018 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:11.016+0000 7fd6d6800640 1 -- 172.21.15.47:0/1225857554 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7fd6b40a2f20 msgr2=0x7fd6b40a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:24:11.018 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:11.016+0000 7fd6d6800640 1 --2- 172.21.15.47:0/1225857554 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7fd6b40a2f20 0x7fd6b40a53e0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fd6d0102700 tx=0x7fd6c0009500 comp rx=0 tx=0).stop 2024-03-21T10:24:11.019 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:11.016+0000 7fd6d6800640 1 -- 172.21.15.47:0/1225857554 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd6d0103260 msgr2=0x7fd6d01a1080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:24:11.019 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:11.016+0000 7fd6d6800640 1 --2- 172.21.15.47:0/1225857554 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd6d0103260 0x7fd6d01a1080 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7fd6c8009230 tx=0x7fd6c8005e40 comp rx=0 tx=0).stop 2024-03-21T10:24:11.019 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:11.016+0000 7fd6d6800640 1 -- 172.21.15.47:0/1225857554 shutdown_connections 2024-03-21T10:24:11.019 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:11.016+0000 7fd6d6800640 1 --2- 172.21.15.47:0/1225857554 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7fd6b40a2f20 0x7fd6b40a53e0 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:24:11.019 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:11.016+0000 7fd6d6800640 1 --2- 172.21.15.47:0/1225857554 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd6d0103260 0x7fd6d01a1080 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:24:11.019 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:11.016+0000 7fd6d6800640 1 --2- 172.21.15.47:0/1225857554 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd6d01028a0 0x7fd6d019a010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:24:11.019 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:11.016+0000 7fd6d6800640 1 --2- 172.21.15.47:0/1225857554 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd6d01016b0 0x7fd6d0199ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:24:11.019 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:11.016+0000 7fd6d6800640 1 -- 172.21.15.47:0/1225857554 >> 172.21.15.47:0/1225857554 conn(0x7fd6d00fcc80 msgr2=0x7fd6d00738d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:24:11.019 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:11.016+0000 7fd6d6800640 1 -- 172.21.15.47:0/1225857554 shutdown_connections 2024-03-21T10:24:11.019 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:11.017+0000 7fd6d6800640 1 -- 172.21.15.47:0/1225857554 wait complete. 2024-03-21T10:24:11.038 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-21T10:24:11.130 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:24:10 smithi047 ceph-mon[31547]: from='client.? 172.21.15.47:0/3868199973' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-21T10:24:11.131 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:24:10 smithi047 ceph-mon[31547]: from='client.? 172.21.15.47:0/3206335175' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-03-21T10:24:11.141 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:24:10 smithi157 ceph-mon[35581]: from='client.? 172.21.15.47:0/3868199973' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-21T10:24:11.141 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:24:10 smithi157 ceph-mon[35581]: from='client.? 172.21.15.47:0/3206335175' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-03-21T10:24:11.229 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:24:10 smithi100 ceph-mon[35193]: from='client.? 172.21.15.47:0/3868199973' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-21T10:24:11.230 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:24:10 smithi100 ceph-mon[35193]: from='client.? 172.21.15.47:0/3206335175' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-03-21T10:24:11.272 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:11.270+0000 7fe00f200640 1 -- 172.21.15.47:0/558365059 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe010101f70 msgr2=0x7fe010104360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:24:11.272 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:11.270+0000 7fe00f200640 1 --2- 172.21.15.47:0/558365059 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe010101f70 0x7fe010104360 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fe000009ca0 tx=0x7fe00002c100 comp rx=0 tx=0).stop 2024-03-21T10:24:11.272 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:11.270+0000 7fe00f200640 1 -- 172.21.15.47:0/558365059 shutdown_connections 2024-03-21T10:24:11.272 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:11.270+0000 7fe00f200640 1 --2- 172.21.15.47:0/558365059 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe0101071d0 0x7fe010109620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:24:11.273 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:11.270+0000 7fe00f200640 1 --2- 172.21.15.47:0/558365059 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe0101048a0 0x7fe010106c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:24:11.273 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:11.270+0000 7fe00f200640 1 --2- 172.21.15.47:0/558365059 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe010101f70 0x7fe010104360 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:24:11.273 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:11.270+0000 7fe00f200640 1 -- 172.21.15.47:0/558365059 >> 172.21.15.47:0/558365059 conn(0x7fe0100fb980 msgr2=0x7fe0100fdda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:24:11.273 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:11.270+0000 7fe00f200640 1 -- 172.21.15.47:0/558365059 shutdown_connections 2024-03-21T10:24:11.273 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:11.270+0000 7fe00f200640 1 -- 172.21.15.47:0/558365059 wait complete. 2024-03-21T10:24:11.273 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:11.271+0000 7fe00f200640 1 Processor -- start 2024-03-21T10:24:11.273 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:11.272+0000 7fe00f200640 1 -- start start 2024-03-21T10:24:11.275 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:11.272+0000 7fe00f200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe010101f70 0x7fe010197740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:24:11.275 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:11.272+0000 7fe00f200640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe0101048a0 0x7fe010197c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:24:11.275 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:11.272+0000 7fe00f200640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe0101071d0 0x7fe01019ecf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:24:11.275 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:11.272+0000 7fe00f200640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe01010d030 con 0x7fe0101071d0 2024-03-21T10:24:11.275 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:11.272+0000 7fe00f200640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe01010ceb0 con 0x7fe010101f70 2024-03-21T10:24:11.275 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:11.272+0000 7fe00f200640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe01010d1b0 con 0x7fe0101048a0 2024-03-21T10:24:11.275 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:11.272+0000 7fe00de00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe010101f70 0x7fe010197740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:24:11.275 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:11.272+0000 7fe00de00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe010101f70 0x7fe010197740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:53922/0 (socket says 172.21.15.47:53922) 2024-03-21T10:24:11.276 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:11.272+0000 7fe00de00640 1 -- 172.21.15.47:0/1090268519 learned_addr learned my addr 172.21.15.47:0/1090268519 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:24:11.276 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:11.272+0000 7fe00e800640 1 --2- 172.21.15.47:0/1090268519 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe0101071d0 0x7fe01019ecf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:24:11.276 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:11.273+0000 7fe00d400640 1 --2- 172.21.15.47:0/1090268519 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe0101048a0 0x7fe010197c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:24:11.276 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:11.273+0000 7fe00de00640 1 -- 172.21.15.47:0/1090268519 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe0101048a0 msgr2=0x7fe010197c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:24:11.276 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:11.273+0000 7fe00de00640 1 --2- 172.21.15.47:0/1090268519 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe0101048a0 0x7fe010197c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:24:11.276 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:11.273+0000 7fe00de00640 1 -- 172.21.15.47:0/1090268519 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe0101071d0 msgr2=0x7fe01019ecf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:24:11.276 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:11.273+0000 7fe00de00640 1 --2- 172.21.15.47:0/1090268519 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe0101071d0 0x7fe01019ecf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:24:11.276 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:11.273+0000 7fe00de00640 1 -- 172.21.15.47:0/1090268519 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe01019f360 con 0x7fe010101f70 2024-03-21T10:24:11.276 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:11.273+0000 7fe00e800640 1 --2- 172.21.15.47:0/1090268519 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe0101071d0 0x7fe01019ecf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T10:24:11.276 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:11.273+0000 7fe00de00640 1 --2- 172.21.15.47:0/1090268519 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe010101f70 0x7fe010197740 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fe000005e30 tx=0x7fe000035d60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:24:11.277 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:11.273+0000 7fe006a00640 1 -- 172.21.15.47:0/1090268519 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe000039040 con 0x7fe010101f70 2024-03-21T10:24:11.277 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:11.274+0000 7fe00f200640 1 -- 172.21.15.47:0/1090268519 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe01019f5f0 con 0x7fe010101f70 2024-03-21T10:24:11.277 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:11.274+0000 7fe006a00640 1 -- 172.21.15.47:0/1090268519 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fe000003970 con 0x7fe010101f70 2024-03-21T10:24:11.277 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:11.274+0000 7fe006a00640 1 -- 172.21.15.47:0/1090268519 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe000004330 con 0x7fe010101f70 2024-03-21T10:24:11.277 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:11.274+0000 7fe00f200640 1 -- 172.21.15.47:0/1090268519 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe01019fb00 con 0x7fe010101f70 2024-03-21T10:24:11.277 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:11.275+0000 7fe00f200640 1 -- 172.21.15.47:0/1090268519 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe0100fd470 con 0x7fe010101f70 2024-03-21T10:24:11.279 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:11.277+0000 7fe006a00640 1 -- 172.21.15.47:0/1090268519 <== mon.2 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 18) v1 ==== 137854+0+0 (secure 0 0 0) 0x7fe000046030 con 0x7fe010101f70 2024-03-21T10:24:11.280 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:11.278+0000 7fe006a00640 1 --2- 172.21.15.47:0/1090268519 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7fdff40a2f20 0x7fdff40a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:24:11.281 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:11.279+0000 7fe006a00640 1 -- 172.21.15.47:0/1090268519 <== mon.2 v2:172.21.15.100:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9216+0+0 (secure 0 0 0) 0x7fe0000f1b70 con 0x7fe010101f70 2024-03-21T10:24:11.281 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:11.279+0000 7fe00d400640 1 --2- 172.21.15.47:0/1090268519 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7fdff40a2f20 0x7fdff40a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:24:11.281 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:11.280+0000 7fe00d400640 1 --2- 172.21.15.47:0/1090268519 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7fdff40a2f20 0x7fdff40a53e0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fdff8005fd0 tx=0x7fdff8005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:24:11.288 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:11.286+0000 7fe006a00640 1 -- 172.21.15.47:0/1090268519 <== mon.2 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fe0000be620 con 0x7fe010101f70 2024-03-21T10:24:11.535 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:11.533+0000 7fe00f200640 1 -- 172.21.15.47:0/1090268519 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fe01002d240 con 0x7fe010101f70 2024-03-21T10:24:11.538 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:11.536+0000 7fe006a00640 1 -- 172.21.15.47:0/1090268519 <== mon.2 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v95) v1 ==== 74+0+24849 (secure 0 0 0) 0x7fe0000bdf20 con 0x7fe010101f70 2024-03-21T10:24:11.539 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-21T10:24:11.539 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":95,"fsid":"3056d51a-e76b-11ee-95cd-87774f69a715","created":"2024-03-21T10:11:17.019965+0000","modified":"2024-03-21T10:23:27.619804+0000","last_up_change":"2024-03-21T10:20:52.675328+0000","last_in_change":"2024-03-21T10:20:27.648212+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-21T10:15:40.553114+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-21T10:21:56.399518+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-21T10:21:57.812940+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-21T10:22:01.822568+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"91","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"b0c7a102-8d89-428b-b793-cbbe67151f50","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6802","nonce":775837313},{"type":"v1","addr":"172.21.15.47:6803","nonce":775837313}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6804","nonce":775837313},{"type":"v1","addr":"172.21.15.47:6805","nonce":775837313}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6808","nonce":775837313},{"type":"v1","addr":"172.21.15.47:6809","nonce":775837313}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6806","nonce":775837313},{"type":"v1","addr":"172.21.15.47:6807","nonce":775837313}]},"public_addr":"172.21.15.47:6803/775837313","cluster_addr":"172.21.15.47:6805/775837313","heartbeat_back_addr":"172.21.15.47:6809/775837313","heartbeat_front_addr":"172.21.15.47:6807/775837313","state":["exists","up"]},{"osd":1,"uuid":"07bb81bb-7cf3-4bbc-ac9e-802f95185d11","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6800","nonce":2923304710},{"type":"v1","addr":"172.21.15.100:6801","nonce":2923304710}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":2923304710},{"type":"v1","addr":"172.21.15.100:6803","nonce":2923304710}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":2923304710},{"type":"v1","addr":"172.21.15.100:6807","nonce":2923304710}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6804","nonce":2923304710},{"type":"v1","addr":"172.21.15.100:6805","nonce":2923304710}]},"public_addr":"172.21.15.100:6801/2923304710","cluster_addr":"172.21.15.100:6803/2923304710","heartbeat_back_addr":"172.21.15.100:6807/2923304710","heartbeat_front_addr":"172.21.15.100:6805/2923304710","state":["exists","up"]},{"osd":2,"uuid":"88ce76b9-6081-4836-aad0-8c249c56e8a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":171377005},{"type":"v1","addr":"172.21.15.157:6801","nonce":171377005}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":171377005},{"type":"v1","addr":"172.21.15.157:6803","nonce":171377005}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":171377005},{"type":"v1","addr":"172.21.15.157:6807","nonce":171377005}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":171377005},{"type":"v1","addr":"172.21.15.157:6805","nonce":171377005}]},"public_addr":"172.21.15.157:6801/171377005","cluster_addr":"172.21.15.157:6803/171377005","heartbeat_back_addr":"172.21.15.157:6807/171377005","heartbeat_front_addr":"172.21.15.157:6805/171377005","state":["exists","up"]},{"osd":3,"uuid":"518aa33d-263f-45e6-aab5-2201ceb40e74","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":27,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6810","nonce":640760101},{"type":"v1","addr":"172.21.15.47:6811","nonce":640760101}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6812","nonce":640760101},{"type":"v1","addr":"172.21.15.47:6813","nonce":640760101}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6816","nonce":640760101},{"type":"v1","addr":"172.21.15.47:6817","nonce":640760101}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6814","nonce":640760101},{"type":"v1","addr":"172.21.15.47:6815","nonce":640760101}]},"public_addr":"172.21.15.47:6811/640760101","cluster_addr":"172.21.15.47:6813/640760101","heartbeat_back_addr":"172.21.15.47:6817/640760101","heartbeat_front_addr":"172.21.15.47:6815/640760101","state":["exists","up"]},{"osd":4,"uuid":"987bde83-ecc6-42e5-bd56-c143b3de4226","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":35,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6808","nonce":2785478639},{"type":"v1","addr":"172.21.15.100:6809","nonce":2785478639}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6810","nonce":2785478639},{"type":"v1","addr":"172.21.15.100:6811","nonce":2785478639}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6814","nonce":2785478639},{"type":"v1","addr":"172.21.15.100:6815","nonce":2785478639}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":2785478639},{"type":"v1","addr":"172.21.15.100:6813","nonce":2785478639}]},"public_addr":"172.21.15.100:6809/2785478639","cluster_addr":"172.21.15.100:6811/2785478639","heartbeat_back_addr":"172.21.15.100:6815/2785478639","heartbeat_front_addr":"172.21.15.100:6813/2785478639","state":["exists","up"]},{"osd":5,"uuid":"012d2188-3a61-4a5f-8d01-2001c15ffbe2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":40,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":1525645410},{"type":"v1","addr":"172.21.15.157:6809","nonce":1525645410}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":1525645410},{"type":"v1","addr":"172.21.15.157:6811","nonce":1525645410}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":1525645410},{"type":"v1","addr":"172.21.15.157:6815","nonce":1525645410}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":1525645410},{"type":"v1","addr":"172.21.15.157:6813","nonce":1525645410}]},"public_addr":"172.21.15.157:6809/1525645410","cluster_addr":"172.21.15.157:6811/1525645410","heartbeat_back_addr":"172.21.15.157:6815/1525645410","heartbeat_front_addr":"172.21.15.157:6813/1525645410","state":["exists","up"]},{"osd":6,"uuid":"c12c14b4-85f0-4d88-937c-de4f556437de","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6818","nonce":2321642790},{"type":"v1","addr":"172.21.15.47:6819","nonce":2321642790}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6820","nonce":2321642790},{"type":"v1","addr":"172.21.15.47:6821","nonce":2321642790}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6824","nonce":2321642790},{"type":"v1","addr":"172.21.15.47:6825","nonce":2321642790}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6822","nonce":2321642790},{"type":"v1","addr":"172.21.15.47:6823","nonce":2321642790}]},"public_addr":"172.21.15.47:6819/2321642790","cluster_addr":"172.21.15.47:6821/2321642790","heartbeat_back_addr":"172.21.15.47:6825/2321642790","heartbeat_front_addr":"172.21.15.47:6823/2321642790","state":["exists","up"]},{"osd":7,"uuid":"3c87c6d0-1d1f-4294-9917-aaaad7dda824","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6816","nonce":4104727330},{"type":"v1","addr":"172.21.15.100:6817","nonce":4104727330}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":4104727330},{"type":"v1","addr":"172.21.15.100:6819","nonce":4104727330}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":4104727330},{"type":"v1","addr":"172.21.15.100:6823","nonce":4104727330}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6820","nonce":4104727330},{"type":"v1","addr":"172.21.15.100:6821","nonce":4104727330}]},"public_addr":"172.21.15.100:6817/4104727330","cluster_addr":"172.21.15.100:6819/4104727330","heartbeat_back_addr":"172.21.15.100:6823/4104727330","heartbeat_front_addr":"172.21.15.100:6821/4104727330","state":["exists","up"]},{"osd":8,"uuid":"502d65cf-53c8-4936-8de4-e11b2a6e1e18","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":61,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":4067715347},{"type":"v1","addr":"172.21.15.157:6817","nonce":4067715347}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":4067715347},{"type":"v1","addr":"172.21.15.157:6819","nonce":4067715347}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":4067715347},{"type":"v1","addr":"172.21.15.157:6823","nonce":4067715347}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":4067715347},{"type":"v1","addr":"172.21.15.157:6821","nonce":4067715347}]},"public_addr":"172.21.15.157:6817/4067715347","cluster_addr":"172.21.15.157:6819/4067715347","heartbeat_back_addr":"172.21.15.157:6823/4067715347","heartbeat_front_addr":"172.21.15.157:6821/4067715347","state":["exists","up"]},{"osd":9,"uuid":"5ea1f5be-8458-4c93-9e61-c2e867a514f3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6826","nonce":1982108738},{"type":"v1","addr":"172.21.15.47:6827","nonce":1982108738}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6828","nonce":1982108738},{"type":"v1","addr":"172.21.15.47:6829","nonce":1982108738}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6832","nonce":1982108738},{"type":"v1","addr":"172.21.15.47:6833","nonce":1982108738}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6830","nonce":1982108738},{"type":"v1","addr":"172.21.15.47:6831","nonce":1982108738}]},"public_addr":"172.21.15.47:6827/1982108738","cluster_addr":"172.21.15.47:6829/1982108738","heartbeat_back_addr":"172.21.15.47:6833/1982108738","heartbeat_front_addr":"172.21.15.47:6831/1982108738","state":["exists","up"]},{"osd":10,"uuid":"2da3da90-a0be-4a54-a9ff-b2d6c9b36870","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":74,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":396391028},{"type":"v1","addr":"172.21.15.100:6825","nonce":396391028}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":396391028},{"type":"v1","addr":"172.21.15.100:6827","nonce":396391028}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6830","nonce":396391028},{"type":"v1","addr":"172.21.15.100:6831","nonce":396391028}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":396391028},{"type":"v1","addr":"172.21.15.100:6829","nonce":396391028}]},"public_addr":"172.21.15.100:6825/396391028","cluster_addr":"172.21.15.100:6827/396391028","heartbeat_back_addr":"172.21.15.100:6831/396391028","heartbeat_front_addr":"172.21.15.100:6829/396391028","state":["exists","up"]},{"osd":11,"uuid":"11798d41-7f7e-4319-95b8-8d3d0946bde9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":82,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":2671029483},{"type":"v1","addr":"172.21.15.157:6825","nonce":2671029483}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":2671029483},{"type":"v1","addr":"172.21.15.157:6827","nonce":2671029483}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":2671029483},{"type":"v1","addr":"172.21.15.157:6831","nonce":2671029483}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":2671029483},{"type":"v1","addr":"172.21.15.157:6829","nonce":2671029483}]},"public_addr":"172.21.15.157:6825/2671029483","cluster_addr":"172.21.15.157:6827/2671029483","heartbeat_back_addr":"172.21.15.157:6831/2671029483","heartbeat_front_addr":"172.21.15.157:6829/2671029483","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T10:14:31.822399+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T10:15:04.175721+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T10:15:35.902595+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T10:16:12.088891+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T10:16:46.428704+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T10:17:20.158270+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T10:17:55.915253+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T10:18:30.064641+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T10:19:04.410004+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T10:19:40.605181+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T10:20:15.636605+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T10:20:50.151365+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.11","mappings":[{"from":7,"to":11}]},{"pgid":"2.14","mappings":[{"from":2,"to":4}]},{"pgid":"2.1a","mappings":[{"from":8,"to":1}]},{"pgid":"2.1b","mappings":[{"from":2,"to":4}]},{"pgid":"2.1d","mappings":[{"from":8,"to":11}]},{"pgid":"3.0","mappings":[{"from":4,"to":8},{"from":5,"to":9}]},{"pgid":"3.6","mappings":[{"from":3,"to":7}]},{"pgid":"3.1b","mappings":[{"from":4,"to":6}]},{"pgid":"3.33","mappings":[{"from":5,"to":9}]},{"pgid":"3.3c","mappings":[{"from":4,"to":9}]},{"pgid":"4.c","mappings":[{"from":9,"to":10}]},{"pgid":"4.f","mappings":[{"from":9,"to":8}]},{"pgid":"4.19","mappings":[{"from":9,"to":8}]},{"pgid":"4.1d","mappings":[{"from":5,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.47:0/356822431":"2024-03-22T10:12:25.844530+0000","172.21.15.47:0/1167490879":"2024-03-22T10:12:25.844530+0000","172.21.15.47:0/1540718138":"2024-03-22T10:12:25.844530+0000","172.21.15.47:6801/3836903972":"2024-03-22T10:12:25.844530+0000","172.21.15.47:0/235436602":"2024-03-22T10:11:50.301838+0000","172.21.15.47:0/2052815572":"2024-03-22T10:11:50.301838+0000","172.21.15.47:6800/3942658432":"2024-03-22T10:11:50.301838+0000","172.21.15.47:6801/3942658432":"2024-03-22T10:11:50.301838+0000","172.21.15.47:0/1419516257":"2024-03-22T10:11:50.301838+0000","172.21.15.47:6800/3836903972":"2024-03-22T10:12:25.844530+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-21T10:24:11.541 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:11.539+0000 7fe00f200640 1 -- 172.21.15.47:0/1090268519 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7fdff40a2f20 msgr2=0x7fdff40a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:24:11.541 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:11.539+0000 7fe00f200640 1 --2- 172.21.15.47:0/1090268519 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7fdff40a2f20 0x7fdff40a53e0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fdff8005fd0 tx=0x7fdff8005850 comp rx=0 tx=0).stop 2024-03-21T10:24:11.542 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:11.539+0000 7fe00f200640 1 -- 172.21.15.47:0/1090268519 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe010101f70 msgr2=0x7fe010197740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:24:11.542 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:11.539+0000 7fe00f200640 1 --2- 172.21.15.47:0/1090268519 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe010101f70 0x7fe010197740 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fe000005e30 tx=0x7fe000035d60 comp rx=0 tx=0).stop 2024-03-21T10:24:11.542 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:11.539+0000 7fe00f200640 1 -- 172.21.15.47:0/1090268519 shutdown_connections 2024-03-21T10:24:11.542 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:11.539+0000 7fe00f200640 1 --2- 172.21.15.47:0/1090268519 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7fdff40a2f20 0x7fdff40a53e0 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:24:11.542 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:11.539+0000 7fe00f200640 1 --2- 172.21.15.47:0/1090268519 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe0101071d0 0x7fe01019ecf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:24:11.542 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:11.539+0000 7fe00f200640 1 --2- 172.21.15.47:0/1090268519 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe0101048a0 0x7fe010197c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:24:11.542 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:11.539+0000 7fe00f200640 1 --2- 172.21.15.47:0/1090268519 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe010101f70 0x7fe010197740 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:24:11.542 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:11.539+0000 7fe00f200640 1 -- 172.21.15.47:0/1090268519 >> 172.21.15.47:0/1090268519 conn(0x7fe0100fb980 msgr2=0x7fe0101029f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:24:11.542 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:11.539+0000 7fe00f200640 1 -- 172.21.15.47:0/1090268519 shutdown_connections 2024-03-21T10:24:11.542 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:11.539+0000 7fe00f200640 1 -- 172.21.15.47:0/1090268519 wait complete. 2024-03-21T10:24:11.561 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-21T10:24:11.774 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:11.771+0000 7f14eaa00640 1 -- 172.21.15.47:0/3981880370 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f14e4101510 msgr2=0x7f14e4101930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:24:11.774 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:11.771+0000 7f14eaa00640 1 --2- 172.21.15.47:0/3981880370 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f14e4101510 0x7f14e4101930 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f14d8009900 tx=0x7f14d802bc30 comp rx=0 tx=0).stop 2024-03-21T10:24:11.774 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:11.771+0000 7f14eaa00640 1 -- 172.21.15.47:0/3981880370 shutdown_connections 2024-03-21T10:24:11.774 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:11.771+0000 7f14eaa00640 1 --2- 172.21.15.47:0/3981880370 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f14e41030c0 0x7f14e4109950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:24:11.774 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:11.771+0000 7f14eaa00640 1 --2- 172.21.15.47:0/3981880370 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f14e4102700 0x7f14e4102b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:24:11.774 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:11.771+0000 7f14eaa00640 1 --2- 172.21.15.47:0/3981880370 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f14e4101510 0x7f14e4101930 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:24:11.774 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:11.771+0000 7f14eaa00640 1 -- 172.21.15.47:0/3981880370 >> 172.21.15.47:0/3981880370 conn(0x7f14e40fca80 msgr2=0x7f14e40feec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:24:11.774 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:11.771+0000 7f14eaa00640 1 -- 172.21.15.47:0/3981880370 shutdown_connections 2024-03-21T10:24:11.774 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:11.771+0000 7f14eaa00640 1 -- 172.21.15.47:0/3981880370 wait complete. 2024-03-21T10:24:11.776 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:11.772+0000 7f14eaa00640 1 Processor -- start 2024-03-21T10:24:11.777 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:11.772+0000 7f14eaa00640 1 -- start start 2024-03-21T10:24:11.777 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:11.773+0000 7f14eaa00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f14e4101510 0x7f14e4197750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:24:11.777 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:11.773+0000 7f14eaa00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f14e4102700 0x7f14e4197c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:24:11.777 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:11.773+0000 7f14eaa00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f14e41030c0 0x7f14e419ed00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:24:11.777 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:11.773+0000 7f14eaa00640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14e410d0a0 con 0x7f14e4102700 2024-03-21T10:24:11.777 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:11.773+0000 7f14eaa00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14e410cf20 con 0x7f14e4101510 2024-03-21T10:24:11.777 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:11.773+0000 7f14eaa00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14e410d220 con 0x7f14e41030c0 2024-03-21T10:24:11.777 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:11.773+0000 7f14e8c00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f14e4102700 0x7f14e4197c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:24:11.777 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:11.773+0000 7f14e9600640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f14e4101510 0x7f14e4197750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:24:11.778 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:11.773+0000 7f14e8c00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f14e4102700 0x7f14e4197c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:42394/0 (socket says 172.21.15.47:42394) 2024-03-21T10:24:11.778 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:11.773+0000 7f14e9600640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f14e4101510 0x7f14e4197750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:53950/0 (socket says 172.21.15.47:53950) 2024-03-21T10:24:11.778 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:11.773+0000 7f14e8c00640 1 -- 172.21.15.47:0/869124167 learned_addr learned my addr 172.21.15.47:0/869124167 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:24:11.778 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:11.773+0000 7f14ea000640 1 --2- 172.21.15.47:0/869124167 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f14e41030c0 0x7f14e419ed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:24:11.778 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:11.773+0000 7f14e8c00640 1 -- 172.21.15.47:0/869124167 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f14e41030c0 msgr2=0x7f14e419ed00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:24:11.778 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:11.774+0000 7f14e8c00640 1 --2- 172.21.15.47:0/869124167 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f14e41030c0 0x7f14e419ed00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:24:11.778 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:11.774+0000 7f14e8c00640 1 -- 172.21.15.47:0/869124167 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f14e4101510 msgr2=0x7f14e4197750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:24:11.778 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:11.774+0000 7f14e8c00640 1 --2- 172.21.15.47:0/869124167 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f14e4101510 0x7f14e4197750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:24:11.778 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:11.774+0000 7f14e8c00640 1 -- 172.21.15.47:0/869124167 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f14e419f370 con 0x7f14e4102700 2024-03-21T10:24:11.778 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:11.774+0000 7f14e8c00640 1 --2- 172.21.15.47:0/869124167 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f14e4102700 0x7f14e4197c90 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f14d000ea60 tx=0x7f14d000ef20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:24:11.778 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:11.774+0000 7f14e2000640 1 -- 172.21.15.47:0/869124167 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14d000cde0 con 0x7f14e4102700 2024-03-21T10:24:11.778 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:11.774+0000 7f14e2000640 1 -- 172.21.15.47:0/869124167 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f14d00035f0 con 0x7f14e4102700 2024-03-21T10:24:11.778 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:11.774+0000 7f14e2000640 1 -- 172.21.15.47:0/869124167 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14d0010690 con 0x7f14e4102700 2024-03-21T10:24:11.778 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:11.774+0000 7f14eaa00640 1 -- 172.21.15.47:0/869124167 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f14e419f660 con 0x7f14e4102700 2024-03-21T10:24:11.779 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:11.774+0000 7f14eaa00640 1 -- 172.21.15.47:0/869124167 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f14e419fc00 con 0x7f14e4102700 2024-03-21T10:24:11.779 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:11.775+0000 7f14eaa00640 1 -- 172.21.15.47:0/869124167 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f14e41088d0 con 0x7f14e4102700 2024-03-21T10:24:11.780 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:11.778+0000 7f14e2000640 1 -- 172.21.15.47:0/869124167 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 18) v1 ==== 137854+0+0 (secure 0 0 0) 0x7f14d00037a0 con 0x7f14e4102700 2024-03-21T10:24:11.781 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:11.780+0000 7f14e2000640 1 --2- 172.21.15.47:0/869124167 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f14cc0a2fa0 0x7f14cc0a5460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:24:11.782 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:11.780+0000 7f14e2000640 1 -- 172.21.15.47:0/869124167 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9216+0+0 (secure 0 0 0) 0x7f14d00cfa90 con 0x7f14e4102700 2024-03-21T10:24:11.782 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:11.780+0000 7f14e9600640 1 --2- 172.21.15.47:0/869124167 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f14cc0a2fa0 0x7f14cc0a5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:24:11.782 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:11.781+0000 7f14e9600640 1 --2- 172.21.15.47:0/869124167 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f14cc0a2fa0 0x7f14cc0a5460 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f14d802c240 tx=0x7f14d802c7e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:24:11.786 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:11.784+0000 7f14e2000640 1 -- 172.21.15.47:0/869124167 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f14d009c4b0 con 0x7f14e4102700 2024-03-21T10:24:12.033 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:12.031+0000 7f14eaa00640 1 -- 172.21.15.47:0/869124167 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f14e402d070 con 0x7f14e4102700 2024-03-21T10:24:12.034 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:12.032+0000 7f14e2000640 1 -- 172.21.15.47:0/869124167 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) v1 ==== 94+0+11471 (secure 0 0 0) 0x7f14d009bdb0 con 0x7f14e4102700 2024-03-21T10:24:12.034 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-21T10:24:12.034 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":29,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14511,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6835/735724037","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6834","nonce":735724037},{"type":"v1","addr":"172.21.15.47:6835","nonce":735724037}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14529,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6837/2248191724","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6836","nonce":2248191724},{"type":"v1","addr":"172.21.15.47:6837","nonce":2248191724}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":29,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-21T10:21:58.329063+0000","modified":"2024-03-21T10:22:29.588509+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":34535,"mds_1":34493,"mds_2":34454,"mds_3":34427,"mds_4":14619},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14604":{"gid":14604,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.47:6839/2881335356","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6838","nonce":2881335356},{"type":"v1","addr":"172.21.15.47:6839","nonce":2881335356}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_14619":{"gid":14619,"name":"a","rank":4,"incarnation":27,"state":"up:active","state_seq":11,"addr":"172.21.15.47:6841/3401578978","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6840","nonce":3401578978},{"type":"v1","addr":"172.21.15.47:6841","nonce":3401578978}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24406":{"gid":24406,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.157:6833/4000010136","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":4000010136},{"type":"v1","addr":"172.21.15.157:6833","nonce":4000010136}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24457":{"gid":24457,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.157:6835/3902868651","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":3902868651},{"type":"v1","addr":"172.21.15.157:6835","nonce":3902868651}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24475":{"gid":24475,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.157:6837/1908087204","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6836","nonce":1908087204},{"type":"v1","addr":"172.21.15.157:6837","nonce":1908087204}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24499":{"gid":24499,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.157:6839/4064242876","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6838","nonce":4064242876},{"type":"v1","addr":"172.21.15.157:6839","nonce":4064242876}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_34427":{"gid":34427,"name":"k","rank":3,"incarnation":25,"state":"up:active","state_seq":20,"addr":"172.21.15.100:6833/2723302038","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6832","nonce":2723302038},{"type":"v1","addr":"172.21.15.100:6833","nonce":2723302038}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_34454":{"gid":34454,"name":"e","rank":2,"incarnation":22,"state":"up:active","state_seq":17,"addr":"172.21.15.100:6835/2597849957","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":2597849957},{"type":"v1","addr":"172.21.15.100:6835","nonce":2597849957}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_34493":{"gid":34493,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":13,"addr":"172.21.15.100:6837/3497644162","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6836","nonce":3497644162},{"type":"v1","addr":"172.21.15.100:6837","nonce":3497644162}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_34535":{"gid":34535,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.100:6839/4173168440","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6838","nonce":4173168440},{"type":"v1","addr":"172.21.15.100:6839","nonce":4173168440}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":34535,"qdb_cluster":[34493,34535,34454,34427,14619]},"id":1}]} 2024-03-21T10:24:12.035 INFO:teuthology.orchestra.run.smithi047.stderr:dumped fsmap epoch 29 2024-03-21T10:24:12.036 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:12.034+0000 7f14eaa00640 1 -- 172.21.15.47:0/869124167 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f14cc0a2fa0 msgr2=0x7f14cc0a5460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:24:12.037 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:12.034+0000 7f14eaa00640 1 --2- 172.21.15.47:0/869124167 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f14cc0a2fa0 0x7f14cc0a5460 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f14d802c240 tx=0x7f14d802c7e0 comp rx=0 tx=0).stop 2024-03-21T10:24:12.037 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:12.034+0000 7f14eaa00640 1 -- 172.21.15.47:0/869124167 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f14e4102700 msgr2=0x7f14e4197c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:24:12.037 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:12.034+0000 7f14eaa00640 1 --2- 172.21.15.47:0/869124167 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f14e4102700 0x7f14e4197c90 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f14d000ea60 tx=0x7f14d000ef20 comp rx=0 tx=0).stop 2024-03-21T10:24:12.037 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:12.034+0000 7f14eaa00640 1 -- 172.21.15.47:0/869124167 shutdown_connections 2024-03-21T10:24:12.037 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:12.034+0000 7f14eaa00640 1 --2- 172.21.15.47:0/869124167 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f14cc0a2fa0 0x7f14cc0a5460 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:24:12.037 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:12.034+0000 7f14eaa00640 1 --2- 172.21.15.47:0/869124167 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f14e41030c0 0x7f14e419ed00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:24:12.037 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:12.034+0000 7f14eaa00640 1 --2- 172.21.15.47:0/869124167 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f14e4102700 0x7f14e4197c90 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:24:12.037 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:12.034+0000 7f14eaa00640 1 --2- 172.21.15.47:0/869124167 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f14e4101510 0x7f14e4197750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:24:12.037 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:12.034+0000 7f14eaa00640 1 -- 172.21.15.47:0/869124167 >> 172.21.15.47:0/869124167 conn(0x7f14e40fca80 msgr2=0x7f14e4105aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:24:12.037 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:12.035+0000 7f14eaa00640 1 -- 172.21.15.47:0/869124167 shutdown_connections 2024-03-21T10:24:12.037 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:12.035+0000 7f14eaa00640 1 -- 172.21.15.47:0/869124167 wait complete. 2024-03-21T10:24:12.055 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 29, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-03-21T10:21:58.329063+0000', 'modified': '2024-03-21T10:22:29.588509+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 5, 'in': [0, 1, 2, 3, 4], 'up': {'mds_0': 34535, 'mds_1': 34493, 'mds_2': 34454, 'mds_3': 34427, 'mds_4': 14619}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_14604': {'gid': 14604, 'name': 'g', 'rank': 4, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.47:6839/2881335356', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.47:6838', 'nonce': 2881335356}, {'type': 'v1', 'addr': '172.21.15.47:6839', 'nonce': 2881335356}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_14619': {'gid': 14619, 'name': 'a', 'rank': 4, 'incarnation': 27, 'state': 'up:active', 'state_seq': 11, 'addr': '172.21.15.47:6841/3401578978', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.47:6840', 'nonce': 3401578978}, {'type': 'v1', 'addr': '172.21.15.47:6841', 'nonce': 3401578978}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_24406': {'gid': 24406, 'name': 'l', 'rank': 3, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.157:6833/4000010136', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.157:6832', 'nonce': 4000010136}, {'type': 'v1', 'addr': '172.21.15.157:6833', 'nonce': 4000010136}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_24457': {'gid': 24457, 'name': 'f', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.157:6835/3902868651', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.157:6834', 'nonce': 3902868651}, {'type': 'v1', 'addr': '172.21.15.157:6835', 'nonce': 3902868651}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_24475': {'gid': 24475, 'name': 'c', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.157:6837/1908087204', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.157:6836', 'nonce': 1908087204}, {'type': 'v1', 'addr': '172.21.15.157:6837', 'nonce': 1908087204}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_24499': {'gid': 24499, 'name': 'i', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.157:6839/4064242876', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.157:6838', 'nonce': 4064242876}, {'type': 'v1', 'addr': '172.21.15.157:6839', 'nonce': 4064242876}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_34427': {'gid': 34427, 'name': 'k', 'rank': 3, 'incarnation': 25, 'state': 'up:active', 'state_seq': 20, 'addr': '172.21.15.100:6833/2723302038', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.100:6832', 'nonce': 2723302038}, {'type': 'v1', 'addr': '172.21.15.100:6833', 'nonce': 2723302038}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_34454': {'gid': 34454, 'name': 'e', 'rank': 2, 'incarnation': 22, 'state': 'up:active', 'state_seq': 17, 'addr': '172.21.15.100:6835/2597849957', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.100:6834', 'nonce': 2597849957}, {'type': 'v1', 'addr': '172.21.15.100:6835', 'nonce': 2597849957}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_34493': {'gid': 34493, 'name': 'h', 'rank': 1, 'incarnation': 20, 'state': 'up:active', 'state_seq': 13, 'addr': '172.21.15.100:6837/3497644162', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.100:6836', 'nonce': 3497644162}, {'type': 'v1', 'addr': '172.21.15.100:6837', 'nonce': 3497644162}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_34535': {'gid': 34535, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.100:6839/4173168440', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.100:6838', 'nonce': 4173168440}, {'type': 'v1', 'addr': '172.21.15.100:6839', 'nonce': 4173168440}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 34535, 'qdb_cluster': [34493, 34535, 34454, 34427, 14619]} 2024-03-21T10:24:12.056 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 5/5 2024-03-21T10:24:12.056 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2024-03-21T10:24:12.214 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:24:11 smithi047 ceph-mon[31547]: pgmap v437: 129 pgs: 129 active+clean; 642 KiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 426 B/s wr, 11 op/s 2024-03-21T10:24:12.214 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:24:11 smithi047 ceph-mon[31547]: from='client.? 172.21.15.47:0/1225857554' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-21T10:24:12.214 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:24:11 smithi047 ceph-mon[31547]: from='client.? 172.21.15.47:0/1090268519' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-21T10:24:12.229 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:24:11 smithi100 ceph-mon[35193]: pgmap v437: 129 pgs: 129 active+clean; 642 KiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 426 B/s wr, 11 op/s 2024-03-21T10:24:12.229 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:24:11 smithi100 ceph-mon[35193]: from='client.? 172.21.15.47:0/1225857554' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-21T10:24:12.229 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:24:11 smithi100 ceph-mon[35193]: from='client.? 172.21.15.47:0/1090268519' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-21T10:24:12.241 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:24:11 smithi157 ceph-mon[35581]: pgmap v437: 129 pgs: 129 active+clean; 642 KiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 426 B/s wr, 11 op/s 2024-03-21T10:24:12.241 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:24:11 smithi157 ceph-mon[35581]: from='client.? 172.21.15.47:0/1225857554' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-21T10:24:12.241 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:24:11 smithi157 ceph-mon[35581]: from='client.? 172.21.15.47:0/1090268519' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-21T10:24:12.270 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:12.267+0000 7f5eb2a00640 1 -- 172.21.15.47:0/1948055373 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5eac102000 msgr2=0x7f5eac1043f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:24:12.270 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:12.267+0000 7f5eb2a00640 1 --2- 172.21.15.47:0/1948055373 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5eac102000 0x7f5eac1043f0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f5ea0009ca0 tx=0x7f5ea002c0f0 comp rx=0 tx=0).stop 2024-03-21T10:24:12.270 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:12.268+0000 7f5eb2a00640 1 -- 172.21.15.47:0/1948055373 shutdown_connections 2024-03-21T10:24:12.270 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:12.268+0000 7f5eb2a00640 1 --2- 172.21.15.47:0/1948055373 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5eac107260 0x7f5eac1096b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:24:12.270 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:12.268+0000 7f5eb2a00640 1 --2- 172.21.15.47:0/1948055373 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5eac104930 0x7f5eac106d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:24:12.270 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:12.268+0000 7f5eb2a00640 1 --2- 172.21.15.47:0/1948055373 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5eac102000 0x7f5eac1043f0 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:24:12.270 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:12.268+0000 7f5eb2a00640 1 -- 172.21.15.47:0/1948055373 >> 172.21.15.47:0/1948055373 conn(0x7f5eac0fba10 msgr2=0x7f5eac0fde30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:24:12.270 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:12.268+0000 7f5eb2a00640 1 -- 172.21.15.47:0/1948055373 shutdown_connections 2024-03-21T10:24:12.270 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:12.268+0000 7f5eb2a00640 1 -- 172.21.15.47:0/1948055373 wait complete. 2024-03-21T10:24:12.272 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:12.269+0000 7f5eb2a00640 1 Processor -- start 2024-03-21T10:24:12.272 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:12.269+0000 7f5eb2a00640 1 -- start start 2024-03-21T10:24:12.272 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:12.269+0000 7f5eb2a00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5eac102000 0x7f5eac1999d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:24:12.272 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:12.269+0000 7f5eb2a00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5eac104930 0x7f5eac199f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:24:12.272 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:12.269+0000 7f5eb2a00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5eac107260 0x7f5eac1a0f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:24:12.272 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:12.269+0000 7f5eb2a00640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5eac10d0c0 con 0x7f5eac102000 2024-03-21T10:24:12.273 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:12.269+0000 7f5eb2a00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5eac10cf40 con 0x7f5eac107260 2024-03-21T10:24:12.273 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:12.269+0000 7f5eb2a00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5eac10d240 con 0x7f5eac104930 2024-03-21T10:24:12.273 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:12.269+0000 7f5eb1600640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5eac102000 0x7f5eac1999d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:24:12.273 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:12.269+0000 7f5eb1600640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5eac102000 0x7f5eac1999d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:40814/0 (socket says 172.21.15.47:40814) 2024-03-21T10:24:12.273 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:12.269+0000 7f5eb1600640 1 -- 172.21.15.47:0/3130541723 learned_addr learned my addr 172.21.15.47:0/3130541723 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:24:12.273 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:12.270+0000 7f5eb2000640 1 --2- 172.21.15.47:0/3130541723 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5eac107260 0x7f5eac1a0f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:24:12.273 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:12.270+0000 7f5eb0c00640 1 --2- 172.21.15.47:0/3130541723 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5eac104930 0x7f5eac199f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:24:12.273 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:12.270+0000 7f5eb1600640 1 -- 172.21.15.47:0/3130541723 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5eac104930 msgr2=0x7f5eac199f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:24:12.273 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:12.270+0000 7f5eb1600640 1 --2- 172.21.15.47:0/3130541723 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5eac104930 0x7f5eac199f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:24:12.273 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:12.270+0000 7f5eb1600640 1 -- 172.21.15.47:0/3130541723 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5eac107260 msgr2=0x7f5eac1a0f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:24:12.273 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:12.270+0000 7f5eb1600640 1 --2- 172.21.15.47:0/3130541723 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5eac107260 0x7f5eac1a0f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:24:12.273 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:12.270+0000 7f5eb1600640 1 -- 172.21.15.47:0/3130541723 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5eac1a15f0 con 0x7f5eac102000 2024-03-21T10:24:12.273 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:12.270+0000 7f5eb1600640 1 --2- 172.21.15.47:0/3130541723 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5eac102000 0x7f5eac1999d0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f5ea0000f80 tx=0x7f5ea00328a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:24:12.274 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:12.271+0000 7f5eaa000640 1 -- 172.21.15.47:0/3130541723 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ea0041020 con 0x7f5eac102000 2024-03-21T10:24:12.274 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:12.271+0000 7f5eaa000640 1 -- 172.21.15.47:0/3130541723 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f5ea0003e20 con 0x7f5eac102000 2024-03-21T10:24:12.274 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:12.271+0000 7f5eaa000640 1 -- 172.21.15.47:0/3130541723 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ea0039040 con 0x7f5eac102000 2024-03-21T10:24:12.274 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:12.271+0000 7f5eb2a00640 1 -- 172.21.15.47:0/3130541723 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5eac1a1880 con 0x7f5eac102000 2024-03-21T10:24:12.274 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:12.271+0000 7f5eb2a00640 1 -- 172.21.15.47:0/3130541723 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5eac1a1d20 con 0x7f5eac102000 2024-03-21T10:24:12.278 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:12.274+0000 7f5eb6e35740 1 Processor -- start 2024-03-21T10:24:12.278 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:12.274+0000 7f5eaa000640 1 -- 172.21.15.47:0/3130541723 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 18) v1 ==== 137854+0+0 (secure 0 0 0) 0x7f5ea0044020 con 0x7f5eac102000 2024-03-21T10:24:12.278 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:12.274+0000 7f5eb6e35740 1 -- start start 2024-03-21T10:24:12.278 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:12.275+0000 7f5eb6e35740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5590398c5d80 0x5590398c61a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:24:12.278 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:12.275+0000 7f5eb6e35740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5590398c6770 0x5590398cb270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:24:12.278 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:12.275+0000 7f5eb6e35740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5590398cb8e0 0x5590398cdd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:24:12.278 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:12.275+0000 7f5eb6e35740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x55903982ac90 con 0x5590398c5d80 2024-03-21T10:24:12.278 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:12.275+0000 7f5eb6e35740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x5590397d3980 con 0x5590398c6770 2024-03-21T10:24:12.278 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:12.275+0000 7f5eb6e35740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x559039826d30 con 0x5590398cb8e0 2024-03-21T10:24:12.278 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:12.275+0000 7f5eb2000640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5590398c5d80 0x5590398c61a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:24:12.278 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:12.275+0000 7f5eb2000640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5590398c5d80 0x5590398c61a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:40818/0 (socket says 172.21.15.47:40818) 2024-03-21T10:24:12.279 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:12.275+0000 7f5eb2000640 1 -- 172.21.15.47:0/2944785114 learned_addr learned my addr 172.21.15.47:0/2944785114 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:24:12.279 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:12.275+0000 7f5eb1600640 1 --2- 172.21.15.47:0/2944785114 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5590398c6770 0x5590398cb270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:24:12.279 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:12.275+0000 7f5eb0c00640 1 --2- 172.21.15.47:0/2944785114 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5590398cb8e0 0x5590398cdd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:24:12.279 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:12.276+0000 7f5eb2000640 1 -- 172.21.15.47:0/2944785114 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5590398cb8e0 msgr2=0x5590398cdd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:24:12.279 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:12.276+0000 7f5eb2000640 1 --2- 172.21.15.47:0/2944785114 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5590398cb8e0 0x5590398cdd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:24:12.279 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:12.276+0000 7f5eb2000640 1 -- 172.21.15.47:0/2944785114 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5590398c6770 msgr2=0x5590398cb270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:24:12.279 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:12.276+0000 7f5eb2000640 1 --2- 172.21.15.47:0/2944785114 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5590398c6770 0x5590398cb270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:24:12.279 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:12.276+0000 7f5eb2000640 1 -- 172.21.15.47:0/2944785114 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5590397eb510 con 0x5590398c5d80 2024-03-21T10:24:12.279 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:12.276+0000 7f5eaa000640 1 --2- 172.21.15.47:0/3130541723 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f5e940a2fa0 0x7f5e940a5460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:24:12.282 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:12.276+0000 7f5eb2000640 1 --2- 172.21.15.47:0/2944785114 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5590398c5d80 0x5590398c61a0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f5ea4009ca0 tx=0x7f5ea4011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fbf323adbf8e2edf server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:24:12.282 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:12.276+0000 7f5eaa000640 1 -- 172.21.15.47:0/3130541723 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9216+0+0 (secure 0 0 0) 0x7f5ea00f19c0 con 0x7f5eac102000 2024-03-21T10:24:12.282 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:12.276+0000 7f5eb1600640 1 --2- 172.21.15.47:0/3130541723 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f5e940a2fa0 0x7f5e940a5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:24:12.282 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:12.276+0000 7f5e9e600640 1 -- 172.21.15.47:0/2944785114 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ea401e070 con 0x5590398c5d80 2024-03-21T10:24:12.282 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:12.277+0000 7f5e9e600640 1 -- 172.21.15.47:0/2944785114 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f5ea4009e50 con 0x5590398c5d80 2024-03-21T10:24:12.282 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:12.277+0000 7f5e9e600640 1 -- 172.21.15.47:0/2944785114 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ea400ea50 con 0x5590398c5d80 2024-03-21T10:24:12.282 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:12.277+0000 7f5eb6e35740 1 -- 172.21.15.47:0/2944785114 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5590398c5d80 msgr2=0x5590398c61a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:24:12.282 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:12.277+0000 7f5eb1600640 1 --2- 172.21.15.47:0/3130541723 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f5e940a2fa0 0x7f5e940a5460 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f5ea00bd9e0 tx=0x7f5ea00f7530 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:24:12.282 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:12.277+0000 7f5eb6e35740 1 --2- 172.21.15.47:0/2944785114 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5590398c5d80 0x5590398c61a0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f5ea4009ca0 tx=0x7f5ea4011810 comp rx=0 tx=0).stop 2024-03-21T10:24:12.282 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:12.278+0000 7f5eb6e35740 1 -- 172.21.15.47:0/2944785114 shutdown_connections 2024-03-21T10:24:12.282 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:12.278+0000 7f5eb6e35740 1 --2- 172.21.15.47:0/2944785114 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5590398cb8e0 0x5590398cdd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:24:12.283 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:12.278+0000 7f5eb6e35740 1 --2- 172.21.15.47:0/2944785114 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5590398c6770 0x5590398cb270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:24:12.283 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:12.278+0000 7f5eb6e35740 1 --2- 172.21.15.47:0/2944785114 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5590398c5d80 0x5590398c61a0 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:24:12.283 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:12.278+0000 7f5eb6e35740 1 -- 172.21.15.47:0/2944785114 >> 172.21.15.47:0/2944785114 conn(0x5590398c12f0 msgr2=0x5590398c16f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:24:12.283 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:12.278+0000 7f5eb6e35740 1 -- 172.21.15.47:0/2944785114 shutdown_connections 2024-03-21T10:24:12.283 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:12.278+0000 7f5eb6e35740 1 -- 172.21.15.47:0/2944785114 wait complete. 2024-03-21T10:24:12.283 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:12.278+0000 7f5eb6e35740 1 -- start start 2024-03-21T10:24:12.283 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:12.278+0000 7f5eb6e35740 1 Processor -- start 2024-03-21T10:24:12.283 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:12.279+0000 7f5eb6e35740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5590398c5d80 0x5590398d02b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:24:12.283 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:12.279+0000 7f5eb6e35740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5590398c6770 0x5590398d07f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:24:12.283 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:12.279+0000 7f5eb6e35740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5590398cb8e0 0x5590398d7830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:24:12.283 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:12.279+0000 7f5eb6e35740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x5590398ce890 con 0x5590398cb8e0 2024-03-21T10:24:12.283 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:12.279+0000 7f5eb6e35740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x5590397e6dd0 con 0x5590398c6770 2024-03-21T10:24:12.283 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:12.279+0000 7f5eb6e35740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x5590398cea10 con 0x5590398c5d80 2024-03-21T10:24:12.283 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:12.279+0000 7f5eb6e35740 10 client.0 _ll_register_callbacks cb 0x55903953eac0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T10:24:12.284 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:12.279+0000 7f5eb2000640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5590398cb8e0 0x5590398d7830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:24:12.284 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:12.279+0000 7f5eb2000640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5590398cb8e0 0x5590398d7830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:40824/0 (socket says 172.21.15.47:40824) 2024-03-21T10:24:12.284 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:12.279+0000 7f5eb2000640 1 -- 172.21.15.47:0/4064596706 learned_addr learned my addr 172.21.15.47:0/4064596706 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:24:12.284 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:12.279+0000 7f5eb0c00640 1 --2- 172.21.15.47:0/4064596706 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5590398c6770 0x5590398d07f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:24:12.284 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:12.279+0000 7f5eb2000640 1 --2- 172.21.15.47:0/4064596706 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5590398c5d80 0x5590398d02b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:24:12.284 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:12.280+0000 7f5eb2000640 1 -- 172.21.15.47:0/4064596706 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5590398c5d80 msgr2=0x5590398d02b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:24:12.284 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:12.280+0000 7f5eb2000640 1 --2- 172.21.15.47:0/4064596706 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5590398c5d80 0x5590398d02b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:24:12.284 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:12.280+0000 7f5eb2000640 1 -- 172.21.15.47:0/4064596706 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5590398c6770 msgr2=0x5590398d07f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:24:12.284 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:12.280+0000 7f5eb2000640 1 --2- 172.21.15.47:0/4064596706 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5590398c6770 0x5590398d07f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:24:12.284 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:12.280+0000 7f5eb2000640 1 -- 172.21.15.47:0/4064596706 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55903953f2d0 con 0x5590398cb8e0 2024-03-21T10:24:12.284 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:12.280+0000 7f5eb2000640 1 --2- 172.21.15.47:0/4064596706 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5590398cb8e0 0x5590398d7830 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f5ea40099a0 tx=0x7f5ea400e3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=af1a6c3aa307f936 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:24:12.284 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:12.280+0000 7f5e92a00640 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-03-21T10:24:12.286 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:12.280+0000 7f5e92a00640 1 -- 172.21.15.47:0/4064596706 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ea401e030 con 0x5590398cb8e0 2024-03-21T10:24:12.287 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:12.280+0000 7f5e92a00640 1 -- 172.21.15.47:0/4064596706 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9216+0+0 (secure 0 0 0) 0x7f5ea4004260 con 0x5590398cb8e0 2024-03-21T10:24:12.287 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:12.280+0000 7f5eb6e35740 1 -- 172.21.15.47:0/4064596706 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f5ea4009e50 con 0x5590398cb8e0 2024-03-21T10:24:12.287 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:12.281+0000 7f5e92a00640 1 -- 172.21.15.47:0/4064596706 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5ea4004460 con 0x5590398cb8e0 2024-03-21T10:24:12.287 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:12.281+0000 7f5eb6e35740 10 client.15198 fetch_fsmap learned FSMap version 29 2024-03-21T10:24:12.287 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:12.281+0000 7f5eb6e35740 1 -- 172.21.15.47:0/4064596706 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x5590396096b0 con 0x5590398cb8e0 2024-03-21T10:24:12.287 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:12.281+0000 7f5e92a00640 1 -- 172.21.15.47:0/4064596706 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f5ea4009e50 con 0x5590398cb8e0 2024-03-21T10:24:12.287 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:12.281+0000 7f5e92a00640 1 -- 172.21.15.47:0/4064596706 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 29) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f5ea4004bb0 con 0x5590398cb8e0 2024-03-21T10:24:12.287 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:12.281+0000 7f5eb6e35740 10 client.15198 fetch_fsmap finished waiting for FSMap version 29 2024-03-21T10:24:12.287 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:12.281+0000 7f5eb6e35740 10 client.15198 resolve_mds: resolved name 'a' to daemon mds.a 2024-03-21T10:24:12.287 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:12.281+0000 7f5eb6e35740 20 client.15198 populate_metadata read hostname 'smithi047' 2024-03-21T10:24:12.287 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:12.281+0000 7f5eb6e35740 1 --2- 172.21.15.47:0/4064596706 >> [v2:172.21.15.47:6840/3401578978,v1:172.21.15.47:6841/3401578978] conn(0x5590398dc6f0 0x5590398e0b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:24:12.287 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:12.281+0000 7f5eb6e35740 4 client.15198 mds_command: new command op to 14619 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T10:24:12.287 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:12.281+0000 7f5eb6e35740 1 -- 172.21.15.47:0/4064596706 --> [v2:172.21.15.47:6840/3401578978,v1:172.21.15.47:6841/3401578978] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5590397cc520 con 0x5590398dc6f0 2024-03-21T10:24:12.288 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:12.282+0000 7f5eb0c00640 1 --2- 172.21.15.47:0/4064596706 >> [v2:172.21.15.47:6840/3401578978,v1:172.21.15.47:6841/3401578978] conn(0x5590398dc6f0 0x5590398e0b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:24:12.288 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:12.282+0000 7f5eb0c00640 1 --2- 172.21.15.47:0/4064596706 >> [v2:172.21.15.47:6840/3401578978,v1:172.21.15.47:6841/3401578978] conn(0x5590398dc6f0 0x5590398e0b70 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=822a61324ffeff83 server_cookie=dea8fa5554f59219 in_seq=0 out_seq=0 2024-03-21T10:24:12.288 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:12.282+0000 7f5e92a00640 10 client.15198 ms_handle_connect on v2:172.21.15.47:6840/3401578978 2024-03-21T10:24:12.288 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:12.283+0000 7f5e92a00640 1 -- 172.21.15.47:0/4064596706 <== mds.4 v2:172.21.15.47:6840/3401578978 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x5590397cc520 con 0x5590398dc6f0 2024-03-21T10:24:12.288 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:12.283+0000 7f5e92a00640 10 client.15198 handle_command_reply: tid=0 2024-03-21T10:24:12.288 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:12.283+0000 7f5eb6e35740 1 client.15198 shutdown 2024-03-21T10:24:12.288 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:12.283+0000 7f5eb6e35740 1 -- 172.21.15.47:0/4064596706 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5590398cb8e0 msgr2=0x5590398d7830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:24:12.288 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:12.283+0000 7f5eb6e35740 1 --2- 172.21.15.47:0/4064596706 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5590398cb8e0 0x5590398d7830 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f5ea40099a0 tx=0x7f5ea400e3b0 comp rx=0 tx=0).stop 2024-03-21T10:24:12.288 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:12.283+0000 7f5eb6e35740 1 -- 172.21.15.47:0/4064596706 shutdown_connections 2024-03-21T10:24:12.288 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:12.283+0000 7f5eb6e35740 1 --2- 172.21.15.47:0/4064596706 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5590398cb8e0 0x5590398d7830 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:24:12.288 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:12.283+0000 7f5eb6e35740 1 --2- 172.21.15.47:0/4064596706 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5590398c6770 0x5590398d07f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:24:12.288 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:12.283+0000 7f5eb6e35740 1 --2- 172.21.15.47:0/4064596706 >> [v2:172.21.15.47:6840/3401578978,v1:172.21.15.47:6841/3401578978] conn(0x5590398dc6f0 0x5590398e0b70 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:24:12.288 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:12.283+0000 7f5eb6e35740 1 --2- 172.21.15.47:0/4064596706 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5590398c5d80 0x5590398d02b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:24:12.288 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:12.283+0000 7f5eb6e35740 1 -- 172.21.15.47:0/4064596706 >> 172.21.15.47:0/4064596706 conn(0x5590398c12f0 msgr2=0x5590398c04f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:24:12.289 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:12.283+0000 7f5e92a00640 0 client.15198 ms_handle_reset on v2:172.21.15.47:6840/3401578978 2024-03-21T10:24:12.289 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:12.284+0000 7f5eb6e35740 1 -- 172.21.15.47:0/4064596706 shutdown_connections 2024-03-21T10:24:12.289 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:12.284+0000 7f5eb6e35740 1 -- 172.21.15.47:0/4064596706 wait complete. 2024-03-21T10:24:12.289 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:12.284+0000 7f5eb6e35740 20 client.15198 trim_cache size 0 max 16384 2024-03-21T10:24:12.305 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:12.302+0000 7f5eb6e35740 1 Processor -- start 2024-03-21T10:24:12.305 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:12.302+0000 7f5eb6e35740 1 -- start start 2024-03-21T10:24:12.305 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:12.302+0000 7f5eb6e35740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5590398c5d80 0x5590398cedb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:24:12.305 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:12.302+0000 7f5eb6e35740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5590398c6770 0x5590398d77c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:24:12.305 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:12.302+0000 7f5eb6e35740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5590398cb8e0 0x5590398c16d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:24:12.306 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:12.302+0000 7f5eb6e35740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x5590398dbf10 con 0x5590398c6770 2024-03-21T10:24:12.306 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:12.302+0000 7f5eb6e35740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x5590398dc570 con 0x5590398cb8e0 2024-03-21T10:24:12.306 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:12.302+0000 7f5eb6e35740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x5590398dcbd0 con 0x5590398c5d80 2024-03-21T10:24:12.306 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:12.303+0000 7f5eb0c00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5590398c6770 0x5590398d77c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:24:12.306 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:12.303+0000 7f5eb0c00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5590398c6770 0x5590398d77c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:40826/0 (socket says 172.21.15.47:40826) 2024-03-21T10:24:12.306 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:12.303+0000 7f5eb0c00640 1 -- 172.21.15.47:0/2782968969 learned_addr learned my addr 172.21.15.47:0/2782968969 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:24:12.306 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:12.303+0000 7f5eb2000640 1 --2- 172.21.15.47:0/2782968969 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5590398cb8e0 0x5590398c16d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:24:12.306 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:12.303+0000 7f5eb2000640 1 --2- 172.21.15.47:0/2782968969 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5590398c5d80 0x5590398cedb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:24:12.306 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:12.303+0000 7f5eb0c00640 1 -- 172.21.15.47:0/2782968969 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5590398c5d80 msgr2=0x5590398cedb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:24:12.306 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:12.303+0000 7f5eb0c00640 1 --2- 172.21.15.47:0/2782968969 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5590398c5d80 0x5590398cedb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:24:12.306 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:12.303+0000 7f5eb0c00640 1 -- 172.21.15.47:0/2782968969 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5590398cb8e0 msgr2=0x5590398c16d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:24:12.306 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:12.303+0000 7f5eb0c00640 1 --2- 172.21.15.47:0/2782968969 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5590398cb8e0 0x5590398c16d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:24:12.306 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:12.303+0000 7f5eb0c00640 1 -- 172.21.15.47:0/2782968969 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5590398b3170 con 0x5590398c6770 2024-03-21T10:24:12.306 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:12.303+0000 7f5eb2000640 1 --2- 172.21.15.47:0/2782968969 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5590398cb8e0 0x5590398c16d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T10:24:12.308 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:12.303+0000 7f5eb0c00640 1 --2- 172.21.15.47:0/2782968969 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5590398c6770 0x5590398d77c0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x5590398c2260 tx=0x7f5e9801eb80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=29b037dfb988332a server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:24:12.308 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:12.304+0000 7f5e9dc00640 1 -- 172.21.15.47:0/2782968969 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e980142d0 con 0x5590398c6770 2024-03-21T10:24:12.308 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:12.304+0000 7f5e9dc00640 1 -- 172.21.15.47:0/2782968969 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f5e9801ee30 con 0x5590398c6770 2024-03-21T10:24:12.308 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:12.304+0000 7f5e9dc00640 1 -- 172.21.15.47:0/2782968969 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e980128b0 con 0x5590398c6770 2024-03-21T10:24:12.308 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:12.305+0000 7f5eb6e35740 1 -- 172.21.15.47:0/2782968969 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5590398c6770 msgr2=0x5590398d77c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:24:12.308 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:12.305+0000 7f5eb6e35740 1 --2- 172.21.15.47:0/2782968969 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5590398c6770 0x5590398d77c0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x5590398c2260 tx=0x7f5e9801eb80 comp rx=0 tx=0).stop 2024-03-21T10:24:12.308 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:12.305+0000 7f5eb6e35740 1 -- 172.21.15.47:0/2782968969 shutdown_connections 2024-03-21T10:24:12.308 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:12.305+0000 7f5eb6e35740 1 --2- 172.21.15.47:0/2782968969 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5590398cb8e0 0x5590398c16d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:24:12.308 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:12.305+0000 7f5eb6e35740 1 --2- 172.21.15.47:0/2782968969 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5590398c6770 0x5590398d77c0 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:24:12.308 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:12.305+0000 7f5eb6e35740 1 --2- 172.21.15.47:0/2782968969 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5590398c5d80 0x5590398cedb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:24:12.308 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:12.305+0000 7f5eb6e35740 1 -- 172.21.15.47:0/2782968969 >> 172.21.15.47:0/2782968969 conn(0x5590398c12f0 msgr2=0x5590398c7210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:24:12.308 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:12.305+0000 7f5eb6e35740 1 -- 172.21.15.47:0/2782968969 shutdown_connections 2024-03-21T10:24:12.308 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:12.305+0000 7f5eb6e35740 1 -- 172.21.15.47:0/2782968969 wait complete. 2024-03-21T10:24:12.309 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:12.306+0000 7f5eb6e35740 1 -- start start 2024-03-21T10:24:12.311 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:12.306+0000 7f5eb6e35740 1 Processor -- start 2024-03-21T10:24:12.311 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:12.307+0000 7f5eb6e35740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5590398c5d80 0x5590398d5790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:24:12.312 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:12.307+0000 7f5eb6e35740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5590398c6770 0x559039933c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:24:12.312 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:12.307+0000 7f5eb6e35740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5590398cb8e0 0x559039936290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:24:12.312 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:12.307+0000 7f5eb6e35740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x5590398dd890 con 0x5590398c5d80 2024-03-21T10:24:12.312 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:12.307+0000 7f5eb6e35740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x5590398dd230 con 0x5590398cb8e0 2024-03-21T10:24:12.312 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:12.307+0000 7f5eb6e35740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x5590398ddef0 con 0x5590398c6770 2024-03-21T10:24:12.312 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:12.307+0000 7f5eb6e35740 10 client.0 _ll_register_callbacks cb 0x7f5ea401c070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T10:24:12.312 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:12.307+0000 7f5eb2000640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5590398c5d80 0x5590398d5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:24:12.312 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:12.307+0000 7f5eb2000640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5590398c5d80 0x5590398d5790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:40832/0 (socket says 172.21.15.47:40832) 2024-03-21T10:24:12.312 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:12.307+0000 7f5eb2000640 1 -- 172.21.15.47:0/2605345353 learned_addr learned my addr 172.21.15.47:0/2605345353 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:24:12.312 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:12.307+0000 7f5eb2000640 1 --2- 172.21.15.47:0/2605345353 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5590398cb8e0 0x559039936290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:24:12.312 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:12.307+0000 7f5eb0c00640 1 --2- 172.21.15.47:0/2605345353 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5590398c6770 0x559039933c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:24:12.312 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:12.307+0000 7f5eb2000640 1 -- 172.21.15.47:0/2605345353 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5590398c6770 msgr2=0x559039933c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:24:12.312 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:12.307+0000 7f5eb2000640 1 --2- 172.21.15.47:0/2605345353 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5590398c6770 0x559039933c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:24:12.313 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:12.307+0000 7f5eb2000640 1 -- 172.21.15.47:0/2605345353 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5590398cb8e0 msgr2=0x559039936290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:24:12.313 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:12.307+0000 7f5eb2000640 1 --2- 172.21.15.47:0/2605345353 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5590398cb8e0 0x559039936290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:24:12.313 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:12.307+0000 7f5eb2000640 1 -- 172.21.15.47:0/2605345353 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x559039936ac0 con 0x5590398c5d80 2024-03-21T10:24:12.313 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:12.308+0000 7f5eb2000640 1 --2- 172.21.15.47:0/2605345353 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5590398c5d80 0x5590398d5790 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f5ea400ebc0 tx=0x7f5ea400e3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=33cd2c2d76eac616 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:24:12.313 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:12.308+0000 7f5e92a00640 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-03-21T10:24:12.313 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:12.308+0000 7f5e92a00640 1 -- 172.21.15.47:0/2605345353 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ea401e040 con 0x5590398c5d80 2024-03-21T10:24:12.313 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:12.308+0000 7f5e92a00640 1 -- 172.21.15.47:0/2605345353 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9216+0+0 (secure 0 0 0) 0x7f5ea4003b10 con 0x5590398c5d80 2024-03-21T10:24:12.313 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:12.308+0000 7f5eb6e35740 1 -- 172.21.15.47:0/2605345353 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f5e9801ee30 con 0x5590398c5d80 2024-03-21T10:24:12.313 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:12.308+0000 7f5e92a00640 1 -- 172.21.15.47:0/2605345353 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5ea4041050 con 0x5590398c5d80 2024-03-21T10:24:12.313 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:12.308+0000 7f5eb6e35740 10 client.15204 fetch_fsmap learned FSMap version 29 2024-03-21T10:24:12.313 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:12.308+0000 7f5eb6e35740 1 -- 172.21.15.47:0/2605345353 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x559039937df0 con 0x5590398c5d80 2024-03-21T10:24:12.313 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:12.309+0000 7f5e92a00640 1 -- 172.21.15.47:0/2605345353 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f5e9801ee30 con 0x5590398c5d80 2024-03-21T10:24:12.313 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:12.309+0000 7f5e92a00640 1 -- 172.21.15.47:0/2605345353 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 29) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f5ea4004600 con 0x5590398c5d80 2024-03-21T10:24:12.313 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:12.309+0000 7f5eb6e35740 10 client.15204 fetch_fsmap finished waiting for FSMap version 29 2024-03-21T10:24:12.314 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:12.309+0000 7f5eb6e35740 10 client.15204 resolve_mds: resolved name 'a' to daemon mds.a 2024-03-21T10:24:12.314 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:12.309+0000 7f5eb6e35740 20 client.15204 populate_metadata read hostname 'smithi047' 2024-03-21T10:24:12.314 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:12.309+0000 7f5eb6e35740 1 --2- 172.21.15.47:0/2605345353 >> [v2:172.21.15.47:6840/3401578978,v1:172.21.15.47:6841/3401578978] conn(0x55903993c4a0 0x5590399380e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:24:12.314 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:12.309+0000 7f5eb6e35740 4 client.15204 mds_command: new command op to 14619 tid=0 multi_id=0 [{"prefix": "status"}] 2024-03-21T10:24:12.314 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:12.309+0000 7f5eb6e35740 1 -- 172.21.15.47:0/2605345353 --> [v2:172.21.15.47:6840/3401578978,v1:172.21.15.47:6841/3401578978] -- command(tid 0: {"prefix": "status"}) v1 -- 0x5590398381d0 con 0x55903993c4a0 2024-03-21T10:24:12.314 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:12.309+0000 7f5eb0c00640 1 --2- 172.21.15.47:0/2605345353 >> [v2:172.21.15.47:6840/3401578978,v1:172.21.15.47:6841/3401578978] conn(0x55903993c4a0 0x5590399380e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:24:12.315 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:12.310+0000 7f5eb0c00640 1 --2- 172.21.15.47:0/2605345353 >> [v2:172.21.15.47:6840/3401578978,v1:172.21.15.47:6841/3401578978] conn(0x55903993c4a0 0x5590399380e0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=fce5af68a32b44e6 server_cookie=d460c2b7679f99e1 in_seq=0 out_seq=0 2024-03-21T10:24:12.315 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:12.310+0000 7f5e92a00640 10 client.15204 ms_handle_connect on v2:172.21.15.47:6840/3401578978 2024-03-21T10:24:12.315 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:12.310+0000 7f5e92a00640 1 -- 172.21.15.47:0/2605345353 <== mds.4 v2:172.21.15.47:6840/3401578978 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+322 (crc 0 0 0) 0x5590398381d0 con 0x55903993c4a0 2024-03-21T10:24:12.315 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:12.310+0000 7f5e92a00640 10 client.15204 handle_command_reply: tid=0 2024-03-21T10:24:12.316 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:12.310+0000 7f5eb6e35740 1 client.15204 shutdown 2024-03-21T10:24:12.316 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:12.311+0000 7f5eb6e35740 1 -- 172.21.15.47:0/2605345353 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5590398c5d80 msgr2=0x5590398d5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:24:12.316 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:12.311+0000 7f5eb6e35740 1 --2- 172.21.15.47:0/2605345353 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5590398c5d80 0x5590398d5790 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f5ea400ebc0 tx=0x7f5ea400e3b0 comp rx=0 tx=0).stop 2024-03-21T10:24:12.316 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:12.311+0000 7f5eb6e35740 1 -- 172.21.15.47:0/2605345353 shutdown_connections 2024-03-21T10:24:12.316 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:12.311+0000 7f5eb6e35740 1 --2- 172.21.15.47:0/2605345353 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5590398cb8e0 0x559039936290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:24:12.316 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:12.311+0000 7f5eb6e35740 1 --2- 172.21.15.47:0/2605345353 >> [v2:172.21.15.47:6840/3401578978,v1:172.21.15.47:6841/3401578978] conn(0x55903993c4a0 0x5590399380e0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:24:12.316 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:12.311+0000 7f5eb6e35740 1 --2- 172.21.15.47:0/2605345353 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5590398c6770 0x559039933c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:24:12.316 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:12.311+0000 7f5eb6e35740 1 --2- 172.21.15.47:0/2605345353 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5590398c5d80 0x5590398d5790 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:24:12.316 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:12.311+0000 7f5e92a00640 0 client.15204 ms_handle_reset on v2:172.21.15.47:6840/3401578978 2024-03-21T10:24:12.316 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:12.311+0000 7f5eb6e35740 1 -- 172.21.15.47:0/2605345353 >> 172.21.15.47:0/2605345353 conn(0x5590398c12f0 msgr2=0x5590398cedb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:24:12.316 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:12.311+0000 7f5eb6e35740 1 -- 172.21.15.47:0/2605345353 shutdown_connections 2024-03-21T10:24:12.317 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:12.311+0000 7f5eb6e35740 1 -- 172.21.15.47:0/2605345353 wait complete. 2024-03-21T10:24:12.317 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:12.311+0000 7f5eb6e35740 20 client.15204 trim_cache size 0 max 16384 2024-03-21T10:24:12.317 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-03-21T10:24:12.317 INFO:teuthology.orchestra.run.smithi047.stdout: "cluster_fsid": "3056d51a-e76b-11ee-95cd-87774f69a715", 2024-03-21T10:24:12.317 INFO:teuthology.orchestra.run.smithi047.stdout: "whoami": 4, 2024-03-21T10:24:12.317 INFO:teuthology.orchestra.run.smithi047.stdout: "id": 14619, 2024-03-21T10:24:12.317 INFO:teuthology.orchestra.run.smithi047.stdout: "want_state": "up:active", 2024-03-21T10:24:12.318 INFO:teuthology.orchestra.run.smithi047.stdout: "state": "up:active", 2024-03-21T10:24:12.318 INFO:teuthology.orchestra.run.smithi047.stdout: "fs_name": "cephfs", 2024-03-21T10:24:12.318 INFO:teuthology.orchestra.run.smithi047.stdout: "rank_uptime": 103.730875626, 2024-03-21T10:24:12.318 INFO:teuthology.orchestra.run.smithi047.stdout: "mdsmap_epoch": 29, 2024-03-21T10:24:12.318 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap_epoch": 95, 2024-03-21T10:24:12.318 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap_epoch_barrier": 93, 2024-03-21T10:24:12.318 INFO:teuthology.orchestra.run.smithi047.stdout: "uptime": 142.736479753 2024-03-21T10:24:12.318 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-03-21T10:24:12.319 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:12.312+0000 7f5eb2a00640 1 -- 172.21.15.47:0/3130541723 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f5e940a2fa0 msgr2=0x7f5e940a5460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:24:12.319 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:12.312+0000 7f5eb2a00640 1 --2- 172.21.15.47:0/3130541723 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f5e940a2fa0 0x7f5e940a5460 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f5ea00bd9e0 tx=0x7f5ea00f7530 comp rx=0 tx=0).stop 2024-03-21T10:24:12.319 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:12.312+0000 7f5eb2a00640 1 -- 172.21.15.47:0/3130541723 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5eac102000 msgr2=0x7f5eac1999d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:24:12.319 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:12.312+0000 7f5eb2a00640 1 --2- 172.21.15.47:0/3130541723 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5eac102000 0x7f5eac1999d0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f5ea0000f80 tx=0x7f5ea00328a0 comp rx=0 tx=0).stop 2024-03-21T10:24:12.319 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:12.312+0000 7f5eb2a00640 1 -- 172.21.15.47:0/3130541723 shutdown_connections 2024-03-21T10:24:12.319 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:12.312+0000 7f5eb2a00640 1 --2- 172.21.15.47:0/3130541723 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f5e940a2fa0 0x7f5e940a5460 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:24:12.319 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:12.312+0000 7f5eb2a00640 1 --2- 172.21.15.47:0/3130541723 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5eac107260 0x7f5eac1a0f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:24:12.319 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:12.312+0000 7f5eb2a00640 1 --2- 172.21.15.47:0/3130541723 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5eac104930 0x7f5eac199f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:24:12.319 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:12.312+0000 7f5eb2a00640 1 --2- 172.21.15.47:0/3130541723 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5eac102000 0x7f5eac1999d0 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:24:12.319 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:12.312+0000 7f5eb2a00640 1 -- 172.21.15.47:0/3130541723 >> 172.21.15.47:0/3130541723 conn(0x7f5eac0fba10 msgr2=0x7f5eac1016d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:24:12.319 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:12.312+0000 7f5eb2a00640 1 -- 172.21.15.47:0/3130541723 shutdown_connections 2024-03-21T10:24:12.320 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:12.312+0000 7f5eb2a00640 1 -- 172.21.15.47:0/3130541723 wait complete. 2024-03-21T10:24:12.332 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.k status 2024-03-21T10:24:12.546 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:12.544+0000 7f5d87e00640 1 -- 172.21.15.47:0/85529228 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5d80103260 msgr2=0x7f5d80109af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:24:12.547 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:12.544+0000 7f5d87e00640 1 --2- 172.21.15.47:0/85529228 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5d80103260 0x7f5d80109af0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f5d78009900 tx=0x7f5d7802bc30 comp rx=0 tx=0).stop 2024-03-21T10:24:12.547 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:12.544+0000 7f5d87e00640 1 -- 172.21.15.47:0/85529228 shutdown_connections 2024-03-21T10:24:12.547 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:12.544+0000 7f5d87e00640 1 --2- 172.21.15.47:0/85529228 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5d80103260 0x7f5d80109af0 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:24:12.547 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:12.544+0000 7f5d87e00640 1 --2- 172.21.15.47:0/85529228 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5d801028a0 0x7f5d80102d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:24:12.547 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:12.544+0000 7f5d87e00640 1 --2- 172.21.15.47:0/85529228 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5d801016b0 0x7f5d80101ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:24:12.547 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:12.544+0000 7f5d87e00640 1 -- 172.21.15.47:0/85529228 >> 172.21.15.47:0/85529228 conn(0x7f5d800fcc40 msgr2=0x7f5d800ff060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:24:12.547 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:12.544+0000 7f5d87e00640 1 -- 172.21.15.47:0/85529228 shutdown_connections 2024-03-21T10:24:12.547 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:12.544+0000 7f5d87e00640 1 -- 172.21.15.47:0/85529228 wait complete. 2024-03-21T10:24:12.549 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:12.545+0000 7f5d87e00640 1 Processor -- start 2024-03-21T10:24:12.549 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:12.545+0000 7f5d87e00640 1 -- start start 2024-03-21T10:24:12.549 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:12.546+0000 7f5d87e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5d801016b0 0x7f5d80199a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:24:12.550 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:12.546+0000 7f5d87e00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5d801028a0 0x7f5d80199fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:24:12.550 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:12.546+0000 7f5d87e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5d80103260 0x7f5d801a1030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:24:12.550 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:12.546+0000 7f5d87e00640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d8010d1c0 con 0x7f5d801016b0 2024-03-21T10:24:12.550 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:12.546+0000 7f5d87e00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d8010d040 con 0x7f5d80103260 2024-03-21T10:24:12.550 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:12.546+0000 7f5d87e00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d8010d340 con 0x7f5d801028a0 2024-03-21T10:24:12.550 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:12.546+0000 7f5d85400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5d801016b0 0x7f5d80199a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:24:12.550 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:12.546+0000 7f5d85400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5d801016b0 0x7f5d80199a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:40846/0 (socket says 172.21.15.47:40846) 2024-03-21T10:24:12.550 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:12.546+0000 7f5d85400640 1 -- 172.21.15.47:0/267205244 learned_addr learned my addr 172.21.15.47:0/267205244 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:24:12.550 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:12.546+0000 7f5d85e00640 1 --2- 172.21.15.47:0/267205244 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5d80103260 0x7f5d801a1030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:24:12.550 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:12.546+0000 7f5d84a00640 1 --2- 172.21.15.47:0/267205244 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5d801028a0 0x7f5d80199fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:24:12.550 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:12.547+0000 7f5d85400640 1 -- 172.21.15.47:0/267205244 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5d801028a0 msgr2=0x7f5d80199fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:24:12.550 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:12.547+0000 7f5d85400640 1 --2- 172.21.15.47:0/267205244 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5d801028a0 0x7f5d80199fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:24:12.550 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:12.547+0000 7f5d85400640 1 -- 172.21.15.47:0/267205244 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5d80103260 msgr2=0x7f5d801a1030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:24:12.551 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:12.547+0000 7f5d85400640 1 --2- 172.21.15.47:0/267205244 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5d80103260 0x7f5d801a1030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:24:12.551 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:12.547+0000 7f5d85400640 1 -- 172.21.15.47:0/267205244 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d801a16a0 con 0x7f5d801016b0 2024-03-21T10:24:12.551 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:12.547+0000 7f5d85400640 1 --2- 172.21.15.47:0/267205244 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5d801016b0 0x7f5d80199a80 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f5d74009fd0 tx=0x7f5d7400c270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:24:12.551 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:12.547+0000 7f5d7e000640 1 -- 172.21.15.47:0/267205244 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d74011470 con 0x7f5d801016b0 2024-03-21T10:24:12.551 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:12.547+0000 7f5d7e000640 1 -- 172.21.15.47:0/267205244 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f5d74011ac0 con 0x7f5d801016b0 2024-03-21T10:24:12.551 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:12.547+0000 7f5d87e00640 1 -- 172.21.15.47:0/267205244 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5d801a1990 con 0x7f5d801016b0 2024-03-21T10:24:12.551 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:12.547+0000 7f5d87e00640 1 -- 172.21.15.47:0/267205244 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5d801a1ee0 con 0x7f5d801016b0 2024-03-21T10:24:12.551 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:12.548+0000 7f5d7e000640 1 -- 172.21.15.47:0/267205244 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d7401aab0 con 0x7f5d801016b0 2024-03-21T10:24:12.555 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:12.552+0000 7f5d8abc7740 1 Processor -- start 2024-03-21T10:24:12.555 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:12.552+0000 7f5d8abc7740 1 -- start start 2024-03-21T10:24:12.555 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:12.552+0000 7f5d8abc7740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558a03fe7ba0 0x558a03fe7fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:24:12.555 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:12.552+0000 7f5d8abc7740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x558a03fe8590 0x558a03fed090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:24:12.555 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:12.552+0000 7f5d8abc7740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558a03fed700 0x558a03fefb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:24:12.555 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:12.553+0000 7f5d8abc7740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x558a03f4cc90 con 0x558a03fe8590 2024-03-21T10:24:12.555 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:12.553+0000 7f5d8abc7740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x558a03ef5980 con 0x558a03fed700 2024-03-21T10:24:12.555 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:12.553+0000 7f5d8abc7740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x558a03f48d30 con 0x558a03fe7ba0 2024-03-21T10:24:12.556 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:12.553+0000 7f5d7e000640 1 -- 172.21.15.47:0/267205244 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 18) v1 ==== 137854+0+0 (secure 0 0 0) 0x7f5d74011c70 con 0x7f5d801016b0 2024-03-21T10:24:12.556 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:12.553+0000 7f5d84a00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558a03fed700 0x558a03fefb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:24:12.556 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:12.553+0000 7f5d84a00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558a03fed700 0x558a03fefb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:41208/0 (socket says 172.21.15.47:41208) 2024-03-21T10:24:12.556 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:12.553+0000 7f5d84a00640 1 -- 172.21.15.47:0/717128399 learned_addr learned my addr 172.21.15.47:0/717128399 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:24:12.556 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:12.553+0000 7f5d85400640 1 --2- 172.21.15.47:0/717128399 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x558a03fe8590 0x558a03fed090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:24:12.556 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:12.553+0000 7f5d85e00640 1 --2- 172.21.15.47:0/717128399 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558a03fe7ba0 0x558a03fe7fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:24:12.558 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:12.554+0000 7f5d84a00640 1 -- 172.21.15.47:0/717128399 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558a03fe7ba0 msgr2=0x558a03fe7fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:24:12.558 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:12.554+0000 7f5d84a00640 1 --2- 172.21.15.47:0/717128399 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558a03fe7ba0 0x558a03fe7fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:24:12.558 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:12.554+0000 7f5d84a00640 1 -- 172.21.15.47:0/717128399 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x558a03fe8590 msgr2=0x558a03fed090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:24:12.558 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:12.554+0000 7f5d84a00640 1 --2- 172.21.15.47:0/717128399 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x558a03fe8590 0x558a03fed090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:24:12.558 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:12.554+0000 7f5d84a00640 1 -- 172.21.15.47:0/717128399 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558a03f0d510 con 0x558a03fed700 2024-03-21T10:24:12.558 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:12.554+0000 7f5d85400640 1 --2- 172.21.15.47:0/717128399 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x558a03fe8590 0x558a03fed090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T10:24:12.558 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:12.554+0000 7f5d84a00640 1 --2- 172.21.15.47:0/717128399 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558a03fed700 0x558a03fefb90 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f5d6c009ba0 tx=0x7f5d6c0117b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=eb12f3d8a6460958 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:24:12.558 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:12.554+0000 7f5d7e000640 1 --2- 172.21.15.47:0/267205244 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f5d680a2f20 0x7f5d680a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:24:12.558 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:12.554+0000 7f5d72600640 1 -- 172.21.15.47:0/717128399 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d6c004020 con 0x558a03fed700 2024-03-21T10:24:12.558 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:12.555+0000 7f5d72600640 1 -- 172.21.15.47:0/717128399 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f5d6c0041c0 con 0x558a03fed700 2024-03-21T10:24:12.558 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:12.555+0000 7f5d72600640 1 -- 172.21.15.47:0/717128399 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d6c0126b0 con 0x558a03fed700 2024-03-21T10:24:12.558 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:12.555+0000 7f5d7e000640 1 -- 172.21.15.47:0/267205244 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9216+0+0 (secure 0 0 0) 0x7f5d740cfd60 con 0x7f5d801016b0 2024-03-21T10:24:12.559 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:12.555+0000 7f5d85e00640 1 --2- 172.21.15.47:0/267205244 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f5d680a2f20 0x7f5d680a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:24:12.559 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:12.555+0000 7f5d8abc7740 1 -- 172.21.15.47:0/717128399 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558a03fed700 msgr2=0x558a03fefb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:24:12.559 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:12.555+0000 7f5d8abc7740 1 --2- 172.21.15.47:0/717128399 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558a03fed700 0x558a03fefb90 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f5d6c009ba0 tx=0x7f5d6c0117b0 comp rx=0 tx=0).stop 2024-03-21T10:24:12.559 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:12.556+0000 7f5d8abc7740 1 -- 172.21.15.47:0/717128399 shutdown_connections 2024-03-21T10:24:12.559 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:12.556+0000 7f5d8abc7740 1 --2- 172.21.15.47:0/717128399 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558a03fed700 0x558a03fefb90 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:24:12.559 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:12.556+0000 7f5d8abc7740 1 --2- 172.21.15.47:0/717128399 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x558a03fe8590 0x558a03fed090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:24:12.559 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:12.556+0000 7f5d8abc7740 1 --2- 172.21.15.47:0/717128399 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558a03fe7ba0 0x558a03fe7fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:24:12.559 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:12.556+0000 7f5d8abc7740 1 -- 172.21.15.47:0/717128399 >> 172.21.15.47:0/717128399 conn(0x558a03fe3110 msgr2=0x558a03fe3510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:24:12.559 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:12.556+0000 7f5d85e00640 1 --2- 172.21.15.47:0/267205244 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f5d680a2f20 0x7f5d680a53e0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f5d78006010 tx=0x7f5d7802c7e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:24:12.559 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:12.556+0000 7f5d8abc7740 1 -- 172.21.15.47:0/717128399 shutdown_connections 2024-03-21T10:24:12.559 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:12.556+0000 7f5d8abc7740 1 -- 172.21.15.47:0/717128399 wait complete. 2024-03-21T10:24:12.562 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:12.556+0000 7f5d8abc7740 1 -- start start 2024-03-21T10:24:12.562 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:12.556+0000 7f5d8abc7740 1 Processor -- start 2024-03-21T10:24:12.562 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:12.557+0000 7f5d8abc7740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x558a03fe7ba0 0x558a03ff2090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:24:12.563 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:12.557+0000 7f5d8abc7740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558a03fe8590 0x558a03ff25d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:24:12.563 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:12.557+0000 7f5d8abc7740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558a03fed700 0x558a03ff9610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:24:12.563 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:12.557+0000 7f5d8abc7740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x558a03ff06b0 con 0x558a03fe7ba0 2024-03-21T10:24:12.563 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:12.557+0000 7f5d8abc7740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x558a03f08dd0 con 0x558a03fed700 2024-03-21T10:24:12.563 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:12.557+0000 7f5d8abc7740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x558a03ff0830 con 0x558a03fe8590 2024-03-21T10:24:12.563 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:12.557+0000 7f5d8abc7740 10 client.0 _ll_register_callbacks cb 0x558a03c60ac0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T10:24:12.563 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:12.557+0000 7f5d85400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x558a03fe7ba0 0x558a03ff2090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:24:12.563 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:12.557+0000 7f5d85400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x558a03fe7ba0 0x558a03ff2090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:40866/0 (socket says 172.21.15.47:40866) 2024-03-21T10:24:12.563 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:12.557+0000 7f5d85400640 1 -- 172.21.15.47:0/340879013 learned_addr learned my addr 172.21.15.47:0/340879013 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:24:12.563 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:12.557+0000 7f5d85e00640 1 --2- 172.21.15.47:0/340879013 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558a03fed700 0x558a03ff9610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:24:12.563 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:12.557+0000 7f5d84a00640 1 --2- 172.21.15.47:0/340879013 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558a03fe8590 0x558a03ff25d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:24:12.563 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:12.558+0000 7f5d85400640 1 -- 172.21.15.47:0/340879013 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558a03fe8590 msgr2=0x558a03ff25d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:24:12.564 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:12.558+0000 7f5d85400640 1 --2- 172.21.15.47:0/340879013 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558a03fe8590 0x558a03ff25d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:24:12.564 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:12.558+0000 7f5d85400640 1 -- 172.21.15.47:0/340879013 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558a03fed700 msgr2=0x558a03ff9610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:24:12.564 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:12.558+0000 7f5d85400640 1 --2- 172.21.15.47:0/340879013 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558a03fed700 0x558a03ff9610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:24:12.564 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:12.558+0000 7f5d85400640 1 -- 172.21.15.47:0/340879013 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x558a03c612d0 con 0x558a03fe7ba0 2024-03-21T10:24:12.564 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:12.558+0000 7f5d84a00640 1 --2- 172.21.15.47:0/340879013 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558a03fe8590 0x558a03ff25d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T10:24:12.564 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:12.558+0000 7f5d85400640 1 --2- 172.21.15.47:0/340879013 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x558a03fe7ba0 0x558a03ff2090 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f5d740984e0 tx=0x7f5d74098320 comp rx=0 tx=0).ready entity=mon.0 client_cookie=abde549b7392c4ef server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:24:12.564 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:12.558+0000 7f5d66a00640 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-03-21T10:24:12.564 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:12.558+0000 7f5d66a00640 1 -- 172.21.15.47:0/340879013 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d7409b9e0 con 0x558a03fe7ba0 2024-03-21T10:24:12.564 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:12.558+0000 7f5d66a00640 1 -- 172.21.15.47:0/340879013 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9216+0+0 (secure 0 0 0) 0x7f5d740bc0b0 con 0x558a03fe7ba0 2024-03-21T10:24:12.564 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:12.558+0000 7f5d8abc7740 1 -- 172.21.15.47:0/340879013 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f5d6c0041c0 con 0x558a03fe7ba0 2024-03-21T10:24:12.564 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:12.559+0000 7f5d66a00640 1 -- 172.21.15.47:0/340879013 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5d740bc2b0 con 0x558a03fe7ba0 2024-03-21T10:24:12.564 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:12.559+0000 7f5d8abc7740 10 client.15216 fetch_fsmap learned FSMap version 29 2024-03-21T10:24:12.564 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:12.559+0000 7f5d8abc7740 1 -- 172.21.15.47:0/340879013 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x558a03d2b6b0 con 0x558a03fe7ba0 2024-03-21T10:24:12.564 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:12.559+0000 7f5d66a00640 1 -- 172.21.15.47:0/340879013 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f5d6c0041c0 con 0x558a03fe7ba0 2024-03-21T10:24:12.565 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:12.559+0000 7f5d66a00640 1 -- 172.21.15.47:0/340879013 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 29) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f5d74022030 con 0x558a03fe7ba0 2024-03-21T10:24:12.565 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:12.559+0000 7f5d8abc7740 10 client.15216 fetch_fsmap finished waiting for FSMap version 29 2024-03-21T10:24:12.565 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:12.559+0000 7f5d8abc7740 10 client.15216 resolve_mds: resolved name 'k' to daemon mds.k 2024-03-21T10:24:12.565 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:12.559+0000 7f5d8abc7740 20 client.15216 populate_metadata read hostname 'smithi047' 2024-03-21T10:24:12.565 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:12.559+0000 7f5d8abc7740 1 --2- 172.21.15.47:0/340879013 >> [v2:172.21.15.100:6832/2723302038,v1:172.21.15.100:6833/2723302038] conn(0x558a03ffe920 0x558a03ffda00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:24:12.565 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:12.559+0000 7f5d8abc7740 4 client.15216 mds_command: new command op to 34427 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T10:24:12.565 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:12.559+0000 7f5d8abc7740 1 -- 172.21.15.47:0/340879013 --> [v2:172.21.15.100:6832/2723302038,v1:172.21.15.100:6833/2723302038] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x558a03eee520 con 0x558a03ffe920 2024-03-21T10:24:12.565 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:12.560+0000 7f5d84a00640 1 --2- 172.21.15.47:0/340879013 >> [v2:172.21.15.100:6832/2723302038,v1:172.21.15.100:6833/2723302038] conn(0x558a03ffe920 0x558a03ffda00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:24:12.566 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:12.561+0000 7f5d84a00640 1 --2- 172.21.15.47:0/340879013 >> [v2:172.21.15.100:6832/2723302038,v1:172.21.15.100:6833/2723302038] conn(0x558a03ffe920 0x558a03ffda00 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=c43b68981fb55db6 server_cookie=ae61c00b6d8a25e3 in_seq=0 out_seq=0 2024-03-21T10:24:12.566 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:12.561+0000 7f5d66a00640 10 client.15216 ms_handle_connect on v2:172.21.15.100:6832/2723302038 2024-03-21T10:24:12.566 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:12.563+0000 7f5d66a00640 1 -- 172.21.15.47:0/340879013 <== mds.3 v2:172.21.15.100:6832/2723302038 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x558a03eee520 con 0x558a03ffe920 2024-03-21T10:24:12.567 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:12.563+0000 7f5d66a00640 10 client.15216 handle_command_reply: tid=0 2024-03-21T10:24:12.567 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:12.563+0000 7f5d8abc7740 1 client.15216 shutdown 2024-03-21T10:24:12.567 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:12.564+0000 7f5d8abc7740 1 -- 172.21.15.47:0/340879013 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x558a03fe7ba0 msgr2=0x558a03ff2090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:24:12.567 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:12.564+0000 7f5d8abc7740 1 --2- 172.21.15.47:0/340879013 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x558a03fe7ba0 0x558a03ff2090 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f5d740984e0 tx=0x7f5d74098320 comp rx=0 tx=0).stop 2024-03-21T10:24:12.567 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:12.564+0000 7f5d8abc7740 1 -- 172.21.15.47:0/340879013 shutdown_connections 2024-03-21T10:24:12.567 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:12.564+0000 7f5d8abc7740 1 --2- 172.21.15.47:0/340879013 >> [v2:172.21.15.100:6832/2723302038,v1:172.21.15.100:6833/2723302038] conn(0x558a03ffe920 0x558a03ffda00 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:24:12.567 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:12.564+0000 7f5d8abc7740 1 --2- 172.21.15.47:0/340879013 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558a03fed700 0x558a03ff9610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:24:12.567 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:12.564+0000 7f5d8abc7740 1 --2- 172.21.15.47:0/340879013 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558a03fe8590 0x558a03ff25d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:24:12.567 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:12.564+0000 7f5d8abc7740 1 --2- 172.21.15.47:0/340879013 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x558a03fe7ba0 0x558a03ff2090 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:24:12.567 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:12.564+0000 7f5d66a00640 0 client.15216 ms_handle_reset on v2:172.21.15.100:6832/2723302038 2024-03-21T10:24:12.567 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:12.564+0000 7f5d8abc7740 1 -- 172.21.15.47:0/340879013 >> 172.21.15.47:0/340879013 conn(0x558a03fe3110 msgr2=0x558a03fe2310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:24:12.567 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:12.564+0000 7f5d8abc7740 1 -- 172.21.15.47:0/340879013 shutdown_connections 2024-03-21T10:24:12.567 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:12.564+0000 7f5d8abc7740 1 -- 172.21.15.47:0/340879013 wait complete. 2024-03-21T10:24:12.567 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:12.564+0000 7f5d8abc7740 20 client.15216 trim_cache size 0 max 16384 2024-03-21T10:24:12.585 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:12.582+0000 7f5d8abc7740 1 Processor -- start 2024-03-21T10:24:12.585 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:12.583+0000 7f5d8abc7740 1 -- start start 2024-03-21T10:24:12.585 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:12.583+0000 7f5d8abc7740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558a03fe7ba0 0x558a03fe8e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:24:12.585 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:12.583+0000 7f5d8abc7740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x558a03fe8590 0x558a03ff95d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:24:12.585 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:12.583+0000 7f5d8abc7740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558a03fed700 0x558a03fe34f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:24:12.585 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:12.583+0000 7f5d8abc7740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x558a03ffe140 con 0x558a03fe8590 2024-03-21T10:24:12.586 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:12.583+0000 7f5d8abc7740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x558a03ffe7a0 con 0x558a03fe7ba0 2024-03-21T10:24:12.586 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:12.583+0000 7f5d8abc7740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x558a03ffee00 con 0x558a03fed700 2024-03-21T10:24:12.586 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:12.583+0000 7f5d84a00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x558a03fe8590 0x558a03ff95d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:24:12.586 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:12.583+0000 7f5d84a00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x558a03fe8590 0x558a03ff95d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:40868/0 (socket says 172.21.15.47:40868) 2024-03-21T10:24:12.586 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:12.583+0000 7f5d84a00640 1 -- 172.21.15.47:0/1914130997 learned_addr learned my addr 172.21.15.47:0/1914130997 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:24:12.586 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:12.583+0000 7f5d85400640 1 --2- 172.21.15.47:0/1914130997 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558a03fe7ba0 0x558a03fe8e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:24:12.587 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:12.583+0000 7f5d84a00640 1 -- 172.21.15.47:0/1914130997 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558a03fed700 msgr2=0x558a03fe34f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:24:12.587 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:12.583+0000 7f5d84a00640 1 --2- 172.21.15.47:0/1914130997 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558a03fed700 0x558a03fe34f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:24:12.587 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:12.583+0000 7f5d84a00640 1 -- 172.21.15.47:0/1914130997 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558a03fe7ba0 msgr2=0x558a03fe8e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:24:12.588 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:12.584+0000 7f5d84a00640 1 --2- 172.21.15.47:0/1914130997 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558a03fe7ba0 0x558a03fe8e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:24:12.588 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:12.584+0000 7f5d84a00640 1 -- 172.21.15.47:0/1914130997 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558a03fe8970 con 0x558a03fe8590 2024-03-21T10:24:12.588 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:12.584+0000 7f5d84a00640 1 --2- 172.21.15.47:0/1914130997 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x558a03fe8590 0x558a03ff95d0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f5d6c011cb0 tx=0x7f5d6c009ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3024edb9d0539a73 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:24:12.588 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:12.584+0000 7f5d71c00640 1 -- 172.21.15.47:0/1914130997 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d6c02e070 con 0x558a03fe8590 2024-03-21T10:24:12.588 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:12.584+0000 7f5d71c00640 1 -- 172.21.15.47:0/1914130997 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f5d6c029210 con 0x558a03fe8590 2024-03-21T10:24:12.588 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:12.584+0000 7f5d71c00640 1 -- 172.21.15.47:0/1914130997 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d6c029070 con 0x558a03fe8590 2024-03-21T10:24:12.588 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:12.585+0000 7f5d8abc7740 1 -- 172.21.15.47:0/1914130997 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x558a03fe8590 msgr2=0x558a03ff95d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:24:12.588 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:12.585+0000 7f5d8abc7740 1 --2- 172.21.15.47:0/1914130997 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x558a03fe8590 0x558a03ff95d0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f5d6c011cb0 tx=0x7f5d6c009ad0 comp rx=0 tx=0).stop 2024-03-21T10:24:12.588 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:12.585+0000 7f5d8abc7740 1 -- 172.21.15.47:0/1914130997 shutdown_connections 2024-03-21T10:24:12.588 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:12.585+0000 7f5d8abc7740 1 --2- 172.21.15.47:0/1914130997 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558a03fed700 0x558a03fe34f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:24:12.588 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:12.585+0000 7f5d8abc7740 1 --2- 172.21.15.47:0/1914130997 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x558a03fe8590 0x558a03ff95d0 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:24:12.588 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:12.585+0000 7f5d8abc7740 1 --2- 172.21.15.47:0/1914130997 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558a03fe7ba0 0x558a03fe8e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:24:12.588 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:12.585+0000 7f5d8abc7740 1 -- 172.21.15.47:0/1914130997 >> 172.21.15.47:0/1914130997 conn(0x558a03fe3110 msgr2=0x558a03ffda00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:24:12.589 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:12.585+0000 7f5d8abc7740 1 -- 172.21.15.47:0/1914130997 shutdown_connections 2024-03-21T10:24:12.589 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:12.585+0000 7f5d8abc7740 1 -- 172.21.15.47:0/1914130997 wait complete. 2024-03-21T10:24:12.589 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:12.585+0000 7f5d8abc7740 1 -- start start 2024-03-21T10:24:12.589 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:12.586+0000 7f5d8abc7740 1 Processor -- start 2024-03-21T10:24:12.593 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:12.586+0000 7f5d8abc7740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558a03fe7ba0 0x558a03ff8ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:24:12.593 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:12.586+0000 7f5d8abc7740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558a03fe8590 0x558a03ff9410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:24:12.593 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:12.586+0000 7f5d8abc7740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x558a03fed700 0x558a03ff59c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:24:12.593 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:12.586+0000 7f5d8abc7740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x558a03fffac0 con 0x558a03fed700 2024-03-21T10:24:12.593 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:12.586+0000 7f5d8abc7740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x558a03fff460 con 0x558a03fe8590 2024-03-21T10:24:12.593 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:12.586+0000 7f5d8abc7740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x558a04000120 con 0x558a03fe7ba0 2024-03-21T10:24:12.593 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:12.586+0000 7f5d8abc7740 10 client.0 _ll_register_callbacks cb 0x7f5d7401a400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T10:24:12.594 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:12.586+0000 7f5d85e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x558a03fed700 0x558a03ff59c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:24:12.594 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:12.586+0000 7f5d85e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x558a03fed700 0x558a03ff59c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:40878/0 (socket says 172.21.15.47:40878) 2024-03-21T10:24:12.594 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:12.586+0000 7f5d85e00640 1 -- 172.21.15.47:0/2433551387 learned_addr learned my addr 172.21.15.47:0/2433551387 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:24:12.594 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:12.586+0000 7f5d84a00640 1 --2- 172.21.15.47:0/2433551387 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558a03fe8590 0x558a03ff9410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:24:12.594 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:12.587+0000 7f5d85400640 1 --2- 172.21.15.47:0/2433551387 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558a03fe7ba0 0x558a03ff8ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:24:12.594 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:12.587+0000 7f5d85e00640 1 -- 172.21.15.47:0/2433551387 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558a03fe7ba0 msgr2=0x558a03ff8ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:24:12.594 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:12.587+0000 7f5d85e00640 1 --2- 172.21.15.47:0/2433551387 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558a03fe7ba0 0x558a03ff8ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:24:12.594 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:12.587+0000 7f5d85e00640 1 -- 172.21.15.47:0/2433551387 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558a03fe8590 msgr2=0x558a03ff9410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:24:12.594 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:12.587+0000 7f5d85e00640 1 --2- 172.21.15.47:0/2433551387 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558a03fe8590 0x558a03ff9410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:24:12.595 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:12.587+0000 7f5d85e00640 1 -- 172.21.15.47:0/2433551387 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x558a03ff6130 con 0x558a03fed700 2024-03-21T10:24:12.595 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:12.587+0000 7f5d85e00640 1 --2- 172.21.15.47:0/2433551387 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x558a03fed700 0x558a03ff59c0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f5d78009900 tx=0x7f5d78004580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3782e1520522d3f1 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:24:12.595 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:12.587+0000 7f5d66a00640 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-03-21T10:24:12.595 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:12.587+0000 7f5d66a00640 1 -- 172.21.15.47:0/2433551387 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d780046c0 con 0x558a03fed700 2024-03-21T10:24:12.595 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:12.587+0000 7f5d8abc7740 1 -- 172.21.15.47:0/2433551387 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f5d6c029210 con 0x558a03fed700 2024-03-21T10:24:12.595 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:12.588+0000 7f5d66a00640 1 -- 172.21.15.47:0/2433551387 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9216+0+0 (secure 0 0 0) 0x7f5d78004860 con 0x558a03fed700 2024-03-21T10:24:12.595 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:12.588+0000 7f5d66a00640 1 -- 172.21.15.47:0/2433551387 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5d78004aa0 con 0x558a03fed700 2024-03-21T10:24:12.595 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:12.588+0000 7f5d8abc7740 10 client.15222 fetch_fsmap learned FSMap version 29 2024-03-21T10:24:12.595 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:12.588+0000 7f5d8abc7740 1 -- 172.21.15.47:0/2433551387 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x558a0405a6a0 con 0x558a03fed700 2024-03-21T10:24:12.595 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:12.588+0000 7f5d66a00640 1 -- 172.21.15.47:0/2433551387 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f5d6c029210 con 0x558a03fed700 2024-03-21T10:24:12.595 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:12.588+0000 7f5d66a00640 1 -- 172.21.15.47:0/2433551387 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 29) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f5d78034440 con 0x558a03fed700 2024-03-21T10:24:12.595 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:12.588+0000 7f5d8abc7740 10 client.15222 fetch_fsmap finished waiting for FSMap version 29 2024-03-21T10:24:12.595 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:12.589+0000 7f5d8abc7740 10 client.15222 resolve_mds: resolved name 'k' to daemon mds.k 2024-03-21T10:24:12.595 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:12.589+0000 7f5d8abc7740 20 client.15222 populate_metadata read hostname 'smithi047' 2024-03-21T10:24:12.596 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:12.589+0000 7f5d8abc7740 1 --2- 172.21.15.47:0/2433551387 >> [v2:172.21.15.100:6832/2723302038,v1:172.21.15.100:6833/2723302038] conn(0x558a0405edc0 0x558a0405a9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:24:12.596 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:12.589+0000 7f5d8abc7740 4 client.15222 mds_command: new command op to 34427 tid=0 multi_id=0 [{"prefix": "status"}] 2024-03-21T10:24:12.596 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:12.589+0000 7f5d8abc7740 1 -- 172.21.15.47:0/2433551387 --> [v2:172.21.15.100:6832/2723302038,v1:172.21.15.100:6833/2723302038] -- command(tid 0: {"prefix": "status"}) v1 -- 0x558a03f5a1d0 con 0x558a0405edc0 2024-03-21T10:24:12.596 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:12.589+0000 7f5d85400640 1 --2- 172.21.15.47:0/2433551387 >> [v2:172.21.15.100:6832/2723302038,v1:172.21.15.100:6833/2723302038] conn(0x558a0405edc0 0x558a0405a9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:24:12.596 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:12.590+0000 7f5d85400640 1 --2- 172.21.15.47:0/2433551387 >> [v2:172.21.15.100:6832/2723302038,v1:172.21.15.100:6833/2723302038] conn(0x558a0405edc0 0x558a0405a9e0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=13ebfbca840fbe1f server_cookie=dcbe4264aafc531d in_seq=0 out_seq=0 2024-03-21T10:24:12.596 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:12.590+0000 7f5d66a00640 10 client.15222 ms_handle_connect on v2:172.21.15.100:6832/2723302038 2024-03-21T10:24:12.596 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:12.590+0000 7f5d66a00640 1 -- 172.21.15.47:0/2433551387 <== mds.3 v2:172.21.15.100:6832/2723302038 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+332 (crc 0 0 0) 0x558a03f5a1d0 con 0x558a0405edc0 2024-03-21T10:24:12.596 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:12.590+0000 7f5d66a00640 10 client.15222 handle_command_reply: tid=0 2024-03-21T10:24:12.596 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:12.590+0000 7f5d8abc7740 1 client.15222 shutdown 2024-03-21T10:24:12.596 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:12.590+0000 7f5d8abc7740 1 -- 172.21.15.47:0/2433551387 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x558a03fed700 msgr2=0x558a03ff59c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:24:12.596 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:12.590+0000 7f5d8abc7740 1 --2- 172.21.15.47:0/2433551387 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x558a03fed700 0x558a03ff59c0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f5d78009900 tx=0x7f5d78004580 comp rx=0 tx=0).stop 2024-03-21T10:24:12.596 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:12.591+0000 7f5d8abc7740 1 -- 172.21.15.47:0/2433551387 shutdown_connections 2024-03-21T10:24:12.596 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:12.591+0000 7f5d8abc7740 1 --2- 172.21.15.47:0/2433551387 >> [v2:172.21.15.100:6832/2723302038,v1:172.21.15.100:6833/2723302038] conn(0x558a0405edc0 0x558a0405a9e0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:24:12.596 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:12.591+0000 7f5d8abc7740 1 --2- 172.21.15.47:0/2433551387 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x558a03fed700 0x558a03ff59c0 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:24:12.597 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:12.591+0000 7f5d8abc7740 1 --2- 172.21.15.47:0/2433551387 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558a03fe8590 0x558a03ff9410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:24:12.597 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:12.591+0000 7f5d8abc7740 1 --2- 172.21.15.47:0/2433551387 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558a03fe7ba0 0x558a03ff8ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:24:12.597 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:12.591+0000 7f5d66a00640 0 client.15222 ms_handle_reset on v2:172.21.15.100:6832/2723302038 2024-03-21T10:24:12.597 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:12.591+0000 7f5d8abc7740 1 -- 172.21.15.47:0/2433551387 >> 172.21.15.47:0/2433551387 conn(0x558a03fe3110 msgr2=0x558a03fe34f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:24:12.597 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:12.591+0000 7f5d8abc7740 1 -- 172.21.15.47:0/2433551387 shutdown_connections 2024-03-21T10:24:12.597 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:12.591+0000 7f5d8abc7740 1 -- 172.21.15.47:0/2433551387 wait complete. 2024-03-21T10:24:12.597 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:12.591+0000 7f5d8abc7740 20 client.15222 trim_cache size 0 max 16384 2024-03-21T10:24:12.597 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-03-21T10:24:12.597 INFO:teuthology.orchestra.run.smithi047.stdout: "cluster_fsid": "3056d51a-e76b-11ee-95cd-87774f69a715", 2024-03-21T10:24:12.597 INFO:teuthology.orchestra.run.smithi047.stdout: "whoami": 3, 2024-03-21T10:24:12.598 INFO:teuthology.orchestra.run.smithi047.stdout: "id": 34427, 2024-03-21T10:24:12.598 INFO:teuthology.orchestra.run.smithi047.stdout: "want_state": "up:active", 2024-03-21T10:24:12.598 INFO:teuthology.orchestra.run.smithi047.stdout: "state": "up:active", 2024-03-21T10:24:12.598 INFO:teuthology.orchestra.run.smithi047.stdout: "fs_name": "cephfs", 2024-03-21T10:24:12.598 INFO:teuthology.orchestra.run.smithi047.stdout: "rank_uptime": 104.36974087500001, 2024-03-21T10:24:12.598 INFO:teuthology.orchestra.run.smithi047.stdout: "mdsmap_epoch": 29, 2024-03-21T10:24:12.598 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap_epoch": 95, 2024-03-21T10:24:12.598 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap_epoch_barrier": 93, 2024-03-21T10:24:12.598 INFO:teuthology.orchestra.run.smithi047.stdout: "uptime": 179.29813447800001 2024-03-21T10:24:12.598 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-03-21T10:24:12.599 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:12.592+0000 7f5d87e00640 1 -- 172.21.15.47:0/267205244 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f5d680a2f20 msgr2=0x7f5d680a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:24:12.599 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:12.592+0000 7f5d87e00640 1 --2- 172.21.15.47:0/267205244 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f5d680a2f20 0x7f5d680a53e0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f5d78006010 tx=0x7f5d7802c7e0 comp rx=0 tx=0).stop 2024-03-21T10:24:12.599 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:12.592+0000 7f5d87e00640 1 -- 172.21.15.47:0/267205244 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5d801016b0 msgr2=0x7f5d80199a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:24:12.599 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:12.592+0000 7f5d87e00640 1 --2- 172.21.15.47:0/267205244 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5d801016b0 0x7f5d80199a80 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f5d74009fd0 tx=0x7f5d7400c270 comp rx=0 tx=0).stop 2024-03-21T10:24:12.599 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:12.592+0000 7f5d87e00640 1 -- 172.21.15.47:0/267205244 shutdown_connections 2024-03-21T10:24:12.599 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:12.592+0000 7f5d87e00640 1 --2- 172.21.15.47:0/267205244 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f5d680a2f20 0x7f5d680a53e0 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:24:12.599 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:12.592+0000 7f5d87e00640 1 --2- 172.21.15.47:0/267205244 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5d80103260 0x7f5d801a1030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:24:12.599 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:12.592+0000 7f5d87e00640 1 --2- 172.21.15.47:0/267205244 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5d801028a0 0x7f5d80199fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:24:12.599 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:12.592+0000 7f5d87e00640 1 --2- 172.21.15.47:0/267205244 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5d801016b0 0x7f5d80199a80 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:24:12.599 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:12.592+0000 7f5d87e00640 1 -- 172.21.15.47:0/267205244 >> 172.21.15.47:0/267205244 conn(0x7f5d800fcc40 msgr2=0x7f5d800738d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:24:12.600 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:12.592+0000 7f5d87e00640 1 -- 172.21.15.47:0/267205244 shutdown_connections 2024-03-21T10:24:12.600 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:12.592+0000 7f5d87e00640 1 -- 172.21.15.47:0/267205244 wait complete. 2024-03-21T10:24:12.612 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.e status 2024-03-21T10:24:12.817 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:12.814+0000 7fd493e00640 1 -- 172.21.15.47:0/1714092398 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd494074100 msgr2=0x7fd494074580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:24:12.817 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:12.814+0000 7fd493e00640 1 --2- 172.21.15.47:0/1714092398 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd494074100 0x7fd494074580 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fd47c009930 tx=0x7fd47c02bc30 comp rx=0 tx=0).stop 2024-03-21T10:24:12.817 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:12.815+0000 7fd493e00640 1 -- 172.21.15.47:0/1714092398 shutdown_connections 2024-03-21T10:24:12.817 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:12.815+0000 7fd493e00640 1 --2- 172.21.15.47:0/1714092398 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd49410db70 0x7fd49410ff60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:24:12.817 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:12.815+0000 7fd493e00640 1 --2- 172.21.15.47:0/1714092398 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd494074100 0x7fd494074580 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:24:12.817 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:12.815+0000 7fd493e00640 1 --2- 172.21.15.47:0/1714092398 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd4940756e0 0x7fd494073b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:24:12.817 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:12.815+0000 7fd493e00640 1 -- 172.21.15.47:0/1714092398 >> 172.21.15.47:0/1714092398 conn(0x7fd4940fcb10 msgr2=0x7fd4940fef50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:24:12.818 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:12.815+0000 7fd493e00640 1 -- 172.21.15.47:0/1714092398 shutdown_connections 2024-03-21T10:24:12.818 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:12.815+0000 7fd493e00640 1 -- 172.21.15.47:0/1714092398 wait complete. 2024-03-21T10:24:12.820 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:12.816+0000 7fd493e00640 1 Processor -- start 2024-03-21T10:24:12.820 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:12.816+0000 7fd493e00640 1 -- start start 2024-03-21T10:24:12.820 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:12.816+0000 7fd493e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd494074100 0x7fd49419de80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:24:12.820 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:12.816+0000 7fd493e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd4940756e0 0x7fd49419e3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:24:12.820 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:12.816+0000 7fd493e00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd49410db70 0x7fd4941a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:24:12.820 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:12.816+0000 7fd493e00640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4941136c0 con 0x7fd4940756e0 2024-03-21T10:24:12.820 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:12.816+0000 7fd493e00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd494113540 con 0x7fd494074100 2024-03-21T10:24:12.820 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:12.816+0000 7fd493e00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd494113840 con 0x7fd49410db70 2024-03-21T10:24:12.820 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:12.817+0000 7fd48be00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd4940756e0 0x7fd49419e3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:24:12.821 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:12.817+0000 7fd492a00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd494074100 0x7fd49419de80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:24:12.821 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:12.817+0000 7fd48be00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd4940756e0 0x7fd49419e3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:40902/0 (socket says 172.21.15.47:40902) 2024-03-21T10:24:12.821 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:12.817+0000 7fd48be00640 1 -- 172.21.15.47:0/481010747 learned_addr learned my addr 172.21.15.47:0/481010747 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:24:12.821 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:12.817+0000 7fd493400640 1 --2- 172.21.15.47:0/481010747 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd49410db70 0x7fd4941a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:24:12.821 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:12.817+0000 7fd48be00640 1 -- 172.21.15.47:0/481010747 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd49410db70 msgr2=0x7fd4941a53e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:24:12.821 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:12.817+0000 7fd48be00640 1 --2- 172.21.15.47:0/481010747 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd49410db70 0x7fd4941a53e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:24:12.821 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:12.817+0000 7fd48be00640 1 -- 172.21.15.47:0/481010747 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd494074100 msgr2=0x7fd49419de80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:24:12.821 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:12.817+0000 7fd48be00640 1 --2- 172.21.15.47:0/481010747 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd494074100 0x7fd49419de80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:24:12.821 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:12.817+0000 7fd48be00640 1 -- 172.21.15.47:0/481010747 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4941a5a50 con 0x7fd4940756e0 2024-03-21T10:24:12.821 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:12.817+0000 7fd493400640 1 --2- 172.21.15.47:0/481010747 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd49410db70 0x7fd4941a53e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T10:24:12.821 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:12.817+0000 7fd492a00640 1 --2- 172.21.15.47:0/481010747 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd494074100 0x7fd49419de80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T10:24:12.821 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:12.817+0000 7fd48be00640 1 --2- 172.21.15.47:0/481010747 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd4940756e0 0x7fd49419e3c0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7fd47c009230 tx=0x7fd47c0040c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:24:12.821 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:12.818+0000 7fd48b400640 1 -- 172.21.15.47:0/481010747 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd47c043020 con 0x7fd4940756e0 2024-03-21T10:24:12.822 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:12.818+0000 7fd48b400640 1 -- 172.21.15.47:0/481010747 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fd47c0044c0 con 0x7fd4940756e0 2024-03-21T10:24:12.822 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:12.818+0000 7fd493e00640 1 -- 172.21.15.47:0/481010747 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd4941a5ce0 con 0x7fd4940756e0 2024-03-21T10:24:12.822 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:12.818+0000 7fd48b400640 1 -- 172.21.15.47:0/481010747 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd47c003bd0 con 0x7fd4940756e0 2024-03-21T10:24:12.822 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:12.818+0000 7fd493e00640 1 -- 172.21.15.47:0/481010747 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd4941a6220 con 0x7fd4940756e0 2024-03-21T10:24:12.823 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:12.821+0000 7fd49c5cd740 1 Processor -- start 2024-03-21T10:24:12.823 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:12.821+0000 7fd49c5cd740 1 -- start start 2024-03-21T10:24:12.824 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:12.822+0000 7fd49c5cd740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a3749f2d80 0x55a3749f31a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:24:12.824 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:12.822+0000 7fd48b400640 1 -- 172.21.15.47:0/481010747 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 18) v1 ==== 137854+0+0 (secure 0 0 0) 0x7fd47c046020 con 0x7fd4940756e0 2024-03-21T10:24:12.825 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:12.822+0000 7fd49c5cd740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a3749f3770 0x55a3749f8270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:24:12.825 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:12.822+0000 7fd49c5cd740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a3749f88e0 0x55a3749fad70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:24:12.825 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:12.822+0000 7fd49c5cd740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a374957c90 con 0x55a3749f2d80 2024-03-21T10:24:12.825 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:12.822+0000 7fd49c5cd740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a374900980 con 0x55a3749f3770 2024-03-21T10:24:12.825 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:12.822+0000 7fd49c5cd740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a374953d30 con 0x55a3749f88e0 2024-03-21T10:24:12.825 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:12.822+0000 7fd493400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a3749f2d80 0x55a3749f31a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:24:12.825 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:12.822+0000 7fd493400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a3749f2d80 0x55a3749f31a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:40904/0 (socket says 172.21.15.47:40904) 2024-03-21T10:24:12.825 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:12.822+0000 7fd493400640 1 -- 172.21.15.47:0/2595506547 learned_addr learned my addr 172.21.15.47:0/2595506547 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:24:12.825 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:12.822+0000 7fd492a00640 1 --2- 172.21.15.47:0/2595506547 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a3749f3770 0x55a3749f8270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:24:12.825 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:12.822+0000 7fd48be00640 1 --2- 172.21.15.47:0/2595506547 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a3749f88e0 0x55a3749fad70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:24:12.825 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:12.823+0000 7fd493400640 1 -- 172.21.15.47:0/2595506547 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a3749f88e0 msgr2=0x55a3749fad70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:24:12.825 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:12.823+0000 7fd493400640 1 --2- 172.21.15.47:0/2595506547 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a3749f88e0 0x55a3749fad70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:24:12.825 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:12.823+0000 7fd493400640 1 -- 172.21.15.47:0/2595506547 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a3749f3770 msgr2=0x55a3749f8270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:24:12.825 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:12.823+0000 7fd493400640 1 --2- 172.21.15.47:0/2595506547 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a3749f3770 0x55a3749f8270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:24:12.826 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:12.823+0000 7fd493400640 1 -- 172.21.15.47:0/2595506547 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a374918510 con 0x55a3749f2d80 2024-03-21T10:24:12.828 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:12.823+0000 7fd493400640 1 --2- 172.21.15.47:0/2595506547 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a3749f2d80 0x55a3749f31a0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7fd48c009f90 tx=0x7fd48c00f730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3de99a5eef85ff44 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:24:12.829 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:12.823+0000 7fd48b400640 1 --2- 172.21.15.47:0/481010747 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7fd4780a2fa0 0x7fd4780a5460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:24:12.829 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:12.823+0000 7fd483e00640 1 -- 172.21.15.47:0/2595506547 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd48c003b70 con 0x55a3749f2d80 2024-03-21T10:24:12.829 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:12.823+0000 7fd483e00640 1 -- 172.21.15.47:0/2595506547 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fd48c003d10 con 0x55a3749f2d80 2024-03-21T10:24:12.829 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:12.823+0000 7fd483e00640 1 -- 172.21.15.47:0/2595506547 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd48c011840 con 0x55a3749f2d80 2024-03-21T10:24:12.829 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:12.824+0000 7fd48b400640 1 -- 172.21.15.47:0/481010747 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9216+0+0 (secure 0 0 0) 0x7fd47c0f1a70 con 0x7fd4940756e0 2024-03-21T10:24:12.829 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:12.824+0000 7fd492a00640 1 --2- 172.21.15.47:0/481010747 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7fd4780a2fa0 0x7fd4780a5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:24:12.829 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:12.824+0000 7fd49c5cd740 1 -- 172.21.15.47:0/2595506547 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a3749f2d80 msgr2=0x55a3749f31a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:24:12.829 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:12.824+0000 7fd492a00640 1 --2- 172.21.15.47:0/481010747 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7fd4780a2fa0 0x7fd4780a5460 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x55a3749f3b70 tx=0x7fd48400b390 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:24:12.829 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:12.824+0000 7fd49c5cd740 1 --2- 172.21.15.47:0/2595506547 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a3749f2d80 0x55a3749f31a0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7fd48c009f90 tx=0x7fd48c00f730 comp rx=0 tx=0).stop 2024-03-21T10:24:12.829 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:12.825+0000 7fd49c5cd740 1 -- 172.21.15.47:0/2595506547 shutdown_connections 2024-03-21T10:24:12.829 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:12.825+0000 7fd49c5cd740 1 --2- 172.21.15.47:0/2595506547 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a3749f88e0 0x55a3749fad70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:24:12.829 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:12.825+0000 7fd49c5cd740 1 --2- 172.21.15.47:0/2595506547 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a3749f3770 0x55a3749f8270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:24:12.830 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:12.825+0000 7fd49c5cd740 1 --2- 172.21.15.47:0/2595506547 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a3749f2d80 0x55a3749f31a0 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:24:12.830 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:12.825+0000 7fd49c5cd740 1 -- 172.21.15.47:0/2595506547 >> 172.21.15.47:0/2595506547 conn(0x55a3749ee2f0 msgr2=0x55a3749ee6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:24:12.830 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:12.825+0000 7fd49c5cd740 1 -- 172.21.15.47:0/2595506547 shutdown_connections 2024-03-21T10:24:12.830 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:12.825+0000 7fd49c5cd740 1 -- 172.21.15.47:0/2595506547 wait complete. 2024-03-21T10:24:12.830 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:12.825+0000 7fd49c5cd740 1 -- start start 2024-03-21T10:24:12.830 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:12.825+0000 7fd49c5cd740 1 Processor -- start 2024-03-21T10:24:12.830 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:12.826+0000 7fd49c5cd740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a3749f2d80 0x55a3749fd2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:24:12.830 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:12.826+0000 7fd49c5cd740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a3749f3770 0x55a3749fd810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:24:12.830 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:12.826+0000 7fd49c5cd740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a3749f88e0 0x55a374a04850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:24:12.830 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:12.826+0000 7fd49c5cd740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a3749fb890 con 0x55a3749f3770 2024-03-21T10:24:12.830 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:12.826+0000 7fd49c5cd740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a374913dd0 con 0x55a3749f88e0 2024-03-21T10:24:12.830 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:12.826+0000 7fd49c5cd740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a3749fba10 con 0x55a3749f2d80 2024-03-21T10:24:12.830 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:12.826+0000 7fd49c5cd740 10 client.0 _ll_register_callbacks cb 0x55a37466bac0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T10:24:12.831 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:12.826+0000 7fd48be00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a3749f3770 0x55a3749fd810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:24:12.831 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:12.826+0000 7fd48be00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a3749f3770 0x55a3749fd810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:40918/0 (socket says 172.21.15.47:40918) 2024-03-21T10:24:12.831 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:12.826+0000 7fd48be00640 1 -- 172.21.15.47:0/893387342 learned_addr learned my addr 172.21.15.47:0/893387342 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:24:12.831 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:12.826+0000 7fd493400640 1 --2- 172.21.15.47:0/893387342 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a3749f88e0 0x55a374a04850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:24:12.831 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:12.826+0000 7fd492a00640 1 --2- 172.21.15.47:0/893387342 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a3749f2d80 0x55a3749fd2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:24:12.831 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:12.827+0000 7fd48be00640 1 -- 172.21.15.47:0/893387342 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a3749f2d80 msgr2=0x55a3749fd2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:24:12.831 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:12.827+0000 7fd48be00640 1 --2- 172.21.15.47:0/893387342 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a3749f2d80 0x55a3749fd2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:24:12.831 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:12.827+0000 7fd48be00640 1 -- 172.21.15.47:0/893387342 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a3749f88e0 msgr2=0x55a374a04850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:24:12.831 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:12.827+0000 7fd48be00640 1 --2- 172.21.15.47:0/893387342 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a3749f88e0 0x55a374a04850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:24:12.831 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:12.827+0000 7fd48be00640 1 -- 172.21.15.47:0/893387342 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55a37466c2d0 con 0x55a3749f3770 2024-03-21T10:24:12.835 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:12.827+0000 7fd48be00640 1 --2- 172.21.15.47:0/893387342 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a3749f3770 0x55a3749fd810 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7fd47c0bdad0 tx=0x7fd47c0de130 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fec12b96fd28b619 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:24:12.835 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:12.827+0000 7fd480c00640 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-03-21T10:24:12.835 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:12.827+0000 7fd480c00640 1 -- 172.21.15.47:0/893387342 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd47c0bd730 con 0x55a3749f3770 2024-03-21T10:24:12.835 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:12.827+0000 7fd480c00640 1 -- 172.21.15.47:0/893387342 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9216+0+0 (secure 0 0 0) 0x7fd47c0c6350 con 0x55a3749f3770 2024-03-21T10:24:12.835 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:12.827+0000 7fd49c5cd740 1 -- 172.21.15.47:0/893387342 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fd48c003d10 con 0x55a3749f3770 2024-03-21T10:24:12.835 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:12.828+0000 7fd480c00640 1 -- 172.21.15.47:0/893387342 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd47c0be4d0 con 0x55a3749f3770 2024-03-21T10:24:12.835 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:12.828+0000 7fd49c5cd740 10 client.15234 fetch_fsmap learned FSMap version 29 2024-03-21T10:24:12.835 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:12.828+0000 7fd49c5cd740 1 -- 172.21.15.47:0/893387342 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x55a3747366b0 con 0x55a3749f3770 2024-03-21T10:24:12.835 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:12.828+0000 7fd480c00640 1 -- 172.21.15.47:0/893387342 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fd48c003d10 con 0x55a3749f3770 2024-03-21T10:24:12.835 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:12.828+0000 7fd480c00640 1 -- 172.21.15.47:0/893387342 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 29) v1 ==== 7942+0+0 (secure 0 0 0) 0x7fd47c0bd220 con 0x55a3749f3770 2024-03-21T10:24:12.835 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:12.828+0000 7fd49c5cd740 10 client.15234 fetch_fsmap finished waiting for FSMap version 29 2024-03-21T10:24:12.835 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:12.828+0000 7fd49c5cd740 10 client.15234 resolve_mds: resolved name 'e' to daemon mds.e 2024-03-21T10:24:12.836 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:12.828+0000 7fd49c5cd740 20 client.15234 populate_metadata read hostname 'smithi047' 2024-03-21T10:24:12.836 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:12.828+0000 7fd49c5cd740 1 --2- 172.21.15.47:0/893387342 >> [v2:172.21.15.100:6834/2597849957,v1:172.21.15.100:6835/2597849957] conn(0x55a374a09ad0 0x55a374a089b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:24:12.836 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:12.828+0000 7fd49c5cd740 4 client.15234 mds_command: new command op to 34454 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T10:24:12.836 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:12.828+0000 7fd49c5cd740 1 -- 172.21.15.47:0/893387342 --> [v2:172.21.15.100:6834/2597849957,v1:172.21.15.100:6835/2597849957] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55a3748f9520 con 0x55a374a09ad0 2024-03-21T10:24:12.836 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:12.829+0000 7fd492a00640 1 --2- 172.21.15.47:0/893387342 >> [v2:172.21.15.100:6834/2597849957,v1:172.21.15.100:6835/2597849957] conn(0x55a374a09ad0 0x55a374a089b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:24:12.836 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:12.830+0000 7fd492a00640 1 --2- 172.21.15.47:0/893387342 >> [v2:172.21.15.100:6834/2597849957,v1:172.21.15.100:6835/2597849957] conn(0x55a374a09ad0 0x55a374a089b0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=47a71e4cbad137b5 server_cookie=741cddfc55f2a33 in_seq=0 out_seq=0 2024-03-21T10:24:12.836 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:12.830+0000 7fd480c00640 10 client.15234 ms_handle_connect on v2:172.21.15.100:6834/2597849957 2024-03-21T10:24:12.836 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:12.832+0000 7fd480c00640 1 -- 172.21.15.47:0/893387342 <== mds.2 v2:172.21.15.100:6834/2597849957 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55a3748f9520 con 0x55a374a09ad0 2024-03-21T10:24:12.836 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:12.832+0000 7fd480c00640 10 client.15234 handle_command_reply: tid=0 2024-03-21T10:24:12.836 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:12.832+0000 7fd49c5cd740 1 client.15234 shutdown 2024-03-21T10:24:12.836 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:12.832+0000 7fd49c5cd740 1 -- 172.21.15.47:0/893387342 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a3749f3770 msgr2=0x55a3749fd810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:24:12.836 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:12.832+0000 7fd49c5cd740 1 --2- 172.21.15.47:0/893387342 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a3749f3770 0x55a3749fd810 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7fd47c0bdad0 tx=0x7fd47c0de130 comp rx=0 tx=0).stop 2024-03-21T10:24:12.836 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:12.832+0000 7fd49c5cd740 1 -- 172.21.15.47:0/893387342 shutdown_connections 2024-03-21T10:24:12.836 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:12.832+0000 7fd49c5cd740 1 --2- 172.21.15.47:0/893387342 >> [v2:172.21.15.100:6834/2597849957,v1:172.21.15.100:6835/2597849957] conn(0x55a374a09ad0 0x55a374a089b0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:24:12.836 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:12.832+0000 7fd49c5cd740 1 --2- 172.21.15.47:0/893387342 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a3749f88e0 0x55a374a04850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:24:12.836 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:12.832+0000 7fd49c5cd740 1 --2- 172.21.15.47:0/893387342 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a3749f3770 0x55a3749fd810 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:24:12.837 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:12.832+0000 7fd49c5cd740 1 --2- 172.21.15.47:0/893387342 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a3749f2d80 0x55a3749fd2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:24:12.837 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:12.832+0000 7fd480c00640 0 client.15234 ms_handle_reset on v2:172.21.15.100:6834/2597849957 2024-03-21T10:24:12.837 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:12.832+0000 7fd49c5cd740 1 -- 172.21.15.47:0/893387342 >> 172.21.15.47:0/893387342 conn(0x55a3749ee2f0 msgr2=0x55a3749ed4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:24:12.837 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:12.832+0000 7fd49c5cd740 1 -- 172.21.15.47:0/893387342 shutdown_connections 2024-03-21T10:24:12.837 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:12.832+0000 7fd49c5cd740 1 -- 172.21.15.47:0/893387342 wait complete. 2024-03-21T10:24:12.837 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:12.832+0000 7fd49c5cd740 20 client.15234 trim_cache size 0 max 16384 2024-03-21T10:24:12.854 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:12.851+0000 7fd49c5cd740 1 Processor -- start 2024-03-21T10:24:12.854 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:12.851+0000 7fd49c5cd740 1 -- start start 2024-03-21T10:24:12.855 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:12.851+0000 7fd49c5cd740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a3749f2d80 0x55a3749f1c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:24:12.855 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:12.851+0000 7fd49c5cd740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a3749f3770 0x55a374a04810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:24:12.855 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:12.851+0000 7fd49c5cd740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a3749f88e0 0x55a374a11f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:24:12.855 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:12.851+0000 7fd49c5cd740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a374a092f0 con 0x55a3749f88e0 2024-03-21T10:24:12.855 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:12.851+0000 7fd49c5cd740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a374a09950 con 0x55a3749f2d80 2024-03-21T10:24:12.855 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:12.851+0000 7fd49c5cd740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a374a09fb0 con 0x55a3749f3770 2024-03-21T10:24:12.855 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:12.852+0000 7fd493400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a3749f88e0 0x55a374a11f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:24:12.855 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:12.852+0000 7fd492a00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a3749f2d80 0x55a3749f1c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:24:12.855 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:12.852+0000 7fd493400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a3749f88e0 0x55a374a11f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:40926/0 (socket says 172.21.15.47:40926) 2024-03-21T10:24:12.855 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:12.852+0000 7fd493400640 1 -- 172.21.15.47:0/2378632537 learned_addr learned my addr 172.21.15.47:0/2378632537 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:24:12.855 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:12.852+0000 7fd48be00640 1 --2- 172.21.15.47:0/2378632537 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a3749f3770 0x55a374a04810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:24:12.855 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:12.852+0000 7fd492a00640 1 -- 172.21.15.47:0/2378632537 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a3749f3770 msgr2=0x55a374a04810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:24:12.856 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:12.852+0000 7fd492a00640 1 --2- 172.21.15.47:0/2378632537 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a3749f3770 0x55a374a04810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:24:12.856 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:12.852+0000 7fd492a00640 1 -- 172.21.15.47:0/2378632537 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a3749f88e0 msgr2=0x55a374a11f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:24:12.856 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:12.852+0000 7fd492a00640 1 --2- 172.21.15.47:0/2378632537 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a3749f88e0 0x55a374a11f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:24:12.856 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:12.852+0000 7fd492a00640 1 -- 172.21.15.47:0/2378632537 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a3749f4000 con 0x55a3749f2d80 2024-03-21T10:24:12.856 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:12.852+0000 7fd493400640 1 --2- 172.21.15.47:0/2378632537 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a3749f88e0 0x55a374a11f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T10:24:12.856 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:12.852+0000 7fd48be00640 1 --2- 172.21.15.47:0/2378632537 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a3749f3770 0x55a374a04810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T10:24:12.856 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:12.852+0000 7fd492a00640 1 --2- 172.21.15.47:0/2378632537 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a3749f2d80 0x55a3749f1c60 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x55a374a0b720 tx=0x7fd484016000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e5e5eb3c1bd9714c server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:24:12.859 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:12.853+0000 7fd483400640 1 -- 172.21.15.47:0/2378632537 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd48402cdf0 con 0x55a3749f2d80 2024-03-21T10:24:12.860 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:12.853+0000 7fd483400640 1 -- 172.21.15.47:0/2378632537 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fd484030030 con 0x55a3749f2d80 2024-03-21T10:24:12.860 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:12.853+0000 7fd483400640 1 -- 172.21.15.47:0/2378632537 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd48401cc40 con 0x55a3749f2d80 2024-03-21T10:24:12.860 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:12.853+0000 7fd49c5cd740 1 -- 172.21.15.47:0/2378632537 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a3749f2d80 msgr2=0x55a3749f1c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:24:12.860 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:12.853+0000 7fd49c5cd740 1 --2- 172.21.15.47:0/2378632537 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a3749f2d80 0x55a3749f1c60 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x55a374a0b720 tx=0x7fd484016000 comp rx=0 tx=0).stop 2024-03-21T10:24:12.860 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:12.854+0000 7fd49c5cd740 1 -- 172.21.15.47:0/2378632537 shutdown_connections 2024-03-21T10:24:12.860 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:12.854+0000 7fd49c5cd740 1 --2- 172.21.15.47:0/2378632537 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a3749f88e0 0x55a374a11f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:24:12.860 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:12.854+0000 7fd49c5cd740 1 --2- 172.21.15.47:0/2378632537 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a3749f3770 0x55a374a04810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:24:12.860 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:12.854+0000 7fd49c5cd740 1 --2- 172.21.15.47:0/2378632537 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a3749f2d80 0x55a3749f1c60 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:24:12.860 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:12.854+0000 7fd49c5cd740 1 -- 172.21.15.47:0/2378632537 >> 172.21.15.47:0/2378632537 conn(0x55a3749ee2f0 msgr2=0x55a3749f1550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:24:12.860 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:12.854+0000 7fd49c5cd740 1 -- 172.21.15.47:0/2378632537 shutdown_connections 2024-03-21T10:24:12.860 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:12.854+0000 7fd49c5cd740 1 -- 172.21.15.47:0/2378632537 wait complete. 2024-03-21T10:24:12.860 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:12.854+0000 7fd49c5cd740 1 -- start start 2024-03-21T10:24:12.861 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:12.854+0000 7fd49c5cd740 1 Processor -- start 2024-03-21T10:24:12.861 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:12.855+0000 7fd49c5cd740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a3749f2d80 0x55a374a04700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:24:12.861 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:12.855+0000 7fd49c5cd740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a3749f3770 0x55a374a011f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:24:12.861 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:12.855+0000 7fd49c5cd740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a3749f88e0 0x55a374a01730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:24:12.861 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:12.855+0000 7fd49c5cd740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a374a0ac70 con 0x55a3749f2d80 2024-03-21T10:24:12.861 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:12.855+0000 7fd49c5cd740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a374a0a610 con 0x55a3749f88e0 2024-03-21T10:24:12.861 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:12.855+0000 7fd49c5cd740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a374a0b2d0 con 0x55a3749f3770 2024-03-21T10:24:12.861 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:12.855+0000 7fd49c5cd740 10 client.0 _ll_register_callbacks cb 0x7fd47c0057a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T10:24:12.861 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:12.855+0000 7fd492a00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a3749f2d80 0x55a374a04700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:24:12.861 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:12.855+0000 7fd492a00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a3749f2d80 0x55a374a04700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:40934/0 (socket says 172.21.15.47:40934) 2024-03-21T10:24:12.861 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:12.855+0000 7fd492a00640 1 -- 172.21.15.47:0/4182054917 learned_addr learned my addr 172.21.15.47:0/4182054917 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:24:12.861 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:12.855+0000 7fd493400640 1 --2- 172.21.15.47:0/4182054917 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a3749f88e0 0x55a374a01730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:24:12.861 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:12.855+0000 7fd48be00640 1 --2- 172.21.15.47:0/4182054917 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a3749f3770 0x55a374a011f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:24:12.862 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:12.855+0000 7fd492a00640 1 -- 172.21.15.47:0/4182054917 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a3749f3770 msgr2=0x55a374a011f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:24:12.862 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:12.855+0000 7fd492a00640 1 --2- 172.21.15.47:0/4182054917 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a3749f3770 0x55a374a011f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:24:12.862 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:12.855+0000 7fd492a00640 1 -- 172.21.15.47:0/4182054917 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a3749f88e0 msgr2=0x55a374a01730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:24:12.862 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:12.855+0000 7fd492a00640 1 --2- 172.21.15.47:0/4182054917 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a3749f88e0 0x55a374a01730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:24:12.862 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:12.855+0000 7fd492a00640 1 -- 172.21.15.47:0/4182054917 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55a374a64b30 con 0x55a3749f2d80 2024-03-21T10:24:12.862 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:12.855+0000 7fd48be00640 1 --2- 172.21.15.47:0/4182054917 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a3749f3770 0x55a374a011f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T10:24:12.862 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:12.856+0000 7fd492a00640 1 --2- 172.21.15.47:0/4182054917 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a3749f2d80 0x55a374a04700 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7fd484018040 tx=0x7fd48401c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5d52d5c45d5783a server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:24:12.862 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:12.856+0000 7fd480c00640 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-03-21T10:24:12.862 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:12.856+0000 7fd480c00640 1 -- 172.21.15.47:0/4182054917 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd48403c070 con 0x55a3749f2d80 2024-03-21T10:24:12.862 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:12.856+0000 7fd480c00640 1 -- 172.21.15.47:0/4182054917 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9216+0+0 (secure 0 0 0) 0x7fd48401c980 con 0x55a3749f2d80 2024-03-21T10:24:12.862 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:12.856+0000 7fd49c5cd740 1 -- 172.21.15.47:0/4182054917 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fd484030030 con 0x55a3749f2d80 2024-03-21T10:24:12.862 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:12.856+0000 7fd480c00640 1 -- 172.21.15.47:0/4182054917 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd484048050 con 0x55a3749f2d80 2024-03-21T10:24:12.862 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:12.856+0000 7fd49c5cd740 10 client.15240 fetch_fsmap learned FSMap version 29 2024-03-21T10:24:12.863 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:12.856+0000 7fd49c5cd740 1 -- 172.21.15.47:0/4182054917 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x55a374a65d70 con 0x55a3749f2d80 2024-03-21T10:24:12.863 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:12.857+0000 7fd480c00640 1 -- 172.21.15.47:0/4182054917 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fd484030030 con 0x55a3749f2d80 2024-03-21T10:24:12.863 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:12.857+0000 7fd480c00640 1 -- 172.21.15.47:0/4182054917 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 29) v1 ==== 7942+0+0 (secure 0 0 0) 0x7fd4840051f0 con 0x55a3749f2d80 2024-03-21T10:24:12.863 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:12.857+0000 7fd49c5cd740 10 client.15240 fetch_fsmap finished waiting for FSMap version 29 2024-03-21T10:24:12.863 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:12.857+0000 7fd49c5cd740 10 client.15240 resolve_mds: resolved name 'e' to daemon mds.e 2024-03-21T10:24:12.863 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:12.857+0000 7fd49c5cd740 20 client.15240 populate_metadata read hostname 'smithi047' 2024-03-21T10:24:12.863 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:12.857+0000 7fd49c5cd740 1 --2- 172.21.15.47:0/4182054917 >> [v2:172.21.15.100:6834/2597849957,v1:172.21.15.100:6835/2597849957] conn(0x55a374a6a0e0 0x55a374a66050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:24:12.863 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:12.857+0000 7fd49c5cd740 4 client.15240 mds_command: new command op to 34454 tid=0 multi_id=0 [{"prefix": "status"}] 2024-03-21T10:24:12.863 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:12.857+0000 7fd49c5cd740 1 -- 172.21.15.47:0/4182054917 --> [v2:172.21.15.100:6834/2597849957,v1:172.21.15.100:6835/2597849957] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55a3749651d0 con 0x55a374a6a0e0 2024-03-21T10:24:12.863 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:12.857+0000 7fd48be00640 1 --2- 172.21.15.47:0/4182054917 >> [v2:172.21.15.100:6834/2597849957,v1:172.21.15.100:6835/2597849957] conn(0x55a374a6a0e0 0x55a374a66050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:24:12.864 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:12.858+0000 7fd48be00640 1 --2- 172.21.15.47:0/4182054917 >> [v2:172.21.15.100:6834/2597849957,v1:172.21.15.100:6835/2597849957] conn(0x55a374a6a0e0 0x55a374a66050 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=66f3174fa7341d53 server_cookie=4db5c7267b759af0 in_seq=0 out_seq=0 2024-03-21T10:24:12.864 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:12.858+0000 7fd480c00640 10 client.15240 ms_handle_connect on v2:172.21.15.100:6834/2597849957 2024-03-21T10:24:12.865 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:12.858+0000 7fd480c00640 1 -- 172.21.15.47:0/4182054917 <== mds.2 v2:172.21.15.100:6834/2597849957 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+322 (crc 0 0 0) 0x55a3749651d0 con 0x55a374a6a0e0 2024-03-21T10:24:12.865 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:12.858+0000 7fd480c00640 10 client.15240 handle_command_reply: tid=0 2024-03-21T10:24:12.865 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:12.858+0000 7fd49c5cd740 1 client.15240 shutdown 2024-03-21T10:24:12.865 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:12.859+0000 7fd49c5cd740 1 -- 172.21.15.47:0/4182054917 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a3749f2d80 msgr2=0x55a374a04700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:24:12.865 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:12.859+0000 7fd49c5cd740 1 --2- 172.21.15.47:0/4182054917 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a3749f2d80 0x55a374a04700 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7fd484018040 tx=0x7fd48401c4a0 comp rx=0 tx=0).stop 2024-03-21T10:24:12.865 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:12.859+0000 7fd49c5cd740 1 -- 172.21.15.47:0/4182054917 shutdown_connections 2024-03-21T10:24:12.865 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:12.859+0000 7fd49c5cd740 1 --2- 172.21.15.47:0/4182054917 >> [v2:172.21.15.100:6834/2597849957,v1:172.21.15.100:6835/2597849957] conn(0x55a374a6a0e0 0x55a374a66050 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:24:12.865 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:12.859+0000 7fd49c5cd740 1 --2- 172.21.15.47:0/4182054917 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a3749f88e0 0x55a374a01730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:24:12.865 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:12.859+0000 7fd49c5cd740 1 --2- 172.21.15.47:0/4182054917 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a3749f3770 0x55a374a011f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:24:12.865 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:12.859+0000 7fd49c5cd740 1 --2- 172.21.15.47:0/4182054917 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a3749f2d80 0x55a374a04700 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:24:12.865 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:12.859+0000 7fd480c00640 0 client.15240 ms_handle_reset on v2:172.21.15.100:6834/2597849957 2024-03-21T10:24:12.865 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:12.859+0000 7fd49c5cd740 1 -- 172.21.15.47:0/4182054917 >> 172.21.15.47:0/4182054917 conn(0x55a3749ee2f0 msgr2=0x55a3749f1de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:24:12.865 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:12.859+0000 7fd49c5cd740 1 -- 172.21.15.47:0/4182054917 shutdown_connections 2024-03-21T10:24:12.865 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:12.859+0000 7fd49c5cd740 1 -- 172.21.15.47:0/4182054917 wait complete. 2024-03-21T10:24:12.865 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:12.859+0000 7fd49c5cd740 20 client.15240 trim_cache size 0 max 16384 2024-03-21T10:24:12.866 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-03-21T10:24:12.866 INFO:teuthology.orchestra.run.smithi047.stdout: "cluster_fsid": "3056d51a-e76b-11ee-95cd-87774f69a715", 2024-03-21T10:24:12.866 INFO:teuthology.orchestra.run.smithi047.stdout: "whoami": 2, 2024-03-21T10:24:12.866 INFO:teuthology.orchestra.run.smithi047.stdout: "id": 34454, 2024-03-21T10:24:12.866 INFO:teuthology.orchestra.run.smithi047.stdout: "want_state": "up:active", 2024-03-21T10:24:12.866 INFO:teuthology.orchestra.run.smithi047.stdout: "state": "up:active", 2024-03-21T10:24:12.866 INFO:teuthology.orchestra.run.smithi047.stdout: "fs_name": "cephfs", 2024-03-21T10:24:12.866 INFO:teuthology.orchestra.run.smithi047.stdout: "rank_uptime": 106.677781188, 2024-03-21T10:24:12.866 INFO:teuthology.orchestra.run.smithi047.stdout: "mdsmap_epoch": 29, 2024-03-21T10:24:12.866 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap_epoch": 95, 2024-03-21T10:24:12.867 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap_epoch_barrier": 93, 2024-03-21T10:24:12.867 INFO:teuthology.orchestra.run.smithi047.stdout: "uptime": 167.213034292 2024-03-21T10:24:12.867 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-03-21T10:24:12.867 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:12.860+0000 7fd493e00640 1 -- 172.21.15.47:0/481010747 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7fd4780a2fa0 msgr2=0x7fd4780a5460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:24:12.867 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:12.860+0000 7fd493e00640 1 --2- 172.21.15.47:0/481010747 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7fd4780a2fa0 0x7fd4780a5460 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x55a3749f3b70 tx=0x7fd48400b390 comp rx=0 tx=0).stop 2024-03-21T10:24:12.867 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:12.860+0000 7fd493e00640 1 -- 172.21.15.47:0/481010747 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd4940756e0 msgr2=0x7fd49419e3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:24:12.868 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:12.860+0000 7fd493e00640 1 --2- 172.21.15.47:0/481010747 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd4940756e0 0x7fd49419e3c0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7fd47c009230 tx=0x7fd47c0040c0 comp rx=0 tx=0).stop 2024-03-21T10:24:12.868 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:12.860+0000 7fd493e00640 1 -- 172.21.15.47:0/481010747 shutdown_connections 2024-03-21T10:24:12.868 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:12.860+0000 7fd493e00640 1 --2- 172.21.15.47:0/481010747 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7fd4780a2fa0 0x7fd4780a5460 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:24:12.868 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:12.860+0000 7fd493e00640 1 --2- 172.21.15.47:0/481010747 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd49410db70 0x7fd4941a53e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:24:12.868 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:12.860+0000 7fd493e00640 1 --2- 172.21.15.47:0/481010747 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd4940756e0 0x7fd49419e3c0 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:24:12.868 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:12.860+0000 7fd493e00640 1 --2- 172.21.15.47:0/481010747 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd494074100 0x7fd49419de80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:24:12.868 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:12.860+0000 7fd493e00640 1 -- 172.21.15.47:0/481010747 >> 172.21.15.47:0/481010747 conn(0x7fd4940fcb10 msgr2=0x7fd494100e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:24:12.868 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:12.860+0000 7fd493e00640 1 -- 172.21.15.47:0/481010747 shutdown_connections 2024-03-21T10:24:12.868 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:12.860+0000 7fd493e00640 1 -- 172.21.15.47:0/481010747 wait complete. 2024-03-21T10:24:12.879 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.h status 2024-03-21T10:24:13.085 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:13.082+0000 7f616d400640 1 -- 172.21.15.47:0/2173661547 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6168103150 msgr2=0x7f61681099e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:24:13.085 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:13.082+0000 7f616d400640 1 --2- 172.21.15.47:0/2173661547 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6168103150 0x7f61681099e0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f6160009ca0 tx=0x7f616002c0f0 comp rx=0 tx=0).stop 2024-03-21T10:24:13.085 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:13.082+0000 7f616d400640 1 -- 172.21.15.47:0/2173661547 shutdown_connections 2024-03-21T10:24:13.085 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:13.082+0000 7f616d400640 1 --2- 172.21.15.47:0/2173661547 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6168103150 0x7f61681099e0 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:24:13.085 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:13.082+0000 7f616d400640 1 --2- 172.21.15.47:0/2173661547 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6168102790 0x7f6168102c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:24:13.085 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:13.082+0000 7f616d400640 1 --2- 172.21.15.47:0/2173661547 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f61681015a0 0x7f61681019c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:24:13.085 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:13.082+0000 7f616d400640 1 -- 172.21.15.47:0/2173661547 >> 172.21.15.47:0/2173661547 conn(0x7f61680fcb10 msgr2=0x7f61680fef50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:24:13.085 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:13.082+0000 7f616d400640 1 -- 172.21.15.47:0/2173661547 shutdown_connections 2024-03-21T10:24:13.086 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:13.082+0000 7f616d400640 1 -- 172.21.15.47:0/2173661547 wait complete. 2024-03-21T10:24:13.088 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:13.084+0000 7f616d400640 1 Processor -- start 2024-03-21T10:24:13.088 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:13.084+0000 7f616d400640 1 -- start start 2024-03-21T10:24:13.088 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:13.084+0000 7f616d400640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f61681015a0 0x7f61681999c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:24:13.088 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:13.085+0000 7f616d400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6168102790 0x7f6168199f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:24:13.089 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:13.085+0000 7f616d400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6168103150 0x7f61681a0f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:24:13.089 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:13.085+0000 7f616d400640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f616810d0b0 con 0x7f6168103150 2024-03-21T10:24:13.089 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:13.085+0000 7f616d400640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f616810cf30 con 0x7f6168102790 2024-03-21T10:24:13.089 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:13.085+0000 7f616d400640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f616810d230 con 0x7f61681015a0 2024-03-21T10:24:13.089 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:13.085+0000 7f616ca00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6168103150 0x7f61681a0f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:24:13.089 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:13.085+0000 7f616ca00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6168103150 0x7f61681a0f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:40950/0 (socket says 172.21.15.47:40950) 2024-03-21T10:24:13.089 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:13.085+0000 7f616ca00640 1 -- 172.21.15.47:0/64977336 learned_addr learned my addr 172.21.15.47:0/64977336 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:24:13.089 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:13.085+0000 7f615f400640 1 --2- 172.21.15.47:0/64977336 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6168102790 0x7f6168199f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:24:13.089 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:13.085+0000 7f6167e00640 1 --2- 172.21.15.47:0/64977336 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f61681015a0 0x7f61681999c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:24:13.089 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:13.085+0000 7f616ca00640 1 -- 172.21.15.47:0/64977336 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f61681015a0 msgr2=0x7f61681999c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:24:13.089 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:13.085+0000 7f616ca00640 1 --2- 172.21.15.47:0/64977336 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f61681015a0 0x7f61681999c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:24:13.089 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:13.085+0000 7f616ca00640 1 -- 172.21.15.47:0/64977336 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6168102790 msgr2=0x7f6168199f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:24:13.089 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:13.085+0000 7f616ca00640 1 --2- 172.21.15.47:0/64977336 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6168102790 0x7f6168199f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:24:13.089 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:13.085+0000 7f616ca00640 1 -- 172.21.15.47:0/64977336 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61681a15e0 con 0x7f6168103150 2024-03-21T10:24:13.090 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:13.085+0000 7f615f400640 1 --2- 172.21.15.47:0/64977336 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6168102790 0x7f6168199f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T10:24:13.090 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:13.086+0000 7f616ca00640 1 --2- 172.21.15.47:0/64977336 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6168103150 0x7f61681a0f70 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f6160032550 tx=0x7f6160032820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:24:13.090 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:13.086+0000 7f6165600640 1 -- 172.21.15.47:0/64977336 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6160041020 con 0x7f6168103150 2024-03-21T10:24:13.090 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:13.086+0000 7f6165600640 1 -- 172.21.15.47:0/64977336 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f6160003e20 con 0x7f6168103150 2024-03-21T10:24:13.090 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:13.086+0000 7f6165600640 1 -- 172.21.15.47:0/64977336 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6160039040 con 0x7f6168103150 2024-03-21T10:24:13.090 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:13.086+0000 7f616d400640 1 -- 172.21.15.47:0/64977336 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f61681a1870 con 0x7f6168103150 2024-03-21T10:24:13.090 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:13.086+0000 7f616d400640 1 -- 172.21.15.47:0/64977336 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f61681a1d10 con 0x7f6168103150 2024-03-21T10:24:13.092 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:13.089+0000 7f6165600640 1 -- 172.21.15.47:0/64977336 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 18) v1 ==== 137854+0+0 (secure 0 0 0) 0x7f6160044020 con 0x7f6168103150 2024-03-21T10:24:13.092 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:13.090+0000 7f6171790740 1 Processor -- start 2024-03-21T10:24:13.092 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:13.090+0000 7f6171790740 1 -- start start 2024-03-21T10:24:13.094 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:13.090+0000 7f6171790740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f89e91ed80 0x55f89e91f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:24:13.094 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:13.090+0000 7f6171790740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f89e91f770 0x55f89e924270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:24:13.095 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:13.090+0000 7f6171790740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f89e9248e0 0x55f89e926d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:24:13.095 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:13.090+0000 7f6171790740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f89e883c90 con 0x55f89e9248e0 2024-03-21T10:24:13.095 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:13.090+0000 7f6171790740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f89e82c980 con 0x55f89e91ed80 2024-03-21T10:24:13.095 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:13.090+0000 7f6171790740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f89e87fd30 con 0x55f89e91f770 2024-03-21T10:24:13.095 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:13.091+0000 7f615f400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f89e91ed80 0x55f89e91f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:24:13.095 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:13.091+0000 7f615f400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f89e91ed80 0x55f89e91f1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:41344/0 (socket says 172.21.15.47:41344) 2024-03-21T10:24:13.095 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:13.091+0000 7f615f400640 1 -- 172.21.15.47:0/293204674 learned_addr learned my addr 172.21.15.47:0/293204674 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:24:13.095 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:13.091+0000 7f615f400640 1 --2- 172.21.15.47:0/293204674 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f89e9248e0 0x55f89e926d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:24:13.095 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:13.091+0000 7f6167e00640 1 --2- 172.21.15.47:0/293204674 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f89e91f770 0x55f89e924270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:24:13.095 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:13.091+0000 7f615f400640 1 -- 172.21.15.47:0/293204674 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f89e91f770 msgr2=0x55f89e924270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:24:13.095 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:13.091+0000 7f615f400640 1 --2- 172.21.15.47:0/293204674 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f89e91f770 0x55f89e924270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:24:13.095 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:13.091+0000 7f615f400640 1 -- 172.21.15.47:0/293204674 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f89e9248e0 msgr2=0x55f89e926d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:24:13.096 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:13.091+0000 7f615f400640 1 --2- 172.21.15.47:0/293204674 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f89e9248e0 0x55f89e926d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:24:13.096 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:13.091+0000 7f615f400640 1 -- 172.21.15.47:0/293204674 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f89e844510 con 0x55f89e91ed80 2024-03-21T10:24:13.096 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:13.092+0000 7f615f400640 1 --2- 172.21.15.47:0/293204674 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f89e91ed80 0x55f89e91f1a0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f6150014a30 tx=0x7f615000c570 comp rx=0 tx=0).ready entity=mon.2 client_cookie=389fc3069efbb268 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:24:13.096 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:13.092+0000 7f6165600640 1 --2- 172.21.15.47:0/64977336 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f61480a2fa0 0x7f61480a5460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:24:13.096 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:13.092+0000 7f6165600640 1 -- 172.21.15.47:0/64977336 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9216+0+0 (secure 0 0 0) 0x7f61600f1b00 con 0x7f6168103150 2024-03-21T10:24:13.096 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:13.092+0000 7f6167e00640 1 --2- 172.21.15.47:0/64977336 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f61480a2fa0 0x7f61480a5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:24:13.096 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:13.092+0000 7f615d200640 1 -- 172.21.15.47:0/293204674 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61500142d0 con 0x55f89e91ed80 2024-03-21T10:24:13.096 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:13.092+0000 7f615d200640 1 -- 172.21.15.47:0/293204674 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f615000f820 con 0x55f89e91ed80 2024-03-21T10:24:13.096 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:13.092+0000 7f615d200640 1 -- 172.21.15.47:0/293204674 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6150003830 con 0x55f89e91ed80 2024-03-21T10:24:13.102 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:13.093+0000 7f6171790740 1 -- 172.21.15.47:0/293204674 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f89e91ed80 msgr2=0x55f89e91f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:24:13.102 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:13.093+0000 7f6167e00640 1 --2- 172.21.15.47:0/64977336 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f61480a2fa0 0x7f61480a5460 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x55f89e91fb70 tx=0x7f6158005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:24:13.102 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:13.093+0000 7f6171790740 1 --2- 172.21.15.47:0/293204674 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f89e91ed80 0x55f89e91f1a0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f6150014a30 tx=0x7f615000c570 comp rx=0 tx=0).stop 2024-03-21T10:24:13.102 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:13.093+0000 7f6171790740 1 -- 172.21.15.47:0/293204674 shutdown_connections 2024-03-21T10:24:13.102 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:13.093+0000 7f6171790740 1 --2- 172.21.15.47:0/293204674 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f89e9248e0 0x55f89e926d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:24:13.102 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:13.093+0000 7f6171790740 1 --2- 172.21.15.47:0/293204674 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f89e91f770 0x55f89e924270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:24:13.103 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:13.093+0000 7f6171790740 1 --2- 172.21.15.47:0/293204674 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f89e91ed80 0x55f89e91f1a0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:24:13.103 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:13.093+0000 7f6171790740 1 -- 172.21.15.47:0/293204674 >> 172.21.15.47:0/293204674 conn(0x55f89e91a2f0 msgr2=0x55f89e91a6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:24:13.103 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:13.093+0000 7f6171790740 1 -- 172.21.15.47:0/293204674 shutdown_connections 2024-03-21T10:24:13.103 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:13.093+0000 7f6171790740 1 -- 172.21.15.47:0/293204674 wait complete. 2024-03-21T10:24:13.103 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:13.094+0000 7f6171790740 1 -- start start 2024-03-21T10:24:13.103 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:13.094+0000 7f6171790740 1 Processor -- start 2024-03-21T10:24:13.103 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:13.094+0000 7f6171790740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f89e91ed80 0x55f89e929240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:24:13.103 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:13.094+0000 7f6171790740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f89e91f770 0x55f89e929780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:24:13.103 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:13.094+0000 7f6171790740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f89e9248e0 0x55f89e9307c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:24:13.103 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:13.094+0000 7f6171790740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f89e927830 con 0x55f89e91f770 2024-03-21T10:24:13.103 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:13.094+0000 7f6171790740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f89e83fdd0 con 0x55f89e91ed80 2024-03-21T10:24:13.103 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:13.094+0000 7f6171790740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f89e9279b0 con 0x55f89e9248e0 2024-03-21T10:24:13.103 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:13.094+0000 7f6171790740 10 client.0 _ll_register_callbacks cb 0x55f89e597ac0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T10:24:13.103 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:13.095+0000 7f615f400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f89e91f770 0x55f89e929780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:24:13.104 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:13.095+0000 7f6167e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f89e91ed80 0x55f89e929240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:24:13.104 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:13.095+0000 7f6167e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f89e91ed80 0x55f89e929240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:41346/0 (socket says 172.21.15.47:41346) 2024-03-21T10:24:13.104 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:13.095+0000 7f6167e00640 1 -- 172.21.15.47:0/3459180534 learned_addr learned my addr 172.21.15.47:0/3459180534 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:24:13.104 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:13.095+0000 7f615f400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f89e91f770 0x55f89e929780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:40968/0 (socket says 172.21.15.47:40968) 2024-03-21T10:24:13.104 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:13.095+0000 7f616ca00640 1 --2- 172.21.15.47:0/3459180534 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f89e9248e0 0x55f89e9307c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:24:13.104 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:13.095+0000 7f615f400640 1 -- 172.21.15.47:0/3459180534 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f89e9248e0 msgr2=0x55f89e9307c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:24:13.104 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:13.095+0000 7f615f400640 1 --2- 172.21.15.47:0/3459180534 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f89e9248e0 0x55f89e9307c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:24:13.104 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:13.095+0000 7f615f400640 1 -- 172.21.15.47:0/3459180534 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f89e91ed80 msgr2=0x55f89e929240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:24:13.104 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:13.095+0000 7f615f400640 1 --2- 172.21.15.47:0/3459180534 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f89e91ed80 0x55f89e929240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:24:13.104 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:13.095+0000 7f615f400640 1 -- 172.21.15.47:0/3459180534 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55f89e5982d0 con 0x55f89e91f770 2024-03-21T10:24:13.104 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:13.095+0000 7f616ca00640 1 --2- 172.21.15.47:0/3459180534 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f89e9248e0 0x55f89e9307c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T10:24:13.104 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:13.095+0000 7f6167e00640 1 --2- 172.21.15.47:0/3459180534 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f89e91ed80 0x55f89e929240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T10:24:13.104 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:13.096+0000 7f615f400640 1 --2- 172.21.15.47:0/3459180534 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f89e91f770 0x55f89e929780 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f6150014a30 tx=0x7f6150014380 comp rx=0 tx=0).ready entity=mon.0 client_cookie=77b292f2fad3de9 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:24:13.104 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:13.096+0000 7f6155600640 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-03-21T10:24:13.104 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:13.096+0000 7f6155600640 1 -- 172.21.15.47:0/3459180534 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6150004030 con 0x55f89e91f770 2024-03-21T10:24:13.105 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:13.096+0000 7f6171790740 1 -- 172.21.15.47:0/3459180534 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f615000f820 con 0x55f89e91f770 2024-03-21T10:24:13.105 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:13.096+0000 7f6155600640 1 -- 172.21.15.47:0/3459180534 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9216+0+0 (secure 0 0 0) 0x7f61500041d0 con 0x55f89e91f770 2024-03-21T10:24:13.105 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:13.096+0000 7f6155600640 1 -- 172.21.15.47:0/3459180534 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f61500043d0 con 0x55f89e91f770 2024-03-21T10:24:13.105 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:13.096+0000 7f6171790740 10 client.15252 fetch_fsmap learned FSMap version 29 2024-03-21T10:24:13.105 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:13.096+0000 7f6171790740 1 -- 172.21.15.47:0/3459180534 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x55f89e6626b0 con 0x55f89e91f770 2024-03-21T10:24:13.105 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:13.097+0000 7f6155600640 1 -- 172.21.15.47:0/3459180534 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f615000f820 con 0x55f89e91f770 2024-03-21T10:24:13.105 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:13.097+0000 7f6155600640 1 -- 172.21.15.47:0/3459180534 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 29) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f61500284e0 con 0x55f89e91f770 2024-03-21T10:24:13.105 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:13.097+0000 7f6171790740 10 client.15252 fetch_fsmap finished waiting for FSMap version 29 2024-03-21T10:24:13.105 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:13.097+0000 7f6171790740 10 client.15252 resolve_mds: resolved name 'h' to daemon mds.h 2024-03-21T10:24:13.105 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:13.097+0000 7f6171790740 20 client.15252 populate_metadata read hostname 'smithi047' 2024-03-21T10:24:13.105 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:13.097+0000 7f6171790740 1 --2- 172.21.15.47:0/3459180534 >> [v2:172.21.15.100:6836/3497644162,v1:172.21.15.100:6837/3497644162] conn(0x55f89e935590 0x55f89e939a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:24:13.105 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:13.097+0000 7f6171790740 4 client.15252 mds_command: new command op to 34493 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T10:24:13.105 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:13.097+0000 7f6171790740 1 -- 172.21.15.47:0/3459180534 --> [v2:172.21.15.100:6836/3497644162,v1:172.21.15.100:6837/3497644162] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55f89e825520 con 0x55f89e935590 2024-03-21T10:24:13.105 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:13.098+0000 7f6167e00640 1 --2- 172.21.15.47:0/3459180534 >> [v2:172.21.15.100:6836/3497644162,v1:172.21.15.100:6837/3497644162] conn(0x55f89e935590 0x55f89e939a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:24:13.105 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:13.098+0000 7f6167e00640 1 --2- 172.21.15.47:0/3459180534 >> [v2:172.21.15.100:6836/3497644162,v1:172.21.15.100:6837/3497644162] conn(0x55f89e935590 0x55f89e939a10 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=3bea872754f508ad server_cookie=d05faf5c32682ad2 in_seq=0 out_seq=0 2024-03-21T10:24:13.106 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:13.098+0000 7f6155600640 10 client.15252 ms_handle_connect on v2:172.21.15.100:6836/3497644162 2024-03-21T10:24:13.106 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:13.099+0000 7f6155600640 1 -- 172.21.15.47:0/3459180534 <== mds.1 v2:172.21.15.100:6836/3497644162 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55f89e825520 con 0x55f89e935590 2024-03-21T10:24:13.106 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:13.099+0000 7f6155600640 10 client.15252 handle_command_reply: tid=0 2024-03-21T10:24:13.106 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:13.100+0000 7f6171790740 1 client.15252 shutdown 2024-03-21T10:24:13.106 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:13.100+0000 7f6171790740 1 -- 172.21.15.47:0/3459180534 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f89e91f770 msgr2=0x55f89e929780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:24:13.106 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:13.100+0000 7f6171790740 1 --2- 172.21.15.47:0/3459180534 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f89e91f770 0x55f89e929780 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f6150014a30 tx=0x7f6150014380 comp rx=0 tx=0).stop 2024-03-21T10:24:13.106 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:13.100+0000 7f6171790740 1 -- 172.21.15.47:0/3459180534 shutdown_connections 2024-03-21T10:24:13.106 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:13.100+0000 7f6171790740 1 --2- 172.21.15.47:0/3459180534 >> [v2:172.21.15.100:6836/3497644162,v1:172.21.15.100:6837/3497644162] conn(0x55f89e935590 0x55f89e939a10 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:24:13.106 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:13.100+0000 7f6171790740 1 --2- 172.21.15.47:0/3459180534 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f89e9248e0 0x55f89e9307c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:24:13.106 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:13.100+0000 7f6171790740 1 --2- 172.21.15.47:0/3459180534 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f89e91f770 0x55f89e929780 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:24:13.106 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:13.100+0000 7f6155600640 0 client.15252 ms_handle_reset on v2:172.21.15.100:6836/3497644162 2024-03-21T10:24:13.106 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:13.100+0000 7f6171790740 1 --2- 172.21.15.47:0/3459180534 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f89e91ed80 0x55f89e929240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:24:13.106 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:13.100+0000 7f6171790740 1 -- 172.21.15.47:0/3459180534 >> 172.21.15.47:0/3459180534 conn(0x55f89e91a2f0 msgr2=0x55f89e9194f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:24:13.106 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:13.100+0000 7f6171790740 1 -- 172.21.15.47:0/3459180534 shutdown_connections 2024-03-21T10:24:13.107 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:13.100+0000 7f6171790740 1 -- 172.21.15.47:0/3459180534 wait complete. 2024-03-21T10:24:13.107 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:13.100+0000 7f6171790740 20 client.15252 trim_cache size 0 max 16384 2024-03-21T10:24:13.122 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:13.118+0000 7f6171790740 1 Processor -- start 2024-03-21T10:24:13.123 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:13.118+0000 7f6171790740 1 -- start start 2024-03-21T10:24:13.123 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:13.119+0000 7f6171790740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f89e91ed80 0x55f89e927d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:24:13.123 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:13.119+0000 7f6171790740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f89e91f770 0x55f89e930780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:24:13.123 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:13.119+0000 7f6171790740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f89e9248e0 0x55f89e9201e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:24:13.123 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:13.119+0000 7f6171790740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f89e934db0 con 0x55f89e91ed80 2024-03-21T10:24:13.123 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:13.119+0000 7f6171790740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f89e935410 con 0x55f89e9248e0 2024-03-21T10:24:13.123 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:13.119+0000 7f6171790740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f89e935a70 con 0x55f89e91f770 2024-03-21T10:24:13.123 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:13.119+0000 7f6167e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f89e91ed80 0x55f89e927d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:24:13.123 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:13.119+0000 7f6167e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f89e91ed80 0x55f89e927d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:40970/0 (socket says 172.21.15.47:40970) 2024-03-21T10:24:13.124 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:13.119+0000 7f6167e00640 1 -- 172.21.15.47:0/1410985071 learned_addr learned my addr 172.21.15.47:0/1410985071 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:24:13.124 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:13.119+0000 7f616ca00640 1 --2- 172.21.15.47:0/1410985071 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f89e9248e0 0x55f89e9201e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:24:13.124 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:13.119+0000 7f6167e00640 1 -- 172.21.15.47:0/1410985071 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f89e91f770 msgr2=0x55f89e930780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:24:13.124 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:13.119+0000 7f615f400640 1 --2- 172.21.15.47:0/1410985071 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f89e91f770 0x55f89e930780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:24:13.124 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:13.119+0000 7f6167e00640 1 --2- 172.21.15.47:0/1410985071 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f89e91f770 0x55f89e930780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:24:13.124 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:13.119+0000 7f6167e00640 1 -- 172.21.15.47:0/1410985071 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f89e9248e0 msgr2=0x55f89e9201e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:24:13.124 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:13.119+0000 7f6167e00640 1 --2- 172.21.15.47:0/1410985071 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f89e9248e0 0x55f89e9201e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:24:13.124 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:13.119+0000 7f6167e00640 1 -- 172.21.15.47:0/1410985071 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f89e90c170 con 0x55f89e91ed80 2024-03-21T10:24:13.124 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:13.120+0000 7f6167e00640 1 --2- 172.21.15.47:0/1410985071 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f89e91ed80 0x55f89e927d80 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x55f89e936f80 tx=0x7f6158016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6c6397994b5ed4a6 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:24:13.124 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:13.120+0000 7f6157e00640 1 -- 172.21.15.47:0/1410985071 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f615802e070 con 0x55f89e91ed80 2024-03-21T10:24:13.124 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:13.120+0000 7f6157e00640 1 -- 172.21.15.47:0/1410985071 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f6158011e90 con 0x55f89e91ed80 2024-03-21T10:24:13.124 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:13.120+0000 7f6157e00640 1 -- 172.21.15.47:0/1410985071 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6158029910 con 0x55f89e91ed80 2024-03-21T10:24:13.128 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:13.121+0000 7f6171790740 1 -- 172.21.15.47:0/1410985071 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f89e91ed80 msgr2=0x55f89e927d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:24:13.128 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:13.121+0000 7f6171790740 1 --2- 172.21.15.47:0/1410985071 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f89e91ed80 0x55f89e927d80 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x55f89e936f80 tx=0x7f6158016000 comp rx=0 tx=0).stop 2024-03-21T10:24:13.128 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:13.121+0000 7f6171790740 1 -- 172.21.15.47:0/1410985071 shutdown_connections 2024-03-21T10:24:13.128 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:13.121+0000 7f6171790740 1 --2- 172.21.15.47:0/1410985071 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f89e9248e0 0x55f89e9201e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:24:13.128 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:13.121+0000 7f6171790740 1 --2- 172.21.15.47:0/1410985071 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f89e91f770 0x55f89e930780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:24:13.128 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:13.121+0000 7f6171790740 1 --2- 172.21.15.47:0/1410985071 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f89e91ed80 0x55f89e927d80 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:24:13.128 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:13.121+0000 7f6171790740 1 -- 172.21.15.47:0/1410985071 >> 172.21.15.47:0/1410985071 conn(0x55f89e91a2f0 msgr2=0x55f89e926cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:24:13.128 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:13.121+0000 7f6171790740 1 -- 172.21.15.47:0/1410985071 shutdown_connections 2024-03-21T10:24:13.128 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:13.121+0000 7f6171790740 1 -- 172.21.15.47:0/1410985071 wait complete. 2024-03-21T10:24:13.129 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:13.122+0000 7f6171790740 1 -- start start 2024-03-21T10:24:13.129 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:13.122+0000 7f6171790740 1 Processor -- start 2024-03-21T10:24:13.129 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:13.122+0000 7f6171790740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f89e91ed80 0x55f89e92c9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:24:13.129 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:13.123+0000 7f6171790740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f89e91f770 0x55f89e92cef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:24:13.129 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:13.123+0000 7f6171790740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f89e9248e0 0x55f89e92fbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:24:13.129 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:13.123+0000 7f6171790740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f89e936730 con 0x55f89e91ed80 2024-03-21T10:24:13.129 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:13.123+0000 7f6171790740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f89e9360d0 con 0x55f89e9248e0 2024-03-21T10:24:13.129 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:13.123+0000 7f6171790740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f89e936d90 con 0x55f89e91f770 2024-03-21T10:24:13.130 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:13.123+0000 7f6171790740 10 client.0 _ll_register_callbacks cb 0x7f6150014470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T10:24:13.130 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:13.123+0000 7f6167e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f89e91ed80 0x55f89e92c9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:24:13.130 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:13.123+0000 7f6167e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f89e91ed80 0x55f89e92c9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:40980/0 (socket says 172.21.15.47:40980) 2024-03-21T10:24:13.130 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:13.123+0000 7f6167e00640 1 -- 172.21.15.47:0/922841408 learned_addr learned my addr 172.21.15.47:0/922841408 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:24:13.130 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:13.123+0000 7f616ca00640 1 --2- 172.21.15.47:0/922841408 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f89e9248e0 0x55f89e92fbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:24:13.130 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:13.123+0000 7f615f400640 1 --2- 172.21.15.47:0/922841408 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f89e91f770 0x55f89e92cef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:24:13.130 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:13.123+0000 7f6167e00640 1 -- 172.21.15.47:0/922841408 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f89e91f770 msgr2=0x55f89e92cef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:24:13.130 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:13.123+0000 7f6167e00640 1 --2- 172.21.15.47:0/922841408 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f89e91f770 0x55f89e92cef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:24:13.130 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:13.123+0000 7f6167e00640 1 -- 172.21.15.47:0/922841408 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f89e9248e0 msgr2=0x55f89e92fbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:24:13.130 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:13.123+0000 7f6167e00640 1 --2- 172.21.15.47:0/922841408 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f89e9248e0 0x55f89e92fbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:24:13.131 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:13.123+0000 7f6167e00640 1 -- 172.21.15.47:0/922841408 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55f89e930220 con 0x55f89e91ed80 2024-03-21T10:24:13.131 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:13.124+0000 7f6167e00640 1 --2- 172.21.15.47:0/922841408 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f89e91ed80 0x55f89e92c9b0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f6158029370 tx=0x7f61580293a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2449739313604a03 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:24:13.131 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:13.124+0000 7f6155600640 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-03-21T10:24:13.131 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:13.124+0000 7f6155600640 1 -- 172.21.15.47:0/922841408 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6158036070 con 0x55f89e91ed80 2024-03-21T10:24:13.131 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:13.124+0000 7f6171790740 1 -- 172.21.15.47:0/922841408 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f6158011e90 con 0x55f89e91ed80 2024-03-21T10:24:13.131 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:13.124+0000 7f6155600640 1 -- 172.21.15.47:0/922841408 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9216+0+0 (secure 0 0 0) 0x7f61580054c0 con 0x55f89e91ed80 2024-03-21T10:24:13.131 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:13.124+0000 7f6155600640 1 -- 172.21.15.47:0/922841408 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6158041050 con 0x55f89e91ed80 2024-03-21T10:24:13.131 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:13.124+0000 7f6171790740 10 client.15258 fetch_fsmap learned FSMap version 29 2024-03-21T10:24:13.131 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:13.124+0000 7f6171790740 1 -- 172.21.15.47:0/922841408 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x55f89e999b60 con 0x55f89e91ed80 2024-03-21T10:24:13.131 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:13.125+0000 7f6155600640 1 -- 172.21.15.47:0/922841408 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f6158011e90 con 0x55f89e91ed80 2024-03-21T10:24:13.131 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:13.125+0000 7f6155600640 1 -- 172.21.15.47:0/922841408 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 29) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f6158004a50 con 0x55f89e91ed80 2024-03-21T10:24:13.131 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:13.125+0000 7f6171790740 10 client.15258 fetch_fsmap finished waiting for FSMap version 29 2024-03-21T10:24:13.132 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:13.125+0000 7f6171790740 10 client.15258 resolve_mds: resolved name 'h' to daemon mds.h 2024-03-21T10:24:13.132 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:13.125+0000 7f6171790740 20 client.15258 populate_metadata read hostname 'smithi047' 2024-03-21T10:24:13.132 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:13.125+0000 7f6171790740 1 --2- 172.21.15.47:0/922841408 >> [v2:172.21.15.100:6836/3497644162,v1:172.21.15.100:6837/3497644162] conn(0x55f89e99de40 0x55f89e999d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:24:13.132 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:13.125+0000 7f6171790740 4 client.15258 mds_command: new command op to 34493 tid=0 multi_id=0 [{"prefix": "status"}] 2024-03-21T10:24:13.132 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:13.125+0000 7f6171790740 1 -- 172.21.15.47:0/922841408 --> [v2:172.21.15.100:6836/3497644162,v1:172.21.15.100:6837/3497644162] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55f89e8911d0 con 0x55f89e99de40 2024-03-21T10:24:13.132 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:13.126+0000 7f615f400640 1 --2- 172.21.15.47:0/922841408 >> [v2:172.21.15.100:6836/3497644162,v1:172.21.15.100:6837/3497644162] conn(0x55f89e99de40 0x55f89e999d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:24:13.134 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:13.126+0000 7f615f400640 1 --2- 172.21.15.47:0/922841408 >> [v2:172.21.15.100:6836/3497644162,v1:172.21.15.100:6837/3497644162] conn(0x55f89e99de40 0x55f89e999d30 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=5dad6e5353980553 server_cookie=86ba9aecef3a6540 in_seq=0 out_seq=0 2024-03-21T10:24:13.134 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:13.126+0000 7f6155600640 10 client.15258 ms_handle_connect on v2:172.21.15.100:6836/3497644162 2024-03-21T10:24:13.134 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:13.127+0000 7f6155600640 1 -- 172.21.15.47:0/922841408 <== mds.1 v2:172.21.15.100:6836/3497644162 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+332 (crc 0 0 0) 0x55f89e8911d0 con 0x55f89e99de40 2024-03-21T10:24:13.134 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:13.127+0000 7f6155600640 10 client.15258 handle_command_reply: tid=0 2024-03-21T10:24:13.134 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:13.127+0000 7f6171790740 1 client.15258 shutdown 2024-03-21T10:24:13.134 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:13.127+0000 7f6171790740 1 -- 172.21.15.47:0/922841408 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f89e91ed80 msgr2=0x55f89e92c9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:24:13.134 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:13.127+0000 7f6171790740 1 --2- 172.21.15.47:0/922841408 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f89e91ed80 0x55f89e92c9b0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f6158029370 tx=0x7f61580293a0 comp rx=0 tx=0).stop 2024-03-21T10:24:13.134 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:13.127+0000 7f6171790740 1 -- 172.21.15.47:0/922841408 shutdown_connections 2024-03-21T10:24:13.134 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:13.127+0000 7f6171790740 1 --2- 172.21.15.47:0/922841408 >> [v2:172.21.15.100:6836/3497644162,v1:172.21.15.100:6837/3497644162] conn(0x55f89e99de40 0x55f89e999d30 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:24:13.134 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:13.127+0000 7f6171790740 1 --2- 172.21.15.47:0/922841408 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f89e9248e0 0x55f89e92fbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:24:13.134 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:13.127+0000 7f6171790740 1 --2- 172.21.15.47:0/922841408 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f89e91f770 0x55f89e92cef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:24:13.134 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:13.127+0000 7f6171790740 1 --2- 172.21.15.47:0/922841408 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f89e91ed80 0x55f89e92c9b0 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:24:13.135 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:13.127+0000 7f6171790740 1 -- 172.21.15.47:0/922841408 >> 172.21.15.47:0/922841408 conn(0x55f89e91a2f0 msgr2=0x55f89e927d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:24:13.135 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:13.127+0000 7f6155600640 0 client.15258 ms_handle_reset on v2:172.21.15.100:6836/3497644162 2024-03-21T10:24:13.135 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:13.127+0000 7f6171790740 1 -- 172.21.15.47:0/922841408 shutdown_connections 2024-03-21T10:24:13.135 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:13.127+0000 7f6171790740 1 -- 172.21.15.47:0/922841408 wait complete. 2024-03-21T10:24:13.135 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:13.127+0000 7f6171790740 20 client.15258 trim_cache size 0 max 16384 2024-03-21T10:24:13.135 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-03-21T10:24:13.135 INFO:teuthology.orchestra.run.smithi047.stdout: "cluster_fsid": "3056d51a-e76b-11ee-95cd-87774f69a715", 2024-03-21T10:24:13.135 INFO:teuthology.orchestra.run.smithi047.stdout: "whoami": 1, 2024-03-21T10:24:13.135 INFO:teuthology.orchestra.run.smithi047.stdout: "id": 34493, 2024-03-21T10:24:13.136 INFO:teuthology.orchestra.run.smithi047.stdout: "want_state": "up:active", 2024-03-21T10:24:13.136 INFO:teuthology.orchestra.run.smithi047.stdout: "state": "up:active", 2024-03-21T10:24:13.136 INFO:teuthology.orchestra.run.smithi047.stdout: "fs_name": "cephfs", 2024-03-21T10:24:13.136 INFO:teuthology.orchestra.run.smithi047.stdout: "rank_uptime": 107.95780639100001, 2024-03-21T10:24:13.136 INFO:teuthology.orchestra.run.smithi047.stdout: "mdsmap_epoch": 29, 2024-03-21T10:24:13.136 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap_epoch": 95, 2024-03-21T10:24:13.136 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap_epoch_barrier": 93, 2024-03-21T10:24:13.136 INFO:teuthology.orchestra.run.smithi047.stdout: "uptime": 155.54860370700001 2024-03-21T10:24:13.136 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-03-21T10:24:13.137 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:13.128+0000 7f616d400640 1 -- 172.21.15.47:0/64977336 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f61480a2fa0 msgr2=0x7f61480a5460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:24:13.137 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:13.128+0000 7f616d400640 1 --2- 172.21.15.47:0/64977336 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f61480a2fa0 0x7f61480a5460 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x55f89e91fb70 tx=0x7f6158005f90 comp rx=0 tx=0).stop 2024-03-21T10:24:13.137 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:13.128+0000 7f616d400640 1 -- 172.21.15.47:0/64977336 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6168103150 msgr2=0x7f61681a0f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:24:13.137 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:13.128+0000 7f616d400640 1 --2- 172.21.15.47:0/64977336 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6168103150 0x7f61681a0f70 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f6160032550 tx=0x7f6160032820 comp rx=0 tx=0).stop 2024-03-21T10:24:13.137 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:13.128+0000 7f616d400640 1 -- 172.21.15.47:0/64977336 shutdown_connections 2024-03-21T10:24:13.137 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:13.128+0000 7f616d400640 1 --2- 172.21.15.47:0/64977336 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f61480a2fa0 0x7f61480a5460 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:24:13.137 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:13.128+0000 7f616d400640 1 --2- 172.21.15.47:0/64977336 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6168103150 0x7f61681a0f70 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:24:13.137 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:13.128+0000 7f616d400640 1 --2- 172.21.15.47:0/64977336 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6168102790 0x7f6168199f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:24:13.137 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:13.128+0000 7f616d400640 1 --2- 172.21.15.47:0/64977336 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f61681015a0 0x7f61681999c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:24:13.137 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:13.128+0000 7f616d400640 1 -- 172.21.15.47:0/64977336 >> 172.21.15.47:0/64977336 conn(0x7f61680fcb10 msgr2=0x7f61680738d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:24:13.138 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:13.128+0000 7f616d400640 1 -- 172.21.15.47:0/64977336 shutdown_connections 2024-03-21T10:24:13.138 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:13.128+0000 7f616d400640 1 -- 172.21.15.47:0/64977336 wait complete. 2024-03-21T10:24:13.151 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:24:12 smithi047 ceph-mon[31547]: from='client.? 172.21.15.47:0/869124167' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-21T10:24:13.154 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2024-03-21T10:24:13.229 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:24:12 smithi100 ceph-mon[35193]: from='client.? 172.21.15.47:0/869124167' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-21T10:24:13.240 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:24:12 smithi157 ceph-mon[35581]: from='client.? 172.21.15.47:0/869124167' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-21T10:24:13.367 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:13.364+0000 7f901ae00640 1 -- 172.21.15.47:0/1144673237 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9014102030 msgr2=0x7f90141088c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:24:13.367 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:13.364+0000 7f901ae00640 1 --2- 172.21.15.47:0/1144673237 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9014102030 0x7f90141088c0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f900c009ca0 tx=0x7f900c02c100 comp rx=0 tx=0).stop 2024-03-21T10:24:13.367 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:13.364+0000 7f901ae00640 1 -- 172.21.15.47:0/1144673237 shutdown_connections 2024-03-21T10:24:13.367 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:13.364+0000 7f901ae00640 1 --2- 172.21.15.47:0/1144673237 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9014102030 0x7f90141088c0 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:24:13.367 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:13.364+0000 7f901ae00640 1 --2- 172.21.15.47:0/1144673237 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9014101670 0x7f9014101af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:24:13.367 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:13.364+0000 7f901ae00640 1 --2- 172.21.15.47:0/1144673237 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9014100480 0x7f90141008a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:24:13.367 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:13.364+0000 7f901ae00640 1 -- 172.21.15.47:0/1144673237 >> 172.21.15.47:0/1144673237 conn(0x7f90140fba10 msgr2=0x7f90140fde30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:24:13.367 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:13.364+0000 7f901ae00640 1 -- 172.21.15.47:0/1144673237 shutdown_connections 2024-03-21T10:24:13.367 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:13.365+0000 7f901ae00640 1 -- 172.21.15.47:0/1144673237 wait complete. 2024-03-21T10:24:13.369 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:13.366+0000 7f901ae00640 1 Processor -- start 2024-03-21T10:24:13.369 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:13.366+0000 7f901ae00640 1 -- start start 2024-03-21T10:24:13.369 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:13.366+0000 7f901ae00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9014100480 0x7f9014199970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:24:13.369 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:13.366+0000 7f901ae00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9014101670 0x7f9014199eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:24:13.369 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:13.366+0000 7f901ae00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9014102030 0x7f90141a0f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:24:13.369 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:13.366+0000 7f901ae00640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f901410d0d0 con 0x7f9014101670 2024-03-21T10:24:13.369 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:13.366+0000 7f901ae00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f901410cf50 con 0x7f9014100480 2024-03-21T10:24:13.369 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:13.366+0000 7f901ae00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f901410d250 con 0x7f9014102030 2024-03-21T10:24:13.369 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:13.366+0000 7f9019000640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9014101670 0x7f9014199eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:24:13.369 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:13.367+0000 7f9019000640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9014101670 0x7f9014199eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:40988/0 (socket says 172.21.15.47:40988) 2024-03-21T10:24:13.369 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:13.367+0000 7f9019000640 1 -- 172.21.15.47:0/2770599993 learned_addr learned my addr 172.21.15.47:0/2770599993 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:24:13.369 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:13.367+0000 7f9019a00640 1 --2- 172.21.15.47:0/2770599993 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9014100480 0x7f9014199970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:24:13.370 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:13.367+0000 7f901a400640 1 --2- 172.21.15.47:0/2770599993 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9014102030 0x7f90141a0f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:24:13.370 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:13.367+0000 7f9019000640 1 -- 172.21.15.47:0/2770599993 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9014102030 msgr2=0x7f90141a0f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:24:13.371 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:13.367+0000 7f9019000640 1 --2- 172.21.15.47:0/2770599993 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9014102030 0x7f90141a0f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:24:13.371 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:13.367+0000 7f9019000640 1 -- 172.21.15.47:0/2770599993 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9014100480 msgr2=0x7f9014199970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:24:13.371 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:13.367+0000 7f9019000640 1 --2- 172.21.15.47:0/2770599993 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9014100480 0x7f9014199970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:24:13.371 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:13.367+0000 7f9019000640 1 -- 172.21.15.47:0/2770599993 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f90141a1590 con 0x7f9014101670 2024-03-21T10:24:13.371 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:13.368+0000 7f9019000640 1 --2- 172.21.15.47:0/2770599993 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9014101670 0x7f9014199eb0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f900000d950 tx=0x7f900000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:24:13.371 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:13.368+0000 7f9012000640 1 -- 172.21.15.47:0/2770599993 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9000017070 con 0x7f9014101670 2024-03-21T10:24:13.371 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:13.368+0000 7f9012000640 1 -- 172.21.15.47:0/2770599993 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f9000009c30 con 0x7f9014101670 2024-03-21T10:24:13.371 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:13.368+0000 7f9012000640 1 -- 172.21.15.47:0/2770599993 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f900000f620 con 0x7f9014101670 2024-03-21T10:24:13.371 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:13.368+0000 7f901ae00640 1 -- 172.21.15.47:0/2770599993 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f90141a1880 con 0x7f9014101670 2024-03-21T10:24:13.371 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:13.368+0000 7f901ae00640 1 -- 172.21.15.47:0/2770599993 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f90141a1d80 con 0x7f9014101670 2024-03-21T10:24:13.373 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:13.371+0000 7f901f21e740 1 Processor -- start 2024-03-21T10:24:13.373 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:13.371+0000 7f901f21e740 1 -- start start 2024-03-21T10:24:13.374 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:13.372+0000 7f9012000640 1 -- 172.21.15.47:0/2770599993 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 18) v1 ==== 137854+0+0 (secure 0 0 0) 0x7f90000109a0 con 0x7f9014101670 2024-03-21T10:24:13.374 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:13.372+0000 7f901f21e740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55664f6aad80 0x55664f6ab1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:24:13.374 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:13.372+0000 7f901f21e740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55664f6ab770 0x55664f6b0270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:24:13.374 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:13.372+0000 7f901f21e740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55664f6b08e0 0x55664f6b2d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:24:13.375 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:13.372+0000 7f901f21e740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x55664f60fc90 con 0x55664f6aad80 2024-03-21T10:24:13.375 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:13.372+0000 7f901f21e740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55664f5b8980 con 0x55664f6ab770 2024-03-21T10:24:13.375 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:13.372+0000 7f901f21e740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55664f60bd30 con 0x55664f6b08e0 2024-03-21T10:24:13.375 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:13.372+0000 7f901a400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55664f6aad80 0x55664f6ab1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:24:13.375 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:13.372+0000 7f901a400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55664f6aad80 0x55664f6ab1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:41004/0 (socket says 172.21.15.47:41004) 2024-03-21T10:24:13.375 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:13.372+0000 7f901a400640 1 -- 172.21.15.47:0/3033618880 learned_addr learned my addr 172.21.15.47:0/3033618880 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:24:13.375 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:13.372+0000 7f9019a00640 1 --2- 172.21.15.47:0/3033618880 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55664f6ab770 0x55664f6b0270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:24:13.375 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:13.372+0000 7f9019000640 1 --2- 172.21.15.47:0/3033618880 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55664f6b08e0 0x55664f6b2d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:24:13.377 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:13.373+0000 7f901a400640 1 -- 172.21.15.47:0/3033618880 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55664f6b08e0 msgr2=0x55664f6b2d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:24:13.377 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:13.373+0000 7f901a400640 1 --2- 172.21.15.47:0/3033618880 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55664f6b08e0 0x55664f6b2d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:24:13.377 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:13.373+0000 7f901a400640 1 -- 172.21.15.47:0/3033618880 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55664f6ab770 msgr2=0x55664f6b0270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:24:13.377 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:13.373+0000 7f901a400640 1 --2- 172.21.15.47:0/3033618880 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55664f6ab770 0x55664f6b0270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:24:13.377 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:13.373+0000 7f901a400640 1 -- 172.21.15.47:0/3033618880 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55664f5d0510 con 0x55664f6aad80 2024-03-21T10:24:13.377 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:13.373+0000 7f901a400640 1 --2- 172.21.15.47:0/3033618880 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55664f6aad80 0x55664f6ab1a0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f900c005e30 tx=0x7f900c005f80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cb5381988177e0a2 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:24:13.377 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:13.373+0000 7f9012000640 1 --2- 172.21.15.47:0/2770599993 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f8ffc0a2fa0 0x7f8ffc0a5460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:24:13.377 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:13.374+0000 7f9006a00640 1 -- 172.21.15.47:0/3033618880 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f900c043020 con 0x55664f6aad80 2024-03-21T10:24:13.377 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:13.374+0000 7f9006a00640 1 -- 172.21.15.47:0/3033618880 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f900c032dc0 con 0x55664f6aad80 2024-03-21T10:24:13.377 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:13.374+0000 7f9006a00640 1 -- 172.21.15.47:0/3033618880 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f900c039040 con 0x55664f6aad80 2024-03-21T10:24:13.377 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:13.374+0000 7f9012000640 1 -- 172.21.15.47:0/2770599993 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9216+0+0 (secure 0 0 0) 0x7f90000cf3e0 con 0x7f9014101670 2024-03-21T10:24:13.378 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:13.374+0000 7f9019a00640 1 --2- 172.21.15.47:0/2770599993 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f8ffc0a2fa0 0x7f8ffc0a5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:24:13.378 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:13.374+0000 7f901f21e740 1 -- 172.21.15.47:0/3033618880 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55664f6aad80 msgr2=0x55664f6ab1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:24:13.378 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:13.374+0000 7f901f21e740 1 --2- 172.21.15.47:0/3033618880 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55664f6aad80 0x55664f6ab1a0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f900c005e30 tx=0x7f900c005f80 comp rx=0 tx=0).stop 2024-03-21T10:24:13.378 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:13.375+0000 7f901f21e740 1 -- 172.21.15.47:0/3033618880 shutdown_connections 2024-03-21T10:24:13.378 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:13.375+0000 7f901f21e740 1 --2- 172.21.15.47:0/3033618880 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55664f6b08e0 0x55664f6b2d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:24:13.378 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:13.375+0000 7f901f21e740 1 --2- 172.21.15.47:0/3033618880 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55664f6ab770 0x55664f6b0270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:24:13.378 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:13.375+0000 7f901f21e740 1 --2- 172.21.15.47:0/3033618880 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55664f6aad80 0x55664f6ab1a0 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:24:13.378 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:13.375+0000 7f901f21e740 1 -- 172.21.15.47:0/3033618880 >> 172.21.15.47:0/3033618880 conn(0x55664f6a62f0 msgr2=0x55664f6a66f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:24:13.378 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:13.375+0000 7f9019a00640 1 --2- 172.21.15.47:0/2770599993 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f8ffc0a2fa0 0x7f8ffc0a5460 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x55664f6abb70 tx=0x7f9008005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:24:13.378 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:13.375+0000 7f901f21e740 1 -- 172.21.15.47:0/3033618880 shutdown_connections 2024-03-21T10:24:13.378 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:13.375+0000 7f901f21e740 1 -- 172.21.15.47:0/3033618880 wait complete. 2024-03-21T10:24:13.381 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:13.375+0000 7f901f21e740 1 -- start start 2024-03-21T10:24:13.381 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:13.376+0000 7f901f21e740 1 Processor -- start 2024-03-21T10:24:13.381 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:13.376+0000 7f901f21e740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55664f6aad80 0x55664f6b5280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:24:13.381 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:13.376+0000 7f901f21e740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55664f6ab770 0x55664f6b57c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:24:13.381 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:13.376+0000 7f901f21e740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55664f6b08e0 0x55664f6bc800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:24:13.382 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:13.376+0000 7f901f21e740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x55664f6b3830 con 0x55664f6ab770 2024-03-21T10:24:13.382 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:13.376+0000 7f901f21e740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55664f5cbdd0 con 0x55664f6b08e0 2024-03-21T10:24:13.382 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:13.376+0000 7f901f21e740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55664f6b39b0 con 0x55664f6aad80 2024-03-21T10:24:13.382 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:13.377+0000 7f901f21e740 10 client.0 _ll_register_callbacks cb 0x55664f323ac0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T10:24:13.382 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:13.377+0000 7f9019000640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55664f6ab770 0x55664f6b57c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:24:13.382 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:13.377+0000 7f9019000640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55664f6ab770 0x55664f6b57c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:41020/0 (socket says 172.21.15.47:41020) 2024-03-21T10:24:13.382 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:13.377+0000 7f9019000640 1 -- 172.21.15.47:0/1467338264 learned_addr learned my addr 172.21.15.47:0/1467338264 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:24:13.382 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:13.377+0000 7f901a400640 1 --2- 172.21.15.47:0/1467338264 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55664f6b08e0 0x55664f6bc800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:24:13.382 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:13.377+0000 7f9019a00640 1 --2- 172.21.15.47:0/1467338264 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55664f6aad80 0x55664f6b5280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:24:13.382 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:13.377+0000 7f9019000640 1 -- 172.21.15.47:0/1467338264 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55664f6aad80 msgr2=0x55664f6b5280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:24:13.382 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:13.377+0000 7f9019000640 1 --2- 172.21.15.47:0/1467338264 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55664f6aad80 0x55664f6b5280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:24:13.382 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:13.377+0000 7f9019000640 1 -- 172.21.15.47:0/1467338264 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55664f6b08e0 msgr2=0x55664f6bc800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:24:13.382 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:13.377+0000 7f9019000640 1 --2- 172.21.15.47:0/1467338264 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55664f6b08e0 0x55664f6bc800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:24:13.383 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:13.377+0000 7f9019000640 1 -- 172.21.15.47:0/1467338264 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55664f3242d0 con 0x55664f6ab770 2024-03-21T10:24:13.383 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:13.378+0000 7f9019000640 1 --2- 172.21.15.47:0/1467338264 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55664f6ab770 0x55664f6b57c0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f9000098110 tx=0x7f90000973a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5faf3271cf7653b3 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:24:13.383 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:13.378+0000 7f8ffb400640 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-03-21T10:24:13.383 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:13.378+0000 7f8ffb400640 1 -- 172.21.15.47:0/1467338264 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9000020030 con 0x55664f6ab770 2024-03-21T10:24:13.383 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:13.378+0000 7f901f21e740 1 -- 172.21.15.47:0/1467338264 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f900c032dc0 con 0x55664f6ab770 2024-03-21T10:24:13.383 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:13.378+0000 7f8ffb400640 1 -- 172.21.15.47:0/1467338264 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9216+0+0 (secure 0 0 0) 0x7f90000ccb90 con 0x55664f6ab770 2024-03-21T10:24:13.383 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:13.378+0000 7f8ffb400640 1 -- 172.21.15.47:0/1467338264 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f900009b0a0 con 0x55664f6ab770 2024-03-21T10:24:13.383 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:13.378+0000 7f901f21e740 10 client.15270 fetch_fsmap learned FSMap version 29 2024-03-21T10:24:13.383 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:13.378+0000 7f901f21e740 1 -- 172.21.15.47:0/1467338264 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x55664f3ee6b0 con 0x55664f6ab770 2024-03-21T10:24:13.383 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:13.378+0000 7f8ffb400640 1 -- 172.21.15.47:0/1467338264 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f900c032dc0 con 0x55664f6ab770 2024-03-21T10:24:13.383 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:13.379+0000 7f8ffb400640 1 -- 172.21.15.47:0/1467338264 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 29) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f9000097db0 con 0x55664f6ab770 2024-03-21T10:24:13.383 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:13.379+0000 7f901f21e740 10 client.15270 fetch_fsmap finished waiting for FSMap version 29 2024-03-21T10:24:13.383 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:13.379+0000 7f901f21e740 10 client.15270 resolve_mds: resolved name 'b' to daemon mds.b 2024-03-21T10:24:13.383 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:13.379+0000 7f901f21e740 20 client.15270 populate_metadata read hostname 'smithi047' 2024-03-21T10:24:13.383 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:13.379+0000 7f901f21e740 1 --2- 172.21.15.47:0/1467338264 >> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] conn(0x55664f6c1a50 0x55664f6c0b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:24:13.384 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:13.379+0000 7f901f21e740 4 client.15270 mds_command: new command op to 34535 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T10:24:13.384 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:13.379+0000 7f901f21e740 1 -- 172.21.15.47:0/1467338264 --> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55664f5b1520 con 0x55664f6c1a50 2024-03-21T10:24:13.384 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:13.379+0000 7f9019a00640 1 --2- 172.21.15.47:0/1467338264 >> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] conn(0x55664f6c1a50 0x55664f6c0b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:24:13.385 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:13.380+0000 7f9019a00640 1 --2- 172.21.15.47:0/1467338264 >> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] conn(0x55664f6c1a50 0x55664f6c0b40 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4fe70365cf85860b server_cookie=426aaadd4845c59b in_seq=0 out_seq=0 2024-03-21T10:24:13.385 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:13.380+0000 7f8ffb400640 10 client.15270 ms_handle_connect on v2:172.21.15.100:6838/4173168440 2024-03-21T10:24:13.385 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:13.382+0000 7f8ffb400640 1 -- 172.21.15.47:0/1467338264 <== mds.0 v2:172.21.15.100:6838/4173168440 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55664f5b1520 con 0x55664f6c1a50 2024-03-21T10:24:13.385 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:13.382+0000 7f8ffb400640 10 client.15270 handle_command_reply: tid=0 2024-03-21T10:24:13.385 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:13.382+0000 7f901f21e740 1 client.15270 shutdown 2024-03-21T10:24:13.385 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:13.382+0000 7f901f21e740 1 -- 172.21.15.47:0/1467338264 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55664f6ab770 msgr2=0x55664f6b57c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:24:13.385 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:13.382+0000 7f901f21e740 1 --2- 172.21.15.47:0/1467338264 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55664f6ab770 0x55664f6b57c0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f9000098110 tx=0x7f90000973a0 comp rx=0 tx=0).stop 2024-03-21T10:24:13.385 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:13.382+0000 7f901f21e740 1 -- 172.21.15.47:0/1467338264 shutdown_connections 2024-03-21T10:24:13.385 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:13.382+0000 7f901f21e740 1 --2- 172.21.15.47:0/1467338264 >> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] conn(0x55664f6c1a50 0x55664f6c0b40 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:24:13.385 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:13.382+0000 7f901f21e740 1 --2- 172.21.15.47:0/1467338264 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55664f6b08e0 0x55664f6bc800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:24:13.385 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:13.382+0000 7f901f21e740 1 --2- 172.21.15.47:0/1467338264 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55664f6ab770 0x55664f6b57c0 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:24:13.386 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:13.382+0000 7f901f21e740 1 --2- 172.21.15.47:0/1467338264 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55664f6aad80 0x55664f6b5280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:24:13.386 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:13.382+0000 7f8ffb400640 0 client.15270 ms_handle_reset on v2:172.21.15.100:6838/4173168440 2024-03-21T10:24:13.386 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:13.382+0000 7f901f21e740 1 -- 172.21.15.47:0/1467338264 >> 172.21.15.47:0/1467338264 conn(0x55664f6a62f0 msgr2=0x55664f6a54f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:24:13.386 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:13.382+0000 7f901f21e740 1 -- 172.21.15.47:0/1467338264 shutdown_connections 2024-03-21T10:24:13.386 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:13.382+0000 7f901f21e740 1 -- 172.21.15.47:0/1467338264 wait complete. 2024-03-21T10:24:13.386 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:13.383+0000 7f901f21e740 20 client.15270 trim_cache size 0 max 16384 2024-03-21T10:24:13.405 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:13.401+0000 7f901f21e740 1 Processor -- start 2024-03-21T10:24:13.405 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:13.401+0000 7f901f21e740 1 -- start start 2024-03-21T10:24:13.405 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:13.401+0000 7f901f21e740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55664f6aad80 0x55664f6ac210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:24:13.406 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:13.401+0000 7f901f21e740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55664f6ab770 0x55664f6a66d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:24:13.406 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:13.401+0000 7f901f21e740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55664f6b08e0 0x55664f6a4c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:24:13.406 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:13.402+0000 7f901f21e740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x55664f6c1270 con 0x55664f6ab770 2024-03-21T10:24:13.406 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:13.402+0000 7f901f21e740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55664f6c18d0 con 0x55664f6b08e0 2024-03-21T10:24:13.406 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:13.402+0000 7f901f21e740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55664f6c1f30 con 0x55664f6aad80 2024-03-21T10:24:13.406 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:13.402+0000 7f9019000640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55664f6ab770 0x55664f6a66d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:24:13.406 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:13.402+0000 7f9019000640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55664f6ab770 0x55664f6a66d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:41022/0 (socket says 172.21.15.47:41022) 2024-03-21T10:24:13.406 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:13.402+0000 7f9019000640 1 -- 172.21.15.47:0/3016658101 learned_addr learned my addr 172.21.15.47:0/3016658101 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:24:13.406 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:13.402+0000 7f901a400640 1 --2- 172.21.15.47:0/3016658101 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55664f6b08e0 0x55664f6a4c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:24:13.406 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:13.402+0000 7f9019a00640 1 --2- 172.21.15.47:0/3016658101 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55664f6aad80 0x55664f6ac210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:24:13.406 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:13.402+0000 7f9019000640 1 -- 172.21.15.47:0/3016658101 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55664f6aad80 msgr2=0x55664f6ac210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:24:13.406 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:13.402+0000 7f9019000640 1 --2- 172.21.15.47:0/3016658101 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55664f6aad80 0x55664f6ac210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:24:13.406 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:13.402+0000 7f9019000640 1 -- 172.21.15.47:0/3016658101 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55664f6b08e0 msgr2=0x55664f6a4c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:24:13.407 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:13.402+0000 7f9019000640 1 --2- 172.21.15.47:0/3016658101 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55664f6b08e0 0x55664f6a4c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:24:13.407 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:13.402+0000 7f9019000640 1 -- 172.21.15.47:0/3016658101 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55664f698170 con 0x55664f6ab770 2024-03-21T10:24:13.407 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:13.402+0000 7f9019a00640 1 --2- 172.21.15.47:0/3016658101 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55664f6aad80 0x55664f6ac210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T10:24:13.407 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:13.403+0000 7f9019000640 1 --2- 172.21.15.47:0/3016658101 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55664f6ab770 0x55664f6a66d0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f90000860b0 tx=0x7f90000d2ac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b3530fe15725c0d1 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:24:13.407 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:13.403+0000 7f9006000640 1 -- 172.21.15.47:0/3016658101 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90000b3070 con 0x55664f6ab770 2024-03-21T10:24:13.407 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:13.403+0000 7f9006000640 1 -- 172.21.15.47:0/3016658101 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f900009be40 con 0x55664f6ab770 2024-03-21T10:24:13.407 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:13.403+0000 7f9006000640 1 -- 172.21.15.47:0/3016658101 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9000020070 con 0x55664f6ab770 2024-03-21T10:24:13.410 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:13.404+0000 7f901f21e740 1 -- 172.21.15.47:0/3016658101 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55664f6ab770 msgr2=0x55664f6a66d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:24:13.410 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:13.404+0000 7f901f21e740 1 --2- 172.21.15.47:0/3016658101 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55664f6ab770 0x55664f6a66d0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f90000860b0 tx=0x7f90000d2ac0 comp rx=0 tx=0).stop 2024-03-21T10:24:13.410 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:13.404+0000 7f901f21e740 1 -- 172.21.15.47:0/3016658101 shutdown_connections 2024-03-21T10:24:13.410 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:13.404+0000 7f901f21e740 1 --2- 172.21.15.47:0/3016658101 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55664f6b08e0 0x55664f6a4c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:24:13.410 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:13.404+0000 7f901f21e740 1 --2- 172.21.15.47:0/3016658101 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55664f6ab770 0x55664f6a66d0 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:24:13.410 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:13.404+0000 7f901f21e740 1 --2- 172.21.15.47:0/3016658101 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55664f6aad80 0x55664f6ac210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:24:13.410 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:13.404+0000 7f901f21e740 1 -- 172.21.15.47:0/3016658101 >> 172.21.15.47:0/3016658101 conn(0x55664f6a62f0 msgr2=0x55664f6b3de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:24:13.410 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:13.404+0000 7f901f21e740 1 -- 172.21.15.47:0/3016658101 shutdown_connections 2024-03-21T10:24:13.411 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:13.404+0000 7f901f21e740 1 -- 172.21.15.47:0/3016658101 wait complete. 2024-03-21T10:24:13.411 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:13.404+0000 7f901f21e740 1 -- start start 2024-03-21T10:24:13.411 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:13.405+0000 7f901f21e740 1 Processor -- start 2024-03-21T10:24:13.411 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:13.405+0000 7f901f21e740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55664f6aad80 0x55664f6bba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:24:13.411 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:13.405+0000 7f901f21e740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55664f6ab770 0x55664f6bbfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:24:13.411 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:13.405+0000 7f901f21e740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55664f6b08e0 0x55664f6b8a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:24:13.411 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:13.405+0000 7f901f21e740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x55664f6c2bf0 con 0x55664f6b08e0 2024-03-21T10:24:13.411 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:13.405+0000 7f901f21e740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55664f6c2590 con 0x55664f6ab770 2024-03-21T10:24:13.411 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:13.405+0000 7f901f21e740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55664f6c3250 con 0x55664f6aad80 2024-03-21T10:24:13.411 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:13.405+0000 7f901f21e740 10 client.0 _ll_register_callbacks cb 0x7f90000d2b50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T10:24:13.411 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:13.405+0000 7f901a400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55664f6b08e0 0x55664f6b8a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:24:13.411 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:13.405+0000 7f901a400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55664f6b08e0 0x55664f6b8a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:41030/0 (socket says 172.21.15.47:41030) 2024-03-21T10:24:13.411 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:13.405+0000 7f901a400640 1 -- 172.21.15.47:0/2295928095 learned_addr learned my addr 172.21.15.47:0/2295928095 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:24:13.412 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:13.406+0000 7f9019000640 1 --2- 172.21.15.47:0/2295928095 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55664f6ab770 0x55664f6bbfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:24:13.412 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:13.406+0000 7f9019a00640 1 --2- 172.21.15.47:0/2295928095 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55664f6aad80 0x55664f6bba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:24:13.412 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:13.406+0000 7f901a400640 1 -- 172.21.15.47:0/2295928095 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55664f6aad80 msgr2=0x55664f6bba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:24:13.412 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:13.406+0000 7f901a400640 1 --2- 172.21.15.47:0/2295928095 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55664f6aad80 0x55664f6bba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:24:13.412 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:13.406+0000 7f901a400640 1 -- 172.21.15.47:0/2295928095 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55664f6ab770 msgr2=0x55664f6bbfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:24:13.412 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:13.406+0000 7f901a400640 1 --2- 172.21.15.47:0/2295928095 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55664f6ab770 0x55664f6bbfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:24:13.412 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:13.406+0000 7f901a400640 1 -- 172.21.15.47:0/2295928095 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55664f6b9230 con 0x55664f6b08e0 2024-03-21T10:24:13.412 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:13.406+0000 7f901a400640 1 --2- 172.21.15.47:0/2295928095 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55664f6b08e0 0x55664f6b8a00 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f900c009a60 tx=0x7f900c032fa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=25d389d7aeafc8e4 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:24:13.412 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:13.406+0000 7f8ffb400640 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-03-21T10:24:13.412 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:13.407+0000 7f8ffb400640 1 -- 172.21.15.47:0/2295928095 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f900c043050 con 0x55664f6b08e0 2024-03-21T10:24:13.412 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:13.407+0000 7f8ffb400640 1 -- 172.21.15.47:0/2295928095 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9216+0+0 (secure 0 0 0) 0x7f900c003970 con 0x55664f6b08e0 2024-03-21T10:24:13.412 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:13.407+0000 7f901f21e740 1 -- 172.21.15.47:0/2295928095 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f900009be40 con 0x55664f6b08e0 2024-03-21T10:24:13.412 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:13.407+0000 7f8ffb400640 1 -- 172.21.15.47:0/2295928095 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f900c057050 con 0x55664f6b08e0 2024-03-21T10:24:13.412 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:13.407+0000 7f901f21e740 10 client.15276 fetch_fsmap learned FSMap version 29 2024-03-21T10:24:13.412 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:13.407+0000 7f901f21e740 1 -- 172.21.15.47:0/2295928095 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x55664f71d840 con 0x55664f6b08e0 2024-03-21T10:24:13.413 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:13.407+0000 7f8ffb400640 1 -- 172.21.15.47:0/2295928095 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f900009be40 con 0x55664f6b08e0 2024-03-21T10:24:13.413 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:13.407+0000 7f8ffb400640 1 -- 172.21.15.47:0/2295928095 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 29) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f900c03f470 con 0x55664f6b08e0 2024-03-21T10:24:13.413 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:13.408+0000 7f901f21e740 10 client.15276 fetch_fsmap finished waiting for FSMap version 29 2024-03-21T10:24:13.413 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:13.408+0000 7f901f21e740 10 client.15276 resolve_mds: resolved name 'b' to daemon mds.b 2024-03-21T10:24:13.413 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:13.408+0000 7f901f21e740 20 client.15276 populate_metadata read hostname 'smithi047' 2024-03-21T10:24:13.413 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:13.408+0000 7f901f21e740 1 --2- 172.21.15.47:0/2295928095 >> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] conn(0x55664f721a20 0x55664f71da10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:24:13.413 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:13.408+0000 7f901f21e740 4 client.15276 mds_command: new command op to 34535 tid=0 multi_id=0 [{"prefix": "status"}] 2024-03-21T10:24:13.413 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:13.408+0000 7f901f21e740 1 -- 172.21.15.47:0/2295928095 --> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55664f61d1d0 con 0x55664f721a20 2024-03-21T10:24:13.413 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:13.408+0000 7f9019a00640 1 --2- 172.21.15.47:0/2295928095 >> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] conn(0x55664f721a20 0x55664f71da10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:24:13.414 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:13.409+0000 7f9019a00640 1 --2- 172.21.15.47:0/2295928095 >> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] conn(0x55664f721a20 0x55664f71da10 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dcd3addc272b8a3e server_cookie=2ed0f95d7803a0d9 in_seq=0 out_seq=0 2024-03-21T10:24:13.414 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:13.409+0000 7f8ffb400640 10 client.15276 ms_handle_connect on v2:172.21.15.100:6838/4173168440 2024-03-21T10:24:13.415 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:13.409+0000 7f8ffb400640 1 -- 172.21.15.47:0/2295928095 <== mds.0 v2:172.21.15.100:6838/4173168440 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+332 (crc 0 0 0) 0x55664f61d1d0 con 0x55664f721a20 2024-03-21T10:24:13.415 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:13.409+0000 7f8ffb400640 10 client.15276 handle_command_reply: tid=0 2024-03-21T10:24:13.415 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:13.409+0000 7f901f21e740 1 client.15276 shutdown 2024-03-21T10:24:13.415 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:13.409+0000 7f901f21e740 1 -- 172.21.15.47:0/2295928095 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55664f6b08e0 msgr2=0x55664f6b8a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:24:13.415 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:13.409+0000 7f901f21e740 1 --2- 172.21.15.47:0/2295928095 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55664f6b08e0 0x55664f6b8a00 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f900c009a60 tx=0x7f900c032fa0 comp rx=0 tx=0).stop 2024-03-21T10:24:13.415 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:13.410+0000 7f901f21e740 1 -- 172.21.15.47:0/2295928095 shutdown_connections 2024-03-21T10:24:13.415 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:13.410+0000 7f901f21e740 1 --2- 172.21.15.47:0/2295928095 >> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] conn(0x55664f721a20 0x55664f71da10 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:24:13.415 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:13.410+0000 7f901f21e740 1 --2- 172.21.15.47:0/2295928095 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55664f6b08e0 0x55664f6b8a00 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:24:13.415 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:13.410+0000 7f901f21e740 1 --2- 172.21.15.47:0/2295928095 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55664f6ab770 0x55664f6bbfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:24:13.415 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:13.410+0000 7f901f21e740 1 --2- 172.21.15.47:0/2295928095 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55664f6aad80 0x55664f6bba60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:24:13.415 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:13.410+0000 7f901f21e740 1 -- 172.21.15.47:0/2295928095 >> 172.21.15.47:0/2295928095 conn(0x55664f6a62f0 msgr2=0x55664f6ac210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:24:13.415 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:13.410+0000 7f8ffb400640 0 client.15276 ms_handle_reset on v2:172.21.15.100:6838/4173168440 2024-03-21T10:24:13.415 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:13.410+0000 7f901f21e740 1 -- 172.21.15.47:0/2295928095 shutdown_connections 2024-03-21T10:24:13.415 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:13.410+0000 7f901f21e740 1 -- 172.21.15.47:0/2295928095 wait complete. 2024-03-21T10:24:13.416 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:13.410+0000 7f901f21e740 20 client.15276 trim_cache size 0 max 16384 2024-03-21T10:24:13.416 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-03-21T10:24:13.416 INFO:teuthology.orchestra.run.smithi047.stdout: "cluster_fsid": "3056d51a-e76b-11ee-95cd-87774f69a715", 2024-03-21T10:24:13.416 INFO:teuthology.orchestra.run.smithi047.stdout: "whoami": 0, 2024-03-21T10:24:13.416 INFO:teuthology.orchestra.run.smithi047.stdout: "id": 34535, 2024-03-21T10:24:13.416 INFO:teuthology.orchestra.run.smithi047.stdout: "want_state": "up:active", 2024-03-21T10:24:13.416 INFO:teuthology.orchestra.run.smithi047.stdout: "state": "up:active", 2024-03-21T10:24:13.416 INFO:teuthology.orchestra.run.smithi047.stdout: "fs_name": "cephfs", 2024-03-21T10:24:13.416 INFO:teuthology.orchestra.run.smithi047.stdout: "rank_uptime": 134.15376021500001, 2024-03-21T10:24:13.416 INFO:teuthology.orchestra.run.smithi047.stdout: "mdsmap_epoch": 29, 2024-03-21T10:24:13.416 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap_epoch": 95, 2024-03-21T10:24:13.417 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap_epoch_barrier": 93, 2024-03-21T10:24:13.417 INFO:teuthology.orchestra.run.smithi047.stdout: "uptime": 139.81555909799999 2024-03-21T10:24:13.417 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-03-21T10:24:13.417 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:13.411+0000 7f901ae00640 1 -- 172.21.15.47:0/2770599993 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f8ffc0a2fa0 msgr2=0x7f8ffc0a5460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:24:13.417 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:13.411+0000 7f901ae00640 1 --2- 172.21.15.47:0/2770599993 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f8ffc0a2fa0 0x7f8ffc0a5460 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x55664f6abb70 tx=0x7f9008005f90 comp rx=0 tx=0).stop 2024-03-21T10:24:13.418 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:13.411+0000 7f901ae00640 1 -- 172.21.15.47:0/2770599993 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9014101670 msgr2=0x7f9014199eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:24:13.418 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:13.411+0000 7f901ae00640 1 --2- 172.21.15.47:0/2770599993 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9014101670 0x7f9014199eb0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f900000d950 tx=0x7f900000de10 comp rx=0 tx=0).stop 2024-03-21T10:24:13.418 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:13.411+0000 7f901ae00640 1 -- 172.21.15.47:0/2770599993 shutdown_connections 2024-03-21T10:24:13.418 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:13.411+0000 7f901ae00640 1 --2- 172.21.15.47:0/2770599993 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f8ffc0a2fa0 0x7f8ffc0a5460 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:24:13.418 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:13.411+0000 7f901ae00640 1 --2- 172.21.15.47:0/2770599993 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9014102030 0x7f90141a0f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:24:13.418 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:13.411+0000 7f901ae00640 1 --2- 172.21.15.47:0/2770599993 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9014101670 0x7f9014199eb0 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:24:13.418 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:13.411+0000 7f901ae00640 1 --2- 172.21.15.47:0/2770599993 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9014100480 0x7f9014199970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:24:13.418 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:13.411+0000 7f901ae00640 1 -- 172.21.15.47:0/2770599993 >> 172.21.15.47:0/2770599993 conn(0x7f90140fba10 msgr2=0x7f9014109280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:24:13.418 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:13.411+0000 7f901ae00640 1 -- 172.21.15.47:0/2770599993 shutdown_connections 2024-03-21T10:24:13.418 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:13.411+0000 7f901ae00640 1 -- 172.21.15.47:0/2770599993 wait complete. 2024-03-21T10:24:13.432 INFO:tasks.cephadm:Setup complete, yielding 2024-03-21T10:24:13.433 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-03-21T10:24:13.442 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi047.front.sepia.ceph.com 2024-03-21T10:24:13.443 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2edfee68ad7243bed422a049b6f5f58619300f11 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3056d51a-e76b-11ee-95cd-87774f69a715 -- bash -c 'ceph orch status' 2024-03-21T10:24:13.918 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/3056d51a-e76b-11ee-95cd-87774f69a715/mon.a/config 2024-03-21T10:24:14.214 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:24:13 smithi047 ceph-mon[31547]: pgmap v438: 129 pgs: 129 active+clean; 642 KiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-21T10:24:14.229 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:24:13 smithi100 ceph-mon[35193]: pgmap v438: 129 pgs: 129 active+clean; 642 KiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-21T10:24:14.240 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:24:13 smithi157 ceph-mon[35581]: pgmap v438: 129 pgs: 129 active+clean; 642 KiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-21T10:24:15.214 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:24:14 smithi047 ceph-mon[31547]: pgmap v439: 129 pgs: 129 active+clean; 642 KiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-03-21T10:24:15.229 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:24:14 smithi100 ceph-mon[35193]: pgmap v439: 129 pgs: 129 active+clean; 642 KiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-03-21T10:24:15.240 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:24:14 smithi157 ceph-mon[35581]: pgmap v439: 129 pgs: 129 active+clean; 642 KiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-03-21T10:24:16.398 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:16.394+0000 7fa7af200700 1 -- 172.21.15.47:0/3682379643 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa7a81021e0 msgr2=0x7fa7a8102600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:24:16.398 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:16.394+0000 7fa7af200700 1 --2- 172.21.15.47:0/3682379643 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa7a81021e0 0x7fa7a8102600 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7fa7980099c0 tx=0x7fa798009cd0 comp rx=0 tx=0).stop 2024-03-21T10:24:16.398 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:16.395+0000 7fa7af200700 1 -- 172.21.15.47:0/3682379643 shutdown_connections 2024-03-21T10:24:16.398 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:16.395+0000 7fa7af200700 1 --2- 172.21.15.47:0/3682379643 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa7a8103d80 0x7fa7a810a630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:24:16.398 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:16.395+0000 7fa7af200700 1 --2- 172.21.15.47:0/3682379643 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa7a81021e0 0x7fa7a8102600 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:24:16.398 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:16.395+0000 7fa7af200700 1 --2- 172.21.15.47:0/3682379643 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa7a81033e0 0x7fa7a8103840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:24:16.398 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:16.395+0000 7fa7af200700 1 -- 172.21.15.47:0/3682379643 >> 172.21.15.47:0/3682379643 conn(0x7fa7a80fd7a0 msgr2=0x7fa7a80ffbc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:24:16.398 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:16.395+0000 7fa7af200700 1 -- 172.21.15.47:0/3682379643 shutdown_connections 2024-03-21T10:24:16.398 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:16.395+0000 7fa7af200700 1 -- 172.21.15.47:0/3682379643 wait complete. 2024-03-21T10:24:16.400 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:16.396+0000 7fa7af200700 1 Processor -- start 2024-03-21T10:24:16.400 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:16.396+0000 7fa7af200700 1 -- start start 2024-03-21T10:24:16.400 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:16.396+0000 7fa7af200700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa7a81021e0 0x7fa7a819a590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:24:16.400 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:16.396+0000 7fa7af200700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa7a81033e0 0x7fa7a819aad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:24:16.400 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:16.397+0000 7fa7af200700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa7a8103d80 0x7fa7a81a1b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:24:16.400 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:16.397+0000 7fa7af200700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7a810dca0 con 0x7fa7a81033e0 2024-03-21T10:24:16.400 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:16.397+0000 7fa7af200700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7a810db20 con 0x7fa7a8103d80 2024-03-21T10:24:16.400 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:16.397+0000 7fa7af200700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7a810de20 con 0x7fa7a81021e0 2024-03-21T10:24:16.400 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:16.397+0000 7fa79fe00700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa7a81033e0 0x7fa7a819aad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:24:16.401 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:16.397+0000 7fa79fe00700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa7a81033e0 0x7fa7a819aad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:41048/0 (socket says 172.21.15.47:41048) 2024-03-21T10:24:16.401 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:16.397+0000 7fa79fe00700 1 -- 172.21.15.47:0/2752618529 learned_addr learned my addr 172.21.15.47:0/2752618529 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:24:16.401 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:16.397+0000 7fa7ad200700 1 --2- 172.21.15.47:0/2752618529 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa7a8103d80 0x7fa7a81a1b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:24:16.401 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:16.397+0000 7fa7a7e00700 1 --2- 172.21.15.47:0/2752618529 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa7a81021e0 0x7fa7a819a590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:24:16.401 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:16.397+0000 7fa79fe00700 1 -- 172.21.15.47:0/2752618529 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa7a81021e0 msgr2=0x7fa7a819a590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:24:16.401 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:16.397+0000 7fa79fe00700 1 --2- 172.21.15.47:0/2752618529 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa7a81021e0 0x7fa7a819a590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:24:16.401 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:16.397+0000 7fa79fe00700 1 -- 172.21.15.47:0/2752618529 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa7a8103d80 msgr2=0x7fa7a81a1b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:24:16.401 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:16.397+0000 7fa79fe00700 1 --2- 172.21.15.47:0/2752618529 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa7a8103d80 0x7fa7a81a1b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:24:16.401 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:16.398+0000 7fa79fe00700 1 -- 172.21.15.47:0/2752618529 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7a81a21f0 con 0x7fa7a81033e0 2024-03-21T10:24:16.401 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:16.398+0000 7fa79fe00700 1 --2- 172.21.15.47:0/2752618529 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa7a81033e0 0x7fa7a819aad0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7fa79000d980 tx=0x7fa79000dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:24:16.402 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:16.398+0000 7fa7a5600700 1 -- 172.21.15.47:0/2752618529 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa790016070 con 0x7fa7a81033e0 2024-03-21T10:24:16.402 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:16.398+0000 7fa7a5600700 1 -- 172.21.15.47:0/2752618529 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fa790009d80 con 0x7fa7a81033e0 2024-03-21T10:24:16.402 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:16.398+0000 7fa7af200700 1 -- 172.21.15.47:0/2752618529 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa7a81a24e0 con 0x7fa7a81033e0 2024-03-21T10:24:16.402 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:16.398+0000 7fa7a5600700 1 -- 172.21.15.47:0/2752618529 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa79000f3c0 con 0x7fa7a81033e0 2024-03-21T10:24:16.402 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:16.399+0000 7fa7af200700 1 -- 172.21.15.47:0/2752618529 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa7a81a2a20 con 0x7fa7a81033e0 2024-03-21T10:24:16.402 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:16.400+0000 7fa7af200700 1 -- 172.21.15.47:0/2752618529 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa7a8068e50 con 0x7fa7a81033e0 2024-03-21T10:24:16.404 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:16.402+0000 7fa7a5600700 1 -- 172.21.15.47:0/2752618529 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 18) v1 ==== 137854+0+0 (secure 0 0 0) 0x7fa790019070 con 0x7fa7a81033e0 2024-03-21T10:24:16.405 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:16.403+0000 7fa7a5600700 1 --2- 172.21.15.47:0/2752618529 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7fa78c0a2fb0 0x7fa78c0a5470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:24:16.405 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:16.403+0000 7fa7a5600700 1 -- 172.21.15.47:0/2752618529 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9216+0+0 (secure 0 0 0) 0x7fa7900cf2a0 con 0x7fa7a81033e0 2024-03-21T10:24:16.406 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:16.403+0000 7fa7a7e00700 1 --2- 172.21.15.47:0/2752618529 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7fa78c0a2fb0 0x7fa78c0a5470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:24:16.406 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:16.404+0000 7fa7a7e00700 1 --2- 172.21.15.47:0/2752618529 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7fa78c0a2fb0 0x7fa78c0a5470 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fa79800b580 tx=0x7fa798005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:24:16.411 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:16.409+0000 7fa7a5600700 1 -- 172.21.15.47:0/2752618529 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fa79009bcc0 con 0x7fa7a81033e0 2024-03-21T10:24:16.656 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:16.653+0000 7fa7af200700 1 -- 172.21.15.47:0/2752618529 --> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] -- mgr_command(tid 0: {"prefix": "orch status", "target": ["mon-mgr", ""]}) v1 -- 0x7fa7a8075ae0 con 0x7fa78c0a2fb0 2024-03-21T10:24:16.658 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:16.656+0000 7fa7a5600700 1 -- 172.21.15.47:0/2752618529 <== mgr.14152 v2:172.21.15.47:6800/2823885994 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+43 (secure 0 0 0) 0x7fa7a8075ae0 con 0x7fa78c0a2fb0 2024-03-21T10:24:16.658 INFO:teuthology.orchestra.run.smithi047.stdout:Backend: cephadm 2024-03-21T10:24:16.658 INFO:teuthology.orchestra.run.smithi047.stdout:Available: Yes 2024-03-21T10:24:16.659 INFO:teuthology.orchestra.run.smithi047.stdout:Paused: No 2024-03-21T10:24:16.662 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:16.659+0000 7fa7af200700 1 -- 172.21.15.47:0/2752618529 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7fa78c0a2fb0 msgr2=0x7fa78c0a5470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:24:16.663 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:16.659+0000 7fa7af200700 1 --2- 172.21.15.47:0/2752618529 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7fa78c0a2fb0 0x7fa78c0a5470 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fa79800b580 tx=0x7fa798005fb0 comp rx=0 tx=0).stop 2024-03-21T10:24:16.663 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:16.659+0000 7fa7af200700 1 -- 172.21.15.47:0/2752618529 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa7a81033e0 msgr2=0x7fa7a819aad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:24:16.663 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:16.659+0000 7fa7af200700 1 --2- 172.21.15.47:0/2752618529 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa7a81033e0 0x7fa7a819aad0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7fa79000d980 tx=0x7fa79000dc90 comp rx=0 tx=0).stop 2024-03-21T10:24:16.663 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:16.660+0000 7fa7af200700 1 -- 172.21.15.47:0/2752618529 shutdown_connections 2024-03-21T10:24:16.663 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:16.660+0000 7fa7af200700 1 --2- 172.21.15.47:0/2752618529 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7fa78c0a2fb0 0x7fa78c0a5470 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:24:16.663 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:16.660+0000 7fa7af200700 1 --2- 172.21.15.47:0/2752618529 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa7a8103d80 0x7fa7a81a1b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:24:16.663 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:16.660+0000 7fa7af200700 1 --2- 172.21.15.47:0/2752618529 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa7a81021e0 0x7fa7a819a590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:24:16.663 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:16.660+0000 7fa7af200700 1 --2- 172.21.15.47:0/2752618529 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa7a81033e0 0x7fa7a819aad0 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:24:16.664 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:16.660+0000 7fa7af200700 1 -- 172.21.15.47:0/2752618529 >> 172.21.15.47:0/2752618529 conn(0x7fa7a80fd7a0 msgr2=0x7fa7a80743c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:24:16.664 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:16.660+0000 7fa7af200700 1 -- 172.21.15.47:0/2752618529 shutdown_connections 2024-03-21T10:24:16.664 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:16.660+0000 7fa7af200700 1 -- 172.21.15.47:0/2752618529 wait complete. 2024-03-21T10:24:17.464 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:24:17 smithi047 ceph-mon[31547]: pgmap v440: 129 pgs: 129 active+clean; 642 KiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-21T10:24:17.464 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:24:17 smithi047 ceph-mon[31547]: from='client.15282 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:24:17.479 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:24:17 smithi100 ceph-mon[35193]: pgmap v440: 129 pgs: 129 active+clean; 642 KiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-21T10:24:17.479 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:24:17 smithi100 ceph-mon[35193]: from='client.15282 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:24:17.489 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:24:17 smithi157 ceph-mon[35581]: pgmap v440: 129 pgs: 129 active+clean; 642 KiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-21T10:24:17.490 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:24:17 smithi157 ceph-mon[35581]: from='client.15282 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:24:17.532 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2edfee68ad7243bed422a049b6f5f58619300f11 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3056d51a-e76b-11ee-95cd-87774f69a715 -- bash -c 'ceph orch ps' 2024-03-21T10:24:18.002 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/3056d51a-e76b-11ee-95cd-87774f69a715/mon.a/config 2024-03-21T10:24:19.445 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:24:19 smithi047 ceph-mon[31547]: pgmap v441: 129 pgs: 129 active+clean; 642 KiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-03-21T10:24:19.479 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:24:19 smithi100 ceph-mon[35193]: pgmap v441: 129 pgs: 129 active+clean; 642 KiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-03-21T10:24:19.490 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:24:19 smithi157 ceph-mon[35581]: pgmap v441: 129 pgs: 129 active+clean; 642 KiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-03-21T10:24:20.248 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:20.246+0000 7fbafac00700 1 -- 172.21.15.47:0/2250187732 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbaf4102d10 msgr2=0x7fbaf4105100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:24:20.249 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:20.246+0000 7fbafac00700 1 --2- 172.21.15.47:0/2250187732 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbaf4102d10 0x7fbaf4105100 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fbae40098f0 tx=0x7fbae4009c00 comp rx=0 tx=0).stop 2024-03-21T10:24:20.249 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:20.247+0000 7fbafac00700 1 -- 172.21.15.47:0/2250187732 shutdown_connections 2024-03-21T10:24:20.249 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:20.247+0000 7fbafac00700 1 --2- 172.21.15.47:0/2250187732 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbaf4107f70 0x7fbaf410a360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:24:20.249 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:20.247+0000 7fbafac00700 1 --2- 172.21.15.47:0/2250187732 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbaf4102d10 0x7fbaf4105100 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:24:20.249 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:20.247+0000 7fbafac00700 1 --2- 172.21.15.47:0/2250187732 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbaf4105640 0x7fbaf4107a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:24:20.249 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:20.247+0000 7fbafac00700 1 -- 172.21.15.47:0/2250187732 >> 172.21.15.47:0/2250187732 conn(0x7fbaf40fc660 msgr2=0x7fbaf40feaa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:24:20.249 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:20.247+0000 7fbafac00700 1 -- 172.21.15.47:0/2250187732 shutdown_connections 2024-03-21T10:24:20.249 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:20.247+0000 7fbafac00700 1 -- 172.21.15.47:0/2250187732 wait complete. 2024-03-21T10:24:20.250 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:20.248+0000 7fbafac00700 1 Processor -- start 2024-03-21T10:24:20.250 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:20.248+0000 7fbafac00700 1 -- start start 2024-03-21T10:24:20.250 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:20.249+0000 7fbafac00700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbaf4102d10 0x7fbaf41983d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:24:20.250 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:20.249+0000 7fbafac00700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbaf4105640 0x7fbaf4198910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:24:20.250 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:20.249+0000 7fbafac00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbaf4107f70 0x7fbaf419f990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:24:20.250 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:20.249+0000 7fbafac00700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbaf410dc90 con 0x7fbaf4102d10 2024-03-21T10:24:20.251 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:20.249+0000 7fbafac00700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbaf410db10 con 0x7fbaf4105640 2024-03-21T10:24:20.251 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:20.249+0000 7fbafac00700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbaf410de10 con 0x7fbaf4107f70 2024-03-21T10:24:20.251 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:20.249+0000 7fbaf3e00700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbaf4102d10 0x7fbaf41983d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:24:20.251 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:20.249+0000 7fbaf3e00700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbaf4102d10 0x7fbaf41983d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:41066/0 (socket says 172.21.15.47:41066) 2024-03-21T10:24:20.251 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:20.249+0000 7fbaf3e00700 1 -- 172.21.15.47:0/3562023762 learned_addr learned my addr 172.21.15.47:0/3562023762 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:24:20.251 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:20.249+0000 7fbaf3400700 1 --2- 172.21.15.47:0/3562023762 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbaf4105640 0x7fbaf4198910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:24:20.251 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:20.249+0000 7fbaf8c00700 1 --2- 172.21.15.47:0/3562023762 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbaf4107f70 0x7fbaf419f990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:24:20.252 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:20.249+0000 7fbaf3e00700 1 -- 172.21.15.47:0/3562023762 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbaf4107f70 msgr2=0x7fbaf419f990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:24:20.252 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:20.249+0000 7fbaf3e00700 1 --2- 172.21.15.47:0/3562023762 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbaf4107f70 0x7fbaf419f990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:24:20.252 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:20.249+0000 7fbaf3e00700 1 -- 172.21.15.47:0/3562023762 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbaf4105640 msgr2=0x7fbaf4198910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:24:20.252 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:20.249+0000 7fbaf3e00700 1 --2- 172.21.15.47:0/3562023762 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbaf4105640 0x7fbaf4198910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:24:20.252 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:20.249+0000 7fbaf3e00700 1 -- 172.21.15.47:0/3562023762 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbaf41a0030 con 0x7fbaf4102d10 2024-03-21T10:24:20.252 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:20.250+0000 7fbaf3e00700 1 --2- 172.21.15.47:0/3562023762 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbaf4102d10 0x7fbaf41983d0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7fbae4005270 tx=0x7fbae40052a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:24:20.252 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:20.250+0000 7fbaf0c00700 1 -- 172.21.15.47:0/3562023762 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbae400b560 con 0x7fbaf4102d10 2024-03-21T10:24:20.252 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:20.250+0000 7fbaf0c00700 1 -- 172.21.15.47:0/3562023762 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fbae400bbb0 con 0x7fbaf4102d10 2024-03-21T10:24:20.252 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:20.250+0000 7fbaf0c00700 1 -- 172.21.15.47:0/3562023762 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbae4017400 con 0x7fbaf4102d10 2024-03-21T10:24:20.252 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:20.250+0000 7fbafac00700 1 -- 172.21.15.47:0/3562023762 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbaf41a02c0 con 0x7fbaf4102d10 2024-03-21T10:24:20.252 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:20.250+0000 7fbafac00700 1 -- 172.21.15.47:0/3562023762 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbaf41a0710 con 0x7fbaf4102d10 2024-03-21T10:24:20.253 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:20.251+0000 7fbafac00700 1 -- 172.21.15.47:0/3562023762 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbaf4068e50 con 0x7fbaf4102d10 2024-03-21T10:24:20.256 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:20.255+0000 7fbaf0c00700 1 -- 172.21.15.47:0/3562023762 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 18) v1 ==== 137854+0+0 (secure 0 0 0) 0x7fbae400b700 con 0x7fbaf4102d10 2024-03-21T10:24:20.258 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:20.256+0000 7fbaf0c00700 1 --2- 172.21.15.47:0/3562023762 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7fbad80a2fb0 0x7fbad80a5470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:24:20.258 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:20.256+0000 7fbaf0c00700 1 -- 172.21.15.47:0/3562023762 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9216+0+0 (secure 0 0 0) 0x7fbae40d0fa0 con 0x7fbaf4102d10 2024-03-21T10:24:20.258 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:20.256+0000 7fbaf3400700 1 --2- 172.21.15.47:0/3562023762 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7fbad80a2fb0 0x7fbad80a5470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:24:20.258 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:20.257+0000 7fbaf3400700 1 --2- 172.21.15.47:0/3562023762 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7fbad80a2fb0 0x7fbad80a5470 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fbae0009750 tx=0x7fbae0009310 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:24:20.263 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:20.262+0000 7fbaf0c00700 1 -- 172.21.15.47:0/3562023762 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fbae4013030 con 0x7fbaf4102d10 2024-03-21T10:24:20.501 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:20.499+0000 7fbafac00700 1 -- 172.21.15.47:0/3562023762 --> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7fbaf40625d0 con 0x7fbad80a2fb0 2024-03-21T10:24:20.541 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:20.539+0000 7fbaf0c00700 1 -- 172.21.15.47:0/3562023762 <== mgr.14152 v2:172.21.15.47:6800/2823885994 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+4123 (secure 0 0 0) 0x7fbaf40625d0 con 0x7fbad80a2fb0 2024-03-21T10:24:20.542 INFO:teuthology.orchestra.run.smithi047.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-03-21T10:24:20.542 INFO:teuthology.orchestra.run.smithi047.stdout:mds.a smithi047 running (2m) 40s ago 2m 15.7M - 19.0.0-2231-g2edfee68 237e43c7a65f 4ecabcd2ea34 2024-03-21T10:24:20.543 INFO:teuthology.orchestra.run.smithi047.stdout:mds.b smithi100 running (2m) 74s ago 2m 24.6M - 19.0.0-2231-g2edfee68 237e43c7a65f 030eeb395aa2 2024-03-21T10:24:20.543 INFO:teuthology.orchestra.run.smithi047.stdout:mds.c smithi157 running (2m) 75s ago 2m 14.4M - 19.0.0-2231-g2edfee68 237e43c7a65f 55d8b6b1a13b 2024-03-21T10:24:20.543 INFO:teuthology.orchestra.run.smithi047.stdout:mds.d smithi047 running (2m) 41s ago 2m 12.5M - 19.0.0-2231-g2edfee68 237e43c7a65f 4ae97ca5479a 2024-03-21T10:24:20.543 INFO:teuthology.orchestra.run.smithi047.stdout:mds.e smithi100 running (2m) 74s ago 2m 15.8M - 19.0.0-2231-g2edfee68 237e43c7a65f bce3049fc805 2024-03-21T10:24:20.543 INFO:teuthology.orchestra.run.smithi047.stdout:mds.f smithi157 running (2m) 75s ago 2m 14.4M - 19.0.0-2231-g2edfee68 237e43c7a65f 96a8befe1f79 2024-03-21T10:24:20.543 INFO:teuthology.orchestra.run.smithi047.stdout:mds.g smithi047 running (2m) 40s ago 2m 15.0M - 19.0.0-2231-g2edfee68 237e43c7a65f b018904203b9 2024-03-21T10:24:20.543 INFO:teuthology.orchestra.run.smithi047.stdout:mds.h smithi100 running (2m) 74s ago 2m 15.8M - 19.0.0-2231-g2edfee68 237e43c7a65f 8a41fac68642 2024-03-21T10:24:20.543 INFO:teuthology.orchestra.run.smithi047.stdout:mds.i smithi157 running (2m) 75s ago 2m 15.9M - 19.0.0-2231-g2edfee68 237e43c7a65f 9a440ce57d23 2024-03-21T10:24:20.543 INFO:teuthology.orchestra.run.smithi047.stdout:mds.j smithi047 running (3m) 41s ago 3m 12.5M - 19.0.0-2231-g2edfee68 237e43c7a65f 2488f82e2f50 2024-03-21T10:24:20.543 INFO:teuthology.orchestra.run.smithi047.stdout:mds.k smithi100 running (3m) 74s ago 3m 15.5M - 19.0.0-2231-g2edfee68 237e43c7a65f 6867ea9076c0 2024-03-21T10:24:20.543 INFO:teuthology.orchestra.run.smithi047.stdout:mds.l smithi157 running (3m) 75s ago 3m 14.5M - 19.0.0-2231-g2edfee68 237e43c7a65f 49f83b9ee100 2024-03-21T10:24:20.544 INFO:teuthology.orchestra.run.smithi047.stdout:mgr.x smithi047 *:9283,8765 running (12m) 41s ago 12m 515M - 19.0.0-2231-g2edfee68 237e43c7a65f d2faba9824e0 2024-03-21T10:24:20.544 INFO:teuthology.orchestra.run.smithi047.stdout:mgr.y smithi100 *:8443,8765 running (10m) 74s ago 10m 451M - 19.0.0-2231-g2edfee68 237e43c7a65f f904fc9002ac 2024-03-21T10:24:20.544 INFO:teuthology.orchestra.run.smithi047.stdout:mgr.z smithi157 *:8443,8765 running (10m) 75s ago 10m 452M - 19.0.0-2231-g2edfee68 237e43c7a65f 7392304e277d 2024-03-21T10:24:20.544 INFO:teuthology.orchestra.run.smithi047.stdout:mon.a smithi047 running (12m) 41s ago 13m 51.0M 2048M 19.0.0-2231-g2edfee68 237e43c7a65f e43a05e9bf7b 2024-03-21T10:24:20.544 INFO:teuthology.orchestra.run.smithi047.stdout:mon.b smithi100 running (10m) 74s ago 10m 40.3M 2048M 19.0.0-2231-g2edfee68 237e43c7a65f 5aa332c64f45 2024-03-21T10:24:20.544 INFO:teuthology.orchestra.run.smithi047.stdout:mon.c smithi157 running (10m) 75s ago 10m 40.4M 2048M 19.0.0-2231-g2edfee68 237e43c7a65f d9e77d928b1e 2024-03-21T10:24:20.544 INFO:teuthology.orchestra.run.smithi047.stdout:osd.0 smithi047 running (9m) 41s ago 9m 45.1M 4273M 19.0.0-2231-g2edfee68 237e43c7a65f dff9fdcb42dc 2024-03-21T10:24:20.544 INFO:teuthology.orchestra.run.smithi047.stdout:osd.1 smithi100 running (9m) 74s ago 9m 55.4M 4272M 19.0.0-2231-g2edfee68 237e43c7a65f e02373ec2d73 2024-03-21T10:24:20.544 INFO:teuthology.orchestra.run.smithi047.stdout:osd.2 smithi157 running (8m) 75s ago 8m 52.7M 4272M 19.0.0-2231-g2edfee68 237e43c7a65f 7205552e07a6 2024-03-21T10:24:20.544 INFO:teuthology.orchestra.run.smithi047.stdout:osd.3 smithi047 running (8m) 41s ago 8m 51.3M 4273M 19.0.0-2231-g2edfee68 237e43c7a65f 042c8ec94546 2024-03-21T10:24:20.544 INFO:teuthology.orchestra.run.smithi047.stdout:osd.4 smithi100 running (7m) 74s ago 7m 42.7M 4272M 19.0.0-2231-g2edfee68 237e43c7a65f e1dd02fadaab 2024-03-21T10:24:20.544 INFO:teuthology.orchestra.run.smithi047.stdout:osd.5 smithi157 running (7m) 75s ago 7m 50.7M 4272M 19.0.0-2231-g2edfee68 237e43c7a65f a4e2eea4f685 2024-03-21T10:24:20.544 INFO:teuthology.orchestra.run.smithi047.stdout:osd.6 smithi047 running (6m) 41s ago 6m 49.3M 4273M 19.0.0-2231-g2edfee68 237e43c7a65f ef71a58e6ea5 2024-03-21T10:24:20.544 INFO:teuthology.orchestra.run.smithi047.stdout:osd.7 smithi100 running (5m) 74s ago 5m 43.3M 4272M 19.0.0-2231-g2edfee68 237e43c7a65f a1542225a4f5 2024-03-21T10:24:20.544 INFO:teuthology.orchestra.run.smithi047.stdout:osd.8 smithi157 running (5m) 75s ago 5m 49.9M 4272M 19.0.0-2231-g2edfee68 237e43c7a65f 5255a1eb55e0 2024-03-21T10:24:20.545 INFO:teuthology.orchestra.run.smithi047.stdout:osd.9 smithi047 running (4m) 41s ago 4m 44.3M 4273M 19.0.0-2231-g2edfee68 237e43c7a65f 831d8f0ad59f 2024-03-21T10:24:20.545 INFO:teuthology.orchestra.run.smithi047.stdout:osd.10 smithi100 running (4m) 74s ago 4m 51.3M 4272M 19.0.0-2231-g2edfee68 237e43c7a65f d27e4913dd06 2024-03-21T10:24:20.545 INFO:teuthology.orchestra.run.smithi047.stdout:osd.11 smithi157 running (3m) 75s ago 3m 47.7M 4272M 19.0.0-2231-g2edfee68 237e43c7a65f 32f5dd1c5653 2024-03-21T10:24:20.545 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:20.542+0000 7fbafac00700 1 -- 172.21.15.47:0/3562023762 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7fbad80a2fb0 msgr2=0x7fbad80a5470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:24:20.545 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:20.542+0000 7fbafac00700 1 --2- 172.21.15.47:0/3562023762 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7fbad80a2fb0 0x7fbad80a5470 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fbae0009750 tx=0x7fbae0009310 comp rx=0 tx=0).stop 2024-03-21T10:24:20.545 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:20.542+0000 7fbafac00700 1 -- 172.21.15.47:0/3562023762 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbaf4102d10 msgr2=0x7fbaf41983d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:24:20.545 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:20.542+0000 7fbafac00700 1 --2- 172.21.15.47:0/3562023762 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbaf4102d10 0x7fbaf41983d0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7fbae4005270 tx=0x7fbae40052a0 comp rx=0 tx=0).stop 2024-03-21T10:24:20.546 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:20.542+0000 7fbafac00700 1 -- 172.21.15.47:0/3562023762 shutdown_connections 2024-03-21T10:24:20.546 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:20.542+0000 7fbafac00700 1 --2- 172.21.15.47:0/3562023762 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7fbad80a2fb0 0x7fbad80a5470 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:24:20.546 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:20.542+0000 7fbafac00700 1 --2- 172.21.15.47:0/3562023762 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbaf4107f70 0x7fbaf419f990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:24:20.546 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:20.542+0000 7fbafac00700 1 --2- 172.21.15.47:0/3562023762 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbaf4102d10 0x7fbaf41983d0 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:24:20.546 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:20.542+0000 7fbafac00700 1 --2- 172.21.15.47:0/3562023762 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbaf4105640 0x7fbaf4198910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:24:20.546 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:20.542+0000 7fbafac00700 1 -- 172.21.15.47:0/3562023762 >> 172.21.15.47:0/3562023762 conn(0x7fbaf40fc660 msgr2=0x7fbaf41037b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:24:20.546 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:20.542+0000 7fbafac00700 1 -- 172.21.15.47:0/3562023762 shutdown_connections 2024-03-21T10:24:20.546 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:20.542+0000 7fbafac00700 1 -- 172.21.15.47:0/3562023762 wait complete. 2024-03-21T10:24:21.464 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:24:21 smithi047 ceph-mon[31547]: pgmap v442: 129 pgs: 129 active+clean; 642 KiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-03-21T10:24:21.464 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:24:21 smithi047 ceph-mon[31547]: from='client.15288 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:24:21.480 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:24:21 smithi100 ceph-mon[35193]: pgmap v442: 129 pgs: 129 active+clean; 642 KiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-03-21T10:24:21.480 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:24:21 smithi100 ceph-mon[35193]: from='client.15288 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:24:21.490 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:24:21 smithi157 ceph-mon[35581]: pgmap v442: 129 pgs: 129 active+clean; 642 KiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-03-21T10:24:21.490 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:24:21 smithi157 ceph-mon[35581]: from='client.15288 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:24:21.593 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2edfee68ad7243bed422a049b6f5f58619300f11 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3056d51a-e76b-11ee-95cd-87774f69a715 -- bash -c 'ceph orch ls' 2024-03-21T10:24:22.101 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/3056d51a-e76b-11ee-95cd-87774f69a715/mon.a/config 2024-03-21T10:24:23.464 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:24:23 smithi047 ceph-mon[31547]: pgmap v443: 129 pgs: 129 active+clean; 642 KiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-21T10:24:23.479 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:24:23 smithi100 ceph-mon[35193]: pgmap v443: 129 pgs: 129 active+clean; 642 KiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-21T10:24:23.489 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:24:23 smithi157 ceph-mon[35581]: pgmap v443: 129 pgs: 129 active+clean; 642 KiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-21T10:24:24.567 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:24.565+0000 7f8126600700 1 -- 172.21.15.47:0/4197927843 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8120102c50 msgr2=0x7f8120105040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:24:24.568 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:24.565+0000 7f8126600700 1 --2- 172.21.15.47:0/4197927843 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8120102c50 0x7f8120105040 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f8110009990 tx=0x7f8110009ca0 comp rx=0 tx=0).stop 2024-03-21T10:24:24.568 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:24.565+0000 7f8126600700 1 -- 172.21.15.47:0/4197927843 shutdown_connections 2024-03-21T10:24:24.568 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:24.565+0000 7f8126600700 1 --2- 172.21.15.47:0/4197927843 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8120107eb0 0x7f812010a320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:24:24.568 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:24.565+0000 7f8126600700 1 --2- 172.21.15.47:0/4197927843 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8120102c50 0x7f8120105040 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:24:24.568 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:24.565+0000 7f8126600700 1 --2- 172.21.15.47:0/4197927843 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8120105580 0x7f8120107970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:24:24.568 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:24.565+0000 7f8126600700 1 -- 172.21.15.47:0/4197927843 >> 172.21.15.47:0/4197927843 conn(0x7f81200fc660 msgr2=0x7f81200feaa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:24:24.568 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:24.565+0000 7f8126600700 1 -- 172.21.15.47:0/4197927843 shutdown_connections 2024-03-21T10:24:24.568 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:24.565+0000 7f8126600700 1 -- 172.21.15.47:0/4197927843 wait complete. 2024-03-21T10:24:24.570 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:24.567+0000 7f8126600700 1 Processor -- start 2024-03-21T10:24:24.570 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:24.567+0000 7f8126600700 1 -- start start 2024-03-21T10:24:24.570 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:24.567+0000 7f8126600700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8120102c50 0x7f81201983b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:24:24.570 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:24.567+0000 7f8126600700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8120105580 0x7f81201988f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:24:24.570 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:24.567+0000 7f8126600700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8120107eb0 0x7f812019f970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:24:24.570 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:24.567+0000 7f8126600700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f812010dcd0 con 0x7f8120107eb0 2024-03-21T10:24:24.570 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:24.567+0000 7f8126600700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f812010db50 con 0x7f8120102c50 2024-03-21T10:24:24.570 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:24.567+0000 7f8126600700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f812010de50 con 0x7f8120105580 2024-03-21T10:24:24.571 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:24.568+0000 7f811fe00700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8120107eb0 0x7f812019f970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:24:24.571 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:24.568+0000 7f811fe00700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8120107eb0 0x7f812019f970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:55296/0 (socket says 172.21.15.47:55296) 2024-03-21T10:24:24.571 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:24.568+0000 7f811fe00700 1 -- 172.21.15.47:0/1673379125 learned_addr learned my addr 172.21.15.47:0/1673379125 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:24:24.571 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:24.568+0000 7f811f400700 1 --2- 172.21.15.47:0/1673379125 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8120102c50 0x7f81201983b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:24:24.571 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:24.568+0000 7f811ea00700 1 --2- 172.21.15.47:0/1673379125 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8120105580 0x7f81201988f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:24:24.571 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:24.568+0000 7f811fe00700 1 -- 172.21.15.47:0/1673379125 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8120105580 msgr2=0x7f81201988f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:24:24.571 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:24.568+0000 7f811fe00700 1 --2- 172.21.15.47:0/1673379125 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8120105580 0x7f81201988f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:24:24.571 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:24.568+0000 7f811fe00700 1 -- 172.21.15.47:0/1673379125 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8120102c50 msgr2=0x7f81201983b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:24:24.571 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:24.568+0000 7f811fe00700 1 --2- 172.21.15.47:0/1673379125 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8120102c50 0x7f81201983b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:24:24.571 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:24.568+0000 7f811fe00700 1 -- 172.21.15.47:0/1673379125 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81201a0010 con 0x7f8120107eb0 2024-03-21T10:24:24.571 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:24.568+0000 7f811f400700 1 --2- 172.21.15.47:0/1673379125 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8120102c50 0x7f81201983b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T10:24:24.571 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:24.569+0000 7f811fe00700 1 --2- 172.21.15.47:0/1673379125 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8120107eb0 0x7f812019f970 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f811800efc0 tx=0x7f811800c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:24:24.572 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:24.569+0000 7f8117e00700 1 -- 172.21.15.47:0/1673379125 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81180092e0 con 0x7f8120107eb0 2024-03-21T10:24:24.572 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:24.569+0000 7f8126600700 1 -- 172.21.15.47:0/1673379125 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f81201a02a0 con 0x7f8120107eb0 2024-03-21T10:24:24.572 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:24.569+0000 7f8117e00700 1 -- 172.21.15.47:0/1673379125 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f8118004bb0 con 0x7f8120107eb0 2024-03-21T10:24:24.572 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:24.569+0000 7f8117e00700 1 -- 172.21.15.47:0/1673379125 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8118004280 con 0x7f8120107eb0 2024-03-21T10:24:24.572 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:24.570+0000 7f8126600700 1 -- 172.21.15.47:0/1673379125 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8120100a40 con 0x7f8120107eb0 2024-03-21T10:24:24.572 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:24.571+0000 7f8126600700 1 -- 172.21.15.47:0/1673379125 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8120068e50 con 0x7f8120107eb0 2024-03-21T10:24:24.575 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:24.574+0000 7f8117e00700 1 -- 172.21.15.47:0/1673379125 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 18) v1 ==== 137854+0+0 (secure 0 0 0) 0x7f811801b030 con 0x7f8120107eb0 2024-03-21T10:24:24.576 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:24.575+0000 7f8117e00700 1 --2- 172.21.15.47:0/1673379125 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f81040a3030 0x7f81040a54f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:24:24.576 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:24.575+0000 7f8117e00700 1 -- 172.21.15.47:0/1673379125 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9216+0+0 (secure 0 0 0) 0x7f81180cf330 con 0x7f8120107eb0 2024-03-21T10:24:24.576 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:24.575+0000 7f811f400700 1 --2- 172.21.15.47:0/1673379125 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f81040a3030 0x7f81040a54f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:24:24.577 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:24.576+0000 7f811f400700 1 --2- 172.21.15.47:0/1673379125 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f81040a3030 0x7f81040a54f0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f811000b2e0 tx=0x7f811001a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:24:24.581 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:24.580+0000 7f8117e00700 1 -- 172.21.15.47:0/1673379125 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f8118022020 con 0x7f8120107eb0 2024-03-21T10:24:24.822 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:24.820+0000 7f8126600700 1 -- 172.21.15.47:0/1673379125 --> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] -- mgr_command(tid 0: {"prefix": "orch ls", "target": ["mon-mgr", ""]}) v1 -- 0x7f81200625d0 con 0x7f81040a3030 2024-03-21T10:24:24.832 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:24.831+0000 7f8117e00700 1 -- 172.21.15.47:0/1673379125 <== mgr.14152 v2:172.21.15.47:6800/2823885994 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+980 (secure 0 0 0) 0x7f81200625d0 con 0x7f81040a3030 2024-03-21T10:24:24.833 INFO:teuthology.orchestra.run.smithi047.stdout:NAME PORTS RUNNING REFRESHED AGE PLACEMENT 2024-03-21T10:24:24.833 INFO:teuthology.orchestra.run.smithi047.stdout:mds.all 12/12 79s ago 3m smithi047=a;smithi047=d;smithi047=g;smithi047=j;smithi100=b;smithi100=e;smithi100=h;smithi100=k;smithi157=c;smithi157=f;smithi157=i;smithi157=l;count:12 2024-03-21T10:24:24.833 INFO:teuthology.orchestra.run.smithi047.stdout:mgr 3/3 79s ago 10m smithi047=x;smithi100=y;smithi157=z;count:3 2024-03-21T10:24:24.833 INFO:teuthology.orchestra.run.smithi047.stdout:mon 3/3 79s ago 11m smithi047:172.21.15.47=a;smithi100:172.21.15.100=b;smithi157:172.21.15.157=c;count:3 2024-03-21T10:24:24.833 INFO:teuthology.orchestra.run.smithi047.stdout:osd 12 79s ago - 2024-03-21T10:24:24.835 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:24.834+0000 7f8126600700 1 -- 172.21.15.47:0/1673379125 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f81040a3030 msgr2=0x7f81040a54f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:24:24.835 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:24.834+0000 7f8126600700 1 --2- 172.21.15.47:0/1673379125 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f81040a3030 0x7f81040a54f0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f811000b2e0 tx=0x7f811001a040 comp rx=0 tx=0).stop 2024-03-21T10:24:24.835 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:24.834+0000 7f8126600700 1 -- 172.21.15.47:0/1673379125 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8120107eb0 msgr2=0x7f812019f970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:24:24.836 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:24.834+0000 7f8126600700 1 --2- 172.21.15.47:0/1673379125 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8120107eb0 0x7f812019f970 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f811800efc0 tx=0x7f811800c3a0 comp rx=0 tx=0).stop 2024-03-21T10:24:24.836 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:24.834+0000 7f8126600700 1 -- 172.21.15.47:0/1673379125 shutdown_connections 2024-03-21T10:24:24.836 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:24.834+0000 7f8126600700 1 --2- 172.21.15.47:0/1673379125 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f81040a3030 0x7f81040a54f0 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:24:24.836 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:24.834+0000 7f8126600700 1 --2- 172.21.15.47:0/1673379125 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8120107eb0 0x7f812019f970 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:24:24.836 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:24.834+0000 7f8126600700 1 --2- 172.21.15.47:0/1673379125 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8120102c50 0x7f81201983b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:24:24.836 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:24.834+0000 7f8126600700 1 --2- 172.21.15.47:0/1673379125 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8120105580 0x7f81201988f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:24:24.836 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:24.834+0000 7f8126600700 1 -- 172.21.15.47:0/1673379125 >> 172.21.15.47:0/1673379125 conn(0x7f81200fc660 msgr2=0x7f8120104af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:24:24.837 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:24.834+0000 7f8126600700 1 -- 172.21.15.47:0/1673379125 shutdown_connections 2024-03-21T10:24:24.837 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:24.835+0000 7f8126600700 1 -- 172.21.15.47:0/1673379125 wait complete. 2024-03-21T10:24:25.464 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:24:25 smithi047 ceph-mon[31547]: pgmap v444: 129 pgs: 129 active+clean; 642 KiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-03-21T10:24:25.464 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:24:25 smithi047 ceph-mon[31547]: from='client.15294 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:24:25.479 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:24:25 smithi100 ceph-mon[35193]: pgmap v444: 129 pgs: 129 active+clean; 642 KiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-03-21T10:24:25.479 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:24:25 smithi100 ceph-mon[35193]: from='client.15294 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:24:25.489 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:24:25 smithi157 ceph-mon[35581]: pgmap v444: 129 pgs: 129 active+clean; 642 KiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-03-21T10:24:25.490 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:24:25 smithi157 ceph-mon[35581]: from='client.15294 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:24:25.787 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2edfee68ad7243bed422a049b6f5f58619300f11 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3056d51a-e76b-11ee-95cd-87774f69a715 -- bash -c 'ceph orch host ls' 2024-03-21T10:24:26.255 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/3056d51a-e76b-11ee-95cd-87774f69a715/mon.a/config 2024-03-21T10:24:27.461 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:24:27 smithi047 ceph-mon[31547]: pgmap v445: 129 pgs: 129 active+clean; 642 KiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-21T10:24:27.479 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:24:27 smithi100 ceph-mon[35193]: pgmap v445: 129 pgs: 129 active+clean; 642 KiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-21T10:24:27.490 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:24:27 smithi157 ceph-mon[35581]: pgmap v445: 129 pgs: 129 active+clean; 642 KiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-21T10:24:28.740 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:28.738+0000 7f08e3000700 1 -- 172.21.15.47:0/1912854900 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f08dc104410 msgr2=0x7f08dc104830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:24:28.741 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:28.738+0000 7f08e3000700 1 --2- 172.21.15.47:0/1912854900 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f08dc104410 0x7f08dc104830 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f08cc0099c0 tx=0x7f08cc009cd0 comp rx=0 tx=0).stop 2024-03-21T10:24:28.741 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:28.738+0000 7f08e3000700 1 -- 172.21.15.47:0/1912854900 shutdown_connections 2024-03-21T10:24:28.741 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:28.738+0000 7f08e3000700 1 --2- 172.21.15.47:0/1912854900 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f08dc105fb0 0x7f08dc10c860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:24:28.741 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:28.738+0000 7f08e3000700 1 --2- 172.21.15.47:0/1912854900 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f08dc104410 0x7f08dc104830 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:24:28.741 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:28.738+0000 7f08e3000700 1 --2- 172.21.15.47:0/1912854900 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f08dc105610 0x7f08dc105a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:24:28.741 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:28.738+0000 7f08e3000700 1 -- 172.21.15.47:0/1912854900 >> 172.21.15.47:0/1912854900 conn(0x7f08dc0ff9b0 msgr2=0x7f08dc101df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:24:28.741 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:28.738+0000 7f08e3000700 1 -- 172.21.15.47:0/1912854900 shutdown_connections 2024-03-21T10:24:28.741 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:28.739+0000 7f08e3000700 1 -- 172.21.15.47:0/1912854900 wait complete. 2024-03-21T10:24:28.741 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:28.740+0000 7f08e3000700 1 Processor -- start 2024-03-21T10:24:28.742 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:28.740+0000 7f08e3000700 1 -- start start 2024-03-21T10:24:28.743 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:28.740+0000 7f08e3000700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f08dc104410 0x7f08dc19a520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:24:28.743 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:28.740+0000 7f08e3000700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f08dc105610 0x7f08dc19aa60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:24:28.743 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:28.740+0000 7f08e3000700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f08dc105fb0 0x7f08dc1a1ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:24:28.743 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:28.740+0000 7f08e3000700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08dc10fee0 con 0x7f08dc105fb0 2024-03-21T10:24:28.743 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:28.740+0000 7f08e3000700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08dc10fd60 con 0x7f08dc104410 2024-03-21T10:24:28.743 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:28.740+0000 7f08e3000700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08dc110060 con 0x7f08dc105610 2024-03-21T10:24:28.743 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:28.741+0000 7f08e1000700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f08dc105fb0 0x7f08dc1a1ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:24:28.743 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:28.741+0000 7f08dbe00700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f08dc104410 0x7f08dc19a520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:24:28.743 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:28.741+0000 7f08e1000700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f08dc105fb0 0x7f08dc1a1ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:55302/0 (socket says 172.21.15.47:55302) 2024-03-21T10:24:28.743 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:28.741+0000 7f08e1000700 1 -- 172.21.15.47:0/1059173586 learned_addr learned my addr 172.21.15.47:0/1059173586 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:24:28.743 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:28.741+0000 7f08e1000700 1 -- 172.21.15.47:0/1059173586 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f08dc105610 msgr2=0x7f08dc19aa60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:24:28.743 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:28.741+0000 7f08e1000700 1 --2- 172.21.15.47:0/1059173586 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f08dc105610 0x7f08dc19aa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:24:28.743 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:28.741+0000 7f08e1000700 1 -- 172.21.15.47:0/1059173586 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f08dc104410 msgr2=0x7f08dc19a520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:24:28.743 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:28.741+0000 7f08e1000700 1 --2- 172.21.15.47:0/1059173586 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f08dc104410 0x7f08dc19a520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:24:28.744 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:28.741+0000 7f08e1000700 1 -- 172.21.15.47:0/1059173586 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08dc1a2180 con 0x7f08dc105fb0 2024-03-21T10:24:28.744 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:28.741+0000 7f08dbe00700 1 --2- 172.21.15.47:0/1059173586 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f08dc104410 0x7f08dc19a520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T10:24:28.744 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:28.742+0000 7f08e1000700 1 --2- 172.21.15.47:0/1059173586 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f08dc105fb0 0x7f08dc1a1ae0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f08d4007f10 tx=0x7f08d400d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:24:28.744 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:28.742+0000 7f08d8c00700 1 -- 172.21.15.47:0/1059173586 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08d4004bb0 con 0x7f08dc105fb0 2024-03-21T10:24:28.744 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:28.742+0000 7f08d8c00700 1 -- 172.21.15.47:0/1059173586 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f08d4004d50 con 0x7f08dc105fb0 2024-03-21T10:24:28.745 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:28.742+0000 7f08e3000700 1 -- 172.21.15.47:0/1059173586 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f08dc1a2470 con 0x7f08dc105fb0 2024-03-21T10:24:28.745 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:28.742+0000 7f08d8c00700 1 -- 172.21.15.47:0/1059173586 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08d4005990 con 0x7f08dc105fb0 2024-03-21T10:24:28.745 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:28.742+0000 7f08e3000700 1 -- 172.21.15.47:0/1059173586 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f08dc1a29b0 con 0x7f08dc105fb0 2024-03-21T10:24:28.745 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:28.744+0000 7f08e3000700 1 -- 172.21.15.47:0/1059173586 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f08dc068e50 con 0x7f08dc105fb0 2024-03-21T10:24:28.747 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:28.746+0000 7f08d8c00700 1 -- 172.21.15.47:0/1059173586 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 18) v1 ==== 137854+0+0 (secure 0 0 0) 0x7f08d401a020 con 0x7f08dc105fb0 2024-03-21T10:24:28.749 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:28.748+0000 7f08d8c00700 1 --2- 172.21.15.47:0/1059173586 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f08c00a2fb0 0x7f08c00a5470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:24:28.749 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:28.748+0000 7f08d8c00700 1 -- 172.21.15.47:0/1059173586 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9216+0+0 (secure 0 0 0) 0x7f08d40ceef0 con 0x7f08dc105fb0 2024-03-21T10:24:28.749 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:28.748+0000 7f08dbe00700 1 --2- 172.21.15.47:0/1059173586 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f08c00a2fb0 0x7f08c00a5470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:24:28.750 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:28.749+0000 7f08dbe00700 1 --2- 172.21.15.47:0/1059173586 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f08c00a2fb0 0x7f08c00a5470 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f08cc000c00 tx=0x7f08cc005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:24:28.754 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:28.753+0000 7f08d8c00700 1 -- 172.21.15.47:0/1059173586 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f08d4010070 con 0x7f08dc105fb0 2024-03-21T10:24:29.094 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:29.092+0000 7f08e3000700 1 -- 172.21.15.47:0/1059173586 --> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""]}) v1 -- 0x7f08dc0625d0 con 0x7f08c00a2fb0 2024-03-21T10:24:29.096 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:29.095+0000 7f08d8c00700 1 -- 172.21.15.47:0/1059173586 <== mgr.14152 v2:172.21.15.47:6800/2823885994 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+191 (secure 0 0 0) 0x7f08dc0625d0 con 0x7f08c00a2fb0 2024-03-21T10:24:29.097 INFO:teuthology.orchestra.run.smithi047.stdout:HOST ADDR LABELS STATUS 2024-03-21T10:24:29.097 INFO:teuthology.orchestra.run.smithi047.stdout:smithi047 172.21.15.47 2024-03-21T10:24:29.097 INFO:teuthology.orchestra.run.smithi047.stdout:smithi100 172.21.15.100 2024-03-21T10:24:29.097 INFO:teuthology.orchestra.run.smithi047.stdout:smithi157 172.21.15.157 2024-03-21T10:24:29.097 INFO:teuthology.orchestra.run.smithi047.stdout:3 hosts in cluster 2024-03-21T10:24:29.100 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:29.098+0000 7f08e3000700 1 -- 172.21.15.47:0/1059173586 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f08c00a2fb0 msgr2=0x7f08c00a5470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:24:29.100 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:29.098+0000 7f08e3000700 1 --2- 172.21.15.47:0/1059173586 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f08c00a2fb0 0x7f08c00a5470 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f08cc000c00 tx=0x7f08cc005fb0 comp rx=0 tx=0).stop 2024-03-21T10:24:29.100 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:29.098+0000 7f08e3000700 1 -- 172.21.15.47:0/1059173586 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f08dc105fb0 msgr2=0x7f08dc1a1ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:24:29.100 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:29.098+0000 7f08e3000700 1 --2- 172.21.15.47:0/1059173586 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f08dc105fb0 0x7f08dc1a1ae0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f08d4007f10 tx=0x7f08d400d3b0 comp rx=0 tx=0).stop 2024-03-21T10:24:29.100 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:29.098+0000 7f08e3000700 1 -- 172.21.15.47:0/1059173586 shutdown_connections 2024-03-21T10:24:29.100 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:29.098+0000 7f08e3000700 1 --2- 172.21.15.47:0/1059173586 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f08c00a2fb0 0x7f08c00a5470 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:24:29.100 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:29.098+0000 7f08e3000700 1 --2- 172.21.15.47:0/1059173586 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f08dc105fb0 0x7f08dc1a1ae0 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:24:29.100 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:29.098+0000 7f08e3000700 1 --2- 172.21.15.47:0/1059173586 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f08dc104410 0x7f08dc19a520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:24:29.100 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:29.098+0000 7f08e3000700 1 --2- 172.21.15.47:0/1059173586 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f08dc105610 0x7f08dc19aa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:24:29.100 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:29.098+0000 7f08e3000700 1 -- 172.21.15.47:0/1059173586 >> 172.21.15.47:0/1059173586 conn(0x7f08dc0ff9b0 msgr2=0x7f08dc108910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:24:29.100 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:29.099+0000 7f08e3000700 1 -- 172.21.15.47:0/1059173586 shutdown_connections 2024-03-21T10:24:29.101 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:29.099+0000 7f08e3000700 1 -- 172.21.15.47:0/1059173586 wait complete. 2024-03-21T10:24:29.464 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:24:29 smithi047 ceph-mon[31547]: pgmap v446: 129 pgs: 129 active+clean; 642 KiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-03-21T10:24:29.479 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:24:29 smithi100 ceph-mon[35193]: pgmap v446: 129 pgs: 129 active+clean; 642 KiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-03-21T10:24:29.490 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:24:29 smithi157 ceph-mon[35581]: pgmap v446: 129 pgs: 129 active+clean; 642 KiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-03-21T10:24:30.058 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2edfee68ad7243bed422a049b6f5f58619300f11 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3056d51a-e76b-11ee-95cd-87774f69a715 -- bash -c 'ceph orch device ls' 2024-03-21T10:24:30.463 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:24:30 smithi047 ceph-mon[31547]: from='client.15300 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:24:30.479 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:24:30 smithi100 ceph-mon[35193]: from='client.15300 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:24:30.490 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:24:30 smithi157 ceph-mon[35581]: from='client.15300 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:24:30.533 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/3056d51a-e76b-11ee-95cd-87774f69a715/mon.a/config 2024-03-21T10:24:31.464 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:24:31 smithi047 ceph-mon[31547]: pgmap v447: 129 pgs: 129 active+clean; 642 KiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-03-21T10:24:31.479 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:24:31 smithi100 ceph-mon[35193]: pgmap v447: 129 pgs: 129 active+clean; 642 KiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-03-21T10:24:31.490 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:24:31 smithi157 ceph-mon[35581]: pgmap v447: 129 pgs: 129 active+clean; 642 KiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-03-21T10:24:32.874 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:32.873+0000 7f0237400700 1 -- 172.21.15.47:0/804778397 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0230105580 msgr2=0x7f0230107970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:24:32.875 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:32.873+0000 7f0237400700 1 --2- 172.21.15.47:0/804778397 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0230105580 0x7f0230107970 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f021c009990 tx=0x7f021c009ca0 comp rx=0 tx=0).stop 2024-03-21T10:24:32.875 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:32.873+0000 7f0237400700 1 -- 172.21.15.47:0/804778397 shutdown_connections 2024-03-21T10:24:32.875 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:32.873+0000 7f0237400700 1 --2- 172.21.15.47:0/804778397 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0230107eb0 0x7f023010a320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:24:32.875 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:32.873+0000 7f0237400700 1 --2- 172.21.15.47:0/804778397 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0230102c50 0x7f0230105040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:24:32.875 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:32.873+0000 7f0237400700 1 --2- 172.21.15.47:0/804778397 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0230105580 0x7f0230107970 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:24:32.875 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:32.873+0000 7f0237400700 1 -- 172.21.15.47:0/804778397 >> 172.21.15.47:0/804778397 conn(0x7f02300fc660 msgr2=0x7f02300feaa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:24:32.875 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:32.873+0000 7f0237400700 1 -- 172.21.15.47:0/804778397 shutdown_connections 2024-03-21T10:24:32.875 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:32.873+0000 7f0237400700 1 -- 172.21.15.47:0/804778397 wait complete. 2024-03-21T10:24:32.876 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:32.874+0000 7f0237400700 1 Processor -- start 2024-03-21T10:24:32.876 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:32.874+0000 7f0237400700 1 -- start start 2024-03-21T10:24:32.876 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:32.875+0000 7f0237400700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0230102c50 0x7f023019a5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:24:32.876 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:32.875+0000 7f0237400700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0230105580 0x7f023019ab20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:24:32.876 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:32.875+0000 7f0237400700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0230107eb0 0x7f02301a1ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:24:32.876 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:32.875+0000 7f0237400700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f023010dcd0 con 0x7f0230102c50 2024-03-21T10:24:32.876 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:32.875+0000 7f0237400700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f023010db50 con 0x7f0230105580 2024-03-21T10:24:32.876 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:32.875+0000 7f0237400700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f023010de50 con 0x7f0230107eb0 2024-03-21T10:24:32.877 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:32.875+0000 7f022fe00700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0230105580 0x7f023019ab20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:24:32.877 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:32.875+0000 7f022fe00700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0230105580 0x7f023019ab20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:35206/0 (socket says 172.21.15.47:35206) 2024-03-21T10:24:32.878 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:32.875+0000 7f022fe00700 1 -- 172.21.15.47:0/917309313 learned_addr learned my addr 172.21.15.47:0/917309313 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:24:32.878 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:32.875+0000 7f0234a00700 1 --2- 172.21.15.47:0/917309313 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0230102c50 0x7f023019a5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:24:32.878 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:32.875+0000 7f0235400700 1 --2- 172.21.15.47:0/917309313 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0230107eb0 0x7f02301a1ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:24:32.878 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:32.876+0000 7f022fe00700 1 -- 172.21.15.47:0/917309313 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0230107eb0 msgr2=0x7f02301a1ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:24:32.878 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:32.876+0000 7f022fe00700 1 --2- 172.21.15.47:0/917309313 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0230107eb0 0x7f02301a1ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:24:32.878 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:32.876+0000 7f022fe00700 1 -- 172.21.15.47:0/917309313 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0230102c50 msgr2=0x7f023019a5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:24:32.878 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:32.876+0000 7f022fe00700 1 --2- 172.21.15.47:0/917309313 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0230102c50 0x7f023019a5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:24:32.878 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:32.876+0000 7f022fe00700 1 -- 172.21.15.47:0/917309313 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f02301a2240 con 0x7f0230105580 2024-03-21T10:24:32.878 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:32.876+0000 7f022fe00700 1 --2- 172.21.15.47:0/917309313 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0230105580 0x7f023019ab20 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f021c0098e0 tx=0x7f021c0057c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:24:32.879 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:32.877+0000 7f022d600700 1 -- 172.21.15.47:0/917309313 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f021c027070 con 0x7f0230105580 2024-03-21T10:24:32.879 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:32.877+0000 7f0237400700 1 -- 172.21.15.47:0/917309313 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f02301a24d0 con 0x7f0230105580 2024-03-21T10:24:32.879 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:32.877+0000 7f0237400700 1 -- 172.21.15.47:0/917309313 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f02301a2980 con 0x7f0230105580 2024-03-21T10:24:32.879 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:32.877+0000 7f022d600700 1 -- 172.21.15.47:0/917309313 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f021c004010 con 0x7f0230105580 2024-03-21T10:24:32.879 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:32.877+0000 7f022d600700 1 -- 172.21.15.47:0/917309313 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f021c022d60 con 0x7f0230105580 2024-03-21T10:24:32.879 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:32.878+0000 7f0237400700 1 -- 172.21.15.47:0/917309313 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0210005330 con 0x7f0230105580 2024-03-21T10:24:32.882 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:32.881+0000 7f022d600700 1 -- 172.21.15.47:0/917309313 <== mon.2 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 18) v1 ==== 137854+0+0 (secure 0 0 0) 0x7f021c02a070 con 0x7f0230105580 2024-03-21T10:24:32.884 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:32.882+0000 7f022d600700 1 --2- 172.21.15.47:0/917309313 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f02180a2fb0 0x7f02180a5470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:24:32.884 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:32.883+0000 7f022d600700 1 -- 172.21.15.47:0/917309313 <== mon.2 v2:172.21.15.100:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9216+0+0 (secure 0 0 0) 0x7f021c0d0c10 con 0x7f0230105580 2024-03-21T10:24:32.884 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:32.883+0000 7f0234a00700 1 --2- 172.21.15.47:0/917309313 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f02180a2fb0 0x7f02180a5470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:24:32.885 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:32.884+0000 7f0234a00700 1 --2- 172.21.15.47:0/917309313 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f02180a2fb0 0x7f02180a5470 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f0224004630 tx=0x7f022400b3c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:24:32.896 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:32.895+0000 7f022d600700 1 -- 172.21.15.47:0/917309313 <== mon.2 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f021c031020 con 0x7f0230105580 2024-03-21T10:24:33.140 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:33.138+0000 7f0237400700 1 -- 172.21.15.47:0/917309313 --> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] -- mgr_command(tid 0: {"prefix": "orch device ls", "target": ["mon-mgr", ""]}) v1 -- 0x7f0210000bc0 con 0x7f02180a2fb0 2024-03-21T10:24:33.144 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:33.143+0000 7f022d600700 1 -- 172.21.15.47:0/917309313 <== mgr.14152 v2:172.21.15.47:6800/2823885994 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+532 (secure 0 0 0) 0x7f0210000bc0 con 0x7f02180a2fb0 2024-03-21T10:24:33.144 INFO:teuthology.orchestra.run.smithi047.stdout:HOST PATH TYPE DEVICE ID SIZE AVAILABLE REFRESHED REJECT REASONS 2024-03-21T10:24:33.145 INFO:teuthology.orchestra.run.smithi047.stdout:smithi047 /dev/nvme0n1 ssd INTEL_SSDPED1D480GA_PHMB75130022480DGN 447G No 4m ago Has a FileSystem, LVM detected 2024-03-21T10:24:33.145 INFO:teuthology.orchestra.run.smithi047.stdout:smithi100 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_PHFT620400WR400BGN 372G No 4m ago Has a FileSystem, LVM detected 2024-03-21T10:24:33.145 INFO:teuthology.orchestra.run.smithi047.stdout:smithi157 /dev/nvme0n1 ssd INTEL_SSDPED1D480GA_PHMB7476002V480DGN 447G No 3m ago Has a FileSystem, LVM detected 2024-03-21T10:24:33.148 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:33.147+0000 7f0237400700 1 -- 172.21.15.47:0/917309313 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f02180a2fb0 msgr2=0x7f02180a5470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:24:33.148 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:33.147+0000 7f0237400700 1 --2- 172.21.15.47:0/917309313 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f02180a2fb0 0x7f02180a5470 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f0224004630 tx=0x7f022400b3c0 comp rx=0 tx=0).stop 2024-03-21T10:24:33.148 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:33.147+0000 7f0237400700 1 -- 172.21.15.47:0/917309313 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0230105580 msgr2=0x7f023019ab20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:24:33.149 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:33.147+0000 7f0237400700 1 --2- 172.21.15.47:0/917309313 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0230105580 0x7f023019ab20 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f021c0098e0 tx=0x7f021c0057c0 comp rx=0 tx=0).stop 2024-03-21T10:24:33.149 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:33.147+0000 7f0237400700 1 -- 172.21.15.47:0/917309313 shutdown_connections 2024-03-21T10:24:33.149 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:33.147+0000 7f0237400700 1 --2- 172.21.15.47:0/917309313 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f02180a2fb0 0x7f02180a5470 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:24:33.149 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:33.147+0000 7f0237400700 1 --2- 172.21.15.47:0/917309313 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0230107eb0 0x7f02301a1ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:24:33.149 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:33.147+0000 7f0237400700 1 --2- 172.21.15.47:0/917309313 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0230102c50 0x7f023019a5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:24:33.149 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:33.147+0000 7f0237400700 1 --2- 172.21.15.47:0/917309313 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0230105580 0x7f023019ab20 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:24:33.149 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:33.147+0000 7f0237400700 1 -- 172.21.15.47:0/917309313 >> 172.21.15.47:0/917309313 conn(0x7f02300fc660 msgr2=0x7f0230102540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:24:33.149 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:33.147+0000 7f0237400700 1 -- 172.21.15.47:0/917309313 shutdown_connections 2024-03-21T10:24:33.149 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:33.147+0000 7f0237400700 1 -- 172.21.15.47:0/917309313 wait complete. 2024-03-21T10:24:33.464 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:24:33 smithi047 ceph-mon[31547]: pgmap v448: 129 pgs: 129 active+clean; 642 KiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-21T10:24:33.479 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:24:33 smithi100 ceph-mon[35193]: pgmap v448: 129 pgs: 129 active+clean; 642 KiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-21T10:24:33.490 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:24:33 smithi157 ceph-mon[35581]: pgmap v448: 129 pgs: 129 active+clean; 642 KiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-21T10:24:34.211 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-03-21T10:24:34.224 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi047.front.sepia.ceph.com 2024-03-21T10:24:34.224 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2edfee68ad7243bed422a049b6f5f58619300f11 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3056d51a-e76b-11ee-95cd-87774f69a715 -- bash -c 'ceph fs dump' 2024-03-21T10:24:34.404 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:24:34 smithi047 ceph-mon[31547]: from='client.35150 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:24:34.479 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:24:34 smithi100 ceph-mon[35193]: from='client.35150 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:24:34.490 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:24:34 smithi157 ceph-mon[35581]: from='client.35150 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:24:34.691 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/3056d51a-e76b-11ee-95cd-87774f69a715/mon.a/config 2024-03-21T10:24:35.464 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:24:35 smithi047 ceph-mon[31547]: pgmap v449: 129 pgs: 129 active+clean; 642 KiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-03-21T10:24:35.479 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:24:35 smithi100 ceph-mon[35193]: pgmap v449: 129 pgs: 129 active+clean; 642 KiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-03-21T10:24:35.490 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:24:35 smithi157 ceph-mon[35581]: pgmap v449: 129 pgs: 129 active+clean; 642 KiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-03-21T10:24:37.095 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:37.093+0000 7f771ae00700 1 -- 172.21.15.47:0/2077924852 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7714107eb0 msgr2=0x7f771410a320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:24:37.096 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:37.093+0000 7f771ae00700 1 --2- 172.21.15.47:0/2077924852 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7714107eb0 0x7f771410a320 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f770c00b0a0 tx=0x7f770c00b3b0 comp rx=0 tx=0).stop 2024-03-21T10:24:37.096 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:37.094+0000 7f771ae00700 1 -- 172.21.15.47:0/2077924852 shutdown_connections 2024-03-21T10:24:37.096 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:37.094+0000 7f771ae00700 1 --2- 172.21.15.47:0/2077924852 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7714107eb0 0x7f771410a320 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:24:37.096 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:37.094+0000 7f771ae00700 1 --2- 172.21.15.47:0/2077924852 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7714102c50 0x7f7714105040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:24:37.096 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:37.094+0000 7f771ae00700 1 --2- 172.21.15.47:0/2077924852 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7714105580 0x7f7714107970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:24:37.096 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:37.094+0000 7f771ae00700 1 -- 172.21.15.47:0/2077924852 >> 172.21.15.47:0/2077924852 conn(0x7f77140fc660 msgr2=0x7f77140feaa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:24:37.096 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:37.094+0000 7f771ae00700 1 -- 172.21.15.47:0/2077924852 shutdown_connections 2024-03-21T10:24:37.096 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:37.094+0000 7f771ae00700 1 -- 172.21.15.47:0/2077924852 wait complete. 2024-03-21T10:24:37.097 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:37.095+0000 7f771ae00700 1 Processor -- start 2024-03-21T10:24:37.097 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:37.095+0000 7f771ae00700 1 -- start start 2024-03-21T10:24:37.097 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:37.096+0000 7f771ae00700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7714102c50 0x7f771419a520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:24:37.097 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:37.096+0000 7f771ae00700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7714105580 0x7f771419aa60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:24:37.097 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:37.096+0000 7f771ae00700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7714107eb0 0x7f77141a1ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:24:37.097 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:37.096+0000 7f771ae00700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f771410dca0 con 0x7f7714102c50 2024-03-21T10:24:37.097 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:37.096+0000 7f771ae00700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f771410db20 con 0x7f7714105580 2024-03-21T10:24:37.097 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:37.096+0000 7f771ae00700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f771410de20 con 0x7f7714107eb0 2024-03-21T10:24:37.099 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:37.096+0000 7f7713e00700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7714102c50 0x7f771419a520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:24:37.099 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:37.096+0000 7f7713e00700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7714102c50 0x7f771419a520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:35260/0 (socket says 172.21.15.47:35260) 2024-03-21T10:24:37.099 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:37.096+0000 7f7713e00700 1 -- 172.21.15.47:0/1055250429 learned_addr learned my addr 172.21.15.47:0/1055250429 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:24:37.099 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:37.096+0000 7f7713400700 1 --2- 172.21.15.47:0/1055250429 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7714105580 0x7f771419aa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:24:37.099 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:37.097+0000 7f7718e00700 1 --2- 172.21.15.47:0/1055250429 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7714107eb0 0x7f77141a1ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:24:37.099 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:37.097+0000 7f7713e00700 1 -- 172.21.15.47:0/1055250429 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7714107eb0 msgr2=0x7f77141a1ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:24:37.099 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:37.097+0000 7f7713e00700 1 --2- 172.21.15.47:0/1055250429 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7714107eb0 0x7f77141a1ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:24:37.099 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:37.097+0000 7f7713e00700 1 -- 172.21.15.47:0/1055250429 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7714105580 msgr2=0x7f771419aa60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:24:37.099 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:37.097+0000 7f7713e00700 1 --2- 172.21.15.47:0/1055250429 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7714105580 0x7f771419aa60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:24:37.099 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:37.097+0000 7f7713e00700 1 -- 172.21.15.47:0/1055250429 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77141a2180 con 0x7f7714102c50 2024-03-21T10:24:37.099 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:37.097+0000 7f7718e00700 1 --2- 172.21.15.47:0/1055250429 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7714107eb0 0x7f77141a1ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T10:24:37.100 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:37.097+0000 7f7713400700 1 --2- 172.21.15.47:0/1055250429 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7714105580 0x7f771419aa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T10:24:37.100 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:37.097+0000 7f7713e00700 1 --2- 172.21.15.47:0/1055250429 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7714102c50 0x7f771419a520 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f770400d9b0 tx=0x7f770400dd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:24:37.100 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:37.098+0000 7f7710c00700 1 -- 172.21.15.47:0/1055250429 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7704017070 con 0x7f7714102c50 2024-03-21T10:24:37.100 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:37.098+0000 7f7710c00700 1 -- 172.21.15.47:0/1055250429 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f7704004830 con 0x7f7714102c50 2024-03-21T10:24:37.100 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:37.098+0000 7f771ae00700 1 -- 172.21.15.47:0/1055250429 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f77141a2470 con 0x7f7714102c50 2024-03-21T10:24:37.100 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:37.098+0000 7f7710c00700 1 -- 172.21.15.47:0/1055250429 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77040053a0 con 0x7f7714102c50 2024-03-21T10:24:37.100 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:37.098+0000 7f771ae00700 1 -- 172.21.15.47:0/1055250429 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f77141a2960 con 0x7f7714102c50 2024-03-21T10:24:37.100 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:37.099+0000 7f771ae00700 1 -- 172.21.15.47:0/1055250429 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7714068e50 con 0x7f7714102c50 2024-03-21T10:24:37.102 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:37.101+0000 7f7710c00700 1 -- 172.21.15.47:0/1055250429 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 18) v1 ==== 137854+0+0 (secure 0 0 0) 0x7f77040049e0 con 0x7f7714102c50 2024-03-21T10:24:37.104 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:37.104+0000 7f7710c00700 1 --2- 172.21.15.47:0/1055250429 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f76f80a2fb0 0x7f76f80a5470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:24:37.105 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:37.104+0000 7f7710c00700 1 -- 172.21.15.47:0/1055250429 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9216+0+0 (secure 0 0 0) 0x7f77040cf3c0 con 0x7f7714102c50 2024-03-21T10:24:37.105 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:37.104+0000 7f7713400700 1 --2- 172.21.15.47:0/1055250429 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f76f80a2fb0 0x7f76f80a5470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:24:37.105 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:37.105+0000 7f7713400700 1 --2- 172.21.15.47:0/1055250429 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f76f80a2fb0 0x7f76f80a5470 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f7700009cb0 tx=0x7f7700009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:24:37.108 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:37.107+0000 7f7710c00700 1 -- 172.21.15.47:0/1055250429 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f7704020020 con 0x7f7714102c50 2024-03-21T10:24:37.434 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:37.433+0000 7f771ae00700 1 -- 172.21.15.47:0/1055250429 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f7714062da0 con 0x7f7714102c50 2024-03-21T10:24:37.434 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:24:37 smithi047 ceph-mon[31547]: pgmap v450: 129 pgs: 129 active+clean; 642 KiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-03-21T10:24:37.435 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:37.434+0000 7f7710c00700 1 -- 172.21.15.47:0/1055250429 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 29 v29) v1 ==== 76+0+3046 (secure 0 0 0) 0x7f770409bde0 con 0x7f7714102c50 2024-03-21T10:24:37.436 INFO:teuthology.orchestra.run.smithi047.stdout:e29 2024-03-21T10:24:37.436 INFO:teuthology.orchestra.run.smithi047.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-03-21T10:24:37.436 INFO:teuthology.orchestra.run.smithi047.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments} 2024-03-21T10:24:37.436 INFO:teuthology.orchestra.run.smithi047.stdout:legacy client fscid: 1 2024-03-21T10:24:37.436 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-21T10:24:37.436 INFO:teuthology.orchestra.run.smithi047.stdout:Filesystem 'cephfs' (1) 2024-03-21T10:24:37.436 INFO:teuthology.orchestra.run.smithi047.stdout:fs_name cephfs 2024-03-21T10:24:37.437 INFO:teuthology.orchestra.run.smithi047.stdout:epoch 29 2024-03-21T10:24:37.437 INFO:teuthology.orchestra.run.smithi047.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-03-21T10:24:37.437 INFO:teuthology.orchestra.run.smithi047.stdout:created 2024-03-21T10:21:58.329063+0000 2024-03-21T10:24:37.437 INFO:teuthology.orchestra.run.smithi047.stdout:modified 2024-03-21T10:22:29.588509+0000 2024-03-21T10:24:37.437 INFO:teuthology.orchestra.run.smithi047.stdout:tableserver 0 2024-03-21T10:24:37.437 INFO:teuthology.orchestra.run.smithi047.stdout:root 0 2024-03-21T10:24:37.437 INFO:teuthology.orchestra.run.smithi047.stdout:session_timeout 300 2024-03-21T10:24:37.437 INFO:teuthology.orchestra.run.smithi047.stdout:session_autoclose 300 2024-03-21T10:24:37.437 INFO:teuthology.orchestra.run.smithi047.stdout:max_file_size 1099511627776 2024-03-21T10:24:37.437 INFO:teuthology.orchestra.run.smithi047.stdout:max_xattr_size 65536 2024-03-21T10:24:37.437 INFO:teuthology.orchestra.run.smithi047.stdout:required_client_features {} 2024-03-21T10:24:37.437 INFO:teuthology.orchestra.run.smithi047.stdout:last_failure 0 2024-03-21T10:24:37.437 INFO:teuthology.orchestra.run.smithi047.stdout:last_failure_osd_epoch 0 2024-03-21T10:24:37.438 INFO:teuthology.orchestra.run.smithi047.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments} 2024-03-21T10:24:37.438 INFO:teuthology.orchestra.run.smithi047.stdout:max_mds 5 2024-03-21T10:24:37.438 INFO:teuthology.orchestra.run.smithi047.stdout:in 0,1,2,3,4 2024-03-21T10:24:37.438 INFO:teuthology.orchestra.run.smithi047.stdout:up {0=34535,1=34493,2=34454,3=34427,4=14619} 2024-03-21T10:24:37.438 INFO:teuthology.orchestra.run.smithi047.stdout:failed 2024-03-21T10:24:37.438 INFO:teuthology.orchestra.run.smithi047.stdout:damaged 2024-03-21T10:24:37.438 INFO:teuthology.orchestra.run.smithi047.stdout:stopped 2024-03-21T10:24:37.438 INFO:teuthology.orchestra.run.smithi047.stdout:data_pools [3,4] 2024-03-21T10:24:37.438 INFO:teuthology.orchestra.run.smithi047.stdout:metadata_pool 2 2024-03-21T10:24:37.438 INFO:teuthology.orchestra.run.smithi047.stdout:inline_data disabled 2024-03-21T10:24:37.438 INFO:teuthology.orchestra.run.smithi047.stdout:balancer 2024-03-21T10:24:37.438 INFO:teuthology.orchestra.run.smithi047.stdout:bal_rank_mask -1 2024-03-21T10:24:37.438 INFO:teuthology.orchestra.run.smithi047.stdout:standby_count_wanted 0 2024-03-21T10:24:37.438 INFO:teuthology.orchestra.run.smithi047.stdout:qdb_cluster leader: 34535 members: 34493,34535,34454,34427,14619 2024-03-21T10:24:37.439 INFO:teuthology.orchestra.run.smithi047.stdout:[mds.b{0:34535} state up:active seq 3 addr [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] compat {c=[1],r=[1],i=[fff]}] 2024-03-21T10:24:37.439 INFO:teuthology.orchestra.run.smithi047.stdout:[mds.i{0:24499} state up:standby-replay seq 1 addr [v2:172.21.15.157:6838/4064242876,v1:172.21.15.157:6839/4064242876] compat {c=[1],r=[1],i=[fff]}] 2024-03-21T10:24:37.439 INFO:teuthology.orchestra.run.smithi047.stdout:[mds.h{1:34493} state up:active seq 13 addr [v2:172.21.15.100:6836/3497644162,v1:172.21.15.100:6837/3497644162] compat {c=[1],r=[1],i=[fff]}] 2024-03-21T10:24:37.439 INFO:teuthology.orchestra.run.smithi047.stdout:[mds.c{1:24475} state up:standby-replay seq 1 addr [v2:172.21.15.157:6836/1908087204,v1:172.21.15.157:6837/1908087204] compat {c=[1],r=[1],i=[fff]}] 2024-03-21T10:24:37.439 INFO:teuthology.orchestra.run.smithi047.stdout:[mds.e{2:34454} state up:active seq 17 addr [v2:172.21.15.100:6834/2597849957,v1:172.21.15.100:6835/2597849957] compat {c=[1],r=[1],i=[fff]}] 2024-03-21T10:24:37.439 INFO:teuthology.orchestra.run.smithi047.stdout:[mds.f{2:24457} state up:standby-replay seq 1 addr [v2:172.21.15.157:6834/3902868651,v1:172.21.15.157:6835/3902868651] compat {c=[1],r=[1],i=[fff]}] 2024-03-21T10:24:37.439 INFO:teuthology.orchestra.run.smithi047.stdout:[mds.k{3:34427} state up:active seq 20 addr [v2:172.21.15.100:6832/2723302038,v1:172.21.15.100:6833/2723302038] compat {c=[1],r=[1],i=[fff]}] 2024-03-21T10:24:37.439 INFO:teuthology.orchestra.run.smithi047.stdout:[mds.l{3:24406} state up:standby-replay seq 1 addr [v2:172.21.15.157:6832/4000010136,v1:172.21.15.157:6833/4000010136] compat {c=[1],r=[1],i=[fff]}] 2024-03-21T10:24:37.439 INFO:teuthology.orchestra.run.smithi047.stdout:[mds.a{4:14619} state up:active seq 11 addr [v2:172.21.15.47:6840/3401578978,v1:172.21.15.47:6841/3401578978] compat {c=[1],r=[1],i=[fff]}] 2024-03-21T10:24:37.439 INFO:teuthology.orchestra.run.smithi047.stdout:[mds.g{4:14604} state up:standby-replay seq 1 addr [v2:172.21.15.47:6838/2881335356,v1:172.21.15.47:6839/2881335356] compat {c=[1],r=[1],i=[fff]}] 2024-03-21T10:24:37.439 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-21T10:24:37.439 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-21T10:24:37.439 INFO:teuthology.orchestra.run.smithi047.stdout:Standby daemons: 2024-03-21T10:24:37.439 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-21T10:24:37.439 INFO:teuthology.orchestra.run.smithi047.stdout:[mds.j{-1:14511} state up:standby seq 1 addr [v2:172.21.15.47:6834/735724037,v1:172.21.15.47:6835/735724037] compat {c=[1],r=[1],i=[fff]}] 2024-03-21T10:24:37.440 INFO:teuthology.orchestra.run.smithi047.stdout:[mds.d{-1:14529} state up:standby seq 1 addr [v2:172.21.15.47:6836/2248191724,v1:172.21.15.47:6837/2248191724] compat {c=[1],r=[1],i=[fff]}] 2024-03-21T10:24:37.440 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:37.437+0000 7f771ae00700 1 -- 172.21.15.47:0/1055250429 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f76f80a2fb0 msgr2=0x7f76f80a5470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:24:37.440 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:37.437+0000 7f771ae00700 1 --2- 172.21.15.47:0/1055250429 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f76f80a2fb0 0x7f76f80a5470 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f7700009cb0 tx=0x7f7700009260 comp rx=0 tx=0).stop 2024-03-21T10:24:37.441 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:37.437+0000 7f771ae00700 1 -- 172.21.15.47:0/1055250429 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7714102c50 msgr2=0x7f771419a520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:24:37.441 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:37.437+0000 7f771ae00700 1 --2- 172.21.15.47:0/1055250429 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7714102c50 0x7f771419a520 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f770400d9b0 tx=0x7f770400dd70 comp rx=0 tx=0).stop 2024-03-21T10:24:37.441 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:37.437+0000 7f771ae00700 1 -- 172.21.15.47:0/1055250429 shutdown_connections 2024-03-21T10:24:37.441 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:37.437+0000 7f771ae00700 1 --2- 172.21.15.47:0/1055250429 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f76f80a2fb0 0x7f76f80a5470 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:24:37.441 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:37.437+0000 7f771ae00700 1 --2- 172.21.15.47:0/1055250429 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7714107eb0 0x7f77141a1ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:24:37.441 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:37.437+0000 7f771ae00700 1 --2- 172.21.15.47:0/1055250429 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7714102c50 0x7f771419a520 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:24:37.441 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:37.437+0000 7f771ae00700 1 --2- 172.21.15.47:0/1055250429 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7714105580 0x7f771419aa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:24:37.441 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:37.437+0000 7f771ae00700 1 -- 172.21.15.47:0/1055250429 >> 172.21.15.47:0/1055250429 conn(0x7f77140fc660 msgr2=0x7f7714102330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:24:37.441 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:37.438+0000 7f771ae00700 1 -- 172.21.15.47:0/1055250429 shutdown_connections 2024-03-21T10:24:37.441 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:37.438+0000 7f771ae00700 1 -- 172.21.15.47:0/1055250429 wait complete. 2024-03-21T10:24:37.442 INFO:teuthology.orchestra.run.smithi047.stderr:dumped fsmap epoch 29 2024-03-21T10:24:37.479 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:24:37 smithi100 ceph-mon[35193]: pgmap v450: 129 pgs: 129 active+clean; 642 KiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-03-21T10:24:37.489 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:24:37 smithi157 ceph-mon[35581]: pgmap v450: 129 pgs: 129 active+clean; 642 KiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-03-21T10:24:38.356 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2edfee68ad7243bed422a049b6f5f58619300f11 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3056d51a-e76b-11ee-95cd-87774f69a715 -- bash -c 'ceph osd dump' 2024-03-21T10:24:38.464 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:24:38 smithi047 ceph-mon[31547]: from='client.? 172.21.15.47:0/1055250429' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-21T10:24:38.479 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:24:38 smithi100 ceph-mon[35193]: from='client.? 172.21.15.47:0/1055250429' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-21T10:24:38.489 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:24:38 smithi157 ceph-mon[35581]: from='client.? 172.21.15.47:0/1055250429' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-21T10:24:38.839 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/3056d51a-e76b-11ee-95cd-87774f69a715/mon.a/config 2024-03-21T10:24:39.352 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:24:39 smithi047 ceph-mon[31547]: pgmap v451: 129 pgs: 129 active+clean; 642 KiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 9 op/s 2024-03-21T10:24:39.479 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:24:39 smithi100 ceph-mon[35193]: pgmap v451: 129 pgs: 129 active+clean; 642 KiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 9 op/s 2024-03-21T10:24:39.489 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:24:39 smithi157 ceph-mon[35581]: pgmap v451: 129 pgs: 129 active+clean; 642 KiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 9 op/s 2024-03-21T10:24:41.105 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:41.104+0000 7f0247400700 1 -- 172.21.15.47:0/2210729781 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0240102c50 msgr2=0x7f0240105040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:24:41.105 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:41.104+0000 7f0247400700 1 --2- 172.21.15.47:0/2210729781 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0240102c50 0x7f0240105040 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f0234009990 tx=0x7f0234009ca0 comp rx=0 tx=0).stop 2024-03-21T10:24:41.106 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:41.104+0000 7f0247400700 1 -- 172.21.15.47:0/2210729781 shutdown_connections 2024-03-21T10:24:41.106 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:41.104+0000 7f0247400700 1 --2- 172.21.15.47:0/2210729781 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0240107eb0 0x7f024010a320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:24:41.106 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:41.104+0000 7f0247400700 1 --2- 172.21.15.47:0/2210729781 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0240102c50 0x7f0240105040 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:24:41.106 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:41.104+0000 7f0247400700 1 --2- 172.21.15.47:0/2210729781 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0240105580 0x7f0240107970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:24:41.106 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:41.104+0000 7f0247400700 1 -- 172.21.15.47:0/2210729781 >> 172.21.15.47:0/2210729781 conn(0x7f02400fc660 msgr2=0x7f02400feaa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:24:41.106 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:41.104+0000 7f0247400700 1 -- 172.21.15.47:0/2210729781 shutdown_connections 2024-03-21T10:24:41.106 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:41.104+0000 7f0247400700 1 -- 172.21.15.47:0/2210729781 wait complete. 2024-03-21T10:24:41.107 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:41.105+0000 7f0247400700 1 Processor -- start 2024-03-21T10:24:41.107 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:41.105+0000 7f0247400700 1 -- start start 2024-03-21T10:24:41.107 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:41.106+0000 7f0247400700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0240102c50 0x7f024019a5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:24:41.107 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:41.106+0000 7f0247400700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0240105580 0x7f024019aaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:24:41.107 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:41.106+0000 7f0247400700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0240107eb0 0x7f02401a1b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:24:41.108 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:41.106+0000 7f0247400700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f024010dce0 con 0x7f0240105580 2024-03-21T10:24:41.108 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:41.106+0000 7f0247400700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f024010db60 con 0x7f0240107eb0 2024-03-21T10:24:41.108 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:41.106+0000 7f0247400700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f024010de60 con 0x7f0240102c50 2024-03-21T10:24:41.108 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:41.106+0000 7f023fe00700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0240105580 0x7f024019aaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:24:41.108 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:41.106+0000 7f023fe00700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0240105580 0x7f024019aaf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:35284/0 (socket says 172.21.15.47:35284) 2024-03-21T10:24:41.108 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:41.106+0000 7f023fe00700 1 -- 172.21.15.47:0/2415018052 learned_addr learned my addr 172.21.15.47:0/2415018052 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:24:41.108 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:41.106+0000 7f0245400700 1 --2- 172.21.15.47:0/2415018052 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0240107eb0 0x7f02401a1b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:24:41.108 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:41.106+0000 7f0244a00700 1 --2- 172.21.15.47:0/2415018052 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0240102c50 0x7f024019a5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:24:41.109 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:41.107+0000 7f023fe00700 1 -- 172.21.15.47:0/2415018052 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0240102c50 msgr2=0x7f024019a5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:24:41.109 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:41.107+0000 7f023fe00700 1 --2- 172.21.15.47:0/2415018052 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0240102c50 0x7f024019a5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:24:41.109 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:41.107+0000 7f023fe00700 1 -- 172.21.15.47:0/2415018052 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0240107eb0 msgr2=0x7f02401a1b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:24:41.109 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:41.107+0000 7f023fe00700 1 --2- 172.21.15.47:0/2415018052 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0240107eb0 0x7f02401a1b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:24:41.109 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:41.107+0000 7f023fe00700 1 -- 172.21.15.47:0/2415018052 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f02401a2210 con 0x7f0240105580 2024-03-21T10:24:41.110 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:41.107+0000 7f023fe00700 1 --2- 172.21.15.47:0/2415018052 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0240105580 0x7f024019aaf0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f022c00d9b0 tx=0x7f022c00dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:24:41.110 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:41.107+0000 7f023d600700 1 -- 172.21.15.47:0/2415018052 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f022c017070 con 0x7f0240105580 2024-03-21T10:24:41.110 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:41.108+0000 7f023d600700 1 -- 172.21.15.47:0/2415018052 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f022c005020 con 0x7f0240105580 2024-03-21T10:24:41.110 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:41.108+0000 7f023d600700 1 -- 172.21.15.47:0/2415018052 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f022c009a20 con 0x7f0240105580 2024-03-21T10:24:41.110 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:41.108+0000 7f0247400700 1 -- 172.21.15.47:0/2415018052 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f02401a2500 con 0x7f0240105580 2024-03-21T10:24:41.110 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:41.108+0000 7f0247400700 1 -- 172.21.15.47:0/2415018052 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f02401a29f0 con 0x7f0240105580 2024-03-21T10:24:41.110 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:41.109+0000 7f0247400700 1 -- 172.21.15.47:0/2415018052 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0240068e50 con 0x7f0240105580 2024-03-21T10:24:41.112 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:41.111+0000 7f023d600700 1 -- 172.21.15.47:0/2415018052 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 18) v1 ==== 137854+0+0 (secure 0 0 0) 0x7f022c0051d0 con 0x7f0240105580 2024-03-21T10:24:41.113 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:41.112+0000 7f023d600700 1 --2- 172.21.15.47:0/2415018052 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f02280a3030 0x7f02280a54f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:24:41.114 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:41.113+0000 7f023d600700 1 -- 172.21.15.47:0/2415018052 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9216+0+0 (secure 0 0 0) 0x7f022c0cf550 con 0x7f0240105580 2024-03-21T10:24:41.114 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:41.113+0000 7f0244a00700 1 --2- 172.21.15.47:0/2415018052 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f02280a3030 0x7f02280a54f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:24:41.114 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:41.114+0000 7f0244a00700 1 --2- 172.21.15.47:0/2415018052 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f02280a3030 0x7f02280a54f0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f023400b5c0 tx=0x7f0234003850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:24:41.118 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:41.117+0000 7f023d600700 1 -- 172.21.15.47:0/2415018052 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f022c020020 con 0x7f0240105580 2024-03-21T10:24:41.360 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:41.358+0000 7f0247400700 1 -- 172.21.15.47:0/2415018052 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd dump"} v 0) v1 -- 0x7f0240062da0 con 0x7f0240105580 2024-03-21T10:24:41.361 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:41.360+0000 7f023d600700 1 -- 172.21.15.47:0/2415018052 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump"}]=0 v95) v1 ==== 56+0+5706 (secure 0 0 0) 0x7f022c09bf70 con 0x7f0240105580 2024-03-21T10:24:41.362 INFO:teuthology.orchestra.run.smithi047.stdout:epoch 95 2024-03-21T10:24:41.362 INFO:teuthology.orchestra.run.smithi047.stdout:fsid 3056d51a-e76b-11ee-95cd-87774f69a715 2024-03-21T10:24:41.362 INFO:teuthology.orchestra.run.smithi047.stdout:created 2024-03-21T10:11:17.019965+0000 2024-03-21T10:24:41.362 INFO:teuthology.orchestra.run.smithi047.stdout:modified 2024-03-21T10:23:27.619804+0000 2024-03-21T10:24:41.362 INFO:teuthology.orchestra.run.smithi047.stdout:flags sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit 2024-03-21T10:24:41.362 INFO:teuthology.orchestra.run.smithi047.stdout:crush_version 27 2024-03-21T10:24:41.362 INFO:teuthology.orchestra.run.smithi047.stdout:full_ratio 0.95 2024-03-21T10:24:41.362 INFO:teuthology.orchestra.run.smithi047.stdout:backfillfull_ratio 0.9 2024-03-21T10:24:41.362 INFO:teuthology.orchestra.run.smithi047.stdout:nearfull_ratio 0.85 2024-03-21T10:24:41.363 INFO:teuthology.orchestra.run.smithi047.stdout:require_min_compat_client luminous 2024-03-21T10:24:41.363 INFO:teuthology.orchestra.run.smithi047.stdout:min_compat_client luminous 2024-03-21T10:24:41.363 INFO:teuthology.orchestra.run.smithi047.stdout:require_osd_release squid 2024-03-21T10:24:41.363 INFO:teuthology.orchestra.run.smithi047.stdout:stretch_mode_enabled false 2024-03-21T10:24:41.363 INFO:teuthology.orchestra.run.smithi047.stdout:pool 1 '.mgr' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 1 pgp_num 1 autoscale_mode off last_change 22 flags hashpspool stripe_width 0 pg_num_max 32 pg_num_min 1 application mgr read_balance_score 12.00 2024-03-21T10:24:41.363 INFO:teuthology.orchestra.run.smithi047.stdout:pool 2 'cephfs_metadata' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 32 pgp_num 32 autoscale_mode off last_change 86 flags hashpspool stripe_width 0 pg_autoscale_bias 4 pg_num_min 16 recovery_priority 5 application cephfs read_balance_score 1.88 2024-03-21T10:24:41.363 INFO:teuthology.orchestra.run.smithi047.stdout:pool 3 'cephfs_data' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 64 pgp_num 64 autoscale_mode off last_change 87 flags hashpspool stripe_width 0 pg_num_min 64 target_size_ratio 0.05 application cephfs read_balance_score 1.50 2024-03-21T10:24:41.363 INFO:teuthology.orchestra.run.smithi047.stdout:pool 4 'cephfs_data_ec' erasure profile cephfs_data_ec size 4 min_size 3 crush_rule 1 object_hash rjenkins pg_num 32 pgp_num 32 autoscale_mode off last_change 91 flags hashpspool,ec_overwrites stripe_width 8192 pg_num_min 64 target_size_ratio 0.9 application cephfs 2024-03-21T10:24:41.363 INFO:teuthology.orchestra.run.smithi047.stdout:max_osd 12 2024-03-21T10:24:41.363 INFO:teuthology.orchestra.run.smithi047.stdout:osd.0 up in weight 1 up_from 8 up_thru 92 down_at 0 last_clean_interval [0,0) [v2:172.21.15.47:6802/775837313,v1:172.21.15.47:6803/775837313] [v2:172.21.15.47:6804/775837313,v1:172.21.15.47:6805/775837313] exists,up b0c7a102-8d89-428b-b793-cbbe67151f50 2024-03-21T10:24:41.363 INFO:teuthology.orchestra.run.smithi047.stdout:osd.1 up in weight 1 up_from 13 up_thru 89 down_at 0 last_clean_interval [0,0) [v2:172.21.15.100:6800/2923304710,v1:172.21.15.100:6801/2923304710] [v2:172.21.15.100:6802/2923304710,v1:172.21.15.100:6803/2923304710] exists,up 07bb81bb-7cf3-4bbc-ac9e-802f95185d11 2024-03-21T10:24:41.363 INFO:teuthology.orchestra.run.smithi047.stdout:osd.2 up in weight 1 up_from 18 up_thru 94 down_at 0 last_clean_interval [0,0) [v2:172.21.15.157:6800/171377005,v1:172.21.15.157:6801/171377005] [v2:172.21.15.157:6802/171377005,v1:172.21.15.157:6803/171377005] exists,up 88ce76b9-6081-4836-aad0-8c249c56e8a1 2024-03-21T10:24:41.363 INFO:teuthology.orchestra.run.smithi047.stdout:osd.3 up in weight 1 up_from 27 up_thru 89 down_at 0 last_clean_interval [0,0) [v2:172.21.15.47:6810/640760101,v1:172.21.15.47:6811/640760101] [v2:172.21.15.47:6812/640760101,v1:172.21.15.47:6813/640760101] exists,up 518aa33d-263f-45e6-aab5-2201ceb40e74 2024-03-21T10:24:41.364 INFO:teuthology.orchestra.run.smithi047.stdout:osd.4 up in weight 1 up_from 35 up_thru 94 down_at 0 last_clean_interval [0,0) [v2:172.21.15.100:6808/2785478639,v1:172.21.15.100:6809/2785478639] [v2:172.21.15.100:6810/2785478639,v1:172.21.15.100:6811/2785478639] exists,up 987bde83-ecc6-42e5-bd56-c143b3de4226 2024-03-21T10:24:41.364 INFO:teuthology.orchestra.run.smithi047.stdout:osd.5 up in weight 1 up_from 40 up_thru 92 down_at 0 last_clean_interval [0,0) [v2:172.21.15.157:6808/1525645410,v1:172.21.15.157:6809/1525645410] [v2:172.21.15.157:6810/1525645410,v1:172.21.15.157:6811/1525645410] exists,up 012d2188-3a61-4a5f-8d01-2001c15ffbe2 2024-03-21T10:24:41.364 INFO:teuthology.orchestra.run.smithi047.stdout:osd.6 up in weight 1 up_from 48 up_thru 92 down_at 0 last_clean_interval [0,0) [v2:172.21.15.47:6818/2321642790,v1:172.21.15.47:6819/2321642790] [v2:172.21.15.47:6820/2321642790,v1:172.21.15.47:6821/2321642790] exists,up c12c14b4-85f0-4d88-937c-de4f556437de 2024-03-21T10:24:41.364 INFO:teuthology.orchestra.run.smithi047.stdout:osd.7 up in weight 1 up_from 53 up_thru 89 down_at 0 last_clean_interval [0,0) [v2:172.21.15.100:6816/4104727330,v1:172.21.15.100:6817/4104727330] [v2:172.21.15.100:6818/4104727330,v1:172.21.15.100:6819/4104727330] exists,up 3c87c6d0-1d1f-4294-9917-aaaad7dda824 2024-03-21T10:24:41.364 INFO:teuthology.orchestra.run.smithi047.stdout:osd.8 up in weight 1 up_from 61 up_thru 94 down_at 0 last_clean_interval [0,0) [v2:172.21.15.157:6816/4067715347,v1:172.21.15.157:6817/4067715347] [v2:172.21.15.157:6818/4067715347,v1:172.21.15.157:6819/4067715347] exists,up 502d65cf-53c8-4936-8de4-e11b2a6e1e18 2024-03-21T10:24:41.364 INFO:teuthology.orchestra.run.smithi047.stdout:osd.9 up in weight 1 up_from 69 up_thru 94 down_at 0 last_clean_interval [0,0) [v2:172.21.15.47:6826/1982108738,v1:172.21.15.47:6827/1982108738] [v2:172.21.15.47:6828/1982108738,v1:172.21.15.47:6829/1982108738] exists,up 5ea1f5be-8458-4c93-9e61-c2e867a514f3 2024-03-21T10:24:41.364 INFO:teuthology.orchestra.run.smithi047.stdout:osd.10 up in weight 1 up_from 74 up_thru 92 down_at 0 last_clean_interval [0,0) [v2:172.21.15.100:6824/396391028,v1:172.21.15.100:6825/396391028] [v2:172.21.15.100:6826/396391028,v1:172.21.15.100:6827/396391028] exists,up 2da3da90-a0be-4a54-a9ff-b2d6c9b36870 2024-03-21T10:24:41.364 INFO:teuthology.orchestra.run.smithi047.stdout:osd.11 up in weight 1 up_from 82 up_thru 92 down_at 0 last_clean_interval [0,0) [v2:172.21.15.157:6824/2671029483,v1:172.21.15.157:6825/2671029483] [v2:172.21.15.157:6826/2671029483,v1:172.21.15.157:6827/2671029483] exists,up 11798d41-7f7e-4319-95b8-8d3d0946bde9 2024-03-21T10:24:41.364 INFO:teuthology.orchestra.run.smithi047.stdout:pg_upmap_items 2.11 [7,11] 2024-03-21T10:24:41.364 INFO:teuthology.orchestra.run.smithi047.stdout:pg_upmap_items 2.14 [2,4] 2024-03-21T10:24:41.364 INFO:teuthology.orchestra.run.smithi047.stdout:pg_upmap_items 2.1a [8,1] 2024-03-21T10:24:41.364 INFO:teuthology.orchestra.run.smithi047.stdout:pg_upmap_items 2.1b [2,4] 2024-03-21T10:24:41.364 INFO:teuthology.orchestra.run.smithi047.stdout:pg_upmap_items 2.1d [8,11] 2024-03-21T10:24:41.364 INFO:teuthology.orchestra.run.smithi047.stdout:pg_upmap_items 3.0 [4,8,5,9] 2024-03-21T10:24:41.365 INFO:teuthology.orchestra.run.smithi047.stdout:pg_upmap_items 3.6 [3,7] 2024-03-21T10:24:41.365 INFO:teuthology.orchestra.run.smithi047.stdout:pg_upmap_items 3.1b [4,6] 2024-03-21T10:24:41.365 INFO:teuthology.orchestra.run.smithi047.stdout:pg_upmap_items 3.33 [5,9] 2024-03-21T10:24:41.365 INFO:teuthology.orchestra.run.smithi047.stdout:pg_upmap_items 3.3c [4,9] 2024-03-21T10:24:41.365 INFO:teuthology.orchestra.run.smithi047.stdout:pg_upmap_items 4.c [9,10] 2024-03-21T10:24:41.365 INFO:teuthology.orchestra.run.smithi047.stdout:pg_upmap_items 4.f [9,8] 2024-03-21T10:24:41.365 INFO:teuthology.orchestra.run.smithi047.stdout:pg_upmap_items 4.19 [9,8] 2024-03-21T10:24:41.365 INFO:teuthology.orchestra.run.smithi047.stdout:pg_upmap_items 4.1d [5,8] 2024-03-21T10:24:41.365 INFO:teuthology.orchestra.run.smithi047.stdout:blocklist 172.21.15.47:0/356822431 expires 2024-03-22T10:12:25.844530+0000 2024-03-21T10:24:41.365 INFO:teuthology.orchestra.run.smithi047.stdout:blocklist 172.21.15.47:0/1167490879 expires 2024-03-22T10:12:25.844530+0000 2024-03-21T10:24:41.365 INFO:teuthology.orchestra.run.smithi047.stdout:blocklist 172.21.15.47:0/1540718138 expires 2024-03-22T10:12:25.844530+0000 2024-03-21T10:24:41.365 INFO:teuthology.orchestra.run.smithi047.stdout:blocklist 172.21.15.47:6801/3836903972 expires 2024-03-22T10:12:25.844530+0000 2024-03-21T10:24:41.365 INFO:teuthology.orchestra.run.smithi047.stdout:blocklist 172.21.15.47:0/235436602 expires 2024-03-22T10:11:50.301838+0000 2024-03-21T10:24:41.365 INFO:teuthology.orchestra.run.smithi047.stdout:blocklist 172.21.15.47:0/2052815572 expires 2024-03-22T10:11:50.301838+0000 2024-03-21T10:24:41.365 INFO:teuthology.orchestra.run.smithi047.stdout:blocklist 172.21.15.47:6800/3942658432 expires 2024-03-22T10:11:50.301838+0000 2024-03-21T10:24:41.365 INFO:teuthology.orchestra.run.smithi047.stdout:blocklist 172.21.15.47:6801/3942658432 expires 2024-03-22T10:11:50.301838+0000 2024-03-21T10:24:41.366 INFO:teuthology.orchestra.run.smithi047.stdout:blocklist 172.21.15.47:0/1419516257 expires 2024-03-22T10:11:50.301838+0000 2024-03-21T10:24:41.366 INFO:teuthology.orchestra.run.smithi047.stdout:blocklist 172.21.15.47:6800/3836903972 expires 2024-03-22T10:12:25.844530+0000 2024-03-21T10:24:41.366 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:41.363+0000 7f0247400700 1 -- 172.21.15.47:0/2415018052 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f02280a3030 msgr2=0x7f02280a54f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:24:41.367 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:41.363+0000 7f0247400700 1 --2- 172.21.15.47:0/2415018052 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f02280a3030 0x7f02280a54f0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f023400b5c0 tx=0x7f0234003850 comp rx=0 tx=0).stop 2024-03-21T10:24:41.367 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:41.363+0000 7f0247400700 1 -- 172.21.15.47:0/2415018052 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0240105580 msgr2=0x7f024019aaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:24:41.367 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:41.363+0000 7f0247400700 1 --2- 172.21.15.47:0/2415018052 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0240105580 0x7f024019aaf0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f022c00d9b0 tx=0x7f022c00dcc0 comp rx=0 tx=0).stop 2024-03-21T10:24:41.367 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:41.363+0000 7f0247400700 1 -- 172.21.15.47:0/2415018052 shutdown_connections 2024-03-21T10:24:41.367 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:41.363+0000 7f0247400700 1 --2- 172.21.15.47:0/2415018052 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f02280a3030 0x7f02280a54f0 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:24:41.367 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:41.363+0000 7f0247400700 1 --2- 172.21.15.47:0/2415018052 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0240107eb0 0x7f02401a1b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:24:41.367 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:41.363+0000 7f0247400700 1 --2- 172.21.15.47:0/2415018052 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0240102c50 0x7f024019a5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:24:41.367 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:41.363+0000 7f0247400700 1 --2- 172.21.15.47:0/2415018052 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0240105580 0x7f024019aaf0 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:24:41.367 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:41.363+0000 7f0247400700 1 -- 172.21.15.47:0/2415018052 >> 172.21.15.47:0/2415018052 conn(0x7f02400fc660 msgr2=0x7f02401023e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:24:41.367 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:41.364+0000 7f0247400700 1 -- 172.21.15.47:0/2415018052 shutdown_connections 2024-03-21T10:24:41.367 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:41.364+0000 7f0247400700 1 -- 172.21.15.47:0/2415018052 wait complete. 2024-03-21T10:24:41.479 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:24:41 smithi100 ceph-mon[35193]: pgmap v452: 129 pgs: 129 active+clean; 642 KiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 11 op/s 2024-03-21T10:24:41.490 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:24:41 smithi157 ceph-mon[35581]: pgmap v452: 129 pgs: 129 active+clean; 642 KiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 11 op/s 2024-03-21T10:24:41.713 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:24:41 smithi047 ceph-mon[31547]: pgmap v452: 129 pgs: 129 active+clean; 642 KiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 11 op/s 2024-03-21T10:24:42.314 INFO:teuthology.run_tasks:Running task fs.ready... 2024-03-21T10:24:42.326 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-21T10:24:42.490 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:24:42 smithi157 ceph-mon[35581]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T10:24:42.490 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:24:42 smithi157 ceph-mon[35581]: from='client.? 172.21.15.47:0/2415018052' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-03-21T10:24:42.490 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:24:42 smithi157 ceph-mon[35581]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:24:42.490 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:24:42 smithi157 ceph-mon[35581]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:24:42.527 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:42.526+0000 7fb10c600640 1 -- 172.21.15.47:0/301278005 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb104104ad0 msgr2=0x7fb104104f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:24:42.528 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:42.526+0000 7fb10c600640 1 --2- 172.21.15.47:0/301278005 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb104104ad0 0x7fb104104f50 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7fb0f0009900 tx=0x7fb0f002bc30 comp rx=0 tx=0).stop 2024-03-21T10:24:42.528 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:42.526+0000 7fb10c600640 1 -- 172.21.15.47:0/301278005 shutdown_connections 2024-03-21T10:24:42.528 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:42.526+0000 7fb10c600640 1 --2- 172.21.15.47:0/301278005 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb104105490 0x7fb10410bd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:24:42.528 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:42.526+0000 7fb10c600640 1 --2- 172.21.15.47:0/301278005 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb104104ad0 0x7fb104104f50 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:24:42.528 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:42.526+0000 7fb10c600640 1 --2- 172.21.15.47:0/301278005 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb1041038e0 0x7fb104103d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:24:42.528 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:42.526+0000 7fb10c600640 1 -- 172.21.15.47:0/301278005 >> 172.21.15.47:0/301278005 conn(0x7fb1040fee50 msgr2=0x7fb104101290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:24:42.528 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:42.526+0000 7fb10c600640 1 -- 172.21.15.47:0/301278005 shutdown_connections 2024-03-21T10:24:42.528 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:42.526+0000 7fb10c600640 1 -- 172.21.15.47:0/301278005 wait complete. 2024-03-21T10:24:42.530 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:42.527+0000 7fb10c600640 1 Processor -- start 2024-03-21T10:24:42.530 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:42.527+0000 7fb10c600640 1 -- start start 2024-03-21T10:24:42.530 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:42.528+0000 7fb10c600640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb1041038e0 0x7fb104199af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:24:42.530 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:42.528+0000 7fb10c600640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb104104ad0 0x7fb10419a030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:24:42.530 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:42.528+0000 7fb10c600640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb104105490 0x7fb1041a10a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:24:42.530 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:42.528+0000 7fb10c600640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb10410f410 con 0x7fb1041038e0 2024-03-21T10:24:42.530 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:42.528+0000 7fb10c600640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb10410f290 con 0x7fb104104ad0 2024-03-21T10:24:42.530 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:42.528+0000 7fb10c600640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb10410f590 con 0x7fb104105490 2024-03-21T10:24:42.530 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:42.528+0000 7fb109c00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb1041038e0 0x7fb104199af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:24:42.530 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:42.528+0000 7fb109c00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb1041038e0 0x7fb104199af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:44160/0 (socket says 172.21.15.47:44160) 2024-03-21T10:24:42.530 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:42.528+0000 7fb109200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb104104ad0 0x7fb10419a030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:24:42.531 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:42.528+0000 7fb109c00640 1 -- 172.21.15.47:0/2610121739 learned_addr learned my addr 172.21.15.47:0/2610121739 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:24:42.531 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:42.528+0000 7fb10a600640 1 --2- 172.21.15.47:0/2610121739 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb104105490 0x7fb1041a10a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:24:42.531 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:42.528+0000 7fb109c00640 1 -- 172.21.15.47:0/2610121739 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb104105490 msgr2=0x7fb1041a10a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:24:42.531 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:42.528+0000 7fb109c00640 1 --2- 172.21.15.47:0/2610121739 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb104105490 0x7fb1041a10a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:24:42.531 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:42.528+0000 7fb109c00640 1 -- 172.21.15.47:0/2610121739 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb104104ad0 msgr2=0x7fb10419a030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:24:42.531 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:42.528+0000 7fb109c00640 1 --2- 172.21.15.47:0/2610121739 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb104104ad0 0x7fb10419a030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:24:42.531 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:42.529+0000 7fb109c00640 1 -- 172.21.15.47:0/2610121739 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1041a1710 con 0x7fb1041038e0 2024-03-21T10:24:42.531 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:42.529+0000 7fb109c00640 1 --2- 172.21.15.47:0/2610121739 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb1041038e0 0x7fb104199af0 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7fb0f800e9c0 tx=0x7fb0f800ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:24:42.532 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:42.529+0000 7fb102000640 1 -- 172.21.15.47:0/2610121739 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0f800cde0 con 0x7fb1041038e0 2024-03-21T10:24:42.532 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:42.529+0000 7fb10c600640 1 -- 172.21.15.47:0/2610121739 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb1041a1a00 con 0x7fb1041038e0 2024-03-21T10:24:42.532 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:42.529+0000 7fb102000640 1 -- 172.21.15.47:0/2610121739 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fb0f80108e0 con 0x7fb1041038e0 2024-03-21T10:24:42.532 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:42.529+0000 7fb102000640 1 -- 172.21.15.47:0/2610121739 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0f80188e0 con 0x7fb1041038e0 2024-03-21T10:24:42.532 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:42.529+0000 7fb10c600640 1 -- 172.21.15.47:0/2610121739 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb1041a1f70 con 0x7fb1041038e0 2024-03-21T10:24:42.532 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:42.531+0000 7fb10c600640 1 -- 172.21.15.47:0/2610121739 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb1040688e0 con 0x7fb1041038e0 2024-03-21T10:24:42.536 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:42.535+0000 7fb102000640 1 -- 172.21.15.47:0/2610121739 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 18) v1 ==== 137854+0+0 (secure 0 0 0) 0x7fb0f801f020 con 0x7fb1041038e0 2024-03-21T10:24:42.537 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:42.536+0000 7fb102000640 1 --2- 172.21.15.47:0/2610121739 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7fb0ec0a2f20 0x7fb0ec0a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:24:42.538 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:42.536+0000 7fb102000640 1 -- 172.21.15.47:0/2610121739 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9216+0+0 (secure 0 0 0) 0x7fb0f80d0060 con 0x7fb1041038e0 2024-03-21T10:24:42.538 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:42.537+0000 7fb109200640 1 --2- 172.21.15.47:0/2610121739 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7fb0ec0a2f20 0x7fb0ec0a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:24:42.538 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:42.537+0000 7fb102000640 1 -- 172.21.15.47:0/2610121739 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fb0f809ca80 con 0x7fb1041038e0 2024-03-21T10:24:42.538 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:42.538+0000 7fb109200640 1 --2- 172.21.15.47:0/2610121739 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7fb0ec0a2f20 0x7fb0ec0a53e0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fb0f0009230 tx=0x7fb0f0005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:24:42.626 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:24:42 smithi100 ceph-mon[35193]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T10:24:42.626 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:24:42 smithi100 ceph-mon[35193]: from='client.? 172.21.15.47:0/2415018052' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-03-21T10:24:42.626 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:24:42 smithi100 ceph-mon[35193]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:24:42.626 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:24:42 smithi100 ceph-mon[35193]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:24:42.714 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:24:42 smithi047 ceph-mon[31547]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T10:24:42.714 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:24:42 smithi047 ceph-mon[31547]: from='client.? 172.21.15.47:0/2415018052' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-03-21T10:24:42.714 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:24:42 smithi047 ceph-mon[31547]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:24:42.714 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:24:42 smithi047 ceph-mon[31547]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:24:42.757 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:42.756+0000 7fb10c600640 1 -- 172.21.15.47:0/2610121739 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fb1040622a0 con 0x7fb1041038e0 2024-03-21T10:24:42.761 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:42.760+0000 7fb102000640 1 -- 172.21.15.47:0/2610121739 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v95) v1 ==== 74+0+24849 (secure 0 0 0) 0x7fb0f809c380 con 0x7fb1041038e0 2024-03-21T10:24:42.762 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-21T10:24:42.762 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":95,"fsid":"3056d51a-e76b-11ee-95cd-87774f69a715","created":"2024-03-21T10:11:17.019965+0000","modified":"2024-03-21T10:23:27.619804+0000","last_up_change":"2024-03-21T10:20:52.675328+0000","last_in_change":"2024-03-21T10:20:27.648212+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-21T10:15:40.553114+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-21T10:21:56.399518+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-21T10:21:57.812940+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-21T10:22:01.822568+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"91","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"b0c7a102-8d89-428b-b793-cbbe67151f50","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6802","nonce":775837313},{"type":"v1","addr":"172.21.15.47:6803","nonce":775837313}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6804","nonce":775837313},{"type":"v1","addr":"172.21.15.47:6805","nonce":775837313}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6808","nonce":775837313},{"type":"v1","addr":"172.21.15.47:6809","nonce":775837313}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6806","nonce":775837313},{"type":"v1","addr":"172.21.15.47:6807","nonce":775837313}]},"public_addr":"172.21.15.47:6803/775837313","cluster_addr":"172.21.15.47:6805/775837313","heartbeat_back_addr":"172.21.15.47:6809/775837313","heartbeat_front_addr":"172.21.15.47:6807/775837313","state":["exists","up"]},{"osd":1,"uuid":"07bb81bb-7cf3-4bbc-ac9e-802f95185d11","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6800","nonce":2923304710},{"type":"v1","addr":"172.21.15.100:6801","nonce":2923304710}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":2923304710},{"type":"v1","addr":"172.21.15.100:6803","nonce":2923304710}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":2923304710},{"type":"v1","addr":"172.21.15.100:6807","nonce":2923304710}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6804","nonce":2923304710},{"type":"v1","addr":"172.21.15.100:6805","nonce":2923304710}]},"public_addr":"172.21.15.100:6801/2923304710","cluster_addr":"172.21.15.100:6803/2923304710","heartbeat_back_addr":"172.21.15.100:6807/2923304710","heartbeat_front_addr":"172.21.15.100:6805/2923304710","state":["exists","up"]},{"osd":2,"uuid":"88ce76b9-6081-4836-aad0-8c249c56e8a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":171377005},{"type":"v1","addr":"172.21.15.157:6801","nonce":171377005}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":171377005},{"type":"v1","addr":"172.21.15.157:6803","nonce":171377005}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":171377005},{"type":"v1","addr":"172.21.15.157:6807","nonce":171377005}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":171377005},{"type":"v1","addr":"172.21.15.157:6805","nonce":171377005}]},"public_addr":"172.21.15.157:6801/171377005","cluster_addr":"172.21.15.157:6803/171377005","heartbeat_back_addr":"172.21.15.157:6807/171377005","heartbeat_front_addr":"172.21.15.157:6805/171377005","state":["exists","up"]},{"osd":3,"uuid":"518aa33d-263f-45e6-aab5-2201ceb40e74","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":27,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6810","nonce":640760101},{"type":"v1","addr":"172.21.15.47:6811","nonce":640760101}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6812","nonce":640760101},{"type":"v1","addr":"172.21.15.47:6813","nonce":640760101}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6816","nonce":640760101},{"type":"v1","addr":"172.21.15.47:6817","nonce":640760101}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6814","nonce":640760101},{"type":"v1","addr":"172.21.15.47:6815","nonce":640760101}]},"public_addr":"172.21.15.47:6811/640760101","cluster_addr":"172.21.15.47:6813/640760101","heartbeat_back_addr":"172.21.15.47:6817/640760101","heartbeat_front_addr":"172.21.15.47:6815/640760101","state":["exists","up"]},{"osd":4,"uuid":"987bde83-ecc6-42e5-bd56-c143b3de4226","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":35,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6808","nonce":2785478639},{"type":"v1","addr":"172.21.15.100:6809","nonce":2785478639}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6810","nonce":2785478639},{"type":"v1","addr":"172.21.15.100:6811","nonce":2785478639}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6814","nonce":2785478639},{"type":"v1","addr":"172.21.15.100:6815","nonce":2785478639}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":2785478639},{"type":"v1","addr":"172.21.15.100:6813","nonce":2785478639}]},"public_addr":"172.21.15.100:6809/2785478639","cluster_addr":"172.21.15.100:6811/2785478639","heartbeat_back_addr":"172.21.15.100:6815/2785478639","heartbeat_front_addr":"172.21.15.100:6813/2785478639","state":["exists","up"]},{"osd":5,"uuid":"012d2188-3a61-4a5f-8d01-2001c15ffbe2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":40,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":1525645410},{"type":"v1","addr":"172.21.15.157:6809","nonce":1525645410}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":1525645410},{"type":"v1","addr":"172.21.15.157:6811","nonce":1525645410}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":1525645410},{"type":"v1","addr":"172.21.15.157:6815","nonce":1525645410}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":1525645410},{"type":"v1","addr":"172.21.15.157:6813","nonce":1525645410}]},"public_addr":"172.21.15.157:6809/1525645410","cluster_addr":"172.21.15.157:6811/1525645410","heartbeat_back_addr":"172.21.15.157:6815/1525645410","heartbeat_front_addr":"172.21.15.157:6813/1525645410","state":["exists","up"]},{"osd":6,"uuid":"c12c14b4-85f0-4d88-937c-de4f556437de","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6818","nonce":2321642790},{"type":"v1","addr":"172.21.15.47:6819","nonce":2321642790}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6820","nonce":2321642790},{"type":"v1","addr":"172.21.15.47:6821","nonce":2321642790}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6824","nonce":2321642790},{"type":"v1","addr":"172.21.15.47:6825","nonce":2321642790}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6822","nonce":2321642790},{"type":"v1","addr":"172.21.15.47:6823","nonce":2321642790}]},"public_addr":"172.21.15.47:6819/2321642790","cluster_addr":"172.21.15.47:6821/2321642790","heartbeat_back_addr":"172.21.15.47:6825/2321642790","heartbeat_front_addr":"172.21.15.47:6823/2321642790","state":["exists","up"]},{"osd":7,"uuid":"3c87c6d0-1d1f-4294-9917-aaaad7dda824","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6816","nonce":4104727330},{"type":"v1","addr":"172.21.15.100:6817","nonce":4104727330}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":4104727330},{"type":"v1","addr":"172.21.15.100:6819","nonce":4104727330}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":4104727330},{"type":"v1","addr":"172.21.15.100:6823","nonce":4104727330}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6820","nonce":4104727330},{"type":"v1","addr":"172.21.15.100:6821","nonce":4104727330}]},"public_addr":"172.21.15.100:6817/4104727330","cluster_addr":"172.21.15.100:6819/4104727330","heartbeat_back_addr":"172.21.15.100:6823/4104727330","heartbeat_front_addr":"172.21.15.100:6821/4104727330","state":["exists","up"]},{"osd":8,"uuid":"502d65cf-53c8-4936-8de4-e11b2a6e1e18","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":61,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":4067715347},{"type":"v1","addr":"172.21.15.157:6817","nonce":4067715347}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":4067715347},{"type":"v1","addr":"172.21.15.157:6819","nonce":4067715347}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":4067715347},{"type":"v1","addr":"172.21.15.157:6823","nonce":4067715347}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":4067715347},{"type":"v1","addr":"172.21.15.157:6821","nonce":4067715347}]},"public_addr":"172.21.15.157:6817/4067715347","cluster_addr":"172.21.15.157:6819/4067715347","heartbeat_back_addr":"172.21.15.157:6823/4067715347","heartbeat_front_addr":"172.21.15.157:6821/4067715347","state":["exists","up"]},{"osd":9,"uuid":"5ea1f5be-8458-4c93-9e61-c2e867a514f3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6826","nonce":1982108738},{"type":"v1","addr":"172.21.15.47:6827","nonce":1982108738}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6828","nonce":1982108738},{"type":"v1","addr":"172.21.15.47:6829","nonce":1982108738}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6832","nonce":1982108738},{"type":"v1","addr":"172.21.15.47:6833","nonce":1982108738}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6830","nonce":1982108738},{"type":"v1","addr":"172.21.15.47:6831","nonce":1982108738}]},"public_addr":"172.21.15.47:6827/1982108738","cluster_addr":"172.21.15.47:6829/1982108738","heartbeat_back_addr":"172.21.15.47:6833/1982108738","heartbeat_front_addr":"172.21.15.47:6831/1982108738","state":["exists","up"]},{"osd":10,"uuid":"2da3da90-a0be-4a54-a9ff-b2d6c9b36870","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":74,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":396391028},{"type":"v1","addr":"172.21.15.100:6825","nonce":396391028}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":396391028},{"type":"v1","addr":"172.21.15.100:6827","nonce":396391028}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6830","nonce":396391028},{"type":"v1","addr":"172.21.15.100:6831","nonce":396391028}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":396391028},{"type":"v1","addr":"172.21.15.100:6829","nonce":396391028}]},"public_addr":"172.21.15.100:6825/396391028","cluster_addr":"172.21.15.100:6827/396391028","heartbeat_back_addr":"172.21.15.100:6831/396391028","heartbeat_front_addr":"172.21.15.100:6829/396391028","state":["exists","up"]},{"osd":11,"uuid":"11798d41-7f7e-4319-95b8-8d3d0946bde9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":82,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":2671029483},{"type":"v1","addr":"172.21.15.157:6825","nonce":2671029483}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":2671029483},{"type":"v1","addr":"172.21.15.157:6827","nonce":2671029483}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":2671029483},{"type":"v1","addr":"172.21.15.157:6831","nonce":2671029483}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":2671029483},{"type":"v1","addr":"172.21.15.157:6829","nonce":2671029483}]},"public_addr":"172.21.15.157:6825/2671029483","cluster_addr":"172.21.15.157:6827/2671029483","heartbeat_back_addr":"172.21.15.157:6831/2671029483","heartbeat_front_addr":"172.21.15.157:6829/2671029483","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T10:14:31.822399+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T10:15:04.175721+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T10:15:35.902595+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T10:16:12.088891+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T10:16:46.428704+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T10:17:20.158270+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T10:17:55.915253+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T10:18:30.064641+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T10:19:04.410004+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T10:19:40.605181+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T10:20:15.636605+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T10:20:50.151365+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.11","mappings":[{"from":7,"to":11}]},{"pgid":"2.14","mappings":[{"from":2,"to":4}]},{"pgid":"2.1a","mappings":[{"from":8,"to":1}]},{"pgid":"2.1b","mappings":[{"from":2,"to":4}]},{"pgid":"2.1d","mappings":[{"from":8,"to":11}]},{"pgid":"3.0","mappings":[{"from":4,"to":8},{"from":5,"to":9}]},{"pgid":"3.6","mappings":[{"from":3,"to":7}]},{"pgid":"3.1b","mappings":[{"from":4,"to":6}]},{"pgid":"3.33","mappings":[{"from":5,"to":9}]},{"pgid":"3.3c","mappings":[{"from":4,"to":9}]},{"pgid":"4.c","mappings":[{"from":9,"to":10}]},{"pgid":"4.f","mappings":[{"from":9,"to":8}]},{"pgid":"4.19","mappings":[{"from":9,"to":8}]},{"pgid":"4.1d","mappings":[{"from":5,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.47:0/356822431":"2024-03-22T10:12:25.844530+0000","172.21.15.47:0/1167490879":"2024-03-22T10:12:25.844530+0000","172.21.15.47:0/1540718138":"2024-03-22T10:12:25.844530+0000","172.21.15.47:6801/3836903972":"2024-03-22T10:12:25.844530+0000","172.21.15.47:0/235436602":"2024-03-22T10:11:50.301838+0000","172.21.15.47:0/2052815572":"2024-03-22T10:11:50.301838+0000","172.21.15.47:6800/3942658432":"2024-03-22T10:11:50.301838+0000","172.21.15.47:6801/3942658432":"2024-03-22T10:11:50.301838+0000","172.21.15.47:0/1419516257":"2024-03-22T10:11:50.301838+0000","172.21.15.47:6800/3836903972":"2024-03-22T10:12:25.844530+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-21T10:24:42.764 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:42.763+0000 7fb10c600640 1 -- 172.21.15.47:0/2610121739 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7fb0ec0a2f20 msgr2=0x7fb0ec0a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:24:42.764 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:42.763+0000 7fb10c600640 1 --2- 172.21.15.47:0/2610121739 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7fb0ec0a2f20 0x7fb0ec0a53e0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fb0f0009230 tx=0x7fb0f0005fd0 comp rx=0 tx=0).stop 2024-03-21T10:24:42.765 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:42.763+0000 7fb10c600640 1 -- 172.21.15.47:0/2610121739 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb1041038e0 msgr2=0x7fb104199af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:24:42.765 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:42.763+0000 7fb10c600640 1 --2- 172.21.15.47:0/2610121739 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb1041038e0 0x7fb104199af0 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7fb0f800e9c0 tx=0x7fb0f800ee80 comp rx=0 tx=0).stop 2024-03-21T10:24:42.765 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:42.763+0000 7fb10c600640 1 -- 172.21.15.47:0/2610121739 shutdown_connections 2024-03-21T10:24:42.765 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:42.763+0000 7fb10c600640 1 --2- 172.21.15.47:0/2610121739 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7fb0ec0a2f20 0x7fb0ec0a53e0 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:24:42.765 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:42.763+0000 7fb10c600640 1 --2- 172.21.15.47:0/2610121739 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb104105490 0x7fb1041a10a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:24:42.765 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:42.763+0000 7fb10c600640 1 --2- 172.21.15.47:0/2610121739 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb104104ad0 0x7fb10419a030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:24:42.765 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:42.763+0000 7fb10c600640 1 --2- 172.21.15.47:0/2610121739 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb1041038e0 0x7fb104199af0 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:24:42.765 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:42.763+0000 7fb10c600640 1 -- 172.21.15.47:0/2610121739 >> 172.21.15.47:0/2610121739 conn(0x7fb1040fee50 msgr2=0x7fb104107e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:24:42.765 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:42.764+0000 7fb10c600640 1 -- 172.21.15.47:0/2610121739 shutdown_connections 2024-03-21T10:24:42.765 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:42.764+0000 7fb10c600640 1 -- 172.21.15.47:0/2610121739 wait complete. 2024-03-21T10:24:42.790 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-21T10:15:40.553114+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-21T10:21:56.399518+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '86', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-21T10:21:57.812940+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '87', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-03-21T10:22:01.822568+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '91', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-03-21T10:24:42.791 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-21T10:24:43.016 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:43.014+0000 7fb4f6800640 1 -- 172.21.15.47:0/2811715716 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb4f01016b0 msgr2=0x7fb4f0101ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:24:43.016 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:43.014+0000 7fb4f6800640 1 --2- 172.21.15.47:0/2811715716 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb4f01016b0 0x7fb4f0101ad0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fb4e0009860 tx=0x7fb4e002bb70 comp rx=0 tx=0).stop 2024-03-21T10:24:43.016 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:43.015+0000 7fb4f6800640 1 -- 172.21.15.47:0/2811715716 shutdown_connections 2024-03-21T10:24:43.016 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:43.015+0000 7fb4f6800640 1 --2- 172.21.15.47:0/2811715716 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb4f0103260 0x7fb4f0109af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:24:43.016 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:43.015+0000 7fb4f6800640 1 --2- 172.21.15.47:0/2811715716 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb4f01028a0 0x7fb4f0102d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:24:43.016 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:43.015+0000 7fb4f6800640 1 --2- 172.21.15.47:0/2811715716 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb4f01016b0 0x7fb4f0101ad0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:24:43.016 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:43.015+0000 7fb4f6800640 1 -- 172.21.15.47:0/2811715716 >> 172.21.15.47:0/2811715716 conn(0x7fb4f00fcc40 msgr2=0x7fb4f00ff060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:24:43.017 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:43.015+0000 7fb4f6800640 1 -- 172.21.15.47:0/2811715716 shutdown_connections 2024-03-21T10:24:43.017 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:43.015+0000 7fb4f6800640 1 -- 172.21.15.47:0/2811715716 wait complete. 2024-03-21T10:24:43.018 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:43.016+0000 7fb4f6800640 1 Processor -- start 2024-03-21T10:24:43.018 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:43.016+0000 7fb4f6800640 1 -- start start 2024-03-21T10:24:43.019 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:43.016+0000 7fb4f6800640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb4f01016b0 0x7fb4f0197930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:24:43.019 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:43.016+0000 7fb4f6800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb4f01028a0 0x7fb4f0197e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:24:43.019 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:43.016+0000 7fb4f6800640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb4f0103260 0x7fb4f019eee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:24:43.019 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:43.016+0000 7fb4f6800640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4f010d1e0 con 0x7fb4f01016b0 2024-03-21T10:24:43.019 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:43.016+0000 7fb4f6800640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4f010d060 con 0x7fb4f01028a0 2024-03-21T10:24:43.019 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:43.016+0000 7fb4f6800640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4f010d360 con 0x7fb4f0103260 2024-03-21T10:24:43.019 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:43.016+0000 7fb4ef400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb4f01016b0 0x7fb4f0197930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:24:43.019 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:43.017+0000 7fb4ef400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb4f01016b0 0x7fb4f0197930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:44172/0 (socket says 172.21.15.47:44172) 2024-03-21T10:24:43.019 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:43.017+0000 7fb4ef400640 1 -- 172.21.15.47:0/2619460381 learned_addr learned my addr 172.21.15.47:0/2619460381 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:24:43.019 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:43.017+0000 7fb4e7e00640 1 --2- 172.21.15.47:0/2619460381 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb4f01028a0 0x7fb4f0197e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:24:43.019 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:43.017+0000 7fb4efe00640 1 --2- 172.21.15.47:0/2619460381 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb4f0103260 0x7fb4f019eee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:24:43.019 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:43.017+0000 7fb4ef400640 1 -- 172.21.15.47:0/2619460381 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb4f0103260 msgr2=0x7fb4f019eee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:24:43.019 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:43.017+0000 7fb4ef400640 1 --2- 172.21.15.47:0/2619460381 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb4f0103260 0x7fb4f019eee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:24:43.020 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:43.017+0000 7fb4ef400640 1 -- 172.21.15.47:0/2619460381 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb4f01028a0 msgr2=0x7fb4f0197e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:24:43.020 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:43.017+0000 7fb4ef400640 1 --2- 172.21.15.47:0/2619460381 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb4f01028a0 0x7fb4f0197e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:24:43.020 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:43.017+0000 7fb4ef400640 1 -- 172.21.15.47:0/2619460381 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb4f019f550 con 0x7fb4f01016b0 2024-03-21T10:24:43.020 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:43.017+0000 7fb4e7e00640 1 --2- 172.21.15.47:0/2619460381 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb4f01028a0 0x7fb4f0197e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T10:24:43.020 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:43.017+0000 7fb4ef400640 1 --2- 172.21.15.47:0/2619460381 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb4f01016b0 0x7fb4f0197930 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7fb4e0005940 tx=0x7fb4e00042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:24:43.020 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:43.018+0000 7fb4ecc00640 1 -- 172.21.15.47:0/2619460381 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4e0041020 con 0x7fb4f01016b0 2024-03-21T10:24:43.021 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:43.018+0000 7fb4f6800640 1 -- 172.21.15.47:0/2619460381 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb4f019f7e0 con 0x7fb4f01016b0 2024-03-21T10:24:43.021 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:43.018+0000 7fb4ecc00640 1 -- 172.21.15.47:0/2619460381 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fb4e00035f0 con 0x7fb4f01016b0 2024-03-21T10:24:43.021 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:43.018+0000 7fb4ecc00640 1 -- 172.21.15.47:0/2619460381 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4e0039040 con 0x7fb4f01016b0 2024-03-21T10:24:43.021 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:43.018+0000 7fb4f6800640 1 -- 172.21.15.47:0/2619460381 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb4f019fd20 con 0x7fb4f01016b0 2024-03-21T10:24:43.021 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:43.020+0000 7fb4f6800640 1 -- 172.21.15.47:0/2619460381 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb4f0108a90 con 0x7fb4f01016b0 2024-03-21T10:24:43.022 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:43.021+0000 7fb4ecc00640 1 -- 172.21.15.47:0/2619460381 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 18) v1 ==== 137854+0+0 (secure 0 0 0) 0x7fb4e0044020 con 0x7fb4f01016b0 2024-03-21T10:24:43.023 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:43.023+0000 7fb4ecc00640 1 --2- 172.21.15.47:0/2619460381 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7fb4d40a2f20 0x7fb4d40a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:24:43.024 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:43.023+0000 7fb4ecc00640 1 -- 172.21.15.47:0/2619460381 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9216+0+0 (secure 0 0 0) 0x7fb4e00f11a0 con 0x7fb4f01016b0 2024-03-21T10:24:43.024 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:43.023+0000 7fb4e7e00640 1 --2- 172.21.15.47:0/2619460381 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7fb4d40a2f20 0x7fb4d40a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:24:43.025 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:43.024+0000 7fb4e7e00640 1 --2- 172.21.15.47:0/2619460381 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7fb4d40a2f20 0x7fb4d40a53e0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fb4d80096c0 tx=0x7fb4d8009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:24:43.028 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:43.027+0000 7fb4ecc00640 1 -- 172.21.15.47:0/2619460381 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fb4e004b020 con 0x7fb4f01016b0 2024-03-21T10:24:43.245 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:43.244+0000 7fb4f6800640 1 -- 172.21.15.47:0/2619460381 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fb4f00622a0 con 0x7fb4f01016b0 2024-03-21T10:24:43.245 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:43.244+0000 7fb4ecc00640 1 -- 172.21.15.47:0/2619460381 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v95) v1 ==== 93+0+10 (secure 0 0 0) 0x7fb4e00bdbc0 con 0x7fb4f01016b0 2024-03-21T10:24:43.246 INFO:teuthology.orchestra.run.smithi047.stdout:pg_num: 1 2024-03-21T10:24:43.249 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:43.248+0000 7fb4f6800640 1 -- 172.21.15.47:0/2619460381 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7fb4d40a2f20 msgr2=0x7fb4d40a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:24:43.249 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:43.248+0000 7fb4f6800640 1 --2- 172.21.15.47:0/2619460381 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7fb4d40a2f20 0x7fb4d40a53e0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fb4d80096c0 tx=0x7fb4d8009260 comp rx=0 tx=0).stop 2024-03-21T10:24:43.249 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:43.248+0000 7fb4f6800640 1 -- 172.21.15.47:0/2619460381 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb4f01016b0 msgr2=0x7fb4f0197930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:24:43.249 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:43.248+0000 7fb4f6800640 1 --2- 172.21.15.47:0/2619460381 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb4f01016b0 0x7fb4f0197930 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7fb4e0005940 tx=0x7fb4e00042f0 comp rx=0 tx=0).stop 2024-03-21T10:24:43.249 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:43.248+0000 7fb4f6800640 1 -- 172.21.15.47:0/2619460381 shutdown_connections 2024-03-21T10:24:43.249 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:43.248+0000 7fb4f6800640 1 --2- 172.21.15.47:0/2619460381 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7fb4d40a2f20 0x7fb4d40a53e0 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:24:43.249 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:43.248+0000 7fb4f6800640 1 --2- 172.21.15.47:0/2619460381 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb4f0103260 0x7fb4f019eee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:24:43.250 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:43.248+0000 7fb4f6800640 1 --2- 172.21.15.47:0/2619460381 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb4f01028a0 0x7fb4f0197e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:24:43.250 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:43.248+0000 7fb4f6800640 1 --2- 172.21.15.47:0/2619460381 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb4f01016b0 0x7fb4f0197930 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:24:43.250 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:43.248+0000 7fb4f6800640 1 -- 172.21.15.47:0/2619460381 >> 172.21.15.47:0/2619460381 conn(0x7fb4f00fcc40 msgr2=0x7fb4f0105c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:24:43.250 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:43.248+0000 7fb4f6800640 1 -- 172.21.15.47:0/2619460381 shutdown_connections 2024-03-21T10:24:43.250 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:43.248+0000 7fb4f6800640 1 -- 172.21.15.47:0/2619460381 wait complete. 2024-03-21T10:24:43.270 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-21T10:24:43.345 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:24:43 smithi047 ceph-mon[31547]: pgmap v453: 129 pgs: 129 active+clean; 642 KiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-03-21T10:24:43.345 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:24:43 smithi047 ceph-mon[31547]: from='client.? 172.21.15.47:0/2610121739' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-21T10:24:43.487 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:43.486+0000 7f8567400640 1 -- 172.21.15.47:0/2069776250 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f85601028a0 msgr2=0x7f8560102d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:24:43.487 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:43.486+0000 7f8567400640 1 --2- 172.21.15.47:0/2069776250 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f85601028a0 0x7f8560102d20 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f854c009900 tx=0x7f854c02bc30 comp rx=0 tx=0).stop 2024-03-21T10:24:43.488 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:43.486+0000 7f8567400640 1 -- 172.21.15.47:0/2069776250 shutdown_connections 2024-03-21T10:24:43.488 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:43.486+0000 7f8567400640 1 --2- 172.21.15.47:0/2069776250 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8560103260 0x7f8560109af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:24:43.488 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:43.486+0000 7f8567400640 1 --2- 172.21.15.47:0/2069776250 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f85601028a0 0x7f8560102d20 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:24:43.488 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:43.486+0000 7f8567400640 1 --2- 172.21.15.47:0/2069776250 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f85601016b0 0x7f8560101ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:24:43.488 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:43.486+0000 7f8567400640 1 -- 172.21.15.47:0/2069776250 >> 172.21.15.47:0/2069776250 conn(0x7f85600fcc40 msgr2=0x7f85600ff060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:24:43.488 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:43.486+0000 7f8567400640 1 -- 172.21.15.47:0/2069776250 shutdown_connections 2024-03-21T10:24:43.488 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:43.486+0000 7f8567400640 1 -- 172.21.15.47:0/2069776250 wait complete. 2024-03-21T10:24:43.490 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:43.487+0000 7f8567400640 1 Processor -- start 2024-03-21T10:24:43.490 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:43.487+0000 7f8567400640 1 -- start start 2024-03-21T10:24:43.490 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:43.488+0000 7f8567400640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f85601016b0 0x7f8560199b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:24:43.490 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:43.488+0000 7f8567400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f85601028a0 0x7f856019a0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:24:43.490 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:43.488+0000 7f8567400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8560103260 0x7f85601a1130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:24:43.490 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:43.488+0000 7f8567400640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f856010d210 con 0x7f85601028a0 2024-03-21T10:24:43.490 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:43.488+0000 7f8567400640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f856010d090 con 0x7f8560103260 2024-03-21T10:24:43.490 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:43.488+0000 7f8567400640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f856010d390 con 0x7f85601016b0 2024-03-21T10:24:43.490 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:43.488+0000 7f855fe00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f85601028a0 0x7f856019a0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:24:43.490 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:43.488+0000 7f855fe00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f85601028a0 0x7f856019a0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:44184/0 (socket says 172.21.15.47:44184) 2024-03-21T10:24:43.490 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:43.488+0000 7f855fe00640 1 -- 172.21.15.47:0/4216885630 learned_addr learned my addr 172.21.15.47:0/4216885630 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:24:43.490 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:43.488+0000 7f8564a00640 1 --2- 172.21.15.47:0/4216885630 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f85601016b0 0x7f8560199b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:24:43.490 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:43.488+0000 7f8565400640 1 --2- 172.21.15.47:0/4216885630 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8560103260 0x7f85601a1130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:24:43.491 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:43.489+0000 7f855fe00640 1 -- 172.21.15.47:0/4216885630 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f85601016b0 msgr2=0x7f8560199b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:24:43.491 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:43.489+0000 7f855fe00640 1 --2- 172.21.15.47:0/4216885630 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f85601016b0 0x7f8560199b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:24:43.491 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:43.489+0000 7f855fe00640 1 -- 172.21.15.47:0/4216885630 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8560103260 msgr2=0x7f85601a1130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:24:43.491 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:43.489+0000 7f855fe00640 1 --2- 172.21.15.47:0/4216885630 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8560103260 0x7f85601a1130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:24:43.491 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:43.489+0000 7f855fe00640 1 -- 172.21.15.47:0/4216885630 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f85601a17a0 con 0x7f85601028a0 2024-03-21T10:24:43.491 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:24:43 smithi157 ceph-mon[35581]: pgmap v453: 129 pgs: 129 active+clean; 642 KiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-03-21T10:24:43.492 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:24:43 smithi157 ceph-mon[35581]: from='client.? 172.21.15.47:0/2610121739' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-21T10:24:43.492 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:43.489+0000 7f855fe00640 1 --2- 172.21.15.47:0/4216885630 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f85601028a0 0x7f856019a0c0 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f854c000c00 tx=0x7f854c0042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:24:43.492 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:43.490+0000 7f855d600640 1 -- 172.21.15.47:0/4216885630 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f854c043020 con 0x7f85601028a0 2024-03-21T10:24:43.492 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:43.490+0000 7f8567400640 1 -- 172.21.15.47:0/4216885630 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f85601a1a30 con 0x7f85601028a0 2024-03-21T10:24:43.492 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:43.490+0000 7f855d600640 1 -- 172.21.15.47:0/4216885630 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f854c004aa0 con 0x7f85601028a0 2024-03-21T10:24:43.492 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:43.490+0000 7f855d600640 1 -- 172.21.15.47:0/4216885630 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f854c003d70 con 0x7f85601028a0 2024-03-21T10:24:43.492 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:43.490+0000 7f8567400640 1 -- 172.21.15.47:0/4216885630 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f85601a1fd0 con 0x7f85601028a0 2024-03-21T10:24:43.493 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:43.492+0000 7f8567400640 1 -- 172.21.15.47:0/4216885630 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f85601a21a0 con 0x7f85601028a0 2024-03-21T10:24:43.495 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:43.494+0000 7f855d600640 1 -- 172.21.15.47:0/4216885630 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 18) v1 ==== 137854+0+0 (secure 0 0 0) 0x7f854c046020 con 0x7f85601028a0 2024-03-21T10:24:43.496 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:43.495+0000 7f855d600640 1 --2- 172.21.15.47:0/4216885630 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f85480a2fa0 0x7f85480a5460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:24:43.496 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:43.495+0000 7f855d600640 1 -- 172.21.15.47:0/4216885630 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9216+0+0 (secure 0 0 0) 0x7f854c0f1ba0 con 0x7f85601028a0 2024-03-21T10:24:43.496 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:43.496+0000 7f8564a00640 1 --2- 172.21.15.47:0/4216885630 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f85480a2fa0 0x7f85480a5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:24:43.497 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:43.497+0000 7f8564a00640 1 --2- 172.21.15.47:0/4216885630 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f85480a2fa0 0x7f85480a5460 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f8560102700 tx=0x7f8554005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:24:43.499 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:43.498+0000 7f855d600640 1 -- 172.21.15.47:0/4216885630 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f854c04d030 con 0x7f85601028a0 2024-03-21T10:24:43.729 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:24:43 smithi100 ceph-mon[35193]: pgmap v453: 129 pgs: 129 active+clean; 642 KiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-03-21T10:24:43.729 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:24:43 smithi100 ceph-mon[35193]: from='client.? 172.21.15.47:0/2610121739' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-21T10:24:43.750 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:43.749+0000 7f8567400640 1 -- 172.21.15.47:0/4216885630 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f85601a2370 con 0x7f85601028a0 2024-03-21T10:24:43.750 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:43.750+0000 7f855d600640 1 -- 172.21.15.47:0/4216885630 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v95) v1 ==== 104+0+11 (secure 0 0 0) 0x7f854c0be5c0 con 0x7f85601028a0 2024-03-21T10:24:43.751 INFO:teuthology.orchestra.run.smithi047.stdout:pg_num: 32 2024-03-21T10:24:43.754 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:43.752+0000 7f8567400640 1 -- 172.21.15.47:0/4216885630 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f85480a2fa0 msgr2=0x7f85480a5460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:24:43.754 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:43.752+0000 7f8567400640 1 --2- 172.21.15.47:0/4216885630 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f85480a2fa0 0x7f85480a5460 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f8560102700 tx=0x7f8554005f90 comp rx=0 tx=0).stop 2024-03-21T10:24:43.754 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:43.752+0000 7f8567400640 1 -- 172.21.15.47:0/4216885630 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f85601028a0 msgr2=0x7f856019a0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:24:43.754 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:43.752+0000 7f8567400640 1 --2- 172.21.15.47:0/4216885630 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f85601028a0 0x7f856019a0c0 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f854c000c00 tx=0x7f854c0042f0 comp rx=0 tx=0).stop 2024-03-21T10:24:43.754 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:43.753+0000 7f8567400640 1 -- 172.21.15.47:0/4216885630 shutdown_connections 2024-03-21T10:24:43.754 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:43.753+0000 7f8567400640 1 --2- 172.21.15.47:0/4216885630 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f85480a2fa0 0x7f85480a5460 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:24:43.754 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:43.753+0000 7f8567400640 1 --2- 172.21.15.47:0/4216885630 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8560103260 0x7f85601a1130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:24:43.754 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:43.753+0000 7f8567400640 1 --2- 172.21.15.47:0/4216885630 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f85601028a0 0x7f856019a0c0 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:24:43.754 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:43.753+0000 7f8567400640 1 --2- 172.21.15.47:0/4216885630 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f85601016b0 0x7f8560199b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:24:43.754 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:43.753+0000 7f8567400640 1 -- 172.21.15.47:0/4216885630 >> 172.21.15.47:0/4216885630 conn(0x7f85600fcc40 msgr2=0x7f85600738d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:24:43.755 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:43.753+0000 7f8567400640 1 -- 172.21.15.47:0/4216885630 shutdown_connections 2024-03-21T10:24:43.755 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:43.753+0000 7f8567400640 1 -- 172.21.15.47:0/4216885630 wait complete. 2024-03-21T10:24:43.779 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-21T10:24:43.992 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:43.990+0000 7f74f6200640 1 -- 172.21.15.47:0/1628760066 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f74f00756a0 msgr2=0x7f74f0073af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:24:43.992 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:43.990+0000 7f74f6200640 1 --2- 172.21.15.47:0/1628760066 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f74f00756a0 0x7f74f0073af0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f74e0009900 tx=0x7f74e002bc30 comp rx=0 tx=0).stop 2024-03-21T10:24:43.992 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:43.990+0000 7f74f6200640 1 -- 172.21.15.47:0/1628760066 shutdown_connections 2024-03-21T10:24:43.992 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:43.990+0000 7f74f6200640 1 --2- 172.21.15.47:0/1628760066 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f74f0109560 0x7f74f010b9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:24:43.992 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:43.990+0000 7f74f6200640 1 --2- 172.21.15.47:0/1628760066 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f74f00740c0 0x7f74f0074540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:24:43.992 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:43.990+0000 7f74f6200640 1 --2- 172.21.15.47:0/1628760066 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f74f00756a0 0x7f74f0073af0 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:24:43.992 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:43.990+0000 7f74f6200640 1 -- 172.21.15.47:0/1628760066 >> 172.21.15.47:0/1628760066 conn(0x7f74f00fcc40 msgr2=0x7f74f00ff060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:24:43.992 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:43.991+0000 7f74f6200640 1 -- 172.21.15.47:0/1628760066 shutdown_connections 2024-03-21T10:24:43.992 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:43.991+0000 7f74f6200640 1 -- 172.21.15.47:0/1628760066 wait complete. 2024-03-21T10:24:43.994 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:43.992+0000 7f74f6200640 1 Processor -- start 2024-03-21T10:24:43.994 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:43.992+0000 7f74f6200640 1 -- start start 2024-03-21T10:24:43.994 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:43.992+0000 7f74f6200640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f74f00740c0 0x7f74f0199b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:24:43.994 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:43.992+0000 7f74f6200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f74f00756a0 0x7f74f019a050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:24:43.994 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:43.992+0000 7f74f6200640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f74f0109560 0x7f74f01a10c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:24:43.994 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:43.992+0000 7f74f6200640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74f010f410 con 0x7f74f0109560 2024-03-21T10:24:43.994 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:43.992+0000 7f74f6200640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74f010f290 con 0x7f74f00756a0 2024-03-21T10:24:43.994 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:43.992+0000 7f74f6200640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74f010f590 con 0x7f74f00740c0 2024-03-21T10:24:43.994 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:43.993+0000 7f74ef400640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f74f00740c0 0x7f74f0199b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:24:43.994 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:43.993+0000 7f74efe00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f74f0109560 0x7f74f01a10c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:24:43.994 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:43.993+0000 7f74eea00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f74f00756a0 0x7f74f019a050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:24:43.994 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:43.993+0000 7f74ef400640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f74f00740c0 0x7f74f0199b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.47:32830/0 (socket says 172.21.15.47:32830) 2024-03-21T10:24:43.995 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:43.993+0000 7f74efe00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f74f0109560 0x7f74f01a10c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:44206/0 (socket says 172.21.15.47:44206) 2024-03-21T10:24:43.995 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:43.993+0000 7f74ef400640 1 -- 172.21.15.47:0/1964306843 learned_addr learned my addr 172.21.15.47:0/1964306843 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:24:43.996 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:43.993+0000 7f74ef400640 1 -- 172.21.15.47:0/1964306843 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f74f00756a0 msgr2=0x7f74f019a050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:24:43.996 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:43.993+0000 7f74ef400640 1 --2- 172.21.15.47:0/1964306843 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f74f00756a0 0x7f74f019a050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:24:43.996 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:43.993+0000 7f74ef400640 1 -- 172.21.15.47:0/1964306843 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f74f0109560 msgr2=0x7f74f01a10c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:24:43.996 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:43.993+0000 7f74ef400640 1 --2- 172.21.15.47:0/1964306843 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f74f0109560 0x7f74f01a10c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:24:43.996 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:43.993+0000 7f74ef400640 1 -- 172.21.15.47:0/1964306843 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f74f01a1730 con 0x7f74f00740c0 2024-03-21T10:24:43.996 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:43.993+0000 7f74eea00640 1 --2- 172.21.15.47:0/1964306843 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f74f00756a0 0x7f74f019a050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T10:24:43.996 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:43.993+0000 7f74efe00640 1 --2- 172.21.15.47:0/1964306843 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f74f0109560 0x7f74f01a10c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T10:24:43.996 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:43.994+0000 7f74ef400640 1 --2- 172.21.15.47:0/1964306843 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f74f00740c0 0x7f74f0199b10 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f74e0000c00 tx=0x7f74e00040c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:24:43.996 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:43.994+0000 7f74e7e00640 1 -- 172.21.15.47:0/1964306843 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74e0043020 con 0x7f74f00740c0 2024-03-21T10:24:43.996 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:43.994+0000 7f74e7e00640 1 -- 172.21.15.47:0/1964306843 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f74e00044c0 con 0x7f74f00740c0 2024-03-21T10:24:43.996 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:43.994+0000 7f74f6200640 1 -- 172.21.15.47:0/1964306843 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f74f01a19c0 con 0x7f74f00740c0 2024-03-21T10:24:43.996 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:43.994+0000 7f74e7e00640 1 -- 172.21.15.47:0/1964306843 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74e0003c60 con 0x7f74f00740c0 2024-03-21T10:24:43.996 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:43.994+0000 7f74f6200640 1 -- 172.21.15.47:0/1964306843 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f74f01a1f60 con 0x7f74f00740c0 2024-03-21T10:24:43.997 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:43.995+0000 7f74f6200640 1 -- 172.21.15.47:0/1964306843 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f74f00688e0 con 0x7f74f00740c0 2024-03-21T10:24:43.998 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:43.997+0000 7f74e7e00640 1 -- 172.21.15.47:0/1964306843 <== mon.1 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 18) v1 ==== 137854+0+0 (secure 0 0 0) 0x7f74e0046020 con 0x7f74f00740c0 2024-03-21T10:24:44.000 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:43.999+0000 7f74e7e00640 1 --2- 172.21.15.47:0/1964306843 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f74d40a2fa0 0x7f74d40a5460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:24:44.000 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:43.999+0000 7f74e7e00640 1 -- 172.21.15.47:0/1964306843 <== mon.1 v2:172.21.15.157:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9216+0+0 (secure 0 0 0) 0x7f74e00f19a0 con 0x7f74f00740c0 2024-03-21T10:24:44.000 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:43.999+0000 7f74eea00640 1 --2- 172.21.15.47:0/1964306843 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f74d40a2fa0 0x7f74d40a5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:24:44.001 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:44.000+0000 7f74eea00640 1 --2- 172.21.15.47:0/1964306843 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f74d40a2fa0 0x7f74d40a5460 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f74f019b020 tx=0x7f74dc005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:24:44.005 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:44.004+0000 7f74e7e00640 1 -- 172.21.15.47:0/1964306843 <== mon.1 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f74e004d020 con 0x7f74f00740c0 2024-03-21T10:24:44.261 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:44.260+0000 7f74f6200640 1 -- 172.21.15.47:0/1964306843 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f74f00622a0 con 0x7f74f00740c0 2024-03-21T10:24:44.261 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:44.261+0000 7f74e7e00640 1 -- 172.21.15.47:0/1964306843 <== mon.1 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v95) v1 ==== 100+0+11 (secure 0 0 0) 0x7f74e00be3c0 con 0x7f74f00740c0 2024-03-21T10:24:44.262 INFO:teuthology.orchestra.run.smithi047.stdout:pg_num: 64 2024-03-21T10:24:44.264 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:44.263+0000 7f74f6200640 1 -- 172.21.15.47:0/1964306843 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f74d40a2fa0 msgr2=0x7f74d40a5460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:24:44.264 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:44.263+0000 7f74f6200640 1 --2- 172.21.15.47:0/1964306843 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f74d40a2fa0 0x7f74d40a5460 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f74f019b020 tx=0x7f74dc005f90 comp rx=0 tx=0).stop 2024-03-21T10:24:44.264 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:44.263+0000 7f74f6200640 1 -- 172.21.15.47:0/1964306843 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f74f00740c0 msgr2=0x7f74f0199b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:24:44.264 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:44.263+0000 7f74f6200640 1 --2- 172.21.15.47:0/1964306843 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f74f00740c0 0x7f74f0199b10 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f74e0000c00 tx=0x7f74e00040c0 comp rx=0 tx=0).stop 2024-03-21T10:24:44.265 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:44.263+0000 7f74f6200640 1 -- 172.21.15.47:0/1964306843 shutdown_connections 2024-03-21T10:24:44.265 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:44.263+0000 7f74f6200640 1 --2- 172.21.15.47:0/1964306843 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f74d40a2fa0 0x7f74d40a5460 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:24:44.265 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:44.263+0000 7f74f6200640 1 --2- 172.21.15.47:0/1964306843 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f74f0109560 0x7f74f01a10c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:24:44.265 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:44.263+0000 7f74f6200640 1 --2- 172.21.15.47:0/1964306843 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f74f00756a0 0x7f74f019a050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:24:44.265 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:44.263+0000 7f74f6200640 1 --2- 172.21.15.47:0/1964306843 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f74f00740c0 0x7f74f0199b10 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:24:44.265 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:44.263+0000 7f74f6200640 1 -- 172.21.15.47:0/1964306843 >> 172.21.15.47:0/1964306843 conn(0x7f74f00fcc40 msgr2=0x7f74f01054b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:24:44.265 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:44.263+0000 7f74f6200640 1 -- 172.21.15.47:0/1964306843 shutdown_connections 2024-03-21T10:24:44.265 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:44.263+0000 7f74f6200640 1 -- 172.21.15.47:0/1964306843 wait complete. 2024-03-21T10:24:44.288 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-03-21T10:24:44.487 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:44.486+0000 7f0a2e200640 1 -- 172.21.15.47:0/3731520460 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0a281028a0 msgr2=0x7f0a28102d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:24:44.487 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:44.486+0000 7f0a2e200640 1 --2- 172.21.15.47:0/3731520460 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0a281028a0 0x7f0a28102d20 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f0a14009900 tx=0x7f0a1402bc30 comp rx=0 tx=0).stop 2024-03-21T10:24:44.488 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:44.486+0000 7f0a2e200640 1 -- 172.21.15.47:0/3731520460 shutdown_connections 2024-03-21T10:24:44.488 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:44.486+0000 7f0a2e200640 1 --2- 172.21.15.47:0/3731520460 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0a28103260 0x7f0a28109af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:24:44.488 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:44.486+0000 7f0a2e200640 1 --2- 172.21.15.47:0/3731520460 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0a281028a0 0x7f0a28102d20 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:24:44.488 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:44.486+0000 7f0a2e200640 1 --2- 172.21.15.47:0/3731520460 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0a281016b0 0x7f0a28101ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:24:44.488 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:44.486+0000 7f0a2e200640 1 -- 172.21.15.47:0/3731520460 >> 172.21.15.47:0/3731520460 conn(0x7f0a280fcc40 msgr2=0x7f0a280ff060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:24:44.488 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:44.486+0000 7f0a2e200640 1 -- 172.21.15.47:0/3731520460 shutdown_connections 2024-03-21T10:24:44.488 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:44.486+0000 7f0a2e200640 1 -- 172.21.15.47:0/3731520460 wait complete. 2024-03-21T10:24:44.489 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:44.487+0000 7f0a2e200640 1 Processor -- start 2024-03-21T10:24:44.489 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:44.487+0000 7f0a2e200640 1 -- start start 2024-03-21T10:24:44.489 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:44.487+0000 7f0a2e200640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0a281016b0 0x7f0a28199b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:24:44.489 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:44.487+0000 7f0a2e200640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0a281028a0 0x7f0a2819a080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:24:44.490 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:44.487+0000 7f0a2e200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0a28103260 0x7f0a281a10f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:24:44.490 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:44.488+0000 7f0a2e200640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a2810d230 con 0x7f0a281016b0 2024-03-21T10:24:44.490 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:44.488+0000 7f0a2e200640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a2810d0b0 con 0x7f0a28103260 2024-03-21T10:24:44.490 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:44.488+0000 7f0a2e200640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a2810d3b0 con 0x7f0a281028a0 2024-03-21T10:24:44.490 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:44.488+0000 7f0a27400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0a281016b0 0x7f0a28199b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:24:44.490 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:44.488+0000 7f0a27400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0a281016b0 0x7f0a28199b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:44222/0 (socket says 172.21.15.47:44222) 2024-03-21T10:24:44.490 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:44.488+0000 7f0a27400640 1 -- 172.21.15.47:0/405536471 learned_addr learned my addr 172.21.15.47:0/405536471 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:24:44.490 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:44.488+0000 7f0a26a00640 1 --2- 172.21.15.47:0/405536471 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0a281028a0 0x7f0a2819a080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:24:44.490 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:44.488+0000 7f0a27e00640 1 --2- 172.21.15.47:0/405536471 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0a28103260 0x7f0a281a10f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:24:44.490 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:44.488+0000 7f0a27400640 1 -- 172.21.15.47:0/405536471 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0a281028a0 msgr2=0x7f0a2819a080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:24:44.490 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:44.488+0000 7f0a27400640 1 --2- 172.21.15.47:0/405536471 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0a281028a0 0x7f0a2819a080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:24:44.490 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:44.488+0000 7f0a27400640 1 -- 172.21.15.47:0/405536471 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0a28103260 msgr2=0x7f0a281a10f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:24:44.490 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:44.488+0000 7f0a27400640 1 --2- 172.21.15.47:0/405536471 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0a28103260 0x7f0a281a10f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:24:44.491 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:44.488+0000 7f0a27400640 1 -- 172.21.15.47:0/405536471 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a281a1760 con 0x7f0a281016b0 2024-03-21T10:24:44.491 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:44.489+0000 7f0a27400640 1 --2- 172.21.15.47:0/405536471 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0a281016b0 0x7f0a28199b40 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f0a1800d770 tx=0x7f0a1800dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:24:44.491 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:44.489+0000 7f0a1fe00640 1 -- 172.21.15.47:0/405536471 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a18017070 con 0x7f0a281016b0 2024-03-21T10:24:44.491 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:44.489+0000 7f0a1fe00640 1 -- 172.21.15.47:0/405536471 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f0a18009c30 con 0x7f0a281016b0 2024-03-21T10:24:44.492 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:44.489+0000 7f0a1fe00640 1 -- 172.21.15.47:0/405536471 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a18003d00 con 0x7f0a281016b0 2024-03-21T10:24:44.492 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:44.489+0000 7f0a2e200640 1 -- 172.21.15.47:0/405536471 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0a281a1a50 con 0x7f0a281016b0 2024-03-21T10:24:44.492 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:44.489+0000 7f0a2e200640 1 -- 172.21.15.47:0/405536471 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0a281a1fc0 con 0x7f0a281016b0 2024-03-21T10:24:44.492 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:44.491+0000 7f0a2e200640 1 -- 172.21.15.47:0/405536471 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0a281a21f0 con 0x7f0a281016b0 2024-03-21T10:24:44.494 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:44.493+0000 7f0a1fe00640 1 -- 172.21.15.47:0/405536471 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 18) v1 ==== 137854+0+0 (secure 0 0 0) 0x7f0a18003ea0 con 0x7f0a281016b0 2024-03-21T10:24:44.495 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:44.494+0000 7f0a1fe00640 1 --2- 172.21.15.47:0/405536471 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f0a0c0a2fa0 0x7f0a0c0a5460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:24:44.495 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:44.495+0000 7f0a1fe00640 1 -- 172.21.15.47:0/405536471 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9216+0+0 (secure 0 0 0) 0x7f0a180cf3f0 con 0x7f0a281016b0 2024-03-21T10:24:44.496 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:44.495+0000 7f0a26a00640 1 --2- 172.21.15.47:0/405536471 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f0a0c0a2fa0 0x7f0a0c0a5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:24:44.496 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:44.496+0000 7f0a26a00640 1 --2- 172.21.15.47:0/405536471 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f0a0c0a2fa0 0x7f0a0c0a5460 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f0a14000c00 tx=0x7f0a1402c7e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:24:44.500 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:44.500+0000 7f0a1fe00640 1 -- 172.21.15.47:0/405536471 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f0a18009de0 con 0x7f0a281016b0 2024-03-21T10:24:44.708 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:44.707+0000 7f0a2e200640 1 -- 172.21.15.47:0/405536471 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f0a281a2580 con 0x7f0a281016b0 2024-03-21T10:24:44.709 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:44.708+0000 7f0a1fe00640 1 -- 172.21.15.47:0/405536471 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v95) v1 ==== 103+0+11 (secure 0 0 0) 0x7f0a1809b7c0 con 0x7f0a281016b0 2024-03-21T10:24:44.709 INFO:teuthology.orchestra.run.smithi047.stdout:pg_num: 32 2024-03-21T10:24:44.713 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:44.712+0000 7f0a2e200640 1 -- 172.21.15.47:0/405536471 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f0a0c0a2fa0 msgr2=0x7f0a0c0a5460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:24:44.713 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:44.712+0000 7f0a2e200640 1 --2- 172.21.15.47:0/405536471 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f0a0c0a2fa0 0x7f0a0c0a5460 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f0a14000c00 tx=0x7f0a1402c7e0 comp rx=0 tx=0).stop 2024-03-21T10:24:44.713 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:44.712+0000 7f0a2e200640 1 -- 172.21.15.47:0/405536471 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0a281016b0 msgr2=0x7f0a28199b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:24:44.713 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:44.712+0000 7f0a2e200640 1 --2- 172.21.15.47:0/405536471 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0a281016b0 0x7f0a28199b40 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f0a1800d770 tx=0x7f0a1800dc30 comp rx=0 tx=0).stop 2024-03-21T10:24:44.714 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:24:44 smithi047 ceph-mon[31547]: from='client.? 172.21.15.47:0/2619460381' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-21T10:24:44.714 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:24:44 smithi047 ceph-mon[31547]: from='client.? 172.21.15.47:0/4216885630' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-21T10:24:44.714 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:24:44 smithi047 ceph-mon[31547]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:24:44.714 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:24:44 smithi047 ceph-mon[31547]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:24:44.715 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:44.713+0000 7f0a2e200640 1 -- 172.21.15.47:0/405536471 shutdown_connections 2024-03-21T10:24:44.715 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:44.713+0000 7f0a2e200640 1 --2- 172.21.15.47:0/405536471 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f0a0c0a2fa0 0x7f0a0c0a5460 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:24:44.715 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:44.713+0000 7f0a2e200640 1 --2- 172.21.15.47:0/405536471 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0a28103260 0x7f0a281a10f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:24:44.715 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:44.713+0000 7f0a2e200640 1 --2- 172.21.15.47:0/405536471 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0a281028a0 0x7f0a2819a080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:24:44.715 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:44.713+0000 7f0a2e200640 1 --2- 172.21.15.47:0/405536471 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0a281016b0 0x7f0a28199b40 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:24:44.715 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:44.713+0000 7f0a2e200640 1 -- 172.21.15.47:0/405536471 >> 172.21.15.47:0/405536471 conn(0x7f0a280fcc40 msgr2=0x7f0a280738d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:24:44.715 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:44.713+0000 7f0a2e200640 1 -- 172.21.15.47:0/405536471 shutdown_connections 2024-03-21T10:24:44.715 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:44.713+0000 7f0a2e200640 1 -- 172.21.15.47:0/405536471 wait complete. 2024-03-21T10:24:44.729 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:24:44 smithi100 ceph-mon[35193]: from='client.? 172.21.15.47:0/2619460381' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-21T10:24:44.729 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:24:44 smithi100 ceph-mon[35193]: from='client.? 172.21.15.47:0/4216885630' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-21T10:24:44.729 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:24:44 smithi100 ceph-mon[35193]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:24:44.729 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:24:44 smithi100 ceph-mon[35193]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:24:44.739 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:24:44 smithi157 ceph-mon[35581]: from='client.? 172.21.15.47:0/2619460381' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-21T10:24:44.739 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:24:44 smithi157 ceph-mon[35581]: from='client.? 172.21.15.47:0/4216885630' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-21T10:24:44.740 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:24:44 smithi157 ceph-mon[35581]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:24:44.740 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:24:44 smithi157 ceph-mon[35581]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:24:44.743 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-21T10:24:44.969 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:44.968+0000 7f3df7200640 1 -- 172.21.15.47:0/4150853381 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3df80756a0 msgr2=0x7f3df8073af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:24:44.969 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:44.968+0000 7f3df7200640 1 --2- 172.21.15.47:0/4150853381 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3df80756a0 0x7f3df8073af0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f3de8009900 tx=0x7f3de802bc30 comp rx=0 tx=0).stop 2024-03-21T10:24:44.969 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:44.968+0000 7f3df7200640 1 -- 172.21.15.47:0/4150853381 shutdown_connections 2024-03-21T10:24:44.969 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:44.968+0000 7f3df7200640 1 --2- 172.21.15.47:0/4150853381 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3df81093b0 0x7f3df810b820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:24:44.970 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:44.968+0000 7f3df7200640 1 --2- 172.21.15.47:0/4150853381 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3df80740c0 0x7f3df8074540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:24:44.970 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:44.968+0000 7f3df7200640 1 --2- 172.21.15.47:0/4150853381 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3df80756a0 0x7f3df8073af0 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:24:44.970 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:44.968+0000 7f3df7200640 1 -- 172.21.15.47:0/4150853381 >> 172.21.15.47:0/4150853381 conn(0x7f3df80fca70 msgr2=0x7f3df80feeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:24:44.970 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:44.968+0000 7f3df7200640 1 -- 172.21.15.47:0/4150853381 shutdown_connections 2024-03-21T10:24:44.970 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:44.968+0000 7f3df7200640 1 -- 172.21.15.47:0/4150853381 wait complete. 2024-03-21T10:24:44.971 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:44.969+0000 7f3df7200640 1 Processor -- start 2024-03-21T10:24:44.972 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:44.969+0000 7f3df7200640 1 -- start start 2024-03-21T10:24:44.972 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:44.969+0000 7f3df7200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3df80740c0 0x7f3df8199980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:24:44.972 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:44.969+0000 7f3df7200640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3df80756a0 0x7f3df8199ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:24:44.972 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:44.970+0000 7f3df7200640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3df81093b0 0x7f3df81a0ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:24:44.972 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:44.970+0000 7f3df7200640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3df810f220 con 0x7f3df80756a0 2024-03-21T10:24:44.972 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:44.970+0000 7f3df7200640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3df810f0a0 con 0x7f3df80740c0 2024-03-21T10:24:44.972 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:44.970+0000 7f3df7200640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3df810f3a0 con 0x7f3df81093b0 2024-03-21T10:24:44.972 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:44.970+0000 7f3df5e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3df80740c0 0x7f3df8199980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:24:44.972 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:44.970+0000 7f3df5400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3df80756a0 0x7f3df8199ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:24:44.972 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:44.970+0000 7f3df5e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3df80740c0 0x7f3df8199980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:56256/0 (socket says 172.21.15.47:56256) 2024-03-21T10:24:44.972 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:44.970+0000 7f3df5e00640 1 -- 172.21.15.47:0/622781934 learned_addr learned my addr 172.21.15.47:0/622781934 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:24:44.972 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:44.970+0000 7f3df6800640 1 --2- 172.21.15.47:0/622781934 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3df81093b0 0x7f3df81a0ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:24:44.973 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:44.970+0000 7f3df5e00640 1 -- 172.21.15.47:0/622781934 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3df81093b0 msgr2=0x7f3df81a0ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:24:44.973 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:44.970+0000 7f3df5e00640 1 --2- 172.21.15.47:0/622781934 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3df81093b0 0x7f3df81a0ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:24:44.973 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:44.970+0000 7f3df5e00640 1 -- 172.21.15.47:0/622781934 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3df80756a0 msgr2=0x7f3df8199ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:24:44.973 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:44.970+0000 7f3df5e00640 1 --2- 172.21.15.47:0/622781934 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3df80756a0 0x7f3df8199ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:24:44.973 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:44.970+0000 7f3df5e00640 1 -- 172.21.15.47:0/622781934 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3df81a1550 con 0x7f3df80740c0 2024-03-21T10:24:44.973 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:44.970+0000 7f3df6800640 1 --2- 172.21.15.47:0/622781934 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3df81093b0 0x7f3df81a0ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T10:24:44.973 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:44.970+0000 7f3df5400640 1 --2- 172.21.15.47:0/622781934 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3df80756a0 0x7f3df8199ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T10:24:44.973 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:44.971+0000 7f3df5e00640 1 --2- 172.21.15.47:0/622781934 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3df80740c0 0x7f3df8199980 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f3de8006010 tx=0x7f3de8005e40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:24:44.974 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:44.971+0000 7f3deea00640 1 -- 172.21.15.47:0/622781934 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3de8043020 con 0x7f3df80740c0 2024-03-21T10:24:44.974 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:44.971+0000 7f3df7200640 1 -- 172.21.15.47:0/622781934 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3df81a17e0 con 0x7f3df80740c0 2024-03-21T10:24:44.974 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:44.971+0000 7f3deea00640 1 -- 172.21.15.47:0/622781934 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f3de80043d0 con 0x7f3df80740c0 2024-03-21T10:24:44.974 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:44.971+0000 7f3deea00640 1 -- 172.21.15.47:0/622781934 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3de8003b80 con 0x7f3df80740c0 2024-03-21T10:24:44.974 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:44.971+0000 7f3df7200640 1 -- 172.21.15.47:0/622781934 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3df81a1d30 con 0x7f3df80740c0 2024-03-21T10:24:44.975 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:44.972+0000 7f3df7200640 1 -- 172.21.15.47:0/622781934 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3df8108130 con 0x7f3df80740c0 2024-03-21T10:24:44.977 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:44.976+0000 7f3deea00640 1 -- 172.21.15.47:0/622781934 <== mon.2 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 18) v1 ==== 137854+0+0 (secure 0 0 0) 0x7f3de8046020 con 0x7f3df80740c0 2024-03-21T10:24:44.978 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:44.978+0000 7f3deea00640 1 --2- 172.21.15.47:0/622781934 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f3ddc0a2f20 0x7f3ddc0a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:24:44.979 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:44.978+0000 7f3deea00640 1 -- 172.21.15.47:0/622781934 <== mon.2 v2:172.21.15.100:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9216+0+0 (secure 0 0 0) 0x7f3de80f1a80 con 0x7f3df80740c0 2024-03-21T10:24:44.982 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:44.978+0000 7f3df5400640 1 --2- 172.21.15.47:0/622781934 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f3ddc0a2f20 0x7f3ddc0a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:24:44.982 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:44.979+0000 7f3df5400640 1 --2- 172.21.15.47:0/622781934 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f3ddc0a2f20 0x7f3ddc0a53e0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f3df819ae40 tx=0x7f3de4005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:24:44.990 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:44.989+0000 7f3deea00640 1 -- 172.21.15.47:0/622781934 <== mon.2 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f3de80f3e20 con 0x7f3df80740c0 2024-03-21T10:24:45.258 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:45.257+0000 7f3df7200640 1 -- 172.21.15.47:0/622781934 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f3df802d070 con 0x7f3df80740c0 2024-03-21T10:24:45.259 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:45.258+0000 7f3deea00640 1 -- 172.21.15.47:0/622781934 <== mon.2 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) v1 ==== 94+0+11471 (secure 0 0 0) 0x7f3de804d020 con 0x7f3df80740c0 2024-03-21T10:24:45.260 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-21T10:24:45.260 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":29,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14511,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6835/735724037","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6834","nonce":735724037},{"type":"v1","addr":"172.21.15.47:6835","nonce":735724037}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14529,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6837/2248191724","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6836","nonce":2248191724},{"type":"v1","addr":"172.21.15.47:6837","nonce":2248191724}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":29,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-21T10:21:58.329063+0000","modified":"2024-03-21T10:22:29.588509+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":34535,"mds_1":34493,"mds_2":34454,"mds_3":34427,"mds_4":14619},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14604":{"gid":14604,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.47:6839/2881335356","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6838","nonce":2881335356},{"type":"v1","addr":"172.21.15.47:6839","nonce":2881335356}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_14619":{"gid":14619,"name":"a","rank":4,"incarnation":27,"state":"up:active","state_seq":11,"addr":"172.21.15.47:6841/3401578978","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6840","nonce":3401578978},{"type":"v1","addr":"172.21.15.47:6841","nonce":3401578978}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24406":{"gid":24406,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.157:6833/4000010136","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":4000010136},{"type":"v1","addr":"172.21.15.157:6833","nonce":4000010136}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24457":{"gid":24457,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.157:6835/3902868651","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":3902868651},{"type":"v1","addr":"172.21.15.157:6835","nonce":3902868651}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24475":{"gid":24475,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.157:6837/1908087204","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6836","nonce":1908087204},{"type":"v1","addr":"172.21.15.157:6837","nonce":1908087204}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24499":{"gid":24499,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.157:6839/4064242876","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6838","nonce":4064242876},{"type":"v1","addr":"172.21.15.157:6839","nonce":4064242876}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_34427":{"gid":34427,"name":"k","rank":3,"incarnation":25,"state":"up:active","state_seq":20,"addr":"172.21.15.100:6833/2723302038","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6832","nonce":2723302038},{"type":"v1","addr":"172.21.15.100:6833","nonce":2723302038}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_34454":{"gid":34454,"name":"e","rank":2,"incarnation":22,"state":"up:active","state_seq":17,"addr":"172.21.15.100:6835/2597849957","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":2597849957},{"type":"v1","addr":"172.21.15.100:6835","nonce":2597849957}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_34493":{"gid":34493,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":13,"addr":"172.21.15.100:6837/3497644162","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6836","nonce":3497644162},{"type":"v1","addr":"172.21.15.100:6837","nonce":3497644162}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_34535":{"gid":34535,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.100:6839/4173168440","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6838","nonce":4173168440},{"type":"v1","addr":"172.21.15.100:6839","nonce":4173168440}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":34535,"qdb_cluster":[34493,34535,34454,34427,14619]},"id":1}]} 2024-03-21T10:24:45.260 INFO:teuthology.orchestra.run.smithi047.stderr:dumped fsmap epoch 29 2024-03-21T10:24:45.263 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:45.262+0000 7f3df7200640 1 -- 172.21.15.47:0/622781934 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f3ddc0a2f20 msgr2=0x7f3ddc0a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:24:45.263 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:45.262+0000 7f3df7200640 1 --2- 172.21.15.47:0/622781934 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f3ddc0a2f20 0x7f3ddc0a53e0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f3df819ae40 tx=0x7f3de4005f90 comp rx=0 tx=0).stop 2024-03-21T10:24:45.263 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:45.262+0000 7f3df7200640 1 -- 172.21.15.47:0/622781934 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3df80740c0 msgr2=0x7f3df8199980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:24:45.263 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:45.262+0000 7f3df7200640 1 --2- 172.21.15.47:0/622781934 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3df80740c0 0x7f3df8199980 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f3de8006010 tx=0x7f3de8005e40 comp rx=0 tx=0).stop 2024-03-21T10:24:45.263 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:45.262+0000 7f3df7200640 1 -- 172.21.15.47:0/622781934 shutdown_connections 2024-03-21T10:24:45.264 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:45.262+0000 7f3df7200640 1 --2- 172.21.15.47:0/622781934 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f3ddc0a2f20 0x7f3ddc0a53e0 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:24:45.264 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:45.262+0000 7f3df7200640 1 --2- 172.21.15.47:0/622781934 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3df81093b0 0x7f3df81a0ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:24:45.264 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:45.262+0000 7f3df7200640 1 --2- 172.21.15.47:0/622781934 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3df80756a0 0x7f3df8199ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:24:45.264 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:45.262+0000 7f3df7200640 1 --2- 172.21.15.47:0/622781934 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3df80740c0 0x7f3df8199980 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:24:45.264 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:45.262+0000 7f3df7200640 1 -- 172.21.15.47:0/622781934 >> 172.21.15.47:0/622781934 conn(0x7f3df80fca70 msgr2=0x7f3df8105300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:24:45.264 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:45.262+0000 7f3df7200640 1 -- 172.21.15.47:0/622781934 shutdown_connections 2024-03-21T10:24:45.264 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:45.262+0000 7f3df7200640 1 -- 172.21.15.47:0/622781934 wait complete. 2024-03-21T10:24:45.283 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-21T10:24:45.481 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:45.480+0000 7f04d6200640 1 -- 172.21.15.47:0/2609211049 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f04d0102160 msgr2=0x7f04d0104550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:24:45.481 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:45.480+0000 7f04d6200640 1 --2- 172.21.15.47:0/2609211049 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f04d0102160 0x7f04d0104550 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f04c0009860 tx=0x7f04c002bb70 comp rx=0 tx=0).stop 2024-03-21T10:24:45.481 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:45.480+0000 7f04d6200640 1 -- 172.21.15.47:0/2609211049 shutdown_connections 2024-03-21T10:24:45.481 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:45.480+0000 7f04d6200640 1 --2- 172.21.15.47:0/2609211049 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f04d01073c0 0x7f04d0109810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:24:45.482 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:45.480+0000 7f04d6200640 1 --2- 172.21.15.47:0/2609211049 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f04d0104a90 0x7f04d0106e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:24:45.482 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:45.480+0000 7f04d6200640 1 --2- 172.21.15.47:0/2609211049 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f04d0102160 0x7f04d0104550 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:24:45.482 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:45.480+0000 7f04d6200640 1 -- 172.21.15.47:0/2609211049 >> 172.21.15.47:0/2609211049 conn(0x7f04d00fbb50 msgr2=0x7f04d00fdf90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:24:45.482 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:45.480+0000 7f04d6200640 1 -- 172.21.15.47:0/2609211049 shutdown_connections 2024-03-21T10:24:45.484 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:45.484+0000 7f04d6200640 1 -- 172.21.15.47:0/2609211049 wait complete. 2024-03-21T10:24:45.486 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:45.485+0000 7f04d6200640 1 Processor -- start 2024-03-21T10:24:45.487 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:45.485+0000 7f04d6200640 1 -- start start 2024-03-21T10:24:45.487 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:45.485+0000 7f04d6200640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f04d0102160 0x7f04d0199b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:24:45.487 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:45.485+0000 7f04d6200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f04d0104a90 0x7f04d019a080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:24:45.487 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:45.485+0000 7f04d6200640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f04d01073c0 0x7f04d01a10f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:24:45.487 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:45.485+0000 7f04d6200640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04d010d230 con 0x7f04d0102160 2024-03-21T10:24:45.487 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:45.485+0000 7f04d6200640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04d010d0b0 con 0x7f04d0104a90 2024-03-21T10:24:45.487 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:45.485+0000 7f04d6200640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04d010d3b0 con 0x7f04d01073c0 2024-03-21T10:24:45.487 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:45.485+0000 7f04cf400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f04d0102160 0x7f04d0199b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:24:45.487 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:45.485+0000 7f04cea00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f04d0104a90 0x7f04d019a080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:24:45.487 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:45.486+0000 7f04cf400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f04d0102160 0x7f04d0199b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:44242/0 (socket says 172.21.15.47:44242) 2024-03-21T10:24:45.487 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:45.486+0000 7f04cf400640 1 -- 172.21.15.47:0/2287603473 learned_addr learned my addr 172.21.15.47:0/2287603473 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:24:45.487 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:45.486+0000 7f04cfe00640 1 --2- 172.21.15.47:0/2287603473 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f04d01073c0 0x7f04d01a10f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:24:45.488 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:45.486+0000 7f04cf400640 1 -- 172.21.15.47:0/2287603473 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f04d01073c0 msgr2=0x7f04d01a10f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:24:45.489 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:45.486+0000 7f04cf400640 1 --2- 172.21.15.47:0/2287603473 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f04d01073c0 0x7f04d01a10f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:24:45.489 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:45.486+0000 7f04cf400640 1 -- 172.21.15.47:0/2287603473 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f04d0104a90 msgr2=0x7f04d019a080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:24:45.489 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:45.486+0000 7f04cf400640 1 --2- 172.21.15.47:0/2287603473 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f04d0104a90 0x7f04d019a080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:24:45.489 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:45.486+0000 7f04cf400640 1 -- 172.21.15.47:0/2287603473 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f04d01a1760 con 0x7f04d0102160 2024-03-21T10:24:45.489 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:45.486+0000 7f04cea00640 1 --2- 172.21.15.47:0/2287603473 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f04d0104a90 0x7f04d019a080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T10:24:45.489 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:45.486+0000 7f04cfe00640 1 --2- 172.21.15.47:0/2287603473 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f04d01073c0 0x7f04d01a10f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T10:24:45.489 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:45.487+0000 7f04cf400640 1 --2- 172.21.15.47:0/2287603473 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f04d0102160 0x7f04d0199b40 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f04c00057c0 tx=0x7f04c00042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:24:45.489 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:45.487+0000 7f04c7e00640 1 -- 172.21.15.47:0/2287603473 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04c0041030 con 0x7f04d0102160 2024-03-21T10:24:45.489 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:45.487+0000 7f04c7e00640 1 -- 172.21.15.47:0/2287603473 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f04c002c9c0 con 0x7f04d0102160 2024-03-21T10:24:45.489 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:45.487+0000 7f04d6200640 1 -- 172.21.15.47:0/2287603473 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f04d01a19f0 con 0x7f04d0102160 2024-03-21T10:24:45.490 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:45.487+0000 7f04c7e00640 1 -- 172.21.15.47:0/2287603473 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04c0039040 con 0x7f04d0102160 2024-03-21T10:24:45.490 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:45.487+0000 7f04d6200640 1 -- 172.21.15.47:0/2287603473 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f04d01a1ee0 con 0x7f04d0102160 2024-03-21T10:24:45.490 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:45.489+0000 7f04d6200640 1 -- 172.21.15.47:0/2287603473 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f04d0108f10 con 0x7f04d0102160 2024-03-21T10:24:45.491 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:45.491+0000 7f04c7e00640 1 -- 172.21.15.47:0/2287603473 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 18) v1 ==== 137854+0+0 (secure 0 0 0) 0x7f04c0044070 con 0x7f04d0102160 2024-03-21T10:24:45.494 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:45.493+0000 7f04c7e00640 1 --2- 172.21.15.47:0/2287603473 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f04b40a2f20 0x7f04b40a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:24:45.494 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:45.493+0000 7f04c7e00640 1 -- 172.21.15.47:0/2287603473 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9216+0+0 (secure 0 0 0) 0x7f04c00f0ee0 con 0x7f04d0102160 2024-03-21T10:24:45.494 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:45.493+0000 7f04cea00640 1 --2- 172.21.15.47:0/2287603473 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f04b40a2f20 0x7f04b40a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:24:45.495 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:45.494+0000 7f04cea00640 1 --2- 172.21.15.47:0/2287603473 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f04b40a2f20 0x7f04b40a53e0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f04d019b050 tx=0x7f04bc00b390 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:24:45.497 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:45.496+0000 7f04c7e00640 1 -- 172.21.15.47:0/2287603473 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f04c00bd900 con 0x7f04d0102160 2024-03-21T10:24:45.714 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:24:45 smithi047 ceph-mon[31547]: pgmap v454: 129 pgs: 129 active+clean; 642 KiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-03-21T10:24:45.714 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:24:45 smithi047 ceph-mon[31547]: from='client.? 172.21.15.47:0/1964306843' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-21T10:24:45.714 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:24:45 smithi047 ceph-mon[31547]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:24:45.714 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:24:45 smithi047 ceph-mon[31547]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:24:45.715 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:24:45 smithi047 ceph-mon[31547]: from='client.? 172.21.15.47:0/405536471' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-03-21T10:24:45.715 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:24:45 smithi047 ceph-mon[31547]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:24:45.715 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:24:45 smithi047 ceph-mon[31547]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:24:45.729 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:24:45 smithi100 ceph-mon[35193]: pgmap v454: 129 pgs: 129 active+clean; 642 KiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-03-21T10:24:45.729 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:24:45 smithi100 ceph-mon[35193]: from='client.? 172.21.15.47:0/1964306843' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-21T10:24:45.729 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:24:45 smithi100 ceph-mon[35193]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:24:45.729 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:24:45 smithi100 ceph-mon[35193]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:24:45.729 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:24:45 smithi100 ceph-mon[35193]: from='client.? 172.21.15.47:0/405536471' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-03-21T10:24:45.730 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:24:45 smithi100 ceph-mon[35193]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:24:45.730 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:24:45 smithi100 ceph-mon[35193]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:24:45.731 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:45.730+0000 7f04d6200640 1 -- 172.21.15.47:0/2287603473 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f04d00622a0 con 0x7f04d0102160 2024-03-21T10:24:45.733 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:45.732+0000 7f04c7e00640 1 -- 172.21.15.47:0/2287603473 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v95) v1 ==== 74+0+24849 (secure 0 0 0) 0x7f04c00bd200 con 0x7f04d0102160 2024-03-21T10:24:45.734 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-21T10:24:45.735 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":95,"fsid":"3056d51a-e76b-11ee-95cd-87774f69a715","created":"2024-03-21T10:11:17.019965+0000","modified":"2024-03-21T10:23:27.619804+0000","last_up_change":"2024-03-21T10:20:52.675328+0000","last_in_change":"2024-03-21T10:20:27.648212+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-21T10:15:40.553114+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-21T10:21:56.399518+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-21T10:21:57.812940+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-21T10:22:01.822568+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"91","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"b0c7a102-8d89-428b-b793-cbbe67151f50","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6802","nonce":775837313},{"type":"v1","addr":"172.21.15.47:6803","nonce":775837313}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6804","nonce":775837313},{"type":"v1","addr":"172.21.15.47:6805","nonce":775837313}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6808","nonce":775837313},{"type":"v1","addr":"172.21.15.47:6809","nonce":775837313}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6806","nonce":775837313},{"type":"v1","addr":"172.21.15.47:6807","nonce":775837313}]},"public_addr":"172.21.15.47:6803/775837313","cluster_addr":"172.21.15.47:6805/775837313","heartbeat_back_addr":"172.21.15.47:6809/775837313","heartbeat_front_addr":"172.21.15.47:6807/775837313","state":["exists","up"]},{"osd":1,"uuid":"07bb81bb-7cf3-4bbc-ac9e-802f95185d11","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6800","nonce":2923304710},{"type":"v1","addr":"172.21.15.100:6801","nonce":2923304710}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":2923304710},{"type":"v1","addr":"172.21.15.100:6803","nonce":2923304710}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":2923304710},{"type":"v1","addr":"172.21.15.100:6807","nonce":2923304710}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6804","nonce":2923304710},{"type":"v1","addr":"172.21.15.100:6805","nonce":2923304710}]},"public_addr":"172.21.15.100:6801/2923304710","cluster_addr":"172.21.15.100:6803/2923304710","heartbeat_back_addr":"172.21.15.100:6807/2923304710","heartbeat_front_addr":"172.21.15.100:6805/2923304710","state":["exists","up"]},{"osd":2,"uuid":"88ce76b9-6081-4836-aad0-8c249c56e8a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":171377005},{"type":"v1","addr":"172.21.15.157:6801","nonce":171377005}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":171377005},{"type":"v1","addr":"172.21.15.157:6803","nonce":171377005}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":171377005},{"type":"v1","addr":"172.21.15.157:6807","nonce":171377005}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":171377005},{"type":"v1","addr":"172.21.15.157:6805","nonce":171377005}]},"public_addr":"172.21.15.157:6801/171377005","cluster_addr":"172.21.15.157:6803/171377005","heartbeat_back_addr":"172.21.15.157:6807/171377005","heartbeat_front_addr":"172.21.15.157:6805/171377005","state":["exists","up"]},{"osd":3,"uuid":"518aa33d-263f-45e6-aab5-2201ceb40e74","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":27,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6810","nonce":640760101},{"type":"v1","addr":"172.21.15.47:6811","nonce":640760101}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6812","nonce":640760101},{"type":"v1","addr":"172.21.15.47:6813","nonce":640760101}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6816","nonce":640760101},{"type":"v1","addr":"172.21.15.47:6817","nonce":640760101}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6814","nonce":640760101},{"type":"v1","addr":"172.21.15.47:6815","nonce":640760101}]},"public_addr":"172.21.15.47:6811/640760101","cluster_addr":"172.21.15.47:6813/640760101","heartbeat_back_addr":"172.21.15.47:6817/640760101","heartbeat_front_addr":"172.21.15.47:6815/640760101","state":["exists","up"]},{"osd":4,"uuid":"987bde83-ecc6-42e5-bd56-c143b3de4226","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":35,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6808","nonce":2785478639},{"type":"v1","addr":"172.21.15.100:6809","nonce":2785478639}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6810","nonce":2785478639},{"type":"v1","addr":"172.21.15.100:6811","nonce":2785478639}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6814","nonce":2785478639},{"type":"v1","addr":"172.21.15.100:6815","nonce":2785478639}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":2785478639},{"type":"v1","addr":"172.21.15.100:6813","nonce":2785478639}]},"public_addr":"172.21.15.100:6809/2785478639","cluster_addr":"172.21.15.100:6811/2785478639","heartbeat_back_addr":"172.21.15.100:6815/2785478639","heartbeat_front_addr":"172.21.15.100:6813/2785478639","state":["exists","up"]},{"osd":5,"uuid":"012d2188-3a61-4a5f-8d01-2001c15ffbe2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":40,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":1525645410},{"type":"v1","addr":"172.21.15.157:6809","nonce":1525645410}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":1525645410},{"type":"v1","addr":"172.21.15.157:6811","nonce":1525645410}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":1525645410},{"type":"v1","addr":"172.21.15.157:6815","nonce":1525645410}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":1525645410},{"type":"v1","addr":"172.21.15.157:6813","nonce":1525645410}]},"public_addr":"172.21.15.157:6809/1525645410","cluster_addr":"172.21.15.157:6811/1525645410","heartbeat_back_addr":"172.21.15.157:6815/1525645410","heartbeat_front_addr":"172.21.15.157:6813/1525645410","state":["exists","up"]},{"osd":6,"uuid":"c12c14b4-85f0-4d88-937c-de4f556437de","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6818","nonce":2321642790},{"type":"v1","addr":"172.21.15.47:6819","nonce":2321642790}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6820","nonce":2321642790},{"type":"v1","addr":"172.21.15.47:6821","nonce":2321642790}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6824","nonce":2321642790},{"type":"v1","addr":"172.21.15.47:6825","nonce":2321642790}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6822","nonce":2321642790},{"type":"v1","addr":"172.21.15.47:6823","nonce":2321642790}]},"public_addr":"172.21.15.47:6819/2321642790","cluster_addr":"172.21.15.47:6821/2321642790","heartbeat_back_addr":"172.21.15.47:6825/2321642790","heartbeat_front_addr":"172.21.15.47:6823/2321642790","state":["exists","up"]},{"osd":7,"uuid":"3c87c6d0-1d1f-4294-9917-aaaad7dda824","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6816","nonce":4104727330},{"type":"v1","addr":"172.21.15.100:6817","nonce":4104727330}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":4104727330},{"type":"v1","addr":"172.21.15.100:6819","nonce":4104727330}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":4104727330},{"type":"v1","addr":"172.21.15.100:6823","nonce":4104727330}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6820","nonce":4104727330},{"type":"v1","addr":"172.21.15.100:6821","nonce":4104727330}]},"public_addr":"172.21.15.100:6817/4104727330","cluster_addr":"172.21.15.100:6819/4104727330","heartbeat_back_addr":"172.21.15.100:6823/4104727330","heartbeat_front_addr":"172.21.15.100:6821/4104727330","state":["exists","up"]},{"osd":8,"uuid":"502d65cf-53c8-4936-8de4-e11b2a6e1e18","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":61,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":4067715347},{"type":"v1","addr":"172.21.15.157:6817","nonce":4067715347}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":4067715347},{"type":"v1","addr":"172.21.15.157:6819","nonce":4067715347}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":4067715347},{"type":"v1","addr":"172.21.15.157:6823","nonce":4067715347}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":4067715347},{"type":"v1","addr":"172.21.15.157:6821","nonce":4067715347}]},"public_addr":"172.21.15.157:6817/4067715347","cluster_addr":"172.21.15.157:6819/4067715347","heartbeat_back_addr":"172.21.15.157:6823/4067715347","heartbeat_front_addr":"172.21.15.157:6821/4067715347","state":["exists","up"]},{"osd":9,"uuid":"5ea1f5be-8458-4c93-9e61-c2e867a514f3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6826","nonce":1982108738},{"type":"v1","addr":"172.21.15.47:6827","nonce":1982108738}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6828","nonce":1982108738},{"type":"v1","addr":"172.21.15.47:6829","nonce":1982108738}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6832","nonce":1982108738},{"type":"v1","addr":"172.21.15.47:6833","nonce":1982108738}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6830","nonce":1982108738},{"type":"v1","addr":"172.21.15.47:6831","nonce":1982108738}]},"public_addr":"172.21.15.47:6827/1982108738","cluster_addr":"172.21.15.47:6829/1982108738","heartbeat_back_addr":"172.21.15.47:6833/1982108738","heartbeat_front_addr":"172.21.15.47:6831/1982108738","state":["exists","up"]},{"osd":10,"uuid":"2da3da90-a0be-4a54-a9ff-b2d6c9b36870","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":74,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":396391028},{"type":"v1","addr":"172.21.15.100:6825","nonce":396391028}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":396391028},{"type":"v1","addr":"172.21.15.100:6827","nonce":396391028}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6830","nonce":396391028},{"type":"v1","addr":"172.21.15.100:6831","nonce":396391028}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":396391028},{"type":"v1","addr":"172.21.15.100:6829","nonce":396391028}]},"public_addr":"172.21.15.100:6825/396391028","cluster_addr":"172.21.15.100:6827/396391028","heartbeat_back_addr":"172.21.15.100:6831/396391028","heartbeat_front_addr":"172.21.15.100:6829/396391028","state":["exists","up"]},{"osd":11,"uuid":"11798d41-7f7e-4319-95b8-8d3d0946bde9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":82,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":2671029483},{"type":"v1","addr":"172.21.15.157:6825","nonce":2671029483}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":2671029483},{"type":"v1","addr":"172.21.15.157:6827","nonce":2671029483}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":2671029483},{"type":"v1","addr":"172.21.15.157:6831","nonce":2671029483}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":2671029483},{"type":"v1","addr":"172.21.15.157:6829","nonce":2671029483}]},"public_addr":"172.21.15.157:6825/2671029483","cluster_addr":"172.21.15.157:6827/2671029483","heartbeat_back_addr":"172.21.15.157:6831/2671029483","heartbeat_front_addr":"172.21.15.157:6829/2671029483","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T10:14:31.822399+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T10:15:04.175721+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T10:15:35.902595+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T10:16:12.088891+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T10:16:46.428704+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T10:17:20.158270+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T10:17:55.915253+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T10:18:30.064641+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T10:19:04.410004+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T10:19:40.605181+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T10:20:15.636605+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T10:20:50.151365+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.11","mappings":[{"from":7,"to":11}]},{"pgid":"2.14","mappings":[{"from":2,"to":4}]},{"pgid":"2.1a","mappings":[{"from":8,"to":1}]},{"pgid":"2.1b","mappings":[{"from":2,"to":4}]},{"pgid":"2.1d","mappings":[{"from":8,"to":11}]},{"pgid":"3.0","mappings":[{"from":4,"to":8},{"from":5,"to":9}]},{"pgid":"3.6","mappings":[{"from":3,"to":7}]},{"pgid":"3.1b","mappings":[{"from":4,"to":6}]},{"pgid":"3.33","mappings":[{"from":5,"to":9}]},{"pgid":"3.3c","mappings":[{"from":4,"to":9}]},{"pgid":"4.c","mappings":[{"from":9,"to":10}]},{"pgid":"4.f","mappings":[{"from":9,"to":8}]},{"pgid":"4.19","mappings":[{"from":9,"to":8}]},{"pgid":"4.1d","mappings":[{"from":5,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.47:0/356822431":"2024-03-22T10:12:25.844530+0000","172.21.15.47:0/1167490879":"2024-03-22T10:12:25.844530+0000","172.21.15.47:0/1540718138":"2024-03-22T10:12:25.844530+0000","172.21.15.47:6801/3836903972":"2024-03-22T10:12:25.844530+0000","172.21.15.47:0/235436602":"2024-03-22T10:11:50.301838+0000","172.21.15.47:0/2052815572":"2024-03-22T10:11:50.301838+0000","172.21.15.47:6800/3942658432":"2024-03-22T10:11:50.301838+0000","172.21.15.47:6801/3942658432":"2024-03-22T10:11:50.301838+0000","172.21.15.47:0/1419516257":"2024-03-22T10:11:50.301838+0000","172.21.15.47:6800/3836903972":"2024-03-22T10:12:25.844530+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-21T10:24:45.737 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:45.736+0000 7f04d6200640 1 -- 172.21.15.47:0/2287603473 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f04b40a2f20 msgr2=0x7f04b40a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:24:45.737 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:45.736+0000 7f04d6200640 1 --2- 172.21.15.47:0/2287603473 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f04b40a2f20 0x7f04b40a53e0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f04d019b050 tx=0x7f04bc00b390 comp rx=0 tx=0).stop 2024-03-21T10:24:45.737 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:45.736+0000 7f04d6200640 1 -- 172.21.15.47:0/2287603473 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f04d0102160 msgr2=0x7f04d0199b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:24:45.738 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:45.736+0000 7f04d6200640 1 --2- 172.21.15.47:0/2287603473 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f04d0102160 0x7f04d0199b40 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f04c00057c0 tx=0x7f04c00042f0 comp rx=0 tx=0).stop 2024-03-21T10:24:45.738 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:45.736+0000 7f04d6200640 1 -- 172.21.15.47:0/2287603473 shutdown_connections 2024-03-21T10:24:45.738 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:45.736+0000 7f04d6200640 1 --2- 172.21.15.47:0/2287603473 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f04b40a2f20 0x7f04b40a53e0 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:24:45.738 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:45.736+0000 7f04d6200640 1 --2- 172.21.15.47:0/2287603473 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f04d01073c0 0x7f04d01a10f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:24:45.738 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:45.736+0000 7f04d6200640 1 --2- 172.21.15.47:0/2287603473 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f04d0104a90 0x7f04d019a080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:24:45.738 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:45.736+0000 7f04d6200640 1 --2- 172.21.15.47:0/2287603473 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f04d0102160 0x7f04d0199b40 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:24:45.738 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:45.736+0000 7f04d6200640 1 -- 172.21.15.47:0/2287603473 >> 172.21.15.47:0/2287603473 conn(0x7f04d00fbb50 msgr2=0x7f04d01018d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:24:45.738 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:45.736+0000 7f04d6200640 1 -- 172.21.15.47:0/2287603473 shutdown_connections 2024-03-21T10:24:45.738 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:45.736+0000 7f04d6200640 1 -- 172.21.15.47:0/2287603473 wait complete. 2024-03-21T10:24:45.739 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:24:45 smithi157 ceph-mon[35581]: pgmap v454: 129 pgs: 129 active+clean; 642 KiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-03-21T10:24:45.740 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:24:45 smithi157 ceph-mon[35581]: from='client.? 172.21.15.47:0/1964306843' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-21T10:24:45.740 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:24:45 smithi157 ceph-mon[35581]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:24:45.740 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:24:45 smithi157 ceph-mon[35581]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:24:45.740 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:24:45 smithi157 ceph-mon[35581]: from='client.? 172.21.15.47:0/405536471' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-03-21T10:24:45.740 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:24:45 smithi157 ceph-mon[35581]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:24:45.740 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:24:45 smithi157 ceph-mon[35581]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:24:45.764 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-21T10:15:40.553114+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-21T10:21:56.399518+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '86', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-21T10:21:57.812940+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '87', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-03-21T10:22:01.822568+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '91', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-03-21T10:24:45.765 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-21T10:24:46.032 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:46.030+0000 7ff413000640 1 -- 172.21.15.47:0/1865910796 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff40c102000 msgr2=0x7ff40c1043f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:24:46.033 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:46.030+0000 7ff413000640 1 --2- 172.21.15.47:0/1865910796 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff40c102000 0x7ff40c1043f0 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7ff400009900 tx=0x7ff40002bc30 comp rx=0 tx=0).stop 2024-03-21T10:24:46.033 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:46.030+0000 7ff413000640 1 -- 172.21.15.47:0/1865910796 shutdown_connections 2024-03-21T10:24:46.033 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:46.030+0000 7ff413000640 1 --2- 172.21.15.47:0/1865910796 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff40c107260 0x7ff40c1096b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:24:46.033 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:46.030+0000 7ff413000640 1 --2- 172.21.15.47:0/1865910796 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff40c104930 0x7ff40c106d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:24:46.033 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:46.030+0000 7ff413000640 1 --2- 172.21.15.47:0/1865910796 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff40c102000 0x7ff40c1043f0 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:24:46.033 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:46.030+0000 7ff413000640 1 -- 172.21.15.47:0/1865910796 >> 172.21.15.47:0/1865910796 conn(0x7ff40c0fba10 msgr2=0x7ff40c0fde30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:24:46.033 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:46.030+0000 7ff413000640 1 -- 172.21.15.47:0/1865910796 shutdown_connections 2024-03-21T10:24:46.033 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:46.030+0000 7ff413000640 1 -- 172.21.15.47:0/1865910796 wait complete. 2024-03-21T10:24:46.033 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:46.031+0000 7ff413000640 1 Processor -- start 2024-03-21T10:24:46.033 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:46.031+0000 7ff413000640 1 -- start start 2024-03-21T10:24:46.033 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:46.031+0000 7ff413000640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff40c102000 0x7ff40c101970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:24:46.034 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:46.031+0000 7ff413000640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff40c104930 0x7ff40c0fffb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:24:46.034 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:46.031+0000 7ff413000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff40c107260 0x7ff40c1004f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:24:46.034 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:46.031+0000 7ff413000640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff40c10d0c0 con 0x7ff40c102000 2024-03-21T10:24:46.034 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:46.031+0000 7ff413000640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff40c10cf40 con 0x7ff40c107260 2024-03-21T10:24:46.034 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:46.031+0000 7ff413000640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff40c10d240 con 0x7ff40c104930 2024-03-21T10:24:46.036 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:46.032+0000 7ff411c00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff40c102000 0x7ff40c101970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:24:46.036 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:46.032+0000 7ff411c00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff40c102000 0x7ff40c101970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:44254/0 (socket says 172.21.15.47:44254) 2024-03-21T10:24:46.036 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:46.032+0000 7ff411c00640 1 -- 172.21.15.47:0/1953208673 learned_addr learned my addr 172.21.15.47:0/1953208673 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:24:46.036 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:46.032+0000 7ff412600640 1 --2- 172.21.15.47:0/1953208673 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff40c107260 0x7ff40c1004f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:24:46.036 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:46.032+0000 7ff411200640 1 --2- 172.21.15.47:0/1953208673 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff40c104930 0x7ff40c0fffb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:24:46.036 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:46.032+0000 7ff411c00640 1 -- 172.21.15.47:0/1953208673 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff40c104930 msgr2=0x7ff40c0fffb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:24:46.036 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:46.032+0000 7ff411c00640 1 --2- 172.21.15.47:0/1953208673 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff40c104930 0x7ff40c0fffb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:24:46.036 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:46.032+0000 7ff411c00640 1 -- 172.21.15.47:0/1953208673 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff40c107260 msgr2=0x7ff40c1004f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:24:46.036 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:46.032+0000 7ff411c00640 1 --2- 172.21.15.47:0/1953208673 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff40c107260 0x7ff40c1004f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:24:46.036 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:46.032+0000 7ff411c00640 1 -- 172.21.15.47:0/1953208673 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff40c100d80 con 0x7ff40c102000 2024-03-21T10:24:46.036 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:46.032+0000 7ff411c00640 1 --2- 172.21.15.47:0/1953208673 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff40c102000 0x7ff40c101970 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7ff400006010 tx=0x7ff400004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:24:46.036 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:46.033+0000 7ff40a000640 1 -- 172.21.15.47:0/1953208673 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff400043020 con 0x7ff40c102000 2024-03-21T10:24:46.037 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:46.033+0000 7ff413000640 1 -- 172.21.15.47:0/1953208673 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff40c1a18e0 con 0x7ff40c102000 2024-03-21T10:24:46.037 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:46.033+0000 7ff40a000640 1 -- 172.21.15.47:0/1953208673 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7ff400004440 con 0x7ff40c102000 2024-03-21T10:24:46.037 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:46.033+0000 7ff40a000640 1 -- 172.21.15.47:0/1953208673 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff400003bf0 con 0x7ff40c102000 2024-03-21T10:24:46.037 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:46.033+0000 7ff413000640 1 -- 172.21.15.47:0/1953208673 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff40c1a1e80 con 0x7ff40c102000 2024-03-21T10:24:46.037 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:46.035+0000 7ff413000640 1 -- 172.21.15.47:0/1953208673 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff40c0fd500 con 0x7ff40c102000 2024-03-21T10:24:46.037 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:46.036+0000 7ff40a000640 1 -- 172.21.15.47:0/1953208673 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 18) v1 ==== 137854+0+0 (secure 0 0 0) 0x7ff400046020 con 0x7ff40c102000 2024-03-21T10:24:46.039 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:46.037+0000 7ff40a000640 1 --2- 172.21.15.47:0/1953208673 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7ff3f40a2fa0 0x7ff3f40a5460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:24:46.039 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:46.038+0000 7ff40a000640 1 -- 172.21.15.47:0/1953208673 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9216+0+0 (secure 0 0 0) 0x7ff4000f1af0 con 0x7ff40c102000 2024-03-21T10:24:46.039 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:46.038+0000 7ff411200640 1 --2- 172.21.15.47:0/1953208673 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7ff3f40a2fa0 0x7ff3f40a5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:24:46.039 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:46.039+0000 7ff411200640 1 --2- 172.21.15.47:0/1953208673 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7ff3f40a2fa0 0x7ff3f40a5460 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7ff3f80095f0 tx=0x7ff3f8005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:24:46.043 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:46.042+0000 7ff40a000640 1 -- 172.21.15.47:0/1953208673 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7ff40004d020 con 0x7ff40c102000 2024-03-21T10:24:46.298 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:46.297+0000 7ff413000640 1 -- 172.21.15.47:0/1953208673 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7ff40c0622a0 con 0x7ff40c102000 2024-03-21T10:24:46.298 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:46.297+0000 7ff40a000640 1 -- 172.21.15.47:0/1953208673 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v95) v1 ==== 93+0+10 (secure 0 0 0) 0x7ff4000be550 con 0x7ff40c102000 2024-03-21T10:24:46.298 INFO:teuthology.orchestra.run.smithi047.stdout:pg_num: 1 2024-03-21T10:24:46.302 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:46.301+0000 7ff413000640 1 -- 172.21.15.47:0/1953208673 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7ff3f40a2fa0 msgr2=0x7ff3f40a5460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:24:46.302 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:46.301+0000 7ff413000640 1 --2- 172.21.15.47:0/1953208673 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7ff3f40a2fa0 0x7ff3f40a5460 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7ff3f80095f0 tx=0x7ff3f8005f90 comp rx=0 tx=0).stop 2024-03-21T10:24:46.302 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:46.301+0000 7ff413000640 1 -- 172.21.15.47:0/1953208673 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff40c102000 msgr2=0x7ff40c101970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:24:46.302 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:46.301+0000 7ff413000640 1 --2- 172.21.15.47:0/1953208673 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff40c102000 0x7ff40c101970 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7ff400006010 tx=0x7ff400004040 comp rx=0 tx=0).stop 2024-03-21T10:24:46.302 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:46.301+0000 7ff413000640 1 -- 172.21.15.47:0/1953208673 shutdown_connections 2024-03-21T10:24:46.302 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:46.301+0000 7ff413000640 1 --2- 172.21.15.47:0/1953208673 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7ff3f40a2fa0 0x7ff3f40a5460 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:24:46.303 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:46.301+0000 7ff413000640 1 --2- 172.21.15.47:0/1953208673 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff40c107260 0x7ff40c1004f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:24:46.303 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:46.301+0000 7ff413000640 1 --2- 172.21.15.47:0/1953208673 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff40c104930 0x7ff40c0fffb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:24:46.303 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:46.301+0000 7ff413000640 1 --2- 172.21.15.47:0/1953208673 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff40c102000 0x7ff40c101970 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:24:46.303 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:46.301+0000 7ff413000640 1 -- 172.21.15.47:0/1953208673 >> 172.21.15.47:0/1953208673 conn(0x7ff40c0fba10 msgr2=0x7ff40c102b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:24:46.303 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:46.301+0000 7ff413000640 1 -- 172.21.15.47:0/1953208673 shutdown_connections 2024-03-21T10:24:46.303 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:46.301+0000 7ff413000640 1 -- 172.21.15.47:0/1953208673 wait complete. 2024-03-21T10:24:46.325 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-21T10:24:46.541 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:46.539+0000 7fd47ec00640 1 -- 172.21.15.47:0/3639981204 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd478102000 msgr2=0x7fd4781043f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:24:46.541 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:46.539+0000 7fd47ec00640 1 --2- 172.21.15.47:0/3639981204 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd478102000 0x7fd4781043f0 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7fd46c009900 tx=0x7fd46c02bc30 comp rx=0 tx=0).stop 2024-03-21T10:24:46.541 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:46.539+0000 7fd47ec00640 1 -- 172.21.15.47:0/3639981204 shutdown_connections 2024-03-21T10:24:46.541 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:46.539+0000 7fd47ec00640 1 --2- 172.21.15.47:0/3639981204 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd478107260 0x7fd4781096b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:24:46.541 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:46.539+0000 7fd47ec00640 1 --2- 172.21.15.47:0/3639981204 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd478104930 0x7fd478106d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:24:46.541 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:46.539+0000 7fd47ec00640 1 --2- 172.21.15.47:0/3639981204 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd478102000 0x7fd4781043f0 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:24:46.541 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:46.539+0000 7fd47ec00640 1 -- 172.21.15.47:0/3639981204 >> 172.21.15.47:0/3639981204 conn(0x7fd4780fba10 msgr2=0x7fd4780fde30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:24:46.541 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:46.540+0000 7fd47ec00640 1 -- 172.21.15.47:0/3639981204 shutdown_connections 2024-03-21T10:24:46.541 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:46.540+0000 7fd47ec00640 1 -- 172.21.15.47:0/3639981204 wait complete. 2024-03-21T10:24:46.543 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:46.541+0000 7fd47ec00640 1 Processor -- start 2024-03-21T10:24:46.543 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:46.541+0000 7fd47ec00640 1 -- start start 2024-03-21T10:24:46.543 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:46.541+0000 7fd47ec00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd478102000 0x7fd478101970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:24:46.543 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:46.541+0000 7fd47ec00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd478104930 0x7fd4780fffb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:24:46.543 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:46.541+0000 7fd47ec00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd478107260 0x7fd4781004f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:24:46.543 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:46.541+0000 7fd47ec00640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd47810d110 con 0x7fd478102000 2024-03-21T10:24:46.543 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:46.541+0000 7fd47ec00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd47810cf90 con 0x7fd478104930 2024-03-21T10:24:46.543 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:46.541+0000 7fd47ec00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd47810d290 con 0x7fd478107260 2024-03-21T10:24:46.543 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:46.541+0000 7fd47d800640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd478102000 0x7fd478101970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:24:46.543 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:46.542+0000 7fd47d800640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd478102000 0x7fd478101970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:44280/0 (socket says 172.21.15.47:44280) 2024-03-21T10:24:46.543 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:46.542+0000 7fd47d800640 1 -- 172.21.15.47:0/210893938 learned_addr learned my addr 172.21.15.47:0/210893938 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:24:46.544 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:46.542+0000 7fd47ce00640 1 --2- 172.21.15.47:0/210893938 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd478104930 0x7fd4780fffb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:24:46.544 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:46.542+0000 7fd47d800640 1 -- 172.21.15.47:0/210893938 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd478107260 msgr2=0x7fd4781004f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:24:46.544 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:46.542+0000 7fd47d800640 1 --2- 172.21.15.47:0/210893938 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd478107260 0x7fd4781004f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:24:46.544 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:46.542+0000 7fd47d800640 1 -- 172.21.15.47:0/210893938 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd478104930 msgr2=0x7fd4780fffb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:24:46.544 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:46.542+0000 7fd47d800640 1 --2- 172.21.15.47:0/210893938 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd478104930 0x7fd4780fffb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:24:46.544 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:46.542+0000 7fd47d800640 1 -- 172.21.15.47:0/210893938 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd478100d80 con 0x7fd478102000 2024-03-21T10:24:46.545 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:46.542+0000 7fd47d800640 1 --2- 172.21.15.47:0/210893938 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd478102000 0x7fd478101970 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7fd46c000c00 tx=0x7fd46c0042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:24:46.545 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:46.543+0000 7fd476000640 1 -- 172.21.15.47:0/210893938 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd46c043020 con 0x7fd478102000 2024-03-21T10:24:46.545 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:46.543+0000 7fd476000640 1 -- 172.21.15.47:0/210893938 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fd46c004aa0 con 0x7fd478102000 2024-03-21T10:24:46.545 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:46.543+0000 7fd476000640 1 -- 172.21.15.47:0/210893938 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd46c003d70 con 0x7fd478102000 2024-03-21T10:24:46.545 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:46.543+0000 7fd47ec00640 1 -- 172.21.15.47:0/210893938 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd4781a18d0 con 0x7fd478102000 2024-03-21T10:24:46.545 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:46.543+0000 7fd47ec00640 1 -- 172.21.15.47:0/210893938 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd4781a1dd0 con 0x7fd478102000 2024-03-21T10:24:46.545 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:46.544+0000 7fd47ec00640 1 -- 172.21.15.47:0/210893938 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd4780fd500 con 0x7fd478102000 2024-03-21T10:24:46.547 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:46.546+0000 7fd476000640 1 -- 172.21.15.47:0/210893938 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 18) v1 ==== 137854+0+0 (secure 0 0 0) 0x7fd46c046020 con 0x7fd478102000 2024-03-21T10:24:46.549 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:46.548+0000 7fd476000640 1 --2- 172.21.15.47:0/210893938 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7fd4600a2f20 0x7fd4600a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:24:46.549 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:46.548+0000 7fd476000640 1 -- 172.21.15.47:0/210893938 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9216+0+0 (secure 0 0 0) 0x7fd46c0f1b30 con 0x7fd478102000 2024-03-21T10:24:46.549 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:46.548+0000 7fd47ce00640 1 --2- 172.21.15.47:0/210893938 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7fd4600a2f20 0x7fd4600a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:24:46.549 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:46.549+0000 7fd47ce00640 1 --2- 172.21.15.47:0/210893938 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7fd4600a2f20 0x7fd4600a53e0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fd4781016e0 tx=0x7fd464005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:24:46.553 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:46.552+0000 7fd476000640 1 -- 172.21.15.47:0/210893938 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fd46c04d020 con 0x7fd478102000 2024-03-21T10:24:46.714 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:24:46 smithi047 ceph-mon[31547]: from='client.? 172.21.15.47:0/622781934' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-21T10:24:46.714 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:24:46 smithi047 ceph-mon[31547]: from='client.? 172.21.15.47:0/2287603473' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-21T10:24:46.714 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:24:46 smithi047 ceph-mon[31547]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T10:24:46.714 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:24:46 smithi047 ceph-mon[31547]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T10:24:46.714 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:24:46 smithi047 ceph-mon[31547]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:24:46.729 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:24:46 smithi100 ceph-mon[35193]: from='client.? 172.21.15.47:0/622781934' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-21T10:24:46.729 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:24:46 smithi100 ceph-mon[35193]: from='client.? 172.21.15.47:0/2287603473' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-21T10:24:46.730 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:24:46 smithi100 ceph-mon[35193]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T10:24:46.730 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:24:46 smithi100 ceph-mon[35193]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T10:24:46.730 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:24:46 smithi100 ceph-mon[35193]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:24:46.740 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:24:46 smithi157 ceph-mon[35581]: from='client.? 172.21.15.47:0/622781934' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-21T10:24:46.740 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:24:46 smithi157 ceph-mon[35581]: from='client.? 172.21.15.47:0/2287603473' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-21T10:24:46.740 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:24:46 smithi157 ceph-mon[35581]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T10:24:46.740 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:24:46 smithi157 ceph-mon[35581]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T10:24:46.740 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:24:46 smithi157 ceph-mon[35581]: from='mgr.14152 172.21.15.47:0/3821269' entity='mgr.x' 2024-03-21T10:24:46.780 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:46.779+0000 7fd47ec00640 1 -- 172.21.15.47:0/210893938 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fd4780622a0 con 0x7fd478102000 2024-03-21T10:24:46.781 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:46.780+0000 7fd476000640 1 -- 172.21.15.47:0/210893938 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v95) v1 ==== 104+0+11 (secure 0 0 0) 0x7fd46c0be550 con 0x7fd478102000 2024-03-21T10:24:46.781 INFO:teuthology.orchestra.run.smithi047.stdout:pg_num: 32 2024-03-21T10:24:46.784 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:46.783+0000 7fd47ec00640 1 -- 172.21.15.47:0/210893938 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7fd4600a2f20 msgr2=0x7fd4600a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:24:46.784 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:46.783+0000 7fd47ec00640 1 --2- 172.21.15.47:0/210893938 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7fd4600a2f20 0x7fd4600a53e0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fd4781016e0 tx=0x7fd464005f90 comp rx=0 tx=0).stop 2024-03-21T10:24:46.784 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:46.783+0000 7fd47ec00640 1 -- 172.21.15.47:0/210893938 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd478102000 msgr2=0x7fd478101970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:24:46.784 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:46.783+0000 7fd47ec00640 1 --2- 172.21.15.47:0/210893938 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd478102000 0x7fd478101970 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7fd46c000c00 tx=0x7fd46c0042f0 comp rx=0 tx=0).stop 2024-03-21T10:24:46.784 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:46.783+0000 7fd47ec00640 1 -- 172.21.15.47:0/210893938 shutdown_connections 2024-03-21T10:24:46.784 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:46.783+0000 7fd47ec00640 1 --2- 172.21.15.47:0/210893938 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7fd4600a2f20 0x7fd4600a53e0 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:24:46.785 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:46.783+0000 7fd47ec00640 1 --2- 172.21.15.47:0/210893938 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd478107260 0x7fd4781004f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:24:46.785 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:46.783+0000 7fd47ec00640 1 --2- 172.21.15.47:0/210893938 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd478104930 0x7fd4780fffb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:24:46.785 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:46.783+0000 7fd47ec00640 1 --2- 172.21.15.47:0/210893938 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd478102000 0x7fd478101970 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:24:46.785 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:46.783+0000 7fd47ec00640 1 -- 172.21.15.47:0/210893938 >> 172.21.15.47:0/210893938 conn(0x7fd4780fba10 msgr2=0x7fd4781028c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:24:46.785 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:46.783+0000 7fd47ec00640 1 -- 172.21.15.47:0/210893938 shutdown_connections 2024-03-21T10:24:46.785 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:46.783+0000 7fd47ec00640 1 -- 172.21.15.47:0/210893938 wait complete. 2024-03-21T10:24:46.802 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-21T10:24:47.029 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:47.028+0000 7f167a400640 1 -- 172.21.15.47:0/1806499869 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f16740756e0 msgr2=0x7f1674073b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:24:47.029 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:47.028+0000 7f167a400640 1 --2- 172.21.15.47:0/1806499869 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f16740756e0 0x7f1674073b30 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f1668009900 tx=0x7f166802bc30 comp rx=0 tx=0).stop 2024-03-21T10:24:47.029 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:47.028+0000 7f167a400640 1 -- 172.21.15.47:0/1806499869 shutdown_connections 2024-03-21T10:24:47.029 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:47.028+0000 7f167a400640 1 --2- 172.21.15.47:0/1806499869 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1674109460 0x7f167410b8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:24:47.029 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:47.028+0000 7f167a400640 1 --2- 172.21.15.47:0/1806499869 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1674074100 0x7f1674074580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:24:47.029 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:47.028+0000 7f167a400640 1 --2- 172.21.15.47:0/1806499869 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f16740756e0 0x7f1674073b30 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:24:47.030 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:47.028+0000 7f167a400640 1 -- 172.21.15.47:0/1806499869 >> 172.21.15.47:0/1806499869 conn(0x7f16740fcb10 msgr2=0x7f16740fef50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:24:47.030 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:47.028+0000 7f167a400640 1 -- 172.21.15.47:0/1806499869 shutdown_connections 2024-03-21T10:24:47.030 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:47.028+0000 7f167a400640 1 -- 172.21.15.47:0/1806499869 wait complete. 2024-03-21T10:24:47.030 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:47.029+0000 7f167a400640 1 Processor -- start 2024-03-21T10:24:47.030 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:47.029+0000 7f167a400640 1 -- start start 2024-03-21T10:24:47.035 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:47.030+0000 7f167a400640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1674074100 0x7f1674199a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:24:47.035 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:47.030+0000 7f167a400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f16740756e0 0x7f1674199f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:24:47.035 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:47.030+0000 7f167a400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1674109460 0x7f16741a0fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:24:47.035 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:47.030+0000 7f167a400640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f167410f360 con 0x7f1674109460 2024-03-21T10:24:47.035 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:47.030+0000 7f167a400640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f167410f1e0 con 0x7f16740756e0 2024-03-21T10:24:47.035 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:47.030+0000 7f167a400640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f167410f4e0 con 0x7f1674074100 2024-03-21T10:24:47.035 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:47.030+0000 7f1679a00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1674109460 0x7f16741a0fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:24:47.035 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:47.030+0000 7f1679a00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1674109460 0x7f16741a0fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:44294/0 (socket says 172.21.15.47:44294) 2024-03-21T10:24:47.035 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:47.030+0000 7f1679a00640 1 -- 172.21.15.47:0/3438265290 learned_addr learned my addr 172.21.15.47:0/3438265290 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:24:47.035 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:47.030+0000 7f1673e00640 1 --2- 172.21.15.47:0/3438265290 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f16740756e0 0x7f1674199f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:24:47.035 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:47.031+0000 7f1679000640 1 --2- 172.21.15.47:0/3438265290 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1674074100 0x7f1674199a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:24:47.035 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:47.031+0000 7f1679a00640 1 -- 172.21.15.47:0/3438265290 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1674074100 msgr2=0x7f1674199a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:24:47.036 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:47.031+0000 7f1679a00640 1 --2- 172.21.15.47:0/3438265290 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1674074100 0x7f1674199a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:24:47.036 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:47.031+0000 7f1679a00640 1 -- 172.21.15.47:0/3438265290 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f16740756e0 msgr2=0x7f1674199f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:24:47.036 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:47.031+0000 7f1679a00640 1 --2- 172.21.15.47:0/3438265290 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f16740756e0 0x7f1674199f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:24:47.036 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:47.031+0000 7f1679a00640 1 -- 172.21.15.47:0/3438265290 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f16741a1650 con 0x7f1674109460 2024-03-21T10:24:47.036 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:47.031+0000 7f1679a00640 1 --2- 172.21.15.47:0/3438265290 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1674109460 0x7f16741a0fe0 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f166c00eb10 tx=0x7f166c00ef10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:24:47.036 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:47.031+0000 7f1671600640 1 -- 172.21.15.47:0/3438265290 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f166c00ce20 con 0x7f1674109460 2024-03-21T10:24:47.036 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:47.031+0000 7f1671600640 1 -- 172.21.15.47:0/3438265290 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f166c005020 con 0x7f1674109460 2024-03-21T10:24:47.036 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:47.032+0000 7f167a400640 1 -- 172.21.15.47:0/3438265290 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f16741a1940 con 0x7f1674109460 2024-03-21T10:24:47.036 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:47.032+0000 7f1671600640 1 -- 172.21.15.47:0/3438265290 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f166c010690 con 0x7f1674109460 2024-03-21T10:24:47.036 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:47.032+0000 7f167a400640 1 -- 172.21.15.47:0/3438265290 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f16741a1ec0 con 0x7f1674109460 2024-03-21T10:24:47.036 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:47.033+0000 7f167a400640 1 -- 172.21.15.47:0/3438265290 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1674108190 con 0x7f1674109460 2024-03-21T10:24:47.037 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:47.036+0000 7f1671600640 1 -- 172.21.15.47:0/3438265290 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 18) v1 ==== 137854+0+0 (secure 0 0 0) 0x7f166c0051d0 con 0x7f1674109460 2024-03-21T10:24:47.039 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:47.038+0000 7f1671600640 1 --2- 172.21.15.47:0/3438265290 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f165c0a2f20 0x7f165c0a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:24:47.039 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:47.038+0000 7f1671600640 1 -- 172.21.15.47:0/3438265290 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9216+0+0 (secure 0 0 0) 0x7f166c0cfe00 con 0x7f1674109460 2024-03-21T10:24:47.039 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:47.038+0000 7f1679000640 1 --2- 172.21.15.47:0/3438265290 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f165c0a2f20 0x7f165c0a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:24:47.040 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:47.039+0000 7f1679000640 1 --2- 172.21.15.47:0/3438265290 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f165c0a2f20 0x7f165c0a53e0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f1668006010 tx=0x7f1668000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:24:47.047 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:47.046+0000 7f1671600640 1 -- 172.21.15.47:0/3438265290 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f166c09c860 con 0x7f1674109460 2024-03-21T10:24:47.259 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:47.258+0000 7f167a400640 1 -- 172.21.15.47:0/3438265290 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f16740622a0 con 0x7f1674109460 2024-03-21T10:24:47.262 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:47.261+0000 7f1671600640 1 -- 172.21.15.47:0/3438265290 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v95) v1 ==== 100+0+11 (secure 0 0 0) 0x7f166c09c160 con 0x7f1674109460 2024-03-21T10:24:47.262 INFO:teuthology.orchestra.run.smithi047.stdout:pg_num: 64 2024-03-21T10:24:47.266 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:47.264+0000 7f167a400640 1 -- 172.21.15.47:0/3438265290 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f165c0a2f20 msgr2=0x7f165c0a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:24:47.266 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:47.264+0000 7f167a400640 1 --2- 172.21.15.47:0/3438265290 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f165c0a2f20 0x7f165c0a53e0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f1668006010 tx=0x7f1668000bc0 comp rx=0 tx=0).stop 2024-03-21T10:24:47.266 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:47.264+0000 7f167a400640 1 -- 172.21.15.47:0/3438265290 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1674109460 msgr2=0x7f16741a0fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:24:47.266 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:47.264+0000 7f167a400640 1 --2- 172.21.15.47:0/3438265290 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1674109460 0x7f16741a0fe0 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f166c00eb10 tx=0x7f166c00ef10 comp rx=0 tx=0).stop 2024-03-21T10:24:47.266 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:47.265+0000 7f167a400640 1 -- 172.21.15.47:0/3438265290 shutdown_connections 2024-03-21T10:24:47.266 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:47.265+0000 7f167a400640 1 --2- 172.21.15.47:0/3438265290 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f165c0a2f20 0x7f165c0a53e0 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:24:47.266 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:47.265+0000 7f167a400640 1 --2- 172.21.15.47:0/3438265290 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1674109460 0x7f16741a0fe0 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:24:47.266 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:47.265+0000 7f167a400640 1 --2- 172.21.15.47:0/3438265290 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f16740756e0 0x7f1674199f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:24:47.266 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:47.265+0000 7f167a400640 1 --2- 172.21.15.47:0/3438265290 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1674074100 0x7f1674199a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:24:47.267 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:47.265+0000 7f167a400640 1 -- 172.21.15.47:0/3438265290 >> 172.21.15.47:0/3438265290 conn(0x7f16740fcb10 msgr2=0x7f1674105360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:24:47.267 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:47.265+0000 7f167a400640 1 -- 172.21.15.47:0/3438265290 shutdown_connections 2024-03-21T10:24:47.267 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:47.265+0000 7f167a400640 1 -- 172.21.15.47:0/3438265290 wait complete. 2024-03-21T10:24:47.294 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-03-21T10:24:47.519 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:47.517+0000 7faadaa00640 1 -- 172.21.15.47:0/31432706 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7faad4105350 msgr2=0x7faad410bbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:24:47.519 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:47.517+0000 7faadaa00640 1 --2- 172.21.15.47:0/31432706 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7faad4105350 0x7faad410bbe0 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7faacc009ca0 tx=0x7faacc02c0f0 comp rx=0 tx=0).stop 2024-03-21T10:24:47.519 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:47.517+0000 7faadaa00640 1 -- 172.21.15.47:0/31432706 shutdown_connections 2024-03-21T10:24:47.519 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:47.517+0000 7faadaa00640 1 --2- 172.21.15.47:0/31432706 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7faad4105350 0x7faad410bbe0 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:24:47.519 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:47.517+0000 7faadaa00640 1 --2- 172.21.15.47:0/31432706 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7faad4104990 0x7faad4104e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:24:47.519 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:47.517+0000 7faadaa00640 1 --2- 172.21.15.47:0/31432706 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7faad41037a0 0x7faad4103bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:24:47.519 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:47.517+0000 7faadaa00640 1 -- 172.21.15.47:0/31432706 >> 172.21.15.47:0/31432706 conn(0x7faad40fed10 msgr2=0x7faad4101150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:24:47.519 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:47.517+0000 7faadaa00640 1 -- 172.21.15.47:0/31432706 shutdown_connections 2024-03-21T10:24:47.519 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:47.517+0000 7faadaa00640 1 -- 172.21.15.47:0/31432706 wait complete. 2024-03-21T10:24:47.521 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:47.518+0000 7faadaa00640 1 Processor -- start 2024-03-21T10:24:47.521 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:47.518+0000 7faadaa00640 1 -- start start 2024-03-21T10:24:47.521 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:47.519+0000 7faadaa00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7faad41037a0 0x7faad4199980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:24:47.521 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:47.519+0000 7faadaa00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7faad4104990 0x7faad4199ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:24:47.522 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:47.519+0000 7faadaa00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7faad4105350 0x7faad41a0f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:24:47.522 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:47.519+0000 7faadaa00640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faad410f2b0 con 0x7faad41037a0 2024-03-21T10:24:47.522 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:47.519+0000 7faadaa00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faad410f130 con 0x7faad4105350 2024-03-21T10:24:47.522 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:47.519+0000 7faadaa00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faad410f430 con 0x7faad4104990 2024-03-21T10:24:47.522 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:47.519+0000 7faad9600640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7faad41037a0 0x7faad4199980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:24:47.522 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:47.519+0000 7faad9600640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7faad41037a0 0x7faad4199980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:44314/0 (socket says 172.21.15.47:44314) 2024-03-21T10:24:47.522 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:47.519+0000 7faad9600640 1 -- 172.21.15.47:0/261397232 learned_addr learned my addr 172.21.15.47:0/261397232 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:24:47.522 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:47.519+0000 7faada000640 1 --2- 172.21.15.47:0/261397232 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7faad4105350 0x7faad41a0f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:24:47.522 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:47.519+0000 7faad8c00640 1 --2- 172.21.15.47:0/261397232 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7faad4104990 0x7faad4199ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:24:47.522 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:47.520+0000 7faad9600640 1 -- 172.21.15.47:0/261397232 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7faad4104990 msgr2=0x7faad4199ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:24:47.522 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:47.520+0000 7faad9600640 1 --2- 172.21.15.47:0/261397232 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7faad4104990 0x7faad4199ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:24:47.522 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:47.520+0000 7faad9600640 1 -- 172.21.15.47:0/261397232 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7faad4105350 msgr2=0x7faad41a0f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:24:47.522 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:47.520+0000 7faad9600640 1 --2- 172.21.15.47:0/261397232 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7faad4105350 0x7faad41a0f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:24:47.523 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:47.520+0000 7faad9600640 1 -- 172.21.15.47:0/261397232 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faad41a15a0 con 0x7faad41037a0 2024-03-21T10:24:47.523 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:47.520+0000 7faad9600640 1 --2- 172.21.15.47:0/261397232 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7faad41037a0 0x7faad4199980 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7faac800d770 tx=0x7faac800dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:24:47.523 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:47.520+0000 7faad2000640 1 -- 172.21.15.47:0/261397232 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faac8017070 con 0x7faad41037a0 2024-03-21T10:24:47.523 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:47.520+0000 7faadaa00640 1 -- 172.21.15.47:0/261397232 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faad41a1890 con 0x7faad41037a0 2024-03-21T10:24:47.523 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:47.520+0000 7faad2000640 1 -- 172.21.15.47:0/261397232 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7faac8009c30 con 0x7faad41037a0 2024-03-21T10:24:47.523 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:47.520+0000 7faad2000640 1 -- 172.21.15.47:0/261397232 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faac8003d00 con 0x7faad41037a0 2024-03-21T10:24:47.523 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:47.520+0000 7faadaa00640 1 -- 172.21.15.47:0/261397232 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faad41a1e30 con 0x7faad41037a0 2024-03-21T10:24:47.523 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:47.522+0000 7faadaa00640 1 -- 172.21.15.47:0/261397232 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faad4068800 con 0x7faad41037a0 2024-03-21T10:24:47.525 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:47.524+0000 7faad2000640 1 -- 172.21.15.47:0/261397232 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 18) v1 ==== 137854+0+0 (secure 0 0 0) 0x7faac8003ea0 con 0x7faad41037a0 2024-03-21T10:24:47.526 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:47.525+0000 7faad2000640 1 --2- 172.21.15.47:0/261397232 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7faabc0a2f20 0x7faabc0a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:24:47.526 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:47.526+0000 7faad2000640 1 -- 172.21.15.47:0/261397232 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9216+0+0 (secure 0 0 0) 0x7faac80cf5c0 con 0x7faad41037a0 2024-03-21T10:24:47.526 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:47.526+0000 7faad8c00640 1 --2- 172.21.15.47:0/261397232 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7faabc0a2f20 0x7faabc0a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:24:47.527 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:47.526+0000 7faad8c00640 1 --2- 172.21.15.47:0/261397232 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7faabc0a2f20 0x7faabc0a53e0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7faad419ae90 tx=0x7faac0005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:24:47.529 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:47.528+0000 7faad2000640 1 -- 172.21.15.47:0/261397232 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7faac809bfe0 con 0x7faad41037a0 2024-03-21T10:24:47.714 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:24:47 smithi047 ceph-mon[31547]: pgmap v455: 129 pgs: 129 active+clean; 642 KiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-21T10:24:47.714 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:24:47 smithi047 ceph-mon[31547]: from='client.? 172.21.15.47:0/1953208673' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-21T10:24:47.714 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:24:47 smithi047 ceph-mon[31547]: from='client.? 172.21.15.47:0/210893938' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-21T10:24:47.729 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:24:47 smithi100 ceph-mon[35193]: pgmap v455: 129 pgs: 129 active+clean; 642 KiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-21T10:24:47.729 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:24:47 smithi100 ceph-mon[35193]: from='client.? 172.21.15.47:0/1953208673' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-21T10:24:47.729 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:24:47 smithi100 ceph-mon[35193]: from='client.? 172.21.15.47:0/210893938' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-21T10:24:47.731 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:47.731+0000 7faadaa00640 1 -- 172.21.15.47:0/261397232 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7faad402d070 con 0x7faad41037a0 2024-03-21T10:24:47.732 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:47.731+0000 7faad2000640 1 -- 172.21.15.47:0/261397232 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v95) v1 ==== 103+0+11 (secure 0 0 0) 0x7faac8009de0 con 0x7faad41037a0 2024-03-21T10:24:47.733 INFO:teuthology.orchestra.run.smithi047.stdout:pg_num: 32 2024-03-21T10:24:47.735 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:47.734+0000 7faadaa00640 1 -- 172.21.15.47:0/261397232 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7faabc0a2f20 msgr2=0x7faabc0a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:24:47.735 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:47.734+0000 7faadaa00640 1 --2- 172.21.15.47:0/261397232 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7faabc0a2f20 0x7faabc0a53e0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7faad419ae90 tx=0x7faac0005f90 comp rx=0 tx=0).stop 2024-03-21T10:24:47.735 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:47.734+0000 7faadaa00640 1 -- 172.21.15.47:0/261397232 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7faad41037a0 msgr2=0x7faad4199980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:24:47.736 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:47.734+0000 7faadaa00640 1 --2- 172.21.15.47:0/261397232 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7faad41037a0 0x7faad4199980 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7faac800d770 tx=0x7faac800dc30 comp rx=0 tx=0).stop 2024-03-21T10:24:47.736 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:47.734+0000 7faadaa00640 1 -- 172.21.15.47:0/261397232 shutdown_connections 2024-03-21T10:24:47.736 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:47.734+0000 7faadaa00640 1 --2- 172.21.15.47:0/261397232 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7faabc0a2f20 0x7faabc0a53e0 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:24:47.736 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:47.734+0000 7faadaa00640 1 --2- 172.21.15.47:0/261397232 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7faad4105350 0x7faad41a0f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:24:47.736 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:47.734+0000 7faadaa00640 1 --2- 172.21.15.47:0/261397232 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7faad4104990 0x7faad4199ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:24:47.736 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:47.734+0000 7faadaa00640 1 --2- 172.21.15.47:0/261397232 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7faad41037a0 0x7faad4199980 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:24:47.736 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:47.734+0000 7faadaa00640 1 -- 172.21.15.47:0/261397232 >> 172.21.15.47:0/261397232 conn(0x7faad40fed10 msgr2=0x7faad4107d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:24:47.736 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:47.734+0000 7faadaa00640 1 -- 172.21.15.47:0/261397232 shutdown_connections 2024-03-21T10:24:47.736 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:47.735+0000 7faadaa00640 1 -- 172.21.15.47:0/261397232 wait complete. 2024-03-21T10:24:47.740 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:24:47 smithi157 ceph-mon[35581]: pgmap v455: 129 pgs: 129 active+clean; 642 KiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-21T10:24:47.740 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:24:47 smithi157 ceph-mon[35581]: from='client.? 172.21.15.47:0/1953208673' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-21T10:24:47.740 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:24:47 smithi157 ceph-mon[35581]: from='client.? 172.21.15.47:0/210893938' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-21T10:24:47.756 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-21T10:24:47.979 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:47.978+0000 7f8deee00640 1 -- 172.21.15.47:0/2268120856 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8de8074080 msgr2=0x7f8de80744e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:24:47.980 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:47.978+0000 7f8deee00640 1 --2- 172.21.15.47:0/2268120856 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8de8074080 0x7f8de80744e0 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f8dd4009900 tx=0x7f8dd402bcd0 comp rx=0 tx=0).stop 2024-03-21T10:24:47.980 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:47.978+0000 7f8deee00640 1 -- 172.21.15.47:0/2268120856 shutdown_connections 2024-03-21T10:24:47.980 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:47.978+0000 7f8deee00640 1 --2- 172.21.15.47:0/2268120856 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8de8109560 0x7f8de810b9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:24:47.980 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:47.978+0000 7f8deee00640 1 --2- 172.21.15.47:0/2268120856 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8de8074080 0x7f8de80744e0 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:24:47.980 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:47.978+0000 7f8deee00640 1 --2- 172.21.15.47:0/2268120856 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8de8075660 0x7f8de8073ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:24:47.980 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:47.978+0000 7f8deee00640 1 -- 172.21.15.47:0/2268120856 >> 172.21.15.47:0/2268120856 conn(0x7f8de80fcc80 msgr2=0x7f8de80ff0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:24:47.980 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:47.978+0000 7f8deee00640 1 -- 172.21.15.47:0/2268120856 shutdown_connections 2024-03-21T10:24:47.980 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:47.978+0000 7f8deee00640 1 -- 172.21.15.47:0/2268120856 wait complete. 2024-03-21T10:24:47.982 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:47.979+0000 7f8deee00640 1 Processor -- start 2024-03-21T10:24:47.982 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:47.980+0000 7f8deee00640 1 -- start start 2024-03-21T10:24:47.982 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:47.980+0000 7f8deee00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8de8074080 0x7f8de8199a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:24:47.982 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:47.980+0000 7f8deee00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8de8075660 0x7f8de8199fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:24:47.982 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:47.980+0000 7f8deee00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8de8109560 0x7f8de81a1010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:24:47.982 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:47.980+0000 7f8deee00640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8de810f410 con 0x7f8de8109560 2024-03-21T10:24:47.982 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:47.980+0000 7f8deee00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8de810f290 con 0x7f8de8074080 2024-03-21T10:24:47.982 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:47.980+0000 7f8deee00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8de810f590 con 0x7f8de8075660 2024-03-21T10:24:47.982 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:47.980+0000 7f8dece00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8de8109560 0x7f8de81a1010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:24:47.983 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:47.980+0000 7f8dece00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8de8109560 0x7f8de81a1010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:44330/0 (socket says 172.21.15.47:44330) 2024-03-21T10:24:47.983 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:47.980+0000 7f8dece00640 1 -- 172.21.15.47:0/3573292364 learned_addr learned my addr 172.21.15.47:0/3573292364 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:24:47.983 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:47.980+0000 7f8de7e00640 1 --2- 172.21.15.47:0/3573292364 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8de8074080 0x7f8de8199a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:24:47.983 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:47.981+0000 7f8de7400640 1 --2- 172.21.15.47:0/3573292364 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8de8075660 0x7f8de8199fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:24:47.983 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:47.981+0000 7f8dece00640 1 -- 172.21.15.47:0/3573292364 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8de8075660 msgr2=0x7f8de8199fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:24:47.983 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:47.981+0000 7f8dece00640 1 --2- 172.21.15.47:0/3573292364 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8de8075660 0x7f8de8199fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:24:47.983 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:47.981+0000 7f8dece00640 1 -- 172.21.15.47:0/3573292364 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8de8074080 msgr2=0x7f8de8199a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:24:47.983 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:47.981+0000 7f8dece00640 1 --2- 172.21.15.47:0/3573292364 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8de8074080 0x7f8de8199a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:24:47.983 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:47.981+0000 7f8dece00640 1 -- 172.21.15.47:0/3573292364 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8de81a1680 con 0x7f8de8109560 2024-03-21T10:24:47.983 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:47.981+0000 7f8dece00640 1 --2- 172.21.15.47:0/3573292364 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8de8109560 0x7f8de81a1010 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f8de000cce0 tx=0x7f8de000a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:24:47.984 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:47.981+0000 7f8de4c00640 1 -- 172.21.15.47:0/3573292364 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8de000f450 con 0x7f8de8109560 2024-03-21T10:24:47.984 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:47.981+0000 7f8de4c00640 1 -- 172.21.15.47:0/3573292364 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f8de000faa0 con 0x7f8de8109560 2024-03-21T10:24:47.984 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:47.981+0000 7f8de4c00640 1 -- 172.21.15.47:0/3573292364 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8de0018b70 con 0x7f8de8109560 2024-03-21T10:24:47.984 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:47.981+0000 7f8deee00640 1 -- 172.21.15.47:0/3573292364 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8de81a1970 con 0x7f8de8109560 2024-03-21T10:24:47.984 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:47.982+0000 7f8deee00640 1 -- 172.21.15.47:0/3573292364 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8de81a1e90 con 0x7f8de8109560 2024-03-21T10:24:47.984 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:47.983+0000 7f8deee00640 1 -- 172.21.15.47:0/3573292364 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8de8108320 con 0x7f8de8109560 2024-03-21T10:24:47.986 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:47.985+0000 7f8de4c00640 1 -- 172.21.15.47:0/3573292364 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 18) v1 ==== 137854+0+0 (secure 0 0 0) 0x7f8de0018d10 con 0x7f8de8109560 2024-03-21T10:24:47.988 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:47.987+0000 7f8de4c00640 1 --2- 172.21.15.47:0/3573292364 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f8dcc0a2fa0 0x7f8dcc0a5460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:24:47.988 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:47.987+0000 7f8de4c00640 1 -- 172.21.15.47:0/3573292364 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9216+0+0 (secure 0 0 0) 0x7f8de00cfbb0 con 0x7f8de8109560 2024-03-21T10:24:47.988 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:47.988+0000 7f8de7e00640 1 --2- 172.21.15.47:0/3573292364 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f8dcc0a2fa0 0x7f8dcc0a5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:24:47.989 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:47.988+0000 7f8de7e00640 1 --2- 172.21.15.47:0/3573292364 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f8dcc0a2fa0 0x7f8dcc0a5460 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f8dd80095f0 tx=0x7f8dd8005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:24:47.993 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:47.992+0000 7f8de4c00640 1 -- 172.21.15.47:0/3573292364 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f8de0014030 con 0x7f8de8109560 2024-03-21T10:24:48.237 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:48.236+0000 7f8deee00640 1 -- 172.21.15.47:0/3573292364 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f8de802d070 con 0x7f8de8109560 2024-03-21T10:24:48.238 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:48.237+0000 7f8de4c00640 1 -- 172.21.15.47:0/3573292364 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) v1 ==== 94+0+11471 (secure 0 0 0) 0x7f8de009c610 con 0x7f8de8109560 2024-03-21T10:24:48.238 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-21T10:24:48.238 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":29,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14511,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6835/735724037","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6834","nonce":735724037},{"type":"v1","addr":"172.21.15.47:6835","nonce":735724037}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14529,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6837/2248191724","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6836","nonce":2248191724},{"type":"v1","addr":"172.21.15.47:6837","nonce":2248191724}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":29,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-21T10:21:58.329063+0000","modified":"2024-03-21T10:22:29.588509+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":34535,"mds_1":34493,"mds_2":34454,"mds_3":34427,"mds_4":14619},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14604":{"gid":14604,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.47:6839/2881335356","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6838","nonce":2881335356},{"type":"v1","addr":"172.21.15.47:6839","nonce":2881335356}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_14619":{"gid":14619,"name":"a","rank":4,"incarnation":27,"state":"up:active","state_seq":11,"addr":"172.21.15.47:6841/3401578978","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6840","nonce":3401578978},{"type":"v1","addr":"172.21.15.47:6841","nonce":3401578978}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24406":{"gid":24406,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.157:6833/4000010136","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":4000010136},{"type":"v1","addr":"172.21.15.157:6833","nonce":4000010136}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24457":{"gid":24457,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.157:6835/3902868651","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":3902868651},{"type":"v1","addr":"172.21.15.157:6835","nonce":3902868651}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24475":{"gid":24475,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.157:6837/1908087204","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6836","nonce":1908087204},{"type":"v1","addr":"172.21.15.157:6837","nonce":1908087204}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24499":{"gid":24499,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.157:6839/4064242876","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6838","nonce":4064242876},{"type":"v1","addr":"172.21.15.157:6839","nonce":4064242876}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_34427":{"gid":34427,"name":"k","rank":3,"incarnation":25,"state":"up:active","state_seq":20,"addr":"172.21.15.100:6833/2723302038","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6832","nonce":2723302038},{"type":"v1","addr":"172.21.15.100:6833","nonce":2723302038}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_34454":{"gid":34454,"name":"e","rank":2,"incarnation":22,"state":"up:active","state_seq":17,"addr":"172.21.15.100:6835/2597849957","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":2597849957},{"type":"v1","addr":"172.21.15.100:6835","nonce":2597849957}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_34493":{"gid":34493,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":13,"addr":"172.21.15.100:6837/3497644162","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6836","nonce":3497644162},{"type":"v1","addr":"172.21.15.100:6837","nonce":3497644162}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_34535":{"gid":34535,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.100:6839/4173168440","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6838","nonce":4173168440},{"type":"v1","addr":"172.21.15.100:6839","nonce":4173168440}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":34535,"qdb_cluster":[34493,34535,34454,34427,14619]},"id":1}]} 2024-03-21T10:24:48.239 INFO:teuthology.orchestra.run.smithi047.stderr:dumped fsmap epoch 29 2024-03-21T10:24:48.242 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:48.240+0000 7f8deee00640 1 -- 172.21.15.47:0/3573292364 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f8dcc0a2fa0 msgr2=0x7f8dcc0a5460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:24:48.242 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:48.240+0000 7f8deee00640 1 --2- 172.21.15.47:0/3573292364 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f8dcc0a2fa0 0x7f8dcc0a5460 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f8dd80095f0 tx=0x7f8dd8005f90 comp rx=0 tx=0).stop 2024-03-21T10:24:48.242 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:48.240+0000 7f8deee00640 1 -- 172.21.15.47:0/3573292364 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8de8109560 msgr2=0x7f8de81a1010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:24:48.242 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:48.240+0000 7f8deee00640 1 --2- 172.21.15.47:0/3573292364 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8de8109560 0x7f8de81a1010 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f8de000cce0 tx=0x7f8de000a620 comp rx=0 tx=0).stop 2024-03-21T10:24:48.242 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:48.240+0000 7f8deee00640 1 -- 172.21.15.47:0/3573292364 shutdown_connections 2024-03-21T10:24:48.242 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:48.240+0000 7f8deee00640 1 --2- 172.21.15.47:0/3573292364 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f8dcc0a2fa0 0x7f8dcc0a5460 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:24:48.242 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:48.240+0000 7f8deee00640 1 --2- 172.21.15.47:0/3573292364 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8de8109560 0x7f8de81a1010 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:24:48.242 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:48.240+0000 7f8deee00640 1 --2- 172.21.15.47:0/3573292364 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8de8075660 0x7f8de8199fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:24:48.242 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:48.240+0000 7f8deee00640 1 --2- 172.21.15.47:0/3573292364 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8de8074080 0x7f8de8199a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:24:48.243 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:48.241+0000 7f8deee00640 1 -- 172.21.15.47:0/3573292364 >> 172.21.15.47:0/3573292364 conn(0x7f8de80fcc80 msgr2=0x7f8de81054f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:24:48.243 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:48.241+0000 7f8deee00640 1 -- 172.21.15.47:0/3573292364 shutdown_connections 2024-03-21T10:24:48.243 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:48.241+0000 7f8deee00640 1 -- 172.21.15.47:0/3573292364 wait complete. 2024-03-21T10:24:48.266 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-21T10:24:48.478 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:48.477+0000 7f5a05e00640 1 -- 172.21.15.47:0/2188428113 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5a00104ad0 msgr2=0x7f5a00104f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:24:48.478 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:48.477+0000 7f5a05e00640 1 --2- 172.21.15.47:0/2188428113 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5a00104ad0 0x7f5a00104f50 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f59ec009900 tx=0x7f59ec02bc40 comp rx=0 tx=0).stop 2024-03-21T10:24:48.478 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:48.477+0000 7f5a05e00640 1 -- 172.21.15.47:0/2188428113 shutdown_connections 2024-03-21T10:24:48.479 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:48.477+0000 7f5a05e00640 1 --2- 172.21.15.47:0/2188428113 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5a00105490 0x7f5a0010bd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:24:48.479 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:48.477+0000 7f5a05e00640 1 --2- 172.21.15.47:0/2188428113 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5a00104ad0 0x7f5a00104f50 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:24:48.479 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:48.477+0000 7f5a05e00640 1 --2- 172.21.15.47:0/2188428113 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5a001038e0 0x7f5a00103d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:24:48.479 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:48.477+0000 7f5a05e00640 1 -- 172.21.15.47:0/2188428113 >> 172.21.15.47:0/2188428113 conn(0x7f5a000fee50 msgr2=0x7f5a00101290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:24:48.479 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:48.477+0000 7f5a05e00640 1 -- 172.21.15.47:0/2188428113 shutdown_connections 2024-03-21T10:24:48.479 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:48.477+0000 7f5a05e00640 1 -- 172.21.15.47:0/2188428113 wait complete. 2024-03-21T10:24:48.481 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:48.478+0000 7f5a05e00640 1 Processor -- start 2024-03-21T10:24:48.481 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:48.478+0000 7f5a05e00640 1 -- start start 2024-03-21T10:24:48.481 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:48.479+0000 7f5a05e00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5a001038e0 0x7f5a00199b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:24:48.481 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:48.479+0000 7f5a05e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5a00104ad0 0x7f5a0019a040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:24:48.481 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:48.479+0000 7f5a05e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5a00105490 0x7f5a001a10b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:24:48.481 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:48.479+0000 7f5a05e00640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a0010f3f0 con 0x7f5a00105490 2024-03-21T10:24:48.481 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:48.479+0000 7f5a05e00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a0010f270 con 0x7f5a00104ad0 2024-03-21T10:24:48.481 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:48.479+0000 7f5a05e00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a0010f570 con 0x7f5a001038e0 2024-03-21T10:24:48.481 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:48.479+0000 7f59ff400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5a00105490 0x7f5a001a10b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:24:48.481 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:48.479+0000 7f59ff400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5a00105490 0x7f5a001a10b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:44358/0 (socket says 172.21.15.47:44358) 2024-03-21T10:24:48.481 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:48.479+0000 7f59ff400640 1 -- 172.21.15.47:0/4107755992 learned_addr learned my addr 172.21.15.47:0/4107755992 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:24:48.481 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:48.479+0000 7f59fe000640 1 --2- 172.21.15.47:0/4107755992 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5a00104ad0 0x7f5a0019a040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:24:48.481 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:48.479+0000 7f59fea00640 1 --2- 172.21.15.47:0/4107755992 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5a001038e0 0x7f5a00199b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:24:48.482 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:48.479+0000 7f59ff400640 1 -- 172.21.15.47:0/4107755992 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5a001038e0 msgr2=0x7f5a00199b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:24:48.482 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:48.479+0000 7f59ff400640 1 --2- 172.21.15.47:0/4107755992 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5a001038e0 0x7f5a00199b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:24:48.482 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:48.479+0000 7f59ff400640 1 -- 172.21.15.47:0/4107755992 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5a00104ad0 msgr2=0x7f5a0019a040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:24:48.482 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:48.479+0000 7f59ff400640 1 --2- 172.21.15.47:0/4107755992 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5a00104ad0 0x7f5a0019a040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:24:48.482 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:48.479+0000 7f59ff400640 1 -- 172.21.15.47:0/4107755992 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a001a1720 con 0x7f5a00105490 2024-03-21T10:24:48.482 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:48.480+0000 7f59ff400640 1 --2- 172.21.15.47:0/4107755992 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5a00105490 0x7f5a001a10b0 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f59f800eaa0 tx=0x7f59f800ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:24:48.483 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:48.480+0000 7f5a04a00640 1 -- 172.21.15.47:0/4107755992 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59f800cdb0 con 0x7f5a00105490 2024-03-21T10:24:48.483 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:48.480+0000 7f5a04a00640 1 -- 172.21.15.47:0/4107755992 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f59f8005020 con 0x7f5a00105490 2024-03-21T10:24:48.483 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:48.480+0000 7f5a04a00640 1 -- 172.21.15.47:0/4107755992 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59f80116d0 con 0x7f5a00105490 2024-03-21T10:24:48.483 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:48.480+0000 7f5a05e00640 1 -- 172.21.15.47:0/4107755992 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5a001a1a10 con 0x7f5a00105490 2024-03-21T10:24:48.483 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:48.480+0000 7f5a05e00640 1 -- 172.21.15.47:0/4107755992 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5a001a1f10 con 0x7f5a00105490 2024-03-21T10:24:48.483 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:48.482+0000 7f5a05e00640 1 -- 172.21.15.47:0/4107755992 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5a0010ac50 con 0x7f5a00105490 2024-03-21T10:24:48.484 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:48.484+0000 7f5a04a00640 1 -- 172.21.15.47:0/4107755992 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 18) v1 ==== 137854+0+0 (secure 0 0 0) 0x7f59f80051d0 con 0x7f5a00105490 2024-03-21T10:24:48.486 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:48.485+0000 7f5a04a00640 1 --2- 172.21.15.47:0/4107755992 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f59e40a2f20 0x7f59e40a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:24:48.486 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:48.485+0000 7f5a04a00640 1 -- 172.21.15.47:0/4107755992 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9216+0+0 (secure 0 0 0) 0x7f59f80cfa70 con 0x7f5a00105490 2024-03-21T10:24:48.486 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:48.485+0000 7f59fea00640 1 --2- 172.21.15.47:0/4107755992 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f59e40a2f20 0x7f59e40a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:24:48.487 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:48.486+0000 7f59fea00640 1 --2- 172.21.15.47:0/4107755992 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f59e40a2f20 0x7f59e40a53e0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f59f0005fd0 tx=0x7f59f0005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:24:48.492 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:48.491+0000 7f5a04a00640 1 -- 172.21.15.47:0/4107755992 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f59f8016030 con 0x7f5a00105490 2024-03-21T10:24:48.696 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:48.695+0000 7f5a05e00640 1 -- 172.21.15.47:0/4107755992 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f5a0002d070 con 0x7f5a00105490 2024-03-21T10:24:48.699 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:48.698+0000 7f5a04a00640 1 -- 172.21.15.47:0/4107755992 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v95) v1 ==== 74+0+24849 (secure 0 0 0) 0x7f59f809c4d0 con 0x7f5a00105490 2024-03-21T10:24:48.700 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-21T10:24:48.700 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":95,"fsid":"3056d51a-e76b-11ee-95cd-87774f69a715","created":"2024-03-21T10:11:17.019965+0000","modified":"2024-03-21T10:23:27.619804+0000","last_up_change":"2024-03-21T10:20:52.675328+0000","last_in_change":"2024-03-21T10:20:27.648212+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-21T10:15:40.553114+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-21T10:21:56.399518+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-21T10:21:57.812940+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-21T10:22:01.822568+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"91","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"b0c7a102-8d89-428b-b793-cbbe67151f50","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6802","nonce":775837313},{"type":"v1","addr":"172.21.15.47:6803","nonce":775837313}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6804","nonce":775837313},{"type":"v1","addr":"172.21.15.47:6805","nonce":775837313}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6808","nonce":775837313},{"type":"v1","addr":"172.21.15.47:6809","nonce":775837313}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6806","nonce":775837313},{"type":"v1","addr":"172.21.15.47:6807","nonce":775837313}]},"public_addr":"172.21.15.47:6803/775837313","cluster_addr":"172.21.15.47:6805/775837313","heartbeat_back_addr":"172.21.15.47:6809/775837313","heartbeat_front_addr":"172.21.15.47:6807/775837313","state":["exists","up"]},{"osd":1,"uuid":"07bb81bb-7cf3-4bbc-ac9e-802f95185d11","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6800","nonce":2923304710},{"type":"v1","addr":"172.21.15.100:6801","nonce":2923304710}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":2923304710},{"type":"v1","addr":"172.21.15.100:6803","nonce":2923304710}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":2923304710},{"type":"v1","addr":"172.21.15.100:6807","nonce":2923304710}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6804","nonce":2923304710},{"type":"v1","addr":"172.21.15.100:6805","nonce":2923304710}]},"public_addr":"172.21.15.100:6801/2923304710","cluster_addr":"172.21.15.100:6803/2923304710","heartbeat_back_addr":"172.21.15.100:6807/2923304710","heartbeat_front_addr":"172.21.15.100:6805/2923304710","state":["exists","up"]},{"osd":2,"uuid":"88ce76b9-6081-4836-aad0-8c249c56e8a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":171377005},{"type":"v1","addr":"172.21.15.157:6801","nonce":171377005}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":171377005},{"type":"v1","addr":"172.21.15.157:6803","nonce":171377005}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":171377005},{"type":"v1","addr":"172.21.15.157:6807","nonce":171377005}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":171377005},{"type":"v1","addr":"172.21.15.157:6805","nonce":171377005}]},"public_addr":"172.21.15.157:6801/171377005","cluster_addr":"172.21.15.157:6803/171377005","heartbeat_back_addr":"172.21.15.157:6807/171377005","heartbeat_front_addr":"172.21.15.157:6805/171377005","state":["exists","up"]},{"osd":3,"uuid":"518aa33d-263f-45e6-aab5-2201ceb40e74","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":27,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6810","nonce":640760101},{"type":"v1","addr":"172.21.15.47:6811","nonce":640760101}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6812","nonce":640760101},{"type":"v1","addr":"172.21.15.47:6813","nonce":640760101}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6816","nonce":640760101},{"type":"v1","addr":"172.21.15.47:6817","nonce":640760101}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6814","nonce":640760101},{"type":"v1","addr":"172.21.15.47:6815","nonce":640760101}]},"public_addr":"172.21.15.47:6811/640760101","cluster_addr":"172.21.15.47:6813/640760101","heartbeat_back_addr":"172.21.15.47:6817/640760101","heartbeat_front_addr":"172.21.15.47:6815/640760101","state":["exists","up"]},{"osd":4,"uuid":"987bde83-ecc6-42e5-bd56-c143b3de4226","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":35,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6808","nonce":2785478639},{"type":"v1","addr":"172.21.15.100:6809","nonce":2785478639}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6810","nonce":2785478639},{"type":"v1","addr":"172.21.15.100:6811","nonce":2785478639}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6814","nonce":2785478639},{"type":"v1","addr":"172.21.15.100:6815","nonce":2785478639}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":2785478639},{"type":"v1","addr":"172.21.15.100:6813","nonce":2785478639}]},"public_addr":"172.21.15.100:6809/2785478639","cluster_addr":"172.21.15.100:6811/2785478639","heartbeat_back_addr":"172.21.15.100:6815/2785478639","heartbeat_front_addr":"172.21.15.100:6813/2785478639","state":["exists","up"]},{"osd":5,"uuid":"012d2188-3a61-4a5f-8d01-2001c15ffbe2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":40,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":1525645410},{"type":"v1","addr":"172.21.15.157:6809","nonce":1525645410}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":1525645410},{"type":"v1","addr":"172.21.15.157:6811","nonce":1525645410}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":1525645410},{"type":"v1","addr":"172.21.15.157:6815","nonce":1525645410}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":1525645410},{"type":"v1","addr":"172.21.15.157:6813","nonce":1525645410}]},"public_addr":"172.21.15.157:6809/1525645410","cluster_addr":"172.21.15.157:6811/1525645410","heartbeat_back_addr":"172.21.15.157:6815/1525645410","heartbeat_front_addr":"172.21.15.157:6813/1525645410","state":["exists","up"]},{"osd":6,"uuid":"c12c14b4-85f0-4d88-937c-de4f556437de","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6818","nonce":2321642790},{"type":"v1","addr":"172.21.15.47:6819","nonce":2321642790}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6820","nonce":2321642790},{"type":"v1","addr":"172.21.15.47:6821","nonce":2321642790}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6824","nonce":2321642790},{"type":"v1","addr":"172.21.15.47:6825","nonce":2321642790}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6822","nonce":2321642790},{"type":"v1","addr":"172.21.15.47:6823","nonce":2321642790}]},"public_addr":"172.21.15.47:6819/2321642790","cluster_addr":"172.21.15.47:6821/2321642790","heartbeat_back_addr":"172.21.15.47:6825/2321642790","heartbeat_front_addr":"172.21.15.47:6823/2321642790","state":["exists","up"]},{"osd":7,"uuid":"3c87c6d0-1d1f-4294-9917-aaaad7dda824","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6816","nonce":4104727330},{"type":"v1","addr":"172.21.15.100:6817","nonce":4104727330}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":4104727330},{"type":"v1","addr":"172.21.15.100:6819","nonce":4104727330}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":4104727330},{"type":"v1","addr":"172.21.15.100:6823","nonce":4104727330}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6820","nonce":4104727330},{"type":"v1","addr":"172.21.15.100:6821","nonce":4104727330}]},"public_addr":"172.21.15.100:6817/4104727330","cluster_addr":"172.21.15.100:6819/4104727330","heartbeat_back_addr":"172.21.15.100:6823/4104727330","heartbeat_front_addr":"172.21.15.100:6821/4104727330","state":["exists","up"]},{"osd":8,"uuid":"502d65cf-53c8-4936-8de4-e11b2a6e1e18","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":61,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":4067715347},{"type":"v1","addr":"172.21.15.157:6817","nonce":4067715347}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":4067715347},{"type":"v1","addr":"172.21.15.157:6819","nonce":4067715347}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":4067715347},{"type":"v1","addr":"172.21.15.157:6823","nonce":4067715347}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":4067715347},{"type":"v1","addr":"172.21.15.157:6821","nonce":4067715347}]},"public_addr":"172.21.15.157:6817/4067715347","cluster_addr":"172.21.15.157:6819/4067715347","heartbeat_back_addr":"172.21.15.157:6823/4067715347","heartbeat_front_addr":"172.21.15.157:6821/4067715347","state":["exists","up"]},{"osd":9,"uuid":"5ea1f5be-8458-4c93-9e61-c2e867a514f3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6826","nonce":1982108738},{"type":"v1","addr":"172.21.15.47:6827","nonce":1982108738}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6828","nonce":1982108738},{"type":"v1","addr":"172.21.15.47:6829","nonce":1982108738}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6832","nonce":1982108738},{"type":"v1","addr":"172.21.15.47:6833","nonce":1982108738}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6830","nonce":1982108738},{"type":"v1","addr":"172.21.15.47:6831","nonce":1982108738}]},"public_addr":"172.21.15.47:6827/1982108738","cluster_addr":"172.21.15.47:6829/1982108738","heartbeat_back_addr":"172.21.15.47:6833/1982108738","heartbeat_front_addr":"172.21.15.47:6831/1982108738","state":["exists","up"]},{"osd":10,"uuid":"2da3da90-a0be-4a54-a9ff-b2d6c9b36870","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":74,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":396391028},{"type":"v1","addr":"172.21.15.100:6825","nonce":396391028}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":396391028},{"type":"v1","addr":"172.21.15.100:6827","nonce":396391028}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6830","nonce":396391028},{"type":"v1","addr":"172.21.15.100:6831","nonce":396391028}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":396391028},{"type":"v1","addr":"172.21.15.100:6829","nonce":396391028}]},"public_addr":"172.21.15.100:6825/396391028","cluster_addr":"172.21.15.100:6827/396391028","heartbeat_back_addr":"172.21.15.100:6831/396391028","heartbeat_front_addr":"172.21.15.100:6829/396391028","state":["exists","up"]},{"osd":11,"uuid":"11798d41-7f7e-4319-95b8-8d3d0946bde9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":82,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":2671029483},{"type":"v1","addr":"172.21.15.157:6825","nonce":2671029483}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":2671029483},{"type":"v1","addr":"172.21.15.157:6827","nonce":2671029483}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":2671029483},{"type":"v1","addr":"172.21.15.157:6831","nonce":2671029483}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":2671029483},{"type":"v1","addr":"172.21.15.157:6829","nonce":2671029483}]},"public_addr":"172.21.15.157:6825/2671029483","cluster_addr":"172.21.15.157:6827/2671029483","heartbeat_back_addr":"172.21.15.157:6831/2671029483","heartbeat_front_addr":"172.21.15.157:6829/2671029483","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T10:14:31.822399+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T10:15:04.175721+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T10:15:35.902595+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T10:16:12.088891+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T10:16:46.428704+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T10:17:20.158270+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T10:17:55.915253+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T10:18:30.064641+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T10:19:04.410004+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T10:19:40.605181+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T10:20:15.636605+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T10:20:50.151365+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.11","mappings":[{"from":7,"to":11}]},{"pgid":"2.14","mappings":[{"from":2,"to":4}]},{"pgid":"2.1a","mappings":[{"from":8,"to":1}]},{"pgid":"2.1b","mappings":[{"from":2,"to":4}]},{"pgid":"2.1d","mappings":[{"from":8,"to":11}]},{"pgid":"3.0","mappings":[{"from":4,"to":8},{"from":5,"to":9}]},{"pgid":"3.6","mappings":[{"from":3,"to":7}]},{"pgid":"3.1b","mappings":[{"from":4,"to":6}]},{"pgid":"3.33","mappings":[{"from":5,"to":9}]},{"pgid":"3.3c","mappings":[{"from":4,"to":9}]},{"pgid":"4.c","mappings":[{"from":9,"to":10}]},{"pgid":"4.f","mappings":[{"from":9,"to":8}]},{"pgid":"4.19","mappings":[{"from":9,"to":8}]},{"pgid":"4.1d","mappings":[{"from":5,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.47:0/356822431":"2024-03-22T10:12:25.844530+0000","172.21.15.47:0/1167490879":"2024-03-22T10:12:25.844530+0000","172.21.15.47:0/1540718138":"2024-03-22T10:12:25.844530+0000","172.21.15.47:6801/3836903972":"2024-03-22T10:12:25.844530+0000","172.21.15.47:0/235436602":"2024-03-22T10:11:50.301838+0000","172.21.15.47:0/2052815572":"2024-03-22T10:11:50.301838+0000","172.21.15.47:6800/3942658432":"2024-03-22T10:11:50.301838+0000","172.21.15.47:6801/3942658432":"2024-03-22T10:11:50.301838+0000","172.21.15.47:0/1419516257":"2024-03-22T10:11:50.301838+0000","172.21.15.47:6800/3836903972":"2024-03-22T10:12:25.844530+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-21T10:24:48.702 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:48.701+0000 7f5a05e00640 1 -- 172.21.15.47:0/4107755992 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f59e40a2f20 msgr2=0x7f59e40a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:24:48.702 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:48.701+0000 7f5a05e00640 1 --2- 172.21.15.47:0/4107755992 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f59e40a2f20 0x7f59e40a53e0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f59f0005fd0 tx=0x7f59f0005850 comp rx=0 tx=0).stop 2024-03-21T10:24:48.703 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:48.701+0000 7f5a05e00640 1 -- 172.21.15.47:0/4107755992 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5a00105490 msgr2=0x7f5a001a10b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:24:48.703 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:48.701+0000 7f5a05e00640 1 --2- 172.21.15.47:0/4107755992 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5a00105490 0x7f5a001a10b0 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f59f800eaa0 tx=0x7f59f800ef60 comp rx=0 tx=0).stop 2024-03-21T10:24:48.703 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:48.701+0000 7f5a05e00640 1 -- 172.21.15.47:0/4107755992 shutdown_connections 2024-03-21T10:24:48.703 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:48.701+0000 7f5a05e00640 1 --2- 172.21.15.47:0/4107755992 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f59e40a2f20 0x7f59e40a53e0 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:24:48.703 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:48.701+0000 7f5a05e00640 1 --2- 172.21.15.47:0/4107755992 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5a00105490 0x7f5a001a10b0 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:24:48.703 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:48.701+0000 7f5a05e00640 1 --2- 172.21.15.47:0/4107755992 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5a00104ad0 0x7f5a0019a040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:24:48.703 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:48.701+0000 7f5a05e00640 1 --2- 172.21.15.47:0/4107755992 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5a001038e0 0x7f5a00199b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:24:48.703 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:48.701+0000 7f5a05e00640 1 -- 172.21.15.47:0/4107755992 >> 172.21.15.47:0/4107755992 conn(0x7f5a000fee50 msgr2=0x7f5a00107e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:24:48.703 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:48.701+0000 7f5a05e00640 1 -- 172.21.15.47:0/4107755992 shutdown_connections 2024-03-21T10:24:48.703 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:48.701+0000 7f5a05e00640 1 -- 172.21.15.47:0/4107755992 wait complete. 2024-03-21T10:24:48.713 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:24:48 smithi047 ceph-mon[31547]: from='client.? 172.21.15.47:0/3438265290' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-21T10:24:48.714 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:24:48 smithi047 ceph-mon[31547]: from='client.? 172.21.15.47:0/261397232' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-03-21T10:24:48.714 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:24:48 smithi047 ceph-mon[31547]: from='client.? 172.21.15.47:0/3573292364' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-21T10:24:48.726 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 29, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-03-21T10:21:58.329063+0000', 'modified': '2024-03-21T10:22:29.588509+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 5, 'in': [0, 1, 2, 3, 4], 'up': {'mds_0': 34535, 'mds_1': 34493, 'mds_2': 34454, 'mds_3': 34427, 'mds_4': 14619}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_14604': {'gid': 14604, 'name': 'g', 'rank': 4, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.47:6839/2881335356', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.47:6838', 'nonce': 2881335356}, {'type': 'v1', 'addr': '172.21.15.47:6839', 'nonce': 2881335356}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_14619': {'gid': 14619, 'name': 'a', 'rank': 4, 'incarnation': 27, 'state': 'up:active', 'state_seq': 11, 'addr': '172.21.15.47:6841/3401578978', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.47:6840', 'nonce': 3401578978}, {'type': 'v1', 'addr': '172.21.15.47:6841', 'nonce': 3401578978}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_24406': {'gid': 24406, 'name': 'l', 'rank': 3, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.157:6833/4000010136', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.157:6832', 'nonce': 4000010136}, {'type': 'v1', 'addr': '172.21.15.157:6833', 'nonce': 4000010136}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_24457': {'gid': 24457, 'name': 'f', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.157:6835/3902868651', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.157:6834', 'nonce': 3902868651}, {'type': 'v1', 'addr': '172.21.15.157:6835', 'nonce': 3902868651}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_24475': {'gid': 24475, 'name': 'c', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.157:6837/1908087204', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.157:6836', 'nonce': 1908087204}, {'type': 'v1', 'addr': '172.21.15.157:6837', 'nonce': 1908087204}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_24499': {'gid': 24499, 'name': 'i', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.157:6839/4064242876', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.157:6838', 'nonce': 4064242876}, {'type': 'v1', 'addr': '172.21.15.157:6839', 'nonce': 4064242876}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_34427': {'gid': 34427, 'name': 'k', 'rank': 3, 'incarnation': 25, 'state': 'up:active', 'state_seq': 20, 'addr': '172.21.15.100:6833/2723302038', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.100:6832', 'nonce': 2723302038}, {'type': 'v1', 'addr': '172.21.15.100:6833', 'nonce': 2723302038}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_34454': {'gid': 34454, 'name': 'e', 'rank': 2, 'incarnation': 22, 'state': 'up:active', 'state_seq': 17, 'addr': '172.21.15.100:6835/2597849957', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.100:6834', 'nonce': 2597849957}, {'type': 'v1', 'addr': '172.21.15.100:6835', 'nonce': 2597849957}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_34493': {'gid': 34493, 'name': 'h', 'rank': 1, 'incarnation': 20, 'state': 'up:active', 'state_seq': 13, 'addr': '172.21.15.100:6837/3497644162', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.100:6836', 'nonce': 3497644162}, {'type': 'v1', 'addr': '172.21.15.100:6837', 'nonce': 3497644162}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_34535': {'gid': 34535, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.100:6839/4173168440', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.100:6838', 'nonce': 4173168440}, {'type': 'v1', 'addr': '172.21.15.100:6839', 'nonce': 4173168440}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 34535, 'qdb_cluster': [34493, 34535, 34454, 34427, 14619]} 2024-03-21T10:24:48.726 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 5/5 2024-03-21T10:24:48.726 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2024-03-21T10:24:48.729 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:24:48 smithi100 ceph-mon[35193]: from='client.? 172.21.15.47:0/3438265290' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-21T10:24:48.729 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:24:48 smithi100 ceph-mon[35193]: from='client.? 172.21.15.47:0/261397232' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-03-21T10:24:48.729 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:24:48 smithi100 ceph-mon[35193]: from='client.? 172.21.15.47:0/3573292364' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-21T10:24:48.739 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:24:48 smithi157 ceph-mon[35581]: from='client.? 172.21.15.47:0/3438265290' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-21T10:24:48.740 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:24:48 smithi157 ceph-mon[35581]: from='client.? 172.21.15.47:0/261397232' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-03-21T10:24:48.740 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:24:48 smithi157 ceph-mon[35581]: from='client.? 172.21.15.47:0/3573292364' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-21T10:24:48.955 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:48.954+0000 7f8130e00640 1 -- 172.21.15.47:0/74026245 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f812c0ffdf0 msgr2=0x7f812c100210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:24:48.955 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:48.954+0000 7f8130e00640 1 --2- 172.21.15.47:0/74026245 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f812c0ffdf0 0x7f812c100210 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f811c009900 tx=0x7f811c02bc30 comp rx=0 tx=0).stop 2024-03-21T10:24:48.955 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:48.954+0000 7f8130e00640 1 -- 172.21.15.47:0/74026245 shutdown_connections 2024-03-21T10:24:48.955 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:48.954+0000 7f8130e00640 1 --2- 172.21.15.47:0/74026245 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f812c1010f0 0x7f812c101570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:24:48.955 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:48.954+0000 7f8130e00640 1 --2- 172.21.15.47:0/74026245 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f812c100750 0x7f812c100bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:24:48.956 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:48.954+0000 7f8130e00640 1 --2- 172.21.15.47:0/74026245 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f812c0ffdf0 0x7f812c100210 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:24:48.956 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:48.954+0000 7f8130e00640 1 -- 172.21.15.47:0/74026245 >> 172.21.15.47:0/74026245 conn(0x7f812c0fba10 msgr2=0x7f812c0fde30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:24:48.956 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:48.954+0000 7f8130e00640 1 -- 172.21.15.47:0/74026245 shutdown_connections 2024-03-21T10:24:48.956 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:48.954+0000 7f8130e00640 1 -- 172.21.15.47:0/74026245 wait complete. 2024-03-21T10:24:48.958 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:48.955+0000 7f8130e00640 1 Processor -- start 2024-03-21T10:24:48.959 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:48.955+0000 7f8130e00640 1 -- start start 2024-03-21T10:24:48.959 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:48.955+0000 7f8130e00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f812c0ffdf0 0x7f812c1999c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:24:48.959 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:48.955+0000 7f8130e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f812c100750 0x7f812c199f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:24:48.959 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:48.955+0000 7f8130e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f812c1010f0 0x7f812c1a0f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:24:48.959 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:48.955+0000 7f8130e00640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f812c10f2e0 con 0x7f812c100750 2024-03-21T10:24:48.959 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:48.955+0000 7f8130e00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f812c10f160 con 0x7f812c1010f0 2024-03-21T10:24:48.959 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:48.955+0000 7f8130e00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f812c10f460 con 0x7f812c0ffdf0 2024-03-21T10:24:48.959 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:48.956+0000 7f812aa00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f812c100750 0x7f812c199f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:24:48.959 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:48.956+0000 7f812aa00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f812c100750 0x7f812c199f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:44378/0 (socket says 172.21.15.47:44378) 2024-03-21T10:24:48.959 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:48.956+0000 7f812aa00640 1 -- 172.21.15.47:0/269807618 learned_addr learned my addr 172.21.15.47:0/269807618 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:24:48.959 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:48.956+0000 7f812be00640 1 --2- 172.21.15.47:0/269807618 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f812c1010f0 0x7f812c1a0f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:24:48.959 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:48.956+0000 7f812b400640 1 --2- 172.21.15.47:0/269807618 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f812c0ffdf0 0x7f812c1999c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:24:48.959 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:48.956+0000 7f812aa00640 1 -- 172.21.15.47:0/269807618 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f812c0ffdf0 msgr2=0x7f812c1999c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:24:48.960 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:48.956+0000 7f812aa00640 1 --2- 172.21.15.47:0/269807618 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f812c0ffdf0 0x7f812c1999c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:24:48.960 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:48.956+0000 7f812aa00640 1 -- 172.21.15.47:0/269807618 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f812c1010f0 msgr2=0x7f812c1a0f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:24:48.960 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:48.956+0000 7f812aa00640 1 --2- 172.21.15.47:0/269807618 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f812c1010f0 0x7f812c1a0f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:24:48.960 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:48.956+0000 7f812aa00640 1 -- 172.21.15.47:0/269807618 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f812c1a15e0 con 0x7f812c100750 2024-03-21T10:24:48.960 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:48.956+0000 7f812be00640 1 --2- 172.21.15.47:0/269807618 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f812c1010f0 0x7f812c1a0f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T10:24:48.960 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:48.957+0000 7f812aa00640 1 --2- 172.21.15.47:0/269807618 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f812c100750 0x7f812c199f00 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f811800d770 tx=0x7f811800dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:24:48.960 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:48.957+0000 7f8123e00640 1 -- 172.21.15.47:0/269807618 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8118017070 con 0x7f812c100750 2024-03-21T10:24:48.960 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:48.957+0000 7f8123e00640 1 -- 172.21.15.47:0/269807618 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f81180054d0 con 0x7f812c100750 2024-03-21T10:24:48.960 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:48.957+0000 7f8130e00640 1 -- 172.21.15.47:0/269807618 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f812c1a18d0 con 0x7f812c100750 2024-03-21T10:24:48.960 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:48.957+0000 7f8123e00640 1 -- 172.21.15.47:0/269807618 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8118004cf0 con 0x7f812c100750 2024-03-21T10:24:48.960 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:48.957+0000 7f8130e00640 1 -- 172.21.15.47:0/269807618 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f812c1a1dd0 con 0x7f812c100750 2024-03-21T10:24:48.962 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:48.961+0000 7f8135039740 1 Processor -- start 2024-03-21T10:24:48.962 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:48.961+0000 7f8135039740 1 -- start start 2024-03-21T10:24:48.963 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:48.961+0000 7f8135039740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x558992e75d80 0x558992e761a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:24:48.963 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:48.961+0000 7f8135039740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558992e76770 0x558992e7b270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:24:48.963 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:48.961+0000 7f8135039740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558992e7b8e0 0x558992e7dd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:24:48.963 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:48.961+0000 7f8135039740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x558992ddac90 con 0x558992e75d80 2024-03-21T10:24:48.963 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:48.961+0000 7f8135039740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x558992d83980 con 0x558992e7b8e0 2024-03-21T10:24:48.964 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:48.961+0000 7f8135039740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x558992dd6d30 con 0x558992e76770 2024-03-21T10:24:48.964 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:48.961+0000 7f8123e00640 1 -- 172.21.15.47:0/269807618 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 18) v1 ==== 137854+0+0 (secure 0 0 0) 0x7f8118004e90 con 0x7f812c100750 2024-03-21T10:24:48.964 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:48.962+0000 7f812be00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x558992e75d80 0x558992e761a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:24:48.964 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:48.962+0000 7f812be00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x558992e75d80 0x558992e761a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:44384/0 (socket says 172.21.15.47:44384) 2024-03-21T10:24:48.964 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:48.962+0000 7f812be00640 1 -- 172.21.15.47:0/1582449832 learned_addr learned my addr 172.21.15.47:0/1582449832 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:24:48.964 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:48.962+0000 7f812aa00640 1 --2- 172.21.15.47:0/1582449832 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558992e7b8e0 0x558992e7dd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:24:48.964 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:48.962+0000 7f812b400640 1 --2- 172.21.15.47:0/1582449832 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558992e76770 0x558992e7b270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:24:48.964 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:48.962+0000 7f812be00640 1 -- 172.21.15.47:0/1582449832 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558992e76770 msgr2=0x558992e7b270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:24:48.964 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:48.962+0000 7f812be00640 1 --2- 172.21.15.47:0/1582449832 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558992e76770 0x558992e7b270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:24:48.964 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:48.962+0000 7f812be00640 1 -- 172.21.15.47:0/1582449832 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558992e7b8e0 msgr2=0x558992e7dd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:24:48.964 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:48.962+0000 7f812be00640 1 --2- 172.21.15.47:0/1582449832 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558992e7b8e0 0x558992e7dd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:24:48.964 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:48.962+0000 7f812be00640 1 -- 172.21.15.47:0/1582449832 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558992d9b510 con 0x558992e75d80 2024-03-21T10:24:48.964 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:48.962+0000 7f812b400640 1 --2- 172.21.15.47:0/1582449832 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558992e76770 0x558992e7b270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T10:24:48.966 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:48.963+0000 7f812be00640 1 --2- 172.21.15.47:0/1582449832 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x558992e75d80 0x558992e761a0 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f8124009fd0 tx=0x7f81240108e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fe590a4b1605735c server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:24:48.966 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:48.963+0000 7f8120c00640 1 -- 172.21.15.47:0/1582449832 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8124011760 con 0x558992e75d80 2024-03-21T10:24:48.966 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:48.963+0000 7f8123e00640 1 --2- 172.21.15.47:0/269807618 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f81100a2f20 0x7f81100a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:24:48.966 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:48.963+0000 7f8120c00640 1 -- 172.21.15.47:0/1582449832 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f8124011db0 con 0x558992e75d80 2024-03-21T10:24:48.966 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:48.963+0000 7f8120c00640 1 -- 172.21.15.47:0/1582449832 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f812400e3c0 con 0x558992e75d80 2024-03-21T10:24:48.966 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:48.963+0000 7f8123e00640 1 -- 172.21.15.47:0/269807618 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9216+0+0 (secure 0 0 0) 0x7f81180cf2e0 con 0x7f812c100750 2024-03-21T10:24:48.966 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:48.963+0000 7f812b400640 1 --2- 172.21.15.47:0/269807618 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f81100a2f20 0x7f81100a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:24:48.966 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:48.964+0000 7f8135039740 1 -- 172.21.15.47:0/1582449832 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x558992e75d80 msgr2=0x558992e761a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:24:48.966 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:48.964+0000 7f8135039740 1 --2- 172.21.15.47:0/1582449832 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x558992e75d80 0x558992e761a0 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f8124009fd0 tx=0x7f81240108e0 comp rx=0 tx=0).stop 2024-03-21T10:24:48.967 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:48.964+0000 7f8135039740 1 -- 172.21.15.47:0/1582449832 shutdown_connections 2024-03-21T10:24:48.967 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:48.964+0000 7f8135039740 1 --2- 172.21.15.47:0/1582449832 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558992e7b8e0 0x558992e7dd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:24:48.967 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:48.964+0000 7f812b400640 1 --2- 172.21.15.47:0/269807618 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f81100a2f20 0x7f81100a53e0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f811c009230 tx=0x7f811c02c7e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:24:48.967 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:48.964+0000 7f8135039740 1 --2- 172.21.15.47:0/1582449832 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558992e76770 0x558992e7b270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:24:48.967 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:48.964+0000 7f8135039740 1 --2- 172.21.15.47:0/1582449832 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x558992e75d80 0x558992e761a0 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:24:48.967 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:48.964+0000 7f8135039740 1 -- 172.21.15.47:0/1582449832 >> 172.21.15.47:0/1582449832 conn(0x558992e712f0 msgr2=0x558992e716f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:24:48.967 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:48.964+0000 7f8135039740 1 -- 172.21.15.47:0/1582449832 shutdown_connections 2024-03-21T10:24:48.967 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:48.964+0000 7f8135039740 1 -- 172.21.15.47:0/1582449832 wait complete. 2024-03-21T10:24:48.967 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:48.965+0000 7f8135039740 1 -- start start 2024-03-21T10:24:48.967 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:48.965+0000 7f8135039740 1 Processor -- start 2024-03-21T10:24:48.970 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:48.965+0000 7f8135039740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558992e75d80 0x558992e802c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:24:48.970 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:48.965+0000 7f8135039740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x558992e76770 0x558992e80800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:24:48.970 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:48.966+0000 7f8135039740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558992e7b8e0 0x558992e87840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:24:48.970 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:48.966+0000 7f8135039740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x558992e7e890 con 0x558992e76770 2024-03-21T10:24:48.970 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:48.966+0000 7f8135039740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x558992d96dd0 con 0x558992e75d80 2024-03-21T10:24:48.970 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:48.966+0000 7f8135039740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x558992e7ea10 con 0x558992e7b8e0 2024-03-21T10:24:48.970 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:48.966+0000 7f8135039740 10 client.0 _ll_register_callbacks cb 0x558992aeeac0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T10:24:48.970 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:48.966+0000 7f812aa00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x558992e76770 0x558992e80800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:24:48.970 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:48.966+0000 7f812b400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558992e75d80 0x558992e802c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:24:48.970 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:48.966+0000 7f812aa00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x558992e76770 0x558992e80800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:44400/0 (socket says 172.21.15.47:44400) 2024-03-21T10:24:48.970 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:48.966+0000 7f812aa00640 1 -- 172.21.15.47:0/3553281337 learned_addr learned my addr 172.21.15.47:0/3553281337 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:24:48.971 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:48.966+0000 7f812aa00640 1 -- 172.21.15.47:0/3553281337 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558992e7b8e0 msgr2=0x558992e87840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:24:48.971 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:48.966+0000 7f812aa00640 1 --2- 172.21.15.47:0/3553281337 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558992e7b8e0 0x558992e87840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:24:48.971 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:48.966+0000 7f812aa00640 1 -- 172.21.15.47:0/3553281337 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558992e75d80 msgr2=0x558992e802c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:24:48.971 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:48.966+0000 7f812aa00640 1 --2- 172.21.15.47:0/3553281337 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558992e75d80 0x558992e802c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:24:48.971 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:48.966+0000 7f812aa00640 1 -- 172.21.15.47:0/3553281337 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x558992aef2d0 con 0x558992e76770 2024-03-21T10:24:48.971 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:48.967+0000 7f812aa00640 1 --2- 172.21.15.47:0/3553281337 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x558992e76770 0x558992e80800 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f8118097fd0 tx=0x7f8118005740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b746d7653ab5fd07 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:24:48.971 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:48.967+0000 7f8115600640 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-03-21T10:24:48.971 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:48.967+0000 7f8115600640 1 -- 172.21.15.47:0/3553281337 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8118020070 con 0x558992e76770 2024-03-21T10:24:48.971 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:48.967+0000 7f8115600640 1 -- 172.21.15.47:0/3553281337 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9216+0+0 (secure 0 0 0) 0x7f8118084b90 con 0x558992e76770 2024-03-21T10:24:48.971 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:48.967+0000 7f8135039740 1 -- 172.21.15.47:0/3553281337 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f8124011db0 con 0x558992e76770 2024-03-21T10:24:48.971 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:48.967+0000 7f8115600640 1 -- 172.21.15.47:0/3553281337 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8118097040 con 0x558992e76770 2024-03-21T10:24:48.971 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:48.967+0000 7f8135039740 10 client.15408 fetch_fsmap learned FSMap version 29 2024-03-21T10:24:48.971 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:48.967+0000 7f8135039740 1 -- 172.21.15.47:0/3553281337 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x558992bb96b0 con 0x558992e76770 2024-03-21T10:24:48.971 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:48.967+0000 7f8115600640 1 -- 172.21.15.47:0/3553281337 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f8124011db0 con 0x558992e76770 2024-03-21T10:24:48.972 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:48.968+0000 7f8115600640 1 -- 172.21.15.47:0/3553281337 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 29) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f8118097350 con 0x558992e76770 2024-03-21T10:24:48.972 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:48.968+0000 7f8135039740 10 client.15408 fetch_fsmap finished waiting for FSMap version 29 2024-03-21T10:24:48.972 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:48.968+0000 7f8135039740 10 client.15408 resolve_mds: resolved name 'a' to daemon mds.a 2024-03-21T10:24:48.972 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:48.968+0000 7f8135039740 20 client.15408 populate_metadata read hostname 'smithi047' 2024-03-21T10:24:48.972 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:48.968+0000 7f8135039740 1 --2- 172.21.15.47:0/3553281337 >> [v2:172.21.15.47:6840/3401578978,v1:172.21.15.47:6841/3401578978] conn(0x558992e8ca50 0x558992e8bbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:24:48.972 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:48.968+0000 7f8135039740 4 client.15408 mds_command: new command op to 14619 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T10:24:48.972 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:48.968+0000 7f8135039740 1 -- 172.21.15.47:0/3553281337 --> [v2:172.21.15.47:6840/3401578978,v1:172.21.15.47:6841/3401578978] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x558992d7c520 con 0x558992e8ca50 2024-03-21T10:24:48.972 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:48.968+0000 7f812b400640 1 --2- 172.21.15.47:0/3553281337 >> [v2:172.21.15.47:6840/3401578978,v1:172.21.15.47:6841/3401578978] conn(0x558992e8ca50 0x558992e8bbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:24:48.972 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:48.969+0000 7f8115600640 10 client.15408 ms_handle_connect on v2:172.21.15.47:6840/3401578978 2024-03-21T10:24:48.972 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:48.969+0000 7f812b400640 1 --2- 172.21.15.47:0/3553281337 >> [v2:172.21.15.47:6840/3401578978,v1:172.21.15.47:6841/3401578978] conn(0x558992e8ca50 0x558992e8bbd0 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=794a00bbedc62f00 server_cookie=e148af7150ce07e in_seq=0 out_seq=0 2024-03-21T10:24:48.973 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:48.970+0000 7f8115600640 1 -- 172.21.15.47:0/3553281337 <== mds.4 v2:172.21.15.47:6840/3401578978 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x558992d7c520 con 0x558992e8ca50 2024-03-21T10:24:48.973 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:48.970+0000 7f8115600640 10 client.15408 handle_command_reply: tid=0 2024-03-21T10:24:48.973 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:48.970+0000 7f8135039740 1 client.15408 shutdown 2024-03-21T10:24:48.973 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:48.970+0000 7f8135039740 1 -- 172.21.15.47:0/3553281337 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x558992e76770 msgr2=0x558992e80800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:24:48.974 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:48.970+0000 7f8135039740 1 --2- 172.21.15.47:0/3553281337 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x558992e76770 0x558992e80800 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f8118097fd0 tx=0x7f8118005740 comp rx=0 tx=0).stop 2024-03-21T10:24:48.974 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:48.970+0000 7f8135039740 1 -- 172.21.15.47:0/3553281337 shutdown_connections 2024-03-21T10:24:48.974 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:48.970+0000 7f8135039740 1 --2- 172.21.15.47:0/3553281337 >> [v2:172.21.15.47:6840/3401578978,v1:172.21.15.47:6841/3401578978] conn(0x558992e8ca50 0x558992e8bbd0 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:24:48.974 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:48.970+0000 7f8135039740 1 --2- 172.21.15.47:0/3553281337 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558992e7b8e0 0x558992e87840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:24:48.974 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:48.970+0000 7f8135039740 1 --2- 172.21.15.47:0/3553281337 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x558992e76770 0x558992e80800 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:24:48.974 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:48.970+0000 7f8135039740 1 --2- 172.21.15.47:0/3553281337 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558992e75d80 0x558992e802c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:24:48.974 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:48.970+0000 7f8135039740 1 -- 172.21.15.47:0/3553281337 >> 172.21.15.47:0/3553281337 conn(0x558992e712f0 msgr2=0x558992e704f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:24:48.974 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:48.970+0000 7f8115600640 0 client.15408 ms_handle_reset on v2:172.21.15.47:6840/3401578978 2024-03-21T10:24:48.974 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:48.970+0000 7f8135039740 1 -- 172.21.15.47:0/3553281337 shutdown_connections 2024-03-21T10:24:48.974 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:48.970+0000 7f8135039740 1 -- 172.21.15.47:0/3553281337 wait complete. 2024-03-21T10:24:48.974 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:48.971+0000 7f8135039740 20 client.15408 trim_cache size 0 max 16384 2024-03-21T10:24:48.990 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:48.989+0000 7f8135039740 1 Processor -- start 2024-03-21T10:24:48.990 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:48.989+0000 7f8135039740 1 -- start start 2024-03-21T10:24:48.990 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:48.989+0000 7f8135039740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558992e75d80 0x558992e7edb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:24:48.991 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:48.989+0000 7f8135039740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x558992e76770 0x558992e87800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:24:48.991 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:48.989+0000 7f8135039740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558992e7b8e0 0x558992e7f2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:24:48.991 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:48.989+0000 7f8135039740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x558992e8c270 con 0x558992e76770 2024-03-21T10:24:48.991 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:48.989+0000 7f8135039740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x558992e8c8d0 con 0x558992e75d80 2024-03-21T10:24:48.991 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:48.989+0000 7f8135039740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x558992e8cf30 con 0x558992e7b8e0 2024-03-21T10:24:48.991 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:48.990+0000 7f812aa00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x558992e76770 0x558992e87800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:24:48.991 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:48.990+0000 7f812aa00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x558992e76770 0x558992e87800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:44414/0 (socket says 172.21.15.47:44414) 2024-03-21T10:24:48.991 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:48.990+0000 7f812aa00640 1 -- 172.21.15.47:0/2537949653 learned_addr learned my addr 172.21.15.47:0/2537949653 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:24:48.991 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:48.990+0000 7f812b400640 1 --2- 172.21.15.47:0/2537949653 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558992e75d80 0x558992e7edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:24:48.991 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:48.990+0000 7f812be00640 1 --2- 172.21.15.47:0/2537949653 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558992e7b8e0 0x558992e7f2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:24:48.992 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:48.990+0000 7f812aa00640 1 -- 172.21.15.47:0/2537949653 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558992e7b8e0 msgr2=0x558992e7f2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:24:48.992 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:48.990+0000 7f812aa00640 1 --2- 172.21.15.47:0/2537949653 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558992e7b8e0 0x558992e7f2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:24:48.992 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:48.990+0000 7f812aa00640 1 -- 172.21.15.47:0/2537949653 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558992e75d80 msgr2=0x558992e7edb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:24:48.993 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:48.990+0000 7f812aa00640 1 --2- 172.21.15.47:0/2537949653 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558992e75d80 0x558992e7edb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:24:48.993 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:48.990+0000 7f812aa00640 1 -- 172.21.15.47:0/2537949653 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558992e72060 con 0x558992e76770 2024-03-21T10:24:48.993 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:48.990+0000 7f812aa00640 1 --2- 172.21.15.47:0/2537949653 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x558992e76770 0x558992e87800 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f81180998b0 tx=0x7f8118097270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4162711e46585656 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:24:48.993 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:48.991+0000 7f8117e00640 1 -- 172.21.15.47:0/2537949653 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8118020070 con 0x558992e76770 2024-03-21T10:24:48.993 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:48.991+0000 7f8117e00640 1 -- 172.21.15.47:0/2537949653 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f8118009c30 con 0x558992e76770 2024-03-21T10:24:48.993 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:48.991+0000 7f8117e00640 1 -- 172.21.15.47:0/2537949653 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f811809bd00 con 0x558992e76770 2024-03-21T10:24:48.993 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:48.991+0000 7f8135039740 1 -- 172.21.15.47:0/2537949653 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x558992e76770 msgr2=0x558992e87800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:24:48.993 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:48.991+0000 7f8135039740 1 --2- 172.21.15.47:0/2537949653 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x558992e76770 0x558992e87800 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f81180998b0 tx=0x7f8118097270 comp rx=0 tx=0).stop 2024-03-21T10:24:48.993 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:48.991+0000 7f8135039740 1 -- 172.21.15.47:0/2537949653 shutdown_connections 2024-03-21T10:24:48.993 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:48.991+0000 7f8135039740 1 --2- 172.21.15.47:0/2537949653 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558992e7b8e0 0x558992e7f2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:24:48.993 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:48.991+0000 7f8135039740 1 --2- 172.21.15.47:0/2537949653 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x558992e76770 0x558992e87800 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:24:48.993 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:48.991+0000 7f8135039740 1 --2- 172.21.15.47:0/2537949653 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558992e75d80 0x558992e7edb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:24:48.993 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:48.991+0000 7f8135039740 1 -- 172.21.15.47:0/2537949653 >> 172.21.15.47:0/2537949653 conn(0x558992e712f0 msgr2=0x558992e716d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:24:48.993 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:48.992+0000 7f8135039740 1 -- 172.21.15.47:0/2537949653 shutdown_connections 2024-03-21T10:24:48.993 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:48.992+0000 7f8135039740 1 -- 172.21.15.47:0/2537949653 wait complete. 2024-03-21T10:24:48.996 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:48.992+0000 7f8135039740 1 -- start start 2024-03-21T10:24:48.996 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:48.992+0000 7f8135039740 1 Processor -- start 2024-03-21T10:24:48.996 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:48.993+0000 7f8135039740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558992e75d80 0x558992e875a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:24:48.996 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:48.993+0000 7f8135039740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x558992e76770 0x558992e84040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:24:48.996 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:48.993+0000 7f8135039740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558992e7b8e0 0x558992e84580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:24:48.996 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:48.993+0000 7f8135039740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x558992e8dbf0 con 0x558992e76770 2024-03-21T10:24:48.996 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:48.993+0000 7f8135039740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x558992e8d590 con 0x558992e7b8e0 2024-03-21T10:24:48.996 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:48.993+0000 7f8135039740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x558992e8e250 con 0x558992e75d80 2024-03-21T10:24:48.997 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:48.993+0000 7f8135039740 10 client.0 _ll_register_callbacks cb 0x7f811800d6c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T10:24:48.997 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:48.993+0000 7f812aa00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x558992e76770 0x558992e84040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:24:48.997 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:48.993+0000 7f812aa00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x558992e76770 0x558992e84040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:44416/0 (socket says 172.21.15.47:44416) 2024-03-21T10:24:48.997 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:48.993+0000 7f812aa00640 1 -- 172.21.15.47:0/2009524885 learned_addr learned my addr 172.21.15.47:0/2009524885 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:24:48.997 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:48.993+0000 7f812be00640 1 --2- 172.21.15.47:0/2009524885 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558992e7b8e0 0x558992e84580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:24:48.997 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:48.993+0000 7f812aa00640 1 -- 172.21.15.47:0/2009524885 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558992e75d80 msgr2=0x558992e875a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:24:48.997 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:48.993+0000 7f812b400640 1 --2- 172.21.15.47:0/2009524885 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558992e75d80 0x558992e875a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:24:48.997 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:48.993+0000 7f812aa00640 1 --2- 172.21.15.47:0/2009524885 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558992e75d80 0x558992e875a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:24:48.997 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:48.993+0000 7f812aa00640 1 -- 172.21.15.47:0/2009524885 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558992e7b8e0 msgr2=0x558992e84580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:24:48.997 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:48.993+0000 7f812aa00640 1 --2- 172.21.15.47:0/2009524885 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558992e7b8e0 0x558992e84580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:24:48.997 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:48.993+0000 7f812aa00640 1 -- 172.21.15.47:0/2009524885 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x558992e84b90 con 0x558992e76770 2024-03-21T10:24:48.997 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:48.993+0000 7f812be00640 1 --2- 172.21.15.47:0/2009524885 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558992e7b8e0 0x558992e84580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T10:24:48.997 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:48.994+0000 7f812aa00640 1 --2- 172.21.15.47:0/2009524885 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x558992e76770 0x558992e84040 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f81180ccab0 tx=0x7f81180cf5d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=21dd9afbb05e253c server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:24:48.997 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:48.994+0000 7f8115600640 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-03-21T10:24:48.997 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:48.994+0000 7f8115600640 1 -- 172.21.15.47:0/2009524885 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8118020070 con 0x558992e76770 2024-03-21T10:24:48.998 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:48.994+0000 7f8135039740 1 -- 172.21.15.47:0/2009524885 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x558992de81d0 con 0x558992e76770 2024-03-21T10:24:48.998 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:48.994+0000 7f8115600640 1 -- 172.21.15.47:0/2009524885 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9216+0+0 (secure 0 0 0) 0x7f8118084b90 con 0x558992e76770 2024-03-21T10:24:48.998 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:48.994+0000 7f8115600640 1 -- 172.21.15.47:0/2009524885 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f81180ba140 con 0x558992e76770 2024-03-21T10:24:48.998 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:48.994+0000 7f8135039740 10 client.15414 fetch_fsmap learned FSMap version 29 2024-03-21T10:24:48.998 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:48.994+0000 7f8135039740 1 -- 172.21.15.47:0/2009524885 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x558992ee8b40 con 0x558992e76770 2024-03-21T10:24:48.998 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:48.995+0000 7f8115600640 1 -- 172.21.15.47:0/2009524885 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x558992de81d0 con 0x558992e76770 2024-03-21T10:24:48.998 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:48.995+0000 7f8115600640 1 -- 172.21.15.47:0/2009524885 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 29) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f811809a4c0 con 0x558992e76770 2024-03-21T10:24:48.998 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:48.995+0000 7f8135039740 10 client.15414 fetch_fsmap finished waiting for FSMap version 29 2024-03-21T10:24:48.998 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:48.995+0000 7f8135039740 10 client.15414 resolve_mds: resolved name 'a' to daemon mds.a 2024-03-21T10:24:48.998 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:48.995+0000 7f8135039740 20 client.15414 populate_metadata read hostname 'smithi047' 2024-03-21T10:24:48.998 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:48.995+0000 7f8135039740 1 --2- 172.21.15.47:0/2009524885 >> [v2:172.21.15.47:6840/3401578978,v1:172.21.15.47:6841/3401578978] conn(0x558992eecc10 0x558992ee8dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:24:48.998 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:48.995+0000 7f8135039740 4 client.15414 mds_command: new command op to 14619 tid=0 multi_id=0 [{"prefix": "status"}] 2024-03-21T10:24:48.998 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:48.995+0000 7f8135039740 1 -- 172.21.15.47:0/2009524885 --> [v2:172.21.15.47:6840/3401578978,v1:172.21.15.47:6841/3401578978] -- command(tid 0: {"prefix": "status"}) v1 -- 0x558992de2520 con 0x558992eecc10 2024-03-21T10:24:48.998 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:48.995+0000 7f812b400640 1 --2- 172.21.15.47:0/2009524885 >> [v2:172.21.15.47:6840/3401578978,v1:172.21.15.47:6841/3401578978] conn(0x558992eecc10 0x558992ee8dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:24:48.999 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:48.996+0000 7f8115600640 10 client.15414 ms_handle_connect on v2:172.21.15.47:6840/3401578978 2024-03-21T10:24:49.000 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:48.996+0000 7f812b400640 1 --2- 172.21.15.47:0/2009524885 >> [v2:172.21.15.47:6840/3401578978,v1:172.21.15.47:6841/3401578978] conn(0x558992eecc10 0x558992ee8dd0 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=ad32afdb80a6e6cc server_cookie=6e58326407497656 in_seq=0 out_seq=0 2024-03-21T10:24:49.000 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:48.996+0000 7f8115600640 1 -- 172.21.15.47:0/2009524885 <== mds.4 v2:172.21.15.47:6840/3401578978 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+327 (crc 0 0 0) 0x558992de2520 con 0x558992eecc10 2024-03-21T10:24:49.000 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:48.996+0000 7f8115600640 10 client.15414 handle_command_reply: tid=0 2024-03-21T10:24:49.000 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:48.996+0000 7f8135039740 1 client.15414 shutdown 2024-03-21T10:24:49.000 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:48.996+0000 7f8135039740 1 -- 172.21.15.47:0/2009524885 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x558992e76770 msgr2=0x558992e84040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:24:49.000 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:48.996+0000 7f8135039740 1 --2- 172.21.15.47:0/2009524885 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x558992e76770 0x558992e84040 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f81180ccab0 tx=0x7f81180cf5d0 comp rx=0 tx=0).stop 2024-03-21T10:24:49.000 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:48.996+0000 7f8135039740 1 -- 172.21.15.47:0/2009524885 shutdown_connections 2024-03-21T10:24:49.000 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:48.996+0000 7f8135039740 1 --2- 172.21.15.47:0/2009524885 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558992e7b8e0 0x558992e84580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:24:49.000 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:48.996+0000 7f8135039740 1 --2- 172.21.15.47:0/2009524885 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x558992e76770 0x558992e84040 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:24:49.000 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:48.996+0000 7f8135039740 1 --2- 172.21.15.47:0/2009524885 >> [v2:172.21.15.47:6840/3401578978,v1:172.21.15.47:6841/3401578978] conn(0x558992eecc10 0x558992ee8dd0 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:24:49.000 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:48.996+0000 7f8135039740 1 --2- 172.21.15.47:0/2009524885 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558992e75d80 0x558992e875a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:24:49.000 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:48.996+0000 7f8135039740 1 -- 172.21.15.47:0/2009524885 >> 172.21.15.47:0/2009524885 conn(0x558992e712f0 msgr2=0x558992e74550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:24:49.000 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:48.996+0000 7f8115600640 0 client.15414 ms_handle_reset on v2:172.21.15.47:6840/3401578978 2024-03-21T10:24:49.000 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:48.997+0000 7f8135039740 1 -- 172.21.15.47:0/2009524885 shutdown_connections 2024-03-21T10:24:49.000 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:48.997+0000 7f8135039740 1 -- 172.21.15.47:0/2009524885 wait complete. 2024-03-21T10:24:49.001 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:48.997+0000 7f8135039740 20 client.15414 trim_cache size 0 max 16384 2024-03-21T10:24:49.001 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-03-21T10:24:49.001 INFO:teuthology.orchestra.run.smithi047.stdout: "cluster_fsid": "3056d51a-e76b-11ee-95cd-87774f69a715", 2024-03-21T10:24:49.001 INFO:teuthology.orchestra.run.smithi047.stdout: "whoami": 4, 2024-03-21T10:24:49.001 INFO:teuthology.orchestra.run.smithi047.stdout: "id": 14619, 2024-03-21T10:24:49.001 INFO:teuthology.orchestra.run.smithi047.stdout: "want_state": "up:active", 2024-03-21T10:24:49.001 INFO:teuthology.orchestra.run.smithi047.stdout: "state": "up:active", 2024-03-21T10:24:49.001 INFO:teuthology.orchestra.run.smithi047.stdout: "fs_name": "cephfs", 2024-03-21T10:24:49.001 INFO:teuthology.orchestra.run.smithi047.stdout: "rank_uptime": 140.415584774, 2024-03-21T10:24:49.001 INFO:teuthology.orchestra.run.smithi047.stdout: "mdsmap_epoch": 29, 2024-03-21T10:24:49.001 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap_epoch": 95, 2024-03-21T10:24:49.001 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap_epoch_barrier": 93, 2024-03-21T10:24:49.002 INFO:teuthology.orchestra.run.smithi047.stdout: "uptime": 179.42119508299999 2024-03-21T10:24:49.002 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-03-21T10:24:49.002 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:48.997+0000 7f8130e00640 1 -- 172.21.15.47:0/269807618 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f81100a2f20 msgr2=0x7f81100a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:24:49.002 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:48.997+0000 7f8130e00640 1 --2- 172.21.15.47:0/269807618 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f81100a2f20 0x7f81100a53e0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f811c009230 tx=0x7f811c02c7e0 comp rx=0 tx=0).stop 2024-03-21T10:24:49.002 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:48.997+0000 7f8130e00640 1 -- 172.21.15.47:0/269807618 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f812c100750 msgr2=0x7f812c199f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:24:49.002 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:48.997+0000 7f8130e00640 1 --2- 172.21.15.47:0/269807618 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f812c100750 0x7f812c199f00 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f811800d770 tx=0x7f811800dc30 comp rx=0 tx=0).stop 2024-03-21T10:24:49.003 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:48.998+0000 7f8130e00640 1 -- 172.21.15.47:0/269807618 shutdown_connections 2024-03-21T10:24:49.003 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:48.998+0000 7f8130e00640 1 --2- 172.21.15.47:0/269807618 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f81100a2f20 0x7f81100a53e0 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:24:49.003 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:48.998+0000 7f8130e00640 1 --2- 172.21.15.47:0/269807618 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f812c1010f0 0x7f812c1a0f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:24:49.003 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:48.998+0000 7f8130e00640 1 --2- 172.21.15.47:0/269807618 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f812c100750 0x7f812c199f00 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:24:49.003 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:48.998+0000 7f8130e00640 1 --2- 172.21.15.47:0/269807618 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f812c0ffdf0 0x7f812c1999c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:24:49.003 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:48.998+0000 7f8130e00640 1 -- 172.21.15.47:0/269807618 >> 172.21.15.47:0/269807618 conn(0x7f812c0fba10 msgr2=0x7f812c0fd1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:24:49.003 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:48.998+0000 7f8130e00640 1 -- 172.21.15.47:0/269807618 shutdown_connections 2024-03-21T10:24:49.003 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:48.998+0000 7f8130e00640 1 -- 172.21.15.47:0/269807618 wait complete. 2024-03-21T10:24:49.017 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.k status 2024-03-21T10:24:49.234 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:49.231+0000 7f4ed3e00640 1 -- 172.21.15.47:0/1112511611 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4ed41037c0 msgr2=0x7f4ed4103be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:24:49.234 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:49.231+0000 7f4ed3e00640 1 --2- 172.21.15.47:0/1112511611 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4ed41037c0 0x7f4ed4103be0 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f4ec4009900 tx=0x7f4ec402bc30 comp rx=0 tx=0).stop 2024-03-21T10:24:49.234 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:49.231+0000 7f4ed3e00640 1 -- 172.21.15.47:0/1112511611 shutdown_connections 2024-03-21T10:24:49.234 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:49.231+0000 7f4ed3e00640 1 --2- 172.21.15.47:0/1112511611 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4ed4105370 0x7f4ed410bc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:24:49.234 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:49.231+0000 7f4ed3e00640 1 --2- 172.21.15.47:0/1112511611 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4ed41049b0 0x7f4ed4104e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:24:49.235 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:49.231+0000 7f4ed3e00640 1 --2- 172.21.15.47:0/1112511611 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4ed41037c0 0x7f4ed4103be0 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:24:49.235 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:49.231+0000 7f4ed3e00640 1 -- 172.21.15.47:0/1112511611 >> 172.21.15.47:0/1112511611 conn(0x7f4ed40fed50 msgr2=0x7f4ed4101170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:24:49.235 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:49.232+0000 7f4ed3e00640 1 -- 172.21.15.47:0/1112511611 shutdown_connections 2024-03-21T10:24:49.235 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:49.232+0000 7f4ed3e00640 1 -- 172.21.15.47:0/1112511611 wait complete. 2024-03-21T10:24:49.235 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:49.233+0000 7f4ed3e00640 1 Processor -- start 2024-03-21T10:24:49.235 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:49.233+0000 7f4ed3e00640 1 -- start start 2024-03-21T10:24:49.243 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:49.233+0000 7f4ed3e00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4ed41037c0 0x7f4ed41999f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:24:49.243 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:49.233+0000 7f4ed3e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4ed41049b0 0x7f4ed4199f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:24:49.243 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:49.233+0000 7f4ed3e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4ed4105370 0x7f4ed41a0fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:24:49.243 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:49.233+0000 7f4ed3e00640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ed410f2e0 con 0x7f4ed41049b0 2024-03-21T10:24:49.244 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:49.233+0000 7f4ed3e00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ed410f160 con 0x7f4ed4105370 2024-03-21T10:24:49.244 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:49.233+0000 7f4ed3e00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ed410f460 con 0x7f4ed41037c0 2024-03-21T10:24:49.244 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:49.233+0000 7f4ed2000640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4ed41049b0 0x7f4ed4199f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:24:49.244 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:49.234+0000 7f4ed2000640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4ed41049b0 0x7f4ed4199f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:44436/0 (socket says 172.21.15.47:44436) 2024-03-21T10:24:49.244 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:49.234+0000 7f4ed2000640 1 -- 172.21.15.47:0/2069555612 learned_addr learned my addr 172.21.15.47:0/2069555612 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:24:49.244 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:49.234+0000 7f4ed3400640 1 --2- 172.21.15.47:0/2069555612 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4ed4105370 0x7f4ed41a0fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:24:49.244 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:49.234+0000 7f4ed2a00640 1 --2- 172.21.15.47:0/2069555612 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4ed41037c0 0x7f4ed41999f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:24:49.244 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:49.234+0000 7f4ed2000640 1 -- 172.21.15.47:0/2069555612 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4ed41037c0 msgr2=0x7f4ed41999f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:24:49.244 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:49.234+0000 7f4ed2000640 1 --2- 172.21.15.47:0/2069555612 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4ed41037c0 0x7f4ed41999f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:24:49.244 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:49.234+0000 7f4ed2000640 1 -- 172.21.15.47:0/2069555612 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4ed4105370 msgr2=0x7f4ed41a0fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:24:49.244 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:49.234+0000 7f4ed2000640 1 --2- 172.21.15.47:0/2069555612 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4ed4105370 0x7f4ed41a0fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:24:49.244 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:49.234+0000 7f4ed2000640 1 -- 172.21.15.47:0/2069555612 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ed41a1610 con 0x7f4ed41049b0 2024-03-21T10:24:49.244 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:49.234+0000 7f4ed3400640 1 --2- 172.21.15.47:0/2069555612 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4ed4105370 0x7f4ed41a0fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T10:24:49.244 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:49.235+0000 7f4ed2000640 1 --2- 172.21.15.47:0/2069555612 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4ed41049b0 0x7f4ed4199f30 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f4ec000d770 tx=0x7f4ec000dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:24:49.245 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:49.235+0000 7f4ecb400640 1 -- 172.21.15.47:0/2069555612 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ec0017070 con 0x7f4ed41049b0 2024-03-21T10:24:49.245 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:49.235+0000 7f4ecb400640 1 -- 172.21.15.47:0/2069555612 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f4ec00054d0 con 0x7f4ed41049b0 2024-03-21T10:24:49.245 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:49.235+0000 7f4ed3e00640 1 -- 172.21.15.47:0/2069555612 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4ed41a1900 con 0x7f4ed41049b0 2024-03-21T10:24:49.245 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:49.235+0000 7f4ecb400640 1 -- 172.21.15.47:0/2069555612 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ec0004cf0 con 0x7f4ed41049b0 2024-03-21T10:24:49.245 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:49.235+0000 7f4ed3e00640 1 -- 172.21.15.47:0/2069555612 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4ed41a1e00 con 0x7f4ed41049b0 2024-03-21T10:24:49.245 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:49.238+0000 7f4edc41d740 1 Processor -- start 2024-03-21T10:24:49.245 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:49.238+0000 7f4edc41d740 1 -- start start 2024-03-21T10:24:49.245 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:49.239+0000 7f4edc41d740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556ba40efd80 0x556ba40f01a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:24:49.245 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:49.239+0000 7f4edc41d740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x556ba40f0770 0x556ba40f5270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:24:49.245 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:49.239+0000 7f4edc41d740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556ba40f58e0 0x556ba40f7d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:24:49.245 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:49.239+0000 7f4edc41d740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x556ba4054c90 con 0x556ba40f0770 2024-03-21T10:24:49.245 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:49.239+0000 7f4edc41d740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x556ba3ffd980 con 0x556ba40efd80 2024-03-21T10:24:49.245 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:49.239+0000 7f4edc41d740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x556ba4050d30 con 0x556ba40f58e0 2024-03-21T10:24:49.245 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:49.239+0000 7f4ecb400640 1 -- 172.21.15.47:0/2069555612 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 18) v1 ==== 137854+0+0 (secure 0 0 0) 0x7f4ec0004e90 con 0x7f4ed41049b0 2024-03-21T10:24:49.245 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:49.239+0000 7f4ed2a00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x556ba40f0770 0x556ba40f5270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:24:49.245 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:49.239+0000 7f4ed3400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556ba40efd80 0x556ba40f01a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:24:49.246 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:49.239+0000 7f4ed2a00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x556ba40f0770 0x556ba40f5270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:44442/0 (socket says 172.21.15.47:44442) 2024-03-21T10:24:49.246 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:49.239+0000 7f4ed2a00640 1 -- 172.21.15.47:0/2942980653 learned_addr learned my addr 172.21.15.47:0/2942980653 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:24:49.246 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:49.240+0000 7f4ed2000640 1 --2- 172.21.15.47:0/2942980653 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556ba40f58e0 0x556ba40f7d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:24:49.246 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:49.240+0000 7f4ed2a00640 1 -- 172.21.15.47:0/2942980653 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556ba40f58e0 msgr2=0x556ba40f7d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:24:49.246 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:49.240+0000 7f4ed2a00640 1 --2- 172.21.15.47:0/2942980653 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556ba40f58e0 0x556ba40f7d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:24:49.246 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:49.240+0000 7f4ed2a00640 1 -- 172.21.15.47:0/2942980653 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556ba40efd80 msgr2=0x556ba40f01a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:24:49.246 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:49.240+0000 7f4ed2a00640 1 --2- 172.21.15.47:0/2942980653 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556ba40efd80 0x556ba40f01a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:24:49.246 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:49.240+0000 7f4ed2a00640 1 -- 172.21.15.47:0/2942980653 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556ba4015510 con 0x556ba40f0770 2024-03-21T10:24:49.246 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:49.240+0000 7f4ed3400640 1 --2- 172.21.15.47:0/2942980653 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556ba40efd80 0x556ba40f01a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T10:24:49.246 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:49.240+0000 7f4ed2a00640 1 --2- 172.21.15.47:0/2942980653 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x556ba40f0770 0x556ba40f5270 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f4ec4005ab0 tx=0x7f4ec40095e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f4a8ad4e89751149 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:24:49.246 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:49.241+0000 7f4ebf400640 1 -- 172.21.15.47:0/2942980653 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ec4044020 con 0x556ba40f0770 2024-03-21T10:24:49.246 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:49.241+0000 7f4ebf400640 1 -- 172.21.15.47:0/2942980653 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f4ec402cba0 con 0x556ba40f0770 2024-03-21T10:24:49.246 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:49.241+0000 7f4ebf400640 1 -- 172.21.15.47:0/2942980653 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ec40046c0 con 0x556ba40f0770 2024-03-21T10:24:49.247 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:49.241+0000 7f4ecb400640 1 --2- 172.21.15.47:0/2069555612 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f4eb80a2f20 0x7f4eb80a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:24:49.247 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:49.241+0000 7f4ecb400640 1 -- 172.21.15.47:0/2069555612 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9216+0+0 (secure 0 0 0) 0x7f4ec00cf2b0 con 0x7f4ed41049b0 2024-03-21T10:24:49.247 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:49.241+0000 7f4ed3400640 1 --2- 172.21.15.47:0/2069555612 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f4eb80a2f20 0x7f4eb80a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:24:49.247 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:49.242+0000 7f4edc41d740 1 -- 172.21.15.47:0/2942980653 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x556ba40f0770 msgr2=0x556ba40f5270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:24:49.247 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:49.242+0000 7f4edc41d740 1 --2- 172.21.15.47:0/2942980653 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x556ba40f0770 0x556ba40f5270 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f4ec4005ab0 tx=0x7f4ec40095e0 comp rx=0 tx=0).stop 2024-03-21T10:24:49.247 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:49.242+0000 7f4ed3400640 1 --2- 172.21.15.47:0/2069555612 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f4eb80a2f20 0x7f4eb80a53e0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x556ba40f0fd0 tx=0x7f4ecc009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:24:49.247 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:49.242+0000 7f4edc41d740 1 -- 172.21.15.47:0/2942980653 shutdown_connections 2024-03-21T10:24:49.247 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:49.242+0000 7f4edc41d740 1 --2- 172.21.15.47:0/2942980653 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556ba40f58e0 0x556ba40f7d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:24:49.247 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:49.242+0000 7f4edc41d740 1 --2- 172.21.15.47:0/2942980653 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x556ba40f0770 0x556ba40f5270 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:24:49.247 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:49.242+0000 7f4edc41d740 1 --2- 172.21.15.47:0/2942980653 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556ba40efd80 0x556ba40f01a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:24:49.247 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:49.242+0000 7f4edc41d740 1 -- 172.21.15.47:0/2942980653 >> 172.21.15.47:0/2942980653 conn(0x556ba40eb2f0 msgr2=0x556ba40eb6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:24:49.247 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:49.242+0000 7f4edc41d740 1 -- 172.21.15.47:0/2942980653 shutdown_connections 2024-03-21T10:24:49.247 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:49.242+0000 7f4edc41d740 1 -- 172.21.15.47:0/2942980653 wait complete. 2024-03-21T10:24:49.247 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:49.242+0000 7f4edc41d740 1 -- start start 2024-03-21T10:24:49.247 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:49.243+0000 7f4edc41d740 1 Processor -- start 2024-03-21T10:24:49.251 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:49.243+0000 7f4edc41d740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x556ba40efd80 0x556ba40fa200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:24:49.251 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:49.243+0000 7f4edc41d740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556ba40f0770 0x556ba40fa740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:24:49.251 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:49.243+0000 7f4edc41d740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556ba40f58e0 0x556ba4101780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:24:49.251 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:49.243+0000 7f4edc41d740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x556ba40f8830 con 0x556ba40efd80 2024-03-21T10:24:49.251 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:49.243+0000 7f4edc41d740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x556ba4010dd0 con 0x556ba40f58e0 2024-03-21T10:24:49.251 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:49.243+0000 7f4edc41d740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x556ba40f89b0 con 0x556ba40f0770 2024-03-21T10:24:49.251 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:49.243+0000 7f4edc41d740 10 client.0 _ll_register_callbacks cb 0x556ba3d68ac0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T10:24:49.251 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:49.243+0000 7f4ed2a00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x556ba40efd80 0x556ba40fa200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:24:49.251 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:49.243+0000 7f4ed2a00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x556ba40efd80 0x556ba40fa200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:44458/0 (socket says 172.21.15.47:44458) 2024-03-21T10:24:49.251 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:49.243+0000 7f4ed2a00640 1 -- 172.21.15.47:0/3278995247 learned_addr learned my addr 172.21.15.47:0/3278995247 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:24:49.251 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:49.244+0000 7f4ed3400640 1 --2- 172.21.15.47:0/3278995247 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556ba40f58e0 0x556ba4101780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:24:49.251 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:49.244+0000 7f4ed2a00640 1 -- 172.21.15.47:0/3278995247 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556ba40f0770 msgr2=0x556ba40fa740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:24:49.252 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:49.244+0000 7f4ed2000640 1 --2- 172.21.15.47:0/3278995247 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556ba40f0770 0x556ba40fa740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:24:49.252 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:49.244+0000 7f4ed2a00640 1 --2- 172.21.15.47:0/3278995247 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556ba40f0770 0x556ba40fa740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:24:49.252 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:49.244+0000 7f4ed2a00640 1 -- 172.21.15.47:0/3278995247 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556ba40f58e0 msgr2=0x556ba4101780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:24:49.252 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:49.244+0000 7f4ed2a00640 1 --2- 172.21.15.47:0/3278995247 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556ba40f58e0 0x556ba4101780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:24:49.252 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:49.244+0000 7f4ed2a00640 1 -- 172.21.15.47:0/3278995247 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x556ba3d692d0 con 0x556ba40efd80 2024-03-21T10:24:49.252 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:49.244+0000 7f4ed3400640 1 --2- 172.21.15.47:0/3278995247 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556ba40f58e0 0x556ba4101780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T10:24:49.252 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:49.244+0000 7f4ed2a00640 1 --2- 172.21.15.47:0/3278995247 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x556ba40efd80 0x556ba40fa200 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f4ec4000c00 tx=0x7f4ec402ca20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7ceaf19cd58ef6c3 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:24:49.252 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:49.244+0000 7f4eb3e00640 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-03-21T10:24:49.252 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:49.245+0000 7f4eb3e00640 1 -- 172.21.15.47:0/3278995247 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ec404f030 con 0x556ba40efd80 2024-03-21T10:24:49.252 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:49.245+0000 7f4eb3e00640 1 -- 172.21.15.47:0/3278995247 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9216+0+0 (secure 0 0 0) 0x7f4ec40035f0 con 0x556ba40efd80 2024-03-21T10:24:49.252 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:49.245+0000 7f4edc41d740 1 -- 172.21.15.47:0/3278995247 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f4ec402cba0 con 0x556ba40efd80 2024-03-21T10:24:49.252 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:49.245+0000 7f4eb3e00640 1 -- 172.21.15.47:0/3278995247 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4ec4059050 con 0x556ba40efd80 2024-03-21T10:24:49.252 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:49.245+0000 7f4edc41d740 10 client.15426 fetch_fsmap learned FSMap version 29 2024-03-21T10:24:49.252 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:49.245+0000 7f4edc41d740 1 -- 172.21.15.47:0/3278995247 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x556ba3e336b0 con 0x556ba40efd80 2024-03-21T10:24:49.252 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:49.245+0000 7f4eb3e00640 1 -- 172.21.15.47:0/3278995247 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f4ec402cba0 con 0x556ba40efd80 2024-03-21T10:24:49.253 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:49.245+0000 7f4eb3e00640 1 -- 172.21.15.47:0/3278995247 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 29) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f4ec4040a80 con 0x556ba40efd80 2024-03-21T10:24:49.253 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:49.246+0000 7f4edc41d740 10 client.15426 fetch_fsmap finished waiting for FSMap version 29 2024-03-21T10:24:49.253 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:49.246+0000 7f4edc41d740 10 client.15426 resolve_mds: resolved name 'k' to daemon mds.k 2024-03-21T10:24:49.253 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:49.246+0000 7f4edc41d740 20 client.15426 populate_metadata read hostname 'smithi047' 2024-03-21T10:24:49.253 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:49.246+0000 7f4edc41d740 1 --2- 172.21.15.47:0/3278995247 >> [v2:172.21.15.100:6832/2723302038,v1:172.21.15.100:6833/2723302038] conn(0x556ba41058e0 0x556ba410a990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:24:49.253 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:49.246+0000 7f4edc41d740 4 client.15426 mds_command: new command op to 34427 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T10:24:49.253 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:49.246+0000 7f4edc41d740 1 -- 172.21.15.47:0/3278995247 --> [v2:172.21.15.100:6832/2723302038,v1:172.21.15.100:6833/2723302038] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x556ba3ff6520 con 0x556ba41058e0 2024-03-21T10:24:49.253 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:49.246+0000 7f4ed2000640 1 --2- 172.21.15.47:0/3278995247 >> [v2:172.21.15.100:6832/2723302038,v1:172.21.15.100:6833/2723302038] conn(0x556ba41058e0 0x556ba410a990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:24:49.253 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:49.247+0000 7f4ed2000640 1 --2- 172.21.15.47:0/3278995247 >> [v2:172.21.15.100:6832/2723302038,v1:172.21.15.100:6833/2723302038] conn(0x556ba41058e0 0x556ba410a990 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=588c2bf1d302eb61 server_cookie=d276f7bcb72cbc02 in_seq=0 out_seq=0 2024-03-21T10:24:49.253 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:49.247+0000 7f4eb3e00640 10 client.15426 ms_handle_connect on v2:172.21.15.100:6832/2723302038 2024-03-21T10:24:49.253 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:49.248+0000 7f4eb3e00640 1 -- 172.21.15.47:0/3278995247 <== mds.3 v2:172.21.15.100:6832/2723302038 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x556ba3ff6520 con 0x556ba41058e0 2024-03-21T10:24:49.253 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:49.248+0000 7f4eb3e00640 10 client.15426 handle_command_reply: tid=0 2024-03-21T10:24:49.253 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:49.249+0000 7f4edc41d740 1 client.15426 shutdown 2024-03-21T10:24:49.253 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:49.249+0000 7f4edc41d740 1 -- 172.21.15.47:0/3278995247 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x556ba40efd80 msgr2=0x556ba40fa200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:24:49.253 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:49.249+0000 7f4edc41d740 1 --2- 172.21.15.47:0/3278995247 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x556ba40efd80 0x556ba40fa200 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f4ec4000c00 tx=0x7f4ec402ca20 comp rx=0 tx=0).stop 2024-03-21T10:24:49.254 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:49.249+0000 7f4edc41d740 1 -- 172.21.15.47:0/3278995247 shutdown_connections 2024-03-21T10:24:49.254 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:49.249+0000 7f4edc41d740 1 --2- 172.21.15.47:0/3278995247 >> [v2:172.21.15.100:6832/2723302038,v1:172.21.15.100:6833/2723302038] conn(0x556ba41058e0 0x556ba410a990 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:24:49.254 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:49.249+0000 7f4edc41d740 1 --2- 172.21.15.47:0/3278995247 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556ba40f58e0 0x556ba4101780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:24:49.254 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:49.249+0000 7f4edc41d740 1 --2- 172.21.15.47:0/3278995247 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556ba40f0770 0x556ba40fa740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:24:49.254 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:49.249+0000 7f4edc41d740 1 --2- 172.21.15.47:0/3278995247 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x556ba40efd80 0x556ba40fa200 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:24:49.254 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:49.249+0000 7f4eb3e00640 0 client.15426 ms_handle_reset on v2:172.21.15.100:6832/2723302038 2024-03-21T10:24:49.254 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:49.249+0000 7f4edc41d740 1 -- 172.21.15.47:0/3278995247 >> 172.21.15.47:0/3278995247 conn(0x556ba40eb2f0 msgr2=0x556ba40ea4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:24:49.254 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:49.249+0000 7f4edc41d740 1 -- 172.21.15.47:0/3278995247 shutdown_connections 2024-03-21T10:24:49.254 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:49.249+0000 7f4edc41d740 1 -- 172.21.15.47:0/3278995247 wait complete. 2024-03-21T10:24:49.254 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:49.249+0000 7f4edc41d740 20 client.15426 trim_cache size 0 max 16384 2024-03-21T10:24:49.269 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:49.268+0000 7f4edc41d740 1 Processor -- start 2024-03-21T10:24:49.269 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:49.268+0000 7f4edc41d740 1 -- start start 2024-03-21T10:24:49.269 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:49.269+0000 7f4edc41d740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556ba40efd80 0x556ba40eb6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:24:49.270 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:49.269+0000 7f4edc41d740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556ba40f0770 0x556ba4101740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:24:49.270 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:49.269+0000 7f4edc41d740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x556ba40f58e0 0x556ba40e9c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:24:49.270 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:49.269+0000 7f4edc41d740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x556ba4105d30 con 0x556ba40f58e0 2024-03-21T10:24:49.270 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:49.269+0000 7f4edc41d740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x556ba4106390 con 0x556ba40f0770 2024-03-21T10:24:49.270 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:49.269+0000 7f4edc41d740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x556ba41069f0 con 0x556ba40efd80 2024-03-21T10:24:49.271 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:49.269+0000 7f4ed3400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x556ba40f58e0 0x556ba40e9c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:24:49.271 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:49.269+0000 7f4ed3400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x556ba40f58e0 0x556ba40e9c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:44472/0 (socket says 172.21.15.47:44472) 2024-03-21T10:24:49.271 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:49.269+0000 7f4ed3400640 1 -- 172.21.15.47:0/4285802704 learned_addr learned my addr 172.21.15.47:0/4285802704 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:24:49.271 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:49.269+0000 7f4ed2000640 1 --2- 172.21.15.47:0/4285802704 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556ba40f0770 0x556ba4101740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:24:49.271 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:49.269+0000 7f4ed2a00640 1 --2- 172.21.15.47:0/4285802704 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556ba40efd80 0x556ba40eb6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:24:49.271 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:49.270+0000 7f4ed3400640 1 -- 172.21.15.47:0/4285802704 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556ba40efd80 msgr2=0x556ba40eb6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:24:49.271 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:49.270+0000 7f4ed3400640 1 --2- 172.21.15.47:0/4285802704 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556ba40efd80 0x556ba40eb6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:24:49.271 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:49.270+0000 7f4ed3400640 1 -- 172.21.15.47:0/4285802704 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556ba40f0770 msgr2=0x556ba4101740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:24:49.271 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:49.270+0000 7f4ed3400640 1 --2- 172.21.15.47:0/4285802704 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556ba40f0770 0x556ba4101740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:24:49.271 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:49.270+0000 7f4ed3400640 1 -- 172.21.15.47:0/4285802704 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556ba40dd170 con 0x556ba40f58e0 2024-03-21T10:24:49.271 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:49.270+0000 7f4ed3400640 1 --2- 172.21.15.47:0/4285802704 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x556ba40f58e0 0x556ba40e9c60 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f4ecc010830 tx=0x7f4ecc016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ba91b87e7f8adca1 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:24:49.272 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:49.270+0000 7f4ebea00640 1 -- 172.21.15.47:0/4285802704 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ecc010e70 con 0x556ba40f58e0 2024-03-21T10:24:49.272 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:49.271+0000 7f4ebea00640 1 -- 172.21.15.47:0/4285802704 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f4ecc01edd0 con 0x556ba40f58e0 2024-03-21T10:24:49.272 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:49.271+0000 7f4ebea00640 1 -- 172.21.15.47:0/4285802704 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ecc01c890 con 0x556ba40f58e0 2024-03-21T10:24:49.272 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:49.271+0000 7f4edc41d740 1 -- 172.21.15.47:0/4285802704 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x556ba40f58e0 msgr2=0x556ba40e9c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:24:49.273 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:49.271+0000 7f4edc41d740 1 --2- 172.21.15.47:0/4285802704 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x556ba40f58e0 0x556ba40e9c60 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f4ecc010830 tx=0x7f4ecc016000 comp rx=0 tx=0).stop 2024-03-21T10:24:49.273 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:49.271+0000 7f4edc41d740 1 -- 172.21.15.47:0/4285802704 shutdown_connections 2024-03-21T10:24:49.273 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:49.271+0000 7f4edc41d740 1 --2- 172.21.15.47:0/4285802704 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x556ba40f58e0 0x556ba40e9c60 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:24:49.273 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:49.271+0000 7f4edc41d740 1 --2- 172.21.15.47:0/4285802704 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556ba40f0770 0x556ba4101740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:24:49.273 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:49.271+0000 7f4edc41d740 1 --2- 172.21.15.47:0/4285802704 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556ba40efd80 0x556ba40eb6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:24:49.273 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:49.271+0000 7f4edc41d740 1 -- 172.21.15.47:0/4285802704 >> 172.21.15.47:0/4285802704 conn(0x556ba40eb2f0 msgr2=0x556ba40f8d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:24:49.273 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:49.272+0000 7f4edc41d740 1 -- 172.21.15.47:0/4285802704 shutdown_connections 2024-03-21T10:24:49.273 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:49.272+0000 7f4edc41d740 1 -- 172.21.15.47:0/4285802704 wait complete. 2024-03-21T10:24:49.274 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:49.272+0000 7f4edc41d740 1 -- start start 2024-03-21T10:24:49.274 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:49.272+0000 7f4edc41d740 1 Processor -- start 2024-03-21T10:24:49.274 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:49.273+0000 7f4edc41d740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x556ba40efd80 0x556ba40fd470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:24:49.274 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:49.273+0000 7f4edc41d740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556ba40f0770 0x556ba40fd9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:24:49.274 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:49.273+0000 7f4edc41d740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556ba40f58e0 0x556ba41009d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:24:49.274 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:49.273+0000 7f4edc41d740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x556ba41076b0 con 0x556ba40efd80 2024-03-21T10:24:49.275 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:49.273+0000 7f4edc41d740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x556ba4107050 con 0x556ba40f0770 2024-03-21T10:24:49.275 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:49.273+0000 7f4edc41d740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x556ba4107d10 con 0x556ba40f58e0 2024-03-21T10:24:49.275 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:49.273+0000 7f4edc41d740 10 client.0 _ll_register_callbacks cb 0x7f4ecc0099e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T10:24:49.275 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:49.273+0000 7f4ed2a00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x556ba40efd80 0x556ba40fd470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:24:49.275 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:49.273+0000 7f4ed2000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556ba40f0770 0x556ba40fd9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:24:49.275 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:49.273+0000 7f4ed2a00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x556ba40efd80 0x556ba40fd470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:44482/0 (socket says 172.21.15.47:44482) 2024-03-21T10:24:49.275 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:49.273+0000 7f4ed2a00640 1 -- 172.21.15.47:0/3765502317 learned_addr learned my addr 172.21.15.47:0/3765502317 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:24:49.275 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:49.273+0000 7f4ed2a00640 1 -- 172.21.15.47:0/3765502317 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556ba40f58e0 msgr2=0x556ba41009d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:24:49.275 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:49.273+0000 7f4ed2a00640 1 --2- 172.21.15.47:0/3765502317 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556ba40f58e0 0x556ba41009d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:24:49.275 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:49.273+0000 7f4ed2a00640 1 -- 172.21.15.47:0/3765502317 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556ba40f0770 msgr2=0x556ba40fd9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:24:49.275 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:49.273+0000 7f4ed2a00640 1 --2- 172.21.15.47:0/3765502317 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556ba40f0770 0x556ba40fd9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:24:49.275 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:49.273+0000 7f4ed2a00640 1 -- 172.21.15.47:0/3765502317 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x556ba4100fc0 con 0x556ba40efd80 2024-03-21T10:24:49.277 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:49.274+0000 7f4ed2a00640 1 --2- 172.21.15.47:0/3765502317 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x556ba40efd80 0x556ba40fd470 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f4ec4005ec0 tx=0x7f4ec402cdb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2065acfce4e9db04 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:24:49.277 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:49.274+0000 7f4eb3e00640 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-03-21T10:24:49.277 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:49.274+0000 7f4eb3e00640 1 -- 172.21.15.47:0/3765502317 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ec404f070 con 0x556ba40efd80 2024-03-21T10:24:49.277 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:49.274+0000 7f4eb3e00640 1 -- 172.21.15.47:0/3765502317 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9216+0+0 (secure 0 0 0) 0x7f4ec404ba10 con 0x556ba40efd80 2024-03-21T10:24:49.277 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:49.274+0000 7f4edc41d740 1 -- 172.21.15.47:0/3765502317 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f4ecc01edd0 con 0x556ba40efd80 2024-03-21T10:24:49.277 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:49.274+0000 7f4eb3e00640 1 -- 172.21.15.47:0/3765502317 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4ec4064050 con 0x556ba40efd80 2024-03-21T10:24:49.277 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:49.274+0000 7f4edc41d740 10 client.15432 fetch_fsmap learned FSMap version 29 2024-03-21T10:24:49.277 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:49.274+0000 7f4edc41d740 1 -- 172.21.15.47:0/3765502317 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x556ba4162160 con 0x556ba40efd80 2024-03-21T10:24:49.277 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:49.274+0000 7f4eb3e00640 1 -- 172.21.15.47:0/3765502317 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f4ecc01edd0 con 0x556ba40efd80 2024-03-21T10:24:49.277 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:49.275+0000 7f4eb3e00640 1 -- 172.21.15.47:0/3765502317 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 29) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f4ec40035f0 con 0x556ba40efd80 2024-03-21T10:24:49.277 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:49.275+0000 7f4edc41d740 10 client.15432 fetch_fsmap finished waiting for FSMap version 29 2024-03-21T10:24:49.277 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:49.275+0000 7f4edc41d740 10 client.15432 resolve_mds: resolved name 'k' to daemon mds.k 2024-03-21T10:24:49.277 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:49.275+0000 7f4edc41d740 20 client.15432 populate_metadata read hostname 'smithi047' 2024-03-21T10:24:49.277 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:49.275+0000 7f4edc41d740 1 --2- 172.21.15.47:0/3765502317 >> [v2:172.21.15.100:6832/2723302038,v1:172.21.15.100:6833/2723302038] conn(0x556ba4166810 0x556ba41623e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:24:49.277 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:49.275+0000 7f4edc41d740 4 client.15432 mds_command: new command op to 34427 tid=0 multi_id=0 [{"prefix": "status"}] 2024-03-21T10:24:49.278 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:49.275+0000 7f4edc41d740 1 -- 172.21.15.47:0/3765502317 --> [v2:172.21.15.100:6832/2723302038,v1:172.21.15.100:6833/2723302038] -- command(tid 0: {"prefix": "status"}) v1 -- 0x556ba40621d0 con 0x556ba4166810 2024-03-21T10:24:49.278 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:49.275+0000 7f4ed2000640 1 --2- 172.21.15.47:0/3765502317 >> [v2:172.21.15.100:6832/2723302038,v1:172.21.15.100:6833/2723302038] conn(0x556ba4166810 0x556ba41623e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:24:49.279 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:49.276+0000 7f4ed2000640 1 --2- 172.21.15.47:0/3765502317 >> [v2:172.21.15.100:6832/2723302038,v1:172.21.15.100:6833/2723302038] conn(0x556ba4166810 0x556ba41623e0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=5583b52f1dcacae6 server_cookie=d0039788f7984e76 in_seq=0 out_seq=0 2024-03-21T10:24:49.279 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:49.276+0000 7f4eb3e00640 10 client.15432 ms_handle_connect on v2:172.21.15.100:6832/2723302038 2024-03-21T10:24:49.279 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:49.277+0000 7f4eb3e00640 1 -- 172.21.15.47:0/3765502317 <== mds.3 v2:172.21.15.100:6832/2723302038 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+332 (crc 0 0 0) 0x556ba40621d0 con 0x556ba4166810 2024-03-21T10:24:49.279 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:49.277+0000 7f4eb3e00640 10 client.15432 handle_command_reply: tid=0 2024-03-21T10:24:49.279 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:49.277+0000 7f4edc41d740 1 client.15432 shutdown 2024-03-21T10:24:49.279 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:49.277+0000 7f4edc41d740 1 -- 172.21.15.47:0/3765502317 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x556ba40efd80 msgr2=0x556ba40fd470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:24:49.279 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:49.277+0000 7f4edc41d740 1 --2- 172.21.15.47:0/3765502317 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x556ba40efd80 0x556ba40fd470 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f4ec4005ec0 tx=0x7f4ec402cdb0 comp rx=0 tx=0).stop 2024-03-21T10:24:49.279 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:49.277+0000 7f4edc41d740 1 -- 172.21.15.47:0/3765502317 shutdown_connections 2024-03-21T10:24:49.279 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:49.277+0000 7f4edc41d740 1 --2- 172.21.15.47:0/3765502317 >> [v2:172.21.15.100:6832/2723302038,v1:172.21.15.100:6833/2723302038] conn(0x556ba4166810 0x556ba41623e0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:24:49.279 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:49.277+0000 7f4edc41d740 1 --2- 172.21.15.47:0/3765502317 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556ba40f58e0 0x556ba41009d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:24:49.279 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:49.277+0000 7f4edc41d740 1 --2- 172.21.15.47:0/3765502317 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556ba40f0770 0x556ba40fd9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:24:49.279 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:49.277+0000 7f4edc41d740 1 --2- 172.21.15.47:0/3765502317 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x556ba40efd80 0x556ba40fd470 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:24:49.280 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:49.277+0000 7f4eb3e00640 0 client.15432 ms_handle_reset on v2:172.21.15.100:6832/2723302038 2024-03-21T10:24:49.280 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:49.277+0000 7f4edc41d740 1 -- 172.21.15.47:0/3765502317 >> 172.21.15.47:0/3765502317 conn(0x556ba40eb2f0 msgr2=0x556ba40eb6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:24:49.280 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:49.277+0000 7f4edc41d740 1 -- 172.21.15.47:0/3765502317 shutdown_connections 2024-03-21T10:24:49.280 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:49.277+0000 7f4edc41d740 1 -- 172.21.15.47:0/3765502317 wait complete. 2024-03-21T10:24:49.280 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:49.277+0000 7f4edc41d740 20 client.15432 trim_cache size 0 max 16384 2024-03-21T10:24:49.280 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-03-21T10:24:49.280 INFO:teuthology.orchestra.run.smithi047.stdout: "cluster_fsid": "3056d51a-e76b-11ee-95cd-87774f69a715", 2024-03-21T10:24:49.280 INFO:teuthology.orchestra.run.smithi047.stdout: "whoami": 3, 2024-03-21T10:24:49.280 INFO:teuthology.orchestra.run.smithi047.stdout: "id": 34427, 2024-03-21T10:24:49.280 INFO:teuthology.orchestra.run.smithi047.stdout: "want_state": "up:active", 2024-03-21T10:24:49.280 INFO:teuthology.orchestra.run.smithi047.stdout: "state": "up:active", 2024-03-21T10:24:49.280 INFO:teuthology.orchestra.run.smithi047.stdout: "fs_name": "cephfs", 2024-03-21T10:24:49.281 INFO:teuthology.orchestra.run.smithi047.stdout: "rank_uptime": 141.05507982500001, 2024-03-21T10:24:49.281 INFO:teuthology.orchestra.run.smithi047.stdout: "mdsmap_epoch": 29, 2024-03-21T10:24:49.281 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap_epoch": 95, 2024-03-21T10:24:49.281 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap_epoch_barrier": 93, 2024-03-21T10:24:49.281 INFO:teuthology.orchestra.run.smithi047.stdout: "uptime": 215.98347414200001 2024-03-21T10:24:49.281 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-03-21T10:24:49.281 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:49.278+0000 7f4ed3e00640 1 -- 172.21.15.47:0/2069555612 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f4eb80a2f20 msgr2=0x7f4eb80a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:24:49.282 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:49.278+0000 7f4ed3e00640 1 --2- 172.21.15.47:0/2069555612 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f4eb80a2f20 0x7f4eb80a53e0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x556ba40f0fd0 tx=0x7f4ecc009500 comp rx=0 tx=0).stop 2024-03-21T10:24:49.282 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:49.278+0000 7f4ed3e00640 1 -- 172.21.15.47:0/2069555612 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4ed41049b0 msgr2=0x7f4ed4199f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:24:49.282 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:49.278+0000 7f4ed3e00640 1 --2- 172.21.15.47:0/2069555612 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4ed41049b0 0x7f4ed4199f30 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f4ec000d770 tx=0x7f4ec000dc30 comp rx=0 tx=0).stop 2024-03-21T10:24:49.282 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:49.278+0000 7f4ed3e00640 1 -- 172.21.15.47:0/2069555612 shutdown_connections 2024-03-21T10:24:49.282 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:49.278+0000 7f4ed3e00640 1 --2- 172.21.15.47:0/2069555612 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f4eb80a2f20 0x7f4eb80a53e0 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:24:49.282 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:49.278+0000 7f4ed3e00640 1 --2- 172.21.15.47:0/2069555612 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4ed4105370 0x7f4ed41a0fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:24:49.282 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:49.278+0000 7f4ed3e00640 1 --2- 172.21.15.47:0/2069555612 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4ed41049b0 0x7f4ed4199f30 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:24:49.282 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:49.278+0000 7f4ed3e00640 1 --2- 172.21.15.47:0/2069555612 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4ed41037c0 0x7f4ed41999f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:24:49.282 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:49.278+0000 7f4ed3e00640 1 -- 172.21.15.47:0/2069555612 >> 172.21.15.47:0/2069555612 conn(0x7f4ed40fed50 msgr2=0x7f4ed4107da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:24:49.282 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:49.278+0000 7f4ed3e00640 1 -- 172.21.15.47:0/2069555612 shutdown_connections 2024-03-21T10:24:49.282 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:49.278+0000 7f4ed3e00640 1 -- 172.21.15.47:0/2069555612 wait complete. 2024-03-21T10:24:49.299 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.e status 2024-03-21T10:24:49.513 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:49.511+0000 7f6de6000640 1 -- 172.21.15.47:0/1219807733 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6de0102790 msgr2=0x7f6de0102c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:24:49.513 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:49.511+0000 7f6de6000640 1 --2- 172.21.15.47:0/1219807733 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6de0102790 0x7f6de0102c10 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f6dcc009900 tx=0x7f6dcc02bc30 comp rx=0 tx=0).stop 2024-03-21T10:24:49.513 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:49.511+0000 7f6de6000640 1 -- 172.21.15.47:0/1219807733 shutdown_connections 2024-03-21T10:24:49.513 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:49.511+0000 7f6de6000640 1 --2- 172.21.15.47:0/1219807733 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6de0103150 0x7f6de01099e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:24:49.513 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:49.511+0000 7f6de6000640 1 --2- 172.21.15.47:0/1219807733 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6de0102790 0x7f6de0102c10 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:24:49.513 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:49.511+0000 7f6de6000640 1 --2- 172.21.15.47:0/1219807733 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6de01015a0 0x7f6de01019c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:24:49.513 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:49.511+0000 7f6de6000640 1 -- 172.21.15.47:0/1219807733 >> 172.21.15.47:0/1219807733 conn(0x7f6de00fcb10 msgr2=0x7f6de00fef50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:24:49.513 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:49.511+0000 7f6de6000640 1 -- 172.21.15.47:0/1219807733 shutdown_connections 2024-03-21T10:24:49.513 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:49.511+0000 7f6de6000640 1 -- 172.21.15.47:0/1219807733 wait complete. 2024-03-21T10:24:49.515 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:49.512+0000 7f6de6000640 1 Processor -- start 2024-03-21T10:24:49.516 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:49.513+0000 7f6de6000640 1 -- start start 2024-03-21T10:24:49.516 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:49.513+0000 7f6de6000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6de01015a0 0x7f6de01977c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:24:49.516 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:49.513+0000 7f6de6000640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6de0102790 0x7f6de0197d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:24:49.516 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:49.513+0000 7f6de6000640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6de0103150 0x7f6de019ed70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:24:49.516 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:49.513+0000 7f6de6000640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6de010d0e0 con 0x7f6de0103150 2024-03-21T10:24:49.516 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:49.513+0000 7f6de6000640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6de010cf60 con 0x7f6de01015a0 2024-03-21T10:24:49.516 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:49.513+0000 7f6de6000640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6de010d260 con 0x7f6de0102790 2024-03-21T10:24:49.516 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:49.513+0000 7f6de5600640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6de0103150 0x7f6de019ed70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:24:49.516 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:49.513+0000 7f6de5600640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6de0103150 0x7f6de019ed70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:44490/0 (socket says 172.21.15.47:44490) 2024-03-21T10:24:49.516 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:49.513+0000 7f6de5600640 1 -- 172.21.15.47:0/2909926818 learned_addr learned my addr 172.21.15.47:0/2909926818 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:24:49.516 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:49.514+0000 7f6ddfe00640 1 --2- 172.21.15.47:0/2909926818 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6de0102790 0x7f6de0197d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:24:49.516 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:49.514+0000 7f6de4c00640 1 --2- 172.21.15.47:0/2909926818 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6de01015a0 0x7f6de01977c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:24:49.516 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:49.514+0000 7f6de5600640 1 -- 172.21.15.47:0/2909926818 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6de0102790 msgr2=0x7f6de0197d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:24:49.517 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:49.514+0000 7f6de5600640 1 --2- 172.21.15.47:0/2909926818 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6de0102790 0x7f6de0197d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:24:49.517 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:49.514+0000 7f6de5600640 1 -- 172.21.15.47:0/2909926818 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6de01015a0 msgr2=0x7f6de01977c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:24:49.517 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:49.514+0000 7f6de5600640 1 --2- 172.21.15.47:0/2909926818 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6de01015a0 0x7f6de01977c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:24:49.517 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:49.514+0000 7f6de5600640 1 -- 172.21.15.47:0/2909926818 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6de019f3e0 con 0x7f6de0103150 2024-03-21T10:24:49.517 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:49.514+0000 7f6de4c00640 1 --2- 172.21.15.47:0/2909926818 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6de01015a0 0x7f6de01977c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T10:24:49.517 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:49.514+0000 7f6ddfe00640 1 --2- 172.21.15.47:0/2909926818 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6de0102790 0x7f6de0197d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T10:24:49.517 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:49.514+0000 7f6de5600640 1 --2- 172.21.15.47:0/2909926818 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6de0103150 0x7f6de019ed70 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f6dd800eb10 tx=0x7f6dd800c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:24:49.517 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:49.514+0000 7f6ddd600640 1 -- 172.21.15.47:0/2909926818 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6dd8004b20 con 0x7f6de0103150 2024-03-21T10:24:49.517 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:49.515+0000 7f6de6000640 1 -- 172.21.15.47:0/2909926818 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6de019f6d0 con 0x7f6de0103150 2024-03-21T10:24:49.517 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:49.515+0000 7f6ddd600640 1 -- 172.21.15.47:0/2909926818 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f6dd8004cc0 con 0x7f6de0103150 2024-03-21T10:24:49.517 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:49.515+0000 7f6ddd600640 1 -- 172.21.15.47:0/2909926818 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6dd8018630 con 0x7f6de0103150 2024-03-21T10:24:49.517 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:49.515+0000 7f6de6000640 1 -- 172.21.15.47:0/2909926818 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6de019fbd0 con 0x7f6de0103150 2024-03-21T10:24:49.519 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:49.518+0000 7f6dea20e740 1 Processor -- start 2024-03-21T10:24:49.519 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:49.518+0000 7f6ddd600640 1 -- 172.21.15.47:0/2909926818 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 18) v1 ==== 137854+0+0 (secure 0 0 0) 0x7f6dd8005000 con 0x7f6de0103150 2024-03-21T10:24:49.519 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:49.518+0000 7f6dea20e740 1 -- start start 2024-03-21T10:24:49.521 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:49.518+0000 7f6dea20e740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ba7f4b5d80 0x55ba7f4b61a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:24:49.521 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:49.518+0000 7f6dea20e740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ba7f4b6770 0x55ba7f4bb270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:24:49.521 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:49.519+0000 7f6dea20e740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ba7f4bb8e0 0x55ba7f4bdd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:24:49.521 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:49.519+0000 7f6dea20e740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ba7f41ac90 con 0x55ba7f4b5d80 2024-03-21T10:24:49.521 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:49.519+0000 7f6dea20e740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ba7f3c3980 con 0x55ba7f4bb8e0 2024-03-21T10:24:49.521 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:49.519+0000 7f6dea20e740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ba7f416d30 con 0x55ba7f4b6770 2024-03-21T10:24:49.521 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:49.519+0000 7f6ddfe00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ba7f4b5d80 0x55ba7f4b61a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:24:49.521 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:49.519+0000 7f6ddfe00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ba7f4b5d80 0x55ba7f4b61a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:44506/0 (socket says 172.21.15.47:44506) 2024-03-21T10:24:49.521 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:49.519+0000 7f6ddfe00640 1 -- 172.21.15.47:0/3409360575 learned_addr learned my addr 172.21.15.47:0/3409360575 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:24:49.521 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:49.519+0000 7f6de4c00640 1 --2- 172.21.15.47:0/3409360575 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ba7f4b6770 0x55ba7f4bb270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:24:49.521 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:49.519+0000 7f6ddfe00640 1 --2- 172.21.15.47:0/3409360575 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ba7f4bb8e0 0x55ba7f4bdd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:24:49.522 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:49.519+0000 7f6ddd600640 1 --2- 172.21.15.47:0/2909926818 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f6dc80a2f20 0x7f6dc80a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:24:49.522 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:49.520+0000 7f6ddfe00640 1 -- 172.21.15.47:0/3409360575 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ba7f4b6770 msgr2=0x55ba7f4bb270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:24:49.522 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:49.520+0000 7f6ddfe00640 1 --2- 172.21.15.47:0/3409360575 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ba7f4b6770 0x55ba7f4bb270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:24:49.522 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:49.520+0000 7f6ddfe00640 1 -- 172.21.15.47:0/3409360575 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ba7f4bb8e0 msgr2=0x55ba7f4bdd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:24:49.522 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:49.520+0000 7f6ddfe00640 1 --2- 172.21.15.47:0/3409360575 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ba7f4bb8e0 0x55ba7f4bdd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:24:49.522 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:49.520+0000 7f6ddfe00640 1 -- 172.21.15.47:0/3409360575 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ba7f3db510 con 0x55ba7f4b5d80 2024-03-21T10:24:49.522 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:49.520+0000 7f6de4c00640 1 --2- 172.21.15.47:0/3409360575 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ba7f4b6770 0x55ba7f4bb270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T10:24:49.522 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:49.520+0000 7f6ddd600640 1 -- 172.21.15.47:0/2909926818 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9216+0+0 (secure 0 0 0) 0x7f6dd80cfff0 con 0x7f6de0103150 2024-03-21T10:24:49.522 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:49.520+0000 7f6de5600640 1 --2- 172.21.15.47:0/2909926818 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f6dc80a2f20 0x7f6dc80a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:24:49.522 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:49.520+0000 7f6ddfe00640 1 --2- 172.21.15.47:0/3409360575 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ba7f4b5d80 0x55ba7f4b61a0 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f6dcc005ab0 tx=0x7f6dcc02c5d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e20ed6be703106ef server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:24:49.524 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:49.521+0000 7f6de5600640 1 --2- 172.21.15.47:0/2909926818 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f6dc80a2f20 0x7f6dc80a53e0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f6dd809c030 tx=0x7f6dd8009a70 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:24:49.524 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:49.521+0000 7f6dd1c00640 1 -- 172.21.15.47:0/3409360575 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6dcc048070 con 0x55ba7f4b5d80 2024-03-21T10:24:49.524 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:49.521+0000 7f6dd1c00640 1 -- 172.21.15.47:0/3409360575 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f6dcc0054f0 con 0x55ba7f4b5d80 2024-03-21T10:24:49.524 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:49.521+0000 7f6dd1c00640 1 -- 172.21.15.47:0/3409360575 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6dcc004ab0 con 0x55ba7f4b5d80 2024-03-21T10:24:49.524 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:49.522+0000 7f6dea20e740 1 -- 172.21.15.47:0/3409360575 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ba7f4b5d80 msgr2=0x55ba7f4b61a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:24:49.524 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:49.522+0000 7f6dea20e740 1 --2- 172.21.15.47:0/3409360575 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ba7f4b5d80 0x55ba7f4b61a0 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f6dcc005ab0 tx=0x7f6dcc02c5d0 comp rx=0 tx=0).stop 2024-03-21T10:24:49.524 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:49.522+0000 7f6dea20e740 1 -- 172.21.15.47:0/3409360575 shutdown_connections 2024-03-21T10:24:49.524 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:49.522+0000 7f6dea20e740 1 --2- 172.21.15.47:0/3409360575 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ba7f4bb8e0 0x55ba7f4bdd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:24:49.524 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:49.522+0000 7f6dea20e740 1 --2- 172.21.15.47:0/3409360575 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ba7f4b6770 0x55ba7f4bb270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:24:49.524 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:49.522+0000 7f6dea20e740 1 --2- 172.21.15.47:0/3409360575 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ba7f4b5d80 0x55ba7f4b61a0 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:24:49.524 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:49.522+0000 7f6dea20e740 1 -- 172.21.15.47:0/3409360575 >> 172.21.15.47:0/3409360575 conn(0x55ba7f4b12f0 msgr2=0x55ba7f4b16f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:24:49.524 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:49.522+0000 7f6dea20e740 1 -- 172.21.15.47:0/3409360575 shutdown_connections 2024-03-21T10:24:49.524 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:49.522+0000 7f6dea20e740 1 -- 172.21.15.47:0/3409360575 wait complete. 2024-03-21T10:24:49.525 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:49.523+0000 7f6dea20e740 1 -- start start 2024-03-21T10:24:49.525 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:49.523+0000 7f6dea20e740 1 Processor -- start 2024-03-21T10:24:49.527 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:49.523+0000 7f6dea20e740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ba7f4b5d80 0x55ba7f4c4760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:24:49.527 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:49.523+0000 7f6dea20e740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ba7f4b6770 0x55ba7f4c4ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:24:49.528 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:49.523+0000 7f6dea20e740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ba7f4bb8e0 0x55ba7f4cbce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:24:49.528 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:49.523+0000 7f6dea20e740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ba7f4be830 con 0x55ba7f4bb8e0 2024-03-21T10:24:49.528 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:49.523+0000 7f6dea20e740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ba7f3d6dd0 con 0x55ba7f4b5d80 2024-03-21T10:24:49.528 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:49.523+0000 7f6dea20e740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ba7f4be9b0 con 0x55ba7f4b6770 2024-03-21T10:24:49.528 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:49.523+0000 7f6dea20e740 10 client.0 _ll_register_callbacks cb 0x55ba7f12eac0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T10:24:49.528 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:49.524+0000 7f6ddfe00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ba7f4bb8e0 0x55ba7f4cbce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:24:49.528 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:49.524+0000 7f6ddfe00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ba7f4b5d80 0x55ba7f4c4760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:24:49.528 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:49.524+0000 7f6ddfe00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ba7f4b5d80 0x55ba7f4c4760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:56532/0 (socket says 172.21.15.47:56532) 2024-03-21T10:24:49.528 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:49.524+0000 7f6ddfe00640 1 -- 172.21.15.47:0/1470285162 learned_addr learned my addr 172.21.15.47:0/1470285162 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:24:49.528 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:49.524+0000 7f6de4c00640 1 --2- 172.21.15.47:0/1470285162 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ba7f4b6770 0x55ba7f4c4ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:24:49.528 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:49.524+0000 7f6ddfe00640 1 -- 172.21.15.47:0/1470285162 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ba7f4b6770 msgr2=0x55ba7f4c4ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:24:49.528 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:49.524+0000 7f6ddfe00640 1 --2- 172.21.15.47:0/1470285162 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ba7f4b6770 0x55ba7f4c4ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:24:49.528 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:49.524+0000 7f6ddfe00640 1 -- 172.21.15.47:0/1470285162 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ba7f4bb8e0 msgr2=0x55ba7f4cbce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:24:49.529 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:49.524+0000 7f6ddfe00640 1 --2- 172.21.15.47:0/1470285162 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ba7f4bb8e0 0x55ba7f4cbce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:24:49.529 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:49.524+0000 7f6ddfe00640 1 -- 172.21.15.47:0/1470285162 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55ba7f12f2d0 con 0x55ba7f4b5d80 2024-03-21T10:24:49.529 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:49.524+0000 7f6ddfe00640 1 --2- 172.21.15.47:0/1470285162 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ba7f4b5d80 0x55ba7f4c4760 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f6dcc005a80 tx=0x7f6dcc005fd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3f522e54db77a7d2 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:24:49.529 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:49.525+0000 7f6dc6000640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-21T10:24:49.529 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:49.525+0000 7f6dc6000640 1 -- 172.21.15.47:0/1470285162 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6dcc048070 con 0x55ba7f4b5d80 2024-03-21T10:24:49.529 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:49.525+0000 7f6dea20e740 1 -- 172.21.15.47:0/1470285162 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f6dcc0054f0 con 0x55ba7f4b5d80 2024-03-21T10:24:49.529 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:49.525+0000 7f6dc6000640 1 -- 172.21.15.47:0/1470285162 <== mon.2 v2:172.21.15.100:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9216+0+0 (secure 0 0 0) 0x7f6dcc043440 con 0x55ba7f4b5d80 2024-03-21T10:24:49.529 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:49.525+0000 7f6dc6000640 1 -- 172.21.15.47:0/1470285162 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6dcc005320 con 0x55ba7f4b5d80 2024-03-21T10:24:49.529 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:49.525+0000 7f6dea20e740 10 client.35279 fetch_fsmap learned FSMap version 29 2024-03-21T10:24:49.529 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:49.525+0000 7f6dea20e740 1 -- 172.21.15.47:0/1470285162 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x55ba7f1f96b0 con 0x55ba7f4b5d80 2024-03-21T10:24:49.529 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:49.526+0000 7f6dc6000640 1 -- 172.21.15.47:0/1470285162 <== mon.2 v2:172.21.15.100:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f6dcc0054f0 con 0x55ba7f4b5d80 2024-03-21T10:24:49.529 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:49.526+0000 7f6dc6000640 1 -- 172.21.15.47:0/1470285162 <== mon.2 v2:172.21.15.100:3300/0 5 ==== fsmap(e 29) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f6dcc053560 con 0x55ba7f4b5d80 2024-03-21T10:24:49.529 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:49.526+0000 7f6dea20e740 10 client.35279 fetch_fsmap finished waiting for FSMap version 29 2024-03-21T10:24:49.530 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:49.526+0000 7f6dea20e740 10 client.35279 resolve_mds: resolved name 'e' to daemon mds.e 2024-03-21T10:24:49.530 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:49.526+0000 7f6dea20e740 20 client.35279 populate_metadata read hostname 'smithi047' 2024-03-21T10:24:49.530 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:49.526+0000 7f6dea20e740 1 --2- 172.21.15.47:0/1470285162 >> [v2:172.21.15.100:6834/2597849957,v1:172.21.15.100:6835/2597849957] conn(0x55ba7f4d0870 0x55ba7f4d4cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:24:49.530 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:49.526+0000 7f6dea20e740 4 client.35279 mds_command: new command op to 34454 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T10:24:49.530 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:49.526+0000 7f6dea20e740 1 -- 172.21.15.47:0/1470285162 --> [v2:172.21.15.100:6834/2597849957,v1:172.21.15.100:6835/2597849957] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55ba7f3bc520 con 0x55ba7f4d0870 2024-03-21T10:24:49.530 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:49.527+0000 7f6de4c00640 1 --2- 172.21.15.47:0/1470285162 >> [v2:172.21.15.100:6834/2597849957,v1:172.21.15.100:6835/2597849957] conn(0x55ba7f4d0870 0x55ba7f4d4cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:24:49.531 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:49.528+0000 7f6de4c00640 1 --2- 172.21.15.47:0/1470285162 >> [v2:172.21.15.100:6834/2597849957,v1:172.21.15.100:6835/2597849957] conn(0x55ba7f4d0870 0x55ba7f4d4cf0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=1c3d7db4211a124c server_cookie=89905f81a49d3895 in_seq=0 out_seq=0 2024-03-21T10:24:49.531 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:49.528+0000 7f6dc6000640 10 client.35279 ms_handle_connect on v2:172.21.15.100:6834/2597849957 2024-03-21T10:24:49.531 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:49.529+0000 7f6dc6000640 1 -- 172.21.15.47:0/1470285162 <== mds.2 v2:172.21.15.100:6834/2597849957 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55ba7f3bc520 con 0x55ba7f4d0870 2024-03-21T10:24:49.531 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:49.529+0000 7f6dc6000640 10 client.35279 handle_command_reply: tid=0 2024-03-21T10:24:49.531 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:49.529+0000 7f6dea20e740 1 client.35279 shutdown 2024-03-21T10:24:49.531 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:49.529+0000 7f6dea20e740 1 -- 172.21.15.47:0/1470285162 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ba7f4b5d80 msgr2=0x55ba7f4c4760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:24:49.532 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:49.529+0000 7f6dea20e740 1 --2- 172.21.15.47:0/1470285162 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ba7f4b5d80 0x55ba7f4c4760 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f6dcc005a80 tx=0x7f6dcc005fd0 comp rx=0 tx=0).stop 2024-03-21T10:24:49.532 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:49.529+0000 7f6dea20e740 1 -- 172.21.15.47:0/1470285162 shutdown_connections 2024-03-21T10:24:49.532 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:49.529+0000 7f6dea20e740 1 --2- 172.21.15.47:0/1470285162 >> [v2:172.21.15.100:6834/2597849957,v1:172.21.15.100:6835/2597849957] conn(0x55ba7f4d0870 0x55ba7f4d4cf0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:24:49.532 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:49.529+0000 7f6dea20e740 1 --2- 172.21.15.47:0/1470285162 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ba7f4bb8e0 0x55ba7f4cbce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:24:49.532 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:49.529+0000 7f6dea20e740 1 --2- 172.21.15.47:0/1470285162 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ba7f4b6770 0x55ba7f4c4ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:24:49.532 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:49.529+0000 7f6dea20e740 1 --2- 172.21.15.47:0/1470285162 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ba7f4b5d80 0x55ba7f4c4760 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:24:49.532 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:49.529+0000 7f6dc6000640 0 client.35279 ms_handle_reset on v2:172.21.15.100:6834/2597849957 2024-03-21T10:24:49.532 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:49.529+0000 7f6dea20e740 1 -- 172.21.15.47:0/1470285162 >> 172.21.15.47:0/1470285162 conn(0x55ba7f4b12f0 msgr2=0x55ba7f4b04f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:24:49.532 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:49.529+0000 7f6dea20e740 1 -- 172.21.15.47:0/1470285162 shutdown_connections 2024-03-21T10:24:49.532 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:49.529+0000 7f6dea20e740 1 -- 172.21.15.47:0/1470285162 wait complete. 2024-03-21T10:24:49.532 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:49.530+0000 7f6dea20e740 20 client.35279 trim_cache size 0 max 16384 2024-03-21T10:24:49.552 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:49.551+0000 7f6dea20e740 1 Processor -- start 2024-03-21T10:24:49.552 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:49.551+0000 7f6dea20e740 1 -- start start 2024-03-21T10:24:49.552 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:49.551+0000 7f6dea20e740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ba7f4b5d80 0x55ba7f4b16d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:24:49.553 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:49.551+0000 7f6dea20e740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ba7f4b6770 0x55ba7f4cbca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:24:49.553 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:49.551+0000 7f6dea20e740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ba7f4bb8e0 0x55ba7f4c4370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:24:49.553 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:49.551+0000 7f6dea20e740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ba7f4d0090 con 0x55ba7f4bb8e0 2024-03-21T10:24:49.553 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:49.551+0000 7f6dea20e740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ba7f4d06f0 con 0x55ba7f4b5d80 2024-03-21T10:24:49.553 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:49.551+0000 7f6dea20e740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ba7f4d0d50 con 0x55ba7f4b6770 2024-03-21T10:24:49.555 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:49.552+0000 7f6ddfe00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ba7f4bb8e0 0x55ba7f4c4370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:24:49.555 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:49.552+0000 7f6ddfe00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ba7f4bb8e0 0x55ba7f4c4370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:44528/0 (socket says 172.21.15.47:44528) 2024-03-21T10:24:49.555 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:49.552+0000 7f6ddfe00640 1 -- 172.21.15.47:0/1661207493 learned_addr learned my addr 172.21.15.47:0/1661207493 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:24:49.555 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:49.552+0000 7f6ddfe00640 1 --2- 172.21.15.47:0/1661207493 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ba7f4b5d80 0x55ba7f4b16d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:24:49.555 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:49.552+0000 7f6de4c00640 1 --2- 172.21.15.47:0/1661207493 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ba7f4b6770 0x55ba7f4cbca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:24:49.555 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:49.552+0000 7f6ddfe00640 1 -- 172.21.15.47:0/1661207493 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ba7f4b6770 msgr2=0x55ba7f4cbca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:24:49.555 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:49.552+0000 7f6ddfe00640 1 --2- 172.21.15.47:0/1661207493 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ba7f4b6770 0x55ba7f4cbca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:24:49.555 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:49.552+0000 7f6ddfe00640 1 -- 172.21.15.47:0/1661207493 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ba7f4b5d80 msgr2=0x55ba7f4b16d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:24:49.555 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:49.552+0000 7f6ddfe00640 1 --2- 172.21.15.47:0/1661207493 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ba7f4b5d80 0x55ba7f4b16d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:24:49.555 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:49.552+0000 7f6ddfe00640 1 -- 172.21.15.47:0/1661207493 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ba7f4bb620 con 0x55ba7f4bb8e0 2024-03-21T10:24:49.555 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:49.553+0000 7f6ddfe00640 1 --2- 172.21.15.47:0/1661207493 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ba7f4bb8e0 0x55ba7f4c4370 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f6dcc02c840 tx=0x7f6dcc02bc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e0f7c497d13d9d2 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:24:49.556 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:49.553+0000 7f6dd1200640 1 -- 172.21.15.47:0/1661207493 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6dcc048070 con 0x55ba7f4bb8e0 2024-03-21T10:24:49.556 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:49.553+0000 7f6dd1200640 1 -- 172.21.15.47:0/1661207493 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f6dcc0054c0 con 0x55ba7f4bb8e0 2024-03-21T10:24:49.556 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:49.553+0000 7f6dd1200640 1 -- 172.21.15.47:0/1661207493 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6dcc0346a0 con 0x55ba7f4bb8e0 2024-03-21T10:24:49.556 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:49.553+0000 7f6dea20e740 1 -- 172.21.15.47:0/1661207493 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ba7f4bb8e0 msgr2=0x55ba7f4c4370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:24:49.556 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:49.553+0000 7f6dea20e740 1 --2- 172.21.15.47:0/1661207493 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ba7f4bb8e0 0x55ba7f4c4370 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f6dcc02c840 tx=0x7f6dcc02bc30 comp rx=0 tx=0).stop 2024-03-21T10:24:49.556 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:49.553+0000 7f6dea20e740 1 -- 172.21.15.47:0/1661207493 shutdown_connections 2024-03-21T10:24:49.556 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:49.554+0000 7f6dea20e740 1 --2- 172.21.15.47:0/1661207493 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ba7f4bb8e0 0x55ba7f4c4370 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:24:49.556 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:49.554+0000 7f6dea20e740 1 --2- 172.21.15.47:0/1661207493 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ba7f4b6770 0x55ba7f4cbca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:24:49.556 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:49.554+0000 7f6dea20e740 1 --2- 172.21.15.47:0/1661207493 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ba7f4b5d80 0x55ba7f4b16d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:24:49.556 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:49.554+0000 7f6dea20e740 1 -- 172.21.15.47:0/1661207493 >> 172.21.15.47:0/1661207493 conn(0x55ba7f4b12f0 msgr2=0x55ba7f4bdcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:24:49.556 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:49.554+0000 7f6dea20e740 1 -- 172.21.15.47:0/1661207493 shutdown_connections 2024-03-21T10:24:49.556 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:49.554+0000 7f6dea20e740 1 -- 172.21.15.47:0/1661207493 wait complete. 2024-03-21T10:24:49.556 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:49.554+0000 7f6dea20e740 1 -- start start 2024-03-21T10:24:49.556 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:49.554+0000 7f6dea20e740 1 Processor -- start 2024-03-21T10:24:49.559 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:49.555+0000 7f6dea20e740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ba7f4b5d80 0x55ba7f4cb730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:24:49.559 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:49.555+0000 7f6dea20e740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ba7f4b6770 0x55ba7f4cbc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:24:49.559 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:49.555+0000 7f6dea20e740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ba7f4c8220 0x55ba7f4c8680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:24:49.559 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:49.555+0000 7f6dea20e740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ba7f4d1a10 con 0x55ba7f4c8220 2024-03-21T10:24:49.559 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:49.555+0000 7f6dea20e740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ba7f4d13b0 con 0x55ba7f4b6770 2024-03-21T10:24:49.559 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:49.555+0000 7f6dea20e740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ba7f4d2070 con 0x55ba7f4b5d80 2024-03-21T10:24:49.559 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:49.555+0000 7f6dea20e740 10 client.0 _ll_register_callbacks cb 0x7f6dcc046070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T10:24:49.559 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:49.555+0000 7f6ddfe00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ba7f4c8220 0x55ba7f4c8680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:24:49.559 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:49.555+0000 7f6de4c00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ba7f4b6770 0x55ba7f4cbc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:24:49.559 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:49.555+0000 7f6ddfe00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ba7f4c8220 0x55ba7f4c8680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:44542/0 (socket says 172.21.15.47:44542) 2024-03-21T10:24:49.560 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:49.555+0000 7f6ddfe00640 1 -- 172.21.15.47:0/1389397391 learned_addr learned my addr 172.21.15.47:0/1389397391 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:24:49.560 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:49.555+0000 7f6ddfe00640 1 --2- 172.21.15.47:0/1389397391 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ba7f4b5d80 0x55ba7f4cb730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:24:49.560 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:49.555+0000 7f6ddfe00640 1 -- 172.21.15.47:0/1389397391 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ba7f4b5d80 msgr2=0x55ba7f4cb730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:24:49.560 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:49.555+0000 7f6ddfe00640 1 --2- 172.21.15.47:0/1389397391 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ba7f4b5d80 0x55ba7f4cb730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:24:49.560 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:49.555+0000 7f6ddfe00640 1 -- 172.21.15.47:0/1389397391 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ba7f4b6770 msgr2=0x55ba7f4cbc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:24:49.560 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:49.555+0000 7f6ddfe00640 1 --2- 172.21.15.47:0/1389397391 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ba7f4b6770 0x55ba7f4cbc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:24:49.560 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:49.555+0000 7f6ddfe00640 1 -- 172.21.15.47:0/1389397391 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55ba7f52be10 con 0x55ba7f4c8220 2024-03-21T10:24:49.560 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:49.556+0000 7f6ddfe00640 1 --2- 172.21.15.47:0/1389397391 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ba7f4c8220 0x55ba7f4c8680 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f6dcc053980 tx=0x7f6dcc045040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f0c041baf9397a24 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:24:49.560 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:49.556+0000 7f6dc6000640 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-03-21T10:24:49.560 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:49.556+0000 7f6dc6000640 1 -- 172.21.15.47:0/1389397391 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6dcc048070 con 0x55ba7f4c8220 2024-03-21T10:24:49.560 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:49.556+0000 7f6dc6000640 1 -- 172.21.15.47:0/1389397391 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9216+0+0 (secure 0 0 0) 0x7f6dcc063480 con 0x55ba7f4c8220 2024-03-21T10:24:49.560 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:49.556+0000 7f6dea20e740 1 -- 172.21.15.47:0/1389397391 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f6dcc0054c0 con 0x55ba7f4c8220 2024-03-21T10:24:49.560 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:49.556+0000 7f6dc6000640 1 -- 172.21.15.47:0/1389397391 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6dcc005d50 con 0x55ba7f4c8220 2024-03-21T10:24:49.560 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:49.556+0000 7f6dea20e740 10 client.15450 fetch_fsmap learned FSMap version 29 2024-03-21T10:24:49.561 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:49.556+0000 7f6dea20e740 1 -- 172.21.15.47:0/1389397391 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x55ba7f52d040 con 0x55ba7f4c8220 2024-03-21T10:24:49.561 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:49.557+0000 7f6dc6000640 1 -- 172.21.15.47:0/1389397391 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f6dcc06b070 con 0x55ba7f4c8220 2024-03-21T10:24:49.561 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:49.557+0000 7f6dc6000640 1 -- 172.21.15.47:0/1389397391 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 29) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f6dcc034440 con 0x55ba7f4c8220 2024-03-21T10:24:49.561 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:49.557+0000 7f6dea20e740 10 client.15450 fetch_fsmap finished waiting for FSMap version 29 2024-03-21T10:24:49.561 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:49.557+0000 7f6dea20e740 10 client.15450 resolve_mds: resolved name 'e' to daemon mds.e 2024-03-21T10:24:49.561 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:49.557+0000 7f6dea20e740 20 client.15450 populate_metadata read hostname 'smithi047' 2024-03-21T10:24:49.561 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:49.557+0000 7f6dea20e740 1 --2- 172.21.15.47:0/1389397391 >> [v2:172.21.15.100:6834/2597849957,v1:172.21.15.100:6835/2597849957] conn(0x55ba7f5311c0 0x55ba7f52d210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:24:49.561 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:49.557+0000 7f6dea20e740 4 client.15450 mds_command: new command op to 34454 tid=0 multi_id=0 [{"prefix": "status"}] 2024-03-21T10:24:49.561 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:49.557+0000 7f6dea20e740 1 -- 172.21.15.47:0/1389397391 --> [v2:172.21.15.100:6834/2597849957,v1:172.21.15.100:6835/2597849957] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55ba7f4281d0 con 0x55ba7f5311c0 2024-03-21T10:24:49.561 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:49.557+0000 7f6de4c00640 1 --2- 172.21.15.47:0/1389397391 >> [v2:172.21.15.100:6834/2597849957,v1:172.21.15.100:6835/2597849957] conn(0x55ba7f5311c0 0x55ba7f52d210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:24:49.561 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:49.558+0000 7f6de4c00640 1 --2- 172.21.15.47:0/1389397391 >> [v2:172.21.15.100:6834/2597849957,v1:172.21.15.100:6835/2597849957] conn(0x55ba7f5311c0 0x55ba7f52d210 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=8282ef3eba4874fb server_cookie=eb930185d249c683 in_seq=0 out_seq=0 2024-03-21T10:24:49.561 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:49.558+0000 7f6dc6000640 10 client.15450 ms_handle_connect on v2:172.21.15.100:6834/2597849957 2024-03-21T10:24:49.562 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:49.558+0000 7f6dc6000640 1 -- 172.21.15.47:0/1389397391 <== mds.2 v2:172.21.15.100:6834/2597849957 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+332 (crc 0 0 0) 0x55ba7f4281d0 con 0x55ba7f5311c0 2024-03-21T10:24:49.562 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:49.558+0000 7f6dc6000640 10 client.15450 handle_command_reply: tid=0 2024-03-21T10:24:49.562 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:49.558+0000 7f6dea20e740 1 client.15450 shutdown 2024-03-21T10:24:49.563 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:49.559+0000 7f6dea20e740 1 -- 172.21.15.47:0/1389397391 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ba7f4c8220 msgr2=0x55ba7f4c8680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:24:49.563 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:49.559+0000 7f6dea20e740 1 --2- 172.21.15.47:0/1389397391 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ba7f4c8220 0x55ba7f4c8680 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f6dcc053980 tx=0x7f6dcc045040 comp rx=0 tx=0).stop 2024-03-21T10:24:49.563 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:49.559+0000 7f6dea20e740 1 -- 172.21.15.47:0/1389397391 shutdown_connections 2024-03-21T10:24:49.563 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:49.559+0000 7f6dea20e740 1 --2- 172.21.15.47:0/1389397391 >> [v2:172.21.15.100:6834/2597849957,v1:172.21.15.100:6835/2597849957] conn(0x55ba7f5311c0 0x55ba7f52d210 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:24:49.563 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:49.559+0000 7f6dea20e740 1 --2- 172.21.15.47:0/1389397391 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ba7f4c8220 0x55ba7f4c8680 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:24:49.563 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:49.559+0000 7f6dea20e740 1 --2- 172.21.15.47:0/1389397391 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ba7f4b6770 0x55ba7f4cbc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:24:49.563 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:49.559+0000 7f6dc6000640 0 client.15450 ms_handle_reset on v2:172.21.15.100:6834/2597849957 2024-03-21T10:24:49.563 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:49.559+0000 7f6dea20e740 1 --2- 172.21.15.47:0/1389397391 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ba7f4b5d80 0x55ba7f4cb730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:24:49.563 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:49.559+0000 7f6dea20e740 1 -- 172.21.15.47:0/1389397391 >> 172.21.15.47:0/1389397391 conn(0x55ba7f4b12f0 msgr2=0x55ba7f4b16d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:24:49.563 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:49.559+0000 7f6dea20e740 1 -- 172.21.15.47:0/1389397391 shutdown_connections 2024-03-21T10:24:49.563 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:49.559+0000 7f6dea20e740 1 -- 172.21.15.47:0/1389397391 wait complete. 2024-03-21T10:24:49.563 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:49.559+0000 7f6dea20e740 20 client.15450 trim_cache size 0 max 16384 2024-03-21T10:24:49.564 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-03-21T10:24:49.564 INFO:teuthology.orchestra.run.smithi047.stdout: "cluster_fsid": "3056d51a-e76b-11ee-95cd-87774f69a715", 2024-03-21T10:24:49.564 INFO:teuthology.orchestra.run.smithi047.stdout: "whoami": 2, 2024-03-21T10:24:49.564 INFO:teuthology.orchestra.run.smithi047.stdout: "id": 34454, 2024-03-21T10:24:49.564 INFO:teuthology.orchestra.run.smithi047.stdout: "want_state": "up:active", 2024-03-21T10:24:49.564 INFO:teuthology.orchestra.run.smithi047.stdout: "state": "up:active", 2024-03-21T10:24:49.564 INFO:teuthology.orchestra.run.smithi047.stdout: "fs_name": "cephfs", 2024-03-21T10:24:49.564 INFO:teuthology.orchestra.run.smithi047.stdout: "rank_uptime": 143.37681060700001, 2024-03-21T10:24:49.564 INFO:teuthology.orchestra.run.smithi047.stdout: "mdsmap_epoch": 29, 2024-03-21T10:24:49.564 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap_epoch": 95, 2024-03-21T10:24:49.564 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap_epoch_barrier": 93, 2024-03-21T10:24:49.564 INFO:teuthology.orchestra.run.smithi047.stdout: "uptime": 203.91205960299999 2024-03-21T10:24:49.564 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-03-21T10:24:49.565 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:49.560+0000 7f6de6000640 1 -- 172.21.15.47:0/2909926818 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f6dc80a2f20 msgr2=0x7f6dc80a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:24:49.565 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:49.560+0000 7f6de6000640 1 --2- 172.21.15.47:0/2909926818 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f6dc80a2f20 0x7f6dc80a53e0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f6dd809c030 tx=0x7f6dd8009a70 comp rx=0 tx=0).stop 2024-03-21T10:24:49.565 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:49.560+0000 7f6de6000640 1 -- 172.21.15.47:0/2909926818 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6de0103150 msgr2=0x7f6de019ed70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:24:49.565 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:49.560+0000 7f6de6000640 1 --2- 172.21.15.47:0/2909926818 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6de0103150 0x7f6de019ed70 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f6dd800eb10 tx=0x7f6dd800c2d0 comp rx=0 tx=0).stop 2024-03-21T10:24:49.565 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:49.560+0000 7f6de6000640 1 -- 172.21.15.47:0/2909926818 shutdown_connections 2024-03-21T10:24:49.565 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:49.560+0000 7f6de6000640 1 --2- 172.21.15.47:0/2909926818 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f6dc80a2f20 0x7f6dc80a53e0 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:24:49.566 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:49.560+0000 7f6de6000640 1 --2- 172.21.15.47:0/2909926818 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6de0103150 0x7f6de019ed70 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:24:49.566 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:49.560+0000 7f6de6000640 1 --2- 172.21.15.47:0/2909926818 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6de0102790 0x7f6de0197d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:24:49.566 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:49.560+0000 7f6de6000640 1 --2- 172.21.15.47:0/2909926818 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6de01015a0 0x7f6de01977c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:24:49.566 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:49.560+0000 7f6de6000640 1 -- 172.21.15.47:0/2909926818 >> 172.21.15.47:0/2909926818 conn(0x7f6de00fcb10 msgr2=0x7f6de0105b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:24:49.566 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:49.560+0000 7f6de6000640 1 -- 172.21.15.47:0/2909926818 shutdown_connections 2024-03-21T10:24:49.566 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:49.560+0000 7f6de6000640 1 -- 172.21.15.47:0/2909926818 wait complete. 2024-03-21T10:24:49.576 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:24:49 smithi047 ceph-mon[31547]: pgmap v456: 129 pgs: 129 active+clean; 642 KiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-03-21T10:24:49.576 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:24:49 smithi047 ceph-mon[31547]: from='client.? 172.21.15.47:0/4107755992' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-21T10:24:49.578 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.h status 2024-03-21T10:24:49.630 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:24:49 smithi100 ceph-mon[35193]: pgmap v456: 129 pgs: 129 active+clean; 642 KiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-03-21T10:24:49.631 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:24:49 smithi100 ceph-mon[35193]: from='client.? 172.21.15.47:0/4107755992' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-21T10:24:49.740 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:24:49 smithi157 ceph-mon[35581]: pgmap v456: 129 pgs: 129 active+clean; 642 KiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-03-21T10:24:49.740 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:24:49 smithi157 ceph-mon[35581]: from='client.? 172.21.15.47:0/4107755992' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-21T10:24:49.787 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:49.784+0000 7f3bc1000640 1 -- 172.21.15.47:0/468082310 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3bbc103130 msgr2=0x7f3bbc1099c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:24:49.787 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:49.784+0000 7f3bc1000640 1 --2- 172.21.15.47:0/468082310 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3bbc103130 0x7f3bbc1099c0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f3bb4009900 tx=0x7f3bb402bc40 comp rx=0 tx=0).stop 2024-03-21T10:24:49.787 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:49.784+0000 7f3bc1000640 1 -- 172.21.15.47:0/468082310 shutdown_connections 2024-03-21T10:24:49.787 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:49.784+0000 7f3bc1000640 1 --2- 172.21.15.47:0/468082310 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3bbc103130 0x7f3bbc1099c0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:24:49.787 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:49.784+0000 7f3bc1000640 1 --2- 172.21.15.47:0/468082310 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3bbc102770 0x7f3bbc102bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:24:49.787 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:49.784+0000 7f3bc1000640 1 --2- 172.21.15.47:0/468082310 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3bbc101580 0x7f3bbc1019a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:24:49.787 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:49.784+0000 7f3bc1000640 1 -- 172.21.15.47:0/468082310 >> 172.21.15.47:0/468082310 conn(0x7f3bbc0fcb10 msgr2=0x7f3bbc0fef30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:24:49.787 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:49.784+0000 7f3bc1000640 1 -- 172.21.15.47:0/468082310 shutdown_connections 2024-03-21T10:24:49.788 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:49.784+0000 7f3bc1000640 1 -- 172.21.15.47:0/468082310 wait complete. 2024-03-21T10:24:49.790 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:49.785+0000 7f3bc1000640 1 Processor -- start 2024-03-21T10:24:49.790 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:49.785+0000 7f3bc1000640 1 -- start start 2024-03-21T10:24:49.791 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:49.786+0000 7f3bc1000640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3bbc101580 0x7f3bbc075450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:24:49.791 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:49.786+0000 7f3bc1000640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3bbc102770 0x7f3bbc073a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:24:49.791 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:49.786+0000 7f3bc1000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3bbc103130 0x7f3bbc073fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:24:49.791 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:49.786+0000 7f3bc1000640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3bbc10d100 con 0x7f3bbc102770 2024-03-21T10:24:49.791 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:49.786+0000 7f3bc1000640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3bbc10cf80 con 0x7f3bbc103130 2024-03-21T10:24:49.791 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:49.786+0000 7f3bc1000640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3bbc10d280 con 0x7f3bbc101580 2024-03-21T10:24:49.791 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:49.786+0000 7f3bb3e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3bbc102770 0x7f3bbc073a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:24:49.791 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:49.786+0000 7f3bb3e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3bbc102770 0x7f3bbc073a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:44562/0 (socket says 172.21.15.47:44562) 2024-03-21T10:24:49.791 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:49.786+0000 7f3bb3e00640 1 -- 172.21.15.47:0/2850281165 learned_addr learned my addr 172.21.15.47:0/2850281165 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:24:49.792 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:49.786+0000 7f3bbb400640 1 --2- 172.21.15.47:0/2850281165 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3bbc101580 0x7f3bbc075450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:24:49.792 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:49.786+0000 7f3bbbe00640 1 --2- 172.21.15.47:0/2850281165 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3bbc103130 0x7f3bbc073fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:24:49.792 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:49.787+0000 7f3bb3e00640 1 -- 172.21.15.47:0/2850281165 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3bbc101580 msgr2=0x7f3bbc075450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:24:49.792 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:49.787+0000 7f3bb3e00640 1 --2- 172.21.15.47:0/2850281165 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3bbc101580 0x7f3bbc075450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:24:49.792 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:49.787+0000 7f3bb3e00640 1 -- 172.21.15.47:0/2850281165 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3bbc103130 msgr2=0x7f3bbc073fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:24:49.792 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:49.787+0000 7f3bb3e00640 1 --2- 172.21.15.47:0/2850281165 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3bbc103130 0x7f3bbc073fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:24:49.792 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:49.787+0000 7f3bb3e00640 1 -- 172.21.15.47:0/2850281165 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3bbc074860 con 0x7f3bbc102770 2024-03-21T10:24:49.792 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:49.787+0000 7f3bb3e00640 1 --2- 172.21.15.47:0/2850281165 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3bbc102770 0x7f3bbc073a90 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f3ba400e9f0 tx=0x7f3ba400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:24:49.792 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:49.787+0000 7f3bb8c00640 1 -- 172.21.15.47:0/2850281165 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ba400cd80 con 0x7f3bbc102770 2024-03-21T10:24:49.793 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:49.787+0000 7f3bc1000640 1 -- 172.21.15.47:0/2850281165 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3bbc1a1890 con 0x7f3bbc102770 2024-03-21T10:24:49.793 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:49.787+0000 7f3bb8c00640 1 -- 172.21.15.47:0/2850281165 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f3ba40108e0 con 0x7f3bbc102770 2024-03-21T10:24:49.793 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:49.787+0000 7f3bb8c00640 1 -- 172.21.15.47:0/2850281165 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ba4018920 con 0x7f3bbc102770 2024-03-21T10:24:49.793 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:49.788+0000 7f3bc1000640 1 -- 172.21.15.47:0/2850281165 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3bbc1a1e30 con 0x7f3bbc102770 2024-03-21T10:24:49.795 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:49.791+0000 7f3bc53f2740 1 Processor -- start 2024-03-21T10:24:49.795 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:49.791+0000 7f3bc53f2740 1 -- start start 2024-03-21T10:24:49.795 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:49.792+0000 7f3bc53f2740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55bb6cd47d80 0x55bb6cd481a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:24:49.795 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:49.792+0000 7f3bc53f2740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55bb6cd48770 0x55bb6cd4d270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:24:49.795 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:49.792+0000 7f3bc53f2740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55bb6cd4d8e0 0x55bb6cd4fd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:24:49.795 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:49.792+0000 7f3bc53f2740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bb6ccacc90 con 0x55bb6cd48770 2024-03-21T10:24:49.795 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:49.792+0000 7f3bc53f2740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bb6cc55980 con 0x55bb6cd4d8e0 2024-03-21T10:24:49.795 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:49.792+0000 7f3bc53f2740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bb6cca8d30 con 0x55bb6cd47d80 2024-03-21T10:24:49.795 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:49.792+0000 7f3bb8c00640 1 -- 172.21.15.47:0/2850281165 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 18) v1 ==== 137854+0+0 (secure 0 0 0) 0x7f3ba4010a90 con 0x7f3bbc102770 2024-03-21T10:24:49.795 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:49.792+0000 7f3bbb400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55bb6cd48770 0x55bb6cd4d270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:24:49.796 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:49.792+0000 7f3bbb400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55bb6cd48770 0x55bb6cd4d270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:44574/0 (socket says 172.21.15.47:44574) 2024-03-21T10:24:49.796 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:49.792+0000 7f3bbb400640 1 -- 172.21.15.47:0/2955821466 learned_addr learned my addr 172.21.15.47:0/2955821466 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:24:49.796 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:49.792+0000 7f3bbbe00640 1 --2- 172.21.15.47:0/2955821466 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55bb6cd47d80 0x55bb6cd481a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:24:49.796 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:49.792+0000 7f3bb3e00640 1 --2- 172.21.15.47:0/2955821466 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55bb6cd4d8e0 0x55bb6cd4fd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:24:49.796 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:49.793+0000 7f3bbb400640 1 -- 172.21.15.47:0/2955821466 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55bb6cd47d80 msgr2=0x55bb6cd481a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:24:49.796 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:49.793+0000 7f3bbb400640 1 --2- 172.21.15.47:0/2955821466 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55bb6cd47d80 0x55bb6cd481a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:24:49.796 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:49.793+0000 7f3bbb400640 1 -- 172.21.15.47:0/2955821466 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55bb6cd4d8e0 msgr2=0x55bb6cd4fd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:24:49.796 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:49.793+0000 7f3bbb400640 1 --2- 172.21.15.47:0/2955821466 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55bb6cd4d8e0 0x55bb6cd4fd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:24:49.796 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:49.793+0000 7f3bbb400640 1 -- 172.21.15.47:0/2955821466 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55bb6cc6d510 con 0x55bb6cd48770 2024-03-21T10:24:49.797 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:49.793+0000 7f3bbb400640 1 --2- 172.21.15.47:0/2955821466 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55bb6cd48770 0x55bb6cd4d270 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f3bac011470 tx=0x7f3bac011930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aa71ca6cff7d0688 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:24:49.805 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:49.793+0000 7f3bb0c00640 1 -- 172.21.15.47:0/2955821466 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3bac003970 con 0x55bb6cd48770 2024-03-21T10:24:49.805 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:49.794+0000 7f3bb0c00640 1 -- 172.21.15.47:0/2955821466 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f3bac004020 con 0x55bb6cd48770 2024-03-21T10:24:49.805 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:49.794+0000 7f3bb0c00640 1 -- 172.21.15.47:0/2955821466 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3bac00f840 con 0x55bb6cd48770 2024-03-21T10:24:49.805 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:49.794+0000 7f3bb8c00640 1 --2- 172.21.15.47:0/2850281165 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f3ba00a2fa0 0x7f3ba00a5460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:24:49.806 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:49.794+0000 7f3bbbe00640 1 --2- 172.21.15.47:0/2850281165 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f3ba00a2fa0 0x7f3ba00a5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:24:49.806 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:49.794+0000 7f3bb8c00640 1 -- 172.21.15.47:0/2850281165 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9216+0+0 (secure 0 0 0) 0x7f3ba40cfa00 con 0x7f3bbc102770 2024-03-21T10:24:49.806 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:49.794+0000 7f3bc53f2740 1 -- 172.21.15.47:0/2955821466 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55bb6cd48770 msgr2=0x55bb6cd4d270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:24:49.806 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:49.794+0000 7f3bc53f2740 1 --2- 172.21.15.47:0/2955821466 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55bb6cd48770 0x55bb6cd4d270 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f3bac011470 tx=0x7f3bac011930 comp rx=0 tx=0).stop 2024-03-21T10:24:49.806 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:49.794+0000 7f3bc53f2740 1 -- 172.21.15.47:0/2955821466 shutdown_connections 2024-03-21T10:24:49.806 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:49.794+0000 7f3bc53f2740 1 --2- 172.21.15.47:0/2955821466 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55bb6cd4d8e0 0x55bb6cd4fd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:24:49.807 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:49.794+0000 7f3bc53f2740 1 --2- 172.21.15.47:0/2955821466 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55bb6cd48770 0x55bb6cd4d270 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:24:49.807 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:49.794+0000 7f3bc53f2740 1 --2- 172.21.15.47:0/2955821466 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55bb6cd47d80 0x55bb6cd481a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:24:49.807 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:49.794+0000 7f3bc53f2740 1 -- 172.21.15.47:0/2955821466 >> 172.21.15.47:0/2955821466 conn(0x55bb6cd432f0 msgr2=0x55bb6cd436f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:24:49.807 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:49.795+0000 7f3bc53f2740 1 -- 172.21.15.47:0/2955821466 shutdown_connections 2024-03-21T10:24:49.807 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:49.795+0000 7f3bc53f2740 1 -- 172.21.15.47:0/2955821466 wait complete. 2024-03-21T10:24:49.807 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:49.795+0000 7f3bbbe00640 1 --2- 172.21.15.47:0/2850281165 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f3ba00a2fa0 0x7f3ba00a5460 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f3bb4006010 tx=0x7f3bb402c810 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:24:49.807 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:49.795+0000 7f3bc53f2740 1 -- start start 2024-03-21T10:24:49.807 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:49.795+0000 7f3bc53f2740 1 Processor -- start 2024-03-21T10:24:49.807 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:49.796+0000 7f3bc53f2740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55bb6cd47d80 0x55bb6cd52240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:24:49.807 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:49.796+0000 7f3bc53f2740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55bb6cd48770 0x55bb6cd52780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:24:49.807 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:49.796+0000 7f3bc53f2740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55bb6cd4d8e0 0x55bb6cd597c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:24:49.807 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:49.796+0000 7f3bc53f2740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bb6cd50890 con 0x55bb6cd4d8e0 2024-03-21T10:24:49.808 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:49.796+0000 7f3bc53f2740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bb6cc68dd0 con 0x55bb6cd47d80 2024-03-21T10:24:49.808 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:49.796+0000 7f3bc53f2740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bb6cd50a10 con 0x55bb6cd48770 2024-03-21T10:24:49.808 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:49.796+0000 7f3bc53f2740 10 client.0 _ll_register_callbacks cb 0x55bb6c9c0ac0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T10:24:49.808 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:49.796+0000 7f3bbbe00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55bb6cd4d8e0 0x55bb6cd597c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:24:49.808 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:49.796+0000 7f3bbbe00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55bb6cd4d8e0 0x55bb6cd597c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:44588/0 (socket says 172.21.15.47:44588) 2024-03-21T10:24:49.808 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:49.796+0000 7f3bbbe00640 1 -- 172.21.15.47:0/2411802117 learned_addr learned my addr 172.21.15.47:0/2411802117 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:24:49.808 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:49.796+0000 7f3bbb400640 1 --2- 172.21.15.47:0/2411802117 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55bb6cd47d80 0x55bb6cd52240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:24:49.808 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:49.797+0000 7f3bb3e00640 1 --2- 172.21.15.47:0/2411802117 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55bb6cd48770 0x55bb6cd52780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:24:49.808 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:49.797+0000 7f3bbbe00640 1 -- 172.21.15.47:0/2411802117 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55bb6cd48770 msgr2=0x55bb6cd52780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:24:49.808 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:49.797+0000 7f3bbbe00640 1 --2- 172.21.15.47:0/2411802117 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55bb6cd48770 0x55bb6cd52780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:24:49.808 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:49.797+0000 7f3bbbe00640 1 -- 172.21.15.47:0/2411802117 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55bb6cd47d80 msgr2=0x55bb6cd52240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:24:49.808 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:49.797+0000 7f3bbbe00640 1 --2- 172.21.15.47:0/2411802117 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55bb6cd47d80 0x55bb6cd52240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:24:49.808 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:49.797+0000 7f3bbbe00640 1 -- 172.21.15.47:0/2411802117 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55bb6c9c12d0 con 0x55bb6cd4d8e0 2024-03-21T10:24:49.809 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:49.797+0000 7f3bbbe00640 1 --2- 172.21.15.47:0/2411802117 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55bb6cd4d8e0 0x55bb6cd597c0 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f3bb4035000 tx=0x7f3bb402bc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e45c1a55241cd7f2 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:24:49.809 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:49.797+0000 7f3ba9600640 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-03-21T10:24:49.809 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:49.797+0000 7f3ba9600640 1 -- 172.21.15.47:0/2411802117 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3bb4050030 con 0x55bb6cd4d8e0 2024-03-21T10:24:49.809 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:49.798+0000 7f3ba9600640 1 -- 172.21.15.47:0/2411802117 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9216+0+0 (secure 0 0 0) 0x7f3bb403a040 con 0x55bb6cd4d8e0 2024-03-21T10:24:49.809 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:49.798+0000 7f3bc53f2740 1 -- 172.21.15.47:0/2411802117 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f3bac004020 con 0x55bb6cd4d8e0 2024-03-21T10:24:49.809 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:49.798+0000 7f3ba9600640 1 -- 172.21.15.47:0/2411802117 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3bb405d050 con 0x55bb6cd4d8e0 2024-03-21T10:24:49.809 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:49.798+0000 7f3bc53f2740 10 client.15462 fetch_fsmap learned FSMap version 29 2024-03-21T10:24:49.809 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:49.798+0000 7f3bc53f2740 1 -- 172.21.15.47:0/2411802117 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x55bb6ca8b6b0 con 0x55bb6cd4d8e0 2024-03-21T10:24:49.809 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:49.798+0000 7f3ba9600640 1 -- 172.21.15.47:0/2411802117 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f3bac004020 con 0x55bb6cd4d8e0 2024-03-21T10:24:49.809 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:49.798+0000 7f3ba9600640 1 -- 172.21.15.47:0/2411802117 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 29) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f3bb40038b0 con 0x55bb6cd4d8e0 2024-03-21T10:24:49.809 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:49.799+0000 7f3bc53f2740 10 client.15462 fetch_fsmap finished waiting for FSMap version 29 2024-03-21T10:24:49.809 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:49.799+0000 7f3bc53f2740 10 client.15462 resolve_mds: resolved name 'h' to daemon mds.h 2024-03-21T10:24:49.809 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:49.799+0000 7f3bc53f2740 20 client.15462 populate_metadata read hostname 'smithi047' 2024-03-21T10:24:49.809 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:49.799+0000 7f3bc53f2740 1 --2- 172.21.15.47:0/2411802117 >> [v2:172.21.15.100:6836/3497644162,v1:172.21.15.100:6837/3497644162] conn(0x55bb6cd5eac0 0x55bb6cd5db00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:24:49.810 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:49.799+0000 7f3bc53f2740 4 client.15462 mds_command: new command op to 34493 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T10:24:49.810 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:49.799+0000 7f3bc53f2740 1 -- 172.21.15.47:0/2411802117 --> [v2:172.21.15.100:6836/3497644162,v1:172.21.15.100:6837/3497644162] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55bb6cc4e520 con 0x55bb6cd5eac0 2024-03-21T10:24:49.810 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:49.799+0000 7f3bbb400640 1 --2- 172.21.15.47:0/2411802117 >> [v2:172.21.15.100:6836/3497644162,v1:172.21.15.100:6837/3497644162] conn(0x55bb6cd5eac0 0x55bb6cd5db00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:24:49.810 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:49.800+0000 7f3bbb400640 1 --2- 172.21.15.47:0/2411802117 >> [v2:172.21.15.100:6836/3497644162,v1:172.21.15.100:6837/3497644162] conn(0x55bb6cd5eac0 0x55bb6cd5db00 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=f426739e404d612 server_cookie=a15490836867818a in_seq=0 out_seq=0 2024-03-21T10:24:49.810 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:49.800+0000 7f3ba9600640 10 client.15462 ms_handle_connect on v2:172.21.15.100:6836/3497644162 2024-03-21T10:24:49.810 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:49.801+0000 7f3ba9600640 1 -- 172.21.15.47:0/2411802117 <== mds.1 v2:172.21.15.100:6836/3497644162 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55bb6cc4e520 con 0x55bb6cd5eac0 2024-03-21T10:24:49.810 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:49.801+0000 7f3ba9600640 10 client.15462 handle_command_reply: tid=0 2024-03-21T10:24:49.810 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:49.801+0000 7f3bc53f2740 1 client.15462 shutdown 2024-03-21T10:24:49.810 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:49.801+0000 7f3bc53f2740 1 -- 172.21.15.47:0/2411802117 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55bb6cd4d8e0 msgr2=0x55bb6cd597c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:24:49.810 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:49.801+0000 7f3bc53f2740 1 --2- 172.21.15.47:0/2411802117 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55bb6cd4d8e0 0x55bb6cd597c0 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f3bb4035000 tx=0x7f3bb402bc40 comp rx=0 tx=0).stop 2024-03-21T10:24:49.810 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:49.802+0000 7f3bc53f2740 1 -- 172.21.15.47:0/2411802117 shutdown_connections 2024-03-21T10:24:49.810 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:49.802+0000 7f3bc53f2740 1 --2- 172.21.15.47:0/2411802117 >> [v2:172.21.15.100:6836/3497644162,v1:172.21.15.100:6837/3497644162] conn(0x55bb6cd5eac0 0x55bb6cd5db00 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:24:49.810 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:49.802+0000 7f3bc53f2740 1 --2- 172.21.15.47:0/2411802117 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55bb6cd4d8e0 0x55bb6cd597c0 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:24:49.810 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:49.802+0000 7f3bc53f2740 1 --2- 172.21.15.47:0/2411802117 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55bb6cd48770 0x55bb6cd52780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:24:49.810 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:49.802+0000 7f3bc53f2740 1 --2- 172.21.15.47:0/2411802117 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55bb6cd47d80 0x55bb6cd52240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:24:49.811 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:49.802+0000 7f3bc53f2740 1 -- 172.21.15.47:0/2411802117 >> 172.21.15.47:0/2411802117 conn(0x55bb6cd432f0 msgr2=0x55bb6cd424f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:24:49.811 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:49.802+0000 7f3ba9600640 0 client.15462 ms_handle_reset on v2:172.21.15.100:6836/3497644162 2024-03-21T10:24:49.811 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:49.802+0000 7f3bc53f2740 1 -- 172.21.15.47:0/2411802117 shutdown_connections 2024-03-21T10:24:49.811 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:49.802+0000 7f3bc53f2740 1 -- 172.21.15.47:0/2411802117 wait complete. 2024-03-21T10:24:49.811 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:49.802+0000 7f3bc53f2740 20 client.15462 trim_cache size 0 max 16384 2024-03-21T10:24:49.835 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:49.833+0000 7f3bc53f2740 1 Processor -- start 2024-03-21T10:24:49.835 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:49.833+0000 7f3bc53f2740 1 -- start start 2024-03-21T10:24:49.835 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:49.833+0000 7f3bc53f2740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55bb6cd47d80 0x55bb6cd436d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:24:49.835 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:49.833+0000 7f3bc53f2740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55bb6cd48770 0x55bb6cd596e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:24:49.835 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:49.833+0000 7f3bc53f2740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55bb6cd4d8e0 0x55bb6cd51e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:24:49.835 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:49.833+0000 7f3bc53f2740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bb6cd5e2e0 con 0x55bb6cd47d80 2024-03-21T10:24:49.835 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:49.833+0000 7f3bc53f2740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bb6cd5e940 con 0x55bb6cd4d8e0 2024-03-21T10:24:49.835 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:49.833+0000 7f3bc53f2740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bb6cd5efa0 con 0x55bb6cd48770 2024-03-21T10:24:49.836 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:49.834+0000 7f3bb3e00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55bb6cd48770 0x55bb6cd596e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:24:49.837 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:49.834+0000 7f3bbbe00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55bb6cd4d8e0 0x55bb6cd51e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:24:49.837 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:49.834+0000 7f3bbb400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55bb6cd47d80 0x55bb6cd436d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:24:49.837 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:49.834+0000 7f3bb3e00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55bb6cd48770 0x55bb6cd596e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.47:33196/0 (socket says 172.21.15.47:33196) 2024-03-21T10:24:49.837 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:49.834+0000 7f3bb3e00640 1 -- 172.21.15.47:0/2952338370 learned_addr learned my addr 172.21.15.47:0/2952338370 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:24:49.837 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:49.834+0000 7f3bbb400640 1 -- 172.21.15.47:0/2952338370 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55bb6cd48770 msgr2=0x55bb6cd596e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:24:49.837 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:49.834+0000 7f3bbb400640 1 --2- 172.21.15.47:0/2952338370 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55bb6cd48770 0x55bb6cd596e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:24:49.837 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:49.834+0000 7f3bbb400640 1 -- 172.21.15.47:0/2952338370 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55bb6cd4d8e0 msgr2=0x55bb6cd51e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:24:49.837 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:49.834+0000 7f3bbb400640 1 --2- 172.21.15.47:0/2952338370 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55bb6cd4d8e0 0x55bb6cd51e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:24:49.837 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:49.834+0000 7f3bbb400640 1 -- 172.21.15.47:0/2952338370 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55bb6cd35170 con 0x55bb6cd47d80 2024-03-21T10:24:49.838 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:49.834+0000 7f3bbb400640 1 --2- 172.21.15.47:0/2952338370 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55bb6cd47d80 0x55bb6cd436d0 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x55bb6cd44260 tx=0x7f3bac011e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c7ece7a00506bbc server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:24:49.838 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:49.835+0000 7f3babe00640 1 -- 172.21.15.47:0/2952338370 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3bac031070 con 0x55bb6cd47d80 2024-03-21T10:24:49.838 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:49.835+0000 7f3babe00640 1 -- 172.21.15.47:0/2952338370 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f3bac004e00 con 0x55bb6cd47d80 2024-03-21T10:24:49.838 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:49.835+0000 7f3babe00640 1 -- 172.21.15.47:0/2952338370 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3bac003f40 con 0x55bb6cd47d80 2024-03-21T10:24:49.841 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:49.835+0000 7f3bc53f2740 1 -- 172.21.15.47:0/2952338370 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55bb6cd47d80 msgr2=0x55bb6cd436d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:24:49.842 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:49.835+0000 7f3bc53f2740 1 --2- 172.21.15.47:0/2952338370 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55bb6cd47d80 0x55bb6cd436d0 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x55bb6cd44260 tx=0x7f3bac011e30 comp rx=0 tx=0).stop 2024-03-21T10:24:49.842 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:49.836+0000 7f3bc53f2740 1 -- 172.21.15.47:0/2952338370 shutdown_connections 2024-03-21T10:24:49.842 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:49.836+0000 7f3bc53f2740 1 --2- 172.21.15.47:0/2952338370 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55bb6cd4d8e0 0x55bb6cd51e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:24:49.842 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:49.836+0000 7f3bc53f2740 1 --2- 172.21.15.47:0/2952338370 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55bb6cd48770 0x55bb6cd596e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:24:49.842 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:49.836+0000 7f3bc53f2740 1 --2- 172.21.15.47:0/2952338370 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55bb6cd47d80 0x55bb6cd436d0 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:24:49.842 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:49.836+0000 7f3bc53f2740 1 -- 172.21.15.47:0/2952338370 >> 172.21.15.47:0/2952338370 conn(0x55bb6cd432f0 msgr2=0x55bb6cd5db00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:24:49.842 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:49.836+0000 7f3bc53f2740 1 -- 172.21.15.47:0/2952338370 shutdown_connections 2024-03-21T10:24:49.842 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:49.836+0000 7f3bc53f2740 1 -- 172.21.15.47:0/2952338370 wait complete. 2024-03-21T10:24:49.842 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:49.836+0000 7f3bc53f2740 1 -- start start 2024-03-21T10:24:49.842 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:49.836+0000 7f3bc53f2740 1 Processor -- start 2024-03-21T10:24:49.842 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:49.837+0000 7f3bc53f2740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55bb6cd47d80 0x55bb6cd55630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:24:49.842 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:49.837+0000 7f3bc53f2740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55bb6cd48770 0x55bb6cd55b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:24:49.843 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:49.837+0000 7f3bc53f2740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55bb6cd4d8e0 0x55bb6cd58b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:24:49.843 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:49.837+0000 7f3bc53f2740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bb6cd5fc60 con 0x55bb6cd48770 2024-03-21T10:24:49.843 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:49.837+0000 7f3bc53f2740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bb6cd5f600 con 0x55bb6cd4d8e0 2024-03-21T10:24:49.843 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:49.837+0000 7f3bc53f2740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bb6cd602c0 con 0x55bb6cd47d80 2024-03-21T10:24:49.843 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:49.837+0000 7f3bc53f2740 10 client.0 _ll_register_callbacks cb 0x7f3bb4044050 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T10:24:49.843 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:49.837+0000 7f3bb3e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55bb6cd48770 0x55bb6cd55b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:24:49.843 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:49.837+0000 7f3bb3e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55bb6cd48770 0x55bb6cd55b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:44610/0 (socket says 172.21.15.47:44610) 2024-03-21T10:24:49.843 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:49.837+0000 7f3bb3e00640 1 -- 172.21.15.47:0/1918404938 learned_addr learned my addr 172.21.15.47:0/1918404938 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:24:49.843 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:49.837+0000 7f3bbb400640 1 --2- 172.21.15.47:0/1918404938 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55bb6cd47d80 0x55bb6cd55630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:24:49.843 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:49.837+0000 7f3bbbe00640 1 --2- 172.21.15.47:0/1918404938 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55bb6cd4d8e0 0x55bb6cd58b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:24:49.843 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:49.837+0000 7f3bb3e00640 1 -- 172.21.15.47:0/1918404938 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55bb6cd47d80 msgr2=0x55bb6cd55630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:24:49.843 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:49.837+0000 7f3bb3e00640 1 --2- 172.21.15.47:0/1918404938 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55bb6cd47d80 0x55bb6cd55630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:24:49.843 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:49.837+0000 7f3bb3e00640 1 -- 172.21.15.47:0/1918404938 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55bb6cd4d8e0 msgr2=0x55bb6cd58b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:24:49.843 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:49.837+0000 7f3bb3e00640 1 --2- 172.21.15.47:0/1918404938 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55bb6cd4d8e0 0x55bb6cd58b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:24:49.844 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:49.837+0000 7f3bb3e00640 1 -- 172.21.15.47:0/1918404938 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55bb6cd59180 con 0x55bb6cd48770 2024-03-21T10:24:49.844 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:49.837+0000 7f3bbbe00640 1 --2- 172.21.15.47:0/1918404938 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55bb6cd4d8e0 0x55bb6cd58b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T10:24:49.844 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:49.838+0000 7f3bb3e00640 1 --2- 172.21.15.47:0/1918404938 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55bb6cd48770 0x55bb6cd55b70 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f3ba400e9c0 tx=0x7f3ba409ba10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=35e29e020a5d6d26 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:24:49.844 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:49.838+0000 7f3ba9600640 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-03-21T10:24:49.844 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:49.838+0000 7f3ba9600640 1 -- 172.21.15.47:0/1918404938 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ba40a44e0 con 0x55bb6cd48770 2024-03-21T10:24:49.844 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:49.838+0000 7f3ba9600640 1 -- 172.21.15.47:0/1918404938 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9216+0+0 (secure 0 0 0) 0x7f3ba4010430 con 0x55bb6cd48770 2024-03-21T10:24:49.844 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:49.838+0000 7f3bc53f2740 1 -- 172.21.15.47:0/1918404938 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f3bac004e00 con 0x55bb6cd48770 2024-03-21T10:24:49.844 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:49.838+0000 7f3ba9600640 1 -- 172.21.15.47:0/1918404938 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3ba409be10 con 0x55bb6cd48770 2024-03-21T10:24:49.844 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:49.838+0000 7f3bc53f2740 10 client.15468 fetch_fsmap learned FSMap version 29 2024-03-21T10:24:49.844 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:49.838+0000 7f3bc53f2740 1 -- 172.21.15.47:0/1918404938 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x55bb6cdba270 con 0x55bb6cd48770 2024-03-21T10:24:49.844 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:49.839+0000 7f3ba9600640 1 -- 172.21.15.47:0/1918404938 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f3bac004e00 con 0x55bb6cd48770 2024-03-21T10:24:49.844 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:49.839+0000 7f3ba9600640 1 -- 172.21.15.47:0/1918404938 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 29) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f3ba4097b60 con 0x55bb6cd48770 2024-03-21T10:24:49.844 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:49.839+0000 7f3bc53f2740 10 client.15468 fetch_fsmap finished waiting for FSMap version 29 2024-03-21T10:24:49.844 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:49.839+0000 7f3bc53f2740 10 client.15468 resolve_mds: resolved name 'h' to daemon mds.h 2024-03-21T10:24:49.845 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:49.839+0000 7f3bc53f2740 20 client.15468 populate_metadata read hostname 'smithi047' 2024-03-21T10:24:49.845 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:49.839+0000 7f3bc53f2740 1 --2- 172.21.15.47:0/1918404938 >> [v2:172.21.15.100:6836/3497644162,v1:172.21.15.100:6837/3497644162] conn(0x55bb6cdbe9c0 0x55bb6cdba550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:24:49.845 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:49.839+0000 7f3bc53f2740 4 client.15468 mds_command: new command op to 34493 tid=0 multi_id=0 [{"prefix": "status"}] 2024-03-21T10:24:49.845 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:49.839+0000 7f3bc53f2740 1 -- 172.21.15.47:0/1918404938 --> [v2:172.21.15.100:6836/3497644162,v1:172.21.15.100:6837/3497644162] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55bb6ccba1d0 con 0x55bb6cdbe9c0 2024-03-21T10:24:49.845 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:49.840+0000 7f3bbb400640 1 --2- 172.21.15.47:0/1918404938 >> [v2:172.21.15.100:6836/3497644162,v1:172.21.15.100:6837/3497644162] conn(0x55bb6cdbe9c0 0x55bb6cdba550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:24:49.846 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:49.841+0000 7f3bbb400640 1 --2- 172.21.15.47:0/1918404938 >> [v2:172.21.15.100:6836/3497644162,v1:172.21.15.100:6837/3497644162] conn(0x55bb6cdbe9c0 0x55bb6cdba550 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=1d39dcdc0ebf0cdf server_cookie=bc3123918b6a8ac9 in_seq=0 out_seq=0 2024-03-21T10:24:49.846 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:49.841+0000 7f3ba9600640 10 client.15468 ms_handle_connect on v2:172.21.15.100:6836/3497644162 2024-03-21T10:24:49.846 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:49.841+0000 7f3ba9600640 1 -- 172.21.15.47:0/1918404938 <== mds.1 v2:172.21.15.100:6836/3497644162 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+327 (crc 0 0 0) 0x55bb6ccba1d0 con 0x55bb6cdbe9c0 2024-03-21T10:24:49.846 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:49.841+0000 7f3ba9600640 10 client.15468 handle_command_reply: tid=0 2024-03-21T10:24:49.846 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:49.841+0000 7f3bc53f2740 1 client.15468 shutdown 2024-03-21T10:24:49.846 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:49.841+0000 7f3bc53f2740 1 -- 172.21.15.47:0/1918404938 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55bb6cd48770 msgr2=0x55bb6cd55b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:24:49.847 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:49.841+0000 7f3bc53f2740 1 --2- 172.21.15.47:0/1918404938 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55bb6cd48770 0x55bb6cd55b70 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f3ba400e9c0 tx=0x7f3ba409ba10 comp rx=0 tx=0).stop 2024-03-21T10:24:49.847 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:49.841+0000 7f3bc53f2740 1 -- 172.21.15.47:0/1918404938 shutdown_connections 2024-03-21T10:24:49.847 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:49.841+0000 7f3bc53f2740 1 --2- 172.21.15.47:0/1918404938 >> [v2:172.21.15.100:6836/3497644162,v1:172.21.15.100:6837/3497644162] conn(0x55bb6cdbe9c0 0x55bb6cdba550 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:24:49.847 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:49.841+0000 7f3bc53f2740 1 --2- 172.21.15.47:0/1918404938 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55bb6cd4d8e0 0x55bb6cd58b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:24:49.847 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:49.841+0000 7f3ba9600640 0 client.15468 ms_handle_reset on v2:172.21.15.100:6836/3497644162 2024-03-21T10:24:49.847 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:49.841+0000 7f3bc53f2740 1 --2- 172.21.15.47:0/1918404938 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55bb6cd48770 0x55bb6cd55b70 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:24:49.847 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:49.841+0000 7f3bc53f2740 1 --2- 172.21.15.47:0/1918404938 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55bb6cd47d80 0x55bb6cd55630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:24:49.847 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:49.841+0000 7f3bc53f2740 1 -- 172.21.15.47:0/1918404938 >> 172.21.15.47:0/1918404938 conn(0x55bb6cd432f0 msgr2=0x55bb6cd436d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:24:49.847 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:49.842+0000 7f3bc53f2740 1 -- 172.21.15.47:0/1918404938 shutdown_connections 2024-03-21T10:24:49.847 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:49.842+0000 7f3bc53f2740 1 -- 172.21.15.47:0/1918404938 wait complete. 2024-03-21T10:24:49.847 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:49.842+0000 7f3bc53f2740 20 client.15468 trim_cache size 0 max 16384 2024-03-21T10:24:49.847 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-03-21T10:24:49.848 INFO:teuthology.orchestra.run.smithi047.stdout: "cluster_fsid": "3056d51a-e76b-11ee-95cd-87774f69a715", 2024-03-21T10:24:49.848 INFO:teuthology.orchestra.run.smithi047.stdout: "whoami": 1, 2024-03-21T10:24:49.848 INFO:teuthology.orchestra.run.smithi047.stdout: "id": 34493, 2024-03-21T10:24:49.848 INFO:teuthology.orchestra.run.smithi047.stdout: "want_state": "up:active", 2024-03-21T10:24:49.848 INFO:teuthology.orchestra.run.smithi047.stdout: "state": "up:active", 2024-03-21T10:24:49.848 INFO:teuthology.orchestra.run.smithi047.stdout: "fs_name": "cephfs", 2024-03-21T10:24:49.848 INFO:teuthology.orchestra.run.smithi047.stdout: "rank_uptime": 144.672024456, 2024-03-21T10:24:49.848 INFO:teuthology.orchestra.run.smithi047.stdout: "mdsmap_epoch": 29, 2024-03-21T10:24:49.848 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap_epoch": 95, 2024-03-21T10:24:49.848 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap_epoch_barrier": 93, 2024-03-21T10:24:49.848 INFO:teuthology.orchestra.run.smithi047.stdout: "uptime": 192.26282201999999 2024-03-21T10:24:49.848 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-03-21T10:24:49.849 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:49.842+0000 7f3bc1000640 1 -- 172.21.15.47:0/2850281165 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f3ba00a2fa0 msgr2=0x7f3ba00a5460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:24:49.849 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:49.842+0000 7f3bc1000640 1 --2- 172.21.15.47:0/2850281165 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f3ba00a2fa0 0x7f3ba00a5460 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f3bb4006010 tx=0x7f3bb402c810 comp rx=0 tx=0).stop 2024-03-21T10:24:49.849 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:49.842+0000 7f3bc1000640 1 -- 172.21.15.47:0/2850281165 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3bbc102770 msgr2=0x7f3bbc073a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:24:49.849 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:49.842+0000 7f3bc1000640 1 --2- 172.21.15.47:0/2850281165 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3bbc102770 0x7f3bbc073a90 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f3ba400e9f0 tx=0x7f3ba400eeb0 comp rx=0 tx=0).stop 2024-03-21T10:24:49.849 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:49.843+0000 7f3bc1000640 1 -- 172.21.15.47:0/2850281165 shutdown_connections 2024-03-21T10:24:49.850 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:49.843+0000 7f3bc1000640 1 --2- 172.21.15.47:0/2850281165 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f3ba00a2fa0 0x7f3ba00a5460 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:24:49.850 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:49.843+0000 7f3bc1000640 1 --2- 172.21.15.47:0/2850281165 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3bbc103130 0x7f3bbc073fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:24:49.850 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:49.843+0000 7f3bc1000640 1 --2- 172.21.15.47:0/2850281165 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3bbc102770 0x7f3bbc073a90 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:24:49.850 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:49.843+0000 7f3bc1000640 1 --2- 172.21.15.47:0/2850281165 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3bbc101580 0x7f3bbc075450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:24:49.850 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:49.843+0000 7f3bc1000640 1 -- 172.21.15.47:0/2850281165 >> 172.21.15.47:0/2850281165 conn(0x7f3bbc0fcb10 msgr2=0x7f3bbc105a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:24:49.850 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:49.843+0000 7f3bc1000640 1 -- 172.21.15.47:0/2850281165 shutdown_connections 2024-03-21T10:24:49.850 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:49.843+0000 7f3bc1000640 1 -- 172.21.15.47:0/2850281165 wait complete. 2024-03-21T10:24:49.861 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2024-03-21T10:24:50.064 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:50.062+0000 7fe38d000640 1 -- 172.21.15.47:0/3000171592 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe388109460 msgr2=0x7fe38810b8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:24:50.065 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:50.062+0000 7fe38d000640 1 --2- 172.21.15.47:0/3000171592 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe388109460 0x7fe38810b8f0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fe380009ca0 tx=0x7fe38002c0f0 comp rx=0 tx=0).stop 2024-03-21T10:24:50.065 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:50.062+0000 7fe38d000640 1 -- 172.21.15.47:0/3000171592 shutdown_connections 2024-03-21T10:24:50.065 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:50.062+0000 7fe38d000640 1 --2- 172.21.15.47:0/3000171592 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe388109460 0x7fe38810b8f0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:24:50.065 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:50.062+0000 7fe38d000640 1 --2- 172.21.15.47:0/3000171592 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe388074100 0x7fe388074580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:24:50.065 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:50.062+0000 7fe38d000640 1 --2- 172.21.15.47:0/3000171592 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe3880756e0 0x7fe388073b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:24:50.065 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:50.062+0000 7fe38d000640 1 -- 172.21.15.47:0/3000171592 >> 172.21.15.47:0/3000171592 conn(0x7fe3880fcb10 msgr2=0x7fe3880fef50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:24:50.065 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:50.062+0000 7fe38d000640 1 -- 172.21.15.47:0/3000171592 shutdown_connections 2024-03-21T10:24:50.065 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:50.062+0000 7fe38d000640 1 -- 172.21.15.47:0/3000171592 wait complete. 2024-03-21T10:24:50.066 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:50.063+0000 7fe38d000640 1 Processor -- start 2024-03-21T10:24:50.066 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:50.063+0000 7fe38d000640 1 -- start start 2024-03-21T10:24:50.066 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:50.064+0000 7fe38d000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe388074100 0x7fe388199980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:24:50.066 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:50.064+0000 7fe38d000640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe3880756e0 0x7fe388199ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:24:50.066 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:50.064+0000 7fe38d000640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe388109460 0x7fe3881a0ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:24:50.066 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:50.064+0000 7fe38d000640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe38810f2b0 con 0x7fe388109460 2024-03-21T10:24:50.067 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:50.064+0000 7fe38d000640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe38810f130 con 0x7fe388074100 2024-03-21T10:24:50.067 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:50.064+0000 7fe38d000640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe38810f430 con 0x7fe3880756e0 2024-03-21T10:24:50.067 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:50.064+0000 7fe387400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe388074100 0x7fe388199980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:24:50.067 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:50.064+0000 7fe387400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe388074100 0x7fe388199980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:56600/0 (socket says 172.21.15.47:56600) 2024-03-21T10:24:50.067 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:50.064+0000 7fe387e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe388109460 0x7fe3881a0ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:24:50.067 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:50.064+0000 7fe387400640 1 -- 172.21.15.47:0/2939267742 learned_addr learned my addr 172.21.15.47:0/2939267742 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:24:50.067 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:50.064+0000 7fe386a00640 1 --2- 172.21.15.47:0/2939267742 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe3880756e0 0x7fe388199ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:24:50.068 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:50.065+0000 7fe387400640 1 -- 172.21.15.47:0/2939267742 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe3880756e0 msgr2=0x7fe388199ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:24:50.068 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:50.065+0000 7fe387400640 1 --2- 172.21.15.47:0/2939267742 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe3880756e0 0x7fe388199ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:24:50.068 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:50.065+0000 7fe387400640 1 -- 172.21.15.47:0/2939267742 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe388109460 msgr2=0x7fe3881a0ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:24:50.068 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:50.065+0000 7fe387400640 1 --2- 172.21.15.47:0/2939267742 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe388109460 0x7fe3881a0ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:24:50.068 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:50.065+0000 7fe387400640 1 -- 172.21.15.47:0/2939267742 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3881a1550 con 0x7fe388074100 2024-03-21T10:24:50.069 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:50.065+0000 7fe387400640 1 --2- 172.21.15.47:0/2939267742 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe388074100 0x7fe388199980 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fe37800e9f0 tx=0x7fe37800eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:24:50.069 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:50.065+0000 7fe37fe00640 1 -- 172.21.15.47:0/2939267742 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe37800cd80 con 0x7fe388074100 2024-03-21T10:24:50.069 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:50.066+0000 7fe38d000640 1 -- 172.21.15.47:0/2939267742 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe3881a1840 con 0x7fe388074100 2024-03-21T10:24:50.069 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:50.066+0000 7fe37fe00640 1 -- 172.21.15.47:0/2939267742 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fe3780108e0 con 0x7fe388074100 2024-03-21T10:24:50.069 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:50.066+0000 7fe37fe00640 1 -- 172.21.15.47:0/2939267742 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe378018920 con 0x7fe388074100 2024-03-21T10:24:50.069 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:50.066+0000 7fe38d000640 1 -- 172.21.15.47:0/2939267742 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe3881a1de0 con 0x7fe388074100 2024-03-21T10:24:50.070 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:50.069+0000 7fe3913d4740 1 Processor -- start 2024-03-21T10:24:50.071 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:50.069+0000 7fe3913d4740 1 -- start start 2024-03-21T10:24:50.072 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:50.069+0000 7fe3913d4740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f087dd8d80 0x55f087dd91a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:24:50.072 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:50.069+0000 7fe3913d4740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f087dd9770 0x55f087dde270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:24:50.072 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:50.069+0000 7fe3913d4740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f087dde8e0 0x55f087de0d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:24:50.072 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:50.069+0000 7fe3913d4740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f087d3dc90 con 0x55f087dde8e0 2024-03-21T10:24:50.073 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:50.069+0000 7fe3913d4740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f087ce6980 con 0x55f087dd9770 2024-03-21T10:24:50.073 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:50.069+0000 7fe3913d4740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f087d39d30 con 0x55f087dd8d80 2024-03-21T10:24:50.073 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:50.069+0000 7fe37fe00640 1 -- 172.21.15.47:0/2939267742 <== mon.2 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 18) v1 ==== 137854+0+0 (secure 0 0 0) 0x7fe378010a90 con 0x7fe388074100 2024-03-21T10:24:50.073 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:50.069+0000 7fe386a00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f087dde8e0 0x55f087de0d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:24:50.073 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:50.070+0000 7fe386a00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f087dde8e0 0x55f087de0d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:44624/0 (socket says 172.21.15.47:44624) 2024-03-21T10:24:50.073 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:50.070+0000 7fe386a00640 1 -- 172.21.15.47:0/2496458904 learned_addr learned my addr 172.21.15.47:0/2496458904 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:24:50.073 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:50.070+0000 7fe387e00640 1 --2- 172.21.15.47:0/2496458904 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f087dd8d80 0x55f087dd91a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:24:50.073 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:50.070+0000 7fe387400640 1 --2- 172.21.15.47:0/2496458904 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f087dd9770 0x55f087dde270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:24:50.073 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:50.070+0000 7fe386a00640 1 -- 172.21.15.47:0/2496458904 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f087dd8d80 msgr2=0x55f087dd91a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:24:50.073 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:50.070+0000 7fe386a00640 1 --2- 172.21.15.47:0/2496458904 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f087dd8d80 0x55f087dd91a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:24:50.074 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:50.070+0000 7fe386a00640 1 -- 172.21.15.47:0/2496458904 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f087dd9770 msgr2=0x55f087dde270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:24:50.074 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:50.070+0000 7fe386a00640 1 --2- 172.21.15.47:0/2496458904 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f087dd9770 0x55f087dde270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:24:50.074 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:50.070+0000 7fe386a00640 1 -- 172.21.15.47:0/2496458904 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f087cfe510 con 0x55f087dde8e0 2024-03-21T10:24:50.074 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:50.070+0000 7fe386a00640 1 --2- 172.21.15.47:0/2496458904 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f087dde8e0 0x55f087de0d70 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7fe374009c30 tx=0x7fe3740117b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=17dbfea5f8e6e81c server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:24:50.074 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:50.071+0000 7fe37cc00640 1 -- 172.21.15.47:0/2496458904 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe374012450 con 0x55f087dde8e0 2024-03-21T10:24:50.074 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:50.071+0000 7fe37cc00640 1 -- 172.21.15.47:0/2496458904 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fe374012aa0 con 0x55f087dde8e0 2024-03-21T10:24:50.074 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:50.071+0000 7fe37cc00640 1 -- 172.21.15.47:0/2496458904 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe37400fae0 con 0x55f087dde8e0 2024-03-21T10:24:50.077 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:50.071+0000 7fe37fe00640 1 --2- 172.21.15.47:0/2939267742 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7fe36c0a2f20 0x7fe36c0a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:24:50.077 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:50.071+0000 7fe37fe00640 1 -- 172.21.15.47:0/2939267742 <== mon.2 v2:172.21.15.100:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9216+0+0 (secure 0 0 0) 0x7fe3780cfb40 con 0x7fe388074100 2024-03-21T10:24:50.077 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:50.071+0000 7fe3913d4740 1 -- 172.21.15.47:0/2496458904 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f087dde8e0 msgr2=0x55f087de0d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:24:50.077 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:50.071+0000 7fe3913d4740 1 --2- 172.21.15.47:0/2496458904 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f087dde8e0 0x55f087de0d70 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7fe374009c30 tx=0x7fe3740117b0 comp rx=0 tx=0).stop 2024-03-21T10:24:50.077 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:50.071+0000 7fe387e00640 1 --2- 172.21.15.47:0/2939267742 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7fe36c0a2f20 0x7fe36c0a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:24:50.077 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:50.072+0000 7fe3913d4740 1 -- 172.21.15.47:0/2496458904 shutdown_connections 2024-03-21T10:24:50.077 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:50.072+0000 7fe3913d4740 1 --2- 172.21.15.47:0/2496458904 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f087dde8e0 0x55f087de0d70 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:24:50.077 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:50.072+0000 7fe3913d4740 1 --2- 172.21.15.47:0/2496458904 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f087dd9770 0x55f087dde270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:24:50.078 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:50.072+0000 7fe3913d4740 1 --2- 172.21.15.47:0/2496458904 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f087dd8d80 0x55f087dd91a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:24:50.078 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:50.072+0000 7fe3913d4740 1 -- 172.21.15.47:0/2496458904 >> 172.21.15.47:0/2496458904 conn(0x55f087dd42f0 msgr2=0x55f087dd46f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:24:50.078 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:50.072+0000 7fe3913d4740 1 -- 172.21.15.47:0/2496458904 shutdown_connections 2024-03-21T10:24:50.078 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:50.072+0000 7fe3913d4740 1 -- 172.21.15.47:0/2496458904 wait complete. 2024-03-21T10:24:50.078 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:50.072+0000 7fe3913d4740 1 -- start start 2024-03-21T10:24:50.078 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:50.072+0000 7fe387e00640 1 --2- 172.21.15.47:0/2939267742 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7fe36c0a2f20 0x7fe36c0a53e0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fe380006010 tx=0x7fe38002e040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:24:50.078 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:50.072+0000 7fe3913d4740 1 Processor -- start 2024-03-21T10:24:50.078 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:50.073+0000 7fe3913d4740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f087dd8d80 0x55f087de3260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:24:50.078 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:50.073+0000 7fe3913d4740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f087dd9770 0x55f087de37a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:24:50.078 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:50.073+0000 7fe3913d4740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f087dde8e0 0x55f087dea7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:24:50.078 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:50.073+0000 7fe3913d4740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f087de1890 con 0x55f087dd9770 2024-03-21T10:24:50.079 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:50.073+0000 7fe3913d4740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f087cf9dd0 con 0x55f087dde8e0 2024-03-21T10:24:50.079 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:50.073+0000 7fe3913d4740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f087de1a10 con 0x55f087dd8d80 2024-03-21T10:24:50.079 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:50.073+0000 7fe3913d4740 10 client.0 _ll_register_callbacks cb 0x55f087a51ac0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T10:24:50.079 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:50.073+0000 7fe386a00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f087dd9770 0x55f087de37a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:24:50.079 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:50.073+0000 7fe386a00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f087dd9770 0x55f087de37a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:44634/0 (socket says 172.21.15.47:44634) 2024-03-21T10:24:50.079 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:50.073+0000 7fe386a00640 1 -- 172.21.15.47:0/2034797254 learned_addr learned my addr 172.21.15.47:0/2034797254 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:24:50.079 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:50.074+0000 7fe387400640 1 --2- 172.21.15.47:0/2034797254 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f087dd8d80 0x55f087de3260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:24:50.079 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:50.074+0000 7fe387e00640 1 --2- 172.21.15.47:0/2034797254 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f087dde8e0 0x55f087dea7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:24:50.079 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:50.074+0000 7fe386a00640 1 -- 172.21.15.47:0/2034797254 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f087dd8d80 msgr2=0x55f087de3260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:24:50.079 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:50.074+0000 7fe386a00640 1 --2- 172.21.15.47:0/2034797254 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f087dd8d80 0x55f087de3260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:24:50.079 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:50.074+0000 7fe386a00640 1 -- 172.21.15.47:0/2034797254 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f087dde8e0 msgr2=0x55f087dea7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:24:50.079 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:50.074+0000 7fe386a00640 1 --2- 172.21.15.47:0/2034797254 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f087dde8e0 0x55f087dea7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:24:50.079 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:50.074+0000 7fe386a00640 1 -- 172.21.15.47:0/2034797254 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55f087a522d0 con 0x55f087dd9770 2024-03-21T10:24:50.080 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:50.074+0000 7fe387e00640 1 --2- 172.21.15.47:0/2034797254 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f087dde8e0 0x55f087dea7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T10:24:50.080 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:50.074+0000 7fe386a00640 1 --2- 172.21.15.47:0/2034797254 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f087dd9770 0x55f087de37a0 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7fe374009c00 tx=0x7fe374012e60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=add921dadd1999d2 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:24:50.080 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:50.074+0000 7fe371600640 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-03-21T10:24:50.080 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:50.074+0000 7fe371600640 1 -- 172.21.15.47:0/2034797254 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe374028070 con 0x55f087dd9770 2024-03-21T10:24:50.080 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:50.075+0000 7fe371600640 1 -- 172.21.15.47:0/2034797254 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9216+0+0 (secure 0 0 0) 0x7fe37400f820 con 0x55f087dd9770 2024-03-21T10:24:50.080 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:50.075+0000 7fe3913d4740 1 -- 172.21.15.47:0/2034797254 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fe374012aa0 con 0x55f087dd9770 2024-03-21T10:24:50.080 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:50.075+0000 7fe371600640 1 -- 172.21.15.47:0/2034797254 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe374034050 con 0x55f087dd9770 2024-03-21T10:24:50.080 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:50.075+0000 7fe3913d4740 10 client.15480 fetch_fsmap learned FSMap version 29 2024-03-21T10:24:50.080 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:50.075+0000 7fe3913d4740 1 -- 172.21.15.47:0/2034797254 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x55f087b1c6b0 con 0x55f087dd9770 2024-03-21T10:24:50.080 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:50.075+0000 7fe371600640 1 -- 172.21.15.47:0/2034797254 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fe374012aa0 con 0x55f087dd9770 2024-03-21T10:24:50.082 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:50.076+0000 7fe371600640 1 -- 172.21.15.47:0/2034797254 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 29) v1 ==== 7942+0+0 (secure 0 0 0) 0x7fe3740046c0 con 0x55f087dd9770 2024-03-21T10:24:50.082 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:50.076+0000 7fe3913d4740 10 client.15480 fetch_fsmap finished waiting for FSMap version 29 2024-03-21T10:24:50.082 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:50.076+0000 7fe3913d4740 10 client.15480 resolve_mds: resolved name 'b' to daemon mds.b 2024-03-21T10:24:50.082 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:50.076+0000 7fe3913d4740 20 client.15480 populate_metadata read hostname 'smithi047' 2024-03-21T10:24:50.082 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:50.076+0000 7fe3913d4740 1 --2- 172.21.15.47:0/2034797254 >> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] conn(0x55f087deea00 0x55f087df3a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:24:50.082 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:50.076+0000 7fe3913d4740 4 client.15480 mds_command: new command op to 34535 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T10:24:50.082 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:50.076+0000 7fe3913d4740 1 -- 172.21.15.47:0/2034797254 --> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55f087cdf520 con 0x55f087deea00 2024-03-21T10:24:50.082 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:50.076+0000 7fe387400640 1 --2- 172.21.15.47:0/2034797254 >> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] conn(0x55f087deea00 0x55f087df3a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:24:50.082 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:50.077+0000 7fe387400640 1 --2- 172.21.15.47:0/2034797254 >> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] conn(0x55f087deea00 0x55f087df3a90 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d64aaae3270b3c90 server_cookie=6c9c481ade146075 in_seq=0 out_seq=0 2024-03-21T10:24:50.082 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:50.077+0000 7fe371600640 10 client.15480 ms_handle_connect on v2:172.21.15.100:6838/4173168440 2024-03-21T10:24:50.083 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:50.078+0000 7fe371600640 1 -- 172.21.15.47:0/2034797254 <== mds.0 v2:172.21.15.100:6838/4173168440 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55f087cdf520 con 0x55f087deea00 2024-03-21T10:24:50.083 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:50.078+0000 7fe371600640 10 client.15480 handle_command_reply: tid=0 2024-03-21T10:24:50.083 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:50.078+0000 7fe3913d4740 1 client.15480 shutdown 2024-03-21T10:24:50.083 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:50.078+0000 7fe3913d4740 1 -- 172.21.15.47:0/2034797254 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f087dd9770 msgr2=0x55f087de37a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:24:50.083 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:50.078+0000 7fe3913d4740 1 --2- 172.21.15.47:0/2034797254 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f087dd9770 0x55f087de37a0 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7fe374009c00 tx=0x7fe374012e60 comp rx=0 tx=0).stop 2024-03-21T10:24:50.083 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:50.079+0000 7fe3913d4740 1 -- 172.21.15.47:0/2034797254 shutdown_connections 2024-03-21T10:24:50.083 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:50.079+0000 7fe3913d4740 1 --2- 172.21.15.47:0/2034797254 >> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] conn(0x55f087deea00 0x55f087df3a90 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:24:50.083 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:50.079+0000 7fe3913d4740 1 --2- 172.21.15.47:0/2034797254 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f087dde8e0 0x55f087dea7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:24:50.083 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:50.079+0000 7fe371600640 0 client.15480 ms_handle_reset on v2:172.21.15.100:6838/4173168440 2024-03-21T10:24:50.083 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:50.079+0000 7fe3913d4740 1 --2- 172.21.15.47:0/2034797254 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f087dd9770 0x55f087de37a0 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:24:50.083 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:50.079+0000 7fe3913d4740 1 --2- 172.21.15.47:0/2034797254 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f087dd8d80 0x55f087de3260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:24:50.083 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:50.079+0000 7fe3913d4740 1 -- 172.21.15.47:0/2034797254 >> 172.21.15.47:0/2034797254 conn(0x55f087dd42f0 msgr2=0x55f087dd34f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:24:50.083 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:50.079+0000 7fe3913d4740 1 -- 172.21.15.47:0/2034797254 shutdown_connections 2024-03-21T10:24:50.083 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:50.079+0000 7fe3913d4740 1 -- 172.21.15.47:0/2034797254 wait complete. 2024-03-21T10:24:50.084 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:50.079+0000 7fe3913d4740 20 client.15480 trim_cache size 0 max 16384 2024-03-21T10:24:50.100 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:50.097+0000 7fe3913d4740 1 Processor -- start 2024-03-21T10:24:50.100 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:50.098+0000 7fe3913d4740 1 -- start start 2024-03-21T10:24:50.101 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:50.098+0000 7fe3913d4740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f087dd8d80 0x55f087de0cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:24:50.101 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:50.098+0000 7fe3913d4740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f087dd9770 0x55f087dda210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:24:50.101 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:50.098+0000 7fe3913d4740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f087dde8e0 0x55f087dd46d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:24:50.101 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:50.098+0000 7fe3913d4740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f087deee30 con 0x55f087dd9770 2024-03-21T10:24:50.101 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:50.098+0000 7fe3913d4740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f087def490 con 0x55f087dd8d80 2024-03-21T10:24:50.101 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:50.098+0000 7fe3913d4740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f087defaf0 con 0x55f087dde8e0 2024-03-21T10:24:50.101 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:50.098+0000 7fe386a00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f087dd9770 0x55f087dda210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:24:50.101 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:50.098+0000 7fe386a00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f087dd9770 0x55f087dda210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:44646/0 (socket says 172.21.15.47:44646) 2024-03-21T10:24:50.101 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:50.098+0000 7fe387400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f087dd8d80 0x55f087de0cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:24:50.101 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:50.098+0000 7fe386a00640 1 -- 172.21.15.47:0/2430445441 learned_addr learned my addr 172.21.15.47:0/2430445441 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:24:50.101 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:50.098+0000 7fe387e00640 1 --2- 172.21.15.47:0/2430445441 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f087dde8e0 0x55f087dd46d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:24:50.101 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:50.099+0000 7fe386a00640 1 -- 172.21.15.47:0/2430445441 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f087dde8e0 msgr2=0x55f087dd46d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:24:50.101 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:50.099+0000 7fe386a00640 1 --2- 172.21.15.47:0/2430445441 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f087dde8e0 0x55f087dd46d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:24:50.102 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:50.099+0000 7fe386a00640 1 -- 172.21.15.47:0/2430445441 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f087dd8d80 msgr2=0x55f087de0cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:24:50.102 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:50.099+0000 7fe386a00640 1 --2- 172.21.15.47:0/2430445441 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f087dd8d80 0x55f087de0cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:24:50.102 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:50.099+0000 7fe386a00640 1 -- 172.21.15.47:0/2430445441 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f087dd5060 con 0x55f087dd9770 2024-03-21T10:24:50.113 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:50.099+0000 7fe386a00640 1 --2- 172.21.15.47:0/2430445441 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f087dd9770 0x55f087dda210 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7fe374009c30 tx=0x7fe3740098e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4dc52af15ff954ae server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:24:50.114 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:50.099+0000 7fe373e00640 1 -- 172.21.15.47:0/2430445441 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe37403d070 con 0x55f087dd9770 2024-03-21T10:24:50.114 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:50.099+0000 7fe373e00640 1 -- 172.21.15.47:0/2430445441 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fe374004d80 con 0x55f087dd9770 2024-03-21T10:24:50.114 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:50.099+0000 7fe373e00640 1 -- 172.21.15.47:0/2430445441 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe374028040 con 0x55f087dd9770 2024-03-21T10:24:50.114 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:50.100+0000 7fe3913d4740 1 -- 172.21.15.47:0/2430445441 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f087dd9770 msgr2=0x55f087dda210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:24:50.114 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:50.100+0000 7fe3913d4740 1 --2- 172.21.15.47:0/2430445441 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f087dd9770 0x55f087dda210 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7fe374009c30 tx=0x7fe3740098e0 comp rx=0 tx=0).stop 2024-03-21T10:24:50.114 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:50.100+0000 7fe3913d4740 1 -- 172.21.15.47:0/2430445441 shutdown_connections 2024-03-21T10:24:50.115 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:50.100+0000 7fe3913d4740 1 --2- 172.21.15.47:0/2430445441 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f087dde8e0 0x55f087dd46d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:24:50.115 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:50.100+0000 7fe3913d4740 1 --2- 172.21.15.47:0/2430445441 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f087dd9770 0x55f087dda210 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:24:50.115 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:50.100+0000 7fe3913d4740 1 --2- 172.21.15.47:0/2430445441 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f087dd8d80 0x55f087de0cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:24:50.115 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:50.100+0000 7fe3913d4740 1 -- 172.21.15.47:0/2430445441 >> 172.21.15.47:0/2430445441 conn(0x55f087dd42f0 msgr2=0x55f087de1db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:24:50.115 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:50.100+0000 7fe3913d4740 1 -- 172.21.15.47:0/2430445441 shutdown_connections 2024-03-21T10:24:50.115 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:50.100+0000 7fe3913d4740 1 -- 172.21.15.47:0/2430445441 wait complete. 2024-03-21T10:24:50.116 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:50.101+0000 7fe3913d4740 1 -- start start 2024-03-21T10:24:50.116 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:50.101+0000 7fe3913d4740 1 Processor -- start 2024-03-21T10:24:50.116 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:50.101+0000 7fe3913d4740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f087dd8d80 0x55f087de5800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:24:50.116 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:50.101+0000 7fe3913d4740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f087dd9770 0x55f087de5d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:24:50.116 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:50.101+0000 7fe3913d4740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f087dde8e0 0x55f087de8d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:24:50.116 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:50.101+0000 7fe3913d4740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f087df07b0 con 0x55f087dd8d80 2024-03-21T10:24:50.116 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:50.101+0000 7fe3913d4740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f087df0150 con 0x55f087dd9770 2024-03-21T10:24:50.116 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:50.101+0000 7fe3913d4740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f087df0e10 con 0x55f087dde8e0 2024-03-21T10:24:50.117 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:50.101+0000 7fe3913d4740 10 client.0 _ll_register_callbacks cb 0x7fe374009a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T10:24:50.117 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:50.102+0000 7fe387400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f087dd8d80 0x55f087de5800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:24:50.117 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:50.102+0000 7fe386a00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f087dd9770 0x55f087de5d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:24:50.117 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:50.102+0000 7fe387400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f087dd8d80 0x55f087de5800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:44650/0 (socket says 172.21.15.47:44650) 2024-03-21T10:24:50.117 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:50.102+0000 7fe387400640 1 -- 172.21.15.47:0/28011730 learned_addr learned my addr 172.21.15.47:0/28011730 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:24:50.117 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:50.102+0000 7fe387e00640 1 --2- 172.21.15.47:0/28011730 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f087dde8e0 0x55f087de8d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:24:50.117 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:50.102+0000 7fe386a00640 1 -- 172.21.15.47:0/28011730 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f087dde8e0 msgr2=0x55f087de8d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:24:50.118 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:50.102+0000 7fe386a00640 1 --2- 172.21.15.47:0/28011730 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f087dde8e0 0x55f087de8d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:24:50.118 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:50.102+0000 7fe386a00640 1 -- 172.21.15.47:0/28011730 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f087dd8d80 msgr2=0x55f087de5800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:24:50.118 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:50.102+0000 7fe386a00640 1 --2- 172.21.15.47:0/28011730 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f087dd8d80 0x55f087de5800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:24:50.118 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:50.102+0000 7fe386a00640 1 -- 172.21.15.47:0/28011730 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55f087de9350 con 0x55f087dd9770 2024-03-21T10:24:50.118 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:50.102+0000 7fe386a00640 1 --2- 172.21.15.47:0/28011730 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f087dd9770 0x55f087de5d40 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x55f087de4400 tx=0x7fe374009760 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5c04ec591e61d24 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:24:50.118 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:50.102+0000 7fe371600640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-21T10:24:50.118 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:50.103+0000 7fe371600640 1 -- 172.21.15.47:0/28011730 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe37403d030 con 0x55f087dd9770 2024-03-21T10:24:50.119 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:50.103+0000 7fe3913d4740 1 -- 172.21.15.47:0/28011730 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fe374004d80 con 0x55f087dd9770 2024-03-21T10:24:50.119 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:50.103+0000 7fe371600640 1 -- 172.21.15.47:0/28011730 <== mon.2 v2:172.21.15.100:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9216+0+0 (secure 0 0 0) 0x7fe374012970 con 0x55f087dd9770 2024-03-21T10:24:50.119 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:50.103+0000 7fe371600640 1 -- 172.21.15.47:0/28011730 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe37404a050 con 0x55f087dd9770 2024-03-21T10:24:50.119 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:50.103+0000 7fe3913d4740 10 client.35318 fetch_fsmap learned FSMap version 29 2024-03-21T10:24:50.119 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:50.103+0000 7fe3913d4740 1 -- 172.21.15.47:0/28011730 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x55f087e4ad00 con 0x55f087dd9770 2024-03-21T10:24:50.119 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:50.103+0000 7fe371600640 1 -- 172.21.15.47:0/28011730 <== mon.2 v2:172.21.15.100:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fe374004d80 con 0x55f087dd9770 2024-03-21T10:24:50.119 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:50.104+0000 7fe371600640 1 -- 172.21.15.47:0/28011730 <== mon.2 v2:172.21.15.100:3300/0 5 ==== fsmap(e 29) v1 ==== 7942+0+0 (secure 0 0 0) 0x7fe374004530 con 0x55f087dd9770 2024-03-21T10:24:50.120 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:50.104+0000 7fe3913d4740 10 client.35318 fetch_fsmap finished waiting for FSMap version 29 2024-03-21T10:24:50.120 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:50.104+0000 7fe3913d4740 10 client.35318 resolve_mds: resolved name 'b' to daemon mds.b 2024-03-21T10:24:50.120 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:50.104+0000 7fe3913d4740 20 client.35318 populate_metadata read hostname 'smithi047' 2024-03-21T10:24:50.120 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:50.104+0000 7fe3913d4740 1 --2- 172.21.15.47:0/28011730 >> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] conn(0x55f087e4f030 0x55f087e4af80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:24:50.120 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:50.104+0000 7fe3913d4740 4 client.35318 mds_command: new command op to 34535 tid=0 multi_id=0 [{"prefix": "status"}] 2024-03-21T10:24:50.120 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:50.104+0000 7fe3913d4740 1 -- 172.21.15.47:0/28011730 --> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55f087d4b1d0 con 0x55f087e4f030 2024-03-21T10:24:50.120 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:50.104+0000 7fe387400640 1 --2- 172.21.15.47:0/28011730 >> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] conn(0x55f087e4f030 0x55f087e4af80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:24:50.120 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:50.105+0000 7fe387400640 1 --2- 172.21.15.47:0/28011730 >> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] conn(0x55f087e4f030 0x55f087e4af80 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=322e360da7053f2c server_cookie=4014832f051040ea in_seq=0 out_seq=0 2024-03-21T10:24:50.121 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:50.105+0000 7fe371600640 10 client.35318 ms_handle_connect on v2:172.21.15.100:6838/4173168440 2024-03-21T10:24:50.121 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:50.105+0000 7fe371600640 1 -- 172.21.15.47:0/28011730 <== mds.0 v2:172.21.15.100:6838/4173168440 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+332 (crc 0 0 0) 0x55f087d4b1d0 con 0x55f087e4f030 2024-03-21T10:24:50.121 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:50.105+0000 7fe371600640 10 client.35318 handle_command_reply: tid=0 2024-03-21T10:24:50.121 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:50.105+0000 7fe3913d4740 1 client.35318 shutdown 2024-03-21T10:24:50.121 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:50.105+0000 7fe3913d4740 1 -- 172.21.15.47:0/28011730 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f087dd9770 msgr2=0x55f087de5d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:24:50.121 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:50.105+0000 7fe3913d4740 1 --2- 172.21.15.47:0/28011730 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f087dd9770 0x55f087de5d40 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x55f087de4400 tx=0x7fe374009760 comp rx=0 tx=0).stop 2024-03-21T10:24:50.121 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:50.106+0000 7fe3913d4740 1 -- 172.21.15.47:0/28011730 shutdown_connections 2024-03-21T10:24:50.121 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:50.106+0000 7fe3913d4740 1 --2- 172.21.15.47:0/28011730 >> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] conn(0x55f087e4f030 0x55f087e4af80 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:24:50.122 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:50.106+0000 7fe3913d4740 1 --2- 172.21.15.47:0/28011730 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f087dde8e0 0x55f087de8d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:24:50.122 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:50.106+0000 7fe3913d4740 1 --2- 172.21.15.47:0/28011730 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f087dd9770 0x55f087de5d40 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:24:50.122 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:50.106+0000 7fe3913d4740 1 --2- 172.21.15.47:0/28011730 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f087dd8d80 0x55f087de5800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:24:50.122 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:50.106+0000 7fe3913d4740 1 -- 172.21.15.47:0/28011730 >> 172.21.15.47:0/28011730 conn(0x55f087dd42f0 msgr2=0x55f087dd46d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:24:50.122 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:50.106+0000 7fe371600640 0 client.35318 ms_handle_reset on v2:172.21.15.100:6838/4173168440 2024-03-21T10:24:50.122 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:50.106+0000 7fe3913d4740 1 -- 172.21.15.47:0/28011730 shutdown_connections 2024-03-21T10:24:50.122 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:50.106+0000 7fe3913d4740 1 -- 172.21.15.47:0/28011730 wait complete. 2024-03-21T10:24:50.123 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:50.106+0000 7fe3913d4740 20 client.35318 trim_cache size 0 max 16384 2024-03-21T10:24:50.123 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-03-21T10:24:50.123 INFO:teuthology.orchestra.run.smithi047.stdout: "cluster_fsid": "3056d51a-e76b-11ee-95cd-87774f69a715", 2024-03-21T10:24:50.123 INFO:teuthology.orchestra.run.smithi047.stdout: "whoami": 0, 2024-03-21T10:24:50.123 INFO:teuthology.orchestra.run.smithi047.stdout: "id": 34535, 2024-03-21T10:24:50.124 INFO:teuthology.orchestra.run.smithi047.stdout: "want_state": "up:active", 2024-03-21T10:24:50.124 INFO:teuthology.orchestra.run.smithi047.stdout: "state": "up:active", 2024-03-21T10:24:50.124 INFO:teuthology.orchestra.run.smithi047.stdout: "fs_name": "cephfs", 2024-03-21T10:24:50.124 INFO:teuthology.orchestra.run.smithi047.stdout: "rank_uptime": 170.84990610099999, 2024-03-21T10:24:50.124 INFO:teuthology.orchestra.run.smithi047.stdout: "mdsmap_epoch": 29, 2024-03-21T10:24:50.124 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap_epoch": 95, 2024-03-21T10:24:50.124 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap_epoch_barrier": 93, 2024-03-21T10:24:50.124 INFO:teuthology.orchestra.run.smithi047.stdout: "uptime": 176.51169727300001 2024-03-21T10:24:50.125 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-03-21T10:24:50.126 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:50.106+0000 7fe38d000640 1 -- 172.21.15.47:0/2939267742 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7fe36c0a2f20 msgr2=0x7fe36c0a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:24:50.127 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:50.106+0000 7fe38d000640 1 --2- 172.21.15.47:0/2939267742 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7fe36c0a2f20 0x7fe36c0a53e0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fe380006010 tx=0x7fe38002e040 comp rx=0 tx=0).stop 2024-03-21T10:24:50.127 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:50.107+0000 7fe38d000640 1 -- 172.21.15.47:0/2939267742 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe388074100 msgr2=0x7fe388199980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:24:50.127 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:50.107+0000 7fe38d000640 1 --2- 172.21.15.47:0/2939267742 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe388074100 0x7fe388199980 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fe37800e9f0 tx=0x7fe37800eeb0 comp rx=0 tx=0).stop 2024-03-21T10:24:50.127 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:50.107+0000 7fe38d000640 1 -- 172.21.15.47:0/2939267742 shutdown_connections 2024-03-21T10:24:50.127 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:50.107+0000 7fe38d000640 1 --2- 172.21.15.47:0/2939267742 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7fe36c0a2f20 0x7fe36c0a53e0 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:24:50.127 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:50.107+0000 7fe38d000640 1 --2- 172.21.15.47:0/2939267742 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe388109460 0x7fe3881a0ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:24:50.127 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:50.107+0000 7fe38d000640 1 --2- 172.21.15.47:0/2939267742 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe3880756e0 0x7fe388199ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:24:50.128 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:50.107+0000 7fe38d000640 1 --2- 172.21.15.47:0/2939267742 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe388074100 0x7fe388199980 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:24:50.128 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:50.107+0000 7fe38d000640 1 -- 172.21.15.47:0/2939267742 >> 172.21.15.47:0/2939267742 conn(0x7fe3880fcb10 msgr2=0x7fe388105360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:24:50.128 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:50.107+0000 7fe38d000640 1 -- 172.21.15.47:0/2939267742 shutdown_connections 2024-03-21T10:24:50.128 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:50.107+0000 7fe38d000640 1 -- 172.21.15.47:0/2939267742 wait complete. 2024-03-21T10:24:50.128 INFO:teuthology.run_tasks:Running task ceph-fuse... 2024-03-21T10:24:50.141 INFO:tasks.ceph_fuse:Running ceph_fuse task... 2024-03-21T10:24:50.141 INFO:tasks.ceph_fuse:config is {'client.0': {}} 2024-03-21T10:24:50.141 INFO:tasks.ceph_fuse:client.0 config is {'mount_subvol_num': 0} 2024-03-21T10:24:50.142 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-03-21T10:24:50.142 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-03-21T10:24:50.142 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-21T10:24:50.142 DEBUG:teuthology.orchestra.run.smithi047:> ip netns list 2024-03-21T10:24:50.194 INFO:teuthology.orchestra.run.smithi047.stdout:ceph-ns--home-ubuntu-cephtest-mnt.admin (id: 0) 2024-03-21T10:24:50.195 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-21T10:24:50.195 DEBUG:teuthology.orchestra.run.smithi047:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-03-21T10:24:50.278 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-21T10:24:50.278 DEBUG:teuthology.orchestra.run.smithi047:> sudo ip link delete ceph-brx 2024-03-21T10:24:50.402 INFO:tasks.ceph_fuse:Mounting ceph-fuse clients... 2024-03-21T10:24:50.403 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2edfee68ad7243bed422a049b6f5f58619300f11 shell --fsid 3056d51a-e76b-11ee-95cd-87774f69a715 -- ceph fs ls 2024-03-21T10:24:50.874 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/3056d51a-e76b-11ee-95cd-87774f69a715/mon.a/config 2024-03-21T10:24:51.282 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:24:51 smithi047 ceph-mon[31547]: pgmap v457: 129 pgs: 129 active+clean; 642 KiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-03-21T10:24:51.479 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:24:51 smithi100 ceph-mon[35193]: pgmap v457: 129 pgs: 129 active+clean; 642 KiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-03-21T10:24:51.740 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:24:51 smithi157 ceph-mon[35581]: pgmap v457: 129 pgs: 129 active+clean; 642 KiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-03-21T10:24:53.337 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:53.334+0000 7fef7f000700 1 -- 172.21.15.47:0/1864122262 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fef7806a7e0 msgr2=0x7fef7810f020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:24:53.337 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:53.334+0000 7fef7f000700 1 --2- 172.21.15.47:0/1864122262 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fef7806a7e0 0x7fef7810f020 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7fef7000b0d0 tx=0x7fef7000b3e0 comp rx=0 tx=0).stop 2024-03-21T10:24:53.337 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:53.334+0000 7fef7f000700 1 -- 172.21.15.47:0/1864122262 shutdown_connections 2024-03-21T10:24:53.337 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:53.334+0000 7fef7f000700 1 --2- 172.21.15.47:0/1864122262 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fef7806a7e0 0x7fef7810f020 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:24:53.337 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:53.334+0000 7fef7f000700 1 --2- 172.21.15.47:0/1864122262 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fef781081e0 0x7fef781085c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:24:53.337 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:53.334+0000 7fef7f000700 1 --2- 172.21.15.47:0/1864122262 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fef78069e40 0x7fef7806a2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:24:53.338 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:53.334+0000 7fef7f000700 1 -- 172.21.15.47:0/1864122262 >> 172.21.15.47:0/1864122262 conn(0x7fef78077fc0 msgr2=0x7fef780783d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:24:53.338 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:53.335+0000 7fef7f000700 1 -- 172.21.15.47:0/1864122262 shutdown_connections 2024-03-21T10:24:53.338 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:53.335+0000 7fef7f000700 1 -- 172.21.15.47:0/1864122262 wait complete. 2024-03-21T10:24:53.338 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:53.336+0000 7fef7f000700 1 Processor -- start 2024-03-21T10:24:53.338 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:53.336+0000 7fef7f000700 1 -- start start 2024-03-21T10:24:53.339 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:53.337+0000 7fef7f000700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fef78069e40 0x7fef7819e450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:24:53.339 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:53.337+0000 7fef7f000700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fef7806a7e0 0x7fef7819e990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:24:53.339 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:53.337+0000 7fef7f000700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fef781081e0 0x7fef78198610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:24:53.340 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:53.337+0000 7fef7f000700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef78112330 con 0x7fef78069e40 2024-03-21T10:24:53.340 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:53.337+0000 7fef7f000700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef781121b0 con 0x7fef781081e0 2024-03-21T10:24:53.340 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:53.337+0000 7fef7f000700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef781124b0 con 0x7fef7806a7e0 2024-03-21T10:24:53.340 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:53.337+0000 7fef77e00700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fef78069e40 0x7fef7819e450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:24:53.340 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:53.337+0000 7fef77e00700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fef78069e40 0x7fef7819e450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:34390/0 (socket says 172.21.15.47:34390) 2024-03-21T10:24:53.340 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:53.337+0000 7fef77e00700 1 -- 172.21.15.47:0/3721063798 learned_addr learned my addr 172.21.15.47:0/3721063798 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:24:53.340 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:53.337+0000 7fef6fe00700 1 --2- 172.21.15.47:0/3721063798 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fef7806a7e0 0x7fef7819e990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:24:53.341 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:53.337+0000 7fef7d000700 1 --2- 172.21.15.47:0/3721063798 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fef781081e0 0x7fef78198610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:24:53.341 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:53.338+0000 7fef77e00700 1 -- 172.21.15.47:0/3721063798 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fef7806a7e0 msgr2=0x7fef7819e990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:24:53.341 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:53.338+0000 7fef77e00700 1 --2- 172.21.15.47:0/3721063798 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fef7806a7e0 0x7fef7819e990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:24:53.341 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:53.338+0000 7fef77e00700 1 -- 172.21.15.47:0/3721063798 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fef781081e0 msgr2=0x7fef78198610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:24:53.341 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:53.338+0000 7fef77e00700 1 --2- 172.21.15.47:0/3721063798 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fef781081e0 0x7fef78198610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:24:53.341 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:53.338+0000 7fef77e00700 1 -- 172.21.15.47:0/3721063798 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef78198d70 con 0x7fef78069e40 2024-03-21T10:24:53.341 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:53.338+0000 7fef77e00700 1 --2- 172.21.15.47:0/3721063798 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fef78069e40 0x7fef7819e450 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7fef6800d950 tx=0x7fef6800dd10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:24:53.341 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:53.338+0000 7fef75600700 1 -- 172.21.15.47:0/3721063798 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef68016070 con 0x7fef78069e40 2024-03-21T10:24:53.341 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:53.338+0000 7fef75600700 1 -- 172.21.15.47:0/3721063798 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fef68010460 con 0x7fef78069e40 2024-03-21T10:24:53.341 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:53.338+0000 7fef7f000700 1 -- 172.21.15.47:0/3721063798 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fef78199060 con 0x7fef78069e40 2024-03-21T10:24:53.342 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:53.339+0000 7fef75600700 1 -- 172.21.15.47:0/3721063798 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef6800f620 con 0x7fef78069e40 2024-03-21T10:24:53.342 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:53.339+0000 7fef7f000700 1 -- 172.21.15.47:0/3721063798 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fef781a5320 con 0x7fef78069e40 2024-03-21T10:24:53.342 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:53.340+0000 7fef7f000700 1 -- 172.21.15.47:0/3721063798 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fef78068e50 con 0x7fef78069e40 2024-03-21T10:24:53.344 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:53.342+0000 7fef75600700 1 -- 172.21.15.47:0/3721063798 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 18) v1 ==== 137854+0+0 (secure 0 0 0) 0x7fef68019070 con 0x7fef78069e40 2024-03-21T10:24:53.345 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:53.343+0000 7fef75600700 1 --2- 172.21.15.47:0/3721063798 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7fef5c0a3030 0x7fef5c0a54f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:24:53.346 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:53.344+0000 7fef75600700 1 -- 172.21.15.47:0/3721063798 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9216+0+0 (secure 0 0 0) 0x7fef680cf6f0 con 0x7fef78069e40 2024-03-21T10:24:53.346 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:53.344+0000 7fef6fe00700 1 --2- 172.21.15.47:0/3721063798 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7fef5c0a3030 0x7fef5c0a54f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:24:53.347 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:53.345+0000 7fef6fe00700 1 --2- 172.21.15.47:0/3721063798 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7fef5c0a3030 0x7fef5c0a54f0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fef780785a0 tx=0x7fef60009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:24:53.350 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:53.348+0000 7fef75600700 1 -- 172.21.15.47:0/3721063798 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fef6809c110 con 0x7fef78069e40 2024-03-21T10:24:53.463 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:24:53 smithi047 ceph-mon[31547]: pgmap v458: 129 pgs: 129 active+clean; 642 KiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-21T10:24:53.620 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:53.617+0000 7fef7f000700 1 -- 172.21.15.47:0/3721063798 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fef7802da30 con 0x7fef78069e40 2024-03-21T10:24:53.620 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:53.618+0000 7fef75600700 1 -- 172.21.15.47:0/3721063798 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v29) v1 ==== 53+0+88 (secure 0 0 0) 0x7fef6809ba10 con 0x7fef78069e40 2024-03-21T10:24:53.621 INFO:teuthology.orchestra.run.smithi047.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2024-03-21T10:24:53.623 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:53.620+0000 7fef7f000700 1 -- 172.21.15.47:0/3721063798 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7fef5c0a3030 msgr2=0x7fef5c0a54f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:24:53.623 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:53.620+0000 7fef7f000700 1 --2- 172.21.15.47:0/3721063798 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7fef5c0a3030 0x7fef5c0a54f0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fef780785a0 tx=0x7fef60009260 comp rx=0 tx=0).stop 2024-03-21T10:24:53.623 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:53.620+0000 7fef7f000700 1 -- 172.21.15.47:0/3721063798 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fef78069e40 msgr2=0x7fef7819e450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:24:53.623 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:53.620+0000 7fef7f000700 1 --2- 172.21.15.47:0/3721063798 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fef78069e40 0x7fef7819e450 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7fef6800d950 tx=0x7fef6800dd10 comp rx=0 tx=0).stop 2024-03-21T10:24:53.624 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:53.622+0000 7fef7f000700 1 -- 172.21.15.47:0/3721063798 shutdown_connections 2024-03-21T10:24:53.624 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:53.622+0000 7fef7f000700 1 --2- 172.21.15.47:0/3721063798 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7fef5c0a3030 0x7fef5c0a54f0 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:24:53.624 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:53.622+0000 7fef7f000700 1 --2- 172.21.15.47:0/3721063798 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fef781081e0 0x7fef78198610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:24:53.624 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:53.622+0000 7fef7f000700 1 --2- 172.21.15.47:0/3721063798 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fef78069e40 0x7fef7819e450 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:24:53.624 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:53.622+0000 7fef7f000700 1 --2- 172.21.15.47:0/3721063798 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fef7806a7e0 0x7fef7819e990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:24:53.624 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:53.622+0000 7fef7f000700 1 -- 172.21.15.47:0/3721063798 >> 172.21.15.47:0/3721063798 conn(0x7fef78077fc0 msgr2=0x7fef780ffc90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:24:53.624 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:53.622+0000 7fef7f000700 1 -- 172.21.15.47:0/3721063798 shutdown_connections 2024-03-21T10:24:53.624 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:53.622+0000 7fef7f000700 1 -- 172.21.15.47:0/3721063798 wait complete. 2024-03-21T10:24:53.729 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:24:53 smithi100 ceph-mon[35193]: pgmap v458: 129 pgs: 129 active+clean; 642 KiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-21T10:24:53.739 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:24:53 smithi157 ceph-mon[35581]: pgmap v458: 129 pgs: 129 active+clean; 642 KiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-21T10:24:54.452 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:24:54 smithi047 ceph-mon[31547]: from='client.? 172.21.15.47:0/3721063798' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-03-21T10:24:54.513 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-03-21T10:24:54.514 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-03-21T10:24:54.514 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi047.front.sepia.ceph.com 2024-03-21T10:24:54.514 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-03-21T10:24:54.514 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-03-21T10:24:54.514 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-03-21T10:24:54.514 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-03-21T10:24:54.514 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-03-21T10:24:54.514 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.0' 2024-03-21T10:24:54.515 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-21T10:24:54.515 DEBUG:teuthology.orchestra.run.smithi047:> ip addr 2024-03-21T10:24:54.577 INFO:teuthology.orchestra.run.smithi047.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-03-21T10:24:54.578 INFO:teuthology.orchestra.run.smithi047.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-03-21T10:24:54.578 INFO:teuthology.orchestra.run.smithi047.stdout: inet 127.0.0.1/8 scope host lo 2024-03-21T10:24:54.578 INFO:teuthology.orchestra.run.smithi047.stdout: valid_lft forever preferred_lft forever 2024-03-21T10:24:54.578 INFO:teuthology.orchestra.run.smithi047.stdout: inet6 ::1/128 scope host 2024-03-21T10:24:54.578 INFO:teuthology.orchestra.run.smithi047.stdout: valid_lft forever preferred_lft forever 2024-03-21T10:24:54.578 INFO:teuthology.orchestra.run.smithi047.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-21T10:24:54.578 INFO:teuthology.orchestra.run.smithi047.stdout: link/ether 0c:c4:7a:6b:ff:78 brd ff:ff:ff:ff:ff:ff 2024-03-21T10:24:54.578 INFO:teuthology.orchestra.run.smithi047.stdout: altname enp7s0f0 2024-03-21T10:24:54.578 INFO:teuthology.orchestra.run.smithi047.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-21T10:24:54.578 INFO:teuthology.orchestra.run.smithi047.stdout: link/ether 0c:c4:7a:6b:ff:79 brd ff:ff:ff:ff:ff:ff 2024-03-21T10:24:54.578 INFO:teuthology.orchestra.run.smithi047.stdout: altname enp7s0f1 2024-03-21T10:24:54.579 INFO:teuthology.orchestra.run.smithi047.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-03-21T10:24:54.579 INFO:teuthology.orchestra.run.smithi047.stdout: link/ether 0c:c4:7a:bd:14:8e brd ff:ff:ff:ff:ff:ff 2024-03-21T10:24:54.579 INFO:teuthology.orchestra.run.smithi047.stdout: altname enp1s0f0 2024-03-21T10:24:54.579 INFO:teuthology.orchestra.run.smithi047.stdout: inet 172.21.15.47/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-03-21T10:24:54.579 INFO:teuthology.orchestra.run.smithi047.stdout: valid_lft 41745sec preferred_lft 41745sec 2024-03-21T10:24:54.579 INFO:teuthology.orchestra.run.smithi047.stdout: inet6 fe80::ec4:7aff:febd:148e/64 scope link noprefixroute 2024-03-21T10:24:54.579 INFO:teuthology.orchestra.run.smithi047.stdout: valid_lft forever preferred_lft forever 2024-03-21T10:24:54.579 INFO:teuthology.orchestra.run.smithi047.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-21T10:24:54.579 INFO:teuthology.orchestra.run.smithi047.stdout: link/ether 0c:c4:7a:bd:14:8f brd ff:ff:ff:ff:ff:ff 2024-03-21T10:24:54.579 INFO:teuthology.orchestra.run.smithi047.stdout: altname enp1s0f1 2024-03-21T10:24:54.580 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-03-21T10:24:54.580 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-21T10:24:54.580 DEBUG:teuthology.orchestra.run.smithi047:> (cd / && exec bash -c ' 2024-03-21T10:24:54.580 DEBUG:teuthology.orchestra.run.smithi047:> set -e 2024-03-21T10:24:54.580 DEBUG:teuthology.orchestra.run.smithi047:> sudo ip link add name ceph-brx type bridge 2024-03-21T10:24:54.580 DEBUG:teuthology.orchestra.run.smithi047:> sudo ip addr flush dev ceph-brx 2024-03-21T10:24:54.580 DEBUG:teuthology.orchestra.run.smithi047:> sudo ip link set ceph-brx up 2024-03-21T10:24:54.580 DEBUG:teuthology.orchestra.run.smithi047:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-03-21T10:24:54.580 DEBUG:teuthology.orchestra.run.smithi047:> ') 2024-03-21T10:24:54.729 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:24:54 smithi100 ceph-mon[35193]: from='client.? 172.21.15.47:0/3721063798' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-03-21T10:24:54.739 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:24:54 smithi157 ceph-mon[35581]: from='client.? 172.21.15.47:0/3721063798' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-03-21T10:24:54.815 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-21T10:24:54.816 DEBUG:teuthology.orchestra.run.smithi047:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-03-21T10:24:54.898 INFO:teuthology.orchestra.run.smithi047.stdout:1 2024-03-21T10:24:54.903 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-21T10:24:54.903 DEBUG:teuthology.orchestra.run.smithi047:> route 2024-03-21T10:24:54.955 INFO:teuthology.orchestra.run.smithi047.stdout:Kernel IP routing table 2024-03-21T10:24:54.955 INFO:teuthology.orchestra.run.smithi047.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-03-21T10:24:54.955 INFO:teuthology.orchestra.run.smithi047.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-03-21T10:24:54.955 INFO:teuthology.orchestra.run.smithi047.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-03-21T10:24:54.955 INFO:teuthology.orchestra.run.smithi047.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-03-21T10:24:54.956 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-21T10:24:54.956 DEBUG:teuthology.orchestra.run.smithi047:> (cd / && exec bash -c ' 2024-03-21T10:24:54.956 DEBUG:teuthology.orchestra.run.smithi047:> set -e 2024-03-21T10:24:54.957 DEBUG:teuthology.orchestra.run.smithi047:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-03-21T10:24:54.957 DEBUG:teuthology.orchestra.run.smithi047:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-03-21T10:24:54.957 DEBUG:teuthology.orchestra.run.smithi047:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-03-21T10:24:54.957 DEBUG:teuthology.orchestra.run.smithi047:> ') 2024-03-21T10:24:55.133 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-21T10:24:55.133 DEBUG:teuthology.orchestra.run.smithi047:> ip netns list 2024-03-21T10:24:55.183 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-21T10:24:55.184 DEBUG:teuthology.orchestra.run.smithi047:> ip netns list-id 2024-03-21T10:24:55.234 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-21T10:24:55.234 DEBUG:teuthology.orchestra.run.smithi047:> (cd / && exec bash -c ' 2024-03-21T10:24:55.235 DEBUG:teuthology.orchestra.run.smithi047:> set -e 2024-03-21T10:24:55.235 DEBUG:teuthology.orchestra.run.smithi047:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0 2024-03-21T10:24:55.235 DEBUG:teuthology.orchestra.run.smithi047:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0 2024-03-21T10:24:55.235 DEBUG:teuthology.orchestra.run.smithi047:> ') 2024-03-21T10:24:55.381 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2024-03-21T10:24:55.382 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-21T10:24:55.382 DEBUG:teuthology.orchestra.run.smithi047:> (cd / && exec bash -c ' 2024-03-21T10:24:55.382 DEBUG:teuthology.orchestra.run.smithi047:> set -e 2024-03-21T10:24:55.382 DEBUG:teuthology.orchestra.run.smithi047:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0 2024-03-21T10:24:55.382 DEBUG:teuthology.orchestra.run.smithi047:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-03-21T10:24:55.382 DEBUG:teuthology.orchestra.run.smithi047:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up 2024-03-21T10:24:55.382 DEBUG:teuthology.orchestra.run.smithi047:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up 2024-03-21T10:24:55.382 DEBUG:teuthology.orchestra.run.smithi047:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254 2024-03-21T10:24:55.382 DEBUG:teuthology.orchestra.run.smithi047:> ') 2024-03-21T10:24:55.552 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:24:55 smithi047 ceph-mon[31547]: pgmap v459: 129 pgs: 129 active+clean; 642 KiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-03-21T10:24:55.649 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-21T10:24:55.650 DEBUG:teuthology.orchestra.run.smithi047:> (cd / && exec bash -c ' 2024-03-21T10:24:55.650 DEBUG:teuthology.orchestra.run.smithi047:> set -e 2024-03-21T10:24:55.650 DEBUG:teuthology.orchestra.run.smithi047:> sudo ip link set brx.0 up 2024-03-21T10:24:55.650 DEBUG:teuthology.orchestra.run.smithi047:> sudo ip link set dev brx.0 master ceph-brx 2024-03-21T10:24:55.650 DEBUG:teuthology.orchestra.run.smithi047:> ') 2024-03-21T10:24:55.729 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:24:55 smithi100 ceph-mon[35193]: pgmap v459: 129 pgs: 129 active+clean; 642 KiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-03-21T10:24:55.739 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:24:55 smithi157 ceph-mon[35581]: pgmap v459: 129 pgs: 129 active+clean; 642 KiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-03-21T10:24:55.812 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {'mount_subvol_num': 0} 2024-03-21T10:24:55.813 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-03-21T10:24:55.813 DEBUG:teuthology.orchestra.run.smithi047:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-03-21T10:24:55.859 INFO:teuthology.orchestra.run.smithi047.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-03-21T10:24:55.859 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-03-21T10:24:55.860 DEBUG:teuthology.orchestra.run.smithi047:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-03-21T10:24:55.952 DEBUG:teuthology.orchestra.run.smithi047:> sudo modprobe fuse 2024-03-21T10:24:56.047 DEBUG:teuthology.orchestra.run.smithi047:> cat /proc/self/mounts | awk '{print $2}' 2024-03-21T10:24:56.102 INFO:teuthology.orchestra.run.smithi047.stdout:/proc 2024-03-21T10:24:56.103 INFO:teuthology.orchestra.run.smithi047.stdout:/sys 2024-03-21T10:24:56.103 INFO:teuthology.orchestra.run.smithi047.stdout:/dev 2024-03-21T10:24:56.103 INFO:teuthology.orchestra.run.smithi047.stdout:/sys/kernel/security 2024-03-21T10:24:56.103 INFO:teuthology.orchestra.run.smithi047.stdout:/dev/shm 2024-03-21T10:24:56.103 INFO:teuthology.orchestra.run.smithi047.stdout:/dev/pts 2024-03-21T10:24:56.103 INFO:teuthology.orchestra.run.smithi047.stdout:/run 2024-03-21T10:24:56.103 INFO:teuthology.orchestra.run.smithi047.stdout:/sys/fs/cgroup 2024-03-21T10:24:56.103 INFO:teuthology.orchestra.run.smithi047.stdout:/sys/fs/pstore 2024-03-21T10:24:56.103 INFO:teuthology.orchestra.run.smithi047.stdout:/sys/fs/bpf 2024-03-21T10:24:56.103 INFO:teuthology.orchestra.run.smithi047.stdout:/ 2024-03-21T10:24:56.103 INFO:teuthology.orchestra.run.smithi047.stdout:/sys/fs/selinux 2024-03-21T10:24:56.103 INFO:teuthology.orchestra.run.smithi047.stdout:/proc/sys/fs/binfmt_misc 2024-03-21T10:24:56.104 INFO:teuthology.orchestra.run.smithi047.stdout:/dev/mqueue 2024-03-21T10:24:56.104 INFO:teuthology.orchestra.run.smithi047.stdout:/dev/hugepages 2024-03-21T10:24:56.104 INFO:teuthology.orchestra.run.smithi047.stdout:/sys/kernel/debug 2024-03-21T10:24:56.104 INFO:teuthology.orchestra.run.smithi047.stdout:/sys/kernel/tracing 2024-03-21T10:24:56.104 INFO:teuthology.orchestra.run.smithi047.stdout:/sys/fs/fuse/connections 2024-03-21T10:24:56.104 INFO:teuthology.orchestra.run.smithi047.stdout:/sys/kernel/config 2024-03-21T10:24:56.104 INFO:teuthology.orchestra.run.smithi047.stdout:/run/credentials/systemd-sysctl.service 2024-03-21T10:24:56.104 INFO:teuthology.orchestra.run.smithi047.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-03-21T10:24:56.104 INFO:teuthology.orchestra.run.smithi047.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-03-21T10:24:56.104 INFO:teuthology.orchestra.run.smithi047.stdout:/var/lib/nfs/rpc_pipefs 2024-03-21T10:24:56.104 INFO:teuthology.orchestra.run.smithi047.stdout:/run/user/1000 2024-03-21T10:24:56.104 INFO:teuthology.orchestra.run.smithi047.stdout:/var/lib/ceph 2024-03-21T10:24:56.104 INFO:teuthology.orchestra.run.smithi047.stdout:/var/lib/containers/storage/overlay 2024-03-21T10:24:56.105 INFO:teuthology.orchestra.run.smithi047.stdout:/var/lib/containers/storage/overlay/10ab6f66d550c87573f945f803c4f92650c76c2142b200e308dd5734436cc6e8/merged 2024-03-21T10:24:56.105 INFO:teuthology.orchestra.run.smithi047.stdout:/var/lib/containers/storage/overlay/4193f3bc34f41e5fb90c176b8210c5c797208467c6f372b377075907f20e100c/merged 2024-03-21T10:24:56.105 INFO:teuthology.orchestra.run.smithi047.stdout:/run/user/0 2024-03-21T10:24:56.105 INFO:teuthology.orchestra.run.smithi047.stdout:/proc/sys/fs/binfmt_misc 2024-03-21T10:24:56.105 INFO:teuthology.orchestra.run.smithi047.stdout:/var/lib/containers/storage/overlay/fa63c9447d222c5abaf50fb70c1cc34ea356854abf3426eb0c8a7f015a95e543/merged 2024-03-21T10:24:56.105 INFO:teuthology.orchestra.run.smithi047.stdout:/var/lib/containers/storage/overlay/cab8c1c11ddc44b3f7251050e29e61c724a07774891943f3f169428cf41f19ce/merged 2024-03-21T10:24:56.105 INFO:teuthology.orchestra.run.smithi047.stdout:/var/lib/containers/storage/overlay/8bc14524d7f02db40a42ab65f069f5682514962833182f9de83d3508887ef2e7/merged 2024-03-21T10:24:56.105 INFO:teuthology.orchestra.run.smithi047.stdout:/var/lib/containers/storage/overlay/225ab22923163649720ee3c99e303986cfdc6df3ece1f25ae6bea1c7386ea563/merged 2024-03-21T10:24:56.105 INFO:teuthology.orchestra.run.smithi047.stdout:/var/lib/containers/storage/overlay/a670d6b1797f7b111d0422282a9f2fc87507a063f306170c368d80948f2e4034/merged 2024-03-21T10:24:56.105 INFO:teuthology.orchestra.run.smithi047.stdout:/var/lib/containers/storage/overlay/d41f794868e6e4c013a6f5180d396f28874ea4f7b53dc71022d9b672a7f62644/merged 2024-03-21T10:24:56.105 INFO:teuthology.orchestra.run.smithi047.stdout:/var/lib/containers/storage/overlay/d2f5e9cf3dce6353de30b11e940b05e7d1241191f5cbdf874907eb57db79323c/merged 2024-03-21T10:24:56.105 INFO:teuthology.orchestra.run.smithi047.stdout:/var/lib/containers/storage/overlay/decd4140254122fd73d80c489f7134415c0f4da62e417409ccd425e24cd125cd/merged 2024-03-21T10:24:56.105 INFO:teuthology.orchestra.run.smithi047.stdout:/run/netns 2024-03-21T10:24:56.106 INFO:teuthology.orchestra.run.smithi047.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-03-21T10:24:56.106 INFO:teuthology.orchestra.run.smithi047.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-03-21T10:24:56.107 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-21T10:24:56.107 DEBUG:teuthology.orchestra.run.smithi047:> ls /sys/fs/fuse/connections 2024-03-21T10:24:56.156 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-03-21T10:24:56.156 DEBUG:teuthology.orchestra.run.smithi047:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0 --client_mountpoint=/volumes/qa/sv_0/2e602e66-0b27-4e4c-a210-4a1ed53947cf --client_fs=cephfs) 2024-03-21T10:24:56.161 DEBUG:teuthology.orchestra.run.smithi047:> sudo modprobe fuse 2024-03-21T10:24:56.251 DEBUG:teuthology.orchestra.run.smithi047:> cat /proc/self/mounts | awk '{print $2}' 2024-03-21T10:24:56.301 INFO:teuthology.orchestra.run.smithi047.stdout:/proc 2024-03-21T10:24:56.301 INFO:teuthology.orchestra.run.smithi047.stdout:/sys 2024-03-21T10:24:56.301 INFO:teuthology.orchestra.run.smithi047.stdout:/dev 2024-03-21T10:24:56.301 INFO:teuthology.orchestra.run.smithi047.stdout:/sys/kernel/security 2024-03-21T10:24:56.301 INFO:teuthology.orchestra.run.smithi047.stdout:/dev/shm 2024-03-21T10:24:56.301 INFO:teuthology.orchestra.run.smithi047.stdout:/dev/pts 2024-03-21T10:24:56.302 INFO:teuthology.orchestra.run.smithi047.stdout:/run 2024-03-21T10:24:56.302 INFO:teuthology.orchestra.run.smithi047.stdout:/sys/fs/cgroup 2024-03-21T10:24:56.302 INFO:teuthology.orchestra.run.smithi047.stdout:/sys/fs/pstore 2024-03-21T10:24:56.302 INFO:teuthology.orchestra.run.smithi047.stdout:/sys/fs/bpf 2024-03-21T10:24:56.302 INFO:teuthology.orchestra.run.smithi047.stdout:/ 2024-03-21T10:24:56.302 INFO:teuthology.orchestra.run.smithi047.stdout:/sys/fs/selinux 2024-03-21T10:24:56.302 INFO:teuthology.orchestra.run.smithi047.stdout:/proc/sys/fs/binfmt_misc 2024-03-21T10:24:56.302 INFO:teuthology.orchestra.run.smithi047.stdout:/dev/mqueue 2024-03-21T10:24:56.302 INFO:teuthology.orchestra.run.smithi047.stdout:/dev/hugepages 2024-03-21T10:24:56.302 INFO:teuthology.orchestra.run.smithi047.stdout:/sys/kernel/debug 2024-03-21T10:24:56.302 INFO:teuthology.orchestra.run.smithi047.stdout:/sys/kernel/tracing 2024-03-21T10:24:56.303 INFO:teuthology.orchestra.run.smithi047.stdout:/sys/fs/fuse/connections 2024-03-21T10:24:56.303 INFO:teuthology.orchestra.run.smithi047.stdout:/sys/kernel/config 2024-03-21T10:24:56.303 INFO:teuthology.orchestra.run.smithi047.stdout:/run/credentials/systemd-sysctl.service 2024-03-21T10:24:56.303 INFO:teuthology.orchestra.run.smithi047.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-03-21T10:24:56.303 INFO:teuthology.orchestra.run.smithi047.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-03-21T10:24:56.303 INFO:teuthology.orchestra.run.smithi047.stdout:/var/lib/nfs/rpc_pipefs 2024-03-21T10:24:56.303 INFO:teuthology.orchestra.run.smithi047.stdout:/run/user/1000 2024-03-21T10:24:56.303 INFO:teuthology.orchestra.run.smithi047.stdout:/var/lib/ceph 2024-03-21T10:24:56.303 INFO:teuthology.orchestra.run.smithi047.stdout:/var/lib/containers/storage/overlay 2024-03-21T10:24:56.303 INFO:teuthology.orchestra.run.smithi047.stdout:/var/lib/containers/storage/overlay/10ab6f66d550c87573f945f803c4f92650c76c2142b200e308dd5734436cc6e8/merged 2024-03-21T10:24:56.303 INFO:teuthology.orchestra.run.smithi047.stdout:/var/lib/containers/storage/overlay/4193f3bc34f41e5fb90c176b8210c5c797208467c6f372b377075907f20e100c/merged 2024-03-21T10:24:56.303 INFO:teuthology.orchestra.run.smithi047.stdout:/run/user/0 2024-03-21T10:24:56.303 INFO:teuthology.orchestra.run.smithi047.stdout:/proc/sys/fs/binfmt_misc 2024-03-21T10:24:56.303 INFO:teuthology.orchestra.run.smithi047.stdout:/var/lib/containers/storage/overlay/fa63c9447d222c5abaf50fb70c1cc34ea356854abf3426eb0c8a7f015a95e543/merged 2024-03-21T10:24:56.304 INFO:teuthology.orchestra.run.smithi047.stdout:/var/lib/containers/storage/overlay/cab8c1c11ddc44b3f7251050e29e61c724a07774891943f3f169428cf41f19ce/merged 2024-03-21T10:24:56.304 INFO:teuthology.orchestra.run.smithi047.stdout:/var/lib/containers/storage/overlay/8bc14524d7f02db40a42ab65f069f5682514962833182f9de83d3508887ef2e7/merged 2024-03-21T10:24:56.304 INFO:teuthology.orchestra.run.smithi047.stdout:/var/lib/containers/storage/overlay/225ab22923163649720ee3c99e303986cfdc6df3ece1f25ae6bea1c7386ea563/merged 2024-03-21T10:24:56.304 INFO:teuthology.orchestra.run.smithi047.stdout:/var/lib/containers/storage/overlay/a670d6b1797f7b111d0422282a9f2fc87507a063f306170c368d80948f2e4034/merged 2024-03-21T10:24:56.304 INFO:teuthology.orchestra.run.smithi047.stdout:/var/lib/containers/storage/overlay/d41f794868e6e4c013a6f5180d396f28874ea4f7b53dc71022d9b672a7f62644/merged 2024-03-21T10:24:56.304 INFO:teuthology.orchestra.run.smithi047.stdout:/var/lib/containers/storage/overlay/d2f5e9cf3dce6353de30b11e940b05e7d1241191f5cbdf874907eb57db79323c/merged 2024-03-21T10:24:56.304 INFO:teuthology.orchestra.run.smithi047.stdout:/var/lib/containers/storage/overlay/decd4140254122fd73d80c489f7134415c0f4da62e417409ccd425e24cd125cd/merged 2024-03-21T10:24:56.304 INFO:teuthology.orchestra.run.smithi047.stdout:/run/netns 2024-03-21T10:24:56.304 INFO:teuthology.orchestra.run.smithi047.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-03-21T10:24:56.304 INFO:teuthology.orchestra.run.smithi047.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-03-21T10:24:56.305 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-21T10:24:56.305 DEBUG:teuthology.orchestra.run.smithi047:> ls /sys/fs/fuse/connections 2024-03-21T10:24:56.374 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi047.stderr:2024-03-21T10:24:56.373+0000 7f33a97b8480 -1 init, newargv = 0x5621affc93a0 newargc=15 2024-03-21T10:24:56.375 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi047.stderr:ceph-fuse[83961]: starting ceph client 2024-03-21T10:24:56.417 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi047.stderr:ceph-fuse[83961]: starting fuse 2024-03-21T10:24:57.450 DEBUG:teuthology.orchestra.run.smithi047:> sudo modprobe fuse 2024-03-21T10:24:57.549 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:24:57 smithi047 ceph-mon[31547]: pgmap v460: 129 pgs: 129 active+clean; 642 KiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-21T10:24:57.552 DEBUG:teuthology.orchestra.run.smithi047:> cat /proc/self/mounts | awk '{print $2}' 2024-03-21T10:24:57.611 INFO:teuthology.orchestra.run.smithi047.stdout:/proc 2024-03-21T10:24:57.611 INFO:teuthology.orchestra.run.smithi047.stdout:/sys 2024-03-21T10:24:57.611 INFO:teuthology.orchestra.run.smithi047.stdout:/dev 2024-03-21T10:24:57.611 INFO:teuthology.orchestra.run.smithi047.stdout:/sys/kernel/security 2024-03-21T10:24:57.611 INFO:teuthology.orchestra.run.smithi047.stdout:/dev/shm 2024-03-21T10:24:57.611 INFO:teuthology.orchestra.run.smithi047.stdout:/dev/pts 2024-03-21T10:24:57.612 INFO:teuthology.orchestra.run.smithi047.stdout:/run 2024-03-21T10:24:57.612 INFO:teuthology.orchestra.run.smithi047.stdout:/sys/fs/cgroup 2024-03-21T10:24:57.612 INFO:teuthology.orchestra.run.smithi047.stdout:/sys/fs/pstore 2024-03-21T10:24:57.612 INFO:teuthology.orchestra.run.smithi047.stdout:/sys/fs/bpf 2024-03-21T10:24:57.612 INFO:teuthology.orchestra.run.smithi047.stdout:/ 2024-03-21T10:24:57.612 INFO:teuthology.orchestra.run.smithi047.stdout:/sys/fs/selinux 2024-03-21T10:24:57.612 INFO:teuthology.orchestra.run.smithi047.stdout:/proc/sys/fs/binfmt_misc 2024-03-21T10:24:57.612 INFO:teuthology.orchestra.run.smithi047.stdout:/dev/mqueue 2024-03-21T10:24:57.612 INFO:teuthology.orchestra.run.smithi047.stdout:/dev/hugepages 2024-03-21T10:24:57.612 INFO:teuthology.orchestra.run.smithi047.stdout:/sys/kernel/debug 2024-03-21T10:24:57.612 INFO:teuthology.orchestra.run.smithi047.stdout:/sys/kernel/tracing 2024-03-21T10:24:57.612 INFO:teuthology.orchestra.run.smithi047.stdout:/sys/fs/fuse/connections 2024-03-21T10:24:57.612 INFO:teuthology.orchestra.run.smithi047.stdout:/sys/kernel/config 2024-03-21T10:24:57.613 INFO:teuthology.orchestra.run.smithi047.stdout:/run/credentials/systemd-sysctl.service 2024-03-21T10:24:57.613 INFO:teuthology.orchestra.run.smithi047.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-03-21T10:24:57.613 INFO:teuthology.orchestra.run.smithi047.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-03-21T10:24:57.613 INFO:teuthology.orchestra.run.smithi047.stdout:/var/lib/nfs/rpc_pipefs 2024-03-21T10:24:57.613 INFO:teuthology.orchestra.run.smithi047.stdout:/run/user/1000 2024-03-21T10:24:57.613 INFO:teuthology.orchestra.run.smithi047.stdout:/var/lib/ceph 2024-03-21T10:24:57.613 INFO:teuthology.orchestra.run.smithi047.stdout:/var/lib/containers/storage/overlay 2024-03-21T10:24:57.613 INFO:teuthology.orchestra.run.smithi047.stdout:/var/lib/containers/storage/overlay/10ab6f66d550c87573f945f803c4f92650c76c2142b200e308dd5734436cc6e8/merged 2024-03-21T10:24:57.613 INFO:teuthology.orchestra.run.smithi047.stdout:/var/lib/containers/storage/overlay/4193f3bc34f41e5fb90c176b8210c5c797208467c6f372b377075907f20e100c/merged 2024-03-21T10:24:57.613 INFO:teuthology.orchestra.run.smithi047.stdout:/run/user/0 2024-03-21T10:24:57.613 INFO:teuthology.orchestra.run.smithi047.stdout:/proc/sys/fs/binfmt_misc 2024-03-21T10:24:57.613 INFO:teuthology.orchestra.run.smithi047.stdout:/var/lib/containers/storage/overlay/fa63c9447d222c5abaf50fb70c1cc34ea356854abf3426eb0c8a7f015a95e543/merged 2024-03-21T10:24:57.613 INFO:teuthology.orchestra.run.smithi047.stdout:/var/lib/containers/storage/overlay/cab8c1c11ddc44b3f7251050e29e61c724a07774891943f3f169428cf41f19ce/merged 2024-03-21T10:24:57.613 INFO:teuthology.orchestra.run.smithi047.stdout:/var/lib/containers/storage/overlay/8bc14524d7f02db40a42ab65f069f5682514962833182f9de83d3508887ef2e7/merged 2024-03-21T10:24:57.613 INFO:teuthology.orchestra.run.smithi047.stdout:/var/lib/containers/storage/overlay/225ab22923163649720ee3c99e303986cfdc6df3ece1f25ae6bea1c7386ea563/merged 2024-03-21T10:24:57.614 INFO:teuthology.orchestra.run.smithi047.stdout:/var/lib/containers/storage/overlay/a670d6b1797f7b111d0422282a9f2fc87507a063f306170c368d80948f2e4034/merged 2024-03-21T10:24:57.614 INFO:teuthology.orchestra.run.smithi047.stdout:/var/lib/containers/storage/overlay/d41f794868e6e4c013a6f5180d396f28874ea4f7b53dc71022d9b672a7f62644/merged 2024-03-21T10:24:57.614 INFO:teuthology.orchestra.run.smithi047.stdout:/var/lib/containers/storage/overlay/d2f5e9cf3dce6353de30b11e940b05e7d1241191f5cbdf874907eb57db79323c/merged 2024-03-21T10:24:57.614 INFO:teuthology.orchestra.run.smithi047.stdout:/var/lib/containers/storage/overlay/decd4140254122fd73d80c489f7134415c0f4da62e417409ccd425e24cd125cd/merged 2024-03-21T10:24:57.614 INFO:teuthology.orchestra.run.smithi047.stdout:/run/netns 2024-03-21T10:24:57.614 INFO:teuthology.orchestra.run.smithi047.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-03-21T10:24:57.614 INFO:teuthology.orchestra.run.smithi047.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-03-21T10:24:57.614 INFO:teuthology.orchestra.run.smithi047.stdout:/home/ubuntu/cephtest/mnt.0 2024-03-21T10:24:57.615 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-21T10:24:57.615 DEBUG:teuthology.orchestra.run.smithi047:> ls /sys/fs/fuse/connections 2024-03-21T10:24:57.666 INFO:teuthology.orchestra.run.smithi047.stdout:118 2024-03-21T10:24:57.667 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [118] 2024-03-21T10:24:57.667 DEBUG:teuthology.orchestra.run.smithi047:> sudo stdin-killer -- python3 -c ' 2024-03-21T10:24:57.667 DEBUG:teuthology.orchestra.run.smithi047:> import glob 2024-03-21T10:24:57.667 DEBUG:teuthology.orchestra.run.smithi047:> import re 2024-03-21T10:24:57.667 DEBUG:teuthology.orchestra.run.smithi047:> import os 2024-03-21T10:24:57.667 DEBUG:teuthology.orchestra.run.smithi047:> import subprocess 2024-03-21T10:24:57.667 DEBUG:teuthology.orchestra.run.smithi047:> 2024-03-21T10:24:57.667 DEBUG:teuthology.orchestra.run.smithi047:> def _find_admin_socket(client_name): 2024-03-21T10:24:57.667 DEBUG:teuthology.orchestra.run.smithi047:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-03-21T10:24:57.667 DEBUG:teuthology.orchestra.run.smithi047:> files = glob.glob(asok_path) 2024-03-21T10:24:57.668 DEBUG:teuthology.orchestra.run.smithi047:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-03-21T10:24:57.668 DEBUG:teuthology.orchestra.run.smithi047:> 2024-03-21T10:24:57.668 DEBUG:teuthology.orchestra.run.smithi047:> # Given a non-glob path, it better be there 2024-03-21T10:24:57.668 DEBUG:teuthology.orchestra.run.smithi047:> if "*" not in asok_path: 2024-03-21T10:24:57.668 DEBUG:teuthology.orchestra.run.smithi047:> assert(len(files) == 1) 2024-03-21T10:24:57.668 DEBUG:teuthology.orchestra.run.smithi047:> return files[0] 2024-03-21T10:24:57.668 DEBUG:teuthology.orchestra.run.smithi047:> 2024-03-21T10:24:57.668 DEBUG:teuthology.orchestra.run.smithi047:> for f in files: 2024-03-21T10:24:57.668 DEBUG:teuthology.orchestra.run.smithi047:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-03-21T10:24:57.668 DEBUG:teuthology.orchestra.run.smithi047:> if os.path.exists("/proc/{0}".format(pid)): 2024-03-21T10:24:57.668 DEBUG:teuthology.orchestra.run.smithi047:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-03-21T10:24:57.668 DEBUG:teuthology.orchestra.run.smithi047:> contents = proc_f.read() 2024-03-21T10:24:57.668 DEBUG:teuthology.orchestra.run.smithi047:> if mountpoint in contents: 2024-03-21T10:24:57.668 DEBUG:teuthology.orchestra.run.smithi047:> return f 2024-03-21T10:24:57.668 DEBUG:teuthology.orchestra.run.smithi047:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-03-21T10:24:57.669 DEBUG:teuthology.orchestra.run.smithi047:> 2024-03-21T10:24:57.669 DEBUG:teuthology.orchestra.run.smithi047:> print(_find_admin_socket("client.0")) 2024-03-21T10:24:57.669 DEBUG:teuthology.orchestra.run.smithi047:> ' 2024-03-21T10:24:57.729 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:24:57 smithi100 ceph-mon[35193]: pgmap v460: 129 pgs: 129 active+clean; 642 KiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-21T10:24:57.739 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:24:57 smithi157 ceph-mon[35581]: pgmap v460: 129 pgs: 129 active+clean; 642 KiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-21T10:24:57.850 INFO:teuthology.orchestra.run.smithi047.stdout:/var/run/ceph/ceph-client.0.83961.asok 2024-03-21T10:24:57.855 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:57 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-21T10:24:57.869 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.83961.asok 2024-03-21T10:24:57.869 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-21T10:24:57.869 DEBUG:teuthology.orchestra.run.smithi047:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.83961.asok status 2024-03-21T10:24:58.114 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-03-21T10:24:58.114 INFO:teuthology.orchestra.run.smithi047.stdout: "metadata": { 2024-03-21T10:24:58.114 INFO:teuthology.orchestra.run.smithi047.stdout: "ceph_sha1": "2edfee68ad7243bed422a049b6f5f58619300f11", 2024-03-21T10:24:58.114 INFO:teuthology.orchestra.run.smithi047.stdout: "ceph_version": "ceph version 19.0.0-2231-g2edfee68 (2edfee68ad7243bed422a049b6f5f58619300f11) squid (dev)", 2024-03-21T10:24:58.114 INFO:teuthology.orchestra.run.smithi047.stdout: "entity_id": "0", 2024-03-21T10:24:58.114 INFO:teuthology.orchestra.run.smithi047.stdout: "hostname": "smithi047", 2024-03-21T10:24:58.114 INFO:teuthology.orchestra.run.smithi047.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2024-03-21T10:24:58.114 INFO:teuthology.orchestra.run.smithi047.stdout: "pid": "83961", 2024-03-21T10:24:58.115 INFO:teuthology.orchestra.run.smithi047.stdout: "root": "/volumes/qa/sv_0/2e602e66-0b27-4e4c-a210-4a1ed53947cf" 2024-03-21T10:24:58.115 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:24:58.115 INFO:teuthology.orchestra.run.smithi047.stdout: "dentry_count": 0, 2024-03-21T10:24:58.115 INFO:teuthology.orchestra.run.smithi047.stdout: "dentry_pinned_count": 0, 2024-03-21T10:24:58.115 INFO:teuthology.orchestra.run.smithi047.stdout: "id": 15498, 2024-03-21T10:24:58.115 INFO:teuthology.orchestra.run.smithi047.stdout: "inst": { 2024-03-21T10:24:58.115 INFO:teuthology.orchestra.run.smithi047.stdout: "name": { 2024-03-21T10:24:58.115 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "client", 2024-03-21T10:24:58.115 INFO:teuthology.orchestra.run.smithi047.stdout: "num": 15498 2024-03-21T10:24:58.115 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:24:58.115 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": { 2024-03-21T10:24:58.115 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-03-21T10:24:58.115 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:0", 2024-03-21T10:24:58.115 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 3340746132 2024-03-21T10:24:58.115 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-21T10:24:58.116 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:24:58.116 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": { 2024-03-21T10:24:58.116 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-03-21T10:24:58.116 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:0", 2024-03-21T10:24:58.116 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 3340746132 2024-03-21T10:24:58.116 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:24:58.116 INFO:teuthology.orchestra.run.smithi047.stdout: "inst_str": "client.15498 172.21.15.47:0/3340746132", 2024-03-21T10:24:58.116 INFO:teuthology.orchestra.run.smithi047.stdout: "addr_str": "172.21.15.47:0/3340746132", 2024-03-21T10:24:58.116 INFO:teuthology.orchestra.run.smithi047.stdout: "inode_count": 5, 2024-03-21T10:24:58.116 INFO:teuthology.orchestra.run.smithi047.stdout: "mds_epoch": 29, 2024-03-21T10:24:58.116 INFO:teuthology.orchestra.run.smithi047.stdout: "osd_epoch": 95, 2024-03-21T10:24:58.116 INFO:teuthology.orchestra.run.smithi047.stdout: "osd_epoch_barrier": 0, 2024-03-21T10:24:58.116 INFO:teuthology.orchestra.run.smithi047.stdout: "blocklisted": false, 2024-03-21T10:24:58.116 INFO:teuthology.orchestra.run.smithi047.stdout: "fs_name": "cephfs" 2024-03-21T10:24:58.117 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-03-21T10:24:58.130 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-21T10:24:58.130 DEBUG:teuthology.orchestra.run.smithi047:> stat --file-system '--printf=%T 2024-03-21T10:24:58.130 DEBUG:teuthology.orchestra.run.smithi047:> ' -- /home/ubuntu/cephtest/mnt.0 2024-03-21T10:24:58.181 INFO:teuthology.orchestra.run.smithi047.stdout:fuseblk 2024-03-21T10:24:58.181 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2024-03-21T10:24:58.181 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-21T10:24:58.182 DEBUG:teuthology.orchestra.run.smithi047:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-03-21T10:24:58.268 INFO:teuthology.run_tasks:Running task exec... 2024-03-21T10:24:58.278 INFO:teuthology.task.exec:Executing custom commands... 2024-03-21T10:24:58.278 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi047.front.sepia.ceph.com 2024-03-21T10:24:58.278 DEBUG:teuthology.orchestra.run.smithi047:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs set cephfs balance_automate true' 2024-03-21T10:24:58.497 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:58.494+0000 7fd0df400640 1 -- 172.21.15.47:0/3739082351 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd0d8104a90 msgr2=0x7fd0d8106e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:24:58.497 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:58.494+0000 7fd0df400640 1 --2- 172.21.15.47:0/3739082351 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd0d8104a90 0x7fd0d8106e80 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fd0c4009860 tx=0x7fd0c402bb70 comp rx=0 tx=0).stop 2024-03-21T10:24:58.497 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:58.495+0000 7fd0df400640 1 -- 172.21.15.47:0/3739082351 shutdown_connections 2024-03-21T10:24:58.497 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:58.495+0000 7fd0df400640 1 --2- 172.21.15.47:0/3739082351 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd0d81073c0 0x7fd0d8109810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:24:58.497 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:58.495+0000 7fd0df400640 1 --2- 172.21.15.47:0/3739082351 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd0d8104a90 0x7fd0d8106e80 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:24:58.497 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:58.495+0000 7fd0df400640 1 --2- 172.21.15.47:0/3739082351 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd0d8102160 0x7fd0d8104550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:24:58.497 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:58.495+0000 7fd0df400640 1 -- 172.21.15.47:0/3739082351 >> 172.21.15.47:0/3739082351 conn(0x7fd0d80fbb50 msgr2=0x7fd0d80fdf90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:24:58.497 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:58.495+0000 7fd0df400640 1 -- 172.21.15.47:0/3739082351 shutdown_connections 2024-03-21T10:24:58.498 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:58.495+0000 7fd0df400640 1 -- 172.21.15.47:0/3739082351 wait complete. 2024-03-21T10:24:58.498 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:58.497+0000 7fd0df400640 1 Processor -- start 2024-03-21T10:24:58.499 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:58.497+0000 7fd0df400640 1 -- start start 2024-03-21T10:24:58.501 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:58.498+0000 7fd0df400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd0d8102160 0x7fd0d8101ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:24:58.501 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:58.498+0000 7fd0df400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd0d8104a90 0x7fd0d8100110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:24:58.501 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:58.498+0000 7fd0df400640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd0d81073c0 0x7fd0d8100650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:24:58.501 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:58.498+0000 7fd0df400640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0d810d210 con 0x7fd0d8102160 2024-03-21T10:24:58.501 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:58.498+0000 7fd0df400640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0d810d090 con 0x7fd0d8104a90 2024-03-21T10:24:58.501 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:58.498+0000 7fd0df400640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0d810d390 con 0x7fd0d81073c0 2024-03-21T10:24:58.502 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:58.498+0000 7fd0dca00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd0d8102160 0x7fd0d8101ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:24:58.502 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:58.498+0000 7fd0dca00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd0d8102160 0x7fd0d8101ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:34412/0 (socket says 172.21.15.47:34412) 2024-03-21T10:24:58.502 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:58.498+0000 7fd0dca00640 1 -- 172.21.15.47:0/441655783 learned_addr learned my addr 172.21.15.47:0/441655783 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:24:58.502 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:58.498+0000 7fd0d7e00640 1 --2- 172.21.15.47:0/441655783 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd0d8104a90 0x7fd0d8100110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:24:58.502 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:58.498+0000 7fd0dd400640 1 --2- 172.21.15.47:0/441655783 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd0d81073c0 0x7fd0d8100650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:24:58.502 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:58.498+0000 7fd0dca00640 1 -- 172.21.15.47:0/441655783 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd0d81073c0 msgr2=0x7fd0d8100650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:24:58.502 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:58.498+0000 7fd0dca00640 1 --2- 172.21.15.47:0/441655783 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd0d81073c0 0x7fd0d8100650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:24:58.502 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:58.498+0000 7fd0dca00640 1 -- 172.21.15.47:0/441655783 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd0d8104a90 msgr2=0x7fd0d8100110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:24:58.502 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:58.498+0000 7fd0dca00640 1 --2- 172.21.15.47:0/441655783 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd0d8104a90 0x7fd0d8100110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:24:58.502 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:58.498+0000 7fd0dca00640 1 -- 172.21.15.47:0/441655783 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd0d8100ee0 con 0x7fd0d8102160 2024-03-21T10:24:58.502 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:58.498+0000 7fd0d7e00640 1 --2- 172.21.15.47:0/441655783 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd0d8104a90 0x7fd0d8100110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T10:24:58.502 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:58.498+0000 7fd0dd400640 1 --2- 172.21.15.47:0/441655783 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd0d81073c0 0x7fd0d8100650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T10:24:58.503 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:58.499+0000 7fd0dca00640 1 --2- 172.21.15.47:0/441655783 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd0d8102160 0x7fd0d8101ad0 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7fd0cc00c420 tx=0x7fd0cc009f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:24:58.503 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:58.499+0000 7fd0d5600640 1 -- 172.21.15.47:0/441655783 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0cc003970 con 0x7fd0d8102160 2024-03-21T10:24:58.503 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:58.499+0000 7fd0d5600640 1 -- 172.21.15.47:0/441655783 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fd0cc004020 con 0x7fd0d8102160 2024-03-21T10:24:58.503 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:58.499+0000 7fd0df400640 1 -- 172.21.15.47:0/441655783 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd0d81a19a0 con 0x7fd0d8102160 2024-03-21T10:24:58.503 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:58.499+0000 7fd0d5600640 1 -- 172.21.15.47:0/441655783 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0cc00b960 con 0x7fd0d8102160 2024-03-21T10:24:58.503 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:58.499+0000 7fd0df400640 1 -- 172.21.15.47:0/441655783 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd0d81a1f40 con 0x7fd0d8102160 2024-03-21T10:24:58.503 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:58.501+0000 7fd0df400640 1 -- 172.21.15.47:0/441655783 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd0d80fd640 con 0x7fd0d8102160 2024-03-21T10:24:58.504 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:58.503+0000 7fd0d5600640 1 -- 172.21.15.47:0/441655783 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 18) v1 ==== 137854+0+0 (secure 0 0 0) 0x7fd0cc0041d0 con 0x7fd0d8102160 2024-03-21T10:24:58.507 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:58.505+0000 7fd0d5600640 1 --2- 172.21.15.47:0/441655783 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7fd0c00a3040 0x7fd0c00a5500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:24:58.507 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:58.505+0000 7fd0d5600640 1 -- 172.21.15.47:0/441655783 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9216+0+0 (secure 0 0 0) 0x7fd0cc0cfb00 con 0x7fd0d8102160 2024-03-21T10:24:58.507 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:58.505+0000 7fd0d7e00640 1 --2- 172.21.15.47:0/441655783 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7fd0c00a3040 0x7fd0c00a5500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:24:58.508 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:58.506+0000 7fd0d7e00640 1 --2- 172.21.15.47:0/441655783 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7fd0c00a3040 0x7fd0c00a5500 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fd0c402c0b0 tx=0x7fd0c40058b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:24:58.511 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:58.509+0000 7fd0d5600640 1 -- 172.21.15.47:0/441655783 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fd0cc09c520 con 0x7fd0d8102160 2024-03-21T10:24:58.786 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:58.784+0000 7fd0df400640 1 -- 172.21.15.47:0/441655783 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"} v 0) v1 -- 0x7fd0d802d070 con 0x7fd0d8102160 2024-03-21T10:24:59.313 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:59.311+0000 7fd0d5600640 1 -- 172.21.15.47:0/441655783 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]=0 v30) v1 ==== 117+0+0 (secure 0 0 0) 0x7fd0d802d070 con 0x7fd0d8102160 2024-03-21T10:24:59.319 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:59.316+0000 7fd0df400640 1 -- 172.21.15.47:0/441655783 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7fd0c00a3040 msgr2=0x7fd0c00a5500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:24:59.319 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:59.316+0000 7fd0df400640 1 --2- 172.21.15.47:0/441655783 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7fd0c00a3040 0x7fd0c00a5500 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fd0c402c0b0 tx=0x7fd0c40058b0 comp rx=0 tx=0).stop 2024-03-21T10:24:59.319 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:59.316+0000 7fd0df400640 1 -- 172.21.15.47:0/441655783 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd0d8102160 msgr2=0x7fd0d8101ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:24:59.319 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:59.316+0000 7fd0df400640 1 --2- 172.21.15.47:0/441655783 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd0d8102160 0x7fd0d8101ad0 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7fd0cc00c420 tx=0x7fd0cc009f90 comp rx=0 tx=0).stop 2024-03-21T10:24:59.319 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:59.316+0000 7fd0df400640 1 -- 172.21.15.47:0/441655783 shutdown_connections 2024-03-21T10:24:59.319 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:59.316+0000 7fd0df400640 1 --2- 172.21.15.47:0/441655783 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7fd0c00a3040 0x7fd0c00a5500 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:24:59.319 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:59.316+0000 7fd0df400640 1 --2- 172.21.15.47:0/441655783 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd0d81073c0 0x7fd0d8100650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:24:59.319 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:59.316+0000 7fd0df400640 1 --2- 172.21.15.47:0/441655783 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd0d8104a90 0x7fd0d8100110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:24:59.319 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:59.316+0000 7fd0df400640 1 --2- 172.21.15.47:0/441655783 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd0d8102160 0x7fd0d8101ad0 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:24:59.320 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:59.316+0000 7fd0df400640 1 -- 172.21.15.47:0/441655783 >> 172.21.15.47:0/441655783 conn(0x7fd0d80fbb50 msgr2=0x7fd0d8102b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:24:59.320 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:59.317+0000 7fd0df400640 1 -- 172.21.15.47:0/441655783 shutdown_connections 2024-03-21T10:24:59.320 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:59.317+0000 7fd0df400640 1 -- 172.21.15.47:0/441655783 wait complete. 2024-03-21T10:24:59.343 INFO:teuthology.run_tasks:Running task check-counter... 2024-03-21T10:24:59.354 INFO:teuthology.run_tasks:Running task fwd_scrub... 2024-03-21T10:24:59.365 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-21T10:24:59.575 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:59.573+0000 7f12c9a00640 1 -- 172.21.15.47:0/2440422520 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f12c41026f0 msgr2=0x7f12c4102b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:24:59.575 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:59.573+0000 7f12c9a00640 1 --2- 172.21.15.47:0/2440422520 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f12c41026f0 0x7f12c4102b70 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f12b0009900 tx=0x7f12b002bc30 comp rx=0 tx=0).stop 2024-03-21T10:24:59.576 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:59.573+0000 7f12c9a00640 1 -- 172.21.15.47:0/2440422520 shutdown_connections 2024-03-21T10:24:59.576 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:59.573+0000 7f12c9a00640 1 --2- 172.21.15.47:0/2440422520 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f12c41030b0 0x7f12c4109940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:24:59.576 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:59.573+0000 7f12c9a00640 1 --2- 172.21.15.47:0/2440422520 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f12c41026f0 0x7f12c4102b70 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:24:59.576 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:59.573+0000 7f12c9a00640 1 --2- 172.21.15.47:0/2440422520 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f12c4101500 0x7f12c4101920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:24:59.576 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:59.573+0000 7f12c9a00640 1 -- 172.21.15.47:0/2440422520 >> 172.21.15.47:0/2440422520 conn(0x7f12c40fca70 msgr2=0x7f12c40feeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:24:59.576 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:59.573+0000 7f12c9a00640 1 -- 172.21.15.47:0/2440422520 shutdown_connections 2024-03-21T10:24:59.576 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:59.573+0000 7f12c9a00640 1 -- 172.21.15.47:0/2440422520 wait complete. 2024-03-21T10:24:59.576 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:59.575+0000 7f12c9a00640 1 Processor -- start 2024-03-21T10:24:59.577 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:59.575+0000 7f12c9a00640 1 -- start start 2024-03-21T10:24:59.578 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:59.575+0000 7f12c9a00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f12c4101500 0x7f12c4075450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:24:59.578 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:59.575+0000 7f12c9a00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f12c41026f0 0x7f12c4073a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:24:59.578 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:59.576+0000 7f12c9a00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f12c41030b0 0x7f12c4073fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:24:59.579 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:59.576+0000 7f12c9a00640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12c410d090 con 0x7f12c4101500 2024-03-21T10:24:59.579 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:59.576+0000 7f12c9a00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12c410cf10 con 0x7f12c41026f0 2024-03-21T10:24:59.579 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:59.576+0000 7f12c9a00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12c410d210 con 0x7f12c41030b0 2024-03-21T10:24:59.579 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:59.576+0000 7f12c3e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f12c4101500 0x7f12c4075450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:24:59.579 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:59.576+0000 7f12c3e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f12c4101500 0x7f12c4075450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:34434/0 (socket says 172.21.15.47:34434) 2024-03-21T10:24:59.579 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:59.576+0000 7f12c3e00640 1 -- 172.21.15.47:0/440140478 learned_addr learned my addr 172.21.15.47:0/440140478 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:24:59.579 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:59.576+0000 7f12c3400640 1 --2- 172.21.15.47:0/440140478 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f12c41026f0 0x7f12c4073a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:24:59.579 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:59.576+0000 7f12c9000640 1 --2- 172.21.15.47:0/440140478 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f12c41030b0 0x7f12c4073fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:24:59.579 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:59.576+0000 7f12c3e00640 1 -- 172.21.15.47:0/440140478 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f12c41030b0 msgr2=0x7f12c4073fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:24:59.579 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:59.576+0000 7f12c3e00640 1 --2- 172.21.15.47:0/440140478 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f12c41030b0 0x7f12c4073fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:24:59.579 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:59.576+0000 7f12c3e00640 1 -- 172.21.15.47:0/440140478 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f12c41026f0 msgr2=0x7f12c4073a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:24:59.579 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:59.576+0000 7f12c3e00640 1 --2- 172.21.15.47:0/440140478 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f12c41026f0 0x7f12c4073a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:24:59.579 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:59.576+0000 7f12c3e00640 1 -- 172.21.15.47:0/440140478 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f12c4074860 con 0x7f12c4101500 2024-03-21T10:24:59.579 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:59.576+0000 7f12c9000640 1 --2- 172.21.15.47:0/440140478 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f12c41030b0 0x7f12c4073fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T10:24:59.580 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:59.577+0000 7f12c3e00640 1 --2- 172.21.15.47:0/440140478 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f12c4101500 0x7f12c4075450 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f12b400d950 tx=0x7f12b400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:24:59.580 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:59.577+0000 7f12c0c00640 1 -- 172.21.15.47:0/440140478 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12b4017070 con 0x7f12c4101500 2024-03-21T10:24:59.580 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:59.577+0000 7f12c9a00640 1 -- 172.21.15.47:0/440140478 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f12c41a1860 con 0x7f12c4101500 2024-03-21T10:24:59.580 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:59.577+0000 7f12c0c00640 1 -- 172.21.15.47:0/440140478 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f12b4009c30 con 0x7f12c4101500 2024-03-21T10:24:59.581 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:59.577+0000 7f12c0c00640 1 -- 172.21.15.47:0/440140478 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12b400f620 con 0x7f12c4101500 2024-03-21T10:24:59.581 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:59.577+0000 7f12c9a00640 1 -- 172.21.15.47:0/440140478 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f12c41a1d80 con 0x7f12c4101500 2024-03-21T10:24:59.581 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:59.578+0000 7f12c9a00640 1 -- 172.21.15.47:0/440140478 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f12c4108870 con 0x7f12c4101500 2024-03-21T10:24:59.582 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:59.580+0000 7f12c0c00640 1 -- 172.21.15.47:0/440140478 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 18) v1 ==== 137854+0+0 (secure 0 0 0) 0x7f12b401a070 con 0x7f12c4101500 2024-03-21T10:24:59.584 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:59.582+0000 7f12c0c00640 1 --2- 172.21.15.47:0/440140478 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f12a80a2fa0 0x7f12a80a5460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:24:59.584 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:59.583+0000 7f12c0c00640 1 -- 172.21.15.47:0/440140478 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9216+0+0 (secure 0 0 0) 0x7f12b40cf1f0 con 0x7f12c4101500 2024-03-21T10:24:59.585 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:59.583+0000 7f12c3400640 1 --2- 172.21.15.47:0/440140478 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f12a80a2fa0 0x7f12a80a5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:24:59.585 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:59.584+0000 7f12c3400640 1 --2- 172.21.15.47:0/440140478 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f12a80a2fa0 0x7f12a80a5460 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f12b0000c00 tx=0x7f12b002c7e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:24:59.589 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:59.588+0000 7f12c0c00640 1 -- 172.21.15.47:0/440140478 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f12b4009de0 con 0x7f12c4101500 2024-03-21T10:24:59.713 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:24:59 smithi047 ceph-mon[31547]: pgmap v461: 129 pgs: 129 active+clean; 643 KiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 426 B/s wr, 10 op/s 2024-03-21T10:24:59.714 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:24:59 smithi047 ceph-mon[31547]: from='client.? 172.21.15.47:0/441655783' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]: dispatch 2024-03-21T10:24:59.729 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:24:59 smithi100 ceph-mon[35193]: pgmap v461: 129 pgs: 129 active+clean; 643 KiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 426 B/s wr, 10 op/s 2024-03-21T10:24:59.729 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:24:59 smithi100 ceph-mon[35193]: from='client.? 172.21.15.47:0/441655783' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]: dispatch 2024-03-21T10:24:59.739 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:24:59 smithi157 ceph-mon[35581]: pgmap v461: 129 pgs: 129 active+clean; 643 KiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 426 B/s wr, 10 op/s 2024-03-21T10:24:59.739 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:24:59 smithi157 ceph-mon[35581]: from='client.? 172.21.15.47:0/441655783' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]: dispatch 2024-03-21T10:24:59.797 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:59.795+0000 7f12c9a00640 1 -- 172.21.15.47:0/440140478 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f12c40622a0 con 0x7f12c4101500 2024-03-21T10:24:59.799 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:59.797+0000 7f12c0c00640 1 -- 172.21.15.47:0/440140478 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v95) v1 ==== 74+0+24849 (secure 0 0 0) 0x7f12b409bc10 con 0x7f12c4101500 2024-03-21T10:24:59.800 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-21T10:24:59.800 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":95,"fsid":"3056d51a-e76b-11ee-95cd-87774f69a715","created":"2024-03-21T10:11:17.019965+0000","modified":"2024-03-21T10:23:27.619804+0000","last_up_change":"2024-03-21T10:20:52.675328+0000","last_in_change":"2024-03-21T10:20:27.648212+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-21T10:15:40.553114+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-21T10:21:56.399518+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-21T10:21:57.812940+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-21T10:22:01.822568+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"91","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"b0c7a102-8d89-428b-b793-cbbe67151f50","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6802","nonce":775837313},{"type":"v1","addr":"172.21.15.47:6803","nonce":775837313}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6804","nonce":775837313},{"type":"v1","addr":"172.21.15.47:6805","nonce":775837313}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6808","nonce":775837313},{"type":"v1","addr":"172.21.15.47:6809","nonce":775837313}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6806","nonce":775837313},{"type":"v1","addr":"172.21.15.47:6807","nonce":775837313}]},"public_addr":"172.21.15.47:6803/775837313","cluster_addr":"172.21.15.47:6805/775837313","heartbeat_back_addr":"172.21.15.47:6809/775837313","heartbeat_front_addr":"172.21.15.47:6807/775837313","state":["exists","up"]},{"osd":1,"uuid":"07bb81bb-7cf3-4bbc-ac9e-802f95185d11","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6800","nonce":2923304710},{"type":"v1","addr":"172.21.15.100:6801","nonce":2923304710}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":2923304710},{"type":"v1","addr":"172.21.15.100:6803","nonce":2923304710}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":2923304710},{"type":"v1","addr":"172.21.15.100:6807","nonce":2923304710}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6804","nonce":2923304710},{"type":"v1","addr":"172.21.15.100:6805","nonce":2923304710}]},"public_addr":"172.21.15.100:6801/2923304710","cluster_addr":"172.21.15.100:6803/2923304710","heartbeat_back_addr":"172.21.15.100:6807/2923304710","heartbeat_front_addr":"172.21.15.100:6805/2923304710","state":["exists","up"]},{"osd":2,"uuid":"88ce76b9-6081-4836-aad0-8c249c56e8a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":171377005},{"type":"v1","addr":"172.21.15.157:6801","nonce":171377005}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":171377005},{"type":"v1","addr":"172.21.15.157:6803","nonce":171377005}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":171377005},{"type":"v1","addr":"172.21.15.157:6807","nonce":171377005}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":171377005},{"type":"v1","addr":"172.21.15.157:6805","nonce":171377005}]},"public_addr":"172.21.15.157:6801/171377005","cluster_addr":"172.21.15.157:6803/171377005","heartbeat_back_addr":"172.21.15.157:6807/171377005","heartbeat_front_addr":"172.21.15.157:6805/171377005","state":["exists","up"]},{"osd":3,"uuid":"518aa33d-263f-45e6-aab5-2201ceb40e74","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":27,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6810","nonce":640760101},{"type":"v1","addr":"172.21.15.47:6811","nonce":640760101}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6812","nonce":640760101},{"type":"v1","addr":"172.21.15.47:6813","nonce":640760101}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6816","nonce":640760101},{"type":"v1","addr":"172.21.15.47:6817","nonce":640760101}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6814","nonce":640760101},{"type":"v1","addr":"172.21.15.47:6815","nonce":640760101}]},"public_addr":"172.21.15.47:6811/640760101","cluster_addr":"172.21.15.47:6813/640760101","heartbeat_back_addr":"172.21.15.47:6817/640760101","heartbeat_front_addr":"172.21.15.47:6815/640760101","state":["exists","up"]},{"osd":4,"uuid":"987bde83-ecc6-42e5-bd56-c143b3de4226","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":35,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6808","nonce":2785478639},{"type":"v1","addr":"172.21.15.100:6809","nonce":2785478639}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6810","nonce":2785478639},{"type":"v1","addr":"172.21.15.100:6811","nonce":2785478639}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6814","nonce":2785478639},{"type":"v1","addr":"172.21.15.100:6815","nonce":2785478639}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":2785478639},{"type":"v1","addr":"172.21.15.100:6813","nonce":2785478639}]},"public_addr":"172.21.15.100:6809/2785478639","cluster_addr":"172.21.15.100:6811/2785478639","heartbeat_back_addr":"172.21.15.100:6815/2785478639","heartbeat_front_addr":"172.21.15.100:6813/2785478639","state":["exists","up"]},{"osd":5,"uuid":"012d2188-3a61-4a5f-8d01-2001c15ffbe2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":40,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":1525645410},{"type":"v1","addr":"172.21.15.157:6809","nonce":1525645410}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":1525645410},{"type":"v1","addr":"172.21.15.157:6811","nonce":1525645410}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":1525645410},{"type":"v1","addr":"172.21.15.157:6815","nonce":1525645410}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":1525645410},{"type":"v1","addr":"172.21.15.157:6813","nonce":1525645410}]},"public_addr":"172.21.15.157:6809/1525645410","cluster_addr":"172.21.15.157:6811/1525645410","heartbeat_back_addr":"172.21.15.157:6815/1525645410","heartbeat_front_addr":"172.21.15.157:6813/1525645410","state":["exists","up"]},{"osd":6,"uuid":"c12c14b4-85f0-4d88-937c-de4f556437de","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6818","nonce":2321642790},{"type":"v1","addr":"172.21.15.47:6819","nonce":2321642790}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6820","nonce":2321642790},{"type":"v1","addr":"172.21.15.47:6821","nonce":2321642790}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6824","nonce":2321642790},{"type":"v1","addr":"172.21.15.47:6825","nonce":2321642790}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6822","nonce":2321642790},{"type":"v1","addr":"172.21.15.47:6823","nonce":2321642790}]},"public_addr":"172.21.15.47:6819/2321642790","cluster_addr":"172.21.15.47:6821/2321642790","heartbeat_back_addr":"172.21.15.47:6825/2321642790","heartbeat_front_addr":"172.21.15.47:6823/2321642790","state":["exists","up"]},{"osd":7,"uuid":"3c87c6d0-1d1f-4294-9917-aaaad7dda824","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6816","nonce":4104727330},{"type":"v1","addr":"172.21.15.100:6817","nonce":4104727330}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":4104727330},{"type":"v1","addr":"172.21.15.100:6819","nonce":4104727330}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":4104727330},{"type":"v1","addr":"172.21.15.100:6823","nonce":4104727330}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6820","nonce":4104727330},{"type":"v1","addr":"172.21.15.100:6821","nonce":4104727330}]},"public_addr":"172.21.15.100:6817/4104727330","cluster_addr":"172.21.15.100:6819/4104727330","heartbeat_back_addr":"172.21.15.100:6823/4104727330","heartbeat_front_addr":"172.21.15.100:6821/4104727330","state":["exists","up"]},{"osd":8,"uuid":"502d65cf-53c8-4936-8de4-e11b2a6e1e18","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":61,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":4067715347},{"type":"v1","addr":"172.21.15.157:6817","nonce":4067715347}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":4067715347},{"type":"v1","addr":"172.21.15.157:6819","nonce":4067715347}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":4067715347},{"type":"v1","addr":"172.21.15.157:6823","nonce":4067715347}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":4067715347},{"type":"v1","addr":"172.21.15.157:6821","nonce":4067715347}]},"public_addr":"172.21.15.157:6817/4067715347","cluster_addr":"172.21.15.157:6819/4067715347","heartbeat_back_addr":"172.21.15.157:6823/4067715347","heartbeat_front_addr":"172.21.15.157:6821/4067715347","state":["exists","up"]},{"osd":9,"uuid":"5ea1f5be-8458-4c93-9e61-c2e867a514f3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6826","nonce":1982108738},{"type":"v1","addr":"172.21.15.47:6827","nonce":1982108738}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6828","nonce":1982108738},{"type":"v1","addr":"172.21.15.47:6829","nonce":1982108738}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6832","nonce":1982108738},{"type":"v1","addr":"172.21.15.47:6833","nonce":1982108738}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6830","nonce":1982108738},{"type":"v1","addr":"172.21.15.47:6831","nonce":1982108738}]},"public_addr":"172.21.15.47:6827/1982108738","cluster_addr":"172.21.15.47:6829/1982108738","heartbeat_back_addr":"172.21.15.47:6833/1982108738","heartbeat_front_addr":"172.21.15.47:6831/1982108738","state":["exists","up"]},{"osd":10,"uuid":"2da3da90-a0be-4a54-a9ff-b2d6c9b36870","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":74,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":396391028},{"type":"v1","addr":"172.21.15.100:6825","nonce":396391028}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":396391028},{"type":"v1","addr":"172.21.15.100:6827","nonce":396391028}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6830","nonce":396391028},{"type":"v1","addr":"172.21.15.100:6831","nonce":396391028}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":396391028},{"type":"v1","addr":"172.21.15.100:6829","nonce":396391028}]},"public_addr":"172.21.15.100:6825/396391028","cluster_addr":"172.21.15.100:6827/396391028","heartbeat_back_addr":"172.21.15.100:6831/396391028","heartbeat_front_addr":"172.21.15.100:6829/396391028","state":["exists","up"]},{"osd":11,"uuid":"11798d41-7f7e-4319-95b8-8d3d0946bde9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":82,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":2671029483},{"type":"v1","addr":"172.21.15.157:6825","nonce":2671029483}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":2671029483},{"type":"v1","addr":"172.21.15.157:6827","nonce":2671029483}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":2671029483},{"type":"v1","addr":"172.21.15.157:6831","nonce":2671029483}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":2671029483},{"type":"v1","addr":"172.21.15.157:6829","nonce":2671029483}]},"public_addr":"172.21.15.157:6825/2671029483","cluster_addr":"172.21.15.157:6827/2671029483","heartbeat_back_addr":"172.21.15.157:6831/2671029483","heartbeat_front_addr":"172.21.15.157:6829/2671029483","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T10:14:31.822399+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T10:15:04.175721+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T10:15:35.902595+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T10:16:12.088891+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T10:16:46.428704+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T10:17:20.158270+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T10:17:55.915253+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T10:18:30.064641+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T10:19:04.410004+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T10:19:40.605181+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T10:20:15.636605+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T10:20:50.151365+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.11","mappings":[{"from":7,"to":11}]},{"pgid":"2.14","mappings":[{"from":2,"to":4}]},{"pgid":"2.1a","mappings":[{"from":8,"to":1}]},{"pgid":"2.1b","mappings":[{"from":2,"to":4}]},{"pgid":"2.1d","mappings":[{"from":8,"to":11}]},{"pgid":"3.0","mappings":[{"from":4,"to":8},{"from":5,"to":9}]},{"pgid":"3.6","mappings":[{"from":3,"to":7}]},{"pgid":"3.1b","mappings":[{"from":4,"to":6}]},{"pgid":"3.33","mappings":[{"from":5,"to":9}]},{"pgid":"3.3c","mappings":[{"from":4,"to":9}]},{"pgid":"4.c","mappings":[{"from":9,"to":10}]},{"pgid":"4.f","mappings":[{"from":9,"to":8}]},{"pgid":"4.19","mappings":[{"from":9,"to":8}]},{"pgid":"4.1d","mappings":[{"from":5,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.47:0/356822431":"2024-03-22T10:12:25.844530+0000","172.21.15.47:0/1167490879":"2024-03-22T10:12:25.844530+0000","172.21.15.47:0/1540718138":"2024-03-22T10:12:25.844530+0000","172.21.15.47:6801/3836903972":"2024-03-22T10:12:25.844530+0000","172.21.15.47:0/235436602":"2024-03-22T10:11:50.301838+0000","172.21.15.47:0/2052815572":"2024-03-22T10:11:50.301838+0000","172.21.15.47:6800/3942658432":"2024-03-22T10:11:50.301838+0000","172.21.15.47:6801/3942658432":"2024-03-22T10:11:50.301838+0000","172.21.15.47:0/1419516257":"2024-03-22T10:11:50.301838+0000","172.21.15.47:6800/3836903972":"2024-03-22T10:12:25.844530+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-21T10:24:59.803 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:59.801+0000 7f12c9a00640 1 -- 172.21.15.47:0/440140478 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f12a80a2fa0 msgr2=0x7f12a80a5460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:24:59.804 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:59.801+0000 7f12c9a00640 1 --2- 172.21.15.47:0/440140478 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f12a80a2fa0 0x7f12a80a5460 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f12b0000c00 tx=0x7f12b002c7e0 comp rx=0 tx=0).stop 2024-03-21T10:24:59.804 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:59.801+0000 7f12c9a00640 1 -- 172.21.15.47:0/440140478 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f12c4101500 msgr2=0x7f12c4075450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:24:59.804 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:59.801+0000 7f12c9a00640 1 --2- 172.21.15.47:0/440140478 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f12c4101500 0x7f12c4075450 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f12b400d950 tx=0x7f12b400de10 comp rx=0 tx=0).stop 2024-03-21T10:24:59.804 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:59.801+0000 7f12c9a00640 1 -- 172.21.15.47:0/440140478 shutdown_connections 2024-03-21T10:24:59.804 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:59.801+0000 7f12c9a00640 1 --2- 172.21.15.47:0/440140478 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f12a80a2fa0 0x7f12a80a5460 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:24:59.804 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:59.801+0000 7f12c9a00640 1 --2- 172.21.15.47:0/440140478 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f12c41030b0 0x7f12c4073fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:24:59.804 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:59.801+0000 7f12c9a00640 1 --2- 172.21.15.47:0/440140478 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f12c41026f0 0x7f12c4073a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:24:59.804 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:59.801+0000 7f12c9a00640 1 --2- 172.21.15.47:0/440140478 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f12c4101500 0x7f12c4075450 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:24:59.804 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:59.801+0000 7f12c9a00640 1 -- 172.21.15.47:0/440140478 >> 172.21.15.47:0/440140478 conn(0x7f12c40fca70 msgr2=0x7f12c4105a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:24:59.804 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:59.801+0000 7f12c9a00640 1 -- 172.21.15.47:0/440140478 shutdown_connections 2024-03-21T10:24:59.804 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:24:59.801+0000 7f12c9a00640 1 -- 172.21.15.47:0/440140478 wait complete. 2024-03-21T10:24:59.823 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-21T10:15:40.553114+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-21T10:21:56.399518+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '86', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-21T10:21:57.812940+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '87', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-03-21T10:22:01.822568+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '91', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-03-21T10:24:59.824 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-21T10:25:00.045 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:00.043+0000 7f64e4e00640 1 -- 172.21.15.47:0/1636187192 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f64e0102000 msgr2=0x7f64e01043f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:25:00.046 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:00.043+0000 7f64e4e00640 1 --2- 172.21.15.47:0/1636187192 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f64e0102000 0x7f64e01043f0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f64d0009900 tx=0x7f64d002bc40 comp rx=0 tx=0).stop 2024-03-21T10:25:00.046 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:00.043+0000 7f64e4e00640 1 -- 172.21.15.47:0/1636187192 shutdown_connections 2024-03-21T10:25:00.046 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:00.043+0000 7f64e4e00640 1 --2- 172.21.15.47:0/1636187192 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f64e0107260 0x7f64e01096b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:25:00.046 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:00.043+0000 7f64e4e00640 1 --2- 172.21.15.47:0/1636187192 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f64e0104930 0x7f64e0106d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:25:00.046 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:00.043+0000 7f64e4e00640 1 --2- 172.21.15.47:0/1636187192 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f64e0102000 0x7f64e01043f0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:25:00.047 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:00.043+0000 7f64e4e00640 1 -- 172.21.15.47:0/1636187192 >> 172.21.15.47:0/1636187192 conn(0x7f64e00fba10 msgr2=0x7f64e00fde30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:25:00.047 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:00.043+0000 7f64e4e00640 1 -- 172.21.15.47:0/1636187192 shutdown_connections 2024-03-21T10:25:00.047 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:00.043+0000 7f64e4e00640 1 -- 172.21.15.47:0/1636187192 wait complete. 2024-03-21T10:25:00.051 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:00.045+0000 7f64e4e00640 1 Processor -- start 2024-03-21T10:25:00.051 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:00.045+0000 7f64e4e00640 1 -- start start 2024-03-21T10:25:00.051 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:00.045+0000 7f64e4e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f64e0102000 0x7f64e01977e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:25:00.051 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:00.045+0000 7f64e4e00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f64e0104930 0x7f64e0197d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:25:00.052 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:00.045+0000 7f64e4e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f64e0107260 0x7f64e019ed90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:25:00.052 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:00.045+0000 7f64e4e00640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64e010d110 con 0x7f64e0107260 2024-03-21T10:25:00.052 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:00.045+0000 7f64e4e00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64e010cf90 con 0x7f64e0102000 2024-03-21T10:25:00.052 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:00.045+0000 7f64e4e00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64e010d290 con 0x7f64e0104930 2024-03-21T10:25:00.052 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:00.046+0000 7f64df400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f64e0102000 0x7f64e01977e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:25:00.052 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:00.046+0000 7f64df400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f64e0102000 0x7f64e01977e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:36146/0 (socket says 172.21.15.47:36146) 2024-03-21T10:25:00.053 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:00.046+0000 7f64df400640 1 -- 172.21.15.47:0/1774881222 learned_addr learned my addr 172.21.15.47:0/1774881222 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:25:00.053 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:00.046+0000 7f64dfe00640 1 --2- 172.21.15.47:0/1774881222 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f64e0107260 0x7f64e019ed90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:25:00.053 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:00.046+0000 7f64dea00640 1 --2- 172.21.15.47:0/1774881222 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f64e0104930 0x7f64e0197d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:25:00.053 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:00.046+0000 7f64df400640 1 -- 172.21.15.47:0/1774881222 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f64e0104930 msgr2=0x7f64e0197d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:25:00.053 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:00.046+0000 7f64df400640 1 --2- 172.21.15.47:0/1774881222 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f64e0104930 0x7f64e0197d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:25:00.053 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:00.046+0000 7f64df400640 1 -- 172.21.15.47:0/1774881222 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f64e0107260 msgr2=0x7f64e019ed90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:25:00.054 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:00.046+0000 7f64df400640 1 --2- 172.21.15.47:0/1774881222 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f64e0107260 0x7f64e019ed90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:25:00.054 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:00.046+0000 7f64df400640 1 -- 172.21.15.47:0/1774881222 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64e019f400 con 0x7f64e0102000 2024-03-21T10:25:00.054 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:00.047+0000 7f64df400640 1 --2- 172.21.15.47:0/1774881222 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f64e0102000 0x7f64e01977e0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f64d0000c00 tx=0x7f64d00042f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:25:00.054 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:00.047+0000 7f64d7e00640 1 -- 172.21.15.47:0/1774881222 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64d0043020 con 0x7f64e0102000 2024-03-21T10:25:00.054 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:00.047+0000 7f64e4e00640 1 -- 172.21.15.47:0/1774881222 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f64e019f690 con 0x7f64e0102000 2024-03-21T10:25:00.054 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:00.047+0000 7f64d7e00640 1 -- 172.21.15.47:0/1774881222 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f64d00054c0 con 0x7f64e0102000 2024-03-21T10:25:00.054 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:00.047+0000 7f64d7e00640 1 -- 172.21.15.47:0/1774881222 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64d0003d70 con 0x7f64e0102000 2024-03-21T10:25:00.055 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:00.047+0000 7f64e4e00640 1 -- 172.21.15.47:0/1774881222 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f64e019fc30 con 0x7f64e0102000 2024-03-21T10:25:00.055 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:00.048+0000 7f64e4e00640 1 -- 172.21.15.47:0/1774881222 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f64e00fd500 con 0x7f64e0102000 2024-03-21T10:25:00.056 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:00.051+0000 7f64d7e00640 1 -- 172.21.15.47:0/1774881222 <== mon.2 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 18) v1 ==== 137854+0+0 (secure 0 0 0) 0x7f64d0046020 con 0x7f64e0102000 2024-03-21T10:25:00.057 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:00.052+0000 7f64d7e00640 1 --2- 172.21.15.47:0/1774881222 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f64c40a2fa0 0x7f64c40a5460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:25:00.057 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:00.053+0000 7f64d7e00640 1 -- 172.21.15.47:0/1774881222 <== mon.2 v2:172.21.15.100:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9216+0+0 (secure 0 0 0) 0x7f64d00f2100 con 0x7f64e0102000 2024-03-21T10:25:00.057 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:00.053+0000 7f64dea00640 1 --2- 172.21.15.47:0/1774881222 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f64c40a2fa0 0x7f64c40a5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:25:00.057 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:00.054+0000 7f64dea00640 1 --2- 172.21.15.47:0/1774881222 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f64c40a2fa0 0x7f64c40a5460 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f64cc005fd0 tx=0x7f64cc005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:25:00.066 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:00.065+0000 7f64d7e00640 1 -- 172.21.15.47:0/1774881222 <== mon.2 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f64d004d020 con 0x7f64e0102000 2024-03-21T10:25:00.276 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:00.274+0000 7f64e4e00640 1 -- 172.21.15.47:0/1774881222 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f64e00622a0 con 0x7f64e0102000 2024-03-21T10:25:00.277 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:00.275+0000 7f64d7e00640 1 -- 172.21.15.47:0/1774881222 <== mon.2 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v95) v1 ==== 93+0+10 (secure 0 0 0) 0x7f64d00bec40 con 0x7f64e0102000 2024-03-21T10:25:00.277 INFO:teuthology.orchestra.run.smithi047.stdout:pg_num: 1 2024-03-21T10:25:00.280 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:00.278+0000 7f64e4e00640 1 -- 172.21.15.47:0/1774881222 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f64c40a2fa0 msgr2=0x7f64c40a5460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:25:00.280 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:00.278+0000 7f64e4e00640 1 --2- 172.21.15.47:0/1774881222 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f64c40a2fa0 0x7f64c40a5460 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f64cc005fd0 tx=0x7f64cc005850 comp rx=0 tx=0).stop 2024-03-21T10:25:00.280 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:00.278+0000 7f64e4e00640 1 -- 172.21.15.47:0/1774881222 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f64e0102000 msgr2=0x7f64e01977e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:25:00.281 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:00.278+0000 7f64e4e00640 1 --2- 172.21.15.47:0/1774881222 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f64e0102000 0x7f64e01977e0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f64d0000c00 tx=0x7f64d00042f0 comp rx=0 tx=0).stop 2024-03-21T10:25:00.281 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:00.278+0000 7f64e4e00640 1 -- 172.21.15.47:0/1774881222 shutdown_connections 2024-03-21T10:25:00.281 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:00.278+0000 7f64e4e00640 1 --2- 172.21.15.47:0/1774881222 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f64c40a2fa0 0x7f64c40a5460 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:25:00.281 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:00.278+0000 7f64e4e00640 1 --2- 172.21.15.47:0/1774881222 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f64e0107260 0x7f64e019ed90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:25:00.281 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:00.278+0000 7f64e4e00640 1 --2- 172.21.15.47:0/1774881222 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f64e0104930 0x7f64e0197d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:25:00.281 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:00.278+0000 7f64e4e00640 1 --2- 172.21.15.47:0/1774881222 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f64e0102000 0x7f64e01977e0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:25:00.281 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:00.278+0000 7f64e4e00640 1 -- 172.21.15.47:0/1774881222 >> 172.21.15.47:0/1774881222 conn(0x7f64e00fba10 msgr2=0x7f64e0102a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:25:00.281 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:00.278+0000 7f64e4e00640 1 -- 172.21.15.47:0/1774881222 shutdown_connections 2024-03-21T10:25:00.281 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:00.278+0000 7f64e4e00640 1 -- 172.21.15.47:0/1774881222 wait complete. 2024-03-21T10:25:00.305 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-21T10:25:00.526 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:00.523+0000 7f687a000640 1 -- 172.21.15.47:0/3674084052 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6874104940 msgr2=0x7f6874106d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:25:00.526 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:00.523+0000 7f687a000640 1 --2- 172.21.15.47:0/3674084052 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6874104940 0x7f6874106d30 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f6860009860 tx=0x7f686002bb70 comp rx=0 tx=0).stop 2024-03-21T10:25:00.526 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:00.523+0000 7f687a000640 1 -- 172.21.15.47:0/3674084052 shutdown_connections 2024-03-21T10:25:00.526 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:00.523+0000 7f687a000640 1 --2- 172.21.15.47:0/3674084052 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6874107270 0x7f68741096c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:25:00.526 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:00.523+0000 7f687a000640 1 --2- 172.21.15.47:0/3674084052 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6874104940 0x7f6874106d30 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:25:00.527 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:00.523+0000 7f687a000640 1 --2- 172.21.15.47:0/3674084052 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6874102010 0x7f6874104400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:25:00.527 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:00.523+0000 7f687a000640 1 -- 172.21.15.47:0/3674084052 >> 172.21.15.47:0/3674084052 conn(0x7f68740fba20 msgr2=0x7f68740fde40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:25:00.527 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:00.523+0000 7f687a000640 1 -- 172.21.15.47:0/3674084052 shutdown_connections 2024-03-21T10:25:00.527 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:00.523+0000 7f687a000640 1 -- 172.21.15.47:0/3674084052 wait complete. 2024-03-21T10:25:00.527 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:00.524+0000 7f687a000640 1 Processor -- start 2024-03-21T10:25:00.529 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:00.525+0000 7f687a000640 1 -- start start 2024-03-21T10:25:00.529 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:00.525+0000 7f687a000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6874102010 0x7f6874199a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:25:00.529 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:00.525+0000 7f687a000640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6874104940 0x7f6874199f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:25:00.529 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:00.525+0000 7f687a000640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6874107270 0x7f68741a0fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:25:00.529 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:00.525+0000 7f687a000640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f687410d0d0 con 0x7f6874104940 2024-03-21T10:25:00.529 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:00.525+0000 7f687a000640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f687410cf50 con 0x7f6874102010 2024-03-21T10:25:00.529 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:00.525+0000 7f687a000640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f687410d250 con 0x7f6874107270 2024-03-21T10:25:00.529 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:00.525+0000 7f6873e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6874104940 0x7f6874199f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:25:00.529 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:00.525+0000 7f6878c00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6874102010 0x7f6874199a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:25:00.529 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:00.525+0000 7f6873e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6874104940 0x7f6874199f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:34468/0 (socket says 172.21.15.47:34468) 2024-03-21T10:25:00.529 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:00.525+0000 7f6873e00640 1 -- 172.21.15.47:0/1703830837 learned_addr learned my addr 172.21.15.47:0/1703830837 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:25:00.530 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:00.526+0000 7f6879600640 1 --2- 172.21.15.47:0/1703830837 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6874107270 0x7f68741a0fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:25:00.530 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:00.526+0000 7f6873e00640 1 -- 172.21.15.47:0/1703830837 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6874107270 msgr2=0x7f68741a0fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:25:00.530 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:00.526+0000 7f6873e00640 1 --2- 172.21.15.47:0/1703830837 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6874107270 0x7f68741a0fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:25:00.530 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:00.526+0000 7f6873e00640 1 -- 172.21.15.47:0/1703830837 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6874102010 msgr2=0x7f6874199a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:25:00.530 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:00.526+0000 7f6873e00640 1 --2- 172.21.15.47:0/1703830837 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6874102010 0x7f6874199a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:25:00.530 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:00.526+0000 7f6873e00640 1 -- 172.21.15.47:0/1703830837 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68741a1620 con 0x7f6874104940 2024-03-21T10:25:00.530 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:00.526+0000 7f6878c00640 1 --2- 172.21.15.47:0/1703830837 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6874102010 0x7f6874199a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T10:25:00.530 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:00.526+0000 7f6873e00640 1 --2- 172.21.15.47:0/1703830837 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6874104940 0x7f6874199f40 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f68600057c0 tx=0x7f68600042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:25:00.530 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:00.527+0000 7f6871600640 1 -- 172.21.15.47:0/1703830837 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6860041030 con 0x7f6874104940 2024-03-21T10:25:00.530 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:00.527+0000 7f687a000640 1 -- 172.21.15.47:0/1703830837 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f68741a18b0 con 0x7f6874104940 2024-03-21T10:25:00.531 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:00.527+0000 7f6871600640 1 -- 172.21.15.47:0/1703830837 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f686002c9c0 con 0x7f6874104940 2024-03-21T10:25:00.531 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:00.527+0000 7f6871600640 1 -- 172.21.15.47:0/1703830837 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6860039040 con 0x7f6874104940 2024-03-21T10:25:00.531 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:00.527+0000 7f687a000640 1 -- 172.21.15.47:0/1703830837 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f68741a1da0 con 0x7f6874104940 2024-03-21T10:25:00.531 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:00.528+0000 7f687a000640 1 -- 172.21.15.47:0/1703830837 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6874108dc0 con 0x7f6874104940 2024-03-21T10:25:00.533 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:00.532+0000 7f6871600640 1 -- 172.21.15.47:0/1703830837 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 18) v1 ==== 137854+0+0 (secure 0 0 0) 0x7f6860044070 con 0x7f6874104940 2024-03-21T10:25:00.535 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:00.533+0000 7f6871600640 1 --2- 172.21.15.47:0/1703830837 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f685c0a2f20 0x7f685c0a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:25:00.535 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:00.533+0000 7f6871600640 1 -- 172.21.15.47:0/1703830837 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9216+0+0 (secure 0 0 0) 0x7f68600f0ec0 con 0x7f6874104940 2024-03-21T10:25:00.535 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:00.533+0000 7f6878c00640 1 --2- 172.21.15.47:0/1703830837 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f685c0a2f20 0x7f685c0a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:25:00.536 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:00.534+0000 7f6878c00640 1 --2- 172.21.15.47:0/1703830837 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f685c0a2f20 0x7f685c0a53e0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f68680096c0 tx=0x7f6868009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:25:00.540 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:00.538+0000 7f6871600640 1 -- 172.21.15.47:0/1703830837 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f68600bd8e0 con 0x7f6874104940 2024-03-21T10:25:00.713 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:25:00 smithi047 ceph-mon[31547]: from='client.? 172.21.15.47:0/441655783' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]': finished 2024-03-21T10:25:00.714 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:25:00 smithi047 ceph-mon[31547]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-21T10:25:00.714 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:25:00 smithi047 ceph-mon[31547]: from='client.? 172.21.15.47:0/440140478' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-21T10:25:00.714 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:25:00 smithi047 ceph-mon[31547]: from='client.? 172.21.15.47:0/1774881222' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-21T10:25:00.729 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:25:00 smithi100 ceph-mon[35193]: from='client.? 172.21.15.47:0/441655783' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]': finished 2024-03-21T10:25:00.729 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:25:00 smithi100 ceph-mon[35193]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-21T10:25:00.729 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:25:00 smithi100 ceph-mon[35193]: from='client.? 172.21.15.47:0/440140478' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-21T10:25:00.729 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:25:00 smithi100 ceph-mon[35193]: from='client.? 172.21.15.47:0/1774881222' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-21T10:25:00.739 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:25:00 smithi157 ceph-mon[35581]: from='client.? 172.21.15.47:0/441655783' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]': finished 2024-03-21T10:25:00.739 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:25:00 smithi157 ceph-mon[35581]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-21T10:25:00.740 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:25:00 smithi157 ceph-mon[35581]: from='client.? 172.21.15.47:0/440140478' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-21T10:25:00.740 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:25:00 smithi157 ceph-mon[35581]: from='client.? 172.21.15.47:0/1774881222' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-21T10:25:00.765 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:00.763+0000 7f687a000640 1 -- 172.21.15.47:0/1703830837 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f68740622a0 con 0x7f6874104940 2024-03-21T10:25:00.766 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:00.764+0000 7f6871600640 1 -- 172.21.15.47:0/1703830837 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v95) v1 ==== 104+0+11 (secure 0 0 0) 0x7f68600bd1e0 con 0x7f6874104940 2024-03-21T10:25:00.766 INFO:teuthology.orchestra.run.smithi047.stdout:pg_num: 32 2024-03-21T10:25:00.770 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:00.768+0000 7f687a000640 1 -- 172.21.15.47:0/1703830837 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f685c0a2f20 msgr2=0x7f685c0a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:25:00.770 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:00.768+0000 7f687a000640 1 --2- 172.21.15.47:0/1703830837 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f685c0a2f20 0x7f685c0a53e0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f68680096c0 tx=0x7f6868009260 comp rx=0 tx=0).stop 2024-03-21T10:25:00.770 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:00.768+0000 7f687a000640 1 -- 172.21.15.47:0/1703830837 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6874104940 msgr2=0x7f6874199f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:25:00.770 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:00.768+0000 7f687a000640 1 --2- 172.21.15.47:0/1703830837 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6874104940 0x7f6874199f40 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f68600057c0 tx=0x7f68600042f0 comp rx=0 tx=0).stop 2024-03-21T10:25:00.770 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:00.768+0000 7f687a000640 1 -- 172.21.15.47:0/1703830837 shutdown_connections 2024-03-21T10:25:00.770 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:00.768+0000 7f687a000640 1 --2- 172.21.15.47:0/1703830837 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f685c0a2f20 0x7f685c0a53e0 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:25:00.770 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:00.768+0000 7f687a000640 1 --2- 172.21.15.47:0/1703830837 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6874107270 0x7f68741a0fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:25:00.770 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:00.768+0000 7f687a000640 1 --2- 172.21.15.47:0/1703830837 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6874104940 0x7f6874199f40 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:25:00.770 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:00.768+0000 7f687a000640 1 --2- 172.21.15.47:0/1703830837 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6874102010 0x7f6874199a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:25:00.770 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:00.768+0000 7f687a000640 1 -- 172.21.15.47:0/1703830837 >> 172.21.15.47:0/1703830837 conn(0x7f68740fba20 msgr2=0x7f68741018a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:25:00.771 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:00.768+0000 7f687a000640 1 -- 172.21.15.47:0/1703830837 shutdown_connections 2024-03-21T10:25:00.771 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:00.768+0000 7f687a000640 1 -- 172.21.15.47:0/1703830837 wait complete. 2024-03-21T10:25:00.789 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-21T10:25:01.007 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:01.005+0000 7f2c7aa00640 1 -- 172.21.15.47:0/2625900929 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2c74107380 msgr2=0x7f2c741097d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:25:01.008 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:01.005+0000 7f2c7aa00640 1 --2- 172.21.15.47:0/2625900929 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2c74107380 0x7f2c741097d0 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f2c6c009be0 tx=0x7f2c6c02bef0 comp rx=0 tx=0).stop 2024-03-21T10:25:01.008 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:01.005+0000 7f2c7aa00640 1 -- 172.21.15.47:0/2625900929 shutdown_connections 2024-03-21T10:25:01.008 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:01.005+0000 7f2c7aa00640 1 --2- 172.21.15.47:0/2625900929 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2c74107380 0x7f2c741097d0 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:25:01.008 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:01.005+0000 7f2c7aa00640 1 --2- 172.21.15.47:0/2625900929 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2c74104a50 0x7f2c74106e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:25:01.008 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:01.005+0000 7f2c7aa00640 1 --2- 172.21.15.47:0/2625900929 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2c74102120 0x7f2c74104510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:25:01.008 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:01.005+0000 7f2c7aa00640 1 -- 172.21.15.47:0/2625900929 >> 172.21.15.47:0/2625900929 conn(0x7f2c740fbb30 msgr2=0x7f2c740fdf50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:25:01.008 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:01.005+0000 7f2c7aa00640 1 -- 172.21.15.47:0/2625900929 shutdown_connections 2024-03-21T10:25:01.008 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:01.005+0000 7f2c7aa00640 1 -- 172.21.15.47:0/2625900929 wait complete. 2024-03-21T10:25:01.010 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:01.006+0000 7f2c7aa00640 1 Processor -- start 2024-03-21T10:25:01.011 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:01.007+0000 7f2c7aa00640 1 -- start start 2024-03-21T10:25:01.011 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:01.007+0000 7f2c7aa00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2c74102120 0x7f2c74197910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:25:01.011 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:01.007+0000 7f2c7aa00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2c74104a50 0x7f2c74197e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:25:01.011 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:01.007+0000 7f2c7aa00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2c74107380 0x7f2c7419eec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:25:01.011 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:01.007+0000 7f2c7aa00640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c7410d240 con 0x7f2c74107380 2024-03-21T10:25:01.011 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:01.007+0000 7f2c7aa00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c7410d0c0 con 0x7f2c74102120 2024-03-21T10:25:01.011 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:01.007+0000 7f2c7aa00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c7410d3c0 con 0x7f2c74104a50 2024-03-21T10:25:01.011 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:01.007+0000 7f2c78a00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2c74107380 0x7f2c7419eec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:25:01.011 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:01.007+0000 7f2c73e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2c74102120 0x7f2c74197910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:25:01.012 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:01.007+0000 7f2c78a00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2c74107380 0x7f2c7419eec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:34492/0 (socket says 172.21.15.47:34492) 2024-03-21T10:25:01.012 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:01.007+0000 7f2c78a00640 1 -- 172.21.15.47:0/2095192730 learned_addr learned my addr 172.21.15.47:0/2095192730 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:25:01.012 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:01.007+0000 7f2c73e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2c74102120 0x7f2c74197910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:36162/0 (socket says 172.21.15.47:36162) 2024-03-21T10:25:01.012 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:01.008+0000 7f2c73400640 1 --2- 172.21.15.47:0/2095192730 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2c74104a50 0x7f2c74197e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:25:01.012 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:01.008+0000 7f2c78a00640 1 -- 172.21.15.47:0/2095192730 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2c74104a50 msgr2=0x7f2c74197e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:25:01.012 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:01.008+0000 7f2c78a00640 1 --2- 172.21.15.47:0/2095192730 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2c74104a50 0x7f2c74197e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:25:01.012 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:01.008+0000 7f2c78a00640 1 -- 172.21.15.47:0/2095192730 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2c74102120 msgr2=0x7f2c74197910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:25:01.012 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:01.008+0000 7f2c78a00640 1 --2- 172.21.15.47:0/2095192730 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2c74102120 0x7f2c74197910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:25:01.012 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:01.008+0000 7f2c78a00640 1 -- 172.21.15.47:0/2095192730 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c7419f530 con 0x7f2c74107380 2024-03-21T10:25:01.012 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:01.008+0000 7f2c73e00640 1 --2- 172.21.15.47:0/2095192730 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2c74102120 0x7f2c74197910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T10:25:01.012 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:01.008+0000 7f2c73400640 1 --2- 172.21.15.47:0/2095192730 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2c74104a50 0x7f2c74197e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T10:25:01.012 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:01.008+0000 7f2c78a00640 1 --2- 172.21.15.47:0/2095192730 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2c74107380 0x7f2c7419eec0 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f2c6c034fd0 tx=0x7f2c6c02cbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:25:01.012 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:01.009+0000 7f2c70c00640 1 -- 172.21.15.47:0/2095192730 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c6c043020 con 0x7f2c74107380 2024-03-21T10:25:01.012 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:01.009+0000 7f2c70c00640 1 -- 172.21.15.47:0/2095192730 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f2c6c003e20 con 0x7f2c74107380 2024-03-21T10:25:01.013 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:01.009+0000 7f2c7aa00640 1 -- 172.21.15.47:0/2095192730 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2c7419f7c0 con 0x7f2c74107380 2024-03-21T10:25:01.013 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:01.009+0000 7f2c70c00640 1 -- 172.21.15.47:0/2095192730 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c6c034430 con 0x7f2c74107380 2024-03-21T10:25:01.013 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:01.009+0000 7f2c7aa00640 1 -- 172.21.15.47:0/2095192730 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2c7419fd10 con 0x7f2c74107380 2024-03-21T10:25:01.013 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:01.010+0000 7f2c7aa00640 1 -- 172.21.15.47:0/2095192730 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2c740fd620 con 0x7f2c74107380 2024-03-21T10:25:01.014 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:01.012+0000 7f2c70c00640 1 -- 172.21.15.47:0/2095192730 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 18) v1 ==== 137854+0+0 (secure 0 0 0) 0x7f2c6c046020 con 0x7f2c74107380 2024-03-21T10:25:01.015 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:01.013+0000 7f2c70c00640 1 --2- 172.21.15.47:0/2095192730 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f2c580a2f20 0x7f2c580a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:25:01.016 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:01.014+0000 7f2c70c00640 1 -- 172.21.15.47:0/2095192730 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9216+0+0 (secure 0 0 0) 0x7f2c6c0f2790 con 0x7f2c74107380 2024-03-21T10:25:01.016 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:01.014+0000 7f2c73e00640 1 --2- 172.21.15.47:0/2095192730 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f2c580a2f20 0x7f2c580a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:25:01.016 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:01.015+0000 7f2c73e00640 1 --2- 172.21.15.47:0/2095192730 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f2c580a2f20 0x7f2c580a53e0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f2c64005fd0 tx=0x7f2c64009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:25:01.019 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:01.016+0000 7f2c70c00640 1 -- 172.21.15.47:0/2095192730 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f2c6c04d020 con 0x7f2c74107380 2024-03-21T10:25:01.280 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:01.278+0000 7f2c7aa00640 1 -- 172.21.15.47:0/2095192730 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f2c7402d070 con 0x7f2c74107380 2024-03-21T10:25:01.281 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:01.279+0000 7f2c70c00640 1 -- 172.21.15.47:0/2095192730 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v95) v1 ==== 100+0+11 (secure 0 0 0) 0x7f2c6c0473b0 con 0x7f2c74107380 2024-03-21T10:25:01.281 INFO:teuthology.orchestra.run.smithi047.stdout:pg_num: 64 2024-03-21T10:25:01.283 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:01.282+0000 7f2c7aa00640 1 -- 172.21.15.47:0/2095192730 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f2c580a2f20 msgr2=0x7f2c580a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:25:01.283 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:01.282+0000 7f2c7aa00640 1 --2- 172.21.15.47:0/2095192730 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f2c580a2f20 0x7f2c580a53e0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f2c64005fd0 tx=0x7f2c64009260 comp rx=0 tx=0).stop 2024-03-21T10:25:01.283 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:01.282+0000 7f2c7aa00640 1 -- 172.21.15.47:0/2095192730 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2c74107380 msgr2=0x7f2c7419eec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:25:01.283 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:01.282+0000 7f2c7aa00640 1 --2- 172.21.15.47:0/2095192730 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2c74107380 0x7f2c7419eec0 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f2c6c034fd0 tx=0x7f2c6c02cbc0 comp rx=0 tx=0).stop 2024-03-21T10:25:01.284 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:01.282+0000 7f2c7aa00640 1 -- 172.21.15.47:0/2095192730 shutdown_connections 2024-03-21T10:25:01.285 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:01.282+0000 7f2c7aa00640 1 --2- 172.21.15.47:0/2095192730 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f2c580a2f20 0x7f2c580a53e0 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:25:01.285 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:01.282+0000 7f2c7aa00640 1 --2- 172.21.15.47:0/2095192730 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2c74107380 0x7f2c7419eec0 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:25:01.285 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:01.282+0000 7f2c7aa00640 1 --2- 172.21.15.47:0/2095192730 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2c74104a50 0x7f2c74197e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:25:01.285 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:01.282+0000 7f2c7aa00640 1 --2- 172.21.15.47:0/2095192730 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2c74102120 0x7f2c74197910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:25:01.285 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:01.282+0000 7f2c7aa00640 1 -- 172.21.15.47:0/2095192730 >> 172.21.15.47:0/2095192730 conn(0x7f2c740fbb30 msgr2=0x7f2c74102a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:25:01.285 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:01.282+0000 7f2c7aa00640 1 -- 172.21.15.47:0/2095192730 shutdown_connections 2024-03-21T10:25:01.285 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:01.282+0000 7f2c7aa00640 1 -- 172.21.15.47:0/2095192730 wait complete. 2024-03-21T10:25:01.304 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-03-21T10:25:01.539 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:01.537+0000 7ff621600640 1 -- 172.21.15.47:0/733019693 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff61c103260 msgr2=0x7ff61c109af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:25:01.539 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:01.537+0000 7ff621600640 1 --2- 172.21.15.47:0/733019693 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff61c103260 0x7ff61c109af0 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7ff614009900 tx=0x7ff61402bc40 comp rx=0 tx=0).stop 2024-03-21T10:25:01.540 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:01.537+0000 7ff621600640 1 -- 172.21.15.47:0/733019693 shutdown_connections 2024-03-21T10:25:01.540 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:01.537+0000 7ff621600640 1 --2- 172.21.15.47:0/733019693 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff61c103260 0x7ff61c109af0 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:25:01.540 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:01.537+0000 7ff621600640 1 --2- 172.21.15.47:0/733019693 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff61c1028a0 0x7ff61c102d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:25:01.540 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:01.537+0000 7ff621600640 1 --2- 172.21.15.47:0/733019693 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff61c1016b0 0x7ff61c101ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:25:01.540 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:01.537+0000 7ff621600640 1 -- 172.21.15.47:0/733019693 >> 172.21.15.47:0/733019693 conn(0x7ff61c0fcc40 msgr2=0x7ff61c0ff060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:25:01.540 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:01.537+0000 7ff621600640 1 -- 172.21.15.47:0/733019693 shutdown_connections 2024-03-21T10:25:01.540 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:01.537+0000 7ff621600640 1 -- 172.21.15.47:0/733019693 wait complete. 2024-03-21T10:25:01.541 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:01.538+0000 7ff621600640 1 Processor -- start 2024-03-21T10:25:01.542 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:01.538+0000 7ff621600640 1 -- start start 2024-03-21T10:25:01.542 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:01.539+0000 7ff621600640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff61c1016b0 0x7ff61c199ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:25:01.542 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:01.539+0000 7ff621600640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff61c1028a0 0x7ff61c19a020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:25:01.542 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:01.539+0000 7ff621600640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff61c103260 0x7ff61c1a1090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:25:01.542 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:01.539+0000 7ff621600640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff61c10d230 con 0x7ff61c1028a0 2024-03-21T10:25:01.542 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:01.539+0000 7ff621600640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff61c10d0b0 con 0x7ff61c103260 2024-03-21T10:25:01.542 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:01.539+0000 7ff621600640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff61c10d3b0 con 0x7ff61c1016b0 2024-03-21T10:25:01.542 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:01.539+0000 7ff612000640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff61c1028a0 0x7ff61c19a020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:25:01.542 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:01.539+0000 7ff612000640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff61c1028a0 0x7ff61c19a020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:34506/0 (socket says 172.21.15.47:34506) 2024-03-21T10:25:01.542 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:01.539+0000 7ff612000640 1 -- 172.21.15.47:0/1426786599 learned_addr learned my addr 172.21.15.47:0/1426786599 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:25:01.542 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:01.539+0000 7ff61b400640 1 --2- 172.21.15.47:0/1426786599 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff61c103260 0x7ff61c1a1090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:25:01.542 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:01.539+0000 7ff61aa00640 1 --2- 172.21.15.47:0/1426786599 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff61c1016b0 0x7ff61c199ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:25:01.542 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:01.540+0000 7ff612000640 1 -- 172.21.15.47:0/1426786599 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff61c1016b0 msgr2=0x7ff61c199ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:25:01.543 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:01.540+0000 7ff612000640 1 --2- 172.21.15.47:0/1426786599 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff61c1016b0 0x7ff61c199ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:25:01.543 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:01.540+0000 7ff612000640 1 -- 172.21.15.47:0/1426786599 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff61c103260 msgr2=0x7ff61c1a1090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:25:01.543 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:01.540+0000 7ff612000640 1 --2- 172.21.15.47:0/1426786599 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff61c103260 0x7ff61c1a1090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:25:01.543 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:01.540+0000 7ff612000640 1 -- 172.21.15.47:0/1426786599 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff61c1a1700 con 0x7ff61c1028a0 2024-03-21T10:25:01.544 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:01.540+0000 7ff612000640 1 --2- 172.21.15.47:0/1426786599 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff61c1028a0 0x7ff61c19a020 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7ff60400e9f0 tx=0x7ff60400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:25:01.544 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:01.540+0000 7ff613e00640 1 -- 172.21.15.47:0/1426786599 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff60400cd80 con 0x7ff61c1028a0 2024-03-21T10:25:01.544 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:01.541+0000 7ff621600640 1 -- 172.21.15.47:0/1426786599 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff61c1a19f0 con 0x7ff61c1028a0 2024-03-21T10:25:01.544 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:01.541+0000 7ff613e00640 1 -- 172.21.15.47:0/1426786599 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7ff6040108e0 con 0x7ff61c1028a0 2024-03-21T10:25:01.544 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:01.541+0000 7ff613e00640 1 -- 172.21.15.47:0/1426786599 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff604018920 con 0x7ff61c1028a0 2024-03-21T10:25:01.544 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:01.541+0000 7ff621600640 1 -- 172.21.15.47:0/1426786599 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff61c1a1f40 con 0x7ff61c1028a0 2024-03-21T10:25:01.544 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:01.542+0000 7ff621600640 1 -- 172.21.15.47:0/1426786599 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff61c1a2170 con 0x7ff61c1028a0 2024-03-21T10:25:01.546 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:01.544+0000 7ff613e00640 1 -- 172.21.15.47:0/1426786599 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 18) v1 ==== 137854+0+0 (secure 0 0 0) 0x7ff604010a90 con 0x7ff61c1028a0 2024-03-21T10:25:01.547 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:01.545+0000 7ff613e00640 1 --2- 172.21.15.47:0/1426786599 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7ff6000a2f20 0x7ff6000a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:25:01.547 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:01.545+0000 7ff613e00640 1 -- 172.21.15.47:0/1426786599 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9216+0+0 (secure 0 0 0) 0x7ff6040cfff0 con 0x7ff61c1028a0 2024-03-21T10:25:01.547 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:01.546+0000 7ff61aa00640 1 --2- 172.21.15.47:0/1426786599 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7ff6000a2f20 0x7ff6000a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:25:01.548 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:01.546+0000 7ff61aa00640 1 --2- 172.21.15.47:0/1426786599 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7ff6000a2f20 0x7ff6000a53e0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7ff61c102700 tx=0x7ff60c009870 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:25:01.551 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:01.549+0000 7ff613e00640 1 -- 172.21.15.47:0/1426786599 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7ff60409ca10 con 0x7ff61c1028a0 2024-03-21T10:25:01.713 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:25:01 smithi047 ceph-mon[31547]: pgmap v462: 129 pgs: 129 active+clean; 645 KiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 1.2 KiB/s wr, 13 op/s 2024-03-21T10:25:01.714 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:25:01 smithi047 ceph-mon[31547]: from='client.? 172.21.15.47:0/1703830837' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-21T10:25:01.714 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:25:01 smithi047 ceph-mon[31547]: from='client.? 172.21.15.47:0/2095192730' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-21T10:25:01.729 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:25:01 smithi100 ceph-mon[35193]: pgmap v462: 129 pgs: 129 active+clean; 645 KiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 1.2 KiB/s wr, 13 op/s 2024-03-21T10:25:01.729 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:25:01 smithi100 ceph-mon[35193]: from='client.? 172.21.15.47:0/1703830837' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-21T10:25:01.729 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:25:01 smithi100 ceph-mon[35193]: from='client.? 172.21.15.47:0/2095192730' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-21T10:25:01.739 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:25:01 smithi157 ceph-mon[35581]: pgmap v462: 129 pgs: 129 active+clean; 645 KiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 1.2 KiB/s wr, 13 op/s 2024-03-21T10:25:01.740 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:25:01 smithi157 ceph-mon[35581]: from='client.? 172.21.15.47:0/1703830837' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-21T10:25:01.740 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:25:01 smithi157 ceph-mon[35581]: from='client.? 172.21.15.47:0/2095192730' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-21T10:25:01.754 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:01.753+0000 7ff621600640 1 -- 172.21.15.47:0/1426786599 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7ff61c1a2340 con 0x7ff61c1028a0 2024-03-21T10:25:01.755 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:01.753+0000 7ff613e00640 1 -- 172.21.15.47:0/1426786599 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v95) v1 ==== 103+0+11 (secure 0 0 0) 0x7ff60409c310 con 0x7ff61c1028a0 2024-03-21T10:25:01.755 INFO:teuthology.orchestra.run.smithi047.stdout:pg_num: 32 2024-03-21T10:25:01.759 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:01.757+0000 7ff621600640 1 -- 172.21.15.47:0/1426786599 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7ff6000a2f20 msgr2=0x7ff6000a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:25:01.759 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:01.757+0000 7ff621600640 1 --2- 172.21.15.47:0/1426786599 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7ff6000a2f20 0x7ff6000a53e0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7ff61c102700 tx=0x7ff60c009870 comp rx=0 tx=0).stop 2024-03-21T10:25:01.759 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:01.757+0000 7ff621600640 1 -- 172.21.15.47:0/1426786599 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff61c1028a0 msgr2=0x7ff61c19a020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:25:01.759 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:01.757+0000 7ff621600640 1 --2- 172.21.15.47:0/1426786599 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff61c1028a0 0x7ff61c19a020 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7ff60400e9f0 tx=0x7ff60400eeb0 comp rx=0 tx=0).stop 2024-03-21T10:25:01.760 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:01.757+0000 7ff621600640 1 -- 172.21.15.47:0/1426786599 shutdown_connections 2024-03-21T10:25:01.760 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:01.757+0000 7ff621600640 1 --2- 172.21.15.47:0/1426786599 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7ff6000a2f20 0x7ff6000a53e0 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:25:01.760 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:01.757+0000 7ff621600640 1 --2- 172.21.15.47:0/1426786599 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff61c103260 0x7ff61c1a1090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:25:01.760 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:01.757+0000 7ff621600640 1 --2- 172.21.15.47:0/1426786599 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff61c1028a0 0x7ff61c19a020 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:25:01.760 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:01.757+0000 7ff621600640 1 --2- 172.21.15.47:0/1426786599 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff61c1016b0 0x7ff61c199ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:25:01.760 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:01.757+0000 7ff621600640 1 -- 172.21.15.47:0/1426786599 >> 172.21.15.47:0/1426786599 conn(0x7ff61c0fcc40 msgr2=0x7ff61c108040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:25:01.760 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:01.758+0000 7ff621600640 1 -- 172.21.15.47:0/1426786599 shutdown_connections 2024-03-21T10:25:01.760 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:01.758+0000 7ff621600640 1 -- 172.21.15.47:0/1426786599 wait complete. 2024-03-21T10:25:01.780 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-21T10:25:01.985 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:01.982+0000 7faadac00640 1 -- 172.21.15.47:0/3109907237 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7faad4104a70 msgr2=0x7faad4106e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:25:01.985 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:01.982+0000 7faadac00640 1 --2- 172.21.15.47:0/3109907237 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7faad4104a70 0x7faad4106e60 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7faabc009900 tx=0x7faabc02bc50 comp rx=0 tx=0).stop 2024-03-21T10:25:01.985 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:01.983+0000 7faadac00640 1 -- 172.21.15.47:0/3109907237 shutdown_connections 2024-03-21T10:25:01.985 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:01.983+0000 7faadac00640 1 --2- 172.21.15.47:0/3109907237 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7faad41073a0 0x7faad41097f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:25:01.985 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:01.983+0000 7faadac00640 1 --2- 172.21.15.47:0/3109907237 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7faad4104a70 0x7faad4106e60 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:25:01.985 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:01.983+0000 7faadac00640 1 --2- 172.21.15.47:0/3109907237 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7faad4102140 0x7faad4104530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:25:01.985 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:01.983+0000 7faadac00640 1 -- 172.21.15.47:0/3109907237 >> 172.21.15.47:0/3109907237 conn(0x7faad40fbb30 msgr2=0x7faad40fdf70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:25:01.985 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:01.983+0000 7faadac00640 1 -- 172.21.15.47:0/3109907237 shutdown_connections 2024-03-21T10:25:01.985 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:01.983+0000 7faadac00640 1 -- 172.21.15.47:0/3109907237 wait complete. 2024-03-21T10:25:01.987 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:01.984+0000 7faadac00640 1 Processor -- start 2024-03-21T10:25:01.988 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:01.984+0000 7faadac00640 1 -- start start 2024-03-21T10:25:01.988 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:01.984+0000 7faadac00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7faad4102140 0x7faad4199b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:25:01.988 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:01.984+0000 7faadac00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7faad4104a70 0x7faad419a0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:25:01.988 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:01.984+0000 7faadac00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7faad41073a0 0x7faad41a1130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:25:01.988 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:01.984+0000 7faadac00640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faad410d200 con 0x7faad4104a70 2024-03-21T10:25:01.988 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:01.984+0000 7faadac00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faad410d080 con 0x7faad41073a0 2024-03-21T10:25:01.988 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:01.984+0000 7faadac00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faad410d380 con 0x7faad4102140 2024-03-21T10:25:01.988 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:01.984+0000 7faacbe00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7faad4104a70 0x7faad419a0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:25:01.988 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:01.985+0000 7faad8c00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7faad41073a0 0x7faad41a1130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:25:01.988 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:01.985+0000 7faacbe00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7faad4104a70 0x7faad419a0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:56168/0 (socket says 172.21.15.47:56168) 2024-03-21T10:25:01.988 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:01.985+0000 7faacbe00640 1 -- 172.21.15.47:0/1149401013 learned_addr learned my addr 172.21.15.47:0/1149401013 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:25:01.988 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:01.985+0000 7faad3e00640 1 --2- 172.21.15.47:0/1149401013 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7faad4102140 0x7faad4199b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:25:01.988 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:01.985+0000 7faacbe00640 1 -- 172.21.15.47:0/1149401013 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7faad4102140 msgr2=0x7faad4199b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:25:01.989 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:01.985+0000 7faacbe00640 1 --2- 172.21.15.47:0/1149401013 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7faad4102140 0x7faad4199b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:25:01.989 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:01.985+0000 7faacbe00640 1 -- 172.21.15.47:0/1149401013 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7faad41073a0 msgr2=0x7faad41a1130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:25:01.989 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:01.985+0000 7faacbe00640 1 --2- 172.21.15.47:0/1149401013 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7faad41073a0 0x7faad41a1130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:25:01.989 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:01.985+0000 7faacbe00640 1 -- 172.21.15.47:0/1149401013 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faad41a17a0 con 0x7faad4104a70 2024-03-21T10:25:01.989 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:01.985+0000 7faacbe00640 1 --2- 172.21.15.47:0/1149401013 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7faad4104a70 0x7faad419a0c0 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7faabc000c00 tx=0x7faabc004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:25:01.989 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:01.986+0000 7faad1600640 1 -- 172.21.15.47:0/1149401013 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faabc043020 con 0x7faad4104a70 2024-03-21T10:25:01.989 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:01.986+0000 7faad1600640 1 -- 172.21.15.47:0/1149401013 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7faabc004440 con 0x7faad4104a70 2024-03-21T10:25:01.989 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:01.986+0000 7faad1600640 1 -- 172.21.15.47:0/1149401013 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faabc003c90 con 0x7faad4104a70 2024-03-21T10:25:01.989 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:01.986+0000 7faadac00640 1 -- 172.21.15.47:0/1149401013 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faad41a1a30 con 0x7faad4104a70 2024-03-21T10:25:01.989 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:01.986+0000 7faadac00640 1 -- 172.21.15.47:0/1149401013 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faad41a1f30 con 0x7faad4104a70 2024-03-21T10:25:01.989 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:01.987+0000 7faadac00640 1 -- 172.21.15.47:0/1149401013 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faad4108ef0 con 0x7faad4104a70 2024-03-21T10:25:01.991 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:01.989+0000 7faad1600640 1 -- 172.21.15.47:0/1149401013 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 18) v1 ==== 137854+0+0 (secure 0 0 0) 0x7faabc046020 con 0x7faad4104a70 2024-03-21T10:25:01.992 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:01.990+0000 7faad1600640 1 --2- 172.21.15.47:0/1149401013 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7faab80a2fa0 0x7faab80a5460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:25:01.992 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:01.990+0000 7faad1600640 1 -- 172.21.15.47:0/1149401013 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9216+0+0 (secure 0 0 0) 0x7faabc0f1dd0 con 0x7faad4104a70 2024-03-21T10:25:01.992 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:01.991+0000 7faad3e00640 1 --2- 172.21.15.47:0/1149401013 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7faab80a2fa0 0x7faab80a5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:25:01.993 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:01.991+0000 7faad3e00640 1 --2- 172.21.15.47:0/1149401013 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7faab80a2fa0 0x7faab80a5460 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7faac4005fd0 tx=0x7faac4005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:25:01.999 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:01.997+0000 7faad1600640 1 -- 172.21.15.47:0/1149401013 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7faabc04d020 con 0x7faad4104a70 2024-03-21T10:25:02.205 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:02.203+0000 7faadac00640 1 -- 172.21.15.47:0/1149401013 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7faad402d2c0 con 0x7faad4104a70 2024-03-21T10:25:02.207 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:02.206+0000 7faad1600640 1 -- 172.21.15.47:0/1149401013 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v95) v1 ==== 74+0+24849 (secure 0 0 0) 0x7faabc0be7f0 con 0x7faad4104a70 2024-03-21T10:25:02.208 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-21T10:25:02.208 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":95,"fsid":"3056d51a-e76b-11ee-95cd-87774f69a715","created":"2024-03-21T10:11:17.019965+0000","modified":"2024-03-21T10:23:27.619804+0000","last_up_change":"2024-03-21T10:20:52.675328+0000","last_in_change":"2024-03-21T10:20:27.648212+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-21T10:15:40.553114+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-21T10:21:56.399518+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-21T10:21:57.812940+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-21T10:22:01.822568+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"91","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"b0c7a102-8d89-428b-b793-cbbe67151f50","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6802","nonce":775837313},{"type":"v1","addr":"172.21.15.47:6803","nonce":775837313}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6804","nonce":775837313},{"type":"v1","addr":"172.21.15.47:6805","nonce":775837313}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6808","nonce":775837313},{"type":"v1","addr":"172.21.15.47:6809","nonce":775837313}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6806","nonce":775837313},{"type":"v1","addr":"172.21.15.47:6807","nonce":775837313}]},"public_addr":"172.21.15.47:6803/775837313","cluster_addr":"172.21.15.47:6805/775837313","heartbeat_back_addr":"172.21.15.47:6809/775837313","heartbeat_front_addr":"172.21.15.47:6807/775837313","state":["exists","up"]},{"osd":1,"uuid":"07bb81bb-7cf3-4bbc-ac9e-802f95185d11","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6800","nonce":2923304710},{"type":"v1","addr":"172.21.15.100:6801","nonce":2923304710}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":2923304710},{"type":"v1","addr":"172.21.15.100:6803","nonce":2923304710}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":2923304710},{"type":"v1","addr":"172.21.15.100:6807","nonce":2923304710}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6804","nonce":2923304710},{"type":"v1","addr":"172.21.15.100:6805","nonce":2923304710}]},"public_addr":"172.21.15.100:6801/2923304710","cluster_addr":"172.21.15.100:6803/2923304710","heartbeat_back_addr":"172.21.15.100:6807/2923304710","heartbeat_front_addr":"172.21.15.100:6805/2923304710","state":["exists","up"]},{"osd":2,"uuid":"88ce76b9-6081-4836-aad0-8c249c56e8a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":171377005},{"type":"v1","addr":"172.21.15.157:6801","nonce":171377005}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":171377005},{"type":"v1","addr":"172.21.15.157:6803","nonce":171377005}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":171377005},{"type":"v1","addr":"172.21.15.157:6807","nonce":171377005}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":171377005},{"type":"v1","addr":"172.21.15.157:6805","nonce":171377005}]},"public_addr":"172.21.15.157:6801/171377005","cluster_addr":"172.21.15.157:6803/171377005","heartbeat_back_addr":"172.21.15.157:6807/171377005","heartbeat_front_addr":"172.21.15.157:6805/171377005","state":["exists","up"]},{"osd":3,"uuid":"518aa33d-263f-45e6-aab5-2201ceb40e74","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":27,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6810","nonce":640760101},{"type":"v1","addr":"172.21.15.47:6811","nonce":640760101}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6812","nonce":640760101},{"type":"v1","addr":"172.21.15.47:6813","nonce":640760101}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6816","nonce":640760101},{"type":"v1","addr":"172.21.15.47:6817","nonce":640760101}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6814","nonce":640760101},{"type":"v1","addr":"172.21.15.47:6815","nonce":640760101}]},"public_addr":"172.21.15.47:6811/640760101","cluster_addr":"172.21.15.47:6813/640760101","heartbeat_back_addr":"172.21.15.47:6817/640760101","heartbeat_front_addr":"172.21.15.47:6815/640760101","state":["exists","up"]},{"osd":4,"uuid":"987bde83-ecc6-42e5-bd56-c143b3de4226","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":35,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6808","nonce":2785478639},{"type":"v1","addr":"172.21.15.100:6809","nonce":2785478639}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6810","nonce":2785478639},{"type":"v1","addr":"172.21.15.100:6811","nonce":2785478639}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6814","nonce":2785478639},{"type":"v1","addr":"172.21.15.100:6815","nonce":2785478639}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":2785478639},{"type":"v1","addr":"172.21.15.100:6813","nonce":2785478639}]},"public_addr":"172.21.15.100:6809/2785478639","cluster_addr":"172.21.15.100:6811/2785478639","heartbeat_back_addr":"172.21.15.100:6815/2785478639","heartbeat_front_addr":"172.21.15.100:6813/2785478639","state":["exists","up"]},{"osd":5,"uuid":"012d2188-3a61-4a5f-8d01-2001c15ffbe2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":40,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":1525645410},{"type":"v1","addr":"172.21.15.157:6809","nonce":1525645410}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":1525645410},{"type":"v1","addr":"172.21.15.157:6811","nonce":1525645410}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":1525645410},{"type":"v1","addr":"172.21.15.157:6815","nonce":1525645410}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":1525645410},{"type":"v1","addr":"172.21.15.157:6813","nonce":1525645410}]},"public_addr":"172.21.15.157:6809/1525645410","cluster_addr":"172.21.15.157:6811/1525645410","heartbeat_back_addr":"172.21.15.157:6815/1525645410","heartbeat_front_addr":"172.21.15.157:6813/1525645410","state":["exists","up"]},{"osd":6,"uuid":"c12c14b4-85f0-4d88-937c-de4f556437de","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6818","nonce":2321642790},{"type":"v1","addr":"172.21.15.47:6819","nonce":2321642790}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6820","nonce":2321642790},{"type":"v1","addr":"172.21.15.47:6821","nonce":2321642790}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6824","nonce":2321642790},{"type":"v1","addr":"172.21.15.47:6825","nonce":2321642790}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6822","nonce":2321642790},{"type":"v1","addr":"172.21.15.47:6823","nonce":2321642790}]},"public_addr":"172.21.15.47:6819/2321642790","cluster_addr":"172.21.15.47:6821/2321642790","heartbeat_back_addr":"172.21.15.47:6825/2321642790","heartbeat_front_addr":"172.21.15.47:6823/2321642790","state":["exists","up"]},{"osd":7,"uuid":"3c87c6d0-1d1f-4294-9917-aaaad7dda824","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6816","nonce":4104727330},{"type":"v1","addr":"172.21.15.100:6817","nonce":4104727330}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":4104727330},{"type":"v1","addr":"172.21.15.100:6819","nonce":4104727330}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":4104727330},{"type":"v1","addr":"172.21.15.100:6823","nonce":4104727330}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6820","nonce":4104727330},{"type":"v1","addr":"172.21.15.100:6821","nonce":4104727330}]},"public_addr":"172.21.15.100:6817/4104727330","cluster_addr":"172.21.15.100:6819/4104727330","heartbeat_back_addr":"172.21.15.100:6823/4104727330","heartbeat_front_addr":"172.21.15.100:6821/4104727330","state":["exists","up"]},{"osd":8,"uuid":"502d65cf-53c8-4936-8de4-e11b2a6e1e18","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":61,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":4067715347},{"type":"v1","addr":"172.21.15.157:6817","nonce":4067715347}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":4067715347},{"type":"v1","addr":"172.21.15.157:6819","nonce":4067715347}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":4067715347},{"type":"v1","addr":"172.21.15.157:6823","nonce":4067715347}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":4067715347},{"type":"v1","addr":"172.21.15.157:6821","nonce":4067715347}]},"public_addr":"172.21.15.157:6817/4067715347","cluster_addr":"172.21.15.157:6819/4067715347","heartbeat_back_addr":"172.21.15.157:6823/4067715347","heartbeat_front_addr":"172.21.15.157:6821/4067715347","state":["exists","up"]},{"osd":9,"uuid":"5ea1f5be-8458-4c93-9e61-c2e867a514f3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6826","nonce":1982108738},{"type":"v1","addr":"172.21.15.47:6827","nonce":1982108738}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6828","nonce":1982108738},{"type":"v1","addr":"172.21.15.47:6829","nonce":1982108738}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6832","nonce":1982108738},{"type":"v1","addr":"172.21.15.47:6833","nonce":1982108738}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6830","nonce":1982108738},{"type":"v1","addr":"172.21.15.47:6831","nonce":1982108738}]},"public_addr":"172.21.15.47:6827/1982108738","cluster_addr":"172.21.15.47:6829/1982108738","heartbeat_back_addr":"172.21.15.47:6833/1982108738","heartbeat_front_addr":"172.21.15.47:6831/1982108738","state":["exists","up"]},{"osd":10,"uuid":"2da3da90-a0be-4a54-a9ff-b2d6c9b36870","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":74,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":396391028},{"type":"v1","addr":"172.21.15.100:6825","nonce":396391028}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":396391028},{"type":"v1","addr":"172.21.15.100:6827","nonce":396391028}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6830","nonce":396391028},{"type":"v1","addr":"172.21.15.100:6831","nonce":396391028}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":396391028},{"type":"v1","addr":"172.21.15.100:6829","nonce":396391028}]},"public_addr":"172.21.15.100:6825/396391028","cluster_addr":"172.21.15.100:6827/396391028","heartbeat_back_addr":"172.21.15.100:6831/396391028","heartbeat_front_addr":"172.21.15.100:6829/396391028","state":["exists","up"]},{"osd":11,"uuid":"11798d41-7f7e-4319-95b8-8d3d0946bde9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":82,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":2671029483},{"type":"v1","addr":"172.21.15.157:6825","nonce":2671029483}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":2671029483},{"type":"v1","addr":"172.21.15.157:6827","nonce":2671029483}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":2671029483},{"type":"v1","addr":"172.21.15.157:6831","nonce":2671029483}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":2671029483},{"type":"v1","addr":"172.21.15.157:6829","nonce":2671029483}]},"public_addr":"172.21.15.157:6825/2671029483","cluster_addr":"172.21.15.157:6827/2671029483","heartbeat_back_addr":"172.21.15.157:6831/2671029483","heartbeat_front_addr":"172.21.15.157:6829/2671029483","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T10:14:31.822399+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T10:15:04.175721+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T10:15:35.902595+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T10:16:12.088891+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T10:16:46.428704+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T10:17:20.158270+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T10:17:55.915253+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T10:18:30.064641+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T10:19:04.410004+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T10:19:40.605181+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T10:20:15.636605+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T10:20:50.151365+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.11","mappings":[{"from":7,"to":11}]},{"pgid":"2.14","mappings":[{"from":2,"to":4}]},{"pgid":"2.1a","mappings":[{"from":8,"to":1}]},{"pgid":"2.1b","mappings":[{"from":2,"to":4}]},{"pgid":"2.1d","mappings":[{"from":8,"to":11}]},{"pgid":"3.0","mappings":[{"from":4,"to":8},{"from":5,"to":9}]},{"pgid":"3.6","mappings":[{"from":3,"to":7}]},{"pgid":"3.1b","mappings":[{"from":4,"to":6}]},{"pgid":"3.33","mappings":[{"from":5,"to":9}]},{"pgid":"3.3c","mappings":[{"from":4,"to":9}]},{"pgid":"4.c","mappings":[{"from":9,"to":10}]},{"pgid":"4.f","mappings":[{"from":9,"to":8}]},{"pgid":"4.19","mappings":[{"from":9,"to":8}]},{"pgid":"4.1d","mappings":[{"from":5,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.47:0/356822431":"2024-03-22T10:12:25.844530+0000","172.21.15.47:0/1167490879":"2024-03-22T10:12:25.844530+0000","172.21.15.47:0/1540718138":"2024-03-22T10:12:25.844530+0000","172.21.15.47:6801/3836903972":"2024-03-22T10:12:25.844530+0000","172.21.15.47:0/235436602":"2024-03-22T10:11:50.301838+0000","172.21.15.47:0/2052815572":"2024-03-22T10:11:50.301838+0000","172.21.15.47:6800/3942658432":"2024-03-22T10:11:50.301838+0000","172.21.15.47:6801/3942658432":"2024-03-22T10:11:50.301838+0000","172.21.15.47:0/1419516257":"2024-03-22T10:11:50.301838+0000","172.21.15.47:6800/3836903972":"2024-03-22T10:12:25.844530+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-21T10:25:02.211 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:02.210+0000 7faadac00640 1 -- 172.21.15.47:0/1149401013 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7faab80a2fa0 msgr2=0x7faab80a5460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:25:02.211 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:02.210+0000 7faadac00640 1 --2- 172.21.15.47:0/1149401013 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7faab80a2fa0 0x7faab80a5460 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7faac4005fd0 tx=0x7faac4005850 comp rx=0 tx=0).stop 2024-03-21T10:25:02.211 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:02.210+0000 7faadac00640 1 -- 172.21.15.47:0/1149401013 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7faad4104a70 msgr2=0x7faad419a0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:25:02.211 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:02.210+0000 7faadac00640 1 --2- 172.21.15.47:0/1149401013 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7faad4104a70 0x7faad419a0c0 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7faabc000c00 tx=0x7faabc004040 comp rx=0 tx=0).stop 2024-03-21T10:25:02.212 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:02.211+0000 7faadac00640 1 -- 172.21.15.47:0/1149401013 shutdown_connections 2024-03-21T10:25:02.212 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:02.211+0000 7faadac00640 1 --2- 172.21.15.47:0/1149401013 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7faab80a2fa0 0x7faab80a5460 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:25:02.212 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:02.211+0000 7faadac00640 1 --2- 172.21.15.47:0/1149401013 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7faad41073a0 0x7faad41a1130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:25:02.212 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:02.211+0000 7faadac00640 1 --2- 172.21.15.47:0/1149401013 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7faad4104a70 0x7faad419a0c0 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:25:02.212 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:02.211+0000 7faadac00640 1 --2- 172.21.15.47:0/1149401013 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7faad4102140 0x7faad4199b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:25:02.212 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:02.211+0000 7faadac00640 1 -- 172.21.15.47:0/1149401013 >> 172.21.15.47:0/1149401013 conn(0x7faad40fbb30 msgr2=0x7faad4101a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:25:02.212 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:02.211+0000 7faadac00640 1 -- 172.21.15.47:0/1149401013 shutdown_connections 2024-03-21T10:25:02.212 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:02.211+0000 7faadac00640 1 -- 172.21.15.47:0/1149401013 wait complete. 2024-03-21T10:25:02.235 INFO:tasks.fwd_scrub.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-21T10:15:40.553114+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-21T10:21:56.399518+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '86', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-21T10:21:57.812940+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '87', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-03-21T10:22:01.822568+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '91', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-03-21T10:25:02.236 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-21T10:25:02.459 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:02.458+0000 7fc932800640 1 -- 172.21.15.47:0/110364711 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc92c074100 msgr2=0x7fc92c074580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:25:02.460 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:02.458+0000 7fc932800640 1 --2- 172.21.15.47:0/110364711 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc92c074100 0x7fc92c074580 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7fc918009930 tx=0x7fc91802bc30 comp rx=0 tx=0).stop 2024-03-21T10:25:02.460 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:02.458+0000 7fc932800640 1 -- 172.21.15.47:0/110364711 shutdown_connections 2024-03-21T10:25:02.460 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:02.458+0000 7fc932800640 1 --2- 172.21.15.47:0/110364711 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc92c10db70 0x7fc92c10ff60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:25:02.460 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:02.458+0000 7fc932800640 1 --2- 172.21.15.47:0/110364711 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc92c074100 0x7fc92c074580 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:25:02.460 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:02.458+0000 7fc932800640 1 --2- 172.21.15.47:0/110364711 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc92c0756e0 0x7fc92c073b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:25:02.460 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:02.458+0000 7fc932800640 1 -- 172.21.15.47:0/110364711 >> 172.21.15.47:0/110364711 conn(0x7fc92c0fcb10 msgr2=0x7fc92c0fef50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:25:02.460 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:02.458+0000 7fc932800640 1 -- 172.21.15.47:0/110364711 shutdown_connections 2024-03-21T10:25:02.460 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:02.459+0000 7fc932800640 1 -- 172.21.15.47:0/110364711 wait complete. 2024-03-21T10:25:02.462 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:02.459+0000 7fc932800640 1 Processor -- start 2024-03-21T10:25:02.462 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:02.460+0000 7fc932800640 1 -- start start 2024-03-21T10:25:02.462 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:02.460+0000 7fc932800640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc92c074100 0x7fc92c19de30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:25:02.462 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:02.460+0000 7fc932800640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc92c0756e0 0x7fc92c19e370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:25:02.463 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:02.460+0000 7fc932800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc92c10db70 0x7fc92c1a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:25:02.463 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:02.460+0000 7fc932800640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc92c1136a0 con 0x7fc92c0756e0 2024-03-21T10:25:02.463 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:02.460+0000 7fc932800640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc92c113520 con 0x7fc92c10db70 2024-03-21T10:25:02.463 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:02.460+0000 7fc932800640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc92c113820 con 0x7fc92c074100 2024-03-21T10:25:02.463 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:02.460+0000 7fc930a00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc92c0756e0 0x7fc92c19e370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:25:02.463 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:02.460+0000 7fc930a00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc92c0756e0 0x7fc92c19e370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:56184/0 (socket says 172.21.15.47:56184) 2024-03-21T10:25:02.463 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:02.460+0000 7fc930a00640 1 -- 172.21.15.47:0/261408825 learned_addr learned my addr 172.21.15.47:0/261408825 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:25:02.463 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:02.460+0000 7fc931400640 1 --2- 172.21.15.47:0/261408825 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc92c074100 0x7fc92c19de30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:25:02.463 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:02.460+0000 7fc931e00640 1 --2- 172.21.15.47:0/261408825 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc92c10db70 0x7fc92c1a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:25:02.463 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:02.461+0000 7fc930a00640 1 -- 172.21.15.47:0/261408825 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc92c074100 msgr2=0x7fc92c19de30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:25:02.463 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:02.461+0000 7fc930a00640 1 --2- 172.21.15.47:0/261408825 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc92c074100 0x7fc92c19de30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:25:02.463 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:02.461+0000 7fc930a00640 1 -- 172.21.15.47:0/261408825 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc92c10db70 msgr2=0x7fc92c1a53e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:25:02.463 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:02.461+0000 7fc930a00640 1 --2- 172.21.15.47:0/261408825 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc92c10db70 0x7fc92c1a53e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:25:02.463 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:02.461+0000 7fc930a00640 1 -- 172.21.15.47:0/261408825 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc92c1a5a50 con 0x7fc92c0756e0 2024-03-21T10:25:02.463 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:02.461+0000 7fc931e00640 1 --2- 172.21.15.47:0/261408825 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc92c10db70 0x7fc92c1a53e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T10:25:02.464 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:02.461+0000 7fc930a00640 1 --2- 172.21.15.47:0/261408825 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc92c0756e0 0x7fc92c19e370 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7fc918009230 tx=0x7fc9180040c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:25:02.464 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:02.461+0000 7fc92a000640 1 -- 172.21.15.47:0/261408825 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc918043020 con 0x7fc92c0756e0 2024-03-21T10:25:02.464 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:02.461+0000 7fc92a000640 1 -- 172.21.15.47:0/261408825 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fc9180044c0 con 0x7fc92c0756e0 2024-03-21T10:25:02.464 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:02.462+0000 7fc92a000640 1 -- 172.21.15.47:0/261408825 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc918003bd0 con 0x7fc92c0756e0 2024-03-21T10:25:02.464 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:02.462+0000 7fc932800640 1 -- 172.21.15.47:0/261408825 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc92c1a5ce0 con 0x7fc92c0756e0 2024-03-21T10:25:02.464 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:02.462+0000 7fc932800640 1 -- 172.21.15.47:0/261408825 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc92c1a61d0 con 0x7fc92c0756e0 2024-03-21T10:25:02.464 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:02.463+0000 7fc932800640 1 -- 172.21.15.47:0/261408825 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc92c103d40 con 0x7fc92c0756e0 2024-03-21T10:25:02.466 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:02.465+0000 7fc92a000640 1 -- 172.21.15.47:0/261408825 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 18) v1 ==== 137854+0+0 (secure 0 0 0) 0x7fc918046020 con 0x7fc92c0756e0 2024-03-21T10:25:02.468 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:02.468+0000 7fc92a000640 1 --2- 172.21.15.47:0/261408825 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7fc9140a2f20 0x7fc9140a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:25:02.469 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:02.468+0000 7fc92a000640 1 -- 172.21.15.47:0/261408825 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9216+0+0 (secure 0 0 0) 0x7fc9180f1f90 con 0x7fc92c0756e0 2024-03-21T10:25:02.469 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:02.468+0000 7fc931400640 1 --2- 172.21.15.47:0/261408825 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7fc9140a2f20 0x7fc9140a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:25:02.469 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:02.469+0000 7fc931400640 1 --2- 172.21.15.47:0/261408825 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7fc9140a2f20 0x7fc9140a53e0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fc920005fd0 tx=0x7fc920005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:25:02.470 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:02.469+0000 7fc92a000640 1 -- 172.21.15.47:0/261408825 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fc91804d020 con 0x7fc92c0756e0 2024-03-21T10:25:02.714 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:25:02 smithi047 ceph-mon[31547]: from='client.? 172.21.15.47:0/1426786599' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-03-21T10:25:02.714 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:25:02 smithi047 ceph-mon[31547]: from='client.? 172.21.15.47:0/1149401013' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-21T10:25:02.729 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:25:02 smithi100 ceph-mon[35193]: from='client.? 172.21.15.47:0/1426786599' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-03-21T10:25:02.729 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:25:02 smithi100 ceph-mon[35193]: from='client.? 172.21.15.47:0/1149401013' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-21T10:25:02.739 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:25:02 smithi157 ceph-mon[35581]: from='client.? 172.21.15.47:0/1426786599' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-03-21T10:25:02.740 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:25:02 smithi157 ceph-mon[35581]: from='client.? 172.21.15.47:0/1149401013' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-21T10:25:02.762 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:02.761+0000 7fc932800640 1 -- 172.21.15.47:0/261408825 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fc92c02d2c0 con 0x7fc92c0756e0 2024-03-21T10:25:02.763 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:02.762+0000 7fc92a000640 1 -- 172.21.15.47:0/261408825 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v95) v1 ==== 93+0+10 (secure 0 0 0) 0x7fc9180b8e20 con 0x7fc92c0756e0 2024-03-21T10:25:02.763 INFO:teuthology.orchestra.run.smithi047.stdout:pg_num: 1 2024-03-21T10:25:02.766 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:02.764+0000 7fc932800640 1 -- 172.21.15.47:0/261408825 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7fc9140a2f20 msgr2=0x7fc9140a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:25:02.766 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:02.764+0000 7fc932800640 1 --2- 172.21.15.47:0/261408825 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7fc9140a2f20 0x7fc9140a53e0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fc920005fd0 tx=0x7fc920005850 comp rx=0 tx=0).stop 2024-03-21T10:25:02.766 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:02.765+0000 7fc932800640 1 -- 172.21.15.47:0/261408825 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc92c0756e0 msgr2=0x7fc92c19e370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:25:02.767 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:02.765+0000 7fc932800640 1 --2- 172.21.15.47:0/261408825 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc92c0756e0 0x7fc92c19e370 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7fc918009230 tx=0x7fc9180040c0 comp rx=0 tx=0).stop 2024-03-21T10:25:02.767 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:02.765+0000 7fc932800640 1 -- 172.21.15.47:0/261408825 shutdown_connections 2024-03-21T10:25:02.767 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:02.765+0000 7fc932800640 1 --2- 172.21.15.47:0/261408825 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7fc9140a2f20 0x7fc9140a53e0 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:25:02.767 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:02.765+0000 7fc932800640 1 --2- 172.21.15.47:0/261408825 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc92c10db70 0x7fc92c1a53e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:25:02.767 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:02.765+0000 7fc932800640 1 --2- 172.21.15.47:0/261408825 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc92c0756e0 0x7fc92c19e370 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:25:02.767 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:02.765+0000 7fc932800640 1 --2- 172.21.15.47:0/261408825 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc92c074100 0x7fc92c19de30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:25:02.767 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:02.765+0000 7fc932800640 1 -- 172.21.15.47:0/261408825 >> 172.21.15.47:0/261408825 conn(0x7fc92c0fcb10 msgr2=0x7fc92c100f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:25:02.767 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:02.765+0000 7fc932800640 1 -- 172.21.15.47:0/261408825 shutdown_connections 2024-03-21T10:25:02.767 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:02.765+0000 7fc932800640 1 -- 172.21.15.47:0/261408825 wait complete. 2024-03-21T10:25:02.787 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-21T10:25:03.009 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:03.007+0000 7fcea5600640 1 -- 172.21.15.47:0/1910222863 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fcea0105490 msgr2=0x7fcea010bd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:25:03.009 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:03.007+0000 7fcea5600640 1 --2- 172.21.15.47:0/1910222863 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fcea0105490 0x7fcea010bd20 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7fce98009900 tx=0x7fce9802bc30 comp rx=0 tx=0).stop 2024-03-21T10:25:03.009 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:03.007+0000 7fcea5600640 1 -- 172.21.15.47:0/1910222863 shutdown_connections 2024-03-21T10:25:03.009 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:03.007+0000 7fcea5600640 1 --2- 172.21.15.47:0/1910222863 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fcea0105490 0x7fcea010bd20 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:25:03.009 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:03.007+0000 7fcea5600640 1 --2- 172.21.15.47:0/1910222863 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcea0104ad0 0x7fcea0104f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:25:03.009 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:03.007+0000 7fcea5600640 1 --2- 172.21.15.47:0/1910222863 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcea01038e0 0x7fcea0103d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:25:03.010 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:03.007+0000 7fcea5600640 1 -- 172.21.15.47:0/1910222863 >> 172.21.15.47:0/1910222863 conn(0x7fcea00fee50 msgr2=0x7fcea0101290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:25:03.010 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:03.007+0000 7fcea5600640 1 -- 172.21.15.47:0/1910222863 shutdown_connections 2024-03-21T10:25:03.010 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:03.008+0000 7fcea5600640 1 -- 172.21.15.47:0/1910222863 wait complete. 2024-03-21T10:25:03.012 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:03.009+0000 7fcea5600640 1 Processor -- start 2024-03-21T10:25:03.012 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:03.009+0000 7fcea5600640 1 -- start start 2024-03-21T10:25:03.012 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:03.009+0000 7fcea5600640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcea01038e0 0x7fcea0199af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:25:03.012 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:03.009+0000 7fcea5600640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcea0104ad0 0x7fcea019a030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:25:03.012 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:03.009+0000 7fcea5600640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fcea0105490 0x7fcea01a10a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:25:03.012 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:03.009+0000 7fcea5600640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcea010f3f0 con 0x7fcea0105490 2024-03-21T10:25:03.012 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:03.009+0000 7fcea5600640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcea010f270 con 0x7fcea0104ad0 2024-03-21T10:25:03.012 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:03.009+0000 7fcea5600640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcea010f570 con 0x7fcea01038e0 2024-03-21T10:25:03.012 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:03.009+0000 7fce9f400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fcea0105490 0x7fcea01a10a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:25:03.012 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:03.009+0000 7fce96000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcea0104ad0 0x7fcea019a030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:25:03.013 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:03.010+0000 7fce96000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcea0104ad0 0x7fcea019a030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:50024/0 (socket says 172.21.15.47:50024) 2024-03-21T10:25:03.013 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:03.010+0000 7fce96000640 1 -- 172.21.15.47:0/784843382 learned_addr learned my addr 172.21.15.47:0/784843382 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:25:03.013 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:03.010+0000 7fce9ea00640 1 --2- 172.21.15.47:0/784843382 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcea01038e0 0x7fcea0199af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:25:03.013 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:03.010+0000 7fce9f400640 1 -- 172.21.15.47:0/784843382 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcea01038e0 msgr2=0x7fcea0199af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:25:03.013 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:03.010+0000 7fce9f400640 1 --2- 172.21.15.47:0/784843382 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcea01038e0 0x7fcea0199af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:25:03.013 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:03.010+0000 7fce9f400640 1 -- 172.21.15.47:0/784843382 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcea0104ad0 msgr2=0x7fcea019a030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:25:03.013 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:03.010+0000 7fce9f400640 1 --2- 172.21.15.47:0/784843382 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcea0104ad0 0x7fcea019a030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:25:03.013 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:03.010+0000 7fce9f400640 1 -- 172.21.15.47:0/784843382 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcea01a1710 con 0x7fcea0105490 2024-03-21T10:25:03.013 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:03.010+0000 7fce9f400640 1 --2- 172.21.15.47:0/784843382 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fcea0105490 0x7fcea01a10a0 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7fce98009230 tx=0x7fce980042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:25:03.013 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:03.011+0000 7fce97e00640 1 -- 172.21.15.47:0/784843382 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce98043020 con 0x7fcea0105490 2024-03-21T10:25:03.013 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:03.011+0000 7fce97e00640 1 -- 172.21.15.47:0/784843382 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fce98004aa0 con 0x7fcea0105490 2024-03-21T10:25:03.013 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:03.011+0000 7fce97e00640 1 -- 172.21.15.47:0/784843382 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce98003c40 con 0x7fcea0105490 2024-03-21T10:25:03.013 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:03.011+0000 7fcea5600640 1 -- 172.21.15.47:0/784843382 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcea01a19a0 con 0x7fcea0105490 2024-03-21T10:25:03.013 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:03.011+0000 7fcea5600640 1 -- 172.21.15.47:0/784843382 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcea01a1ea0 con 0x7fcea0105490 2024-03-21T10:25:03.014 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:03.012+0000 7fcea5600640 1 -- 172.21.15.47:0/784843382 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcea010acc0 con 0x7fcea0105490 2024-03-21T10:25:03.016 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:03.016+0000 7fce97e00640 1 -- 172.21.15.47:0/784843382 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 18) v1 ==== 137854+0+0 (secure 0 0 0) 0x7fce98046020 con 0x7fcea0105490 2024-03-21T10:25:03.019 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:03.018+0000 7fce97e00640 1 --2- 172.21.15.47:0/784843382 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7fce840a2f20 0x7fce840a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:25:03.019 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:03.018+0000 7fce97e00640 1 -- 172.21.15.47:0/784843382 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9216+0+0 (secure 0 0 0) 0x7fce980f1e80 con 0x7fcea0105490 2024-03-21T10:25:03.019 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:03.019+0000 7fce9ea00640 1 --2- 172.21.15.47:0/784843382 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7fce840a2f20 0x7fce840a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:25:03.020 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:03.019+0000 7fce9ea00640 1 --2- 172.21.15.47:0/784843382 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7fce840a2f20 0x7fce840a53e0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fcea0104930 tx=0x7fce90009670 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:25:03.020 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:03.020+0000 7fce97e00640 1 -- 172.21.15.47:0/784843382 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fce9804d020 con 0x7fcea0105490 2024-03-21T10:25:03.235 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:03.234+0000 7fcea5600640 1 -- 172.21.15.47:0/784843382 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fcea002d200 con 0x7fcea0105490 2024-03-21T10:25:03.235 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:03.234+0000 7fce97e00640 1 -- 172.21.15.47:0/784843382 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v95) v1 ==== 104+0+11 (secure 0 0 0) 0x7fce980be8e0 con 0x7fcea0105490 2024-03-21T10:25:03.235 INFO:teuthology.orchestra.run.smithi047.stdout:pg_num: 32 2024-03-21T10:25:03.238 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:03.237+0000 7fcea5600640 1 -- 172.21.15.47:0/784843382 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7fce840a2f20 msgr2=0x7fce840a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:25:03.238 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:03.237+0000 7fcea5600640 1 --2- 172.21.15.47:0/784843382 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7fce840a2f20 0x7fce840a53e0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fcea0104930 tx=0x7fce90009670 comp rx=0 tx=0).stop 2024-03-21T10:25:03.238 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:03.237+0000 7fcea5600640 1 -- 172.21.15.47:0/784843382 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fcea0105490 msgr2=0x7fcea01a10a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:25:03.239 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:03.237+0000 7fcea5600640 1 --2- 172.21.15.47:0/784843382 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fcea0105490 0x7fcea01a10a0 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7fce98009230 tx=0x7fce980042f0 comp rx=0 tx=0).stop 2024-03-21T10:25:03.239 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:03.237+0000 7fcea5600640 1 -- 172.21.15.47:0/784843382 shutdown_connections 2024-03-21T10:25:03.239 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:03.237+0000 7fcea5600640 1 --2- 172.21.15.47:0/784843382 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7fce840a2f20 0x7fce840a53e0 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:25:03.239 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:03.237+0000 7fcea5600640 1 --2- 172.21.15.47:0/784843382 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fcea0105490 0x7fcea01a10a0 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:25:03.239 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:03.237+0000 7fcea5600640 1 --2- 172.21.15.47:0/784843382 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcea0104ad0 0x7fcea019a030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:25:03.239 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:03.237+0000 7fcea5600640 1 --2- 172.21.15.47:0/784843382 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcea01038e0 0x7fcea0199af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:25:03.239 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:03.237+0000 7fcea5600640 1 -- 172.21.15.47:0/784843382 >> 172.21.15.47:0/784843382 conn(0x7fcea00fee50 msgr2=0x7fcea0107e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:25:03.239 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:03.238+0000 7fcea5600640 1 -- 172.21.15.47:0/784843382 shutdown_connections 2024-03-21T10:25:03.239 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:03.238+0000 7fcea5600640 1 -- 172.21.15.47:0/784843382 wait complete. 2024-03-21T10:25:03.259 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-21T10:25:03.493 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:03.492+0000 7f350be00640 1 -- 172.21.15.47:0/1400683106 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f350c105370 msgr2=0x7f350c10bc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:25:03.493 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:03.492+0000 7f350be00640 1 --2- 172.21.15.47:0/1400683106 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f350c105370 0x7f350c10bc00 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f3504009ca0 tx=0x7f350402c100 comp rx=0 tx=0).stop 2024-03-21T10:25:03.494 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:03.492+0000 7f350be00640 1 -- 172.21.15.47:0/1400683106 shutdown_connections 2024-03-21T10:25:03.494 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:03.492+0000 7f350be00640 1 --2- 172.21.15.47:0/1400683106 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f350c105370 0x7f350c10bc00 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:25:03.494 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:03.492+0000 7f350be00640 1 --2- 172.21.15.47:0/1400683106 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f350c1049b0 0x7f350c104e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:25:03.494 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:03.492+0000 7f350be00640 1 --2- 172.21.15.47:0/1400683106 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f350c1037c0 0x7f350c103be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:25:03.494 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:03.492+0000 7f350be00640 1 -- 172.21.15.47:0/1400683106 >> 172.21.15.47:0/1400683106 conn(0x7f350c0fed50 msgr2=0x7f350c101170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:25:03.494 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:03.492+0000 7f350be00640 1 -- 172.21.15.47:0/1400683106 shutdown_connections 2024-03-21T10:25:03.494 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:03.492+0000 7f350be00640 1 -- 172.21.15.47:0/1400683106 wait complete. 2024-03-21T10:25:03.496 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:03.493+0000 7f350be00640 1 Processor -- start 2024-03-21T10:25:03.496 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:03.493+0000 7f350be00640 1 -- start start 2024-03-21T10:25:03.496 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:03.494+0000 7f350be00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f350c1037c0 0x7f350c199960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:25:03.496 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:03.494+0000 7f350be00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f350c1049b0 0x7f350c199ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:25:03.496 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:03.494+0000 7f350be00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f350c105370 0x7f350c1a0f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:25:03.496 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:03.494+0000 7f350be00640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f350c10f2f0 con 0x7f350c1049b0 2024-03-21T10:25:03.496 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:03.494+0000 7f350be00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f350c10f170 con 0x7f350c1037c0 2024-03-21T10:25:03.496 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:03.494+0000 7f350be00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f350c10f470 con 0x7f350c105370 2024-03-21T10:25:03.497 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:03.494+0000 7f3503e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f350c1049b0 0x7f350c199ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:25:03.497 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:03.494+0000 7f350aa00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f350c1037c0 0x7f350c199960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:25:03.497 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:03.494+0000 7f3503e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f350c1049b0 0x7f350c199ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:56214/0 (socket says 172.21.15.47:56214) 2024-03-21T10:25:03.497 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:03.494+0000 7f3503e00640 1 -- 172.21.15.47:0/2723417398 learned_addr learned my addr 172.21.15.47:0/2723417398 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:25:03.497 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:03.495+0000 7f350b400640 1 --2- 172.21.15.47:0/2723417398 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f350c105370 0x7f350c1a0f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:25:03.497 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:03.495+0000 7f3503e00640 1 -- 172.21.15.47:0/2723417398 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f350c105370 msgr2=0x7f350c1a0f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:25:03.497 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:03.495+0000 7f3503e00640 1 --2- 172.21.15.47:0/2723417398 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f350c105370 0x7f350c1a0f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:25:03.497 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:03.495+0000 7f3503e00640 1 -- 172.21.15.47:0/2723417398 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f350c1037c0 msgr2=0x7f350c199960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:25:03.497 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:03.495+0000 7f3503e00640 1 --2- 172.21.15.47:0/2723417398 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f350c1037c0 0x7f350c199960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:25:03.497 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:03.495+0000 7f3503e00640 1 -- 172.21.15.47:0/2723417398 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f350c1a1580 con 0x7f350c1049b0 2024-03-21T10:25:03.497 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:03.495+0000 7f350aa00640 1 --2- 172.21.15.47:0/2723417398 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f350c1037c0 0x7f350c199960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T10:25:03.497 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:03.495+0000 7f3503e00640 1 --2- 172.21.15.47:0/2723417398 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f350c1049b0 0x7f350c199ea0 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f34f400e9f0 tx=0x7f34f400edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:25:03.498 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:03.495+0000 7f3503400640 1 -- 172.21.15.47:0/2723417398 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34f400cd80 con 0x7f350c1049b0 2024-03-21T10:25:03.498 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:03.496+0000 7f3503400640 1 -- 172.21.15.47:0/2723417398 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f34f40054d0 con 0x7f350c1049b0 2024-03-21T10:25:03.498 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:03.496+0000 7f350be00640 1 -- 172.21.15.47:0/2723417398 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f350c1a1870 con 0x7f350c1049b0 2024-03-21T10:25:03.498 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:03.496+0000 7f3503400640 1 -- 172.21.15.47:0/2723417398 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34f4010490 con 0x7f350c1049b0 2024-03-21T10:25:03.498 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:03.496+0000 7f350be00640 1 -- 172.21.15.47:0/2723417398 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f350c1a1d70 con 0x7f350c1049b0 2024-03-21T10:25:03.499 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:03.497+0000 7f350be00640 1 -- 172.21.15.47:0/2723417398 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f350c10aae0 con 0x7f350c1049b0 2024-03-21T10:25:03.500 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:03.499+0000 7f3503400640 1 -- 172.21.15.47:0/2723417398 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 18) v1 ==== 137854+0+0 (secure 0 0 0) 0x7f34f4005020 con 0x7f350c1049b0 2024-03-21T10:25:03.502 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:03.501+0000 7f3503400640 1 --2- 172.21.15.47:0/2723417398 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f34f00a2f20 0x7f34f00a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:25:03.502 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:03.501+0000 7f3503400640 1 -- 172.21.15.47:0/2723417398 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9216+0+0 (secure 0 0 0) 0x7f34f40cfbf0 con 0x7f350c1049b0 2024-03-21T10:25:03.502 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:03.502+0000 7f350aa00640 1 --2- 172.21.15.47:0/2723417398 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f34f00a2f20 0x7f34f00a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:25:03.503 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:03.502+0000 7f350aa00640 1 --2- 172.21.15.47:0/2723417398 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f34f00a2f20 0x7f34f00a53e0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f350c104810 tx=0x7f34fc005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:25:03.507 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:03.506+0000 7f3503400640 1 -- 172.21.15.47:0/2723417398 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f34f409c610 con 0x7f350c1049b0 2024-03-21T10:25:03.713 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:25:03 smithi047 ceph-mon[31547]: pgmap v463: 129 pgs: 129 active+clean; 645 KiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 1.2 KiB/s wr, 10 op/s 2024-03-21T10:25:03.714 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:25:03 smithi047 ceph-mon[31547]: from='client.? 172.21.15.47:0/261408825' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-21T10:25:03.714 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:25:03 smithi047 ceph-mon[31547]: from='client.? 172.21.15.47:0/784843382' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-21T10:25:03.714 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:03.712+0000 7f350be00640 1 -- 172.21.15.47:0/2723417398 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f350c0622a0 con 0x7f350c1049b0 2024-03-21T10:25:03.714 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:03.713+0000 7f3503400640 1 -- 172.21.15.47:0/2723417398 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v95) v1 ==== 100+0+11 (secure 0 0 0) 0x7f34f409bf10 con 0x7f350c1049b0 2024-03-21T10:25:03.714 INFO:teuthology.orchestra.run.smithi047.stdout:pg_num: 64 2024-03-21T10:25:03.719 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:03.717+0000 7f350be00640 1 -- 172.21.15.47:0/2723417398 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f34f00a2f20 msgr2=0x7f34f00a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:25:03.719 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:03.717+0000 7f350be00640 1 --2- 172.21.15.47:0/2723417398 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f34f00a2f20 0x7f34f00a53e0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f350c104810 tx=0x7f34fc005f90 comp rx=0 tx=0).stop 2024-03-21T10:25:03.719 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:03.717+0000 7f350be00640 1 -- 172.21.15.47:0/2723417398 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f350c1049b0 msgr2=0x7f350c199ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:25:03.719 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:03.717+0000 7f350be00640 1 --2- 172.21.15.47:0/2723417398 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f350c1049b0 0x7f350c199ea0 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f34f400e9f0 tx=0x7f34f400edf0 comp rx=0 tx=0).stop 2024-03-21T10:25:03.719 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:03.717+0000 7f350be00640 1 -- 172.21.15.47:0/2723417398 shutdown_connections 2024-03-21T10:25:03.720 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:03.717+0000 7f350be00640 1 --2- 172.21.15.47:0/2723417398 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f34f00a2f20 0x7f34f00a53e0 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:25:03.720 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:03.717+0000 7f350be00640 1 --2- 172.21.15.47:0/2723417398 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f350c105370 0x7f350c1a0f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:25:03.720 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:03.717+0000 7f350be00640 1 --2- 172.21.15.47:0/2723417398 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f350c1049b0 0x7f350c199ea0 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:25:03.720 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:03.717+0000 7f350be00640 1 --2- 172.21.15.47:0/2723417398 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f350c1037c0 0x7f350c199960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:25:03.720 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:03.717+0000 7f350be00640 1 -- 172.21.15.47:0/2723417398 >> 172.21.15.47:0/2723417398 conn(0x7f350c0fed50 msgr2=0x7f350c107cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:25:03.720 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:03.717+0000 7f350be00640 1 -- 172.21.15.47:0/2723417398 shutdown_connections 2024-03-21T10:25:03.720 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:03.717+0000 7f350be00640 1 -- 172.21.15.47:0/2723417398 wait complete. 2024-03-21T10:25:03.729 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:25:03 smithi100 ceph-mon[35193]: pgmap v463: 129 pgs: 129 active+clean; 645 KiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 1.2 KiB/s wr, 10 op/s 2024-03-21T10:25:03.729 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:25:03 smithi100 ceph-mon[35193]: from='client.? 172.21.15.47:0/261408825' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-21T10:25:03.729 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:25:03 smithi100 ceph-mon[35193]: from='client.? 172.21.15.47:0/784843382' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-21T10:25:03.739 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:25:03 smithi157 ceph-mon[35581]: pgmap v463: 129 pgs: 129 active+clean; 645 KiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 1.2 KiB/s wr, 10 op/s 2024-03-21T10:25:03.740 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:25:03 smithi157 ceph-mon[35581]: from='client.? 172.21.15.47:0/261408825' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-21T10:25:03.740 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:25:03 smithi157 ceph-mon[35581]: from='client.? 172.21.15.47:0/784843382' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-21T10:25:03.741 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-03-21T10:25:03.964 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:03.963+0000 7f2ce2200640 1 -- 172.21.15.47:0/2440529675 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2cdc077c40 msgr2=0x7f2cdc0784e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:25:03.964 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:03.963+0000 7f2ce2200640 1 --2- 172.21.15.47:0/2440529675 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2cdc077c40 0x7f2cdc0784e0 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f2cd4009ca0 tx=0x7f2cd402c0f0 comp rx=0 tx=0).stop 2024-03-21T10:25:03.965 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:03.963+0000 7f2ce2200640 1 -- 172.21.15.47:0/2440529675 shutdown_connections 2024-03-21T10:25:03.965 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:03.963+0000 7f2ce2200640 1 --2- 172.21.15.47:0/2440529675 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2cdc077c40 0x7f2cdc0784e0 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:25:03.965 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:03.963+0000 7f2ce2200640 1 --2- 172.21.15.47:0/2440529675 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2cdc0772a0 0x7f2cdc077700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:25:03.965 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:03.963+0000 7f2ce2200640 1 --2- 172.21.15.47:0/2440529675 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2cdc076060 0x7f2cdc076480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:25:03.965 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:03.963+0000 7f2ce2200640 1 -- 172.21.15.47:0/2440529675 >> 172.21.15.47:0/2440529675 conn(0x7f2cdc0fed10 msgr2=0x7f2cdc101150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:25:03.965 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:03.963+0000 7f2ce2200640 1 -- 172.21.15.47:0/2440529675 shutdown_connections 2024-03-21T10:25:03.965 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:03.963+0000 7f2ce2200640 1 -- 172.21.15.47:0/2440529675 wait complete. 2024-03-21T10:25:03.967 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:03.964+0000 7f2ce2200640 1 Processor -- start 2024-03-21T10:25:03.967 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:03.964+0000 7f2ce2200640 1 -- start start 2024-03-21T10:25:03.967 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:03.965+0000 7f2ce2200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2cdc076060 0x7f2cdc19dd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:25:03.967 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:03.965+0000 7f2ce2200640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2cdc0772a0 0x7f2cdc19e290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:25:03.967 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:03.965+0000 7f2ce2200640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2cdc077c40 0x7f2cdc1a52c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:25:03.968 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:03.965+0000 7f2ce2200640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2cdc1136e0 con 0x7f2cdc0772a0 2024-03-21T10:25:03.968 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:03.965+0000 7f2ce2200640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2cdc113560 con 0x7f2cdc076060 2024-03-21T10:25:03.968 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:03.965+0000 7f2ce2200640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2cdc113860 con 0x7f2cdc077c40 2024-03-21T10:25:03.968 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:03.965+0000 7f2cdbe00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2cdc0772a0 0x7f2cdc19e290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:25:03.968 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:03.965+0000 7f2cdbe00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2cdc0772a0 0x7f2cdc19e290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:56232/0 (socket says 172.21.15.47:56232) 2024-03-21T10:25:03.968 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:03.965+0000 7f2cdbe00640 1 -- 172.21.15.47:0/4105408016 learned_addr learned my addr 172.21.15.47:0/4105408016 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:25:03.968 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:03.965+0000 7f2ce0e00640 1 --2- 172.21.15.47:0/4105408016 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2cdc076060 0x7f2cdc19dd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:25:03.968 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:03.965+0000 7f2ce1800640 1 --2- 172.21.15.47:0/4105408016 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2cdc077c40 0x7f2cdc1a52c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:25:03.968 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:03.965+0000 7f2cdbe00640 1 -- 172.21.15.47:0/4105408016 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2cdc077c40 msgr2=0x7f2cdc1a52c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:25:03.968 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:03.965+0000 7f2cdbe00640 1 --2- 172.21.15.47:0/4105408016 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2cdc077c40 0x7f2cdc1a52c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:25:03.968 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:03.966+0000 7f2cdbe00640 1 -- 172.21.15.47:0/4105408016 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2cdc076060 msgr2=0x7f2cdc19dd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:25:03.968 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:03.966+0000 7f2cdbe00640 1 --2- 172.21.15.47:0/4105408016 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2cdc076060 0x7f2cdc19dd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:25:03.968 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:03.966+0000 7f2cdbe00640 1 -- 172.21.15.47:0/4105408016 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2cdc1a5960 con 0x7f2cdc0772a0 2024-03-21T10:25:03.969 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:03.966+0000 7f2cdbe00640 1 --2- 172.21.15.47:0/4105408016 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2cdc0772a0 0x7f2cdc19e290 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f2cc400d770 tx=0x7f2cc400dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:25:03.969 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:03.966+0000 7f2cd9600640 1 -- 172.21.15.47:0/4105408016 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2cc4017070 con 0x7f2cdc0772a0 2024-03-21T10:25:03.969 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:03.966+0000 7f2cd9600640 1 -- 172.21.15.47:0/4105408016 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f2cc40054d0 con 0x7f2cdc0772a0 2024-03-21T10:25:03.969 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:03.966+0000 7f2cd9600640 1 -- 172.21.15.47:0/4105408016 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2cc4004cf0 con 0x7f2cdc0772a0 2024-03-21T10:25:03.969 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:03.966+0000 7f2ce2200640 1 -- 172.21.15.47:0/4105408016 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2cdc1a5c50 con 0x7f2cdc0772a0 2024-03-21T10:25:03.969 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:03.966+0000 7f2ce2200640 1 -- 172.21.15.47:0/4105408016 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2cdc1a6150 con 0x7f2cdc0772a0 2024-03-21T10:25:03.969 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:03.968+0000 7f2ce2200640 1 -- 172.21.15.47:0/4105408016 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2cdc110110 con 0x7f2cdc0772a0 2024-03-21T10:25:03.972 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:03.971+0000 7f2cd9600640 1 -- 172.21.15.47:0/4105408016 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 18) v1 ==== 137854+0+0 (secure 0 0 0) 0x7f2cc4004e90 con 0x7f2cdc0772a0 2024-03-21T10:25:03.975 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:03.973+0000 7f2cd9600640 1 --2- 172.21.15.47:0/4105408016 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f2cc00a2fa0 0x7f2cc00a5460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:25:03.975 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:03.974+0000 7f2cd9600640 1 -- 172.21.15.47:0/4105408016 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9216+0+0 (secure 0 0 0) 0x7f2cc40cf250 con 0x7f2cdc0772a0 2024-03-21T10:25:03.975 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:03.974+0000 7f2ce0e00640 1 --2- 172.21.15.47:0/4105408016 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f2cc00a2fa0 0x7f2cc00a5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:25:03.975 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:03.974+0000 7f2ce0e00640 1 --2- 172.21.15.47:0/4105408016 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f2cc00a2fa0 0x7f2cc00a5460 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f2cdc19f220 tx=0x7f2ccc00b390 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:25:03.979 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:03.979+0000 7f2cd9600640 1 -- 172.21.15.47:0/4105408016 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f2cc409bc70 con 0x7f2cdc0772a0 2024-03-21T10:25:04.180 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:04.179+0000 7f2ce2200640 1 -- 172.21.15.47:0/4105408016 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f2cdc02d320 con 0x7f2cdc0772a0 2024-03-21T10:25:04.180 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:04.179+0000 7f2cd9600640 1 -- 172.21.15.47:0/4105408016 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v95) v1 ==== 103+0+11 (secure 0 0 0) 0x7f2cc409b570 con 0x7f2cdc0772a0 2024-03-21T10:25:04.181 INFO:teuthology.orchestra.run.smithi047.stdout:pg_num: 32 2024-03-21T10:25:04.182 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:04.182+0000 7f2ce2200640 1 -- 172.21.15.47:0/4105408016 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f2cc00a2fa0 msgr2=0x7f2cc00a5460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:25:04.183 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:04.182+0000 7f2ce2200640 1 --2- 172.21.15.47:0/4105408016 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f2cc00a2fa0 0x7f2cc00a5460 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f2cdc19f220 tx=0x7f2ccc00b390 comp rx=0 tx=0).stop 2024-03-21T10:25:04.183 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:04.182+0000 7f2ce2200640 1 -- 172.21.15.47:0/4105408016 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2cdc0772a0 msgr2=0x7f2cdc19e290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:25:04.183 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:04.182+0000 7f2ce2200640 1 --2- 172.21.15.47:0/4105408016 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2cdc0772a0 0x7f2cdc19e290 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f2cc400d770 tx=0x7f2cc400dc30 comp rx=0 tx=0).stop 2024-03-21T10:25:04.183 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:04.182+0000 7f2ce2200640 1 -- 172.21.15.47:0/4105408016 shutdown_connections 2024-03-21T10:25:04.183 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:04.182+0000 7f2ce2200640 1 --2- 172.21.15.47:0/4105408016 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f2cc00a2fa0 0x7f2cc00a5460 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:25:04.184 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:04.182+0000 7f2ce2200640 1 --2- 172.21.15.47:0/4105408016 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2cdc077c40 0x7f2cdc1a52c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:25:04.184 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:04.182+0000 7f2ce2200640 1 --2- 172.21.15.47:0/4105408016 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2cdc0772a0 0x7f2cdc19e290 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:25:04.184 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:04.182+0000 7f2ce2200640 1 --2- 172.21.15.47:0/4105408016 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2cdc076060 0x7f2cdc19dd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:25:04.184 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:04.182+0000 7f2ce2200640 1 -- 172.21.15.47:0/4105408016 >> 172.21.15.47:0/4105408016 conn(0x7f2cdc0fed10 msgr2=0x7f2cdc100810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:25:04.184 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:04.182+0000 7f2ce2200640 1 -- 172.21.15.47:0/4105408016 shutdown_connections 2024-03-21T10:25:04.184 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:04.182+0000 7f2ce2200640 1 -- 172.21.15.47:0/4105408016 wait complete. 2024-03-21T10:25:04.201 INFO:tasks.fwd_scrub:Wait for all MDSs to reach steady state... 2024-03-21T10:25:04.202 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-21T10:25:04.421 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:04.420+0000 7efddd800640 1 -- 172.21.15.47:0/2416013182 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7efdd8107250 msgr2=0x7efdd8109640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:25:04.422 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:04.420+0000 7efddd800640 1 --2- 172.21.15.47:0/2416013182 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7efdd8107250 0x7efdd8109640 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7efdd0009ca0 tx=0x7efdd002c0f0 comp rx=0 tx=0).stop 2024-03-21T10:25:04.422 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:04.420+0000 7efddd800640 1 -- 172.21.15.47:0/2416013182 shutdown_connections 2024-03-21T10:25:04.422 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:04.420+0000 7efddd800640 1 --2- 172.21.15.47:0/2416013182 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7efdd8107250 0x7efdd8109640 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:25:04.422 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:04.420+0000 7efddd800640 1 --2- 172.21.15.47:0/2416013182 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7efdd8104920 0x7efdd8106d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:25:04.422 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:04.420+0000 7efddd800640 1 --2- 172.21.15.47:0/2416013182 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7efdd8101ff0 0x7efdd81043e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:25:04.422 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:04.420+0000 7efddd800640 1 -- 172.21.15.47:0/2416013182 >> 172.21.15.47:0/2416013182 conn(0x7efdd80fb980 msgr2=0x7efdd80fdda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:25:04.422 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:04.421+0000 7efddd800640 1 -- 172.21.15.47:0/2416013182 shutdown_connections 2024-03-21T10:25:04.422 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:04.421+0000 7efddd800640 1 -- 172.21.15.47:0/2416013182 wait complete. 2024-03-21T10:25:04.424 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:04.422+0000 7efddd800640 1 Processor -- start 2024-03-21T10:25:04.424 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:04.422+0000 7efddd800640 1 -- start start 2024-03-21T10:25:04.424 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:04.422+0000 7efddd800640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7efdd8101ff0 0x7efdd8199930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:25:04.424 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:04.422+0000 7efddd800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7efdd8104920 0x7efdd8199e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:25:04.424 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:04.422+0000 7efddd800640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7efdd8107250 0x7efdd81a0ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:25:04.424 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:04.422+0000 7efddd800640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efdd810d030 con 0x7efdd8101ff0 2024-03-21T10:25:04.424 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:04.422+0000 7efddd800640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efdd810ceb0 con 0x7efdd8104920 2024-03-21T10:25:04.424 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:04.422+0000 7efddd800640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efdd810d1b0 con 0x7efdd8107250 2024-03-21T10:25:04.425 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:04.422+0000 7efdd7e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7efdd8101ff0 0x7efdd8199930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:25:04.425 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:04.422+0000 7efdd7e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7efdd8101ff0 0x7efdd8199930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:56250/0 (socket says 172.21.15.47:56250) 2024-03-21T10:25:04.425 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:04.422+0000 7efdd7e00640 1 -- 172.21.15.47:0/1123821566 learned_addr learned my addr 172.21.15.47:0/1123821566 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:25:04.425 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:04.423+0000 7efdcfe00640 1 --2- 172.21.15.47:0/1123821566 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7efdd8104920 0x7efdd8199e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:25:04.425 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:04.423+0000 7efddce00640 1 --2- 172.21.15.47:0/1123821566 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7efdd8107250 0x7efdd81a0ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:25:04.425 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:04.423+0000 7efdd7e00640 1 -- 172.21.15.47:0/1123821566 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7efdd8107250 msgr2=0x7efdd81a0ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:25:04.425 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:04.423+0000 7efdd7e00640 1 --2- 172.21.15.47:0/1123821566 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7efdd8107250 0x7efdd81a0ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:25:04.426 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:04.423+0000 7efdd7e00640 1 -- 172.21.15.47:0/1123821566 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7efdd8104920 msgr2=0x7efdd8199e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:25:04.426 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:04.423+0000 7efdd7e00640 1 --2- 172.21.15.47:0/1123821566 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7efdd8104920 0x7efdd8199e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:25:04.426 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:04.423+0000 7efdd7e00640 1 -- 172.21.15.47:0/1123821566 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efdd81a1550 con 0x7efdd8101ff0 2024-03-21T10:25:04.426 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:04.423+0000 7efddce00640 1 --2- 172.21.15.47:0/1123821566 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7efdd8107250 0x7efdd81a0ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T10:25:04.426 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:04.423+0000 7efdd7e00640 1 --2- 172.21.15.47:0/1123821566 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7efdd8101ff0 0x7efdd8199930 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7efdc800e9f0 tx=0x7efdc800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:25:04.427 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:04.424+0000 7efdd5600640 1 -- 172.21.15.47:0/1123821566 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efdc800cd80 con 0x7efdd8101ff0 2024-03-21T10:25:04.427 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:04.424+0000 7efdd5600640 1 -- 172.21.15.47:0/1123821566 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7efdc80108e0 con 0x7efdd8101ff0 2024-03-21T10:25:04.427 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:04.424+0000 7efddd800640 1 -- 172.21.15.47:0/1123821566 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efdd81a1840 con 0x7efdd8101ff0 2024-03-21T10:25:04.427 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:04.424+0000 7efdd5600640 1 -- 172.21.15.47:0/1123821566 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efdc8018920 con 0x7efdd8101ff0 2024-03-21T10:25:04.427 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:04.424+0000 7efddd800640 1 -- 172.21.15.47:0/1123821566 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efdd81a1d90 con 0x7efdd8101ff0 2024-03-21T10:25:04.427 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:04.425+0000 7efddd800640 1 -- 172.21.15.47:0/1123821566 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efdd8108d40 con 0x7efdd8101ff0 2024-03-21T10:25:04.428 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:04.427+0000 7efdd5600640 1 -- 172.21.15.47:0/1123821566 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 18) v1 ==== 137854+0+0 (secure 0 0 0) 0x7efdc8010a90 con 0x7efdd8101ff0 2024-03-21T10:25:04.430 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:04.429+0000 7efdd5600640 1 --2- 172.21.15.47:0/1123821566 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7efdbc0a2f20 0x7efdbc0a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:25:04.430 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:04.430+0000 7efdd5600640 1 -- 172.21.15.47:0/1123821566 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9216+0+0 (secure 0 0 0) 0x7efdc80d0010 con 0x7efdd8101ff0 2024-03-21T10:25:04.431 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:04.430+0000 7efdcfe00640 1 --2- 172.21.15.47:0/1123821566 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7efdbc0a2f20 0x7efdbc0a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:25:04.431 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:04.431+0000 7efdcfe00640 1 --2- 172.21.15.47:0/1123821566 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7efdbc0a2f20 0x7efdbc0a53e0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7efdd819ae40 tx=0x7efdc000b390 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:25:04.433 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:04.433+0000 7efdd5600640 1 -- 172.21.15.47:0/1123821566 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7efdc809ca70 con 0x7efdd8101ff0 2024-03-21T10:25:04.463 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:25:04 smithi047 ceph-mon[31547]: from='client.? 172.21.15.47:0/2723417398' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-21T10:25:04.463 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:25:04 smithi047 ceph-mon[31547]: from='client.? 172.21.15.47:0/4105408016' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-03-21T10:25:04.678 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:04.678+0000 7efddd800640 1 -- 172.21.15.47:0/1123821566 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7efdd802d070 con 0x7efdd8101ff0 2024-03-21T10:25:04.679 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:04.679+0000 7efdd5600640 1 -- 172.21.15.47:0/1123821566 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) v1 ==== 94+0+11471 (secure 0 0 0) 0x7efdc809c370 con 0x7efdd8101ff0 2024-03-21T10:25:04.680 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-21T10:25:04.680 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":30,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14511,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6835/735724037","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6834","nonce":735724037},{"type":"v1","addr":"172.21.15.47:6835","nonce":735724037}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14529,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6837/2248191724","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6836","nonce":2248191724},{"type":"v1","addr":"172.21.15.47:6837","nonce":2248191724}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":30,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-21T10:21:58.329063+0000","modified":"2024-03-21T10:24:58.786909+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":34535,"mds_1":34493,"mds_2":34454,"mds_3":34427,"mds_4":14619},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14604":{"gid":14604,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.47:6839/2881335356","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6838","nonce":2881335356},{"type":"v1","addr":"172.21.15.47:6839","nonce":2881335356}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_14619":{"gid":14619,"name":"a","rank":4,"incarnation":27,"state":"up:active","state_seq":11,"addr":"172.21.15.47:6841/3401578978","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6840","nonce":3401578978},{"type":"v1","addr":"172.21.15.47:6841","nonce":3401578978}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24406":{"gid":24406,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.157:6833/4000010136","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":4000010136},{"type":"v1","addr":"172.21.15.157:6833","nonce":4000010136}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24457":{"gid":24457,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.157:6835/3902868651","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":3902868651},{"type":"v1","addr":"172.21.15.157:6835","nonce":3902868651}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24475":{"gid":24475,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.157:6837/1908087204","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6836","nonce":1908087204},{"type":"v1","addr":"172.21.15.157:6837","nonce":1908087204}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24499":{"gid":24499,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.157:6839/4064242876","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6838","nonce":4064242876},{"type":"v1","addr":"172.21.15.157:6839","nonce":4064242876}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_34427":{"gid":34427,"name":"k","rank":3,"incarnation":25,"state":"up:active","state_seq":20,"addr":"172.21.15.100:6833/2723302038","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6832","nonce":2723302038},{"type":"v1","addr":"172.21.15.100:6833","nonce":2723302038}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_34454":{"gid":34454,"name":"e","rank":2,"incarnation":22,"state":"up:active","state_seq":17,"addr":"172.21.15.100:6835/2597849957","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":2597849957},{"type":"v1","addr":"172.21.15.100:6835","nonce":2597849957}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_34493":{"gid":34493,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":13,"addr":"172.21.15.100:6837/3497644162","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6836","nonce":3497644162},{"type":"v1","addr":"172.21.15.100:6837","nonce":3497644162}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_34535":{"gid":34535,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.100:6839/4173168440","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6838","nonce":4173168440},{"type":"v1","addr":"172.21.15.100:6839","nonce":4173168440}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":34535,"qdb_cluster":[14619,34427,34454,34493,34535]},"id":1}]} 2024-03-21T10:25:04.680 INFO:teuthology.orchestra.run.smithi047.stderr:dumped fsmap epoch 30 2024-03-21T10:25:04.682 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:04.681+0000 7efddd800640 1 -- 172.21.15.47:0/1123821566 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7efdbc0a2f20 msgr2=0x7efdbc0a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:25:04.683 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:04.681+0000 7efddd800640 1 --2- 172.21.15.47:0/1123821566 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7efdbc0a2f20 0x7efdbc0a53e0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7efdd819ae40 tx=0x7efdc000b390 comp rx=0 tx=0).stop 2024-03-21T10:25:04.683 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:04.681+0000 7efddd800640 1 -- 172.21.15.47:0/1123821566 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7efdd8101ff0 msgr2=0x7efdd8199930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:25:04.683 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:04.681+0000 7efddd800640 1 --2- 172.21.15.47:0/1123821566 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7efdd8101ff0 0x7efdd8199930 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7efdc800e9f0 tx=0x7efdc800eeb0 comp rx=0 tx=0).stop 2024-03-21T10:25:04.683 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:04.681+0000 7efddd800640 1 -- 172.21.15.47:0/1123821566 shutdown_connections 2024-03-21T10:25:04.683 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:04.681+0000 7efddd800640 1 --2- 172.21.15.47:0/1123821566 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7efdbc0a2f20 0x7efdbc0a53e0 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:25:04.683 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:04.681+0000 7efddd800640 1 --2- 172.21.15.47:0/1123821566 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7efdd8107250 0x7efdd81a0ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:25:04.683 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:04.681+0000 7efddd800640 1 --2- 172.21.15.47:0/1123821566 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7efdd8104920 0x7efdd8199e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:25:04.683 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:04.681+0000 7efddd800640 1 --2- 172.21.15.47:0/1123821566 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7efdd8101ff0 0x7efdd8199930 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:25:04.683 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:04.681+0000 7efddd800640 1 -- 172.21.15.47:0/1123821566 >> 172.21.15.47:0/1123821566 conn(0x7efdd80fb980 msgr2=0x7efdd8101800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:25:04.683 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:04.682+0000 7efddd800640 1 -- 172.21.15.47:0/1123821566 shutdown_connections 2024-03-21T10:25:04.683 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:04.682+0000 7efddd800640 1 -- 172.21.15.47:0/1123821566 wait complete. 2024-03-21T10:25:04.703 INFO:tasks.fwd_scrub:Ready to start scrub thrashing 2024-03-21T10:25:04.703 INFO:tasks.fwd_scrub.ceph_manager:waiting for clean 2024-03-21T10:25:04.703 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-21T10:25:04.729 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:25:04 smithi100 ceph-mon[35193]: from='client.? 172.21.15.47:0/2723417398' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-21T10:25:04.729 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:25:04 smithi100 ceph-mon[35193]: from='client.? 172.21.15.47:0/4105408016' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-03-21T10:25:04.739 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:25:04 smithi157 ceph-mon[35581]: from='client.? 172.21.15.47:0/2723417398' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-21T10:25:04.740 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:25:04 smithi157 ceph-mon[35581]: from='client.? 172.21.15.47:0/4105408016' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-03-21T10:25:04.925 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:04.924+0000 7f996bc00640 1 -- 172.21.15.47:0/3387149092 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f99641038e0 msgr2=0x7f9964103d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:25:04.925 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:04.924+0000 7f996bc00640 1 --2- 172.21.15.47:0/3387149092 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f99641038e0 0x7f9964103d00 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f9958009900 tx=0x7f995802bc30 comp rx=0 tx=0).stop 2024-03-21T10:25:04.925 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:04.924+0000 7f996bc00640 1 -- 172.21.15.47:0/3387149092 shutdown_connections 2024-03-21T10:25:04.926 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:04.924+0000 7f996bc00640 1 --2- 172.21.15.47:0/3387149092 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9964105490 0x7f996410bd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:25:04.926 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:04.924+0000 7f996bc00640 1 --2- 172.21.15.47:0/3387149092 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9964104ad0 0x7f9964104f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:25:04.926 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:04.924+0000 7f996bc00640 1 --2- 172.21.15.47:0/3387149092 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f99641038e0 0x7f9964103d00 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:25:04.926 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:04.924+0000 7f996bc00640 1 -- 172.21.15.47:0/3387149092 >> 172.21.15.47:0/3387149092 conn(0x7f99640fee90 msgr2=0x7f99641012b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:25:04.926 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:04.924+0000 7f996bc00640 1 -- 172.21.15.47:0/3387149092 shutdown_connections 2024-03-21T10:25:04.926 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:04.924+0000 7f996bc00640 1 -- 172.21.15.47:0/3387149092 wait complete. 2024-03-21T10:25:04.927 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:04.925+0000 7f996bc00640 1 Processor -- start 2024-03-21T10:25:04.927 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:04.925+0000 7f996bc00640 1 -- start start 2024-03-21T10:25:04.928 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:04.926+0000 7f996bc00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f99641038e0 0x7f9964199b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:25:04.928 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:04.926+0000 7f996bc00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9964104ad0 0x7f996419a090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:25:04.928 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:04.926+0000 7f996bc00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9964105490 0x7f99641a1100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:25:04.928 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:04.926+0000 7f996bc00640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f996410f450 con 0x7f99641038e0 2024-03-21T10:25:04.928 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:04.926+0000 7f996bc00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f996410f2d0 con 0x7f9964104ad0 2024-03-21T10:25:04.928 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:04.926+0000 7f996bc00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f996410f5d0 con 0x7f9964105490 2024-03-21T10:25:04.928 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:04.926+0000 7f9969200640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f99641038e0 0x7f9964199b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:25:04.928 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:04.926+0000 7f9969200640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f99641038e0 0x7f9964199b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:56268/0 (socket says 172.21.15.47:56268) 2024-03-21T10:25:04.928 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:04.926+0000 7f9969200640 1 -- 172.21.15.47:0/4210899655 learned_addr learned my addr 172.21.15.47:0/4210899655 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:25:04.928 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:04.926+0000 7f9963e00640 1 --2- 172.21.15.47:0/4210899655 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9964104ad0 0x7f996419a090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:25:04.928 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:04.926+0000 7f9969c00640 1 --2- 172.21.15.47:0/4210899655 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9964105490 0x7f99641a1100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:25:04.928 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:04.927+0000 7f9969200640 1 -- 172.21.15.47:0/4210899655 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9964105490 msgr2=0x7f99641a1100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:25:04.928 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:04.927+0000 7f9969200640 1 --2- 172.21.15.47:0/4210899655 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9964105490 0x7f99641a1100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:25:04.929 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:04.927+0000 7f9969200640 1 -- 172.21.15.47:0/4210899655 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9964104ad0 msgr2=0x7f996419a090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:25:04.929 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:04.927+0000 7f9969200640 1 --2- 172.21.15.47:0/4210899655 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9964104ad0 0x7f996419a090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:25:04.929 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:04.927+0000 7f9969200640 1 -- 172.21.15.47:0/4210899655 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99641a1770 con 0x7f99641038e0 2024-03-21T10:25:04.929 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:04.927+0000 7f9969200640 1 --2- 172.21.15.47:0/4210899655 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f99641038e0 0x7f9964199b50 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f9958000c00 tx=0x7f99580040c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:25:04.930 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:04.927+0000 7f9961600640 1 -- 172.21.15.47:0/4210899655 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9958043020 con 0x7f99641038e0 2024-03-21T10:25:04.930 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:04.927+0000 7f9961600640 1 -- 172.21.15.47:0/4210899655 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f99580044c0 con 0x7f99641038e0 2024-03-21T10:25:04.930 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:04.927+0000 7f996bc00640 1 -- 172.21.15.47:0/4210899655 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f99641a1a00 con 0x7f99641038e0 2024-03-21T10:25:04.930 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:04.927+0000 7f9961600640 1 -- 172.21.15.47:0/4210899655 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9958003c90 con 0x7f99641038e0 2024-03-21T10:25:04.930 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:04.928+0000 7f996bc00640 1 -- 172.21.15.47:0/4210899655 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f99641a1f50 con 0x7f99641038e0 2024-03-21T10:25:04.930 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:04.929+0000 7f996bc00640 1 -- 172.21.15.47:0/4210899655 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f996410acd0 con 0x7f99641038e0 2024-03-21T10:25:04.931 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:04.931+0000 7f9961600640 1 -- 172.21.15.47:0/4210899655 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 18) v1 ==== 137854+0+0 (secure 0 0 0) 0x7f9958046020 con 0x7f99641038e0 2024-03-21T10:25:04.933 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:04.932+0000 7f9961600640 1 --2- 172.21.15.47:0/4210899655 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f994c0a2f20 0x7f994c0a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:25:04.933 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:04.932+0000 7f9961600640 1 -- 172.21.15.47:0/4210899655 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9216+0+0 (secure 0 0 0) 0x7f99580f1ac0 con 0x7f99641038e0 2024-03-21T10:25:04.933 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:04.932+0000 7f9963e00640 1 --2- 172.21.15.47:0/4210899655 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f994c0a2f20 0x7f994c0a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:25:04.934 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:04.933+0000 7f9963e00640 1 --2- 172.21.15.47:0/4210899655 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f994c0a2f20 0x7f994c0a53e0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f996419b060 tx=0x7f995000b390 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:25:04.939 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:04.938+0000 7f9961600640 1 -- 172.21.15.47:0/4210899655 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f995804d020 con 0x7f99641038e0 2024-03-21T10:25:05.140 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:05.140+0000 7f996bc00640 1 -- 172.21.15.47:0/4210899655 --> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f9964061ae0 con 0x7f994c0a2f20 2024-03-21T10:25:05.152 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:05.151+0000 7f9961600640 1 -- 172.21.15.47:0/4210899655 <== mgr.14152 v2:172.21.15.47:6800/2823885994 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+340554 (secure 0 0 0) 0x7f9964061ae0 con 0x7f994c0a2f20 2024-03-21T10:25:05.157 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-21T10:25:05.159 INFO:teuthology.orchestra.run.smithi047.stderr:dumped all 2024-03-21T10:25:05.161 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:05.157+0000 7f996bc00640 1 -- 172.21.15.47:0/4210899655 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f994c0a2f20 msgr2=0x7f994c0a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:25:05.161 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:05.157+0000 7f996bc00640 1 --2- 172.21.15.47:0/4210899655 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f994c0a2f20 0x7f994c0a53e0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f996419b060 tx=0x7f995000b390 comp rx=0 tx=0).stop 2024-03-21T10:25:05.161 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:05.157+0000 7f996bc00640 1 -- 172.21.15.47:0/4210899655 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f99641038e0 msgr2=0x7f9964199b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:25:05.161 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:05.157+0000 7f996bc00640 1 --2- 172.21.15.47:0/4210899655 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f99641038e0 0x7f9964199b50 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f9958000c00 tx=0x7f99580040c0 comp rx=0 tx=0).stop 2024-03-21T10:25:05.161 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:05.157+0000 7f996bc00640 1 -- 172.21.15.47:0/4210899655 shutdown_connections 2024-03-21T10:25:05.161 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:05.157+0000 7f996bc00640 1 --2- 172.21.15.47:0/4210899655 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f994c0a2f20 0x7f994c0a53e0 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:25:05.161 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:05.157+0000 7f996bc00640 1 --2- 172.21.15.47:0/4210899655 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9964105490 0x7f99641a1100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:25:05.161 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:05.157+0000 7f996bc00640 1 --2- 172.21.15.47:0/4210899655 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9964104ad0 0x7f996419a090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:25:05.162 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:05.157+0000 7f996bc00640 1 --2- 172.21.15.47:0/4210899655 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f99641038e0 0x7f9964199b50 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:25:05.162 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:05.157+0000 7f996bc00640 1 -- 172.21.15.47:0/4210899655 >> 172.21.15.47:0/4210899655 conn(0x7f99640fee90 msgr2=0x7f9964107ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:25:05.162 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:05.158+0000 7f996bc00640 1 -- 172.21.15.47:0/4210899655 shutdown_connections 2024-03-21T10:25:05.162 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:05.158+0000 7f996bc00640 1 -- 172.21.15.47:0/4210899655 wait complete. 2024-03-21T10:25:05.187 INFO:teuthology.orchestra.run.smithi047.stdout:{"pg_ready":true,"pg_map":{"version":464,"stamp":"2024-03-21T10:25:04.096428+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":664376,"num_objects":102,"num_object_clones":0,"num_object_copies":308,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":1721,"num_read_kb":995,"num_write":451,"num_write_kb":3101,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13,"num_bytes_recovered":1185317,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":869,"ondisk_log_size":869,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":397244,"kb_used_data":14812,"kb_used_omap":18,"kb_used_meta":382317,"kb_avail":1124544580,"statfs":{"total":1151940427776,"available":1151533649920,"internally_reserved":0,"allocated":15167488,"data_stored":8282996,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19142,"internal_metadata":391492922},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":6779,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":151,"num_read_kb":85,"num_write":21,"num_write_kb":20,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.005301"},"pg_stats":[{"pgid":"3.3f","version":"95'8","reported_seq":38,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T10:23:56.049062+0000","last_change":"2024-03-21T10:21:58.872480+0000","last_active":"2024-03-21T10:23:56.049062+0000","last_peered":"2024-03-21T10:23:56.049062+0000","last_clean":"2024-03-21T10:23:56.049062+0000","last_became_active":"2024-03-21T10:21:58.872143+0000","last_became_peered":"2024-03-21T10:21:58.872143+0000","last_unstale":"2024-03-21T10:23:56.049062+0000","last_undegraded":"2024-03-21T10:23:56.049062+0000","last_fullsized":"2024-03-21T10:23:56.049062+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_clean_scrub_stamp":"2024-03-21T10:21:57.813712+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T15:20:00.409201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"95'4","reported_seq":34,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T10:23:56.027679+0000","last_change":"2024-03-21T10:21:58.871130+0000","last_active":"2024-03-21T10:23:56.027679+0000","last_peered":"2024-03-21T10:23:56.027679+0000","last_clean":"2024-03-21T10:23:56.027679+0000","last_became_active":"2024-03-21T10:21:58.870750+0000","last_became_peered":"2024-03-21T10:21:58.870750+0000","last_unstale":"2024-03-21T10:23:56.027679+0000","last_undegraded":"2024-03-21T10:23:56.027679+0000","last_fullsized":"2024-03-21T10:23:56.027679+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_clean_scrub_stamp":"2024-03-21T10:21:57.813712+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T17:38:04.830212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"95'7","reported_seq":37,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T10:23:56.048931+0000","last_change":"2024-03-21T10:21:58.868026+0000","last_active":"2024-03-21T10:23:56.048931+0000","last_peered":"2024-03-21T10:23:56.048931+0000","last_clean":"2024-03-21T10:23:56.048931+0000","last_became_active":"2024-03-21T10:21:58.859877+0000","last_became_peered":"2024-03-21T10:21:58.859877+0000","last_unstale":"2024-03-21T10:23:56.048931+0000","last_undegraded":"2024-03-21T10:23:56.048931+0000","last_fullsized":"2024-03-21T10:23:56.048931+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_clean_scrub_stamp":"2024-03-21T10:21:57.813712+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T14:45:12.870111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"95'8","reported_seq":24,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T10:23:56.009656+0000","last_change":"2024-03-21T10:22:27.231695+0000","last_active":"2024-03-21T10:23:56.009656+0000","last_peered":"2024-03-21T10:23:56.009656+0000","last_clean":"2024-03-21T10:23:56.009656+0000","last_became_active":"2024-03-21T10:22:27.231149+0000","last_became_peered":"2024-03-21T10:22:27.231149+0000","last_unstale":"2024-03-21T10:23:56.009656+0000","last_undegraded":"2024-03-21T10:23:56.009656+0000","last_fullsized":"2024-03-21T10:23:56.009656+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_clean_scrub_stamp":"2024-03-21T10:21:57.813712+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T18:19:19.105366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,2],"acting":[9,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.3b","version":"95'4","reported_seq":34,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T10:23:56.012895+0000","last_change":"2024-03-21T10:21:58.853998+0000","last_active":"2024-03-21T10:23:56.012895+0000","last_peered":"2024-03-21T10:23:56.012895+0000","last_clean":"2024-03-21T10:23:56.012895+0000","last_became_active":"2024-03-21T10:21:58.853059+0000","last_became_peered":"2024-03-21T10:21:58.853059+0000","last_unstale":"2024-03-21T10:23:56.012895+0000","last_undegraded":"2024-03-21T10:23:56.012895+0000","last_fullsized":"2024-03-21T10:23:56.012895+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_clean_scrub_stamp":"2024-03-21T10:21:57.813712+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T17:06:36.859522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"95'6","reported_seq":36,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T10:23:56.024543+0000","last_change":"2024-03-21T10:21:58.855535+0000","last_active":"2024-03-21T10:23:56.024543+0000","last_peered":"2024-03-21T10:23:56.024543+0000","last_clean":"2024-03-21T10:23:56.024543+0000","last_became_active":"2024-03-21T10:21:58.855181+0000","last_became_peered":"2024-03-21T10:21:58.855181+0000","last_unstale":"2024-03-21T10:23:56.024543+0000","last_undegraded":"2024-03-21T10:23:56.024543+0000","last_fullsized":"2024-03-21T10:23:56.024543+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_clean_scrub_stamp":"2024-03-21T10:21:57.813712+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T14:46:35.045646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"95'5","reported_seq":35,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T10:23:56.041045+0000","last_change":"2024-03-21T10:21:58.856527+0000","last_active":"2024-03-21T10:23:56.041045+0000","last_peered":"2024-03-21T10:23:56.041045+0000","last_clean":"2024-03-21T10:23:56.041045+0000","last_became_active":"2024-03-21T10:21:58.856176+0000","last_became_peered":"2024-03-21T10:21:58.856176+0000","last_unstale":"2024-03-21T10:23:56.041045+0000","last_undegraded":"2024-03-21T10:23:56.041045+0000","last_fullsized":"2024-03-21T10:23:56.041045+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_clean_scrub_stamp":"2024-03-21T10:21:57.813712+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T21:34:07.804293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"95'9","reported_seq":39,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T10:23:56.027007+0000","last_change":"2024-03-21T10:21:58.846260+0000","last_active":"2024-03-21T10:23:56.027007+0000","last_peered":"2024-03-21T10:23:56.027007+0000","last_clean":"2024-03-21T10:23:56.027007+0000","last_became_active":"2024-03-21T10:21:58.844245+0000","last_became_peered":"2024-03-21T10:21:58.844245+0000","last_unstale":"2024-03-21T10:23:56.027007+0000","last_undegraded":"2024-03-21T10:23:56.027007+0000","last_fullsized":"2024-03-21T10:23:56.027007+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_clean_scrub_stamp":"2024-03-21T10:21:57.813712+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T16:37:07.480867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"95'11","reported_seq":41,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T10:23:56.049788+0000","last_change":"2024-03-21T10:21:58.858413+0000","last_active":"2024-03-21T10:23:56.049788+0000","last_peered":"2024-03-21T10:23:56.049788+0000","last_clean":"2024-03-21T10:23:56.049788+0000","last_became_active":"2024-03-21T10:21:58.858007+0000","last_became_peered":"2024-03-21T10:21:58.858007+0000","last_unstale":"2024-03-21T10:23:56.049788+0000","last_undegraded":"2024-03-21T10:23:56.049788+0000","last_fullsized":"2024-03-21T10:23:56.049788+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_clean_scrub_stamp":"2024-03-21T10:21:57.813712+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T21:49:12.428918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"95'16","reported_seq":46,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T10:23:56.026737+0000","last_change":"2024-03-21T10:21:58.864393+0000","last_active":"2024-03-21T10:23:56.026737+0000","last_peered":"2024-03-21T10:23:56.026737+0000","last_clean":"2024-03-21T10:23:56.026737+0000","last_became_active":"2024-03-21T10:21:58.864061+0000","last_became_peered":"2024-03-21T10:21:58.864061+0000","last_unstale":"2024-03-21T10:23:56.026737+0000","last_undegraded":"2024-03-21T10:23:56.026737+0000","last_fullsized":"2024-03-21T10:23:56.026737+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_clean_scrub_stamp":"2024-03-21T10:21:57.813712+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T20:19:44.209612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"95'6","reported_seq":36,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T10:23:56.048906+0000","last_change":"2024-03-21T10:21:58.873885+0000","last_active":"2024-03-21T10:23:56.048906+0000","last_peered":"2024-03-21T10:23:56.048906+0000","last_clean":"2024-03-21T10:23:56.048906+0000","last_became_active":"2024-03-21T10:21:58.869738+0000","last_became_peered":"2024-03-21T10:21:58.869738+0000","last_unstale":"2024-03-21T10:23:56.048906+0000","last_undegraded":"2024-03-21T10:23:56.048906+0000","last_fullsized":"2024-03-21T10:23:56.048906+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_clean_scrub_stamp":"2024-03-21T10:21:57.813712+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T13:33:26.816720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"95'7","reported_seq":37,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T10:23:56.007359+0000","last_change":"2024-03-21T10:21:58.867563+0000","last_active":"2024-03-21T10:23:56.007359+0000","last_peered":"2024-03-21T10:23:56.007359+0000","last_clean":"2024-03-21T10:23:56.007359+0000","last_became_active":"2024-03-21T10:21:58.863455+0000","last_became_peered":"2024-03-21T10:21:58.863455+0000","last_unstale":"2024-03-21T10:23:56.007359+0000","last_undegraded":"2024-03-21T10:23:56.007359+0000","last_fullsized":"2024-03-21T10:23:56.007359+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_clean_scrub_stamp":"2024-03-21T10:21:57.813712+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T15:04:16.636126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"95'10","reported_seq":46,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T10:23:56.019337+0000","last_change":"2024-03-21T10:22:27.213460+0000","last_active":"2024-03-21T10:23:56.019337+0000","last_peered":"2024-03-21T10:23:56.019337+0000","last_clean":"2024-03-21T10:23:56.019337+0000","last_became_active":"2024-03-21T10:22:27.212961+0000","last_became_peered":"2024-03-21T10:22:27.212961+0000","last_unstale":"2024-03-21T10:23:56.019337+0000","last_undegraded":"2024-03-21T10:23:56.019337+0000","last_fullsized":"2024-03-21T10:23:56.019337+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_clean_scrub_stamp":"2024-03-21T10:21:57.813712+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T18:26:03.881767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,9],"acting":[6,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"95'7","reported_seq":37,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T10:23:56.024681+0000","last_change":"2024-03-21T10:21:58.870851+0000","last_active":"2024-03-21T10:23:56.024681+0000","last_peered":"2024-03-21T10:23:56.024681+0000","last_clean":"2024-03-21T10:23:56.024681+0000","last_became_active":"2024-03-21T10:21:58.870132+0000","last_became_peered":"2024-03-21T10:21:58.870132+0000","last_unstale":"2024-03-21T10:23:56.024681+0000","last_undegraded":"2024-03-21T10:23:56.024681+0000","last_fullsized":"2024-03-21T10:23:56.024681+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_clean_scrub_stamp":"2024-03-21T10:21:57.813712+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T20:20:04.207090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"95'7","reported_seq":37,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T10:23:56.020213+0000","last_change":"2024-03-21T10:21:58.865084+0000","last_active":"2024-03-21T10:23:56.020213+0000","last_peered":"2024-03-21T10:23:56.020213+0000","last_clean":"2024-03-21T10:23:56.020213+0000","last_became_active":"2024-03-21T10:21:58.864583+0000","last_became_peered":"2024-03-21T10:21:58.864583+0000","last_unstale":"2024-03-21T10:23:56.020213+0000","last_undegraded":"2024-03-21T10:23:56.020213+0000","last_fullsized":"2024-03-21T10:23:56.020213+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_clean_scrub_stamp":"2024-03-21T10:21:57.813712+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T14:09:16.781462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"95'9","reported_seq":39,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T10:23:56.048651+0000","last_change":"2024-03-21T10:21:58.870370+0000","last_active":"2024-03-21T10:23:56.048651+0000","last_peered":"2024-03-21T10:23:56.048651+0000","last_clean":"2024-03-21T10:23:56.048651+0000","last_became_active":"2024-03-21T10:21:58.870044+0000","last_became_peered":"2024-03-21T10:21:58.870044+0000","last_unstale":"2024-03-21T10:23:56.048651+0000","last_undegraded":"2024-03-21T10:23:56.048651+0000","last_fullsized":"2024-03-21T10:23:56.048651+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_clean_scrub_stamp":"2024-03-21T10:21:57.813712+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T16:02:00.342426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"95'8","reported_seq":38,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T10:23:56.049052+0000","last_change":"2024-03-21T10:21:58.872787+0000","last_active":"2024-03-21T10:23:56.049052+0000","last_peered":"2024-03-21T10:23:56.049052+0000","last_clean":"2024-03-21T10:23:56.049052+0000","last_became_active":"2024-03-21T10:21:58.872390+0000","last_became_peered":"2024-03-21T10:21:58.872390+0000","last_unstale":"2024-03-21T10:23:56.049052+0000","last_undegraded":"2024-03-21T10:23:56.049052+0000","last_fullsized":"2024-03-21T10:23:56.049052+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_clean_scrub_stamp":"2024-03-21T10:21:57.813712+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T15:56:51.798173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"95'10","reported_seq":40,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T10:23:56.042614+0000","last_change":"2024-03-21T10:21:58.858474+0000","last_active":"2024-03-21T10:23:56.042614+0000","last_peered":"2024-03-21T10:23:56.042614+0000","last_clean":"2024-03-21T10:23:56.042614+0000","last_became_active":"2024-03-21T10:21:58.858076+0000","last_became_peered":"2024-03-21T10:21:58.858076+0000","last_unstale":"2024-03-21T10:23:56.042614+0000","last_undegraded":"2024-03-21T10:23:56.042614+0000","last_fullsized":"2024-03-21T10:23:56.042614+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_clean_scrub_stamp":"2024-03-21T10:21:57.813712+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T12:25:17.160405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"95'7","reported_seq":37,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T10:23:56.027042+0000","last_change":"2024-03-21T10:21:58.854156+0000","last_active":"2024-03-21T10:23:56.027042+0000","last_peered":"2024-03-21T10:23:56.027042+0000","last_clean":"2024-03-21T10:23:56.027042+0000","last_became_active":"2024-03-21T10:21:58.853758+0000","last_became_peered":"2024-03-21T10:21:58.853758+0000","last_unstale":"2024-03-21T10:23:56.027042+0000","last_undegraded":"2024-03-21T10:23:56.027042+0000","last_fullsized":"2024-03-21T10:23:56.027042+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_clean_scrub_stamp":"2024-03-21T10:21:57.813712+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T16:58:54.854826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":30,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T10:23:27.840549+0000","last_change":"2024-03-21T10:23:27.838445+0000","last_active":"2024-03-21T10:23:27.840549+0000","last_peered":"2024-03-21T10:23:27.840549+0000","last_clean":"2024-03-21T10:23:27.840549+0000","last_became_active":"2024-03-21T10:23:27.836992+0000","last_became_peered":"2024-03-21T10:23:27.836992+0000","last_unstale":"2024-03-21T10:23:27.840549+0000","last_undegraded":"2024-03-21T10:23:27.840549+0000","last_fullsized":"2024-03-21T10:23:27.840549+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:22:02.269614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:22:02.269614+0000","last_clean_scrub_stamp":"2024-03-21T10:22:02.269614+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T15:34:51.615332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8,0],"acting":[2,3,8,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"95'14","reported_seq":200,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T10:25:00.942184+0000","last_change":"2024-03-21T10:21:57.859554+0000","last_active":"2024-03-21T10:25:00.942184+0000","last_peered":"2024-03-21T10:25:00.942184+0000","last_clean":"2024-03-21T10:25:00.942184+0000","last_became_active":"2024-03-21T10:21:57.857997+0000","last_became_peered":"2024-03-21T10:21:57.857997+0000","last_unstale":"2024-03-21T10:25:00.942184+0000","last_undegraded":"2024-03-21T10:25:00.942184+0000","last_fullsized":"2024-03-21T10:25:00.942184+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:21:56.788908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:21:56.788908+0000","last_clean_scrub_stamp":"2024-03-21T10:21:56.788908+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T14:27:47.516832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":154,"num_read_kb":154,"num_write":15,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"95'5","reported_seq":35,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T10:23:56.021709+0000","last_change":"2024-03-21T10:21:58.850197+0000","last_active":"2024-03-21T10:23:56.021709+0000","last_peered":"2024-03-21T10:23:56.021709+0000","last_clean":"2024-03-21T10:23:56.021709+0000","last_became_active":"2024-03-21T10:21:58.849029+0000","last_became_peered":"2024-03-21T10:21:58.849029+0000","last_unstale":"2024-03-21T10:23:56.021709+0000","last_undegraded":"2024-03-21T10:23:56.021709+0000","last_fullsized":"2024-03-21T10:23:56.021709+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_clean_scrub_stamp":"2024-03-21T10:21:57.813712+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T15:46:49.077382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":23,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T10:23:27.640067+0000","last_change":"2024-03-21T10:22:03.320716+0000","last_active":"2024-03-21T10:23:27.640067+0000","last_peered":"2024-03-21T10:23:27.640067+0000","last_clean":"2024-03-21T10:23:27.640067+0000","last_became_active":"2024-03-21T10:22:03.320262+0000","last_became_peered":"2024-03-21T10:22:03.320262+0000","last_unstale":"2024-03-21T10:23:27.640067+0000","last_undegraded":"2024-03-21T10:23:27.640067+0000","last_fullsized":"2024-03-21T10:23:27.640067+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:22:02.269614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:22:02.269614+0000","last_clean_scrub_stamp":"2024-03-21T10:22:02.269614+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T16:16:07.423168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"92'2","reported_seq":34,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T10:23:27.639016+0000","last_change":"2024-03-21T10:21:57.863642+0000","last_active":"2024-03-21T10:23:27.639016+0000","last_peered":"2024-03-21T10:23:27.639016+0000","last_clean":"2024-03-21T10:23:27.639016+0000","last_became_active":"2024-03-21T10:21:57.863330+0000","last_became_peered":"2024-03-21T10:21:57.863330+0000","last_unstale":"2024-03-21T10:23:27.639016+0000","last_undegraded":"2024-03-21T10:23:27.639016+0000","last_fullsized":"2024-03-21T10:23:27.639016+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:21:56.788908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:21:56.788908+0000","last_clean_scrub_stamp":"2024-03-21T10:21:56.788908+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T14:45:02.131458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"95'6","reported_seq":36,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T10:23:56.048702+0000","last_change":"2024-03-21T10:21:58.862940+0000","last_active":"2024-03-21T10:23:56.048702+0000","last_peered":"2024-03-21T10:23:56.048702+0000","last_clean":"2024-03-21T10:23:56.048702+0000","last_became_active":"2024-03-21T10:21:58.862489+0000","last_became_peered":"2024-03-21T10:21:58.862489+0000","last_unstale":"2024-03-21T10:23:56.048702+0000","last_undegraded":"2024-03-21T10:23:56.048702+0000","last_fullsized":"2024-03-21T10:23:56.048702+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_clean_scrub_stamp":"2024-03-21T10:21:57.813712+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T19:28:02.175642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":23,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T10:23:27.640509+0000","last_change":"2024-03-21T10:22:03.318660+0000","last_active":"2024-03-21T10:23:27.640509+0000","last_peered":"2024-03-21T10:23:27.640509+0000","last_clean":"2024-03-21T10:23:27.640509+0000","last_became_active":"2024-03-21T10:22:03.318216+0000","last_became_peered":"2024-03-21T10:22:03.318216+0000","last_unstale":"2024-03-21T10:23:27.640509+0000","last_undegraded":"2024-03-21T10:23:27.640509+0000","last_fullsized":"2024-03-21T10:23:27.640509+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:22:02.269614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:22:02.269614+0000","last_clean_scrub_stamp":"2024-03-21T10:22:02.269614+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T16:36:39.930726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"93'12","reported_seq":31,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T10:23:27.837288+0000","last_change":"2024-03-21T10:22:27.239342+0000","last_active":"2024-03-21T10:23:27.837288+0000","last_peered":"2024-03-21T10:23:27.837288+0000","last_clean":"2024-03-21T10:23:27.837288+0000","last_became_active":"2024-03-21T10:22:27.224086+0000","last_became_peered":"2024-03-21T10:22:27.224086+0000","last_unstale":"2024-03-21T10:23:27.837288+0000","last_undegraded":"2024-03-21T10:23:27.837288+0000","last_fullsized":"2024-03-21T10:23:27.837288+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:21:56.788908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:21:56.788908+0000","last_clean_scrub_stamp":"2024-03-21T10:21:56.788908+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T20:46:45.951485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,7],"acting":[11,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1c","version":"95'9","reported_seq":39,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T10:23:56.044535+0000","last_change":"2024-03-21T10:21:58.873239+0000","last_active":"2024-03-21T10:23:56.044535+0000","last_peered":"2024-03-21T10:23:56.044535+0000","last_clean":"2024-03-21T10:23:56.044535+0000","last_became_active":"2024-03-21T10:21:58.872877+0000","last_became_peered":"2024-03-21T10:21:58.872877+0000","last_unstale":"2024-03-21T10:23:56.044535+0000","last_undegraded":"2024-03-21T10:23:56.044535+0000","last_fullsized":"2024-03-21T10:23:56.044535+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_clean_scrub_stamp":"2024-03-21T10:21:57.813712+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T16:22:49.070697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":23,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T10:23:27.640201+0000","last_change":"2024-03-21T10:22:03.311589+0000","last_active":"2024-03-21T10:23:27.640201+0000","last_peered":"2024-03-21T10:23:27.640201+0000","last_clean":"2024-03-21T10:23:27.640201+0000","last_became_active":"2024-03-21T10:22:03.311134+0000","last_became_peered":"2024-03-21T10:22:03.311134+0000","last_unstale":"2024-03-21T10:23:27.640201+0000","last_undegraded":"2024-03-21T10:23:27.640201+0000","last_fullsized":"2024-03-21T10:23:27.640201+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:22:02.269614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:22:02.269614+0000","last_clean_scrub_stamp":"2024-03-21T10:22:02.269614+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T10:22:29.138348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"93'5","reported_seq":38,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T10:23:27.646290+0000","last_change":"2024-03-21T10:21:57.858981+0000","last_active":"2024-03-21T10:23:27.646290+0000","last_peered":"2024-03-21T10:23:27.646290+0000","last_clean":"2024-03-21T10:23:27.646290+0000","last_became_active":"2024-03-21T10:21:57.858564+0000","last_became_peered":"2024-03-21T10:21:57.858564+0000","last_unstale":"2024-03-21T10:23:27.646290+0000","last_undegraded":"2024-03-21T10:23:27.646290+0000","last_fullsized":"2024-03-21T10:23:27.646290+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:21:56.788908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:21:56.788908+0000","last_clean_scrub_stamp":"2024-03-21T10:21:56.788908+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T10:45:09.488194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"95'15","reported_seq":45,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T10:23:56.039689+0000","last_change":"2024-03-21T10:21:58.858053+0000","last_active":"2024-03-21T10:23:56.039689+0000","last_peered":"2024-03-21T10:23:56.039689+0000","last_clean":"2024-03-21T10:23:56.039689+0000","last_became_active":"2024-03-21T10:21:58.857688+0000","last_became_peered":"2024-03-21T10:21:58.857688+0000","last_unstale":"2024-03-21T10:23:56.039689+0000","last_undegraded":"2024-03-21T10:23:56.039689+0000","last_fullsized":"2024-03-21T10:23:56.039689+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_clean_scrub_stamp":"2024-03-21T10:21:57.813712+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T19:45:13.043034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":30,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T10:23:27.843397+0000","last_change":"2024-03-21T10:23:27.842200+0000","last_active":"2024-03-21T10:23:27.843397+0000","last_peered":"2024-03-21T10:23:27.843397+0000","last_clean":"2024-03-21T10:23:27.843397+0000","last_became_active":"2024-03-21T10:23:27.840749+0000","last_became_peered":"2024-03-21T10:23:27.840749+0000","last_unstale":"2024-03-21T10:23:27.843397+0000","last_undegraded":"2024-03-21T10:23:27.843397+0000","last_fullsized":"2024-03-21T10:23:27.843397+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:22:02.269614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:22:02.269614+0000","last_clean_scrub_stamp":"2024-03-21T10:22:02.269614+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T22:01:31.714771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,7,11],"acting":[2,8,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"95'6","reported_seq":207,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T10:25:00.996435+0000","last_change":"2024-03-21T10:22:27.235744+0000","last_active":"2024-03-21T10:25:00.996435+0000","last_peered":"2024-03-21T10:25:00.996435+0000","last_clean":"2024-03-21T10:25:00.996435+0000","last_became_active":"2024-03-21T10:22:27.219999+0000","last_became_peered":"2024-03-21T10:22:27.219999+0000","last_unstale":"2024-03-21T10:25:00.996435+0000","last_undegraded":"2024-03-21T10:25:00.996435+0000","last_fullsized":"2024-03-21T10:25:00.996435+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:21:56.788908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:21:56.788908+0000","last_clean_scrub_stamp":"2024-03-21T10:21:56.788908+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T18:15:41.288034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1115,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":157,"num_read_kb":3,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":622,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7],"acting":[5,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"95'7","reported_seq":37,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T10:23:55.996971+0000","last_change":"2024-03-21T10:21:58.854752+0000","last_active":"2024-03-21T10:23:55.996971+0000","last_peered":"2024-03-21T10:23:55.996971+0000","last_clean":"2024-03-21T10:23:55.996971+0000","last_became_active":"2024-03-21T10:21:58.854442+0000","last_became_peered":"2024-03-21T10:21:58.854442+0000","last_unstale":"2024-03-21T10:23:55.996971+0000","last_undegraded":"2024-03-21T10:23:55.996971+0000","last_fullsized":"2024-03-21T10:23:55.996971+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_clean_scrub_stamp":"2024-03-21T10:21:57.813712+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T15:12:40.590585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":23,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T10:23:27.648343+0000","last_change":"2024-03-21T10:22:03.319247+0000","last_active":"2024-03-21T10:23:27.648343+0000","last_peered":"2024-03-21T10:23:27.648343+0000","last_clean":"2024-03-21T10:23:27.648343+0000","last_became_active":"2024-03-21T10:22:03.318731+0000","last_became_peered":"2024-03-21T10:22:03.318731+0000","last_unstale":"2024-03-21T10:23:27.648343+0000","last_undegraded":"2024-03-21T10:23:27.648343+0000","last_fullsized":"2024-03-21T10:23:27.648343+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:22:02.269614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:22:02.269614+0000","last_clean_scrub_stamp":"2024-03-21T10:22:02.269614+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T20:06:36.519559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"93'7","reported_seq":45,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T10:23:27.744422+0000","last_change":"2024-03-21T10:22:27.222981+0000","last_active":"2024-03-21T10:23:27.744422+0000","last_peered":"2024-03-21T10:23:27.744422+0000","last_clean":"2024-03-21T10:23:27.744422+0000","last_became_active":"2024-03-21T10:22:27.221088+0000","last_became_peered":"2024-03-21T10:22:27.221088+0000","last_unstale":"2024-03-21T10:23:27.744422+0000","last_undegraded":"2024-03-21T10:23:27.744422+0000","last_fullsized":"2024-03-21T10:23:27.744422+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:21:56.788908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:21:56.788908+0000","last_clean_scrub_stamp":"2024-03-21T10:21:56.788908+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T15:43:11.740353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,6],"acting":[10,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"95'8","reported_seq":44,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T10:23:56.027376+0000","last_change":"2024-03-21T10:22:27.226733+0000","last_active":"2024-03-21T10:23:56.027376+0000","last_peered":"2024-03-21T10:23:56.027376+0000","last_clean":"2024-03-21T10:23:56.027376+0000","last_became_active":"2024-03-21T10:22:27.225576+0000","last_became_peered":"2024-03-21T10:22:27.225576+0000","last_unstale":"2024-03-21T10:23:56.027376+0000","last_undegraded":"2024-03-21T10:23:56.027376+0000","last_fullsized":"2024-03-21T10:23:56.027376+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_clean_scrub_stamp":"2024-03-21T10:21:57.813712+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T20:06:09.534268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,3],"acting":[5,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":23,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T10:23:27.836282+0000","last_change":"2024-03-21T10:22:03.321595+0000","last_active":"2024-03-21T10:23:27.836282+0000","last_peered":"2024-03-21T10:23:27.836282+0000","last_clean":"2024-03-21T10:23:27.836282+0000","last_became_active":"2024-03-21T10:22:03.320897+0000","last_became_peered":"2024-03-21T10:22:03.320897+0000","last_unstale":"2024-03-21T10:23:27.836282+0000","last_undegraded":"2024-03-21T10:23:27.836282+0000","last_fullsized":"2024-03-21T10:23:27.836282+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:22:02.269614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:22:02.269614+0000","last_clean_scrub_stamp":"2024-03-21T10:22:02.269614+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T17:37:02.633709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"93'3","reported_seq":35,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T10:23:27.742215+0000","last_change":"2024-03-21T10:21:57.854801+0000","last_active":"2024-03-21T10:23:27.742215+0000","last_peered":"2024-03-21T10:23:27.742215+0000","last_clean":"2024-03-21T10:23:27.742215+0000","last_became_active":"2024-03-21T10:21:57.854443+0000","last_became_peered":"2024-03-21T10:21:57.854443+0000","last_unstale":"2024-03-21T10:23:27.742215+0000","last_undegraded":"2024-03-21T10:23:27.742215+0000","last_fullsized":"2024-03-21T10:23:27.742215+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:21:56.788908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:21:56.788908+0000","last_clean_scrub_stamp":"2024-03-21T10:21:56.788908+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T15:49:23.573360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"95'8","reported_seq":38,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T10:23:56.026817+0000","last_change":"2024-03-21T10:21:58.857167+0000","last_active":"2024-03-21T10:23:56.026817+0000","last_peered":"2024-03-21T10:23:56.026817+0000","last_clean":"2024-03-21T10:23:56.026817+0000","last_became_active":"2024-03-21T10:21:58.856827+0000","last_became_peered":"2024-03-21T10:21:58.856827+0000","last_unstale":"2024-03-21T10:23:56.026817+0000","last_undegraded":"2024-03-21T10:23:56.026817+0000","last_fullsized":"2024-03-21T10:23:56.026817+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_clean_scrub_stamp":"2024-03-21T10:21:57.813712+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T15:04:43.848209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":23,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T10:23:27.834956+0000","last_change":"2024-03-21T10:22:03.306554+0000","last_active":"2024-03-21T10:23:27.834956+0000","last_peered":"2024-03-21T10:23:27.834956+0000","last_clean":"2024-03-21T10:23:27.834956+0000","last_became_active":"2024-03-21T10:22:03.306087+0000","last_became_peered":"2024-03-21T10:22:03.306087+0000","last_unstale":"2024-03-21T10:23:27.834956+0000","last_undegraded":"2024-03-21T10:23:27.834956+0000","last_fullsized":"2024-03-21T10:23:27.834956+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:22:02.269614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:22:02.269614+0000","last_clean_scrub_stamp":"2024-03-21T10:22:02.269614+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T11:36:35.344770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"93'3","reported_seq":36,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T10:23:27.751080+0000","last_change":"2024-03-21T10:21:57.860343+0000","last_active":"2024-03-21T10:23:27.751080+0000","last_peered":"2024-03-21T10:23:27.751080+0000","last_clean":"2024-03-21T10:23:27.751080+0000","last_became_active":"2024-03-21T10:21:57.859808+0000","last_became_peered":"2024-03-21T10:21:57.859808+0000","last_unstale":"2024-03-21T10:23:27.751080+0000","last_undegraded":"2024-03-21T10:23:27.751080+0000","last_fullsized":"2024-03-21T10:23:27.751080+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:21:56.788908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:21:56.788908+0000","last_clean_scrub_stamp":"2024-03-21T10:21:56.788908+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T14:41:24.162795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"95'11","reported_seq":41,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T10:23:56.023593+0000","last_change":"2024-03-21T10:21:58.850991+0000","last_active":"2024-03-21T10:23:56.023593+0000","last_peered":"2024-03-21T10:23:56.023593+0000","last_clean":"2024-03-21T10:23:56.023593+0000","last_became_active":"2024-03-21T10:21:58.850581+0000","last_became_peered":"2024-03-21T10:21:58.850581+0000","last_unstale":"2024-03-21T10:23:56.023593+0000","last_undegraded":"2024-03-21T10:23:56.023593+0000","last_fullsized":"2024-03-21T10:23:56.023593+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_clean_scrub_stamp":"2024-03-21T10:21:57.813712+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T17:07:11.350806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":23,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T10:23:27.638522+0000","last_change":"2024-03-21T10:22:03.315333+0000","last_active":"2024-03-21T10:23:27.638522+0000","last_peered":"2024-03-21T10:23:27.638522+0000","last_clean":"2024-03-21T10:23:27.638522+0000","last_became_active":"2024-03-21T10:22:03.313553+0000","last_became_peered":"2024-03-21T10:22:03.313553+0000","last_unstale":"2024-03-21T10:23:27.638522+0000","last_undegraded":"2024-03-21T10:23:27.638522+0000","last_fullsized":"2024-03-21T10:23:27.638522+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:22:02.269614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:22:02.269614+0000","last_clean_scrub_stamp":"2024-03-21T10:22:02.269614+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T15:25:24.290789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"95'6","reported_seq":348,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T10:25:03.000542+0000","last_change":"2024-03-21T10:21:57.864930+0000","last_active":"2024-03-21T10:25:03.000542+0000","last_peered":"2024-03-21T10:25:03.000542+0000","last_clean":"2024-03-21T10:25:03.000542+0000","last_became_active":"2024-03-21T10:21:57.863769+0000","last_became_peered":"2024-03-21T10:21:57.863769+0000","last_unstale":"2024-03-21T10:25:03.000542+0000","last_undegraded":"2024-03-21T10:25:03.000542+0000","last_fullsized":"2024-03-21T10:25:03.000542+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:21:56.788908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:21:56.788908+0000","last_clean_scrub_stamp":"2024-03-21T10:21:56.788908+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T21:35:39.144065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":310,"num_read_kb":310,"num_write":7,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"95'10","reported_seq":40,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T10:23:56.047740+0000","last_change":"2024-03-21T10:21:58.872084+0000","last_active":"2024-03-21T10:23:56.047740+0000","last_peered":"2024-03-21T10:23:56.047740+0000","last_clean":"2024-03-21T10:23:56.047740+0000","last_became_active":"2024-03-21T10:21:58.871725+0000","last_became_peered":"2024-03-21T10:21:58.871725+0000","last_unstale":"2024-03-21T10:23:56.047740+0000","last_undegraded":"2024-03-21T10:23:56.047740+0000","last_fullsized":"2024-03-21T10:23:56.047740+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_clean_scrub_stamp":"2024-03-21T10:21:57.813712+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T16:19:37.240053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":23,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T10:23:27.641039+0000","last_change":"2024-03-21T10:22:03.317808+0000","last_active":"2024-03-21T10:23:27.641039+0000","last_peered":"2024-03-21T10:23:27.641039+0000","last_clean":"2024-03-21T10:23:27.641039+0000","last_became_active":"2024-03-21T10:22:03.313751+0000","last_became_peered":"2024-03-21T10:22:03.313751+0000","last_unstale":"2024-03-21T10:23:27.641039+0000","last_undegraded":"2024-03-21T10:23:27.641039+0000","last_fullsized":"2024-03-21T10:23:27.641039+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:22:02.269614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:22:02.269614+0000","last_clean_scrub_stamp":"2024-03-21T10:22:02.269614+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T20:05:08.105795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"93'4","reported_seq":36,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T10:23:27.646634+0000","last_change":"2024-03-21T10:21:57.865462+0000","last_active":"2024-03-21T10:23:27.646634+0000","last_peered":"2024-03-21T10:23:27.646634+0000","last_clean":"2024-03-21T10:23:27.646634+0000","last_became_active":"2024-03-21T10:21:57.862922+0000","last_became_peered":"2024-03-21T10:21:57.862922+0000","last_unstale":"2024-03-21T10:23:27.646634+0000","last_undegraded":"2024-03-21T10:23:27.646634+0000","last_fullsized":"2024-03-21T10:23:27.646634+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:21:56.788908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:21:56.788908+0000","last_clean_scrub_stamp":"2024-03-21T10:21:56.788908+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T15:31:17.167806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"95'9","reported_seq":39,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T10:23:56.022903+0000","last_change":"2024-03-21T10:21:58.865459+0000","last_active":"2024-03-21T10:23:56.022903+0000","last_peered":"2024-03-21T10:23:56.022903+0000","last_clean":"2024-03-21T10:23:56.022903+0000","last_became_active":"2024-03-21T10:21:58.860909+0000","last_became_peered":"2024-03-21T10:21:58.860909+0000","last_unstale":"2024-03-21T10:23:56.022903+0000","last_undegraded":"2024-03-21T10:23:56.022903+0000","last_fullsized":"2024-03-21T10:23:56.022903+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_clean_scrub_stamp":"2024-03-21T10:21:57.813712+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T15:53:35.110404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":23,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T10:23:27.640633+0000","last_change":"2024-03-21T10:22:03.318651+0000","last_active":"2024-03-21T10:23:27.640633+0000","last_peered":"2024-03-21T10:23:27.640633+0000","last_clean":"2024-03-21T10:23:27.640633+0000","last_became_active":"2024-03-21T10:22:03.318108+0000","last_became_peered":"2024-03-21T10:22:03.318108+0000","last_unstale":"2024-03-21T10:23:27.640633+0000","last_undegraded":"2024-03-21T10:23:27.640633+0000","last_fullsized":"2024-03-21T10:23:27.640633+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:22:02.269614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:22:02.269614+0000","last_clean_scrub_stamp":"2024-03-21T10:22:02.269614+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T19:00:04.216749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"93'5","reported_seq":38,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T10:23:27.751795+0000","last_change":"2024-03-21T10:21:57.865516+0000","last_active":"2024-03-21T10:23:27.751795+0000","last_peered":"2024-03-21T10:23:27.751795+0000","last_clean":"2024-03-21T10:23:27.751795+0000","last_became_active":"2024-03-21T10:21:57.865012+0000","last_became_peered":"2024-03-21T10:21:57.865012+0000","last_unstale":"2024-03-21T10:23:27.751795+0000","last_undegraded":"2024-03-21T10:23:27.751795+0000","last_fullsized":"2024-03-21T10:23:27.751795+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:21:56.788908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:21:56.788908+0000","last_clean_scrub_stamp":"2024-03-21T10:21:56.788908+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T21:38:02.813453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"95'11","reported_seq":41,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T10:23:56.048118+0000","last_change":"2024-03-21T10:21:58.870876+0000","last_active":"2024-03-21T10:23:56.048118+0000","last_peered":"2024-03-21T10:23:56.048118+0000","last_clean":"2024-03-21T10:23:56.048118+0000","last_became_active":"2024-03-21T10:21:58.868240+0000","last_became_peered":"2024-03-21T10:21:58.868240+0000","last_unstale":"2024-03-21T10:23:56.048118+0000","last_undegraded":"2024-03-21T10:23:56.048118+0000","last_fullsized":"2024-03-21T10:23:56.048118+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_clean_scrub_stamp":"2024-03-21T10:21:57.813712+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T15:16:59.496020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":23,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T10:23:27.641221+0000","last_change":"2024-03-21T10:22:03.314854+0000","last_active":"2024-03-21T10:23:27.641221+0000","last_peered":"2024-03-21T10:23:27.641221+0000","last_clean":"2024-03-21T10:23:27.641221+0000","last_became_active":"2024-03-21T10:22:03.314422+0000","last_became_peered":"2024-03-21T10:22:03.314422+0000","last_unstale":"2024-03-21T10:23:27.641221+0000","last_undegraded":"2024-03-21T10:23:27.641221+0000","last_fullsized":"2024-03-21T10:23:27.641221+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:22:02.269614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:22:02.269614+0000","last_clean_scrub_stamp":"2024-03-21T10:22:02.269614+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T12:34:31.519118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.14","version":"95'38","reported_seq":407,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T10:25:02.999025+0000","last_change":"2024-03-21T10:22:27.243817+0000","last_active":"2024-03-21T10:25:02.999025+0000","last_peered":"2024-03-21T10:25:02.999025+0000","last_clean":"2024-03-21T10:25:02.999025+0000","last_became_active":"2024-03-21T10:22:27.218873+0000","last_became_peered":"2024-03-21T10:22:27.218873+0000","last_unstale":"2024-03-21T10:25:02.999025+0000","last_undegraded":"2024-03-21T10:25:02.999025+0000","last_fullsized":"2024-03-21T10:25:02.999025+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:21:56.788908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:21:56.788908+0000","last_clean_scrub_stamp":"2024-03-21T10:21:56.788908+0000","objects_scrubbed":0,"log_size":38,"log_dups_size":0,"ondisk_log_size":38,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T12:27:43.574451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":64848,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":319,"num_read_kb":223,"num_write":40,"num_write_kb":82,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":3842,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,4],"acting":[10,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"95'6","reported_seq":36,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T10:23:56.011976+0000","last_change":"2024-03-21T10:21:58.860386+0000","last_active":"2024-03-21T10:23:56.011976+0000","last_peered":"2024-03-21T10:23:56.011976+0000","last_clean":"2024-03-21T10:23:56.011976+0000","last_became_active":"2024-03-21T10:21:58.860013+0000","last_became_peered":"2024-03-21T10:21:58.860013+0000","last_unstale":"2024-03-21T10:23:56.011976+0000","last_undegraded":"2024-03-21T10:23:56.011976+0000","last_fullsized":"2024-03-21T10:23:56.011976+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_clean_scrub_stamp":"2024-03-21T10:21:57.813712+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T18:02:33.920374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":23,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T10:23:27.752685+0000","last_change":"2024-03-21T10:22:03.314083+0000","last_active":"2024-03-21T10:23:27.752685+0000","last_peered":"2024-03-21T10:23:27.752685+0000","last_clean":"2024-03-21T10:23:27.752685+0000","last_became_active":"2024-03-21T10:22:03.313652+0000","last_became_peered":"2024-03-21T10:22:03.313652+0000","last_unstale":"2024-03-21T10:23:27.752685+0000","last_undegraded":"2024-03-21T10:23:27.752685+0000","last_fullsized":"2024-03-21T10:23:27.752685+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:22:02.269614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:22:02.269614+0000","last_clean_scrub_stamp":"2024-03-21T10:22:02.269614+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T13:23:51.256535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"93'9","reported_seq":43,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T10:23:27.648194+0000","last_change":"2024-03-21T10:21:57.866716+0000","last_active":"2024-03-21T10:23:27.648194+0000","last_peered":"2024-03-21T10:23:27.648194+0000","last_clean":"2024-03-21T10:23:27.648194+0000","last_became_active":"2024-03-21T10:21:57.860900+0000","last_became_peered":"2024-03-21T10:21:57.860900+0000","last_unstale":"2024-03-21T10:23:27.648194+0000","last_undegraded":"2024-03-21T10:23:27.648194+0000","last_fullsized":"2024-03-21T10:23:27.648194+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:21:56.788908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:21:56.788908+0000","last_clean_scrub_stamp":"2024-03-21T10:21:56.788908+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T13:13:04.937120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"95'9","reported_seq":39,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T10:23:56.016553+0000","last_change":"2024-03-21T10:21:58.858815+0000","last_active":"2024-03-21T10:23:56.016553+0000","last_peered":"2024-03-21T10:23:56.016553+0000","last_clean":"2024-03-21T10:23:56.016553+0000","last_became_active":"2024-03-21T10:21:58.856483+0000","last_became_peered":"2024-03-21T10:21:58.856483+0000","last_unstale":"2024-03-21T10:23:56.016553+0000","last_undegraded":"2024-03-21T10:23:56.016553+0000","last_fullsized":"2024-03-21T10:23:56.016553+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_clean_scrub_stamp":"2024-03-21T10:21:57.813712+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T15:42:38.208093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":23,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T10:23:27.836558+0000","last_change":"2024-03-21T10:22:03.319719+0000","last_active":"2024-03-21T10:23:27.836558+0000","last_peered":"2024-03-21T10:23:27.836558+0000","last_clean":"2024-03-21T10:23:27.836558+0000","last_became_active":"2024-03-21T10:22:03.319234+0000","last_became_peered":"2024-03-21T10:22:03.319234+0000","last_unstale":"2024-03-21T10:23:27.836558+0000","last_undegraded":"2024-03-21T10:23:27.836558+0000","last_fullsized":"2024-03-21T10:23:27.836558+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:22:02.269614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:22:02.269614+0000","last_clean_scrub_stamp":"2024-03-21T10:22:02.269614+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T14:11:21.609594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"93'5","reported_seq":38,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T10:23:27.751203+0000","last_change":"2024-03-21T10:21:57.862744+0000","last_active":"2024-03-21T10:23:27.751203+0000","last_peered":"2024-03-21T10:23:27.751203+0000","last_clean":"2024-03-21T10:23:27.751203+0000","last_became_active":"2024-03-21T10:21:57.862352+0000","last_became_peered":"2024-03-21T10:21:57.862352+0000","last_unstale":"2024-03-21T10:23:27.751203+0000","last_undegraded":"2024-03-21T10:23:27.751203+0000","last_fullsized":"2024-03-21T10:23:27.751203+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:21:56.788908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:21:56.788908+0000","last_clean_scrub_stamp":"2024-03-21T10:21:56.788908+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T20:19:50.138904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"95'15","reported_seq":45,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T10:23:56.046387+0000","last_change":"2024-03-21T10:21:58.863514+0000","last_active":"2024-03-21T10:23:56.046387+0000","last_peered":"2024-03-21T10:23:56.046387+0000","last_clean":"2024-03-21T10:23:56.046387+0000","last_became_active":"2024-03-21T10:21:58.861981+0000","last_became_peered":"2024-03-21T10:21:58.861981+0000","last_unstale":"2024-03-21T10:23:56.046387+0000","last_undegraded":"2024-03-21T10:23:56.046387+0000","last_fullsized":"2024-03-21T10:23:56.046387+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_clean_scrub_stamp":"2024-03-21T10:21:57.813712+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T15:00:33.813456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":23,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T10:23:27.642326+0000","last_change":"2024-03-21T10:22:03.309844+0000","last_active":"2024-03-21T10:23:27.642326+0000","last_peered":"2024-03-21T10:23:27.642326+0000","last_clean":"2024-03-21T10:23:27.642326+0000","last_became_active":"2024-03-21T10:22:03.309448+0000","last_became_peered":"2024-03-21T10:22:03.309448+0000","last_unstale":"2024-03-21T10:23:27.642326+0000","last_undegraded":"2024-03-21T10:23:27.642326+0000","last_fullsized":"2024-03-21T10:23:27.642326+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:22:02.269614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:22:02.269614+0000","last_clean_scrub_stamp":"2024-03-21T10:22:02.269614+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T18:38:51.167166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.11","version":"95'8","reported_seq":201,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T10:24:57.986043+0000","last_change":"2024-03-21T10:22:27.223337+0000","last_active":"2024-03-21T10:24:57.986043+0000","last_peered":"2024-03-21T10:24:57.986043+0000","last_clean":"2024-03-21T10:24:57.986043+0000","last_became_active":"2024-03-21T10:22:27.222999+0000","last_became_peered":"2024-03-21T10:22:27.222999+0000","last_unstale":"2024-03-21T10:24:57.986043+0000","last_undegraded":"2024-03-21T10:24:57.986043+0000","last_fullsized":"2024-03-21T10:24:57.986043+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:21:56.788908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:21:56.788908+0000","last_clean_scrub_stamp":"2024-03-21T10:21:56.788908+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T20:40:45.780696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1613,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":156,"num_read_kb":5,"num_write":8,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,10],"acting":[4,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"95'6","reported_seq":36,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T10:23:56.037159+0000","last_change":"2024-03-21T10:21:58.862225+0000","last_active":"2024-03-21T10:23:56.037159+0000","last_peered":"2024-03-21T10:23:56.037159+0000","last_clean":"2024-03-21T10:23:56.037159+0000","last_became_active":"2024-03-21T10:21:58.861816+0000","last_became_peered":"2024-03-21T10:21:58.861816+0000","last_unstale":"2024-03-21T10:23:56.037159+0000","last_undegraded":"2024-03-21T10:23:56.037159+0000","last_fullsized":"2024-03-21T10:23:56.037159+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_clean_scrub_stamp":"2024-03-21T10:21:57.813712+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T18:36:13.420867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":23,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T10:23:27.742956+0000","last_change":"2024-03-21T10:22:03.315306+0000","last_active":"2024-03-21T10:23:27.742956+0000","last_peered":"2024-03-21T10:23:27.742956+0000","last_clean":"2024-03-21T10:23:27.742956+0000","last_became_active":"2024-03-21T10:22:03.314839+0000","last_became_peered":"2024-03-21T10:22:03.314839+0000","last_unstale":"2024-03-21T10:23:27.742956+0000","last_undegraded":"2024-03-21T10:23:27.742956+0000","last_fullsized":"2024-03-21T10:23:27.742956+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:22:02.269614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:22:02.269614+0000","last_clean_scrub_stamp":"2024-03-21T10:22:02.269614+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T15:47:45.690434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"93'17","reported_seq":50,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T10:23:27.644536+0000","last_change":"2024-03-21T10:21:57.870998+0000","last_active":"2024-03-21T10:23:27.644536+0000","last_peered":"2024-03-21T10:23:27.644536+0000","last_clean":"2024-03-21T10:23:27.644536+0000","last_became_active":"2024-03-21T10:21:57.870605+0000","last_became_peered":"2024-03-21T10:21:57.870605+0000","last_unstale":"2024-03-21T10:23:27.644536+0000","last_undegraded":"2024-03-21T10:23:27.644536+0000","last_fullsized":"2024-03-21T10:23:27.644536+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:21:56.788908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:21:56.788908+0000","last_clean_scrub_stamp":"2024-03-21T10:21:56.788908+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T13:33:54.775122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"95'6","reported_seq":36,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T10:23:56.030441+0000","last_change":"2024-03-21T10:21:58.858675+0000","last_active":"2024-03-21T10:23:56.030441+0000","last_peered":"2024-03-21T10:23:56.030441+0000","last_clean":"2024-03-21T10:23:56.030441+0000","last_became_active":"2024-03-21T10:21:58.856907+0000","last_became_peered":"2024-03-21T10:21:58.856907+0000","last_unstale":"2024-03-21T10:23:56.030441+0000","last_undegraded":"2024-03-21T10:23:56.030441+0000","last_fullsized":"2024-03-21T10:23:56.030441+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_clean_scrub_stamp":"2024-03-21T10:21:57.813712+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T18:17:31.208728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":23,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T10:23:27.750746+0000","last_change":"2024-03-21T10:22:03.318257+0000","last_active":"2024-03-21T10:23:27.750746+0000","last_peered":"2024-03-21T10:23:27.750746+0000","last_clean":"2024-03-21T10:23:27.750746+0000","last_became_active":"2024-03-21T10:22:03.317622+0000","last_became_peered":"2024-03-21T10:22:03.317622+0000","last_unstale":"2024-03-21T10:23:27.750746+0000","last_undegraded":"2024-03-21T10:23:27.750746+0000","last_fullsized":"2024-03-21T10:23:27.750746+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:22:02.269614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:22:02.269614+0000","last_clean_scrub_stamp":"2024-03-21T10:22:02.269614+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T17:15:50.272193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"95'3","reported_seq":36,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T10:24:12.259722+0000","last_change":"2024-03-21T10:21:57.868014+0000","last_active":"2024-03-21T10:24:12.259722+0000","last_peered":"2024-03-21T10:24:12.259722+0000","last_clean":"2024-03-21T10:24:12.259722+0000","last_became_active":"2024-03-21T10:21:57.867189+0000","last_became_peered":"2024-03-21T10:21:57.867189+0000","last_unstale":"2024-03-21T10:24:12.259722+0000","last_undegraded":"2024-03-21T10:24:12.259722+0000","last_fullsized":"2024-03-21T10:24:12.259722+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:21:56.788908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:21:56.788908+0000","last_clean_scrub_stamp":"2024-03-21T10:21:56.788908+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T22:02:48.704932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"95'5","reported_seq":35,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T10:23:56.004037+0000","last_change":"2024-03-21T10:21:58.862205+0000","last_active":"2024-03-21T10:23:56.004037+0000","last_peered":"2024-03-21T10:23:56.004037+0000","last_clean":"2024-03-21T10:23:56.004037+0000","last_became_active":"2024-03-21T10:21:58.861186+0000","last_became_peered":"2024-03-21T10:21:58.861186+0000","last_unstale":"2024-03-21T10:23:56.004037+0000","last_undegraded":"2024-03-21T10:23:56.004037+0000","last_fullsized":"2024-03-21T10:23:56.004037+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_clean_scrub_stamp":"2024-03-21T10:21:57.813712+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T19:44:49.719845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":23,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T10:23:27.643556+0000","last_change":"2024-03-21T10:22:03.636064+0000","last_active":"2024-03-21T10:23:27.643556+0000","last_peered":"2024-03-21T10:23:27.643556+0000","last_clean":"2024-03-21T10:23:27.643556+0000","last_became_active":"2024-03-21T10:22:03.633596+0000","last_became_peered":"2024-03-21T10:22:03.633596+0000","last_unstale":"2024-03-21T10:23:27.643556+0000","last_undegraded":"2024-03-21T10:23:27.643556+0000","last_fullsized":"2024-03-21T10:23:27.643556+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:22:02.269614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:22:02.269614+0000","last_clean_scrub_stamp":"2024-03-21T10:22:02.269614+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T19:32:31.844066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"93'1","reported_seq":33,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T10:23:27.642192+0000","last_change":"2024-03-21T10:21:57.851955+0000","last_active":"2024-03-21T10:23:27.642192+0000","last_peered":"2024-03-21T10:23:27.642192+0000","last_clean":"2024-03-21T10:23:27.642192+0000","last_became_active":"2024-03-21T10:21:57.851571+0000","last_became_peered":"2024-03-21T10:21:57.851571+0000","last_unstale":"2024-03-21T10:23:27.642192+0000","last_undegraded":"2024-03-21T10:23:27.642192+0000","last_fullsized":"2024-03-21T10:23:27.642192+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:21:56.788908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:21:56.788908+0000","last_clean_scrub_stamp":"2024-03-21T10:21:56.788908+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T17:23:40.441895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"95'4","reported_seq":34,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T10:23:56.015141+0000","last_change":"2024-03-21T10:21:58.856393+0000","last_active":"2024-03-21T10:23:56.015141+0000","last_peered":"2024-03-21T10:23:56.015141+0000","last_clean":"2024-03-21T10:23:56.015141+0000","last_became_active":"2024-03-21T10:21:58.855139+0000","last_became_peered":"2024-03-21T10:21:58.855139+0000","last_unstale":"2024-03-21T10:23:56.015141+0000","last_undegraded":"2024-03-21T10:23:56.015141+0000","last_fullsized":"2024-03-21T10:23:56.015141+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_clean_scrub_stamp":"2024-03-21T10:21:57.813712+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T14:24:55.663576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":23,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T10:23:27.742416+0000","last_change":"2024-03-21T10:22:03.315586+0000","last_active":"2024-03-21T10:23:27.742416+0000","last_peered":"2024-03-21T10:23:27.742416+0000","last_clean":"2024-03-21T10:23:27.742416+0000","last_became_active":"2024-03-21T10:22:03.315198+0000","last_became_peered":"2024-03-21T10:22:03.315198+0000","last_unstale":"2024-03-21T10:23:27.742416+0000","last_undegraded":"2024-03-21T10:23:27.742416+0000","last_fullsized":"2024-03-21T10:23:27.742416+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:22:02.269614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:22:02.269614+0000","last_clean_scrub_stamp":"2024-03-21T10:22:02.269614+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T14:25:13.534311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"95'6","reported_seq":193,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T10:25:00.943126+0000","last_change":"2024-03-21T10:21:57.862094+0000","last_active":"2024-03-21T10:25:00.943126+0000","last_peered":"2024-03-21T10:25:00.943126+0000","last_clean":"2024-03-21T10:25:00.943126+0000","last_became_active":"2024-03-21T10:21:57.860307+0000","last_became_peered":"2024-03-21T10:21:57.860307+0000","last_unstale":"2024-03-21T10:25:00.943126+0000","last_undegraded":"2024-03-21T10:25:00.943126+0000","last_fullsized":"2024-03-21T10:25:00.943126+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:21:56.788908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:21:56.788908+0000","last_clean_scrub_stamp":"2024-03-21T10:21:56.788908+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T12:58:50.701218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1081,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":155,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"95'7","reported_seq":37,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T10:23:56.024789+0000","last_change":"2024-03-21T10:21:58.870880+0000","last_active":"2024-03-21T10:23:56.024789+0000","last_peered":"2024-03-21T10:23:56.024789+0000","last_clean":"2024-03-21T10:23:56.024789+0000","last_became_active":"2024-03-21T10:21:58.869882+0000","last_became_peered":"2024-03-21T10:21:58.869882+0000","last_unstale":"2024-03-21T10:23:56.024789+0000","last_undegraded":"2024-03-21T10:23:56.024789+0000","last_fullsized":"2024-03-21T10:23:56.024789+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_clean_scrub_stamp":"2024-03-21T10:21:57.813712+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T18:01:42.613732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":23,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T10:23:27.742882+0000","last_change":"2024-03-21T10:22:03.634425+0000","last_active":"2024-03-21T10:23:27.742882+0000","last_peered":"2024-03-21T10:23:27.742882+0000","last_clean":"2024-03-21T10:23:27.742882+0000","last_became_active":"2024-03-21T10:22:03.632464+0000","last_became_peered":"2024-03-21T10:22:03.632464+0000","last_unstale":"2024-03-21T10:23:27.742882+0000","last_undegraded":"2024-03-21T10:23:27.742882+0000","last_fullsized":"2024-03-21T10:23:27.742882+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:22:02.269614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:22:02.269614+0000","last_clean_scrub_stamp":"2024-03-21T10:22:02.269614+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T16:03:28.270757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"95'8","reported_seq":19,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T10:23:56.006280+0000","last_change":"2024-03-21T10:23:27.649397+0000","last_active":"2024-03-21T10:23:56.006280+0000","last_peered":"2024-03-21T10:23:56.006280+0000","last_clean":"2024-03-21T10:23:56.006280+0000","last_became_active":"2024-03-21T10:23:27.649148+0000","last_became_peered":"2024-03-21T10:23:27.649148+0000","last_unstale":"2024-03-21T10:23:56.006280+0000","last_undegraded":"2024-03-21T10:23:56.006280+0000","last_fullsized":"2024-03-21T10:23:56.006280+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_clean_scrub_stamp":"2024-03-21T10:21:57.813712+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T19:28:03.830551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,8],"acting":[9,2,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.1","version":"95'7","reported_seq":195,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T10:25:01.994641+0000","last_change":"2024-03-21T10:21:57.856111+0000","last_active":"2024-03-21T10:25:01.994641+0000","last_peered":"2024-03-21T10:25:01.994641+0000","last_clean":"2024-03-21T10:25:01.994641+0000","last_became_active":"2024-03-21T10:21:57.855584+0000","last_became_peered":"2024-03-21T10:21:57.855584+0000","last_unstale":"2024-03-21T10:25:01.994641+0000","last_undegraded":"2024-03-21T10:25:01.994641+0000","last_fullsized":"2024-03-21T10:25:01.994641+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:21:56.788908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:21:56.788908+0000","last_clean_scrub_stamp":"2024-03-21T10:21:56.788908+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T17:53:48.208554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":156,"num_read_kb":156,"num_write":7,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"93'1","reported_seq":24,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T10:23:27.645180+0000","last_change":"2024-03-21T10:22:03.318831+0000","last_active":"2024-03-21T10:23:27.645180+0000","last_peered":"2024-03-21T10:23:27.645180+0000","last_clean":"2024-03-21T10:23:27.645180+0000","last_became_active":"2024-03-21T10:22:03.318404+0000","last_became_peered":"2024-03-21T10:22:03.318404+0000","last_unstale":"2024-03-21T10:23:27.645180+0000","last_undegraded":"2024-03-21T10:23:27.645180+0000","last_fullsized":"2024-03-21T10:23:27.645180+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:22:02.269614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:22:02.269614+0000","last_clean_scrub_stamp":"2024-03-21T10:22:02.269614+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T12:06:52.043286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"95'8","reported_seq":38,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T10:23:56.038328+0000","last_change":"2024-03-21T10:21:58.856469+0000","last_active":"2024-03-21T10:23:56.038328+0000","last_peered":"2024-03-21T10:23:56.038328+0000","last_clean":"2024-03-21T10:23:56.038328+0000","last_became_active":"2024-03-21T10:21:58.856032+0000","last_became_peered":"2024-03-21T10:21:58.856032+0000","last_unstale":"2024-03-21T10:23:56.038328+0000","last_undegraded":"2024-03-21T10:23:56.038328+0000","last_fullsized":"2024-03-21T10:23:56.038328+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_clean_scrub_stamp":"2024-03-21T10:21:57.813712+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T21:19:59.302418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"93'6","reported_seq":41,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T10:23:27.640569+0000","last_change":"2024-03-21T10:21:57.854210+0000","last_active":"2024-03-21T10:23:27.640569+0000","last_peered":"2024-03-21T10:23:27.640569+0000","last_clean":"2024-03-21T10:23:27.640569+0000","last_became_active":"2024-03-21T10:21:57.853787+0000","last_became_peered":"2024-03-21T10:21:57.853787+0000","last_unstale":"2024-03-21T10:23:27.640569+0000","last_undegraded":"2024-03-21T10:23:27.640569+0000","last_fullsized":"2024-03-21T10:23:27.640569+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:21:56.788908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:21:56.788908+0000","last_clean_scrub_stamp":"2024-03-21T10:21:56.788908+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T12:10:42.926436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"23'137","reported_seq":367,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T10:23:27.643665+0000","last_change":"2024-03-21T10:20:21.585339+0000","last_active":"2024-03-21T10:23:27.643665+0000","last_peered":"2024-03-21T10:23:27.643665+0000","last_clean":"2024-03-21T10:23:27.643665+0000","last_became_active":"2024-03-21T10:20:21.583999+0000","last_became_peered":"2024-03-21T10:20:21.583999+0000","last_unstale":"2024-03-21T10:23:27.643665+0000","last_undegraded":"2024-03-21T10:23:27.643665+0000","last_fullsized":"2024-03-21T10:23:27.643665+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:15:40.696168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:15:40.696168+0000","last_clean_scrub_stamp":"2024-03-21T10:15:40.696168+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T15:31:54.603722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":23,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T10:23:27.752359+0000","last_change":"2024-03-21T10:22:03.634970+0000","last_active":"2024-03-21T10:23:27.752359+0000","last_peered":"2024-03-21T10:23:27.752359+0000","last_clean":"2024-03-21T10:23:27.752359+0000","last_became_active":"2024-03-21T10:22:03.633603+0000","last_became_peered":"2024-03-21T10:22:03.633603+0000","last_unstale":"2024-03-21T10:23:27.752359+0000","last_undegraded":"2024-03-21T10:23:27.752359+0000","last_fullsized":"2024-03-21T10:23:27.752359+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:22:02.269614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:22:02.269614+0000","last_clean_scrub_stamp":"2024-03-21T10:22:02.269614+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T12:13:02.041661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"95'5","reported_seq":35,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T10:23:56.002047+0000","last_change":"2024-03-21T10:21:58.861159+0000","last_active":"2024-03-21T10:23:56.002047+0000","last_peered":"2024-03-21T10:23:56.002047+0000","last_clean":"2024-03-21T10:23:56.002047+0000","last_became_active":"2024-03-21T10:21:58.860690+0000","last_became_peered":"2024-03-21T10:21:58.860690+0000","last_unstale":"2024-03-21T10:23:56.002047+0000","last_undegraded":"2024-03-21T10:23:56.002047+0000","last_fullsized":"2024-03-21T10:23:56.002047+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_clean_scrub_stamp":"2024-03-21T10:21:57.813712+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T16:21:53.632997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"93'4","reported_seq":36,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T10:23:27.643848+0000","last_change":"2024-03-21T10:21:57.863581+0000","last_active":"2024-03-21T10:23:27.643848+0000","last_peered":"2024-03-21T10:23:27.643848+0000","last_clean":"2024-03-21T10:23:27.643848+0000","last_became_active":"2024-03-21T10:21:57.863269+0000","last_became_peered":"2024-03-21T10:21:57.863269+0000","last_unstale":"2024-03-21T10:23:27.643848+0000","last_undegraded":"2024-03-21T10:23:27.643848+0000","last_fullsized":"2024-03-21T10:23:27.643848+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:21:56.788908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:21:56.788908+0000","last_clean_scrub_stamp":"2024-03-21T10:21:56.788908+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T14:03:59.930570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":23,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T10:23:27.835814+0000","last_change":"2024-03-21T10:22:03.313515+0000","last_active":"2024-03-21T10:23:27.835814+0000","last_peered":"2024-03-21T10:23:27.835814+0000","last_clean":"2024-03-21T10:23:27.835814+0000","last_became_active":"2024-03-21T10:22:03.311862+0000","last_became_peered":"2024-03-21T10:22:03.311862+0000","last_unstale":"2024-03-21T10:23:27.835814+0000","last_undegraded":"2024-03-21T10:23:27.835814+0000","last_fullsized":"2024-03-21T10:23:27.835814+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:22:02.269614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:22:02.269614+0000","last_clean_scrub_stamp":"2024-03-21T10:22:02.269614+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T16:47:50.761205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"93'4","reported_seq":37,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T10:23:27.646442+0000","last_change":"2024-03-21T10:21:57.870913+0000","last_active":"2024-03-21T10:23:27.646442+0000","last_peered":"2024-03-21T10:23:27.646442+0000","last_clean":"2024-03-21T10:23:27.646442+0000","last_became_active":"2024-03-21T10:21:57.870578+0000","last_became_peered":"2024-03-21T10:21:57.870578+0000","last_unstale":"2024-03-21T10:23:27.646442+0000","last_undegraded":"2024-03-21T10:23:27.646442+0000","last_fullsized":"2024-03-21T10:23:27.646442+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:21:56.788908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:21:56.788908+0000","last_clean_scrub_stamp":"2024-03-21T10:21:56.788908+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T11:46:48.621501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"95'9","reported_seq":39,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T10:23:56.007067+0000","last_change":"2024-03-21T10:21:58.852247+0000","last_active":"2024-03-21T10:23:56.007067+0000","last_peered":"2024-03-21T10:23:56.007067+0000","last_clean":"2024-03-21T10:23:56.007067+0000","last_became_active":"2024-03-21T10:21:58.851976+0000","last_became_peered":"2024-03-21T10:21:58.851976+0000","last_unstale":"2024-03-21T10:23:56.007067+0000","last_undegraded":"2024-03-21T10:23:56.007067+0000","last_fullsized":"2024-03-21T10:23:56.007067+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_clean_scrub_stamp":"2024-03-21T10:21:57.813712+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T14:27:11.383741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":23,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T10:23:27.638289+0000","last_change":"2024-03-21T10:22:03.637671+0000","last_active":"2024-03-21T10:23:27.638289+0000","last_peered":"2024-03-21T10:23:27.638289+0000","last_clean":"2024-03-21T10:23:27.638289+0000","last_became_active":"2024-03-21T10:22:03.634265+0000","last_became_peered":"2024-03-21T10:22:03.634265+0000","last_unstale":"2024-03-21T10:23:27.638289+0000","last_undegraded":"2024-03-21T10:23:27.638289+0000","last_fullsized":"2024-03-21T10:23:27.638289+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:22:02.269614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:22:02.269614+0000","last_clean_scrub_stamp":"2024-03-21T10:22:02.269614+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T15:38:02.214178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"95'16","reported_seq":48,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T10:24:57.291140+0000","last_change":"2024-03-21T10:21:57.862027+0000","last_active":"2024-03-21T10:24:57.291140+0000","last_peered":"2024-03-21T10:24:57.291140+0000","last_clean":"2024-03-21T10:24:57.291140+0000","last_became_active":"2024-03-21T10:21:57.860097+0000","last_became_peered":"2024-03-21T10:21:57.860097+0000","last_unstale":"2024-03-21T10:24:57.291140+0000","last_undegraded":"2024-03-21T10:24:57.291140+0000","last_fullsized":"2024-03-21T10:24:57.291140+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:21:56.788908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:21:56.788908+0000","last_clean_scrub_stamp":"2024-03-21T10:21:56.788908+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T13:46:19.562452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":28,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"95'5","reported_seq":35,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T10:23:56.040770+0000","last_change":"2024-03-21T10:21:58.856082+0000","last_active":"2024-03-21T10:23:56.040770+0000","last_peered":"2024-03-21T10:23:56.040770+0000","last_clean":"2024-03-21T10:23:56.040770+0000","last_became_active":"2024-03-21T10:21:58.854172+0000","last_became_peered":"2024-03-21T10:21:58.854172+0000","last_unstale":"2024-03-21T10:23:56.040770+0000","last_undegraded":"2024-03-21T10:23:56.040770+0000","last_fullsized":"2024-03-21T10:23:56.040770+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_clean_scrub_stamp":"2024-03-21T10:21:57.813712+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T14:59:33.995716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":23,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T10:23:27.647010+0000","last_change":"2024-03-21T10:22:03.310205+0000","last_active":"2024-03-21T10:23:27.647010+0000","last_peered":"2024-03-21T10:23:27.647010+0000","last_clean":"2024-03-21T10:23:27.647010+0000","last_became_active":"2024-03-21T10:22:03.309039+0000","last_became_peered":"2024-03-21T10:22:03.309039+0000","last_unstale":"2024-03-21T10:23:27.647010+0000","last_undegraded":"2024-03-21T10:23:27.647010+0000","last_fullsized":"2024-03-21T10:23:27.647010+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:22:02.269614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:22:02.269614+0000","last_clean_scrub_stamp":"2024-03-21T10:22:02.269614+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T11:16:46.075260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"93'7","reported_seq":40,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T10:23:27.641787+0000","last_change":"2024-03-21T10:21:57.857890+0000","last_active":"2024-03-21T10:23:27.641787+0000","last_peered":"2024-03-21T10:23:27.641787+0000","last_clean":"2024-03-21T10:23:27.641787+0000","last_became_active":"2024-03-21T10:21:57.857492+0000","last_became_peered":"2024-03-21T10:21:57.857492+0000","last_unstale":"2024-03-21T10:23:27.641787+0000","last_undegraded":"2024-03-21T10:23:27.641787+0000","last_fullsized":"2024-03-21T10:23:27.641787+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:21:56.788908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:21:56.788908+0000","last_clean_scrub_stamp":"2024-03-21T10:21:56.788908+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T21:18:45.566843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"95'7","reported_seq":43,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T10:23:56.040975+0000","last_change":"2024-03-21T10:22:27.219339+0000","last_active":"2024-03-21T10:23:56.040975+0000","last_peered":"2024-03-21T10:23:56.040975+0000","last_clean":"2024-03-21T10:23:56.040975+0000","last_became_active":"2024-03-21T10:22:27.218604+0000","last_became_peered":"2024-03-21T10:22:27.218604+0000","last_unstale":"2024-03-21T10:23:56.040975+0000","last_undegraded":"2024-03-21T10:23:56.040975+0000","last_fullsized":"2024-03-21T10:23:56.040975+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_clean_scrub_stamp":"2024-03-21T10:21:57.813712+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T15:58:53.161767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,4],"acting":[0,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":23,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T10:23:27.641301+0000","last_change":"2024-03-21T10:22:03.633789+0000","last_active":"2024-03-21T10:23:27.641301+0000","last_peered":"2024-03-21T10:23:27.641301+0000","last_clean":"2024-03-21T10:23:27.641301+0000","last_became_active":"2024-03-21T10:22:03.632716+0000","last_became_peered":"2024-03-21T10:22:03.632716+0000","last_unstale":"2024-03-21T10:23:27.641301+0000","last_undegraded":"2024-03-21T10:23:27.641301+0000","last_fullsized":"2024-03-21T10:23:27.641301+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:22:02.269614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:22:02.269614+0000","last_clean_scrub_stamp":"2024-03-21T10:22:02.269614+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T14:08:47.301881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"91'2","reported_seq":34,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T10:23:27.833770+0000","last_change":"2024-03-21T10:21:57.862413+0000","last_active":"2024-03-21T10:23:27.833770+0000","last_peered":"2024-03-21T10:23:27.833770+0000","last_clean":"2024-03-21T10:23:27.833770+0000","last_became_active":"2024-03-21T10:21:57.862104+0000","last_became_peered":"2024-03-21T10:21:57.862104+0000","last_unstale":"2024-03-21T10:23:27.833770+0000","last_undegraded":"2024-03-21T10:23:27.833770+0000","last_fullsized":"2024-03-21T10:23:27.833770+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:21:56.788908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:21:56.788908+0000","last_clean_scrub_stamp":"2024-03-21T10:21:56.788908+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T17:49:21.729492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"95'2","reported_seq":32,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T10:23:56.014082+0000","last_change":"2024-03-21T10:21:58.864688+0000","last_active":"2024-03-21T10:23:56.014082+0000","last_peered":"2024-03-21T10:23:56.014082+0000","last_clean":"2024-03-21T10:23:56.014082+0000","last_became_active":"2024-03-21T10:21:58.862821+0000","last_became_peered":"2024-03-21T10:21:58.862821+0000","last_unstale":"2024-03-21T10:23:56.014082+0000","last_undegraded":"2024-03-21T10:23:56.014082+0000","last_fullsized":"2024-03-21T10:23:56.014082+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_clean_scrub_stamp":"2024-03-21T10:21:57.813712+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T11:17:09.650649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T10:23:27.756711+0000","last_change":"2024-03-21T10:23:27.756711+0000","last_active":"2024-03-21T10:23:27.756711+0000","last_peered":"2024-03-21T10:23:27.756711+0000","last_clean":"2024-03-21T10:23:27.756711+0000","last_became_active":"2024-03-21T10:23:27.754993+0000","last_became_peered":"2024-03-21T10:23:27.754993+0000","last_unstale":"2024-03-21T10:23:27.756711+0000","last_undegraded":"2024-03-21T10:23:27.756711+0000","last_fullsized":"2024-03-21T10:23:27.756711+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:22:02.269614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:22:02.269614+0000","last_clean_scrub_stamp":"2024-03-21T10:22:02.269614+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T11:22:51.851047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,2,5],"acting":[8,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"93'7","reported_seq":39,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T10:23:27.638625+0000","last_change":"2024-03-21T10:21:57.868033+0000","last_active":"2024-03-21T10:23:27.638625+0000","last_peered":"2024-03-21T10:23:27.638625+0000","last_clean":"2024-03-21T10:23:27.638625+0000","last_became_active":"2024-03-21T10:21:57.867279+0000","last_became_peered":"2024-03-21T10:21:57.867279+0000","last_unstale":"2024-03-21T10:23:27.638625+0000","last_undegraded":"2024-03-21T10:23:27.638625+0000","last_fullsized":"2024-03-21T10:23:27.638625+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:21:56.788908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:21:56.788908+0000","last_clean_scrub_stamp":"2024-03-21T10:21:56.788908+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T20:05:37.701443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"95'9","reported_seq":39,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T10:23:56.022588+0000","last_change":"2024-03-21T10:21:58.868455+0000","last_active":"2024-03-21T10:23:56.022588+0000","last_peered":"2024-03-21T10:23:56.022588+0000","last_clean":"2024-03-21T10:23:56.022588+0000","last_became_active":"2024-03-21T10:21:58.867786+0000","last_became_peered":"2024-03-21T10:21:58.867786+0000","last_unstale":"2024-03-21T10:23:56.022588+0000","last_undegraded":"2024-03-21T10:23:56.022588+0000","last_fullsized":"2024-03-21T10:23:56.022588+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_clean_scrub_stamp":"2024-03-21T10:21:57.813712+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T19:16:12.258470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"93'1","reported_seq":35,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T10:23:27.753821+0000","last_change":"2024-03-21T10:23:27.753222+0000","last_active":"2024-03-21T10:23:27.753821+0000","last_peered":"2024-03-21T10:23:27.753821+0000","last_clean":"2024-03-21T10:23:27.753821+0000","last_became_active":"2024-03-21T10:23:27.746251+0000","last_became_peered":"2024-03-21T10:23:27.746251+0000","last_unstale":"2024-03-21T10:23:27.753821+0000","last_undegraded":"2024-03-21T10:23:27.753821+0000","last_fullsized":"2024-03-21T10:23:27.753821+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:22:02.269614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:22:02.269614+0000","last_clean_scrub_stamp":"2024-03-21T10:22:02.269614+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T13:42:00.362336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":117,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,7,8],"acting":[4,10,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"95'5","reported_seq":37,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T10:24:57.202344+0000","last_change":"2024-03-21T10:21:57.862748+0000","last_active":"2024-03-21T10:24:57.202344+0000","last_peered":"2024-03-21T10:24:57.202344+0000","last_clean":"2024-03-21T10:24:57.202344+0000","last_became_active":"2024-03-21T10:21:57.862350+0000","last_became_peered":"2024-03-21T10:21:57.862350+0000","last_unstale":"2024-03-21T10:24:57.202344+0000","last_undegraded":"2024-03-21T10:24:57.202344+0000","last_fullsized":"2024-03-21T10:24:57.202344+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:21:56.788908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:21:56.788908+0000","last_clean_scrub_stamp":"2024-03-21T10:21:56.788908+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T17:08:47.301758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"95'8","reported_seq":38,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T10:23:56.026242+0000","last_change":"2024-03-21T10:21:58.866340+0000","last_active":"2024-03-21T10:23:56.026242+0000","last_peered":"2024-03-21T10:23:56.026242+0000","last_clean":"2024-03-21T10:23:56.026242+0000","last_became_active":"2024-03-21T10:21:58.865965+0000","last_became_peered":"2024-03-21T10:21:58.865965+0000","last_unstale":"2024-03-21T10:23:56.026242+0000","last_undegraded":"2024-03-21T10:23:56.026242+0000","last_fullsized":"2024-03-21T10:23:56.026242+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_clean_scrub_stamp":"2024-03-21T10:21:57.813712+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T10:42:23.854176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":23,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T10:23:27.640308+0000","last_change":"2024-03-21T10:22:03.634108+0000","last_active":"2024-03-21T10:23:27.640308+0000","last_peered":"2024-03-21T10:23:27.640308+0000","last_clean":"2024-03-21T10:23:27.640308+0000","last_became_active":"2024-03-21T10:22:03.633344+0000","last_became_peered":"2024-03-21T10:22:03.633344+0000","last_unstale":"2024-03-21T10:23:27.640308+0000","last_undegraded":"2024-03-21T10:23:27.640308+0000","last_fullsized":"2024-03-21T10:23:27.640308+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:22:02.269614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:22:02.269614+0000","last_clean_scrub_stamp":"2024-03-21T10:22:02.269614+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T21:00:59.893994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"91'4","reported_seq":37,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T10:23:27.644244+0000","last_change":"2024-03-21T10:21:57.871520+0000","last_active":"2024-03-21T10:23:27.644244+0000","last_peered":"2024-03-21T10:23:27.644244+0000","last_clean":"2024-03-21T10:23:27.644244+0000","last_became_active":"2024-03-21T10:21:57.871127+0000","last_became_peered":"2024-03-21T10:21:57.871127+0000","last_unstale":"2024-03-21T10:23:27.644244+0000","last_undegraded":"2024-03-21T10:23:27.644244+0000","last_fullsized":"2024-03-21T10:23:27.644244+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:21:56.788908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:21:56.788908+0000","last_clean_scrub_stamp":"2024-03-21T10:21:56.788908+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T20:43:39.537307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"95'5","reported_seq":35,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T10:23:56.044719+0000","last_change":"2024-03-21T10:21:58.861786+0000","last_active":"2024-03-21T10:23:56.044719+0000","last_peered":"2024-03-21T10:23:56.044719+0000","last_clean":"2024-03-21T10:23:56.044719+0000","last_became_active":"2024-03-21T10:21:58.861428+0000","last_became_peered":"2024-03-21T10:21:58.861428+0000","last_unstale":"2024-03-21T10:23:56.044719+0000","last_undegraded":"2024-03-21T10:23:56.044719+0000","last_fullsized":"2024-03-21T10:23:56.044719+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_clean_scrub_stamp":"2024-03-21T10:21:57.813712+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T15:56:30.256266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":23,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T10:23:27.833726+0000","last_change":"2024-03-21T10:22:03.305085+0000","last_active":"2024-03-21T10:23:27.833726+0000","last_peered":"2024-03-21T10:23:27.833726+0000","last_clean":"2024-03-21T10:23:27.833726+0000","last_became_active":"2024-03-21T10:22:03.304474+0000","last_became_peered":"2024-03-21T10:22:03.304474+0000","last_unstale":"2024-03-21T10:23:27.833726+0000","last_undegraded":"2024-03-21T10:23:27.833726+0000","last_fullsized":"2024-03-21T10:23:27.833726+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:22:02.269614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:22:02.269614+0000","last_clean_scrub_stamp":"2024-03-21T10:22:02.269614+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T18:28:10.480981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"95'6","reported_seq":196,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T10:25:01.999354+0000","last_change":"2024-03-21T10:21:57.860657+0000","last_active":"2024-03-21T10:25:01.999354+0000","last_peered":"2024-03-21T10:25:01.999354+0000","last_clean":"2024-03-21T10:25:01.999354+0000","last_became_active":"2024-03-21T10:21:57.860340+0000","last_became_peered":"2024-03-21T10:21:57.860340+0000","last_unstale":"2024-03-21T10:25:01.999354+0000","last_undegraded":"2024-03-21T10:25:01.999354+0000","last_fullsized":"2024-03-21T10:25:01.999354+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:21:56.788908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:21:56.788908+0000","last_clean_scrub_stamp":"2024-03-21T10:21:56.788908+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T21:58:08.706812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1663,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":158,"num_read_kb":3,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"95'7","reported_seq":37,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T10:23:56.048801+0000","last_change":"2024-03-21T10:21:58.874254+0000","last_active":"2024-03-21T10:23:56.048801+0000","last_peered":"2024-03-21T10:23:56.048801+0000","last_clean":"2024-03-21T10:23:56.048801+0000","last_became_active":"2024-03-21T10:21:58.869795+0000","last_became_peered":"2024-03-21T10:21:58.869795+0000","last_unstale":"2024-03-21T10:23:56.048801+0000","last_undegraded":"2024-03-21T10:23:56.048801+0000","last_fullsized":"2024-03-21T10:23:56.048801+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_clean_scrub_stamp":"2024-03-21T10:21:57.813712+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T19:51:28.296199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":23,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T10:23:27.834356+0000","last_change":"2024-03-21T10:22:03.306761+0000","last_active":"2024-03-21T10:23:27.834356+0000","last_peered":"2024-03-21T10:23:27.834356+0000","last_clean":"2024-03-21T10:23:27.834356+0000","last_became_active":"2024-03-21T10:22:03.305106+0000","last_became_peered":"2024-03-21T10:22:03.305106+0000","last_unstale":"2024-03-21T10:23:27.834356+0000","last_undegraded":"2024-03-21T10:23:27.834356+0000","last_fullsized":"2024-03-21T10:23:27.834356+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:22:02.269614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:22:02.269614+0000","last_clean_scrub_stamp":"2024-03-21T10:22:02.269614+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T21:13:00.846169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"93'3","reported_seq":36,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T10:23:27.752174+0000","last_change":"2024-03-21T10:21:57.868012+0000","last_active":"2024-03-21T10:23:27.752174+0000","last_peered":"2024-03-21T10:23:27.752174+0000","last_clean":"2024-03-21T10:23:27.752174+0000","last_became_active":"2024-03-21T10:21:57.867682+0000","last_became_peered":"2024-03-21T10:21:57.867682+0000","last_unstale":"2024-03-21T10:23:27.752174+0000","last_undegraded":"2024-03-21T10:23:27.752174+0000","last_fullsized":"2024-03-21T10:23:27.752174+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:21:56.788908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:21:56.788908+0000","last_clean_scrub_stamp":"2024-03-21T10:21:56.788908+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T16:59:56.061167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"95'6","reported_seq":36,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T10:23:56.033933+0000","last_change":"2024-03-21T10:21:58.864949+0000","last_active":"2024-03-21T10:23:56.033933+0000","last_peered":"2024-03-21T10:23:56.033933+0000","last_clean":"2024-03-21T10:23:56.033933+0000","last_became_active":"2024-03-21T10:21:58.859796+0000","last_became_peered":"2024-03-21T10:21:58.859796+0000","last_unstale":"2024-03-21T10:23:56.033933+0000","last_undegraded":"2024-03-21T10:23:56.033933+0000","last_fullsized":"2024-03-21T10:23:56.033933+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_clean_scrub_stamp":"2024-03-21T10:21:57.813712+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T18:02:51.873103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":23,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T10:23:27.750167+0000","last_change":"2024-03-21T10:22:03.308451+0000","last_active":"2024-03-21T10:23:27.750167+0000","last_peered":"2024-03-21T10:23:27.750167+0000","last_clean":"2024-03-21T10:23:27.750167+0000","last_became_active":"2024-03-21T10:22:03.307986+0000","last_became_peered":"2024-03-21T10:22:03.307986+0000","last_unstale":"2024-03-21T10:23:27.750167+0000","last_undegraded":"2024-03-21T10:23:27.750167+0000","last_fullsized":"2024-03-21T10:23:27.750167+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:22:02.269614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:22:02.269614+0000","last_clean_scrub_stamp":"2024-03-21T10:22:02.269614+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T21:08:42.241657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"93'5","reported_seq":37,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T10:23:27.742941+0000","last_change":"2024-03-21T10:21:57.851717+0000","last_active":"2024-03-21T10:23:27.742941+0000","last_peered":"2024-03-21T10:23:27.742941+0000","last_clean":"2024-03-21T10:23:27.742941+0000","last_became_active":"2024-03-21T10:21:57.849909+0000","last_became_peered":"2024-03-21T10:21:57.849909+0000","last_unstale":"2024-03-21T10:23:27.742941+0000","last_undegraded":"2024-03-21T10:23:27.742941+0000","last_fullsized":"2024-03-21T10:23:27.742941+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:21:56.788908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:21:56.788908+0000","last_clean_scrub_stamp":"2024-03-21T10:21:56.788908+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T11:41:37.611400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"95'9","reported_seq":39,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T10:23:56.032325+0000","last_change":"2024-03-21T10:21:58.860892+0000","last_active":"2024-03-21T10:23:56.032325+0000","last_peered":"2024-03-21T10:23:56.032325+0000","last_clean":"2024-03-21T10:23:56.032325+0000","last_became_active":"2024-03-21T10:21:58.860403+0000","last_became_peered":"2024-03-21T10:21:58.860403+0000","last_unstale":"2024-03-21T10:23:56.032325+0000","last_undegraded":"2024-03-21T10:23:56.032325+0000","last_fullsized":"2024-03-21T10:23:56.032325+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_clean_scrub_stamp":"2024-03-21T10:21:57.813712+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T15:15:23.303864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"95'9","reported_seq":39,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T10:23:56.010246+0000","last_change":"2024-03-21T10:21:58.869275+0000","last_active":"2024-03-21T10:23:56.010246+0000","last_peered":"2024-03-21T10:23:56.010246+0000","last_clean":"2024-03-21T10:23:56.010246+0000","last_became_active":"2024-03-21T10:21:58.868834+0000","last_became_peered":"2024-03-21T10:21:58.868834+0000","last_unstale":"2024-03-21T10:23:56.010246+0000","last_undegraded":"2024-03-21T10:23:56.010246+0000","last_fullsized":"2024-03-21T10:23:56.010246+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_clean_scrub_stamp":"2024-03-21T10:21:57.813712+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T14:01:11.074702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"95'12","reported_seq":42,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T10:23:56.002992+0000","last_change":"2024-03-21T10:21:58.851658+0000","last_active":"2024-03-21T10:23:56.002992+0000","last_peered":"2024-03-21T10:23:56.002992+0000","last_clean":"2024-03-21T10:23:56.002992+0000","last_became_active":"2024-03-21T10:21:58.851295+0000","last_became_peered":"2024-03-21T10:21:58.851295+0000","last_unstale":"2024-03-21T10:23:56.002992+0000","last_undegraded":"2024-03-21T10:23:56.002992+0000","last_fullsized":"2024-03-21T10:23:56.002992+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_clean_scrub_stamp":"2024-03-21T10:21:57.813712+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T13:15:32.107783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"95'8","reported_seq":38,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T10:23:56.026300+0000","last_change":"2024-03-21T10:21:58.859275+0000","last_active":"2024-03-21T10:23:56.026300+0000","last_peered":"2024-03-21T10:23:56.026300+0000","last_clean":"2024-03-21T10:23:56.026300+0000","last_became_active":"2024-03-21T10:21:58.857054+0000","last_became_peered":"2024-03-21T10:21:58.857054+0000","last_unstale":"2024-03-21T10:23:56.026300+0000","last_undegraded":"2024-03-21T10:23:56.026300+0000","last_fullsized":"2024-03-21T10:23:56.026300+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_clean_scrub_stamp":"2024-03-21T10:21:57.813712+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T20:05:13.819966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"95'7","reported_seq":37,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T10:23:56.022332+0000","last_change":"2024-03-21T10:21:58.869025+0000","last_active":"2024-03-21T10:23:56.022332+0000","last_peered":"2024-03-21T10:23:56.022332+0000","last_clean":"2024-03-21T10:23:56.022332+0000","last_became_active":"2024-03-21T10:21:58.868240+0000","last_became_peered":"2024-03-21T10:21:58.868240+0000","last_unstale":"2024-03-21T10:23:56.022332+0000","last_undegraded":"2024-03-21T10:23:56.022332+0000","last_fullsized":"2024-03-21T10:23:56.022332+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_clean_scrub_stamp":"2024-03-21T10:21:57.813712+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T21:56:45.642916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"95'4","reported_seq":34,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T10:23:56.021712+0000","last_change":"2024-03-21T10:21:58.865024+0000","last_active":"2024-03-21T10:23:56.021712+0000","last_peered":"2024-03-21T10:23:56.021712+0000","last_clean":"2024-03-21T10:23:56.021712+0000","last_became_active":"2024-03-21T10:21:58.864649+0000","last_became_peered":"2024-03-21T10:21:58.864649+0000","last_unstale":"2024-03-21T10:23:56.021712+0000","last_undegraded":"2024-03-21T10:23:56.021712+0000","last_fullsized":"2024-03-21T10:23:56.021712+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_clean_scrub_stamp":"2024-03-21T10:21:57.813712+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T22:09:03.561283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"95'6","reported_seq":36,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T10:23:56.004821+0000","last_change":"2024-03-21T10:21:58.865206+0000","last_active":"2024-03-21T10:23:56.004821+0000","last_peered":"2024-03-21T10:23:56.004821+0000","last_clean":"2024-03-21T10:23:56.004821+0000","last_became_active":"2024-03-21T10:21:58.864669+0000","last_became_peered":"2024-03-21T10:21:58.864669+0000","last_unstale":"2024-03-21T10:23:56.004821+0000","last_undegraded":"2024-03-21T10:23:56.004821+0000","last_fullsized":"2024-03-21T10:23:56.004821+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_clean_scrub_stamp":"2024-03-21T10:21:57.813712+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T13:57:45.668192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"95'15","reported_seq":45,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T10:23:56.044533+0000","last_change":"2024-03-21T10:21:58.862664+0000","last_active":"2024-03-21T10:23:56.044533+0000","last_peered":"2024-03-21T10:23:56.044533+0000","last_clean":"2024-03-21T10:23:56.044533+0000","last_became_active":"2024-03-21T10:21:58.862144+0000","last_became_peered":"2024-03-21T10:21:58.862144+0000","last_unstale":"2024-03-21T10:23:56.044533+0000","last_undegraded":"2024-03-21T10:23:56.044533+0000","last_fullsized":"2024-03-21T10:23:56.044533+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_clean_scrub_stamp":"2024-03-21T10:21:57.813712+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T11:10:10.225102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"95'2","reported_seq":32,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T10:23:56.021964+0000","last_change":"2024-03-21T10:21:58.851000+0000","last_active":"2024-03-21T10:23:56.021964+0000","last_peered":"2024-03-21T10:23:56.021964+0000","last_clean":"2024-03-21T10:23:56.021964+0000","last_became_active":"2024-03-21T10:21:58.850563+0000","last_became_peered":"2024-03-21T10:21:58.850563+0000","last_unstale":"2024-03-21T10:23:56.021964+0000","last_undegraded":"2024-03-21T10:23:56.021964+0000","last_fullsized":"2024-03-21T10:23:56.021964+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_clean_scrub_stamp":"2024-03-21T10:21:57.813712+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T19:39:08.983713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"95'6","reported_seq":36,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T10:23:56.027451+0000","last_change":"2024-03-21T10:21:58.838426+0000","last_active":"2024-03-21T10:23:56.027451+0000","last_peered":"2024-03-21T10:23:56.027451+0000","last_clean":"2024-03-21T10:23:56.027451+0000","last_became_active":"2024-03-21T10:21:58.837625+0000","last_became_peered":"2024-03-21T10:21:58.837625+0000","last_unstale":"2024-03-21T10:23:56.027451+0000","last_undegraded":"2024-03-21T10:23:56.027451+0000","last_fullsized":"2024-03-21T10:23:56.027451+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_clean_scrub_stamp":"2024-03-21T10:21:57.813712+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T16:22:26.523685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"95'8","reported_seq":38,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T10:23:56.022477+0000","last_change":"2024-03-21T10:21:58.854114+0000","last_active":"2024-03-21T10:23:56.022477+0000","last_peered":"2024-03-21T10:23:56.022477+0000","last_clean":"2024-03-21T10:23:56.022477+0000","last_became_active":"2024-03-21T10:21:58.853686+0000","last_became_peered":"2024-03-21T10:21:58.853686+0000","last_unstale":"2024-03-21T10:23:56.022477+0000","last_undegraded":"2024-03-21T10:23:56.022477+0000","last_fullsized":"2024-03-21T10:23:56.022477+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_clean_scrub_stamp":"2024-03-21T10:21:57.813712+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T11:07:27.346766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"95'10","reported_seq":40,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T10:23:56.049981+0000","last_change":"2024-03-21T10:21:58.857778+0000","last_active":"2024-03-21T10:23:56.049981+0000","last_peered":"2024-03-21T10:23:56.049981+0000","last_clean":"2024-03-21T10:23:56.049981+0000","last_became_active":"2024-03-21T10:21:58.857336+0000","last_became_peered":"2024-03-21T10:21:58.857336+0000","last_unstale":"2024-03-21T10:23:56.049981+0000","last_undegraded":"2024-03-21T10:23:56.049981+0000","last_fullsized":"2024-03-21T10:23:56.049981+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_clean_scrub_stamp":"2024-03-21T10:21:57.813712+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T15:22:57.760341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"95'11","reported_seq":41,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T10:23:56.022750+0000","last_change":"2024-03-21T10:21:58.869774+0000","last_active":"2024-03-21T10:23:56.022750+0000","last_peered":"2024-03-21T10:23:56.022750+0000","last_clean":"2024-03-21T10:23:56.022750+0000","last_became_active":"2024-03-21T10:21:58.868815+0000","last_became_peered":"2024-03-21T10:21:58.868815+0000","last_unstale":"2024-03-21T10:23:56.022750+0000","last_undegraded":"2024-03-21T10:23:56.022750+0000","last_fullsized":"2024-03-21T10:23:56.022750+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_clean_scrub_stamp":"2024-03-21T10:21:57.813712+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T16:57:55.082127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"95'8","reported_seq":38,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T10:23:56.004307+0000","last_change":"2024-03-21T10:21:58.864746+0000","last_active":"2024-03-21T10:23:56.004307+0000","last_peered":"2024-03-21T10:23:56.004307+0000","last_clean":"2024-03-21T10:23:56.004307+0000","last_became_active":"2024-03-21T10:21:58.864366+0000","last_became_peered":"2024-03-21T10:21:58.864366+0000","last_unstale":"2024-03-21T10:23:56.004307+0000","last_undegraded":"2024-03-21T10:23:56.004307+0000","last_fullsized":"2024-03-21T10:23:56.004307+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_clean_scrub_stamp":"2024-03-21T10:21:57.813712+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T13:10:27.303843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":117,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":500,"ondisk_log_size":500,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4751360,"data_stored":4722944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":73774,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":1585,"num_read_kb":876,"num_write":268,"num_write_kb":161,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":4464,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":577536,"data_stored":221322,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":230,"ondisk_log_size":230,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":82,"seq":352187318325,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36396,"kb_used_data":1060,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708756,"statfs":{"total":95995035648,"available":95957766144,"internally_reserved":0,"allocated":1085440,"data_stored":529237,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":36174276},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":74,"seq":317827579964,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32916,"kb_used_data":1740,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712236,"statfs":{"total":95995035648,"available":95961329664,"internally_reserved":0,"allocated":1781760,"data_stored":1183916,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":69,"seq":296352743491,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28096,"kb_used_data":1080,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717056,"statfs":{"total":95995035648,"available":95966265344,"internally_reserved":0,"allocated":1105920,"data_stored":527061,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":61,"seq":261993005130,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36412,"kb_used_data":1076,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708740,"statfs":{"total":95995035648,"available":95957749760,"internally_reserved":0,"allocated":1101824,"data_stored":527439,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":53,"seq":227633266770,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28672,"kb_used_data":1656,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716480,"statfs":{"total":95995035648,"available":95965675520,"internally_reserved":0,"allocated":1695744,"data_stored":1123943,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":48,"seq":206158430297,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36392,"kb_used_data":1056,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708760,"statfs":{"total":95995035648,"available":95957770240,"internally_reserved":0,"allocated":1081344,"data_stored":523058,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574970,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32300,"kb_used_data":1124,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712852,"statfs":{"total":95995035648,"available":95961960448,"internally_reserved":0,"allocated":1150976,"data_stored":588936,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738496,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28096,"kb_used_data":1080,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717056,"statfs":{"total":95995035648,"available":95966265344,"internally_reserved":0,"allocated":1105920,"data_stored":524514,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411445,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36400,"kb_used_data":1064,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708752,"statfs":{"total":95995035648,"available":95957762048,"internally_reserved":0,"allocated":1089536,"data_stored":523686,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":27,"seq":115964117101,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36404,"kb_used_data":1068,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708748,"statfs":{"total":95995035648,"available":95957757952,"internally_reserved":0,"allocated":1093632,"data_stored":522273,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":35,"seq":150323855462,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28156,"kb_used_data":1140,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716996,"statfs":{"total":95995035648,"available":95966203904,"internally_reserved":0,"allocated":1167360,"data_stored":589313,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":40,"seq":171798691935,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":37004,"kb_used_data":1668,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708148,"statfs":{"total":95995035648,"available":95957143552,"internally_reserved":0,"allocated":1708032,"data_stored":1119620,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2671,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":86016,"data_stored":67093,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1843,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":102400,"data_stored":67688,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":3313,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1215,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":3540,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":1500,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":118784,"data_stored":67609,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":3298,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-21T10:25:05.193 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-21T10:25:05.447 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:05.446+0000 7fea5dc00640 1 -- 172.21.15.47:0/3073466017 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fea58103270 msgr2=0x7fea58109b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:25:05.447 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:05.446+0000 7fea5dc00640 1 --2- 172.21.15.47:0/3073466017 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fea58103270 0x7fea58109b00 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fea50009ca0 tx=0x7fea5002c100 comp rx=0 tx=0).stop 2024-03-21T10:25:05.447 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:05.446+0000 7fea5dc00640 1 -- 172.21.15.47:0/3073466017 shutdown_connections 2024-03-21T10:25:05.447 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:05.446+0000 7fea5dc00640 1 --2- 172.21.15.47:0/3073466017 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fea58103270 0x7fea58109b00 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:25:05.448 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:05.446+0000 7fea5dc00640 1 --2- 172.21.15.47:0/3073466017 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fea581028b0 0x7fea58102d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:25:05.448 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:05.446+0000 7fea5dc00640 1 --2- 172.21.15.47:0/3073466017 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fea581016c0 0x7fea58101ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:25:05.448 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:05.446+0000 7fea5dc00640 1 -- 172.21.15.47:0/3073466017 >> 172.21.15.47:0/3073466017 conn(0x7fea580fcc50 msgr2=0x7fea580ff070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:25:05.448 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:05.446+0000 7fea5dc00640 1 -- 172.21.15.47:0/3073466017 shutdown_connections 2024-03-21T10:25:05.448 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:05.446+0000 7fea5dc00640 1 -- 172.21.15.47:0/3073466017 wait complete. 2024-03-21T10:25:05.450 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:05.447+0000 7fea5dc00640 1 Processor -- start 2024-03-21T10:25:05.450 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:05.447+0000 7fea5dc00640 1 -- start start 2024-03-21T10:25:05.450 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:05.447+0000 7fea5dc00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fea581016c0 0x7fea58199aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:25:05.450 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:05.447+0000 7fea5dc00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fea581028b0 0x7fea58199fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:25:05.450 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:05.448+0000 7fea5dc00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fea58103270 0x7fea581a1050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:25:05.451 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:05.448+0000 7fea5dc00640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea5810d1f0 con 0x7fea581028b0 2024-03-21T10:25:05.451 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:05.448+0000 7fea5dc00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea5810d070 con 0x7fea581016c0 2024-03-21T10:25:05.451 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:05.448+0000 7fea5dc00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea5810d370 con 0x7fea58103270 2024-03-21T10:25:05.451 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:05.448+0000 7fea4fe00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fea581028b0 0x7fea58199fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:25:05.451 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:05.448+0000 7fea4fe00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fea581028b0 0x7fea58199fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:56292/0 (socket says 172.21.15.47:56292) 2024-03-21T10:25:05.451 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:05.448+0000 7fea4fe00640 1 -- 172.21.15.47:0/17929735 learned_addr learned my addr 172.21.15.47:0/17929735 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:25:05.451 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:05.448+0000 7fea56a00640 1 --2- 172.21.15.47:0/17929735 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fea581016c0 0x7fea58199aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:25:05.451 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:05.448+0000 7fea57400640 1 --2- 172.21.15.47:0/17929735 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fea58103270 0x7fea581a1050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:25:05.451 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:05.448+0000 7fea4fe00640 1 -- 172.21.15.47:0/17929735 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fea58103270 msgr2=0x7fea581a1050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:25:05.451 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:05.448+0000 7fea4fe00640 1 --2- 172.21.15.47:0/17929735 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fea58103270 0x7fea581a1050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:25:05.451 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:05.448+0000 7fea4fe00640 1 -- 172.21.15.47:0/17929735 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fea581016c0 msgr2=0x7fea58199aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:25:05.451 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:05.448+0000 7fea4fe00640 1 --2- 172.21.15.47:0/17929735 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fea581016c0 0x7fea58199aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:25:05.451 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:05.448+0000 7fea4fe00640 1 -- 172.21.15.47:0/17929735 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea581a16c0 con 0x7fea581028b0 2024-03-21T10:25:05.452 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:05.448+0000 7fea57400640 1 --2- 172.21.15.47:0/17929735 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fea58103270 0x7fea581a1050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T10:25:05.452 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:05.448+0000 7fea56a00640 1 --2- 172.21.15.47:0/17929735 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fea581016c0 0x7fea58199aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T10:25:05.452 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:05.449+0000 7fea4fe00640 1 --2- 172.21.15.47:0/17929735 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fea581028b0 0x7fea58199fe0 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7fea4000e9f0 tx=0x7fea4000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:25:05.452 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:05.449+0000 7fea4f400640 1 -- 172.21.15.47:0/17929735 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea4000cd80 con 0x7fea581028b0 2024-03-21T10:25:05.452 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:05.449+0000 7fea4f400640 1 -- 172.21.15.47:0/17929735 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fea400108e0 con 0x7fea581028b0 2024-03-21T10:25:05.452 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:05.449+0000 7fea5dc00640 1 -- 172.21.15.47:0/17929735 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fea581a19b0 con 0x7fea581028b0 2024-03-21T10:25:05.452 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:05.449+0000 7fea4f400640 1 -- 172.21.15.47:0/17929735 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea40018920 con 0x7fea581028b0 2024-03-21T10:25:05.452 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:05.449+0000 7fea5dc00640 1 -- 172.21.15.47:0/17929735 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fea581a1f00 con 0x7fea581028b0 2024-03-21T10:25:05.452 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:05.451+0000 7fea5dc00640 1 -- 172.21.15.47:0/17929735 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fea581a2130 con 0x7fea581028b0 2024-03-21T10:25:05.453 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:05.453+0000 7fea4f400640 1 -- 172.21.15.47:0/17929735 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 18) v1 ==== 137854+0+0 (secure 0 0 0) 0x7fea40010a90 con 0x7fea581028b0 2024-03-21T10:25:05.455 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:05.454+0000 7fea4f400640 1 --2- 172.21.15.47:0/17929735 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7fea3c0a2f20 0x7fea3c0a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:25:05.455 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:05.454+0000 7fea4f400640 1 -- 172.21.15.47:0/17929735 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9216+0+0 (secure 0 0 0) 0x7fea400d00c0 con 0x7fea581028b0 2024-03-21T10:25:05.455 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:05.454+0000 7fea56a00640 1 --2- 172.21.15.47:0/17929735 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7fea3c0a2f20 0x7fea3c0a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:25:05.456 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:05.455+0000 7fea56a00640 1 --2- 172.21.15.47:0/17929735 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7fea3c0a2f20 0x7fea3c0a53e0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7fea480058c0 tx=0x7fea48005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:25:05.459 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:05.459+0000 7fea4f400640 1 -- 172.21.15.47:0/17929735 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fea4009cb20 con 0x7fea581028b0 2024-03-21T10:25:05.463 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:25:05 smithi047 ceph-mon[31547]: pgmap v464: 129 pgs: 129 active+clean; 649 KiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 1.7 KiB/s wr, 14 op/s 2024-03-21T10:25:05.463 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:25:05 smithi047 ceph-mon[31547]: from='client.? 172.21.15.47:0/1123821566' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-21T10:25:05.662 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:05.661+0000 7fea5dc00640 1 -- 172.21.15.47:0/17929735 --> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fea58074ff0 con 0x7fea3c0a2f20 2024-03-21T10:25:05.689 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:05.688+0000 7fea4f400640 1 -- 172.21.15.47:0/17929735 <== mgr.14152 v2:172.21.15.47:6800/2823885994 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+340554 (secure 0 0 0) 0x7fea58074ff0 con 0x7fea3c0a2f20 2024-03-21T10:25:05.693 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-21T10:25:05.698 INFO:teuthology.orchestra.run.smithi047.stderr:dumped all 2024-03-21T10:25:05.699 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:05.697+0000 7fea5dc00640 1 -- 172.21.15.47:0/17929735 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7fea3c0a2f20 msgr2=0x7fea3c0a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:25:05.700 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:05.697+0000 7fea5dc00640 1 --2- 172.21.15.47:0/17929735 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7fea3c0a2f20 0x7fea3c0a53e0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7fea480058c0 tx=0x7fea48005850 comp rx=0 tx=0).stop 2024-03-21T10:25:05.700 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:05.697+0000 7fea5dc00640 1 -- 172.21.15.47:0/17929735 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fea581028b0 msgr2=0x7fea58199fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:25:05.700 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:05.697+0000 7fea5dc00640 1 --2- 172.21.15.47:0/17929735 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fea581028b0 0x7fea58199fe0 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7fea4000e9f0 tx=0x7fea4000eeb0 comp rx=0 tx=0).stop 2024-03-21T10:25:05.700 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:05.698+0000 7fea5dc00640 1 -- 172.21.15.47:0/17929735 shutdown_connections 2024-03-21T10:25:05.700 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:05.698+0000 7fea5dc00640 1 --2- 172.21.15.47:0/17929735 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7fea3c0a2f20 0x7fea3c0a53e0 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:25:05.700 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:05.698+0000 7fea5dc00640 1 --2- 172.21.15.47:0/17929735 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fea58103270 0x7fea581a1050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:25:05.700 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:05.698+0000 7fea5dc00640 1 --2- 172.21.15.47:0/17929735 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fea581028b0 0x7fea58199fe0 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:25:05.700 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:05.698+0000 7fea5dc00640 1 --2- 172.21.15.47:0/17929735 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fea581016c0 0x7fea58199aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:25:05.700 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:05.698+0000 7fea5dc00640 1 -- 172.21.15.47:0/17929735 >> 172.21.15.47:0/17929735 conn(0x7fea580fcc50 msgr2=0x7fea580738d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:25:05.700 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:05.698+0000 7fea5dc00640 1 -- 172.21.15.47:0/17929735 shutdown_connections 2024-03-21T10:25:05.700 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:05.698+0000 7fea5dc00640 1 -- 172.21.15.47:0/17929735 wait complete. 2024-03-21T10:25:05.720 INFO:teuthology.orchestra.run.smithi047.stdout:{"pg_ready":true,"pg_map":{"version":464,"stamp":"2024-03-21T10:25:04.096428+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":664376,"num_objects":102,"num_object_clones":0,"num_object_copies":308,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":1721,"num_read_kb":995,"num_write":451,"num_write_kb":3101,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13,"num_bytes_recovered":1185317,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":869,"ondisk_log_size":869,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":397244,"kb_used_data":14812,"kb_used_omap":18,"kb_used_meta":382317,"kb_avail":1124544580,"statfs":{"total":1151940427776,"available":1151533649920,"internally_reserved":0,"allocated":15167488,"data_stored":8282996,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19142,"internal_metadata":391492922},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":6779,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":151,"num_read_kb":85,"num_write":21,"num_write_kb":20,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.005301"},"pg_stats":[{"pgid":"3.3f","version":"95'8","reported_seq":38,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T10:23:56.049062+0000","last_change":"2024-03-21T10:21:58.872480+0000","last_active":"2024-03-21T10:23:56.049062+0000","last_peered":"2024-03-21T10:23:56.049062+0000","last_clean":"2024-03-21T10:23:56.049062+0000","last_became_active":"2024-03-21T10:21:58.872143+0000","last_became_peered":"2024-03-21T10:21:58.872143+0000","last_unstale":"2024-03-21T10:23:56.049062+0000","last_undegraded":"2024-03-21T10:23:56.049062+0000","last_fullsized":"2024-03-21T10:23:56.049062+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_clean_scrub_stamp":"2024-03-21T10:21:57.813712+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T15:20:00.409201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"95'4","reported_seq":34,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T10:23:56.027679+0000","last_change":"2024-03-21T10:21:58.871130+0000","last_active":"2024-03-21T10:23:56.027679+0000","last_peered":"2024-03-21T10:23:56.027679+0000","last_clean":"2024-03-21T10:23:56.027679+0000","last_became_active":"2024-03-21T10:21:58.870750+0000","last_became_peered":"2024-03-21T10:21:58.870750+0000","last_unstale":"2024-03-21T10:23:56.027679+0000","last_undegraded":"2024-03-21T10:23:56.027679+0000","last_fullsized":"2024-03-21T10:23:56.027679+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_clean_scrub_stamp":"2024-03-21T10:21:57.813712+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T17:38:04.830212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"95'7","reported_seq":37,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T10:23:56.048931+0000","last_change":"2024-03-21T10:21:58.868026+0000","last_active":"2024-03-21T10:23:56.048931+0000","last_peered":"2024-03-21T10:23:56.048931+0000","last_clean":"2024-03-21T10:23:56.048931+0000","last_became_active":"2024-03-21T10:21:58.859877+0000","last_became_peered":"2024-03-21T10:21:58.859877+0000","last_unstale":"2024-03-21T10:23:56.048931+0000","last_undegraded":"2024-03-21T10:23:56.048931+0000","last_fullsized":"2024-03-21T10:23:56.048931+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_clean_scrub_stamp":"2024-03-21T10:21:57.813712+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T14:45:12.870111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"95'8","reported_seq":24,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T10:23:56.009656+0000","last_change":"2024-03-21T10:22:27.231695+0000","last_active":"2024-03-21T10:23:56.009656+0000","last_peered":"2024-03-21T10:23:56.009656+0000","last_clean":"2024-03-21T10:23:56.009656+0000","last_became_active":"2024-03-21T10:22:27.231149+0000","last_became_peered":"2024-03-21T10:22:27.231149+0000","last_unstale":"2024-03-21T10:23:56.009656+0000","last_undegraded":"2024-03-21T10:23:56.009656+0000","last_fullsized":"2024-03-21T10:23:56.009656+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_clean_scrub_stamp":"2024-03-21T10:21:57.813712+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T18:19:19.105366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,2],"acting":[9,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.3b","version":"95'4","reported_seq":34,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T10:23:56.012895+0000","last_change":"2024-03-21T10:21:58.853998+0000","last_active":"2024-03-21T10:23:56.012895+0000","last_peered":"2024-03-21T10:23:56.012895+0000","last_clean":"2024-03-21T10:23:56.012895+0000","last_became_active":"2024-03-21T10:21:58.853059+0000","last_became_peered":"2024-03-21T10:21:58.853059+0000","last_unstale":"2024-03-21T10:23:56.012895+0000","last_undegraded":"2024-03-21T10:23:56.012895+0000","last_fullsized":"2024-03-21T10:23:56.012895+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_clean_scrub_stamp":"2024-03-21T10:21:57.813712+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T17:06:36.859522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"95'6","reported_seq":36,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T10:23:56.024543+0000","last_change":"2024-03-21T10:21:58.855535+0000","last_active":"2024-03-21T10:23:56.024543+0000","last_peered":"2024-03-21T10:23:56.024543+0000","last_clean":"2024-03-21T10:23:56.024543+0000","last_became_active":"2024-03-21T10:21:58.855181+0000","last_became_peered":"2024-03-21T10:21:58.855181+0000","last_unstale":"2024-03-21T10:23:56.024543+0000","last_undegraded":"2024-03-21T10:23:56.024543+0000","last_fullsized":"2024-03-21T10:23:56.024543+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_clean_scrub_stamp":"2024-03-21T10:21:57.813712+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T14:46:35.045646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"95'5","reported_seq":35,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T10:23:56.041045+0000","last_change":"2024-03-21T10:21:58.856527+0000","last_active":"2024-03-21T10:23:56.041045+0000","last_peered":"2024-03-21T10:23:56.041045+0000","last_clean":"2024-03-21T10:23:56.041045+0000","last_became_active":"2024-03-21T10:21:58.856176+0000","last_became_peered":"2024-03-21T10:21:58.856176+0000","last_unstale":"2024-03-21T10:23:56.041045+0000","last_undegraded":"2024-03-21T10:23:56.041045+0000","last_fullsized":"2024-03-21T10:23:56.041045+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_clean_scrub_stamp":"2024-03-21T10:21:57.813712+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T21:34:07.804293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"95'9","reported_seq":39,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T10:23:56.027007+0000","last_change":"2024-03-21T10:21:58.846260+0000","last_active":"2024-03-21T10:23:56.027007+0000","last_peered":"2024-03-21T10:23:56.027007+0000","last_clean":"2024-03-21T10:23:56.027007+0000","last_became_active":"2024-03-21T10:21:58.844245+0000","last_became_peered":"2024-03-21T10:21:58.844245+0000","last_unstale":"2024-03-21T10:23:56.027007+0000","last_undegraded":"2024-03-21T10:23:56.027007+0000","last_fullsized":"2024-03-21T10:23:56.027007+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_clean_scrub_stamp":"2024-03-21T10:21:57.813712+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T16:37:07.480867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"95'11","reported_seq":41,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T10:23:56.049788+0000","last_change":"2024-03-21T10:21:58.858413+0000","last_active":"2024-03-21T10:23:56.049788+0000","last_peered":"2024-03-21T10:23:56.049788+0000","last_clean":"2024-03-21T10:23:56.049788+0000","last_became_active":"2024-03-21T10:21:58.858007+0000","last_became_peered":"2024-03-21T10:21:58.858007+0000","last_unstale":"2024-03-21T10:23:56.049788+0000","last_undegraded":"2024-03-21T10:23:56.049788+0000","last_fullsized":"2024-03-21T10:23:56.049788+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_clean_scrub_stamp":"2024-03-21T10:21:57.813712+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T21:49:12.428918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"95'16","reported_seq":46,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T10:23:56.026737+0000","last_change":"2024-03-21T10:21:58.864393+0000","last_active":"2024-03-21T10:23:56.026737+0000","last_peered":"2024-03-21T10:23:56.026737+0000","last_clean":"2024-03-21T10:23:56.026737+0000","last_became_active":"2024-03-21T10:21:58.864061+0000","last_became_peered":"2024-03-21T10:21:58.864061+0000","last_unstale":"2024-03-21T10:23:56.026737+0000","last_undegraded":"2024-03-21T10:23:56.026737+0000","last_fullsized":"2024-03-21T10:23:56.026737+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_clean_scrub_stamp":"2024-03-21T10:21:57.813712+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T20:19:44.209612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"95'6","reported_seq":36,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T10:23:56.048906+0000","last_change":"2024-03-21T10:21:58.873885+0000","last_active":"2024-03-21T10:23:56.048906+0000","last_peered":"2024-03-21T10:23:56.048906+0000","last_clean":"2024-03-21T10:23:56.048906+0000","last_became_active":"2024-03-21T10:21:58.869738+0000","last_became_peered":"2024-03-21T10:21:58.869738+0000","last_unstale":"2024-03-21T10:23:56.048906+0000","last_undegraded":"2024-03-21T10:23:56.048906+0000","last_fullsized":"2024-03-21T10:23:56.048906+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_clean_scrub_stamp":"2024-03-21T10:21:57.813712+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T13:33:26.816720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"95'7","reported_seq":37,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T10:23:56.007359+0000","last_change":"2024-03-21T10:21:58.867563+0000","last_active":"2024-03-21T10:23:56.007359+0000","last_peered":"2024-03-21T10:23:56.007359+0000","last_clean":"2024-03-21T10:23:56.007359+0000","last_became_active":"2024-03-21T10:21:58.863455+0000","last_became_peered":"2024-03-21T10:21:58.863455+0000","last_unstale":"2024-03-21T10:23:56.007359+0000","last_undegraded":"2024-03-21T10:23:56.007359+0000","last_fullsized":"2024-03-21T10:23:56.007359+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_clean_scrub_stamp":"2024-03-21T10:21:57.813712+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T15:04:16.636126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"95'10","reported_seq":46,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T10:23:56.019337+0000","last_change":"2024-03-21T10:22:27.213460+0000","last_active":"2024-03-21T10:23:56.019337+0000","last_peered":"2024-03-21T10:23:56.019337+0000","last_clean":"2024-03-21T10:23:56.019337+0000","last_became_active":"2024-03-21T10:22:27.212961+0000","last_became_peered":"2024-03-21T10:22:27.212961+0000","last_unstale":"2024-03-21T10:23:56.019337+0000","last_undegraded":"2024-03-21T10:23:56.019337+0000","last_fullsized":"2024-03-21T10:23:56.019337+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_clean_scrub_stamp":"2024-03-21T10:21:57.813712+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T18:26:03.881767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,9],"acting":[6,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"95'7","reported_seq":37,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T10:23:56.024681+0000","last_change":"2024-03-21T10:21:58.870851+0000","last_active":"2024-03-21T10:23:56.024681+0000","last_peered":"2024-03-21T10:23:56.024681+0000","last_clean":"2024-03-21T10:23:56.024681+0000","last_became_active":"2024-03-21T10:21:58.870132+0000","last_became_peered":"2024-03-21T10:21:58.870132+0000","last_unstale":"2024-03-21T10:23:56.024681+0000","last_undegraded":"2024-03-21T10:23:56.024681+0000","last_fullsized":"2024-03-21T10:23:56.024681+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_clean_scrub_stamp":"2024-03-21T10:21:57.813712+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T20:20:04.207090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"95'7","reported_seq":37,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T10:23:56.020213+0000","last_change":"2024-03-21T10:21:58.865084+0000","last_active":"2024-03-21T10:23:56.020213+0000","last_peered":"2024-03-21T10:23:56.020213+0000","last_clean":"2024-03-21T10:23:56.020213+0000","last_became_active":"2024-03-21T10:21:58.864583+0000","last_became_peered":"2024-03-21T10:21:58.864583+0000","last_unstale":"2024-03-21T10:23:56.020213+0000","last_undegraded":"2024-03-21T10:23:56.020213+0000","last_fullsized":"2024-03-21T10:23:56.020213+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_clean_scrub_stamp":"2024-03-21T10:21:57.813712+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T14:09:16.781462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"95'9","reported_seq":39,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T10:23:56.048651+0000","last_change":"2024-03-21T10:21:58.870370+0000","last_active":"2024-03-21T10:23:56.048651+0000","last_peered":"2024-03-21T10:23:56.048651+0000","last_clean":"2024-03-21T10:23:56.048651+0000","last_became_active":"2024-03-21T10:21:58.870044+0000","last_became_peered":"2024-03-21T10:21:58.870044+0000","last_unstale":"2024-03-21T10:23:56.048651+0000","last_undegraded":"2024-03-21T10:23:56.048651+0000","last_fullsized":"2024-03-21T10:23:56.048651+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_clean_scrub_stamp":"2024-03-21T10:21:57.813712+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T16:02:00.342426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"95'8","reported_seq":38,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T10:23:56.049052+0000","last_change":"2024-03-21T10:21:58.872787+0000","last_active":"2024-03-21T10:23:56.049052+0000","last_peered":"2024-03-21T10:23:56.049052+0000","last_clean":"2024-03-21T10:23:56.049052+0000","last_became_active":"2024-03-21T10:21:58.872390+0000","last_became_peered":"2024-03-21T10:21:58.872390+0000","last_unstale":"2024-03-21T10:23:56.049052+0000","last_undegraded":"2024-03-21T10:23:56.049052+0000","last_fullsized":"2024-03-21T10:23:56.049052+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_clean_scrub_stamp":"2024-03-21T10:21:57.813712+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T15:56:51.798173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"95'10","reported_seq":40,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T10:23:56.042614+0000","last_change":"2024-03-21T10:21:58.858474+0000","last_active":"2024-03-21T10:23:56.042614+0000","last_peered":"2024-03-21T10:23:56.042614+0000","last_clean":"2024-03-21T10:23:56.042614+0000","last_became_active":"2024-03-21T10:21:58.858076+0000","last_became_peered":"2024-03-21T10:21:58.858076+0000","last_unstale":"2024-03-21T10:23:56.042614+0000","last_undegraded":"2024-03-21T10:23:56.042614+0000","last_fullsized":"2024-03-21T10:23:56.042614+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_clean_scrub_stamp":"2024-03-21T10:21:57.813712+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T12:25:17.160405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"95'7","reported_seq":37,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T10:23:56.027042+0000","last_change":"2024-03-21T10:21:58.854156+0000","last_active":"2024-03-21T10:23:56.027042+0000","last_peered":"2024-03-21T10:23:56.027042+0000","last_clean":"2024-03-21T10:23:56.027042+0000","last_became_active":"2024-03-21T10:21:58.853758+0000","last_became_peered":"2024-03-21T10:21:58.853758+0000","last_unstale":"2024-03-21T10:23:56.027042+0000","last_undegraded":"2024-03-21T10:23:56.027042+0000","last_fullsized":"2024-03-21T10:23:56.027042+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_clean_scrub_stamp":"2024-03-21T10:21:57.813712+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T16:58:54.854826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":30,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T10:23:27.840549+0000","last_change":"2024-03-21T10:23:27.838445+0000","last_active":"2024-03-21T10:23:27.840549+0000","last_peered":"2024-03-21T10:23:27.840549+0000","last_clean":"2024-03-21T10:23:27.840549+0000","last_became_active":"2024-03-21T10:23:27.836992+0000","last_became_peered":"2024-03-21T10:23:27.836992+0000","last_unstale":"2024-03-21T10:23:27.840549+0000","last_undegraded":"2024-03-21T10:23:27.840549+0000","last_fullsized":"2024-03-21T10:23:27.840549+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:22:02.269614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:22:02.269614+0000","last_clean_scrub_stamp":"2024-03-21T10:22:02.269614+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T15:34:51.615332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8,0],"acting":[2,3,8,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"95'14","reported_seq":200,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T10:25:00.942184+0000","last_change":"2024-03-21T10:21:57.859554+0000","last_active":"2024-03-21T10:25:00.942184+0000","last_peered":"2024-03-21T10:25:00.942184+0000","last_clean":"2024-03-21T10:25:00.942184+0000","last_became_active":"2024-03-21T10:21:57.857997+0000","last_became_peered":"2024-03-21T10:21:57.857997+0000","last_unstale":"2024-03-21T10:25:00.942184+0000","last_undegraded":"2024-03-21T10:25:00.942184+0000","last_fullsized":"2024-03-21T10:25:00.942184+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:21:56.788908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:21:56.788908+0000","last_clean_scrub_stamp":"2024-03-21T10:21:56.788908+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T14:27:47.516832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":154,"num_read_kb":154,"num_write":15,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"95'5","reported_seq":35,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T10:23:56.021709+0000","last_change":"2024-03-21T10:21:58.850197+0000","last_active":"2024-03-21T10:23:56.021709+0000","last_peered":"2024-03-21T10:23:56.021709+0000","last_clean":"2024-03-21T10:23:56.021709+0000","last_became_active":"2024-03-21T10:21:58.849029+0000","last_became_peered":"2024-03-21T10:21:58.849029+0000","last_unstale":"2024-03-21T10:23:56.021709+0000","last_undegraded":"2024-03-21T10:23:56.021709+0000","last_fullsized":"2024-03-21T10:23:56.021709+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_clean_scrub_stamp":"2024-03-21T10:21:57.813712+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T15:46:49.077382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":23,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T10:23:27.640067+0000","last_change":"2024-03-21T10:22:03.320716+0000","last_active":"2024-03-21T10:23:27.640067+0000","last_peered":"2024-03-21T10:23:27.640067+0000","last_clean":"2024-03-21T10:23:27.640067+0000","last_became_active":"2024-03-21T10:22:03.320262+0000","last_became_peered":"2024-03-21T10:22:03.320262+0000","last_unstale":"2024-03-21T10:23:27.640067+0000","last_undegraded":"2024-03-21T10:23:27.640067+0000","last_fullsized":"2024-03-21T10:23:27.640067+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:22:02.269614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:22:02.269614+0000","last_clean_scrub_stamp":"2024-03-21T10:22:02.269614+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T16:16:07.423168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"92'2","reported_seq":34,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T10:23:27.639016+0000","last_change":"2024-03-21T10:21:57.863642+0000","last_active":"2024-03-21T10:23:27.639016+0000","last_peered":"2024-03-21T10:23:27.639016+0000","last_clean":"2024-03-21T10:23:27.639016+0000","last_became_active":"2024-03-21T10:21:57.863330+0000","last_became_peered":"2024-03-21T10:21:57.863330+0000","last_unstale":"2024-03-21T10:23:27.639016+0000","last_undegraded":"2024-03-21T10:23:27.639016+0000","last_fullsized":"2024-03-21T10:23:27.639016+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:21:56.788908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:21:56.788908+0000","last_clean_scrub_stamp":"2024-03-21T10:21:56.788908+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T14:45:02.131458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"95'6","reported_seq":36,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T10:23:56.048702+0000","last_change":"2024-03-21T10:21:58.862940+0000","last_active":"2024-03-21T10:23:56.048702+0000","last_peered":"2024-03-21T10:23:56.048702+0000","last_clean":"2024-03-21T10:23:56.048702+0000","last_became_active":"2024-03-21T10:21:58.862489+0000","last_became_peered":"2024-03-21T10:21:58.862489+0000","last_unstale":"2024-03-21T10:23:56.048702+0000","last_undegraded":"2024-03-21T10:23:56.048702+0000","last_fullsized":"2024-03-21T10:23:56.048702+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_clean_scrub_stamp":"2024-03-21T10:21:57.813712+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T19:28:02.175642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":23,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T10:23:27.640509+0000","last_change":"2024-03-21T10:22:03.318660+0000","last_active":"2024-03-21T10:23:27.640509+0000","last_peered":"2024-03-21T10:23:27.640509+0000","last_clean":"2024-03-21T10:23:27.640509+0000","last_became_active":"2024-03-21T10:22:03.318216+0000","last_became_peered":"2024-03-21T10:22:03.318216+0000","last_unstale":"2024-03-21T10:23:27.640509+0000","last_undegraded":"2024-03-21T10:23:27.640509+0000","last_fullsized":"2024-03-21T10:23:27.640509+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:22:02.269614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:22:02.269614+0000","last_clean_scrub_stamp":"2024-03-21T10:22:02.269614+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T16:36:39.930726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"93'12","reported_seq":31,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T10:23:27.837288+0000","last_change":"2024-03-21T10:22:27.239342+0000","last_active":"2024-03-21T10:23:27.837288+0000","last_peered":"2024-03-21T10:23:27.837288+0000","last_clean":"2024-03-21T10:23:27.837288+0000","last_became_active":"2024-03-21T10:22:27.224086+0000","last_became_peered":"2024-03-21T10:22:27.224086+0000","last_unstale":"2024-03-21T10:23:27.837288+0000","last_undegraded":"2024-03-21T10:23:27.837288+0000","last_fullsized":"2024-03-21T10:23:27.837288+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:21:56.788908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:21:56.788908+0000","last_clean_scrub_stamp":"2024-03-21T10:21:56.788908+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T20:46:45.951485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,7],"acting":[11,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1c","version":"95'9","reported_seq":39,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T10:23:56.044535+0000","last_change":"2024-03-21T10:21:58.873239+0000","last_active":"2024-03-21T10:23:56.044535+0000","last_peered":"2024-03-21T10:23:56.044535+0000","last_clean":"2024-03-21T10:23:56.044535+0000","last_became_active":"2024-03-21T10:21:58.872877+0000","last_became_peered":"2024-03-21T10:21:58.872877+0000","last_unstale":"2024-03-21T10:23:56.044535+0000","last_undegraded":"2024-03-21T10:23:56.044535+0000","last_fullsized":"2024-03-21T10:23:56.044535+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_clean_scrub_stamp":"2024-03-21T10:21:57.813712+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T16:22:49.070697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":23,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T10:23:27.640201+0000","last_change":"2024-03-21T10:22:03.311589+0000","last_active":"2024-03-21T10:23:27.640201+0000","last_peered":"2024-03-21T10:23:27.640201+0000","last_clean":"2024-03-21T10:23:27.640201+0000","last_became_active":"2024-03-21T10:22:03.311134+0000","last_became_peered":"2024-03-21T10:22:03.311134+0000","last_unstale":"2024-03-21T10:23:27.640201+0000","last_undegraded":"2024-03-21T10:23:27.640201+0000","last_fullsized":"2024-03-21T10:23:27.640201+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:22:02.269614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:22:02.269614+0000","last_clean_scrub_stamp":"2024-03-21T10:22:02.269614+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T10:22:29.138348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"93'5","reported_seq":38,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T10:23:27.646290+0000","last_change":"2024-03-21T10:21:57.858981+0000","last_active":"2024-03-21T10:23:27.646290+0000","last_peered":"2024-03-21T10:23:27.646290+0000","last_clean":"2024-03-21T10:23:27.646290+0000","last_became_active":"2024-03-21T10:21:57.858564+0000","last_became_peered":"2024-03-21T10:21:57.858564+0000","last_unstale":"2024-03-21T10:23:27.646290+0000","last_undegraded":"2024-03-21T10:23:27.646290+0000","last_fullsized":"2024-03-21T10:23:27.646290+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:21:56.788908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:21:56.788908+0000","last_clean_scrub_stamp":"2024-03-21T10:21:56.788908+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T10:45:09.488194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"95'15","reported_seq":45,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T10:23:56.039689+0000","last_change":"2024-03-21T10:21:58.858053+0000","last_active":"2024-03-21T10:23:56.039689+0000","last_peered":"2024-03-21T10:23:56.039689+0000","last_clean":"2024-03-21T10:23:56.039689+0000","last_became_active":"2024-03-21T10:21:58.857688+0000","last_became_peered":"2024-03-21T10:21:58.857688+0000","last_unstale":"2024-03-21T10:23:56.039689+0000","last_undegraded":"2024-03-21T10:23:56.039689+0000","last_fullsized":"2024-03-21T10:23:56.039689+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_clean_scrub_stamp":"2024-03-21T10:21:57.813712+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T19:45:13.043034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":30,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T10:23:27.843397+0000","last_change":"2024-03-21T10:23:27.842200+0000","last_active":"2024-03-21T10:23:27.843397+0000","last_peered":"2024-03-21T10:23:27.843397+0000","last_clean":"2024-03-21T10:23:27.843397+0000","last_became_active":"2024-03-21T10:23:27.840749+0000","last_became_peered":"2024-03-21T10:23:27.840749+0000","last_unstale":"2024-03-21T10:23:27.843397+0000","last_undegraded":"2024-03-21T10:23:27.843397+0000","last_fullsized":"2024-03-21T10:23:27.843397+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:22:02.269614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:22:02.269614+0000","last_clean_scrub_stamp":"2024-03-21T10:22:02.269614+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T22:01:31.714771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,7,11],"acting":[2,8,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"95'6","reported_seq":207,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T10:25:00.996435+0000","last_change":"2024-03-21T10:22:27.235744+0000","last_active":"2024-03-21T10:25:00.996435+0000","last_peered":"2024-03-21T10:25:00.996435+0000","last_clean":"2024-03-21T10:25:00.996435+0000","last_became_active":"2024-03-21T10:22:27.219999+0000","last_became_peered":"2024-03-21T10:22:27.219999+0000","last_unstale":"2024-03-21T10:25:00.996435+0000","last_undegraded":"2024-03-21T10:25:00.996435+0000","last_fullsized":"2024-03-21T10:25:00.996435+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:21:56.788908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:21:56.788908+0000","last_clean_scrub_stamp":"2024-03-21T10:21:56.788908+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T18:15:41.288034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1115,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":157,"num_read_kb":3,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":622,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7],"acting":[5,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"95'7","reported_seq":37,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T10:23:55.996971+0000","last_change":"2024-03-21T10:21:58.854752+0000","last_active":"2024-03-21T10:23:55.996971+0000","last_peered":"2024-03-21T10:23:55.996971+0000","last_clean":"2024-03-21T10:23:55.996971+0000","last_became_active":"2024-03-21T10:21:58.854442+0000","last_became_peered":"2024-03-21T10:21:58.854442+0000","last_unstale":"2024-03-21T10:23:55.996971+0000","last_undegraded":"2024-03-21T10:23:55.996971+0000","last_fullsized":"2024-03-21T10:23:55.996971+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_clean_scrub_stamp":"2024-03-21T10:21:57.813712+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T15:12:40.590585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":23,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T10:23:27.648343+0000","last_change":"2024-03-21T10:22:03.319247+0000","last_active":"2024-03-21T10:23:27.648343+0000","last_peered":"2024-03-21T10:23:27.648343+0000","last_clean":"2024-03-21T10:23:27.648343+0000","last_became_active":"2024-03-21T10:22:03.318731+0000","last_became_peered":"2024-03-21T10:22:03.318731+0000","last_unstale":"2024-03-21T10:23:27.648343+0000","last_undegraded":"2024-03-21T10:23:27.648343+0000","last_fullsized":"2024-03-21T10:23:27.648343+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:22:02.269614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:22:02.269614+0000","last_clean_scrub_stamp":"2024-03-21T10:22:02.269614+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T20:06:36.519559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"93'7","reported_seq":45,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T10:23:27.744422+0000","last_change":"2024-03-21T10:22:27.222981+0000","last_active":"2024-03-21T10:23:27.744422+0000","last_peered":"2024-03-21T10:23:27.744422+0000","last_clean":"2024-03-21T10:23:27.744422+0000","last_became_active":"2024-03-21T10:22:27.221088+0000","last_became_peered":"2024-03-21T10:22:27.221088+0000","last_unstale":"2024-03-21T10:23:27.744422+0000","last_undegraded":"2024-03-21T10:23:27.744422+0000","last_fullsized":"2024-03-21T10:23:27.744422+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:21:56.788908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:21:56.788908+0000","last_clean_scrub_stamp":"2024-03-21T10:21:56.788908+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T15:43:11.740353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,6],"acting":[10,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"95'8","reported_seq":44,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T10:23:56.027376+0000","last_change":"2024-03-21T10:22:27.226733+0000","last_active":"2024-03-21T10:23:56.027376+0000","last_peered":"2024-03-21T10:23:56.027376+0000","last_clean":"2024-03-21T10:23:56.027376+0000","last_became_active":"2024-03-21T10:22:27.225576+0000","last_became_peered":"2024-03-21T10:22:27.225576+0000","last_unstale":"2024-03-21T10:23:56.027376+0000","last_undegraded":"2024-03-21T10:23:56.027376+0000","last_fullsized":"2024-03-21T10:23:56.027376+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_clean_scrub_stamp":"2024-03-21T10:21:57.813712+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T20:06:09.534268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,3],"acting":[5,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":23,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T10:23:27.836282+0000","last_change":"2024-03-21T10:22:03.321595+0000","last_active":"2024-03-21T10:23:27.836282+0000","last_peered":"2024-03-21T10:23:27.836282+0000","last_clean":"2024-03-21T10:23:27.836282+0000","last_became_active":"2024-03-21T10:22:03.320897+0000","last_became_peered":"2024-03-21T10:22:03.320897+0000","last_unstale":"2024-03-21T10:23:27.836282+0000","last_undegraded":"2024-03-21T10:23:27.836282+0000","last_fullsized":"2024-03-21T10:23:27.836282+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:22:02.269614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:22:02.269614+0000","last_clean_scrub_stamp":"2024-03-21T10:22:02.269614+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T17:37:02.633709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"93'3","reported_seq":35,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T10:23:27.742215+0000","last_change":"2024-03-21T10:21:57.854801+0000","last_active":"2024-03-21T10:23:27.742215+0000","last_peered":"2024-03-21T10:23:27.742215+0000","last_clean":"2024-03-21T10:23:27.742215+0000","last_became_active":"2024-03-21T10:21:57.854443+0000","last_became_peered":"2024-03-21T10:21:57.854443+0000","last_unstale":"2024-03-21T10:23:27.742215+0000","last_undegraded":"2024-03-21T10:23:27.742215+0000","last_fullsized":"2024-03-21T10:23:27.742215+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:21:56.788908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:21:56.788908+0000","last_clean_scrub_stamp":"2024-03-21T10:21:56.788908+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T15:49:23.573360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"95'8","reported_seq":38,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T10:23:56.026817+0000","last_change":"2024-03-21T10:21:58.857167+0000","last_active":"2024-03-21T10:23:56.026817+0000","last_peered":"2024-03-21T10:23:56.026817+0000","last_clean":"2024-03-21T10:23:56.026817+0000","last_became_active":"2024-03-21T10:21:58.856827+0000","last_became_peered":"2024-03-21T10:21:58.856827+0000","last_unstale":"2024-03-21T10:23:56.026817+0000","last_undegraded":"2024-03-21T10:23:56.026817+0000","last_fullsized":"2024-03-21T10:23:56.026817+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_clean_scrub_stamp":"2024-03-21T10:21:57.813712+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T15:04:43.848209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":23,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T10:23:27.834956+0000","last_change":"2024-03-21T10:22:03.306554+0000","last_active":"2024-03-21T10:23:27.834956+0000","last_peered":"2024-03-21T10:23:27.834956+0000","last_clean":"2024-03-21T10:23:27.834956+0000","last_became_active":"2024-03-21T10:22:03.306087+0000","last_became_peered":"2024-03-21T10:22:03.306087+0000","last_unstale":"2024-03-21T10:23:27.834956+0000","last_undegraded":"2024-03-21T10:23:27.834956+0000","last_fullsized":"2024-03-21T10:23:27.834956+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:22:02.269614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:22:02.269614+0000","last_clean_scrub_stamp":"2024-03-21T10:22:02.269614+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T11:36:35.344770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"93'3","reported_seq":36,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T10:23:27.751080+0000","last_change":"2024-03-21T10:21:57.860343+0000","last_active":"2024-03-21T10:23:27.751080+0000","last_peered":"2024-03-21T10:23:27.751080+0000","last_clean":"2024-03-21T10:23:27.751080+0000","last_became_active":"2024-03-21T10:21:57.859808+0000","last_became_peered":"2024-03-21T10:21:57.859808+0000","last_unstale":"2024-03-21T10:23:27.751080+0000","last_undegraded":"2024-03-21T10:23:27.751080+0000","last_fullsized":"2024-03-21T10:23:27.751080+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:21:56.788908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:21:56.788908+0000","last_clean_scrub_stamp":"2024-03-21T10:21:56.788908+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T14:41:24.162795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"95'11","reported_seq":41,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T10:23:56.023593+0000","last_change":"2024-03-21T10:21:58.850991+0000","last_active":"2024-03-21T10:23:56.023593+0000","last_peered":"2024-03-21T10:23:56.023593+0000","last_clean":"2024-03-21T10:23:56.023593+0000","last_became_active":"2024-03-21T10:21:58.850581+0000","last_became_peered":"2024-03-21T10:21:58.850581+0000","last_unstale":"2024-03-21T10:23:56.023593+0000","last_undegraded":"2024-03-21T10:23:56.023593+0000","last_fullsized":"2024-03-21T10:23:56.023593+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_clean_scrub_stamp":"2024-03-21T10:21:57.813712+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T17:07:11.350806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":23,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T10:23:27.638522+0000","last_change":"2024-03-21T10:22:03.315333+0000","last_active":"2024-03-21T10:23:27.638522+0000","last_peered":"2024-03-21T10:23:27.638522+0000","last_clean":"2024-03-21T10:23:27.638522+0000","last_became_active":"2024-03-21T10:22:03.313553+0000","last_became_peered":"2024-03-21T10:22:03.313553+0000","last_unstale":"2024-03-21T10:23:27.638522+0000","last_undegraded":"2024-03-21T10:23:27.638522+0000","last_fullsized":"2024-03-21T10:23:27.638522+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:22:02.269614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:22:02.269614+0000","last_clean_scrub_stamp":"2024-03-21T10:22:02.269614+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T15:25:24.290789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"95'6","reported_seq":348,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T10:25:03.000542+0000","last_change":"2024-03-21T10:21:57.864930+0000","last_active":"2024-03-21T10:25:03.000542+0000","last_peered":"2024-03-21T10:25:03.000542+0000","last_clean":"2024-03-21T10:25:03.000542+0000","last_became_active":"2024-03-21T10:21:57.863769+0000","last_became_peered":"2024-03-21T10:21:57.863769+0000","last_unstale":"2024-03-21T10:25:03.000542+0000","last_undegraded":"2024-03-21T10:25:03.000542+0000","last_fullsized":"2024-03-21T10:25:03.000542+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:21:56.788908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:21:56.788908+0000","last_clean_scrub_stamp":"2024-03-21T10:21:56.788908+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T21:35:39.144065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":310,"num_read_kb":310,"num_write":7,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"95'10","reported_seq":40,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T10:23:56.047740+0000","last_change":"2024-03-21T10:21:58.872084+0000","last_active":"2024-03-21T10:23:56.047740+0000","last_peered":"2024-03-21T10:23:56.047740+0000","last_clean":"2024-03-21T10:23:56.047740+0000","last_became_active":"2024-03-21T10:21:58.871725+0000","last_became_peered":"2024-03-21T10:21:58.871725+0000","last_unstale":"2024-03-21T10:23:56.047740+0000","last_undegraded":"2024-03-21T10:23:56.047740+0000","last_fullsized":"2024-03-21T10:23:56.047740+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_clean_scrub_stamp":"2024-03-21T10:21:57.813712+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T16:19:37.240053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":23,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T10:23:27.641039+0000","last_change":"2024-03-21T10:22:03.317808+0000","last_active":"2024-03-21T10:23:27.641039+0000","last_peered":"2024-03-21T10:23:27.641039+0000","last_clean":"2024-03-21T10:23:27.641039+0000","last_became_active":"2024-03-21T10:22:03.313751+0000","last_became_peered":"2024-03-21T10:22:03.313751+0000","last_unstale":"2024-03-21T10:23:27.641039+0000","last_undegraded":"2024-03-21T10:23:27.641039+0000","last_fullsized":"2024-03-21T10:23:27.641039+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:22:02.269614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:22:02.269614+0000","last_clean_scrub_stamp":"2024-03-21T10:22:02.269614+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T20:05:08.105795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"93'4","reported_seq":36,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T10:23:27.646634+0000","last_change":"2024-03-21T10:21:57.865462+0000","last_active":"2024-03-21T10:23:27.646634+0000","last_peered":"2024-03-21T10:23:27.646634+0000","last_clean":"2024-03-21T10:23:27.646634+0000","last_became_active":"2024-03-21T10:21:57.862922+0000","last_became_peered":"2024-03-21T10:21:57.862922+0000","last_unstale":"2024-03-21T10:23:27.646634+0000","last_undegraded":"2024-03-21T10:23:27.646634+0000","last_fullsized":"2024-03-21T10:23:27.646634+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:21:56.788908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:21:56.788908+0000","last_clean_scrub_stamp":"2024-03-21T10:21:56.788908+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T15:31:17.167806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"95'9","reported_seq":39,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T10:23:56.022903+0000","last_change":"2024-03-21T10:21:58.865459+0000","last_active":"2024-03-21T10:23:56.022903+0000","last_peered":"2024-03-21T10:23:56.022903+0000","last_clean":"2024-03-21T10:23:56.022903+0000","last_became_active":"2024-03-21T10:21:58.860909+0000","last_became_peered":"2024-03-21T10:21:58.860909+0000","last_unstale":"2024-03-21T10:23:56.022903+0000","last_undegraded":"2024-03-21T10:23:56.022903+0000","last_fullsized":"2024-03-21T10:23:56.022903+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_clean_scrub_stamp":"2024-03-21T10:21:57.813712+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T15:53:35.110404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":23,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T10:23:27.640633+0000","last_change":"2024-03-21T10:22:03.318651+0000","last_active":"2024-03-21T10:23:27.640633+0000","last_peered":"2024-03-21T10:23:27.640633+0000","last_clean":"2024-03-21T10:23:27.640633+0000","last_became_active":"2024-03-21T10:22:03.318108+0000","last_became_peered":"2024-03-21T10:22:03.318108+0000","last_unstale":"2024-03-21T10:23:27.640633+0000","last_undegraded":"2024-03-21T10:23:27.640633+0000","last_fullsized":"2024-03-21T10:23:27.640633+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:22:02.269614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:22:02.269614+0000","last_clean_scrub_stamp":"2024-03-21T10:22:02.269614+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T19:00:04.216749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"93'5","reported_seq":38,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T10:23:27.751795+0000","last_change":"2024-03-21T10:21:57.865516+0000","last_active":"2024-03-21T10:23:27.751795+0000","last_peered":"2024-03-21T10:23:27.751795+0000","last_clean":"2024-03-21T10:23:27.751795+0000","last_became_active":"2024-03-21T10:21:57.865012+0000","last_became_peered":"2024-03-21T10:21:57.865012+0000","last_unstale":"2024-03-21T10:23:27.751795+0000","last_undegraded":"2024-03-21T10:23:27.751795+0000","last_fullsized":"2024-03-21T10:23:27.751795+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:21:56.788908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:21:56.788908+0000","last_clean_scrub_stamp":"2024-03-21T10:21:56.788908+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T21:38:02.813453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"95'11","reported_seq":41,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T10:23:56.048118+0000","last_change":"2024-03-21T10:21:58.870876+0000","last_active":"2024-03-21T10:23:56.048118+0000","last_peered":"2024-03-21T10:23:56.048118+0000","last_clean":"2024-03-21T10:23:56.048118+0000","last_became_active":"2024-03-21T10:21:58.868240+0000","last_became_peered":"2024-03-21T10:21:58.868240+0000","last_unstale":"2024-03-21T10:23:56.048118+0000","last_undegraded":"2024-03-21T10:23:56.048118+0000","last_fullsized":"2024-03-21T10:23:56.048118+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_clean_scrub_stamp":"2024-03-21T10:21:57.813712+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T15:16:59.496020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":23,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T10:23:27.641221+0000","last_change":"2024-03-21T10:22:03.314854+0000","last_active":"2024-03-21T10:23:27.641221+0000","last_peered":"2024-03-21T10:23:27.641221+0000","last_clean":"2024-03-21T10:23:27.641221+0000","last_became_active":"2024-03-21T10:22:03.314422+0000","last_became_peered":"2024-03-21T10:22:03.314422+0000","last_unstale":"2024-03-21T10:23:27.641221+0000","last_undegraded":"2024-03-21T10:23:27.641221+0000","last_fullsized":"2024-03-21T10:23:27.641221+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:22:02.269614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:22:02.269614+0000","last_clean_scrub_stamp":"2024-03-21T10:22:02.269614+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T12:34:31.519118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.14","version":"95'38","reported_seq":407,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T10:25:02.999025+0000","last_change":"2024-03-21T10:22:27.243817+0000","last_active":"2024-03-21T10:25:02.999025+0000","last_peered":"2024-03-21T10:25:02.999025+0000","last_clean":"2024-03-21T10:25:02.999025+0000","last_became_active":"2024-03-21T10:22:27.218873+0000","last_became_peered":"2024-03-21T10:22:27.218873+0000","last_unstale":"2024-03-21T10:25:02.999025+0000","last_undegraded":"2024-03-21T10:25:02.999025+0000","last_fullsized":"2024-03-21T10:25:02.999025+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:21:56.788908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:21:56.788908+0000","last_clean_scrub_stamp":"2024-03-21T10:21:56.788908+0000","objects_scrubbed":0,"log_size":38,"log_dups_size":0,"ondisk_log_size":38,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T12:27:43.574451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":64848,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":319,"num_read_kb":223,"num_write":40,"num_write_kb":82,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":3842,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,4],"acting":[10,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"95'6","reported_seq":36,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T10:23:56.011976+0000","last_change":"2024-03-21T10:21:58.860386+0000","last_active":"2024-03-21T10:23:56.011976+0000","last_peered":"2024-03-21T10:23:56.011976+0000","last_clean":"2024-03-21T10:23:56.011976+0000","last_became_active":"2024-03-21T10:21:58.860013+0000","last_became_peered":"2024-03-21T10:21:58.860013+0000","last_unstale":"2024-03-21T10:23:56.011976+0000","last_undegraded":"2024-03-21T10:23:56.011976+0000","last_fullsized":"2024-03-21T10:23:56.011976+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_clean_scrub_stamp":"2024-03-21T10:21:57.813712+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T18:02:33.920374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":23,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T10:23:27.752685+0000","last_change":"2024-03-21T10:22:03.314083+0000","last_active":"2024-03-21T10:23:27.752685+0000","last_peered":"2024-03-21T10:23:27.752685+0000","last_clean":"2024-03-21T10:23:27.752685+0000","last_became_active":"2024-03-21T10:22:03.313652+0000","last_became_peered":"2024-03-21T10:22:03.313652+0000","last_unstale":"2024-03-21T10:23:27.752685+0000","last_undegraded":"2024-03-21T10:23:27.752685+0000","last_fullsized":"2024-03-21T10:23:27.752685+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:22:02.269614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:22:02.269614+0000","last_clean_scrub_stamp":"2024-03-21T10:22:02.269614+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T13:23:51.256535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"93'9","reported_seq":43,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T10:23:27.648194+0000","last_change":"2024-03-21T10:21:57.866716+0000","last_active":"2024-03-21T10:23:27.648194+0000","last_peered":"2024-03-21T10:23:27.648194+0000","last_clean":"2024-03-21T10:23:27.648194+0000","last_became_active":"2024-03-21T10:21:57.860900+0000","last_became_peered":"2024-03-21T10:21:57.860900+0000","last_unstale":"2024-03-21T10:23:27.648194+0000","last_undegraded":"2024-03-21T10:23:27.648194+0000","last_fullsized":"2024-03-21T10:23:27.648194+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:21:56.788908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:21:56.788908+0000","last_clean_scrub_stamp":"2024-03-21T10:21:56.788908+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T13:13:04.937120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"95'9","reported_seq":39,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T10:23:56.016553+0000","last_change":"2024-03-21T10:21:58.858815+0000","last_active":"2024-03-21T10:23:56.016553+0000","last_peered":"2024-03-21T10:23:56.016553+0000","last_clean":"2024-03-21T10:23:56.016553+0000","last_became_active":"2024-03-21T10:21:58.856483+0000","last_became_peered":"2024-03-21T10:21:58.856483+0000","last_unstale":"2024-03-21T10:23:56.016553+0000","last_undegraded":"2024-03-21T10:23:56.016553+0000","last_fullsized":"2024-03-21T10:23:56.016553+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_clean_scrub_stamp":"2024-03-21T10:21:57.813712+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T15:42:38.208093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":23,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T10:23:27.836558+0000","last_change":"2024-03-21T10:22:03.319719+0000","last_active":"2024-03-21T10:23:27.836558+0000","last_peered":"2024-03-21T10:23:27.836558+0000","last_clean":"2024-03-21T10:23:27.836558+0000","last_became_active":"2024-03-21T10:22:03.319234+0000","last_became_peered":"2024-03-21T10:22:03.319234+0000","last_unstale":"2024-03-21T10:23:27.836558+0000","last_undegraded":"2024-03-21T10:23:27.836558+0000","last_fullsized":"2024-03-21T10:23:27.836558+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:22:02.269614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:22:02.269614+0000","last_clean_scrub_stamp":"2024-03-21T10:22:02.269614+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T14:11:21.609594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"93'5","reported_seq":38,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T10:23:27.751203+0000","last_change":"2024-03-21T10:21:57.862744+0000","last_active":"2024-03-21T10:23:27.751203+0000","last_peered":"2024-03-21T10:23:27.751203+0000","last_clean":"2024-03-21T10:23:27.751203+0000","last_became_active":"2024-03-21T10:21:57.862352+0000","last_became_peered":"2024-03-21T10:21:57.862352+0000","last_unstale":"2024-03-21T10:23:27.751203+0000","last_undegraded":"2024-03-21T10:23:27.751203+0000","last_fullsized":"2024-03-21T10:23:27.751203+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:21:56.788908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:21:56.788908+0000","last_clean_scrub_stamp":"2024-03-21T10:21:56.788908+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T20:19:50.138904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"95'15","reported_seq":45,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T10:23:56.046387+0000","last_change":"2024-03-21T10:21:58.863514+0000","last_active":"2024-03-21T10:23:56.046387+0000","last_peered":"2024-03-21T10:23:56.046387+0000","last_clean":"2024-03-21T10:23:56.046387+0000","last_became_active":"2024-03-21T10:21:58.861981+0000","last_became_peered":"2024-03-21T10:21:58.861981+0000","last_unstale":"2024-03-21T10:23:56.046387+0000","last_undegraded":"2024-03-21T10:23:56.046387+0000","last_fullsized":"2024-03-21T10:23:56.046387+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_clean_scrub_stamp":"2024-03-21T10:21:57.813712+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T15:00:33.813456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":23,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T10:23:27.642326+0000","last_change":"2024-03-21T10:22:03.309844+0000","last_active":"2024-03-21T10:23:27.642326+0000","last_peered":"2024-03-21T10:23:27.642326+0000","last_clean":"2024-03-21T10:23:27.642326+0000","last_became_active":"2024-03-21T10:22:03.309448+0000","last_became_peered":"2024-03-21T10:22:03.309448+0000","last_unstale":"2024-03-21T10:23:27.642326+0000","last_undegraded":"2024-03-21T10:23:27.642326+0000","last_fullsized":"2024-03-21T10:23:27.642326+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:22:02.269614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:22:02.269614+0000","last_clean_scrub_stamp":"2024-03-21T10:22:02.269614+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T18:38:51.167166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.11","version":"95'8","reported_seq":201,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T10:24:57.986043+0000","last_change":"2024-03-21T10:22:27.223337+0000","last_active":"2024-03-21T10:24:57.986043+0000","last_peered":"2024-03-21T10:24:57.986043+0000","last_clean":"2024-03-21T10:24:57.986043+0000","last_became_active":"2024-03-21T10:22:27.222999+0000","last_became_peered":"2024-03-21T10:22:27.222999+0000","last_unstale":"2024-03-21T10:24:57.986043+0000","last_undegraded":"2024-03-21T10:24:57.986043+0000","last_fullsized":"2024-03-21T10:24:57.986043+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:21:56.788908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:21:56.788908+0000","last_clean_scrub_stamp":"2024-03-21T10:21:56.788908+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T20:40:45.780696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1613,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":156,"num_read_kb":5,"num_write":8,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,10],"acting":[4,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"95'6","reported_seq":36,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T10:23:56.037159+0000","last_change":"2024-03-21T10:21:58.862225+0000","last_active":"2024-03-21T10:23:56.037159+0000","last_peered":"2024-03-21T10:23:56.037159+0000","last_clean":"2024-03-21T10:23:56.037159+0000","last_became_active":"2024-03-21T10:21:58.861816+0000","last_became_peered":"2024-03-21T10:21:58.861816+0000","last_unstale":"2024-03-21T10:23:56.037159+0000","last_undegraded":"2024-03-21T10:23:56.037159+0000","last_fullsized":"2024-03-21T10:23:56.037159+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_clean_scrub_stamp":"2024-03-21T10:21:57.813712+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T18:36:13.420867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":23,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T10:23:27.742956+0000","last_change":"2024-03-21T10:22:03.315306+0000","last_active":"2024-03-21T10:23:27.742956+0000","last_peered":"2024-03-21T10:23:27.742956+0000","last_clean":"2024-03-21T10:23:27.742956+0000","last_became_active":"2024-03-21T10:22:03.314839+0000","last_became_peered":"2024-03-21T10:22:03.314839+0000","last_unstale":"2024-03-21T10:23:27.742956+0000","last_undegraded":"2024-03-21T10:23:27.742956+0000","last_fullsized":"2024-03-21T10:23:27.742956+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:22:02.269614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:22:02.269614+0000","last_clean_scrub_stamp":"2024-03-21T10:22:02.269614+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T15:47:45.690434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"93'17","reported_seq":50,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T10:23:27.644536+0000","last_change":"2024-03-21T10:21:57.870998+0000","last_active":"2024-03-21T10:23:27.644536+0000","last_peered":"2024-03-21T10:23:27.644536+0000","last_clean":"2024-03-21T10:23:27.644536+0000","last_became_active":"2024-03-21T10:21:57.870605+0000","last_became_peered":"2024-03-21T10:21:57.870605+0000","last_unstale":"2024-03-21T10:23:27.644536+0000","last_undegraded":"2024-03-21T10:23:27.644536+0000","last_fullsized":"2024-03-21T10:23:27.644536+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:21:56.788908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:21:56.788908+0000","last_clean_scrub_stamp":"2024-03-21T10:21:56.788908+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T13:33:54.775122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"95'6","reported_seq":36,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T10:23:56.030441+0000","last_change":"2024-03-21T10:21:58.858675+0000","last_active":"2024-03-21T10:23:56.030441+0000","last_peered":"2024-03-21T10:23:56.030441+0000","last_clean":"2024-03-21T10:23:56.030441+0000","last_became_active":"2024-03-21T10:21:58.856907+0000","last_became_peered":"2024-03-21T10:21:58.856907+0000","last_unstale":"2024-03-21T10:23:56.030441+0000","last_undegraded":"2024-03-21T10:23:56.030441+0000","last_fullsized":"2024-03-21T10:23:56.030441+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_clean_scrub_stamp":"2024-03-21T10:21:57.813712+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T18:17:31.208728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":23,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T10:23:27.750746+0000","last_change":"2024-03-21T10:22:03.318257+0000","last_active":"2024-03-21T10:23:27.750746+0000","last_peered":"2024-03-21T10:23:27.750746+0000","last_clean":"2024-03-21T10:23:27.750746+0000","last_became_active":"2024-03-21T10:22:03.317622+0000","last_became_peered":"2024-03-21T10:22:03.317622+0000","last_unstale":"2024-03-21T10:23:27.750746+0000","last_undegraded":"2024-03-21T10:23:27.750746+0000","last_fullsized":"2024-03-21T10:23:27.750746+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:22:02.269614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:22:02.269614+0000","last_clean_scrub_stamp":"2024-03-21T10:22:02.269614+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T17:15:50.272193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"95'3","reported_seq":36,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T10:24:12.259722+0000","last_change":"2024-03-21T10:21:57.868014+0000","last_active":"2024-03-21T10:24:12.259722+0000","last_peered":"2024-03-21T10:24:12.259722+0000","last_clean":"2024-03-21T10:24:12.259722+0000","last_became_active":"2024-03-21T10:21:57.867189+0000","last_became_peered":"2024-03-21T10:21:57.867189+0000","last_unstale":"2024-03-21T10:24:12.259722+0000","last_undegraded":"2024-03-21T10:24:12.259722+0000","last_fullsized":"2024-03-21T10:24:12.259722+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:21:56.788908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:21:56.788908+0000","last_clean_scrub_stamp":"2024-03-21T10:21:56.788908+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T22:02:48.704932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"95'5","reported_seq":35,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T10:23:56.004037+0000","last_change":"2024-03-21T10:21:58.862205+0000","last_active":"2024-03-21T10:23:56.004037+0000","last_peered":"2024-03-21T10:23:56.004037+0000","last_clean":"2024-03-21T10:23:56.004037+0000","last_became_active":"2024-03-21T10:21:58.861186+0000","last_became_peered":"2024-03-21T10:21:58.861186+0000","last_unstale":"2024-03-21T10:23:56.004037+0000","last_undegraded":"2024-03-21T10:23:56.004037+0000","last_fullsized":"2024-03-21T10:23:56.004037+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_clean_scrub_stamp":"2024-03-21T10:21:57.813712+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T19:44:49.719845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":23,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T10:23:27.643556+0000","last_change":"2024-03-21T10:22:03.636064+0000","last_active":"2024-03-21T10:23:27.643556+0000","last_peered":"2024-03-21T10:23:27.643556+0000","last_clean":"2024-03-21T10:23:27.643556+0000","last_became_active":"2024-03-21T10:22:03.633596+0000","last_became_peered":"2024-03-21T10:22:03.633596+0000","last_unstale":"2024-03-21T10:23:27.643556+0000","last_undegraded":"2024-03-21T10:23:27.643556+0000","last_fullsized":"2024-03-21T10:23:27.643556+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:22:02.269614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:22:02.269614+0000","last_clean_scrub_stamp":"2024-03-21T10:22:02.269614+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T19:32:31.844066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"93'1","reported_seq":33,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T10:23:27.642192+0000","last_change":"2024-03-21T10:21:57.851955+0000","last_active":"2024-03-21T10:23:27.642192+0000","last_peered":"2024-03-21T10:23:27.642192+0000","last_clean":"2024-03-21T10:23:27.642192+0000","last_became_active":"2024-03-21T10:21:57.851571+0000","last_became_peered":"2024-03-21T10:21:57.851571+0000","last_unstale":"2024-03-21T10:23:27.642192+0000","last_undegraded":"2024-03-21T10:23:27.642192+0000","last_fullsized":"2024-03-21T10:23:27.642192+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:21:56.788908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:21:56.788908+0000","last_clean_scrub_stamp":"2024-03-21T10:21:56.788908+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T17:23:40.441895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"95'4","reported_seq":34,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T10:23:56.015141+0000","last_change":"2024-03-21T10:21:58.856393+0000","last_active":"2024-03-21T10:23:56.015141+0000","last_peered":"2024-03-21T10:23:56.015141+0000","last_clean":"2024-03-21T10:23:56.015141+0000","last_became_active":"2024-03-21T10:21:58.855139+0000","last_became_peered":"2024-03-21T10:21:58.855139+0000","last_unstale":"2024-03-21T10:23:56.015141+0000","last_undegraded":"2024-03-21T10:23:56.015141+0000","last_fullsized":"2024-03-21T10:23:56.015141+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_clean_scrub_stamp":"2024-03-21T10:21:57.813712+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T14:24:55.663576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":23,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T10:23:27.742416+0000","last_change":"2024-03-21T10:22:03.315586+0000","last_active":"2024-03-21T10:23:27.742416+0000","last_peered":"2024-03-21T10:23:27.742416+0000","last_clean":"2024-03-21T10:23:27.742416+0000","last_became_active":"2024-03-21T10:22:03.315198+0000","last_became_peered":"2024-03-21T10:22:03.315198+0000","last_unstale":"2024-03-21T10:23:27.742416+0000","last_undegraded":"2024-03-21T10:23:27.742416+0000","last_fullsized":"2024-03-21T10:23:27.742416+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:22:02.269614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:22:02.269614+0000","last_clean_scrub_stamp":"2024-03-21T10:22:02.269614+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T14:25:13.534311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"95'6","reported_seq":193,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T10:25:00.943126+0000","last_change":"2024-03-21T10:21:57.862094+0000","last_active":"2024-03-21T10:25:00.943126+0000","last_peered":"2024-03-21T10:25:00.943126+0000","last_clean":"2024-03-21T10:25:00.943126+0000","last_became_active":"2024-03-21T10:21:57.860307+0000","last_became_peered":"2024-03-21T10:21:57.860307+0000","last_unstale":"2024-03-21T10:25:00.943126+0000","last_undegraded":"2024-03-21T10:25:00.943126+0000","last_fullsized":"2024-03-21T10:25:00.943126+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:21:56.788908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:21:56.788908+0000","last_clean_scrub_stamp":"2024-03-21T10:21:56.788908+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T12:58:50.701218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1081,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":155,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"95'7","reported_seq":37,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T10:23:56.024789+0000","last_change":"2024-03-21T10:21:58.870880+0000","last_active":"2024-03-21T10:23:56.024789+0000","last_peered":"2024-03-21T10:23:56.024789+0000","last_clean":"2024-03-21T10:23:56.024789+0000","last_became_active":"2024-03-21T10:21:58.869882+0000","last_became_peered":"2024-03-21T10:21:58.869882+0000","last_unstale":"2024-03-21T10:23:56.024789+0000","last_undegraded":"2024-03-21T10:23:56.024789+0000","last_fullsized":"2024-03-21T10:23:56.024789+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_clean_scrub_stamp":"2024-03-21T10:21:57.813712+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T18:01:42.613732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":23,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T10:23:27.742882+0000","last_change":"2024-03-21T10:22:03.634425+0000","last_active":"2024-03-21T10:23:27.742882+0000","last_peered":"2024-03-21T10:23:27.742882+0000","last_clean":"2024-03-21T10:23:27.742882+0000","last_became_active":"2024-03-21T10:22:03.632464+0000","last_became_peered":"2024-03-21T10:22:03.632464+0000","last_unstale":"2024-03-21T10:23:27.742882+0000","last_undegraded":"2024-03-21T10:23:27.742882+0000","last_fullsized":"2024-03-21T10:23:27.742882+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:22:02.269614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:22:02.269614+0000","last_clean_scrub_stamp":"2024-03-21T10:22:02.269614+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T16:03:28.270757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"95'8","reported_seq":19,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T10:23:56.006280+0000","last_change":"2024-03-21T10:23:27.649397+0000","last_active":"2024-03-21T10:23:56.006280+0000","last_peered":"2024-03-21T10:23:56.006280+0000","last_clean":"2024-03-21T10:23:56.006280+0000","last_became_active":"2024-03-21T10:23:27.649148+0000","last_became_peered":"2024-03-21T10:23:27.649148+0000","last_unstale":"2024-03-21T10:23:56.006280+0000","last_undegraded":"2024-03-21T10:23:56.006280+0000","last_fullsized":"2024-03-21T10:23:56.006280+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_clean_scrub_stamp":"2024-03-21T10:21:57.813712+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T19:28:03.830551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,8],"acting":[9,2,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.1","version":"95'7","reported_seq":195,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T10:25:01.994641+0000","last_change":"2024-03-21T10:21:57.856111+0000","last_active":"2024-03-21T10:25:01.994641+0000","last_peered":"2024-03-21T10:25:01.994641+0000","last_clean":"2024-03-21T10:25:01.994641+0000","last_became_active":"2024-03-21T10:21:57.855584+0000","last_became_peered":"2024-03-21T10:21:57.855584+0000","last_unstale":"2024-03-21T10:25:01.994641+0000","last_undegraded":"2024-03-21T10:25:01.994641+0000","last_fullsized":"2024-03-21T10:25:01.994641+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:21:56.788908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:21:56.788908+0000","last_clean_scrub_stamp":"2024-03-21T10:21:56.788908+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T17:53:48.208554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":156,"num_read_kb":156,"num_write":7,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"93'1","reported_seq":24,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T10:23:27.645180+0000","last_change":"2024-03-21T10:22:03.318831+0000","last_active":"2024-03-21T10:23:27.645180+0000","last_peered":"2024-03-21T10:23:27.645180+0000","last_clean":"2024-03-21T10:23:27.645180+0000","last_became_active":"2024-03-21T10:22:03.318404+0000","last_became_peered":"2024-03-21T10:22:03.318404+0000","last_unstale":"2024-03-21T10:23:27.645180+0000","last_undegraded":"2024-03-21T10:23:27.645180+0000","last_fullsized":"2024-03-21T10:23:27.645180+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:22:02.269614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:22:02.269614+0000","last_clean_scrub_stamp":"2024-03-21T10:22:02.269614+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T12:06:52.043286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"95'8","reported_seq":38,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T10:23:56.038328+0000","last_change":"2024-03-21T10:21:58.856469+0000","last_active":"2024-03-21T10:23:56.038328+0000","last_peered":"2024-03-21T10:23:56.038328+0000","last_clean":"2024-03-21T10:23:56.038328+0000","last_became_active":"2024-03-21T10:21:58.856032+0000","last_became_peered":"2024-03-21T10:21:58.856032+0000","last_unstale":"2024-03-21T10:23:56.038328+0000","last_undegraded":"2024-03-21T10:23:56.038328+0000","last_fullsized":"2024-03-21T10:23:56.038328+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_clean_scrub_stamp":"2024-03-21T10:21:57.813712+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T21:19:59.302418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"93'6","reported_seq":41,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T10:23:27.640569+0000","last_change":"2024-03-21T10:21:57.854210+0000","last_active":"2024-03-21T10:23:27.640569+0000","last_peered":"2024-03-21T10:23:27.640569+0000","last_clean":"2024-03-21T10:23:27.640569+0000","last_became_active":"2024-03-21T10:21:57.853787+0000","last_became_peered":"2024-03-21T10:21:57.853787+0000","last_unstale":"2024-03-21T10:23:27.640569+0000","last_undegraded":"2024-03-21T10:23:27.640569+0000","last_fullsized":"2024-03-21T10:23:27.640569+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:21:56.788908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:21:56.788908+0000","last_clean_scrub_stamp":"2024-03-21T10:21:56.788908+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T12:10:42.926436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"23'137","reported_seq":367,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T10:23:27.643665+0000","last_change":"2024-03-21T10:20:21.585339+0000","last_active":"2024-03-21T10:23:27.643665+0000","last_peered":"2024-03-21T10:23:27.643665+0000","last_clean":"2024-03-21T10:23:27.643665+0000","last_became_active":"2024-03-21T10:20:21.583999+0000","last_became_peered":"2024-03-21T10:20:21.583999+0000","last_unstale":"2024-03-21T10:23:27.643665+0000","last_undegraded":"2024-03-21T10:23:27.643665+0000","last_fullsized":"2024-03-21T10:23:27.643665+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:15:40.696168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:15:40.696168+0000","last_clean_scrub_stamp":"2024-03-21T10:15:40.696168+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T15:31:54.603722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":23,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T10:23:27.752359+0000","last_change":"2024-03-21T10:22:03.634970+0000","last_active":"2024-03-21T10:23:27.752359+0000","last_peered":"2024-03-21T10:23:27.752359+0000","last_clean":"2024-03-21T10:23:27.752359+0000","last_became_active":"2024-03-21T10:22:03.633603+0000","last_became_peered":"2024-03-21T10:22:03.633603+0000","last_unstale":"2024-03-21T10:23:27.752359+0000","last_undegraded":"2024-03-21T10:23:27.752359+0000","last_fullsized":"2024-03-21T10:23:27.752359+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:22:02.269614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:22:02.269614+0000","last_clean_scrub_stamp":"2024-03-21T10:22:02.269614+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T12:13:02.041661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"95'5","reported_seq":35,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T10:23:56.002047+0000","last_change":"2024-03-21T10:21:58.861159+0000","last_active":"2024-03-21T10:23:56.002047+0000","last_peered":"2024-03-21T10:23:56.002047+0000","last_clean":"2024-03-21T10:23:56.002047+0000","last_became_active":"2024-03-21T10:21:58.860690+0000","last_became_peered":"2024-03-21T10:21:58.860690+0000","last_unstale":"2024-03-21T10:23:56.002047+0000","last_undegraded":"2024-03-21T10:23:56.002047+0000","last_fullsized":"2024-03-21T10:23:56.002047+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_clean_scrub_stamp":"2024-03-21T10:21:57.813712+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T16:21:53.632997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"93'4","reported_seq":36,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T10:23:27.643848+0000","last_change":"2024-03-21T10:21:57.863581+0000","last_active":"2024-03-21T10:23:27.643848+0000","last_peered":"2024-03-21T10:23:27.643848+0000","last_clean":"2024-03-21T10:23:27.643848+0000","last_became_active":"2024-03-21T10:21:57.863269+0000","last_became_peered":"2024-03-21T10:21:57.863269+0000","last_unstale":"2024-03-21T10:23:27.643848+0000","last_undegraded":"2024-03-21T10:23:27.643848+0000","last_fullsized":"2024-03-21T10:23:27.643848+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:21:56.788908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:21:56.788908+0000","last_clean_scrub_stamp":"2024-03-21T10:21:56.788908+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T14:03:59.930570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":23,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T10:23:27.835814+0000","last_change":"2024-03-21T10:22:03.313515+0000","last_active":"2024-03-21T10:23:27.835814+0000","last_peered":"2024-03-21T10:23:27.835814+0000","last_clean":"2024-03-21T10:23:27.835814+0000","last_became_active":"2024-03-21T10:22:03.311862+0000","last_became_peered":"2024-03-21T10:22:03.311862+0000","last_unstale":"2024-03-21T10:23:27.835814+0000","last_undegraded":"2024-03-21T10:23:27.835814+0000","last_fullsized":"2024-03-21T10:23:27.835814+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:22:02.269614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:22:02.269614+0000","last_clean_scrub_stamp":"2024-03-21T10:22:02.269614+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T16:47:50.761205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"93'4","reported_seq":37,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T10:23:27.646442+0000","last_change":"2024-03-21T10:21:57.870913+0000","last_active":"2024-03-21T10:23:27.646442+0000","last_peered":"2024-03-21T10:23:27.646442+0000","last_clean":"2024-03-21T10:23:27.646442+0000","last_became_active":"2024-03-21T10:21:57.870578+0000","last_became_peered":"2024-03-21T10:21:57.870578+0000","last_unstale":"2024-03-21T10:23:27.646442+0000","last_undegraded":"2024-03-21T10:23:27.646442+0000","last_fullsized":"2024-03-21T10:23:27.646442+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:21:56.788908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:21:56.788908+0000","last_clean_scrub_stamp":"2024-03-21T10:21:56.788908+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T11:46:48.621501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"95'9","reported_seq":39,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T10:23:56.007067+0000","last_change":"2024-03-21T10:21:58.852247+0000","last_active":"2024-03-21T10:23:56.007067+0000","last_peered":"2024-03-21T10:23:56.007067+0000","last_clean":"2024-03-21T10:23:56.007067+0000","last_became_active":"2024-03-21T10:21:58.851976+0000","last_became_peered":"2024-03-21T10:21:58.851976+0000","last_unstale":"2024-03-21T10:23:56.007067+0000","last_undegraded":"2024-03-21T10:23:56.007067+0000","last_fullsized":"2024-03-21T10:23:56.007067+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_clean_scrub_stamp":"2024-03-21T10:21:57.813712+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T14:27:11.383741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":23,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T10:23:27.638289+0000","last_change":"2024-03-21T10:22:03.637671+0000","last_active":"2024-03-21T10:23:27.638289+0000","last_peered":"2024-03-21T10:23:27.638289+0000","last_clean":"2024-03-21T10:23:27.638289+0000","last_became_active":"2024-03-21T10:22:03.634265+0000","last_became_peered":"2024-03-21T10:22:03.634265+0000","last_unstale":"2024-03-21T10:23:27.638289+0000","last_undegraded":"2024-03-21T10:23:27.638289+0000","last_fullsized":"2024-03-21T10:23:27.638289+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:22:02.269614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:22:02.269614+0000","last_clean_scrub_stamp":"2024-03-21T10:22:02.269614+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T15:38:02.214178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"95'16","reported_seq":48,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T10:24:57.291140+0000","last_change":"2024-03-21T10:21:57.862027+0000","last_active":"2024-03-21T10:24:57.291140+0000","last_peered":"2024-03-21T10:24:57.291140+0000","last_clean":"2024-03-21T10:24:57.291140+0000","last_became_active":"2024-03-21T10:21:57.860097+0000","last_became_peered":"2024-03-21T10:21:57.860097+0000","last_unstale":"2024-03-21T10:24:57.291140+0000","last_undegraded":"2024-03-21T10:24:57.291140+0000","last_fullsized":"2024-03-21T10:24:57.291140+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:21:56.788908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:21:56.788908+0000","last_clean_scrub_stamp":"2024-03-21T10:21:56.788908+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T13:46:19.562452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":28,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"95'5","reported_seq":35,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T10:23:56.040770+0000","last_change":"2024-03-21T10:21:58.856082+0000","last_active":"2024-03-21T10:23:56.040770+0000","last_peered":"2024-03-21T10:23:56.040770+0000","last_clean":"2024-03-21T10:23:56.040770+0000","last_became_active":"2024-03-21T10:21:58.854172+0000","last_became_peered":"2024-03-21T10:21:58.854172+0000","last_unstale":"2024-03-21T10:23:56.040770+0000","last_undegraded":"2024-03-21T10:23:56.040770+0000","last_fullsized":"2024-03-21T10:23:56.040770+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_clean_scrub_stamp":"2024-03-21T10:21:57.813712+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T14:59:33.995716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":23,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T10:23:27.647010+0000","last_change":"2024-03-21T10:22:03.310205+0000","last_active":"2024-03-21T10:23:27.647010+0000","last_peered":"2024-03-21T10:23:27.647010+0000","last_clean":"2024-03-21T10:23:27.647010+0000","last_became_active":"2024-03-21T10:22:03.309039+0000","last_became_peered":"2024-03-21T10:22:03.309039+0000","last_unstale":"2024-03-21T10:23:27.647010+0000","last_undegraded":"2024-03-21T10:23:27.647010+0000","last_fullsized":"2024-03-21T10:23:27.647010+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:22:02.269614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:22:02.269614+0000","last_clean_scrub_stamp":"2024-03-21T10:22:02.269614+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T11:16:46.075260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"93'7","reported_seq":40,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T10:23:27.641787+0000","last_change":"2024-03-21T10:21:57.857890+0000","last_active":"2024-03-21T10:23:27.641787+0000","last_peered":"2024-03-21T10:23:27.641787+0000","last_clean":"2024-03-21T10:23:27.641787+0000","last_became_active":"2024-03-21T10:21:57.857492+0000","last_became_peered":"2024-03-21T10:21:57.857492+0000","last_unstale":"2024-03-21T10:23:27.641787+0000","last_undegraded":"2024-03-21T10:23:27.641787+0000","last_fullsized":"2024-03-21T10:23:27.641787+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:21:56.788908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:21:56.788908+0000","last_clean_scrub_stamp":"2024-03-21T10:21:56.788908+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T21:18:45.566843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"95'7","reported_seq":43,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T10:23:56.040975+0000","last_change":"2024-03-21T10:22:27.219339+0000","last_active":"2024-03-21T10:23:56.040975+0000","last_peered":"2024-03-21T10:23:56.040975+0000","last_clean":"2024-03-21T10:23:56.040975+0000","last_became_active":"2024-03-21T10:22:27.218604+0000","last_became_peered":"2024-03-21T10:22:27.218604+0000","last_unstale":"2024-03-21T10:23:56.040975+0000","last_undegraded":"2024-03-21T10:23:56.040975+0000","last_fullsized":"2024-03-21T10:23:56.040975+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_clean_scrub_stamp":"2024-03-21T10:21:57.813712+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T15:58:53.161767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,4],"acting":[0,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":23,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T10:23:27.641301+0000","last_change":"2024-03-21T10:22:03.633789+0000","last_active":"2024-03-21T10:23:27.641301+0000","last_peered":"2024-03-21T10:23:27.641301+0000","last_clean":"2024-03-21T10:23:27.641301+0000","last_became_active":"2024-03-21T10:22:03.632716+0000","last_became_peered":"2024-03-21T10:22:03.632716+0000","last_unstale":"2024-03-21T10:23:27.641301+0000","last_undegraded":"2024-03-21T10:23:27.641301+0000","last_fullsized":"2024-03-21T10:23:27.641301+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:22:02.269614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:22:02.269614+0000","last_clean_scrub_stamp":"2024-03-21T10:22:02.269614+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T14:08:47.301881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"91'2","reported_seq":34,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T10:23:27.833770+0000","last_change":"2024-03-21T10:21:57.862413+0000","last_active":"2024-03-21T10:23:27.833770+0000","last_peered":"2024-03-21T10:23:27.833770+0000","last_clean":"2024-03-21T10:23:27.833770+0000","last_became_active":"2024-03-21T10:21:57.862104+0000","last_became_peered":"2024-03-21T10:21:57.862104+0000","last_unstale":"2024-03-21T10:23:27.833770+0000","last_undegraded":"2024-03-21T10:23:27.833770+0000","last_fullsized":"2024-03-21T10:23:27.833770+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:21:56.788908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:21:56.788908+0000","last_clean_scrub_stamp":"2024-03-21T10:21:56.788908+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T17:49:21.729492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"95'2","reported_seq":32,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T10:23:56.014082+0000","last_change":"2024-03-21T10:21:58.864688+0000","last_active":"2024-03-21T10:23:56.014082+0000","last_peered":"2024-03-21T10:23:56.014082+0000","last_clean":"2024-03-21T10:23:56.014082+0000","last_became_active":"2024-03-21T10:21:58.862821+0000","last_became_peered":"2024-03-21T10:21:58.862821+0000","last_unstale":"2024-03-21T10:23:56.014082+0000","last_undegraded":"2024-03-21T10:23:56.014082+0000","last_fullsized":"2024-03-21T10:23:56.014082+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_clean_scrub_stamp":"2024-03-21T10:21:57.813712+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T11:17:09.650649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T10:23:27.756711+0000","last_change":"2024-03-21T10:23:27.756711+0000","last_active":"2024-03-21T10:23:27.756711+0000","last_peered":"2024-03-21T10:23:27.756711+0000","last_clean":"2024-03-21T10:23:27.756711+0000","last_became_active":"2024-03-21T10:23:27.754993+0000","last_became_peered":"2024-03-21T10:23:27.754993+0000","last_unstale":"2024-03-21T10:23:27.756711+0000","last_undegraded":"2024-03-21T10:23:27.756711+0000","last_fullsized":"2024-03-21T10:23:27.756711+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:22:02.269614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:22:02.269614+0000","last_clean_scrub_stamp":"2024-03-21T10:22:02.269614+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T11:22:51.851047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,2,5],"acting":[8,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"93'7","reported_seq":39,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T10:23:27.638625+0000","last_change":"2024-03-21T10:21:57.868033+0000","last_active":"2024-03-21T10:23:27.638625+0000","last_peered":"2024-03-21T10:23:27.638625+0000","last_clean":"2024-03-21T10:23:27.638625+0000","last_became_active":"2024-03-21T10:21:57.867279+0000","last_became_peered":"2024-03-21T10:21:57.867279+0000","last_unstale":"2024-03-21T10:23:27.638625+0000","last_undegraded":"2024-03-21T10:23:27.638625+0000","last_fullsized":"2024-03-21T10:23:27.638625+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:21:56.788908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:21:56.788908+0000","last_clean_scrub_stamp":"2024-03-21T10:21:56.788908+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T20:05:37.701443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"95'9","reported_seq":39,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T10:23:56.022588+0000","last_change":"2024-03-21T10:21:58.868455+0000","last_active":"2024-03-21T10:23:56.022588+0000","last_peered":"2024-03-21T10:23:56.022588+0000","last_clean":"2024-03-21T10:23:56.022588+0000","last_became_active":"2024-03-21T10:21:58.867786+0000","last_became_peered":"2024-03-21T10:21:58.867786+0000","last_unstale":"2024-03-21T10:23:56.022588+0000","last_undegraded":"2024-03-21T10:23:56.022588+0000","last_fullsized":"2024-03-21T10:23:56.022588+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_clean_scrub_stamp":"2024-03-21T10:21:57.813712+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T19:16:12.258470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"93'1","reported_seq":35,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T10:23:27.753821+0000","last_change":"2024-03-21T10:23:27.753222+0000","last_active":"2024-03-21T10:23:27.753821+0000","last_peered":"2024-03-21T10:23:27.753821+0000","last_clean":"2024-03-21T10:23:27.753821+0000","last_became_active":"2024-03-21T10:23:27.746251+0000","last_became_peered":"2024-03-21T10:23:27.746251+0000","last_unstale":"2024-03-21T10:23:27.753821+0000","last_undegraded":"2024-03-21T10:23:27.753821+0000","last_fullsized":"2024-03-21T10:23:27.753821+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:22:02.269614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:22:02.269614+0000","last_clean_scrub_stamp":"2024-03-21T10:22:02.269614+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T13:42:00.362336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":117,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,7,8],"acting":[4,10,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"95'5","reported_seq":37,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T10:24:57.202344+0000","last_change":"2024-03-21T10:21:57.862748+0000","last_active":"2024-03-21T10:24:57.202344+0000","last_peered":"2024-03-21T10:24:57.202344+0000","last_clean":"2024-03-21T10:24:57.202344+0000","last_became_active":"2024-03-21T10:21:57.862350+0000","last_became_peered":"2024-03-21T10:21:57.862350+0000","last_unstale":"2024-03-21T10:24:57.202344+0000","last_undegraded":"2024-03-21T10:24:57.202344+0000","last_fullsized":"2024-03-21T10:24:57.202344+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:21:56.788908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:21:56.788908+0000","last_clean_scrub_stamp":"2024-03-21T10:21:56.788908+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T17:08:47.301758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"95'8","reported_seq":38,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T10:23:56.026242+0000","last_change":"2024-03-21T10:21:58.866340+0000","last_active":"2024-03-21T10:23:56.026242+0000","last_peered":"2024-03-21T10:23:56.026242+0000","last_clean":"2024-03-21T10:23:56.026242+0000","last_became_active":"2024-03-21T10:21:58.865965+0000","last_became_peered":"2024-03-21T10:21:58.865965+0000","last_unstale":"2024-03-21T10:23:56.026242+0000","last_undegraded":"2024-03-21T10:23:56.026242+0000","last_fullsized":"2024-03-21T10:23:56.026242+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_clean_scrub_stamp":"2024-03-21T10:21:57.813712+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T10:42:23.854176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":23,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T10:23:27.640308+0000","last_change":"2024-03-21T10:22:03.634108+0000","last_active":"2024-03-21T10:23:27.640308+0000","last_peered":"2024-03-21T10:23:27.640308+0000","last_clean":"2024-03-21T10:23:27.640308+0000","last_became_active":"2024-03-21T10:22:03.633344+0000","last_became_peered":"2024-03-21T10:22:03.633344+0000","last_unstale":"2024-03-21T10:23:27.640308+0000","last_undegraded":"2024-03-21T10:23:27.640308+0000","last_fullsized":"2024-03-21T10:23:27.640308+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:22:02.269614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:22:02.269614+0000","last_clean_scrub_stamp":"2024-03-21T10:22:02.269614+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T21:00:59.893994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"91'4","reported_seq":37,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T10:23:27.644244+0000","last_change":"2024-03-21T10:21:57.871520+0000","last_active":"2024-03-21T10:23:27.644244+0000","last_peered":"2024-03-21T10:23:27.644244+0000","last_clean":"2024-03-21T10:23:27.644244+0000","last_became_active":"2024-03-21T10:21:57.871127+0000","last_became_peered":"2024-03-21T10:21:57.871127+0000","last_unstale":"2024-03-21T10:23:27.644244+0000","last_undegraded":"2024-03-21T10:23:27.644244+0000","last_fullsized":"2024-03-21T10:23:27.644244+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:21:56.788908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:21:56.788908+0000","last_clean_scrub_stamp":"2024-03-21T10:21:56.788908+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T20:43:39.537307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"95'5","reported_seq":35,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T10:23:56.044719+0000","last_change":"2024-03-21T10:21:58.861786+0000","last_active":"2024-03-21T10:23:56.044719+0000","last_peered":"2024-03-21T10:23:56.044719+0000","last_clean":"2024-03-21T10:23:56.044719+0000","last_became_active":"2024-03-21T10:21:58.861428+0000","last_became_peered":"2024-03-21T10:21:58.861428+0000","last_unstale":"2024-03-21T10:23:56.044719+0000","last_undegraded":"2024-03-21T10:23:56.044719+0000","last_fullsized":"2024-03-21T10:23:56.044719+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_clean_scrub_stamp":"2024-03-21T10:21:57.813712+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T15:56:30.256266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":23,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T10:23:27.833726+0000","last_change":"2024-03-21T10:22:03.305085+0000","last_active":"2024-03-21T10:23:27.833726+0000","last_peered":"2024-03-21T10:23:27.833726+0000","last_clean":"2024-03-21T10:23:27.833726+0000","last_became_active":"2024-03-21T10:22:03.304474+0000","last_became_peered":"2024-03-21T10:22:03.304474+0000","last_unstale":"2024-03-21T10:23:27.833726+0000","last_undegraded":"2024-03-21T10:23:27.833726+0000","last_fullsized":"2024-03-21T10:23:27.833726+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:22:02.269614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:22:02.269614+0000","last_clean_scrub_stamp":"2024-03-21T10:22:02.269614+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T18:28:10.480981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"95'6","reported_seq":196,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T10:25:01.999354+0000","last_change":"2024-03-21T10:21:57.860657+0000","last_active":"2024-03-21T10:25:01.999354+0000","last_peered":"2024-03-21T10:25:01.999354+0000","last_clean":"2024-03-21T10:25:01.999354+0000","last_became_active":"2024-03-21T10:21:57.860340+0000","last_became_peered":"2024-03-21T10:21:57.860340+0000","last_unstale":"2024-03-21T10:25:01.999354+0000","last_undegraded":"2024-03-21T10:25:01.999354+0000","last_fullsized":"2024-03-21T10:25:01.999354+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:21:56.788908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:21:56.788908+0000","last_clean_scrub_stamp":"2024-03-21T10:21:56.788908+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T21:58:08.706812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1663,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":158,"num_read_kb":3,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"95'7","reported_seq":37,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T10:23:56.048801+0000","last_change":"2024-03-21T10:21:58.874254+0000","last_active":"2024-03-21T10:23:56.048801+0000","last_peered":"2024-03-21T10:23:56.048801+0000","last_clean":"2024-03-21T10:23:56.048801+0000","last_became_active":"2024-03-21T10:21:58.869795+0000","last_became_peered":"2024-03-21T10:21:58.869795+0000","last_unstale":"2024-03-21T10:23:56.048801+0000","last_undegraded":"2024-03-21T10:23:56.048801+0000","last_fullsized":"2024-03-21T10:23:56.048801+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_clean_scrub_stamp":"2024-03-21T10:21:57.813712+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T19:51:28.296199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":23,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T10:23:27.834356+0000","last_change":"2024-03-21T10:22:03.306761+0000","last_active":"2024-03-21T10:23:27.834356+0000","last_peered":"2024-03-21T10:23:27.834356+0000","last_clean":"2024-03-21T10:23:27.834356+0000","last_became_active":"2024-03-21T10:22:03.305106+0000","last_became_peered":"2024-03-21T10:22:03.305106+0000","last_unstale":"2024-03-21T10:23:27.834356+0000","last_undegraded":"2024-03-21T10:23:27.834356+0000","last_fullsized":"2024-03-21T10:23:27.834356+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:22:02.269614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:22:02.269614+0000","last_clean_scrub_stamp":"2024-03-21T10:22:02.269614+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T21:13:00.846169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"93'3","reported_seq":36,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T10:23:27.752174+0000","last_change":"2024-03-21T10:21:57.868012+0000","last_active":"2024-03-21T10:23:27.752174+0000","last_peered":"2024-03-21T10:23:27.752174+0000","last_clean":"2024-03-21T10:23:27.752174+0000","last_became_active":"2024-03-21T10:21:57.867682+0000","last_became_peered":"2024-03-21T10:21:57.867682+0000","last_unstale":"2024-03-21T10:23:27.752174+0000","last_undegraded":"2024-03-21T10:23:27.752174+0000","last_fullsized":"2024-03-21T10:23:27.752174+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:21:56.788908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:21:56.788908+0000","last_clean_scrub_stamp":"2024-03-21T10:21:56.788908+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T16:59:56.061167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"95'6","reported_seq":36,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T10:23:56.033933+0000","last_change":"2024-03-21T10:21:58.864949+0000","last_active":"2024-03-21T10:23:56.033933+0000","last_peered":"2024-03-21T10:23:56.033933+0000","last_clean":"2024-03-21T10:23:56.033933+0000","last_became_active":"2024-03-21T10:21:58.859796+0000","last_became_peered":"2024-03-21T10:21:58.859796+0000","last_unstale":"2024-03-21T10:23:56.033933+0000","last_undegraded":"2024-03-21T10:23:56.033933+0000","last_fullsized":"2024-03-21T10:23:56.033933+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_clean_scrub_stamp":"2024-03-21T10:21:57.813712+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T18:02:51.873103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":23,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T10:23:27.750167+0000","last_change":"2024-03-21T10:22:03.308451+0000","last_active":"2024-03-21T10:23:27.750167+0000","last_peered":"2024-03-21T10:23:27.750167+0000","last_clean":"2024-03-21T10:23:27.750167+0000","last_became_active":"2024-03-21T10:22:03.307986+0000","last_became_peered":"2024-03-21T10:22:03.307986+0000","last_unstale":"2024-03-21T10:23:27.750167+0000","last_undegraded":"2024-03-21T10:23:27.750167+0000","last_fullsized":"2024-03-21T10:23:27.750167+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:22:02.269614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:22:02.269614+0000","last_clean_scrub_stamp":"2024-03-21T10:22:02.269614+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T21:08:42.241657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"93'5","reported_seq":37,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T10:23:27.742941+0000","last_change":"2024-03-21T10:21:57.851717+0000","last_active":"2024-03-21T10:23:27.742941+0000","last_peered":"2024-03-21T10:23:27.742941+0000","last_clean":"2024-03-21T10:23:27.742941+0000","last_became_active":"2024-03-21T10:21:57.849909+0000","last_became_peered":"2024-03-21T10:21:57.849909+0000","last_unstale":"2024-03-21T10:23:27.742941+0000","last_undegraded":"2024-03-21T10:23:27.742941+0000","last_fullsized":"2024-03-21T10:23:27.742941+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:21:56.788908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:21:56.788908+0000","last_clean_scrub_stamp":"2024-03-21T10:21:56.788908+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T11:41:37.611400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"95'9","reported_seq":39,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T10:23:56.032325+0000","last_change":"2024-03-21T10:21:58.860892+0000","last_active":"2024-03-21T10:23:56.032325+0000","last_peered":"2024-03-21T10:23:56.032325+0000","last_clean":"2024-03-21T10:23:56.032325+0000","last_became_active":"2024-03-21T10:21:58.860403+0000","last_became_peered":"2024-03-21T10:21:58.860403+0000","last_unstale":"2024-03-21T10:23:56.032325+0000","last_undegraded":"2024-03-21T10:23:56.032325+0000","last_fullsized":"2024-03-21T10:23:56.032325+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_clean_scrub_stamp":"2024-03-21T10:21:57.813712+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T15:15:23.303864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"95'9","reported_seq":39,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T10:23:56.010246+0000","last_change":"2024-03-21T10:21:58.869275+0000","last_active":"2024-03-21T10:23:56.010246+0000","last_peered":"2024-03-21T10:23:56.010246+0000","last_clean":"2024-03-21T10:23:56.010246+0000","last_became_active":"2024-03-21T10:21:58.868834+0000","last_became_peered":"2024-03-21T10:21:58.868834+0000","last_unstale":"2024-03-21T10:23:56.010246+0000","last_undegraded":"2024-03-21T10:23:56.010246+0000","last_fullsized":"2024-03-21T10:23:56.010246+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_clean_scrub_stamp":"2024-03-21T10:21:57.813712+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T14:01:11.074702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"95'12","reported_seq":42,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T10:23:56.002992+0000","last_change":"2024-03-21T10:21:58.851658+0000","last_active":"2024-03-21T10:23:56.002992+0000","last_peered":"2024-03-21T10:23:56.002992+0000","last_clean":"2024-03-21T10:23:56.002992+0000","last_became_active":"2024-03-21T10:21:58.851295+0000","last_became_peered":"2024-03-21T10:21:58.851295+0000","last_unstale":"2024-03-21T10:23:56.002992+0000","last_undegraded":"2024-03-21T10:23:56.002992+0000","last_fullsized":"2024-03-21T10:23:56.002992+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_clean_scrub_stamp":"2024-03-21T10:21:57.813712+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T13:15:32.107783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"95'8","reported_seq":38,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T10:23:56.026300+0000","last_change":"2024-03-21T10:21:58.859275+0000","last_active":"2024-03-21T10:23:56.026300+0000","last_peered":"2024-03-21T10:23:56.026300+0000","last_clean":"2024-03-21T10:23:56.026300+0000","last_became_active":"2024-03-21T10:21:58.857054+0000","last_became_peered":"2024-03-21T10:21:58.857054+0000","last_unstale":"2024-03-21T10:23:56.026300+0000","last_undegraded":"2024-03-21T10:23:56.026300+0000","last_fullsized":"2024-03-21T10:23:56.026300+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_clean_scrub_stamp":"2024-03-21T10:21:57.813712+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T20:05:13.819966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"95'7","reported_seq":37,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T10:23:56.022332+0000","last_change":"2024-03-21T10:21:58.869025+0000","last_active":"2024-03-21T10:23:56.022332+0000","last_peered":"2024-03-21T10:23:56.022332+0000","last_clean":"2024-03-21T10:23:56.022332+0000","last_became_active":"2024-03-21T10:21:58.868240+0000","last_became_peered":"2024-03-21T10:21:58.868240+0000","last_unstale":"2024-03-21T10:23:56.022332+0000","last_undegraded":"2024-03-21T10:23:56.022332+0000","last_fullsized":"2024-03-21T10:23:56.022332+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_clean_scrub_stamp":"2024-03-21T10:21:57.813712+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T21:56:45.642916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"95'4","reported_seq":34,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T10:23:56.021712+0000","last_change":"2024-03-21T10:21:58.865024+0000","last_active":"2024-03-21T10:23:56.021712+0000","last_peered":"2024-03-21T10:23:56.021712+0000","last_clean":"2024-03-21T10:23:56.021712+0000","last_became_active":"2024-03-21T10:21:58.864649+0000","last_became_peered":"2024-03-21T10:21:58.864649+0000","last_unstale":"2024-03-21T10:23:56.021712+0000","last_undegraded":"2024-03-21T10:23:56.021712+0000","last_fullsized":"2024-03-21T10:23:56.021712+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_clean_scrub_stamp":"2024-03-21T10:21:57.813712+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T22:09:03.561283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"95'6","reported_seq":36,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T10:23:56.004821+0000","last_change":"2024-03-21T10:21:58.865206+0000","last_active":"2024-03-21T10:23:56.004821+0000","last_peered":"2024-03-21T10:23:56.004821+0000","last_clean":"2024-03-21T10:23:56.004821+0000","last_became_active":"2024-03-21T10:21:58.864669+0000","last_became_peered":"2024-03-21T10:21:58.864669+0000","last_unstale":"2024-03-21T10:23:56.004821+0000","last_undegraded":"2024-03-21T10:23:56.004821+0000","last_fullsized":"2024-03-21T10:23:56.004821+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_clean_scrub_stamp":"2024-03-21T10:21:57.813712+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T13:57:45.668192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"95'15","reported_seq":45,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T10:23:56.044533+0000","last_change":"2024-03-21T10:21:58.862664+0000","last_active":"2024-03-21T10:23:56.044533+0000","last_peered":"2024-03-21T10:23:56.044533+0000","last_clean":"2024-03-21T10:23:56.044533+0000","last_became_active":"2024-03-21T10:21:58.862144+0000","last_became_peered":"2024-03-21T10:21:58.862144+0000","last_unstale":"2024-03-21T10:23:56.044533+0000","last_undegraded":"2024-03-21T10:23:56.044533+0000","last_fullsized":"2024-03-21T10:23:56.044533+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_clean_scrub_stamp":"2024-03-21T10:21:57.813712+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T11:10:10.225102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"95'2","reported_seq":32,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T10:23:56.021964+0000","last_change":"2024-03-21T10:21:58.851000+0000","last_active":"2024-03-21T10:23:56.021964+0000","last_peered":"2024-03-21T10:23:56.021964+0000","last_clean":"2024-03-21T10:23:56.021964+0000","last_became_active":"2024-03-21T10:21:58.850563+0000","last_became_peered":"2024-03-21T10:21:58.850563+0000","last_unstale":"2024-03-21T10:23:56.021964+0000","last_undegraded":"2024-03-21T10:23:56.021964+0000","last_fullsized":"2024-03-21T10:23:56.021964+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_clean_scrub_stamp":"2024-03-21T10:21:57.813712+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T19:39:08.983713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"95'6","reported_seq":36,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T10:23:56.027451+0000","last_change":"2024-03-21T10:21:58.838426+0000","last_active":"2024-03-21T10:23:56.027451+0000","last_peered":"2024-03-21T10:23:56.027451+0000","last_clean":"2024-03-21T10:23:56.027451+0000","last_became_active":"2024-03-21T10:21:58.837625+0000","last_became_peered":"2024-03-21T10:21:58.837625+0000","last_unstale":"2024-03-21T10:23:56.027451+0000","last_undegraded":"2024-03-21T10:23:56.027451+0000","last_fullsized":"2024-03-21T10:23:56.027451+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_clean_scrub_stamp":"2024-03-21T10:21:57.813712+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T16:22:26.523685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"95'8","reported_seq":38,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T10:23:56.022477+0000","last_change":"2024-03-21T10:21:58.854114+0000","last_active":"2024-03-21T10:23:56.022477+0000","last_peered":"2024-03-21T10:23:56.022477+0000","last_clean":"2024-03-21T10:23:56.022477+0000","last_became_active":"2024-03-21T10:21:58.853686+0000","last_became_peered":"2024-03-21T10:21:58.853686+0000","last_unstale":"2024-03-21T10:23:56.022477+0000","last_undegraded":"2024-03-21T10:23:56.022477+0000","last_fullsized":"2024-03-21T10:23:56.022477+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_clean_scrub_stamp":"2024-03-21T10:21:57.813712+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T11:07:27.346766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"95'10","reported_seq":40,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T10:23:56.049981+0000","last_change":"2024-03-21T10:21:58.857778+0000","last_active":"2024-03-21T10:23:56.049981+0000","last_peered":"2024-03-21T10:23:56.049981+0000","last_clean":"2024-03-21T10:23:56.049981+0000","last_became_active":"2024-03-21T10:21:58.857336+0000","last_became_peered":"2024-03-21T10:21:58.857336+0000","last_unstale":"2024-03-21T10:23:56.049981+0000","last_undegraded":"2024-03-21T10:23:56.049981+0000","last_fullsized":"2024-03-21T10:23:56.049981+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_clean_scrub_stamp":"2024-03-21T10:21:57.813712+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T15:22:57.760341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"95'11","reported_seq":41,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T10:23:56.022750+0000","last_change":"2024-03-21T10:21:58.869774+0000","last_active":"2024-03-21T10:23:56.022750+0000","last_peered":"2024-03-21T10:23:56.022750+0000","last_clean":"2024-03-21T10:23:56.022750+0000","last_became_active":"2024-03-21T10:21:58.868815+0000","last_became_peered":"2024-03-21T10:21:58.868815+0000","last_unstale":"2024-03-21T10:23:56.022750+0000","last_undegraded":"2024-03-21T10:23:56.022750+0000","last_fullsized":"2024-03-21T10:23:56.022750+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_clean_scrub_stamp":"2024-03-21T10:21:57.813712+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T16:57:55.082127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"95'8","reported_seq":38,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T10:23:56.004307+0000","last_change":"2024-03-21T10:21:58.864746+0000","last_active":"2024-03-21T10:23:56.004307+0000","last_peered":"2024-03-21T10:23:56.004307+0000","last_clean":"2024-03-21T10:23:56.004307+0000","last_became_active":"2024-03-21T10:21:58.864366+0000","last_became_peered":"2024-03-21T10:21:58.864366+0000","last_unstale":"2024-03-21T10:23:56.004307+0000","last_undegraded":"2024-03-21T10:23:56.004307+0000","last_fullsized":"2024-03-21T10:23:56.004307+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_clean_scrub_stamp":"2024-03-21T10:21:57.813712+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T13:10:27.303843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":117,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":500,"ondisk_log_size":500,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4751360,"data_stored":4722944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":73774,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":1585,"num_read_kb":876,"num_write":268,"num_write_kb":161,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":4464,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":577536,"data_stored":221322,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":230,"ondisk_log_size":230,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":82,"seq":352187318325,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36396,"kb_used_data":1060,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708756,"statfs":{"total":95995035648,"available":95957766144,"internally_reserved":0,"allocated":1085440,"data_stored":529237,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":36174276},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":74,"seq":317827579964,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32916,"kb_used_data":1740,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712236,"statfs":{"total":95995035648,"available":95961329664,"internally_reserved":0,"allocated":1781760,"data_stored":1183916,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":69,"seq":296352743491,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28096,"kb_used_data":1080,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717056,"statfs":{"total":95995035648,"available":95966265344,"internally_reserved":0,"allocated":1105920,"data_stored":527061,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":61,"seq":261993005130,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36412,"kb_used_data":1076,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708740,"statfs":{"total":95995035648,"available":95957749760,"internally_reserved":0,"allocated":1101824,"data_stored":527439,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":53,"seq":227633266770,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28672,"kb_used_data":1656,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716480,"statfs":{"total":95995035648,"available":95965675520,"internally_reserved":0,"allocated":1695744,"data_stored":1123943,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":48,"seq":206158430297,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36392,"kb_used_data":1056,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708760,"statfs":{"total":95995035648,"available":95957770240,"internally_reserved":0,"allocated":1081344,"data_stored":523058,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574970,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32300,"kb_used_data":1124,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712852,"statfs":{"total":95995035648,"available":95961960448,"internally_reserved":0,"allocated":1150976,"data_stored":588936,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738496,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28096,"kb_used_data":1080,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717056,"statfs":{"total":95995035648,"available":95966265344,"internally_reserved":0,"allocated":1105920,"data_stored":524514,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411445,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36400,"kb_used_data":1064,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708752,"statfs":{"total":95995035648,"available":95957762048,"internally_reserved":0,"allocated":1089536,"data_stored":523686,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":27,"seq":115964117101,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36404,"kb_used_data":1068,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708748,"statfs":{"total":95995035648,"available":95957757952,"internally_reserved":0,"allocated":1093632,"data_stored":522273,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":35,"seq":150323855462,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28156,"kb_used_data":1140,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716996,"statfs":{"total":95995035648,"available":95966203904,"internally_reserved":0,"allocated":1167360,"data_stored":589313,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":40,"seq":171798691935,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":37004,"kb_used_data":1668,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708148,"statfs":{"total":95995035648,"available":95957143552,"internally_reserved":0,"allocated":1708032,"data_stored":1119620,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2671,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":86016,"data_stored":67093,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1843,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":102400,"data_stored":67688,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":3313,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1215,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":3540,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":1500,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":118784,"data_stored":67609,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":3298,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-21T10:25:05.725 INFO:tasks.fwd_scrub.ceph_manager:clean! 2024-03-21T10:25:05.725 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-21T10:25:05.729 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:25:05 smithi100 ceph-mon[35193]: pgmap v464: 129 pgs: 129 active+clean; 649 KiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 1.7 KiB/s wr, 14 op/s 2024-03-21T10:25:05.729 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:25:05 smithi100 ceph-mon[35193]: from='client.? 172.21.15.47:0/1123821566' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-21T10:25:05.739 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:25:05 smithi157 ceph-mon[35581]: pgmap v464: 129 pgs: 129 active+clean; 649 KiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 1.7 KiB/s wr, 14 op/s 2024-03-21T10:25:05.739 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:25:05 smithi157 ceph-mon[35581]: from='client.? 172.21.15.47:0/1123821566' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-21T10:25:05.979 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:05.978+0000 7f3b87c00640 1 -- 172.21.15.47:0/3458125708 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3b80075660 msgr2=0x7f3b80073ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:25:05.979 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:05.978+0000 7f3b87c00640 1 --2- 172.21.15.47:0/3458125708 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3b80075660 0x7f3b80073ab0 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f3b74009900 tx=0x7f3b7402bcd0 comp rx=0 tx=0).stop 2024-03-21T10:25:05.979 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:05.978+0000 7f3b87c00640 1 -- 172.21.15.47:0/3458125708 shutdown_connections 2024-03-21T10:25:05.979 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:05.978+0000 7f3b87c00640 1 --2- 172.21.15.47:0/3458125708 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3b80109570 0x7f3b8010ba00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:25:05.979 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:05.978+0000 7f3b87c00640 1 --2- 172.21.15.47:0/3458125708 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3b80074080 0x7f3b800744e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:25:05.979 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:05.978+0000 7f3b87c00640 1 --2- 172.21.15.47:0/3458125708 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3b80075660 0x7f3b80073ab0 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:25:05.979 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:05.978+0000 7f3b87c00640 1 -- 172.21.15.47:0/3458125708 >> 172.21.15.47:0/3458125708 conn(0x7f3b800fcc50 msgr2=0x7f3b800ff070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:25:05.980 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:05.978+0000 7f3b87c00640 1 -- 172.21.15.47:0/3458125708 shutdown_connections 2024-03-21T10:25:05.980 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:05.978+0000 7f3b87c00640 1 -- 172.21.15.47:0/3458125708 wait complete. 2024-03-21T10:25:05.982 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:05.979+0000 7f3b87c00640 1 Processor -- start 2024-03-21T10:25:05.982 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:05.979+0000 7f3b87c00640 1 -- start start 2024-03-21T10:25:05.982 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:05.979+0000 7f3b87c00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3b80074080 0x7f3b80199b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:25:05.982 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:05.979+0000 7f3b87c00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3b80075660 0x7f3b8019a040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:25:05.982 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:05.979+0000 7f3b87c00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3b80109570 0x7f3b801a10b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:25:05.982 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:05.979+0000 7f3b87c00640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b8010f440 con 0x7f3b80109570 2024-03-21T10:25:05.982 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:05.979+0000 7f3b87c00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b8010f2c0 con 0x7f3b80075660 2024-03-21T10:25:05.982 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:05.979+0000 7f3b87c00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b8010f5c0 con 0x7f3b80074080 2024-03-21T10:25:05.982 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:05.980+0000 7f3b85c00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3b80109570 0x7f3b801a10b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:25:05.983 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:05.980+0000 7f3b85c00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3b80109570 0x7f3b801a10b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:56304/0 (socket says 172.21.15.47:56304) 2024-03-21T10:25:05.983 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:05.980+0000 7f3b85c00640 1 -- 172.21.15.47:0/3687236998 learned_addr learned my addr 172.21.15.47:0/3687236998 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:25:05.983 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:05.980+0000 7f3b7fe00640 1 --2- 172.21.15.47:0/3687236998 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3b80075660 0x7f3b8019a040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:25:05.983 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:05.980+0000 7f3b85200640 1 --2- 172.21.15.47:0/3687236998 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3b80074080 0x7f3b80199b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:25:05.983 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:05.980+0000 7f3b85c00640 1 -- 172.21.15.47:0/3687236998 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3b80074080 msgr2=0x7f3b80199b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:25:05.983 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:05.980+0000 7f3b85c00640 1 --2- 172.21.15.47:0/3687236998 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3b80074080 0x7f3b80199b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:25:05.983 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:05.980+0000 7f3b85c00640 1 -- 172.21.15.47:0/3687236998 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3b80075660 msgr2=0x7f3b8019a040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:25:05.983 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:05.980+0000 7f3b85c00640 1 --2- 172.21.15.47:0/3687236998 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3b80075660 0x7f3b8019a040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:25:05.983 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:05.980+0000 7f3b85c00640 1 -- 172.21.15.47:0/3687236998 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3b801a1720 con 0x7f3b80109570 2024-03-21T10:25:05.983 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:05.981+0000 7f3b85c00640 1 --2- 172.21.15.47:0/3687236998 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3b80109570 0x7f3b801a10b0 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f3b7800eb10 tx=0x7f3b7800c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:25:05.983 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:05.981+0000 7f3b7d600640 1 -- 172.21.15.47:0/3687236998 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b78004b20 con 0x7f3b80109570 2024-03-21T10:25:05.983 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:05.981+0000 7f3b7d600640 1 -- 172.21.15.47:0/3687236998 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f3b78004cc0 con 0x7f3b80109570 2024-03-21T10:25:05.984 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:05.981+0000 7f3b87c00640 1 -- 172.21.15.47:0/3687236998 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3b801a1a10 con 0x7f3b80109570 2024-03-21T10:25:05.984 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:05.981+0000 7f3b7d600640 1 -- 172.21.15.47:0/3687236998 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b78010690 con 0x7f3b80109570 2024-03-21T10:25:05.984 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:05.981+0000 7f3b87c00640 1 -- 172.21.15.47:0/3687236998 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3b801a1f10 con 0x7f3b80109570 2024-03-21T10:25:05.984 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:05.982+0000 7f3b87c00640 1 -- 172.21.15.47:0/3687236998 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3b80108330 con 0x7f3b80109570 2024-03-21T10:25:05.985 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:05.984+0000 7f3b7d600640 1 -- 172.21.15.47:0/3687236998 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 18) v1 ==== 137854+0+0 (secure 0 0 0) 0x7f3b78005000 con 0x7f3b80109570 2024-03-21T10:25:05.986 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:05.986+0000 7f3b7d600640 1 --2- 172.21.15.47:0/3687236998 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f3b680a2f20 0x7f3b680a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:25:05.986 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:05.986+0000 7f3b7d600640 1 -- 172.21.15.47:0/3687236998 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9216+0+0 (secure 0 0 0) 0x7f3b780d0060 con 0x7f3b80109570 2024-03-21T10:25:05.987 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:05.986+0000 7f3b85200640 1 --2- 172.21.15.47:0/3687236998 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f3b680a2f20 0x7f3b680a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:25:05.987 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:05.987+0000 7f3b85200640 1 --2- 172.21.15.47:0/3687236998 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f3b680a2f20 0x7f3b680a53e0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f3b74006010 tx=0x7f3b74000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:25:05.990 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:05.989+0000 7f3b7d600640 1 -- 172.21.15.47:0/3687236998 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f3b78053740 con 0x7f3b80109570 2024-03-21T10:25:06.189 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:06.188+0000 7f3b87c00640 1 -- 172.21.15.47:0/3687236998 --> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f3b80061ae0 con 0x7f3b680a2f20 2024-03-21T10:25:06.207 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:06.207+0000 7f3b7d600640 1 -- 172.21.15.47:0/3687236998 <== mgr.14152 v2:172.21.15.47:6800/2823885994 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+340554 (secure 0 0 0) 0x7f3b80061ae0 con 0x7f3b680a2f20 2024-03-21T10:25:06.212 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-21T10:25:06.214 INFO:teuthology.orchestra.run.smithi047.stderr:dumped all 2024-03-21T10:25:06.215 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:06.213+0000 7f3b87c00640 1 -- 172.21.15.47:0/3687236998 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f3b680a2f20 msgr2=0x7f3b680a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:25:06.216 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:06.213+0000 7f3b87c00640 1 --2- 172.21.15.47:0/3687236998 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f3b680a2f20 0x7f3b680a53e0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f3b74006010 tx=0x7f3b74000bc0 comp rx=0 tx=0).stop 2024-03-21T10:25:06.216 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:06.213+0000 7f3b87c00640 1 -- 172.21.15.47:0/3687236998 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3b80109570 msgr2=0x7f3b801a10b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:25:06.216 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:06.213+0000 7f3b87c00640 1 --2- 172.21.15.47:0/3687236998 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3b80109570 0x7f3b801a10b0 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f3b7800eb10 tx=0x7f3b7800c2d0 comp rx=0 tx=0).stop 2024-03-21T10:25:06.216 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:06.214+0000 7f3b87c00640 1 -- 172.21.15.47:0/3687236998 shutdown_connections 2024-03-21T10:25:06.216 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:06.214+0000 7f3b87c00640 1 --2- 172.21.15.47:0/3687236998 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f3b680a2f20 0x7f3b680a53e0 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:25:06.216 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:06.214+0000 7f3b87c00640 1 --2- 172.21.15.47:0/3687236998 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3b80109570 0x7f3b801a10b0 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:25:06.216 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:06.214+0000 7f3b87c00640 1 --2- 172.21.15.47:0/3687236998 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3b80075660 0x7f3b8019a040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:25:06.216 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:06.214+0000 7f3b87c00640 1 --2- 172.21.15.47:0/3687236998 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3b80074080 0x7f3b80199b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:25:06.216 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:06.214+0000 7f3b87c00640 1 -- 172.21.15.47:0/3687236998 >> 172.21.15.47:0/3687236998 conn(0x7f3b800fcc50 msgr2=0x7f3b80105500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:25:06.216 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:06.214+0000 7f3b87c00640 1 -- 172.21.15.47:0/3687236998 shutdown_connections 2024-03-21T10:25:06.216 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:06.214+0000 7f3b87c00640 1 -- 172.21.15.47:0/3687236998 wait complete. 2024-03-21T10:25:06.237 INFO:teuthology.orchestra.run.smithi047.stdout:{"pg_ready":true,"pg_map":{"version":465,"stamp":"2024-03-21T10:25:06.097822+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":664376,"num_objects":102,"num_object_clones":0,"num_object_copies":308,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":1726,"num_read_kb":995,"num_write":451,"num_write_kb":3101,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13,"num_bytes_recovered":1185317,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":869,"ondisk_log_size":869,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":397248,"kb_used_data":14816,"kb_used_omap":18,"kb_used_meta":382317,"kb_avail":1124544576,"statfs":{"total":1151940427776,"available":1151533645824,"internally_reserved":0,"allocated":15171584,"data_stored":8287310,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19142,"internal_metadata":391492922},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":6779,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":111,"num_read_kb":60,"num_write":21,"num_write_kb":20,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.005983"},"pg_stats":[{"pgid":"3.3f","version":"95'8","reported_seq":38,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T10:23:56.049062+0000","last_change":"2024-03-21T10:21:58.872480+0000","last_active":"2024-03-21T10:23:56.049062+0000","last_peered":"2024-03-21T10:23:56.049062+0000","last_clean":"2024-03-21T10:23:56.049062+0000","last_became_active":"2024-03-21T10:21:58.872143+0000","last_became_peered":"2024-03-21T10:21:58.872143+0000","last_unstale":"2024-03-21T10:23:56.049062+0000","last_undegraded":"2024-03-21T10:23:56.049062+0000","last_fullsized":"2024-03-21T10:23:56.049062+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_clean_scrub_stamp":"2024-03-21T10:21:57.813712+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T15:20:00.409201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"95'4","reported_seq":34,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T10:23:56.027679+0000","last_change":"2024-03-21T10:21:58.871130+0000","last_active":"2024-03-21T10:23:56.027679+0000","last_peered":"2024-03-21T10:23:56.027679+0000","last_clean":"2024-03-21T10:23:56.027679+0000","last_became_active":"2024-03-21T10:21:58.870750+0000","last_became_peered":"2024-03-21T10:21:58.870750+0000","last_unstale":"2024-03-21T10:23:56.027679+0000","last_undegraded":"2024-03-21T10:23:56.027679+0000","last_fullsized":"2024-03-21T10:23:56.027679+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_clean_scrub_stamp":"2024-03-21T10:21:57.813712+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T17:38:04.830212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"95'7","reported_seq":37,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T10:23:56.048931+0000","last_change":"2024-03-21T10:21:58.868026+0000","last_active":"2024-03-21T10:23:56.048931+0000","last_peered":"2024-03-21T10:23:56.048931+0000","last_clean":"2024-03-21T10:23:56.048931+0000","last_became_active":"2024-03-21T10:21:58.859877+0000","last_became_peered":"2024-03-21T10:21:58.859877+0000","last_unstale":"2024-03-21T10:23:56.048931+0000","last_undegraded":"2024-03-21T10:23:56.048931+0000","last_fullsized":"2024-03-21T10:23:56.048931+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_clean_scrub_stamp":"2024-03-21T10:21:57.813712+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T14:45:12.870111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"95'8","reported_seq":24,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T10:23:56.009656+0000","last_change":"2024-03-21T10:22:27.231695+0000","last_active":"2024-03-21T10:23:56.009656+0000","last_peered":"2024-03-21T10:23:56.009656+0000","last_clean":"2024-03-21T10:23:56.009656+0000","last_became_active":"2024-03-21T10:22:27.231149+0000","last_became_peered":"2024-03-21T10:22:27.231149+0000","last_unstale":"2024-03-21T10:23:56.009656+0000","last_undegraded":"2024-03-21T10:23:56.009656+0000","last_fullsized":"2024-03-21T10:23:56.009656+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_clean_scrub_stamp":"2024-03-21T10:21:57.813712+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T18:19:19.105366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,2],"acting":[9,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.3b","version":"95'4","reported_seq":34,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T10:23:56.012895+0000","last_change":"2024-03-21T10:21:58.853998+0000","last_active":"2024-03-21T10:23:56.012895+0000","last_peered":"2024-03-21T10:23:56.012895+0000","last_clean":"2024-03-21T10:23:56.012895+0000","last_became_active":"2024-03-21T10:21:58.853059+0000","last_became_peered":"2024-03-21T10:21:58.853059+0000","last_unstale":"2024-03-21T10:23:56.012895+0000","last_undegraded":"2024-03-21T10:23:56.012895+0000","last_fullsized":"2024-03-21T10:23:56.012895+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_clean_scrub_stamp":"2024-03-21T10:21:57.813712+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T17:06:36.859522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"95'6","reported_seq":36,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T10:23:56.024543+0000","last_change":"2024-03-21T10:21:58.855535+0000","last_active":"2024-03-21T10:23:56.024543+0000","last_peered":"2024-03-21T10:23:56.024543+0000","last_clean":"2024-03-21T10:23:56.024543+0000","last_became_active":"2024-03-21T10:21:58.855181+0000","last_became_peered":"2024-03-21T10:21:58.855181+0000","last_unstale":"2024-03-21T10:23:56.024543+0000","last_undegraded":"2024-03-21T10:23:56.024543+0000","last_fullsized":"2024-03-21T10:23:56.024543+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_clean_scrub_stamp":"2024-03-21T10:21:57.813712+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T14:46:35.045646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"95'5","reported_seq":35,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T10:23:56.041045+0000","last_change":"2024-03-21T10:21:58.856527+0000","last_active":"2024-03-21T10:23:56.041045+0000","last_peered":"2024-03-21T10:23:56.041045+0000","last_clean":"2024-03-21T10:23:56.041045+0000","last_became_active":"2024-03-21T10:21:58.856176+0000","last_became_peered":"2024-03-21T10:21:58.856176+0000","last_unstale":"2024-03-21T10:23:56.041045+0000","last_undegraded":"2024-03-21T10:23:56.041045+0000","last_fullsized":"2024-03-21T10:23:56.041045+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_clean_scrub_stamp":"2024-03-21T10:21:57.813712+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T21:34:07.804293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"95'9","reported_seq":39,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T10:23:56.027007+0000","last_change":"2024-03-21T10:21:58.846260+0000","last_active":"2024-03-21T10:23:56.027007+0000","last_peered":"2024-03-21T10:23:56.027007+0000","last_clean":"2024-03-21T10:23:56.027007+0000","last_became_active":"2024-03-21T10:21:58.844245+0000","last_became_peered":"2024-03-21T10:21:58.844245+0000","last_unstale":"2024-03-21T10:23:56.027007+0000","last_undegraded":"2024-03-21T10:23:56.027007+0000","last_fullsized":"2024-03-21T10:23:56.027007+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_clean_scrub_stamp":"2024-03-21T10:21:57.813712+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T16:37:07.480867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"95'11","reported_seq":41,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T10:23:56.049788+0000","last_change":"2024-03-21T10:21:58.858413+0000","last_active":"2024-03-21T10:23:56.049788+0000","last_peered":"2024-03-21T10:23:56.049788+0000","last_clean":"2024-03-21T10:23:56.049788+0000","last_became_active":"2024-03-21T10:21:58.858007+0000","last_became_peered":"2024-03-21T10:21:58.858007+0000","last_unstale":"2024-03-21T10:23:56.049788+0000","last_undegraded":"2024-03-21T10:23:56.049788+0000","last_fullsized":"2024-03-21T10:23:56.049788+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_clean_scrub_stamp":"2024-03-21T10:21:57.813712+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T21:49:12.428918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"95'16","reported_seq":46,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T10:23:56.026737+0000","last_change":"2024-03-21T10:21:58.864393+0000","last_active":"2024-03-21T10:23:56.026737+0000","last_peered":"2024-03-21T10:23:56.026737+0000","last_clean":"2024-03-21T10:23:56.026737+0000","last_became_active":"2024-03-21T10:21:58.864061+0000","last_became_peered":"2024-03-21T10:21:58.864061+0000","last_unstale":"2024-03-21T10:23:56.026737+0000","last_undegraded":"2024-03-21T10:23:56.026737+0000","last_fullsized":"2024-03-21T10:23:56.026737+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_clean_scrub_stamp":"2024-03-21T10:21:57.813712+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T20:19:44.209612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"95'6","reported_seq":36,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T10:23:56.048906+0000","last_change":"2024-03-21T10:21:58.873885+0000","last_active":"2024-03-21T10:23:56.048906+0000","last_peered":"2024-03-21T10:23:56.048906+0000","last_clean":"2024-03-21T10:23:56.048906+0000","last_became_active":"2024-03-21T10:21:58.869738+0000","last_became_peered":"2024-03-21T10:21:58.869738+0000","last_unstale":"2024-03-21T10:23:56.048906+0000","last_undegraded":"2024-03-21T10:23:56.048906+0000","last_fullsized":"2024-03-21T10:23:56.048906+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_clean_scrub_stamp":"2024-03-21T10:21:57.813712+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T13:33:26.816720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"95'7","reported_seq":37,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T10:23:56.007359+0000","last_change":"2024-03-21T10:21:58.867563+0000","last_active":"2024-03-21T10:23:56.007359+0000","last_peered":"2024-03-21T10:23:56.007359+0000","last_clean":"2024-03-21T10:23:56.007359+0000","last_became_active":"2024-03-21T10:21:58.863455+0000","last_became_peered":"2024-03-21T10:21:58.863455+0000","last_unstale":"2024-03-21T10:23:56.007359+0000","last_undegraded":"2024-03-21T10:23:56.007359+0000","last_fullsized":"2024-03-21T10:23:56.007359+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_clean_scrub_stamp":"2024-03-21T10:21:57.813712+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T15:04:16.636126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"95'10","reported_seq":46,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T10:23:56.019337+0000","last_change":"2024-03-21T10:22:27.213460+0000","last_active":"2024-03-21T10:23:56.019337+0000","last_peered":"2024-03-21T10:23:56.019337+0000","last_clean":"2024-03-21T10:23:56.019337+0000","last_became_active":"2024-03-21T10:22:27.212961+0000","last_became_peered":"2024-03-21T10:22:27.212961+0000","last_unstale":"2024-03-21T10:23:56.019337+0000","last_undegraded":"2024-03-21T10:23:56.019337+0000","last_fullsized":"2024-03-21T10:23:56.019337+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_clean_scrub_stamp":"2024-03-21T10:21:57.813712+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T18:26:03.881767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,9],"acting":[6,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"95'7","reported_seq":37,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T10:23:56.024681+0000","last_change":"2024-03-21T10:21:58.870851+0000","last_active":"2024-03-21T10:23:56.024681+0000","last_peered":"2024-03-21T10:23:56.024681+0000","last_clean":"2024-03-21T10:23:56.024681+0000","last_became_active":"2024-03-21T10:21:58.870132+0000","last_became_peered":"2024-03-21T10:21:58.870132+0000","last_unstale":"2024-03-21T10:23:56.024681+0000","last_undegraded":"2024-03-21T10:23:56.024681+0000","last_fullsized":"2024-03-21T10:23:56.024681+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_clean_scrub_stamp":"2024-03-21T10:21:57.813712+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T20:20:04.207090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"95'7","reported_seq":37,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T10:23:56.020213+0000","last_change":"2024-03-21T10:21:58.865084+0000","last_active":"2024-03-21T10:23:56.020213+0000","last_peered":"2024-03-21T10:23:56.020213+0000","last_clean":"2024-03-21T10:23:56.020213+0000","last_became_active":"2024-03-21T10:21:58.864583+0000","last_became_peered":"2024-03-21T10:21:58.864583+0000","last_unstale":"2024-03-21T10:23:56.020213+0000","last_undegraded":"2024-03-21T10:23:56.020213+0000","last_fullsized":"2024-03-21T10:23:56.020213+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_clean_scrub_stamp":"2024-03-21T10:21:57.813712+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T14:09:16.781462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"95'9","reported_seq":39,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T10:23:56.048651+0000","last_change":"2024-03-21T10:21:58.870370+0000","last_active":"2024-03-21T10:23:56.048651+0000","last_peered":"2024-03-21T10:23:56.048651+0000","last_clean":"2024-03-21T10:23:56.048651+0000","last_became_active":"2024-03-21T10:21:58.870044+0000","last_became_peered":"2024-03-21T10:21:58.870044+0000","last_unstale":"2024-03-21T10:23:56.048651+0000","last_undegraded":"2024-03-21T10:23:56.048651+0000","last_fullsized":"2024-03-21T10:23:56.048651+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_clean_scrub_stamp":"2024-03-21T10:21:57.813712+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T16:02:00.342426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"95'8","reported_seq":38,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T10:23:56.049052+0000","last_change":"2024-03-21T10:21:58.872787+0000","last_active":"2024-03-21T10:23:56.049052+0000","last_peered":"2024-03-21T10:23:56.049052+0000","last_clean":"2024-03-21T10:23:56.049052+0000","last_became_active":"2024-03-21T10:21:58.872390+0000","last_became_peered":"2024-03-21T10:21:58.872390+0000","last_unstale":"2024-03-21T10:23:56.049052+0000","last_undegraded":"2024-03-21T10:23:56.049052+0000","last_fullsized":"2024-03-21T10:23:56.049052+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_clean_scrub_stamp":"2024-03-21T10:21:57.813712+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T15:56:51.798173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"95'10","reported_seq":40,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T10:23:56.042614+0000","last_change":"2024-03-21T10:21:58.858474+0000","last_active":"2024-03-21T10:23:56.042614+0000","last_peered":"2024-03-21T10:23:56.042614+0000","last_clean":"2024-03-21T10:23:56.042614+0000","last_became_active":"2024-03-21T10:21:58.858076+0000","last_became_peered":"2024-03-21T10:21:58.858076+0000","last_unstale":"2024-03-21T10:23:56.042614+0000","last_undegraded":"2024-03-21T10:23:56.042614+0000","last_fullsized":"2024-03-21T10:23:56.042614+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_clean_scrub_stamp":"2024-03-21T10:21:57.813712+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T12:25:17.160405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"95'7","reported_seq":37,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T10:23:56.027042+0000","last_change":"2024-03-21T10:21:58.854156+0000","last_active":"2024-03-21T10:23:56.027042+0000","last_peered":"2024-03-21T10:23:56.027042+0000","last_clean":"2024-03-21T10:23:56.027042+0000","last_became_active":"2024-03-21T10:21:58.853758+0000","last_became_peered":"2024-03-21T10:21:58.853758+0000","last_unstale":"2024-03-21T10:23:56.027042+0000","last_undegraded":"2024-03-21T10:23:56.027042+0000","last_fullsized":"2024-03-21T10:23:56.027042+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_clean_scrub_stamp":"2024-03-21T10:21:57.813712+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T16:58:54.854826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":30,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T10:23:27.840549+0000","last_change":"2024-03-21T10:23:27.838445+0000","last_active":"2024-03-21T10:23:27.840549+0000","last_peered":"2024-03-21T10:23:27.840549+0000","last_clean":"2024-03-21T10:23:27.840549+0000","last_became_active":"2024-03-21T10:23:27.836992+0000","last_became_peered":"2024-03-21T10:23:27.836992+0000","last_unstale":"2024-03-21T10:23:27.840549+0000","last_undegraded":"2024-03-21T10:23:27.840549+0000","last_fullsized":"2024-03-21T10:23:27.840549+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:22:02.269614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:22:02.269614+0000","last_clean_scrub_stamp":"2024-03-21T10:22:02.269614+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T15:34:51.615332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8,0],"acting":[2,3,8,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"95'14","reported_seq":200,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T10:25:00.942184+0000","last_change":"2024-03-21T10:21:57.859554+0000","last_active":"2024-03-21T10:25:00.942184+0000","last_peered":"2024-03-21T10:25:00.942184+0000","last_clean":"2024-03-21T10:25:00.942184+0000","last_became_active":"2024-03-21T10:21:57.857997+0000","last_became_peered":"2024-03-21T10:21:57.857997+0000","last_unstale":"2024-03-21T10:25:00.942184+0000","last_undegraded":"2024-03-21T10:25:00.942184+0000","last_fullsized":"2024-03-21T10:25:00.942184+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:21:56.788908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:21:56.788908+0000","last_clean_scrub_stamp":"2024-03-21T10:21:56.788908+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T14:27:47.516832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":154,"num_read_kb":154,"num_write":15,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"95'5","reported_seq":35,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T10:23:56.021709+0000","last_change":"2024-03-21T10:21:58.850197+0000","last_active":"2024-03-21T10:23:56.021709+0000","last_peered":"2024-03-21T10:23:56.021709+0000","last_clean":"2024-03-21T10:23:56.021709+0000","last_became_active":"2024-03-21T10:21:58.849029+0000","last_became_peered":"2024-03-21T10:21:58.849029+0000","last_unstale":"2024-03-21T10:23:56.021709+0000","last_undegraded":"2024-03-21T10:23:56.021709+0000","last_fullsized":"2024-03-21T10:23:56.021709+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_clean_scrub_stamp":"2024-03-21T10:21:57.813712+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T15:46:49.077382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":23,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T10:23:27.640067+0000","last_change":"2024-03-21T10:22:03.320716+0000","last_active":"2024-03-21T10:23:27.640067+0000","last_peered":"2024-03-21T10:23:27.640067+0000","last_clean":"2024-03-21T10:23:27.640067+0000","last_became_active":"2024-03-21T10:22:03.320262+0000","last_became_peered":"2024-03-21T10:22:03.320262+0000","last_unstale":"2024-03-21T10:23:27.640067+0000","last_undegraded":"2024-03-21T10:23:27.640067+0000","last_fullsized":"2024-03-21T10:23:27.640067+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:22:02.269614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:22:02.269614+0000","last_clean_scrub_stamp":"2024-03-21T10:22:02.269614+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T16:16:07.423168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"92'2","reported_seq":34,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T10:23:27.639016+0000","last_change":"2024-03-21T10:21:57.863642+0000","last_active":"2024-03-21T10:23:27.639016+0000","last_peered":"2024-03-21T10:23:27.639016+0000","last_clean":"2024-03-21T10:23:27.639016+0000","last_became_active":"2024-03-21T10:21:57.863330+0000","last_became_peered":"2024-03-21T10:21:57.863330+0000","last_unstale":"2024-03-21T10:23:27.639016+0000","last_undegraded":"2024-03-21T10:23:27.639016+0000","last_fullsized":"2024-03-21T10:23:27.639016+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:21:56.788908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:21:56.788908+0000","last_clean_scrub_stamp":"2024-03-21T10:21:56.788908+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T14:45:02.131458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"95'6","reported_seq":36,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T10:23:56.048702+0000","last_change":"2024-03-21T10:21:58.862940+0000","last_active":"2024-03-21T10:23:56.048702+0000","last_peered":"2024-03-21T10:23:56.048702+0000","last_clean":"2024-03-21T10:23:56.048702+0000","last_became_active":"2024-03-21T10:21:58.862489+0000","last_became_peered":"2024-03-21T10:21:58.862489+0000","last_unstale":"2024-03-21T10:23:56.048702+0000","last_undegraded":"2024-03-21T10:23:56.048702+0000","last_fullsized":"2024-03-21T10:23:56.048702+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_clean_scrub_stamp":"2024-03-21T10:21:57.813712+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T19:28:02.175642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":23,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T10:23:27.640509+0000","last_change":"2024-03-21T10:22:03.318660+0000","last_active":"2024-03-21T10:23:27.640509+0000","last_peered":"2024-03-21T10:23:27.640509+0000","last_clean":"2024-03-21T10:23:27.640509+0000","last_became_active":"2024-03-21T10:22:03.318216+0000","last_became_peered":"2024-03-21T10:22:03.318216+0000","last_unstale":"2024-03-21T10:23:27.640509+0000","last_undegraded":"2024-03-21T10:23:27.640509+0000","last_fullsized":"2024-03-21T10:23:27.640509+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:22:02.269614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:22:02.269614+0000","last_clean_scrub_stamp":"2024-03-21T10:22:02.269614+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T16:36:39.930726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"93'12","reported_seq":31,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T10:23:27.837288+0000","last_change":"2024-03-21T10:22:27.239342+0000","last_active":"2024-03-21T10:23:27.837288+0000","last_peered":"2024-03-21T10:23:27.837288+0000","last_clean":"2024-03-21T10:23:27.837288+0000","last_became_active":"2024-03-21T10:22:27.224086+0000","last_became_peered":"2024-03-21T10:22:27.224086+0000","last_unstale":"2024-03-21T10:23:27.837288+0000","last_undegraded":"2024-03-21T10:23:27.837288+0000","last_fullsized":"2024-03-21T10:23:27.837288+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:21:56.788908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:21:56.788908+0000","last_clean_scrub_stamp":"2024-03-21T10:21:56.788908+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T20:46:45.951485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,7],"acting":[11,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1c","version":"95'9","reported_seq":39,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T10:23:56.044535+0000","last_change":"2024-03-21T10:21:58.873239+0000","last_active":"2024-03-21T10:23:56.044535+0000","last_peered":"2024-03-21T10:23:56.044535+0000","last_clean":"2024-03-21T10:23:56.044535+0000","last_became_active":"2024-03-21T10:21:58.872877+0000","last_became_peered":"2024-03-21T10:21:58.872877+0000","last_unstale":"2024-03-21T10:23:56.044535+0000","last_undegraded":"2024-03-21T10:23:56.044535+0000","last_fullsized":"2024-03-21T10:23:56.044535+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_clean_scrub_stamp":"2024-03-21T10:21:57.813712+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T16:22:49.070697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":23,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T10:23:27.640201+0000","last_change":"2024-03-21T10:22:03.311589+0000","last_active":"2024-03-21T10:23:27.640201+0000","last_peered":"2024-03-21T10:23:27.640201+0000","last_clean":"2024-03-21T10:23:27.640201+0000","last_became_active":"2024-03-21T10:22:03.311134+0000","last_became_peered":"2024-03-21T10:22:03.311134+0000","last_unstale":"2024-03-21T10:23:27.640201+0000","last_undegraded":"2024-03-21T10:23:27.640201+0000","last_fullsized":"2024-03-21T10:23:27.640201+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:22:02.269614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:22:02.269614+0000","last_clean_scrub_stamp":"2024-03-21T10:22:02.269614+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T10:22:29.138348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"93'5","reported_seq":38,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T10:23:27.646290+0000","last_change":"2024-03-21T10:21:57.858981+0000","last_active":"2024-03-21T10:23:27.646290+0000","last_peered":"2024-03-21T10:23:27.646290+0000","last_clean":"2024-03-21T10:23:27.646290+0000","last_became_active":"2024-03-21T10:21:57.858564+0000","last_became_peered":"2024-03-21T10:21:57.858564+0000","last_unstale":"2024-03-21T10:23:27.646290+0000","last_undegraded":"2024-03-21T10:23:27.646290+0000","last_fullsized":"2024-03-21T10:23:27.646290+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:21:56.788908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:21:56.788908+0000","last_clean_scrub_stamp":"2024-03-21T10:21:56.788908+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T10:45:09.488194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"95'15","reported_seq":45,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T10:23:56.039689+0000","last_change":"2024-03-21T10:21:58.858053+0000","last_active":"2024-03-21T10:23:56.039689+0000","last_peered":"2024-03-21T10:23:56.039689+0000","last_clean":"2024-03-21T10:23:56.039689+0000","last_became_active":"2024-03-21T10:21:58.857688+0000","last_became_peered":"2024-03-21T10:21:58.857688+0000","last_unstale":"2024-03-21T10:23:56.039689+0000","last_undegraded":"2024-03-21T10:23:56.039689+0000","last_fullsized":"2024-03-21T10:23:56.039689+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_clean_scrub_stamp":"2024-03-21T10:21:57.813712+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T19:45:13.043034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":30,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T10:23:27.843397+0000","last_change":"2024-03-21T10:23:27.842200+0000","last_active":"2024-03-21T10:23:27.843397+0000","last_peered":"2024-03-21T10:23:27.843397+0000","last_clean":"2024-03-21T10:23:27.843397+0000","last_became_active":"2024-03-21T10:23:27.840749+0000","last_became_peered":"2024-03-21T10:23:27.840749+0000","last_unstale":"2024-03-21T10:23:27.843397+0000","last_undegraded":"2024-03-21T10:23:27.843397+0000","last_fullsized":"2024-03-21T10:23:27.843397+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:22:02.269614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:22:02.269614+0000","last_clean_scrub_stamp":"2024-03-21T10:22:02.269614+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T22:01:31.714771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,7,11],"acting":[2,8,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"95'6","reported_seq":207,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T10:25:00.996435+0000","last_change":"2024-03-21T10:22:27.235744+0000","last_active":"2024-03-21T10:25:00.996435+0000","last_peered":"2024-03-21T10:25:00.996435+0000","last_clean":"2024-03-21T10:25:00.996435+0000","last_became_active":"2024-03-21T10:22:27.219999+0000","last_became_peered":"2024-03-21T10:22:27.219999+0000","last_unstale":"2024-03-21T10:25:00.996435+0000","last_undegraded":"2024-03-21T10:25:00.996435+0000","last_fullsized":"2024-03-21T10:25:00.996435+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:21:56.788908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:21:56.788908+0000","last_clean_scrub_stamp":"2024-03-21T10:21:56.788908+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T18:15:41.288034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1115,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":157,"num_read_kb":3,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":622,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7],"acting":[5,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"95'7","reported_seq":37,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T10:23:55.996971+0000","last_change":"2024-03-21T10:21:58.854752+0000","last_active":"2024-03-21T10:23:55.996971+0000","last_peered":"2024-03-21T10:23:55.996971+0000","last_clean":"2024-03-21T10:23:55.996971+0000","last_became_active":"2024-03-21T10:21:58.854442+0000","last_became_peered":"2024-03-21T10:21:58.854442+0000","last_unstale":"2024-03-21T10:23:55.996971+0000","last_undegraded":"2024-03-21T10:23:55.996971+0000","last_fullsized":"2024-03-21T10:23:55.996971+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_clean_scrub_stamp":"2024-03-21T10:21:57.813712+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T15:12:40.590585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":23,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T10:23:27.648343+0000","last_change":"2024-03-21T10:22:03.319247+0000","last_active":"2024-03-21T10:23:27.648343+0000","last_peered":"2024-03-21T10:23:27.648343+0000","last_clean":"2024-03-21T10:23:27.648343+0000","last_became_active":"2024-03-21T10:22:03.318731+0000","last_became_peered":"2024-03-21T10:22:03.318731+0000","last_unstale":"2024-03-21T10:23:27.648343+0000","last_undegraded":"2024-03-21T10:23:27.648343+0000","last_fullsized":"2024-03-21T10:23:27.648343+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:22:02.269614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:22:02.269614+0000","last_clean_scrub_stamp":"2024-03-21T10:22:02.269614+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T20:06:36.519559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"93'7","reported_seq":45,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T10:23:27.744422+0000","last_change":"2024-03-21T10:22:27.222981+0000","last_active":"2024-03-21T10:23:27.744422+0000","last_peered":"2024-03-21T10:23:27.744422+0000","last_clean":"2024-03-21T10:23:27.744422+0000","last_became_active":"2024-03-21T10:22:27.221088+0000","last_became_peered":"2024-03-21T10:22:27.221088+0000","last_unstale":"2024-03-21T10:23:27.744422+0000","last_undegraded":"2024-03-21T10:23:27.744422+0000","last_fullsized":"2024-03-21T10:23:27.744422+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:21:56.788908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:21:56.788908+0000","last_clean_scrub_stamp":"2024-03-21T10:21:56.788908+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T15:43:11.740353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,6],"acting":[10,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"95'8","reported_seq":44,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T10:23:56.027376+0000","last_change":"2024-03-21T10:22:27.226733+0000","last_active":"2024-03-21T10:23:56.027376+0000","last_peered":"2024-03-21T10:23:56.027376+0000","last_clean":"2024-03-21T10:23:56.027376+0000","last_became_active":"2024-03-21T10:22:27.225576+0000","last_became_peered":"2024-03-21T10:22:27.225576+0000","last_unstale":"2024-03-21T10:23:56.027376+0000","last_undegraded":"2024-03-21T10:23:56.027376+0000","last_fullsized":"2024-03-21T10:23:56.027376+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_clean_scrub_stamp":"2024-03-21T10:21:57.813712+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T20:06:09.534268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,3],"acting":[5,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":23,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T10:23:27.836282+0000","last_change":"2024-03-21T10:22:03.321595+0000","last_active":"2024-03-21T10:23:27.836282+0000","last_peered":"2024-03-21T10:23:27.836282+0000","last_clean":"2024-03-21T10:23:27.836282+0000","last_became_active":"2024-03-21T10:22:03.320897+0000","last_became_peered":"2024-03-21T10:22:03.320897+0000","last_unstale":"2024-03-21T10:23:27.836282+0000","last_undegraded":"2024-03-21T10:23:27.836282+0000","last_fullsized":"2024-03-21T10:23:27.836282+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:22:02.269614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:22:02.269614+0000","last_clean_scrub_stamp":"2024-03-21T10:22:02.269614+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T17:37:02.633709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"93'3","reported_seq":35,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T10:23:27.742215+0000","last_change":"2024-03-21T10:21:57.854801+0000","last_active":"2024-03-21T10:23:27.742215+0000","last_peered":"2024-03-21T10:23:27.742215+0000","last_clean":"2024-03-21T10:23:27.742215+0000","last_became_active":"2024-03-21T10:21:57.854443+0000","last_became_peered":"2024-03-21T10:21:57.854443+0000","last_unstale":"2024-03-21T10:23:27.742215+0000","last_undegraded":"2024-03-21T10:23:27.742215+0000","last_fullsized":"2024-03-21T10:23:27.742215+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:21:56.788908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:21:56.788908+0000","last_clean_scrub_stamp":"2024-03-21T10:21:56.788908+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T15:49:23.573360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"95'8","reported_seq":38,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T10:23:56.026817+0000","last_change":"2024-03-21T10:21:58.857167+0000","last_active":"2024-03-21T10:23:56.026817+0000","last_peered":"2024-03-21T10:23:56.026817+0000","last_clean":"2024-03-21T10:23:56.026817+0000","last_became_active":"2024-03-21T10:21:58.856827+0000","last_became_peered":"2024-03-21T10:21:58.856827+0000","last_unstale":"2024-03-21T10:23:56.026817+0000","last_undegraded":"2024-03-21T10:23:56.026817+0000","last_fullsized":"2024-03-21T10:23:56.026817+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_clean_scrub_stamp":"2024-03-21T10:21:57.813712+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T15:04:43.848209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":23,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T10:23:27.834956+0000","last_change":"2024-03-21T10:22:03.306554+0000","last_active":"2024-03-21T10:23:27.834956+0000","last_peered":"2024-03-21T10:23:27.834956+0000","last_clean":"2024-03-21T10:23:27.834956+0000","last_became_active":"2024-03-21T10:22:03.306087+0000","last_became_peered":"2024-03-21T10:22:03.306087+0000","last_unstale":"2024-03-21T10:23:27.834956+0000","last_undegraded":"2024-03-21T10:23:27.834956+0000","last_fullsized":"2024-03-21T10:23:27.834956+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:22:02.269614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:22:02.269614+0000","last_clean_scrub_stamp":"2024-03-21T10:22:02.269614+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T11:36:35.344770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"93'3","reported_seq":36,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T10:23:27.751080+0000","last_change":"2024-03-21T10:21:57.860343+0000","last_active":"2024-03-21T10:23:27.751080+0000","last_peered":"2024-03-21T10:23:27.751080+0000","last_clean":"2024-03-21T10:23:27.751080+0000","last_became_active":"2024-03-21T10:21:57.859808+0000","last_became_peered":"2024-03-21T10:21:57.859808+0000","last_unstale":"2024-03-21T10:23:27.751080+0000","last_undegraded":"2024-03-21T10:23:27.751080+0000","last_fullsized":"2024-03-21T10:23:27.751080+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:21:56.788908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:21:56.788908+0000","last_clean_scrub_stamp":"2024-03-21T10:21:56.788908+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T14:41:24.162795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"95'11","reported_seq":41,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T10:23:56.023593+0000","last_change":"2024-03-21T10:21:58.850991+0000","last_active":"2024-03-21T10:23:56.023593+0000","last_peered":"2024-03-21T10:23:56.023593+0000","last_clean":"2024-03-21T10:23:56.023593+0000","last_became_active":"2024-03-21T10:21:58.850581+0000","last_became_peered":"2024-03-21T10:21:58.850581+0000","last_unstale":"2024-03-21T10:23:56.023593+0000","last_undegraded":"2024-03-21T10:23:56.023593+0000","last_fullsized":"2024-03-21T10:23:56.023593+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_clean_scrub_stamp":"2024-03-21T10:21:57.813712+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T17:07:11.350806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":23,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T10:23:27.638522+0000","last_change":"2024-03-21T10:22:03.315333+0000","last_active":"2024-03-21T10:23:27.638522+0000","last_peered":"2024-03-21T10:23:27.638522+0000","last_clean":"2024-03-21T10:23:27.638522+0000","last_became_active":"2024-03-21T10:22:03.313553+0000","last_became_peered":"2024-03-21T10:22:03.313553+0000","last_unstale":"2024-03-21T10:23:27.638522+0000","last_undegraded":"2024-03-21T10:23:27.638522+0000","last_fullsized":"2024-03-21T10:23:27.638522+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:22:02.269614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:22:02.269614+0000","last_clean_scrub_stamp":"2024-03-21T10:22:02.269614+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T15:25:24.290789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"95'6","reported_seq":348,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T10:25:03.000542+0000","last_change":"2024-03-21T10:21:57.864930+0000","last_active":"2024-03-21T10:25:03.000542+0000","last_peered":"2024-03-21T10:25:03.000542+0000","last_clean":"2024-03-21T10:25:03.000542+0000","last_became_active":"2024-03-21T10:21:57.863769+0000","last_became_peered":"2024-03-21T10:21:57.863769+0000","last_unstale":"2024-03-21T10:25:03.000542+0000","last_undegraded":"2024-03-21T10:25:03.000542+0000","last_fullsized":"2024-03-21T10:25:03.000542+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:21:56.788908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:21:56.788908+0000","last_clean_scrub_stamp":"2024-03-21T10:21:56.788908+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T21:35:39.144065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":310,"num_read_kb":310,"num_write":7,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"95'10","reported_seq":40,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T10:23:56.047740+0000","last_change":"2024-03-21T10:21:58.872084+0000","last_active":"2024-03-21T10:23:56.047740+0000","last_peered":"2024-03-21T10:23:56.047740+0000","last_clean":"2024-03-21T10:23:56.047740+0000","last_became_active":"2024-03-21T10:21:58.871725+0000","last_became_peered":"2024-03-21T10:21:58.871725+0000","last_unstale":"2024-03-21T10:23:56.047740+0000","last_undegraded":"2024-03-21T10:23:56.047740+0000","last_fullsized":"2024-03-21T10:23:56.047740+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_clean_scrub_stamp":"2024-03-21T10:21:57.813712+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T16:19:37.240053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":23,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T10:23:27.641039+0000","last_change":"2024-03-21T10:22:03.317808+0000","last_active":"2024-03-21T10:23:27.641039+0000","last_peered":"2024-03-21T10:23:27.641039+0000","last_clean":"2024-03-21T10:23:27.641039+0000","last_became_active":"2024-03-21T10:22:03.313751+0000","last_became_peered":"2024-03-21T10:22:03.313751+0000","last_unstale":"2024-03-21T10:23:27.641039+0000","last_undegraded":"2024-03-21T10:23:27.641039+0000","last_fullsized":"2024-03-21T10:23:27.641039+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:22:02.269614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:22:02.269614+0000","last_clean_scrub_stamp":"2024-03-21T10:22:02.269614+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T20:05:08.105795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"93'4","reported_seq":36,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T10:23:27.646634+0000","last_change":"2024-03-21T10:21:57.865462+0000","last_active":"2024-03-21T10:23:27.646634+0000","last_peered":"2024-03-21T10:23:27.646634+0000","last_clean":"2024-03-21T10:23:27.646634+0000","last_became_active":"2024-03-21T10:21:57.862922+0000","last_became_peered":"2024-03-21T10:21:57.862922+0000","last_unstale":"2024-03-21T10:23:27.646634+0000","last_undegraded":"2024-03-21T10:23:27.646634+0000","last_fullsized":"2024-03-21T10:23:27.646634+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:21:56.788908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:21:56.788908+0000","last_clean_scrub_stamp":"2024-03-21T10:21:56.788908+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T15:31:17.167806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"95'9","reported_seq":39,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T10:23:56.022903+0000","last_change":"2024-03-21T10:21:58.865459+0000","last_active":"2024-03-21T10:23:56.022903+0000","last_peered":"2024-03-21T10:23:56.022903+0000","last_clean":"2024-03-21T10:23:56.022903+0000","last_became_active":"2024-03-21T10:21:58.860909+0000","last_became_peered":"2024-03-21T10:21:58.860909+0000","last_unstale":"2024-03-21T10:23:56.022903+0000","last_undegraded":"2024-03-21T10:23:56.022903+0000","last_fullsized":"2024-03-21T10:23:56.022903+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_clean_scrub_stamp":"2024-03-21T10:21:57.813712+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T15:53:35.110404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":23,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T10:23:27.640633+0000","last_change":"2024-03-21T10:22:03.318651+0000","last_active":"2024-03-21T10:23:27.640633+0000","last_peered":"2024-03-21T10:23:27.640633+0000","last_clean":"2024-03-21T10:23:27.640633+0000","last_became_active":"2024-03-21T10:22:03.318108+0000","last_became_peered":"2024-03-21T10:22:03.318108+0000","last_unstale":"2024-03-21T10:23:27.640633+0000","last_undegraded":"2024-03-21T10:23:27.640633+0000","last_fullsized":"2024-03-21T10:23:27.640633+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:22:02.269614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:22:02.269614+0000","last_clean_scrub_stamp":"2024-03-21T10:22:02.269614+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T19:00:04.216749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"93'5","reported_seq":38,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T10:23:27.751795+0000","last_change":"2024-03-21T10:21:57.865516+0000","last_active":"2024-03-21T10:23:27.751795+0000","last_peered":"2024-03-21T10:23:27.751795+0000","last_clean":"2024-03-21T10:23:27.751795+0000","last_became_active":"2024-03-21T10:21:57.865012+0000","last_became_peered":"2024-03-21T10:21:57.865012+0000","last_unstale":"2024-03-21T10:23:27.751795+0000","last_undegraded":"2024-03-21T10:23:27.751795+0000","last_fullsized":"2024-03-21T10:23:27.751795+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:21:56.788908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:21:56.788908+0000","last_clean_scrub_stamp":"2024-03-21T10:21:56.788908+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T21:38:02.813453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"95'11","reported_seq":41,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T10:23:56.048118+0000","last_change":"2024-03-21T10:21:58.870876+0000","last_active":"2024-03-21T10:23:56.048118+0000","last_peered":"2024-03-21T10:23:56.048118+0000","last_clean":"2024-03-21T10:23:56.048118+0000","last_became_active":"2024-03-21T10:21:58.868240+0000","last_became_peered":"2024-03-21T10:21:58.868240+0000","last_unstale":"2024-03-21T10:23:56.048118+0000","last_undegraded":"2024-03-21T10:23:56.048118+0000","last_fullsized":"2024-03-21T10:23:56.048118+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_clean_scrub_stamp":"2024-03-21T10:21:57.813712+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T15:16:59.496020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":23,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T10:23:27.641221+0000","last_change":"2024-03-21T10:22:03.314854+0000","last_active":"2024-03-21T10:23:27.641221+0000","last_peered":"2024-03-21T10:23:27.641221+0000","last_clean":"2024-03-21T10:23:27.641221+0000","last_became_active":"2024-03-21T10:22:03.314422+0000","last_became_peered":"2024-03-21T10:22:03.314422+0000","last_unstale":"2024-03-21T10:23:27.641221+0000","last_undegraded":"2024-03-21T10:23:27.641221+0000","last_fullsized":"2024-03-21T10:23:27.641221+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:22:02.269614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:22:02.269614+0000","last_clean_scrub_stamp":"2024-03-21T10:22:02.269614+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T12:34:31.519118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.14","version":"95'38","reported_seq":407,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T10:25:02.999025+0000","last_change":"2024-03-21T10:22:27.243817+0000","last_active":"2024-03-21T10:25:02.999025+0000","last_peered":"2024-03-21T10:25:02.999025+0000","last_clean":"2024-03-21T10:25:02.999025+0000","last_became_active":"2024-03-21T10:22:27.218873+0000","last_became_peered":"2024-03-21T10:22:27.218873+0000","last_unstale":"2024-03-21T10:25:02.999025+0000","last_undegraded":"2024-03-21T10:25:02.999025+0000","last_fullsized":"2024-03-21T10:25:02.999025+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:21:56.788908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:21:56.788908+0000","last_clean_scrub_stamp":"2024-03-21T10:21:56.788908+0000","objects_scrubbed":0,"log_size":38,"log_dups_size":0,"ondisk_log_size":38,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T12:27:43.574451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":64848,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":319,"num_read_kb":223,"num_write":40,"num_write_kb":82,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":3842,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,4],"acting":[10,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"95'6","reported_seq":36,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T10:23:56.011976+0000","last_change":"2024-03-21T10:21:58.860386+0000","last_active":"2024-03-21T10:23:56.011976+0000","last_peered":"2024-03-21T10:23:56.011976+0000","last_clean":"2024-03-21T10:23:56.011976+0000","last_became_active":"2024-03-21T10:21:58.860013+0000","last_became_peered":"2024-03-21T10:21:58.860013+0000","last_unstale":"2024-03-21T10:23:56.011976+0000","last_undegraded":"2024-03-21T10:23:56.011976+0000","last_fullsized":"2024-03-21T10:23:56.011976+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_clean_scrub_stamp":"2024-03-21T10:21:57.813712+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T18:02:33.920374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":23,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T10:23:27.752685+0000","last_change":"2024-03-21T10:22:03.314083+0000","last_active":"2024-03-21T10:23:27.752685+0000","last_peered":"2024-03-21T10:23:27.752685+0000","last_clean":"2024-03-21T10:23:27.752685+0000","last_became_active":"2024-03-21T10:22:03.313652+0000","last_became_peered":"2024-03-21T10:22:03.313652+0000","last_unstale":"2024-03-21T10:23:27.752685+0000","last_undegraded":"2024-03-21T10:23:27.752685+0000","last_fullsized":"2024-03-21T10:23:27.752685+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:22:02.269614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:22:02.269614+0000","last_clean_scrub_stamp":"2024-03-21T10:22:02.269614+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T13:23:51.256535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"93'9","reported_seq":43,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T10:23:27.648194+0000","last_change":"2024-03-21T10:21:57.866716+0000","last_active":"2024-03-21T10:23:27.648194+0000","last_peered":"2024-03-21T10:23:27.648194+0000","last_clean":"2024-03-21T10:23:27.648194+0000","last_became_active":"2024-03-21T10:21:57.860900+0000","last_became_peered":"2024-03-21T10:21:57.860900+0000","last_unstale":"2024-03-21T10:23:27.648194+0000","last_undegraded":"2024-03-21T10:23:27.648194+0000","last_fullsized":"2024-03-21T10:23:27.648194+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:21:56.788908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:21:56.788908+0000","last_clean_scrub_stamp":"2024-03-21T10:21:56.788908+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T13:13:04.937120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"95'9","reported_seq":39,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T10:23:56.016553+0000","last_change":"2024-03-21T10:21:58.858815+0000","last_active":"2024-03-21T10:23:56.016553+0000","last_peered":"2024-03-21T10:23:56.016553+0000","last_clean":"2024-03-21T10:23:56.016553+0000","last_became_active":"2024-03-21T10:21:58.856483+0000","last_became_peered":"2024-03-21T10:21:58.856483+0000","last_unstale":"2024-03-21T10:23:56.016553+0000","last_undegraded":"2024-03-21T10:23:56.016553+0000","last_fullsized":"2024-03-21T10:23:56.016553+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_clean_scrub_stamp":"2024-03-21T10:21:57.813712+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T15:42:38.208093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":23,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T10:23:27.836558+0000","last_change":"2024-03-21T10:22:03.319719+0000","last_active":"2024-03-21T10:23:27.836558+0000","last_peered":"2024-03-21T10:23:27.836558+0000","last_clean":"2024-03-21T10:23:27.836558+0000","last_became_active":"2024-03-21T10:22:03.319234+0000","last_became_peered":"2024-03-21T10:22:03.319234+0000","last_unstale":"2024-03-21T10:23:27.836558+0000","last_undegraded":"2024-03-21T10:23:27.836558+0000","last_fullsized":"2024-03-21T10:23:27.836558+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:22:02.269614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:22:02.269614+0000","last_clean_scrub_stamp":"2024-03-21T10:22:02.269614+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T14:11:21.609594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"93'5","reported_seq":38,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T10:23:27.751203+0000","last_change":"2024-03-21T10:21:57.862744+0000","last_active":"2024-03-21T10:23:27.751203+0000","last_peered":"2024-03-21T10:23:27.751203+0000","last_clean":"2024-03-21T10:23:27.751203+0000","last_became_active":"2024-03-21T10:21:57.862352+0000","last_became_peered":"2024-03-21T10:21:57.862352+0000","last_unstale":"2024-03-21T10:23:27.751203+0000","last_undegraded":"2024-03-21T10:23:27.751203+0000","last_fullsized":"2024-03-21T10:23:27.751203+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:21:56.788908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:21:56.788908+0000","last_clean_scrub_stamp":"2024-03-21T10:21:56.788908+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T20:19:50.138904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"95'15","reported_seq":45,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T10:23:56.046387+0000","last_change":"2024-03-21T10:21:58.863514+0000","last_active":"2024-03-21T10:23:56.046387+0000","last_peered":"2024-03-21T10:23:56.046387+0000","last_clean":"2024-03-21T10:23:56.046387+0000","last_became_active":"2024-03-21T10:21:58.861981+0000","last_became_peered":"2024-03-21T10:21:58.861981+0000","last_unstale":"2024-03-21T10:23:56.046387+0000","last_undegraded":"2024-03-21T10:23:56.046387+0000","last_fullsized":"2024-03-21T10:23:56.046387+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_clean_scrub_stamp":"2024-03-21T10:21:57.813712+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T15:00:33.813456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":23,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T10:23:27.642326+0000","last_change":"2024-03-21T10:22:03.309844+0000","last_active":"2024-03-21T10:23:27.642326+0000","last_peered":"2024-03-21T10:23:27.642326+0000","last_clean":"2024-03-21T10:23:27.642326+0000","last_became_active":"2024-03-21T10:22:03.309448+0000","last_became_peered":"2024-03-21T10:22:03.309448+0000","last_unstale":"2024-03-21T10:23:27.642326+0000","last_undegraded":"2024-03-21T10:23:27.642326+0000","last_fullsized":"2024-03-21T10:23:27.642326+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:22:02.269614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:22:02.269614+0000","last_clean_scrub_stamp":"2024-03-21T10:22:02.269614+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T18:38:51.167166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.11","version":"95'8","reported_seq":206,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T10:25:02.998428+0000","last_change":"2024-03-21T10:22:27.223337+0000","last_active":"2024-03-21T10:25:02.998428+0000","last_peered":"2024-03-21T10:25:02.998428+0000","last_clean":"2024-03-21T10:25:02.998428+0000","last_became_active":"2024-03-21T10:22:27.222999+0000","last_became_peered":"2024-03-21T10:22:27.222999+0000","last_unstale":"2024-03-21T10:25:02.998428+0000","last_undegraded":"2024-03-21T10:25:02.998428+0000","last_fullsized":"2024-03-21T10:25:02.998428+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:21:56.788908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:21:56.788908+0000","last_clean_scrub_stamp":"2024-03-21T10:21:56.788908+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T20:40:45.780696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1613,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":161,"num_read_kb":5,"num_write":8,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,10],"acting":[4,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"95'6","reported_seq":36,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T10:23:56.037159+0000","last_change":"2024-03-21T10:21:58.862225+0000","last_active":"2024-03-21T10:23:56.037159+0000","last_peered":"2024-03-21T10:23:56.037159+0000","last_clean":"2024-03-21T10:23:56.037159+0000","last_became_active":"2024-03-21T10:21:58.861816+0000","last_became_peered":"2024-03-21T10:21:58.861816+0000","last_unstale":"2024-03-21T10:23:56.037159+0000","last_undegraded":"2024-03-21T10:23:56.037159+0000","last_fullsized":"2024-03-21T10:23:56.037159+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_clean_scrub_stamp":"2024-03-21T10:21:57.813712+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T18:36:13.420867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":23,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T10:23:27.742956+0000","last_change":"2024-03-21T10:22:03.315306+0000","last_active":"2024-03-21T10:23:27.742956+0000","last_peered":"2024-03-21T10:23:27.742956+0000","last_clean":"2024-03-21T10:23:27.742956+0000","last_became_active":"2024-03-21T10:22:03.314839+0000","last_became_peered":"2024-03-21T10:22:03.314839+0000","last_unstale":"2024-03-21T10:23:27.742956+0000","last_undegraded":"2024-03-21T10:23:27.742956+0000","last_fullsized":"2024-03-21T10:23:27.742956+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:22:02.269614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:22:02.269614+0000","last_clean_scrub_stamp":"2024-03-21T10:22:02.269614+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T15:47:45.690434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"93'17","reported_seq":50,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T10:23:27.644536+0000","last_change":"2024-03-21T10:21:57.870998+0000","last_active":"2024-03-21T10:23:27.644536+0000","last_peered":"2024-03-21T10:23:27.644536+0000","last_clean":"2024-03-21T10:23:27.644536+0000","last_became_active":"2024-03-21T10:21:57.870605+0000","last_became_peered":"2024-03-21T10:21:57.870605+0000","last_unstale":"2024-03-21T10:23:27.644536+0000","last_undegraded":"2024-03-21T10:23:27.644536+0000","last_fullsized":"2024-03-21T10:23:27.644536+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:21:56.788908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:21:56.788908+0000","last_clean_scrub_stamp":"2024-03-21T10:21:56.788908+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T13:33:54.775122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"95'6","reported_seq":36,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T10:23:56.030441+0000","last_change":"2024-03-21T10:21:58.858675+0000","last_active":"2024-03-21T10:23:56.030441+0000","last_peered":"2024-03-21T10:23:56.030441+0000","last_clean":"2024-03-21T10:23:56.030441+0000","last_became_active":"2024-03-21T10:21:58.856907+0000","last_became_peered":"2024-03-21T10:21:58.856907+0000","last_unstale":"2024-03-21T10:23:56.030441+0000","last_undegraded":"2024-03-21T10:23:56.030441+0000","last_fullsized":"2024-03-21T10:23:56.030441+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_clean_scrub_stamp":"2024-03-21T10:21:57.813712+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T18:17:31.208728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":23,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T10:23:27.750746+0000","last_change":"2024-03-21T10:22:03.318257+0000","last_active":"2024-03-21T10:23:27.750746+0000","last_peered":"2024-03-21T10:23:27.750746+0000","last_clean":"2024-03-21T10:23:27.750746+0000","last_became_active":"2024-03-21T10:22:03.317622+0000","last_became_peered":"2024-03-21T10:22:03.317622+0000","last_unstale":"2024-03-21T10:23:27.750746+0000","last_undegraded":"2024-03-21T10:23:27.750746+0000","last_fullsized":"2024-03-21T10:23:27.750746+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:22:02.269614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:22:02.269614+0000","last_clean_scrub_stamp":"2024-03-21T10:22:02.269614+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T17:15:50.272193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"95'3","reported_seq":36,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T10:24:12.259722+0000","last_change":"2024-03-21T10:21:57.868014+0000","last_active":"2024-03-21T10:24:12.259722+0000","last_peered":"2024-03-21T10:24:12.259722+0000","last_clean":"2024-03-21T10:24:12.259722+0000","last_became_active":"2024-03-21T10:21:57.867189+0000","last_became_peered":"2024-03-21T10:21:57.867189+0000","last_unstale":"2024-03-21T10:24:12.259722+0000","last_undegraded":"2024-03-21T10:24:12.259722+0000","last_fullsized":"2024-03-21T10:24:12.259722+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:21:56.788908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:21:56.788908+0000","last_clean_scrub_stamp":"2024-03-21T10:21:56.788908+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T22:02:48.704932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"95'5","reported_seq":35,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T10:23:56.004037+0000","last_change":"2024-03-21T10:21:58.862205+0000","last_active":"2024-03-21T10:23:56.004037+0000","last_peered":"2024-03-21T10:23:56.004037+0000","last_clean":"2024-03-21T10:23:56.004037+0000","last_became_active":"2024-03-21T10:21:58.861186+0000","last_became_peered":"2024-03-21T10:21:58.861186+0000","last_unstale":"2024-03-21T10:23:56.004037+0000","last_undegraded":"2024-03-21T10:23:56.004037+0000","last_fullsized":"2024-03-21T10:23:56.004037+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_clean_scrub_stamp":"2024-03-21T10:21:57.813712+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T19:44:49.719845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":23,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T10:23:27.643556+0000","last_change":"2024-03-21T10:22:03.636064+0000","last_active":"2024-03-21T10:23:27.643556+0000","last_peered":"2024-03-21T10:23:27.643556+0000","last_clean":"2024-03-21T10:23:27.643556+0000","last_became_active":"2024-03-21T10:22:03.633596+0000","last_became_peered":"2024-03-21T10:22:03.633596+0000","last_unstale":"2024-03-21T10:23:27.643556+0000","last_undegraded":"2024-03-21T10:23:27.643556+0000","last_fullsized":"2024-03-21T10:23:27.643556+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:22:02.269614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:22:02.269614+0000","last_clean_scrub_stamp":"2024-03-21T10:22:02.269614+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T19:32:31.844066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"93'1","reported_seq":33,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T10:23:27.642192+0000","last_change":"2024-03-21T10:21:57.851955+0000","last_active":"2024-03-21T10:23:27.642192+0000","last_peered":"2024-03-21T10:23:27.642192+0000","last_clean":"2024-03-21T10:23:27.642192+0000","last_became_active":"2024-03-21T10:21:57.851571+0000","last_became_peered":"2024-03-21T10:21:57.851571+0000","last_unstale":"2024-03-21T10:23:27.642192+0000","last_undegraded":"2024-03-21T10:23:27.642192+0000","last_fullsized":"2024-03-21T10:23:27.642192+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:21:56.788908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:21:56.788908+0000","last_clean_scrub_stamp":"2024-03-21T10:21:56.788908+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T17:23:40.441895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"95'4","reported_seq":34,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T10:23:56.015141+0000","last_change":"2024-03-21T10:21:58.856393+0000","last_active":"2024-03-21T10:23:56.015141+0000","last_peered":"2024-03-21T10:23:56.015141+0000","last_clean":"2024-03-21T10:23:56.015141+0000","last_became_active":"2024-03-21T10:21:58.855139+0000","last_became_peered":"2024-03-21T10:21:58.855139+0000","last_unstale":"2024-03-21T10:23:56.015141+0000","last_undegraded":"2024-03-21T10:23:56.015141+0000","last_fullsized":"2024-03-21T10:23:56.015141+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_clean_scrub_stamp":"2024-03-21T10:21:57.813712+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T14:24:55.663576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":23,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T10:23:27.742416+0000","last_change":"2024-03-21T10:22:03.315586+0000","last_active":"2024-03-21T10:23:27.742416+0000","last_peered":"2024-03-21T10:23:27.742416+0000","last_clean":"2024-03-21T10:23:27.742416+0000","last_became_active":"2024-03-21T10:22:03.315198+0000","last_became_peered":"2024-03-21T10:22:03.315198+0000","last_unstale":"2024-03-21T10:23:27.742416+0000","last_undegraded":"2024-03-21T10:23:27.742416+0000","last_fullsized":"2024-03-21T10:23:27.742416+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:22:02.269614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:22:02.269614+0000","last_clean_scrub_stamp":"2024-03-21T10:22:02.269614+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T14:25:13.534311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"95'6","reported_seq":193,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T10:25:00.943126+0000","last_change":"2024-03-21T10:21:57.862094+0000","last_active":"2024-03-21T10:25:00.943126+0000","last_peered":"2024-03-21T10:25:00.943126+0000","last_clean":"2024-03-21T10:25:00.943126+0000","last_became_active":"2024-03-21T10:21:57.860307+0000","last_became_peered":"2024-03-21T10:21:57.860307+0000","last_unstale":"2024-03-21T10:25:00.943126+0000","last_undegraded":"2024-03-21T10:25:00.943126+0000","last_fullsized":"2024-03-21T10:25:00.943126+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:21:56.788908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:21:56.788908+0000","last_clean_scrub_stamp":"2024-03-21T10:21:56.788908+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T12:58:50.701218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1081,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":155,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"95'7","reported_seq":37,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T10:23:56.024789+0000","last_change":"2024-03-21T10:21:58.870880+0000","last_active":"2024-03-21T10:23:56.024789+0000","last_peered":"2024-03-21T10:23:56.024789+0000","last_clean":"2024-03-21T10:23:56.024789+0000","last_became_active":"2024-03-21T10:21:58.869882+0000","last_became_peered":"2024-03-21T10:21:58.869882+0000","last_unstale":"2024-03-21T10:23:56.024789+0000","last_undegraded":"2024-03-21T10:23:56.024789+0000","last_fullsized":"2024-03-21T10:23:56.024789+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_clean_scrub_stamp":"2024-03-21T10:21:57.813712+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T18:01:42.613732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":23,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T10:23:27.742882+0000","last_change":"2024-03-21T10:22:03.634425+0000","last_active":"2024-03-21T10:23:27.742882+0000","last_peered":"2024-03-21T10:23:27.742882+0000","last_clean":"2024-03-21T10:23:27.742882+0000","last_became_active":"2024-03-21T10:22:03.632464+0000","last_became_peered":"2024-03-21T10:22:03.632464+0000","last_unstale":"2024-03-21T10:23:27.742882+0000","last_undegraded":"2024-03-21T10:23:27.742882+0000","last_fullsized":"2024-03-21T10:23:27.742882+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:22:02.269614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:22:02.269614+0000","last_clean_scrub_stamp":"2024-03-21T10:22:02.269614+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T16:03:28.270757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"95'8","reported_seq":19,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T10:23:56.006280+0000","last_change":"2024-03-21T10:23:27.649397+0000","last_active":"2024-03-21T10:23:56.006280+0000","last_peered":"2024-03-21T10:23:56.006280+0000","last_clean":"2024-03-21T10:23:56.006280+0000","last_became_active":"2024-03-21T10:23:27.649148+0000","last_became_peered":"2024-03-21T10:23:27.649148+0000","last_unstale":"2024-03-21T10:23:56.006280+0000","last_undegraded":"2024-03-21T10:23:56.006280+0000","last_fullsized":"2024-03-21T10:23:56.006280+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_clean_scrub_stamp":"2024-03-21T10:21:57.813712+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T19:28:03.830551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,8],"acting":[9,2,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.1","version":"95'7","reported_seq":195,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T10:25:01.994641+0000","last_change":"2024-03-21T10:21:57.856111+0000","last_active":"2024-03-21T10:25:01.994641+0000","last_peered":"2024-03-21T10:25:01.994641+0000","last_clean":"2024-03-21T10:25:01.994641+0000","last_became_active":"2024-03-21T10:21:57.855584+0000","last_became_peered":"2024-03-21T10:21:57.855584+0000","last_unstale":"2024-03-21T10:25:01.994641+0000","last_undegraded":"2024-03-21T10:25:01.994641+0000","last_fullsized":"2024-03-21T10:25:01.994641+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:21:56.788908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:21:56.788908+0000","last_clean_scrub_stamp":"2024-03-21T10:21:56.788908+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T17:53:48.208554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":156,"num_read_kb":156,"num_write":7,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"93'1","reported_seq":24,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T10:23:27.645180+0000","last_change":"2024-03-21T10:22:03.318831+0000","last_active":"2024-03-21T10:23:27.645180+0000","last_peered":"2024-03-21T10:23:27.645180+0000","last_clean":"2024-03-21T10:23:27.645180+0000","last_became_active":"2024-03-21T10:22:03.318404+0000","last_became_peered":"2024-03-21T10:22:03.318404+0000","last_unstale":"2024-03-21T10:23:27.645180+0000","last_undegraded":"2024-03-21T10:23:27.645180+0000","last_fullsized":"2024-03-21T10:23:27.645180+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:22:02.269614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:22:02.269614+0000","last_clean_scrub_stamp":"2024-03-21T10:22:02.269614+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T12:06:52.043286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"95'8","reported_seq":38,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T10:23:56.038328+0000","last_change":"2024-03-21T10:21:58.856469+0000","last_active":"2024-03-21T10:23:56.038328+0000","last_peered":"2024-03-21T10:23:56.038328+0000","last_clean":"2024-03-21T10:23:56.038328+0000","last_became_active":"2024-03-21T10:21:58.856032+0000","last_became_peered":"2024-03-21T10:21:58.856032+0000","last_unstale":"2024-03-21T10:23:56.038328+0000","last_undegraded":"2024-03-21T10:23:56.038328+0000","last_fullsized":"2024-03-21T10:23:56.038328+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_clean_scrub_stamp":"2024-03-21T10:21:57.813712+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T21:19:59.302418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"93'6","reported_seq":41,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T10:23:27.640569+0000","last_change":"2024-03-21T10:21:57.854210+0000","last_active":"2024-03-21T10:23:27.640569+0000","last_peered":"2024-03-21T10:23:27.640569+0000","last_clean":"2024-03-21T10:23:27.640569+0000","last_became_active":"2024-03-21T10:21:57.853787+0000","last_became_peered":"2024-03-21T10:21:57.853787+0000","last_unstale":"2024-03-21T10:23:27.640569+0000","last_undegraded":"2024-03-21T10:23:27.640569+0000","last_fullsized":"2024-03-21T10:23:27.640569+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:21:56.788908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:21:56.788908+0000","last_clean_scrub_stamp":"2024-03-21T10:21:56.788908+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T12:10:42.926436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"23'137","reported_seq":367,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T10:23:27.643665+0000","last_change":"2024-03-21T10:20:21.585339+0000","last_active":"2024-03-21T10:23:27.643665+0000","last_peered":"2024-03-21T10:23:27.643665+0000","last_clean":"2024-03-21T10:23:27.643665+0000","last_became_active":"2024-03-21T10:20:21.583999+0000","last_became_peered":"2024-03-21T10:20:21.583999+0000","last_unstale":"2024-03-21T10:23:27.643665+0000","last_undegraded":"2024-03-21T10:23:27.643665+0000","last_fullsized":"2024-03-21T10:23:27.643665+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:15:40.696168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:15:40.696168+0000","last_clean_scrub_stamp":"2024-03-21T10:15:40.696168+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T15:31:54.603722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":23,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T10:23:27.752359+0000","last_change":"2024-03-21T10:22:03.634970+0000","last_active":"2024-03-21T10:23:27.752359+0000","last_peered":"2024-03-21T10:23:27.752359+0000","last_clean":"2024-03-21T10:23:27.752359+0000","last_became_active":"2024-03-21T10:22:03.633603+0000","last_became_peered":"2024-03-21T10:22:03.633603+0000","last_unstale":"2024-03-21T10:23:27.752359+0000","last_undegraded":"2024-03-21T10:23:27.752359+0000","last_fullsized":"2024-03-21T10:23:27.752359+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:22:02.269614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:22:02.269614+0000","last_clean_scrub_stamp":"2024-03-21T10:22:02.269614+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T12:13:02.041661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"95'5","reported_seq":35,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T10:23:56.002047+0000","last_change":"2024-03-21T10:21:58.861159+0000","last_active":"2024-03-21T10:23:56.002047+0000","last_peered":"2024-03-21T10:23:56.002047+0000","last_clean":"2024-03-21T10:23:56.002047+0000","last_became_active":"2024-03-21T10:21:58.860690+0000","last_became_peered":"2024-03-21T10:21:58.860690+0000","last_unstale":"2024-03-21T10:23:56.002047+0000","last_undegraded":"2024-03-21T10:23:56.002047+0000","last_fullsized":"2024-03-21T10:23:56.002047+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_clean_scrub_stamp":"2024-03-21T10:21:57.813712+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T16:21:53.632997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"93'4","reported_seq":36,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T10:23:27.643848+0000","last_change":"2024-03-21T10:21:57.863581+0000","last_active":"2024-03-21T10:23:27.643848+0000","last_peered":"2024-03-21T10:23:27.643848+0000","last_clean":"2024-03-21T10:23:27.643848+0000","last_became_active":"2024-03-21T10:21:57.863269+0000","last_became_peered":"2024-03-21T10:21:57.863269+0000","last_unstale":"2024-03-21T10:23:27.643848+0000","last_undegraded":"2024-03-21T10:23:27.643848+0000","last_fullsized":"2024-03-21T10:23:27.643848+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:21:56.788908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:21:56.788908+0000","last_clean_scrub_stamp":"2024-03-21T10:21:56.788908+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T14:03:59.930570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":23,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T10:23:27.835814+0000","last_change":"2024-03-21T10:22:03.313515+0000","last_active":"2024-03-21T10:23:27.835814+0000","last_peered":"2024-03-21T10:23:27.835814+0000","last_clean":"2024-03-21T10:23:27.835814+0000","last_became_active":"2024-03-21T10:22:03.311862+0000","last_became_peered":"2024-03-21T10:22:03.311862+0000","last_unstale":"2024-03-21T10:23:27.835814+0000","last_undegraded":"2024-03-21T10:23:27.835814+0000","last_fullsized":"2024-03-21T10:23:27.835814+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:22:02.269614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:22:02.269614+0000","last_clean_scrub_stamp":"2024-03-21T10:22:02.269614+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T16:47:50.761205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"93'4","reported_seq":37,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T10:23:27.646442+0000","last_change":"2024-03-21T10:21:57.870913+0000","last_active":"2024-03-21T10:23:27.646442+0000","last_peered":"2024-03-21T10:23:27.646442+0000","last_clean":"2024-03-21T10:23:27.646442+0000","last_became_active":"2024-03-21T10:21:57.870578+0000","last_became_peered":"2024-03-21T10:21:57.870578+0000","last_unstale":"2024-03-21T10:23:27.646442+0000","last_undegraded":"2024-03-21T10:23:27.646442+0000","last_fullsized":"2024-03-21T10:23:27.646442+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:21:56.788908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:21:56.788908+0000","last_clean_scrub_stamp":"2024-03-21T10:21:56.788908+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T11:46:48.621501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"95'9","reported_seq":39,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T10:23:56.007067+0000","last_change":"2024-03-21T10:21:58.852247+0000","last_active":"2024-03-21T10:23:56.007067+0000","last_peered":"2024-03-21T10:23:56.007067+0000","last_clean":"2024-03-21T10:23:56.007067+0000","last_became_active":"2024-03-21T10:21:58.851976+0000","last_became_peered":"2024-03-21T10:21:58.851976+0000","last_unstale":"2024-03-21T10:23:56.007067+0000","last_undegraded":"2024-03-21T10:23:56.007067+0000","last_fullsized":"2024-03-21T10:23:56.007067+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_clean_scrub_stamp":"2024-03-21T10:21:57.813712+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T14:27:11.383741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":23,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T10:23:27.638289+0000","last_change":"2024-03-21T10:22:03.637671+0000","last_active":"2024-03-21T10:23:27.638289+0000","last_peered":"2024-03-21T10:23:27.638289+0000","last_clean":"2024-03-21T10:23:27.638289+0000","last_became_active":"2024-03-21T10:22:03.634265+0000","last_became_peered":"2024-03-21T10:22:03.634265+0000","last_unstale":"2024-03-21T10:23:27.638289+0000","last_undegraded":"2024-03-21T10:23:27.638289+0000","last_fullsized":"2024-03-21T10:23:27.638289+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:22:02.269614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:22:02.269614+0000","last_clean_scrub_stamp":"2024-03-21T10:22:02.269614+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T15:38:02.214178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"95'16","reported_seq":48,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T10:24:57.291140+0000","last_change":"2024-03-21T10:21:57.862027+0000","last_active":"2024-03-21T10:24:57.291140+0000","last_peered":"2024-03-21T10:24:57.291140+0000","last_clean":"2024-03-21T10:24:57.291140+0000","last_became_active":"2024-03-21T10:21:57.860097+0000","last_became_peered":"2024-03-21T10:21:57.860097+0000","last_unstale":"2024-03-21T10:24:57.291140+0000","last_undegraded":"2024-03-21T10:24:57.291140+0000","last_fullsized":"2024-03-21T10:24:57.291140+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:21:56.788908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:21:56.788908+0000","last_clean_scrub_stamp":"2024-03-21T10:21:56.788908+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T13:46:19.562452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":28,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"95'5","reported_seq":35,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T10:23:56.040770+0000","last_change":"2024-03-21T10:21:58.856082+0000","last_active":"2024-03-21T10:23:56.040770+0000","last_peered":"2024-03-21T10:23:56.040770+0000","last_clean":"2024-03-21T10:23:56.040770+0000","last_became_active":"2024-03-21T10:21:58.854172+0000","last_became_peered":"2024-03-21T10:21:58.854172+0000","last_unstale":"2024-03-21T10:23:56.040770+0000","last_undegraded":"2024-03-21T10:23:56.040770+0000","last_fullsized":"2024-03-21T10:23:56.040770+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_clean_scrub_stamp":"2024-03-21T10:21:57.813712+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T14:59:33.995716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":23,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T10:23:27.647010+0000","last_change":"2024-03-21T10:22:03.310205+0000","last_active":"2024-03-21T10:23:27.647010+0000","last_peered":"2024-03-21T10:23:27.647010+0000","last_clean":"2024-03-21T10:23:27.647010+0000","last_became_active":"2024-03-21T10:22:03.309039+0000","last_became_peered":"2024-03-21T10:22:03.309039+0000","last_unstale":"2024-03-21T10:23:27.647010+0000","last_undegraded":"2024-03-21T10:23:27.647010+0000","last_fullsized":"2024-03-21T10:23:27.647010+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:22:02.269614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:22:02.269614+0000","last_clean_scrub_stamp":"2024-03-21T10:22:02.269614+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T11:16:46.075260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"93'7","reported_seq":40,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T10:23:27.641787+0000","last_change":"2024-03-21T10:21:57.857890+0000","last_active":"2024-03-21T10:23:27.641787+0000","last_peered":"2024-03-21T10:23:27.641787+0000","last_clean":"2024-03-21T10:23:27.641787+0000","last_became_active":"2024-03-21T10:21:57.857492+0000","last_became_peered":"2024-03-21T10:21:57.857492+0000","last_unstale":"2024-03-21T10:23:27.641787+0000","last_undegraded":"2024-03-21T10:23:27.641787+0000","last_fullsized":"2024-03-21T10:23:27.641787+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:21:56.788908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:21:56.788908+0000","last_clean_scrub_stamp":"2024-03-21T10:21:56.788908+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T21:18:45.566843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"95'7","reported_seq":43,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T10:23:56.040975+0000","last_change":"2024-03-21T10:22:27.219339+0000","last_active":"2024-03-21T10:23:56.040975+0000","last_peered":"2024-03-21T10:23:56.040975+0000","last_clean":"2024-03-21T10:23:56.040975+0000","last_became_active":"2024-03-21T10:22:27.218604+0000","last_became_peered":"2024-03-21T10:22:27.218604+0000","last_unstale":"2024-03-21T10:23:56.040975+0000","last_undegraded":"2024-03-21T10:23:56.040975+0000","last_fullsized":"2024-03-21T10:23:56.040975+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_clean_scrub_stamp":"2024-03-21T10:21:57.813712+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T15:58:53.161767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,4],"acting":[0,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":23,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T10:23:27.641301+0000","last_change":"2024-03-21T10:22:03.633789+0000","last_active":"2024-03-21T10:23:27.641301+0000","last_peered":"2024-03-21T10:23:27.641301+0000","last_clean":"2024-03-21T10:23:27.641301+0000","last_became_active":"2024-03-21T10:22:03.632716+0000","last_became_peered":"2024-03-21T10:22:03.632716+0000","last_unstale":"2024-03-21T10:23:27.641301+0000","last_undegraded":"2024-03-21T10:23:27.641301+0000","last_fullsized":"2024-03-21T10:23:27.641301+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:22:02.269614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:22:02.269614+0000","last_clean_scrub_stamp":"2024-03-21T10:22:02.269614+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T14:08:47.301881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"91'2","reported_seq":34,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T10:23:27.833770+0000","last_change":"2024-03-21T10:21:57.862413+0000","last_active":"2024-03-21T10:23:27.833770+0000","last_peered":"2024-03-21T10:23:27.833770+0000","last_clean":"2024-03-21T10:23:27.833770+0000","last_became_active":"2024-03-21T10:21:57.862104+0000","last_became_peered":"2024-03-21T10:21:57.862104+0000","last_unstale":"2024-03-21T10:23:27.833770+0000","last_undegraded":"2024-03-21T10:23:27.833770+0000","last_fullsized":"2024-03-21T10:23:27.833770+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:21:56.788908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:21:56.788908+0000","last_clean_scrub_stamp":"2024-03-21T10:21:56.788908+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T17:49:21.729492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"95'2","reported_seq":32,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T10:23:56.014082+0000","last_change":"2024-03-21T10:21:58.864688+0000","last_active":"2024-03-21T10:23:56.014082+0000","last_peered":"2024-03-21T10:23:56.014082+0000","last_clean":"2024-03-21T10:23:56.014082+0000","last_became_active":"2024-03-21T10:21:58.862821+0000","last_became_peered":"2024-03-21T10:21:58.862821+0000","last_unstale":"2024-03-21T10:23:56.014082+0000","last_undegraded":"2024-03-21T10:23:56.014082+0000","last_fullsized":"2024-03-21T10:23:56.014082+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_clean_scrub_stamp":"2024-03-21T10:21:57.813712+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T11:17:09.650649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T10:23:27.756711+0000","last_change":"2024-03-21T10:23:27.756711+0000","last_active":"2024-03-21T10:23:27.756711+0000","last_peered":"2024-03-21T10:23:27.756711+0000","last_clean":"2024-03-21T10:23:27.756711+0000","last_became_active":"2024-03-21T10:23:27.754993+0000","last_became_peered":"2024-03-21T10:23:27.754993+0000","last_unstale":"2024-03-21T10:23:27.756711+0000","last_undegraded":"2024-03-21T10:23:27.756711+0000","last_fullsized":"2024-03-21T10:23:27.756711+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:22:02.269614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:22:02.269614+0000","last_clean_scrub_stamp":"2024-03-21T10:22:02.269614+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T11:22:51.851047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,2,5],"acting":[8,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"93'7","reported_seq":39,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T10:23:27.638625+0000","last_change":"2024-03-21T10:21:57.868033+0000","last_active":"2024-03-21T10:23:27.638625+0000","last_peered":"2024-03-21T10:23:27.638625+0000","last_clean":"2024-03-21T10:23:27.638625+0000","last_became_active":"2024-03-21T10:21:57.867279+0000","last_became_peered":"2024-03-21T10:21:57.867279+0000","last_unstale":"2024-03-21T10:23:27.638625+0000","last_undegraded":"2024-03-21T10:23:27.638625+0000","last_fullsized":"2024-03-21T10:23:27.638625+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:21:56.788908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:21:56.788908+0000","last_clean_scrub_stamp":"2024-03-21T10:21:56.788908+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T20:05:37.701443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"95'9","reported_seq":39,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T10:23:56.022588+0000","last_change":"2024-03-21T10:21:58.868455+0000","last_active":"2024-03-21T10:23:56.022588+0000","last_peered":"2024-03-21T10:23:56.022588+0000","last_clean":"2024-03-21T10:23:56.022588+0000","last_became_active":"2024-03-21T10:21:58.867786+0000","last_became_peered":"2024-03-21T10:21:58.867786+0000","last_unstale":"2024-03-21T10:23:56.022588+0000","last_undegraded":"2024-03-21T10:23:56.022588+0000","last_fullsized":"2024-03-21T10:23:56.022588+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_clean_scrub_stamp":"2024-03-21T10:21:57.813712+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T19:16:12.258470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"93'1","reported_seq":35,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T10:23:27.753821+0000","last_change":"2024-03-21T10:23:27.753222+0000","last_active":"2024-03-21T10:23:27.753821+0000","last_peered":"2024-03-21T10:23:27.753821+0000","last_clean":"2024-03-21T10:23:27.753821+0000","last_became_active":"2024-03-21T10:23:27.746251+0000","last_became_peered":"2024-03-21T10:23:27.746251+0000","last_unstale":"2024-03-21T10:23:27.753821+0000","last_undegraded":"2024-03-21T10:23:27.753821+0000","last_fullsized":"2024-03-21T10:23:27.753821+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:22:02.269614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:22:02.269614+0000","last_clean_scrub_stamp":"2024-03-21T10:22:02.269614+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T13:42:00.362336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":117,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,7,8],"acting":[4,10,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"95'5","reported_seq":37,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T10:24:57.202344+0000","last_change":"2024-03-21T10:21:57.862748+0000","last_active":"2024-03-21T10:24:57.202344+0000","last_peered":"2024-03-21T10:24:57.202344+0000","last_clean":"2024-03-21T10:24:57.202344+0000","last_became_active":"2024-03-21T10:21:57.862350+0000","last_became_peered":"2024-03-21T10:21:57.862350+0000","last_unstale":"2024-03-21T10:24:57.202344+0000","last_undegraded":"2024-03-21T10:24:57.202344+0000","last_fullsized":"2024-03-21T10:24:57.202344+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:21:56.788908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:21:56.788908+0000","last_clean_scrub_stamp":"2024-03-21T10:21:56.788908+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T17:08:47.301758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"95'8","reported_seq":38,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T10:23:56.026242+0000","last_change":"2024-03-21T10:21:58.866340+0000","last_active":"2024-03-21T10:23:56.026242+0000","last_peered":"2024-03-21T10:23:56.026242+0000","last_clean":"2024-03-21T10:23:56.026242+0000","last_became_active":"2024-03-21T10:21:58.865965+0000","last_became_peered":"2024-03-21T10:21:58.865965+0000","last_unstale":"2024-03-21T10:23:56.026242+0000","last_undegraded":"2024-03-21T10:23:56.026242+0000","last_fullsized":"2024-03-21T10:23:56.026242+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_clean_scrub_stamp":"2024-03-21T10:21:57.813712+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T10:42:23.854176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":23,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T10:23:27.640308+0000","last_change":"2024-03-21T10:22:03.634108+0000","last_active":"2024-03-21T10:23:27.640308+0000","last_peered":"2024-03-21T10:23:27.640308+0000","last_clean":"2024-03-21T10:23:27.640308+0000","last_became_active":"2024-03-21T10:22:03.633344+0000","last_became_peered":"2024-03-21T10:22:03.633344+0000","last_unstale":"2024-03-21T10:23:27.640308+0000","last_undegraded":"2024-03-21T10:23:27.640308+0000","last_fullsized":"2024-03-21T10:23:27.640308+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:22:02.269614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:22:02.269614+0000","last_clean_scrub_stamp":"2024-03-21T10:22:02.269614+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T21:00:59.893994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"91'4","reported_seq":37,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T10:23:27.644244+0000","last_change":"2024-03-21T10:21:57.871520+0000","last_active":"2024-03-21T10:23:27.644244+0000","last_peered":"2024-03-21T10:23:27.644244+0000","last_clean":"2024-03-21T10:23:27.644244+0000","last_became_active":"2024-03-21T10:21:57.871127+0000","last_became_peered":"2024-03-21T10:21:57.871127+0000","last_unstale":"2024-03-21T10:23:27.644244+0000","last_undegraded":"2024-03-21T10:23:27.644244+0000","last_fullsized":"2024-03-21T10:23:27.644244+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:21:56.788908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:21:56.788908+0000","last_clean_scrub_stamp":"2024-03-21T10:21:56.788908+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T20:43:39.537307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"95'5","reported_seq":35,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T10:23:56.044719+0000","last_change":"2024-03-21T10:21:58.861786+0000","last_active":"2024-03-21T10:23:56.044719+0000","last_peered":"2024-03-21T10:23:56.044719+0000","last_clean":"2024-03-21T10:23:56.044719+0000","last_became_active":"2024-03-21T10:21:58.861428+0000","last_became_peered":"2024-03-21T10:21:58.861428+0000","last_unstale":"2024-03-21T10:23:56.044719+0000","last_undegraded":"2024-03-21T10:23:56.044719+0000","last_fullsized":"2024-03-21T10:23:56.044719+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_clean_scrub_stamp":"2024-03-21T10:21:57.813712+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T15:56:30.256266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":23,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T10:23:27.833726+0000","last_change":"2024-03-21T10:22:03.305085+0000","last_active":"2024-03-21T10:23:27.833726+0000","last_peered":"2024-03-21T10:23:27.833726+0000","last_clean":"2024-03-21T10:23:27.833726+0000","last_became_active":"2024-03-21T10:22:03.304474+0000","last_became_peered":"2024-03-21T10:22:03.304474+0000","last_unstale":"2024-03-21T10:23:27.833726+0000","last_undegraded":"2024-03-21T10:23:27.833726+0000","last_fullsized":"2024-03-21T10:23:27.833726+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:22:02.269614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:22:02.269614+0000","last_clean_scrub_stamp":"2024-03-21T10:22:02.269614+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T18:28:10.480981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"95'6","reported_seq":196,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T10:25:01.999354+0000","last_change":"2024-03-21T10:21:57.860657+0000","last_active":"2024-03-21T10:25:01.999354+0000","last_peered":"2024-03-21T10:25:01.999354+0000","last_clean":"2024-03-21T10:25:01.999354+0000","last_became_active":"2024-03-21T10:21:57.860340+0000","last_became_peered":"2024-03-21T10:21:57.860340+0000","last_unstale":"2024-03-21T10:25:01.999354+0000","last_undegraded":"2024-03-21T10:25:01.999354+0000","last_fullsized":"2024-03-21T10:25:01.999354+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:21:56.788908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:21:56.788908+0000","last_clean_scrub_stamp":"2024-03-21T10:21:56.788908+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T21:58:08.706812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1663,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":158,"num_read_kb":3,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"95'7","reported_seq":37,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T10:23:56.048801+0000","last_change":"2024-03-21T10:21:58.874254+0000","last_active":"2024-03-21T10:23:56.048801+0000","last_peered":"2024-03-21T10:23:56.048801+0000","last_clean":"2024-03-21T10:23:56.048801+0000","last_became_active":"2024-03-21T10:21:58.869795+0000","last_became_peered":"2024-03-21T10:21:58.869795+0000","last_unstale":"2024-03-21T10:23:56.048801+0000","last_undegraded":"2024-03-21T10:23:56.048801+0000","last_fullsized":"2024-03-21T10:23:56.048801+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_clean_scrub_stamp":"2024-03-21T10:21:57.813712+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T19:51:28.296199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":23,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T10:23:27.834356+0000","last_change":"2024-03-21T10:22:03.306761+0000","last_active":"2024-03-21T10:23:27.834356+0000","last_peered":"2024-03-21T10:23:27.834356+0000","last_clean":"2024-03-21T10:23:27.834356+0000","last_became_active":"2024-03-21T10:22:03.305106+0000","last_became_peered":"2024-03-21T10:22:03.305106+0000","last_unstale":"2024-03-21T10:23:27.834356+0000","last_undegraded":"2024-03-21T10:23:27.834356+0000","last_fullsized":"2024-03-21T10:23:27.834356+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:22:02.269614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:22:02.269614+0000","last_clean_scrub_stamp":"2024-03-21T10:22:02.269614+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T21:13:00.846169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"93'3","reported_seq":36,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T10:23:27.752174+0000","last_change":"2024-03-21T10:21:57.868012+0000","last_active":"2024-03-21T10:23:27.752174+0000","last_peered":"2024-03-21T10:23:27.752174+0000","last_clean":"2024-03-21T10:23:27.752174+0000","last_became_active":"2024-03-21T10:21:57.867682+0000","last_became_peered":"2024-03-21T10:21:57.867682+0000","last_unstale":"2024-03-21T10:23:27.752174+0000","last_undegraded":"2024-03-21T10:23:27.752174+0000","last_fullsized":"2024-03-21T10:23:27.752174+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:21:56.788908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:21:56.788908+0000","last_clean_scrub_stamp":"2024-03-21T10:21:56.788908+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T16:59:56.061167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"95'6","reported_seq":36,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T10:23:56.033933+0000","last_change":"2024-03-21T10:21:58.864949+0000","last_active":"2024-03-21T10:23:56.033933+0000","last_peered":"2024-03-21T10:23:56.033933+0000","last_clean":"2024-03-21T10:23:56.033933+0000","last_became_active":"2024-03-21T10:21:58.859796+0000","last_became_peered":"2024-03-21T10:21:58.859796+0000","last_unstale":"2024-03-21T10:23:56.033933+0000","last_undegraded":"2024-03-21T10:23:56.033933+0000","last_fullsized":"2024-03-21T10:23:56.033933+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_clean_scrub_stamp":"2024-03-21T10:21:57.813712+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T18:02:51.873103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":23,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T10:23:27.750167+0000","last_change":"2024-03-21T10:22:03.308451+0000","last_active":"2024-03-21T10:23:27.750167+0000","last_peered":"2024-03-21T10:23:27.750167+0000","last_clean":"2024-03-21T10:23:27.750167+0000","last_became_active":"2024-03-21T10:22:03.307986+0000","last_became_peered":"2024-03-21T10:22:03.307986+0000","last_unstale":"2024-03-21T10:23:27.750167+0000","last_undegraded":"2024-03-21T10:23:27.750167+0000","last_fullsized":"2024-03-21T10:23:27.750167+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:22:02.269614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:22:02.269614+0000","last_clean_scrub_stamp":"2024-03-21T10:22:02.269614+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T21:08:42.241657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"93'5","reported_seq":37,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T10:23:27.742941+0000","last_change":"2024-03-21T10:21:57.851717+0000","last_active":"2024-03-21T10:23:27.742941+0000","last_peered":"2024-03-21T10:23:27.742941+0000","last_clean":"2024-03-21T10:23:27.742941+0000","last_became_active":"2024-03-21T10:21:57.849909+0000","last_became_peered":"2024-03-21T10:21:57.849909+0000","last_unstale":"2024-03-21T10:23:27.742941+0000","last_undegraded":"2024-03-21T10:23:27.742941+0000","last_fullsized":"2024-03-21T10:23:27.742941+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:21:56.788908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:21:56.788908+0000","last_clean_scrub_stamp":"2024-03-21T10:21:56.788908+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T11:41:37.611400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"95'9","reported_seq":39,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T10:23:56.032325+0000","last_change":"2024-03-21T10:21:58.860892+0000","last_active":"2024-03-21T10:23:56.032325+0000","last_peered":"2024-03-21T10:23:56.032325+0000","last_clean":"2024-03-21T10:23:56.032325+0000","last_became_active":"2024-03-21T10:21:58.860403+0000","last_became_peered":"2024-03-21T10:21:58.860403+0000","last_unstale":"2024-03-21T10:23:56.032325+0000","last_undegraded":"2024-03-21T10:23:56.032325+0000","last_fullsized":"2024-03-21T10:23:56.032325+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_clean_scrub_stamp":"2024-03-21T10:21:57.813712+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T15:15:23.303864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"95'9","reported_seq":39,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T10:23:56.010246+0000","last_change":"2024-03-21T10:21:58.869275+0000","last_active":"2024-03-21T10:23:56.010246+0000","last_peered":"2024-03-21T10:23:56.010246+0000","last_clean":"2024-03-21T10:23:56.010246+0000","last_became_active":"2024-03-21T10:21:58.868834+0000","last_became_peered":"2024-03-21T10:21:58.868834+0000","last_unstale":"2024-03-21T10:23:56.010246+0000","last_undegraded":"2024-03-21T10:23:56.010246+0000","last_fullsized":"2024-03-21T10:23:56.010246+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_clean_scrub_stamp":"2024-03-21T10:21:57.813712+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T14:01:11.074702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"95'12","reported_seq":42,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T10:23:56.002992+0000","last_change":"2024-03-21T10:21:58.851658+0000","last_active":"2024-03-21T10:23:56.002992+0000","last_peered":"2024-03-21T10:23:56.002992+0000","last_clean":"2024-03-21T10:23:56.002992+0000","last_became_active":"2024-03-21T10:21:58.851295+0000","last_became_peered":"2024-03-21T10:21:58.851295+0000","last_unstale":"2024-03-21T10:23:56.002992+0000","last_undegraded":"2024-03-21T10:23:56.002992+0000","last_fullsized":"2024-03-21T10:23:56.002992+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_clean_scrub_stamp":"2024-03-21T10:21:57.813712+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T13:15:32.107783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"95'8","reported_seq":38,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T10:23:56.026300+0000","last_change":"2024-03-21T10:21:58.859275+0000","last_active":"2024-03-21T10:23:56.026300+0000","last_peered":"2024-03-21T10:23:56.026300+0000","last_clean":"2024-03-21T10:23:56.026300+0000","last_became_active":"2024-03-21T10:21:58.857054+0000","last_became_peered":"2024-03-21T10:21:58.857054+0000","last_unstale":"2024-03-21T10:23:56.026300+0000","last_undegraded":"2024-03-21T10:23:56.026300+0000","last_fullsized":"2024-03-21T10:23:56.026300+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_clean_scrub_stamp":"2024-03-21T10:21:57.813712+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T20:05:13.819966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"95'7","reported_seq":37,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T10:23:56.022332+0000","last_change":"2024-03-21T10:21:58.869025+0000","last_active":"2024-03-21T10:23:56.022332+0000","last_peered":"2024-03-21T10:23:56.022332+0000","last_clean":"2024-03-21T10:23:56.022332+0000","last_became_active":"2024-03-21T10:21:58.868240+0000","last_became_peered":"2024-03-21T10:21:58.868240+0000","last_unstale":"2024-03-21T10:23:56.022332+0000","last_undegraded":"2024-03-21T10:23:56.022332+0000","last_fullsized":"2024-03-21T10:23:56.022332+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_clean_scrub_stamp":"2024-03-21T10:21:57.813712+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T21:56:45.642916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"95'4","reported_seq":34,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T10:23:56.021712+0000","last_change":"2024-03-21T10:21:58.865024+0000","last_active":"2024-03-21T10:23:56.021712+0000","last_peered":"2024-03-21T10:23:56.021712+0000","last_clean":"2024-03-21T10:23:56.021712+0000","last_became_active":"2024-03-21T10:21:58.864649+0000","last_became_peered":"2024-03-21T10:21:58.864649+0000","last_unstale":"2024-03-21T10:23:56.021712+0000","last_undegraded":"2024-03-21T10:23:56.021712+0000","last_fullsized":"2024-03-21T10:23:56.021712+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_clean_scrub_stamp":"2024-03-21T10:21:57.813712+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T22:09:03.561283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"95'6","reported_seq":36,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T10:23:56.004821+0000","last_change":"2024-03-21T10:21:58.865206+0000","last_active":"2024-03-21T10:23:56.004821+0000","last_peered":"2024-03-21T10:23:56.004821+0000","last_clean":"2024-03-21T10:23:56.004821+0000","last_became_active":"2024-03-21T10:21:58.864669+0000","last_became_peered":"2024-03-21T10:21:58.864669+0000","last_unstale":"2024-03-21T10:23:56.004821+0000","last_undegraded":"2024-03-21T10:23:56.004821+0000","last_fullsized":"2024-03-21T10:23:56.004821+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_clean_scrub_stamp":"2024-03-21T10:21:57.813712+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T13:57:45.668192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"95'15","reported_seq":45,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T10:23:56.044533+0000","last_change":"2024-03-21T10:21:58.862664+0000","last_active":"2024-03-21T10:23:56.044533+0000","last_peered":"2024-03-21T10:23:56.044533+0000","last_clean":"2024-03-21T10:23:56.044533+0000","last_became_active":"2024-03-21T10:21:58.862144+0000","last_became_peered":"2024-03-21T10:21:58.862144+0000","last_unstale":"2024-03-21T10:23:56.044533+0000","last_undegraded":"2024-03-21T10:23:56.044533+0000","last_fullsized":"2024-03-21T10:23:56.044533+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_clean_scrub_stamp":"2024-03-21T10:21:57.813712+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T11:10:10.225102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"95'2","reported_seq":32,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T10:23:56.021964+0000","last_change":"2024-03-21T10:21:58.851000+0000","last_active":"2024-03-21T10:23:56.021964+0000","last_peered":"2024-03-21T10:23:56.021964+0000","last_clean":"2024-03-21T10:23:56.021964+0000","last_became_active":"2024-03-21T10:21:58.850563+0000","last_became_peered":"2024-03-21T10:21:58.850563+0000","last_unstale":"2024-03-21T10:23:56.021964+0000","last_undegraded":"2024-03-21T10:23:56.021964+0000","last_fullsized":"2024-03-21T10:23:56.021964+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_clean_scrub_stamp":"2024-03-21T10:21:57.813712+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T19:39:08.983713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"95'6","reported_seq":36,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T10:23:56.027451+0000","last_change":"2024-03-21T10:21:58.838426+0000","last_active":"2024-03-21T10:23:56.027451+0000","last_peered":"2024-03-21T10:23:56.027451+0000","last_clean":"2024-03-21T10:23:56.027451+0000","last_became_active":"2024-03-21T10:21:58.837625+0000","last_became_peered":"2024-03-21T10:21:58.837625+0000","last_unstale":"2024-03-21T10:23:56.027451+0000","last_undegraded":"2024-03-21T10:23:56.027451+0000","last_fullsized":"2024-03-21T10:23:56.027451+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_clean_scrub_stamp":"2024-03-21T10:21:57.813712+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T16:22:26.523685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"95'8","reported_seq":38,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T10:23:56.022477+0000","last_change":"2024-03-21T10:21:58.854114+0000","last_active":"2024-03-21T10:23:56.022477+0000","last_peered":"2024-03-21T10:23:56.022477+0000","last_clean":"2024-03-21T10:23:56.022477+0000","last_became_active":"2024-03-21T10:21:58.853686+0000","last_became_peered":"2024-03-21T10:21:58.853686+0000","last_unstale":"2024-03-21T10:23:56.022477+0000","last_undegraded":"2024-03-21T10:23:56.022477+0000","last_fullsized":"2024-03-21T10:23:56.022477+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_clean_scrub_stamp":"2024-03-21T10:21:57.813712+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T11:07:27.346766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"95'10","reported_seq":40,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T10:23:56.049981+0000","last_change":"2024-03-21T10:21:58.857778+0000","last_active":"2024-03-21T10:23:56.049981+0000","last_peered":"2024-03-21T10:23:56.049981+0000","last_clean":"2024-03-21T10:23:56.049981+0000","last_became_active":"2024-03-21T10:21:58.857336+0000","last_became_peered":"2024-03-21T10:21:58.857336+0000","last_unstale":"2024-03-21T10:23:56.049981+0000","last_undegraded":"2024-03-21T10:23:56.049981+0000","last_fullsized":"2024-03-21T10:23:56.049981+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_clean_scrub_stamp":"2024-03-21T10:21:57.813712+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T15:22:57.760341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"95'11","reported_seq":41,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T10:23:56.022750+0000","last_change":"2024-03-21T10:21:58.869774+0000","last_active":"2024-03-21T10:23:56.022750+0000","last_peered":"2024-03-21T10:23:56.022750+0000","last_clean":"2024-03-21T10:23:56.022750+0000","last_became_active":"2024-03-21T10:21:58.868815+0000","last_became_peered":"2024-03-21T10:21:58.868815+0000","last_unstale":"2024-03-21T10:23:56.022750+0000","last_undegraded":"2024-03-21T10:23:56.022750+0000","last_fullsized":"2024-03-21T10:23:56.022750+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_clean_scrub_stamp":"2024-03-21T10:21:57.813712+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T16:57:55.082127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"95'8","reported_seq":38,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-03-21T10:23:56.004307+0000","last_change":"2024-03-21T10:21:58.864746+0000","last_active":"2024-03-21T10:23:56.004307+0000","last_peered":"2024-03-21T10:23:56.004307+0000","last_clean":"2024-03-21T10:23:56.004307+0000","last_became_active":"2024-03-21T10:21:58.864366+0000","last_became_peered":"2024-03-21T10:21:58.864366+0000","last_unstale":"2024-03-21T10:23:56.004307+0000","last_undegraded":"2024-03-21T10:23:56.004307+0000","last_fullsized":"2024-03-21T10:23:56.004307+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:21:57.813712+0000","last_clean_scrub_stamp":"2024-03-21T10:21:57.813712+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T13:10:27.303843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":117,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":500,"ondisk_log_size":500,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4751360,"data_stored":4722944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":73774,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":1590,"num_read_kb":876,"num_write":268,"num_write_kb":161,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":4464,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":577536,"data_stored":221322,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":230,"ondisk_log_size":230,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":82,"seq":352187318325,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36396,"kb_used_data":1060,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708756,"statfs":{"total":95995035648,"available":95957766144,"internally_reserved":0,"allocated":1085440,"data_stored":529237,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":36174276},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":74,"seq":317827579964,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32916,"kb_used_data":1740,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712236,"statfs":{"total":95995035648,"available":95961329664,"internally_reserved":0,"allocated":1781760,"data_stored":1183916,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":69,"seq":296352743491,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28096,"kb_used_data":1080,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717056,"statfs":{"total":95995035648,"available":95966265344,"internally_reserved":0,"allocated":1105920,"data_stored":527061,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":61,"seq":261993005130,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36412,"kb_used_data":1076,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708740,"statfs":{"total":95995035648,"available":95957749760,"internally_reserved":0,"allocated":1101824,"data_stored":527439,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":53,"seq":227633266770,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28672,"kb_used_data":1656,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716480,"statfs":{"total":95995035648,"available":95965675520,"internally_reserved":0,"allocated":1695744,"data_stored":1123943,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":48,"seq":206158430297,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36392,"kb_used_data":1056,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708760,"statfs":{"total":95995035648,"available":95957770240,"internally_reserved":0,"allocated":1081344,"data_stored":523058,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574970,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32300,"kb_used_data":1124,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712852,"statfs":{"total":95995035648,"available":95961960448,"internally_reserved":0,"allocated":1150976,"data_stored":588936,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738497,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28096,"kb_used_data":1080,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717056,"statfs":{"total":95995035648,"available":95966265344,"internally_reserved":0,"allocated":1105920,"data_stored":524514,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411445,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36400,"kb_used_data":1064,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708752,"statfs":{"total":95995035648,"available":95957762048,"internally_reserved":0,"allocated":1089536,"data_stored":523686,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":27,"seq":115964117102,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36404,"kb_used_data":1068,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708748,"statfs":{"total":95995035648,"available":95957757952,"internally_reserved":0,"allocated":1093632,"data_stored":522273,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":35,"seq":150323855463,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28160,"kb_used_data":1144,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716992,"statfs":{"total":95995035648,"available":95966199808,"internally_reserved":0,"allocated":1171456,"data_stored":593627,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":40,"seq":171798691935,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":37004,"kb_used_data":1668,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708148,"statfs":{"total":95995035648,"available":95957143552,"internally_reserved":0,"allocated":1708032,"data_stored":1119620,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2671,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":86016,"data_stored":67093,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1843,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":102400,"data_stored":67688,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":3313,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1215,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":3540,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":1500,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":118784,"data_stored":67609,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":3298,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-21T10:25:06.243 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-21T10:25:06.498 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:06.497+0000 7fb429e00640 1 -- 172.21.15.47:0/3479810411 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb4241052b0 msgr2=0x7fb42410bb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:25:06.498 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:06.497+0000 7fb429e00640 1 --2- 172.21.15.47:0/3479810411 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb4241052b0 0x7fb42410bb40 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7fb41c009ca0 tx=0x7fb41c02c0f0 comp rx=0 tx=0).stop 2024-03-21T10:25:06.498 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:06.497+0000 7fb429e00640 1 -- 172.21.15.47:0/3479810411 shutdown_connections 2024-03-21T10:25:06.498 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:06.497+0000 7fb429e00640 1 --2- 172.21.15.47:0/3479810411 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb4241052b0 0x7fb42410bb40 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:25:06.498 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:06.497+0000 7fb429e00640 1 --2- 172.21.15.47:0/3479810411 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb4241048f0 0x7fb424104d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:25:06.498 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:06.497+0000 7fb429e00640 1 --2- 172.21.15.47:0/3479810411 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb424103700 0x7fb424103b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:25:06.499 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:06.497+0000 7fb429e00640 1 -- 172.21.15.47:0/3479810411 >> 172.21.15.47:0/3479810411 conn(0x7fb4240fec70 msgr2=0x7fb4241010b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:25:06.499 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:06.497+0000 7fb429e00640 1 -- 172.21.15.47:0/3479810411 shutdown_connections 2024-03-21T10:25:06.499 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:06.497+0000 7fb429e00640 1 -- 172.21.15.47:0/3479810411 wait complete. 2024-03-21T10:25:06.501 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:06.498+0000 7fb429e00640 1 Processor -- start 2024-03-21T10:25:06.501 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:06.498+0000 7fb429e00640 1 -- start start 2024-03-21T10:25:06.501 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:06.499+0000 7fb429e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb424103700 0x7fb4241998e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:25:06.501 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:06.499+0000 7fb429e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb4241048f0 0x7fb424199e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:25:06.501 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:06.499+0000 7fb429e00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb4241052b0 0x7fb4241a0e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:25:06.501 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:06.499+0000 7fb429e00640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb42410f230 con 0x7fb4241048f0 2024-03-21T10:25:06.501 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:06.499+0000 7fb429e00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb42410f0b0 con 0x7fb424103700 2024-03-21T10:25:06.502 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:06.499+0000 7fb429e00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb42410f3b0 con 0x7fb4241052b0 2024-03-21T10:25:06.502 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:06.499+0000 7fb41be00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb4241048f0 0x7fb424199e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:25:06.502 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:06.499+0000 7fb41be00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb4241048f0 0x7fb424199e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:56324/0 (socket says 172.21.15.47:56324) 2024-03-21T10:25:06.502 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:06.499+0000 7fb41be00640 1 -- 172.21.15.47:0/2727466052 learned_addr learned my addr 172.21.15.47:0/2727466052 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:25:06.502 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:06.499+0000 7fb428a00640 1 --2- 172.21.15.47:0/2727466052 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb424103700 0x7fb4241998e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:25:06.502 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:06.499+0000 7fb429400640 1 --2- 172.21.15.47:0/2727466052 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb4241052b0 0x7fb4241a0e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:25:06.502 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:06.499+0000 7fb41be00640 1 -- 172.21.15.47:0/2727466052 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb4241052b0 msgr2=0x7fb4241a0e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:25:06.502 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:06.499+0000 7fb41be00640 1 --2- 172.21.15.47:0/2727466052 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb4241052b0 0x7fb4241a0e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:25:06.502 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:06.499+0000 7fb41be00640 1 -- 172.21.15.47:0/2727466052 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb424103700 msgr2=0x7fb4241998e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:25:06.502 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:06.499+0000 7fb41be00640 1 --2- 172.21.15.47:0/2727466052 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb424103700 0x7fb4241998e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:25:06.502 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:06.499+0000 7fb41be00640 1 -- 172.21.15.47:0/2727466052 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb4241a1500 con 0x7fb4241048f0 2024-03-21T10:25:06.502 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:06.499+0000 7fb429400640 1 --2- 172.21.15.47:0/2727466052 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb4241052b0 0x7fb4241a0e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T10:25:06.502 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:06.500+0000 7fb41be00640 1 --2- 172.21.15.47:0/2727466052 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb4241048f0 0x7fb424199e20 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7fb40c00d770 tx=0x7fb40c00dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:25:06.503 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:06.500+0000 7fb422000640 1 -- 172.21.15.47:0/2727466052 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb40c017070 con 0x7fb4241048f0 2024-03-21T10:25:06.503 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:06.500+0000 7fb422000640 1 -- 172.21.15.47:0/2727466052 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fb40c0054d0 con 0x7fb4241048f0 2024-03-21T10:25:06.503 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:06.500+0000 7fb422000640 1 -- 172.21.15.47:0/2727466052 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb40c004cf0 con 0x7fb4241048f0 2024-03-21T10:25:06.503 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:06.500+0000 7fb429e00640 1 -- 172.21.15.47:0/2727466052 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb4241a17f0 con 0x7fb4241048f0 2024-03-21T10:25:06.503 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:06.500+0000 7fb429e00640 1 -- 172.21.15.47:0/2727466052 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb4241a1cf0 con 0x7fb4241048f0 2024-03-21T10:25:06.503 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:06.501+0000 7fb429e00640 1 -- 172.21.15.47:0/2727466052 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb42410aaa0 con 0x7fb4241048f0 2024-03-21T10:25:06.504 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:06.503+0000 7fb422000640 1 -- 172.21.15.47:0/2727466052 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 18) v1 ==== 137854+0+0 (secure 0 0 0) 0x7fb40c004e90 con 0x7fb4241048f0 2024-03-21T10:25:06.506 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:06.505+0000 7fb422000640 1 --2- 172.21.15.47:0/2727466052 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7fb4080a2f20 0x7fb4080a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:25:06.506 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:06.505+0000 7fb422000640 1 -- 172.21.15.47:0/2727466052 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9216+0+0 (secure 0 0 0) 0x7fb40c0cf260 con 0x7fb4241048f0 2024-03-21T10:25:06.506 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:06.505+0000 7fb428a00640 1 --2- 172.21.15.47:0/2727466052 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7fb4080a2f20 0x7fb4080a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:25:06.509 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:06.506+0000 7fb428a00640 1 --2- 172.21.15.47:0/2727466052 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7fb4080a2f20 0x7fb4080a53e0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fb414005d00 tx=0x7fb414005c50 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:25:06.511 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:06.510+0000 7fb422000640 1 -- 172.21.15.47:0/2727466052 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fb40c09bc80 con 0x7fb4241048f0 2024-03-21T10:25:06.713 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:25:06 smithi047 ceph-mon[31547]: from='client.15576 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-21T10:25:06.714 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:25:06 smithi047 ceph-mon[31547]: from='client.15582 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-21T10:25:06.717 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:06.716+0000 7fb429e00640 1 -- 172.21.15.47:0/2727466052 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fb4240622a0 con 0x7fb4241048f0 2024-03-21T10:25:06.719 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:06.718+0000 7fb422000640 1 -- 172.21.15.47:0/2727466052 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v95) v1 ==== 74+0+24849 (secure 0 0 0) 0x7fb40c09b580 con 0x7fb4241048f0 2024-03-21T10:25:06.719 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-21T10:25:06.720 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":95,"fsid":"3056d51a-e76b-11ee-95cd-87774f69a715","created":"2024-03-21T10:11:17.019965+0000","modified":"2024-03-21T10:23:27.619804+0000","last_up_change":"2024-03-21T10:20:52.675328+0000","last_in_change":"2024-03-21T10:20:27.648212+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-21T10:15:40.553114+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-21T10:21:56.399518+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-21T10:21:57.812940+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-21T10:22:01.822568+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"91","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"b0c7a102-8d89-428b-b793-cbbe67151f50","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6802","nonce":775837313},{"type":"v1","addr":"172.21.15.47:6803","nonce":775837313}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6804","nonce":775837313},{"type":"v1","addr":"172.21.15.47:6805","nonce":775837313}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6808","nonce":775837313},{"type":"v1","addr":"172.21.15.47:6809","nonce":775837313}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6806","nonce":775837313},{"type":"v1","addr":"172.21.15.47:6807","nonce":775837313}]},"public_addr":"172.21.15.47:6803/775837313","cluster_addr":"172.21.15.47:6805/775837313","heartbeat_back_addr":"172.21.15.47:6809/775837313","heartbeat_front_addr":"172.21.15.47:6807/775837313","state":["exists","up"]},{"osd":1,"uuid":"07bb81bb-7cf3-4bbc-ac9e-802f95185d11","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6800","nonce":2923304710},{"type":"v1","addr":"172.21.15.100:6801","nonce":2923304710}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":2923304710},{"type":"v1","addr":"172.21.15.100:6803","nonce":2923304710}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":2923304710},{"type":"v1","addr":"172.21.15.100:6807","nonce":2923304710}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6804","nonce":2923304710},{"type":"v1","addr":"172.21.15.100:6805","nonce":2923304710}]},"public_addr":"172.21.15.100:6801/2923304710","cluster_addr":"172.21.15.100:6803/2923304710","heartbeat_back_addr":"172.21.15.100:6807/2923304710","heartbeat_front_addr":"172.21.15.100:6805/2923304710","state":["exists","up"]},{"osd":2,"uuid":"88ce76b9-6081-4836-aad0-8c249c56e8a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":171377005},{"type":"v1","addr":"172.21.15.157:6801","nonce":171377005}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":171377005},{"type":"v1","addr":"172.21.15.157:6803","nonce":171377005}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":171377005},{"type":"v1","addr":"172.21.15.157:6807","nonce":171377005}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":171377005},{"type":"v1","addr":"172.21.15.157:6805","nonce":171377005}]},"public_addr":"172.21.15.157:6801/171377005","cluster_addr":"172.21.15.157:6803/171377005","heartbeat_back_addr":"172.21.15.157:6807/171377005","heartbeat_front_addr":"172.21.15.157:6805/171377005","state":["exists","up"]},{"osd":3,"uuid":"518aa33d-263f-45e6-aab5-2201ceb40e74","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":27,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6810","nonce":640760101},{"type":"v1","addr":"172.21.15.47:6811","nonce":640760101}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6812","nonce":640760101},{"type":"v1","addr":"172.21.15.47:6813","nonce":640760101}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6816","nonce":640760101},{"type":"v1","addr":"172.21.15.47:6817","nonce":640760101}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6814","nonce":640760101},{"type":"v1","addr":"172.21.15.47:6815","nonce":640760101}]},"public_addr":"172.21.15.47:6811/640760101","cluster_addr":"172.21.15.47:6813/640760101","heartbeat_back_addr":"172.21.15.47:6817/640760101","heartbeat_front_addr":"172.21.15.47:6815/640760101","state":["exists","up"]},{"osd":4,"uuid":"987bde83-ecc6-42e5-bd56-c143b3de4226","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":35,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6808","nonce":2785478639},{"type":"v1","addr":"172.21.15.100:6809","nonce":2785478639}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6810","nonce":2785478639},{"type":"v1","addr":"172.21.15.100:6811","nonce":2785478639}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6814","nonce":2785478639},{"type":"v1","addr":"172.21.15.100:6815","nonce":2785478639}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":2785478639},{"type":"v1","addr":"172.21.15.100:6813","nonce":2785478639}]},"public_addr":"172.21.15.100:6809/2785478639","cluster_addr":"172.21.15.100:6811/2785478639","heartbeat_back_addr":"172.21.15.100:6815/2785478639","heartbeat_front_addr":"172.21.15.100:6813/2785478639","state":["exists","up"]},{"osd":5,"uuid":"012d2188-3a61-4a5f-8d01-2001c15ffbe2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":40,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":1525645410},{"type":"v1","addr":"172.21.15.157:6809","nonce":1525645410}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":1525645410},{"type":"v1","addr":"172.21.15.157:6811","nonce":1525645410}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":1525645410},{"type":"v1","addr":"172.21.15.157:6815","nonce":1525645410}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":1525645410},{"type":"v1","addr":"172.21.15.157:6813","nonce":1525645410}]},"public_addr":"172.21.15.157:6809/1525645410","cluster_addr":"172.21.15.157:6811/1525645410","heartbeat_back_addr":"172.21.15.157:6815/1525645410","heartbeat_front_addr":"172.21.15.157:6813/1525645410","state":["exists","up"]},{"osd":6,"uuid":"c12c14b4-85f0-4d88-937c-de4f556437de","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6818","nonce":2321642790},{"type":"v1","addr":"172.21.15.47:6819","nonce":2321642790}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6820","nonce":2321642790},{"type":"v1","addr":"172.21.15.47:6821","nonce":2321642790}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6824","nonce":2321642790},{"type":"v1","addr":"172.21.15.47:6825","nonce":2321642790}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6822","nonce":2321642790},{"type":"v1","addr":"172.21.15.47:6823","nonce":2321642790}]},"public_addr":"172.21.15.47:6819/2321642790","cluster_addr":"172.21.15.47:6821/2321642790","heartbeat_back_addr":"172.21.15.47:6825/2321642790","heartbeat_front_addr":"172.21.15.47:6823/2321642790","state":["exists","up"]},{"osd":7,"uuid":"3c87c6d0-1d1f-4294-9917-aaaad7dda824","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6816","nonce":4104727330},{"type":"v1","addr":"172.21.15.100:6817","nonce":4104727330}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":4104727330},{"type":"v1","addr":"172.21.15.100:6819","nonce":4104727330}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":4104727330},{"type":"v1","addr":"172.21.15.100:6823","nonce":4104727330}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6820","nonce":4104727330},{"type":"v1","addr":"172.21.15.100:6821","nonce":4104727330}]},"public_addr":"172.21.15.100:6817/4104727330","cluster_addr":"172.21.15.100:6819/4104727330","heartbeat_back_addr":"172.21.15.100:6823/4104727330","heartbeat_front_addr":"172.21.15.100:6821/4104727330","state":["exists","up"]},{"osd":8,"uuid":"502d65cf-53c8-4936-8de4-e11b2a6e1e18","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":61,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":4067715347},{"type":"v1","addr":"172.21.15.157:6817","nonce":4067715347}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":4067715347},{"type":"v1","addr":"172.21.15.157:6819","nonce":4067715347}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":4067715347},{"type":"v1","addr":"172.21.15.157:6823","nonce":4067715347}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":4067715347},{"type":"v1","addr":"172.21.15.157:6821","nonce":4067715347}]},"public_addr":"172.21.15.157:6817/4067715347","cluster_addr":"172.21.15.157:6819/4067715347","heartbeat_back_addr":"172.21.15.157:6823/4067715347","heartbeat_front_addr":"172.21.15.157:6821/4067715347","state":["exists","up"]},{"osd":9,"uuid":"5ea1f5be-8458-4c93-9e61-c2e867a514f3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6826","nonce":1982108738},{"type":"v1","addr":"172.21.15.47:6827","nonce":1982108738}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6828","nonce":1982108738},{"type":"v1","addr":"172.21.15.47:6829","nonce":1982108738}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6832","nonce":1982108738},{"type":"v1","addr":"172.21.15.47:6833","nonce":1982108738}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6830","nonce":1982108738},{"type":"v1","addr":"172.21.15.47:6831","nonce":1982108738}]},"public_addr":"172.21.15.47:6827/1982108738","cluster_addr":"172.21.15.47:6829/1982108738","heartbeat_back_addr":"172.21.15.47:6833/1982108738","heartbeat_front_addr":"172.21.15.47:6831/1982108738","state":["exists","up"]},{"osd":10,"uuid":"2da3da90-a0be-4a54-a9ff-b2d6c9b36870","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":74,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":396391028},{"type":"v1","addr":"172.21.15.100:6825","nonce":396391028}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":396391028},{"type":"v1","addr":"172.21.15.100:6827","nonce":396391028}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6830","nonce":396391028},{"type":"v1","addr":"172.21.15.100:6831","nonce":396391028}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":396391028},{"type":"v1","addr":"172.21.15.100:6829","nonce":396391028}]},"public_addr":"172.21.15.100:6825/396391028","cluster_addr":"172.21.15.100:6827/396391028","heartbeat_back_addr":"172.21.15.100:6831/396391028","heartbeat_front_addr":"172.21.15.100:6829/396391028","state":["exists","up"]},{"osd":11,"uuid":"11798d41-7f7e-4319-95b8-8d3d0946bde9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":82,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":2671029483},{"type":"v1","addr":"172.21.15.157:6825","nonce":2671029483}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":2671029483},{"type":"v1","addr":"172.21.15.157:6827","nonce":2671029483}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":2671029483},{"type":"v1","addr":"172.21.15.157:6831","nonce":2671029483}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":2671029483},{"type":"v1","addr":"172.21.15.157:6829","nonce":2671029483}]},"public_addr":"172.21.15.157:6825/2671029483","cluster_addr":"172.21.15.157:6827/2671029483","heartbeat_back_addr":"172.21.15.157:6831/2671029483","heartbeat_front_addr":"172.21.15.157:6829/2671029483","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T10:14:31.822399+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T10:15:04.175721+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T10:15:35.902595+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T10:16:12.088891+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T10:16:46.428704+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T10:17:20.158270+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T10:17:55.915253+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T10:18:30.064641+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T10:19:04.410004+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T10:19:40.605181+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T10:20:15.636605+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T10:20:50.151365+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.11","mappings":[{"from":7,"to":11}]},{"pgid":"2.14","mappings":[{"from":2,"to":4}]},{"pgid":"2.1a","mappings":[{"from":8,"to":1}]},{"pgid":"2.1b","mappings":[{"from":2,"to":4}]},{"pgid":"2.1d","mappings":[{"from":8,"to":11}]},{"pgid":"3.0","mappings":[{"from":4,"to":8},{"from":5,"to":9}]},{"pgid":"3.6","mappings":[{"from":3,"to":7}]},{"pgid":"3.1b","mappings":[{"from":4,"to":6}]},{"pgid":"3.33","mappings":[{"from":5,"to":9}]},{"pgid":"3.3c","mappings":[{"from":4,"to":9}]},{"pgid":"4.c","mappings":[{"from":9,"to":10}]},{"pgid":"4.f","mappings":[{"from":9,"to":8}]},{"pgid":"4.19","mappings":[{"from":9,"to":8}]},{"pgid":"4.1d","mappings":[{"from":5,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.47:0/356822431":"2024-03-22T10:12:25.844530+0000","172.21.15.47:0/1167490879":"2024-03-22T10:12:25.844530+0000","172.21.15.47:0/1540718138":"2024-03-22T10:12:25.844530+0000","172.21.15.47:6801/3836903972":"2024-03-22T10:12:25.844530+0000","172.21.15.47:0/235436602":"2024-03-22T10:11:50.301838+0000","172.21.15.47:0/2052815572":"2024-03-22T10:11:50.301838+0000","172.21.15.47:6800/3942658432":"2024-03-22T10:11:50.301838+0000","172.21.15.47:6801/3942658432":"2024-03-22T10:11:50.301838+0000","172.21.15.47:0/1419516257":"2024-03-22T10:11:50.301838+0000","172.21.15.47:6800/3836903972":"2024-03-22T10:12:25.844530+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-21T10:25:06.722 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:06.721+0000 7fb429e00640 1 -- 172.21.15.47:0/2727466052 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7fb4080a2f20 msgr2=0x7fb4080a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:25:06.722 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:06.721+0000 7fb429e00640 1 --2- 172.21.15.47:0/2727466052 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7fb4080a2f20 0x7fb4080a53e0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fb414005d00 tx=0x7fb414005c50 comp rx=0 tx=0).stop 2024-03-21T10:25:06.722 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:06.721+0000 7fb429e00640 1 -- 172.21.15.47:0/2727466052 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb4241048f0 msgr2=0x7fb424199e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:25:06.722 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:06.721+0000 7fb429e00640 1 --2- 172.21.15.47:0/2727466052 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb4241048f0 0x7fb424199e20 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7fb40c00d770 tx=0x7fb40c00dc30 comp rx=0 tx=0).stop 2024-03-21T10:25:06.723 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:06.721+0000 7fb429e00640 1 -- 172.21.15.47:0/2727466052 shutdown_connections 2024-03-21T10:25:06.723 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:06.721+0000 7fb429e00640 1 --2- 172.21.15.47:0/2727466052 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7fb4080a2f20 0x7fb4080a53e0 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:25:06.723 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:06.721+0000 7fb429e00640 1 --2- 172.21.15.47:0/2727466052 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb4241052b0 0x7fb4241a0e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:25:06.723 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:06.721+0000 7fb429e00640 1 --2- 172.21.15.47:0/2727466052 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb4241048f0 0x7fb424199e20 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:25:06.723 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:06.721+0000 7fb429e00640 1 --2- 172.21.15.47:0/2727466052 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb424103700 0x7fb4241998e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:25:06.723 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:06.721+0000 7fb429e00640 1 -- 172.21.15.47:0/2727466052 >> 172.21.15.47:0/2727466052 conn(0x7fb4240fec70 msgr2=0x7fb424107c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:25:06.723 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:06.721+0000 7fb429e00640 1 -- 172.21.15.47:0/2727466052 shutdown_connections 2024-03-21T10:25:06.723 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:06.722+0000 7fb429e00640 1 -- 172.21.15.47:0/2727466052 wait complete. 2024-03-21T10:25:06.729 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:25:06 smithi100 ceph-mon[35193]: from='client.15576 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-21T10:25:06.729 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:25:06 smithi100 ceph-mon[35193]: from='client.15582 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-21T10:25:06.740 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:25:06 smithi157 ceph-mon[35581]: from='client.15576 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-21T10:25:06.740 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:25:06 smithi157 ceph-mon[35581]: from='client.15582 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-21T10:25:06.743 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-21T10:15:40.553114+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-21T10:21:56.399518+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '86', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-21T10:21:57.812940+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '87', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-03-21T10:22:01.822568+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '91', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-03-21T10:25:06.744 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-21T10:25:06.968 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:06.967+0000 7fed0dc00640 1 -- 172.21.15.47:0/1734203153 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fed08104990 msgr2=0x7fed08104e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:25:06.968 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:06.967+0000 7fed0dc00640 1 --2- 172.21.15.47:0/1734203153 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fed08104990 0x7fed08104e10 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7fecf4009900 tx=0x7fecf402bc30 comp rx=0 tx=0).stop 2024-03-21T10:25:06.968 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:06.967+0000 7fed0dc00640 1 -- 172.21.15.47:0/1734203153 shutdown_connections 2024-03-21T10:25:06.968 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:06.967+0000 7fed0dc00640 1 --2- 172.21.15.47:0/1734203153 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fed08105350 0x7fed0810bbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:25:06.968 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:06.967+0000 7fed0dc00640 1 --2- 172.21.15.47:0/1734203153 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fed08104990 0x7fed08104e10 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:25:06.968 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:06.967+0000 7fed0dc00640 1 --2- 172.21.15.47:0/1734203153 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fed081037a0 0x7fed08103bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:25:06.969 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:06.967+0000 7fed0dc00640 1 -- 172.21.15.47:0/1734203153 >> 172.21.15.47:0/1734203153 conn(0x7fed080fed10 msgr2=0x7fed08101150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:25:06.969 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:06.967+0000 7fed0dc00640 1 -- 172.21.15.47:0/1734203153 shutdown_connections 2024-03-21T10:25:06.969 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:06.967+0000 7fed0dc00640 1 -- 172.21.15.47:0/1734203153 wait complete. 2024-03-21T10:25:06.971 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:06.968+0000 7fed0dc00640 1 Processor -- start 2024-03-21T10:25:06.971 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:06.968+0000 7fed0dc00640 1 -- start start 2024-03-21T10:25:06.971 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:06.968+0000 7fed0dc00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fed081037a0 0x7fed08199990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:25:06.971 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:06.968+0000 7fed0dc00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fed08104990 0x7fed08199ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:25:06.971 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:06.968+0000 7fed0dc00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fed08105350 0x7fed081a0f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:25:06.971 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:06.968+0000 7fed0dc00640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed0810f320 con 0x7fed08105350 2024-03-21T10:25:06.971 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:06.968+0000 7fed0dc00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed0810f1a0 con 0x7fed081037a0 2024-03-21T10:25:06.971 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:06.968+0000 7fed0dc00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed0810f4a0 con 0x7fed08104990 2024-03-21T10:25:06.971 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:06.969+0000 7fed0d200640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fed08105350 0x7fed081a0f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:25:06.971 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:06.969+0000 7fed07e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fed081037a0 0x7fed08199990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:25:06.971 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:06.969+0000 7fed0d200640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fed08105350 0x7fed081a0f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:56352/0 (socket says 172.21.15.47:56352) 2024-03-21T10:25:06.972 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:06.969+0000 7fed0d200640 1 -- 172.21.15.47:0/4096092142 learned_addr learned my addr 172.21.15.47:0/4096092142 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:25:06.972 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:06.969+0000 7fed07400640 1 --2- 172.21.15.47:0/4096092142 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fed08104990 0x7fed08199ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:25:06.972 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:06.969+0000 7fed0d200640 1 -- 172.21.15.47:0/4096092142 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fed08104990 msgr2=0x7fed08199ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:25:06.972 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:06.969+0000 7fed0d200640 1 --2- 172.21.15.47:0/4096092142 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fed08104990 0x7fed08199ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:25:06.972 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:06.969+0000 7fed0d200640 1 -- 172.21.15.47:0/4096092142 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fed081037a0 msgr2=0x7fed08199990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:25:06.972 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:06.969+0000 7fed0d200640 1 --2- 172.21.15.47:0/4096092142 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fed081037a0 0x7fed08199990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:25:06.972 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:06.969+0000 7fed0d200640 1 -- 172.21.15.47:0/4096092142 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed081a15b0 con 0x7fed08105350 2024-03-21T10:25:06.972 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:06.970+0000 7fed0d200640 1 --2- 172.21.15.47:0/4096092142 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fed08105350 0x7fed081a0f40 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7fed0000eaa0 tx=0x7fed0000ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:25:06.972 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:06.970+0000 7fed04c00640 1 -- 172.21.15.47:0/4096092142 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed0000cdb0 con 0x7fed08105350 2024-03-21T10:25:06.972 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:06.970+0000 7fed04c00640 1 -- 172.21.15.47:0/4096092142 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fed00005020 con 0x7fed08105350 2024-03-21T10:25:06.972 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:06.970+0000 7fed04c00640 1 -- 172.21.15.47:0/4096092142 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed000116d0 con 0x7fed08105350 2024-03-21T10:25:06.972 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:06.970+0000 7fed0dc00640 1 -- 172.21.15.47:0/4096092142 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fed081a18a0 con 0x7fed08105350 2024-03-21T10:25:06.972 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:06.970+0000 7fed0dc00640 1 -- 172.21.15.47:0/4096092142 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fed081a1e10 con 0x7fed08105350 2024-03-21T10:25:06.973 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:06.972+0000 7fed0dc00640 1 -- 172.21.15.47:0/4096092142 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fed0810aac0 con 0x7fed08105350 2024-03-21T10:25:06.974 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:06.973+0000 7fed04c00640 1 -- 172.21.15.47:0/4096092142 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 18) v1 ==== 137854+0+0 (secure 0 0 0) 0x7fed000051d0 con 0x7fed08105350 2024-03-21T10:25:06.976 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:06.975+0000 7fed04c00640 1 --2- 172.21.15.47:0/4096092142 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7fecec0a2fa0 0x7fecec0a5460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:25:06.976 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:06.975+0000 7fed04c00640 1 -- 172.21.15.47:0/4096092142 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9216+0+0 (secure 0 0 0) 0x7fed000cf960 con 0x7fed08105350 2024-03-21T10:25:06.976 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:06.975+0000 7fed07e00640 1 --2- 172.21.15.47:0/4096092142 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7fecec0a2fa0 0x7fecec0a5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:25:06.976 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:06.976+0000 7fed07e00640 1 --2- 172.21.15.47:0/4096092142 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7fecec0a2fa0 0x7fecec0a5460 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7fed081047f0 tx=0x7fecf8005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:25:06.980 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:06.980+0000 7fed04c00640 1 -- 172.21.15.47:0/4096092142 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fed00016030 con 0x7fed08105350 2024-03-21T10:25:07.185 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:07.184+0000 7fed0dc00640 1 -- 172.21.15.47:0/4096092142 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fed080622a0 con 0x7fed08105350 2024-03-21T10:25:07.186 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:07.185+0000 7fed04c00640 1 -- 172.21.15.47:0/4096092142 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v95) v1 ==== 93+0+10 (secure 0 0 0) 0x7fed0009c380 con 0x7fed08105350 2024-03-21T10:25:07.186 INFO:teuthology.orchestra.run.smithi047.stdout:pg_num: 1 2024-03-21T10:25:07.189 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:07.188+0000 7fed0dc00640 1 -- 172.21.15.47:0/4096092142 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7fecec0a2fa0 msgr2=0x7fecec0a5460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:25:07.189 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:07.188+0000 7fed0dc00640 1 --2- 172.21.15.47:0/4096092142 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7fecec0a2fa0 0x7fecec0a5460 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7fed081047f0 tx=0x7fecf8005f90 comp rx=0 tx=0).stop 2024-03-21T10:25:07.189 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:07.188+0000 7fed0dc00640 1 -- 172.21.15.47:0/4096092142 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fed08105350 msgr2=0x7fed081a0f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:25:07.189 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:07.188+0000 7fed0dc00640 1 --2- 172.21.15.47:0/4096092142 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fed08105350 0x7fed081a0f40 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7fed0000eaa0 tx=0x7fed0000ef60 comp rx=0 tx=0).stop 2024-03-21T10:25:07.189 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:07.188+0000 7fed0dc00640 1 -- 172.21.15.47:0/4096092142 shutdown_connections 2024-03-21T10:25:07.189 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:07.188+0000 7fed0dc00640 1 --2- 172.21.15.47:0/4096092142 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7fecec0a2fa0 0x7fecec0a5460 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:25:07.190 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:07.188+0000 7fed0dc00640 1 --2- 172.21.15.47:0/4096092142 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fed08105350 0x7fed081a0f40 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:25:07.190 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:07.188+0000 7fed0dc00640 1 --2- 172.21.15.47:0/4096092142 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fed08104990 0x7fed08199ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:25:07.190 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:07.188+0000 7fed0dc00640 1 --2- 172.21.15.47:0/4096092142 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fed081037a0 0x7fed08199990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:25:07.190 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:07.188+0000 7fed0dc00640 1 -- 172.21.15.47:0/4096092142 >> 172.21.15.47:0/4096092142 conn(0x7fed080fed10 msgr2=0x7fed08107c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:25:07.190 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:07.188+0000 7fed0dc00640 1 -- 172.21.15.47:0/4096092142 shutdown_connections 2024-03-21T10:25:07.190 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:07.189+0000 7fed0dc00640 1 -- 172.21.15.47:0/4096092142 wait complete. 2024-03-21T10:25:07.210 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-21T10:25:07.436 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:07.435+0000 7f1509a00640 1 -- 172.21.15.47:0/4244811674 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f15041016b0 msgr2=0x7f1504101ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:25:07.436 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:07.435+0000 7f1509a00640 1 --2- 172.21.15.47:0/4244811674 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f15041016b0 0x7f1504101ad0 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f14f4009900 tx=0x7f14f402bc30 comp rx=0 tx=0).stop 2024-03-21T10:25:07.436 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:07.435+0000 7f1509a00640 1 -- 172.21.15.47:0/4244811674 shutdown_connections 2024-03-21T10:25:07.436 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:07.435+0000 7f1509a00640 1 --2- 172.21.15.47:0/4244811674 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1504103260 0x7f1504109af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:25:07.437 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:07.435+0000 7f1509a00640 1 --2- 172.21.15.47:0/4244811674 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f15041028a0 0x7f1504102d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:25:07.437 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:07.435+0000 7f1509a00640 1 --2- 172.21.15.47:0/4244811674 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f15041016b0 0x7f1504101ad0 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:25:07.437 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:07.435+0000 7f1509a00640 1 -- 172.21.15.47:0/4244811674 >> 172.21.15.47:0/4244811674 conn(0x7f15040fcc40 msgr2=0x7f15040ff060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:25:07.437 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:07.435+0000 7f1509a00640 1 -- 172.21.15.47:0/4244811674 shutdown_connections 2024-03-21T10:25:07.437 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:07.435+0000 7f1509a00640 1 -- 172.21.15.47:0/4244811674 wait complete. 2024-03-21T10:25:07.439 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:07.436+0000 7f1509a00640 1 Processor -- start 2024-03-21T10:25:07.439 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:07.437+0000 7f1509a00640 1 -- start start 2024-03-21T10:25:07.439 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:07.437+0000 7f1509a00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f15041016b0 0x7f1504199b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:25:07.439 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:07.437+0000 7f1509a00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f15041028a0 0x7f150419a090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:25:07.439 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:07.437+0000 7f1509a00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1504103260 0x7f15041a1100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:25:07.439 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:07.437+0000 7f1509a00640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f150410d1b0 con 0x7f15041016b0 2024-03-21T10:25:07.439 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:07.437+0000 7f1509a00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f150410d030 con 0x7f15041028a0 2024-03-21T10:25:07.439 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:07.437+0000 7f1509a00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f150410d330 con 0x7f1504103260 2024-03-21T10:25:07.439 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:07.437+0000 7f1502a00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f15041016b0 0x7f1504199b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:25:07.439 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:07.437+0000 7f1502a00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f15041016b0 0x7f1504199b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:56378/0 (socket says 172.21.15.47:56378) 2024-03-21T10:25:07.439 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:07.437+0000 7f1502a00640 1 -- 172.21.15.47:0/2841534003 learned_addr learned my addr 172.21.15.47:0/2841534003 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:25:07.439 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:07.437+0000 7f14fbe00640 1 --2- 172.21.15.47:0/2841534003 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f15041028a0 0x7f150419a090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:25:07.439 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:07.438+0000 7f1503400640 1 --2- 172.21.15.47:0/2841534003 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1504103260 0x7f15041a1100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:25:07.440 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:07.438+0000 7f1502a00640 1 -- 172.21.15.47:0/2841534003 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1504103260 msgr2=0x7f15041a1100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:25:07.440 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:07.438+0000 7f1502a00640 1 --2- 172.21.15.47:0/2841534003 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1504103260 0x7f15041a1100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:25:07.440 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:07.438+0000 7f1502a00640 1 -- 172.21.15.47:0/2841534003 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f15041028a0 msgr2=0x7f150419a090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:25:07.440 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:07.438+0000 7f1502a00640 1 --2- 172.21.15.47:0/2841534003 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f15041028a0 0x7f150419a090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:25:07.440 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:07.438+0000 7f1502a00640 1 -- 172.21.15.47:0/2841534003 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15041a1770 con 0x7f15041016b0 2024-03-21T10:25:07.440 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:07.438+0000 7f1502a00640 1 --2- 172.21.15.47:0/2841534003 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f15041016b0 0x7f1504199b50 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f14f4000c00 tx=0x7f14f4004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:25:07.441 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:07.438+0000 7f14fb400640 1 -- 172.21.15.47:0/2841534003 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14f4043020 con 0x7f15041016b0 2024-03-21T10:25:07.441 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:07.438+0000 7f14fb400640 1 -- 172.21.15.47:0/2841534003 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f14f4004440 con 0x7f15041016b0 2024-03-21T10:25:07.441 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:07.438+0000 7f1509a00640 1 -- 172.21.15.47:0/2841534003 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f15041a1a00 con 0x7f15041016b0 2024-03-21T10:25:07.441 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:07.438+0000 7f14fb400640 1 -- 172.21.15.47:0/2841534003 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14f4003c90 con 0x7f15041016b0 2024-03-21T10:25:07.441 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:07.439+0000 7f1509a00640 1 -- 172.21.15.47:0/2841534003 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f15041a1f50 con 0x7f15041016b0 2024-03-21T10:25:07.441 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:07.440+0000 7f1509a00640 1 -- 172.21.15.47:0/2841534003 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f15040688e0 con 0x7f15041016b0 2024-03-21T10:25:07.443 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:07.442+0000 7f14fb400640 1 -- 172.21.15.47:0/2841534003 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 18) v1 ==== 137854+0+0 (secure 0 0 0) 0x7f14f4046020 con 0x7f15041016b0 2024-03-21T10:25:07.444 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:07.444+0000 7f14fb400640 1 --2- 172.21.15.47:0/2841534003 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f14e80a2f20 0x7f14e80a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:25:07.445 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:07.444+0000 7f14fb400640 1 -- 172.21.15.47:0/2841534003 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9216+0+0 (secure 0 0 0) 0x7f14f40f1e90 con 0x7f15041016b0 2024-03-21T10:25:07.445 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:07.444+0000 7f14fbe00640 1 --2- 172.21.15.47:0/2841534003 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f14e80a2f20 0x7f14e80a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:25:07.445 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:07.445+0000 7f14fbe00640 1 --2- 172.21.15.47:0/2841534003 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f14e80a2f20 0x7f14e80a53e0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f150419b060 tx=0x7f14ec005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:25:07.449 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:07.449+0000 7f14fb400640 1 -- 172.21.15.47:0/2841534003 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f14f404d020 con 0x7f15041016b0 2024-03-21T10:25:07.463 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:25:07 smithi047 ceph-mon[31547]: pgmap v465: 129 pgs: 129 active+clean; 649 KiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 1.7 KiB/s wr, 10 op/s 2024-03-21T10:25:07.463 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:25:07 smithi047 ceph-mon[31547]: from='client.15588 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-21T10:25:07.463 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:25:07 smithi047 ceph-mon[31547]: from='client.? 172.21.15.47:0/2727466052' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-21T10:25:07.463 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:25:07 smithi047 ceph-mon[31547]: from='client.? 172.21.15.47:0/4096092142' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-21T10:25:07.654 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:07.654+0000 7f1509a00640 1 -- 172.21.15.47:0/2841534003 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f15041a2330 con 0x7f15041016b0 2024-03-21T10:25:07.655 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:07.654+0000 7f14fb400640 1 -- 172.21.15.47:0/2841534003 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v95) v1 ==== 104+0+11 (secure 0 0 0) 0x7f14f40be9d0 con 0x7f15041016b0 2024-03-21T10:25:07.655 INFO:teuthology.orchestra.run.smithi047.stdout:pg_num: 32 2024-03-21T10:25:07.659 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:07.657+0000 7f1509a00640 1 -- 172.21.15.47:0/2841534003 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f14e80a2f20 msgr2=0x7f14e80a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:25:07.659 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:07.657+0000 7f1509a00640 1 --2- 172.21.15.47:0/2841534003 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f14e80a2f20 0x7f14e80a53e0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f150419b060 tx=0x7f14ec005f90 comp rx=0 tx=0).stop 2024-03-21T10:25:07.659 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:07.657+0000 7f1509a00640 1 -- 172.21.15.47:0/2841534003 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f15041016b0 msgr2=0x7f1504199b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:25:07.659 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:07.657+0000 7f1509a00640 1 --2- 172.21.15.47:0/2841534003 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f15041016b0 0x7f1504199b50 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f14f4000c00 tx=0x7f14f4004040 comp rx=0 tx=0).stop 2024-03-21T10:25:07.659 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:07.657+0000 7f1509a00640 1 -- 172.21.15.47:0/2841534003 shutdown_connections 2024-03-21T10:25:07.659 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:07.657+0000 7f1509a00640 1 --2- 172.21.15.47:0/2841534003 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f14e80a2f20 0x7f14e80a53e0 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:25:07.659 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:07.657+0000 7f1509a00640 1 --2- 172.21.15.47:0/2841534003 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1504103260 0x7f15041a1100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:25:07.659 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:07.657+0000 7f1509a00640 1 --2- 172.21.15.47:0/2841534003 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f15041028a0 0x7f150419a090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:25:07.660 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:07.657+0000 7f1509a00640 1 --2- 172.21.15.47:0/2841534003 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f15041016b0 0x7f1504199b50 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:25:07.660 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:07.657+0000 7f1509a00640 1 -- 172.21.15.47:0/2841534003 >> 172.21.15.47:0/2841534003 conn(0x7f15040fcc40 msgr2=0x7f1504108040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:25:07.660 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:07.658+0000 7f1509a00640 1 -- 172.21.15.47:0/2841534003 shutdown_connections 2024-03-21T10:25:07.660 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:07.658+0000 7f1509a00640 1 -- 172.21.15.47:0/2841534003 wait complete. 2024-03-21T10:25:07.677 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-21T10:25:07.729 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:25:07 smithi100 ceph-mon[35193]: pgmap v465: 129 pgs: 129 active+clean; 649 KiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 1.7 KiB/s wr, 10 op/s 2024-03-21T10:25:07.729 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:25:07 smithi100 ceph-mon[35193]: from='client.15588 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-21T10:25:07.729 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:25:07 smithi100 ceph-mon[35193]: from='client.? 172.21.15.47:0/2727466052' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-21T10:25:07.729 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:25:07 smithi100 ceph-mon[35193]: from='client.? 172.21.15.47:0/4096092142' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-21T10:25:07.739 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:25:07 smithi157 ceph-mon[35581]: pgmap v465: 129 pgs: 129 active+clean; 649 KiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 1.7 KiB/s wr, 10 op/s 2024-03-21T10:25:07.740 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:25:07 smithi157 ceph-mon[35581]: from='client.15588 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-21T10:25:07.740 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:25:07 smithi157 ceph-mon[35581]: from='client.? 172.21.15.47:0/2727466052' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-21T10:25:07.740 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:25:07 smithi157 ceph-mon[35581]: from='client.? 172.21.15.47:0/4096092142' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-21T10:25:07.899 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:07.897+0000 7f0734a00640 1 -- 172.21.15.47:0/3342427046 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f07301038e0 msgr2=0x7f0730103d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:25:07.899 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:07.897+0000 7f0734a00640 1 --2- 172.21.15.47:0/3342427046 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f07301038e0 0x7f0730103d00 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f0720009900 tx=0x7f072002bc30 comp rx=0 tx=0).stop 2024-03-21T10:25:07.899 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:07.898+0000 7f0734a00640 1 -- 172.21.15.47:0/3342427046 shutdown_connections 2024-03-21T10:25:07.899 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:07.898+0000 7f0734a00640 1 --2- 172.21.15.47:0/3342427046 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0730105490 0x7f073010bd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:25:07.899 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:07.898+0000 7f0734a00640 1 --2- 172.21.15.47:0/3342427046 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0730104ad0 0x7f0730104f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:25:07.899 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:07.898+0000 7f0734a00640 1 --2- 172.21.15.47:0/3342427046 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f07301038e0 0x7f0730103d00 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:25:07.899 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:07.898+0000 7f0734a00640 1 -- 172.21.15.47:0/3342427046 >> 172.21.15.47:0/3342427046 conn(0x7f07300fee50 msgr2=0x7f0730101290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:25:07.899 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:07.898+0000 7f0734a00640 1 -- 172.21.15.47:0/3342427046 shutdown_connections 2024-03-21T10:25:07.899 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:07.898+0000 7f0734a00640 1 -- 172.21.15.47:0/3342427046 wait complete. 2024-03-21T10:25:07.902 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:07.899+0000 7f0734a00640 1 Processor -- start 2024-03-21T10:25:07.902 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:07.899+0000 7f0734a00640 1 -- start start 2024-03-21T10:25:07.902 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:07.899+0000 7f0734a00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f07301038e0 0x7f0730199b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:25:07.902 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:07.899+0000 7f0734a00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0730104ad0 0x7f073019a050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:25:07.902 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:07.899+0000 7f0734a00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0730105490 0x7f07301a10c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:25:07.902 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:07.899+0000 7f0734a00640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f073010f430 con 0x7f07301038e0 2024-03-21T10:25:07.902 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:07.899+0000 7f0734a00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f073010f2b0 con 0x7f0730104ad0 2024-03-21T10:25:07.902 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:07.899+0000 7f0734a00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f073010f5b0 con 0x7f0730105490 2024-03-21T10:25:07.902 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:07.899+0000 7f072de00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f07301038e0 0x7f0730199b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:25:07.902 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:07.899+0000 7f072de00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f07301038e0 0x7f0730199b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:56384/0 (socket says 172.21.15.47:56384) 2024-03-21T10:25:07.902 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:07.899+0000 7f072de00640 1 -- 172.21.15.47:0/3687199128 learned_addr learned my addr 172.21.15.47:0/3687199128 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:25:07.902 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:07.900+0000 7f072d400640 1 --2- 172.21.15.47:0/3687199128 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0730104ad0 0x7f073019a050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:25:07.903 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:07.900+0000 7f072e800640 1 --2- 172.21.15.47:0/3687199128 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0730105490 0x7f07301a10c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:25:07.903 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:07.900+0000 7f072de00640 1 -- 172.21.15.47:0/3687199128 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0730105490 msgr2=0x7f07301a10c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:25:07.903 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:07.900+0000 7f072de00640 1 --2- 172.21.15.47:0/3687199128 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0730105490 0x7f07301a10c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:25:07.903 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:07.900+0000 7f072de00640 1 -- 172.21.15.47:0/3687199128 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0730104ad0 msgr2=0x7f073019a050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:25:07.903 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:07.900+0000 7f072de00640 1 --2- 172.21.15.47:0/3687199128 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0730104ad0 0x7f073019a050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:25:07.903 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:07.900+0000 7f072de00640 1 -- 172.21.15.47:0/3687199128 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f07301a1730 con 0x7f07301038e0 2024-03-21T10:25:07.903 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:07.900+0000 7f072de00640 1 --2- 172.21.15.47:0/3687199128 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f07301038e0 0x7f0730199b10 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f0720000c00 tx=0x7f07200040c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:25:07.903 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:07.901+0000 7f0726a00640 1 -- 172.21.15.47:0/3687199128 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0720043020 con 0x7f07301038e0 2024-03-21T10:25:07.903 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:07.901+0000 7f0726a00640 1 -- 172.21.15.47:0/3687199128 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f07200044c0 con 0x7f07301038e0 2024-03-21T10:25:07.903 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:07.901+0000 7f0734a00640 1 -- 172.21.15.47:0/3687199128 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f07301a19c0 con 0x7f07301038e0 2024-03-21T10:25:07.903 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:07.901+0000 7f0726a00640 1 -- 172.21.15.47:0/3687199128 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0720003c60 con 0x7f07301038e0 2024-03-21T10:25:07.903 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:07.901+0000 7f0734a00640 1 -- 172.21.15.47:0/3687199128 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f07301a1f60 con 0x7f07301038e0 2024-03-21T10:25:07.904 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:07.902+0000 7f0734a00640 1 -- 172.21.15.47:0/3687199128 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f07300688e0 con 0x7f07301038e0 2024-03-21T10:25:07.904 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:07.904+0000 7f0726a00640 1 -- 172.21.15.47:0/3687199128 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 18) v1 ==== 137854+0+0 (secure 0 0 0) 0x7f0720046020 con 0x7f07301038e0 2024-03-21T10:25:07.907 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:07.906+0000 7f0726a00640 1 --2- 172.21.15.47:0/3687199128 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f07140a2fa0 0x7f07140a5460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:25:07.907 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:07.906+0000 7f0726a00640 1 -- 172.21.15.47:0/3687199128 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9216+0+0 (secure 0 0 0) 0x7f07200f19d0 con 0x7f07301038e0 2024-03-21T10:25:07.907 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:07.906+0000 7f072d400640 1 --2- 172.21.15.47:0/3687199128 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f07140a2fa0 0x7f07140a5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:25:07.908 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:07.907+0000 7f072d400640 1 --2- 172.21.15.47:0/3687199128 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f07140a2fa0 0x7f07140a5460 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f073019b020 tx=0x7f071c00b390 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:25:07.911 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:07.911+0000 7f0726a00640 1 -- 172.21.15.47:0/3687199128 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f072004d020 con 0x7f07301038e0 2024-03-21T10:25:08.124 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:08.123+0000 7f0734a00640 1 -- 172.21.15.47:0/3687199128 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f07300622a0 con 0x7f07301038e0 2024-03-21T10:25:08.124 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:08.124+0000 7f0726a00640 1 -- 172.21.15.47:0/3687199128 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v95) v1 ==== 100+0+11 (secure 0 0 0) 0x7f07200be430 con 0x7f07301038e0 2024-03-21T10:25:08.125 INFO:teuthology.orchestra.run.smithi047.stdout:pg_num: 64 2024-03-21T10:25:08.127 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:08.126+0000 7f0734a00640 1 -- 172.21.15.47:0/3687199128 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f07140a2fa0 msgr2=0x7f07140a5460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:25:08.127 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:08.126+0000 7f0734a00640 1 --2- 172.21.15.47:0/3687199128 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f07140a2fa0 0x7f07140a5460 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f073019b020 tx=0x7f071c00b390 comp rx=0 tx=0).stop 2024-03-21T10:25:08.128 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:08.126+0000 7f0734a00640 1 -- 172.21.15.47:0/3687199128 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f07301038e0 msgr2=0x7f0730199b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:25:08.128 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:08.126+0000 7f0734a00640 1 --2- 172.21.15.47:0/3687199128 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f07301038e0 0x7f0730199b10 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f0720000c00 tx=0x7f07200040c0 comp rx=0 tx=0).stop 2024-03-21T10:25:08.128 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:08.126+0000 7f0734a00640 1 -- 172.21.15.47:0/3687199128 shutdown_connections 2024-03-21T10:25:08.128 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:08.126+0000 7f0734a00640 1 --2- 172.21.15.47:0/3687199128 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f07140a2fa0 0x7f07140a5460 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:25:08.128 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:08.126+0000 7f0734a00640 1 --2- 172.21.15.47:0/3687199128 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0730105490 0x7f07301a10c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:25:08.128 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:08.126+0000 7f0734a00640 1 --2- 172.21.15.47:0/3687199128 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0730104ad0 0x7f073019a050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:25:08.128 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:08.126+0000 7f0734a00640 1 --2- 172.21.15.47:0/3687199128 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f07301038e0 0x7f0730199b10 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:25:08.128 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:08.126+0000 7f0734a00640 1 -- 172.21.15.47:0/3687199128 >> 172.21.15.47:0/3687199128 conn(0x7f07300fee50 msgr2=0x7f0730107e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:25:08.128 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:08.127+0000 7f0734a00640 1 -- 172.21.15.47:0/3687199128 shutdown_connections 2024-03-21T10:25:08.128 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:08.127+0000 7f0734a00640 1 -- 172.21.15.47:0/3687199128 wait complete. 2024-03-21T10:25:08.149 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-03-21T10:25:08.369 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:08.368+0000 7ffa0a000640 1 -- 172.21.15.47:0/4223545562 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ffa04104930 msgr2=0x7ffa04106d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:25:08.369 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:08.368+0000 7ffa0a000640 1 --2- 172.21.15.47:0/4223545562 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ffa04104930 0x7ffa04106d20 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7ff9ec009900 tx=0x7ff9ec02bc30 comp rx=0 tx=0).stop 2024-03-21T10:25:08.369 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:08.368+0000 7ffa0a000640 1 -- 172.21.15.47:0/4223545562 shutdown_connections 2024-03-21T10:25:08.369 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:08.368+0000 7ffa0a000640 1 --2- 172.21.15.47:0/4223545562 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ffa04107260 0x7ffa041096b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:25:08.369 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:08.368+0000 7ffa0a000640 1 --2- 172.21.15.47:0/4223545562 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ffa04104930 0x7ffa04106d20 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:25:08.369 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:08.368+0000 7ffa0a000640 1 --2- 172.21.15.47:0/4223545562 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ffa04102000 0x7ffa041043f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:25:08.370 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:08.368+0000 7ffa0a000640 1 -- 172.21.15.47:0/4223545562 >> 172.21.15.47:0/4223545562 conn(0x7ffa040fba10 msgr2=0x7ffa040fde30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:25:08.370 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:08.368+0000 7ffa0a000640 1 -- 172.21.15.47:0/4223545562 shutdown_connections 2024-03-21T10:25:08.370 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:08.368+0000 7ffa0a000640 1 -- 172.21.15.47:0/4223545562 wait complete. 2024-03-21T10:25:08.370 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:08.370+0000 7ffa0a000640 1 Processor -- start 2024-03-21T10:25:08.371 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:08.370+0000 7ffa0a000640 1 -- start start 2024-03-21T10:25:08.372 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:08.370+0000 7ffa0a000640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ffa04102000 0x7ffa04101970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:25:08.372 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:08.370+0000 7ffa0a000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ffa04104930 0x7ffa040fffb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:25:08.372 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:08.370+0000 7ffa0a000640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ffa04107260 0x7ffa041004f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:25:08.372 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:08.370+0000 7ffa0a000640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa0410d0c0 con 0x7ffa04107260 2024-03-21T10:25:08.372 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:08.370+0000 7ffa0a000640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa0410cf40 con 0x7ffa04104930 2024-03-21T10:25:08.372 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:08.370+0000 7ffa0a000640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa0410d240 con 0x7ffa04102000 2024-03-21T10:25:08.372 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:08.371+0000 7ffa09600640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ffa04107260 0x7ffa041004f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:25:08.372 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:08.371+0000 7ff9fbe00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ffa04104930 0x7ffa040fffb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:25:08.372 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:08.371+0000 7ff9fbe00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ffa04104930 0x7ffa040fffb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:50218/0 (socket says 172.21.15.47:50218) 2024-03-21T10:25:08.372 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:08.371+0000 7ff9fbe00640 1 -- 172.21.15.47:0/242985248 learned_addr learned my addr 172.21.15.47:0/242985248 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:25:08.372 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:08.371+0000 7ffa09600640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ffa04107260 0x7ffa041004f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:56402/0 (socket says 172.21.15.47:56402) 2024-03-21T10:25:08.372 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:08.371+0000 7ffa08c00640 1 --2- 172.21.15.47:0/242985248 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ffa04102000 0x7ffa04101970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:25:08.373 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:08.371+0000 7ff9fbe00640 1 -- 172.21.15.47:0/242985248 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ffa04102000 msgr2=0x7ffa04101970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:25:08.373 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:08.371+0000 7ff9fbe00640 1 --2- 172.21.15.47:0/242985248 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ffa04102000 0x7ffa04101970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:25:08.373 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:08.371+0000 7ff9fbe00640 1 -- 172.21.15.47:0/242985248 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ffa04107260 msgr2=0x7ffa041004f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:25:08.373 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:08.371+0000 7ff9fbe00640 1 --2- 172.21.15.47:0/242985248 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ffa04107260 0x7ffa041004f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:25:08.373 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:08.371+0000 7ff9fbe00640 1 -- 172.21.15.47:0/242985248 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffa04100d80 con 0x7ffa04104930 2024-03-21T10:25:08.373 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:08.371+0000 7ffa09600640 1 --2- 172.21.15.47:0/242985248 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ffa04107260 0x7ffa041004f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T10:25:08.374 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:08.372+0000 7ff9fbe00640 1 --2- 172.21.15.47:0/242985248 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ffa04104930 0x7ffa040fffb0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7ff9ec000c00 tx=0x7ff9ec005dc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:25:08.377 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:08.376+0000 7ffa02000640 1 -- 172.21.15.47:0/242985248 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9ec043020 con 0x7ffa04104930 2024-03-21T10:25:08.377 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:08.376+0000 7ffa02000640 1 -- 172.21.15.47:0/242985248 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7ff9ec004350 con 0x7ffa04104930 2024-03-21T10:25:08.377 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:08.376+0000 7ffa0a000640 1 -- 172.21.15.47:0/242985248 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffa041a1900 con 0x7ffa04104930 2024-03-21T10:25:08.378 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:08.376+0000 7ffa02000640 1 -- 172.21.15.47:0/242985248 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9ec003b80 con 0x7ffa04104930 2024-03-21T10:25:08.378 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:08.377+0000 7ffa0a000640 1 -- 172.21.15.47:0/242985248 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffa041a1e50 con 0x7ffa04104930 2024-03-21T10:25:08.378 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:08.378+0000 7ffa0a000640 1 -- 172.21.15.47:0/242985248 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffa040fd500 con 0x7ffa04104930 2024-03-21T10:25:08.381 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:08.380+0000 7ffa02000640 1 -- 172.21.15.47:0/242985248 <== mon.2 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 18) v1 ==== 137854+0+0 (secure 0 0 0) 0x7ff9ec046020 con 0x7ffa04104930 2024-03-21T10:25:08.382 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:08.381+0000 7ffa02000640 1 --2- 172.21.15.47:0/242985248 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7ff9e80a2f20 0x7ff9e80a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:25:08.382 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:08.382+0000 7ffa02000640 1 -- 172.21.15.47:0/242985248 <== mon.2 v2:172.21.15.100:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9216+0+0 (secure 0 0 0) 0x7ff9ec0f1be0 con 0x7ffa04104930 2024-03-21T10:25:08.382 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:08.382+0000 7ffa08c00640 1 --2- 172.21.15.47:0/242985248 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7ff9e80a2f20 0x7ff9e80a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:25:08.383 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:08.382+0000 7ffa08c00640 1 --2- 172.21.15.47:0/242985248 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7ff9e80a2f20 0x7ff9e80a53e0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7ff9f4009bd0 tx=0x7ff9f4005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:25:08.394 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:08.393+0000 7ffa02000640 1 -- 172.21.15.47:0/242985248 <== mon.2 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7ff9ec04d020 con 0x7ffa04104930 2024-03-21T10:25:08.463 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:25:08 smithi047 ceph-mon[31547]: from='client.? 172.21.15.47:0/2841534003' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-21T10:25:08.463 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:25:08 smithi047 ceph-mon[31547]: from='client.? 172.21.15.47:0/3687199128' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-21T10:25:08.599 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:08.598+0000 7ffa0a000640 1 -- 172.21.15.47:0/242985248 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7ffa0402d2c0 con 0x7ffa04104930 2024-03-21T10:25:08.599 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:08.599+0000 7ffa02000640 1 -- 172.21.15.47:0/242985248 <== mon.2 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v95) v1 ==== 103+0+11 (secure 0 0 0) 0x7ff9ec0be600 con 0x7ffa04104930 2024-03-21T10:25:08.600 INFO:teuthology.orchestra.run.smithi047.stdout:pg_num: 32 2024-03-21T10:25:08.602 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:08.601+0000 7ffa0a000640 1 -- 172.21.15.47:0/242985248 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7ff9e80a2f20 msgr2=0x7ff9e80a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:25:08.602 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:08.601+0000 7ffa0a000640 1 --2- 172.21.15.47:0/242985248 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7ff9e80a2f20 0x7ff9e80a53e0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7ff9f4009bd0 tx=0x7ff9f4005f90 comp rx=0 tx=0).stop 2024-03-21T10:25:08.603 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:08.601+0000 7ffa0a000640 1 -- 172.21.15.47:0/242985248 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ffa04104930 msgr2=0x7ffa040fffb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:25:08.603 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:08.601+0000 7ffa0a000640 1 --2- 172.21.15.47:0/242985248 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ffa04104930 0x7ffa040fffb0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7ff9ec000c00 tx=0x7ff9ec005dc0 comp rx=0 tx=0).stop 2024-03-21T10:25:08.603 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:08.601+0000 7ffa0a000640 1 -- 172.21.15.47:0/242985248 shutdown_connections 2024-03-21T10:25:08.603 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:08.601+0000 7ffa0a000640 1 --2- 172.21.15.47:0/242985248 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7ff9e80a2f20 0x7ff9e80a53e0 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:25:08.603 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:08.601+0000 7ffa0a000640 1 --2- 172.21.15.47:0/242985248 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ffa04107260 0x7ffa041004f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:25:08.603 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:08.601+0000 7ffa0a000640 1 --2- 172.21.15.47:0/242985248 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ffa04104930 0x7ffa040fffb0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:25:08.603 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:08.601+0000 7ffa0a000640 1 --2- 172.21.15.47:0/242985248 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ffa04102000 0x7ffa04101970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:25:08.603 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:08.601+0000 7ffa0a000640 1 -- 172.21.15.47:0/242985248 >> 172.21.15.47:0/242985248 conn(0x7ffa040fba10 msgr2=0x7ffa04103e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:25:08.603 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:08.602+0000 7ffa0a000640 1 -- 172.21.15.47:0/242985248 shutdown_connections 2024-03-21T10:25:08.603 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:08.602+0000 7ffa0a000640 1 -- 172.21.15.47:0/242985248 wait complete. 2024-03-21T10:25:08.624 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-21T10:25:08.729 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:25:08 smithi100 ceph-mon[35193]: from='client.? 172.21.15.47:0/2841534003' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-21T10:25:08.730 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:25:08 smithi100 ceph-mon[35193]: from='client.? 172.21.15.47:0/3687199128' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-21T10:25:08.739 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:25:08 smithi157 ceph-mon[35581]: from='client.? 172.21.15.47:0/2841534003' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-21T10:25:08.740 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:25:08 smithi157 ceph-mon[35581]: from='client.? 172.21.15.47:0/3687199128' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-21T10:25:08.843 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:08.842+0000 7f2097e00640 1 -- 172.21.15.47:0/719967786 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f20981003f0 msgr2=0x7f2098100810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:25:08.844 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:08.842+0000 7f2097e00640 1 --2- 172.21.15.47:0/719967786 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f20981003f0 0x7f2098100810 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f2088009900 tx=0x7f208802bc30 comp rx=0 tx=0).stop 2024-03-21T10:25:08.844 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:08.842+0000 7f2097e00640 1 -- 172.21.15.47:0/719967786 shutdown_connections 2024-03-21T10:25:08.844 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:08.842+0000 7f2097e00640 1 --2- 172.21.15.47:0/719967786 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2098101fa0 0x7f2098108830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:25:08.844 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:08.842+0000 7f2097e00640 1 --2- 172.21.15.47:0/719967786 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f20981015e0 0x7f2098101a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:25:08.844 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:08.842+0000 7f2097e00640 1 --2- 172.21.15.47:0/719967786 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f20981003f0 0x7f2098100810 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:25:08.844 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:08.842+0000 7f2097e00640 1 -- 172.21.15.47:0/719967786 >> 172.21.15.47:0/719967786 conn(0x7f20980fb980 msgr2=0x7f20980fdda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:25:08.844 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:08.842+0000 7f2097e00640 1 -- 172.21.15.47:0/719967786 shutdown_connections 2024-03-21T10:25:08.845 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:08.842+0000 7f2097e00640 1 -- 172.21.15.47:0/719967786 wait complete. 2024-03-21T10:25:08.847 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:08.844+0000 7f2097e00640 1 Processor -- start 2024-03-21T10:25:08.847 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:08.844+0000 7f2097e00640 1 -- start start 2024-03-21T10:25:08.847 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:08.844+0000 7f2097e00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f20981003f0 0x7f2098199910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:25:08.848 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:08.844+0000 7f2097e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f20981015e0 0x7f2098199e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:25:08.848 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:08.844+0000 7f2097e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2098101fa0 0x7f20981a0ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:25:08.848 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:08.844+0000 7f2097e00640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f209810ae20 con 0x7f2098101fa0 2024-03-21T10:25:08.848 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:08.844+0000 7f2097e00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f209810aca0 con 0x7f20981015e0 2024-03-21T10:25:08.848 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:08.844+0000 7f2097e00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f209810afa0 con 0x7f20981003f0 2024-03-21T10:25:08.848 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:08.844+0000 7f2097400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2098101fa0 0x7f20981a0ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:25:08.848 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:08.844+0000 7f2097400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2098101fa0 0x7f20981a0ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:56422/0 (socket says 172.21.15.47:56422) 2024-03-21T10:25:08.848 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:08.844+0000 7f2097400640 1 -- 172.21.15.47:0/1238013384 learned_addr learned my addr 172.21.15.47:0/1238013384 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:25:08.849 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:08.845+0000 7f2096000640 1 --2- 172.21.15.47:0/1238013384 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f20981015e0 0x7f2098199e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:25:08.849 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:08.845+0000 7f2096a00640 1 --2- 172.21.15.47:0/1238013384 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f20981003f0 0x7f2098199910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:25:08.849 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:08.845+0000 7f2097400640 1 -- 172.21.15.47:0/1238013384 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f20981003f0 msgr2=0x7f2098199910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:25:08.849 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:08.845+0000 7f2097400640 1 --2- 172.21.15.47:0/1238013384 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f20981003f0 0x7f2098199910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:25:08.849 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:08.845+0000 7f2097400640 1 -- 172.21.15.47:0/1238013384 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f20981015e0 msgr2=0x7f2098199e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:25:08.849 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:08.845+0000 7f2097400640 1 --2- 172.21.15.47:0/1238013384 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f20981015e0 0x7f2098199e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:25:08.849 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:08.845+0000 7f2097400640 1 -- 172.21.15.47:0/1238013384 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20981a1530 con 0x7f2098101fa0 2024-03-21T10:25:08.849 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:08.845+0000 7f2097400640 1 --2- 172.21.15.47:0/1238013384 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2098101fa0 0x7f20981a0ec0 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f209000eb10 tx=0x7f209000c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:25:08.849 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:08.846+0000 7f208f400640 1 -- 172.21.15.47:0/1238013384 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2090004b20 con 0x7f2098101fa0 2024-03-21T10:25:08.850 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:08.846+0000 7f2097e00640 1 -- 172.21.15.47:0/1238013384 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f20981a1820 con 0x7f2098101fa0 2024-03-21T10:25:08.850 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:08.846+0000 7f208f400640 1 -- 172.21.15.47:0/1238013384 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f2090004cc0 con 0x7f2098101fa0 2024-03-21T10:25:08.850 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:08.846+0000 7f208f400640 1 -- 172.21.15.47:0/1238013384 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2090010690 con 0x7f2098101fa0 2024-03-21T10:25:08.850 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:08.846+0000 7f2097e00640 1 -- 172.21.15.47:0/1238013384 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f20981a1dc0 con 0x7f2098101fa0 2024-03-21T10:25:08.850 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:08.847+0000 7f2097e00640 1 -- 172.21.15.47:0/1238013384 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2098107780 con 0x7f2098101fa0 2024-03-21T10:25:08.850 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:08.849+0000 7f208f400640 1 -- 172.21.15.47:0/1238013384 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 18) v1 ==== 137854+0+0 (secure 0 0 0) 0x7f2090005020 con 0x7f2098101fa0 2024-03-21T10:25:08.851 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:08.850+0000 7f208f400640 1 --2- 172.21.15.47:0/1238013384 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f207c0a2f00 0x7f207c0a53c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:25:08.851 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:08.850+0000 7f208f400640 1 -- 172.21.15.47:0/1238013384 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9216+0+0 (secure 0 0 0) 0x7f20900cfeb0 con 0x7f2098101fa0 2024-03-21T10:25:08.851 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:08.850+0000 7f2096a00640 1 --2- 172.21.15.47:0/1238013384 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f207c0a2f00 0x7f207c0a53c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:25:08.852 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:08.851+0000 7f2096a00640 1 --2- 172.21.15.47:0/1238013384 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f207c0a2f00 0x7f207c0a53c0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f2088005ab0 tx=0x7f2088005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:25:08.856 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:08.855+0000 7f208f400640 1 -- 172.21.15.47:0/1238013384 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f209009c8d0 con 0x7f2098101fa0 2024-03-21T10:25:09.120 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:09.119+0000 7f2097e00640 1 -- 172.21.15.47:0/1238013384 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f20980622a0 con 0x7f2098101fa0 2024-03-21T10:25:09.122 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:09.121+0000 7f208f400640 1 -- 172.21.15.47:0/1238013384 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) v1 ==== 94+0+11471 (secure 0 0 0) 0x7f209009c1d0 con 0x7f2098101fa0 2024-03-21T10:25:09.122 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-21T10:25:09.122 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":30,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14511,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6835/735724037","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6834","nonce":735724037},{"type":"v1","addr":"172.21.15.47:6835","nonce":735724037}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14529,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6837/2248191724","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6836","nonce":2248191724},{"type":"v1","addr":"172.21.15.47:6837","nonce":2248191724}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":30,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-21T10:21:58.329063+0000","modified":"2024-03-21T10:24:58.786909+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":34535,"mds_1":34493,"mds_2":34454,"mds_3":34427,"mds_4":14619},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14604":{"gid":14604,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.47:6839/2881335356","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6838","nonce":2881335356},{"type":"v1","addr":"172.21.15.47:6839","nonce":2881335356}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_14619":{"gid":14619,"name":"a","rank":4,"incarnation":27,"state":"up:active","state_seq":11,"addr":"172.21.15.47:6841/3401578978","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6840","nonce":3401578978},{"type":"v1","addr":"172.21.15.47:6841","nonce":3401578978}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24406":{"gid":24406,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.157:6833/4000010136","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":4000010136},{"type":"v1","addr":"172.21.15.157:6833","nonce":4000010136}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24457":{"gid":24457,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.157:6835/3902868651","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":3902868651},{"type":"v1","addr":"172.21.15.157:6835","nonce":3902868651}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24475":{"gid":24475,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.157:6837/1908087204","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6836","nonce":1908087204},{"type":"v1","addr":"172.21.15.157:6837","nonce":1908087204}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24499":{"gid":24499,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.157:6839/4064242876","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6838","nonce":4064242876},{"type":"v1","addr":"172.21.15.157:6839","nonce":4064242876}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_34427":{"gid":34427,"name":"k","rank":3,"incarnation":25,"state":"up:active","state_seq":20,"addr":"172.21.15.100:6833/2723302038","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6832","nonce":2723302038},{"type":"v1","addr":"172.21.15.100:6833","nonce":2723302038}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_34454":{"gid":34454,"name":"e","rank":2,"incarnation":22,"state":"up:active","state_seq":17,"addr":"172.21.15.100:6835/2597849957","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":2597849957},{"type":"v1","addr":"172.21.15.100:6835","nonce":2597849957}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_34493":{"gid":34493,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":13,"addr":"172.21.15.100:6837/3497644162","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6836","nonce":3497644162},{"type":"v1","addr":"172.21.15.100:6837","nonce":3497644162}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_34535":{"gid":34535,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.100:6839/4173168440","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6838","nonce":4173168440},{"type":"v1","addr":"172.21.15.100:6839","nonce":4173168440}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":34535,"qdb_cluster":[14619,34427,34454,34493,34535]},"id":1}]} 2024-03-21T10:25:09.123 INFO:teuthology.orchestra.run.smithi047.stderr:dumped fsmap epoch 30 2024-03-21T10:25:09.124 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:09.123+0000 7f2097e00640 1 -- 172.21.15.47:0/1238013384 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f207c0a2f00 msgr2=0x7f207c0a53c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:25:09.124 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:09.123+0000 7f2097e00640 1 --2- 172.21.15.47:0/1238013384 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f207c0a2f00 0x7f207c0a53c0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f2088005ab0 tx=0x7f2088005fd0 comp rx=0 tx=0).stop 2024-03-21T10:25:09.124 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:09.124+0000 7f2097e00640 1 -- 172.21.15.47:0/1238013384 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2098101fa0 msgr2=0x7f20981a0ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:25:09.124 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:09.124+0000 7f2097e00640 1 --2- 172.21.15.47:0/1238013384 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2098101fa0 0x7f20981a0ec0 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f209000eb10 tx=0x7f209000c2d0 comp rx=0 tx=0).stop 2024-03-21T10:25:09.125 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:09.124+0000 7f2097e00640 1 -- 172.21.15.47:0/1238013384 shutdown_connections 2024-03-21T10:25:09.125 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:09.124+0000 7f2097e00640 1 --2- 172.21.15.47:0/1238013384 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f207c0a2f00 0x7f207c0a53c0 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:25:09.125 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:09.124+0000 7f2097e00640 1 --2- 172.21.15.47:0/1238013384 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2098101fa0 0x7f20981a0ec0 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:25:09.125 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:09.124+0000 7f2097e00640 1 --2- 172.21.15.47:0/1238013384 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f20981015e0 0x7f2098199e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:25:09.125 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:09.124+0000 7f2097e00640 1 --2- 172.21.15.47:0/1238013384 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f20981003f0 0x7f2098199910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:25:09.125 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:09.124+0000 7f2097e00640 1 -- 172.21.15.47:0/1238013384 >> 172.21.15.47:0/1238013384 conn(0x7f20980fb980 msgr2=0x7f2098104950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:25:09.126 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:09.124+0000 7f2097e00640 1 -- 172.21.15.47:0/1238013384 shutdown_connections 2024-03-21T10:25:09.126 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:09.124+0000 7f2097e00640 1 -- 172.21.15.47:0/1238013384 wait complete. 2024-03-21T10:25:09.147 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-21T10:25:09.370 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:09.369+0000 7f4b6f200640 1 -- 172.21.15.47:0/1996070405 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4b70102030 msgr2=0x7f4b70104420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:25:09.371 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:09.369+0000 7f4b6f200640 1 --2- 172.21.15.47:0/1996070405 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4b70102030 0x7f4b70104420 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f4b60009900 tx=0x7f4b6002bc30 comp rx=0 tx=0).stop 2024-03-21T10:25:09.371 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:09.369+0000 7f4b6f200640 1 -- 172.21.15.47:0/1996070405 shutdown_connections 2024-03-21T10:25:09.371 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:09.369+0000 7f4b6f200640 1 --2- 172.21.15.47:0/1996070405 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4b70107290 0x7f4b70109680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:25:09.371 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:09.369+0000 7f4b6f200640 1 --2- 172.21.15.47:0/1996070405 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4b70104960 0x7f4b70106d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:25:09.371 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:09.369+0000 7f4b6f200640 1 --2- 172.21.15.47:0/1996070405 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4b70102030 0x7f4b70104420 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:25:09.371 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:09.369+0000 7f4b6f200640 1 -- 172.21.15.47:0/1996070405 >> 172.21.15.47:0/1996070405 conn(0x7f4b700fb980 msgr2=0x7f4b700fdda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:25:09.371 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:09.370+0000 7f4b6f200640 1 -- 172.21.15.47:0/1996070405 shutdown_connections 2024-03-21T10:25:09.371 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:09.370+0000 7f4b6f200640 1 -- 172.21.15.47:0/1996070405 wait complete. 2024-03-21T10:25:09.375 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:09.371+0000 7f4b6f200640 1 Processor -- start 2024-03-21T10:25:09.375 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:09.372+0000 7f4b6f200640 1 -- start start 2024-03-21T10:25:09.375 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:09.372+0000 7f4b6f200640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4b70102030 0x7f4b701019a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:25:09.375 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:09.372+0000 7f4b6f200640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4b70104960 0x7f4b700fffe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:25:09.375 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:09.372+0000 7f4b6f200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4b70107290 0x7f4b70100520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:25:09.376 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:09.372+0000 7f4b6f200640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b7010d090 con 0x7f4b70104960 2024-03-21T10:25:09.376 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:09.372+0000 7f4b6f200640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b7010cf10 con 0x7f4b70107290 2024-03-21T10:25:09.376 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:09.372+0000 7f4b6f200640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b7010d210 con 0x7f4b70102030 2024-03-21T10:25:09.376 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:09.372+0000 7f4b67e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4b70104960 0x7f4b700fffe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:25:09.376 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:09.373+0000 7f4b67e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4b70104960 0x7f4b700fffe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:56446/0 (socket says 172.21.15.47:56446) 2024-03-21T10:25:09.376 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:09.373+0000 7f4b67e00640 1 -- 172.21.15.47:0/1966066287 learned_addr learned my addr 172.21.15.47:0/1966066287 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:25:09.376 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:09.373+0000 7f4b6e800640 1 --2- 172.21.15.47:0/1966066287 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4b70107290 0x7f4b70100520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:25:09.376 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:09.373+0000 7f4b6de00640 1 --2- 172.21.15.47:0/1966066287 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4b70102030 0x7f4b701019a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:25:09.376 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:09.373+0000 7f4b67e00640 1 -- 172.21.15.47:0/1966066287 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4b70102030 msgr2=0x7f4b701019a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:25:09.376 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:09.373+0000 7f4b67e00640 1 --2- 172.21.15.47:0/1966066287 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4b70102030 0x7f4b701019a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:25:09.376 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:09.373+0000 7f4b67e00640 1 -- 172.21.15.47:0/1966066287 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4b70107290 msgr2=0x7f4b70100520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:25:09.376 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:09.373+0000 7f4b67e00640 1 --2- 172.21.15.47:0/1966066287 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4b70107290 0x7f4b70100520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:25:09.376 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:09.373+0000 7f4b67e00640 1 -- 172.21.15.47:0/1966066287 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b70100db0 con 0x7f4b70104960 2024-03-21T10:25:09.376 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:09.373+0000 7f4b67e00640 1 --2- 172.21.15.47:0/1966066287 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4b70104960 0x7f4b700fffe0 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f4b5800e9c0 tx=0x7f4b5800ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:25:09.377 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:09.374+0000 7f4b66a00640 1 -- 172.21.15.47:0/1966066287 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b5800cd80 con 0x7f4b70104960 2024-03-21T10:25:09.377 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:09.374+0000 7f4b66a00640 1 -- 172.21.15.47:0/1966066287 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f4b580054d0 con 0x7f4b70104960 2024-03-21T10:25:09.377 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:09.374+0000 7f4b66a00640 1 -- 172.21.15.47:0/1966066287 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b58010430 con 0x7f4b70104960 2024-03-21T10:25:09.377 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:09.374+0000 7f4b6f200640 1 -- 172.21.15.47:0/1966066287 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4b701a1850 con 0x7f4b70104960 2024-03-21T10:25:09.377 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:09.374+0000 7f4b6f200640 1 -- 172.21.15.47:0/1966066287 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4b701a1d50 con 0x7f4b70104960 2024-03-21T10:25:09.377 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:09.375+0000 7f4b6f200640 1 -- 172.21.15.47:0/1966066287 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4b700fd470 con 0x7f4b70104960 2024-03-21T10:25:09.378 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:09.377+0000 7f4b66a00640 1 -- 172.21.15.47:0/1966066287 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 18) v1 ==== 137854+0+0 (secure 0 0 0) 0x7f4b58004fd0 con 0x7f4b70104960 2024-03-21T10:25:09.379 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:09.378+0000 7f4b66a00640 1 --2- 172.21.15.47:0/1966066287 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f4b540a2f20 0x7f4b540a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:25:09.379 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:09.379+0000 7f4b66a00640 1 -- 172.21.15.47:0/1966066287 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9216+0+0 (secure 0 0 0) 0x7f4b580cfce0 con 0x7f4b70104960 2024-03-21T10:25:09.379 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:09.379+0000 7f4b6de00640 1 --2- 172.21.15.47:0/1966066287 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f4b540a2f20 0x7f4b540a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:25:09.380 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:09.380+0000 7f4b6de00640 1 --2- 172.21.15.47:0/1966066287 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f4b540a2f20 0x7f4b540a53e0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f4b60006010 tx=0x7f4b60000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:25:09.386 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:09.385+0000 7f4b66a00640 1 -- 172.21.15.47:0/1966066287 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f4b5809c700 con 0x7f4b70104960 2024-03-21T10:25:09.463 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:25:09 smithi047 ceph-mon[31547]: pgmap v466: 129 pgs: 129 active+clean; 649 KiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 1.7 KiB/s wr, 11 op/s 2024-03-21T10:25:09.464 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:25:09 smithi047 ceph-mon[31547]: from='client.? 172.21.15.47:0/242985248' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-03-21T10:25:09.464 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:25:09 smithi047 ceph-mon[31547]: from='client.? 172.21.15.47:0/1238013384' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-21T10:25:09.592 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:09.591+0000 7f4b6f200640 1 -- 172.21.15.47:0/1966066287 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f4b700622d0 con 0x7f4b70104960 2024-03-21T10:25:09.594 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:09.594+0000 7f4b66a00640 1 -- 172.21.15.47:0/1966066287 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v95) v1 ==== 74+0+24849 (secure 0 0 0) 0x7f4b5809c000 con 0x7f4b70104960 2024-03-21T10:25:09.595 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-21T10:25:09.595 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":95,"fsid":"3056d51a-e76b-11ee-95cd-87774f69a715","created":"2024-03-21T10:11:17.019965+0000","modified":"2024-03-21T10:23:27.619804+0000","last_up_change":"2024-03-21T10:20:52.675328+0000","last_in_change":"2024-03-21T10:20:27.648212+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-21T10:15:40.553114+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-21T10:21:56.399518+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-21T10:21:57.812940+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-21T10:22:01.822568+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"91","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"b0c7a102-8d89-428b-b793-cbbe67151f50","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6802","nonce":775837313},{"type":"v1","addr":"172.21.15.47:6803","nonce":775837313}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6804","nonce":775837313},{"type":"v1","addr":"172.21.15.47:6805","nonce":775837313}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6808","nonce":775837313},{"type":"v1","addr":"172.21.15.47:6809","nonce":775837313}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6806","nonce":775837313},{"type":"v1","addr":"172.21.15.47:6807","nonce":775837313}]},"public_addr":"172.21.15.47:6803/775837313","cluster_addr":"172.21.15.47:6805/775837313","heartbeat_back_addr":"172.21.15.47:6809/775837313","heartbeat_front_addr":"172.21.15.47:6807/775837313","state":["exists","up"]},{"osd":1,"uuid":"07bb81bb-7cf3-4bbc-ac9e-802f95185d11","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6800","nonce":2923304710},{"type":"v1","addr":"172.21.15.100:6801","nonce":2923304710}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":2923304710},{"type":"v1","addr":"172.21.15.100:6803","nonce":2923304710}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":2923304710},{"type":"v1","addr":"172.21.15.100:6807","nonce":2923304710}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6804","nonce":2923304710},{"type":"v1","addr":"172.21.15.100:6805","nonce":2923304710}]},"public_addr":"172.21.15.100:6801/2923304710","cluster_addr":"172.21.15.100:6803/2923304710","heartbeat_back_addr":"172.21.15.100:6807/2923304710","heartbeat_front_addr":"172.21.15.100:6805/2923304710","state":["exists","up"]},{"osd":2,"uuid":"88ce76b9-6081-4836-aad0-8c249c56e8a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":171377005},{"type":"v1","addr":"172.21.15.157:6801","nonce":171377005}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":171377005},{"type":"v1","addr":"172.21.15.157:6803","nonce":171377005}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":171377005},{"type":"v1","addr":"172.21.15.157:6807","nonce":171377005}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":171377005},{"type":"v1","addr":"172.21.15.157:6805","nonce":171377005}]},"public_addr":"172.21.15.157:6801/171377005","cluster_addr":"172.21.15.157:6803/171377005","heartbeat_back_addr":"172.21.15.157:6807/171377005","heartbeat_front_addr":"172.21.15.157:6805/171377005","state":["exists","up"]},{"osd":3,"uuid":"518aa33d-263f-45e6-aab5-2201ceb40e74","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":27,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6810","nonce":640760101},{"type":"v1","addr":"172.21.15.47:6811","nonce":640760101}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6812","nonce":640760101},{"type":"v1","addr":"172.21.15.47:6813","nonce":640760101}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6816","nonce":640760101},{"type":"v1","addr":"172.21.15.47:6817","nonce":640760101}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6814","nonce":640760101},{"type":"v1","addr":"172.21.15.47:6815","nonce":640760101}]},"public_addr":"172.21.15.47:6811/640760101","cluster_addr":"172.21.15.47:6813/640760101","heartbeat_back_addr":"172.21.15.47:6817/640760101","heartbeat_front_addr":"172.21.15.47:6815/640760101","state":["exists","up"]},{"osd":4,"uuid":"987bde83-ecc6-42e5-bd56-c143b3de4226","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":35,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6808","nonce":2785478639},{"type":"v1","addr":"172.21.15.100:6809","nonce":2785478639}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6810","nonce":2785478639},{"type":"v1","addr":"172.21.15.100:6811","nonce":2785478639}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6814","nonce":2785478639},{"type":"v1","addr":"172.21.15.100:6815","nonce":2785478639}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":2785478639},{"type":"v1","addr":"172.21.15.100:6813","nonce":2785478639}]},"public_addr":"172.21.15.100:6809/2785478639","cluster_addr":"172.21.15.100:6811/2785478639","heartbeat_back_addr":"172.21.15.100:6815/2785478639","heartbeat_front_addr":"172.21.15.100:6813/2785478639","state":["exists","up"]},{"osd":5,"uuid":"012d2188-3a61-4a5f-8d01-2001c15ffbe2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":40,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":1525645410},{"type":"v1","addr":"172.21.15.157:6809","nonce":1525645410}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":1525645410},{"type":"v1","addr":"172.21.15.157:6811","nonce":1525645410}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":1525645410},{"type":"v1","addr":"172.21.15.157:6815","nonce":1525645410}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":1525645410},{"type":"v1","addr":"172.21.15.157:6813","nonce":1525645410}]},"public_addr":"172.21.15.157:6809/1525645410","cluster_addr":"172.21.15.157:6811/1525645410","heartbeat_back_addr":"172.21.15.157:6815/1525645410","heartbeat_front_addr":"172.21.15.157:6813/1525645410","state":["exists","up"]},{"osd":6,"uuid":"c12c14b4-85f0-4d88-937c-de4f556437de","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6818","nonce":2321642790},{"type":"v1","addr":"172.21.15.47:6819","nonce":2321642790}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6820","nonce":2321642790},{"type":"v1","addr":"172.21.15.47:6821","nonce":2321642790}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6824","nonce":2321642790},{"type":"v1","addr":"172.21.15.47:6825","nonce":2321642790}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6822","nonce":2321642790},{"type":"v1","addr":"172.21.15.47:6823","nonce":2321642790}]},"public_addr":"172.21.15.47:6819/2321642790","cluster_addr":"172.21.15.47:6821/2321642790","heartbeat_back_addr":"172.21.15.47:6825/2321642790","heartbeat_front_addr":"172.21.15.47:6823/2321642790","state":["exists","up"]},{"osd":7,"uuid":"3c87c6d0-1d1f-4294-9917-aaaad7dda824","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6816","nonce":4104727330},{"type":"v1","addr":"172.21.15.100:6817","nonce":4104727330}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":4104727330},{"type":"v1","addr":"172.21.15.100:6819","nonce":4104727330}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":4104727330},{"type":"v1","addr":"172.21.15.100:6823","nonce":4104727330}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6820","nonce":4104727330},{"type":"v1","addr":"172.21.15.100:6821","nonce":4104727330}]},"public_addr":"172.21.15.100:6817/4104727330","cluster_addr":"172.21.15.100:6819/4104727330","heartbeat_back_addr":"172.21.15.100:6823/4104727330","heartbeat_front_addr":"172.21.15.100:6821/4104727330","state":["exists","up"]},{"osd":8,"uuid":"502d65cf-53c8-4936-8de4-e11b2a6e1e18","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":61,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":4067715347},{"type":"v1","addr":"172.21.15.157:6817","nonce":4067715347}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":4067715347},{"type":"v1","addr":"172.21.15.157:6819","nonce":4067715347}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":4067715347},{"type":"v1","addr":"172.21.15.157:6823","nonce":4067715347}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":4067715347},{"type":"v1","addr":"172.21.15.157:6821","nonce":4067715347}]},"public_addr":"172.21.15.157:6817/4067715347","cluster_addr":"172.21.15.157:6819/4067715347","heartbeat_back_addr":"172.21.15.157:6823/4067715347","heartbeat_front_addr":"172.21.15.157:6821/4067715347","state":["exists","up"]},{"osd":9,"uuid":"5ea1f5be-8458-4c93-9e61-c2e867a514f3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6826","nonce":1982108738},{"type":"v1","addr":"172.21.15.47:6827","nonce":1982108738}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6828","nonce":1982108738},{"type":"v1","addr":"172.21.15.47:6829","nonce":1982108738}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6832","nonce":1982108738},{"type":"v1","addr":"172.21.15.47:6833","nonce":1982108738}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6830","nonce":1982108738},{"type":"v1","addr":"172.21.15.47:6831","nonce":1982108738}]},"public_addr":"172.21.15.47:6827/1982108738","cluster_addr":"172.21.15.47:6829/1982108738","heartbeat_back_addr":"172.21.15.47:6833/1982108738","heartbeat_front_addr":"172.21.15.47:6831/1982108738","state":["exists","up"]},{"osd":10,"uuid":"2da3da90-a0be-4a54-a9ff-b2d6c9b36870","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":74,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":396391028},{"type":"v1","addr":"172.21.15.100:6825","nonce":396391028}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":396391028},{"type":"v1","addr":"172.21.15.100:6827","nonce":396391028}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6830","nonce":396391028},{"type":"v1","addr":"172.21.15.100:6831","nonce":396391028}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":396391028},{"type":"v1","addr":"172.21.15.100:6829","nonce":396391028}]},"public_addr":"172.21.15.100:6825/396391028","cluster_addr":"172.21.15.100:6827/396391028","heartbeat_back_addr":"172.21.15.100:6831/396391028","heartbeat_front_addr":"172.21.15.100:6829/396391028","state":["exists","up"]},{"osd":11,"uuid":"11798d41-7f7e-4319-95b8-8d3d0946bde9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":82,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":2671029483},{"type":"v1","addr":"172.21.15.157:6825","nonce":2671029483}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":2671029483},{"type":"v1","addr":"172.21.15.157:6827","nonce":2671029483}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":2671029483},{"type":"v1","addr":"172.21.15.157:6831","nonce":2671029483}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":2671029483},{"type":"v1","addr":"172.21.15.157:6829","nonce":2671029483}]},"public_addr":"172.21.15.157:6825/2671029483","cluster_addr":"172.21.15.157:6827/2671029483","heartbeat_back_addr":"172.21.15.157:6831/2671029483","heartbeat_front_addr":"172.21.15.157:6829/2671029483","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T10:14:31.822399+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T10:15:04.175721+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T10:15:35.902595+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T10:16:12.088891+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T10:16:46.428704+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T10:17:20.158270+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T10:17:55.915253+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T10:18:30.064641+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T10:19:04.410004+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T10:19:40.605181+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T10:20:15.636605+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T10:20:50.151365+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.11","mappings":[{"from":7,"to":11}]},{"pgid":"2.14","mappings":[{"from":2,"to":4}]},{"pgid":"2.1a","mappings":[{"from":8,"to":1}]},{"pgid":"2.1b","mappings":[{"from":2,"to":4}]},{"pgid":"2.1d","mappings":[{"from":8,"to":11}]},{"pgid":"3.0","mappings":[{"from":4,"to":8},{"from":5,"to":9}]},{"pgid":"3.6","mappings":[{"from":3,"to":7}]},{"pgid":"3.1b","mappings":[{"from":4,"to":6}]},{"pgid":"3.33","mappings":[{"from":5,"to":9}]},{"pgid":"3.3c","mappings":[{"from":4,"to":9}]},{"pgid":"4.c","mappings":[{"from":9,"to":10}]},{"pgid":"4.f","mappings":[{"from":9,"to":8}]},{"pgid":"4.19","mappings":[{"from":9,"to":8}]},{"pgid":"4.1d","mappings":[{"from":5,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.47:0/356822431":"2024-03-22T10:12:25.844530+0000","172.21.15.47:0/1167490879":"2024-03-22T10:12:25.844530+0000","172.21.15.47:0/1540718138":"2024-03-22T10:12:25.844530+0000","172.21.15.47:6801/3836903972":"2024-03-22T10:12:25.844530+0000","172.21.15.47:0/235436602":"2024-03-22T10:11:50.301838+0000","172.21.15.47:0/2052815572":"2024-03-22T10:11:50.301838+0000","172.21.15.47:6800/3942658432":"2024-03-22T10:11:50.301838+0000","172.21.15.47:6801/3942658432":"2024-03-22T10:11:50.301838+0000","172.21.15.47:0/1419516257":"2024-03-22T10:11:50.301838+0000","172.21.15.47:6800/3836903972":"2024-03-22T10:12:25.844530+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-21T10:25:09.598 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:09.596+0000 7f4b6f200640 1 -- 172.21.15.47:0/1966066287 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f4b540a2f20 msgr2=0x7f4b540a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:25:09.598 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:09.596+0000 7f4b6f200640 1 --2- 172.21.15.47:0/1966066287 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f4b540a2f20 0x7f4b540a53e0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f4b60006010 tx=0x7f4b60000bc0 comp rx=0 tx=0).stop 2024-03-21T10:25:09.598 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:09.596+0000 7f4b6f200640 1 -- 172.21.15.47:0/1966066287 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4b70104960 msgr2=0x7f4b700fffe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:25:09.598 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:09.596+0000 7f4b6f200640 1 --2- 172.21.15.47:0/1966066287 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4b70104960 0x7f4b700fffe0 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f4b5800e9c0 tx=0x7f4b5800ee80 comp rx=0 tx=0).stop 2024-03-21T10:25:09.598 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:09.597+0000 7f4b6f200640 1 -- 172.21.15.47:0/1966066287 shutdown_connections 2024-03-21T10:25:09.598 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:09.597+0000 7f4b6f200640 1 --2- 172.21.15.47:0/1966066287 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f4b540a2f20 0x7f4b540a53e0 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:25:09.598 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:09.597+0000 7f4b6f200640 1 --2- 172.21.15.47:0/1966066287 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4b70107290 0x7f4b70100520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:25:09.598 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:09.597+0000 7f4b6f200640 1 --2- 172.21.15.47:0/1966066287 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4b70104960 0x7f4b700fffe0 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:25:09.598 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:09.597+0000 7f4b6f200640 1 --2- 172.21.15.47:0/1966066287 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4b70102030 0x7f4b701019a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:25:09.598 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:09.597+0000 7f4b6f200640 1 -- 172.21.15.47:0/1966066287 >> 172.21.15.47:0/1966066287 conn(0x7f4b700fb980 msgr2=0x7f4b70102a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:25:09.598 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:09.597+0000 7f4b6f200640 1 -- 172.21.15.47:0/1966066287 shutdown_connections 2024-03-21T10:25:09.599 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:09.597+0000 7f4b6f200640 1 -- 172.21.15.47:0/1966066287 wait complete. 2024-03-21T10:25:09.617 DEBUG:tasks.fwd_scrub:Yielding 2024-03-21T10:25:09.617 INFO:teuthology.run_tasks:Running task exec... 2024-03-21T10:25:09.625 INFO:teuthology.task.exec:Executing custom commands... 2024-03-21T10:25:09.626 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi047.front.sepia.ceph.com 2024-03-21T10:25:09.626 DEBUG:teuthology.orchestra.run.smithi047:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph mgr module enable snap_schedule' 2024-03-21T10:25:09.626 INFO:tasks.fwd_scrub.fs.[cephfs]:start scrubbing fs: cephfs 2024-03-21T10:25:09.626 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-03-21T10:25:09.626 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-21T10:25:09.626 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-03-21T10:25:09.729 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:25:09 smithi100 ceph-mon[35193]: pgmap v466: 129 pgs: 129 active+clean; 649 KiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 1.7 KiB/s wr, 11 op/s 2024-03-21T10:25:09.729 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:25:09 smithi100 ceph-mon[35193]: from='client.? 172.21.15.47:0/242985248' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-03-21T10:25:09.730 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:25:09 smithi100 ceph-mon[35193]: from='client.? 172.21.15.47:0/1238013384' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-21T10:25:09.739 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:25:09 smithi157 ceph-mon[35581]: pgmap v466: 129 pgs: 129 active+clean; 649 KiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 1.7 KiB/s wr, 11 op/s 2024-03-21T10:25:09.740 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:25:09 smithi157 ceph-mon[35581]: from='client.? 172.21.15.47:0/242985248' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-03-21T10:25:09.740 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:25:09 smithi157 ceph-mon[35581]: from='client.? 172.21.15.47:0/1238013384' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-21T10:25:09.834 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:09.833+0000 7f7a27200640 1 -- 172.21.15.47:0/44692680 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7a28102770 msgr2=0x7f7a28102bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:25:09.834 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:09.833+0000 7f7a27200640 1 --2- 172.21.15.47:0/44692680 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7a28102770 0x7f7a28102bf0 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f7a10009900 tx=0x7f7a1002bc30 comp rx=0 tx=0).stop 2024-03-21T10:25:09.835 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:09.833+0000 7f7a27200640 1 -- 172.21.15.47:0/44692680 shutdown_connections 2024-03-21T10:25:09.835 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:09.833+0000 7f7a27200640 1 --2- 172.21.15.47:0/44692680 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7a28103130 0x7f7a281099c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:25:09.835 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:09.833+0000 7f7a27200640 1 --2- 172.21.15.47:0/44692680 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7a28102770 0x7f7a28102bf0 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:25:09.835 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:09.833+0000 7f7a27200640 1 --2- 172.21.15.47:0/44692680 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7a28101580 0x7f7a281019a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:25:09.835 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:09.833+0000 7f7a27200640 1 -- 172.21.15.47:0/44692680 >> 172.21.15.47:0/44692680 conn(0x7f7a280fcb10 msgr2=0x7f7a280fef30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:25:09.835 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:09.833+0000 7f7a27200640 1 -- 172.21.15.47:0/44692680 shutdown_connections 2024-03-21T10:25:09.835 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:09.833+0000 7f7a27200640 1 -- 172.21.15.47:0/44692680 wait complete. 2024-03-21T10:25:09.837 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:09.834+0000 7f7a27200640 1 Processor -- start 2024-03-21T10:25:09.837 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:09.834+0000 7f7a27200640 1 -- start start 2024-03-21T10:25:09.837 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:09.835+0000 7f7a27200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7a28101580 0x7f7a28075450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:25:09.837 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:09.835+0000 7f7a27200640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7a28102770 0x7f7a28073a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:25:09.838 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:09.835+0000 7f7a27200640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7a28103130 0x7f7a28073fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:25:09.838 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:09.835+0000 7f7a27200640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a2810d0e0 con 0x7f7a28102770 2024-03-21T10:25:09.838 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:09.835+0000 7f7a27200640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a2810cf60 con 0x7f7a28101580 2024-03-21T10:25:09.838 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:09.835+0000 7f7a27200640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a2810d260 con 0x7f7a28103130 2024-03-21T10:25:09.838 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:09.835+0000 7f7a25400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7a28102770 0x7f7a28073a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:25:09.838 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:09.835+0000 7f7a25400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7a28102770 0x7f7a28073a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:56468/0 (socket says 172.21.15.47:56468) 2024-03-21T10:25:09.838 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:09.835+0000 7f7a25400640 1 -- 172.21.15.47:0/628536271 learned_addr learned my addr 172.21.15.47:0/628536271 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:25:09.838 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:09.835+0000 7f7a25e00640 1 --2- 172.21.15.47:0/628536271 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7a28101580 0x7f7a28075450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:25:09.838 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:09.835+0000 7f7a26800640 1 --2- 172.21.15.47:0/628536271 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7a28103130 0x7f7a28073fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:25:09.838 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:09.836+0000 7f7a25400640 1 -- 172.21.15.47:0/628536271 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7a28103130 msgr2=0x7f7a28073fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:25:09.838 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:09.836+0000 7f7a25400640 1 --2- 172.21.15.47:0/628536271 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7a28103130 0x7f7a28073fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:25:09.838 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:09.836+0000 7f7a25400640 1 -- 172.21.15.47:0/628536271 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7a28101580 msgr2=0x7f7a28075450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:25:09.838 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:09.836+0000 7f7a25400640 1 --2- 172.21.15.47:0/628536271 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7a28101580 0x7f7a28075450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:25:09.839 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:09.836+0000 7f7a25400640 1 -- 172.21.15.47:0/628536271 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a28074860 con 0x7f7a28102770 2024-03-21T10:25:09.839 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:09.836+0000 7f7a26800640 1 --2- 172.21.15.47:0/628536271 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7a28103130 0x7f7a28073fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T10:25:09.839 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:09.836+0000 7f7a25400640 1 --2- 172.21.15.47:0/628536271 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7a28102770 0x7f7a28073a90 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f7a10000c00 tx=0x7f7a100042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:25:09.839 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:09.836+0000 7f7a1ea00640 1 -- 172.21.15.47:0/628536271 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a10043020 con 0x7f7a28102770 2024-03-21T10:25:09.839 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:09.836+0000 7f7a27200640 1 -- 172.21.15.47:0/628536271 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7a281a1870 con 0x7f7a28102770 2024-03-21T10:25:09.839 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:09.836+0000 7f7a1ea00640 1 -- 172.21.15.47:0/628536271 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f7a10004aa0 con 0x7f7a28102770 2024-03-21T10:25:09.839 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:09.836+0000 7f7a1ea00640 1 -- 172.21.15.47:0/628536271 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a10003d70 con 0x7f7a28102770 2024-03-21T10:25:09.839 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:09.837+0000 7f7a27200640 1 -- 172.21.15.47:0/628536271 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7a281a1dc0 con 0x7f7a28102770 2024-03-21T10:25:09.841 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:09.840+0000 7f7a1ea00640 1 -- 172.21.15.47:0/628536271 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 18) v1 ==== 137854+0+0 (secure 0 0 0) 0x7f7a10046020 con 0x7f7a28102770 2024-03-21T10:25:09.841 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:09.840+0000 7f7a2fbfa740 1 Processor -- start 2024-03-21T10:25:09.841 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:09.840+0000 7f7a2fbfa740 1 -- start start 2024-03-21T10:25:09.843 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:09.841+0000 7f7a2fbfa740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f33cba82b0 0x55f33cba86d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:25:09.843 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:09.841+0000 7f7a2fbfa740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f33cba8ca0 0x55f33cbad7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:25:09.843 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:09.841+0000 7f7a2fbfa740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f33cbade10 0x55f33cbb02a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:25:09.843 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:09.841+0000 7f7a2fbfa740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f33cb0deb0 con 0x55f33cba82b0 2024-03-21T10:25:09.843 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:09.841+0000 7f7a2fbfa740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f33cab6a60 con 0x55f33cba8ca0 2024-03-21T10:25:09.843 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:09.841+0000 7f7a2fbfa740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f33cb09f50 con 0x55f33cbade10 2024-03-21T10:25:09.843 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:09.841+0000 7f7a26800640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f33cba82b0 0x55f33cba86d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:25:09.843 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:09.841+0000 7f7a26800640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f33cba82b0 0x55f33cba86d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:56476/0 (socket says 172.21.15.47:56476) 2024-03-21T10:25:09.843 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:09.841+0000 7f7a25e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f33cba8ca0 0x55f33cbad7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:25:09.843 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:09.841+0000 7f7a26800640 1 -- 172.21.15.47:0/2363391916 learned_addr learned my addr 172.21.15.47:0/2363391916 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:25:09.843 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:09.841+0000 7f7a25400640 1 --2- 172.21.15.47:0/2363391916 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f33cbade10 0x55f33cbb02a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:25:09.844 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:09.842+0000 7f7a26800640 1 -- 172.21.15.47:0/2363391916 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f33cbade10 msgr2=0x55f33cbb02a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:25:09.844 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:09.842+0000 7f7a26800640 1 --2- 172.21.15.47:0/2363391916 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f33cbade10 0x55f33cbb02a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:25:09.844 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:09.842+0000 7f7a26800640 1 -- 172.21.15.47:0/2363391916 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f33cba8ca0 msgr2=0x55f33cbad7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:25:09.844 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:09.842+0000 7f7a26800640 1 --2- 172.21.15.47:0/2363391916 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f33cba8ca0 0x55f33cbad7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:25:09.844 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:09.842+0000 7f7a26800640 1 -- 172.21.15.47:0/2363391916 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f33c8ea7e0 con 0x55f33cba82b0 2024-03-21T10:25:09.844 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:09.842+0000 7f7a25e00640 1 --2- 172.21.15.47:0/2363391916 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f33cba8ca0 0x55f33cbad7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T10:25:09.844 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:09.842+0000 7f7a1ea00640 1 --2- 172.21.15.47:0/628536271 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f7a0c0a2f20 0x7f7a0c0a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:25:09.844 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:09.842+0000 7f7a1ea00640 1 -- 172.21.15.47:0/628536271 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9216+0+0 (secure 0 0 0) 0x7f7a100f1b40 con 0x7f7a28102770 2024-03-21T10:25:09.844 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:09.842+0000 7f7a26800640 1 --2- 172.21.15.47:0/2363391916 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f33cba82b0 0x55f33cba86d0 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f7a2000ac90 tx=0x7f7a2000e720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b9e6bd36cf88a2c7 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:25:09.844 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:09.842+0000 7f7a25e00640 1 --2- 172.21.15.47:0/628536271 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f7a0c0a2f20 0x7f7a0c0a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:25:09.848 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:09.842+0000 7f7a17400640 1 -- 172.21.15.47:0/2363391916 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a20005220 con 0x55f33cba82b0 2024-03-21T10:25:09.848 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:09.843+0000 7f7a17400640 1 -- 172.21.15.47:0/2363391916 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f7a20004c50 con 0x55f33cba82b0 2024-03-21T10:25:09.848 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:09.843+0000 7f7a17400640 1 -- 172.21.15.47:0/2363391916 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a20010800 con 0x55f33cba82b0 2024-03-21T10:25:09.848 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:09.843+0000 7f7a2fbfa740 1 -- 172.21.15.47:0/2363391916 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f33cba82b0 msgr2=0x55f33cba86d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:25:09.848 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:09.843+0000 7f7a25e00640 1 --2- 172.21.15.47:0/628536271 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f7a0c0a2f20 0x7f7a0c0a53e0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f7a18009fd0 tx=0x7f7a18009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:25:09.849 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:09.843+0000 7f7a2fbfa740 1 --2- 172.21.15.47:0/2363391916 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f33cba82b0 0x55f33cba86d0 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f7a2000ac90 tx=0x7f7a2000e720 comp rx=0 tx=0).stop 2024-03-21T10:25:09.849 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:09.843+0000 7f7a2fbfa740 1 -- 172.21.15.47:0/2363391916 shutdown_connections 2024-03-21T10:25:09.849 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:09.843+0000 7f7a2fbfa740 1 --2- 172.21.15.47:0/2363391916 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f33cbade10 0x55f33cbb02a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:25:09.849 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:09.843+0000 7f7a2fbfa740 1 --2- 172.21.15.47:0/2363391916 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f33cba8ca0 0x55f33cbad7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:25:09.849 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:09.843+0000 7f7a2fbfa740 1 --2- 172.21.15.47:0/2363391916 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f33cba82b0 0x55f33cba86d0 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:25:09.849 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:09.843+0000 7f7a2fbfa740 1 -- 172.21.15.47:0/2363391916 >> 172.21.15.47:0/2363391916 conn(0x55f33cba3820 msgr2=0x55f33cba3c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:25:09.849 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:09.844+0000 7f7a2fbfa740 1 -- 172.21.15.47:0/2363391916 shutdown_connections 2024-03-21T10:25:09.849 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:09.844+0000 7f7a2fbfa740 1 -- 172.21.15.47:0/2363391916 wait complete. 2024-03-21T10:25:09.849 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:09.844+0000 7f7a2fbfa740 1 -- start start 2024-03-21T10:25:09.849 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:09.844+0000 7f7a2fbfa740 1 Processor -- start 2024-03-21T10:25:09.849 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:09.845+0000 7f7a2fbfa740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f33cba82b0 0x55f33cbb2520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:25:09.849 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:09.845+0000 7f7a2fbfa740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f33cba8ca0 0x55f33cbb2a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:25:09.849 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:09.845+0000 7f7a2fbfa740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f33cbade10 0x55f33cbb9aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:25:09.850 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:09.845+0000 7f7a2fbfa740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f33cbb0dc0 con 0x55f33cba82b0 2024-03-21T10:25:09.850 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:09.845+0000 7f7a2fbfa740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f33cac9eb0 con 0x55f33cbade10 2024-03-21T10:25:09.850 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:09.845+0000 7f7a2fbfa740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f33cbb0f40 con 0x55f33cba8ca0 2024-03-21T10:25:09.850 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:09.845+0000 7f7a2fbfa740 10 client.0 _ll_register_callbacks cb 0x55f33cabe150 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T10:25:09.850 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:09.845+0000 7f7a25e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f33cba82b0 0x55f33cbb2520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:25:09.850 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:09.845+0000 7f7a25e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f33cba82b0 0x55f33cbb2520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:56492/0 (socket says 172.21.15.47:56492) 2024-03-21T10:25:09.850 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:09.845+0000 7f7a25e00640 1 -- 172.21.15.47:0/3401275646 learned_addr learned my addr 172.21.15.47:0/3401275646 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:25:09.850 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:09.845+0000 7f7a26800640 1 --2- 172.21.15.47:0/3401275646 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f33cbade10 0x55f33cbb9aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:25:09.850 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:09.846+0000 7f7a25400640 1 --2- 172.21.15.47:0/3401275646 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f33cba8ca0 0x55f33cbb2a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:25:09.850 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:09.846+0000 7f7a25e00640 1 -- 172.21.15.47:0/3401275646 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f33cba8ca0 msgr2=0x55f33cbb2a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:25:09.850 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:09.846+0000 7f7a25e00640 1 --2- 172.21.15.47:0/3401275646 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f33cba8ca0 0x55f33cbb2a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:25:09.850 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:09.846+0000 7f7a25e00640 1 -- 172.21.15.47:0/3401275646 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f33cbade10 msgr2=0x55f33cbb9aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:25:09.850 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:09.846+0000 7f7a25e00640 1 --2- 172.21.15.47:0/3401275646 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f33cbade10 0x55f33cbb9aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:25:09.850 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:09.846+0000 7f7a25e00640 1 -- 172.21.15.47:0/3401275646 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55f33c94af70 con 0x55f33cba82b0 2024-03-21T10:25:09.851 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:09.846+0000 7f7a26800640 1 --2- 172.21.15.47:0/3401275646 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f33cbade10 0x55f33cbb9aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T10:25:09.851 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:09.846+0000 7f7a25e00640 1 --2- 172.21.15.47:0/3401275646 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f33cba82b0 0x55f33cbb2520 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f7a1800f6d0 tx=0x7f7a18004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=49095ded67c700ab server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:25:09.851 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:09.846+0000 7f7a0be00640 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-03-21T10:25:09.851 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:09.846+0000 7f7a0be00640 1 -- 172.21.15.47:0/3401275646 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a180118d0 con 0x55f33cba82b0 2024-03-21T10:25:09.851 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:09.846+0000 7f7a0be00640 1 -- 172.21.15.47:0/3401275646 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9216+0+0 (secure 0 0 0) 0x7f7a18011a70 con 0x55f33cba82b0 2024-03-21T10:25:09.851 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:09.847+0000 7f7a2fbfa740 1 -- 172.21.15.47:0/3401275646 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f7a20004c50 con 0x55f33cba82b0 2024-03-21T10:25:09.851 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:09.847+0000 7f7a0be00640 1 -- 172.21.15.47:0/3401275646 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7a18011c70 con 0x55f33cba82b0 2024-03-21T10:25:09.851 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:09.847+0000 7f7a2fbfa740 10 client.15642 fetch_fsmap learned FSMap version 30 2024-03-21T10:25:09.851 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:09.847+0000 7f7a2fbfa740 1 -- 172.21.15.47:0/3401275646 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x55f33c7e2170 con 0x55f33cba82b0 2024-03-21T10:25:09.851 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:09.847+0000 7f7a0be00640 1 -- 172.21.15.47:0/3401275646 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f7a20004c50 con 0x55f33cba82b0 2024-03-21T10:25:09.851 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:09.847+0000 7f7a0be00640 1 -- 172.21.15.47:0/3401275646 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 30) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f7a18027070 con 0x55f33cba82b0 2024-03-21T10:25:09.851 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:09.847+0000 7f7a2fbfa740 10 client.15642 fetch_fsmap finished waiting for FSMap version 30 2024-03-21T10:25:09.851 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:09.847+0000 7f7a2fbfa740 10 client.15642 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-21T10:25:09.851 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:09.847+0000 7f7a2fbfa740 20 client.15642 populate_metadata read hostname 'smithi047' 2024-03-21T10:25:09.851 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:09.847+0000 7f7a2fbfa740 1 --2- 172.21.15.47:0/3401275646 >> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] conn(0x55f33cbbe3f0 0x55f33cbc0850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:25:09.852 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:09.847+0000 7f7a2fbfa740 4 client.15642 mds_command: new command op to 34535 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T10:25:09.852 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:09.847+0000 7f7a2fbfa740 1 -- 172.21.15.47:0/3401275646 --> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55f33cb1b3f0 con 0x55f33cbbe3f0 2024-03-21T10:25:09.853 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:09.848+0000 7f7a25400640 1 --2- 172.21.15.47:0/3401275646 >> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] conn(0x55f33cbbe3f0 0x55f33cbc0850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:25:09.853 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:09.849+0000 7f7a25400640 1 --2- 172.21.15.47:0/3401275646 >> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] conn(0x55f33cbbe3f0 0x55f33cbc0850 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6e4fb23b2b3f6f73 server_cookie=554dc287730a454 in_seq=0 out_seq=0 2024-03-21T10:25:09.853 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:09.849+0000 7f7a0be00640 10 client.15642 ms_handle_connect on v2:172.21.15.100:6838/4173168440 2024-03-21T10:25:09.853 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:09.850+0000 7f7a0be00640 1 -- 172.21.15.47:0/3401275646 <== mds.0 v2:172.21.15.100:6838/4173168440 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55f33cb1b3f0 con 0x55f33cbbe3f0 2024-03-21T10:25:09.853 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:09.850+0000 7f7a0be00640 10 client.15642 handle_command_reply: tid=0 2024-03-21T10:25:09.853 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:09.850+0000 7f7a2fbfa740 1 client.15642 shutdown 2024-03-21T10:25:09.853 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:09.850+0000 7f7a2fbfa740 1 -- 172.21.15.47:0/3401275646 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f33cba82b0 msgr2=0x55f33cbb2520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:25:09.853 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:09.850+0000 7f7a2fbfa740 1 --2- 172.21.15.47:0/3401275646 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f33cba82b0 0x55f33cbb2520 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f7a1800f6d0 tx=0x7f7a18004ab0 comp rx=0 tx=0).stop 2024-03-21T10:25:09.853 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:09.850+0000 7f7a2fbfa740 1 -- 172.21.15.47:0/3401275646 shutdown_connections 2024-03-21T10:25:09.854 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:09.850+0000 7f7a2fbfa740 1 --2- 172.21.15.47:0/3401275646 >> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] conn(0x55f33cbbe3f0 0x55f33cbc0850 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:25:09.854 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:09.850+0000 7f7a2fbfa740 1 --2- 172.21.15.47:0/3401275646 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f33cbade10 0x55f33cbb9aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:25:09.854 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:09.850+0000 7f7a2fbfa740 1 --2- 172.21.15.47:0/3401275646 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f33cba8ca0 0x55f33cbb2a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:25:09.854 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:09.850+0000 7f7a2fbfa740 1 --2- 172.21.15.47:0/3401275646 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f33cba82b0 0x55f33cbb2520 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:25:09.854 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:09.850+0000 7f7a0be00640 0 client.15642 ms_handle_reset on v2:172.21.15.100:6838/4173168440 2024-03-21T10:25:09.854 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:09.850+0000 7f7a2fbfa740 1 -- 172.21.15.47:0/3401275646 >> 172.21.15.47:0/3401275646 conn(0x55f33cba3820 msgr2=0x55f33cba2a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:25:09.854 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:09.851+0000 7f7a2fbfa740 1 -- 172.21.15.47:0/3401275646 shutdown_connections 2024-03-21T10:25:09.854 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:09.851+0000 7f7a2fbfa740 1 -- 172.21.15.47:0/3401275646 wait complete. 2024-03-21T10:25:09.854 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:09.851+0000 7f7a2fbfa740 20 client.15642 trim_cache size 0 max 16384 2024-03-21T10:25:09.860 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:09.859+0000 7f8686800640 1 -- 172.21.15.47:0/402672547 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f86801054b0 msgr2=0x7f868010bd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:25:09.861 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:09.859+0000 7f8686800640 1 --2- 172.21.15.47:0/402672547 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f86801054b0 0x7f868010bd40 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f8678009ca0 tx=0x7f867802c0f0 comp rx=0 tx=0).stop 2024-03-21T10:25:09.861 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:09.860+0000 7f8686800640 1 -- 172.21.15.47:0/402672547 shutdown_connections 2024-03-21T10:25:09.861 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:09.860+0000 7f8686800640 1 --2- 172.21.15.47:0/402672547 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f86801054b0 0x7f868010bd40 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:25:09.861 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:09.860+0000 7f8686800640 1 --2- 172.21.15.47:0/402672547 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8680104af0 0x7f8680104f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:25:09.861 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:09.860+0000 7f8686800640 1 --2- 172.21.15.47:0/402672547 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8680103900 0x7f8680103d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:25:09.861 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:09.860+0000 7f8686800640 1 -- 172.21.15.47:0/402672547 >> 172.21.15.47:0/402672547 conn(0x7f86800fee90 msgr2=0x7f86801012b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:25:09.861 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:09.860+0000 7f8686800640 1 -- 172.21.15.47:0/402672547 shutdown_connections 2024-03-21T10:25:09.861 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:09.860+0000 7f8686800640 1 -- 172.21.15.47:0/402672547 wait complete. 2024-03-21T10:25:09.861 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:09.861+0000 7f8686800640 1 Processor -- start 2024-03-21T10:25:09.863 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:09.861+0000 7f8686800640 1 -- start start 2024-03-21T10:25:09.863 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:09.861+0000 7f8686800640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8680103900 0x7f86801999e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:25:09.863 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:09.861+0000 7f8686800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8680104af0 0x7f8680199f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:25:09.863 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:09.862+0000 7f8686800640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f86801054b0 0x7f86801a0f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:25:09.863 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:09.862+0000 7f8686800640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f868010f430 con 0x7f86801054b0 2024-03-21T10:25:09.863 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:09.862+0000 7f8686800640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f868010f2b0 con 0x7f8680104af0 2024-03-21T10:25:09.863 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:09.862+0000 7f8686800640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f868010f5b0 con 0x7f8680103900 2024-03-21T10:25:09.863 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:09.862+0000 7f867fe00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f86801054b0 0x7f86801a0f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:25:09.863 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:09.862+0000 7f867fe00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f86801054b0 0x7f86801a0f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:56514/0 (socket says 172.21.15.47:56514) 2024-03-21T10:25:09.863 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:09.862+0000 7f867fe00640 1 -- 172.21.15.47:0/1749301017 learned_addr learned my addr 172.21.15.47:0/1749301017 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:25:09.863 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:09.862+0000 7f867ea00640 1 --2- 172.21.15.47:0/1749301017 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8680104af0 0x7f8680199f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:25:09.863 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:09.862+0000 7f867f400640 1 --2- 172.21.15.47:0/1749301017 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8680103900 0x7f86801999e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:25:09.865 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:09.862+0000 7f867fe00640 1 -- 172.21.15.47:0/1749301017 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8680103900 msgr2=0x7f86801999e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:25:09.865 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:09.862+0000 7f867fe00640 1 --2- 172.21.15.47:0/1749301017 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8680103900 0x7f86801999e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:25:09.865 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:09.862+0000 7f867fe00640 1 -- 172.21.15.47:0/1749301017 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8680104af0 msgr2=0x7f8680199f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:25:09.865 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:09.862+0000 7f867fe00640 1 --2- 172.21.15.47:0/1749301017 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8680104af0 0x7f8680199f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:25:09.865 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:09.862+0000 7f867fe00640 1 -- 172.21.15.47:0/1749301017 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86801a1600 con 0x7f86801054b0 2024-03-21T10:25:09.865 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:09.863+0000 7f867fe00640 1 --2- 172.21.15.47:0/1749301017 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f86801054b0 0x7f86801a0f90 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f8678032380 tx=0x7f8678032820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:25:09.865 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:09.863+0000 7f8677e00640 1 -- 172.21.15.47:0/1749301017 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8678041020 con 0x7f86801054b0 2024-03-21T10:25:09.865 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:09.863+0000 7f8677e00640 1 -- 172.21.15.47:0/1749301017 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f8678003e20 con 0x7f86801054b0 2024-03-21T10:25:09.865 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:09.863+0000 7f8686800640 1 -- 172.21.15.47:0/1749301017 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f86801a1890 con 0x7f86801054b0 2024-03-21T10:25:09.865 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:09.863+0000 7f8677e00640 1 -- 172.21.15.47:0/1749301017 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8678039040 con 0x7f86801054b0 2024-03-21T10:25:09.865 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:09.863+0000 7f8686800640 1 -- 172.21.15.47:0/1749301017 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f86801a1dd0 con 0x7f86801054b0 2024-03-21T10:25:09.866 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:09.865+0000 7f8686800640 1 -- 172.21.15.47:0/1749301017 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f868010ac20 con 0x7f86801054b0 2024-03-21T10:25:09.869 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:09.869+0000 7f8677e00640 1 -- 172.21.15.47:0/1749301017 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 18) v1 ==== 137854+0+0 (secure 0 0 0) 0x7f8678044020 con 0x7f86801054b0 2024-03-21T10:25:09.871 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:09.870+0000 7f8677e00640 1 --2- 172.21.15.47:0/1749301017 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f86640a3040 0x7f86640a5500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:25:09.871 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:09.871+0000 7f8677e00640 1 -- 172.21.15.47:0/1749301017 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9216+0+0 (secure 0 0 0) 0x7f86780f1950 con 0x7f86801054b0 2024-03-21T10:25:09.872 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:09.871+0000 7f867f400640 1 --2- 172.21.15.47:0/1749301017 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f86640a3040 0x7f86640a5500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:25:09.873 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:09.872+0000 7f867f400640 1 --2- 172.21.15.47:0/1749301017 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f86640a3040 0x7f86640a5500 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f8670005fd0 tx=0x7f867000b390 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:25:09.877 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:09.876+0000 7f8677e00640 1 -- 172.21.15.47:0/1749301017 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f867804b020 con 0x7f86801054b0 2024-03-21T10:25:09.878 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:09.877+0000 7f7a2fbfa740 1 Processor -- start 2024-03-21T10:25:09.878 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:09.877+0000 7f7a2fbfa740 1 -- start start 2024-03-21T10:25:09.878 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:09.877+0000 7f7a2fbfa740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f33cba82b0 0x55f33cba2860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:25:09.878 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:09.877+0000 7f7a2fbfa740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f33cba8ca0 0x55f33cbb9a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:25:09.878 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:09.877+0000 7f7a2fbfa740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f33cbade10 0x55f33cba9530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:25:09.878 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:09.877+0000 7f7a2fbfa740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a281ac0f0 con 0x55f33cbade10 2024-03-21T10:25:09.878 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:09.877+0000 7f7a2fbfa740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a281ada70 con 0x55f33cba8ca0 2024-03-21T10:25:09.879 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:09.877+0000 7f7a2fbfa740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a281ac750 con 0x55f33cba82b0 2024-03-21T10:25:09.880 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:09.878+0000 7f7a25400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f33cba8ca0 0x55f33cbb9a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:25:09.880 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:09.878+0000 7f7a25400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f33cba8ca0 0x55f33cbb9a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:50304/0 (socket says 172.21.15.47:50304) 2024-03-21T10:25:09.880 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:09.878+0000 7f7a25400640 1 -- 172.21.15.47:0/3943462546 learned_addr learned my addr 172.21.15.47:0/3943462546 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:25:09.880 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:09.878+0000 7f7a26800640 1 --2- 172.21.15.47:0/3943462546 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f33cbade10 0x55f33cba9530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:25:09.880 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:09.878+0000 7f7a25e00640 1 --2- 172.21.15.47:0/3943462546 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f33cba82b0 0x55f33cba2860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:25:09.880 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:09.879+0000 7f7a25400640 1 -- 172.21.15.47:0/3943462546 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f33cba82b0 msgr2=0x55f33cba2860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:25:09.880 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:09.879+0000 7f7a25400640 1 --2- 172.21.15.47:0/3943462546 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f33cba82b0 0x55f33cba2860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:25:09.880 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:09.879+0000 7f7a25400640 1 -- 172.21.15.47:0/3943462546 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f33cbade10 msgr2=0x55f33cba9530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:25:09.881 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:09.879+0000 7f7a25400640 1 --2- 172.21.15.47:0/3943462546 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f33cbade10 0x55f33cba9530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:25:09.881 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:09.879+0000 7f7a25400640 1 -- 172.21.15.47:0/3943462546 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f33cbbe1d0 con 0x55f33cba8ca0 2024-03-21T10:25:09.881 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:09.879+0000 7f7a25400640 1 --2- 172.21.15.47:0/3943462546 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f33cba8ca0 0x55f33cbb9a60 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f7a1002ce10 tx=0x7f7a100f3fd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cba3f64bc47135b2 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:25:09.881 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:09.880+0000 7f7a16a00640 1 -- 172.21.15.47:0/3943462546 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a100044e0 con 0x55f33cba8ca0 2024-03-21T10:25:09.881 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:09.880+0000 7f7a16a00640 1 -- 172.21.15.47:0/3943462546 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f7a100f9040 con 0x55f33cba8ca0 2024-03-21T10:25:09.881 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:09.880+0000 7f7a16a00640 1 -- 172.21.15.47:0/3943462546 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a100c1ff0 con 0x55f33cba8ca0 2024-03-21T10:25:09.882 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:09.881+0000 7f7a2fbfa740 1 -- 172.21.15.47:0/3943462546 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f33cba8ca0 msgr2=0x55f33cbb9a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:25:09.882 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:09.881+0000 7f7a2fbfa740 1 --2- 172.21.15.47:0/3943462546 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f33cba8ca0 0x55f33cbb9a60 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f7a1002ce10 tx=0x7f7a100f3fd0 comp rx=0 tx=0).stop 2024-03-21T10:25:09.882 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:09.881+0000 7f7a2fbfa740 1 -- 172.21.15.47:0/3943462546 shutdown_connections 2024-03-21T10:25:09.882 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:09.881+0000 7f7a2fbfa740 1 --2- 172.21.15.47:0/3943462546 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f33cbade10 0x55f33cba9530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:25:09.883 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:09.881+0000 7f7a2fbfa740 1 --2- 172.21.15.47:0/3943462546 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f33cba8ca0 0x55f33cbb9a60 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:25:09.883 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:09.881+0000 7f7a2fbfa740 1 --2- 172.21.15.47:0/3943462546 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f33cba82b0 0x55f33cba2860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:25:09.883 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:09.881+0000 7f7a2fbfa740 1 -- 172.21.15.47:0/3943462546 >> 172.21.15.47:0/3943462546 conn(0x55f33cba3820 msgr2=0x55f33cba7310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:25:09.883 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:09.881+0000 7f7a2fbfa740 1 -- 172.21.15.47:0/3943462546 shutdown_connections 2024-03-21T10:25:09.883 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:09.881+0000 7f7a2fbfa740 1 -- 172.21.15.47:0/3943462546 wait complete. 2024-03-21T10:25:09.884 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:09.882+0000 7f7a2fbfa740 1 -- start start 2024-03-21T10:25:09.884 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:09.882+0000 7f7a2fbfa740 1 Processor -- start 2024-03-21T10:25:09.884 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:09.883+0000 7f7a2fbfa740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f33cba82b0 0x55f33cbb8390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:25:09.884 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:09.883+0000 7f7a2fbfa740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f33cba8ca0 0x55f33cbb88d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:25:09.884 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:09.883+0000 7f7a2fbfa740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f33cbade10 0x55f33cbb8e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:25:09.884 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:09.883+0000 7f7a2fbfa740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a281ad410 con 0x55f33cba8ca0 2024-03-21T10:25:09.884 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:09.883+0000 7f7a2fbfa740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a281acdb0 con 0x55f33cba82b0 2024-03-21T10:25:09.885 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:09.883+0000 7f7a2fbfa740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a281aa290 con 0x55f33cbade10 2024-03-21T10:25:09.885 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:09.883+0000 7f7a2fbfa740 10 client.0 _ll_register_callbacks cb 0x7f7a18009ca0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T10:25:09.885 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:09.883+0000 7f7a25e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f33cba82b0 0x55f33cbb8390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:25:09.885 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:09.883+0000 7f7a25400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f33cba8ca0 0x55f33cbb88d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:25:09.885 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:09.883+0000 7f7a25e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f33cba82b0 0x55f33cbb8390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:50316/0 (socket says 172.21.15.47:50316) 2024-03-21T10:25:09.885 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:09.883+0000 7f7a25400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f33cba8ca0 0x55f33cbb88d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:56522/0 (socket says 172.21.15.47:56522) 2024-03-21T10:25:09.885 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:09.883+0000 7f7a25e00640 1 -- 172.21.15.47:0/3461646476 learned_addr learned my addr 172.21.15.47:0/3461646476 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:25:09.885 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:09.883+0000 7f7a26800640 1 --2- 172.21.15.47:0/3461646476 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f33cbade10 0x55f33cbb8e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:25:09.887 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:09.884+0000 7f7a25e00640 1 -- 172.21.15.47:0/3461646476 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f33cbade10 msgr2=0x55f33cbb8e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:25:09.887 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:09.884+0000 7f7a25e00640 1 --2- 172.21.15.47:0/3461646476 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f33cbade10 0x55f33cbb8e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:25:09.887 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:09.884+0000 7f7a25e00640 1 -- 172.21.15.47:0/3461646476 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f33cba8ca0 msgr2=0x55f33cbb88d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:25:09.887 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:09.884+0000 7f7a25e00640 1 --2- 172.21.15.47:0/3461646476 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f33cba8ca0 0x55f33cbb88d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:25:09.887 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:09.884+0000 7f7a25e00640 1 -- 172.21.15.47:0/3461646476 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55f33cba4290 con 0x55f33cba82b0 2024-03-21T10:25:09.887 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:09.884+0000 7f7a25400640 1 --2- 172.21.15.47:0/3461646476 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f33cba8ca0 0x55f33cbb88d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T10:25:09.888 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:09.884+0000 7f7a25e00640 1 --2- 172.21.15.47:0/3461646476 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f33cba82b0 0x55f33cbb8390 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f7a18029ef0 tx=0x7f7a18011be0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b36d74d2606e4196 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:25:09.888 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:09.884+0000 7f7a0be00640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-21T10:25:09.888 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:09.884+0000 7f7a0be00640 1 -- 172.21.15.47:0/3461646476 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a18029990 con 0x55f33cba82b0 2024-03-21T10:25:09.888 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:09.884+0000 7f7a2fbfa740 1 -- 172.21.15.47:0/3461646476 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f7a100f9040 con 0x55f33cba82b0 2024-03-21T10:25:09.888 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:09.885+0000 7f7a0be00640 1 -- 172.21.15.47:0/3461646476 <== mon.2 v2:172.21.15.100:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9216+0+0 (secure 0 0 0) 0x7f7a18029b30 con 0x55f33cba82b0 2024-03-21T10:25:09.888 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:09.885+0000 7f7a0be00640 1 -- 172.21.15.47:0/3461646476 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7a18045050 con 0x55f33cba82b0 2024-03-21T10:25:09.888 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:09.885+0000 7f7a2fbfa740 10 client.35483 fetch_fsmap learned FSMap version 30 2024-03-21T10:25:09.888 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:09.885+0000 7f7a2fbfa740 1 -- 172.21.15.47:0/3461646476 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x55f33cc1b040 con 0x55f33cba82b0 2024-03-21T10:25:09.888 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:09.885+0000 7f7a0be00640 1 -- 172.21.15.47:0/3461646476 <== mon.2 v2:172.21.15.100:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f7a100f9040 con 0x55f33cba82b0 2024-03-21T10:25:09.888 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:09.886+0000 7f7a0be00640 1 -- 172.21.15.47:0/3461646476 <== mon.2 v2:172.21.15.100:3300/0 5 ==== fsmap(e 30) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f7a18027040 con 0x55f33cba82b0 2024-03-21T10:25:09.888 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:09.886+0000 7f7a2fbfa740 10 client.35483 fetch_fsmap finished waiting for FSMap version 30 2024-03-21T10:25:09.888 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:09.886+0000 7f7a2fbfa740 10 client.35483 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-21T10:25:09.888 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:09.886+0000 7f7a2fbfa740 20 client.35483 populate_metadata read hostname 'smithi047' 2024-03-21T10:25:09.888 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:09.886+0000 7f7a2fbfa740 1 --2- 172.21.15.47:0/3461646476 >> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] conn(0x55f33cc1b9c0 0x55f33cc1dde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:25:09.888 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:09.886+0000 7f7a2fbfa740 4 client.35483 mds_command: new command op to 34535 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-03-21T10:25:09.889 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:09.886+0000 7f7a2fbfa740 1 -- 172.21.15.47:0/3461646476 --> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x55f33cb15740 con 0x55f33cc1b9c0 2024-03-21T10:25:09.889 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:09.886+0000 7f7a25400640 1 --2- 172.21.15.47:0/3461646476 >> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] conn(0x55f33cc1b9c0 0x55f33cc1dde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:25:09.889 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:09.887+0000 7f7a25400640 1 --2- 172.21.15.47:0/3461646476 >> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] conn(0x55f33cc1b9c0 0x55f33cc1dde0 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7852c18320f3fe2e server_cookie=8e307660e698386f in_seq=0 out_seq=0 2024-03-21T10:25:09.889 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:09.887+0000 7f7a0be00640 10 client.35483 ms_handle_connect on v2:172.21.15.100:6838/4173168440 2024-03-21T10:25:09.892 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:09.890+0000 7f7a0be00640 1 -- 172.21.15.47:0/3461646476 <== mds.0 v2:172.21.15.100:6838/4173168440 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x55f33cb15740 con 0x55f33cc1b9c0 2024-03-21T10:25:09.892 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:09.890+0000 7f7a0be00640 10 client.35483 handle_command_reply: tid=0 2024-03-21T10:25:09.892 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:09.891+0000 7f7a2fbfa740 1 client.35483 shutdown 2024-03-21T10:25:09.892 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:09.891+0000 7f7a2fbfa740 1 -- 172.21.15.47:0/3461646476 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f33cba82b0 msgr2=0x55f33cbb8390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:25:09.892 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:09.891+0000 7f7a2fbfa740 1 --2- 172.21.15.47:0/3461646476 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f33cba82b0 0x55f33cbb8390 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f7a18029ef0 tx=0x7f7a18011be0 comp rx=0 tx=0).stop 2024-03-21T10:25:09.893 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:09.891+0000 7f7a2fbfa740 1 -- 172.21.15.47:0/3461646476 shutdown_connections 2024-03-21T10:25:09.893 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:09.891+0000 7f7a2fbfa740 1 --2- 172.21.15.47:0/3461646476 >> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] conn(0x55f33cc1b9c0 0x55f33cc1dde0 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:25:09.893 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:09.891+0000 7f7a2fbfa740 1 --2- 172.21.15.47:0/3461646476 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f33cbade10 0x55f33cbb8e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:25:09.893 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:09.891+0000 7f7a2fbfa740 1 --2- 172.21.15.47:0/3461646476 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f33cba8ca0 0x55f33cbb88d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:25:09.893 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:09.891+0000 7f7a2fbfa740 1 --2- 172.21.15.47:0/3461646476 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f33cba82b0 0x55f33cbb8390 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:25:09.893 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:09.891+0000 7f7a0be00640 0 client.35483 ms_handle_reset on v2:172.21.15.100:6838/4173168440 2024-03-21T10:25:09.893 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:09.891+0000 7f7a2fbfa740 1 -- 172.21.15.47:0/3461646476 >> 172.21.15.47:0/3461646476 conn(0x55f33cba3820 msgr2=0x55f33cbc5110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:25:09.893 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:09.891+0000 7f7a2fbfa740 1 -- 172.21.15.47:0/3461646476 shutdown_connections 2024-03-21T10:25:09.893 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:09.891+0000 7f7a2fbfa740 1 -- 172.21.15.47:0/3461646476 wait complete. 2024-03-21T10:25:09.893 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:09.891+0000 7f7a2fbfa740 20 client.35483 trim_cache size 0 max 16384 2024-03-21T10:25:09.893 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-03-21T10:25:09.894 INFO:teuthology.orchestra.run.smithi047.stdout: "return_code": 0, 2024-03-21T10:25:09.894 INFO:teuthology.orchestra.run.smithi047.stdout: "scrub_tag": "64186214-1d58-46f0-8165-8cd56e92c6b1", 2024-03-21T10:25:09.894 INFO:teuthology.orchestra.run.smithi047.stdout: "mode": "asynchronous" 2024-03-21T10:25:09.894 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-03-21T10:25:09.894 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:09.892+0000 7f7a27200640 1 -- 172.21.15.47:0/628536271 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f7a0c0a2f20 msgr2=0x7f7a0c0a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:25:09.895 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:09.892+0000 7f7a27200640 1 --2- 172.21.15.47:0/628536271 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f7a0c0a2f20 0x7f7a0c0a53e0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f7a18009fd0 tx=0x7f7a18009500 comp rx=0 tx=0).stop 2024-03-21T10:25:09.895 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:09.892+0000 7f7a27200640 1 -- 172.21.15.47:0/628536271 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7a28102770 msgr2=0x7f7a28073a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:25:09.895 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:09.892+0000 7f7a27200640 1 --2- 172.21.15.47:0/628536271 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7a28102770 0x7f7a28073a90 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f7a10000c00 tx=0x7f7a100042f0 comp rx=0 tx=0).stop 2024-03-21T10:25:09.895 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:09.892+0000 7f7a27200640 1 -- 172.21.15.47:0/628536271 shutdown_connections 2024-03-21T10:25:09.895 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:09.892+0000 7f7a27200640 1 --2- 172.21.15.47:0/628536271 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f7a0c0a2f20 0x7f7a0c0a53e0 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:25:09.895 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:09.892+0000 7f7a27200640 1 --2- 172.21.15.47:0/628536271 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7a28103130 0x7f7a28073fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:25:09.895 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:09.892+0000 7f7a27200640 1 --2- 172.21.15.47:0/628536271 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7a28102770 0x7f7a28073a90 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:25:09.895 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:09.892+0000 7f7a27200640 1 --2- 172.21.15.47:0/628536271 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7a28101580 0x7f7a28075450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:25:09.895 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:09.892+0000 7f7a27200640 1 -- 172.21.15.47:0/628536271 >> 172.21.15.47:0/628536271 conn(0x7f7a280fcb10 msgr2=0x7f7a28105a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:25:09.895 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:09.893+0000 7f7a27200640 1 -- 172.21.15.47:0/628536271 shutdown_connections 2024-03-21T10:25:09.895 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:09.893+0000 7f7a27200640 1 -- 172.21.15.47:0/628536271 wait complete. 2024-03-21T10:25:09.911 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-21T10:25:09.912 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-21T10:25:10.125 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:10.123+0000 7f172d800640 1 -- 172.21.15.47:0/3939137516 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1728107390 msgr2=0x7f17281097e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:25:10.125 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:10.123+0000 7f172d800640 1 --2- 172.21.15.47:0/3939137516 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1728107390 0x7f17281097e0 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f1720009ca0 tx=0x7f172002c0f0 comp rx=0 tx=0).stop 2024-03-21T10:25:10.125 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:10.123+0000 7f172d800640 1 -- 172.21.15.47:0/3939137516 shutdown_connections 2024-03-21T10:25:10.125 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:10.123+0000 7f172d800640 1 --2- 172.21.15.47:0/3939137516 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1728107390 0x7f17281097e0 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:25:10.125 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:10.123+0000 7f172d800640 1 --2- 172.21.15.47:0/3939137516 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1728104a60 0x7f1728106e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:25:10.125 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:10.123+0000 7f172d800640 1 --2- 172.21.15.47:0/3939137516 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1728102130 0x7f1728104520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:25:10.125 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:10.123+0000 7f172d800640 1 -- 172.21.15.47:0/3939137516 >> 172.21.15.47:0/3939137516 conn(0x7f17280fbb40 msgr2=0x7f17280fdf60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:25:10.125 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:10.123+0000 7f172d800640 1 -- 172.21.15.47:0/3939137516 shutdown_connections 2024-03-21T10:25:10.125 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:10.123+0000 7f172d800640 1 -- 172.21.15.47:0/3939137516 wait complete. 2024-03-21T10:25:10.127 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:10.125+0000 7f172d800640 1 Processor -- start 2024-03-21T10:25:10.127 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:10.125+0000 7f172d800640 1 -- start start 2024-03-21T10:25:10.127 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:10.125+0000 7f172d800640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1728102130 0x7f1728101aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:25:10.127 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:10.126+0000 7f172d800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1728104a60 0x7f17281000e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:25:10.128 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:10.126+0000 7f172d800640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1728107390 0x7f1728100620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:25:10.128 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:10.126+0000 7f172d800640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f172810d1e0 con 0x7f1728107390 2024-03-21T10:25:10.128 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:10.126+0000 7f172d800640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f172810d060 con 0x7f1728104a60 2024-03-21T10:25:10.128 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:10.126+0000 7f172d800640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f172810d360 con 0x7f1728102130 2024-03-21T10:25:10.128 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:10.126+0000 7f1726000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1728104a60 0x7f17281000e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:25:10.128 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:10.126+0000 7f1727400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1728107390 0x7f1728100620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:25:10.128 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:10.126+0000 7f1726000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1728104a60 0x7f17281000e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:50332/0 (socket says 172.21.15.47:50332) 2024-03-21T10:25:10.128 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:10.126+0000 7f1726000640 1 -- 172.21.15.47:0/3609720064 learned_addr learned my addr 172.21.15.47:0/3609720064 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:25:10.128 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:10.126+0000 7f1726a00640 1 --2- 172.21.15.47:0/3609720064 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1728102130 0x7f1728101aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:25:10.128 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:10.126+0000 7f1727400640 1 -- 172.21.15.47:0/3609720064 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1728102130 msgr2=0x7f1728101aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:25:10.129 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:10.127+0000 7f1727400640 1 --2- 172.21.15.47:0/3609720064 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1728102130 0x7f1728101aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:25:10.129 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:10.127+0000 7f1727400640 1 -- 172.21.15.47:0/3609720064 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1728104a60 msgr2=0x7f17281000e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:25:10.129 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:10.127+0000 7f1727400640 1 --2- 172.21.15.47:0/3609720064 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1728104a60 0x7f17281000e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:25:10.129 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:10.127+0000 7f1727400640 1 -- 172.21.15.47:0/3609720064 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1728100eb0 con 0x7f1728107390 2024-03-21T10:25:10.129 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:10.127+0000 7f1726a00640 1 --2- 172.21.15.47:0/3609720064 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1728102130 0x7f1728101aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T10:25:10.129 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:10.127+0000 7f1726000640 1 --2- 172.21.15.47:0/3609720064 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1728104a60 0x7f17281000e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T10:25:10.130 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:10.127+0000 7f1727400640 1 --2- 172.21.15.47:0/3609720064 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1728107390 0x7f1728100620 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f1720037040 tx=0x7f1720034880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:25:10.130 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:10.127+0000 7f171f400640 1 -- 172.21.15.47:0/3609720064 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1720041020 con 0x7f1728107390 2024-03-21T10:25:10.130 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:10.127+0000 7f171f400640 1 -- 172.21.15.47:0/3609720064 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f1720003e20 con 0x7f1728107390 2024-03-21T10:25:10.130 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:10.127+0000 7f172d800640 1 -- 172.21.15.47:0/3609720064 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f17281a19a0 con 0x7f1728107390 2024-03-21T10:25:10.130 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:10.127+0000 7f171f400640 1 -- 172.21.15.47:0/3609720064 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1720039040 con 0x7f1728107390 2024-03-21T10:25:10.130 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:10.128+0000 7f172d800640 1 -- 172.21.15.47:0/3609720064 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f17281a1ec0 con 0x7f1728107390 2024-03-21T10:25:10.132 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:10.131+0000 7f171f400640 1 -- 172.21.15.47:0/3609720064 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 18) v1 ==== 137854+0+0 (secure 0 0 0) 0x7f1720044020 con 0x7f1728107390 2024-03-21T10:25:10.133 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:10.132+0000 7f17305bb740 1 Processor -- start 2024-03-21T10:25:10.133 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:10.133+0000 7f171f400640 1 --2- 172.21.15.47:0/3609720064 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f170c0a2f20 0x7f170c0a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:25:10.133 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:10.133+0000 7f17305bb740 1 -- start start 2024-03-21T10:25:10.134 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:10.133+0000 7f171f400640 1 -- 172.21.15.47:0/3609720064 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9216+0+0 (secure 0 0 0) 0x7f17200f19c0 con 0x7f1728107390 2024-03-21T10:25:10.134 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:10.133+0000 7f1726000640 1 --2- 172.21.15.47:0/3609720064 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f170c0a2f20 0x7f170c0a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:25:10.134 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:10.133+0000 7f17305bb740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5653be6b7e80 0x5653be6b82a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:25:10.134 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:10.133+0000 7f17305bb740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5653be6b8870 0x5653be6bd370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:25:10.134 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:10.133+0000 7f17305bb740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5653be6bd9e0 0x5653be6bfe70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:25:10.134 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:10.133+0000 7f17305bb740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x5653be61d250 con 0x5653be6bd9e0 2024-03-21T10:25:10.134 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:10.133+0000 7f17305bb740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x5653be5c5a30 con 0x5653be6b8870 2024-03-21T10:25:10.135 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:10.133+0000 7f17305bb740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x5653be6192f0 con 0x5653be6b7e80 2024-03-21T10:25:10.137 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:10.133+0000 7f1727400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5653be6bd9e0 0x5653be6bfe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:25:10.137 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:10.134+0000 7f1727400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5653be6bd9e0 0x5653be6bfe70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:56532/0 (socket says 172.21.15.47:56532) 2024-03-21T10:25:10.137 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:10.134+0000 7f1727400640 1 -- 172.21.15.47:0/784042129 learned_addr learned my addr 172.21.15.47:0/784042129 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:25:10.137 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:10.134+0000 7f1726000640 1 --2- 172.21.15.47:0/784042129 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5653be6b8870 0x5653be6bd370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:25:10.137 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:10.134+0000 7f1726a00640 1 --2- 172.21.15.47:0/784042129 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5653be6b7e80 0x5653be6b82a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:25:10.138 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:10.134+0000 7f1726000640 1 --2- 172.21.15.47:0/3609720064 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f170c0a2f20 0x7f170c0a53e0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f1728101810 tx=0x7f17140108d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:25:10.138 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:10.134+0000 7f1727400640 1 -- 172.21.15.47:0/784042129 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5653be6b7e80 msgr2=0x5653be6b82a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:25:10.138 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:10.134+0000 7f1727400640 1 --2- 172.21.15.47:0/784042129 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5653be6b7e80 0x5653be6b82a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:25:10.138 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:10.134+0000 7f1727400640 1 -- 172.21.15.47:0/784042129 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5653be6b8870 msgr2=0x5653be6bd370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:25:10.138 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:10.134+0000 7f1727400640 1 --2- 172.21.15.47:0/784042129 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5653be6b8870 0x5653be6bd370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:25:10.138 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:10.134+0000 7f1727400640 1 -- 172.21.15.47:0/784042129 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5653be5b0690 con 0x5653be6bd9e0 2024-03-21T10:25:10.139 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:10.134+0000 7f1726000640 1 --2- 172.21.15.47:0/784042129 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5653be6b8870 0x5653be6bd370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T10:25:10.139 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:10.134+0000 7f1727400640 1 --2- 172.21.15.47:0/784042129 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5653be6bd9e0 0x5653be6bfe70 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f17200bd9e0 tx=0x7f172002c0f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7d8ff5a176506e39 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:25:10.139 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:10.135+0000 7f1713e00640 1 -- 172.21.15.47:0/784042129 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1720032920 con 0x5653be6bd9e0 2024-03-21T10:25:10.139 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:10.135+0000 7f1713e00640 1 -- 172.21.15.47:0/784042129 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f17200be3e0 con 0x5653be6bd9e0 2024-03-21T10:25:10.139 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:10.135+0000 7f1713e00640 1 -- 172.21.15.47:0/784042129 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17200ba350 con 0x5653be6bd9e0 2024-03-21T10:25:10.139 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:10.135+0000 7f17305bb740 1 -- 172.21.15.47:0/784042129 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5653be6bd9e0 msgr2=0x5653be6bfe70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:25:10.140 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:10.135+0000 7f17305bb740 1 --2- 172.21.15.47:0/784042129 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5653be6bd9e0 0x5653be6bfe70 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f17200bd9e0 tx=0x7f172002c0f0 comp rx=0 tx=0).stop 2024-03-21T10:25:10.140 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:10.136+0000 7f17305bb740 1 -- 172.21.15.47:0/784042129 shutdown_connections 2024-03-21T10:25:10.140 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:10.136+0000 7f17305bb740 1 --2- 172.21.15.47:0/784042129 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5653be6bd9e0 0x5653be6bfe70 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:25:10.140 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:10.136+0000 7f17305bb740 1 --2- 172.21.15.47:0/784042129 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5653be6b8870 0x5653be6bd370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:25:10.140 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:10.136+0000 7f17305bb740 1 --2- 172.21.15.47:0/784042129 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5653be6b7e80 0x5653be6b82a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:25:10.140 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:10.136+0000 7f17305bb740 1 -- 172.21.15.47:0/784042129 >> 172.21.15.47:0/784042129 conn(0x5653be680200 msgr2=0x5653be6b37f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:25:10.140 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:10.136+0000 7f17305bb740 1 -- 172.21.15.47:0/784042129 shutdown_connections 2024-03-21T10:25:10.141 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:10.136+0000 7f17305bb740 1 -- 172.21.15.47:0/784042129 wait complete. 2024-03-21T10:25:10.141 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:10.136+0000 7f17305bb740 1 -- start start 2024-03-21T10:25:10.141 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:10.136+0000 7f17305bb740 1 Processor -- start 2024-03-21T10:25:10.144 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:10.137+0000 7f17305bb740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5653be6b7e80 0x5653be6c1ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:25:10.145 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:10.137+0000 7f17305bb740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5653be6b8870 0x5653be6c2000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:25:10.145 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:10.137+0000 7f17305bb740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5653be6bd9e0 0x5653be6c9040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:25:10.145 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:10.137+0000 7f17305bb740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x5653be6c0750 con 0x5653be6b7e80 2024-03-21T10:25:10.145 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:10.137+0000 7f17305bb740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x5653be5d8e80 con 0x5653be6bd9e0 2024-03-21T10:25:10.145 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:10.137+0000 7f17305bb740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x5653be6c08d0 con 0x5653be6b8870 2024-03-21T10:25:10.145 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:10.137+0000 7f17305bb740 10 client.0 _ll_register_callbacks cb 0x5653be5eb130 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T10:25:10.145 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:10.137+0000 7f1726a00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5653be6b7e80 0x5653be6c1ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:25:10.145 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:10.137+0000 7f1726a00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5653be6b7e80 0x5653be6c1ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:56548/0 (socket says 172.21.15.47:56548) 2024-03-21T10:25:10.145 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:10.137+0000 7f1726a00640 1 -- 172.21.15.47:0/2069944182 learned_addr learned my addr 172.21.15.47:0/2069944182 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:25:10.145 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:10.137+0000 7f1727400640 1 --2- 172.21.15.47:0/2069944182 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5653be6bd9e0 0x5653be6c9040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:25:10.146 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:10.137+0000 7f1726000640 1 --2- 172.21.15.47:0/2069944182 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5653be6b8870 0x5653be6c2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:25:10.146 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:10.138+0000 7f1726a00640 1 -- 172.21.15.47:0/2069944182 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5653be6b8870 msgr2=0x5653be6c2000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:25:10.146 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:10.138+0000 7f1726a00640 1 --2- 172.21.15.47:0/2069944182 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5653be6b8870 0x5653be6c2000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:25:10.146 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:10.138+0000 7f1726a00640 1 -- 172.21.15.47:0/2069944182 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5653be6bd9e0 msgr2=0x5653be6c9040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:25:10.146 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:10.138+0000 7f1726a00640 1 --2- 172.21.15.47:0/2069944182 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5653be6bd9e0 0x5653be6c9040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:25:10.146 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:10.138+0000 7f1726a00640 1 -- 172.21.15.47:0/2069944182 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5653be3fb7d0 con 0x5653be6b7e80 2024-03-21T10:25:10.146 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:10.138+0000 7f1726a00640 1 --2- 172.21.15.47:0/2069944182 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5653be6b7e80 0x5653be6c1ac0 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f171800bf40 tx=0x7f1718011870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=456e49078141407c server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:25:10.146 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:10.138+0000 7f1710c00640 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-03-21T10:25:10.146 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:10.139+0000 7f1710c00640 1 -- 172.21.15.47:0/2069944182 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1718004440 con 0x5653be6b7e80 2024-03-21T10:25:10.146 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:10.139+0000 7f1710c00640 1 -- 172.21.15.47:0/2069944182 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9216+0+0 (secure 0 0 0) 0x7f17180035f0 con 0x5653be6b7e80 2024-03-21T10:25:10.146 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:10.139+0000 7f17305bb740 1 -- 172.21.15.47:0/2069944182 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f17200be3e0 con 0x5653be6b7e80 2024-03-21T10:25:10.147 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:10.139+0000 7f1710c00640 1 -- 172.21.15.47:0/2069944182 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1718024050 con 0x5653be6b7e80 2024-03-21T10:25:10.147 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:10.139+0000 7f17305bb740 10 client.15666 fetch_fsmap learned FSMap version 30 2024-03-21T10:25:10.147 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:10.139+0000 7f17305bb740 1 -- 172.21.15.47:0/2069944182 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x5653be3313f0 con 0x5653be6b7e80 2024-03-21T10:25:10.147 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:10.139+0000 7f1710c00640 1 -- 172.21.15.47:0/2069944182 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f17200be3e0 con 0x5653be6b7e80 2024-03-21T10:25:10.147 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:10.140+0000 7f1710c00640 1 -- 172.21.15.47:0/2069944182 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 30) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f171801ba00 con 0x5653be6b7e80 2024-03-21T10:25:10.147 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:10.140+0000 7f17305bb740 10 client.15666 fetch_fsmap finished waiting for FSMap version 30 2024-03-21T10:25:10.147 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:10.140+0000 7f17305bb740 10 client.15666 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-21T10:25:10.147 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:10.140+0000 7f17305bb740 20 client.15666 populate_metadata read hostname 'smithi047' 2024-03-21T10:25:10.147 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:10.140+0000 7f17305bb740 1 --2- 172.21.15.47:0/2069944182 >> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] conn(0x5653be6cd960 0x5653be6cfde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:25:10.147 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:10.140+0000 7f17305bb740 4 client.15666 mds_command: new command op to 34535 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T10:25:10.147 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:10.140+0000 7f17305bb740 1 -- 172.21.15.47:0/2069944182 --> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5653be62a790 con 0x5653be6cd960 2024-03-21T10:25:10.148 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:10.140+0000 7f1726000640 1 --2- 172.21.15.47:0/2069944182 >> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] conn(0x5653be6cd960 0x5653be6cfde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:25:10.148 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:10.141+0000 7f1726000640 1 --2- 172.21.15.47:0/2069944182 >> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] conn(0x5653be6cd960 0x5653be6cfde0 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2bb4292ad325978f server_cookie=d3e7c34159ec5f49 in_seq=0 out_seq=0 2024-03-21T10:25:10.148 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:10.141+0000 7f1710c00640 10 client.15666 ms_handle_connect on v2:172.21.15.100:6838/4173168440 2024-03-21T10:25:10.148 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:10.142+0000 7f1710c00640 1 -- 172.21.15.47:0/2069944182 <== mds.0 v2:172.21.15.100:6838/4173168440 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x5653be62a790 con 0x5653be6cd960 2024-03-21T10:25:10.148 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:10.142+0000 7f1710c00640 10 client.15666 handle_command_reply: tid=0 2024-03-21T10:25:10.148 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:10.142+0000 7f17305bb740 1 client.15666 shutdown 2024-03-21T10:25:10.148 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:10.142+0000 7f17305bb740 1 -- 172.21.15.47:0/2069944182 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5653be6b7e80 msgr2=0x5653be6c1ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:25:10.148 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:10.142+0000 7f17305bb740 1 --2- 172.21.15.47:0/2069944182 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5653be6b7e80 0x5653be6c1ac0 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f171800bf40 tx=0x7f1718011870 comp rx=0 tx=0).stop 2024-03-21T10:25:10.148 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:10.142+0000 7f17305bb740 1 -- 172.21.15.47:0/2069944182 shutdown_connections 2024-03-21T10:25:10.148 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:10.142+0000 7f17305bb740 1 --2- 172.21.15.47:0/2069944182 >> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] conn(0x5653be6cd960 0x5653be6cfde0 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:25:10.148 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:10.142+0000 7f17305bb740 1 --2- 172.21.15.47:0/2069944182 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5653be6bd9e0 0x5653be6c9040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:25:10.148 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:10.142+0000 7f17305bb740 1 --2- 172.21.15.47:0/2069944182 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5653be6b8870 0x5653be6c2000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:25:10.148 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:10.142+0000 7f17305bb740 1 --2- 172.21.15.47:0/2069944182 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5653be6b7e80 0x5653be6c1ac0 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:25:10.148 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:10.142+0000 7f1710c00640 0 client.15666 ms_handle_reset on v2:172.21.15.100:6838/4173168440 2024-03-21T10:25:10.149 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:10.142+0000 7f17305bb740 1 -- 172.21.15.47:0/2069944182 >> 172.21.15.47:0/2069944182 conn(0x5653be680200 msgr2=0x5653be6b2a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:25:10.149 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:10.142+0000 7f17305bb740 1 -- 172.21.15.47:0/2069944182 shutdown_connections 2024-03-21T10:25:10.149 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:10.142+0000 7f17305bb740 1 -- 172.21.15.47:0/2069944182 wait complete. 2024-03-21T10:25:10.149 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:10.143+0000 7f17305bb740 20 client.15666 trim_cache size 0 max 16384 2024-03-21T10:25:10.149 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:10.146+0000 7f8686800640 1 -- 172.21.15.47:0/1749301017 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "snap_schedule"} v 0) v1 -- 0x7f86800622a0 con 0x7f86801054b0 2024-03-21T10:25:10.164 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:10.160+0000 7f17305bb740 1 Processor -- start 2024-03-21T10:25:10.164 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:10.160+0000 7f17305bb740 1 -- start start 2024-03-21T10:25:10.164 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:10.160+0000 7f17305bb740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5653be6b7e80 0x5653be6b2910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:25:10.164 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:10.160+0000 7f17305bb740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5653be6b8870 0x5653be6c8f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:25:10.164 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:10.160+0000 7f17305bb740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5653be6bd9e0 0x5653be6d46e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:25:10.164 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:10.160+0000 7f17305bb740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17281ac280 con 0x5653be6b8870 2024-03-21T10:25:10.164 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:10.160+0000 7f17305bb740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17281adc00 con 0x5653be6bd9e0 2024-03-21T10:25:10.164 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:10.160+0000 7f17305bb740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17281ac8e0 con 0x5653be6b7e80 2024-03-21T10:25:10.165 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:10.161+0000 7f1726000640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5653be6b8870 0x5653be6c8f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:25:10.165 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:10.161+0000 7f1726000640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5653be6b8870 0x5653be6c8f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:56558/0 (socket says 172.21.15.47:56558) 2024-03-21T10:25:10.165 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:10.161+0000 7f1726000640 1 -- 172.21.15.47:0/1044674727 learned_addr learned my addr 172.21.15.47:0/1044674727 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:25:10.165 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:10.161+0000 7f1727400640 1 --2- 172.21.15.47:0/1044674727 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5653be6bd9e0 0x5653be6d46e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:25:10.165 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:10.161+0000 7f1726a00640 1 --2- 172.21.15.47:0/1044674727 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5653be6b7e80 0x5653be6b2910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:25:10.165 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:10.161+0000 7f1726000640 1 -- 172.21.15.47:0/1044674727 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5653be6b7e80 msgr2=0x5653be6b2910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:25:10.165 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:10.161+0000 7f1726000640 1 --2- 172.21.15.47:0/1044674727 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5653be6b7e80 0x5653be6b2910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:25:10.165 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:10.161+0000 7f1726000640 1 -- 172.21.15.47:0/1044674727 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5653be6bd9e0 msgr2=0x5653be6d46e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:25:10.165 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:10.161+0000 7f1726000640 1 --2- 172.21.15.47:0/1044674727 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5653be6bd9e0 0x5653be6d46e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:25:10.165 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:10.161+0000 7f1726000640 1 -- 172.21.15.47:0/1044674727 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5653be2f10f0 con 0x5653be6b8870 2024-03-21T10:25:10.165 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:10.161+0000 7f1727400640 1 --2- 172.21.15.47:0/1044674727 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5653be6bd9e0 0x5653be6d46e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T10:25:10.165 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:10.162+0000 7f1726000640 1 --2- 172.21.15.47:0/1044674727 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5653be6b8870 0x5653be6c8f60 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f1714005bc0 tx=0x7f1714015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a0696281cbe16faa server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:25:10.165 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:10.162+0000 7f1713400640 1 -- 172.21.15.47:0/1044674727 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f171402cc20 con 0x5653be6b8870 2024-03-21T10:25:10.165 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:10.162+0000 7f1713400640 1 -- 172.21.15.47:0/1044674727 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f1714032070 con 0x5653be6b8870 2024-03-21T10:25:10.166 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:10.162+0000 7f1713400640 1 -- 172.21.15.47:0/1044674727 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f171401f430 con 0x5653be6b8870 2024-03-21T10:25:10.166 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:10.163+0000 7f17305bb740 1 -- 172.21.15.47:0/1044674727 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5653be6b8870 msgr2=0x5653be6c8f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:25:10.166 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:10.163+0000 7f17305bb740 1 --2- 172.21.15.47:0/1044674727 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5653be6b8870 0x5653be6c8f60 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f1714005bc0 tx=0x7f1714015000 comp rx=0 tx=0).stop 2024-03-21T10:25:10.166 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:10.163+0000 7f17305bb740 1 -- 172.21.15.47:0/1044674727 shutdown_connections 2024-03-21T10:25:10.166 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:10.163+0000 7f17305bb740 1 --2- 172.21.15.47:0/1044674727 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5653be6bd9e0 0x5653be6d46e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:25:10.166 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:10.163+0000 7f17305bb740 1 --2- 172.21.15.47:0/1044674727 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5653be6b8870 0x5653be6c8f60 unknown :-1 s=CLOSED pgs=501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:25:10.166 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:10.163+0000 7f17305bb740 1 --2- 172.21.15.47:0/1044674727 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5653be6b7e80 0x5653be6b2910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:25:10.167 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:10.163+0000 7f17305bb740 1 -- 172.21.15.47:0/1044674727 >> 172.21.15.47:0/1044674727 conn(0x5653be680200 msgr2=0x5653be6b2200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:25:10.167 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:10.163+0000 7f17305bb740 1 -- 172.21.15.47:0/1044674727 shutdown_connections 2024-03-21T10:25:10.167 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:10.163+0000 7f17305bb740 1 -- 172.21.15.47:0/1044674727 wait complete. 2024-03-21T10:25:10.167 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:10.163+0000 7f17305bb740 1 -- start start 2024-03-21T10:25:10.167 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:10.163+0000 7f17305bb740 1 Processor -- start 2024-03-21T10:25:10.170 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:10.164+0000 7f17305bb740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5653be6b7e80 0x5653be6c6260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:25:10.170 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:10.164+0000 7f17305bb740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5653be6b8870 0x5653be6c67a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:25:10.170 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:10.164+0000 7f17305bb740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5653be6bd9e0 0x5653be6c77b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:25:10.170 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:10.164+0000 7f17305bb740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17281ad5a0 con 0x5653be6bd9e0 2024-03-21T10:25:10.170 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:10.164+0000 7f17305bb740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17281acf40 con 0x5653be6b7e80 2024-03-21T10:25:10.170 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:10.164+0000 7f17305bb740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17281aa420 con 0x5653be6b8870 2024-03-21T10:25:10.171 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:10.164+0000 7f17305bb740 10 client.0 _ll_register_callbacks cb 0x7f171800bad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T10:25:10.171 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:10.164+0000 7f1727400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5653be6bd9e0 0x5653be6c77b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:25:10.171 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:10.164+0000 7f1726a00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5653be6b7e80 0x5653be6c6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:25:10.171 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:10.164+0000 7f1727400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5653be6bd9e0 0x5653be6c77b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:56560/0 (socket says 172.21.15.47:56560) 2024-03-21T10:25:10.171 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:10.164+0000 7f1727400640 1 -- 172.21.15.47:0/3949997127 learned_addr learned my addr 172.21.15.47:0/3949997127 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:25:10.171 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:10.165+0000 7f1726000640 1 --2- 172.21.15.47:0/3949997127 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5653be6b8870 0x5653be6c67a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:25:10.171 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:10.165+0000 7f1727400640 1 -- 172.21.15.47:0/3949997127 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5653be6b8870 msgr2=0x5653be6c67a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:25:10.171 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:10.165+0000 7f1727400640 1 --2- 172.21.15.47:0/3949997127 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5653be6b8870 0x5653be6c67a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:25:10.171 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:10.165+0000 7f1727400640 1 -- 172.21.15.47:0/3949997127 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5653be6b7e80 msgr2=0x5653be6c6260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:25:10.171 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:10.165+0000 7f1727400640 1 --2- 172.21.15.47:0/3949997127 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5653be6b7e80 0x5653be6c6260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:25:10.171 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:10.165+0000 7f1727400640 1 -- 172.21.15.47:0/3949997127 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5653be6c7da0 con 0x5653be6bd9e0 2024-03-21T10:25:10.172 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:10.165+0000 7f1727400640 1 --2- 172.21.15.47:0/3949997127 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5653be6bd9e0 0x5653be6c77b0 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f1720005fa0 tx=0x7f17200099f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e38a05d266e3e03c server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:25:10.172 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:10.165+0000 7f1710c00640 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-03-21T10:25:10.172 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:10.165+0000 7f1710c00640 1 -- 172.21.15.47:0/3949997127 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17200be140 con 0x5653be6bd9e0 2024-03-21T10:25:10.172 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:10.165+0000 7f1710c00640 1 -- 172.21.15.47:0/3949997127 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9216+0+0 (secure 0 0 0) 0x7f17200c6260 con 0x5653be6bd9e0 2024-03-21T10:25:10.172 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:10.165+0000 7f17305bb740 1 -- 172.21.15.47:0/3949997127 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f1714032070 con 0x5653be6bd9e0 2024-03-21T10:25:10.172 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:10.165+0000 7f1710c00640 1 -- 172.21.15.47:0/3949997127 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f17200c1e30 con 0x5653be6bd9e0 2024-03-21T10:25:10.172 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:10.166+0000 7f17305bb740 10 client.15672 fetch_fsmap learned FSMap version 30 2024-03-21T10:25:10.172 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:10.166+0000 7f17305bb740 1 -- 172.21.15.47:0/3949997127 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x5653be6c9120 con 0x5653be6bd9e0 2024-03-21T10:25:10.173 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:10.166+0000 7f1710c00640 1 -- 172.21.15.47:0/3949997127 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f1714032070 con 0x5653be6bd9e0 2024-03-21T10:25:10.173 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:10.166+0000 7f1710c00640 1 -- 172.21.15.47:0/3949997127 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 30) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f172003c030 con 0x5653be6bd9e0 2024-03-21T10:25:10.173 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:10.166+0000 7f17305bb740 10 client.15672 fetch_fsmap finished waiting for FSMap version 30 2024-03-21T10:25:10.173 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:10.166+0000 7f17305bb740 10 client.15672 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-21T10:25:10.173 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:10.166+0000 7f17305bb740 20 client.15672 populate_metadata read hostname 'smithi047' 2024-03-21T10:25:10.173 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:10.166+0000 7f17305bb740 1 --2- 172.21.15.47:0/3949997127 >> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] conn(0x5653be729620 0x5653be72ba40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:25:10.173 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:10.166+0000 7f17305bb740 4 client.15672 mds_command: new command op to 34535 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-21T10:25:10.173 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:10.166+0000 7f17305bb740 1 -- 172.21.15.47:0/3949997127 --> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x5653be624ae0 con 0x5653be729620 2024-03-21T10:25:10.173 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:10.167+0000 7f1726a00640 1 --2- 172.21.15.47:0/3949997127 >> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] conn(0x5653be729620 0x5653be72ba40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:25:10.173 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:10.167+0000 7f1726a00640 1 --2- 172.21.15.47:0/3949997127 >> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] conn(0x5653be729620 0x5653be72ba40 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f99fa2b2586a3149 server_cookie=3e2b4ab69fc000f8 in_seq=0 out_seq=0 2024-03-21T10:25:10.173 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:10.167+0000 7f1710c00640 10 client.15672 ms_handle_connect on v2:172.21.15.100:6838/4173168440 2024-03-21T10:25:10.173 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:10.168+0000 7f1710c00640 1 -- 172.21.15.47:0/3949997127 <== mds.0 v2:172.21.15.100:6838/4173168440 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x5653be624ae0 con 0x5653be729620 2024-03-21T10:25:10.173 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:10.168+0000 7f1710c00640 10 client.15672 handle_command_reply: tid=0 2024-03-21T10:25:10.173 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:10.168+0000 7f17305bb740 1 client.15672 shutdown 2024-03-21T10:25:10.173 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:10.168+0000 7f17305bb740 1 -- 172.21.15.47:0/3949997127 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5653be6bd9e0 msgr2=0x5653be6c77b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:25:10.174 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:10.168+0000 7f17305bb740 1 --2- 172.21.15.47:0/3949997127 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5653be6bd9e0 0x5653be6c77b0 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f1720005fa0 tx=0x7f17200099f0 comp rx=0 tx=0).stop 2024-03-21T10:25:10.174 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:10.168+0000 7f17305bb740 1 -- 172.21.15.47:0/3949997127 shutdown_connections 2024-03-21T10:25:10.174 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:10.168+0000 7f17305bb740 1 --2- 172.21.15.47:0/3949997127 >> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] conn(0x5653be729620 0x5653be72ba40 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:25:10.174 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:10.168+0000 7f17305bb740 1 --2- 172.21.15.47:0/3949997127 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5653be6bd9e0 0x5653be6c77b0 unknown :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:25:10.174 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:10.168+0000 7f17305bb740 1 --2- 172.21.15.47:0/3949997127 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5653be6b8870 0x5653be6c67a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:25:10.174 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:10.168+0000 7f17305bb740 1 --2- 172.21.15.47:0/3949997127 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5653be6b7e80 0x5653be6c6260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:25:10.174 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:10.168+0000 7f1710c00640 0 client.15672 ms_handle_reset on v2:172.21.15.100:6838/4173168440 2024-03-21T10:25:10.174 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:10.168+0000 7f17305bb740 1 -- 172.21.15.47:0/3949997127 >> 172.21.15.47:0/3949997127 conn(0x5653be680200 msgr2=0x5653be6b2a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:25:10.174 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:10.168+0000 7f17305bb740 1 -- 172.21.15.47:0/3949997127 shutdown_connections 2024-03-21T10:25:10.174 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:10.168+0000 7f17305bb740 1 -- 172.21.15.47:0/3949997127 wait complete. 2024-03-21T10:25:10.175 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:10.168+0000 7f17305bb740 20 client.15672 trim_cache size 0 max 16384 2024-03-21T10:25:10.175 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-03-21T10:25:10.175 INFO:teuthology.orchestra.run.smithi047.stdout: "status": "IDLE (waiting for more scrubs)", 2024-03-21T10:25:10.175 INFO:teuthology.orchestra.run.smithi047.stdout: "scrubs": { 2024-03-21T10:25:10.175 INFO:teuthology.orchestra.run.smithi047.stdout: "64186214-1d58-46f0-8165-8cd56e92c6b1": { 2024-03-21T10:25:10.175 INFO:teuthology.orchestra.run.smithi047.stdout: "path": "/", 2024-03-21T10:25:10.175 INFO:teuthology.orchestra.run.smithi047.stdout: "tag": "64186214-1d58-46f0-8165-8cd56e92c6b1", 2024-03-21T10:25:10.175 INFO:teuthology.orchestra.run.smithi047.stdout: "options": "recursive,force" 2024-03-21T10:25:10.175 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-21T10:25:10.175 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-21T10:25:10.176 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-03-21T10:25:10.176 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:10.169+0000 7f172d800640 1 -- 172.21.15.47:0/3609720064 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f170c0a2f20 msgr2=0x7f170c0a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:25:10.176 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:10.169+0000 7f172d800640 1 --2- 172.21.15.47:0/3609720064 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f170c0a2f20 0x7f170c0a53e0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f1728101810 tx=0x7f17140108d0 comp rx=0 tx=0).stop 2024-03-21T10:25:10.176 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:10.169+0000 7f172d800640 1 -- 172.21.15.47:0/3609720064 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1728107390 msgr2=0x7f1728100620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:25:10.177 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:10.169+0000 7f172d800640 1 --2- 172.21.15.47:0/3609720064 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1728107390 0x7f1728100620 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f1720037040 tx=0x7f1720034880 comp rx=0 tx=0).stop 2024-03-21T10:25:10.177 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:10.169+0000 7f172d800640 1 -- 172.21.15.47:0/3609720064 shutdown_connections 2024-03-21T10:25:10.177 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:10.169+0000 7f172d800640 1 --2- 172.21.15.47:0/3609720064 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f170c0a2f20 0x7f170c0a53e0 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:25:10.177 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:10.169+0000 7f172d800640 1 --2- 172.21.15.47:0/3609720064 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1728107390 0x7f1728100620 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:25:10.177 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:10.169+0000 7f172d800640 1 --2- 172.21.15.47:0/3609720064 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1728104a60 0x7f17281000e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:25:10.177 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:10.169+0000 7f172d800640 1 --2- 172.21.15.47:0/3609720064 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1728102130 0x7f1728101aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:25:10.177 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:10.169+0000 7f172d800640 1 -- 172.21.15.47:0/3609720064 >> 172.21.15.47:0/3609720064 conn(0x7f17280fbb40 msgr2=0x7f1728102a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:25:10.177 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:10.169+0000 7f172d800640 1 -- 172.21.15.47:0/3609720064 shutdown_connections 2024-03-21T10:25:10.177 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:10.169+0000 7f172d800640 1 -- 172.21.15.47:0/3609720064 wait complete. 2024-03-21T10:25:10.187 INFO:tasks.cephfs.filesystem:scrub status for tag:64186214-1d58-46f0-8165-8cd56e92c6b1 - {'path': '/', 'tag': '64186214-1d58-46f0-8165-8cd56e92c6b1', 'options': 'recursive,force'} 2024-03-21T10:25:10.406 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:10.405+0000 7f8677e00640 1 -- 172.21.15.47:0/1749301017 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "snap_schedule"}]=0 v19) v1 ==== 92+0+0 (secure 0 0 0) 0x7f86780be370 con 0x7f86801054b0 2024-03-21T10:25:10.410 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:10.408+0000 7f8686800640 1 -- 172.21.15.47:0/1749301017 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f86640a3040 msgr2=0x7f86640a5500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:25:10.410 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:10.408+0000 7f8686800640 1 --2- 172.21.15.47:0/1749301017 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f86640a3040 0x7f86640a5500 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f8670005fd0 tx=0x7f867000b390 comp rx=0 tx=0).stop 2024-03-21T10:25:10.410 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:10.408+0000 7f8686800640 1 -- 172.21.15.47:0/1749301017 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f86801054b0 msgr2=0x7f86801a0f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:25:10.410 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:10.408+0000 7f8686800640 1 --2- 172.21.15.47:0/1749301017 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f86801054b0 0x7f86801a0f90 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f8678032380 tx=0x7f8678032820 comp rx=0 tx=0).stop 2024-03-21T10:25:10.410 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:10.409+0000 7f8686800640 1 -- 172.21.15.47:0/1749301017 shutdown_connections 2024-03-21T10:25:10.410 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:10.409+0000 7f8686800640 1 --2- 172.21.15.47:0/1749301017 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f86640a3040 0x7f86640a5500 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:25:10.410 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:10.409+0000 7f8686800640 1 --2- 172.21.15.47:0/1749301017 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f86801054b0 0x7f86801a0f90 unknown :-1 s=CLOSED pgs=496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:25:10.410 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:10.409+0000 7f8686800640 1 --2- 172.21.15.47:0/1749301017 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8680104af0 0x7f8680199f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:25:10.410 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:10.409+0000 7f8686800640 1 --2- 172.21.15.47:0/1749301017 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8680103900 0x7f86801999e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:25:10.410 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:10.409+0000 7f8686800640 1 -- 172.21.15.47:0/1749301017 >> 172.21.15.47:0/1749301017 conn(0x7f86800fee90 msgr2=0x7f8680107df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:25:10.411 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:10.409+0000 7f8686800640 1 -- 172.21.15.47:0/1749301017 shutdown_connections 2024-03-21T10:25:10.411 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:10.409+0000 7f8686800640 1 -- 172.21.15.47:0/1749301017 wait complete. 2024-03-21T10:25:10.435 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:25:10 smithi047 ceph-mon[31547]: from='client.? 172.21.15.47:0/1966066287' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-21T10:25:10.436 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:25:10 smithi047 ceph-mon[31547]: from='client.? 172.21.15.47:0/1749301017' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "snap_schedule"}]: dispatch 2024-03-21T10:25:10.439 DEBUG:teuthology.orchestra.run.smithi047:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph config set mgr mgr/snap_schedule/allow_m_granularity true' 2024-03-21T10:25:10.641 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:10.640+0000 7f779a800640 1 -- 172.21.15.47:0/3796731273 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7794074100 msgr2=0x7f7794074580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:25:10.641 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:10.640+0000 7f779a800640 1 --2- 172.21.15.47:0/3796731273 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7794074100 0x7f7794074580 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f7780009900 tx=0x7f778002bc30 comp rx=0 tx=0).stop 2024-03-21T10:25:10.641 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:10.640+0000 7f779a800640 1 -- 172.21.15.47:0/3796731273 shutdown_connections 2024-03-21T10:25:10.641 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:10.640+0000 7f779a800640 1 --2- 172.21.15.47:0/3796731273 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f77941093d0 0x7f779410b860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:25:10.641 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:10.640+0000 7f779a800640 1 --2- 172.21.15.47:0/3796731273 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7794074100 0x7f7794074580 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:25:10.641 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:10.640+0000 7f779a800640 1 --2- 172.21.15.47:0/3796731273 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f77940756e0 0x7f7794073b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:25:10.641 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:10.640+0000 7f779a800640 1 -- 172.21.15.47:0/3796731273 >> 172.21.15.47:0/3796731273 conn(0x7f77940fca80 msgr2=0x7f77940feec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:25:10.642 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:10.640+0000 7f779a800640 1 -- 172.21.15.47:0/3796731273 shutdown_connections 2024-03-21T10:25:10.642 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:10.640+0000 7f779a800640 1 -- 172.21.15.47:0/3796731273 wait complete. 2024-03-21T10:25:10.644 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:10.641+0000 7f779a800640 1 Processor -- start 2024-03-21T10:25:10.644 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:10.641+0000 7f779a800640 1 -- start start 2024-03-21T10:25:10.644 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:10.641+0000 7f779a800640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7794074100 0x7f7794199930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:25:10.644 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:10.642+0000 7f779a800640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f77940756e0 0x7f7794199e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:25:10.644 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:10.642+0000 7f779a800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f77941093d0 0x7f77941a0ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:25:10.644 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:10.642+0000 7f779a800640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f779410f270 con 0x7f7794074100 2024-03-21T10:25:10.644 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:10.642+0000 7f779a800640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f779410f0f0 con 0x7f77941093d0 2024-03-21T10:25:10.644 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:10.642+0000 7f779a800640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f779410f3f0 con 0x7f77940756e0 2024-03-21T10:25:10.644 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:10.642+0000 7f7799400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7794074100 0x7f7794199930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:25:10.644 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:10.642+0000 7f7799400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7794074100 0x7f7794199930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:56608/0 (socket says 172.21.15.47:56608) 2024-03-21T10:25:10.645 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:10.642+0000 7f7799400640 1 -- 172.21.15.47:0/1098362985 learned_addr learned my addr 172.21.15.47:0/1098362985 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:25:10.645 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:10.642+0000 7f7799e00640 1 --2- 172.21.15.47:0/1098362985 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f77941093d0 0x7f77941a0ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:25:10.645 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:10.642+0000 7f7798a00640 1 --2- 172.21.15.47:0/1098362985 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f77940756e0 0x7f7794199e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:25:10.645 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:10.642+0000 7f7799400640 1 -- 172.21.15.47:0/1098362985 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f77940756e0 msgr2=0x7f7794199e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:25:10.645 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:10.642+0000 7f7799400640 1 --2- 172.21.15.47:0/1098362985 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f77940756e0 0x7f7794199e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:25:10.645 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:10.642+0000 7f7799400640 1 -- 172.21.15.47:0/1098362985 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f77941093d0 msgr2=0x7f77941a0ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:25:10.645 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:10.642+0000 7f7799400640 1 --2- 172.21.15.47:0/1098362985 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f77941093d0 0x7f77941a0ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:25:10.645 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:10.642+0000 7f7799400640 1 -- 172.21.15.47:0/1098362985 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77941a1550 con 0x7f7794074100 2024-03-21T10:25:10.645 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:10.643+0000 7f7799400640 1 --2- 172.21.15.47:0/1098362985 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7794074100 0x7f7794199930 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f778800e9f0 tx=0x7f778800edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:25:10.645 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:10.643+0000 7f7792000640 1 -- 172.21.15.47:0/1098362985 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f778800ccc0 con 0x7f7794074100 2024-03-21T10:25:10.646 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:10.643+0000 7f779a800640 1 -- 172.21.15.47:0/1098362985 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f77941a1840 con 0x7f7794074100 2024-03-21T10:25:10.646 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:10.643+0000 7f7792000640 1 -- 172.21.15.47:0/1098362985 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f778800ce60 con 0x7f7794074100 2024-03-21T10:25:10.646 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:10.643+0000 7f7792000640 1 -- 172.21.15.47:0/1098362985 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7788010430 con 0x7f7794074100 2024-03-21T10:25:10.646 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:10.643+0000 7f779a800640 1 -- 172.21.15.47:0/1098362985 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f77941a1d10 con 0x7f7794074100 2024-03-21T10:25:10.646 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:10.645+0000 7f779a800640 1 -- 172.21.15.47:0/1098362985 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7794108100 con 0x7f7794074100 2024-03-21T10:25:10.647 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:10.647+0000 7f7792000640 1 -- 172.21.15.47:0/1098362985 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 19) v1 ==== 137871+0+0 (secure 0 0 0) 0x7f7788005000 con 0x7f7794074100 2024-03-21T10:25:10.649 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:10.648+0000 7f7792000640 1 --2- 172.21.15.47:0/1098362985 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f777c0a2f60 0x7f777c0a5420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:25:10.649 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:10.648+0000 7f7798a00640 1 -- 172.21.15.47:0/1098362985 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f777c0a2f60 msgr2=0x7f777c0a5420 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.47:6800/2823885994 2024-03-21T10:25:10.649 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:10.648+0000 7f7792000640 1 -- 172.21.15.47:0/1098362985 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9216+0+0 (secure 0 0 0) 0x7f77880cfee0 con 0x7f7794074100 2024-03-21T10:25:10.649 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:10.648+0000 7f7798a00640 1 --2- 172.21.15.47:0/1098362985 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f777c0a2f60 0x7f777c0a5420 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-21T10:25:10.652 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:10.651+0000 7f7792000640 1 -- 172.21.15.47:0/1098362985 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f7788053740 con 0x7f7794074100 2024-03-21T10:25:10.713 INFO:journalctl@ceph.mgr.x.smithi047.stdout:Mar 21 10:25:10 smithi047 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-mgr-x[31773]: ignoring --setuser ceph since I am not root 2024-03-21T10:25:10.714 INFO:journalctl@ceph.mgr.x.smithi047.stdout:Mar 21 10:25:10 smithi047 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-mgr-x[31773]: ignoring --setgroup ceph since I am not root 2024-03-21T10:25:10.729 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:25:10 smithi100 ceph-mon[35193]: from='client.? 172.21.15.47:0/1966066287' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-21T10:25:10.729 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:25:10 smithi100 ceph-mon[35193]: from='client.? 172.21.15.47:0/1749301017' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "snap_schedule"}]: dispatch 2024-03-21T10:25:10.729 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Mar 21 10:25:10 smithi100 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-mgr-y[36578]: ignoring --setuser ceph since I am not root 2024-03-21T10:25:10.730 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Mar 21 10:25:10 smithi100 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-mgr-y[36578]: ignoring --setgroup ceph since I am not root 2024-03-21T10:25:10.730 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Mar 21 10:25:10 smithi100 ceph-mgr[36582]: -- 172.21.15.100:0/3906006842 <== mon.1 v2:172.21.15.157:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55bdaff865a0 con 0x55bdaff5a800 2024-03-21T10:25:10.739 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:25:10 smithi157 ceph-mon[35581]: from='client.? 172.21.15.47:0/1966066287' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-21T10:25:10.740 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:25:10 smithi157 ceph-mon[35581]: from='client.? 172.21.15.47:0/1749301017' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "snap_schedule"}]: dispatch 2024-03-21T10:25:10.740 INFO:journalctl@ceph.mgr.z.smithi157.stdout:Mar 21 10:25:10 smithi157 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-mgr-z[37318]: ignoring --setuser ceph since I am not root 2024-03-21T10:25:10.740 INFO:journalctl@ceph.mgr.z.smithi157.stdout:Mar 21 10:25:10 smithi157 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-mgr-z[37318]: ignoring --setgroup ceph since I am not root 2024-03-21T10:25:10.740 INFO:journalctl@ceph.mgr.z.smithi157.stdout:Mar 21 10:25:10 smithi157 ceph-mgr[37322]: -- 172.21.15.157:0/3255067031 <== mon.0 v2:172.21.15.47:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55c1827285a0 con 0x55c1826fc800 2024-03-21T10:25:10.850 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:10.849+0000 7f7798a00640 1 -- 172.21.15.47:0/1098362985 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f777c0a2f60 msgr2=0x7f777c0a5420 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.47:6800/2823885994 2024-03-21T10:25:10.850 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:10.849+0000 7f7798a00640 1 --2- 172.21.15.47:0/1098362985 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f777c0a2f60 0x7f777c0a5420 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-21T10:25:10.858 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:10.857+0000 7f779a800640 1 -- 172.21.15.47:0/1098362985 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}] v 0) v1 -- 0x7f779402d070 con 0x7f7794074100 2024-03-21T10:25:10.867 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:10.866+0000 7f7792000640 1 -- 172.21.15.47:0/1098362985 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}]=0 v31)=0 v31) v1 ==== 138+0+0 (secure 0 0 0) 0x7f778809c900 con 0x7f7794074100 2024-03-21T10:25:10.870 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:10.869+0000 7f779a800640 1 -- 172.21.15.47:0/1098362985 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f777c0a2f60 msgr2=0x7f777c0a5420 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-21T10:25:10.870 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:10.869+0000 7f779a800640 1 --2- 172.21.15.47:0/1098362985 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f777c0a2f60 0x7f777c0a5420 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:25:10.870 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:10.869+0000 7f779a800640 1 -- 172.21.15.47:0/1098362985 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7794074100 msgr2=0x7f7794199930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:25:10.871 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:10.869+0000 7f779a800640 1 --2- 172.21.15.47:0/1098362985 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7794074100 0x7f7794199930 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f778800e9f0 tx=0x7f778800edf0 comp rx=0 tx=0).stop 2024-03-21T10:25:10.871 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:10.869+0000 7f779a800640 1 -- 172.21.15.47:0/1098362985 shutdown_connections 2024-03-21T10:25:10.871 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:10.869+0000 7f779a800640 1 --2- 172.21.15.47:0/1098362985 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f777c0a2f60 0x7f777c0a5420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:25:10.871 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:10.869+0000 7f779a800640 1 --2- 172.21.15.47:0/1098362985 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f77941093d0 0x7f77941a0ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:25:10.871 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:10.869+0000 7f779a800640 1 --2- 172.21.15.47:0/1098362985 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f77940756e0 0x7f7794199e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:25:10.871 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:10.869+0000 7f779a800640 1 --2- 172.21.15.47:0/1098362985 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7794074100 0x7f7794199930 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:25:10.871 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:10.869+0000 7f779a800640 1 -- 172.21.15.47:0/1098362985 >> 172.21.15.47:0/1098362985 conn(0x7f77940fca80 msgr2=0x7f77941052d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:25:10.871 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:10.869+0000 7f779a800640 1 -- 172.21.15.47:0/1098362985 shutdown_connections 2024-03-21T10:25:10.871 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:10.870+0000 7f779a800640 1 -- 172.21.15.47:0/1098362985 wait complete. 2024-03-21T10:25:10.894 DEBUG:teuthology.orchestra.run.smithi047:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph config set mgr mgr/snap_schedule/dump_on_update true' 2024-03-21T10:25:11.081 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:11.080+0000 7f4caa400640 1 -- 172.21.15.47:0/3128776123 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4ca4074080 msgr2=0x7f4ca40744e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:25:11.081 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:11.080+0000 7f4caa400640 1 --2- 172.21.15.47:0/3128776123 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4ca4074080 0x7f4ca40744e0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f4c90009900 tx=0x7f4c9002bcc0 comp rx=0 tx=0).stop 2024-03-21T10:25:11.081 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:11.080+0000 7f4caa400640 1 -- 172.21.15.47:0/3128776123 shutdown_connections 2024-03-21T10:25:11.081 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:11.080+0000 7f4caa400640 1 --2- 172.21.15.47:0/3128776123 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4ca4109580 0x7f4ca410ba10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:25:11.082 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:11.080+0000 7f4caa400640 1 --2- 172.21.15.47:0/3128776123 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4ca4074080 0x7f4ca40744e0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:25:11.082 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:11.080+0000 7f4caa400640 1 --2- 172.21.15.47:0/3128776123 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4ca4075660 0x7f4ca4073ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:25:11.082 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:11.080+0000 7f4caa400640 1 -- 172.21.15.47:0/3128776123 >> 172.21.15.47:0/3128776123 conn(0x7f4ca40fcc60 msgr2=0x7f4ca40ff080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:25:11.082 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:11.080+0000 7f4caa400640 1 -- 172.21.15.47:0/3128776123 shutdown_connections 2024-03-21T10:25:11.082 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:11.080+0000 7f4caa400640 1 -- 172.21.15.47:0/3128776123 wait complete. 2024-03-21T10:25:11.084 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:11.081+0000 7f4caa400640 1 Processor -- start 2024-03-21T10:25:11.084 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:11.082+0000 7f4caa400640 1 -- start start 2024-03-21T10:25:11.084 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:11.082+0000 7f4caa400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4ca4074080 0x7f4ca4199a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:25:11.084 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:11.082+0000 7f4caa400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4ca4075660 0x7f4ca4199f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:25:11.084 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:11.082+0000 7f4caa400640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4ca4109580 0x7f4ca41a0fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:25:11.085 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:11.082+0000 7f4caa400640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ca410f3e0 con 0x7f4ca4075660 2024-03-21T10:25:11.085 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:11.082+0000 7f4caa400640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ca410f260 con 0x7f4ca4074080 2024-03-21T10:25:11.085 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:11.082+0000 7f4caa400640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ca410f560 con 0x7f4ca4109580 2024-03-21T10:25:11.085 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:11.082+0000 7f4ca2a00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4ca4075660 0x7f4ca4199f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:25:11.085 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:11.082+0000 7f4ca2a00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4ca4075660 0x7f4ca4199f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:56622/0 (socket says 172.21.15.47:56622) 2024-03-21T10:25:11.085 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:11.082+0000 7f4ca2a00640 1 -- 172.21.15.47:0/3115353101 learned_addr learned my addr 172.21.15.47:0/3115353101 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:25:11.085 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:11.083+0000 7f4ca3e00640 1 --2- 172.21.15.47:0/3115353101 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4ca4109580 0x7f4ca41a0fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:25:11.085 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:11.083+0000 7f4ca3400640 1 --2- 172.21.15.47:0/3115353101 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4ca4074080 0x7f4ca4199a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:25:11.085 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:11.083+0000 7f4ca2a00640 1 -- 172.21.15.47:0/3115353101 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4ca4109580 msgr2=0x7f4ca41a0fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:25:11.085 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:11.083+0000 7f4ca2a00640 1 --2- 172.21.15.47:0/3115353101 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4ca4109580 0x7f4ca41a0fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:25:11.085 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:11.083+0000 7f4ca2a00640 1 -- 172.21.15.47:0/3115353101 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4ca4074080 msgr2=0x7f4ca4199a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:25:11.085 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:11.083+0000 7f4ca2a00640 1 --2- 172.21.15.47:0/3115353101 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4ca4074080 0x7f4ca4199a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:25:11.085 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:11.083+0000 7f4ca2a00640 1 -- 172.21.15.47:0/3115353101 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ca41a1650 con 0x7f4ca4075660 2024-03-21T10:25:11.086 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:11.083+0000 7f4ca3e00640 1 --2- 172.21.15.47:0/3115353101 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4ca4109580 0x7f4ca41a0fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T10:25:11.086 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:11.084+0000 7f4ca2a00640 1 --2- 172.21.15.47:0/3115353101 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4ca4075660 0x7f4ca4199f70 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f4c90006010 tx=0x7f4c90005d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:25:11.086 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:11.084+0000 7f4c9be00640 1 -- 172.21.15.47:0/3115353101 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c90043020 con 0x7f4ca4075660 2024-03-21T10:25:11.086 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:11.084+0000 7f4caa400640 1 -- 172.21.15.47:0/3115353101 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4ca41a18e0 con 0x7f4ca4075660 2024-03-21T10:25:11.086 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:11.084+0000 7f4c9be00640 1 -- 172.21.15.47:0/3115353101 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f4c9002dc10 con 0x7f4ca4075660 2024-03-21T10:25:11.087 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:11.084+0000 7f4c9be00640 1 -- 172.21.15.47:0/3115353101 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c90003a10 con 0x7f4ca4075660 2024-03-21T10:25:11.087 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:11.084+0000 7f4caa400640 1 -- 172.21.15.47:0/3115353101 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4ca41a1e30 con 0x7f4ca4075660 2024-03-21T10:25:11.087 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:11.085+0000 7f4caa400640 1 -- 172.21.15.47:0/3115353101 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4ca4108340 con 0x7f4ca4075660 2024-03-21T10:25:11.090 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:11.089+0000 7f4c9be00640 1 -- 172.21.15.47:0/3115353101 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 19) v1 ==== 137871+0+0 (secure 0 0 0) 0x7f4c90046020 con 0x7f4ca4075660 2024-03-21T10:25:11.091 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:11.090+0000 7f4c9be00640 1 --2- 172.21.15.47:0/3115353101 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f4c880a3010 0x7f4c880a54d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:25:11.091 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:11.090+0000 7f4ca3400640 1 -- 172.21.15.47:0/3115353101 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f4c880a3010 msgr2=0x7f4c880a54d0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.47:6800/2823885994 2024-03-21T10:25:11.091 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:11.090+0000 7f4c9be00640 1 -- 172.21.15.47:0/3115353101 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9216+0+0 (secure 0 0 0) 0x7f4c900f1fa0 con 0x7f4ca4075660 2024-03-21T10:25:11.091 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:11.090+0000 7f4ca3400640 1 --2- 172.21.15.47:0/3115353101 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f4c880a3010 0x7f4c880a54d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-21T10:25:11.096 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:11.095+0000 7f4c9be00640 1 -- 172.21.15.47:0/3115353101 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f4c9004d020 con 0x7f4ca4075660 2024-03-21T10:25:11.292 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:11.291+0000 7f4ca3400640 1 -- 172.21.15.47:0/3115353101 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f4c880a3010 msgr2=0x7f4c880a54d0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.47:6800/2823885994 2024-03-21T10:25:11.292 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:11.291+0000 7f4ca3400640 1 --2- 172.21.15.47:0/3115353101 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f4c880a3010 0x7f4c880a54d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-21T10:25:11.318 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:11.317+0000 7f4caa400640 1 -- 172.21.15.47:0/3115353101 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/dump_on_update}] v 0) v1 -- 0x7f4ca40622a0 con 0x7f4ca4075660 2024-03-21T10:25:11.323 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:11.322+0000 7f4c9be00640 1 -- 172.21.15.47:0/3115353101 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/dump_on_update}]=0 v32)=0 v32) v1 ==== 133+0+0 (secure 0 0 0) 0x7f4c900be9c0 con 0x7f4ca4075660 2024-03-21T10:25:11.326 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:11.325+0000 7f4caa400640 1 -- 172.21.15.47:0/3115353101 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f4c880a3010 msgr2=0x7f4c880a54d0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-21T10:25:11.326 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:11.325+0000 7f4caa400640 1 --2- 172.21.15.47:0/3115353101 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f4c880a3010 0x7f4c880a54d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:25:11.326 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:11.325+0000 7f4caa400640 1 -- 172.21.15.47:0/3115353101 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4ca4075660 msgr2=0x7f4ca4199f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:25:11.326 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:11.325+0000 7f4caa400640 1 --2- 172.21.15.47:0/3115353101 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4ca4075660 0x7f4ca4199f70 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f4c90006010 tx=0x7f4c90005d50 comp rx=0 tx=0).stop 2024-03-21T10:25:11.326 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:11.325+0000 7f4caa400640 1 -- 172.21.15.47:0/3115353101 shutdown_connections 2024-03-21T10:25:11.326 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:11.325+0000 7f4caa400640 1 --2- 172.21.15.47:0/3115353101 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f4c880a3010 0x7f4c880a54d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:25:11.326 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:11.325+0000 7f4caa400640 1 --2- 172.21.15.47:0/3115353101 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4ca4109580 0x7f4ca41a0fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:25:11.327 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:11.325+0000 7f4caa400640 1 --2- 172.21.15.47:0/3115353101 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4ca4075660 0x7f4ca4199f70 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:25:11.327 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:11.325+0000 7f4caa400640 1 --2- 172.21.15.47:0/3115353101 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4ca4074080 0x7f4ca4199a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:25:11.327 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:11.325+0000 7f4caa400640 1 -- 172.21.15.47:0/3115353101 >> 172.21.15.47:0/3115353101 conn(0x7f4ca40fcc60 msgr2=0x7f4ca4105510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:25:11.327 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:11.325+0000 7f4caa400640 1 -- 172.21.15.47:0/3115353101 shutdown_connections 2024-03-21T10:25:11.327 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:11.325+0000 7f4caa400640 1 -- 172.21.15.47:0/3115353101 wait complete. 2024-03-21T10:25:11.350 DEBUG:teuthology.orchestra.run.smithi047:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule add --fs=cephfs --path=/ --snap_schedule=1m' 2024-03-21T10:25:11.533 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:11.532+0000 7f33d1c00640 1 -- 172.21.15.47:0/4157775919 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f33cc104ae0 msgr2=0x7f33cc104f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:25:11.534 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:11.532+0000 7f33d1c00640 1 --2- 172.21.15.47:0/4157775919 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f33cc104ae0 0x7f33cc104f60 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f33b4009900 tx=0x7f33b402bc30 comp rx=0 tx=0).stop 2024-03-21T10:25:11.534 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:11.532+0000 7f33d1c00640 1 -- 172.21.15.47:0/4157775919 shutdown_connections 2024-03-21T10:25:11.534 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:11.532+0000 7f33d1c00640 1 --2- 172.21.15.47:0/4157775919 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f33cc1054a0 0x7f33cc10bd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:25:11.534 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:11.532+0000 7f33d1c00640 1 --2- 172.21.15.47:0/4157775919 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f33cc104ae0 0x7f33cc104f60 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:25:11.534 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:11.532+0000 7f33d1c00640 1 --2- 172.21.15.47:0/4157775919 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f33cc1038f0 0x7f33cc103d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:25:11.534 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:11.532+0000 7f33d1c00640 1 -- 172.21.15.47:0/4157775919 >> 172.21.15.47:0/4157775919 conn(0x7f33cc0fee60 msgr2=0x7f33cc1012a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:25:11.534 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:11.532+0000 7f33d1c00640 1 -- 172.21.15.47:0/4157775919 shutdown_connections 2024-03-21T10:25:11.534 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:11.532+0000 7f33d1c00640 1 -- 172.21.15.47:0/4157775919 wait complete. 2024-03-21T10:25:11.535 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:11.534+0000 7f33d1c00640 1 Processor -- start 2024-03-21T10:25:11.535 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:11.534+0000 7f33d1c00640 1 -- start start 2024-03-21T10:25:11.535 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:11.534+0000 7f33d1c00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f33cc1038f0 0x7f33cc199a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:25:11.535 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:11.534+0000 7f33d1c00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f33cc104ae0 0x7f33cc199fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:25:11.535 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:11.534+0000 7f33d1c00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f33cc1054a0 0x7f33cc1a1040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:25:11.535 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:11.534+0000 7f33d1c00640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33cc10f450 con 0x7f33cc1038f0 2024-03-21T10:25:11.535 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:11.534+0000 7f33d1c00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33cc10f2d0 con 0x7f33cc104ae0 2024-03-21T10:25:11.536 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:11.534+0000 7f33d1c00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33cc10f5d0 con 0x7f33cc1054a0 2024-03-21T10:25:11.537 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:11.535+0000 7f33caa00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f33cc1038f0 0x7f33cc199a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:25:11.537 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:11.535+0000 7f33c3e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f33cc104ae0 0x7f33cc199fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:25:11.537 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:11.535+0000 7f33caa00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f33cc1038f0 0x7f33cc199a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:56638/0 (socket says 172.21.15.47:56638) 2024-03-21T10:25:11.537 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:11.535+0000 7f33caa00640 1 -- 172.21.15.47:0/1613331918 learned_addr learned my addr 172.21.15.47:0/1613331918 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:25:11.537 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:11.535+0000 7f33cb400640 1 --2- 172.21.15.47:0/1613331918 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f33cc1054a0 0x7f33cc1a1040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:25:11.537 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:11.535+0000 7f33caa00640 1 -- 172.21.15.47:0/1613331918 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f33cc1054a0 msgr2=0x7f33cc1a1040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:25:11.537 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:11.535+0000 7f33caa00640 1 --2- 172.21.15.47:0/1613331918 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f33cc1054a0 0x7f33cc1a1040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:25:11.537 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:11.535+0000 7f33caa00640 1 -- 172.21.15.47:0/1613331918 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f33cc104ae0 msgr2=0x7f33cc199fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:25:11.538 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:11.535+0000 7f33caa00640 1 --2- 172.21.15.47:0/1613331918 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f33cc104ae0 0x7f33cc199fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:25:11.538 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:11.535+0000 7f33caa00640 1 -- 172.21.15.47:0/1613331918 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f33cc1a16b0 con 0x7f33cc1038f0 2024-03-21T10:25:11.538 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:11.536+0000 7f33caa00640 1 --2- 172.21.15.47:0/1613331918 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f33cc1038f0 0x7f33cc199a90 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f33bc00d950 tx=0x7f33bc00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:25:11.538 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:11.536+0000 7f33c3400640 1 -- 172.21.15.47:0/1613331918 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33bc017020 con 0x7f33cc1038f0 2024-03-21T10:25:11.538 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:11.536+0000 7f33c3400640 1 -- 172.21.15.47:0/1613331918 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f33bc009c30 con 0x7f33cc1038f0 2024-03-21T10:25:11.538 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:11.536+0000 7f33d1c00640 1 -- 172.21.15.47:0/1613331918 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f33cc1a19a0 con 0x7f33cc1038f0 2024-03-21T10:25:11.538 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:11.536+0000 7f33c3400640 1 -- 172.21.15.47:0/1613331918 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33bc00f6b0 con 0x7f33cc1038f0 2024-03-21T10:25:11.538 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:11.536+0000 7f33d1c00640 1 -- 172.21.15.47:0/1613331918 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f33cc1a1ec0 con 0x7f33cc1038f0 2024-03-21T10:25:11.538 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:11.537+0000 7f33d1c00640 1 -- 172.21.15.47:0/1613331918 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f33cc10ac10 con 0x7f33cc1038f0 2024-03-21T10:25:11.540 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:11.539+0000 7f33c3400640 1 -- 172.21.15.47:0/1613331918 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 19) v1 ==== 137871+0+0 (secure 0 0 0) 0x7f33bc01a020 con 0x7f33cc1038f0 2024-03-21T10:25:11.542 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:11.541+0000 7f33c3400640 1 --2- 172.21.15.47:0/1613331918 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f33b00a3060 0x7f33b00a5520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:25:11.542 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:11.542+0000 7f33c3e00640 1 -- 172.21.15.47:0/1613331918 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f33b00a3060 msgr2=0x7f33b00a5520 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.47:6800/2823885994 2024-03-21T10:25:11.543 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:11.542+0000 7f33c3400640 1 -- 172.21.15.47:0/1613331918 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9216+0+0 (secure 0 0 0) 0x7f33bc0cf7d0 con 0x7f33cc1038f0 2024-03-21T10:25:11.543 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:11.542+0000 7f33c3e00640 1 --2- 172.21.15.47:0/1613331918 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f33b00a3060 0x7f33b00a5520 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-21T10:25:11.545 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:11.545+0000 7f33c3400640 1 -- 172.21.15.47:0/1613331918 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f33bc021020 con 0x7f33cc1038f0 2024-03-21T10:25:11.641 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:25:11 smithi157 ceph-mon[35581]: scrub queued for path: / 2024-03-21T10:25:11.642 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:25:11 smithi157 ceph-mon[35581]: scrub summary: idle+waiting paths [/] 2024-03-21T10:25:11.642 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:25:11 smithi157 ceph-mon[35581]: scrub summary: active paths [/] 2024-03-21T10:25:11.642 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:25:11 smithi157 ceph-mon[35581]: scrub summary: idle+waiting paths [/] 2024-03-21T10:25:11.642 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:25:11 smithi157 ceph-mon[35581]: from='client.? 172.21.15.47:0/1749301017' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "snap_schedule"}]': finished 2024-03-21T10:25:11.642 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:25:11 smithi157 ceph-mon[35581]: mgrmap e19: x(active, since 12m), standbys: z, y 2024-03-21T10:25:11.642 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:25:11 smithi157 ceph-mon[35581]: from='client.? 172.21.15.47:0/1098362985' entity='client.admin' 2024-03-21T10:25:11.642 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:25:11 smithi157 ceph-mon[35581]: from='client.? 172.21.15.47:0/3115353101' entity='client.admin' 2024-03-21T10:25:11.643 INFO:journalctl@ceph.mgr.z.smithi157.stdout:Mar 21 10:25:11 smithi157 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-mgr-z[37318]: 2024-03-21T10:25:11.248+0000 7f00b7c23200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-03-21T10:25:11.643 INFO:journalctl@ceph.mgr.z.smithi157.stdout:Mar 21 10:25:11 smithi157 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-mgr-z[37318]: 2024-03-21T10:25:11.369+0000 7f00b7c23200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-03-21T10:25:11.708 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:25:11 smithi100 ceph-mon[35193]: scrub queued for path: / 2024-03-21T10:25:11.708 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:25:11 smithi100 ceph-mon[35193]: scrub summary: idle+waiting paths [/] 2024-03-21T10:25:11.708 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:25:11 smithi100 ceph-mon[35193]: scrub summary: active paths [/] 2024-03-21T10:25:11.708 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:25:11 smithi100 ceph-mon[35193]: scrub summary: idle+waiting paths [/] 2024-03-21T10:25:11.709 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:25:11 smithi100 ceph-mon[35193]: from='client.? 172.21.15.47:0/1749301017' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "snap_schedule"}]': finished 2024-03-21T10:25:11.709 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:25:11 smithi100 ceph-mon[35193]: mgrmap e19: x(active, since 12m), standbys: z, y 2024-03-21T10:25:11.709 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:25:11 smithi100 ceph-mon[35193]: from='client.? 172.21.15.47:0/1098362985' entity='client.admin' 2024-03-21T10:25:11.709 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:25:11 smithi100 ceph-mon[35193]: from='client.? 172.21.15.47:0/3115353101' entity='client.admin' 2024-03-21T10:25:11.709 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Mar 21 10:25:11 smithi100 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-mgr-y[36578]: 2024-03-21T10:25:11.504+0000 7f2a4f872200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-03-21T10:25:11.713 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:25:11 smithi047 ceph-mon[31547]: scrub queued for path: / 2024-03-21T10:25:11.713 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:25:11 smithi047 ceph-mon[31547]: scrub summary: idle+waiting paths [/] 2024-03-21T10:25:11.713 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:25:11 smithi047 ceph-mon[31547]: scrub summary: active paths [/] 2024-03-21T10:25:11.713 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:25:11 smithi047 ceph-mon[31547]: scrub summary: idle+waiting paths [/] 2024-03-21T10:25:11.714 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:25:11 smithi047 ceph-mon[31547]: from='client.? 172.21.15.47:0/1749301017' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "snap_schedule"}]': finished 2024-03-21T10:25:11.714 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:25:11 smithi047 ceph-mon[31547]: mgrmap e19: x(active, since 12m), standbys: z, y 2024-03-21T10:25:11.714 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:25:11 smithi047 ceph-mon[31547]: from='client.? 172.21.15.47:0/1098362985' entity='client.admin' 2024-03-21T10:25:11.714 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:25:11 smithi047 ceph-mon[31547]: from='client.? 172.21.15.47:0/3115353101' entity='client.admin' 2024-03-21T10:25:11.714 INFO:journalctl@ceph.mgr.x.smithi047.stdout:Mar 21 10:25:11 smithi047 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-mgr-x[31773]: 2024-03-21T10:25:11.351+0000 7ffb20de9200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-03-21T10:25:11.714 INFO:journalctl@ceph.mgr.x.smithi047.stdout:Mar 21 10:25:11 smithi047 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-mgr-x[31773]: 2024-03-21T10:25:11.499+0000 7ffb20de9200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-03-21T10:25:11.744 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:11.743+0000 7f33c3e00640 1 -- 172.21.15.47:0/1613331918 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f33b00a3060 msgr2=0x7f33b00a5520 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.47:6800/2823885994 2024-03-21T10:25:11.744 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:11.743+0000 7f33c3e00640 1 --2- 172.21.15.47:0/1613331918 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f33b00a3060 0x7f33b00a5520 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-21T10:25:11.809 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:11.808+0000 7f33d1c00640 1 -- 172.21.15.47:0/1613331918 --> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}) v1 -- 0x7f33cc061ae0 con 0x7f33b00a3060 2024-03-21T10:25:11.979 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Mar 21 10:25:11 smithi100 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-mgr-y[36578]: 2024-03-21T10:25:11.706+0000 7f2a4f872200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-03-21T10:25:12.145 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:12.144+0000 7f33c3e00640 1 -- 172.21.15.47:0/1613331918 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f33b00a3060 msgr2=0x7f33b00a5520 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.47:6800/2823885994 2024-03-21T10:25:12.145 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:12.144+0000 7f33c3e00640 1 --2- 172.21.15.47:0/1613331918 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f33b00a3060 0x7f33b00a5520 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-21T10:25:12.458 INFO:journalctl@ceph.mgr.z.smithi157.stdout:Mar 21 10:25:12 smithi157 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-mgr-z[37318]: 2024-03-21T10:25:12.157+0000 7f00b7c23200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-03-21T10:25:12.458 INFO:journalctl@ceph.mgr.z.smithi157.stdout:Mar 21 10:25:12 smithi157 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-mgr-z[37318]: 2024-03-21T10:25:12.278+0000 7f00b7c23200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-03-21T10:25:12.458 INFO:journalctl@ceph.mgr.z.smithi157.stdout:Mar 21 10:25:12 smithi157 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-mgr-z[37318]: 2024-03-21T10:25:12.455+0000 7f00b7c23200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-03-21T10:25:12.684 INFO:journalctl@ceph.mgr.x.smithi047.stdout:Mar 21 10:25:12 smithi047 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-mgr-x[31773]: 2024-03-21T10:25:12.372+0000 7ffb20de9200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-03-21T10:25:12.684 INFO:journalctl@ceph.mgr.x.smithi047.stdout:Mar 21 10:25:12 smithi047 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-mgr-x[31773]: 2024-03-21T10:25:12.499+0000 7ffb20de9200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-03-21T10:25:12.740 INFO:journalctl@ceph.mgr.z.smithi157.stdout:Mar 21 10:25:12 smithi157 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-mgr-z[37318]: 2024-03-21T10:25:12.580+0000 7f00b7c23200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-03-21T10:25:12.740 INFO:journalctl@ceph.mgr.z.smithi157.stdout:Mar 21 10:25:12 smithi157 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-mgr-z[37318]: 2024-03-21T10:25:12.691+0000 7f00b7c23200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-03-21T10:25:12.946 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:12.945+0000 7f33c3e00640 1 -- 172.21.15.47:0/1613331918 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f33b00a3060 msgr2=0x7f33b00a5520 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.47:6800/2823885994 2024-03-21T10:25:12.946 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:12.945+0000 7f33c3e00640 1 --2- 172.21.15.47:0/1613331918 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f33b00a3060 0x7f33b00a5520 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-21T10:25:12.964 INFO:journalctl@ceph.mgr.x.smithi047.stdout:Mar 21 10:25:12 smithi047 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-mgr-x[31773]: 2024-03-21T10:25:12.682+0000 7ffb20de9200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-03-21T10:25:12.964 INFO:journalctl@ceph.mgr.x.smithi047.stdout:Mar 21 10:25:12 smithi047 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-mgr-x[31773]: 2024-03-21T10:25:12.813+0000 7ffb20de9200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-03-21T10:25:12.964 INFO:journalctl@ceph.mgr.x.smithi047.stdout:Mar 21 10:25:12 smithi047 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-mgr-x[31773]: 2024-03-21T10:25:12.929+0000 7ffb20de9200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-03-21T10:25:13.137 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Mar 21 10:25:12 smithi100 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-mgr-y[36578]: 2024-03-21T10:25:12.739+0000 7f2a4f872200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-03-21T10:25:13.137 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Mar 21 10:25:12 smithi100 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-mgr-y[36578]: 2024-03-21T10:25:12.916+0000 7f2a4f872200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-03-21T10:25:13.137 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Mar 21 10:25:13 smithi100 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-mgr-y[36578]: 2024-03-21T10:25:13.135+0000 7f2a4f872200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-03-21T10:25:13.306 INFO:journalctl@ceph.mgr.z.smithi157.stdout:Mar 21 10:25:13 smithi157 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-mgr-z[37318]: 2024-03-21T10:25:13.042+0000 7f00b7c23200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-03-21T10:25:13.307 INFO:journalctl@ceph.mgr.z.smithi157.stdout:Mar 21 10:25:13 smithi157 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-mgr-z[37318]: 2024-03-21T10:25:13.188+0000 7f00b7c23200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-03-21T10:25:13.307 INFO:journalctl@ceph.mgr.z.smithi157.stdout:Mar 21 10:25:13 smithi157 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-mgr-z[37318]: 2024-03-21T10:25:13.305+0000 7f00b7c23200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-03-21T10:25:13.479 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Mar 21 10:25:13 smithi100 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-mgr-y[36578]: 2024-03-21T10:25:13.327+0000 7f2a4f872200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-03-21T10:25:13.538 INFO:journalctl@ceph.mgr.x.smithi047.stdout:Mar 21 10:25:13 smithi047 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-mgr-x[31773]: 2024-03-21T10:25:13.273+0000 7ffb20de9200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-03-21T10:25:13.538 INFO:journalctl@ceph.mgr.x.smithi047.stdout:Mar 21 10:25:13 smithi047 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-mgr-x[31773]: 2024-03-21T10:25:13.419+0000 7ffb20de9200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-03-21T10:25:13.538 INFO:journalctl@ceph.mgr.x.smithi047.stdout:Mar 21 10:25:13 smithi047 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-mgr-x[31773]: 2024-03-21T10:25:13.536+0000 7ffb20de9200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-03-21T10:25:13.666 INFO:journalctl@ceph.mgr.z.smithi157.stdout:Mar 21 10:25:13 smithi157 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-mgr-z[37318]: 2024-03-21T10:25:13.434+0000 7f00b7c23200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-03-21T10:25:13.894 INFO:journalctl@ceph.mgr.x.smithi047.stdout:Mar 21 10:25:13 smithi047 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-mgr-x[31773]: 2024-03-21T10:25:13.668+0000 7ffb20de9200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-03-21T10:25:13.979 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Mar 21 10:25:13 smithi100 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-mgr-y[36578]: 2024-03-21T10:25:13.496+0000 7f2a4f872200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-03-21T10:25:13.989 INFO:journalctl@ceph.mgr.z.smithi157.stdout:Mar 21 10:25:13 smithi157 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-mgr-z[37318]: 2024-03-21T10:25:13.664+0000 7f00b7c23200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-03-21T10:25:14.214 INFO:journalctl@ceph.mgr.x.smithi047.stdout:Mar 21 10:25:13 smithi047 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-mgr-x[31773]: 2024-03-21T10:25:13.892+0000 7ffb20de9200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-03-21T10:25:14.362 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Mar 21 10:25:13 smithi100 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-mgr-y[36578]: 2024-03-21T10:25:13.984+0000 7f2a4f872200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-03-21T10:25:14.363 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Mar 21 10:25:14 smithi100 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-mgr-y[36578]: 2024-03-21T10:25:14.185+0000 7f2a4f872200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-03-21T10:25:14.548 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:14.547+0000 7f33c3e00640 1 -- 172.21.15.47:0/1613331918 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f33b00a3060 msgr2=0x7f33b00a5520 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.47:6800/2823885994 2024-03-21T10:25:14.548 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:14.547+0000 7f33c3e00640 1 --2- 172.21.15.47:0/1613331918 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f33b00a3060 0x7f33b00a5520 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2024-03-21T10:25:14.729 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Mar 21 10:25:14 smithi100 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-mgr-y[36578]: 2024-03-21T10:25:14.361+0000 7f2a4f872200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-03-21T10:25:14.729 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Mar 21 10:25:14 smithi100 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-mgr-y[36578]: 2024-03-21T10:25:14.536+0000 7f2a4f872200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-03-21T10:25:15.229 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Mar 21 10:25:14 smithi100 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-mgr-y[36578]: 2024-03-21T10:25:14.848+0000 7f2a4f872200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-03-21T10:25:15.239 INFO:journalctl@ceph.mgr.z.smithi157.stdout:Mar 21 10:25:14 smithi157 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-mgr-z[37318]: 2024-03-21T10:25:14.977+0000 7f00b7c23200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-03-21T10:25:15.239 INFO:journalctl@ceph.mgr.z.smithi157.stdout:Mar 21 10:25:15 smithi157 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-mgr-z[37318]: 2024-03-21T10:25:15.148+0000 7f00b7c23200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-03-21T10:25:15.714 INFO:journalctl@ceph.mgr.x.smithi047.stdout:Mar 21 10:25:15 smithi047 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-mgr-x[31773]: 2024-03-21T10:25:15.224+0000 7ffb20de9200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-03-21T10:25:15.714 INFO:journalctl@ceph.mgr.x.smithi047.stdout:Mar 21 10:25:15 smithi047 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-mgr-x[31773]: 2024-03-21T10:25:15.409+0000 7ffb20de9200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-03-21T10:25:15.943 INFO:journalctl@ceph.mgr.z.smithi157.stdout:Mar 21 10:25:15 smithi157 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-mgr-z[37318]: 2024-03-21T10:25:15.570+0000 7f00b7c23200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-03-21T10:25:15.943 INFO:journalctl@ceph.mgr.z.smithi157.stdout:Mar 21 10:25:15 smithi157 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-mgr-z[37318]: 2024-03-21T10:25:15.687+0000 7f00b7c23200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-03-21T10:25:16.213 INFO:journalctl@ceph.mgr.x.smithi047.stdout:Mar 21 10:25:15 smithi047 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-mgr-x[31773]: 2024-03-21T10:25:15.844+0000 7ffb20de9200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-03-21T10:25:16.213 INFO:journalctl@ceph.mgr.x.smithi047.stdout:Mar 21 10:25:15 smithi047 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-mgr-x[31773]: 2024-03-21T10:25:15.968+0000 7ffb20de9200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-03-21T10:25:16.239 INFO:journalctl@ceph.mgr.z.smithi157.stdout:Mar 21 10:25:15 smithi157 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-mgr-z[37318]: 2024-03-21T10:25:15.941+0000 7f00b7c23200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-03-21T10:25:16.239 INFO:journalctl@ceph.mgr.z.smithi157.stdout:Mar 21 10:25:16 smithi157 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-mgr-z[37318]: 2024-03-21T10:25:16.183+0000 7f00b7c23200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-03-21T10:25:16.499 INFO:journalctl@ceph.mgr.x.smithi047.stdout:Mar 21 10:25:16 smithi047 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-mgr-x[31773]: 2024-03-21T10:25:16.238+0000 7ffb20de9200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-03-21T10:25:16.499 INFO:journalctl@ceph.mgr.x.smithi047.stdout:Mar 21 10:25:16 smithi047 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-mgr-x[31773]: 2024-03-21T10:25:16.497+0000 7ffb20de9200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-03-21T10:25:16.834 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Mar 21 10:25:16 smithi100 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-mgr-y[36578]: 2024-03-21T10:25:16.567+0000 7f2a4f872200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-03-21T10:25:16.873 INFO:journalctl@ceph.mgr.z.smithi157.stdout:Mar 21 10:25:16 smithi157 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-mgr-z[37318]: 2024-03-21T10:25:16.551+0000 7f00b7c23200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-03-21T10:25:16.911 INFO:journalctl@ceph.mgr.x.smithi047.stdout:Mar 21 10:25:16 smithi047 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-mgr-x[31773]: 2024-03-21T10:25:16.909+0000 7ffb20de9200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-03-21T10:25:17.177 INFO:journalctl@ceph.mgr.z.smithi157.stdout:Mar 21 10:25:16 smithi157 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-mgr-z[37318]: 2024-03-21T10:25:16.871+0000 7f00b7c23200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-03-21T10:25:17.229 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Mar 21 10:25:16 smithi100 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-mgr-y[36578]: 2024-03-21T10:25:16.833+0000 7f2a4f872200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-03-21T10:25:17.489 INFO:journalctl@ceph.mgr.z.smithi157.stdout:Mar 21 10:25:17 smithi157 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-mgr-z[37318]: 2024-03-21T10:25:17.175+0000 7f00b7c23200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-03-21T10:25:17.561 INFO:journalctl@ceph.mgr.x.smithi047.stdout:Mar 21 10:25:17 smithi047 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-mgr-x[31773]: 2024-03-21T10:25:17.237+0000 7ffb20de9200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-03-21T10:25:17.729 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Mar 21 10:25:17 smithi100 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-mgr-y[36578]: 2024-03-21T10:25:17.350+0000 7f2a4f872200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-03-21T10:25:17.729 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Mar 21 10:25:17 smithi100 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-mgr-y[36578]: 2024-03-21T10:25:17.475+0000 7f2a4f872200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-03-21T10:25:17.751 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:17.750+0000 7f33c3e00640 1 -- 172.21.15.47:0/1613331918 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f33b00a3060 msgr2=0x7f33b00a5520 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.47:6800/2823885994 2024-03-21T10:25:17.751 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:17.750+0000 7f33c3e00640 1 --2- 172.21.15.47:0/1613331918 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f33b00a3060 0x7f33b00a5520 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 6.400000 2024-03-21T10:25:17.963 INFO:journalctl@ceph.mgr.x.smithi047.stdout:Mar 21 10:25:17 smithi047 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-mgr-x[31773]: 2024-03-21T10:25:17.558+0000 7ffb20de9200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-03-21T10:25:18.139 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Mar 21 10:25:17 smithi100 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-mgr-y[36578]: 2024-03-21T10:25:17.780+0000 7f2a4f872200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-03-21T10:25:18.140 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Mar 21 10:25:18 smithi100 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-mgr-y[36578]: 2024-03-21T10:25:18.138+0000 7f2a4f872200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-03-21T10:25:18.489 INFO:journalctl@ceph.mgr.z.smithi157.stdout:Mar 21 10:25:18 smithi157 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-mgr-z[37318]: 2024-03-21T10:25:18.196+0000 7f00b7c23200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-03-21T10:25:18.963 INFO:journalctl@ceph.mgr.x.smithi047.stdout:Mar 21 10:25:18 smithi047 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-mgr-x[31773]: 2024-03-21T10:25:18.656+0000 7ffb20de9200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-03-21T10:25:18.979 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Mar 21 10:25:18 smithi100 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-mgr-y[36578]: 2024-03-21T10:25:18.666+0000 7f2a4f872200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-03-21T10:25:19.239 INFO:journalctl@ceph.mgr.z.smithi157.stdout:Mar 21 10:25:18 smithi157 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-mgr-z[37318]: 2024-03-21T10:25:18.897+0000 7f00b7c23200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-03-21T10:25:19.240 INFO:journalctl@ceph.mgr.z.smithi157.stdout:Mar 21 10:25:19 smithi157 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-mgr-z[37318]: 2024-03-21T10:25:19.024+0000 7f00b7c23200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-03-21T10:25:19.385 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:19.384+0000 7f33c3400640 1 -- 172.21.15.47:0/1613331918 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mgrmap(e 20) v1 ==== 137871+0+0 (secure 0 0 0) 0x7f33bc098160 con 0x7f33cc1038f0 2024-03-21T10:25:19.389 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Mar 21 10:25:19 smithi100 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-mgr-y[36578]: 2024-03-21T10:25:19.059+0000 7f2a4f872200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-03-21T10:25:19.713 INFO:journalctl@ceph.mgr.x.smithi047.stdout:Mar 21 10:25:19 smithi047 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-mgr-x[31773]: 2024-03-21T10:25:19.465+0000 7ffb20de9200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-03-21T10:25:19.713 INFO:journalctl@ceph.mgr.x.smithi047.stdout:Mar 21 10:25:19 smithi047 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-mgr-x[31773]: 2024-03-21T10:25:19.603+0000 7ffb20de9200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-03-21T10:25:19.714 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:25:19 smithi047 ceph-mon[31547]: Standby manager daemon z restarted 2024-03-21T10:25:19.714 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:25:19 smithi047 ceph-mon[31547]: Standby manager daemon z started 2024-03-21T10:25:19.714 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:25:19 smithi047 ceph-mon[31547]: from='mgr.? 172.21.15.157:0/839892404' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-03-21T10:25:19.714 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:25:19 smithi047 ceph-mon[31547]: from='mgr.? 172.21.15.157:0/839892404' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-03-21T10:25:19.714 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:25:19 smithi047 ceph-mon[31547]: from='mgr.? 172.21.15.157:0/839892404' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-03-21T10:25:19.714 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:25:19 smithi047 ceph-mon[31547]: from='mgr.? 172.21.15.157:0/839892404' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-03-21T10:25:19.729 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Mar 21 10:25:19 smithi100 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-mgr-y[36578]: 2024-03-21T10:25:19.432+0000 7f2a4f872200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-03-21T10:25:19.729 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:25:19 smithi100 ceph-mon[35193]: Standby manager daemon z restarted 2024-03-21T10:25:19.729 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:25:19 smithi100 ceph-mon[35193]: Standby manager daemon z started 2024-03-21T10:25:19.729 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:25:19 smithi100 ceph-mon[35193]: from='mgr.? 172.21.15.157:0/839892404' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-03-21T10:25:19.729 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:25:19 smithi100 ceph-mon[35193]: from='mgr.? 172.21.15.157:0/839892404' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-03-21T10:25:19.729 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:25:19 smithi100 ceph-mon[35193]: from='mgr.? 172.21.15.157:0/839892404' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-03-21T10:25:19.730 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:25:19 smithi100 ceph-mon[35193]: from='mgr.? 172.21.15.157:0/839892404' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-03-21T10:25:19.739 INFO:journalctl@ceph.mgr.z.smithi157.stdout:Mar 21 10:25:19 smithi157 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-mgr-z[37318]: 2024-03-21T10:25:19.304+0000 7f00b7c23200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-03-21T10:25:19.740 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:25:19 smithi157 ceph-mon[35581]: Standby manager daemon z restarted 2024-03-21T10:25:19.740 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:25:19 smithi157 ceph-mon[35581]: Standby manager daemon z started 2024-03-21T10:25:19.740 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:25:19 smithi157 ceph-mon[35581]: from='mgr.? 172.21.15.157:0/839892404' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-03-21T10:25:19.740 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:25:19 smithi157 ceph-mon[35581]: from='mgr.? 172.21.15.157:0/839892404' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-03-21T10:25:19.740 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:25:19 smithi157 ceph-mon[35581]: from='mgr.? 172.21.15.157:0/839892404' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-03-21T10:25:19.740 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:25:19 smithi157 ceph-mon[35581]: from='mgr.? 172.21.15.157:0/839892404' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-03-21T10:25:19.942 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:19.941+0000 7f33c3400640 1 -- 172.21.15.47:0/1613331918 <== mon.0 v2:172.21.15.47:3300/0 8 ==== mgrmap(e 21) v1 ==== 137539+0+0 (secure 0 0 0) 0x7f33bc0ccf40 con 0x7f33cc1038f0 2024-03-21T10:25:19.944 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:19.943+0000 7f33c3400640 1 -- 172.21.15.47:0/1613331918 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f33b00a3060 msgr2=0x7f33b00a5520 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-21T10:25:19.944 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:19.943+0000 7f33c3400640 1 --2- 172.21.15.47:0/1613331918 >> [v2:172.21.15.47:6800/2823885994,v1:172.21.15.47:6801/2823885994] conn(0x7f33b00a3060 0x7f33b00a5520 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:25:20.163 INFO:journalctl@ceph.mgr.x.smithi047.stdout:Mar 21 10:25:19 smithi047 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-mgr-x[31773]: 2024-03-21T10:25:19.909+0000 7ffb20de9200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-03-21T10:25:20.465 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:25:20 smithi047 ceph-mon[31547]: mgrmap e20: x(active, since 12m), standbys: z, y 2024-03-21T10:25:20.465 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:25:20 smithi047 ceph-mon[31547]: Active manager daemon x restarted 2024-03-21T10:25:20.465 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:25:20 smithi047 ceph-mon[31547]: Activating manager daemon x 2024-03-21T10:25:20.465 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:25:20 smithi047 ceph-mon[31547]: osdmap e96: 12 total, 12 up, 12 in 2024-03-21T10:25:20.465 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:25:20 smithi047 ceph-mon[31547]: mgrmap e21: x(active, starting, since 0.0318817s), standbys: z, y 2024-03-21T10:25:20.465 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:25:20 smithi047 ceph-mon[31547]: from='mgr.15687 172.21.15.47:0/8730661' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-21T10:25:20.465 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:25:20 smithi047 ceph-mon[31547]: from='mgr.15687 172.21.15.47:0/8730661' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-21T10:25:20.465 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:25:20 smithi047 ceph-mon[31547]: from='mgr.15687 172.21.15.47:0/8730661' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-21T10:25:20.465 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:25:20 smithi047 ceph-mon[31547]: from='mgr.15687 172.21.15.47:0/8730661' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-03-21T10:25:20.465 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:25:20 smithi047 ceph-mon[31547]: from='mgr.15687 172.21.15.47:0/8730661' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-03-21T10:25:20.465 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:25:20 smithi047 ceph-mon[31547]: from='mgr.15687 172.21.15.47:0/8730661' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-03-21T10:25:20.465 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:25:20 smithi047 ceph-mon[31547]: from='mgr.15687 172.21.15.47:0/8730661' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-03-21T10:25:20.465 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:25:20 smithi047 ceph-mon[31547]: from='mgr.15687 172.21.15.47:0/8730661' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-03-21T10:25:20.466 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:25:20 smithi047 ceph-mon[31547]: from='mgr.15687 172.21.15.47:0/8730661' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-03-21T10:25:20.466 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:25:20 smithi047 ceph-mon[31547]: from='mgr.15687 172.21.15.47:0/8730661' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-03-21T10:25:20.466 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:25:20 smithi047 ceph-mon[31547]: from='mgr.15687 172.21.15.47:0/8730661' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-03-21T10:25:20.466 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:25:20 smithi047 ceph-mon[31547]: from='mgr.15687 172.21.15.47:0/8730661' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-03-21T10:25:20.466 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:25:20 smithi047 ceph-mon[31547]: from='mgr.15687 172.21.15.47:0/8730661' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-03-21T10:25:20.466 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:25:20 smithi047 ceph-mon[31547]: from='mgr.15687 172.21.15.47:0/8730661' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-03-21T10:25:20.466 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:25:20 smithi047 ceph-mon[31547]: from='mgr.15687 172.21.15.47:0/8730661' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-03-21T10:25:20.466 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:25:20 smithi047 ceph-mon[31547]: from='mgr.15687 172.21.15.47:0/8730661' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-03-21T10:25:20.466 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:25:20 smithi047 ceph-mon[31547]: from='mgr.15687 172.21.15.47:0/8730661' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-03-21T10:25:20.466 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:25:20 smithi047 ceph-mon[31547]: from='mgr.15687 172.21.15.47:0/8730661' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-03-21T10:25:20.466 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:25:20 smithi047 ceph-mon[31547]: from='mgr.15687 172.21.15.47:0/8730661' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-21T10:25:20.466 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:25:20 smithi047 ceph-mon[31547]: from='mgr.15687 172.21.15.47:0/8730661' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-21T10:25:20.466 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:25:20 smithi047 ceph-mon[31547]: from='mgr.15687 172.21.15.47:0/8730661' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-21T10:25:20.466 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:25:20 smithi047 ceph-mon[31547]: from='mgr.15687 172.21.15.47:0/8730661' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-21T10:25:20.467 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:25:20 smithi047 ceph-mon[31547]: from='mgr.15687 172.21.15.47:0/8730661' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-21T10:25:20.467 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:25:20 smithi047 ceph-mon[31547]: from='mgr.15687 172.21.15.47:0/8730661' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-21T10:25:20.467 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:25:20 smithi047 ceph-mon[31547]: from='mgr.15687 172.21.15.47:0/8730661' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-21T10:25:20.467 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:25:20 smithi047 ceph-mon[31547]: from='mgr.15687 172.21.15.47:0/8730661' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-21T10:25:20.467 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:25:20 smithi047 ceph-mon[31547]: from='mgr.15687 172.21.15.47:0/8730661' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-21T10:25:20.467 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:25:20 smithi047 ceph-mon[31547]: from='mgr.15687 172.21.15.47:0/8730661' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-21T10:25:20.467 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:25:20 smithi047 ceph-mon[31547]: from='mgr.15687 172.21.15.47:0/8730661' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-21T10:25:20.467 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:25:20 smithi047 ceph-mon[31547]: from='mgr.15687 172.21.15.47:0/8730661' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-21T10:25:20.467 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:25:20 smithi047 ceph-mon[31547]: from='mgr.15687 172.21.15.47:0/8730661' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-03-21T10:25:20.467 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:25:20 smithi047 ceph-mon[31547]: from='mgr.15687 172.21.15.47:0/8730661' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-03-21T10:25:20.468 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:25:20 smithi047 ceph-mon[31547]: from='mgr.15687 172.21.15.47:0/8730661' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-03-21T10:25:20.468 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:25:20 smithi047 ceph-mon[31547]: Manager daemon x is now available 2024-03-21T10:25:20.468 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:25:20 smithi047 ceph-mon[31547]: from='mgr.15687 172.21.15.47:0/8730661' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T10:25:20.468 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:25:20 smithi047 ceph-mon[31547]: from='mgr.15687 172.21.15.47:0/8730661' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-03-21T10:25:20.468 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:25:20 smithi047 ceph-mon[31547]: from='mgr.15687 172.21.15.47:0/8730661' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-03-21T10:25:20.480 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:25:20 smithi100 ceph-mon[35193]: mgrmap e20: x(active, since 12m), standbys: z, y 2024-03-21T10:25:20.480 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:25:20 smithi100 ceph-mon[35193]: Active manager daemon x restarted 2024-03-21T10:25:20.480 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:25:20 smithi100 ceph-mon[35193]: Activating manager daemon x 2024-03-21T10:25:20.480 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:25:20 smithi100 ceph-mon[35193]: osdmap e96: 12 total, 12 up, 12 in 2024-03-21T10:25:20.480 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:25:20 smithi100 ceph-mon[35193]: mgrmap e21: x(active, starting, since 0.0318817s), standbys: z, y 2024-03-21T10:25:20.480 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:25:20 smithi100 ceph-mon[35193]: from='mgr.15687 172.21.15.47:0/8730661' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-21T10:25:20.480 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:25:20 smithi100 ceph-mon[35193]: from='mgr.15687 172.21.15.47:0/8730661' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-21T10:25:20.480 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:25:20 smithi100 ceph-mon[35193]: from='mgr.15687 172.21.15.47:0/8730661' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-21T10:25:20.480 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:25:20 smithi100 ceph-mon[35193]: from='mgr.15687 172.21.15.47:0/8730661' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-03-21T10:25:20.481 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:25:20 smithi100 ceph-mon[35193]: from='mgr.15687 172.21.15.47:0/8730661' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-03-21T10:25:20.481 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:25:20 smithi100 ceph-mon[35193]: from='mgr.15687 172.21.15.47:0/8730661' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-03-21T10:25:20.481 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:25:20 smithi100 ceph-mon[35193]: from='mgr.15687 172.21.15.47:0/8730661' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-03-21T10:25:20.481 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:25:20 smithi100 ceph-mon[35193]: from='mgr.15687 172.21.15.47:0/8730661' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-03-21T10:25:20.481 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:25:20 smithi100 ceph-mon[35193]: from='mgr.15687 172.21.15.47:0/8730661' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-03-21T10:25:20.481 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:25:20 smithi100 ceph-mon[35193]: from='mgr.15687 172.21.15.47:0/8730661' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-03-21T10:25:20.481 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:25:20 smithi100 ceph-mon[35193]: from='mgr.15687 172.21.15.47:0/8730661' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-03-21T10:25:20.481 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:25:20 smithi100 ceph-mon[35193]: from='mgr.15687 172.21.15.47:0/8730661' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-03-21T10:25:20.481 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:25:20 smithi100 ceph-mon[35193]: from='mgr.15687 172.21.15.47:0/8730661' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-03-21T10:25:20.481 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:25:20 smithi100 ceph-mon[35193]: from='mgr.15687 172.21.15.47:0/8730661' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-03-21T10:25:20.481 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:25:20 smithi100 ceph-mon[35193]: from='mgr.15687 172.21.15.47:0/8730661' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-03-21T10:25:20.481 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:25:20 smithi100 ceph-mon[35193]: from='mgr.15687 172.21.15.47:0/8730661' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-03-21T10:25:20.481 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:25:20 smithi100 ceph-mon[35193]: from='mgr.15687 172.21.15.47:0/8730661' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-03-21T10:25:20.481 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:25:20 smithi100 ceph-mon[35193]: from='mgr.15687 172.21.15.47:0/8730661' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-03-21T10:25:20.482 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:25:20 smithi100 ceph-mon[35193]: from='mgr.15687 172.21.15.47:0/8730661' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-21T10:25:20.482 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:25:20 smithi100 ceph-mon[35193]: from='mgr.15687 172.21.15.47:0/8730661' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-21T10:25:20.482 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:25:20 smithi100 ceph-mon[35193]: from='mgr.15687 172.21.15.47:0/8730661' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-21T10:25:20.482 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:25:20 smithi100 ceph-mon[35193]: from='mgr.15687 172.21.15.47:0/8730661' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-21T10:25:20.482 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:25:20 smithi100 ceph-mon[35193]: from='mgr.15687 172.21.15.47:0/8730661' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-21T10:25:20.482 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:25:20 smithi100 ceph-mon[35193]: from='mgr.15687 172.21.15.47:0/8730661' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-21T10:25:20.482 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:25:20 smithi100 ceph-mon[35193]: from='mgr.15687 172.21.15.47:0/8730661' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-21T10:25:20.482 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:25:20 smithi100 ceph-mon[35193]: from='mgr.15687 172.21.15.47:0/8730661' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-21T10:25:20.482 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:25:20 smithi100 ceph-mon[35193]: from='mgr.15687 172.21.15.47:0/8730661' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-21T10:25:20.482 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:25:20 smithi100 ceph-mon[35193]: from='mgr.15687 172.21.15.47:0/8730661' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-21T10:25:20.482 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:25:20 smithi100 ceph-mon[35193]: from='mgr.15687 172.21.15.47:0/8730661' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-21T10:25:20.482 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:25:20 smithi100 ceph-mon[35193]: from='mgr.15687 172.21.15.47:0/8730661' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-21T10:25:20.482 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:25:20 smithi100 ceph-mon[35193]: from='mgr.15687 172.21.15.47:0/8730661' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-03-21T10:25:20.483 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:25:20 smithi100 ceph-mon[35193]: from='mgr.15687 172.21.15.47:0/8730661' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-03-21T10:25:20.483 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:25:20 smithi100 ceph-mon[35193]: from='mgr.15687 172.21.15.47:0/8730661' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-03-21T10:25:20.483 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:25:20 smithi100 ceph-mon[35193]: Manager daemon x is now available 2024-03-21T10:25:20.483 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:25:20 smithi100 ceph-mon[35193]: from='mgr.15687 172.21.15.47:0/8730661' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T10:25:20.483 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:25:20 smithi100 ceph-mon[35193]: from='mgr.15687 172.21.15.47:0/8730661' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-03-21T10:25:20.483 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:25:20 smithi100 ceph-mon[35193]: from='mgr.15687 172.21.15.47:0/8730661' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-03-21T10:25:20.490 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:25:20 smithi157 ceph-mon[35581]: mgrmap e20: x(active, since 12m), standbys: z, y 2024-03-21T10:25:20.490 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:25:20 smithi157 ceph-mon[35581]: Active manager daemon x restarted 2024-03-21T10:25:20.490 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:25:20 smithi157 ceph-mon[35581]: Activating manager daemon x 2024-03-21T10:25:20.491 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:25:20 smithi157 ceph-mon[35581]: osdmap e96: 12 total, 12 up, 12 in 2024-03-21T10:25:20.491 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:25:20 smithi157 ceph-mon[35581]: mgrmap e21: x(active, starting, since 0.0318817s), standbys: z, y 2024-03-21T10:25:20.491 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:25:20 smithi157 ceph-mon[35581]: from='mgr.15687 172.21.15.47:0/8730661' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-21T10:25:20.491 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:25:20 smithi157 ceph-mon[35581]: from='mgr.15687 172.21.15.47:0/8730661' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-21T10:25:20.491 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:25:20 smithi157 ceph-mon[35581]: from='mgr.15687 172.21.15.47:0/8730661' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-21T10:25:20.491 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:25:20 smithi157 ceph-mon[35581]: from='mgr.15687 172.21.15.47:0/8730661' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-03-21T10:25:20.491 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:25:20 smithi157 ceph-mon[35581]: from='mgr.15687 172.21.15.47:0/8730661' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-03-21T10:25:20.491 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:25:20 smithi157 ceph-mon[35581]: from='mgr.15687 172.21.15.47:0/8730661' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-03-21T10:25:20.491 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:25:20 smithi157 ceph-mon[35581]: from='mgr.15687 172.21.15.47:0/8730661' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-03-21T10:25:20.491 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:25:20 smithi157 ceph-mon[35581]: from='mgr.15687 172.21.15.47:0/8730661' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-03-21T10:25:20.491 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:25:20 smithi157 ceph-mon[35581]: from='mgr.15687 172.21.15.47:0/8730661' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-03-21T10:25:20.491 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:25:20 smithi157 ceph-mon[35581]: from='mgr.15687 172.21.15.47:0/8730661' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-03-21T10:25:20.491 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:25:20 smithi157 ceph-mon[35581]: from='mgr.15687 172.21.15.47:0/8730661' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-03-21T10:25:20.491 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:25:20 smithi157 ceph-mon[35581]: from='mgr.15687 172.21.15.47:0/8730661' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-03-21T10:25:20.491 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:25:20 smithi157 ceph-mon[35581]: from='mgr.15687 172.21.15.47:0/8730661' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-03-21T10:25:20.492 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:25:20 smithi157 ceph-mon[35581]: from='mgr.15687 172.21.15.47:0/8730661' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-03-21T10:25:20.492 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:25:20 smithi157 ceph-mon[35581]: from='mgr.15687 172.21.15.47:0/8730661' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-03-21T10:25:20.492 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:25:20 smithi157 ceph-mon[35581]: from='mgr.15687 172.21.15.47:0/8730661' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-03-21T10:25:20.492 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:25:20 smithi157 ceph-mon[35581]: from='mgr.15687 172.21.15.47:0/8730661' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-03-21T10:25:20.492 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:25:20 smithi157 ceph-mon[35581]: from='mgr.15687 172.21.15.47:0/8730661' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-03-21T10:25:20.492 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:25:20 smithi157 ceph-mon[35581]: from='mgr.15687 172.21.15.47:0/8730661' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-21T10:25:20.492 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:25:20 smithi157 ceph-mon[35581]: from='mgr.15687 172.21.15.47:0/8730661' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-21T10:25:20.492 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:25:20 smithi157 ceph-mon[35581]: from='mgr.15687 172.21.15.47:0/8730661' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-21T10:25:20.492 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:25:20 smithi157 ceph-mon[35581]: from='mgr.15687 172.21.15.47:0/8730661' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-21T10:25:20.492 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:25:20 smithi157 ceph-mon[35581]: from='mgr.15687 172.21.15.47:0/8730661' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-21T10:25:20.492 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:25:20 smithi157 ceph-mon[35581]: from='mgr.15687 172.21.15.47:0/8730661' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-21T10:25:20.493 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:25:20 smithi157 ceph-mon[35581]: from='mgr.15687 172.21.15.47:0/8730661' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-21T10:25:20.493 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:25:20 smithi157 ceph-mon[35581]: from='mgr.15687 172.21.15.47:0/8730661' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-21T10:25:20.493 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:25:20 smithi157 ceph-mon[35581]: from='mgr.15687 172.21.15.47:0/8730661' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-21T10:25:20.493 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:25:20 smithi157 ceph-mon[35581]: from='mgr.15687 172.21.15.47:0/8730661' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-21T10:25:20.493 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:25:20 smithi157 ceph-mon[35581]: from='mgr.15687 172.21.15.47:0/8730661' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-21T10:25:20.493 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:25:20 smithi157 ceph-mon[35581]: from='mgr.15687 172.21.15.47:0/8730661' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-21T10:25:20.493 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:25:20 smithi157 ceph-mon[35581]: from='mgr.15687 172.21.15.47:0/8730661' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-03-21T10:25:20.493 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:25:20 smithi157 ceph-mon[35581]: from='mgr.15687 172.21.15.47:0/8730661' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-03-21T10:25:20.493 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:25:20 smithi157 ceph-mon[35581]: from='mgr.15687 172.21.15.47:0/8730661' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-03-21T10:25:20.493 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:25:20 smithi157 ceph-mon[35581]: Manager daemon x is now available 2024-03-21T10:25:20.493 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:25:20 smithi157 ceph-mon[35581]: from='mgr.15687 172.21.15.47:0/8730661' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T10:25:20.493 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:25:20 smithi157 ceph-mon[35581]: from='mgr.15687 172.21.15.47:0/8730661' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-03-21T10:25:20.493 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:25:20 smithi157 ceph-mon[35581]: from='mgr.15687 172.21.15.47:0/8730661' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-03-21T10:25:20.952 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:20.949+0000 7f33c3400640 1 -- 172.21.15.47:0/1613331918 <== mon.0 v2:172.21.15.47:3300/0 9 ==== mgrmap(e 22) v1 ==== 137666+0+0 (secure 0 0 0) 0x7f33bc08f640 con 0x7f33cc1038f0 2024-03-21T10:25:20.952 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:20.951+0000 7f33c3400640 1 --2- 172.21.15.47:0/1613331918 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f33b0035a10 0x7f33b0035df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:25:20.953 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:20.951+0000 7f33c3400640 1 -- 172.21.15.47:0/1613331918 --> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}) v1 -- 0x7f33b003b4e0 con 0x7f33b0035a10 2024-03-21T10:25:20.957 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:20.956+0000 7f33c3e00640 1 --2- 172.21.15.47:0/1613331918 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f33b0035a10 0x7f33b0035df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:25:20.961 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:20.961+0000 7f33c3e00640 1 --2- 172.21.15.47:0/1613331918 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f33b0035a10 0x7f33b0035df0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f33b4000c00 tx=0x7f33b4005fd0 comp rx=0 tx=0).ready entity=mgr.15687 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:25:20.964 INFO:journalctl@ceph.mgr.x.smithi047.stdout:Mar 21 10:25:20 smithi047 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-mgr-x[31773]: 2024-03-21T10:25:20.516+0000 7ffa75400700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-21T10:25:20.964 INFO:journalctl@ceph.mgr.x.smithi047.stdout:Mar 21 10:25:20 smithi047 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-mgr-x[31773]: 2024-03-21T10:25:20.516+0000 7ffa75400700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-21T10:25:20.964 INFO:journalctl@ceph.mgr.x.smithi047.stdout:Mar 21 10:25:20 smithi047 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-mgr-x[31773]: 2024-03-21T10:25:20.516+0000 7ffa75400700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-21T10:25:20.964 INFO:journalctl@ceph.mgr.x.smithi047.stdout:Mar 21 10:25:20 smithi047 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-mgr-x[31773]: 2024-03-21T10:25:20.516+0000 7ffa75400700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-21T10:25:20.965 INFO:journalctl@ceph.mgr.x.smithi047.stdout:Mar 21 10:25:20 smithi047 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-mgr-x[31773]: 2024-03-21T10:25:20.516+0000 7ffa75400700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-21T10:25:20.965 INFO:journalctl@ceph.mgr.x.smithi047.stdout:Mar 21 10:25:20 smithi047 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-mgr-x[31773]: 2024-03-21T10:25:20.516+0000 7ffa73600700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-21T10:25:20.965 INFO:journalctl@ceph.mgr.x.smithi047.stdout:Mar 21 10:25:20 smithi047 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-mgr-x[31773]: 2024-03-21T10:25:20.516+0000 7ffa73600700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-21T10:25:20.965 INFO:journalctl@ceph.mgr.x.smithi047.stdout:Mar 21 10:25:20 smithi047 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-mgr-x[31773]: 2024-03-21T10:25:20.516+0000 7ffa73600700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-21T10:25:20.965 INFO:journalctl@ceph.mgr.x.smithi047.stdout:Mar 21 10:25:20 smithi047 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-mgr-x[31773]: 2024-03-21T10:25:20.516+0000 7ffa73600700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-21T10:25:20.965 INFO:journalctl@ceph.mgr.x.smithi047.stdout:Mar 21 10:25:20 smithi047 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-mgr-x[31773]: 2024-03-21T10:25:20.516+0000 7ffa73600700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-21T10:25:21.080 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:21.079+0000 7f33c3400640 1 -- 172.21.15.47:0/1613331918 <== mgr.15687 v2:172.21.15.47:6800/196172753 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+24 (secure 0 0 0) 0x7f33b003b4e0 con 0x7f33b0035a10 2024-03-21T10:25:21.080 INFO:teuthology.orchestra.run.smithi047.stdout:Schedule set for path / 2024-03-21T10:25:21.083 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:21.082+0000 7f33d1c00640 1 -- 172.21.15.47:0/1613331918 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f33b0035a10 msgr2=0x7f33b0035df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:25:21.083 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:21.082+0000 7f33d1c00640 1 --2- 172.21.15.47:0/1613331918 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f33b0035a10 0x7f33b0035df0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f33b4000c00 tx=0x7f33b4005fd0 comp rx=0 tx=0).stop 2024-03-21T10:25:21.083 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:21.082+0000 7f33d1c00640 1 -- 172.21.15.47:0/1613331918 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f33cc1038f0 msgr2=0x7f33cc199a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:25:21.083 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:21.082+0000 7f33d1c00640 1 --2- 172.21.15.47:0/1613331918 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f33cc1038f0 0x7f33cc199a90 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f33bc00d950 tx=0x7f33bc00de10 comp rx=0 tx=0).stop 2024-03-21T10:25:21.083 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:21.082+0000 7f33cb400640 1 -- 172.21.15.47:0/1613331918 reap_dead start 2024-03-21T10:25:21.083 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:21.082+0000 7f33d1c00640 1 -- 172.21.15.47:0/1613331918 shutdown_connections 2024-03-21T10:25:21.083 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:21.082+0000 7f33d1c00640 1 -- 172.21.15.47:0/1613331918 >> 172.21.15.47:0/1613331918 conn(0x7f33cc0fee60 msgr2=0x7f33cc107de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:25:21.083 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:21.082+0000 7f33d1c00640 1 -- 172.21.15.47:0/1613331918 shutdown_connections 2024-03-21T10:25:21.084 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:21.082+0000 7f33d1c00640 1 -- 172.21.15.47:0/1613331918 wait complete. 2024-03-21T10:25:21.102 DEBUG:teuthology.orchestra.run.smithi047:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule retention add --fs=cephfs --path=/ --retention-spec-or-period=6m3h' 2024-03-21T10:25:21.229 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Mar 21 10:25:20 smithi100 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-mgr-y[36578]: 2024-03-21T10:25:20.859+0000 7f2a4f872200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-03-21T10:25:21.301 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:21.299+0000 7f7f1f600640 1 -- 172.21.15.47:0/2596094364 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7f181028c0 msgr2=0x7f7f18102d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:25:21.301 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:21.299+0000 7f7f1f600640 1 --2- 172.21.15.47:0/2596094364 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7f181028c0 0x7f7f18102d40 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f7f04009900 tx=0x7f7f0402bc40 comp rx=0 tx=0).stop 2024-03-21T10:25:21.301 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:21.300+0000 7f7f1f600640 1 -- 172.21.15.47:0/2596094364 shutdown_connections 2024-03-21T10:25:21.301 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:21.300+0000 7f7f1f600640 1 --2- 172.21.15.47:0/2596094364 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7f18103280 0x7f7f18109b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:25:21.301 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:21.300+0000 7f7f1f600640 1 --2- 172.21.15.47:0/2596094364 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7f181028c0 0x7f7f18102d40 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:25:21.301 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:21.300+0000 7f7f1f600640 1 --2- 172.21.15.47:0/2596094364 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7f181016d0 0x7f7f18101af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:25:21.301 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:21.300+0000 7f7f1f600640 1 -- 172.21.15.47:0/2596094364 >> 172.21.15.47:0/2596094364 conn(0x7f7f180fcc60 msgr2=0x7f7f180ff080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:25:21.301 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:21.300+0000 7f7f1f600640 1 -- 172.21.15.47:0/2596094364 shutdown_connections 2024-03-21T10:25:21.301 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:21.300+0000 7f7f1f600640 1 -- 172.21.15.47:0/2596094364 wait complete. 2024-03-21T10:25:21.302 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:21.301+0000 7f7f1f600640 1 Processor -- start 2024-03-21T10:25:21.302 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:21.302+0000 7f7f1f600640 1 -- start start 2024-03-21T10:25:21.303 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:21.302+0000 7f7f1f600640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7f181016d0 0x7f7f18199a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:25:21.303 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:21.302+0000 7f7f1f600640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7f181028c0 0x7f7f18199fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:25:21.303 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:21.302+0000 7f7f1f600640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7f18103280 0x7f7f181a1030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:25:21.303 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:21.302+0000 7f7f1f600640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f1810d250 con 0x7f7f181028c0 2024-03-21T10:25:21.303 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:21.302+0000 7f7f1f600640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f1810d0d0 con 0x7f7f181016d0 2024-03-21T10:25:21.303 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:21.302+0000 7f7f1f600640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f1810d3d0 con 0x7f7f18103280 2024-03-21T10:25:21.304 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:21.302+0000 7f7f17e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7f181028c0 0x7f7f18199fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:25:21.304 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:21.302+0000 7f7f17e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7f181028c0 0x7f7f18199fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:43254/0 (socket says 172.21.15.47:43254) 2024-03-21T10:25:21.304 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:21.302+0000 7f7f17e00640 1 -- 172.21.15.47:0/2773831627 learned_addr learned my addr 172.21.15.47:0/2773831627 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:25:21.304 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:21.302+0000 7f7f1cc00640 1 --2- 172.21.15.47:0/2773831627 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7f181016d0 0x7f7f18199a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:25:21.304 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:21.303+0000 7f7f1d600640 1 --2- 172.21.15.47:0/2773831627 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7f18103280 0x7f7f181a1030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:25:21.305 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:21.303+0000 7f7f17e00640 1 -- 172.21.15.47:0/2773831627 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7f18103280 msgr2=0x7f7f181a1030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:25:21.305 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:21.303+0000 7f7f17e00640 1 --2- 172.21.15.47:0/2773831627 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7f18103280 0x7f7f181a1030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:25:21.305 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:21.303+0000 7f7f17e00640 1 -- 172.21.15.47:0/2773831627 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7f181016d0 msgr2=0x7f7f18199a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:25:21.305 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:21.303+0000 7f7f17e00640 1 --2- 172.21.15.47:0/2773831627 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7f181016d0 0x7f7f18199a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:25:21.305 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:21.303+0000 7f7f17e00640 1 -- 172.21.15.47:0/2773831627 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f181a16a0 con 0x7f7f181028c0 2024-03-21T10:25:21.305 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:21.303+0000 7f7f17e00640 1 --2- 172.21.15.47:0/2773831627 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7f181028c0 0x7f7f18199fc0 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f7f04000c00 tx=0x7f7f040040c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:25:21.305 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:21.304+0000 7f7f15600640 1 -- 172.21.15.47:0/2773831627 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f04043020 con 0x7f7f181028c0 2024-03-21T10:25:21.305 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:21.304+0000 7f7f15600640 1 -- 172.21.15.47:0/2773831627 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f7f040044c0 con 0x7f7f181028c0 2024-03-21T10:25:21.305 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:21.304+0000 7f7f1f600640 1 -- 172.21.15.47:0/2773831627 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7f181a1930 con 0x7f7f181028c0 2024-03-21T10:25:21.305 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:21.304+0000 7f7f15600640 1 -- 172.21.15.47:0/2773831627 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f04003c30 con 0x7f7f181028c0 2024-03-21T10:25:21.305 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:21.304+0000 7f7f1f600640 1 -- 172.21.15.47:0/2773831627 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7f181a1e80 con 0x7f7f181028c0 2024-03-21T10:25:21.306 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:21.305+0000 7f7f1f600640 1 -- 172.21.15.47:0/2773831627 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7f1804f400 con 0x7f7f181028c0 2024-03-21T10:25:21.309 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:21.309+0000 7f7f15600640 1 -- 172.21.15.47:0/2773831627 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 22) v1 ==== 137666+0+0 (secure 0 0 0) 0x7f7f04046020 con 0x7f7f181028c0 2024-03-21T10:25:21.313 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:21.313+0000 7f7f15600640 1 --2- 172.21.15.47:0/2773831627 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f7f000a2dd0 0x7f7f000a5290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:25:21.313 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:21.313+0000 7f7f15600640 1 -- 172.21.15.47:0/2773831627 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 9474+0+0 (secure 0 0 0) 0x7f7f040f1bb0 con 0x7f7f181028c0 2024-03-21T10:25:21.314 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:21.313+0000 7f7f1cc00640 1 --2- 172.21.15.47:0/2773831627 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f7f000a2dd0 0x7f7f000a5290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:25:21.314 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:21.313+0000 7f7f15600640 1 -- 172.21.15.47:0/2773831627 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f7f0404d020 con 0x7f7f181028c0 2024-03-21T10:25:21.314 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:21.314+0000 7f7f1cc00640 1 --2- 172.21.15.47:0/2773831627 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f7f000a2dd0 0x7f7f000a5290 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f7f18102720 tx=0x7f7f0c005f90 comp rx=0 tx=0).ready entity=mgr.15687 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:25:21.613 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:21.612+0000 7f7f1f600640 1 -- 172.21.15.47:0/2773831627 --> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] -- mgr_command(tid 0: {"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}) v1 -- 0x7f7f18075090 con 0x7f7f000a2dd0 2024-03-21T10:25:21.737 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:21.736+0000 7f7f15600640 1 -- 172.21.15.47:0/2773831627 <== mgr.15687 v2:172.21.15.47:6800/196172753 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+26 (secure 0 0 0) 0x7f7f18075090 con 0x7f7f000a2dd0 2024-03-21T10:25:21.737 INFO:teuthology.orchestra.run.smithi047.stdout:Retention added to path / 2024-03-21T10:25:21.740 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:21.739+0000 7f7f1f600640 1 -- 172.21.15.47:0/2773831627 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f7f000a2dd0 msgr2=0x7f7f000a5290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:25:21.740 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:21.739+0000 7f7f1f600640 1 --2- 172.21.15.47:0/2773831627 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f7f000a2dd0 0x7f7f000a5290 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f7f18102720 tx=0x7f7f0c005f90 comp rx=0 tx=0).stop 2024-03-21T10:25:21.740 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:21.739+0000 7f7f1f600640 1 -- 172.21.15.47:0/2773831627 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7f181028c0 msgr2=0x7f7f18199fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:25:21.740 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:21.739+0000 7f7f1f600640 1 --2- 172.21.15.47:0/2773831627 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7f181028c0 0x7f7f18199fc0 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f7f04000c00 tx=0x7f7f040040c0 comp rx=0 tx=0).stop 2024-03-21T10:25:21.740 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:21.739+0000 7f7f1f600640 1 -- 172.21.15.47:0/2773831627 shutdown_connections 2024-03-21T10:25:21.741 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:21.739+0000 7f7f1f600640 1 --2- 172.21.15.47:0/2773831627 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7f18103280 0x7f7f181a1030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:25:21.741 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:21.739+0000 7f7f1f600640 1 --2- 172.21.15.47:0/2773831627 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7f181028c0 0x7f7f18199fc0 unknown :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:25:21.741 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:21.739+0000 7f7f1f600640 1 --2- 172.21.15.47:0/2773831627 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f7f000a2dd0 0x7f7f000a5290 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:25:21.741 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:21.739+0000 7f7f1f600640 1 --2- 172.21.15.47:0/2773831627 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7f181016d0 0x7f7f18199a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:25:21.741 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:21.739+0000 7f7f1f600640 1 -- 172.21.15.47:0/2773831627 >> 172.21.15.47:0/2773831627 conn(0x7f7f180fcc60 msgr2=0x7f7f180738d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:25:21.741 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:21.739+0000 7f7f1f600640 1 -- 172.21.15.47:0/2773831627 shutdown_connections 2024-03-21T10:25:21.741 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:21.740+0000 7f7f1f600640 1 -- 172.21.15.47:0/2773831627 wait complete. 2024-03-21T10:25:21.758 DEBUG:teuthology.orchestra.run.smithi047:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule status --fs=cephfs --path=/' 2024-03-21T10:25:21.977 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:21.976+0000 7f4491600640 1 -- 172.21.15.47:0/2092945855 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f448c103290 msgr2=0x7f448c109b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:25:21.977 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:21.976+0000 7f4491600640 1 --2- 172.21.15.47:0/2092945855 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f448c103290 0x7f448c109b20 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f4484009ca0 tx=0x7f448402c0f0 comp rx=0 tx=0).stop 2024-03-21T10:25:21.978 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:21.976+0000 7f4491600640 1 -- 172.21.15.47:0/2092945855 shutdown_connections 2024-03-21T10:25:21.978 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:21.976+0000 7f4491600640 1 --2- 172.21.15.47:0/2092945855 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f448c103290 0x7f448c109b20 unknown :-1 s=CLOSED pgs=522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:25:21.978 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:21.976+0000 7f4491600640 1 --2- 172.21.15.47:0/2092945855 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f448c1028d0 0x7f448c102d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:25:21.978 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:21.976+0000 7f4491600640 1 --2- 172.21.15.47:0/2092945855 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f448c1016e0 0x7f448c101b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:25:21.978 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:21.976+0000 7f4491600640 1 -- 172.21.15.47:0/2092945855 >> 172.21.15.47:0/2092945855 conn(0x7f448c0fcc30 msgr2=0x7f448c0ff090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:25:21.978 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:21.976+0000 7f4491600640 1 -- 172.21.15.47:0/2092945855 shutdown_connections 2024-03-21T10:25:21.978 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:21.976+0000 7f4491600640 1 -- 172.21.15.47:0/2092945855 wait complete. 2024-03-21T10:25:21.980 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:21.977+0000 7f4491600640 1 Processor -- start 2024-03-21T10:25:21.980 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:21.977+0000 7f4491600640 1 -- start start 2024-03-21T10:25:21.980 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:21.978+0000 7f4491600640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f448c1016e0 0x7f448c199a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:25:21.980 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:21.978+0000 7f4491600640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f448c1028d0 0x7f448c199f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:25:21.980 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:21.978+0000 7f4491600640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f448c103290 0x7f448c1a0fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:25:21.980 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:21.978+0000 7f4491600640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f448c10d210 con 0x7f448c1028d0 2024-03-21T10:25:21.981 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:21.978+0000 7f4491600640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f448c10d090 con 0x7f448c1016e0 2024-03-21T10:25:21.981 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:21.978+0000 7f4491600640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f448c10d390 con 0x7f448c103290 2024-03-21T10:25:21.981 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:21.978+0000 7f448a000640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f448c1028d0 0x7f448c199f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:25:21.981 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:21.978+0000 7f448aa00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f448c1016e0 0x7f448c199a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:25:21.981 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:21.978+0000 7f448a000640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f448c1028d0 0x7f448c199f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:48596/0 (socket says 172.21.15.47:48596) 2024-03-21T10:25:21.981 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:21.978+0000 7f448a000640 1 -- 172.21.15.47:0/1128631044 learned_addr learned my addr 172.21.15.47:0/1128631044 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:25:21.981 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:21.978+0000 7f448b400640 1 --2- 172.21.15.47:0/1128631044 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f448c103290 0x7f448c1a0fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:25:21.981 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:21.979+0000 7f448a000640 1 -- 172.21.15.47:0/1128631044 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f448c103290 msgr2=0x7f448c1a0fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:25:21.981 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:21.979+0000 7f448a000640 1 --2- 172.21.15.47:0/1128631044 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f448c103290 0x7f448c1a0fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:25:21.981 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:21.979+0000 7f448a000640 1 -- 172.21.15.47:0/1128631044 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f448c1016e0 msgr2=0x7f448c199a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:25:21.981 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:21.979+0000 7f448a000640 1 --2- 172.21.15.47:0/1128631044 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f448c1016e0 0x7f448c199a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:25:21.981 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:21.979+0000 7f448a000640 1 -- 172.21.15.47:0/1128631044 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f448c1a1630 con 0x7f448c1028d0 2024-03-21T10:25:21.981 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:21.979+0000 7f448b400640 1 --2- 172.21.15.47:0/1128631044 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f448c103290 0x7f448c1a0fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T10:25:21.982 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:21.979+0000 7f448aa00640 1 --2- 172.21.15.47:0/1128631044 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f448c1016e0 0x7f448c199a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T10:25:21.982 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:21.979+0000 7f448a000640 1 --2- 172.21.15.47:0/1128631044 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f448c1028d0 0x7f448c199f50 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f447800e9f0 tx=0x7f447800edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:25:21.982 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:21.979+0000 7f4483400640 1 -- 172.21.15.47:0/1128631044 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f447800cd80 con 0x7f448c1028d0 2024-03-21T10:25:21.982 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:21.979+0000 7f4491600640 1 -- 172.21.15.47:0/1128631044 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f448c1a1920 con 0x7f448c1028d0 2024-03-21T10:25:21.982 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:21.979+0000 7f4483400640 1 -- 172.21.15.47:0/1128631044 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f44780054d0 con 0x7f448c1028d0 2024-03-21T10:25:21.982 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:21.979+0000 7f4483400640 1 -- 172.21.15.47:0/1128631044 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4478010490 con 0x7f448c1028d0 2024-03-21T10:25:21.982 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:21.979+0000 7f4491600640 1 -- 172.21.15.47:0/1128631044 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f448c1a1e20 con 0x7f448c1028d0 2024-03-21T10:25:21.982 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:21.981+0000 7f4491600640 1 -- 172.21.15.47:0/1128631044 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f448c1a20b0 con 0x7f448c1028d0 2024-03-21T10:25:21.983 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:21.982+0000 7f4483400640 1 -- 172.21.15.47:0/1128631044 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 23) v1 ==== 137974+0+0 (secure 0 0 0) 0x7f4478005020 con 0x7f448c1028d0 2024-03-21T10:25:21.984 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:21.984+0000 7f4483400640 1 --2- 172.21.15.47:0/1128631044 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f44700a3130 0x7f44700a55f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:25:21.984 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:21.984+0000 7f4483400640 1 -- 172.21.15.47:0/1128631044 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 9474+0+0 (secure 0 0 0) 0x7f44780cff80 con 0x7f448c1028d0 2024-03-21T10:25:21.985 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:21.984+0000 7f448aa00640 1 --2- 172.21.15.47:0/1128631044 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f44700a3130 0x7f44700a55f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:25:21.985 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:21.985+0000 7f448aa00640 1 --2- 172.21.15.47:0/1128631044 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f44700a3130 0x7f44700a55f0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f447c005fd0 tx=0x7f447c00b410 comp rx=0 tx=0).ready entity=mgr.15687 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:25:21.989 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:21.988+0000 7f4483400640 1 -- 172.21.15.47:0/1128631044 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f447809c650 con 0x7f448c1028d0 2024-03-21T10:25:22.006 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:25:21 smithi047 ceph-mon[31547]: [21/Mar/2024:10:25:20] ENGINE Bus STARTING 2024-03-21T10:25:22.006 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:25:21 smithi047 ceph-mon[31547]: [21/Mar/2024:10:25:20] ENGINE Serving on https://172.21.15.47:7150 2024-03-21T10:25:22.006 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:25:21 smithi047 ceph-mon[31547]: [21/Mar/2024:10:25:20] ENGINE Serving on http://172.21.15.47:8765 2024-03-21T10:25:22.006 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:25:21 smithi047 ceph-mon[31547]: [21/Mar/2024:10:25:20] ENGINE Bus STARTED 2024-03-21T10:25:22.006 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:25:21 smithi047 ceph-mon[31547]: mgrmap e22: x(active, since 1.03912s), standbys: z, y 2024-03-21T10:25:22.006 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:25:21 smithi047 ceph-mon[31547]: from='client.15705 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:25:22.006 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:25:21 smithi047 ceph-mon[31547]: pgmap v3: 129 pgs: 129 active+clean; 1.1 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-21T10:25:22.229 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Mar 21 10:25:21 smithi100 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-mgr-y[36578]: 2024-03-21T10:25:21.836+0000 7f2a4f872200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-03-21T10:25:22.229 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Mar 21 10:25:22 smithi100 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-mgr-y[36578]: 2024-03-21T10:25:22.019+0000 7f2a4f872200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-03-21T10:25:22.229 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:25:21 smithi100 ceph-mon[35193]: [21/Mar/2024:10:25:20] ENGINE Bus STARTING 2024-03-21T10:25:22.229 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:25:21 smithi100 ceph-mon[35193]: [21/Mar/2024:10:25:20] ENGINE Serving on https://172.21.15.47:7150 2024-03-21T10:25:22.230 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:25:21 smithi100 ceph-mon[35193]: [21/Mar/2024:10:25:20] ENGINE Serving on http://172.21.15.47:8765 2024-03-21T10:25:22.230 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:25:21 smithi100 ceph-mon[35193]: [21/Mar/2024:10:25:20] ENGINE Bus STARTED 2024-03-21T10:25:22.230 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:25:21 smithi100 ceph-mon[35193]: mgrmap e22: x(active, since 1.03912s), standbys: z, y 2024-03-21T10:25:22.230 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:25:21 smithi100 ceph-mon[35193]: from='client.15705 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:25:22.230 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:25:21 smithi100 ceph-mon[35193]: pgmap v3: 129 pgs: 129 active+clean; 1.1 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-21T10:25:22.234 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:22.234+0000 7f4491600640 1 -- 172.21.15.47:0/1128631044 --> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}) v1 -- 0x7f448c1a2280 con 0x7f44700a3130 2024-03-21T10:25:22.240 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:25:21 smithi157 ceph-mon[35581]: [21/Mar/2024:10:25:20] ENGINE Bus STARTING 2024-03-21T10:25:22.240 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:25:21 smithi157 ceph-mon[35581]: [21/Mar/2024:10:25:20] ENGINE Serving on https://172.21.15.47:7150 2024-03-21T10:25:22.240 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:25:21 smithi157 ceph-mon[35581]: [21/Mar/2024:10:25:20] ENGINE Serving on http://172.21.15.47:8765 2024-03-21T10:25:22.240 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:25:21 smithi157 ceph-mon[35581]: [21/Mar/2024:10:25:20] ENGINE Bus STARTED 2024-03-21T10:25:22.240 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:25:21 smithi157 ceph-mon[35581]: mgrmap e22: x(active, since 1.03912s), standbys: z, y 2024-03-21T10:25:22.240 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:25:21 smithi157 ceph-mon[35581]: from='client.15705 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:25:22.240 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:25:21 smithi157 ceph-mon[35581]: pgmap v3: 129 pgs: 129 active+clean; 1.1 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-21T10:25:22.250 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:22.250+0000 7f4483400640 1 -- 172.21.15.47:0/1128631044 <== mgr.15687 v2:172.21.15.47:6800/196172753 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+298 (secure 0 0 0) 0x7f448c1a2280 con 0x7f44700a3130 2024-03-21T10:25:22.251 INFO:teuthology.orchestra.run.smithi047.stdout:{"fs": "cephfs", "subvol": null, "group": null, "path": "/", "rel_path": "/", "schedule": "1m", "retention": {"m": 6, "h": 3}, "start": "2024-03-21T00:00:00", "created": "2024-03-21T10:25:20", "first": null, "last": null, "last_pruned": null, "created_count": 0, "pruned_count": 0, "active": true} 2024-03-21T10:25:22.253 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:22.252+0000 7f4491600640 1 -- 172.21.15.47:0/1128631044 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f44700a3130 msgr2=0x7f44700a55f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:25:22.253 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:22.252+0000 7f4491600640 1 --2- 172.21.15.47:0/1128631044 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f44700a3130 0x7f44700a55f0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f447c005fd0 tx=0x7f447c00b410 comp rx=0 tx=0).stop 2024-03-21T10:25:22.253 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:22.252+0000 7f4491600640 1 -- 172.21.15.47:0/1128631044 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f448c1028d0 msgr2=0x7f448c199f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:25:22.254 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:22.252+0000 7f4491600640 1 --2- 172.21.15.47:0/1128631044 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f448c1028d0 0x7f448c199f50 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f447800e9f0 tx=0x7f447800edf0 comp rx=0 tx=0).stop 2024-03-21T10:25:22.254 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:22.252+0000 7f4491600640 1 -- 172.21.15.47:0/1128631044 shutdown_connections 2024-03-21T10:25:22.254 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:22.252+0000 7f4491600640 1 --2- 172.21.15.47:0/1128631044 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f448c103290 0x7f448c1a0fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:25:22.254 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:22.252+0000 7f4491600640 1 --2- 172.21.15.47:0/1128631044 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f448c1028d0 0x7f448c199f50 unknown :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:25:22.254 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:22.252+0000 7f4491600640 1 --2- 172.21.15.47:0/1128631044 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f44700a3130 0x7f44700a55f0 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:25:22.254 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:22.252+0000 7f4491600640 1 --2- 172.21.15.47:0/1128631044 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f448c1016e0 0x7f448c199a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:25:22.254 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:22.252+0000 7f4491600640 1 -- 172.21.15.47:0/1128631044 >> 172.21.15.47:0/1128631044 conn(0x7f448c0fcc30 msgr2=0x7f448c0738d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:25:22.254 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:22.253+0000 7f4491600640 1 -- 172.21.15.47:0/1128631044 shutdown_connections 2024-03-21T10:25:22.254 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:22.253+0000 7f4491600640 1 -- 172.21.15.47:0/1128631044 wait complete. 2024-03-21T10:25:22.272 DEBUG:teuthology.orchestra.run.smithi047:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true' 2024-03-21T10:25:22.475 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:22.474+0000 7f7aad400640 1 -- 172.21.15.47:0/2936317694 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7aa81016f0 msgr2=0x7f7aa8101b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:25:22.475 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:22.474+0000 7f7aad400640 1 --2- 172.21.15.47:0/2936317694 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7aa81016f0 0x7f7aa8101b10 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f7a98009900 tx=0x7f7a9802bc30 comp rx=0 tx=0).stop 2024-03-21T10:25:22.476 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:22.474+0000 7f7aad400640 1 -- 172.21.15.47:0/2936317694 shutdown_connections 2024-03-21T10:25:22.476 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:22.474+0000 7f7aad400640 1 --2- 172.21.15.47:0/2936317694 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7aa81032a0 0x7f7aa8109b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:25:22.476 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:22.474+0000 7f7aad400640 1 --2- 172.21.15.47:0/2936317694 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7aa81028e0 0x7f7aa8102d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:25:22.476 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:22.474+0000 7f7aad400640 1 --2- 172.21.15.47:0/2936317694 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7aa81016f0 0x7f7aa8101b10 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:25:22.476 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:22.474+0000 7f7aad400640 1 -- 172.21.15.47:0/2936317694 >> 172.21.15.47:0/2936317694 conn(0x7f7aa80fcc60 msgr2=0x7f7aa80ff0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:25:22.476 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:22.474+0000 7f7aad400640 1 -- 172.21.15.47:0/2936317694 shutdown_connections 2024-03-21T10:25:22.476 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:22.474+0000 7f7aad400640 1 -- 172.21.15.47:0/2936317694 wait complete. 2024-03-21T10:25:22.476 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:22.476+0000 7f7aad400640 1 Processor -- start 2024-03-21T10:25:22.477 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:22.476+0000 7f7aad400640 1 -- start start 2024-03-21T10:25:22.478 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:22.476+0000 7f7aad400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7aa81016f0 0x7f7aa8199ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:25:22.478 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:22.476+0000 7f7aad400640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7aa81028e0 0x7f7aa819a010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:25:22.478 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:22.476+0000 7f7aad400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7aa81032a0 0x7f7aa81a1080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:25:22.478 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:22.476+0000 7f7aad400640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7aa810d1f0 con 0x7f7aa81016f0 2024-03-21T10:25:22.478 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:22.476+0000 7f7aad400640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7aa810d070 con 0x7f7aa81032a0 2024-03-21T10:25:22.478 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:22.476+0000 7f7aad400640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7aa810d370 con 0x7f7aa81028e0 2024-03-21T10:25:22.478 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:22.477+0000 7f7aa6800640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7aa81016f0 0x7f7aa8199ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:25:22.478 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:22.477+0000 7f7aa6800640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7aa81016f0 0x7f7aa8199ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:48630/0 (socket says 172.21.15.47:48630) 2024-03-21T10:25:22.478 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:22.477+0000 7f7aa6800640 1 -- 172.21.15.47:0/4193197468 learned_addr learned my addr 172.21.15.47:0/4193197468 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:25:22.478 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:22.477+0000 7f7aa5e00640 1 --2- 172.21.15.47:0/4193197468 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7aa81028e0 0x7f7aa819a010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:25:22.479 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:22.477+0000 7f7aa7200640 1 --2- 172.21.15.47:0/4193197468 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7aa81032a0 0x7f7aa81a1080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:25:22.479 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:22.477+0000 7f7aa6800640 1 -- 172.21.15.47:0/4193197468 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7aa81028e0 msgr2=0x7f7aa819a010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:25:22.479 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:22.477+0000 7f7aa6800640 1 --2- 172.21.15.47:0/4193197468 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7aa81028e0 0x7f7aa819a010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:25:22.479 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:22.477+0000 7f7aa6800640 1 -- 172.21.15.47:0/4193197468 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7aa81032a0 msgr2=0x7f7aa81a1080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:25:22.479 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:22.477+0000 7f7aa6800640 1 --2- 172.21.15.47:0/4193197468 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7aa81032a0 0x7f7aa81a1080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:25:22.479 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:22.477+0000 7f7aa6800640 1 -- 172.21.15.47:0/4193197468 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7aa81a16f0 con 0x7f7aa81016f0 2024-03-21T10:25:22.483 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:22.478+0000 7f7aa6800640 1 --2- 172.21.15.47:0/4193197468 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7aa81016f0 0x7f7aa8199ad0 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f7a98000c00 tx=0x7f7a98005dc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:25:22.483 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:22.478+0000 7f7a9f400640 1 -- 172.21.15.47:0/4193197468 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a98043020 con 0x7f7aa81016f0 2024-03-21T10:25:22.483 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:22.478+0000 7f7aad400640 1 -- 172.21.15.47:0/4193197468 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7aa81a1980 con 0x7f7aa81016f0 2024-03-21T10:25:22.484 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:22.478+0000 7f7a9f400640 1 -- 172.21.15.47:0/4193197468 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f7a98004350 con 0x7f7aa81016f0 2024-03-21T10:25:22.484 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:22.478+0000 7f7a9f400640 1 -- 172.21.15.47:0/4193197468 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a98003b80 con 0x7f7aa81016f0 2024-03-21T10:25:22.484 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:22.478+0000 7f7aad400640 1 -- 172.21.15.47:0/4193197468 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7aa81a1f20 con 0x7f7aa81016f0 2024-03-21T10:25:22.484 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:22.480+0000 7f7aad400640 1 -- 172.21.15.47:0/4193197468 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7aa81a21b0 con 0x7f7aa81016f0 2024-03-21T10:25:22.484 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:22.481+0000 7f7a9f400640 1 -- 172.21.15.47:0/4193197468 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 23) v1 ==== 137974+0+0 (secure 0 0 0) 0x7f7a98046020 con 0x7f7aa81016f0 2024-03-21T10:25:22.484 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:22.483+0000 7f7a9f400640 1 --2- 172.21.15.47:0/4193197468 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f7a8c0a3130 0x7f7a8c0a55f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:25:22.484 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:22.483+0000 7f7a9f400640 1 -- 172.21.15.47:0/4193197468 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 9474+0+0 (secure 0 0 0) 0x7f7a980f1bd0 con 0x7f7aa81016f0 2024-03-21T10:25:22.485 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:22.483+0000 7f7aa5e00640 1 --2- 172.21.15.47:0/4193197468 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f7a8c0a3130 0x7f7a8c0a55f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:25:22.485 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:22.484+0000 7f7aa5e00640 1 --2- 172.21.15.47:0/4193197468 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f7a8c0a3130 0x7f7a8c0a55f0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f7aa819afe0 tx=0x7f7a94005f90 comp rx=0 tx=0).ready entity=mgr.15687 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:25:22.488 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:22.488+0000 7f7a9f400640 1 -- 172.21.15.47:0/4193197468 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f7a9804d020 con 0x7f7aa81016f0 2024-03-21T10:25:22.729 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Mar 21 10:25:22 smithi100 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-mgr-y[36578]: 2024-03-21T10:25:22.435+0000 7f2a4f872200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-03-21T10:25:22.798 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:22.797+0000 7f7aad400640 1 -- 172.21.15.47:0/4193197468 --> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] -- mgr_command(tid 0: {"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}) v1 -- 0x7f7aa8074ff0 con 0x7f7a8c0a3130 2024-03-21T10:25:22.812 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:22.811+0000 7f7a9f400640 1 -- 172.21.15.47:0/4193197468 <== mgr.15687 v2:172.21.15.47:6800/196172753 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+10 (secure 0 0 0) 0x7f7aa8074ff0 con 0x7f7a8c0a3130 2024-03-21T10:25:22.812 INFO:teuthology.orchestra.run.smithi047.stdout:/ 1m 6m3h 2024-03-21T10:25:22.816 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:22.815+0000 7f7aad400640 1 -- 172.21.15.47:0/4193197468 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f7a8c0a3130 msgr2=0x7f7a8c0a55f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:25:22.816 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:22.815+0000 7f7aad400640 1 --2- 172.21.15.47:0/4193197468 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f7a8c0a3130 0x7f7a8c0a55f0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f7aa819afe0 tx=0x7f7a94005f90 comp rx=0 tx=0).stop 2024-03-21T10:25:22.816 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:22.815+0000 7f7aad400640 1 -- 172.21.15.47:0/4193197468 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7aa81016f0 msgr2=0x7f7aa8199ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:25:22.816 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:22.815+0000 7f7aad400640 1 --2- 172.21.15.47:0/4193197468 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7aa81016f0 0x7f7aa8199ad0 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f7a98000c00 tx=0x7f7a98005dc0 comp rx=0 tx=0).stop 2024-03-21T10:25:22.817 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:22.815+0000 7f7aad400640 1 -- 172.21.15.47:0/4193197468 shutdown_connections 2024-03-21T10:25:22.817 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:22.815+0000 7f7aad400640 1 --2- 172.21.15.47:0/4193197468 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f7a8c0a3130 0x7f7a8c0a55f0 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:25:22.817 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:22.815+0000 7f7aad400640 1 --2- 172.21.15.47:0/4193197468 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7aa81032a0 0x7f7aa81a1080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:25:22.817 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:22.815+0000 7f7aad400640 1 --2- 172.21.15.47:0/4193197468 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7aa81028e0 0x7f7aa819a010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:25:22.817 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:22.815+0000 7f7aad400640 1 --2- 172.21.15.47:0/4193197468 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7aa81016f0 0x7f7aa8199ad0 unknown :-1 s=CLOSED pgs=526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:25:22.817 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:22.815+0000 7f7aad400640 1 -- 172.21.15.47:0/4193197468 >> 172.21.15.47:0/4193197468 conn(0x7f7aa80fcc60 msgr2=0x7f7aa80738d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:25:22.817 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:22.815+0000 7f7aad400640 1 -- 172.21.15.47:0/4193197468 shutdown_connections 2024-03-21T10:25:22.817 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:22.816+0000 7f7aad400640 1 -- 172.21.15.47:0/4193197468 wait complete. 2024-03-21T10:25:22.843 DEBUG:teuthology.orchestra.run.smithi047:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'date +%s > START_TIME' 2024-03-21T10:25:22.922 INFO:teuthology.run_tasks:Running task full_sequential_finally... 2024-03-21T10:25:22.932 INFO:teuthology.run_tasks:Running task background_exec... 2024-03-21T10:25:22.942 INFO:teuthology.task.background_exec:Running background command on role mon.a host ubuntu@smithi047.front.sepia.ceph.com 2024-03-21T10:25:22.942 DEBUG:teuthology.orchestra.run.smithi047:> sudo TESTDIR=/home/ubuntu/cephtest daemon-helper kill --kill-group bash -c 'while sleep 13; do ceph tell mds.cephfs:0 flush journal; done' 2024-03-21T10:25:22.947 INFO:teuthology.run_tasks:Running task workunit... 2024-03-21T10:25:22.959 INFO:tasks.workunit:Pulling workunits from ref 2edfee68ad7243bed422a049b6f5f58619300f11 2024-03-21T10:25:22.960 INFO:tasks.workunit:Making a separate scratch dir for every client... 2024-03-21T10:25:22.960 INFO:tasks.workunit:timeout=3h 2024-03-21T10:25:22.960 INFO:tasks.workunit:cleanup=True 2024-03-21T10:25:22.960 DEBUG:teuthology.orchestra.run.smithi047:> stat -- /home/ubuntu/cephtest/mnt.0 2024-03-21T10:25:23.009 INFO:teuthology.orchestra.run.smithi047.stdout: File: /home/ubuntu/cephtest/mnt.0 2024-03-21T10:25:23.010 INFO:teuthology.orchestra.run.smithi047.stdout: Size: 0 Blocks: 1 IO Block: 4194304 directory 2024-03-21T10:25:23.010 INFO:teuthology.orchestra.run.smithi047.stdout:Device: 76h/118d Inode: 1 Links: 2 2024-03-21T10:25:23.010 INFO:teuthology.orchestra.run.smithi047.stdout:Access: (1777/drwxrwxrwt) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-21T10:25:23.010 INFO:teuthology.orchestra.run.smithi047.stdout:Context: system_u:object_r:fusefs_t:s0 2024-03-21T10:25:23.010 INFO:teuthology.orchestra.run.smithi047.stdout:Access: 2024-03-21 10:22:28.748979164 +0000 2024-03-21T10:25:23.010 INFO:teuthology.orchestra.run.smithi047.stdout:Modify: 2024-03-21 10:22:28.748979164 +0000 2024-03-21T10:25:23.010 INFO:teuthology.orchestra.run.smithi047.stdout:Change: 2024-03-21 10:24:58.261126581 +0000 2024-03-21T10:25:23.010 INFO:teuthology.orchestra.run.smithi047.stdout: Birth: - 2024-03-21T10:25:23.010 INFO:tasks.workunit:Did not need to create dir /home/ubuntu/cephtest/mnt.0 2024-03-21T10:25:23.010 DEBUG:teuthology.orchestra.run.smithi047:> cd -- /home/ubuntu/cephtest/mnt.0 && sudo install -d -m 0755 --owner=ubuntu -- client.0 2024-03-21T10:25:23.119 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:25:22 smithi047 ceph-mon[31547]: from='client.15747 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:25:23.119 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:25:22 smithi047 ceph-mon[31547]: pgmap v4: 129 pgs: 129 active+clean; 1.1 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-21T10:25:23.119 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:25:22 smithi047 ceph-mon[31547]: mgrmap e23: x(active, since 2s), standbys: z, y 2024-03-21T10:25:23.119 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:25:22 smithi047 ceph-mon[31547]: from='client.15753 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:25:23.119 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:25:22 smithi047 ceph-mon[31547]: Standby manager daemon y restarted 2024-03-21T10:25:23.119 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:25:22 smithi047 ceph-mon[31547]: Standby manager daemon y started 2024-03-21T10:25:23.119 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:25:22 smithi047 ceph-mon[31547]: from='mgr.? 172.21.15.100:0/72489826' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-03-21T10:25:23.119 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:25:22 smithi047 ceph-mon[31547]: from='mgr.? 172.21.15.100:0/72489826' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-03-21T10:25:23.120 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:25:22 smithi047 ceph-mon[31547]: from='mgr.? 172.21.15.100:0/72489826' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-03-21T10:25:23.120 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:25:22 smithi047 ceph-mon[31547]: from='mgr.? 172.21.15.100:0/72489826' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-03-21T10:25:23.141 DEBUG:teuthology.orchestra.run.smithi047:> rm -rf /home/ubuntu/cephtest/clone.client.0 && git clone https://git.ceph.com/ceph-ci.git /home/ubuntu/cephtest/clone.client.0 && cd /home/ubuntu/cephtest/clone.client.0 && git checkout 2edfee68ad7243bed422a049b6f5f58619300f11 2024-03-21T10:25:23.186 INFO:tasks.workunit.client.0.smithi047.stderr:Cloning into '/home/ubuntu/cephtest/clone.client.0'... 2024-03-21T10:25:23.229 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:25:22 smithi100 ceph-mon[35193]: from='client.15747 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:25:23.229 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:25:22 smithi100 ceph-mon[35193]: pgmap v4: 129 pgs: 129 active+clean; 1.1 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-21T10:25:23.229 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:25:22 smithi100 ceph-mon[35193]: mgrmap e23: x(active, since 2s), standbys: z, y 2024-03-21T10:25:23.229 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:25:22 smithi100 ceph-mon[35193]: from='client.15753 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:25:23.230 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:25:22 smithi100 ceph-mon[35193]: Standby manager daemon y restarted 2024-03-21T10:25:23.230 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:25:22 smithi100 ceph-mon[35193]: Standby manager daemon y started 2024-03-21T10:25:23.230 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:25:22 smithi100 ceph-mon[35193]: from='mgr.? 172.21.15.100:0/72489826' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-03-21T10:25:23.230 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:25:22 smithi100 ceph-mon[35193]: from='mgr.? 172.21.15.100:0/72489826' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-03-21T10:25:23.230 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:25:22 smithi100 ceph-mon[35193]: from='mgr.? 172.21.15.100:0/72489826' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-03-21T10:25:23.230 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:25:22 smithi100 ceph-mon[35193]: from='mgr.? 172.21.15.100:0/72489826' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-03-21T10:25:23.239 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:25:22 smithi157 ceph-mon[35581]: from='client.15747 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:25:23.239 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:25:22 smithi157 ceph-mon[35581]: pgmap v4: 129 pgs: 129 active+clean; 1.1 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-21T10:25:23.239 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:25:22 smithi157 ceph-mon[35581]: mgrmap e23: x(active, since 2s), standbys: z, y 2024-03-21T10:25:23.239 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:25:22 smithi157 ceph-mon[35581]: from='client.15753 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:25:23.239 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:25:22 smithi157 ceph-mon[35581]: Standby manager daemon y restarted 2024-03-21T10:25:23.239 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:25:22 smithi157 ceph-mon[35581]: Standby manager daemon y started 2024-03-21T10:25:23.240 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:25:22 smithi157 ceph-mon[35581]: from='mgr.? 172.21.15.100:0/72489826' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-03-21T10:25:23.240 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:25:22 smithi157 ceph-mon[35581]: from='mgr.? 172.21.15.100:0/72489826' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-03-21T10:25:23.240 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:25:22 smithi157 ceph-mon[35581]: from='mgr.? 172.21.15.100:0/72489826' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-03-21T10:25:23.240 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:25:22 smithi157 ceph-mon[35581]: from='mgr.? 172.21.15.100:0/72489826' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-03-21T10:25:24.213 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:25:23 smithi047 ceph-mon[31547]: from='client.15765 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:25:24.214 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:25:23 smithi047 ceph-mon[31547]: mgrmap e24: x(active, since 3s), standbys: y, z 2024-03-21T10:25:24.214 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:25:23 smithi047 ceph-mon[31547]: from='mgr.15687 172.21.15.47:0/8730661' entity='mgr.x' 2024-03-21T10:25:24.214 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:25:23 smithi047 ceph-mon[31547]: from='mgr.15687 172.21.15.47:0/8730661' entity='mgr.x' 2024-03-21T10:25:24.214 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:25:23 smithi047 ceph-mon[31547]: from='mgr.15687 172.21.15.47:0/8730661' entity='mgr.x' 2024-03-21T10:25:24.214 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:25:23 smithi047 ceph-mon[31547]: from='mgr.15687 172.21.15.47:0/8730661' entity='mgr.x' 2024-03-21T10:25:24.214 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:25:23 smithi047 ceph-mon[31547]: from='mgr.15687 172.21.15.47:0/8730661' entity='mgr.x' 2024-03-21T10:25:24.214 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:25:23 smithi047 ceph-mon[31547]: from='mgr.15687 172.21.15.47:0/8730661' entity='mgr.x' 2024-03-21T10:25:24.214 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:25:23 smithi047 ceph-mon[31547]: scrub summary: idle 2024-03-21T10:25:24.229 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:25:23 smithi100 ceph-mon[35193]: from='client.15765 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:25:24.229 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:25:23 smithi100 ceph-mon[35193]: mgrmap e24: x(active, since 3s), standbys: y, z 2024-03-21T10:25:24.229 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:25:23 smithi100 ceph-mon[35193]: from='mgr.15687 172.21.15.47:0/8730661' entity='mgr.x' 2024-03-21T10:25:24.229 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:25:23 smithi100 ceph-mon[35193]: from='mgr.15687 172.21.15.47:0/8730661' entity='mgr.x' 2024-03-21T10:25:24.229 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:25:23 smithi100 ceph-mon[35193]: from='mgr.15687 172.21.15.47:0/8730661' entity='mgr.x' 2024-03-21T10:25:24.229 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:25:23 smithi100 ceph-mon[35193]: from='mgr.15687 172.21.15.47:0/8730661' entity='mgr.x' 2024-03-21T10:25:24.229 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:25:23 smithi100 ceph-mon[35193]: from='mgr.15687 172.21.15.47:0/8730661' entity='mgr.x' 2024-03-21T10:25:24.229 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:25:23 smithi100 ceph-mon[35193]: from='mgr.15687 172.21.15.47:0/8730661' entity='mgr.x' 2024-03-21T10:25:24.229 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:25:23 smithi100 ceph-mon[35193]: scrub summary: idle 2024-03-21T10:25:24.239 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:25:23 smithi157 ceph-mon[35581]: from='client.15765 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:25:24.240 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:25:23 smithi157 ceph-mon[35581]: mgrmap e24: x(active, since 3s), standbys: y, z 2024-03-21T10:25:24.240 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:25:23 smithi157 ceph-mon[35581]: from='mgr.15687 172.21.15.47:0/8730661' entity='mgr.x' 2024-03-21T10:25:24.240 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:25:23 smithi157 ceph-mon[35581]: from='mgr.15687 172.21.15.47:0/8730661' entity='mgr.x' 2024-03-21T10:25:24.240 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:25:23 smithi157 ceph-mon[35581]: from='mgr.15687 172.21.15.47:0/8730661' entity='mgr.x' 2024-03-21T10:25:24.240 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:25:23 smithi157 ceph-mon[35581]: from='mgr.15687 172.21.15.47:0/8730661' entity='mgr.x' 2024-03-21T10:25:24.240 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:25:23 smithi157 ceph-mon[35581]: from='mgr.15687 172.21.15.47:0/8730661' entity='mgr.x' 2024-03-21T10:25:24.240 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:25:23 smithi157 ceph-mon[35581]: from='mgr.15687 172.21.15.47:0/8730661' entity='mgr.x' 2024-03-21T10:25:24.240 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:25:23 smithi157 ceph-mon[35581]: scrub summary: idle 2024-03-21T10:25:25.963 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:25:25 smithi047 ceph-mon[31547]: pgmap v5: 129 pgs: 129 active+clean; 1.1 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-21T10:25:25.964 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:25:25 smithi047 ceph-mon[31547]: mgrmap e25: x(active, since 4s), standbys: y, z 2024-03-21T10:25:25.964 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:25:25 smithi047 ceph-mon[31547]: from='mgr.15687 172.21.15.47:0/8730661' entity='mgr.x' 2024-03-21T10:25:25.964 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:25:25 smithi047 ceph-mon[31547]: from='mgr.15687 172.21.15.47:0/8730661' entity='mgr.x' 2024-03-21T10:25:25.964 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:25:25 smithi047 ceph-mon[31547]: from='mgr.15687 172.21.15.47:0/8730661' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-21T10:25:25.964 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:25:25 smithi047 ceph-mon[31547]: from='mgr.15687 172.21.15.47:0/8730661' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-21T10:25:25.964 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:25:25 smithi047 ceph-mon[31547]: from='mgr.15687 172.21.15.47:0/8730661' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-03-21T10:25:25.964 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:25:25 smithi047 ceph-mon[31547]: from='mgr.15687 172.21.15.47:0/8730661' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-03-21T10:25:25.964 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:25:25 smithi047 ceph-mon[31547]: from='mgr.15687 172.21.15.47:0/8730661' entity='mgr.x' 2024-03-21T10:25:25.964 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:25:25 smithi047 ceph-mon[31547]: from='mgr.15687 172.21.15.47:0/8730661' entity='mgr.x' 2024-03-21T10:25:25.964 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:25:25 smithi047 ceph-mon[31547]: from='mgr.15687 172.21.15.47:0/8730661' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-21T10:25:25.964 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:25:25 smithi047 ceph-mon[31547]: from='mgr.15687 172.21.15.47:0/8730661' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-21T10:25:25.964 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:25:25 smithi047 ceph-mon[31547]: from='mgr.15687 172.21.15.47:0/8730661' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-21T10:25:25.964 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:25:25 smithi047 ceph-mon[31547]: from='mgr.15687 172.21.15.47:0/8730661' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-03-21T10:25:25.979 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:25:25 smithi100 ceph-mon[35193]: pgmap v5: 129 pgs: 129 active+clean; 1.1 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-21T10:25:25.979 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:25:25 smithi100 ceph-mon[35193]: mgrmap e25: x(active, since 4s), standbys: y, z 2024-03-21T10:25:25.979 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:25:25 smithi100 ceph-mon[35193]: from='mgr.15687 172.21.15.47:0/8730661' entity='mgr.x' 2024-03-21T10:25:25.979 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:25:25 smithi100 ceph-mon[35193]: from='mgr.15687 172.21.15.47:0/8730661' entity='mgr.x' 2024-03-21T10:25:25.980 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:25:25 smithi100 ceph-mon[35193]: from='mgr.15687 172.21.15.47:0/8730661' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-21T10:25:25.980 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:25:25 smithi100 ceph-mon[35193]: from='mgr.15687 172.21.15.47:0/8730661' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-21T10:25:25.980 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:25:25 smithi100 ceph-mon[35193]: from='mgr.15687 172.21.15.47:0/8730661' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-03-21T10:25:25.980 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:25:25 smithi100 ceph-mon[35193]: from='mgr.15687 172.21.15.47:0/8730661' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-03-21T10:25:25.980 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:25:25 smithi100 ceph-mon[35193]: from='mgr.15687 172.21.15.47:0/8730661' entity='mgr.x' 2024-03-21T10:25:25.980 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:25:25 smithi100 ceph-mon[35193]: from='mgr.15687 172.21.15.47:0/8730661' entity='mgr.x' 2024-03-21T10:25:25.980 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:25:25 smithi100 ceph-mon[35193]: from='mgr.15687 172.21.15.47:0/8730661' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-21T10:25:25.980 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:25:25 smithi100 ceph-mon[35193]: from='mgr.15687 172.21.15.47:0/8730661' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-21T10:25:25.980 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:25:25 smithi100 ceph-mon[35193]: from='mgr.15687 172.21.15.47:0/8730661' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-21T10:25:25.980 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:25:25 smithi100 ceph-mon[35193]: from='mgr.15687 172.21.15.47:0/8730661' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-03-21T10:25:25.989 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:25:25 smithi157 ceph-mon[35581]: pgmap v5: 129 pgs: 129 active+clean; 1.1 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-21T10:25:25.989 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:25:25 smithi157 ceph-mon[35581]: mgrmap e25: x(active, since 4s), standbys: y, z 2024-03-21T10:25:25.989 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:25:25 smithi157 ceph-mon[35581]: from='mgr.15687 172.21.15.47:0/8730661' entity='mgr.x' 2024-03-21T10:25:25.989 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:25:25 smithi157 ceph-mon[35581]: from='mgr.15687 172.21.15.47:0/8730661' entity='mgr.x' 2024-03-21T10:25:25.989 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:25:25 smithi157 ceph-mon[35581]: from='mgr.15687 172.21.15.47:0/8730661' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-21T10:25:25.989 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:25:25 smithi157 ceph-mon[35581]: from='mgr.15687 172.21.15.47:0/8730661' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-21T10:25:25.989 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:25:25 smithi157 ceph-mon[35581]: from='mgr.15687 172.21.15.47:0/8730661' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-03-21T10:25:25.989 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:25:25 smithi157 ceph-mon[35581]: from='mgr.15687 172.21.15.47:0/8730661' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-03-21T10:25:25.990 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:25:25 smithi157 ceph-mon[35581]: from='mgr.15687 172.21.15.47:0/8730661' entity='mgr.x' 2024-03-21T10:25:25.990 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:25:25 smithi157 ceph-mon[35581]: from='mgr.15687 172.21.15.47:0/8730661' entity='mgr.x' 2024-03-21T10:25:25.990 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:25:25 smithi157 ceph-mon[35581]: from='mgr.15687 172.21.15.47:0/8730661' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-21T10:25:25.990 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:25:25 smithi157 ceph-mon[35581]: from='mgr.15687 172.21.15.47:0/8730661' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-21T10:25:25.990 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:25:25 smithi157 ceph-mon[35581]: from='mgr.15687 172.21.15.47:0/8730661' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-21T10:25:25.990 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:25:25 smithi157 ceph-mon[35581]: from='mgr.15687 172.21.15.47:0/8730661' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-03-21T10:25:26.963 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:25:26 smithi047 ceph-mon[31547]: Adjusting osd_memory_target on smithi157 to 176.8M 2024-03-21T10:25:26.963 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:25:26 smithi047 ceph-mon[31547]: Unable to set osd_memory_target on smithi157 to 185389568: error parsing value: Value '185389568' is below minimum 939524096 2024-03-21T10:25:26.963 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:25:26 smithi047 ceph-mon[31547]: Adjusting osd_memory_target on smithi047 to 177.8M 2024-03-21T10:25:26.963 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:25:26 smithi047 ceph-mon[31547]: Unable to set osd_memory_target on smithi047 to 186497740: error parsing value: Value '186497740' is below minimum 939524096 2024-03-21T10:25:26.964 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:25:26 smithi047 ceph-mon[31547]: from='mgr.15687 172.21.15.47:0/8730661' entity='mgr.x' 2024-03-21T10:25:26.964 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:25:26 smithi047 ceph-mon[31547]: from='mgr.15687 172.21.15.47:0/8730661' entity='mgr.x' 2024-03-21T10:25:26.964 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:25:26 smithi047 ceph-mon[31547]: from='mgr.15687 172.21.15.47:0/8730661' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-21T10:25:26.964 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:25:26 smithi047 ceph-mon[31547]: from='mgr.15687 172.21.15.47:0/8730661' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-21T10:25:26.964 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:25:26 smithi047 ceph-mon[31547]: from='mgr.15687 172.21.15.47:0/8730661' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-21T10:25:26.964 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:25:26 smithi047 ceph-mon[31547]: from='mgr.15687 172.21.15.47:0/8730661' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-03-21T10:25:26.964 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:25:26 smithi047 ceph-mon[31547]: from='mgr.15687 172.21.15.47:0/8730661' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T10:25:26.964 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:25:26 smithi047 ceph-mon[31547]: from='mgr.15687 172.21.15.47:0/8730661' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T10:25:26.980 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:25:26 smithi100 ceph-mon[35193]: Adjusting osd_memory_target on smithi157 to 176.8M 2024-03-21T10:25:26.980 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:25:26 smithi100 ceph-mon[35193]: Unable to set osd_memory_target on smithi157 to 185389568: error parsing value: Value '185389568' is below minimum 939524096 2024-03-21T10:25:26.980 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:25:26 smithi100 ceph-mon[35193]: Adjusting osd_memory_target on smithi047 to 177.8M 2024-03-21T10:25:26.980 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:25:26 smithi100 ceph-mon[35193]: Unable to set osd_memory_target on smithi047 to 186497740: error parsing value: Value '186497740' is below minimum 939524096 2024-03-21T10:25:26.980 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:25:26 smithi100 ceph-mon[35193]: from='mgr.15687 172.21.15.47:0/8730661' entity='mgr.x' 2024-03-21T10:25:26.980 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:25:26 smithi100 ceph-mon[35193]: from='mgr.15687 172.21.15.47:0/8730661' entity='mgr.x' 2024-03-21T10:25:26.980 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:25:26 smithi100 ceph-mon[35193]: from='mgr.15687 172.21.15.47:0/8730661' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-21T10:25:26.980 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:25:26 smithi100 ceph-mon[35193]: from='mgr.15687 172.21.15.47:0/8730661' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-21T10:25:26.980 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:25:26 smithi100 ceph-mon[35193]: from='mgr.15687 172.21.15.47:0/8730661' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-21T10:25:26.980 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:25:26 smithi100 ceph-mon[35193]: from='mgr.15687 172.21.15.47:0/8730661' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-03-21T10:25:26.980 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:25:26 smithi100 ceph-mon[35193]: from='mgr.15687 172.21.15.47:0/8730661' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T10:25:26.980 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:25:26 smithi100 ceph-mon[35193]: from='mgr.15687 172.21.15.47:0/8730661' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T10:25:26.989 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:25:26 smithi157 ceph-mon[35581]: Adjusting osd_memory_target on smithi157 to 176.8M 2024-03-21T10:25:26.990 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:25:26 smithi157 ceph-mon[35581]: Unable to set osd_memory_target on smithi157 to 185389568: error parsing value: Value '185389568' is below minimum 939524096 2024-03-21T10:25:26.990 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:25:26 smithi157 ceph-mon[35581]: Adjusting osd_memory_target on smithi047 to 177.8M 2024-03-21T10:25:26.990 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:25:26 smithi157 ceph-mon[35581]: Unable to set osd_memory_target on smithi047 to 186497740: error parsing value: Value '186497740' is below minimum 939524096 2024-03-21T10:25:26.990 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:25:26 smithi157 ceph-mon[35581]: from='mgr.15687 172.21.15.47:0/8730661' entity='mgr.x' 2024-03-21T10:25:26.990 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:25:26 smithi157 ceph-mon[35581]: from='mgr.15687 172.21.15.47:0/8730661' entity='mgr.x' 2024-03-21T10:25:26.990 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:25:26 smithi157 ceph-mon[35581]: from='mgr.15687 172.21.15.47:0/8730661' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-21T10:25:26.990 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:25:26 smithi157 ceph-mon[35581]: from='mgr.15687 172.21.15.47:0/8730661' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-21T10:25:26.990 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:25:26 smithi157 ceph-mon[35581]: from='mgr.15687 172.21.15.47:0/8730661' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-21T10:25:26.990 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:25:26 smithi157 ceph-mon[35581]: from='mgr.15687 172.21.15.47:0/8730661' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-03-21T10:25:26.990 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:25:26 smithi157 ceph-mon[35581]: from='mgr.15687 172.21.15.47:0/8730661' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T10:25:26.990 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:25:26 smithi157 ceph-mon[35581]: from='mgr.15687 172.21.15.47:0/8730661' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T10:25:27.963 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:25:27 smithi047 ceph-mon[31547]: Adjusting osd_memory_target on smithi100 to 176.7M 2024-03-21T10:25:27.964 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:25:27 smithi047 ceph-mon[31547]: Unable to set osd_memory_target on smithi100 to 185329356: error parsing value: Value '185329356' is below minimum 939524096 2024-03-21T10:25:27.964 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:25:27 smithi047 ceph-mon[31547]: Updating smithi047:/etc/ceph/ceph.conf 2024-03-21T10:25:27.964 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:25:27 smithi047 ceph-mon[31547]: Updating smithi100:/etc/ceph/ceph.conf 2024-03-21T10:25:27.964 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:25:27 smithi047 ceph-mon[31547]: Updating smithi157:/etc/ceph/ceph.conf 2024-03-21T10:25:27.964 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:25:27 smithi047 ceph-mon[31547]: pgmap v6: 129 pgs: 129 active+clean; 1.1 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-21T10:25:27.964 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:25:27 smithi047 ceph-mon[31547]: Updating smithi100:/var/lib/ceph/3056d51a-e76b-11ee-95cd-87774f69a715/config/ceph.conf 2024-03-21T10:25:27.964 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:25:27 smithi047 ceph-mon[31547]: Updating smithi047:/var/lib/ceph/3056d51a-e76b-11ee-95cd-87774f69a715/config/ceph.conf 2024-03-21T10:25:27.964 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:25:27 smithi047 ceph-mon[31547]: Updating smithi157:/var/lib/ceph/3056d51a-e76b-11ee-95cd-87774f69a715/config/ceph.conf 2024-03-21T10:25:27.979 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:25:27 smithi100 ceph-mon[35193]: Adjusting osd_memory_target on smithi100 to 176.7M 2024-03-21T10:25:27.979 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:25:27 smithi100 ceph-mon[35193]: Unable to set osd_memory_target on smithi100 to 185329356: error parsing value: Value '185329356' is below minimum 939524096 2024-03-21T10:25:27.979 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:25:27 smithi100 ceph-mon[35193]: Updating smithi047:/etc/ceph/ceph.conf 2024-03-21T10:25:27.979 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:25:27 smithi100 ceph-mon[35193]: Updating smithi100:/etc/ceph/ceph.conf 2024-03-21T10:25:27.979 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:25:27 smithi100 ceph-mon[35193]: Updating smithi157:/etc/ceph/ceph.conf 2024-03-21T10:25:27.979 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:25:27 smithi100 ceph-mon[35193]: pgmap v6: 129 pgs: 129 active+clean; 1.1 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-21T10:25:27.980 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:25:27 smithi100 ceph-mon[35193]: Updating smithi100:/var/lib/ceph/3056d51a-e76b-11ee-95cd-87774f69a715/config/ceph.conf 2024-03-21T10:25:27.980 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:25:27 smithi100 ceph-mon[35193]: Updating smithi047:/var/lib/ceph/3056d51a-e76b-11ee-95cd-87774f69a715/config/ceph.conf 2024-03-21T10:25:27.980 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:25:27 smithi100 ceph-mon[35193]: Updating smithi157:/var/lib/ceph/3056d51a-e76b-11ee-95cd-87774f69a715/config/ceph.conf 2024-03-21T10:25:27.989 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:25:27 smithi157 ceph-mon[35581]: Adjusting osd_memory_target on smithi100 to 176.7M 2024-03-21T10:25:27.989 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:25:27 smithi157 ceph-mon[35581]: Unable to set osd_memory_target on smithi100 to 185329356: error parsing value: Value '185329356' is below minimum 939524096 2024-03-21T10:25:27.989 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:25:27 smithi157 ceph-mon[35581]: Updating smithi047:/etc/ceph/ceph.conf 2024-03-21T10:25:27.989 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:25:27 smithi157 ceph-mon[35581]: Updating smithi100:/etc/ceph/ceph.conf 2024-03-21T10:25:27.990 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:25:27 smithi157 ceph-mon[35581]: Updating smithi157:/etc/ceph/ceph.conf 2024-03-21T10:25:27.990 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:25:27 smithi157 ceph-mon[35581]: pgmap v6: 129 pgs: 129 active+clean; 1.1 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-21T10:25:27.990 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:25:27 smithi157 ceph-mon[35581]: Updating smithi100:/var/lib/ceph/3056d51a-e76b-11ee-95cd-87774f69a715/config/ceph.conf 2024-03-21T10:25:27.990 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:25:27 smithi157 ceph-mon[35581]: Updating smithi047:/var/lib/ceph/3056d51a-e76b-11ee-95cd-87774f69a715/config/ceph.conf 2024-03-21T10:25:27.990 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:25:27 smithi157 ceph-mon[35581]: Updating smithi157:/var/lib/ceph/3056d51a-e76b-11ee-95cd-87774f69a715/config/ceph.conf 2024-03-21T10:25:28.963 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:25:28 smithi047 ceph-mon[31547]: Updating smithi100:/etc/ceph/ceph.client.admin.keyring 2024-03-21T10:25:28.964 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:25:28 smithi047 ceph-mon[31547]: Updating smithi047:/etc/ceph/ceph.client.admin.keyring 2024-03-21T10:25:28.964 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:25:28 smithi047 ceph-mon[31547]: Updating smithi157:/etc/ceph/ceph.client.admin.keyring 2024-03-21T10:25:28.964 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:25:28 smithi047 ceph-mon[31547]: from='mgr.15687 172.21.15.47:0/8730661' entity='mgr.x' 2024-03-21T10:25:28.964 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:25:28 smithi047 ceph-mon[31547]: from='mgr.15687 172.21.15.47:0/8730661' entity='mgr.x' 2024-03-21T10:25:28.964 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:25:28 smithi047 ceph-mon[31547]: from='mgr.15687 172.21.15.47:0/8730661' entity='mgr.x' 2024-03-21T10:25:28.964 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:25:28 smithi047 ceph-mon[31547]: from='mgr.15687 172.21.15.47:0/8730661' entity='mgr.x' 2024-03-21T10:25:28.979 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:25:28 smithi100 ceph-mon[35193]: Updating smithi100:/etc/ceph/ceph.client.admin.keyring 2024-03-21T10:25:28.979 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:25:28 smithi100 ceph-mon[35193]: Updating smithi047:/etc/ceph/ceph.client.admin.keyring 2024-03-21T10:25:28.979 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:25:28 smithi100 ceph-mon[35193]: Updating smithi157:/etc/ceph/ceph.client.admin.keyring 2024-03-21T10:25:28.979 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:25:28 smithi100 ceph-mon[35193]: from='mgr.15687 172.21.15.47:0/8730661' entity='mgr.x' 2024-03-21T10:25:28.979 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:25:28 smithi100 ceph-mon[35193]: from='mgr.15687 172.21.15.47:0/8730661' entity='mgr.x' 2024-03-21T10:25:28.979 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:25:28 smithi100 ceph-mon[35193]: from='mgr.15687 172.21.15.47:0/8730661' entity='mgr.x' 2024-03-21T10:25:28.979 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:25:28 smithi100 ceph-mon[35193]: from='mgr.15687 172.21.15.47:0/8730661' entity='mgr.x' 2024-03-21T10:25:28.990 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:25:28 smithi157 ceph-mon[35581]: Updating smithi100:/etc/ceph/ceph.client.admin.keyring 2024-03-21T10:25:28.990 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:25:28 smithi157 ceph-mon[35581]: Updating smithi047:/etc/ceph/ceph.client.admin.keyring 2024-03-21T10:25:28.990 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:25:28 smithi157 ceph-mon[35581]: Updating smithi157:/etc/ceph/ceph.client.admin.keyring 2024-03-21T10:25:28.990 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:25:28 smithi157 ceph-mon[35581]: from='mgr.15687 172.21.15.47:0/8730661' entity='mgr.x' 2024-03-21T10:25:28.990 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:25:28 smithi157 ceph-mon[35581]: from='mgr.15687 172.21.15.47:0/8730661' entity='mgr.x' 2024-03-21T10:25:28.990 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:25:28 smithi157 ceph-mon[35581]: from='mgr.15687 172.21.15.47:0/8730661' entity='mgr.x' 2024-03-21T10:25:28.990 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:25:28 smithi157 ceph-mon[35581]: from='mgr.15687 172.21.15.47:0/8730661' entity='mgr.x' 2024-03-21T10:25:29.964 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:25:29 smithi047 ceph-mon[31547]: Updating smithi100:/var/lib/ceph/3056d51a-e76b-11ee-95cd-87774f69a715/config/ceph.client.admin.keyring 2024-03-21T10:25:29.964 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:25:29 smithi047 ceph-mon[31547]: Updating smithi047:/var/lib/ceph/3056d51a-e76b-11ee-95cd-87774f69a715/config/ceph.client.admin.keyring 2024-03-21T10:25:29.964 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:25:29 smithi047 ceph-mon[31547]: pgmap v7: 129 pgs: 129 active+clean; 1.3 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 68 KiB/s rd, 132 KiB/s wr, 80 op/s 2024-03-21T10:25:29.964 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:25:29 smithi047 ceph-mon[31547]: Updating smithi157:/var/lib/ceph/3056d51a-e76b-11ee-95cd-87774f69a715/config/ceph.client.admin.keyring 2024-03-21T10:25:29.964 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:25:29 smithi047 ceph-mon[31547]: from='mgr.15687 172.21.15.47:0/8730661' entity='mgr.x' 2024-03-21T10:25:29.964 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:25:29 smithi047 ceph-mon[31547]: from='mgr.15687 172.21.15.47:0/8730661' entity='mgr.x' 2024-03-21T10:25:29.964 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:25:29 smithi047 ceph-mon[31547]: from='mgr.15687 172.21.15.47:0/8730661' entity='mgr.x' 2024-03-21T10:25:29.965 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:25:29 smithi047 ceph-mon[31547]: from='mgr.15687 172.21.15.47:0/8730661' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T10:25:29.965 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:25:29 smithi047 ceph-mon[31547]: from='mgr.15687 172.21.15.47:0/8730661' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T10:25:29.965 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:25:29 smithi047 ceph-mon[31547]: from='mgr.15687 172.21.15.47:0/8730661' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T10:25:29.965 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:25:29 smithi047 ceph-mon[31547]: from='mgr.15687 172.21.15.47:0/8730661' entity='mgr.x' 2024-03-21T10:25:29.979 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:25:29 smithi100 ceph-mon[35193]: Updating smithi100:/var/lib/ceph/3056d51a-e76b-11ee-95cd-87774f69a715/config/ceph.client.admin.keyring 2024-03-21T10:25:29.980 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:25:29 smithi100 ceph-mon[35193]: Updating smithi047:/var/lib/ceph/3056d51a-e76b-11ee-95cd-87774f69a715/config/ceph.client.admin.keyring 2024-03-21T10:25:29.980 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:25:29 smithi100 ceph-mon[35193]: pgmap v7: 129 pgs: 129 active+clean; 1.3 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 68 KiB/s rd, 132 KiB/s wr, 80 op/s 2024-03-21T10:25:29.980 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:25:29 smithi100 ceph-mon[35193]: Updating smithi157:/var/lib/ceph/3056d51a-e76b-11ee-95cd-87774f69a715/config/ceph.client.admin.keyring 2024-03-21T10:25:29.980 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:25:29 smithi100 ceph-mon[35193]: from='mgr.15687 172.21.15.47:0/8730661' entity='mgr.x' 2024-03-21T10:25:29.980 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:25:29 smithi100 ceph-mon[35193]: from='mgr.15687 172.21.15.47:0/8730661' entity='mgr.x' 2024-03-21T10:25:29.980 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:25:29 smithi100 ceph-mon[35193]: from='mgr.15687 172.21.15.47:0/8730661' entity='mgr.x' 2024-03-21T10:25:29.980 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:25:29 smithi100 ceph-mon[35193]: from='mgr.15687 172.21.15.47:0/8730661' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T10:25:29.980 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:25:29 smithi100 ceph-mon[35193]: from='mgr.15687 172.21.15.47:0/8730661' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T10:25:29.980 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:25:29 smithi100 ceph-mon[35193]: from='mgr.15687 172.21.15.47:0/8730661' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T10:25:29.980 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:25:29 smithi100 ceph-mon[35193]: from='mgr.15687 172.21.15.47:0/8730661' entity='mgr.x' 2024-03-21T10:25:29.990 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:25:29 smithi157 ceph-mon[35581]: Updating smithi100:/var/lib/ceph/3056d51a-e76b-11ee-95cd-87774f69a715/config/ceph.client.admin.keyring 2024-03-21T10:25:29.990 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:25:29 smithi157 ceph-mon[35581]: Updating smithi047:/var/lib/ceph/3056d51a-e76b-11ee-95cd-87774f69a715/config/ceph.client.admin.keyring 2024-03-21T10:25:29.990 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:25:29 smithi157 ceph-mon[35581]: pgmap v7: 129 pgs: 129 active+clean; 1.3 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 68 KiB/s rd, 132 KiB/s wr, 80 op/s 2024-03-21T10:25:29.990 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:25:29 smithi157 ceph-mon[35581]: Updating smithi157:/var/lib/ceph/3056d51a-e76b-11ee-95cd-87774f69a715/config/ceph.client.admin.keyring 2024-03-21T10:25:29.990 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:25:29 smithi157 ceph-mon[35581]: from='mgr.15687 172.21.15.47:0/8730661' entity='mgr.x' 2024-03-21T10:25:29.990 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:25:29 smithi157 ceph-mon[35581]: from='mgr.15687 172.21.15.47:0/8730661' entity='mgr.x' 2024-03-21T10:25:29.990 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:25:29 smithi157 ceph-mon[35581]: from='mgr.15687 172.21.15.47:0/8730661' entity='mgr.x' 2024-03-21T10:25:29.990 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:25:29 smithi157 ceph-mon[35581]: from='mgr.15687 172.21.15.47:0/8730661' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T10:25:29.990 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:25:29 smithi157 ceph-mon[35581]: from='mgr.15687 172.21.15.47:0/8730661' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T10:25:29.990 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:25:29 smithi157 ceph-mon[35581]: from='mgr.15687 172.21.15.47:0/8730661' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T10:25:29.990 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:25:29 smithi157 ceph-mon[35581]: from='mgr.15687 172.21.15.47:0/8730661' entity='mgr.x' 2024-03-21T10:25:31.963 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:25:31 smithi047 ceph-mon[31547]: pgmap v8: 129 pgs: 129 active+clean; 1.3 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 103 KiB/s wr, 62 op/s 2024-03-21T10:25:31.979 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:25:31 smithi100 ceph-mon[35193]: pgmap v8: 129 pgs: 129 active+clean; 1.3 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 103 KiB/s wr, 62 op/s 2024-03-21T10:25:31.989 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:25:31 smithi157 ceph-mon[35581]: pgmap v8: 129 pgs: 129 active+clean; 1.3 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 103 KiB/s wr, 62 op/s 2024-03-21T10:25:32.963 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:25:32 smithi047 ceph-mon[31547]: pgmap v9: 129 pgs: 129 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 85 KiB/s wr, 55 op/s 2024-03-21T10:25:32.979 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:25:32 smithi100 ceph-mon[35193]: pgmap v9: 129 pgs: 129 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 85 KiB/s wr, 55 op/s 2024-03-21T10:25:32.989 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:25:32 smithi157 ceph-mon[35581]: pgmap v9: 129 pgs: 129 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 85 KiB/s wr, 55 op/s 2024-03-21T10:25:35.463 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:25:35 smithi047 ceph-mon[31547]: pgmap v10: 129 pgs: 129 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 77 KiB/s wr, 51 op/s 2024-03-21T10:25:35.479 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:25:35 smithi100 ceph-mon[35193]: pgmap v10: 129 pgs: 129 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 77 KiB/s wr, 51 op/s 2024-03-21T10:25:35.489 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:25:35 smithi157 ceph-mon[35581]: pgmap v10: 129 pgs: 129 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 77 KiB/s wr, 51 op/s 2024-03-21T10:25:36.213 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:25:36.210+0000 7fd044c00640 1 -- 172.21.15.47:0/477297122 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd040104040 msgr2=0x7fd040110550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:25:36.213 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:25:36.210+0000 7fd044c00640 1 --2- 172.21.15.47:0/477297122 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd040104040 0x7fd040110550 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7fd038009cd0 tx=0x7fd03802c0f0 comp rx=0 tx=0).stop 2024-03-21T10:25:36.213 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:25:36.211+0000 7fd044c00640 1 -- 172.21.15.47:0/477297122 shutdown_connections 2024-03-21T10:25:36.213 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:25:36.211+0000 7fd044c00640 1 --2- 172.21.15.47:0/477297122 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd040104040 0x7fd040110550 unknown :-1 s=CLOSED pgs=527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:25:36.213 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:25:36.211+0000 7fd044c00640 1 --2- 172.21.15.47:0/477297122 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd040103680 0x7fd040103b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:25:36.213 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:25:36.211+0000 7fd044c00640 1 --2- 172.21.15.47:0/477297122 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd040109690 0x7fd040109a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:25:36.213 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:25:36.211+0000 7fd044c00640 1 -- 172.21.15.47:0/477297122 >> 172.21.15.47:0/477297122 conn(0x7fd0400ff110 msgr2=0x7fd040101530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:25:36.213 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:25:36.211+0000 7fd044c00640 1 -- 172.21.15.47:0/477297122 shutdown_connections 2024-03-21T10:25:36.213 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:25:36.211+0000 7fd044c00640 1 -- 172.21.15.47:0/477297122 wait complete. 2024-03-21T10:25:36.216 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:25:36.212+0000 7fd044c00640 1 Processor -- start 2024-03-21T10:25:36.216 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:25:36.212+0000 7fd044c00640 1 -- start start 2024-03-21T10:25:36.216 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:25:36.212+0000 7fd044c00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd040103680 0x7fd040199e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:25:36.216 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:25:36.212+0000 7fd044c00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd040104040 0x7fd04019a3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:25:36.216 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:25:36.212+0000 7fd044c00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd040109690 0x7fd04019e750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:25:36.216 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:25:36.212+0000 7fd044c00640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd040113910 con 0x7fd040109690 2024-03-21T10:25:36.216 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:25:36.212+0000 7fd044c00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd040113790 con 0x7fd040104040 2024-03-21T10:25:36.216 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:25:36.212+0000 7fd044c00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd040113a90 con 0x7fd040103680 2024-03-21T10:25:36.216 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:25:36.212+0000 7fd03fe00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd040109690 0x7fd04019e750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:25:36.216 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:25:36.212+0000 7fd03fe00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd040109690 0x7fd04019e750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:55872/0 (socket says 172.21.15.47:55872) 2024-03-21T10:25:36.216 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:25:36.212+0000 7fd03fe00640 1 -- 172.21.15.47:0/4023408273 learned_addr learned my addr 172.21.15.47:0/4023408273 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:25:36.217 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:25:36.213+0000 7fd03ea00640 1 --2- 172.21.15.47:0/4023408273 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd040104040 0x7fd04019a3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:25:36.217 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:25:36.213+0000 7fd03f400640 1 --2- 172.21.15.47:0/4023408273 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd040103680 0x7fd040199e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:25:36.217 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:25:36.213+0000 7fd03fe00640 1 -- 172.21.15.47:0/4023408273 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd040103680 msgr2=0x7fd040199e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:25:36.217 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:25:36.213+0000 7fd03fe00640 1 --2- 172.21.15.47:0/4023408273 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd040103680 0x7fd040199e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:25:36.217 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:25:36.213+0000 7fd03fe00640 1 -- 172.21.15.47:0/4023408273 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd040104040 msgr2=0x7fd04019a3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:25:36.217 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:25:36.213+0000 7fd03fe00640 1 --2- 172.21.15.47:0/4023408273 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd040104040 0x7fd04019a3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:25:36.217 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:25:36.213+0000 7fd03fe00640 1 -- 172.21.15.47:0/4023408273 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd04019ee80 con 0x7fd040109690 2024-03-21T10:25:36.217 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:25:36.213+0000 7fd03ea00640 1 --2- 172.21.15.47:0/4023408273 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd040104040 0x7fd04019a3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T10:25:36.217 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:25:36.213+0000 7fd03fe00640 1 --2- 172.21.15.47:0/4023408273 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd040109690 0x7fd04019e750 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7fd038005e30 tx=0x7fd038035ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:25:36.217 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:25:36.214+0000 7fd037e00640 1 -- 172.21.15.47:0/4023408273 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd038039040 con 0x7fd040109690 2024-03-21T10:25:36.218 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:25:36.214+0000 7fd044c00640 1 -- 172.21.15.47:0/4023408273 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd04019f110 con 0x7fd040109690 2024-03-21T10:25:36.218 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:25:36.214+0000 7fd037e00640 1 -- 172.21.15.47:0/4023408273 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fd038044070 con 0x7fd040109690 2024-03-21T10:25:36.218 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:25:36.214+0000 7fd037e00640 1 -- 172.21.15.47:0/4023408273 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0380042a0 con 0x7fd040109690 2024-03-21T10:25:36.218 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:25:36.214+0000 7fd044c00640 1 -- 172.21.15.47:0/4023408273 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd0401a6a00 con 0x7fd040109690 2024-03-21T10:25:36.218 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:25:36.217+0000 7fd048e5f740 1 Processor -- start 2024-03-21T10:25:36.218 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:25:36.217+0000 7fd037e00640 1 -- 172.21.15.47:0/4023408273 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 25) v1 ==== 138030+0+0 (secure 0 0 0) 0x7fd038004440 con 0x7fd040109690 2024-03-21T10:25:36.219 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:25:36.217+0000 7fd048e5f740 1 -- start start 2024-03-21T10:25:36.219 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:25:36.218+0000 7fd048e5f740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563ea7a441e0 0x563ea7a445c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:25:36.220 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:25:36.218+0000 7fd048e5f740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563ea7a44b00 0x563ea7a4d390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:25:36.220 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:25:36.218+0000 7fd048e5f740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x563ea7a4d8d0 0x563ea7a4fcc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:25:36.220 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:25:36.218+0000 7fd048e5f740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x563ea7a063b0 con 0x563ea7a4d8d0 2024-03-21T10:25:36.220 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:25:36.218+0000 7fd048e5f740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x563ea79a8830 con 0x563ea7a441e0 2024-03-21T10:25:36.220 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:25:36.218+0000 7fd048e5f740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x563ea79515d0 con 0x563ea7a44b00 2024-03-21T10:25:36.221 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:25:36.218+0000 7fd03ea00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x563ea7a4d8d0 0x563ea7a4fcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:25:36.221 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:25:36.218+0000 7fd03ea00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x563ea7a4d8d0 0x563ea7a4fcc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:55888/0 (socket says 172.21.15.47:55888) 2024-03-21T10:25:36.222 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:25:36.218+0000 7fd03ea00640 1 -- 172.21.15.47:0/1114113675 learned_addr learned my addr 172.21.15.47:0/1114113675 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:25:36.222 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:25:36.218+0000 7fd03ea00640 1 --2- 172.21.15.47:0/1114113675 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563ea7a441e0 0x563ea7a445c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:25:36.222 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:25:36.219+0000 7fd03f400640 1 --2- 172.21.15.47:0/1114113675 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563ea7a44b00 0x563ea7a4d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:25:36.222 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:25:36.219+0000 7fd03ea00640 1 -- 172.21.15.47:0/1114113675 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563ea7a44b00 msgr2=0x563ea7a4d390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:25:36.222 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:25:36.219+0000 7fd03ea00640 1 --2- 172.21.15.47:0/1114113675 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563ea7a44b00 0x563ea7a4d390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:25:36.222 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:25:36.219+0000 7fd03ea00640 1 -- 172.21.15.47:0/1114113675 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563ea7a441e0 msgr2=0x563ea7a445c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:25:36.222 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:25:36.219+0000 7fd03ea00640 1 --2- 172.21.15.47:0/1114113675 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563ea7a441e0 0x563ea7a445c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:25:36.222 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:25:36.219+0000 7fd037e00640 1 --2- 172.21.15.47:0/4023408273 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7fd0240a3110 0x7fd0240a55d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:25:36.222 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:25:36.219+0000 7fd03ea00640 1 -- 172.21.15.47:0/1114113675 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563ea79692c0 con 0x563ea7a4d8d0 2024-03-21T10:25:36.222 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:25:36.219+0000 7fd037e00640 1 -- 172.21.15.47:0/4023408273 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 9474+0+0 (secure 0 0 0) 0x7fd0380f1ff0 con 0x7fd040109690 2024-03-21T10:25:36.222 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:25:36.219+0000 7fd03ea00640 1 --2- 172.21.15.47:0/1114113675 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x563ea7a4d8d0 0x563ea7a4fcc0 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7fd028014a30 tx=0x7fd02800c570 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4f66ab8bfd779283 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:25:36.222 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:25:36.219+0000 7fd03f400640 1 --2- 172.21.15.47:0/4023408273 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7fd0240a3110 0x7fd0240a55d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:25:36.223 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:25:36.220+0000 7fd02fe00640 1 -- 172.21.15.47:0/1114113675 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0280142d0 con 0x563ea7a4d8d0 2024-03-21T10:25:36.223 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:25:36.220+0000 7fd02fe00640 1 -- 172.21.15.47:0/1114113675 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fd02800f820 con 0x563ea7a4d8d0 2024-03-21T10:25:36.223 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:25:36.220+0000 7fd02fe00640 1 -- 172.21.15.47:0/1114113675 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd028003830 con 0x563ea7a4d8d0 2024-03-21T10:25:36.226 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:25:36.220+0000 7fd03f400640 1 --2- 172.21.15.47:0/4023408273 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7fd0240a3110 0x7fd0240a55d0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x563ea7a43c90 tx=0x7fd030005f90 comp rx=0 tx=0).ready entity=mgr.15687 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:25:36.226 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:25:36.220+0000 7fd048e5f740 1 -- 172.21.15.47:0/1114113675 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x563ea7a4d8d0 msgr2=0x563ea7a4fcc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:25:36.227 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:25:36.221+0000 7fd048e5f740 1 --2- 172.21.15.47:0/1114113675 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x563ea7a4d8d0 0x563ea7a4fcc0 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7fd028014a30 tx=0x7fd02800c570 comp rx=0 tx=0).stop 2024-03-21T10:25:36.227 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:25:36.221+0000 7fd048e5f740 1 -- 172.21.15.47:0/1114113675 shutdown_connections 2024-03-21T10:25:36.227 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:25:36.221+0000 7fd048e5f740 1 --2- 172.21.15.47:0/1114113675 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x563ea7a4d8d0 0x563ea7a4fcc0 unknown :-1 s=CLOSED pgs=529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:25:36.227 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:25:36.221+0000 7fd048e5f740 1 --2- 172.21.15.47:0/1114113675 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563ea7a44b00 0x563ea7a4d390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:25:36.227 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:25:36.221+0000 7fd048e5f740 1 --2- 172.21.15.47:0/1114113675 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563ea7a441e0 0x563ea7a445c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:25:36.227 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:25:36.221+0000 7fd048e5f740 1 -- 172.21.15.47:0/1114113675 >> 172.21.15.47:0/1114113675 conn(0x563ea7a3eb00 msgr2=0x563ea7a3eee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:25:36.227 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:25:36.221+0000 7fd048e5f740 1 -- 172.21.15.47:0/1114113675 shutdown_connections 2024-03-21T10:25:36.227 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:25:36.221+0000 7fd048e5f740 1 -- 172.21.15.47:0/1114113675 wait complete. 2024-03-21T10:25:36.227 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:25:36.221+0000 7fd048e5f740 1 -- start start 2024-03-21T10:25:36.227 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:25:36.221+0000 7fd048e5f740 1 Processor -- start 2024-03-21T10:25:36.227 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:25:36.222+0000 7fd048e5f740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563ea7a441e0 0x563ea7a4a900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:25:36.227 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:25:36.222+0000 7fd048e5f740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x563ea7a44b00 0x563ea7a4ae40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:25:36.228 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:25:36.222+0000 7fd048e5f740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563ea7a4d8d0 0x563ea7a4b380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:25:36.228 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:25:36.222+0000 7fd048e5f740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x563ea7964a20 con 0x563ea7a44b00 2024-03-21T10:25:36.228 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:25:36.222+0000 7fd048e5f740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x563ea79a48d0 con 0x563ea7a4d8d0 2024-03-21T10:25:36.228 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:25:36.222+0000 7fd048e5f740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x563ea7a504a0 con 0x563ea7a441e0 2024-03-21T10:25:36.228 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:25:36.222+0000 7fd048e5f740 10 client.0 _ll_register_callbacks cb 0x563ea79a11c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T10:25:36.228 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:25:36.222+0000 7fd03ea00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x563ea7a44b00 0x563ea7a4ae40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:25:36.228 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:25:36.222+0000 7fd03ea00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x563ea7a44b00 0x563ea7a4ae40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:55900/0 (socket says 172.21.15.47:55900) 2024-03-21T10:25:36.228 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:25:36.222+0000 7fd03ea00640 1 -- 172.21.15.47:0/3349687166 learned_addr learned my addr 172.21.15.47:0/3349687166 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:25:36.228 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:25:36.222+0000 7fd03fe00640 1 --2- 172.21.15.47:0/3349687166 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563ea7a4d8d0 0x563ea7a4b380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:25:36.228 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:25:36.222+0000 7fd03f400640 1 --2- 172.21.15.47:0/3349687166 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563ea7a441e0 0x563ea7a4a900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:25:36.228 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:25:36.223+0000 7fd03ea00640 1 -- 172.21.15.47:0/3349687166 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563ea7a441e0 msgr2=0x563ea7a4a900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:25:36.228 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:25:36.223+0000 7fd03ea00640 1 --2- 172.21.15.47:0/3349687166 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563ea7a441e0 0x563ea7a4a900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:25:36.228 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:25:36.223+0000 7fd03ea00640 1 -- 172.21.15.47:0/3349687166 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563ea7a4d8d0 msgr2=0x563ea7a4b380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:25:36.228 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:25:36.223+0000 7fd03ea00640 1 --2- 172.21.15.47:0/3349687166 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563ea7a4d8d0 0x563ea7a4b380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:25:36.229 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:25:36.223+0000 7fd03ea00640 1 -- 172.21.15.47:0/3349687166 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x563ea76bd210 con 0x563ea7a44b00 2024-03-21T10:25:36.229 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:25:36.223+0000 7fd03ea00640 1 --2- 172.21.15.47:0/3349687166 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x563ea7a44b00 0x563ea7a4ae40 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7fd0280142e0 tx=0x7fd028014310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d503a8f77288e908 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:25:36.229 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:25:36.223+0000 7fd02cc00640 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-03-21T10:25:36.229 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:25:36.223+0000 7fd02cc00640 1 -- 172.21.15.47:0/3349687166 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0280146d0 con 0x563ea7a44b00 2024-03-21T10:25:36.229 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:25:36.223+0000 7fd02cc00640 1 -- 172.21.15.47:0/3349687166 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 9474+0+0 (secure 0 0 0) 0x7fd028004030 con 0x563ea7a44b00 2024-03-21T10:25:36.229 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:25:36.224+0000 7fd048e5f740 1 -- 172.21.15.47:0/3349687166 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fd02800f820 con 0x563ea7a44b00 2024-03-21T10:25:36.229 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:25:36.224+0000 7fd02cc00640 1 -- 172.21.15.47:0/3349687166 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd028030050 con 0x563ea7a44b00 2024-03-21T10:25:36.229 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:25:36.224+0000 7fd048e5f740 10 client.15777 fetch_fsmap learned FSMap version 30 2024-03-21T10:25:36.229 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:25:36.224+0000 7fd048e5f740 1 -- 172.21.15.47:0/3349687166 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x563ea7789620 con 0x563ea7a44b00 2024-03-21T10:25:36.229 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:25:36.224+0000 7fd02cc00640 1 -- 172.21.15.47:0/3349687166 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fd02800f820 con 0x563ea7a44b00 2024-03-21T10:25:36.229 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:25:36.224+0000 7fd02cc00640 1 -- 172.21.15.47:0/3349687166 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 30) v1 ==== 7942+0+0 (secure 0 0 0) 0x7fd0280047c0 con 0x563ea7a44b00 2024-03-21T10:25:36.229 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:25:36.224+0000 7fd048e5f740 10 client.15777 fetch_fsmap finished waiting for FSMap version 30 2024-03-21T10:25:36.229 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:25:36.224+0000 7fd048e5f740 10 client.15777 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-21T10:25:36.229 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:25:36.224+0000 7fd048e5f740 20 client.15777 populate_metadata read hostname 'smithi047' 2024-03-21T10:25:36.230 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:25:36.225+0000 7fd048e5f740 1 --2- 172.21.15.47:0/3349687166 >> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] conn(0x563ea7a5d830 0x563ea7a5fc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:25:36.230 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:25:36.225+0000 7fd048e5f740 4 client.15777 mds_command: new command op to 34535 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T10:25:36.230 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:25:36.225+0000 7fd048e5f740 1 -- 172.21.15.47:0/3349687166 --> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x563ea79b5d70 con 0x563ea7a5d830 2024-03-21T10:25:36.230 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:25:36.225+0000 7fd03f400640 1 --2- 172.21.15.47:0/3349687166 >> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] conn(0x563ea7a5d830 0x563ea7a5fc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:25:36.231 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:25:36.226+0000 7fd03f400640 1 --2- 172.21.15.47:0/3349687166 >> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] conn(0x563ea7a5d830 0x563ea7a5fc20 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7ad5d11f6bd357d9 server_cookie=3c5e8c50552281fe in_seq=0 out_seq=0 2024-03-21T10:25:36.231 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:25:36.226+0000 7fd02cc00640 10 client.15777 ms_handle_connect on v2:172.21.15.100:6838/4173168440 2024-03-21T10:25:36.231 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:25:36.226+0000 7fd02cc00640 1 -- 172.21.15.47:0/3349687166 <== mds.0 v2:172.21.15.100:6838/4173168440 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x563ea79b5d70 con 0x563ea7a5d830 2024-03-21T10:25:36.231 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:25:36.226+0000 7fd02cc00640 10 client.15777 handle_command_reply: tid=0 2024-03-21T10:25:36.231 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:25:36.226+0000 7fd048e5f740 1 client.15777 shutdown 2024-03-21T10:25:36.231 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:25:36.227+0000 7fd048e5f740 1 -- 172.21.15.47:0/3349687166 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x563ea7a44b00 msgr2=0x563ea7a4ae40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:25:36.231 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:25:36.227+0000 7fd048e5f740 1 --2- 172.21.15.47:0/3349687166 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x563ea7a44b00 0x563ea7a4ae40 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7fd0280142e0 tx=0x7fd028014310 comp rx=0 tx=0).stop 2024-03-21T10:25:36.231 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:25:36.227+0000 7fd048e5f740 1 -- 172.21.15.47:0/3349687166 shutdown_connections 2024-03-21T10:25:36.231 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:25:36.227+0000 7fd048e5f740 1 --2- 172.21.15.47:0/3349687166 >> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] conn(0x563ea7a5d830 0x563ea7a5fc20 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:25:36.232 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:25:36.227+0000 7fd048e5f740 1 --2- 172.21.15.47:0/3349687166 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563ea7a4d8d0 0x563ea7a4b380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:25:36.232 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:25:36.227+0000 7fd048e5f740 1 --2- 172.21.15.47:0/3349687166 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x563ea7a44b00 0x563ea7a4ae40 unknown :-1 s=CLOSED pgs=530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:25:36.232 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:25:36.227+0000 7fd048e5f740 1 --2- 172.21.15.47:0/3349687166 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563ea7a441e0 0x563ea7a4a900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:25:36.232 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:25:36.227+0000 7fd02cc00640 0 client.15777 ms_handle_reset on v2:172.21.15.100:6838/4173168440 2024-03-21T10:25:36.232 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:25:36.227+0000 7fd048e5f740 1 -- 172.21.15.47:0/3349687166 >> 172.21.15.47:0/3349687166 conn(0x563ea7a3eb00 msgr2=0x563ea7a40850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:25:36.232 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:25:36.227+0000 7fd048e5f740 1 -- 172.21.15.47:0/3349687166 shutdown_connections 2024-03-21T10:25:36.232 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:25:36.227+0000 7fd048e5f740 1 -- 172.21.15.47:0/3349687166 wait complete. 2024-03-21T10:25:36.232 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:25:36.227+0000 7fd048e5f740 20 client.15777 trim_cache size 0 max 16384 2024-03-21T10:25:36.244 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:25:36.242+0000 7fd048e5f740 1 Processor -- start 2024-03-21T10:25:36.244 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:25:36.242+0000 7fd048e5f740 1 -- start start 2024-03-21T10:25:36.245 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:25:36.243+0000 7fd048e5f740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x563ea7a441e0 0x563ea7a4d030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:25:36.245 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:25:36.243+0000 7fd048e5f740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563ea7a44b00 0x563ea7a47ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:25:36.245 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:25:36.243+0000 7fd048e5f740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563ea7a4d8d0 0x563ea7a480e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:25:36.245 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:25:36.243+0000 7fd048e5f740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0401b0900 con 0x563ea7a441e0 2024-03-21T10:25:36.245 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:25:36.243+0000 7fd048e5f740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0401b2280 con 0x563ea7a4d8d0 2024-03-21T10:25:36.245 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:25:36.243+0000 7fd048e5f740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0401b0f60 con 0x563ea7a44b00 2024-03-21T10:25:36.245 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:25:36.243+0000 7fd03f400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x563ea7a441e0 0x563ea7a4d030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:25:36.245 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:25:36.243+0000 7fd03f400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x563ea7a441e0 0x563ea7a4d030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:55912/0 (socket says 172.21.15.47:55912) 2024-03-21T10:25:36.245 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:25:36.243+0000 7fd03f400640 1 -- 172.21.15.47:0/1684412767 learned_addr learned my addr 172.21.15.47:0/1684412767 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:25:36.245 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:25:36.243+0000 7fd03fe00640 1 --2- 172.21.15.47:0/1684412767 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563ea7a4d8d0 0x563ea7a480e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:25:36.245 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:25:36.243+0000 7fd03f400640 1 -- 172.21.15.47:0/1684412767 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563ea7a44b00 msgr2=0x563ea7a47ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:25:36.246 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:25:36.243+0000 7fd03ea00640 1 --2- 172.21.15.47:0/1684412767 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563ea7a44b00 0x563ea7a47ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:25:36.246 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:25:36.243+0000 7fd03f400640 1 --2- 172.21.15.47:0/1684412767 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563ea7a44b00 0x563ea7a47ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:25:36.246 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:25:36.243+0000 7fd03f400640 1 -- 172.21.15.47:0/1684412767 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563ea7a4d8d0 msgr2=0x563ea7a480e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:25:36.246 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:25:36.243+0000 7fd03f400640 1 --2- 172.21.15.47:0/1684412767 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563ea7a4d8d0 0x563ea7a480e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:25:36.246 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:25:36.243+0000 7fd03f400640 1 -- 172.21.15.47:0/1684412767 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563ea7a65430 con 0x563ea7a441e0 2024-03-21T10:25:36.248 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:25:36.244+0000 7fd03f400640 1 --2- 172.21.15.47:0/1684412767 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x563ea7a441e0 0x563ea7a4d030 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7fd03002b7f0 tx=0x7fd030017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4d3b049386696b4 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:25:36.248 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:25:36.244+0000 7fd02f400640 1 -- 172.21.15.47:0/1684412767 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd030031070 con 0x563ea7a441e0 2024-03-21T10:25:36.249 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:25:36.244+0000 7fd02f400640 1 -- 172.21.15.47:0/1684412767 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fd03002be30 con 0x563ea7a441e0 2024-03-21T10:25:36.249 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:25:36.244+0000 7fd02f400640 1 -- 172.21.15.47:0/1684412767 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd03001f850 con 0x563ea7a441e0 2024-03-21T10:25:36.249 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:25:36.245+0000 7fd048e5f740 1 -- 172.21.15.47:0/1684412767 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x563ea7a441e0 msgr2=0x563ea7a4d030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:25:36.249 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:25:36.245+0000 7fd048e5f740 1 --2- 172.21.15.47:0/1684412767 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x563ea7a441e0 0x563ea7a4d030 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7fd03002b7f0 tx=0x7fd030017000 comp rx=0 tx=0).stop 2024-03-21T10:25:36.249 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:25:36.245+0000 7fd048e5f740 1 -- 172.21.15.47:0/1684412767 shutdown_connections 2024-03-21T10:25:36.249 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:25:36.245+0000 7fd048e5f740 1 --2- 172.21.15.47:0/1684412767 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563ea7a4d8d0 0x563ea7a480e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:25:36.249 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:25:36.245+0000 7fd048e5f740 1 --2- 172.21.15.47:0/1684412767 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563ea7a44b00 0x563ea7a47ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:25:36.249 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:25:36.245+0000 7fd048e5f740 1 --2- 172.21.15.47:0/1684412767 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x563ea7a441e0 0x563ea7a4d030 unknown :-1 s=CLOSED pgs=531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:25:36.249 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:25:36.245+0000 7fd048e5f740 1 -- 172.21.15.47:0/1684412767 >> 172.21.15.47:0/1684412767 conn(0x563ea7a3eb00 msgr2=0x563ea7a45060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:25:36.249 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:25:36.245+0000 7fd048e5f740 1 -- 172.21.15.47:0/1684412767 shutdown_connections 2024-03-21T10:25:36.249 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:25:36.245+0000 7fd048e5f740 1 -- 172.21.15.47:0/1684412767 wait complete. 2024-03-21T10:25:36.249 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:25:36.245+0000 7fd048e5f740 1 -- start start 2024-03-21T10:25:36.250 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:25:36.245+0000 7fd048e5f740 1 Processor -- start 2024-03-21T10:25:36.250 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:25:36.246+0000 7fd048e5f740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563ea7a441e0 0x563ea7a60c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:25:36.250 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:25:36.246+0000 7fd048e5f740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x563ea7a44b00 0x563ea7a61190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:25:36.250 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:25:36.246+0000 7fd048e5f740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563ea7a4d8d0 0x563ea7a620a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:25:36.250 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:25:36.246+0000 7fd048e5f740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0401b1c20 con 0x563ea7a44b00 2024-03-21T10:25:36.250 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:25:36.246+0000 7fd048e5f740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0401b15c0 con 0x563ea7a4d8d0 2024-03-21T10:25:36.250 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:25:36.246+0000 7fd048e5f740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0401aeac0 con 0x563ea7a441e0 2024-03-21T10:25:36.250 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:25:36.246+0000 7fd048e5f740 10 client.0 _ll_register_callbacks cb 0x7fd028014470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T10:25:36.250 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:25:36.246+0000 7fd03ea00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x563ea7a44b00 0x563ea7a61190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:25:36.250 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:25:36.246+0000 7fd03ea00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x563ea7a44b00 0x563ea7a61190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:55914/0 (socket says 172.21.15.47:55914) 2024-03-21T10:25:36.250 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:25:36.246+0000 7fd03ea00640 1 -- 172.21.15.47:0/2163056757 learned_addr learned my addr 172.21.15.47:0/2163056757 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:25:36.250 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:25:36.246+0000 7fd03fe00640 1 --2- 172.21.15.47:0/2163056757 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563ea7a4d8d0 0x563ea7a620a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:25:36.250 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:25:36.247+0000 7fd03f400640 1 --2- 172.21.15.47:0/2163056757 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563ea7a441e0 0x563ea7a60c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:25:36.251 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:25:36.247+0000 7fd03ea00640 1 -- 172.21.15.47:0/2163056757 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563ea7a441e0 msgr2=0x563ea7a60c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:25:36.251 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:25:36.247+0000 7fd03ea00640 1 --2- 172.21.15.47:0/2163056757 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563ea7a441e0 0x563ea7a60c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:25:36.251 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:25:36.247+0000 7fd03ea00640 1 -- 172.21.15.47:0/2163056757 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563ea7a4d8d0 msgr2=0x563ea7a620a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:25:36.251 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:25:36.247+0000 7fd03ea00640 1 --2- 172.21.15.47:0/2163056757 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563ea7a4d8d0 0x563ea7a620a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:25:36.251 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:25:36.247+0000 7fd03ea00640 1 -- 172.21.15.47:0/2163056757 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x563ea7a628a0 con 0x563ea7a44b00 2024-03-21T10:25:36.251 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:25:36.247+0000 7fd03ea00640 1 --2- 172.21.15.47:0/2163056757 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x563ea7a44b00 0x563ea7a61190 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7fd028010010 tx=0x7fd02800f820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8b523f79dc4421b8 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:25:36.251 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:25:36.247+0000 7fd02cc00640 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-03-21T10:25:36.252 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:25:36.247+0000 7fd02cc00640 1 -- 172.21.15.47:0/2163056757 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd02800c430 con 0x563ea7a44b00 2024-03-21T10:25:36.253 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:25:36.247+0000 7fd02cc00640 1 -- 172.21.15.47:0/2163056757 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 9474+0+0 (secure 0 0 0) 0x7fd02800fbe0 con 0x563ea7a44b00 2024-03-21T10:25:36.253 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:25:36.247+0000 7fd048e5f740 1 -- 172.21.15.47:0/2163056757 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fd03002be30 con 0x563ea7a44b00 2024-03-21T10:25:36.253 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:25:36.248+0000 7fd02cc00640 1 -- 172.21.15.47:0/2163056757 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd028004680 con 0x563ea7a44b00 2024-03-21T10:25:36.253 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:25:36.248+0000 7fd048e5f740 10 client.15783 fetch_fsmap learned FSMap version 30 2024-03-21T10:25:36.253 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:25:36.248+0000 7fd048e5f740 1 -- 172.21.15.47:0/2163056757 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x563ea7ab9930 con 0x563ea7a44b00 2024-03-21T10:25:36.253 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:25:36.248+0000 7fd02cc00640 1 -- 172.21.15.47:0/2163056757 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fd03002be30 con 0x563ea7a44b00 2024-03-21T10:25:36.253 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:25:36.248+0000 7fd02cc00640 1 -- 172.21.15.47:0/2163056757 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 30) v1 ==== 7942+0+0 (secure 0 0 0) 0x7fd028005240 con 0x563ea7a44b00 2024-03-21T10:25:36.253 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:25:36.248+0000 7fd048e5f740 10 client.15783 fetch_fsmap finished waiting for FSMap version 30 2024-03-21T10:25:36.253 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:25:36.248+0000 7fd048e5f740 10 client.15783 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-21T10:25:36.253 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:25:36.248+0000 7fd048e5f740 20 client.15783 populate_metadata read hostname 'smithi047' 2024-03-21T10:25:36.253 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:25:36.248+0000 7fd048e5f740 1 --2- 172.21.15.47:0/2163056757 >> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] conn(0x563ea7aba050 0x563ea7abc470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:25:36.253 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:25:36.248+0000 7fd048e5f740 4 client.15783 mds_command: new command op to 34535 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-21T10:25:36.253 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:25:36.248+0000 7fd048e5f740 1 -- 172.21.15.47:0/2163056757 --> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x563ea79b00c0 con 0x563ea7aba050 2024-03-21T10:25:36.254 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:25:36.249+0000 7fd03f400640 1 --2- 172.21.15.47:0/2163056757 >> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] conn(0x563ea7aba050 0x563ea7abc470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:25:36.254 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:25:36.250+0000 7fd03f400640 1 --2- 172.21.15.47:0/2163056757 >> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] conn(0x563ea7aba050 0x563ea7abc470 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d27e56eb7f02c96 server_cookie=fe4bbcfd439e30c2 in_seq=0 out_seq=0 2024-03-21T10:25:36.254 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:25:36.250+0000 7fd02cc00640 10 client.15783 ms_handle_connect on v2:172.21.15.100:6838/4173168440 2024-03-21T10:25:36.279 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:25:36.277+0000 7fd02cc00640 1 -- 172.21.15.47:0/2163056757 <== mds.0 v2:172.21.15.100:6838/4173168440 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x563ea79b00c0 con 0x563ea7aba050 2024-03-21T10:25:36.279 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:25:36.277+0000 7fd02cc00640 10 client.15783 handle_command_reply: tid=0 2024-03-21T10:25:36.279 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:25:36.277+0000 7fd048e5f740 1 client.15783 shutdown 2024-03-21T10:25:36.279 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:25:36.277+0000 7fd048e5f740 1 -- 172.21.15.47:0/2163056757 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x563ea7a44b00 msgr2=0x563ea7a61190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:25:36.280 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:25:36.277+0000 7fd048e5f740 1 --2- 172.21.15.47:0/2163056757 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x563ea7a44b00 0x563ea7a61190 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7fd028010010 tx=0x7fd02800f820 comp rx=0 tx=0).stop 2024-03-21T10:25:36.280 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:25:36.277+0000 7fd048e5f740 1 -- 172.21.15.47:0/2163056757 shutdown_connections 2024-03-21T10:25:36.280 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:25:36.277+0000 7fd048e5f740 1 --2- 172.21.15.47:0/2163056757 >> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] conn(0x563ea7aba050 0x563ea7abc470 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:25:36.280 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:25:36.277+0000 7fd048e5f740 1 --2- 172.21.15.47:0/2163056757 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563ea7a4d8d0 0x563ea7a620a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:25:36.280 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:25:36.277+0000 7fd048e5f740 1 --2- 172.21.15.47:0/2163056757 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x563ea7a44b00 0x563ea7a61190 unknown :-1 s=CLOSED pgs=532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:25:36.280 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:25:36.277+0000 7fd02cc00640 0 client.15783 ms_handle_reset on v2:172.21.15.100:6838/4173168440 2024-03-21T10:25:36.280 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:25:36.277+0000 7fd048e5f740 1 --2- 172.21.15.47:0/2163056757 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563ea7a441e0 0x563ea7a60c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:25:36.280 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:25:36.277+0000 7fd048e5f740 1 -- 172.21.15.47:0/2163056757 >> 172.21.15.47:0/2163056757 conn(0x563ea7a3eb00 msgr2=0x563ea7a3ff00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:25:36.280 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:25:36.278+0000 7fd048e5f740 1 -- 172.21.15.47:0/2163056757 shutdown_connections 2024-03-21T10:25:36.280 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:25:36.278+0000 7fd048e5f740 1 -- 172.21.15.47:0/2163056757 wait complete. 2024-03-21T10:25:36.280 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:25:36.278+0000 7fd048e5f740 20 client.15783 trim_cache size 0 max 16384 2024-03-21T10:25:36.281 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout:{ 2024-03-21T10:25:36.281 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout: "message": "", 2024-03-21T10:25:36.281 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout: "return_code": 0 2024-03-21T10:25:36.281 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout:} 2024-03-21T10:25:36.282 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:25:36.279+0000 7fd044c00640 1 -- 172.21.15.47:0/4023408273 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7fd0240a3110 msgr2=0x7fd0240a55d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:25:36.282 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:25:36.279+0000 7fd044c00640 1 --2- 172.21.15.47:0/4023408273 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7fd0240a3110 0x7fd0240a55d0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x563ea7a43c90 tx=0x7fd030005f90 comp rx=0 tx=0).stop 2024-03-21T10:25:36.282 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:25:36.279+0000 7fd044c00640 1 -- 172.21.15.47:0/4023408273 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd040109690 msgr2=0x7fd04019e750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:25:36.282 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:25:36.279+0000 7fd044c00640 1 --2- 172.21.15.47:0/4023408273 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd040109690 0x7fd04019e750 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7fd038005e30 tx=0x7fd038035ce0 comp rx=0 tx=0).stop 2024-03-21T10:25:36.282 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:25:36.279+0000 7fd044c00640 1 -- 172.21.15.47:0/4023408273 shutdown_connections 2024-03-21T10:25:36.282 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:25:36.279+0000 7fd044c00640 1 --2- 172.21.15.47:0/4023408273 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd040109690 0x7fd04019e750 unknown :-1 s=CLOSED pgs=528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:25:36.282 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:25:36.279+0000 7fd044c00640 1 --2- 172.21.15.47:0/4023408273 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7fd0240a3110 0x7fd0240a55d0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:25:36.282 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:25:36.279+0000 7fd044c00640 1 --2- 172.21.15.47:0/4023408273 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd040104040 0x7fd04019a3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:25:36.282 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:25:36.279+0000 7fd044c00640 1 --2- 172.21.15.47:0/4023408273 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd040103680 0x7fd040199e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:25:36.282 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:25:36.279+0000 7fd044c00640 1 -- 172.21.15.47:0/4023408273 >> 172.21.15.47:0/4023408273 conn(0x7fd0400ff110 msgr2=0x7fd0400ff990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:25:36.282 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:25:36.279+0000 7fd044c00640 1 -- 172.21.15.47:0/4023408273 shutdown_connections 2024-03-21T10:25:36.282 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:25:36.279+0000 7fd044c00640 1 -- 172.21.15.47:0/4023408273 wait complete. 2024-03-21T10:25:37.463 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:25:37 smithi047 ceph-mon[31547]: pgmap v11: 129 pgs: 129 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 77 KiB/s wr, 51 op/s 2024-03-21T10:25:37.479 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:25:37 smithi100 ceph-mon[35193]: pgmap v11: 129 pgs: 129 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 77 KiB/s wr, 51 op/s 2024-03-21T10:25:37.489 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:25:37 smithi157 ceph-mon[35581]: pgmap v11: 129 pgs: 129 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 77 KiB/s wr, 51 op/s 2024-03-21T10:25:39.463 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:25:39 smithi047 ceph-mon[31547]: pgmap v12: 129 pgs: 129 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 77 KiB/s wr, 55 op/s 2024-03-21T10:25:39.479 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:25:39 smithi100 ceph-mon[35193]: pgmap v12: 129 pgs: 129 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 77 KiB/s wr, 55 op/s 2024-03-21T10:25:39.489 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:25:39 smithi157 ceph-mon[35581]: pgmap v12: 129 pgs: 129 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 77 KiB/s wr, 55 op/s 2024-03-21T10:25:40.188 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-21T10:25:40.188 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-21T10:25:40.425 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:40.423+0000 7f509e200640 1 -- 172.21.15.47:0/1605161204 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5098103250 msgr2=0x7f509810b730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:25:40.425 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:40.423+0000 7f509e200640 1 --2- 172.21.15.47:0/1605161204 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5098103250 0x7f509810b730 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f5084009900 tx=0x7f508402bcc0 comp rx=0 tx=0).stop 2024-03-21T10:25:40.425 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:40.423+0000 7f509e200640 1 -- 172.21.15.47:0/1605161204 shutdown_connections 2024-03-21T10:25:40.425 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:40.423+0000 7f509e200640 1 --2- 172.21.15.47:0/1605161204 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f509810bd00 0x7f509810e130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:25:40.426 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:40.423+0000 7f509e200640 1 --2- 172.21.15.47:0/1605161204 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5098103250 0x7f509810b730 unknown :-1 s=CLOSED pgs=533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:25:40.426 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:40.423+0000 7f509e200640 1 --2- 172.21.15.47:0/1605161204 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5098102930 0x7f5098102d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:25:40.426 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:40.423+0000 7f509e200640 1 -- 172.21.15.47:0/1605161204 >> 172.21.15.47:0/1605161204 conn(0x7f5098077700 msgr2=0x7f5098077b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:25:40.426 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:40.423+0000 7f509e200640 1 -- 172.21.15.47:0/1605161204 shutdown_connections 2024-03-21T10:25:40.426 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:40.423+0000 7f509e200640 1 -- 172.21.15.47:0/1605161204 wait complete. 2024-03-21T10:25:40.428 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:40.424+0000 7f509e200640 1 Processor -- start 2024-03-21T10:25:40.428 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:40.425+0000 7f509e200640 1 -- start start 2024-03-21T10:25:40.428 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:40.425+0000 7f509e200640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5098102930 0x7f5098197f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:25:40.428 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:40.425+0000 7f509e200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5098103250 0x7f5098198450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:25:40.428 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:40.425+0000 7f509e200640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f509810bd00 0x7f509819c7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:25:40.428 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:40.425+0000 7f509e200640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5098111940 con 0x7f5098102930 2024-03-21T10:25:40.428 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:40.425+0000 7f509e200640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50981117c0 con 0x7f5098103250 2024-03-21T10:25:40.428 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:40.425+0000 7f509e200640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5098111ac0 con 0x7f509810bd00 2024-03-21T10:25:40.428 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:40.425+0000 7f5097400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5098102930 0x7f5098197f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:25:40.428 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:40.425+0000 7f5097400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5098102930 0x7f5098197f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:55926/0 (socket says 172.21.15.47:55926) 2024-03-21T10:25:40.428 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:40.425+0000 7f5097400640 1 -- 172.21.15.47:0/407496896 learned_addr learned my addr 172.21.15.47:0/407496896 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:25:40.428 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:40.425+0000 7f5096a00640 1 --2- 172.21.15.47:0/407496896 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5098103250 0x7f5098198450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:25:40.428 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:40.426+0000 7f5097e00640 1 --2- 172.21.15.47:0/407496896 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f509810bd00 0x7f509819c7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:25:40.429 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:40.426+0000 7f5096a00640 1 -- 172.21.15.47:0/407496896 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f509810bd00 msgr2=0x7f509819c7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:25:40.429 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:40.426+0000 7f5096a00640 1 --2- 172.21.15.47:0/407496896 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f509810bd00 0x7f509819c7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:25:40.429 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:40.426+0000 7f5096a00640 1 -- 172.21.15.47:0/407496896 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5098102930 msgr2=0x7f5098197f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:25:40.429 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:40.426+0000 7f5096a00640 1 --2- 172.21.15.47:0/407496896 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5098102930 0x7f5098197f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:25:40.429 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:40.426+0000 7f5096a00640 1 -- 172.21.15.47:0/407496896 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f509819cf00 con 0x7f5098103250 2024-03-21T10:25:40.429 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:40.426+0000 7f5097400640 1 --2- 172.21.15.47:0/407496896 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5098102930 0x7f5098197f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T10:25:40.429 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:40.426+0000 7f5096a00640 1 --2- 172.21.15.47:0/407496896 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5098103250 0x7f5098198450 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f5084036040 tx=0x7f50840045e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:25:40.430 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:40.427+0000 7f508fe00640 1 -- 172.21.15.47:0/407496896 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5084043020 con 0x7f5098103250 2024-03-21T10:25:40.430 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:40.427+0000 7f509e200640 1 -- 172.21.15.47:0/407496896 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f509819d190 con 0x7f5098103250 2024-03-21T10:25:40.430 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:40.427+0000 7f508fe00640 1 -- 172.21.15.47:0/407496896 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f5084004e00 con 0x7f5098103250 2024-03-21T10:25:40.430 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:40.427+0000 7f508fe00640 1 -- 172.21.15.47:0/407496896 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50840038e0 con 0x7f5098103250 2024-03-21T10:25:40.430 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:40.427+0000 7f509e200640 1 -- 172.21.15.47:0/407496896 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f50981a4a80 con 0x7f5098103250 2024-03-21T10:25:40.432 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:40.431+0000 7f508fe00640 1 -- 172.21.15.47:0/407496896 <== mon.2 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 25) v1 ==== 138030+0+0 (secure 0 0 0) 0x7f5084046020 con 0x7f5098103250 2024-03-21T10:25:40.432 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:40.431+0000 7f50a0e72740 1 Processor -- start 2024-03-21T10:25:40.432 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:40.431+0000 7f50a0e72740 1 -- start start 2024-03-21T10:25:40.434 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:40.431+0000 7f50a0e72740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ad6fe83c00 0x55ad6fe82950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:25:40.434 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:40.432+0000 7f50a0e72740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ad6fe82f20 0x55ad6fe833a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:25:40.434 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:40.432+0000 7f50a0e72740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ad6fe906a0 0x55ad6fe92a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:25:40.434 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:40.432+0000 7f50a0e72740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ad6fde7250 con 0x55ad6fe82f20 2024-03-21T10:25:40.434 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:40.432+0000 7f50a0e72740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ad6fd8fa30 con 0x55ad6fe906a0 2024-03-21T10:25:40.434 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:40.432+0000 7f50a0e72740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ad6fde32f0 con 0x55ad6fe83c00 2024-03-21T10:25:40.435 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:40.432+0000 7f5097400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ad6fe82f20 0x55ad6fe833a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:25:40.435 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:40.432+0000 7f5097400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ad6fe82f20 0x55ad6fe833a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:55930/0 (socket says 172.21.15.47:55930) 2024-03-21T10:25:40.435 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:40.432+0000 7f5096a00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ad6fe906a0 0x55ad6fe92a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:25:40.435 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:40.432+0000 7f5097400640 1 -- 172.21.15.47:0/1905950532 learned_addr learned my addr 172.21.15.47:0/1905950532 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:25:40.435 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:40.432+0000 7f5097e00640 1 --2- 172.21.15.47:0/1905950532 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ad6fe83c00 0x55ad6fe82950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:25:40.435 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:40.432+0000 7f508fe00640 1 --2- 172.21.15.47:0/407496896 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f507c0a31a0 0x7f507c0a5660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:25:40.435 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:40.433+0000 7f5097400640 1 -- 172.21.15.47:0/1905950532 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ad6fe83c00 msgr2=0x55ad6fe82950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:25:40.435 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:40.433+0000 7f5097400640 1 --2- 172.21.15.47:0/1905950532 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ad6fe83c00 0x55ad6fe82950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:25:40.435 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:40.433+0000 7f5097400640 1 -- 172.21.15.47:0/1905950532 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ad6fe906a0 msgr2=0x55ad6fe92a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:25:40.435 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:40.433+0000 7f5097400640 1 --2- 172.21.15.47:0/1905950532 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ad6fe906a0 0x55ad6fe92a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:25:40.435 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:40.433+0000 7f5097400640 1 -- 172.21.15.47:0/1905950532 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ad6fd7a690 con 0x55ad6fe82f20 2024-03-21T10:25:40.435 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:40.433+0000 7f508fe00640 1 -- 172.21.15.47:0/407496896 <== mon.2 v2:172.21.15.100:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 9474+0+0 (secure 0 0 0) 0x7f50840f1b70 con 0x7f5098103250 2024-03-21T10:25:40.435 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:40.433+0000 7f5097e00640 1 --2- 172.21.15.47:0/407496896 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f507c0a31a0 0x7f507c0a5660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:25:40.439 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:40.433+0000 7f5097400640 1 --2- 172.21.15.47:0/1905950532 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ad6fe82f20 0x55ad6fe833a0 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f5088011630 tx=0x7f5088011af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=63b51ec4011adf2d server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:25:40.439 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:40.433+0000 7f5083e00640 1 -- 172.21.15.47:0/1905950532 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f508801e070 con 0x55ad6fe82f20 2024-03-21T10:25:40.439 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:40.434+0000 7f5083e00640 1 -- 172.21.15.47:0/1905950532 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f508800fba0 con 0x55ad6fe82f20 2024-03-21T10:25:40.439 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:40.434+0000 7f5083e00640 1 -- 172.21.15.47:0/1905950532 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5088019c80 con 0x55ad6fe82f20 2024-03-21T10:25:40.439 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:40.434+0000 7f50a0e72740 1 -- 172.21.15.47:0/1905950532 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ad6fe82f20 msgr2=0x55ad6fe833a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:25:40.439 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:40.434+0000 7f5097e00640 1 --2- 172.21.15.47:0/407496896 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f507c0a31a0 0x7f507c0a5660 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f50900099a0 tx=0x7f5090004a40 comp rx=0 tx=0).ready entity=mgr.15687 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:25:40.439 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:40.434+0000 7f50a0e72740 1 --2- 172.21.15.47:0/1905950532 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ad6fe82f20 0x55ad6fe833a0 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f5088011630 tx=0x7f5088011af0 comp rx=0 tx=0).stop 2024-03-21T10:25:40.439 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:40.434+0000 7f50a0e72740 1 -- 172.21.15.47:0/1905950532 shutdown_connections 2024-03-21T10:25:40.440 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:40.434+0000 7f50a0e72740 1 --2- 172.21.15.47:0/1905950532 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ad6fe906a0 0x55ad6fe92a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:25:40.440 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:40.434+0000 7f50a0e72740 1 --2- 172.21.15.47:0/1905950532 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ad6fe82f20 0x55ad6fe833a0 unknown :-1 s=CLOSED pgs=534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:25:40.440 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:40.434+0000 7f50a0e72740 1 --2- 172.21.15.47:0/1905950532 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ad6fe83c00 0x55ad6fe82950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:25:40.440 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:40.434+0000 7f50a0e72740 1 -- 172.21.15.47:0/1905950532 >> 172.21.15.47:0/1905950532 conn(0x55ad6fe4a200 msgr2=0x55ad6fe7d970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:25:40.440 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:40.434+0000 7f50a0e72740 1 -- 172.21.15.47:0/1905950532 shutdown_connections 2024-03-21T10:25:40.440 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:40.434+0000 7f50a0e72740 1 -- 172.21.15.47:0/1905950532 wait complete. 2024-03-21T10:25:40.440 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:40.435+0000 7f50a0e72740 1 -- start start 2024-03-21T10:25:40.440 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:40.435+0000 7f50a0e72740 1 Processor -- start 2024-03-21T10:25:40.440 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:40.435+0000 7f50a0e72740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ad6fe82f20 0x55ad6fe8fcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:25:40.440 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:40.436+0000 7f50a0e72740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ad6fe83c00 0x55ad6fe8d080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:25:40.440 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:40.436+0000 7f50a0e72740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ad6fe906a0 0x55ad6fe8d5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:25:40.440 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:40.436+0000 7f50a0e72740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ad6fe93510 con 0x55ad6fe82f20 2024-03-21T10:25:40.441 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:40.436+0000 7f50a0e72740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ad6fda2e80 con 0x55ad6fe906a0 2024-03-21T10:25:40.441 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:40.436+0000 7f50a0e72740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ad6fe93690 con 0x55ad6fe83c00 2024-03-21T10:25:40.441 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:40.436+0000 7f50a0e72740 10 client.0 _ll_register_callbacks cb 0x55ad6fdb5130 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T10:25:40.441 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:40.436+0000 7f5097400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ad6fe82f20 0x55ad6fe8fcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:25:40.441 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:40.436+0000 7f5097400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ad6fe82f20 0x55ad6fe8fcf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:55938/0 (socket says 172.21.15.47:55938) 2024-03-21T10:25:40.441 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:40.436+0000 7f5097400640 1 -- 172.21.15.47:0/2684831004 learned_addr learned my addr 172.21.15.47:0/2684831004 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:25:40.441 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:40.436+0000 7f5097e00640 1 --2- 172.21.15.47:0/2684831004 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ad6fe906a0 0x55ad6fe8d5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:25:40.441 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:40.436+0000 7f5096a00640 1 --2- 172.21.15.47:0/2684831004 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ad6fe83c00 0x55ad6fe8d080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:25:40.441 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:40.436+0000 7f5097400640 1 -- 172.21.15.47:0/2684831004 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ad6fe83c00 msgr2=0x55ad6fe8d080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:25:40.441 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:40.436+0000 7f5097400640 1 --2- 172.21.15.47:0/2684831004 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ad6fe83c00 0x55ad6fe8d080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:25:40.441 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:40.436+0000 7f5097400640 1 -- 172.21.15.47:0/2684831004 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ad6fe906a0 msgr2=0x55ad6fe8d5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:25:40.441 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:40.436+0000 7f5097400640 1 --2- 172.21.15.47:0/2684831004 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ad6fe906a0 0x55ad6fe8d5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:25:40.441 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:40.436+0000 7f5097400640 1 -- 172.21.15.47:0/2684831004 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55ad6fbc57d0 con 0x55ad6fe82f20 2024-03-21T10:25:40.441 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:40.436+0000 7f5096a00640 1 --2- 172.21.15.47:0/2684831004 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ad6fe83c00 0x55ad6fe8d080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T10:25:40.442 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:40.437+0000 7f5097400640 1 --2- 172.21.15.47:0/2684831004 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ad6fe82f20 0x55ad6fe8fcf0 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f5088009810 tx=0x7f5088009f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e63109b22226a1b5 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:25:40.442 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:40.437+0000 7f5080c00640 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-03-21T10:25:40.442 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:40.437+0000 7f5080c00640 1 -- 172.21.15.47:0/2684831004 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5088026020 con 0x55ad6fe82f20 2024-03-21T10:25:40.442 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:40.437+0000 7f5080c00640 1 -- 172.21.15.47:0/2684831004 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 9474+0+0 (secure 0 0 0) 0x7f508800fd50 con 0x55ad6fe82f20 2024-03-21T10:25:40.442 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:40.437+0000 7f50a0e72740 1 -- 172.21.15.47:0/2684831004 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f508800fba0 con 0x55ad6fe82f20 2024-03-21T10:25:40.442 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:40.437+0000 7f5080c00640 1 -- 172.21.15.47:0/2684831004 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5088031050 con 0x55ad6fe82f20 2024-03-21T10:25:40.442 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:40.438+0000 7f50a0e72740 10 client.15795 fetch_fsmap learned FSMap version 30 2024-03-21T10:25:40.442 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:40.438+0000 7f50a0e72740 1 -- 172.21.15.47:0/2684831004 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x55ad6fafb3f0 con 0x55ad6fe82f20 2024-03-21T10:25:40.444 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:40.438+0000 7f5080c00640 1 -- 172.21.15.47:0/2684831004 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f508800fba0 con 0x55ad6fe82f20 2024-03-21T10:25:40.444 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:40.438+0000 7f5080c00640 1 -- 172.21.15.47:0/2684831004 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 30) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f5088021050 con 0x55ad6fe82f20 2024-03-21T10:25:40.444 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:40.438+0000 7f50a0e72740 10 client.15795 fetch_fsmap finished waiting for FSMap version 30 2024-03-21T10:25:40.444 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:40.438+0000 7f50a0e72740 10 client.15795 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-21T10:25:40.444 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:40.438+0000 7f50a0e72740 20 client.15795 populate_metadata read hostname 'smithi047' 2024-03-21T10:25:40.444 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:40.438+0000 7f50a0e72740 1 --2- 172.21.15.47:0/2684831004 >> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] conn(0x55ad6fea04b0 0x55ad6fea28a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:25:40.444 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:40.438+0000 7f50a0e72740 4 client.15795 mds_command: new command op to 34535 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T10:25:40.444 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:40.438+0000 7f50a0e72740 1 -- 172.21.15.47:0/2684831004 --> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55ad6fdf4790 con 0x55ad6fea04b0 2024-03-21T10:25:40.444 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:40.439+0000 7f5096a00640 1 --2- 172.21.15.47:0/2684831004 >> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] conn(0x55ad6fea04b0 0x55ad6fea28a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:25:40.444 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:40.439+0000 7f5096a00640 1 --2- 172.21.15.47:0/2684831004 >> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] conn(0x55ad6fea04b0 0x55ad6fea28a0 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=60a38fda21e5ce63 server_cookie=ae7bf0f65c248aec in_seq=0 out_seq=0 2024-03-21T10:25:40.444 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:40.439+0000 7f5080c00640 10 client.15795 ms_handle_connect on v2:172.21.15.100:6838/4173168440 2024-03-21T10:25:40.445 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:40.441+0000 7f5080c00640 1 -- 172.21.15.47:0/2684831004 <== mds.0 v2:172.21.15.100:6838/4173168440 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55ad6fdf4790 con 0x55ad6fea04b0 2024-03-21T10:25:40.445 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:40.441+0000 7f5080c00640 10 client.15795 handle_command_reply: tid=0 2024-03-21T10:25:40.445 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:40.441+0000 7f50a0e72740 1 client.15795 shutdown 2024-03-21T10:25:40.445 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:40.441+0000 7f50a0e72740 1 -- 172.21.15.47:0/2684831004 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ad6fe82f20 msgr2=0x55ad6fe8fcf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:25:40.445 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:40.441+0000 7f50a0e72740 1 --2- 172.21.15.47:0/2684831004 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ad6fe82f20 0x55ad6fe8fcf0 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f5088009810 tx=0x7f5088009f10 comp rx=0 tx=0).stop 2024-03-21T10:25:40.445 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:40.441+0000 7f50a0e72740 1 -- 172.21.15.47:0/2684831004 shutdown_connections 2024-03-21T10:25:40.445 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:40.441+0000 7f50a0e72740 1 --2- 172.21.15.47:0/2684831004 >> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] conn(0x55ad6fea04b0 0x55ad6fea28a0 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:25:40.445 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:40.441+0000 7f50a0e72740 1 --2- 172.21.15.47:0/2684831004 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ad6fe906a0 0x55ad6fe8d5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:25:40.445 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:40.441+0000 7f50a0e72740 1 --2- 172.21.15.47:0/2684831004 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ad6fe83c00 0x55ad6fe8d080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:25:40.445 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:40.441+0000 7f50a0e72740 1 --2- 172.21.15.47:0/2684831004 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ad6fe82f20 0x55ad6fe8fcf0 unknown :-1 s=CLOSED pgs=535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:25:40.445 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:40.441+0000 7f50a0e72740 1 -- 172.21.15.47:0/2684831004 >> 172.21.15.47:0/2684831004 conn(0x55ad6fe4a200 msgr2=0x55ad6fe7f6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:25:40.445 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:40.441+0000 7f5080c00640 0 client.15795 ms_handle_reset on v2:172.21.15.100:6838/4173168440 2024-03-21T10:25:40.445 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:40.441+0000 7f50a0e72740 1 -- 172.21.15.47:0/2684831004 shutdown_connections 2024-03-21T10:25:40.445 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:40.441+0000 7f50a0e72740 1 -- 172.21.15.47:0/2684831004 wait complete. 2024-03-21T10:25:40.446 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:40.442+0000 7f50a0e72740 20 client.15795 trim_cache size 0 max 16384 2024-03-21T10:25:40.479 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:40.476+0000 7f50a0e72740 1 Processor -- start 2024-03-21T10:25:40.479 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:40.477+0000 7f50a0e72740 1 -- start start 2024-03-21T10:25:40.479 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:40.477+0000 7f50a0e72740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ad6fe82f20 0x55ad6fe8f7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:25:40.479 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:40.477+0000 7f50a0e72740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ad6fe83c00 0x55ad6fe8fcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:25:40.479 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:40.477+0000 7f50a0e72740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ad6fe906a0 0x55ad6fe91a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:25:40.479 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:40.477+0000 7f50a0e72740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50981ac6c0 con 0x55ad6fe82f20 2024-03-21T10:25:40.479 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:40.477+0000 7f50a0e72740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50981ae040 con 0x55ad6fe83c00 2024-03-21T10:25:40.479 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:40.477+0000 7f50a0e72740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50981acd20 con 0x55ad6fe906a0 2024-03-21T10:25:40.479 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:40.477+0000 7f5097400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ad6fe82f20 0x55ad6fe8f7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:25:40.479 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:40.477+0000 7f5097400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ad6fe82f20 0x55ad6fe8f7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:55948/0 (socket says 172.21.15.47:55948) 2024-03-21T10:25:40.480 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:40.477+0000 7f5097400640 1 -- 172.21.15.47:0/1993105544 learned_addr learned my addr 172.21.15.47:0/1993105544 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:25:40.480 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:40.478+0000 7f5096a00640 1 --2- 172.21.15.47:0/1993105544 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ad6fe83c00 0x55ad6fe8fcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:25:40.481 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:40.478+0000 7f5097e00640 1 --2- 172.21.15.47:0/1993105544 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ad6fe906a0 0x55ad6fe91a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:25:40.481 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:40.478+0000 7f5097400640 1 -- 172.21.15.47:0/1993105544 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ad6fe906a0 msgr2=0x55ad6fe91a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:25:40.481 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:40.478+0000 7f5097400640 1 --2- 172.21.15.47:0/1993105544 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ad6fe906a0 0x55ad6fe91a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:25:40.482 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:40.478+0000 7f5097400640 1 -- 172.21.15.47:0/1993105544 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ad6fe83c00 msgr2=0x55ad6fe8fcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:25:40.482 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:40.478+0000 7f5097400640 1 --2- 172.21.15.47:0/1993105544 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ad6fe83c00 0x55ad6fe8fcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:25:40.482 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:40.478+0000 7f5097400640 1 -- 172.21.15.47:0/1993105544 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ad6fabb0f0 con 0x55ad6fe82f20 2024-03-21T10:25:40.482 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:40.478+0000 7f5096a00640 1 --2- 172.21.15.47:0/1993105544 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ad6fe83c00 0x55ad6fe8fcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T10:25:40.482 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:40.478+0000 7f5097400640 1 --2- 172.21.15.47:0/1993105544 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ad6fe82f20 0x55ad6fe8f7b0 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f5088009530 tx=0x7f5088009560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1749e83b8d33ed3d server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:25:40.482 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:40.478+0000 7f5083400640 1 -- 172.21.15.47:0/1993105544 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5088016070 con 0x55ad6fe82f20 2024-03-21T10:25:40.482 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:40.479+0000 7f5083400640 1 -- 172.21.15.47:0/1993105544 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f5088004ad0 con 0x55ad6fe82f20 2024-03-21T10:25:40.482 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:40.479+0000 7f5083400640 1 -- 172.21.15.47:0/1993105544 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5088026040 con 0x55ad6fe82f20 2024-03-21T10:25:40.482 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:40.479+0000 7f50a0e72740 1 -- 172.21.15.47:0/1993105544 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ad6fe82f20 msgr2=0x55ad6fe8f7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:25:40.482 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:40.479+0000 7f50a0e72740 1 --2- 172.21.15.47:0/1993105544 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ad6fe82f20 0x55ad6fe8f7b0 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f5088009530 tx=0x7f5088009560 comp rx=0 tx=0).stop 2024-03-21T10:25:40.482 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:40.479+0000 7f50a0e72740 1 -- 172.21.15.47:0/1993105544 shutdown_connections 2024-03-21T10:25:40.482 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:40.479+0000 7f50a0e72740 1 --2- 172.21.15.47:0/1993105544 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ad6fe906a0 0x55ad6fe91a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:25:40.482 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:40.479+0000 7f50a0e72740 1 --2- 172.21.15.47:0/1993105544 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ad6fe83c00 0x55ad6fe8fcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:25:40.482 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:40.479+0000 7f50a0e72740 1 --2- 172.21.15.47:0/1993105544 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ad6fe82f20 0x55ad6fe8f7b0 unknown :-1 s=CLOSED pgs=536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:25:40.483 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:40.479+0000 7f50a0e72740 1 -- 172.21.15.47:0/1993105544 >> 172.21.15.47:0/1993105544 conn(0x55ad6fe4a200 msgr2=0x55ad6fe7d4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:25:40.483 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:40.479+0000 7f50a0e72740 1 -- 172.21.15.47:0/1993105544 shutdown_connections 2024-03-21T10:25:40.483 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:40.479+0000 7f50a0e72740 1 -- 172.21.15.47:0/1993105544 wait complete. 2024-03-21T10:25:40.483 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:40.480+0000 7f50a0e72740 1 -- start start 2024-03-21T10:25:40.485 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:40.480+0000 7f50a0e72740 1 Processor -- start 2024-03-21T10:25:40.486 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:40.481+0000 7f50a0e72740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ad6fe82f20 0x55ad6fe985c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:25:40.486 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:40.481+0000 7f50a0e72740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ad6fe83c00 0x55ad6fe93940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:25:40.486 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:40.481+0000 7f50a0e72740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ad6fe906a0 0x55ad6fe93e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:25:40.486 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:40.481+0000 7f50a0e72740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50981ad9e0 con 0x55ad6fe82f20 2024-03-21T10:25:40.486 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:40.481+0000 7f50a0e72740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50981ad380 con 0x55ad6fe83c00 2024-03-21T10:25:40.486 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:40.481+0000 7f50a0e72740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50981aa880 con 0x55ad6fe906a0 2024-03-21T10:25:40.486 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:40.481+0000 7f50a0e72740 10 client.0 _ll_register_callbacks cb 0x7f508801c070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T10:25:40.486 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:40.481+0000 7f5097400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ad6fe82f20 0x55ad6fe985c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:25:40.486 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:40.481+0000 7f5097400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ad6fe82f20 0x55ad6fe985c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:55960/0 (socket says 172.21.15.47:55960) 2024-03-21T10:25:40.486 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:40.481+0000 7f5097400640 1 -- 172.21.15.47:0/401093586 learned_addr learned my addr 172.21.15.47:0/401093586 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:25:40.486 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:40.481+0000 7f5096a00640 1 --2- 172.21.15.47:0/401093586 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ad6fe83c00 0x55ad6fe93940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:25:40.486 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:40.482+0000 7f5097e00640 1 --2- 172.21.15.47:0/401093586 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ad6fe906a0 0x55ad6fe93e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:25:40.486 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:40.482+0000 7f5097400640 1 -- 172.21.15.47:0/401093586 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ad6fe906a0 msgr2=0x55ad6fe93e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:25:40.487 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:40.482+0000 7f5097400640 1 --2- 172.21.15.47:0/401093586 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ad6fe906a0 0x55ad6fe93e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:25:40.487 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:40.482+0000 7f5097400640 1 -- 172.21.15.47:0/401093586 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ad6fe83c00 msgr2=0x55ad6fe93940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:25:40.487 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:40.482+0000 7f5097400640 1 --2- 172.21.15.47:0/401093586 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ad6fe83c00 0x55ad6fe93940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:25:40.487 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:40.482+0000 7f5097400640 1 -- 172.21.15.47:0/401093586 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55ad6fe94520 con 0x55ad6fe82f20 2024-03-21T10:25:40.487 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:40.482+0000 7f5096a00640 1 --2- 172.21.15.47:0/401093586 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ad6fe83c00 0x55ad6fe93940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T10:25:40.487 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:40.482+0000 7f5097400640 1 --2- 172.21.15.47:0/401093586 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ad6fe82f20 0x55ad6fe985c0 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f5088014040 tx=0x7f5088009f70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6801f0ef0edba504 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:25:40.487 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:40.482+0000 7f5080c00640 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-03-21T10:25:40.487 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:40.482+0000 7f5080c00640 1 -- 172.21.15.47:0/401093586 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5088016070 con 0x55ad6fe82f20 2024-03-21T10:25:40.487 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:40.483+0000 7f5080c00640 1 -- 172.21.15.47:0/401093586 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 9474+0+0 (secure 0 0 0) 0x7f508800fa70 con 0x55ad6fe82f20 2024-03-21T10:25:40.487 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:40.483+0000 7f50a0e72740 1 -- 172.21.15.47:0/401093586 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f5088004ad0 con 0x55ad6fe82f20 2024-03-21T10:25:40.487 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:40.483+0000 7f5080c00640 1 -- 172.21.15.47:0/401093586 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5088043050 con 0x55ad6fe82f20 2024-03-21T10:25:40.487 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:40.483+0000 7f50a0e72740 10 client.15801 fetch_fsmap learned FSMap version 30 2024-03-21T10:25:40.487 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:40.483+0000 7f50a0e72740 1 -- 172.21.15.47:0/401093586 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x55ad6fefba90 con 0x55ad6fe82f20 2024-03-21T10:25:40.488 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:40.483+0000 7f5080c00640 1 -- 172.21.15.47:0/401093586 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f5088004ad0 con 0x55ad6fe82f20 2024-03-21T10:25:40.488 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:40.483+0000 7f5080c00640 1 -- 172.21.15.47:0/401093586 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 30) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f5088021070 con 0x55ad6fe82f20 2024-03-21T10:25:40.488 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:40.483+0000 7f50a0e72740 10 client.15801 fetch_fsmap finished waiting for FSMap version 30 2024-03-21T10:25:40.488 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:40.483+0000 7f50a0e72740 10 client.15801 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-21T10:25:40.488 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:40.483+0000 7f50a0e72740 20 client.15801 populate_metadata read hostname 'smithi047' 2024-03-21T10:25:40.488 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:40.483+0000 7f50a0e72740 1 --2- 172.21.15.47:0/401093586 >> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] conn(0x55ad6fefc0c0 0x55ad6fefe4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:25:40.488 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:40.484+0000 7f50a0e72740 4 client.15801 mds_command: new command op to 34535 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-21T10:25:40.488 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:40.484+0000 7f50a0e72740 1 -- 172.21.15.47:0/401093586 --> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55ad6fdeeae0 con 0x55ad6fefc0c0 2024-03-21T10:25:40.488 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:40.484+0000 7f5096a00640 1 --2- 172.21.15.47:0/401093586 >> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] conn(0x55ad6fefc0c0 0x55ad6fefe4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:25:40.489 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:40.484+0000 7f5096a00640 1 --2- 172.21.15.47:0/401093586 >> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] conn(0x55ad6fefc0c0 0x55ad6fefe4e0 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9ef1f020bfd2bb1f server_cookie=23e067e33407d8b7 in_seq=0 out_seq=0 2024-03-21T10:25:40.489 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:40.484+0000 7f5080c00640 10 client.15801 ms_handle_connect on v2:172.21.15.100:6838/4173168440 2024-03-21T10:25:40.489 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:40.485+0000 7f5080c00640 1 -- 172.21.15.47:0/401093586 <== mds.0 v2:172.21.15.100:6838/4173168440 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x55ad6fdeeae0 con 0x55ad6fefc0c0 2024-03-21T10:25:40.489 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:40.485+0000 7f5080c00640 10 client.15801 handle_command_reply: tid=0 2024-03-21T10:25:40.490 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:40.485+0000 7f50a0e72740 1 client.15801 shutdown 2024-03-21T10:25:40.490 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:40.485+0000 7f50a0e72740 1 -- 172.21.15.47:0/401093586 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ad6fe82f20 msgr2=0x55ad6fe985c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:25:40.490 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:40.485+0000 7f50a0e72740 1 --2- 172.21.15.47:0/401093586 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ad6fe82f20 0x55ad6fe985c0 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f5088014040 tx=0x7f5088009f70 comp rx=0 tx=0).stop 2024-03-21T10:25:40.490 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:40.485+0000 7f50a0e72740 1 -- 172.21.15.47:0/401093586 shutdown_connections 2024-03-21T10:25:40.490 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:40.485+0000 7f50a0e72740 1 --2- 172.21.15.47:0/401093586 >> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] conn(0x55ad6fefc0c0 0x55ad6fefe4e0 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:25:40.490 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:40.485+0000 7f50a0e72740 1 --2- 172.21.15.47:0/401093586 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ad6fe906a0 0x55ad6fe93e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:25:40.490 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:40.485+0000 7f50a0e72740 1 --2- 172.21.15.47:0/401093586 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ad6fe83c00 0x55ad6fe93940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:25:40.490 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:40.485+0000 7f50a0e72740 1 --2- 172.21.15.47:0/401093586 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ad6fe82f20 0x55ad6fe985c0 unknown :-1 s=CLOSED pgs=537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:25:40.490 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:40.485+0000 7f5080c00640 0 client.15801 ms_handle_reset on v2:172.21.15.100:6838/4173168440 2024-03-21T10:25:40.490 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:40.485+0000 7f50a0e72740 1 -- 172.21.15.47:0/401093586 >> 172.21.15.47:0/401093586 conn(0x55ad6fe4a200 msgr2=0x55ad6fe8b550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:25:40.490 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:40.485+0000 7f50a0e72740 1 -- 172.21.15.47:0/401093586 shutdown_connections 2024-03-21T10:25:40.490 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:40.486+0000 7f50a0e72740 1 -- 172.21.15.47:0/401093586 wait complete. 2024-03-21T10:25:40.490 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:40.486+0000 7f50a0e72740 20 client.15801 trim_cache size 0 max 16384 2024-03-21T10:25:40.491 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-03-21T10:25:40.491 INFO:teuthology.orchestra.run.smithi047.stdout: "status": "no active scrubs running", 2024-03-21T10:25:40.491 INFO:teuthology.orchestra.run.smithi047.stdout: "scrubs": {} 2024-03-21T10:25:40.491 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-03-21T10:25:40.492 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:40.486+0000 7f509e200640 1 -- 172.21.15.47:0/407496896 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f507c0a31a0 msgr2=0x7f507c0a5660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:25:40.492 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:40.486+0000 7f509e200640 1 --2- 172.21.15.47:0/407496896 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f507c0a31a0 0x7f507c0a5660 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f50900099a0 tx=0x7f5090004a40 comp rx=0 tx=0).stop 2024-03-21T10:25:40.492 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:40.486+0000 7f509e200640 1 -- 172.21.15.47:0/407496896 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5098103250 msgr2=0x7f5098198450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:25:40.492 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:40.486+0000 7f509e200640 1 --2- 172.21.15.47:0/407496896 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5098103250 0x7f5098198450 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f5084036040 tx=0x7f50840045e0 comp rx=0 tx=0).stop 2024-03-21T10:25:40.492 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:40.487+0000 7f509e200640 1 -- 172.21.15.47:0/407496896 shutdown_connections 2024-03-21T10:25:40.492 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:40.487+0000 7f509e200640 1 --2- 172.21.15.47:0/407496896 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f509810bd00 0x7f509819c7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:25:40.492 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:40.487+0000 7f509e200640 1 --2- 172.21.15.47:0/407496896 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f507c0a31a0 0x7f507c0a5660 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:25:40.492 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:40.487+0000 7f509e200640 1 --2- 172.21.15.47:0/407496896 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5098103250 0x7f5098198450 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:25:40.492 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:40.487+0000 7f509e200640 1 --2- 172.21.15.47:0/407496896 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5098102930 0x7f5098197f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:25:40.492 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:40.487+0000 7f509e200640 1 -- 172.21.15.47:0/407496896 >> 172.21.15.47:0/407496896 conn(0x7f5098077700 msgr2=0x7f50980fd770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:25:40.492 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:40.487+0000 7f509e200640 1 -- 172.21.15.47:0/407496896 shutdown_connections 2024-03-21T10:25:40.492 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:40.487+0000 7f509e200640 1 -- 172.21.15.47:0/407496896 wait complete. 2024-03-21T10:25:40.507 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-03-21T10:25:40.507 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-21T10:25:40.723 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:40.721+0000 7fc77b200640 1 -- 172.21.15.47:0/614873598 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc77c105800 msgr2=0x7fc77c105c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:25:40.723 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:40.721+0000 7fc77b200640 1 --2- 172.21.15.47:0/614873598 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc77c105800 0x7fc77c105c90 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fc774009ca0 tx=0x7fc77402c190 comp rx=0 tx=0).stop 2024-03-21T10:25:40.723 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:40.721+0000 7fc77b200640 1 -- 172.21.15.47:0/614873598 shutdown_connections 2024-03-21T10:25:40.723 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:40.721+0000 7fc77b200640 1 --2- 172.21.15.47:0/614873598 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc77c105800 0x7fc77c105c90 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:25:40.723 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:40.721+0000 7fc77b200640 1 --2- 172.21.15.47:0/614873598 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc77c100d20 0x7fc77c1051e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:25:40.723 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:40.721+0000 7fc77b200640 1 --2- 172.21.15.47:0/614873598 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc77c100400 0x7fc77c1007e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:25:40.723 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:40.721+0000 7fc77b200640 1 -- 172.21.15.47:0/614873598 >> 172.21.15.47:0/614873598 conn(0x7fc77c0fbe50 msgr2=0x7fc77c0fe270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:25:40.723 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:40.721+0000 7fc77b200640 1 -- 172.21.15.47:0/614873598 shutdown_connections 2024-03-21T10:25:40.723 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:40.721+0000 7fc77b200640 1 -- 172.21.15.47:0/614873598 wait complete. 2024-03-21T10:25:40.724 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:40.722+0000 7fc77b200640 1 Processor -- start 2024-03-21T10:25:40.724 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:40.722+0000 7fc77b200640 1 -- start start 2024-03-21T10:25:40.726 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:40.723+0000 7fc77b200640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc77c100400 0x7fc77c19e230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:25:40.726 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:40.723+0000 7fc77b200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc77c100d20 0x7fc77c19e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:25:40.726 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:40.723+0000 7fc77b200640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc77c105800 0x7fc77c1a2af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:25:40.726 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:40.723+0000 7fc77b200640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc77c117c90 con 0x7fc77c105800 2024-03-21T10:25:40.726 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:40.723+0000 7fc77b200640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc77c117b10 con 0x7fc77c100d20 2024-03-21T10:25:40.726 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:40.723+0000 7fc77b200640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc77c117e10 con 0x7fc77c100400 2024-03-21T10:25:40.726 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:40.723+0000 7fc77a800640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc77c105800 0x7fc77c1a2af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:25:40.726 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:40.723+0000 7fc77a800640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc77c105800 0x7fc77c1a2af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:55980/0 (socket says 172.21.15.47:55980) 2024-03-21T10:25:40.726 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:40.723+0000 7fc77a800640 1 -- 172.21.15.47:0/2658942557 learned_addr learned my addr 172.21.15.47:0/2658942557 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:25:40.726 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:40.724+0000 7fc773e00640 1 --2- 172.21.15.47:0/2658942557 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc77c100d20 0x7fc77c19e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:25:40.726 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:40.724+0000 7fc779e00640 1 --2- 172.21.15.47:0/2658942557 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc77c100400 0x7fc77c19e230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:25:40.727 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:40.724+0000 7fc77a800640 1 -- 172.21.15.47:0/2658942557 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc77c100400 msgr2=0x7fc77c19e230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:25:40.727 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:40.724+0000 7fc77a800640 1 --2- 172.21.15.47:0/2658942557 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc77c100400 0x7fc77c19e230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:25:40.727 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:40.724+0000 7fc77a800640 1 -- 172.21.15.47:0/2658942557 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc77c100d20 msgr2=0x7fc77c19e770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:25:40.727 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:40.724+0000 7fc77a800640 1 --2- 172.21.15.47:0/2658942557 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc77c100d20 0x7fc77c19e770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:25:40.727 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:40.724+0000 7fc77a800640 1 -- 172.21.15.47:0/2658942557 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc77c1a3220 con 0x7fc77c105800 2024-03-21T10:25:40.727 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:40.724+0000 7fc77a800640 1 --2- 172.21.15.47:0/2658942557 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc77c105800 0x7fc77c1a2af0 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7fc774032590 tx=0x7fc774035cc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:25:40.728 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:40.725+0000 7fc772a00640 1 -- 172.21.15.47:0/2658942557 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc774039040 con 0x7fc77c105800 2024-03-21T10:25:40.728 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:40.725+0000 7fc772a00640 1 -- 172.21.15.47:0/2658942557 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fc774044070 con 0x7fc77c105800 2024-03-21T10:25:40.728 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:40.725+0000 7fc77b200640 1 -- 172.21.15.47:0/2658942557 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc77c1a34b0 con 0x7fc77c105800 2024-03-21T10:25:40.728 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:40.725+0000 7fc772a00640 1 -- 172.21.15.47:0/2658942557 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc774032850 con 0x7fc77c105800 2024-03-21T10:25:40.728 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:40.725+0000 7fc77b200640 1 -- 172.21.15.47:0/2658942557 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc77c1aada0 con 0x7fc77c105800 2024-03-21T10:25:40.728 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:40.726+0000 7fc77b200640 1 -- 172.21.15.47:0/2658942557 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc77c068800 con 0x7fc77c105800 2024-03-21T10:25:40.730 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:40.728+0000 7fc772a00640 1 -- 172.21.15.47:0/2658942557 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 25) v1 ==== 138030+0+0 (secure 0 0 0) 0x7fc774004040 con 0x7fc77c105800 2024-03-21T10:25:40.732 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:40.730+0000 7fc772a00640 1 --2- 172.21.15.47:0/2658942557 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7fc7600a3120 0x7fc7600a55e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:25:40.732 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:40.730+0000 7fc772a00640 1 -- 172.21.15.47:0/2658942557 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 9474+0+0 (secure 0 0 0) 0x7fc7740f2030 con 0x7fc77c105800 2024-03-21T10:25:40.732 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:40.731+0000 7fc779e00640 1 --2- 172.21.15.47:0/2658942557 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7fc7600a3120 0x7fc7600a55e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:25:40.733 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:40.731+0000 7fc779e00640 1 --2- 172.21.15.47:0/2658942557 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7fc7600a3120 0x7fc7600a55e0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fc76c009bd0 tx=0x7fc76c005f90 comp rx=0 tx=0).ready entity=mgr.15687 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:25:40.736 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:40.735+0000 7fc772a00640 1 -- 172.21.15.47:0/2658942557 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fc7740be6a0 con 0x7fc77c105800 2024-03-21T10:25:40.985 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:40.983+0000 7fc77b200640 1 -- 172.21.15.47:0/2658942557 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc77c1aaf70 con 0x7fc77c105800 2024-03-21T10:25:40.985 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:40.984+0000 7fc772a00640 1 -- 172.21.15.47:0/2658942557 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) v1 ==== 94+0+11471 (secure 0 0 0) 0x7fc7740bdfa0 con 0x7fc77c105800 2024-03-21T10:25:40.986 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-21T10:25:40.986 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":30,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14511,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6835/735724037","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6834","nonce":735724037},{"type":"v1","addr":"172.21.15.47:6835","nonce":735724037}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14529,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6837/2248191724","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6836","nonce":2248191724},{"type":"v1","addr":"172.21.15.47:6837","nonce":2248191724}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":30,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-21T10:21:58.329063+0000","modified":"2024-03-21T10:24:58.786909+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":34535,"mds_1":34493,"mds_2":34454,"mds_3":34427,"mds_4":14619},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14604":{"gid":14604,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.47:6839/2881335356","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6838","nonce":2881335356},{"type":"v1","addr":"172.21.15.47:6839","nonce":2881335356}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_14619":{"gid":14619,"name":"a","rank":4,"incarnation":27,"state":"up:active","state_seq":11,"addr":"172.21.15.47:6841/3401578978","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6840","nonce":3401578978},{"type":"v1","addr":"172.21.15.47:6841","nonce":3401578978}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24406":{"gid":24406,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.157:6833/4000010136","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":4000010136},{"type":"v1","addr":"172.21.15.157:6833","nonce":4000010136}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24457":{"gid":24457,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.157:6835/3902868651","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":3902868651},{"type":"v1","addr":"172.21.15.157:6835","nonce":3902868651}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24475":{"gid":24475,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.157:6837/1908087204","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6836","nonce":1908087204},{"type":"v1","addr":"172.21.15.157:6837","nonce":1908087204}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24499":{"gid":24499,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.157:6839/4064242876","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6838","nonce":4064242876},{"type":"v1","addr":"172.21.15.157:6839","nonce":4064242876}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_34427":{"gid":34427,"name":"k","rank":3,"incarnation":25,"state":"up:active","state_seq":20,"addr":"172.21.15.100:6833/2723302038","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6832","nonce":2723302038},{"type":"v1","addr":"172.21.15.100:6833","nonce":2723302038}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_34454":{"gid":34454,"name":"e","rank":2,"incarnation":22,"state":"up:active","state_seq":17,"addr":"172.21.15.100:6835/2597849957","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":2597849957},{"type":"v1","addr":"172.21.15.100:6835","nonce":2597849957}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_34493":{"gid":34493,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":13,"addr":"172.21.15.100:6837/3497644162","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6836","nonce":3497644162},{"type":"v1","addr":"172.21.15.100:6837","nonce":3497644162}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_34535":{"gid":34535,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.100:6839/4173168440","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6838","nonce":4173168440},{"type":"v1","addr":"172.21.15.100:6839","nonce":4173168440}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":34535,"qdb_cluster":[14619,34427,34454,34493,34535]},"id":1}]} 2024-03-21T10:25:40.986 INFO:teuthology.orchestra.run.smithi047.stderr:dumped fsmap epoch 30 2024-03-21T10:25:40.988 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:40.987+0000 7fc77b200640 1 -- 172.21.15.47:0/2658942557 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7fc7600a3120 msgr2=0x7fc7600a55e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:25:40.988 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:40.987+0000 7fc77b200640 1 --2- 172.21.15.47:0/2658942557 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7fc7600a3120 0x7fc7600a55e0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fc76c009bd0 tx=0x7fc76c005f90 comp rx=0 tx=0).stop 2024-03-21T10:25:40.988 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:40.987+0000 7fc77b200640 1 -- 172.21.15.47:0/2658942557 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc77c105800 msgr2=0x7fc77c1a2af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:25:40.988 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:40.987+0000 7fc77b200640 1 --2- 172.21.15.47:0/2658942557 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc77c105800 0x7fc77c1a2af0 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7fc774032590 tx=0x7fc774035cc0 comp rx=0 tx=0).stop 2024-03-21T10:25:40.989 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:40.987+0000 7fc77b200640 1 -- 172.21.15.47:0/2658942557 shutdown_connections 2024-03-21T10:25:40.989 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:40.987+0000 7fc77b200640 1 --2- 172.21.15.47:0/2658942557 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc77c105800 0x7fc77c1a2af0 unknown :-1 s=CLOSED pgs=538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:25:40.989 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:40.987+0000 7fc77b200640 1 --2- 172.21.15.47:0/2658942557 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7fc7600a3120 0x7fc7600a55e0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:25:40.989 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:40.987+0000 7fc77b200640 1 --2- 172.21.15.47:0/2658942557 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc77c100d20 0x7fc77c19e770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:25:40.989 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:40.987+0000 7fc77b200640 1 --2- 172.21.15.47:0/2658942557 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc77c100400 0x7fc77c19e230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:25:40.989 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:40.987+0000 7fc77b200640 1 -- 172.21.15.47:0/2658942557 >> 172.21.15.47:0/2658942557 conn(0x7fc77c0fbe50 msgr2=0x7fc77c0fcec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:25:40.989 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:40.987+0000 7fc77b200640 1 -- 172.21.15.47:0/2658942557 shutdown_connections 2024-03-21T10:25:40.990 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:40.987+0000 7fc77b200640 1 -- 172.21.15.47:0/2658942557 wait complete. 2024-03-21T10:25:41.012 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-21T10:25:41.013 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-03-21T10:25:41.249 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:41.247+0000 7f98e2800640 1 -- 172.21.15.47:0/309719362 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f98dc1026d0 msgr2=0x7f98dc102ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:25:41.250 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:41.247+0000 7f98e2800640 1 --2- 172.21.15.47:0/309719362 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f98dc1026d0 0x7f98dc102ab0 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f98d0009900 tx=0x7f98d002bcd0 comp rx=0 tx=0).stop 2024-03-21T10:25:41.250 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:41.248+0000 7f98e2800640 1 -- 172.21.15.47:0/309719362 shutdown_connections 2024-03-21T10:25:41.250 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:41.248+0000 7f98e2800640 1 --2- 172.21.15.47:0/309719362 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f98dc10baf0 0x7f98dc10df90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:25:41.250 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:41.248+0000 7f98e2800640 1 --2- 172.21.15.47:0/309719362 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f98dc102ff0 0x7f98dc10b4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:25:41.250 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:41.248+0000 7f98e2800640 1 --2- 172.21.15.47:0/309719362 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f98dc1026d0 0x7f98dc102ab0 unknown :-1 s=CLOSED pgs=539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:25:41.250 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:41.248+0000 7f98e2800640 1 -- 172.21.15.47:0/309719362 >> 172.21.15.47:0/309719362 conn(0x7f98dc0fbef0 msgr2=0x7f98dc0fe310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:25:41.250 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:41.248+0000 7f98e2800640 1 -- 172.21.15.47:0/309719362 shutdown_connections 2024-03-21T10:25:41.250 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:41.248+0000 7f98e2800640 1 -- 172.21.15.47:0/309719362 wait complete. 2024-03-21T10:25:41.252 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:41.249+0000 7f98e2800640 1 Processor -- start 2024-03-21T10:25:41.252 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:41.249+0000 7f98e2800640 1 -- start start 2024-03-21T10:25:41.252 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:41.249+0000 7f98e2800640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f98dc1026d0 0x7f98dc10a820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:25:41.252 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:41.249+0000 7f98e2800640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f98dc102ff0 0x7f98dc10ad60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:25:41.252 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:41.249+0000 7f98e2800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f98dc10baf0 0x7f98dc1a2b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:25:41.252 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:41.249+0000 7f98e2800640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98dc1117d0 con 0x7f98dc1026d0 2024-03-21T10:25:41.252 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:41.249+0000 7f98e2800640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98dc111650 con 0x7f98dc10baf0 2024-03-21T10:25:41.252 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:41.249+0000 7f98e2800640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98dc111950 con 0x7f98dc102ff0 2024-03-21T10:25:41.252 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:41.250+0000 7f98e1400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f98dc1026d0 0x7f98dc10a820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:25:41.252 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:41.250+0000 7f98e1400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f98dc1026d0 0x7f98dc10a820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:56000/0 (socket says 172.21.15.47:56000) 2024-03-21T10:25:41.252 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:41.250+0000 7f98e1400640 1 -- 172.21.15.47:0/1724084671 learned_addr learned my addr 172.21.15.47:0/1724084671 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:25:41.252 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:41.250+0000 7f98e1e00640 1 --2- 172.21.15.47:0/1724084671 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f98dc10baf0 0x7f98dc1a2b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:25:41.253 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:41.250+0000 7f98e0a00640 1 --2- 172.21.15.47:0/1724084671 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f98dc102ff0 0x7f98dc10ad60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:25:41.253 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:41.250+0000 7f98e1400640 1 -- 172.21.15.47:0/1724084671 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f98dc102ff0 msgr2=0x7f98dc10ad60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:25:41.253 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:41.250+0000 7f98e1400640 1 --2- 172.21.15.47:0/1724084671 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f98dc102ff0 0x7f98dc10ad60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:25:41.253 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:41.250+0000 7f98e1400640 1 -- 172.21.15.47:0/1724084671 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f98dc10baf0 msgr2=0x7f98dc1a2b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:25:41.253 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:41.250+0000 7f98e1400640 1 --2- 172.21.15.47:0/1724084671 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f98dc10baf0 0x7f98dc1a2b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:25:41.253 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:41.250+0000 7f98e1400640 1 -- 172.21.15.47:0/1724084671 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98dc1a32b0 con 0x7f98dc1026d0 2024-03-21T10:25:41.253 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:41.250+0000 7f98e1e00640 1 --2- 172.21.15.47:0/1724084671 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f98dc10baf0 0x7f98dc1a2b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T10:25:41.254 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:41.251+0000 7f98e1400640 1 --2- 172.21.15.47:0/1724084671 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f98dc1026d0 0x7f98dc10a820 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f98d0000c00 tx=0x7f98d002c760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:25:41.254 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:41.251+0000 7f98da000640 1 -- 172.21.15.47:0/1724084671 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98d0043020 con 0x7f98dc1026d0 2024-03-21T10:25:41.254 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:41.251+0000 7f98da000640 1 -- 172.21.15.47:0/1724084671 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f98d002dc60 con 0x7f98dc1026d0 2024-03-21T10:25:41.254 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:41.251+0000 7f98e2800640 1 -- 172.21.15.47:0/1724084671 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f98dc1a3540 con 0x7f98dc1026d0 2024-03-21T10:25:41.254 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:41.251+0000 7f98da000640 1 -- 172.21.15.47:0/1724084671 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98d00039e0 con 0x7f98dc1026d0 2024-03-21T10:25:41.254 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:41.251+0000 7f98e2800640 1 -- 172.21.15.47:0/1724084671 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f98dc1aae30 con 0x7f98dc1026d0 2024-03-21T10:25:41.257 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:41.255+0000 7f98e6bff740 1 Processor -- start 2024-03-21T10:25:41.258 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:41.256+0000 7f98e6bff740 1 -- start start 2024-03-21T10:25:41.258 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:41.256+0000 7f98da000640 1 -- 172.21.15.47:0/1724084671 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 25) v1 ==== 138030+0+0 (secure 0 0 0) 0x7f98d0046020 con 0x7f98dc1026d0 2024-03-21T10:25:41.258 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:41.256+0000 7f98e6bff740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5628430d68b0 0x5628430d6c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:25:41.258 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:41.256+0000 7f98e6bff740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5628430d7260 0x5628430df810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:25:41.258 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:41.256+0000 7f98e6bff740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5628430dfde0 0x5628430e2230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:25:41.258 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:41.256+0000 7f98e6bff740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x56284303a2c0 con 0x5628430dfde0 2024-03-21T10:25:41.258 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:41.256+0000 7f98e6bff740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x562842fe2c90 con 0x5628430d68b0 2024-03-21T10:25:41.258 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:41.256+0000 7f98e6bff740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x562843036360 con 0x5628430d7260 2024-03-21T10:25:41.260 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:41.257+0000 7f98e1e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5628430d68b0 0x5628430d6c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:25:41.260 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:41.257+0000 7f98e1e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5628430d68b0 0x5628430d6c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:37350/0 (socket says 172.21.15.47:37350) 2024-03-21T10:25:41.260 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:41.257+0000 7f98e1e00640 1 -- 172.21.15.47:0/809583554 learned_addr learned my addr 172.21.15.47:0/809583554 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:25:41.260 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:41.257+0000 7f98e0a00640 1 --2- 172.21.15.47:0/809583554 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5628430dfde0 0x5628430e2230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:25:41.260 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:41.257+0000 7f98e1400640 1 --2- 172.21.15.47:0/809583554 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5628430d7260 0x5628430df810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:25:41.260 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:41.257+0000 7f98e1e00640 1 -- 172.21.15.47:0/809583554 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5628430d7260 msgr2=0x5628430df810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:25:41.261 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:41.257+0000 7f98e1e00640 1 --2- 172.21.15.47:0/809583554 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5628430d7260 0x5628430df810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:25:41.261 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:41.257+0000 7f98e1e00640 1 -- 172.21.15.47:0/809583554 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5628430dfde0 msgr2=0x5628430e2230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:25:41.261 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:41.257+0000 7f98e1e00640 1 --2- 172.21.15.47:0/809583554 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5628430dfde0 0x5628430e2230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:25:41.261 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:41.257+0000 7f98e1e00640 1 -- 172.21.15.47:0/809583554 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562843007080 con 0x5628430d68b0 2024-03-21T10:25:41.261 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:41.257+0000 7f98e1e00640 1 --2- 172.21.15.47:0/809583554 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5628430d68b0 0x5628430d6c90 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f98d4011790 tx=0x7f98d4011c50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9206afb321d9c412 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:25:41.261 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:41.258+0000 7f98ce600640 1 -- 172.21.15.47:0/809583554 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98d4003c30 con 0x5628430d68b0 2024-03-21T10:25:41.261 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:41.258+0000 7f98ce600640 1 -- 172.21.15.47:0/809583554 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f98d400fe50 con 0x5628430d68b0 2024-03-21T10:25:41.261 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:41.258+0000 7f98ce600640 1 -- 172.21.15.47:0/809583554 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98d4004820 con 0x5628430d68b0 2024-03-21T10:25:41.261 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:41.258+0000 7f98da000640 1 --2- 172.21.15.47:0/1724084671 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f98c40a31a0 0x7f98c40a5660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:25:41.261 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:41.258+0000 7f98da000640 1 -- 172.21.15.47:0/1724084671 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 9474+0+0 (secure 0 0 0) 0x7f98d00f1c60 con 0x7f98dc1026d0 2024-03-21T10:25:41.261 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:41.258+0000 7f98e1400640 1 --2- 172.21.15.47:0/1724084671 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f98c40a31a0 0x7f98c40a5660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:25:41.261 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:41.259+0000 7f98e6bff740 1 -- 172.21.15.47:0/809583554 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5628430d68b0 msgr2=0x5628430d6c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:25:41.261 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:41.259+0000 7f98e6bff740 1 --2- 172.21.15.47:0/809583554 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5628430d68b0 0x5628430d6c90 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f98d4011790 tx=0x7f98d4011c50 comp rx=0 tx=0).stop 2024-03-21T10:25:41.261 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:41.259+0000 7f98e6bff740 1 -- 172.21.15.47:0/809583554 shutdown_connections 2024-03-21T10:25:41.262 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:41.259+0000 7f98e6bff740 1 --2- 172.21.15.47:0/809583554 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5628430dfde0 0x5628430e2230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:25:41.262 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:41.259+0000 7f98e6bff740 1 --2- 172.21.15.47:0/809583554 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5628430d7260 0x5628430df810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:25:41.262 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:41.259+0000 7f98e6bff740 1 --2- 172.21.15.47:0/809583554 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5628430d68b0 0x5628430d6c90 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:25:41.262 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:41.259+0000 7f98e6bff740 1 -- 172.21.15.47:0/809583554 >> 172.21.15.47:0/809583554 conn(0x5628430d0f80 msgr2=0x5628430d1360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:25:41.265 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:41.259+0000 7f98e6bff740 1 -- 172.21.15.47:0/809583554 shutdown_connections 2024-03-21T10:25:41.265 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:41.259+0000 7f98e6bff740 1 -- 172.21.15.47:0/809583554 wait complete. 2024-03-21T10:25:41.265 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:41.259+0000 7f98e1400640 1 --2- 172.21.15.47:0/1724084671 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f98c40a31a0 0x7f98c40a5660 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f98d00ef040 tx=0x7f98d0009260 comp rx=0 tx=0).ready entity=mgr.15687 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:25:41.265 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:41.260+0000 7f98e6bff740 1 -- start start 2024-03-21T10:25:41.266 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:41.260+0000 7f98e6bff740 1 Processor -- start 2024-03-21T10:25:41.266 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:41.260+0000 7f98e6bff740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5628430d68b0 0x5628430df740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:25:41.266 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:41.260+0000 7f98e6bff740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5628430d7260 0x5628430dcad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:25:41.266 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:41.260+0000 7f98e6bff740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5628430dfde0 0x5628430dd0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:25:41.266 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:41.260+0000 7f98e6bff740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x5628430e2d50 con 0x5628430d68b0 2024-03-21T10:25:41.266 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:41.260+0000 7f98e6bff740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x562842ff60d0 con 0x5628430dfde0 2024-03-21T10:25:41.266 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:41.260+0000 7f98e6bff740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x5628430e2ed0 con 0x5628430d7260 2024-03-21T10:25:41.266 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:41.260+0000 7f98e6bff740 10 client.0 _ll_register_callbacks cb 0x562842ffae80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T10:25:41.266 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:41.261+0000 7f98e1e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5628430d68b0 0x5628430df740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:25:41.266 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:41.261+0000 7f98e1e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5628430d68b0 0x5628430df740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:56018/0 (socket says 172.21.15.47:56018) 2024-03-21T10:25:41.266 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:41.261+0000 7f98e1e00640 1 -- 172.21.15.47:0/1353640879 learned_addr learned my addr 172.21.15.47:0/1353640879 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:25:41.266 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:41.261+0000 7f98e0a00640 1 --2- 172.21.15.47:0/1353640879 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5628430d7260 0x5628430dcad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:25:41.266 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:41.261+0000 7f98e1e00640 1 -- 172.21.15.47:0/1353640879 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5628430d7260 msgr2=0x5628430dcad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:25:41.267 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:41.261+0000 7f98e1e00640 1 --2- 172.21.15.47:0/1353640879 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5628430d7260 0x5628430dcad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:25:41.267 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:41.261+0000 7f98e1e00640 1 -- 172.21.15.47:0/1353640879 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5628430dfde0 msgr2=0x5628430dd0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:25:41.267 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:41.261+0000 7f98e1e00640 1 --2- 172.21.15.47:0/1353640879 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5628430dfde0 0x5628430dd0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:25:41.267 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:41.261+0000 7f98e1e00640 1 -- 172.21.15.47:0/1353640879 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x562842d4e3f0 con 0x5628430d68b0 2024-03-21T10:25:41.267 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:41.261+0000 7f98e0a00640 1 --2- 172.21.15.47:0/1353640879 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5628430d7260 0x5628430dcad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T10:25:41.267 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:41.261+0000 7f98e1e00640 1 --2- 172.21.15.47:0/1353640879 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5628430d68b0 0x5628430df740 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f98d400a010 tx=0x7f98d4005ae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7b1af7e862a9693a server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:25:41.267 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:41.261+0000 7f98c2a00640 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-03-21T10:25:41.267 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:41.262+0000 7f98c2a00640 1 -- 172.21.15.47:0/1353640879 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98d401f040 con 0x5628430d68b0 2024-03-21T10:25:41.267 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:41.262+0000 7f98c2a00640 1 -- 172.21.15.47:0/1353640879 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 9474+0+0 (secure 0 0 0) 0x7f98d4011240 con 0x5628430d68b0 2024-03-21T10:25:41.267 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:41.262+0000 7f98e6bff740 1 -- 172.21.15.47:0/1353640879 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f98d400fe50 con 0x5628430d68b0 2024-03-21T10:25:41.267 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:41.262+0000 7f98c2a00640 1 -- 172.21.15.47:0/1353640879 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f98d4003a30 con 0x5628430d68b0 2024-03-21T10:25:41.267 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:41.262+0000 7f98e6bff740 10 client.15819 fetch_fsmap learned FSMap version 30 2024-03-21T10:25:41.267 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:41.263+0000 7f98e6bff740 1 -- 172.21.15.47:0/1353640879 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x562842e187d0 con 0x5628430d68b0 2024-03-21T10:25:41.267 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:41.263+0000 7f98c2a00640 1 -- 172.21.15.47:0/1353640879 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f98d400fe50 con 0x5628430d68b0 2024-03-21T10:25:41.268 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:41.263+0000 7f98c2a00640 1 -- 172.21.15.47:0/1353640879 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 30) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f98d402e030 con 0x5628430d68b0 2024-03-21T10:25:41.268 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:41.263+0000 7f98e6bff740 10 client.15819 fetch_fsmap finished waiting for FSMap version 30 2024-03-21T10:25:41.268 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:41.263+0000 7f98e6bff740 10 client.15819 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.a 2024-03-21T10:25:41.268 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:41.263+0000 7f98e6bff740 20 client.15819 populate_metadata read hostname 'smithi047' 2024-03-21T10:25:41.268 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:41.263+0000 7f98e6bff740 1 --2- 172.21.15.47:0/1353640879 >> [v2:172.21.15.47:6840/3401578978,v1:172.21.15.47:6841/3401578978] conn(0x5628430efbe0 0x5628430f1fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:25:41.268 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:41.263+0000 7f98e6bff740 4 client.15819 mds_command: new command op to 14619 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T10:25:41.268 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:41.263+0000 7f98e6bff740 1 -- 172.21.15.47:0/1353640879 --> [v2:172.21.15.47:6840/3401578978,v1:172.21.15.47:6841/3401578978] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x562843047800 con 0x5628430efbe0 2024-03-21T10:25:41.268 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:41.264+0000 7f98e0a00640 1 --2- 172.21.15.47:0/1353640879 >> [v2:172.21.15.47:6840/3401578978,v1:172.21.15.47:6841/3401578978] conn(0x5628430efbe0 0x5628430f1fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:25:41.269 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:41.264+0000 7f98e0a00640 1 --2- 172.21.15.47:0/1353640879 >> [v2:172.21.15.47:6840/3401578978,v1:172.21.15.47:6841/3401578978] conn(0x5628430efbe0 0x5628430f1fd0 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=7a32cc21d1262d6b server_cookie=6f13acc2ad1931e7 in_seq=0 out_seq=0 2024-03-21T10:25:41.269 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:41.264+0000 7f98c2a00640 10 client.15819 ms_handle_connect on v2:172.21.15.47:6840/3401578978 2024-03-21T10:25:41.269 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:41.265+0000 7f98c2a00640 1 -- 172.21.15.47:0/1353640879 <== mds.4 v2:172.21.15.47:6840/3401578978 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x562843047800 con 0x5628430efbe0 2024-03-21T10:25:41.270 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:41.265+0000 7f98c2a00640 10 client.15819 handle_command_reply: tid=0 2024-03-21T10:25:41.270 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:41.265+0000 7f98e6bff740 1 client.15819 shutdown 2024-03-21T10:25:41.270 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:41.265+0000 7f98e6bff740 1 -- 172.21.15.47:0/1353640879 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5628430d68b0 msgr2=0x5628430df740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:25:41.270 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:41.265+0000 7f98e6bff740 1 --2- 172.21.15.47:0/1353640879 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5628430d68b0 0x5628430df740 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f98d400a010 tx=0x7f98d4005ae0 comp rx=0 tx=0).stop 2024-03-21T10:25:41.270 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:41.266+0000 7f98e6bff740 1 -- 172.21.15.47:0/1353640879 shutdown_connections 2024-03-21T10:25:41.270 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:41.266+0000 7f98e6bff740 1 --2- 172.21.15.47:0/1353640879 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5628430dfde0 0x5628430dd0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:25:41.270 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:41.266+0000 7f98e6bff740 1 --2- 172.21.15.47:0/1353640879 >> [v2:172.21.15.47:6840/3401578978,v1:172.21.15.47:6841/3401578978] conn(0x5628430efbe0 0x5628430f1fd0 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:25:41.270 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:41.266+0000 7f98e6bff740 1 --2- 172.21.15.47:0/1353640879 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5628430d7260 0x5628430dcad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:25:41.270 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:41.266+0000 7f98e6bff740 1 --2- 172.21.15.47:0/1353640879 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5628430d68b0 0x5628430df740 unknown :-1 s=CLOSED pgs=541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:25:41.270 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:41.266+0000 7f98c2a00640 0 client.15819 ms_handle_reset on v2:172.21.15.47:6840/3401578978 2024-03-21T10:25:41.270 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:41.266+0000 7f98e6bff740 1 -- 172.21.15.47:0/1353640879 >> 172.21.15.47:0/1353640879 conn(0x5628430d0f80 msgr2=0x5628430d2cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:25:41.270 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:41.266+0000 7f98e6bff740 1 -- 172.21.15.47:0/1353640879 shutdown_connections 2024-03-21T10:25:41.271 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:41.266+0000 7f98e6bff740 1 -- 172.21.15.47:0/1353640879 wait complete. 2024-03-21T10:25:41.271 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:41.266+0000 7f98e6bff740 20 client.15819 trim_cache size 0 max 16384 2024-03-21T10:25:41.281 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:41.279+0000 7f98e6bff740 1 Processor -- start 2024-03-21T10:25:41.281 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:41.280+0000 7f98e6bff740 1 -- start start 2024-03-21T10:25:41.281 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:41.280+0000 7f98e6bff740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5628430d68b0 0x5628430d0160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:25:41.282 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:41.280+0000 7f98e6bff740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5628430d7260 0x5628430df200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:25:41.282 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:41.280+0000 7f98e6bff740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5628430dfde0 0x5628430df740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:25:41.282 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:41.280+0000 7f98e6bff740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98dc1b4d80 con 0x5628430d7260 2024-03-21T10:25:41.282 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:41.280+0000 7f98e6bff740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98dc1b6700 con 0x5628430d68b0 2024-03-21T10:25:41.282 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:41.280+0000 7f98e6bff740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98dc1b53e0 con 0x5628430dfde0 2024-03-21T10:25:41.284 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:41.280+0000 7f98e0a00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5628430d7260 0x5628430df200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:25:41.284 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:41.280+0000 7f98e0a00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5628430d7260 0x5628430df200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:56022/0 (socket says 172.21.15.47:56022) 2024-03-21T10:25:41.284 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:41.280+0000 7f98e0a00640 1 -- 172.21.15.47:0/2007228233 learned_addr learned my addr 172.21.15.47:0/2007228233 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:25:41.284 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:41.280+0000 7f98e1e00640 1 --2- 172.21.15.47:0/2007228233 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5628430d68b0 0x5628430d0160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:25:41.284 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:41.281+0000 7f98e1e00640 1 --2- 172.21.15.47:0/2007228233 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5628430dfde0 0x5628430df740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:25:41.284 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:41.281+0000 7f98e0a00640 1 -- 172.21.15.47:0/2007228233 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5628430dfde0 msgr2=0x5628430df740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:25:41.284 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:41.281+0000 7f98e0a00640 1 --2- 172.21.15.47:0/2007228233 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5628430dfde0 0x5628430df740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:25:41.285 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:41.281+0000 7f98e0a00640 1 -- 172.21.15.47:0/2007228233 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5628430d68b0 msgr2=0x5628430d0160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:25:41.285 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:41.281+0000 7f98e0a00640 1 --2- 172.21.15.47:0/2007228233 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5628430d68b0 0x5628430d0160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:25:41.285 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:41.281+0000 7f98e0a00640 1 -- 172.21.15.47:0/2007228233 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5628430d5a60 con 0x5628430d7260 2024-03-21T10:25:41.285 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:41.281+0000 7f98e0a00640 1 --2- 172.21.15.47:0/2007228233 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5628430d7260 0x5628430df200 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f98c8005fd0 tx=0x7f98c800b4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9b2339665e9075ce server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:25:41.285 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:41.281+0000 7f98cdc00640 1 -- 172.21.15.47:0/2007228233 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98c801f070 con 0x5628430d7260 2024-03-21T10:25:41.285 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:41.281+0000 7f98cdc00640 1 -- 172.21.15.47:0/2007228233 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f98c801b2c0 con 0x5628430d7260 2024-03-21T10:25:41.285 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:41.281+0000 7f98cdc00640 1 -- 172.21.15.47:0/2007228233 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98c8011a80 con 0x5628430d7260 2024-03-21T10:25:41.285 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:41.282+0000 7f98e6bff740 1 -- 172.21.15.47:0/2007228233 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5628430d7260 msgr2=0x5628430df200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:25:41.285 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:41.282+0000 7f98e6bff740 1 --2- 172.21.15.47:0/2007228233 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5628430d7260 0x5628430df200 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f98c8005fd0 tx=0x7f98c800b4d0 comp rx=0 tx=0).stop 2024-03-21T10:25:41.285 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:41.282+0000 7f98e6bff740 1 -- 172.21.15.47:0/2007228233 shutdown_connections 2024-03-21T10:25:41.285 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:41.282+0000 7f98e6bff740 1 --2- 172.21.15.47:0/2007228233 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5628430dfde0 0x5628430df740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:25:41.285 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:41.282+0000 7f98e6bff740 1 --2- 172.21.15.47:0/2007228233 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5628430d7260 0x5628430df200 unknown :-1 s=CLOSED pgs=542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:25:41.285 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:41.282+0000 7f98e6bff740 1 --2- 172.21.15.47:0/2007228233 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5628430d68b0 0x5628430d0160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:25:41.285 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:41.282+0000 7f98e6bff740 1 -- 172.21.15.47:0/2007228233 >> 172.21.15.47:0/2007228233 conn(0x5628430d0f80 msgr2=0x5628430cc060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:25:41.285 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:41.282+0000 7f98e6bff740 1 -- 172.21.15.47:0/2007228233 shutdown_connections 2024-03-21T10:25:41.286 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:41.282+0000 7f98e6bff740 1 -- 172.21.15.47:0/2007228233 wait complete. 2024-03-21T10:25:41.286 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:41.283+0000 7f98e6bff740 1 -- start start 2024-03-21T10:25:41.286 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:41.283+0000 7f98e6bff740 1 Processor -- start 2024-03-21T10:25:41.289 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:41.283+0000 7f98e6bff740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5628430d68b0 0x5628430e5cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:25:41.289 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:41.283+0000 7f98e6bff740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5628430d7260 0x5628430e6200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:25:41.289 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:41.283+0000 7f98e6bff740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5628430dfde0 0x56284314a290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:25:41.289 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:41.283+0000 7f98e6bff740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98dc1b60a0 con 0x5628430dfde0 2024-03-21T10:25:41.289 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:41.283+0000 7f98e6bff740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98dc1b5a40 con 0x5628430d7260 2024-03-21T10:25:41.290 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:41.283+0000 7f98e6bff740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98dc1b2f40 con 0x5628430d68b0 2024-03-21T10:25:41.290 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:41.283+0000 7f98e6bff740 10 client.0 _ll_register_callbacks cb 0x7f98dc069860 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T10:25:41.290 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:41.284+0000 7f98e1e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5628430dfde0 0x56284314a290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:25:41.290 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:41.284+0000 7f98e1e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5628430dfde0 0x56284314a290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:56038/0 (socket says 172.21.15.47:56038) 2024-03-21T10:25:41.290 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:41.284+0000 7f98e1e00640 1 -- 172.21.15.47:0/1201756243 learned_addr learned my addr 172.21.15.47:0/1201756243 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:25:41.290 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:41.284+0000 7f98e0a00640 1 --2- 172.21.15.47:0/1201756243 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5628430d7260 0x5628430e6200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:25:41.290 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:41.284+0000 7f98e1e00640 1 --2- 172.21.15.47:0/1201756243 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5628430d68b0 0x5628430e5cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:25:41.291 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:41.284+0000 7f98e1e00640 1 -- 172.21.15.47:0/1201756243 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5628430d68b0 msgr2=0x5628430e5cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:25:41.291 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:41.284+0000 7f98e1e00640 1 --2- 172.21.15.47:0/1201756243 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5628430d68b0 0x5628430e5cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:25:41.291 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:41.284+0000 7f98e1e00640 1 -- 172.21.15.47:0/1201756243 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5628430d7260 msgr2=0x5628430e6200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:25:41.291 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:41.284+0000 7f98e1e00640 1 --2- 172.21.15.47:0/1201756243 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5628430d7260 0x5628430e6200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:25:41.291 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:41.284+0000 7f98e1e00640 1 -- 172.21.15.47:0/1201756243 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56284314a820 con 0x5628430dfde0 2024-03-21T10:25:41.291 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:41.284+0000 7f98e0a00640 1 --2- 172.21.15.47:0/1201756243 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5628430d7260 0x5628430e6200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T10:25:41.291 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:41.284+0000 7f98e1e00640 1 --2- 172.21.15.47:0/1201756243 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5628430dfde0 0x56284314a290 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f98d4005c10 tx=0x7f98d401c040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=35990edb50d9583a server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:25:41.291 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:41.284+0000 7f98c2a00640 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-03-21T10:25:41.291 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:41.285+0000 7f98c2a00640 1 -- 172.21.15.47:0/1201756243 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98d4009500 con 0x5628430dfde0 2024-03-21T10:25:41.291 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:41.285+0000 7f98e6bff740 1 -- 172.21.15.47:0/1201756243 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f98c801b2c0 con 0x5628430dfde0 2024-03-21T10:25:41.291 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:41.285+0000 7f98c2a00640 1 -- 172.21.15.47:0/1201756243 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 9474+0+0 (secure 0 0 0) 0x7f98d401a650 con 0x5628430dfde0 2024-03-21T10:25:41.291 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:41.285+0000 7f98c2a00640 1 -- 172.21.15.47:0/1201756243 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f98d4011040 con 0x5628430dfde0 2024-03-21T10:25:41.291 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:41.285+0000 7f98e6bff740 10 client.15825 fetch_fsmap learned FSMap version 30 2024-03-21T10:25:41.292 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:41.285+0000 7f98e6bff740 1 -- 172.21.15.47:0/1201756243 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x56284314bb20 con 0x5628430dfde0 2024-03-21T10:25:41.292 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:41.285+0000 7f98c2a00640 1 -- 172.21.15.47:0/1201756243 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f98c801b2c0 con 0x5628430dfde0 2024-03-21T10:25:41.292 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:41.285+0000 7f98c2a00640 1 -- 172.21.15.47:0/1201756243 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 30) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f98d402e070 con 0x5628430dfde0 2024-03-21T10:25:41.292 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:41.285+0000 7f98e6bff740 10 client.15825 fetch_fsmap finished waiting for FSMap version 30 2024-03-21T10:25:41.292 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:41.285+0000 7f98e6bff740 10 client.15825 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.a 2024-03-21T10:25:41.292 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:41.285+0000 7f98e6bff740 20 client.15825 populate_metadata read hostname 'smithi047' 2024-03-21T10:25:41.292 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:41.286+0000 7f98e6bff740 1 --2- 172.21.15.47:0/1201756243 >> [v2:172.21.15.47:6840/3401578978,v1:172.21.15.47:6841/3401578978] conn(0x56284314c1c0 0x56284314e5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:25:41.292 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:41.286+0000 7f98e6bff740 4 client.15825 mds_command: new command op to 14619 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-21T10:25:41.292 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:41.286+0000 7f98e6bff740 1 -- 172.21.15.47:0/1201756243 --> [v2:172.21.15.47:6840/3401578978,v1:172.21.15.47:6841/3401578978] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x562843041b50 con 0x56284314c1c0 2024-03-21T10:25:41.292 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:41.286+0000 7f98e0a00640 1 --2- 172.21.15.47:0/1201756243 >> [v2:172.21.15.47:6840/3401578978,v1:172.21.15.47:6841/3401578978] conn(0x56284314c1c0 0x56284314e5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:25:41.292 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:41.286+0000 7f98e0a00640 1 --2- 172.21.15.47:0/1201756243 >> [v2:172.21.15.47:6840/3401578978,v1:172.21.15.47:6841/3401578978] conn(0x56284314c1c0 0x56284314e5e0 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=11efa0a14e77673c server_cookie=5482f4ae7111633a in_seq=0 out_seq=0 2024-03-21T10:25:41.292 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:41.286+0000 7f98c2a00640 10 client.15825 ms_handle_connect on v2:172.21.15.47:6840/3401578978 2024-03-21T10:25:41.292 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:41.287+0000 7f98c2a00640 1 -- 172.21.15.47:0/1201756243 <== mds.4 v2:172.21.15.47:6840/3401578978 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x562843041b50 con 0x56284314c1c0 2024-03-21T10:25:41.292 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:41.287+0000 7f98c2a00640 10 client.15825 handle_command_reply: tid=0 2024-03-21T10:25:41.293 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:41.287+0000 7f98e6bff740 1 client.15825 shutdown 2024-03-21T10:25:41.293 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:41.287+0000 7f98e6bff740 1 -- 172.21.15.47:0/1201756243 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5628430dfde0 msgr2=0x56284314a290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:25:41.293 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:41.287+0000 7f98e6bff740 1 --2- 172.21.15.47:0/1201756243 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5628430dfde0 0x56284314a290 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f98d4005c10 tx=0x7f98d401c040 comp rx=0 tx=0).stop 2024-03-21T10:25:41.293 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:41.287+0000 7f98e6bff740 1 -- 172.21.15.47:0/1201756243 shutdown_connections 2024-03-21T10:25:41.293 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:41.287+0000 7f98e6bff740 1 --2- 172.21.15.47:0/1201756243 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5628430dfde0 0x56284314a290 unknown :-1 s=CLOSED pgs=543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:25:41.293 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:41.287+0000 7f98e6bff740 1 --2- 172.21.15.47:0/1201756243 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5628430d7260 0x5628430e6200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:25:41.293 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:41.287+0000 7f98e6bff740 1 --2- 172.21.15.47:0/1201756243 >> [v2:172.21.15.47:6840/3401578978,v1:172.21.15.47:6841/3401578978] conn(0x56284314c1c0 0x56284314e5e0 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:25:41.293 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:41.287+0000 7f98e6bff740 1 --2- 172.21.15.47:0/1201756243 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5628430d68b0 0x5628430e5cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:25:41.293 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:41.287+0000 7f98c2a00640 0 client.15825 ms_handle_reset on v2:172.21.15.47:6840/3401578978 2024-03-21T10:25:41.293 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:41.287+0000 7f98e6bff740 1 -- 172.21.15.47:0/1201756243 >> 172.21.15.47:0/1201756243 conn(0x5628430d0f80 msgr2=0x5628430d0160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:25:41.293 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:41.287+0000 7f98e6bff740 1 -- 172.21.15.47:0/1201756243 shutdown_connections 2024-03-21T10:25:41.293 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:41.287+0000 7f98e6bff740 1 -- 172.21.15.47:0/1201756243 wait complete. 2024-03-21T10:25:41.293 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:41.287+0000 7f98e6bff740 20 client.15825 trim_cache size 0 max 16384 2024-03-21T10:25:41.294 INFO:teuthology.orchestra.run.smithi047.stdout:[] 2024-03-21T10:25:41.294 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:41.288+0000 7f98e2800640 1 -- 172.21.15.47:0/1724084671 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f98c40a31a0 msgr2=0x7f98c40a5660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:25:41.294 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:41.288+0000 7f98e2800640 1 --2- 172.21.15.47:0/1724084671 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f98c40a31a0 0x7f98c40a5660 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f98d00ef040 tx=0x7f98d0009260 comp rx=0 tx=0).stop 2024-03-21T10:25:41.295 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:41.288+0000 7f98e2800640 1 -- 172.21.15.47:0/1724084671 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f98dc1026d0 msgr2=0x7f98dc10a820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:25:41.295 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:41.288+0000 7f98e2800640 1 --2- 172.21.15.47:0/1724084671 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f98dc1026d0 0x7f98dc10a820 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f98d0000c00 tx=0x7f98d002c760 comp rx=0 tx=0).stop 2024-03-21T10:25:41.295 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:41.288+0000 7f98e2800640 1 -- 172.21.15.47:0/1724084671 shutdown_connections 2024-03-21T10:25:41.295 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:41.288+0000 7f98e2800640 1 --2- 172.21.15.47:0/1724084671 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f98c40a31a0 0x7f98c40a5660 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:25:41.295 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:41.288+0000 7f98e2800640 1 --2- 172.21.15.47:0/1724084671 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f98dc10baf0 0x7f98dc1a2b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:25:41.295 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:41.288+0000 7f98e2800640 1 --2- 172.21.15.47:0/1724084671 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f98dc102ff0 0x7f98dc10ad60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:25:41.295 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:41.288+0000 7f98e2800640 1 --2- 172.21.15.47:0/1724084671 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f98dc1026d0 0x7f98dc10a820 unknown :-1 s=CLOSED pgs=540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:25:41.295 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:41.288+0000 7f98e2800640 1 -- 172.21.15.47:0/1724084671 >> 172.21.15.47:0/1724084671 conn(0x7f98dc0fbef0 msgr2=0x7f98dc10cde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:25:41.295 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:41.288+0000 7f98e2800640 1 -- 172.21.15.47:0/1724084671 shutdown_connections 2024-03-21T10:25:41.295 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:41.288+0000 7f98e2800640 1 -- 172.21.15.47:0/1724084671 wait complete. 2024-03-21T10:25:41.296 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:25:41 smithi157 ceph-mon[35581]: pgmap v13: 129 pgs: 129 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 597 B/s wr, 8 op/s 2024-03-21T10:25:41.296 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:25:41 smithi157 ceph-mon[35581]: from='client.? 172.21.15.47:0/2658942557' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-21T10:25:41.304 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:25:41 smithi047 ceph-mon[31547]: pgmap v13: 129 pgs: 129 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 597 B/s wr, 8 op/s 2024-03-21T10:25:41.304 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:25:41 smithi047 ceph-mon[31547]: from='client.? 172.21.15.47:0/2658942557' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-21T10:25:41.306 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-21T10:25:41.306 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-03-21T10:25:41.479 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:25:41 smithi100 ceph-mon[35193]: pgmap v13: 129 pgs: 129 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 597 B/s wr, 8 op/s 2024-03-21T10:25:41.479 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:25:41 smithi100 ceph-mon[35193]: from='client.? 172.21.15.47:0/2658942557' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-21T10:25:41.522 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:41.520+0000 7f27dd600640 1 -- 172.21.15.47:0/2249212581 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f27d8101530 msgr2=0x7f27d81019b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:25:41.522 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:41.520+0000 7f27dd600640 1 --2- 172.21.15.47:0/2249212581 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f27d8101530 0x7f27d81019b0 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f27c4009be0 tx=0x7f27c402bef0 comp rx=0 tx=0).stop 2024-03-21T10:25:41.522 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:41.520+0000 7f27dd600640 1 -- 172.21.15.47:0/2249212581 shutdown_connections 2024-03-21T10:25:41.522 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:41.520+0000 7f27dd600640 1 --2- 172.21.15.47:0/2249212581 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f27d8101ef0 0x7f27d810e3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:25:41.523 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:41.520+0000 7f27dd600640 1 --2- 172.21.15.47:0/2249212581 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f27d8101530 0x7f27d81019b0 unknown :-1 s=CLOSED pgs=544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:25:41.523 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:41.520+0000 7f27dd600640 1 --2- 172.21.15.47:0/2249212581 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f27d81075f0 0x7f27d81079d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:25:41.523 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:41.520+0000 7f27dd600640 1 -- 172.21.15.47:0/2249212581 >> 172.21.15.47:0/2249212581 conn(0x7f27d80fd000 msgr2=0x7f27d80ff420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:25:41.523 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:41.520+0000 7f27dd600640 1 -- 172.21.15.47:0/2249212581 shutdown_connections 2024-03-21T10:25:41.523 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:41.520+0000 7f27dd600640 1 -- 172.21.15.47:0/2249212581 wait complete. 2024-03-21T10:25:41.541 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:41.521+0000 7f27dd600640 1 Processor -- start 2024-03-21T10:25:41.541 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:41.522+0000 7f27dd600640 1 -- start start 2024-03-21T10:25:41.541 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:41.522+0000 7f27dd600640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f27d8101530 0x7f27d819d8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:25:41.541 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:41.522+0000 7f27dd600640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f27d8101ef0 0x7f27d819de20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:25:41.541 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:41.522+0000 7f27dd600640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f27d81075f0 0x7f27d8197a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:25:41.542 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:41.522+0000 7f27dd600640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27d81117d0 con 0x7f27d8101530 2024-03-21T10:25:41.542 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:41.522+0000 7f27dd600640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27d8111650 con 0x7f27d81075f0 2024-03-21T10:25:41.542 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:41.522+0000 7f27dd600640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27d8111950 con 0x7f27d8101ef0 2024-03-21T10:25:41.542 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:41.522+0000 7f27d7e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f27d8101530 0x7f27d819d8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:25:41.542 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:41.523+0000 7f27d7e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f27d8101530 0x7f27d819d8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:56060/0 (socket says 172.21.15.47:56060) 2024-03-21T10:25:41.543 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:41.523+0000 7f27d7e00640 1 -- 172.21.15.47:0/1146247716 learned_addr learned my addr 172.21.15.47:0/1146247716 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:25:41.543 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:41.523+0000 7f27dcc00640 1 --2- 172.21.15.47:0/1146247716 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f27d81075f0 0x7f27d8197a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:25:41.543 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:41.523+0000 7f27d7400640 1 --2- 172.21.15.47:0/1146247716 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f27d8101ef0 0x7f27d819de20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:25:41.543 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:41.523+0000 7f27d7e00640 1 -- 172.21.15.47:0/1146247716 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f27d8101ef0 msgr2=0x7f27d819de20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:25:41.543 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:41.523+0000 7f27d7e00640 1 --2- 172.21.15.47:0/1146247716 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f27d8101ef0 0x7f27d819de20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:25:41.543 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:41.523+0000 7f27d7e00640 1 -- 172.21.15.47:0/1146247716 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f27d81075f0 msgr2=0x7f27d8197a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:25:41.543 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:41.523+0000 7f27d7e00640 1 --2- 172.21.15.47:0/1146247716 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f27d81075f0 0x7f27d8197a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:25:41.544 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:41.523+0000 7f27d7e00640 1 -- 172.21.15.47:0/1146247716 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f27d8198210 con 0x7f27d8101530 2024-03-21T10:25:41.544 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:41.523+0000 7f27dcc00640 1 --2- 172.21.15.47:0/1146247716 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f27d81075f0 0x7f27d8197a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T10:25:41.544 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:41.523+0000 7f27d7e00640 1 --2- 172.21.15.47:0/1146247716 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f27d8101530 0x7f27d819d8e0 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f27c800e9c0 tx=0x7f27c800ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:25:41.544 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:41.524+0000 7f27d4c00640 1 -- 172.21.15.47:0/1146247716 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27c800cde0 con 0x7f27d8101530 2024-03-21T10:25:41.544 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:41.524+0000 7f27d4c00640 1 -- 172.21.15.47:0/1146247716 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f27c80035f0 con 0x7f27d8101530 2024-03-21T10:25:41.544 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:41.524+0000 7f27dd600640 1 -- 172.21.15.47:0/1146247716 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f27d8198500 con 0x7f27d8101530 2024-03-21T10:25:41.544 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:41.524+0000 7f27d4c00640 1 -- 172.21.15.47:0/1146247716 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27c8010690 con 0x7f27d8101530 2024-03-21T10:25:41.545 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:41.524+0000 7f27dd600640 1 -- 172.21.15.47:0/1146247716 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f27d81a47a0 con 0x7f27d8101530 2024-03-21T10:25:41.545 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:41.527+0000 7f27d4c00640 1 -- 172.21.15.47:0/1146247716 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 25) v1 ==== 138030+0+0 (secure 0 0 0) 0x7f27c80037a0 con 0x7f27d8101530 2024-03-21T10:25:41.545 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:41.527+0000 7f27e1979740 1 Processor -- start 2024-03-21T10:25:41.545 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:41.527+0000 7f27e1979740 1 -- start start 2024-03-21T10:25:41.545 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:41.528+0000 7f27e1979740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563a38ddfcc0 0x563a38de00a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:25:41.545 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:41.528+0000 7f27e1979740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x563a38de0670 0x563a38de8c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:25:41.546 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:41.528+0000 7f27e1979740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563a38de91f0 0x563a38deb640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:25:41.546 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:41.528+0000 7f27e1979740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x563a38d3f2c0 con 0x563a38de0670 2024-03-21T10:25:41.546 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:41.528+0000 7f27e1979740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x563a38ce7c90 con 0x563a38ddfcc0 2024-03-21T10:25:41.546 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:41.528+0000 7f27e1979740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x563a38d3b360 con 0x563a38de91f0 2024-03-21T10:25:41.546 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:41.528+0000 7f27d7e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x563a38de0670 0x563a38de8c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:25:41.546 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:41.528+0000 7f27dcc00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563a38ddfcc0 0x563a38de00a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:25:41.547 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:41.528+0000 7f27dcc00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563a38ddfcc0 0x563a38de00a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:37408/0 (socket says 172.21.15.47:37408) 2024-03-21T10:25:41.547 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:41.528+0000 7f27dcc00640 1 -- 172.21.15.47:0/1133971319 learned_addr learned my addr 172.21.15.47:0/1133971319 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:25:41.547 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:41.528+0000 7f27d7400640 1 --2- 172.21.15.47:0/1133971319 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563a38de91f0 0x563a38deb640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:25:41.547 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:41.529+0000 7f27dcc00640 1 -- 172.21.15.47:0/1133971319 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563a38de91f0 msgr2=0x563a38deb640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:25:41.547 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:41.529+0000 7f27dcc00640 1 --2- 172.21.15.47:0/1133971319 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563a38de91f0 0x563a38deb640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:25:41.548 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:41.529+0000 7f27dcc00640 1 -- 172.21.15.47:0/1133971319 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x563a38de0670 msgr2=0x563a38de8c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:25:41.548 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:41.529+0000 7f27dcc00640 1 --2- 172.21.15.47:0/1133971319 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x563a38de0670 0x563a38de8c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:25:41.548 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:41.529+0000 7f27dcc00640 1 -- 172.21.15.47:0/1133971319 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563a38d0c080 con 0x563a38ddfcc0 2024-03-21T10:25:41.548 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:41.529+0000 7f27dcc00640 1 --2- 172.21.15.47:0/1133971319 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563a38ddfcc0 0x563a38de00a0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f27d000bcf0 tx=0x7f27d000f730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3c3cefe498604849 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:25:41.548 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:41.529+0000 7f27d4c00640 1 --2- 172.21.15.47:0/1146247716 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f27bc0a31a0 0x7f27bc0a5660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:25:41.548 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:41.530+0000 7f27cd200640 1 -- 172.21.15.47:0/1133971319 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27d0015070 con 0x563a38ddfcc0 2024-03-21T10:25:41.549 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:41.530+0000 7f27cd200640 1 -- 172.21.15.47:0/1133971319 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f27d00118f0 con 0x563a38ddfcc0 2024-03-21T10:25:41.549 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:41.530+0000 7f27d4c00640 1 -- 172.21.15.47:0/1146247716 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 9474+0+0 (secure 0 0 0) 0x7f27c80d0110 con 0x7f27d8101530 2024-03-21T10:25:41.549 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:41.530+0000 7f27cd200640 1 -- 172.21.15.47:0/1133971319 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27d0010840 con 0x563a38ddfcc0 2024-03-21T10:25:41.549 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:41.530+0000 7f27d7e00640 1 --2- 172.21.15.47:0/1146247716 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f27bc0a31a0 0x7f27bc0a5660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:25:41.549 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:41.530+0000 7f27e1979740 1 -- 172.21.15.47:0/1133971319 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563a38ddfcc0 msgr2=0x563a38de00a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:25:41.549 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:41.530+0000 7f27e1979740 1 --2- 172.21.15.47:0/1133971319 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563a38ddfcc0 0x563a38de00a0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f27d000bcf0 tx=0x7f27d000f730 comp rx=0 tx=0).stop 2024-03-21T10:25:41.550 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:41.531+0000 7f27d7e00640 1 --2- 172.21.15.47:0/1146247716 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f27bc0a31a0 0x7f27bc0a5660 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f27c80cd440 tx=0x7f27c80d3b90 comp rx=0 tx=0).ready entity=mgr.15687 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:25:41.550 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:41.531+0000 7f27e1979740 1 -- 172.21.15.47:0/1133971319 shutdown_connections 2024-03-21T10:25:41.550 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:41.531+0000 7f27e1979740 1 --2- 172.21.15.47:0/1133971319 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563a38de91f0 0x563a38deb640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:25:41.550 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:41.531+0000 7f27e1979740 1 --2- 172.21.15.47:0/1133971319 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x563a38de0670 0x563a38de8c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:25:41.550 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:41.531+0000 7f27e1979740 1 --2- 172.21.15.47:0/1133971319 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563a38ddfcc0 0x563a38de00a0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:25:41.550 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:41.531+0000 7f27e1979740 1 -- 172.21.15.47:0/1133971319 >> 172.21.15.47:0/1133971319 conn(0x563a38dda390 msgr2=0x563a38dda770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:25:41.551 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:41.531+0000 7f27e1979740 1 -- 172.21.15.47:0/1133971319 shutdown_connections 2024-03-21T10:25:41.551 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:41.531+0000 7f27e1979740 1 -- 172.21.15.47:0/1133971319 wait complete. 2024-03-21T10:25:41.551 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:41.531+0000 7f27e1979740 1 -- start start 2024-03-21T10:25:41.551 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:41.531+0000 7f27e1979740 1 Processor -- start 2024-03-21T10:25:41.551 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:41.532+0000 7f27e1979740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563a38ddfcc0 0x563a38de8a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:25:41.552 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:41.532+0000 7f27e1979740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563a38de0670 0x563a38de5e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:25:41.552 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:41.532+0000 7f27e1979740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x563a38de91f0 0x563a38de6340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:25:41.552 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:41.532+0000 7f27e1979740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x563a38dec160 con 0x563a38de91f0 2024-03-21T10:25:41.552 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:41.532+0000 7f27e1979740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x563a38cfb0d0 con 0x563a38ddfcc0 2024-03-21T10:25:41.552 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:41.532+0000 7f27e1979740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x563a38dec2e0 con 0x563a38de0670 2024-03-21T10:25:41.552 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:41.532+0000 7f27e1979740 10 client.0 _ll_register_callbacks cb 0x563a38cffe80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T10:25:41.553 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:41.532+0000 7f27dcc00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x563a38de91f0 0x563a38de6340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:25:41.553 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:41.532+0000 7f27dcc00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x563a38de91f0 0x563a38de6340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:56082/0 (socket says 172.21.15.47:56082) 2024-03-21T10:25:41.553 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:41.532+0000 7f27dcc00640 1 -- 172.21.15.47:0/2833153426 learned_addr learned my addr 172.21.15.47:0/2833153426 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:25:41.553 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:41.532+0000 7f27dcc00640 1 --2- 172.21.15.47:0/2833153426 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563a38ddfcc0 0x563a38de8a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:25:41.553 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:41.533+0000 7f27d7400640 1 --2- 172.21.15.47:0/2833153426 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563a38de0670 0x563a38de5e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:25:41.553 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:41.533+0000 7f27dcc00640 1 -- 172.21.15.47:0/2833153426 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563a38de0670 msgr2=0x563a38de5e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:25:41.554 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:41.533+0000 7f27dcc00640 1 --2- 172.21.15.47:0/2833153426 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563a38de0670 0x563a38de5e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:25:41.554 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:41.533+0000 7f27dcc00640 1 -- 172.21.15.47:0/2833153426 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563a38ddfcc0 msgr2=0x563a38de8a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:25:41.554 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:41.533+0000 7f27dcc00640 1 --2- 172.21.15.47:0/2833153426 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563a38ddfcc0 0x563a38de8a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:25:41.555 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:41.533+0000 7f27dcc00640 1 -- 172.21.15.47:0/2833153426 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x563a38a533f0 con 0x563a38de91f0 2024-03-21T10:25:41.555 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:41.533+0000 7f27dcc00640 1 --2- 172.21.15.47:0/2833153426 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x563a38de91f0 0x563a38de6340 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f27d0005d80 tx=0x7f27d0005d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cecda16d010ed520 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:25:41.555 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:41.533+0000 7f27c1600640 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-03-21T10:25:41.555 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:41.534+0000 7f27c1600640 1 -- 172.21.15.47:0/2833153426 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27d0015040 con 0x563a38de91f0 2024-03-21T10:25:41.555 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:41.534+0000 7f27e1979740 1 -- 172.21.15.47:0/2833153426 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f27d00118f0 con 0x563a38de91f0 2024-03-21T10:25:41.555 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:41.534+0000 7f27c1600640 1 -- 172.21.15.47:0/2833153426 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 9474+0+0 (secure 0 0 0) 0x7f27d0011440 con 0x563a38de91f0 2024-03-21T10:25:41.556 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:41.534+0000 7f27c1600640 1 -- 172.21.15.47:0/2833153426 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f27d0011640 con 0x563a38de91f0 2024-03-21T10:25:41.556 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:41.534+0000 7f27e1979740 10 client.15837 fetch_fsmap learned FSMap version 30 2024-03-21T10:25:41.556 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:41.534+0000 7f27e1979740 1 -- 172.21.15.47:0/2833153426 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x563a38b1d7d0 con 0x563a38de91f0 2024-03-21T10:25:41.556 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:41.534+0000 7f27c1600640 1 -- 172.21.15.47:0/2833153426 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f27d00118f0 con 0x563a38de91f0 2024-03-21T10:25:41.556 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:41.535+0000 7f27c1600640 1 -- 172.21.15.47:0/2833153426 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 30) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f27d001a460 con 0x563a38de91f0 2024-03-21T10:25:41.556 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:41.535+0000 7f27e1979740 10 client.15837 fetch_fsmap finished waiting for FSMap version 30 2024-03-21T10:25:41.556 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:41.535+0000 7f27e1979740 10 client.15837 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.k 2024-03-21T10:25:41.556 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:41.535+0000 7f27e1979740 20 client.15837 populate_metadata read hostname 'smithi047' 2024-03-21T10:25:41.556 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:41.535+0000 7f27e1979740 1 --2- 172.21.15.47:0/2833153426 >> [v2:172.21.15.100:6832/2723302038,v1:172.21.15.100:6833/2723302038] conn(0x563a38df8ed0 0x563a38dfb2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:25:41.557 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:41.535+0000 7f27e1979740 4 client.15837 mds_command: new command op to 34427 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T10:25:41.557 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:41.535+0000 7f27e1979740 1 -- 172.21.15.47:0/2833153426 --> [v2:172.21.15.100:6832/2723302038,v1:172.21.15.100:6833/2723302038] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x563a38d4c800 con 0x563a38df8ed0 2024-03-21T10:25:41.557 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:41.535+0000 7f27d7400640 1 --2- 172.21.15.47:0/2833153426 >> [v2:172.21.15.100:6832/2723302038,v1:172.21.15.100:6833/2723302038] conn(0x563a38df8ed0 0x563a38dfb2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:25:41.557 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:41.536+0000 7f27d7400640 1 --2- 172.21.15.47:0/2833153426 >> [v2:172.21.15.100:6832/2723302038,v1:172.21.15.100:6833/2723302038] conn(0x563a38df8ed0 0x563a38dfb2c0 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=d22ca62619f0e10f server_cookie=9ba676312c9bea03 in_seq=0 out_seq=0 2024-03-21T10:25:41.557 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:41.536+0000 7f27c1600640 10 client.15837 ms_handle_connect on v2:172.21.15.100:6832/2723302038 2024-03-21T10:25:41.557 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:41.538+0000 7f27c1600640 1 -- 172.21.15.47:0/2833153426 <== mds.3 v2:172.21.15.100:6832/2723302038 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x563a38d4c800 con 0x563a38df8ed0 2024-03-21T10:25:41.557 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:41.538+0000 7f27c1600640 10 client.15837 handle_command_reply: tid=0 2024-03-21T10:25:41.557 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:41.538+0000 7f27e1979740 1 client.15837 shutdown 2024-03-21T10:25:41.557 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:41.538+0000 7f27e1979740 1 -- 172.21.15.47:0/2833153426 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x563a38de91f0 msgr2=0x563a38de6340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:25:41.558 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:41.538+0000 7f27e1979740 1 --2- 172.21.15.47:0/2833153426 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x563a38de91f0 0x563a38de6340 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f27d0005d80 tx=0x7f27d0005d10 comp rx=0 tx=0).stop 2024-03-21T10:25:41.558 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:41.538+0000 7f27e1979740 1 -- 172.21.15.47:0/2833153426 shutdown_connections 2024-03-21T10:25:41.558 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:41.538+0000 7f27e1979740 1 --2- 172.21.15.47:0/2833153426 >> [v2:172.21.15.100:6832/2723302038,v1:172.21.15.100:6833/2723302038] conn(0x563a38df8ed0 0x563a38dfb2c0 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:25:41.558 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:41.538+0000 7f27e1979740 1 --2- 172.21.15.47:0/2833153426 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x563a38de91f0 0x563a38de6340 unknown :-1 s=CLOSED pgs=546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:25:41.558 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:41.538+0000 7f27e1979740 1 --2- 172.21.15.47:0/2833153426 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563a38de0670 0x563a38de5e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:25:41.558 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:41.538+0000 7f27e1979740 1 --2- 172.21.15.47:0/2833153426 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563a38ddfcc0 0x563a38de8a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:25:41.558 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:41.538+0000 7f27c1600640 0 client.15837 ms_handle_reset on v2:172.21.15.100:6832/2723302038 2024-03-21T10:25:41.559 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:41.538+0000 7f27e1979740 1 -- 172.21.15.47:0/2833153426 >> 172.21.15.47:0/2833153426 conn(0x563a38dda390 msgr2=0x563a38dd5160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:25:41.559 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:41.539+0000 7f27e1979740 1 -- 172.21.15.47:0/2833153426 shutdown_connections 2024-03-21T10:25:41.559 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:41.539+0000 7f27e1979740 1 -- 172.21.15.47:0/2833153426 wait complete. 2024-03-21T10:25:41.559 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:41.539+0000 7f27e1979740 20 client.15837 trim_cache size 0 max 16384 2024-03-21T10:25:41.564 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:41.556+0000 7f27e1979740 1 Processor -- start 2024-03-21T10:25:41.564 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:41.557+0000 7f27e1979740 1 -- start start 2024-03-21T10:25:41.564 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:41.557+0000 7f27e1979740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563a38ddfcc0 0x563a38de8530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:25:41.564 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:41.557+0000 7f27e1979740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563a38de0670 0x563a38de8a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:25:41.564 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:41.557+0000 7f27e1979740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x563a38de91f0 0x563a38de3060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:25:41.564 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:41.557+0000 7f27e1979740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27d81ac4f0 con 0x563a38de91f0 2024-03-21T10:25:41.564 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:41.557+0000 7f27e1979740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27d81ade70 con 0x563a38de0670 2024-03-21T10:25:41.564 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:41.557+0000 7f27e1979740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27d81acb50 con 0x563a38ddfcc0 2024-03-21T10:25:41.565 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:41.558+0000 7f27d7400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563a38de0670 0x563a38de8a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:25:41.565 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:41.558+0000 7f27dcc00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x563a38de91f0 0x563a38de3060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:25:41.565 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:41.558+0000 7f27d7400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563a38de0670 0x563a38de8a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:37432/0 (socket says 172.21.15.47:37432) 2024-03-21T10:25:41.565 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:41.558+0000 7f27d7400640 1 -- 172.21.15.47:0/29799054 learned_addr learned my addr 172.21.15.47:0/29799054 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:25:41.565 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:41.558+0000 7f27dcc00640 1 --2- 172.21.15.47:0/29799054 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563a38ddfcc0 0x563a38de8530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:25:41.565 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:41.558+0000 7f27dcc00640 1 -- 172.21.15.47:0/29799054 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563a38ddfcc0 msgr2=0x563a38de8530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:25:41.565 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:41.558+0000 7f27dcc00640 1 --2- 172.21.15.47:0/29799054 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563a38ddfcc0 0x563a38de8530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:25:41.565 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:41.558+0000 7f27dcc00640 1 -- 172.21.15.47:0/29799054 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563a38de0670 msgr2=0x563a38de8a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:25:41.565 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:41.558+0000 7f27dcc00640 1 --2- 172.21.15.47:0/29799054 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563a38de0670 0x563a38de8a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:25:41.565 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:41.558+0000 7f27dcc00640 1 -- 172.21.15.47:0/29799054 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563a38de0410 con 0x563a38de91f0 2024-03-21T10:25:41.565 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:41.559+0000 7f27dcc00640 1 --2- 172.21.15.47:0/29799054 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x563a38de91f0 0x563a38de3060 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f27d0001eb0 tx=0x7f27d000f730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ee1165963806e5b1 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:25:41.565 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:41.559+0000 7f27c3e00640 1 -- 172.21.15.47:0/29799054 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27d0015070 con 0x563a38de91f0 2024-03-21T10:25:41.565 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:41.559+0000 7f27c3e00640 1 -- 172.21.15.47:0/29799054 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f27d0011440 con 0x563a38de91f0 2024-03-21T10:25:41.565 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:41.559+0000 7f27c3e00640 1 -- 172.21.15.47:0/29799054 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27d00103d0 con 0x563a38de91f0 2024-03-21T10:25:41.565 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:41.560+0000 7f27e1979740 1 -- 172.21.15.47:0/29799054 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x563a38de91f0 msgr2=0x563a38de3060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:25:41.566 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:41.560+0000 7f27e1979740 1 --2- 172.21.15.47:0/29799054 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x563a38de91f0 0x563a38de3060 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f27d0001eb0 tx=0x7f27d000f730 comp rx=0 tx=0).stop 2024-03-21T10:25:41.566 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:41.560+0000 7f27e1979740 1 -- 172.21.15.47:0/29799054 shutdown_connections 2024-03-21T10:25:41.566 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:41.560+0000 7f27e1979740 1 --2- 172.21.15.47:0/29799054 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x563a38de91f0 0x563a38de3060 unknown :-1 s=CLOSED pgs=547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:25:41.566 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:41.560+0000 7f27e1979740 1 --2- 172.21.15.47:0/29799054 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563a38de0670 0x563a38de8a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:25:41.566 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:41.560+0000 7f27e1979740 1 --2- 172.21.15.47:0/29799054 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563a38ddfcc0 0x563a38de8530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:25:41.566 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:41.560+0000 7f27e1979740 1 -- 172.21.15.47:0/29799054 >> 172.21.15.47:0/29799054 conn(0x563a38dda390 msgr2=0x563a38de12e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:25:41.566 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:41.560+0000 7f27e1979740 1 -- 172.21.15.47:0/29799054 shutdown_connections 2024-03-21T10:25:41.566 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:41.560+0000 7f27e1979740 1 -- 172.21.15.47:0/29799054 wait complete. 2024-03-21T10:25:41.566 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:41.560+0000 7f27e1979740 1 -- start start 2024-03-21T10:25:41.566 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:41.560+0000 7f27e1979740 1 Processor -- start 2024-03-21T10:25:41.566 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:41.561+0000 7f27e1979740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x563a38ddfcc0 0x563a38decfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:25:41.566 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:41.561+0000 7f27e1979740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563a38de0670 0x563a38ded4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:25:41.566 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:41.561+0000 7f27e1979740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563a38de91f0 0x563a38e534e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:25:41.566 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:41.561+0000 7f27e1979740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27d81ad810 con 0x563a38ddfcc0 2024-03-21T10:25:41.567 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:41.561+0000 7f27e1979740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27d81ad1b0 con 0x563a38de91f0 2024-03-21T10:25:41.567 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:41.561+0000 7f27e1979740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27d81aa6b0 con 0x563a38de0670 2024-03-21T10:25:41.567 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:41.561+0000 7f27e1979740 10 client.0 _ll_register_callbacks cb 0x7f27c80cd270 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T10:25:41.567 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:41.561+0000 7f27dcc00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x563a38ddfcc0 0x563a38decfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:25:41.567 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:41.561+0000 7f27dcc00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x563a38ddfcc0 0x563a38decfa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:56100/0 (socket says 172.21.15.47:56100) 2024-03-21T10:25:41.567 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:41.561+0000 7f27dcc00640 1 -- 172.21.15.47:0/521245775 learned_addr learned my addr 172.21.15.47:0/521245775 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:25:41.567 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:41.562+0000 7f27d7400640 1 --2- 172.21.15.47:0/521245775 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563a38de0670 0x563a38ded4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:25:41.567 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:41.562+0000 7f27dcc00640 1 -- 172.21.15.47:0/521245775 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563a38de0670 msgr2=0x563a38ded4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:25:41.567 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:41.562+0000 7f27dcc00640 1 --2- 172.21.15.47:0/521245775 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563a38de0670 0x563a38ded4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:25:41.567 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:41.562+0000 7f27dcc00640 1 -- 172.21.15.47:0/521245775 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563a38de91f0 msgr2=0x563a38e534e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:25:41.567 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:41.562+0000 7f27dcc00640 1 --2- 172.21.15.47:0/521245775 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563a38de91f0 0x563a38e534e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:25:41.567 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:41.562+0000 7f27dcc00640 1 -- 172.21.15.47:0/521245775 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x563a38e53a20 con 0x563a38ddfcc0 2024-03-21T10:25:41.567 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:41.562+0000 7f27d7400640 1 --2- 172.21.15.47:0/521245775 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563a38de0670 0x563a38ded4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T10:25:41.567 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:41.562+0000 7f27dcc00640 1 --2- 172.21.15.47:0/521245775 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x563a38ddfcc0 0x563a38decfa0 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f27d0006010 tx=0x7f27d0010a10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=65e35a6dcaffdce8 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:25:41.567 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:41.562+0000 7f27c1600640 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-03-21T10:25:41.568 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:41.562+0000 7f27c1600640 1 -- 172.21.15.47:0/521245775 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27d0015070 con 0x563a38ddfcc0 2024-03-21T10:25:41.568 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:41.562+0000 7f27c1600640 1 -- 172.21.15.47:0/521245775 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 9474+0+0 (secure 0 0 0) 0x7f27d001a460 con 0x563a38ddfcc0 2024-03-21T10:25:41.568 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:41.562+0000 7f27e1979740 1 -- 172.21.15.47:0/521245775 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f27d0011440 con 0x563a38ddfcc0 2024-03-21T10:25:41.568 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:41.563+0000 7f27c1600640 1 -- 172.21.15.47:0/521245775 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f27d00105d0 con 0x563a38ddfcc0 2024-03-21T10:25:41.568 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:41.563+0000 7f27e1979740 10 client.15843 fetch_fsmap learned FSMap version 30 2024-03-21T10:25:41.568 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:41.563+0000 7f27e1979740 1 -- 172.21.15.47:0/521245775 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x563a38e54c80 con 0x563a38ddfcc0 2024-03-21T10:25:41.570 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:41.563+0000 7f27c1600640 1 -- 172.21.15.47:0/521245775 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f27d0011440 con 0x563a38ddfcc0 2024-03-21T10:25:41.570 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:41.563+0000 7f27c1600640 1 -- 172.21.15.47:0/521245775 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 30) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f27d0011b60 con 0x563a38ddfcc0 2024-03-21T10:25:41.570 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:41.563+0000 7f27e1979740 10 client.15843 fetch_fsmap finished waiting for FSMap version 30 2024-03-21T10:25:41.570 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:41.563+0000 7f27e1979740 10 client.15843 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.k 2024-03-21T10:25:41.570 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:41.563+0000 7f27e1979740 20 client.15843 populate_metadata read hostname 'smithi047' 2024-03-21T10:25:41.571 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:41.563+0000 7f27e1979740 1 --2- 172.21.15.47:0/521245775 >> [v2:172.21.15.100:6832/2723302038,v1:172.21.15.100:6833/2723302038] conn(0x563a38e550f0 0x563a38e574e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:25:41.571 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:41.563+0000 7f27e1979740 4 client.15843 mds_command: new command op to 34427 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-21T10:25:41.571 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:41.563+0000 7f27e1979740 1 -- 172.21.15.47:0/521245775 --> [v2:172.21.15.100:6832/2723302038,v1:172.21.15.100:6833/2723302038] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x563a38d46b50 con 0x563a38e550f0 2024-03-21T10:25:41.571 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:41.564+0000 7f27d7400640 1 --2- 172.21.15.47:0/521245775 >> [v2:172.21.15.100:6832/2723302038,v1:172.21.15.100:6833/2723302038] conn(0x563a38e550f0 0x563a38e574e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:25:41.571 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:41.565+0000 7f27d7400640 1 --2- 172.21.15.47:0/521245775 >> [v2:172.21.15.100:6832/2723302038,v1:172.21.15.100:6833/2723302038] conn(0x563a38e550f0 0x563a38e574e0 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=7f6975424b7eb7a9 server_cookie=7e60b2730346381e in_seq=0 out_seq=0 2024-03-21T10:25:41.571 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:41.565+0000 7f27c1600640 10 client.15843 ms_handle_connect on v2:172.21.15.100:6832/2723302038 2024-03-21T10:25:41.571 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:41.565+0000 7f27c1600640 1 -- 172.21.15.47:0/521245775 <== mds.3 v2:172.21.15.100:6832/2723302038 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x563a38d46b50 con 0x563a38e550f0 2024-03-21T10:25:41.571 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:41.565+0000 7f27c1600640 10 client.15843 handle_command_reply: tid=0 2024-03-21T10:25:41.572 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:41.565+0000 7f27e1979740 1 client.15843 shutdown 2024-03-21T10:25:41.572 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:41.565+0000 7f27e1979740 1 -- 172.21.15.47:0/521245775 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x563a38ddfcc0 msgr2=0x563a38decfa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:25:41.572 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:41.565+0000 7f27e1979740 1 --2- 172.21.15.47:0/521245775 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x563a38ddfcc0 0x563a38decfa0 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f27d0006010 tx=0x7f27d0010a10 comp rx=0 tx=0).stop 2024-03-21T10:25:41.572 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:41.566+0000 7f27e1979740 1 -- 172.21.15.47:0/521245775 shutdown_connections 2024-03-21T10:25:41.572 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:41.566+0000 7f27e1979740 1 --2- 172.21.15.47:0/521245775 >> [v2:172.21.15.100:6832/2723302038,v1:172.21.15.100:6833/2723302038] conn(0x563a38e550f0 0x563a38e574e0 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:25:41.572 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:41.566+0000 7f27e1979740 1 --2- 172.21.15.47:0/521245775 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563a38de91f0 0x563a38e534e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:25:41.572 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:41.566+0000 7f27e1979740 1 --2- 172.21.15.47:0/521245775 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563a38de0670 0x563a38ded4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:25:41.572 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:41.566+0000 7f27e1979740 1 --2- 172.21.15.47:0/521245775 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x563a38ddfcc0 0x563a38decfa0 unknown :-1 s=CLOSED pgs=548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:25:41.573 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:41.566+0000 7f27e1979740 1 -- 172.21.15.47:0/521245775 >> 172.21.15.47:0/521245775 conn(0x563a38dda390 msgr2=0x563a38ddb690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:25:41.573 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:41.566+0000 7f27c1600640 0 client.15843 ms_handle_reset on v2:172.21.15.100:6832/2723302038 2024-03-21T10:25:41.573 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:41.566+0000 7f27e1979740 1 -- 172.21.15.47:0/521245775 shutdown_connections 2024-03-21T10:25:41.573 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:41.566+0000 7f27e1979740 1 -- 172.21.15.47:0/521245775 wait complete. 2024-03-21T10:25:41.573 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:41.566+0000 7f27e1979740 20 client.15843 trim_cache size 0 max 16384 2024-03-21T10:25:41.574 INFO:teuthology.orchestra.run.smithi047.stdout:[] 2024-03-21T10:25:41.575 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:41.567+0000 7f27dd600640 1 -- 172.21.15.47:0/1146247716 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f27bc0a31a0 msgr2=0x7f27bc0a5660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:25:41.575 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:41.567+0000 7f27dd600640 1 --2- 172.21.15.47:0/1146247716 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f27bc0a31a0 0x7f27bc0a5660 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f27c80cd440 tx=0x7f27c80d3b90 comp rx=0 tx=0).stop 2024-03-21T10:25:41.575 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:41.567+0000 7f27dd600640 1 -- 172.21.15.47:0/1146247716 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f27d8101530 msgr2=0x7f27d819d8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:25:41.575 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:41.567+0000 7f27dd600640 1 --2- 172.21.15.47:0/1146247716 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f27d8101530 0x7f27d819d8e0 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f27c800e9c0 tx=0x7f27c800ee80 comp rx=0 tx=0).stop 2024-03-21T10:25:41.576 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:41.567+0000 7f27dd600640 1 -- 172.21.15.47:0/1146247716 shutdown_connections 2024-03-21T10:25:41.576 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:41.567+0000 7f27dd600640 1 --2- 172.21.15.47:0/1146247716 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f27bc0a31a0 0x7f27bc0a5660 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:25:41.576 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:41.567+0000 7f27dd600640 1 --2- 172.21.15.47:0/1146247716 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f27d81075f0 0x7f27d8197a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:25:41.576 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:41.567+0000 7f27dd600640 1 --2- 172.21.15.47:0/1146247716 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f27d8101ef0 0x7f27d819de20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:25:41.576 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:41.567+0000 7f27dd600640 1 --2- 172.21.15.47:0/1146247716 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f27d8101530 0x7f27d819d8e0 unknown :-1 s=CLOSED pgs=545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:25:41.576 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:41.567+0000 7f27dd600640 1 -- 172.21.15.47:0/1146247716 >> 172.21.15.47:0/1146247716 conn(0x7f27d80fd000 msgr2=0x7f27d80fe890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:25:41.576 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:41.567+0000 7f27dd600640 1 -- 172.21.15.47:0/1146247716 shutdown_connections 2024-03-21T10:25:41.577 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:41.567+0000 7f27dd600640 1 -- 172.21.15.47:0/1146247716 wait complete. 2024-03-21T10:25:41.586 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-21T10:25:41.586 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-03-21T10:25:41.806 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:41.804+0000 7fb969000640 1 -- 172.21.15.47:0/2565680199 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb964109730 msgr2=0x7fb964109b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:25:41.806 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:41.804+0000 7fb969000640 1 --2- 172.21.15.47:0/2565680199 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb964109730 0x7fb964109b10 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fb954009900 tx=0x7fb95402bc40 comp rx=0 tx=0).stop 2024-03-21T10:25:41.806 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:41.804+0000 7fb969000640 1 -- 172.21.15.47:0/2565680199 shutdown_connections 2024-03-21T10:25:41.806 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:41.804+0000 7fb969000640 1 --2- 172.21.15.47:0/2565680199 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb9641040e0 0x7fb9641105f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:25:41.806 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:41.804+0000 7fb969000640 1 --2- 172.21.15.47:0/2565680199 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb964103720 0x7fb964103ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:25:41.807 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:41.804+0000 7fb969000640 1 --2- 172.21.15.47:0/2565680199 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb964109730 0x7fb964109b10 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:25:41.807 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:41.804+0000 7fb969000640 1 -- 172.21.15.47:0/2565680199 >> 172.21.15.47:0/2565680199 conn(0x7fb9640ff1b0 msgr2=0x7fb9641015d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:25:41.807 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:41.804+0000 7fb969000640 1 -- 172.21.15.47:0/2565680199 shutdown_connections 2024-03-21T10:25:41.807 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:41.804+0000 7fb969000640 1 -- 172.21.15.47:0/2565680199 wait complete. 2024-03-21T10:25:41.809 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:41.805+0000 7fb969000640 1 Processor -- start 2024-03-21T10:25:41.809 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:41.805+0000 7fb969000640 1 -- start start 2024-03-21T10:25:41.809 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:41.805+0000 7fb969000640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb964103720 0x7fb964199f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:25:41.809 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:41.805+0000 7fb969000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb9641040e0 0x7fb96419a4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:25:41.809 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:41.806+0000 7fb969000640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb964109730 0x7fb96419e820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:25:41.809 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:41.806+0000 7fb969000640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb964113a00 con 0x7fb964103720 2024-03-21T10:25:41.809 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:41.806+0000 7fb969000640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb964113880 con 0x7fb9641040e0 2024-03-21T10:25:41.810 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:41.806+0000 7fb969000640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb964113b80 con 0x7fb964109730 2024-03-21T10:25:41.810 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:41.806+0000 7fb963400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb964103720 0x7fb964199f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:25:41.810 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:41.806+0000 7fb963400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb964103720 0x7fb964199f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:56120/0 (socket says 172.21.15.47:56120) 2024-03-21T10:25:41.810 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:41.806+0000 7fb963400640 1 -- 172.21.15.47:0/1796021017 learned_addr learned my addr 172.21.15.47:0/1796021017 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:25:41.810 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:41.806+0000 7fb962a00640 1 --2- 172.21.15.47:0/1796021017 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb9641040e0 0x7fb96419a4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:25:41.810 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:41.806+0000 7fb963e00640 1 --2- 172.21.15.47:0/1796021017 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb964109730 0x7fb96419e820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:25:41.810 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:41.806+0000 7fb963400640 1 -- 172.21.15.47:0/1796021017 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb964109730 msgr2=0x7fb96419e820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:25:41.810 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:41.806+0000 7fb963400640 1 --2- 172.21.15.47:0/1796021017 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb964109730 0x7fb96419e820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:25:41.810 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:41.806+0000 7fb963400640 1 -- 172.21.15.47:0/1796021017 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb9641040e0 msgr2=0x7fb96419a4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:25:41.810 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:41.806+0000 7fb963400640 1 --2- 172.21.15.47:0/1796021017 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb9641040e0 0x7fb96419a4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:25:41.810 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:41.806+0000 7fb963400640 1 -- 172.21.15.47:0/1796021017 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb96419ef50 con 0x7fb964103720 2024-03-21T10:25:41.810 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:41.807+0000 7fb963400640 1 --2- 172.21.15.47:0/1796021017 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb964103720 0x7fb964199f60 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7fb954000c00 tx=0x7fb9540045e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:25:41.810 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:41.807+0000 7fb95be00640 1 -- 172.21.15.47:0/1796021017 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb954043020 con 0x7fb964103720 2024-03-21T10:25:41.811 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:41.807+0000 7fb969000640 1 -- 172.21.15.47:0/1796021017 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb96419f1e0 con 0x7fb964103720 2024-03-21T10:25:41.811 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:41.807+0000 7fb95be00640 1 -- 172.21.15.47:0/1796021017 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fb95402cbe0 con 0x7fb964103720 2024-03-21T10:25:41.811 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:41.807+0000 7fb95be00640 1 -- 172.21.15.47:0/1796021017 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb954003b80 con 0x7fb964103720 2024-03-21T10:25:41.811 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:41.807+0000 7fb969000640 1 -- 172.21.15.47:0/1796021017 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb9641a6ad0 con 0x7fb964103720 2024-03-21T10:25:41.812 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:41.811+0000 7fb95be00640 1 -- 172.21.15.47:0/1796021017 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 25) v1 ==== 138030+0+0 (secure 0 0 0) 0x7fb954046020 con 0x7fb964103720 2024-03-21T10:25:41.812 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:41.811+0000 7fb96d40b740 1 Processor -- start 2024-03-21T10:25:41.812 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:41.811+0000 7fb96d40b740 1 -- start start 2024-03-21T10:25:41.814 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:41.811+0000 7fb96d40b740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d8149978b0 0x55d814997c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:25:41.814 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:41.811+0000 7fb96d40b740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d814998260 0x55d8149a0810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:25:41.814 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:41.811+0000 7fb96d40b740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d8149a0de0 0x55d8149a3230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:25:41.814 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:41.811+0000 7fb96d40b740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d8148fb2c0 con 0x55d814998260 2024-03-21T10:25:41.814 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:41.811+0000 7fb96d40b740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d8148a3c90 con 0x55d8149a0de0 2024-03-21T10:25:41.814 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:41.811+0000 7fb96d40b740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d8148f7360 con 0x55d8149978b0 2024-03-21T10:25:41.814 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:41.812+0000 7fb963400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d814998260 0x55d8149a0810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:25:41.814 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:41.812+0000 7fb963400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d814998260 0x55d8149a0810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:56126/0 (socket says 172.21.15.47:56126) 2024-03-21T10:25:41.814 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:41.812+0000 7fb963400640 1 -- 172.21.15.47:0/3090261063 learned_addr learned my addr 172.21.15.47:0/3090261063 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:25:41.815 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:41.812+0000 7fb962a00640 1 --2- 172.21.15.47:0/3090261063 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d8149a0de0 0x55d8149a3230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:25:41.815 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:41.812+0000 7fb963e00640 1 --2- 172.21.15.47:0/3090261063 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d8149978b0 0x55d814997c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:25:41.815 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:41.812+0000 7fb963400640 1 -- 172.21.15.47:0/3090261063 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d8149978b0 msgr2=0x55d814997c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:25:41.815 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:41.812+0000 7fb963400640 1 --2- 172.21.15.47:0/3090261063 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d8149978b0 0x55d814997c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:25:41.815 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:41.812+0000 7fb963400640 1 -- 172.21.15.47:0/3090261063 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d8149a0de0 msgr2=0x55d8149a3230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:25:41.815 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:41.812+0000 7fb963400640 1 --2- 172.21.15.47:0/3090261063 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d8149a0de0 0x55d8149a3230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:25:41.815 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:41.812+0000 7fb963400640 1 -- 172.21.15.47:0/3090261063 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d8148c8080 con 0x55d814998260 2024-03-21T10:25:41.815 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:41.813+0000 7fb95be00640 1 --2- 172.21.15.47:0/1796021017 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7fb9480a3120 0x7fb9480a55e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:25:41.819 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:41.813+0000 7fb963400640 1 --2- 172.21.15.47:0/3090261063 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d814998260 0x55d8149a0810 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7fb9540ef330 tx=0x7fb9540fa0e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e67b304e29579f19 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:25:41.819 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:41.813+0000 7fb963e00640 1 --2- 172.21.15.47:0/1796021017 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7fb9480a3120 0x7fb9480a55e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:25:41.819 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:41.813+0000 7fb95be00640 1 -- 172.21.15.47:0/1796021017 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 9474+0+0 (secure 0 0 0) 0x7fb9540f1fb0 con 0x7fb964103720 2024-03-21T10:25:41.819 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:41.813+0000 7fb958c00640 1 -- 172.21.15.47:0/3090261063 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9540ba220 con 0x55d814998260 2024-03-21T10:25:41.819 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:41.813+0000 7fb958c00640 1 -- 172.21.15.47:0/3090261063 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fb9540be680 con 0x55d814998260 2024-03-21T10:25:41.819 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:41.813+0000 7fb958c00640 1 -- 172.21.15.47:0/3090261063 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9540f5b40 con 0x55d814998260 2024-03-21T10:25:41.819 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:41.814+0000 7fb96d40b740 1 -- 172.21.15.47:0/3090261063 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d814998260 msgr2=0x55d8149a0810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:25:41.819 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:41.814+0000 7fb96d40b740 1 --2- 172.21.15.47:0/3090261063 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d814998260 0x55d8149a0810 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7fb9540ef330 tx=0x7fb9540fa0e0 comp rx=0 tx=0).stop 2024-03-21T10:25:41.819 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:41.814+0000 7fb963e00640 1 --2- 172.21.15.47:0/1796021017 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7fb9480a3120 0x7fb9480a55e0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fb95c0099a0 tx=0x7fb95c009500 comp rx=0 tx=0).ready entity=mgr.15687 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:25:41.819 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:41.814+0000 7fb96d40b740 1 -- 172.21.15.47:0/3090261063 shutdown_connections 2024-03-21T10:25:41.819 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:41.814+0000 7fb96d40b740 1 --2- 172.21.15.47:0/3090261063 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d8149a0de0 0x55d8149a3230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:25:41.819 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:41.814+0000 7fb96d40b740 1 --2- 172.21.15.47:0/3090261063 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d814998260 0x55d8149a0810 unknown :-1 s=CLOSED pgs=550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:25:41.820 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:41.814+0000 7fb96d40b740 1 --2- 172.21.15.47:0/3090261063 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d8149978b0 0x55d814997c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:25:41.820 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:41.814+0000 7fb96d40b740 1 -- 172.21.15.47:0/3090261063 >> 172.21.15.47:0/3090261063 conn(0x55d814991f80 msgr2=0x55d814992360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:25:41.820 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:41.814+0000 7fb96d40b740 1 -- 172.21.15.47:0/3090261063 shutdown_connections 2024-03-21T10:25:41.820 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:41.814+0000 7fb96d40b740 1 -- 172.21.15.47:0/3090261063 wait complete. 2024-03-21T10:25:41.820 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:41.815+0000 7fb96d40b740 1 -- start start 2024-03-21T10:25:41.820 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:41.815+0000 7fb96d40b740 1 Processor -- start 2024-03-21T10:25:41.820 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:41.815+0000 7fb96d40b740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d8149978b0 0x55d8149a0640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:25:41.820 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:41.815+0000 7fb96d40b740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d814998260 0x55d81499d9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:25:41.820 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:41.815+0000 7fb96d40b740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d8149a0de0 0x55d81499df10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:25:41.820 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:41.815+0000 7fb96d40b740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d8149a3b10 con 0x55d814998260 2024-03-21T10:25:41.820 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:41.815+0000 7fb96d40b740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d8148b70d0 con 0x55d8149a0de0 2024-03-21T10:25:41.820 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:41.815+0000 7fb96d40b740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d8149a3c90 con 0x55d8149978b0 2024-03-21T10:25:41.820 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:41.815+0000 7fb96d40b740 10 client.0 _ll_register_callbacks cb 0x55d8148bbe80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T10:25:41.820 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:41.816+0000 7fb962a00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d814998260 0x55d81499d9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:25:41.821 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:41.816+0000 7fb962a00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d814998260 0x55d81499d9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:56130/0 (socket says 172.21.15.47:56130) 2024-03-21T10:25:41.821 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:41.816+0000 7fb962a00640 1 -- 172.21.15.47:0/1901905203 learned_addr learned my addr 172.21.15.47:0/1901905203 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:25:41.821 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:41.816+0000 7fb963e00640 1 --2- 172.21.15.47:0/1901905203 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d8149a0de0 0x55d81499df10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:25:41.821 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:41.816+0000 7fb963400640 1 --2- 172.21.15.47:0/1901905203 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d8149978b0 0x55d8149a0640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:25:41.821 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:41.816+0000 7fb962a00640 1 -- 172.21.15.47:0/1901905203 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d8149978b0 msgr2=0x55d8149a0640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:25:41.821 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:41.816+0000 7fb962a00640 1 --2- 172.21.15.47:0/1901905203 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d8149978b0 0x55d8149a0640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:25:41.821 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:41.816+0000 7fb962a00640 1 -- 172.21.15.47:0/1901905203 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d8149a0de0 msgr2=0x55d81499df10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:25:41.821 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:41.816+0000 7fb962a00640 1 --2- 172.21.15.47:0/1901905203 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d8149a0de0 0x55d81499df10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:25:41.821 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:41.816+0000 7fb962a00640 1 -- 172.21.15.47:0/1901905203 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55d81460f3f0 con 0x55d814998260 2024-03-21T10:25:41.821 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:41.816+0000 7fb963400640 1 --2- 172.21.15.47:0/1901905203 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d8149978b0 0x55d8149a0640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T10:25:41.821 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:41.816+0000 7fb962a00640 1 --2- 172.21.15.47:0/1901905203 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d814998260 0x55d81499d9d0 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7fb95000bad0 tx=0x7fb95000bf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fa7f33397fff0e34 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:25:41.821 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:41.817+0000 7fb94d600640 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-03-21T10:25:41.821 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:41.817+0000 7fb94d600640 1 -- 172.21.15.47:0/1901905203 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb950003e20 con 0x55d814998260 2024-03-21T10:25:41.822 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:41.817+0000 7fb94d600640 1 -- 172.21.15.47:0/1901905203 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 9474+0+0 (secure 0 0 0) 0x7fb950005020 con 0x55d814998260 2024-03-21T10:25:41.822 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:41.817+0000 7fb96d40b740 1 -- 172.21.15.47:0/1901905203 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fb9540be680 con 0x55d814998260 2024-03-21T10:25:41.822 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:41.817+0000 7fb94d600640 1 -- 172.21.15.47:0/1901905203 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb950005220 con 0x55d814998260 2024-03-21T10:25:41.822 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:41.817+0000 7fb96d40b740 10 client.15855 fetch_fsmap learned FSMap version 30 2024-03-21T10:25:41.822 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:41.817+0000 7fb96d40b740 1 -- 172.21.15.47:0/1901905203 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x55d8146d97d0 con 0x55d814998260 2024-03-21T10:25:41.822 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:41.817+0000 7fb94d600640 1 -- 172.21.15.47:0/1901905203 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fb9540be680 con 0x55d814998260 2024-03-21T10:25:41.824 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:41.818+0000 7fb94d600640 1 -- 172.21.15.47:0/1901905203 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 30) v1 ==== 7942+0+0 (secure 0 0 0) 0x7fb95001b570 con 0x55d814998260 2024-03-21T10:25:41.824 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:41.818+0000 7fb96d40b740 10 client.15855 fetch_fsmap finished waiting for FSMap version 30 2024-03-21T10:25:41.824 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:41.818+0000 7fb96d40b740 10 client.15855 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.e 2024-03-21T10:25:41.824 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:41.818+0000 7fb96d40b740 20 client.15855 populate_metadata read hostname 'smithi047' 2024-03-21T10:25:41.824 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:41.818+0000 7fb96d40b740 1 --2- 172.21.15.47:0/1901905203 >> [v2:172.21.15.100:6834/2597849957,v1:172.21.15.100:6835/2597849957] conn(0x55d8149b09a0 0x55d8149b2d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:25:41.824 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:41.818+0000 7fb96d40b740 4 client.15855 mds_command: new command op to 34454 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T10:25:41.824 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:41.818+0000 7fb96d40b740 1 -- 172.21.15.47:0/1901905203 --> [v2:172.21.15.100:6834/2597849957,v1:172.21.15.100:6835/2597849957] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55d814908800 con 0x55d8149b09a0 2024-03-21T10:25:41.824 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:41.818+0000 7fb963400640 1 --2- 172.21.15.47:0/1901905203 >> [v2:172.21.15.100:6834/2597849957,v1:172.21.15.100:6835/2597849957] conn(0x55d8149b09a0 0x55d8149b2d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:25:41.824 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:41.819+0000 7fb963400640 1 --2- 172.21.15.47:0/1901905203 >> [v2:172.21.15.100:6834/2597849957,v1:172.21.15.100:6835/2597849957] conn(0x55d8149b09a0 0x55d8149b2d90 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=acbbb05d798391e6 server_cookie=7d03218d3d14e3e1 in_seq=0 out_seq=0 2024-03-21T10:25:41.824 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:41.819+0000 7fb94d600640 10 client.15855 ms_handle_connect on v2:172.21.15.100:6834/2597849957 2024-03-21T10:25:41.825 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:41.821+0000 7fb94d600640 1 -- 172.21.15.47:0/1901905203 <== mds.2 v2:172.21.15.100:6834/2597849957 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55d814908800 con 0x55d8149b09a0 2024-03-21T10:25:41.825 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:41.821+0000 7fb94d600640 10 client.15855 handle_command_reply: tid=0 2024-03-21T10:25:41.825 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:41.821+0000 7fb96d40b740 1 client.15855 shutdown 2024-03-21T10:25:41.825 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:41.821+0000 7fb96d40b740 1 -- 172.21.15.47:0/1901905203 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d814998260 msgr2=0x55d81499d9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:25:41.825 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:41.821+0000 7fb96d40b740 1 --2- 172.21.15.47:0/1901905203 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d814998260 0x55d81499d9d0 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7fb95000bad0 tx=0x7fb95000bf90 comp rx=0 tx=0).stop 2024-03-21T10:25:41.825 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:41.821+0000 7fb96d40b740 1 -- 172.21.15.47:0/1901905203 shutdown_connections 2024-03-21T10:25:41.825 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:41.821+0000 7fb96d40b740 1 --2- 172.21.15.47:0/1901905203 >> [v2:172.21.15.100:6834/2597849957,v1:172.21.15.100:6835/2597849957] conn(0x55d8149b09a0 0x55d8149b2d90 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:25:41.825 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:41.821+0000 7fb96d40b740 1 --2- 172.21.15.47:0/1901905203 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d8149a0de0 0x55d81499df10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:25:41.825 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:41.821+0000 7fb94d600640 0 client.15855 ms_handle_reset on v2:172.21.15.100:6834/2597849957 2024-03-21T10:25:41.825 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:41.821+0000 7fb96d40b740 1 --2- 172.21.15.47:0/1901905203 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d814998260 0x55d81499d9d0 unknown :-1 s=CLOSED pgs=551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:25:41.825 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:41.821+0000 7fb96d40b740 1 --2- 172.21.15.47:0/1901905203 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d8149978b0 0x55d8149a0640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:25:41.825 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:41.821+0000 7fb96d40b740 1 -- 172.21.15.47:0/1901905203 >> 172.21.15.47:0/1901905203 conn(0x55d814991f80 msgr2=0x55d814993cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:25:41.825 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:41.821+0000 7fb96d40b740 1 -- 172.21.15.47:0/1901905203 shutdown_connections 2024-03-21T10:25:41.825 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:41.821+0000 7fb96d40b740 1 -- 172.21.15.47:0/1901905203 wait complete. 2024-03-21T10:25:41.826 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:41.821+0000 7fb96d40b740 20 client.15855 trim_cache size 0 max 16384 2024-03-21T10:25:41.838 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:41.835+0000 7fb96d40b740 1 Processor -- start 2024-03-21T10:25:41.838 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:41.835+0000 7fb96d40b740 1 -- start start 2024-03-21T10:25:41.838 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:41.835+0000 7fb96d40b740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d8149978b0 0x55d8149a0100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:25:41.838 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:41.835+0000 7fb96d40b740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d814998260 0x55d8149a0640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:25:41.838 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:41.835+0000 7fb96d40b740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d8149a0de0 0x55d81499ac50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:25:41.838 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:41.835+0000 7fb96d40b740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9641b0930 con 0x55d8149978b0 2024-03-21T10:25:41.838 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:41.835+0000 7fb96d40b740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9641b22b0 con 0x55d8149a0de0 2024-03-21T10:25:41.838 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:41.835+0000 7fb96d40b740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9641b0f90 con 0x55d814998260 2024-03-21T10:25:41.838 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:41.836+0000 7fb963400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d8149978b0 0x55d8149a0100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:25:41.838 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:41.836+0000 7fb963400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d8149978b0 0x55d8149a0100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:56134/0 (socket says 172.21.15.47:56134) 2024-03-21T10:25:41.838 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:41.836+0000 7fb963400640 1 -- 172.21.15.47:0/2927332123 learned_addr learned my addr 172.21.15.47:0/2927332123 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:25:41.838 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:41.836+0000 7fb963e00640 1 --2- 172.21.15.47:0/2927332123 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d8149a0de0 0x55d81499ac50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:25:41.839 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:41.836+0000 7fb963400640 1 -- 172.21.15.47:0/2927332123 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d814998260 msgr2=0x55d8149a0640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:25:41.839 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:41.836+0000 7fb963400640 1 --2- 172.21.15.47:0/2927332123 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d814998260 0x55d8149a0640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:25:41.839 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:41.836+0000 7fb963400640 1 -- 172.21.15.47:0/2927332123 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d8149a0de0 msgr2=0x55d81499ac50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:25:41.839 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:41.836+0000 7fb963400640 1 --2- 172.21.15.47:0/2927332123 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d8149a0de0 0x55d81499ac50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:25:41.839 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:41.836+0000 7fb963400640 1 -- 172.21.15.47:0/2927332123 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d814996940 con 0x55d8149978b0 2024-03-21T10:25:41.839 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:41.836+0000 7fb963400640 1 --2- 172.21.15.47:0/2927332123 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d8149978b0 0x55d8149a0100 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7fb9540bdc80 tx=0x7fb9540fa0e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f8f021c3d6b4e024 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:25:41.843 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:41.836+0000 7fb94fe00640 1 -- 172.21.15.47:0/2927332123 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9540bd8e0 con 0x55d8149978b0 2024-03-21T10:25:41.843 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:41.837+0000 7fb94fe00640 1 -- 172.21.15.47:0/2927332123 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fb9540be680 con 0x55d8149978b0 2024-03-21T10:25:41.843 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:41.837+0000 7fb94fe00640 1 -- 172.21.15.47:0/2927332123 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9540bdac0 con 0x55d8149978b0 2024-03-21T10:25:41.843 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:41.837+0000 7fb96d40b740 1 -- 172.21.15.47:0/2927332123 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d8149978b0 msgr2=0x55d8149a0100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:25:41.843 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:41.837+0000 7fb96d40b740 1 --2- 172.21.15.47:0/2927332123 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d8149978b0 0x55d8149a0100 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7fb9540bdc80 tx=0x7fb9540fa0e0 comp rx=0 tx=0).stop 2024-03-21T10:25:41.843 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:41.837+0000 7fb96d40b740 1 -- 172.21.15.47:0/2927332123 shutdown_connections 2024-03-21T10:25:41.843 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:41.837+0000 7fb96d40b740 1 --2- 172.21.15.47:0/2927332123 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d8149a0de0 0x55d81499ac50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:25:41.843 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:41.837+0000 7fb96d40b740 1 --2- 172.21.15.47:0/2927332123 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d814998260 0x55d8149a0640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:25:41.843 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:41.837+0000 7fb96d40b740 1 --2- 172.21.15.47:0/2927332123 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d8149978b0 0x55d8149a0100 unknown :-1 s=CLOSED pgs=552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:25:41.843 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:41.837+0000 7fb96d40b740 1 -- 172.21.15.47:0/2927332123 >> 172.21.15.47:0/2927332123 conn(0x55d814991f80 msgr2=0x55d814993c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:25:41.844 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:41.837+0000 7fb96d40b740 1 -- 172.21.15.47:0/2927332123 shutdown_connections 2024-03-21T10:25:41.844 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:41.837+0000 7fb96d40b740 1 -- 172.21.15.47:0/2927332123 wait complete. 2024-03-21T10:25:41.844 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:41.838+0000 7fb96d40b740 1 -- start start 2024-03-21T10:25:41.844 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:41.838+0000 7fb96d40b740 1 Processor -- start 2024-03-21T10:25:41.844 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:41.838+0000 7fb96d40b740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d8149978b0 0x55d814a05570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:25:41.844 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:41.838+0000 7fb96d40b740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d814998260 0x55d814a05ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:25:41.844 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:41.838+0000 7fb96d40b740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d8149a0de0 0x55d8149a4ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:25:41.844 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:41.838+0000 7fb96d40b740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9641b1c50 con 0x55d814998260 2024-03-21T10:25:41.844 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:41.838+0000 7fb96d40b740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9641b15f0 con 0x55d8149a0de0 2024-03-21T10:25:41.844 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:41.838+0000 7fb96d40b740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9641aeaf0 con 0x55d8149978b0 2024-03-21T10:25:41.844 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:41.838+0000 7fb96d40b740 10 client.0 _ll_register_callbacks cb 0x7fb9500040a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T10:25:41.844 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:41.839+0000 7fb962a00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d814998260 0x55d814a05ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:25:41.845 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:41.839+0000 7fb962a00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d814998260 0x55d814a05ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:56150/0 (socket says 172.21.15.47:56150) 2024-03-21T10:25:41.845 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:41.839+0000 7fb962a00640 1 -- 172.21.15.47:0/2065894155 learned_addr learned my addr 172.21.15.47:0/2065894155 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:25:41.845 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:41.839+0000 7fb963e00640 1 --2- 172.21.15.47:0/2065894155 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d8149a0de0 0x55d8149a4ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:25:41.845 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:41.839+0000 7fb963400640 1 --2- 172.21.15.47:0/2065894155 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d8149978b0 0x55d814a05570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:25:41.845 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:41.839+0000 7fb962a00640 1 -- 172.21.15.47:0/2065894155 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d8149978b0 msgr2=0x55d814a05570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:25:41.845 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:41.839+0000 7fb962a00640 1 --2- 172.21.15.47:0/2065894155 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d8149978b0 0x55d814a05570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:25:41.845 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:41.839+0000 7fb962a00640 1 -- 172.21.15.47:0/2065894155 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d8149a0de0 msgr2=0x55d8149a4ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:25:41.845 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:41.839+0000 7fb962a00640 1 --2- 172.21.15.47:0/2065894155 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d8149a0de0 0x55d8149a4ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:25:41.845 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:41.839+0000 7fb962a00640 1 -- 172.21.15.47:0/2065894155 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55d8149a51d0 con 0x55d814998260 2024-03-21T10:25:41.845 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:41.840+0000 7fb962a00640 1 --2- 172.21.15.47:0/2065894155 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d814998260 0x55d814a05ab0 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7fb95001b990 tx=0x7fb950005e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d356ec86122c5a01 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:25:41.845 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:41.840+0000 7fb94d600640 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-03-21T10:25:41.845 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:41.840+0000 7fb94d600640 1 -- 172.21.15.47:0/2065894155 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb950005070 con 0x55d814998260 2024-03-21T10:25:41.845 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:41.840+0000 7fb94d600640 1 -- 172.21.15.47:0/2065894155 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 9474+0+0 (secure 0 0 0) 0x7fb95001b470 con 0x55d814998260 2024-03-21T10:25:41.846 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:41.840+0000 7fb96d40b740 1 -- 172.21.15.47:0/2065894155 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fb9540be680 con 0x55d814998260 2024-03-21T10:25:41.846 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:41.840+0000 7fb94d600640 1 -- 172.21.15.47:0/2065894155 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb950036050 con 0x55d814998260 2024-03-21T10:25:41.846 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:41.840+0000 7fb96d40b740 10 client.15861 fetch_fsmap learned FSMap version 30 2024-03-21T10:25:41.846 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:41.840+0000 7fb96d40b740 1 -- 172.21.15.47:0/2065894155 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x55d814a0cd10 con 0x55d814998260 2024-03-21T10:25:41.846 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:41.841+0000 7fb94d600640 1 -- 172.21.15.47:0/2065894155 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fb9540be680 con 0x55d814998260 2024-03-21T10:25:41.846 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:41.841+0000 7fb94d600640 1 -- 172.21.15.47:0/2065894155 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 30) v1 ==== 7942+0+0 (secure 0 0 0) 0x7fb950004af0 con 0x55d814998260 2024-03-21T10:25:41.846 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:41.841+0000 7fb96d40b740 10 client.15861 fetch_fsmap finished waiting for FSMap version 30 2024-03-21T10:25:41.846 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:41.841+0000 7fb96d40b740 10 client.15861 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.e 2024-03-21T10:25:41.846 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:41.841+0000 7fb96d40b740 20 client.15861 populate_metadata read hostname 'smithi047' 2024-03-21T10:25:41.846 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:41.841+0000 7fb96d40b740 1 --2- 172.21.15.47:0/2065894155 >> [v2:172.21.15.100:6834/2597849957,v1:172.21.15.100:6835/2597849957] conn(0x55d814a0d400 0x55d814a0f820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:25:41.846 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:41.841+0000 7fb96d40b740 4 client.15861 mds_command: new command op to 34454 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-21T10:25:41.846 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:41.841+0000 7fb96d40b740 1 -- 172.21.15.47:0/2065894155 --> [v2:172.21.15.100:6834/2597849957,v1:172.21.15.100:6835/2597849957] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55d814902b50 con 0x55d814a0d400 2024-03-21T10:25:41.847 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:41.842+0000 7fb963400640 1 --2- 172.21.15.47:0/2065894155 >> [v2:172.21.15.100:6834/2597849957,v1:172.21.15.100:6835/2597849957] conn(0x55d814a0d400 0x55d814a0f820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:25:41.848 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:41.842+0000 7fb963400640 1 --2- 172.21.15.47:0/2065894155 >> [v2:172.21.15.100:6834/2597849957,v1:172.21.15.100:6835/2597849957] conn(0x55d814a0d400 0x55d814a0f820 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=5449f595360ddac6 server_cookie=2e99ef85be114030 in_seq=0 out_seq=0 2024-03-21T10:25:41.848 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:41.842+0000 7fb94d600640 10 client.15861 ms_handle_connect on v2:172.21.15.100:6834/2597849957 2024-03-21T10:25:41.848 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:41.843+0000 7fb94d600640 1 -- 172.21.15.47:0/2065894155 <== mds.2 v2:172.21.15.100:6834/2597849957 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55d814902b50 con 0x55d814a0d400 2024-03-21T10:25:41.848 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:41.843+0000 7fb94d600640 10 client.15861 handle_command_reply: tid=0 2024-03-21T10:25:41.848 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:41.843+0000 7fb96d40b740 1 client.15861 shutdown 2024-03-21T10:25:41.848 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:41.843+0000 7fb96d40b740 1 -- 172.21.15.47:0/2065894155 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d814998260 msgr2=0x55d814a05ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:25:41.848 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:41.843+0000 7fb96d40b740 1 --2- 172.21.15.47:0/2065894155 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d814998260 0x55d814a05ab0 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7fb95001b990 tx=0x7fb950005e50 comp rx=0 tx=0).stop 2024-03-21T10:25:41.848 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:41.843+0000 7fb96d40b740 1 -- 172.21.15.47:0/2065894155 shutdown_connections 2024-03-21T10:25:41.848 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:41.843+0000 7fb96d40b740 1 --2- 172.21.15.47:0/2065894155 >> [v2:172.21.15.100:6834/2597849957,v1:172.21.15.100:6835/2597849957] conn(0x55d814a0d400 0x55d814a0f820 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:25:41.848 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:41.843+0000 7fb96d40b740 1 --2- 172.21.15.47:0/2065894155 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d8149a0de0 0x55d8149a4ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:25:41.849 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:41.843+0000 7fb96d40b740 1 --2- 172.21.15.47:0/2065894155 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d814998260 0x55d814a05ab0 unknown :-1 s=CLOSED pgs=553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:25:41.849 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:41.843+0000 7fb96d40b740 1 --2- 172.21.15.47:0/2065894155 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d8149978b0 0x55d814a05570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:25:41.849 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:41.843+0000 7fb94d600640 0 client.15861 ms_handle_reset on v2:172.21.15.100:6834/2597849957 2024-03-21T10:25:41.849 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:41.843+0000 7fb96d40b740 1 -- 172.21.15.47:0/2065894155 >> 172.21.15.47:0/2065894155 conn(0x55d814991f80 msgr2=0x55d8149933b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:25:41.849 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:41.843+0000 7fb96d40b740 1 -- 172.21.15.47:0/2065894155 shutdown_connections 2024-03-21T10:25:41.849 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:41.843+0000 7fb96d40b740 1 -- 172.21.15.47:0/2065894155 wait complete. 2024-03-21T10:25:41.849 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:41.843+0000 7fb96d40b740 20 client.15861 trim_cache size 0 max 16384 2024-03-21T10:25:41.849 INFO:teuthology.orchestra.run.smithi047.stdout:[] 2024-03-21T10:25:41.850 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:41.844+0000 7fb969000640 1 -- 172.21.15.47:0/1796021017 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7fb9480a3120 msgr2=0x7fb9480a55e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:25:41.850 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:41.844+0000 7fb969000640 1 --2- 172.21.15.47:0/1796021017 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7fb9480a3120 0x7fb9480a55e0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fb95c0099a0 tx=0x7fb95c009500 comp rx=0 tx=0).stop 2024-03-21T10:25:41.850 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:41.844+0000 7fb969000640 1 -- 172.21.15.47:0/1796021017 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb964103720 msgr2=0x7fb964199f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:25:41.850 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:41.844+0000 7fb969000640 1 --2- 172.21.15.47:0/1796021017 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb964103720 0x7fb964199f60 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7fb954000c00 tx=0x7fb9540045e0 comp rx=0 tx=0).stop 2024-03-21T10:25:41.851 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:41.844+0000 7fb969000640 1 -- 172.21.15.47:0/1796021017 shutdown_connections 2024-03-21T10:25:41.851 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:41.844+0000 7fb969000640 1 --2- 172.21.15.47:0/1796021017 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb964109730 0x7fb96419e820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:25:41.851 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:41.844+0000 7fb969000640 1 --2- 172.21.15.47:0/1796021017 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7fb9480a3120 0x7fb9480a55e0 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:25:41.851 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:41.844+0000 7fb969000640 1 --2- 172.21.15.47:0/1796021017 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb9641040e0 0x7fb96419a4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:25:41.851 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:41.844+0000 7fb969000640 1 --2- 172.21.15.47:0/1796021017 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb964103720 0x7fb964199f60 unknown :-1 s=CLOSED pgs=549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:25:41.851 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:41.844+0000 7fb969000640 1 -- 172.21.15.47:0/1796021017 >> 172.21.15.47:0/1796021017 conn(0x7fb9640ff1b0 msgr2=0x7fb9640ff9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:25:41.851 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:41.844+0000 7fb969000640 1 -- 172.21.15.47:0/1796021017 shutdown_connections 2024-03-21T10:25:41.851 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:41.844+0000 7fb969000640 1 -- 172.21.15.47:0/1796021017 wait complete. 2024-03-21T10:25:41.863 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-21T10:25:41.863 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-03-21T10:25:42.071 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:42.069+0000 7f68c6a00640 1 -- 172.21.15.47:0/3843136964 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f68c010de60 msgr2=0x7f68c0110250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:25:42.071 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:42.069+0000 7f68c6a00640 1 --2- 172.21.15.47:0/3843136964 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f68c010de60 0x7f68c0110250 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f68b8009ca0 tx=0x7f68b802c0f0 comp rx=0 tx=0).stop 2024-03-21T10:25:42.072 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:42.069+0000 7f68c6a00640 1 -- 172.21.15.47:0/3843136964 shutdown_connections 2024-03-21T10:25:42.072 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:42.069+0000 7f68c6a00640 1 --2- 172.21.15.47:0/3843136964 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f68c010de60 0x7f68c0110250 unknown :-1 s=CLOSED pgs=554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:25:42.072 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:42.069+0000 7f68c6a00640 1 --2- 172.21.15.47:0/3843136964 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f68c0100c60 0x7f68c010d920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:25:42.072 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:42.069+0000 7f68c6a00640 1 --2- 172.21.15.47:0/3843136964 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f68c01002b0 0x7f68c0100690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:25:42.072 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:42.069+0000 7f68c6a00640 1 -- 172.21.15.47:0/3843136964 >> 172.21.15.47:0/3843136964 conn(0x7f68c00fbef0 msgr2=0x7f68c00fe310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:25:42.072 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:42.070+0000 7f68c6a00640 1 -- 172.21.15.47:0/3843136964 shutdown_connections 2024-03-21T10:25:42.072 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:42.070+0000 7f68c6a00640 1 -- 172.21.15.47:0/3843136964 wait complete. 2024-03-21T10:25:42.074 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:42.071+0000 7f68c6a00640 1 Processor -- start 2024-03-21T10:25:42.074 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:42.071+0000 7f68c6a00640 1 -- start start 2024-03-21T10:25:42.075 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:42.071+0000 7f68c6a00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f68c01002b0 0x7f68c0199f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:25:42.075 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:42.071+0000 7f68c6a00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f68c0100c60 0x7f68c019a440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:25:42.075 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:42.071+0000 7f68c6a00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f68c010de60 0x7f68c019e7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:25:42.075 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:42.071+0000 7f68c6a00640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68c01139c0 con 0x7f68c0100c60 2024-03-21T10:25:42.075 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:42.071+0000 7f68c6a00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68c0113840 con 0x7f68c01002b0 2024-03-21T10:25:42.075 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:42.071+0000 7f68c6a00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68c0113b40 con 0x7f68c010de60 2024-03-21T10:25:42.075 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:42.072+0000 7f68c4c00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f68c0100c60 0x7f68c019a440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:25:42.075 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:42.072+0000 7f68c4c00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f68c0100c60 0x7f68c019a440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:54958/0 (socket says 172.21.15.47:54958) 2024-03-21T10:25:42.075 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:42.072+0000 7f68c4c00640 1 -- 172.21.15.47:0/433304381 learned_addr learned my addr 172.21.15.47:0/433304381 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:25:42.075 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:42.072+0000 7f68c5600640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f68c01002b0 0x7f68c0199f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:25:42.075 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:42.072+0000 7f68c6000640 1 --2- 172.21.15.47:0/433304381 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f68c010de60 0x7f68c019e7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:25:42.075 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:42.072+0000 7f68c4c00640 1 -- 172.21.15.47:0/433304381 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f68c010de60 msgr2=0x7f68c019e7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:25:42.075 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:42.072+0000 7f68c4c00640 1 --2- 172.21.15.47:0/433304381 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f68c010de60 0x7f68c019e7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:25:42.076 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:42.072+0000 7f68c4c00640 1 -- 172.21.15.47:0/433304381 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f68c01002b0 msgr2=0x7f68c0199f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:25:42.076 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:42.072+0000 7f68c4c00640 1 --2- 172.21.15.47:0/433304381 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f68c01002b0 0x7f68c0199f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:25:42.076 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:42.072+0000 7f68c4c00640 1 -- 172.21.15.47:0/433304381 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68c019eef0 con 0x7f68c0100c60 2024-03-21T10:25:42.076 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:42.072+0000 7f68c4c00640 1 --2- 172.21.15.47:0/433304381 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f68c0100c60 0x7f68c019a440 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f68ac00d950 tx=0x7f68ac00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:25:42.076 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:42.073+0000 7f68be000640 1 -- 172.21.15.47:0/433304381 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68ac017070 con 0x7f68c0100c60 2024-03-21T10:25:42.076 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:42.073+0000 7f68be000640 1 -- 172.21.15.47:0/433304381 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f68ac009c30 con 0x7f68c0100c60 2024-03-21T10:25:42.076 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:42.073+0000 7f68c6a00640 1 -- 172.21.15.47:0/433304381 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f68c019f1e0 con 0x7f68c0100c60 2024-03-21T10:25:42.076 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:42.073+0000 7f68be000640 1 -- 172.21.15.47:0/433304381 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68ac00f620 con 0x7f68c0100c60 2024-03-21T10:25:42.076 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:42.073+0000 7f68c6a00640 1 -- 172.21.15.47:0/433304381 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f68c01a6a70 con 0x7f68c0100c60 2024-03-21T10:25:42.079 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:42.076+0000 7f68be000640 1 -- 172.21.15.47:0/433304381 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 25) v1 ==== 138030+0+0 (secure 0 0 0) 0x7f68ac0109a0 con 0x7f68c0100c60 2024-03-21T10:25:42.079 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:42.076+0000 7f68cabe0740 1 Processor -- start 2024-03-21T10:25:42.079 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:42.077+0000 7f68cabe0740 1 -- start start 2024-03-21T10:25:42.079 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:42.077+0000 7f68cabe0740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f644bc08b0 0x55f644bc0c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:25:42.080 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:42.077+0000 7f68cabe0740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f644bc1260 0x55f644bc9810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:25:42.080 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:42.077+0000 7f68cabe0740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f644bc9de0 0x55f644bcc230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:25:42.080 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:42.077+0000 7f68cabe0740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f644b242c0 con 0x55f644bc1260 2024-03-21T10:25:42.080 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:42.077+0000 7f68cabe0740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f644accc90 con 0x55f644bc9de0 2024-03-21T10:25:42.080 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:42.077+0000 7f68cabe0740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f644b20360 con 0x55f644bc08b0 2024-03-21T10:25:42.080 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:42.077+0000 7f68c5600640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f644bc1260 0x55f644bc9810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:25:42.080 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:42.077+0000 7f68c5600640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f644bc1260 0x55f644bc9810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:54970/0 (socket says 172.21.15.47:54970) 2024-03-21T10:25:42.080 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:42.077+0000 7f68c5600640 1 -- 172.21.15.47:0/915344106 learned_addr learned my addr 172.21.15.47:0/915344106 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:25:42.080 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:42.078+0000 7f68c4c00640 1 --2- 172.21.15.47:0/915344106 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f644bc9de0 0x55f644bcc230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:25:42.080 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:42.078+0000 7f68c6000640 1 --2- 172.21.15.47:0/915344106 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f644bc08b0 0x55f644bc0c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:25:42.080 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:42.078+0000 7f68c5600640 1 -- 172.21.15.47:0/915344106 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f644bc08b0 msgr2=0x55f644bc0c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:25:42.080 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:42.078+0000 7f68c5600640 1 --2- 172.21.15.47:0/915344106 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f644bc08b0 0x55f644bc0c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:25:42.080 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:42.078+0000 7f68c5600640 1 -- 172.21.15.47:0/915344106 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f644bc9de0 msgr2=0x55f644bcc230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:25:42.080 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:42.078+0000 7f68c5600640 1 --2- 172.21.15.47:0/915344106 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f644bc9de0 0x55f644bcc230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:25:42.081 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:42.078+0000 7f68c5600640 1 -- 172.21.15.47:0/915344106 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f644af1080 con 0x55f644bc1260 2024-03-21T10:25:42.082 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:42.078+0000 7f68c5600640 1 --2- 172.21.15.47:0/915344106 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f644bc1260 0x55f644bc9810 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f68b4009cf0 tx=0x7f68b4011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a5cab132a546c759 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:25:42.082 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:42.078+0000 7f68b2600640 1 -- 172.21.15.47:0/915344106 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68b4004230 con 0x55f644bc1260 2024-03-21T10:25:42.082 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:42.079+0000 7f68b2600640 1 -- 172.21.15.47:0/915344106 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f68b40043d0 con 0x55f644bc1260 2024-03-21T10:25:42.082 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:42.079+0000 7f68b2600640 1 -- 172.21.15.47:0/915344106 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68b40197e0 con 0x55f644bc1260 2024-03-21T10:25:42.083 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:42.079+0000 7f68be000640 1 --2- 172.21.15.47:0/433304381 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f68a80a3120 0x7f68a80a55e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:25:42.083 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:42.079+0000 7f68be000640 1 -- 172.21.15.47:0/433304381 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 9474+0+0 (secure 0 0 0) 0x7f68ac0cf8e0 con 0x7f68c0100c60 2024-03-21T10:25:42.083 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:42.079+0000 7f68c6000640 1 --2- 172.21.15.47:0/433304381 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f68a80a3120 0x7f68a80a55e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:25:42.083 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:42.079+0000 7f68cabe0740 1 -- 172.21.15.47:0/915344106 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f644bc1260 msgr2=0x55f644bc9810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:25:42.083 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:42.079+0000 7f68cabe0740 1 --2- 172.21.15.47:0/915344106 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f644bc1260 0x55f644bc9810 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f68b4009cf0 tx=0x7f68b4011810 comp rx=0 tx=0).stop 2024-03-21T10:25:42.083 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:42.080+0000 7f68cabe0740 1 -- 172.21.15.47:0/915344106 shutdown_connections 2024-03-21T10:25:42.083 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:42.080+0000 7f68cabe0740 1 --2- 172.21.15.47:0/915344106 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f644bc9de0 0x55f644bcc230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:25:42.083 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:42.080+0000 7f68cabe0740 1 --2- 172.21.15.47:0/915344106 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f644bc1260 0x55f644bc9810 unknown :-1 s=CLOSED pgs=556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:25:42.083 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:42.080+0000 7f68cabe0740 1 --2- 172.21.15.47:0/915344106 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f644bc08b0 0x55f644bc0c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:25:42.083 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:42.080+0000 7f68cabe0740 1 -- 172.21.15.47:0/915344106 >> 172.21.15.47:0/915344106 conn(0x55f644bbaf80 msgr2=0x55f644bbb360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:25:42.083 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:42.080+0000 7f68cabe0740 1 -- 172.21.15.47:0/915344106 shutdown_connections 2024-03-21T10:25:42.083 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:42.080+0000 7f68c6000640 1 --2- 172.21.15.47:0/433304381 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f68a80a3120 0x7f68a80a55e0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f68b8006010 tx=0x7f68b8005f80 comp rx=0 tx=0).ready entity=mgr.15687 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:25:42.083 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:42.080+0000 7f68cabe0740 1 -- 172.21.15.47:0/915344106 wait complete. 2024-03-21T10:25:42.083 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:42.080+0000 7f68cabe0740 1 -- start start 2024-03-21T10:25:42.084 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:42.081+0000 7f68cabe0740 1 Processor -- start 2024-03-21T10:25:42.087 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:42.081+0000 7f68cabe0740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f644bc08b0 0x55f644bc9630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:25:42.088 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:42.081+0000 7f68cabe0740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f644bc1260 0x55f644bc69c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:25:42.088 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:42.081+0000 7f68cabe0740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f644bc9de0 0x55f644bc6f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:25:42.088 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:42.081+0000 7f68cabe0740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f644bccd50 con 0x55f644bc9de0 2024-03-21T10:25:42.088 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:42.081+0000 7f68cabe0740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f644ae00d0 con 0x55f644bc1260 2024-03-21T10:25:42.088 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:42.081+0000 7f68cabe0740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f644bcced0 con 0x55f644bc08b0 2024-03-21T10:25:42.088 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:42.081+0000 7f68cabe0740 10 client.0 _ll_register_callbacks cb 0x55f644ae4e80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T10:25:42.088 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:42.082+0000 7f68c6000640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f644bc9de0 0x55f644bc6f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:25:42.088 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:42.082+0000 7f68c4c00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f644bc1260 0x55f644bc69c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:25:42.088 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:42.082+0000 7f68c4c00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f644bc1260 0x55f644bc69c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:39328/0 (socket says 172.21.15.47:39328) 2024-03-21T10:25:42.088 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:42.082+0000 7f68c4c00640 1 -- 172.21.15.47:0/3387443767 learned_addr learned my addr 172.21.15.47:0/3387443767 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:25:42.088 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:42.082+0000 7f68c5600640 1 --2- 172.21.15.47:0/3387443767 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f644bc08b0 0x55f644bc9630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:25:42.088 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:42.082+0000 7f68c4c00640 1 -- 172.21.15.47:0/3387443767 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f644bc08b0 msgr2=0x55f644bc9630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:25:42.088 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:42.082+0000 7f68c4c00640 1 --2- 172.21.15.47:0/3387443767 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f644bc08b0 0x55f644bc9630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:25:42.089 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:42.082+0000 7f68c4c00640 1 -- 172.21.15.47:0/3387443767 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f644bc9de0 msgr2=0x55f644bc6f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:25:42.089 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:42.082+0000 7f68c4c00640 1 --2- 172.21.15.47:0/3387443767 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f644bc9de0 0x55f644bc6f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:25:42.089 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:42.082+0000 7f68c4c00640 1 -- 172.21.15.47:0/3387443767 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55f6448383f0 con 0x55f644bc1260 2024-03-21T10:25:42.089 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:42.082+0000 7f68c4c00640 1 --2- 172.21.15.47:0/3387443767 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f644bc1260 0x55f644bc69c0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f68ac000c00 tx=0x7f68ac09b5e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f95cff5aab91333a server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:25:42.089 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:42.082+0000 7f68a6a00640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-21T10:25:42.089 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:42.083+0000 7f68a6a00640 1 -- 172.21.15.47:0/3387443767 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68ac020050 con 0x55f644bc1260 2024-03-21T10:25:42.089 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:42.083+0000 7f68a6a00640 1 -- 172.21.15.47:0/3387443767 <== mon.2 v2:172.21.15.100:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 9474+0+0 (secure 0 0 0) 0x7f68ac09f580 con 0x55f644bc1260 2024-03-21T10:25:42.089 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:42.083+0000 7f68cabe0740 1 -- 172.21.15.47:0/3387443767 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f68b40043d0 con 0x55f644bc1260 2024-03-21T10:25:42.089 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:42.083+0000 7f68a6a00640 1 -- 172.21.15.47:0/3387443767 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f68ac009de0 con 0x55f644bc1260 2024-03-21T10:25:42.089 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:42.083+0000 7f68cabe0740 10 client.35684 fetch_fsmap learned FSMap version 30 2024-03-21T10:25:42.089 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:42.083+0000 7f68cabe0740 1 -- 172.21.15.47:0/3387443767 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x55f6449027d0 con 0x55f644bc1260 2024-03-21T10:25:42.089 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:42.083+0000 7f68a6a00640 1 -- 172.21.15.47:0/3387443767 <== mon.2 v2:172.21.15.100:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f68b40043d0 con 0x55f644bc1260 2024-03-21T10:25:42.089 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:42.084+0000 7f68a6a00640 1 -- 172.21.15.47:0/3387443767 <== mon.2 v2:172.21.15.100:3300/0 5 ==== fsmap(e 30) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f68ac084ce0 con 0x55f644bc1260 2024-03-21T10:25:42.089 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:42.084+0000 7f68cabe0740 10 client.35684 fetch_fsmap finished waiting for FSMap version 30 2024-03-21T10:25:42.090 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:42.084+0000 7f68cabe0740 10 client.35684 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-03-21T10:25:42.090 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:42.084+0000 7f68cabe0740 20 client.35684 populate_metadata read hostname 'smithi047' 2024-03-21T10:25:42.090 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:42.084+0000 7f68cabe0740 1 --2- 172.21.15.47:0/3387443767 >> [v2:172.21.15.100:6836/3497644162,v1:172.21.15.100:6837/3497644162] conn(0x55f644bd9c30 0x55f644bdc050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:25:42.090 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:42.084+0000 7f68cabe0740 4 client.35684 mds_command: new command op to 34493 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T10:25:42.090 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:42.084+0000 7f68cabe0740 1 -- 172.21.15.47:0/3387443767 --> [v2:172.21.15.100:6836/3497644162,v1:172.21.15.100:6837/3497644162] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55f644b31800 con 0x55f644bd9c30 2024-03-21T10:25:42.090 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:42.084+0000 7f68c5600640 1 --2- 172.21.15.47:0/3387443767 >> [v2:172.21.15.100:6836/3497644162,v1:172.21.15.100:6837/3497644162] conn(0x55f644bd9c30 0x55f644bdc050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:25:42.090 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:42.085+0000 7f68c5600640 1 --2- 172.21.15.47:0/3387443767 >> [v2:172.21.15.100:6836/3497644162,v1:172.21.15.100:6837/3497644162] conn(0x55f644bd9c30 0x55f644bdc050 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=931c41c27bd3ec2a server_cookie=5ed2ed2cd857fddf in_seq=0 out_seq=0 2024-03-21T10:25:42.090 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:42.085+0000 7f68a6a00640 10 client.35684 ms_handle_connect on v2:172.21.15.100:6836/3497644162 2024-03-21T10:25:42.090 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:42.085+0000 7f68a6a00640 1 -- 172.21.15.47:0/3387443767 <== mds.1 v2:172.21.15.100:6836/3497644162 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55f644b31800 con 0x55f644bd9c30 2024-03-21T10:25:42.090 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:42.085+0000 7f68a6a00640 10 client.35684 handle_command_reply: tid=0 2024-03-21T10:25:42.090 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:42.086+0000 7f68cabe0740 1 client.35684 shutdown 2024-03-21T10:25:42.090 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:42.086+0000 7f68cabe0740 1 -- 172.21.15.47:0/3387443767 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f644bc1260 msgr2=0x55f644bc69c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:25:42.090 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:42.086+0000 7f68cabe0740 1 --2- 172.21.15.47:0/3387443767 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f644bc1260 0x55f644bc69c0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f68ac000c00 tx=0x7f68ac09b5e0 comp rx=0 tx=0).stop 2024-03-21T10:25:42.090 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:42.086+0000 7f68cabe0740 1 -- 172.21.15.47:0/3387443767 shutdown_connections 2024-03-21T10:25:42.091 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:42.086+0000 7f68cabe0740 1 --2- 172.21.15.47:0/3387443767 >> [v2:172.21.15.100:6836/3497644162,v1:172.21.15.100:6837/3497644162] conn(0x55f644bd9c30 0x55f644bdc050 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:25:42.091 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:42.086+0000 7f68cabe0740 1 --2- 172.21.15.47:0/3387443767 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f644bc9de0 0x55f644bc6f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:25:42.091 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:42.086+0000 7f68cabe0740 1 --2- 172.21.15.47:0/3387443767 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f644bc1260 0x55f644bc69c0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:25:42.091 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:42.086+0000 7f68cabe0740 1 --2- 172.21.15.47:0/3387443767 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f644bc08b0 0x55f644bc9630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:25:42.091 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:42.086+0000 7f68a6a00640 0 client.35684 ms_handle_reset on v2:172.21.15.100:6836/3497644162 2024-03-21T10:25:42.091 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:42.086+0000 7f68cabe0740 1 -- 172.21.15.47:0/3387443767 >> 172.21.15.47:0/3387443767 conn(0x55f644bbaf80 msgr2=0x55f644bbccd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:25:42.091 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:42.086+0000 7f68cabe0740 1 -- 172.21.15.47:0/3387443767 shutdown_connections 2024-03-21T10:25:42.091 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:42.086+0000 7f68cabe0740 1 -- 172.21.15.47:0/3387443767 wait complete. 2024-03-21T10:25:42.091 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:42.086+0000 7f68cabe0740 20 client.35684 trim_cache size 0 max 16384 2024-03-21T10:25:42.102 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:42.100+0000 7f68cabe0740 1 Processor -- start 2024-03-21T10:25:42.102 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:42.100+0000 7f68cabe0740 1 -- start start 2024-03-21T10:25:42.103 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:42.100+0000 7f68cabe0740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f644bc08b0 0x55f644bc90f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:25:42.103 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:42.100+0000 7f68cabe0740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f644bc1260 0x55f644bc9630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:25:42.103 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:42.100+0000 7f68cabe0740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f644bc9de0 0x55f644bc3c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:25:42.103 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:42.100+0000 7f68cabe0740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68c01b0870 con 0x55f644bc1260 2024-03-21T10:25:42.103 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:42.100+0000 7f68cabe0740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68c01b21f0 con 0x55f644bc9de0 2024-03-21T10:25:42.103 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:42.100+0000 7f68cabe0740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68c01b0ed0 con 0x55f644bc08b0 2024-03-21T10:25:42.103 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:42.100+0000 7f68c4c00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f644bc1260 0x55f644bc9630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:25:42.103 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:42.100+0000 7f68c4c00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f644bc1260 0x55f644bc9630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:54992/0 (socket says 172.21.15.47:54992) 2024-03-21T10:25:42.103 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:42.100+0000 7f68c4c00640 1 -- 172.21.15.47:0/3157620056 learned_addr learned my addr 172.21.15.47:0/3157620056 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:25:42.103 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:42.101+0000 7f68c6000640 1 --2- 172.21.15.47:0/3157620056 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f644bc9de0 0x55f644bc3c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:25:42.103 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:42.101+0000 7f68c5600640 1 --2- 172.21.15.47:0/3157620056 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f644bc08b0 0x55f644bc90f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:25:42.103 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:42.101+0000 7f68c4c00640 1 -- 172.21.15.47:0/3157620056 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f644bc08b0 msgr2=0x55f644bc90f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:25:42.103 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:42.101+0000 7f68c4c00640 1 --2- 172.21.15.47:0/3157620056 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f644bc08b0 0x55f644bc90f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:25:42.104 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:42.101+0000 7f68c4c00640 1 -- 172.21.15.47:0/3157620056 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f644bc9de0 msgr2=0x55f644bc3c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:25:42.104 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:42.101+0000 7f68c4c00640 1 --2- 172.21.15.47:0/3157620056 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f644bc9de0 0x55f644bc3c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:25:42.104 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:42.101+0000 7f68c4c00640 1 -- 172.21.15.47:0/3157620056 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f644bbaab0 con 0x55f644bc1260 2024-03-21T10:25:42.104 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:42.101+0000 7f68c6000640 1 --2- 172.21.15.47:0/3157620056 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f644bc9de0 0x55f644bc3c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T10:25:42.104 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:42.101+0000 7f68c4c00640 1 --2- 172.21.15.47:0/3157620056 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f644bc1260 0x55f644bc9630 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f68ac09b4a0 tx=0x7f68ac0cfc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=34daf5abd302c13c server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:25:42.107 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:42.101+0000 7f68b1c00640 1 -- 172.21.15.47:0/3157620056 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68ac020030 con 0x55f644bc1260 2024-03-21T10:25:42.107 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:42.101+0000 7f68b1c00640 1 -- 172.21.15.47:0/3157620056 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f68ac09f8f0 con 0x55f644bc1260 2024-03-21T10:25:42.107 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:42.102+0000 7f68b1c00640 1 -- 172.21.15.47:0/3157620056 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68ac010460 con 0x55f644bc1260 2024-03-21T10:25:42.107 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:42.102+0000 7f68cabe0740 1 -- 172.21.15.47:0/3157620056 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f644bc1260 msgr2=0x55f644bc9630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:25:42.107 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:42.102+0000 7f68cabe0740 1 --2- 172.21.15.47:0/3157620056 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f644bc1260 0x55f644bc9630 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f68ac09b4a0 tx=0x7f68ac0cfc70 comp rx=0 tx=0).stop 2024-03-21T10:25:42.108 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:42.102+0000 7f68cabe0740 1 -- 172.21.15.47:0/3157620056 shutdown_connections 2024-03-21T10:25:42.108 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:42.102+0000 7f68cabe0740 1 --2- 172.21.15.47:0/3157620056 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f644bc9de0 0x55f644bc3c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:25:42.108 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:42.102+0000 7f68cabe0740 1 --2- 172.21.15.47:0/3157620056 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f644bc1260 0x55f644bc9630 unknown :-1 s=CLOSED pgs=557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:25:42.108 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:42.102+0000 7f68cabe0740 1 --2- 172.21.15.47:0/3157620056 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f644bc08b0 0x55f644bc90f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:25:42.108 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:42.102+0000 7f68cabe0740 1 -- 172.21.15.47:0/3157620056 >> 172.21.15.47:0/3157620056 conn(0x55f644bbaf80 msgr2=0x55f644bbcc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:25:42.108 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:42.102+0000 7f68cabe0740 1 -- 172.21.15.47:0/3157620056 shutdown_connections 2024-03-21T10:25:42.108 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:42.102+0000 7f68cabe0740 1 -- 172.21.15.47:0/3157620056 wait complete. 2024-03-21T10:25:42.108 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:42.103+0000 7f68cabe0740 1 -- start start 2024-03-21T10:25:42.108 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:42.103+0000 7f68cabe0740 1 Processor -- start 2024-03-21T10:25:42.108 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:42.104+0000 7f68cabe0740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f644bc08b0 0x55f644bd1b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:25:42.108 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:42.104+0000 7f68cabe0740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f644bc1260 0x55f644bd2050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:25:42.108 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:42.104+0000 7f68cabe0740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f644bc9de0 0x55f644bce0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:25:42.108 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:42.104+0000 7f68cabe0740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68c01b1b90 con 0x55f644bc9de0 2024-03-21T10:25:42.108 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:42.104+0000 7f68cabe0740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68c01b1530 con 0x55f644bc08b0 2024-03-21T10:25:42.109 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:42.104+0000 7f68cabe0740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68c01aea30 con 0x55f644bc1260 2024-03-21T10:25:42.109 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:42.104+0000 7f68cabe0740 10 client.0 _ll_register_callbacks cb 0x7f68ac0cca40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T10:25:42.109 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:42.104+0000 7f68c6000640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f644bc9de0 0x55f644bce0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:25:42.109 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:42.104+0000 7f68c6000640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f644bc9de0 0x55f644bce0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:55004/0 (socket says 172.21.15.47:55004) 2024-03-21T10:25:42.109 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:42.104+0000 7f68c6000640 1 -- 172.21.15.47:0/1109310375 learned_addr learned my addr 172.21.15.47:0/1109310375 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:25:42.109 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:42.104+0000 7f68c5600640 1 --2- 172.21.15.47:0/1109310375 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f644bc08b0 0x55f644bd1b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:25:42.109 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:42.104+0000 7f68c4c00640 1 --2- 172.21.15.47:0/1109310375 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f644bc1260 0x55f644bd2050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:25:42.109 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:42.105+0000 7f68c6000640 1 -- 172.21.15.47:0/1109310375 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f644bc1260 msgr2=0x55f644bd2050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:25:42.109 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:42.105+0000 7f68c6000640 1 --2- 172.21.15.47:0/1109310375 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f644bc1260 0x55f644bd2050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:25:42.109 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:42.105+0000 7f68c6000640 1 -- 172.21.15.47:0/1109310375 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f644bc08b0 msgr2=0x55f644bd1b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:25:42.109 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:42.105+0000 7f68c6000640 1 --2- 172.21.15.47:0/1109310375 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f644bc08b0 0x55f644bd1b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:25:42.109 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:42.105+0000 7f68c6000640 1 -- 172.21.15.47:0/1109310375 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55f644bce8e0 con 0x55f644bc9de0 2024-03-21T10:25:42.109 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:42.105+0000 7f68c6000640 1 --2- 172.21.15.47:0/1109310375 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f644bc9de0 0x55f644bce0e0 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f68c0102490 tx=0x7f68b802c0f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=717a83a0fec72c2f server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:25:42.109 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:42.105+0000 7f68a6a00640 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-03-21T10:25:42.110 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:42.105+0000 7f68a6a00640 1 -- 172.21.15.47:0/1109310375 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68b8051030 con 0x55f644bc9de0 2024-03-21T10:25:42.110 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:42.105+0000 7f68cabe0740 1 -- 172.21.15.47:0/1109310375 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f68ac09f8f0 con 0x55f644bc9de0 2024-03-21T10:25:42.110 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:42.105+0000 7f68a6a00640 1 -- 172.21.15.47:0/1109310375 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 9474+0+0 (secure 0 0 0) 0x7f68b804d480 con 0x55f644bc9de0 2024-03-21T10:25:42.110 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:42.106+0000 7f68a6a00640 1 -- 172.21.15.47:0/1109310375 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f68b8039040 con 0x55f644bc9de0 2024-03-21T10:25:42.110 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:42.106+0000 7f68cabe0740 10 client.15879 fetch_fsmap learned FSMap version 30 2024-03-21T10:25:42.110 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:42.106+0000 7f68cabe0740 1 -- 172.21.15.47:0/1109310375 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x55f644c3e8f0 con 0x55f644bc9de0 2024-03-21T10:25:42.112 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:42.106+0000 7f68a6a00640 1 -- 172.21.15.47:0/1109310375 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f68ac09f8f0 con 0x55f644bc9de0 2024-03-21T10:25:42.112 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:42.106+0000 7f68a6a00640 1 -- 172.21.15.47:0/1109310375 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 30) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f68b804a050 con 0x55f644bc9de0 2024-03-21T10:25:42.112 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:42.106+0000 7f68cabe0740 10 client.15879 fetch_fsmap finished waiting for FSMap version 30 2024-03-21T10:25:42.112 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:42.106+0000 7f68cabe0740 10 client.15879 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-03-21T10:25:42.112 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:42.106+0000 7f68cabe0740 20 client.15879 populate_metadata read hostname 'smithi047' 2024-03-21T10:25:42.112 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:42.106+0000 7f68cabe0740 1 --2- 172.21.15.47:0/1109310375 >> [v2:172.21.15.100:6836/3497644162,v1:172.21.15.100:6837/3497644162] conn(0x55f644c3f020 0x55f644c41440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:25:42.112 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:42.106+0000 7f68cabe0740 4 client.15879 mds_command: new command op to 34493 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-21T10:25:42.112 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:42.106+0000 7f68cabe0740 1 -- 172.21.15.47:0/1109310375 --> [v2:172.21.15.100:6836/3497644162,v1:172.21.15.100:6837/3497644162] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55f644b2bb50 con 0x55f644c3f020 2024-03-21T10:25:42.112 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:42.107+0000 7f68c5600640 1 --2- 172.21.15.47:0/1109310375 >> [v2:172.21.15.100:6836/3497644162,v1:172.21.15.100:6837/3497644162] conn(0x55f644c3f020 0x55f644c41440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:25:42.112 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:42.108+0000 7f68c5600640 1 --2- 172.21.15.47:0/1109310375 >> [v2:172.21.15.100:6836/3497644162,v1:172.21.15.100:6837/3497644162] conn(0x55f644c3f020 0x55f644c41440 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=90f00dd3bc79db7f server_cookie=4c65e453b94c0f99 in_seq=0 out_seq=0 2024-03-21T10:25:42.112 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:42.108+0000 7f68a6a00640 10 client.15879 ms_handle_connect on v2:172.21.15.100:6836/3497644162 2024-03-21T10:25:42.112 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:42.108+0000 7f68a6a00640 1 -- 172.21.15.47:0/1109310375 <== mds.1 v2:172.21.15.100:6836/3497644162 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55f644b2bb50 con 0x55f644c3f020 2024-03-21T10:25:42.113 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:42.108+0000 7f68a6a00640 10 client.15879 handle_command_reply: tid=0 2024-03-21T10:25:42.113 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:42.108+0000 7f68cabe0740 1 client.15879 shutdown 2024-03-21T10:25:42.113 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:42.108+0000 7f68cabe0740 1 -- 172.21.15.47:0/1109310375 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f644bc9de0 msgr2=0x55f644bce0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:25:42.113 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:42.108+0000 7f68cabe0740 1 --2- 172.21.15.47:0/1109310375 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f644bc9de0 0x55f644bce0e0 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f68c0102490 tx=0x7f68b802c0f0 comp rx=0 tx=0).stop 2024-03-21T10:25:42.113 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:42.108+0000 7f68cabe0740 1 -- 172.21.15.47:0/1109310375 shutdown_connections 2024-03-21T10:25:42.113 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:42.108+0000 7f68cabe0740 1 --2- 172.21.15.47:0/1109310375 >> [v2:172.21.15.100:6836/3497644162,v1:172.21.15.100:6837/3497644162] conn(0x55f644c3f020 0x55f644c41440 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:25:42.113 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:42.108+0000 7f68cabe0740 1 --2- 172.21.15.47:0/1109310375 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f644bc9de0 0x55f644bce0e0 unknown :-1 s=CLOSED pgs=558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:25:42.113 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:42.108+0000 7f68cabe0740 1 --2- 172.21.15.47:0/1109310375 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f644bc1260 0x55f644bd2050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:25:42.113 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:42.108+0000 7f68cabe0740 1 --2- 172.21.15.47:0/1109310375 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f644bc08b0 0x55f644bd1b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:25:42.113 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:42.108+0000 7f68a6a00640 0 client.15879 ms_handle_reset on v2:172.21.15.100:6836/3497644162 2024-03-21T10:25:42.113 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:42.108+0000 7f68cabe0740 1 -- 172.21.15.47:0/1109310375 >> 172.21.15.47:0/1109310375 conn(0x55f644bbaf80 msgr2=0x55f644bbc380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:25:42.113 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:42.108+0000 7f68cabe0740 1 -- 172.21.15.47:0/1109310375 shutdown_connections 2024-03-21T10:25:42.113 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:42.108+0000 7f68cabe0740 1 -- 172.21.15.47:0/1109310375 wait complete. 2024-03-21T10:25:42.113 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:42.109+0000 7f68cabe0740 20 client.15879 trim_cache size 0 max 16384 2024-03-21T10:25:42.114 INFO:teuthology.orchestra.run.smithi047.stdout:[] 2024-03-21T10:25:42.114 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:42.109+0000 7f68c6a00640 1 -- 172.21.15.47:0/433304381 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f68a80a3120 msgr2=0x7f68a80a55e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:25:42.114 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:42.109+0000 7f68c6a00640 1 --2- 172.21.15.47:0/433304381 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f68a80a3120 0x7f68a80a55e0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f68b8006010 tx=0x7f68b8005f80 comp rx=0 tx=0).stop 2024-03-21T10:25:42.115 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:42.109+0000 7f68c6a00640 1 -- 172.21.15.47:0/433304381 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f68c0100c60 msgr2=0x7f68c019a440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:25:42.115 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:42.109+0000 7f68c6a00640 1 --2- 172.21.15.47:0/433304381 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f68c0100c60 0x7f68c019a440 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f68ac00d950 tx=0x7f68ac00de10 comp rx=0 tx=0).stop 2024-03-21T10:25:42.115 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:42.109+0000 7f68c6a00640 1 -- 172.21.15.47:0/433304381 shutdown_connections 2024-03-21T10:25:42.115 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:42.109+0000 7f68c6a00640 1 --2- 172.21.15.47:0/433304381 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f68c010de60 0x7f68c019e7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:25:42.115 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:42.109+0000 7f68c6a00640 1 --2- 172.21.15.47:0/433304381 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f68c0100c60 0x7f68c019a440 unknown :-1 s=CLOSED pgs=555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:25:42.115 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:42.109+0000 7f68c6a00640 1 --2- 172.21.15.47:0/433304381 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f68a80a3120 0x7f68a80a55e0 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:25:42.115 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:42.109+0000 7f68c6a00640 1 --2- 172.21.15.47:0/433304381 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f68c01002b0 0x7f68c0199f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:25:42.115 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:42.109+0000 7f68c6a00640 1 -- 172.21.15.47:0/433304381 >> 172.21.15.47:0/433304381 conn(0x7f68c00fbef0 msgr2=0x7f68c00fe2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:25:42.115 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:42.109+0000 7f68c6a00640 1 -- 172.21.15.47:0/433304381 shutdown_connections 2024-03-21T10:25:42.115 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:42.110+0000 7f68c6a00640 1 -- 172.21.15.47:0/433304381 wait complete. 2024-03-21T10:25:42.130 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-21T10:25:42.130 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-03-21T10:25:42.352 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:42.345+0000 7f08d3800640 1 -- 172.21.15.47:0/858354759 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f08cc1004d0 msgr2=0x7f08cc1008b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:25:42.352 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:42.345+0000 7f08d3800640 1 --2- 172.21.15.47:0/858354759 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f08cc1004d0 0x7f08cc1008b0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f08c0009900 tx=0x7f08c002bc30 comp rx=0 tx=0).stop 2024-03-21T10:25:42.352 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:42.345+0000 7f08d3800640 1 -- 172.21.15.47:0/858354759 shutdown_connections 2024-03-21T10:25:42.352 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:42.345+0000 7f08d3800640 1 --2- 172.21.15.47:0/858354759 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f08cc10e080 0x7f08cc110470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:25:42.352 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:42.345+0000 7f08d3800640 1 --2- 172.21.15.47:0/858354759 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f08cc100df0 0x7f08cc10db40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:25:42.352 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:42.345+0000 7f08d3800640 1 --2- 172.21.15.47:0/858354759 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f08cc1004d0 0x7f08cc1008b0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:25:42.353 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:42.345+0000 7f08d3800640 1 -- 172.21.15.47:0/858354759 >> 172.21.15.47:0/858354759 conn(0x7f08cc077700 msgr2=0x7f08cc077b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:25:42.353 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:42.345+0000 7f08d3800640 1 -- 172.21.15.47:0/858354759 shutdown_connections 2024-03-21T10:25:42.353 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:42.345+0000 7f08d3800640 1 -- 172.21.15.47:0/858354759 wait complete. 2024-03-21T10:25:42.353 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:42.346+0000 7f08d3800640 1 Processor -- start 2024-03-21T10:25:42.353 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:42.347+0000 7f08d3800640 1 -- start start 2024-03-21T10:25:42.353 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:42.347+0000 7f08d3800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f08cc1004d0 0x7f08cc19a100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:25:42.353 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:42.347+0000 7f08d3800640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f08cc100df0 0x7f08cc19a640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:25:42.353 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:42.347+0000 7f08d3800640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f08cc10e080 0x7f08cc19e970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:25:42.353 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:42.347+0000 7f08d3800640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08cc113b60 con 0x7f08cc10e080 2024-03-21T10:25:42.353 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:42.347+0000 7f08d3800640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08cc1139e0 con 0x7f08cc1004d0 2024-03-21T10:25:42.353 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:42.347+0000 7f08d3800640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08cc113ce0 con 0x7f08cc100df0 2024-03-21T10:25:42.353 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:42.347+0000 7f08d1800640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f08cc10e080 0x7f08cc19e970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:25:42.353 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:42.347+0000 7f08d1800640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f08cc10e080 0x7f08cc19e970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:55024/0 (socket says 172.21.15.47:55024) 2024-03-21T10:25:42.353 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:42.347+0000 7f08d1800640 1 -- 172.21.15.47:0/974546312 learned_addr learned my addr 172.21.15.47:0/974546312 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:25:42.354 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:42.347+0000 7f08d0e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f08cc1004d0 0x7f08cc19a100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:25:42.354 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:42.347+0000 7f08cbe00640 1 --2- 172.21.15.47:0/974546312 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f08cc100df0 0x7f08cc19a640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:25:42.354 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:42.348+0000 7f08d1800640 1 -- 172.21.15.47:0/974546312 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f08cc100df0 msgr2=0x7f08cc19a640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:25:42.354 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:42.348+0000 7f08d1800640 1 --2- 172.21.15.47:0/974546312 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f08cc100df0 0x7f08cc19a640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:25:42.354 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:42.348+0000 7f08d1800640 1 -- 172.21.15.47:0/974546312 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f08cc1004d0 msgr2=0x7f08cc19a100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:25:42.354 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:42.348+0000 7f08d1800640 1 --2- 172.21.15.47:0/974546312 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f08cc1004d0 0x7f08cc19a100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:25:42.354 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:42.348+0000 7f08d1800640 1 -- 172.21.15.47:0/974546312 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08cc19f0a0 con 0x7f08cc10e080 2024-03-21T10:25:42.354 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:42.348+0000 7f08d1800640 1 --2- 172.21.15.47:0/974546312 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f08cc10e080 0x7f08cc19e970 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f08c400eb40 tx=0x7f08c400c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:25:42.354 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:42.348+0000 7f08c9600640 1 -- 172.21.15.47:0/974546312 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08c4004b20 con 0x7f08cc10e080 2024-03-21T10:25:42.354 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:42.348+0000 7f08c9600640 1 -- 172.21.15.47:0/974546312 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f08c4004cc0 con 0x7f08cc10e080 2024-03-21T10:25:42.354 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:42.349+0000 7f08d3800640 1 -- 172.21.15.47:0/974546312 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f08cc19f390 con 0x7f08cc10e080 2024-03-21T10:25:42.354 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:42.349+0000 7f08c9600640 1 -- 172.21.15.47:0/974546312 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08c40107b0 con 0x7f08cc10e080 2024-03-21T10:25:42.354 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:42.349+0000 7f08d3800640 1 -- 172.21.15.47:0/974546312 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f08cc1a6c20 con 0x7f08cc10e080 2024-03-21T10:25:42.357 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:42.352+0000 7f08d65dd740 1 Processor -- start 2024-03-21T10:25:42.357 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:42.352+0000 7f08c9600640 1 -- 172.21.15.47:0/974546312 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 25) v1 ==== 138030+0+0 (secure 0 0 0) 0x7f08c4005020 con 0x7f08cc10e080 2024-03-21T10:25:42.357 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:42.352+0000 7f08d65dd740 1 -- start start 2024-03-21T10:25:42.358 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:42.353+0000 7f08d65dd740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557448b6b6d0 0x557448b6bab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:25:42.358 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:42.353+0000 7f08d65dd740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x557448b6c080 0x557448b74630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:25:42.358 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:42.353+0000 7f08d65dd740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557448b74c00 0x557448b77050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:25:42.358 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:42.353+0000 7f08d65dd740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x557448acf2c0 con 0x557448b6c080 2024-03-21T10:25:42.358 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:42.353+0000 7f08d65dd740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x557448a77c90 con 0x557448b74c00 2024-03-21T10:25:42.358 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:42.353+0000 7f08d65dd740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x557448acb360 con 0x557448b6b6d0 2024-03-21T10:25:42.358 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:42.353+0000 7f08d0e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x557448b6c080 0x557448b74630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:25:42.358 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:42.353+0000 7f08d0e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x557448b6c080 0x557448b74630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:55032/0 (socket says 172.21.15.47:55032) 2024-03-21T10:25:42.358 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:42.353+0000 7f08d0e00640 1 -- 172.21.15.47:0/2689924456 learned_addr learned my addr 172.21.15.47:0/2689924456 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:25:42.358 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:42.353+0000 7f08cbe00640 1 --2- 172.21.15.47:0/2689924456 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557448b6b6d0 0x557448b6bab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:25:42.358 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:42.353+0000 7f08cbe00640 1 --2- 172.21.15.47:0/2689924456 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557448b74c00 0x557448b77050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:25:42.358 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:42.354+0000 7f08d0e00640 1 -- 172.21.15.47:0/2689924456 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557448b6b6d0 msgr2=0x557448b6bab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:25:42.359 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:42.354+0000 7f08d0e00640 1 --2- 172.21.15.47:0/2689924456 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557448b6b6d0 0x557448b6bab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:25:42.359 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:42.354+0000 7f08d0e00640 1 -- 172.21.15.47:0/2689924456 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557448b74c00 msgr2=0x557448b77050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:25:42.359 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:42.354+0000 7f08d0e00640 1 --2- 172.21.15.47:0/2689924456 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557448b74c00 0x557448b77050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:25:42.359 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:42.354+0000 7f08d0e00640 1 -- 172.21.15.47:0/2689924456 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557448a9c080 con 0x557448b6c080 2024-03-21T10:25:42.359 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:42.354+0000 7f08cbe00640 1 --2- 172.21.15.47:0/2689924456 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557448b74c00 0x557448b77050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T10:25:42.359 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:42.354+0000 7f08c9600640 1 --2- 172.21.15.47:0/974546312 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f08b40a31a0 0x7f08b40a5660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:25:42.359 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:42.354+0000 7f08d0e00640 1 --2- 172.21.15.47:0/2689924456 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x557448b6c080 0x557448b74630 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f08c0009900 tx=0x7f08c00095f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1bb7fcb4297fc7c3 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:25:42.359 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:42.354+0000 7f08c9600640 1 -- 172.21.15.47:0/974546312 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 9474+0+0 (secure 0 0 0) 0x7f08c40d0430 con 0x7f08cc10e080 2024-03-21T10:25:42.359 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:42.355+0000 7f08cbe00640 1 --2- 172.21.15.47:0/974546312 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f08b40a31a0 0x7f08b40a5660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:25:42.359 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:42.355+0000 7f08be000640 1 -- 172.21.15.47:0/2689924456 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08c0045020 con 0x557448b6c080 2024-03-21T10:25:42.359 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:42.355+0000 7f08be000640 1 -- 172.21.15.47:0/2689924456 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f08c0004040 con 0x557448b6c080 2024-03-21T10:25:42.359 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:42.355+0000 7f08be000640 1 -- 172.21.15.47:0/2689924456 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08c0005570 con 0x557448b6c080 2024-03-21T10:25:42.359 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:42.356+0000 7f08d65dd740 1 -- 172.21.15.47:0/2689924456 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x557448b6c080 msgr2=0x557448b74630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:25:42.360 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:42.356+0000 7f08d65dd740 1 --2- 172.21.15.47:0/2689924456 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x557448b6c080 0x557448b74630 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f08c0009900 tx=0x7f08c00095f0 comp rx=0 tx=0).stop 2024-03-21T10:25:42.360 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:42.356+0000 7f08cbe00640 1 --2- 172.21.15.47:0/974546312 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f08b40a31a0 0x7f08b40a5660 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f08cc100030 tx=0x7f08b8013040 comp rx=0 tx=0).ready entity=mgr.15687 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:25:42.360 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:42.356+0000 7f08d65dd740 1 -- 172.21.15.47:0/2689924456 shutdown_connections 2024-03-21T10:25:42.360 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:42.356+0000 7f08d65dd740 1 --2- 172.21.15.47:0/2689924456 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557448b74c00 0x557448b77050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:25:42.360 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:42.356+0000 7f08d65dd740 1 --2- 172.21.15.47:0/2689924456 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x557448b6c080 0x557448b74630 unknown :-1 s=CLOSED pgs=560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:25:42.360 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:42.356+0000 7f08d65dd740 1 --2- 172.21.15.47:0/2689924456 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557448b6b6d0 0x557448b6bab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:25:42.360 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:42.356+0000 7f08d65dd740 1 -- 172.21.15.47:0/2689924456 >> 172.21.15.47:0/2689924456 conn(0x557448b65da0 msgr2=0x557448b66180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:25:42.360 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:42.356+0000 7f08d65dd740 1 -- 172.21.15.47:0/2689924456 shutdown_connections 2024-03-21T10:25:42.360 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:42.356+0000 7f08d65dd740 1 -- 172.21.15.47:0/2689924456 wait complete. 2024-03-21T10:25:42.363 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:42.356+0000 7f08d65dd740 1 -- start start 2024-03-21T10:25:42.363 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:42.357+0000 7f08d65dd740 1 Processor -- start 2024-03-21T10:25:42.363 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:42.357+0000 7f08d65dd740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557448b6b6d0 0x557448b74470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:25:42.364 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:42.357+0000 7f08d65dd740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x557448b6c080 0x557448b71800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:25:42.364 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:42.357+0000 7f08d65dd740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557448b74c00 0x557448b71d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:25:42.364 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:42.357+0000 7f08d65dd740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x557448b77b10 con 0x557448b6c080 2024-03-21T10:25:42.364 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:42.357+0000 7f08d65dd740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x557448a8b0d0 con 0x557448b6b6d0 2024-03-21T10:25:42.364 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:42.357+0000 7f08d65dd740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x557448b77c90 con 0x557448b74c00 2024-03-21T10:25:42.364 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:42.357+0000 7f08d65dd740 10 client.0 _ll_register_callbacks cb 0x557448a8fe80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T10:25:42.364 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:42.357+0000 7f08cbe00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x557448b6c080 0x557448b71800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:25:42.364 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:42.358+0000 7f08cbe00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x557448b6c080 0x557448b71800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:55044/0 (socket says 172.21.15.47:55044) 2024-03-21T10:25:42.364 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:42.358+0000 7f08cbe00640 1 -- 172.21.15.47:0/4272125624 learned_addr learned my addr 172.21.15.47:0/4272125624 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:25:42.364 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:42.358+0000 7f08d0e00640 1 --2- 172.21.15.47:0/4272125624 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557448b6b6d0 0x557448b74470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:25:42.364 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:42.358+0000 7f08d1800640 1 --2- 172.21.15.47:0/4272125624 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557448b74c00 0x557448b71d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:25:42.364 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:42.358+0000 7f08cbe00640 1 -- 172.21.15.47:0/4272125624 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557448b74c00 msgr2=0x557448b71d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:25:42.364 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:42.358+0000 7f08cbe00640 1 --2- 172.21.15.47:0/4272125624 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557448b74c00 0x557448b71d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:25:42.364 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:42.358+0000 7f08cbe00640 1 -- 172.21.15.47:0/4272125624 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557448b6b6d0 msgr2=0x557448b74470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:25:42.365 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:42.358+0000 7f08cbe00640 1 --2- 172.21.15.47:0/4272125624 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557448b6b6d0 0x557448b74470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:25:42.365 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:42.358+0000 7f08cbe00640 1 -- 172.21.15.47:0/4272125624 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5574487e33f0 con 0x557448b6c080 2024-03-21T10:25:42.365 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:42.358+0000 7f08d0e00640 1 --2- 172.21.15.47:0/4272125624 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557448b6b6d0 0x557448b74470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T10:25:42.365 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:42.358+0000 7f08cbe00640 1 --2- 172.21.15.47:0/4272125624 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x557448b6c080 0x557448b71800 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f08b800fbd0 tx=0x7f08b80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=44f2f033bf2516b5 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:25:42.365 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:42.359+0000 7f08b2a00640 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-03-21T10:25:42.365 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:42.359+0000 7f08b2a00640 1 -- 172.21.15.47:0/4272125624 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08b8021bf0 con 0x557448b6c080 2024-03-21T10:25:42.365 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:42.359+0000 7f08b2a00640 1 -- 172.21.15.47:0/4272125624 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 9474+0+0 (secure 0 0 0) 0x7f08b8021d90 con 0x557448b6c080 2024-03-21T10:25:42.365 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:42.359+0000 7f08d65dd740 1 -- 172.21.15.47:0/4272125624 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f08c0004040 con 0x557448b6c080 2024-03-21T10:25:42.365 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:42.359+0000 7f08b2a00640 1 -- 172.21.15.47:0/4272125624 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f08b8036050 con 0x557448b6c080 2024-03-21T10:25:42.365 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:42.359+0000 7f08d65dd740 10 client.15891 fetch_fsmap learned FSMap version 30 2024-03-21T10:25:42.365 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:42.359+0000 7f08d65dd740 1 -- 172.21.15.47:0/4272125624 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x5574488ad7d0 con 0x557448b6c080 2024-03-21T10:25:42.365 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:42.360+0000 7f08b2a00640 1 -- 172.21.15.47:0/4272125624 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f08c0004040 con 0x557448b6c080 2024-03-21T10:25:42.365 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:42.360+0000 7f08b2a00640 1 -- 172.21.15.47:0/4272125624 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 30) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f08b802b5b0 con 0x557448b6c080 2024-03-21T10:25:42.366 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:42.360+0000 7f08d65dd740 10 client.15891 fetch_fsmap finished waiting for FSMap version 30 2024-03-21T10:25:42.366 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:42.360+0000 7f08d65dd740 10 client.15891 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-21T10:25:42.366 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:42.360+0000 7f08d65dd740 20 client.15891 populate_metadata read hostname 'smithi047' 2024-03-21T10:25:42.366 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:42.360+0000 7f08d65dd740 1 --2- 172.21.15.47:0/4272125624 >> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] conn(0x557448b84a10 0x557448b86e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:25:42.366 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:42.360+0000 7f08d65dd740 4 client.15891 mds_command: new command op to 34535 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T10:25:42.366 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:42.360+0000 7f08d65dd740 1 -- 172.21.15.47:0/4272125624 --> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x557448adc800 con 0x557448b84a10 2024-03-21T10:25:42.366 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:42.361+0000 7f08d0e00640 1 --2- 172.21.15.47:0/4272125624 >> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] conn(0x557448b84a10 0x557448b86e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:25:42.366 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:42.362+0000 7f08d0e00640 1 --2- 172.21.15.47:0/4272125624 >> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] conn(0x557448b84a10 0x557448b86e90 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b7139568f120b455 server_cookie=67f9e44cee111828 in_seq=0 out_seq=0 2024-03-21T10:25:42.366 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:42.362+0000 7f08b2a00640 10 client.15891 ms_handle_connect on v2:172.21.15.100:6838/4173168440 2024-03-21T10:25:42.367 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:42.363+0000 7f08b2a00640 1 -- 172.21.15.47:0/4272125624 <== mds.0 v2:172.21.15.100:6838/4173168440 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x557448adc800 con 0x557448b84a10 2024-03-21T10:25:42.367 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:42.363+0000 7f08b2a00640 10 client.15891 handle_command_reply: tid=0 2024-03-21T10:25:42.367 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:42.363+0000 7f08d65dd740 1 client.15891 shutdown 2024-03-21T10:25:42.368 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:42.363+0000 7f08d65dd740 1 -- 172.21.15.47:0/4272125624 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x557448b6c080 msgr2=0x557448b71800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:25:42.368 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:42.363+0000 7f08d65dd740 1 --2- 172.21.15.47:0/4272125624 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x557448b6c080 0x557448b71800 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f08b800fbd0 tx=0x7f08b80046c0 comp rx=0 tx=0).stop 2024-03-21T10:25:42.368 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:42.363+0000 7f08d65dd740 1 -- 172.21.15.47:0/4272125624 shutdown_connections 2024-03-21T10:25:42.368 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:42.363+0000 7f08d65dd740 1 --2- 172.21.15.47:0/4272125624 >> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] conn(0x557448b84a10 0x557448b86e90 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:25:42.368 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:42.363+0000 7f08d65dd740 1 --2- 172.21.15.47:0/4272125624 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557448b74c00 0x557448b71d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:25:42.368 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:42.363+0000 7f08d65dd740 1 --2- 172.21.15.47:0/4272125624 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x557448b6c080 0x557448b71800 unknown :-1 s=CLOSED pgs=561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:25:42.368 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:42.363+0000 7f08d65dd740 1 --2- 172.21.15.47:0/4272125624 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557448b6b6d0 0x557448b74470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:25:42.368 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:42.363+0000 7f08b2a00640 0 client.15891 ms_handle_reset on v2:172.21.15.100:6838/4173168440 2024-03-21T10:25:42.368 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:42.363+0000 7f08d65dd740 1 -- 172.21.15.47:0/4272125624 >> 172.21.15.47:0/4272125624 conn(0x557448b65da0 msgr2=0x557448b67af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:25:42.368 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:42.363+0000 7f08d65dd740 1 -- 172.21.15.47:0/4272125624 shutdown_connections 2024-03-21T10:25:42.368 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:42.363+0000 7f08d65dd740 1 -- 172.21.15.47:0/4272125624 wait complete. 2024-03-21T10:25:42.368 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:42.363+0000 7f08d65dd740 20 client.15891 trim_cache size 0 max 16384 2024-03-21T10:25:42.379 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:42.377+0000 7f08d65dd740 1 Processor -- start 2024-03-21T10:25:42.379 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:42.377+0000 7f08d65dd740 1 -- start start 2024-03-21T10:25:42.379 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:42.377+0000 7f08d65dd740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557448b6b6d0 0x557448b73f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:25:42.379 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:42.377+0000 7f08d65dd740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557448b6c080 0x557448b74470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:25:42.379 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:42.377+0000 7f08d65dd740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x557448b74c00 0x557448b6ea60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:25:42.379 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:42.377+0000 7f08d65dd740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08cc1b0a10 con 0x557448b74c00 2024-03-21T10:25:42.379 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:42.377+0000 7f08d65dd740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08cc1b2390 con 0x557448b6c080 2024-03-21T10:25:42.379 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:42.377+0000 7f08d65dd740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08cc1b1070 con 0x557448b6b6d0 2024-03-21T10:25:42.380 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:42.378+0000 7f08d1800640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x557448b74c00 0x557448b6ea60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:25:42.380 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:42.378+0000 7f08d1800640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x557448b74c00 0x557448b6ea60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:55046/0 (socket says 172.21.15.47:55046) 2024-03-21T10:25:42.380 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:42.378+0000 7f08d1800640 1 -- 172.21.15.47:0/1101473376 learned_addr learned my addr 172.21.15.47:0/1101473376 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:25:42.381 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:42.378+0000 7f08cbe00640 1 --2- 172.21.15.47:0/1101473376 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557448b6c080 0x557448b74470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:25:42.382 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:42.378+0000 7f08d0e00640 1 --2- 172.21.15.47:0/1101473376 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557448b6b6d0 0x557448b73f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:25:42.382 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:42.378+0000 7f08d1800640 1 -- 172.21.15.47:0/1101473376 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557448b6b6d0 msgr2=0x557448b73f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:25:42.382 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:42.378+0000 7f08d1800640 1 --2- 172.21.15.47:0/1101473376 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557448b6b6d0 0x557448b73f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:25:42.382 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:42.378+0000 7f08d1800640 1 -- 172.21.15.47:0/1101473376 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557448b6c080 msgr2=0x557448b74470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:25:42.382 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:42.378+0000 7f08d1800640 1 --2- 172.21.15.47:0/1101473376 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557448b6c080 0x557448b74470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:25:42.382 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:42.378+0000 7f08d1800640 1 -- 172.21.15.47:0/1101473376 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557448b61700 con 0x557448b74c00 2024-03-21T10:25:42.382 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:42.378+0000 7f08cbe00640 1 --2- 172.21.15.47:0/1101473376 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557448b6c080 0x557448b74470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T10:25:42.382 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:42.378+0000 7f08d1800640 1 --2- 172.21.15.47:0/1101473376 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x557448b74c00 0x557448b6ea60 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f08c40986f0 tx=0x7f08c409be90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=78c63268551a606 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:25:42.382 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:42.379+0000 7f08bd600640 1 -- 172.21.15.47:0/1101473376 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08c40b4070 con 0x557448b74c00 2024-03-21T10:25:42.382 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:42.379+0000 7f08bd600640 1 -- 172.21.15.47:0/1101473376 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f08c409c290 con 0x557448b74c00 2024-03-21T10:25:42.382 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:42.379+0000 7f08bd600640 1 -- 172.21.15.47:0/1101473376 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08c409a620 con 0x557448b74c00 2024-03-21T10:25:42.383 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:42.379+0000 7f08d65dd740 1 -- 172.21.15.47:0/1101473376 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x557448b74c00 msgr2=0x557448b6ea60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:25:42.383 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:42.379+0000 7f08d65dd740 1 --2- 172.21.15.47:0/1101473376 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x557448b74c00 0x557448b6ea60 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f08c40986f0 tx=0x7f08c409be90 comp rx=0 tx=0).stop 2024-03-21T10:25:42.383 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:42.379+0000 7f08d65dd740 1 -- 172.21.15.47:0/1101473376 shutdown_connections 2024-03-21T10:25:42.383 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:42.379+0000 7f08d65dd740 1 --2- 172.21.15.47:0/1101473376 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x557448b74c00 0x557448b6ea60 unknown :-1 s=CLOSED pgs=562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:25:42.383 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:42.379+0000 7f08d65dd740 1 --2- 172.21.15.47:0/1101473376 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557448b6c080 0x557448b74470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:25:42.383 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:42.379+0000 7f08d65dd740 1 --2- 172.21.15.47:0/1101473376 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557448b6b6d0 0x557448b73f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:25:42.383 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:42.380+0000 7f08d65dd740 1 -- 172.21.15.47:0/1101473376 >> 172.21.15.47:0/1101473376 conn(0x557448b65da0 msgr2=0x557448b6cd40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:25:42.383 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:42.380+0000 7f08d65dd740 1 -- 172.21.15.47:0/1101473376 shutdown_connections 2024-03-21T10:25:42.383 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:42.380+0000 7f08d65dd740 1 -- 172.21.15.47:0/1101473376 wait complete. 2024-03-21T10:25:42.383 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:42.380+0000 7f08d65dd740 1 -- start start 2024-03-21T10:25:42.383 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:42.380+0000 7f08d65dd740 1 Processor -- start 2024-03-21T10:25:42.385 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:42.381+0000 7f08d65dd740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557448b6b6d0 0x557448b78a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:25:42.386 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:42.381+0000 7f08d65dd740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x557448b6c080 0x557448b78fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:25:42.386 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:42.381+0000 7f08d65dd740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557448b74c00 0x557448bdf320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:25:42.386 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:42.381+0000 7f08d65dd740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08cc1b1d30 con 0x557448b6c080 2024-03-21T10:25:42.386 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:42.381+0000 7f08d65dd740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08cc1b16d0 con 0x557448b74c00 2024-03-21T10:25:42.386 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:42.381+0000 7f08d65dd740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08cc1aebd0 con 0x557448b6b6d0 2024-03-21T10:25:42.386 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:42.381+0000 7f08d65dd740 10 client.0 _ll_register_callbacks cb 0x7f08b8009ee0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T10:25:42.386 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:42.381+0000 7f08cbe00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x557448b6c080 0x557448b78fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:25:42.386 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:42.381+0000 7f08cbe00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x557448b6c080 0x557448b78fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:55058/0 (socket says 172.21.15.47:55058) 2024-03-21T10:25:42.386 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:42.381+0000 7f08cbe00640 1 -- 172.21.15.47:0/1921815357 learned_addr learned my addr 172.21.15.47:0/1921815357 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:25:42.386 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:42.381+0000 7f08d0e00640 1 --2- 172.21.15.47:0/1921815357 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557448b6b6d0 0x557448b78a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:25:42.386 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:42.381+0000 7f08d1800640 1 --2- 172.21.15.47:0/1921815357 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557448b74c00 0x557448bdf320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:25:42.386 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:42.381+0000 7f08cbe00640 1 -- 172.21.15.47:0/1921815357 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557448b6b6d0 msgr2=0x557448b78a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:25:42.387 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:42.381+0000 7f08cbe00640 1 --2- 172.21.15.47:0/1921815357 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557448b6b6d0 0x557448b78a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:25:42.387 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:42.381+0000 7f08cbe00640 1 -- 172.21.15.47:0/1921815357 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557448b74c00 msgr2=0x557448bdf320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:25:42.387 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:42.381+0000 7f08cbe00640 1 --2- 172.21.15.47:0/1921815357 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557448b74c00 0x557448bdf320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:25:42.387 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:42.381+0000 7f08cbe00640 1 -- 172.21.15.47:0/1921815357 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x557448bdf8b0 con 0x557448b6c080 2024-03-21T10:25:42.387 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:42.382+0000 7f08cbe00640 1 --2- 172.21.15.47:0/1921815357 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x557448b6c080 0x557448b78fc0 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f08b802ba20 tx=0x7f08b802cf00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8b30e9424383d5b server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:25:42.387 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:42.382+0000 7f08b2a00640 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-03-21T10:25:42.387 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:42.382+0000 7f08b2a00640 1 -- 172.21.15.47:0/1921815357 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08b8040020 con 0x557448b6c080 2024-03-21T10:25:42.387 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:42.382+0000 7f08b2a00640 1 -- 172.21.15.47:0/1921815357 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 9474+0+0 (secure 0 0 0) 0x7f08b802b3f0 con 0x557448b6c080 2024-03-21T10:25:42.387 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:42.382+0000 7f08d65dd740 1 -- 172.21.15.47:0/1921815357 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f08c409c290 con 0x557448b6c080 2024-03-21T10:25:42.387 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:42.383+0000 7f08b2a00640 1 -- 172.21.15.47:0/1921815357 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f08b8048050 con 0x557448b6c080 2024-03-21T10:25:42.387 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:42.383+0000 7f08d65dd740 10 client.15897 fetch_fsmap learned FSMap version 30 2024-03-21T10:25:42.387 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:42.383+0000 7f08d65dd740 1 -- 172.21.15.47:0/1921815357 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x557448be0bb0 con 0x557448b6c080 2024-03-21T10:25:42.387 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:42.383+0000 7f08b2a00640 1 -- 172.21.15.47:0/1921815357 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f08c409c290 con 0x557448b6c080 2024-03-21T10:25:42.387 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:42.383+0000 7f08b2a00640 1 -- 172.21.15.47:0/1921815357 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 30) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f08b8005380 con 0x557448b6c080 2024-03-21T10:25:42.387 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:42.383+0000 7f08d65dd740 10 client.15897 fetch_fsmap finished waiting for FSMap version 30 2024-03-21T10:25:42.388 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:42.383+0000 7f08d65dd740 10 client.15897 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-21T10:25:42.388 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:42.383+0000 7f08d65dd740 20 client.15897 populate_metadata read hostname 'smithi047' 2024-03-21T10:25:42.388 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:42.383+0000 7f08d65dd740 1 --2- 172.21.15.47:0/1921815357 >> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] conn(0x557448be12b0 0x557448be36d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:25:42.388 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:42.383+0000 7f08d65dd740 4 client.15897 mds_command: new command op to 34535 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-21T10:25:42.388 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:42.383+0000 7f08d65dd740 1 -- 172.21.15.47:0/1921815357 --> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x557448ad6b50 con 0x557448be12b0 2024-03-21T10:25:42.388 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:42.384+0000 7f08d0e00640 1 --2- 172.21.15.47:0/1921815357 >> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] conn(0x557448be12b0 0x557448be36d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:25:42.389 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:42.385+0000 7f08b2a00640 10 client.15897 ms_handle_connect on v2:172.21.15.100:6838/4173168440 2024-03-21T10:25:42.389 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:42.385+0000 7f08d0e00640 1 --2- 172.21.15.47:0/1921815357 >> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] conn(0x557448be12b0 0x557448be36d0 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c92fa5a8098e396b server_cookie=9b9d10475abdb655 in_seq=0 out_seq=0 2024-03-21T10:25:42.389 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:42.385+0000 7f08b2a00640 1 -- 172.21.15.47:0/1921815357 <== mds.0 v2:172.21.15.100:6838/4173168440 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x557448ad6b50 con 0x557448be12b0 2024-03-21T10:25:42.389 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:42.385+0000 7f08b2a00640 10 client.15897 handle_command_reply: tid=0 2024-03-21T10:25:42.390 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:42.385+0000 7f08d65dd740 1 client.15897 shutdown 2024-03-21T10:25:42.390 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:42.385+0000 7f08d65dd740 1 -- 172.21.15.47:0/1921815357 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x557448b6c080 msgr2=0x557448b78fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:25:42.390 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:42.385+0000 7f08d65dd740 1 --2- 172.21.15.47:0/1921815357 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x557448b6c080 0x557448b78fc0 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f08b802ba20 tx=0x7f08b802cf00 comp rx=0 tx=0).stop 2024-03-21T10:25:42.390 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:42.385+0000 7f08d65dd740 1 -- 172.21.15.47:0/1921815357 shutdown_connections 2024-03-21T10:25:42.390 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:42.385+0000 7f08d65dd740 1 --2- 172.21.15.47:0/1921815357 >> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] conn(0x557448be12b0 0x557448be36d0 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:25:42.390 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:42.385+0000 7f08d65dd740 1 --2- 172.21.15.47:0/1921815357 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557448b74c00 0x557448bdf320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:25:42.390 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:42.385+0000 7f08d65dd740 1 --2- 172.21.15.47:0/1921815357 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x557448b6c080 0x557448b78fc0 unknown :-1 s=CLOSED pgs=563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:25:42.390 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:42.385+0000 7f08d65dd740 1 --2- 172.21.15.47:0/1921815357 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557448b6b6d0 0x557448b78a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:25:42.390 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:42.385+0000 7f08b2a00640 0 client.15897 ms_handle_reset on v2:172.21.15.100:6838/4173168440 2024-03-21T10:25:42.390 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:42.385+0000 7f08d65dd740 1 -- 172.21.15.47:0/1921815357 >> 172.21.15.47:0/1921815357 conn(0x557448b65da0 msgr2=0x557448b64620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:25:42.390 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:42.385+0000 7f08d65dd740 1 -- 172.21.15.47:0/1921815357 shutdown_connections 2024-03-21T10:25:42.390 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:42.385+0000 7f08d65dd740 1 -- 172.21.15.47:0/1921815357 wait complete. 2024-03-21T10:25:42.390 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:42.386+0000 7f08d65dd740 20 client.15897 trim_cache size 0 max 16384 2024-03-21T10:25:42.391 INFO:teuthology.orchestra.run.smithi047.stdout:[] 2024-03-21T10:25:42.391 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:42.386+0000 7f08d3800640 1 -- 172.21.15.47:0/974546312 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f08b40a31a0 msgr2=0x7f08b40a5660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:25:42.392 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:42.386+0000 7f08d3800640 1 --2- 172.21.15.47:0/974546312 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f08b40a31a0 0x7f08b40a5660 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f08cc100030 tx=0x7f08b8013040 comp rx=0 tx=0).stop 2024-03-21T10:25:42.392 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:42.386+0000 7f08d3800640 1 -- 172.21.15.47:0/974546312 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f08cc10e080 msgr2=0x7f08cc19e970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:25:42.392 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:42.386+0000 7f08d3800640 1 --2- 172.21.15.47:0/974546312 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f08cc10e080 0x7f08cc19e970 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f08c400eb40 tx=0x7f08c400c2d0 comp rx=0 tx=0).stop 2024-03-21T10:25:42.392 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:42.386+0000 7f08d3800640 1 -- 172.21.15.47:0/974546312 shutdown_connections 2024-03-21T10:25:42.392 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:42.386+0000 7f08d3800640 1 --2- 172.21.15.47:0/974546312 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f08cc10e080 0x7f08cc19e970 unknown :-1 s=CLOSED pgs=559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:25:42.392 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:42.386+0000 7f08d3800640 1 --2- 172.21.15.47:0/974546312 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f08cc100df0 0x7f08cc19a640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:25:42.392 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:42.386+0000 7f08d3800640 1 --2- 172.21.15.47:0/974546312 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f08b40a31a0 0x7f08b40a5660 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:25:42.392 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:42.386+0000 7f08d3800640 1 --2- 172.21.15.47:0/974546312 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f08cc1004d0 0x7f08cc19a100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:25:42.392 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:42.386+0000 7f08d3800640 1 -- 172.21.15.47:0/974546312 >> 172.21.15.47:0/974546312 conn(0x7f08cc077700 msgr2=0x7f08cc10f6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:25:42.392 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:42.386+0000 7f08d3800640 1 -- 172.21.15.47:0/974546312 shutdown_connections 2024-03-21T10:25:42.392 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:42.387+0000 7f08d3800640 1 -- 172.21.15.47:0/974546312 wait complete. 2024-03-21T10:25:43.239 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:25:43 smithi157 ceph-mon[35581]: pgmap v14: 129 pgs: 129 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 2.3 KiB/s wr, 17 op/s 2024-03-21T10:25:43.408 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-03-21T10:25:43.408 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-21T10:25:43.408 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-03-21T10:25:43.463 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:25:43 smithi047 ceph-mon[31547]: pgmap v14: 129 pgs: 129 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 2.3 KiB/s wr, 17 op/s 2024-03-21T10:25:43.479 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:25:43 smithi100 ceph-mon[35193]: pgmap v14: 129 pgs: 129 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 2.3 KiB/s wr, 17 op/s 2024-03-21T10:25:43.653 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:43.651+0000 7f75f2600640 1 -- 172.21.15.47:0/4075031068 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f75ec1040e0 msgr2=0x7f75ec1105f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:25:43.653 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:43.651+0000 7f75f2600640 1 --2- 172.21.15.47:0/4075031068 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f75ec1040e0 0x7f75ec1105f0 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f75e4009cd0 tx=0x7f75e402c0f0 comp rx=0 tx=0).stop 2024-03-21T10:25:43.653 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:43.651+0000 7f75f2600640 1 -- 172.21.15.47:0/4075031068 shutdown_connections 2024-03-21T10:25:43.653 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:43.651+0000 7f75f2600640 1 --2- 172.21.15.47:0/4075031068 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f75ec1040e0 0x7f75ec1105f0 unknown :-1 s=CLOSED pgs=564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:25:43.653 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:43.651+0000 7f75f2600640 1 --2- 172.21.15.47:0/4075031068 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f75ec103720 0x7f75ec103ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:25:43.653 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:43.651+0000 7f75f2600640 1 --2- 172.21.15.47:0/4075031068 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f75ec109730 0x7f75ec109b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:25:43.653 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:43.651+0000 7f75f2600640 1 -- 172.21.15.47:0/4075031068 >> 172.21.15.47:0/4075031068 conn(0x7f75ec0ff1d0 msgr2=0x7f75ec1015f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:25:43.653 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:43.651+0000 7f75f2600640 1 -- 172.21.15.47:0/4075031068 shutdown_connections 2024-03-21T10:25:43.653 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:43.651+0000 7f75f2600640 1 -- 172.21.15.47:0/4075031068 wait complete. 2024-03-21T10:25:43.656 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:43.652+0000 7f75f2600640 1 Processor -- start 2024-03-21T10:25:43.656 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:43.652+0000 7f75f2600640 1 -- start start 2024-03-21T10:25:43.656 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:43.653+0000 7f75f2600640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f75ec103720 0x7f75ec199f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:25:43.656 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:43.653+0000 7f75f2600640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f75ec1040e0 0x7f75ec19a490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:25:43.656 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:43.653+0000 7f75f2600640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f75ec109730 0x7f75ec19e810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:25:43.656 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:43.653+0000 7f75f2600640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75ec1139f0 con 0x7f75ec1040e0 2024-03-21T10:25:43.656 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:43.653+0000 7f75f2600640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75ec113870 con 0x7f75ec103720 2024-03-21T10:25:43.657 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:43.653+0000 7f75f2600640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75ec113b70 con 0x7f75ec109730 2024-03-21T10:25:43.657 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:43.653+0000 7f75ebe00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f75ec1040e0 0x7f75ec19a490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:25:43.657 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:43.653+0000 7f75ebe00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f75ec1040e0 0x7f75ec19a490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:55082/0 (socket says 172.21.15.47:55082) 2024-03-21T10:25:43.657 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:43.653+0000 7f75ebe00640 1 -- 172.21.15.47:0/1721579884 learned_addr learned my addr 172.21.15.47:0/1721579884 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:25:43.657 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:43.653+0000 7f75f1200640 1 --2- 172.21.15.47:0/1721579884 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f75ec103720 0x7f75ec199f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:25:43.657 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:43.653+0000 7f75f1c00640 1 --2- 172.21.15.47:0/1721579884 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f75ec109730 0x7f75ec19e810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:25:43.657 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:43.654+0000 7f75ebe00640 1 -- 172.21.15.47:0/1721579884 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f75ec109730 msgr2=0x7f75ec19e810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:25:43.657 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:43.654+0000 7f75ebe00640 1 --2- 172.21.15.47:0/1721579884 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f75ec109730 0x7f75ec19e810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:25:43.657 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:43.654+0000 7f75ebe00640 1 -- 172.21.15.47:0/1721579884 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f75ec103720 msgr2=0x7f75ec199f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:25:43.657 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:43.654+0000 7f75ebe00640 1 --2- 172.21.15.47:0/1721579884 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f75ec103720 0x7f75ec199f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:25:43.658 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:43.654+0000 7f75ebe00640 1 -- 172.21.15.47:0/1721579884 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f75ec19ef40 con 0x7f75ec1040e0 2024-03-21T10:25:43.658 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:43.654+0000 7f75ebe00640 1 --2- 172.21.15.47:0/1721579884 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f75ec1040e0 0x7f75ec19a490 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7f75d800d950 tx=0x7f75d800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:25:43.658 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:43.654+0000 7f75e9600640 1 -- 172.21.15.47:0/1721579884 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75d8017070 con 0x7f75ec1040e0 2024-03-21T10:25:43.658 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:43.654+0000 7f75e9600640 1 -- 172.21.15.47:0/1721579884 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f75d8009c30 con 0x7f75ec1040e0 2024-03-21T10:25:43.658 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:43.654+0000 7f75f2600640 1 -- 172.21.15.47:0/1721579884 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f75ec19f230 con 0x7f75ec1040e0 2024-03-21T10:25:43.658 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:43.654+0000 7f75e9600640 1 -- 172.21.15.47:0/1721579884 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75d800f620 con 0x7f75ec1040e0 2024-03-21T10:25:43.658 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:43.655+0000 7f75f2600640 1 -- 172.21.15.47:0/1721579884 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f75ec1a6ac0 con 0x7f75ec1040e0 2024-03-21T10:25:43.660 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:43.659+0000 7f75e9600640 1 -- 172.21.15.47:0/1721579884 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 25) v1 ==== 138030+0+0 (secure 0 0 0) 0x7f75d80109a0 con 0x7f75ec1040e0 2024-03-21T10:25:43.661 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:43.659+0000 7f75f69db740 1 Processor -- start 2024-03-21T10:25:43.661 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:43.660+0000 7f75f69db740 1 -- start start 2024-03-21T10:25:43.663 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:43.660+0000 7f75f69db740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a14c41b290 0x55a14c41b670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:25:43.663 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:43.660+0000 7f75f69db740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a14c41bbb0 0x55a14c424250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:25:43.663 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:43.660+0000 7f75f69db740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a14c424790 0x55a14c426b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:25:43.663 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:43.660+0000 7f75f69db740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a14c37feb0 con 0x55a14c41b290 2024-03-21T10:25:43.663 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:43.660+0000 7f75f69db740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a14c328a60 con 0x55a14c41bbb0 2024-03-21T10:25:43.663 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:43.660+0000 7f75f69db740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a14c37bf50 con 0x55a14c424790 2024-03-21T10:25:43.663 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:43.660+0000 7f75e9600640 1 --2- 172.21.15.47:0/1721579884 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f75d40a3120 0x7f75d40a55e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:25:43.663 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:43.660+0000 7f75e9600640 1 -- 172.21.15.47:0/1721579884 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 9474+0+0 (secure 0 0 0) 0x7f75d80cf9d0 con 0x7f75ec1040e0 2024-03-21T10:25:43.663 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:43.661+0000 7f75f1c00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a14c41b290 0x55a14c41b670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:25:43.663 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:43.661+0000 7f75f1200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a14c41bbb0 0x55a14c424250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:25:43.663 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:43.661+0000 7f75f1c00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a14c41b290 0x55a14c41b670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:55088/0 (socket says 172.21.15.47:55088) 2024-03-21T10:25:43.663 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:43.661+0000 7f75f1c00640 1 -- 172.21.15.47:0/2811834188 learned_addr learned my addr 172.21.15.47:0/2811834188 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:25:43.664 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:43.661+0000 7f75ebe00640 1 --2- 172.21.15.47:0/2811834188 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a14c424790 0x55a14c426b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:25:43.664 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:43.661+0000 7f75f1c00640 1 --2- 172.21.15.47:0/1721579884 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f75d40a3120 0x7f75d40a55e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:25:43.664 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:43.661+0000 7f75f1200640 1 -- 172.21.15.47:0/2811834188 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a14c424790 msgr2=0x55a14c426b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:25:43.664 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:43.661+0000 7f75f1200640 1 --2- 172.21.15.47:0/2811834188 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a14c424790 0x55a14c426b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:25:43.664 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:43.661+0000 7f75f1200640 1 -- 172.21.15.47:0/2811834188 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a14c41b290 msgr2=0x55a14c41b670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:25:43.664 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:43.661+0000 7f75f1200640 1 --2- 172.21.15.47:0/2811834188 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a14c41b290 0x55a14c41b670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:25:43.664 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:43.661+0000 7f75f1200640 1 -- 172.21.15.47:0/2811834188 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a14c15c7e0 con 0x55a14c41bbb0 2024-03-21T10:25:43.664 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:43.661+0000 7f75ebe00640 1 --2- 172.21.15.47:0/2811834188 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a14c424790 0x55a14c426b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T10:25:43.667 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:43.662+0000 7f75f1200640 1 --2- 172.21.15.47:0/2811834188 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a14c41bbb0 0x55a14c424250 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f75e00045b0 tx=0x7f75e000bb20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3196602648ac768 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:25:43.667 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:43.662+0000 7f75f1c00640 1 --2- 172.21.15.47:0/1721579884 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f75d40a3120 0x7f75d40a55e0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f75e4009ca0 tx=0x7f75e40095c0 comp rx=0 tx=0).ready entity=mgr.15687 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:25:43.667 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:43.662+0000 7f75de000640 1 -- 172.21.15.47:0/2811834188 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75e0003bd0 con 0x55a14c41bbb0 2024-03-21T10:25:43.667 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:43.662+0000 7f75de000640 1 -- 172.21.15.47:0/2811834188 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f75e0015070 con 0x55a14c41bbb0 2024-03-21T10:25:43.667 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:43.662+0000 7f75de000640 1 -- 172.21.15.47:0/2811834188 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75e0004cf0 con 0x55a14c41bbb0 2024-03-21T10:25:43.667 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:43.663+0000 7f75f69db740 1 -- 172.21.15.47:0/2811834188 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a14c41bbb0 msgr2=0x55a14c424250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:25:43.667 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:43.663+0000 7f75f69db740 1 --2- 172.21.15.47:0/2811834188 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a14c41bbb0 0x55a14c424250 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f75e00045b0 tx=0x7f75e000bb20 comp rx=0 tx=0).stop 2024-03-21T10:25:43.668 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:43.663+0000 7f75f69db740 1 -- 172.21.15.47:0/2811834188 shutdown_connections 2024-03-21T10:25:43.668 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:43.663+0000 7f75f69db740 1 --2- 172.21.15.47:0/2811834188 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a14c424790 0x55a14c426b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:25:43.668 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:43.663+0000 7f75f69db740 1 --2- 172.21.15.47:0/2811834188 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a14c41bbb0 0x55a14c424250 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:25:43.668 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:43.663+0000 7f75f69db740 1 --2- 172.21.15.47:0/2811834188 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a14c41b290 0x55a14c41b670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:25:43.668 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:43.663+0000 7f75f69db740 1 -- 172.21.15.47:0/2811834188 >> 172.21.15.47:0/2811834188 conn(0x55a14c4159c0 msgr2=0x55a14c415da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:25:43.668 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:43.663+0000 7f75f69db740 1 -- 172.21.15.47:0/2811834188 shutdown_connections 2024-03-21T10:25:43.668 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:43.663+0000 7f75f69db740 1 -- 172.21.15.47:0/2811834188 wait complete. 2024-03-21T10:25:43.668 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:43.664+0000 7f75f69db740 1 -- start start 2024-03-21T10:25:43.668 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:43.664+0000 7f75f69db740 1 Processor -- start 2024-03-21T10:25:43.668 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:43.664+0000 7f75f69db740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a14c41b290 0x55a14c4240f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:25:43.668 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:43.664+0000 7f75f69db740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a14c41bbb0 0x55a14c421480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:25:43.669 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:43.664+0000 7f75f69db740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a14c424790 0x55a14c4219e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:25:43.669 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:43.664+0000 7f75f69db740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a14c427580 con 0x55a14c424790 2024-03-21T10:25:43.669 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:43.664+0000 7f75f69db740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a14c33beb0 con 0x55a14c41b290 2024-03-21T10:25:43.669 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:43.664+0000 7f75f69db740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a14c427700 con 0x55a14c41bbb0 2024-03-21T10:25:43.669 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:43.664+0000 7f75f69db740 10 client.0 _ll_register_callbacks cb 0x55a14c330150 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T10:25:43.669 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:43.665+0000 7f75f1c00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a14c424790 0x55a14c4219e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:25:43.669 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:43.665+0000 7f75f1c00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a14c424790 0x55a14c4219e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:55102/0 (socket says 172.21.15.47:55102) 2024-03-21T10:25:43.669 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:43.665+0000 7f75f1c00640 1 -- 172.21.15.47:0/2681497988 learned_addr learned my addr 172.21.15.47:0/2681497988 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:25:43.669 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:43.665+0000 7f75f1200640 1 --2- 172.21.15.47:0/2681497988 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a14c41b290 0x55a14c4240f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:25:43.669 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:43.665+0000 7f75ebe00640 1 --2- 172.21.15.47:0/2681497988 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a14c41bbb0 0x55a14c421480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:25:43.669 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:43.665+0000 7f75f1c00640 1 -- 172.21.15.47:0/2681497988 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a14c41bbb0 msgr2=0x55a14c421480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:25:43.669 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:43.665+0000 7f75f1c00640 1 --2- 172.21.15.47:0/2681497988 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a14c41bbb0 0x55a14c421480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:25:43.669 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:43.665+0000 7f75f1c00640 1 -- 172.21.15.47:0/2681497988 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a14c41b290 msgr2=0x55a14c4240f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:25:43.669 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:43.665+0000 7f75f1c00640 1 --2- 172.21.15.47:0/2681497988 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a14c41b290 0x55a14c4240f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:25:43.669 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:43.665+0000 7f75f1c00640 1 -- 172.21.15.47:0/2681497988 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55a14c1bcf70 con 0x55a14c424790 2024-03-21T10:25:43.670 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:43.665+0000 7f75ebe00640 1 --2- 172.21.15.47:0/2681497988 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a14c41bbb0 0x55a14c421480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T10:25:43.672 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:43.666+0000 7f75f1c00640 1 --2- 172.21.15.47:0/2681497988 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a14c424790 0x55a14c4219e0 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f75e4037040 tx=0x7f75e402c0f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ca431b3f4e5a7d2b server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:25:43.673 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:43.666+0000 7f75d2a00640 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-03-21T10:25:43.673 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:43.666+0000 7f75d2a00640 1 -- 172.21.15.47:0/2681497988 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75e4054060 con 0x55a14c424790 2024-03-21T10:25:43.673 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:43.666+0000 7f75f69db740 1 -- 172.21.15.47:0/2681497988 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f75e0015070 con 0x55a14c424790 2024-03-21T10:25:43.673 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:43.666+0000 7f75d2a00640 1 -- 172.21.15.47:0/2681497988 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 9474+0+0 (secure 0 0 0) 0x7f75e4004040 con 0x55a14c424790 2024-03-21T10:25:43.673 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:43.666+0000 7f75d2a00640 1 -- 172.21.15.47:0/2681497988 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f75e405b050 con 0x55a14c424790 2024-03-21T10:25:43.673 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:43.666+0000 7f75f69db740 10 client.15909 fetch_fsmap learned FSMap version 30 2024-03-21T10:25:43.673 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:43.667+0000 7f75f69db740 1 -- 172.21.15.47:0/2681497988 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x55a14c054170 con 0x55a14c424790 2024-03-21T10:25:43.673 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:43.667+0000 7f75d2a00640 1 -- 172.21.15.47:0/2681497988 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f75e0015070 con 0x55a14c424790 2024-03-21T10:25:43.673 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:43.667+0000 7f75d2a00640 1 -- 172.21.15.47:0/2681497988 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 30) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f75e404f7e0 con 0x55a14c424790 2024-03-21T10:25:43.673 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:43.667+0000 7f75f69db740 10 client.15909 fetch_fsmap finished waiting for FSMap version 30 2024-03-21T10:25:43.674 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:43.667+0000 7f75f69db740 10 client.15909 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-21T10:25:43.674 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:43.667+0000 7f75f69db740 20 client.15909 populate_metadata read hostname 'smithi047' 2024-03-21T10:25:43.674 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:43.667+0000 7f75f69db740 1 --2- 172.21.15.47:0/2681497988 >> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] conn(0x55a14c434750 0x55a14c436b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:25:43.674 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:43.667+0000 7f75f69db740 4 client.15909 mds_command: new command op to 34535 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T10:25:43.674 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:43.667+0000 7f75f69db740 1 -- 172.21.15.47:0/2681497988 --> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55a14c38d3f0 con 0x55a14c434750 2024-03-21T10:25:43.674 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:43.668+0000 7f75f1200640 1 --2- 172.21.15.47:0/2681497988 >> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] conn(0x55a14c434750 0x55a14c436b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:25:43.674 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:43.668+0000 7f75f1200640 1 --2- 172.21.15.47:0/2681497988 >> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] conn(0x55a14c434750 0x55a14c436b40 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f7728bb9a4a061b9 server_cookie=e9aac4efee277f32 in_seq=0 out_seq=0 2024-03-21T10:25:43.674 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:43.668+0000 7f75d2a00640 10 client.15909 ms_handle_connect on v2:172.21.15.100:6838/4173168440 2024-03-21T10:25:43.674 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:43.669+0000 7f75d2a00640 1 -- 172.21.15.47:0/2681497988 <== mds.0 v2:172.21.15.100:6838/4173168440 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55a14c38d3f0 con 0x55a14c434750 2024-03-21T10:25:43.674 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:43.669+0000 7f75d2a00640 10 client.15909 handle_command_reply: tid=0 2024-03-21T10:25:43.674 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:43.669+0000 7f75f69db740 1 client.15909 shutdown 2024-03-21T10:25:43.675 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:43.670+0000 7f75f69db740 1 -- 172.21.15.47:0/2681497988 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a14c424790 msgr2=0x55a14c4219e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:25:43.675 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:43.670+0000 7f75f69db740 1 --2- 172.21.15.47:0/2681497988 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a14c424790 0x55a14c4219e0 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f75e4037040 tx=0x7f75e402c0f0 comp rx=0 tx=0).stop 2024-03-21T10:25:43.675 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:43.670+0000 7f75f69db740 1 -- 172.21.15.47:0/2681497988 shutdown_connections 2024-03-21T10:25:43.675 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:43.670+0000 7f75f69db740 1 --2- 172.21.15.47:0/2681497988 >> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] conn(0x55a14c434750 0x55a14c436b40 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:25:43.675 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:43.670+0000 7f75f69db740 1 --2- 172.21.15.47:0/2681497988 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a14c424790 0x55a14c4219e0 unknown :-1 s=CLOSED pgs=566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:25:43.675 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:43.670+0000 7f75f69db740 1 --2- 172.21.15.47:0/2681497988 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a14c41bbb0 0x55a14c421480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:25:43.675 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:43.670+0000 7f75f69db740 1 --2- 172.21.15.47:0/2681497988 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a14c41b290 0x55a14c4240f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:25:43.675 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:43.670+0000 7f75d2a00640 0 client.15909 ms_handle_reset on v2:172.21.15.100:6838/4173168440 2024-03-21T10:25:43.675 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:43.670+0000 7f75f69db740 1 -- 172.21.15.47:0/2681497988 >> 172.21.15.47:0/2681497988 conn(0x55a14c4159c0 msgr2=0x55a14c417710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:25:43.676 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:43.670+0000 7f75f69db740 1 -- 172.21.15.47:0/2681497988 shutdown_connections 2024-03-21T10:25:43.676 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:43.670+0000 7f75f69db740 1 -- 172.21.15.47:0/2681497988 wait complete. 2024-03-21T10:25:43.676 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:43.670+0000 7f75f69db740 20 client.15909 trim_cache size 0 max 16384 2024-03-21T10:25:43.690 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:43.688+0000 7f75f69db740 1 Processor -- start 2024-03-21T10:25:43.690 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:43.688+0000 7f75f69db740 1 -- start start 2024-03-21T10:25:43.690 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:43.688+0000 7f75f69db740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a14c41b290 0x55a14c414900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:25:43.690 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:43.688+0000 7f75f69db740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a14c41bbb0 0x55a14c423bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:25:43.690 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:43.688+0000 7f75f69db740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a14c424790 0x55a14c4240f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:25:43.690 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:43.688+0000 7f75f69db740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75ec1b09a0 con 0x55a14c41b290 2024-03-21T10:25:43.690 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:43.688+0000 7f75f69db740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75ec1b2320 con 0x55a14c424790 2024-03-21T10:25:43.690 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:43.688+0000 7f75f69db740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75ec1b1000 con 0x55a14c41bbb0 2024-03-21T10:25:43.691 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:43.689+0000 7f75f1200640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a14c41b290 0x55a14c414900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:25:43.691 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:43.689+0000 7f75f1200640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a14c41b290 0x55a14c414900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:55110/0 (socket says 172.21.15.47:55110) 2024-03-21T10:25:43.692 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:43.689+0000 7f75f1200640 1 -- 172.21.15.47:0/1309103524 learned_addr learned my addr 172.21.15.47:0/1309103524 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:25:43.692 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:43.689+0000 7f75f1c00640 1 --2- 172.21.15.47:0/1309103524 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a14c424790 0x55a14c4240f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:25:43.692 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:43.689+0000 7f75ebe00640 1 --2- 172.21.15.47:0/1309103524 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a14c41bbb0 0x55a14c423bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:25:43.692 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:43.689+0000 7f75f1200640 1 -- 172.21.15.47:0/1309103524 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a14c41bbb0 msgr2=0x55a14c423bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:25:43.692 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:43.689+0000 7f75f1200640 1 --2- 172.21.15.47:0/1309103524 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a14c41bbb0 0x55a14c423bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:25:43.692 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:43.689+0000 7f75f1200640 1 -- 172.21.15.47:0/1309103524 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a14c424790 msgr2=0x55a14c4240f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:25:43.692 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:43.689+0000 7f75f1200640 1 --2- 172.21.15.47:0/1309103524 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a14c424790 0x55a14c4240f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:25:43.692 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:43.689+0000 7f75f1200640 1 -- 172.21.15.47:0/1309103524 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a14c436eb0 con 0x55a14c41b290 2024-03-21T10:25:43.692 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:43.689+0000 7f75ebe00640 1 --2- 172.21.15.47:0/1309103524 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a14c41bbb0 0x55a14c423bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T10:25:43.692 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:43.689+0000 7f75f1200640 1 --2- 172.21.15.47:0/1309103524 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a14c41b290 0x55a14c414900 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f75e000b620 tx=0x7f75e000b390 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cb298f77fe714192 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:25:43.692 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:43.690+0000 7f75dd600640 1 -- 172.21.15.47:0/1309103524 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75e002c020 con 0x55a14c41b290 2024-03-21T10:25:43.692 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:43.690+0000 7f75dd600640 1 -- 172.21.15.47:0/1309103524 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f75e0015070 con 0x55a14c41b290 2024-03-21T10:25:43.692 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:43.690+0000 7f75dd600640 1 -- 172.21.15.47:0/1309103524 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75e00041a0 con 0x55a14c41b290 2024-03-21T10:25:43.697 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:43.690+0000 7f75f69db740 1 -- 172.21.15.47:0/1309103524 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a14c41b290 msgr2=0x55a14c414900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:25:43.697 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:43.690+0000 7f75f69db740 1 --2- 172.21.15.47:0/1309103524 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a14c41b290 0x55a14c414900 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f75e000b620 tx=0x7f75e000b390 comp rx=0 tx=0).stop 2024-03-21T10:25:43.698 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:43.691+0000 7f75f69db740 1 -- 172.21.15.47:0/1309103524 shutdown_connections 2024-03-21T10:25:43.698 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:43.691+0000 7f75f69db740 1 --2- 172.21.15.47:0/1309103524 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a14c424790 0x55a14c4240f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:25:43.698 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:43.691+0000 7f75f69db740 1 --2- 172.21.15.47:0/1309103524 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a14c41bbb0 0x55a14c423bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:25:43.698 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:43.691+0000 7f75f69db740 1 --2- 172.21.15.47:0/1309103524 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a14c41b290 0x55a14c414900 unknown :-1 s=CLOSED pgs=567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:25:43.698 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:43.691+0000 7f75f69db740 1 -- 172.21.15.47:0/1309103524 >> 172.21.15.47:0/1309103524 conn(0x55a14c4159c0 msgr2=0x55a14c4141f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:25:43.698 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:43.691+0000 7f75f69db740 1 -- 172.21.15.47:0/1309103524 shutdown_connections 2024-03-21T10:25:43.698 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:43.691+0000 7f75f69db740 1 -- 172.21.15.47:0/1309103524 wait complete. 2024-03-21T10:25:43.698 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:43.691+0000 7f75f69db740 1 -- start start 2024-03-21T10:25:43.699 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:43.691+0000 7f75f69db740 1 Processor -- start 2024-03-21T10:25:43.699 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:43.692+0000 7f75f69db740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a14c41b290 0x55a14c429420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:25:43.699 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:43.692+0000 7f75f69db740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a14c41bbb0 0x55a14c429960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:25:43.699 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:43.692+0000 7f75f69db740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a14c424790 0x55a14c48ffe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:25:43.699 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:43.692+0000 7f75f69db740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75ec1b1cc0 con 0x55a14c41b290 2024-03-21T10:25:43.699 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:43.692+0000 7f75f69db740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75ec1b1660 con 0x55a14c41bbb0 2024-03-21T10:25:43.699 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:43.692+0000 7f75f69db740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75ec1aeb60 con 0x55a14c424790 2024-03-21T10:25:43.699 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:43.692+0000 7f75f69db740 10 client.0 _ll_register_callbacks cb 0x7f75e4009b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T10:25:43.699 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:43.692+0000 7f75f1200640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a14c41b290 0x55a14c429420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:25:43.700 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:43.692+0000 7f75f1200640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a14c41b290 0x55a14c429420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:55112/0 (socket says 172.21.15.47:55112) 2024-03-21T10:25:43.700 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:43.692+0000 7f75f1200640 1 -- 172.21.15.47:0/3817199646 learned_addr learned my addr 172.21.15.47:0/3817199646 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:25:43.700 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:43.692+0000 7f75f1c00640 1 --2- 172.21.15.47:0/3817199646 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a14c424790 0x55a14c48ffe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:25:43.700 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:43.692+0000 7f75ebe00640 1 --2- 172.21.15.47:0/3817199646 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a14c41bbb0 0x55a14c429960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:25:43.700 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:43.693+0000 7f75f1200640 1 -- 172.21.15.47:0/3817199646 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a14c424790 msgr2=0x55a14c48ffe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:25:43.700 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:43.693+0000 7f75f1200640 1 --2- 172.21.15.47:0/3817199646 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a14c424790 0x55a14c48ffe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:25:43.700 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:43.693+0000 7f75f1200640 1 -- 172.21.15.47:0/3817199646 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a14c41bbb0 msgr2=0x55a14c429960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:25:43.700 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:43.693+0000 7f75f1200640 1 --2- 172.21.15.47:0/3817199646 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a14c41bbb0 0x55a14c429960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:25:43.701 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:43.693+0000 7f75f1200640 1 -- 172.21.15.47:0/3817199646 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55a14c490520 con 0x55a14c41b290 2024-03-21T10:25:43.701 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:43.693+0000 7f75f1c00640 1 --2- 172.21.15.47:0/3817199646 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a14c424790 0x55a14c48ffe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T10:25:43.701 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:43.693+0000 7f75f1200640 1 --2- 172.21.15.47:0/3817199646 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a14c41b290 0x55a14c429420 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f75e000bef0 tx=0x7f75e00280f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6b71413a03210ad server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:25:43.701 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:43.693+0000 7f75d2a00640 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-03-21T10:25:43.701 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:43.693+0000 7f75d2a00640 1 -- 172.21.15.47:0/3817199646 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75e0028310 con 0x55a14c41b290 2024-03-21T10:25:43.701 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:43.693+0000 7f75d2a00640 1 -- 172.21.15.47:0/3817199646 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 9474+0+0 (secure 0 0 0) 0x7f75e0004120 con 0x55a14c41b290 2024-03-21T10:25:43.701 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:43.693+0000 7f75f69db740 1 -- 172.21.15.47:0/3817199646 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f75e0015070 con 0x55a14c41b290 2024-03-21T10:25:43.701 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:43.694+0000 7f75d2a00640 1 -- 172.21.15.47:0/3817199646 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f75e0043050 con 0x55a14c41b290 2024-03-21T10:25:43.701 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:43.694+0000 7f75f69db740 10 client.15915 fetch_fsmap learned FSMap version 30 2024-03-21T10:25:43.702 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:43.694+0000 7f75f69db740 1 -- 172.21.15.47:0/3817199646 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x55a14c491800 con 0x55a14c41b290 2024-03-21T10:25:43.702 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:43.694+0000 7f75d2a00640 1 -- 172.21.15.47:0/3817199646 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f75e0015070 con 0x55a14c41b290 2024-03-21T10:25:43.702 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:43.694+0000 7f75d2a00640 1 -- 172.21.15.47:0/3817199646 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 30) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f75e00317d0 con 0x55a14c41b290 2024-03-21T10:25:43.702 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:43.694+0000 7f75f69db740 10 client.15915 fetch_fsmap finished waiting for FSMap version 30 2024-03-21T10:25:43.702 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:43.694+0000 7f75f69db740 10 client.15915 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-21T10:25:43.702 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:43.694+0000 7f75f69db740 20 client.15915 populate_metadata read hostname 'smithi047' 2024-03-21T10:25:43.702 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:43.694+0000 7f75f69db740 1 --2- 172.21.15.47:0/3817199646 >> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] conn(0x55a14c491d30 0x55a14c494120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:25:43.702 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:43.694+0000 7f75f69db740 4 client.15915 mds_command: new command op to 34535 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-03-21T10:25:43.702 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:43.694+0000 7f75f69db740 1 -- 172.21.15.47:0/3817199646 --> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x55a14c387740 con 0x55a14c491d30 2024-03-21T10:25:43.703 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:43.695+0000 7f75ebe00640 1 --2- 172.21.15.47:0/3817199646 >> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] conn(0x55a14c491d30 0x55a14c494120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:25:43.703 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:43.695+0000 7f75ebe00640 1 --2- 172.21.15.47:0/3817199646 >> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] conn(0x55a14c491d30 0x55a14c494120 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=41a7f0ea6ad4ab95 server_cookie=1a23c7815d6f4c66 in_seq=0 out_seq=0 2024-03-21T10:25:43.703 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:43.695+0000 7f75d2a00640 10 client.15915 ms_handle_connect on v2:172.21.15.100:6838/4173168440 2024-03-21T10:25:43.705 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:43.697+0000 7f75d2a00640 1 -- 172.21.15.47:0/3817199646 <== mds.0 v2:172.21.15.100:6838/4173168440 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x55a14c387740 con 0x55a14c491d30 2024-03-21T10:25:43.705 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:43.697+0000 7f75d2a00640 10 client.15915 handle_command_reply: tid=0 2024-03-21T10:25:43.705 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:43.697+0000 7f75f69db740 1 client.15915 shutdown 2024-03-21T10:25:43.705 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:43.697+0000 7f75f69db740 1 -- 172.21.15.47:0/3817199646 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a14c41b290 msgr2=0x55a14c429420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:25:43.705 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:43.697+0000 7f75f69db740 1 --2- 172.21.15.47:0/3817199646 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a14c41b290 0x55a14c429420 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f75e000bef0 tx=0x7f75e00280f0 comp rx=0 tx=0).stop 2024-03-21T10:25:43.705 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:43.697+0000 7f75f69db740 1 -- 172.21.15.47:0/3817199646 shutdown_connections 2024-03-21T10:25:43.705 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:43.697+0000 7f75f69db740 1 --2- 172.21.15.47:0/3817199646 >> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] conn(0x55a14c491d30 0x55a14c494120 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:25:43.705 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:43.697+0000 7f75f69db740 1 --2- 172.21.15.47:0/3817199646 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a14c424790 0x55a14c48ffe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:25:43.706 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:43.697+0000 7f75f69db740 1 --2- 172.21.15.47:0/3817199646 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a14c41bbb0 0x55a14c429960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:25:43.706 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:43.697+0000 7f75f69db740 1 --2- 172.21.15.47:0/3817199646 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a14c41b290 0x55a14c429420 unknown :-1 s=CLOSED pgs=568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:25:43.706 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:43.697+0000 7f75f69db740 1 -- 172.21.15.47:0/3817199646 >> 172.21.15.47:0/3817199646 conn(0x55a14c4159c0 msgr2=0x55a14c414a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:25:43.706 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:43.697+0000 7f75d2a00640 0 client.15915 ms_handle_reset on v2:172.21.15.100:6838/4173168440 2024-03-21T10:25:43.706 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:43.698+0000 7f75f69db740 1 -- 172.21.15.47:0/3817199646 shutdown_connections 2024-03-21T10:25:43.706 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:43.698+0000 7f75f69db740 1 -- 172.21.15.47:0/3817199646 wait complete. 2024-03-21T10:25:43.706 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:43.698+0000 7f75f69db740 20 client.15915 trim_cache size 0 max 16384 2024-03-21T10:25:43.707 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-03-21T10:25:43.707 INFO:teuthology.orchestra.run.smithi047.stdout: "return_code": 0, 2024-03-21T10:25:43.707 INFO:teuthology.orchestra.run.smithi047.stdout: "scrub_tag": "04dd17ee-543a-4799-b6fc-ae466b0940ad", 2024-03-21T10:25:43.707 INFO:teuthology.orchestra.run.smithi047.stdout: "mode": "asynchronous" 2024-03-21T10:25:43.707 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-03-21T10:25:43.708 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:43.698+0000 7f75f2600640 1 -- 172.21.15.47:0/1721579884 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f75d40a3120 msgr2=0x7f75d40a55e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:25:43.708 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:43.698+0000 7f75f2600640 1 --2- 172.21.15.47:0/1721579884 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f75d40a3120 0x7f75d40a55e0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f75e4009ca0 tx=0x7f75e40095c0 comp rx=0 tx=0).stop 2024-03-21T10:25:43.708 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:43.698+0000 7f75f2600640 1 -- 172.21.15.47:0/1721579884 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f75ec1040e0 msgr2=0x7f75ec19a490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:25:43.708 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:43.698+0000 7f75f2600640 1 --2- 172.21.15.47:0/1721579884 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f75ec1040e0 0x7f75ec19a490 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7f75d800d950 tx=0x7f75d800de10 comp rx=0 tx=0).stop 2024-03-21T10:25:43.708 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:43.699+0000 7f75f2600640 1 -- 172.21.15.47:0/1721579884 shutdown_connections 2024-03-21T10:25:43.708 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:43.699+0000 7f75f2600640 1 --2- 172.21.15.47:0/1721579884 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f75ec109730 0x7f75ec19e810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:25:43.708 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:43.699+0000 7f75f2600640 1 --2- 172.21.15.47:0/1721579884 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f75ec1040e0 0x7f75ec19a490 unknown :-1 s=CLOSED pgs=565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:25:43.708 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:43.699+0000 7f75f2600640 1 --2- 172.21.15.47:0/1721579884 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f75d40a3120 0x7f75d40a55e0 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:25:43.709 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:43.699+0000 7f75f2600640 1 --2- 172.21.15.47:0/1721579884 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f75ec103720 0x7f75ec199f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:25:43.709 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:43.699+0000 7f75f2600640 1 -- 172.21.15.47:0/1721579884 >> 172.21.15.47:0/1721579884 conn(0x7f75ec0ff1d0 msgr2=0x7f75ec0ffaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:25:43.709 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:43.699+0000 7f75f2600640 1 -- 172.21.15.47:0/1721579884 shutdown_connections 2024-03-21T10:25:43.709 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:43.699+0000 7f75f2600640 1 -- 172.21.15.47:0/1721579884 wait complete. 2024-03-21T10:25:43.720 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-21T10:25:43.720 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-21T10:25:43.925 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:43.923+0000 7fd761600640 1 -- 172.21.15.47:0/108679631 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd75c100720 msgr2=0x7fd75c100b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:25:43.925 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:43.923+0000 7fd761600640 1 --2- 172.21.15.47:0/108679631 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd75c100720 0x7fd75c100b00 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7fd74c009860 tx=0x7fd74c02bc10 comp rx=0 tx=0).stop 2024-03-21T10:25:43.925 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:43.923+0000 7fd761600640 1 -- 172.21.15.47:0/108679631 shutdown_connections 2024-03-21T10:25:43.926 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:43.923+0000 7fd761600640 1 --2- 172.21.15.47:0/108679631 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd75c105a40 0x7fd75c105ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:25:43.926 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:43.923+0000 7fd761600640 1 --2- 172.21.15.47:0/108679631 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd75c101040 0x7fd75c105500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:25:43.926 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:43.923+0000 7fd761600640 1 --2- 172.21.15.47:0/108679631 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd75c100720 0x7fd75c100b00 unknown :-1 s=CLOSED pgs=569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:25:43.926 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:43.923+0000 7fd761600640 1 -- 172.21.15.47:0/108679631 >> 172.21.15.47:0/108679631 conn(0x7fd75c077700 msgr2=0x7fd75c077b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:25:43.926 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:43.923+0000 7fd761600640 1 -- 172.21.15.47:0/108679631 shutdown_connections 2024-03-21T10:25:43.926 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:43.923+0000 7fd761600640 1 -- 172.21.15.47:0/108679631 wait complete. 2024-03-21T10:25:43.928 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:43.924+0000 7fd761600640 1 Processor -- start 2024-03-21T10:25:43.928 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:43.924+0000 7fd761600640 1 -- start start 2024-03-21T10:25:43.928 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:43.925+0000 7fd761600640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd75c100720 0x7fd75c19e490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:25:43.929 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:43.925+0000 7fd761600640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd75c101040 0x7fd75c19e9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:25:43.929 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:43.925+0000 7fd761600640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd75c105a40 0x7fd75c1a2d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:25:43.929 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:43.925+0000 7fd761600640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd75c117ed0 con 0x7fd75c101040 2024-03-21T10:25:43.929 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:43.925+0000 7fd761600640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd75c117d50 con 0x7fd75c100720 2024-03-21T10:25:43.929 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:43.925+0000 7fd761600640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd75c118050 con 0x7fd75c105a40 2024-03-21T10:25:43.929 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:43.925+0000 7fd75a000640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd75c101040 0x7fd75c19e9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:25:43.929 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:43.925+0000 7fd75aa00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd75c100720 0x7fd75c19e490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:25:43.929 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:43.925+0000 7fd75a000640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd75c101040 0x7fd75c19e9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:55120/0 (socket says 172.21.15.47:55120) 2024-03-21T10:25:43.929 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:43.925+0000 7fd75a000640 1 -- 172.21.15.47:0/2713801914 learned_addr learned my addr 172.21.15.47:0/2713801914 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:25:43.929 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:43.925+0000 7fd75b400640 1 --2- 172.21.15.47:0/2713801914 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd75c105a40 0x7fd75c1a2d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:25:43.929 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:43.926+0000 7fd75a000640 1 -- 172.21.15.47:0/2713801914 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd75c105a40 msgr2=0x7fd75c1a2d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:25:43.929 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:43.926+0000 7fd75a000640 1 --2- 172.21.15.47:0/2713801914 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd75c105a40 0x7fd75c1a2d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:25:43.930 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:43.926+0000 7fd75a000640 1 -- 172.21.15.47:0/2713801914 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd75c100720 msgr2=0x7fd75c19e490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:25:43.930 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:43.926+0000 7fd75a000640 1 --2- 172.21.15.47:0/2713801914 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd75c100720 0x7fd75c19e490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:25:43.930 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:43.926+0000 7fd75a000640 1 -- 172.21.15.47:0/2713801914 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd75c1a3480 con 0x7fd75c101040 2024-03-21T10:25:43.930 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:43.926+0000 7fd75b400640 1 --2- 172.21.15.47:0/2713801914 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd75c105a40 0x7fd75c1a2d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T10:25:43.930 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:43.926+0000 7fd75aa00640 1 --2- 172.21.15.47:0/2713801914 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd75c100720 0x7fd75c19e490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T10:25:43.930 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:43.926+0000 7fd75a000640 1 --2- 172.21.15.47:0/2713801914 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd75c101040 0x7fd75c19e9d0 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7fd74400c420 tx=0x7fd74400c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:25:43.930 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:43.926+0000 7fd753400640 1 -- 172.21.15.47:0/2713801914 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd744004b20 con 0x7fd75c101040 2024-03-21T10:25:43.930 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:43.927+0000 7fd753400640 1 -- 172.21.15.47:0/2713801914 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fd744003e20 con 0x7fd75c101040 2024-03-21T10:25:43.930 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:43.927+0000 7fd761600640 1 -- 172.21.15.47:0/2713801914 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd75c1a3770 con 0x7fd75c101040 2024-03-21T10:25:43.930 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:43.927+0000 7fd753400640 1 -- 172.21.15.47:0/2713801914 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd74400b630 con 0x7fd75c101040 2024-03-21T10:25:43.930 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:43.927+0000 7fd761600640 1 -- 172.21.15.47:0/2713801914 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd75c1ab000 con 0x7fd75c101040 2024-03-21T10:25:43.931 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:43.930+0000 7fd76437f740 1 Processor -- start 2024-03-21T10:25:43.931 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:43.930+0000 7fd76437f740 1 -- start start 2024-03-21T10:25:43.934 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:43.930+0000 7fd753400640 1 -- 172.21.15.47:0/2713801914 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 25) v1 ==== 138030+0+0 (secure 0 0 0) 0x7fd74401d020 con 0x7fd75c101040 2024-03-21T10:25:43.934 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:43.930+0000 7fd76437f740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556dd94bfc00 0x556dd94be950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:25:43.934 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:43.930+0000 7fd76437f740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556dd94bef20 0x556dd94bf3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:25:43.934 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:43.931+0000 7fd76437f740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x556dd94c8290 0x556dd94ca680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:25:43.934 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:43.931+0000 7fd76437f740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x556dd9423250 con 0x556dd94c8290 2024-03-21T10:25:43.934 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:43.931+0000 7fd76437f740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x556dd93cba30 con 0x556dd94bef20 2024-03-21T10:25:43.934 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:43.931+0000 7fd76437f740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x556dd941f2f0 con 0x556dd94bfc00 2024-03-21T10:25:43.934 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:43.931+0000 7fd75a000640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x556dd94c8290 0x556dd94ca680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:25:43.934 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:43.931+0000 7fd75a000640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x556dd94c8290 0x556dd94ca680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:55124/0 (socket says 172.21.15.47:55124) 2024-03-21T10:25:43.934 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:43.931+0000 7fd75a000640 1 -- 172.21.15.47:0/1837488877 learned_addr learned my addr 172.21.15.47:0/1837488877 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:25:43.934 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:43.931+0000 7fd75aa00640 1 --2- 172.21.15.47:0/1837488877 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556dd94bef20 0x556dd94bf3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:25:43.934 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:43.931+0000 7fd75b400640 1 --2- 172.21.15.47:0/1837488877 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556dd94bfc00 0x556dd94be950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:25:43.935 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:43.931+0000 7fd75a000640 1 -- 172.21.15.47:0/1837488877 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556dd94bfc00 msgr2=0x556dd94be950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:25:43.935 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:43.931+0000 7fd75a000640 1 --2- 172.21.15.47:0/1837488877 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556dd94bfc00 0x556dd94be950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:25:43.935 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:43.931+0000 7fd75a000640 1 -- 172.21.15.47:0/1837488877 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556dd94bef20 msgr2=0x556dd94bf3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:25:43.935 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:43.931+0000 7fd75a000640 1 --2- 172.21.15.47:0/1837488877 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556dd94bef20 0x556dd94bf3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:25:43.935 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:43.931+0000 7fd75a000640 1 -- 172.21.15.47:0/1837488877 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556dd93b6690 con 0x556dd94c8290 2024-03-21T10:25:43.935 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:43.932+0000 7fd75a000640 1 --2- 172.21.15.47:0/1837488877 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x556dd94c8290 0x556dd94ca680 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7fd7440ccf80 tx=0x7fd7440d3ea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f6dbc70ad265f988 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:25:43.935 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:43.932+0000 7fd74b400640 1 -- 172.21.15.47:0/1837488877 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7440d6ae0 con 0x556dd94c8290 2024-03-21T10:25:43.935 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:43.932+0000 7fd74b400640 1 -- 172.21.15.47:0/1837488877 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fd7440d6c80 con 0x556dd94c8290 2024-03-21T10:25:43.935 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:43.932+0000 7fd74b400640 1 -- 172.21.15.47:0/1837488877 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7440e0630 con 0x556dd94c8290 2024-03-21T10:25:43.939 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:43.933+0000 7fd753400640 1 --2- 172.21.15.47:0/2713801914 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7fd73c0a31a0 0x7fd73c0a5660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:25:43.939 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:43.933+0000 7fd753400640 1 -- 172.21.15.47:0/2713801914 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 9474+0+0 (secure 0 0 0) 0x7fd7440cfda0 con 0x7fd75c101040 2024-03-21T10:25:43.939 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:43.933+0000 7fd76437f740 1 -- 172.21.15.47:0/1837488877 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x556dd94c8290 msgr2=0x556dd94ca680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:25:43.939 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:43.933+0000 7fd76437f740 1 --2- 172.21.15.47:0/1837488877 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x556dd94c8290 0x556dd94ca680 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7fd7440ccf80 tx=0x7fd7440d3ea0 comp rx=0 tx=0).stop 2024-03-21T10:25:43.939 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:43.933+0000 7fd75b400640 1 --2- 172.21.15.47:0/2713801914 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7fd73c0a31a0 0x7fd73c0a5660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:25:43.939 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:43.933+0000 7fd76437f740 1 -- 172.21.15.47:0/1837488877 shutdown_connections 2024-03-21T10:25:43.940 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:43.933+0000 7fd76437f740 1 --2- 172.21.15.47:0/1837488877 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x556dd94c8290 0x556dd94ca680 unknown :-1 s=CLOSED pgs=571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:25:43.940 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:43.933+0000 7fd76437f740 1 --2- 172.21.15.47:0/1837488877 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556dd94bef20 0x556dd94bf3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:25:43.940 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:43.933+0000 7fd76437f740 1 --2- 172.21.15.47:0/1837488877 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556dd94bfc00 0x556dd94be950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:25:43.940 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:43.933+0000 7fd76437f740 1 -- 172.21.15.47:0/1837488877 >> 172.21.15.47:0/1837488877 conn(0x556dd9486200 msgr2=0x556dd94b9970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:25:43.940 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:43.933+0000 7fd76437f740 1 -- 172.21.15.47:0/1837488877 shutdown_connections 2024-03-21T10:25:43.940 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:43.933+0000 7fd76437f740 1 -- 172.21.15.47:0/1837488877 wait complete. 2024-03-21T10:25:43.940 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:43.934+0000 7fd75b400640 1 --2- 172.21.15.47:0/2713801914 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7fd73c0a31a0 0x7fd73c0a5660 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fd7540095f0 tx=0x7fd754005f90 comp rx=0 tx=0).ready entity=mgr.15687 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:25:43.940 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:43.934+0000 7fd76437f740 1 -- start start 2024-03-21T10:25:43.940 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:43.934+0000 7fd76437f740 1 Processor -- start 2024-03-21T10:25:43.940 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:43.935+0000 7fd76437f740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x556dd94bef20 0x556dd94c78e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:25:43.940 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:43.935+0000 7fd76437f740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556dd94bfc00 0x556dd94c4c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:25:43.940 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:43.935+0000 7fd76437f740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556dd94c8290 0x556dd94c51b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:25:43.940 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:43.935+0000 7fd76437f740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x556dd94cb100 con 0x556dd94bef20 2024-03-21T10:25:43.940 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:43.935+0000 7fd76437f740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x556dd93dee80 con 0x556dd94c8290 2024-03-21T10:25:43.940 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:43.935+0000 7fd76437f740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x556dd94cb280 con 0x556dd94bfc00 2024-03-21T10:25:43.941 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:43.935+0000 7fd76437f740 10 client.0 _ll_register_callbacks cb 0x556dd93f1130 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T10:25:43.941 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:43.935+0000 7fd75aa00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x556dd94bef20 0x556dd94c78e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:25:43.941 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:43.935+0000 7fd75aa00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x556dd94bef20 0x556dd94c78e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:55132/0 (socket says 172.21.15.47:55132) 2024-03-21T10:25:43.941 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:43.935+0000 7fd75aa00640 1 -- 172.21.15.47:0/3438754726 learned_addr learned my addr 172.21.15.47:0/3438754726 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:25:43.941 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:43.935+0000 7fd75b400640 1 --2- 172.21.15.47:0/3438754726 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556dd94c8290 0x556dd94c51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:25:43.941 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:43.935+0000 7fd75a000640 1 --2- 172.21.15.47:0/3438754726 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556dd94bfc00 0x556dd94c4c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:25:43.941 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:43.935+0000 7fd75aa00640 1 -- 172.21.15.47:0/3438754726 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556dd94bfc00 msgr2=0x556dd94c4c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:25:43.941 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:43.935+0000 7fd75aa00640 1 --2- 172.21.15.47:0/3438754726 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556dd94bfc00 0x556dd94c4c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:25:43.941 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:43.935+0000 7fd75aa00640 1 -- 172.21.15.47:0/3438754726 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556dd94c8290 msgr2=0x556dd94c51b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:25:43.941 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:43.935+0000 7fd75aa00640 1 --2- 172.21.15.47:0/3438754726 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556dd94c8290 0x556dd94c51b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:25:43.941 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:43.936+0000 7fd75aa00640 1 -- 172.21.15.47:0/3438754726 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x556dd92017d0 con 0x556dd94bef20 2024-03-21T10:25:43.941 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:43.936+0000 7fd75aa00640 1 --2- 172.21.15.47:0/3438754726 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x556dd94bef20 0x556dd94c78e0 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7fd74c009780 tx=0x7fd74c034960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dc79d1d5919a9c81 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:25:43.941 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:43.936+0000 7fd743e00640 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-03-21T10:25:43.941 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:43.936+0000 7fd743e00640 1 -- 172.21.15.47:0/3438754726 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd74c045070 con 0x556dd94bef20 2024-03-21T10:25:43.942 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:43.936+0000 7fd743e00640 1 -- 172.21.15.47:0/3438754726 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 9474+0+0 (secure 0 0 0) 0x7fd74c004240 con 0x556dd94bef20 2024-03-21T10:25:43.942 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:43.936+0000 7fd76437f740 1 -- 172.21.15.47:0/3438754726 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fd7440d6c80 con 0x556dd94bef20 2024-03-21T10:25:43.942 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:43.936+0000 7fd743e00640 1 -- 172.21.15.47:0/3438754726 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd74c051050 con 0x556dd94bef20 2024-03-21T10:25:43.942 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:43.937+0000 7fd76437f740 10 client.15927 fetch_fsmap learned FSMap version 30 2024-03-21T10:25:43.942 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:43.937+0000 7fd76437f740 1 -- 172.21.15.47:0/3438754726 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x556dd91373f0 con 0x556dd94bef20 2024-03-21T10:25:43.942 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:43.937+0000 7fd743e00640 1 -- 172.21.15.47:0/3438754726 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fd7440d6c80 con 0x556dd94bef20 2024-03-21T10:25:43.942 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:43.937+0000 7fd743e00640 1 -- 172.21.15.47:0/3438754726 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 30) v1 ==== 7942+0+0 (secure 0 0 0) 0x7fd74c0037b0 con 0x556dd94bef20 2024-03-21T10:25:43.942 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:43.937+0000 7fd76437f740 10 client.15927 fetch_fsmap finished waiting for FSMap version 30 2024-03-21T10:25:43.942 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:43.937+0000 7fd76437f740 10 client.15927 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-21T10:25:43.942 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:43.937+0000 7fd76437f740 20 client.15927 populate_metadata read hostname 'smithi047' 2024-03-21T10:25:43.942 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:43.937+0000 7fd76437f740 1 --2- 172.21.15.47:0/3438754726 >> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] conn(0x556dd94d8080 0x556dd94da4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:25:43.942 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:43.937+0000 7fd76437f740 4 client.15927 mds_command: new command op to 34535 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T10:25:43.942 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:43.937+0000 7fd76437f740 1 -- 172.21.15.47:0/3438754726 --> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x556dd9430790 con 0x556dd94d8080 2024-03-21T10:25:43.943 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:43.938+0000 7fd75a000640 1 --2- 172.21.15.47:0/3438754726 >> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] conn(0x556dd94d8080 0x556dd94da4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:25:43.944 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:43.938+0000 7fd75a000640 1 --2- 172.21.15.47:0/3438754726 >> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] conn(0x556dd94d8080 0x556dd94da4f0 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=215d63d9ef17cee4 server_cookie=a40dcd41fc08d08c in_seq=0 out_seq=0 2024-03-21T10:25:43.944 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:43.938+0000 7fd743e00640 10 client.15927 ms_handle_connect on v2:172.21.15.100:6838/4173168440 2024-03-21T10:25:43.944 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:43.940+0000 7fd743e00640 1 -- 172.21.15.47:0/3438754726 <== mds.0 v2:172.21.15.100:6838/4173168440 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x556dd9430790 con 0x556dd94d8080 2024-03-21T10:25:43.944 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:43.940+0000 7fd743e00640 10 client.15927 handle_command_reply: tid=0 2024-03-21T10:25:43.944 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:43.940+0000 7fd76437f740 1 client.15927 shutdown 2024-03-21T10:25:43.944 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:43.940+0000 7fd76437f740 1 -- 172.21.15.47:0/3438754726 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x556dd94bef20 msgr2=0x556dd94c78e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:25:43.944 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:43.940+0000 7fd76437f740 1 --2- 172.21.15.47:0/3438754726 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x556dd94bef20 0x556dd94c78e0 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7fd74c009780 tx=0x7fd74c034960 comp rx=0 tx=0).stop 2024-03-21T10:25:43.944 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:43.940+0000 7fd76437f740 1 -- 172.21.15.47:0/3438754726 shutdown_connections 2024-03-21T10:25:43.944 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:43.940+0000 7fd76437f740 1 --2- 172.21.15.47:0/3438754726 >> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] conn(0x556dd94d8080 0x556dd94da4f0 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:25:43.944 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:43.940+0000 7fd76437f740 1 --2- 172.21.15.47:0/3438754726 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556dd94c8290 0x556dd94c51b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:25:43.944 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:43.940+0000 7fd76437f740 1 --2- 172.21.15.47:0/3438754726 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556dd94bfc00 0x556dd94c4c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:25:43.944 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:43.940+0000 7fd76437f740 1 --2- 172.21.15.47:0/3438754726 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x556dd94bef20 0x556dd94c78e0 unknown :-1 s=CLOSED pgs=572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:25:43.944 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:43.940+0000 7fd743e00640 0 client.15927 ms_handle_reset on v2:172.21.15.100:6838/4173168440 2024-03-21T10:25:43.945 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:43.940+0000 7fd76437f740 1 -- 172.21.15.47:0/3438754726 >> 172.21.15.47:0/3438754726 conn(0x556dd9486200 msgr2=0x556dd94bb6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:25:43.945 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:43.941+0000 7fd76437f740 1 -- 172.21.15.47:0/3438754726 shutdown_connections 2024-03-21T10:25:43.945 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:43.941+0000 7fd76437f740 1 -- 172.21.15.47:0/3438754726 wait complete. 2024-03-21T10:25:43.945 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:43.941+0000 7fd76437f740 20 client.15927 trim_cache size 0 max 16384 2024-03-21T10:25:43.961 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:43.958+0000 7fd76437f740 1 Processor -- start 2024-03-21T10:25:43.961 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:43.958+0000 7fd76437f740 1 -- start start 2024-03-21T10:25:43.961 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:43.958+0000 7fd76437f740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556dd94bef20 0x556dd94c73a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:25:43.961 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:43.958+0000 7fd76437f740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556dd94bfc00 0x556dd94c78e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:25:43.961 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:43.959+0000 7fd76437f740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x556dd94c8290 0x556dd94c9680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:25:43.961 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:43.959+0000 7fd76437f740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd75c1b4ec0 con 0x556dd94c8290 2024-03-21T10:25:43.961 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:43.959+0000 7fd76437f740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd75c1b6840 con 0x556dd94bfc00 2024-03-21T10:25:43.961 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:43.959+0000 7fd76437f740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd75c1b5520 con 0x556dd94bef20 2024-03-21T10:25:43.962 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:43.959+0000 7fd75b400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x556dd94c8290 0x556dd94c9680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:25:43.962 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:43.959+0000 7fd75b400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x556dd94c8290 0x556dd94c9680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:55142/0 (socket says 172.21.15.47:55142) 2024-03-21T10:25:43.962 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:43.959+0000 7fd75b400640 1 -- 172.21.15.47:0/2255544153 learned_addr learned my addr 172.21.15.47:0/2255544153 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:25:43.962 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:43.959+0000 7fd75aa00640 1 --2- 172.21.15.47:0/2255544153 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556dd94bef20 0x556dd94c73a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:25:43.962 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:43.959+0000 7fd75a000640 1 --2- 172.21.15.47:0/2255544153 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556dd94bfc00 0x556dd94c78e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:25:43.962 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:43.959+0000 7fd75b400640 1 -- 172.21.15.47:0/2255544153 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556dd94bef20 msgr2=0x556dd94c73a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:25:43.962 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:43.959+0000 7fd75b400640 1 --2- 172.21.15.47:0/2255544153 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556dd94bef20 0x556dd94c73a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:25:43.962 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:43.959+0000 7fd75b400640 1 -- 172.21.15.47:0/2255544153 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556dd94bfc00 msgr2=0x556dd94c78e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:25:43.962 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:43.959+0000 7fd75b400640 1 --2- 172.21.15.47:0/2255544153 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556dd94bfc00 0x556dd94c78e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:25:43.962 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:43.959+0000 7fd75b400640 1 -- 172.21.15.47:0/2255544153 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556dd90f70f0 con 0x556dd94c8290 2024-03-21T10:25:43.962 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:43.959+0000 7fd75aa00640 1 --2- 172.21.15.47:0/2255544153 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556dd94bef20 0x556dd94c73a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T10:25:43.962 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:43.960+0000 7fd75b400640 1 --2- 172.21.15.47:0/2255544153 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x556dd94c8290 0x556dd94c9680 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7fd75401f490 tx=0x7fd754016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c8f0f6d0e3f7aa05 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:25:43.966 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:43.960+0000 7fd74aa00640 1 -- 172.21.15.47:0/2255544153 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd75401fa80 con 0x556dd94c8290 2024-03-21T10:25:43.967 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:43.960+0000 7fd74aa00640 1 -- 172.21.15.47:0/2255544153 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fd7540054d0 con 0x556dd94c8290 2024-03-21T10:25:43.967 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:43.960+0000 7fd74aa00640 1 -- 172.21.15.47:0/2255544153 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd754004d50 con 0x556dd94c8290 2024-03-21T10:25:43.967 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:43.960+0000 7fd76437f740 1 -- 172.21.15.47:0/2255544153 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x556dd94c8290 msgr2=0x556dd94c9680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:25:43.967 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:43.960+0000 7fd76437f740 1 --2- 172.21.15.47:0/2255544153 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x556dd94c8290 0x556dd94c9680 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7fd75401f490 tx=0x7fd754016000 comp rx=0 tx=0).stop 2024-03-21T10:25:43.967 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:43.961+0000 7fd76437f740 1 -- 172.21.15.47:0/2255544153 shutdown_connections 2024-03-21T10:25:43.967 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:43.961+0000 7fd76437f740 1 --2- 172.21.15.47:0/2255544153 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x556dd94c8290 0x556dd94c9680 unknown :-1 s=CLOSED pgs=573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:25:43.967 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:43.961+0000 7fd76437f740 1 --2- 172.21.15.47:0/2255544153 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556dd94bfc00 0x556dd94c78e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:25:43.967 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:43.961+0000 7fd76437f740 1 --2- 172.21.15.47:0/2255544153 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556dd94bef20 0x556dd94c73a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:25:43.967 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:43.961+0000 7fd76437f740 1 -- 172.21.15.47:0/2255544153 >> 172.21.15.47:0/2255544153 conn(0x556dd9486200 msgr2=0x556dd94b94a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:25:43.967 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:43.961+0000 7fd76437f740 1 -- 172.21.15.47:0/2255544153 shutdown_connections 2024-03-21T10:25:43.967 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:43.961+0000 7fd76437f740 1 -- 172.21.15.47:0/2255544153 wait complete. 2024-03-21T10:25:43.967 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:43.961+0000 7fd76437f740 1 -- start start 2024-03-21T10:25:43.967 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:43.961+0000 7fd76437f740 1 Processor -- start 2024-03-21T10:25:43.968 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:43.962+0000 7fd76437f740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556dd94bef20 0x556dd94c1b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:25:43.968 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:43.962+0000 7fd76437f740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556dd94bfc00 0x556dd94c20c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:25:43.968 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:43.962+0000 7fd76437f740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x556dd94c8290 0x556dd9532980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:25:43.968 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:43.962+0000 7fd76437f740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd75c1b61e0 con 0x556dd94c8290 2024-03-21T10:25:43.968 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:43.962+0000 7fd76437f740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd75c1b5b80 con 0x556dd94bfc00 2024-03-21T10:25:43.968 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:43.962+0000 7fd76437f740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd75c1b3080 con 0x556dd94bef20 2024-03-21T10:25:43.968 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:43.962+0000 7fd76437f740 10 client.0 _ll_register_callbacks cb 0x7fd744009460 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T10:25:43.968 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:43.962+0000 7fd75b400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x556dd94c8290 0x556dd9532980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:25:43.968 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:43.962+0000 7fd75b400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x556dd94c8290 0x556dd9532980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:55154/0 (socket says 172.21.15.47:55154) 2024-03-21T10:25:43.968 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:43.962+0000 7fd75b400640 1 -- 172.21.15.47:0/4169291574 learned_addr learned my addr 172.21.15.47:0/4169291574 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:25:43.968 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:43.962+0000 7fd75a000640 1 --2- 172.21.15.47:0/4169291574 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556dd94bfc00 0x556dd94c20c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:25:43.968 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:43.962+0000 7fd75aa00640 1 --2- 172.21.15.47:0/4169291574 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556dd94bef20 0x556dd94c1b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:25:43.968 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:43.963+0000 7fd75b400640 1 -- 172.21.15.47:0/4169291574 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556dd94bef20 msgr2=0x556dd94c1b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:25:43.968 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:43.963+0000 7fd75b400640 1 --2- 172.21.15.47:0/4169291574 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556dd94bef20 0x556dd94c1b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:25:43.969 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:43.963+0000 7fd75b400640 1 -- 172.21.15.47:0/4169291574 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556dd94bfc00 msgr2=0x556dd94c20c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:25:43.969 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:43.963+0000 7fd75b400640 1 --2- 172.21.15.47:0/4169291574 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556dd94bfc00 0x556dd94c20c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:25:43.969 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:43.963+0000 7fd75b400640 1 -- 172.21.15.47:0/4169291574 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x556dd9532f40 con 0x556dd94c8290 2024-03-21T10:25:43.969 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:43.963+0000 7fd75a000640 1 --2- 172.21.15.47:0/4169291574 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556dd94bfc00 0x556dd94c20c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T10:25:43.969 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:43.963+0000 7fd75b400640 1 --2- 172.21.15.47:0/4169291574 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x556dd94c8290 0x556dd9532980 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7fd75401f460 tx=0x7fd754004d40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4b768bec79dfcbde server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:25:43.969 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:43.963+0000 7fd743e00640 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-03-21T10:25:43.969 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:43.963+0000 7fd743e00640 1 -- 172.21.15.47:0/4169291574 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd754005170 con 0x556dd94c8290 2024-03-21T10:25:43.969 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:43.963+0000 7fd76437f740 1 -- 172.21.15.47:0/4169291574 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fd7540054d0 con 0x556dd94c8290 2024-03-21T10:25:43.969 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:43.963+0000 7fd743e00640 1 -- 172.21.15.47:0/4169291574 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 9474+0+0 (secure 0 0 0) 0x7fd754025040 con 0x556dd94c8290 2024-03-21T10:25:43.969 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:43.964+0000 7fd743e00640 1 -- 172.21.15.47:0/4169291574 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd75403f050 con 0x556dd94c8290 2024-03-21T10:25:43.969 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:43.964+0000 7fd76437f740 10 client.15933 fetch_fsmap learned FSMap version 30 2024-03-21T10:25:43.969 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:43.964+0000 7fd76437f740 1 -- 172.21.15.47:0/4169291574 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x556dd95341a0 con 0x556dd94c8290 2024-03-21T10:25:43.969 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:43.964+0000 7fd743e00640 1 -- 172.21.15.47:0/4169291574 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fd7540054d0 con 0x556dd94c8290 2024-03-21T10:25:43.969 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:43.964+0000 7fd743e00640 1 -- 172.21.15.47:0/4169291574 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 30) v1 ==== 7942+0+0 (secure 0 0 0) 0x7fd754028410 con 0x556dd94c8290 2024-03-21T10:25:43.970 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:43.964+0000 7fd76437f740 10 client.15933 fetch_fsmap finished waiting for FSMap version 30 2024-03-21T10:25:43.970 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:43.964+0000 7fd76437f740 10 client.15933 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-21T10:25:43.970 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:43.964+0000 7fd76437f740 20 client.15933 populate_metadata read hostname 'smithi047' 2024-03-21T10:25:43.970 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:43.964+0000 7fd76437f740 1 --2- 172.21.15.47:0/4169291574 >> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] conn(0x556dd9534880 0x556dd9536ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:25:43.970 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:43.964+0000 7fd76437f740 4 client.15933 mds_command: new command op to 34535 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-21T10:25:43.970 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:43.964+0000 7fd76437f740 1 -- 172.21.15.47:0/4169291574 --> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x556dd942aae0 con 0x556dd9534880 2024-03-21T10:25:43.970 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:43.965+0000 7fd75aa00640 1 --2- 172.21.15.47:0/4169291574 >> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] conn(0x556dd9534880 0x556dd9536ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:25:43.970 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:43.965+0000 7fd75aa00640 1 --2- 172.21.15.47:0/4169291574 >> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] conn(0x556dd9534880 0x556dd9536ca0 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8666c3e76ff66cd3 server_cookie=8e9dcb8af8c75503 in_seq=0 out_seq=0 2024-03-21T10:25:43.970 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:43.965+0000 7fd743e00640 10 client.15933 ms_handle_connect on v2:172.21.15.100:6838/4173168440 2024-03-21T10:25:43.971 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:43.966+0000 7fd743e00640 1 -- 172.21.15.47:0/4169291574 <== mds.0 v2:172.21.15.100:6838/4173168440 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x556dd942aae0 con 0x556dd9534880 2024-03-21T10:25:43.971 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:43.966+0000 7fd743e00640 10 client.15933 handle_command_reply: tid=0 2024-03-21T10:25:43.971 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:43.966+0000 7fd76437f740 1 client.15933 shutdown 2024-03-21T10:25:43.972 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:43.966+0000 7fd76437f740 1 -- 172.21.15.47:0/4169291574 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x556dd94c8290 msgr2=0x556dd9532980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:25:43.972 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:43.966+0000 7fd76437f740 1 --2- 172.21.15.47:0/4169291574 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x556dd94c8290 0x556dd9532980 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7fd75401f460 tx=0x7fd754004d40 comp rx=0 tx=0).stop 2024-03-21T10:25:43.972 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:43.966+0000 7fd76437f740 1 -- 172.21.15.47:0/4169291574 shutdown_connections 2024-03-21T10:25:43.972 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:43.966+0000 7fd76437f740 1 --2- 172.21.15.47:0/4169291574 >> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] conn(0x556dd9534880 0x556dd9536ca0 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:25:43.972 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:43.966+0000 7fd76437f740 1 --2- 172.21.15.47:0/4169291574 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x556dd94c8290 0x556dd9532980 unknown :-1 s=CLOSED pgs=574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:25:43.972 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:43.966+0000 7fd76437f740 1 --2- 172.21.15.47:0/4169291574 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556dd94bfc00 0x556dd94c20c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:25:43.972 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:43.966+0000 7fd76437f740 1 --2- 172.21.15.47:0/4169291574 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556dd94bef20 0x556dd94c1b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:25:43.972 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:43.966+0000 7fd76437f740 1 -- 172.21.15.47:0/4169291574 >> 172.21.15.47:0/4169291574 conn(0x556dd9486200 msgr2=0x556dd94bac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:25:43.972 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:43.966+0000 7fd743e00640 0 client.15933 ms_handle_reset on v2:172.21.15.100:6838/4173168440 2024-03-21T10:25:43.972 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:43.966+0000 7fd76437f740 1 -- 172.21.15.47:0/4169291574 shutdown_connections 2024-03-21T10:25:43.972 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:43.966+0000 7fd76437f740 1 -- 172.21.15.47:0/4169291574 wait complete. 2024-03-21T10:25:43.972 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:43.966+0000 7fd76437f740 20 client.15933 trim_cache size 0 max 16384 2024-03-21T10:25:43.973 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-03-21T10:25:43.973 INFO:teuthology.orchestra.run.smithi047.stdout: "status": "IDLE (waiting for more scrubs)", 2024-03-21T10:25:43.973 INFO:teuthology.orchestra.run.smithi047.stdout: "scrubs": { 2024-03-21T10:25:43.973 INFO:teuthology.orchestra.run.smithi047.stdout: "04dd17ee-543a-4799-b6fc-ae466b0940ad": { 2024-03-21T10:25:43.973 INFO:teuthology.orchestra.run.smithi047.stdout: "path": "/", 2024-03-21T10:25:43.973 INFO:teuthology.orchestra.run.smithi047.stdout: "tag": "04dd17ee-543a-4799-b6fc-ae466b0940ad", 2024-03-21T10:25:43.973 INFO:teuthology.orchestra.run.smithi047.stdout: "options": "recursive,force" 2024-03-21T10:25:43.973 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-21T10:25:43.973 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-21T10:25:43.973 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-03-21T10:25:43.974 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:43.967+0000 7fd761600640 1 -- 172.21.15.47:0/2713801914 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7fd73c0a31a0 msgr2=0x7fd73c0a5660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:25:43.974 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:43.967+0000 7fd761600640 1 --2- 172.21.15.47:0/2713801914 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7fd73c0a31a0 0x7fd73c0a5660 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fd7540095f0 tx=0x7fd754005f90 comp rx=0 tx=0).stop 2024-03-21T10:25:43.974 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:43.967+0000 7fd761600640 1 -- 172.21.15.47:0/2713801914 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd75c101040 msgr2=0x7fd75c19e9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:25:43.974 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:43.967+0000 7fd761600640 1 --2- 172.21.15.47:0/2713801914 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd75c101040 0x7fd75c19e9d0 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7fd74400c420 tx=0x7fd74400c8e0 comp rx=0 tx=0).stop 2024-03-21T10:25:43.974 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:43.967+0000 7fd761600640 1 -- 172.21.15.47:0/2713801914 shutdown_connections 2024-03-21T10:25:43.974 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:43.967+0000 7fd761600640 1 --2- 172.21.15.47:0/2713801914 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd75c105a40 0x7fd75c1a2d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:25:43.975 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:43.967+0000 7fd761600640 1 --2- 172.21.15.47:0/2713801914 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd75c101040 0x7fd75c19e9d0 unknown :-1 s=CLOSED pgs=570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:25:43.975 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:43.967+0000 7fd761600640 1 --2- 172.21.15.47:0/2713801914 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7fd73c0a31a0 0x7fd73c0a5660 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:25:43.975 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:43.967+0000 7fd761600640 1 --2- 172.21.15.47:0/2713801914 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd75c100720 0x7fd75c19e490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:25:43.975 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:43.967+0000 7fd761600640 1 -- 172.21.15.47:0/2713801914 >> 172.21.15.47:0/2713801914 conn(0x7fd75c077700 msgr2=0x7fd75c1034d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:25:43.975 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:43.967+0000 7fd761600640 1 -- 172.21.15.47:0/2713801914 shutdown_connections 2024-03-21T10:25:43.975 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:25:43.967+0000 7fd761600640 1 -- 172.21.15.47:0/2713801914 wait complete. 2024-03-21T10:25:43.988 INFO:tasks.cephfs.filesystem:scrub status for tag:04dd17ee-543a-4799-b6fc-ae466b0940ad - {'path': '/', 'tag': '04dd17ee-543a-4799-b6fc-ae466b0940ad', 'options': 'recursive,force'} 2024-03-21T10:25:45.463 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:25:45 smithi047 ceph-mon[31547]: scrub queued for path: / 2024-03-21T10:25:45.464 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:25:45 smithi047 ceph-mon[31547]: scrub summary: idle+waiting paths [/] 2024-03-21T10:25:45.464 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:25:45 smithi047 ceph-mon[31547]: scrub summary: active paths [/] 2024-03-21T10:25:45.464 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:25:45 smithi047 ceph-mon[31547]: scrub summary: idle+waiting paths [/] 2024-03-21T10:25:45.464 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:25:45 smithi047 ceph-mon[31547]: pgmap v15: 129 pgs: 129 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 1.7 KiB/s wr, 13 op/s 2024-03-21T10:25:45.479 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:25:45 smithi100 ceph-mon[35193]: scrub queued for path: / 2024-03-21T10:25:45.479 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:25:45 smithi100 ceph-mon[35193]: scrub summary: idle+waiting paths [/] 2024-03-21T10:25:45.479 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:25:45 smithi100 ceph-mon[35193]: scrub summary: active paths [/] 2024-03-21T10:25:45.479 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:25:45 smithi100 ceph-mon[35193]: scrub summary: idle+waiting paths [/] 2024-03-21T10:25:45.479 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:25:45 smithi100 ceph-mon[35193]: pgmap v15: 129 pgs: 129 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 1.7 KiB/s wr, 13 op/s 2024-03-21T10:25:45.490 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:25:45 smithi157 ceph-mon[35581]: scrub queued for path: / 2024-03-21T10:25:45.490 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:25:45 smithi157 ceph-mon[35581]: scrub summary: idle+waiting paths [/] 2024-03-21T10:25:45.490 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:25:45 smithi157 ceph-mon[35581]: scrub summary: active paths [/] 2024-03-21T10:25:45.490 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:25:45 smithi157 ceph-mon[35581]: scrub summary: idle+waiting paths [/] 2024-03-21T10:25:45.490 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:25:45 smithi157 ceph-mon[35581]: pgmap v15: 129 pgs: 129 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 1.7 KiB/s wr, 13 op/s 2024-03-21T10:25:47.463 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:25:47 smithi047 ceph-mon[31547]: pgmap v16: 129 pgs: 129 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 1.7 KiB/s wr, 16 op/s 2024-03-21T10:25:47.479 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:25:47 smithi100 ceph-mon[35193]: pgmap v16: 129 pgs: 129 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 1.7 KiB/s wr, 16 op/s 2024-03-21T10:25:47.489 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:25:47 smithi157 ceph-mon[35581]: pgmap v16: 129 pgs: 129 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 1.7 KiB/s wr, 16 op/s 2024-03-21T10:25:49.432 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:25:49.430+0000 7fcb1e400640 1 -- 172.21.15.47:0/76638270 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fcb18073a40 msgr2=0x7fcb18073e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:25:49.432 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:25:49.430+0000 7fcb1e400640 1 --2- 172.21.15.47:0/76638270 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fcb18073a40 0x7fcb18073e20 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7fcb0c009900 tx=0x7fcb0c02bcd0 comp rx=0 tx=0).stop 2024-03-21T10:25:49.432 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:25:49.431+0000 7fcb1e400640 1 -- 172.21.15.47:0/76638270 shutdown_connections 2024-03-21T10:25:49.432 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:25:49.431+0000 7fcb1e400640 1 --2- 172.21.15.47:0/76638270 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcb18112490 0x7fcb18114880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:25:49.432 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:25:49.431+0000 7fcb1e400640 1 --2- 172.21.15.47:0/76638270 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcb18074360 0x7fcb18111f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:25:49.432 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:25:49.431+0000 7fcb1e400640 1 --2- 172.21.15.47:0/76638270 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fcb18073a40 0x7fcb18073e20 unknown :-1 s=CLOSED pgs=575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:25:49.432 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:25:49.431+0000 7fcb1e400640 1 -- 172.21.15.47:0/76638270 >> 172.21.15.47:0/76638270 conn(0x7fcb180fcff0 msgr2=0x7fcb180ff410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:25:49.432 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:25:49.431+0000 7fcb1e400640 1 -- 172.21.15.47:0/76638270 shutdown_connections 2024-03-21T10:25:49.432 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:25:49.431+0000 7fcb1e400640 1 -- 172.21.15.47:0/76638270 wait complete. 2024-03-21T10:25:49.434 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:25:49.432+0000 7fcb1e400640 1 Processor -- start 2024-03-21T10:25:49.434 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:25:49.432+0000 7fcb1e400640 1 -- start start 2024-03-21T10:25:49.434 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:25:49.432+0000 7fcb1e400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fcb18073a40 0x7fcb1819e330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:25:49.434 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:25:49.432+0000 7fcb1e400640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcb18074360 0x7fcb1819e870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:25:49.434 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:25:49.432+0000 7fcb1e400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcb18112490 0x7fcb181a2bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:25:49.434 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:25:49.432+0000 7fcb1e400640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb18117d90 con 0x7fcb18073a40 2024-03-21T10:25:49.435 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:25:49.432+0000 7fcb1e400640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb18117c10 con 0x7fcb18112490 2024-03-21T10:25:49.435 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:25:49.432+0000 7fcb1e400640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb18117f10 con 0x7fcb18074360 2024-03-21T10:25:49.435 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:25:49.433+0000 7fcb1d000640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fcb18073a40 0x7fcb1819e330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:25:49.435 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:25:49.433+0000 7fcb1d000640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fcb18073a40 0x7fcb1819e330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:55176/0 (socket says 172.21.15.47:55176) 2024-03-21T10:25:49.435 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:25:49.433+0000 7fcb1d000640 1 -- 172.21.15.47:0/1874647489 learned_addr learned my addr 172.21.15.47:0/1874647489 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:25:49.435 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:25:49.433+0000 7fcb1da00640 1 --2- 172.21.15.47:0/1874647489 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcb18112490 0x7fcb181a2bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:25:49.435 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:25:49.433+0000 7fcb17e00640 1 --2- 172.21.15.47:0/1874647489 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcb18074360 0x7fcb1819e870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:25:49.435 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:25:49.433+0000 7fcb1d000640 1 -- 172.21.15.47:0/1874647489 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcb18074360 msgr2=0x7fcb1819e870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:25:49.435 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:25:49.433+0000 7fcb1d000640 1 --2- 172.21.15.47:0/1874647489 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcb18074360 0x7fcb1819e870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:25:49.435 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:25:49.433+0000 7fcb1d000640 1 -- 172.21.15.47:0/1874647489 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcb18112490 msgr2=0x7fcb181a2bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:25:49.435 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:25:49.433+0000 7fcb1d000640 1 --2- 172.21.15.47:0/1874647489 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcb18112490 0x7fcb181a2bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:25:49.435 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:25:49.433+0000 7fcb1d000640 1 -- 172.21.15.47:0/1874647489 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb181a3320 con 0x7fcb18073a40 2024-03-21T10:25:49.435 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:25:49.434+0000 7fcb1d000640 1 --2- 172.21.15.47:0/1874647489 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fcb18073a40 0x7fcb1819e330 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7fcb0c000c00 tx=0x7fcb0c02d890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:25:49.436 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:25:49.434+0000 7fcb15600640 1 -- 172.21.15.47:0/1874647489 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb0c043020 con 0x7fcb18073a40 2024-03-21T10:25:49.436 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:25:49.434+0000 7fcb15600640 1 -- 172.21.15.47:0/1874647489 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fcb0c02dd50 con 0x7fcb18073a40 2024-03-21T10:25:49.436 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:25:49.434+0000 7fcb15600640 1 -- 172.21.15.47:0/1874647489 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb0c003ab0 con 0x7fcb18073a40 2024-03-21T10:25:49.436 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:25:49.434+0000 7fcb1e400640 1 -- 172.21.15.47:0/1874647489 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcb181a35b0 con 0x7fcb18073a40 2024-03-21T10:25:49.436 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:25:49.434+0000 7fcb1e400640 1 -- 172.21.15.47:0/1874647489 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcb181aaea0 con 0x7fcb18073a40 2024-03-21T10:25:49.438 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:25:49.437+0000 7fcb15600640 1 -- 172.21.15.47:0/1874647489 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 25) v1 ==== 138030+0+0 (secure 0 0 0) 0x7fcb0c046020 con 0x7fcb18073a40 2024-03-21T10:25:49.438 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:25:49.438+0000 7fcb227cf740 1 Processor -- start 2024-03-21T10:25:49.438 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:25:49.438+0000 7fcb227cf740 1 -- start start 2024-03-21T10:25:49.439 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:25:49.439+0000 7fcb227cf740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a6a9f171e0 0x55a6a9f175c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:25:49.440 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:25:49.439+0000 7fcb227cf740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a6a9f17b00 0x55a6a9f20390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:25:49.440 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:25:49.439+0000 7fcb227cf740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a6a9f208d0 0x55a6a9f22cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:25:49.440 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:25:49.439+0000 7fcb227cf740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a6a9ed93b0 con 0x55a6a9f208d0 2024-03-21T10:25:49.440 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:25:49.439+0000 7fcb227cf740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a6a9e7b830 con 0x55a6a9f171e0 2024-03-21T10:25:49.440 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:25:49.439+0000 7fcb227cf740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a6a9e245d0 con 0x55a6a9f17b00 2024-03-21T10:25:49.440 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:25:49.439+0000 7fcb17e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a6a9f208d0 0x55a6a9f22cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:25:49.440 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:25:49.439+0000 7fcb17e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a6a9f208d0 0x55a6a9f22cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:55178/0 (socket says 172.21.15.47:55178) 2024-03-21T10:25:49.440 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:25:49.439+0000 7fcb1da00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a6a9f171e0 0x55a6a9f175c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:25:49.440 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:25:49.439+0000 7fcb17e00640 1 -- 172.21.15.47:0/216976720 learned_addr learned my addr 172.21.15.47:0/216976720 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:25:49.440 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:25:49.439+0000 7fcb1d000640 1 --2- 172.21.15.47:0/216976720 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a6a9f17b00 0x55a6a9f20390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:25:49.442 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:25:49.440+0000 7fcb17e00640 1 -- 172.21.15.47:0/216976720 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a6a9f17b00 msgr2=0x55a6a9f20390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:25:49.442 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:25:49.440+0000 7fcb17e00640 1 --2- 172.21.15.47:0/216976720 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a6a9f17b00 0x55a6a9f20390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:25:49.442 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:25:49.440+0000 7fcb15600640 1 --2- 172.21.15.47:0/1874647489 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7fcb000a3110 0x7fcb000a55d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:25:49.442 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:25:49.440+0000 7fcb17e00640 1 -- 172.21.15.47:0/216976720 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a6a9f171e0 msgr2=0x55a6a9f175c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:25:49.442 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:25:49.440+0000 7fcb17e00640 1 --2- 172.21.15.47:0/216976720 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a6a9f171e0 0x55a6a9f175c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:25:49.443 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:25:49.440+0000 7fcb17e00640 1 -- 172.21.15.47:0/216976720 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a6a9e3c2c0 con 0x55a6a9f208d0 2024-03-21T10:25:49.443 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:25:49.440+0000 7fcb1d000640 1 --2- 172.21.15.47:0/216976720 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a6a9f17b00 0x55a6a9f20390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T10:25:49.443 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:25:49.440+0000 7fcb15600640 1 -- 172.21.15.47:0/1874647489 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 9474+0+0 (secure 0 0 0) 0x7fcb0c0f1dc0 con 0x7fcb18073a40 2024-03-21T10:25:49.443 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:25:49.440+0000 7fcb1da00640 1 --2- 172.21.15.47:0/1874647489 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7fcb000a3110 0x7fcb000a55d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:25:49.443 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:25:49.440+0000 7fcb17e00640 1 --2- 172.21.15.47:0/216976720 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a6a9f208d0 0x55a6a9f22cc0 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7fcb0400bc90 tx=0x7fcb0400f730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ffcceb7b4dc01cb9 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:25:49.443 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:25:49.440+0000 7fcb0a000640 1 -- 172.21.15.47:0/216976720 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb04015070 con 0x55a6a9f208d0 2024-03-21T10:25:49.443 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:25:49.441+0000 7fcb0a000640 1 -- 172.21.15.47:0/216976720 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fcb0401a910 con 0x55a6a9f208d0 2024-03-21T10:25:49.443 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:25:49.441+0000 7fcb0a000640 1 -- 172.21.15.47:0/216976720 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb04011810 con 0x55a6a9f208d0 2024-03-21T10:25:49.443 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:25:49.441+0000 7fcb227cf740 1 -- 172.21.15.47:0/216976720 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a6a9f208d0 msgr2=0x55a6a9f22cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:25:49.443 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:25:49.441+0000 7fcb1da00640 1 --2- 172.21.15.47:0/1874647489 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7fcb000a3110 0x7fcb000a55d0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fcb1806aaf0 tx=0x7fcb10004a40 comp rx=0 tx=0).ready entity=mgr.15687 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:25:49.443 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:25:49.441+0000 7fcb227cf740 1 --2- 172.21.15.47:0/216976720 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a6a9f208d0 0x55a6a9f22cc0 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7fcb0400bc90 tx=0x7fcb0400f730 comp rx=0 tx=0).stop 2024-03-21T10:25:49.443 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:25:49.441+0000 7fcb227cf740 1 -- 172.21.15.47:0/216976720 shutdown_connections 2024-03-21T10:25:49.443 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:25:49.441+0000 7fcb227cf740 1 --2- 172.21.15.47:0/216976720 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a6a9f208d0 0x55a6a9f22cc0 unknown :-1 s=CLOSED pgs=577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:25:49.443 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:25:49.441+0000 7fcb227cf740 1 --2- 172.21.15.47:0/216976720 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a6a9f17b00 0x55a6a9f20390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:25:49.444 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:25:49.441+0000 7fcb227cf740 1 --2- 172.21.15.47:0/216976720 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a6a9f171e0 0x55a6a9f175c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:25:49.444 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:25:49.441+0000 7fcb227cf740 1 -- 172.21.15.47:0/216976720 >> 172.21.15.47:0/216976720 conn(0x55a6a9f11b00 msgr2=0x55a6a9f11ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:25:49.444 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:25:49.442+0000 7fcb227cf740 1 -- 172.21.15.47:0/216976720 shutdown_connections 2024-03-21T10:25:49.444 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:25:49.442+0000 7fcb227cf740 1 -- 172.21.15.47:0/216976720 wait complete. 2024-03-21T10:25:49.447 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:25:49.442+0000 7fcb227cf740 1 -- start start 2024-03-21T10:25:49.447 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:25:49.442+0000 7fcb227cf740 1 Processor -- start 2024-03-21T10:25:49.448 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:25:49.443+0000 7fcb227cf740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a6a9f171e0 0x55a6a9f1d940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:25:49.448 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:25:49.443+0000 7fcb227cf740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a6a9f17b00 0x55a6a9f1de80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:25:49.448 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:25:49.443+0000 7fcb227cf740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a6a9f208d0 0x55a6a9f1e3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:25:49.448 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:25:49.443+0000 7fcb227cf740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a6a9e37a20 con 0x55a6a9f17b00 2024-03-21T10:25:49.448 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:25:49.443+0000 7fcb227cf740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a6a9e778d0 con 0x55a6a9f171e0 2024-03-21T10:25:49.448 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:25:49.443+0000 7fcb227cf740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a6a9f23500 con 0x55a6a9f208d0 2024-03-21T10:25:49.448 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:25:49.443+0000 7fcb227cf740 10 client.0 _ll_register_callbacks cb 0x55a6a9e741c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T10:25:49.448 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:25:49.443+0000 7fcb17e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a6a9f17b00 0x55a6a9f1de80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:25:49.448 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:25:49.443+0000 7fcb17e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a6a9f17b00 0x55a6a9f1de80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:55194/0 (socket says 172.21.15.47:55194) 2024-03-21T10:25:49.448 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:25:49.443+0000 7fcb17e00640 1 -- 172.21.15.47:0/2940938904 learned_addr learned my addr 172.21.15.47:0/2940938904 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:25:49.448 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:25:49.443+0000 7fcb1d000640 1 --2- 172.21.15.47:0/2940938904 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a6a9f171e0 0x55a6a9f1d940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:25:49.448 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:25:49.443+0000 7fcb1da00640 1 --2- 172.21.15.47:0/2940938904 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a6a9f208d0 0x55a6a9f1e3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:25:49.448 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:25:49.443+0000 7fcb17e00640 1 -- 172.21.15.47:0/2940938904 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a6a9f208d0 msgr2=0x55a6a9f1e3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:25:49.449 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:25:49.443+0000 7fcb17e00640 1 --2- 172.21.15.47:0/2940938904 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a6a9f208d0 0x55a6a9f1e3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:25:49.449 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:25:49.443+0000 7fcb17e00640 1 -- 172.21.15.47:0/2940938904 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a6a9f171e0 msgr2=0x55a6a9f1d940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:25:49.449 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:25:49.443+0000 7fcb17e00640 1 --2- 172.21.15.47:0/2940938904 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a6a9f171e0 0x55a6a9f1d940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:25:49.449 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:25:49.443+0000 7fcb17e00640 1 -- 172.21.15.47:0/2940938904 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55a6a9b90210 con 0x55a6a9f17b00 2024-03-21T10:25:49.449 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:25:49.443+0000 7fcb1d000640 1 --2- 172.21.15.47:0/2940938904 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a6a9f171e0 0x55a6a9f1d940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T10:25:49.449 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:25:49.443+0000 7fcb1da00640 1 --2- 172.21.15.47:0/2940938904 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a6a9f208d0 0x55a6a9f1e3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T10:25:49.449 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:25:49.444+0000 7fcb17e00640 1 --2- 172.21.15.47:0/2940938904 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a6a9f17b00 0x55a6a9f1de80 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7fcb04006010 tx=0x7fcb0400b9d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=31ac5d1f0f7a7e36 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:25:49.449 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:25:49.444+0000 7fcafea00640 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-03-21T10:25:49.449 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:25:49.444+0000 7fcafea00640 1 -- 172.21.15.47:0/2940938904 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb04028060 con 0x55a6a9f17b00 2024-03-21T10:25:49.449 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:25:49.444+0000 7fcb227cf740 1 -- 172.21.15.47:0/2940938904 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fcb0401a910 con 0x55a6a9f17b00 2024-03-21T10:25:49.449 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:25:49.444+0000 7fcafea00640 1 -- 172.21.15.47:0/2940938904 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 9474+0+0 (secure 0 0 0) 0x7fcb0401aac0 con 0x55a6a9f17b00 2024-03-21T10:25:49.449 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:25:49.444+0000 7fcafea00640 1 -- 172.21.15.47:0/2940938904 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7fcb0402f050 con 0x55a6a9f17b00 2024-03-21T10:25:49.449 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:25:49.444+0000 7fcb227cf740 10 client.15945 fetch_fsmap learned FSMap version 30 2024-03-21T10:25:49.449 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:25:49.444+0000 7fcb227cf740 1 -- 172.21.15.47:0/2940938904 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x55a6a9c5c620 con 0x55a6a9f17b00 2024-03-21T10:25:49.450 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:25:49.445+0000 7fcafea00640 1 -- 172.21.15.47:0/2940938904 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fcb0401a910 con 0x55a6a9f17b00 2024-03-21T10:25:49.450 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:25:49.445+0000 7fcafea00640 1 -- 172.21.15.47:0/2940938904 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 30) v1 ==== 7942+0+0 (secure 0 0 0) 0x7fcb04004850 con 0x55a6a9f17b00 2024-03-21T10:25:49.450 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:25:49.445+0000 7fcb227cf740 10 client.15945 fetch_fsmap finished waiting for FSMap version 30 2024-03-21T10:25:49.450 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:25:49.445+0000 7fcb227cf740 10 client.15945 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-21T10:25:49.450 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:25:49.445+0000 7fcb227cf740 20 client.15945 populate_metadata read hostname 'smithi047' 2024-03-21T10:25:49.450 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:25:49.445+0000 7fcb227cf740 1 --2- 172.21.15.47:0/2940938904 >> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] conn(0x55a6a9f30810 0x55a6a9f32c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:25:49.450 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:25:49.445+0000 7fcb227cf740 4 client.15945 mds_command: new command op to 34535 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T10:25:49.450 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:25:49.445+0000 7fcb227cf740 1 -- 172.21.15.47:0/2940938904 --> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55a6a9e88d70 con 0x55a6a9f30810 2024-03-21T10:25:49.450 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:25:49.446+0000 7fcb1d000640 1 --2- 172.21.15.47:0/2940938904 >> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] conn(0x55a6a9f30810 0x55a6a9f32c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:25:49.450 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:25:49.447+0000 7fcb1d000640 1 --2- 172.21.15.47:0/2940938904 >> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] conn(0x55a6a9f30810 0x55a6a9f32c00 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1dde34bf57399e25 server_cookie=c725c89e66987615 in_seq=0 out_seq=0 2024-03-21T10:25:49.450 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:25:49.447+0000 7fcafea00640 10 client.15945 ms_handle_connect on v2:172.21.15.100:6838/4173168440 2024-03-21T10:25:49.451 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:25:49.448+0000 7fcafea00640 1 -- 172.21.15.47:0/2940938904 <== mds.0 v2:172.21.15.100:6838/4173168440 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55a6a9e88d70 con 0x55a6a9f30810 2024-03-21T10:25:49.451 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:25:49.448+0000 7fcafea00640 10 client.15945 handle_command_reply: tid=0 2024-03-21T10:25:49.452 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:25:49.448+0000 7fcb227cf740 1 client.15945 shutdown 2024-03-21T10:25:49.452 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:25:49.448+0000 7fcb227cf740 1 -- 172.21.15.47:0/2940938904 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a6a9f17b00 msgr2=0x55a6a9f1de80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:25:49.452 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:25:49.448+0000 7fcb227cf740 1 --2- 172.21.15.47:0/2940938904 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a6a9f17b00 0x55a6a9f1de80 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7fcb04006010 tx=0x7fcb0400b9d0 comp rx=0 tx=0).stop 2024-03-21T10:25:49.452 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:25:49.448+0000 7fcb227cf740 1 -- 172.21.15.47:0/2940938904 shutdown_connections 2024-03-21T10:25:49.452 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:25:49.448+0000 7fcb227cf740 1 --2- 172.21.15.47:0/2940938904 >> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] conn(0x55a6a9f30810 0x55a6a9f32c00 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:25:49.452 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:25:49.448+0000 7fcb227cf740 1 --2- 172.21.15.47:0/2940938904 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a6a9f208d0 0x55a6a9f1e3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:25:49.452 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:25:49.448+0000 7fcafea00640 0 client.15945 ms_handle_reset on v2:172.21.15.100:6838/4173168440 2024-03-21T10:25:49.452 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:25:49.448+0000 7fcb227cf740 1 --2- 172.21.15.47:0/2940938904 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a6a9f17b00 0x55a6a9f1de80 unknown :-1 s=CLOSED pgs=578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:25:49.452 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:25:49.448+0000 7fcb227cf740 1 --2- 172.21.15.47:0/2940938904 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a6a9f171e0 0x55a6a9f1d940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:25:49.452 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:25:49.448+0000 7fcb227cf740 1 -- 172.21.15.47:0/2940938904 >> 172.21.15.47:0/2940938904 conn(0x55a6a9f11b00 msgr2=0x55a6a9f13850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:25:49.452 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:25:49.449+0000 7fcb227cf740 1 -- 172.21.15.47:0/2940938904 shutdown_connections 2024-03-21T10:25:49.452 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:25:49.449+0000 7fcb227cf740 1 -- 172.21.15.47:0/2940938904 wait complete. 2024-03-21T10:25:49.452 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:25:49.449+0000 7fcb227cf740 20 client.15945 trim_cache size 0 max 16384 2024-03-21T10:25:49.463 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:25:49 smithi047 ceph-mon[31547]: pgmap v17: 129 pgs: 129 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 1.7 KiB/s wr, 17 op/s 2024-03-21T10:25:49.463 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:25:49 smithi047 ceph-mon[31547]: scrub summary: idle 2024-03-21T10:25:49.474 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:25:49.474+0000 7fcb227cf740 1 Processor -- start 2024-03-21T10:25:49.475 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:25:49.474+0000 7fcb227cf740 1 -- start start 2024-03-21T10:25:49.475 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:25:49.474+0000 7fcb227cf740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a6a9f171e0 0x55a6a9f20070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:25:49.475 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:25:49.474+0000 7fcb227cf740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a6a9f17b00 0x55a6a9f13610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:25:49.475 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:25:49.474+0000 7fcb227cf740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a6a9f208d0 0x55a6a9f1aba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:25:49.475 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:25:49.474+0000 7fcb227cf740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb181b4d80 con 0x55a6a9f171e0 2024-03-21T10:25:49.475 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:25:49.474+0000 7fcb227cf740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb181b6700 con 0x55a6a9f17b00 2024-03-21T10:25:49.475 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:25:49.474+0000 7fcb227cf740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb181b53e0 con 0x55a6a9f208d0 2024-03-21T10:25:49.477 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:25:49.474+0000 7fcb1d000640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a6a9f171e0 0x55a6a9f20070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:25:49.477 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:25:49.475+0000 7fcb1d000640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a6a9f171e0 0x55a6a9f20070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:55208/0 (socket says 172.21.15.47:55208) 2024-03-21T10:25:49.477 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:25:49.475+0000 7fcb1d000640 1 -- 172.21.15.47:0/3867981731 learned_addr learned my addr 172.21.15.47:0/3867981731 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:25:49.477 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:25:49.475+0000 7fcb17e00640 1 --2- 172.21.15.47:0/3867981731 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a6a9f17b00 0x55a6a9f13610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:25:49.477 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:25:49.475+0000 7fcb1d000640 1 -- 172.21.15.47:0/3867981731 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a6a9f208d0 msgr2=0x55a6a9f1aba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:25:49.477 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:25:49.475+0000 7fcb1da00640 1 --2- 172.21.15.47:0/3867981731 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a6a9f208d0 0x55a6a9f1aba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:25:49.477 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:25:49.475+0000 7fcb1d000640 1 --2- 172.21.15.47:0/3867981731 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a6a9f208d0 0x55a6a9f1aba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:25:49.477 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:25:49.475+0000 7fcb1d000640 1 -- 172.21.15.47:0/3867981731 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a6a9f17b00 msgr2=0x55a6a9f13610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:25:49.478 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:25:49.475+0000 7fcb1d000640 1 --2- 172.21.15.47:0/3867981731 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a6a9f17b00 0x55a6a9f13610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:25:49.478 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:25:49.475+0000 7fcb1d000640 1 -- 172.21.15.47:0/3867981731 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a6a9f1be90 con 0x55a6a9f171e0 2024-03-21T10:25:49.478 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:25:49.475+0000 7fcb17e00640 1 --2- 172.21.15.47:0/3867981731 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a6a9f17b00 0x55a6a9f13610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T10:25:49.478 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:25:49.475+0000 7fcb1d000640 1 --2- 172.21.15.47:0/3867981731 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a6a9f171e0 0x55a6a9f20070 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7fcb0c006010 tx=0x7fcb0c0de040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d18eeafb3591c4ff server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:25:49.478 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:25:49.476+0000 7fcb09600640 1 -- 172.21.15.47:0/3867981731 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb0c0bd870 con 0x55a6a9f171e0 2024-03-21T10:25:49.478 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:25:49.476+0000 7fcb09600640 1 -- 172.21.15.47:0/3867981731 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fcb0c0be430 con 0x55a6a9f171e0 2024-03-21T10:25:49.478 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:25:49.476+0000 7fcb09600640 1 -- 172.21.15.47:0/3867981731 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb0c0bd690 con 0x55a6a9f171e0 2024-03-21T10:25:49.478 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:25:49.476+0000 7fcb227cf740 1 -- 172.21.15.47:0/3867981731 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a6a9f171e0 msgr2=0x55a6a9f20070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:25:49.478 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:25:49.476+0000 7fcb227cf740 1 --2- 172.21.15.47:0/3867981731 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a6a9f171e0 0x55a6a9f20070 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7fcb0c006010 tx=0x7fcb0c0de040 comp rx=0 tx=0).stop 2024-03-21T10:25:49.478 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:25:49.476+0000 7fcb227cf740 1 -- 172.21.15.47:0/3867981731 shutdown_connections 2024-03-21T10:25:49.478 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:25:49.476+0000 7fcb227cf740 1 --2- 172.21.15.47:0/3867981731 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a6a9f208d0 0x55a6a9f1aba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:25:49.478 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:25:49.476+0000 7fcb227cf740 1 --2- 172.21.15.47:0/3867981731 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a6a9f17b00 0x55a6a9f13610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:25:49.478 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:25:49.476+0000 7fcb227cf740 1 --2- 172.21.15.47:0/3867981731 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a6a9f171e0 0x55a6a9f20070 unknown :-1 s=CLOSED pgs=579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:25:49.479 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:25:49.476+0000 7fcb227cf740 1 -- 172.21.15.47:0/3867981731 >> 172.21.15.47:0/3867981731 conn(0x55a6a9f11b00 msgr2=0x55a6a9f12f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:25:49.479 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:25:49.477+0000 7fcb227cf740 1 -- 172.21.15.47:0/3867981731 shutdown_connections 2024-03-21T10:25:49.479 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:25:49.477+0000 7fcb227cf740 1 -- 172.21.15.47:0/3867981731 wait complete. 2024-03-21T10:25:49.479 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:25:49 smithi100 ceph-mon[35193]: pgmap v17: 129 pgs: 129 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 1.7 KiB/s wr, 17 op/s 2024-03-21T10:25:49.479 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:25:49 smithi100 ceph-mon[35193]: scrub summary: idle 2024-03-21T10:25:49.482 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:25:49.477+0000 7fcb227cf740 1 -- start start 2024-03-21T10:25:49.482 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:25:49.477+0000 7fcb227cf740 1 Processor -- start 2024-03-21T10:25:49.482 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:25:49.478+0000 7fcb227cf740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a6a9f171e0 0x55a6a9f33770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:25:49.482 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:25:49.478+0000 7fcb227cf740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a6a9f17b00 0x55a6a9f33cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:25:49.482 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:25:49.478+0000 7fcb227cf740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a6a9f208d0 0x55a6a9f8ad40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:25:49.483 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:25:49.478+0000 7fcb227cf740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb181b60a0 con 0x55a6a9f171e0 2024-03-21T10:25:49.483 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:25:49.478+0000 7fcb227cf740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb181b5a40 con 0x55a6a9f208d0 2024-03-21T10:25:49.483 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:25:49.478+0000 7fcb227cf740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb181b2f40 con 0x55a6a9f17b00 2024-03-21T10:25:49.483 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:25:49.478+0000 7fcb227cf740 10 client.0 _ll_register_callbacks cb 0x7fcb10009a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T10:25:49.483 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:25:49.478+0000 7fcb1d000640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a6a9f171e0 0x55a6a9f33770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:25:49.483 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:25:49.478+0000 7fcb1d000640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a6a9f171e0 0x55a6a9f33770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:55218/0 (socket says 172.21.15.47:55218) 2024-03-21T10:25:49.483 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:25:49.478+0000 7fcb1d000640 1 -- 172.21.15.47:0/2258969764 learned_addr learned my addr 172.21.15.47:0/2258969764 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:25:49.483 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:25:49.478+0000 7fcb1da00640 1 --2- 172.21.15.47:0/2258969764 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a6a9f208d0 0x55a6a9f8ad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:25:49.483 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:25:49.478+0000 7fcb17e00640 1 --2- 172.21.15.47:0/2258969764 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a6a9f17b00 0x55a6a9f33cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:25:49.483 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:25:49.479+0000 7fcb1d000640 1 -- 172.21.15.47:0/2258969764 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a6a9f17b00 msgr2=0x55a6a9f33cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:25:49.483 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:25:49.479+0000 7fcb1d000640 1 --2- 172.21.15.47:0/2258969764 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a6a9f17b00 0x55a6a9f33cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:25:49.483 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:25:49.479+0000 7fcb1d000640 1 -- 172.21.15.47:0/2258969764 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a6a9f208d0 msgr2=0x55a6a9f8ad40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:25:49.483 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:25:49.479+0000 7fcb1d000640 1 --2- 172.21.15.47:0/2258969764 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a6a9f208d0 0x55a6a9f8ad40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:25:49.483 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:25:49.479+0000 7fcb1d000640 1 -- 172.21.15.47:0/2258969764 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55a6a9f8b280 con 0x55a6a9f171e0 2024-03-21T10:25:49.484 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:25:49.479+0000 7fcb1d000640 1 --2- 172.21.15.47:0/2258969764 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a6a9f171e0 0x55a6a9f33770 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7fcb0c0bdcc0 tx=0x7fcb0c02c350 comp rx=0 tx=0).ready entity=mon.0 client_cookie=14611d66161e080a server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:25:49.484 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:25:49.479+0000 7fcafea00640 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-03-21T10:25:49.484 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:25:49.479+0000 7fcafea00640 1 -- 172.21.15.47:0/2258969764 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb0c0dc020 con 0x55a6a9f171e0 2024-03-21T10:25:49.484 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:25:49.479+0000 7fcafea00640 1 -- 172.21.15.47:0/2258969764 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 9474+0+0 (secure 0 0 0) 0x7fcb0c0bdf90 con 0x55a6a9f171e0 2024-03-21T10:25:49.484 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:25:49.479+0000 7fcb227cf740 1 -- 172.21.15.47:0/2258969764 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fcb0c0be430 con 0x55a6a9f171e0 2024-03-21T10:25:49.484 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:25:49.479+0000 7fcafea00640 1 -- 172.21.15.47:0/2258969764 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7fcb0c003e50 con 0x55a6a9f171e0 2024-03-21T10:25:49.484 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:25:49.479+0000 7fcb227cf740 10 client.15951 fetch_fsmap learned FSMap version 30 2024-03-21T10:25:49.484 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:25:49.479+0000 7fcb227cf740 1 -- 172.21.15.47:0/2258969764 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x55a6a9f8c540 con 0x55a6a9f171e0 2024-03-21T10:25:49.484 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:25:49.480+0000 7fcafea00640 1 -- 172.21.15.47:0/2258969764 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fcb0c0be430 con 0x55a6a9f171e0 2024-03-21T10:25:49.484 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:25:49.480+0000 7fcafea00640 1 -- 172.21.15.47:0/2258969764 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 30) v1 ==== 7942+0+0 (secure 0 0 0) 0x7fcb0c0f5020 con 0x55a6a9f171e0 2024-03-21T10:25:49.484 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:25:49.480+0000 7fcb227cf740 10 client.15951 fetch_fsmap finished waiting for FSMap version 30 2024-03-21T10:25:49.484 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:25:49.480+0000 7fcb227cf740 10 client.15951 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-21T10:25:49.484 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:25:49.480+0000 7fcb227cf740 20 client.15951 populate_metadata read hostname 'smithi047' 2024-03-21T10:25:49.484 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:25:49.480+0000 7fcb227cf740 1 --2- 172.21.15.47:0/2258969764 >> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] conn(0x55a6a9f8cc10 0x55a6a9f8f000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:25:49.485 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:25:49.480+0000 7fcb227cf740 4 client.15951 mds_command: new command op to 34535 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-21T10:25:49.485 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:25:49.480+0000 7fcb227cf740 1 -- 172.21.15.47:0/2258969764 --> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55a6a9e830c0 con 0x55a6a9f8cc10 2024-03-21T10:25:49.485 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:25:49.480+0000 7fcb17e00640 1 --2- 172.21.15.47:0/2258969764 >> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] conn(0x55a6a9f8cc10 0x55a6a9f8f000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:25:49.485 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:25:49.481+0000 7fcb17e00640 1 --2- 172.21.15.47:0/2258969764 >> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] conn(0x55a6a9f8cc10 0x55a6a9f8f000 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e114fd724dc651f7 server_cookie=452f88025257a424 in_seq=0 out_seq=0 2024-03-21T10:25:49.485 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:25:49.481+0000 7fcafea00640 10 client.15951 ms_handle_connect on v2:172.21.15.100:6838/4173168440 2024-03-21T10:25:49.489 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:25:49 smithi157 ceph-mon[35581]: pgmap v17: 129 pgs: 129 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 1.7 KiB/s wr, 17 op/s 2024-03-21T10:25:49.489 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:25:49 smithi157 ceph-mon[35581]: scrub summary: idle 2024-03-21T10:25:49.493 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:25:49.492+0000 7fcafea00640 1 -- 172.21.15.47:0/2258969764 <== mds.0 v2:172.21.15.100:6838/4173168440 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55a6a9e830c0 con 0x55a6a9f8cc10 2024-03-21T10:25:49.493 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:25:49.492+0000 7fcafea00640 10 client.15951 handle_command_reply: tid=0 2024-03-21T10:25:49.493 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:25:49.492+0000 7fcb227cf740 1 client.15951 shutdown 2024-03-21T10:25:49.493 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:25:49.492+0000 7fcb227cf740 1 -- 172.21.15.47:0/2258969764 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a6a9f171e0 msgr2=0x55a6a9f33770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:25:49.493 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:25:49.492+0000 7fcb227cf740 1 --2- 172.21.15.47:0/2258969764 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a6a9f171e0 0x55a6a9f33770 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7fcb0c0bdcc0 tx=0x7fcb0c02c350 comp rx=0 tx=0).stop 2024-03-21T10:25:49.494 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:25:49.492+0000 7fcb227cf740 1 -- 172.21.15.47:0/2258969764 shutdown_connections 2024-03-21T10:25:49.494 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:25:49.492+0000 7fcb227cf740 1 --2- 172.21.15.47:0/2258969764 >> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] conn(0x55a6a9f8cc10 0x55a6a9f8f000 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:25:49.494 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:25:49.493+0000 7fcb227cf740 1 --2- 172.21.15.47:0/2258969764 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a6a9f208d0 0x55a6a9f8ad40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:25:49.494 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:25:49.493+0000 7fcb227cf740 1 --2- 172.21.15.47:0/2258969764 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a6a9f17b00 0x55a6a9f33cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:25:49.494 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:25:49.493+0000 7fcb227cf740 1 --2- 172.21.15.47:0/2258969764 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a6a9f171e0 0x55a6a9f33770 unknown :-1 s=CLOSED pgs=580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:25:49.494 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:25:49.493+0000 7fcafea00640 0 client.15951 ms_handle_reset on v2:172.21.15.100:6838/4173168440 2024-03-21T10:25:49.494 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:25:49.493+0000 7fcb227cf740 1 -- 172.21.15.47:0/2258969764 >> 172.21.15.47:0/2258969764 conn(0x55a6a9f11b00 msgr2=0x55a6a9f2bfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:25:49.494 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:25:49.493+0000 7fcb227cf740 1 -- 172.21.15.47:0/2258969764 shutdown_connections 2024-03-21T10:25:49.494 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:25:49.493+0000 7fcb227cf740 1 -- 172.21.15.47:0/2258969764 wait complete. 2024-03-21T10:25:49.494 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:25:49.493+0000 7fcb227cf740 20 client.15951 trim_cache size 0 max 16384 2024-03-21T10:25:49.495 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout:{ 2024-03-21T10:25:49.495 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout: "message": "", 2024-03-21T10:25:49.495 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout: "return_code": 0 2024-03-21T10:25:49.495 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout:} 2024-03-21T10:25:49.496 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:25:49.494+0000 7fcb1e400640 1 -- 172.21.15.47:0/1874647489 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7fcb000a3110 msgr2=0x7fcb000a55d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:25:49.496 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:25:49.494+0000 7fcb1e400640 1 --2- 172.21.15.47:0/1874647489 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7fcb000a3110 0x7fcb000a55d0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fcb1806aaf0 tx=0x7fcb10004a40 comp rx=0 tx=0).stop 2024-03-21T10:25:49.496 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:25:49.494+0000 7fcb1e400640 1 -- 172.21.15.47:0/1874647489 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fcb18073a40 msgr2=0x7fcb1819e330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:25:49.496 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:25:49.494+0000 7fcb1e400640 1 --2- 172.21.15.47:0/1874647489 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fcb18073a40 0x7fcb1819e330 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7fcb0c000c00 tx=0x7fcb0c02d890 comp rx=0 tx=0).stop 2024-03-21T10:25:49.496 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:25:49.494+0000 7fcb1e400640 1 -- 172.21.15.47:0/1874647489 shutdown_connections 2024-03-21T10:25:49.496 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:25:49.494+0000 7fcb1e400640 1 --2- 172.21.15.47:0/1874647489 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7fcb000a3110 0x7fcb000a55d0 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:25:49.496 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:25:49.494+0000 7fcb1e400640 1 --2- 172.21.15.47:0/1874647489 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcb18112490 0x7fcb181a2bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:25:49.496 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:25:49.494+0000 7fcb1e400640 1 --2- 172.21.15.47:0/1874647489 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcb18074360 0x7fcb1819e870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:25:49.496 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:25:49.494+0000 7fcb1e400640 1 --2- 172.21.15.47:0/1874647489 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fcb18073a40 0x7fcb1819e330 unknown :-1 s=CLOSED pgs=576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:25:49.496 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:25:49.494+0000 7fcb1e400640 1 -- 172.21.15.47:0/1874647489 >> 172.21.15.47:0/1874647489 conn(0x7fcb180fcff0 msgr2=0x7fcb180fdda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:25:49.496 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:25:49.494+0000 7fcb1e400640 1 -- 172.21.15.47:0/1874647489 shutdown_connections 2024-03-21T10:25:49.496 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:25:49.494+0000 7fcb1e400640 1 -- 172.21.15.47:0/1874647489 wait complete. 2024-03-21T10:25:51.463 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:25:51 smithi047 ceph-mon[31547]: pgmap v18: 129 pgs: 129 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 1.7 KiB/s wr, 14 op/s 2024-03-21T10:25:51.479 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:25:51 smithi100 ceph-mon[35193]: pgmap v18: 129 pgs: 129 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 1.7 KiB/s wr, 14 op/s 2024-03-21T10:25:51.489 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:25:51 smithi157 ceph-mon[35581]: pgmap v18: 129 pgs: 129 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 1.7 KiB/s wr, 14 op/s 2024-03-21T10:25:53.359 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:25:53 smithi157 ceph-mon[35581]: pgmap v19: 129 pgs: 129 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 2.2 KiB/s wr, 19 op/s 2024-03-21T10:25:53.463 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:25:53 smithi047 ceph-mon[31547]: pgmap v19: 129 pgs: 129 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 2.2 KiB/s wr, 19 op/s 2024-03-21T10:25:53.479 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:25:53 smithi100 ceph-mon[35193]: pgmap v19: 129 pgs: 129 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 2.2 KiB/s wr, 19 op/s 2024-03-21T10:25:55.463 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:25:55 smithi047 ceph-mon[31547]: pgmap v20: 129 pgs: 129 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 426 B/s wr, 10 op/s 2024-03-21T10:25:55.478 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:25:55 smithi100 ceph-mon[35193]: pgmap v20: 129 pgs: 129 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 426 B/s wr, 10 op/s 2024-03-21T10:25:55.489 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:25:55 smithi157 ceph-mon[35581]: pgmap v20: 129 pgs: 129 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 426 B/s wr, 10 op/s 2024-03-21T10:25:57.463 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:25:57 smithi047 ceph-mon[31547]: pgmap v21: 129 pgs: 129 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 426 B/s wr, 13 op/s 2024-03-21T10:25:57.479 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:25:57 smithi100 ceph-mon[35193]: pgmap v21: 129 pgs: 129 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 426 B/s wr, 13 op/s 2024-03-21T10:25:57.489 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:25:57 smithi157 ceph-mon[35581]: pgmap v21: 129 pgs: 129 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 426 B/s wr, 13 op/s 2024-03-21T10:25:59.463 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:25:59 smithi047 ceph-mon[31547]: pgmap v22: 129 pgs: 129 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 426 B/s wr, 10 op/s 2024-03-21T10:25:59.478 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:25:59 smithi100 ceph-mon[35193]: pgmap v22: 129 pgs: 129 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 426 B/s wr, 10 op/s 2024-03-21T10:25:59.489 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:25:59 smithi157 ceph-mon[35581]: pgmap v22: 129 pgs: 129 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 426 B/s wr, 10 op/s 2024-03-21T10:26:01.100 INFO:journalctl@ceph.mgr.x.smithi047.stdout:Mar 21 10:26:00 smithi047 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-mgr-x[31773]: 2024-03-21T10:26:00.754+0000 7ffa4ca00700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-21T10:26:01.101 INFO:journalctl@ceph.mgr.x.smithi047.stdout:Mar 21 10:26:00 smithi047 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-mgr-x[31773]: 2024-03-21T10:26:00.754+0000 7ffa4ca00700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-21T10:26:01.101 INFO:journalctl@ceph.mgr.x.smithi047.stdout:Mar 21 10:26:00 smithi047 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-mgr-x[31773]: 2024-03-21T10:26:00.754+0000 7ffa4ca00700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-21T10:26:01.101 INFO:journalctl@ceph.mgr.x.smithi047.stdout:Mar 21 10:26:00 smithi047 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-mgr-x[31773]: 2024-03-21T10:26:00.754+0000 7ffa4ca00700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-21T10:26:01.101 INFO:journalctl@ceph.mgr.x.smithi047.stdout:Mar 21 10:26:00 smithi047 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-mgr-x[31773]: 2024-03-21T10:26:00.754+0000 7ffa4ca00700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-21T10:26:01.463 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:26:01 smithi047 ceph-mon[31547]: pgmap v23: 129 pgs: 129 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 426 B/s wr, 9 op/s 2024-03-21T10:26:01.479 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:26:01 smithi100 ceph-mon[35193]: pgmap v23: 129 pgs: 129 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 426 B/s wr, 9 op/s 2024-03-21T10:26:01.489 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:26:01 smithi157 ceph-mon[35581]: pgmap v23: 129 pgs: 129 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 426 B/s wr, 9 op/s 2024-03-21T10:26:02.637 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:02.636+0000 7f0a91800640 1 -- 172.21.15.47:0/1125705641 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0a8c100d60 msgr2=0x7f0a8c105220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:26:02.637 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:02.636+0000 7f0a91800640 1 --2- 172.21.15.47:0/1125705641 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0a8c100d60 0x7f0a8c105220 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7f0a78009900 tx=0x7f0a7802bd10 comp rx=0 tx=0).stop 2024-03-21T10:26:02.637 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:02.636+0000 7f0a91800640 1 -- 172.21.15.47:0/1125705641 shutdown_connections 2024-03-21T10:26:02.637 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:02.636+0000 7f0a91800640 1 --2- 172.21.15.47:0/1125705641 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0a8c105840 0x7f0a8c105cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:26:02.637 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:02.636+0000 7f0a91800640 1 --2- 172.21.15.47:0/1125705641 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0a8c100d60 0x7f0a8c105220 unknown :-1 s=CLOSED pgs=583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:26:02.637 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:02.636+0000 7f0a91800640 1 --2- 172.21.15.47:0/1125705641 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0a8c100440 0x7f0a8c100820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:26:02.638 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:02.636+0000 7f0a91800640 1 -- 172.21.15.47:0/1125705641 >> 172.21.15.47:0/1125705641 conn(0x7f0a8c0fbeb0 msgr2=0x7f0a8c0fe2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:26:02.638 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:02.636+0000 7f0a91800640 1 -- 172.21.15.47:0/1125705641 shutdown_connections 2024-03-21T10:26:02.638 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:02.637+0000 7f0a91800640 1 -- 172.21.15.47:0/1125705641 wait complete. 2024-03-21T10:26:02.638 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:02.638+0000 7f0a91800640 1 Processor -- start 2024-03-21T10:26:02.638 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:02.638+0000 7f0a91800640 1 -- start start 2024-03-21T10:26:02.639 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:02.639+0000 7f0a91800640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0a8c100440 0x7f0a8c19e2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:26:02.639 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:02.639+0000 7f0a91800640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0a8c100d60 0x7f0a8c19e810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:26:02.640 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:02.639+0000 7f0a91800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0a8c105840 0x7f0a8c1a2b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:26:02.640 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:02.639+0000 7f0a91800640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a8c117d30 con 0x7f0a8c100d60 2024-03-21T10:26:02.640 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:02.639+0000 7f0a91800640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a8c117bb0 con 0x7f0a8c105840 2024-03-21T10:26:02.640 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:02.639+0000 7f0a91800640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a8c117eb0 con 0x7f0a8c100440 2024-03-21T10:26:02.640 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:02.639+0000 7f0a8b400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0a8c100d60 0x7f0a8c19e810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:26:02.640 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:02.639+0000 7f0a8b400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0a8c100d60 0x7f0a8c19e810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:43032/0 (socket says 172.21.15.47:43032) 2024-03-21T10:26:02.640 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:02.639+0000 7f0a8b400640 1 -- 172.21.15.47:0/1689092095 learned_addr learned my addr 172.21.15.47:0/1689092095 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:26:02.640 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:02.639+0000 7f0a90e00640 1 --2- 172.21.15.47:0/1689092095 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0a8c105840 0x7f0a8c1a2b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:26:02.640 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:02.639+0000 7f0a8be00640 1 --2- 172.21.15.47:0/1689092095 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0a8c100440 0x7f0a8c19e2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:26:02.641 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:02.640+0000 7f0a8b400640 1 -- 172.21.15.47:0/1689092095 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0a8c100440 msgr2=0x7f0a8c19e2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:26:02.641 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:02.640+0000 7f0a8b400640 1 --2- 172.21.15.47:0/1689092095 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0a8c100440 0x7f0a8c19e2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:26:02.641 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:02.640+0000 7f0a8b400640 1 -- 172.21.15.47:0/1689092095 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0a8c105840 msgr2=0x7f0a8c1a2b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:26:02.641 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:02.640+0000 7f0a8b400640 1 --2- 172.21.15.47:0/1689092095 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0a8c105840 0x7f0a8c1a2b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:26:02.641 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:02.640+0000 7f0a8b400640 1 -- 172.21.15.47:0/1689092095 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a8c1a32c0 con 0x7f0a8c100d60 2024-03-21T10:26:02.641 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:02.640+0000 7f0a90e00640 1 --2- 172.21.15.47:0/1689092095 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0a8c105840 0x7f0a8c1a2b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T10:26:02.642 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:02.640+0000 7f0a8b400640 1 --2- 172.21.15.47:0/1689092095 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0a8c100d60 0x7f0a8c19e810 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f0a78005940 tx=0x7f0a780045b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:26:02.642 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:02.640+0000 7f0a88c00640 1 -- 172.21.15.47:0/1689092095 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a78042020 con 0x7f0a8c100d60 2024-03-21T10:26:02.642 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:02.640+0000 7f0a88c00640 1 -- 172.21.15.47:0/1689092095 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f0a78004e00 con 0x7f0a8c100d60 2024-03-21T10:26:02.642 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:02.640+0000 7f0a88c00640 1 -- 172.21.15.47:0/1689092095 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a780038e0 con 0x7f0a8c100d60 2024-03-21T10:26:02.642 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:02.640+0000 7f0a91800640 1 -- 172.21.15.47:0/1689092095 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0a8c1a3550 con 0x7f0a8c100d60 2024-03-21T10:26:02.642 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:02.641+0000 7f0a91800640 1 -- 172.21.15.47:0/1689092095 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0a8c1aae40 con 0x7f0a8c100d60 2024-03-21T10:26:02.646 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:02.644+0000 7f0a95c29740 1 Processor -- start 2024-03-21T10:26:02.646 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:02.645+0000 7f0a95c29740 1 -- start start 2024-03-21T10:26:02.646 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:02.645+0000 7f0a95c29740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5599e1a871e0 0x5599e1a875c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:26:02.646 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:02.645+0000 7f0a95c29740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5599e1a87b00 0x5599e1a90390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:26:02.646 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:02.645+0000 7f0a95c29740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5599e1a908d0 0x5599e1a92cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:26:02.646 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:02.645+0000 7f0a95c29740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x5599e1a493b0 con 0x5599e1a908d0 2024-03-21T10:26:02.646 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:02.645+0000 7f0a95c29740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x5599e19eb830 con 0x5599e1a87b00 2024-03-21T10:26:02.646 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:02.645+0000 7f0a95c29740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x5599e19945d0 con 0x5599e1a871e0 2024-03-21T10:26:02.646 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:02.645+0000 7f0a90e00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5599e1a871e0 0x5599e1a875c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:26:02.646 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:02.645+0000 7f0a8be00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5599e1a87b00 0x5599e1a90390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:26:02.647 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:02.646+0000 7f0a8be00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5599e1a87b00 0x5599e1a90390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:43518/0 (socket says 172.21.15.47:43518) 2024-03-21T10:26:02.647 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:02.646+0000 7f0a8be00640 1 -- 172.21.15.47:0/3265479920 learned_addr learned my addr 172.21.15.47:0/3265479920 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:26:02.648 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:02.646+0000 7f0a88c00640 1 -- 172.21.15.47:0/1689092095 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 26) v1 ==== 138087+0+0 (secure 0 0 0) 0x7f0a78045020 con 0x7f0a8c100d60 2024-03-21T10:26:02.648 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:02.646+0000 7f0a8be00640 1 -- 172.21.15.47:0/3265479920 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5599e1a871e0 msgr2=0x5599e1a875c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:26:02.648 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:02.646+0000 7f0a8be00640 1 --2- 172.21.15.47:0/3265479920 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5599e1a871e0 0x5599e1a875c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:26:02.648 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:02.646+0000 7f0a8be00640 1 -- 172.21.15.47:0/3265479920 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5599e1a908d0 msgr2=0x5599e1a92cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:26:02.648 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:02.646+0000 7f0a8be00640 1 --2- 172.21.15.47:0/3265479920 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5599e1a908d0 0x5599e1a92cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:26:02.648 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:02.646+0000 7f0a8be00640 1 -- 172.21.15.47:0/3265479920 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5599e19ac2c0 con 0x5599e1a87b00 2024-03-21T10:26:02.648 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:02.647+0000 7f0a8be00640 1 --2- 172.21.15.47:0/3265479920 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5599e1a87b00 0x5599e1a90390 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f0a7c009c80 tx=0x7f0a7c0117b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=445dba54b97266a6 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:26:02.648 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:02.647+0000 7f0a81600640 1 -- 172.21.15.47:0/3265479920 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a7c012450 con 0x5599e1a87b00 2024-03-21T10:26:02.648 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:02.647+0000 7f0a81600640 1 -- 172.21.15.47:0/3265479920 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f0a7c012aa0 con 0x5599e1a87b00 2024-03-21T10:26:02.648 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:02.647+0000 7f0a81600640 1 -- 172.21.15.47:0/3265479920 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a7c00fb40 con 0x5599e1a87b00 2024-03-21T10:26:02.648 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:02.648+0000 7f0a88c00640 1 --2- 172.21.15.47:0/1689092095 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f0a700a31a0 0x7f0a700a5660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:26:02.649 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:02.648+0000 7f0a88c00640 1 -- 172.21.15.47:0/1689092095 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 9474+0+0 (secure 0 0 0) 0x7f0a780f1a30 con 0x7f0a8c100d60 2024-03-21T10:26:02.649 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:02.648+0000 7f0a90e00640 1 --2- 172.21.15.47:0/1689092095 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f0a700a31a0 0x7f0a700a5660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:26:02.649 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:02.648+0000 7f0a95c29740 1 -- 172.21.15.47:0/3265479920 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5599e1a87b00 msgr2=0x5599e1a90390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:26:02.649 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:02.648+0000 7f0a95c29740 1 --2- 172.21.15.47:0/3265479920 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5599e1a87b00 0x5599e1a90390 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f0a7c009c80 tx=0x7f0a7c0117b0 comp rx=0 tx=0).stop 2024-03-21T10:26:02.649 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:02.648+0000 7f0a95c29740 1 -- 172.21.15.47:0/3265479920 shutdown_connections 2024-03-21T10:26:02.649 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:02.649+0000 7f0a95c29740 1 --2- 172.21.15.47:0/3265479920 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5599e1a908d0 0x5599e1a92cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:26:02.649 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:02.649+0000 7f0a95c29740 1 --2- 172.21.15.47:0/3265479920 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5599e1a87b00 0x5599e1a90390 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:26:02.649 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:02.649+0000 7f0a95c29740 1 --2- 172.21.15.47:0/3265479920 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5599e1a871e0 0x5599e1a875c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:26:02.650 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:02.649+0000 7f0a95c29740 1 -- 172.21.15.47:0/3265479920 >> 172.21.15.47:0/3265479920 conn(0x5599e1a81b00 msgr2=0x5599e1a81ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:26:02.651 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:02.649+0000 7f0a95c29740 1 -- 172.21.15.47:0/3265479920 shutdown_connections 2024-03-21T10:26:02.651 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:02.649+0000 7f0a90e00640 1 --2- 172.21.15.47:0/1689092095 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f0a700a31a0 0x7f0a700a5660 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f0a84005fd0 tx=0x7f0a84009500 comp rx=0 tx=0).ready entity=mgr.15687 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:26:02.651 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:02.649+0000 7f0a95c29740 1 -- 172.21.15.47:0/3265479920 wait complete. 2024-03-21T10:26:02.651 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:02.650+0000 7f0a95c29740 1 -- start start 2024-03-21T10:26:02.651 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:02.650+0000 7f0a95c29740 1 Processor -- start 2024-03-21T10:26:02.651 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:02.650+0000 7f0a95c29740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5599e1a871e0 0x5599e1a8d9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:26:02.651 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:02.650+0000 7f0a95c29740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5599e1a87b00 0x5599e1a8df30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:26:02.651 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:02.650+0000 7f0a95c29740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5599e1a908d0 0x5599e1a8e470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:26:02.651 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:02.650+0000 7f0a95c29740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x5599e19a7a20 con 0x5599e1a908d0 2024-03-21T10:26:02.651 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:02.650+0000 7f0a95c29740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x5599e19e78d0 con 0x5599e1a87b00 2024-03-21T10:26:02.651 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:02.650+0000 7f0a95c29740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x5599e1a93500 con 0x5599e1a871e0 2024-03-21T10:26:02.651 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:02.650+0000 7f0a95c29740 10 client.0 _ll_register_callbacks cb 0x5599e19e41c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T10:26:02.652 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:02.651+0000 7f0a8b400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5599e1a87b00 0x5599e1a8df30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:26:02.653 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:02.651+0000 7f0a90e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5599e1a908d0 0x5599e1a8e470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:26:02.653 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:02.651+0000 7f0a8b400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5599e1a87b00 0x5599e1a8df30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:43530/0 (socket says 172.21.15.47:43530) 2024-03-21T10:26:02.653 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:02.651+0000 7f0a8b400640 1 -- 172.21.15.47:0/1062009732 learned_addr learned my addr 172.21.15.47:0/1062009732 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:26:02.653 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:02.651+0000 7f0a8be00640 1 --2- 172.21.15.47:0/1062009732 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5599e1a871e0 0x5599e1a8d9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:26:02.653 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:02.651+0000 7f0a8b400640 1 -- 172.21.15.47:0/1062009732 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5599e1a871e0 msgr2=0x5599e1a8d9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:26:02.653 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:02.651+0000 7f0a8b400640 1 --2- 172.21.15.47:0/1062009732 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5599e1a871e0 0x5599e1a8d9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:26:02.653 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:02.651+0000 7f0a8b400640 1 -- 172.21.15.47:0/1062009732 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5599e1a908d0 msgr2=0x5599e1a8e470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:26:02.653 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:02.651+0000 7f0a8b400640 1 --2- 172.21.15.47:0/1062009732 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5599e1a908d0 0x5599e1a8e470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:26:02.653 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:02.651+0000 7f0a8b400640 1 -- 172.21.15.47:0/1062009732 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5599e1700210 con 0x5599e1a87b00 2024-03-21T10:26:02.653 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:02.652+0000 7f0a8b400640 1 --2- 172.21.15.47:0/1062009732 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5599e1a87b00 0x5599e1a8df30 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f0a780bd850 tx=0x7f0a780bd330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ea759c947523030f server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:26:02.653 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:02.652+0000 7f0a76000640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-21T10:26:02.653 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:02.652+0000 7f0a76000640 1 -- 172.21.15.47:0/1062009732 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a780c62d0 con 0x5599e1a87b00 2024-03-21T10:26:02.653 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:02.652+0000 7f0a76000640 1 -- 172.21.15.47:0/1062009732 <== mon.2 v2:172.21.15.100:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 9474+0+0 (secure 0 0 0) 0x7f0a780b9f30 con 0x5599e1a87b00 2024-03-21T10:26:02.653 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:02.652+0000 7f0a95c29740 1 -- 172.21.15.47:0/1062009732 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f0a7c012aa0 con 0x5599e1a87b00 2024-03-21T10:26:02.654 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:02.652+0000 7f0a76000640 1 -- 172.21.15.47:0/1062009732 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0a78004fb0 con 0x5599e1a87b00 2024-03-21T10:26:02.654 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:02.652+0000 7f0a95c29740 10 client.35774 fetch_fsmap learned FSMap version 30 2024-03-21T10:26:02.654 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:02.652+0000 7f0a95c29740 1 -- 172.21.15.47:0/1062009732 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x5599e17cc620 con 0x5599e1a87b00 2024-03-21T10:26:02.655 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:02.653+0000 7f0a76000640 1 -- 172.21.15.47:0/1062009732 <== mon.2 v2:172.21.15.100:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f0a7c012aa0 con 0x5599e1a87b00 2024-03-21T10:26:02.655 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:02.653+0000 7f0a76000640 1 -- 172.21.15.47:0/1062009732 <== mon.2 v2:172.21.15.100:3300/0 5 ==== fsmap(e 30) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f0a780c5ed0 con 0x5599e1a87b00 2024-03-21T10:26:02.655 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:02.653+0000 7f0a95c29740 10 client.35774 fetch_fsmap finished waiting for FSMap version 30 2024-03-21T10:26:02.655 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:02.653+0000 7f0a95c29740 10 client.35774 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-21T10:26:02.655 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:02.653+0000 7f0a95c29740 20 client.35774 populate_metadata read hostname 'smithi047' 2024-03-21T10:26:02.655 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:02.653+0000 7f0a95c29740 1 --2- 172.21.15.47:0/1062009732 >> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] conn(0x5599e1aa08f0 0x5599e1aa2d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:26:02.655 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:02.653+0000 7f0a95c29740 4 client.35774 mds_command: new command op to 34535 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T10:26:02.655 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:02.653+0000 7f0a95c29740 1 -- 172.21.15.47:0/1062009732 --> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5599e19f8d70 con 0x5599e1aa08f0 2024-03-21T10:26:02.655 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:02.654+0000 7f0a8be00640 1 --2- 172.21.15.47:0/1062009732 >> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] conn(0x5599e1aa08f0 0x5599e1aa2d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:26:02.655 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:02.655+0000 7f0a8be00640 1 --2- 172.21.15.47:0/1062009732 >> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] conn(0x5599e1aa08f0 0x5599e1aa2d00 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4422f76396b4b9c0 server_cookie=1eef897f049a5ebe in_seq=0 out_seq=0 2024-03-21T10:26:02.656 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:02.655+0000 7f0a76000640 10 client.35774 ms_handle_connect on v2:172.21.15.100:6838/4173168440 2024-03-21T10:26:02.657 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:02.657+0000 7f0a76000640 1 -- 172.21.15.47:0/1062009732 <== mds.0 v2:172.21.15.100:6838/4173168440 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x5599e19f8d70 con 0x5599e1aa08f0 2024-03-21T10:26:02.657 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:02.657+0000 7f0a76000640 10 client.35774 handle_command_reply: tid=0 2024-03-21T10:26:02.657 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:02.657+0000 7f0a95c29740 1 client.35774 shutdown 2024-03-21T10:26:02.658 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:02.657+0000 7f0a95c29740 1 -- 172.21.15.47:0/1062009732 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5599e1a87b00 msgr2=0x5599e1a8df30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:26:02.658 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:02.657+0000 7f0a95c29740 1 --2- 172.21.15.47:0/1062009732 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5599e1a87b00 0x5599e1a8df30 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f0a780bd850 tx=0x7f0a780bd330 comp rx=0 tx=0).stop 2024-03-21T10:26:02.658 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:02.657+0000 7f0a95c29740 1 -- 172.21.15.47:0/1062009732 shutdown_connections 2024-03-21T10:26:02.658 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:02.657+0000 7f0a95c29740 1 --2- 172.21.15.47:0/1062009732 >> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] conn(0x5599e1aa08f0 0x5599e1aa2d00 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:26:02.658 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:02.657+0000 7f0a95c29740 1 --2- 172.21.15.47:0/1062009732 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5599e1a908d0 0x5599e1a8e470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:26:02.658 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:02.657+0000 7f0a95c29740 1 --2- 172.21.15.47:0/1062009732 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5599e1a87b00 0x5599e1a8df30 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:26:02.658 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:02.657+0000 7f0a76000640 0 client.35774 ms_handle_reset on v2:172.21.15.100:6838/4173168440 2024-03-21T10:26:02.658 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:02.657+0000 7f0a95c29740 1 --2- 172.21.15.47:0/1062009732 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5599e1a871e0 0x5599e1a8d9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:26:02.659 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:02.657+0000 7f0a95c29740 1 -- 172.21.15.47:0/1062009732 >> 172.21.15.47:0/1062009732 conn(0x5599e1a81b00 msgr2=0x5599e1a83850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:26:02.659 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:02.658+0000 7f0a95c29740 1 -- 172.21.15.47:0/1062009732 shutdown_connections 2024-03-21T10:26:02.659 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:02.658+0000 7f0a95c29740 1 -- 172.21.15.47:0/1062009732 wait complete. 2024-03-21T10:26:02.659 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:02.658+0000 7f0a95c29740 20 client.35774 trim_cache size 0 max 16384 2024-03-21T10:26:02.678 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:02.676+0000 7f0a95c29740 1 Processor -- start 2024-03-21T10:26:02.678 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:02.676+0000 7f0a95c29740 1 -- start start 2024-03-21T10:26:02.678 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:02.677+0000 7f0a95c29740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5599e1a871e0 0x5599e1a90120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:26:02.678 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:02.677+0000 7f0a95c29740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5599e1a87b00 0x5599e1a8e2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:26:02.678 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:02.677+0000 7f0a95c29740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5599e1a908d0 0x5599e1a9bc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:26:02.678 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:02.677+0000 7f0a95c29740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a8c1b4d70 con 0x5599e1a87b00 2024-03-21T10:26:02.678 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:02.677+0000 7f0a95c29740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a8c1b66f0 con 0x5599e1a871e0 2024-03-21T10:26:02.678 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:02.677+0000 7f0a95c29740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a8c1b53d0 con 0x5599e1a908d0 2024-03-21T10:26:02.678 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:02.677+0000 7f0a8b400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5599e1a87b00 0x5599e1a8e2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:26:02.679 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:02.677+0000 7f0a8b400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5599e1a87b00 0x5599e1a8e2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:43058/0 (socket says 172.21.15.47:43058) 2024-03-21T10:26:02.679 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:02.677+0000 7f0a8b400640 1 -- 172.21.15.47:0/404776956 learned_addr learned my addr 172.21.15.47:0/404776956 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:26:02.679 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:02.677+0000 7f0a8be00640 1 --2- 172.21.15.47:0/404776956 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5599e1a871e0 0x5599e1a90120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:26:02.679 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:02.677+0000 7f0a90e00640 1 --2- 172.21.15.47:0/404776956 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5599e1a908d0 0x5599e1a9bc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:26:02.679 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:02.678+0000 7f0a8b400640 1 -- 172.21.15.47:0/404776956 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5599e1a908d0 msgr2=0x5599e1a9bc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:26:02.679 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:02.678+0000 7f0a8b400640 1 --2- 172.21.15.47:0/404776956 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5599e1a908d0 0x5599e1a9bc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:26:02.679 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:02.678+0000 7f0a8b400640 1 -- 172.21.15.47:0/404776956 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5599e1a871e0 msgr2=0x5599e1a90120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:26:02.679 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:02.678+0000 7f0a8b400640 1 --2- 172.21.15.47:0/404776956 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5599e1a871e0 0x5599e1a90120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:26:02.679 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:02.678+0000 7f0a8b400640 1 -- 172.21.15.47:0/404776956 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5599e1a8bfa0 con 0x5599e1a87b00 2024-03-21T10:26:02.680 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:02.678+0000 7f0a8b400640 1 --2- 172.21.15.47:0/404776956 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5599e1a87b00 0x5599e1a8e2b0 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7f0a78005ec0 tx=0x7f0a780bda00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a2fc4d363876bbd server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:26:02.681 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:02.678+0000 7f0a80c00640 1 -- 172.21.15.47:0/404776956 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a78005150 con 0x5599e1a87b00 2024-03-21T10:26:02.681 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:02.678+0000 7f0a80c00640 1 -- 172.21.15.47:0/404776956 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f0a780db020 con 0x5599e1a87b00 2024-03-21T10:26:02.681 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:02.678+0000 7f0a80c00640 1 -- 172.21.15.47:0/404776956 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a780f10d0 con 0x5599e1a87b00 2024-03-21T10:26:02.681 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:02.679+0000 7f0a95c29740 1 -- 172.21.15.47:0/404776956 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5599e1a87b00 msgr2=0x5599e1a8e2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:26:02.681 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:02.679+0000 7f0a95c29740 1 --2- 172.21.15.47:0/404776956 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5599e1a87b00 0x5599e1a8e2b0 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7f0a78005ec0 tx=0x7f0a780bda00 comp rx=0 tx=0).stop 2024-03-21T10:26:02.681 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:02.679+0000 7f0a95c29740 1 -- 172.21.15.47:0/404776956 shutdown_connections 2024-03-21T10:26:02.681 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:02.679+0000 7f0a95c29740 1 --2- 172.21.15.47:0/404776956 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5599e1a908d0 0x5599e1a9bc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:26:02.681 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:02.679+0000 7f0a95c29740 1 --2- 172.21.15.47:0/404776956 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5599e1a87b00 0x5599e1a8e2b0 unknown :-1 s=CLOSED pgs=585 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:26:02.681 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:02.679+0000 7f0a95c29740 1 --2- 172.21.15.47:0/404776956 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5599e1a871e0 0x5599e1a90120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:26:02.681 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:02.679+0000 7f0a95c29740 1 -- 172.21.15.47:0/404776956 >> 172.21.15.47:0/404776956 conn(0x5599e1a81b00 msgr2=0x5599e1a80b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:26:02.681 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:02.679+0000 7f0a95c29740 1 -- 172.21.15.47:0/404776956 shutdown_connections 2024-03-21T10:26:02.681 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:02.679+0000 7f0a95c29740 1 -- 172.21.15.47:0/404776956 wait complete. 2024-03-21T10:26:02.681 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:02.680+0000 7f0a95c29740 1 -- start start 2024-03-21T10:26:02.682 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:02.680+0000 7f0a95c29740 1 Processor -- start 2024-03-21T10:26:02.685 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:02.681+0000 7f0a95c29740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5599e1a871e0 0x5599e1af5400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:26:02.685 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:02.681+0000 7f0a95c29740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5599e1a87b00 0x5599e1af5940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:26:02.685 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:02.681+0000 7f0a95c29740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5599e1a908d0 0x5599e1a8a820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:26:02.685 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:02.681+0000 7f0a95c29740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a8c1b6090 con 0x5599e1a908d0 2024-03-21T10:26:02.686 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:02.681+0000 7f0a95c29740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a8c1b5a30 con 0x5599e1a87b00 2024-03-21T10:26:02.686 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:02.681+0000 7f0a95c29740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a8c1b2f30 con 0x5599e1a871e0 2024-03-21T10:26:02.686 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:02.681+0000 7f0a95c29740 10 client.0 _ll_register_callbacks cb 0x7f0a780bd420 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T10:26:02.686 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:02.681+0000 7f0a8b400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5599e1a87b00 0x5599e1af5940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:26:02.686 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:02.681+0000 7f0a90e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5599e1a908d0 0x5599e1a8a820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:26:02.686 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:02.681+0000 7f0a8b400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5599e1a87b00 0x5599e1af5940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:43542/0 (socket says 172.21.15.47:43542) 2024-03-21T10:26:02.686 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:02.681+0000 7f0a8b400640 1 -- 172.21.15.47:0/3172860236 learned_addr learned my addr 172.21.15.47:0/3172860236 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:26:02.686 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:02.681+0000 7f0a8be00640 1 --2- 172.21.15.47:0/3172860236 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5599e1a871e0 0x5599e1af5400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:26:02.686 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:02.681+0000 7f0a8b400640 1 -- 172.21.15.47:0/3172860236 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5599e1a871e0 msgr2=0x5599e1af5400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:26:02.686 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:02.682+0000 7f0a8b400640 1 --2- 172.21.15.47:0/3172860236 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5599e1a871e0 0x5599e1af5400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:26:02.686 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:02.682+0000 7f0a8b400640 1 -- 172.21.15.47:0/3172860236 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5599e1a908d0 msgr2=0x5599e1a8a820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:26:02.686 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:02.682+0000 7f0a8b400640 1 --2- 172.21.15.47:0/3172860236 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5599e1a908d0 0x5599e1a8a820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:26:02.686 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:02.682+0000 7f0a8b400640 1 -- 172.21.15.47:0/3172860236 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5599e1a8af10 con 0x5599e1a87b00 2024-03-21T10:26:02.686 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:02.682+0000 7f0a8be00640 1 --2- 172.21.15.47:0/3172860236 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5599e1a871e0 0x5599e1af5400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T10:26:02.687 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:02.682+0000 7f0a8b400640 1 --2- 172.21.15.47:0/3172860236 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5599e1a87b00 0x5599e1af5940 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f0a7802cfd0 tx=0x7f0a780bbdd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d9df10d120198108 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:26:02.687 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:02.682+0000 7f0a76000640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-21T10:26:02.687 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:02.682+0000 7f0a76000640 1 -- 172.21.15.47:0/3172860236 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a780d6030 con 0x5599e1a87b00 2024-03-21T10:26:02.687 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:02.682+0000 7f0a76000640 1 -- 172.21.15.47:0/3172860236 <== mon.2 v2:172.21.15.100:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 9474+0+0 (secure 0 0 0) 0x7f0a780dc040 con 0x5599e1a87b00 2024-03-21T10:26:02.687 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:02.682+0000 7f0a95c29740 1 -- 172.21.15.47:0/3172860236 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f0a780db020 con 0x5599e1a87b00 2024-03-21T10:26:02.687 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:02.683+0000 7f0a76000640 1 -- 172.21.15.47:0/3172860236 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0a78005150 con 0x5599e1a87b00 2024-03-21T10:26:02.687 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:02.683+0000 7f0a95c29740 10 client.35780 fetch_fsmap learned FSMap version 30 2024-03-21T10:26:02.687 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:02.683+0000 7f0a95c29740 1 -- 172.21.15.47:0/3172860236 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x5599e1a8af10 con 0x5599e1a87b00 2024-03-21T10:26:02.687 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:02.683+0000 7f0a76000640 1 -- 172.21.15.47:0/3172860236 <== mon.2 v2:172.21.15.100:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f0a780db020 con 0x5599e1a87b00 2024-03-21T10:26:02.687 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:02.683+0000 7f0a76000640 1 -- 172.21.15.47:0/3172860236 <== mon.2 v2:172.21.15.100:3300/0 5 ==== fsmap(e 30) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f0a780bcd10 con 0x5599e1a87b00 2024-03-21T10:26:02.687 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:02.683+0000 7f0a95c29740 10 client.35780 fetch_fsmap finished waiting for FSMap version 30 2024-03-21T10:26:02.687 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:02.683+0000 7f0a95c29740 10 client.35780 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-21T10:26:02.687 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:02.683+0000 7f0a95c29740 20 client.35780 populate_metadata read hostname 'smithi047' 2024-03-21T10:26:02.688 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:02.683+0000 7f0a95c29740 1 --2- 172.21.15.47:0/3172860236 >> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] conn(0x5599e1afce00 0x5599e1aff1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:26:02.688 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:02.683+0000 7f0a95c29740 4 client.35780 mds_command: new command op to 34535 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-21T10:26:02.688 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:02.683+0000 7f0a95c29740 1 -- 172.21.15.47:0/3172860236 --> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x5599e19f30c0 con 0x5599e1afce00 2024-03-21T10:26:02.688 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:02.684+0000 7f0a8be00640 1 --2- 172.21.15.47:0/3172860236 >> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] conn(0x5599e1afce00 0x5599e1aff1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:26:02.688 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:02.684+0000 7f0a8be00640 1 --2- 172.21.15.47:0/3172860236 >> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] conn(0x5599e1afce00 0x5599e1aff1f0 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=774a02805aae3d40 server_cookie=8b4d98d879a75284 in_seq=0 out_seq=0 2024-03-21T10:26:02.688 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:02.684+0000 7f0a76000640 10 client.35780 ms_handle_connect on v2:172.21.15.100:6838/4173168440 2024-03-21T10:26:03.465 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:26:03 smithi047 ceph-mon[31547]: pgmap v24: 129 pgs: 129 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 23 KiB/s wr, 28 op/s 2024-03-21T10:26:03.466 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:26:03 smithi047 ceph-mon[31547]: mgrmap e26: x(active, since 42s), standbys: y, z 2024-03-21T10:26:03.479 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:26:03 smithi100 ceph-mon[35193]: pgmap v24: 129 pgs: 129 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 23 KiB/s wr, 28 op/s 2024-03-21T10:26:03.479 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:26:03 smithi100 ceph-mon[35193]: mgrmap e26: x(active, since 42s), standbys: y, z 2024-03-21T10:26:03.489 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:26:03 smithi157 ceph-mon[35581]: pgmap v24: 129 pgs: 129 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 23 KiB/s wr, 28 op/s 2024-03-21T10:26:03.489 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:26:03 smithi157 ceph-mon[35581]: mgrmap e26: x(active, since 42s), standbys: y, z 2024-03-21T10:26:03.608 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:03.607+0000 7f0a76000640 1 -- 172.21.15.47:0/3172860236 <== mds.0 v2:172.21.15.100:6838/4173168440 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x5599e19f30c0 con 0x5599e1afce00 2024-03-21T10:26:03.608 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:03.607+0000 7f0a76000640 10 client.35780 handle_command_reply: tid=0 2024-03-21T10:26:03.608 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:03.607+0000 7f0a95c29740 1 client.35780 shutdown 2024-03-21T10:26:03.609 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:03.608+0000 7f0a95c29740 1 -- 172.21.15.47:0/3172860236 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5599e1a87b00 msgr2=0x5599e1af5940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:26:03.609 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:03.608+0000 7f0a95c29740 1 --2- 172.21.15.47:0/3172860236 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5599e1a87b00 0x5599e1af5940 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f0a7802cfd0 tx=0x7f0a780bbdd0 comp rx=0 tx=0).stop 2024-03-21T10:26:03.609 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:03.608+0000 7f0a95c29740 1 -- 172.21.15.47:0/3172860236 shutdown_connections 2024-03-21T10:26:03.609 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:03.608+0000 7f0a95c29740 1 --2- 172.21.15.47:0/3172860236 >> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] conn(0x5599e1afce00 0x5599e1aff1f0 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:26:03.609 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:03.608+0000 7f0a95c29740 1 --2- 172.21.15.47:0/3172860236 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5599e1a908d0 0x5599e1a8a820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:26:03.609 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:03.608+0000 7f0a95c29740 1 --2- 172.21.15.47:0/3172860236 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5599e1a87b00 0x5599e1af5940 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:26:03.610 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:03.608+0000 7f0a95c29740 1 --2- 172.21.15.47:0/3172860236 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5599e1a871e0 0x5599e1af5400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:26:03.610 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:03.608+0000 7f0a76000640 0 client.35780 ms_handle_reset on v2:172.21.15.100:6838/4173168440 2024-03-21T10:26:03.610 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:03.608+0000 7f0a95c29740 1 -- 172.21.15.47:0/3172860236 >> 172.21.15.47:0/3172860236 conn(0x5599e1a81b00 msgr2=0x5599e1a82e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:26:03.610 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:03.608+0000 7f0a95c29740 1 -- 172.21.15.47:0/3172860236 shutdown_connections 2024-03-21T10:26:03.610 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:03.608+0000 7f0a95c29740 1 -- 172.21.15.47:0/3172860236 wait complete. 2024-03-21T10:26:03.610 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:03.609+0000 7f0a95c29740 20 client.35780 trim_cache size 0 max 16384 2024-03-21T10:26:03.612 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout:{ 2024-03-21T10:26:03.612 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout: "message": "", 2024-03-21T10:26:03.612 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout: "return_code": 0 2024-03-21T10:26:03.613 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout:} 2024-03-21T10:26:03.613 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:03.610+0000 7f0a91800640 1 -- 172.21.15.47:0/1689092095 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f0a700a31a0 msgr2=0x7f0a700a5660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:26:03.613 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:03.610+0000 7f0a91800640 1 --2- 172.21.15.47:0/1689092095 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f0a700a31a0 0x7f0a700a5660 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f0a84005fd0 tx=0x7f0a84009500 comp rx=0 tx=0).stop 2024-03-21T10:26:03.613 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:03.610+0000 7f0a91800640 1 -- 172.21.15.47:0/1689092095 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0a8c100d60 msgr2=0x7f0a8c19e810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:26:03.613 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:03.610+0000 7f0a91800640 1 --2- 172.21.15.47:0/1689092095 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0a8c100d60 0x7f0a8c19e810 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f0a78005940 tx=0x7f0a780045b0 comp rx=0 tx=0).stop 2024-03-21T10:26:03.613 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:03.610+0000 7f0a91800640 1 -- 172.21.15.47:0/1689092095 shutdown_connections 2024-03-21T10:26:03.613 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:03.610+0000 7f0a91800640 1 --2- 172.21.15.47:0/1689092095 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f0a700a31a0 0x7f0a700a5660 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:26:03.613 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:03.610+0000 7f0a91800640 1 --2- 172.21.15.47:0/1689092095 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0a8c105840 0x7f0a8c1a2b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:26:03.613 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:03.610+0000 7f0a91800640 1 --2- 172.21.15.47:0/1689092095 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0a8c100d60 0x7f0a8c19e810 unknown :-1 s=CLOSED pgs=584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:26:03.613 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:03.610+0000 7f0a91800640 1 --2- 172.21.15.47:0/1689092095 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0a8c100440 0x7f0a8c19e2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:26:03.613 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:03.610+0000 7f0a91800640 1 -- 172.21.15.47:0/1689092095 >> 172.21.15.47:0/1689092095 conn(0x7f0a8c0fbeb0 msgr2=0x7f0a8c0fcfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:26:03.613 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:03.611+0000 7f0a91800640 1 -- 172.21.15.47:0/1689092095 shutdown_connections 2024-03-21T10:26:03.614 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:03.611+0000 7f0a91800640 1 -- 172.21.15.47:0/1689092095 wait complete. 2024-03-21T10:26:04.491 INFO:journalctl@ceph.mds.i.smithi157.stdout:Mar 21 10:26:04 smithi157 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-mds-i[66599]: *** Caught signal (Segmentation fault) ** 2024-03-21T10:26:04.492 INFO:journalctl@ceph.mds.i.smithi157.stdout:Mar 21 10:26:04 smithi157 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-mds-i[66599]: in thread 7f8df7600700 thread_name:md_log_replay 2024-03-21T10:26:04.492 INFO:journalctl@ceph.mds.i.smithi157.stdout:Mar 21 10:26:04 smithi157 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-mds-i[66599]: ceph version 19.0.0-2231-g2edfee68 (2edfee68ad7243bed422a049b6f5f58619300f11) squid (dev) 2024-03-21T10:26:04.492 INFO:journalctl@ceph.mds.i.smithi157.stdout:Mar 21 10:26:04 smithi157 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-mds-i[66599]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7f8e0bc58d20] 2024-03-21T10:26:04.492 INFO:journalctl@ceph.mds.i.smithi157.stdout:Mar 21 10:26:04 smithi157 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-mds-i[66599]: 2: /usr/bin/ceph-mds(+0x531252) [0x558fb964f252] 2024-03-21T10:26:04.492 INFO:journalctl@ceph.mds.i.smithi157.stdout:Mar 21 10:26:04 smithi157 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-mds-i[66599]: 3: (MDSTableClient::got_journaled_ack(unsigned long)+0x173) [0x558fb9653303] 2024-03-21T10:26:04.492 INFO:journalctl@ceph.mds.i.smithi157.stdout:Mar 21 10:26:04 smithi157 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-mds-i[66599]: 4: (MDLog::_replay_thread()+0x617) [0x558fb96b17a7] 2024-03-21T10:26:04.492 INFO:journalctl@ceph.mds.i.smithi157.stdout:Mar 21 10:26:04 smithi157 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-mds-i[66599]: 5: (MDLog::ReplayThread::entry()+0x11) [0x558fb96b7dc1] 2024-03-21T10:26:04.492 INFO:journalctl@ceph.mds.i.smithi157.stdout:Mar 21 10:26:04 smithi157 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-mds-i[66599]: 6: /lib64/libpthread.so.0(+0x81ca) [0x7f8e0bc4e1ca] 2024-03-21T10:26:04.492 INFO:journalctl@ceph.mds.i.smithi157.stdout:Mar 21 10:26:04 smithi157 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-mds-i[66599]: 7: clone() 2024-03-21T10:26:04.492 INFO:journalctl@ceph.mds.i.smithi157.stdout:Mar 21 10:26:04 smithi157 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-mds-i[66599]: 2024-03-21T10:26:04.157+0000 7f8df7600700 -1 *** Caught signal (Segmentation fault) ** 2024-03-21T10:26:04.492 INFO:journalctl@ceph.mds.i.smithi157.stdout:Mar 21 10:26:04 smithi157 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-mds-i[66599]: in thread 7f8df7600700 thread_name:md_log_replay 2024-03-21T10:26:04.492 INFO:journalctl@ceph.mds.i.smithi157.stdout:Mar 21 10:26:04 smithi157 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-mds-i[66599]: 2024-03-21T10:26:04.493 INFO:journalctl@ceph.mds.i.smithi157.stdout:Mar 21 10:26:04 smithi157 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-mds-i[66599]: ceph version 19.0.0-2231-g2edfee68 (2edfee68ad7243bed422a049b6f5f58619300f11) squid (dev) 2024-03-21T10:26:04.493 INFO:journalctl@ceph.mds.i.smithi157.stdout:Mar 21 10:26:04 smithi157 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-mds-i[66599]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7f8e0bc58d20] 2024-03-21T10:26:04.493 INFO:journalctl@ceph.mds.i.smithi157.stdout:Mar 21 10:26:04 smithi157 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-mds-i[66599]: 2: /usr/bin/ceph-mds(+0x531252) [0x558fb964f252] 2024-03-21T10:26:04.493 INFO:journalctl@ceph.mds.i.smithi157.stdout:Mar 21 10:26:04 smithi157 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-mds-i[66599]: 3: (MDSTableClient::got_journaled_ack(unsigned long)+0x173) [0x558fb9653303] 2024-03-21T10:26:04.493 INFO:journalctl@ceph.mds.i.smithi157.stdout:Mar 21 10:26:04 smithi157 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-mds-i[66599]: 4: (MDLog::_replay_thread()+0x617) [0x558fb96b17a7] 2024-03-21T10:26:04.493 INFO:journalctl@ceph.mds.i.smithi157.stdout:Mar 21 10:26:04 smithi157 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-mds-i[66599]: 5: (MDLog::ReplayThread::entry()+0x11) [0x558fb96b7dc1] 2024-03-21T10:26:04.493 INFO:journalctl@ceph.mds.i.smithi157.stdout:Mar 21 10:26:04 smithi157 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-mds-i[66599]: 6: /lib64/libpthread.so.0(+0x81ca) [0x7f8e0bc4e1ca] 2024-03-21T10:26:04.493 INFO:journalctl@ceph.mds.i.smithi157.stdout:Mar 21 10:26:04 smithi157 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-mds-i[66599]: 7: clone() 2024-03-21T10:26:04.493 INFO:journalctl@ceph.mds.i.smithi157.stdout:Mar 21 10:26:04 smithi157 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-mds-i[66599]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-03-21T10:26:04.493 INFO:journalctl@ceph.mds.i.smithi157.stdout:Mar 21 10:26:04 smithi157 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-mds-i[66599]: 2024-03-21T10:26:04.493 INFO:journalctl@ceph.mds.i.smithi157.stdout:Mar 21 10:26:04 smithi157 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-mds-i[66599]: 0> 2024-03-21T10:26:04.157+0000 7f8df7600700 -1 *** Caught signal (Segmentation fault) ** 2024-03-21T10:26:04.493 INFO:journalctl@ceph.mds.i.smithi157.stdout:Mar 21 10:26:04 smithi157 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-mds-i[66599]: in thread 7f8df7600700 thread_name:md_log_replay 2024-03-21T10:26:04.493 INFO:journalctl@ceph.mds.i.smithi157.stdout:Mar 21 10:26:04 smithi157 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-mds-i[66599]: 2024-03-21T10:26:04.493 INFO:journalctl@ceph.mds.i.smithi157.stdout:Mar 21 10:26:04 smithi157 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-mds-i[66599]: ceph version 19.0.0-2231-g2edfee68 (2edfee68ad7243bed422a049b6f5f58619300f11) squid (dev) 2024-03-21T10:26:04.493 INFO:journalctl@ceph.mds.i.smithi157.stdout:Mar 21 10:26:04 smithi157 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-mds-i[66599]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7f8e0bc58d20] 2024-03-21T10:26:04.494 INFO:journalctl@ceph.mds.i.smithi157.stdout:Mar 21 10:26:04 smithi157 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-mds-i[66599]: 2: /usr/bin/ceph-mds(+0x531252) [0x558fb964f252] 2024-03-21T10:26:04.494 INFO:journalctl@ceph.mds.i.smithi157.stdout:Mar 21 10:26:04 smithi157 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-mds-i[66599]: 3: (MDSTableClient::got_journaled_ack(unsigned long)+0x173) [0x558fb9653303] 2024-03-21T10:26:04.494 INFO:journalctl@ceph.mds.i.smithi157.stdout:Mar 21 10:26:04 smithi157 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-mds-i[66599]: 4: (MDLog::_replay_thread()+0x617) [0x558fb96b17a7] 2024-03-21T10:26:04.494 INFO:journalctl@ceph.mds.i.smithi157.stdout:Mar 21 10:26:04 smithi157 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-mds-i[66599]: 5: (MDLog::ReplayThread::entry()+0x11) [0x558fb96b7dc1] 2024-03-21T10:26:04.494 INFO:journalctl@ceph.mds.i.smithi157.stdout:Mar 21 10:26:04 smithi157 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-mds-i[66599]: 6: /lib64/libpthread.so.0(+0x81ca) [0x7f8e0bc4e1ca] 2024-03-21T10:26:04.494 INFO:journalctl@ceph.mds.i.smithi157.stdout:Mar 21 10:26:04 smithi157 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-mds-i[66599]: 7: clone() 2024-03-21T10:26:04.494 INFO:journalctl@ceph.mds.i.smithi157.stdout:Mar 21 10:26:04 smithi157 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-mds-i[66599]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-03-21T10:26:04.494 INFO:journalctl@ceph.mds.i.smithi157.stdout:Mar 21 10:26:04 smithi157 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-mds-i[66599]: 2024-03-21T10:26:04.494 INFO:journalctl@ceph.mds.i.smithi157.stdout:Mar 21 10:26:04 smithi157 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-mds-i[66599]: 0> 2024-03-21T10:26:04.157+0000 7f8df7600700 -1 *** Caught signal (Segmentation fault) ** 2024-03-21T10:26:04.494 INFO:journalctl@ceph.mds.i.smithi157.stdout:Mar 21 10:26:04 smithi157 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-mds-i[66599]: in thread 7f8df7600700 thread_name:md_log_replay 2024-03-21T10:26:04.495 INFO:journalctl@ceph.mds.i.smithi157.stdout:Mar 21 10:26:04 smithi157 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-mds-i[66599]: 2024-03-21T10:26:04.495 INFO:journalctl@ceph.mds.i.smithi157.stdout:Mar 21 10:26:04 smithi157 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-mds-i[66599]: ceph version 19.0.0-2231-g2edfee68 (2edfee68ad7243bed422a049b6f5f58619300f11) squid (dev) 2024-03-21T10:26:04.495 INFO:journalctl@ceph.mds.i.smithi157.stdout:Mar 21 10:26:04 smithi157 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-mds-i[66599]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7f8e0bc58d20] 2024-03-21T10:26:04.495 INFO:journalctl@ceph.mds.i.smithi157.stdout:Mar 21 10:26:04 smithi157 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-mds-i[66599]: 2: /usr/bin/ceph-mds(+0x531252) [0x558fb964f252] 2024-03-21T10:26:04.495 INFO:journalctl@ceph.mds.i.smithi157.stdout:Mar 21 10:26:04 smithi157 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-mds-i[66599]: 3: (MDSTableClient::got_journaled_ack(unsigned long)+0x173) [0x558fb9653303] 2024-03-21T10:26:04.495 INFO:journalctl@ceph.mds.i.smithi157.stdout:Mar 21 10:26:04 smithi157 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-mds-i[66599]: 4: (MDLog::_replay_thread()+0x617) [0x558fb96b17a7] 2024-03-21T10:26:04.495 INFO:journalctl@ceph.mds.i.smithi157.stdout:Mar 21 10:26:04 smithi157 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-mds-i[66599]: 5: (MDLog::ReplayThread::entry()+0x11) [0x558fb96b7dc1] 2024-03-21T10:26:04.495 INFO:journalctl@ceph.mds.i.smithi157.stdout:Mar 21 10:26:04 smithi157 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-mds-i[66599]: 6: /lib64/libpthread.so.0(+0x81ca) [0x7f8e0bc4e1ca] 2024-03-21T10:26:04.495 INFO:journalctl@ceph.mds.i.smithi157.stdout:Mar 21 10:26:04 smithi157 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-mds-i[66599]: 7: clone() 2024-03-21T10:26:04.495 INFO:journalctl@ceph.mds.i.smithi157.stdout:Mar 21 10:26:04 smithi157 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-mds-i[66599]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-03-21T10:26:04.495 INFO:journalctl@ceph.mds.i.smithi157.stdout:Mar 21 10:26:04 smithi157 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-mds-i[66599]: 2024-03-21T10:26:04.495 INFO:journalctl@ceph.mds.i.smithi157.stdout:Mar 21 10:26:04 smithi157 podman[70145]: 2024-03-21 10:26:04.289629608 +0000 UTC m=+0.042259928 container died 9a440ce57d234172d1c605e0aaa9e951fbf775c9f25ec84d0874e522ec3a2262 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-3056d51a-e76b-11ee-95cd-87774f69a715-mds-i, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, ceph=True, GIT_CLEAN=True, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, org.label-schema.build-date=20240319, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux ) 2024-03-21T10:26:05.463 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:26:05 smithi047 ceph-mon[31547]: pgmap v25: 129 pgs: 129 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 23 KiB/s wr, 23 op/s 2024-03-21T10:26:05.478 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:26:05 smithi100 ceph-mon[35193]: pgmap v25: 129 pgs: 129 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 23 KiB/s wr, 23 op/s 2024-03-21T10:26:05.489 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:26:05 smithi157 ceph-mon[35581]: pgmap v25: 129 pgs: 129 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 23 KiB/s wr, 23 op/s 2024-03-21T10:26:05.489 INFO:journalctl@ceph.mds.i.smithi157.stdout:Mar 21 10:26:05 smithi157 podman[70145]: 2024-03-21 10:26:05.099187213 +0000 UTC m=+0.851817553 container remove 9a440ce57d234172d1c605e0aaa9e951fbf775c9f25ec84d0874e522ec3a2262 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-3056d51a-e76b-11ee-95cd-87774f69a715-mds-i, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, io.buildah.version=1.33.5, org.label-schema.build-date=20240319, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30) 2024-03-21T10:26:05.489 INFO:journalctl@ceph.mds.i.smithi157.stdout:Mar 21 10:26:05 smithi157 systemd[1]: ceph-3056d51a-e76b-11ee-95cd-87774f69a715@mds.i.service: Main process exited, code=exited, status=139/n/a 2024-03-21T10:26:05.989 INFO:journalctl@ceph.mds.i.smithi157.stdout:Mar 21 10:26:05 smithi157 systemd[1]: ceph-3056d51a-e76b-11ee-95cd-87774f69a715@mds.i.service: Failed with result 'exit-code'. 2024-03-21T10:26:05.990 INFO:journalctl@ceph.mds.i.smithi157.stdout:Mar 21 10:26:05 smithi157 systemd[1]: ceph-3056d51a-e76b-11ee-95cd-87774f69a715@mds.i.service: Consumed 2.414s CPU time. 2024-03-21T10:26:07.463 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:26:07 smithi047 ceph-mon[31547]: pgmap v26: 129 pgs: 129 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 23 KiB/s wr, 27 op/s 2024-03-21T10:26:07.479 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:26:07 smithi100 ceph-mon[35193]: pgmap v26: 129 pgs: 129 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 23 KiB/s wr, 27 op/s 2024-03-21T10:26:07.489 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:26:07 smithi157 ceph-mon[35581]: pgmap v26: 129 pgs: 129 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 23 KiB/s wr, 27 op/s 2024-03-21T10:26:09.463 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:26:09 smithi047 ceph-mon[31547]: pgmap v27: 129 pgs: 129 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 23 KiB/s wr, 25 op/s 2024-03-21T10:26:09.478 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:26:09 smithi100 ceph-mon[35193]: pgmap v27: 129 pgs: 129 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 23 KiB/s wr, 25 op/s 2024-03-21T10:26:09.489 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:26:09 smithi157 ceph-mon[35581]: pgmap v27: 129 pgs: 129 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 23 KiB/s wr, 25 op/s 2024-03-21T10:26:11.463 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:26:11 smithi047 ceph-mon[31547]: pgmap v28: 129 pgs: 129 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 23 KiB/s wr, 24 op/s 2024-03-21T10:26:11.478 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:26:11 smithi100 ceph-mon[35193]: pgmap v28: 129 pgs: 129 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 23 KiB/s wr, 24 op/s 2024-03-21T10:26:11.489 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:26:11 smithi157 ceph-mon[35581]: pgmap v28: 129 pgs: 129 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 23 KiB/s wr, 24 op/s 2024-03-21T10:26:13.463 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:26:13 smithi047 ceph-mon[31547]: pgmap v29: 129 pgs: 129 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 23 KiB/s wr, 28 op/s 2024-03-21T10:26:13.479 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:26:13 smithi100 ceph-mon[35193]: pgmap v29: 129 pgs: 129 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 23 KiB/s wr, 28 op/s 2024-03-21T10:26:13.489 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:26:13 smithi157 ceph-mon[35581]: pgmap v29: 129 pgs: 129 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 23 KiB/s wr, 28 op/s 2024-03-21T10:26:13.989 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-21T10:26:13.989 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-21T10:26:14.222 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:14.222+0000 7fa1b7200640 1 -- 172.21.15.47:0/2230171484 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa1b8109730 msgr2=0x7fa1b8109b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:26:14.223 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:14.222+0000 7fa1b7200640 1 --2- 172.21.15.47:0/2230171484 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa1b8109730 0x7fa1b8109b10 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7fa1a8009930 tx=0x7fa1a802bc30 comp rx=0 tx=0).stop 2024-03-21T10:26:14.223 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:14.222+0000 7fa1b7200640 1 -- 172.21.15.47:0/2230171484 shutdown_connections 2024-03-21T10:26:14.223 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:14.222+0000 7fa1b7200640 1 --2- 172.21.15.47:0/2230171484 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa1b81040e0 0x7fa1b81105f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:26:14.223 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:14.222+0000 7fa1b7200640 1 --2- 172.21.15.47:0/2230171484 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa1b8103720 0x7fa1b8103ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:26:14.223 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:14.222+0000 7fa1b7200640 1 --2- 172.21.15.47:0/2230171484 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa1b8109730 0x7fa1b8109b10 unknown :-1 s=CLOSED pgs=586 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:26:14.223 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:14.222+0000 7fa1b7200640 1 -- 172.21.15.47:0/2230171484 >> 172.21.15.47:0/2230171484 conn(0x7fa1b80ff1b0 msgr2=0x7fa1b81015d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:26:14.224 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:14.224+0000 7fa1b7200640 1 -- 172.21.15.47:0/2230171484 shutdown_connections 2024-03-21T10:26:14.224 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:14.224+0000 7fa1b7200640 1 -- 172.21.15.47:0/2230171484 wait complete. 2024-03-21T10:26:14.225 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:14.225+0000 7fa1b7200640 1 Processor -- start 2024-03-21T10:26:14.227 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:14.225+0000 7fa1b7200640 1 -- start start 2024-03-21T10:26:14.227 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:14.226+0000 7fa1b7200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa1b8103720 0x7fa1b819a180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:26:14.227 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:14.226+0000 7fa1b7200640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa1b81040e0 0x7fa1b819a6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:26:14.227 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:14.226+0000 7fa1b7200640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa1b819ea40 0x7fa1b819eef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:26:14.227 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:14.226+0000 7fa1b7200640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1b8113a00 con 0x7fa1b819ea40 2024-03-21T10:26:14.227 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:14.226+0000 7fa1b7200640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1b8113880 con 0x7fa1b8103720 2024-03-21T10:26:14.227 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:14.226+0000 7fa1b7200640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1b8113b80 con 0x7fa1b81040e0 2024-03-21T10:26:14.227 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:14.226+0000 7fa1b5e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa1b8103720 0x7fa1b819a180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:26:14.227 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:14.226+0000 7fa1b5e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa1b8103720 0x7fa1b819a180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:41130/0 (socket says 172.21.15.47:41130) 2024-03-21T10:26:14.227 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:14.226+0000 7fa1b5e00640 1 -- 172.21.15.47:0/2495251814 learned_addr learned my addr 172.21.15.47:0/2495251814 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:26:14.227 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:14.226+0000 7fa1b6800640 1 --2- 172.21.15.47:0/2495251814 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa1b819ea40 0x7fa1b819eef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:26:14.227 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:14.227+0000 7fa1b5400640 1 --2- 172.21.15.47:0/2495251814 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa1b81040e0 0x7fa1b819a6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:26:14.228 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:14.227+0000 7fa1b5e00640 1 -- 172.21.15.47:0/2495251814 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa1b81040e0 msgr2=0x7fa1b819a6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:26:14.229 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:14.227+0000 7fa1b5e00640 1 --2- 172.21.15.47:0/2495251814 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa1b81040e0 0x7fa1b819a6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:26:14.229 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:14.227+0000 7fa1b5e00640 1 -- 172.21.15.47:0/2495251814 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa1b819ea40 msgr2=0x7fa1b819eef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:26:14.229 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:14.227+0000 7fa1b5e00640 1 --2- 172.21.15.47:0/2495251814 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa1b819ea40 0x7fa1b819eef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:26:14.229 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:14.227+0000 7fa1b5e00640 1 -- 172.21.15.47:0/2495251814 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa1b819f530 con 0x7fa1b8103720 2024-03-21T10:26:14.229 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:14.227+0000 7fa1b6800640 1 --2- 172.21.15.47:0/2495251814 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa1b819ea40 0x7fa1b819eef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T10:26:14.229 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:14.228+0000 7fa1b5e00640 1 --2- 172.21.15.47:0/2495251814 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa1b8103720 0x7fa1b819a180 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fa1a8009900 tx=0x7fa1a802c6e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:26:14.229 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:14.228+0000 7fa1aea00640 1 -- 172.21.15.47:0/2495251814 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1a8043020 con 0x7fa1b8103720 2024-03-21T10:26:14.229 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:14.228+0000 7fa1aea00640 1 -- 172.21.15.47:0/2495251814 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fa1a802cb70 con 0x7fa1b8103720 2024-03-21T10:26:14.229 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:14.228+0000 7fa1aea00640 1 -- 172.21.15.47:0/2495251814 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1a8003a50 con 0x7fa1b8103720 2024-03-21T10:26:14.229 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:14.228+0000 7fa1b7200640 1 -- 172.21.15.47:0/2495251814 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa1b819f7c0 con 0x7fa1b8103720 2024-03-21T10:26:14.230 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:14.229+0000 7fa1b7200640 1 -- 172.21.15.47:0/2495251814 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa1b806b300 con 0x7fa1b8103720 2024-03-21T10:26:14.234 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:14.233+0000 7fa1bff92740 1 Processor -- start 2024-03-21T10:26:14.234 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:14.234+0000 7fa1bff92740 1 -- start start 2024-03-21T10:26:14.235 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:14.234+0000 7fa1bff92740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a950652dd0 0x55a950651b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:26:14.235 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:14.234+0000 7fa1bff92740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a9506520f0 0x55a950652570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:26:14.235 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:14.234+0000 7fa1bff92740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a95065b460 0x55a95065d850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:26:14.235 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:14.234+0000 7fa1bff92740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a9505b6250 con 0x55a950652dd0 2024-03-21T10:26:14.235 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:14.234+0000 7fa1bff92740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a95055ea30 con 0x55a9506520f0 2024-03-21T10:26:14.235 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:14.234+0000 7fa1bff92740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a9505b22f0 con 0x55a95065b460 2024-03-21T10:26:14.235 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:14.234+0000 7fa1aea00640 1 -- 172.21.15.47:0/2495251814 <== mon.2 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 26) v1 ==== 138087+0+0 (secure 0 0 0) 0x7fa1a8046020 con 0x7fa1b8103720 2024-03-21T10:26:14.235 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:14.234+0000 7fa1b6800640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a950652dd0 0x55a950651b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:26:14.235 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:14.235+0000 7fa1b6800640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a950652dd0 0x55a950651b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:58424/0 (socket says 172.21.15.47:58424) 2024-03-21T10:26:14.235 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:14.235+0000 7fa1b6800640 1 -- 172.21.15.47:0/1195729315 learned_addr learned my addr 172.21.15.47:0/1195729315 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:26:14.236 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:14.235+0000 7fa1b6800640 1 -- 172.21.15.47:0/1195729315 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a95065b460 msgr2=0x55a95065d850 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-21T10:26:14.236 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:14.235+0000 7fa1b6800640 1 --2- 172.21.15.47:0/1195729315 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a95065b460 0x55a95065d850 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:26:14.236 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:14.235+0000 7fa1b6800640 1 -- 172.21.15.47:0/1195729315 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a9506520f0 msgr2=0x55a950652570 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-21T10:26:14.236 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:14.235+0000 7fa1b6800640 1 --2- 172.21.15.47:0/1195729315 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a9506520f0 0x55a950652570 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:26:14.236 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:14.235+0000 7fa1b6800640 1 -- 172.21.15.47:0/1195729315 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a950549690 con 0x55a950652dd0 2024-03-21T10:26:14.236 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:14.235+0000 7fa1b6800640 1 --2- 172.21.15.47:0/1195729315 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a950652dd0 0x55a950651b20 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7fa1b0011790 tx=0x7fa1b0011c50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e64f8bc7c7893c8b server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:26:14.237 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:14.236+0000 7fa1a3400640 1 -- 172.21.15.47:0/1195729315 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1b0003c30 con 0x55a950652dd0 2024-03-21T10:26:14.237 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:14.236+0000 7fa1a3400640 1 -- 172.21.15.47:0/1195729315 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fa1b000fe50 con 0x55a950652dd0 2024-03-21T10:26:14.237 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:14.236+0000 7fa1a3400640 1 -- 172.21.15.47:0/1195729315 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1b00193e0 con 0x55a950652dd0 2024-03-21T10:26:14.237 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:14.236+0000 7fa1aea00640 1 --2- 172.21.15.47:0/2495251814 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7fa19c0a31b0 0x7fa19c0a5670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:26:14.237 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:14.237+0000 7fa1aea00640 1 -- 172.21.15.47:0/2495251814 <== mon.2 v2:172.21.15.100:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 9474+0+0 (secure 0 0 0) 0x7fa1a80bb780 con 0x7fa1b8103720 2024-03-21T10:26:14.237 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:14.237+0000 7fa1b5e00640 1 --2- 172.21.15.47:0/2495251814 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7fa19c0a31b0 0x7fa19c0a5670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:26:14.238 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:14.237+0000 7fa1bff92740 1 -- 172.21.15.47:0/1195729315 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a950652dd0 msgr2=0x55a950651b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:26:14.238 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:14.237+0000 7fa1bff92740 1 --2- 172.21.15.47:0/1195729315 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a950652dd0 0x55a950651b20 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7fa1b0011790 tx=0x7fa1b0011c50 comp rx=0 tx=0).stop 2024-03-21T10:26:14.238 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:14.237+0000 7fa1bff92740 1 -- 172.21.15.47:0/1195729315 shutdown_connections 2024-03-21T10:26:14.238 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:14.237+0000 7fa1bff92740 1 --2- 172.21.15.47:0/1195729315 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a95065b460 0x55a95065d850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:26:14.238 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:14.237+0000 7fa1bff92740 1 --2- 172.21.15.47:0/1195729315 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a9506520f0 0x55a950652570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:26:14.239 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:14.237+0000 7fa1bff92740 1 --2- 172.21.15.47:0/1195729315 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a950652dd0 0x55a950651b20 unknown :-1 s=CLOSED pgs=587 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:26:14.239 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:14.237+0000 7fa1bff92740 1 -- 172.21.15.47:0/1195729315 >> 172.21.15.47:0/1195729315 conn(0x55a950619200 msgr2=0x55a95064cb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:26:14.239 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:14.238+0000 7fa1bff92740 1 -- 172.21.15.47:0/1195729315 shutdown_connections 2024-03-21T10:26:14.239 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:14.238+0000 7fa1b5e00640 1 --2- 172.21.15.47:0/2495251814 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7fa19c0a31b0 0x7fa19c0a5670 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fa1a80eeae0 tx=0x7fa1a80c1a20 comp rx=0 tx=0).ready entity=mgr.15687 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:26:14.239 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:14.238+0000 7fa1bff92740 1 -- 172.21.15.47:0/1195729315 wait complete. 2024-03-21T10:26:14.240 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:14.238+0000 7fa1bff92740 1 -- start start 2024-03-21T10:26:14.240 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:14.239+0000 7fa1bff92740 1 Processor -- start 2024-03-21T10:26:14.240 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:14.239+0000 7fa1bff92740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a9506520f0 0x55a95065ab60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:26:14.240 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:14.239+0000 7fa1bff92740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a950652dd0 0x55a950657ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:26:14.240 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:14.239+0000 7fa1bff92740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a95065b460 0x55a950658430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:26:14.240 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:14.239+0000 7fa1bff92740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a95065e2d0 con 0x55a9506520f0 2024-03-21T10:26:14.240 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:14.239+0000 7fa1bff92740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a950571e80 con 0x55a950652dd0 2024-03-21T10:26:14.240 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:14.240+0000 7fa1bff92740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a95065e450 con 0x55a95065b460 2024-03-21T10:26:14.240 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:14.240+0000 7fa1bff92740 10 client.0 _ll_register_callbacks cb 0x55a950584130 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T10:26:14.241 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:14.240+0000 7fa1b6800640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a9506520f0 0x55a95065ab60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:26:14.241 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:14.240+0000 7fa1b6800640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a9506520f0 0x55a95065ab60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:58438/0 (socket says 172.21.15.47:58438) 2024-03-21T10:26:14.241 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:14.240+0000 7fa1b6800640 1 -- 172.21.15.47:0/1637602644 learned_addr learned my addr 172.21.15.47:0/1637602644 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:26:14.241 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:14.240+0000 7fa1b5400640 1 --2- 172.21.15.47:0/1637602644 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a950652dd0 0x55a950657ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:26:14.242 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:14.240+0000 7fa1b6800640 1 --2- 172.21.15.47:0/1637602644 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a95065b460 0x55a950658430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:26:14.242 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:14.240+0000 7fa1b6800640 1 -- 172.21.15.47:0/1637602644 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a95065b460 msgr2=0x55a950658430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:26:14.242 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:14.240+0000 7fa1b6800640 1 --2- 172.21.15.47:0/1637602644 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a95065b460 0x55a950658430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:26:14.242 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:14.241+0000 7fa1b6800640 1 -- 172.21.15.47:0/1637602644 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a950652dd0 msgr2=0x55a950657ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:26:14.242 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:14.241+0000 7fa1b6800640 1 --2- 172.21.15.47:0/1637602644 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a950652dd0 0x55a950657ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:26:14.242 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:14.241+0000 7fa1b6800640 1 -- 172.21.15.47:0/1637602644 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55a9503947d0 con 0x55a9506520f0 2024-03-21T10:26:14.242 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:14.241+0000 7fa1b6800640 1 --2- 172.21.15.47:0/1637602644 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a9506520f0 0x55a95065ab60 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7fa1b000fe20 tx=0x7fa1b0004500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=106d861e4b4ab2d2 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:26:14.242 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:14.241+0000 7fa197e00640 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-03-21T10:26:14.243 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:14.241+0000 7fa197e00640 1 -- 172.21.15.47:0/1637602644 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1b0003e40 con 0x55a9506520f0 2024-03-21T10:26:14.243 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:14.241+0000 7fa197e00640 1 -- 172.21.15.47:0/1637602644 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 9474+0+0 (secure 0 0 0) 0x7fa1b0003fe0 con 0x55a9506520f0 2024-03-21T10:26:14.243 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:14.242+0000 7fa1bff92740 1 -- 172.21.15.47:0/1637602644 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fa1b000fe50 con 0x55a9506520f0 2024-03-21T10:26:14.243 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:14.242+0000 7fa197e00640 1 -- 172.21.15.47:0/1637602644 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa1b0004200 con 0x55a9506520f0 2024-03-21T10:26:14.243 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:14.242+0000 7fa1bff92740 10 client.15984 fetch_fsmap learned FSMap version 30 2024-03-21T10:26:14.243 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:14.242+0000 7fa1bff92740 1 -- 172.21.15.47:0/1637602644 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x55a9502ca3f0 con 0x55a9506520f0 2024-03-21T10:26:14.243 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:14.242+0000 7fa197e00640 1 -- 172.21.15.47:0/1637602644 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fa1b000fe50 con 0x55a9506520f0 2024-03-21T10:26:14.244 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:14.243+0000 7fa197e00640 1 -- 172.21.15.47:0/1637602644 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 30) v1 ==== 7942+0+0 (secure 0 0 0) 0x7fa1b0018070 con 0x55a9506520f0 2024-03-21T10:26:14.244 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:14.243+0000 7fa1bff92740 10 client.15984 fetch_fsmap finished waiting for FSMap version 30 2024-03-21T10:26:14.244 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:14.243+0000 7fa1bff92740 10 client.15984 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-21T10:26:14.244 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:14.243+0000 7fa1bff92740 20 client.15984 populate_metadata read hostname 'smithi047' 2024-03-21T10:26:14.244 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:14.244+0000 7fa1bff92740 1 --2- 172.21.15.47:0/1637602644 >> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] conn(0x55a95066b240 0x55a95066d630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:26:14.245 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:14.244+0000 7fa1bff92740 4 client.15984 mds_command: new command op to 34535 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T10:26:14.245 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:14.244+0000 7fa1bff92740 1 -- 172.21.15.47:0/1637602644 --> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55a9505c3790 con 0x55a95066b240 2024-03-21T10:26:14.245 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:14.244+0000 7fa1b5400640 1 --2- 172.21.15.47:0/1637602644 >> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] conn(0x55a95066b240 0x55a95066d630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:26:14.245 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:14.245+0000 7fa1b5400640 1 --2- 172.21.15.47:0/1637602644 >> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] conn(0x55a95066b240 0x55a95066d630 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4714c683ad97fc05 server_cookie=66007a2bbaa943f2 in_seq=0 out_seq=0 2024-03-21T10:26:14.245 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:14.245+0000 7fa197e00640 10 client.15984 ms_handle_connect on v2:172.21.15.100:6838/4173168440 2024-03-21T10:26:14.246 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:14.246+0000 7fa197e00640 1 -- 172.21.15.47:0/1637602644 <== mds.0 v2:172.21.15.100:6838/4173168440 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55a9505c3790 con 0x55a95066b240 2024-03-21T10:26:14.247 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:14.246+0000 7fa197e00640 10 client.15984 handle_command_reply: tid=0 2024-03-21T10:26:14.247 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:14.246+0000 7fa1bff92740 1 client.15984 shutdown 2024-03-21T10:26:14.248 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:14.247+0000 7fa1bff92740 1 -- 172.21.15.47:0/1637602644 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a9506520f0 msgr2=0x55a95065ab60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:26:14.248 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:14.247+0000 7fa1bff92740 1 --2- 172.21.15.47:0/1637602644 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a9506520f0 0x55a95065ab60 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7fa1b000fe20 tx=0x7fa1b0004500 comp rx=0 tx=0).stop 2024-03-21T10:26:14.248 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:14.247+0000 7fa1bff92740 1 -- 172.21.15.47:0/1637602644 shutdown_connections 2024-03-21T10:26:14.248 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:14.247+0000 7fa1bff92740 1 --2- 172.21.15.47:0/1637602644 >> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] conn(0x55a95066b240 0x55a95066d630 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:26:14.248 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:14.247+0000 7fa1bff92740 1 --2- 172.21.15.47:0/1637602644 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a95065b460 0x55a950658430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:26:14.248 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:14.247+0000 7fa1bff92740 1 --2- 172.21.15.47:0/1637602644 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a950652dd0 0x55a950657ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:26:14.248 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:14.247+0000 7fa1bff92740 1 --2- 172.21.15.47:0/1637602644 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a9506520f0 0x55a95065ab60 unknown :-1 s=CLOSED pgs=588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:26:14.248 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:14.247+0000 7fa1bff92740 1 -- 172.21.15.47:0/1637602644 >> 172.21.15.47:0/1637602644 conn(0x55a950619200 msgr2=0x55a95064b330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:26:14.248 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:14.247+0000 7fa197e00640 0 client.15984 ms_handle_reset on v2:172.21.15.100:6838/4173168440 2024-03-21T10:26:14.248 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:14.247+0000 7fa1bff92740 1 -- 172.21.15.47:0/1637602644 shutdown_connections 2024-03-21T10:26:14.248 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:14.247+0000 7fa1bff92740 1 -- 172.21.15.47:0/1637602644 wait complete. 2024-03-21T10:26:14.249 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:14.247+0000 7fa1bff92740 20 client.15984 trim_cache size 0 max 16384 2024-03-21T10:26:14.269 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:14.269+0000 7fa1bff92740 1 Processor -- start 2024-03-21T10:26:14.269 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:14.269+0000 7fa1bff92740 1 -- start start 2024-03-21T10:26:14.270 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:14.270+0000 7fa1bff92740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a9506520f0 0x55a950656ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:26:14.270 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:14.270+0000 7fa1bff92740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a950652dd0 0x55a95065a620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:26:14.270 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:14.270+0000 7fa1bff92740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a95065b460 0x55a95065ab60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:26:14.270 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:14.270+0000 7fa1bff92740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1b81b96c0 con 0x55a950652dd0 2024-03-21T10:26:14.270 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:14.270+0000 7fa1bff92740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1b81bb040 con 0x55a95065b460 2024-03-21T10:26:14.270 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:14.270+0000 7fa1bff92740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1b81b9d20 con 0x55a9506520f0 2024-03-21T10:26:14.272 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:14.270+0000 7fa1b6800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a95065b460 0x55a95065ab60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:26:14.272 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:14.270+0000 7fa1b6800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a95065b460 0x55a95065ab60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:41152/0 (socket says 172.21.15.47:41152) 2024-03-21T10:26:14.272 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:14.270+0000 7fa1b6800640 1 -- 172.21.15.47:0/3632367609 learned_addr learned my addr 172.21.15.47:0/3632367609 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:26:14.272 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:14.270+0000 7fa1b5400640 1 --2- 172.21.15.47:0/3632367609 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a950652dd0 0x55a95065a620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:26:14.272 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:14.270+0000 7fa1b6800640 1 --2- 172.21.15.47:0/3632367609 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a9506520f0 0x55a950656ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:26:14.272 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:14.271+0000 7fa1b6800640 1 -- 172.21.15.47:0/3632367609 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a9506520f0 msgr2=0x55a950656ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:26:14.272 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:14.271+0000 7fa1b6800640 1 --2- 172.21.15.47:0/3632367609 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a9506520f0 0x55a950656ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:26:14.272 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:14.271+0000 7fa1b6800640 1 -- 172.21.15.47:0/3632367609 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a950652dd0 msgr2=0x55a95065a620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:26:14.272 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:14.271+0000 7fa1b6800640 1 --2- 172.21.15.47:0/3632367609 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a950652dd0 0x55a95065a620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:26:14.272 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:14.271+0000 7fa1b6800640 1 -- 172.21.15.47:0/3632367609 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a95028a0f0 con 0x55a95065b460 2024-03-21T10:26:14.272 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:14.271+0000 7fa1b6800640 1 --2- 172.21.15.47:0/3632367609 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a95065b460 0x55a95065ab60 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fa1b0011790 tx=0x7fa1b0014040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=661a04c253f6570d server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:26:14.272 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:14.271+0000 7fa1a2a00640 1 -- 172.21.15.47:0/3632367609 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1b0003e40 con 0x55a95065b460 2024-03-21T10:26:14.273 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:14.271+0000 7fa1a2a00640 1 -- 172.21.15.47:0/3632367609 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fa1b000fe50 con 0x55a95065b460 2024-03-21T10:26:14.273 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:14.271+0000 7fa1a2a00640 1 -- 172.21.15.47:0/3632367609 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1b001a780 con 0x55a95065b460 2024-03-21T10:26:14.274 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:14.272+0000 7fa1bff92740 1 -- 172.21.15.47:0/3632367609 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a95065b460 msgr2=0x55a95065ab60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:26:14.274 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:14.272+0000 7fa1bff92740 1 --2- 172.21.15.47:0/3632367609 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a95065b460 0x55a95065ab60 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fa1b0011790 tx=0x7fa1b0014040 comp rx=0 tx=0).stop 2024-03-21T10:26:14.274 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:14.272+0000 7fa1bff92740 1 -- 172.21.15.47:0/3632367609 shutdown_connections 2024-03-21T10:26:14.274 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:14.272+0000 7fa1bff92740 1 --2- 172.21.15.47:0/3632367609 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a95065b460 0x55a95065ab60 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:26:14.274 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:14.272+0000 7fa1bff92740 1 --2- 172.21.15.47:0/3632367609 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a950652dd0 0x55a95065a620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:26:14.275 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:14.272+0000 7fa1bff92740 1 --2- 172.21.15.47:0/3632367609 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a9506520f0 0x55a950656ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:26:14.275 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:14.272+0000 7fa1bff92740 1 -- 172.21.15.47:0/3632367609 >> 172.21.15.47:0/3632367609 conn(0x55a950619200 msgr2=0x55a9506567c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:26:14.275 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:14.272+0000 7fa1bff92740 1 -- 172.21.15.47:0/3632367609 shutdown_connections 2024-03-21T10:26:14.275 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:14.272+0000 7fa1bff92740 1 -- 172.21.15.47:0/3632367609 wait complete. 2024-03-21T10:26:14.275 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:14.273+0000 7fa1bff92740 1 -- start start 2024-03-21T10:26:14.275 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:14.273+0000 7fa1bff92740 1 Processor -- start 2024-03-21T10:26:14.275 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:14.273+0000 7fa1bff92740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a9506520f0 0x55a950663130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:26:14.275 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:14.274+0000 7fa1bff92740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a950652dd0 0x55a950663670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:26:14.275 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:14.274+0000 7fa1bff92740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a95065b460 0x55a95065e880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:26:14.275 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:14.274+0000 7fa1bff92740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1b81ba9e0 con 0x55a95065b460 2024-03-21T10:26:14.275 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:14.274+0000 7fa1bff92740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1b81ba380 con 0x55a950652dd0 2024-03-21T10:26:14.275 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:14.274+0000 7fa1bff92740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1b8073450 con 0x55a9506520f0 2024-03-21T10:26:14.275 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:14.274+0000 7fa1bff92740 10 client.0 _ll_register_callbacks cb 0x7fa1b0011660 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T10:26:14.275 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:14.274+0000 7fa1b6800640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a95065b460 0x55a95065e880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:26:14.276 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:14.274+0000 7fa1b6800640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a95065b460 0x55a95065e880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:58462/0 (socket says 172.21.15.47:58462) 2024-03-21T10:26:14.276 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:14.274+0000 7fa1b6800640 1 -- 172.21.15.47:0/515418028 learned_addr learned my addr 172.21.15.47:0/515418028 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:26:14.276 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:14.274+0000 7fa1b5400640 1 --2- 172.21.15.47:0/515418028 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a950652dd0 0x55a950663670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:26:14.277 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:14.274+0000 7fa1b6800640 1 -- 172.21.15.47:0/515418028 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a9506520f0 msgr2=0x55a950663130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:26:14.277 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:14.274+0000 7fa1b6800640 1 --2- 172.21.15.47:0/515418028 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a9506520f0 0x55a950663130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:26:14.277 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:14.274+0000 7fa1b6800640 1 -- 172.21.15.47:0/515418028 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a950652dd0 msgr2=0x55a950663670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:26:14.278 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:14.274+0000 7fa1b6800640 1 --2- 172.21.15.47:0/515418028 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a950652dd0 0x55a950663670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:26:14.278 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:14.274+0000 7fa1b6800640 1 -- 172.21.15.47:0/515418028 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55a95065f0b0 con 0x55a95065b460 2024-03-21T10:26:14.278 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:14.275+0000 7fa1b6800640 1 --2- 172.21.15.47:0/515418028 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a95065b460 0x55a95065e880 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7fa1b001a490 tx=0x7fa1b0003970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1d59374a23157acb server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:26:14.278 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:14.275+0000 7fa197e00640 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-03-21T10:26:14.278 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:14.275+0000 7fa197e00640 1 -- 172.21.15.47:0/515418028 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1b003b040 con 0x55a95065b460 2024-03-21T10:26:14.278 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:14.275+0000 7fa197e00640 1 -- 172.21.15.47:0/515418028 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 9474+0+0 (secure 0 0 0) 0x7fa1b001aa30 con 0x55a95065b460 2024-03-21T10:26:14.278 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:14.275+0000 7fa1bff92740 1 -- 172.21.15.47:0/515418028 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fa1b000fe50 con 0x55a95065b460 2024-03-21T10:26:14.278 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:14.276+0000 7fa197e00640 1 -- 172.21.15.47:0/515418028 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa1b001ac30 con 0x55a95065b460 2024-03-21T10:26:14.278 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:14.276+0000 7fa1bff92740 10 client.15990 fetch_fsmap learned FSMap version 30 2024-03-21T10:26:14.278 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:14.276+0000 7fa1bff92740 1 -- 172.21.15.47:0/515418028 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x55a9506c6580 con 0x55a95065b460 2024-03-21T10:26:14.278 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:14.276+0000 7fa197e00640 1 -- 172.21.15.47:0/515418028 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fa1b000fe50 con 0x55a95065b460 2024-03-21T10:26:14.278 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:14.276+0000 7fa197e00640 1 -- 172.21.15.47:0/515418028 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 30) v1 ==== 7942+0+0 (secure 0 0 0) 0x7fa1b0018070 con 0x55a95065b460 2024-03-21T10:26:14.278 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:14.276+0000 7fa1bff92740 10 client.15990 fetch_fsmap finished waiting for FSMap version 30 2024-03-21T10:26:14.279 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:14.276+0000 7fa1bff92740 10 client.15990 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-21T10:26:14.279 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:14.276+0000 7fa1bff92740 20 client.15990 populate_metadata read hostname 'smithi047' 2024-03-21T10:26:14.279 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:14.277+0000 7fa1bff92740 1 --2- 172.21.15.47:0/515418028 >> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] conn(0x55a9506c6be0 0x55a9506c9000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:26:14.279 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:14.277+0000 7fa1bff92740 4 client.15990 mds_command: new command op to 34535 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-21T10:26:14.279 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:14.277+0000 7fa1bff92740 1 -- 172.21.15.47:0/515418028 --> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55a9505bdae0 con 0x55a9506c6be0 2024-03-21T10:26:14.279 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:14.277+0000 7fa1b5400640 1 --2- 172.21.15.47:0/515418028 >> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] conn(0x55a9506c6be0 0x55a9506c9000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:26:14.280 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:14.278+0000 7fa1b5400640 1 --2- 172.21.15.47:0/515418028 >> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] conn(0x55a9506c6be0 0x55a9506c9000 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c270a38c58434267 server_cookie=ec7333064c6c9668 in_seq=0 out_seq=0 2024-03-21T10:26:14.280 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:14.278+0000 7fa197e00640 10 client.15990 ms_handle_connect on v2:172.21.15.100:6838/4173168440 2024-03-21T10:26:14.280 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:14.278+0000 7fa197e00640 1 -- 172.21.15.47:0/515418028 <== mds.0 v2:172.21.15.100:6838/4173168440 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x55a9505bdae0 con 0x55a9506c6be0 2024-03-21T10:26:14.281 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:14.278+0000 7fa197e00640 10 client.15990 handle_command_reply: tid=0 2024-03-21T10:26:14.281 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:14.278+0000 7fa1bff92740 1 client.15990 shutdown 2024-03-21T10:26:14.281 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:14.278+0000 7fa1bff92740 1 -- 172.21.15.47:0/515418028 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a95065b460 msgr2=0x55a95065e880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:26:14.281 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:14.278+0000 7fa1bff92740 1 --2- 172.21.15.47:0/515418028 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a95065b460 0x55a95065e880 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7fa1b001a490 tx=0x7fa1b0003970 comp rx=0 tx=0).stop 2024-03-21T10:26:14.281 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:14.278+0000 7fa1bff92740 1 -- 172.21.15.47:0/515418028 shutdown_connections 2024-03-21T10:26:14.281 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:14.279+0000 7fa1bff92740 1 --2- 172.21.15.47:0/515418028 >> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] conn(0x55a9506c6be0 0x55a9506c9000 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:26:14.281 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:14.279+0000 7fa1bff92740 1 --2- 172.21.15.47:0/515418028 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a95065b460 0x55a95065e880 unknown :-1 s=CLOSED pgs=589 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:26:14.281 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:14.279+0000 7fa197e00640 0 client.15990 ms_handle_reset on v2:172.21.15.100:6838/4173168440 2024-03-21T10:26:14.281 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:14.279+0000 7fa1bff92740 1 --2- 172.21.15.47:0/515418028 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a950652dd0 0x55a950663670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:26:14.281 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:14.279+0000 7fa1bff92740 1 --2- 172.21.15.47:0/515418028 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a9506520f0 0x55a950663130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:26:14.281 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:14.279+0000 7fa1bff92740 1 -- 172.21.15.47:0/515418028 >> 172.21.15.47:0/515418028 conn(0x55a950619200 msgr2=0x55a95064b330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:26:14.281 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:14.279+0000 7fa1bff92740 1 -- 172.21.15.47:0/515418028 shutdown_connections 2024-03-21T10:26:14.281 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:14.279+0000 7fa1bff92740 1 -- 172.21.15.47:0/515418028 wait complete. 2024-03-21T10:26:14.281 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:14.279+0000 7fa1bff92740 20 client.15990 trim_cache size 0 max 16384 2024-03-21T10:26:14.282 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-03-21T10:26:14.282 INFO:teuthology.orchestra.run.smithi047.stdout: "status": "no active scrubs running", 2024-03-21T10:26:14.282 INFO:teuthology.orchestra.run.smithi047.stdout: "scrubs": {} 2024-03-21T10:26:14.282 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-03-21T10:26:14.283 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:14.280+0000 7fa1b7200640 1 -- 172.21.15.47:0/2495251814 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7fa19c0a31b0 msgr2=0x7fa19c0a5670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:26:14.283 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:14.280+0000 7fa1b7200640 1 --2- 172.21.15.47:0/2495251814 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7fa19c0a31b0 0x7fa19c0a5670 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fa1a80eeae0 tx=0x7fa1a80c1a20 comp rx=0 tx=0).stop 2024-03-21T10:26:14.283 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:14.280+0000 7fa1b7200640 1 -- 172.21.15.47:0/2495251814 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa1b8103720 msgr2=0x7fa1b819a180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:26:14.283 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:14.280+0000 7fa1b7200640 1 --2- 172.21.15.47:0/2495251814 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa1b8103720 0x7fa1b819a180 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fa1a8009900 tx=0x7fa1a802c6e0 comp rx=0 tx=0).stop 2024-03-21T10:26:14.283 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:14.280+0000 7fa1b7200640 1 -- 172.21.15.47:0/2495251814 shutdown_connections 2024-03-21T10:26:14.283 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:14.280+0000 7fa1b7200640 1 --2- 172.21.15.47:0/2495251814 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa1b819ea40 0x7fa1b819eef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:26:14.283 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:14.280+0000 7fa1b7200640 1 --2- 172.21.15.47:0/2495251814 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa1b81040e0 0x7fa1b819a6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:26:14.283 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:14.280+0000 7fa1b7200640 1 --2- 172.21.15.47:0/2495251814 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7fa19c0a31b0 0x7fa19c0a5670 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:26:14.283 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:14.280+0000 7fa1b7200640 1 --2- 172.21.15.47:0/2495251814 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa1b8103720 0x7fa1b819a180 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:26:14.283 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:14.280+0000 7fa1b7200640 1 -- 172.21.15.47:0/2495251814 >> 172.21.15.47:0/2495251814 conn(0x7fa1b80ff1b0 msgr2=0x7fa1b80ffb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:26:14.283 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:14.280+0000 7fa1b7200640 1 -- 172.21.15.47:0/2495251814 shutdown_connections 2024-03-21T10:26:14.283 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:14.280+0000 7fa1b7200640 1 -- 172.21.15.47:0/2495251814 wait complete. 2024-03-21T10:26:14.303 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-03-21T10:26:14.303 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-21T10:26:14.506 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:14.505+0000 7f8024a00640 1 -- 172.21.15.47:0/843365277 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f802010dd50 msgr2=0x7f8020110140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:26:14.506 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:14.505+0000 7f8024a00640 1 --2- 172.21.15.47:0/843365277 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f802010dd50 0x7f8020110140 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7f8018009ca0 tx=0x7f801802c180 comp rx=0 tx=0).stop 2024-03-21T10:26:14.507 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:14.506+0000 7f8024a00640 1 -- 172.21.15.47:0/843365277 shutdown_connections 2024-03-21T10:26:14.507 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:14.506+0000 7f8024a00640 1 --2- 172.21.15.47:0/843365277 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f802010dd50 0x7f8020110140 unknown :-1 s=CLOSED pgs=590 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:26:14.507 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:14.506+0000 7f8024a00640 1 --2- 172.21.15.47:0/843365277 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8020074370 0x7f802010d810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:26:14.507 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:14.506+0000 7f8024a00640 1 --2- 172.21.15.47:0/843365277 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8020073a50 0x7f8020073e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:26:14.507 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:14.506+0000 7f8024a00640 1 -- 172.21.15.47:0/843365277 >> 172.21.15.47:0/843365277 conn(0x7f80200fcf80 msgr2=0x7f80200ff3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:26:14.507 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:14.506+0000 7f8024a00640 1 -- 172.21.15.47:0/843365277 shutdown_connections 2024-03-21T10:26:14.507 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:14.506+0000 7f8024a00640 1 -- 172.21.15.47:0/843365277 wait complete. 2024-03-21T10:26:14.507 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:14.507+0000 7f8024a00640 1 Processor -- start 2024-03-21T10:26:14.508 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:14.507+0000 7f8024a00640 1 -- start start 2024-03-21T10:26:14.509 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:14.508+0000 7f8024a00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8020073a50 0x7f8020199ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:26:14.509 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:14.508+0000 7f8024a00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8020074370 0x7f802019a420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:26:14.509 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:14.508+0000 7f8024a00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f802010dd50 0x7f802019e7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:26:14.509 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:14.508+0000 7f8024a00640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8020113920 con 0x7f802010dd50 2024-03-21T10:26:14.509 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:14.508+0000 7f8024a00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80201137a0 con 0x7f8020073a50 2024-03-21T10:26:14.509 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:14.508+0000 7f8024a00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8020113aa0 con 0x7f8020074370 2024-03-21T10:26:14.509 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:14.508+0000 7f801f200640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f802010dd50 0x7f802019e7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:26:14.509 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:14.508+0000 7f801f200640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f802010dd50 0x7f802019e7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:58466/0 (socket says 172.21.15.47:58466) 2024-03-21T10:26:14.509 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:14.508+0000 7f801f200640 1 -- 172.21.15.47:0/1120145976 learned_addr learned my addr 172.21.15.47:0/1120145976 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:26:14.509 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:14.508+0000 7f801e800640 1 --2- 172.21.15.47:0/1120145976 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8020073a50 0x7f8020199ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:26:14.510 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:14.509+0000 7f801de00640 1 --2- 172.21.15.47:0/1120145976 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8020074370 0x7f802019a420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:26:14.510 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:14.509+0000 7f801f200640 1 -- 172.21.15.47:0/1120145976 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8020074370 msgr2=0x7f802019a420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:26:14.510 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:14.509+0000 7f801f200640 1 --2- 172.21.15.47:0/1120145976 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8020074370 0x7f802019a420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:26:14.510 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:14.509+0000 7f801f200640 1 -- 172.21.15.47:0/1120145976 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8020073a50 msgr2=0x7f8020199ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:26:14.510 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:14.509+0000 7f801f200640 1 --2- 172.21.15.47:0/1120145976 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8020073a50 0x7f8020199ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:26:14.510 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:14.509+0000 7f801f200640 1 -- 172.21.15.47:0/1120145976 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f802019eed0 con 0x7f802010dd50 2024-03-21T10:26:14.511 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:14.509+0000 7f801f200640 1 --2- 172.21.15.47:0/1120145976 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f802010dd50 0x7f802019e7a0 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7f8018037040 tx=0x7f8018035d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:26:14.511 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:14.510+0000 7f8017400640 1 -- 172.21.15.47:0/1120145976 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8018039040 con 0x7f802010dd50 2024-03-21T10:26:14.511 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:14.510+0000 7f8017400640 1 -- 172.21.15.47:0/1120145976 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f8018044070 con 0x7f802010dd50 2024-03-21T10:26:14.511 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:14.510+0000 7f8017400640 1 -- 172.21.15.47:0/1120145976 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8018032850 con 0x7f802010dd50 2024-03-21T10:26:14.511 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:14.510+0000 7f8024a00640 1 -- 172.21.15.47:0/1120145976 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f802019f160 con 0x7f802010dd50 2024-03-21T10:26:14.511 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:14.510+0000 7f8024a00640 1 -- 172.21.15.47:0/1120145976 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f80201a6a50 con 0x7f802010dd50 2024-03-21T10:26:14.512 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:14.511+0000 7f8024a00640 1 -- 172.21.15.47:0/1120145976 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f80200683b0 con 0x7f802010dd50 2024-03-21T10:26:14.515 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:14.515+0000 7f8017400640 1 -- 172.21.15.47:0/1120145976 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 26) v1 ==== 138087+0+0 (secure 0 0 0) 0x7f80180329f0 con 0x7f802010dd50 2024-03-21T10:26:14.516 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:14.516+0000 7f8017400640 1 --2- 172.21.15.47:0/1120145976 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f80040a3300 0x7f80040a57c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:26:14.517 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:14.516+0000 7f8017400640 1 -- 172.21.15.47:0/1120145976 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 9474+0+0 (secure 0 0 0) 0x7f80180f1aa0 con 0x7f802010dd50 2024-03-21T10:26:14.517 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:14.517+0000 7f801e800640 1 --2- 172.21.15.47:0/1120145976 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f80040a3300 0x7f80040a57c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:26:14.518 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:14.518+0000 7f801e800640 1 --2- 172.21.15.47:0/1120145976 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f80040a3300 0x7f80040a57c0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f80100095f0 tx=0x7f8010005f90 comp rx=0 tx=0).ready entity=mgr.15687 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:26:14.526 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:14.526+0000 7f8017400640 1 -- 172.21.15.47:0/1120145976 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f801804d020 con 0x7f802010dd50 2024-03-21T10:26:14.891 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:14.891+0000 7f8024a00640 1 -- 172.21.15.47:0/1120145976 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f8020108ed0 con 0x7f802010dd50 2024-03-21T10:26:14.892 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:14.892+0000 7f8017400640 1 -- 172.21.15.47:0/1120145976 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) v1 ==== 94+0+11471 (secure 0 0 0) 0x7f80180be030 con 0x7f802010dd50 2024-03-21T10:26:14.893 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-21T10:26:14.893 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":30,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14511,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6835/735724037","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6834","nonce":735724037},{"type":"v1","addr":"172.21.15.47:6835","nonce":735724037}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14529,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6837/2248191724","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6836","nonce":2248191724},{"type":"v1","addr":"172.21.15.47:6837","nonce":2248191724}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":30,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-21T10:21:58.329063+0000","modified":"2024-03-21T10:24:58.786909+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":34535,"mds_1":34493,"mds_2":34454,"mds_3":34427,"mds_4":14619},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14604":{"gid":14604,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.47:6839/2881335356","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6838","nonce":2881335356},{"type":"v1","addr":"172.21.15.47:6839","nonce":2881335356}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_14619":{"gid":14619,"name":"a","rank":4,"incarnation":27,"state":"up:active","state_seq":11,"addr":"172.21.15.47:6841/3401578978","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6840","nonce":3401578978},{"type":"v1","addr":"172.21.15.47:6841","nonce":3401578978}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24406":{"gid":24406,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.157:6833/4000010136","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":4000010136},{"type":"v1","addr":"172.21.15.157:6833","nonce":4000010136}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24457":{"gid":24457,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.157:6835/3902868651","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":3902868651},{"type":"v1","addr":"172.21.15.157:6835","nonce":3902868651}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24475":{"gid":24475,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.157:6837/1908087204","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6836","nonce":1908087204},{"type":"v1","addr":"172.21.15.157:6837","nonce":1908087204}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24499":{"gid":24499,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.157:6839/4064242876","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6838","nonce":4064242876},{"type":"v1","addr":"172.21.15.157:6839","nonce":4064242876}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_34427":{"gid":34427,"name":"k","rank":3,"incarnation":25,"state":"up:active","state_seq":20,"addr":"172.21.15.100:6833/2723302038","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6832","nonce":2723302038},{"type":"v1","addr":"172.21.15.100:6833","nonce":2723302038}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_34454":{"gid":34454,"name":"e","rank":2,"incarnation":22,"state":"up:active","state_seq":17,"addr":"172.21.15.100:6835/2597849957","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":2597849957},{"type":"v1","addr":"172.21.15.100:6835","nonce":2597849957}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_34493":{"gid":34493,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":13,"addr":"172.21.15.100:6837/3497644162","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6836","nonce":3497644162},{"type":"v1","addr":"172.21.15.100:6837","nonce":3497644162}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_34535":{"gid":34535,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.100:6839/4173168440","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6838","nonce":4173168440},{"type":"v1","addr":"172.21.15.100:6839","nonce":4173168440}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":34535,"qdb_cluster":[14619,34427,34454,34493,34535]},"id":1}]} 2024-03-21T10:26:14.893 INFO:teuthology.orchestra.run.smithi047.stderr:dumped fsmap epoch 30 2024-03-21T10:26:14.894 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:14.894+0000 7f8024a00640 1 -- 172.21.15.47:0/1120145976 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f80040a3300 msgr2=0x7f80040a57c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:26:14.894 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:14.894+0000 7f8024a00640 1 --2- 172.21.15.47:0/1120145976 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f80040a3300 0x7f80040a57c0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f80100095f0 tx=0x7f8010005f90 comp rx=0 tx=0).stop 2024-03-21T10:26:14.894 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:14.894+0000 7f8024a00640 1 -- 172.21.15.47:0/1120145976 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f802010dd50 msgr2=0x7f802019e7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:26:14.894 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:14.894+0000 7f8024a00640 1 --2- 172.21.15.47:0/1120145976 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f802010dd50 0x7f802019e7a0 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7f8018037040 tx=0x7f8018035d50 comp rx=0 tx=0).stop 2024-03-21T10:26:14.895 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:14.894+0000 7f8024a00640 1 -- 172.21.15.47:0/1120145976 shutdown_connections 2024-03-21T10:26:14.895 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:14.894+0000 7f8024a00640 1 --2- 172.21.15.47:0/1120145976 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f802010dd50 0x7f802019e7a0 unknown :-1 s=CLOSED pgs=591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:26:14.895 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:14.894+0000 7f8024a00640 1 --2- 172.21.15.47:0/1120145976 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8020074370 0x7f802019a420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:26:14.895 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:14.894+0000 7f8024a00640 1 --2- 172.21.15.47:0/1120145976 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f80040a3300 0x7f80040a57c0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:26:14.895 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:14.894+0000 7f8024a00640 1 --2- 172.21.15.47:0/1120145976 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8020073a50 0x7f8020199ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:26:14.896 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:14.894+0000 7f8024a00640 1 -- 172.21.15.47:0/1120145976 >> 172.21.15.47:0/1120145976 conn(0x7f80200fcf80 msgr2=0x7f80200fdcf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:26:14.896 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:14.895+0000 7f8024a00640 1 -- 172.21.15.47:0/1120145976 shutdown_connections 2024-03-21T10:26:14.896 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:14.895+0000 7f8024a00640 1 -- 172.21.15.47:0/1120145976 wait complete. 2024-03-21T10:26:14.915 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-21T10:26:14.915 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-03-21T10:26:15.133 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:15.133+0000 7f60f3200640 1 -- 172.21.15.47:0/2131637589 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f60ec073a70 msgr2=0x7f60ec073e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:26:15.134 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:15.133+0000 7f60f3200640 1 --2- 172.21.15.47:0/2131637589 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f60ec073a70 0x7f60ec073e50 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7f60e0009900 tx=0x7f60e002bcd0 comp rx=0 tx=0).stop 2024-03-21T10:26:15.134 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:15.133+0000 7f60f3200640 1 -- 172.21.15.47:0/2131637589 shutdown_connections 2024-03-21T10:26:15.134 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:15.133+0000 7f60f3200640 1 --2- 172.21.15.47:0/2131637589 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f60ec10ddb0 0x7f60ec1101c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:26:15.134 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:15.133+0000 7f60f3200640 1 --2- 172.21.15.47:0/2131637589 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f60ec074390 0x7f60ec10d870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:26:15.134 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:15.133+0000 7f60f3200640 1 --2- 172.21.15.47:0/2131637589 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f60ec073a70 0x7f60ec073e50 unknown :-1 s=CLOSED pgs=592 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:26:15.134 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:15.133+0000 7f60f3200640 1 -- 172.21.15.47:0/2131637589 >> 172.21.15.47:0/2131637589 conn(0x7f60ec0fd040 msgr2=0x7f60ec0ff460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:26:15.134 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:15.133+0000 7f60f3200640 1 -- 172.21.15.47:0/2131637589 shutdown_connections 2024-03-21T10:26:15.134 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:15.133+0000 7f60f3200640 1 -- 172.21.15.47:0/2131637589 wait complete. 2024-03-21T10:26:15.135 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:15.134+0000 7f60f3200640 1 Processor -- start 2024-03-21T10:26:15.135 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:15.135+0000 7f60f3200640 1 -- start start 2024-03-21T10:26:15.136 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:15.135+0000 7f60f3200640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f60ec073a70 0x7f60ec199fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:26:15.136 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:15.135+0000 7f60f3200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f60ec074390 0x7f60ec19a510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:26:15.136 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:15.135+0000 7f60f3200640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f60ec10ddb0 0x7f60ec19e840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:26:15.136 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:15.135+0000 7f60f3200640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60ec113a30 con 0x7f60ec073a70 2024-03-21T10:26:15.136 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:15.135+0000 7f60f3200640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60ec1138b0 con 0x7f60ec074390 2024-03-21T10:26:15.136 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:15.135+0000 7f60f3200640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60ec113bb0 con 0x7f60ec10ddb0 2024-03-21T10:26:15.137 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:15.136+0000 7f60f1400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f60ec074390 0x7f60ec19a510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:26:15.137 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:15.136+0000 7f60f1400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f60ec074390 0x7f60ec19a510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:41204/0 (socket says 172.21.15.47:41204) 2024-03-21T10:26:15.137 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:15.136+0000 7f60f1400640 1 -- 172.21.15.47:0/1944136240 learned_addr learned my addr 172.21.15.47:0/1944136240 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:26:15.137 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:15.136+0000 7f60f1e00640 1 --2- 172.21.15.47:0/1944136240 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f60ec073a70 0x7f60ec199fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:26:15.137 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:15.136+0000 7f60f2800640 1 --2- 172.21.15.47:0/1944136240 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f60ec10ddb0 0x7f60ec19e840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:26:15.137 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:15.136+0000 7f60f1400640 1 -- 172.21.15.47:0/1944136240 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f60ec10ddb0 msgr2=0x7f60ec19e840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:26:15.137 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:15.136+0000 7f60f1400640 1 --2- 172.21.15.47:0/1944136240 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f60ec10ddb0 0x7f60ec19e840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:26:15.137 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:15.136+0000 7f60f1400640 1 -- 172.21.15.47:0/1944136240 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f60ec073a70 msgr2=0x7f60ec199fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:26:15.137 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:15.137+0000 7f60f1400640 1 --2- 172.21.15.47:0/1944136240 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f60ec073a70 0x7f60ec199fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:26:15.137 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:15.137+0000 7f60f1400640 1 -- 172.21.15.47:0/1944136240 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f60ec19ef70 con 0x7f60ec074390 2024-03-21T10:26:15.138 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:15.137+0000 7f60f2800640 1 --2- 172.21.15.47:0/1944136240 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f60ec10ddb0 0x7f60ec19e840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T10:26:15.138 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:15.137+0000 7f60f1e00640 1 --2- 172.21.15.47:0/1944136240 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f60ec073a70 0x7f60ec199fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T10:26:15.138 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:15.137+0000 7f60f1400640 1 --2- 172.21.15.47:0/1944136240 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f60ec074390 0x7f60ec19a510 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f60dc00e9f0 tx=0x7f60dc00edf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:26:15.139 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:15.137+0000 7f60e6a00640 1 -- 172.21.15.47:0/1944136240 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60dc00cd80 con 0x7f60ec074390 2024-03-21T10:26:15.139 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:15.137+0000 7f60f3200640 1 -- 172.21.15.47:0/1944136240 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f60ec19f260 con 0x7f60ec074390 2024-03-21T10:26:15.139 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:15.138+0000 7f60e6a00640 1 -- 172.21.15.47:0/1944136240 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f60dc0054d0 con 0x7f60ec074390 2024-03-21T10:26:15.139 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:15.138+0000 7f60f3200640 1 -- 172.21.15.47:0/1944136240 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f60ec1a6af0 con 0x7f60ec074390 2024-03-21T10:26:15.139 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:15.138+0000 7f60e6a00640 1 -- 172.21.15.47:0/1944136240 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60dc010490 con 0x7f60ec074390 2024-03-21T10:26:15.142 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:15.142+0000 7f60e6a00640 1 -- 172.21.15.47:0/1944136240 <== mon.2 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 26) v1 ==== 138087+0+0 (secure 0 0 0) 0x7f60dc005020 con 0x7f60ec074390 2024-03-21T10:26:15.142 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:15.142+0000 7f60f744b740 1 Processor -- start 2024-03-21T10:26:15.143 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:15.142+0000 7f60f744b740 1 -- start start 2024-03-21T10:26:15.143 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:15.143+0000 7f60f744b740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b37768b8b0 0x55b37768bc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:26:15.143 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:15.143+0000 7f60f744b740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b37768c260 0x55b377694810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:26:15.143 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:15.143+0000 7f60f744b740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b377694de0 0x55b377697230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:26:15.143 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:15.143+0000 7f60f744b740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b3775ef2c0 con 0x55b37768c260 2024-03-21T10:26:15.144 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:15.143+0000 7f60f744b740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b377597c90 con 0x55b377694de0 2024-03-21T10:26:15.144 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:15.143+0000 7f60f744b740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b3775eb360 con 0x55b37768b8b0 2024-03-21T10:26:15.144 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:15.143+0000 7f60f2800640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b37768b8b0 0x55b37768bc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:26:15.145 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:15.144+0000 7f60f1400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b377694de0 0x55b377697230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:26:15.145 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:15.144+0000 7f60f1400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b377694de0 0x55b377697230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:41214/0 (socket says 172.21.15.47:41214) 2024-03-21T10:26:15.145 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:15.144+0000 7f60f1400640 1 -- 172.21.15.47:0/2401746106 learned_addr learned my addr 172.21.15.47:0/2401746106 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:26:15.145 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:15.144+0000 7f60e6a00640 1 --2- 172.21.15.47:0/1944136240 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f60d40a33b0 0x7f60d40a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:26:15.145 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:15.144+0000 7f60f1e00640 1 --2- 172.21.15.47:0/2401746106 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b37768c260 0x55b377694810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:26:15.145 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:15.144+0000 7f60e6a00640 1 -- 172.21.15.47:0/1944136240 <== mon.2 v2:172.21.15.100:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 9474+0+0 (secure 0 0 0) 0x7f60dc004e00 con 0x7f60ec074390 2024-03-21T10:26:15.145 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:15.144+0000 7f60f1e00640 1 -- 172.21.15.47:0/2401746106 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b37768b8b0 msgr2=0x55b37768bc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:26:15.145 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:15.144+0000 7f60f1e00640 1 --2- 172.21.15.47:0/2401746106 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b37768b8b0 0x55b37768bc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:26:15.145 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:15.144+0000 7f60f1e00640 1 -- 172.21.15.47:0/2401746106 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b377694de0 msgr2=0x55b377697230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:26:15.146 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:15.144+0000 7f60f1e00640 1 --2- 172.21.15.47:0/2401746106 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b377694de0 0x55b377697230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:26:15.146 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:15.144+0000 7f60f1e00640 1 -- 172.21.15.47:0/2401746106 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b3775bc080 con 0x55b37768c260 2024-03-21T10:26:15.146 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:15.144+0000 7f60f2800640 1 --2- 172.21.15.47:0/1944136240 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f60d40a33b0 0x7f60d40a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:26:15.147 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:15.145+0000 7f60f1e00640 1 --2- 172.21.15.47:0/2401746106 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b37768c260 0x55b377694810 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7f60e002c210 tx=0x7f60e00095e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4e3e38a574fa6732 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:26:15.147 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:15.145+0000 7f60f2800640 1 --2- 172.21.15.47:0/1944136240 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f60d40a33b0 0x7f60d40a5870 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f60ec06abd0 tx=0x7f60e8004ab0 comp rx=0 tx=0).ready entity=mgr.15687 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:26:15.147 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:15.145+0000 7f60db000640 1 -- 172.21.15.47:0/2401746106 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60e0043030 con 0x55b37768c260 2024-03-21T10:26:15.147 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:15.145+0000 7f60db000640 1 -- 172.21.15.47:0/2401746106 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f60e002c880 con 0x55b37768c260 2024-03-21T10:26:15.147 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:15.145+0000 7f60db000640 1 -- 172.21.15.47:0/2401746106 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60e0003770 con 0x55b37768c260 2024-03-21T10:26:15.147 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:15.146+0000 7f60f744b740 1 -- 172.21.15.47:0/2401746106 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b37768c260 msgr2=0x55b377694810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:26:15.147 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:15.146+0000 7f60f744b740 1 --2- 172.21.15.47:0/2401746106 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b37768c260 0x55b377694810 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7f60e002c210 tx=0x7f60e00095e0 comp rx=0 tx=0).stop 2024-03-21T10:26:15.147 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:15.147+0000 7f60f744b740 1 -- 172.21.15.47:0/2401746106 shutdown_connections 2024-03-21T10:26:15.148 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:15.147+0000 7f60f744b740 1 --2- 172.21.15.47:0/2401746106 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b377694de0 0x55b377697230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:26:15.148 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:15.147+0000 7f60f744b740 1 --2- 172.21.15.47:0/2401746106 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b37768c260 0x55b377694810 unknown :-1 s=CLOSED pgs=593 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:26:15.148 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:15.147+0000 7f60f744b740 1 --2- 172.21.15.47:0/2401746106 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b37768b8b0 0x55b37768bc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:26:15.148 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:15.147+0000 7f60f744b740 1 -- 172.21.15.47:0/2401746106 >> 172.21.15.47:0/2401746106 conn(0x55b377685f80 msgr2=0x55b377686360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:26:15.148 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:15.147+0000 7f60f744b740 1 -- 172.21.15.47:0/2401746106 shutdown_connections 2024-03-21T10:26:15.148 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:15.147+0000 7f60f744b740 1 -- 172.21.15.47:0/2401746106 wait complete. 2024-03-21T10:26:15.149 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:15.147+0000 7f60f744b740 1 -- start start 2024-03-21T10:26:15.149 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:15.148+0000 7f60f744b740 1 Processor -- start 2024-03-21T10:26:15.149 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:15.148+0000 7f60f744b740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b37768b8b0 0x55b377694740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:26:15.149 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:15.149+0000 7f60f744b740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b37768c260 0x55b377691ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:26:15.150 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:15.149+0000 7f60f744b740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b377694de0 0x55b3776920e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:26:15.150 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:15.149+0000 7f60f744b740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b377697cf0 con 0x55b377694de0 2024-03-21T10:26:15.150 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:15.149+0000 7f60f744b740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b3775ab0d0 con 0x55b37768c260 2024-03-21T10:26:15.150 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:15.149+0000 7f60f744b740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b377697e70 con 0x55b37768b8b0 2024-03-21T10:26:15.150 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:15.149+0000 7f60f744b740 10 client.0 _ll_register_callbacks cb 0x55b3775afe80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T10:26:15.150 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:15.149+0000 7f60f1e00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b37768b8b0 0x55b377694740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:26:15.150 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:15.149+0000 7f60f1400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b37768c260 0x55b377691ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:26:15.150 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:15.149+0000 7f60f1400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b37768c260 0x55b377691ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:41224/0 (socket says 172.21.15.47:41224) 2024-03-21T10:26:15.150 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:15.149+0000 7f60f1400640 1 -- 172.21.15.47:0/1500539618 learned_addr learned my addr 172.21.15.47:0/1500539618 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:26:15.150 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:15.150+0000 7f60f2800640 1 --2- 172.21.15.47:0/1500539618 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b377694de0 0x55b3776920e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:26:15.151 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:15.150+0000 7f60f1e00640 1 -- 172.21.15.47:0/1500539618 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b37768c260 msgr2=0x55b377691ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:26:15.151 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:15.150+0000 7f60f1e00640 1 --2- 172.21.15.47:0/1500539618 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b37768c260 0x55b377691ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:26:15.151 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:15.150+0000 7f60f1e00640 1 -- 172.21.15.47:0/1500539618 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b377694de0 msgr2=0x55b3776920e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:26:15.151 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:15.150+0000 7f60f1e00640 1 --2- 172.21.15.47:0/1500539618 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b377694de0 0x55b3776920e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:26:15.151 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:15.150+0000 7f60f1e00640 1 -- 172.21.15.47:0/1500539618 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55b3773033f0 con 0x55b37768b8b0 2024-03-21T10:26:15.152 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:15.151+0000 7f60f1e00640 1 --2- 172.21.15.47:0/1500539618 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b37768b8b0 0x55b377694740 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x55b377686ff0 tx=0x7f60e003f670 comp rx=0 tx=0).ready entity=mon.1 client_cookie=79ad692b68f36e21 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:26:15.152 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:15.151+0000 7f60cf400640 10 client.0 ms_handle_connect on v2:172.21.15.157:3300/0 2024-03-21T10:26:15.156 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:15.156+0000 7f60cf400640 1 -- 172.21.15.47:0/1500539618 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60e0043070 con 0x55b37768b8b0 2024-03-21T10:26:15.156 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:15.156+0000 7f60cf400640 1 -- 172.21.15.47:0/1500539618 <== mon.1 v2:172.21.15.157:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 9474+0+0 (secure 0 0 0) 0x7f60e003faf0 con 0x55b37768b8b0 2024-03-21T10:26:15.156 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:15.156+0000 7f60f744b740 1 -- 172.21.15.47:0/1500539618 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f60e002c880 con 0x55b37768b8b0 2024-03-21T10:26:15.157 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:15.156+0000 7f60cf400640 1 -- 172.21.15.47:0/1500539618 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f60e003fcf0 con 0x55b37768b8b0 2024-03-21T10:26:15.157 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:15.156+0000 7f60f744b740 10 client.25657 fetch_fsmap learned FSMap version 30 2024-03-21T10:26:15.157 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:15.156+0000 7f60f744b740 1 -- 172.21.15.47:0/1500539618 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x55b3773cd7d0 con 0x55b37768b8b0 2024-03-21T10:26:15.157 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:15.157+0000 7f60cf400640 1 -- 172.21.15.47:0/1500539618 <== mon.1 v2:172.21.15.157:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f60e00348f0 con 0x55b37768b8b0 2024-03-21T10:26:15.158 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:15.157+0000 7f60cf400640 1 -- 172.21.15.47:0/1500539618 <== mon.1 v2:172.21.15.157:3300/0 5 ==== fsmap(e 30) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f60e00465d0 con 0x55b37768b8b0 2024-03-21T10:26:15.158 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:15.157+0000 7f60f744b740 10 client.25657 fetch_fsmap finished waiting for FSMap version 30 2024-03-21T10:26:15.158 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:15.157+0000 7f60f744b740 10 client.25657 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.a 2024-03-21T10:26:15.158 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:15.157+0000 7f60f744b740 20 client.25657 populate_metadata read hostname 'smithi047' 2024-03-21T10:26:15.158 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:15.158+0000 7f60f744b740 1 --2- 172.21.15.47:0/1500539618 >> [v2:172.21.15.47:6840/3401578978,v1:172.21.15.47:6841/3401578978] conn(0x55b3776a4ad0 0x55b3776a6ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:26:15.158 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:15.158+0000 7f60f744b740 4 client.25657 mds_command: new command op to 14619 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T10:26:15.158 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:15.158+0000 7f60f744b740 1 -- 172.21.15.47:0/1500539618 --> [v2:172.21.15.47:6840/3401578978,v1:172.21.15.47:6841/3401578978] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7f60e002c880 con 0x55b3776a4ad0 2024-03-21T10:26:15.159 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:15.158+0000 7f60f1400640 1 --2- 172.21.15.47:0/1500539618 >> [v2:172.21.15.47:6840/3401578978,v1:172.21.15.47:6841/3401578978] conn(0x55b3776a4ad0 0x55b3776a6ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:26:15.159 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:15.159+0000 7f60f1400640 1 --2- 172.21.15.47:0/1500539618 >> [v2:172.21.15.47:6840/3401578978,v1:172.21.15.47:6841/3401578978] conn(0x55b3776a4ad0 0x55b3776a6ec0 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=cedea65f3b821925 server_cookie=fe08c780eca48500 in_seq=0 out_seq=0 2024-03-21T10:26:15.159 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:15.159+0000 7f60cf400640 10 client.25657 ms_handle_connect on v2:172.21.15.47:6840/3401578978 2024-03-21T10:26:15.160 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:15.160+0000 7f60cf400640 1 -- 172.21.15.47:0/1500539618 <== mds.4 v2:172.21.15.47:6840/3401578978 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x7f60e002c880 con 0x55b3776a4ad0 2024-03-21T10:26:15.160 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:15.160+0000 7f60cf400640 10 client.25657 handle_command_reply: tid=0 2024-03-21T10:26:15.160 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:15.160+0000 7f60f744b740 1 client.25657 shutdown 2024-03-21T10:26:15.161 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:15.160+0000 7f60f744b740 1 -- 172.21.15.47:0/1500539618 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b37768b8b0 msgr2=0x55b377694740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:26:15.161 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:15.160+0000 7f60f744b740 1 --2- 172.21.15.47:0/1500539618 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b37768b8b0 0x55b377694740 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x55b377686ff0 tx=0x7f60e003f670 comp rx=0 tx=0).stop 2024-03-21T10:26:15.161 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:15.160+0000 7f60f744b740 1 -- 172.21.15.47:0/1500539618 shutdown_connections 2024-03-21T10:26:15.161 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:15.160+0000 7f60f744b740 1 --2- 172.21.15.47:0/1500539618 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b377694de0 0x55b3776920e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:26:15.162 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:15.160+0000 7f60f744b740 1 --2- 172.21.15.47:0/1500539618 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b37768c260 0x55b377691ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:26:15.162 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:15.160+0000 7f60f744b740 1 --2- 172.21.15.47:0/1500539618 >> [v2:172.21.15.47:6840/3401578978,v1:172.21.15.47:6841/3401578978] conn(0x55b3776a4ad0 0x55b3776a6ec0 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:26:15.162 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:15.160+0000 7f60f744b740 1 --2- 172.21.15.47:0/1500539618 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b37768b8b0 0x55b377694740 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:26:15.162 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:15.161+0000 7f60f744b740 1 -- 172.21.15.47:0/1500539618 >> 172.21.15.47:0/1500539618 conn(0x55b377685f80 msgr2=0x55b377687cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:26:15.162 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:15.161+0000 7f60cf400640 0 client.25657 ms_handle_reset on v2:172.21.15.47:6840/3401578978 2024-03-21T10:26:15.162 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:15.161+0000 7f60f744b740 1 -- 172.21.15.47:0/1500539618 shutdown_connections 2024-03-21T10:26:15.162 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:15.161+0000 7f60f744b740 1 -- 172.21.15.47:0/1500539618 wait complete. 2024-03-21T10:26:15.162 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:15.161+0000 7f60f744b740 20 client.25657 trim_cache size 0 max 16384 2024-03-21T10:26:15.188 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:15.187+0000 7f60f744b740 1 Processor -- start 2024-03-21T10:26:15.188 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:15.188+0000 7f60f744b740 1 -- start start 2024-03-21T10:26:15.188 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:15.188+0000 7f60f744b740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b37768b8b0 0x55b3776a10d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:26:15.188 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:15.188+0000 7f60f744b740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b37768c260 0x55b377694200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:26:15.189 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:15.188+0000 7f60f744b740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b377694de0 0x55b377694740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:26:15.189 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:15.188+0000 7f60f744b740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60ec1ac4e0 con 0x55b377694de0 2024-03-21T10:26:15.189 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:15.188+0000 7f60f744b740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60ec1ade60 con 0x55b37768b8b0 2024-03-21T10:26:15.189 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:15.188+0000 7f60f744b740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60ec1acb40 con 0x55b37768c260 2024-03-21T10:26:15.190 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:15.188+0000 7f60f2800640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b377694de0 0x55b377694740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:26:15.190 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:15.189+0000 7f60f2800640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b377694de0 0x55b377694740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:58492/0 (socket says 172.21.15.47:58492) 2024-03-21T10:26:15.190 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:15.189+0000 7f60f2800640 1 -- 172.21.15.47:0/2033296737 learned_addr learned my addr 172.21.15.47:0/2033296737 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:26:15.190 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:15.189+0000 7f60f1e00640 1 --2- 172.21.15.47:0/2033296737 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b37768b8b0 0x55b3776a10d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:26:15.190 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:15.189+0000 7f60f1400640 1 --2- 172.21.15.47:0/2033296737 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b37768c260 0x55b377694200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:26:15.190 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:15.189+0000 7f60f1e00640 1 -- 172.21.15.47:0/2033296737 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b37768c260 msgr2=0x55b377694200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:26:15.191 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:15.189+0000 7f60f1e00640 1 --2- 172.21.15.47:0/2033296737 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b37768c260 0x55b377694200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:26:15.191 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:15.189+0000 7f60f1e00640 1 -- 172.21.15.47:0/2033296737 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b377694de0 msgr2=0x55b377694740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:26:15.191 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:15.189+0000 7f60f1e00640 1 --2- 172.21.15.47:0/2033296737 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b377694de0 0x55b377694740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:26:15.191 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:15.189+0000 7f60f1e00640 1 -- 172.21.15.47:0/2033296737 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b377692410 con 0x55b37768b8b0 2024-03-21T10:26:15.191 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:15.189+0000 7f60f2800640 1 --2- 172.21.15.47:0/2033296737 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b377694de0 0x55b377694740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T10:26:15.191 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:15.190+0000 7f60f1e00640 1 --2- 172.21.15.47:0/2033296737 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b37768b8b0 0x55b3776a10d0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f60e0000c00 tx=0x7f60e0005a20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=94c3cb07b1802c74 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:26:15.191 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:15.190+0000 7f60da600640 1 -- 172.21.15.47:0/2033296737 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60e0043040 con 0x55b37768b8b0 2024-03-21T10:26:15.191 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:15.190+0000 7f60da600640 1 -- 172.21.15.47:0/2033296737 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f60e0046ea0 con 0x55b37768b8b0 2024-03-21T10:26:15.191 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:15.190+0000 7f60da600640 1 -- 172.21.15.47:0/2033296737 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60e00042a0 con 0x55b37768b8b0 2024-03-21T10:26:15.192 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:15.191+0000 7f60f744b740 1 -- 172.21.15.47:0/2033296737 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b37768b8b0 msgr2=0x55b3776a10d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:26:15.192 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:15.191+0000 7f60f744b740 1 --2- 172.21.15.47:0/2033296737 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b37768b8b0 0x55b3776a10d0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f60e0000c00 tx=0x7f60e0005a20 comp rx=0 tx=0).stop 2024-03-21T10:26:15.192 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:15.191+0000 7f60f744b740 1 -- 172.21.15.47:0/2033296737 shutdown_connections 2024-03-21T10:26:15.192 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:15.191+0000 7f60f744b740 1 --2- 172.21.15.47:0/2033296737 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b377694de0 0x55b377694740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:26:15.193 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:15.191+0000 7f60f744b740 1 --2- 172.21.15.47:0/2033296737 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b37768c260 0x55b377694200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:26:15.193 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:15.191+0000 7f60f744b740 1 --2- 172.21.15.47:0/2033296737 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b37768b8b0 0x55b3776a10d0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:26:15.193 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:15.191+0000 7f60f744b740 1 -- 172.21.15.47:0/2033296737 >> 172.21.15.47:0/2033296737 conn(0x55b377685f80 msgr2=0x55b377687020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:26:15.193 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:15.192+0000 7f60f744b740 1 -- 172.21.15.47:0/2033296737 shutdown_connections 2024-03-21T10:26:15.193 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:15.192+0000 7f60f744b740 1 -- 172.21.15.47:0/2033296737 wait complete. 2024-03-21T10:26:15.193 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:15.192+0000 7f60f744b740 1 -- start start 2024-03-21T10:26:15.193 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:15.192+0000 7f60f744b740 1 Processor -- start 2024-03-21T10:26:15.195 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:15.192+0000 7f60f744b740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b37768b8b0 0x55b3776a9560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:26:15.195 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:15.193+0000 7f60f744b740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b37768c260 0x55b3776a9aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:26:15.195 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:15.193+0000 7f60f744b740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b377694de0 0x55b3776a7a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:26:15.195 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:15.193+0000 7f60f744b740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60ec1ad800 con 0x55b37768c260 2024-03-21T10:26:15.195 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:15.193+0000 7f60f744b740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60ec1ad1a0 con 0x55b37768b8b0 2024-03-21T10:26:15.195 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:15.193+0000 7f60f744b740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60ec1aa6a0 con 0x55b377694de0 2024-03-21T10:26:15.196 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:15.193+0000 7f60f744b740 10 client.0 _ll_register_callbacks cb 0x7f60dc004aa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T10:26:15.196 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:15.193+0000 7f60f1e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b37768b8b0 0x55b3776a9560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:26:15.196 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:15.193+0000 7f60f1e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b37768b8b0 0x55b3776a9560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:41242/0 (socket says 172.21.15.47:41242) 2024-03-21T10:26:15.196 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:15.193+0000 7f60f1e00640 1 -- 172.21.15.47:0/3047621700 learned_addr learned my addr 172.21.15.47:0/3047621700 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:26:15.196 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:15.193+0000 7f60f1400640 1 --2- 172.21.15.47:0/3047621700 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b37768c260 0x55b3776a9aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:26:15.196 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:15.193+0000 7f60f2800640 1 --2- 172.21.15.47:0/3047621700 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b377694de0 0x55b3776a7a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:26:15.196 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:15.193+0000 7f60f1e00640 1 -- 172.21.15.47:0/3047621700 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b377694de0 msgr2=0x55b3776a7a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:26:15.196 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:15.193+0000 7f60f1e00640 1 --2- 172.21.15.47:0/3047621700 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b377694de0 0x55b3776a7a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:26:15.196 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:15.193+0000 7f60f1e00640 1 -- 172.21.15.47:0/3047621700 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b37768c260 msgr2=0x55b3776a9aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:26:15.196 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:15.193+0000 7f60f1e00640 1 --2- 172.21.15.47:0/3047621700 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b37768c260 0x55b3776a9aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:26:15.196 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:15.193+0000 7f60f1e00640 1 -- 172.21.15.47:0/3047621700 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55b3776a8110 con 0x55b37768b8b0 2024-03-21T10:26:15.196 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:15.194+0000 7f60f1e00640 1 --2- 172.21.15.47:0/3047621700 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b37768b8b0 0x55b3776a9560 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f60e0005c10 tx=0x7f60e0046d80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9fae1b36c043723c server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:26:15.196 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:15.194+0000 7f60cf400640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-21T10:26:15.197 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:15.194+0000 7f60cf400640 1 -- 172.21.15.47:0/3047621700 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60e005d020 con 0x55b37768b8b0 2024-03-21T10:26:15.197 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:15.194+0000 7f60f744b740 1 -- 172.21.15.47:0/3047621700 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f60e0046ea0 con 0x55b37768b8b0 2024-03-21T10:26:15.197 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:15.194+0000 7f60cf400640 1 -- 172.21.15.47:0/3047621700 <== mon.2 v2:172.21.15.100:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 9474+0+0 (secure 0 0 0) 0x7f60e0046360 con 0x55b37768b8b0 2024-03-21T10:26:15.197 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:15.195+0000 7f60cf400640 1 -- 172.21.15.47:0/3047621700 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f60e0046560 con 0x55b37768b8b0 2024-03-21T10:26:15.197 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:15.195+0000 7f60f744b740 10 client.35819 fetch_fsmap learned FSMap version 30 2024-03-21T10:26:15.197 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:15.195+0000 7f60f744b740 1 -- 172.21.15.47:0/3047621700 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x55b3777009f0 con 0x55b37768b8b0 2024-03-21T10:26:15.197 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:15.195+0000 7f60cf400640 1 -- 172.21.15.47:0/3047621700 <== mon.2 v2:172.21.15.100:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f60e0046ea0 con 0x55b37768b8b0 2024-03-21T10:26:15.198 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:15.195+0000 7f60cf400640 1 -- 172.21.15.47:0/3047621700 <== mon.2 v2:172.21.15.100:3300/0 5 ==== fsmap(e 30) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f60e002d250 con 0x55b37768b8b0 2024-03-21T10:26:15.198 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:15.195+0000 7f60f744b740 10 client.35819 fetch_fsmap finished waiting for FSMap version 30 2024-03-21T10:26:15.198 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:15.195+0000 7f60f744b740 10 client.35819 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.a 2024-03-21T10:26:15.199 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:15.195+0000 7f60f744b740 20 client.35819 populate_metadata read hostname 'smithi047' 2024-03-21T10:26:15.199 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:15.196+0000 7f60f744b740 1 --2- 172.21.15.47:0/3047621700 >> [v2:172.21.15.47:6840/3401578978,v1:172.21.15.47:6841/3401578978] conn(0x55b377700ed0 0x55b3777032f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:26:15.199 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:15.196+0000 7f60f744b740 4 client.35819 mds_command: new command op to 14619 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-21T10:26:15.199 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:15.196+0000 7f60f744b740 1 -- 172.21.15.47:0/3047621700 --> [v2:172.21.15.47:6840/3401578978,v1:172.21.15.47:6841/3401578978] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55b3775fc800 con 0x55b377700ed0 2024-03-21T10:26:15.199 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:15.196+0000 7f60f1400640 1 --2- 172.21.15.47:0/3047621700 >> [v2:172.21.15.47:6840/3401578978,v1:172.21.15.47:6841/3401578978] conn(0x55b377700ed0 0x55b3777032f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:26:15.199 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:15.196+0000 7f60f1400640 1 --2- 172.21.15.47:0/3047621700 >> [v2:172.21.15.47:6840/3401578978,v1:172.21.15.47:6841/3401578978] conn(0x55b377700ed0 0x55b3777032f0 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=5a44870728e941dc server_cookie=792c4d58b264ab44 in_seq=0 out_seq=0 2024-03-21T10:26:15.199 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:15.196+0000 7f60cf400640 10 client.35819 ms_handle_connect on v2:172.21.15.47:6840/3401578978 2024-03-21T10:26:15.199 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:15.197+0000 7f60cf400640 1 -- 172.21.15.47:0/3047621700 <== mds.4 v2:172.21.15.47:6840/3401578978 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55b3775fc800 con 0x55b377700ed0 2024-03-21T10:26:15.199 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:15.197+0000 7f60cf400640 10 client.35819 handle_command_reply: tid=0 2024-03-21T10:26:15.199 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:15.197+0000 7f60f744b740 1 client.35819 shutdown 2024-03-21T10:26:15.199 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:15.197+0000 7f60f744b740 1 -- 172.21.15.47:0/3047621700 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b37768b8b0 msgr2=0x55b3776a9560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:26:15.199 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:15.197+0000 7f60f744b740 1 --2- 172.21.15.47:0/3047621700 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b37768b8b0 0x55b3776a9560 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f60e0005c10 tx=0x7f60e0046d80 comp rx=0 tx=0).stop 2024-03-21T10:26:15.199 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:15.197+0000 7f60f744b740 1 -- 172.21.15.47:0/3047621700 shutdown_connections 2024-03-21T10:26:15.199 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:15.197+0000 7f60f744b740 1 --2- 172.21.15.47:0/3047621700 >> [v2:172.21.15.47:6840/3401578978,v1:172.21.15.47:6841/3401578978] conn(0x55b377700ed0 0x55b3777032f0 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:26:15.199 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:15.197+0000 7f60f744b740 1 --2- 172.21.15.47:0/3047621700 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b377694de0 0x55b3776a7a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:26:15.200 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:15.197+0000 7f60cf400640 0 client.35819 ms_handle_reset on v2:172.21.15.47:6840/3401578978 2024-03-21T10:26:15.200 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:15.197+0000 7f60f744b740 1 --2- 172.21.15.47:0/3047621700 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b37768c260 0x55b3776a9aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:26:15.200 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:15.197+0000 7f60f744b740 1 --2- 172.21.15.47:0/3047621700 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b37768b8b0 0x55b3776a9560 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:26:15.200 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:15.197+0000 7f60f744b740 1 -- 172.21.15.47:0/3047621700 >> 172.21.15.47:0/3047621700 conn(0x55b377685f80 msgr2=0x55b3776878b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:26:15.200 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:15.198+0000 7f60f744b740 1 -- 172.21.15.47:0/3047621700 shutdown_connections 2024-03-21T10:26:15.200 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:15.198+0000 7f60f744b740 1 -- 172.21.15.47:0/3047621700 wait complete. 2024-03-21T10:26:15.200 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:15.198+0000 7f60f744b740 20 client.35819 trim_cache size 0 max 16384 2024-03-21T10:26:15.200 INFO:teuthology.orchestra.run.smithi047.stdout:[] 2024-03-21T10:26:15.201 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:15.198+0000 7f60f3200640 1 -- 172.21.15.47:0/1944136240 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f60d40a33b0 msgr2=0x7f60d40a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:26:15.201 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:15.198+0000 7f60f3200640 1 --2- 172.21.15.47:0/1944136240 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f60d40a33b0 0x7f60d40a5870 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f60ec06abd0 tx=0x7f60e8004ab0 comp rx=0 tx=0).stop 2024-03-21T10:26:15.201 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:15.198+0000 7f60f3200640 1 -- 172.21.15.47:0/1944136240 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f60ec074390 msgr2=0x7f60ec19a510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:26:15.201 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:15.198+0000 7f60f3200640 1 --2- 172.21.15.47:0/1944136240 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f60ec074390 0x7f60ec19a510 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f60dc00e9f0 tx=0x7f60dc00edf0 comp rx=0 tx=0).stop 2024-03-21T10:26:15.201 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:15.198+0000 7f60f3200640 1 -- 172.21.15.47:0/1944136240 shutdown_connections 2024-03-21T10:26:15.201 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:15.198+0000 7f60f3200640 1 --2- 172.21.15.47:0/1944136240 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f60ec10ddb0 0x7f60ec19e840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:26:15.201 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:15.198+0000 7f60f3200640 1 --2- 172.21.15.47:0/1944136240 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f60d40a33b0 0x7f60d40a5870 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:26:15.202 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:15.198+0000 7f60f3200640 1 --2- 172.21.15.47:0/1944136240 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f60ec074390 0x7f60ec19a510 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:26:15.202 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:15.198+0000 7f60f3200640 1 --2- 172.21.15.47:0/1944136240 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f60ec073a70 0x7f60ec199fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:26:15.202 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:15.198+0000 7f60f3200640 1 -- 172.21.15.47:0/1944136240 >> 172.21.15.47:0/1944136240 conn(0x7f60ec0fd040 msgr2=0x7f60ec0fe870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:26:15.202 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:15.199+0000 7f60f3200640 1 -- 172.21.15.47:0/1944136240 shutdown_connections 2024-03-21T10:26:15.202 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:15.199+0000 7f60f3200640 1 -- 172.21.15.47:0/1944136240 wait complete. 2024-03-21T10:26:15.213 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:26:15 smithi047 ceph-mon[31547]: pgmap v30: 129 pgs: 129 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 597 B/s wr, 8 op/s 2024-03-21T10:26:15.213 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:26:15 smithi047 ceph-mon[31547]: from='client.? 172.21.15.47:0/1120145976' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-21T10:26:15.219 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-21T10:26:15.220 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-03-21T10:26:15.437 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:15.436+0000 7f860be00640 1 -- 172.21.15.47:0/3859425265 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f860c1004c0 msgr2=0x7f860c1008a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:26:15.437 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:15.436+0000 7f860be00640 1 --2- 172.21.15.47:0/3859425265 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f860c1004c0 0x7f860c1008a0 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7f85fc009900 tx=0x7f85fc02bd10 comp rx=0 tx=0).stop 2024-03-21T10:26:15.437 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:15.436+0000 7f860be00640 1 -- 172.21.15.47:0/3859425265 shutdown_connections 2024-03-21T10:26:15.437 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:15.436+0000 7f860be00640 1 --2- 172.21.15.47:0/3859425265 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f860c1058c0 0x7f860c105d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:26:15.437 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:15.436+0000 7f860be00640 1 --2- 172.21.15.47:0/3859425265 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f860c100de0 0x7f860c1052a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:26:15.437 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:15.436+0000 7f860be00640 1 --2- 172.21.15.47:0/3859425265 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f860c1004c0 0x7f860c1008a0 unknown :-1 s=CLOSED pgs=594 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:26:15.437 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:15.436+0000 7f860be00640 1 -- 172.21.15.47:0/3859425265 >> 172.21.15.47:0/3859425265 conn(0x7f860c0fbef0 msgr2=0x7f860c0fe310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:26:15.438 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:15.436+0000 7f860be00640 1 -- 172.21.15.47:0/3859425265 shutdown_connections 2024-03-21T10:26:15.438 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:15.436+0000 7f860be00640 1 -- 172.21.15.47:0/3859425265 wait complete. 2024-03-21T10:26:15.438 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:15.438+0000 7f860be00640 1 Processor -- start 2024-03-21T10:26:15.438 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:15.438+0000 7f860be00640 1 -- start start 2024-03-21T10:26:15.439 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:15.438+0000 7f860be00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f860c1004c0 0x7f860c19e360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:26:15.439 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:15.438+0000 7f860be00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f860c100de0 0x7f860c19e8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:26:15.439 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:15.438+0000 7f860be00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f860c1058c0 0x7f860c1a2c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:26:15.439 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:15.439+0000 7f860be00640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f860c117db0 con 0x7f860c1004c0 2024-03-21T10:26:15.440 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:15.439+0000 7f860be00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f860c117c30 con 0x7f860c100de0 2024-03-21T10:26:15.440 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:15.439+0000 7f860be00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f860c117f30 con 0x7f860c1058c0 2024-03-21T10:26:15.440 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:15.439+0000 7f860aa00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f860c1004c0 0x7f860c19e360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:26:15.440 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:15.439+0000 7f860aa00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f860c1004c0 0x7f860c19e360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:58514/0 (socket says 172.21.15.47:58514) 2024-03-21T10:26:15.440 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:15.439+0000 7f860aa00640 1 -- 172.21.15.47:0/3251068331 learned_addr learned my addr 172.21.15.47:0/3251068331 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:26:15.440 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:15.439+0000 7f860a000640 1 --2- 172.21.15.47:0/3251068331 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f860c100de0 0x7f860c19e8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:26:15.440 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:15.439+0000 7f860b400640 1 --2- 172.21.15.47:0/3251068331 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f860c1058c0 0x7f860c1a2c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:26:15.441 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:15.439+0000 7f860aa00640 1 -- 172.21.15.47:0/3251068331 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f860c1058c0 msgr2=0x7f860c1a2c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:26:15.441 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:15.439+0000 7f860aa00640 1 --2- 172.21.15.47:0/3251068331 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f860c1058c0 0x7f860c1a2c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:26:15.441 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:15.440+0000 7f860aa00640 1 -- 172.21.15.47:0/3251068331 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f860c100de0 msgr2=0x7f860c19e8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:26:15.441 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:15.440+0000 7f860aa00640 1 --2- 172.21.15.47:0/3251068331 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f860c100de0 0x7f860c19e8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:26:15.441 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:15.440+0000 7f860aa00640 1 -- 172.21.15.47:0/3251068331 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f860c1a3350 con 0x7f860c1004c0 2024-03-21T10:26:15.441 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:15.440+0000 7f860b400640 1 --2- 172.21.15.47:0/3251068331 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f860c1058c0 0x7f860c1a2c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T10:26:15.441 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:15.440+0000 7f860aa00640 1 --2- 172.21.15.47:0/3251068331 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f860c1004c0 0x7f860c19e360 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7f85fc005940 tx=0x7f85fc02d870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:26:15.441 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:15.441+0000 7f8603400640 1 -- 172.21.15.47:0/3251068331 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85fc043020 con 0x7f860c1004c0 2024-03-21T10:26:15.442 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:15.441+0000 7f8603400640 1 -- 172.21.15.47:0/3251068331 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f85fc02dc70 con 0x7f860c1004c0 2024-03-21T10:26:15.442 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:15.441+0000 7f8603400640 1 -- 172.21.15.47:0/3251068331 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85fc0039f0 con 0x7f860c1004c0 2024-03-21T10:26:15.442 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:15.441+0000 7f860be00640 1 -- 172.21.15.47:0/3251068331 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f860c1a35e0 con 0x7f860c1004c0 2024-03-21T10:26:15.442 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:15.441+0000 7f860be00640 1 -- 172.21.15.47:0/3251068331 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f860c1aaed0 con 0x7f860c1004c0 2024-03-21T10:26:15.446 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:15.445+0000 7f8614426740 1 Processor -- start 2024-03-21T10:26:15.446 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:15.446+0000 7f8614426740 1 -- start start 2024-03-21T10:26:15.446 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:15.446+0000 7f8614426740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e8ab2c48b0 0x55e8ab2c4c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:26:15.447 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:15.446+0000 7f8614426740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e8ab2c5260 0x55e8ab2cd810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:26:15.447 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:15.446+0000 7f8614426740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e8ab2cdde0 0x55e8ab2d0230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:26:15.447 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:15.446+0000 7f8614426740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e8ab2282c0 con 0x55e8ab2cdde0 2024-03-21T10:26:15.447 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:15.446+0000 7f8614426740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e8ab1d0c90 con 0x55e8ab2c48b0 2024-03-21T10:26:15.447 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:15.446+0000 7f8614426740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e8ab224360 con 0x55e8ab2c5260 2024-03-21T10:26:15.448 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:15.446+0000 7f8603400640 1 -- 172.21.15.47:0/3251068331 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 26) v1 ==== 138087+0+0 (secure 0 0 0) 0x7f85fc046020 con 0x7f860c1004c0 2024-03-21T10:26:15.448 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:15.447+0000 7f860b400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e8ab2c48b0 0x55e8ab2c4c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:26:15.448 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:15.447+0000 7f860a000640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e8ab2cdde0 0x55e8ab2d0230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:26:15.448 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:15.447+0000 7f860b400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e8ab2c48b0 0x55e8ab2c4c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:41264/0 (socket says 172.21.15.47:41264) 2024-03-21T10:26:15.448 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:15.447+0000 7f860b400640 1 -- 172.21.15.47:0/104268770 learned_addr learned my addr 172.21.15.47:0/104268770 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:26:15.448 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:15.447+0000 7f860b400640 1 -- 172.21.15.47:0/104268770 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e8ab2c5260 msgr2=0x55e8ab2cd810 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-21T10:26:15.448 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:15.447+0000 7f860b400640 1 --2- 172.21.15.47:0/104268770 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e8ab2c5260 0x55e8ab2cd810 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:26:15.448 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:15.447+0000 7f860b400640 1 -- 172.21.15.47:0/104268770 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e8ab2cdde0 msgr2=0x55e8ab2d0230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:26:15.448 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:15.447+0000 7f860b400640 1 --2- 172.21.15.47:0/104268770 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e8ab2cdde0 0x55e8ab2d0230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:26:15.448 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:15.447+0000 7f860b400640 1 -- 172.21.15.47:0/104268770 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e8ab1f5080 con 0x55e8ab2c48b0 2024-03-21T10:26:15.449 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:15.448+0000 7f860a000640 1 --2- 172.21.15.47:0/104268770 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e8ab2cdde0 0x55e8ab2d0230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T10:26:15.450 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:15.448+0000 7f860b400640 1 --2- 172.21.15.47:0/104268770 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e8ab2c48b0 0x55e8ab2c4c90 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f8604009ea0 tx=0x7f8604009f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bae4036db3ad14b1 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:26:15.450 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:15.448+0000 7f85f7e00640 1 -- 172.21.15.47:0/104268770 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8604016070 con 0x55e8ab2c48b0 2024-03-21T10:26:15.450 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:15.448+0000 7f85f7e00640 1 -- 172.21.15.47:0/104268770 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f860400ee80 con 0x55e8ab2c48b0 2024-03-21T10:26:15.450 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:15.448+0000 7f85f7e00640 1 -- 172.21.15.47:0/104268770 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86040116d0 con 0x55e8ab2c48b0 2024-03-21T10:26:15.450 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:15.449+0000 7f8603400640 1 --2- 172.21.15.47:0/3251068331 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f85f00a3230 0x7f85f00a56f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:26:15.450 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:15.449+0000 7f8603400640 1 -- 172.21.15.47:0/3251068331 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 9474+0+0 (secure 0 0 0) 0x7f85fc0f1a30 con 0x7f860c1004c0 2024-03-21T10:26:15.450 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:15.449+0000 7f8614426740 1 -- 172.21.15.47:0/104268770 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e8ab2c48b0 msgr2=0x55e8ab2c4c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:26:15.450 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:15.449+0000 7f8614426740 1 --2- 172.21.15.47:0/104268770 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e8ab2c48b0 0x55e8ab2c4c90 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f8604009ea0 tx=0x7f8604009f90 comp rx=0 tx=0).stop 2024-03-21T10:26:15.451 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:15.450+0000 7f860aa00640 1 --2- 172.21.15.47:0/3251068331 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f85f00a3230 0x7f85f00a56f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:26:15.451 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:15.450+0000 7f8614426740 1 -- 172.21.15.47:0/104268770 shutdown_connections 2024-03-21T10:26:15.451 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:15.450+0000 7f8614426740 1 --2- 172.21.15.47:0/104268770 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e8ab2cdde0 0x55e8ab2d0230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:26:15.451 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:15.450+0000 7f8614426740 1 --2- 172.21.15.47:0/104268770 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e8ab2c5260 0x55e8ab2cd810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:26:15.451 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:15.450+0000 7f8614426740 1 --2- 172.21.15.47:0/104268770 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e8ab2c48b0 0x55e8ab2c4c90 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:26:15.451 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:15.450+0000 7f8614426740 1 -- 172.21.15.47:0/104268770 >> 172.21.15.47:0/104268770 conn(0x55e8ab2bef80 msgr2=0x55e8ab2bf360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:26:15.451 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:15.450+0000 7f8614426740 1 -- 172.21.15.47:0/104268770 shutdown_connections 2024-03-21T10:26:15.452 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:15.450+0000 7f8614426740 1 -- 172.21.15.47:0/104268770 wait complete. 2024-03-21T10:26:15.453 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:15.450+0000 7f860aa00640 1 --2- 172.21.15.47:0/3251068331 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f85f00a3230 0x7f85f00a56f0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f85fc0eedb0 tx=0x7f85fc0b9bc0 comp rx=0 tx=0).ready entity=mgr.15687 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:26:15.453 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:15.451+0000 7f8614426740 1 -- start start 2024-03-21T10:26:15.453 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:15.451+0000 7f8614426740 1 Processor -- start 2024-03-21T10:26:15.453 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:15.452+0000 7f8614426740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e8ab2c48b0 0x55e8ab2cd6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:26:15.453 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:15.452+0000 7f8614426740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e8ab2c5260 0x55e8ab2caa40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:26:15.453 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:15.452+0000 7f8614426740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e8ab2cdde0 0x55e8ab2cafa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:26:15.453 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:15.452+0000 7f8614426740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e8ab2d0d50 con 0x55e8ab2cdde0 2024-03-21T10:26:15.453 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:15.452+0000 7f8614426740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e8ab1e40d0 con 0x55e8ab2c5260 2024-03-21T10:26:15.453 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:15.452+0000 7f8614426740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e8ab2d0ed0 con 0x55e8ab2c48b0 2024-03-21T10:26:15.453 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:15.452+0000 7f8614426740 10 client.0 _ll_register_callbacks cb 0x55e8ab1e8e80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T10:26:15.453 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:15.452+0000 7f860b400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e8ab2cdde0 0x55e8ab2cafa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:26:15.453 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:15.452+0000 7f860a000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e8ab2c5260 0x55e8ab2caa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:26:15.453 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:15.452+0000 7f860b400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e8ab2cdde0 0x55e8ab2cafa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:58536/0 (socket says 172.21.15.47:58536) 2024-03-21T10:26:15.454 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:15.452+0000 7f860b400640 1 -- 172.21.15.47:0/1611218157 learned_addr learned my addr 172.21.15.47:0/1611218157 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:26:15.454 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:15.452+0000 7f860b400640 1 --2- 172.21.15.47:0/1611218157 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e8ab2c48b0 0x55e8ab2cd6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:26:15.455 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:15.453+0000 7f860a000640 1 -- 172.21.15.47:0/1611218157 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e8ab2c48b0 msgr2=0x55e8ab2cd6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:26:15.455 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:15.453+0000 7f860a000640 1 --2- 172.21.15.47:0/1611218157 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e8ab2c48b0 0x55e8ab2cd6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:26:15.455 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:15.453+0000 7f860a000640 1 -- 172.21.15.47:0/1611218157 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e8ab2cdde0 msgr2=0x55e8ab2cafa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:26:15.455 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:15.453+0000 7f860a000640 1 --2- 172.21.15.47:0/1611218157 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e8ab2cdde0 0x55e8ab2cafa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:26:15.455 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:15.453+0000 7f860a000640 1 -- 172.21.15.47:0/1611218157 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55e8aaf3c3f0 con 0x55e8ab2c5260 2024-03-21T10:26:15.456 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:15.453+0000 7f860b400640 1 --2- 172.21.15.47:0/1611218157 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e8ab2cdde0 0x55e8ab2cafa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T10:26:15.456 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:15.453+0000 7f860a000640 1 --2- 172.21.15.47:0/1611218157 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e8ab2c5260 0x55e8ab2caa40 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f85f8005fd0 tx=0x7f85f800bde0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1e39c668cd3b7602 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:26:15.456 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:15.453+0000 7f85f4c00640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-21T10:26:15.456 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:15.453+0000 7f85f4c00640 1 -- 172.21.15.47:0/1611218157 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85f8005020 con 0x55e8ab2c5260 2024-03-21T10:26:15.456 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:15.454+0000 7f8614426740 1 -- 172.21.15.47:0/1611218157 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55e8ab235800 con 0x55e8ab2c5260 2024-03-21T10:26:15.456 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:15.454+0000 7f85f4c00640 1 -- 172.21.15.47:0/1611218157 <== mon.2 v2:172.21.15.100:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 9474+0+0 (secure 0 0 0) 0x7f85f80051c0 con 0x55e8ab2c5260 2024-03-21T10:26:15.456 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:15.454+0000 7f85f4c00640 1 -- 172.21.15.47:0/1611218157 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f85f8024050 con 0x55e8ab2c5260 2024-03-21T10:26:15.456 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:15.454+0000 7f8614426740 10 client.35831 fetch_fsmap learned FSMap version 30 2024-03-21T10:26:15.456 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:15.454+0000 7f8614426740 1 -- 172.21.15.47:0/1611218157 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x55e8ab0067d0 con 0x55e8ab2c5260 2024-03-21T10:26:15.456 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:15.454+0000 7f85f4c00640 1 -- 172.21.15.47:0/1611218157 <== mon.2 v2:172.21.15.100:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x55e8ab235800 con 0x55e8ab2c5260 2024-03-21T10:26:15.456 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:15.455+0000 7f85f4c00640 1 -- 172.21.15.47:0/1611218157 <== mon.2 v2:172.21.15.100:3300/0 5 ==== fsmap(e 30) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f85f801b470 con 0x55e8ab2c5260 2024-03-21T10:26:15.456 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:15.455+0000 7f8614426740 10 client.35831 fetch_fsmap finished waiting for FSMap version 30 2024-03-21T10:26:15.456 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:15.455+0000 7f8614426740 10 client.35831 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.k 2024-03-21T10:26:15.456 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:15.455+0000 7f8614426740 20 client.35831 populate_metadata read hostname 'smithi047' 2024-03-21T10:26:15.457 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:15.455+0000 7f8614426740 1 --2- 172.21.15.47:0/1611218157 >> [v2:172.21.15.100:6832/2723302038,v1:172.21.15.100:6833/2723302038] conn(0x55e8ab2ddc80 0x55e8ab2e00a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:26:15.457 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:15.455+0000 7f8614426740 4 client.35831 mds_command: new command op to 34427 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T10:26:15.457 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:15.455+0000 7f8614426740 1 -- 172.21.15.47:0/1611218157 --> [v2:172.21.15.100:6832/2723302038,v1:172.21.15.100:6833/2723302038] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55e8ab22fb50 con 0x55e8ab2ddc80 2024-03-21T10:26:15.457 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:15.456+0000 7f860b400640 1 --2- 172.21.15.47:0/1611218157 >> [v2:172.21.15.100:6832/2723302038,v1:172.21.15.100:6833/2723302038] conn(0x55e8ab2ddc80 0x55e8ab2e00a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:26:15.457 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:15.457+0000 7f860b400640 1 --2- 172.21.15.47:0/1611218157 >> [v2:172.21.15.100:6832/2723302038,v1:172.21.15.100:6833/2723302038] conn(0x55e8ab2ddc80 0x55e8ab2e00a0 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=fee602f7e2059b49 server_cookie=42d07b24ff48d2d4 in_seq=0 out_seq=0 2024-03-21T10:26:15.458 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:15.457+0000 7f85f4c00640 10 client.35831 ms_handle_connect on v2:172.21.15.100:6832/2723302038 2024-03-21T10:26:15.458 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:15.458+0000 7f85f4c00640 1 -- 172.21.15.47:0/1611218157 <== mds.3 v2:172.21.15.100:6832/2723302038 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55e8ab22fb50 con 0x55e8ab2ddc80 2024-03-21T10:26:15.458 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:15.458+0000 7f85f4c00640 10 client.35831 handle_command_reply: tid=0 2024-03-21T10:26:15.458 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:15.458+0000 7f8614426740 1 client.35831 shutdown 2024-03-21T10:26:15.459 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:15.459+0000 7f8614426740 1 -- 172.21.15.47:0/1611218157 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e8ab2c5260 msgr2=0x55e8ab2caa40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:26:15.460 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:15.459+0000 7f8614426740 1 --2- 172.21.15.47:0/1611218157 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e8ab2c5260 0x55e8ab2caa40 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f85f8005fd0 tx=0x7f85f800bde0 comp rx=0 tx=0).stop 2024-03-21T10:26:15.460 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:15.459+0000 7f8614426740 1 -- 172.21.15.47:0/1611218157 shutdown_connections 2024-03-21T10:26:15.460 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:15.459+0000 7f8614426740 1 --2- 172.21.15.47:0/1611218157 >> [v2:172.21.15.100:6832/2723302038,v1:172.21.15.100:6833/2723302038] conn(0x55e8ab2ddc80 0x55e8ab2e00a0 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:26:15.460 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:15.459+0000 7f8614426740 1 --2- 172.21.15.47:0/1611218157 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e8ab2cdde0 0x55e8ab2cafa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:26:15.460 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:15.459+0000 7f8614426740 1 --2- 172.21.15.47:0/1611218157 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e8ab2c5260 0x55e8ab2caa40 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:26:15.460 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:15.459+0000 7f8614426740 1 --2- 172.21.15.47:0/1611218157 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e8ab2c48b0 0x55e8ab2cd6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:26:15.460 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:15.459+0000 7f8614426740 1 -- 172.21.15.47:0/1611218157 >> 172.21.15.47:0/1611218157 conn(0x55e8ab2bef80 msgr2=0x55e8ab2c0cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:26:15.460 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:15.459+0000 7f85f4c00640 0 client.35831 ms_handle_reset on v2:172.21.15.100:6832/2723302038 2024-03-21T10:26:15.460 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:15.459+0000 7f8614426740 1 -- 172.21.15.47:0/1611218157 shutdown_connections 2024-03-21T10:26:15.460 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:15.459+0000 7f8614426740 1 -- 172.21.15.47:0/1611218157 wait complete. 2024-03-21T10:26:15.460 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:15.459+0000 7f8614426740 20 client.35831 trim_cache size 0 max 16384 2024-03-21T10:26:15.477 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:15.477+0000 7f8614426740 1 Processor -- start 2024-03-21T10:26:15.477 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:15.477+0000 7f8614426740 1 -- start start 2024-03-21T10:26:15.478 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:15.477+0000 7f8614426740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e8ab2c48b0 0x55e8ab2bdfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:26:15.478 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:15.477+0000 7f8614426740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e8ab2c5260 0x55e8ab2cd170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:26:15.478 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:15.477+0000 7f8614426740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e8ab2cdde0 0x55e8ab2cd6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:26:15.479 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:15.477+0000 7f8614426740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f860c1b4cd0 con 0x55e8ab2c5260 2024-03-21T10:26:15.479 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:15.477+0000 7f8614426740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f860c1b6650 con 0x55e8ab2c48b0 2024-03-21T10:26:15.479 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:15.477+0000 7f8614426740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f860c1b5330 con 0x55e8ab2cdde0 2024-03-21T10:26:15.479 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:15.478+0000 7f860a000640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e8ab2c5260 0x55e8ab2cd170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:26:15.479 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:15.478+0000 7f860a000640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e8ab2c5260 0x55e8ab2cd170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:58538/0 (socket says 172.21.15.47:58538) 2024-03-21T10:26:15.479 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:15.478+0000 7f860a000640 1 -- 172.21.15.47:0/2293508504 learned_addr learned my addr 172.21.15.47:0/2293508504 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:26:15.479 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:15.478+0000 7f860b400640 1 --2- 172.21.15.47:0/2293508504 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e8ab2c48b0 0x55e8ab2bdfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:26:15.479 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:15.478+0000 7f860b400640 1 --2- 172.21.15.47:0/2293508504 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e8ab2cdde0 0x55e8ab2cd6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:26:15.479 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:15.478+0000 7f860a000640 1 -- 172.21.15.47:0/2293508504 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e8ab2cdde0 msgr2=0x55e8ab2cd6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:26:15.479 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:15.478+0000 7f860a000640 1 --2- 172.21.15.47:0/2293508504 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e8ab2cdde0 0x55e8ab2cd6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:26:15.479 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:15.478+0000 7f860a000640 1 -- 172.21.15.47:0/2293508504 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e8ab2c48b0 msgr2=0x55e8ab2bdfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:26:15.479 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:15.478+0000 7f860a000640 1 --2- 172.21.15.47:0/2293508504 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e8ab2c48b0 0x55e8ab2bdfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:26:15.479 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:15.478+0000 7f860a000640 1 -- 172.21.15.47:0/2293508504 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e8ab2c5000 con 0x55e8ab2c5260 2024-03-21T10:26:15.479 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:15.478+0000 7f860b400640 1 --2- 172.21.15.47:0/2293508504 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e8ab2cdde0 0x55e8ab2cd6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T10:26:15.480 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:26:15 smithi100 ceph-mon[35193]: pgmap v30: 129 pgs: 129 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 597 B/s wr, 8 op/s 2024-03-21T10:26:15.480 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:26:15 smithi100 ceph-mon[35193]: from='client.? 172.21.15.47:0/1120145976' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-21T10:26:15.481 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:15.479+0000 7f860a000640 1 --2- 172.21.15.47:0/2293508504 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e8ab2c5260 0x55e8ab2cd170 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7f85f8000c00 tx=0x7f85f800bf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c730f92ea3ae2fc0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:26:15.481 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:15.479+0000 7f85f7400640 1 -- 172.21.15.47:0/2293508504 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85f802c020 con 0x55e8ab2c5260 2024-03-21T10:26:15.481 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:15.479+0000 7f85f7400640 1 -- 172.21.15.47:0/2293508504 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f85f800b350 con 0x55e8ab2c5260 2024-03-21T10:26:15.481 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:15.479+0000 7f85f7400640 1 -- 172.21.15.47:0/2293508504 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85f801be60 con 0x55e8ab2c5260 2024-03-21T10:26:15.481 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:15.480+0000 7f8614426740 1 -- 172.21.15.47:0/2293508504 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e8ab2c5260 msgr2=0x55e8ab2cd170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:26:15.481 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:15.480+0000 7f8614426740 1 --2- 172.21.15.47:0/2293508504 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e8ab2c5260 0x55e8ab2cd170 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7f85f8000c00 tx=0x7f85f800bf90 comp rx=0 tx=0).stop 2024-03-21T10:26:15.482 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:15.480+0000 7f8614426740 1 -- 172.21.15.47:0/2293508504 shutdown_connections 2024-03-21T10:26:15.482 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:15.480+0000 7f8614426740 1 --2- 172.21.15.47:0/2293508504 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e8ab2cdde0 0x55e8ab2cd6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:26:15.482 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:15.480+0000 7f8614426740 1 --2- 172.21.15.47:0/2293508504 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e8ab2c5260 0x55e8ab2cd170 unknown :-1 s=CLOSED pgs=596 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:26:15.482 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:15.480+0000 7f8614426740 1 --2- 172.21.15.47:0/2293508504 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e8ab2c48b0 0x55e8ab2bdfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:26:15.482 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:15.480+0000 7f8614426740 1 -- 172.21.15.47:0/2293508504 >> 172.21.15.47:0/2293508504 conn(0x55e8ab2bef80 msgr2=0x55e8ab2bd8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:26:15.482 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:15.480+0000 7f8614426740 1 -- 172.21.15.47:0/2293508504 shutdown_connections 2024-03-21T10:26:15.482 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:15.480+0000 7f8614426740 1 -- 172.21.15.47:0/2293508504 wait complete. 2024-03-21T10:26:15.482 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:15.481+0000 7f8614426740 1 -- start start 2024-03-21T10:26:15.482 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:15.481+0000 7f8614426740 1 Processor -- start 2024-03-21T10:26:15.483 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:15.482+0000 7f8614426740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e8ab2c48b0 0x55e8ab2d5a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:26:15.483 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:15.482+0000 7f8614426740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e8ab2c5260 0x55e8ab2d5f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:26:15.483 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:15.482+0000 7f8614426740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e8ab2cdde0 0x55e8ab2d1120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:26:15.483 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:15.482+0000 7f8614426740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f860c1b5ff0 con 0x55e8ab2c48b0 2024-03-21T10:26:15.484 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:15.482+0000 7f8614426740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f860c1b5990 con 0x55e8ab2c5260 2024-03-21T10:26:15.484 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:15.482+0000 7f8614426740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f860c1b2e90 con 0x55e8ab2cdde0 2024-03-21T10:26:15.484 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:15.482+0000 7f8614426740 10 client.0 _ll_register_callbacks cb 0x7f85f80042b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T10:26:15.484 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:15.482+0000 7f860b400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e8ab2c48b0 0x55e8ab2d5a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:26:15.484 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:15.482+0000 7f860b400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e8ab2c48b0 0x55e8ab2d5a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:58542/0 (socket says 172.21.15.47:58542) 2024-03-21T10:26:15.484 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:15.482+0000 7f860a000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e8ab2c5260 0x55e8ab2d5f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:26:15.484 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:15.482+0000 7f860b400640 1 -- 172.21.15.47:0/3410927139 learned_addr learned my addr 172.21.15.47:0/3410927139 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:26:15.484 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:15.483+0000 7f860b400640 1 --2- 172.21.15.47:0/3410927139 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e8ab2cdde0 0x55e8ab2d1120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:26:15.484 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:15.483+0000 7f860b400640 1 -- 172.21.15.47:0/3410927139 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e8ab2cdde0 msgr2=0x55e8ab2d1120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:26:15.484 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:15.483+0000 7f860b400640 1 --2- 172.21.15.47:0/3410927139 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e8ab2cdde0 0x55e8ab2d1120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:26:15.484 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:15.483+0000 7f860b400640 1 -- 172.21.15.47:0/3410927139 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e8ab2c5260 msgr2=0x55e8ab2d5f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:26:15.484 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:15.483+0000 7f860b400640 1 --2- 172.21.15.47:0/3410927139 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e8ab2c5260 0x55e8ab2d5f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:26:15.485 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:15.483+0000 7f860b400640 1 -- 172.21.15.47:0/3410927139 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55e8ab2d1950 con 0x55e8ab2c48b0 2024-03-21T10:26:15.485 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:15.483+0000 7f860a000640 1 --2- 172.21.15.47:0/3410927139 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e8ab2c5260 0x55e8ab2d5f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T10:26:15.486 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:15.484+0000 7f860b400640 1 --2- 172.21.15.47:0/3410927139 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e8ab2c48b0 0x55e8ab2d5a00 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7f86040059b0 tx=0x7f860401c040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=45ff2c27cf5c4850 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:26:15.486 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:15.484+0000 7f85f4c00640 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-03-21T10:26:15.486 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:15.484+0000 7f85f4c00640 1 -- 172.21.15.47:0/3410927139 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8604016070 con 0x55e8ab2c48b0 2024-03-21T10:26:15.486 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:15.484+0000 7f85f4c00640 1 -- 172.21.15.47:0/3410927139 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 9474+0+0 (secure 0 0 0) 0x7f8604011b20 con 0x55e8ab2c48b0 2024-03-21T10:26:15.486 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:15.484+0000 7f8614426740 1 -- 172.21.15.47:0/3410927139 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f85f800b350 con 0x55e8ab2c48b0 2024-03-21T10:26:15.486 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:15.484+0000 7f85f4c00640 1 -- 172.21.15.47:0/3410927139 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8604011d20 con 0x55e8ab2c48b0 2024-03-21T10:26:15.486 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:15.484+0000 7f8614426740 10 client.16032 fetch_fsmap learned FSMap version 30 2024-03-21T10:26:15.486 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:15.484+0000 7f8614426740 1 -- 172.21.15.47:0/3410927139 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x55e8ab338f90 con 0x55e8ab2c48b0 2024-03-21T10:26:15.486 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:15.485+0000 7f85f4c00640 1 -- 172.21.15.47:0/3410927139 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f8604003e20 con 0x55e8ab2c48b0 2024-03-21T10:26:15.487 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:15.485+0000 7f85f4c00640 1 -- 172.21.15.47:0/3410927139 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 30) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f8604029040 con 0x55e8ab2c48b0 2024-03-21T10:26:15.487 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:15.485+0000 7f8614426740 10 client.16032 fetch_fsmap finished waiting for FSMap version 30 2024-03-21T10:26:15.487 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:15.485+0000 7f8614426740 10 client.16032 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.k 2024-03-21T10:26:15.487 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:15.485+0000 7f8614426740 20 client.16032 populate_metadata read hostname 'smithi047' 2024-03-21T10:26:15.487 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:15.485+0000 7f8614426740 1 --2- 172.21.15.47:0/3410927139 >> [v2:172.21.15.100:6832/2723302038,v1:172.21.15.100:6833/2723302038] conn(0x55e8ab339390 0x55e8ab33b780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:26:15.487 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:15.485+0000 7f8614426740 4 client.16032 mds_command: new command op to 34427 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-21T10:26:15.487 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:15.485+0000 7f8614426740 1 -- 172.21.15.47:0/3410927139 --> [v2:172.21.15.100:6832/2723302038,v1:172.21.15.100:6833/2723302038] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x7f85f800b350 con 0x55e8ab339390 2024-03-21T10:26:15.487 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:15.485+0000 7f860a000640 1 --2- 172.21.15.47:0/3410927139 >> [v2:172.21.15.100:6832/2723302038,v1:172.21.15.100:6833/2723302038] conn(0x55e8ab339390 0x55e8ab33b780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:26:15.488 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:15.486+0000 7f860a000640 1 --2- 172.21.15.47:0/3410927139 >> [v2:172.21.15.100:6832/2723302038,v1:172.21.15.100:6833/2723302038] conn(0x55e8ab339390 0x55e8ab33b780 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=5aa4e53094f37518 server_cookie=50d3d4b34314afe5 in_seq=0 out_seq=0 2024-03-21T10:26:15.489 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:15.486+0000 7f85f4c00640 10 client.16032 ms_handle_connect on v2:172.21.15.100:6832/2723302038 2024-03-21T10:26:15.489 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:15.487+0000 7f85f4c00640 1 -- 172.21.15.47:0/3410927139 <== mds.3 v2:172.21.15.100:6832/2723302038 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x7f85f800b350 con 0x55e8ab339390 2024-03-21T10:26:15.489 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:15.487+0000 7f85f4c00640 10 client.16032 handle_command_reply: tid=0 2024-03-21T10:26:15.489 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:15.487+0000 7f8614426740 1 client.16032 shutdown 2024-03-21T10:26:15.489 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:15.487+0000 7f8614426740 1 -- 172.21.15.47:0/3410927139 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e8ab2c48b0 msgr2=0x55e8ab2d5a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:26:15.489 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:15.487+0000 7f8614426740 1 --2- 172.21.15.47:0/3410927139 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e8ab2c48b0 0x55e8ab2d5a00 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7f86040059b0 tx=0x7f860401c040 comp rx=0 tx=0).stop 2024-03-21T10:26:15.489 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:15.487+0000 7f8614426740 1 -- 172.21.15.47:0/3410927139 shutdown_connections 2024-03-21T10:26:15.489 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:15.487+0000 7f8614426740 1 --2- 172.21.15.47:0/3410927139 >> [v2:172.21.15.100:6832/2723302038,v1:172.21.15.100:6833/2723302038] conn(0x55e8ab339390 0x55e8ab33b780 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:26:15.489 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:15.487+0000 7f8614426740 1 --2- 172.21.15.47:0/3410927139 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e8ab2cdde0 0x55e8ab2d1120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:26:15.489 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:15.487+0000 7f8614426740 1 --2- 172.21.15.47:0/3410927139 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e8ab2c5260 0x55e8ab2d5f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:26:15.489 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:15.487+0000 7f8614426740 1 --2- 172.21.15.47:0/3410927139 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e8ab2c48b0 0x55e8ab2d5a00 unknown :-1 s=CLOSED pgs=597 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:26:15.490 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:15.487+0000 7f8614426740 1 -- 172.21.15.47:0/3410927139 >> 172.21.15.47:0/3410927139 conn(0x55e8ab2bef80 msgr2=0x55e8ab2be160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:26:15.490 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:15.488+0000 7f85f4c00640 0 client.16032 ms_handle_reset on v2:172.21.15.100:6832/2723302038 2024-03-21T10:26:15.490 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:15.488+0000 7f8614426740 1 -- 172.21.15.47:0/3410927139 shutdown_connections 2024-03-21T10:26:15.490 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:15.488+0000 7f8614426740 1 -- 172.21.15.47:0/3410927139 wait complete. 2024-03-21T10:26:15.490 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:15.488+0000 7f8614426740 20 client.16032 trim_cache size 0 max 16384 2024-03-21T10:26:15.490 INFO:teuthology.orchestra.run.smithi047.stdout:[] 2024-03-21T10:26:15.491 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:15.489+0000 7f860be00640 1 -- 172.21.15.47:0/3251068331 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f85f00a3230 msgr2=0x7f85f00a56f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:26:15.491 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:15.489+0000 7f860be00640 1 --2- 172.21.15.47:0/3251068331 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f85f00a3230 0x7f85f00a56f0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f85fc0eedb0 tx=0x7f85fc0b9bc0 comp rx=0 tx=0).stop 2024-03-21T10:26:15.491 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:15.489+0000 7f860be00640 1 -- 172.21.15.47:0/3251068331 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f860c1004c0 msgr2=0x7f860c19e360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:26:15.491 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:15.489+0000 7f860be00640 1 --2- 172.21.15.47:0/3251068331 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f860c1004c0 0x7f860c19e360 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7f85fc005940 tx=0x7f85fc02d870 comp rx=0 tx=0).stop 2024-03-21T10:26:15.491 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:15.489+0000 7f860be00640 1 -- 172.21.15.47:0/3251068331 shutdown_connections 2024-03-21T10:26:15.491 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:15.489+0000 7f860be00640 1 --2- 172.21.15.47:0/3251068331 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f860c1058c0 0x7f860c1a2c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:26:15.491 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:15.489+0000 7f860be00640 1 --2- 172.21.15.47:0/3251068331 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f85f00a3230 0x7f85f00a56f0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:26:15.491 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:15.489+0000 7f860be00640 1 --2- 172.21.15.47:0/3251068331 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f860c100de0 0x7f860c19e8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:26:15.491 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:15.489+0000 7f860be00640 1 --2- 172.21.15.47:0/3251068331 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f860c1004c0 0x7f860c19e360 unknown :-1 s=CLOSED pgs=595 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:26:15.491 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:15.489+0000 7f860be00640 1 -- 172.21.15.47:0/3251068331 >> 172.21.15.47:0/3251068331 conn(0x7f860c0fbef0 msgr2=0x7f860c10f080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:26:15.492 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:15.490+0000 7f860be00640 1 -- 172.21.15.47:0/3251068331 shutdown_connections 2024-03-21T10:26:15.492 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:15.490+0000 7f860be00640 1 -- 172.21.15.47:0/3251068331 wait complete. 2024-03-21T10:26:15.492 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:26:15 smithi157 ceph-mon[35581]: pgmap v30: 129 pgs: 129 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 597 B/s wr, 8 op/s 2024-03-21T10:26:15.492 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:26:15 smithi157 ceph-mon[35581]: from='client.? 172.21.15.47:0/1120145976' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-21T10:26:15.512 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-21T10:26:15.512 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-03-21T10:26:15.766 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:15.765+0000 7fe067000640 1 -- 172.21.15.47:0/4118788261 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe060101530 msgr2=0x7fe0601019b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:26:15.766 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:15.765+0000 7fe067000640 1 --2- 172.21.15.47:0/4118788261 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe060101530 0x7fe0601019b0 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7fe04c009900 tx=0x7fe04c02bc50 comp rx=0 tx=0).stop 2024-03-21T10:26:15.766 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:15.765+0000 7fe067000640 1 -- 172.21.15.47:0/4118788261 shutdown_connections 2024-03-21T10:26:15.766 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:15.765+0000 7fe067000640 1 --2- 172.21.15.47:0/4118788261 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe060101ef0 0x7fe06010e3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:26:15.766 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:15.765+0000 7fe067000640 1 --2- 172.21.15.47:0/4118788261 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe060101530 0x7fe0601019b0 unknown :-1 s=CLOSED pgs=598 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:26:15.766 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:15.765+0000 7fe067000640 1 --2- 172.21.15.47:0/4118788261 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe0601075f0 0x7fe0601079d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:26:15.766 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:15.765+0000 7fe067000640 1 -- 172.21.15.47:0/4118788261 >> 172.21.15.47:0/4118788261 conn(0x7fe0600fd040 msgr2=0x7fe0600ff460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:26:15.767 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:15.765+0000 7fe067000640 1 -- 172.21.15.47:0/4118788261 shutdown_connections 2024-03-21T10:26:15.767 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:15.765+0000 7fe067000640 1 -- 172.21.15.47:0/4118788261 wait complete. 2024-03-21T10:26:15.767 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:15.766+0000 7fe067000640 1 Processor -- start 2024-03-21T10:26:15.768 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:15.766+0000 7fe067000640 1 -- start start 2024-03-21T10:26:15.768 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:15.767+0000 7fe067000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe060101530 0x7fe0601a3f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:26:15.768 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:15.767+0000 7fe067000640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe060101ef0 0x7fe0601a44a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:26:15.768 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:15.767+0000 7fe067000640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe0601075f0 0x7fe06010d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:26:15.768 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:15.767+0000 7fe067000640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0601117b0 con 0x7fe060101ef0 2024-03-21T10:26:15.768 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:15.767+0000 7fe067000640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe060111630 con 0x7fe060101530 2024-03-21T10:26:15.768 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:15.767+0000 7fe067000640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe060111930 con 0x7fe0601075f0 2024-03-21T10:26:15.768 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:15.767+0000 7fe065c00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe060101530 0x7fe0601a3f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:26:15.768 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:15.767+0000 7fe065c00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe060101530 0x7fe0601a3f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:41308/0 (socket says 172.21.15.47:41308) 2024-03-21T10:26:15.768 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:15.767+0000 7fe065c00640 1 -- 172.21.15.47:0/4259757850 learned_addr learned my addr 172.21.15.47:0/4259757850 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:26:15.768 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:15.767+0000 7fe065200640 1 --2- 172.21.15.47:0/4259757850 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe060101ef0 0x7fe0601a44a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:26:15.768 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:15.767+0000 7fe066600640 1 --2- 172.21.15.47:0/4259757850 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe0601075f0 0x7fe06010d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:26:15.769 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:15.768+0000 7fe065c00640 1 -- 172.21.15.47:0/4259757850 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe0601075f0 msgr2=0x7fe06010d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:26:15.770 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:15.768+0000 7fe065c00640 1 --2- 172.21.15.47:0/4259757850 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe0601075f0 0x7fe06010d5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:26:15.770 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:15.768+0000 7fe065c00640 1 -- 172.21.15.47:0/4259757850 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe060101ef0 msgr2=0x7fe0601a44a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:26:15.770 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:15.768+0000 7fe065c00640 1 --2- 172.21.15.47:0/4259757850 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe060101ef0 0x7fe0601a44a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:26:15.770 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:15.768+0000 7fe065c00640 1 -- 172.21.15.47:0/4259757850 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe06010dd50 con 0x7fe060101530 2024-03-21T10:26:15.770 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:15.768+0000 7fe065200640 1 --2- 172.21.15.47:0/4259757850 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe060101ef0 0x7fe0601a44a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T10:26:15.770 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:15.768+0000 7fe065c00640 1 --2- 172.21.15.47:0/4259757850 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe060101530 0x7fe0601a3f60 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fe05400d950 tx=0x7fe05400de10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:26:15.770 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:15.768+0000 7fe05e000640 1 -- 172.21.15.47:0/4259757850 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe054017070 con 0x7fe060101530 2024-03-21T10:26:15.770 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:15.768+0000 7fe05e000640 1 -- 172.21.15.47:0/4259757850 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fe054009c30 con 0x7fe060101530 2024-03-21T10:26:15.770 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:15.769+0000 7fe05e000640 1 -- 172.21.15.47:0/4259757850 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe05400f620 con 0x7fe060101530 2024-03-21T10:26:15.770 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:15.769+0000 7fe067000640 1 -- 172.21.15.47:0/4259757850 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe06010e040 con 0x7fe060101530 2024-03-21T10:26:15.770 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:15.769+0000 7fe067000640 1 -- 172.21.15.47:0/4259757850 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe0601aadb0 con 0x7fe060101530 2024-03-21T10:26:15.772 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:15.772+0000 7fe06b3ad740 1 Processor -- start 2024-03-21T10:26:15.773 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:15.773+0000 7fe06b3ad740 1 -- start start 2024-03-21T10:26:15.773 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:15.773+0000 7fe06b3ad740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56153a8bc8b0 0x56153a8bcc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:26:15.774 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:15.773+0000 7fe06b3ad740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56153a8bd260 0x56153a8c5810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:26:15.774 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:15.773+0000 7fe06b3ad740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56153a8c5de0 0x56153a8c8230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:26:15.774 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:15.773+0000 7fe06b3ad740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x56153a8202c0 con 0x56153a8bc8b0 2024-03-21T10:26:15.774 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:15.773+0000 7fe06b3ad740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x56153a7c8c90 con 0x56153a8bd260 2024-03-21T10:26:15.774 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:15.773+0000 7fe06b3ad740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x56153a81c360 con 0x56153a8c5de0 2024-03-21T10:26:15.774 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:15.774+0000 7fe066600640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56153a8bc8b0 0x56153a8bcc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:26:15.774 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:15.774+0000 7fe066600640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56153a8bc8b0 0x56153a8bcc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:58576/0 (socket says 172.21.15.47:58576) 2024-03-21T10:26:15.774 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:15.774+0000 7fe05e000640 1 -- 172.21.15.47:0/4259757850 <== mon.2 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 26) v1 ==== 138087+0+0 (secure 0 0 0) 0x7fe0540109a0 con 0x7fe060101530 2024-03-21T10:26:15.774 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:15.774+0000 7fe066600640 1 -- 172.21.15.47:0/1744510389 learned_addr learned my addr 172.21.15.47:0/1744510389 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:26:15.775 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:15.774+0000 7fe066600640 1 -- 172.21.15.47:0/1744510389 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56153a8c5de0 msgr2=0x56153a8c8230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:26:15.775 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:15.774+0000 7fe066600640 1 --2- 172.21.15.47:0/1744510389 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56153a8c5de0 0x56153a8c8230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:26:15.775 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:15.774+0000 7fe066600640 1 -- 172.21.15.47:0/1744510389 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56153a8bd260 msgr2=0x56153a8c5810 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-03-21T10:26:15.775 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:15.774+0000 7fe066600640 1 --2- 172.21.15.47:0/1744510389 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56153a8bd260 0x56153a8c5810 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:26:15.775 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:15.774+0000 7fe066600640 1 -- 172.21.15.47:0/1744510389 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56153a7ed080 con 0x56153a8bc8b0 2024-03-21T10:26:15.775 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:15.775+0000 7fe066600640 1 --2- 172.21.15.47:0/1744510389 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56153a8bc8b0 0x56153a8bcc90 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7fe058009fc0 tx=0x7fe058010860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2b170ba13f9d6f09 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:26:15.776 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:15.775+0000 7fe052a00640 1 -- 172.21.15.47:0/1744510389 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe058003b70 con 0x56153a8bc8b0 2024-03-21T10:26:15.776 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:15.775+0000 7fe052a00640 1 -- 172.21.15.47:0/1744510389 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fe058003d10 con 0x56153a8bc8b0 2024-03-21T10:26:15.776 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:15.775+0000 7fe052a00640 1 -- 172.21.15.47:0/1744510389 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe058011a10 con 0x56153a8bc8b0 2024-03-21T10:26:15.776 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:15.776+0000 7fe05e000640 1 --2- 172.21.15.47:0/4259757850 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7fe0480a32b0 0x7fe0480a5770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:26:15.777 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:15.776+0000 7fe06b3ad740 1 -- 172.21.15.47:0/1744510389 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56153a8bc8b0 msgr2=0x56153a8bcc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:26:15.777 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:15.776+0000 7fe06b3ad740 1 --2- 172.21.15.47:0/1744510389 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56153a8bc8b0 0x56153a8bcc90 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7fe058009fc0 tx=0x7fe058010860 comp rx=0 tx=0).stop 2024-03-21T10:26:15.777 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:15.776+0000 7fe065c00640 1 --2- 172.21.15.47:0/4259757850 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7fe0480a32b0 0x7fe0480a5770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:26:15.778 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:15.777+0000 7fe06b3ad740 1 -- 172.21.15.47:0/1744510389 shutdown_connections 2024-03-21T10:26:15.778 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:15.777+0000 7fe06b3ad740 1 --2- 172.21.15.47:0/1744510389 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56153a8c5de0 0x56153a8c8230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:26:15.778 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:15.777+0000 7fe06b3ad740 1 --2- 172.21.15.47:0/1744510389 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56153a8bd260 0x56153a8c5810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:26:15.778 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:15.777+0000 7fe06b3ad740 1 --2- 172.21.15.47:0/1744510389 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56153a8bc8b0 0x56153a8bcc90 unknown :-1 s=CLOSED pgs=599 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:26:15.778 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:15.777+0000 7fe06b3ad740 1 -- 172.21.15.47:0/1744510389 >> 172.21.15.47:0/1744510389 conn(0x56153a8b6f80 msgr2=0x56153a8b7360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:26:15.778 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:15.777+0000 7fe05e000640 1 -- 172.21.15.47:0/4259757850 <== mon.2 v2:172.21.15.100:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 9474+0+0 (secure 0 0 0) 0x7fe0540cfb20 con 0x7fe060101530 2024-03-21T10:26:15.778 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:15.777+0000 7fe06b3ad740 1 -- 172.21.15.47:0/1744510389 shutdown_connections 2024-03-21T10:26:15.778 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:15.777+0000 7fe06b3ad740 1 -- 172.21.15.47:0/1744510389 wait complete. 2024-03-21T10:26:15.778 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:15.777+0000 7fe065c00640 1 --2- 172.21.15.47:0/4259757850 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7fe0480a32b0 0x7fe0480a5770 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fe0540ccd30 tx=0x7fe0540d3150 comp rx=0 tx=0).ready entity=mgr.15687 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:26:15.778 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:15.778+0000 7fe06b3ad740 1 -- start start 2024-03-21T10:26:15.779 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:15.778+0000 7fe06b3ad740 1 Processor -- start 2024-03-21T10:26:15.779 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:15.779+0000 7fe06b3ad740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56153a8bc8b0 0x56153a8c5670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:26:15.779 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:15.779+0000 7fe06b3ad740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56153a8bd260 0x56153a8c2a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:26:15.779 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:15.779+0000 7fe06b3ad740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56153a8c5de0 0x56153a8c2f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:26:15.779 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:15.779+0000 7fe06b3ad740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x56153a8c8d50 con 0x56153a8c5de0 2024-03-21T10:26:15.779 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:15.779+0000 7fe06b3ad740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x56153a7dc0d0 con 0x56153a8bd260 2024-03-21T10:26:15.780 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:15.779+0000 7fe06b3ad740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x56153a8c8ed0 con 0x56153a8bc8b0 2024-03-21T10:26:15.780 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:15.779+0000 7fe06b3ad740 10 client.0 _ll_register_callbacks cb 0x56153a7e0e80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T10:26:15.780 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:15.779+0000 7fe066600640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56153a8c5de0 0x56153a8c2f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:26:15.780 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:15.779+0000 7fe066600640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56153a8c5de0 0x56153a8c2f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:58580/0 (socket says 172.21.15.47:58580) 2024-03-21T10:26:15.781 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:15.779+0000 7fe066600640 1 -- 172.21.15.47:0/41437458 learned_addr learned my addr 172.21.15.47:0/41437458 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:26:15.781 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:15.779+0000 7fe065200640 1 --2- 172.21.15.47:0/41437458 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56153a8bd260 0x56153a8c2a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:26:15.781 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:15.779+0000 7fe066600640 1 --2- 172.21.15.47:0/41437458 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56153a8bc8b0 0x56153a8c5670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:26:15.781 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:15.780+0000 7fe066600640 1 -- 172.21.15.47:0/41437458 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56153a8bc8b0 msgr2=0x56153a8c5670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:26:15.781 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:15.780+0000 7fe066600640 1 --2- 172.21.15.47:0/41437458 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56153a8bc8b0 0x56153a8c5670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:26:15.781 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:15.780+0000 7fe066600640 1 -- 172.21.15.47:0/41437458 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56153a8bd260 msgr2=0x56153a8c2a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:26:15.781 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:15.780+0000 7fe066600640 1 --2- 172.21.15.47:0/41437458 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56153a8bd260 0x56153a8c2a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:26:15.781 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:15.780+0000 7fe066600640 1 -- 172.21.15.47:0/41437458 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56153a5343f0 con 0x56153a8c5de0 2024-03-21T10:26:15.781 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:15.780+0000 7fe066600640 1 --2- 172.21.15.47:0/41437458 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56153a8c5de0 0x56153a8c2f40 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7fe058011fa0 tx=0x7fe058013040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e1142b3061f1c4e2 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:26:15.781 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:15.780+0000 7fe047400640 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-03-21T10:26:15.782 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:15.781+0000 7fe047400640 1 -- 172.21.15.47:0/41437458 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe058003ec0 con 0x56153a8c5de0 2024-03-21T10:26:15.782 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:15.781+0000 7fe047400640 1 -- 172.21.15.47:0/41437458 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 9474+0+0 (secure 0 0 0) 0x7fe058004060 con 0x56153a8c5de0 2024-03-21T10:26:15.782 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:15.781+0000 7fe06b3ad740 1 -- 172.21.15.47:0/41437458 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fe058003d10 con 0x56153a8c5de0 2024-03-21T10:26:15.783 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:15.781+0000 7fe047400640 1 -- 172.21.15.47:0/41437458 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe058004260 con 0x56153a8c5de0 2024-03-21T10:26:15.783 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:15.781+0000 7fe06b3ad740 10 client.16044 fetch_fsmap learned FSMap version 30 2024-03-21T10:26:15.783 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:15.781+0000 7fe06b3ad740 1 -- 172.21.15.47:0/41437458 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x56153a5fe7d0 con 0x56153a8c5de0 2024-03-21T10:26:15.783 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:15.781+0000 7fe047400640 1 -- 172.21.15.47:0/41437458 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fe058003d10 con 0x56153a8c5de0 2024-03-21T10:26:15.783 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:15.782+0000 7fe047400640 1 -- 172.21.15.47:0/41437458 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 30) v1 ==== 7942+0+0 (secure 0 0 0) 0x7fe058029040 con 0x56153a8c5de0 2024-03-21T10:26:15.783 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:15.782+0000 7fe06b3ad740 10 client.16044 fetch_fsmap finished waiting for FSMap version 30 2024-03-21T10:26:15.783 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:15.782+0000 7fe06b3ad740 10 client.16044 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.e 2024-03-21T10:26:15.783 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:15.782+0000 7fe06b3ad740 20 client.16044 populate_metadata read hostname 'smithi047' 2024-03-21T10:26:15.783 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:15.782+0000 7fe06b3ad740 1 --2- 172.21.15.47:0/41437458 >> [v2:172.21.15.100:6834/2597849957,v1:172.21.15.100:6835/2597849957] conn(0x56153a8d5ac0 0x56153a8d7eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:26:15.783 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:15.782+0000 7fe06b3ad740 4 client.16044 mds_command: new command op to 34454 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T10:26:15.783 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:15.782+0000 7fe06b3ad740 1 -- 172.21.15.47:0/41437458 --> [v2:172.21.15.100:6834/2597849957,v1:172.21.15.100:6835/2597849957] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56153a82d800 con 0x56153a8d5ac0 2024-03-21T10:26:15.784 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:15.783+0000 7fe065200640 1 --2- 172.21.15.47:0/41437458 >> [v2:172.21.15.100:6834/2597849957,v1:172.21.15.100:6835/2597849957] conn(0x56153a8d5ac0 0x56153a8d7eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:26:15.784 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:15.784+0000 7fe065200640 1 --2- 172.21.15.47:0/41437458 >> [v2:172.21.15.100:6834/2597849957,v1:172.21.15.100:6835/2597849957] conn(0x56153a8d5ac0 0x56153a8d7eb0 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=2892e6b566863de2 server_cookie=58089b83da12e10e in_seq=0 out_seq=0 2024-03-21T10:26:15.784 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:15.784+0000 7fe047400640 10 client.16044 ms_handle_connect on v2:172.21.15.100:6834/2597849957 2024-03-21T10:26:15.785 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:15.785+0000 7fe047400640 1 -- 172.21.15.47:0/41437458 <== mds.2 v2:172.21.15.100:6834/2597849957 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x56153a82d800 con 0x56153a8d5ac0 2024-03-21T10:26:15.785 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:15.785+0000 7fe047400640 10 client.16044 handle_command_reply: tid=0 2024-03-21T10:26:15.785 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:15.785+0000 7fe06b3ad740 1 client.16044 shutdown 2024-03-21T10:26:15.785 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:15.785+0000 7fe06b3ad740 1 -- 172.21.15.47:0/41437458 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56153a8c5de0 msgr2=0x56153a8c2f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:26:15.786 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:15.785+0000 7fe06b3ad740 1 --2- 172.21.15.47:0/41437458 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56153a8c5de0 0x56153a8c2f40 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7fe058011fa0 tx=0x7fe058013040 comp rx=0 tx=0).stop 2024-03-21T10:26:15.786 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:15.785+0000 7fe06b3ad740 1 -- 172.21.15.47:0/41437458 shutdown_connections 2024-03-21T10:26:15.786 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:15.785+0000 7fe06b3ad740 1 --2- 172.21.15.47:0/41437458 >> [v2:172.21.15.100:6834/2597849957,v1:172.21.15.100:6835/2597849957] conn(0x56153a8d5ac0 0x56153a8d7eb0 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:26:15.786 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:15.785+0000 7fe06b3ad740 1 --2- 172.21.15.47:0/41437458 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56153a8c5de0 0x56153a8c2f40 unknown :-1 s=CLOSED pgs=600 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:26:15.786 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:15.785+0000 7fe06b3ad740 1 --2- 172.21.15.47:0/41437458 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56153a8bd260 0x56153a8c2a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:26:15.786 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:15.785+0000 7fe06b3ad740 1 --2- 172.21.15.47:0/41437458 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56153a8bc8b0 0x56153a8c5670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:26:15.786 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:15.785+0000 7fe06b3ad740 1 -- 172.21.15.47:0/41437458 >> 172.21.15.47:0/41437458 conn(0x56153a8b6f80 msgr2=0x56153a8b8cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:26:15.786 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:15.785+0000 7fe047400640 0 client.16044 ms_handle_reset on v2:172.21.15.100:6834/2597849957 2024-03-21T10:26:15.786 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:15.786+0000 7fe06b3ad740 1 -- 172.21.15.47:0/41437458 shutdown_connections 2024-03-21T10:26:15.787 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:15.786+0000 7fe06b3ad740 1 -- 172.21.15.47:0/41437458 wait complete. 2024-03-21T10:26:15.787 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:15.786+0000 7fe06b3ad740 20 client.16044 trim_cache size 0 max 16384 2024-03-21T10:26:15.801 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:15.801+0000 7fe06b3ad740 1 Processor -- start 2024-03-21T10:26:15.801 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:15.801+0000 7fe06b3ad740 1 -- start start 2024-03-21T10:26:15.802 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:15.801+0000 7fe06b3ad740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56153a8bc8b0 0x56153a8c5130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:26:15.802 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:15.801+0000 7fe06b3ad740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56153a8bd260 0x56153a8c5670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:26:15.802 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:15.801+0000 7fe06b3ad740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56153a8c5de0 0x56153a8bfc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:26:15.802 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:15.801+0000 7fe06b3ad740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0601b4d60 con 0x56153a8c5de0 2024-03-21T10:26:15.802 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:15.801+0000 7fe06b3ad740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0601b66e0 con 0x56153a8bd260 2024-03-21T10:26:15.802 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:15.801+0000 7fe06b3ad740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0601b53c0 con 0x56153a8bc8b0 2024-03-21T10:26:15.802 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:15.802+0000 7fe065200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56153a8bd260 0x56153a8c5670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:26:15.802 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:15.802+0000 7fe065200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56153a8bd260 0x56153a8c5670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:41346/0 (socket says 172.21.15.47:41346) 2024-03-21T10:26:15.802 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:15.802+0000 7fe065200640 1 -- 172.21.15.47:0/5551812 learned_addr learned my addr 172.21.15.47:0/5551812 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:26:15.804 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:15.802+0000 7fe066600640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56153a8c5de0 0x56153a8bfc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:26:15.804 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:15.802+0000 7fe066600640 1 --2- 172.21.15.47:0/5551812 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56153a8bc8b0 0x56153a8c5130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:26:15.804 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:15.802+0000 7fe065200640 1 -- 172.21.15.47:0/5551812 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56153a8bc8b0 msgr2=0x56153a8c5130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:26:15.804 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:15.802+0000 7fe065200640 1 --2- 172.21.15.47:0/5551812 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56153a8bc8b0 0x56153a8c5130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:26:15.804 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:15.802+0000 7fe065200640 1 -- 172.21.15.47:0/5551812 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56153a8c5de0 msgr2=0x56153a8bfc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:26:15.804 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:15.802+0000 7fe065200640 1 --2- 172.21.15.47:0/5551812 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56153a8c5de0 0x56153a8bfc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:26:15.804 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:15.802+0000 7fe065200640 1 -- 172.21.15.47:0/5551812 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56153a8b7d10 con 0x56153a8bd260 2024-03-21T10:26:15.804 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:15.803+0000 7fe066600640 1 --2- 172.21.15.47:0/5551812 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56153a8c5de0 0x56153a8bfc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T10:26:15.804 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:15.803+0000 7fe065200640 1 --2- 172.21.15.47:0/5551812 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56153a8bd260 0x56153a8c5670 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fe04c02c230 tx=0x7fe04c005be0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7264f644bb363c88 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:26:15.804 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:15.803+0000 7fe052000640 1 -- 172.21.15.47:0/5551812 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe04c0041e0 con 0x56153a8bd260 2024-03-21T10:26:15.804 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:15.803+0000 7fe052000640 1 -- 172.21.15.47:0/5551812 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fe04c02cc50 con 0x56153a8bd260 2024-03-21T10:26:15.804 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:15.803+0000 7fe052000640 1 -- 172.21.15.47:0/5551812 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe04c0046c0 con 0x56153a8bd260 2024-03-21T10:26:15.805 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:15.804+0000 7fe06b3ad740 1 -- 172.21.15.47:0/5551812 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56153a8bd260 msgr2=0x56153a8c5670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:26:15.805 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:15.804+0000 7fe06b3ad740 1 --2- 172.21.15.47:0/5551812 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56153a8bd260 0x56153a8c5670 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fe04c02c230 tx=0x7fe04c005be0 comp rx=0 tx=0).stop 2024-03-21T10:26:15.805 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:15.804+0000 7fe06b3ad740 1 -- 172.21.15.47:0/5551812 shutdown_connections 2024-03-21T10:26:15.806 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:15.804+0000 7fe06b3ad740 1 --2- 172.21.15.47:0/5551812 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56153a8c5de0 0x56153a8bfc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:26:15.806 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:15.804+0000 7fe06b3ad740 1 --2- 172.21.15.47:0/5551812 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56153a8bd260 0x56153a8c5670 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:26:15.806 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:15.804+0000 7fe06b3ad740 1 --2- 172.21.15.47:0/5551812 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56153a8bc8b0 0x56153a8c5130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:26:15.806 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:15.804+0000 7fe06b3ad740 1 -- 172.21.15.47:0/5551812 >> 172.21.15.47:0/5551812 conn(0x56153a8b6f80 msgr2=0x56153a8b6160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:26:15.806 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:15.804+0000 7fe06b3ad740 1 -- 172.21.15.47:0/5551812 shutdown_connections 2024-03-21T10:26:15.806 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:15.804+0000 7fe06b3ad740 1 -- 172.21.15.47:0/5551812 wait complete. 2024-03-21T10:26:15.806 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:15.805+0000 7fe06b3ad740 1 -- start start 2024-03-21T10:26:15.806 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:15.805+0000 7fe06b3ad740 1 Processor -- start 2024-03-21T10:26:15.808 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:15.805+0000 7fe06b3ad740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56153a8bc8b0 0x56153a8c9d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:26:15.808 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:15.806+0000 7fe06b3ad740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56153a8bd260 0x56153a8ca240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:26:15.808 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:15.806+0000 7fe06b3ad740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56153a8c5de0 0x56153a9300a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:26:15.808 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:15.806+0000 7fe06b3ad740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0601b6080 con 0x56153a8bc8b0 2024-03-21T10:26:15.808 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:15.806+0000 7fe06b3ad740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0601b5a20 con 0x56153a8c5de0 2024-03-21T10:26:15.808 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:15.806+0000 7fe06b3ad740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0601b2f20 con 0x56153a8bd260 2024-03-21T10:26:15.808 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:15.806+0000 7fe06b3ad740 10 client.0 _ll_register_callbacks cb 0x7fe058009e90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T10:26:15.809 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:15.806+0000 7fe066600640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56153a8bc8b0 0x56153a8c9d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:26:15.809 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:15.806+0000 7fe066600640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56153a8bc8b0 0x56153a8c9d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:58600/0 (socket says 172.21.15.47:58600) 2024-03-21T10:26:15.809 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:15.806+0000 7fe066600640 1 -- 172.21.15.47:0/250292972 learned_addr learned my addr 172.21.15.47:0/250292972 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:26:15.809 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:15.806+0000 7fe065200640 1 --2- 172.21.15.47:0/250292972 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56153a8bd260 0x56153a8ca240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:26:15.809 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:15.806+0000 7fe066600640 1 --2- 172.21.15.47:0/250292972 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56153a8c5de0 0x56153a9300a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:26:15.809 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:15.806+0000 7fe066600640 1 -- 172.21.15.47:0/250292972 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56153a8bd260 msgr2=0x56153a8ca240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:26:15.809 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:15.806+0000 7fe066600640 1 --2- 172.21.15.47:0/250292972 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56153a8bd260 0x56153a8ca240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:26:15.809 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:15.807+0000 7fe066600640 1 -- 172.21.15.47:0/250292972 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56153a8c5de0 msgr2=0x56153a9300a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:26:15.809 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:15.807+0000 7fe066600640 1 --2- 172.21.15.47:0/250292972 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56153a8c5de0 0x56153a9300a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:26:15.809 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:15.807+0000 7fe066600640 1 -- 172.21.15.47:0/250292972 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56153a9306b0 con 0x56153a8bc8b0 2024-03-21T10:26:15.809 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:15.807+0000 7fe065200640 1 --2- 172.21.15.47:0/250292972 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56153a8bd260 0x56153a8ca240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T10:26:15.809 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:15.807+0000 7fe066600640 1 --2- 172.21.15.47:0/250292972 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56153a8bc8b0 0x56153a8c9d00 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7fe058011fa0 tx=0x7fe058010fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d316c7e1ace194b2 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:26:15.809 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:15.807+0000 7fe047400640 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-03-21T10:26:15.809 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:15.807+0000 7fe047400640 1 -- 172.21.15.47:0/250292972 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe058039040 con 0x56153a8bc8b0 2024-03-21T10:26:15.810 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:15.807+0000 7fe047400640 1 -- 172.21.15.47:0/250292972 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 9474+0+0 (secure 0 0 0) 0x7fe058023040 con 0x56153a8bc8b0 2024-03-21T10:26:15.810 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:15.807+0000 7fe06b3ad740 1 -- 172.21.15.47:0/250292972 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fe04c02cc50 con 0x56153a8bc8b0 2024-03-21T10:26:15.810 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:15.808+0000 7fe047400640 1 -- 172.21.15.47:0/250292972 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe058045050 con 0x56153a8bc8b0 2024-03-21T10:26:15.810 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:15.808+0000 7fe06b3ad740 10 client.16050 fetch_fsmap learned FSMap version 30 2024-03-21T10:26:15.810 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:15.808+0000 7fe06b3ad740 1 -- 172.21.15.47:0/250292972 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x56153a9319b0 con 0x56153a8bc8b0 2024-03-21T10:26:15.810 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:15.808+0000 7fe047400640 1 -- 172.21.15.47:0/250292972 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fe04c02cc50 con 0x56153a8bc8b0 2024-03-21T10:26:15.812 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:15.808+0000 7fe047400640 1 -- 172.21.15.47:0/250292972 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 30) v1 ==== 7942+0+0 (secure 0 0 0) 0x7fe05803e060 con 0x56153a8bc8b0 2024-03-21T10:26:15.812 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:15.808+0000 7fe06b3ad740 10 client.16050 fetch_fsmap finished waiting for FSMap version 30 2024-03-21T10:26:15.812 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:15.808+0000 7fe06b3ad740 10 client.16050 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.e 2024-03-21T10:26:15.812 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:15.808+0000 7fe06b3ad740 20 client.16050 populate_metadata read hostname 'smithi047' 2024-03-21T10:26:15.812 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:15.808+0000 7fe06b3ad740 1 --2- 172.21.15.47:0/250292972 >> [v2:172.21.15.100:6834/2597849957,v1:172.21.15.100:6835/2597849957] conn(0x56153a9320e0 0x56153a934500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:26:15.812 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:15.808+0000 7fe06b3ad740 4 client.16050 mds_command: new command op to 34454 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-21T10:26:15.812 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:15.808+0000 7fe06b3ad740 1 -- 172.21.15.47:0/250292972 --> [v2:172.21.15.100:6834/2597849957,v1:172.21.15.100:6835/2597849957] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x56153a827b50 con 0x56153a9320e0 2024-03-21T10:26:15.812 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:15.809+0000 7fe065200640 1 --2- 172.21.15.47:0/250292972 >> [v2:172.21.15.100:6834/2597849957,v1:172.21.15.100:6835/2597849957] conn(0x56153a9320e0 0x56153a934500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:26:15.812 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:15.810+0000 7fe065200640 1 --2- 172.21.15.47:0/250292972 >> [v2:172.21.15.100:6834/2597849957,v1:172.21.15.100:6835/2597849957] conn(0x56153a9320e0 0x56153a934500 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=21f3be4b0ad608ce server_cookie=1718a4694941dc57 in_seq=0 out_seq=0 2024-03-21T10:26:15.812 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:15.810+0000 7fe047400640 10 client.16050 ms_handle_connect on v2:172.21.15.100:6834/2597849957 2024-03-21T10:26:15.812 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:15.810+0000 7fe047400640 1 -- 172.21.15.47:0/250292972 <== mds.2 v2:172.21.15.100:6834/2597849957 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x56153a827b50 con 0x56153a9320e0 2024-03-21T10:26:15.812 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:15.810+0000 7fe047400640 10 client.16050 handle_command_reply: tid=0 2024-03-21T10:26:15.813 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:15.810+0000 7fe06b3ad740 1 client.16050 shutdown 2024-03-21T10:26:15.813 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:15.811+0000 7fe06b3ad740 1 -- 172.21.15.47:0/250292972 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56153a8bc8b0 msgr2=0x56153a8c9d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:26:15.813 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:15.811+0000 7fe06b3ad740 1 --2- 172.21.15.47:0/250292972 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56153a8bc8b0 0x56153a8c9d00 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7fe058011fa0 tx=0x7fe058010fd0 comp rx=0 tx=0).stop 2024-03-21T10:26:15.813 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:15.811+0000 7fe06b3ad740 1 -- 172.21.15.47:0/250292972 shutdown_connections 2024-03-21T10:26:15.813 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:15.811+0000 7fe06b3ad740 1 --2- 172.21.15.47:0/250292972 >> [v2:172.21.15.100:6834/2597849957,v1:172.21.15.100:6835/2597849957] conn(0x56153a9320e0 0x56153a934500 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:26:15.813 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:15.811+0000 7fe06b3ad740 1 --2- 172.21.15.47:0/250292972 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56153a8c5de0 0x56153a9300a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:26:15.813 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:15.811+0000 7fe06b3ad740 1 --2- 172.21.15.47:0/250292972 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56153a8bd260 0x56153a8ca240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:26:15.813 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:15.811+0000 7fe06b3ad740 1 --2- 172.21.15.47:0/250292972 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56153a8bc8b0 0x56153a8c9d00 unknown :-1 s=CLOSED pgs=601 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:26:15.813 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:15.811+0000 7fe047400640 0 client.16050 ms_handle_reset on v2:172.21.15.100:6834/2597849957 2024-03-21T10:26:15.813 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:15.811+0000 7fe06b3ad740 1 -- 172.21.15.47:0/250292972 >> 172.21.15.47:0/250292972 conn(0x56153a8b6f80 msgr2=0x56153a8bd640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:26:15.813 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:15.811+0000 7fe06b3ad740 1 -- 172.21.15.47:0/250292972 shutdown_connections 2024-03-21T10:26:15.813 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:15.811+0000 7fe06b3ad740 1 -- 172.21.15.47:0/250292972 wait complete. 2024-03-21T10:26:15.813 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:15.811+0000 7fe06b3ad740 20 client.16050 trim_cache size 0 max 16384 2024-03-21T10:26:15.814 INFO:teuthology.orchestra.run.smithi047.stdout:[] 2024-03-21T10:26:15.814 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:15.812+0000 7fe067000640 1 -- 172.21.15.47:0/4259757850 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7fe0480a32b0 msgr2=0x7fe0480a5770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:26:15.815 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:15.812+0000 7fe067000640 1 --2- 172.21.15.47:0/4259757850 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7fe0480a32b0 0x7fe0480a5770 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fe0540ccd30 tx=0x7fe0540d3150 comp rx=0 tx=0).stop 2024-03-21T10:26:15.815 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:15.812+0000 7fe067000640 1 -- 172.21.15.47:0/4259757850 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe060101530 msgr2=0x7fe0601a3f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:26:15.815 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:15.812+0000 7fe067000640 1 --2- 172.21.15.47:0/4259757850 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe060101530 0x7fe0601a3f60 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fe05400d950 tx=0x7fe05400de10 comp rx=0 tx=0).stop 2024-03-21T10:26:15.815 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:15.812+0000 7fe067000640 1 -- 172.21.15.47:0/4259757850 shutdown_connections 2024-03-21T10:26:15.815 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:15.812+0000 7fe067000640 1 --2- 172.21.15.47:0/4259757850 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe0601075f0 0x7fe06010d5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:26:15.815 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:15.812+0000 7fe067000640 1 --2- 172.21.15.47:0/4259757850 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe060101ef0 0x7fe0601a44a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:26:15.815 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:15.812+0000 7fe067000640 1 --2- 172.21.15.47:0/4259757850 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7fe0480a32b0 0x7fe0480a5770 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:26:15.815 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:15.812+0000 7fe067000640 1 --2- 172.21.15.47:0/4259757850 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe060101530 0x7fe0601a3f60 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:26:15.815 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:15.812+0000 7fe067000640 1 -- 172.21.15.47:0/4259757850 >> 172.21.15.47:0/4259757850 conn(0x7fe0600fd040 msgr2=0x7fe0600fe740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:26:15.815 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:15.812+0000 7fe067000640 1 -- 172.21.15.47:0/4259757850 shutdown_connections 2024-03-21T10:26:15.815 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:15.812+0000 7fe067000640 1 -- 172.21.15.47:0/4259757850 wait complete. 2024-03-21T10:26:15.833 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-21T10:26:15.833 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-03-21T10:26:16.020 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:16.020+0000 7f1b5fe00640 1 -- 172.21.15.47:0/3083514645 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1b60103720 msgr2=0x7f1b60103ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:26:16.021 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:16.020+0000 7f1b5fe00640 1 --2- 172.21.15.47:0/3083514645 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1b60103720 0x7f1b60103ba0 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7f1b4c009930 tx=0x7f1b4c02bc30 comp rx=0 tx=0).stop 2024-03-21T10:26:16.021 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:16.020+0000 7f1b5fe00640 1 -- 172.21.15.47:0/3083514645 shutdown_connections 2024-03-21T10:26:16.021 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:16.020+0000 7f1b5fe00640 1 --2- 172.21.15.47:0/3083514645 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1b601040e0 0x7f1b601105f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:26:16.021 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:16.020+0000 7f1b5fe00640 1 --2- 172.21.15.47:0/3083514645 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1b60103720 0x7f1b60103ba0 unknown :-1 s=CLOSED pgs=602 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:26:16.021 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:16.020+0000 7f1b5fe00640 1 --2- 172.21.15.47:0/3083514645 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1b60109730 0x7f1b60109b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:26:16.021 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:16.020+0000 7f1b5fe00640 1 -- 172.21.15.47:0/3083514645 >> 172.21.15.47:0/3083514645 conn(0x7f1b600ff1b0 msgr2=0x7f1b601015d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:26:16.021 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:16.020+0000 7f1b5fe00640 1 -- 172.21.15.47:0/3083514645 shutdown_connections 2024-03-21T10:26:16.021 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:16.020+0000 7f1b5fe00640 1 -- 172.21.15.47:0/3083514645 wait complete. 2024-03-21T10:26:16.021 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:16.021+0000 7f1b5fe00640 1 Processor -- start 2024-03-21T10:26:16.022 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:16.022+0000 7f1b5fe00640 1 -- start start 2024-03-21T10:26:16.023 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:16.022+0000 7f1b5fe00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1b60103720 0x7f1b60110550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:26:16.023 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:16.022+0000 7f1b5fe00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1b601040e0 0x7f1b6010d690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:26:16.023 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:16.022+0000 7f1b5fe00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1b60109730 0x7f1b6010dc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:26:16.023 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:16.022+0000 7f1b5fe00640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b60113a00 con 0x7f1b601040e0 2024-03-21T10:26:16.023 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:16.022+0000 7f1b5fe00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b60113880 con 0x7f1b60103720 2024-03-21T10:26:16.023 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:16.022+0000 7f1b5fe00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b60113b80 con 0x7f1b60109730 2024-03-21T10:26:16.024 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:16.023+0000 7f1b5ea00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1b60103720 0x7f1b60110550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:26:16.024 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:16.023+0000 7f1b5e000640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1b601040e0 0x7f1b6010d690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:26:16.024 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:16.023+0000 7f1b5ea00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1b60103720 0x7f1b60110550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:41374/0 (socket says 172.21.15.47:41374) 2024-03-21T10:26:16.024 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:16.023+0000 7f1b5ea00640 1 -- 172.21.15.47:0/584706725 learned_addr learned my addr 172.21.15.47:0/584706725 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:26:16.024 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:16.023+0000 7f1b5f400640 1 --2- 172.21.15.47:0/584706725 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1b60109730 0x7f1b6010dc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:26:16.024 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:16.023+0000 7f1b5ea00640 1 -- 172.21.15.47:0/584706725 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1b60109730 msgr2=0x7f1b6010dc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:26:16.024 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:16.023+0000 7f1b5ea00640 1 --2- 172.21.15.47:0/584706725 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1b60109730 0x7f1b6010dc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:26:16.024 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:16.023+0000 7f1b5ea00640 1 -- 172.21.15.47:0/584706725 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1b601040e0 msgr2=0x7f1b6010d690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:26:16.024 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:16.023+0000 7f1b5ea00640 1 --2- 172.21.15.47:0/584706725 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1b601040e0 0x7f1b6010d690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:26:16.024 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:16.023+0000 7f1b5ea00640 1 -- 172.21.15.47:0/584706725 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b601aa700 con 0x7f1b60103720 2024-03-21T10:26:16.024 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:16.023+0000 7f1b5e000640 1 --2- 172.21.15.47:0/584706725 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1b601040e0 0x7f1b6010d690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T10:26:16.025 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:16.024+0000 7f1b5ea00640 1 --2- 172.21.15.47:0/584706725 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1b60103720 0x7f1b60110550 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f1b5000d950 tx=0x7f1b5000de10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:26:16.025 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:16.024+0000 7f1b64a00640 1 -- 172.21.15.47:0/584706725 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b50017030 con 0x7f1b60103720 2024-03-21T10:26:16.025 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:16.024+0000 7f1b5fe00640 1 -- 172.21.15.47:0/584706725 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1b601aa8d0 con 0x7f1b60103720 2024-03-21T10:26:16.025 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:16.024+0000 7f1b64a00640 1 -- 172.21.15.47:0/584706725 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f1b50004020 con 0x7f1b60103720 2024-03-21T10:26:16.025 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:16.024+0000 7f1b64a00640 1 -- 172.21.15.47:0/584706725 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b500047a0 con 0x7f1b60103720 2024-03-21T10:26:16.025 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:16.024+0000 7f1b5fe00640 1 -- 172.21.15.47:0/584706725 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1b601aae40 con 0x7f1b60103720 2024-03-21T10:26:16.028 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:16.028+0000 7f1b68b79740 1 Processor -- start 2024-03-21T10:26:16.028 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:16.028+0000 7f1b68b79740 1 -- start start 2024-03-21T10:26:16.028 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:16.028+0000 7f1b68b79740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b4b3cbc8b0 0x55b4b3cbcc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:26:16.029 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:16.028+0000 7f1b68b79740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b4b3cbd260 0x55b4b3cc5810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:26:16.029 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:16.028+0000 7f1b68b79740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b4b3cc5de0 0x55b4b3cc8230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:26:16.029 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:16.028+0000 7f1b68b79740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b4b3c202c0 con 0x55b4b3cbd260 2024-03-21T10:26:16.029 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:16.028+0000 7f1b68b79740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b4b3bc8c90 con 0x55b4b3cbc8b0 2024-03-21T10:26:16.029 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:16.028+0000 7f1b68b79740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b4b3c1c360 con 0x55b4b3cc5de0 2024-03-21T10:26:16.030 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:16.029+0000 7f1b5f400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b4b3cbc8b0 0x55b4b3cbcc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:26:16.030 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:16.029+0000 7f1b5f400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b4b3cbc8b0 0x55b4b3cbcc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:41376/0 (socket says 172.21.15.47:41376) 2024-03-21T10:26:16.030 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:16.029+0000 7f1b5f400640 1 -- 172.21.15.47:0/1421066492 learned_addr learned my addr 172.21.15.47:0/1421066492 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:26:16.030 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:16.029+0000 7f1b5e000640 1 --2- 172.21.15.47:0/1421066492 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b4b3cc5de0 0x55b4b3cc8230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:26:16.030 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:16.029+0000 7f1b5f400640 1 -- 172.21.15.47:0/1421066492 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b4b3cc5de0 msgr2=0x55b4b3cc8230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:26:16.030 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:16.029+0000 7f1b5f400640 1 --2- 172.21.15.47:0/1421066492 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b4b3cc5de0 0x55b4b3cc8230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:26:16.030 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:16.029+0000 7f1b5f400640 1 -- 172.21.15.47:0/1421066492 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b4b3cbd260 msgr2=0x55b4b3cc5810 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-21T10:26:16.030 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:16.029+0000 7f1b5f400640 1 --2- 172.21.15.47:0/1421066492 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b4b3cbd260 0x55b4b3cc5810 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:26:16.030 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:16.029+0000 7f1b5f400640 1 -- 172.21.15.47:0/1421066492 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b4b3bed080 con 0x55b4b3cbc8b0 2024-03-21T10:26:16.030 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:16.029+0000 7f1b5f400640 1 --2- 172.21.15.47:0/1421066492 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b4b3cbc8b0 0x55b4b3cbcc90 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f1b58010370 tx=0x7f1b58010830 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c453b571960ee12e server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:26:16.031 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:16.030+0000 7f1b64a00640 1 -- 172.21.15.47:0/584706725 <== mon.2 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 26) v1 ==== 138087+0+0 (secure 0 0 0) 0x7f1b500041d0 con 0x7f1b60103720 2024-03-21T10:26:16.031 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:16.030+0000 7f1b4be00640 1 -- 172.21.15.47:0/1421066492 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b58016070 con 0x55b4b3cbc8b0 2024-03-21T10:26:16.031 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:16.030+0000 7f1b4be00640 1 -- 172.21.15.47:0/1421066492 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f1b58004d20 con 0x55b4b3cbc8b0 2024-03-21T10:26:16.031 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:16.030+0000 7f1b4be00640 1 -- 172.21.15.47:0/1421066492 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b58005350 con 0x55b4b3cbc8b0 2024-03-21T10:26:16.031 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:16.031+0000 7f1b68b79740 1 -- 172.21.15.47:0/1421066492 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b4b3cbc8b0 msgr2=0x55b4b3cbcc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:26:16.031 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:16.031+0000 7f1b68b79740 1 --2- 172.21.15.47:0/1421066492 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b4b3cbc8b0 0x55b4b3cbcc90 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f1b58010370 tx=0x7f1b58010830 comp rx=0 tx=0).stop 2024-03-21T10:26:16.031 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:16.031+0000 7f1b68b79740 1 -- 172.21.15.47:0/1421066492 shutdown_connections 2024-03-21T10:26:16.032 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:16.031+0000 7f1b68b79740 1 --2- 172.21.15.47:0/1421066492 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b4b3cc5de0 0x55b4b3cc8230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:26:16.032 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:16.031+0000 7f1b68b79740 1 --2- 172.21.15.47:0/1421066492 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b4b3cbd260 0x55b4b3cc5810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:26:16.032 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:16.031+0000 7f1b68b79740 1 --2- 172.21.15.47:0/1421066492 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b4b3cbc8b0 0x55b4b3cbcc90 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:26:16.032 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:16.031+0000 7f1b68b79740 1 -- 172.21.15.47:0/1421066492 >> 172.21.15.47:0/1421066492 conn(0x55b4b3cb6f80 msgr2=0x55b4b3cb7360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:26:16.032 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:16.031+0000 7f1b68b79740 1 -- 172.21.15.47:0/1421066492 shutdown_connections 2024-03-21T10:26:16.032 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:16.031+0000 7f1b68b79740 1 -- 172.21.15.47:0/1421066492 wait complete. 2024-03-21T10:26:16.032 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:16.032+0000 7f1b68b79740 1 -- start start 2024-03-21T10:26:16.033 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:16.032+0000 7f1b64a00640 1 --2- 172.21.15.47:0/584706725 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f1b440a30c0 0x7f1b440a5580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:26:16.033 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:16.032+0000 7f1b68b79740 1 Processor -- start 2024-03-21T10:26:16.033 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:16.032+0000 7f1b64a00640 1 -- 172.21.15.47:0/584706725 <== mon.2 v2:172.21.15.100:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 9474+0+0 (secure 0 0 0) 0x7f1b500cf830 con 0x7f1b60103720 2024-03-21T10:26:16.033 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:16.033+0000 7f1b5f400640 1 --2- 172.21.15.47:0/584706725 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f1b440a30c0 0x7f1b440a5580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:26:16.035 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:16.034+0000 7f1b68b79740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b4b3cbc8b0 0x55b4b3cc57a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:26:16.035 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:16.034+0000 7f1b68b79740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b4b3cbd260 0x55b4b3cc2b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:26:16.035 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:16.034+0000 7f1b68b79740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b4b3cc5de0 0x55b4b3cc3220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:26:16.035 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:16.034+0000 7f1b68b79740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b4b3cc8d50 con 0x55b4b3cbc8b0 2024-03-21T10:26:16.035 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:16.034+0000 7f1b68b79740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b4b3bdc0d0 con 0x55b4b3cbd260 2024-03-21T10:26:16.035 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:16.034+0000 7f1b68b79740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b4b3cc8ed0 con 0x55b4b3cc5de0 2024-03-21T10:26:16.035 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:16.034+0000 7f1b68b79740 10 client.0 _ll_register_callbacks cb 0x55b4b3be0e80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T10:26:16.036 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:16.034+0000 7f1b5f400640 1 --2- 172.21.15.47:0/584706725 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f1b440a30c0 0x7f1b440a5580 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f1b58004ff0 tx=0x7f1b58000f40 comp rx=0 tx=0).ready entity=mgr.15687 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:26:16.036 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:16.034+0000 7f1b5e000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b4b3cbd260 0x55b4b3cc2b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:26:16.036 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:16.034+0000 7f1b5e000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b4b3cbd260 0x55b4b3cc2b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:41384/0 (socket says 172.21.15.47:41384) 2024-03-21T10:26:16.036 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:16.034+0000 7f1b5e000640 1 -- 172.21.15.47:0/31339683 learned_addr learned my addr 172.21.15.47:0/31339683 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:26:16.036 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:16.035+0000 7f1b5f400640 1 --2- 172.21.15.47:0/31339683 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b4b3cc5de0 0x55b4b3cc3220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:26:16.036 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:16.035+0000 7f1b5ea00640 1 --2- 172.21.15.47:0/31339683 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b4b3cbc8b0 0x55b4b3cc57a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:26:16.037 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:16.035+0000 7f1b5e000640 1 -- 172.21.15.47:0/31339683 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b4b3cc5de0 msgr2=0x55b4b3cc3220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:26:16.037 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:16.035+0000 7f1b5e000640 1 --2- 172.21.15.47:0/31339683 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b4b3cc5de0 0x55b4b3cc3220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:26:16.037 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:16.035+0000 7f1b5e000640 1 -- 172.21.15.47:0/31339683 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b4b3cbc8b0 msgr2=0x55b4b3cc57a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:26:16.037 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:16.035+0000 7f1b5e000640 1 --2- 172.21.15.47:0/31339683 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b4b3cbc8b0 0x55b4b3cc57a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:26:16.037 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:16.035+0000 7f1b5e000640 1 -- 172.21.15.47:0/31339683 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55b4b39343f0 con 0x55b4b3cbd260 2024-03-21T10:26:16.037 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:16.035+0000 7f1b5e000640 1 --2- 172.21.15.47:0/31339683 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b4b3cbd260 0x55b4b3cc2b30 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f1b4c005ab0 tx=0x7f1b4c02ca10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5ce0fa25790ae535 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:26:16.037 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:16.035+0000 7f1b48c00640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-21T10:26:16.037 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:16.036+0000 7f1b48c00640 1 -- 172.21.15.47:0/31339683 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b4c02c5d0 con 0x55b4b3cbd260 2024-03-21T10:26:16.037 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:16.036+0000 7f1b48c00640 1 -- 172.21.15.47:0/31339683 <== mon.2 v2:172.21.15.100:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 9474+0+0 (secure 0 0 0) 0x7f1b4c0035f0 con 0x55b4b3cbd260 2024-03-21T10:26:16.038 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:16.036+0000 7f1b68b79740 1 -- 172.21.15.47:0/31339683 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f1b58004d20 con 0x55b4b3cbd260 2024-03-21T10:26:16.038 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:16.036+0000 7f1b48c00640 1 -- 172.21.15.47:0/31339683 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1b4c054050 con 0x55b4b3cbd260 2024-03-21T10:26:16.038 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:16.036+0000 7f1b68b79740 10 client.35864 fetch_fsmap learned FSMap version 30 2024-03-21T10:26:16.038 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:16.036+0000 7f1b68b79740 1 -- 172.21.15.47:0/31339683 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x55b4b39fe7d0 con 0x55b4b3cbd260 2024-03-21T10:26:16.038 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:16.037+0000 7f1b48c00640 1 -- 172.21.15.47:0/31339683 <== mon.2 v2:172.21.15.100:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f1b58004d20 con 0x55b4b3cbd260 2024-03-21T10:26:16.038 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:16.037+0000 7f1b48c00640 1 -- 172.21.15.47:0/31339683 <== mon.2 v2:172.21.15.100:3300/0 5 ==== fsmap(e 30) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f1b4c04ba50 con 0x55b4b3cbd260 2024-03-21T10:26:16.039 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:16.038+0000 7f1b68b79740 10 client.35864 fetch_fsmap finished waiting for FSMap version 30 2024-03-21T10:26:16.039 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:16.038+0000 7f1b68b79740 10 client.35864 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-03-21T10:26:16.039 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:16.038+0000 7f1b68b79740 20 client.35864 populate_metadata read hostname 'smithi047' 2024-03-21T10:26:16.039 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:16.038+0000 7f1b68b79740 1 --2- 172.21.15.47:0/31339683 >> [v2:172.21.15.100:6836/3497644162,v1:172.21.15.100:6837/3497644162] conn(0x55b4b3cd5ce0 0x55b4b3cd80d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:26:16.039 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:16.038+0000 7f1b68b79740 4 client.35864 mds_command: new command op to 34493 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T10:26:16.039 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:16.038+0000 7f1b68b79740 1 -- 172.21.15.47:0/31339683 --> [v2:172.21.15.100:6836/3497644162,v1:172.21.15.100:6837/3497644162] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55b4b3c2d800 con 0x55b4b3cd5ce0 2024-03-21T10:26:16.039 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:16.038+0000 7f1b5ea00640 1 --2- 172.21.15.47:0/31339683 >> [v2:172.21.15.100:6836/3497644162,v1:172.21.15.100:6837/3497644162] conn(0x55b4b3cd5ce0 0x55b4b3cd80d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:26:16.039 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:16.039+0000 7f1b5ea00640 1 --2- 172.21.15.47:0/31339683 >> [v2:172.21.15.100:6836/3497644162,v1:172.21.15.100:6837/3497644162] conn(0x55b4b3cd5ce0 0x55b4b3cd80d0 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=2bb5cda3aac8cbe5 server_cookie=3aa08a8f19503b6b in_seq=0 out_seq=0 2024-03-21T10:26:16.039 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:16.039+0000 7f1b48c00640 10 client.35864 ms_handle_connect on v2:172.21.15.100:6836/3497644162 2024-03-21T10:26:16.041 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:16.040+0000 7f1b48c00640 1 -- 172.21.15.47:0/31339683 <== mds.1 v2:172.21.15.100:6836/3497644162 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55b4b3c2d800 con 0x55b4b3cd5ce0 2024-03-21T10:26:16.041 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:16.041+0000 7f1b48c00640 10 client.35864 handle_command_reply: tid=0 2024-03-21T10:26:16.041 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:16.041+0000 7f1b68b79740 1 client.35864 shutdown 2024-03-21T10:26:16.042 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:16.041+0000 7f1b68b79740 1 -- 172.21.15.47:0/31339683 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b4b3cbd260 msgr2=0x55b4b3cc2b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:26:16.042 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:16.041+0000 7f1b68b79740 1 --2- 172.21.15.47:0/31339683 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b4b3cbd260 0x55b4b3cc2b30 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f1b4c005ab0 tx=0x7f1b4c02ca10 comp rx=0 tx=0).stop 2024-03-21T10:26:16.042 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:16.041+0000 7f1b68b79740 1 -- 172.21.15.47:0/31339683 shutdown_connections 2024-03-21T10:26:16.042 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:16.041+0000 7f1b68b79740 1 --2- 172.21.15.47:0/31339683 >> [v2:172.21.15.100:6836/3497644162,v1:172.21.15.100:6837/3497644162] conn(0x55b4b3cd5ce0 0x55b4b3cd80d0 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:26:16.042 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:16.041+0000 7f1b68b79740 1 --2- 172.21.15.47:0/31339683 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b4b3cc5de0 0x55b4b3cc3220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:26:16.042 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:16.041+0000 7f1b68b79740 1 --2- 172.21.15.47:0/31339683 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b4b3cbd260 0x55b4b3cc2b30 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:26:16.042 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:16.041+0000 7f1b68b79740 1 --2- 172.21.15.47:0/31339683 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b4b3cbc8b0 0x55b4b3cc57a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:26:16.042 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:16.041+0000 7f1b68b79740 1 -- 172.21.15.47:0/31339683 >> 172.21.15.47:0/31339683 conn(0x55b4b3cb6f80 msgr2=0x55b4b3cb8cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:26:16.042 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:16.041+0000 7f1b48c00640 0 client.35864 ms_handle_reset on v2:172.21.15.100:6836/3497644162 2024-03-21T10:26:16.043 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:16.041+0000 7f1b68b79740 1 -- 172.21.15.47:0/31339683 shutdown_connections 2024-03-21T10:26:16.043 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:16.041+0000 7f1b68b79740 1 -- 172.21.15.47:0/31339683 wait complete. 2024-03-21T10:26:16.043 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:16.042+0000 7f1b68b79740 20 client.35864 trim_cache size 0 max 16384 2024-03-21T10:26:16.067 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:16.067+0000 7f1b68b79740 1 Processor -- start 2024-03-21T10:26:16.068 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:16.067+0000 7f1b68b79740 1 -- start start 2024-03-21T10:26:16.068 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:16.068+0000 7f1b68b79740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b4b3cbc8b0 0x55b4b3cc5260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:26:16.068 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:16.068+0000 7f1b68b79740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b4b3cbd260 0x55b4b3cc57a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:26:16.068 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:16.068+0000 7f1b68b79740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b4b3cc5de0 0x55b4b3cbfdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:26:16.068 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:16.068+0000 7f1b68b79740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b601b4da0 con 0x55b4b3cbd260 2024-03-21T10:26:16.068 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:16.068+0000 7f1b68b79740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b601b6720 con 0x55b4b3cc5de0 2024-03-21T10:26:16.068 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:16.068+0000 7f1b68b79740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b601b5400 con 0x55b4b3cbc8b0 2024-03-21T10:26:16.069 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:16.068+0000 7f1b5e000640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b4b3cbd260 0x55b4b3cc57a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:26:16.069 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:16.068+0000 7f1b5f400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b4b3cc5de0 0x55b4b3cbfdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:26:16.069 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:16.068+0000 7f1b5f400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b4b3cc5de0 0x55b4b3cbfdb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:41400/0 (socket says 172.21.15.47:41400) 2024-03-21T10:26:16.069 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:16.068+0000 7f1b5ea00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b4b3cbc8b0 0x55b4b3cc5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:26:16.069 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:16.068+0000 7f1b5f400640 1 -- 172.21.15.47:0/3205502887 learned_addr learned my addr 172.21.15.47:0/3205502887 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:26:16.070 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:16.069+0000 7f1b5f400640 1 -- 172.21.15.47:0/3205502887 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b4b3cbc8b0 msgr2=0x55b4b3cc5260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:26:16.070 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:16.069+0000 7f1b5f400640 1 --2- 172.21.15.47:0/3205502887 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b4b3cbc8b0 0x55b4b3cc5260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:26:16.070 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:16.069+0000 7f1b5f400640 1 -- 172.21.15.47:0/3205502887 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b4b3cbd260 msgr2=0x55b4b3cc57a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:26:16.070 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:16.069+0000 7f1b5f400640 1 --2- 172.21.15.47:0/3205502887 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b4b3cbd260 0x55b4b3cc57a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:26:16.070 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:16.069+0000 7f1b5f400640 1 -- 172.21.15.47:0/3205502887 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b4b3cc8650 con 0x55b4b3cc5de0 2024-03-21T10:26:16.070 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:16.069+0000 7f1b5f400640 1 --2- 172.21.15.47:0/3205502887 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b4b3cc5de0 0x55b4b3cbfdb0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f1b58009a30 tx=0x7f1b58024000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d76bce3cefb64217 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:26:16.070 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:16.070+0000 7f1b4b400640 1 -- 172.21.15.47:0/3205502887 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b58016050 con 0x55b4b3cc5de0 2024-03-21T10:26:16.070 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:16.070+0000 7f1b4b400640 1 -- 172.21.15.47:0/3205502887 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f1b58003e20 con 0x55b4b3cc5de0 2024-03-21T10:26:16.070 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:16.070+0000 7f1b4b400640 1 -- 172.21.15.47:0/3205502887 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b58005280 con 0x55b4b3cc5de0 2024-03-21T10:26:16.071 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:16.071+0000 7f1b68b79740 1 -- 172.21.15.47:0/3205502887 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b4b3cc5de0 msgr2=0x55b4b3cbfdb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:26:16.071 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:16.071+0000 7f1b68b79740 1 --2- 172.21.15.47:0/3205502887 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b4b3cc5de0 0x55b4b3cbfdb0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f1b58009a30 tx=0x7f1b58024000 comp rx=0 tx=0).stop 2024-03-21T10:26:16.071 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:16.071+0000 7f1b68b79740 1 -- 172.21.15.47:0/3205502887 shutdown_connections 2024-03-21T10:26:16.071 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:16.071+0000 7f1b68b79740 1 --2- 172.21.15.47:0/3205502887 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b4b3cc5de0 0x55b4b3cbfdb0 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:26:16.071 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:16.071+0000 7f1b68b79740 1 --2- 172.21.15.47:0/3205502887 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b4b3cbd260 0x55b4b3cc57a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:26:16.072 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:16.071+0000 7f1b68b79740 1 --2- 172.21.15.47:0/3205502887 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b4b3cbc8b0 0x55b4b3cc5260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:26:16.072 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:16.071+0000 7f1b68b79740 1 -- 172.21.15.47:0/3205502887 >> 172.21.15.47:0/3205502887 conn(0x55b4b3cb6f80 msgr2=0x55b4b3cb6160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:26:16.072 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:16.071+0000 7f1b68b79740 1 -- 172.21.15.47:0/3205502887 shutdown_connections 2024-03-21T10:26:16.072 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:16.071+0000 7f1b68b79740 1 -- 172.21.15.47:0/3205502887 wait complete. 2024-03-21T10:26:16.072 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:16.071+0000 7f1b68b79740 1 -- start start 2024-03-21T10:26:16.072 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:16.072+0000 7f1b68b79740 1 Processor -- start 2024-03-21T10:26:16.073 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:16.072+0000 7f1b68b79740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b4b3cbc8b0 0x55b4b3ccc670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:26:16.073 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:16.072+0000 7f1b68b79740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b4b3cbd260 0x55b4b3cccbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:26:16.073 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:16.072+0000 7f1b68b79740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b4b3cc5de0 0x55b4b3ccd0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:26:16.073 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:16.072+0000 7f1b68b79740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b601b60c0 con 0x55b4b3cbc8b0 2024-03-21T10:26:16.073 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:16.072+0000 7f1b68b79740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b601b5a60 con 0x55b4b3cbd260 2024-03-21T10:26:16.073 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:16.072+0000 7f1b68b79740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b601b2f60 con 0x55b4b3cc5de0 2024-03-21T10:26:16.073 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:16.072+0000 7f1b68b79740 10 client.0 _ll_register_callbacks cb 0x7f1b4c009560 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T10:26:16.073 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:16.073+0000 7f1b5e000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b4b3cbd260 0x55b4b3cccbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:26:16.073 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:16.073+0000 7f1b5ea00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b4b3cbc8b0 0x55b4b3ccc670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:26:16.073 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:16.073+0000 7f1b5e000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b4b3cbd260 0x55b4b3cccbb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:41408/0 (socket says 172.21.15.47:41408) 2024-03-21T10:26:16.074 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:16.073+0000 7f1b5e000640 1 -- 172.21.15.47:0/847535256 learned_addr learned my addr 172.21.15.47:0/847535256 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:26:16.075 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:16.073+0000 7f1b5f400640 1 --2- 172.21.15.47:0/847535256 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b4b3cc5de0 0x55b4b3ccd0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:26:16.075 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:16.073+0000 7f1b5ea00640 1 -- 172.21.15.47:0/847535256 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b4b3cc5de0 msgr2=0x55b4b3ccd0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:26:16.075 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:16.073+0000 7f1b5ea00640 1 --2- 172.21.15.47:0/847535256 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b4b3cc5de0 0x55b4b3ccd0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:26:16.075 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:16.073+0000 7f1b5ea00640 1 -- 172.21.15.47:0/847535256 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b4b3cbd260 msgr2=0x55b4b3cccbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:26:16.075 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:16.073+0000 7f1b5ea00640 1 --2- 172.21.15.47:0/847535256 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b4b3cbd260 0x55b4b3cccbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:26:16.075 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:16.073+0000 7f1b5ea00640 1 -- 172.21.15.47:0/847535256 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55b4b3d31690 con 0x55b4b3cbc8b0 2024-03-21T10:26:16.075 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:16.073+0000 7f1b5e000640 1 --2- 172.21.15.47:0/847535256 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b4b3cbd260 0x55b4b3cccbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T10:26:16.075 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:16.074+0000 7f1b5ea00640 1 --2- 172.21.15.47:0/847535256 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b4b3cbc8b0 0x55b4b3ccc670 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7f1b500cc9e0 tx=0x7f1b50086050 comp rx=0 tx=0).ready entity=mon.0 client_cookie=652cb6fd20920c28 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:26:16.075 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:16.074+0000 7f1b48c00640 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-03-21T10:26:16.075 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:16.074+0000 7f1b48c00640 1 -- 172.21.15.47:0/847535256 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b5009f700 con 0x55b4b3cbc8b0 2024-03-21T10:26:16.075 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:16.074+0000 7f1b48c00640 1 -- 172.21.15.47:0/847535256 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 9474+0+0 (secure 0 0 0) 0x7f1b500cf390 con 0x55b4b3cbc8b0 2024-03-21T10:26:16.075 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:16.074+0000 7f1b68b79740 1 -- 172.21.15.47:0/847535256 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f1b58003e20 con 0x55b4b3cbc8b0 2024-03-21T10:26:16.076 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:16.075+0000 7f1b48c00640 1 -- 172.21.15.47:0/847535256 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1b500a3fc0 con 0x55b4b3cbc8b0 2024-03-21T10:26:16.076 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:16.075+0000 7f1b68b79740 10 client.16065 fetch_fsmap learned FSMap version 30 2024-03-21T10:26:16.076 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:16.075+0000 7f1b68b79740 1 -- 172.21.15.47:0/847535256 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x55b4b3d329f0 con 0x55b4b3cbc8b0 2024-03-21T10:26:16.077 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:16.075+0000 7f1b48c00640 1 -- 172.21.15.47:0/847535256 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f1b58003e20 con 0x55b4b3cbc8b0 2024-03-21T10:26:16.077 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:16.075+0000 7f1b48c00640 1 -- 172.21.15.47:0/847535256 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 30) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f1b500cccc0 con 0x55b4b3cbc8b0 2024-03-21T10:26:16.077 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:16.075+0000 7f1b68b79740 10 client.16065 fetch_fsmap finished waiting for FSMap version 30 2024-03-21T10:26:16.077 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:16.075+0000 7f1b68b79740 10 client.16065 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-03-21T10:26:16.077 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:16.075+0000 7f1b68b79740 20 client.16065 populate_metadata read hostname 'smithi047' 2024-03-21T10:26:16.077 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:16.075+0000 7f1b68b79740 1 --2- 172.21.15.47:0/847535256 >> [v2:172.21.15.100:6836/3497644162,v1:172.21.15.100:6837/3497644162] conn(0x55b4b3d33120 0x55b4b3d35540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:26:16.077 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:16.075+0000 7f1b68b79740 4 client.16065 mds_command: new command op to 34493 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-21T10:26:16.077 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:16.075+0000 7f1b68b79740 1 -- 172.21.15.47:0/847535256 --> [v2:172.21.15.100:6836/3497644162,v1:172.21.15.100:6837/3497644162] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55b4b3c27b50 con 0x55b4b3d33120 2024-03-21T10:26:16.077 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:16.076+0000 7f1b5e000640 1 --2- 172.21.15.47:0/847535256 >> [v2:172.21.15.100:6836/3497644162,v1:172.21.15.100:6837/3497644162] conn(0x55b4b3d33120 0x55b4b3d35540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:26:16.077 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:16.077+0000 7f1b5e000640 1 --2- 172.21.15.47:0/847535256 >> [v2:172.21.15.100:6836/3497644162,v1:172.21.15.100:6837/3497644162] conn(0x55b4b3d33120 0x55b4b3d35540 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=5464d64cfed0b66c server_cookie=9ee5a31870d91d1d in_seq=0 out_seq=0 2024-03-21T10:26:16.077 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:16.077+0000 7f1b48c00640 10 client.16065 ms_handle_connect on v2:172.21.15.100:6836/3497644162 2024-03-21T10:26:16.078 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:16.077+0000 7f1b48c00640 1 -- 172.21.15.47:0/847535256 <== mds.1 v2:172.21.15.100:6836/3497644162 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55b4b3c27b50 con 0x55b4b3d33120 2024-03-21T10:26:16.079 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:16.077+0000 7f1b48c00640 10 client.16065 handle_command_reply: tid=0 2024-03-21T10:26:16.079 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:16.077+0000 7f1b68b79740 1 client.16065 shutdown 2024-03-21T10:26:16.079 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:16.077+0000 7f1b68b79740 1 -- 172.21.15.47:0/847535256 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b4b3cbc8b0 msgr2=0x55b4b3ccc670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:26:16.079 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:16.077+0000 7f1b68b79740 1 --2- 172.21.15.47:0/847535256 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b4b3cbc8b0 0x55b4b3ccc670 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7f1b500cc9e0 tx=0x7f1b50086050 comp rx=0 tx=0).stop 2024-03-21T10:26:16.079 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:16.078+0000 7f1b68b79740 1 -- 172.21.15.47:0/847535256 shutdown_connections 2024-03-21T10:26:16.079 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:16.078+0000 7f1b68b79740 1 --2- 172.21.15.47:0/847535256 >> [v2:172.21.15.100:6836/3497644162,v1:172.21.15.100:6837/3497644162] conn(0x55b4b3d33120 0x55b4b3d35540 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:26:16.079 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:16.078+0000 7f1b68b79740 1 --2- 172.21.15.47:0/847535256 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b4b3cc5de0 0x55b4b3ccd0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:26:16.079 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:16.078+0000 7f1b68b79740 1 --2- 172.21.15.47:0/847535256 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b4b3cbd260 0x55b4b3cccbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:26:16.079 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:16.078+0000 7f1b68b79740 1 --2- 172.21.15.47:0/847535256 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b4b3cbc8b0 0x55b4b3ccc670 unknown :-1 s=CLOSED pgs=603 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:26:16.079 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:16.078+0000 7f1b68b79740 1 -- 172.21.15.47:0/847535256 >> 172.21.15.47:0/847535256 conn(0x55b4b3cb6f80 msgr2=0x55b4b3cb58d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:26:16.079 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:16.078+0000 7f1b48c00640 0 client.16065 ms_handle_reset on v2:172.21.15.100:6836/3497644162 2024-03-21T10:26:16.079 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:16.078+0000 7f1b68b79740 1 -- 172.21.15.47:0/847535256 shutdown_connections 2024-03-21T10:26:16.079 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:16.078+0000 7f1b68b79740 1 -- 172.21.15.47:0/847535256 wait complete. 2024-03-21T10:26:16.080 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:16.078+0000 7f1b68b79740 20 client.16065 trim_cache size 0 max 16384 2024-03-21T10:26:16.080 INFO:teuthology.orchestra.run.smithi047.stdout:[] 2024-03-21T10:26:16.080 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:16.079+0000 7f1b5fe00640 1 -- 172.21.15.47:0/584706725 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f1b440a30c0 msgr2=0x7f1b440a5580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:26:16.081 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:16.079+0000 7f1b5fe00640 1 --2- 172.21.15.47:0/584706725 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f1b440a30c0 0x7f1b440a5580 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f1b58004ff0 tx=0x7f1b58000f40 comp rx=0 tx=0).stop 2024-03-21T10:26:16.081 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:16.079+0000 7f1b5fe00640 1 -- 172.21.15.47:0/584706725 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1b60103720 msgr2=0x7f1b60110550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:26:16.081 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:16.079+0000 7f1b5fe00640 1 --2- 172.21.15.47:0/584706725 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1b60103720 0x7f1b60110550 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f1b5000d950 tx=0x7f1b5000de10 comp rx=0 tx=0).stop 2024-03-21T10:26:16.081 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:16.079+0000 7f1b5fe00640 1 -- 172.21.15.47:0/584706725 shutdown_connections 2024-03-21T10:26:16.081 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:16.079+0000 7f1b5fe00640 1 --2- 172.21.15.47:0/584706725 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1b60109730 0x7f1b6010dc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:26:16.081 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:16.079+0000 7f1b5fe00640 1 --2- 172.21.15.47:0/584706725 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1b601040e0 0x7f1b6010d690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:26:16.081 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:16.079+0000 7f1b5fe00640 1 --2- 172.21.15.47:0/584706725 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f1b440a30c0 0x7f1b440a5580 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:26:16.081 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:16.079+0000 7f1b5fe00640 1 --2- 172.21.15.47:0/584706725 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1b60103720 0x7f1b60110550 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:26:16.081 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:16.079+0000 7f1b5fe00640 1 -- 172.21.15.47:0/584706725 >> 172.21.15.47:0/584706725 conn(0x7f1b600ff1b0 msgr2=0x7f1b600ff9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:26:16.081 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:16.079+0000 7f1b5fe00640 1 -- 172.21.15.47:0/584706725 shutdown_connections 2024-03-21T10:26:16.081 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:16.079+0000 7f1b5fe00640 1 -- 172.21.15.47:0/584706725 wait complete. 2024-03-21T10:26:16.097 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-21T10:26:16.098 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-03-21T10:26:16.129 INFO:journalctl@ceph.mds.i.smithi157.stdout:Mar 21 10:26:15 smithi157 systemd[1]: ceph-3056d51a-e76b-11ee-95cd-87774f69a715@mds.i.service: Scheduled restart job, restart counter is at 1. 2024-03-21T10:26:16.130 INFO:journalctl@ceph.mds.i.smithi157.stdout:Mar 21 10:26:15 smithi157 systemd[1]: Stopped Ceph mds.i for 3056d51a-e76b-11ee-95cd-87774f69a715. 2024-03-21T10:26:16.130 INFO:journalctl@ceph.mds.i.smithi157.stdout:Mar 21 10:26:15 smithi157 systemd[1]: ceph-3056d51a-e76b-11ee-95cd-87774f69a715@mds.i.service: Consumed 2.414s CPU time. 2024-03-21T10:26:16.130 INFO:journalctl@ceph.mds.i.smithi157.stdout:Mar 21 10:26:15 smithi157 systemd[1]: Starting Ceph mds.i for 3056d51a-e76b-11ee-95cd-87774f69a715... 2024-03-21T10:26:16.314 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:16.313+0000 7f64b0e00640 1 -- 172.21.15.47:0/726541298 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f64ac073a50 msgr2=0x7f64ac073e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:26:16.315 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:16.313+0000 7f64b0e00640 1 --2- 172.21.15.47:0/726541298 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f64ac073a50 0x7f64ac073e30 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7f649c009900 tx=0x7f649c02bcc0 comp rx=0 tx=0).stop 2024-03-21T10:26:16.315 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:16.314+0000 7f64b0e00640 1 -- 172.21.15.47:0/726541298 shutdown_connections 2024-03-21T10:26:16.315 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:16.314+0000 7f64b0e00640 1 --2- 172.21.15.47:0/726541298 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f64ac10ddd0 0x7f64ac1101c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:26:16.315 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:16.314+0000 7f64b0e00640 1 --2- 172.21.15.47:0/726541298 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f64ac074370 0x7f64ac10d890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:26:16.315 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:16.314+0000 7f64b0e00640 1 --2- 172.21.15.47:0/726541298 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f64ac073a50 0x7f64ac073e30 unknown :-1 s=CLOSED pgs=604 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:26:16.315 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:16.314+0000 7f64b0e00640 1 -- 172.21.15.47:0/726541298 >> 172.21.15.47:0/726541298 conn(0x7f64ac0fd040 msgr2=0x7f64ac0ff460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:26:16.315 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:16.314+0000 7f64b0e00640 1 -- 172.21.15.47:0/726541298 shutdown_connections 2024-03-21T10:26:16.315 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:16.314+0000 7f64b0e00640 1 -- 172.21.15.47:0/726541298 wait complete. 2024-03-21T10:26:16.316 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:16.315+0000 7f64b0e00640 1 Processor -- start 2024-03-21T10:26:16.316 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:16.315+0000 7f64b0e00640 1 -- start start 2024-03-21T10:26:16.316 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:16.316+0000 7f64b0e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f64ac073a50 0x7f64ac199f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:26:16.316 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:16.316+0000 7f64b0e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f64ac074370 0x7f64ac19a490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:26:16.316 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:16.316+0000 7f64b0e00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f64ac10ddd0 0x7f64ac19e810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:26:16.316 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:16.316+0000 7f64b0e00640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64ac1139f0 con 0x7f64ac073a50 2024-03-21T10:26:16.316 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:16.316+0000 7f64b0e00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64ac113870 con 0x7f64ac074370 2024-03-21T10:26:16.316 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:16.316+0000 7f64b0e00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64ac113b70 con 0x7f64ac10ddd0 2024-03-21T10:26:16.317 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:16.316+0000 7f64aaa00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f64ac074370 0x7f64ac19a490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:26:16.317 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:16.316+0000 7f64aaa00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f64ac074370 0x7f64ac19a490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:41426/0 (socket says 172.21.15.47:41426) 2024-03-21T10:26:16.317 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:16.316+0000 7f64aaa00640 1 -- 172.21.15.47:0/2211168829 learned_addr learned my addr 172.21.15.47:0/2211168829 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:26:16.317 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:16.316+0000 7f64abe00640 1 --2- 172.21.15.47:0/2211168829 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f64ac10ddd0 0x7f64ac19e810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:26:16.317 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:16.316+0000 7f64ab400640 1 --2- 172.21.15.47:0/2211168829 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f64ac073a50 0x7f64ac199f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:26:16.317 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:16.317+0000 7f64aaa00640 1 -- 172.21.15.47:0/2211168829 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f64ac10ddd0 msgr2=0x7f64ac19e810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:26:16.317 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:16.317+0000 7f64aaa00640 1 --2- 172.21.15.47:0/2211168829 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f64ac10ddd0 0x7f64ac19e810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:26:16.317 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:16.317+0000 7f64aaa00640 1 -- 172.21.15.47:0/2211168829 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f64ac073a50 msgr2=0x7f64ac199f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:26:16.318 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:16.317+0000 7f64aaa00640 1 --2- 172.21.15.47:0/2211168829 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f64ac073a50 0x7f64ac199f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:26:16.318 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:16.317+0000 7f64aaa00640 1 -- 172.21.15.47:0/2211168829 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64ac19ef40 con 0x7f64ac074370 2024-03-21T10:26:16.318 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:16.317+0000 7f64aaa00640 1 --2- 172.21.15.47:0/2211168829 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f64ac074370 0x7f64ac19a490 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f649800d770 tx=0x7f649800dc30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:26:16.318 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:16.317+0000 7f64a3e00640 1 -- 172.21.15.47:0/2211168829 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6498017070 con 0x7f64ac074370 2024-03-21T10:26:16.318 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:16.318+0000 7f64b0e00640 1 -- 172.21.15.47:0/2211168829 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f64ac19f230 con 0x7f64ac074370 2024-03-21T10:26:16.319 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:16.318+0000 7f64a3e00640 1 -- 172.21.15.47:0/2211168829 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f64980054d0 con 0x7f64ac074370 2024-03-21T10:26:16.319 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:16.318+0000 7f64a3e00640 1 -- 172.21.15.47:0/2211168829 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6498004cf0 con 0x7f64ac074370 2024-03-21T10:26:16.319 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:16.318+0000 7f64b0e00640 1 -- 172.21.15.47:0/2211168829 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f64ac1a6ac0 con 0x7f64ac074370 2024-03-21T10:26:16.321 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:16.321+0000 7f64b502a740 1 Processor -- start 2024-03-21T10:26:16.321 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:16.321+0000 7f64b502a740 1 -- start start 2024-03-21T10:26:16.322 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:16.322+0000 7f64b502a740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5623e66f78b0 0x5623e66f7c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:26:16.322 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:16.322+0000 7f64b502a740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5623e66f8260 0x5623e6700810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:26:16.322 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:16.322+0000 7f64b502a740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5623e6700de0 0x5623e6703230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:26:16.322 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:16.322+0000 7f64b502a740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x5623e665b2c0 con 0x5623e6700de0 2024-03-21T10:26:16.322 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:16.322+0000 7f64b502a740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x5623e6603c90 con 0x5623e66f78b0 2024-03-21T10:26:16.322 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:16.322+0000 7f64b502a740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x5623e6657360 con 0x5623e66f8260 2024-03-21T10:26:16.323 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:16.322+0000 7f64abe00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5623e66f78b0 0x5623e66f7c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:26:16.323 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:16.322+0000 7f64abe00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5623e66f78b0 0x5623e66f7c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:41434/0 (socket says 172.21.15.47:41434) 2024-03-21T10:26:16.323 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:16.322+0000 7f64abe00640 1 -- 172.21.15.47:0/2695581537 learned_addr learned my addr 172.21.15.47:0/2695581537 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:26:16.323 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:16.322+0000 7f64ab400640 1 --2- 172.21.15.47:0/2695581537 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5623e66f8260 0x5623e6700810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:26:16.323 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:16.323+0000 7f64abe00640 1 -- 172.21.15.47:0/2695581537 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5623e66f8260 msgr2=0x5623e6700810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:26:16.323 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:16.323+0000 7f64abe00640 1 --2- 172.21.15.47:0/2695581537 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5623e66f8260 0x5623e6700810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:26:16.323 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:16.323+0000 7f64abe00640 1 -- 172.21.15.47:0/2695581537 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5623e6700de0 msgr2=0x5623e6703230 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-21T10:26:16.324 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:16.323+0000 7f64abe00640 1 --2- 172.21.15.47:0/2695581537 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5623e6700de0 0x5623e6703230 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:26:16.324 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:16.323+0000 7f64abe00640 1 -- 172.21.15.47:0/2695581537 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5623e6628080 con 0x5623e66f78b0 2024-03-21T10:26:16.325 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:16.323+0000 7f64a3e00640 1 -- 172.21.15.47:0/2211168829 <== mon.2 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 26) v1 ==== 138087+0+0 (secure 0 0 0) 0x7f6498004e90 con 0x7f64ac074370 2024-03-21T10:26:16.325 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:16.323+0000 7f64abe00640 1 --2- 172.21.15.47:0/2695581537 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5623e66f78b0 0x5623e66f7c90 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f64a4009f50 tx=0x7f64a40118d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=216eb7a2d1f9e210 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:26:16.325 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:16.323+0000 7f64a0c00640 1 -- 172.21.15.47:0/2695581537 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64a4004b20 con 0x5623e66f78b0 2024-03-21T10:26:16.325 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:16.323+0000 7f64a0c00640 1 -- 172.21.15.47:0/2695581537 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f64a4004cc0 con 0x5623e66f78b0 2024-03-21T10:26:16.325 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:16.324+0000 7f64a0c00640 1 -- 172.21.15.47:0/2695581537 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64a4005020 con 0x5623e66f78b0 2024-03-21T10:26:16.325 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:16.324+0000 7f64b502a740 1 -- 172.21.15.47:0/2695581537 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5623e66f78b0 msgr2=0x5623e66f7c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:26:16.325 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:16.324+0000 7f64b502a740 1 --2- 172.21.15.47:0/2695581537 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5623e66f78b0 0x5623e66f7c90 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f64a4009f50 tx=0x7f64a40118d0 comp rx=0 tx=0).stop 2024-03-21T10:26:16.326 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:16.324+0000 7f64a3e00640 1 --2- 172.21.15.47:0/2211168829 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f64900a3230 0x7f64900a56f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:26:16.326 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:16.325+0000 7f64a3e00640 1 -- 172.21.15.47:0/2211168829 <== mon.2 v2:172.21.15.100:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 9474+0+0 (secure 0 0 0) 0x7f64980cf870 con 0x7f64ac074370 2024-03-21T10:26:16.326 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:16.325+0000 7f64b502a740 1 -- 172.21.15.47:0/2695581537 shutdown_connections 2024-03-21T10:26:16.326 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:16.325+0000 7f64b502a740 1 --2- 172.21.15.47:0/2695581537 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5623e6700de0 0x5623e6703230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:26:16.326 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:16.325+0000 7f64b502a740 1 --2- 172.21.15.47:0/2695581537 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5623e66f8260 0x5623e6700810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:26:16.326 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:16.325+0000 7f64b502a740 1 --2- 172.21.15.47:0/2695581537 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5623e66f78b0 0x5623e66f7c90 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:26:16.326 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:16.325+0000 7f64b502a740 1 -- 172.21.15.47:0/2695581537 >> 172.21.15.47:0/2695581537 conn(0x5623e66f1f80 msgr2=0x5623e66f2360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:26:16.326 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:16.325+0000 7f64abe00640 1 --2- 172.21.15.47:0/2211168829 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f64900a3230 0x7f64900a56f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:26:16.327 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:16.325+0000 7f64b502a740 1 -- 172.21.15.47:0/2695581537 shutdown_connections 2024-03-21T10:26:16.327 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:16.325+0000 7f64b502a740 1 -- 172.21.15.47:0/2695581537 wait complete. 2024-03-21T10:26:16.327 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:16.326+0000 7f64b502a740 1 -- start start 2024-03-21T10:26:16.327 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:16.326+0000 7f64abe00640 1 --2- 172.21.15.47:0/2211168829 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f64900a3230 0x7f64900a56f0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f64a4004f90 tx=0x7f64a40044a0 comp rx=0 tx=0).ready entity=mgr.15687 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:26:16.327 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:16.326+0000 7f64b502a740 1 Processor -- start 2024-03-21T10:26:16.327 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:16.326+0000 7f64b502a740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5623e66f78b0 0x5623e6700720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:26:16.327 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:16.326+0000 7f64b502a740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5623e66f8260 0x5623e66fdab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:26:16.328 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:16.326+0000 7f64b502a740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5623e6700de0 0x5623e66fe0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:26:16.328 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:16.326+0000 7f64b502a740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x5623e6703d50 con 0x5623e66f8260 2024-03-21T10:26:16.328 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:16.326+0000 7f64b502a740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x5623e66170d0 con 0x5623e6700de0 2024-03-21T10:26:16.328 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:16.326+0000 7f64b502a740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x5623e6703ed0 con 0x5623e66f78b0 2024-03-21T10:26:16.328 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:16.327+0000 7f64b502a740 10 client.0 _ll_register_callbacks cb 0x5623e661be80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T10:26:16.328 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:16.327+0000 7f64aaa00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5623e66f8260 0x5623e66fdab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:26:16.328 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:16.327+0000 7f64abe00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5623e6700de0 0x5623e66fe0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:26:16.328 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:16.327+0000 7f64aaa00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5623e66f8260 0x5623e66fdab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:58680/0 (socket says 172.21.15.47:58680) 2024-03-21T10:26:16.328 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:16.327+0000 7f64aaa00640 1 -- 172.21.15.47:0/406719166 learned_addr learned my addr 172.21.15.47:0/406719166 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:26:16.330 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:16.327+0000 7f64ab400640 1 --2- 172.21.15.47:0/406719166 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5623e66f78b0 0x5623e6700720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:26:16.330 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:16.327+0000 7f64aaa00640 1 -- 172.21.15.47:0/406719166 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5623e66f78b0 msgr2=0x5623e6700720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:26:16.330 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:16.327+0000 7f64aaa00640 1 --2- 172.21.15.47:0/406719166 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5623e66f78b0 0x5623e6700720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:26:16.330 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:16.327+0000 7f64aaa00640 1 -- 172.21.15.47:0/406719166 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5623e6700de0 msgr2=0x5623e66fe0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:26:16.330 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:16.327+0000 7f64aaa00640 1 --2- 172.21.15.47:0/406719166 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5623e6700de0 0x5623e66fe0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:26:16.330 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:16.328+0000 7f64aaa00640 1 -- 172.21.15.47:0/406719166 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5623e636f3f0 con 0x5623e66f8260 2024-03-21T10:26:16.330 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:16.328+0000 7f64aaa00640 1 --2- 172.21.15.47:0/406719166 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5623e66f8260 0x5623e66fdab0 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7f6498009610 tx=0x7f649809b1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d83b74526db945a3 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:26:16.330 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:16.328+0000 7f6495600640 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-03-21T10:26:16.330 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:16.328+0000 7f6495600640 1 -- 172.21.15.47:0/406719166 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6498020070 con 0x5623e66f8260 2024-03-21T10:26:16.330 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:16.328+0000 7f6495600640 1 -- 172.21.15.47:0/406719166 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 9474+0+0 (secure 0 0 0) 0x7f64980cf5f0 con 0x5623e66f8260 2024-03-21T10:26:16.330 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:16.328+0000 7f64b502a740 1 -- 172.21.15.47:0/406719166 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f64a4004cc0 con 0x5623e66f8260 2024-03-21T10:26:16.331 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:16.329+0000 7f6495600640 1 -- 172.21.15.47:0/406719166 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6498005180 con 0x5623e66f8260 2024-03-21T10:26:16.331 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:16.329+0000 7f64b502a740 10 client.16074 fetch_fsmap learned FSMap version 30 2024-03-21T10:26:16.331 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:16.329+0000 7f64b502a740 1 -- 172.21.15.47:0/406719166 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x5623e64397d0 con 0x5623e66f8260 2024-03-21T10:26:16.331 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:16.329+0000 7f6495600640 1 -- 172.21.15.47:0/406719166 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f64a4004cc0 con 0x5623e66f8260 2024-03-21T10:26:16.331 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:16.329+0000 7f6495600640 1 -- 172.21.15.47:0/406719166 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 30) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f64980ce030 con 0x5623e66f8260 2024-03-21T10:26:16.331 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:16.329+0000 7f64b502a740 10 client.16074 fetch_fsmap finished waiting for FSMap version 30 2024-03-21T10:26:16.331 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:16.329+0000 7f64b502a740 10 client.16074 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-21T10:26:16.331 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:16.330+0000 7f64b502a740 20 client.16074 populate_metadata read hostname 'smithi047' 2024-03-21T10:26:16.331 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:16.330+0000 7f64b502a740 1 --2- 172.21.15.47:0/406719166 >> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] conn(0x5623e6710c80 0x5623e6713070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:26:16.331 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:16.330+0000 7f64b502a740 4 client.16074 mds_command: new command op to 34535 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T10:26:16.331 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:16.330+0000 7f64b502a740 1 -- 172.21.15.47:0/406719166 --> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5623e6668800 con 0x5623e6710c80 2024-03-21T10:26:16.332 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:16.330+0000 7f64ab400640 1 --2- 172.21.15.47:0/406719166 >> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] conn(0x5623e6710c80 0x5623e6713070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:26:16.332 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:16.331+0000 7f64ab400640 1 --2- 172.21.15.47:0/406719166 >> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] conn(0x5623e6710c80 0x5623e6713070 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3949beffcc0ef410 server_cookie=7d29ae89025e573b in_seq=0 out_seq=0 2024-03-21T10:26:16.332 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:16.331+0000 7f6495600640 10 client.16074 ms_handle_connect on v2:172.21.15.100:6838/4173168440 2024-03-21T10:26:16.333 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:16.332+0000 7f6495600640 1 -- 172.21.15.47:0/406719166 <== mds.0 v2:172.21.15.100:6838/4173168440 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x5623e6668800 con 0x5623e6710c80 2024-03-21T10:26:16.333 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:16.332+0000 7f6495600640 10 client.16074 handle_command_reply: tid=0 2024-03-21T10:26:16.333 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:16.332+0000 7f64b502a740 1 client.16074 shutdown 2024-03-21T10:26:16.333 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:16.332+0000 7f64b502a740 1 -- 172.21.15.47:0/406719166 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5623e66f8260 msgr2=0x5623e66fdab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:26:16.333 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:16.332+0000 7f64b502a740 1 --2- 172.21.15.47:0/406719166 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5623e66f8260 0x5623e66fdab0 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7f6498009610 tx=0x7f649809b1e0 comp rx=0 tx=0).stop 2024-03-21T10:26:16.333 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:16.333+0000 7f64b502a740 1 -- 172.21.15.47:0/406719166 shutdown_connections 2024-03-21T10:26:16.334 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:16.333+0000 7f64b502a740 1 --2- 172.21.15.47:0/406719166 >> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] conn(0x5623e6710c80 0x5623e6713070 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:26:16.334 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:16.333+0000 7f64b502a740 1 --2- 172.21.15.47:0/406719166 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5623e6700de0 0x5623e66fe0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:26:16.334 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:16.333+0000 7f64b502a740 1 --2- 172.21.15.47:0/406719166 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5623e66f8260 0x5623e66fdab0 unknown :-1 s=CLOSED pgs=605 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:26:16.334 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:16.333+0000 7f64b502a740 1 --2- 172.21.15.47:0/406719166 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5623e66f78b0 0x5623e6700720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:26:16.334 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:16.333+0000 7f64b502a740 1 -- 172.21.15.47:0/406719166 >> 172.21.15.47:0/406719166 conn(0x5623e66f1f80 msgr2=0x5623e66f3cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:26:16.334 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:16.333+0000 7f6495600640 0 client.16074 ms_handle_reset on v2:172.21.15.100:6838/4173168440 2024-03-21T10:26:16.334 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:16.333+0000 7f64b502a740 1 -- 172.21.15.47:0/406719166 shutdown_connections 2024-03-21T10:26:16.334 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:16.333+0000 7f64b502a740 1 -- 172.21.15.47:0/406719166 wait complete. 2024-03-21T10:26:16.334 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:16.333+0000 7f64b502a740 20 client.16074 trim_cache size 0 max 16384 2024-03-21T10:26:16.346 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:16.346+0000 7f64b502a740 1 Processor -- start 2024-03-21T10:26:16.347 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:16.346+0000 7f64b502a740 1 -- start start 2024-03-21T10:26:16.347 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:16.347+0000 7f64b502a740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5623e66f78b0 0x5623e66f0fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:26:16.347 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:16.347+0000 7f64b502a740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5623e66f8260 0x5623e67001e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:26:16.347 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:16.347+0000 7f64b502a740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5623e6700de0 0x5623e6700720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:26:16.347 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:16.347+0000 7f64b502a740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64ac1b4cb0 con 0x5623e6700de0 2024-03-21T10:26:16.347 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:16.347+0000 7f64b502a740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64ac1b6630 con 0x5623e66f78b0 2024-03-21T10:26:16.347 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:16.347+0000 7f64b502a740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64ac1b5310 con 0x5623e66f8260 2024-03-21T10:26:16.348 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:16.347+0000 7f64ab400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5623e66f78b0 0x5623e66f0fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:26:16.348 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:16.347+0000 7f64ab400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5623e66f78b0 0x5623e66f0fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:41456/0 (socket says 172.21.15.47:41456) 2024-03-21T10:26:16.348 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:16.347+0000 7f64ab400640 1 -- 172.21.15.47:0/4202871514 learned_addr learned my addr 172.21.15.47:0/4202871514 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:26:16.348 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:16.347+0000 7f64abe00640 1 --2- 172.21.15.47:0/4202871514 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5623e6700de0 0x5623e6700720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:26:16.349 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:16.348+0000 7f64aaa00640 1 --2- 172.21.15.47:0/4202871514 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5623e66f8260 0x5623e67001e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:26:16.349 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:16.348+0000 7f64ab400640 1 -- 172.21.15.47:0/4202871514 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5623e66f8260 msgr2=0x5623e67001e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:26:16.349 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:16.348+0000 7f64ab400640 1 --2- 172.21.15.47:0/4202871514 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5623e66f8260 0x5623e67001e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:26:16.349 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:16.348+0000 7f64ab400640 1 -- 172.21.15.47:0/4202871514 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5623e6700de0 msgr2=0x5623e6700720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:26:16.349 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:16.348+0000 7f64ab400640 1 --2- 172.21.15.47:0/4202871514 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5623e6700de0 0x5623e6700720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:26:16.349 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:16.348+0000 7f64ab400640 1 -- 172.21.15.47:0/4202871514 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5623e66fe430 con 0x5623e66f78b0 2024-03-21T10:26:16.349 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:16.348+0000 7f64ab400640 1 --2- 172.21.15.47:0/4202871514 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5623e66f78b0 0x5623e66f0fe0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f649c006010 tx=0x7f649c009260 comp rx=0 tx=0).ready entity=mon.2 client_cookie=30d545eec070397a server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:26:16.350 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:16.348+0000 7f6497e00640 1 -- 172.21.15.47:0/4202871514 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f649c0041e0 con 0x5623e66f78b0 2024-03-21T10:26:16.350 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:16.349+0000 7f6497e00640 1 -- 172.21.15.47:0/4202871514 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f649c02cd20 con 0x5623e66f78b0 2024-03-21T10:26:16.350 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:16.349+0000 7f6497e00640 1 -- 172.21.15.47:0/4202871514 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f649c0046c0 con 0x5623e66f78b0 2024-03-21T10:26:16.350 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:16.349+0000 7f64b502a740 1 -- 172.21.15.47:0/4202871514 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5623e66f78b0 msgr2=0x5623e66f0fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:26:16.351 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:16.349+0000 7f64b502a740 1 --2- 172.21.15.47:0/4202871514 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5623e66f78b0 0x5623e66f0fe0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f649c006010 tx=0x7f649c009260 comp rx=0 tx=0).stop 2024-03-21T10:26:16.351 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:16.350+0000 7f64b502a740 1 -- 172.21.15.47:0/4202871514 shutdown_connections 2024-03-21T10:26:16.351 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:16.350+0000 7f64b502a740 1 --2- 172.21.15.47:0/4202871514 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5623e6700de0 0x5623e6700720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:26:16.351 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:16.350+0000 7f64b502a740 1 --2- 172.21.15.47:0/4202871514 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5623e66f8260 0x5623e67001e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:26:16.351 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:16.350+0000 7f64b502a740 1 --2- 172.21.15.47:0/4202871514 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5623e66f78b0 0x5623e66f0fe0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:26:16.351 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:16.350+0000 7f64b502a740 1 -- 172.21.15.47:0/4202871514 >> 172.21.15.47:0/4202871514 conn(0x5623e66f1f80 msgr2=0x5623e66f08d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:26:16.351 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:16.350+0000 7f64b502a740 1 -- 172.21.15.47:0/4202871514 shutdown_connections 2024-03-21T10:26:16.351 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:16.350+0000 7f64b502a740 1 -- 172.21.15.47:0/4202871514 wait complete. 2024-03-21T10:26:16.352 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:16.350+0000 7f64b502a740 1 -- start start 2024-03-21T10:26:16.352 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:16.351+0000 7f64b502a740 1 Processor -- start 2024-03-21T10:26:16.352 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:16.351+0000 7f64b502a740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5623e66f78b0 0x5623e6708fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:26:16.352 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:16.351+0000 7f64b502a740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5623e66f8260 0x5623e6706360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:26:16.352 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:16.351+0000 7f64b502a740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5623e6700de0 0x5623e6706970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:26:16.352 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:16.351+0000 7f64b502a740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64ac1b5fd0 con 0x5623e66f8260 2024-03-21T10:26:16.353 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:16.351+0000 7f64b502a740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64ac1b5970 con 0x5623e66f78b0 2024-03-21T10:26:16.353 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:16.351+0000 7f64b502a740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64ac1b2e90 con 0x5623e6700de0 2024-03-21T10:26:16.353 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:16.351+0000 7f64b502a740 10 client.0 _ll_register_callbacks cb 0x7f6498009f50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T10:26:16.353 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:16.352+0000 7f64ab400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5623e66f78b0 0x5623e6708fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:26:16.353 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:16.352+0000 7f64aaa00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5623e66f8260 0x5623e6706360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:26:16.353 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:16.352+0000 7f64aaa00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5623e66f8260 0x5623e6706360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:58692/0 (socket says 172.21.15.47:58692) 2024-03-21T10:26:16.353 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:16.352+0000 7f64aaa00640 1 -- 172.21.15.47:0/2605930902 learned_addr learned my addr 172.21.15.47:0/2605930902 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:26:16.353 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:16.352+0000 7f64abe00640 1 --2- 172.21.15.47:0/2605930902 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5623e6700de0 0x5623e6706970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:26:16.354 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:16.352+0000 7f64ab400640 1 -- 172.21.15.47:0/2605930902 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5623e6700de0 msgr2=0x5623e6706970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:26:16.354 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:16.352+0000 7f64ab400640 1 --2- 172.21.15.47:0/2605930902 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5623e6700de0 0x5623e6706970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:26:16.354 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:16.352+0000 7f64ab400640 1 -- 172.21.15.47:0/2605930902 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5623e66f8260 msgr2=0x5623e6706360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:26:16.354 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:16.352+0000 7f64ab400640 1 --2- 172.21.15.47:0/2605930902 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5623e66f8260 0x5623e6706360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:26:16.355 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:16.352+0000 7f64ab400640 1 -- 172.21.15.47:0/2605930902 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5623e67071a0 con 0x5623e66f78b0 2024-03-21T10:26:16.355 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:16.353+0000 7f64abe00640 1 --2- 172.21.15.47:0/2605930902 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5623e6700de0 0x5623e6706970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T10:26:16.355 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:16.353+0000 7f64aaa00640 1 --2- 172.21.15.47:0/2605930902 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5623e66f8260 0x5623e6706360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T10:26:16.355 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:16.353+0000 7f64ab400640 1 --2- 172.21.15.47:0/2605930902 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5623e66f78b0 0x5623e6708fd0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f649c02c200 tx=0x7f649c0050c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3e7807c17aadd80 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:26:16.355 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:16.353+0000 7f6495600640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-21T10:26:16.355 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:16.353+0000 7f6495600640 1 -- 172.21.15.47:0/2605930902 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f649c04f070 con 0x5623e66f78b0 2024-03-21T10:26:16.355 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:16.353+0000 7f64b502a740 1 -- 172.21.15.47:0/2605930902 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f649c02cd20 con 0x5623e66f78b0 2024-03-21T10:26:16.355 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:16.353+0000 7f6495600640 1 -- 172.21.15.47:0/2605930902 <== mon.2 v2:172.21.15.100:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 9474+0+0 (secure 0 0 0) 0x7f649c0043e0 con 0x5623e66f78b0 2024-03-21T10:26:16.355 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:16.354+0000 7f6495600640 1 -- 172.21.15.47:0/2605930902 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f649c02c870 con 0x5623e66f78b0 2024-03-21T10:26:16.356 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:16.354+0000 7f64b502a740 10 client.35888 fetch_fsmap learned FSMap version 30 2024-03-21T10:26:16.356 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:16.354+0000 7f64b502a740 1 -- 172.21.15.47:0/2605930902 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x5623e676c550 con 0x5623e66f78b0 2024-03-21T10:26:16.357 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:16.354+0000 7f6495600640 1 -- 172.21.15.47:0/2605930902 <== mon.2 v2:172.21.15.100:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f649c02cd20 con 0x5623e66f78b0 2024-03-21T10:26:16.358 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:16.355+0000 7f6495600640 1 -- 172.21.15.47:0/2605930902 <== mon.2 v2:172.21.15.100:3300/0 5 ==== fsmap(e 30) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f649c047070 con 0x5623e66f78b0 2024-03-21T10:26:16.358 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:16.355+0000 7f64b502a740 10 client.35888 fetch_fsmap finished waiting for FSMap version 30 2024-03-21T10:26:16.358 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:16.355+0000 7f64b502a740 10 client.35888 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-21T10:26:16.358 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:16.355+0000 7f64b502a740 20 client.35888 populate_metadata read hostname 'smithi047' 2024-03-21T10:26:16.358 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:16.355+0000 7f64b502a740 1 --2- 172.21.15.47:0/2605930902 >> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] conn(0x5623e676cc20 0x5623e676f040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:26:16.358 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:16.355+0000 7f64b502a740 4 client.35888 mds_command: new command op to 34535 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-21T10:26:16.358 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:16.355+0000 7f64b502a740 1 -- 172.21.15.47:0/2605930902 --> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x5623e6662b50 con 0x5623e676cc20 2024-03-21T10:26:16.358 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:16.355+0000 7f64aaa00640 1 --2- 172.21.15.47:0/2605930902 >> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] conn(0x5623e676cc20 0x5623e676f040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:26:16.358 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:16.356+0000 7f64aaa00640 1 --2- 172.21.15.47:0/2605930902 >> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] conn(0x5623e676cc20 0x5623e676f040 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=aeafd2f677ccb15 server_cookie=f3df83fe5f2b2f5b in_seq=0 out_seq=0 2024-03-21T10:26:16.358 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:16.356+0000 7f6495600640 10 client.35888 ms_handle_connect on v2:172.21.15.100:6838/4173168440 2024-03-21T10:26:16.358 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:16.356+0000 7f6495600640 1 -- 172.21.15.47:0/2605930902 <== mds.0 v2:172.21.15.100:6838/4173168440 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x5623e6662b50 con 0x5623e676cc20 2024-03-21T10:26:16.358 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:16.356+0000 7f6495600640 10 client.35888 handle_command_reply: tid=0 2024-03-21T10:26:16.358 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:16.356+0000 7f64b502a740 1 client.35888 shutdown 2024-03-21T10:26:16.358 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:16.357+0000 7f64b502a740 1 -- 172.21.15.47:0/2605930902 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5623e66f78b0 msgr2=0x5623e6708fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:26:16.359 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:16.357+0000 7f64b502a740 1 --2- 172.21.15.47:0/2605930902 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5623e66f78b0 0x5623e6708fd0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f649c02c200 tx=0x7f649c0050c0 comp rx=0 tx=0).stop 2024-03-21T10:26:16.359 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:16.357+0000 7f64b502a740 1 -- 172.21.15.47:0/2605930902 shutdown_connections 2024-03-21T10:26:16.359 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:16.357+0000 7f64b502a740 1 --2- 172.21.15.47:0/2605930902 >> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] conn(0x5623e676cc20 0x5623e676f040 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:26:16.359 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:16.357+0000 7f64b502a740 1 --2- 172.21.15.47:0/2605930902 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5623e6700de0 0x5623e6706970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:26:16.359 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:16.357+0000 7f64b502a740 1 --2- 172.21.15.47:0/2605930902 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5623e66f8260 0x5623e6706360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:26:16.359 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:16.357+0000 7f64b502a740 1 --2- 172.21.15.47:0/2605930902 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5623e66f78b0 0x5623e6708fd0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:26:16.359 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:16.357+0000 7f64b502a740 1 -- 172.21.15.47:0/2605930902 >> 172.21.15.47:0/2605930902 conn(0x5623e66f1f80 msgr2=0x5623e66f1160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:26:16.359 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:16.357+0000 7f6495600640 0 client.35888 ms_handle_reset on v2:172.21.15.100:6838/4173168440 2024-03-21T10:26:16.359 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:16.357+0000 7f64b502a740 1 -- 172.21.15.47:0/2605930902 shutdown_connections 2024-03-21T10:26:16.359 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:16.357+0000 7f64b502a740 1 -- 172.21.15.47:0/2605930902 wait complete. 2024-03-21T10:26:16.359 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:16.357+0000 7f64b502a740 20 client.35888 trim_cache size 0 max 16384 2024-03-21T10:26:16.360 INFO:teuthology.orchestra.run.smithi047.stdout:[] 2024-03-21T10:26:16.360 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:16.358+0000 7f64b0e00640 1 -- 172.21.15.47:0/2211168829 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f64900a3230 msgr2=0x7f64900a56f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:26:16.360 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:16.359+0000 7f64b0e00640 1 --2- 172.21.15.47:0/2211168829 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f64900a3230 0x7f64900a56f0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f64a4004f90 tx=0x7f64a40044a0 comp rx=0 tx=0).stop 2024-03-21T10:26:16.360 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:16.359+0000 7f64b0e00640 1 -- 172.21.15.47:0/2211168829 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f64ac074370 msgr2=0x7f64ac19a490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:26:16.361 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:16.359+0000 7f64b0e00640 1 --2- 172.21.15.47:0/2211168829 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f64ac074370 0x7f64ac19a490 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f649800d770 tx=0x7f649800dc30 comp rx=0 tx=0).stop 2024-03-21T10:26:16.361 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:16.359+0000 7f64b0e00640 1 -- 172.21.15.47:0/2211168829 shutdown_connections 2024-03-21T10:26:16.361 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:16.359+0000 7f64b0e00640 1 --2- 172.21.15.47:0/2211168829 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f64ac10ddd0 0x7f64ac19e810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:26:16.361 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:16.359+0000 7f64b0e00640 1 --2- 172.21.15.47:0/2211168829 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f64900a3230 0x7f64900a56f0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:26:16.361 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:16.359+0000 7f64b0e00640 1 --2- 172.21.15.47:0/2211168829 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f64ac074370 0x7f64ac19a490 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:26:16.361 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:16.359+0000 7f64b0e00640 1 --2- 172.21.15.47:0/2211168829 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f64ac073a50 0x7f64ac199f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:26:16.361 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:16.359+0000 7f64b0e00640 1 -- 172.21.15.47:0/2211168829 >> 172.21.15.47:0/2211168829 conn(0x7f64ac0fd040 msgr2=0x7f64ac0fddb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:26:16.361 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:16.359+0000 7f64b0e00640 1 -- 172.21.15.47:0/2211168829 shutdown_connections 2024-03-21T10:26:16.361 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:16.359+0000 7f64b0e00640 1 -- 172.21.15.47:0/2211168829 wait complete. 2024-03-21T10:26:16.423 INFO:journalctl@ceph.mds.i.smithi157.stdout:Mar 21 10:26:16 smithi157 podman[70239]: 2024-03-21 10:26:16.033845792 +0000 UTC m=+0.032524382 image pull 237e43c7a65ff48534d94b97c1bfdbe4b7be5672e3d9a613745083ea3702aa54 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232 2024-03-21T10:26:16.423 INFO:journalctl@ceph.mds.i.smithi157.stdout:Mar 21 10:26:16 smithi157 podman[70239]: 2024-03-21 10:26:16.141380425 +0000 UTC m=+0.140059048 container create ab79b2d30930c056f856b8a2c0014be41d4218687dffd31341cb4ba04a345c1b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-3056d51a-e76b-11ee-95cd-87774f69a715-mds-i, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.build-date=20240319, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, ceph=True, CEPH_POINT_RELEASE=) 2024-03-21T10:26:16.740 INFO:journalctl@ceph.mds.i.smithi157.stdout:Mar 21 10:26:16 smithi157 podman[70239]: 2024-03-21 10:26:16.422451444 +0000 UTC m=+0.421130113 container init ab79b2d30930c056f856b8a2c0014be41d4218687dffd31341cb4ba04a345c1b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-3056d51a-e76b-11ee-95cd-87774f69a715-mds-i, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.build-date=20240319, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, org.label-schema.license=GPLv2, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD) 2024-03-21T10:26:16.740 INFO:journalctl@ceph.mds.i.smithi157.stdout:Mar 21 10:26:16 smithi157 podman[70239]: 2024-03-21 10:26:16.43247046 +0000 UTC m=+0.431149090 container start ab79b2d30930c056f856b8a2c0014be41d4218687dffd31341cb4ba04a345c1b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-3056d51a-e76b-11ee-95cd-87774f69a715-mds-i, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240319, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, ceph=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD) 2024-03-21T10:26:16.740 INFO:journalctl@ceph.mds.i.smithi157.stdout:Mar 21 10:26:16 smithi157 ceph-mds[70258]: -- 172.21.15.157:0/3475339658 <== mon.1 v2:172.21.15.157:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55b02830ba40 con 0x55b0282de800 2024-03-21T10:26:16.740 INFO:journalctl@ceph.mds.i.smithi157.stdout:Mar 21 10:26:16 smithi157 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-mds-i[70254]: starting mds.i at 2024-03-21T10:26:16.740 INFO:journalctl@ceph.mds.i.smithi157.stdout:Mar 21 10:26:16 smithi157 bash[70239]: ab79b2d30930c056f856b8a2c0014be41d4218687dffd31341cb4ba04a345c1b 2024-03-21T10:26:16.740 INFO:journalctl@ceph.mds.i.smithi157.stdout:Mar 21 10:26:16 smithi157 systemd[1]: Started Ceph mds.i for 3056d51a-e76b-11ee-95cd-87774f69a715. 2024-03-21T10:26:16.758 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:16.757+0000 7f5f0cc00640 1 -- 172.21.15.47:0/2527109820 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5f08074370 msgr2=0x7f5f0810d820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:26:16.758 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:16.757+0000 7f5f0cc00640 1 --2- 172.21.15.47:0/2527109820 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5f08074370 0x7f5f0810d820 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7f5ef4009860 tx=0x7f5ef402bc20 comp rx=0 tx=0).stop 2024-03-21T10:26:16.758 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:16.757+0000 7f5f0cc00640 1 -- 172.21.15.47:0/2527109820 shutdown_connections 2024-03-21T10:26:16.759 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:16.757+0000 7f5f0cc00640 1 --2- 172.21.15.47:0/2527109820 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5f0810dd60 0x7f5f08110150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:26:16.759 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:16.757+0000 7f5f0cc00640 1 --2- 172.21.15.47:0/2527109820 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5f08074370 0x7f5f0810d820 unknown :-1 s=CLOSED pgs=606 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:26:16.759 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:16.757+0000 7f5f0cc00640 1 --2- 172.21.15.47:0/2527109820 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5f08073a50 0x7f5f08073e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:26:16.759 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:16.757+0000 7f5f0cc00640 1 -- 172.21.15.47:0/2527109820 >> 172.21.15.47:0/2527109820 conn(0x7f5f080fcfb0 msgr2=0x7f5f080ff3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:26:16.759 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:16.757+0000 7f5f0cc00640 1 -- 172.21.15.47:0/2527109820 shutdown_connections 2024-03-21T10:26:16.759 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:16.757+0000 7f5f0cc00640 1 -- 172.21.15.47:0/2527109820 wait complete. 2024-03-21T10:26:16.760 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:16.759+0000 7f5f0cc00640 1 Processor -- start 2024-03-21T10:26:16.760 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:16.759+0000 7f5f0cc00640 1 -- start start 2024-03-21T10:26:16.760 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:16.760+0000 7f5f0cc00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5f08073a50 0x7f5f08199ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:26:16.760 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:16.760+0000 7f5f0cc00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5f08074370 0x7f5f0819a420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:26:16.760 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:16.760+0000 7f5f0cc00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5f0810dd60 0x7f5f0819e7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:26:16.760 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:16.760+0000 7f5f0cc00640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f08113930 con 0x7f5f0810dd60 2024-03-21T10:26:16.760 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:16.760+0000 7f5f0cc00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f081137b0 con 0x7f5f08074370 2024-03-21T10:26:16.761 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:16.760+0000 7f5f0cc00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f08113ab0 con 0x7f5f08073a50 2024-03-21T10:26:16.761 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:16.760+0000 7f5f07e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5f0810dd60 0x7f5f0819e7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:26:16.761 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:16.760+0000 7f5f07e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5f0810dd60 0x7f5f0819e7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:58710/0 (socket says 172.21.15.47:58710) 2024-03-21T10:26:16.761 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:16.760+0000 7f5f07e00640 1 -- 172.21.15.47:0/921812920 learned_addr learned my addr 172.21.15.47:0/921812920 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:26:16.762 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:16.760+0000 7f5f07400640 1 --2- 172.21.15.47:0/921812920 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5f08073a50 0x7f5f08199ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:26:16.762 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:16.760+0000 7f5f06a00640 1 --2- 172.21.15.47:0/921812920 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5f08074370 0x7f5f0819a420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:26:16.762 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:16.761+0000 7f5f07e00640 1 -- 172.21.15.47:0/921812920 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5f08073a50 msgr2=0x7f5f08199ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:26:16.762 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:16.761+0000 7f5f07e00640 1 --2- 172.21.15.47:0/921812920 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5f08073a50 0x7f5f08199ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:26:16.762 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:16.761+0000 7f5f07e00640 1 -- 172.21.15.47:0/921812920 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5f08074370 msgr2=0x7f5f0819a420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:26:16.762 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:16.761+0000 7f5f07e00640 1 --2- 172.21.15.47:0/921812920 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5f08074370 0x7f5f0819a420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:26:16.762 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:16.761+0000 7f5f07e00640 1 -- 172.21.15.47:0/921812920 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f0819eed0 con 0x7f5f0810dd60 2024-03-21T10:26:16.762 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:16.761+0000 7f5f07e00640 1 --2- 172.21.15.47:0/921812920 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5f0810dd60 0x7f5f0819e7a0 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7f5f00009fd0 tx=0x7f5f0000c800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:26:16.762 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:16.761+0000 7f5effe00640 1 -- 172.21.15.47:0/921812920 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f00004b20 con 0x7f5f0810dd60 2024-03-21T10:26:16.763 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:16.761+0000 7f5effe00640 1 -- 172.21.15.47:0/921812920 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f5f00003e20 con 0x7f5f0810dd60 2024-03-21T10:26:16.763 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:16.761+0000 7f5effe00640 1 -- 172.21.15.47:0/921812920 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f00004820 con 0x7f5f0810dd60 2024-03-21T10:26:16.763 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:16.761+0000 7f5f0cc00640 1 -- 172.21.15.47:0/921812920 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5f0819f1c0 con 0x7f5f0810dd60 2024-03-21T10:26:16.763 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:16.762+0000 7f5f0cc00640 1 -- 172.21.15.47:0/921812920 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5f081a6a50 con 0x7f5f0810dd60 2024-03-21T10:26:16.766 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:16.766+0000 7f5f10f8d740 1 Processor -- start 2024-03-21T10:26:16.767 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:16.767+0000 7f5effe00640 1 -- 172.21.15.47:0/921812920 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 26) v1 ==== 138087+0+0 (secure 0 0 0) 0x7f5f0001d020 con 0x7f5f0810dd60 2024-03-21T10:26:16.767 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:16.767+0000 7f5f10f8d740 1 -- start start 2024-03-21T10:26:16.767 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:16.767+0000 7f5f10f8d740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55bfce3d29f0 0x55bfce3d2dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:26:16.768 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:16.767+0000 7f5f10f8d740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55bfce3d3310 0x55bfce3dbba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:26:16.768 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:16.767+0000 7f5f10f8d740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55bfce3dc0e0 0x55bfce3de4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:26:16.768 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:16.767+0000 7f5f10f8d740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bfce38c3b0 con 0x55bfce3dc0e0 2024-03-21T10:26:16.768 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:16.767+0000 7f5f10f8d740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bfce32e830 con 0x55bfce3d3310 2024-03-21T10:26:16.768 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:16.767+0000 7f5f10f8d740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bfce2d75d0 con 0x55bfce3d29f0 2024-03-21T10:26:16.768 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:16.768+0000 7f5f07400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55bfce3d3310 0x55bfce3dbba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:26:16.769 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:16.768+0000 7f5f07400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55bfce3d3310 0x55bfce3dbba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:41492/0 (socket says 172.21.15.47:41492) 2024-03-21T10:26:16.769 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:16.768+0000 7f5f07400640 1 -- 172.21.15.47:0/1686357578 learned_addr learned my addr 172.21.15.47:0/1686357578 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:26:16.769 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:16.768+0000 7f5f06a00640 1 --2- 172.21.15.47:0/1686357578 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55bfce3dc0e0 0x55bfce3de4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:26:16.769 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:16.768+0000 7f5f07400640 1 -- 172.21.15.47:0/1686357578 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55bfce3d29f0 msgr2=0x55bfce3d2dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:26:16.769 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:16.768+0000 7f5f07400640 1 --2- 172.21.15.47:0/1686357578 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55bfce3d29f0 0x55bfce3d2dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:26:16.769 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:16.768+0000 7f5f07400640 1 -- 172.21.15.47:0/1686357578 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55bfce3dc0e0 msgr2=0x55bfce3de4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:26:16.769 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:16.768+0000 7f5f07400640 1 --2- 172.21.15.47:0/1686357578 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55bfce3dc0e0 0x55bfce3de4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:26:16.770 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:16.769+0000 7f5f07400640 1 -- 172.21.15.47:0/1686357578 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55bfce2ef2c0 con 0x55bfce3d3310 2024-03-21T10:26:16.770 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:16.769+0000 7f5f06a00640 1 --2- 172.21.15.47:0/1686357578 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55bfce3dc0e0 0x55bfce3de4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T10:26:16.770 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:16.769+0000 7f5effe00640 1 --2- 172.21.15.47:0/921812920 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f5eec0a32a0 0x7f5eec0a5760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:26:16.770 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:16.769+0000 7f5effe00640 1 -- 172.21.15.47:0/921812920 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 9474+0+0 (secure 0 0 0) 0x7f5f000cfd30 con 0x7f5f0810dd60 2024-03-21T10:26:16.770 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:16.769+0000 7f5f06a00640 1 --2- 172.21.15.47:0/921812920 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f5eec0a32a0 0x7f5eec0a5760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:26:16.770 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:16.769+0000 7f5f07400640 1 --2- 172.21.15.47:0/1686357578 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55bfce3d3310 0x55bfce3dbba0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f5ef8009d20 tx=0x7f5ef8011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=86ed301be0305064 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:26:16.771 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:16.770+0000 7f5ef3e00640 1 -- 172.21.15.47:0/1686357578 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ef801e070 con 0x55bfce3d3310 2024-03-21T10:26:16.771 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:16.770+0000 7f5ef3e00640 1 -- 172.21.15.47:0/1686357578 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f5ef8019ae0 con 0x55bfce3d3310 2024-03-21T10:26:16.771 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:16.770+0000 7f5ef3e00640 1 -- 172.21.15.47:0/1686357578 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ef800eac0 con 0x55bfce3d3310 2024-03-21T10:26:16.771 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:16.771+0000 7f5f10f8d740 1 -- 172.21.15.47:0/1686357578 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55bfce3d3310 msgr2=0x55bfce3dbba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:26:16.771 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:16.771+0000 7f5f06a00640 1 --2- 172.21.15.47:0/921812920 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f5eec0a32a0 0x7f5eec0a5760 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f5ef402c230 tx=0x7f5ef402e040 comp rx=0 tx=0).ready entity=mgr.15687 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:26:16.771 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:16.771+0000 7f5f10f8d740 1 --2- 172.21.15.47:0/1686357578 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55bfce3d3310 0x55bfce3dbba0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f5ef8009d20 tx=0x7f5ef8011810 comp rx=0 tx=0).stop 2024-03-21T10:26:16.778 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:16.778+0000 7f5f10f8d740 1 -- 172.21.15.47:0/1686357578 shutdown_connections 2024-03-21T10:26:16.778 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:16.778+0000 7f5f10f8d740 1 --2- 172.21.15.47:0/1686357578 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55bfce3dc0e0 0x55bfce3de4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:26:16.778 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:16.778+0000 7f5f10f8d740 1 --2- 172.21.15.47:0/1686357578 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55bfce3d3310 0x55bfce3dbba0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:26:16.778 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:16.778+0000 7f5f10f8d740 1 --2- 172.21.15.47:0/1686357578 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55bfce3d29f0 0x55bfce3d2dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:26:16.778 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:16.778+0000 7f5f10f8d740 1 -- 172.21.15.47:0/1686357578 >> 172.21.15.47:0/1686357578 conn(0x55bfce3cd310 msgr2=0x55bfce3cd6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:26:16.779 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:16.778+0000 7f5f10f8d740 1 -- 172.21.15.47:0/1686357578 shutdown_connections 2024-03-21T10:26:16.779 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:16.778+0000 7f5f10f8d740 1 -- 172.21.15.47:0/1686357578 wait complete. 2024-03-21T10:26:16.779 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:16.778+0000 7f5f10f8d740 1 -- start start 2024-03-21T10:26:16.780 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:16.778+0000 7f5f10f8d740 1 Processor -- start 2024-03-21T10:26:16.780 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:16.779+0000 7f5f10f8d740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55bfce3d29f0 0x55bfce3d91f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:26:16.780 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:16.779+0000 7f5f10f8d740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55bfce3d3310 0x55bfce3d9730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:26:16.780 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:16.779+0000 7f5f10f8d740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55bfce3dc0e0 0x55bfce3d9c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:26:16.780 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:16.779+0000 7f5f10f8d740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bfce2eaa20 con 0x55bfce3dc0e0 2024-03-21T10:26:16.780 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:16.779+0000 7f5f10f8d740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bfce32a8d0 con 0x55bfce3d3310 2024-03-21T10:26:16.780 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:16.779+0000 7f5f10f8d740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bfce3ded10 con 0x55bfce3d29f0 2024-03-21T10:26:16.781 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:16.779+0000 7f5f10f8d740 10 client.0 _ll_register_callbacks cb 0x55bfce3271c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T10:26:16.781 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:16.780+0000 7f5f07e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55bfce3dc0e0 0x55bfce3d9c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:26:16.781 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:16.780+0000 7f5f07e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55bfce3dc0e0 0x55bfce3d9c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:58718/0 (socket says 172.21.15.47:58718) 2024-03-21T10:26:16.781 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:16.780+0000 7f5f07e00640 1 -- 172.21.15.47:0/2564444636 learned_addr learned my addr 172.21.15.47:0/2564444636 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:26:16.781 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:16.780+0000 7f5f06a00640 1 --2- 172.21.15.47:0/2564444636 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55bfce3d3310 0x55bfce3d9730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:26:16.781 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:16.780+0000 7f5f07400640 1 --2- 172.21.15.47:0/2564444636 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55bfce3d29f0 0x55bfce3d91f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:26:16.782 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:16.780+0000 7f5f07e00640 1 -- 172.21.15.47:0/2564444636 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55bfce3d29f0 msgr2=0x55bfce3d91f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:26:16.782 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:16.780+0000 7f5f07e00640 1 --2- 172.21.15.47:0/2564444636 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55bfce3d29f0 0x55bfce3d91f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:26:16.782 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:16.780+0000 7f5f07e00640 1 -- 172.21.15.47:0/2564444636 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55bfce3d3310 msgr2=0x55bfce3d9730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:26:16.782 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:16.780+0000 7f5f07e00640 1 --2- 172.21.15.47:0/2564444636 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55bfce3d3310 0x55bfce3d9730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:26:16.782 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:16.780+0000 7f5f07e00640 1 -- 172.21.15.47:0/2564444636 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55bfce043210 con 0x55bfce3dc0e0 2024-03-21T10:26:16.782 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:16.781+0000 7f5f07e00640 1 --2- 172.21.15.47:0/2564444636 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55bfce3dc0e0 0x55bfce3d9c70 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7f5f0009b7b0 tx=0x7f5f0009b6b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cad0256c235475be server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:26:16.782 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:16.781+0000 7f5ef0c00640 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-03-21T10:26:16.782 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:16.781+0000 7f5ef0c00640 1 -- 172.21.15.47:0/2564444636 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f0001a040 con 0x55bfce3dc0e0 2024-03-21T10:26:16.783 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:16.781+0000 7f5f10f8d740 1 -- 172.21.15.47:0/2564444636 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f5ef8019ae0 con 0x55bfce3dc0e0 2024-03-21T10:26:16.783 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:16.781+0000 7f5ef0c00640 1 -- 172.21.15.47:0/2564444636 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 9474+0+0 (secure 0 0 0) 0x7f5f00003970 con 0x55bfce3dc0e0 2024-03-21T10:26:16.783 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:16.781+0000 7f5ef0c00640 1 -- 172.21.15.47:0/2564444636 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5f0009bab0 con 0x55bfce3dc0e0 2024-03-21T10:26:16.783 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:16.782+0000 7f5f10f8d740 10 client.16098 fetch_fsmap learned FSMap version 30 2024-03-21T10:26:16.783 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:16.782+0000 7f5f10f8d740 1 -- 172.21.15.47:0/2564444636 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x55bfce10f620 con 0x55bfce3dc0e0 2024-03-21T10:26:16.783 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:16.782+0000 7f5ef0c00640 1 -- 172.21.15.47:0/2564444636 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f5ef8019ae0 con 0x55bfce3dc0e0 2024-03-21T10:26:16.784 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:16.782+0000 7f5ef0c00640 1 -- 172.21.15.47:0/2564444636 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 30) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f5f0000b040 con 0x55bfce3dc0e0 2024-03-21T10:26:16.784 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:16.782+0000 7f5f10f8d740 10 client.16098 fetch_fsmap finished waiting for FSMap version 30 2024-03-21T10:26:16.784 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:16.782+0000 7f5f10f8d740 10 client.16098 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-21T10:26:16.784 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:16.782+0000 7f5f10f8d740 20 client.16098 populate_metadata read hostname 'smithi047' 2024-03-21T10:26:16.784 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:16.782+0000 7f5f10f8d740 1 --2- 172.21.15.47:0/2564444636 >> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] conn(0x55bfce3ec1c0 0x55bfce3ee5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:26:16.784 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:16.782+0000 7f5f10f8d740 4 client.16098 mds_command: new command op to 34535 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T10:26:16.784 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:16.782+0000 7f5f10f8d740 1 -- 172.21.15.47:0/2564444636 --> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55bfce33bd70 con 0x55bfce3ec1c0 2024-03-21T10:26:16.784 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:16.783+0000 7f5f07400640 1 --2- 172.21.15.47:0/2564444636 >> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] conn(0x55bfce3ec1c0 0x55bfce3ee5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:26:16.785 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:16.784+0000 7f5f07400640 1 --2- 172.21.15.47:0/2564444636 >> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] conn(0x55bfce3ec1c0 0x55bfce3ee5b0 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=875aa0f380917745 server_cookie=f5915d555ecb893e in_seq=0 out_seq=0 2024-03-21T10:26:16.785 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:16.784+0000 7f5ef0c00640 10 client.16098 ms_handle_connect on v2:172.21.15.100:6838/4173168440 2024-03-21T10:26:16.785 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:16.785+0000 7f5ef0c00640 1 -- 172.21.15.47:0/2564444636 <== mds.0 v2:172.21.15.100:6838/4173168440 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55bfce33bd70 con 0x55bfce3ec1c0 2024-03-21T10:26:16.785 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:16.785+0000 7f5ef0c00640 10 client.16098 handle_command_reply: tid=0 2024-03-21T10:26:16.785 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:16.785+0000 7f5f10f8d740 1 client.16098 shutdown 2024-03-21T10:26:16.785 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:16.785+0000 7f5f10f8d740 1 -- 172.21.15.47:0/2564444636 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55bfce3dc0e0 msgr2=0x55bfce3d9c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:26:16.785 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:16.785+0000 7f5f10f8d740 1 --2- 172.21.15.47:0/2564444636 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55bfce3dc0e0 0x55bfce3d9c70 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7f5f0009b7b0 tx=0x7f5f0009b6b0 comp rx=0 tx=0).stop 2024-03-21T10:26:16.786 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:16.785+0000 7f5f10f8d740 1 -- 172.21.15.47:0/2564444636 shutdown_connections 2024-03-21T10:26:16.786 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:16.785+0000 7f5f10f8d740 1 --2- 172.21.15.47:0/2564444636 >> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] conn(0x55bfce3ec1c0 0x55bfce3ee5b0 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:26:16.786 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:16.785+0000 7f5f10f8d740 1 --2- 172.21.15.47:0/2564444636 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55bfce3dc0e0 0x55bfce3d9c70 unknown :-1 s=CLOSED pgs=608 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:26:16.786 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:16.785+0000 7f5f10f8d740 1 --2- 172.21.15.47:0/2564444636 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55bfce3d3310 0x55bfce3d9730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:26:16.786 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:16.785+0000 7f5f10f8d740 1 --2- 172.21.15.47:0/2564444636 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55bfce3d29f0 0x55bfce3d91f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:26:16.786 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:16.785+0000 7f5f10f8d740 1 -- 172.21.15.47:0/2564444636 >> 172.21.15.47:0/2564444636 conn(0x55bfce3cd310 msgr2=0x55bfce3cf060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:26:16.786 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:16.785+0000 7f5ef0c00640 0 client.16098 ms_handle_reset on v2:172.21.15.100:6838/4173168440 2024-03-21T10:26:16.787 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:16.785+0000 7f5f10f8d740 1 -- 172.21.15.47:0/2564444636 shutdown_connections 2024-03-21T10:26:16.787 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:16.785+0000 7f5f10f8d740 1 -- 172.21.15.47:0/2564444636 wait complete. 2024-03-21T10:26:16.787 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:16.785+0000 7f5f10f8d740 20 client.16098 trim_cache size 0 max 16384 2024-03-21T10:26:16.809 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:16.808+0000 7f5f10f8d740 1 Processor -- start 2024-03-21T10:26:16.809 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:16.808+0000 7f5f10f8d740 1 -- start start 2024-03-21T10:26:16.809 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:16.809+0000 7f5f10f8d740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55bfce3d29f0 0x55bfce3db920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:26:16.809 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:16.809+0000 7f5f10f8d740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55bfce3d3310 0x55bfce3cee00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:26:16.809 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:16.809+0000 7f5f10f8d740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55bfce3dc0e0 0x55bfce3e7420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:26:16.809 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:16.809+0000 7f5f10f8d740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f081ac580 con 0x55bfce3dc0e0 2024-03-21T10:26:16.810 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:16.809+0000 7f5f10f8d740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f081adf00 con 0x55bfce3d3310 2024-03-21T10:26:16.810 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:16.809+0000 7f5f10f8d740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f081acbe0 con 0x55bfce3d29f0 2024-03-21T10:26:16.811 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:16.809+0000 7f5f07e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55bfce3dc0e0 0x55bfce3e7420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:26:16.811 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:16.809+0000 7f5f07e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55bfce3dc0e0 0x55bfce3e7420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:58732/0 (socket says 172.21.15.47:58732) 2024-03-21T10:26:16.811 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:16.809+0000 7f5f07e00640 1 -- 172.21.15.47:0/4085529171 learned_addr learned my addr 172.21.15.47:0/4085529171 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:26:16.811 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:16.809+0000 7f5f06a00640 1 --2- 172.21.15.47:0/4085529171 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55bfce3d3310 0x55bfce3cee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:26:16.811 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:16.809+0000 7f5f07400640 1 --2- 172.21.15.47:0/4085529171 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55bfce3d29f0 0x55bfce3db920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:26:16.811 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:16.809+0000 7f5f07e00640 1 -- 172.21.15.47:0/4085529171 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55bfce3d29f0 msgr2=0x55bfce3db920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:26:16.812 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:16.810+0000 7f5f07e00640 1 --2- 172.21.15.47:0/4085529171 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55bfce3d29f0 0x55bfce3db920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:26:16.812 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:16.810+0000 7f5f07e00640 1 -- 172.21.15.47:0/4085529171 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55bfce3d3310 msgr2=0x55bfce3cee00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:26:16.812 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:16.810+0000 7f5f07e00640 1 --2- 172.21.15.47:0/4085529171 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55bfce3d3310 0x55bfce3cee00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:26:16.812 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:16.810+0000 7f5f07e00640 1 -- 172.21.15.47:0/4085529171 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55bfce3d3140 con 0x55bfce3dc0e0 2024-03-21T10:26:16.812 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:16.810+0000 7f5f07400640 1 --2- 172.21.15.47:0/4085529171 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55bfce3d29f0 0x55bfce3db920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T10:26:16.812 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:16.810+0000 7f5f07e00640 1 --2- 172.21.15.47:0/4085529171 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55bfce3dc0e0 0x55bfce3e7420 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7f5f0000cd00 tx=0x7f5f0009b720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9b8ddedd90102fa1 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:26:16.812 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:16.810+0000 7f5ef3400640 1 -- 172.21.15.47:0/4085529171 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f0001a070 con 0x55bfce3dc0e0 2024-03-21T10:26:16.812 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:16.810+0000 7f5ef3400640 1 -- 172.21.15.47:0/4085529171 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f5f000a4240 con 0x55bfce3dc0e0 2024-03-21T10:26:16.812 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:16.811+0000 7f5ef3400640 1 -- 172.21.15.47:0/4085529171 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f0000e0c0 con 0x55bfce3dc0e0 2024-03-21T10:26:16.813 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:16.811+0000 7f5f10f8d740 1 -- 172.21.15.47:0/4085529171 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55bfce3dc0e0 msgr2=0x55bfce3e7420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:26:16.813 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:16.811+0000 7f5f10f8d740 1 --2- 172.21.15.47:0/4085529171 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55bfce3dc0e0 0x55bfce3e7420 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7f5f0000cd00 tx=0x7f5f0009b720 comp rx=0 tx=0).stop 2024-03-21T10:26:16.813 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:16.811+0000 7f5f10f8d740 1 -- 172.21.15.47:0/4085529171 shutdown_connections 2024-03-21T10:26:16.814 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:16.811+0000 7f5f10f8d740 1 --2- 172.21.15.47:0/4085529171 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55bfce3dc0e0 0x55bfce3e7420 unknown :-1 s=CLOSED pgs=609 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:26:16.814 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:16.811+0000 7f5f10f8d740 1 --2- 172.21.15.47:0/4085529171 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55bfce3d3310 0x55bfce3cee00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:26:16.814 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:16.811+0000 7f5f10f8d740 1 --2- 172.21.15.47:0/4085529171 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55bfce3d29f0 0x55bfce3db920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:26:16.814 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:16.812+0000 7f5f10f8d740 1 -- 172.21.15.47:0/4085529171 >> 172.21.15.47:0/4085529171 conn(0x55bfce3cd310 msgr2=0x55bfce3ce6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:26:16.814 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:16.812+0000 7f5f10f8d740 1 -- 172.21.15.47:0/4085529171 shutdown_connections 2024-03-21T10:26:16.814 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:16.812+0000 7f5f10f8d740 1 -- 172.21.15.47:0/4085529171 wait complete. 2024-03-21T10:26:16.814 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:16.812+0000 7f5f10f8d740 1 -- start start 2024-03-21T10:26:16.814 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:16.812+0000 7f5f10f8d740 1 Processor -- start 2024-03-21T10:26:16.814 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:16.813+0000 7f5f10f8d740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55bfce3d29f0 0x55bfce3df520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:26:16.814 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:16.813+0000 7f5f10f8d740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55bfce3d3310 0x55bfce3dfa60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:26:16.814 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:16.813+0000 7f5f10f8d740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55bfce3dc0e0 0x55bfce3dffa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:26:16.814 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:16.813+0000 7f5f10f8d740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f081ad8a0 con 0x55bfce3dc0e0 2024-03-21T10:26:16.814 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:16.813+0000 7f5f10f8d740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f081ad240 con 0x55bfce3d3310 2024-03-21T10:26:16.815 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:16.813+0000 7f5f10f8d740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f081aa740 con 0x55bfce3d29f0 2024-03-21T10:26:16.815 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:16.813+0000 7f5f10f8d740 10 client.0 _ll_register_callbacks cb 0x7f5f00004020 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T10:26:16.816 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:16.814+0000 7f5f07e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55bfce3dc0e0 0x55bfce3dffa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:26:16.816 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:16.814+0000 7f5f07e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55bfce3dc0e0 0x55bfce3dffa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:58734/0 (socket says 172.21.15.47:58734) 2024-03-21T10:26:16.816 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:16.814+0000 7f5f07e00640 1 -- 172.21.15.47:0/1220287651 learned_addr learned my addr 172.21.15.47:0/1220287651 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:26:16.817 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:16.814+0000 7f5f06a00640 1 --2- 172.21.15.47:0/1220287651 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55bfce3d3310 0x55bfce3dfa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:26:16.817 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:16.814+0000 7f5f07e00640 1 -- 172.21.15.47:0/1220287651 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55bfce3d29f0 msgr2=0x55bfce3df520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:26:16.817 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:16.814+0000 7f5f07e00640 1 --2- 172.21.15.47:0/1220287651 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55bfce3d29f0 0x55bfce3df520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:26:16.817 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:16.814+0000 7f5f07e00640 1 -- 172.21.15.47:0/1220287651 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55bfce3d3310 msgr2=0x55bfce3dfa60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:26:16.817 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:16.814+0000 7f5f07e00640 1 --2- 172.21.15.47:0/1220287651 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55bfce3d3310 0x55bfce3dfa60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:26:16.817 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:16.814+0000 7f5f07e00640 1 -- 172.21.15.47:0/1220287651 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55bfce446970 con 0x55bfce3dc0e0 2024-03-21T10:26:16.817 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:16.814+0000 7f5f07e00640 1 --2- 172.21.15.47:0/1220287651 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55bfce3dc0e0 0x55bfce3dffa0 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7f5f00009fa0 tx=0x7f5f00004140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cdf71709fa3a2aa6 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:26:16.817 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:16.815+0000 7f5ef0c00640 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-03-21T10:26:16.817 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:16.815+0000 7f5ef0c00640 1 -- 172.21.15.47:0/1220287651 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f0001a070 con 0x55bfce3dc0e0 2024-03-21T10:26:16.817 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:16.815+0000 7f5ef0c00640 1 -- 172.21.15.47:0/1220287651 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 9474+0+0 (secure 0 0 0) 0x7f5f000cce20 con 0x55bfce3dc0e0 2024-03-21T10:26:16.817 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:16.815+0000 7f5f10f8d740 1 -- 172.21.15.47:0/1220287651 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f5f000a4240 con 0x55bfce3dc0e0 2024-03-21T10:26:16.817 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:16.815+0000 7f5ef0c00640 1 -- 172.21.15.47:0/1220287651 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5f0009fc00 con 0x55bfce3dc0e0 2024-03-21T10:26:16.817 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:16.815+0000 7f5f10f8d740 10 client.16104 fetch_fsmap learned FSMap version 30 2024-03-21T10:26:16.818 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:16.815+0000 7f5f10f8d740 1 -- 172.21.15.47:0/1220287651 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x55bfce446970 con 0x55bfce3dc0e0 2024-03-21T10:26:16.818 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:16.816+0000 7f5ef0c00640 1 -- 172.21.15.47:0/1220287651 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f5f000a4240 con 0x55bfce3dc0e0 2024-03-21T10:26:16.818 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:16.816+0000 7f5ef0c00640 1 -- 172.21.15.47:0/1220287651 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 30) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f5f00097800 con 0x55bfce3dc0e0 2024-03-21T10:26:16.818 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:16.816+0000 7f5f10f8d740 10 client.16104 fetch_fsmap finished waiting for FSMap version 30 2024-03-21T10:26:16.818 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:16.816+0000 7f5f10f8d740 10 client.16104 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-21T10:26:16.818 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:16.816+0000 7f5f10f8d740 20 client.16104 populate_metadata read hostname 'smithi047' 2024-03-21T10:26:16.818 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:16.816+0000 7f5f10f8d740 1 --2- 172.21.15.47:0/1220287651 >> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] conn(0x55bfce447e40 0x55bfce44a230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:26:16.818 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:16.816+0000 7f5f10f8d740 4 client.16104 mds_command: new command op to 34535 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-21T10:26:16.818 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:16.816+0000 7f5f10f8d740 1 -- 172.21.15.47:0/1220287651 --> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55bfce3360c0 con 0x55bfce447e40 2024-03-21T10:26:16.819 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:16.817+0000 7f5f07400640 1 --2- 172.21.15.47:0/1220287651 >> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] conn(0x55bfce447e40 0x55bfce44a230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:26:16.819 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:16.817+0000 7f5f07400640 1 --2- 172.21.15.47:0/1220287651 >> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] conn(0x55bfce447e40 0x55bfce44a230 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=36cf987113049f2f server_cookie=eacd107d4eba7e1d in_seq=0 out_seq=0 2024-03-21T10:26:16.819 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:16.817+0000 7f5ef0c00640 10 client.16104 ms_handle_connect on v2:172.21.15.100:6838/4173168440 2024-03-21T10:26:16.824 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:16.824+0000 7f5ef0c00640 1 -- 172.21.15.47:0/1220287651 <== mds.0 v2:172.21.15.100:6838/4173168440 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55bfce3360c0 con 0x55bfce447e40 2024-03-21T10:26:16.824 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:16.824+0000 7f5ef0c00640 10 client.16104 handle_command_reply: tid=0 2024-03-21T10:26:16.824 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:16.824+0000 7f5f10f8d740 1 client.16104 shutdown 2024-03-21T10:26:16.825 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:16.824+0000 7f5f10f8d740 1 -- 172.21.15.47:0/1220287651 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55bfce3dc0e0 msgr2=0x55bfce3dffa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:26:16.825 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:16.824+0000 7f5f10f8d740 1 --2- 172.21.15.47:0/1220287651 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55bfce3dc0e0 0x55bfce3dffa0 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7f5f00009fa0 tx=0x7f5f00004140 comp rx=0 tx=0).stop 2024-03-21T10:26:16.825 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:16.824+0000 7f5f10f8d740 1 -- 172.21.15.47:0/1220287651 shutdown_connections 2024-03-21T10:26:16.825 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:16.824+0000 7f5f10f8d740 1 --2- 172.21.15.47:0/1220287651 >> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] conn(0x55bfce447e40 0x55bfce44a230 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:26:16.825 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:16.824+0000 7f5ef0c00640 0 client.16104 ms_handle_reset on v2:172.21.15.100:6838/4173168440 2024-03-21T10:26:16.825 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:16.824+0000 7f5f10f8d740 1 --2- 172.21.15.47:0/1220287651 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55bfce3dc0e0 0x55bfce3dffa0 unknown :-1 s=CLOSED pgs=610 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:26:16.825 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:16.824+0000 7f5f10f8d740 1 --2- 172.21.15.47:0/1220287651 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55bfce3d3310 0x55bfce3dfa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:26:16.825 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:16.824+0000 7f5f10f8d740 1 --2- 172.21.15.47:0/1220287651 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55bfce3d29f0 0x55bfce3df520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:26:16.826 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:16.824+0000 7f5f10f8d740 1 -- 172.21.15.47:0/1220287651 >> 172.21.15.47:0/1220287651 conn(0x55bfce3cd310 msgr2=0x55bfce3d9a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:26:16.826 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:16.825+0000 7f5f10f8d740 1 -- 172.21.15.47:0/1220287651 shutdown_connections 2024-03-21T10:26:16.826 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:16.825+0000 7f5f10f8d740 1 -- 172.21.15.47:0/1220287651 wait complete. 2024-03-21T10:26:16.826 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:16.825+0000 7f5f10f8d740 20 client.16104 trim_cache size 0 max 16384 2024-03-21T10:26:16.826 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout:{ 2024-03-21T10:26:16.826 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout: "message": "", 2024-03-21T10:26:16.826 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout: "return_code": 0 2024-03-21T10:26:16.826 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout:} 2024-03-21T10:26:16.827 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:16.825+0000 7f5f0cc00640 1 -- 172.21.15.47:0/921812920 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f5eec0a32a0 msgr2=0x7f5eec0a5760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:26:16.827 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:16.825+0000 7f5f0cc00640 1 --2- 172.21.15.47:0/921812920 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f5eec0a32a0 0x7f5eec0a5760 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f5ef402c230 tx=0x7f5ef402e040 comp rx=0 tx=0).stop 2024-03-21T10:26:16.828 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:16.825+0000 7f5f0cc00640 1 -- 172.21.15.47:0/921812920 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5f0810dd60 msgr2=0x7f5f0819e7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:26:16.828 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:16.825+0000 7f5f0cc00640 1 --2- 172.21.15.47:0/921812920 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5f0810dd60 0x7f5f0819e7a0 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7f5f00009fd0 tx=0x7f5f0000c800 comp rx=0 tx=0).stop 2024-03-21T10:26:16.828 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:16.826+0000 7f5f0cc00640 1 -- 172.21.15.47:0/921812920 shutdown_connections 2024-03-21T10:26:16.828 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:16.826+0000 7f5f0cc00640 1 --2- 172.21.15.47:0/921812920 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5f0810dd60 0x7f5f0819e7a0 unknown :-1 s=CLOSED pgs=607 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:26:16.828 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:16.826+0000 7f5f0cc00640 1 --2- 172.21.15.47:0/921812920 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f5eec0a32a0 0x7f5eec0a5760 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:26:16.828 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:16.826+0000 7f5f0cc00640 1 --2- 172.21.15.47:0/921812920 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5f08074370 0x7f5f0819a420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:26:16.828 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:16.826+0000 7f5f0cc00640 1 --2- 172.21.15.47:0/921812920 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5f08073a50 0x7f5f08199ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:26:16.828 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:16.826+0000 7f5f0cc00640 1 -- 172.21.15.47:0/921812920 >> 172.21.15.47:0/921812920 conn(0x7f5f080fcfb0 msgr2=0x7f5f080fddc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:26:16.828 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:16.826+0000 7f5f0cc00640 1 -- 172.21.15.47:0/921812920 shutdown_connections 2024-03-21T10:26:16.828 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:16.826+0000 7f5f0cc00640 1 -- 172.21.15.47:0/921812920 wait complete. 2024-03-21T10:26:17.380 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-03-21T10:26:17.380 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-21T10:26:17.380 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-03-21T10:26:17.452 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:26:17 smithi047 ceph-mon[31547]: pgmap v31: 129 pgs: 129 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 597 B/s wr, 11 op/s 2024-03-21T10:26:17.452 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:26:17 smithi047 ceph-mon[31547]: daemon mds.i restarted 2024-03-21T10:26:17.478 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:26:17 smithi100 ceph-mon[35193]: pgmap v31: 129 pgs: 129 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 597 B/s wr, 11 op/s 2024-03-21T10:26:17.479 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:26:17 smithi100 ceph-mon[35193]: daemon mds.i restarted 2024-03-21T10:26:17.489 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:26:17 smithi157 ceph-mon[35581]: pgmap v31: 129 pgs: 129 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 597 B/s wr, 11 op/s 2024-03-21T10:26:17.489 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:26:17 smithi157 ceph-mon[35581]: daemon mds.i restarted 2024-03-21T10:26:17.624 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:17.624+0000 7fe439600640 1 -- 172.21.15.47:0/4166674305 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe4341075e0 msgr2=0x7fe4341079c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:26:17.625 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:17.624+0000 7fe439600640 1 --2- 172.21.15.47:0/4166674305 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe4341075e0 0x7fe4341079c0 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7fe424009900 tx=0x7fe42402bc30 comp rx=0 tx=0).stop 2024-03-21T10:26:17.625 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:17.624+0000 7fe439600640 1 -- 172.21.15.47:0/4166674305 shutdown_connections 2024-03-21T10:26:17.625 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:17.624+0000 7fe439600640 1 --2- 172.21.15.47:0/4166674305 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe434101ee0 0x7fe43410e3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:26:17.625 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:17.624+0000 7fe439600640 1 --2- 172.21.15.47:0/4166674305 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe434101520 0x7fe4341019a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:26:17.625 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:17.624+0000 7fe439600640 1 --2- 172.21.15.47:0/4166674305 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe4341075e0 0x7fe4341079c0 unknown :-1 s=CLOSED pgs=611 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:26:17.625 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:17.624+0000 7fe439600640 1 -- 172.21.15.47:0/4166674305 >> 172.21.15.47:0/4166674305 conn(0x7fe4340fd010 msgr2=0x7fe4340ff430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:26:17.625 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:17.624+0000 7fe439600640 1 -- 172.21.15.47:0/4166674305 shutdown_connections 2024-03-21T10:26:17.625 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:17.624+0000 7fe439600640 1 -- 172.21.15.47:0/4166674305 wait complete. 2024-03-21T10:26:17.626 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:17.625+0000 7fe439600640 1 Processor -- start 2024-03-21T10:26:17.626 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:17.625+0000 7fe439600640 1 -- start start 2024-03-21T10:26:17.627 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:17.626+0000 7fe439600640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe434101520 0x7fe4340739a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:26:17.627 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:17.626+0000 7fe439600640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe434101ee0 0x7fe434073ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:26:17.627 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:17.626+0000 7fe439600640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe4341075e0 0x7fe434074420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:26:17.627 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:17.626+0000 7fe439600640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe434111770 con 0x7fe4341075e0 2024-03-21T10:26:17.627 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:17.626+0000 7fe439600640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4341115f0 con 0x7fe434101520 2024-03-21T10:26:17.627 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:17.626+0000 7fe439600640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4341118f0 con 0x7fe434101ee0 2024-03-21T10:26:17.627 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:17.626+0000 7fe438c00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe4341075e0 0x7fe434074420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:26:17.627 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:17.626+0000 7fe438c00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe4341075e0 0x7fe434074420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:58742/0 (socket says 172.21.15.47:58742) 2024-03-21T10:26:17.627 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:17.626+0000 7fe438c00640 1 -- 172.21.15.47:0/810055140 learned_addr learned my addr 172.21.15.47:0/810055140 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:26:17.627 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:17.627+0000 7fe433e00640 1 --2- 172.21.15.47:0/810055140 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe434101520 0x7fe4340739a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:26:17.628 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:17.627+0000 7fe433400640 1 --2- 172.21.15.47:0/810055140 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe434101ee0 0x7fe434073ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:26:17.628 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:17.627+0000 7fe438c00640 1 -- 172.21.15.47:0/810055140 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe434101ee0 msgr2=0x7fe434073ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:26:17.628 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:17.627+0000 7fe438c00640 1 --2- 172.21.15.47:0/810055140 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe434101ee0 0x7fe434073ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:26:17.628 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:17.627+0000 7fe438c00640 1 -- 172.21.15.47:0/810055140 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe434101520 msgr2=0x7fe4340739a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:26:17.628 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:17.627+0000 7fe438c00640 1 --2- 172.21.15.47:0/810055140 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe434101520 0x7fe4340739a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:26:17.628 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:17.627+0000 7fe438c00640 1 -- 172.21.15.47:0/810055140 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe4341a6110 con 0x7fe4341075e0 2024-03-21T10:26:17.628 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:17.627+0000 7fe433e00640 1 --2- 172.21.15.47:0/810055140 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe434101520 0x7fe4340739a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T10:26:17.629 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:17.628+0000 7fe438c00640 1 --2- 172.21.15.47:0/810055140 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe4341075e0 0x7fe434074420 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7fe42c00eaa0 tx=0x7fe42c00ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:26:17.629 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:17.628+0000 7fe430c00640 1 -- 172.21.15.47:0/810055140 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe42c00cdb0 con 0x7fe4341075e0 2024-03-21T10:26:17.629 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:17.628+0000 7fe430c00640 1 -- 172.21.15.47:0/810055140 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fe42c005020 con 0x7fe4341075e0 2024-03-21T10:26:17.629 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:17.628+0000 7fe439600640 1 -- 172.21.15.47:0/810055140 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe4341a6400 con 0x7fe4341075e0 2024-03-21T10:26:17.629 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:17.628+0000 7fe430c00640 1 -- 172.21.15.47:0/810055140 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe42c0116d0 con 0x7fe4341075e0 2024-03-21T10:26:17.630 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:17.629+0000 7fe439600640 1 -- 172.21.15.47:0/810055140 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe4341a69a0 con 0x7fe4341075e0 2024-03-21T10:26:17.634 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:17.634+0000 7fe43d9f3740 1 Processor -- start 2024-03-21T10:26:17.634 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:17.634+0000 7fe430c00640 1 -- 172.21.15.47:0/810055140 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 26) v1 ==== 138087+0+0 (secure 0 0 0) 0x7fe42c0051d0 con 0x7fe4341075e0 2024-03-21T10:26:17.634 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:17.634+0000 7fe43d9f3740 1 -- start start 2024-03-21T10:26:17.635 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:17.634+0000 7fe43d9f3740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e428380290 0x55e428380670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:26:17.635 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:17.634+0000 7fe43d9f3740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e428380bb0 0x55e428389250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:26:17.635 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:17.634+0000 7fe43d9f3740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e428389790 0x55e42838bb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:26:17.635 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:17.634+0000 7fe43d9f3740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e4282e4eb0 con 0x55e428380290 2024-03-21T10:26:17.635 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:17.634+0000 7fe43d9f3740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e42828da60 con 0x55e428380bb0 2024-03-21T10:26:17.635 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:17.634+0000 7fe43d9f3740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e4282e0f50 con 0x55e428389790 2024-03-21T10:26:17.635 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:17.635+0000 7fe433400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e428380290 0x55e428380670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:26:17.635 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:17.635+0000 7fe433e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e428380bb0 0x55e428389250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:26:17.635 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:17.635+0000 7fe433e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e428380bb0 0x55e428389250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:41546/0 (socket says 172.21.15.47:41546) 2024-03-21T10:26:17.636 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:17.635+0000 7fe433e00640 1 -- 172.21.15.47:0/963545502 learned_addr learned my addr 172.21.15.47:0/963545502 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:26:17.636 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:17.636+0000 7fe433e00640 1 -- 172.21.15.47:0/963545502 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e428389790 msgr2=0x55e42838bb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:26:17.636 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:17.636+0000 7fe433400640 1 --2- 172.21.15.47:0/963545502 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e428389790 0x55e42838bb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:26:17.636 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:17.636+0000 7fe433e00640 1 --2- 172.21.15.47:0/963545502 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e428389790 0x55e42838bb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:26:17.636 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:17.636+0000 7fe433e00640 1 -- 172.21.15.47:0/963545502 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e428380290 msgr2=0x55e428380670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:26:17.636 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:17.636+0000 7fe433e00640 1 --2- 172.21.15.47:0/963545502 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e428380290 0x55e428380670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:26:17.636 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:17.636+0000 7fe433e00640 1 -- 172.21.15.47:0/963545502 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e4280c17e0 con 0x55e428380bb0 2024-03-21T10:26:17.637 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:17.636+0000 7fe433e00640 1 --2- 172.21.15.47:0/963545502 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e428380bb0 0x55e428389250 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fe424009900 tx=0x7fe424009610 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2a50f275a778e52b server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:26:17.637 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:17.636+0000 7fe430c00640 1 --2- 172.21.15.47:0/810055140 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7fe4180a32e0 0x7fe4180a57a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:26:17.637 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:17.637+0000 7fe429200640 1 -- 172.21.15.47:0/963545502 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe42402c900 con 0x55e428380bb0 2024-03-21T10:26:17.637 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:17.637+0000 7fe430c00640 1 -- 172.21.15.47:0/810055140 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 9474+0+0 (secure 0 0 0) 0x7fe42c0cfb00 con 0x7fe4341075e0 2024-03-21T10:26:17.637 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:17.637+0000 7fe429200640 1 -- 172.21.15.47:0/963545502 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fe42402caa0 con 0x55e428380bb0 2024-03-21T10:26:17.637 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:17.637+0000 7fe429200640 1 -- 172.21.15.47:0/963545502 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe424003e50 con 0x55e428380bb0 2024-03-21T10:26:17.638 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:17.637+0000 7fe433400640 1 --2- 172.21.15.47:0/810055140 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7fe4180a32e0 0x7fe4180a57a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:26:17.638 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:17.638+0000 7fe43d9f3740 1 -- 172.21.15.47:0/963545502 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e428380bb0 msgr2=0x55e428389250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:26:17.638 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:17.638+0000 7fe43d9f3740 1 --2- 172.21.15.47:0/963545502 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e428380bb0 0x55e428389250 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fe424009900 tx=0x7fe424009610 comp rx=0 tx=0).stop 2024-03-21T10:26:17.638 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:17.638+0000 7fe43d9f3740 1 -- 172.21.15.47:0/963545502 shutdown_connections 2024-03-21T10:26:17.638 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:17.638+0000 7fe43d9f3740 1 --2- 172.21.15.47:0/963545502 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e428389790 0x55e42838bb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:26:17.638 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:17.638+0000 7fe43d9f3740 1 --2- 172.21.15.47:0/963545502 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e428380bb0 0x55e428389250 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:26:17.638 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:17.638+0000 7fe43d9f3740 1 --2- 172.21.15.47:0/963545502 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e428380290 0x55e428380670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:26:17.638 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:17.638+0000 7fe43d9f3740 1 -- 172.21.15.47:0/963545502 >> 172.21.15.47:0/963545502 conn(0x55e42837a9c0 msgr2=0x55e42837ada0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:26:17.639 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:17.638+0000 7fe433400640 1 --2- 172.21.15.47:0/810055140 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7fe4180a32e0 0x7fe4180a57a0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x55e42837bb50 tx=0x7fe42000b650 comp rx=0 tx=0).ready entity=mgr.15687 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:26:17.639 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:17.639+0000 7fe43d9f3740 1 -- 172.21.15.47:0/963545502 shutdown_connections 2024-03-21T10:26:17.639 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:17.639+0000 7fe43d9f3740 1 -- 172.21.15.47:0/963545502 wait complete. 2024-03-21T10:26:17.640 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:17.639+0000 7fe43d9f3740 1 -- start start 2024-03-21T10:26:17.640 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:17.639+0000 7fe43d9f3740 1 Processor -- start 2024-03-21T10:26:17.640 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:17.640+0000 7fe43d9f3740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e428380290 0x55e428389100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:26:17.640 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:17.640+0000 7fe43d9f3740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e428380bb0 0x55e428386490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:26:17.641 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:17.640+0000 7fe43d9f3740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e428389790 0x55e4283869f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:26:17.641 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:17.640+0000 7fe43d9f3740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e42838c520 con 0x55e428380290 2024-03-21T10:26:17.641 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:17.640+0000 7fe43d9f3740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e4282a0eb0 con 0x55e428380bb0 2024-03-21T10:26:17.641 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:17.640+0000 7fe43d9f3740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e42838c6a0 con 0x55e428389790 2024-03-21T10:26:17.642 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:17.640+0000 7fe43d9f3740 10 client.0 _ll_register_callbacks cb 0x55e428295150 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T10:26:17.642 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:17.640+0000 7fe433e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e428380290 0x55e428389100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:26:17.642 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:17.640+0000 7fe433e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e428380290 0x55e428389100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:58772/0 (socket says 172.21.15.47:58772) 2024-03-21T10:26:17.642 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:17.640+0000 7fe433e00640 1 -- 172.21.15.47:0/542658689 learned_addr learned my addr 172.21.15.47:0/542658689 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:26:17.642 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:17.641+0000 7fe438c00640 1 --2- 172.21.15.47:0/542658689 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e428389790 0x55e4283869f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:26:17.642 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:17.641+0000 7fe433e00640 1 -- 172.21.15.47:0/542658689 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e428389790 msgr2=0x55e4283869f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:26:17.642 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:17.641+0000 7fe433e00640 1 --2- 172.21.15.47:0/542658689 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e428389790 0x55e4283869f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:26:17.642 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:17.641+0000 7fe433e00640 1 -- 172.21.15.47:0/542658689 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e428380bb0 msgr2=0x55e428386490 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-21T10:26:17.642 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:17.641+0000 7fe433e00640 1 --2- 172.21.15.47:0/542658689 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e428380bb0 0x55e428386490 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:26:17.642 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:17.641+0000 7fe433e00640 1 -- 172.21.15.47:0/542658689 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55e428121f70 con 0x55e428380290 2024-03-21T10:26:17.642 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:17.641+0000 7fe438c00640 1 --2- 172.21.15.47:0/542658689 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e428389790 0x55e4283869f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T10:26:17.643 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:17.642+0000 7fe433e00640 1 --2- 172.21.15.47:0/542658689 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e428380290 0x55e428389100 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7fe424006010 tx=0x7fe424003e00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6bc14eba112ba68a server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:26:17.643 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:17.642+0000 7fe41d600640 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-03-21T10:26:17.643 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:17.642+0000 7fe41d600640 1 -- 172.21.15.47:0/542658689 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe42404d070 con 0x55e428380290 2024-03-21T10:26:17.643 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:17.642+0000 7fe41d600640 1 -- 172.21.15.47:0/542658689 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(97..97 src has 1..97) v4 ==== 9474+0+0 (secure 0 0 0) 0x7fe424004400 con 0x55e428380290 2024-03-21T10:26:17.643 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:17.642+0000 7fe43d9f3740 1 -- 172.21.15.47:0/542658689 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fe42402caa0 con 0x55e428380290 2024-03-21T10:26:17.644 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:17.643+0000 7fe41d600640 1 -- 172.21.15.47:0/542658689 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe424058050 con 0x55e428380290 2024-03-21T10:26:17.644 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:17.643+0000 7fe43d9f3740 10 client.16116 fetch_fsmap learned FSMap version 32 2024-03-21T10:26:17.644 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:17.643+0000 7fe43d9f3740 1 -- 172.21.15.47:0/542658689 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x55e427fb9170 con 0x55e428380290 2024-03-21T10:26:17.644 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:17.643+0000 7fe41d600640 1 -- 172.21.15.47:0/542658689 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fe42402caa0 con 0x55e428380290 2024-03-21T10:26:17.644 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:17.643+0000 7fe41d600640 1 -- 172.21.15.47:0/542658689 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 32) v1 ==== 7942+0+0 (secure 0 0 0) 0x7fe424034440 con 0x55e428380290 2024-03-21T10:26:17.645 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:17.644+0000 7fe43d9f3740 10 client.16116 fetch_fsmap finished waiting for FSMap version 32 2024-03-21T10:26:17.645 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:17.644+0000 7fe43d9f3740 10 client.16116 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-21T10:26:17.645 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:17.644+0000 7fe43d9f3740 20 client.16116 populate_metadata read hostname 'smithi047' 2024-03-21T10:26:17.645 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:17.644+0000 7fe43d9f3740 1 --2- 172.21.15.47:0/542658689 >> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] conn(0x55e4283a1e50 0x55e4283a4240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:26:17.645 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:17.644+0000 7fe43d9f3740 4 client.16116 mds_command: new command op to 34535 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T10:26:17.645 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:17.644+0000 7fe43d9f3740 1 -- 172.21.15.47:0/542658689 --> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55e4282f23f0 con 0x55e4283a1e50 2024-03-21T10:26:17.645 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:17.644+0000 7fe433400640 1 --2- 172.21.15.47:0/542658689 >> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] conn(0x55e4283a1e50 0x55e4283a4240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:26:17.645 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:17.645+0000 7fe433400640 1 --2- 172.21.15.47:0/542658689 >> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] conn(0x55e4283a1e50 0x55e4283a4240 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=434634795f266194 server_cookie=7d9ee73d20451961 in_seq=0 out_seq=0 2024-03-21T10:26:17.646 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:17.645+0000 7fe41d600640 10 client.16116 ms_handle_connect on v2:172.21.15.100:6838/4173168440 2024-03-21T10:26:17.646 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:17.646+0000 7fe41d600640 1 -- 172.21.15.47:0/542658689 <== mds.0 v2:172.21.15.100:6838/4173168440 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55e4282f23f0 con 0x55e4283a1e50 2024-03-21T10:26:17.646 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:17.646+0000 7fe41d600640 10 client.16116 handle_command_reply: tid=0 2024-03-21T10:26:17.646 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:17.646+0000 7fe43d9f3740 1 client.16116 shutdown 2024-03-21T10:26:17.647 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:17.646+0000 7fe43d9f3740 1 -- 172.21.15.47:0/542658689 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e428380290 msgr2=0x55e428389100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:26:17.647 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:17.646+0000 7fe43d9f3740 1 --2- 172.21.15.47:0/542658689 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e428380290 0x55e428389100 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7fe424006010 tx=0x7fe424003e00 comp rx=0 tx=0).stop 2024-03-21T10:26:17.648 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:17.647+0000 7fe43d9f3740 1 -- 172.21.15.47:0/542658689 shutdown_connections 2024-03-21T10:26:17.648 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:17.647+0000 7fe43d9f3740 1 --2- 172.21.15.47:0/542658689 >> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] conn(0x55e4283a1e50 0x55e4283a4240 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:26:17.648 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:17.647+0000 7fe43d9f3740 1 --2- 172.21.15.47:0/542658689 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e428389790 0x55e4283869f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:26:17.648 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:17.647+0000 7fe43d9f3740 1 --2- 172.21.15.47:0/542658689 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e428380bb0 0x55e428386490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:26:17.648 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:17.647+0000 7fe43d9f3740 1 --2- 172.21.15.47:0/542658689 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e428380290 0x55e428389100 unknown :-1 s=CLOSED pgs=613 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:26:17.648 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:17.647+0000 7fe41d600640 0 client.16116 ms_handle_reset on v2:172.21.15.100:6838/4173168440 2024-03-21T10:26:17.648 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:17.647+0000 7fe43d9f3740 1 -- 172.21.15.47:0/542658689 >> 172.21.15.47:0/542658689 conn(0x55e42837a9c0 msgr2=0x55e42837c710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:26:17.648 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:17.647+0000 7fe43d9f3740 1 -- 172.21.15.47:0/542658689 shutdown_connections 2024-03-21T10:26:17.648 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:17.647+0000 7fe43d9f3740 1 -- 172.21.15.47:0/542658689 wait complete. 2024-03-21T10:26:17.648 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:17.647+0000 7fe43d9f3740 20 client.16116 trim_cache size 0 max 16384 2024-03-21T10:26:17.680 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:17.679+0000 7fe43d9f3740 1 Processor -- start 2024-03-21T10:26:17.680 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:17.679+0000 7fe43d9f3740 1 -- start start 2024-03-21T10:26:17.680 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:17.679+0000 7fe43d9f3740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e428380290 0x55e428379900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:26:17.680 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:17.679+0000 7fe43d9f3740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e428380bb0 0x55e428388b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:26:17.680 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:17.679+0000 7fe43d9f3740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e428389790 0x55e428389090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:26:17.680 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:17.679+0000 7fe43d9f3740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4341ac3f0 con 0x55e428389790 2024-03-21T10:26:17.680 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:17.679+0000 7fe43d9f3740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4341add70 con 0x55e428380bb0 2024-03-21T10:26:17.680 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:17.679+0000 7fe43d9f3740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4341aca50 con 0x55e428380290 2024-03-21T10:26:17.681 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:17.680+0000 7fe438c00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e428389790 0x55e428389090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:26:17.681 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:17.680+0000 7fe438c00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e428389790 0x55e428389090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:58786/0 (socket says 172.21.15.47:58786) 2024-03-21T10:26:17.681 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:17.680+0000 7fe438c00640 1 -- 172.21.15.47:0/1644441450 learned_addr learned my addr 172.21.15.47:0/1644441450 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:26:17.681 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:17.680+0000 7fe433400640 1 --2- 172.21.15.47:0/1644441450 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e428380bb0 0x55e428388b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:26:17.681 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:17.680+0000 7fe433e00640 1 --2- 172.21.15.47:0/1644441450 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e428380290 0x55e428379900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:26:17.681 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:17.680+0000 7fe438c00640 1 -- 172.21.15.47:0/1644441450 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e428380290 msgr2=0x55e428379900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:26:17.681 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:17.680+0000 7fe438c00640 1 --2- 172.21.15.47:0/1644441450 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e428380290 0x55e428379900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:26:17.682 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:17.680+0000 7fe438c00640 1 -- 172.21.15.47:0/1644441450 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e428380bb0 msgr2=0x55e428388b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:26:17.682 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:17.680+0000 7fe438c00640 1 --2- 172.21.15.47:0/1644441450 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e428380bb0 0x55e428388b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:26:17.682 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:17.680+0000 7fe438c00640 1 -- 172.21.15.47:0/1644441450 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e428386d60 con 0x55e428389790 2024-03-21T10:26:17.682 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:17.681+0000 7fe433400640 1 --2- 172.21.15.47:0/1644441450 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e428380bb0 0x55e428388b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T10:26:17.682 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:17.681+0000 7fe438c00640 1 --2- 172.21.15.47:0/1644441450 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e428389790 0x55e428389090 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7fe42c097d90 tx=0x7fe42c09b9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fc25a665270bde9e server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:26:17.683 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:17.681+0000 7fe41fe00640 1 -- 172.21.15.47:0/1644441450 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe42c0b3020 con 0x55e428389790 2024-03-21T10:26:17.683 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:17.681+0000 7fe41fe00640 1 -- 172.21.15.47:0/1644441450 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fe42c005560 con 0x55e428389790 2024-03-21T10:26:17.683 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:17.681+0000 7fe41fe00640 1 -- 172.21.15.47:0/1644441450 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe42c01f070 con 0x55e428389790 2024-03-21T10:26:17.683 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:17.682+0000 7fe43d9f3740 1 -- 172.21.15.47:0/1644441450 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e428389790 msgr2=0x55e428389090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:26:17.683 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:17.682+0000 7fe43d9f3740 1 --2- 172.21.15.47:0/1644441450 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e428389790 0x55e428389090 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7fe42c097d90 tx=0x7fe42c09b9f0 comp rx=0 tx=0).stop 2024-03-21T10:26:17.683 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:17.682+0000 7fe43d9f3740 1 -- 172.21.15.47:0/1644441450 shutdown_connections 2024-03-21T10:26:17.683 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:17.682+0000 7fe43d9f3740 1 --2- 172.21.15.47:0/1644441450 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e428389790 0x55e428389090 unknown :-1 s=CLOSED pgs=614 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:26:17.683 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:17.682+0000 7fe43d9f3740 1 --2- 172.21.15.47:0/1644441450 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e428380bb0 0x55e428388b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:26:17.683 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:17.682+0000 7fe43d9f3740 1 --2- 172.21.15.47:0/1644441450 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e428380290 0x55e428379900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:26:17.683 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:17.682+0000 7fe43d9f3740 1 -- 172.21.15.47:0/1644441450 >> 172.21.15.47:0/1644441450 conn(0x55e42837a9c0 msgr2=0x55e4283791f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:26:17.683 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:17.682+0000 7fe43d9f3740 1 -- 172.21.15.47:0/1644441450 shutdown_connections 2024-03-21T10:26:17.684 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:17.683+0000 7fe43d9f3740 1 -- 172.21.15.47:0/1644441450 wait complete. 2024-03-21T10:26:17.685 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:17.683+0000 7fe43d9f3740 1 -- start start 2024-03-21T10:26:17.685 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:17.683+0000 7fe43d9f3740 1 Processor -- start 2024-03-21T10:26:17.685 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:17.684+0000 7fe43d9f3740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e428380290 0x55e4283a5880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:26:17.685 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:17.684+0000 7fe43d9f3740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e428380bb0 0x55e4283a5dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:26:17.685 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:17.684+0000 7fe43d9f3740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e428389790 0x55e4283a6300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:26:17.685 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:17.684+0000 7fe43d9f3740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4341ad710 con 0x55e428380bb0 2024-03-21T10:26:17.685 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:17.684+0000 7fe43d9f3740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4341ad0b0 con 0x55e428380290 2024-03-21T10:26:17.685 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:17.684+0000 7fe43d9f3740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4341aa5d0 con 0x55e428389790 2024-03-21T10:26:17.685 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:17.684+0000 7fe43d9f3740 10 client.0 _ll_register_callbacks cb 0x7fe42000b810 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T10:26:17.685 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:17.684+0000 7fe433e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e428380290 0x55e4283a5880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:26:17.685 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:17.684+0000 7fe433400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e428380bb0 0x55e4283a5dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:26:17.686 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:17.684+0000 7fe433e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e428380290 0x55e4283a5880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:41574/0 (socket says 172.21.15.47:41574) 2024-03-21T10:26:17.686 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:17.684+0000 7fe433e00640 1 -- 172.21.15.47:0/4085614069 learned_addr learned my addr 172.21.15.47:0/4085614069 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:26:17.686 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:17.684+0000 7fe438c00640 1 --2- 172.21.15.47:0/4085614069 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e428389790 0x55e4283a6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:26:17.686 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:17.685+0000 7fe433e00640 1 -- 172.21.15.47:0/4085614069 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e428389790 msgr2=0x55e4283a6300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:26:17.686 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:17.685+0000 7fe433e00640 1 --2- 172.21.15.47:0/4085614069 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e428389790 0x55e4283a6300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:26:17.686 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:17.685+0000 7fe433e00640 1 -- 172.21.15.47:0/4085614069 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e428380bb0 msgr2=0x55e4283a5dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:26:17.686 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:17.685+0000 7fe433e00640 1 --2- 172.21.15.47:0/4085614069 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e428380bb0 0x55e4283a5dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:26:17.686 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:17.685+0000 7fe433e00640 1 -- 172.21.15.47:0/4085614069 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55e4283fd5b0 con 0x55e428380290 2024-03-21T10:26:17.686 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:17.685+0000 7fe438c00640 1 --2- 172.21.15.47:0/4085614069 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e428389790 0x55e4283a6300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T10:26:17.686 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:17.685+0000 7fe433400640 1 --2- 172.21.15.47:0/4085614069 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e428380bb0 0x55e4283a5dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T10:26:17.687 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:17.685+0000 7fe433e00640 1 --2- 172.21.15.47:0/4085614069 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e428380290 0x55e4283a5880 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fe424006010 tx=0x7fe424034c10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1a1516b9d4ba1c2c server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:26:17.687 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:17.685+0000 7fe41d600640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-21T10:26:17.688 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:17.685+0000 7fe41d600640 1 -- 172.21.15.47:0/4085614069 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe42404d040 con 0x55e428380290 2024-03-21T10:26:17.688 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:17.685+0000 7fe41d600640 1 -- 172.21.15.47:0/4085614069 <== mon.2 v2:172.21.15.100:3300/0 2 ==== osd_map(97..97 src has 1..97) v4 ==== 9474+0+0 (secure 0 0 0) 0x7fe424004400 con 0x55e428380290 2024-03-21T10:26:17.688 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:17.685+0000 7fe43d9f3740 1 -- 172.21.15.47:0/4085614069 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fe42c005560 con 0x55e428380290 2024-03-21T10:26:17.688 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:17.686+0000 7fe41d600640 1 -- 172.21.15.47:0/4085614069 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe424067050 con 0x55e428380290 2024-03-21T10:26:17.688 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:17.686+0000 7fe43d9f3740 10 client.35927 fetch_fsmap learned FSMap version 32 2024-03-21T10:26:17.688 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:17.686+0000 7fe43d9f3740 1 -- 172.21.15.47:0/4085614069 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x55e4283fe8d0 con 0x55e428380290 2024-03-21T10:26:17.688 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:17.686+0000 7fe41d600640 1 -- 172.21.15.47:0/4085614069 <== mon.2 v2:172.21.15.100:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fe42c005560 con 0x55e428380290 2024-03-21T10:26:17.688 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:17.686+0000 7fe41d600640 1 -- 172.21.15.47:0/4085614069 <== mon.2 v2:172.21.15.100:3300/0 5 ==== fsmap(e 32) v1 ==== 7942+0+0 (secure 0 0 0) 0x7fe42403f900 con 0x55e428380290 2024-03-21T10:26:17.688 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:17.687+0000 7fe43d9f3740 10 client.35927 fetch_fsmap finished waiting for FSMap version 32 2024-03-21T10:26:17.688 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:17.687+0000 7fe43d9f3740 10 client.35927 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-21T10:26:17.688 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:17.687+0000 7fe43d9f3740 20 client.35927 populate_metadata read hostname 'smithi047' 2024-03-21T10:26:17.688 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:17.687+0000 7fe43d9f3740 1 --2- 172.21.15.47:0/4085614069 >> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] conn(0x55e4283fefc0 0x55e4284013e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:26:17.688 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:17.687+0000 7fe43d9f3740 4 client.35927 mds_command: new command op to 34535 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-03-21T10:26:17.688 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:17.687+0000 7fe43d9f3740 1 -- 172.21.15.47:0/4085614069 --> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x55e4282ec740 con 0x55e4283fefc0 2024-03-21T10:26:17.689 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:17.687+0000 7fe433400640 1 --2- 172.21.15.47:0/4085614069 >> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] conn(0x55e4283fefc0 0x55e4284013e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:26:17.689 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:17.688+0000 7fe433400640 1 --2- 172.21.15.47:0/4085614069 >> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] conn(0x55e4283fefc0 0x55e4284013e0 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8836f15efd67b9ba server_cookie=acbc757c5e9b69dd in_seq=0 out_seq=0 2024-03-21T10:26:17.689 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:17.688+0000 7fe41d600640 10 client.35927 ms_handle_connect on v2:172.21.15.100:6838/4173168440 2024-03-21T10:26:17.690 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:17.690+0000 7fe41d600640 1 -- 172.21.15.47:0/4085614069 <== mds.0 v2:172.21.15.100:6838/4173168440 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x55e4282ec740 con 0x55e4283fefc0 2024-03-21T10:26:17.690 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:17.690+0000 7fe41d600640 10 client.35927 handle_command_reply: tid=0 2024-03-21T10:26:17.690 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:17.690+0000 7fe43d9f3740 1 client.35927 shutdown 2024-03-21T10:26:17.690 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:17.690+0000 7fe43d9f3740 1 -- 172.21.15.47:0/4085614069 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e428380290 msgr2=0x55e4283a5880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:26:17.691 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:17.690+0000 7fe43d9f3740 1 --2- 172.21.15.47:0/4085614069 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e428380290 0x55e4283a5880 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fe424006010 tx=0x7fe424034c10 comp rx=0 tx=0).stop 2024-03-21T10:26:17.691 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:17.690+0000 7fe43d9f3740 1 -- 172.21.15.47:0/4085614069 shutdown_connections 2024-03-21T10:26:17.691 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:17.690+0000 7fe43d9f3740 1 --2- 172.21.15.47:0/4085614069 >> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] conn(0x55e4283fefc0 0x55e4284013e0 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:26:17.692 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:17.690+0000 7fe41d600640 0 client.35927 ms_handle_reset on v2:172.21.15.100:6838/4173168440 2024-03-21T10:26:17.692 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:17.690+0000 7fe43d9f3740 1 --2- 172.21.15.47:0/4085614069 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e428389790 0x55e4283a6300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:26:17.692 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:17.690+0000 7fe43d9f3740 1 --2- 172.21.15.47:0/4085614069 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e428380bb0 0x55e4283a5dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:26:17.692 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:17.690+0000 7fe43d9f3740 1 --2- 172.21.15.47:0/4085614069 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e428380290 0x55e4283a5880 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:26:17.692 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:17.690+0000 7fe43d9f3740 1 -- 172.21.15.47:0/4085614069 >> 172.21.15.47:0/4085614069 conn(0x55e42837a9c0 msgr2=0x55e428381960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:26:17.692 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:17.691+0000 7fe43d9f3740 1 -- 172.21.15.47:0/4085614069 shutdown_connections 2024-03-21T10:26:17.692 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:17.691+0000 7fe43d9f3740 1 -- 172.21.15.47:0/4085614069 wait complete. 2024-03-21T10:26:17.692 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:17.691+0000 7fe43d9f3740 20 client.35927 trim_cache size 0 max 16384 2024-03-21T10:26:17.692 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-03-21T10:26:17.693 INFO:teuthology.orchestra.run.smithi047.stdout: "return_code": 0, 2024-03-21T10:26:17.693 INFO:teuthology.orchestra.run.smithi047.stdout: "scrub_tag": "b2f16d03-ed21-4368-aadf-b29c35bab8ee", 2024-03-21T10:26:17.693 INFO:teuthology.orchestra.run.smithi047.stdout: "mode": "asynchronous" 2024-03-21T10:26:17.693 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-03-21T10:26:17.693 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:17.693+0000 7fe439600640 1 -- 172.21.15.47:0/810055140 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7fe4180a32e0 msgr2=0x7fe4180a57a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:26:17.693 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:17.693+0000 7fe439600640 1 --2- 172.21.15.47:0/810055140 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7fe4180a32e0 0x7fe4180a57a0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x55e42837bb50 tx=0x7fe42000b650 comp rx=0 tx=0).stop 2024-03-21T10:26:17.693 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:17.693+0000 7fe439600640 1 -- 172.21.15.47:0/810055140 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe4341075e0 msgr2=0x7fe434074420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:26:17.693 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:17.693+0000 7fe439600640 1 --2- 172.21.15.47:0/810055140 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe4341075e0 0x7fe434074420 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7fe42c00eaa0 tx=0x7fe42c00ef60 comp rx=0 tx=0).stop 2024-03-21T10:26:17.694 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:17.694+0000 7fe439600640 1 -- 172.21.15.47:0/810055140 shutdown_connections 2024-03-21T10:26:17.694 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:17.694+0000 7fe439600640 1 --2- 172.21.15.47:0/810055140 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe4341075e0 0x7fe434074420 unknown :-1 s=CLOSED pgs=612 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:26:17.694 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:17.694+0000 7fe439600640 1 --2- 172.21.15.47:0/810055140 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe434101ee0 0x7fe434073ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:26:17.694 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:17.694+0000 7fe439600640 1 --2- 172.21.15.47:0/810055140 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7fe4180a32e0 0x7fe4180a57a0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:26:17.694 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:17.694+0000 7fe439600640 1 --2- 172.21.15.47:0/810055140 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe434101520 0x7fe4340739a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:26:17.695 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:17.694+0000 7fe439600640 1 -- 172.21.15.47:0/810055140 >> 172.21.15.47:0/810055140 conn(0x7fe4340fd010 msgr2=0x7fe4340fe820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:26:17.695 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:17.694+0000 7fe439600640 1 -- 172.21.15.47:0/810055140 shutdown_connections 2024-03-21T10:26:17.695 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:17.694+0000 7fe439600640 1 -- 172.21.15.47:0/810055140 wait complete. 2024-03-21T10:26:17.716 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-21T10:26:17.716 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-21T10:26:17.942 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:17.941+0000 7fb227200640 1 -- 172.21.15.47:0/4218330338 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb228102ff0 msgr2=0x7fb22810b4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:26:17.942 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:17.941+0000 7fb227200640 1 --2- 172.21.15.47:0/4218330338 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb228102ff0 0x7fb22810b4d0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fb214009900 tx=0x7fb21402bcd0 comp rx=0 tx=0).stop 2024-03-21T10:26:17.942 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:17.941+0000 7fb227200640 1 -- 172.21.15.47:0/4218330338 shutdown_connections 2024-03-21T10:26:17.942 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:17.941+0000 7fb227200640 1 --2- 172.21.15.47:0/4218330338 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb22810baf0 0x7fb22810df90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:26:17.942 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:17.941+0000 7fb227200640 1 --2- 172.21.15.47:0/4218330338 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb228102ff0 0x7fb22810b4d0 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:26:17.942 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:17.941+0000 7fb227200640 1 --2- 172.21.15.47:0/4218330338 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb2281026d0 0x7fb228102ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:26:17.942 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:17.941+0000 7fb227200640 1 -- 172.21.15.47:0/4218330338 >> 172.21.15.47:0/4218330338 conn(0x7fb2280fbef0 msgr2=0x7fb2280fe310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:26:17.942 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:17.941+0000 7fb227200640 1 -- 172.21.15.47:0/4218330338 shutdown_connections 2024-03-21T10:26:17.943 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:17.941+0000 7fb227200640 1 -- 172.21.15.47:0/4218330338 wait complete. 2024-03-21T10:26:17.943 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:17.943+0000 7fb227200640 1 Processor -- start 2024-03-21T10:26:17.944 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:17.943+0000 7fb227200640 1 -- start start 2024-03-21T10:26:17.944 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:17.943+0000 7fb227200640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb2281026d0 0x7fb22810a870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:26:17.944 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:17.943+0000 7fb227200640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb228102ff0 0x7fb22810adb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:26:17.944 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:17.943+0000 7fb227200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb22810baf0 0x7fb2281a2b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:26:17.944 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:17.943+0000 7fb227200640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb228111760 con 0x7fb228102ff0 2024-03-21T10:26:17.944 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:17.943+0000 7fb227200640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2281115e0 con 0x7fb22810baf0 2024-03-21T10:26:17.944 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:17.943+0000 7fb227200640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2281118e0 con 0x7fb2281026d0 2024-03-21T10:26:17.945 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:17.944+0000 7fb225400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb228102ff0 0x7fb22810adb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:26:17.945 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:17.944+0000 7fb226800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb22810baf0 0x7fb2281a2b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:26:17.945 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:17.944+0000 7fb225400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb228102ff0 0x7fb22810adb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:58820/0 (socket says 172.21.15.47:58820) 2024-03-21T10:26:17.945 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:17.944+0000 7fb225400640 1 -- 172.21.15.47:0/4086140512 learned_addr learned my addr 172.21.15.47:0/4086140512 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:26:17.945 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:17.944+0000 7fb225e00640 1 --2- 172.21.15.47:0/4086140512 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb2281026d0 0x7fb22810a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:26:17.945 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:17.944+0000 7fb225400640 1 -- 172.21.15.47:0/4086140512 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb2281026d0 msgr2=0x7fb22810a870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:26:17.945 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:17.944+0000 7fb225400640 1 --2- 172.21.15.47:0/4086140512 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb2281026d0 0x7fb22810a870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:26:17.946 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:17.944+0000 7fb225400640 1 -- 172.21.15.47:0/4086140512 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb22810baf0 msgr2=0x7fb2281a2b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:26:17.946 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:17.944+0000 7fb225400640 1 --2- 172.21.15.47:0/4086140512 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb22810baf0 0x7fb2281a2b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:26:17.946 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:17.944+0000 7fb225400640 1 -- 172.21.15.47:0/4086140512 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2281a3290 con 0x7fb228102ff0 2024-03-21T10:26:17.946 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:17.944+0000 7fb226800640 1 --2- 172.21.15.47:0/4086140512 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb22810baf0 0x7fb2281a2b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T10:26:17.947 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:17.945+0000 7fb225e00640 1 --2- 172.21.15.47:0/4086140512 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb2281026d0 0x7fb22810a870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T10:26:17.947 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:17.945+0000 7fb225400640 1 --2- 172.21.15.47:0/4086140512 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb228102ff0 0x7fb22810adb0 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7fb214005940 tx=0x7fb21402c690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:26:17.947 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:17.945+0000 7fb21ea00640 1 -- 172.21.15.47:0/4086140512 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb214043020 con 0x7fb228102ff0 2024-03-21T10:26:17.947 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:17.945+0000 7fb21ea00640 1 -- 172.21.15.47:0/4086140512 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fb214004e00 con 0x7fb228102ff0 2024-03-21T10:26:17.947 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:17.945+0000 7fb227200640 1 -- 172.21.15.47:0/4086140512 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb2281a3520 con 0x7fb228102ff0 2024-03-21T10:26:17.947 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:17.946+0000 7fb21ea00640 1 -- 172.21.15.47:0/4086140512 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2140038c0 con 0x7fb228102ff0 2024-03-21T10:26:17.947 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:17.946+0000 7fb227200640 1 -- 172.21.15.47:0/4086140512 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb2281aae10 con 0x7fb228102ff0 2024-03-21T10:26:17.952 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:17.950+0000 7fb22f7af740 1 Processor -- start 2024-03-21T10:26:17.952 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:17.950+0000 7fb22f7af740 1 -- start start 2024-03-21T10:26:17.952 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:17.951+0000 7fb21ea00640 1 -- 172.21.15.47:0/4086140512 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 26) v1 ==== 138087+0+0 (secure 0 0 0) 0x7fb214046020 con 0x7fb228102ff0 2024-03-21T10:26:17.952 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:17.951+0000 7fb22f7af740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d1879cddd0 0x55d1879ccb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:26:17.952 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:17.951+0000 7fb22f7af740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d1879cd0f0 0x55d1879cd570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:26:17.952 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:17.951+0000 7fb22f7af740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d1879e7470 0x55d1879e9860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:26:17.952 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:17.951+0000 7fb22f7af740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d187931250 con 0x55d1879e7470 2024-03-21T10:26:17.952 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:17.951+0000 7fb22f7af740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d1878d9a30 con 0x55d1879cd0f0 2024-03-21T10:26:17.952 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:17.951+0000 7fb22f7af740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d18792d2f0 con 0x55d1879cddd0 2024-03-21T10:26:17.953 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:17.951+0000 7fb225e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d1879cd0f0 0x55d1879cd570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:26:17.953 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:17.952+0000 7fb225e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d1879cd0f0 0x55d1879cd570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:41590/0 (socket says 172.21.15.47:41590) 2024-03-21T10:26:17.953 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:17.952+0000 7fb225e00640 1 -- 172.21.15.47:0/4034700206 learned_addr learned my addr 172.21.15.47:0/4034700206 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:26:17.954 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:17.952+0000 7fb225400640 1 --2- 172.21.15.47:0/4034700206 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d1879e7470 0x55d1879e9860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:26:17.954 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:17.952+0000 7fb226800640 1 --2- 172.21.15.47:0/4034700206 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d1879cddd0 0x55d1879ccb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:26:17.954 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:17.952+0000 7fb225e00640 1 -- 172.21.15.47:0/4034700206 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d1879cddd0 msgr2=0x55d1879ccb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:26:17.954 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:17.952+0000 7fb225e00640 1 --2- 172.21.15.47:0/4034700206 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d1879cddd0 0x55d1879ccb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:26:17.954 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:17.952+0000 7fb225e00640 1 -- 172.21.15.47:0/4034700206 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d1879e7470 msgr2=0x55d1879e9860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:26:17.954 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:17.952+0000 7fb225e00640 1 --2- 172.21.15.47:0/4034700206 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d1879e7470 0x55d1879e9860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:26:17.954 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:17.952+0000 7fb225e00640 1 -- 172.21.15.47:0/4034700206 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d1878c4690 con 0x55d1879cd0f0 2024-03-21T10:26:17.954 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:17.952+0000 7fb225e00640 1 --2- 172.21.15.47:0/4034700206 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d1879cd0f0 0x55d1879cd570 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fb218009d50 tx=0x7fb2180117b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7a616ba4ba405279 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:26:17.954 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:17.953+0000 7fb213000640 1 -- 172.21.15.47:0/4034700206 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb218012650 con 0x55d1879cd0f0 2024-03-21T10:26:17.954 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:17.953+0000 7fb213000640 1 -- 172.21.15.47:0/4034700206 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fb218009e40 con 0x55d1879cd0f0 2024-03-21T10:26:17.954 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:17.953+0000 7fb213000640 1 -- 172.21.15.47:0/4034700206 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb21800fc00 con 0x55d1879cd0f0 2024-03-21T10:26:17.954 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:17.953+0000 7fb21ea00640 1 --2- 172.21.15.47:0/4086140512 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7fb20c0a32b0 0x7fb20c0a5770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:26:17.954 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:17.953+0000 7fb21ea00640 1 -- 172.21.15.47:0/4086140512 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 9474+0+0 (secure 0 0 0) 0x7fb2140f1c10 con 0x7fb228102ff0 2024-03-21T10:26:17.956 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:17.954+0000 7fb22f7af740 1 -- 172.21.15.47:0/4034700206 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d1879cd0f0 msgr2=0x55d1879cd570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:26:17.956 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:17.954+0000 7fb22f7af740 1 --2- 172.21.15.47:0/4034700206 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d1879cd0f0 0x55d1879cd570 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fb218009d50 tx=0x7fb2180117b0 comp rx=0 tx=0).stop 2024-03-21T10:26:17.956 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:17.954+0000 7fb226800640 1 --2- 172.21.15.47:0/4086140512 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7fb20c0a32b0 0x7fb20c0a5770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:26:17.956 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:17.954+0000 7fb22f7af740 1 -- 172.21.15.47:0/4034700206 shutdown_connections 2024-03-21T10:26:17.956 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:17.954+0000 7fb22f7af740 1 --2- 172.21.15.47:0/4034700206 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d1879e7470 0x55d1879e9860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:26:17.956 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:17.954+0000 7fb22f7af740 1 --2- 172.21.15.47:0/4034700206 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d1879cd0f0 0x55d1879cd570 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:26:17.956 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:17.954+0000 7fb22f7af740 1 --2- 172.21.15.47:0/4034700206 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d1879cddd0 0x55d1879ccb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:26:17.956 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:17.954+0000 7fb22f7af740 1 -- 172.21.15.47:0/4034700206 >> 172.21.15.47:0/4034700206 conn(0x55d187994200 msgr2=0x55d1879c7b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:26:17.957 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:17.954+0000 7fb22f7af740 1 -- 172.21.15.47:0/4034700206 shutdown_connections 2024-03-21T10:26:17.957 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:17.954+0000 7fb22f7af740 1 -- 172.21.15.47:0/4034700206 wait complete. 2024-03-21T10:26:17.957 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:17.954+0000 7fb22f7af740 1 -- start start 2024-03-21T10:26:17.957 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:17.955+0000 7fb22f7af740 1 Processor -- start 2024-03-21T10:26:17.957 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:17.955+0000 7fb22f7af740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d1879cd0f0 0x55d1879e6ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:26:17.957 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:17.955+0000 7fb22f7af740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d1879cddd0 0x55d1879e3e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:26:17.957 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:17.955+0000 7fb22f7af740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d1879e7470 0x55d1879e43a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:26:17.957 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:17.955+0000 7fb22f7af740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d1879ea2e0 con 0x55d1879cd0f0 2024-03-21T10:26:17.957 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:17.955+0000 7fb226800640 1 --2- 172.21.15.47:0/4086140512 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7fb20c0a32b0 0x7fb20c0a5770 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fb220007fd0 tx=0x7fb220007520 comp rx=0 tx=0).ready entity=mgr.15687 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:26:17.957 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:17.955+0000 7fb22f7af740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d1878ece80 con 0x55d1879e7470 2024-03-21T10:26:17.957 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:17.955+0000 7fb22f7af740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d1879ea460 con 0x55d1879cddd0 2024-03-21T10:26:17.957 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:17.955+0000 7fb22f7af740 10 client.0 _ll_register_callbacks cb 0x55d1878ff130 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T10:26:17.957 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:17.956+0000 7fb226800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d1879e7470 0x55d1879e43a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:26:17.957 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:17.956+0000 7fb226800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d1879e7470 0x55d1879e43a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:41602/0 (socket says 172.21.15.47:41602) 2024-03-21T10:26:17.958 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:17.956+0000 7fb226800640 1 -- 172.21.15.47:0/2249989632 learned_addr learned my addr 172.21.15.47:0/2249989632 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:26:17.958 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:17.956+0000 7fb225e00640 1 --2- 172.21.15.47:0/2249989632 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d1879cd0f0 0x55d1879e6ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:26:17.958 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:17.956+0000 7fb225400640 1 --2- 172.21.15.47:0/2249989632 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d1879cddd0 0x55d1879e3e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:26:17.960 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:17.956+0000 7fb226800640 1 -- 172.21.15.47:0/2249989632 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d1879cddd0 msgr2=0x55d1879e3e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:26:17.960 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:17.956+0000 7fb226800640 1 --2- 172.21.15.47:0/2249989632 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d1879cddd0 0x55d1879e3e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:26:17.960 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:17.956+0000 7fb226800640 1 -- 172.21.15.47:0/2249989632 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d1879cd0f0 msgr2=0x55d1879e6ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:26:17.960 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:17.956+0000 7fb226800640 1 --2- 172.21.15.47:0/2249989632 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d1879cd0f0 0x55d1879e6ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:26:17.960 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:17.956+0000 7fb226800640 1 -- 172.21.15.47:0/2249989632 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55d18770f7d0 con 0x55d1879e7470 2024-03-21T10:26:17.960 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:17.956+0000 7fb225e00640 1 --2- 172.21.15.47:0/2249989632 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d1879cd0f0 0x55d1879e6ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T10:26:17.960 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:17.957+0000 7fb226800640 1 --2- 172.21.15.47:0/2249989632 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d1879e7470 0x55d1879e43a0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fb22000ff20 tx=0x7fb220004a40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9b2db46bbe8bd5f0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:26:17.960 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:17.957+0000 7fb207400640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-21T10:26:17.960 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:17.957+0000 7fb207400640 1 -- 172.21.15.47:0/2249989632 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb22002a060 con 0x55d1879e7470 2024-03-21T10:26:17.960 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:17.957+0000 7fb22f7af740 1 -- 172.21.15.47:0/2249989632 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55d18793e790 con 0x55d1879e7470 2024-03-21T10:26:17.960 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:17.957+0000 7fb207400640 1 -- 172.21.15.47:0/2249989632 <== mon.2 v2:172.21.15.100:3300/0 2 ==== osd_map(97..97 src has 1..97) v4 ==== 9474+0+0 (secure 0 0 0) 0x7fb22001b430 con 0x55d1879e7470 2024-03-21T10:26:17.960 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:17.957+0000 7fb207400640 1 -- 172.21.15.47:0/2249989632 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb220031050 con 0x55d1879e7470 2024-03-21T10:26:17.960 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:17.957+0000 7fb22f7af740 10 client.35939 fetch_fsmap learned FSMap version 32 2024-03-21T10:26:17.961 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:17.957+0000 7fb22f7af740 1 -- 172.21.15.47:0/2249989632 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x55d1876453f0 con 0x55d1879e7470 2024-03-21T10:26:17.961 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:17.958+0000 7fb207400640 1 -- 172.21.15.47:0/2249989632 <== mon.2 v2:172.21.15.100:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x55d18793e790 con 0x55d1879e7470 2024-03-21T10:26:17.961 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:17.958+0000 7fb207400640 1 -- 172.21.15.47:0/2249989632 <== mon.2 v2:172.21.15.100:3300/0 5 ==== fsmap(e 32) v1 ==== 7942+0+0 (secure 0 0 0) 0x7fb220024030 con 0x55d1879e7470 2024-03-21T10:26:17.961 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:17.958+0000 7fb22f7af740 10 client.35939 fetch_fsmap finished waiting for FSMap version 32 2024-03-21T10:26:17.961 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:17.958+0000 7fb22f7af740 10 client.35939 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-21T10:26:17.961 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:17.958+0000 7fb22f7af740 20 client.35939 populate_metadata read hostname 'smithi047' 2024-03-21T10:26:17.961 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:17.958+0000 7fb22f7af740 1 --2- 172.21.15.47:0/2249989632 >> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] conn(0x55d187a083a0 0x55d187a0a7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:26:17.961 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:17.958+0000 7fb22f7af740 4 client.35939 mds_command: new command op to 34535 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T10:26:17.961 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:17.958+0000 7fb22f7af740 1 -- 172.21.15.47:0/2249989632 --> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55d187938ae0 con 0x55d187a083a0 2024-03-21T10:26:17.961 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:17.959+0000 7fb225e00640 1 --2- 172.21.15.47:0/2249989632 >> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] conn(0x55d187a083a0 0x55d187a0a7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:26:17.961 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:17.960+0000 7fb225e00640 1 --2- 172.21.15.47:0/2249989632 >> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] conn(0x55d187a083a0 0x55d187a0a7f0 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=84819b229fdbd2ef server_cookie=fc12a6b868b86f88 in_seq=0 out_seq=0 2024-03-21T10:26:17.961 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:17.960+0000 7fb207400640 10 client.35939 ms_handle_connect on v2:172.21.15.100:6838/4173168440 2024-03-21T10:26:17.962 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:17.961+0000 7fb207400640 1 -- 172.21.15.47:0/2249989632 <== mds.0 v2:172.21.15.100:6838/4173168440 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55d187938ae0 con 0x55d187a083a0 2024-03-21T10:26:17.962 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:17.961+0000 7fb207400640 10 client.35939 handle_command_reply: tid=0 2024-03-21T10:26:17.962 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:17.961+0000 7fb22f7af740 1 client.35939 shutdown 2024-03-21T10:26:17.962 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:17.962+0000 7fb22f7af740 1 -- 172.21.15.47:0/2249989632 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d1879e7470 msgr2=0x55d1879e43a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:26:17.962 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:17.962+0000 7fb22f7af740 1 --2- 172.21.15.47:0/2249989632 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d1879e7470 0x55d1879e43a0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fb22000ff20 tx=0x7fb220004a40 comp rx=0 tx=0).stop 2024-03-21T10:26:17.963 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:17.962+0000 7fb22f7af740 1 -- 172.21.15.47:0/2249989632 shutdown_connections 2024-03-21T10:26:17.963 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:17.962+0000 7fb22f7af740 1 --2- 172.21.15.47:0/2249989632 >> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] conn(0x55d187a083a0 0x55d187a0a7f0 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:26:17.963 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:17.962+0000 7fb22f7af740 1 --2- 172.21.15.47:0/2249989632 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d1879e7470 0x55d1879e43a0 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:26:17.963 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:17.962+0000 7fb22f7af740 1 --2- 172.21.15.47:0/2249989632 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d1879cddd0 0x55d1879e3e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:26:17.963 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:17.962+0000 7fb22f7af740 1 --2- 172.21.15.47:0/2249989632 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d1879cd0f0 0x55d1879e6ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:26:17.963 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:17.962+0000 7fb22f7af740 1 -- 172.21.15.47:0/2249989632 >> 172.21.15.47:0/2249989632 conn(0x55d187994200 msgr2=0x55d1879c9890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:26:17.963 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:17.962+0000 7fb207400640 0 client.35939 ms_handle_reset on v2:172.21.15.100:6838/4173168440 2024-03-21T10:26:17.963 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:17.962+0000 7fb22f7af740 1 -- 172.21.15.47:0/2249989632 shutdown_connections 2024-03-21T10:26:17.963 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:17.962+0000 7fb22f7af740 1 -- 172.21.15.47:0/2249989632 wait complete. 2024-03-21T10:26:17.963 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:17.962+0000 7fb22f7af740 20 client.35939 trim_cache size 0 max 16384 2024-03-21T10:26:17.985 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:17.984+0000 7fb22f7af740 1 Processor -- start 2024-03-21T10:26:17.985 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:17.984+0000 7fb22f7af740 1 -- start start 2024-03-21T10:26:17.985 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:17.984+0000 7fb22f7af740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d1879cd0f0 0x55d187a0a3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:26:17.985 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:17.984+0000 7fb22f7af740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d1879cddd0 0x55d1879e8860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:26:17.985 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:17.984+0000 7fb22f7af740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d1879e7470 0x55d1879e8da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:26:17.985 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:17.984+0000 7fb22f7af740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb20800fd30 con 0x55d1879cd0f0 2024-03-21T10:26:17.985 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:17.984+0000 7fb22f7af740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2080116b0 con 0x55d1879e7470 2024-03-21T10:26:17.985 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:17.984+0000 7fb22f7af740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb208010390 con 0x55d1879cddd0 2024-03-21T10:26:17.987 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:17.985+0000 7fb225e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d1879cd0f0 0x55d187a0a3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:26:17.987 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:17.985+0000 7fb225e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d1879cd0f0 0x55d187a0a3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:58840/0 (socket says 172.21.15.47:58840) 2024-03-21T10:26:17.987 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:17.985+0000 7fb225e00640 1 -- 172.21.15.47:0/3341545293 learned_addr learned my addr 172.21.15.47:0/3341545293 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:26:17.987 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:17.985+0000 7fb226800640 1 --2- 172.21.15.47:0/3341545293 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d1879e7470 0x55d1879e8da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:26:17.987 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:17.985+0000 7fb225400640 1 --2- 172.21.15.47:0/3341545293 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d1879cddd0 0x55d1879e8860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:26:17.987 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:17.985+0000 7fb226800640 1 -- 172.21.15.47:0/3341545293 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d1879cddd0 msgr2=0x55d1879e8860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:26:17.987 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:17.985+0000 7fb226800640 1 --2- 172.21.15.47:0/3341545293 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d1879cddd0 0x55d1879e8860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:26:17.987 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:17.985+0000 7fb226800640 1 -- 172.21.15.47:0/3341545293 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d1879cd0f0 msgr2=0x55d187a0a3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:26:17.987 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:17.985+0000 7fb226800640 1 --2- 172.21.15.47:0/3341545293 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d1879cd0f0 0x55d187a0a3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:26:17.987 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:17.985+0000 7fb226800640 1 -- 172.21.15.47:0/3341545293 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d1876050f0 con 0x55d1879e7470 2024-03-21T10:26:17.987 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:17.985+0000 7fb225e00640 1 --2- 172.21.15.47:0/3341545293 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d1879cd0f0 0x55d187a0a3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T10:26:17.988 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:17.986+0000 7fb226800640 1 --2- 172.21.15.47:0/3341545293 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d1879e7470 0x55d1879e8da0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fb220010010 tx=0x7fb220000f40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d8c6d337e7922509 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:26:17.988 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:17.986+0000 7fb212600640 1 -- 172.21.15.47:0/3341545293 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb22003a070 con 0x55d1879e7470 2024-03-21T10:26:17.988 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:17.986+0000 7fb212600640 1 -- 172.21.15.47:0/3341545293 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fb22000fae0 con 0x55d1879e7470 2024-03-21T10:26:17.988 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:17.986+0000 7fb212600640 1 -- 172.21.15.47:0/3341545293 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb220004040 con 0x55d1879e7470 2024-03-21T10:26:17.989 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:17.987+0000 7fb22f7af740 1 -- 172.21.15.47:0/3341545293 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d1879e7470 msgr2=0x55d1879e8da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:26:17.989 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:17.987+0000 7fb22f7af740 1 --2- 172.21.15.47:0/3341545293 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d1879e7470 0x55d1879e8da0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fb220010010 tx=0x7fb220000f40 comp rx=0 tx=0).stop 2024-03-21T10:26:17.989 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:17.987+0000 7fb22f7af740 1 -- 172.21.15.47:0/3341545293 shutdown_connections 2024-03-21T10:26:17.989 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:17.987+0000 7fb22f7af740 1 --2- 172.21.15.47:0/3341545293 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d1879e7470 0x55d1879e8da0 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:26:17.989 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:17.987+0000 7fb22f7af740 1 --2- 172.21.15.47:0/3341545293 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d1879cddd0 0x55d1879e8860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:26:17.989 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:17.987+0000 7fb22f7af740 1 --2- 172.21.15.47:0/3341545293 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d1879cd0f0 0x55d187a0a3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:26:17.989 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:17.987+0000 7fb22f7af740 1 -- 172.21.15.47:0/3341545293 >> 172.21.15.47:0/3341545293 conn(0x55d187994200 msgr2=0x55d1879c7670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:26:17.989 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:17.987+0000 7fb22f7af740 1 -- 172.21.15.47:0/3341545293 shutdown_connections 2024-03-21T10:26:17.989 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:17.988+0000 7fb22f7af740 1 -- 172.21.15.47:0/3341545293 wait complete. 2024-03-21T10:26:17.989 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:17.988+0000 7fb22f7af740 1 -- start start 2024-03-21T10:26:17.989 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:17.988+0000 7fb22f7af740 1 Processor -- start 2024-03-21T10:26:17.989 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:17.988+0000 7fb22f7af740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d1879cd0f0 0x55d1879f2d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:26:17.989 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:17.988+0000 7fb22f7af740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d1879cddd0 0x55d1879f32d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:26:17.990 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:17.989+0000 7fb22f7af740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d1879e7470 0x55d1879e1340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:26:17.990 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:17.989+0000 7fb22f7af740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb208011050 con 0x55d1879e7470 2024-03-21T10:26:17.990 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:17.989+0000 7fb22f7af740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2080109f0 con 0x55d1879cd0f0 2024-03-21T10:26:17.990 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:17.989+0000 7fb22f7af740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb20800def0 con 0x55d1879cddd0 2024-03-21T10:26:17.991 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:17.989+0000 7fb22f7af740 10 client.0 _ll_register_callbacks cb 0x7fb220007eb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T10:26:17.991 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:17.989+0000 7fb226800640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d1879e7470 0x55d1879e1340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:26:17.991 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:17.989+0000 7fb226800640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d1879e7470 0x55d1879e1340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:58854/0 (socket says 172.21.15.47:58854) 2024-03-21T10:26:17.991 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:17.989+0000 7fb226800640 1 -- 172.21.15.47:0/3033281743 learned_addr learned my addr 172.21.15.47:0/3033281743 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:26:17.991 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:17.989+0000 7fb225e00640 1 --2- 172.21.15.47:0/3033281743 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d1879cd0f0 0x55d1879f2d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:26:17.991 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:17.989+0000 7fb225400640 1 --2- 172.21.15.47:0/3033281743 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d1879cddd0 0x55d1879f32d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:26:17.991 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:17.990+0000 7fb226800640 1 -- 172.21.15.47:0/3033281743 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d1879cddd0 msgr2=0x55d1879f32d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:26:17.991 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:17.990+0000 7fb226800640 1 --2- 172.21.15.47:0/3033281743 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d1879cddd0 0x55d1879f32d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:26:17.991 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:17.990+0000 7fb226800640 1 -- 172.21.15.47:0/3033281743 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d1879cd0f0 msgr2=0x55d1879f2d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:26:17.991 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:17.990+0000 7fb226800640 1 --2- 172.21.15.47:0/3033281743 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d1879cd0f0 0x55d1879f2d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:26:17.991 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:17.990+0000 7fb226800640 1 -- 172.21.15.47:0/3033281743 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55d1879e1b70 con 0x55d1879e7470 2024-03-21T10:26:17.992 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:17.990+0000 7fb226800640 1 --2- 172.21.15.47:0/3033281743 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d1879e7470 0x55d1879e1340 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7fb220007e80 tx=0x7fb220025b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3a1c13b32a334b4b server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:26:17.993 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:17.991+0000 7fb207400640 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-03-21T10:26:17.993 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:17.991+0000 7fb207400640 1 -- 172.21.15.47:0/3033281743 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb22003a070 con 0x55d1879e7470 2024-03-21T10:26:17.993 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:17.991+0000 7fb22f7af740 1 -- 172.21.15.47:0/3033281743 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55d187a0ab80 con 0x55d1879e7470 2024-03-21T10:26:17.993 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:17.991+0000 7fb207400640 1 -- 172.21.15.47:0/3033281743 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(97..97 src has 1..97) v4 ==== 9474+0+0 (secure 0 0 0) 0x7fb220003d20 con 0x55d1879e7470 2024-03-21T10:26:17.993 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:17.991+0000 7fb207400640 1 -- 172.21.15.47:0/3033281743 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb22002a030 con 0x55d1879e7470 2024-03-21T10:26:17.993 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:17.991+0000 7fb22f7af740 10 client.16140 fetch_fsmap learned FSMap version 32 2024-03-21T10:26:17.993 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:17.991+0000 7fb22f7af740 1 -- 172.21.15.47:0/3033281743 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x55d1879f2970 con 0x55d1879e7470 2024-03-21T10:26:17.993 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:17.992+0000 7fb207400640 1 -- 172.21.15.47:0/3033281743 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x55d187a0ab80 con 0x55d1879e7470 2024-03-21T10:26:17.993 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:17.992+0000 7fb207400640 1 -- 172.21.15.47:0/3033281743 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 32) v1 ==== 7942+0+0 (secure 0 0 0) 0x7fb220024070 con 0x55d1879e7470 2024-03-21T10:26:17.993 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:17.992+0000 7fb22f7af740 10 client.16140 fetch_fsmap finished waiting for FSMap version 32 2024-03-21T10:26:17.993 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:17.992+0000 7fb22f7af740 10 client.16140 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-21T10:26:17.993 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:17.992+0000 7fb22f7af740 20 client.16140 populate_metadata read hostname 'smithi047' 2024-03-21T10:26:17.993 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:17.992+0000 7fb22f7af740 1 --2- 172.21.15.47:0/3033281743 >> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] conn(0x55d187a63fc0 0x55d187a663e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:26:17.993 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:17.992+0000 7fb22f7af740 4 client.16140 mds_command: new command op to 34535 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-21T10:26:17.994 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:17.992+0000 7fb22f7af740 1 -- 172.21.15.47:0/3033281743 --> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55d1878b63a0 con 0x55d187a63fc0 2024-03-21T10:26:17.994 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:17.993+0000 7fb225e00640 1 --2- 172.21.15.47:0/3033281743 >> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] conn(0x55d187a63fc0 0x55d187a663e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:26:17.994 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:17.994+0000 7fb207400640 10 client.16140 ms_handle_connect on v2:172.21.15.100:6838/4173168440 2024-03-21T10:26:17.995 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:17.994+0000 7fb225e00640 1 --2- 172.21.15.47:0/3033281743 >> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] conn(0x55d187a63fc0 0x55d187a663e0 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ef27b02b70a2f686 server_cookie=e420b325ad23b57a in_seq=0 out_seq=0 2024-03-21T10:26:17.995 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:17.994+0000 7fb207400640 1 -- 172.21.15.47:0/3033281743 <== mds.0 v2:172.21.15.100:6838/4173168440 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x55d1878b63a0 con 0x55d187a63fc0 2024-03-21T10:26:17.995 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:17.994+0000 7fb207400640 10 client.16140 handle_command_reply: tid=0 2024-03-21T10:26:17.995 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:17.994+0000 7fb22f7af740 1 client.16140 shutdown 2024-03-21T10:26:17.995 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:17.994+0000 7fb22f7af740 1 -- 172.21.15.47:0/3033281743 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d1879e7470 msgr2=0x55d1879e1340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:26:17.995 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:17.994+0000 7fb22f7af740 1 --2- 172.21.15.47:0/3033281743 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d1879e7470 0x55d1879e1340 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7fb220007e80 tx=0x7fb220025b20 comp rx=0 tx=0).stop 2024-03-21T10:26:17.996 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:17.995+0000 7fb22f7af740 1 -- 172.21.15.47:0/3033281743 shutdown_connections 2024-03-21T10:26:17.996 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:17.995+0000 7fb22f7af740 1 --2- 172.21.15.47:0/3033281743 >> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] conn(0x55d187a63fc0 0x55d187a663e0 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:26:17.996 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:17.995+0000 7fb22f7af740 1 --2- 172.21.15.47:0/3033281743 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d1879e7470 0x55d1879e1340 unknown :-1 s=CLOSED pgs=616 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:26:17.996 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:17.995+0000 7fb22f7af740 1 --2- 172.21.15.47:0/3033281743 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d1879cddd0 0x55d1879f32d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:26:17.996 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:17.995+0000 7fb22f7af740 1 --2- 172.21.15.47:0/3033281743 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d1879cd0f0 0x55d1879f2d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:26:17.996 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:17.995+0000 7fb22f7af740 1 -- 172.21.15.47:0/3033281743 >> 172.21.15.47:0/3033281743 conn(0x55d187994200 msgr2=0x55d1879e2680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:26:17.996 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:17.995+0000 7fb207400640 0 client.16140 ms_handle_reset on v2:172.21.15.100:6838/4173168440 2024-03-21T10:26:17.996 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:17.995+0000 7fb22f7af740 1 -- 172.21.15.47:0/3033281743 shutdown_connections 2024-03-21T10:26:17.996 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:17.995+0000 7fb22f7af740 1 -- 172.21.15.47:0/3033281743 wait complete. 2024-03-21T10:26:17.996 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:17.995+0000 7fb22f7af740 20 client.16140 trim_cache size 0 max 16384 2024-03-21T10:26:17.997 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-03-21T10:26:17.997 INFO:teuthology.orchestra.run.smithi047.stdout: "status": "IDLE (waiting for more scrubs)", 2024-03-21T10:26:17.997 INFO:teuthology.orchestra.run.smithi047.stdout: "scrubs": { 2024-03-21T10:26:17.997 INFO:teuthology.orchestra.run.smithi047.stdout: "b2f16d03-ed21-4368-aadf-b29c35bab8ee": { 2024-03-21T10:26:17.997 INFO:teuthology.orchestra.run.smithi047.stdout: "path": "/", 2024-03-21T10:26:17.997 INFO:teuthology.orchestra.run.smithi047.stdout: "tag": "b2f16d03-ed21-4368-aadf-b29c35bab8ee", 2024-03-21T10:26:17.997 INFO:teuthology.orchestra.run.smithi047.stdout: "options": "recursive,force" 2024-03-21T10:26:17.997 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-21T10:26:17.997 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-21T10:26:17.997 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-03-21T10:26:17.998 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:17.996+0000 7fb227200640 1 -- 172.21.15.47:0/4086140512 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7fb20c0a32b0 msgr2=0x7fb20c0a5770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:26:17.998 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:17.996+0000 7fb227200640 1 --2- 172.21.15.47:0/4086140512 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7fb20c0a32b0 0x7fb20c0a5770 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fb220007fd0 tx=0x7fb220007520 comp rx=0 tx=0).stop 2024-03-21T10:26:17.998 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:17.996+0000 7fb227200640 1 -- 172.21.15.47:0/4086140512 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb228102ff0 msgr2=0x7fb22810adb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:26:17.998 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:17.996+0000 7fb227200640 1 --2- 172.21.15.47:0/4086140512 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb228102ff0 0x7fb22810adb0 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7fb214005940 tx=0x7fb21402c690 comp rx=0 tx=0).stop 2024-03-21T10:26:17.998 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:17.996+0000 7fb227200640 1 -- 172.21.15.47:0/4086140512 shutdown_connections 2024-03-21T10:26:17.998 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:17.996+0000 7fb227200640 1 --2- 172.21.15.47:0/4086140512 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7fb20c0a32b0 0x7fb20c0a5770 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:26:17.998 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:17.996+0000 7fb227200640 1 --2- 172.21.15.47:0/4086140512 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb22810baf0 0x7fb2281a2b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:26:17.999 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:17.996+0000 7fb227200640 1 --2- 172.21.15.47:0/4086140512 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb228102ff0 0x7fb22810adb0 unknown :-1 s=CLOSED pgs=615 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:26:17.999 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:17.996+0000 7fb227200640 1 --2- 172.21.15.47:0/4086140512 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb2281026d0 0x7fb22810a870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:26:17.999 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:17.996+0000 7fb227200640 1 -- 172.21.15.47:0/4086140512 >> 172.21.15.47:0/4086140512 conn(0x7fb2280fbef0 msgr2=0x7fb2280fe2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:26:17.999 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:17.997+0000 7fb227200640 1 -- 172.21.15.47:0/4086140512 shutdown_connections 2024-03-21T10:26:17.999 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:17.997+0000 7fb227200640 1 -- 172.21.15.47:0/4086140512 wait complete. 2024-03-21T10:26:18.019 INFO:tasks.cephfs.filesystem:scrub status for tag:b2f16d03-ed21-4368-aadf-b29c35bab8ee - {'path': '/', 'tag': 'b2f16d03-ed21-4368-aadf-b29c35bab8ee', 'options': 'recursive,force'} 2024-03-21T10:26:18.463 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:26:18 smithi047 ceph-mon[31547]: osdmap e97: 12 total, 12 up, 12 in 2024-03-21T10:26:18.463 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:26:18 smithi047 ceph-mon[31547]: mds.? [v2:172.21.15.157:6838/1998408416,v1:172.21.15.157:6839/1998408416] up:boot 2024-03-21T10:26:18.463 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:26:18 smithi047 ceph-mon[31547]: fsmap cephfs:5 4 up:standby-replay 3 up:standby 5 up:active 2024-03-21T10:26:18.463 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:26:18 smithi047 ceph-mon[31547]: from='mgr.15687 172.21.15.47:0/8730661' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-03-21T10:26:18.463 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:26:18 smithi047 ceph-mon[31547]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-21T10:26:18.479 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:26:18 smithi100 ceph-mon[35193]: osdmap e97: 12 total, 12 up, 12 in 2024-03-21T10:26:18.479 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:26:18 smithi100 ceph-mon[35193]: mds.? [v2:172.21.15.157:6838/1998408416,v1:172.21.15.157:6839/1998408416] up:boot 2024-03-21T10:26:18.479 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:26:18 smithi100 ceph-mon[35193]: fsmap cephfs:5 4 up:standby-replay 3 up:standby 5 up:active 2024-03-21T10:26:18.479 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:26:18 smithi100 ceph-mon[35193]: from='mgr.15687 172.21.15.47:0/8730661' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-03-21T10:26:18.479 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:26:18 smithi100 ceph-mon[35193]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-21T10:26:18.489 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:26:18 smithi157 ceph-mon[35581]: osdmap e97: 12 total, 12 up, 12 in 2024-03-21T10:26:18.489 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:26:18 smithi157 ceph-mon[35581]: mds.? [v2:172.21.15.157:6838/1998408416,v1:172.21.15.157:6839/1998408416] up:boot 2024-03-21T10:26:18.489 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:26:18 smithi157 ceph-mon[35581]: fsmap cephfs:5 4 up:standby-replay 3 up:standby 5 up:active 2024-03-21T10:26:18.489 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:26:18 smithi157 ceph-mon[35581]: from='mgr.15687 172.21.15.47:0/8730661' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-03-21T10:26:18.489 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:26:18 smithi157 ceph-mon[35581]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-21T10:26:19.463 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:26:19 smithi047 ceph-mon[31547]: scrub queued for path: / 2024-03-21T10:26:19.463 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:26:19 smithi047 ceph-mon[31547]: scrub summary: idle+waiting paths [/] 2024-03-21T10:26:19.463 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:26:19 smithi047 ceph-mon[31547]: scrub summary: active paths [/] 2024-03-21T10:26:19.463 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:26:19 smithi047 ceph-mon[31547]: scrub summary: idle+waiting paths [/] 2024-03-21T10:26:19.463 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:26:19 smithi047 ceph-mon[31547]: pgmap v33: 129 pgs: 129 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 7 op/s 2024-03-21T10:26:19.463 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:26:19 smithi047 ceph-mon[31547]: scrub summary: idle 2024-03-21T10:26:19.479 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:26:19 smithi100 ceph-mon[35193]: scrub queued for path: / 2024-03-21T10:26:19.479 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:26:19 smithi100 ceph-mon[35193]: scrub summary: idle+waiting paths [/] 2024-03-21T10:26:19.479 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:26:19 smithi100 ceph-mon[35193]: scrub summary: active paths [/] 2024-03-21T10:26:19.479 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:26:19 smithi100 ceph-mon[35193]: scrub summary: idle+waiting paths [/] 2024-03-21T10:26:19.479 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:26:19 smithi100 ceph-mon[35193]: pgmap v33: 129 pgs: 129 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 7 op/s 2024-03-21T10:26:19.479 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:26:19 smithi100 ceph-mon[35193]: scrub summary: idle 2024-03-21T10:26:19.489 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:26:19 smithi157 ceph-mon[35581]: scrub queued for path: / 2024-03-21T10:26:19.489 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:26:19 smithi157 ceph-mon[35581]: scrub summary: idle+waiting paths [/] 2024-03-21T10:26:19.489 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:26:19 smithi157 ceph-mon[35581]: scrub summary: active paths [/] 2024-03-21T10:26:19.490 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:26:19 smithi157 ceph-mon[35581]: scrub summary: idle+waiting paths [/] 2024-03-21T10:26:19.490 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:26:19 smithi157 ceph-mon[35581]: pgmap v33: 129 pgs: 129 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 7 op/s 2024-03-21T10:26:19.490 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:26:19 smithi157 ceph-mon[35581]: scrub summary: idle 2024-03-21T10:26:21.463 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:26:21 smithi047 ceph-mon[31547]: pgmap v34: 129 pgs: 129 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 7 op/s 2024-03-21T10:26:21.478 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:26:21 smithi100 ceph-mon[35193]: pgmap v34: 129 pgs: 129 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 7 op/s 2024-03-21T10:26:21.489 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:26:21 smithi157 ceph-mon[35581]: pgmap v34: 129 pgs: 129 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 7 op/s 2024-03-21T10:26:23.463 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:26:23 smithi047 ceph-mon[31547]: pgmap v35: 129 pgs: 129 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 307 B/s wr, 11 op/s 2024-03-21T10:26:23.478 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:26:23 smithi100 ceph-mon[35193]: pgmap v35: 129 pgs: 129 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 307 B/s wr, 11 op/s 2024-03-21T10:26:23.489 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:26:23 smithi157 ceph-mon[35581]: pgmap v35: 129 pgs: 129 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 307 B/s wr, 11 op/s 2024-03-21T10:26:25.463 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:26:25 smithi047 ceph-mon[31547]: pgmap v36: 129 pgs: 129 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 307 B/s wr, 11 op/s 2024-03-21T10:26:25.478 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:26:25 smithi100 ceph-mon[35193]: pgmap v36: 129 pgs: 129 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 307 B/s wr, 11 op/s 2024-03-21T10:26:25.489 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:26:25 smithi157 ceph-mon[35581]: pgmap v36: 129 pgs: 129 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 307 B/s wr, 11 op/s 2024-03-21T10:26:27.463 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:26:27 smithi047 ceph-mon[31547]: pgmap v37: 129 pgs: 129 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 307 B/s wr, 12 op/s 2024-03-21T10:26:27.478 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:26:27 smithi100 ceph-mon[35193]: pgmap v37: 129 pgs: 129 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 307 B/s wr, 12 op/s 2024-03-21T10:26:27.489 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:26:27 smithi157 ceph-mon[35581]: pgmap v37: 129 pgs: 129 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 307 B/s wr, 12 op/s 2024-03-21T10:26:29.463 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:26:29 smithi047 ceph-mon[31547]: pgmap v38: 129 pgs: 129 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 283 B/s wr, 11 op/s 2024-03-21T10:26:29.478 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:26:29 smithi100 ceph-mon[35193]: pgmap v38: 129 pgs: 129 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 283 B/s wr, 11 op/s 2024-03-21T10:26:29.489 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:26:29 smithi157 ceph-mon[35581]: pgmap v38: 129 pgs: 129 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 283 B/s wr, 11 op/s 2024-03-21T10:26:29.968 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:29.966+0000 7f025b200640 1 -- 172.21.15.47:0/884625304 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f02541056c0 msgr2=0x7f025410d290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:26:29.968 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:29.966+0000 7f025b200640 1 --2- 172.21.15.47:0/884625304 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f02541056c0 0x7f025410d290 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f0250009900 tx=0x7f025002bc60 comp rx=0 tx=0).stop 2024-03-21T10:26:29.969 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:29.967+0000 7f025b200640 1 -- 172.21.15.47:0/884625304 shutdown_connections 2024-03-21T10:26:29.969 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:29.967+0000 7f025b200640 1 --2- 172.21.15.47:0/884625304 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f02541056c0 0x7f025410d290 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:26:29.969 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:29.967+0000 7f025b200640 1 --2- 172.21.15.47:0/884625304 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0254100fa0 0x7f0254104fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:26:29.969 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:29.967+0000 7f025b200640 1 --2- 172.21.15.47:0/884625304 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f02541005f0 0x7f02541009d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:26:29.969 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:29.967+0000 7f025b200640 1 -- 172.21.15.47:0/884625304 >> 172.21.15.47:0/884625304 conn(0x7f02540fbeb0 msgr2=0x7f02540fe2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:26:29.969 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:29.968+0000 7f025b200640 1 -- 172.21.15.47:0/884625304 shutdown_connections 2024-03-21T10:26:29.969 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:29.968+0000 7f025b200640 1 -- 172.21.15.47:0/884625304 wait complete. 2024-03-21T10:26:29.970 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:29.969+0000 7f025b200640 1 Processor -- start 2024-03-21T10:26:29.970 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:29.969+0000 7f025b200640 1 -- start start 2024-03-21T10:26:29.971 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:29.970+0000 7f025b200640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f02541005f0 0x7f025419da70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:26:29.971 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:29.970+0000 7f025b200640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f02541056c0 0x7f025419dfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:26:29.971 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:29.970+0000 7f025b200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0254197b80 0x7f0254198000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:26:29.971 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:29.970+0000 7f025b200640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02541117d0 con 0x7f02541056c0 2024-03-21T10:26:29.972 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:29.970+0000 7f025b200640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0254111650 con 0x7f0254197b80 2024-03-21T10:26:29.972 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:29.970+0000 7f025b200640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0254111950 con 0x7f02541005f0 2024-03-21T10:26:29.972 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:29.971+0000 7f0259e00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f02541005f0 0x7f025419da70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:26:29.972 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:29.971+0000 7f0259e00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f02541005f0 0x7f025419da70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.47:47354/0 (socket says 172.21.15.47:47354) 2024-03-21T10:26:29.972 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:29.971+0000 7f0259e00640 1 -- 172.21.15.47:0/1157795786 learned_addr learned my addr 172.21.15.47:0/1157795786 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:26:29.972 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:29.971+0000 7f0259400640 1 --2- 172.21.15.47:0/1157795786 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f02541056c0 0x7f025419dfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:26:29.973 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:29.971+0000 7f0259e00640 1 -- 172.21.15.47:0/1157795786 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0254197b80 msgr2=0x7f0254198000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:26:29.973 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:29.971+0000 7f025a800640 1 --2- 172.21.15.47:0/1157795786 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0254197b80 0x7f0254198000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:26:29.973 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:29.972+0000 7f0259e00640 1 --2- 172.21.15.47:0/1157795786 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0254197b80 0x7f0254198000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:26:29.973 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:29.972+0000 7f0259e00640 1 -- 172.21.15.47:0/1157795786 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f02541056c0 msgr2=0x7f025419dfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:26:29.973 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:29.972+0000 7f0259e00640 1 --2- 172.21.15.47:0/1157795786 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f02541056c0 0x7f025419dfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:26:29.973 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:29.972+0000 7f0259e00640 1 -- 172.21.15.47:0/1157795786 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0254198780 con 0x7f02541005f0 2024-03-21T10:26:29.974 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:29.972+0000 7f025a800640 1 --2- 172.21.15.47:0/1157795786 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0254197b80 0x7f0254198000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T10:26:29.974 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:29.972+0000 7f0259400640 1 --2- 172.21.15.47:0/1157795786 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f02541056c0 0x7f025419dfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T10:26:29.974 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:29.972+0000 7f0259e00640 1 --2- 172.21.15.47:0/1157795786 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f02541005f0 0x7f025419da70 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f024800ba50 tx=0x7f024800bf10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:26:29.974 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:29.973+0000 7f024ea00640 1 -- 172.21.15.47:0/1157795786 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0248019070 con 0x7f02541005f0 2024-03-21T10:26:29.974 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:29.973+0000 7f025b200640 1 -- 172.21.15.47:0/1157795786 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f02541a4810 con 0x7f02541005f0 2024-03-21T10:26:29.974 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:29.973+0000 7f025b200640 1 -- 172.21.15.47:0/1157795786 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f02541a4c70 con 0x7f02541005f0 2024-03-21T10:26:29.975 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:29.973+0000 7f024ea00640 1 -- 172.21.15.47:0/1157795786 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f024800cc10 con 0x7f02541005f0 2024-03-21T10:26:29.976 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:29.973+0000 7f024ea00640 1 -- 172.21.15.47:0/1157795786 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02480075d0 con 0x7f02541005f0 2024-03-21T10:26:29.979 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:29.978+0000 7f025f639740 1 Processor -- start 2024-03-21T10:26:29.980 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:29.978+0000 7f025f639740 1 -- start start 2024-03-21T10:26:29.980 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:29.979+0000 7f025f639740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fb6ec91860 0x55fb6ec91c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:26:29.980 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:29.979+0000 7f025f639740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55fb6ec92210 0x55fb6ec7e5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:26:29.981 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:29.979+0000 7f024ea00640 1 -- 172.21.15.47:0/1157795786 <== mon.1 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 26) v1 ==== 138087+0+0 (secure 0 0 0) 0x7f024801c070 con 0x7f02541005f0 2024-03-21T10:26:29.981 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:29.980+0000 7f025f639740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fb6ec7ebd0 0x55fb6ec7f030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:26:29.981 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:29.980+0000 7f025f639740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fb6ec463b0 con 0x55fb6ec92210 2024-03-21T10:26:29.981 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:29.980+0000 7f025f639740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fb6ebe8830 con 0x55fb6ec91860 2024-03-21T10:26:29.981 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:29.980+0000 7f025f639740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fb6eb915d0 con 0x55fb6ec7ebd0 2024-03-21T10:26:29.981 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:29.980+0000 7f025a800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fb6ec91860 0x55fb6ec91c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:26:29.982 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:29.980+0000 7f0259400640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fb6ec7ebd0 0x55fb6ec7f030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:26:29.982 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:29.980+0000 7f0259400640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fb6ec7ebd0 0x55fb6ec7f030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.47:47364/0 (socket says 172.21.15.47:47364) 2024-03-21T10:26:29.982 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:29.981+0000 7f0259400640 1 -- 172.21.15.47:0/3375465117 learned_addr learned my addr 172.21.15.47:0/3375465117 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:26:29.983 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:29.981+0000 7f0259400640 1 -- 172.21.15.47:0/3375465117 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fb6ec91860 msgr2=0x55fb6ec91c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:26:29.983 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:29.981+0000 7f0259400640 1 --2- 172.21.15.47:0/3375465117 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fb6ec91860 0x55fb6ec91c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:26:29.983 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:29.981+0000 7f0259400640 1 -- 172.21.15.47:0/3375465117 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55fb6ec92210 msgr2=0x55fb6ec7e5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:26:29.983 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:29.981+0000 7f0259400640 1 --2- 172.21.15.47:0/3375465117 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55fb6ec92210 0x55fb6ec7e5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:26:29.983 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:29.981+0000 7f0259400640 1 -- 172.21.15.47:0/3375465117 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55fb6eba92c0 con 0x55fb6ec7ebd0 2024-03-21T10:26:29.986 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:29.983+0000 7f024ea00640 1 --2- 172.21.15.47:0/1157795786 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f023c0a32a0 0x7f023c0a5760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:26:29.986 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:29.983+0000 7f0259400640 1 --2- 172.21.15.47:0/3375465117 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fb6ec7ebd0 0x55fb6ec7f030 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f0244010480 tx=0x7f024400bf90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=82129b961f010c53 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:26:29.986 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:29.983+0000 7f024ea00640 1 -- 172.21.15.47:0/1157795786 <== mon.1 v2:172.21.15.157:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 9474+0+0 (secure 0 0 0) 0x7f02480cf710 con 0x7f02541005f0 2024-03-21T10:26:29.986 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:29.983+0000 7f0243000640 1 -- 172.21.15.47:0/3375465117 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0244011760 con 0x55fb6ec7ebd0 2024-03-21T10:26:29.986 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:29.983+0000 7f0243000640 1 -- 172.21.15.47:0/3375465117 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f0244011db0 con 0x55fb6ec7ebd0 2024-03-21T10:26:29.986 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:29.984+0000 7f025a800640 1 --2- 172.21.15.47:0/1157795786 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f023c0a32a0 0x7f023c0a5760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:26:29.987 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:29.984+0000 7f025f639740 1 -- 172.21.15.47:0/3375465117 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fb6ec7ebd0 msgr2=0x55fb6ec7f030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:26:29.987 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:29.985+0000 7f0243000640 1 -- 172.21.15.47:0/3375465117 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0244011760 con 0x55fb6ec7ebd0 2024-03-21T10:26:29.987 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:29.985+0000 7f025f639740 1 --2- 172.21.15.47:0/3375465117 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fb6ec7ebd0 0x55fb6ec7f030 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f0244010480 tx=0x7f024400bf90 comp rx=0 tx=0).stop 2024-03-21T10:26:29.988 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:29.985+0000 7f025a800640 1 --2- 172.21.15.47:0/1157795786 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f023c0a32a0 0x7f023c0a5760 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f0250009230 tx=0x7f0250005fd0 comp rx=0 tx=0).ready entity=mgr.15687 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:26:29.988 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:29.985+0000 7f025f639740 1 -- 172.21.15.47:0/3375465117 shutdown_connections 2024-03-21T10:26:29.988 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:29.985+0000 7f025f639740 1 --2- 172.21.15.47:0/3375465117 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fb6ec7ebd0 0x55fb6ec7f030 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:26:29.988 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:29.985+0000 7f025f639740 1 --2- 172.21.15.47:0/3375465117 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55fb6ec92210 0x55fb6ec7e5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:26:29.988 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:29.985+0000 7f025f639740 1 --2- 172.21.15.47:0/3375465117 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fb6ec91860 0x55fb6ec91c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:26:29.988 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:29.985+0000 7f025f639740 1 -- 172.21.15.47:0/3375465117 >> 172.21.15.47:0/3375465117 conn(0x55fb6ec81070 msgr2=0x55fb6ec81450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:26:29.988 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:29.986+0000 7f025f639740 1 -- 172.21.15.47:0/3375465117 shutdown_connections 2024-03-21T10:26:29.988 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:29.986+0000 7f025f639740 1 -- 172.21.15.47:0/3375465117 wait complete. 2024-03-21T10:26:29.989 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:29.986+0000 7f025f639740 1 -- start start 2024-03-21T10:26:29.989 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:29.987+0000 7f025f639740 1 Processor -- start 2024-03-21T10:26:29.989 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:29.987+0000 7f025f639740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fb6ec7ebd0 0x55fb6ec95730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:26:29.990 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:29.987+0000 7f025f639740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fb6ec91860 0x55fb6ec95c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:26:29.990 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:29.988+0000 7f025f639740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55fb6ec92210 0x55fb6ec961b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:26:29.990 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:29.988+0000 7f025f639740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fb6eba4a20 con 0x55fb6ec92210 2024-03-21T10:26:29.990 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:29.988+0000 7f025f639740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fb6ebe48d0 con 0x55fb6ec91860 2024-03-21T10:26:29.990 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:29.988+0000 7f025f639740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fb6ec98cf0 con 0x55fb6ec7ebd0 2024-03-21T10:26:29.990 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:29.988+0000 7f025f639740 10 client.0 _ll_register_callbacks cb 0x55fb6ebe11c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T10:26:29.990 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:29.988+0000 7f025a800640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55fb6ec92210 0x55fb6ec961b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:26:29.990 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:29.988+0000 7f0259e00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fb6ec7ebd0 0x55fb6ec95730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:26:29.990 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:29.989+0000 7f025a800640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55fb6ec92210 0x55fb6ec961b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:53030/0 (socket says 172.21.15.47:53030) 2024-03-21T10:26:29.990 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:29.989+0000 7f025a800640 1 -- 172.21.15.47:0/4246051894 learned_addr learned my addr 172.21.15.47:0/4246051894 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:26:29.990 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:29.989+0000 7f0259e00640 1 --2- 172.21.15.47:0/4246051894 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fb6ec7ebd0 0x55fb6ec95730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.47:47372/0 (socket says 172.21.15.47:47372) 2024-03-21T10:26:29.991 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:29.989+0000 7f0259400640 1 --2- 172.21.15.47:0/4246051894 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fb6ec91860 0x55fb6ec95c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:26:29.991 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:29.989+0000 7f025a800640 1 -- 172.21.15.47:0/4246051894 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fb6ec7ebd0 msgr2=0x55fb6ec95730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:26:29.991 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:29.989+0000 7f025a800640 1 --2- 172.21.15.47:0/4246051894 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fb6ec7ebd0 0x55fb6ec95730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:26:29.991 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:29.989+0000 7f025a800640 1 -- 172.21.15.47:0/4246051894 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fb6ec91860 msgr2=0x55fb6ec95c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:26:29.991 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:29.989+0000 7f025a800640 1 --2- 172.21.15.47:0/4246051894 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fb6ec91860 0x55fb6ec95c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:26:29.991 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:29.989+0000 7f025a800640 1 -- 172.21.15.47:0/4246051894 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55fb6e8fd210 con 0x55fb6ec92210 2024-03-21T10:26:29.992 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:29.990+0000 7f025a800640 1 --2- 172.21.15.47:0/4246051894 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55fb6ec92210 0x55fb6ec961b0 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7f0250035000 tx=0x7f025002bc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9aec10511498d684 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:26:29.992 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:29.990+0000 7f0237400640 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-03-21T10:26:29.992 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:29.990+0000 7f0237400640 1 -- 172.21.15.47:0/4246051894 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f025004f020 con 0x55fb6ec92210 2024-03-21T10:26:29.993 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:29.990+0000 7f0237400640 1 -- 172.21.15.47:0/4246051894 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(97..97 src has 1..97) v4 ==== 9474+0+0 (secure 0 0 0) 0x7f02500048d0 con 0x55fb6ec92210 2024-03-21T10:26:29.993 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:29.990+0000 7f025f639740 1 -- 172.21.15.47:0/4246051894 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f0244011db0 con 0x55fb6ec92210 2024-03-21T10:26:29.993 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:29.991+0000 7f0237400640 1 -- 172.21.15.47:0/4246051894 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f025005a050 con 0x55fb6ec92210 2024-03-21T10:26:29.993 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:29.991+0000 7f025f639740 10 client.16149 fetch_fsmap learned FSMap version 32 2024-03-21T10:26:29.993 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:29.991+0000 7f025f639740 1 -- 172.21.15.47:0/4246051894 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x55fb6e9c9620 con 0x55fb6ec92210 2024-03-21T10:26:29.993 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:29.991+0000 7f0237400640 1 -- 172.21.15.47:0/4246051894 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f0244011db0 con 0x55fb6ec92210 2024-03-21T10:26:29.993 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:29.992+0000 7f0237400640 1 -- 172.21.15.47:0/4246051894 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 32) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f0250003ca0 con 0x55fb6ec92210 2024-03-21T10:26:29.993 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:29.992+0000 7f025f639740 10 client.16149 fetch_fsmap finished waiting for FSMap version 32 2024-03-21T10:26:29.993 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:29.992+0000 7f025f639740 10 client.16149 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-21T10:26:29.993 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:29.992+0000 7f025f639740 20 client.16149 populate_metadata read hostname 'smithi047' 2024-03-21T10:26:29.994 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:29.992+0000 7f025f639740 1 --2- 172.21.15.47:0/4246051894 >> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] conn(0x55fb6eca6250 0x55fb6eca8680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:26:29.994 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:29.992+0000 7f025f639740 4 client.16149 mds_command: new command op to 34535 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T10:26:29.994 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:29.992+0000 7f025f639740 1 -- 172.21.15.47:0/4246051894 --> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55fb6ebf5d70 con 0x55fb6eca6250 2024-03-21T10:26:29.994 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:29.993+0000 7f0259e00640 1 --2- 172.21.15.47:0/4246051894 >> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] conn(0x55fb6eca6250 0x55fb6eca8680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:26:29.994 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:29.993+0000 7f0259e00640 1 --2- 172.21.15.47:0/4246051894 >> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] conn(0x55fb6eca6250 0x55fb6eca8680 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=74d2a8c01140c793 server_cookie=2973b397b0591e1c in_seq=0 out_seq=0 2024-03-21T10:26:29.994 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:29.993+0000 7f0237400640 10 client.16149 ms_handle_connect on v2:172.21.15.100:6838/4173168440 2024-03-21T10:26:29.995 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:29.994+0000 7f0237400640 1 -- 172.21.15.47:0/4246051894 <== mds.0 v2:172.21.15.100:6838/4173168440 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55fb6ebf5d70 con 0x55fb6eca6250 2024-03-21T10:26:29.996 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:29.994+0000 7f0237400640 10 client.16149 handle_command_reply: tid=0 2024-03-21T10:26:29.996 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:29.994+0000 7f025f639740 1 client.16149 shutdown 2024-03-21T10:26:29.997 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:29.995+0000 7f025f639740 1 -- 172.21.15.47:0/4246051894 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55fb6ec92210 msgr2=0x55fb6ec961b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:26:29.997 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:29.995+0000 7f025f639740 1 --2- 172.21.15.47:0/4246051894 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55fb6ec92210 0x55fb6ec961b0 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7f0250035000 tx=0x7f025002bc60 comp rx=0 tx=0).stop 2024-03-21T10:26:29.997 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:29.995+0000 7f025f639740 1 -- 172.21.15.47:0/4246051894 shutdown_connections 2024-03-21T10:26:29.997 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:29.995+0000 7f025f639740 1 --2- 172.21.15.47:0/4246051894 >> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] conn(0x55fb6eca6250 0x55fb6eca8680 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:26:29.997 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:29.995+0000 7f025f639740 1 --2- 172.21.15.47:0/4246051894 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55fb6ec92210 0x55fb6ec961b0 unknown :-1 s=CLOSED pgs=617 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:26:29.997 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:29.995+0000 7f025f639740 1 --2- 172.21.15.47:0/4246051894 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fb6ec91860 0x55fb6ec95c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:26:29.997 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:29.995+0000 7f025f639740 1 --2- 172.21.15.47:0/4246051894 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fb6ec7ebd0 0x55fb6ec95730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:26:29.997 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:29.995+0000 7f025f639740 1 -- 172.21.15.47:0/4246051894 >> 172.21.15.47:0/4246051894 conn(0x55fb6ec81070 msgr2=0x55fb6ec807d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:26:29.997 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:29.995+0000 7f0237400640 0 client.16149 ms_handle_reset on v2:172.21.15.100:6838/4173168440 2024-03-21T10:26:29.997 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:29.995+0000 7f025f639740 1 -- 172.21.15.47:0/4246051894 shutdown_connections 2024-03-21T10:26:29.997 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:29.996+0000 7f025f639740 1 -- 172.21.15.47:0/4246051894 wait complete. 2024-03-21T10:26:29.997 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:29.996+0000 7f025f639740 20 client.16149 trim_cache size 0 max 16384 2024-03-21T10:26:30.027 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:30.025+0000 7f025f639740 1 Processor -- start 2024-03-21T10:26:30.027 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:30.025+0000 7f025f639740 1 -- start start 2024-03-21T10:26:30.027 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:30.026+0000 7f025f639740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fb6ec7ebd0 0x55fb6ec90f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:26:30.027 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:30.026+0000 7f025f639740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fb6ec91860 0x55fb6ec96e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:26:30.028 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:30.026+0000 7f025f639740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55fb6ec92210 0x55fb6ec97390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:26:30.028 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:30.026+0000 7f025f639740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f023800fd80 con 0x55fb6ec92210 2024-03-21T10:26:30.028 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:30.026+0000 7f025f639740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0238011700 con 0x55fb6ec91860 2024-03-21T10:26:30.028 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:30.026+0000 7f025f639740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02380103e0 con 0x55fb6ec7ebd0 2024-03-21T10:26:30.029 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:30.026+0000 7f0259e00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fb6ec7ebd0 0x55fb6ec90f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:26:30.030 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:30.026+0000 7f0259e00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fb6ec7ebd0 0x55fb6ec90f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.47:47380/0 (socket says 172.21.15.47:47380) 2024-03-21T10:26:30.030 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:30.026+0000 7f0259e00640 1 -- 172.21.15.47:0/1038480622 learned_addr learned my addr 172.21.15.47:0/1038480622 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:26:30.030 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:30.026+0000 7f0259400640 1 --2- 172.21.15.47:0/1038480622 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fb6ec91860 0x55fb6ec96e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:26:30.030 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:30.027+0000 7f025a800640 1 --2- 172.21.15.47:0/1038480622 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55fb6ec92210 0x55fb6ec97390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:26:30.030 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:30.027+0000 7f0259400640 1 -- 172.21.15.47:0/1038480622 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fb6ec7ebd0 msgr2=0x55fb6ec90f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:26:30.030 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:30.027+0000 7f0259400640 1 --2- 172.21.15.47:0/1038480622 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fb6ec7ebd0 0x55fb6ec90f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:26:30.030 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:30.027+0000 7f0259400640 1 -- 172.21.15.47:0/1038480622 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55fb6ec92210 msgr2=0x55fb6ec97390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:26:30.030 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:30.027+0000 7f0259400640 1 --2- 172.21.15.47:0/1038480622 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55fb6ec92210 0x55fb6ec97390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:26:30.030 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:30.027+0000 7f0259400640 1 -- 172.21.15.47:0/1038480622 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55fb6ec93ce0 con 0x55fb6ec91860 2024-03-21T10:26:30.030 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:30.027+0000 7f025a800640 1 --2- 172.21.15.47:0/1038480622 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55fb6ec92210 0x55fb6ec97390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T10:26:30.031 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:30.028+0000 7f0259400640 1 --2- 172.21.15.47:0/1038480622 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fb6ec91860 0x55fb6ec96e50 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f024401aa30 tx=0x7f024401ab10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5180079012e239d2 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:26:30.031 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:30.029+0000 7f0242600640 1 -- 172.21.15.47:0/1038480622 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f024402c070 con 0x55fb6ec91860 2024-03-21T10:26:30.032 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:30.029+0000 7f0242600640 1 -- 172.21.15.47:0/1038480622 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f024401a830 con 0x55fb6ec91860 2024-03-21T10:26:30.032 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:30.029+0000 7f0242600640 1 -- 172.21.15.47:0/1038480622 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02440048c0 con 0x55fb6ec91860 2024-03-21T10:26:30.032 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:30.030+0000 7f025f639740 1 -- 172.21.15.47:0/1038480622 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fb6ec91860 msgr2=0x55fb6ec96e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:26:30.032 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:30.030+0000 7f025f639740 1 --2- 172.21.15.47:0/1038480622 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fb6ec91860 0x55fb6ec96e50 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f024401aa30 tx=0x7f024401ab10 comp rx=0 tx=0).stop 2024-03-21T10:26:30.032 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:30.030+0000 7f025f639740 1 -- 172.21.15.47:0/1038480622 shutdown_connections 2024-03-21T10:26:30.032 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:30.030+0000 7f025f639740 1 --2- 172.21.15.47:0/1038480622 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55fb6ec92210 0x55fb6ec97390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:26:30.032 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:30.030+0000 7f025f639740 1 --2- 172.21.15.47:0/1038480622 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fb6ec91860 0x55fb6ec96e50 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:26:30.032 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:30.030+0000 7f025f639740 1 --2- 172.21.15.47:0/1038480622 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fb6ec7ebd0 0x55fb6ec90f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:26:30.033 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:30.030+0000 7f025f639740 1 -- 172.21.15.47:0/1038480622 >> 172.21.15.47:0/1038480622 conn(0x55fb6ec81070 msgr2=0x55fb6ec90820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:26:30.033 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:30.030+0000 7f025f639740 1 -- 172.21.15.47:0/1038480622 shutdown_connections 2024-03-21T10:26:30.033 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:30.030+0000 7f025f639740 1 -- 172.21.15.47:0/1038480622 wait complete. 2024-03-21T10:26:30.033 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:30.031+0000 7f025f639740 1 -- start start 2024-03-21T10:26:30.034 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:30.031+0000 7f025f639740 1 Processor -- start 2024-03-21T10:26:30.034 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:30.032+0000 7f025f639740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fb6ec7ebd0 0x55fb6ec8eaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:26:30.034 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:30.032+0000 7f025f639740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fb6ec92210 0x55fb6eca1160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:26:30.035 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:30.032+0000 7f025f639740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55fb6eca16a0 0x55fb6eca1b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:26:30.035 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:30.032+0000 7f0259e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fb6ec7ebd0 0x55fb6ec8eaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:26:30.035 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:30.032+0000 7f025f639740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02380110a0 con 0x55fb6eca16a0 2024-03-21T10:26:30.035 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:30.032+0000 7f025f639740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0238010a40 con 0x55fb6ec7ebd0 2024-03-21T10:26:30.035 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:30.032+0000 7f025f639740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f023800df40 con 0x55fb6ec92210 2024-03-21T10:26:30.035 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:30.032+0000 7f025f639740 10 client.0 _ll_register_callbacks cb 0x7f0250009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T10:26:30.035 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:30.033+0000 7f0259400640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fb6ec92210 0x55fb6eca1160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:26:30.035 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:30.033+0000 7f0259e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fb6ec7ebd0 0x55fb6ec8eaf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:60512/0 (socket says 172.21.15.47:60512) 2024-03-21T10:26:30.035 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:30.033+0000 7f0259e00640 1 -- 172.21.15.47:0/1386262291 learned_addr learned my addr 172.21.15.47:0/1386262291 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:26:30.035 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:30.033+0000 7f025a800640 1 --2- 172.21.15.47:0/1386262291 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55fb6eca16a0 0x55fb6eca1b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:26:30.035 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:30.033+0000 7f0259e00640 1 -- 172.21.15.47:0/1386262291 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fb6ec92210 msgr2=0x55fb6eca1160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:26:30.035 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:30.033+0000 7f0259e00640 1 --2- 172.21.15.47:0/1386262291 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fb6ec92210 0x55fb6eca1160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:26:30.036 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:30.033+0000 7f0259e00640 1 -- 172.21.15.47:0/1386262291 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55fb6eca16a0 msgr2=0x55fb6eca1b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:26:30.036 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:30.033+0000 7f0259e00640 1 --2- 172.21.15.47:0/1386262291 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55fb6eca16a0 0x55fb6eca1b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:26:30.036 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:30.033+0000 7f0259e00640 1 -- 172.21.15.47:0/1386262291 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55fb6ec8d2a0 con 0x55fb6ec7ebd0 2024-03-21T10:26:30.038 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:30.034+0000 7f0259e00640 1 --2- 172.21.15.47:0/1386262291 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fb6ec7ebd0 0x55fb6ec8eaf0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f024809b350 tx=0x7f02480972f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=994bf38bad977342 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:26:30.038 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:30.034+0000 7f0237400640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-21T10:26:30.038 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:30.034+0000 7f0237400640 1 -- 172.21.15.47:0/1386262291 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f024807f060 con 0x55fb6ec7ebd0 2024-03-21T10:26:30.038 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:30.034+0000 7f0237400640 1 -- 172.21.15.47:0/1386262291 <== mon.2 v2:172.21.15.100:3300/0 2 ==== osd_map(97..97 src has 1..97) v4 ==== 9474+0+0 (secure 0 0 0) 0x7f0248097cc0 con 0x55fb6ec7ebd0 2024-03-21T10:26:30.039 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:30.034+0000 7f025f639740 1 -- 172.21.15.47:0/1386262291 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f024401a830 con 0x55fb6ec7ebd0 2024-03-21T10:26:30.039 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:30.035+0000 7f0237400640 1 -- 172.21.15.47:0/1386262291 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f02480b8030 con 0x55fb6ec7ebd0 2024-03-21T10:26:30.039 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:30.035+0000 7f025f639740 10 client.35960 fetch_fsmap learned FSMap version 32 2024-03-21T10:26:30.039 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:30.035+0000 7f025f639740 1 -- 172.21.15.47:0/1386262291 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x55fb6ed02fe0 con 0x55fb6ec7ebd0 2024-03-21T10:26:30.039 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:30.035+0000 7f0237400640 1 -- 172.21.15.47:0/1386262291 <== mon.2 v2:172.21.15.100:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f024401a830 con 0x55fb6ec7ebd0 2024-03-21T10:26:30.039 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:30.036+0000 7f0237400640 1 -- 172.21.15.47:0/1386262291 <== mon.2 v2:172.21.15.100:3300/0 5 ==== fsmap(e 32) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f02480ccba0 con 0x55fb6ec7ebd0 2024-03-21T10:26:30.039 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:30.036+0000 7f025f639740 10 client.35960 fetch_fsmap finished waiting for FSMap version 32 2024-03-21T10:26:30.039 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:30.036+0000 7f025f639740 10 client.35960 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-21T10:26:30.039 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:30.036+0000 7f025f639740 20 client.35960 populate_metadata read hostname 'smithi047' 2024-03-21T10:26:30.039 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:30.036+0000 7f025f639740 1 --2- 172.21.15.47:0/1386262291 >> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] conn(0x55fb6ed03660 0x55fb6ed05a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:26:30.039 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:30.036+0000 7f025f639740 4 client.35960 mds_command: new command op to 34535 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-21T10:26:30.039 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:30.036+0000 7f025f639740 1 -- 172.21.15.47:0/1386262291 --> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55fb6ebf00c0 con 0x55fb6ed03660 2024-03-21T10:26:30.040 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:30.037+0000 7f0259400640 1 --2- 172.21.15.47:0/1386262291 >> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] conn(0x55fb6ed03660 0x55fb6ed05a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:26:30.040 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:30.038+0000 7f0237400640 10 client.35960 ms_handle_connect on v2:172.21.15.100:6838/4173168440 2024-03-21T10:26:30.040 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:30.038+0000 7f0259400640 1 --2- 172.21.15.47:0/1386262291 >> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] conn(0x55fb6ed03660 0x55fb6ed05a80 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a6397f5778cb4d7d server_cookie=d1137454e42a8685 in_seq=0 out_seq=0 2024-03-21T10:26:30.045 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:30.044+0000 7f0237400640 1 -- 172.21.15.47:0/1386262291 <== mds.0 v2:172.21.15.100:6838/4173168440 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55fb6ebf00c0 con 0x55fb6ed03660 2024-03-21T10:26:30.045 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:30.044+0000 7f0237400640 10 client.35960 handle_command_reply: tid=0 2024-03-21T10:26:30.045 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:30.044+0000 7f025f639740 1 client.35960 shutdown 2024-03-21T10:26:30.046 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:30.045+0000 7f025f639740 1 -- 172.21.15.47:0/1386262291 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fb6ec7ebd0 msgr2=0x55fb6ec8eaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:26:30.046 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:30.045+0000 7f025f639740 1 --2- 172.21.15.47:0/1386262291 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fb6ec7ebd0 0x55fb6ec8eaf0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f024809b350 tx=0x7f02480972f0 comp rx=0 tx=0).stop 2024-03-21T10:26:30.046 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:30.045+0000 7f025f639740 1 -- 172.21.15.47:0/1386262291 shutdown_connections 2024-03-21T10:26:30.046 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:30.045+0000 7f025f639740 1 --2- 172.21.15.47:0/1386262291 >> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] conn(0x55fb6ed03660 0x55fb6ed05a80 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:26:30.046 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:30.045+0000 7f0237400640 0 client.35960 ms_handle_reset on v2:172.21.15.100:6838/4173168440 2024-03-21T10:26:30.047 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:30.045+0000 7f025f639740 1 --2- 172.21.15.47:0/1386262291 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55fb6eca16a0 0x55fb6eca1b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:26:30.047 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:30.045+0000 7f025f639740 1 --2- 172.21.15.47:0/1386262291 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fb6ec92210 0x55fb6eca1160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:26:30.047 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:30.045+0000 7f025f639740 1 --2- 172.21.15.47:0/1386262291 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fb6ec7ebd0 0x55fb6ec8eaf0 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:26:30.047 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:30.045+0000 7f025f639740 1 -- 172.21.15.47:0/1386262291 >> 172.21.15.47:0/1386262291 conn(0x55fb6ec81070 msgr2=0x55fb6ec956f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:26:30.047 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:30.045+0000 7f025f639740 1 -- 172.21.15.47:0/1386262291 shutdown_connections 2024-03-21T10:26:30.048 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:30.046+0000 7f025f639740 1 -- 172.21.15.47:0/1386262291 wait complete. 2024-03-21T10:26:30.048 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:30.046+0000 7f025f639740 20 client.35960 trim_cache size 0 max 16384 2024-03-21T10:26:30.048 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout:{ 2024-03-21T10:26:30.048 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout: "message": "", 2024-03-21T10:26:30.049 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout: "return_code": 0 2024-03-21T10:26:30.049 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout:} 2024-03-21T10:26:30.049 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:30.047+0000 7f025b200640 1 -- 172.21.15.47:0/1157795786 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f023c0a32a0 msgr2=0x7f023c0a5760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:26:30.049 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:30.047+0000 7f025b200640 1 --2- 172.21.15.47:0/1157795786 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f023c0a32a0 0x7f023c0a5760 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f0250009230 tx=0x7f0250005fd0 comp rx=0 tx=0).stop 2024-03-21T10:26:30.050 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:30.047+0000 7f025b200640 1 -- 172.21.15.47:0/1157795786 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f02541005f0 msgr2=0x7f025419da70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:26:30.050 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:30.047+0000 7f025b200640 1 --2- 172.21.15.47:0/1157795786 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f02541005f0 0x7f025419da70 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f024800ba50 tx=0x7f024800bf10 comp rx=0 tx=0).stop 2024-03-21T10:26:30.050 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:30.047+0000 7f025b200640 1 -- 172.21.15.47:0/1157795786 shutdown_connections 2024-03-21T10:26:30.050 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:30.047+0000 7f025b200640 1 --2- 172.21.15.47:0/1157795786 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f023c0a32a0 0x7f023c0a5760 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:26:30.050 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:30.047+0000 7f025b200640 1 --2- 172.21.15.47:0/1157795786 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0254197b80 0x7f0254198000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:26:30.050 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:30.047+0000 7f025b200640 1 --2- 172.21.15.47:0/1157795786 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f02541056c0 0x7f025419dfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:26:30.050 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:30.047+0000 7f025b200640 1 --2- 172.21.15.47:0/1157795786 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f02541005f0 0x7f025419da70 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:26:30.050 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:30.047+0000 7f025b200640 1 -- 172.21.15.47:0/1157795786 >> 172.21.15.47:0/1157795786 conn(0x7f02540fbeb0 msgr2=0x7f02540fd280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:26:30.050 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:30.048+0000 7f025b200640 1 -- 172.21.15.47:0/1157795786 shutdown_connections 2024-03-21T10:26:30.050 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:30.048+0000 7f025b200640 1 -- 172.21.15.47:0/1157795786 wait complete. 2024-03-21T10:26:30.478 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:26:30 smithi100 ceph-mon[35193]: from='mgr.15687 172.21.15.47:0/8730661' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T10:26:30.489 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:26:30 smithi157 ceph-mon[35581]: from='mgr.15687 172.21.15.47:0/8730661' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T10:26:30.713 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:26:30 smithi047 ceph-mon[31547]: from='mgr.15687 172.21.15.47:0/8730661' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T10:26:31.478 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:26:31 smithi100 ceph-mon[35193]: pgmap v39: 129 pgs: 129 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 255 B/s wr, 10 op/s 2024-03-21T10:26:31.479 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:26:31 smithi100 ceph-mon[35193]: from='mgr.15687 172.21.15.47:0/8730661' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T10:26:31.479 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:26:31 smithi100 ceph-mon[35193]: from='mgr.15687 172.21.15.47:0/8730661' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T10:26:31.479 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:26:31 smithi100 ceph-mon[35193]: from='mgr.15687 172.21.15.47:0/8730661' entity='mgr.x' 2024-03-21T10:26:31.489 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:26:31 smithi157 ceph-mon[35581]: pgmap v39: 129 pgs: 129 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 255 B/s wr, 10 op/s 2024-03-21T10:26:31.489 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:26:31 smithi157 ceph-mon[35581]: from='mgr.15687 172.21.15.47:0/8730661' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T10:26:31.489 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:26:31 smithi157 ceph-mon[35581]: from='mgr.15687 172.21.15.47:0/8730661' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T10:26:31.489 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:26:31 smithi157 ceph-mon[35581]: from='mgr.15687 172.21.15.47:0/8730661' entity='mgr.x' 2024-03-21T10:26:31.713 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:26:31 smithi047 ceph-mon[31547]: pgmap v39: 129 pgs: 129 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 255 B/s wr, 10 op/s 2024-03-21T10:26:31.713 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:26:31 smithi047 ceph-mon[31547]: from='mgr.15687 172.21.15.47:0/8730661' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T10:26:31.713 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:26:31 smithi047 ceph-mon[31547]: from='mgr.15687 172.21.15.47:0/8730661' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T10:26:31.713 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:26:31 smithi047 ceph-mon[31547]: from='mgr.15687 172.21.15.47:0/8730661' entity='mgr.x' 2024-03-21T10:26:33.478 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:26:33 smithi100 ceph-mon[35193]: pgmap v40: 129 pgs: 129 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 8.1 KiB/s rd, 511 B/s wr, 14 op/s 2024-03-21T10:26:33.489 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:26:33 smithi157 ceph-mon[35581]: pgmap v40: 129 pgs: 129 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 8.1 KiB/s rd, 511 B/s wr, 14 op/s 2024-03-21T10:26:33.713 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:26:33 smithi047 ceph-mon[31547]: pgmap v40: 129 pgs: 129 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 8.1 KiB/s rd, 511 B/s wr, 14 op/s 2024-03-21T10:26:35.489 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:26:35 smithi157 ceph-mon[35581]: pgmap v41: 129 pgs: 129 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-03-21T10:26:35.713 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:26:35 smithi047 ceph-mon[31547]: pgmap v41: 129 pgs: 129 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-03-21T10:26:35.728 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:26:35 smithi100 ceph-mon[35193]: pgmap v41: 129 pgs: 129 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-03-21T10:26:37.489 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:26:37 smithi157 ceph-mon[35581]: pgmap v42: 129 pgs: 129 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 255 B/s wr, 12 op/s 2024-03-21T10:26:37.712 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:26:37 smithi047 ceph-mon[31547]: pgmap v42: 129 pgs: 129 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 255 B/s wr, 12 op/s 2024-03-21T10:26:37.728 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:26:37 smithi100 ceph-mon[35193]: pgmap v42: 129 pgs: 129 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 255 B/s wr, 12 op/s 2024-03-21T10:26:39.713 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:26:39 smithi047 ceph-mon[31547]: pgmap v43: 129 pgs: 129 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 9 op/s 2024-03-21T10:26:39.728 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:26:39 smithi100 ceph-mon[35193]: pgmap v43: 129 pgs: 129 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 9 op/s 2024-03-21T10:26:39.739 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:26:39 smithi157 ceph-mon[35581]: pgmap v43: 129 pgs: 129 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 9 op/s 2024-03-21T10:26:41.542 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:26:41 smithi157 ceph-mon[35581]: pgmap v44: 129 pgs: 129 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-03-21T10:26:41.712 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:26:41 smithi047 ceph-mon[31547]: pgmap v44: 129 pgs: 129 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-03-21T10:26:41.728 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:26:41 smithi100 ceph-mon[35193]: pgmap v44: 129 pgs: 129 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-03-21T10:26:43.240 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:43.239+0000 7f5f8d600640 1 -- 172.21.15.47:0/761208178 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5f88112510 msgr2=0x7f5f88114900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:26:43.240 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:43.239+0000 7f5f8d600640 1 --2- 172.21.15.47:0/761208178 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5f88112510 0x7f5f88114900 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7f5f80009ca0 tx=0x7f5f8002c120 comp rx=0 tx=0).stop 2024-03-21T10:26:43.240 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:43.240+0000 7f5f8d600640 1 -- 172.21.15.47:0/761208178 shutdown_connections 2024-03-21T10:26:43.240 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:43.240+0000 7f5f8d600640 1 --2- 172.21.15.47:0/761208178 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5f88112510 0x7f5f88114900 unknown :-1 s=CLOSED pgs=618 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:26:43.240 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:43.240+0000 7f5f8d600640 1 --2- 172.21.15.47:0/761208178 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5f880764f0 0x7f5f88111fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:26:43.240 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:43.240+0000 7f5f8d600640 1 --2- 172.21.15.47:0/761208178 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5f88075bd0 0x7f5f88075fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:26:43.241 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:43.240+0000 7f5f8d600640 1 -- 172.21.15.47:0/761208178 >> 172.21.15.47:0/761208178 conn(0x7f5f880ff360 msgr2=0x7f5f88101780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:26:43.241 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:43.240+0000 7f5f8d600640 1 -- 172.21.15.47:0/761208178 shutdown_connections 2024-03-21T10:26:43.241 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:43.240+0000 7f5f8d600640 1 -- 172.21.15.47:0/761208178 wait complete. 2024-03-21T10:26:43.241 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:43.241+0000 7f5f8d600640 1 Processor -- start 2024-03-21T10:26:43.242 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:43.242+0000 7f5f8d600640 1 -- start start 2024-03-21T10:26:43.242 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:43.242+0000 7f5f8d600640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5f88075bd0 0x7f5f8819e4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:26:43.243 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:43.242+0000 7f5f8d600640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5f880764f0 0x7f5f8819e9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:26:43.243 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:43.242+0000 7f5f8d600640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5f88112510 0x7f5f881a2d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:26:43.243 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:43.242+0000 7f5f8d600640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f88117fa0 con 0x7f5f88112510 2024-03-21T10:26:43.243 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:43.242+0000 7f5f8d600640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f88117e20 con 0x7f5f88075bd0 2024-03-21T10:26:43.243 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:43.242+0000 7f5f8d600640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f88118120 con 0x7f5f880764f0 2024-03-21T10:26:43.243 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:43.243+0000 7f5f86000640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5f880764f0 0x7f5f8819e9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:26:43.243 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:43.243+0000 7f5f86000640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5f880764f0 0x7f5f8819e9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.47:45750/0 (socket says 172.21.15.47:45750) 2024-03-21T10:26:43.243 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:43.243+0000 7f5f86000640 1 -- 172.21.15.47:0/180331855 learned_addr learned my addr 172.21.15.47:0/180331855 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:26:43.244 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:43.243+0000 7f5f87400640 1 --2- 172.21.15.47:0/180331855 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5f88112510 0x7f5f881a2d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:26:43.244 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:43.244+0000 7f5f87400640 1 -- 172.21.15.47:0/180331855 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5f880764f0 msgr2=0x7f5f8819e9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:26:43.244 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:43.244+0000 7f5f87400640 1 --2- 172.21.15.47:0/180331855 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5f880764f0 0x7f5f8819e9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:26:43.244 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:43.244+0000 7f5f87400640 1 -- 172.21.15.47:0/180331855 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5f88075bd0 msgr2=0x7f5f8819e4a0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-03-21T10:26:43.244 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:43.244+0000 7f5f87400640 1 --2- 172.21.15.47:0/180331855 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5f88075bd0 0x7f5f8819e4a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:26:43.244 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:43.244+0000 7f5f87400640 1 -- 172.21.15.47:0/180331855 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f881a3490 con 0x7f5f88112510 2024-03-21T10:26:43.244 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:43.244+0000 7f5f87400640 1 --2- 172.21.15.47:0/180331855 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5f88112510 0x7f5f881a2d60 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7f5f8002c660 tx=0x7f5f80035d60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:26:43.245 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:43.245+0000 7f5f7f400640 1 -- 172.21.15.47:0/180331855 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f80039040 con 0x7f5f88112510 2024-03-21T10:26:43.245 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:43.245+0000 7f5f7f400640 1 -- 172.21.15.47:0/180331855 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f5f80044070 con 0x7f5f88112510 2024-03-21T10:26:43.245 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:43.245+0000 7f5f7f400640 1 -- 172.21.15.47:0/180331855 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f80004460 con 0x7f5f88112510 2024-03-21T10:26:43.245 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:43.245+0000 7f5f8d600640 1 -- 172.21.15.47:0/180331855 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5f881a3720 con 0x7f5f88112510 2024-03-21T10:26:43.245 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:43.245+0000 7f5f8d600640 1 -- 172.21.15.47:0/180331855 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5f881a3b90 con 0x7f5f88112510 2024-03-21T10:26:43.249 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:43.249+0000 7f5f903d5740 1 Processor -- start 2024-03-21T10:26:43.251 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:43.251+0000 7f5f7f400640 1 -- 172.21.15.47:0/180331855 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 26) v1 ==== 138087+0+0 (secure 0 0 0) 0x7f5f800326f0 con 0x7f5f88112510 2024-03-21T10:26:43.251 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:43.251+0000 7f5f903d5740 1 -- start start 2024-03-21T10:26:43.252 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:43.252+0000 7f5f903d5740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55904a209fd0 0x55904a20a3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:26:43.254 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:43.252+0000 7f5f903d5740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55904a20a8f0 0x55904a213180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:26:43.254 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:43.252+0000 7f5f903d5740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55904a2136c0 0x55904a215ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:26:43.254 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:43.252+0000 7f5f903d5740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x55904a1cc3b0 con 0x55904a2136c0 2024-03-21T10:26:43.254 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:43.252+0000 7f5f903d5740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55904a16e830 con 0x55904a209fd0 2024-03-21T10:26:43.254 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:43.252+0000 7f5f903d5740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55904a1175d0 con 0x55904a20a8f0 2024-03-21T10:26:43.254 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:43.253+0000 7f5f86a00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55904a20a8f0 0x55904a213180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:26:43.254 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:43.254+0000 7f5f86a00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55904a20a8f0 0x55904a213180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.47:45758/0 (socket says 172.21.15.47:45758) 2024-03-21T10:26:43.254 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:43.254+0000 7f5f86a00640 1 -- 172.21.15.47:0/864710557 learned_addr learned my addr 172.21.15.47:0/864710557 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:26:43.254 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:43.254+0000 7f5f86000640 1 --2- 172.21.15.47:0/864710557 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55904a2136c0 0x55904a215ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:26:43.255 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:43.254+0000 7f5f7f400640 1 --2- 172.21.15.47:0/180331855 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f5f6c0a32d0 0x7f5f6c0a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:26:43.255 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:43.254+0000 7f5f86000640 1 --2- 172.21.15.47:0/864710557 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55904a209fd0 0x55904a20a3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:26:43.255 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:43.255+0000 7f5f7f400640 1 -- 172.21.15.47:0/180331855 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 9474+0+0 (secure 0 0 0) 0x7f5f800f2320 con 0x7f5f88112510 2024-03-21T10:26:43.257 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:43.255+0000 7f5f87400640 1 --2- 172.21.15.47:0/180331855 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f5f6c0a32d0 0x7f5f6c0a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:26:43.257 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:43.255+0000 7f5f86a00640 1 -- 172.21.15.47:0/864710557 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55904a209fd0 msgr2=0x55904a20a3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:26:43.257 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:43.255+0000 7f5f86a00640 1 --2- 172.21.15.47:0/864710557 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55904a209fd0 0x55904a20a3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:26:43.257 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:43.255+0000 7f5f86a00640 1 -- 172.21.15.47:0/864710557 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55904a2136c0 msgr2=0x55904a215ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:26:43.257 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:43.255+0000 7f5f86a00640 1 --2- 172.21.15.47:0/864710557 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55904a2136c0 0x55904a215ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:26:43.257 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:43.255+0000 7f5f86a00640 1 -- 172.21.15.47:0/864710557 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55904a12f2c0 con 0x55904a20a8f0 2024-03-21T10:26:43.257 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:43.256+0000 7f5f86a00640 1 --2- 172.21.15.47:0/864710557 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55904a20a8f0 0x55904a213180 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f5f78009770 tx=0x7f5f78009c30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b0bafac4c4620e6b server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:26:43.257 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:43.256+0000 7f5f87400640 1 --2- 172.21.15.47:0/180331855 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f5f6c0a32d0 0x7f5f6c0a5790 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f5f800ef640 tx=0x7f5f80035f50 comp rx=0 tx=0).ready entity=mgr.15687 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:26:43.261 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:43.261+0000 7f5f73400640 1 -- 172.21.15.47:0/864710557 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f78015020 con 0x55904a20a8f0 2024-03-21T10:26:43.261 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:43.261+0000 7f5f73400640 1 -- 172.21.15.47:0/864710557 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f5f78011420 con 0x55904a20a8f0 2024-03-21T10:26:43.261 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:43.261+0000 7f5f73400640 1 -- 172.21.15.47:0/864710557 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f780105a0 con 0x55904a20a8f0 2024-03-21T10:26:43.262 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:43.262+0000 7f5f903d5740 1 -- 172.21.15.47:0/864710557 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55904a20a8f0 msgr2=0x55904a213180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:26:43.262 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:43.262+0000 7f5f903d5740 1 --2- 172.21.15.47:0/864710557 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55904a20a8f0 0x55904a213180 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f5f78009770 tx=0x7f5f78009c30 comp rx=0 tx=0).stop 2024-03-21T10:26:43.263 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:43.262+0000 7f5f903d5740 1 -- 172.21.15.47:0/864710557 shutdown_connections 2024-03-21T10:26:43.263 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:43.262+0000 7f5f903d5740 1 --2- 172.21.15.47:0/864710557 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55904a2136c0 0x55904a215ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:26:43.263 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:43.262+0000 7f5f903d5740 1 --2- 172.21.15.47:0/864710557 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55904a20a8f0 0x55904a213180 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:26:43.263 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:43.262+0000 7f5f903d5740 1 --2- 172.21.15.47:0/864710557 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55904a209fd0 0x55904a20a3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:26:43.263 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:43.262+0000 7f5f903d5740 1 -- 172.21.15.47:0/864710557 >> 172.21.15.47:0/864710557 conn(0x55904a2048f0 msgr2=0x55904a204cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:26:43.263 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:43.262+0000 7f5f903d5740 1 -- 172.21.15.47:0/864710557 shutdown_connections 2024-03-21T10:26:43.263 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:43.262+0000 7f5f903d5740 1 -- 172.21.15.47:0/864710557 wait complete. 2024-03-21T10:26:43.263 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:43.263+0000 7f5f903d5740 1 -- start start 2024-03-21T10:26:43.264 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:43.263+0000 7f5f903d5740 1 Processor -- start 2024-03-21T10:26:43.264 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:43.264+0000 7f5f903d5740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55904a209fd0 0x55904a210660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:26:43.265 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:43.264+0000 7f5f903d5740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55904a20a8f0 0x55904a210ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:26:43.265 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:43.264+0000 7f5f903d5740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55904a2136c0 0x55904a2110e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:26:43.265 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:43.264+0000 7f5f903d5740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x55904a12aa20 con 0x55904a209fd0 2024-03-21T10:26:43.265 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:43.264+0000 7f5f903d5740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55904a16a8d0 con 0x55904a20a8f0 2024-03-21T10:26:43.265 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:43.264+0000 7f5f903d5740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55904a216340 con 0x55904a2136c0 2024-03-21T10:26:43.265 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:43.264+0000 7f5f903d5740 10 client.0 _ll_register_callbacks cb 0x55904a1671c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T10:26:43.265 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:43.264+0000 7f5f86000640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55904a209fd0 0x55904a210660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:26:43.265 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:43.264+0000 7f5f86a00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55904a20a8f0 0x55904a210ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:26:43.265 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:43.264+0000 7f5f86a00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55904a20a8f0 0x55904a210ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:52134/0 (socket says 172.21.15.47:52134) 2024-03-21T10:26:43.265 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:43.264+0000 7f5f86a00640 1 -- 172.21.15.47:0/535593245 learned_addr learned my addr 172.21.15.47:0/535593245 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:26:43.266 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:43.265+0000 7f5f86000640 1 -- 172.21.15.47:0/535593245 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55904a2136c0 msgr2=0x55904a2110e0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-21T10:26:43.266 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:43.265+0000 7f5f86000640 1 --2- 172.21.15.47:0/535593245 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55904a2136c0 0x55904a2110e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:26:43.266 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:43.265+0000 7f5f86000640 1 -- 172.21.15.47:0/535593245 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55904a20a8f0 msgr2=0x55904a210ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:26:43.266 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:43.265+0000 7f5f86000640 1 --2- 172.21.15.47:0/535593245 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55904a20a8f0 0x55904a210ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:26:43.266 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:43.265+0000 7f5f86000640 1 -- 172.21.15.47:0/535593245 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x559049e83210 con 0x55904a209fd0 2024-03-21T10:26:43.266 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:43.265+0000 7f5f86000640 1 --2- 172.21.15.47:0/535593245 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55904a209fd0 0x55904a210660 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x55904a205a80 tx=0x7f5f7400fdd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=18e5232c733dd58 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:26:43.267 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:43.265+0000 7f5f67e00640 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-03-21T10:26:43.267 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:43.266+0000 7f5f67e00640 1 -- 172.21.15.47:0/535593245 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f74005220 con 0x55904a209fd0 2024-03-21T10:26:43.267 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:43.266+0000 7f5f67e00640 1 -- 172.21.15.47:0/535593245 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(97..97 src has 1..97) v4 ==== 9474+0+0 (secure 0 0 0) 0x7f5f740053c0 con 0x55904a209fd0 2024-03-21T10:26:43.267 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:43.266+0000 7f5f903d5740 1 -- 172.21.15.47:0/535593245 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f5f78011420 con 0x55904a209fd0 2024-03-21T10:26:43.267 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:43.267+0000 7f5f67e00640 1 -- 172.21.15.47:0/535593245 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5f74028050 con 0x55904a209fd0 2024-03-21T10:26:43.268 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:43.268+0000 7f5f903d5740 10 client.16167 fetch_fsmap learned FSMap version 32 2024-03-21T10:26:43.268 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:43.268+0000 7f5f903d5740 1 -- 172.21.15.47:0/535593245 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x559049f4f620 con 0x55904a209fd0 2024-03-21T10:26:43.268 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:43.268+0000 7f5f67e00640 1 -- 172.21.15.47:0/535593245 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f5f78011420 con 0x55904a209fd0 2024-03-21T10:26:43.269 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:43.268+0000 7f5f67e00640 1 -- 172.21.15.47:0/535593245 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 32) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f5f7401c030 con 0x55904a209fd0 2024-03-21T10:26:43.269 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:43.269+0000 7f5f903d5740 10 client.16167 fetch_fsmap finished waiting for FSMap version 32 2024-03-21T10:26:43.269 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:43.269+0000 7f5f903d5740 10 client.16167 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-21T10:26:43.270 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:43.269+0000 7f5f903d5740 20 client.16167 populate_metadata read hostname 'smithi047' 2024-03-21T10:26:43.270 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:43.269+0000 7f5f903d5740 1 --2- 172.21.15.47:0/535593245 >> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] conn(0x55904a223790 0x55904a225b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:26:43.270 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:43.269+0000 7f5f903d5740 4 client.16167 mds_command: new command op to 34535 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T10:26:43.270 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:43.269+0000 7f5f903d5740 1 -- 172.21.15.47:0/535593245 --> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55904a17bd70 con 0x55904a223790 2024-03-21T10:26:43.270 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:43.270+0000 7f5f86a00640 1 --2- 172.21.15.47:0/535593245 >> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] conn(0x55904a223790 0x55904a225b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:26:43.270 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:43.270+0000 7f5f86a00640 1 --2- 172.21.15.47:0/535593245 >> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] conn(0x55904a223790 0x55904a225b80 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1a367671725aadd9 server_cookie=de8c1572837b3f5d in_seq=0 out_seq=0 2024-03-21T10:26:43.270 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:43.270+0000 7f5f67e00640 10 client.16167 ms_handle_connect on v2:172.21.15.100:6838/4173168440 2024-03-21T10:26:43.271 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:43.271+0000 7f5f67e00640 1 -- 172.21.15.47:0/535593245 <== mds.0 v2:172.21.15.100:6838/4173168440 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55904a17bd70 con 0x55904a223790 2024-03-21T10:26:43.272 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:43.271+0000 7f5f67e00640 10 client.16167 handle_command_reply: tid=0 2024-03-21T10:26:43.272 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:43.272+0000 7f5f903d5740 1 client.16167 shutdown 2024-03-21T10:26:43.272 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:43.272+0000 7f5f903d5740 1 -- 172.21.15.47:0/535593245 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55904a209fd0 msgr2=0x55904a210660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:26:43.272 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:43.272+0000 7f5f903d5740 1 --2- 172.21.15.47:0/535593245 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55904a209fd0 0x55904a210660 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x55904a205a80 tx=0x7f5f7400fdd0 comp rx=0 tx=0).stop 2024-03-21T10:26:43.273 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:43.273+0000 7f5f903d5740 1 -- 172.21.15.47:0/535593245 shutdown_connections 2024-03-21T10:26:43.273 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:43.273+0000 7f5f903d5740 1 --2- 172.21.15.47:0/535593245 >> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] conn(0x55904a223790 0x55904a225b80 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:26:43.273 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:43.273+0000 7f5f903d5740 1 --2- 172.21.15.47:0/535593245 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55904a2136c0 0x55904a2110e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:26:43.273 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:43.273+0000 7f5f903d5740 1 --2- 172.21.15.47:0/535593245 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55904a20a8f0 0x55904a210ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:26:43.273 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:43.273+0000 7f5f903d5740 1 --2- 172.21.15.47:0/535593245 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55904a209fd0 0x55904a210660 unknown :-1 s=CLOSED pgs=620 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:26:43.273 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:43.273+0000 7f5f903d5740 1 -- 172.21.15.47:0/535593245 >> 172.21.15.47:0/535593245 conn(0x55904a2048f0 msgr2=0x55904a206640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:26:43.273 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:43.273+0000 7f5f67e00640 0 client.16167 ms_handle_reset on v2:172.21.15.100:6838/4173168440 2024-03-21T10:26:43.274 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:43.273+0000 7f5f903d5740 1 -- 172.21.15.47:0/535593245 shutdown_connections 2024-03-21T10:26:43.274 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:43.273+0000 7f5f903d5740 1 -- 172.21.15.47:0/535593245 wait complete. 2024-03-21T10:26:43.274 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:43.274+0000 7f5f903d5740 20 client.16167 trim_cache size 0 max 16384 2024-03-21T10:26:43.301 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:43.300+0000 7f5f903d5740 1 Processor -- start 2024-03-21T10:26:43.301 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:43.300+0000 7f5f903d5740 1 -- start start 2024-03-21T10:26:43.301 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:43.300+0000 7f5f903d5740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55904a209fd0 0x55904a212d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:26:43.301 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:43.301+0000 7f5f903d5740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55904a20a8f0 0x55904a21ea80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:26:43.301 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:43.301+0000 7f5f903d5740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55904a2136c0 0x55904a21efc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:26:43.301 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:43.301+0000 7f5f903d5740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f881b4de0 con 0x55904a20a8f0 2024-03-21T10:26:43.301 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:43.301+0000 7f5f903d5740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f881b6760 con 0x55904a2136c0 2024-03-21T10:26:43.301 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:43.301+0000 7f5f903d5740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f881b5440 con 0x55904a209fd0 2024-03-21T10:26:43.302 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:43.301+0000 7f5f86a00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55904a20a8f0 0x55904a21ea80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:26:43.302 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:43.301+0000 7f5f86a00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55904a20a8f0 0x55904a21ea80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:57842/0 (socket says 172.21.15.47:57842) 2024-03-21T10:26:43.302 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:43.301+0000 7f5f86a00640 1 -- 172.21.15.47:0/4027034060 learned_addr learned my addr 172.21.15.47:0/4027034060 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:26:43.302 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:43.301+0000 7f5f86000640 1 --2- 172.21.15.47:0/4027034060 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55904a2136c0 0x55904a21efc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:26:43.302 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:43.301+0000 7f5f86000640 1 --2- 172.21.15.47:0/4027034060 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55904a209fd0 0x55904a212d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:26:43.302 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:43.302+0000 7f5f86000640 1 -- 172.21.15.47:0/4027034060 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55904a209fd0 msgr2=0x55904a212d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:26:43.302 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:43.302+0000 7f5f86000640 1 --2- 172.21.15.47:0/4027034060 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55904a209fd0 0x55904a212d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:26:43.302 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:43.302+0000 7f5f86000640 1 -- 172.21.15.47:0/4027034060 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55904a20a8f0 msgr2=0x55904a21ea80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:26:43.303 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:43.302+0000 7f5f86000640 1 --2- 172.21.15.47:0/4027034060 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55904a20a8f0 0x55904a21ea80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:26:43.303 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:43.302+0000 7f5f86000640 1 -- 172.21.15.47:0/4027034060 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55904a20ebf0 con 0x55904a2136c0 2024-03-21T10:26:43.303 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:43.302+0000 7f5f86000640 1 --2- 172.21.15.47:0/4027034060 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55904a2136c0 0x55904a21efc0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f5f7400bf00 tx=0x7f5f74014040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4149d80e28e8e79d server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:26:43.304 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:43.302+0000 7f5f72a00640 1 -- 172.21.15.47:0/4027034060 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f74010430 con 0x55904a2136c0 2024-03-21T10:26:43.304 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:43.302+0000 7f5f72a00640 1 -- 172.21.15.47:0/4027034060 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f5f74010a80 con 0x55904a2136c0 2024-03-21T10:26:43.304 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:43.302+0000 7f5f72a00640 1 -- 172.21.15.47:0/4027034060 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f7402e7c0 con 0x55904a2136c0 2024-03-21T10:26:43.304 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:43.303+0000 7f5f903d5740 1 -- 172.21.15.47:0/4027034060 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55904a2136c0 msgr2=0x55904a21efc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:26:43.304 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:43.303+0000 7f5f903d5740 1 --2- 172.21.15.47:0/4027034060 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55904a2136c0 0x55904a21efc0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f5f7400bf00 tx=0x7f5f74014040 comp rx=0 tx=0).stop 2024-03-21T10:26:43.304 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:43.303+0000 7f5f903d5740 1 -- 172.21.15.47:0/4027034060 shutdown_connections 2024-03-21T10:26:43.304 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:43.303+0000 7f5f903d5740 1 --2- 172.21.15.47:0/4027034060 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55904a2136c0 0x55904a21efc0 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:26:43.304 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:43.303+0000 7f5f903d5740 1 --2- 172.21.15.47:0/4027034060 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55904a20a8f0 0x55904a21ea80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:26:43.304 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:43.303+0000 7f5f903d5740 1 --2- 172.21.15.47:0/4027034060 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55904a209fd0 0x55904a212d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:26:43.304 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:43.303+0000 7f5f903d5740 1 -- 172.21.15.47:0/4027034060 >> 172.21.15.47:0/4027034060 conn(0x55904a2048f0 msgr2=0x55904a20b940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:26:43.304 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:43.303+0000 7f5f903d5740 1 -- 172.21.15.47:0/4027034060 shutdown_connections 2024-03-21T10:26:43.305 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:43.303+0000 7f5f903d5740 1 -- 172.21.15.47:0/4027034060 wait complete. 2024-03-21T10:26:43.305 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:43.304+0000 7f5f903d5740 1 -- start start 2024-03-21T10:26:43.305 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:43.304+0000 7f5f903d5740 1 Processor -- start 2024-03-21T10:26:43.306 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:43.304+0000 7f5f903d5740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55904a209fd0 0x55904a20e3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:26:43.306 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:43.304+0000 7f5f903d5740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55904a20a8f0 0x55904a20cd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:26:43.306 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:43.304+0000 7f5f903d5740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55904a2136c0 0x55904a20d460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:26:43.307 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:43.304+0000 7f5f903d5740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f881b6100 con 0x55904a209fd0 2024-03-21T10:26:43.307 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:43.304+0000 7f5f903d5740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f881b5aa0 con 0x55904a2136c0 2024-03-21T10:26:43.307 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:43.304+0000 7f5f903d5740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f881b2fa0 con 0x55904a20a8f0 2024-03-21T10:26:43.307 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:43.305+0000 7f5f903d5740 10 client.0 _ll_register_callbacks cb 0x7f5f7400bdb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T10:26:43.307 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:43.305+0000 7f5f86000640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55904a209fd0 0x55904a20e3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:26:43.307 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:43.305+0000 7f5f86000640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55904a209fd0 0x55904a20e3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:57844/0 (socket says 172.21.15.47:57844) 2024-03-21T10:26:43.307 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:43.305+0000 7f5f86000640 1 -- 172.21.15.47:0/2281900968 learned_addr learned my addr 172.21.15.47:0/2281900968 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:26:43.307 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:43.305+0000 7f5f86a00640 1 --2- 172.21.15.47:0/2281900968 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55904a20a8f0 0x55904a20cd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:26:43.307 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:43.305+0000 7f5f86000640 1 --2- 172.21.15.47:0/2281900968 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55904a2136c0 0x55904a20d460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:26:43.307 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:43.306+0000 7f5f86000640 1 -- 172.21.15.47:0/2281900968 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55904a20a8f0 msgr2=0x55904a20cd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:26:43.307 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:43.306+0000 7f5f86000640 1 --2- 172.21.15.47:0/2281900968 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55904a20a8f0 0x55904a20cd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:26:43.307 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:43.306+0000 7f5f86000640 1 -- 172.21.15.47:0/2281900968 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55904a2136c0 msgr2=0x55904a20d460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:26:43.307 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:43.306+0000 7f5f86000640 1 --2- 172.21.15.47:0/2281900968 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55904a2136c0 0x55904a20d460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:26:43.308 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:43.306+0000 7f5f86000640 1 -- 172.21.15.47:0/2281900968 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55904a27dfa0 con 0x55904a209fd0 2024-03-21T10:26:43.308 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:43.306+0000 7f5f86a00640 1 --2- 172.21.15.47:0/2281900968 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55904a20a8f0 0x55904a20cd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T10:26:43.308 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:43.306+0000 7f5f86000640 1 --2- 172.21.15.47:0/2281900968 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55904a209fd0 0x55904a20e3d0 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7f5f7400deb0 tx=0x7f5f7400f8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c25a12cb45aaa786 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:26:43.308 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:43.306+0000 7f5f67e00640 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-03-21T10:26:43.308 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:43.306+0000 7f5f67e00640 1 -- 172.21.15.47:0/2281900968 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f74010800 con 0x55904a209fd0 2024-03-21T10:26:43.308 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:43.306+0000 7f5f67e00640 1 -- 172.21.15.47:0/2281900968 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(97..97 src has 1..97) v4 ==== 9474+0+0 (secure 0 0 0) 0x7f5f740120d0 con 0x55904a209fd0 2024-03-21T10:26:43.308 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:43.306+0000 7f5f903d5740 1 -- 172.21.15.47:0/2281900968 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f5f74010a80 con 0x55904a209fd0 2024-03-21T10:26:43.309 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:43.307+0000 7f5f67e00640 1 -- 172.21.15.47:0/2281900968 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5f74044050 con 0x55904a209fd0 2024-03-21T10:26:43.309 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:43.307+0000 7f5f903d5740 10 client.16173 fetch_fsmap learned FSMap version 32 2024-03-21T10:26:43.309 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:43.307+0000 7f5f903d5740 1 -- 172.21.15.47:0/2281900968 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x55904a27f1b0 con 0x55904a209fd0 2024-03-21T10:26:43.309 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:43.307+0000 7f5f67e00640 1 -- 172.21.15.47:0/2281900968 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f5f74010a80 con 0x55904a209fd0 2024-03-21T10:26:43.309 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:43.307+0000 7f5f67e00640 1 -- 172.21.15.47:0/2281900968 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 32) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f5f7401c040 con 0x55904a209fd0 2024-03-21T10:26:43.310 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:43.311+0000 7f5f903d5740 10 client.16173 fetch_fsmap finished waiting for FSMap version 32 2024-03-21T10:26:43.311 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:43.311+0000 7f5f903d5740 10 client.16173 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-21T10:26:43.311 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:43.311+0000 7f5f903d5740 20 client.16173 populate_metadata read hostname 'smithi047' 2024-03-21T10:26:43.311 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:43.311+0000 7f5f903d5740 1 --2- 172.21.15.47:0/2281900968 >> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] conn(0x55904a27f710 0x55904a281b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:26:43.311 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:43.311+0000 7f5f903d5740 4 client.16173 mds_command: new command op to 34535 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-21T10:26:43.311 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:43.311+0000 7f5f903d5740 1 -- 172.21.15.47:0/2281900968 --> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55904a1760c0 con 0x55904a27f710 2024-03-21T10:26:43.311 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:43.311+0000 7f5f86a00640 1 --2- 172.21.15.47:0/2281900968 >> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] conn(0x55904a27f710 0x55904a281b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:26:43.312 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:43.312+0000 7f5f86a00640 1 --2- 172.21.15.47:0/2281900968 >> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] conn(0x55904a27f710 0x55904a281b00 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1fd2ee5322ee00ef server_cookie=13bf6532261c2c5c in_seq=0 out_seq=0 2024-03-21T10:26:43.312 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:43.312+0000 7f5f67e00640 10 client.16173 ms_handle_connect on v2:172.21.15.100:6838/4173168440 2024-03-21T10:26:43.317 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:43.317+0000 7f5f67e00640 1 -- 172.21.15.47:0/2281900968 <== mds.0 v2:172.21.15.100:6838/4173168440 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55904a1760c0 con 0x55904a27f710 2024-03-21T10:26:43.317 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:43.317+0000 7f5f67e00640 10 client.16173 handle_command_reply: tid=0 2024-03-21T10:26:43.317 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:43.317+0000 7f5f903d5740 1 client.16173 shutdown 2024-03-21T10:26:43.318 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:43.318+0000 7f5f903d5740 1 -- 172.21.15.47:0/2281900968 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55904a209fd0 msgr2=0x55904a20e3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:26:43.318 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:43.318+0000 7f5f903d5740 1 --2- 172.21.15.47:0/2281900968 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55904a209fd0 0x55904a20e3d0 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7f5f7400deb0 tx=0x7f5f7400f8d0 comp rx=0 tx=0).stop 2024-03-21T10:26:43.319 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:43.318+0000 7f5f903d5740 1 -- 172.21.15.47:0/2281900968 shutdown_connections 2024-03-21T10:26:43.319 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:43.318+0000 7f5f903d5740 1 --2- 172.21.15.47:0/2281900968 >> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] conn(0x55904a27f710 0x55904a281b00 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:26:43.319 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:43.318+0000 7f5f903d5740 1 --2- 172.21.15.47:0/2281900968 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55904a2136c0 0x55904a20d460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:26:43.319 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:43.318+0000 7f5f903d5740 1 --2- 172.21.15.47:0/2281900968 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55904a20a8f0 0x55904a20cd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:26:43.319 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:43.319+0000 7f5f67e00640 0 client.16173 ms_handle_reset on v2:172.21.15.100:6838/4173168440 2024-03-21T10:26:43.319 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:43.319+0000 7f5f903d5740 1 --2- 172.21.15.47:0/2281900968 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55904a209fd0 0x55904a20e3d0 unknown :-1 s=CLOSED pgs=621 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:26:43.320 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:43.319+0000 7f5f903d5740 1 -- 172.21.15.47:0/2281900968 >> 172.21.15.47:0/2281900968 conn(0x55904a2048f0 msgr2=0x55904a205d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:26:43.320 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:43.319+0000 7f5f903d5740 1 -- 172.21.15.47:0/2281900968 shutdown_connections 2024-03-21T10:26:43.320 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:43.320+0000 7f5f903d5740 1 -- 172.21.15.47:0/2281900968 wait complete. 2024-03-21T10:26:43.320 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:43.320+0000 7f5f903d5740 20 client.16173 trim_cache size 0 max 16384 2024-03-21T10:26:43.321 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout:{ 2024-03-21T10:26:43.321 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout: "message": "", 2024-03-21T10:26:43.321 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout: "return_code": 0 2024-03-21T10:26:43.321 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout:} 2024-03-21T10:26:43.322 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:43.321+0000 7f5f8d600640 1 -- 172.21.15.47:0/180331855 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f5f6c0a32d0 msgr2=0x7f5f6c0a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:26:43.322 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:43.321+0000 7f5f8d600640 1 --2- 172.21.15.47:0/180331855 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f5f6c0a32d0 0x7f5f6c0a5790 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f5f800ef640 tx=0x7f5f80035f50 comp rx=0 tx=0).stop 2024-03-21T10:26:43.322 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:43.321+0000 7f5f8d600640 1 -- 172.21.15.47:0/180331855 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5f88112510 msgr2=0x7f5f881a2d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:26:43.322 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:43.321+0000 7f5f8d600640 1 --2- 172.21.15.47:0/180331855 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5f88112510 0x7f5f881a2d60 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7f5f8002c660 tx=0x7f5f80035d60 comp rx=0 tx=0).stop 2024-03-21T10:26:43.322 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:43.321+0000 7f5f8d600640 1 -- 172.21.15.47:0/180331855 shutdown_connections 2024-03-21T10:26:43.322 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:43.321+0000 7f5f8d600640 1 --2- 172.21.15.47:0/180331855 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5f88112510 0x7f5f881a2d60 unknown :-1 s=CLOSED pgs=619 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:26:43.322 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:43.321+0000 7f5f8d600640 1 --2- 172.21.15.47:0/180331855 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5f880764f0 0x7f5f8819e9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:26:43.322 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:43.321+0000 7f5f8d600640 1 --2- 172.21.15.47:0/180331855 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f5f6c0a32d0 0x7f5f6c0a5790 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:26:43.322 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:43.321+0000 7f5f8d600640 1 --2- 172.21.15.47:0/180331855 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5f88075bd0 0x7f5f8819e4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:26:43.322 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:43.321+0000 7f5f8d600640 1 -- 172.21.15.47:0/180331855 >> 172.21.15.47:0/180331855 conn(0x7f5f880ff360 msgr2=0x7f5f88113180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:26:43.323 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:43.322+0000 7f5f8d600640 1 -- 172.21.15.47:0/180331855 shutdown_connections 2024-03-21T10:26:43.323 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:43.322+0000 7f5f8d600640 1 -- 172.21.15.47:0/180331855 wait complete. 2024-03-21T10:26:43.489 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:26:43 smithi157 ceph-mon[35581]: pgmap v45: 129 pgs: 129 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 255 B/s wr, 12 op/s 2024-03-21T10:26:43.712 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:26:43 smithi047 ceph-mon[31547]: pgmap v45: 129 pgs: 129 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 255 B/s wr, 12 op/s 2024-03-21T10:26:43.728 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:26:43 smithi100 ceph-mon[35193]: pgmap v45: 129 pgs: 129 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 255 B/s wr, 12 op/s 2024-03-21T10:26:45.713 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:26:45 smithi047 ceph-mon[31547]: pgmap v46: 129 pgs: 129 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-21T10:26:45.728 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:26:45 smithi100 ceph-mon[35193]: pgmap v46: 129 pgs: 129 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-21T10:26:45.738 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:26:45 smithi157 ceph-mon[35581]: pgmap v46: 129 pgs: 129 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-21T10:26:47.713 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:26:47 smithi047 ceph-mon[31547]: pgmap v47: 129 pgs: 129 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 255 B/s wr, 12 op/s 2024-03-21T10:26:47.729 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:26:47 smithi100 ceph-mon[35193]: pgmap v47: 129 pgs: 129 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 255 B/s wr, 12 op/s 2024-03-21T10:26:47.739 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:26:47 smithi157 ceph-mon[35581]: pgmap v47: 129 pgs: 129 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 255 B/s wr, 12 op/s 2024-03-21T10:26:48.020 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-21T10:26:48.021 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-21T10:26:48.232 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:48.232+0000 7f2596000640 1 -- 172.21.15.47:0/4017692861 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2590112640 msgr2=0x7f2590114a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:26:48.233 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:48.232+0000 7f2596000640 1 --2- 172.21.15.47:0/4017692861 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2590112640 0x7f2590114a30 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7f2580009900 tx=0x7f258002bcd0 comp rx=0 tx=0).stop 2024-03-21T10:26:48.233 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:48.232+0000 7f2596000640 1 -- 172.21.15.47:0/4017692861 shutdown_connections 2024-03-21T10:26:48.233 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:48.232+0000 7f2596000640 1 --2- 172.21.15.47:0/4017692861 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2590112640 0x7f2590114a30 unknown :-1 s=CLOSED pgs=622 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:26:48.233 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:48.232+0000 7f2596000640 1 --2- 172.21.15.47:0/4017692861 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2590074360 0x7f2590112100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:26:48.233 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:48.232+0000 7f2596000640 1 --2- 172.21.15.47:0/4017692861 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2590073a40 0x7f2590073e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:26:48.233 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:48.232+0000 7f2596000640 1 -- 172.21.15.47:0/4017692861 >> 172.21.15.47:0/4017692861 conn(0x7f25900fd140 msgr2=0x7f25900ff560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:26:48.233 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:48.232+0000 7f2596000640 1 -- 172.21.15.47:0/4017692861 shutdown_connections 2024-03-21T10:26:48.233 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:48.232+0000 7f2596000640 1 -- 172.21.15.47:0/4017692861 wait complete. 2024-03-21T10:26:48.234 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:48.234+0000 7f2596000640 1 Processor -- start 2024-03-21T10:26:48.234 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:48.234+0000 7f2596000640 1 -- start start 2024-03-21T10:26:48.238 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:48.234+0000 7f2596000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2590073a40 0x7f259019e4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:26:48.238 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:48.234+0000 7f2596000640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2590074360 0x7f259019ea00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:26:48.238 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:48.234+0000 7f2596000640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2590112640 0x7f25901a2d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:26:48.238 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:48.234+0000 7f2596000640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2590117f40 con 0x7f2590112640 2024-03-21T10:26:48.238 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:48.234+0000 7f2596000640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2590117dc0 con 0x7f2590073a40 2024-03-21T10:26:48.238 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:48.234+0000 7f2596000640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25901180c0 con 0x7f2590074360 2024-03-21T10:26:48.238 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:48.235+0000 7f258fe00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2590112640 0x7f25901a2d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:26:48.238 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:48.235+0000 7f258fe00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2590112640 0x7f25901a2d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:57860/0 (socket says 172.21.15.47:57860) 2024-03-21T10:26:48.238 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:48.235+0000 7f258fe00640 1 -- 172.21.15.47:0/4149706227 learned_addr learned my addr 172.21.15.47:0/4149706227 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:26:48.238 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:48.235+0000 7f258f400640 1 --2- 172.21.15.47:0/4149706227 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2590073a40 0x7f259019e4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:26:48.239 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:48.235+0000 7f258ea00640 1 --2- 172.21.15.47:0/4149706227 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2590074360 0x7f259019ea00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:26:48.239 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:48.235+0000 7f258fe00640 1 -- 172.21.15.47:0/4149706227 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2590074360 msgr2=0x7f259019ea00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:26:48.239 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:48.235+0000 7f258fe00640 1 --2- 172.21.15.47:0/4149706227 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2590074360 0x7f259019ea00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:26:48.239 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:48.235+0000 7f258fe00640 1 -- 172.21.15.47:0/4149706227 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2590073a40 msgr2=0x7f259019e4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:26:48.239 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:48.235+0000 7f258fe00640 1 --2- 172.21.15.47:0/4149706227 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2590073a40 0x7f259019e4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:26:48.239 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:48.235+0000 7f258fe00640 1 -- 172.21.15.47:0/4149706227 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25901a34b0 con 0x7f2590112640 2024-03-21T10:26:48.239 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:48.235+0000 7f258fe00640 1 --2- 172.21.15.47:0/4149706227 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2590112640 0x7f25901a2d80 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7f258002c210 tx=0x7f25800045e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:26:48.239 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:48.236+0000 7f2587e00640 1 -- 172.21.15.47:0/4149706227 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2580043020 con 0x7f2590112640 2024-03-21T10:26:48.239 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:48.236+0000 7f2587e00640 1 -- 172.21.15.47:0/4149706227 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f258002dc60 con 0x7f2590112640 2024-03-21T10:26:48.239 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:48.236+0000 7f2596000640 1 -- 172.21.15.47:0/4149706227 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f25901a3740 con 0x7f2590112640 2024-03-21T10:26:48.239 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:48.236+0000 7f2587e00640 1 -- 172.21.15.47:0/4149706227 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2580003950 con 0x7f2590112640 2024-03-21T10:26:48.239 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:48.236+0000 7f2596000640 1 -- 172.21.15.47:0/4149706227 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f25901ab030 con 0x7f2590112640 2024-03-21T10:26:48.240 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:48.239+0000 7f2598d91740 1 Processor -- start 2024-03-21T10:26:48.240 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:48.240+0000 7f2587e00640 1 -- 172.21.15.47:0/4149706227 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 26) v1 ==== 138087+0+0 (secure 0 0 0) 0x7f2580046020 con 0x7f2590112640 2024-03-21T10:26:48.240 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:48.240+0000 7f2598d91740 1 -- start start 2024-03-21T10:26:48.241 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:48.240+0000 7f2598d91740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55788d3c4c00 0x55788d3c3950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:26:48.241 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:48.241+0000 7f2598d91740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55788d3c3f20 0x55788d3c43a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:26:48.241 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:48.241+0000 7f2598d91740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55788d3cd290 0x55788d3cf680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:26:48.241 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:48.241+0000 7f2598d91740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x55788d328250 con 0x55788d3c3f20 2024-03-21T10:26:48.241 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:48.241+0000 7f2598d91740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55788d2d0a30 con 0x55788d3cd290 2024-03-21T10:26:48.241 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:48.241+0000 7f2598d91740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55788d3242f0 con 0x55788d3c4c00 2024-03-21T10:26:48.241 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:48.241+0000 7f258f400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55788d3c3f20 0x55788d3c43a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:26:48.242 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:48.241+0000 7f258ea00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55788d3cd290 0x55788d3cf680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:26:48.242 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:48.241+0000 7f258f400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55788d3c3f20 0x55788d3c43a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:57874/0 (socket says 172.21.15.47:57874) 2024-03-21T10:26:48.242 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:48.241+0000 7f258f400640 1 -- 172.21.15.47:0/250189328 learned_addr learned my addr 172.21.15.47:0/250189328 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:26:48.242 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:48.241+0000 7f258ea00640 1 --2- 172.21.15.47:0/250189328 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55788d3c4c00 0x55788d3c3950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:26:48.243 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:48.241+0000 7f258f400640 1 -- 172.21.15.47:0/250189328 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55788d3c4c00 msgr2=0x55788d3c3950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:26:48.243 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:48.241+0000 7f258f400640 1 --2- 172.21.15.47:0/250189328 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55788d3c4c00 0x55788d3c3950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:26:48.244 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:48.241+0000 7f258f400640 1 -- 172.21.15.47:0/250189328 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55788d3cd290 msgr2=0x55788d3cf680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:26:48.244 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:48.241+0000 7f258f400640 1 --2- 172.21.15.47:0/250189328 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55788d3cd290 0x55788d3cf680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:26:48.244 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:48.241+0000 7f258f400640 1 -- 172.21.15.47:0/250189328 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55788d2bb690 con 0x55788d3c3f20 2024-03-21T10:26:48.244 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:48.241+0000 7f258ea00640 1 --2- 172.21.15.47:0/250189328 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55788d3c4c00 0x55788d3c3950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T10:26:48.244 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:48.242+0000 7f2587e00640 1 --2- 172.21.15.47:0/4149706227 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f25740a3230 0x7f25740a56f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:26:48.244 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:48.242+0000 7f258f400640 1 --2- 172.21.15.47:0/250189328 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55788d3c3f20 0x55788d3c43a0 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7f25880103f0 tx=0x7f25880108b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e05f8fbdc2313d90 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:26:48.244 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:48.242+0000 7f2587e00640 1 -- 172.21.15.47:0/4149706227 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 9474+0+0 (secure 0 0 0) 0x7f25800f1dd0 con 0x7f2590112640 2024-03-21T10:26:48.244 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:48.242+0000 7f258ea00640 1 --2- 172.21.15.47:0/4149706227 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f25740a3230 0x7f25740a56f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:26:48.244 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:48.242+0000 7f257fe00640 1 -- 172.21.15.47:0/250189328 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2588016070 con 0x55788d3c3f20 2024-03-21T10:26:48.244 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:48.242+0000 7f257fe00640 1 -- 172.21.15.47:0/250189328 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f2588004d20 con 0x55788d3c3f20 2024-03-21T10:26:48.244 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:48.242+0000 7f257fe00640 1 -- 172.21.15.47:0/250189328 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25880053e0 con 0x55788d3c3f20 2024-03-21T10:26:48.245 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:48.243+0000 7f2598d91740 1 -- 172.21.15.47:0/250189328 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55788d3c3f20 msgr2=0x55788d3c43a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:26:48.245 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:48.243+0000 7f258ea00640 1 --2- 172.21.15.47:0/4149706227 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f25740a3230 0x7f25740a56f0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f25900ff8a0 tx=0x7f25780095e0 comp rx=0 tx=0).ready entity=mgr.15687 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:26:48.245 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:48.243+0000 7f2598d91740 1 --2- 172.21.15.47:0/250189328 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55788d3c3f20 0x55788d3c43a0 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7f25880103f0 tx=0x7f25880108b0 comp rx=0 tx=0).stop 2024-03-21T10:26:48.245 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:48.243+0000 7f2598d91740 1 -- 172.21.15.47:0/250189328 shutdown_connections 2024-03-21T10:26:48.245 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:48.243+0000 7f2598d91740 1 --2- 172.21.15.47:0/250189328 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55788d3cd290 0x55788d3cf680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:26:48.245 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:48.243+0000 7f2598d91740 1 --2- 172.21.15.47:0/250189328 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55788d3c3f20 0x55788d3c43a0 unknown :-1 s=CLOSED pgs=624 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:26:48.245 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:48.243+0000 7f2598d91740 1 --2- 172.21.15.47:0/250189328 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55788d3c4c00 0x55788d3c3950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:26:48.245 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:48.243+0000 7f2598d91740 1 -- 172.21.15.47:0/250189328 >> 172.21.15.47:0/250189328 conn(0x55788d38b200 msgr2=0x55788d3be970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:26:48.245 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:48.243+0000 7f2598d91740 1 -- 172.21.15.47:0/250189328 shutdown_connections 2024-03-21T10:26:48.245 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:48.244+0000 7f2598d91740 1 -- 172.21.15.47:0/250189328 wait complete. 2024-03-21T10:26:48.248 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:48.244+0000 7f2598d91740 1 -- start start 2024-03-21T10:26:48.249 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:48.244+0000 7f2598d91740 1 Processor -- start 2024-03-21T10:26:48.249 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:48.244+0000 7f2598d91740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55788d3c3f20 0x55788d3cc920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:26:48.249 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:48.244+0000 7f2598d91740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55788d3c4c00 0x55788d3c9cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:26:48.249 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:48.245+0000 7f2598d91740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55788d3cd290 0x55788d3ca1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:26:48.249 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:48.245+0000 7f2598d91740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x55788d3d0100 con 0x55788d3c3f20 2024-03-21T10:26:48.249 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:48.245+0000 7f2598d91740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55788d2e3e80 con 0x55788d3c4c00 2024-03-21T10:26:48.249 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:48.245+0000 7f2598d91740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55788d3d0280 con 0x55788d3cd290 2024-03-21T10:26:48.249 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:48.245+0000 7f2598d91740 10 client.0 _ll_register_callbacks cb 0x55788d2f6130 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T10:26:48.249 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:48.245+0000 7f258f400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55788d3c3f20 0x55788d3cc920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:26:48.249 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:48.245+0000 7f258f400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55788d3c3f20 0x55788d3cc920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:57880/0 (socket says 172.21.15.47:57880) 2024-03-21T10:26:48.249 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:48.245+0000 7f258f400640 1 -- 172.21.15.47:0/32582145 learned_addr learned my addr 172.21.15.47:0/32582145 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:26:48.249 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:48.245+0000 7f258ea00640 1 --2- 172.21.15.47:0/32582145 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55788d3c4c00 0x55788d3c9cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:26:48.249 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:48.245+0000 7f258fe00640 1 --2- 172.21.15.47:0/32582145 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55788d3cd290 0x55788d3ca1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:26:48.249 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:48.245+0000 7f258f400640 1 -- 172.21.15.47:0/32582145 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55788d3cd290 msgr2=0x55788d3ca1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:26:48.250 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:48.245+0000 7f258f400640 1 --2- 172.21.15.47:0/32582145 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55788d3cd290 0x55788d3ca1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:26:48.250 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:48.245+0000 7f258f400640 1 -- 172.21.15.47:0/32582145 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55788d3c4c00 msgr2=0x55788d3c9cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:26:48.250 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:48.245+0000 7f258f400640 1 --2- 172.21.15.47:0/32582145 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55788d3c4c00 0x55788d3c9cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:26:48.250 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:48.245+0000 7f258f400640 1 -- 172.21.15.47:0/32582145 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55788d1067d0 con 0x55788d3c3f20 2024-03-21T10:26:48.250 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:48.245+0000 7f258fe00640 1 --2- 172.21.15.47:0/32582145 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55788d3cd290 0x55788d3ca1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T10:26:48.250 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:48.246+0000 7f258f400640 1 --2- 172.21.15.47:0/32582145 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55788d3c3f20 0x55788d3cc920 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7f2588005f40 tx=0x7f2588004c70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ec749c6c05c37b0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:26:48.250 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:48.246+0000 7f257cc00640 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-03-21T10:26:48.250 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:48.246+0000 7f257cc00640 1 -- 172.21.15.47:0/32582145 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2588016070 con 0x55788d3c3f20 2024-03-21T10:26:48.250 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:48.246+0000 7f257cc00640 1 -- 172.21.15.47:0/32582145 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(97..97 src has 1..97) v4 ==== 9474+0+0 (secure 0 0 0) 0x7f2588003e00 con 0x55788d3c3f20 2024-03-21T10:26:48.250 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:48.246+0000 7f2598d91740 1 -- 172.21.15.47:0/32582145 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f2588004d20 con 0x55788d3c3f20 2024-03-21T10:26:48.250 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:48.246+0000 7f257cc00640 1 -- 172.21.15.47:0/32582145 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f258802f050 con 0x55788d3c3f20 2024-03-21T10:26:48.250 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:48.246+0000 7f2598d91740 10 client.16185 fetch_fsmap learned FSMap version 32 2024-03-21T10:26:48.250 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:48.246+0000 7f2598d91740 1 -- 172.21.15.47:0/32582145 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x55788d03c3f0 con 0x55788d3c3f20 2024-03-21T10:26:48.251 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:48.247+0000 7f257cc00640 1 -- 172.21.15.47:0/32582145 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f2588004d20 con 0x55788d3c3f20 2024-03-21T10:26:48.251 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:48.247+0000 7f257cc00640 1 -- 172.21.15.47:0/32582145 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 32) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f258800f040 con 0x55788d3c3f20 2024-03-21T10:26:48.251 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:48.247+0000 7f2598d91740 10 client.16185 fetch_fsmap finished waiting for FSMap version 32 2024-03-21T10:26:48.251 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:48.247+0000 7f2598d91740 10 client.16185 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-21T10:26:48.251 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:48.247+0000 7f2598d91740 20 client.16185 populate_metadata read hostname 'smithi047' 2024-03-21T10:26:48.251 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:48.247+0000 7f2598d91740 1 --2- 172.21.15.47:0/32582145 >> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] conn(0x55788d3dd040 0x55788d3df430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:26:48.251 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:48.247+0000 7f2598d91740 4 client.16185 mds_command: new command op to 34535 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T10:26:48.251 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:48.247+0000 7f2598d91740 1 -- 172.21.15.47:0/32582145 --> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55788d335790 con 0x55788d3dd040 2024-03-21T10:26:48.251 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:48.247+0000 7f258ea00640 1 --2- 172.21.15.47:0/32582145 >> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] conn(0x55788d3dd040 0x55788d3df430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:26:48.251 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:48.248+0000 7f258ea00640 1 --2- 172.21.15.47:0/32582145 >> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] conn(0x55788d3dd040 0x55788d3df430 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ccacd2319987793e server_cookie=7d89b9da43ec51ad in_seq=0 out_seq=0 2024-03-21T10:26:48.251 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:48.248+0000 7f257cc00640 10 client.16185 ms_handle_connect on v2:172.21.15.100:6838/4173168440 2024-03-21T10:26:48.252 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:48.250+0000 7f257cc00640 1 -- 172.21.15.47:0/32582145 <== mds.0 v2:172.21.15.100:6838/4173168440 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55788d335790 con 0x55788d3dd040 2024-03-21T10:26:48.252 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:48.250+0000 7f257cc00640 10 client.16185 handle_command_reply: tid=0 2024-03-21T10:26:48.252 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:48.250+0000 7f2598d91740 1 client.16185 shutdown 2024-03-21T10:26:48.252 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:48.250+0000 7f2598d91740 1 -- 172.21.15.47:0/32582145 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55788d3c3f20 msgr2=0x55788d3cc920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:26:48.253 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:48.250+0000 7f2598d91740 1 --2- 172.21.15.47:0/32582145 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55788d3c3f20 0x55788d3cc920 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7f2588005f40 tx=0x7f2588004c70 comp rx=0 tx=0).stop 2024-03-21T10:26:48.253 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:48.250+0000 7f2598d91740 1 -- 172.21.15.47:0/32582145 shutdown_connections 2024-03-21T10:26:48.253 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:48.250+0000 7f2598d91740 1 --2- 172.21.15.47:0/32582145 >> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] conn(0x55788d3dd040 0x55788d3df430 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:26:48.253 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:48.250+0000 7f2598d91740 1 --2- 172.21.15.47:0/32582145 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55788d3cd290 0x55788d3ca1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:26:48.253 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:48.250+0000 7f2598d91740 1 --2- 172.21.15.47:0/32582145 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55788d3c4c00 0x55788d3c9cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:26:48.253 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:48.250+0000 7f2598d91740 1 --2- 172.21.15.47:0/32582145 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55788d3c3f20 0x55788d3cc920 unknown :-1 s=CLOSED pgs=625 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:26:48.253 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:48.250+0000 7f2598d91740 1 -- 172.21.15.47:0/32582145 >> 172.21.15.47:0/32582145 conn(0x55788d38b200 msgr2=0x55788d3c06c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:26:48.253 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:48.250+0000 7f257cc00640 0 client.16185 ms_handle_reset on v2:172.21.15.100:6838/4173168440 2024-03-21T10:26:48.253 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:48.250+0000 7f2598d91740 1 -- 172.21.15.47:0/32582145 shutdown_connections 2024-03-21T10:26:48.253 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:48.250+0000 7f2598d91740 1 -- 172.21.15.47:0/32582145 wait complete. 2024-03-21T10:26:48.253 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:48.250+0000 7f2598d91740 20 client.16185 trim_cache size 0 max 16384 2024-03-21T10:26:48.271 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:48.270+0000 7f2598d91740 1 Processor -- start 2024-03-21T10:26:48.272 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:48.270+0000 7f2598d91740 1 -- start start 2024-03-21T10:26:48.272 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:48.270+0000 7f2598d91740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55788d3c3f20 0x55788d3cc3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:26:48.272 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:48.270+0000 7f2598d91740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55788d3c4c00 0x55788d3cc920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:26:48.272 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:48.270+0000 7f2598d91740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55788d3cd290 0x55788d3ce680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:26:48.272 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:48.270+0000 7f2598d91740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25901b4f40 con 0x55788d3c4c00 2024-03-21T10:26:48.272 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:48.270+0000 7f2598d91740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25901b68c0 con 0x55788d3cd290 2024-03-21T10:26:48.272 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:48.270+0000 7f2598d91740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25901b55a0 con 0x55788d3c3f20 2024-03-21T10:26:48.272 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:48.271+0000 7f258ea00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55788d3c4c00 0x55788d3cc920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:26:48.272 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:48.271+0000 7f258ea00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55788d3c4c00 0x55788d3cc920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:57882/0 (socket says 172.21.15.47:57882) 2024-03-21T10:26:48.272 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:48.271+0000 7f258ea00640 1 -- 172.21.15.47:0/1271430108 learned_addr learned my addr 172.21.15.47:0/1271430108 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:26:48.272 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:48.271+0000 7f258fe00640 1 --2- 172.21.15.47:0/1271430108 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55788d3cd290 0x55788d3ce680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:26:48.272 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:48.271+0000 7f258f400640 1 --2- 172.21.15.47:0/1271430108 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55788d3c3f20 0x55788d3cc3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:26:48.272 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:48.271+0000 7f258ea00640 1 -- 172.21.15.47:0/1271430108 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55788d3c3f20 msgr2=0x55788d3cc3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:26:48.273 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:48.271+0000 7f258ea00640 1 --2- 172.21.15.47:0/1271430108 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55788d3c3f20 0x55788d3cc3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:26:48.273 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:48.271+0000 7f258ea00640 1 -- 172.21.15.47:0/1271430108 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55788d3cd290 msgr2=0x55788d3ce680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:26:48.273 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:48.271+0000 7f258ea00640 1 --2- 172.21.15.47:0/1271430108 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55788d3cd290 0x55788d3ce680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:26:48.273 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:48.271+0000 7f258ea00640 1 -- 172.21.15.47:0/1271430108 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55788cffc0f0 con 0x55788d3c4c00 2024-03-21T10:26:48.273 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:48.271+0000 7f258fe00640 1 --2- 172.21.15.47:0/1271430108 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55788d3cd290 0x55788d3ce680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T10:26:48.273 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:48.271+0000 7f258f400640 1 --2- 172.21.15.47:0/1271430108 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55788d3c3f20 0x55788d3cc3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T10:26:48.275 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:48.272+0000 7f258ea00640 1 --2- 172.21.15.47:0/1271430108 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55788d3c4c00 0x55788d3cc920 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x55788d3c9510 tx=0x7f25780046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a21e677b6bc08e09 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:26:48.276 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:48.272+0000 7f257f400640 1 -- 172.21.15.47:0/1271430108 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f257802bcd0 con 0x55788d3c4c00 2024-03-21T10:26:48.276 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:48.272+0000 7f257f400640 1 -- 172.21.15.47:0/1271430108 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f2578031070 con 0x55788d3c4c00 2024-03-21T10:26:48.276 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:48.272+0000 7f257f400640 1 -- 172.21.15.47:0/1271430108 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f257800f420 con 0x55788d3c4c00 2024-03-21T10:26:48.276 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:48.273+0000 7f2598d91740 1 -- 172.21.15.47:0/1271430108 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55788d3c4c00 msgr2=0x55788d3cc920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:26:48.276 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:48.273+0000 7f2598d91740 1 --2- 172.21.15.47:0/1271430108 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55788d3c4c00 0x55788d3cc920 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x55788d3c9510 tx=0x7f25780046c0 comp rx=0 tx=0).stop 2024-03-21T10:26:48.276 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:48.273+0000 7f2598d91740 1 -- 172.21.15.47:0/1271430108 shutdown_connections 2024-03-21T10:26:48.276 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:48.273+0000 7f2598d91740 1 --2- 172.21.15.47:0/1271430108 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55788d3cd290 0x55788d3ce680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:26:48.276 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:48.273+0000 7f2598d91740 1 --2- 172.21.15.47:0/1271430108 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55788d3c4c00 0x55788d3cc920 unknown :-1 s=CLOSED pgs=626 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:26:48.276 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:48.273+0000 7f2598d91740 1 --2- 172.21.15.47:0/1271430108 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55788d3c3f20 0x55788d3cc3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:26:48.276 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:48.273+0000 7f2598d91740 1 -- 172.21.15.47:0/1271430108 >> 172.21.15.47:0/1271430108 conn(0x55788d38b200 msgr2=0x55788d3c8580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:26:48.276 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:48.273+0000 7f2598d91740 1 -- 172.21.15.47:0/1271430108 shutdown_connections 2024-03-21T10:26:48.276 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:48.273+0000 7f2598d91740 1 -- 172.21.15.47:0/1271430108 wait complete. 2024-03-21T10:26:48.276 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:48.274+0000 7f2598d91740 1 -- start start 2024-03-21T10:26:48.277 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:48.274+0000 7f2598d91740 1 Processor -- start 2024-03-21T10:26:48.277 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:48.274+0000 7f2598d91740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55788d3c3f20 0x55788d3c6b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:26:48.277 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:48.274+0000 7f2598d91740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55788d3c4c00 0x55788d3c70d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:26:48.277 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:48.275+0000 7f2598d91740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55788d3cd290 0x55788d4315f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:26:48.277 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:48.275+0000 7f2598d91740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25901b6260 con 0x55788d3c3f20 2024-03-21T10:26:48.277 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:48.275+0000 7f2598d91740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25901b5c00 con 0x55788d3c4c00 2024-03-21T10:26:48.277 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:48.275+0000 7f2598d91740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25901b3100 con 0x55788d3cd290 2024-03-21T10:26:48.277 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:48.275+0000 7f2598d91740 10 client.0 _ll_register_callbacks cb 0x7f2578009910 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T10:26:48.277 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:48.275+0000 7f258f400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55788d3c3f20 0x55788d3c6b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:26:48.277 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:48.275+0000 7f258ea00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55788d3c4c00 0x55788d3c70d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:26:48.277 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:48.275+0000 7f258f400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55788d3c3f20 0x55788d3c6b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:57890/0 (socket says 172.21.15.47:57890) 2024-03-21T10:26:48.277 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:48.275+0000 7f258ea00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55788d3c4c00 0x55788d3c70d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:52216/0 (socket says 172.21.15.47:52216) 2024-03-21T10:26:48.277 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:48.275+0000 7f258f400640 1 -- 172.21.15.47:0/4064317362 learned_addr learned my addr 172.21.15.47:0/4064317362 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:26:48.277 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:48.275+0000 7f258fe00640 1 --2- 172.21.15.47:0/4064317362 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55788d3cd290 0x55788d4315f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:26:48.278 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:48.275+0000 7f258f400640 1 -- 172.21.15.47:0/4064317362 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55788d3cd290 msgr2=0x55788d4315f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:26:48.278 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:48.275+0000 7f258f400640 1 --2- 172.21.15.47:0/4064317362 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55788d3cd290 0x55788d4315f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:26:48.278 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:48.275+0000 7f258f400640 1 -- 172.21.15.47:0/4064317362 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55788d3c4c00 msgr2=0x55788d3c70d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:26:48.278 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:48.275+0000 7f258f400640 1 --2- 172.21.15.47:0/4064317362 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55788d3c4c00 0x55788d3c70d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:26:48.278 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:48.275+0000 7f258f400640 1 -- 172.21.15.47:0/4064317362 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55788d431c60 con 0x55788d3c3f20 2024-03-21T10:26:48.278 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:48.275+0000 7f258ea00640 1 --2- 172.21.15.47:0/4064317362 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55788d3c4c00 0x55788d3c70d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T10:26:48.278 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:48.276+0000 7f258f400640 1 --2- 172.21.15.47:0/4064317362 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55788d3c3f20 0x55788d3c6b90 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7f2588012e40 tx=0x7f2588012e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8e5e89b087cb7c19 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:26:48.278 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:48.276+0000 7f257cc00640 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-03-21T10:26:48.280 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:48.276+0000 7f257cc00640 1 -- 172.21.15.47:0/4064317362 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2588039060 con 0x55788d3c3f20 2024-03-21T10:26:48.280 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:48.276+0000 7f257cc00640 1 -- 172.21.15.47:0/4064317362 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(97..97 src has 1..97) v4 ==== 9474+0+0 (secure 0 0 0) 0x7f2588012710 con 0x55788d3c3f20 2024-03-21T10:26:48.280 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:48.276+0000 7f2598d91740 1 -- 172.21.15.47:0/4064317362 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f2578031070 con 0x55788d3c3f20 2024-03-21T10:26:48.281 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:48.276+0000 7f257cc00640 1 -- 172.21.15.47:0/4064317362 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2588016040 con 0x55788d3c3f20 2024-03-21T10:26:48.281 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:48.276+0000 7f2598d91740 10 client.16191 fetch_fsmap learned FSMap version 32 2024-03-21T10:26:48.281 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:48.276+0000 7f2598d91740 1 -- 172.21.15.47:0/4064317362 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x55788d438f80 con 0x55788d3c3f20 2024-03-21T10:26:48.281 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:48.277+0000 7f257cc00640 1 -- 172.21.15.47:0/4064317362 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f2578031070 con 0x55788d3c3f20 2024-03-21T10:26:48.281 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:48.277+0000 7f257cc00640 1 -- 172.21.15.47:0/4064317362 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 32) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f258801b570 con 0x55788d3c3f20 2024-03-21T10:26:48.281 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:48.277+0000 7f2598d91740 10 client.16191 fetch_fsmap finished waiting for FSMap version 32 2024-03-21T10:26:48.281 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:48.277+0000 7f2598d91740 10 client.16191 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-21T10:26:48.281 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:48.277+0000 7f2598d91740 20 client.16191 populate_metadata read hostname 'smithi047' 2024-03-21T10:26:48.281 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:48.277+0000 7f2598d91740 1 --2- 172.21.15.47:0/4064317362 >> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] conn(0x55788d439650 0x55788d43ba70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:26:48.281 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:48.277+0000 7f2598d91740 4 client.16191 mds_command: new command op to 34535 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-21T10:26:48.281 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:48.277+0000 7f2598d91740 1 -- 172.21.15.47:0/4064317362 --> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55788d32fae0 con 0x55788d439650 2024-03-21T10:26:48.281 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:48.277+0000 7f258ea00640 1 --2- 172.21.15.47:0/4064317362 >> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] conn(0x55788d439650 0x55788d43ba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:26:48.281 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:48.278+0000 7f258ea00640 1 --2- 172.21.15.47:0/4064317362 >> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] conn(0x55788d439650 0x55788d43ba70 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c05c565bdae51ec0 server_cookie=8e7578244a71c5b9 in_seq=0 out_seq=0 2024-03-21T10:26:48.281 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:48.278+0000 7f257cc00640 10 client.16191 ms_handle_connect on v2:172.21.15.100:6838/4173168440 2024-03-21T10:26:48.281 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:48.278+0000 7f257cc00640 1 -- 172.21.15.47:0/4064317362 <== mds.0 v2:172.21.15.100:6838/4173168440 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x55788d32fae0 con 0x55788d439650 2024-03-21T10:26:48.282 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:48.278+0000 7f257cc00640 10 client.16191 handle_command_reply: tid=0 2024-03-21T10:26:48.282 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:48.278+0000 7f2598d91740 1 client.16191 shutdown 2024-03-21T10:26:48.282 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:48.279+0000 7f2598d91740 1 -- 172.21.15.47:0/4064317362 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55788d3c3f20 msgr2=0x55788d3c6b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:26:48.282 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:48.279+0000 7f2598d91740 1 --2- 172.21.15.47:0/4064317362 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55788d3c3f20 0x55788d3c6b90 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7f2588012e40 tx=0x7f2588012e70 comp rx=0 tx=0).stop 2024-03-21T10:26:48.282 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:48.279+0000 7f2598d91740 1 -- 172.21.15.47:0/4064317362 shutdown_connections 2024-03-21T10:26:48.282 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:48.279+0000 7f2598d91740 1 --2- 172.21.15.47:0/4064317362 >> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] conn(0x55788d439650 0x55788d43ba70 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:26:48.282 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:48.279+0000 7f2598d91740 1 --2- 172.21.15.47:0/4064317362 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55788d3cd290 0x55788d4315f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:26:48.282 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:48.279+0000 7f2598d91740 1 --2- 172.21.15.47:0/4064317362 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55788d3c4c00 0x55788d3c70d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:26:48.282 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:48.279+0000 7f2598d91740 1 --2- 172.21.15.47:0/4064317362 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55788d3c3f20 0x55788d3c6b90 unknown :-1 s=CLOSED pgs=627 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:26:48.282 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:48.279+0000 7f2598d91740 1 -- 172.21.15.47:0/4064317362 >> 172.21.15.47:0/4064317362 conn(0x55788d38b200 msgr2=0x55788d3bd200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:26:48.282 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:48.279+0000 7f257cc00640 0 client.16191 ms_handle_reset on v2:172.21.15.100:6838/4173168440 2024-03-21T10:26:48.282 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:48.279+0000 7f2598d91740 1 -- 172.21.15.47:0/4064317362 shutdown_connections 2024-03-21T10:26:48.282 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:48.279+0000 7f2598d91740 1 -- 172.21.15.47:0/4064317362 wait complete. 2024-03-21T10:26:48.282 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:48.279+0000 7f2598d91740 20 client.16191 trim_cache size 0 max 16384 2024-03-21T10:26:48.283 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-03-21T10:26:48.283 INFO:teuthology.orchestra.run.smithi047.stdout: "status": "no active scrubs running", 2024-03-21T10:26:48.283 INFO:teuthology.orchestra.run.smithi047.stdout: "scrubs": {} 2024-03-21T10:26:48.283 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-03-21T10:26:48.284 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:48.280+0000 7f2596000640 1 -- 172.21.15.47:0/4149706227 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f25740a3230 msgr2=0x7f25740a56f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:26:48.284 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:48.280+0000 7f2596000640 1 --2- 172.21.15.47:0/4149706227 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f25740a3230 0x7f25740a56f0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f25900ff8a0 tx=0x7f25780095e0 comp rx=0 tx=0).stop 2024-03-21T10:26:48.284 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:48.280+0000 7f2596000640 1 -- 172.21.15.47:0/4149706227 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2590112640 msgr2=0x7f25901a2d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:26:48.284 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:48.280+0000 7f2596000640 1 --2- 172.21.15.47:0/4149706227 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2590112640 0x7f25901a2d80 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7f258002c210 tx=0x7f25800045e0 comp rx=0 tx=0).stop 2024-03-21T10:26:48.284 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:48.280+0000 7f2596000640 1 -- 172.21.15.47:0/4149706227 shutdown_connections 2024-03-21T10:26:48.284 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:48.280+0000 7f2596000640 1 --2- 172.21.15.47:0/4149706227 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2590112640 0x7f25901a2d80 unknown :-1 s=CLOSED pgs=623 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:26:48.284 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:48.280+0000 7f2596000640 1 --2- 172.21.15.47:0/4149706227 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2590074360 0x7f259019ea00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:26:48.284 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:48.280+0000 7f2596000640 1 --2- 172.21.15.47:0/4149706227 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f25740a3230 0x7f25740a56f0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:26:48.284 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:48.280+0000 7f2596000640 1 --2- 172.21.15.47:0/4149706227 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2590073a40 0x7f259019e4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:26:48.284 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:48.280+0000 7f2596000640 1 -- 172.21.15.47:0/4149706227 >> 172.21.15.47:0/4149706227 conn(0x7f25900fd140 msgr2=0x7f25900fea40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:26:48.284 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:48.280+0000 7f2596000640 1 -- 172.21.15.47:0/4149706227 shutdown_connections 2024-03-21T10:26:48.285 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:48.280+0000 7f2596000640 1 -- 172.21.15.47:0/4149706227 wait complete. 2024-03-21T10:26:48.298 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-03-21T10:26:48.298 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-21T10:26:48.506 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:48.505+0000 7f34d5a00640 1 -- 172.21.15.47:0/3598883971 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f34d01056e0 msgr2=0x7f34d010d2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:26:48.506 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:48.505+0000 7f34d5a00640 1 --2- 172.21.15.47:0/3598883971 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f34d01056e0 0x7f34d010d2b0 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7f34c8009ca0 tx=0x7f34c802c0f0 comp rx=0 tx=0).stop 2024-03-21T10:26:48.506 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:48.505+0000 7f34d5a00640 1 -- 172.21.15.47:0/3598883971 shutdown_connections 2024-03-21T10:26:48.506 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:48.505+0000 7f34d5a00640 1 --2- 172.21.15.47:0/3598883971 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f34d01056e0 0x7f34d010d2b0 unknown :-1 s=CLOSED pgs=628 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:26:48.506 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:48.505+0000 7f34d5a00640 1 --2- 172.21.15.47:0/3598883971 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f34d0101270 0x7f34d0105000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:26:48.506 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:48.505+0000 7f34d5a00640 1 --2- 172.21.15.47:0/3598883971 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f34d01008c0 0x7f34d0100ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:26:48.506 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:48.505+0000 7f34d5a00640 1 -- 172.21.15.47:0/3598883971 >> 172.21.15.47:0/3598883971 conn(0x7f34d0075fb0 msgr2=0x7f34d00763c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:26:48.506 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:48.505+0000 7f34d5a00640 1 -- 172.21.15.47:0/3598883971 shutdown_connections 2024-03-21T10:26:48.506 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:48.505+0000 7f34d5a00640 1 -- 172.21.15.47:0/3598883971 wait complete. 2024-03-21T10:26:48.508 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:48.506+0000 7f34d5a00640 1 Processor -- start 2024-03-21T10:26:48.508 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:48.507+0000 7f34d5a00640 1 -- start start 2024-03-21T10:26:48.509 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:48.507+0000 7f34d5a00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f34d01008c0 0x7f34d019d900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:26:48.509 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:48.507+0000 7f34d5a00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f34d01056e0 0x7f34d019de40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:26:48.509 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:48.507+0000 7f34d5a00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f34d0197ab0 0x7f34d0197f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:26:48.509 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:48.507+0000 7f34d5a00640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34d0110650 con 0x7f34d0197ab0 2024-03-21T10:26:48.509 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:48.507+0000 7f34d5a00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34d01104d0 con 0x7f34d01056e0 2024-03-21T10:26:48.509 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:48.507+0000 7f34d5a00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34d01107d0 con 0x7f34d01008c0 2024-03-21T10:26:48.509 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:48.507+0000 7f34d5000640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f34d0197ab0 0x7f34d0197f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:26:48.509 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:48.507+0000 7f34d5000640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f34d0197ab0 0x7f34d0197f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:57912/0 (socket says 172.21.15.47:57912) 2024-03-21T10:26:48.509 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:48.507+0000 7f34d5000640 1 -- 172.21.15.47:0/1919726689 learned_addr learned my addr 172.21.15.47:0/1919726689 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:26:48.509 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:48.507+0000 7f34c7e00640 1 --2- 172.21.15.47:0/1919726689 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f34d01056e0 0x7f34d019de40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:26:48.509 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:48.507+0000 7f34cfe00640 1 --2- 172.21.15.47:0/1919726689 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f34d01008c0 0x7f34d019d900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:26:48.509 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:48.508+0000 7f34d5000640 1 -- 172.21.15.47:0/1919726689 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f34d01008c0 msgr2=0x7f34d019d900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:26:48.509 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:48.508+0000 7f34d5000640 1 --2- 172.21.15.47:0/1919726689 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f34d01008c0 0x7f34d019d900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:26:48.509 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:48.508+0000 7f34d5000640 1 -- 172.21.15.47:0/1919726689 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f34d01056e0 msgr2=0x7f34d019de40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:26:48.510 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:48.508+0000 7f34d5000640 1 --2- 172.21.15.47:0/1919726689 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f34d01056e0 0x7f34d019de40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:26:48.510 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:48.508+0000 7f34d5000640 1 -- 172.21.15.47:0/1919726689 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f34d01985d0 con 0x7f34d0197ab0 2024-03-21T10:26:48.510 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:48.508+0000 7f34d5000640 1 --2- 172.21.15.47:0/1919726689 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f34d0197ab0 0x7f34d0197f60 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7f34c802c700 tx=0x7f34c8035d60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:26:48.510 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:48.508+0000 7f34cd600640 1 -- 172.21.15.47:0/1919726689 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34c8039040 con 0x7f34d0197ab0 2024-03-21T10:26:48.510 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:48.508+0000 7f34d5a00640 1 -- 172.21.15.47:0/1919726689 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f34d0198860 con 0x7f34d0197ab0 2024-03-21T10:26:48.510 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:48.508+0000 7f34cd600640 1 -- 172.21.15.47:0/1919726689 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f34c8044070 con 0x7f34d0197ab0 2024-03-21T10:26:48.510 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:48.508+0000 7f34cd600640 1 -- 172.21.15.47:0/1919726689 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34c8004330 con 0x7f34d0197ab0 2024-03-21T10:26:48.510 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:48.509+0000 7f34d5a00640 1 -- 172.21.15.47:0/1919726689 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f34d01a4ab0 con 0x7f34d0197ab0 2024-03-21T10:26:48.510 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:48.510+0000 7f34d5a00640 1 -- 172.21.15.47:0/1919726689 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f34d00683b0 con 0x7f34d0197ab0 2024-03-21T10:26:48.511 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:48.512+0000 7f34cd600640 1 -- 172.21.15.47:0/1919726689 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 26) v1 ==== 138087+0+0 (secure 0 0 0) 0x7f34c8003970 con 0x7f34d0197ab0 2024-03-21T10:26:48.513 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:48.513+0000 7f34cd600640 1 --2- 172.21.15.47:0/1919726689 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f34b40a3230 0x7f34b40a56f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:26:48.513 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:48.513+0000 7f34cd600640 1 -- 172.21.15.47:0/1919726689 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 9474+0+0 (secure 0 0 0) 0x7f34c80f28d0 con 0x7f34d0197ab0 2024-03-21T10:26:48.513 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:48.513+0000 7f34cfe00640 1 --2- 172.21.15.47:0/1919726689 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f34b40a3230 0x7f34b40a56f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:26:48.514 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:48.514+0000 7f34cfe00640 1 --2- 172.21.15.47:0/1919726689 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f34b40a3230 0x7f34b40a56f0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f34c0005fd0 tx=0x7f34c0005850 comp rx=0 tx=0).ready entity=mgr.15687 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:26:48.521 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:48.521+0000 7f34cd600640 1 -- 172.21.15.47:0/1919726689 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f34c80bee60 con 0x7f34d0197ab0 2024-03-21T10:26:48.757 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:48.757+0000 7f34d5a00640 1 -- 172.21.15.47:0/1919726689 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f34d0198fe0 con 0x7f34d0197ab0 2024-03-21T10:26:48.759 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:48.759+0000 7f34cd600640 1 -- 172.21.15.47:0/1919726689 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 32 v32) v1 ==== 94+0+11471 (secure 0 0 0) 0x7f34d0198fe0 con 0x7f34d0197ab0 2024-03-21T10:26:48.760 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-21T10:26:48.760 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":32,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14511,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6835/735724037","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6834","nonce":735724037},{"type":"v1","addr":"172.21.15.47:6835","nonce":735724037}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14529,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6837/2248191724","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6836","nonce":2248191724},{"type":"v1","addr":"172.21.15.47:6837","nonce":2248191724}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":32,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-21T10:21:58.329063+0000","modified":"2024-03-21T10:26:17.169892+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":34535,"mds_1":34493,"mds_2":34454,"mds_3":34427,"mds_4":14619},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14604":{"gid":14604,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.47:6839/2881335356","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6838","nonce":2881335356},{"type":"v1","addr":"172.21.15.47:6839","nonce":2881335356}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_14619":{"gid":14619,"name":"a","rank":4,"incarnation":27,"state":"up:active","state_seq":11,"addr":"172.21.15.47:6841/3401578978","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6840","nonce":3401578978},{"type":"v1","addr":"172.21.15.47:6841","nonce":3401578978}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24406":{"gid":24406,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.157:6833/4000010136","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":4000010136},{"type":"v1","addr":"172.21.15.157:6833","nonce":4000010136}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24457":{"gid":24457,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.157:6835/3902868651","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":3902868651},{"type":"v1","addr":"172.21.15.157:6835","nonce":3902868651}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24475":{"gid":24475,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.157:6837/1908087204","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6836","nonce":1908087204},{"type":"v1","addr":"172.21.15.157:6837","nonce":1908087204}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_25729":{"gid":25729,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.157:6839/1998408416","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6838","nonce":1998408416},{"type":"v1","addr":"172.21.15.157:6839","nonce":1998408416}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_34427":{"gid":34427,"name":"k","rank":3,"incarnation":25,"state":"up:active","state_seq":20,"addr":"172.21.15.100:6833/2723302038","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6832","nonce":2723302038},{"type":"v1","addr":"172.21.15.100:6833","nonce":2723302038}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_34454":{"gid":34454,"name":"e","rank":2,"incarnation":22,"state":"up:active","state_seq":17,"addr":"172.21.15.100:6835/2597849957","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":2597849957},{"type":"v1","addr":"172.21.15.100:6835","nonce":2597849957}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_34493":{"gid":34493,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":13,"addr":"172.21.15.100:6837/3497644162","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6836","nonce":3497644162},{"type":"v1","addr":"172.21.15.100:6837","nonce":3497644162}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_34535":{"gid":34535,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.100:6839/4173168440","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6838","nonce":4173168440},{"type":"v1","addr":"172.21.15.100:6839","nonce":4173168440}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":34535,"qdb_cluster":[34427,14619,34454,34535,34493]},"id":1}]} 2024-03-21T10:26:48.760 INFO:teuthology.orchestra.run.smithi047.stderr:dumped fsmap epoch 32 2024-03-21T10:26:48.762 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:48.762+0000 7f34d5a00640 1 -- 172.21.15.47:0/1919726689 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f34b40a3230 msgr2=0x7f34b40a56f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:26:48.762 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:48.762+0000 7f34d5a00640 1 --2- 172.21.15.47:0/1919726689 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f34b40a3230 0x7f34b40a56f0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f34c0005fd0 tx=0x7f34c0005850 comp rx=0 tx=0).stop 2024-03-21T10:26:48.762 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:48.762+0000 7f34d5a00640 1 -- 172.21.15.47:0/1919726689 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f34d0197ab0 msgr2=0x7f34d0197f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:26:48.762 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:48.762+0000 7f34d5a00640 1 --2- 172.21.15.47:0/1919726689 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f34d0197ab0 0x7f34d0197f60 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7f34c802c700 tx=0x7f34c8035d60 comp rx=0 tx=0).stop 2024-03-21T10:26:48.763 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:48.762+0000 7f34d5a00640 1 -- 172.21.15.47:0/1919726689 shutdown_connections 2024-03-21T10:26:48.763 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:48.762+0000 7f34d5a00640 1 --2- 172.21.15.47:0/1919726689 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f34d0197ab0 0x7f34d0197f60 unknown :-1 s=CLOSED pgs=629 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:26:48.763 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:48.762+0000 7f34d5a00640 1 --2- 172.21.15.47:0/1919726689 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f34b40a3230 0x7f34b40a56f0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:26:48.763 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:48.762+0000 7f34d5a00640 1 --2- 172.21.15.47:0/1919726689 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f34d01056e0 0x7f34d019de40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:26:48.763 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:48.762+0000 7f34d5a00640 1 --2- 172.21.15.47:0/1919726689 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f34d01008c0 0x7f34d019d900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:26:48.764 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:48.763+0000 7f34d5a00640 1 -- 172.21.15.47:0/1919726689 >> 172.21.15.47:0/1919726689 conn(0x7f34d0075fb0 msgr2=0x7f34d0068b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:26:48.764 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:48.763+0000 7f34d5a00640 1 -- 172.21.15.47:0/1919726689 shutdown_connections 2024-03-21T10:26:48.764 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:48.763+0000 7f34d5a00640 1 -- 172.21.15.47:0/1919726689 wait complete. 2024-03-21T10:26:48.780 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-21T10:26:48.780 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-03-21T10:26:48.980 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:48.979+0000 7fef9cc00640 1 -- 172.21.15.47:0/14342114 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fef98074370 msgr2=0x7fef9810d890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:26:48.980 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:48.979+0000 7fef9cc00640 1 --2- 172.21.15.47:0/14342114 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fef98074370 0x7fef9810d890 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7fef84009900 tx=0x7fef8402bcd0 comp rx=0 tx=0).stop 2024-03-21T10:26:48.980 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:48.979+0000 7fef9cc00640 1 -- 172.21.15.47:0/14342114 shutdown_connections 2024-03-21T10:26:48.980 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:48.979+0000 7fef9cc00640 1 --2- 172.21.15.47:0/14342114 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fef9810ddd0 0x7fef981101c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:26:48.980 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:48.979+0000 7fef9cc00640 1 --2- 172.21.15.47:0/14342114 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fef98074370 0x7fef9810d890 unknown :-1 s=CLOSED pgs=630 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:26:48.980 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:48.979+0000 7fef9cc00640 1 --2- 172.21.15.47:0/14342114 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fef98073a50 0x7fef98073e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:26:48.981 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:48.979+0000 7fef9cc00640 1 -- 172.21.15.47:0/14342114 >> 172.21.15.47:0/14342114 conn(0x7fef980fd040 msgr2=0x7fef980ff460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:26:48.981 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:48.979+0000 7fef9cc00640 1 -- 172.21.15.47:0/14342114 shutdown_connections 2024-03-21T10:26:48.981 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:48.979+0000 7fef9cc00640 1 -- 172.21.15.47:0/14342114 wait complete. 2024-03-21T10:26:48.983 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:48.980+0000 7fef9cc00640 1 Processor -- start 2024-03-21T10:26:48.983 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:48.980+0000 7fef9cc00640 1 -- start start 2024-03-21T10:26:48.983 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:48.981+0000 7fef9cc00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fef98073a50 0x7fef981014f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:26:48.983 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:48.981+0000 7fef9cc00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fef98074370 0x7fef98101a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:26:48.983 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:48.981+0000 7fef9cc00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fef9810ddd0 0x7fef98101f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:26:48.983 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:48.981+0000 7fef9cc00640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef981139a0 con 0x7fef98074370 2024-03-21T10:26:48.983 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:48.981+0000 7fef9cc00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef98113820 con 0x7fef98073a50 2024-03-21T10:26:48.983 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:48.981+0000 7fef9cc00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef98113b20 con 0x7fef9810ddd0 2024-03-21T10:26:48.984 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:48.981+0000 7fef96a00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fef98074370 0x7fef98101a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:26:48.984 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:48.981+0000 7fef96a00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fef98074370 0x7fef98101a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:57928/0 (socket says 172.21.15.47:57928) 2024-03-21T10:26:48.984 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:48.981+0000 7fef96a00640 1 -- 172.21.15.47:0/2488326357 learned_addr learned my addr 172.21.15.47:0/2488326357 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:26:48.984 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:48.981+0000 7fef97400640 1 --2- 172.21.15.47:0/2488326357 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fef98073a50 0x7fef981014f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:26:48.984 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:48.982+0000 7fef97e00640 1 --2- 172.21.15.47:0/2488326357 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fef9810ddd0 0x7fef98101f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:26:48.984 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:48.982+0000 7fef96a00640 1 -- 172.21.15.47:0/2488326357 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fef9810ddd0 msgr2=0x7fef98101f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:26:48.984 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:48.982+0000 7fef96a00640 1 --2- 172.21.15.47:0/2488326357 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fef9810ddd0 0x7fef98101f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:26:48.984 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:48.982+0000 7fef96a00640 1 -- 172.21.15.47:0/2488326357 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fef98073a50 msgr2=0x7fef981014f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:26:48.985 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:48.982+0000 7fef96a00640 1 --2- 172.21.15.47:0/2488326357 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fef98073a50 0x7fef981014f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:26:48.985 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:48.982+0000 7fef96a00640 1 -- 172.21.15.47:0/2488326357 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef981aa5e0 con 0x7fef98074370 2024-03-21T10:26:48.985 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:48.982+0000 7fef96a00640 1 --2- 172.21.15.47:0/2488326357 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fef98074370 0x7fef98101a30 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7fef84006010 tx=0x7fef840045e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:26:48.985 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:48.982+0000 7fef8fe00640 1 -- 172.21.15.47:0/2488326357 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef84042020 con 0x7fef98074370 2024-03-21T10:26:48.985 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:48.982+0000 7fef8fe00640 1 -- 172.21.15.47:0/2488326357 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fef8402dba0 con 0x7fef98074370 2024-03-21T10:26:48.985 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:48.982+0000 7fef8fe00640 1 -- 172.21.15.47:0/2488326357 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef84003970 con 0x7fef98074370 2024-03-21T10:26:48.985 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:48.982+0000 7fef9cc00640 1 -- 172.21.15.47:0/2488326357 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fef981aa870 con 0x7fef98074370 2024-03-21T10:26:48.985 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:48.983+0000 7fef9cc00640 1 -- 172.21.15.47:0/2488326357 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fef981aae10 con 0x7fef98074370 2024-03-21T10:26:48.986 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:48.986+0000 7fefa0e46740 1 Processor -- start 2024-03-21T10:26:48.986 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:48.986+0000 7fefa0e46740 1 -- start start 2024-03-21T10:26:48.987 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:48.986+0000 7fef8fe00640 1 -- 172.21.15.47:0/2488326357 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 26) v1 ==== 138087+0+0 (secure 0 0 0) 0x7fef84045020 con 0x7fef98074370 2024-03-21T10:26:48.987 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:48.986+0000 7fefa0e46740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5626c62858b0 0x5626c6285c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:26:48.987 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:48.986+0000 7fefa0e46740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5626c6286260 0x5626c628e810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:26:48.987 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:48.986+0000 7fefa0e46740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5626c628ede0 0x5626c6291230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:26:48.987 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:48.986+0000 7fefa0e46740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x5626c61e92c0 con 0x5626c628ede0 2024-03-21T10:26:48.987 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:48.986+0000 7fefa0e46740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x5626c6191c90 con 0x5626c62858b0 2024-03-21T10:26:48.987 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:48.986+0000 7fefa0e46740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x5626c61e5360 con 0x5626c6286260 2024-03-21T10:26:48.990 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:48.986+0000 7fef96a00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5626c628ede0 0x5626c6291230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:26:48.990 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:48.986+0000 7fef97e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5626c62858b0 0x5626c6285c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:26:48.990 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:48.987+0000 7fef97e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5626c62858b0 0x5626c6285c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:52258/0 (socket says 172.21.15.47:52258) 2024-03-21T10:26:48.990 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:48.987+0000 7fef97e00640 1 -- 172.21.15.47:0/3453211489 learned_addr learned my addr 172.21.15.47:0/3453211489 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:26:48.990 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:48.987+0000 7fef97400640 1 --2- 172.21.15.47:0/3453211489 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5626c6286260 0x5626c628e810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:26:48.991 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:48.987+0000 7fef97e00640 1 -- 172.21.15.47:0/3453211489 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5626c6286260 msgr2=0x5626c628e810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:26:48.991 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:48.987+0000 7fef97e00640 1 --2- 172.21.15.47:0/3453211489 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5626c6286260 0x5626c628e810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:26:48.991 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:48.987+0000 7fef97e00640 1 -- 172.21.15.47:0/3453211489 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5626c628ede0 msgr2=0x5626c6291230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:26:48.991 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:48.987+0000 7fef97e00640 1 --2- 172.21.15.47:0/3453211489 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5626c628ede0 0x5626c6291230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:26:48.991 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:48.987+0000 7fef97e00640 1 -- 172.21.15.47:0/3453211489 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5626c61b6080 con 0x5626c62858b0 2024-03-21T10:26:48.991 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:48.987+0000 7fef96a00640 1 --2- 172.21.15.47:0/3453211489 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5626c628ede0 0x5626c6291230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T10:26:48.992 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:48.987+0000 7fef97e00640 1 --2- 172.21.15.47:0/3453211489 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5626c62858b0 0x5626c6285c90 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fef90009e60 tx=0x7fef90010740 comp rx=0 tx=0).ready entity=mon.2 client_cookie=85886edeea2e7dfa server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:26:48.992 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:48.988+0000 7fef8fe00640 1 --2- 172.21.15.47:0/2488326357 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7fef7c0a3230 0x7fef7c0a56f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:26:48.992 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:48.988+0000 7fef83e00640 1 -- 172.21.15.47:0/3453211489 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef90016070 con 0x5626c62858b0 2024-03-21T10:26:48.992 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:48.988+0000 7fef83e00640 1 -- 172.21.15.47:0/3453211489 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fef90004d20 con 0x5626c62858b0 2024-03-21T10:26:48.992 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:48.988+0000 7fef83e00640 1 -- 172.21.15.47:0/3453211489 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef90012740 con 0x5626c62858b0 2024-03-21T10:26:48.992 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:48.988+0000 7fef8fe00640 1 -- 172.21.15.47:0/2488326357 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 9474+0+0 (secure 0 0 0) 0x7fef840f1af0 con 0x7fef98074370 2024-03-21T10:26:48.993 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:48.988+0000 7fef97400640 1 --2- 172.21.15.47:0/2488326357 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7fef7c0a3230 0x7fef7c0a56f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:26:48.993 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:48.989+0000 7fefa0e46740 1 -- 172.21.15.47:0/3453211489 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5626c62858b0 msgr2=0x5626c6285c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:26:48.993 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:48.989+0000 7fefa0e46740 1 --2- 172.21.15.47:0/3453211489 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5626c62858b0 0x5626c6285c90 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fef90009e60 tx=0x7fef90010740 comp rx=0 tx=0).stop 2024-03-21T10:26:48.993 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:48.989+0000 7fefa0e46740 1 -- 172.21.15.47:0/3453211489 shutdown_connections 2024-03-21T10:26:48.993 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:48.989+0000 7fefa0e46740 1 --2- 172.21.15.47:0/3453211489 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5626c628ede0 0x5626c6291230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:26:48.993 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:48.989+0000 7fefa0e46740 1 --2- 172.21.15.47:0/3453211489 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5626c6286260 0x5626c628e810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:26:48.994 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:48.989+0000 7fefa0e46740 1 --2- 172.21.15.47:0/3453211489 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5626c62858b0 0x5626c6285c90 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:26:48.994 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:48.989+0000 7fefa0e46740 1 -- 172.21.15.47:0/3453211489 >> 172.21.15.47:0/3453211489 conn(0x5626c627ff80 msgr2=0x5626c6280360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:26:48.994 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:48.989+0000 7fef97400640 1 --2- 172.21.15.47:0/2488326357 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7fef7c0a3230 0x7fef7c0a56f0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x5626c6280ff0 tx=0x7fef88005f90 comp rx=0 tx=0).ready entity=mgr.15687 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:26:48.994 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:48.989+0000 7fefa0e46740 1 -- 172.21.15.47:0/3453211489 shutdown_connections 2024-03-21T10:26:48.994 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:48.989+0000 7fefa0e46740 1 -- 172.21.15.47:0/3453211489 wait complete. 2024-03-21T10:26:48.994 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:48.989+0000 7fefa0e46740 1 -- start start 2024-03-21T10:26:48.994 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:48.990+0000 7fefa0e46740 1 Processor -- start 2024-03-21T10:26:49.000 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:48.990+0000 7fefa0e46740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5626c62858b0 0x5626c628e6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:26:49.000 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:48.990+0000 7fefa0e46740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5626c6286260 0x5626c628ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:26:49.000 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:48.990+0000 7fefa0e46740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5626c628ede0 0x5626c628bfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:26:49.000 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:48.990+0000 7fefa0e46740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x5626c6291d50 con 0x5626c628ede0 2024-03-21T10:26:49.001 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:48.990+0000 7fefa0e46740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x5626c61a50d0 con 0x5626c6286260 2024-03-21T10:26:49.001 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:48.990+0000 7fefa0e46740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x5626c6291ed0 con 0x5626c62858b0 2024-03-21T10:26:49.001 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:48.991+0000 7fefa0e46740 10 client.0 _ll_register_callbacks cb 0x5626c61a9e80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T10:26:49.001 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:48.991+0000 7fef97e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5626c628ede0 0x5626c628bfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:26:49.001 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:48.991+0000 7fef96a00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5626c6286260 0x5626c628ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:26:49.001 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:48.991+0000 7fef97e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5626c628ede0 0x5626c628bfe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:57952/0 (socket says 172.21.15.47:57952) 2024-03-21T10:26:49.001 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:48.991+0000 7fef97e00640 1 -- 172.21.15.47:0/1720749730 learned_addr learned my addr 172.21.15.47:0/1720749730 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:26:49.002 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:48.991+0000 7fef97400640 1 --2- 172.21.15.47:0/1720749730 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5626c62858b0 0x5626c628e6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:26:49.002 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:48.991+0000 7fef97e00640 1 -- 172.21.15.47:0/1720749730 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5626c62858b0 msgr2=0x5626c628e6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:26:49.002 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:48.991+0000 7fef97e00640 1 --2- 172.21.15.47:0/1720749730 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5626c62858b0 0x5626c628e6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:26:49.002 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:48.991+0000 7fef97e00640 1 -- 172.21.15.47:0/1720749730 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5626c6286260 msgr2=0x5626c628ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:26:49.002 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:48.991+0000 7fef97e00640 1 --2- 172.21.15.47:0/1720749730 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5626c6286260 0x5626c628ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:26:49.002 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:48.991+0000 7fef97e00640 1 -- 172.21.15.47:0/1720749730 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5626c5efd3f0 con 0x5626c628ede0 2024-03-21T10:26:49.002 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:48.992+0000 7fef97e00640 1 --2- 172.21.15.47:0/1720749730 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5626c628ede0 0x5626c628bfe0 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7fef90005b40 tx=0x7fef90004c70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3d8f80d63a69c3f3 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:26:49.003 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:48.992+0000 7fef80c00640 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-03-21T10:26:49.003 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:48.992+0000 7fef80c00640 1 -- 172.21.15.47:0/1720749730 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef90029060 con 0x5626c628ede0 2024-03-21T10:26:49.003 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:48.992+0000 7fef80c00640 1 -- 172.21.15.47:0/1720749730 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(97..97 src has 1..97) v4 ==== 9474+0+0 (secure 0 0 0) 0x7fef900051a0 con 0x5626c628ede0 2024-03-21T10:26:49.003 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:48.992+0000 7fefa0e46740 1 -- 172.21.15.47:0/1720749730 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fef90004d20 con 0x5626c628ede0 2024-03-21T10:26:49.003 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:48.992+0000 7fef80c00640 1 -- 172.21.15.47:0/1720749730 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7fef90030050 con 0x5626c628ede0 2024-03-21T10:26:49.003 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:48.992+0000 7fefa0e46740 10 client.16209 fetch_fsmap learned FSMap version 32 2024-03-21T10:26:49.003 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:48.992+0000 7fefa0e46740 1 -- 172.21.15.47:0/1720749730 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x5626c5fc77d0 con 0x5626c628ede0 2024-03-21T10:26:49.003 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:48.993+0000 7fef80c00640 1 -- 172.21.15.47:0/1720749730 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fef90004d20 con 0x5626c628ede0 2024-03-21T10:26:49.003 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:48.993+0000 7fef80c00640 1 -- 172.21.15.47:0/1720749730 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 32) v1 ==== 7942+0+0 (secure 0 0 0) 0x7fef9000f040 con 0x5626c628ede0 2024-03-21T10:26:49.004 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:48.993+0000 7fefa0e46740 10 client.16209 fetch_fsmap finished waiting for FSMap version 32 2024-03-21T10:26:49.004 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:48.993+0000 7fefa0e46740 10 client.16209 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.a 2024-03-21T10:26:49.004 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:48.993+0000 7fefa0e46740 20 client.16209 populate_metadata read hostname 'smithi047' 2024-03-21T10:26:49.004 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:48.993+0000 7fefa0e46740 1 --2- 172.21.15.47:0/1720749730 >> [v2:172.21.15.47:6840/3401578978,v1:172.21.15.47:6841/3401578978] conn(0x5626c629ebb0 0x5626c62a0fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:26:49.004 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:48.993+0000 7fefa0e46740 4 client.16209 mds_command: new command op to 14619 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T10:26:49.004 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:48.993+0000 7fefa0e46740 1 -- 172.21.15.47:0/1720749730 --> [v2:172.21.15.47:6840/3401578978,v1:172.21.15.47:6841/3401578978] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5626c61f6800 con 0x5626c629ebb0 2024-03-21T10:26:49.004 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:48.993+0000 7fef97400640 1 --2- 172.21.15.47:0/1720749730 >> [v2:172.21.15.47:6840/3401578978,v1:172.21.15.47:6841/3401578978] conn(0x5626c629ebb0 0x5626c62a0fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:26:49.004 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:48.994+0000 7fef97400640 1 --2- 172.21.15.47:0/1720749730 >> [v2:172.21.15.47:6840/3401578978,v1:172.21.15.47:6841/3401578978] conn(0x5626c629ebb0 0x5626c62a0fa0 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=c7dc14cde446cb0b server_cookie=727ea788e859464a in_seq=0 out_seq=0 2024-03-21T10:26:49.004 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:48.994+0000 7fef80c00640 10 client.16209 ms_handle_connect on v2:172.21.15.47:6840/3401578978 2024-03-21T10:26:49.005 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:48.995+0000 7fef80c00640 1 -- 172.21.15.47:0/1720749730 <== mds.4 v2:172.21.15.47:6840/3401578978 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x5626c61f6800 con 0x5626c629ebb0 2024-03-21T10:26:49.005 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:48.995+0000 7fef80c00640 10 client.16209 handle_command_reply: tid=0 2024-03-21T10:26:49.005 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:48.995+0000 7fefa0e46740 1 client.16209 shutdown 2024-03-21T10:26:49.005 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:48.995+0000 7fefa0e46740 1 -- 172.21.15.47:0/1720749730 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5626c628ede0 msgr2=0x5626c628bfe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:26:49.005 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:48.995+0000 7fefa0e46740 1 --2- 172.21.15.47:0/1720749730 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5626c628ede0 0x5626c628bfe0 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7fef90005b40 tx=0x7fef90004c70 comp rx=0 tx=0).stop 2024-03-21T10:26:49.005 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:48.995+0000 7fefa0e46740 1 -- 172.21.15.47:0/1720749730 shutdown_connections 2024-03-21T10:26:49.005 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:48.995+0000 7fefa0e46740 1 --2- 172.21.15.47:0/1720749730 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5626c628ede0 0x5626c628bfe0 unknown :-1 s=CLOSED pgs=632 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:26:49.005 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:48.995+0000 7fefa0e46740 1 --2- 172.21.15.47:0/1720749730 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5626c6286260 0x5626c628ba60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:26:49.005 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:48.995+0000 7fefa0e46740 1 --2- 172.21.15.47:0/1720749730 >> [v2:172.21.15.47:6840/3401578978,v1:172.21.15.47:6841/3401578978] conn(0x5626c629ebb0 0x5626c62a0fa0 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:26:49.006 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:48.995+0000 7fefa0e46740 1 --2- 172.21.15.47:0/1720749730 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5626c62858b0 0x5626c628e6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:26:49.006 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:48.995+0000 7fefa0e46740 1 -- 172.21.15.47:0/1720749730 >> 172.21.15.47:0/1720749730 conn(0x5626c627ff80 msgr2=0x5626c6281cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:26:49.006 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:48.995+0000 7fef80c00640 0 client.16209 ms_handle_reset on v2:172.21.15.47:6840/3401578978 2024-03-21T10:26:49.006 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:48.995+0000 7fefa0e46740 1 -- 172.21.15.47:0/1720749730 shutdown_connections 2024-03-21T10:26:49.006 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:48.995+0000 7fefa0e46740 1 -- 172.21.15.47:0/1720749730 wait complete. 2024-03-21T10:26:49.006 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:48.996+0000 7fefa0e46740 20 client.16209 trim_cache size 0 max 16384 2024-03-21T10:26:49.009 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.009+0000 7fefa0e46740 1 Processor -- start 2024-03-21T10:26:49.009 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.009+0000 7fefa0e46740 1 -- start start 2024-03-21T10:26:49.009 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.009+0000 7fefa0e46740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5626c62858b0 0x5626c627efe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:26:49.009 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.009+0000 7fefa0e46740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5626c6286260 0x5626c628e190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:26:49.009 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.009+0000 7fefa0e46740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5626c628ede0 0x5626c628e6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:26:49.009 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.009+0000 7fefa0e46740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef981b4d00 con 0x5626c62858b0 2024-03-21T10:26:49.009 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.009+0000 7fefa0e46740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef981b6680 con 0x5626c6286260 2024-03-21T10:26:49.009 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.009+0000 7fefa0e46740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef981b5360 con 0x5626c628ede0 2024-03-21T10:26:49.011 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.009+0000 7fef97400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5626c62858b0 0x5626c627efe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:26:49.011 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.009+0000 7fef97400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5626c62858b0 0x5626c627efe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:57962/0 (socket says 172.21.15.47:57962) 2024-03-21T10:26:49.011 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.009+0000 7fef97400640 1 -- 172.21.15.47:0/3478997603 learned_addr learned my addr 172.21.15.47:0/3478997603 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:26:49.011 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.010+0000 7fef96a00640 1 --2- 172.21.15.47:0/3478997603 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5626c6286260 0x5626c628e190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:26:49.011 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.010+0000 7fef97e00640 1 --2- 172.21.15.47:0/3478997603 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5626c628ede0 0x5626c628e6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:26:49.011 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.010+0000 7fef97400640 1 -- 172.21.15.47:0/3478997603 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5626c628ede0 msgr2=0x5626c628e6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:26:49.011 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.010+0000 7fef97400640 1 --2- 172.21.15.47:0/3478997603 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5626c628ede0 0x5626c628e6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:26:49.011 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.010+0000 7fef97400640 1 -- 172.21.15.47:0/3478997603 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5626c6286260 msgr2=0x5626c628e190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:26:49.011 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.010+0000 7fef97400640 1 --2- 172.21.15.47:0/3478997603 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5626c6286260 0x5626c628e190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:26:49.012 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.010+0000 7fef97400640 1 -- 172.21.15.47:0/3478997603 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5626c6286000 con 0x5626c62858b0 2024-03-21T10:26:49.012 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.010+0000 7fef97400640 1 --2- 172.21.15.47:0/3478997603 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5626c62858b0 0x5626c627efe0 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x5626c6291fe0 tx=0x7fef88017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b4804380cfa14ca1 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:26:49.012 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.011+0000 7fef83400640 1 -- 172.21.15.47:0/3478997603 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef88004380 con 0x5626c62858b0 2024-03-21T10:26:49.012 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.011+0000 7fef83400640 1 -- 172.21.15.47:0/3478997603 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fef88030070 con 0x5626c62858b0 2024-03-21T10:26:49.012 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.011+0000 7fef83400640 1 -- 172.21.15.47:0/3478997603 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef8801e7b0 con 0x5626c62858b0 2024-03-21T10:26:49.014 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.011+0000 7fefa0e46740 1 -- 172.21.15.47:0/3478997603 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5626c62858b0 msgr2=0x5626c627efe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:26:49.014 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.011+0000 7fefa0e46740 1 --2- 172.21.15.47:0/3478997603 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5626c62858b0 0x5626c627efe0 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x5626c6291fe0 tx=0x7fef88017000 comp rx=0 tx=0).stop 2024-03-21T10:26:49.014 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.012+0000 7fefa0e46740 1 -- 172.21.15.47:0/3478997603 shutdown_connections 2024-03-21T10:26:49.014 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.012+0000 7fefa0e46740 1 --2- 172.21.15.47:0/3478997603 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5626c628ede0 0x5626c628e6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:26:49.014 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.012+0000 7fefa0e46740 1 --2- 172.21.15.47:0/3478997603 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5626c6286260 0x5626c628e190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:26:49.014 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.012+0000 7fefa0e46740 1 --2- 172.21.15.47:0/3478997603 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5626c62858b0 0x5626c627efe0 unknown :-1 s=CLOSED pgs=633 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:26:49.014 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.012+0000 7fefa0e46740 1 -- 172.21.15.47:0/3478997603 >> 172.21.15.47:0/3478997603 conn(0x5626c627ff80 msgr2=0x5626c627e8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:26:49.014 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.012+0000 7fefa0e46740 1 -- 172.21.15.47:0/3478997603 shutdown_connections 2024-03-21T10:26:49.014 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.012+0000 7fefa0e46740 1 -- 172.21.15.47:0/3478997603 wait complete. 2024-03-21T10:26:49.014 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.012+0000 7fefa0e46740 1 -- start start 2024-03-21T10:26:49.014 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.012+0000 7fefa0e46740 1 Processor -- start 2024-03-21T10:26:49.015 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.013+0000 7fefa0e46740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5626c62858b0 0x5626c62a3e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:26:49.015 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.013+0000 7fefa0e46740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5626c6286260 0x5626c62a21f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:26:49.015 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.013+0000 7fefa0e46740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5626c628ede0 0x5626c62a2820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:26:49.015 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.013+0000 7fefa0e46740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef981b6020 con 0x5626c6286260 2024-03-21T10:26:49.015 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.013+0000 7fefa0e46740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef981b59c0 con 0x5626c62858b0 2024-03-21T10:26:49.015 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.013+0000 7fefa0e46740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef981b2ec0 con 0x5626c628ede0 2024-03-21T10:26:49.015 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.013+0000 7fefa0e46740 10 client.0 _ll_register_callbacks cb 0x7fef840f5450 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T10:26:49.015 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.013+0000 7fef96a00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5626c6286260 0x5626c62a21f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:26:49.015 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.013+0000 7fef97400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5626c62858b0 0x5626c62a3e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:26:49.015 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.013+0000 7fef96a00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5626c6286260 0x5626c62a21f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:57976/0 (socket says 172.21.15.47:57976) 2024-03-21T10:26:49.015 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.013+0000 7fef96a00640 1 -- 172.21.15.47:0/3095224109 learned_addr learned my addr 172.21.15.47:0/3095224109 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:26:49.015 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.013+0000 7fef97e00640 1 --2- 172.21.15.47:0/3095224109 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5626c628ede0 0x5626c62a2820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:26:49.015 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.013+0000 7fef96a00640 1 -- 172.21.15.47:0/3095224109 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5626c628ede0 msgr2=0x5626c62a2820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:26:49.016 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.014+0000 7fef96a00640 1 --2- 172.21.15.47:0/3095224109 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5626c628ede0 0x5626c62a2820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:26:49.016 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.014+0000 7fef96a00640 1 -- 172.21.15.47:0/3095224109 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5626c62858b0 msgr2=0x5626c62a3e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:26:49.016 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.014+0000 7fef96a00640 1 --2- 172.21.15.47:0/3095224109 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5626c62858b0 0x5626c62a3e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:26:49.016 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.014+0000 7fef96a00640 1 -- 172.21.15.47:0/3095224109 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5626c62a3020 con 0x5626c6286260 2024-03-21T10:26:49.016 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.014+0000 7fef97400640 1 --2- 172.21.15.47:0/3095224109 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5626c62858b0 0x5626c62a3e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T10:26:49.016 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.014+0000 7fef96a00640 1 --2- 172.21.15.47:0/3095224109 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5626c6286260 0x5626c62a21f0 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7fef840bd910 tx=0x7fef840b9c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=88f701217cfcd038 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:26:49.016 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.014+0000 7fef80c00640 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-03-21T10:26:49.018 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.014+0000 7fef80c00640 1 -- 172.21.15.47:0/3095224109 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef840bd2e0 con 0x5626c6286260 2024-03-21T10:26:49.018 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.014+0000 7fef80c00640 1 -- 172.21.15.47:0/3095224109 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(97..97 src has 1..97) v4 ==== 9474+0+0 (secure 0 0 0) 0x7fef840c6100 con 0x5626c6286260 2024-03-21T10:26:49.018 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.014+0000 7fefa0e46740 1 -- 172.21.15.47:0/3095224109 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fef88030070 con 0x5626c6286260 2024-03-21T10:26:49.019 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.014+0000 7fef80c00640 1 -- 172.21.15.47:0/3095224109 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7fef840be080 con 0x5626c6286260 2024-03-21T10:26:49.019 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.014+0000 7fefa0e46740 10 client.16215 fetch_fsmap learned FSMap version 32 2024-03-21T10:26:49.019 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.014+0000 7fefa0e46740 1 -- 172.21.15.47:0/3095224109 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x5626c62fb250 con 0x5626c6286260 2024-03-21T10:26:49.019 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.015+0000 7fef80c00640 1 -- 172.21.15.47:0/3095224109 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fef88030070 con 0x5626c6286260 2024-03-21T10:26:49.019 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.015+0000 7fef80c00640 1 -- 172.21.15.47:0/3095224109 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 32) v1 ==== 7942+0+0 (secure 0 0 0) 0x7fef840bcdd0 con 0x5626c6286260 2024-03-21T10:26:49.019 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.015+0000 7fefa0e46740 10 client.16215 fetch_fsmap finished waiting for FSMap version 32 2024-03-21T10:26:49.019 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.015+0000 7fefa0e46740 10 client.16215 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.a 2024-03-21T10:26:49.019 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.015+0000 7fefa0e46740 20 client.16215 populate_metadata read hostname 'smithi047' 2024-03-21T10:26:49.019 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.015+0000 7fefa0e46740 1 --2- 172.21.15.47:0/3095224109 >> [v2:172.21.15.47:6840/3401578978,v1:172.21.15.47:6841/3401578978] conn(0x5626c62fb9d0 0x5626c62fddf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:26:49.019 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.015+0000 7fefa0e46740 4 client.16215 mds_command: new command op to 14619 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-21T10:26:49.019 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.015+0000 7fefa0e46740 1 -- 172.21.15.47:0/3095224109 --> [v2:172.21.15.47:6840/3401578978,v1:172.21.15.47:6841/3401578978] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x5626c61f0b50 con 0x5626c62fb9d0 2024-03-21T10:26:49.019 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.015+0000 7fef97400640 1 --2- 172.21.15.47:0/3095224109 >> [v2:172.21.15.47:6840/3401578978,v1:172.21.15.47:6841/3401578978] conn(0x5626c62fb9d0 0x5626c62fddf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:26:49.019 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.016+0000 7fef97400640 1 --2- 172.21.15.47:0/3095224109 >> [v2:172.21.15.47:6840/3401578978,v1:172.21.15.47:6841/3401578978] conn(0x5626c62fb9d0 0x5626c62fddf0 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=e824d3dcf3131524 server_cookie=5129889636e79735 in_seq=0 out_seq=0 2024-03-21T10:26:49.019 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.016+0000 7fef80c00640 10 client.16215 ms_handle_connect on v2:172.21.15.47:6840/3401578978 2024-03-21T10:26:49.020 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.016+0000 7fef80c00640 1 -- 172.21.15.47:0/3095224109 <== mds.4 v2:172.21.15.47:6840/3401578978 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x5626c61f0b50 con 0x5626c62fb9d0 2024-03-21T10:26:49.020 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.016+0000 7fef80c00640 10 client.16215 handle_command_reply: tid=0 2024-03-21T10:26:49.020 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.016+0000 7fefa0e46740 1 client.16215 shutdown 2024-03-21T10:26:49.020 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.017+0000 7fefa0e46740 1 -- 172.21.15.47:0/3095224109 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5626c6286260 msgr2=0x5626c62a21f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:26:49.020 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.017+0000 7fefa0e46740 1 --2- 172.21.15.47:0/3095224109 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5626c6286260 0x5626c62a21f0 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7fef840bd910 tx=0x7fef840b9c20 comp rx=0 tx=0).stop 2024-03-21T10:26:49.020 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.017+0000 7fefa0e46740 1 -- 172.21.15.47:0/3095224109 shutdown_connections 2024-03-21T10:26:49.020 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.017+0000 7fefa0e46740 1 --2- 172.21.15.47:0/3095224109 >> [v2:172.21.15.47:6840/3401578978,v1:172.21.15.47:6841/3401578978] conn(0x5626c62fb9d0 0x5626c62fddf0 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:26:49.020 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.017+0000 7fefa0e46740 1 --2- 172.21.15.47:0/3095224109 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5626c628ede0 0x5626c62a2820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:26:49.020 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.017+0000 7fefa0e46740 1 --2- 172.21.15.47:0/3095224109 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5626c6286260 0x5626c62a21f0 unknown :-1 s=CLOSED pgs=634 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:26:49.020 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.017+0000 7fefa0e46740 1 --2- 172.21.15.47:0/3095224109 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5626c62858b0 0x5626c62a3e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:26:49.020 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.017+0000 7fefa0e46740 1 -- 172.21.15.47:0/3095224109 >> 172.21.15.47:0/3095224109 conn(0x5626c627ff80 msgr2=0x5626c627f160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:26:49.020 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.017+0000 7fef80c00640 0 client.16215 ms_handle_reset on v2:172.21.15.47:6840/3401578978 2024-03-21T10:26:49.020 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.017+0000 7fefa0e46740 1 -- 172.21.15.47:0/3095224109 shutdown_connections 2024-03-21T10:26:49.020 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.017+0000 7fefa0e46740 1 -- 172.21.15.47:0/3095224109 wait complete. 2024-03-21T10:26:49.020 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.017+0000 7fefa0e46740 20 client.16215 trim_cache size 0 max 16384 2024-03-21T10:26:49.021 INFO:teuthology.orchestra.run.smithi047.stdout:[] 2024-03-21T10:26:49.021 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.018+0000 7fef9cc00640 1 -- 172.21.15.47:0/2488326357 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7fef7c0a3230 msgr2=0x7fef7c0a56f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:26:49.022 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.018+0000 7fef9cc00640 1 --2- 172.21.15.47:0/2488326357 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7fef7c0a3230 0x7fef7c0a56f0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x5626c6280ff0 tx=0x7fef88005f90 comp rx=0 tx=0).stop 2024-03-21T10:26:49.022 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.018+0000 7fef9cc00640 1 -- 172.21.15.47:0/2488326357 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fef98074370 msgr2=0x7fef98101a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:26:49.022 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.018+0000 7fef9cc00640 1 --2- 172.21.15.47:0/2488326357 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fef98074370 0x7fef98101a30 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7fef84006010 tx=0x7fef840045e0 comp rx=0 tx=0).stop 2024-03-21T10:26:49.022 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.018+0000 7fef9cc00640 1 -- 172.21.15.47:0/2488326357 shutdown_connections 2024-03-21T10:26:49.022 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.018+0000 7fef9cc00640 1 --2- 172.21.15.47:0/2488326357 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fef9810ddd0 0x7fef98101f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:26:49.022 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.018+0000 7fef9cc00640 1 --2- 172.21.15.47:0/2488326357 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fef98074370 0x7fef98101a30 unknown :-1 s=CLOSED pgs=631 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:26:49.022 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.018+0000 7fef9cc00640 1 --2- 172.21.15.47:0/2488326357 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7fef7c0a3230 0x7fef7c0a56f0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:26:49.022 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.018+0000 7fef9cc00640 1 --2- 172.21.15.47:0/2488326357 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fef98073a50 0x7fef981014f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:26:49.022 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.018+0000 7fef9cc00640 1 -- 172.21.15.47:0/2488326357 >> 172.21.15.47:0/2488326357 conn(0x7fef980fd040 msgr2=0x7fef980fde50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:26:49.022 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.018+0000 7fef9cc00640 1 -- 172.21.15.47:0/2488326357 shutdown_connections 2024-03-21T10:26:49.023 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.018+0000 7fef9cc00640 1 -- 172.21.15.47:0/2488326357 wait complete. 2024-03-21T10:26:49.035 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-21T10:26:49.036 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-03-21T10:26:49.219 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.218+0000 7fa629600640 1 -- 172.21.15.47:0/4113210869 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa62410bcb0 msgr2=0x7fa62410e0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:26:49.219 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.218+0000 7fa629600640 1 --2- 172.21.15.47:0/4113210869 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa62410bcb0 0x7fa62410e0e0 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7fa61c009ca0 tx=0x7fa61c02c190 comp rx=0 tx=0).stop 2024-03-21T10:26:49.219 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.218+0000 7fa629600640 1 -- 172.21.15.47:0/4113210869 shutdown_connections 2024-03-21T10:26:49.219 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.218+0000 7fa629600640 1 --2- 172.21.15.47:0/4113210869 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa62410bcb0 0x7fa62410e0e0 unknown :-1 s=CLOSED pgs=635 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:26:49.220 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.218+0000 7fa629600640 1 --2- 172.21.15.47:0/4113210869 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa624103200 0x7fa62410b6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:26:49.220 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.218+0000 7fa629600640 1 --2- 172.21.15.47:0/4113210869 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa6241028e0 0x7fa624102cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:26:49.220 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.218+0000 7fa629600640 1 -- 172.21.15.47:0/4113210869 >> 172.21.15.47:0/4113210869 conn(0x7fa624076920 msgr2=0x7fa624076d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:26:49.220 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.219+0000 7fa629600640 1 -- 172.21.15.47:0/4113210869 shutdown_connections 2024-03-21T10:26:49.220 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.219+0000 7fa629600640 1 -- 172.21.15.47:0/4113210869 wait complete. 2024-03-21T10:26:49.221 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.220+0000 7fa629600640 1 Processor -- start 2024-03-21T10:26:49.222 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.220+0000 7fa629600640 1 -- start start 2024-03-21T10:26:49.222 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.220+0000 7fa629600640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa6241028e0 0x7fa62419a010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:26:49.222 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.220+0000 7fa629600640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa624103200 0x7fa62419a550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:26:49.222 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.220+0000 7fa629600640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa62410bcb0 0x7fa62419e8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:26:49.222 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.220+0000 7fa629600640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6241118f0 con 0x7fa62410bcb0 2024-03-21T10:26:49.222 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.220+0000 7fa629600640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa624111770 con 0x7fa624103200 2024-03-21T10:26:49.222 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.220+0000 7fa629600640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa624111a70 con 0x7fa6241028e0 2024-03-21T10:26:49.222 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.220+0000 7fa623400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa62410bcb0 0x7fa62419e8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:26:49.222 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.221+0000 7fa623400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa62410bcb0 0x7fa62419e8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:57998/0 (socket says 172.21.15.47:57998) 2024-03-21T10:26:49.222 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.221+0000 7fa623400640 1 -- 172.21.15.47:0/3564912294 learned_addr learned my addr 172.21.15.47:0/3564912294 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:26:49.222 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.221+0000 7fa622000640 1 --2- 172.21.15.47:0/3564912294 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa624103200 0x7fa62419a550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:26:49.222 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.221+0000 7fa622a00640 1 --2- 172.21.15.47:0/3564912294 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa6241028e0 0x7fa62419a010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:26:49.222 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.221+0000 7fa623400640 1 -- 172.21.15.47:0/3564912294 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa6241028e0 msgr2=0x7fa62419a010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:26:49.222 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.221+0000 7fa623400640 1 --2- 172.21.15.47:0/3564912294 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa6241028e0 0x7fa62419a010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:26:49.223 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.221+0000 7fa623400640 1 -- 172.21.15.47:0/3564912294 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa624103200 msgr2=0x7fa62419a550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:26:49.223 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.221+0000 7fa623400640 1 --2- 172.21.15.47:0/3564912294 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa624103200 0x7fa62419a550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:26:49.223 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.221+0000 7fa623400640 1 -- 172.21.15.47:0/3564912294 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa62419f000 con 0x7fa62410bcb0 2024-03-21T10:26:49.223 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.222+0000 7fa623400640 1 --2- 172.21.15.47:0/3564912294 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa62410bcb0 0x7fa62419e8d0 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7fa61c02c7a0 tx=0x7fa61c035e20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:26:49.223 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.222+0000 7fa61b400640 1 -- 172.21.15.47:0/3564912294 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa61c039040 con 0x7fa62410bcb0 2024-03-21T10:26:49.224 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.222+0000 7fa61b400640 1 -- 172.21.15.47:0/3564912294 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fa61c044070 con 0x7fa62410bcb0 2024-03-21T10:26:49.224 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.222+0000 7fa629600640 1 -- 172.21.15.47:0/3564912294 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa62419f290 con 0x7fa62410bcb0 2024-03-21T10:26:49.224 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.222+0000 7fa61b400640 1 -- 172.21.15.47:0/3564912294 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa61c032850 con 0x7fa62410bcb0 2024-03-21T10:26:49.224 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.222+0000 7fa629600640 1 -- 172.21.15.47:0/3564912294 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa6241a6b80 con 0x7fa62410bcb0 2024-03-21T10:26:49.225 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.225+0000 7fa61b400640 1 -- 172.21.15.47:0/3564912294 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 26) v1 ==== 138087+0+0 (secure 0 0 0) 0x7fa61c0329f0 con 0x7fa62410bcb0 2024-03-21T10:26:49.225 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.225+0000 7fa62c43a740 1 Processor -- start 2024-03-21T10:26:49.225 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.226+0000 7fa62c43a740 1 -- start start 2024-03-21T10:26:49.227 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.226+0000 7fa62c43a740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5566ea5af6d0 0x5566ea5afab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:26:49.227 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.226+0000 7fa62c43a740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5566ea5b0080 0x5566ea5b8630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:26:49.227 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.226+0000 7fa62c43a740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5566ea5b8c00 0x5566ea5bb050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:26:49.227 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.226+0000 7fa62c43a740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x5566ea5132c0 con 0x5566ea5b0080 2024-03-21T10:26:49.227 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.226+0000 7fa62c43a740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x5566ea4bbc90 con 0x5566ea5b8c00 2024-03-21T10:26:49.227 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.226+0000 7fa62c43a740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x5566ea50f360 con 0x5566ea5af6d0 2024-03-21T10:26:49.227 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.226+0000 7fa622000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5566ea5b8c00 0x5566ea5bb050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:26:49.227 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.227+0000 7fa622000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5566ea5b8c00 0x5566ea5bb050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:52312/0 (socket says 172.21.15.47:52312) 2024-03-21T10:26:49.227 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.227+0000 7fa622000640 1 -- 172.21.15.47:0/3954331594 learned_addr learned my addr 172.21.15.47:0/3954331594 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:26:49.228 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.227+0000 7fa622a00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5566ea5b0080 0x5566ea5b8630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:26:49.228 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.227+0000 7fa622000640 1 --2- 172.21.15.47:0/3954331594 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5566ea5af6d0 0x5566ea5afab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:26:49.228 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.227+0000 7fa622000640 1 -- 172.21.15.47:0/3954331594 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5566ea5af6d0 msgr2=0x5566ea5afab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:26:49.228 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.227+0000 7fa622000640 1 --2- 172.21.15.47:0/3954331594 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5566ea5af6d0 0x5566ea5afab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:26:49.228 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.227+0000 7fa622000640 1 -- 172.21.15.47:0/3954331594 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5566ea5b0080 msgr2=0x5566ea5b8630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:26:49.228 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.227+0000 7fa622000640 1 --2- 172.21.15.47:0/3954331594 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5566ea5b0080 0x5566ea5b8630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:26:49.228 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.227+0000 7fa622000640 1 -- 172.21.15.47:0/3954331594 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5566ea4e0080 con 0x5566ea5b8c00 2024-03-21T10:26:49.228 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.227+0000 7fa622a00640 1 --2- 172.21.15.47:0/3954331594 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5566ea5b0080 0x5566ea5b8630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T10:26:49.231 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.227+0000 7fa61b400640 1 --2- 172.21.15.47:0/3564912294 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7fa6080a3230 0x7fa6080a56f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:26:49.231 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.227+0000 7fa622000640 1 --2- 172.21.15.47:0/3954331594 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5566ea5b8c00 0x5566ea5bb050 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fa6100116f0 tx=0x7fa610009770 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7d323344e0c0e99c server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:26:49.231 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.228+0000 7fa61b400640 1 -- 172.21.15.47:0/3564912294 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 9474+0+0 (secure 0 0 0) 0x7fa61c0f1f50 con 0x7fa62410bcb0 2024-03-21T10:26:49.231 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.228+0000 7fa60f400640 1 -- 172.21.15.47:0/3954331594 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa610004030 con 0x5566ea5b8c00 2024-03-21T10:26:49.231 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.228+0000 7fa622a00640 1 --2- 172.21.15.47:0/3564912294 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7fa6080a3230 0x7fa6080a56f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:26:49.231 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.228+0000 7fa60f400640 1 -- 172.21.15.47:0/3954331594 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fa61000fcf0 con 0x5566ea5b8c00 2024-03-21T10:26:49.231 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.228+0000 7fa60f400640 1 -- 172.21.15.47:0/3954331594 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6100038d0 con 0x5566ea5b8c00 2024-03-21T10:26:49.231 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.229+0000 7fa62c43a740 1 -- 172.21.15.47:0/3954331594 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5566ea5b8c00 msgr2=0x5566ea5bb050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:26:49.231 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.229+0000 7fa622a00640 1 --2- 172.21.15.47:0/3564912294 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7fa6080a3230 0x7fa6080a56f0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x5566ea5aae10 tx=0x7fa61400b390 comp rx=0 tx=0).ready entity=mgr.15687 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:26:49.231 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.229+0000 7fa62c43a740 1 --2- 172.21.15.47:0/3954331594 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5566ea5b8c00 0x5566ea5bb050 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fa6100116f0 tx=0x7fa610009770 comp rx=0 tx=0).stop 2024-03-21T10:26:49.232 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.229+0000 7fa62c43a740 1 -- 172.21.15.47:0/3954331594 shutdown_connections 2024-03-21T10:26:49.232 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.229+0000 7fa62c43a740 1 --2- 172.21.15.47:0/3954331594 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5566ea5b8c00 0x5566ea5bb050 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:26:49.232 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.229+0000 7fa62c43a740 1 --2- 172.21.15.47:0/3954331594 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5566ea5b0080 0x5566ea5b8630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:26:49.232 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.229+0000 7fa62c43a740 1 --2- 172.21.15.47:0/3954331594 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5566ea5af6d0 0x5566ea5afab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:26:49.232 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.229+0000 7fa62c43a740 1 -- 172.21.15.47:0/3954331594 >> 172.21.15.47:0/3954331594 conn(0x5566ea5a9da0 msgr2=0x5566ea5aa180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:26:49.232 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.229+0000 7fa62c43a740 1 -- 172.21.15.47:0/3954331594 shutdown_connections 2024-03-21T10:26:49.232 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.229+0000 7fa62c43a740 1 -- 172.21.15.47:0/3954331594 wait complete. 2024-03-21T10:26:49.232 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.229+0000 7fa62c43a740 1 -- start start 2024-03-21T10:26:49.232 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.230+0000 7fa62c43a740 1 Processor -- start 2024-03-21T10:26:49.232 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.230+0000 7fa62c43a740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5566ea5af6d0 0x5566ea5b8560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:26:49.232 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.230+0000 7fa62c43a740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5566ea5b0080 0x5566ea5b58f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:26:49.232 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.230+0000 7fa62c43a740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5566ea5b8c00 0x5566ea5b5f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:26:49.232 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.230+0000 7fa62c43a740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x5566ea5bbb70 con 0x5566ea5b8c00 2024-03-21T10:26:49.232 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.230+0000 7fa62c43a740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x5566ea4cf0d0 con 0x5566ea5b0080 2024-03-21T10:26:49.233 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.230+0000 7fa62c43a740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x5566ea5bbcf0 con 0x5566ea5af6d0 2024-03-21T10:26:49.233 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.230+0000 7fa62c43a740 10 client.0 _ll_register_callbacks cb 0x5566ea4d3e80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T10:26:49.233 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.230+0000 7fa622000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5566ea5b0080 0x5566ea5b58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:26:49.233 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.230+0000 7fa623400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5566ea5b8c00 0x5566ea5b5f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:26:49.233 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.230+0000 7fa622000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5566ea5b0080 0x5566ea5b58f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:52320/0 (socket says 172.21.15.47:52320) 2024-03-21T10:26:49.233 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.230+0000 7fa622000640 1 -- 172.21.15.47:0/2277801906 learned_addr learned my addr 172.21.15.47:0/2277801906 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:26:49.233 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.231+0000 7fa622a00640 1 --2- 172.21.15.47:0/2277801906 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5566ea5af6d0 0x5566ea5b8560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:26:49.233 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.231+0000 7fa623400640 1 -- 172.21.15.47:0/2277801906 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5566ea5af6d0 msgr2=0x5566ea5b8560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:26:49.233 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.231+0000 7fa623400640 1 --2- 172.21.15.47:0/2277801906 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5566ea5af6d0 0x5566ea5b8560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:26:49.233 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.231+0000 7fa623400640 1 -- 172.21.15.47:0/2277801906 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5566ea5b0080 msgr2=0x5566ea5b58f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:26:49.233 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.231+0000 7fa623400640 1 --2- 172.21.15.47:0/2277801906 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5566ea5b0080 0x5566ea5b58f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:26:49.233 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.231+0000 7fa623400640 1 -- 172.21.15.47:0/2277801906 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5566ea2273f0 con 0x5566ea5b8c00 2024-03-21T10:26:49.236 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.231+0000 7fa623400640 1 --2- 172.21.15.47:0/2277801906 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5566ea5b8c00 0x5566ea5b5f00 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7fa61c0ef210 tx=0x7fa61c0bd9e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7ef6f5fe1346218c server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:26:49.236 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.231+0000 7fa603e00640 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-03-21T10:26:49.236 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.231+0000 7fa603e00640 1 -- 172.21.15.47:0/2277801906 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa61c003e10 con 0x5566ea5b8c00 2024-03-21T10:26:49.236 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.231+0000 7fa603e00640 1 -- 172.21.15.47:0/2277801906 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(97..97 src has 1..97) v4 ==== 9474+0+0 (secure 0 0 0) 0x7fa61c0be250 con 0x5566ea5b8c00 2024-03-21T10:26:49.236 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.231+0000 7fa62c43a740 1 -- 172.21.15.47:0/2277801906 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fa61000fcf0 con 0x5566ea5b8c00 2024-03-21T10:26:49.236 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.232+0000 7fa603e00640 1 -- 172.21.15.47:0/2277801906 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa61c0be4e0 con 0x5566ea5b8c00 2024-03-21T10:26:49.236 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.232+0000 7fa62c43a740 10 client.16227 fetch_fsmap learned FSMap version 32 2024-03-21T10:26:49.236 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.232+0000 7fa62c43a740 1 -- 172.21.15.47:0/2277801906 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x5566ea2f17d0 con 0x5566ea5b8c00 2024-03-21T10:26:49.236 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.232+0000 7fa603e00640 1 -- 172.21.15.47:0/2277801906 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fa61000fcf0 con 0x5566ea5b8c00 2024-03-21T10:26:49.236 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.232+0000 7fa603e00640 1 -- 172.21.15.47:0/2277801906 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 32) v1 ==== 7942+0+0 (secure 0 0 0) 0x7fa61c03c030 con 0x5566ea5b8c00 2024-03-21T10:26:49.236 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.232+0000 7fa62c43a740 10 client.16227 fetch_fsmap finished waiting for FSMap version 32 2024-03-21T10:26:49.236 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.232+0000 7fa62c43a740 10 client.16227 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.k 2024-03-21T10:26:49.236 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.232+0000 7fa62c43a740 20 client.16227 populate_metadata read hostname 'smithi047' 2024-03-21T10:26:49.237 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.232+0000 7fa62c43a740 1 --2- 172.21.15.47:0/2277801906 >> [v2:172.21.15.100:6832/2723302038,v1:172.21.15.100:6833/2723302038] conn(0x5566ea5c8b20 0x5566ea5caf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:26:49.237 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.232+0000 7fa62c43a740 4 client.16227 mds_command: new command op to 34427 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T10:26:49.237 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.232+0000 7fa62c43a740 1 -- 172.21.15.47:0/2277801906 --> [v2:172.21.15.100:6832/2723302038,v1:172.21.15.100:6833/2723302038] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5566ea520800 con 0x5566ea5c8b20 2024-03-21T10:26:49.237 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.233+0000 7fa622a00640 1 --2- 172.21.15.47:0/2277801906 >> [v2:172.21.15.100:6832/2723302038,v1:172.21.15.100:6833/2723302038] conn(0x5566ea5c8b20 0x5566ea5caf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:26:49.237 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.233+0000 7fa622a00640 1 --2- 172.21.15.47:0/2277801906 >> [v2:172.21.15.100:6832/2723302038,v1:172.21.15.100:6833/2723302038] conn(0x5566ea5c8b20 0x5566ea5caf40 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=a99c09e4e02dc65d server_cookie=314a9b606d140141 in_seq=0 out_seq=0 2024-03-21T10:26:49.237 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.233+0000 7fa603e00640 10 client.16227 ms_handle_connect on v2:172.21.15.100:6832/2723302038 2024-03-21T10:26:49.237 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.235+0000 7fa603e00640 1 -- 172.21.15.47:0/2277801906 <== mds.3 v2:172.21.15.100:6832/2723302038 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x5566ea520800 con 0x5566ea5c8b20 2024-03-21T10:26:49.237 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.235+0000 7fa603e00640 10 client.16227 handle_command_reply: tid=0 2024-03-21T10:26:49.237 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.235+0000 7fa62c43a740 1 client.16227 shutdown 2024-03-21T10:26:49.237 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.235+0000 7fa62c43a740 1 -- 172.21.15.47:0/2277801906 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5566ea5b8c00 msgr2=0x5566ea5b5f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:26:49.237 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.235+0000 7fa62c43a740 1 --2- 172.21.15.47:0/2277801906 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5566ea5b8c00 0x5566ea5b5f00 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7fa61c0ef210 tx=0x7fa61c0bd9e0 comp rx=0 tx=0).stop 2024-03-21T10:26:49.237 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.236+0000 7fa62c43a740 1 -- 172.21.15.47:0/2277801906 shutdown_connections 2024-03-21T10:26:49.237 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.236+0000 7fa62c43a740 1 --2- 172.21.15.47:0/2277801906 >> [v2:172.21.15.100:6832/2723302038,v1:172.21.15.100:6833/2723302038] conn(0x5566ea5c8b20 0x5566ea5caf40 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:26:49.237 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.236+0000 7fa62c43a740 1 --2- 172.21.15.47:0/2277801906 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5566ea5b8c00 0x5566ea5b5f00 unknown :-1 s=CLOSED pgs=637 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:26:49.237 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.236+0000 7fa62c43a740 1 --2- 172.21.15.47:0/2277801906 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5566ea5b0080 0x5566ea5b58f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:26:49.238 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.236+0000 7fa62c43a740 1 --2- 172.21.15.47:0/2277801906 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5566ea5af6d0 0x5566ea5b8560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:26:49.238 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.236+0000 7fa62c43a740 1 -- 172.21.15.47:0/2277801906 >> 172.21.15.47:0/2277801906 conn(0x5566ea5a9da0 msgr2=0x5566ea5abaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:26:49.238 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.236+0000 7fa603e00640 0 client.16227 ms_handle_reset on v2:172.21.15.100:6832/2723302038 2024-03-21T10:26:49.238 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.236+0000 7fa62c43a740 1 -- 172.21.15.47:0/2277801906 shutdown_connections 2024-03-21T10:26:49.238 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.236+0000 7fa62c43a740 1 -- 172.21.15.47:0/2277801906 wait complete. 2024-03-21T10:26:49.238 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.236+0000 7fa62c43a740 20 client.16227 trim_cache size 0 max 16384 2024-03-21T10:26:49.257 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.256+0000 7fa62c43a740 1 Processor -- start 2024-03-21T10:26:49.257 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.256+0000 7fa62c43a740 1 -- start start 2024-03-21T10:26:49.257 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.256+0000 7fa62c43a740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5566ea5af6d0 0x5566ea5b8020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:26:49.257 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.256+0000 7fa62c43a740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5566ea5b0080 0x5566ea5b8560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:26:49.257 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.256+0000 7fa62c43a740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5566ea5b8c00 0x5566ea5b2b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:26:49.257 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.256+0000 7fa62c43a740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6241b0a10 con 0x5566ea5af6d0 2024-03-21T10:26:49.257 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.256+0000 7fa62c43a740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6241b2390 con 0x5566ea5b8c00 2024-03-21T10:26:49.257 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.256+0000 7fa62c43a740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6241b1070 con 0x5566ea5b0080 2024-03-21T10:26:49.257 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.256+0000 7fa622a00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5566ea5af6d0 0x5566ea5b8020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:26:49.257 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.256+0000 7fa622a00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5566ea5af6d0 0x5566ea5b8020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:58022/0 (socket says 172.21.15.47:58022) 2024-03-21T10:26:49.257 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.256+0000 7fa622a00640 1 -- 172.21.15.47:0/3844754939 learned_addr learned my addr 172.21.15.47:0/3844754939 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:26:49.257 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.256+0000 7fa623400640 1 --2- 172.21.15.47:0/3844754939 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5566ea5b8c00 0x5566ea5b2b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:26:49.257 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.257+0000 7fa622a00640 1 -- 172.21.15.47:0/3844754939 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5566ea5b0080 msgr2=0x5566ea5b8560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:26:49.258 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.257+0000 7fa622a00640 1 --2- 172.21.15.47:0/3844754939 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5566ea5b0080 0x5566ea5b8560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:26:49.258 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.257+0000 7fa622a00640 1 -- 172.21.15.47:0/3844754939 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5566ea5b8c00 msgr2=0x5566ea5b2b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:26:49.258 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.257+0000 7fa622a00640 1 --2- 172.21.15.47:0/3844754939 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5566ea5b8c00 0x5566ea5b2b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:26:49.258 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.257+0000 7fa622a00640 1 -- 172.21.15.47:0/3844754939 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5566ea5cb5c0 con 0x5566ea5af6d0 2024-03-21T10:26:49.258 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.257+0000 7fa622a00640 1 --2- 172.21.15.47:0/3844754939 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5566ea5af6d0 0x5566ea5b8020 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x5566ea5b0480 tx=0x7fa614016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5a16fa924cdec8db server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:26:49.260 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.257+0000 7fa60ea00640 1 -- 172.21.15.47:0/3844754939 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa61401e420 con 0x5566ea5af6d0 2024-03-21T10:26:49.260 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.257+0000 7fa60ea00640 1 -- 172.21.15.47:0/3844754939 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fa614030070 con 0x5566ea5af6d0 2024-03-21T10:26:49.260 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.257+0000 7fa60ea00640 1 -- 172.21.15.47:0/3844754939 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa61401c950 con 0x5566ea5af6d0 2024-03-21T10:26:49.260 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.258+0000 7fa62c43a740 1 -- 172.21.15.47:0/3844754939 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5566ea5af6d0 msgr2=0x5566ea5b8020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:26:49.260 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.258+0000 7fa62c43a740 1 --2- 172.21.15.47:0/3844754939 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5566ea5af6d0 0x5566ea5b8020 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x5566ea5b0480 tx=0x7fa614016000 comp rx=0 tx=0).stop 2024-03-21T10:26:49.260 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.258+0000 7fa62c43a740 1 -- 172.21.15.47:0/3844754939 shutdown_connections 2024-03-21T10:26:49.260 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.258+0000 7fa62c43a740 1 --2- 172.21.15.47:0/3844754939 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5566ea5b8c00 0x5566ea5b2b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:26:49.260 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.258+0000 7fa62c43a740 1 --2- 172.21.15.47:0/3844754939 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5566ea5b0080 0x5566ea5b8560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:26:49.261 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.258+0000 7fa62c43a740 1 --2- 172.21.15.47:0/3844754939 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5566ea5af6d0 0x5566ea5b8020 unknown :-1 s=CLOSED pgs=638 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:26:49.261 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.258+0000 7fa62c43a740 1 -- 172.21.15.47:0/3844754939 >> 172.21.15.47:0/3844754939 conn(0x5566ea5a9da0 msgr2=0x5566ea5a8eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:26:49.261 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.258+0000 7fa62c43a740 1 -- 172.21.15.47:0/3844754939 shutdown_connections 2024-03-21T10:26:49.261 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.258+0000 7fa62c43a740 1 -- 172.21.15.47:0/3844754939 wait complete. 2024-03-21T10:26:49.261 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.258+0000 7fa62c43a740 1 -- start start 2024-03-21T10:26:49.261 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.259+0000 7fa62c43a740 1 Processor -- start 2024-03-21T10:26:49.261 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.259+0000 7fa62c43a740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5566ea5af6d0 0x5566ea5bec10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:26:49.261 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.259+0000 7fa62c43a740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5566ea5b0080 0x5566ea5bf150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:26:49.261 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.259+0000 7fa62c43a740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5566ea5b8c00 0x5566ea61d460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:26:49.261 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.259+0000 7fa62c43a740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6241b1d30 con 0x5566ea5b0080 2024-03-21T10:26:49.261 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.259+0000 7fa62c43a740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6241b16d0 con 0x5566ea5b8c00 2024-03-21T10:26:49.261 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.259+0000 7fa62c43a740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6241aebd0 con 0x5566ea5af6d0 2024-03-21T10:26:49.261 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.259+0000 7fa62c43a740 10 client.0 _ll_register_callbacks cb 0x7fa61c0ba080 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T10:26:49.261 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.260+0000 7fa622000640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5566ea5b0080 0x5566ea5bf150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:26:49.262 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.260+0000 7fa622000640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5566ea5b0080 0x5566ea5bf150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:58038/0 (socket says 172.21.15.47:58038) 2024-03-21T10:26:49.262 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.260+0000 7fa622000640 1 -- 172.21.15.47:0/3052903628 learned_addr learned my addr 172.21.15.47:0/3052903628 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:26:49.262 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.260+0000 7fa623400640 1 --2- 172.21.15.47:0/3052903628 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5566ea5b8c00 0x5566ea61d460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:26:49.262 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.260+0000 7fa622a00640 1 --2- 172.21.15.47:0/3052903628 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5566ea5af6d0 0x5566ea5bec10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:26:49.262 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.260+0000 7fa622000640 1 -- 172.21.15.47:0/3052903628 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5566ea5af6d0 msgr2=0x5566ea5bec10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:26:49.262 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.260+0000 7fa622000640 1 --2- 172.21.15.47:0/3052903628 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5566ea5af6d0 0x5566ea5bec10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:26:49.262 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.260+0000 7fa622000640 1 -- 172.21.15.47:0/3052903628 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5566ea5b8c00 msgr2=0x5566ea61d460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:26:49.262 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.260+0000 7fa622000640 1 --2- 172.21.15.47:0/3052903628 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5566ea5b8c00 0x5566ea61d460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:26:49.262 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.260+0000 7fa622000640 1 -- 172.21.15.47:0/3052903628 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5566ea61daa0 con 0x5566ea5b0080 2024-03-21T10:26:49.264 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.260+0000 7fa622000640 1 --2- 172.21.15.47:0/3052903628 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5566ea5b0080 0x5566ea5bf150 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7fa610005b90 tx=0x7fa610005540 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a3112d2a6d164098 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:26:49.265 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.261+0000 7fa603e00640 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-03-21T10:26:49.265 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.261+0000 7fa603e00640 1 -- 172.21.15.47:0/3052903628 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa610009c80 con 0x5566ea5b0080 2024-03-21T10:26:49.265 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.261+0000 7fa62c43a740 1 -- 172.21.15.47:0/3052903628 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fa614030070 con 0x5566ea5b0080 2024-03-21T10:26:49.265 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.261+0000 7fa603e00640 1 -- 172.21.15.47:0/3052903628 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(97..97 src has 1..97) v4 ==== 9474+0+0 (secure 0 0 0) 0x7fa61000fc50 con 0x5566ea5b0080 2024-03-21T10:26:49.265 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.261+0000 7fa603e00640 1 -- 172.21.15.47:0/3052903628 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa610033050 con 0x5566ea5b0080 2024-03-21T10:26:49.265 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.261+0000 7fa62c43a740 10 client.16233 fetch_fsmap learned FSMap version 32 2024-03-21T10:26:49.265 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.261+0000 7fa62c43a740 1 -- 172.21.15.47:0/3052903628 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x5566ea624e50 con 0x5566ea5b0080 2024-03-21T10:26:49.265 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.262+0000 7fa603e00640 1 -- 172.21.15.47:0/3052903628 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fa614030070 con 0x5566ea5b0080 2024-03-21T10:26:49.265 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.262+0000 7fa603e00640 1 -- 172.21.15.47:0/3052903628 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 32) v1 ==== 7942+0+0 (secure 0 0 0) 0x7fa610028420 con 0x5566ea5b0080 2024-03-21T10:26:49.265 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.262+0000 7fa62c43a740 10 client.16233 fetch_fsmap finished waiting for FSMap version 32 2024-03-21T10:26:49.265 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.262+0000 7fa62c43a740 10 client.16233 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.k 2024-03-21T10:26:49.265 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.262+0000 7fa62c43a740 20 client.16233 populate_metadata read hostname 'smithi047' 2024-03-21T10:26:49.265 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.262+0000 7fa62c43a740 1 --2- 172.21.15.47:0/3052903628 >> [v2:172.21.15.100:6832/2723302038,v1:172.21.15.100:6833/2723302038] conn(0x5566ea625380 0x5566ea6277a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:26:49.266 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.262+0000 7fa62c43a740 4 client.16233 mds_command: new command op to 34427 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-21T10:26:49.266 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.262+0000 7fa62c43a740 1 -- 172.21.15.47:0/3052903628 --> [v2:172.21.15.100:6832/2723302038,v1:172.21.15.100:6833/2723302038] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x5566ea51ab50 con 0x5566ea625380 2024-03-21T10:26:49.266 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.262+0000 7fa622a00640 1 --2- 172.21.15.47:0/3052903628 >> [v2:172.21.15.100:6832/2723302038,v1:172.21.15.100:6833/2723302038] conn(0x5566ea625380 0x5566ea6277a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:26:49.266 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.263+0000 7fa622a00640 1 --2- 172.21.15.47:0/3052903628 >> [v2:172.21.15.100:6832/2723302038,v1:172.21.15.100:6833/2723302038] conn(0x5566ea625380 0x5566ea6277a0 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=6210304f216810ab server_cookie=709467657bd08f90 in_seq=0 out_seq=0 2024-03-21T10:26:49.266 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.263+0000 7fa603e00640 10 client.16233 ms_handle_connect on v2:172.21.15.100:6832/2723302038 2024-03-21T10:26:49.266 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.263+0000 7fa603e00640 1 -- 172.21.15.47:0/3052903628 <== mds.3 v2:172.21.15.100:6832/2723302038 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x5566ea51ab50 con 0x5566ea625380 2024-03-21T10:26:49.266 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.263+0000 7fa603e00640 10 client.16233 handle_command_reply: tid=0 2024-03-21T10:26:49.266 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.263+0000 7fa62c43a740 1 client.16233 shutdown 2024-03-21T10:26:49.266 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.264+0000 7fa62c43a740 1 -- 172.21.15.47:0/3052903628 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5566ea5b0080 msgr2=0x5566ea5bf150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:26:49.266 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.264+0000 7fa62c43a740 1 --2- 172.21.15.47:0/3052903628 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5566ea5b0080 0x5566ea5bf150 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7fa610005b90 tx=0x7fa610005540 comp rx=0 tx=0).stop 2024-03-21T10:26:49.266 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.264+0000 7fa62c43a740 1 -- 172.21.15.47:0/3052903628 shutdown_connections 2024-03-21T10:26:49.266 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.264+0000 7fa62c43a740 1 --2- 172.21.15.47:0/3052903628 >> [v2:172.21.15.100:6832/2723302038,v1:172.21.15.100:6833/2723302038] conn(0x5566ea625380 0x5566ea6277a0 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:26:49.266 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.264+0000 7fa62c43a740 1 --2- 172.21.15.47:0/3052903628 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5566ea5b8c00 0x5566ea61d460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:26:49.267 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.264+0000 7fa62c43a740 1 --2- 172.21.15.47:0/3052903628 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5566ea5b0080 0x5566ea5bf150 unknown :-1 s=CLOSED pgs=639 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:26:49.267 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.264+0000 7fa62c43a740 1 --2- 172.21.15.47:0/3052903628 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5566ea5af6d0 0x5566ea5bec10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:26:49.267 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.264+0000 7fa62c43a740 1 -- 172.21.15.47:0/3052903628 >> 172.21.15.47:0/3052903628 conn(0x5566ea5a9da0 msgr2=0x5566ea5a8620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:26:49.267 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.264+0000 7fa603e00640 0 client.16233 ms_handle_reset on v2:172.21.15.100:6832/2723302038 2024-03-21T10:26:49.267 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.264+0000 7fa62c43a740 1 -- 172.21.15.47:0/3052903628 shutdown_connections 2024-03-21T10:26:49.267 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.264+0000 7fa62c43a740 1 -- 172.21.15.47:0/3052903628 wait complete. 2024-03-21T10:26:49.267 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.264+0000 7fa62c43a740 20 client.16233 trim_cache size 0 max 16384 2024-03-21T10:26:49.267 INFO:teuthology.orchestra.run.smithi047.stdout:[] 2024-03-21T10:26:49.268 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.265+0000 7fa629600640 1 -- 172.21.15.47:0/3564912294 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7fa6080a3230 msgr2=0x7fa6080a56f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:26:49.268 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.265+0000 7fa629600640 1 --2- 172.21.15.47:0/3564912294 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7fa6080a3230 0x7fa6080a56f0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x5566ea5aae10 tx=0x7fa61400b390 comp rx=0 tx=0).stop 2024-03-21T10:26:49.268 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.265+0000 7fa629600640 1 -- 172.21.15.47:0/3564912294 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa62410bcb0 msgr2=0x7fa62419e8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:26:49.268 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.265+0000 7fa629600640 1 --2- 172.21.15.47:0/3564912294 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa62410bcb0 0x7fa62419e8d0 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7fa61c02c7a0 tx=0x7fa61c035e20 comp rx=0 tx=0).stop 2024-03-21T10:26:49.268 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.265+0000 7fa629600640 1 -- 172.21.15.47:0/3564912294 shutdown_connections 2024-03-21T10:26:49.268 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.265+0000 7fa629600640 1 --2- 172.21.15.47:0/3564912294 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa62410bcb0 0x7fa62419e8d0 unknown :-1 s=CLOSED pgs=636 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:26:49.268 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.265+0000 7fa629600640 1 --2- 172.21.15.47:0/3564912294 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7fa6080a3230 0x7fa6080a56f0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:26:49.268 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.265+0000 7fa629600640 1 --2- 172.21.15.47:0/3564912294 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa624103200 0x7fa62419a550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:26:49.268 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.265+0000 7fa629600640 1 --2- 172.21.15.47:0/3564912294 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa6241028e0 0x7fa62419a010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:26:49.269 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.265+0000 7fa629600640 1 -- 172.21.15.47:0/3564912294 >> 172.21.15.47:0/3564912294 conn(0x7fa624076920 msgr2=0x7fa6241004c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:26:49.269 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.265+0000 7fa629600640 1 -- 172.21.15.47:0/3564912294 shutdown_connections 2024-03-21T10:26:49.269 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.265+0000 7fa629600640 1 -- 172.21.15.47:0/3564912294 wait complete. 2024-03-21T10:26:49.282 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-21T10:26:49.282 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-03-21T10:26:49.341 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:26:49 smithi047 ceph-mon[31547]: pgmap v48: 129 pgs: 129 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 9 op/s 2024-03-21T10:26:49.341 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:26:49 smithi047 ceph-mon[31547]: from='client.? 172.21.15.47:0/1919726689' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-21T10:26:49.473 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.472+0000 7f10a5a00640 1 -- 172.21.15.47:0/386675919 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f10a010e040 msgr2=0x7f10a0110430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:26:49.473 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.472+0000 7f10a5a00640 1 --2- 172.21.15.47:0/386675919 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f10a010e040 0x7f10a0110430 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7f1098009ca0 tx=0x7f109802c100 comp rx=0 tx=0).stop 2024-03-21T10:26:49.473 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.473+0000 7f10a5a00640 1 -- 172.21.15.47:0/386675919 shutdown_connections 2024-03-21T10:26:49.473 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.473+0000 7f10a5a00640 1 --2- 172.21.15.47:0/386675919 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f10a010e040 0x7f10a0110430 unknown :-1 s=CLOSED pgs=640 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:26:49.473 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.473+0000 7f10a5a00640 1 --2- 172.21.15.47:0/386675919 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f10a00691a0 0x7f10a010db00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:26:49.473 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.473+0000 7f10a5a00640 1 --2- 172.21.15.47:0/386675919 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f10a0068880 0x7f10a0068c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:26:49.474 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.473+0000 7f10a5a00640 1 -- 172.21.15.47:0/386675919 >> 172.21.15.47:0/386675919 conn(0x7f10a00776a0 msgr2=0x7f10a0077ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:26:49.474 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.473+0000 7f10a5a00640 1 -- 172.21.15.47:0/386675919 shutdown_connections 2024-03-21T10:26:49.474 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.473+0000 7f10a5a00640 1 -- 172.21.15.47:0/386675919 wait complete. 2024-03-21T10:26:49.476 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.474+0000 7f10a5a00640 1 Processor -- start 2024-03-21T10:26:49.476 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.474+0000 7f10a5a00640 1 -- start start 2024-03-21T10:26:49.476 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.474+0000 7f10a5a00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f10a0068880 0x7f10a019a050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:26:49.476 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.474+0000 7f10a5a00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f10a00691a0 0x7f10a019a590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:26:49.476 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.474+0000 7f10a5a00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f10a010e040 0x7f10a019e910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:26:49.476 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.474+0000 7f10a5a00640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10a0113ab0 con 0x7f10a00691a0 2024-03-21T10:26:49.476 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.474+0000 7f10a5a00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10a0113930 con 0x7f10a010e040 2024-03-21T10:26:49.476 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.474+0000 7f10a5a00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10a0113c30 con 0x7f10a0068880 2024-03-21T10:26:49.477 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.474+0000 7f109e000640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f10a00691a0 0x7f10a019a590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:26:49.477 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.474+0000 7f109e000640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f10a00691a0 0x7f10a019a590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:58066/0 (socket says 172.21.15.47:58066) 2024-03-21T10:26:49.477 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.474+0000 7f109e000640 1 -- 172.21.15.47:0/1300194518 learned_addr learned my addr 172.21.15.47:0/1300194518 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:26:49.477 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.475+0000 7f109f400640 1 --2- 172.21.15.47:0/1300194518 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f10a010e040 0x7f10a019e910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:26:49.477 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.475+0000 7f109ea00640 1 --2- 172.21.15.47:0/1300194518 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f10a0068880 0x7f10a019a050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:26:49.477 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.475+0000 7f109e000640 1 -- 172.21.15.47:0/1300194518 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f10a0068880 msgr2=0x7f10a019a050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:26:49.477 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.475+0000 7f109e000640 1 --2- 172.21.15.47:0/1300194518 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f10a0068880 0x7f10a019a050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:26:49.477 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.475+0000 7f109e000640 1 -- 172.21.15.47:0/1300194518 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f10a010e040 msgr2=0x7f10a019e910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:26:49.477 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.475+0000 7f109e000640 1 --2- 172.21.15.47:0/1300194518 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f10a010e040 0x7f10a019e910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:26:49.477 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.475+0000 7f109e000640 1 -- 172.21.15.47:0/1300194518 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f10a019f040 con 0x7f10a00691a0 2024-03-21T10:26:49.477 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.475+0000 7f109e000640 1 --2- 172.21.15.47:0/1300194518 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f10a00691a0 0x7f10a019a590 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7f108c00e9f0 tx=0x7f108c00edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:26:49.477 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.475+0000 7f1097400640 1 -- 172.21.15.47:0/1300194518 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f108c00cd80 con 0x7f10a00691a0 2024-03-21T10:26:49.477 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.476+0000 7f1097400640 1 -- 172.21.15.47:0/1300194518 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f108c0054d0 con 0x7f10a00691a0 2024-03-21T10:26:49.477 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.476+0000 7f1097400640 1 -- 172.21.15.47:0/1300194518 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f108c010490 con 0x7f10a00691a0 2024-03-21T10:26:49.477 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.476+0000 7f10a5a00640 1 -- 172.21.15.47:0/1300194518 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f10a019f330 con 0x7f10a00691a0 2024-03-21T10:26:49.478 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.476+0000 7f10a5a00640 1 -- 172.21.15.47:0/1300194518 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f10a01a6bc0 con 0x7f10a00691a0 2024-03-21T10:26:49.479 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.479+0000 7f10a87b2740 1 Processor -- start 2024-03-21T10:26:49.479 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.479+0000 7f10a87b2740 1 -- start start 2024-03-21T10:26:49.480 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.480+0000 7f1097400640 1 -- 172.21.15.47:0/1300194518 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 26) v1 ==== 138087+0+0 (secure 0 0 0) 0x7f108c005020 con 0x7f10a00691a0 2024-03-21T10:26:49.480 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.480+0000 7f10a87b2740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f43a0fa6d0 0x55f43a0faab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:26:49.481 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.480+0000 7f10a87b2740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f43a0fb080 0x55f43a103630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:26:49.481 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.480+0000 7f10a87b2740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f43a103c00 0x55f43a106050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:26:49.481 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.480+0000 7f10a87b2740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f43a05e2c0 con 0x55f43a0fb080 2024-03-21T10:26:49.481 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.480+0000 7f10a87b2740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f43a006c90 con 0x55f43a103c00 2024-03-21T10:26:49.481 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.480+0000 7f10a87b2740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f43a05a360 con 0x55f43a0fa6d0 2024-03-21T10:26:49.481 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.480+0000 7f109ea00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f43a0fb080 0x55f43a103630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:26:49.481 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.480+0000 7f109ea00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f43a0fb080 0x55f43a103630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:58070/0 (socket says 172.21.15.47:58070) 2024-03-21T10:26:49.481 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.480+0000 7f109ea00640 1 -- 172.21.15.47:0/2169263615 learned_addr learned my addr 172.21.15.47:0/2169263615 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:26:49.481 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.480+0000 7f109e000640 1 --2- 172.21.15.47:0/2169263615 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f43a103c00 0x55f43a106050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:26:49.481 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.480+0000 7f109f400640 1 --2- 172.21.15.47:0/2169263615 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f43a0fa6d0 0x55f43a0faab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:26:49.481 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.480+0000 7f109ea00640 1 -- 172.21.15.47:0/2169263615 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f43a0fa6d0 msgr2=0x55f43a0faab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:26:49.481 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.480+0000 7f109ea00640 1 --2- 172.21.15.47:0/2169263615 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f43a0fa6d0 0x55f43a0faab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:26:49.481 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.480+0000 7f109ea00640 1 -- 172.21.15.47:0/2169263615 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f43a103c00 msgr2=0x55f43a106050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:26:49.481 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.480+0000 7f109ea00640 1 --2- 172.21.15.47:0/2169263615 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f43a103c00 0x55f43a106050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:26:49.481 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.480+0000 7f109ea00640 1 -- 172.21.15.47:0/2169263615 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f43a02b080 con 0x55f43a0fb080 2024-03-21T10:26:49.485 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.481+0000 7f109ea00640 1 --2- 172.21.15.47:0/2169263615 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f43a0fb080 0x55f43a103630 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7f1090009c80 tx=0x7f10900117b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d5c2e79906dda8fb server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:26:49.485 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.481+0000 7f108be00640 1 -- 172.21.15.47:0/2169263615 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1090012450 con 0x55f43a0fb080 2024-03-21T10:26:49.485 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.481+0000 7f108be00640 1 -- 172.21.15.47:0/2169263615 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f1090012aa0 con 0x55f43a0fb080 2024-03-21T10:26:49.485 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.481+0000 7f108be00640 1 -- 172.21.15.47:0/2169263615 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f109000fb30 con 0x55f43a0fb080 2024-03-21T10:26:49.485 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.481+0000 7f1097400640 1 --2- 172.21.15.47:0/1300194518 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f10840a31b0 0x7f10840a5670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:26:49.485 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.482+0000 7f1097400640 1 -- 172.21.15.47:0/1300194518 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 9474+0+0 (secure 0 0 0) 0x7f108c0d01c0 con 0x7f10a00691a0 2024-03-21T10:26:49.486 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.482+0000 7f109f400640 1 --2- 172.21.15.47:0/1300194518 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f10840a31b0 0x7f10840a5670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:26:49.486 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.482+0000 7f10a87b2740 1 -- 172.21.15.47:0/2169263615 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f43a0fb080 msgr2=0x55f43a103630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:26:49.486 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.482+0000 7f10a87b2740 1 --2- 172.21.15.47:0/2169263615 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f43a0fb080 0x55f43a103630 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7f1090009c80 tx=0x7f10900117b0 comp rx=0 tx=0).stop 2024-03-21T10:26:49.486 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.482+0000 7f10a87b2740 1 -- 172.21.15.47:0/2169263615 shutdown_connections 2024-03-21T10:26:49.486 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.482+0000 7f10a87b2740 1 --2- 172.21.15.47:0/2169263615 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f43a103c00 0x55f43a106050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:26:49.486 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.482+0000 7f10a87b2740 1 --2- 172.21.15.47:0/2169263615 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f43a0fb080 0x55f43a103630 unknown :-1 s=CLOSED pgs=642 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:26:49.486 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.482+0000 7f10a87b2740 1 --2- 172.21.15.47:0/2169263615 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f43a0fa6d0 0x55f43a0faab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:26:49.486 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.482+0000 7f10a87b2740 1 -- 172.21.15.47:0/2169263615 >> 172.21.15.47:0/2169263615 conn(0x55f43a0f4da0 msgr2=0x55f43a0f5180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:26:49.486 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.482+0000 7f10a87b2740 1 -- 172.21.15.47:0/2169263615 shutdown_connections 2024-03-21T10:26:49.486 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.482+0000 7f109f400640 1 --2- 172.21.15.47:0/1300194518 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f10840a31b0 0x7f10840a5670 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f1098005cc0 tx=0x7f109802e040 comp rx=0 tx=0).ready entity=mgr.15687 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:26:49.486 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.482+0000 7f10a87b2740 1 -- 172.21.15.47:0/2169263615 wait complete. 2024-03-21T10:26:49.486 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.483+0000 7f10a87b2740 1 -- start start 2024-03-21T10:26:49.486 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.483+0000 7f10a87b2740 1 Processor -- start 2024-03-21T10:26:49.486 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.483+0000 7f10a87b2740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f43a0fa6d0 0x55f43a1035d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:26:49.487 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.483+0000 7f10a87b2740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f43a0fb080 0x55f43a100960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:26:49.487 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.483+0000 7f10a87b2740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f43a103c00 0x55f43a101080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:26:49.487 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.484+0000 7f10a87b2740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f43a106b70 con 0x55f43a103c00 2024-03-21T10:26:49.487 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.484+0000 7f10a87b2740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f43a01a0d0 con 0x55f43a0fa6d0 2024-03-21T10:26:49.487 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.484+0000 7f10a87b2740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f43a106cf0 con 0x55f43a0fb080 2024-03-21T10:26:49.487 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.484+0000 7f10a87b2740 10 client.0 _ll_register_callbacks cb 0x55f43a01ee80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T10:26:49.487 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.484+0000 7f109f400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f43a103c00 0x55f43a101080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:26:49.487 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.484+0000 7f109f400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f43a103c00 0x55f43a101080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:58074/0 (socket says 172.21.15.47:58074) 2024-03-21T10:26:49.487 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.484+0000 7f109f400640 1 -- 172.21.15.47:0/1983295916 learned_addr learned my addr 172.21.15.47:0/1983295916 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:26:49.487 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.484+0000 7f109ea00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f43a0fa6d0 0x55f43a1035d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:26:49.487 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.484+0000 7f109e000640 1 --2- 172.21.15.47:0/1983295916 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f43a0fb080 0x55f43a100960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:26:49.487 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.484+0000 7f109f400640 1 -- 172.21.15.47:0/1983295916 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f43a0fb080 msgr2=0x55f43a100960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:26:49.487 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.484+0000 7f109f400640 1 --2- 172.21.15.47:0/1983295916 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f43a0fb080 0x55f43a100960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:26:49.487 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.484+0000 7f109f400640 1 -- 172.21.15.47:0/1983295916 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f43a0fa6d0 msgr2=0x55f43a1035d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:26:49.488 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.484+0000 7f109f400640 1 --2- 172.21.15.47:0/1983295916 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f43a0fa6d0 0x55f43a1035d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:26:49.488 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.484+0000 7f109f400640 1 -- 172.21.15.47:0/1983295916 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55f439d723f0 con 0x55f43a103c00 2024-03-21T10:26:49.488 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.484+0000 7f109e000640 1 --2- 172.21.15.47:0/1983295916 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f43a0fb080 0x55f43a100960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T10:26:49.488 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.485+0000 7f109f400640 1 --2- 172.21.15.47:0/1983295916 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f43a103c00 0x55f43a101080 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7f1098006010 tx=0x7f109802c100 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fe657b0f7753e20 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:26:49.488 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.485+0000 7f1088c00640 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-03-21T10:26:49.488 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.485+0000 7f1088c00640 1 -- 172.21.15.47:0/1983295916 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1098039040 con 0x55f43a103c00 2024-03-21T10:26:49.488 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.485+0000 7f1088c00640 1 -- 172.21.15.47:0/1983295916 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(97..97 src has 1..97) v4 ==== 9474+0+0 (secure 0 0 0) 0x7f109804c660 con 0x55f43a103c00 2024-03-21T10:26:49.488 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.485+0000 7f10a87b2740 1 -- 172.21.15.47:0/1983295916 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f1090012aa0 con 0x55f43a103c00 2024-03-21T10:26:49.488 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.485+0000 7f1088c00640 1 -- 172.21.15.47:0/1983295916 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f109805a050 con 0x55f43a103c00 2024-03-21T10:26:49.488 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.485+0000 7f10a87b2740 10 client.16245 fetch_fsmap learned FSMap version 32 2024-03-21T10:26:49.488 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.485+0000 7f10a87b2740 1 -- 172.21.15.47:0/1983295916 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x55f439e3c7d0 con 0x55f43a103c00 2024-03-21T10:26:49.490 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.485+0000 7f1088c00640 1 -- 172.21.15.47:0/1983295916 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f1090012aa0 con 0x55f43a103c00 2024-03-21T10:26:49.490 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.486+0000 7f1088c00640 1 -- 172.21.15.47:0/1983295916 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 32) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f1098040a40 con 0x55f43a103c00 2024-03-21T10:26:49.490 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.486+0000 7f10a87b2740 10 client.16245 fetch_fsmap finished waiting for FSMap version 32 2024-03-21T10:26:49.490 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.486+0000 7f10a87b2740 10 client.16245 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.e 2024-03-21T10:26:49.490 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.486+0000 7f10a87b2740 20 client.16245 populate_metadata read hostname 'smithi047' 2024-03-21T10:26:49.491 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.486+0000 7f10a87b2740 1 --2- 172.21.15.47:0/1983295916 >> [v2:172.21.15.100:6834/2597849957,v1:172.21.15.100:6835/2597849957] conn(0x55f43a113b40 0x55f43a115f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:26:49.491 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.486+0000 7f10a87b2740 4 client.16245 mds_command: new command op to 34454 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T10:26:49.491 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.486+0000 7f10a87b2740 1 -- 172.21.15.47:0/1983295916 --> [v2:172.21.15.100:6834/2597849957,v1:172.21.15.100:6835/2597849957] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55f43a06b800 con 0x55f43a113b40 2024-03-21T10:26:49.491 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.486+0000 7f109ea00640 1 --2- 172.21.15.47:0/1983295916 >> [v2:172.21.15.100:6834/2597849957,v1:172.21.15.100:6835/2597849957] conn(0x55f43a113b40 0x55f43a115f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:26:49.491 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.487+0000 7f1088c00640 10 client.16245 ms_handle_connect on v2:172.21.15.100:6834/2597849957 2024-03-21T10:26:49.491 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.487+0000 7f109ea00640 1 --2- 172.21.15.47:0/1983295916 >> [v2:172.21.15.100:6834/2597849957,v1:172.21.15.100:6835/2597849957] conn(0x55f43a113b40 0x55f43a115f50 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=838fe871ab8d97cd server_cookie=fb94358d9042b798 in_seq=0 out_seq=0 2024-03-21T10:26:49.491 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.488+0000 7f1088c00640 1 -- 172.21.15.47:0/1983295916 <== mds.2 v2:172.21.15.100:6834/2597849957 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55f43a06b800 con 0x55f43a113b40 2024-03-21T10:26:49.491 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.488+0000 7f1088c00640 10 client.16245 handle_command_reply: tid=0 2024-03-21T10:26:49.491 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.488+0000 7f10a87b2740 1 client.16245 shutdown 2024-03-21T10:26:49.491 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.489+0000 7f10a87b2740 1 -- 172.21.15.47:0/1983295916 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f43a103c00 msgr2=0x55f43a101080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:26:49.491 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.489+0000 7f10a87b2740 1 --2- 172.21.15.47:0/1983295916 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f43a103c00 0x55f43a101080 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7f1098006010 tx=0x7f109802c100 comp rx=0 tx=0).stop 2024-03-21T10:26:49.491 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.489+0000 7f10a87b2740 1 -- 172.21.15.47:0/1983295916 shutdown_connections 2024-03-21T10:26:49.491 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.489+0000 7f10a87b2740 1 --2- 172.21.15.47:0/1983295916 >> [v2:172.21.15.100:6834/2597849957,v1:172.21.15.100:6835/2597849957] conn(0x55f43a113b40 0x55f43a115f50 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:26:49.491 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.489+0000 7f10a87b2740 1 --2- 172.21.15.47:0/1983295916 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f43a103c00 0x55f43a101080 unknown :-1 s=CLOSED pgs=643 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:26:49.492 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.489+0000 7f10a87b2740 1 --2- 172.21.15.47:0/1983295916 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f43a0fb080 0x55f43a100960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:26:49.492 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.489+0000 7f10a87b2740 1 --2- 172.21.15.47:0/1983295916 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f43a0fa6d0 0x55f43a1035d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:26:49.492 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.489+0000 7f1088c00640 0 client.16245 ms_handle_reset on v2:172.21.15.100:6834/2597849957 2024-03-21T10:26:49.492 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.489+0000 7f10a87b2740 1 -- 172.21.15.47:0/1983295916 >> 172.21.15.47:0/1983295916 conn(0x55f43a0f4da0 msgr2=0x55f43a0f6af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:26:49.492 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.490+0000 7f10a87b2740 1 -- 172.21.15.47:0/1983295916 shutdown_connections 2024-03-21T10:26:49.492 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.490+0000 7f10a87b2740 1 -- 172.21.15.47:0/1983295916 wait complete. 2024-03-21T10:26:49.492 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.490+0000 7f10a87b2740 20 client.16245 trim_cache size 0 max 16384 2024-03-21T10:26:49.504 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.503+0000 7f10a87b2740 1 Processor -- start 2024-03-21T10:26:49.504 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.503+0000 7f10a87b2740 1 -- start start 2024-03-21T10:26:49.504 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.503+0000 7f10a87b2740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f43a0fa6d0 0x55f43a0fb4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:26:49.504 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.503+0000 7f10a87b2740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f43a0fb080 0x55f43a103090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:26:49.504 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.503+0000 7f10a87b2740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f43a103c00 0x55f43a1035d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:26:49.504 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.503+0000 7f10a87b2740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10a01b0aa0 con 0x55f43a0fa6d0 2024-03-21T10:26:49.504 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.503+0000 7f10a87b2740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10a01b2420 con 0x55f43a0fb080 2024-03-21T10:26:49.504 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.503+0000 7f10a87b2740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10a01b1100 con 0x55f43a103c00 2024-03-21T10:26:49.504 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.503+0000 7f109ea00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f43a0fa6d0 0x55f43a0fb4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:26:49.505 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.503+0000 7f109ea00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f43a0fa6d0 0x55f43a0fb4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:58086/0 (socket says 172.21.15.47:58086) 2024-03-21T10:26:49.505 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.503+0000 7f109ea00640 1 -- 172.21.15.47:0/656303252 learned_addr learned my addr 172.21.15.47:0/656303252 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:26:49.505 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.504+0000 7f109e000640 1 --2- 172.21.15.47:0/656303252 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f43a0fb080 0x55f43a103090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:26:49.505 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.504+0000 7f109ea00640 1 -- 172.21.15.47:0/656303252 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f43a103c00 msgr2=0x55f43a1035d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:26:49.505 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.504+0000 7f109f400640 1 --2- 172.21.15.47:0/656303252 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f43a103c00 0x55f43a1035d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:26:49.505 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.504+0000 7f109ea00640 1 --2- 172.21.15.47:0/656303252 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f43a103c00 0x55f43a1035d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:26:49.505 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.504+0000 7f109ea00640 1 -- 172.21.15.47:0/656303252 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f43a0fb080 msgr2=0x55f43a103090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:26:49.505 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.504+0000 7f109ea00640 1 --2- 172.21.15.47:0/656303252 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f43a0fb080 0x55f43a103090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:26:49.505 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.504+0000 7f109ea00640 1 -- 172.21.15.47:0/656303252 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f43a0fae20 con 0x55f43a0fa6d0 2024-03-21T10:26:49.505 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.504+0000 7f109ea00640 1 --2- 172.21.15.47:0/656303252 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f43a0fa6d0 0x55f43a0fb4b0 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7f1090009c80 tx=0x7f1090009530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2fedc3de05f5eca2 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:26:49.508 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.504+0000 7f108b400640 1 -- 172.21.15.47:0/656303252 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f109002a2b0 con 0x55f43a0fa6d0 2024-03-21T10:26:49.508 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.504+0000 7f108b400640 1 -- 172.21.15.47:0/656303252 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f1090009df0 con 0x55f43a0fa6d0 2024-03-21T10:26:49.508 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.504+0000 7f108b400640 1 -- 172.21.15.47:0/656303252 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1090005370 con 0x55f43a0fa6d0 2024-03-21T10:26:49.509 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.505+0000 7f10a87b2740 1 -- 172.21.15.47:0/656303252 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f43a0fa6d0 msgr2=0x55f43a0fb4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:26:49.509 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.505+0000 7f10a87b2740 1 --2- 172.21.15.47:0/656303252 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f43a0fa6d0 0x55f43a0fb4b0 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7f1090009c80 tx=0x7f1090009530 comp rx=0 tx=0).stop 2024-03-21T10:26:49.509 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.505+0000 7f10a87b2740 1 -- 172.21.15.47:0/656303252 shutdown_connections 2024-03-21T10:26:49.509 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.505+0000 7f10a87b2740 1 --2- 172.21.15.47:0/656303252 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f43a103c00 0x55f43a1035d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:26:49.509 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.505+0000 7f10a87b2740 1 --2- 172.21.15.47:0/656303252 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f43a0fb080 0x55f43a103090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:26:49.509 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.505+0000 7f10a87b2740 1 --2- 172.21.15.47:0/656303252 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f43a0fa6d0 0x55f43a0fb4b0 unknown :-1 s=CLOSED pgs=644 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:26:49.509 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.505+0000 7f10a87b2740 1 -- 172.21.15.47:0/656303252 >> 172.21.15.47:0/656303252 conn(0x55f43a0f4da0 msgr2=0x55f43a0f66d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:26:49.509 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.505+0000 7f10a87b2740 1 -- 172.21.15.47:0/656303252 shutdown_connections 2024-03-21T10:26:49.509 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.505+0000 7f10a87b2740 1 -- 172.21.15.47:0/656303252 wait complete. 2024-03-21T10:26:49.509 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.505+0000 7f10a87b2740 1 -- start start 2024-03-21T10:26:49.509 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.506+0000 7f10a87b2740 1 Processor -- start 2024-03-21T10:26:49.509 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.506+0000 7f10a87b2740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f43a0fa6d0 0x55f43a109a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:26:49.509 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.506+0000 7f10a87b2740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f43a0fb080 0x55f43a109fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:26:49.510 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.506+0000 7f10a87b2740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f43a103c00 0x55f43a16e3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:26:49.510 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.506+0000 7f10a87b2740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10a01b1dc0 con 0x55f43a0fa6d0 2024-03-21T10:26:49.510 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.506+0000 7f10a87b2740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10a01b1760 con 0x55f43a103c00 2024-03-21T10:26:49.510 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.506+0000 7f10a87b2740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10a01aec60 con 0x55f43a0fb080 2024-03-21T10:26:49.510 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.506+0000 7f10a87b2740 10 client.0 _ll_register_callbacks cb 0x7f1090005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T10:26:49.510 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.506+0000 7f109ea00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f43a0fa6d0 0x55f43a109a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:26:49.510 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.506+0000 7f109ea00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f43a0fa6d0 0x55f43a109a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:58100/0 (socket says 172.21.15.47:58100) 2024-03-21T10:26:49.510 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.506+0000 7f109ea00640 1 -- 172.21.15.47:0/1302735686 learned_addr learned my addr 172.21.15.47:0/1302735686 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:26:49.510 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.506+0000 7f109f400640 1 --2- 172.21.15.47:0/1302735686 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f43a103c00 0x55f43a16e3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:26:49.510 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.507+0000 7f109e000640 1 --2- 172.21.15.47:0/1302735686 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f43a0fb080 0x55f43a109fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:26:49.510 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.507+0000 7f109ea00640 1 -- 172.21.15.47:0/1302735686 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f43a0fb080 msgr2=0x55f43a109fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:26:49.510 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.507+0000 7f109ea00640 1 --2- 172.21.15.47:0/1302735686 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f43a0fb080 0x55f43a109fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:26:49.510 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.507+0000 7f109ea00640 1 -- 172.21.15.47:0/1302735686 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f43a103c00 msgr2=0x55f43a16e3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:26:49.510 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.507+0000 7f109ea00640 1 --2- 172.21.15.47:0/1302735686 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f43a103c00 0x55f43a16e3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:26:49.511 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.507+0000 7f109ea00640 1 -- 172.21.15.47:0/1302735686 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55f43a16e930 con 0x55f43a0fa6d0 2024-03-21T10:26:49.511 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.507+0000 7f109ea00640 1 --2- 172.21.15.47:0/1302735686 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f43a0fa6d0 0x55f43a109a80 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7f1090009650 tx=0x7f1090009680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b045801c84a85fbd server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:26:49.511 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.507+0000 7f1088c00640 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-03-21T10:26:49.511 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.507+0000 7f1088c00640 1 -- 172.21.15.47:0/1302735686 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f109002ad10 con 0x55f43a0fa6d0 2024-03-21T10:26:49.511 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.507+0000 7f1088c00640 1 -- 172.21.15.47:0/1302735686 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(97..97 src has 1..97) v4 ==== 9474+0+0 (secure 0 0 0) 0x7f10900050b0 con 0x55f43a0fa6d0 2024-03-21T10:26:49.511 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.507+0000 7f10a87b2740 1 -- 172.21.15.47:0/1302735686 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f1090009df0 con 0x55f43a0fa6d0 2024-03-21T10:26:49.511 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.508+0000 7f1088c00640 1 -- 172.21.15.47:0/1302735686 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f10900052b0 con 0x55f43a0fa6d0 2024-03-21T10:26:49.511 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.508+0000 7f10a87b2740 10 client.16251 fetch_fsmap learned FSMap version 32 2024-03-21T10:26:49.511 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.508+0000 7f10a87b2740 1 -- 172.21.15.47:0/1302735686 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x55f43a16fc30 con 0x55f43a0fa6d0 2024-03-21T10:26:49.511 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.508+0000 7f1088c00640 1 -- 172.21.15.47:0/1302735686 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f1090009df0 con 0x55f43a0fa6d0 2024-03-21T10:26:49.511 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.508+0000 7f1088c00640 1 -- 172.21.15.47:0/1302735686 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 32) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f10900035f0 con 0x55f43a0fa6d0 2024-03-21T10:26:49.511 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.508+0000 7f10a87b2740 10 client.16251 fetch_fsmap finished waiting for FSMap version 32 2024-03-21T10:26:49.511 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.508+0000 7f10a87b2740 10 client.16251 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.e 2024-03-21T10:26:49.511 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.508+0000 7f10a87b2740 20 client.16251 populate_metadata read hostname 'smithi047' 2024-03-21T10:26:49.511 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.508+0000 7f10a87b2740 1 --2- 172.21.15.47:0/1302735686 >> [v2:172.21.15.100:6834/2597849957,v1:172.21.15.100:6835/2597849957] conn(0x55f43a170150 0x55f43a172540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:26:49.512 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.508+0000 7f10a87b2740 4 client.16251 mds_command: new command op to 34454 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-21T10:26:49.512 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.508+0000 7f10a87b2740 1 -- 172.21.15.47:0/1302735686 --> [v2:172.21.15.100:6834/2597849957,v1:172.21.15.100:6835/2597849957] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55f43a065b50 con 0x55f43a170150 2024-03-21T10:26:49.513 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.509+0000 7f109e000640 1 --2- 172.21.15.47:0/1302735686 >> [v2:172.21.15.100:6834/2597849957,v1:172.21.15.100:6835/2597849957] conn(0x55f43a170150 0x55f43a172540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:26:49.513 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.510+0000 7f109e000640 1 --2- 172.21.15.47:0/1302735686 >> [v2:172.21.15.100:6834/2597849957,v1:172.21.15.100:6835/2597849957] conn(0x55f43a170150 0x55f43a172540 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=846c6de2c394fe59 server_cookie=3ecd57673b15fe23 in_seq=0 out_seq=0 2024-03-21T10:26:49.513 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.510+0000 7f1088c00640 10 client.16251 ms_handle_connect on v2:172.21.15.100:6834/2597849957 2024-03-21T10:26:49.513 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.510+0000 7f1088c00640 1 -- 172.21.15.47:0/1302735686 <== mds.2 v2:172.21.15.100:6834/2597849957 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55f43a065b50 con 0x55f43a170150 2024-03-21T10:26:49.513 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.510+0000 7f1088c00640 10 client.16251 handle_command_reply: tid=0 2024-03-21T10:26:49.513 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.510+0000 7f10a87b2740 1 client.16251 shutdown 2024-03-21T10:26:49.513 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.510+0000 7f10a87b2740 1 -- 172.21.15.47:0/1302735686 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f43a0fa6d0 msgr2=0x55f43a109a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:26:49.514 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.510+0000 7f10a87b2740 1 --2- 172.21.15.47:0/1302735686 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f43a0fa6d0 0x55f43a109a80 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7f1090009650 tx=0x7f1090009680 comp rx=0 tx=0).stop 2024-03-21T10:26:49.514 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.510+0000 7f10a87b2740 1 -- 172.21.15.47:0/1302735686 shutdown_connections 2024-03-21T10:26:49.514 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.510+0000 7f10a87b2740 1 --2- 172.21.15.47:0/1302735686 >> [v2:172.21.15.100:6834/2597849957,v1:172.21.15.100:6835/2597849957] conn(0x55f43a170150 0x55f43a172540 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:26:49.514 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.510+0000 7f10a87b2740 1 --2- 172.21.15.47:0/1302735686 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f43a103c00 0x55f43a16e3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:26:49.514 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.510+0000 7f10a87b2740 1 --2- 172.21.15.47:0/1302735686 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f43a0fb080 0x55f43a109fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:26:49.514 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.510+0000 7f10a87b2740 1 --2- 172.21.15.47:0/1302735686 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f43a0fa6d0 0x55f43a109a80 unknown :-1 s=CLOSED pgs=645 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:26:49.514 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.510+0000 7f1088c00640 0 client.16251 ms_handle_reset on v2:172.21.15.100:6834/2597849957 2024-03-21T10:26:49.514 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.510+0000 7f10a87b2740 1 -- 172.21.15.47:0/1302735686 >> 172.21.15.47:0/1302735686 conn(0x55f43a0f4da0 msgr2=0x55f43a11b0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:26:49.514 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.510+0000 7f10a87b2740 1 -- 172.21.15.47:0/1302735686 shutdown_connections 2024-03-21T10:26:49.514 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.510+0000 7f10a87b2740 1 -- 172.21.15.47:0/1302735686 wait complete. 2024-03-21T10:26:49.514 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.511+0000 7f10a87b2740 20 client.16251 trim_cache size 0 max 16384 2024-03-21T10:26:49.515 INFO:teuthology.orchestra.run.smithi047.stdout:[] 2024-03-21T10:26:49.515 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.511+0000 7f10a5a00640 1 -- 172.21.15.47:0/1300194518 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f10840a31b0 msgr2=0x7f10840a5670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:26:49.515 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.511+0000 7f10a5a00640 1 --2- 172.21.15.47:0/1300194518 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f10840a31b0 0x7f10840a5670 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f1098005cc0 tx=0x7f109802e040 comp rx=0 tx=0).stop 2024-03-21T10:26:49.515 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.511+0000 7f10a5a00640 1 -- 172.21.15.47:0/1300194518 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f10a00691a0 msgr2=0x7f10a019a590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:26:49.515 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.511+0000 7f10a5a00640 1 --2- 172.21.15.47:0/1300194518 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f10a00691a0 0x7f10a019a590 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7f108c00e9f0 tx=0x7f108c00edf0 comp rx=0 tx=0).stop 2024-03-21T10:26:49.516 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.511+0000 7f10a5a00640 1 -- 172.21.15.47:0/1300194518 shutdown_connections 2024-03-21T10:26:49.516 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.511+0000 7f10a5a00640 1 --2- 172.21.15.47:0/1300194518 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f10840a31b0 0x7f10840a5670 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:26:49.516 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.511+0000 7f10a5a00640 1 --2- 172.21.15.47:0/1300194518 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f10a010e040 0x7f10a019e910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:26:49.516 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.511+0000 7f10a5a00640 1 --2- 172.21.15.47:0/1300194518 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f10a00691a0 0x7f10a019a590 unknown :-1 s=CLOSED pgs=641 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:26:49.516 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.511+0000 7f10a5a00640 1 --2- 172.21.15.47:0/1300194518 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f10a0068880 0x7f10a019a050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:26:49.516 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.511+0000 7f10a5a00640 1 -- 172.21.15.47:0/1300194518 >> 172.21.15.47:0/1300194518 conn(0x7f10a00776a0 msgr2=0x7f10a010f670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:26:49.516 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.511+0000 7f10a5a00640 1 -- 172.21.15.47:0/1300194518 shutdown_connections 2024-03-21T10:26:49.516 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.511+0000 7f10a5a00640 1 -- 172.21.15.47:0/1300194518 wait complete. 2024-03-21T10:26:49.527 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-21T10:26:49.527 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-03-21T10:26:49.529 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:26:49 smithi100 ceph-mon[35193]: pgmap v48: 129 pgs: 129 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 9 op/s 2024-03-21T10:26:49.529 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:26:49 smithi100 ceph-mon[35193]: from='client.? 172.21.15.47:0/1919726689' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-21T10:26:49.714 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.714+0000 7fa27cc00640 1 -- 172.21.15.47:0/1052143508 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa278102ff0 msgr2=0x7fa27810b4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:26:49.715 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.714+0000 7fa27cc00640 1 --2- 172.21.15.47:0/1052143508 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa278102ff0 0x7fa27810b4d0 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7fa264009900 tx=0x7fa26402bcd0 comp rx=0 tx=0).stop 2024-03-21T10:26:49.715 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.714+0000 7fa27cc00640 1 -- 172.21.15.47:0/1052143508 shutdown_connections 2024-03-21T10:26:49.715 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.714+0000 7fa27cc00640 1 --2- 172.21.15.47:0/1052143508 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa27810baf0 0x7fa27810df90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:26:49.715 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.714+0000 7fa27cc00640 1 --2- 172.21.15.47:0/1052143508 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa278102ff0 0x7fa27810b4d0 unknown :-1 s=CLOSED pgs=646 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:26:49.715 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.714+0000 7fa27cc00640 1 --2- 172.21.15.47:0/1052143508 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa2781026d0 0x7fa278102ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:26:49.715 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.714+0000 7fa27cc00640 1 -- 172.21.15.47:0/1052143508 >> 172.21.15.47:0/1052143508 conn(0x7fa2780fbef0 msgr2=0x7fa2780fe310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:26:49.715 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.714+0000 7fa27cc00640 1 -- 172.21.15.47:0/1052143508 shutdown_connections 2024-03-21T10:26:49.715 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.714+0000 7fa27cc00640 1 -- 172.21.15.47:0/1052143508 wait complete. 2024-03-21T10:26:49.716 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.716+0000 7fa27cc00640 1 Processor -- start 2024-03-21T10:26:49.716 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.716+0000 7fa27cc00640 1 -- start start 2024-03-21T10:26:49.717 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.716+0000 7fa27cc00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa2781026d0 0x7fa278199f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:26:49.717 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.716+0000 7fa27cc00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa278102ff0 0x7fa27819a490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:26:49.717 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.716+0000 7fa27cc00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa27810baf0 0x7fa27819e810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:26:49.717 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.716+0000 7fa27cc00640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2781117d0 con 0x7fa2781026d0 2024-03-21T10:26:49.718 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.716+0000 7fa27cc00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa278111650 con 0x7fa27810baf0 2024-03-21T10:26:49.718 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.716+0000 7fa27cc00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa278111950 con 0x7fa278102ff0 2024-03-21T10:26:49.718 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.717+0000 7fa277400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa2781026d0 0x7fa278199f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:26:49.718 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.717+0000 7fa277400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa2781026d0 0x7fa278199f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:58124/0 (socket says 172.21.15.47:58124) 2024-03-21T10:26:49.718 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.717+0000 7fa277400640 1 -- 172.21.15.47:0/3965418487 learned_addr learned my addr 172.21.15.47:0/3965418487 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:26:49.718 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.717+0000 7fa277e00640 1 --2- 172.21.15.47:0/3965418487 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa27810baf0 0x7fa27819e810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:26:49.718 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.717+0000 7fa276a00640 1 --2- 172.21.15.47:0/3965418487 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa278102ff0 0x7fa27819a490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:26:49.718 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.717+0000 7fa277400640 1 -- 172.21.15.47:0/3965418487 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa278102ff0 msgr2=0x7fa27819a490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:26:49.718 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.717+0000 7fa277400640 1 --2- 172.21.15.47:0/3965418487 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa278102ff0 0x7fa27819a490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:26:49.718 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.717+0000 7fa277400640 1 -- 172.21.15.47:0/3965418487 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa27810baf0 msgr2=0x7fa27819e810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:26:49.718 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.717+0000 7fa277400640 1 --2- 172.21.15.47:0/3965418487 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa27810baf0 0x7fa27819e810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:26:49.718 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.717+0000 7fa277400640 1 -- 172.21.15.47:0/3965418487 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa27819ef40 con 0x7fa2781026d0 2024-03-21T10:26:49.718 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.717+0000 7fa277e00640 1 --2- 172.21.15.47:0/3965418487 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa27810baf0 0x7fa27819e810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T10:26:49.719 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.717+0000 7fa277400640 1 --2- 172.21.15.47:0/3965418487 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa2781026d0 0x7fa278199f50 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7fa26800d770 tx=0x7fa26800dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:26:49.719 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.718+0000 7fa26fe00640 1 -- 172.21.15.47:0/3965418487 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa268017070 con 0x7fa2781026d0 2024-03-21T10:26:49.719 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.718+0000 7fa26fe00640 1 -- 172.21.15.47:0/3965418487 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fa2680054d0 con 0x7fa2781026d0 2024-03-21T10:26:49.719 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.718+0000 7fa26fe00640 1 -- 172.21.15.47:0/3965418487 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa268004cf0 con 0x7fa2781026d0 2024-03-21T10:26:49.719 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.718+0000 7fa27cc00640 1 -- 172.21.15.47:0/3965418487 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa27819f230 con 0x7fa2781026d0 2024-03-21T10:26:49.719 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.718+0000 7fa27cc00640 1 -- 172.21.15.47:0/3965418487 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa2781a6ac0 con 0x7fa2781026d0 2024-03-21T10:26:49.721 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.721+0000 7fa280e55740 1 Processor -- start 2024-03-21T10:26:49.721 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.721+0000 7fa280e55740 1 -- start start 2024-03-21T10:26:49.722 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.721+0000 7fa280e55740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562725e7a8b0 0x562725e7ac90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:26:49.722 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.721+0000 7fa280e55740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562725e7b260 0x562725e83810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:26:49.722 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.721+0000 7fa280e55740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562725e83de0 0x562725e86230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:26:49.722 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.721+0000 7fa280e55740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x562725dde2c0 con 0x562725e7b260 2024-03-21T10:26:49.722 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.721+0000 7fa280e55740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x562725d86c90 con 0x562725e7a8b0 2024-03-21T10:26:49.723 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.721+0000 7fa280e55740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x562725dda360 con 0x562725e83de0 2024-03-21T10:26:49.723 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.721+0000 7fa26fe00640 1 -- 172.21.15.47:0/3965418487 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 26) v1 ==== 138087+0+0 (secure 0 0 0) 0x7fa268004e90 con 0x7fa2781026d0 2024-03-21T10:26:49.723 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.722+0000 7fa277e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562725e7a8b0 0x562725e7ac90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:26:49.723 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.722+0000 7fa277400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562725e7b260 0x562725e83810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:26:49.723 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.722+0000 7fa277e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562725e7a8b0 0x562725e7ac90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:52424/0 (socket says 172.21.15.47:52424) 2024-03-21T10:26:49.723 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.722+0000 7fa277e00640 1 -- 172.21.15.47:0/457140652 learned_addr learned my addr 172.21.15.47:0/457140652 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:26:49.723 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.722+0000 7fa276a00640 1 --2- 172.21.15.47:0/457140652 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562725e83de0 0x562725e86230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:26:49.723 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.722+0000 7fa277e00640 1 -- 172.21.15.47:0/457140652 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562725e83de0 msgr2=0x562725e86230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:26:49.723 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.722+0000 7fa277e00640 1 --2- 172.21.15.47:0/457140652 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562725e83de0 0x562725e86230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:26:49.723 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.722+0000 7fa277e00640 1 -- 172.21.15.47:0/457140652 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562725e7b260 msgr2=0x562725e83810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:26:49.723 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.722+0000 7fa277e00640 1 --2- 172.21.15.47:0/457140652 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562725e7b260 0x562725e83810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:26:49.723 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.722+0000 7fa277e00640 1 -- 172.21.15.47:0/457140652 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562725dab080 con 0x562725e7a8b0 2024-03-21T10:26:49.726 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.723+0000 7fa277e00640 1 --2- 172.21.15.47:0/457140652 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562725e7a8b0 0x562725e7ac90 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fa270009fd0 tx=0x7fa270010860 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6db1823a0af50b87 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:26:49.727 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.723+0000 7fa263e00640 1 -- 172.21.15.47:0/457140652 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa270011700 con 0x562725e7a8b0 2024-03-21T10:26:49.727 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.723+0000 7fa263e00640 1 -- 172.21.15.47:0/457140652 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fa270011d50 con 0x562725e7a8b0 2024-03-21T10:26:49.727 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.723+0000 7fa263e00640 1 -- 172.21.15.47:0/457140652 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa27000e3c0 con 0x562725e7a8b0 2024-03-21T10:26:49.727 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.723+0000 7fa26fe00640 1 --2- 172.21.15.47:0/3965418487 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7fa25c0a3230 0x7fa25c0a56f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:26:49.727 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.723+0000 7fa26fe00640 1 -- 172.21.15.47:0/3965418487 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 9474+0+0 (secure 0 0 0) 0x7fa2680cf950 con 0x7fa2781026d0 2024-03-21T10:26:49.727 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.724+0000 7fa280e55740 1 -- 172.21.15.47:0/457140652 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562725e7a8b0 msgr2=0x562725e7ac90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:26:49.727 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.724+0000 7fa280e55740 1 --2- 172.21.15.47:0/457140652 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562725e7a8b0 0x562725e7ac90 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fa270009fd0 tx=0x7fa270010860 comp rx=0 tx=0).stop 2024-03-21T10:26:49.727 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.724+0000 7fa277400640 1 --2- 172.21.15.47:0/3965418487 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7fa25c0a3230 0x7fa25c0a56f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:26:49.727 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.724+0000 7fa280e55740 1 -- 172.21.15.47:0/457140652 shutdown_connections 2024-03-21T10:26:49.727 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.724+0000 7fa280e55740 1 --2- 172.21.15.47:0/457140652 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562725e83de0 0x562725e86230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:26:49.727 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.724+0000 7fa280e55740 1 --2- 172.21.15.47:0/457140652 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562725e7b260 0x562725e83810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:26:49.727 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.724+0000 7fa280e55740 1 --2- 172.21.15.47:0/457140652 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562725e7a8b0 0x562725e7ac90 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:26:49.727 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.724+0000 7fa280e55740 1 -- 172.21.15.47:0/457140652 >> 172.21.15.47:0/457140652 conn(0x562725e74f80 msgr2=0x562725e75360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:26:49.728 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.724+0000 7fa280e55740 1 -- 172.21.15.47:0/457140652 shutdown_connections 2024-03-21T10:26:49.728 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.724+0000 7fa280e55740 1 -- 172.21.15.47:0/457140652 wait complete. 2024-03-21T10:26:49.728 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.725+0000 7fa280e55740 1 -- start start 2024-03-21T10:26:49.728 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.725+0000 7fa277400640 1 --2- 172.21.15.47:0/3965418487 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7fa25c0a3230 0x7fa25c0a56f0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fa2680ccb60 tx=0x7fa2680d5530 comp rx=0 tx=0).ready entity=mgr.15687 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:26:49.728 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.725+0000 7fa280e55740 1 Processor -- start 2024-03-21T10:26:49.728 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.725+0000 7fa280e55740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562725e7a8b0 0x562725e83750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:26:49.728 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.725+0000 7fa280e55740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562725e7b260 0x562725e80ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:26:49.728 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.725+0000 7fa280e55740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562725e83de0 0x562725e810f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:26:49.728 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.725+0000 7fa280e55740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x562725e86d50 con 0x562725e83de0 2024-03-21T10:26:49.728 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.725+0000 7fa280e55740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x562725d9a0d0 con 0x562725e7b260 2024-03-21T10:26:49.728 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.725+0000 7fa280e55740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x562725e86ed0 con 0x562725e7a8b0 2024-03-21T10:26:49.728 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.725+0000 7fa280e55740 10 client.0 _ll_register_callbacks cb 0x562725d9ee80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T10:26:49.728 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.726+0000 7fa277e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562725e83de0 0x562725e810f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:26:49.728 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.726+0000 7fa277e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562725e83de0 0x562725e810f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:58130/0 (socket says 172.21.15.47:58130) 2024-03-21T10:26:49.728 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.726+0000 7fa277e00640 1 -- 172.21.15.47:0/1857317711 learned_addr learned my addr 172.21.15.47:0/1857317711 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:26:49.729 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.726+0000 7fa276a00640 1 --2- 172.21.15.47:0/1857317711 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562725e7b260 0x562725e80ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:26:49.729 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.726+0000 7fa277e00640 1 --2- 172.21.15.47:0/1857317711 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562725e7a8b0 0x562725e83750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:26:49.729 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.726+0000 7fa277e00640 1 -- 172.21.15.47:0/1857317711 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562725e7a8b0 msgr2=0x562725e83750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:26:49.729 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.726+0000 7fa277e00640 1 --2- 172.21.15.47:0/1857317711 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562725e7a8b0 0x562725e83750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:26:49.729 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.726+0000 7fa277e00640 1 -- 172.21.15.47:0/1857317711 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562725e7b260 msgr2=0x562725e80ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:26:49.729 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.726+0000 7fa277e00640 1 --2- 172.21.15.47:0/1857317711 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562725e7b260 0x562725e80ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:26:49.729 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.726+0000 7fa277e00640 1 -- 172.21.15.47:0/1857317711 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x562725af23f0 con 0x562725e83de0 2024-03-21T10:26:49.729 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.726+0000 7fa276a00640 1 --2- 172.21.15.47:0/1857317711 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562725e7b260 0x562725e80ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T10:26:49.729 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.726+0000 7fa277e00640 1 --2- 172.21.15.47:0/1857317711 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562725e83de0 0x562725e810f0 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7fa270010d60 tx=0x7fa270013040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5f04f88e82a22cc1 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:26:49.729 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.726+0000 7fa260c00640 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-03-21T10:26:49.729 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.727+0000 7fa260c00640 1 -- 172.21.15.47:0/1857317711 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa27002c060 con 0x562725e83de0 2024-03-21T10:26:49.729 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.727+0000 7fa280e55740 1 -- 172.21.15.47:0/1857317711 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fa270011d50 con 0x562725e83de0 2024-03-21T10:26:49.729 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.727+0000 7fa260c00640 1 -- 172.21.15.47:0/1857317711 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(97..97 src has 1..97) v4 ==== 9474+0+0 (secure 0 0 0) 0x7fa27001b640 con 0x562725e83de0 2024-03-21T10:26:49.732 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.727+0000 7fa260c00640 1 -- 172.21.15.47:0/1857317711 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa270033050 con 0x562725e83de0 2024-03-21T10:26:49.732 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.727+0000 7fa280e55740 10 client.16263 fetch_fsmap learned FSMap version 32 2024-03-21T10:26:49.732 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.727+0000 7fa280e55740 1 -- 172.21.15.47:0/1857317711 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x562725bbc7d0 con 0x562725e83de0 2024-03-21T10:26:49.732 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.727+0000 7fa260c00640 1 -- 172.21.15.47:0/1857317711 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fa270011d50 con 0x562725e83de0 2024-03-21T10:26:49.732 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.727+0000 7fa260c00640 1 -- 172.21.15.47:0/1857317711 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 32) v1 ==== 7942+0+0 (secure 0 0 0) 0x7fa270026070 con 0x562725e83de0 2024-03-21T10:26:49.732 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.728+0000 7fa280e55740 10 client.16263 fetch_fsmap finished waiting for FSMap version 32 2024-03-21T10:26:49.732 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.728+0000 7fa280e55740 10 client.16263 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-03-21T10:26:49.732 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.728+0000 7fa280e55740 20 client.16263 populate_metadata read hostname 'smithi047' 2024-03-21T10:26:49.732 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.728+0000 7fa280e55740 1 --2- 172.21.15.47:0/1857317711 >> [v2:172.21.15.100:6836/3497644162,v1:172.21.15.100:6837/3497644162] conn(0x562725e93b30 0x562725e95f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:26:49.732 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.728+0000 7fa280e55740 4 client.16263 mds_command: new command op to 34493 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T10:26:49.732 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.728+0000 7fa280e55740 1 -- 172.21.15.47:0/1857317711 --> [v2:172.21.15.100:6836/3497644162,v1:172.21.15.100:6837/3497644162] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x562725deb800 con 0x562725e93b30 2024-03-21T10:26:49.732 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.728+0000 7fa276a00640 1 --2- 172.21.15.47:0/1857317711 >> [v2:172.21.15.100:6836/3497644162,v1:172.21.15.100:6837/3497644162] conn(0x562725e93b30 0x562725e95f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:26:49.732 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.729+0000 7fa276a00640 1 --2- 172.21.15.47:0/1857317711 >> [v2:172.21.15.100:6836/3497644162,v1:172.21.15.100:6837/3497644162] conn(0x562725e93b30 0x562725e95f20 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=bb268968d1774083 server_cookie=b524ff264b480dd in_seq=0 out_seq=0 2024-03-21T10:26:49.733 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.729+0000 7fa260c00640 10 client.16263 ms_handle_connect on v2:172.21.15.100:6836/3497644162 2024-03-21T10:26:49.733 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.730+0000 7fa260c00640 1 -- 172.21.15.47:0/1857317711 <== mds.1 v2:172.21.15.100:6836/3497644162 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x562725deb800 con 0x562725e93b30 2024-03-21T10:26:49.733 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.730+0000 7fa260c00640 10 client.16263 handle_command_reply: tid=0 2024-03-21T10:26:49.733 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.730+0000 7fa280e55740 1 client.16263 shutdown 2024-03-21T10:26:49.733 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.730+0000 7fa280e55740 1 -- 172.21.15.47:0/1857317711 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562725e83de0 msgr2=0x562725e810f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:26:49.733 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.730+0000 7fa280e55740 1 --2- 172.21.15.47:0/1857317711 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562725e83de0 0x562725e810f0 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7fa270010d60 tx=0x7fa270013040 comp rx=0 tx=0).stop 2024-03-21T10:26:49.733 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.730+0000 7fa280e55740 1 -- 172.21.15.47:0/1857317711 shutdown_connections 2024-03-21T10:26:49.733 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.730+0000 7fa280e55740 1 --2- 172.21.15.47:0/1857317711 >> [v2:172.21.15.100:6836/3497644162,v1:172.21.15.100:6837/3497644162] conn(0x562725e93b30 0x562725e95f20 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:26:49.733 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.730+0000 7fa280e55740 1 --2- 172.21.15.47:0/1857317711 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562725e83de0 0x562725e810f0 unknown :-1 s=CLOSED pgs=648 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:26:49.733 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.730+0000 7fa280e55740 1 --2- 172.21.15.47:0/1857317711 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562725e7b260 0x562725e80ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:26:49.733 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.730+0000 7fa280e55740 1 --2- 172.21.15.47:0/1857317711 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562725e7a8b0 0x562725e83750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:26:49.733 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.730+0000 7fa280e55740 1 -- 172.21.15.47:0/1857317711 >> 172.21.15.47:0/1857317711 conn(0x562725e74f80 msgr2=0x562725e76cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:26:49.733 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.730+0000 7fa260c00640 0 client.16263 ms_handle_reset on v2:172.21.15.100:6836/3497644162 2024-03-21T10:26:49.733 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.731+0000 7fa280e55740 1 -- 172.21.15.47:0/1857317711 shutdown_connections 2024-03-21T10:26:49.733 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.731+0000 7fa280e55740 1 -- 172.21.15.47:0/1857317711 wait complete. 2024-03-21T10:26:49.734 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.731+0000 7fa280e55740 20 client.16263 trim_cache size 0 max 16384 2024-03-21T10:26:49.738 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:26:49 smithi157 ceph-mon[35581]: pgmap v48: 129 pgs: 129 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 9 op/s 2024-03-21T10:26:49.739 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:26:49 smithi157 ceph-mon[35581]: from='client.? 172.21.15.47:0/1919726689' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-21T10:26:49.744 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.744+0000 7fa280e55740 1 Processor -- start 2024-03-21T10:26:49.744 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.744+0000 7fa280e55740 1 -- start start 2024-03-21T10:26:49.744 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.744+0000 7fa280e55740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562725e7a8b0 0x562725e803c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:26:49.744 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.744+0000 7fa280e55740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562725e7b260 0x562725e738d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:26:49.745 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.744+0000 7fa280e55740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562725e83de0 0x562725e83210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:26:49.745 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.744+0000 7fa280e55740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2781b09c0 con 0x562725e7a8b0 2024-03-21T10:26:49.745 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.744+0000 7fa280e55740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2781b2340 con 0x562725e7b260 2024-03-21T10:26:49.745 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.744+0000 7fa280e55740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2781b1020 con 0x562725e83de0 2024-03-21T10:26:49.746 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.745+0000 7fa277e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562725e7a8b0 0x562725e803c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:26:49.746 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.745+0000 7fa277e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562725e7a8b0 0x562725e803c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:58142/0 (socket says 172.21.15.47:58142) 2024-03-21T10:26:49.746 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.745+0000 7fa277e00640 1 -- 172.21.15.47:0/2372146466 learned_addr learned my addr 172.21.15.47:0/2372146466 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:26:49.746 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.745+0000 7fa276a00640 1 --2- 172.21.15.47:0/2372146466 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562725e7b260 0x562725e738d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:26:49.746 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.745+0000 7fa277e00640 1 -- 172.21.15.47:0/2372146466 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562725e83de0 msgr2=0x562725e83210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:26:49.746 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.745+0000 7fa277e00640 1 --2- 172.21.15.47:0/2372146466 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562725e83de0 0x562725e83210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:26:49.746 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.745+0000 7fa277e00640 1 -- 172.21.15.47:0/2372146466 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562725e7b260 msgr2=0x562725e738d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:26:49.747 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.745+0000 7fa277e00640 1 --2- 172.21.15.47:0/2372146466 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562725e7b260 0x562725e738d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:26:49.747 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.745+0000 7fa277e00640 1 -- 172.21.15.47:0/2372146466 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562725e86690 con 0x562725e7a8b0 2024-03-21T10:26:49.747 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.746+0000 7fa277e00640 1 --2- 172.21.15.47:0/2372146466 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562725e7a8b0 0x562725e803c0 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7fa270006010 tx=0x7fa270009e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d86f9129278ba0ff server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:26:49.747 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.746+0000 7fa263400640 1 -- 172.21.15.47:0/2372146466 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa27002c040 con 0x562725e7a8b0 2024-03-21T10:26:49.747 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.746+0000 7fa263400640 1 -- 172.21.15.47:0/2372146466 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fa270011d20 con 0x562725e7a8b0 2024-03-21T10:26:49.747 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.746+0000 7fa263400640 1 -- 172.21.15.47:0/2372146466 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa27001b8a0 con 0x562725e7a8b0 2024-03-21T10:26:49.750 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.747+0000 7fa280e55740 1 -- 172.21.15.47:0/2372146466 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562725e7a8b0 msgr2=0x562725e803c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:26:49.750 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.747+0000 7fa280e55740 1 --2- 172.21.15.47:0/2372146466 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562725e7a8b0 0x562725e803c0 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7fa270006010 tx=0x7fa270009e70 comp rx=0 tx=0).stop 2024-03-21T10:26:49.750 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.747+0000 7fa280e55740 1 -- 172.21.15.47:0/2372146466 shutdown_connections 2024-03-21T10:26:49.751 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.747+0000 7fa280e55740 1 --2- 172.21.15.47:0/2372146466 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562725e83de0 0x562725e83210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:26:49.751 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.747+0000 7fa280e55740 1 --2- 172.21.15.47:0/2372146466 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562725e7b260 0x562725e738d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:26:49.751 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.747+0000 7fa280e55740 1 --2- 172.21.15.47:0/2372146466 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562725e7a8b0 0x562725e803c0 unknown :-1 s=CLOSED pgs=649 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:26:49.751 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.747+0000 7fa280e55740 1 -- 172.21.15.47:0/2372146466 >> 172.21.15.47:0/2372146466 conn(0x562725e74f80 msgr2=0x562725e76bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:26:49.751 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.747+0000 7fa280e55740 1 -- 172.21.15.47:0/2372146466 shutdown_connections 2024-03-21T10:26:49.751 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.747+0000 7fa280e55740 1 -- 172.21.15.47:0/2372146466 wait complete. 2024-03-21T10:26:49.751 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.747+0000 7fa280e55740 1 -- start start 2024-03-21T10:26:49.751 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.747+0000 7fa280e55740 1 Processor -- start 2024-03-21T10:26:49.751 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.748+0000 7fa280e55740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562725e7a8b0 0x562725e89610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:26:49.751 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.748+0000 7fa280e55740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562725e7b260 0x562725e89b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:26:49.751 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.748+0000 7fa280e55740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562725e83de0 0x562725e8a090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:26:49.751 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.748+0000 7fa280e55740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2781b1ce0 con 0x562725e7b260 2024-03-21T10:26:49.751 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.748+0000 7fa280e55740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2781b1680 con 0x562725e83de0 2024-03-21T10:26:49.752 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.748+0000 7fa280e55740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2781aeb80 con 0x562725e7a8b0 2024-03-21T10:26:49.752 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.748+0000 7fa280e55740 10 client.0 _ll_register_callbacks cb 0x7fa2680d5400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T10:26:49.752 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.748+0000 7fa276a00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562725e7b260 0x562725e89b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:26:49.752 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.748+0000 7fa276a00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562725e7b260 0x562725e89b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:58148/0 (socket says 172.21.15.47:58148) 2024-03-21T10:26:49.752 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.748+0000 7fa276a00640 1 -- 172.21.15.47:0/3154489124 learned_addr learned my addr 172.21.15.47:0/3154489124 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:26:49.752 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.748+0000 7fa277e00640 1 --2- 172.21.15.47:0/3154489124 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562725e83de0 0x562725e8a090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:26:49.752 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.748+0000 7fa277e00640 1 --2- 172.21.15.47:0/3154489124 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562725e7a8b0 0x562725e89610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:26:49.752 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.748+0000 7fa276a00640 1 -- 172.21.15.47:0/3154489124 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562725e7a8b0 msgr2=0x562725e89610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:26:49.752 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.748+0000 7fa276a00640 1 --2- 172.21.15.47:0/3154489124 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562725e7a8b0 0x562725e89610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:26:49.752 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.748+0000 7fa276a00640 1 -- 172.21.15.47:0/3154489124 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562725e83de0 msgr2=0x562725e8a090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:26:49.752 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.749+0000 7fa276a00640 1 --2- 172.21.15.47:0/3154489124 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562725e83de0 0x562725e8a090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:26:49.752 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.749+0000 7fa276a00640 1 -- 172.21.15.47:0/3154489124 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x562725eee220 con 0x562725e7b260 2024-03-21T10:26:49.752 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.749+0000 7fa277e00640 1 --2- 172.21.15.47:0/3154489124 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562725e83de0 0x562725e8a090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T10:26:49.752 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.749+0000 7fa276a00640 1 --2- 172.21.15.47:0/3154489124 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562725e7b260 0x562725e89b50 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7fa2780fe330 tx=0x7fa2640043a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=884cbb5c2a1093f2 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:26:49.753 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.749+0000 7fa260c00640 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-03-21T10:26:49.753 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.749+0000 7fa260c00640 1 -- 172.21.15.47:0/3154489124 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2640432d0 con 0x562725e7b260 2024-03-21T10:26:49.753 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.749+0000 7fa260c00640 1 -- 172.21.15.47:0/3154489124 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(97..97 src has 1..97) v4 ==== 9474+0+0 (secure 0 0 0) 0x7fa26402c880 con 0x562725e7b260 2024-03-21T10:26:49.753 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.749+0000 7fa280e55740 1 -- 172.21.15.47:0/3154489124 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fa270011d20 con 0x562725e7b260 2024-03-21T10:26:49.753 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.749+0000 7fa260c00640 1 -- 172.21.15.47:0/3154489124 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa264053050 con 0x562725e7b260 2024-03-21T10:26:49.753 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.750+0000 7fa280e55740 10 client.16269 fetch_fsmap learned FSMap version 32 2024-03-21T10:26:49.753 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.750+0000 7fa280e55740 1 -- 172.21.15.47:0/3154489124 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x562725eef5b0 con 0x562725e7b260 2024-03-21T10:26:49.753 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.750+0000 7fa260c00640 1 -- 172.21.15.47:0/3154489124 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fa270011d20 con 0x562725e7b260 2024-03-21T10:26:49.753 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.750+0000 7fa260c00640 1 -- 172.21.15.47:0/3154489124 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 32) v1 ==== 7942+0+0 (secure 0 0 0) 0x7fa264005040 con 0x562725e7b260 2024-03-21T10:26:49.753 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.750+0000 7fa280e55740 10 client.16269 fetch_fsmap finished waiting for FSMap version 32 2024-03-21T10:26:49.753 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.750+0000 7fa280e55740 10 client.16269 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-03-21T10:26:49.753 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.750+0000 7fa280e55740 20 client.16269 populate_metadata read hostname 'smithi047' 2024-03-21T10:26:49.753 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.750+0000 7fa280e55740 1 --2- 172.21.15.47:0/3154489124 >> [v2:172.21.15.100:6836/3497644162,v1:172.21.15.100:6837/3497644162] conn(0x562725eefcc0 0x562725ef20e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:26:49.753 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.750+0000 7fa280e55740 4 client.16269 mds_command: new command op to 34493 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-21T10:26:49.754 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.750+0000 7fa280e55740 1 -- 172.21.15.47:0/3154489124 --> [v2:172.21.15.100:6836/3497644162,v1:172.21.15.100:6837/3497644162] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x562725de5b50 con 0x562725eefcc0 2024-03-21T10:26:49.754 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.751+0000 7fa277e00640 1 --2- 172.21.15.47:0/3154489124 >> [v2:172.21.15.100:6836/3497644162,v1:172.21.15.100:6837/3497644162] conn(0x562725eefcc0 0x562725ef20e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:26:49.755 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.751+0000 7fa277e00640 1 --2- 172.21.15.47:0/3154489124 >> [v2:172.21.15.100:6836/3497644162,v1:172.21.15.100:6837/3497644162] conn(0x562725eefcc0 0x562725ef20e0 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=54604415a6ac4d9 server_cookie=75c003e5cb819383 in_seq=0 out_seq=0 2024-03-21T10:26:49.755 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.751+0000 7fa260c00640 10 client.16269 ms_handle_connect on v2:172.21.15.100:6836/3497644162 2024-03-21T10:26:49.755 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.751+0000 7fa260c00640 1 -- 172.21.15.47:0/3154489124 <== mds.1 v2:172.21.15.100:6836/3497644162 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x562725de5b50 con 0x562725eefcc0 2024-03-21T10:26:49.755 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.751+0000 7fa260c00640 10 client.16269 handle_command_reply: tid=0 2024-03-21T10:26:49.755 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.751+0000 7fa280e55740 1 client.16269 shutdown 2024-03-21T10:26:49.755 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.752+0000 7fa280e55740 1 -- 172.21.15.47:0/3154489124 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562725e7b260 msgr2=0x562725e89b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:26:49.755 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.752+0000 7fa280e55740 1 --2- 172.21.15.47:0/3154489124 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562725e7b260 0x562725e89b50 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7fa2780fe330 tx=0x7fa2640043a0 comp rx=0 tx=0).stop 2024-03-21T10:26:49.755 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.752+0000 7fa280e55740 1 -- 172.21.15.47:0/3154489124 shutdown_connections 2024-03-21T10:26:49.756 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.752+0000 7fa280e55740 1 --2- 172.21.15.47:0/3154489124 >> [v2:172.21.15.100:6836/3497644162,v1:172.21.15.100:6837/3497644162] conn(0x562725eefcc0 0x562725ef20e0 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:26:49.756 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.752+0000 7fa280e55740 1 --2- 172.21.15.47:0/3154489124 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562725e83de0 0x562725e8a090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:26:49.756 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.752+0000 7fa280e55740 1 --2- 172.21.15.47:0/3154489124 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562725e7b260 0x562725e89b50 unknown :-1 s=CLOSED pgs=650 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:26:49.756 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.752+0000 7fa280e55740 1 --2- 172.21.15.47:0/3154489124 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562725e7a8b0 0x562725e89610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:26:49.756 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.752+0000 7fa280e55740 1 -- 172.21.15.47:0/3154489124 >> 172.21.15.47:0/3154489124 conn(0x562725e74f80 msgr2=0x562725e7b640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:26:49.756 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.752+0000 7fa260c00640 0 client.16269 ms_handle_reset on v2:172.21.15.100:6836/3497644162 2024-03-21T10:26:49.756 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.752+0000 7fa280e55740 1 -- 172.21.15.47:0/3154489124 shutdown_connections 2024-03-21T10:26:49.756 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.752+0000 7fa280e55740 1 -- 172.21.15.47:0/3154489124 wait complete. 2024-03-21T10:26:49.756 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.752+0000 7fa280e55740 20 client.16269 trim_cache size 0 max 16384 2024-03-21T10:26:49.756 INFO:teuthology.orchestra.run.smithi047.stdout:[] 2024-03-21T10:26:49.757 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.753+0000 7fa27cc00640 1 -- 172.21.15.47:0/3965418487 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7fa25c0a3230 msgr2=0x7fa25c0a56f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:26:49.757 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.753+0000 7fa27cc00640 1 --2- 172.21.15.47:0/3965418487 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7fa25c0a3230 0x7fa25c0a56f0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fa2680ccb60 tx=0x7fa2680d5530 comp rx=0 tx=0).stop 2024-03-21T10:26:49.757 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.753+0000 7fa27cc00640 1 -- 172.21.15.47:0/3965418487 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa2781026d0 msgr2=0x7fa278199f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:26:49.757 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.753+0000 7fa27cc00640 1 --2- 172.21.15.47:0/3965418487 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa2781026d0 0x7fa278199f50 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7fa26800d770 tx=0x7fa26800dc30 comp rx=0 tx=0).stop 2024-03-21T10:26:49.757 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.753+0000 7fa27cc00640 1 -- 172.21.15.47:0/3965418487 shutdown_connections 2024-03-21T10:26:49.757 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.753+0000 7fa27cc00640 1 --2- 172.21.15.47:0/3965418487 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7fa25c0a3230 0x7fa25c0a56f0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:26:49.757 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.753+0000 7fa27cc00640 1 --2- 172.21.15.47:0/3965418487 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa27810baf0 0x7fa27819e810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:26:49.758 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.753+0000 7fa27cc00640 1 --2- 172.21.15.47:0/3965418487 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa278102ff0 0x7fa27819a490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:26:49.758 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.753+0000 7fa27cc00640 1 --2- 172.21.15.47:0/3965418487 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa2781026d0 0x7fa278199f50 unknown :-1 s=CLOSED pgs=647 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:26:49.758 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.753+0000 7fa27cc00640 1 -- 172.21.15.47:0/3965418487 >> 172.21.15.47:0/3965418487 conn(0x7fa2780fbef0 msgr2=0x7fa27810cd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:26:49.758 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.753+0000 7fa27cc00640 1 -- 172.21.15.47:0/3965418487 shutdown_connections 2024-03-21T10:26:49.758 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.753+0000 7fa27cc00640 1 -- 172.21.15.47:0/3965418487 wait complete. 2024-03-21T10:26:49.770 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-21T10:26:49.770 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-03-21T10:26:49.972 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.972+0000 7f0223200640 1 -- 172.21.15.47:0/3481078638 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0224101ef0 msgr2=0x7f022410e3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:26:49.973 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.972+0000 7f0223200640 1 --2- 172.21.15.47:0/3481078638 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0224101ef0 0x7f022410e3b0 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7f021c009ca0 tx=0x7f021c02c0f0 comp rx=0 tx=0).stop 2024-03-21T10:26:49.973 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.972+0000 7f0223200640 1 -- 172.21.15.47:0/3481078638 shutdown_connections 2024-03-21T10:26:49.973 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.972+0000 7f0223200640 1 --2- 172.21.15.47:0/3481078638 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0224101ef0 0x7f022410e3b0 unknown :-1 s=CLOSED pgs=651 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:26:49.973 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.972+0000 7f0223200640 1 --2- 172.21.15.47:0/3481078638 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0224101530 0x7f02241019b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:26:49.973 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.972+0000 7f0223200640 1 --2- 172.21.15.47:0/3481078638 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f02241075f0 0x7f02241079d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:26:49.973 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.972+0000 7f0223200640 1 -- 172.21.15.47:0/3481078638 >> 172.21.15.47:0/3481078638 conn(0x7f02240fd040 msgr2=0x7f02240ff460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:26:49.973 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.972+0000 7f0223200640 1 -- 172.21.15.47:0/3481078638 shutdown_connections 2024-03-21T10:26:49.973 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.972+0000 7f0223200640 1 -- 172.21.15.47:0/3481078638 wait complete. 2024-03-21T10:26:49.975 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.973+0000 7f0223200640 1 Processor -- start 2024-03-21T10:26:49.976 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.974+0000 7f0223200640 1 -- start start 2024-03-21T10:26:49.976 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.974+0000 7f0223200640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0224101530 0x7f02240739a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:26:49.976 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.974+0000 7f0223200640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0224101ef0 0x7f0224073ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:26:49.976 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.974+0000 7f0223200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f02241075f0 0x7f0224074420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:26:49.976 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.974+0000 7f0223200640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0224111760 con 0x7f0224101530 2024-03-21T10:26:49.976 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.974+0000 7f0223200640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02241115e0 con 0x7f02241075f0 2024-03-21T10:26:49.976 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.974+0000 7f0223200640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02241118e0 con 0x7f0224101ef0 2024-03-21T10:26:49.976 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.974+0000 7f0221e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0224101530 0x7f02240739a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:26:49.976 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.974+0000 7f0221e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0224101530 0x7f02240739a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:58164/0 (socket says 172.21.15.47:58164) 2024-03-21T10:26:49.976 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.974+0000 7f0221e00640 1 -- 172.21.15.47:0/1782016614 learned_addr learned my addr 172.21.15.47:0/1782016614 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:26:49.976 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.975+0000 7f0222800640 1 --2- 172.21.15.47:0/1782016614 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f02241075f0 0x7f0224074420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:26:49.976 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.975+0000 7f0221400640 1 --2- 172.21.15.47:0/1782016614 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0224101ef0 0x7f0224073ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:26:49.977 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.975+0000 7f0221e00640 1 -- 172.21.15.47:0/1782016614 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0224101ef0 msgr2=0x7f0224073ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:26:49.977 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.975+0000 7f0221e00640 1 --2- 172.21.15.47:0/1782016614 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0224101ef0 0x7f0224073ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:26:49.977 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.975+0000 7f0221e00640 1 -- 172.21.15.47:0/1782016614 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f02241075f0 msgr2=0x7f0224074420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:26:49.977 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.975+0000 7f0221e00640 1 --2- 172.21.15.47:0/1782016614 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f02241075f0 0x7f0224074420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:26:49.977 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.975+0000 7f0221e00640 1 -- 172.21.15.47:0/1782016614 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f02241a6120 con 0x7f0224101530 2024-03-21T10:26:49.977 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.975+0000 7f0221400640 1 --2- 172.21.15.47:0/1782016614 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0224101ef0 0x7f0224073ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T10:26:49.977 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.975+0000 7f0221e00640 1 --2- 172.21.15.47:0/1782016614 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0224101530 0x7f02240739a0 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7f021400c890 tx=0x7f021400cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:26:49.977 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.975+0000 7f021aa00640 1 -- 172.21.15.47:0/1782016614 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0214004220 con 0x7f0224101530 2024-03-21T10:26:49.977 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.975+0000 7f0223200640 1 -- 172.21.15.47:0/1782016614 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f02241a6410 con 0x7f0224101530 2024-03-21T10:26:49.977 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.975+0000 7f021aa00640 1 -- 172.21.15.47:0/1782016614 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f02140043c0 con 0x7f0224101530 2024-03-21T10:26:49.977 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.976+0000 7f021aa00640 1 -- 172.21.15.47:0/1782016614 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0214004a00 con 0x7f0224101530 2024-03-21T10:26:49.977 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.976+0000 7f0223200640 1 -- 172.21.15.47:0/1782016614 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f02241a69b0 con 0x7f0224101530 2024-03-21T10:26:49.978 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.979+0000 7f022b5fd740 1 Processor -- start 2024-03-21T10:26:49.980 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.979+0000 7f022b5fd740 1 -- start start 2024-03-21T10:26:49.980 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.979+0000 7f022b5fd740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56330e6d18b0 0x56330e6d1c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:26:49.980 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.979+0000 7f022b5fd740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56330e6d2260 0x56330e6da810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:26:49.980 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.979+0000 7f022b5fd740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56330e6dade0 0x56330e6dd230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:26:49.980 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.979+0000 7f022b5fd740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x56330e6352c0 con 0x56330e6dade0 2024-03-21T10:26:49.980 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.979+0000 7f022b5fd740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x56330e5ddc90 con 0x56330e6d2260 2024-03-21T10:26:49.981 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.979+0000 7f022b5fd740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x56330e631360 con 0x56330e6d18b0 2024-03-21T10:26:49.981 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.979+0000 7f0221400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56330e6dade0 0x56330e6dd230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:26:49.981 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.980+0000 7f0221400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56330e6dade0 0x56330e6dd230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:58180/0 (socket says 172.21.15.47:58180) 2024-03-21T10:26:49.981 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.980+0000 7f0221400640 1 -- 172.21.15.47:0/1098418435 learned_addr learned my addr 172.21.15.47:0/1098418435 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:26:49.981 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.980+0000 7f0222800640 1 --2- 172.21.15.47:0/1098418435 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56330e6d18b0 0x56330e6d1c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:26:49.981 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.980+0000 7f0221400640 1 -- 172.21.15.47:0/1098418435 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56330e6d18b0 msgr2=0x56330e6d1c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:26:49.981 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.980+0000 7f0221400640 1 --2- 172.21.15.47:0/1098418435 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56330e6d18b0 0x56330e6d1c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:26:49.981 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.980+0000 7f0221400640 1 -- 172.21.15.47:0/1098418435 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56330e6d2260 msgr2=0x56330e6da810 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-21T10:26:49.981 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.980+0000 7f0221400640 1 --2- 172.21.15.47:0/1098418435 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56330e6d2260 0x56330e6da810 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:26:49.981 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.980+0000 7f0221400640 1 -- 172.21.15.47:0/1098418435 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56330e602080 con 0x56330e6dade0 2024-03-21T10:26:49.983 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.980+0000 7f0221400640 1 --2- 172.21.15.47:0/1098418435 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56330e6dade0 0x56330e6dd230 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7f0210009c20 tx=0x7f021000e6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=33b9d8520d6b360f server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:26:49.983 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.981+0000 7f020f000640 1 -- 172.21.15.47:0/1098418435 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0210016070 con 0x56330e6dade0 2024-03-21T10:26:49.983 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.981+0000 7f020f000640 1 -- 172.21.15.47:0/1098418435 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f0210011920 con 0x56330e6dade0 2024-03-21T10:26:49.983 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.981+0000 7f021aa00640 1 -- 172.21.15.47:0/1782016614 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 26) v1 ==== 138087+0+0 (secure 0 0 0) 0x7f0214004ba0 con 0x7f0224101530 2024-03-21T10:26:49.983 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.981+0000 7f020f000640 1 -- 172.21.15.47:0/1098418435 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02100107a0 con 0x56330e6dade0 2024-03-21T10:26:49.984 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.981+0000 7f022b5fd740 1 -- 172.21.15.47:0/1098418435 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56330e6dade0 msgr2=0x56330e6dd230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:26:49.984 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.981+0000 7f022b5fd740 1 --2- 172.21.15.47:0/1098418435 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56330e6dade0 0x56330e6dd230 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7f0210009c20 tx=0x7f021000e6c0 comp rx=0 tx=0).stop 2024-03-21T10:26:49.984 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.981+0000 7f022b5fd740 1 -- 172.21.15.47:0/1098418435 shutdown_connections 2024-03-21T10:26:49.984 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.981+0000 7f022b5fd740 1 --2- 172.21.15.47:0/1098418435 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56330e6dade0 0x56330e6dd230 unknown :-1 s=CLOSED pgs=653 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:26:49.984 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.981+0000 7f022b5fd740 1 --2- 172.21.15.47:0/1098418435 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56330e6d2260 0x56330e6da810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:26:49.984 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.981+0000 7f022b5fd740 1 --2- 172.21.15.47:0/1098418435 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56330e6d18b0 0x56330e6d1c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:26:49.984 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.981+0000 7f022b5fd740 1 -- 172.21.15.47:0/1098418435 >> 172.21.15.47:0/1098418435 conn(0x56330e6cbf80 msgr2=0x56330e6cc360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:26:49.984 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.982+0000 7f022b5fd740 1 -- 172.21.15.47:0/1098418435 shutdown_connections 2024-03-21T10:26:49.984 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.982+0000 7f022b5fd740 1 -- 172.21.15.47:0/1098418435 wait complete. 2024-03-21T10:26:49.984 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.982+0000 7f022b5fd740 1 -- start start 2024-03-21T10:26:49.984 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.983+0000 7f022b5fd740 1 Processor -- start 2024-03-21T10:26:49.984 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.983+0000 7f021aa00640 1 --2- 172.21.15.47:0/1782016614 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f02080a30c0 0x7f02080a5580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:26:49.984 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.983+0000 7f021aa00640 1 -- 172.21.15.47:0/1782016614 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 9474+0+0 (secure 0 0 0) 0x7f02140cfb40 con 0x7f0224101530 2024-03-21T10:26:49.984 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.983+0000 7f0222800640 1 --2- 172.21.15.47:0/1782016614 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f02080a30c0 0x7f02080a5580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:26:49.984 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.983+0000 7f022b5fd740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56330e6d18b0 0x56330e6da6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:26:49.985 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.983+0000 7f022b5fd740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56330e6d2260 0x56330e6d7a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:26:49.985 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.983+0000 7f022b5fd740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56330e6dade0 0x56330e6d7fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:26:49.985 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.983+0000 7f022b5fd740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x56330e6ddd50 con 0x56330e6d18b0 2024-03-21T10:26:49.985 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.983+0000 7f022b5fd740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x56330e5f10d0 con 0x56330e6d2260 2024-03-21T10:26:49.985 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.983+0000 7f022b5fd740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x56330e6dded0 con 0x56330e6dade0 2024-03-21T10:26:49.985 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.983+0000 7f022b5fd740 10 client.0 _ll_register_callbacks cb 0x56330e5f5e80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T10:26:49.987 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.984+0000 7f0221e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56330e6d18b0 0x56330e6da6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:26:49.987 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.984+0000 7f0221e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56330e6d18b0 0x56330e6da6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:58186/0 (socket says 172.21.15.47:58186) 2024-03-21T10:26:49.987 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.984+0000 7f0221e00640 1 -- 172.21.15.47:0/2020663883 learned_addr learned my addr 172.21.15.47:0/2020663883 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:26:49.987 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.984+0000 7f0221400640 1 --2- 172.21.15.47:0/2020663883 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56330e6d2260 0x56330e6d7a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:26:49.987 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.984+0000 7f0222800640 1 --2- 172.21.15.47:0/1782016614 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f02080a30c0 0x7f02080a5580 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f021c004af0 tx=0x7f021c02e040 comp rx=0 tx=0).ready entity=mgr.15687 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:26:49.987 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.984+0000 7f0221e00640 1 -- 172.21.15.47:0/2020663883 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56330e6dade0 msgr2=0x56330e6d7fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:26:49.987 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.984+0000 7f0221e00640 1 --2- 172.21.15.47:0/2020663883 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56330e6dade0 0x56330e6d7fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:26:49.987 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.984+0000 7f0221e00640 1 -- 172.21.15.47:0/2020663883 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56330e6d2260 msgr2=0x56330e6d7a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:26:49.988 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.984+0000 7f0221e00640 1 --2- 172.21.15.47:0/2020663883 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56330e6d2260 0x56330e6d7a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:26:49.988 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.984+0000 7f0221e00640 1 -- 172.21.15.47:0/2020663883 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56330e3493f0 con 0x56330e6d18b0 2024-03-21T10:26:49.988 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.984+0000 7f0221e00640 1 --2- 172.21.15.47:0/2020663883 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56330e6d18b0 0x56330e6da6d0 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7f0214005fd0 tx=0x7f021409bfc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6f37c30daef004ba server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:26:49.988 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.984+0000 7f0203400640 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-03-21T10:26:49.988 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.985+0000 7f0203400640 1 -- 172.21.15.47:0/2020663883 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f021401d040 con 0x56330e6d18b0 2024-03-21T10:26:49.988 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.985+0000 7f0203400640 1 -- 172.21.15.47:0/2020663883 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(97..97 src has 1..97) v4 ==== 9474+0+0 (secure 0 0 0) 0x7f0214004e90 con 0x56330e6d18b0 2024-03-21T10:26:49.988 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.985+0000 7f022b5fd740 1 -- 172.21.15.47:0/2020663883 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f0210011920 con 0x56330e6d18b0 2024-03-21T10:26:49.988 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.985+0000 7f0203400640 1 -- 172.21.15.47:0/2020663883 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f021409fa10 con 0x56330e6d18b0 2024-03-21T10:26:49.988 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.985+0000 7f022b5fd740 10 client.16281 fetch_fsmap learned FSMap version 32 2024-03-21T10:26:49.988 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.985+0000 7f022b5fd740 1 -- 172.21.15.47:0/2020663883 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x56330e4137d0 con 0x56330e6d18b0 2024-03-21T10:26:49.988 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.985+0000 7f0203400640 1 -- 172.21.15.47:0/2020663883 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f0210011920 con 0x56330e6d18b0 2024-03-21T10:26:49.988 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.986+0000 7f0203400640 1 -- 172.21.15.47:0/2020663883 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 32) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f02140ba110 con 0x56330e6d18b0 2024-03-21T10:26:49.988 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.986+0000 7f022b5fd740 10 client.16281 fetch_fsmap finished waiting for FSMap version 32 2024-03-21T10:26:49.988 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.986+0000 7f022b5fd740 10 client.16281 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-21T10:26:49.988 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.986+0000 7f022b5fd740 20 client.16281 populate_metadata read hostname 'smithi047' 2024-03-21T10:26:49.989 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.986+0000 7f022b5fd740 1 --2- 172.21.15.47:0/2020663883 >> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] conn(0x56330e6eac20 0x56330e6ed010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:26:49.989 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.986+0000 7f022b5fd740 4 client.16281 mds_command: new command op to 34535 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T10:26:49.989 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.986+0000 7f022b5fd740 1 -- 172.21.15.47:0/2020663883 --> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56330e642800 con 0x56330e6eac20 2024-03-21T10:26:49.989 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.986+0000 7f0221400640 1 --2- 172.21.15.47:0/2020663883 >> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] conn(0x56330e6eac20 0x56330e6ed010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:26:49.989 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.987+0000 7f0221400640 1 --2- 172.21.15.47:0/2020663883 >> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] conn(0x56330e6eac20 0x56330e6ed010 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dcad1b42df009d37 server_cookie=fc195952d80d1fb8 in_seq=0 out_seq=0 2024-03-21T10:26:49.989 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.987+0000 7f0203400640 10 client.16281 ms_handle_connect on v2:172.21.15.100:6838/4173168440 2024-03-21T10:26:49.990 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.988+0000 7f0203400640 1 -- 172.21.15.47:0/2020663883 <== mds.0 v2:172.21.15.100:6838/4173168440 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x56330e642800 con 0x56330e6eac20 2024-03-21T10:26:49.990 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.988+0000 7f0203400640 10 client.16281 handle_command_reply: tid=0 2024-03-21T10:26:49.990 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.989+0000 7f022b5fd740 1 client.16281 shutdown 2024-03-21T10:26:49.990 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.989+0000 7f022b5fd740 1 -- 172.21.15.47:0/2020663883 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56330e6d18b0 msgr2=0x56330e6da6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:26:49.990 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.989+0000 7f022b5fd740 1 --2- 172.21.15.47:0/2020663883 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56330e6d18b0 0x56330e6da6d0 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7f0214005fd0 tx=0x7f021409bfc0 comp rx=0 tx=0).stop 2024-03-21T10:26:49.990 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.989+0000 7f022b5fd740 1 -- 172.21.15.47:0/2020663883 shutdown_connections 2024-03-21T10:26:49.990 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.989+0000 7f022b5fd740 1 --2- 172.21.15.47:0/2020663883 >> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] conn(0x56330e6eac20 0x56330e6ed010 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:26:49.990 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.989+0000 7f022b5fd740 1 --2- 172.21.15.47:0/2020663883 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56330e6dade0 0x56330e6d7fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:26:49.991 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.989+0000 7f022b5fd740 1 --2- 172.21.15.47:0/2020663883 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56330e6d2260 0x56330e6d7a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:26:49.991 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.989+0000 7f022b5fd740 1 --2- 172.21.15.47:0/2020663883 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56330e6d18b0 0x56330e6da6d0 unknown :-1 s=CLOSED pgs=654 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:26:49.991 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.989+0000 7f022b5fd740 1 -- 172.21.15.47:0/2020663883 >> 172.21.15.47:0/2020663883 conn(0x56330e6cbf80 msgr2=0x56330e6cdcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:26:49.991 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.989+0000 7f0203400640 0 client.16281 ms_handle_reset on v2:172.21.15.100:6838/4173168440 2024-03-21T10:26:49.991 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.989+0000 7f022b5fd740 1 -- 172.21.15.47:0/2020663883 shutdown_connections 2024-03-21T10:26:49.991 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.989+0000 7f022b5fd740 1 -- 172.21.15.47:0/2020663883 wait complete. 2024-03-21T10:26:49.991 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:49.989+0000 7f022b5fd740 20 client.16281 trim_cache size 0 max 16384 2024-03-21T10:26:50.004 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:50.003+0000 7f022b5fd740 1 Processor -- start 2024-03-21T10:26:50.004 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:50.003+0000 7f022b5fd740 1 -- start start 2024-03-21T10:26:50.004 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:50.003+0000 7f022b5fd740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56330e6d18b0 0x56330e6cdae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:26:50.004 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:50.003+0000 7f022b5fd740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56330e6d2260 0x56330e6da190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:26:50.004 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:50.003+0000 7f022b5fd740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56330e6dade0 0x56330e6da6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:26:50.004 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:50.003+0000 7f022b5fd740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02241b08d0 con 0x56330e6d18b0 2024-03-21T10:26:50.004 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:50.003+0000 7f022b5fd740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02241b2250 con 0x56330e6dade0 2024-03-21T10:26:50.004 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:50.003+0000 7f022b5fd740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02241b0f30 con 0x56330e6d2260 2024-03-21T10:26:50.004 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:50.004+0000 7f0221e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56330e6d18b0 0x56330e6cdae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:26:50.004 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:50.004+0000 7f0221e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56330e6d18b0 0x56330e6cdae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:58188/0 (socket says 172.21.15.47:58188) 2024-03-21T10:26:50.004 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:50.004+0000 7f0221e00640 1 -- 172.21.15.47:0/1730933460 learned_addr learned my addr 172.21.15.47:0/1730933460 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:26:50.006 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:50.004+0000 7f0222800640 1 --2- 172.21.15.47:0/1730933460 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56330e6dade0 0x56330e6da6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:26:50.006 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:50.004+0000 7f0221e00640 1 -- 172.21.15.47:0/1730933460 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56330e6d2260 msgr2=0x56330e6da190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:26:50.006 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:50.004+0000 7f0221e00640 1 --2- 172.21.15.47:0/1730933460 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56330e6d2260 0x56330e6da190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:26:50.006 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:50.004+0000 7f0221e00640 1 -- 172.21.15.47:0/1730933460 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56330e6dade0 msgr2=0x56330e6da6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:26:50.006 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:50.004+0000 7f0221e00640 1 --2- 172.21.15.47:0/1730933460 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56330e6dade0 0x56330e6da6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:26:50.006 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:50.004+0000 7f0221e00640 1 -- 172.21.15.47:0/1730933460 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56330e6d8330 con 0x56330e6d18b0 2024-03-21T10:26:50.006 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:50.004+0000 7f0222800640 1 --2- 172.21.15.47:0/1730933460 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56330e6dade0 0x56330e6da6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T10:26:50.006 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:50.004+0000 7f0221e00640 1 --2- 172.21.15.47:0/1730933460 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56330e6d18b0 0x56330e6cdae0 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7f0214097d20 tx=0x7f0214097610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e106465141cf7f5e server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:26:50.007 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:50.005+0000 7f020e600640 1 -- 172.21.15.47:0/1730933460 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f021401d070 con 0x56330e6d18b0 2024-03-21T10:26:50.007 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:50.005+0000 7f020e600640 1 -- 172.21.15.47:0/1730933460 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f0214004e90 con 0x56330e6d18b0 2024-03-21T10:26:50.007 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:50.005+0000 7f020e600640 1 -- 172.21.15.47:0/1730933460 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02140860d0 con 0x56330e6d18b0 2024-03-21T10:26:50.007 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:50.005+0000 7f022b5fd740 1 -- 172.21.15.47:0/1730933460 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56330e6d18b0 msgr2=0x56330e6cdae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:26:50.007 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:50.005+0000 7f022b5fd740 1 --2- 172.21.15.47:0/1730933460 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56330e6d18b0 0x56330e6cdae0 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7f0214097d20 tx=0x7f0214097610 comp rx=0 tx=0).stop 2024-03-21T10:26:50.007 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:50.005+0000 7f022b5fd740 1 -- 172.21.15.47:0/1730933460 shutdown_connections 2024-03-21T10:26:50.007 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:50.005+0000 7f022b5fd740 1 --2- 172.21.15.47:0/1730933460 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56330e6dade0 0x56330e6da6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:26:50.007 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:50.005+0000 7f022b5fd740 1 --2- 172.21.15.47:0/1730933460 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56330e6d2260 0x56330e6da190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:26:50.007 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:50.005+0000 7f022b5fd740 1 --2- 172.21.15.47:0/1730933460 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56330e6d18b0 0x56330e6cdae0 unknown :-1 s=CLOSED pgs=655 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:26:50.007 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:50.005+0000 7f022b5fd740 1 -- 172.21.15.47:0/1730933460 >> 172.21.15.47:0/1730933460 conn(0x56330e6cbf80 msgr2=0x56330e6cd3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:26:50.007 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:50.006+0000 7f022b5fd740 1 -- 172.21.15.47:0/1730933460 shutdown_connections 2024-03-21T10:26:50.007 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:50.006+0000 7f022b5fd740 1 -- 172.21.15.47:0/1730933460 wait complete. 2024-03-21T10:26:50.007 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:50.006+0000 7f022b5fd740 1 -- start start 2024-03-21T10:26:50.007 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:50.006+0000 7f022b5fd740 1 Processor -- start 2024-03-21T10:26:50.011 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:50.007+0000 7f022b5fd740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56330e6d18b0 0x56330e6e2810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:26:50.011 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:50.007+0000 7f022b5fd740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56330e6d2260 0x56330e6e2d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:26:50.011 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:50.007+0000 7f022b5fd740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56330e6dade0 0x56330e6dec90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:26:50.011 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:50.007+0000 7f022b5fd740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02241b1bf0 con 0x56330e6dade0 2024-03-21T10:26:50.011 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:50.007+0000 7f022b5fd740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02241b1590 con 0x56330e6d18b0 2024-03-21T10:26:50.011 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:50.007+0000 7f022b5fd740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02241aea90 con 0x56330e6d2260 2024-03-21T10:26:50.012 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:50.007+0000 7f022b5fd740 10 client.0 _ll_register_callbacks cb 0x7f02140046c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T10:26:50.012 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:50.007+0000 7f0222800640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56330e6dade0 0x56330e6dec90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:26:50.012 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:50.007+0000 7f0222800640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56330e6dade0 0x56330e6dec90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:58202/0 (socket says 172.21.15.47:58202) 2024-03-21T10:26:50.012 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:50.007+0000 7f0222800640 1 -- 172.21.15.47:0/3166781810 learned_addr learned my addr 172.21.15.47:0/3166781810 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:26:50.012 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:50.007+0000 7f0221e00640 1 --2- 172.21.15.47:0/3166781810 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56330e6d18b0 0x56330e6e2810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:26:50.012 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:50.007+0000 7f0221400640 1 --2- 172.21.15.47:0/3166781810 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56330e6d2260 0x56330e6e2d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:26:50.012 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:50.007+0000 7f0222800640 1 -- 172.21.15.47:0/3166781810 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56330e6d2260 msgr2=0x56330e6e2d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:26:50.012 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:50.007+0000 7f0222800640 1 --2- 172.21.15.47:0/3166781810 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56330e6d2260 0x56330e6e2d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:26:50.012 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:50.007+0000 7f0222800640 1 -- 172.21.15.47:0/3166781810 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56330e6d18b0 msgr2=0x56330e6e2810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:26:50.012 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:50.007+0000 7f0222800640 1 --2- 172.21.15.47:0/3166781810 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56330e6d18b0 0x56330e6e2810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:26:50.012 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:50.007+0000 7f0222800640 1 -- 172.21.15.47:0/3166781810 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56330e6df470 con 0x56330e6dade0 2024-03-21T10:26:50.012 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:50.008+0000 7f0222800640 1 --2- 172.21.15.47:0/3166781810 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56330e6dade0 0x56330e6dec90 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7f021c037040 tx=0x7f021c005c50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fd1d00cd81cf39ab server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:26:50.012 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:50.008+0000 7f0203400640 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-03-21T10:26:50.012 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:50.008+0000 7f0203400640 1 -- 172.21.15.47:0/3166781810 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f021c039040 con 0x56330e6dade0 2024-03-21T10:26:50.012 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:50.008+0000 7f0203400640 1 -- 172.21.15.47:0/3166781810 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(97..97 src has 1..97) v4 ==== 9474+0+0 (secure 0 0 0) 0x7f021c004040 con 0x56330e6dade0 2024-03-21T10:26:50.013 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:50.008+0000 7f022b5fd740 1 -- 172.21.15.47:0/3166781810 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f0214004e90 con 0x56330e6dade0 2024-03-21T10:26:50.013 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:50.008+0000 7f0203400640 1 -- 172.21.15.47:0/3166781810 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f021c059050 con 0x56330e6dade0 2024-03-21T10:26:50.013 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:50.008+0000 7f022b5fd740 10 client.16287 fetch_fsmap learned FSMap version 32 2024-03-21T10:26:50.013 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:50.008+0000 7f022b5fd740 1 -- 172.21.15.47:0/3166781810 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x56330e746e60 con 0x56330e6dade0 2024-03-21T10:26:50.013 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:50.009+0000 7f0203400640 1 -- 172.21.15.47:0/3166781810 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f0214004e90 con 0x56330e6dade0 2024-03-21T10:26:50.013 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:50.009+0000 7f0203400640 1 -- 172.21.15.47:0/3166781810 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 32) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f021c054030 con 0x56330e6dade0 2024-03-21T10:26:50.013 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:50.009+0000 7f022b5fd740 10 client.16287 fetch_fsmap finished waiting for FSMap version 32 2024-03-21T10:26:50.013 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:50.009+0000 7f022b5fd740 10 client.16287 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-21T10:26:50.013 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:50.009+0000 7f022b5fd740 20 client.16287 populate_metadata read hostname 'smithi047' 2024-03-21T10:26:50.013 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:50.009+0000 7f022b5fd740 1 --2- 172.21.15.47:0/3166781810 >> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] conn(0x56330e7475b0 0x56330e7499d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:26:50.013 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:50.009+0000 7f022b5fd740 4 client.16287 mds_command: new command op to 34535 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-21T10:26:50.013 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:50.009+0000 7f022b5fd740 1 -- 172.21.15.47:0/3166781810 --> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x56330e63cb50 con 0x56330e7475b0 2024-03-21T10:26:50.013 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:50.009+0000 7f0221e00640 1 --2- 172.21.15.47:0/3166781810 >> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] conn(0x56330e7475b0 0x56330e7499d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:26:50.013 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:50.010+0000 7f0221e00640 1 --2- 172.21.15.47:0/3166781810 >> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] conn(0x56330e7475b0 0x56330e7499d0 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f21908d94ac7a5d7 server_cookie=c64475ad5804e6de in_seq=0 out_seq=0 2024-03-21T10:26:50.014 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:50.010+0000 7f0203400640 10 client.16287 ms_handle_connect on v2:172.21.15.100:6838/4173168440 2024-03-21T10:26:50.014 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:50.010+0000 7f0203400640 1 -- 172.21.15.47:0/3166781810 <== mds.0 v2:172.21.15.100:6838/4173168440 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x56330e63cb50 con 0x56330e7475b0 2024-03-21T10:26:50.014 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:50.010+0000 7f0203400640 10 client.16287 handle_command_reply: tid=0 2024-03-21T10:26:50.014 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:50.010+0000 7f022b5fd740 1 client.16287 shutdown 2024-03-21T10:26:50.014 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:50.010+0000 7f022b5fd740 1 -- 172.21.15.47:0/3166781810 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56330e6dade0 msgr2=0x56330e6dec90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:26:50.014 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:50.010+0000 7f022b5fd740 1 --2- 172.21.15.47:0/3166781810 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56330e6dade0 0x56330e6dec90 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7f021c037040 tx=0x7f021c005c50 comp rx=0 tx=0).stop 2024-03-21T10:26:50.014 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:50.010+0000 7f022b5fd740 1 -- 172.21.15.47:0/3166781810 shutdown_connections 2024-03-21T10:26:50.014 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:50.011+0000 7f022b5fd740 1 --2- 172.21.15.47:0/3166781810 >> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] conn(0x56330e7475b0 0x56330e7499d0 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:26:50.014 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:50.011+0000 7f022b5fd740 1 --2- 172.21.15.47:0/3166781810 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56330e6dade0 0x56330e6dec90 unknown :-1 s=CLOSED pgs=656 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:26:50.014 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:50.011+0000 7f022b5fd740 1 --2- 172.21.15.47:0/3166781810 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56330e6d2260 0x56330e6e2d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:26:50.014 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:50.011+0000 7f022b5fd740 1 --2- 172.21.15.47:0/3166781810 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56330e6d18b0 0x56330e6e2810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:26:50.014 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:50.011+0000 7f0203400640 0 client.16287 ms_handle_reset on v2:172.21.15.100:6838/4173168440 2024-03-21T10:26:50.014 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:50.011+0000 7f022b5fd740 1 -- 172.21.15.47:0/3166781810 >> 172.21.15.47:0/3166781810 conn(0x56330e6cbf80 msgr2=0x56330e6cdc30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:26:50.015 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:50.011+0000 7f022b5fd740 1 -- 172.21.15.47:0/3166781810 shutdown_connections 2024-03-21T10:26:50.015 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:50.011+0000 7f022b5fd740 1 -- 172.21.15.47:0/3166781810 wait complete. 2024-03-21T10:26:50.015 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:50.011+0000 7f022b5fd740 20 client.16287 trim_cache size 0 max 16384 2024-03-21T10:26:50.015 INFO:teuthology.orchestra.run.smithi047.stdout:[] 2024-03-21T10:26:50.016 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:50.011+0000 7f0223200640 1 -- 172.21.15.47:0/1782016614 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f02080a30c0 msgr2=0x7f02080a5580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:26:50.016 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:50.011+0000 7f0223200640 1 --2- 172.21.15.47:0/1782016614 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f02080a30c0 0x7f02080a5580 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f021c004af0 tx=0x7f021c02e040 comp rx=0 tx=0).stop 2024-03-21T10:26:50.016 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:50.011+0000 7f0223200640 1 -- 172.21.15.47:0/1782016614 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0224101530 msgr2=0x7f02240739a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:26:50.016 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:50.011+0000 7f0223200640 1 --2- 172.21.15.47:0/1782016614 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0224101530 0x7f02240739a0 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7f021400c890 tx=0x7f021400cd50 comp rx=0 tx=0).stop 2024-03-21T10:26:50.016 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:50.012+0000 7f0223200640 1 -- 172.21.15.47:0/1782016614 shutdown_connections 2024-03-21T10:26:50.016 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:50.012+0000 7f0223200640 1 --2- 172.21.15.47:0/1782016614 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f02080a30c0 0x7f02080a5580 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:26:50.016 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:50.012+0000 7f0223200640 1 --2- 172.21.15.47:0/1782016614 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f02241075f0 0x7f0224074420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:26:50.016 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:50.012+0000 7f0223200640 1 --2- 172.21.15.47:0/1782016614 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0224101ef0 0x7f0224073ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:26:50.017 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:50.012+0000 7f0223200640 1 --2- 172.21.15.47:0/1782016614 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0224101530 0x7f02240739a0 unknown :-1 s=CLOSED pgs=652 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:26:50.017 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:50.012+0000 7f0223200640 1 -- 172.21.15.47:0/1782016614 >> 172.21.15.47:0/1782016614 conn(0x7f02240fd040 msgr2=0x7f02240fe890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:26:50.017 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:50.012+0000 7f0223200640 1 -- 172.21.15.47:0/1782016614 shutdown_connections 2024-03-21T10:26:50.017 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:50.012+0000 7f0223200640 1 -- 172.21.15.47:0/1782016614 wait complete. 2024-03-21T10:26:51.029 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-03-21T10:26:51.029 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-21T10:26:51.029 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-03-21T10:26:51.258 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:51.257+0000 7f6593e00640 1 -- 172.21.15.47:0/2006478555 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f659410cea0 msgr2=0x7f6594114a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:26:51.258 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:51.257+0000 7f6593e00640 1 --2- 172.21.15.47:0/2006478555 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f659410cea0 0x7f6594114a70 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f658c009be0 tx=0x7f658c02bef0 comp rx=0 tx=0).stop 2024-03-21T10:26:51.258 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:51.257+0000 7f6593e00640 1 -- 172.21.15.47:0/2006478555 shutdown_connections 2024-03-21T10:26:51.258 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:51.257+0000 7f6593e00640 1 --2- 172.21.15.47:0/2006478555 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f659410cea0 0x7f6594114a70 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:26:51.258 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:51.257+0000 7f6593e00640 1 --2- 172.21.15.47:0/2006478555 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6594108800 0x7f659410c7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:26:51.258 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:51.257+0000 7f6593e00640 1 --2- 172.21.15.47:0/2006478555 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6594107e50 0x7f6594108230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:26:51.258 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:51.257+0000 7f6593e00640 1 -- 172.21.15.47:0/2006478555 >> 172.21.15.47:0/2006478555 conn(0x7f65940774c0 msgr2=0x7f65941064c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:26:51.258 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:51.258+0000 7f6593e00640 1 -- 172.21.15.47:0/2006478555 shutdown_connections 2024-03-21T10:26:51.258 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:51.258+0000 7f6593e00640 1 -- 172.21.15.47:0/2006478555 wait complete. 2024-03-21T10:26:51.260 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:51.258+0000 7f6593e00640 1 Processor -- start 2024-03-21T10:26:51.260 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:51.259+0000 7f6593e00640 1 -- start start 2024-03-21T10:26:51.260 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:51.259+0000 7f6593e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6594107e50 0x7f659419e300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:26:51.260 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:51.259+0000 7f6593e00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f659410cea0 0x7f659419e840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:26:51.260 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:51.259+0000 7f6593e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f65941a2b70 0x7f65941a3020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:26:51.260 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:51.259+0000 7f6593e00640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6594117e20 con 0x7f65941a2b70 2024-03-21T10:26:51.260 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:51.259+0000 7f6593e00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6594117ca0 con 0x7f6594107e50 2024-03-21T10:26:51.260 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:51.259+0000 7f6593e00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6594117fa0 con 0x7f659410cea0 2024-03-21T10:26:51.260 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:51.260+0000 7f6593400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f65941a2b70 0x7f65941a3020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:26:51.260 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:51.260+0000 7f6593400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f65941a2b70 0x7f65941a3020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:58226/0 (socket says 172.21.15.47:58226) 2024-03-21T10:26:51.260 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:51.260+0000 7f6593400640 1 -- 172.21.15.47:0/1025440226 learned_addr learned my addr 172.21.15.47:0/1025440226 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:26:51.260 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:51.260+0000 7f6592a00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6594107e50 0x7f659419e300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:26:51.261 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:51.260+0000 7f6592000640 1 --2- 172.21.15.47:0/1025440226 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f659410cea0 0x7f659419e840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:26:51.262 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:51.260+0000 7f6593400640 1 -- 172.21.15.47:0/1025440226 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f659410cea0 msgr2=0x7f659419e840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:26:51.262 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:51.260+0000 7f6593400640 1 --2- 172.21.15.47:0/1025440226 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f659410cea0 0x7f659419e840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:26:51.262 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:51.260+0000 7f6593400640 1 -- 172.21.15.47:0/1025440226 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6594107e50 msgr2=0x7f659419e300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:26:51.262 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:51.260+0000 7f6593400640 1 --2- 172.21.15.47:0/1025440226 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6594107e50 0x7f659419e300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:26:51.262 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:51.260+0000 7f6593400640 1 -- 172.21.15.47:0/1025440226 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65941a3660 con 0x7f65941a2b70 2024-03-21T10:26:51.262 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:51.260+0000 7f6592a00640 1 --2- 172.21.15.47:0/1025440226 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6594107e50 0x7f659419e300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T10:26:51.262 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:51.261+0000 7f6593400640 1 --2- 172.21.15.47:0/1025440226 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f65941a2b70 0x7f65941a3020 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7f658c009d00 tx=0x7f658c02cb30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:26:51.262 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:51.261+0000 7f658b400640 1 -- 172.21.15.47:0/1025440226 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f658c043020 con 0x7f65941a2b70 2024-03-21T10:26:51.262 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:51.261+0000 7f6593e00640 1 -- 172.21.15.47:0/1025440226 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f65941a38f0 con 0x7f65941a2b70 2024-03-21T10:26:51.262 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:51.261+0000 7f658b400640 1 -- 172.21.15.47:0/1025440226 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f658c003e20 con 0x7f65941a2b70 2024-03-21T10:26:51.262 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:51.261+0000 7f658b400640 1 -- 172.21.15.47:0/1025440226 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f658c034430 con 0x7f65941a2b70 2024-03-21T10:26:51.263 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:51.261+0000 7f6593e00640 1 -- 172.21.15.47:0/1025440226 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f65941ab0f0 con 0x7f65941a2b70 2024-03-21T10:26:51.264 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:51.264+0000 7f659c439740 1 Processor -- start 2024-03-21T10:26:51.264 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:51.264+0000 7f658b400640 1 -- 172.21.15.47:0/1025440226 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 26) v1 ==== 138087+0+0 (secure 0 0 0) 0x7f658c046020 con 0x7f65941a2b70 2024-03-21T10:26:51.265 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:51.265+0000 7f659c439740 1 -- start start 2024-03-21T10:26:51.266 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:51.265+0000 7f659c439740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557ead944290 0x557ead944670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:26:51.266 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:51.265+0000 7f659c439740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x557ead944bb0 0x557ead94d250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:26:51.267 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:51.265+0000 7f659c439740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557ead94d790 0x557ead94fb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:26:51.267 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:51.265+0000 7f659c439740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x557ead8a8eb0 con 0x557ead944bb0 2024-03-21T10:26:51.267 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:51.265+0000 7f659c439740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x557ead851a60 con 0x557ead94d790 2024-03-21T10:26:51.267 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:51.265+0000 7f659c439740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x557ead8a4f50 con 0x557ead944290 2024-03-21T10:26:51.267 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:51.265+0000 7f6592a00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x557ead944bb0 0x557ead94d250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:26:51.267 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:51.265+0000 7f6592a00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x557ead944bb0 0x557ead94d250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:58238/0 (socket says 172.21.15.47:58238) 2024-03-21T10:26:51.267 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:51.265+0000 7f6592a00640 1 -- 172.21.15.47:0/2988821648 learned_addr learned my addr 172.21.15.47:0/2988821648 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:26:51.267 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:51.265+0000 7f6592000640 1 --2- 172.21.15.47:0/2988821648 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557ead94d790 0x557ead94fb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:26:51.267 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:51.266+0000 7f6592000640 1 --2- 172.21.15.47:0/2988821648 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557ead944290 0x557ead944670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:26:51.267 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:51.266+0000 7f6592a00640 1 -- 172.21.15.47:0/2988821648 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557ead944290 msgr2=0x557ead944670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:26:51.267 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:51.266+0000 7f6592a00640 1 --2- 172.21.15.47:0/2988821648 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557ead944290 0x557ead944670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:26:51.267 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:51.266+0000 7f6592a00640 1 -- 172.21.15.47:0/2988821648 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557ead94d790 msgr2=0x557ead94fb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:26:51.267 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:51.266+0000 7f6592a00640 1 --2- 172.21.15.47:0/2988821648 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557ead94d790 0x557ead94fb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:26:51.267 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:51.266+0000 7f6592a00640 1 -- 172.21.15.47:0/2988821648 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557ead6857e0 con 0x557ead944bb0 2024-03-21T10:26:51.268 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:51.266+0000 7f6592a00640 1 --2- 172.21.15.47:0/2988821648 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x557ead944bb0 0x557ead94d250 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7f65840115c0 tx=0x7f6584011a80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a2c87a68a4998fdf server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:26:51.268 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:51.266+0000 7f658b400640 1 --2- 172.21.15.47:0/1025440226 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f65780a31b0 0x7f65780a5670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:26:51.268 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:51.266+0000 7f658b400640 1 -- 172.21.15.47:0/1025440226 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 9474+0+0 (secure 0 0 0) 0x7f658c0f22b0 con 0x7f65941a2b70 2024-03-21T10:26:51.269 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:51.267+0000 7f657f400640 1 -- 172.21.15.47:0/2988821648 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f658400f640 con 0x557ead944bb0 2024-03-21T10:26:51.269 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:51.267+0000 7f6592000640 1 --2- 172.21.15.47:0/1025440226 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f65780a31b0 0x7f65780a5670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:26:51.269 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:51.267+0000 7f657f400640 1 -- 172.21.15.47:0/2988821648 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f658400fc90 con 0x557ead944bb0 2024-03-21T10:26:51.269 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:51.267+0000 7f657f400640 1 -- 172.21.15.47:0/2988821648 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6584019c60 con 0x557ead944bb0 2024-03-21T10:26:51.269 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:51.268+0000 7f659c439740 1 -- 172.21.15.47:0/2988821648 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x557ead944bb0 msgr2=0x557ead94d250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:26:51.269 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:51.268+0000 7f659c439740 1 --2- 172.21.15.47:0/2988821648 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x557ead944bb0 0x557ead94d250 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7f65840115c0 tx=0x7f6584011a80 comp rx=0 tx=0).stop 2024-03-21T10:26:51.269 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:51.268+0000 7f6592000640 1 --2- 172.21.15.47:0/1025440226 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f65780a31b0 0x7f65780a5670 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x557ead93fb50 tx=0x7f6580009550 comp rx=0 tx=0).ready entity=mgr.15687 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:26:51.269 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:51.268+0000 7f659c439740 1 -- 172.21.15.47:0/2988821648 shutdown_connections 2024-03-21T10:26:51.269 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:51.268+0000 7f659c439740 1 --2- 172.21.15.47:0/2988821648 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557ead94d790 0x557ead94fb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:26:51.270 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:51.268+0000 7f659c439740 1 --2- 172.21.15.47:0/2988821648 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x557ead944bb0 0x557ead94d250 unknown :-1 s=CLOSED pgs=658 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:26:51.270 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:51.268+0000 7f659c439740 1 --2- 172.21.15.47:0/2988821648 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557ead944290 0x557ead944670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:26:51.270 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:51.268+0000 7f659c439740 1 -- 172.21.15.47:0/2988821648 >> 172.21.15.47:0/2988821648 conn(0x557ead93e9c0 msgr2=0x557ead93eda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:26:51.270 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:51.268+0000 7f659c439740 1 -- 172.21.15.47:0/2988821648 shutdown_connections 2024-03-21T10:26:51.270 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:51.268+0000 7f659c439740 1 -- 172.21.15.47:0/2988821648 wait complete. 2024-03-21T10:26:51.270 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:51.269+0000 7f659c439740 1 -- start start 2024-03-21T10:26:51.270 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:51.269+0000 7f659c439740 1 Processor -- start 2024-03-21T10:26:51.272 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:51.269+0000 7f659c439740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557ead944290 0x557ead94d140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:26:51.272 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:51.269+0000 7f659c439740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557ead944bb0 0x557ead94a4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:26:51.272 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:51.269+0000 7f659c439740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x557ead94d790 0x557ead94aa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:26:51.272 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:51.269+0000 7f659c439740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x557ead950580 con 0x557ead94d790 2024-03-21T10:26:51.272 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:51.269+0000 7f659c439740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x557ead864eb0 con 0x557ead944290 2024-03-21T10:26:51.272 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:51.269+0000 7f659c439740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x557ead950700 con 0x557ead944bb0 2024-03-21T10:26:51.272 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:51.270+0000 7f659c439740 10 client.0 _ll_register_callbacks cb 0x557ead859150 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T10:26:51.272 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:51.270+0000 7f6593400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x557ead94d790 0x557ead94aa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:26:51.272 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:51.270+0000 7f6592a00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557ead944290 0x557ead94d140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:26:51.272 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:51.270+0000 7f6592a00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557ead944290 0x557ead94d140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:52514/0 (socket says 172.21.15.47:52514) 2024-03-21T10:26:51.272 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:51.270+0000 7f6592a00640 1 -- 172.21.15.47:0/67201065 learned_addr learned my addr 172.21.15.47:0/67201065 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:26:51.273 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:51.270+0000 7f6593400640 1 -- 172.21.15.47:0/67201065 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557ead944bb0 msgr2=0x557ead94a4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:26:51.273 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:51.270+0000 7f6592000640 1 --2- 172.21.15.47:0/67201065 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557ead944bb0 0x557ead94a4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:26:51.273 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:51.270+0000 7f6593400640 1 --2- 172.21.15.47:0/67201065 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557ead944bb0 0x557ead94a4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:26:51.273 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:51.270+0000 7f6593400640 1 -- 172.21.15.47:0/67201065 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557ead944290 msgr2=0x557ead94d140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:26:51.273 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:51.270+0000 7f6593400640 1 --2- 172.21.15.47:0/67201065 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557ead944290 0x557ead94d140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:26:51.273 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:51.270+0000 7f6593400640 1 -- 172.21.15.47:0/67201065 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x557ead6e5f70 con 0x557ead94d790 2024-03-21T10:26:51.273 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:51.270+0000 7f6592a00640 1 --2- 172.21.15.47:0/67201065 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557ead944290 0x557ead94d140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T10:26:51.273 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:51.270+0000 7f6592000640 1 --2- 172.21.15.47:0/67201065 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557ead944bb0 0x557ead94a4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T10:26:51.273 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:51.271+0000 7f6593400640 1 --2- 172.21.15.47:0/67201065 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x557ead94d790 0x557ead94aa70 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7f658c0ef570 tx=0x7f658c0bdbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=429c9009def7ca2b server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:26:51.273 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:51.271+0000 7f6573e00640 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-03-21T10:26:51.274 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:51.274+0000 7f6573e00640 1 -- 172.21.15.47:0/67201065 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f658c0be1d0 con 0x557ead94d790 2024-03-21T10:26:51.274 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:51.274+0000 7f6573e00640 1 -- 172.21.15.47:0/67201065 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(97..97 src has 1..97) v4 ==== 9474+0+0 (secure 0 0 0) 0x7f658c0de0e0 con 0x557ead94d790 2024-03-21T10:26:51.275 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:51.274+0000 7f659c439740 1 -- 172.21.15.47:0/67201065 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f658400fc90 con 0x557ead94d790 2024-03-21T10:26:51.275 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:51.275+0000 7f6573e00640 1 -- 172.21.15.47:0/67201065 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f658c0c2320 con 0x557ead94d790 2024-03-21T10:26:51.275 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:51.275+0000 7f659c439740 10 client.16299 fetch_fsmap learned FSMap version 32 2024-03-21T10:26:51.275 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:51.275+0000 7f659c439740 1 -- 172.21.15.47:0/67201065 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x557ead57d170 con 0x557ead94d790 2024-03-21T10:26:51.276 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:51.275+0000 7f6573e00640 1 -- 172.21.15.47:0/67201065 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f658400fc90 con 0x557ead94d790 2024-03-21T10:26:51.276 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:51.275+0000 7f6573e00640 1 -- 172.21.15.47:0/67201065 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 32) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f658c0ef740 con 0x557ead94d790 2024-03-21T10:26:51.276 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:51.275+0000 7f659c439740 10 client.16299 fetch_fsmap finished waiting for FSMap version 32 2024-03-21T10:26:51.276 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:51.275+0000 7f659c439740 10 client.16299 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-21T10:26:51.276 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:51.275+0000 7f659c439740 20 client.16299 populate_metadata read hostname 'smithi047' 2024-03-21T10:26:51.276 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:51.275+0000 7f659c439740 1 --2- 172.21.15.47:0/67201065 >> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] conn(0x557ead95d6b0 0x557ead95fac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:26:51.276 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:51.275+0000 7f659c439740 4 client.16299 mds_command: new command op to 34535 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T10:26:51.276 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:51.275+0000 7f659c439740 1 -- 172.21.15.47:0/67201065 --> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x557ead8b63f0 con 0x557ead95d6b0 2024-03-21T10:26:51.276 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:51.276+0000 7f6592a00640 1 --2- 172.21.15.47:0/67201065 >> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] conn(0x557ead95d6b0 0x557ead95fac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:26:51.277 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:51.277+0000 7f6573e00640 10 client.16299 ms_handle_connect on v2:172.21.15.100:6838/4173168440 2024-03-21T10:26:51.277 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:51.277+0000 7f6592a00640 1 --2- 172.21.15.47:0/67201065 >> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] conn(0x557ead95d6b0 0x557ead95fac0 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=450f9c2af36e72de server_cookie=72935c4a86d614c in_seq=0 out_seq=0 2024-03-21T10:26:51.278 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:51.277+0000 7f6573e00640 1 -- 172.21.15.47:0/67201065 <== mds.0 v2:172.21.15.100:6838/4173168440 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x557ead8b63f0 con 0x557ead95d6b0 2024-03-21T10:26:51.278 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:51.277+0000 7f6573e00640 10 client.16299 handle_command_reply: tid=0 2024-03-21T10:26:51.278 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:51.277+0000 7f659c439740 1 client.16299 shutdown 2024-03-21T10:26:51.278 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:51.278+0000 7f659c439740 1 -- 172.21.15.47:0/67201065 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x557ead94d790 msgr2=0x557ead94aa70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:26:51.278 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:51.278+0000 7f659c439740 1 --2- 172.21.15.47:0/67201065 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x557ead94d790 0x557ead94aa70 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7f658c0ef570 tx=0x7f658c0bdbf0 comp rx=0 tx=0).stop 2024-03-21T10:26:51.278 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:51.278+0000 7f659c439740 1 -- 172.21.15.47:0/67201065 shutdown_connections 2024-03-21T10:26:51.278 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:51.278+0000 7f659c439740 1 --2- 172.21.15.47:0/67201065 >> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] conn(0x557ead95d6b0 0x557ead95fac0 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:26:51.279 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:51.278+0000 7f659c439740 1 --2- 172.21.15.47:0/67201065 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x557ead94d790 0x557ead94aa70 unknown :-1 s=CLOSED pgs=659 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:26:51.279 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:51.278+0000 7f659c439740 1 --2- 172.21.15.47:0/67201065 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557ead944bb0 0x557ead94a4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:26:51.279 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:51.278+0000 7f659c439740 1 --2- 172.21.15.47:0/67201065 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557ead944290 0x557ead94d140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:26:51.279 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:51.278+0000 7f659c439740 1 -- 172.21.15.47:0/67201065 >> 172.21.15.47:0/67201065 conn(0x557ead93e9c0 msgr2=0x557ead940710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:26:51.279 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:51.278+0000 7f6573e00640 0 client.16299 ms_handle_reset on v2:172.21.15.100:6838/4173168440 2024-03-21T10:26:51.279 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:51.278+0000 7f659c439740 1 -- 172.21.15.47:0/67201065 shutdown_connections 2024-03-21T10:26:51.279 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:51.278+0000 7f659c439740 1 -- 172.21.15.47:0/67201065 wait complete. 2024-03-21T10:26:51.279 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:51.278+0000 7f659c439740 20 client.16299 trim_cache size 0 max 16384 2024-03-21T10:26:51.296 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:51.296+0000 7f659c439740 1 Processor -- start 2024-03-21T10:26:51.297 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:51.296+0000 7f659c439740 1 -- start start 2024-03-21T10:26:51.297 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:51.296+0000 7f659c439740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557ead944290 0x557ead94cc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:26:51.297 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:51.296+0000 7f659c439740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x557ead944bb0 0x557ead94d140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:26:51.297 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:51.296+0000 7f659c439740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557ead94d790 0x557ead940300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:26:51.297 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:51.296+0000 7f659c439740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65941b4be0 con 0x557ead944bb0 2024-03-21T10:26:51.297 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:51.296+0000 7f659c439740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65941b6560 con 0x557ead944290 2024-03-21T10:26:51.297 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:51.297+0000 7f659c439740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65941b5240 con 0x557ead94d790 2024-03-21T10:26:51.299 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:51.297+0000 7f6592000640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x557ead944bb0 0x557ead94d140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:26:51.299 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:51.297+0000 7f6592000640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x557ead944bb0 0x557ead94d140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:58252/0 (socket says 172.21.15.47:58252) 2024-03-21T10:26:51.299 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:51.297+0000 7f6592000640 1 -- 172.21.15.47:0/3421258768 learned_addr learned my addr 172.21.15.47:0/3421258768 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:26:51.299 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:51.297+0000 7f6592a00640 1 --2- 172.21.15.47:0/3421258768 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557ead944290 0x557ead94cc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:26:51.299 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:51.297+0000 7f6593400640 1 --2- 172.21.15.47:0/3421258768 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557ead94d790 0x557ead940300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:26:51.299 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:51.297+0000 7f6592000640 1 -- 172.21.15.47:0/3421258768 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557ead94d790 msgr2=0x557ead940300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:26:51.299 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:51.297+0000 7f6592000640 1 --2- 172.21.15.47:0/3421258768 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557ead94d790 0x557ead940300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:26:51.299 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:51.297+0000 7f6592000640 1 -- 172.21.15.47:0/3421258768 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557ead944290 msgr2=0x557ead94cc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:26:51.300 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:51.297+0000 7f6592000640 1 --2- 172.21.15.47:0/3421258768 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557ead944290 0x557ead94cc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:26:51.300 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:51.297+0000 7f6592000640 1 -- 172.21.15.47:0/3421258768 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557ead949480 con 0x557ead944bb0 2024-03-21T10:26:51.300 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:51.298+0000 7f6592000640 1 --2- 172.21.15.47:0/3421258768 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x557ead944bb0 0x557ead94d140 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7f6580009fd0 tx=0x7f6580004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7830a26f212fc643 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:26:51.300 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:51.298+0000 7f657ea00640 1 -- 172.21.15.47:0/3421258768 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6580004030 con 0x557ead944bb0 2024-03-21T10:26:51.300 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:51.298+0000 7f657ea00640 1 -- 172.21.15.47:0/3421258768 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f6580026030 con 0x557ead944bb0 2024-03-21T10:26:51.300 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:51.298+0000 7f657ea00640 1 -- 172.21.15.47:0/3421258768 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6580022740 con 0x557ead944bb0 2024-03-21T10:26:51.300 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:51.299+0000 7f659c439740 1 -- 172.21.15.47:0/3421258768 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x557ead944bb0 msgr2=0x557ead94d140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:26:51.300 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:51.299+0000 7f659c439740 1 --2- 172.21.15.47:0/3421258768 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x557ead944bb0 0x557ead94d140 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7f6580009fd0 tx=0x7f6580004730 comp rx=0 tx=0).stop 2024-03-21T10:26:51.300 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:51.299+0000 7f659c439740 1 -- 172.21.15.47:0/3421258768 shutdown_connections 2024-03-21T10:26:51.300 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:51.299+0000 7f659c439740 1 --2- 172.21.15.47:0/3421258768 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557ead94d790 0x557ead940300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:26:51.300 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:51.299+0000 7f659c439740 1 --2- 172.21.15.47:0/3421258768 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x557ead944bb0 0x557ead94d140 unknown :-1 s=CLOSED pgs=660 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:26:51.300 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:51.299+0000 7f659c439740 1 --2- 172.21.15.47:0/3421258768 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557ead944290 0x557ead94cc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:26:51.300 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:51.299+0000 7f659c439740 1 -- 172.21.15.47:0/3421258768 >> 172.21.15.47:0/3421258768 conn(0x557ead93e9c0 msgr2=0x557ead93fbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:26:51.300 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:51.299+0000 7f659c439740 1 -- 172.21.15.47:0/3421258768 shutdown_connections 2024-03-21T10:26:51.301 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:51.299+0000 7f659c439740 1 -- 172.21.15.47:0/3421258768 wait complete. 2024-03-21T10:26:51.303 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:51.299+0000 7f659c439740 1 -- start start 2024-03-21T10:26:51.303 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:51.300+0000 7f659c439740 1 Processor -- start 2024-03-21T10:26:51.303 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:51.300+0000 7f659c439740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557ead944290 0x557ead952920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:26:51.303 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:51.300+0000 7f659c439740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557ead944bb0 0x557ead952e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:26:51.303 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:51.300+0000 7f659c439740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x557ead94d790 0x557ead9533a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:26:51.303 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:51.300+0000 7f659c439740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65941b5f00 con 0x557ead94d790 2024-03-21T10:26:51.303 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:51.300+0000 7f659c439740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65941b58a0 con 0x557ead944290 2024-03-21T10:26:51.303 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:51.300+0000 7f659c439740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65941b2da0 con 0x557ead944bb0 2024-03-21T10:26:51.303 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:51.300+0000 7f659c439740 10 client.0 _ll_register_callbacks cb 0x7f658c005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T10:26:51.303 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:51.300+0000 7f6593400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x557ead94d790 0x557ead9533a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:26:51.303 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:51.301+0000 7f6593400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x557ead94d790 0x557ead9533a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:58260/0 (socket says 172.21.15.47:58260) 2024-03-21T10:26:51.304 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:51.301+0000 7f6593400640 1 -- 172.21.15.47:0/897616344 learned_addr learned my addr 172.21.15.47:0/897616344 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:26:51.304 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:51.301+0000 7f6592a00640 1 --2- 172.21.15.47:0/897616344 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557ead944290 0x557ead952920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:26:51.304 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:51.301+0000 7f6593400640 1 -- 172.21.15.47:0/897616344 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557ead944bb0 msgr2=0x557ead952e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:26:51.304 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:51.301+0000 7f6593400640 1 --2- 172.21.15.47:0/897616344 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557ead944bb0 0x557ead952e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:26:51.304 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:51.301+0000 7f6593400640 1 -- 172.21.15.47:0/897616344 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557ead944290 msgr2=0x557ead952920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:26:51.304 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:51.301+0000 7f6593400640 1 --2- 172.21.15.47:0/897616344 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557ead944290 0x557ead952920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:26:51.304 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:51.301+0000 7f6593400640 1 -- 172.21.15.47:0/897616344 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x557ead9b9010 con 0x557ead94d790 2024-03-21T10:26:51.304 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:51.301+0000 7f6593400640 1 --2- 172.21.15.47:0/897616344 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x557ead94d790 0x557ead9533a0 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7f658c034fd0 tx=0x7f658c0de2e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dd524af9162b912 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:26:51.304 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:51.301+0000 7f6573e00640 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-03-21T10:26:51.304 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:51.302+0000 7f6573e00640 1 -- 172.21.15.47:0/897616344 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f658c004d90 con 0x557ead94d790 2024-03-21T10:26:51.304 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:51.302+0000 7f6573e00640 1 -- 172.21.15.47:0/897616344 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(97..97 src has 1..97) v4 ==== 9474+0+0 (secure 0 0 0) 0x7f658c003970 con 0x557ead94d790 2024-03-21T10:26:51.304 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:51.302+0000 7f659c439740 1 -- 172.21.15.47:0/897616344 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f6580026030 con 0x557ead94d790 2024-03-21T10:26:51.304 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:51.302+0000 7f6573e00640 1 -- 172.21.15.47:0/897616344 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f658c0be1d0 con 0x557ead94d790 2024-03-21T10:26:51.304 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:51.302+0000 7f659c439740 10 client.16305 fetch_fsmap learned FSMap version 32 2024-03-21T10:26:51.304 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:51.302+0000 7f659c439740 1 -- 172.21.15.47:0/897616344 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x557ead9ba330 con 0x557ead94d790 2024-03-21T10:26:51.305 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:51.302+0000 7f6573e00640 1 -- 172.21.15.47:0/897616344 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f6580026030 con 0x557ead94d790 2024-03-21T10:26:51.305 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:51.303+0000 7f6573e00640 1 -- 172.21.15.47:0/897616344 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 32) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f658c0b9f20 con 0x557ead94d790 2024-03-21T10:26:51.305 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:51.303+0000 7f659c439740 10 client.16305 fetch_fsmap finished waiting for FSMap version 32 2024-03-21T10:26:51.305 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:51.303+0000 7f659c439740 10 client.16305 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-21T10:26:51.305 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:51.303+0000 7f659c439740 20 client.16305 populate_metadata read hostname 'smithi047' 2024-03-21T10:26:51.305 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:51.303+0000 7f659c439740 1 --2- 172.21.15.47:0/897616344 >> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] conn(0x557ead9ba890 0x557ead9bccd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:26:51.305 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:51.303+0000 7f659c439740 4 client.16305 mds_command: new command op to 34535 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-03-21T10:26:51.305 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:51.303+0000 7f659c439740 1 -- 172.21.15.47:0/897616344 --> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x557ead8b0740 con 0x557ead9ba890 2024-03-21T10:26:51.306 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:51.303+0000 7f6592a00640 1 --2- 172.21.15.47:0/897616344 >> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] conn(0x557ead9ba890 0x557ead9bccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:26:51.306 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:51.304+0000 7f6592a00640 1 --2- 172.21.15.47:0/897616344 >> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] conn(0x557ead9ba890 0x557ead9bccd0 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=131d99e5ac4f4219 server_cookie=574b1ee42c873bb1 in_seq=0 out_seq=0 2024-03-21T10:26:51.306 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:51.304+0000 7f6573e00640 10 client.16305 ms_handle_connect on v2:172.21.15.100:6838/4173168440 2024-03-21T10:26:51.308 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:51.307+0000 7f6573e00640 1 -- 172.21.15.47:0/897616344 <== mds.0 v2:172.21.15.100:6838/4173168440 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x557ead8b0740 con 0x557ead9ba890 2024-03-21T10:26:51.308 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:51.307+0000 7f6573e00640 10 client.16305 handle_command_reply: tid=0 2024-03-21T10:26:51.308 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:51.307+0000 7f659c439740 1 client.16305 shutdown 2024-03-21T10:26:51.308 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:51.307+0000 7f659c439740 1 -- 172.21.15.47:0/897616344 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x557ead94d790 msgr2=0x557ead9533a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:26:51.308 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:51.307+0000 7f659c439740 1 --2- 172.21.15.47:0/897616344 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x557ead94d790 0x557ead9533a0 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7f658c034fd0 tx=0x7f658c0de2e0 comp rx=0 tx=0).stop 2024-03-21T10:26:51.308 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:51.308+0000 7f659c439740 1 -- 172.21.15.47:0/897616344 shutdown_connections 2024-03-21T10:26:51.308 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:51.308+0000 7f659c439740 1 --2- 172.21.15.47:0/897616344 >> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] conn(0x557ead9ba890 0x557ead9bccd0 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:26:51.308 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:51.308+0000 7f659c439740 1 --2- 172.21.15.47:0/897616344 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x557ead94d790 0x557ead9533a0 unknown :-1 s=CLOSED pgs=661 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:26:51.309 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:51.308+0000 7f659c439740 1 --2- 172.21.15.47:0/897616344 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557ead944bb0 0x557ead952e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:26:51.309 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:51.308+0000 7f659c439740 1 --2- 172.21.15.47:0/897616344 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557ead944290 0x557ead952920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:26:51.309 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:51.308+0000 7f659c439740 1 -- 172.21.15.47:0/897616344 >> 172.21.15.47:0/897616344 conn(0x557ead93e9c0 msgr2=0x557ead940410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:26:51.309 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:51.308+0000 7f6573e00640 0 client.16305 ms_handle_reset on v2:172.21.15.100:6838/4173168440 2024-03-21T10:26:51.309 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:51.308+0000 7f659c439740 1 -- 172.21.15.47:0/897616344 shutdown_connections 2024-03-21T10:26:51.309 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:51.308+0000 7f659c439740 1 -- 172.21.15.47:0/897616344 wait complete. 2024-03-21T10:26:51.309 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:51.308+0000 7f659c439740 20 client.16305 trim_cache size 0 max 16384 2024-03-21T10:26:51.309 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-03-21T10:26:51.309 INFO:teuthology.orchestra.run.smithi047.stdout: "return_code": 0, 2024-03-21T10:26:51.309 INFO:teuthology.orchestra.run.smithi047.stdout: "scrub_tag": "f5bf1b30-ae33-4b25-9bcc-66a784b8a87c", 2024-03-21T10:26:51.310 INFO:teuthology.orchestra.run.smithi047.stdout: "mode": "asynchronous" 2024-03-21T10:26:51.310 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-03-21T10:26:51.310 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:51.309+0000 7f6593e00640 1 -- 172.21.15.47:0/1025440226 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f65780a31b0 msgr2=0x7f65780a5670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:26:51.310 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:51.309+0000 7f6593e00640 1 --2- 172.21.15.47:0/1025440226 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f65780a31b0 0x7f65780a5670 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x557ead93fb50 tx=0x7f6580009550 comp rx=0 tx=0).stop 2024-03-21T10:26:51.310 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:51.309+0000 7f6593e00640 1 -- 172.21.15.47:0/1025440226 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f65941a2b70 msgr2=0x7f65941a3020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:26:51.311 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:51.309+0000 7f6593e00640 1 --2- 172.21.15.47:0/1025440226 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f65941a2b70 0x7f65941a3020 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7f658c009d00 tx=0x7f658c02cb30 comp rx=0 tx=0).stop 2024-03-21T10:26:51.311 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:51.309+0000 7f6593e00640 1 -- 172.21.15.47:0/1025440226 shutdown_connections 2024-03-21T10:26:51.311 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:51.309+0000 7f6593e00640 1 --2- 172.21.15.47:0/1025440226 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f65941a2b70 0x7f65941a3020 unknown :-1 s=CLOSED pgs=657 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:26:51.311 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:51.309+0000 7f6593e00640 1 --2- 172.21.15.47:0/1025440226 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f659410cea0 0x7f659419e840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:26:51.311 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:51.309+0000 7f6593e00640 1 --2- 172.21.15.47:0/1025440226 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f65780a31b0 0x7f65780a5670 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:26:51.311 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:51.309+0000 7f6593e00640 1 --2- 172.21.15.47:0/1025440226 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6594107e50 0x7f659419e300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:26:51.311 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:51.309+0000 7f6593e00640 1 -- 172.21.15.47:0/1025440226 >> 172.21.15.47:0/1025440226 conn(0x7f65940774c0 msgr2=0x7f6594079080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:26:51.311 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:51.309+0000 7f6593e00640 1 -- 172.21.15.47:0/1025440226 shutdown_connections 2024-03-21T10:26:51.311 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:51.309+0000 7f6593e00640 1 -- 172.21.15.47:0/1025440226 wait complete. 2024-03-21T10:26:51.327 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-21T10:26:51.327 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-21T10:26:51.388 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:26:51 smithi047 ceph-mon[31547]: pgmap v49: 129 pgs: 129 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-03-21T10:26:51.515 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:51.514+0000 7f17b0600640 1 -- 172.21.15.47:0/279066925 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f17a806a590 msgr2=0x7f17a810e520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:26:51.515 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:51.514+0000 7f17b0600640 1 --2- 172.21.15.47:0/279066925 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f17a806a590 0x7f17a810e520 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7f17a0009cd0 tx=0x7f17a002c0f0 comp rx=0 tx=0).stop 2024-03-21T10:26:51.515 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:51.514+0000 7f17b0600640 1 -- 172.21.15.47:0/279066925 shutdown_connections 2024-03-21T10:26:51.515 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:51.514+0000 7f17b0600640 1 --2- 172.21.15.47:0/279066925 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f17a806a590 0x7f17a810e520 unknown :-1 s=CLOSED pgs=662 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:26:51.515 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:51.514+0000 7f17b0600640 1 --2- 172.21.15.47:0/279066925 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f17a8069c10 0x7f17a806a050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:26:51.515 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:51.514+0000 7f17b0600640 1 --2- 172.21.15.47:0/279066925 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f17a8107660 0x7f17a8107a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:26:51.515 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:51.514+0000 7f17b0600640 1 -- 172.21.15.47:0/279066925 >> 172.21.15.47:0/279066925 conn(0x7f17a8077580 msgr2=0x7f17a8077990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:26:51.515 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:51.514+0000 7f17b0600640 1 -- 172.21.15.47:0/279066925 shutdown_connections 2024-03-21T10:26:51.515 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:51.514+0000 7f17b0600640 1 -- 172.21.15.47:0/279066925 wait complete. 2024-03-21T10:26:51.517 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:51.516+0000 7f17b0600640 1 Processor -- start 2024-03-21T10:26:51.517 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:51.516+0000 7f17b0600640 1 -- start start 2024-03-21T10:26:51.517 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:51.516+0000 7f17b0600640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f17a8069c10 0x7f17a819a0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:26:51.517 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:51.516+0000 7f17b0600640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f17a806a590 0x7f17a819a610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:26:51.517 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:51.516+0000 7f17b0600640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f17a8107660 0x7f17a819e8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:26:51.517 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:51.516+0000 7f17b0600640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17a8111910 con 0x7f17a806a590 2024-03-21T10:26:51.517 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:51.516+0000 7f17b0600640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17a8111790 con 0x7f17a8069c10 2024-03-21T10:26:51.517 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:51.516+0000 7f17b0600640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17a8111a90 con 0x7f17a8107660 2024-03-21T10:26:51.517 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:51.516+0000 7f17ad200640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f17a806a590 0x7f17a819a610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:26:51.518 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:51.517+0000 7f17ad200640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f17a806a590 0x7f17a819a610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:58266/0 (socket says 172.21.15.47:58266) 2024-03-21T10:26:51.518 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:51.517+0000 7f17ad200640 1 -- 172.21.15.47:0/477920246 learned_addr learned my addr 172.21.15.47:0/477920246 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:26:51.518 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:51.517+0000 7f17adc00640 1 --2- 172.21.15.47:0/477920246 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f17a8069c10 0x7f17a819a0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:26:51.518 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:51.517+0000 7f17ae600640 1 --2- 172.21.15.47:0/477920246 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f17a8107660 0x7f17a819e8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:26:51.518 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:51.517+0000 7f17ad200640 1 -- 172.21.15.47:0/477920246 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f17a8107660 msgr2=0x7f17a819e8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:26:51.518 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:51.517+0000 7f17ad200640 1 --2- 172.21.15.47:0/477920246 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f17a8107660 0x7f17a819e8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:26:51.518 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:51.517+0000 7f17ad200640 1 -- 172.21.15.47:0/477920246 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f17a8069c10 msgr2=0x7f17a819a0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:26:51.518 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:51.517+0000 7f17ad200640 1 --2- 172.21.15.47:0/477920246 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f17a8069c10 0x7f17a819a0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:26:51.518 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:51.517+0000 7f17ad200640 1 -- 172.21.15.47:0/477920246 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f17a819f020 con 0x7f17a806a590 2024-03-21T10:26:51.518 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:51.517+0000 7f17ae600640 1 --2- 172.21.15.47:0/477920246 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f17a8107660 0x7f17a819e8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T10:26:51.519 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:51.517+0000 7f17ad200640 1 --2- 172.21.15.47:0/477920246 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f17a806a590 0x7f17a819a610 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7f179400d770 tx=0x7f179400dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:26:51.519 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:51.518+0000 7f17a6000640 1 -- 172.21.15.47:0/477920246 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1794017070 con 0x7f17a806a590 2024-03-21T10:26:51.519 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:51.518+0000 7f17a6000640 1 -- 172.21.15.47:0/477920246 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f17940054d0 con 0x7f17a806a590 2024-03-21T10:26:51.519 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:51.518+0000 7f17b0600640 1 -- 172.21.15.47:0/477920246 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f17a819f310 con 0x7f17a806a590 2024-03-21T10:26:51.519 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:51.518+0000 7f17a6000640 1 -- 172.21.15.47:0/477920246 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1794004cf0 con 0x7f17a806a590 2024-03-21T10:26:51.519 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:51.518+0000 7f17b0600640 1 -- 172.21.15.47:0/477920246 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f17a81a6ba0 con 0x7f17a806a590 2024-03-21T10:26:51.522 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:51.522+0000 7f17a6000640 1 -- 172.21.15.47:0/477920246 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 26) v1 ==== 138087+0+0 (secure 0 0 0) 0x7f1794004e90 con 0x7f17a806a590 2024-03-21T10:26:51.522 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:51.522+0000 7f17b3242740 1 Processor -- start 2024-03-21T10:26:51.523 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:51.523+0000 7f17b3242740 1 -- start start 2024-03-21T10:26:51.523 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:51.523+0000 7f17b3242740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d0e760bc00 0x55d0e760a950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:26:51.523 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:51.523+0000 7f17b3242740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d0e760af20 0x55d0e760b3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:26:51.523 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:51.523+0000 7f17b3242740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d0e7614290 0x55d0e7616680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:26:51.523 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:51.523+0000 7f17b3242740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d0e756f250 con 0x55d0e7614290 2024-03-21T10:26:51.523 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:51.523+0000 7f17b3242740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d0e7517a30 con 0x55d0e760af20 2024-03-21T10:26:51.523 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:51.523+0000 7f17b3242740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d0e756b2f0 con 0x55d0e760bc00 2024-03-21T10:26:51.525 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:51.524+0000 7f17ad200640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d0e7614290 0x55d0e7616680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:26:51.525 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:51.524+0000 7f17adc00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d0e760af20 0x55d0e760b3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:26:51.525 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:51.524+0000 7f17adc00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d0e760af20 0x55d0e760b3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:52560/0 (socket says 172.21.15.47:52560) 2024-03-21T10:26:51.525 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:51.524+0000 7f17adc00640 1 -- 172.21.15.47:0/1096647608 learned_addr learned my addr 172.21.15.47:0/1096647608 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:26:51.525 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:51.524+0000 7f17a6000640 1 --2- 172.21.15.47:0/477920246 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f17900a3230 0x7f17900a56f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:26:51.525 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:51.524+0000 7f17ae600640 1 --2- 172.21.15.47:0/1096647608 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d0e760bc00 0x55d0e760a950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:26:51.525 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:51.524+0000 7f17a6000640 1 -- 172.21.15.47:0/477920246 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 9474+0+0 (secure 0 0 0) 0x7f17940cf8a0 con 0x7f17a806a590 2024-03-21T10:26:51.525 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:51.524+0000 7f17adc00640 1 -- 172.21.15.47:0/1096647608 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d0e760bc00 msgr2=0x55d0e760a950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:26:51.525 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:51.524+0000 7f17adc00640 1 --2- 172.21.15.47:0/1096647608 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d0e760bc00 0x55d0e760a950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:26:51.526 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:51.524+0000 7f17adc00640 1 -- 172.21.15.47:0/1096647608 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d0e7614290 msgr2=0x55d0e7616680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:26:51.526 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:51.524+0000 7f17adc00640 1 --2- 172.21.15.47:0/1096647608 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d0e7614290 0x55d0e7616680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:26:51.526 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:51.524+0000 7f17adc00640 1 -- 172.21.15.47:0/1096647608 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d0e7502690 con 0x55d0e760af20 2024-03-21T10:26:51.526 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:51.524+0000 7f17ae600640 1 --2- 172.21.15.47:0/1096647608 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d0e760bc00 0x55d0e760a950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T10:26:51.526 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:51.524+0000 7f17ae600640 1 --2- 172.21.15.47:0/477920246 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f17900a3230 0x7f17900a56f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:26:51.526 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:51.525+0000 7f17adc00640 1 --2- 172.21.15.47:0/1096647608 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d0e760af20 0x55d0e760b3a0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f179c009c70 tx=0x7f179c011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c7e046710ae38d2d server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:26:51.526 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:51.525+0000 7f179aa00640 1 -- 172.21.15.47:0/1096647608 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f179c004230 con 0x55d0e760af20 2024-03-21T10:26:51.526 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:51.525+0000 7f179aa00640 1 -- 172.21.15.47:0/1096647608 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f179c0043d0 con 0x55d0e760af20 2024-03-21T10:26:51.526 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:51.525+0000 7f179aa00640 1 -- 172.21.15.47:0/1096647608 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f179c0196d0 con 0x55d0e760af20 2024-03-21T10:26:51.527 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:51.526+0000 7f17b3242740 1 -- 172.21.15.47:0/1096647608 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d0e760af20 msgr2=0x55d0e760b3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:26:51.527 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:51.526+0000 7f17b3242740 1 --2- 172.21.15.47:0/1096647608 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d0e760af20 0x55d0e760b3a0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f179c009c70 tx=0x7f179c011810 comp rx=0 tx=0).stop 2024-03-21T10:26:51.527 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:51.526+0000 7f17ae600640 1 --2- 172.21.15.47:0/477920246 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f17900a3230 0x7f17900a56f0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f17a0005e30 tx=0x7f17a00095c0 comp rx=0 tx=0).ready entity=mgr.15687 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:26:51.527 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:51.526+0000 7f17b3242740 1 -- 172.21.15.47:0/1096647608 shutdown_connections 2024-03-21T10:26:51.527 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:51.526+0000 7f17b3242740 1 --2- 172.21.15.47:0/1096647608 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d0e7614290 0x55d0e7616680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:26:51.527 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:51.526+0000 7f17b3242740 1 --2- 172.21.15.47:0/1096647608 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d0e760af20 0x55d0e760b3a0 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:26:51.528 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:51.526+0000 7f17b3242740 1 --2- 172.21.15.47:0/1096647608 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d0e760bc00 0x55d0e760a950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:26:51.528 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:51.526+0000 7f17b3242740 1 -- 172.21.15.47:0/1096647608 >> 172.21.15.47:0/1096647608 conn(0x55d0e75d2200 msgr2=0x55d0e7605970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:26:51.528 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:51.526+0000 7f17b3242740 1 -- 172.21.15.47:0/1096647608 shutdown_connections 2024-03-21T10:26:51.528 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:51.526+0000 7f17b3242740 1 -- 172.21.15.47:0/1096647608 wait complete. 2024-03-21T10:26:51.528 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:51.527+0000 7f17b3242740 1 -- start start 2024-03-21T10:26:51.528 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:51.527+0000 7f17b3242740 1 Processor -- start 2024-03-21T10:26:51.532 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:51.528+0000 7f17b3242740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d0e760af20 0x55d0e7613830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:26:51.532 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:51.528+0000 7f17b3242740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d0e760bc00 0x55d0e7610bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:26:51.532 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:51.528+0000 7f17b3242740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d0e7614290 0x55d0e7611100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:26:51.532 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:51.528+0000 7f17b3242740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d0e7617100 con 0x55d0e760bc00 2024-03-21T10:26:51.532 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:51.528+0000 7f17b3242740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d0e752ae80 con 0x55d0e760af20 2024-03-21T10:26:51.532 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:51.528+0000 7f17b3242740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d0e7617280 con 0x55d0e7614290 2024-03-21T10:26:51.532 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:51.528+0000 7f17b3242740 10 client.0 _ll_register_callbacks cb 0x55d0e753d130 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T10:26:51.532 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:51.528+0000 7f17ad200640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d0e760bc00 0x55d0e7610bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:26:51.532 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:51.528+0000 7f17ad200640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d0e760bc00 0x55d0e7610bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:58286/0 (socket says 172.21.15.47:58286) 2024-03-21T10:26:51.532 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:51.528+0000 7f17ad200640 1 -- 172.21.15.47:0/3577797640 learned_addr learned my addr 172.21.15.47:0/3577797640 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:26:51.532 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:51.528+0000 7f17adc00640 1 --2- 172.21.15.47:0/3577797640 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d0e760af20 0x55d0e7613830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:26:51.533 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:51.529+0000 7f17ae600640 1 --2- 172.21.15.47:0/3577797640 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d0e7614290 0x55d0e7611100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:26:51.533 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:51.529+0000 7f17ad200640 1 -- 172.21.15.47:0/3577797640 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d0e7614290 msgr2=0x55d0e7611100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:26:51.533 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:51.529+0000 7f17ad200640 1 --2- 172.21.15.47:0/3577797640 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d0e7614290 0x55d0e7611100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:26:51.533 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:51.529+0000 7f17ad200640 1 -- 172.21.15.47:0/3577797640 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d0e760af20 msgr2=0x55d0e7613830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:26:51.533 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:51.529+0000 7f17ad200640 1 --2- 172.21.15.47:0/3577797640 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d0e760af20 0x55d0e7613830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:26:51.533 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:51.529+0000 7f17ad200640 1 -- 172.21.15.47:0/3577797640 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55d0e734d7d0 con 0x55d0e760bc00 2024-03-21T10:26:51.533 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:51.529+0000 7f17ae600640 1 --2- 172.21.15.47:0/3577797640 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d0e7614290 0x55d0e7611100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T10:26:51.533 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:51.529+0000 7f17ad200640 1 --2- 172.21.15.47:0/3577797640 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d0e760bc00 0x55d0e7610bc0 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7f179400d740 tx=0x7f179400df40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8e44927836c81c29 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:26:51.533 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:51.529+0000 7f178f400640 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-03-21T10:26:51.533 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:51.529+0000 7f178f400640 1 -- 172.21.15.47:0/3577797640 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1794020040 con 0x55d0e760bc00 2024-03-21T10:26:51.533 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:51.529+0000 7f178f400640 1 -- 172.21.15.47:0/3577797640 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(97..97 src has 1..97) v4 ==== 9474+0+0 (secure 0 0 0) 0x7f17940cf620 con 0x55d0e760bc00 2024-03-21T10:26:51.534 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:51.529+0000 7f17b3242740 1 -- 172.21.15.47:0/3577797640 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f179c0043d0 con 0x55d0e760bc00 2024-03-21T10:26:51.534 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:51.530+0000 7f178f400640 1 -- 172.21.15.47:0/3577797640 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f17940d2e60 con 0x55d0e760bc00 2024-03-21T10:26:51.534 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:51.530+0000 7f17b3242740 10 client.16317 fetch_fsmap learned FSMap version 32 2024-03-21T10:26:51.534 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:51.530+0000 7f17b3242740 1 -- 172.21.15.47:0/3577797640 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x55d0e72833f0 con 0x55d0e760bc00 2024-03-21T10:26:51.534 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:51.530+0000 7f178f400640 1 -- 172.21.15.47:0/3577797640 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f179c0043d0 con 0x55d0e760bc00 2024-03-21T10:26:51.534 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:51.530+0000 7f178f400640 1 -- 172.21.15.47:0/3577797640 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 32) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f1794084b60 con 0x55d0e760bc00 2024-03-21T10:26:51.534 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:51.530+0000 7f17b3242740 10 client.16317 fetch_fsmap finished waiting for FSMap version 32 2024-03-21T10:26:51.534 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:51.530+0000 7f17b3242740 10 client.16317 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-21T10:26:51.534 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:51.530+0000 7f17b3242740 20 client.16317 populate_metadata read hostname 'smithi047' 2024-03-21T10:26:51.534 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:51.530+0000 7f17b3242740 1 --2- 172.21.15.47:0/3577797640 >> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] conn(0x55d0e7624100 0x55d0e76264f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:26:51.535 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:51.530+0000 7f17b3242740 4 client.16317 mds_command: new command op to 34535 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T10:26:51.535 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:51.530+0000 7f17b3242740 1 -- 172.21.15.47:0/3577797640 --> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55d0e757c790 con 0x55d0e7624100 2024-03-21T10:26:51.535 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:51.531+0000 7f17adc00640 1 --2- 172.21.15.47:0/3577797640 >> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] conn(0x55d0e7624100 0x55d0e76264f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:26:51.535 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:51.532+0000 7f17adc00640 1 --2- 172.21.15.47:0/3577797640 >> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] conn(0x55d0e7624100 0x55d0e76264f0 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5dbf4bbcfd0f0264 server_cookie=59dcc6dcf395f5a8 in_seq=0 out_seq=0 2024-03-21T10:26:51.535 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:51.532+0000 7f178f400640 10 client.16317 ms_handle_connect on v2:172.21.15.100:6838/4173168440 2024-03-21T10:26:51.536 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:51.533+0000 7f178f400640 1 -- 172.21.15.47:0/3577797640 <== mds.0 v2:172.21.15.100:6838/4173168440 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55d0e757c790 con 0x55d0e7624100 2024-03-21T10:26:51.536 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:51.533+0000 7f178f400640 10 client.16317 handle_command_reply: tid=0 2024-03-21T10:26:51.537 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:51.533+0000 7f17b3242740 1 client.16317 shutdown 2024-03-21T10:26:51.537 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:51.533+0000 7f17b3242740 1 -- 172.21.15.47:0/3577797640 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d0e760bc00 msgr2=0x55d0e7610bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:26:51.537 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:51.533+0000 7f17b3242740 1 --2- 172.21.15.47:0/3577797640 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d0e760bc00 0x55d0e7610bc0 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7f179400d740 tx=0x7f179400df40 comp rx=0 tx=0).stop 2024-03-21T10:26:51.537 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:51.533+0000 7f17b3242740 1 -- 172.21.15.47:0/3577797640 shutdown_connections 2024-03-21T10:26:51.537 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:51.533+0000 7f17b3242740 1 --2- 172.21.15.47:0/3577797640 >> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] conn(0x55d0e7624100 0x55d0e76264f0 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:26:51.537 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:51.533+0000 7f17b3242740 1 --2- 172.21.15.47:0/3577797640 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d0e7614290 0x55d0e7611100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:26:51.537 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:51.533+0000 7f17b3242740 1 --2- 172.21.15.47:0/3577797640 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d0e760bc00 0x55d0e7610bc0 unknown :-1 s=CLOSED pgs=664 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:26:51.537 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:51.533+0000 7f17b3242740 1 --2- 172.21.15.47:0/3577797640 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d0e760af20 0x55d0e7613830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:26:51.537 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:51.533+0000 7f17b3242740 1 -- 172.21.15.47:0/3577797640 >> 172.21.15.47:0/3577797640 conn(0x55d0e75d2200 msgr2=0x55d0e76076c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:26:51.537 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:51.533+0000 7f178f400640 0 client.16317 ms_handle_reset on v2:172.21.15.100:6838/4173168440 2024-03-21T10:26:51.537 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:51.533+0000 7f17b3242740 1 -- 172.21.15.47:0/3577797640 shutdown_connections 2024-03-21T10:26:51.537 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:51.533+0000 7f17b3242740 1 -- 172.21.15.47:0/3577797640 wait complete. 2024-03-21T10:26:51.538 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:51.533+0000 7f17b3242740 20 client.16317 trim_cache size 0 max 16384 2024-03-21T10:26:51.552 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:51.551+0000 7f17b3242740 1 Processor -- start 2024-03-21T10:26:51.552 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:51.551+0000 7f17b3242740 1 -- start start 2024-03-21T10:26:51.552 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:51.551+0000 7f17b3242740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d0e75d2200 0x55d0e760a560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:26:51.552 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:51.551+0000 7f17b3242740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d0e760af20 0x55d0e7615680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:26:51.553 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:51.551+0000 7f17b3242740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d0e760bc00 0x55d0e7613280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:26:51.553 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:51.551+0000 7f17b3242740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17a81b0910 con 0x55d0e75d2200 2024-03-21T10:26:51.553 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:51.551+0000 7f17b3242740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17a81b2290 con 0x55d0e760af20 2024-03-21T10:26:51.553 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:51.551+0000 7f17b3242740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17a81b0f70 con 0x55d0e760bc00 2024-03-21T10:26:51.553 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:51.552+0000 7f17adc00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d0e75d2200 0x55d0e760a560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:26:51.553 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:51.552+0000 7f17adc00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d0e75d2200 0x55d0e760a560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:58290/0 (socket says 172.21.15.47:58290) 2024-03-21T10:26:51.553 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:51.552+0000 7f17adc00640 1 -- 172.21.15.47:0/1070323496 learned_addr learned my addr 172.21.15.47:0/1070323496 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:26:51.553 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:51.552+0000 7f17ad200640 1 --2- 172.21.15.47:0/1070323496 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d0e760af20 0x55d0e7615680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:26:51.553 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:51.552+0000 7f17ae600640 1 --2- 172.21.15.47:0/1070323496 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d0e760bc00 0x55d0e7613280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:26:51.553 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:51.552+0000 7f17adc00640 1 -- 172.21.15.47:0/1070323496 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d0e760bc00 msgr2=0x55d0e7613280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:26:51.554 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:51.552+0000 7f17adc00640 1 --2- 172.21.15.47:0/1070323496 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d0e760bc00 0x55d0e7613280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:26:51.554 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:51.552+0000 7f17adc00640 1 -- 172.21.15.47:0/1070323496 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d0e760af20 msgr2=0x55d0e7615680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:26:51.554 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:51.552+0000 7f17adc00640 1 --2- 172.21.15.47:0/1070323496 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d0e760af20 0x55d0e7615680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:26:51.554 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:51.552+0000 7f17adc00640 1 -- 172.21.15.47:0/1070323496 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d0e72430f0 con 0x55d0e75d2200 2024-03-21T10:26:51.554 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:51.552+0000 7f17ae600640 1 --2- 172.21.15.47:0/1070323496 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d0e760bc00 0x55d0e7613280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T10:26:51.554 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:51.552+0000 7f17adc00640 1 --2- 172.21.15.47:0/1070323496 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d0e75d2200 0x55d0e760a560 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7f179c009c70 tx=0x7f179c009bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=22f4b0fcd14d05a4 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:26:51.558 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:51.553+0000 7f179a000640 1 -- 172.21.15.47:0/1070323496 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f179c031070 con 0x55d0e75d2200 2024-03-21T10:26:51.558 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:51.553+0000 7f179a000640 1 -- 172.21.15.47:0/1070323496 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f179c009e20 con 0x55d0e75d2200 2024-03-21T10:26:51.558 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:51.553+0000 7f179a000640 1 -- 172.21.15.47:0/1070323496 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f179c003920 con 0x55d0e75d2200 2024-03-21T10:26:51.559 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:51.554+0000 7f17b3242740 1 -- 172.21.15.47:0/1070323496 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d0e75d2200 msgr2=0x55d0e760a560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:26:51.559 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:51.554+0000 7f17b3242740 1 --2- 172.21.15.47:0/1070323496 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d0e75d2200 0x55d0e760a560 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7f179c009c70 tx=0x7f179c009bf0 comp rx=0 tx=0).stop 2024-03-21T10:26:51.559 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:51.554+0000 7f17b3242740 1 -- 172.21.15.47:0/1070323496 shutdown_connections 2024-03-21T10:26:51.559 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:51.554+0000 7f17b3242740 1 --2- 172.21.15.47:0/1070323496 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d0e760bc00 0x55d0e7613280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:26:51.559 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:51.554+0000 7f17b3242740 1 --2- 172.21.15.47:0/1070323496 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d0e760af20 0x55d0e7615680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:26:51.559 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:51.554+0000 7f17b3242740 1 --2- 172.21.15.47:0/1070323496 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d0e75d2200 0x55d0e760a560 unknown :-1 s=CLOSED pgs=665 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:26:51.559 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:51.554+0000 7f17b3242740 1 -- 172.21.15.47:0/1070323496 >> 172.21.15.47:0/1070323496 conn(0x55d0e7611260 msgr2=0x55d0e76054a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:26:51.559 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:51.554+0000 7f17b3242740 1 -- 172.21.15.47:0/1070323496 shutdown_connections 2024-03-21T10:26:51.559 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:51.554+0000 7f17b3242740 1 -- 172.21.15.47:0/1070323496 wait complete. 2024-03-21T10:26:51.560 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:51.554+0000 7f17b3242740 1 -- start start 2024-03-21T10:26:51.560 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:51.555+0000 7f17b3242740 1 Processor -- start 2024-03-21T10:26:51.560 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:51.555+0000 7f17b3242740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d0e75d2200 0x55d0e7617b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:26:51.560 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:51.555+0000 7f17b3242740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d0e760af20 0x55d0e7618050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:26:51.560 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:51.555+0000 7f17b3242740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d0e760bc00 0x55d0e7618590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:26:51.560 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:51.555+0000 7f17b3242740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17a81b1c30 con 0x55d0e760af20 2024-03-21T10:26:51.560 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:51.555+0000 7f17b3242740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17a81b15d0 con 0x55d0e75d2200 2024-03-21T10:26:51.560 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:51.555+0000 7f17b3242740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17a81aead0 con 0x55d0e760bc00 2024-03-21T10:26:51.560 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:51.555+0000 7f17b3242740 10 client.0 _ll_register_callbacks cb 0x7f1794097200 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T10:26:51.560 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:51.555+0000 7f17ad200640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d0e760af20 0x55d0e7618050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:26:51.560 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:51.556+0000 7f17ad200640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d0e760af20 0x55d0e7618050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:58304/0 (socket says 172.21.15.47:58304) 2024-03-21T10:26:51.561 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:51.556+0000 7f17ad200640 1 -- 172.21.15.47:0/376412797 learned_addr learned my addr 172.21.15.47:0/376412797 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:26:51.561 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:51.556+0000 7f17adc00640 1 --2- 172.21.15.47:0/376412797 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d0e75d2200 0x55d0e7617b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:26:51.561 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:51.556+0000 7f17ae600640 1 --2- 172.21.15.47:0/376412797 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d0e760bc00 0x55d0e7618590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:26:51.561 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:51.556+0000 7f17ad200640 1 -- 172.21.15.47:0/376412797 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d0e760bc00 msgr2=0x55d0e7618590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:26:51.561 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:51.556+0000 7f17ad200640 1 --2- 172.21.15.47:0/376412797 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d0e760bc00 0x55d0e7618590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:26:51.561 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:51.556+0000 7f17ad200640 1 -- 172.21.15.47:0/376412797 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d0e75d2200 msgr2=0x55d0e7617b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:26:51.561 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:51.556+0000 7f17ad200640 1 --2- 172.21.15.47:0/376412797 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d0e75d2200 0x55d0e7617b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:26:51.561 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:51.556+0000 7f17ad200640 1 -- 172.21.15.47:0/376412797 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55d0e767eb10 con 0x55d0e760af20 2024-03-21T10:26:51.561 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:51.556+0000 7f17ad200640 1 --2- 172.21.15.47:0/376412797 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d0e760af20 0x55d0e7618050 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7f17940cfc60 tx=0x7f17940cf650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a05caac6462f60ea server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:26:51.561 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:51.556+0000 7f178f400640 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-03-21T10:26:51.561 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:51.557+0000 7f178f400640 1 -- 172.21.15.47:0/376412797 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1794014030 con 0x55d0e760af20 2024-03-21T10:26:51.561 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:51.557+0000 7f178f400640 1 -- 172.21.15.47:0/376412797 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(97..97 src has 1..97) v4 ==== 9474+0+0 (secure 0 0 0) 0x7f1794015070 con 0x55d0e760af20 2024-03-21T10:26:51.562 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:51.557+0000 7f17b3242740 1 -- 172.21.15.47:0/376412797 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f179c009e20 con 0x55d0e760af20 2024-03-21T10:26:51.562 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:51.557+0000 7f178f400640 1 -- 172.21.15.47:0/376412797 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1794082050 con 0x55d0e760af20 2024-03-21T10:26:51.562 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:51.557+0000 7f17b3242740 10 client.16323 fetch_fsmap learned FSMap version 32 2024-03-21T10:26:51.562 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:51.557+0000 7f17b3242740 1 -- 172.21.15.47:0/376412797 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x55d0e767fd30 con 0x55d0e760af20 2024-03-21T10:26:51.562 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:51.557+0000 7f178f400640 1 -- 172.21.15.47:0/376412797 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f179c009e20 con 0x55d0e760af20 2024-03-21T10:26:51.562 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:51.557+0000 7f178f400640 1 -- 172.21.15.47:0/376412797 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 32) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f1794097370 con 0x55d0e760af20 2024-03-21T10:26:51.562 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:51.558+0000 7f17b3242740 10 client.16323 fetch_fsmap finished waiting for FSMap version 32 2024-03-21T10:26:51.562 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:51.558+0000 7f17b3242740 10 client.16323 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-21T10:26:51.562 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:51.558+0000 7f17b3242740 20 client.16323 populate_metadata read hostname 'smithi047' 2024-03-21T10:26:51.562 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:51.558+0000 7f17b3242740 1 --2- 172.21.15.47:0/376412797 >> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] conn(0x55d0e7614290 0x55d0e7682380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:26:51.562 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:51.558+0000 7f17b3242740 4 client.16323 mds_command: new command op to 34535 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-21T10:26:51.562 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:51.558+0000 7f17b3242740 1 -- 172.21.15.47:0/376412797 --> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55d0e7576ae0 con 0x55d0e7614290 2024-03-21T10:26:51.563 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:51.558+0000 7f17adc00640 1 --2- 172.21.15.47:0/376412797 >> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] conn(0x55d0e7614290 0x55d0e7682380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:26:51.564 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:51.559+0000 7f17adc00640 1 --2- 172.21.15.47:0/376412797 >> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] conn(0x55d0e7614290 0x55d0e7682380 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9c7ef869b5d478b6 server_cookie=4f11ef298892b633 in_seq=0 out_seq=0 2024-03-21T10:26:51.564 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:51.559+0000 7f178f400640 10 client.16323 ms_handle_connect on v2:172.21.15.100:6838/4173168440 2024-03-21T10:26:51.564 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:51.559+0000 7f178f400640 1 -- 172.21.15.47:0/376412797 <== mds.0 v2:172.21.15.100:6838/4173168440 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x55d0e7576ae0 con 0x55d0e7614290 2024-03-21T10:26:51.564 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:51.559+0000 7f178f400640 10 client.16323 handle_command_reply: tid=0 2024-03-21T10:26:51.564 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:51.559+0000 7f17b3242740 1 client.16323 shutdown 2024-03-21T10:26:51.564 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:51.559+0000 7f17b3242740 1 -- 172.21.15.47:0/376412797 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d0e760af20 msgr2=0x55d0e7618050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:26:51.564 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:51.559+0000 7f17b3242740 1 --2- 172.21.15.47:0/376412797 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d0e760af20 0x55d0e7618050 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7f17940cfc60 tx=0x7f17940cf650 comp rx=0 tx=0).stop 2024-03-21T10:26:51.564 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:51.559+0000 7f17b3242740 1 -- 172.21.15.47:0/376412797 shutdown_connections 2024-03-21T10:26:51.564 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:51.559+0000 7f17b3242740 1 --2- 172.21.15.47:0/376412797 >> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] conn(0x55d0e7614290 0x55d0e7682380 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:26:51.565 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:51.559+0000 7f17b3242740 1 --2- 172.21.15.47:0/376412797 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d0e760bc00 0x55d0e7618590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:26:51.565 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:51.559+0000 7f17b3242740 1 --2- 172.21.15.47:0/376412797 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d0e760af20 0x55d0e7618050 unknown :-1 s=CLOSED pgs=666 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:26:51.565 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:51.559+0000 7f17b3242740 1 --2- 172.21.15.47:0/376412797 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d0e75d2200 0x55d0e7617b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:26:51.565 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:51.559+0000 7f178f400640 0 client.16323 ms_handle_reset on v2:172.21.15.100:6838/4173168440 2024-03-21T10:26:51.565 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:51.559+0000 7f17b3242740 1 -- 172.21.15.47:0/376412797 >> 172.21.15.47:0/376412797 conn(0x55d0e7611260 msgr2=0x55d0e7616680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:26:51.565 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:51.560+0000 7f17b3242740 1 -- 172.21.15.47:0/376412797 shutdown_connections 2024-03-21T10:26:51.565 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:51.560+0000 7f17b3242740 1 -- 172.21.15.47:0/376412797 wait complete. 2024-03-21T10:26:51.565 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:51.560+0000 7f17b3242740 20 client.16323 trim_cache size 0 max 16384 2024-03-21T10:26:51.565 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-03-21T10:26:51.566 INFO:teuthology.orchestra.run.smithi047.stdout: "status": "IDLE (waiting for more scrubs)", 2024-03-21T10:26:51.566 INFO:teuthology.orchestra.run.smithi047.stdout: "scrubs": { 2024-03-21T10:26:51.566 INFO:teuthology.orchestra.run.smithi047.stdout: "f5bf1b30-ae33-4b25-9bcc-66a784b8a87c": { 2024-03-21T10:26:51.566 INFO:teuthology.orchestra.run.smithi047.stdout: "path": "/", 2024-03-21T10:26:51.566 INFO:teuthology.orchestra.run.smithi047.stdout: "tag": "f5bf1b30-ae33-4b25-9bcc-66a784b8a87c", 2024-03-21T10:26:51.566 INFO:teuthology.orchestra.run.smithi047.stdout: "options": "recursive,force" 2024-03-21T10:26:51.566 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-21T10:26:51.566 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-21T10:26:51.566 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-03-21T10:26:51.567 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:51.560+0000 7f17b0600640 1 -- 172.21.15.47:0/477920246 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f17900a3230 msgr2=0x7f17900a56f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:26:51.567 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:51.560+0000 7f17b0600640 1 --2- 172.21.15.47:0/477920246 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f17900a3230 0x7f17900a56f0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f17a0005e30 tx=0x7f17a00095c0 comp rx=0 tx=0).stop 2024-03-21T10:26:51.567 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:51.560+0000 7f17b0600640 1 -- 172.21.15.47:0/477920246 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f17a806a590 msgr2=0x7f17a819a610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:26:51.567 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:51.560+0000 7f17b0600640 1 --2- 172.21.15.47:0/477920246 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f17a806a590 0x7f17a819a610 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7f179400d770 tx=0x7f179400dc30 comp rx=0 tx=0).stop 2024-03-21T10:26:51.567 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:51.561+0000 7f17b0600640 1 -- 172.21.15.47:0/477920246 shutdown_connections 2024-03-21T10:26:51.567 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:51.561+0000 7f17b0600640 1 --2- 172.21.15.47:0/477920246 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f17a8107660 0x7f17a819e8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:26:51.568 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:51.561+0000 7f17b0600640 1 --2- 172.21.15.47:0/477920246 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f17a806a590 0x7f17a819a610 unknown :-1 s=CLOSED pgs=663 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:26:51.568 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:51.561+0000 7f17b0600640 1 --2- 172.21.15.47:0/477920246 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f17900a3230 0x7f17900a56f0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:26:51.568 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:51.561+0000 7f17b0600640 1 --2- 172.21.15.47:0/477920246 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f17a8069c10 0x7f17a819a0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:26:51.568 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:51.561+0000 7f17b0600640 1 -- 172.21.15.47:0/477920246 >> 172.21.15.47:0/477920246 conn(0x7f17a8077580 msgr2=0x7f17a810cdb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:26:51.568 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:51.561+0000 7f17b0600640 1 -- 172.21.15.47:0/477920246 shutdown_connections 2024-03-21T10:26:51.568 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:26:51.561+0000 7f17b0600640 1 -- 172.21.15.47:0/477920246 wait complete. 2024-03-21T10:26:51.578 INFO:tasks.cephfs.filesystem:scrub status for tag:f5bf1b30-ae33-4b25-9bcc-66a784b8a87c - {'path': '/', 'tag': 'f5bf1b30-ae33-4b25-9bcc-66a784b8a87c', 'options': 'recursive,force'} 2024-03-21T10:26:51.728 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:26:51 smithi100 ceph-mon[35193]: pgmap v49: 129 pgs: 129 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-03-21T10:26:51.738 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:26:51 smithi157 ceph-mon[35581]: pgmap v49: 129 pgs: 129 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-03-21T10:26:52.121 INFO:tasks.workunit.client.0.smithi047.stderr:Updating files: 54% (6747/12271) Updating files: 55% (6750/12271) Updating files: 56% (6872/12271) Updating files: 57% (6995/12271) Updating files: 58% (7118/12271) Updating files: 59% (7240/12271) Updating files: 60% (7363/12271) Updating files: 61% (7486/12271) Updating files: 62% (7609/12271) Updating files: 63% (7731/12271) Updating files: 64% (7854/12271) Updating files: 65% (7977/12271) Updating files: 66% (8099/12271) Updating files: 67% (8222/12271) Updating files: 68% (8345/12271) Updating files: 69% (8467/12271) Updating files: 70% (8590/12271) Updating files: 71% (8713/12271) Updating files: 72% (8836/12271) Updating files: 73% (8958/12271) Updating files: 74% (9081/12271) Updating files: 75% (9204/12271) Updating files: 76% (9326/12271) Updating files: 77% (9449/12271) Updating files: 78% (9572/12271) Updating files: 79% (9695/12271) Updating files: 80% (9817/12271) Updating files: 81% (9940/12271) Updating files: 82% (10063/12271) Updating files: 83% (10185/12271) Updating files: 84% (10308/12271) Updating files: 85% (10431/12271) Updating files: 86% (10554/12271) Updating files: 87% (10676/12271) Updating files: 87% (10729/12271) Updating files: 88% (10799/12271) Updating files: 89% (10922/12271) Updating files: 90% (11044/12271) Updating files: 91% (11167/12271) Updating files: 92% (11290/12271) Updating files: 93% (11413/12271) Updating files: 94% (11535/12271) Updating files: 95% (11658/12271) Updating files: 96% (11781/12271) Updating files: 97% (11903/12271) Updating files: 98% (12026/12271) Updating files: 99% (12149/12271) Updating files: 100% (12271/12271) Updating files: 100% (12271/12271), done. 2024-03-21T10:26:52.572 INFO:tasks.workunit.client.0.smithi047.stderr:Note: switching to '2edfee68ad7243bed422a049b6f5f58619300f11'. 2024-03-21T10:26:52.573 INFO:tasks.workunit.client.0.smithi047.stderr: 2024-03-21T10:26:52.573 INFO:tasks.workunit.client.0.smithi047.stderr:You are in 'detached HEAD' state. You can look around, make experimental 2024-03-21T10:26:52.573 INFO:tasks.workunit.client.0.smithi047.stderr:changes and commit them, and you can discard any commits you make in this 2024-03-21T10:26:52.573 INFO:tasks.workunit.client.0.smithi047.stderr:state without impacting any branches by switching back to a branch. 2024-03-21T10:26:52.573 INFO:tasks.workunit.client.0.smithi047.stderr: 2024-03-21T10:26:52.573 INFO:tasks.workunit.client.0.smithi047.stderr:If you want to create a new branch to retain commits you create, you may 2024-03-21T10:26:52.573 INFO:tasks.workunit.client.0.smithi047.stderr:do so (now or later) by using -c with the switch command. Example: 2024-03-21T10:26:52.573 INFO:tasks.workunit.client.0.smithi047.stderr: 2024-03-21T10:26:52.573 INFO:tasks.workunit.client.0.smithi047.stderr: git switch -c 2024-03-21T10:26:52.573 INFO:tasks.workunit.client.0.smithi047.stderr: 2024-03-21T10:26:52.573 INFO:tasks.workunit.client.0.smithi047.stderr:Or undo this operation with: 2024-03-21T10:26:52.573 INFO:tasks.workunit.client.0.smithi047.stderr: 2024-03-21T10:26:52.573 INFO:tasks.workunit.client.0.smithi047.stderr: git switch - 2024-03-21T10:26:52.573 INFO:tasks.workunit.client.0.smithi047.stderr: 2024-03-21T10:26:52.573 INFO:tasks.workunit.client.0.smithi047.stderr:Turn off this advice by setting config variable advice.detachedHead to false 2024-03-21T10:26:52.574 INFO:tasks.workunit.client.0.smithi047.stderr: 2024-03-21T10:26:52.574 INFO:tasks.workunit.client.0.smithi047.stderr:HEAD is now at 2edfee68ad7 Merge PR #55768 into wip-vshankar-testing-20240320.053002 2024-03-21T10:26:52.584 DEBUG:teuthology.orchestra.run.smithi047:> cd -- /home/ubuntu/cephtest/clone.client.0/qa/workunits && if test -e Makefile ; then make ; fi && find -executable -type f -printf '%P\0' >/home/ubuntu/cephtest/workunits.list.client.0 2024-03-21T10:26:52.713 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:26:52 smithi047 ceph-mon[31547]: scrub queued for path: / 2024-03-21T10:26:52.713 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:26:52 smithi047 ceph-mon[31547]: scrub summary: idle+waiting paths [/] 2024-03-21T10:26:52.713 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:26:52 smithi047 ceph-mon[31547]: scrub summary: active paths [/] 2024-03-21T10:26:52.713 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:26:52 smithi047 ceph-mon[31547]: scrub summary: idle+waiting paths [/] 2024-03-21T10:26:52.717 INFO:tasks.workunit.client.0.smithi047.stdout:for d in direct_io fs ; do ( cd $d ; make all ) ; done 2024-03-21T10:26:52.724 INFO:tasks.workunit.client.0.smithi047.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-03-21T10:26:52.724 INFO:tasks.workunit.client.0.smithi047.stdout:cc -Wall -Wextra -D_GNU_SOURCE direct_io_test.c -o direct_io_test 2024-03-21T10:26:52.728 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:26:52 smithi100 ceph-mon[35193]: scrub queued for path: / 2024-03-21T10:26:52.728 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:26:52 smithi100 ceph-mon[35193]: scrub summary: idle+waiting paths [/] 2024-03-21T10:26:52.729 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:26:52 smithi100 ceph-mon[35193]: scrub summary: active paths [/] 2024-03-21T10:26:52.729 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:26:52 smithi100 ceph-mon[35193]: scrub summary: idle+waiting paths [/] 2024-03-21T10:26:52.739 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:26:52 smithi157 ceph-mon[35581]: scrub queued for path: / 2024-03-21T10:26:52.739 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:26:52 smithi157 ceph-mon[35581]: scrub summary: idle+waiting paths [/] 2024-03-21T10:26:52.739 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:26:52 smithi157 ceph-mon[35581]: scrub summary: active paths [/] 2024-03-21T10:26:52.739 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:26:52 smithi157 ceph-mon[35581]: scrub summary: idle+waiting paths [/] 2024-03-21T10:26:53.589 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:26:53 smithi157 ceph-mon[35581]: pgmap v50: 129 pgs: 129 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 853 B/s wr, 13 op/s 2024-03-21T10:26:53.590 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:26:53 smithi157 ceph-mon[35581]: mgrmap e27: x(active, since 92s), standbys: y, z 2024-03-21T10:26:53.614 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:26:53 smithi047 ceph-mon[31547]: pgmap v50: 129 pgs: 129 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 853 B/s wr, 13 op/s 2024-03-21T10:26:53.614 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:26:53 smithi047 ceph-mon[31547]: mgrmap e27: x(active, since 92s), standbys: y, z 2024-03-21T10:26:53.728 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:26:53 smithi100 ceph-mon[35193]: pgmap v50: 129 pgs: 129 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 853 B/s wr, 13 op/s 2024-03-21T10:26:53.728 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:26:53 smithi100 ceph-mon[35193]: mgrmap e27: x(active, since 92s), standbys: y, z 2024-03-21T10:26:53.781 INFO:tasks.workunit.client.0.smithi047.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_sync_io.c -o test_sync_io 2024-03-21T10:26:53.957 INFO:tasks.workunit.client.0.smithi047.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_short_dio_read.c -o test_short_dio_read 2024-03-21T10:26:54.044 INFO:tasks.workunit.client.0.smithi047.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-03-21T10:26:54.050 INFO:tasks.workunit.client.0.smithi047.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-03-21T10:26:54.050 INFO:tasks.workunit.client.0.smithi047.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_o_trunc.c -o test_o_trunc 2024-03-21T10:26:54.133 INFO:tasks.workunit.client.0.smithi047.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-03-21T10:26:54.143 DEBUG:teuthology.orchestra.run.smithi047:> set -ex 2024-03-21T10:26:54.144 DEBUG:teuthology.orchestra.run.smithi047:> dd if=/home/ubuntu/cephtest/workunits.list.client.0 of=/dev/stdout 2024-03-21T10:26:54.192 INFO:tasks.workunit:Running workunits matching suites/fsync-tester.sh on client.0... 2024-03-21T10:26:54.193 INFO:tasks.workunit:Running workunit suites/fsync-tester.sh... 2024-03-21T10:26:54.193 DEBUG:teuthology.orchestra.run.smithi047:workunit test suites/fsync-tester.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=2edfee68ad7243bed422a049b6f5f58619300f11 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/fsync-tester.sh 2024-03-21T10:26:54.319 INFO:tasks.workunit.client.0.smithi047.stderr:+ wget http://download.ceph.com/qa/fsync-tester.c 2024-03-21T10:26:54.362 INFO:tasks.workunit.client.0.smithi047.stderr:--2024-03-21 10:26:54-- http://download.ceph.com/qa/fsync-tester.c 2024-03-21T10:26:54.363 INFO:tasks.workunit.client.0.smithi047.stderr:Resolving download.ceph.com (download.ceph.com)... 158.69.68.124, 2607:5300:201:2000::3:58a1 2024-03-21T10:26:54.385 INFO:tasks.workunit.client.0.smithi047.stderr:Connecting to download.ceph.com (download.ceph.com)|158.69.68.124|:80... connected. 2024-03-21T10:26:54.405 INFO:tasks.workunit.client.0.smithi047.stderr:HTTP request sent, awaiting response... 200 OK 2024-03-21T10:26:54.406 INFO:tasks.workunit.client.0.smithi047.stderr:Length: 3024 (3.0K) [application/octet-stream] 2024-03-21T10:26:54.413 INFO:tasks.workunit.client.0.smithi047.stderr:Saving to: ‘fsync-tester.c’ 2024-03-21T10:26:54.424 INFO:tasks.workunit.client.0.smithi047.stderr: 2024-03-21T10:26:54.424 INFO:tasks.workunit.client.0.smithi047.stderr: 0K .. 100% 79.6M=0s 2024-03-21T10:26:54.424 INFO:tasks.workunit.client.0.smithi047.stderr: 2024-03-21T10:26:54.425 INFO:tasks.workunit.client.0.smithi047.stderr:2024-03-21 10:26:54 (79.6 MB/s) - ‘fsync-tester.c’ saved [3024/3024] 2024-03-21T10:26:54.425 INFO:tasks.workunit.client.0.smithi047.stderr: 2024-03-21T10:26:54.427 INFO:tasks.workunit.client.0.smithi047.stderr:+ gcc -D_GNU_SOURCE fsync-tester.c -o fsync-tester 2024-03-21T10:26:54.619 INFO:tasks.workunit.client.0.smithi047.stderr:+ ./fsync-tester 2024-03-21T10:26:54.713 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:26:54 smithi047 ceph-mon[31547]: scrub summary: idle 2024-03-21T10:26:54.728 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:26:54 smithi100 ceph-mon[35193]: scrub summary: idle 2024-03-21T10:26:54.739 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:26:54 smithi157 ceph-mon[35581]: scrub summary: idle 2024-03-21T10:26:55.713 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:26:55 smithi047 ceph-mon[31547]: pgmap v51: 129 pgs: 129 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 852 B/s wr, 9 op/s 2024-03-21T10:26:55.728 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:26:55 smithi100 ceph-mon[35193]: pgmap v51: 129 pgs: 129 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 852 B/s wr, 9 op/s 2024-03-21T10:26:55.739 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:26:55 smithi157 ceph-mon[35581]: pgmap v51: 129 pgs: 129 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 852 B/s wr, 9 op/s 2024-03-21T10:26:56.481 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:56.480+0000 7fdb8ee00640 1 -- 172.21.15.47:0/3943401152 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdb88103690 msgr2=0x7fdb88103b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:26:56.481 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:56.480+0000 7fdb8ee00640 1 --2- 172.21.15.47:0/3943401152 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdb88103690 0x7fdb88103b10 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7fdb74009930 tx=0x7fdb7402bc30 comp rx=0 tx=0).stop 2024-03-21T10:26:56.481 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:56.480+0000 7fdb8ee00640 1 -- 172.21.15.47:0/3943401152 shutdown_connections 2024-03-21T10:26:56.481 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:56.480+0000 7fdb8ee00640 1 --2- 172.21.15.47:0/3943401152 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdb88104050 0x7fdb88110560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:26:56.481 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:56.480+0000 7fdb8ee00640 1 --2- 172.21.15.47:0/3943401152 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdb88103690 0x7fdb88103b10 unknown :-1 s=CLOSED pgs=667 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:26:56.481 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:56.480+0000 7fdb8ee00640 1 --2- 172.21.15.47:0/3943401152 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdb881096a0 0x7fdb88109a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:26:56.481 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:56.480+0000 7fdb8ee00640 1 -- 172.21.15.47:0/3943401152 >> 172.21.15.47:0/3943401152 conn(0x7fdb880ff180 msgr2=0x7fdb881015a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:26:56.481 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:56.480+0000 7fdb8ee00640 1 -- 172.21.15.47:0/3943401152 shutdown_connections 2024-03-21T10:26:56.481 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:56.480+0000 7fdb8ee00640 1 -- 172.21.15.47:0/3943401152 wait complete. 2024-03-21T10:26:56.482 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:56.482+0000 7fdb8ee00640 1 Processor -- start 2024-03-21T10:26:56.483 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:56.482+0000 7fdb8ee00640 1 -- start start 2024-03-21T10:26:56.483 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:56.482+0000 7fdb8ee00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdb88103690 0x7fdb88199f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:26:56.483 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:56.482+0000 7fdb8ee00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdb88104050 0x7fdb8819a450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:26:56.483 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:56.482+0000 7fdb8ee00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdb881096a0 0x7fdb8819e7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:26:56.483 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:56.482+0000 7fdb8ee00640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb881139b0 con 0x7fdb881096a0 2024-03-21T10:26:56.483 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:56.482+0000 7fdb8ee00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb88113830 con 0x7fdb88104050 2024-03-21T10:26:56.483 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:56.482+0000 7fdb8ee00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb88113b30 con 0x7fdb88103690 2024-03-21T10:26:56.483 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:56.483+0000 7fdb8e400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdb881096a0 0x7fdb8819e7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:26:56.483 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:56.483+0000 7fdb8d000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdb88104050 0x7fdb8819a450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:26:56.483 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:56.483+0000 7fdb8e400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdb881096a0 0x7fdb8819e7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:57930/0 (socket says 172.21.15.47:57930) 2024-03-21T10:26:56.483 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:56.483+0000 7fdb8e400640 1 -- 172.21.15.47:0/1577131815 learned_addr learned my addr 172.21.15.47:0/1577131815 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:26:56.485 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:56.483+0000 7fdb8da00640 1 --2- 172.21.15.47:0/1577131815 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdb88103690 0x7fdb88199f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:26:56.485 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:56.483+0000 7fdb8d000640 1 -- 172.21.15.47:0/1577131815 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdb88103690 msgr2=0x7fdb88199f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:26:56.485 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:56.483+0000 7fdb8d000640 1 --2- 172.21.15.47:0/1577131815 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdb88103690 0x7fdb88199f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:26:56.485 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:56.483+0000 7fdb8d000640 1 -- 172.21.15.47:0/1577131815 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdb881096a0 msgr2=0x7fdb8819e7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:26:56.485 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:56.483+0000 7fdb8d000640 1 --2- 172.21.15.47:0/1577131815 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdb881096a0 0x7fdb8819e7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:26:56.485 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:56.483+0000 7fdb8d000640 1 -- 172.21.15.47:0/1577131815 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdb8819ef00 con 0x7fdb88104050 2024-03-21T10:26:56.486 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:56.483+0000 7fdb8e400640 1 --2- 172.21.15.47:0/1577131815 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdb881096a0 0x7fdb8819e7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T10:26:56.486 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:56.484+0000 7fdb8d000640 1 --2- 172.21.15.47:0/1577131815 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdb88104050 0x7fdb8819a450 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fdb74036040 tx=0x7fdb740045e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:26:56.486 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:56.484+0000 7fdb86000640 1 -- 172.21.15.47:0/1577131815 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb74043020 con 0x7fdb88104050 2024-03-21T10:26:56.486 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:56.484+0000 7fdb8ee00640 1 -- 172.21.15.47:0/1577131815 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdb8819f190 con 0x7fdb88104050 2024-03-21T10:26:56.486 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:56.484+0000 7fdb86000640 1 -- 172.21.15.47:0/1577131815 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fdb7402cb70 con 0x7fdb88104050 2024-03-21T10:26:56.486 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:56.484+0000 7fdb86000640 1 -- 172.21.15.47:0/1577131815 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb74003ae0 con 0x7fdb88104050 2024-03-21T10:26:56.486 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:56.484+0000 7fdb8ee00640 1 -- 172.21.15.47:0/1577131815 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdb881a6a80 con 0x7fdb88104050 2024-03-21T10:26:56.488 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:56.488+0000 7fdb9306f740 1 Processor -- start 2024-03-21T10:26:56.488 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:56.488+0000 7fdb9306f740 1 -- start start 2024-03-21T10:26:56.489 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:56.489+0000 7fdb9306f740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555cf5ae01e0 0x555cf5ae05c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:26:56.489 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:56.489+0000 7fdb9306f740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x555cf5ae0b00 0x555cf5af1ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:26:56.489 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:56.489+0000 7fdb9306f740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555cf5af20e0 0x555cf5af44d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:26:56.490 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:56.489+0000 7fdb9306f740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x555cf5aa23b0 con 0x555cf5ae0b00 2024-03-21T10:26:56.490 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:56.489+0000 7fdb9306f740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x555cf5a44830 con 0x555cf5af20e0 2024-03-21T10:26:56.490 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:56.489+0000 7fdb9306f740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x555cf59ed5d0 con 0x555cf5ae01e0 2024-03-21T10:26:56.490 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:56.489+0000 7fdb8da00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x555cf5ae0b00 0x555cf5af1ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:26:56.490 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:56.489+0000 7fdb8da00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x555cf5ae0b00 0x555cf5af1ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:57940/0 (socket says 172.21.15.47:57940) 2024-03-21T10:26:56.490 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:56.489+0000 7fdb8da00640 1 -- 172.21.15.47:0/1440525013 learned_addr learned my addr 172.21.15.47:0/1440525013 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:26:56.490 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:56.490+0000 7fdb8e400640 1 --2- 172.21.15.47:0/1440525013 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555cf5ae01e0 0x555cf5ae05c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:26:56.491 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:56.490+0000 7fdb8da00640 1 -- 172.21.15.47:0/1440525013 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555cf5ae01e0 msgr2=0x555cf5ae05c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:26:56.491 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:56.490+0000 7fdb86000640 1 -- 172.21.15.47:0/1577131815 <== mon.2 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 27) v1 ==== 137985+0+0 (secure 0 0 0) 0x7fdb74046020 con 0x7fdb88104050 2024-03-21T10:26:56.491 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:56.490+0000 7fdb8da00640 1 --2- 172.21.15.47:0/1440525013 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555cf5ae01e0 0x555cf5ae05c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:26:56.491 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:56.490+0000 7fdb8da00640 1 -- 172.21.15.47:0/1440525013 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555cf5af20e0 msgr2=0x555cf5af44d0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-21T10:26:56.491 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:56.490+0000 7fdb8da00640 1 --2- 172.21.15.47:0/1440525013 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555cf5af20e0 0x555cf5af44d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:26:56.491 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:56.490+0000 7fdb8da00640 1 -- 172.21.15.47:0/1440525013 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555cf5a052c0 con 0x555cf5ae0b00 2024-03-21T10:26:56.491 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:56.490+0000 7fdb8da00640 1 --2- 172.21.15.47:0/1440525013 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x555cf5ae0b00 0x555cf5af1ba0 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7fdb7c00bc60 tx=0x7fdb7c00f730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e181370c4e88c6ea server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:26:56.491 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:56.491+0000 7fdb7aa00640 1 -- 172.21.15.47:0/1440525013 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb7c015070 con 0x555cf5ae0b00 2024-03-21T10:26:56.492 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:56.491+0000 7fdb7aa00640 1 -- 172.21.15.47:0/1440525013 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fdb7c0118f0 con 0x555cf5ae0b00 2024-03-21T10:26:56.492 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:56.491+0000 7fdb7aa00640 1 -- 172.21.15.47:0/1440525013 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb7c0107f0 con 0x555cf5ae0b00 2024-03-21T10:26:56.493 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:56.492+0000 7fdb86000640 1 --2- 172.21.15.47:0/1577131815 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7fdb700a3100 0x7fdb700a55c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:26:56.493 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:56.492+0000 7fdb9306f740 1 -- 172.21.15.47:0/1440525013 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x555cf5ae0b00 msgr2=0x555cf5af1ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:26:56.493 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:56.492+0000 7fdb9306f740 1 --2- 172.21.15.47:0/1440525013 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x555cf5ae0b00 0x555cf5af1ba0 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7fdb7c00bc60 tx=0x7fdb7c00f730 comp rx=0 tx=0).stop 2024-03-21T10:26:56.493 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:56.492+0000 7fdb86000640 1 -- 172.21.15.47:0/1577131815 <== mon.2 v2:172.21.15.100:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 9474+0+0 (secure 0 0 0) 0x7fdb740f1a70 con 0x7fdb88104050 2024-03-21T10:26:56.493 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:56.492+0000 7fdb9306f740 1 -- 172.21.15.47:0/1440525013 shutdown_connections 2024-03-21T10:26:56.493 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:56.492+0000 7fdb9306f740 1 --2- 172.21.15.47:0/1440525013 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555cf5af20e0 0x555cf5af44d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:26:56.493 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:56.492+0000 7fdb9306f740 1 --2- 172.21.15.47:0/1440525013 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x555cf5ae0b00 0x555cf5af1ba0 unknown :-1 s=CLOSED pgs=668 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:26:56.494 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:56.492+0000 7fdb9306f740 1 --2- 172.21.15.47:0/1440525013 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555cf5ae01e0 0x555cf5ae05c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:26:56.494 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:56.492+0000 7fdb9306f740 1 -- 172.21.15.47:0/1440525013 >> 172.21.15.47:0/1440525013 conn(0x555cf5adab00 msgr2=0x555cf5adaee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:26:56.494 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:56.492+0000 7fdb8e400640 1 --2- 172.21.15.47:0/1577131815 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7fdb700a3100 0x7fdb700a55c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:26:56.494 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:56.492+0000 7fdb9306f740 1 -- 172.21.15.47:0/1440525013 shutdown_connections 2024-03-21T10:26:56.494 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:56.492+0000 7fdb9306f740 1 -- 172.21.15.47:0/1440525013 wait complete. 2024-03-21T10:26:56.494 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:56.493+0000 7fdb9306f740 1 -- start start 2024-03-21T10:26:56.494 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:56.493+0000 7fdb8e400640 1 --2- 172.21.15.47:0/1577131815 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7fdb700a3100 0x7fdb700a55c0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fdb800049d0 tx=0x7fdb80009500 comp rx=0 tx=0).ready entity=mgr.15687 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:26:56.494 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:56.493+0000 7fdb9306f740 1 Processor -- start 2024-03-21T10:26:56.497 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:56.494+0000 7fdb9306f740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555cf5ae01e0 0x555cf5aef230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:26:56.497 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:56.494+0000 7fdb9306f740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x555cf5ae0b00 0x555cf5aef770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:26:56.497 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:56.494+0000 7fdb9306f740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555cf5af20e0 0x555cf5aefcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:26:56.497 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:56.494+0000 7fdb9306f740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x555cf5a00a20 con 0x555cf5ae0b00 2024-03-21T10:26:56.497 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:56.494+0000 7fdb9306f740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x555cf5a408d0 con 0x555cf5ae01e0 2024-03-21T10:26:56.497 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:56.494+0000 7fdb9306f740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x555cf5af4d10 con 0x555cf5af20e0 2024-03-21T10:26:56.497 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:56.494+0000 7fdb9306f740 10 client.0 _ll_register_callbacks cb 0x555cf5a3d1c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T10:26:56.498 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:56.494+0000 7fdb8d000640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x555cf5ae0b00 0x555cf5aef770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:26:56.498 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:56.494+0000 7fdb8da00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555cf5ae01e0 0x555cf5aef230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:26:56.498 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:56.494+0000 7fdb8da00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555cf5ae01e0 0x555cf5aef230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:38446/0 (socket says 172.21.15.47:38446) 2024-03-21T10:26:56.498 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:56.494+0000 7fdb8da00640 1 -- 172.21.15.47:0/142532605 learned_addr learned my addr 172.21.15.47:0/142532605 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:26:56.498 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:56.494+0000 7fdb8d000640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x555cf5ae0b00 0x555cf5aef770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:57944/0 (socket says 172.21.15.47:57944) 2024-03-21T10:26:56.498 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:56.495+0000 7fdb8e400640 1 --2- 172.21.15.47:0/142532605 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555cf5af20e0 0x555cf5aefcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:26:56.498 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:56.495+0000 7fdb8da00640 1 -- 172.21.15.47:0/142532605 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555cf5af20e0 msgr2=0x555cf5aefcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:26:56.498 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:56.495+0000 7fdb8da00640 1 --2- 172.21.15.47:0/142532605 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555cf5af20e0 0x555cf5aefcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:26:56.498 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:56.495+0000 7fdb8da00640 1 -- 172.21.15.47:0/142532605 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x555cf5ae0b00 msgr2=0x555cf5aef770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:26:56.498 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:56.495+0000 7fdb8da00640 1 --2- 172.21.15.47:0/142532605 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x555cf5ae0b00 0x555cf5aef770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:26:56.498 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:56.495+0000 7fdb8da00640 1 -- 172.21.15.47:0/142532605 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x555cf5759210 con 0x555cf5ae01e0 2024-03-21T10:26:56.498 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:56.495+0000 7fdb8d000640 1 --2- 172.21.15.47:0/142532605 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x555cf5ae0b00 0x555cf5aef770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T10:26:56.498 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:56.495+0000 7fdb8da00640 1 --2- 172.21.15.47:0/142532605 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555cf5ae01e0 0x555cf5aef230 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fdb7c00bec0 tx=0x7fdb7c00bef0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1ba144368285f93f server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:26:56.498 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:56.495+0000 7fdb6f400640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-21T10:26:56.499 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:56.496+0000 7fdb6f400640 1 -- 172.21.15.47:0/142532605 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb7c015070 con 0x555cf5ae01e0 2024-03-21T10:26:56.499 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:56.496+0000 7fdb9306f740 1 -- 172.21.15.47:0/142532605 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fdb7c0118f0 con 0x555cf5ae01e0 2024-03-21T10:26:56.499 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:56.496+0000 7fdb6f400640 1 -- 172.21.15.47:0/142532605 <== mon.2 v2:172.21.15.100:3300/0 2 ==== osd_map(97..97 src has 1..97) v4 ==== 9474+0+0 (secure 0 0 0) 0x7fdb7c011650 con 0x555cf5ae01e0 2024-03-21T10:26:56.499 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:56.496+0000 7fdb6f400640 1 -- 172.21.15.47:0/142532605 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7fdb7c011aa0 con 0x555cf5ae01e0 2024-03-21T10:26:56.499 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:56.496+0000 7fdb9306f740 10 client.36122 fetch_fsmap learned FSMap version 32 2024-03-21T10:26:56.499 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:56.496+0000 7fdb9306f740 1 -- 172.21.15.47:0/142532605 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x555cf5825620 con 0x555cf5ae01e0 2024-03-21T10:26:56.499 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:56.497+0000 7fdb6f400640 1 -- 172.21.15.47:0/142532605 <== mon.2 v2:172.21.15.100:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fdb7c0118f0 con 0x555cf5ae01e0 2024-03-21T10:26:56.499 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:56.497+0000 7fdb6f400640 1 -- 172.21.15.47:0/142532605 <== mon.2 v2:172.21.15.100:3300/0 5 ==== fsmap(e 32) v1 ==== 7942+0+0 (secure 0 0 0) 0x7fdb7c0048c0 con 0x555cf5ae01e0 2024-03-21T10:26:56.499 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:56.497+0000 7fdb9306f740 10 client.36122 fetch_fsmap finished waiting for FSMap version 32 2024-03-21T10:26:56.499 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:56.497+0000 7fdb9306f740 10 client.36122 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-21T10:26:56.499 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:56.497+0000 7fdb9306f740 20 client.36122 populate_metadata read hostname 'smithi047' 2024-03-21T10:26:56.499 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:56.497+0000 7fdb9306f740 1 --2- 172.21.15.47:0/142532605 >> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] conn(0x555cf5b02160 0x555cf5b04550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:26:56.499 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:56.497+0000 7fdb9306f740 4 client.36122 mds_command: new command op to 34535 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T10:26:56.499 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:56.497+0000 7fdb9306f740 1 -- 172.21.15.47:0/142532605 --> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x555cf5a51d70 con 0x555cf5b02160 2024-03-21T10:26:56.501 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:56.498+0000 7fdb8d000640 1 --2- 172.21.15.47:0/142532605 >> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] conn(0x555cf5b02160 0x555cf5b04550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:26:56.501 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:56.498+0000 7fdb8d000640 1 --2- 172.21.15.47:0/142532605 >> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] conn(0x555cf5b02160 0x555cf5b04550 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=345e59dc6b428841 server_cookie=d6bc5fa3f767fe29 in_seq=0 out_seq=0 2024-03-21T10:26:56.501 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:56.498+0000 7fdb6f400640 10 client.36122 ms_handle_connect on v2:172.21.15.100:6838/4173168440 2024-03-21T10:26:56.501 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:56.500+0000 7fdb6f400640 1 -- 172.21.15.47:0/142532605 <== mds.0 v2:172.21.15.100:6838/4173168440 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x555cf5a51d70 con 0x555cf5b02160 2024-03-21T10:26:56.501 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:56.500+0000 7fdb6f400640 10 client.36122 handle_command_reply: tid=0 2024-03-21T10:26:56.501 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:56.500+0000 7fdb9306f740 1 client.36122 shutdown 2024-03-21T10:26:56.501 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:56.500+0000 7fdb9306f740 1 -- 172.21.15.47:0/142532605 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555cf5ae01e0 msgr2=0x555cf5aef230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:26:56.501 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:56.500+0000 7fdb9306f740 1 --2- 172.21.15.47:0/142532605 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555cf5ae01e0 0x555cf5aef230 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fdb7c00bec0 tx=0x7fdb7c00bef0 comp rx=0 tx=0).stop 2024-03-21T10:26:56.501 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:56.500+0000 7fdb9306f740 1 -- 172.21.15.47:0/142532605 shutdown_connections 2024-03-21T10:26:56.501 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:56.500+0000 7fdb9306f740 1 --2- 172.21.15.47:0/142532605 >> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] conn(0x555cf5b02160 0x555cf5b04550 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:26:56.502 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:56.500+0000 7fdb9306f740 1 --2- 172.21.15.47:0/142532605 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555cf5af20e0 0x555cf5aefcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:26:56.502 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:56.500+0000 7fdb9306f740 1 --2- 172.21.15.47:0/142532605 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x555cf5ae0b00 0x555cf5aef770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:26:56.502 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:56.500+0000 7fdb9306f740 1 --2- 172.21.15.47:0/142532605 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555cf5ae01e0 0x555cf5aef230 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:26:56.502 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:56.500+0000 7fdb6f400640 0 client.36122 ms_handle_reset on v2:172.21.15.100:6838/4173168440 2024-03-21T10:26:56.502 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:56.500+0000 7fdb9306f740 1 -- 172.21.15.47:0/142532605 >> 172.21.15.47:0/142532605 conn(0x555cf5adab00 msgr2=0x555cf5adc850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:26:56.502 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:56.500+0000 7fdb9306f740 1 -- 172.21.15.47:0/142532605 shutdown_connections 2024-03-21T10:26:56.502 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:56.500+0000 7fdb9306f740 1 -- 172.21.15.47:0/142532605 wait complete. 2024-03-21T10:26:56.502 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:56.501+0000 7fdb9306f740 20 client.36122 trim_cache size 0 max 16384 2024-03-21T10:26:56.519 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:56.518+0000 7fdb9306f740 1 Processor -- start 2024-03-21T10:26:56.520 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:56.518+0000 7fdb9306f740 1 -- start start 2024-03-21T10:26:56.520 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:56.518+0000 7fdb9306f740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555cf5ae01e0 0x555cf5af1960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:26:56.520 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:56.518+0000 7fdb9306f740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555cf5ae0b00 0x555cf5afd670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:26:56.520 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:56.518+0000 7fdb9306f740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x555cf5af20e0 0x555cf5aec4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:26:56.520 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:56.518+0000 7fdb9306f740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb881ac580 con 0x555cf5af20e0 2024-03-21T10:26:56.520 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:56.518+0000 7fdb9306f740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb881adf00 con 0x555cf5ae0b00 2024-03-21T10:26:56.520 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:56.518+0000 7fdb9306f740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb881acbe0 con 0x555cf5ae01e0 2024-03-21T10:26:56.520 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:56.519+0000 7fdb8e400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x555cf5af20e0 0x555cf5aec4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:26:56.521 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:56.519+0000 7fdb8d000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555cf5ae0b00 0x555cf5afd670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:26:56.521 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:56.519+0000 7fdb8e400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x555cf5af20e0 0x555cf5aec4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:57948/0 (socket says 172.21.15.47:57948) 2024-03-21T10:26:56.521 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:56.519+0000 7fdb8e400640 1 -- 172.21.15.47:0/887150757 learned_addr learned my addr 172.21.15.47:0/887150757 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:26:56.521 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:56.519+0000 7fdb8da00640 1 --2- 172.21.15.47:0/887150757 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555cf5ae01e0 0x555cf5af1960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:26:56.521 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:56.519+0000 7fdb8e400640 1 -- 172.21.15.47:0/887150757 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555cf5ae01e0 msgr2=0x555cf5af1960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:26:56.521 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:56.519+0000 7fdb8e400640 1 --2- 172.21.15.47:0/887150757 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555cf5ae01e0 0x555cf5af1960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:26:56.521 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:56.519+0000 7fdb8e400640 1 -- 172.21.15.47:0/887150757 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555cf5ae0b00 msgr2=0x555cf5afd670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:26:56.521 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:56.519+0000 7fdb8e400640 1 --2- 172.21.15.47:0/887150757 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555cf5ae0b00 0x555cf5afd670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:26:56.521 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:56.519+0000 7fdb8e400640 1 -- 172.21.15.47:0/887150757 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555cf5aeae50 con 0x555cf5af20e0 2024-03-21T10:26:56.521 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:56.519+0000 7fdb8d000640 1 --2- 172.21.15.47:0/887150757 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555cf5ae0b00 0x555cf5afd670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T10:26:56.521 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:56.519+0000 7fdb8da00640 1 --2- 172.21.15.47:0/887150757 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555cf5ae01e0 0x555cf5af1960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T10:26:56.523 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:56.520+0000 7fdb8e400640 1 --2- 172.21.15.47:0/887150757 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x555cf5af20e0 0x555cf5aec4f0 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7fdb80012500 tx=0x7fdb80017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eeb6ebea9b9cf848 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:26:56.523 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:56.520+0000 7fdb7a000640 1 -- 172.21.15.47:0/887150757 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb80011940 con 0x555cf5af20e0 2024-03-21T10:26:56.523 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:56.520+0000 7fdb7a000640 1 -- 172.21.15.47:0/887150757 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fdb8001f420 con 0x555cf5af20e0 2024-03-21T10:26:56.523 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:56.520+0000 7fdb7a000640 1 -- 172.21.15.47:0/887150757 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb8001d400 con 0x555cf5af20e0 2024-03-21T10:26:56.523 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:56.521+0000 7fdb9306f740 1 -- 172.21.15.47:0/887150757 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x555cf5af20e0 msgr2=0x555cf5aec4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:26:56.523 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:56.521+0000 7fdb9306f740 1 --2- 172.21.15.47:0/887150757 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x555cf5af20e0 0x555cf5aec4f0 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7fdb80012500 tx=0x7fdb80017000 comp rx=0 tx=0).stop 2024-03-21T10:26:56.523 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:56.521+0000 7fdb9306f740 1 -- 172.21.15.47:0/887150757 shutdown_connections 2024-03-21T10:26:56.523 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:56.521+0000 7fdb9306f740 1 --2- 172.21.15.47:0/887150757 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x555cf5af20e0 0x555cf5aec4f0 unknown :-1 s=CLOSED pgs=669 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:26:56.523 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:56.521+0000 7fdb9306f740 1 --2- 172.21.15.47:0/887150757 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555cf5ae0b00 0x555cf5afd670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:26:56.524 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:56.521+0000 7fdb9306f740 1 --2- 172.21.15.47:0/887150757 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555cf5ae01e0 0x555cf5af1960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:26:56.524 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:56.521+0000 7fdb9306f740 1 -- 172.21.15.47:0/887150757 >> 172.21.15.47:0/887150757 conn(0x555cf5adab00 msgr2=0x555cf5afcf60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:26:56.524 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:56.522+0000 7fdb9306f740 1 -- 172.21.15.47:0/887150757 shutdown_connections 2024-03-21T10:26:56.524 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:56.522+0000 7fdb9306f740 1 -- 172.21.15.47:0/887150757 wait complete. 2024-03-21T10:26:56.524 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:56.522+0000 7fdb9306f740 1 -- start start 2024-03-21T10:26:56.524 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:56.523+0000 7fdb9306f740 1 Processor -- start 2024-03-21T10:26:56.527 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:56.523+0000 7fdb9306f740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x555cf5ae01e0 0x555cf5b04c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:26:56.527 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:56.523+0000 7fdb9306f740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555cf5ae0b00 0x555cf5b05170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:26:56.527 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:56.523+0000 7fdb9306f740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555cf5af20e0 0x555cf5b056b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:26:56.527 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:56.523+0000 7fdb9306f740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb881ad8a0 con 0x555cf5ae01e0 2024-03-21T10:26:56.527 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:56.523+0000 7fdb9306f740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb881ad240 con 0x555cf5ae0b00 2024-03-21T10:26:56.527 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:56.523+0000 7fdb9306f740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb881aa740 con 0x555cf5af20e0 2024-03-21T10:26:56.527 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:56.523+0000 7fdb9306f740 10 client.0 _ll_register_callbacks cb 0x7fdb7c00bab0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T10:26:56.528 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:56.523+0000 7fdb8da00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x555cf5ae01e0 0x555cf5b04c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:26:56.528 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:56.524+0000 7fdb8da00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x555cf5ae01e0 0x555cf5b04c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:57954/0 (socket says 172.21.15.47:57954) 2024-03-21T10:26:56.528 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:56.524+0000 7fdb8da00640 1 -- 172.21.15.47:0/1670967192 learned_addr learned my addr 172.21.15.47:0/1670967192 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:26:56.528 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:56.524+0000 7fdb8d000640 1 --2- 172.21.15.47:0/1670967192 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555cf5ae0b00 0x555cf5b05170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:26:56.528 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:56.524+0000 7fdb8e400640 1 --2- 172.21.15.47:0/1670967192 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555cf5af20e0 0x555cf5b056b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:26:56.528 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:56.524+0000 7fdb8da00640 1 -- 172.21.15.47:0/1670967192 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555cf5af20e0 msgr2=0x555cf5b056b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:26:56.528 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:56.524+0000 7fdb8da00640 1 --2- 172.21.15.47:0/1670967192 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555cf5af20e0 0x555cf5b056b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:26:56.528 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:56.524+0000 7fdb8da00640 1 -- 172.21.15.47:0/1670967192 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555cf5ae0b00 msgr2=0x555cf5b05170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:26:56.528 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:56.524+0000 7fdb8da00640 1 --2- 172.21.15.47:0/1670967192 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555cf5ae0b00 0x555cf5b05170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:26:56.528 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:56.524+0000 7fdb8da00640 1 -- 172.21.15.47:0/1670967192 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x555cf5b5c930 con 0x555cf5ae01e0 2024-03-21T10:26:56.528 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:56.524+0000 7fdb8da00640 1 --2- 172.21.15.47:0/1670967192 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x555cf5ae01e0 0x555cf5b04c30 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7fdb7c00f770 tx=0x7fdb7c005cd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d6ae2ac92e41ac7 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:26:56.528 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:56.524+0000 7fdb6f400640 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-03-21T10:26:56.528 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:56.525+0000 7fdb6f400640 1 -- 172.21.15.47:0/1670967192 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb7c036060 con 0x555cf5ae01e0 2024-03-21T10:26:56.528 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:56.525+0000 7fdb6f400640 1 -- 172.21.15.47:0/1670967192 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(97..97 src has 1..97) v4 ==== 9474+0+0 (secure 0 0 0) 0x7fdb7c0104e0 con 0x555cf5ae01e0 2024-03-21T10:26:56.529 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:56.525+0000 7fdb9306f740 1 -- 172.21.15.47:0/1670967192 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fdb8001f420 con 0x555cf5ae01e0 2024-03-21T10:26:56.529 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:56.525+0000 7fdb6f400640 1 -- 172.21.15.47:0/1670967192 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7fdb7c03d050 con 0x555cf5ae01e0 2024-03-21T10:26:56.529 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:56.525+0000 7fdb9306f740 10 client.16341 fetch_fsmap learned FSMap version 32 2024-03-21T10:26:56.529 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:56.525+0000 7fdb9306f740 1 -- 172.21.15.47:0/1670967192 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x555cf5b5dc40 con 0x555cf5ae01e0 2024-03-21T10:26:56.529 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:56.526+0000 7fdb6f400640 1 -- 172.21.15.47:0/1670967192 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fdb8001f420 con 0x555cf5ae01e0 2024-03-21T10:26:56.529 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:56.526+0000 7fdb6f400640 1 -- 172.21.15.47:0/1670967192 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 32) v1 ==== 7942+0+0 (secure 0 0 0) 0x7fdb7c004330 con 0x555cf5ae01e0 2024-03-21T10:26:56.529 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:56.526+0000 7fdb9306f740 10 client.16341 fetch_fsmap finished waiting for FSMap version 32 2024-03-21T10:26:56.529 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:56.526+0000 7fdb9306f740 10 client.16341 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-21T10:26:56.529 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:56.526+0000 7fdb9306f740 20 client.16341 populate_metadata read hostname 'smithi047' 2024-03-21T10:26:56.529 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:56.526+0000 7fdb9306f740 1 --2- 172.21.15.47:0/1670967192 >> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] conn(0x555cf5b5e310 0x555cf5b60730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:26:56.529 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:56.526+0000 7fdb9306f740 4 client.16341 mds_command: new command op to 34535 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-21T10:26:56.529 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:56.526+0000 7fdb9306f740 1 -- 172.21.15.47:0/1670967192 --> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x555cf5a4c0c0 con 0x555cf5b5e310 2024-03-21T10:26:56.529 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:56.527+0000 7fdb8d000640 1 --2- 172.21.15.47:0/1670967192 >> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] conn(0x555cf5b5e310 0x555cf5b60730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:26:56.530 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:56.527+0000 7fdb8d000640 1 --2- 172.21.15.47:0/1670967192 >> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] conn(0x555cf5b5e310 0x555cf5b60730 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cfff3c8a3bc583bb server_cookie=99cadd97c40b3f3c in_seq=0 out_seq=0 2024-03-21T10:26:56.530 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:56.527+0000 7fdb6f400640 10 client.16341 ms_handle_connect on v2:172.21.15.100:6838/4173168440 2024-03-21T10:26:56.551 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:56.550+0000 7fdb6f400640 1 -- 172.21.15.47:0/1670967192 <== mds.0 v2:172.21.15.100:6838/4173168440 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x555cf5a4c0c0 con 0x555cf5b5e310 2024-03-21T10:26:56.551 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:56.550+0000 7fdb6f400640 10 client.16341 handle_command_reply: tid=0 2024-03-21T10:26:56.551 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:56.550+0000 7fdb9306f740 1 client.16341 shutdown 2024-03-21T10:26:56.551 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:56.550+0000 7fdb9306f740 1 -- 172.21.15.47:0/1670967192 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x555cf5ae01e0 msgr2=0x555cf5b04c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:26:56.551 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:56.550+0000 7fdb9306f740 1 --2- 172.21.15.47:0/1670967192 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x555cf5ae01e0 0x555cf5b04c30 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7fdb7c00f770 tx=0x7fdb7c005cd0 comp rx=0 tx=0).stop 2024-03-21T10:26:56.552 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:56.550+0000 7fdb9306f740 1 -- 172.21.15.47:0/1670967192 shutdown_connections 2024-03-21T10:26:56.552 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:56.550+0000 7fdb9306f740 1 --2- 172.21.15.47:0/1670967192 >> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] conn(0x555cf5b5e310 0x555cf5b60730 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:26:56.552 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:56.550+0000 7fdb9306f740 1 --2- 172.21.15.47:0/1670967192 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555cf5af20e0 0x555cf5b056b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:26:56.552 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:56.550+0000 7fdb9306f740 1 --2- 172.21.15.47:0/1670967192 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555cf5ae0b00 0x555cf5b05170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:26:56.552 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:56.550+0000 7fdb9306f740 1 --2- 172.21.15.47:0/1670967192 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x555cf5ae01e0 0x555cf5b04c30 unknown :-1 s=CLOSED pgs=670 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:26:56.552 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:56.550+0000 7fdb9306f740 1 -- 172.21.15.47:0/1670967192 >> 172.21.15.47:0/1670967192 conn(0x555cf5adab00 msgr2=0x555cf5afd7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:26:56.552 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:56.550+0000 7fdb6f400640 0 client.16341 ms_handle_reset on v2:172.21.15.100:6838/4173168440 2024-03-21T10:26:56.552 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:56.551+0000 7fdb9306f740 1 -- 172.21.15.47:0/1670967192 shutdown_connections 2024-03-21T10:26:56.552 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:56.551+0000 7fdb9306f740 1 -- 172.21.15.47:0/1670967192 wait complete. 2024-03-21T10:26:56.553 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:56.551+0000 7fdb9306f740 20 client.16341 trim_cache size 0 max 16384 2024-03-21T10:26:56.553 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout:{ 2024-03-21T10:26:56.553 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout: "message": "", 2024-03-21T10:26:56.553 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout: "return_code": 0 2024-03-21T10:26:56.553 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout:} 2024-03-21T10:26:56.554 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:56.552+0000 7fdb8ee00640 1 -- 172.21.15.47:0/1577131815 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7fdb700a3100 msgr2=0x7fdb700a55c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:26:56.554 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:56.552+0000 7fdb8ee00640 1 --2- 172.21.15.47:0/1577131815 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7fdb700a3100 0x7fdb700a55c0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fdb800049d0 tx=0x7fdb80009500 comp rx=0 tx=0).stop 2024-03-21T10:26:56.554 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:56.552+0000 7fdb8ee00640 1 -- 172.21.15.47:0/1577131815 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdb88104050 msgr2=0x7fdb8819a450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:26:56.554 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:56.552+0000 7fdb8ee00640 1 --2- 172.21.15.47:0/1577131815 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdb88104050 0x7fdb8819a450 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fdb74036040 tx=0x7fdb740045e0 comp rx=0 tx=0).stop 2024-03-21T10:26:56.554 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:56.552+0000 7fdb8ee00640 1 -- 172.21.15.47:0/1577131815 shutdown_connections 2024-03-21T10:26:56.554 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:56.552+0000 7fdb8ee00640 1 --2- 172.21.15.47:0/1577131815 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdb881096a0 0x7fdb8819e7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:26:56.554 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:56.552+0000 7fdb8ee00640 1 --2- 172.21.15.47:0/1577131815 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7fdb700a3100 0x7fdb700a55c0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:26:56.554 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:56.552+0000 7fdb8ee00640 1 --2- 172.21.15.47:0/1577131815 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdb88104050 0x7fdb8819a450 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:26:56.555 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:56.552+0000 7fdb8ee00640 1 --2- 172.21.15.47:0/1577131815 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdb88103690 0x7fdb88199f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:26:56.555 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:56.552+0000 7fdb8ee00640 1 -- 172.21.15.47:0/1577131815 >> 172.21.15.47:0/1577131815 conn(0x7fdb880ff180 msgr2=0x7fdb880ffa90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:26:56.555 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:56.552+0000 7fdb8ee00640 1 -- 172.21.15.47:0/1577131815 shutdown_connections 2024-03-21T10:26:56.555 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:26:56.552+0000 7fdb8ee00640 1 -- 172.21.15.47:0/1577131815 wait complete. 2024-03-21T10:26:57.713 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:26:57 smithi047 ceph-mon[31547]: pgmap v52: 129 pgs: 129 active+clean; 1.4 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 9.6 KiB/s rd, 6.1 KiB/s wr, 16 op/s 2024-03-21T10:26:57.728 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:26:57 smithi100 ceph-mon[35193]: pgmap v52: 129 pgs: 129 active+clean; 1.4 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 9.6 KiB/s rd, 6.1 KiB/s wr, 16 op/s 2024-03-21T10:26:57.739 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:26:57 smithi157 ceph-mon[35581]: pgmap v52: 129 pgs: 129 active+clean; 1.4 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 9.6 KiB/s rd, 6.1 KiB/s wr, 16 op/s 2024-03-21T10:26:59.713 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:26:59 smithi047 ceph-mon[31547]: pgmap v53: 129 pgs: 129 active+clean; 1.4 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 5.8 KiB/s wr, 12 op/s 2024-03-21T10:26:59.728 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:26:59 smithi100 ceph-mon[35193]: pgmap v53: 129 pgs: 129 active+clean; 1.4 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 5.8 KiB/s wr, 12 op/s 2024-03-21T10:26:59.738 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:26:59 smithi157 ceph-mon[35581]: pgmap v53: 129 pgs: 129 active+clean; 1.4 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 5.8 KiB/s wr, 12 op/s 2024-03-21T10:27:01.713 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:27:01 smithi047 ceph-mon[31547]: pgmap v54: 129 pgs: 129 active+clean; 1.4 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 5.8 KiB/s wr, 12 op/s 2024-03-21T10:27:01.728 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:27:01 smithi100 ceph-mon[35193]: pgmap v54: 129 pgs: 129 active+clean; 1.4 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 5.8 KiB/s wr, 12 op/s 2024-03-21T10:27:01.738 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:27:01 smithi157 ceph-mon[35581]: pgmap v54: 129 pgs: 129 active+clean; 1.4 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 5.8 KiB/s wr, 12 op/s 2024-03-21T10:27:03.713 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:27:03 smithi047 ceph-mon[31547]: pgmap v55: 129 pgs: 129 active+clean; 29 MiB data, 447 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 2.4 MiB/s wr, 25 op/s 2024-03-21T10:27:03.728 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:27:03 smithi100 ceph-mon[35193]: pgmap v55: 129 pgs: 129 active+clean; 29 MiB data, 447 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 2.4 MiB/s wr, 25 op/s 2024-03-21T10:27:03.738 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:27:03 smithi157 ceph-mon[35581]: pgmap v55: 129 pgs: 129 active+clean; 29 MiB data, 447 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 2.4 MiB/s wr, 25 op/s 2024-03-21T10:27:05.713 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:27:05 smithi047 ceph-mon[31547]: pgmap v56: 129 pgs: 129 active+clean; 29 MiB data, 447 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 2.4 MiB/s wr, 20 op/s 2024-03-21T10:27:05.728 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:27:05 smithi100 ceph-mon[35193]: pgmap v56: 129 pgs: 129 active+clean; 29 MiB data, 447 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 2.4 MiB/s wr, 20 op/s 2024-03-21T10:27:05.738 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:27:05 smithi157 ceph-mon[35581]: pgmap v56: 129 pgs: 129 active+clean; 29 MiB data, 447 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 2.4 MiB/s wr, 20 op/s 2024-03-21T10:27:07.713 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:27:07 smithi047 ceph-mon[31547]: pgmap v57: 129 pgs: 129 active+clean; 57 MiB data, 501 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 4.7 MiB/s wr, 36 op/s 2024-03-21T10:27:07.728 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:27:07 smithi100 ceph-mon[35193]: pgmap v57: 129 pgs: 129 active+clean; 57 MiB data, 501 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 4.7 MiB/s wr, 36 op/s 2024-03-21T10:27:07.738 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:27:07 smithi157 ceph-mon[35581]: pgmap v57: 129 pgs: 129 active+clean; 57 MiB data, 501 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 4.7 MiB/s wr, 36 op/s 2024-03-21T10:27:09.708 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:09.707+0000 7fcd95800640 1 -- 172.21.15.47:0/1355934233 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcd901096a0 msgr2=0x7fcd90109a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:27:09.708 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:09.707+0000 7fcd95800640 1 --2- 172.21.15.47:0/1355934233 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcd901096a0 0x7fcd90109a80 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fcd80009900 tx=0x7fcd8002bc40 comp rx=0 tx=0).stop 2024-03-21T10:27:09.708 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:09.707+0000 7fcd95800640 1 -- 172.21.15.47:0/1355934233 shutdown_connections 2024-03-21T10:27:09.708 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:09.707+0000 7fcd95800640 1 --2- 172.21.15.47:0/1355934233 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fcd90104050 0x7fcd90110560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:27:09.708 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:09.707+0000 7fcd95800640 1 --2- 172.21.15.47:0/1355934233 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcd90103690 0x7fcd90103b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:27:09.708 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:09.707+0000 7fcd95800640 1 --2- 172.21.15.47:0/1355934233 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcd901096a0 0x7fcd90109a80 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:27:09.708 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:09.707+0000 7fcd95800640 1 -- 172.21.15.47:0/1355934233 >> 172.21.15.47:0/1355934233 conn(0x7fcd900ff180 msgr2=0x7fcd901015a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:27:09.709 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:09.707+0000 7fcd95800640 1 -- 172.21.15.47:0/1355934233 shutdown_connections 2024-03-21T10:27:09.709 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:09.707+0000 7fcd95800640 1 -- 172.21.15.47:0/1355934233 wait complete. 2024-03-21T10:27:09.709 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:09.708+0000 7fcd95800640 1 Processor -- start 2024-03-21T10:27:09.709 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:09.708+0000 7fcd95800640 1 -- start start 2024-03-21T10:27:09.710 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:09.709+0000 7fcd95800640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcd90103690 0x7fcd90199ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:27:09.710 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:09.709+0000 7fcd95800640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fcd90104050 0x7fcd9019a3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:27:09.710 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:09.709+0000 7fcd95800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcd901096a0 0x7fcd9019e760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:27:09.710 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:09.709+0000 7fcd95800640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd90113920 con 0x7fcd90104050 2024-03-21T10:27:09.711 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:09.709+0000 7fcd95800640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd901137a0 con 0x7fcd901096a0 2024-03-21T10:27:09.711 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:09.709+0000 7fcd95800640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd90113aa0 con 0x7fcd90103690 2024-03-21T10:27:09.711 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:09.709+0000 7fcd8f400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fcd90104050 0x7fcd9019a3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:27:09.711 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:09.709+0000 7fcd94e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcd901096a0 0x7fcd9019e760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:27:09.711 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:09.709+0000 7fcd94e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcd901096a0 0x7fcd9019e760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:43286/0 (socket says 172.21.15.47:43286) 2024-03-21T10:27:09.711 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:09.709+0000 7fcd94e00640 1 -- 172.21.15.47:0/1052330591 learned_addr learned my addr 172.21.15.47:0/1052330591 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:27:09.711 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:09.709+0000 7fcd8fe00640 1 --2- 172.21.15.47:0/1052330591 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcd90103690 0x7fcd90199ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:27:09.712 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:09.710+0000 7fcd94e00640 1 -- 172.21.15.47:0/1052330591 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcd90103690 msgr2=0x7fcd90199ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:27:09.712 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:09.710+0000 7fcd94e00640 1 --2- 172.21.15.47:0/1052330591 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcd90103690 0x7fcd90199ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:27:09.712 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:09.710+0000 7fcd94e00640 1 -- 172.21.15.47:0/1052330591 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fcd90104050 msgr2=0x7fcd9019a3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:27:09.712 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:09.710+0000 7fcd94e00640 1 --2- 172.21.15.47:0/1052330591 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fcd90104050 0x7fcd9019a3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:27:09.712 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:09.710+0000 7fcd94e00640 1 -- 172.21.15.47:0/1052330591 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd9019ee90 con 0x7fcd901096a0 2024-03-21T10:27:09.713 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:09.710+0000 7fcd8f400640 1 --2- 172.21.15.47:0/1052330591 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fcd90104050 0x7fcd9019a3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T10:27:09.713 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:09.710+0000 7fcd94e00640 1 --2- 172.21.15.47:0/1052330591 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcd901096a0 0x7fcd9019e760 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fcd8800eb10 tx=0x7fcd8800c2d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:27:09.713 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:09.710+0000 7fcd8cc00640 1 -- 172.21.15.47:0/1052330591 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd88004b20 con 0x7fcd901096a0 2024-03-21T10:27:09.713 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:09.711+0000 7fcd95800640 1 -- 172.21.15.47:0/1052330591 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcd9019f180 con 0x7fcd901096a0 2024-03-21T10:27:09.713 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:09.711+0000 7fcd8cc00640 1 -- 172.21.15.47:0/1052330591 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fcd88004cc0 con 0x7fcd901096a0 2024-03-21T10:27:09.713 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:09.711+0000 7fcd8cc00640 1 -- 172.21.15.47:0/1052330591 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd88010690 con 0x7fcd901096a0 2024-03-21T10:27:09.713 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:09.711+0000 7fcd95800640 1 -- 172.21.15.47:0/1052330591 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcd901a6a10 con 0x7fcd901096a0 2024-03-21T10:27:09.713 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:27:09 smithi047 ceph-mon[31547]: pgmap v58: 129 pgs: 129 active+clean; 57 MiB data, 501 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 4.7 MiB/s wr, 28 op/s 2024-03-21T10:27:09.716 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:09.715+0000 7fcd8cc00640 1 -- 172.21.15.47:0/1052330591 <== mon.2 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 27) v1 ==== 137985+0+0 (secure 0 0 0) 0x7fcd88005020 con 0x7fcd901096a0 2024-03-21T10:27:09.718 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:09.716+0000 7fcd99a09740 1 Processor -- start 2024-03-21T10:27:09.718 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:09.716+0000 7fcd99a09740 1 -- start start 2024-03-21T10:27:09.718 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:09.716+0000 7fcd8cc00640 1 --2- 172.21.15.47:0/1052330591 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7fcd740a3080 0x7fcd740a5540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:27:09.718 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:09.717+0000 7fcd99a09740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d3147c21e0 0x55d3147c25c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:27:09.719 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:09.717+0000 7fcd99a09740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d3147c2b00 0x55d3147cb390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:27:09.719 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:09.717+0000 7fcd8cc00640 1 -- 172.21.15.47:0/1052330591 <== mon.2 v2:172.21.15.100:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 9474+0+0 (secure 0 0 0) 0x7fcd880d00f0 con 0x7fcd901096a0 2024-03-21T10:27:09.719 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:09.717+0000 7fcd99a09740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d3147cb8d0 0x55d3147cdcc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:27:09.719 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:09.717+0000 7fcd99a09740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d3147843b0 con 0x55d3147c21e0 2024-03-21T10:27:09.719 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:09.717+0000 7fcd99a09740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d314726830 con 0x55d3147c2b00 2024-03-21T10:27:09.719 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:09.717+0000 7fcd99a09740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d3146cf5d0 con 0x55d3147cb8d0 2024-03-21T10:27:09.719 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:09.717+0000 7fcd8f400640 1 --2- 172.21.15.47:0/1052330591 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7fcd740a3080 0x7fcd740a5540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:27:09.719 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:09.717+0000 7fcd8fe00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d3147c21e0 0x55d3147c25c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:27:09.719 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:09.717+0000 7fcd8fe00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d3147c21e0 0x55d3147c25c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:49854/0 (socket says 172.21.15.47:49854) 2024-03-21T10:27:09.719 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:09.717+0000 7fcd8fe00640 1 -- 172.21.15.47:0/29329290 learned_addr learned my addr 172.21.15.47:0/29329290 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:27:09.719 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:09.717+0000 7fcd8f400640 1 --2- 172.21.15.47:0/29329290 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d3147c2b00 0x55d3147cb390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:27:09.719 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:09.717+0000 7fcd94e00640 1 --2- 172.21.15.47:0/29329290 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d3147cb8d0 0x55d3147cdcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:27:09.721 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:09.718+0000 7fcd8fe00640 1 -- 172.21.15.47:0/29329290 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d3147cb8d0 msgr2=0x55d3147cdcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:27:09.721 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:09.718+0000 7fcd8fe00640 1 --2- 172.21.15.47:0/29329290 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d3147cb8d0 0x55d3147cdcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:27:09.721 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:09.718+0000 7fcd8fe00640 1 -- 172.21.15.47:0/29329290 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d3147c2b00 msgr2=0x55d3147cb390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:27:09.721 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:09.718+0000 7fcd8fe00640 1 --2- 172.21.15.47:0/29329290 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d3147c2b00 0x55d3147cb390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:27:09.722 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:09.718+0000 7fcd8fe00640 1 -- 172.21.15.47:0/29329290 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d3146e72c0 con 0x55d3147c21e0 2024-03-21T10:27:09.722 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:09.718+0000 7fcd8f400640 1 --2- 172.21.15.47:0/1052330591 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7fcd740a3080 0x7fcd740a5540 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fcd90101880 tx=0x7fcd7c00bf10 comp rx=0 tx=0).ready entity=mgr.15687 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:27:09.722 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:09.718+0000 7fcd8fe00640 1 --2- 172.21.15.47:0/29329290 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d3147c21e0 0x55d3147c25c0 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7fcd80009900 tx=0x7fcd80009560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bbb5e035d4513f58 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:27:09.722 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:09.719+0000 7fcd85600640 1 -- 172.21.15.47:0/29329290 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd8002c810 con 0x55d3147c21e0 2024-03-21T10:27:09.722 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:09.719+0000 7fcd85600640 1 -- 172.21.15.47:0/29329290 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fcd8002c9b0 con 0x55d3147c21e0 2024-03-21T10:27:09.722 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:09.719+0000 7fcd85600640 1 -- 172.21.15.47:0/29329290 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd800046c0 con 0x55d3147c21e0 2024-03-21T10:27:09.722 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:09.719+0000 7fcd99a09740 1 -- 172.21.15.47:0/29329290 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d3147c21e0 msgr2=0x55d3147c25c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:27:09.722 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:09.719+0000 7fcd99a09740 1 --2- 172.21.15.47:0/29329290 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d3147c21e0 0x55d3147c25c0 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7fcd80009900 tx=0x7fcd80009560 comp rx=0 tx=0).stop 2024-03-21T10:27:09.722 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:09.720+0000 7fcd99a09740 1 -- 172.21.15.47:0/29329290 shutdown_connections 2024-03-21T10:27:09.723 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:09.720+0000 7fcd99a09740 1 --2- 172.21.15.47:0/29329290 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d3147cb8d0 0x55d3147cdcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:27:09.723 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:09.720+0000 7fcd99a09740 1 --2- 172.21.15.47:0/29329290 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d3147c2b00 0x55d3147cb390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:27:09.723 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:09.720+0000 7fcd99a09740 1 --2- 172.21.15.47:0/29329290 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d3147c21e0 0x55d3147c25c0 unknown :-1 s=CLOSED pgs=671 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:27:09.723 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:09.720+0000 7fcd99a09740 1 -- 172.21.15.47:0/29329290 >> 172.21.15.47:0/29329290 conn(0x55d3147bcb00 msgr2=0x55d3147bcee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:27:09.723 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:09.720+0000 7fcd99a09740 1 -- 172.21.15.47:0/29329290 shutdown_connections 2024-03-21T10:27:09.723 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:09.720+0000 7fcd99a09740 1 -- 172.21.15.47:0/29329290 wait complete. 2024-03-21T10:27:09.723 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:09.720+0000 7fcd99a09740 1 -- start start 2024-03-21T10:27:09.723 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:09.720+0000 7fcd99a09740 1 Processor -- start 2024-03-21T10:27:09.725 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:09.721+0000 7fcd99a09740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d3147c21e0 0x55d3147c8850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:27:09.726 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:09.721+0000 7fcd99a09740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d3147c2b00 0x55d3147c8d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:27:09.726 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:09.721+0000 7fcd99a09740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d3147cb8d0 0x55d3147c92d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:27:09.726 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:09.721+0000 7fcd99a09740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d3146e2a20 con 0x55d3147cb8d0 2024-03-21T10:27:09.726 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:09.721+0000 7fcd99a09740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d3147228d0 con 0x55d3147c2b00 2024-03-21T10:27:09.726 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:09.721+0000 7fcd99a09740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d3147ce4a0 con 0x55d3147c21e0 2024-03-21T10:27:09.726 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:09.721+0000 7fcd99a09740 10 client.0 _ll_register_callbacks cb 0x55d31471f1c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T10:27:09.726 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:09.721+0000 7fcd94e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d3147cb8d0 0x55d3147c92d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:27:09.726 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:09.721+0000 7fcd8f400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d3147c2b00 0x55d3147c8d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:27:09.726 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:09.721+0000 7fcd94e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d3147cb8d0 0x55d3147c92d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:49860/0 (socket says 172.21.15.47:49860) 2024-03-21T10:27:09.726 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:09.721+0000 7fcd94e00640 1 -- 172.21.15.47:0/2381539875 learned_addr learned my addr 172.21.15.47:0/2381539875 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:27:09.726 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:09.721+0000 7fcd8fe00640 1 --2- 172.21.15.47:0/2381539875 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d3147c21e0 0x55d3147c8850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:27:09.727 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:09.722+0000 7fcd94e00640 1 -- 172.21.15.47:0/2381539875 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d3147c21e0 msgr2=0x55d3147c8850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:27:09.727 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:09.722+0000 7fcd94e00640 1 --2- 172.21.15.47:0/2381539875 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d3147c21e0 0x55d3147c8850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:27:09.727 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:09.722+0000 7fcd94e00640 1 -- 172.21.15.47:0/2381539875 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d3147c2b00 msgr2=0x55d3147c8d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:27:09.727 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:09.722+0000 7fcd94e00640 1 --2- 172.21.15.47:0/2381539875 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d3147c2b00 0x55d3147c8d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:27:09.727 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:09.722+0000 7fcd94e00640 1 -- 172.21.15.47:0/2381539875 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55d31443b210 con 0x55d3147cb8d0 2024-03-21T10:27:09.727 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:09.722+0000 7fcd8f400640 1 --2- 172.21.15.47:0/2381539875 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d3147c2b00 0x55d3147c8d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T10:27:09.727 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:09.722+0000 7fcd94e00640 1 --2- 172.21.15.47:0/2381539875 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d3147cb8d0 0x55d3147c92d0 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7fcd8809c050 tx=0x7fcd8809bae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bea72b54278194ae server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:27:09.727 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:09.722+0000 7fcd7a000640 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-03-21T10:27:09.727 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:09.723+0000 7fcd7a000640 1 -- 172.21.15.47:0/2381539875 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd880bc2c0 con 0x55d3147cb8d0 2024-03-21T10:27:09.727 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:09.723+0000 7fcd7a000640 1 -- 172.21.15.47:0/2381539875 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(97..97 src has 1..97) v4 ==== 9474+0+0 (secure 0 0 0) 0x7fcd8809c320 con 0x55d3147cb8d0 2024-03-21T10:27:09.727 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:09.723+0000 7fcd99a09740 1 -- 172.21.15.47:0/2381539875 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fcd8002c9b0 con 0x55d3147cb8d0 2024-03-21T10:27:09.727 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:09.723+0000 7fcd7a000640 1 -- 172.21.15.47:0/2381539875 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7fcd880a0210 con 0x55d3147cb8d0 2024-03-21T10:27:09.727 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:09.723+0000 7fcd99a09740 10 client.16353 fetch_fsmap learned FSMap version 32 2024-03-21T10:27:09.728 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:09.723+0000 7fcd99a09740 1 -- 172.21.15.47:0/2381539875 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x55d314507620 con 0x55d3147cb8d0 2024-03-21T10:27:09.728 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:09.723+0000 7fcd7a000640 1 -- 172.21.15.47:0/2381539875 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fcd8002c9b0 con 0x55d3147cb8d0 2024-03-21T10:27:09.728 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:09.723+0000 7fcd7a000640 1 -- 172.21.15.47:0/2381539875 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 32) v1 ==== 7942+0+0 (secure 0 0 0) 0x7fcd88005310 con 0x55d3147cb8d0 2024-03-21T10:27:09.728 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:09.724+0000 7fcd99a09740 10 client.16353 fetch_fsmap finished waiting for FSMap version 32 2024-03-21T10:27:09.728 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:09.724+0000 7fcd99a09740 10 client.16353 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-21T10:27:09.728 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:09.724+0000 7fcd99a09740 20 client.16353 populate_metadata read hostname 'smithi047' 2024-03-21T10:27:09.728 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:09.724+0000 7fcd99a09740 1 --2- 172.21.15.47:0/2381539875 >> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] conn(0x55d3147db7c0 0x55d3147ddbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:27:09.728 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:09.724+0000 7fcd99a09740 4 client.16353 mds_command: new command op to 34535 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T10:27:09.728 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:09.724+0000 7fcd99a09740 1 -- 172.21.15.47:0/2381539875 --> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55d314733d70 con 0x55d3147db7c0 2024-03-21T10:27:09.728 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:09.724+0000 7fcd8fe00640 1 --2- 172.21.15.47:0/2381539875 >> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] conn(0x55d3147db7c0 0x55d3147ddbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:27:09.728 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:09.725+0000 7fcd8fe00640 1 --2- 172.21.15.47:0/2381539875 >> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] conn(0x55d3147db7c0 0x55d3147ddbb0 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=97e67b9ed9e7528 server_cookie=1837a8ba7b6e8fe7 in_seq=0 out_seq=0 2024-03-21T10:27:09.728 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:09.725+0000 7fcd7a000640 10 client.16353 ms_handle_connect on v2:172.21.15.100:6838/4173168440 2024-03-21T10:27:09.729 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:27:09 smithi100 ceph-mon[35193]: pgmap v58: 129 pgs: 129 active+clean; 57 MiB data, 501 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 4.7 MiB/s wr, 28 op/s 2024-03-21T10:27:09.730 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:09.726+0000 7fcd7a000640 1 -- 172.21.15.47:0/2381539875 <== mds.0 v2:172.21.15.100:6838/4173168440 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55d314733d70 con 0x55d3147db7c0 2024-03-21T10:27:09.730 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:09.726+0000 7fcd7a000640 10 client.16353 handle_command_reply: tid=0 2024-03-21T10:27:09.730 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:09.726+0000 7fcd99a09740 1 client.16353 shutdown 2024-03-21T10:27:09.730 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:09.726+0000 7fcd99a09740 1 -- 172.21.15.47:0/2381539875 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d3147cb8d0 msgr2=0x55d3147c92d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:27:09.730 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:09.726+0000 7fcd99a09740 1 --2- 172.21.15.47:0/2381539875 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d3147cb8d0 0x55d3147c92d0 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7fcd8809c050 tx=0x7fcd8809bae0 comp rx=0 tx=0).stop 2024-03-21T10:27:09.730 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:09.726+0000 7fcd99a09740 1 -- 172.21.15.47:0/2381539875 shutdown_connections 2024-03-21T10:27:09.731 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:09.726+0000 7fcd99a09740 1 --2- 172.21.15.47:0/2381539875 >> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] conn(0x55d3147db7c0 0x55d3147ddbb0 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:27:09.731 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:09.726+0000 7fcd99a09740 1 --2- 172.21.15.47:0/2381539875 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d3147cb8d0 0x55d3147c92d0 unknown :-1 s=CLOSED pgs=672 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:27:09.731 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:09.726+0000 7fcd99a09740 1 --2- 172.21.15.47:0/2381539875 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d3147c2b00 0x55d3147c8d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:27:09.731 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:09.726+0000 7fcd99a09740 1 --2- 172.21.15.47:0/2381539875 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d3147c21e0 0x55d3147c8850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:27:09.731 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:09.726+0000 7fcd99a09740 1 -- 172.21.15.47:0/2381539875 >> 172.21.15.47:0/2381539875 conn(0x55d3147bcb00 msgr2=0x55d3147be850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:27:09.731 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:09.726+0000 7fcd7a000640 0 client.16353 ms_handle_reset on v2:172.21.15.100:6838/4173168440 2024-03-21T10:27:09.731 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:09.727+0000 7fcd99a09740 1 -- 172.21.15.47:0/2381539875 shutdown_connections 2024-03-21T10:27:09.731 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:09.727+0000 7fcd99a09740 1 -- 172.21.15.47:0/2381539875 wait complete. 2024-03-21T10:27:09.731 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:09.727+0000 7fcd99a09740 20 client.16353 trim_cache size 0 max 16384 2024-03-21T10:27:09.738 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:27:09 smithi157 ceph-mon[35581]: pgmap v58: 129 pgs: 129 active+clean; 57 MiB data, 501 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 4.7 MiB/s wr, 28 op/s 2024-03-21T10:27:09.745 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:09.743+0000 7fcd99a09740 1 Processor -- start 2024-03-21T10:27:09.745 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:09.743+0000 7fcd99a09740 1 -- start start 2024-03-21T10:27:09.745 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:09.744+0000 7fcd99a09740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d3147c21e0 0x55d3147caf80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:27:09.745 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:09.744+0000 7fcd99a09740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d3147c2b00 0x55d3147c3bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:27:09.745 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:09.744+0000 7fcd99a09740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d3147cb8d0 0x55d3147d6bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:27:09.745 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:09.744+0000 7fcd99a09740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd901b09b0 con 0x55d3147cb8d0 2024-03-21T10:27:09.745 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:09.744+0000 7fcd99a09740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd901b2330 con 0x55d3147c21e0 2024-03-21T10:27:09.745 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:09.744+0000 7fcd99a09740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd901b1010 con 0x55d3147c2b00 2024-03-21T10:27:09.748 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:09.744+0000 7fcd94e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d3147cb8d0 0x55d3147d6bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:27:09.748 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:09.744+0000 7fcd8fe00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d3147c21e0 0x55d3147caf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:27:09.748 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:09.744+0000 7fcd8fe00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d3147c21e0 0x55d3147caf80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:43318/0 (socket says 172.21.15.47:43318) 2024-03-21T10:27:09.748 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:09.744+0000 7fcd94e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d3147cb8d0 0x55d3147d6bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:49864/0 (socket says 172.21.15.47:49864) 2024-03-21T10:27:09.748 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:09.744+0000 7fcd8fe00640 1 -- 172.21.15.47:0/3852734675 learned_addr learned my addr 172.21.15.47:0/3852734675 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:27:09.748 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:09.745+0000 7fcd8f400640 1 --2- 172.21.15.47:0/3852734675 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d3147c2b00 0x55d3147c3bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:27:09.748 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:09.745+0000 7fcd8fe00640 1 -- 172.21.15.47:0/3852734675 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d3147c2b00 msgr2=0x55d3147c3bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:27:09.749 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:09.745+0000 7fcd8fe00640 1 --2- 172.21.15.47:0/3852734675 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d3147c2b00 0x55d3147c3bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:27:09.749 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:09.745+0000 7fcd8fe00640 1 -- 172.21.15.47:0/3852734675 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d3147cb8d0 msgr2=0x55d3147d6bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:27:09.749 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:09.745+0000 7fcd8fe00640 1 --2- 172.21.15.47:0/3852734675 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d3147cb8d0 0x55d3147d6bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:27:09.749 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:09.745+0000 7fcd8fe00640 1 -- 172.21.15.47:0/3852734675 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d3147c2930 con 0x55d3147c21e0 2024-03-21T10:27:09.749 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:09.745+0000 7fcd94e00640 1 --2- 172.21.15.47:0/3852734675 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d3147cb8d0 0x55d3147d6bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T10:27:09.749 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:09.745+0000 7fcd8fe00640 1 --2- 172.21.15.47:0/3852734675 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d3147c21e0 0x55d3147caf80 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fcd80005330 tx=0x7fcd80005360 comp rx=0 tx=0).ready entity=mon.2 client_cookie=65890388766b8f5a server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:27:09.749 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:09.745+0000 7fcd84c00640 1 -- 172.21.15.47:0/3852734675 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd80050070 con 0x55d3147c21e0 2024-03-21T10:27:09.749 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:09.746+0000 7fcd84c00640 1 -- 172.21.15.47:0/3852734675 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fcd80005490 con 0x55d3147c21e0 2024-03-21T10:27:09.749 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:09.746+0000 7fcd84c00640 1 -- 172.21.15.47:0/3852734675 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd80003970 con 0x55d3147c21e0 2024-03-21T10:27:09.749 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:09.746+0000 7fcd99a09740 1 -- 172.21.15.47:0/3852734675 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d3147c21e0 msgr2=0x55d3147caf80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:27:09.749 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:09.746+0000 7fcd99a09740 1 --2- 172.21.15.47:0/3852734675 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d3147c21e0 0x55d3147caf80 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fcd80005330 tx=0x7fcd80005360 comp rx=0 tx=0).stop 2024-03-21T10:27:09.750 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:09.746+0000 7fcd99a09740 1 -- 172.21.15.47:0/3852734675 shutdown_connections 2024-03-21T10:27:09.750 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:09.746+0000 7fcd99a09740 1 --2- 172.21.15.47:0/3852734675 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d3147cb8d0 0x55d3147d6bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:27:09.750 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:09.746+0000 7fcd99a09740 1 --2- 172.21.15.47:0/3852734675 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d3147c2b00 0x55d3147c3bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:27:09.750 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:09.746+0000 7fcd99a09740 1 --2- 172.21.15.47:0/3852734675 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d3147c21e0 0x55d3147caf80 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:27:09.750 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:09.746+0000 7fcd99a09740 1 -- 172.21.15.47:0/3852734675 >> 172.21.15.47:0/3852734675 conn(0x55d3147bcb00 msgr2=0x55d3147b76a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:27:09.750 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:09.747+0000 7fcd99a09740 1 -- 172.21.15.47:0/3852734675 shutdown_connections 2024-03-21T10:27:09.750 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:09.747+0000 7fcd99a09740 1 -- 172.21.15.47:0/3852734675 wait complete. 2024-03-21T10:27:09.750 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:09.747+0000 7fcd99a09740 1 -- start start 2024-03-21T10:27:09.753 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:09.747+0000 7fcd99a09740 1 Processor -- start 2024-03-21T10:27:09.753 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:09.748+0000 7fcd99a09740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d3147c21e0 0x55d3147c6260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:27:09.753 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:09.748+0000 7fcd99a09740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d3147c2b00 0x55d3147c4b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:27:09.753 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:09.748+0000 7fcd99a09740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d3147cb8d0 0x55d3147c5060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:27:09.753 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:09.748+0000 7fcd99a09740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd901b1cd0 con 0x55d3147c2b00 2024-03-21T10:27:09.753 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:09.748+0000 7fcd99a09740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd901b1670 con 0x55d3147cb8d0 2024-03-21T10:27:09.753 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:09.748+0000 7fcd99a09740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd901aeb70 con 0x55d3147c21e0 2024-03-21T10:27:09.754 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:09.748+0000 7fcd99a09740 10 client.0 _ll_register_callbacks cb 0x7fcd88004a40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T10:27:09.754 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:09.748+0000 7fcd94e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d3147cb8d0 0x55d3147c5060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:27:09.754 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:09.748+0000 7fcd8f400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d3147c2b00 0x55d3147c4b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:27:09.754 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:09.748+0000 7fcd94e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d3147cb8d0 0x55d3147c5060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:43324/0 (socket says 172.21.15.47:43324) 2024-03-21T10:27:09.754 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:09.749+0000 7fcd94e00640 1 -- 172.21.15.47:0/1888278960 learned_addr learned my addr 172.21.15.47:0/1888278960 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:27:09.754 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:09.749+0000 7fcd94e00640 1 -- 172.21.15.47:0/1888278960 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d3147c21e0 msgr2=0x55d3147c6260 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-03-21T10:27:09.754 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:09.749+0000 7fcd94e00640 1 --2- 172.21.15.47:0/1888278960 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d3147c21e0 0x55d3147c6260 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:27:09.754 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:09.749+0000 7fcd94e00640 1 -- 172.21.15.47:0/1888278960 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d3147c2b00 msgr2=0x55d3147c4b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:27:09.754 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:09.749+0000 7fcd94e00640 1 --2- 172.21.15.47:0/1888278960 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d3147c2b00 0x55d3147c4b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:27:09.754 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:09.749+0000 7fcd94e00640 1 -- 172.21.15.47:0/1888278960 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55d3147c5720 con 0x55d3147cb8d0 2024-03-21T10:27:09.754 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:09.749+0000 7fcd8f400640 1 --2- 172.21.15.47:0/1888278960 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d3147c2b00 0x55d3147c4b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T10:27:09.754 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:09.749+0000 7fcd94e00640 1 --2- 172.21.15.47:0/1888278960 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d3147cb8d0 0x55d3147c5060 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fcd88004af0 tx=0x7fcd8800efd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6836ef25bc43047b server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:27:09.754 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:09.749+0000 7fcd7a000640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-21T10:27:09.755 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:09.750+0000 7fcd7a000640 1 -- 172.21.15.47:0/1888278960 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd8809c520 con 0x55d3147cb8d0 2024-03-21T10:27:09.755 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:09.750+0000 7fcd7a000640 1 -- 172.21.15.47:0/1888278960 <== mon.2 v2:172.21.15.100:3300/0 2 ==== osd_map(97..97 src has 1..97) v4 ==== 9474+0+0 (secure 0 0 0) 0x7fcd8809b510 con 0x55d3147cb8d0 2024-03-21T10:27:09.755 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:09.750+0000 7fcd99a09740 1 -- 172.21.15.47:0/1888278960 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fcd80005490 con 0x55d3147cb8d0 2024-03-21T10:27:09.755 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:09.750+0000 7fcd7a000640 1 -- 172.21.15.47:0/1888278960 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7fcd8809be90 con 0x55d3147cb8d0 2024-03-21T10:27:09.755 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:09.750+0000 7fcd99a09740 10 client.36146 fetch_fsmap learned FSMap version 32 2024-03-21T10:27:09.755 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:09.750+0000 7fcd99a09740 1 -- 172.21.15.47:0/1888278960 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x55d3147c5720 con 0x55d3147cb8d0 2024-03-21T10:27:09.755 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:09.750+0000 7fcd7a000640 1 -- 172.21.15.47:0/1888278960 <== mon.2 v2:172.21.15.100:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fcd80005490 con 0x55d3147cb8d0 2024-03-21T10:27:09.755 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:09.750+0000 7fcd7a000640 1 -- 172.21.15.47:0/1888278960 <== mon.2 v2:172.21.15.100:3300/0 5 ==== fsmap(e 32) v1 ==== 7942+0+0 (secure 0 0 0) 0x7fcd88097e10 con 0x55d3147cb8d0 2024-03-21T10:27:09.755 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:09.750+0000 7fcd99a09740 10 client.36146 fetch_fsmap finished waiting for FSMap version 32 2024-03-21T10:27:09.755 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:09.750+0000 7fcd99a09740 10 client.36146 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-21T10:27:09.755 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:09.751+0000 7fcd99a09740 20 client.36146 populate_metadata read hostname 'smithi047' 2024-03-21T10:27:09.755 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:09.751+0000 7fcd99a09740 1 --2- 172.21.15.47:0/1888278960 >> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] conn(0x55d314837210 0x55d314839600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:27:09.755 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:09.751+0000 7fcd99a09740 4 client.36146 mds_command: new command op to 34535 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-21T10:27:09.755 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:09.751+0000 7fcd99a09740 1 -- 172.21.15.47:0/1888278960 --> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55d31472e0c0 con 0x55d314837210 2024-03-21T10:27:09.756 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:09.751+0000 7fcd8fe00640 1 --2- 172.21.15.47:0/1888278960 >> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] conn(0x55d314837210 0x55d314839600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:27:09.756 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:09.752+0000 7fcd8fe00640 1 --2- 172.21.15.47:0/1888278960 >> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] conn(0x55d314837210 0x55d314839600 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=549ada7832902db7 server_cookie=fc85294ea2a532ff in_seq=0 out_seq=0 2024-03-21T10:27:09.756 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:09.752+0000 7fcd7a000640 10 client.36146 ms_handle_connect on v2:172.21.15.100:6838/4173168440 2024-03-21T10:27:09.770 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:09.769+0000 7fcd7a000640 1 -- 172.21.15.47:0/1888278960 <== mds.0 v2:172.21.15.100:6838/4173168440 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55d31472e0c0 con 0x55d314837210 2024-03-21T10:27:09.771 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:09.769+0000 7fcd7a000640 10 client.36146 handle_command_reply: tid=0 2024-03-21T10:27:09.771 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:09.769+0000 7fcd99a09740 1 client.36146 shutdown 2024-03-21T10:27:09.771 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:09.769+0000 7fcd99a09740 1 -- 172.21.15.47:0/1888278960 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d3147cb8d0 msgr2=0x55d3147c5060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:27:09.771 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:09.769+0000 7fcd99a09740 1 --2- 172.21.15.47:0/1888278960 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d3147cb8d0 0x55d3147c5060 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fcd88004af0 tx=0x7fcd8800efd0 comp rx=0 tx=0).stop 2024-03-21T10:27:09.771 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:09.769+0000 7fcd99a09740 1 -- 172.21.15.47:0/1888278960 shutdown_connections 2024-03-21T10:27:09.771 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:09.769+0000 7fcd99a09740 1 --2- 172.21.15.47:0/1888278960 >> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] conn(0x55d314837210 0x55d314839600 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:27:09.771 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:09.769+0000 7fcd99a09740 1 --2- 172.21.15.47:0/1888278960 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d3147cb8d0 0x55d3147c5060 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:27:09.771 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:09.769+0000 7fcd99a09740 1 --2- 172.21.15.47:0/1888278960 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d3147c2b00 0x55d3147c4b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:27:09.771 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:09.769+0000 7fcd99a09740 1 --2- 172.21.15.47:0/1888278960 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d3147c21e0 0x55d3147c6260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:27:09.771 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:09.769+0000 7fcd7a000640 0 client.36146 ms_handle_reset on v2:172.21.15.100:6838/4173168440 2024-03-21T10:27:09.771 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:09.769+0000 7fcd99a09740 1 -- 172.21.15.47:0/1888278960 >> 172.21.15.47:0/1888278960 conn(0x55d3147bcb00 msgr2=0x55d3147c3bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:27:09.771 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:09.770+0000 7fcd99a09740 1 -- 172.21.15.47:0/1888278960 shutdown_connections 2024-03-21T10:27:09.772 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:09.770+0000 7fcd99a09740 1 -- 172.21.15.47:0/1888278960 wait complete. 2024-03-21T10:27:09.772 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:09.770+0000 7fcd99a09740 20 client.36146 trim_cache size 0 max 16384 2024-03-21T10:27:09.772 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout:{ 2024-03-21T10:27:09.772 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout: "message": "", 2024-03-21T10:27:09.772 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout: "return_code": 0 2024-03-21T10:27:09.772 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout:} 2024-03-21T10:27:09.773 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:09.771+0000 7fcd95800640 1 -- 172.21.15.47:0/1052330591 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7fcd740a3080 msgr2=0x7fcd740a5540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:27:09.773 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:09.771+0000 7fcd95800640 1 --2- 172.21.15.47:0/1052330591 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7fcd740a3080 0x7fcd740a5540 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fcd90101880 tx=0x7fcd7c00bf10 comp rx=0 tx=0).stop 2024-03-21T10:27:09.773 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:09.771+0000 7fcd95800640 1 -- 172.21.15.47:0/1052330591 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcd901096a0 msgr2=0x7fcd9019e760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:27:09.773 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:09.771+0000 7fcd95800640 1 --2- 172.21.15.47:0/1052330591 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcd901096a0 0x7fcd9019e760 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fcd8800eb10 tx=0x7fcd8800c2d0 comp rx=0 tx=0).stop 2024-03-21T10:27:09.773 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:09.771+0000 7fcd95800640 1 -- 172.21.15.47:0/1052330591 shutdown_connections 2024-03-21T10:27:09.773 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:09.771+0000 7fcd95800640 1 --2- 172.21.15.47:0/1052330591 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7fcd740a3080 0x7fcd740a5540 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:27:09.773 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:09.771+0000 7fcd95800640 1 --2- 172.21.15.47:0/1052330591 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcd901096a0 0x7fcd9019e760 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:27:09.773 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:09.771+0000 7fcd95800640 1 --2- 172.21.15.47:0/1052330591 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fcd90104050 0x7fcd9019a3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:27:09.773 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:09.771+0000 7fcd95800640 1 --2- 172.21.15.47:0/1052330591 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcd90103690 0x7fcd90199ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:27:09.774 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:09.771+0000 7fcd95800640 1 -- 172.21.15.47:0/1052330591 >> 172.21.15.47:0/1052330591 conn(0x7fcd900ff180 msgr2=0x7fcd900ff9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:27:09.774 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:09.771+0000 7fcd95800640 1 -- 172.21.15.47:0/1052330591 shutdown_connections 2024-03-21T10:27:09.774 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:09.771+0000 7fcd95800640 1 -- 172.21.15.47:0/1052330591 wait complete. 2024-03-21T10:27:11.713 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:27:11 smithi047 ceph-mon[31547]: pgmap v59: 129 pgs: 129 active+clean; 57 MiB data, 501 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 4.7 MiB/s wr, 28 op/s 2024-03-21T10:27:11.728 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:27:11 smithi100 ceph-mon[35193]: pgmap v59: 129 pgs: 129 active+clean; 57 MiB data, 501 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 4.7 MiB/s wr, 28 op/s 2024-03-21T10:27:11.738 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:27:11 smithi157 ceph-mon[35581]: pgmap v59: 129 pgs: 129 active+clean; 57 MiB data, 501 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 4.7 MiB/s wr, 28 op/s 2024-03-21T10:27:13.713 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:27:13 smithi047 ceph-mon[31547]: pgmap v60: 129 pgs: 129 active+clean; 85 MiB data, 561 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 7.0 MiB/s wr, 36 op/s 2024-03-21T10:27:13.728 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:27:13 smithi100 ceph-mon[35193]: pgmap v60: 129 pgs: 129 active+clean; 85 MiB data, 561 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 7.0 MiB/s wr, 36 op/s 2024-03-21T10:27:13.738 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:27:13 smithi157 ceph-mon[35581]: pgmap v60: 129 pgs: 129 active+clean; 85 MiB data, 561 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 7.0 MiB/s wr, 36 op/s 2024-03-21T10:27:15.713 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:27:15 smithi047 ceph-mon[31547]: pgmap v61: 129 pgs: 129 active+clean; 85 MiB data, 561 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 4.7 MiB/s wr, 23 op/s 2024-03-21T10:27:15.728 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:27:15 smithi100 ceph-mon[35193]: pgmap v61: 129 pgs: 129 active+clean; 85 MiB data, 561 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 4.7 MiB/s wr, 23 op/s 2024-03-21T10:27:15.738 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:27:15 smithi157 ceph-mon[35581]: pgmap v61: 129 pgs: 129 active+clean; 85 MiB data, 561 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 4.7 MiB/s wr, 23 op/s 2024-03-21T10:27:17.713 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:27:17 smithi047 ceph-mon[31547]: pgmap v62: 129 pgs: 129 active+clean; 113 MiB data, 617 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 7.0 MiB/s wr, 27 op/s 2024-03-21T10:27:17.728 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:27:17 smithi100 ceph-mon[35193]: pgmap v62: 129 pgs: 129 active+clean; 113 MiB data, 617 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 7.0 MiB/s wr, 27 op/s 2024-03-21T10:27:17.738 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:27:17 smithi157 ceph-mon[35581]: pgmap v62: 129 pgs: 129 active+clean; 113 MiB data, 617 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 7.0 MiB/s wr, 27 op/s 2024-03-21T10:27:19.713 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:27:19 smithi047 ceph-mon[31547]: pgmap v63: 129 pgs: 129 active+clean; 113 MiB data, 617 MiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 4.7 MiB/s wr, 12 op/s 2024-03-21T10:27:19.728 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:27:19 smithi100 ceph-mon[35193]: pgmap v63: 129 pgs: 129 active+clean; 113 MiB data, 617 MiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 4.7 MiB/s wr, 12 op/s 2024-03-21T10:27:19.738 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:27:19 smithi157 ceph-mon[35581]: pgmap v63: 129 pgs: 129 active+clean; 113 MiB data, 617 MiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 4.7 MiB/s wr, 12 op/s 2024-03-21T10:27:21.579 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-21T10:27:21.579 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-21T10:27:21.672 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:27:21 smithi047 ceph-mon[31547]: pgmap v64: 129 pgs: 129 active+clean; 113 MiB data, 617 MiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 4.7 MiB/s wr, 12 op/s 2024-03-21T10:27:21.728 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:27:21 smithi100 ceph-mon[35193]: pgmap v64: 129 pgs: 129 active+clean; 113 MiB data, 617 MiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 4.7 MiB/s wr, 12 op/s 2024-03-21T10:27:21.738 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:27:21 smithi157 ceph-mon[35581]: pgmap v64: 129 pgs: 129 active+clean; 113 MiB data, 617 MiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 4.7 MiB/s wr, 12 op/s 2024-03-21T10:27:21.843 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:21.841+0000 7f942b200640 1 -- 172.21.15.47:0/3573570521 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f942c1075f0 msgr2=0x7f942c1079d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:27:21.843 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:21.841+0000 7f942b200640 1 --2- 172.21.15.47:0/3573570521 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f942c1075f0 0x7f942c1079d0 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7f941c009ca0 tx=0x7f941c02c100 comp rx=0 tx=0).stop 2024-03-21T10:27:21.843 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:21.842+0000 7f942b200640 1 -- 172.21.15.47:0/3573570521 shutdown_connections 2024-03-21T10:27:21.843 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:21.842+0000 7f942b200640 1 --2- 172.21.15.47:0/3573570521 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f942c101ef0 0x7f942c10e3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:27:21.844 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:21.842+0000 7f942b200640 1 --2- 172.21.15.47:0/3573570521 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f942c101530 0x7f942c1019b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:27:21.844 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:21.842+0000 7f942b200640 1 --2- 172.21.15.47:0/3573570521 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f942c1075f0 0x7f942c1079d0 unknown :-1 s=CLOSED pgs=673 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:27:21.844 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:21.842+0000 7f942b200640 1 -- 172.21.15.47:0/3573570521 >> 172.21.15.47:0/3573570521 conn(0x7f942c0fd040 msgr2=0x7f942c0ff460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:27:21.844 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:21.842+0000 7f942b200640 1 -- 172.21.15.47:0/3573570521 shutdown_connections 2024-03-21T10:27:21.844 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:21.842+0000 7f942b200640 1 -- 172.21.15.47:0/3573570521 wait complete. 2024-03-21T10:27:21.844 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:21.843+0000 7f942b200640 1 Processor -- start 2024-03-21T10:27:21.844 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:21.843+0000 7f942b200640 1 -- start start 2024-03-21T10:27:21.848 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:21.843+0000 7f942b200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f942c101530 0x7f942c199e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:27:21.848 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:21.843+0000 7f942b200640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f942c101ef0 0x7f942c19a3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:27:21.848 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:21.843+0000 7f942b200640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f942c1075f0 0x7f942c19e740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:27:21.848 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:21.843+0000 7f942b200640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f942c111780 con 0x7f942c1075f0 2024-03-21T10:27:21.848 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:21.843+0000 7f942b200640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f942c111600 con 0x7f942c101530 2024-03-21T10:27:21.848 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:21.843+0000 7f942b200640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f942c111900 con 0x7f942c101ef0 2024-03-21T10:27:21.849 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:21.844+0000 7f9429e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f942c101530 0x7f942c199e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:27:21.849 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:21.844+0000 7f942a800640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f942c1075f0 0x7f942c19e740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:27:21.849 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:21.844+0000 7f9429e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f942c101530 0x7f942c199e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:32896/0 (socket says 172.21.15.47:32896) 2024-03-21T10:27:21.849 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:21.844+0000 7f9429e00640 1 -- 172.21.15.47:0/1326372883 learned_addr learned my addr 172.21.15.47:0/1326372883 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:27:21.850 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:21.844+0000 7f9429400640 1 --2- 172.21.15.47:0/1326372883 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f942c101ef0 0x7f942c19a3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:27:21.850 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:21.844+0000 7f9429e00640 1 -- 172.21.15.47:0/1326372883 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f942c101ef0 msgr2=0x7f942c19a3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:27:21.850 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:21.844+0000 7f9429e00640 1 --2- 172.21.15.47:0/1326372883 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f942c101ef0 0x7f942c19a3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:27:21.850 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:21.844+0000 7f9429e00640 1 -- 172.21.15.47:0/1326372883 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f942c1075f0 msgr2=0x7f942c19e740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:27:21.850 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:21.844+0000 7f9429e00640 1 --2- 172.21.15.47:0/1326372883 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f942c1075f0 0x7f942c19e740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:27:21.850 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:21.844+0000 7f9429e00640 1 -- 172.21.15.47:0/1326372883 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f942c19ee70 con 0x7f942c101530 2024-03-21T10:27:21.850 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:21.845+0000 7f9429e00640 1 --2- 172.21.15.47:0/1326372883 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f942c101530 0x7f942c199e80 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f941c005e30 tx=0x7f941c035d60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:27:21.851 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:21.845+0000 7f9422a00640 1 -- 172.21.15.47:0/1326372883 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f941c039040 con 0x7f942c101530 2024-03-21T10:27:21.851 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:21.845+0000 7f942b200640 1 -- 172.21.15.47:0/1326372883 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f942c19f100 con 0x7f942c101530 2024-03-21T10:27:21.851 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:21.845+0000 7f9422a00640 1 -- 172.21.15.47:0/1326372883 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f941c044070 con 0x7f942c101530 2024-03-21T10:27:21.851 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:21.845+0000 7f9422a00640 1 -- 172.21.15.47:0/1326372883 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f941c004280 con 0x7f942c101530 2024-03-21T10:27:21.851 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:21.845+0000 7f942b200640 1 -- 172.21.15.47:0/1326372883 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f942c1a69f0 con 0x7f942c101530 2024-03-21T10:27:21.864 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:21.848+0000 7f9433bce740 1 Processor -- start 2024-03-21T10:27:21.864 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:21.849+0000 7f9433bce740 1 -- start start 2024-03-21T10:27:21.864 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:21.849+0000 7f9433bce740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564f0cb8fdd0 0x564f0cb8eb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:27:21.864 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:21.849+0000 7f9433bce740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x564f0cb8f0f0 0x564f0cb8f570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:27:21.864 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:21.849+0000 7f9433bce740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564f0cb98460 0x564f0cb9a850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:27:21.864 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:21.849+0000 7f9433bce740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x564f0caf3250 con 0x564f0cb8f0f0 2024-03-21T10:27:21.864 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:21.849+0000 7f9433bce740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x564f0ca9ba30 con 0x564f0cb8fdd0 2024-03-21T10:27:21.865 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:21.849+0000 7f9433bce740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x564f0caef2f0 con 0x564f0cb98460 2024-03-21T10:27:21.865 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:21.850+0000 7f942a800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564f0cb8fdd0 0x564f0cb8eb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:27:21.865 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:21.850+0000 7f942a800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564f0cb8fdd0 0x564f0cb8eb20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:32898/0 (socket says 172.21.15.47:32898) 2024-03-21T10:27:21.865 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:21.850+0000 7f942a800640 1 -- 172.21.15.47:0/3042397211 learned_addr learned my addr 172.21.15.47:0/3042397211 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:27:21.865 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:21.850+0000 7f9429400640 1 --2- 172.21.15.47:0/3042397211 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564f0cb98460 0x564f0cb9a850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:27:21.865 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:21.850+0000 7f942a800640 1 -- 172.21.15.47:0/3042397211 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564f0cb98460 msgr2=0x564f0cb9a850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:27:21.866 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:21.850+0000 7f942a800640 1 --2- 172.21.15.47:0/3042397211 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564f0cb98460 0x564f0cb9a850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:27:21.866 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:21.850+0000 7f942a800640 1 -- 172.21.15.47:0/3042397211 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x564f0cb8f0f0 msgr2=0x564f0cb8f570 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-21T10:27:21.866 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:21.850+0000 7f942a800640 1 --2- 172.21.15.47:0/3042397211 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x564f0cb8f0f0 0x564f0cb8f570 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:27:21.866 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:21.850+0000 7f942a800640 1 -- 172.21.15.47:0/3042397211 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564f0ca86690 con 0x564f0cb8fdd0 2024-03-21T10:27:21.866 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:21.851+0000 7f9422a00640 1 -- 172.21.15.47:0/1326372883 <== mon.2 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 27) v1 ==== 137985+0+0 (secure 0 0 0) 0x7f941c004420 con 0x7f942c101530 2024-03-21T10:27:21.867 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:21.851+0000 7f942a800640 1 --2- 172.21.15.47:0/3042397211 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564f0cb8fdd0 0x564f0cb8eb20 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f942400b700 tx=0x7f942400bbc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6c2a995110c67b9c server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:27:21.867 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:21.851+0000 7f941b400640 1 -- 172.21.15.47:0/3042397211 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9424003dd0 con 0x564f0cb8fdd0 2024-03-21T10:27:21.867 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:21.851+0000 7f941b400640 1 -- 172.21.15.47:0/3042397211 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f9424015070 con 0x564f0cb8fdd0 2024-03-21T10:27:21.867 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:21.851+0000 7f941b400640 1 -- 172.21.15.47:0/3042397211 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94240106a0 con 0x564f0cb8fdd0 2024-03-21T10:27:21.867 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:21.852+0000 7f9433bce740 1 -- 172.21.15.47:0/3042397211 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564f0cb8fdd0 msgr2=0x564f0cb8eb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:27:21.867 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:21.852+0000 7f9433bce740 1 --2- 172.21.15.47:0/3042397211 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564f0cb8fdd0 0x564f0cb8eb20 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f942400b700 tx=0x7f942400bbc0 comp rx=0 tx=0).stop 2024-03-21T10:27:21.867 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:21.852+0000 7f9433bce740 1 -- 172.21.15.47:0/3042397211 shutdown_connections 2024-03-21T10:27:21.868 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:21.852+0000 7f9433bce740 1 --2- 172.21.15.47:0/3042397211 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564f0cb98460 0x564f0cb9a850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:27:21.868 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:21.852+0000 7f9433bce740 1 --2- 172.21.15.47:0/3042397211 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x564f0cb8f0f0 0x564f0cb8f570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:27:21.868 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:21.852+0000 7f9433bce740 1 --2- 172.21.15.47:0/3042397211 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564f0cb8fdd0 0x564f0cb8eb20 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:27:21.868 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:21.852+0000 7f9433bce740 1 -- 172.21.15.47:0/3042397211 >> 172.21.15.47:0/3042397211 conn(0x564f0cb56200 msgr2=0x564f0cb89b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:27:21.868 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:21.852+0000 7f9422a00640 1 --2- 172.21.15.47:0/1326372883 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f94100a3090 0x7f94100a5550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:27:21.868 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:21.852+0000 7f9433bce740 1 -- 172.21.15.47:0/3042397211 shutdown_connections 2024-03-21T10:27:21.869 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:21.853+0000 7f9422a00640 1 -- 172.21.15.47:0/1326372883 <== mon.2 v2:172.21.15.100:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 9474+0+0 (secure 0 0 0) 0x7f941c0f1ec0 con 0x7f942c101530 2024-03-21T10:27:21.869 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:21.853+0000 7f9433bce740 1 -- 172.21.15.47:0/3042397211 wait complete. 2024-03-21T10:27:21.869 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:21.853+0000 7f9429400640 1 --2- 172.21.15.47:0/1326372883 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f94100a3090 0x7f94100a5550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:27:21.869 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:21.853+0000 7f9433bce740 1 -- start start 2024-03-21T10:27:21.869 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:21.854+0000 7f9433bce740 1 Processor -- start 2024-03-21T10:27:21.869 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:21.854+0000 7f9429400640 1 --2- 172.21.15.47:0/1326372883 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f94100a3090 0x7f94100a5550 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f942c10e7f0 tx=0x7f9414005f90 comp rx=0 tx=0).ready entity=mgr.15687 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:27:21.869 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:21.854+0000 7f9433bce740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x564f0cb8f0f0 0x564f0cb97c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:27:21.870 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:21.854+0000 7f9433bce740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564f0cb8fdd0 0x564f0cb94f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:27:21.870 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:21.854+0000 7f9433bce740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564f0cb98460 0x564f0cb954d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:27:21.870 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:21.854+0000 7f9433bce740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x564f0cb9b2d0 con 0x564f0cb8f0f0 2024-03-21T10:27:21.870 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:21.854+0000 7f9433bce740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x564f0caaee80 con 0x564f0cb98460 2024-03-21T10:27:21.870 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:21.854+0000 7f9433bce740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x564f0cb9b450 con 0x564f0cb8fdd0 2024-03-21T10:27:21.871 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:21.854+0000 7f9433bce740 10 client.0 _ll_register_callbacks cb 0x564f0cac1130 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T10:27:21.871 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:21.855+0000 7f9429e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x564f0cb8f0f0 0x564f0cb97c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:27:21.871 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:21.855+0000 7f942a800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564f0cb98460 0x564f0cb954d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:27:21.871 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:21.855+0000 7f942a800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564f0cb98460 0x564f0cb954d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:32908/0 (socket says 172.21.15.47:32908) 2024-03-21T10:27:21.871 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:21.855+0000 7f942a800640 1 -- 172.21.15.47:0/1324260369 learned_addr learned my addr 172.21.15.47:0/1324260369 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:27:21.871 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:21.855+0000 7f9429400640 1 --2- 172.21.15.47:0/1324260369 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564f0cb8fdd0 0x564f0cb94f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:27:21.871 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:21.855+0000 7f942a800640 1 -- 172.21.15.47:0/1324260369 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564f0cb8fdd0 msgr2=0x564f0cb94f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:27:21.872 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:21.855+0000 7f942a800640 1 --2- 172.21.15.47:0/1324260369 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564f0cb8fdd0 0x564f0cb94f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:27:21.872 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:21.855+0000 7f942a800640 1 -- 172.21.15.47:0/1324260369 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x564f0cb8f0f0 msgr2=0x564f0cb97c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:27:21.872 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:21.855+0000 7f942a800640 1 --2- 172.21.15.47:0/1324260369 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x564f0cb8f0f0 0x564f0cb97c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:27:21.872 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:21.855+0000 7f942a800640 1 -- 172.21.15.47:0/1324260369 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x564f0c8d17d0 con 0x564f0cb98460 2024-03-21T10:27:21.872 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:21.855+0000 7f9429e00640 1 --2- 172.21.15.47:0/1324260369 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x564f0cb8f0f0 0x564f0cb97c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T10:27:21.872 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:21.856+0000 7f942a800640 1 --2- 172.21.15.47:0/1324260369 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564f0cb98460 0x564f0cb954d0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f942400b6d0 tx=0x7f9424003e80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=997d00a25577bf75 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:27:21.873 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:21.856+0000 7f940fe00640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-21T10:27:21.873 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:21.856+0000 7f940fe00640 1 -- 172.21.15.47:0/1324260369 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9424028060 con 0x564f0cb98460 2024-03-21T10:27:21.873 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:21.856+0000 7f9433bce740 1 -- 172.21.15.47:0/1324260369 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f9424015070 con 0x564f0cb98460 2024-03-21T10:27:21.873 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:21.856+0000 7f940fe00640 1 -- 172.21.15.47:0/1324260369 <== mon.2 v2:172.21.15.100:3300/0 2 ==== osd_map(97..97 src has 1..97) v4 ==== 9474+0+0 (secure 0 0 0) 0x7f9424011650 con 0x564f0cb98460 2024-03-21T10:27:21.873 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:21.857+0000 7f940fe00640 1 -- 172.21.15.47:0/1324260369 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f942402f050 con 0x564f0cb98460 2024-03-21T10:27:21.873 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:21.857+0000 7f9433bce740 10 client.36158 fetch_fsmap learned FSMap version 32 2024-03-21T10:27:21.873 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:21.857+0000 7f9433bce740 1 -- 172.21.15.47:0/1324260369 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x564f0c8073f0 con 0x564f0cb98460 2024-03-21T10:27:21.874 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:21.857+0000 7f940fe00640 1 -- 172.21.15.47:0/1324260369 <== mon.2 v2:172.21.15.100:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f9424015070 con 0x564f0cb98460 2024-03-21T10:27:21.874 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:21.857+0000 7f940fe00640 1 -- 172.21.15.47:0/1324260369 <== mon.2 v2:172.21.15.100:3300/0 5 ==== fsmap(e 32) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f942401ab80 con 0x564f0cb98460 2024-03-21T10:27:21.874 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:21.857+0000 7f9433bce740 10 client.36158 fetch_fsmap finished waiting for FSMap version 32 2024-03-21T10:27:21.874 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:21.857+0000 7f9433bce740 10 client.36158 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-21T10:27:21.874 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:21.857+0000 7f9433bce740 20 client.36158 populate_metadata read hostname 'smithi047' 2024-03-21T10:27:21.874 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:21.858+0000 7f9433bce740 1 --2- 172.21.15.47:0/1324260369 >> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] conn(0x564f0cba8360 0x564f0cbaa750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:27:21.875 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:21.858+0000 7f9433bce740 4 client.36158 mds_command: new command op to 34535 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T10:27:21.875 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:21.858+0000 7f9433bce740 1 -- 172.21.15.47:0/1324260369 --> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x564f0cb00790 con 0x564f0cba8360 2024-03-21T10:27:21.875 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:21.858+0000 7f9429e00640 1 --2- 172.21.15.47:0/1324260369 >> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] conn(0x564f0cba8360 0x564f0cbaa750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:27:21.875 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:21.859+0000 7f9429e00640 1 --2- 172.21.15.47:0/1324260369 >> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] conn(0x564f0cba8360 0x564f0cbaa750 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4cc6a998da37a694 server_cookie=beda88fbff475dcd in_seq=0 out_seq=0 2024-03-21T10:27:21.875 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:21.859+0000 7f940fe00640 10 client.36158 ms_handle_connect on v2:172.21.15.100:6838/4173168440 2024-03-21T10:27:21.875 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:21.860+0000 7f940fe00640 1 -- 172.21.15.47:0/1324260369 <== mds.0 v2:172.21.15.100:6838/4173168440 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x564f0cb00790 con 0x564f0cba8360 2024-03-21T10:27:21.875 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:21.860+0000 7f940fe00640 10 client.36158 handle_command_reply: tid=0 2024-03-21T10:27:21.876 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:21.860+0000 7f9433bce740 1 client.36158 shutdown 2024-03-21T10:27:21.876 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:21.860+0000 7f9433bce740 1 -- 172.21.15.47:0/1324260369 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564f0cb98460 msgr2=0x564f0cb954d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:27:21.876 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:21.860+0000 7f9433bce740 1 --2- 172.21.15.47:0/1324260369 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564f0cb98460 0x564f0cb954d0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f942400b6d0 tx=0x7f9424003e80 comp rx=0 tx=0).stop 2024-03-21T10:27:21.876 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:21.860+0000 7f9433bce740 1 -- 172.21.15.47:0/1324260369 shutdown_connections 2024-03-21T10:27:21.876 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:21.860+0000 7f9433bce740 1 --2- 172.21.15.47:0/1324260369 >> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] conn(0x564f0cba8360 0x564f0cbaa750 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:27:21.876 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:21.860+0000 7f9433bce740 1 --2- 172.21.15.47:0/1324260369 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564f0cb98460 0x564f0cb954d0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:27:21.877 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:21.860+0000 7f9433bce740 1 --2- 172.21.15.47:0/1324260369 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564f0cb8fdd0 0x564f0cb94f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:27:21.877 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:21.860+0000 7f9433bce740 1 --2- 172.21.15.47:0/1324260369 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x564f0cb8f0f0 0x564f0cb97c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:27:21.877 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:21.860+0000 7f940fe00640 0 client.36158 ms_handle_reset on v2:172.21.15.100:6838/4173168440 2024-03-21T10:27:21.877 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:21.860+0000 7f9433bce740 1 -- 172.21.15.47:0/1324260369 >> 172.21.15.47:0/1324260369 conn(0x564f0cb56200 msgr2=0x564f0cb8b890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:27:21.877 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:21.860+0000 7f9433bce740 1 -- 172.21.15.47:0/1324260369 shutdown_connections 2024-03-21T10:27:21.877 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:21.860+0000 7f9433bce740 1 -- 172.21.15.47:0/1324260369 wait complete. 2024-03-21T10:27:21.878 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:21.861+0000 7f9433bce740 20 client.36158 trim_cache size 0 max 16384 2024-03-21T10:27:21.899 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:21.888+0000 7f9433bce740 1 Processor -- start 2024-03-21T10:27:21.899 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:21.888+0000 7f9433bce740 1 -- start start 2024-03-21T10:27:21.900 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:21.889+0000 7f9433bce740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x564f0cb8f0f0 0x564f0cb976c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:27:21.900 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:21.889+0000 7f9433bce740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564f0cb8fdd0 0x564f0cb97c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:27:21.900 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:21.889+0000 7f9433bce740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564f0cb98460 0x564f0cba3ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:27:21.900 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:21.889+0000 7f9433bce740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f942c1b0840 con 0x564f0cb8f0f0 2024-03-21T10:27:21.900 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:21.889+0000 7f9433bce740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f942c1b21c0 con 0x564f0cb98460 2024-03-21T10:27:21.900 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:21.889+0000 7f9433bce740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f942c1b0ea0 con 0x564f0cb8fdd0 2024-03-21T10:27:21.900 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:21.889+0000 7f9429e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x564f0cb8f0f0 0x564f0cb976c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:27:21.901 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:21.889+0000 7f9429e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x564f0cb8f0f0 0x564f0cb976c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:34566/0 (socket says 172.21.15.47:34566) 2024-03-21T10:27:21.901 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:21.889+0000 7f9429e00640 1 -- 172.21.15.47:0/2598206285 learned_addr learned my addr 172.21.15.47:0/2598206285 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:27:21.901 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:21.889+0000 7f942a800640 1 --2- 172.21.15.47:0/2598206285 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564f0cb98460 0x564f0cba3ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:27:21.901 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:21.889+0000 7f9429400640 1 --2- 172.21.15.47:0/2598206285 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564f0cb8fdd0 0x564f0cb97c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:27:21.901 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:21.890+0000 7f9429e00640 1 -- 172.21.15.47:0/2598206285 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564f0cb8fdd0 msgr2=0x564f0cb97c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:27:21.901 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:21.890+0000 7f9429e00640 1 --2- 172.21.15.47:0/2598206285 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564f0cb8fdd0 0x564f0cb97c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:27:21.902 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:21.890+0000 7f9429e00640 1 -- 172.21.15.47:0/2598206285 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564f0cb98460 msgr2=0x564f0cba3ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:27:21.902 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:21.890+0000 7f9429e00640 1 --2- 172.21.15.47:0/2598206285 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564f0cb98460 0x564f0cba3ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:27:21.902 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:21.890+0000 7f9429e00640 1 -- 172.21.15.47:0/2598206285 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564f0cb9ac10 con 0x564f0cb8f0f0 2024-03-21T10:27:21.902 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:21.890+0000 7f9429e00640 1 --2- 172.21.15.47:0/2598206285 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x564f0cb8f0f0 0x564f0cb976c0 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7f941c0bdca0 tx=0x7f941c035f50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=261410bb465d3169 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:27:21.902 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:21.890+0000 7f941aa00640 1 -- 172.21.15.47:0/2598206285 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f941c0bdfa0 con 0x564f0cb8f0f0 2024-03-21T10:27:21.902 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:21.890+0000 7f941aa00640 1 -- 172.21.15.47:0/2598206285 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f941c0c20f0 con 0x564f0cb8f0f0 2024-03-21T10:27:21.902 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:21.890+0000 7f941aa00640 1 -- 172.21.15.47:0/2598206285 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f941c0ba610 con 0x564f0cb8f0f0 2024-03-21T10:27:21.903 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:21.891+0000 7f9433bce740 1 -- 172.21.15.47:0/2598206285 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x564f0cb8f0f0 msgr2=0x564f0cb976c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:27:21.903 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:21.891+0000 7f9433bce740 1 --2- 172.21.15.47:0/2598206285 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x564f0cb8f0f0 0x564f0cb976c0 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7f941c0bdca0 tx=0x7f941c035f50 comp rx=0 tx=0).stop 2024-03-21T10:27:21.903 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:21.891+0000 7f9433bce740 1 -- 172.21.15.47:0/2598206285 shutdown_connections 2024-03-21T10:27:21.903 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:21.891+0000 7f9433bce740 1 --2- 172.21.15.47:0/2598206285 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564f0cb98460 0x564f0cba3ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:27:21.903 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:21.891+0000 7f9433bce740 1 --2- 172.21.15.47:0/2598206285 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564f0cb8fdd0 0x564f0cb97c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:27:21.903 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:21.891+0000 7f9433bce740 1 --2- 172.21.15.47:0/2598206285 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x564f0cb8f0f0 0x564f0cb976c0 unknown :-1 s=CLOSED pgs=674 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:27:21.904 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:21.891+0000 7f9433bce740 1 -- 172.21.15.47:0/2598206285 >> 172.21.15.47:0/2598206285 conn(0x564f0cb56200 msgr2=0x564f0cb89670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:27:21.904 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:21.891+0000 7f9433bce740 1 -- 172.21.15.47:0/2598206285 shutdown_connections 2024-03-21T10:27:21.904 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:21.891+0000 7f9433bce740 1 -- 172.21.15.47:0/2598206285 wait complete. 2024-03-21T10:27:21.904 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:21.892+0000 7f9433bce740 1 -- start start 2024-03-21T10:27:21.904 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:21.892+0000 7f9433bce740 1 Processor -- start 2024-03-21T10:27:21.905 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:21.893+0000 7f9433bce740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x564f0cb8f0f0 0x564f0cb99fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:27:21.905 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:21.893+0000 7f9433bce740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564f0cb8fdd0 0x564f0cb98840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:27:21.905 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:21.893+0000 7f9433bce740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564f0cb98460 0x564f0cb98d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:27:21.905 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:21.893+0000 7f9433bce740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f942c1b1b60 con 0x564f0cb8f0f0 2024-03-21T10:27:21.905 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:21.893+0000 7f9433bce740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f942c1b1500 con 0x564f0cb98460 2024-03-21T10:27:21.905 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:21.893+0000 7f9433bce740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f942c1aea00 con 0x564f0cb8fdd0 2024-03-21T10:27:21.906 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:21.893+0000 7f9433bce740 10 client.0 _ll_register_callbacks cb 0x7f9414009590 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T10:27:21.906 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:21.893+0000 7f9429e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x564f0cb8f0f0 0x564f0cb99fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:27:21.906 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:21.893+0000 7f9429e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x564f0cb8f0f0 0x564f0cb99fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:34582/0 (socket says 172.21.15.47:34582) 2024-03-21T10:27:21.906 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:21.893+0000 7f9429e00640 1 -- 172.21.15.47:0/1755241631 learned_addr learned my addr 172.21.15.47:0/1755241631 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:27:21.906 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:21.893+0000 7f942a800640 1 --2- 172.21.15.47:0/1755241631 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564f0cb98460 0x564f0cb98d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:27:21.907 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:21.893+0000 7f9429400640 1 --2- 172.21.15.47:0/1755241631 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564f0cb8fdd0 0x564f0cb98840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:27:21.907 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:21.893+0000 7f9429e00640 1 -- 172.21.15.47:0/1755241631 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564f0cb8fdd0 msgr2=0x564f0cb98840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:27:21.907 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:21.894+0000 7f9429e00640 1 --2- 172.21.15.47:0/1755241631 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564f0cb8fdd0 0x564f0cb98840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:27:21.907 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:21.894+0000 7f9429e00640 1 -- 172.21.15.47:0/1755241631 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564f0cb98460 msgr2=0x564f0cb98d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:27:21.907 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:21.894+0000 7f9429e00640 1 --2- 172.21.15.47:0/1755241631 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564f0cb98460 0x564f0cb98d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:27:21.907 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:21.894+0000 7f9429e00640 1 -- 172.21.15.47:0/1755241631 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x564f0c7c70f0 con 0x564f0cb8f0f0 2024-03-21T10:27:21.908 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:21.894+0000 7f942a800640 1 --2- 172.21.15.47:0/1755241631 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564f0cb98460 0x564f0cb98d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T10:27:21.908 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:21.894+0000 7f9429e00640 1 --2- 172.21.15.47:0/1755241631 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x564f0cb8f0f0 0x564f0cb99fc0 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7f941c0bdbd0 tx=0x7f941c009a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d2ab988934ab9b2f server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:27:21.908 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:21.894+0000 7f940fe00640 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-03-21T10:27:21.908 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:21.894+0000 7f940fe00640 1 -- 172.21.15.47:0/1755241631 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f941c0bdfa0 con 0x564f0cb8f0f0 2024-03-21T10:27:21.908 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:21.894+0000 7f940fe00640 1 -- 172.21.15.47:0/1755241631 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(97..97 src has 1..97) v4 ==== 9474+0+0 (secure 0 0 0) 0x7f941c0de050 con 0x564f0cb8f0f0 2024-03-21T10:27:21.908 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:21.894+0000 7f9433bce740 1 -- 172.21.15.47:0/1755241631 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f941c0c20f0 con 0x564f0cb8f0f0 2024-03-21T10:27:21.909 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:21.894+0000 7f940fe00640 1 -- 172.21.15.47:0/1755241631 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f941c0c69b0 con 0x564f0cb8f0f0 2024-03-21T10:27:21.909 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:21.895+0000 7f9433bce740 10 client.16374 fetch_fsmap learned FSMap version 32 2024-03-21T10:27:21.909 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:21.895+0000 7f9433bce740 1 -- 172.21.15.47:0/1755241631 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x564f0cba0440 con 0x564f0cb8f0f0 2024-03-21T10:27:21.909 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:21.895+0000 7f940fe00640 1 -- 172.21.15.47:0/1755241631 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f941c0c20f0 con 0x564f0cb8f0f0 2024-03-21T10:27:21.909 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:21.895+0000 7f940fe00640 1 -- 172.21.15.47:0/1755241631 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 32) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f941c03c030 con 0x564f0cb8f0f0 2024-03-21T10:27:21.909 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:21.895+0000 7f9433bce740 10 client.16374 fetch_fsmap finished waiting for FSMap version 32 2024-03-21T10:27:21.910 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:21.895+0000 7f9433bce740 10 client.16374 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-21T10:27:21.910 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:21.895+0000 7f9433bce740 20 client.16374 populate_metadata read hostname 'smithi047' 2024-03-21T10:27:21.910 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:21.895+0000 7f9433bce740 1 --2- 172.21.15.47:0/1755241631 >> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] conn(0x564f0cc03d20 0x564f0cc06140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:27:21.910 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:21.895+0000 7f9433bce740 4 client.16374 mds_command: new command op to 34535 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-21T10:27:21.910 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:21.895+0000 7f9433bce740 1 -- 172.21.15.47:0/1755241631 --> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x564f0cafaae0 con 0x564f0cc03d20 2024-03-21T10:27:21.910 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:21.896+0000 7f9429400640 1 --2- 172.21.15.47:0/1755241631 >> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] conn(0x564f0cc03d20 0x564f0cc06140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:27:21.910 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:21.896+0000 7f9429400640 1 --2- 172.21.15.47:0/1755241631 >> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] conn(0x564f0cc03d20 0x564f0cc06140 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=84e654d94801e597 server_cookie=5a7fe5e71b9f4ed9 in_seq=0 out_seq=0 2024-03-21T10:27:21.910 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:21.896+0000 7f940fe00640 10 client.16374 ms_handle_connect on v2:172.21.15.100:6838/4173168440 2024-03-21T10:27:21.910 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:21.897+0000 7f940fe00640 1 -- 172.21.15.47:0/1755241631 <== mds.0 v2:172.21.15.100:6838/4173168440 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x564f0cafaae0 con 0x564f0cc03d20 2024-03-21T10:27:21.910 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:21.897+0000 7f940fe00640 10 client.16374 handle_command_reply: tid=0 2024-03-21T10:27:21.910 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:21.897+0000 7f9433bce740 1 client.16374 shutdown 2024-03-21T10:27:21.911 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:21.897+0000 7f9433bce740 1 -- 172.21.15.47:0/1755241631 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x564f0cb8f0f0 msgr2=0x564f0cb99fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:27:21.911 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:21.897+0000 7f9433bce740 1 --2- 172.21.15.47:0/1755241631 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x564f0cb8f0f0 0x564f0cb99fc0 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7f941c0bdbd0 tx=0x7f941c009a20 comp rx=0 tx=0).stop 2024-03-21T10:27:21.911 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:21.897+0000 7f9433bce740 1 -- 172.21.15.47:0/1755241631 shutdown_connections 2024-03-21T10:27:21.911 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:21.897+0000 7f9433bce740 1 --2- 172.21.15.47:0/1755241631 >> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] conn(0x564f0cc03d20 0x564f0cc06140 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:27:21.911 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:21.897+0000 7f9433bce740 1 --2- 172.21.15.47:0/1755241631 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564f0cb98460 0x564f0cb98d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:27:21.911 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:21.897+0000 7f9433bce740 1 --2- 172.21.15.47:0/1755241631 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564f0cb8fdd0 0x564f0cb98840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:27:21.911 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:21.897+0000 7f9433bce740 1 --2- 172.21.15.47:0/1755241631 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x564f0cb8f0f0 0x564f0cb99fc0 unknown :-1 s=CLOSED pgs=675 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:27:21.911 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:21.897+0000 7f9433bce740 1 -- 172.21.15.47:0/1755241631 >> 172.21.15.47:0/1755241631 conn(0x564f0cb56200 msgr2=0x564f0cb94d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:27:21.911 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:21.897+0000 7f940fe00640 0 client.16374 ms_handle_reset on v2:172.21.15.100:6838/4173168440 2024-03-21T10:27:21.911 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:21.897+0000 7f9433bce740 1 -- 172.21.15.47:0/1755241631 shutdown_connections 2024-03-21T10:27:21.911 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:21.897+0000 7f9433bce740 1 -- 172.21.15.47:0/1755241631 wait complete. 2024-03-21T10:27:21.911 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:21.898+0000 7f9433bce740 20 client.16374 trim_cache size 0 max 16384 2024-03-21T10:27:21.912 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-03-21T10:27:21.912 INFO:teuthology.orchestra.run.smithi047.stdout: "status": "no active scrubs running", 2024-03-21T10:27:21.912 INFO:teuthology.orchestra.run.smithi047.stdout: "scrubs": {} 2024-03-21T10:27:21.912 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-03-21T10:27:21.912 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:21.898+0000 7f942b200640 1 -- 172.21.15.47:0/1326372883 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f94100a3090 msgr2=0x7f94100a5550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:27:21.913 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:21.898+0000 7f942b200640 1 --2- 172.21.15.47:0/1326372883 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f94100a3090 0x7f94100a5550 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f942c10e7f0 tx=0x7f9414005f90 comp rx=0 tx=0).stop 2024-03-21T10:27:21.913 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:21.898+0000 7f942b200640 1 -- 172.21.15.47:0/1326372883 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f942c101530 msgr2=0x7f942c199e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:27:21.913 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:21.898+0000 7f942b200640 1 --2- 172.21.15.47:0/1326372883 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f942c101530 0x7f942c199e80 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f941c005e30 tx=0x7f941c035d60 comp rx=0 tx=0).stop 2024-03-21T10:27:21.913 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:21.898+0000 7f942b200640 1 -- 172.21.15.47:0/1326372883 shutdown_connections 2024-03-21T10:27:21.913 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:21.898+0000 7f942b200640 1 --2- 172.21.15.47:0/1326372883 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f942c1075f0 0x7f942c19e740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:27:21.913 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:21.898+0000 7f942b200640 1 --2- 172.21.15.47:0/1326372883 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f942c101ef0 0x7f942c19a3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:27:21.913 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:21.898+0000 7f942b200640 1 --2- 172.21.15.47:0/1326372883 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f94100a3090 0x7f94100a5550 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:27:21.913 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:21.898+0000 7f942b200640 1 --2- 172.21.15.47:0/1326372883 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f942c101530 0x7f942c199e80 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:27:21.913 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:21.898+0000 7f942b200640 1 -- 172.21.15.47:0/1326372883 >> 172.21.15.47:0/1326372883 conn(0x7f942c0fd040 msgr2=0x7f942c0fe660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:27:21.913 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:21.899+0000 7f942b200640 1 -- 172.21.15.47:0/1326372883 shutdown_connections 2024-03-21T10:27:21.913 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:21.899+0000 7f942b200640 1 -- 172.21.15.47:0/1326372883 wait complete. 2024-03-21T10:27:21.919 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-03-21T10:27:21.919 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-21T10:27:22.171 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:22.171+0000 7fbac3800640 1 -- 172.21.15.47:0/2529948461 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbabc1041f0 msgr2=0x7fbabc110700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:27:22.172 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:22.171+0000 7fbac3800640 1 --2- 172.21.15.47:0/2529948461 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbabc1041f0 0x7fbabc110700 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7fbab4009cd0 tx=0x7fbab402c0e0 comp rx=0 tx=0).stop 2024-03-21T10:27:22.172 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:22.171+0000 7fbac3800640 1 -- 172.21.15.47:0/2529948461 shutdown_connections 2024-03-21T10:27:22.172 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:22.171+0000 7fbac3800640 1 --2- 172.21.15.47:0/2529948461 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbabc1041f0 0x7fbabc110700 unknown :-1 s=CLOSED pgs=676 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:27:22.172 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:22.171+0000 7fbac3800640 1 --2- 172.21.15.47:0/2529948461 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbabc103830 0x7fbabc103cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:27:22.172 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:22.171+0000 7fbac3800640 1 --2- 172.21.15.47:0/2529948461 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbabc109840 0x7fbabc109c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:27:22.172 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:22.171+0000 7fbac3800640 1 -- 172.21.15.47:0/2529948461 >> 172.21.15.47:0/2529948461 conn(0x7fbabc0ff300 msgr2=0x7fbabc101720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:27:22.172 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:22.171+0000 7fbac3800640 1 -- 172.21.15.47:0/2529948461 shutdown_connections 2024-03-21T10:27:22.172 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:22.171+0000 7fbac3800640 1 -- 172.21.15.47:0/2529948461 wait complete. 2024-03-21T10:27:22.173 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:22.173+0000 7fbac3800640 1 Processor -- start 2024-03-21T10:27:22.173 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:22.173+0000 7fbac3800640 1 -- start start 2024-03-21T10:27:22.174 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:22.173+0000 7fbac3800640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbabc103830 0x7fbabc19a080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:27:22.174 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:22.173+0000 7fbac3800640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbabc1041f0 0x7fbabc19a5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:27:22.174 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:22.173+0000 7fbac3800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbabc109840 0x7fbabc19e940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:27:22.174 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:22.173+0000 7fbac3800640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbabc113b00 con 0x7fbabc1041f0 2024-03-21T10:27:22.174 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:22.173+0000 7fbac3800640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbabc113980 con 0x7fbabc109840 2024-03-21T10:27:22.174 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:22.173+0000 7fbac3800640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbabc113c80 con 0x7fbabc103830 2024-03-21T10:27:22.174 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:22.174+0000 7fbac1800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbabc109840 0x7fbabc19e940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:27:22.174 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:22.174+0000 7fbabbe00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbabc1041f0 0x7fbabc19a5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:27:22.174 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:22.174+0000 7fbac1800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbabc109840 0x7fbabc19e940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:44994/0 (socket says 172.21.15.47:44994) 2024-03-21T10:27:22.174 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:22.174+0000 7fbac1800640 1 -- 172.21.15.47:0/1024567717 learned_addr learned my addr 172.21.15.47:0/1024567717 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:27:22.175 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:22.174+0000 7fbac0e00640 1 --2- 172.21.15.47:0/1024567717 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbabc103830 0x7fbabc19a080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:27:22.176 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:22.175+0000 7fbabbe00640 1 -- 172.21.15.47:0/1024567717 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbabc103830 msgr2=0x7fbabc19a080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:27:22.176 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:22.175+0000 7fbabbe00640 1 --2- 172.21.15.47:0/1024567717 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbabc103830 0x7fbabc19a080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:27:22.176 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:22.175+0000 7fbabbe00640 1 -- 172.21.15.47:0/1024567717 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbabc109840 msgr2=0x7fbabc19e940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:27:22.176 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:22.175+0000 7fbabbe00640 1 --2- 172.21.15.47:0/1024567717 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbabc109840 0x7fbabc19e940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:27:22.176 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:22.175+0000 7fbabbe00640 1 -- 172.21.15.47:0/1024567717 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbabc19f070 con 0x7fbabc1041f0 2024-03-21T10:27:22.176 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:22.175+0000 7fbac1800640 1 --2- 172.21.15.47:0/1024567717 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbabc109840 0x7fbabc19e940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T10:27:22.176 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:22.175+0000 7fbabbe00640 1 --2- 172.21.15.47:0/1024567717 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbabc1041f0 0x7fbabc19a5c0 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7fbaa800e9f0 tx=0x7fbaa800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:27:22.176 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:22.175+0000 7fbab9600640 1 -- 172.21.15.47:0/1024567717 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbaa800ccc0 con 0x7fbabc1041f0 2024-03-21T10:27:22.176 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:22.175+0000 7fbac3800640 1 -- 172.21.15.47:0/1024567717 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbabc19f360 con 0x7fbabc1041f0 2024-03-21T10:27:22.177 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:22.175+0000 7fbab9600640 1 -- 172.21.15.47:0/1024567717 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fbaa80108e0 con 0x7fbabc1041f0 2024-03-21T10:27:22.177 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:22.175+0000 7fbab9600640 1 -- 172.21.15.47:0/1024567717 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbaa8018880 con 0x7fbabc1041f0 2024-03-21T10:27:22.177 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:22.176+0000 7fbac3800640 1 -- 172.21.15.47:0/1024567717 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbabc1a6bf0 con 0x7fbabc1041f0 2024-03-21T10:27:22.177 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:22.177+0000 7fbac3800640 1 -- 172.21.15.47:0/1024567717 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbabc0688e0 con 0x7fbabc1041f0 2024-03-21T10:27:22.182 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:22.183+0000 7fbab9600640 1 -- 172.21.15.47:0/1024567717 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 27) v1 ==== 137985+0+0 (secure 0 0 0) 0x7fbaa8010a90 con 0x7fbabc1041f0 2024-03-21T10:27:22.185 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:22.185+0000 7fbab9600640 1 --2- 172.21.15.47:0/1024567717 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7fbaa40a3090 0x7fbaa40a5550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:27:22.185 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:22.185+0000 7fbab9600640 1 -- 172.21.15.47:0/1024567717 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 9474+0+0 (secure 0 0 0) 0x7fbaa80d03a0 con 0x7fbabc1041f0 2024-03-21T10:27:22.185 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:22.185+0000 7fbac0e00640 1 --2- 172.21.15.47:0/1024567717 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7fbaa40a3090 0x7fbaa40a5550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:27:22.186 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:22.186+0000 7fbac0e00640 1 --2- 172.21.15.47:0/1024567717 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7fbaa40a3090 0x7fbaa40a5550 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fbab0005fd0 tx=0x7fbab000b390 comp rx=0 tx=0).ready entity=mgr.15687 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:27:22.187 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:22.188+0000 7fbab9600640 1 -- 172.21.15.47:0/1024567717 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fbaa809c480 con 0x7fbabc1041f0 2024-03-21T10:27:22.508 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:22.508+0000 7fbac3800640 1 -- 172.21.15.47:0/1024567717 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fbabc10bdc0 con 0x7fbabc1041f0 2024-03-21T10:27:22.510 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:22.510+0000 7fbab9600640 1 -- 172.21.15.47:0/1024567717 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 32 v32) v1 ==== 94+0+11471 (secure 0 0 0) 0x7fbaa80a0520 con 0x7fbabc1041f0 2024-03-21T10:27:22.510 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-21T10:27:22.510 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":32,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14511,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6835/735724037","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6834","nonce":735724037},{"type":"v1","addr":"172.21.15.47:6835","nonce":735724037}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14529,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6837/2248191724","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6836","nonce":2248191724},{"type":"v1","addr":"172.21.15.47:6837","nonce":2248191724}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":32,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-21T10:21:58.329063+0000","modified":"2024-03-21T10:26:17.169892+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":34535,"mds_1":34493,"mds_2":34454,"mds_3":34427,"mds_4":14619},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14604":{"gid":14604,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.47:6839/2881335356","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6838","nonce":2881335356},{"type":"v1","addr":"172.21.15.47:6839","nonce":2881335356}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_14619":{"gid":14619,"name":"a","rank":4,"incarnation":27,"state":"up:active","state_seq":11,"addr":"172.21.15.47:6841/3401578978","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6840","nonce":3401578978},{"type":"v1","addr":"172.21.15.47:6841","nonce":3401578978}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24406":{"gid":24406,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.157:6833/4000010136","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":4000010136},{"type":"v1","addr":"172.21.15.157:6833","nonce":4000010136}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24457":{"gid":24457,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.157:6835/3902868651","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":3902868651},{"type":"v1","addr":"172.21.15.157:6835","nonce":3902868651}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24475":{"gid":24475,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.157:6837/1908087204","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6836","nonce":1908087204},{"type":"v1","addr":"172.21.15.157:6837","nonce":1908087204}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_25729":{"gid":25729,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.157:6839/1998408416","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6838","nonce":1998408416},{"type":"v1","addr":"172.21.15.157:6839","nonce":1998408416}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_34427":{"gid":34427,"name":"k","rank":3,"incarnation":25,"state":"up:active","state_seq":20,"addr":"172.21.15.100:6833/2723302038","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6832","nonce":2723302038},{"type":"v1","addr":"172.21.15.100:6833","nonce":2723302038}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_34454":{"gid":34454,"name":"e","rank":2,"incarnation":22,"state":"up:active","state_seq":17,"addr":"172.21.15.100:6835/2597849957","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":2597849957},{"type":"v1","addr":"172.21.15.100:6835","nonce":2597849957}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_34493":{"gid":34493,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":13,"addr":"172.21.15.100:6837/3497644162","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6836","nonce":3497644162},{"type":"v1","addr":"172.21.15.100:6837","nonce":3497644162}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_34535":{"gid":34535,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.100:6839/4173168440","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6838","nonce":4173168440},{"type":"v1","addr":"172.21.15.100:6839","nonce":4173168440}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":34535,"qdb_cluster":[34427,14619,34454,34535,34493]},"id":1}]} 2024-03-21T10:27:22.511 INFO:teuthology.orchestra.run.smithi047.stderr:dumped fsmap epoch 32 2024-03-21T10:27:22.513 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:22.513+0000 7fbac3800640 1 -- 172.21.15.47:0/1024567717 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7fbaa40a3090 msgr2=0x7fbaa40a5550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:27:22.513 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:22.513+0000 7fbac3800640 1 --2- 172.21.15.47:0/1024567717 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7fbaa40a3090 0x7fbaa40a5550 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fbab0005fd0 tx=0x7fbab000b390 comp rx=0 tx=0).stop 2024-03-21T10:27:22.513 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:22.513+0000 7fbac3800640 1 -- 172.21.15.47:0/1024567717 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbabc1041f0 msgr2=0x7fbabc19a5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:27:22.513 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:22.513+0000 7fbac3800640 1 --2- 172.21.15.47:0/1024567717 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbabc1041f0 0x7fbabc19a5c0 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7fbaa800e9f0 tx=0x7fbaa800eeb0 comp rx=0 tx=0).stop 2024-03-21T10:27:22.513 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:22.513+0000 7fbac3800640 1 -- 172.21.15.47:0/1024567717 shutdown_connections 2024-03-21T10:27:22.513 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:22.513+0000 7fbac3800640 1 --2- 172.21.15.47:0/1024567717 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7fbaa40a3090 0x7fbaa40a5550 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:27:22.513 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:22.513+0000 7fbac3800640 1 --2- 172.21.15.47:0/1024567717 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbabc109840 0x7fbabc19e940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:27:22.514 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:22.513+0000 7fbac3800640 1 --2- 172.21.15.47:0/1024567717 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbabc1041f0 0x7fbabc19a5c0 unknown :-1 s=CLOSED pgs=677 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:27:22.514 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:22.513+0000 7fbac3800640 1 --2- 172.21.15.47:0/1024567717 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbabc103830 0x7fbabc19a080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:27:22.514 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:22.513+0000 7fbac3800640 1 -- 172.21.15.47:0/1024567717 >> 172.21.15.47:0/1024567717 conn(0x7fbabc0ff300 msgr2=0x7fbabc0ffb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:27:22.514 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:22.514+0000 7fbac3800640 1 -- 172.21.15.47:0/1024567717 shutdown_connections 2024-03-21T10:27:22.514 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:22.514+0000 7fbac3800640 1 -- 172.21.15.47:0/1024567717 wait complete. 2024-03-21T10:27:22.536 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-21T10:27:22.537 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-03-21T10:27:22.777 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:22.776+0000 7f47d9000640 1 -- 172.21.15.47:0/4088322182 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f47d41075f0 msgr2=0x7f47d41079d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:27:22.777 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:22.776+0000 7f47d9000640 1 --2- 172.21.15.47:0/4088322182 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f47d41075f0 0x7f47d41079d0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f47c4009930 tx=0x7f47c402bc30 comp rx=0 tx=0).stop 2024-03-21T10:27:22.777 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:22.777+0000 7f47d9000640 1 -- 172.21.15.47:0/4088322182 shutdown_connections 2024-03-21T10:27:22.777 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:22.777+0000 7f47d9000640 1 --2- 172.21.15.47:0/4088322182 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f47d4101ef0 0x7f47d410e3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:27:22.777 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:22.777+0000 7f47d9000640 1 --2- 172.21.15.47:0/4088322182 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f47d4101530 0x7f47d41019b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:27:22.777 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:22.777+0000 7f47d9000640 1 --2- 172.21.15.47:0/4088322182 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f47d41075f0 0x7f47d41079d0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:27:22.777 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:22.777+0000 7f47d9000640 1 -- 172.21.15.47:0/4088322182 >> 172.21.15.47:0/4088322182 conn(0x7f47d40fd040 msgr2=0x7f47d40ff460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:27:22.778 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:22.777+0000 7f47d9000640 1 -- 172.21.15.47:0/4088322182 shutdown_connections 2024-03-21T10:27:22.778 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:22.777+0000 7f47d9000640 1 -- 172.21.15.47:0/4088322182 wait complete. 2024-03-21T10:27:22.779 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:22.778+0000 7f47d9000640 1 Processor -- start 2024-03-21T10:27:22.779 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:22.778+0000 7f47d9000640 1 -- start start 2024-03-21T10:27:22.779 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:22.779+0000 7f47d9000640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f47d4101530 0x7f47d419d910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:27:22.779 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:22.779+0000 7f47d9000640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f47d4101ef0 0x7f47d419de50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:27:22.779 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:22.779+0000 7f47d9000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f47d41075f0 0x7f47d4197b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:27:22.779 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:22.779+0000 7f47d9000640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47d41117b0 con 0x7f47d4101530 2024-03-21T10:27:22.779 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:22.779+0000 7f47d9000640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47d4111630 con 0x7f47d41075f0 2024-03-21T10:27:22.780 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:22.779+0000 7f47d9000640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47d4111930 con 0x7f47d4101ef0 2024-03-21T10:27:22.780 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:22.779+0000 7f47d3400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f47d4101530 0x7f47d419d910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:27:22.780 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:22.779+0000 7f47d3400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f47d4101530 0x7f47d419d910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:34608/0 (socket says 172.21.15.47:34608) 2024-03-21T10:27:22.780 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:22.779+0000 7f47d3400640 1 -- 172.21.15.47:0/2648129614 learned_addr learned my addr 172.21.15.47:0/2648129614 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:27:22.780 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:22.779+0000 7f47d3e00640 1 --2- 172.21.15.47:0/2648129614 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f47d41075f0 0x7f47d4197b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:27:22.780 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:22.779+0000 7f47d2a00640 1 --2- 172.21.15.47:0/2648129614 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f47d4101ef0 0x7f47d419de50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:27:22.781 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:22.780+0000 7f47d3400640 1 -- 172.21.15.47:0/2648129614 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f47d4101ef0 msgr2=0x7f47d419de50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:27:22.781 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:22.780+0000 7f47d3400640 1 --2- 172.21.15.47:0/2648129614 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f47d4101ef0 0x7f47d419de50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:27:22.781 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:22.780+0000 7f47d3400640 1 -- 172.21.15.47:0/2648129614 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f47d41075f0 msgr2=0x7f47d4197b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:27:22.781 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:22.780+0000 7f47d3400640 1 --2- 172.21.15.47:0/2648129614 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f47d41075f0 0x7f47d4197b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:27:22.781 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:22.780+0000 7f47d3400640 1 -- 172.21.15.47:0/2648129614 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47d4198290 con 0x7f47d4101530 2024-03-21T10:27:22.781 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:22.780+0000 7f47d3e00640 1 --2- 172.21.15.47:0/2648129614 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f47d41075f0 0x7f47d4197b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T10:27:22.781 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:22.780+0000 7f47d3400640 1 --2- 172.21.15.47:0/2648129614 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f47d4101530 0x7f47d419d910 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7f47c4000c00 tx=0x7f47c402c660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:27:22.781 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:22.780+0000 7f47cbe00640 1 -- 172.21.15.47:0/2648129614 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47c4043020 con 0x7f47d4101530 2024-03-21T10:27:22.782 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:22.781+0000 7f47d9000640 1 -- 172.21.15.47:0/2648129614 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f47d4198520 con 0x7f47d4101530 2024-03-21T10:27:22.782 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:22.781+0000 7f47cbe00640 1 -- 172.21.15.47:0/2648129614 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f47c402caf0 con 0x7f47d4101530 2024-03-21T10:27:22.782 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:22.781+0000 7f47d9000640 1 -- 172.21.15.47:0/2648129614 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f47d41a47d0 con 0x7f47d4101530 2024-03-21T10:27:22.782 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:22.781+0000 7f47cbe00640 1 -- 172.21.15.47:0/2648129614 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47c4003a20 con 0x7f47d4101530 2024-03-21T10:27:22.784 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:22.785+0000 7f47cbe00640 1 -- 172.21.15.47:0/2648129614 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 27) v1 ==== 137985+0+0 (secure 0 0 0) 0x7f47c4046020 con 0x7f47d4101530 2024-03-21T10:27:22.785 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:22.785+0000 7f47dd261740 1 Processor -- start 2024-03-21T10:27:22.785 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:22.785+0000 7f47dd261740 1 -- start start 2024-03-21T10:27:22.786 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:22.785+0000 7f47dd261740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557495fddcc0 0x557495fde0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:27:22.786 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:22.785+0000 7f47dd261740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x557495fde670 0x557495fe6c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:27:22.786 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:22.785+0000 7f47dd261740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557495fe71f0 0x557495fe9640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:27:22.786 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:22.785+0000 7f47dd261740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x557495f3d2c0 con 0x557495fde670 2024-03-21T10:27:22.786 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:22.786+0000 7f47dd261740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x557495ee5c90 con 0x557495fddcc0 2024-03-21T10:27:22.786 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:22.786+0000 7f47dd261740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x557495f39360 con 0x557495fe71f0 2024-03-21T10:27:22.786 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:22.786+0000 7f47d3400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x557495fde670 0x557495fe6c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:27:22.786 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:22.786+0000 7f47d3400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x557495fde670 0x557495fe6c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:34616/0 (socket says 172.21.15.47:34616) 2024-03-21T10:27:22.786 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:22.786+0000 7f47d3400640 1 -- 172.21.15.47:0/2343280568 learned_addr learned my addr 172.21.15.47:0/2343280568 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:27:22.786 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:22.786+0000 7f47d3e00640 1 --2- 172.21.15.47:0/2343280568 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557495fddcc0 0x557495fde0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:27:22.786 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:22.786+0000 7f47d2a00640 1 --2- 172.21.15.47:0/2343280568 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557495fe71f0 0x557495fe9640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:27:22.787 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:22.787+0000 7f47cbe00640 1 --2- 172.21.15.47:0/2648129614 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f47b80a3090 0x7f47b80a5550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:27:22.788 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:22.787+0000 7f47d3400640 1 -- 172.21.15.47:0/2343280568 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557495fe71f0 msgr2=0x557495fe9640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:27:22.788 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:22.787+0000 7f47d3400640 1 --2- 172.21.15.47:0/2343280568 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557495fe71f0 0x557495fe9640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:27:22.788 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:22.787+0000 7f47d3400640 1 -- 172.21.15.47:0/2343280568 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557495fddcc0 msgr2=0x557495fde0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:27:22.788 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:22.787+0000 7f47d3400640 1 --2- 172.21.15.47:0/2343280568 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557495fddcc0 0x557495fde0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:27:22.788 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:22.787+0000 7f47d3400640 1 -- 172.21.15.47:0/2343280568 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557495f0a080 con 0x557495fde670 2024-03-21T10:27:22.788 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:22.787+0000 7f47cbe00640 1 -- 172.21.15.47:0/2648129614 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 9474+0+0 (secure 0 0 0) 0x7f47c40f1bd0 con 0x7f47d4101530 2024-03-21T10:27:22.788 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:22.787+0000 7f47d3e00640 1 --2- 172.21.15.47:0/2648129614 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f47b80a3090 0x7f47b80a5550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:27:22.788 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:22.787+0000 7f47d3400640 1 --2- 172.21.15.47:0/2343280568 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x557495fde670 0x557495fe6c20 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7f47c40eef50 tx=0x7f47c408d780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5ea11348c76f55c4 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:27:22.788 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:22.788+0000 7f47c8c00640 1 -- 172.21.15.47:0/2343280568 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47c40bd7f0 con 0x557495fde670 2024-03-21T10:27:22.788 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:22.788+0000 7f47c8c00640 1 -- 172.21.15.47:0/2343280568 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f47c40be3b0 con 0x557495fde670 2024-03-21T10:27:22.788 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:22.788+0000 7f47c8c00640 1 -- 172.21.15.47:0/2343280568 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47c402cca0 con 0x557495fde670 2024-03-21T10:27:22.789 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:22.789+0000 7f47dd261740 1 -- 172.21.15.47:0/2343280568 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x557495fde670 msgr2=0x557495fe6c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:27:22.789 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:22.789+0000 7f47dd261740 1 --2- 172.21.15.47:0/2343280568 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x557495fde670 0x557495fe6c20 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7f47c40eef50 tx=0x7f47c408d780 comp rx=0 tx=0).stop 2024-03-21T10:27:22.789 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:22.789+0000 7f47d3e00640 1 --2- 172.21.15.47:0/2648129614 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f47b80a3090 0x7f47b80a5550 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f47cc0099a0 tx=0x7f47cc004a40 comp rx=0 tx=0).ready entity=mgr.15687 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:27:22.789 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:22.789+0000 7f47dd261740 1 -- 172.21.15.47:0/2343280568 shutdown_connections 2024-03-21T10:27:22.789 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:22.789+0000 7f47dd261740 1 --2- 172.21.15.47:0/2343280568 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557495fe71f0 0x557495fe9640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:27:22.789 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:22.789+0000 7f47dd261740 1 --2- 172.21.15.47:0/2343280568 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x557495fde670 0x557495fe6c20 unknown :-1 s=CLOSED pgs=679 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:27:22.789 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:22.789+0000 7f47dd261740 1 --2- 172.21.15.47:0/2343280568 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557495fddcc0 0x557495fde0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:27:22.790 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:22.789+0000 7f47dd261740 1 -- 172.21.15.47:0/2343280568 >> 172.21.15.47:0/2343280568 conn(0x557495fd8390 msgr2=0x557495fd8770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:27:22.790 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:22.789+0000 7f47dd261740 1 -- 172.21.15.47:0/2343280568 shutdown_connections 2024-03-21T10:27:22.790 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:22.789+0000 7f47dd261740 1 -- 172.21.15.47:0/2343280568 wait complete. 2024-03-21T10:27:22.790 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:22.789+0000 7f47dd261740 1 -- start start 2024-03-21T10:27:22.791 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:22.790+0000 7f47dd261740 1 Processor -- start 2024-03-21T10:27:22.791 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:22.790+0000 7f47dd261740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x557495fddcc0 0x557495fe6ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:27:22.791 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:22.790+0000 7f47dd261740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557495fde670 0x557495fe3e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:27:22.791 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:22.790+0000 7f47dd261740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557495fe71f0 0x557495fe43b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:27:22.791 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:22.790+0000 7f47dd261740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x557495fe9f20 con 0x557495fddcc0 2024-03-21T10:27:22.791 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:22.790+0000 7f47dd261740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x557495ef90d0 con 0x557495fde670 2024-03-21T10:27:22.791 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:22.790+0000 7f47dd261740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x557495fea0a0 con 0x557495fe71f0 2024-03-21T10:27:22.791 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:22.791+0000 7f47dd261740 10 client.0 _ll_register_callbacks cb 0x557495efde80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T10:27:22.791 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:22.791+0000 7f47d3400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x557495fddcc0 0x557495fe6ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:27:22.791 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:22.791+0000 7f47d3400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x557495fddcc0 0x557495fe6ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:34630/0 (socket says 172.21.15.47:34630) 2024-03-21T10:27:22.791 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:22.791+0000 7f47d3400640 1 -- 172.21.15.47:0/3597272936 learned_addr learned my addr 172.21.15.47:0/3597272936 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:27:22.791 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:22.791+0000 7f47d2a00640 1 --2- 172.21.15.47:0/3597272936 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557495fde670 0x557495fe3e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:27:22.792 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:22.791+0000 7f47d3e00640 1 --2- 172.21.15.47:0/3597272936 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557495fe71f0 0x557495fe43b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:27:22.793 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:22.792+0000 7f47d3400640 1 -- 172.21.15.47:0/3597272936 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557495fe71f0 msgr2=0x557495fe43b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:27:22.793 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:22.792+0000 7f47d3400640 1 --2- 172.21.15.47:0/3597272936 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557495fe71f0 0x557495fe43b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:27:22.794 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:22.792+0000 7f47d3400640 1 -- 172.21.15.47:0/3597272936 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557495fde670 msgr2=0x557495fe3e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:27:22.794 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:22.792+0000 7f47d3400640 1 --2- 172.21.15.47:0/3597272936 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557495fde670 0x557495fe3e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:27:22.794 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:22.792+0000 7f47d3400640 1 -- 172.21.15.47:0/3597272936 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x557495c513f0 con 0x557495fddcc0 2024-03-21T10:27:22.794 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:22.792+0000 7f47d3400640 1 --2- 172.21.15.47:0/3597272936 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x557495fddcc0 0x557495fe6ac0 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7f47c4041000 tx=0x7f47c40f1e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=14bf3e1adc101bbd server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:27:22.794 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:22.792+0000 7f47bd600640 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-03-21T10:27:22.794 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:22.792+0000 7f47bd600640 1 -- 172.21.15.47:0/3597272936 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47c408b8e0 con 0x557495fddcc0 2024-03-21T10:27:22.794 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:22.792+0000 7f47dd261740 1 -- 172.21.15.47:0/3597272936 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f47c40be3b0 con 0x557495fddcc0 2024-03-21T10:27:22.794 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:22.793+0000 7f47bd600640 1 -- 172.21.15.47:0/3597272936 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(97..97 src has 1..97) v4 ==== 9474+0+0 (secure 0 0 0) 0x7f47c4050dd0 con 0x557495fddcc0 2024-03-21T10:27:22.794 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:22.793+0000 7f47bd600640 1 -- 172.21.15.47:0/3597272936 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f47c408bb40 con 0x557495fddcc0 2024-03-21T10:27:22.794 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:22.793+0000 7f47dd261740 10 client.16392 fetch_fsmap learned FSMap version 32 2024-03-21T10:27:22.794 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:22.793+0000 7f47dd261740 1 -- 172.21.15.47:0/3597272936 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x557495d1b7d0 con 0x557495fddcc0 2024-03-21T10:27:22.794 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:22.793+0000 7f47bd600640 1 -- 172.21.15.47:0/3597272936 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f47c40be3b0 con 0x557495fddcc0 2024-03-21T10:27:22.794 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:22.793+0000 7f47bd600640 1 -- 172.21.15.47:0/3597272936 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 32) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f47c40bdf10 con 0x557495fddcc0 2024-03-21T10:27:22.794 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:22.794+0000 7f47dd261740 10 client.16392 fetch_fsmap finished waiting for FSMap version 32 2024-03-21T10:27:22.794 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:22.794+0000 7f47dd261740 10 client.16392 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.a 2024-03-21T10:27:22.795 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:22.794+0000 7f47dd261740 20 client.16392 populate_metadata read hostname 'smithi047' 2024-03-21T10:27:22.795 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:22.794+0000 7f47dd261740 1 --2- 172.21.15.47:0/3597272936 >> [v2:172.21.15.47:6840/3401578978,v1:172.21.15.47:6841/3401578978] conn(0x557495ff6b70 0x557495ff8f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:27:22.795 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:22.794+0000 7f47dd261740 4 client.16392 mds_command: new command op to 14619 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T10:27:22.795 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:22.794+0000 7f47dd261740 1 -- 172.21.15.47:0/3597272936 --> [v2:172.21.15.47:6840/3401578978,v1:172.21.15.47:6841/3401578978] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x557495f4a800 con 0x557495ff6b70 2024-03-21T10:27:22.795 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:22.794+0000 7f47d2a00640 1 --2- 172.21.15.47:0/3597272936 >> [v2:172.21.15.47:6840/3401578978,v1:172.21.15.47:6841/3401578978] conn(0x557495ff6b70 0x557495ff8f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:27:22.795 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:22.795+0000 7f47d2a00640 1 --2- 172.21.15.47:0/3597272936 >> [v2:172.21.15.47:6840/3401578978,v1:172.21.15.47:6841/3401578978] conn(0x557495ff6b70 0x557495ff8f60 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=b8815a5a86a3381b server_cookie=15f686447ff8357b in_seq=0 out_seq=0 2024-03-21T10:27:22.795 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:22.795+0000 7f47bd600640 10 client.16392 ms_handle_connect on v2:172.21.15.47:6840/3401578978 2024-03-21T10:27:22.796 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:22.796+0000 7f47bd600640 1 -- 172.21.15.47:0/3597272936 <== mds.4 v2:172.21.15.47:6840/3401578978 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x557495f4a800 con 0x557495ff6b70 2024-03-21T10:27:22.796 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:22.796+0000 7f47bd600640 10 client.16392 handle_command_reply: tid=0 2024-03-21T10:27:22.796 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:22.796+0000 7f47dd261740 1 client.16392 shutdown 2024-03-21T10:27:22.797 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:22.796+0000 7f47dd261740 1 -- 172.21.15.47:0/3597272936 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x557495fddcc0 msgr2=0x557495fe6ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:27:22.797 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:22.796+0000 7f47dd261740 1 --2- 172.21.15.47:0/3597272936 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x557495fddcc0 0x557495fe6ac0 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7f47c4041000 tx=0x7f47c40f1e50 comp rx=0 tx=0).stop 2024-03-21T10:27:22.797 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:22.797+0000 7f47dd261740 1 -- 172.21.15.47:0/3597272936 shutdown_connections 2024-03-21T10:27:22.798 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:22.797+0000 7f47dd261740 1 --2- 172.21.15.47:0/3597272936 >> [v2:172.21.15.47:6840/3401578978,v1:172.21.15.47:6841/3401578978] conn(0x557495ff6b70 0x557495ff8f60 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:27:22.798 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:22.797+0000 7f47dd261740 1 --2- 172.21.15.47:0/3597272936 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557495fe71f0 0x557495fe43b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:27:22.798 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:22.797+0000 7f47dd261740 1 --2- 172.21.15.47:0/3597272936 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557495fde670 0x557495fe3e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:27:22.798 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:22.797+0000 7f47dd261740 1 --2- 172.21.15.47:0/3597272936 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x557495fddcc0 0x557495fe6ac0 unknown :-1 s=CLOSED pgs=680 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:27:22.798 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:22.797+0000 7f47dd261740 1 -- 172.21.15.47:0/3597272936 >> 172.21.15.47:0/3597272936 conn(0x557495fd8390 msgr2=0x557495fd9850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:27:22.798 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:22.797+0000 7f47bd600640 0 client.16392 ms_handle_reset on v2:172.21.15.47:6840/3401578978 2024-03-21T10:27:22.798 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:22.797+0000 7f47dd261740 1 -- 172.21.15.47:0/3597272936 shutdown_connections 2024-03-21T10:27:22.798 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:22.797+0000 7f47dd261740 1 -- 172.21.15.47:0/3597272936 wait complete. 2024-03-21T10:27:22.798 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:22.797+0000 7f47dd261740 20 client.16392 trim_cache size 0 max 16384 2024-03-21T10:27:22.817 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:22.816+0000 7f47dd261740 1 Processor -- start 2024-03-21T10:27:22.817 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:22.816+0000 7f47dd261740 1 -- start start 2024-03-21T10:27:22.817 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:22.817+0000 7f47dd261740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557495fddcc0 0x557495fd3160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:27:22.817 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:22.817+0000 7f47dd261740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557495fde670 0x557495fe2e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:27:22.817 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:22.817+0000 7f47dd261740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x557495fe71f0 0x557495fe6580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:27:22.817 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:22.817+0000 7f47dd261740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47d41ac470 con 0x557495fe71f0 2024-03-21T10:27:22.817 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:22.817+0000 7f47dd261740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47d41addf0 con 0x557495fde670 2024-03-21T10:27:22.817 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:22.817+0000 7f47dd261740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47d41acad0 con 0x557495fddcc0 2024-03-21T10:27:22.819 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:22.817+0000 7f47d3e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x557495fe71f0 0x557495fe6580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:27:22.819 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:22.817+0000 7f47d2a00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557495fde670 0x557495fe2e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:27:22.819 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:22.817+0000 7f47d3e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x557495fe71f0 0x557495fe6580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:34640/0 (socket says 172.21.15.47:34640) 2024-03-21T10:27:22.819 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:22.817+0000 7f47d2a00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557495fde670 0x557495fe2e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:45050/0 (socket says 172.21.15.47:45050) 2024-03-21T10:27:22.819 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:22.817+0000 7f47d3e00640 1 -- 172.21.15.47:0/1208898380 learned_addr learned my addr 172.21.15.47:0/1208898380 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:27:22.819 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:22.818+0000 7f47d3400640 1 --2- 172.21.15.47:0/1208898380 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557495fddcc0 0x557495fd3160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:27:22.819 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:22.818+0000 7f47d3e00640 1 -- 172.21.15.47:0/1208898380 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557495fddcc0 msgr2=0x557495fd3160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:27:22.819 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:22.818+0000 7f47d3e00640 1 --2- 172.21.15.47:0/1208898380 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557495fddcc0 0x557495fd3160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:27:22.819 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:22.818+0000 7f47d3e00640 1 -- 172.21.15.47:0/1208898380 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557495fde670 msgr2=0x557495fe2e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:27:22.819 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:22.818+0000 7f47d3e00640 1 --2- 172.21.15.47:0/1208898380 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557495fde670 0x557495fe2e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:27:22.819 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:22.818+0000 7f47d3e00640 1 -- 172.21.15.47:0/1208898380 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557495fe4720 con 0x557495fe71f0 2024-03-21T10:27:22.819 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:22.818+0000 7f47d2a00640 1 --2- 172.21.15.47:0/1208898380 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557495fde670 0x557495fe2e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T10:27:22.820 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:22.818+0000 7f47d3e00640 1 --2- 172.21.15.47:0/1208898380 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x557495fe71f0 0x557495fe6580 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7f47cc013430 tx=0x7f47cc016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c1e68d708708cb67 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:27:22.820 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:22.819+0000 7f47bfe00640 1 -- 172.21.15.47:0/1208898380 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47cc011500 con 0x557495fe71f0 2024-03-21T10:27:22.820 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:22.819+0000 7f47bfe00640 1 -- 172.21.15.47:0/1208898380 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f47cc011b50 con 0x557495fe71f0 2024-03-21T10:27:22.820 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:22.819+0000 7f47bfe00640 1 -- 172.21.15.47:0/1208898380 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47cc01ba50 con 0x557495fe71f0 2024-03-21T10:27:22.820 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:22.820+0000 7f47dd261740 1 -- 172.21.15.47:0/1208898380 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x557495fe71f0 msgr2=0x557495fe6580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:27:22.821 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:22.820+0000 7f47dd261740 1 --2- 172.21.15.47:0/1208898380 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x557495fe71f0 0x557495fe6580 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7f47cc013430 tx=0x7f47cc016000 comp rx=0 tx=0).stop 2024-03-21T10:27:22.821 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:22.820+0000 7f47dd261740 1 -- 172.21.15.47:0/1208898380 shutdown_connections 2024-03-21T10:27:22.821 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:22.820+0000 7f47dd261740 1 --2- 172.21.15.47:0/1208898380 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x557495fe71f0 0x557495fe6580 unknown :-1 s=CLOSED pgs=681 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:27:22.821 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:22.820+0000 7f47dd261740 1 --2- 172.21.15.47:0/1208898380 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557495fde670 0x557495fe2e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:27:22.821 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:22.820+0000 7f47dd261740 1 --2- 172.21.15.47:0/1208898380 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557495fddcc0 0x557495fd3160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:27:22.821 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:22.820+0000 7f47dd261740 1 -- 172.21.15.47:0/1208898380 >> 172.21.15.47:0/1208898380 conn(0x557495fd8390 msgr2=0x557495fe2720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:27:22.821 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:22.820+0000 7f47dd261740 1 -- 172.21.15.47:0/1208898380 shutdown_connections 2024-03-21T10:27:22.821 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:22.820+0000 7f47dd261740 1 -- 172.21.15.47:0/1208898380 wait complete. 2024-03-21T10:27:22.821 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:22.821+0000 7f47dd261740 1 -- start start 2024-03-21T10:27:22.822 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:22.821+0000 7f47dd261740 1 Processor -- start 2024-03-21T10:27:22.822 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:22.822+0000 7f47dd261740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557495fddcc0 0x557495fec980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:27:22.823 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:22.822+0000 7f47dd261740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x557495fde670 0x557495fecec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:27:22.823 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:22.822+0000 7f47dd261740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557495fe71f0 0x557496051390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:27:22.823 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:22.822+0000 7f47dd261740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47d41ad790 con 0x557495fde670 2024-03-21T10:27:22.823 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:22.822+0000 7f47dd261740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47d41ad130 con 0x557495fddcc0 2024-03-21T10:27:22.823 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:22.822+0000 7f47dd261740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47d41aa630 con 0x557495fe71f0 2024-03-21T10:27:22.823 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:22.822+0000 7f47dd261740 10 client.0 _ll_register_callbacks cb 0x7f47cc009a70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T10:27:22.823 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:22.823+0000 7f47d3400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557495fddcc0 0x557495fec980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:27:22.823 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:22.823+0000 7f47d2a00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x557495fde670 0x557495fecec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:27:22.823 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:22.823+0000 7f47d2a00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x557495fde670 0x557495fecec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:34644/0 (socket says 172.21.15.47:34644) 2024-03-21T10:27:22.823 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:22.823+0000 7f47d2a00640 1 -- 172.21.15.47:0/3027400196 learned_addr learned my addr 172.21.15.47:0/3027400196 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:27:22.823 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:22.823+0000 7f47d3400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557495fddcc0 0x557495fec980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:45060/0 (socket says 172.21.15.47:45060) 2024-03-21T10:27:22.824 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:22.823+0000 7f47d2a00640 1 -- 172.21.15.47:0/3027400196 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557495fe71f0 msgr2=0x557496051390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:27:22.824 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:22.823+0000 7f47d2a00640 1 --2- 172.21.15.47:0/3027400196 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557495fe71f0 0x557496051390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:27:22.824 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:22.823+0000 7f47d2a00640 1 -- 172.21.15.47:0/3027400196 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557495fddcc0 msgr2=0x557495fec980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:27:22.824 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:22.823+0000 7f47d2a00640 1 --2- 172.21.15.47:0/3027400196 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557495fddcc0 0x557495fec980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:27:22.824 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:22.823+0000 7f47d2a00640 1 -- 172.21.15.47:0/3027400196 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5574960518d0 con 0x557495fde670 2024-03-21T10:27:22.825 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:22.824+0000 7f47d2a00640 1 --2- 172.21.15.47:0/3027400196 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x557495fde670 0x557495fecec0 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x557495fe36b0 tx=0x7f47c0005bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cd7af29ecfb4e239 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:27:22.825 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:22.824+0000 7f47bd600640 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-03-21T10:27:22.825 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:22.824+0000 7f47bd600640 1 -- 172.21.15.47:0/3027400196 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47c0004220 con 0x557495fde670 2024-03-21T10:27:22.825 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:22.824+0000 7f47bd600640 1 -- 172.21.15.47:0/3027400196 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(97..97 src has 1..97) v4 ==== 9474+0+0 (secure 0 0 0) 0x7f47c00043c0 con 0x557495fde670 2024-03-21T10:27:22.825 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:22.824+0000 7f47dd261740 1 -- 172.21.15.47:0/3027400196 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f47cc011b50 con 0x557495fde670 2024-03-21T10:27:22.826 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:22.825+0000 7f47bd600640 1 -- 172.21.15.47:0/3027400196 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f47c002f050 con 0x557495fde670 2024-03-21T10:27:22.826 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:22.825+0000 7f47dd261740 10 client.16398 fetch_fsmap learned FSMap version 32 2024-03-21T10:27:22.826 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:22.825+0000 7f47dd261740 1 -- 172.21.15.47:0/3027400196 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x557496052b00 con 0x557495fde670 2024-03-21T10:27:22.826 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:22.825+0000 7f47bd600640 1 -- 172.21.15.47:0/3027400196 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f47c0012b10 con 0x557495fde670 2024-03-21T10:27:22.826 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:22.825+0000 7f47bd600640 1 -- 172.21.15.47:0/3027400196 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 32) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f47c00039d0 con 0x557495fde670 2024-03-21T10:27:22.826 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:22.826+0000 7f47dd261740 10 client.16398 fetch_fsmap finished waiting for FSMap version 32 2024-03-21T10:27:22.827 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:22.826+0000 7f47dd261740 10 client.16398 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.a 2024-03-21T10:27:22.827 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:22.826+0000 7f47dd261740 20 client.16398 populate_metadata read hostname 'smithi047' 2024-03-21T10:27:22.827 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:22.826+0000 7f47dd261740 1 --2- 172.21.15.47:0/3027400196 >> [v2:172.21.15.47:6840/3401578978,v1:172.21.15.47:6841/3401578978] conn(0x557496053260 0x557496055680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:27:22.827 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:22.826+0000 7f47dd261740 4 client.16398 mds_command: new command op to 14619 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-21T10:27:22.827 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:22.826+0000 7f47dd261740 1 -- 172.21.15.47:0/3027400196 --> [v2:172.21.15.47:6840/3401578978,v1:172.21.15.47:6841/3401578978] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x7f47cc011b50 con 0x557496053260 2024-03-21T10:27:22.827 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:22.826+0000 7f47d3400640 1 --2- 172.21.15.47:0/3027400196 >> [v2:172.21.15.47:6840/3401578978,v1:172.21.15.47:6841/3401578978] conn(0x557496053260 0x557496055680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:27:22.828 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:22.827+0000 7f47d3400640 1 --2- 172.21.15.47:0/3027400196 >> [v2:172.21.15.47:6840/3401578978,v1:172.21.15.47:6841/3401578978] conn(0x557496053260 0x557496055680 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=803f13b0d951519c server_cookie=6c199549d6cbbc85 in_seq=0 out_seq=0 2024-03-21T10:27:22.828 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:22.827+0000 7f47bd600640 10 client.16398 ms_handle_connect on v2:172.21.15.47:6840/3401578978 2024-03-21T10:27:22.828 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:22.827+0000 7f47bd600640 1 -- 172.21.15.47:0/3027400196 <== mds.4 v2:172.21.15.47:6840/3401578978 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x7f47cc011b50 con 0x557496053260 2024-03-21T10:27:22.828 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:22.827+0000 7f47bd600640 10 client.16398 handle_command_reply: tid=0 2024-03-21T10:27:22.828 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:22.827+0000 7f47dd261740 1 client.16398 shutdown 2024-03-21T10:27:22.828 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:22.828+0000 7f47dd261740 1 -- 172.21.15.47:0/3027400196 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x557495fde670 msgr2=0x557495fecec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:27:22.828 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:22.828+0000 7f47dd261740 1 --2- 172.21.15.47:0/3027400196 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x557495fde670 0x557495fecec0 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x557495fe36b0 tx=0x7f47c0005bb0 comp rx=0 tx=0).stop 2024-03-21T10:27:22.828 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:22.828+0000 7f47dd261740 1 -- 172.21.15.47:0/3027400196 shutdown_connections 2024-03-21T10:27:22.828 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:22.828+0000 7f47dd261740 1 --2- 172.21.15.47:0/3027400196 >> [v2:172.21.15.47:6840/3401578978,v1:172.21.15.47:6841/3401578978] conn(0x557496053260 0x557496055680 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:27:22.828 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:22.828+0000 7f47dd261740 1 --2- 172.21.15.47:0/3027400196 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557495fe71f0 0x557496051390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:27:22.828 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:22.828+0000 7f47dd261740 1 --2- 172.21.15.47:0/3027400196 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x557495fde670 0x557495fecec0 unknown :-1 s=CLOSED pgs=682 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:27:22.828 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:22.828+0000 7f47dd261740 1 --2- 172.21.15.47:0/3027400196 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557495fddcc0 0x557495fec980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:27:22.828 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:22.828+0000 7f47dd261740 1 -- 172.21.15.47:0/3027400196 >> 172.21.15.47:0/3027400196 conn(0x557495fd8390 msgr2=0x557495fd93e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:27:22.828 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:22.828+0000 7f47bd600640 0 client.16398 ms_handle_reset on v2:172.21.15.47:6840/3401578978 2024-03-21T10:27:22.829 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:22.828+0000 7f47dd261740 1 -- 172.21.15.47:0/3027400196 shutdown_connections 2024-03-21T10:27:22.829 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:22.828+0000 7f47dd261740 1 -- 172.21.15.47:0/3027400196 wait complete. 2024-03-21T10:27:22.829 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:22.828+0000 7f47dd261740 20 client.16398 trim_cache size 0 max 16384 2024-03-21T10:27:22.830 INFO:teuthology.orchestra.run.smithi047.stdout:[] 2024-03-21T10:27:22.830 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:22.829+0000 7f47d9000640 1 -- 172.21.15.47:0/2648129614 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f47b80a3090 msgr2=0x7f47b80a5550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:27:22.830 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:22.829+0000 7f47d9000640 1 --2- 172.21.15.47:0/2648129614 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f47b80a3090 0x7f47b80a5550 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f47cc0099a0 tx=0x7f47cc004a40 comp rx=0 tx=0).stop 2024-03-21T10:27:22.830 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:22.829+0000 7f47d9000640 1 -- 172.21.15.47:0/2648129614 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f47d4101530 msgr2=0x7f47d419d910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:27:22.830 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:22.829+0000 7f47d9000640 1 --2- 172.21.15.47:0/2648129614 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f47d4101530 0x7f47d419d910 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7f47c4000c00 tx=0x7f47c402c660 comp rx=0 tx=0).stop 2024-03-21T10:27:22.831 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:22.830+0000 7f47d9000640 1 -- 172.21.15.47:0/2648129614 shutdown_connections 2024-03-21T10:27:22.831 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:22.830+0000 7f47d9000640 1 --2- 172.21.15.47:0/2648129614 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f47b80a3090 0x7f47b80a5550 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:27:22.831 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:22.830+0000 7f47d9000640 1 --2- 172.21.15.47:0/2648129614 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f47d41075f0 0x7f47d4197b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:27:22.831 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:22.830+0000 7f47d9000640 1 --2- 172.21.15.47:0/2648129614 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f47d4101ef0 0x7f47d419de50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:27:22.831 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:22.830+0000 7f47d9000640 1 --2- 172.21.15.47:0/2648129614 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f47d4101530 0x7f47d419d910 unknown :-1 s=CLOSED pgs=678 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:27:22.831 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:22.830+0000 7f47d9000640 1 -- 172.21.15.47:0/2648129614 >> 172.21.15.47:0/2648129614 conn(0x7f47d40fd040 msgr2=0x7f47d40fe7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:27:22.831 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:22.830+0000 7f47d9000640 1 -- 172.21.15.47:0/2648129614 shutdown_connections 2024-03-21T10:27:22.831 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:22.830+0000 7f47d9000640 1 -- 172.21.15.47:0/2648129614 wait complete. 2024-03-21T10:27:22.857 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-21T10:27:22.857 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-03-21T10:27:22.942 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:22.942+0000 7f1c0d600640 1 -- 172.21.15.47:0/576780026 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1c08073a70 msgr2=0x7f1c08073e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:27:22.942 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:22.942+0000 7f1c0d600640 1 --2- 172.21.15.47:0/576780026 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1c08073a70 0x7f1c08073e50 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7f1bf8009900 tx=0x7f1bf802bcd0 comp rx=0 tx=0).stop 2024-03-21T10:27:22.942 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:22.942+0000 7f1c0d600640 1 -- 172.21.15.47:0/576780026 shutdown_connections 2024-03-21T10:27:22.942 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:22.942+0000 7f1c0d600640 1 --2- 172.21.15.47:0/576780026 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1c0810dd40 0x7f1c08110150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:27:22.942 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:22.942+0000 7f1c0d600640 1 --2- 172.21.15.47:0/576780026 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1c08074390 0x7f1c0810d800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:27:22.942 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:22.942+0000 7f1c0d600640 1 --2- 172.21.15.47:0/576780026 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1c08073a70 0x7f1c08073e50 unknown :-1 s=CLOSED pgs=683 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:27:22.943 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:22.942+0000 7f1c0d600640 1 -- 172.21.15.47:0/576780026 >> 172.21.15.47:0/576780026 conn(0x7f1c080fcfb0 msgr2=0x7f1c080ff3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:27:22.943 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:22.942+0000 7f1c0d600640 1 -- 172.21.15.47:0/576780026 shutdown_connections 2024-03-21T10:27:22.943 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:22.942+0000 7f1c0d600640 1 -- 172.21.15.47:0/576780026 wait complete. 2024-03-21T10:27:22.944 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:22.943+0000 7f1c0d600640 1 Processor -- start 2024-03-21T10:27:22.944 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:22.943+0000 7f1c0d600640 1 -- start start 2024-03-21T10:27:22.944 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:22.944+0000 7f1c0d600640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1c08073a70 0x7f1c08199f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:27:22.944 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:22.944+0000 7f1c0d600640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1c08074390 0x7f1c0819a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:27:22.945 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:22.944+0000 7f1c0d600640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1c0810dd40 0x7f1c0819e7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:27:22.945 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:22.944+0000 7f1c0d600640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c08113950 con 0x7f1c08074390 2024-03-21T10:27:22.945 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:22.944+0000 7f1c0d600640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c081137d0 con 0x7f1c0810dd40 2024-03-21T10:27:22.945 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:22.944+0000 7f1c0d600640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c08113ad0 con 0x7f1c08073a70 2024-03-21T10:27:22.945 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:22.944+0000 7f1c07400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1c08074390 0x7f1c0819a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:27:22.945 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:22.944+0000 7f1c07400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1c08074390 0x7f1c0819a480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:34656/0 (socket says 172.21.15.47:34656) 2024-03-21T10:27:22.945 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:22.944+0000 7f1c07400640 1 -- 172.21.15.47:0/1295712256 learned_addr learned my addr 172.21.15.47:0/1295712256 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:27:22.945 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:22.944+0000 7f1c0cc00640 1 --2- 172.21.15.47:0/1295712256 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1c0810dd40 0x7f1c0819e7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:27:22.945 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:22.944+0000 7f1c07e00640 1 --2- 172.21.15.47:0/1295712256 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1c08073a70 0x7f1c08199f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:27:22.946 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:22.945+0000 7f1c07400640 1 -- 172.21.15.47:0/1295712256 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1c08073a70 msgr2=0x7f1c08199f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:27:22.946 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:22.945+0000 7f1c07400640 1 --2- 172.21.15.47:0/1295712256 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1c08073a70 0x7f1c08199f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:27:22.946 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:22.945+0000 7f1c07400640 1 -- 172.21.15.47:0/1295712256 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1c0810dd40 msgr2=0x7f1c0819e7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:27:22.946 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:22.945+0000 7f1c07400640 1 --2- 172.21.15.47:0/1295712256 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1c0810dd40 0x7f1c0819e7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:27:22.947 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:22.945+0000 7f1c07400640 1 -- 172.21.15.47:0/1295712256 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c0819eee0 con 0x7f1c08074390 2024-03-21T10:27:22.947 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:22.945+0000 7f1c07400640 1 --2- 172.21.15.47:0/1295712256 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1c08074390 0x7f1c0819a480 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7f1bf400e9f0 tx=0x7f1bf400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:27:22.947 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:22.946+0000 7f1c04c00640 1 -- 172.21.15.47:0/1295712256 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1bf400cd80 con 0x7f1c08074390 2024-03-21T10:27:22.947 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:22.946+0000 7f1c04c00640 1 -- 172.21.15.47:0/1295712256 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f1bf40108e0 con 0x7f1c08074390 2024-03-21T10:27:22.947 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:22.946+0000 7f1c04c00640 1 -- 172.21.15.47:0/1295712256 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1bf4018920 con 0x7f1c08074390 2024-03-21T10:27:22.947 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:22.946+0000 7f1c0d600640 1 -- 172.21.15.47:0/1295712256 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1c0819f1d0 con 0x7f1c08074390 2024-03-21T10:27:22.947 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:22.946+0000 7f1c0d600640 1 -- 172.21.15.47:0/1295712256 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1c081a6a60 con 0x7f1c08074390 2024-03-21T10:27:22.951 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:22.951+0000 7f1c11a26740 1 Processor -- start 2024-03-21T10:27:22.952 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:22.952+0000 7f1c04c00640 1 -- 172.21.15.47:0/1295712256 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 27) v1 ==== 137985+0+0 (secure 0 0 0) 0x7f1bf4010a90 con 0x7f1c08074390 2024-03-21T10:27:22.953 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:22.953+0000 7f1c11a26740 1 -- start start 2024-03-21T10:27:22.954 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:22.954+0000 7f1c04c00640 1 --2- 172.21.15.47:0/1295712256 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f1bec0a3080 0x7f1bec0a5540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:27:22.955 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:22.954+0000 7f1c04c00640 1 -- 172.21.15.47:0/1295712256 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 9474+0+0 (secure 0 0 0) 0x7f1bf40cff00 con 0x7f1c08074390 2024-03-21T10:27:22.955 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:22.955+0000 7f1c11a26740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56004b7639f0 0x56004b763dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:27:22.955 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:22.955+0000 7f1c11a26740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56004b764310 0x56004b76cba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:27:22.955 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:22.955+0000 7f1c11a26740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56004b76d0e0 0x56004b76f4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:27:22.955 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:22.955+0000 7f1c11a26740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x56004b71d3b0 con 0x56004b764310 2024-03-21T10:27:22.955 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:22.955+0000 7f1c11a26740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x56004b6bf830 con 0x56004b76d0e0 2024-03-21T10:27:22.955 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:22.955+0000 7f1c11a26740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x56004b6685d0 con 0x56004b7639f0 2024-03-21T10:27:22.956 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:22.955+0000 7f1c0cc00640 1 --2- 172.21.15.47:0/1295712256 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f1bec0a3080 0x7f1bec0a5540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:27:22.956 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:22.956+0000 7f1c07e00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56004b7639f0 0x56004b763dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:27:22.956 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:22.956+0000 7f1c07400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56004b764310 0x56004b76cba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:27:22.956 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:22.956+0000 7f1c07400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56004b764310 0x56004b76cba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:34664/0 (socket says 172.21.15.47:34664) 2024-03-21T10:27:22.956 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:22.956+0000 7f1c07400640 1 -- 172.21.15.47:0/3988829355 learned_addr learned my addr 172.21.15.47:0/3988829355 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:27:22.956 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:22.956+0000 7f1c0cc00640 1 --2- 172.21.15.47:0/3988829355 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56004b76d0e0 0x56004b76f4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:27:22.957 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:22.957+0000 7f1c07400640 1 -- 172.21.15.47:0/3988829355 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56004b7639f0 msgr2=0x56004b763dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:27:22.957 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:22.957+0000 7f1c07400640 1 --2- 172.21.15.47:0/3988829355 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56004b7639f0 0x56004b763dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:27:22.957 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:22.957+0000 7f1c07400640 1 -- 172.21.15.47:0/3988829355 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56004b76d0e0 msgr2=0x56004b76f4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:27:22.957 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:22.957+0000 7f1c07400640 1 --2- 172.21.15.47:0/3988829355 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56004b76d0e0 0x56004b76f4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:27:22.957 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:22.957+0000 7f1c07400640 1 -- 172.21.15.47:0/3988829355 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56004b6802c0 con 0x56004b764310 2024-03-21T10:27:22.957 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:22.957+0000 7f1c0cc00640 1 --2- 172.21.15.47:0/1295712256 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f1bec0a3080 0x7f1bec0a5540 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f1c08069860 tx=0x7f1c00011810 comp rx=0 tx=0).ready entity=mgr.15687 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:27:22.957 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:22.957+0000 7f1c07400640 1 --2- 172.21.15.47:0/3988829355 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56004b764310 0x56004b76cba0 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7f1bf40a43b0 tx=0x7f1bf409c040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c2fa91bb1504298b server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:27:22.958 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:22.957+0000 7f1bfd200640 1 -- 172.21.15.47:0/3988829355 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1bf409a280 con 0x56004b764310 2024-03-21T10:27:22.958 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:22.958+0000 7f1bfd200640 1 -- 172.21.15.47:0/3988829355 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f1bf40cd4b0 con 0x56004b764310 2024-03-21T10:27:22.958 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:22.958+0000 7f1bfd200640 1 -- 172.21.15.47:0/3988829355 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1bf4096af0 con 0x56004b764310 2024-03-21T10:27:22.958 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:22.959+0000 7f1c11a26740 1 -- 172.21.15.47:0/3988829355 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56004b764310 msgr2=0x56004b76cba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:27:22.959 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:22.959+0000 7f1c11a26740 1 --2- 172.21.15.47:0/3988829355 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56004b764310 0x56004b76cba0 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7f1bf40a43b0 tx=0x7f1bf409c040 comp rx=0 tx=0).stop 2024-03-21T10:27:22.959 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:22.959+0000 7f1c11a26740 1 -- 172.21.15.47:0/3988829355 shutdown_connections 2024-03-21T10:27:22.959 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:22.959+0000 7f1c11a26740 1 --2- 172.21.15.47:0/3988829355 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56004b76d0e0 0x56004b76f4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:27:22.959 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:22.959+0000 7f1c11a26740 1 --2- 172.21.15.47:0/3988829355 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56004b764310 0x56004b76cba0 unknown :-1 s=CLOSED pgs=685 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:27:22.959 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:22.959+0000 7f1c11a26740 1 --2- 172.21.15.47:0/3988829355 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56004b7639f0 0x56004b763dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:27:22.959 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:22.959+0000 7f1c11a26740 1 -- 172.21.15.47:0/3988829355 >> 172.21.15.47:0/3988829355 conn(0x56004b75e310 msgr2=0x56004b75e6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:27:22.960 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:22.959+0000 7f1c11a26740 1 -- 172.21.15.47:0/3988829355 shutdown_connections 2024-03-21T10:27:22.960 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:22.959+0000 7f1c11a26740 1 -- 172.21.15.47:0/3988829355 wait complete. 2024-03-21T10:27:22.960 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:22.960+0000 7f1c11a26740 1 -- start start 2024-03-21T10:27:22.960 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:22.960+0000 7f1c11a26740 1 Processor -- start 2024-03-21T10:27:22.961 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:22.961+0000 7f1c11a26740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56004b7639f0 0x56004b76a170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:27:22.961 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:22.961+0000 7f1c11a26740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56004b764310 0x56004b76a6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:27:22.961 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:22.961+0000 7f1c11a26740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56004b76d0e0 0x56004b76abf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:27:22.961 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:22.961+0000 7f1c11a26740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x56004b67ba20 con 0x56004b76d0e0 2024-03-21T10:27:22.961 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:22.961+0000 7f1c11a26740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x56004b6bb8d0 con 0x56004b764310 2024-03-21T10:27:22.962 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:22.961+0000 7f1c11a26740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x56004b76fd10 con 0x56004b7639f0 2024-03-21T10:27:22.962 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:22.961+0000 7f1c11a26740 10 client.0 _ll_register_callbacks cb 0x56004b6b81c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T10:27:22.963 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:22.962+0000 7f1c07e00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56004b7639f0 0x56004b76a170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:27:22.963 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:22.962+0000 7f1c07400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56004b764310 0x56004b76a6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:27:22.963 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:22.962+0000 7f1c07400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56004b764310 0x56004b76a6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:45072/0 (socket says 172.21.15.47:45072) 2024-03-21T10:27:22.963 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:22.962+0000 7f1c07400640 1 -- 172.21.15.47:0/3504052808 learned_addr learned my addr 172.21.15.47:0/3504052808 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:27:22.963 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:22.962+0000 7f1c0cc00640 1 --2- 172.21.15.47:0/3504052808 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56004b76d0e0 0x56004b76abf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:27:22.965 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:22.963+0000 7f1c07400640 1 -- 172.21.15.47:0/3504052808 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56004b7639f0 msgr2=0x56004b76a170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:27:22.965 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:22.963+0000 7f1c07400640 1 --2- 172.21.15.47:0/3504052808 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56004b7639f0 0x56004b76a170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:27:22.965 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:22.963+0000 7f1c07400640 1 -- 172.21.15.47:0/3504052808 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56004b76d0e0 msgr2=0x56004b76abf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:27:22.966 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:22.963+0000 7f1c07400640 1 --2- 172.21.15.47:0/3504052808 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56004b76d0e0 0x56004b76abf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:27:22.966 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:22.963+0000 7f1c07400640 1 -- 172.21.15.47:0/3504052808 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56004b3d4210 con 0x56004b764310 2024-03-21T10:27:22.966 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:22.963+0000 7f1c0cc00640 1 --2- 172.21.15.47:0/3504052808 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56004b76d0e0 0x56004b76abf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T10:27:22.966 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:22.963+0000 7f1c07400640 1 --2- 172.21.15.47:0/3504052808 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56004b764310 0x56004b76a6b0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f1bf4098950 tx=0x7f1bf40a01a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c9109ede201ce4e7 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:27:22.966 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:22.963+0000 7f1bf1600640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-21T10:27:22.966 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:22.963+0000 7f1bf1600640 1 -- 172.21.15.47:0/3504052808 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1bf4010630 con 0x56004b764310 2024-03-21T10:27:22.966 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:22.963+0000 7f1c11a26740 1 -- 172.21.15.47:0/3504052808 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f1bf40cd4b0 con 0x56004b764310 2024-03-21T10:27:22.966 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:22.963+0000 7f1bf1600640 1 -- 172.21.15.47:0/3504052808 <== mon.2 v2:172.21.15.100:3300/0 2 ==== osd_map(97..97 src has 1..97) v4 ==== 9474+0+0 (secure 0 0 0) 0x7f1bf4096af0 con 0x56004b764310 2024-03-21T10:27:22.966 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:22.964+0000 7f1bf1600640 1 -- 172.21.15.47:0/3504052808 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1bf4096cf0 con 0x56004b764310 2024-03-21T10:27:22.966 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:22.964+0000 7f1c11a26740 10 client.36200 fetch_fsmap learned FSMap version 32 2024-03-21T10:27:22.966 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:22.964+0000 7f1c11a26740 1 -- 172.21.15.47:0/3504052808 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x56004b4a0620 con 0x56004b764310 2024-03-21T10:27:22.966 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:22.964+0000 7f1bf1600640 1 -- 172.21.15.47:0/3504052808 <== mon.2 v2:172.21.15.100:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f1bf40cd4b0 con 0x56004b764310 2024-03-21T10:27:22.966 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:22.964+0000 7f1bf1600640 1 -- 172.21.15.47:0/3504052808 <== mon.2 v2:172.21.15.100:3300/0 5 ==== fsmap(e 32) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f1bf409ae40 con 0x56004b764310 2024-03-21T10:27:22.966 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:22.964+0000 7f1c11a26740 10 client.36200 fetch_fsmap finished waiting for FSMap version 32 2024-03-21T10:27:22.967 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:22.964+0000 7f1c11a26740 10 client.36200 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-21T10:27:22.967 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:22.964+0000 7f1c11a26740 20 client.36200 populate_metadata read hostname 'smithi047' 2024-03-21T10:27:22.967 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:22.965+0000 7f1c11a26740 1 --2- 172.21.15.47:0/3504052808 >> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] conn(0x56004b77cfd0 0x56004b77f3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:27:22.967 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:22.965+0000 7f1c11a26740 4 client.36200 mds_command: new command op to 34535 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T10:27:22.967 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:22.965+0000 7f1c11a26740 1 -- 172.21.15.47:0/3504052808 --> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56004b6ccd70 con 0x56004b77cfd0 2024-03-21T10:27:22.968 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:22.966+0000 7f1c07e00640 1 --2- 172.21.15.47:0/3504052808 >> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] conn(0x56004b77cfd0 0x56004b77f3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:27:22.968 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:22.967+0000 7f1c07e00640 1 --2- 172.21.15.47:0/3504052808 >> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] conn(0x56004b77cfd0 0x56004b77f3c0 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9ed4c1dac8b588c5 server_cookie=132f0e6c9ffc0631 in_seq=0 out_seq=0 2024-03-21T10:27:22.968 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:22.967+0000 7f1bf1600640 10 client.36200 ms_handle_connect on v2:172.21.15.100:6838/4173168440 2024-03-21T10:27:22.968 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:22.968+0000 7f1bf1600640 1 -- 172.21.15.47:0/3504052808 <== mds.0 v2:172.21.15.100:6838/4173168440 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x56004b6ccd70 con 0x56004b77cfd0 2024-03-21T10:27:22.968 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:22.968+0000 7f1bf1600640 10 client.36200 handle_command_reply: tid=0 2024-03-21T10:27:22.968 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:22.968+0000 7f1c11a26740 1 client.36200 shutdown 2024-03-21T10:27:22.969 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:22.968+0000 7f1c11a26740 1 -- 172.21.15.47:0/3504052808 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56004b764310 msgr2=0x56004b76a6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:27:22.969 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:22.968+0000 7f1c11a26740 1 --2- 172.21.15.47:0/3504052808 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56004b764310 0x56004b76a6b0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f1bf4098950 tx=0x7f1bf40a01a0 comp rx=0 tx=0).stop 2024-03-21T10:27:22.969 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:22.968+0000 7f1c11a26740 1 -- 172.21.15.47:0/3504052808 shutdown_connections 2024-03-21T10:27:22.969 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:22.968+0000 7f1c11a26740 1 --2- 172.21.15.47:0/3504052808 >> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] conn(0x56004b77cfd0 0x56004b77f3c0 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:27:22.969 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:22.969+0000 7f1c11a26740 1 --2- 172.21.15.47:0/3504052808 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56004b76d0e0 0x56004b76abf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:27:22.969 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:22.969+0000 7f1c11a26740 1 --2- 172.21.15.47:0/3504052808 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56004b764310 0x56004b76a6b0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:27:22.969 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:22.969+0000 7f1c11a26740 1 --2- 172.21.15.47:0/3504052808 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56004b7639f0 0x56004b76a170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:27:22.969 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:22.969+0000 7f1bf1600640 0 client.36200 ms_handle_reset on v2:172.21.15.100:6838/4173168440 2024-03-21T10:27:22.969 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:22.969+0000 7f1c11a26740 1 -- 172.21.15.47:0/3504052808 >> 172.21.15.47:0/3504052808 conn(0x56004b75e310 msgr2=0x56004b760060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:27:22.969 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:22.969+0000 7f1c11a26740 1 -- 172.21.15.47:0/3504052808 shutdown_connections 2024-03-21T10:27:22.969 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:22.969+0000 7f1c11a26740 1 -- 172.21.15.47:0/3504052808 wait complete. 2024-03-21T10:27:22.970 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:22.969+0000 7f1c11a26740 20 client.36200 trim_cache size 0 max 16384 2024-03-21T10:27:22.993 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:22.992+0000 7f1c11a26740 1 Processor -- start 2024-03-21T10:27:22.993 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:22.992+0000 7f1c11a26740 1 -- start start 2024-03-21T10:27:22.993 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:22.993+0000 7f1c11a26740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56004b7639f0 0x56004b76c8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:27:22.993 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:22.993+0000 7f1c11a26740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56004b764310 0x56004b778420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:27:22.993 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:22.993+0000 7f1c11a26740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56004b76d0e0 0x56004b778960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:27:22.993 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:22.993+0000 7f1c11a26740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c081ac670 con 0x56004b764310 2024-03-21T10:27:22.993 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:22.993+0000 7f1c11a26740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c081adff0 con 0x56004b76d0e0 2024-03-21T10:27:22.993 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:22.993+0000 7f1c11a26740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c081accd0 con 0x56004b7639f0 2024-03-21T10:27:22.993 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:22.993+0000 7f1c07400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56004b764310 0x56004b778420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:27:22.995 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:22.993+0000 7f1c07400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56004b764310 0x56004b778420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:34680/0 (socket says 172.21.15.47:34680) 2024-03-21T10:27:22.995 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:22.993+0000 7f1c07400640 1 -- 172.21.15.47:0/3624981717 learned_addr learned my addr 172.21.15.47:0/3624981717 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:27:22.995 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:22.993+0000 7f1c0cc00640 1 --2- 172.21.15.47:0/3624981717 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56004b76d0e0 0x56004b778960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:27:22.995 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:22.994+0000 7f1c07e00640 1 --2- 172.21.15.47:0/3624981717 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56004b7639f0 0x56004b76c8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:27:22.995 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:22.994+0000 7f1c07400640 1 -- 172.21.15.47:0/3624981717 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56004b7639f0 msgr2=0x56004b76c8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:27:22.995 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:22.994+0000 7f1c07400640 1 --2- 172.21.15.47:0/3624981717 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56004b7639f0 0x56004b76c8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:27:22.995 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:22.994+0000 7f1c07400640 1 -- 172.21.15.47:0/3624981717 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56004b76d0e0 msgr2=0x56004b778960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:27:22.996 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:22.994+0000 7f1c07400640 1 --2- 172.21.15.47:0/3624981717 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56004b76d0e0 0x56004b778960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:27:22.996 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:22.994+0000 7f1c07400640 1 -- 172.21.15.47:0/3624981717 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56004b753f80 con 0x56004b764310 2024-03-21T10:27:22.996 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:22.994+0000 7f1c07400640 1 --2- 172.21.15.47:0/3624981717 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56004b764310 0x56004b778420 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7f1bf409b260 tx=0x7f1bf409a490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b5a1d68651919de1 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:27:22.996 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:22.995+0000 7f1bf3e00640 1 -- 172.21.15.47:0/3624981717 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1bf4098220 con 0x56004b764310 2024-03-21T10:27:22.996 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:22.995+0000 7f1bf3e00640 1 -- 172.21.15.47:0/3624981717 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f1bf40a1460 con 0x56004b764310 2024-03-21T10:27:22.996 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:22.995+0000 7f1bf3e00640 1 -- 172.21.15.47:0/3624981717 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1bf4010690 con 0x56004b764310 2024-03-21T10:27:22.997 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:22.996+0000 7f1c11a26740 1 -- 172.21.15.47:0/3624981717 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56004b764310 msgr2=0x56004b778420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:27:22.997 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:22.996+0000 7f1c11a26740 1 --2- 172.21.15.47:0/3624981717 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56004b764310 0x56004b778420 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7f1bf409b260 tx=0x7f1bf409a490 comp rx=0 tx=0).stop 2024-03-21T10:27:22.997 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:22.996+0000 7f1c11a26740 1 -- 172.21.15.47:0/3624981717 shutdown_connections 2024-03-21T10:27:22.997 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:22.996+0000 7f1c11a26740 1 --2- 172.21.15.47:0/3624981717 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56004b76d0e0 0x56004b778960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:27:22.997 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:22.996+0000 7f1c11a26740 1 --2- 172.21.15.47:0/3624981717 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56004b764310 0x56004b778420 unknown :-1 s=CLOSED pgs=686 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:27:22.997 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:22.996+0000 7f1c11a26740 1 --2- 172.21.15.47:0/3624981717 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56004b7639f0 0x56004b76c8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:27:22.997 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:22.996+0000 7f1c11a26740 1 -- 172.21.15.47:0/3624981717 >> 172.21.15.47:0/3624981717 conn(0x56004b75e310 msgr2=0x56004b7654a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:27:22.997 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:22.996+0000 7f1c11a26740 1 -- 172.21.15.47:0/3624981717 shutdown_connections 2024-03-21T10:27:22.997 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:22.996+0000 7f1c11a26740 1 -- 172.21.15.47:0/3624981717 wait complete. 2024-03-21T10:27:22.997 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:22.997+0000 7f1c11a26740 1 -- start start 2024-03-21T10:27:22.997 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:22.997+0000 7f1c11a26740 1 Processor -- start 2024-03-21T10:27:23.000 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:22.997+0000 7f1c11a26740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56004b7639f0 0x56004b766ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:27:23.000 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:22.997+0000 7f1c11a26740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56004b764310 0x56004b767400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:27:23.000 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:22.998+0000 7f1c11a26740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56004b76d0e0 0x56004b7d7550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:27:23.000 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:22.998+0000 7f1c11a26740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c081ad990 con 0x56004b764310 2024-03-21T10:27:23.000 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:22.998+0000 7f1c11a26740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c081ad330 con 0x56004b76d0e0 2024-03-21T10:27:23.000 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:22.998+0000 7f1c11a26740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c081aa830 con 0x56004b7639f0 2024-03-21T10:27:23.000 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:22.998+0000 7f1c11a26740 10 client.0 _ll_register_callbacks cb 0x7f1bf4010de0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T10:27:23.000 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:22.998+0000 7f1c07400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56004b764310 0x56004b767400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:27:23.000 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:22.998+0000 7f1c07400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56004b764310 0x56004b767400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:34696/0 (socket says 172.21.15.47:34696) 2024-03-21T10:27:23.000 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:22.998+0000 7f1c07400640 1 -- 172.21.15.47:0/2485982677 learned_addr learned my addr 172.21.15.47:0/2485982677 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:27:23.000 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:22.998+0000 7f1c07e00640 1 --2- 172.21.15.47:0/2485982677 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56004b7639f0 0x56004b766ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:27:23.000 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:22.998+0000 7f1c0cc00640 1 --2- 172.21.15.47:0/2485982677 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56004b76d0e0 0x56004b7d7550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:27:23.000 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:22.999+0000 7f1c07400640 1 -- 172.21.15.47:0/2485982677 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56004b7639f0 msgr2=0x56004b766ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:27:23.001 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:22.999+0000 7f1c07400640 1 --2- 172.21.15.47:0/2485982677 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56004b7639f0 0x56004b766ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:27:23.001 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:22.999+0000 7f1c07400640 1 -- 172.21.15.47:0/2485982677 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56004b76d0e0 msgr2=0x56004b7d7550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:27:23.001 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:22.999+0000 7f1c07400640 1 --2- 172.21.15.47:0/2485982677 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56004b76d0e0 0x56004b7d7550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:27:23.001 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:22.999+0000 7f1c07400640 1 -- 172.21.15.47:0/2485982677 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56004b7d7a90 con 0x56004b764310 2024-03-21T10:27:23.001 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:22.999+0000 7f1c07e00640 1 --2- 172.21.15.47:0/2485982677 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56004b7639f0 0x56004b766ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T10:27:23.001 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:22.999+0000 7f1c07400640 1 --2- 172.21.15.47:0/2485982677 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56004b764310 0x56004b767400 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7f1bf400c6a0 tx=0x7f1bf409b520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e506e48cbaf5b32c server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:27:23.001 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:22.999+0000 7f1bf1600640 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-03-21T10:27:23.001 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:22.999+0000 7f1bf1600640 1 -- 172.21.15.47:0/2485982677 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1bf4098220 con 0x56004b764310 2024-03-21T10:27:23.001 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:22.999+0000 7f1bf1600640 1 -- 172.21.15.47:0/2485982677 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(97..97 src has 1..97) v4 ==== 9474+0+0 (secure 0 0 0) 0x7f1bf4010430 con 0x56004b764310 2024-03-21T10:27:23.001 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:22.999+0000 7f1c11a26740 1 -- 172.21.15.47:0/2485982677 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f1bf40a1460 con 0x56004b764310 2024-03-21T10:27:23.001 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:23.000+0000 7f1bf1600640 1 -- 172.21.15.47:0/2485982677 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1bf40cd6c0 con 0x56004b764310 2024-03-21T10:27:23.001 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:23.000+0000 7f1c11a26740 10 client.16416 fetch_fsmap learned FSMap version 32 2024-03-21T10:27:23.001 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:23.000+0000 7f1c11a26740 1 -- 172.21.15.47:0/2485982677 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x56004b7d8d10 con 0x56004b764310 2024-03-21T10:27:23.001 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:23.000+0000 7f1bf1600640 1 -- 172.21.15.47:0/2485982677 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f1bf40a1460 con 0x56004b764310 2024-03-21T10:27:23.002 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:23.000+0000 7f1bf1600640 1 -- 172.21.15.47:0/2485982677 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 32) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f1bf409a540 con 0x56004b764310 2024-03-21T10:27:23.002 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:23.001+0000 7f1c11a26740 10 client.16416 fetch_fsmap finished waiting for FSMap version 32 2024-03-21T10:27:23.002 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:23.001+0000 7f1c11a26740 10 client.16416 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-21T10:27:23.003 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:23.001+0000 7f1c11a26740 20 client.16416 populate_metadata read hostname 'smithi047' 2024-03-21T10:27:23.003 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:23.001+0000 7f1c11a26740 1 --2- 172.21.15.47:0/2485982677 >> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] conn(0x56004b7d9120 0x56004b7db540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:27:23.003 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:23.001+0000 7f1c11a26740 4 client.16416 mds_command: new command op to 34535 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-21T10:27:23.003 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:23.001+0000 7f1c11a26740 1 -- 172.21.15.47:0/2485982677 --> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x56004b6c70c0 con 0x56004b7d9120 2024-03-21T10:27:23.003 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:23.001+0000 7f1c07e00640 1 --2- 172.21.15.47:0/2485982677 >> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] conn(0x56004b7d9120 0x56004b7db540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:27:23.003 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:23.002+0000 7f1bf1600640 10 client.16416 ms_handle_connect on v2:172.21.15.100:6838/4173168440 2024-03-21T10:27:23.003 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:23.002+0000 7f1c07e00640 1 --2- 172.21.15.47:0/2485982677 >> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] conn(0x56004b7d9120 0x56004b7db540 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=74cfab00088d004 server_cookie=9efe4860148cbc07 in_seq=0 out_seq=0 2024-03-21T10:27:23.018 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:23.018+0000 7f1bf1600640 1 -- 172.21.15.47:0/2485982677 <== mds.0 v2:172.21.15.100:6838/4173168440 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x56004b6c70c0 con 0x56004b7d9120 2024-03-21T10:27:23.019 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:23.018+0000 7f1bf1600640 10 client.16416 handle_command_reply: tid=0 2024-03-21T10:27:23.019 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:23.018+0000 7f1c11a26740 1 client.16416 shutdown 2024-03-21T10:27:23.019 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:23.018+0000 7f1c11a26740 1 -- 172.21.15.47:0/2485982677 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56004b764310 msgr2=0x56004b767400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:27:23.019 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:23.018+0000 7f1c11a26740 1 --2- 172.21.15.47:0/2485982677 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56004b764310 0x56004b767400 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7f1bf400c6a0 tx=0x7f1bf409b520 comp rx=0 tx=0).stop 2024-03-21T10:27:23.019 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:23.019+0000 7f1c11a26740 1 -- 172.21.15.47:0/2485982677 shutdown_connections 2024-03-21T10:27:23.019 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:23.019+0000 7f1c11a26740 1 --2- 172.21.15.47:0/2485982677 >> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] conn(0x56004b7d9120 0x56004b7db540 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:27:23.019 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:23.019+0000 7f1c11a26740 1 --2- 172.21.15.47:0/2485982677 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56004b76d0e0 0x56004b7d7550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:27:23.019 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:23.019+0000 7f1c11a26740 1 --2- 172.21.15.47:0/2485982677 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56004b764310 0x56004b767400 unknown :-1 s=CLOSED pgs=687 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:27:23.019 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:23.019+0000 7f1c11a26740 1 --2- 172.21.15.47:0/2485982677 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56004b7639f0 0x56004b766ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:27:23.019 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:23.019+0000 7f1c11a26740 1 -- 172.21.15.47:0/2485982677 >> 172.21.15.47:0/2485982677 conn(0x56004b75e310 msgr2=0x56004b7542e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:27:23.019 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:23.019+0000 7f1bf1600640 0 client.16416 ms_handle_reset on v2:172.21.15.100:6838/4173168440 2024-03-21T10:27:23.020 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:23.019+0000 7f1c11a26740 1 -- 172.21.15.47:0/2485982677 shutdown_connections 2024-03-21T10:27:23.020 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:23.019+0000 7f1c11a26740 1 -- 172.21.15.47:0/2485982677 wait complete. 2024-03-21T10:27:23.020 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:23.019+0000 7f1c11a26740 20 client.16416 trim_cache size 0 max 16384 2024-03-21T10:27:23.020 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout:{ 2024-03-21T10:27:23.020 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout: "message": "", 2024-03-21T10:27:23.021 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout: "return_code": 0 2024-03-21T10:27:23.021 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout:} 2024-03-21T10:27:23.021 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:23.020+0000 7f1c0d600640 1 -- 172.21.15.47:0/1295712256 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f1bec0a3080 msgr2=0x7f1bec0a5540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:27:23.021 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:23.020+0000 7f1c0d600640 1 --2- 172.21.15.47:0/1295712256 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f1bec0a3080 0x7f1bec0a5540 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f1c08069860 tx=0x7f1c00011810 comp rx=0 tx=0).stop 2024-03-21T10:27:23.021 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:23.020+0000 7f1c0d600640 1 -- 172.21.15.47:0/1295712256 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1c08074390 msgr2=0x7f1c0819a480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:27:23.021 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:23.020+0000 7f1c0d600640 1 --2- 172.21.15.47:0/1295712256 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1c08074390 0x7f1c0819a480 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7f1bf400e9f0 tx=0x7f1bf400eeb0 comp rx=0 tx=0).stop 2024-03-21T10:27:23.022 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:23.020+0000 7f1c0d600640 1 -- 172.21.15.47:0/1295712256 shutdown_connections 2024-03-21T10:27:23.022 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:23.020+0000 7f1c0d600640 1 --2- 172.21.15.47:0/1295712256 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f1bec0a3080 0x7f1bec0a5540 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:27:23.022 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:23.020+0000 7f1c0d600640 1 --2- 172.21.15.47:0/1295712256 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1c0810dd40 0x7f1c0819e7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:27:23.022 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:23.020+0000 7f1c0d600640 1 --2- 172.21.15.47:0/1295712256 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1c08074390 0x7f1c0819a480 unknown :-1 s=CLOSED pgs=684 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:27:23.022 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:23.020+0000 7f1c0d600640 1 --2- 172.21.15.47:0/1295712256 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1c08073a70 0x7f1c08199f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:27:23.022 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:23.020+0000 7f1c0d600640 1 -- 172.21.15.47:0/1295712256 >> 172.21.15.47:0/1295712256 conn(0x7f1c080fcfb0 msgr2=0x7f1c080fde00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:27:23.022 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:23.020+0000 7f1c0d600640 1 -- 172.21.15.47:0/1295712256 shutdown_connections 2024-03-21T10:27:23.022 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:23.021+0000 7f1c0d600640 1 -- 172.21.15.47:0/1295712256 wait complete. 2024-03-21T10:27:23.107 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:23.106+0000 7f9d57e00640 1 -- 172.21.15.47:0/3308666520 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9d58109730 msgr2=0x7f9d58109b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:27:23.107 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:23.106+0000 7f9d57e00640 1 --2- 172.21.15.47:0/3308666520 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9d58109730 0x7f9d58109b10 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7f9d48009930 tx=0x7f9d4802bc30 comp rx=0 tx=0).stop 2024-03-21T10:27:23.107 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:23.107+0000 7f9d57e00640 1 -- 172.21.15.47:0/3308666520 shutdown_connections 2024-03-21T10:27:23.107 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:23.107+0000 7f9d57e00640 1 --2- 172.21.15.47:0/3308666520 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9d581040e0 0x7f9d581105f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:27:23.107 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:23.107+0000 7f9d57e00640 1 --2- 172.21.15.47:0/3308666520 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9d58103720 0x7f9d58103ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:27:23.107 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:23.107+0000 7f9d57e00640 1 --2- 172.21.15.47:0/3308666520 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9d58109730 0x7f9d58109b10 unknown :-1 s=CLOSED pgs=688 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:27:23.107 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:23.107+0000 7f9d57e00640 1 -- 172.21.15.47:0/3308666520 >> 172.21.15.47:0/3308666520 conn(0x7f9d580ff1b0 msgr2=0x7f9d581015d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:27:23.107 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:23.107+0000 7f9d57e00640 1 -- 172.21.15.47:0/3308666520 shutdown_connections 2024-03-21T10:27:23.108 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:23.107+0000 7f9d57e00640 1 -- 172.21.15.47:0/3308666520 wait complete. 2024-03-21T10:27:23.109 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:23.108+0000 7f9d57e00640 1 Processor -- start 2024-03-21T10:27:23.109 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:23.108+0000 7f9d57e00640 1 -- start start 2024-03-21T10:27:23.109 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:23.108+0000 7f9d57e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9d58103720 0x7f9d58199fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:27:23.109 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:23.108+0000 7f9d57e00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9d581040e0 0x7f9d5819a500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:27:23.109 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:23.108+0000 7f9d57e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9d58109730 0x7f9d5819e880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:27:23.110 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:23.108+0000 7f9d57e00640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d58113a40 con 0x7f9d58103720 2024-03-21T10:27:23.110 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:23.108+0000 7f9d57e00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d581138c0 con 0x7f9d58109730 2024-03-21T10:27:23.110 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:23.108+0000 7f9d57e00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d58113bc0 con 0x7f9d581040e0 2024-03-21T10:27:23.110 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:23.109+0000 7f9d56a00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9d58103720 0x7f9d58199fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:27:23.110 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:23.109+0000 7f9d57400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9d58109730 0x7f9d5819e880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:27:23.110 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:23.109+0000 7f9d56a00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9d58103720 0x7f9d58199fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:34728/0 (socket says 172.21.15.47:34728) 2024-03-21T10:27:23.110 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:23.109+0000 7f9d56a00640 1 -- 172.21.15.47:0/3369102929 learned_addr learned my addr 172.21.15.47:0/3369102929 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:27:23.110 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:23.109+0000 7f9d56000640 1 --2- 172.21.15.47:0/3369102929 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9d581040e0 0x7f9d5819a500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:27:23.110 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:23.109+0000 7f9d56a00640 1 -- 172.21.15.47:0/3369102929 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9d581040e0 msgr2=0x7f9d5819a500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:27:23.110 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:23.109+0000 7f9d56a00640 1 --2- 172.21.15.47:0/3369102929 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9d581040e0 0x7f9d5819a500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:27:23.110 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:23.109+0000 7f9d56a00640 1 -- 172.21.15.47:0/3369102929 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9d58109730 msgr2=0x7f9d5819e880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:27:23.110 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:23.109+0000 7f9d56a00640 1 --2- 172.21.15.47:0/3369102929 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9d58109730 0x7f9d5819e880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:27:23.110 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:23.109+0000 7f9d56a00640 1 -- 172.21.15.47:0/3369102929 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d5819efb0 con 0x7f9d58103720 2024-03-21T10:27:23.111 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:23.109+0000 7f9d57400640 1 --2- 172.21.15.47:0/3369102929 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9d58109730 0x7f9d5819e880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T10:27:23.111 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:23.109+0000 7f9d56000640 1 --2- 172.21.15.47:0/3369102929 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9d581040e0 0x7f9d5819a500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T10:27:23.111 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:23.110+0000 7f9d56a00640 1 --2- 172.21.15.47:0/3369102929 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9d58103720 0x7f9d58199fc0 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7f9d48000c00 tx=0x7f9d480045e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:27:23.111 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:23.110+0000 7f9d4f400640 1 -- 172.21.15.47:0/3369102929 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d48043020 con 0x7f9d58103720 2024-03-21T10:27:23.111 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:23.110+0000 7f9d57e00640 1 -- 172.21.15.47:0/3369102929 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9d5819f240 con 0x7f9d58103720 2024-03-21T10:27:23.111 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:23.110+0000 7f9d4f400640 1 -- 172.21.15.47:0/3369102929 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f9d4802cb70 con 0x7f9d58103720 2024-03-21T10:27:23.111 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:23.110+0000 7f9d57e00640 1 -- 172.21.15.47:0/3369102929 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9d581a6b30 con 0x7f9d58103720 2024-03-21T10:27:23.112 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:23.110+0000 7f9d4f400640 1 -- 172.21.15.47:0/3369102929 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d48003b20 con 0x7f9d58103720 2024-03-21T10:27:23.113 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:23.114+0000 7f9d4f400640 1 -- 172.21.15.47:0/3369102929 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 27) v1 ==== 137985+0+0 (secure 0 0 0) 0x7f9d48046020 con 0x7f9d58103720 2024-03-21T10:27:23.114 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:23.114+0000 7f9d6076b740 1 Processor -- start 2024-03-21T10:27:23.114 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:23.115+0000 7f9d6076b740 1 -- start start 2024-03-21T10:27:23.115 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:23.115+0000 7f9d6076b740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d919e8b8b0 0x55d919e8bc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:27:23.116 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:23.115+0000 7f9d6076b740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d919e8c260 0x55d919e94810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:27:23.116 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:23.115+0000 7f9d6076b740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d919e94de0 0x55d919e97230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:27:23.116 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:23.115+0000 7f9d6076b740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d919def2c0 con 0x55d919e8c260 2024-03-21T10:27:23.116 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:23.115+0000 7f9d6076b740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d919d97c90 con 0x55d919e94de0 2024-03-21T10:27:23.116 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:23.115+0000 7f9d6076b740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d919deb360 con 0x55d919e8b8b0 2024-03-21T10:27:23.116 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:23.116+0000 7f9d56a00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d919e8c260 0x55d919e94810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:27:23.116 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:23.116+0000 7f9d56a00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d919e8c260 0x55d919e94810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:34730/0 (socket says 172.21.15.47:34730) 2024-03-21T10:27:23.116 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:23.116+0000 7f9d56a00640 1 -- 172.21.15.47:0/45433875 learned_addr learned my addr 172.21.15.47:0/45433875 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:27:23.116 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:23.116+0000 7f9d4f400640 1 --2- 172.21.15.47:0/3369102929 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f9d3c0a3090 0x7f9d3c0a5550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:27:23.116 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:23.116+0000 7f9d56000640 1 --2- 172.21.15.47:0/45433875 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d919e94de0 0x55d919e97230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:27:23.117 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:23.116+0000 7f9d57400640 1 --2- 172.21.15.47:0/45433875 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d919e8b8b0 0x55d919e8bc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:27:23.118 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:23.116+0000 7f9d57400640 1 --2- 172.21.15.47:0/3369102929 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f9d3c0a3090 0x7f9d3c0a5550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:27:23.118 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:23.116+0000 7f9d56000640 1 -- 172.21.15.47:0/45433875 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d919e8b8b0 msgr2=0x55d919e8bc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:27:23.118 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:23.116+0000 7f9d56000640 1 --2- 172.21.15.47:0/45433875 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d919e8b8b0 0x55d919e8bc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:27:23.118 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:23.116+0000 7f9d56000640 1 -- 172.21.15.47:0/45433875 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d919e8c260 msgr2=0x55d919e94810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:27:23.118 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:23.116+0000 7f9d56000640 1 --2- 172.21.15.47:0/45433875 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d919e8c260 0x55d919e94810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:27:23.118 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:23.117+0000 7f9d56000640 1 -- 172.21.15.47:0/45433875 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d919dbc080 con 0x55d919e94de0 2024-03-21T10:27:23.118 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:23.117+0000 7f9d4f400640 1 -- 172.21.15.47:0/3369102929 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 9474+0+0 (secure 0 0 0) 0x7f9d480f1bf0 con 0x7f9d58103720 2024-03-21T10:27:23.118 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:23.117+0000 7f9d56000640 1 --2- 172.21.15.47:0/45433875 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d919e94de0 0x55d919e97230 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f9d44009b60 tx=0x7f9d4400e6c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5f18ec27be0e23b5 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:27:23.118 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:23.117+0000 7f9d57400640 1 --2- 172.21.15.47:0/3369102929 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f9d3c0a3090 0x7f9d3c0a5550 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f9d500094d0 tx=0x7f9d50004ab0 comp rx=0 tx=0).ready entity=mgr.15687 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:27:23.118 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:23.117+0000 7f9d43e00640 1 -- 172.21.15.47:0/45433875 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d44016070 con 0x55d919e94de0 2024-03-21T10:27:23.118 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:23.118+0000 7f9d43e00640 1 -- 172.21.15.47:0/45433875 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f9d44009e50 con 0x55d919e94de0 2024-03-21T10:27:23.118 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:23.118+0000 7f9d43e00640 1 -- 172.21.15.47:0/45433875 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d44010700 con 0x55d919e94de0 2024-03-21T10:27:23.119 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:23.119+0000 7f9d6076b740 1 -- 172.21.15.47:0/45433875 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d919e94de0 msgr2=0x55d919e97230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:27:23.119 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:23.119+0000 7f9d6076b740 1 --2- 172.21.15.47:0/45433875 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d919e94de0 0x55d919e97230 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f9d44009b60 tx=0x7f9d4400e6c0 comp rx=0 tx=0).stop 2024-03-21T10:27:23.119 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:23.119+0000 7f9d6076b740 1 -- 172.21.15.47:0/45433875 shutdown_connections 2024-03-21T10:27:23.119 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:23.119+0000 7f9d6076b740 1 --2- 172.21.15.47:0/45433875 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d919e94de0 0x55d919e97230 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:27:23.120 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:23.119+0000 7f9d6076b740 1 --2- 172.21.15.47:0/45433875 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d919e8c260 0x55d919e94810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:27:23.120 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:23.119+0000 7f9d6076b740 1 --2- 172.21.15.47:0/45433875 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d919e8b8b0 0x55d919e8bc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:27:23.120 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:23.119+0000 7f9d6076b740 1 -- 172.21.15.47:0/45433875 >> 172.21.15.47:0/45433875 conn(0x55d919e85f80 msgr2=0x55d919e86360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:27:23.120 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:23.119+0000 7f9d6076b740 1 -- 172.21.15.47:0/45433875 shutdown_connections 2024-03-21T10:27:23.120 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:23.119+0000 7f9d6076b740 1 -- 172.21.15.47:0/45433875 wait complete. 2024-03-21T10:27:23.121 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:23.120+0000 7f9d6076b740 1 -- start start 2024-03-21T10:27:23.121 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:23.120+0000 7f9d6076b740 1 Processor -- start 2024-03-21T10:27:23.121 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:23.121+0000 7f9d6076b740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d919e8b8b0 0x55d919e94640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:27:23.121 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:23.121+0000 7f9d6076b740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d919e8c260 0x55d919e919d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:27:23.121 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:23.121+0000 7f9d6076b740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d919e94de0 0x55d919e91f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:27:23.122 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:23.121+0000 7f9d6076b740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d919e97d50 con 0x55d919e8b8b0 2024-03-21T10:27:23.122 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:23.121+0000 7f9d6076b740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d919dab0d0 con 0x55d919e8c260 2024-03-21T10:27:23.122 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:23.121+0000 7f9d6076b740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d919e97ed0 con 0x55d919e94de0 2024-03-21T10:27:23.122 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:23.121+0000 7f9d6076b740 10 client.0 _ll_register_callbacks cb 0x55d919dafe80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T10:27:23.122 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:23.121+0000 7f9d56000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d919e8c260 0x55d919e919d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:27:23.122 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:23.121+0000 7f9d56000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d919e8c260 0x55d919e919d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:45136/0 (socket says 172.21.15.47:45136) 2024-03-21T10:27:23.122 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:23.121+0000 7f9d57400640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d919e94de0 0x55d919e91f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:27:23.122 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:23.121+0000 7f9d56000640 1 -- 172.21.15.47:0/4251839933 learned_addr learned my addr 172.21.15.47:0/4251839933 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:27:23.122 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:23.121+0000 7f9d56a00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d919e8b8b0 0x55d919e94640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:27:23.123 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:23.122+0000 7f9d57400640 1 -- 172.21.15.47:0/4251839933 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d919e8c260 msgr2=0x55d919e919d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:27:23.123 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:23.122+0000 7f9d57400640 1 --2- 172.21.15.47:0/4251839933 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d919e8c260 0x55d919e919d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:27:23.123 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:23.122+0000 7f9d57400640 1 -- 172.21.15.47:0/4251839933 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d919e8b8b0 msgr2=0x55d919e94640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:27:23.124 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:23.122+0000 7f9d57400640 1 --2- 172.21.15.47:0/4251839933 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d919e8b8b0 0x55d919e94640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:27:23.124 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:23.122+0000 7f9d57400640 1 -- 172.21.15.47:0/4251839933 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55d919b033f0 con 0x55d919e94de0 2024-03-21T10:27:23.124 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:23.122+0000 7f9d56000640 1 --2- 172.21.15.47:0/4251839933 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d919e8c260 0x55d919e919d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T10:27:23.124 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:23.122+0000 7f9d56a00640 1 --2- 172.21.15.47:0/4251839933 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d919e8b8b0 0x55d919e94640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T10:27:23.124 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:23.123+0000 7f9d57400640 1 --2- 172.21.15.47:0/4251839933 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d919e94de0 0x55d919e91f10 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f9d50005fd0 tx=0x7f9d50004a40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2634b87985a42a1c server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:27:23.124 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:23.123+0000 7f9d40c00640 10 client.0 ms_handle_connect on v2:172.21.15.157:3300/0 2024-03-21T10:27:23.124 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:23.123+0000 7f9d40c00640 1 -- 172.21.15.47:0/4251839933 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d50026030 con 0x55d919e94de0 2024-03-21T10:27:23.124 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:23.123+0000 7f9d6076b740 1 -- 172.21.15.47:0/4251839933 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f9d44009e50 con 0x55d919e94de0 2024-03-21T10:27:23.124 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:23.123+0000 7f9d40c00640 1 -- 172.21.15.47:0/4251839933 <== mon.1 v2:172.21.15.157:3300/0 2 ==== osd_map(97..97 src has 1..97) v4 ==== 9474+0+0 (secure 0 0 0) 0x7f9d50003e00 con 0x55d919e94de0 2024-03-21T10:27:23.125 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:23.124+0000 7f9d40c00640 1 -- 172.21.15.47:0/4251839933 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9d50030050 con 0x55d919e94de0 2024-03-21T10:27:23.125 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:23.124+0000 7f9d6076b740 10 client.25984 fetch_fsmap learned FSMap version 32 2024-03-21T10:27:23.125 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:23.124+0000 7f9d6076b740 1 -- 172.21.15.47:0/4251839933 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x55d919bcd7d0 con 0x55d919e94de0 2024-03-21T10:27:23.125 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:23.124+0000 7f9d40c00640 1 -- 172.21.15.47:0/4251839933 <== mon.1 v2:172.21.15.157:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f9d44009e50 con 0x55d919e94de0 2024-03-21T10:27:23.125 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:23.125+0000 7f9d40c00640 1 -- 172.21.15.47:0/4251839933 <== mon.1 v2:172.21.15.157:3300/0 5 ==== fsmap(e 32) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f9d5001f070 con 0x55d919e94de0 2024-03-21T10:27:23.125 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:23.125+0000 7f9d6076b740 10 client.25984 fetch_fsmap finished waiting for FSMap version 32 2024-03-21T10:27:23.125 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:23.125+0000 7f9d6076b740 10 client.25984 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.k 2024-03-21T10:27:23.126 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:23.125+0000 7f9d6076b740 20 client.25984 populate_metadata read hostname 'smithi047' 2024-03-21T10:27:23.126 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:23.125+0000 7f9d6076b740 1 --2- 172.21.15.47:0/4251839933 >> [v2:172.21.15.100:6832/2723302038,v1:172.21.15.100:6833/2723302038] conn(0x55d919ea4c10 0x55d919ea7050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:27:23.126 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:23.125+0000 7f9d6076b740 4 client.25984 mds_command: new command op to 34427 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T10:27:23.126 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:23.125+0000 7f9d6076b740 1 -- 172.21.15.47:0/4251839933 --> [v2:172.21.15.100:6832/2723302038,v1:172.21.15.100:6833/2723302038] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55d919dfc800 con 0x55d919ea4c10 2024-03-21T10:27:23.126 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:23.125+0000 7f9d56a00640 1 --2- 172.21.15.47:0/4251839933 >> [v2:172.21.15.100:6832/2723302038,v1:172.21.15.100:6833/2723302038] conn(0x55d919ea4c10 0x55d919ea7050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:27:23.126 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:23.126+0000 7f9d56a00640 1 --2- 172.21.15.47:0/4251839933 >> [v2:172.21.15.100:6832/2723302038,v1:172.21.15.100:6833/2723302038] conn(0x55d919ea4c10 0x55d919ea7050 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=81e2496056f7a685 server_cookie=a3ae60e38bff2159 in_seq=0 out_seq=0 2024-03-21T10:27:23.126 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:23.126+0000 7f9d40c00640 10 client.25984 ms_handle_connect on v2:172.21.15.100:6832/2723302038 2024-03-21T10:27:23.128 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:23.128+0000 7f9d40c00640 1 -- 172.21.15.47:0/4251839933 <== mds.3 v2:172.21.15.100:6832/2723302038 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55d919dfc800 con 0x55d919ea4c10 2024-03-21T10:27:23.128 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:23.128+0000 7f9d40c00640 10 client.25984 handle_command_reply: tid=0 2024-03-21T10:27:23.128 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:23.128+0000 7f9d6076b740 1 client.25984 shutdown 2024-03-21T10:27:23.129 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:23.128+0000 7f9d6076b740 1 -- 172.21.15.47:0/4251839933 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d919e94de0 msgr2=0x55d919e91f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:27:23.129 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:23.128+0000 7f9d6076b740 1 --2- 172.21.15.47:0/4251839933 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d919e94de0 0x55d919e91f10 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f9d50005fd0 tx=0x7f9d50004a40 comp rx=0 tx=0).stop 2024-03-21T10:27:23.129 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:23.129+0000 7f9d6076b740 1 -- 172.21.15.47:0/4251839933 shutdown_connections 2024-03-21T10:27:23.129 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:23.129+0000 7f9d6076b740 1 --2- 172.21.15.47:0/4251839933 >> [v2:172.21.15.100:6832/2723302038,v1:172.21.15.100:6833/2723302038] conn(0x55d919ea4c10 0x55d919ea7050 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:27:23.129 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:23.129+0000 7f9d6076b740 1 --2- 172.21.15.47:0/4251839933 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d919e94de0 0x55d919e91f10 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:27:23.129 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:23.129+0000 7f9d6076b740 1 --2- 172.21.15.47:0/4251839933 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d919e8c260 0x55d919e919d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:27:23.129 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:23.129+0000 7f9d6076b740 1 --2- 172.21.15.47:0/4251839933 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d919e8b8b0 0x55d919e94640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:27:23.129 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:23.129+0000 7f9d40c00640 0 client.25984 ms_handle_reset on v2:172.21.15.100:6832/2723302038 2024-03-21T10:27:23.129 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:23.129+0000 7f9d6076b740 1 -- 172.21.15.47:0/4251839933 >> 172.21.15.47:0/4251839933 conn(0x55d919e85f80 msgr2=0x55d919e87cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:27:23.130 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:23.129+0000 7f9d6076b740 1 -- 172.21.15.47:0/4251839933 shutdown_connections 2024-03-21T10:27:23.130 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:23.129+0000 7f9d6076b740 1 -- 172.21.15.47:0/4251839933 wait complete. 2024-03-21T10:27:23.130 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:23.129+0000 7f9d6076b740 20 client.25984 trim_cache size 0 max 16384 2024-03-21T10:27:23.146 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:23.146+0000 7f9d6076b740 1 Processor -- start 2024-03-21T10:27:23.146 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:23.146+0000 7f9d6076b740 1 -- start start 2024-03-21T10:27:23.147 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:23.146+0000 7f9d6076b740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d919e8b8b0 0x55d919e94100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:27:23.147 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:23.146+0000 7f9d6076b740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d919e8c260 0x55d919e94640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:27:23.147 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:23.146+0000 7f9d6076b740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d919e94de0 0x55d919e8ec50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:27:23.147 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:23.146+0000 7f9d6076b740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d581b0970 con 0x55d919e8c260 2024-03-21T10:27:23.147 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:23.146+0000 7f9d6076b740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d581b22f0 con 0x55d919e94de0 2024-03-21T10:27:23.147 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:23.146+0000 7f9d6076b740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d581b0fd0 con 0x55d919e8b8b0 2024-03-21T10:27:23.147 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:23.147+0000 7f9d57400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d919e94de0 0x55d919e8ec50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:27:23.148 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:23.147+0000 7f9d56000640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d919e8c260 0x55d919e94640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:27:23.148 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:23.147+0000 7f9d57400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d919e94de0 0x55d919e8ec50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:45148/0 (socket says 172.21.15.47:45148) 2024-03-21T10:27:23.148 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:23.147+0000 7f9d57400640 1 -- 172.21.15.47:0/2591875829 learned_addr learned my addr 172.21.15.47:0/2591875829 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:27:23.148 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:23.147+0000 7f9d56a00640 1 --2- 172.21.15.47:0/2591875829 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d919e8b8b0 0x55d919e94100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:27:23.148 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:23.147+0000 7f9d57400640 1 -- 172.21.15.47:0/2591875829 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d919e8b8b0 msgr2=0x55d919e94100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:27:23.148 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:23.147+0000 7f9d57400640 1 --2- 172.21.15.47:0/2591875829 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d919e8b8b0 0x55d919e94100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:27:23.148 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:23.147+0000 7f9d57400640 1 -- 172.21.15.47:0/2591875829 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d919e8c260 msgr2=0x55d919e94640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:27:23.148 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:23.147+0000 7f9d57400640 1 --2- 172.21.15.47:0/2591875829 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d919e8c260 0x55d919e94640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:27:23.148 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:23.147+0000 7f9d57400640 1 -- 172.21.15.47:0/2591875829 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d919e80570 con 0x55d919e94de0 2024-03-21T10:27:23.149 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:23.148+0000 7f9d57400640 1 --2- 172.21.15.47:0/2591875829 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d919e94de0 0x55d919e8ec50 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f9d50000f80 tx=0x7f9d50005ac0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3f8504d36dec6417 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:27:23.149 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:23.148+0000 7f9d43400640 1 -- 172.21.15.47:0/2591875829 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d50026040 con 0x55d919e94de0 2024-03-21T10:27:23.150 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:23.148+0000 7f9d43400640 1 -- 172.21.15.47:0/2591875829 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f9d500047e0 con 0x55d919e94de0 2024-03-21T10:27:23.150 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:23.148+0000 7f9d43400640 1 -- 172.21.15.47:0/2591875829 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d50010690 con 0x55d919e94de0 2024-03-21T10:27:23.150 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:23.149+0000 7f9d6076b740 1 -- 172.21.15.47:0/2591875829 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d919e94de0 msgr2=0x55d919e8ec50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:27:23.150 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:23.149+0000 7f9d6076b740 1 --2- 172.21.15.47:0/2591875829 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d919e94de0 0x55d919e8ec50 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f9d50000f80 tx=0x7f9d50005ac0 comp rx=0 tx=0).stop 2024-03-21T10:27:23.150 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:23.149+0000 7f9d6076b740 1 -- 172.21.15.47:0/2591875829 shutdown_connections 2024-03-21T10:27:23.150 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:23.149+0000 7f9d6076b740 1 --2- 172.21.15.47:0/2591875829 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d919e94de0 0x55d919e8ec50 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:27:23.150 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:23.149+0000 7f9d6076b740 1 --2- 172.21.15.47:0/2591875829 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d919e8c260 0x55d919e94640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:27:23.150 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:23.149+0000 7f9d6076b740 1 --2- 172.21.15.47:0/2591875829 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d919e8b8b0 0x55d919e94100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:27:23.150 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:23.149+0000 7f9d6076b740 1 -- 172.21.15.47:0/2591875829 >> 172.21.15.47:0/2591875829 conn(0x55d919e85f80 msgr2=0x55d919e85160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:27:23.150 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:23.149+0000 7f9d6076b740 1 -- 172.21.15.47:0/2591875829 shutdown_connections 2024-03-21T10:27:23.150 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:23.149+0000 7f9d6076b740 1 -- 172.21.15.47:0/2591875829 wait complete. 2024-03-21T10:27:23.150 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:23.149+0000 7f9d6076b740 1 -- start start 2024-03-21T10:27:23.150 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:23.150+0000 7f9d6076b740 1 Processor -- start 2024-03-21T10:27:23.153 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:23.150+0000 7f9d6076b740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d919e8b8b0 0x55d919e98720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:27:23.153 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:23.150+0000 7f9d6076b740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d919e8c260 0x55d919e98c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:27:23.153 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:23.150+0000 7f9d6076b740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d919e94de0 0x55d919e991a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:27:23.153 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:23.150+0000 7f9d6076b740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d581b1c90 con 0x55d919e8c260 2024-03-21T10:27:23.153 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:23.150+0000 7f9d6076b740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d581b1630 con 0x55d919e94de0 2024-03-21T10:27:23.153 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:23.150+0000 7f9d6076b740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d581aeb30 con 0x55d919e8b8b0 2024-03-21T10:27:23.153 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:23.151+0000 7f9d6076b740 10 client.0 _ll_register_callbacks cb 0x7f9d500136e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T10:27:23.153 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:23.151+0000 7f9d56000640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d919e8c260 0x55d919e98c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:27:23.153 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:23.151+0000 7f9d57400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d919e94de0 0x55d919e991a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:27:23.153 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:23.151+0000 7f9d57400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d919e94de0 0x55d919e991a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:45152/0 (socket says 172.21.15.47:45152) 2024-03-21T10:27:23.154 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:23.151+0000 7f9d57400640 1 -- 172.21.15.47:0/1276137612 learned_addr learned my addr 172.21.15.47:0/1276137612 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:27:23.154 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:23.151+0000 7f9d56a00640 1 --2- 172.21.15.47:0/1276137612 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d919e8b8b0 0x55d919e98720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:27:23.154 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:23.151+0000 7f9d57400640 1 -- 172.21.15.47:0/1276137612 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d919e8b8b0 msgr2=0x55d919e98720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:27:23.154 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:23.151+0000 7f9d57400640 1 --2- 172.21.15.47:0/1276137612 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d919e8b8b0 0x55d919e98720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:27:23.154 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:23.151+0000 7f9d57400640 1 -- 172.21.15.47:0/1276137612 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d919e8c260 msgr2=0x55d919e98c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:27:23.154 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:23.151+0000 7f9d57400640 1 --2- 172.21.15.47:0/1276137612 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d919e8c260 0x55d919e98c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:27:23.154 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:23.151+0000 7f9d57400640 1 -- 172.21.15.47:0/1276137612 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55d919eff6e0 con 0x55d919e94de0 2024-03-21T10:27:23.154 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:23.151+0000 7f9d56000640 1 --2- 172.21.15.47:0/1276137612 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d919e8c260 0x55d919e98c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T10:27:23.154 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:23.152+0000 7f9d57400640 1 --2- 172.21.15.47:0/1276137612 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d919e94de0 0x55d919e991a0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f9d50005fd0 tx=0x7f9d50022e20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4eec3ade3269f5b9 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:27:23.154 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:23.152+0000 7f9d40c00640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-21T10:27:23.154 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:23.152+0000 7f9d40c00640 1 -- 172.21.15.47:0/1276137612 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d50041070 con 0x55d919e94de0 2024-03-21T10:27:23.154 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:23.152+0000 7f9d6076b740 1 -- 172.21.15.47:0/1276137612 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f9d500047e0 con 0x55d919e94de0 2024-03-21T10:27:23.154 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:23.152+0000 7f9d40c00640 1 -- 172.21.15.47:0/1276137612 <== mon.2 v2:172.21.15.100:3300/0 2 ==== osd_map(97..97 src has 1..97) v4 ==== 9474+0+0 (secure 0 0 0) 0x7f9d50022820 con 0x55d919e94de0 2024-03-21T10:27:23.154 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:23.152+0000 7f9d40c00640 1 -- 172.21.15.47:0/1276137612 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9d5004d050 con 0x55d919e94de0 2024-03-21T10:27:23.155 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:23.152+0000 7f9d6076b740 10 client.36224 fetch_fsmap learned FSMap version 32 2024-03-21T10:27:23.155 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:23.152+0000 7f9d6076b740 1 -- 172.21.15.47:0/1276137612 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x55d919f009a0 con 0x55d919e94de0 2024-03-21T10:27:23.155 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:23.153+0000 7f9d40c00640 1 -- 172.21.15.47:0/1276137612 <== mon.2 v2:172.21.15.100:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f9d500047e0 con 0x55d919e94de0 2024-03-21T10:27:23.155 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:23.153+0000 7f9d40c00640 1 -- 172.21.15.47:0/1276137612 <== mon.2 v2:172.21.15.100:3300/0 5 ==== fsmap(e 32) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f9d5001f040 con 0x55d919e94de0 2024-03-21T10:27:23.155 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:23.153+0000 7f9d6076b740 10 client.36224 fetch_fsmap finished waiting for FSMap version 32 2024-03-21T10:27:23.155 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:23.153+0000 7f9d6076b740 10 client.36224 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.k 2024-03-21T10:27:23.155 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:23.153+0000 7f9d6076b740 20 client.36224 populate_metadata read hostname 'smithi047' 2024-03-21T10:27:23.155 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:23.153+0000 7f9d6076b740 1 --2- 172.21.15.47:0/1276137612 >> [v2:172.21.15.100:6832/2723302038,v1:172.21.15.100:6833/2723302038] conn(0x55d919f00f00 0x55d919f03320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:27:23.155 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:23.153+0000 7f9d6076b740 4 client.36224 mds_command: new command op to 34427 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-21T10:27:23.155 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:23.153+0000 7f9d6076b740 1 -- 172.21.15.47:0/1276137612 --> [v2:172.21.15.100:6832/2723302038,v1:172.21.15.100:6833/2723302038] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55d919df6b50 con 0x55d919f00f00 2024-03-21T10:27:23.157 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:23.153+0000 7f9d56a00640 1 --2- 172.21.15.47:0/1276137612 >> [v2:172.21.15.100:6832/2723302038,v1:172.21.15.100:6833/2723302038] conn(0x55d919f00f00 0x55d919f03320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:27:23.157 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:23.154+0000 7f9d56a00640 1 --2- 172.21.15.47:0/1276137612 >> [v2:172.21.15.100:6832/2723302038,v1:172.21.15.100:6833/2723302038] conn(0x55d919f00f00 0x55d919f03320 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=ca63a7457f6a4aae server_cookie=f915d738ad76331d in_seq=0 out_seq=0 2024-03-21T10:27:23.157 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:23.154+0000 7f9d40c00640 10 client.36224 ms_handle_connect on v2:172.21.15.100:6832/2723302038 2024-03-21T10:27:23.157 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:23.154+0000 7f9d40c00640 1 -- 172.21.15.47:0/1276137612 <== mds.3 v2:172.21.15.100:6832/2723302038 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55d919df6b50 con 0x55d919f00f00 2024-03-21T10:27:23.157 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:23.154+0000 7f9d40c00640 10 client.36224 handle_command_reply: tid=0 2024-03-21T10:27:23.157 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:23.154+0000 7f9d6076b740 1 client.36224 shutdown 2024-03-21T10:27:23.157 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:23.155+0000 7f9d6076b740 1 -- 172.21.15.47:0/1276137612 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d919e94de0 msgr2=0x55d919e991a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:27:23.157 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:23.155+0000 7f9d6076b740 1 --2- 172.21.15.47:0/1276137612 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d919e94de0 0x55d919e991a0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f9d50005fd0 tx=0x7f9d50022e20 comp rx=0 tx=0).stop 2024-03-21T10:27:23.157 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:23.155+0000 7f9d6076b740 1 -- 172.21.15.47:0/1276137612 shutdown_connections 2024-03-21T10:27:23.157 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:23.155+0000 7f9d6076b740 1 --2- 172.21.15.47:0/1276137612 >> [v2:172.21.15.100:6832/2723302038,v1:172.21.15.100:6833/2723302038] conn(0x55d919f00f00 0x55d919f03320 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:27:23.157 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:23.155+0000 7f9d6076b740 1 --2- 172.21.15.47:0/1276137612 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d919e94de0 0x55d919e991a0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:27:23.157 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:23.155+0000 7f9d6076b740 1 --2- 172.21.15.47:0/1276137612 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d919e8c260 0x55d919e98c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:27:23.157 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:23.155+0000 7f9d6076b740 1 --2- 172.21.15.47:0/1276137612 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d919e8b8b0 0x55d919e98720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:27:23.158 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:23.155+0000 7f9d40c00640 0 client.36224 ms_handle_reset on v2:172.21.15.100:6832/2723302038 2024-03-21T10:27:23.158 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:23.155+0000 7f9d6076b740 1 -- 172.21.15.47:0/1276137612 >> 172.21.15.47:0/1276137612 conn(0x55d919e85f80 msgr2=0x55d919e8c640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:27:23.158 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:23.155+0000 7f9d6076b740 1 -- 172.21.15.47:0/1276137612 shutdown_connections 2024-03-21T10:27:23.158 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:23.155+0000 7f9d6076b740 1 -- 172.21.15.47:0/1276137612 wait complete. 2024-03-21T10:27:23.158 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:23.155+0000 7f9d6076b740 20 client.36224 trim_cache size 0 max 16384 2024-03-21T10:27:23.158 INFO:teuthology.orchestra.run.smithi047.stdout:[] 2024-03-21T10:27:23.159 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:23.156+0000 7f9d57e00640 1 -- 172.21.15.47:0/3369102929 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f9d3c0a3090 msgr2=0x7f9d3c0a5550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:27:23.159 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:23.156+0000 7f9d57e00640 1 --2- 172.21.15.47:0/3369102929 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f9d3c0a3090 0x7f9d3c0a5550 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f9d500094d0 tx=0x7f9d50004ab0 comp rx=0 tx=0).stop 2024-03-21T10:27:23.159 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:23.156+0000 7f9d57e00640 1 -- 172.21.15.47:0/3369102929 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9d58103720 msgr2=0x7f9d58199fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:27:23.159 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:23.156+0000 7f9d57e00640 1 --2- 172.21.15.47:0/3369102929 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9d58103720 0x7f9d58199fc0 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7f9d48000c00 tx=0x7f9d480045e0 comp rx=0 tx=0).stop 2024-03-21T10:27:23.159 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:23.156+0000 7f9d57e00640 1 -- 172.21.15.47:0/3369102929 shutdown_connections 2024-03-21T10:27:23.159 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:23.156+0000 7f9d57e00640 1 --2- 172.21.15.47:0/3369102929 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f9d3c0a3090 0x7f9d3c0a5550 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:27:23.159 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:23.156+0000 7f9d57e00640 1 --2- 172.21.15.47:0/3369102929 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9d58109730 0x7f9d5819e880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:27:23.159 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:23.156+0000 7f9d57e00640 1 --2- 172.21.15.47:0/3369102929 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9d581040e0 0x7f9d5819a500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:27:23.159 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:23.156+0000 7f9d57e00640 1 --2- 172.21.15.47:0/3369102929 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9d58103720 0x7f9d58199fc0 unknown :-1 s=CLOSED pgs=689 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:27:23.159 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:23.156+0000 7f9d57e00640 1 -- 172.21.15.47:0/3369102929 >> 172.21.15.47:0/3369102929 conn(0x7f9d580ff1b0 msgr2=0x7f9d580ffa20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:27:23.160 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:23.156+0000 7f9d57e00640 1 -- 172.21.15.47:0/3369102929 shutdown_connections 2024-03-21T10:27:23.160 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:23.156+0000 7f9d57e00640 1 -- 172.21.15.47:0/3369102929 wait complete. 2024-03-21T10:27:23.175 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-21T10:27:23.176 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-03-21T10:27:23.397 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:23.397+0000 7ff9c4e00640 1 -- 172.21.15.47:0/2035513673 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff9c0100c60 msgr2=0x7ff9c010d920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:27:23.397 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:23.397+0000 7ff9c4e00640 1 --2- 172.21.15.47:0/2035513673 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff9c0100c60 0x7ff9c010d920 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7ff9ac009900 tx=0x7ff9ac02bc30 comp rx=0 tx=0).stop 2024-03-21T10:27:23.397 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:23.397+0000 7ff9c4e00640 1 -- 172.21.15.47:0/2035513673 shutdown_connections 2024-03-21T10:27:23.398 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:23.397+0000 7ff9c4e00640 1 --2- 172.21.15.47:0/2035513673 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff9c010de60 0x7ff9c0110250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:27:23.398 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:23.397+0000 7ff9c4e00640 1 --2- 172.21.15.47:0/2035513673 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff9c0100c60 0x7ff9c010d920 unknown :-1 s=CLOSED pgs=690 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:27:23.398 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:23.397+0000 7ff9c4e00640 1 --2- 172.21.15.47:0/2035513673 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff9c01002b0 0x7ff9c0100690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:27:23.398 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:23.397+0000 7ff9c4e00640 1 -- 172.21.15.47:0/2035513673 >> 172.21.15.47:0/2035513673 conn(0x7ff9c00fbef0 msgr2=0x7ff9c00fe310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:27:23.398 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:23.397+0000 7ff9c4e00640 1 -- 172.21.15.47:0/2035513673 shutdown_connections 2024-03-21T10:27:23.398 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:23.397+0000 7ff9c4e00640 1 -- 172.21.15.47:0/2035513673 wait complete. 2024-03-21T10:27:23.398 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:23.398+0000 7ff9c4e00640 1 Processor -- start 2024-03-21T10:27:23.399 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:23.399+0000 7ff9c4e00640 1 -- start start 2024-03-21T10:27:23.399 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:23.399+0000 7ff9c4e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff9c01002b0 0x7ff9c0199f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:27:23.400 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:23.399+0000 7ff9c4e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff9c0100c60 0x7ff9c019a460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:27:23.400 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:23.399+0000 7ff9c4e00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff9c010de60 0x7ff9c019e790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:27:23.400 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:23.399+0000 7ff9c4e00640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9c01139c0 con 0x7ff9c01002b0 2024-03-21T10:27:23.400 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:23.399+0000 7ff9c4e00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9c0113840 con 0x7ff9c0100c60 2024-03-21T10:27:23.400 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:23.399+0000 7ff9c4e00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9c0113b40 con 0x7ff9c010de60 2024-03-21T10:27:23.400 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:23.400+0000 7ff9bf400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff9c01002b0 0x7ff9c0199f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:27:23.400 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:23.400+0000 7ff9bf400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff9c01002b0 0x7ff9c0199f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:34762/0 (socket says 172.21.15.47:34762) 2024-03-21T10:27:23.400 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:23.400+0000 7ff9bf400640 1 -- 172.21.15.47:0/2546762233 learned_addr learned my addr 172.21.15.47:0/2546762233 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:27:23.400 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:23.400+0000 7ff9bea00640 1 --2- 172.21.15.47:0/2546762233 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff9c0100c60 0x7ff9c019a460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:27:23.402 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:23.400+0000 7ff9bfe00640 1 --2- 172.21.15.47:0/2546762233 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff9c010de60 0x7ff9c019e790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:27:23.402 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:23.400+0000 7ff9bf400640 1 -- 172.21.15.47:0/2546762233 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff9c010de60 msgr2=0x7ff9c019e790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:27:23.402 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:23.400+0000 7ff9bf400640 1 --2- 172.21.15.47:0/2546762233 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff9c010de60 0x7ff9c019e790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:27:23.402 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:23.400+0000 7ff9bf400640 1 -- 172.21.15.47:0/2546762233 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff9c0100c60 msgr2=0x7ff9c019a460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:27:23.402 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:23.400+0000 7ff9bf400640 1 --2- 172.21.15.47:0/2546762233 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff9c0100c60 0x7ff9c019a460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:27:23.402 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:23.400+0000 7ff9bf400640 1 -- 172.21.15.47:0/2546762233 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9c019eec0 con 0x7ff9c01002b0 2024-03-21T10:27:23.402 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:23.400+0000 7ff9bea00640 1 --2- 172.21.15.47:0/2546762233 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff9c0100c60 0x7ff9c019a460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T10:27:23.402 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:23.401+0000 7ff9bf400640 1 --2- 172.21.15.47:0/2546762233 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff9c01002b0 0x7ff9c0199f20 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7ff9b000ea60 tx=0x7ff9b000ef20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:27:23.402 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:23.401+0000 7ff9b7e00640 1 -- 172.21.15.47:0/2546762233 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9b000cde0 con 0x7ff9c01002b0 2024-03-21T10:27:23.402 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:23.401+0000 7ff9b7e00640 1 -- 172.21.15.47:0/2546762233 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7ff9b00035f0 con 0x7ff9c01002b0 2024-03-21T10:27:23.402 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:23.401+0000 7ff9b7e00640 1 -- 172.21.15.47:0/2546762233 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9b0010690 con 0x7ff9c01002b0 2024-03-21T10:27:23.402 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:23.401+0000 7ff9c4e00640 1 -- 172.21.15.47:0/2546762233 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff9c019f1b0 con 0x7ff9c01002b0 2024-03-21T10:27:23.402 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:23.401+0000 7ff9c4e00640 1 -- 172.21.15.47:0/2546762233 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff9c01a6a40 con 0x7ff9c01002b0 2024-03-21T10:27:23.406 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:23.406+0000 7ff9c9010740 1 Processor -- start 2024-03-21T10:27:23.406 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:23.406+0000 7ff9c9010740 1 -- start start 2024-03-21T10:27:23.406 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:23.406+0000 7ff9b7e00640 1 -- 172.21.15.47:0/2546762233 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 27) v1 ==== 137985+0+0 (secure 0 0 0) 0x7ff9b00037a0 con 0x7ff9c01002b0 2024-03-21T10:27:23.406 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:23.406+0000 7ff9c9010740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56194d3198b0 0x56194d319c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:27:23.407 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:23.406+0000 7ff9c9010740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56194d31a260 0x56194d322810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:27:23.407 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:23.407+0000 7ff9c9010740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56194d322de0 0x56194d325230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:27:23.407 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:23.407+0000 7ff9c9010740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x56194d27d2c0 con 0x56194d322de0 2024-03-21T10:27:23.407 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:23.407+0000 7ff9c9010740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x56194d225c90 con 0x56194d3198b0 2024-03-21T10:27:23.407 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:23.407+0000 7ff9c9010740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x56194d279360 con 0x56194d31a260 2024-03-21T10:27:23.408 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:23.407+0000 7ff9bfe00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56194d3198b0 0x56194d319c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:27:23.408 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:23.407+0000 7ff9bfe00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56194d3198b0 0x56194d319c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:45174/0 (socket says 172.21.15.47:45174) 2024-03-21T10:27:23.408 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:23.407+0000 7ff9bfe00640 1 -- 172.21.15.47:0/3829897529 learned_addr learned my addr 172.21.15.47:0/3829897529 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:27:23.408 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:23.407+0000 7ff9bea00640 1 --2- 172.21.15.47:0/3829897529 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56194d322de0 0x56194d325230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:27:23.408 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:23.407+0000 7ff9bf400640 1 --2- 172.21.15.47:0/3829897529 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56194d31a260 0x56194d322810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:27:23.408 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:23.408+0000 7ff9bea00640 1 -- 172.21.15.47:0/3829897529 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56194d31a260 msgr2=0x56194d322810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:27:23.408 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:23.408+0000 7ff9bea00640 1 --2- 172.21.15.47:0/3829897529 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56194d31a260 0x56194d322810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:27:23.408 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:23.408+0000 7ff9bea00640 1 -- 172.21.15.47:0/3829897529 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56194d3198b0 msgr2=0x56194d319c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:27:23.408 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:23.408+0000 7ff9bea00640 1 --2- 172.21.15.47:0/3829897529 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56194d3198b0 0x56194d319c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:27:23.408 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:23.408+0000 7ff9bea00640 1 -- 172.21.15.47:0/3829897529 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56194d24a080 con 0x56194d322de0 2024-03-21T10:27:23.408 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:23.408+0000 7ff9bea00640 1 --2- 172.21.15.47:0/3829897529 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56194d322de0 0x56194d325230 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7ff9ac000c00 tx=0x7ff9ac02c5f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c6b4848ad0048bd8 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:27:23.410 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:23.408+0000 7ff9b4c00640 1 -- 172.21.15.47:0/3829897529 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9ac045070 con 0x56194d322de0 2024-03-21T10:27:23.410 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:23.409+0000 7ff9b4c00640 1 -- 172.21.15.47:0/3829897529 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7ff9ac02c7e0 con 0x56194d322de0 2024-03-21T10:27:23.410 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:23.409+0000 7ff9b4c00640 1 -- 172.21.15.47:0/3829897529 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9ac003c60 con 0x56194d322de0 2024-03-21T10:27:23.410 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:23.409+0000 7ff9b7e00640 1 --2- 172.21.15.47:0/2546762233 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7ff9a40a3090 0x7ff9a40a5550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:27:23.410 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:23.409+0000 7ff9b7e00640 1 -- 172.21.15.47:0/2546762233 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 9474+0+0 (secure 0 0 0) 0x7ff9b00cff30 con 0x7ff9c01002b0 2024-03-21T10:27:23.410 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:23.409+0000 7ff9bfe00640 1 --2- 172.21.15.47:0/2546762233 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7ff9a40a3090 0x7ff9a40a5550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:27:23.410 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:23.409+0000 7ff9c9010740 1 -- 172.21.15.47:0/3829897529 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56194d322de0 msgr2=0x56194d325230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:27:23.410 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:23.409+0000 7ff9c9010740 1 --2- 172.21.15.47:0/3829897529 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56194d322de0 0x56194d325230 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7ff9ac000c00 tx=0x7ff9ac02c5f0 comp rx=0 tx=0).stop 2024-03-21T10:27:23.410 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:23.410+0000 7ff9c9010740 1 -- 172.21.15.47:0/3829897529 shutdown_connections 2024-03-21T10:27:23.411 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:23.410+0000 7ff9c9010740 1 --2- 172.21.15.47:0/3829897529 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56194d322de0 0x56194d325230 unknown :-1 s=CLOSED pgs=692 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:27:23.411 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:23.410+0000 7ff9c9010740 1 --2- 172.21.15.47:0/3829897529 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56194d31a260 0x56194d322810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:27:23.411 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:23.410+0000 7ff9c9010740 1 --2- 172.21.15.47:0/3829897529 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56194d3198b0 0x56194d319c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:27:23.411 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:23.410+0000 7ff9c9010740 1 -- 172.21.15.47:0/3829897529 >> 172.21.15.47:0/3829897529 conn(0x56194d313f80 msgr2=0x56194d314360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:27:23.411 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:23.410+0000 7ff9c9010740 1 -- 172.21.15.47:0/3829897529 shutdown_connections 2024-03-21T10:27:23.411 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:23.410+0000 7ff9c9010740 1 -- 172.21.15.47:0/3829897529 wait complete. 2024-03-21T10:27:23.411 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:23.410+0000 7ff9bfe00640 1 --2- 172.21.15.47:0/2546762233 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7ff9a40a3090 0x7ff9a40a5550 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7ff9b8009db0 tx=0x7ff9b8009600 comp rx=0 tx=0).ready entity=mgr.15687 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:27:23.414 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:23.410+0000 7ff9c9010740 1 -- start start 2024-03-21T10:27:23.414 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:23.411+0000 7ff9c9010740 1 Processor -- start 2024-03-21T10:27:23.414 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:23.411+0000 7ff9c9010740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56194d3198b0 0x56194d3227f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:27:23.414 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:23.411+0000 7ff9c9010740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56194d31a260 0x56194d31fb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:27:23.415 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:23.412+0000 7ff9c9010740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56194d322de0 0x56194d320270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:27:23.415 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:23.412+0000 7ff9c9010740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x56194d325cf0 con 0x56194d31a260 2024-03-21T10:27:23.415 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:23.412+0000 7ff9c9010740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x56194d2390d0 con 0x56194d3198b0 2024-03-21T10:27:23.415 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:23.412+0000 7ff9c9010740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x56194d325e70 con 0x56194d322de0 2024-03-21T10:27:23.415 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:23.412+0000 7ff9c9010740 10 client.0 _ll_register_callbacks cb 0x56194d23de80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T10:27:23.415 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:23.412+0000 7ff9bf400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56194d3198b0 0x56194d3227f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:27:23.415 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:23.412+0000 7ff9bf400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56194d3198b0 0x56194d3227f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:45188/0 (socket says 172.21.15.47:45188) 2024-03-21T10:27:23.415 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:23.412+0000 7ff9bf400640 1 -- 172.21.15.47:0/230190174 learned_addr learned my addr 172.21.15.47:0/230190174 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:27:23.415 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:23.412+0000 7ff9bea00640 1 --2- 172.21.15.47:0/230190174 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56194d31a260 0x56194d31fb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:27:23.415 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:23.412+0000 7ff9bfe00640 1 --2- 172.21.15.47:0/230190174 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56194d322de0 0x56194d320270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:27:23.415 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:23.412+0000 7ff9bf400640 1 -- 172.21.15.47:0/230190174 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56194d322de0 msgr2=0x56194d320270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:27:23.416 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:23.412+0000 7ff9bf400640 1 --2- 172.21.15.47:0/230190174 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56194d322de0 0x56194d320270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:27:23.416 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:23.412+0000 7ff9bf400640 1 -- 172.21.15.47:0/230190174 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56194d31a260 msgr2=0x56194d31fb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:27:23.416 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:23.412+0000 7ff9bf400640 1 --2- 172.21.15.47:0/230190174 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56194d31a260 0x56194d31fb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:27:23.416 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:23.412+0000 7ff9bf400640 1 -- 172.21.15.47:0/230190174 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56194cf913f0 con 0x56194d3198b0 2024-03-21T10:27:23.416 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:23.412+0000 7ff9bfe00640 1 --2- 172.21.15.47:0/230190174 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56194d322de0 0x56194d320270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T10:27:23.416 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:23.413+0000 7ff9bf400640 1 --2- 172.21.15.47:0/230190174 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56194d3198b0 0x56194d3227f0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7ff9b000ea30 tx=0x7ff9b0098290 comp rx=0 tx=0).ready entity=mon.2 client_cookie=81e49ea409461830 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:27:23.416 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:23.413+0000 7ff9a9600640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-21T10:27:23.416 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:23.413+0000 7ff9a9600640 1 -- 172.21.15.47:0/230190174 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9b0003b30 con 0x56194d3198b0 2024-03-21T10:27:23.416 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:23.413+0000 7ff9a9600640 1 -- 172.21.15.47:0/230190174 <== mon.2 v2:172.21.15.100:3300/0 2 ==== osd_map(97..97 src has 1..97) v4 ==== 9474+0+0 (secure 0 0 0) 0x7ff9b00cd4e0 con 0x56194d3198b0 2024-03-21T10:27:23.416 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:23.413+0000 7ff9c9010740 1 -- 172.21.15.47:0/230190174 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ff9ac02c7e0 con 0x56194d3198b0 2024-03-21T10:27:23.416 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:23.413+0000 7ff9a9600640 1 -- 172.21.15.47:0/230190174 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff9b00cd6e0 con 0x56194d3198b0 2024-03-21T10:27:23.416 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:23.413+0000 7ff9c9010740 10 client.36236 fetch_fsmap learned FSMap version 32 2024-03-21T10:27:23.416 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:23.413+0000 7ff9c9010740 1 -- 172.21.15.47:0/230190174 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x56194d05b7d0 con 0x56194d3198b0 2024-03-21T10:27:23.417 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:23.414+0000 7ff9a9600640 1 -- 172.21.15.47:0/230190174 <== mon.2 v2:172.21.15.100:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7ff9ac02c7e0 con 0x56194d3198b0 2024-03-21T10:27:23.417 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:23.414+0000 7ff9a9600640 1 -- 172.21.15.47:0/230190174 <== mon.2 v2:172.21.15.100:3300/0 5 ==== fsmap(e 32) v1 ==== 7942+0+0 (secure 0 0 0) 0x7ff9b009b350 con 0x56194d3198b0 2024-03-21T10:27:23.417 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:23.414+0000 7ff9c9010740 10 client.36236 fetch_fsmap finished waiting for FSMap version 32 2024-03-21T10:27:23.417 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:23.414+0000 7ff9c9010740 10 client.36236 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.e 2024-03-21T10:27:23.417 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:23.414+0000 7ff9c9010740 20 client.36236 populate_metadata read hostname 'smithi047' 2024-03-21T10:27:23.417 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:23.414+0000 7ff9c9010740 1 --2- 172.21.15.47:0/230190174 >> [v2:172.21.15.100:6834/2597849957,v1:172.21.15.100:6835/2597849957] conn(0x56194d332c90 0x56194d335080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:27:23.417 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:23.414+0000 7ff9c9010740 4 client.36236 mds_command: new command op to 34454 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T10:27:23.417 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:23.414+0000 7ff9c9010740 1 -- 172.21.15.47:0/230190174 --> [v2:172.21.15.100:6834/2597849957,v1:172.21.15.100:6835/2597849957] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56194d28a800 con 0x56194d332c90 2024-03-21T10:27:23.418 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:23.415+0000 7ff9bea00640 1 --2- 172.21.15.47:0/230190174 >> [v2:172.21.15.100:6834/2597849957,v1:172.21.15.100:6835/2597849957] conn(0x56194d332c90 0x56194d335080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:27:23.418 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:23.415+0000 7ff9bea00640 1 --2- 172.21.15.47:0/230190174 >> [v2:172.21.15.100:6834/2597849957,v1:172.21.15.100:6835/2597849957] conn(0x56194d332c90 0x56194d335080 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=aec8e42e52d8816b server_cookie=df0ed83d12f841c in_seq=0 out_seq=0 2024-03-21T10:27:23.418 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:23.415+0000 7ff9a9600640 10 client.36236 ms_handle_connect on v2:172.21.15.100:6834/2597849957 2024-03-21T10:27:23.419 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:23.417+0000 7ff9a9600640 1 -- 172.21.15.47:0/230190174 <== mds.2 v2:172.21.15.100:6834/2597849957 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x56194d28a800 con 0x56194d332c90 2024-03-21T10:27:23.419 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:23.417+0000 7ff9a9600640 10 client.36236 handle_command_reply: tid=0 2024-03-21T10:27:23.419 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:23.417+0000 7ff9c9010740 1 client.36236 shutdown 2024-03-21T10:27:23.419 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:23.417+0000 7ff9c9010740 1 -- 172.21.15.47:0/230190174 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56194d3198b0 msgr2=0x56194d3227f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:27:23.419 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:23.417+0000 7ff9c9010740 1 --2- 172.21.15.47:0/230190174 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56194d3198b0 0x56194d3227f0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7ff9b000ea30 tx=0x7ff9b0098290 comp rx=0 tx=0).stop 2024-03-21T10:27:23.419 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:23.417+0000 7ff9c9010740 1 -- 172.21.15.47:0/230190174 shutdown_connections 2024-03-21T10:27:23.419 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:23.417+0000 7ff9c9010740 1 --2- 172.21.15.47:0/230190174 >> [v2:172.21.15.100:6834/2597849957,v1:172.21.15.100:6835/2597849957] conn(0x56194d332c90 0x56194d335080 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:27:23.419 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:23.417+0000 7ff9c9010740 1 --2- 172.21.15.47:0/230190174 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56194d322de0 0x56194d320270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:27:23.419 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:23.417+0000 7ff9c9010740 1 --2- 172.21.15.47:0/230190174 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56194d31a260 0x56194d31fb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:27:23.419 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:23.417+0000 7ff9c9010740 1 --2- 172.21.15.47:0/230190174 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56194d3198b0 0x56194d3227f0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:27:23.419 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:23.417+0000 7ff9a9600640 0 client.36236 ms_handle_reset on v2:172.21.15.100:6834/2597849957 2024-03-21T10:27:23.419 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:23.417+0000 7ff9c9010740 1 -- 172.21.15.47:0/230190174 >> 172.21.15.47:0/230190174 conn(0x56194d313f80 msgr2=0x56194d315cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:27:23.419 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:23.418+0000 7ff9c9010740 1 -- 172.21.15.47:0/230190174 shutdown_connections 2024-03-21T10:27:23.419 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:23.418+0000 7ff9c9010740 1 -- 172.21.15.47:0/230190174 wait complete. 2024-03-21T10:27:23.419 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:23.418+0000 7ff9c9010740 20 client.36236 trim_cache size 0 max 16384 2024-03-21T10:27:23.437 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:23.437+0000 7ff9c9010740 1 Processor -- start 2024-03-21T10:27:23.437 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:23.437+0000 7ff9c9010740 1 -- start start 2024-03-21T10:27:23.439 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:23.438+0000 7ff9c9010740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56194d3198b0 0x56194d315c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:27:23.439 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:23.438+0000 7ff9c9010740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56194d31a260 0x56194d312fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:27:23.439 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:23.438+0000 7ff9c9010740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56194d322de0 0x56194d3222b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:27:23.439 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:23.438+0000 7ff9c9010740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9c01b0870 con 0x56194d31a260 2024-03-21T10:27:23.439 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:23.438+0000 7ff9c9010740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9c01b21f0 con 0x56194d322de0 2024-03-21T10:27:23.440 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:23.438+0000 7ff9c9010740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9c01b0ed0 con 0x56194d3198b0 2024-03-21T10:27:23.440 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:23.438+0000 7ff9bea00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56194d31a260 0x56194d312fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:27:23.440 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:23.438+0000 7ff9bea00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56194d31a260 0x56194d312fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:34782/0 (socket says 172.21.15.47:34782) 2024-03-21T10:27:23.440 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:23.438+0000 7ff9bea00640 1 -- 172.21.15.47:0/27758915 learned_addr learned my addr 172.21.15.47:0/27758915 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:27:23.440 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:23.438+0000 7ff9bfe00640 1 --2- 172.21.15.47:0/27758915 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56194d322de0 0x56194d3222b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:27:23.440 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:23.438+0000 7ff9bf400640 1 --2- 172.21.15.47:0/27758915 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56194d3198b0 0x56194d315c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:27:23.440 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:23.439+0000 7ff9bea00640 1 -- 172.21.15.47:0/27758915 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56194d3198b0 msgr2=0x56194d315c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:27:23.440 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:23.439+0000 7ff9bea00640 1 --2- 172.21.15.47:0/27758915 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56194d3198b0 0x56194d315c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:27:23.440 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:23.439+0000 7ff9bea00640 1 -- 172.21.15.47:0/27758915 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56194d322de0 msgr2=0x56194d3222b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:27:23.440 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:23.439+0000 7ff9bea00640 1 --2- 172.21.15.47:0/27758915 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56194d322de0 0x56194d3222b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:27:23.440 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:23.439+0000 7ff9bea00640 1 -- 172.21.15.47:0/27758915 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56194d3205e0 con 0x56194d31a260 2024-03-21T10:27:23.440 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:23.439+0000 7ff9bfe00640 1 --2- 172.21.15.47:0/27758915 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56194d322de0 0x56194d3222b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T10:27:23.440 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:23.439+0000 7ff9bea00640 1 --2- 172.21.15.47:0/27758915 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56194d31a260 0x56194d312fe0 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7ff9ac000c00 tx=0x7ff9ac009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=64b9968615adc44c server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:27:23.441 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:23.439+0000 7ff9abe00640 1 -- 172.21.15.47:0/27758915 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9ac050070 con 0x56194d31a260 2024-03-21T10:27:23.441 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:23.439+0000 7ff9abe00640 1 -- 172.21.15.47:0/27758915 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7ff9ac02c7e0 con 0x56194d31a260 2024-03-21T10:27:23.441 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:23.439+0000 7ff9abe00640 1 -- 172.21.15.47:0/27758915 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9ac045040 con 0x56194d31a260 2024-03-21T10:27:23.444 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:23.440+0000 7ff9c9010740 1 -- 172.21.15.47:0/27758915 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56194d31a260 msgr2=0x56194d312fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:27:23.444 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:23.440+0000 7ff9c9010740 1 --2- 172.21.15.47:0/27758915 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56194d31a260 0x56194d312fe0 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7ff9ac000c00 tx=0x7ff9ac009260 comp rx=0 tx=0).stop 2024-03-21T10:27:23.444 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:23.440+0000 7ff9c9010740 1 -- 172.21.15.47:0/27758915 shutdown_connections 2024-03-21T10:27:23.444 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:23.440+0000 7ff9c9010740 1 --2- 172.21.15.47:0/27758915 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56194d322de0 0x56194d3222b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:27:23.444 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:23.440+0000 7ff9c9010740 1 --2- 172.21.15.47:0/27758915 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56194d31a260 0x56194d312fe0 unknown :-1 s=CLOSED pgs=693 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:27:23.444 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:23.440+0000 7ff9c9010740 1 --2- 172.21.15.47:0/27758915 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56194d3198b0 0x56194d315c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:27:23.444 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:23.440+0000 7ff9c9010740 1 -- 172.21.15.47:0/27758915 >> 172.21.15.47:0/27758915 conn(0x56194d313f80 msgr2=0x56194d3128d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:27:23.444 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:23.440+0000 7ff9c9010740 1 -- 172.21.15.47:0/27758915 shutdown_connections 2024-03-21T10:27:23.444 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:23.441+0000 7ff9c9010740 1 -- 172.21.15.47:0/27758915 wait complete. 2024-03-21T10:27:23.444 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:23.441+0000 7ff9c9010740 1 -- start start 2024-03-21T10:27:23.445 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:23.441+0000 7ff9c9010740 1 Processor -- start 2024-03-21T10:27:23.445 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:23.441+0000 7ff9c9010740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56194d3198b0 0x56194d328930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:27:23.445 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:23.442+0000 7ff9c9010740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56194d31a260 0x56194d328e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:27:23.445 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:23.442+0000 7ff9c9010740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56194d322de0 0x56194d38d520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:27:23.445 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:23.442+0000 7ff9c9010740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9c01b1b90 con 0x56194d3198b0 2024-03-21T10:27:23.445 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:23.442+0000 7ff9c9010740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9c01b1530 con 0x56194d31a260 2024-03-21T10:27:23.445 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:23.442+0000 7ff9c9010740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9c01aea30 con 0x56194d322de0 2024-03-21T10:27:23.445 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:23.442+0000 7ff9c9010740 10 client.0 _ll_register_callbacks cb 0x7ff9b0009490 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T10:27:23.445 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:23.442+0000 7ff9bf400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56194d3198b0 0x56194d328930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:27:23.445 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:23.442+0000 7ff9bf400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56194d3198b0 0x56194d328930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:34798/0 (socket says 172.21.15.47:34798) 2024-03-21T10:27:23.445 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:23.442+0000 7ff9bf400640 1 -- 172.21.15.47:0/574414217 learned_addr learned my addr 172.21.15.47:0/574414217 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:27:23.445 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:23.442+0000 7ff9bea00640 1 --2- 172.21.15.47:0/574414217 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56194d31a260 0x56194d328e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:27:23.446 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:23.442+0000 7ff9bf400640 1 -- 172.21.15.47:0/574414217 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56194d322de0 msgr2=0x56194d38d520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:27:23.446 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:23.442+0000 7ff9bf400640 1 --2- 172.21.15.47:0/574414217 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56194d322de0 0x56194d38d520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:27:23.446 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:23.442+0000 7ff9bf400640 1 -- 172.21.15.47:0/574414217 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56194d31a260 msgr2=0x56194d328e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:27:23.446 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:23.442+0000 7ff9bf400640 1 --2- 172.21.15.47:0/574414217 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56194d31a260 0x56194d328e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:27:23.446 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:23.442+0000 7ff9bf400640 1 -- 172.21.15.47:0/574414217 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56194d38da60 con 0x56194d3198b0 2024-03-21T10:27:23.446 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:23.443+0000 7ff9bf400640 1 --2- 172.21.15.47:0/574414217 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56194d3198b0 0x56194d328930 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7ff9b00988d0 tx=0x7ff9b009b580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dd7553238d78e80f server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:27:23.446 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:23.443+0000 7ff9a9600640 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-03-21T10:27:23.446 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:23.443+0000 7ff9a9600640 1 -- 172.21.15.47:0/574414217 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9b00cd6e0 con 0x56194d3198b0 2024-03-21T10:27:23.446 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:23.443+0000 7ff9a9600640 1 -- 172.21.15.47:0/574414217 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(97..97 src has 1..97) v4 ==== 9474+0+0 (secure 0 0 0) 0x7ff9b00d3020 con 0x56194d3198b0 2024-03-21T10:27:23.446 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:23.443+0000 7ff9c9010740 1 -- 172.21.15.47:0/574414217 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ff9ac02c7e0 con 0x56194d3198b0 2024-03-21T10:27:23.447 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:23.443+0000 7ff9a9600640 1 -- 172.21.15.47:0/574414217 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff9b009c600 con 0x56194d3198b0 2024-03-21T10:27:23.447 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:23.443+0000 7ff9c9010740 10 client.16452 fetch_fsmap learned FSMap version 32 2024-03-21T10:27:23.447 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:23.443+0000 7ff9c9010740 1 -- 172.21.15.47:0/574414217 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x56194d38ec20 con 0x56194d3198b0 2024-03-21T10:27:23.447 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:23.443+0000 7ff9a9600640 1 -- 172.21.15.47:0/574414217 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7ff9ac02c7e0 con 0x56194d3198b0 2024-03-21T10:27:23.447 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:23.444+0000 7ff9a9600640 1 -- 172.21.15.47:0/574414217 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 32) v1 ==== 7942+0+0 (secure 0 0 0) 0x7ff9b0097c50 con 0x56194d3198b0 2024-03-21T10:27:23.447 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:23.444+0000 7ff9c9010740 10 client.16452 fetch_fsmap finished waiting for FSMap version 32 2024-03-21T10:27:23.447 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:23.444+0000 7ff9c9010740 10 client.16452 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.e 2024-03-21T10:27:23.447 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:23.444+0000 7ff9c9010740 20 client.16452 populate_metadata read hostname 'smithi047' 2024-03-21T10:27:23.447 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:23.444+0000 7ff9c9010740 1 --2- 172.21.15.47:0/574414217 >> [v2:172.21.15.100:6834/2597849957,v1:172.21.15.100:6835/2597849957] conn(0x56194d38f2e0 0x56194d391720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:27:23.447 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:23.444+0000 7ff9c9010740 4 client.16452 mds_command: new command op to 34454 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-21T10:27:23.447 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:23.444+0000 7ff9c9010740 1 -- 172.21.15.47:0/574414217 --> [v2:172.21.15.100:6834/2597849957,v1:172.21.15.100:6835/2597849957] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x56194d284b50 con 0x56194d38f2e0 2024-03-21T10:27:23.447 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:23.444+0000 7ff9bea00640 1 --2- 172.21.15.47:0/574414217 >> [v2:172.21.15.100:6834/2597849957,v1:172.21.15.100:6835/2597849957] conn(0x56194d38f2e0 0x56194d391720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:27:23.448 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:23.445+0000 7ff9bea00640 1 --2- 172.21.15.47:0/574414217 >> [v2:172.21.15.100:6834/2597849957,v1:172.21.15.100:6835/2597849957] conn(0x56194d38f2e0 0x56194d391720 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=466ed56c13414410 server_cookie=350e8eb9493ee7d8 in_seq=0 out_seq=0 2024-03-21T10:27:23.449 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:23.445+0000 7ff9a9600640 10 client.16452 ms_handle_connect on v2:172.21.15.100:6834/2597849957 2024-03-21T10:27:23.449 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:23.445+0000 7ff9a9600640 1 -- 172.21.15.47:0/574414217 <== mds.2 v2:172.21.15.100:6834/2597849957 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x56194d284b50 con 0x56194d38f2e0 2024-03-21T10:27:23.449 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:23.445+0000 7ff9a9600640 10 client.16452 handle_command_reply: tid=0 2024-03-21T10:27:23.449 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:23.445+0000 7ff9c9010740 1 client.16452 shutdown 2024-03-21T10:27:23.449 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:23.446+0000 7ff9c9010740 1 -- 172.21.15.47:0/574414217 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56194d3198b0 msgr2=0x56194d328930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:27:23.449 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:23.446+0000 7ff9c9010740 1 --2- 172.21.15.47:0/574414217 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56194d3198b0 0x56194d328930 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7ff9b00988d0 tx=0x7ff9b009b580 comp rx=0 tx=0).stop 2024-03-21T10:27:23.449 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:23.446+0000 7ff9c9010740 1 -- 172.21.15.47:0/574414217 shutdown_connections 2024-03-21T10:27:23.449 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:23.446+0000 7ff9c9010740 1 --2- 172.21.15.47:0/574414217 >> [v2:172.21.15.100:6834/2597849957,v1:172.21.15.100:6835/2597849957] conn(0x56194d38f2e0 0x56194d391720 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:27:23.449 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:23.446+0000 7ff9c9010740 1 --2- 172.21.15.47:0/574414217 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56194d322de0 0x56194d38d520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:27:23.449 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:23.446+0000 7ff9c9010740 1 --2- 172.21.15.47:0/574414217 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56194d31a260 0x56194d328e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:27:23.449 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:23.446+0000 7ff9c9010740 1 --2- 172.21.15.47:0/574414217 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56194d3198b0 0x56194d328930 unknown :-1 s=CLOSED pgs=694 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:27:23.449 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:23.446+0000 7ff9c9010740 1 -- 172.21.15.47:0/574414217 >> 172.21.15.47:0/574414217 conn(0x56194d313f80 msgr2=0x56194d31a640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:27:23.449 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:23.446+0000 7ff9a9600640 0 client.16452 ms_handle_reset on v2:172.21.15.100:6834/2597849957 2024-03-21T10:27:23.450 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:23.446+0000 7ff9c9010740 1 -- 172.21.15.47:0/574414217 shutdown_connections 2024-03-21T10:27:23.450 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:23.446+0000 7ff9c9010740 1 -- 172.21.15.47:0/574414217 wait complete. 2024-03-21T10:27:23.450 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:23.446+0000 7ff9c9010740 20 client.16452 trim_cache size 0 max 16384 2024-03-21T10:27:23.450 INFO:teuthology.orchestra.run.smithi047.stdout:[] 2024-03-21T10:27:23.451 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:23.447+0000 7ff9c4e00640 1 -- 172.21.15.47:0/2546762233 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7ff9a40a3090 msgr2=0x7ff9a40a5550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:27:23.451 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:23.447+0000 7ff9c4e00640 1 --2- 172.21.15.47:0/2546762233 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7ff9a40a3090 0x7ff9a40a5550 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7ff9b8009db0 tx=0x7ff9b8009600 comp rx=0 tx=0).stop 2024-03-21T10:27:23.451 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:23.447+0000 7ff9c4e00640 1 -- 172.21.15.47:0/2546762233 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff9c01002b0 msgr2=0x7ff9c0199f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:27:23.451 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:23.447+0000 7ff9c4e00640 1 --2- 172.21.15.47:0/2546762233 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff9c01002b0 0x7ff9c0199f20 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7ff9b000ea60 tx=0x7ff9b000ef20 comp rx=0 tx=0).stop 2024-03-21T10:27:23.451 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:23.447+0000 7ff9c4e00640 1 -- 172.21.15.47:0/2546762233 shutdown_connections 2024-03-21T10:27:23.451 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:23.447+0000 7ff9c4e00640 1 --2- 172.21.15.47:0/2546762233 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff9c010de60 0x7ff9c019e790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:27:23.451 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:23.447+0000 7ff9c4e00640 1 --2- 172.21.15.47:0/2546762233 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7ff9a40a3090 0x7ff9a40a5550 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:27:23.451 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:23.447+0000 7ff9c4e00640 1 --2- 172.21.15.47:0/2546762233 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff9c0100c60 0x7ff9c019a460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:27:23.451 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:23.447+0000 7ff9c4e00640 1 --2- 172.21.15.47:0/2546762233 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff9c01002b0 0x7ff9c0199f20 unknown :-1 s=CLOSED pgs=691 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:27:23.451 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:23.447+0000 7ff9c4e00640 1 -- 172.21.15.47:0/2546762233 >> 172.21.15.47:0/2546762233 conn(0x7ff9c00fbef0 msgr2=0x7ff9c00fe2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:27:23.451 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:23.447+0000 7ff9c4e00640 1 -- 172.21.15.47:0/2546762233 shutdown_connections 2024-03-21T10:27:23.451 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:23.447+0000 7ff9c4e00640 1 -- 172.21.15.47:0/2546762233 wait complete. 2024-03-21T10:27:23.462 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:27:23 smithi047 ceph-mon[31547]: pgmap v65: 129 pgs: 129 active+clean; 142 MiB data, 667 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 7.0 MiB/s wr, 17 op/s 2024-03-21T10:27:23.462 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:27:23 smithi047 ceph-mon[31547]: from='client.? 172.21.15.47:0/1024567717' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-21T10:27:23.469 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-21T10:27:23.469 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-03-21T10:27:23.692 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:23.691+0000 7fdac3e00640 1 -- 172.21.15.47:0/2598534767 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdac4101ee0 msgr2=0x7fdac410e3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:27:23.692 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:23.691+0000 7fdac3e00640 1 --2- 172.21.15.47:0/2598534767 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdac4101ee0 0x7fdac410e3a0 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7fdabc009ca0 tx=0x7fdabc02c0f0 comp rx=0 tx=0).stop 2024-03-21T10:27:23.692 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:23.691+0000 7fdac3e00640 1 -- 172.21.15.47:0/2598534767 shutdown_connections 2024-03-21T10:27:23.692 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:23.691+0000 7fdac3e00640 1 --2- 172.21.15.47:0/2598534767 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdac4101ee0 0x7fdac410e3a0 unknown :-1 s=CLOSED pgs=695 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:27:23.692 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:23.691+0000 7fdac3e00640 1 --2- 172.21.15.47:0/2598534767 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdac4101520 0x7fdac41019a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:27:23.692 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:23.691+0000 7fdac3e00640 1 --2- 172.21.15.47:0/2598534767 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdac41075e0 0x7fdac41079c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:27:23.692 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:23.691+0000 7fdac3e00640 1 -- 172.21.15.47:0/2598534767 >> 172.21.15.47:0/2598534767 conn(0x7fdac40fd010 msgr2=0x7fdac40ff430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:27:23.692 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:23.692+0000 7fdac3e00640 1 -- 172.21.15.47:0/2598534767 shutdown_connections 2024-03-21T10:27:23.692 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:23.692+0000 7fdac3e00640 1 -- 172.21.15.47:0/2598534767 wait complete. 2024-03-21T10:27:23.693 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:23.693+0000 7fdac3e00640 1 Processor -- start 2024-03-21T10:27:23.693 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:23.693+0000 7fdac3e00640 1 -- start start 2024-03-21T10:27:23.694 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:23.694+0000 7fdac3e00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdac4101520 0x7fdac4199e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:27:23.694 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:23.694+0000 7fdac3e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdac4101ee0 0x7fdac419a380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:27:23.695 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:23.694+0000 7fdac3e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdac41075e0 0x7fdac419e700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:27:23.695 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:23.694+0000 7fdac3e00640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdac4111750 con 0x7fdac4101ee0 2024-03-21T10:27:23.695 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:23.694+0000 7fdac3e00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdac41115d0 con 0x7fdac41075e0 2024-03-21T10:27:23.695 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:23.694+0000 7fdac3e00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdac41118d0 con 0x7fdac4101520 2024-03-21T10:27:23.695 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:23.694+0000 7fdac2000640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdac4101ee0 0x7fdac419a380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:27:23.695 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:23.694+0000 7fdac2000640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdac4101ee0 0x7fdac419a380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:34818/0 (socket says 172.21.15.47:34818) 2024-03-21T10:27:23.695 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:23.694+0000 7fdac2000640 1 -- 172.21.15.47:0/3264069423 learned_addr learned my addr 172.21.15.47:0/3264069423 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:27:23.695 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:23.694+0000 7fdac2a00640 1 --2- 172.21.15.47:0/3264069423 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdac4101520 0x7fdac4199e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:27:23.695 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:23.695+0000 7fdac3400640 1 --2- 172.21.15.47:0/3264069423 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdac41075e0 0x7fdac419e700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:27:23.695 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:23.695+0000 7fdac2000640 1 -- 172.21.15.47:0/3264069423 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdac4101520 msgr2=0x7fdac4199e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:27:23.695 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:23.695+0000 7fdac2000640 1 --2- 172.21.15.47:0/3264069423 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdac4101520 0x7fdac4199e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:27:23.695 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:23.695+0000 7fdac2000640 1 -- 172.21.15.47:0/3264069423 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdac41075e0 msgr2=0x7fdac419e700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:27:23.695 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:23.695+0000 7fdac2000640 1 --2- 172.21.15.47:0/3264069423 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdac41075e0 0x7fdac419e700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:27:23.695 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:23.695+0000 7fdac2000640 1 -- 172.21.15.47:0/3264069423 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdac419ee30 con 0x7fdac4101ee0 2024-03-21T10:27:23.696 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:23.695+0000 7fdac2000640 1 --2- 172.21.15.47:0/3264069423 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdac4101ee0 0x7fdac419a380 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7fdab000e9f0 tx=0x7fdab000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:27:23.696 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:23.695+0000 7fdabb400640 1 -- 172.21.15.47:0/3264069423 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdab000cd80 con 0x7fdac4101ee0 2024-03-21T10:27:23.697 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:23.696+0000 7fdabb400640 1 -- 172.21.15.47:0/3264069423 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fdab00108e0 con 0x7fdac4101ee0 2024-03-21T10:27:23.697 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:23.696+0000 7fdabb400640 1 -- 172.21.15.47:0/3264069423 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdab0018920 con 0x7fdac4101ee0 2024-03-21T10:27:23.697 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:23.696+0000 7fdac3e00640 1 -- 172.21.15.47:0/3264069423 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdac419f120 con 0x7fdac4101ee0 2024-03-21T10:27:23.697 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:23.696+0000 7fdac3e00640 1 -- 172.21.15.47:0/3264069423 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdac41a69b0 con 0x7fdac4101ee0 2024-03-21T10:27:23.699 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:23.699+0000 7fdacc379740 1 Processor -- start 2024-03-21T10:27:23.701 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:23.700+0000 7fdacc379740 1 -- start start 2024-03-21T10:27:23.701 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:23.700+0000 7fdacc379740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b6f27da8b0 0x55b6f27dac90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:27:23.701 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:23.700+0000 7fdacc379740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b6f27db260 0x55b6f27e3810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:27:23.701 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:23.700+0000 7fdacc379740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b6f27e3de0 0x55b6f27e6230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:27:23.701 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:23.700+0000 7fdacc379740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b6f273e2c0 con 0x55b6f27db260 2024-03-21T10:27:23.701 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:23.700+0000 7fdacc379740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b6f26e6c90 con 0x55b6f27da8b0 2024-03-21T10:27:23.701 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:23.700+0000 7fdacc379740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b6f273a360 con 0x55b6f27e3de0 2024-03-21T10:27:23.701 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:23.700+0000 7fdabb400640 1 -- 172.21.15.47:0/3264069423 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 27) v1 ==== 137985+0+0 (secure 0 0 0) 0x7fdab0010a90 con 0x7fdac4101ee0 2024-03-21T10:27:23.701 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:23.700+0000 7fdac2a00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b6f27db260 0x55b6f27e3810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:27:23.702 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:23.701+0000 7fdac2a00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b6f27db260 0x55b6f27e3810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:34830/0 (socket says 172.21.15.47:34830) 2024-03-21T10:27:23.702 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:23.701+0000 7fdac2a00640 1 -- 172.21.15.47:0/786962370 learned_addr learned my addr 172.21.15.47:0/786962370 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:27:23.702 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:23.701+0000 7fdac3400640 1 --2- 172.21.15.47:0/786962370 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b6f27da8b0 0x55b6f27dac90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:27:23.702 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:23.701+0000 7fdac2000640 1 --2- 172.21.15.47:0/786962370 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b6f27e3de0 0x55b6f27e6230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:27:23.702 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:23.701+0000 7fdac2a00640 1 -- 172.21.15.47:0/786962370 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b6f27e3de0 msgr2=0x55b6f27e6230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:27:23.702 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:23.701+0000 7fdac2a00640 1 --2- 172.21.15.47:0/786962370 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b6f27e3de0 0x55b6f27e6230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:27:23.702 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:23.701+0000 7fdac2a00640 1 -- 172.21.15.47:0/786962370 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b6f27da8b0 msgr2=0x55b6f27dac90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:27:23.702 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:23.701+0000 7fdac2a00640 1 --2- 172.21.15.47:0/786962370 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b6f27da8b0 0x55b6f27dac90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:27:23.702 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:23.701+0000 7fdac2a00640 1 -- 172.21.15.47:0/786962370 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b6f270b080 con 0x55b6f27db260 2024-03-21T10:27:23.703 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:23.702+0000 7fdac2a00640 1 --2- 172.21.15.47:0/786962370 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b6f27db260 0x55b6f27e3810 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7fdab4009c80 tx=0x7fdab40117b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=417754f17cf76c0f server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:27:23.704 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:23.702+0000 7fdaaf400640 1 -- 172.21.15.47:0/786962370 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdab4012450 con 0x55b6f27db260 2024-03-21T10:27:23.704 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:23.702+0000 7fdaaf400640 1 -- 172.21.15.47:0/786962370 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fdab4012aa0 con 0x55b6f27db260 2024-03-21T10:27:23.704 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:23.702+0000 7fdaaf400640 1 -- 172.21.15.47:0/786962370 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdab400fb30 con 0x55b6f27db260 2024-03-21T10:27:23.704 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:23.702+0000 7fdabb400640 1 --2- 172.21.15.47:0/3264069423 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7fdaa80a3090 0x7fdaa80a5550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:27:23.704 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:23.702+0000 7fdabb400640 1 -- 172.21.15.47:0/3264069423 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 9474+0+0 (secure 0 0 0) 0x7fdab00d0060 con 0x7fdac4101ee0 2024-03-21T10:27:23.704 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:23.703+0000 7fdac3400640 1 --2- 172.21.15.47:0/3264069423 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7fdaa80a3090 0x7fdaa80a5550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:27:23.704 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:23.703+0000 7fdacc379740 1 -- 172.21.15.47:0/786962370 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b6f27db260 msgr2=0x55b6f27e3810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:27:23.704 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:23.703+0000 7fdacc379740 1 --2- 172.21.15.47:0/786962370 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b6f27db260 0x55b6f27e3810 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7fdab4009c80 tx=0x7fdab40117b0 comp rx=0 tx=0).stop 2024-03-21T10:27:23.704 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:23.703+0000 7fdacc379740 1 -- 172.21.15.47:0/786962370 shutdown_connections 2024-03-21T10:27:23.704 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:23.703+0000 7fdacc379740 1 --2- 172.21.15.47:0/786962370 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b6f27e3de0 0x55b6f27e6230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:27:23.704 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:23.703+0000 7fdacc379740 1 --2- 172.21.15.47:0/786962370 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b6f27db260 0x55b6f27e3810 unknown :-1 s=CLOSED pgs=697 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:27:23.704 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:23.703+0000 7fdacc379740 1 --2- 172.21.15.47:0/786962370 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b6f27da8b0 0x55b6f27dac90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:27:23.704 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:23.703+0000 7fdacc379740 1 -- 172.21.15.47:0/786962370 >> 172.21.15.47:0/786962370 conn(0x55b6f27d4f80 msgr2=0x55b6f27d5360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:27:23.704 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:23.704+0000 7fdac3400640 1 --2- 172.21.15.47:0/3264069423 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7fdaa80a3090 0x7fdaa80a5550 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fdabc006010 tx=0x7fdabc004a40 comp rx=0 tx=0).ready entity=mgr.15687 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:27:23.705 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:23.704+0000 7fdacc379740 1 -- 172.21.15.47:0/786962370 shutdown_connections 2024-03-21T10:27:23.705 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:23.704+0000 7fdacc379740 1 -- 172.21.15.47:0/786962370 wait complete. 2024-03-21T10:27:23.706 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:23.704+0000 7fdacc379740 1 -- start start 2024-03-21T10:27:23.706 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:23.704+0000 7fdacc379740 1 Processor -- start 2024-03-21T10:27:23.706 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:23.705+0000 7fdacc379740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b6f27da8b0 0x55b6f27e3720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:27:23.706 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:23.705+0000 7fdacc379740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b6f27db260 0x55b6f27e0ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:27:23.706 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:23.705+0000 7fdacc379740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b6f27e3de0 0x55b6f27e10c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:27:23.706 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:23.705+0000 7fdacc379740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b6f27e6d50 con 0x55b6f27db260 2024-03-21T10:27:23.706 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:23.705+0000 7fdacc379740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b6f26fa0d0 con 0x55b6f27e3de0 2024-03-21T10:27:23.707 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:23.705+0000 7fdacc379740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b6f27e6ed0 con 0x55b6f27da8b0 2024-03-21T10:27:23.707 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:23.705+0000 7fdacc379740 10 client.0 _ll_register_callbacks cb 0x55b6f26fee80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T10:27:23.707 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:23.705+0000 7fdac2000640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b6f27db260 0x55b6f27e0ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:27:23.707 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:23.705+0000 7fdac2000640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b6f27db260 0x55b6f27e0ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:34834/0 (socket says 172.21.15.47:34834) 2024-03-21T10:27:23.707 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:23.705+0000 7fdac2000640 1 -- 172.21.15.47:0/3447967576 learned_addr learned my addr 172.21.15.47:0/3447967576 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:27:23.707 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:23.705+0000 7fdac3400640 1 --2- 172.21.15.47:0/3447967576 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b6f27e3de0 0x55b6f27e10c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:27:23.707 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:23.706+0000 7fdac2a00640 1 --2- 172.21.15.47:0/3447967576 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b6f27da8b0 0x55b6f27e3720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:27:23.707 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:23.706+0000 7fdac2000640 1 -- 172.21.15.47:0/3447967576 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b6f27da8b0 msgr2=0x55b6f27e3720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:27:23.707 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:23.706+0000 7fdac2000640 1 --2- 172.21.15.47:0/3447967576 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b6f27da8b0 0x55b6f27e3720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:27:23.707 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:23.706+0000 7fdac2000640 1 -- 172.21.15.47:0/3447967576 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b6f27e3de0 msgr2=0x55b6f27e10c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:27:23.707 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:23.706+0000 7fdac2000640 1 --2- 172.21.15.47:0/3447967576 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b6f27e3de0 0x55b6f27e10c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:27:23.707 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:23.706+0000 7fdac2000640 1 -- 172.21.15.47:0/3447967576 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55b6f24523f0 con 0x55b6f27db260 2024-03-21T10:27:23.707 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:23.706+0000 7fdac2000640 1 --2- 172.21.15.47:0/3447967576 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b6f27db260 0x55b6f27e0ab0 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7fdab0098a00 tx=0x7fdab009bc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a7590db171c282a8 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:27:23.709 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:23.706+0000 7fdaa3e00640 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-03-21T10:27:23.709 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:23.707+0000 7fdaa3e00640 1 -- 172.21.15.47:0/3447967576 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdab009c030 con 0x55b6f27db260 2024-03-21T10:27:23.709 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:23.707+0000 7fdaa3e00640 1 -- 172.21.15.47:0/3447967576 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(97..97 src has 1..97) v4 ==== 9474+0+0 (secure 0 0 0) 0x7fdab00cd610 con 0x55b6f27db260 2024-03-21T10:27:23.709 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:23.707+0000 7fdacc379740 1 -- 172.21.15.47:0/3447967576 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fdab4012aa0 con 0x55b6f27db260 2024-03-21T10:27:23.709 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:23.707+0000 7fdaa3e00640 1 -- 172.21.15.47:0/3447967576 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7fdab00cd810 con 0x55b6f27db260 2024-03-21T10:27:23.709 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:23.707+0000 7fdacc379740 10 client.16464 fetch_fsmap learned FSMap version 32 2024-03-21T10:27:23.709 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:23.707+0000 7fdacc379740 1 -- 172.21.15.47:0/3447967576 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x55b6f251c7d0 con 0x55b6f27db260 2024-03-21T10:27:23.709 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:23.708+0000 7fdaa3e00640 1 -- 172.21.15.47:0/3447967576 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fdab4012aa0 con 0x55b6f27db260 2024-03-21T10:27:23.710 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:23.708+0000 7fdaa3e00640 1 -- 172.21.15.47:0/3447967576 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 32) v1 ==== 7942+0+0 (secure 0 0 0) 0x7fdab009b480 con 0x55b6f27db260 2024-03-21T10:27:23.710 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:23.708+0000 7fdacc379740 10 client.16464 fetch_fsmap finished waiting for FSMap version 32 2024-03-21T10:27:23.710 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:23.708+0000 7fdacc379740 10 client.16464 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-03-21T10:27:23.710 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:23.708+0000 7fdacc379740 20 client.16464 populate_metadata read hostname 'smithi047' 2024-03-21T10:27:23.710 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:23.708+0000 7fdacc379740 1 --2- 172.21.15.47:0/3447967576 >> [v2:172.21.15.100:6836/3497644162,v1:172.21.15.100:6837/3497644162] conn(0x55b6f27f3bd0 0x55b6f27f5fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:27:23.710 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:23.708+0000 7fdacc379740 4 client.16464 mds_command: new command op to 34493 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T10:27:23.710 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:23.708+0000 7fdacc379740 1 -- 172.21.15.47:0/3447967576 --> [v2:172.21.15.100:6836/3497644162,v1:172.21.15.100:6837/3497644162] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55b6f274b800 con 0x55b6f27f3bd0 2024-03-21T10:27:23.710 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:23.708+0000 7fdac2a00640 1 --2- 172.21.15.47:0/3447967576 >> [v2:172.21.15.100:6836/3497644162,v1:172.21.15.100:6837/3497644162] conn(0x55b6f27f3bd0 0x55b6f27f5fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:27:23.710 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:23.709+0000 7fdac2a00640 1 --2- 172.21.15.47:0/3447967576 >> [v2:172.21.15.100:6836/3497644162,v1:172.21.15.100:6837/3497644162] conn(0x55b6f27f3bd0 0x55b6f27f5fc0 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=e6c28d1a4274d65a server_cookie=b87536194cc600 in_seq=0 out_seq=0 2024-03-21T10:27:23.710 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:23.709+0000 7fdaa3e00640 10 client.16464 ms_handle_connect on v2:172.21.15.100:6836/3497644162 2024-03-21T10:27:23.711 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:23.711+0000 7fdaa3e00640 1 -- 172.21.15.47:0/3447967576 <== mds.1 v2:172.21.15.100:6836/3497644162 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55b6f274b800 con 0x55b6f27f3bd0 2024-03-21T10:27:23.712 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:23.711+0000 7fdaa3e00640 10 client.16464 handle_command_reply: tid=0 2024-03-21T10:27:23.712 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:23.711+0000 7fdacc379740 1 client.16464 shutdown 2024-03-21T10:27:23.712 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:23.711+0000 7fdacc379740 1 -- 172.21.15.47:0/3447967576 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b6f27db260 msgr2=0x55b6f27e0ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:27:23.712 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:23.711+0000 7fdacc379740 1 --2- 172.21.15.47:0/3447967576 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b6f27db260 0x55b6f27e0ab0 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7fdab0098a00 tx=0x7fdab009bc60 comp rx=0 tx=0).stop 2024-03-21T10:27:23.712 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:23.711+0000 7fdacc379740 1 -- 172.21.15.47:0/3447967576 shutdown_connections 2024-03-21T10:27:23.712 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:23.711+0000 7fdacc379740 1 --2- 172.21.15.47:0/3447967576 >> [v2:172.21.15.100:6836/3497644162,v1:172.21.15.100:6837/3497644162] conn(0x55b6f27f3bd0 0x55b6f27f5fc0 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:27:23.712 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:23.711+0000 7fdacc379740 1 --2- 172.21.15.47:0/3447967576 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b6f27e3de0 0x55b6f27e10c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:27:23.712 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:23.711+0000 7fdacc379740 1 --2- 172.21.15.47:0/3447967576 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b6f27db260 0x55b6f27e0ab0 unknown :-1 s=CLOSED pgs=698 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:27:23.712 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:23.711+0000 7fdacc379740 1 --2- 172.21.15.47:0/3447967576 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b6f27da8b0 0x55b6f27e3720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:27:23.712 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:23.711+0000 7fdacc379740 1 -- 172.21.15.47:0/3447967576 >> 172.21.15.47:0/3447967576 conn(0x55b6f27d4f80 msgr2=0x55b6f27d6cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:27:23.712 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:23.711+0000 7fdaa3e00640 0 client.16464 ms_handle_reset on v2:172.21.15.100:6836/3497644162 2024-03-21T10:27:23.712 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:23.711+0000 7fdacc379740 1 -- 172.21.15.47:0/3447967576 shutdown_connections 2024-03-21T10:27:23.712 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:23.712+0000 7fdacc379740 1 -- 172.21.15.47:0/3447967576 wait complete. 2024-03-21T10:27:23.713 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:23.712+0000 7fdacc379740 20 client.16464 trim_cache size 0 max 16384 2024-03-21T10:27:23.728 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:27:23 smithi100 ceph-mon[35193]: pgmap v65: 129 pgs: 129 active+clean; 142 MiB data, 667 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 7.0 MiB/s wr, 17 op/s 2024-03-21T10:27:23.728 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:27:23 smithi100 ceph-mon[35193]: from='client.? 172.21.15.47:0/1024567717' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-21T10:27:23.734 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:23.733+0000 7fdacc379740 1 Processor -- start 2024-03-21T10:27:23.734 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:23.733+0000 7fdacc379740 1 -- start start 2024-03-21T10:27:23.734 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:23.733+0000 7fdacc379740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b6f27da8b0 0x55b6f27e31e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:27:23.734 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:23.733+0000 7fdacc379740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b6f27db260 0x55b6f27e3720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:27:23.734 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:23.733+0000 7fdacc379740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b6f27e3de0 0x55b6f27ddd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:27:23.734 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:23.733+0000 7fdacc379740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdac41b08d0 con 0x55b6f27db260 2024-03-21T10:27:23.734 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:23.733+0000 7fdacc379740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdac41b2250 con 0x55b6f27da8b0 2024-03-21T10:27:23.735 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:23.733+0000 7fdacc379740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdac41b0f30 con 0x55b6f27e3de0 2024-03-21T10:27:23.735 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:23.734+0000 7fdac2000640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b6f27db260 0x55b6f27e3720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:27:23.735 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:23.734+0000 7fdac2a00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b6f27da8b0 0x55b6f27e31e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:27:23.735 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:23.734+0000 7fdac2000640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b6f27db260 0x55b6f27e3720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:34840/0 (socket says 172.21.15.47:34840) 2024-03-21T10:27:23.735 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:23.734+0000 7fdac2000640 1 -- 172.21.15.47:0/4063619876 learned_addr learned my addr 172.21.15.47:0/4063619876 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:27:23.735 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:23.734+0000 7fdac3400640 1 --2- 172.21.15.47:0/4063619876 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b6f27e3de0 0x55b6f27ddd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:27:23.735 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:23.734+0000 7fdac2000640 1 -- 172.21.15.47:0/4063619876 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b6f27e3de0 msgr2=0x55b6f27ddd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:27:23.735 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:23.734+0000 7fdac2000640 1 --2- 172.21.15.47:0/4063619876 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b6f27e3de0 0x55b6f27ddd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:27:23.735 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:23.734+0000 7fdac2000640 1 -- 172.21.15.47:0/4063619876 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b6f27da8b0 msgr2=0x55b6f27e31e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:27:23.735 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:23.734+0000 7fdac2000640 1 --2- 172.21.15.47:0/4063619876 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b6f27da8b0 0x55b6f27e31e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:27:23.735 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:23.734+0000 7fdac2000640 1 -- 172.21.15.47:0/4063619876 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b6f27d4ab0 con 0x55b6f27db260 2024-03-21T10:27:23.737 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:23.734+0000 7fdac2000640 1 --2- 172.21.15.47:0/4063619876 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b6f27db260 0x55b6f27e3720 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7fdab009a2e0 tx=0x7fdab000cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6424b5febbbf8608 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:27:23.737 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:23.735+0000 7fdaaea00640 1 -- 172.21.15.47:0/4063619876 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdab00a4a40 con 0x55b6f27db260 2024-03-21T10:27:23.737 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:23.735+0000 7fdaaea00640 1 -- 172.21.15.47:0/4063619876 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fdab00cd610 con 0x55b6f27db260 2024-03-21T10:27:23.737 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:23.735+0000 7fdaaea00640 1 -- 172.21.15.47:0/4063619876 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdab0097fe0 con 0x55b6f27db260 2024-03-21T10:27:23.737 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:23.736+0000 7fdacc379740 1 -- 172.21.15.47:0/4063619876 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b6f27db260 msgr2=0x55b6f27e3720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:27:23.737 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:23.736+0000 7fdacc379740 1 --2- 172.21.15.47:0/4063619876 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b6f27db260 0x55b6f27e3720 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7fdab009a2e0 tx=0x7fdab000cf90 comp rx=0 tx=0).stop 2024-03-21T10:27:23.737 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:23.736+0000 7fdacc379740 1 -- 172.21.15.47:0/4063619876 shutdown_connections 2024-03-21T10:27:23.737 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:23.736+0000 7fdacc379740 1 --2- 172.21.15.47:0/4063619876 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b6f27e3de0 0x55b6f27ddd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:27:23.737 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:23.736+0000 7fdacc379740 1 --2- 172.21.15.47:0/4063619876 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b6f27db260 0x55b6f27e3720 unknown :-1 s=CLOSED pgs=699 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:27:23.737 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:23.736+0000 7fdacc379740 1 --2- 172.21.15.47:0/4063619876 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b6f27da8b0 0x55b6f27e31e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:27:23.737 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:23.736+0000 7fdacc379740 1 -- 172.21.15.47:0/4063619876 >> 172.21.15.47:0/4063619876 conn(0x55b6f27d4f80 msgr2=0x55b6f27d4160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:27:23.737 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:23.736+0000 7fdacc379740 1 -- 172.21.15.47:0/4063619876 shutdown_connections 2024-03-21T10:27:23.738 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:23.736+0000 7fdacc379740 1 -- 172.21.15.47:0/4063619876 wait complete. 2024-03-21T10:27:23.738 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:23.736+0000 7fdacc379740 1 -- start start 2024-03-21T10:27:23.738 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:23.737+0000 7fdacc379740 1 Processor -- start 2024-03-21T10:27:23.740 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:23.737+0000 7fdacc379740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b6f27da8b0 0x55b6f27ebc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:27:23.740 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:23.737+0000 7fdacc379740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b6f27db260 0x55b6f27ea000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:27:23.740 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:23.737+0000 7fdacc379740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b6f27e3de0 0x55b6f27ea540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:27:23.740 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:23.737+0000 7fdacc379740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdac41b1bf0 con 0x55b6f27db260 2024-03-21T10:27:23.740 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:23.737+0000 7fdacc379740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdac41b1590 con 0x55b6f27da8b0 2024-03-21T10:27:23.740 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:23.737+0000 7fdacc379740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdac41aea90 con 0x55b6f27e3de0 2024-03-21T10:27:23.740 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:23.737+0000 7fdacc379740 10 client.0 _ll_register_callbacks cb 0x7fdab0009700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T10:27:23.740 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:23.738+0000 7fdac2000640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b6f27db260 0x55b6f27ea000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:27:23.741 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:23.738+0000 7fdac2a00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b6f27da8b0 0x55b6f27ebc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:27:23.741 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:23.738+0000 7fdac2000640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b6f27db260 0x55b6f27ea000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:34854/0 (socket says 172.21.15.47:34854) 2024-03-21T10:27:23.741 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:23.738+0000 7fdac2000640 1 -- 172.21.15.47:0/2993342499 learned_addr learned my addr 172.21.15.47:0/2993342499 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:27:23.741 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:23.738+0000 7fdac3400640 1 --2- 172.21.15.47:0/2993342499 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b6f27e3de0 0x55b6f27ea540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:27:23.741 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:23.738+0000 7fdac2000640 1 -- 172.21.15.47:0/2993342499 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b6f27e3de0 msgr2=0x55b6f27ea540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:27:23.741 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:23.738+0000 7fdac2000640 1 --2- 172.21.15.47:0/2993342499 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b6f27e3de0 0x55b6f27ea540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:27:23.741 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:23.738+0000 7fdac2000640 1 -- 172.21.15.47:0/2993342499 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b6f27da8b0 msgr2=0x55b6f27ebc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:27:23.741 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:23.738+0000 7fdac2000640 1 --2- 172.21.15.47:0/2993342499 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b6f27da8b0 0x55b6f27ebc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:27:23.741 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:23.738+0000 7fdac2000640 1 -- 172.21.15.47:0/2993342499 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55b6f27eaab0 con 0x55b6f27db260 2024-03-21T10:27:23.741 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:23.739+0000 7fdac2000640 1 --2- 172.21.15.47:0/2993342499 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b6f27db260 0x55b6f27ea000 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7fdab0098a00 tx=0x7fdab0096db0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f59e6f7492d8a341 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:27:23.742 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:23.739+0000 7fdaa3e00640 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-03-21T10:27:23.742 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:23.739+0000 7fdaa3e00640 1 -- 172.21.15.47:0/2993342499 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdab00a4a40 con 0x55b6f27db260 2024-03-21T10:27:23.742 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:23.739+0000 7fdaa3e00640 1 -- 172.21.15.47:0/2993342499 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(97..97 src has 1..97) v4 ==== 9474+0+0 (secure 0 0 0) 0x7fdab0010640 con 0x55b6f27db260 2024-03-21T10:27:23.742 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:23.739+0000 7fdacc379740 1 -- 172.21.15.47:0/2993342499 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fdab00cd610 con 0x55b6f27db260 2024-03-21T10:27:23.742 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:23.739+0000 7fdaa3e00640 1 -- 172.21.15.47:0/2993342499 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7fdab00cdb30 con 0x55b6f27db260 2024-03-21T10:27:23.742 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:23.739+0000 7fdacc379740 10 client.16470 fetch_fsmap learned FSMap version 32 2024-03-21T10:27:23.742 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:23.739+0000 7fdacc379740 1 -- 172.21.15.47:0/2993342499 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x55b6f2850170 con 0x55b6f27db260 2024-03-21T10:27:23.742 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:23.740+0000 7fdaa3e00640 1 -- 172.21.15.47:0/2993342499 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fdab00cd610 con 0x55b6f27db260 2024-03-21T10:27:23.742 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:23.740+0000 7fdaa3e00640 1 -- 172.21.15.47:0/2993342499 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 32) v1 ==== 7942+0+0 (secure 0 0 0) 0x7fdab0018b30 con 0x55b6f27db260 2024-03-21T10:27:23.742 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:23.740+0000 7fdacc379740 10 client.16470 fetch_fsmap finished waiting for FSMap version 32 2024-03-21T10:27:23.742 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:23.740+0000 7fdacc379740 10 client.16470 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-03-21T10:27:23.743 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:23.740+0000 7fdacc379740 20 client.16470 populate_metadata read hostname 'smithi047' 2024-03-21T10:27:23.743 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:23.740+0000 7fdacc379740 1 --2- 172.21.15.47:0/2993342499 >> [v2:172.21.15.100:6836/3497644162,v1:172.21.15.100:6837/3497644162] conn(0x55b6f2850440 0x55b6f2852860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:27:23.743 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:23.740+0000 7fdacc379740 4 client.16470 mds_command: new command op to 34493 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-21T10:27:23.743 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:23.740+0000 7fdacc379740 1 -- 172.21.15.47:0/2993342499 --> [v2:172.21.15.100:6836/3497644162,v1:172.21.15.100:6837/3497644162] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55b6f2745b50 con 0x55b6f2850440 2024-03-21T10:27:23.744 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:27:23 smithi157 ceph-mon[35581]: pgmap v65: 129 pgs: 129 active+clean; 142 MiB data, 667 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 7.0 MiB/s wr, 17 op/s 2024-03-21T10:27:23.744 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:27:23 smithi157 ceph-mon[35581]: from='client.? 172.21.15.47:0/1024567717' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-21T10:27:23.745 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:23.741+0000 7fdac2a00640 1 --2- 172.21.15.47:0/2993342499 >> [v2:172.21.15.100:6836/3497644162,v1:172.21.15.100:6837/3497644162] conn(0x55b6f2850440 0x55b6f2852860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:27:23.745 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:23.741+0000 7fdaa3e00640 10 client.16470 ms_handle_connect on v2:172.21.15.100:6836/3497644162 2024-03-21T10:27:23.745 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:23.741+0000 7fdac2a00640 1 --2- 172.21.15.47:0/2993342499 >> [v2:172.21.15.100:6836/3497644162,v1:172.21.15.100:6837/3497644162] conn(0x55b6f2850440 0x55b6f2852860 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=4802b541b38a39e5 server_cookie=859cd9c7e2ca5ec3 in_seq=0 out_seq=0 2024-03-21T10:27:23.745 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:23.742+0000 7fdaa3e00640 1 -- 172.21.15.47:0/2993342499 <== mds.1 v2:172.21.15.100:6836/3497644162 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55b6f2745b50 con 0x55b6f2850440 2024-03-21T10:27:23.745 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:23.742+0000 7fdaa3e00640 10 client.16470 handle_command_reply: tid=0 2024-03-21T10:27:23.745 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:23.742+0000 7fdacc379740 1 client.16470 shutdown 2024-03-21T10:27:23.745 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:23.742+0000 7fdacc379740 1 -- 172.21.15.47:0/2993342499 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b6f27db260 msgr2=0x55b6f27ea000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:27:23.745 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:23.742+0000 7fdacc379740 1 --2- 172.21.15.47:0/2993342499 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b6f27db260 0x55b6f27ea000 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7fdab0098a00 tx=0x7fdab0096db0 comp rx=0 tx=0).stop 2024-03-21T10:27:23.745 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:23.742+0000 7fdacc379740 1 -- 172.21.15.47:0/2993342499 shutdown_connections 2024-03-21T10:27:23.745 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:23.742+0000 7fdacc379740 1 --2- 172.21.15.47:0/2993342499 >> [v2:172.21.15.100:6836/3497644162,v1:172.21.15.100:6837/3497644162] conn(0x55b6f2850440 0x55b6f2852860 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:27:23.746 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:23.742+0000 7fdacc379740 1 --2- 172.21.15.47:0/2993342499 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b6f27e3de0 0x55b6f27ea540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:27:23.746 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:23.742+0000 7fdacc379740 1 --2- 172.21.15.47:0/2993342499 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b6f27db260 0x55b6f27ea000 unknown :-1 s=CLOSED pgs=700 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:27:23.746 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:23.742+0000 7fdacc379740 1 --2- 172.21.15.47:0/2993342499 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b6f27da8b0 0x55b6f27ebc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:27:23.746 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:23.742+0000 7fdaa3e00640 0 client.16470 ms_handle_reset on v2:172.21.15.100:6836/3497644162 2024-03-21T10:27:23.746 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:23.742+0000 7fdacc379740 1 -- 172.21.15.47:0/2993342499 >> 172.21.15.47:0/2993342499 conn(0x55b6f27d4f80 msgr2=0x55b6f27db690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:27:23.746 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:23.742+0000 7fdacc379740 1 -- 172.21.15.47:0/2993342499 shutdown_connections 2024-03-21T10:27:23.746 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:23.742+0000 7fdacc379740 1 -- 172.21.15.47:0/2993342499 wait complete. 2024-03-21T10:27:23.746 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:23.742+0000 7fdacc379740 20 client.16470 trim_cache size 0 max 16384 2024-03-21T10:27:23.746 INFO:teuthology.orchestra.run.smithi047.stdout:[] 2024-03-21T10:27:23.747 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:23.743+0000 7fdac3e00640 1 -- 172.21.15.47:0/3264069423 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7fdaa80a3090 msgr2=0x7fdaa80a5550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:27:23.747 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:23.743+0000 7fdac3e00640 1 --2- 172.21.15.47:0/3264069423 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7fdaa80a3090 0x7fdaa80a5550 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fdabc006010 tx=0x7fdabc004a40 comp rx=0 tx=0).stop 2024-03-21T10:27:23.747 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:23.743+0000 7fdac3e00640 1 -- 172.21.15.47:0/3264069423 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdac4101ee0 msgr2=0x7fdac419a380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:27:23.747 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:23.743+0000 7fdac3e00640 1 --2- 172.21.15.47:0/3264069423 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdac4101ee0 0x7fdac419a380 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7fdab000e9f0 tx=0x7fdab000eeb0 comp rx=0 tx=0).stop 2024-03-21T10:27:23.747 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:23.743+0000 7fdac3e00640 1 -- 172.21.15.47:0/3264069423 shutdown_connections 2024-03-21T10:27:23.748 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:23.743+0000 7fdac3e00640 1 --2- 172.21.15.47:0/3264069423 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7fdaa80a3090 0x7fdaa80a5550 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:27:23.748 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:23.743+0000 7fdac3e00640 1 --2- 172.21.15.47:0/3264069423 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdac41075e0 0x7fdac419e700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:27:23.748 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:23.743+0000 7fdac3e00640 1 --2- 172.21.15.47:0/3264069423 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdac4101ee0 0x7fdac419a380 unknown :-1 s=CLOSED pgs=696 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:27:23.748 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:23.743+0000 7fdac3e00640 1 --2- 172.21.15.47:0/3264069423 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdac4101520 0x7fdac4199e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:27:23.748 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:23.743+0000 7fdac3e00640 1 -- 172.21.15.47:0/3264069423 >> 172.21.15.47:0/3264069423 conn(0x7fdac40fd010 msgr2=0x7fdac40fe540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:27:23.748 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:23.743+0000 7fdac3e00640 1 -- 172.21.15.47:0/3264069423 shutdown_connections 2024-03-21T10:27:23.748 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:23.743+0000 7fdac3e00640 1 -- 172.21.15.47:0/3264069423 wait complete. 2024-03-21T10:27:23.761 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-21T10:27:23.761 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-03-21T10:27:23.985 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:23.984+0000 7f0f85600640 1 -- 172.21.15.47:0/1904519214 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0f80073a40 msgr2=0x7f0f80073e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:27:23.985 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:23.984+0000 7f0f85600640 1 --2- 172.21.15.47:0/1904519214 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0f80073a40 0x7f0f80073e20 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7f0f70009900 tx=0x7f0f7002bcd0 comp rx=0 tx=0).stop 2024-03-21T10:27:23.985 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:23.984+0000 7f0f85600640 1 -- 172.21.15.47:0/1904519214 shutdown_connections 2024-03-21T10:27:23.985 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:23.984+0000 7f0f85600640 1 --2- 172.21.15.47:0/1904519214 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0f80112520 0x7f0f80114910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:27:23.985 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:23.984+0000 7f0f85600640 1 --2- 172.21.15.47:0/1904519214 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0f80074360 0x7f0f80111fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:27:23.985 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:23.984+0000 7f0f85600640 1 --2- 172.21.15.47:0/1904519214 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0f80073a40 0x7f0f80073e20 unknown :-1 s=CLOSED pgs=701 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:27:23.985 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:23.984+0000 7f0f85600640 1 -- 172.21.15.47:0/1904519214 >> 172.21.15.47:0/1904519214 conn(0x7f0f800fd040 msgr2=0x7f0f800ff460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:27:23.985 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:23.985+0000 7f0f85600640 1 -- 172.21.15.47:0/1904519214 shutdown_connections 2024-03-21T10:27:23.985 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:23.985+0000 7f0f85600640 1 -- 172.21.15.47:0/1904519214 wait complete. 2024-03-21T10:27:23.986 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:23.986+0000 7f0f85600640 1 Processor -- start 2024-03-21T10:27:23.986 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:23.986+0000 7f0f85600640 1 -- start start 2024-03-21T10:27:23.987 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:23.987+0000 7f0f85600640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0f80073a40 0x7f0f8019e3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:27:23.987 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:23.987+0000 7f0f85600640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0f80074360 0x7f0f8019e900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:27:23.987 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:23.987+0000 7f0f85600640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0f80112520 0x7f0f801a2c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:27:23.987 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:23.987+0000 7f0f85600640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f80117e20 con 0x7f0f80073a40 2024-03-21T10:27:23.987 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:23.987+0000 7f0f85600640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f80117ca0 con 0x7f0f80074360 2024-03-21T10:27:23.987 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:23.987+0000 7f0f85600640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f80117fa0 con 0x7f0f80112520 2024-03-21T10:27:23.987 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:23.987+0000 7f0f7fe00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0f80073a40 0x7f0f8019e3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:27:23.987 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:23.987+0000 7f0f7fe00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0f80073a40 0x7f0f8019e3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:34880/0 (socket says 172.21.15.47:34880) 2024-03-21T10:27:23.988 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:23.987+0000 7f0f7fe00640 1 -- 172.21.15.47:0/3265578614 learned_addr learned my addr 172.21.15.47:0/3265578614 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:27:23.989 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:23.987+0000 7f0f7f400640 1 --2- 172.21.15.47:0/3265578614 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0f80074360 0x7f0f8019e900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:27:23.989 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:23.988+0000 7f0f84c00640 1 --2- 172.21.15.47:0/3265578614 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0f80112520 0x7f0f801a2c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:27:23.989 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:23.988+0000 7f0f7fe00640 1 -- 172.21.15.47:0/3265578614 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0f80112520 msgr2=0x7f0f801a2c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:27:23.989 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:23.988+0000 7f0f7fe00640 1 --2- 172.21.15.47:0/3265578614 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0f80112520 0x7f0f801a2c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:27:23.989 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:23.988+0000 7f0f7fe00640 1 -- 172.21.15.47:0/3265578614 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0f80074360 msgr2=0x7f0f8019e900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:27:23.990 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:23.988+0000 7f0f7fe00640 1 --2- 172.21.15.47:0/3265578614 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0f80074360 0x7f0f8019e900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:27:23.990 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:23.988+0000 7f0f7fe00640 1 -- 172.21.15.47:0/3265578614 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f801a3360 con 0x7f0f80073a40 2024-03-21T10:27:23.990 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:23.988+0000 7f0f7f400640 1 --2- 172.21.15.47:0/3265578614 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0f80074360 0x7f0f8019e900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T10:27:23.990 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:23.988+0000 7f0f7fe00640 1 --2- 172.21.15.47:0/3265578614 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0f80073a40 0x7f0f8019e3c0 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7f0f70000c00 tx=0x7f0f7002d890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:27:23.990 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:23.989+0000 7f0f7cc00640 1 -- 172.21.15.47:0/3265578614 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f70043020 con 0x7f0f80073a40 2024-03-21T10:27:23.990 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:23.989+0000 7f0f7cc00640 1 -- 172.21.15.47:0/3265578614 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f0f7002dd50 con 0x7f0f80073a40 2024-03-21T10:27:23.990 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:23.989+0000 7f0f7cc00640 1 -- 172.21.15.47:0/3265578614 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f70003ab0 con 0x7f0f80073a40 2024-03-21T10:27:23.990 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:23.989+0000 7f0f85600640 1 -- 172.21.15.47:0/3265578614 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0f801a35f0 con 0x7f0f80073a40 2024-03-21T10:27:23.990 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:23.989+0000 7f0f85600640 1 -- 172.21.15.47:0/3265578614 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0f801aaee0 con 0x7f0f80073a40 2024-03-21T10:27:23.992 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:23.993+0000 7f0f7cc00640 1 -- 172.21.15.47:0/3265578614 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 27) v1 ==== 137985+0+0 (secure 0 0 0) 0x7f0f70046020 con 0x7f0f80073a40 2024-03-21T10:27:23.993 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:23.993+0000 7f0f8997f740 1 Processor -- start 2024-03-21T10:27:23.993 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:23.993+0000 7f0f8997f740 1 -- start start 2024-03-21T10:27:23.994 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:23.994+0000 7f0f8997f740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5612597e68b0 0x5612597e6c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:27:23.994 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:23.994+0000 7f0f8997f740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5612597e7260 0x5612597ef810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:27:23.994 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:23.994+0000 7f0f8997f740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5612597efde0 0x5612597f2230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:27:23.994 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:23.994+0000 7f0f8997f740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x56125974a2c0 con 0x5612597efde0 2024-03-21T10:27:23.994 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:23.994+0000 7f0f8997f740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x5612596f2c90 con 0x5612597e7260 2024-03-21T10:27:23.994 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:23.994+0000 7f0f8997f740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x561259746360 con 0x5612597e68b0 2024-03-21T10:27:23.995 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:23.994+0000 7f0f7f400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5612597efde0 0x5612597f2230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:27:23.995 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:23.994+0000 7f0f7f400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5612597efde0 0x5612597f2230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:34892/0 (socket says 172.21.15.47:34892) 2024-03-21T10:27:23.995 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:23.994+0000 7f0f7cc00640 1 --2- 172.21.15.47:0/3265578614 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f0f640a3090 0x7f0f640a5550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:27:23.996 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:23.994+0000 7f0f7f400640 1 -- 172.21.15.47:0/2540855867 learned_addr learned my addr 172.21.15.47:0/2540855867 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:27:23.996 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:23.995+0000 7f0f7fe00640 1 --2- 172.21.15.47:0/2540855867 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5612597e7260 0x5612597ef810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:27:23.996 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:23.995+0000 7f0f84c00640 1 --2- 172.21.15.47:0/2540855867 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5612597e68b0 0x5612597e6c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:27:23.996 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:23.995+0000 7f0f7cc00640 1 -- 172.21.15.47:0/3265578614 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 9474+0+0 (secure 0 0 0) 0x7f0f700f1c80 con 0x7f0f80073a40 2024-03-21T10:27:23.996 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:23.995+0000 7f0f84c00640 1 --2- 172.21.15.47:0/3265578614 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f0f640a3090 0x7f0f640a5550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:27:23.996 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:23.995+0000 7f0f7f400640 1 -- 172.21.15.47:0/2540855867 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5612597e68b0 msgr2=0x5612597e6c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:27:23.996 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:23.995+0000 7f0f7f400640 1 --2- 172.21.15.47:0/2540855867 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5612597e68b0 0x5612597e6c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:27:23.996 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:23.995+0000 7f0f7f400640 1 -- 172.21.15.47:0/2540855867 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5612597e7260 msgr2=0x5612597ef810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:27:23.996 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:23.995+0000 7f0f7f400640 1 --2- 172.21.15.47:0/2540855867 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5612597e7260 0x5612597ef810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:27:23.996 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:23.995+0000 7f0f7f400640 1 -- 172.21.15.47:0/2540855867 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561259717080 con 0x5612597efde0 2024-03-21T10:27:23.996 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:23.996+0000 7f0f7f400640 1 --2- 172.21.15.47:0/2540855867 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5612597efde0 0x5612597f2230 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7f0f6c00bd30 tx=0x7f0f6c00f730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d844d3dd3c5304db server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:27:23.999 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:23.996+0000 7f0f84c00640 1 --2- 172.21.15.47:0/3265578614 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f0f640a3090 0x7f0f640a5550 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f0f78009370 tx=0x7f0f78009240 comp rx=0 tx=0).ready entity=mgr.15687 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:27:23.999 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:23.996+0000 7f0f75200640 1 -- 172.21.15.47:0/2540855867 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f6c015070 con 0x5612597efde0 2024-03-21T10:27:24.000 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:23.996+0000 7f0f75200640 1 -- 172.21.15.47:0/2540855867 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f0f6c01a910 con 0x5612597efde0 2024-03-21T10:27:24.000 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:23.996+0000 7f0f75200640 1 -- 172.21.15.47:0/2540855867 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f6c011840 con 0x5612597efde0 2024-03-21T10:27:24.000 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:23.997+0000 7f0f8997f740 1 -- 172.21.15.47:0/2540855867 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5612597efde0 msgr2=0x5612597f2230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:27:24.000 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:23.997+0000 7f0f8997f740 1 --2- 172.21.15.47:0/2540855867 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5612597efde0 0x5612597f2230 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7f0f6c00bd30 tx=0x7f0f6c00f730 comp rx=0 tx=0).stop 2024-03-21T10:27:24.000 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:23.997+0000 7f0f8997f740 1 -- 172.21.15.47:0/2540855867 shutdown_connections 2024-03-21T10:27:24.000 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:23.997+0000 7f0f8997f740 1 --2- 172.21.15.47:0/2540855867 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5612597efde0 0x5612597f2230 unknown :-1 s=CLOSED pgs=703 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:27:24.000 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:23.997+0000 7f0f8997f740 1 --2- 172.21.15.47:0/2540855867 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5612597e7260 0x5612597ef810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:27:24.000 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:23.997+0000 7f0f8997f740 1 --2- 172.21.15.47:0/2540855867 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5612597e68b0 0x5612597e6c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:27:24.000 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:23.997+0000 7f0f8997f740 1 -- 172.21.15.47:0/2540855867 >> 172.21.15.47:0/2540855867 conn(0x5612597e0f80 msgr2=0x5612597e1360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:27:24.000 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:23.997+0000 7f0f8997f740 1 -- 172.21.15.47:0/2540855867 shutdown_connections 2024-03-21T10:27:24.000 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:23.997+0000 7f0f8997f740 1 -- 172.21.15.47:0/2540855867 wait complete. 2024-03-21T10:27:24.000 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:23.998+0000 7f0f8997f740 1 -- start start 2024-03-21T10:27:24.000 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:23.998+0000 7f0f8997f740 1 Processor -- start 2024-03-21T10:27:24.000 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:23.998+0000 7f0f8997f740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5612597e68b0 0x5612597ef6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:27:24.001 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:23.998+0000 7f0f8997f740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5612597e7260 0x5612597eca50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:27:24.001 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:23.998+0000 7f0f8997f740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5612597efde0 0x5612597ecfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:27:24.001 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:23.999+0000 7f0f8997f740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x5612597f2d50 con 0x5612597e7260 2024-03-21T10:27:24.001 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:23.999+0000 7f0f8997f740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x5612597060d0 con 0x5612597efde0 2024-03-21T10:27:24.001 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:23.999+0000 7f0f8997f740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x5612597f2ed0 con 0x5612597e68b0 2024-03-21T10:27:24.001 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:23.999+0000 7f0f8997f740 10 client.0 _ll_register_callbacks cb 0x56125970ae80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T10:27:24.001 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:23.999+0000 7f0f7f400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5612597e7260 0x5612597eca50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:27:24.001 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:23.999+0000 7f0f7f400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5612597e7260 0x5612597eca50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:34902/0 (socket says 172.21.15.47:34902) 2024-03-21T10:27:24.001 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:23.999+0000 7f0f7f400640 1 -- 172.21.15.47:0/1078130350 learned_addr learned my addr 172.21.15.47:0/1078130350 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:27:24.001 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:23.999+0000 7f0f7fe00640 1 --2- 172.21.15.47:0/1078130350 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5612597e68b0 0x5612597ef6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:27:24.001 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:23.999+0000 7f0f84c00640 1 --2- 172.21.15.47:0/1078130350 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5612597efde0 0x5612597ecfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:27:24.001 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:23.999+0000 7f0f7f400640 1 -- 172.21.15.47:0/1078130350 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5612597e68b0 msgr2=0x5612597ef6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:27:24.001 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:23.999+0000 7f0f7f400640 1 --2- 172.21.15.47:0/1078130350 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5612597e68b0 0x5612597ef6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:27:24.001 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:23.999+0000 7f0f7f400640 1 -- 172.21.15.47:0/1078130350 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5612597efde0 msgr2=0x5612597ecfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:27:24.002 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:23.999+0000 7f0f7f400640 1 --2- 172.21.15.47:0/1078130350 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5612597efde0 0x5612597ecfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:27:24.002 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:23.999+0000 7f0f7f400640 1 -- 172.21.15.47:0/1078130350 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56125945e3f0 con 0x5612597e7260 2024-03-21T10:27:24.002 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:24.000+0000 7f0f7f400640 1 --2- 172.21.15.47:0/1078130350 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5612597e7260 0x5612597eca50 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7f0f6c00ba70 tx=0x7f0f6c00bf50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fede0d274ceedb7c server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:27:24.002 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:24.000+0000 7f0f69600640 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-03-21T10:27:24.004 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:24.000+0000 7f0f69600640 1 -- 172.21.15.47:0/1078130350 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f6c028060 con 0x5612597e7260 2024-03-21T10:27:24.004 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:24.000+0000 7f0f69600640 1 -- 172.21.15.47:0/1078130350 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(97..97 src has 1..97) v4 ==== 9474+0+0 (secure 0 0 0) 0x7f0f6c01aac0 con 0x5612597e7260 2024-03-21T10:27:24.004 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:24.000+0000 7f0f8997f740 1 -- 172.21.15.47:0/1078130350 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f0f6c01a910 con 0x5612597e7260 2024-03-21T10:27:24.004 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:24.001+0000 7f0f69600640 1 -- 172.21.15.47:0/1078130350 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0f6c02f050 con 0x5612597e7260 2024-03-21T10:27:24.004 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:24.001+0000 7f0f8997f740 10 client.16482 fetch_fsmap learned FSMap version 32 2024-03-21T10:27:24.004 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:24.001+0000 7f0f8997f740 1 -- 172.21.15.47:0/1078130350 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x5612595287d0 con 0x5612597e7260 2024-03-21T10:27:24.005 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:24.001+0000 7f0f69600640 1 -- 172.21.15.47:0/1078130350 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f0f6c01a910 con 0x5612597e7260 2024-03-21T10:27:24.005 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:24.001+0000 7f0f69600640 1 -- 172.21.15.47:0/1078130350 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 32) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f0f6c004880 con 0x5612597e7260 2024-03-21T10:27:24.005 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:24.001+0000 7f0f8997f740 10 client.16482 fetch_fsmap finished waiting for FSMap version 32 2024-03-21T10:27:24.005 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:24.001+0000 7f0f8997f740 10 client.16482 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-21T10:27:24.005 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:24.001+0000 7f0f8997f740 20 client.16482 populate_metadata read hostname 'smithi047' 2024-03-21T10:27:24.005 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:24.001+0000 7f0f8997f740 1 --2- 172.21.15.47:0/1078130350 >> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] conn(0x5612597ffab0 0x561259801ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:27:24.005 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:24.001+0000 7f0f8997f740 4 client.16482 mds_command: new command op to 34535 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T10:27:24.005 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:24.001+0000 7f0f8997f740 1 -- 172.21.15.47:0/1078130350 --> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x561259757800 con 0x5612597ffab0 2024-03-21T10:27:24.005 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:24.002+0000 7f0f7fe00640 1 --2- 172.21.15.47:0/1078130350 >> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] conn(0x5612597ffab0 0x561259801ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:27:24.005 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:24.002+0000 7f0f7fe00640 1 --2- 172.21.15.47:0/1078130350 >> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] conn(0x5612597ffab0 0x561259801ea0 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=52f017cb55955eff server_cookie=95cd6949ea43b298 in_seq=0 out_seq=0 2024-03-21T10:27:24.006 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:24.002+0000 7f0f69600640 10 client.16482 ms_handle_connect on v2:172.21.15.100:6838/4173168440 2024-03-21T10:27:24.006 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:24.003+0000 7f0f69600640 1 -- 172.21.15.47:0/1078130350 <== mds.0 v2:172.21.15.100:6838/4173168440 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x561259757800 con 0x5612597ffab0 2024-03-21T10:27:24.006 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:24.003+0000 7f0f69600640 10 client.16482 handle_command_reply: tid=0 2024-03-21T10:27:24.006 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:24.003+0000 7f0f8997f740 1 client.16482 shutdown 2024-03-21T10:27:24.006 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:24.004+0000 7f0f8997f740 1 -- 172.21.15.47:0/1078130350 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5612597e7260 msgr2=0x5612597eca50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:27:24.006 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:24.004+0000 7f0f8997f740 1 --2- 172.21.15.47:0/1078130350 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5612597e7260 0x5612597eca50 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7f0f6c00ba70 tx=0x7f0f6c00bf50 comp rx=0 tx=0).stop 2024-03-21T10:27:24.006 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:24.004+0000 7f0f8997f740 1 -- 172.21.15.47:0/1078130350 shutdown_connections 2024-03-21T10:27:24.006 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:24.004+0000 7f0f8997f740 1 --2- 172.21.15.47:0/1078130350 >> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] conn(0x5612597ffab0 0x561259801ea0 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:27:24.006 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:24.004+0000 7f0f8997f740 1 --2- 172.21.15.47:0/1078130350 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5612597efde0 0x5612597ecfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:27:24.006 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:24.004+0000 7f0f8997f740 1 --2- 172.21.15.47:0/1078130350 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5612597e7260 0x5612597eca50 unknown :-1 s=CLOSED pgs=704 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:27:24.006 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:24.004+0000 7f0f8997f740 1 --2- 172.21.15.47:0/1078130350 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5612597e68b0 0x5612597ef6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:27:24.007 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:24.004+0000 7f0f8997f740 1 -- 172.21.15.47:0/1078130350 >> 172.21.15.47:0/1078130350 conn(0x5612597e0f80 msgr2=0x5612597e2cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:27:24.007 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:24.004+0000 7f0f69600640 0 client.16482 ms_handle_reset on v2:172.21.15.100:6838/4173168440 2024-03-21T10:27:24.007 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:24.004+0000 7f0f8997f740 1 -- 172.21.15.47:0/1078130350 shutdown_connections 2024-03-21T10:27:24.007 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:24.004+0000 7f0f8997f740 1 -- 172.21.15.47:0/1078130350 wait complete. 2024-03-21T10:27:24.007 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:24.004+0000 7f0f8997f740 20 client.16482 trim_cache size 0 max 16384 2024-03-21T10:27:24.023 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:24.022+0000 7f0f8997f740 1 Processor -- start 2024-03-21T10:27:24.023 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:24.022+0000 7f0f8997f740 1 -- start start 2024-03-21T10:27:24.023 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:24.023+0000 7f0f8997f740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5612597e68b0 0x5612597ebbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:27:24.023 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:24.023+0000 7f0f8997f740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5612597e7260 0x5612597dffe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:27:24.024 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:24.023+0000 7f0f8997f740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5612597efde0 0x5612597ef180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:27:24.024 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:24.023+0000 7f0f8997f740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f801b4cc0 con 0x5612597e7260 2024-03-21T10:27:24.024 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:24.023+0000 7f0f8997f740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f801b6640 con 0x5612597efde0 2024-03-21T10:27:24.024 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:24.023+0000 7f0f8997f740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f801b5320 con 0x5612597e68b0 2024-03-21T10:27:24.024 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:24.023+0000 7f0f7f400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5612597e7260 0x5612597dffe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:27:24.024 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:24.023+0000 7f0f84c00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5612597efde0 0x5612597ef180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:27:24.024 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:24.023+0000 7f0f7f400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5612597e7260 0x5612597dffe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:34912/0 (socket says 172.21.15.47:34912) 2024-03-21T10:27:24.024 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:24.023+0000 7f0f7f400640 1 -- 172.21.15.47:0/1462380041 learned_addr learned my addr 172.21.15.47:0/1462380041 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:27:24.026 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:24.023+0000 7f0f7fe00640 1 --2- 172.21.15.47:0/1462380041 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5612597e68b0 0x5612597ebbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:27:24.026 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:24.024+0000 7f0f7f400640 1 -- 172.21.15.47:0/1462380041 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5612597e68b0 msgr2=0x5612597ebbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:27:24.026 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:24.024+0000 7f0f7f400640 1 --2- 172.21.15.47:0/1462380041 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5612597e68b0 0x5612597ebbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:27:24.026 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:24.024+0000 7f0f7f400640 1 -- 172.21.15.47:0/1462380041 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5612597efde0 msgr2=0x5612597ef180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:27:24.026 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:24.024+0000 7f0f7f400640 1 --2- 172.21.15.47:0/1462380041 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5612597efde0 0x5612597ef180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:27:24.026 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:24.024+0000 7f0f7f400640 1 -- 172.21.15.47:0/1462380041 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5612597e0520 con 0x5612597e7260 2024-03-21T10:27:24.027 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:24.024+0000 7f0f7fe00640 1 --2- 172.21.15.47:0/1462380041 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5612597e68b0 0x5612597ebbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T10:27:24.027 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:24.024+0000 7f0f7f400640 1 --2- 172.21.15.47:0/1462380041 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5612597e7260 0x5612597dffe0 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7f0f6c00bac0 tx=0x7f0f6c011650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e78ed60c06d5c320 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:27:24.027 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:24.024+0000 7f0f6be00640 1 -- 172.21.15.47:0/1462380041 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f6c028070 con 0x5612597e7260 2024-03-21T10:27:24.027 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:24.024+0000 7f0f6be00640 1 -- 172.21.15.47:0/1462380041 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f0f6c004dc0 con 0x5612597e7260 2024-03-21T10:27:24.027 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:24.024+0000 7f0f6be00640 1 -- 172.21.15.47:0/1462380041 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f6c015070 con 0x5612597e7260 2024-03-21T10:27:24.027 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:24.025+0000 7f0f8997f740 1 -- 172.21.15.47:0/1462380041 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5612597e7260 msgr2=0x5612597dffe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:27:24.027 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:24.025+0000 7f0f8997f740 1 --2- 172.21.15.47:0/1462380041 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5612597e7260 0x5612597dffe0 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7f0f6c00bac0 tx=0x7f0f6c011650 comp rx=0 tx=0).stop 2024-03-21T10:27:24.027 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:24.025+0000 7f0f8997f740 1 -- 172.21.15.47:0/1462380041 shutdown_connections 2024-03-21T10:27:24.027 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:24.025+0000 7f0f8997f740 1 --2- 172.21.15.47:0/1462380041 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5612597efde0 0x5612597ef180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:27:24.027 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:24.025+0000 7f0f8997f740 1 --2- 172.21.15.47:0/1462380041 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5612597e7260 0x5612597dffe0 unknown :-1 s=CLOSED pgs=705 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:27:24.027 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:24.025+0000 7f0f8997f740 1 --2- 172.21.15.47:0/1462380041 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5612597e68b0 0x5612597ebbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:27:24.028 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:24.025+0000 7f0f8997f740 1 -- 172.21.15.47:0/1462380041 >> 172.21.15.47:0/1462380041 conn(0x5612597e0f80 msgr2=0x5612597df8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:27:24.028 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:24.026+0000 7f0f8997f740 1 -- 172.21.15.47:0/1462380041 shutdown_connections 2024-03-21T10:27:24.028 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:24.026+0000 7f0f8997f740 1 -- 172.21.15.47:0/1462380041 wait complete. 2024-03-21T10:27:24.028 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:24.026+0000 7f0f8997f740 1 -- start start 2024-03-21T10:27:24.028 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:24.026+0000 7f0f8997f740 1 Processor -- start 2024-03-21T10:27:24.030 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:24.027+0000 7f0f8997f740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5612597e68b0 0x5612597f37c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:27:24.031 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:24.027+0000 7f0f8997f740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5612597e7260 0x5612597f3d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:27:24.031 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:24.027+0000 7f0f8997f740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5612597efde0 0x5612597f4240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:27:24.031 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:24.027+0000 7f0f8997f740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f801b5fe0 con 0x5612597efde0 2024-03-21T10:27:24.031 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:24.027+0000 7f0f8997f740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f801b5980 con 0x5612597e68b0 2024-03-21T10:27:24.031 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:24.027+0000 7f0f8997f740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f801b2e80 con 0x5612597e7260 2024-03-21T10:27:24.034 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:24.027+0000 7f0f8997f740 10 client.0 _ll_register_callbacks cb 0x7f0f6c00bb80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T10:27:24.034 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:24.027+0000 7f0f7fe00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5612597e68b0 0x5612597f37c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:27:24.035 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:24.027+0000 7f0f84c00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5612597efde0 0x5612597f4240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:27:24.035 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:24.027+0000 7f0f7fe00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5612597e68b0 0x5612597f37c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:45296/0 (socket says 172.21.15.47:45296) 2024-03-21T10:27:24.035 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:24.027+0000 7f0f7fe00640 1 -- 172.21.15.47:0/635194702 learned_addr learned my addr 172.21.15.47:0/635194702 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:27:24.035 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:24.027+0000 7f0f7f400640 1 --2- 172.21.15.47:0/635194702 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5612597e7260 0x5612597f3d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:27:24.035 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:24.028+0000 7f0f7fe00640 1 -- 172.21.15.47:0/635194702 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5612597e7260 msgr2=0x5612597f3d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:27:24.035 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:24.028+0000 7f0f7fe00640 1 --2- 172.21.15.47:0/635194702 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5612597e7260 0x5612597f3d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:27:24.035 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:24.028+0000 7f0f7fe00640 1 -- 172.21.15.47:0/635194702 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5612597efde0 msgr2=0x5612597f4240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:27:24.035 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:24.028+0000 7f0f7fe00640 1 --2- 172.21.15.47:0/635194702 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5612597efde0 0x5612597f4240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:27:24.035 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:24.028+0000 7f0f7fe00640 1 -- 172.21.15.47:0/635194702 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56125985a2a0 con 0x5612597e68b0 2024-03-21T10:27:24.036 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:24.028+0000 7f0f84c00640 1 --2- 172.21.15.47:0/635194702 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5612597efde0 0x5612597f4240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T10:27:24.036 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:24.028+0000 7f0f7fe00640 1 --2- 172.21.15.47:0/635194702 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5612597e68b0 0x5612597f37c0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f0f700ba1c0 tx=0x7f0f7008d830 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5ad18dc1fc1ea3d5 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:27:24.036 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:24.028+0000 7f0f69600640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-21T10:27:24.036 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:24.028+0000 7f0f69600640 1 -- 172.21.15.47:0/635194702 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f70088cc0 con 0x5612597e68b0 2024-03-21T10:27:24.036 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:24.028+0000 7f0f8997f740 1 -- 172.21.15.47:0/635194702 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f0f6c004dc0 con 0x5612597e68b0 2024-03-21T10:27:24.036 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:24.028+0000 7f0f69600640 1 -- 172.21.15.47:0/635194702 <== mon.2 v2:172.21.15.100:3300/0 2 ==== osd_map(97..97 src has 1..97) v4 ==== 9474+0+0 (secure 0 0 0) 0x7f0f700ba3d0 con 0x5612597e68b0 2024-03-21T10:27:24.036 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:24.029+0000 7f0f69600640 1 -- 172.21.15.47:0/635194702 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0f700be460 con 0x5612597e68b0 2024-03-21T10:27:24.037 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:24.029+0000 7f0f8997f740 10 client.36278 fetch_fsmap learned FSMap version 32 2024-03-21T10:27:24.037 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:24.029+0000 7f0f8997f740 1 -- 172.21.15.47:0/635194702 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x56125985b5e0 con 0x5612597e68b0 2024-03-21T10:27:24.037 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:24.029+0000 7f0f69600640 1 -- 172.21.15.47:0/635194702 <== mon.2 v2:172.21.15.100:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f0f6c004dc0 con 0x5612597e68b0 2024-03-21T10:27:24.037 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:24.029+0000 7f0f69600640 1 -- 172.21.15.47:0/635194702 <== mon.2 v2:172.21.15.100:3300/0 5 ==== fsmap(e 32) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f0f70003c50 con 0x5612597e68b0 2024-03-21T10:27:24.037 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:24.029+0000 7f0f8997f740 10 client.36278 fetch_fsmap finished waiting for FSMap version 32 2024-03-21T10:27:24.037 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:24.029+0000 7f0f8997f740 10 client.36278 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-21T10:27:24.037 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:24.029+0000 7f0f8997f740 20 client.36278 populate_metadata read hostname 'smithi047' 2024-03-21T10:27:24.037 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:24.029+0000 7f0f8997f740 1 --2- 172.21.15.47:0/635194702 >> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] conn(0x56125985bc60 0x56125985e050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:27:24.037 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:24.029+0000 7f0f8997f740 4 client.36278 mds_command: new command op to 34535 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-21T10:27:24.038 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:24.029+0000 7f0f8997f740 1 -- 172.21.15.47:0/635194702 --> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x561259751b50 con 0x56125985bc60 2024-03-21T10:27:24.038 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:24.030+0000 7f0f7f400640 1 --2- 172.21.15.47:0/635194702 >> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] conn(0x56125985bc60 0x56125985e050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:27:24.038 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:24.031+0000 7f0f7f400640 1 --2- 172.21.15.47:0/635194702 >> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] conn(0x56125985bc60 0x56125985e050 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5611e4a4092e68f server_cookie=575d9a2e26f57b9 in_seq=0 out_seq=0 2024-03-21T10:27:24.038 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:24.031+0000 7f0f69600640 10 client.36278 ms_handle_connect on v2:172.21.15.100:6838/4173168440 2024-03-21T10:27:24.040 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:24.031+0000 7f0f69600640 1 -- 172.21.15.47:0/635194702 <== mds.0 v2:172.21.15.100:6838/4173168440 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x561259751b50 con 0x56125985bc60 2024-03-21T10:27:24.041 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:24.031+0000 7f0f69600640 10 client.36278 handle_command_reply: tid=0 2024-03-21T10:27:24.041 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:24.031+0000 7f0f8997f740 1 client.36278 shutdown 2024-03-21T10:27:24.041 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:24.031+0000 7f0f8997f740 1 -- 172.21.15.47:0/635194702 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5612597e68b0 msgr2=0x5612597f37c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:27:24.041 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:24.031+0000 7f0f8997f740 1 --2- 172.21.15.47:0/635194702 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5612597e68b0 0x5612597f37c0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f0f700ba1c0 tx=0x7f0f7008d830 comp rx=0 tx=0).stop 2024-03-21T10:27:24.041 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:24.031+0000 7f0f8997f740 1 -- 172.21.15.47:0/635194702 shutdown_connections 2024-03-21T10:27:24.041 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:24.031+0000 7f0f8997f740 1 --2- 172.21.15.47:0/635194702 >> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] conn(0x56125985bc60 0x56125985e050 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:27:24.042 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:24.031+0000 7f0f8997f740 1 --2- 172.21.15.47:0/635194702 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5612597efde0 0x5612597f4240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:27:24.042 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:24.031+0000 7f0f8997f740 1 --2- 172.21.15.47:0/635194702 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5612597e7260 0x5612597f3d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:27:24.042 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:24.031+0000 7f0f8997f740 1 --2- 172.21.15.47:0/635194702 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5612597e68b0 0x5612597f37c0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:27:24.042 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:24.031+0000 7f0f8997f740 1 -- 172.21.15.47:0/635194702 >> 172.21.15.47:0/635194702 conn(0x5612597e0f80 msgr2=0x5612597ebbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:27:24.042 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:24.031+0000 7f0f69600640 0 client.36278 ms_handle_reset on v2:172.21.15.100:6838/4173168440 2024-03-21T10:27:24.042 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:24.032+0000 7f0f8997f740 1 -- 172.21.15.47:0/635194702 shutdown_connections 2024-03-21T10:27:24.042 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:24.032+0000 7f0f8997f740 1 -- 172.21.15.47:0/635194702 wait complete. 2024-03-21T10:27:24.043 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:24.032+0000 7f0f8997f740 20 client.36278 trim_cache size 0 max 16384 2024-03-21T10:27:24.043 INFO:teuthology.orchestra.run.smithi047.stdout:[] 2024-03-21T10:27:24.045 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:24.032+0000 7f0f85600640 1 -- 172.21.15.47:0/3265578614 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f0f640a3090 msgr2=0x7f0f640a5550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:27:24.045 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:24.032+0000 7f0f85600640 1 --2- 172.21.15.47:0/3265578614 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f0f640a3090 0x7f0f640a5550 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f0f78009370 tx=0x7f0f78009240 comp rx=0 tx=0).stop 2024-03-21T10:27:24.045 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:24.032+0000 7f0f85600640 1 -- 172.21.15.47:0/3265578614 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0f80073a40 msgr2=0x7f0f8019e3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:27:24.045 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:24.033+0000 7f0f85600640 1 --2- 172.21.15.47:0/3265578614 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0f80073a40 0x7f0f8019e3c0 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7f0f70000c00 tx=0x7f0f7002d890 comp rx=0 tx=0).stop 2024-03-21T10:27:24.045 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:24.033+0000 7f0f85600640 1 -- 172.21.15.47:0/3265578614 shutdown_connections 2024-03-21T10:27:24.046 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:24.033+0000 7f0f85600640 1 --2- 172.21.15.47:0/3265578614 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0f80112520 0x7f0f801a2c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:27:24.046 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:24.033+0000 7f0f85600640 1 --2- 172.21.15.47:0/3265578614 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f0f640a3090 0x7f0f640a5550 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:27:24.046 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:24.033+0000 7f0f85600640 1 --2- 172.21.15.47:0/3265578614 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0f80074360 0x7f0f8019e900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:27:24.046 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:24.033+0000 7f0f85600640 1 --2- 172.21.15.47:0/3265578614 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0f80073a40 0x7f0f8019e3c0 unknown :-1 s=CLOSED pgs=702 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:27:24.046 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:24.033+0000 7f0f85600640 1 -- 172.21.15.47:0/3265578614 >> 172.21.15.47:0/3265578614 conn(0x7f0f800fd040 msgr2=0x7f0f800fe940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:27:24.046 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:24.033+0000 7f0f85600640 1 -- 172.21.15.47:0/3265578614 shutdown_connections 2024-03-21T10:27:24.047 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:24.033+0000 7f0f85600640 1 -- 172.21.15.47:0/3265578614 wait complete. 2024-03-21T10:27:25.054 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-03-21T10:27:25.054 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-21T10:27:25.055 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-03-21T10:27:25.280 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:25.279+0000 7fd053400640 1 -- 172.21.15.47:0/1721280550 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd04c068880 msgr2=0x7fd04c068d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:27:25.280 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:25.279+0000 7fd053400640 1 --2- 172.21.15.47:0/1721280550 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd04c068880 0x7fd04c068d00 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7fd038009930 tx=0x7fd03802bc20 comp rx=0 tx=0).stop 2024-03-21T10:27:25.280 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:25.279+0000 7fd053400640 1 -- 172.21.15.47:0/1721280550 shutdown_connections 2024-03-21T10:27:25.280 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:25.279+0000 7fd053400640 1 --2- 172.21.15.47:0/1721280550 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd04c069240 0x7fd04c10e4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:27:25.280 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:25.279+0000 7fd053400640 1 --2- 172.21.15.47:0/1721280550 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd04c068880 0x7fd04c068d00 unknown :-1 s=CLOSED pgs=706 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:27:25.281 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:25.279+0000 7fd053400640 1 --2- 172.21.15.47:0/1721280550 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd04c1076b0 0x7fd04c107a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:27:25.281 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:25.279+0000 7fd053400640 1 -- 172.21.15.47:0/1721280550 >> 172.21.15.47:0/1721280550 conn(0x7fd04c0776a0 msgr2=0x7fd04c077ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:27:25.281 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:25.279+0000 7fd053400640 1 -- 172.21.15.47:0/1721280550 shutdown_connections 2024-03-21T10:27:25.281 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:25.279+0000 7fd053400640 1 -- 172.21.15.47:0/1721280550 wait complete. 2024-03-21T10:27:25.284 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:25.280+0000 7fd053400640 1 Processor -- start 2024-03-21T10:27:25.284 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:25.280+0000 7fd053400640 1 -- start start 2024-03-21T10:27:25.284 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:25.281+0000 7fd053400640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd04c068880 0x7fd04c197e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:27:25.284 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:25.281+0000 7fd053400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd04c069240 0x7fd04c1983b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:27:25.284 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:25.281+0000 7fd053400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd04c1076b0 0x7fd04c19c6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:27:25.285 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:25.281+0000 7fd053400640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd04c1118f0 con 0x7fd04c1076b0 2024-03-21T10:27:25.285 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:25.281+0000 7fd053400640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd04c111770 con 0x7fd04c069240 2024-03-21T10:27:25.285 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:25.281+0000 7fd053400640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd04c111a70 con 0x7fd04c068880 2024-03-21T10:27:25.285 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:25.281+0000 7fd051400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd04c1076b0 0x7fd04c19c6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:27:25.285 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:25.281+0000 7fd051400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd04c1076b0 0x7fd04c19c6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:34946/0 (socket says 172.21.15.47:34946) 2024-03-21T10:27:25.285 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:25.281+0000 7fd051400640 1 -- 172.21.15.47:0/4278473528 learned_addr learned my addr 172.21.15.47:0/4278473528 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:27:25.285 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:25.282+0000 7fd050a00640 1 --2- 172.21.15.47:0/4278473528 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd04c068880 0x7fd04c197e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:27:25.285 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:25.282+0000 7fd04be00640 1 --2- 172.21.15.47:0/4278473528 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd04c069240 0x7fd04c1983b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:27:25.285 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:25.282+0000 7fd051400640 1 -- 172.21.15.47:0/4278473528 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd04c068880 msgr2=0x7fd04c197e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:27:25.285 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:25.282+0000 7fd051400640 1 --2- 172.21.15.47:0/4278473528 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd04c068880 0x7fd04c197e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:27:25.285 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:25.282+0000 7fd051400640 1 -- 172.21.15.47:0/4278473528 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd04c069240 msgr2=0x7fd04c1983b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:27:25.285 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:25.282+0000 7fd051400640 1 --2- 172.21.15.47:0/4278473528 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd04c069240 0x7fd04c1983b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:27:25.285 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:25.282+0000 7fd051400640 1 -- 172.21.15.47:0/4278473528 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd04c19ce10 con 0x7fd04c1076b0 2024-03-21T10:27:25.285 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:25.282+0000 7fd051400640 1 --2- 172.21.15.47:0/4278473528 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd04c1076b0 0x7fd04c19c6e0 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7fd04400eb10 tx=0x7fd04400c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:27:25.285 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:25.283+0000 7fd049600640 1 -- 172.21.15.47:0/4278473528 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd044004b20 con 0x7fd04c1076b0 2024-03-21T10:27:25.286 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:25.283+0000 7fd049600640 1 -- 172.21.15.47:0/4278473528 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fd044004cc0 con 0x7fd04c1076b0 2024-03-21T10:27:25.286 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:25.283+0000 7fd053400640 1 -- 172.21.15.47:0/4278473528 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd04c19d100 con 0x7fd04c1076b0 2024-03-21T10:27:25.286 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:25.283+0000 7fd049600640 1 -- 172.21.15.47:0/4278473528 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd044010690 con 0x7fd04c1076b0 2024-03-21T10:27:25.286 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:25.283+0000 7fd053400640 1 -- 172.21.15.47:0/4278473528 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd04c1a4990 con 0x7fd04c1076b0 2024-03-21T10:27:25.286 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:25.286+0000 7fd055d22740 1 Processor -- start 2024-03-21T10:27:25.287 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:25.288+0000 7fd049600640 1 -- 172.21.15.47:0/4278473528 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 27) v1 ==== 137985+0+0 (secure 0 0 0) 0x7fd044005020 con 0x7fd04c1076b0 2024-03-21T10:27:25.288 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:25.288+0000 7fd055d22740 1 -- start start 2024-03-21T10:27:25.288 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:25.288+0000 7fd055d22740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55dd0cf73490 0x55dd0cf73870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:27:25.289 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:25.288+0000 7fd055d22740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55dd0cf73db0 0x55dd0cf7c450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:27:25.289 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:25.288+0000 7fd055d22740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55dd0cf7c990 0x55dd0cf7ed80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:27:25.289 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:25.288+0000 7fd055d22740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dd0ced3eb0 con 0x55dd0cf73490 2024-03-21T10:27:25.289 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:25.288+0000 7fd055d22740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dd0ce7ca60 con 0x55dd0cf73db0 2024-03-21T10:27:25.289 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:25.288+0000 7fd055d22740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dd0cecff50 con 0x55dd0cf7c990 2024-03-21T10:27:25.289 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:25.289+0000 7fd04be00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55dd0cf73490 0x55dd0cf73870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:27:25.289 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:25.289+0000 7fd04be00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55dd0cf73490 0x55dd0cf73870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:34954/0 (socket says 172.21.15.47:34954) 2024-03-21T10:27:25.289 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:25.289+0000 7fd04be00640 1 -- 172.21.15.47:0/2175903028 learned_addr learned my addr 172.21.15.47:0/2175903028 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:27:25.289 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:25.289+0000 7fd050a00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55dd0cf73db0 0x55dd0cf7c450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:27:25.289 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:25.289+0000 7fd04be00640 1 --2- 172.21.15.47:0/2175903028 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55dd0cf7c990 0x55dd0cf7ed80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:27:25.290 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:25.290+0000 7fd04be00640 1 -- 172.21.15.47:0/2175903028 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55dd0cf7c990 msgr2=0x55dd0cf7ed80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:27:25.290 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:25.290+0000 7fd04be00640 1 --2- 172.21.15.47:0/2175903028 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55dd0cf7c990 0x55dd0cf7ed80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:27:25.291 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:25.290+0000 7fd04be00640 1 -- 172.21.15.47:0/2175903028 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55dd0cf73db0 msgr2=0x55dd0cf7c450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:27:25.291 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:25.290+0000 7fd04be00640 1 --2- 172.21.15.47:0/2175903028 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55dd0cf73db0 0x55dd0cf7c450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:27:25.291 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:25.290+0000 7fd049600640 1 --2- 172.21.15.47:0/4278473528 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7fd0340a3090 0x7fd0340a5550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:27:25.291 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:25.290+0000 7fd04be00640 1 -- 172.21.15.47:0/2175903028 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55dd0ccb07e0 con 0x55dd0cf73490 2024-03-21T10:27:25.291 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:25.290+0000 7fd049600640 1 -- 172.21.15.47:0/4278473528 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 9474+0+0 (secure 0 0 0) 0x7fd0440d01d0 con 0x7fd04c1076b0 2024-03-21T10:27:25.291 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:25.291+0000 7fd04be00640 1 --2- 172.21.15.47:0/2175903028 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55dd0cf73490 0x55dd0cf73870 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7fd038005ab0 tx=0x7fd038005e00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8a0c2d45a2b1ffbc server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:27:25.291 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:25.291+0000 7fd050a00640 1 --2- 172.21.15.47:0/4278473528 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7fd0340a3090 0x7fd0340a5550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:27:25.292 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:25.291+0000 7fd03dc00640 1 -- 172.21.15.47:0/2175903028 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd03804a070 con 0x55dd0cf73490 2024-03-21T10:27:25.292 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:25.291+0000 7fd03dc00640 1 -- 172.21.15.47:0/2175903028 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fd038005050 con 0x55dd0cf73490 2024-03-21T10:27:25.292 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:25.292+0000 7fd03dc00640 1 -- 172.21.15.47:0/2175903028 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd038004970 con 0x55dd0cf73490 2024-03-21T10:27:25.292 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:25.292+0000 7fd050a00640 1 --2- 172.21.15.47:0/4278473528 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7fd0340a3090 0x7fd0340a5550 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x55dd0cf72f60 tx=0x7fd04000c430 comp rx=0 tx=0).ready entity=mgr.15687 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:27:25.292 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:25.292+0000 7fd055d22740 1 -- 172.21.15.47:0/2175903028 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55dd0cf73490 msgr2=0x55dd0cf73870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:27:25.292 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:25.292+0000 7fd055d22740 1 --2- 172.21.15.47:0/2175903028 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55dd0cf73490 0x55dd0cf73870 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7fd038005ab0 tx=0x7fd038005e00 comp rx=0 tx=0).stop 2024-03-21T10:27:25.293 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:25.292+0000 7fd055d22740 1 -- 172.21.15.47:0/2175903028 shutdown_connections 2024-03-21T10:27:25.293 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:25.292+0000 7fd055d22740 1 --2- 172.21.15.47:0/2175903028 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55dd0cf7c990 0x55dd0cf7ed80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:27:25.293 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:25.292+0000 7fd055d22740 1 --2- 172.21.15.47:0/2175903028 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55dd0cf73db0 0x55dd0cf7c450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:27:25.293 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:25.292+0000 7fd055d22740 1 --2- 172.21.15.47:0/2175903028 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55dd0cf73490 0x55dd0cf73870 unknown :-1 s=CLOSED pgs=708 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:27:25.293 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:25.293+0000 7fd055d22740 1 -- 172.21.15.47:0/2175903028 >> 172.21.15.47:0/2175903028 conn(0x55dd0cf6dbc0 msgr2=0x55dd0cf6dfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:27:25.293 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:25.293+0000 7fd055d22740 1 -- 172.21.15.47:0/2175903028 shutdown_connections 2024-03-21T10:27:25.293 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:25.293+0000 7fd055d22740 1 -- 172.21.15.47:0/2175903028 wait complete. 2024-03-21T10:27:25.293 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:25.293+0000 7fd055d22740 1 -- start start 2024-03-21T10:27:25.295 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:25.294+0000 7fd055d22740 1 Processor -- start 2024-03-21T10:27:25.295 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:25.294+0000 7fd055d22740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55dd0cf73db0 0x55dd0cf79750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:27:25.295 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:25.294+0000 7fd055d22740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55dd0cf7c990 0x55dd0cf79c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:27:25.295 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:25.294+0000 7fd055d22740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55dd0cf7a1d0 0x55dd0cf88b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:27:25.295 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:25.294+0000 7fd055d22740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dd0cf7f720 con 0x55dd0cf73db0 2024-03-21T10:27:25.295 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:25.294+0000 7fd055d22740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dd0ce8feb0 con 0x55dd0cf7c990 2024-03-21T10:27:25.295 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:25.294+0000 7fd055d22740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dd0cf7f8a0 con 0x55dd0cf7a1d0 2024-03-21T10:27:25.295 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:25.294+0000 7fd055d22740 10 client.0 _ll_register_callbacks cb 0x55dd0ce84150 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T10:27:25.295 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:25.295+0000 7fd050a00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55dd0cf73db0 0x55dd0cf79750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:27:25.296 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:25.295+0000 7fd050a00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55dd0cf73db0 0x55dd0cf79750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:34964/0 (socket says 172.21.15.47:34964) 2024-03-21T10:27:25.296 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:25.295+0000 7fd050a00640 1 -- 172.21.15.47:0/2154906376 learned_addr learned my addr 172.21.15.47:0/2154906376 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:27:25.296 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:25.295+0000 7fd04be00640 1 --2- 172.21.15.47:0/2154906376 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55dd0cf7c990 0x55dd0cf79c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:27:25.297 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:25.295+0000 7fd051400640 1 --2- 172.21.15.47:0/2154906376 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55dd0cf7a1d0 0x55dd0cf88b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:27:25.297 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:25.296+0000 7fd050a00640 1 -- 172.21.15.47:0/2154906376 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55dd0cf7a1d0 msgr2=0x55dd0cf88b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:27:25.297 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:25.296+0000 7fd050a00640 1 --2- 172.21.15.47:0/2154906376 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55dd0cf7a1d0 0x55dd0cf88b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:27:25.297 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:25.296+0000 7fd050a00640 1 -- 172.21.15.47:0/2154906376 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55dd0cf7c990 msgr2=0x55dd0cf79c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:27:25.297 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:25.296+0000 7fd050a00640 1 --2- 172.21.15.47:0/2154906376 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55dd0cf7c990 0x55dd0cf79c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:27:25.297 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:25.296+0000 7fd050a00640 1 -- 172.21.15.47:0/2154906376 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55dd0cd10f70 con 0x55dd0cf73db0 2024-03-21T10:27:25.297 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:25.296+0000 7fd050a00640 1 --2- 172.21.15.47:0/2154906376 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55dd0cf73db0 0x55dd0cf79750 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7fd04001ce00 tx=0x7fd0400046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7d40b474510e134 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:27:25.297 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:25.296+0000 7fd032000640 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-03-21T10:27:25.297 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:25.297+0000 7fd032000640 1 -- 172.21.15.47:0/2154906376 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd040025850 con 0x55dd0cf73db0 2024-03-21T10:27:25.298 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:25.297+0000 7fd032000640 1 -- 172.21.15.47:0/2154906376 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(97..97 src has 1..97) v4 ==== 9474+0+0 (secure 0 0 0) 0x7fd0400259f0 con 0x55dd0cf73db0 2024-03-21T10:27:25.298 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:25.297+0000 7fd055d22740 1 -- 172.21.15.47:0/2154906376 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fd038005050 con 0x55dd0cf73db0 2024-03-21T10:27:25.298 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:25.297+0000 7fd032000640 1 -- 172.21.15.47:0/2154906376 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd040038050 con 0x55dd0cf73db0 2024-03-21T10:27:25.298 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:25.297+0000 7fd055d22740 10 client.16500 fetch_fsmap learned FSMap version 32 2024-03-21T10:27:25.298 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:25.297+0000 7fd055d22740 1 -- 172.21.15.47:0/2154906376 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x55dd0cba8170 con 0x55dd0cf73db0 2024-03-21T10:27:25.298 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:25.297+0000 7fd032000640 1 -- 172.21.15.47:0/2154906376 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fd040025bf0 con 0x55dd0cf73db0 2024-03-21T10:27:25.299 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:25.298+0000 7fd032000640 1 -- 172.21.15.47:0/2154906376 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 32) v1 ==== 7942+0+0 (secure 0 0 0) 0x7fd040018040 con 0x55dd0cf73db0 2024-03-21T10:27:25.299 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:25.298+0000 7fd055d22740 10 client.16500 fetch_fsmap finished waiting for FSMap version 32 2024-03-21T10:27:25.299 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:25.298+0000 7fd055d22740 10 client.16500 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-21T10:27:25.299 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:25.298+0000 7fd055d22740 20 client.16500 populate_metadata read hostname 'smithi047' 2024-03-21T10:27:25.299 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:25.298+0000 7fd055d22740 1 --2- 172.21.15.47:0/2154906376 >> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] conn(0x55dd0cf6a3d0 0x55dd0cf6c800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:27:25.299 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:25.298+0000 7fd055d22740 4 client.16500 mds_command: new command op to 34535 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T10:27:25.299 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:25.298+0000 7fd055d22740 1 -- 172.21.15.47:0/2154906376 --> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7fd038005050 con 0x55dd0cf6a3d0 2024-03-21T10:27:25.299 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:25.299+0000 7fd04be00640 1 --2- 172.21.15.47:0/2154906376 >> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] conn(0x55dd0cf6a3d0 0x55dd0cf6c800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:27:25.299 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:25.300+0000 7fd04be00640 1 --2- 172.21.15.47:0/2154906376 >> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] conn(0x55dd0cf6a3d0 0x55dd0cf6c800 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=271344b327ba201f server_cookie=4c98ce6b55ce02be in_seq=0 out_seq=0 2024-03-21T10:27:25.300 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:25.300+0000 7fd032000640 10 client.16500 ms_handle_connect on v2:172.21.15.100:6838/4173168440 2024-03-21T10:27:25.301 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:25.301+0000 7fd032000640 1 -- 172.21.15.47:0/2154906376 <== mds.0 v2:172.21.15.100:6838/4173168440 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x7fd038005050 con 0x55dd0cf6a3d0 2024-03-21T10:27:25.301 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:25.301+0000 7fd032000640 10 client.16500 handle_command_reply: tid=0 2024-03-21T10:27:25.301 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:25.301+0000 7fd055d22740 1 client.16500 shutdown 2024-03-21T10:27:25.301 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:25.301+0000 7fd055d22740 1 -- 172.21.15.47:0/2154906376 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55dd0cf73db0 msgr2=0x55dd0cf79750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:27:25.301 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:25.301+0000 7fd055d22740 1 --2- 172.21.15.47:0/2154906376 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55dd0cf73db0 0x55dd0cf79750 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7fd04001ce00 tx=0x7fd0400046c0 comp rx=0 tx=0).stop 2024-03-21T10:27:25.302 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:25.302+0000 7fd055d22740 1 -- 172.21.15.47:0/2154906376 shutdown_connections 2024-03-21T10:27:25.302 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:25.302+0000 7fd055d22740 1 --2- 172.21.15.47:0/2154906376 >> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] conn(0x55dd0cf6a3d0 0x55dd0cf6c800 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:27:25.302 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:25.302+0000 7fd055d22740 1 --2- 172.21.15.47:0/2154906376 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55dd0cf7a1d0 0x55dd0cf88b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:27:25.302 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:25.302+0000 7fd055d22740 1 --2- 172.21.15.47:0/2154906376 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55dd0cf7c990 0x55dd0cf79c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:27:25.302 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:25.302+0000 7fd055d22740 1 --2- 172.21.15.47:0/2154906376 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55dd0cf73db0 0x55dd0cf79750 unknown :-1 s=CLOSED pgs=709 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:27:25.303 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:25.302+0000 7fd055d22740 1 -- 172.21.15.47:0/2154906376 >> 172.21.15.47:0/2154906376 conn(0x55dd0cf6dbc0 msgr2=0x55dd0cf6f910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:27:25.303 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:25.302+0000 7fd032000640 0 client.16500 ms_handle_reset on v2:172.21.15.100:6838/4173168440 2024-03-21T10:27:25.303 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:25.302+0000 7fd055d22740 1 -- 172.21.15.47:0/2154906376 shutdown_connections 2024-03-21T10:27:25.303 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:25.302+0000 7fd055d22740 1 -- 172.21.15.47:0/2154906376 wait complete. 2024-03-21T10:27:25.303 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:25.302+0000 7fd055d22740 20 client.16500 trim_cache size 0 max 16384 2024-03-21T10:27:25.324 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:25.324+0000 7fd055d22740 1 Processor -- start 2024-03-21T10:27:25.324 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:25.324+0000 7fd055d22740 1 -- start start 2024-03-21T10:27:25.325 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:25.324+0000 7fd055d22740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55dd0cf73db0 0x55dd0cf88b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:27:25.325 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:25.324+0000 7fd055d22740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55dd0cf7a1d0 0x55dd0cf749e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:27:25.325 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:25.324+0000 7fd055d22740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55dd0cf7c990 0x55dd0cf74f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:27:25.325 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:25.324+0000 7fd055d22740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd04c1ac610 con 0x55dd0cf73db0 2024-03-21T10:27:25.325 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:25.324+0000 7fd055d22740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd04c1adf90 con 0x55dd0cf7c990 2024-03-21T10:27:25.325 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:25.324+0000 7fd055d22740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd04c1acc70 con 0x55dd0cf7a1d0 2024-03-21T10:27:25.325 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:25.325+0000 7fd050a00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55dd0cf73db0 0x55dd0cf88b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:27:25.325 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:25.325+0000 7fd050a00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55dd0cf73db0 0x55dd0cf88b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:34980/0 (socket says 172.21.15.47:34980) 2024-03-21T10:27:25.325 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:25.325+0000 7fd050a00640 1 -- 172.21.15.47:0/485230105 learned_addr learned my addr 172.21.15.47:0/485230105 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:27:25.325 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:25.325+0000 7fd04be00640 1 --2- 172.21.15.47:0/485230105 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55dd0cf7a1d0 0x55dd0cf749e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:27:25.326 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:25.325+0000 7fd051400640 1 --2- 172.21.15.47:0/485230105 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55dd0cf7c990 0x55dd0cf74f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:27:25.326 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:25.326+0000 7fd050a00640 1 -- 172.21.15.47:0/485230105 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55dd0cf7a1d0 msgr2=0x55dd0cf749e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:27:25.326 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:25.326+0000 7fd050a00640 1 --2- 172.21.15.47:0/485230105 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55dd0cf7a1d0 0x55dd0cf749e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:27:25.326 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:25.326+0000 7fd050a00640 1 -- 172.21.15.47:0/485230105 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55dd0cf7c990 msgr2=0x55dd0cf74f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:27:25.326 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:25.326+0000 7fd050a00640 1 --2- 172.21.15.47:0/485230105 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55dd0cf7c990 0x55dd0cf74f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:27:25.326 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:25.326+0000 7fd050a00640 1 -- 172.21.15.47:0/485230105 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55dd0cf7c7c0 con 0x55dd0cf73db0 2024-03-21T10:27:25.328 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:25.326+0000 7fd050a00640 1 --2- 172.21.15.47:0/485230105 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55dd0cf73db0 0x55dd0cf88b00 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7fd04001c820 tx=0x7fd04001ca60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9a0eadf90b471e8b server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:27:25.328 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:25.328+0000 7fd03d200640 1 -- 172.21.15.47:0/485230105 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd040040070 con 0x55dd0cf73db0 2024-03-21T10:27:25.328 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:25.328+0000 7fd03d200640 1 -- 172.21.15.47:0/485230105 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fd040025d00 con 0x55dd0cf73db0 2024-03-21T10:27:25.329 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:25.328+0000 7fd03d200640 1 -- 172.21.15.47:0/485230105 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd04001a820 con 0x55dd0cf73db0 2024-03-21T10:27:25.329 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:25.329+0000 7fd055d22740 1 -- 172.21.15.47:0/485230105 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55dd0cf73db0 msgr2=0x55dd0cf88b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:27:25.329 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:25.329+0000 7fd055d22740 1 --2- 172.21.15.47:0/485230105 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55dd0cf73db0 0x55dd0cf88b00 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7fd04001c820 tx=0x7fd04001ca60 comp rx=0 tx=0).stop 2024-03-21T10:27:25.329 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:25.329+0000 7fd055d22740 1 -- 172.21.15.47:0/485230105 shutdown_connections 2024-03-21T10:27:25.329 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:25.329+0000 7fd055d22740 1 --2- 172.21.15.47:0/485230105 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55dd0cf7c990 0x55dd0cf74f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:27:25.329 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:25.329+0000 7fd055d22740 1 --2- 172.21.15.47:0/485230105 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55dd0cf7a1d0 0x55dd0cf749e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:27:25.330 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:25.329+0000 7fd055d22740 1 --2- 172.21.15.47:0/485230105 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55dd0cf73db0 0x55dd0cf88b00 unknown :-1 s=CLOSED pgs=710 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:27:25.330 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:25.329+0000 7fd055d22740 1 -- 172.21.15.47:0/485230105 >> 172.21.15.47:0/485230105 conn(0x55dd0cf6dbc0 msgr2=0x55dd0cf742d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:27:25.330 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:25.329+0000 7fd055d22740 1 -- 172.21.15.47:0/485230105 shutdown_connections 2024-03-21T10:27:25.330 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:25.330+0000 7fd055d22740 1 -- 172.21.15.47:0/485230105 wait complete. 2024-03-21T10:27:25.330 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:25.330+0000 7fd055d22740 1 -- start start 2024-03-21T10:27:25.330 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:25.331+0000 7fd055d22740 1 Processor -- start 2024-03-21T10:27:25.332 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:25.332+0000 7fd055d22740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55dd0cf73db0 0x55dd0cf8f0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:27:25.332 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:25.332+0000 7fd055d22740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55dd0cf7a1d0 0x55dd0cf8f600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:27:25.332 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:25.332+0000 7fd055d22740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55dd0cf7c990 0x55dd0cf8fb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:27:25.332 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:25.332+0000 7fd055d22740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd04c1ad930 con 0x55dd0cf7c990 2024-03-21T10:27:25.332 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:25.332+0000 7fd055d22740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd04c1ad2d0 con 0x55dd0cf7a1d0 2024-03-21T10:27:25.332 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:25.332+0000 7fd055d22740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd04c1aa7d0 con 0x55dd0cf73db0 2024-03-21T10:27:25.332 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:25.332+0000 7fd055d22740 10 client.0 _ll_register_callbacks cb 0x7fd04000c630 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T10:27:25.332 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:25.332+0000 7fd04be00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55dd0cf7a1d0 0x55dd0cf8f600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:27:25.332 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:25.332+0000 7fd04be00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55dd0cf7a1d0 0x55dd0cf8f600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:45342/0 (socket says 172.21.15.47:45342) 2024-03-21T10:27:25.332 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:25.332+0000 7fd04be00640 1 -- 172.21.15.47:0/630550326 learned_addr learned my addr 172.21.15.47:0/630550326 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:27:25.334 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:25.332+0000 7fd051400640 1 --2- 172.21.15.47:0/630550326 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55dd0cf7c990 0x55dd0cf8fb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:27:25.334 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:25.332+0000 7fd050a00640 1 --2- 172.21.15.47:0/630550326 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55dd0cf73db0 0x55dd0cf8f0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:27:25.334 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:25.333+0000 7fd04be00640 1 -- 172.21.15.47:0/630550326 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55dd0cf73db0 msgr2=0x55dd0cf8f0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:27:25.335 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:25.333+0000 7fd04be00640 1 --2- 172.21.15.47:0/630550326 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55dd0cf73db0 0x55dd0cf8f0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:27:25.335 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:25.333+0000 7fd04be00640 1 -- 172.21.15.47:0/630550326 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55dd0cf7c990 msgr2=0x55dd0cf8fb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:27:25.335 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:25.333+0000 7fd04be00640 1 --2- 172.21.15.47:0/630550326 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55dd0cf7c990 0x55dd0cf8fb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:27:25.335 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:25.333+0000 7fd04be00640 1 -- 172.21.15.47:0/630550326 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55dd0cf64c20 con 0x55dd0cf7a1d0 2024-03-21T10:27:25.335 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:25.333+0000 7fd051400640 1 --2- 172.21.15.47:0/630550326 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55dd0cf7c990 0x55dd0cf8fb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T10:27:25.335 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:25.333+0000 7fd050a00640 1 --2- 172.21.15.47:0/630550326 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55dd0cf73db0 0x55dd0cf8f0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T10:27:25.335 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:25.333+0000 7fd04be00640 1 --2- 172.21.15.47:0/630550326 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55dd0cf7a1d0 0x55dd0cf8f600 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fd03802c160 tx=0x7fd038005a20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8eaff10d4737f44f server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:27:25.335 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:25.333+0000 7fd032000640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-21T10:27:25.335 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:25.334+0000 7fd032000640 1 -- 172.21.15.47:0/630550326 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd03804a040 con 0x55dd0cf7a1d0 2024-03-21T10:27:25.335 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:25.334+0000 7fd032000640 1 -- 172.21.15.47:0/630550326 <== mon.2 v2:172.21.15.100:3300/0 2 ==== osd_map(97..97 src has 1..97) v4 ==== 9474+0+0 (secure 0 0 0) 0x7fd038053220 con 0x55dd0cf7a1d0 2024-03-21T10:27:25.335 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:25.334+0000 7fd055d22740 1 -- 172.21.15.47:0/630550326 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fd040025d00 con 0x55dd0cf7a1d0 2024-03-21T10:27:25.335 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:25.334+0000 7fd032000640 1 -- 172.21.15.47:0/630550326 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd038061050 con 0x55dd0cf7a1d0 2024-03-21T10:27:25.335 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:25.334+0000 7fd055d22740 10 client.36296 fetch_fsmap learned FSMap version 32 2024-03-21T10:27:25.335 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:25.334+0000 7fd055d22740 1 -- 172.21.15.47:0/630550326 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x55dd0cfe5a70 con 0x55dd0cf7a1d0 2024-03-21T10:27:25.336 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:25.335+0000 7fd032000640 1 -- 172.21.15.47:0/630550326 <== mon.2 v2:172.21.15.100:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fd040025d00 con 0x55dd0cf7a1d0 2024-03-21T10:27:25.336 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:25.335+0000 7fd032000640 1 -- 172.21.15.47:0/630550326 <== mon.2 v2:172.21.15.100:3300/0 5 ==== fsmap(e 32) v1 ==== 7942+0+0 (secure 0 0 0) 0x7fd038040040 con 0x55dd0cf7a1d0 2024-03-21T10:27:25.336 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:25.335+0000 7fd055d22740 10 client.36296 fetch_fsmap finished waiting for FSMap version 32 2024-03-21T10:27:25.336 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:25.335+0000 7fd055d22740 10 client.36296 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-21T10:27:25.336 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:25.335+0000 7fd055d22740 20 client.36296 populate_metadata read hostname 'smithi047' 2024-03-21T10:27:25.337 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:25.335+0000 7fd055d22740 1 --2- 172.21.15.47:0/630550326 >> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] conn(0x55dd0cfe5ca0 0x55dd0cfe8090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:27:25.337 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:25.335+0000 7fd055d22740 4 client.36296 mds_command: new command op to 34535 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-03-21T10:27:25.337 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:25.335+0000 7fd055d22740 1 -- 172.21.15.47:0/630550326 --> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x55dd0cee13f0 con 0x55dd0cfe5ca0 2024-03-21T10:27:25.337 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:25.335+0000 7fd050a00640 1 --2- 172.21.15.47:0/630550326 >> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] conn(0x55dd0cfe5ca0 0x55dd0cfe8090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:27:25.337 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:25.336+0000 7fd032000640 10 client.36296 ms_handle_connect on v2:172.21.15.100:6838/4173168440 2024-03-21T10:27:25.337 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:25.336+0000 7fd050a00640 1 --2- 172.21.15.47:0/630550326 >> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] conn(0x55dd0cfe5ca0 0x55dd0cfe8090 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f73eb4850e840d31 server_cookie=f25f9182f719e39e in_seq=0 out_seq=0 2024-03-21T10:27:25.337 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:25.338+0000 7fd032000640 1 -- 172.21.15.47:0/630550326 <== mds.0 v2:172.21.15.100:6838/4173168440 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x55dd0cee13f0 con 0x55dd0cfe5ca0 2024-03-21T10:27:25.338 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:25.338+0000 7fd032000640 10 client.36296 handle_command_reply: tid=0 2024-03-21T10:27:25.338 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:25.338+0000 7fd055d22740 1 client.36296 shutdown 2024-03-21T10:27:25.338 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:25.338+0000 7fd055d22740 1 -- 172.21.15.47:0/630550326 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55dd0cf7a1d0 msgr2=0x55dd0cf8f600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:27:25.338 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:25.338+0000 7fd055d22740 1 --2- 172.21.15.47:0/630550326 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55dd0cf7a1d0 0x55dd0cf8f600 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fd03802c160 tx=0x7fd038005a20 comp rx=0 tx=0).stop 2024-03-21T10:27:25.339 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:25.338+0000 7fd055d22740 1 -- 172.21.15.47:0/630550326 shutdown_connections 2024-03-21T10:27:25.339 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:25.338+0000 7fd055d22740 1 --2- 172.21.15.47:0/630550326 >> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] conn(0x55dd0cfe5ca0 0x55dd0cfe8090 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:27:25.339 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:25.338+0000 7fd055d22740 1 --2- 172.21.15.47:0/630550326 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55dd0cf7c990 0x55dd0cf8fb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:27:25.339 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:25.338+0000 7fd055d22740 1 --2- 172.21.15.47:0/630550326 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55dd0cf7a1d0 0x55dd0cf8f600 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:27:25.339 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:25.338+0000 7fd055d22740 1 --2- 172.21.15.47:0/630550326 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55dd0cf73db0 0x55dd0cf8f0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:27:25.339 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:25.338+0000 7fd055d22740 1 -- 172.21.15.47:0/630550326 >> 172.21.15.47:0/630550326 conn(0x55dd0cf6dbc0 msgr2=0x55dd0cf74b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:27:25.339 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:25.338+0000 7fd032000640 0 client.36296 ms_handle_reset on v2:172.21.15.100:6838/4173168440 2024-03-21T10:27:25.339 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:25.339+0000 7fd055d22740 1 -- 172.21.15.47:0/630550326 shutdown_connections 2024-03-21T10:27:25.339 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:25.339+0000 7fd055d22740 1 -- 172.21.15.47:0/630550326 wait complete. 2024-03-21T10:27:25.339 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:25.339+0000 7fd055d22740 20 client.36296 trim_cache size 0 max 16384 2024-03-21T10:27:25.340 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-03-21T10:27:25.340 INFO:teuthology.orchestra.run.smithi047.stdout: "return_code": 0, 2024-03-21T10:27:25.340 INFO:teuthology.orchestra.run.smithi047.stdout: "scrub_tag": "72616ced-3c4d-450c-8c3e-77e93553bb86", 2024-03-21T10:27:25.340 INFO:teuthology.orchestra.run.smithi047.stdout: "mode": "asynchronous" 2024-03-21T10:27:25.340 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-03-21T10:27:25.341 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:25.340+0000 7fd053400640 1 -- 172.21.15.47:0/4278473528 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7fd0340a3090 msgr2=0x7fd0340a5550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:27:25.341 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:25.340+0000 7fd053400640 1 --2- 172.21.15.47:0/4278473528 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7fd0340a3090 0x7fd0340a5550 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x55dd0cf72f60 tx=0x7fd04000c430 comp rx=0 tx=0).stop 2024-03-21T10:27:25.341 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:25.340+0000 7fd053400640 1 -- 172.21.15.47:0/4278473528 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd04c1076b0 msgr2=0x7fd04c19c6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:27:25.341 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:25.340+0000 7fd053400640 1 --2- 172.21.15.47:0/4278473528 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd04c1076b0 0x7fd04c19c6e0 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7fd04400eb10 tx=0x7fd04400c2d0 comp rx=0 tx=0).stop 2024-03-21T10:27:25.341 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:25.340+0000 7fd053400640 1 -- 172.21.15.47:0/4278473528 shutdown_connections 2024-03-21T10:27:25.341 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:25.340+0000 7fd053400640 1 --2- 172.21.15.47:0/4278473528 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd04c1076b0 0x7fd04c19c6e0 unknown :-1 s=CLOSED pgs=707 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:27:25.341 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:25.340+0000 7fd053400640 1 --2- 172.21.15.47:0/4278473528 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7fd0340a3090 0x7fd0340a5550 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:27:25.341 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:25.340+0000 7fd053400640 1 --2- 172.21.15.47:0/4278473528 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd04c069240 0x7fd04c1983b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:27:25.341 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:25.340+0000 7fd053400640 1 --2- 172.21.15.47:0/4278473528 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd04c068880 0x7fd04c197e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:27:25.341 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:25.340+0000 7fd053400640 1 -- 172.21.15.47:0/4278473528 >> 172.21.15.47:0/4278473528 conn(0x7fd04c0776a0 msgr2=0x7fd04c0ff160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:27:25.341 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:25.340+0000 7fd053400640 1 -- 172.21.15.47:0/4278473528 shutdown_connections 2024-03-21T10:27:25.341 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:25.340+0000 7fd053400640 1 -- 172.21.15.47:0/4278473528 wait complete. 2024-03-21T10:27:25.361 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-21T10:27:25.361 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-21T10:27:25.587 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:25.586+0000 7f195f200640 1 -- 172.21.15.47:0/3808731779 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f19601075e0 msgr2=0x7f19601079c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:27:25.587 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:25.586+0000 7f195f200640 1 --2- 172.21.15.47:0/3808731779 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f19601075e0 0x7f19601079c0 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7f1950009900 tx=0x7f195002bc30 comp rx=0 tx=0).stop 2024-03-21T10:27:25.587 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:25.587+0000 7f195f200640 1 -- 172.21.15.47:0/3808731779 shutdown_connections 2024-03-21T10:27:25.587 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:25.587+0000 7f195f200640 1 --2- 172.21.15.47:0/3808731779 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1960101ee0 0x7f196010e3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:27:25.587 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:25.587+0000 7f195f200640 1 --2- 172.21.15.47:0/3808731779 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1960101520 0x7f19601019a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:27:25.587 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:25.587+0000 7f195f200640 1 --2- 172.21.15.47:0/3808731779 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f19601075e0 0x7f19601079c0 unknown :-1 s=CLOSED pgs=711 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:27:25.587 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:25.587+0000 7f195f200640 1 -- 172.21.15.47:0/3808731779 >> 172.21.15.47:0/3808731779 conn(0x7f19600fd010 msgr2=0x7f19600ff430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:27:25.587 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:25.587+0000 7f195f200640 1 -- 172.21.15.47:0/3808731779 shutdown_connections 2024-03-21T10:27:25.587 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:25.587+0000 7f195f200640 1 -- 172.21.15.47:0/3808731779 wait complete. 2024-03-21T10:27:25.589 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:25.588+0000 7f195f200640 1 Processor -- start 2024-03-21T10:27:25.589 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:25.588+0000 7f195f200640 1 -- start start 2024-03-21T10:27:25.589 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:25.588+0000 7f195f200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1960101520 0x7f196019d910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:27:25.589 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:25.589+0000 7f195f200640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1960101ee0 0x7f196019de50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:27:25.589 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:25.589+0000 7f195f200640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f19601075e0 0x7f1960197ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:27:25.589 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:25.589+0000 7f195f200640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19601117c0 con 0x7f19601075e0 2024-03-21T10:27:25.589 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:25.589+0000 7f195f200640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1960111640 con 0x7f1960101520 2024-03-21T10:27:25.589 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:25.589+0000 7f195f200640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1960111940 con 0x7f1960101ee0 2024-03-21T10:27:25.589 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:25.589+0000 7f195de00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1960101520 0x7f196019d910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:27:25.589 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:25.589+0000 7f195de00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1960101520 0x7f196019d910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:45372/0 (socket says 172.21.15.47:45372) 2024-03-21T10:27:25.590 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:25.589+0000 7f195de00640 1 -- 172.21.15.47:0/4253981533 learned_addr learned my addr 172.21.15.47:0/4253981533 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:27:25.590 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:25.589+0000 7f195e800640 1 --2- 172.21.15.47:0/4253981533 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f19601075e0 0x7f1960197ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:27:25.590 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:25.589+0000 7f195d400640 1 --2- 172.21.15.47:0/4253981533 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1960101ee0 0x7f196019de50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:27:25.591 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:25.589+0000 7f195de00640 1 -- 172.21.15.47:0/4253981533 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1960101ee0 msgr2=0x7f196019de50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:27:25.591 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:25.589+0000 7f195de00640 1 --2- 172.21.15.47:0/4253981533 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1960101ee0 0x7f196019de50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:27:25.591 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:25.589+0000 7f195de00640 1 -- 172.21.15.47:0/4253981533 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f19601075e0 msgr2=0x7f1960197ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:27:25.591 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:25.589+0000 7f195de00640 1 --2- 172.21.15.47:0/4253981533 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f19601075e0 0x7f1960197ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:27:25.591 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:25.589+0000 7f195de00640 1 -- 172.21.15.47:0/4253981533 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1960198240 con 0x7f1960101520 2024-03-21T10:27:25.591 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:25.590+0000 7f195de00640 1 --2- 172.21.15.47:0/4253981533 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1960101520 0x7f196019d910 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f1950000c00 tx=0x7f19500045e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:27:25.591 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:25.590+0000 7f1956a00640 1 -- 172.21.15.47:0/4253981533 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1950043020 con 0x7f1960101520 2024-03-21T10:27:25.591 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:25.590+0000 7f1956a00640 1 -- 172.21.15.47:0/4253981533 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f195002cb10 con 0x7f1960101520 2024-03-21T10:27:25.591 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:25.590+0000 7f1956a00640 1 -- 172.21.15.47:0/4253981533 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1950003a80 con 0x7f1960101520 2024-03-21T10:27:25.591 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:25.590+0000 7f195f200640 1 -- 172.21.15.47:0/4253981533 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f19601984d0 con 0x7f1960101520 2024-03-21T10:27:25.591 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:25.591+0000 7f195f200640 1 -- 172.21.15.47:0/4253981533 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f19601a47d0 con 0x7f1960101520 2024-03-21T10:27:25.594 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:25.594+0000 7f1956a00640 1 -- 172.21.15.47:0/4253981533 <== mon.2 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 27) v1 ==== 137985+0+0 (secure 0 0 0) 0x7f1950046020 con 0x7f1960101520 2024-03-21T10:27:25.594 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:25.594+0000 7f196778c740 1 Processor -- start 2024-03-21T10:27:25.595 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:25.595+0000 7f196778c740 1 -- start start 2024-03-21T10:27:25.595 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:25.595+0000 7f196778c740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560b3c8c6dd0 0x560b3c8c5b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:27:25.595 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:25.595+0000 7f196778c740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560b3c8c60f0 0x560b3c8c6570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:27:25.595 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:25.595+0000 7f196778c740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x560b3c8cf460 0x560b3c8d1850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:27:25.595 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:25.595+0000 7f196778c740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x560b3c82a250 con 0x560b3c8cf460 2024-03-21T10:27:25.596 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:25.595+0000 7f196778c740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x560b3c7d2a30 con 0x560b3c8c60f0 2024-03-21T10:27:25.596 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:25.595+0000 7f196778c740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x560b3c8262f0 con 0x560b3c8c6dd0 2024-03-21T10:27:25.596 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:25.595+0000 7f195d400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x560b3c8cf460 0x560b3c8d1850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:27:25.596 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:25.595+0000 7f195d400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x560b3c8cf460 0x560b3c8d1850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:35022/0 (socket says 172.21.15.47:35022) 2024-03-21T10:27:25.596 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:25.595+0000 7f195d400640 1 -- 172.21.15.47:0/424055146 learned_addr learned my addr 172.21.15.47:0/424055146 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:27:25.596 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:25.596+0000 7f195de00640 1 --2- 172.21.15.47:0/424055146 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560b3c8c60f0 0x560b3c8c6570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:27:25.597 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:25.596+0000 7f195e800640 1 --2- 172.21.15.47:0/424055146 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560b3c8c6dd0 0x560b3c8c5b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:27:25.597 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:25.596+0000 7f195d400640 1 -- 172.21.15.47:0/424055146 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560b3c8c6dd0 msgr2=0x560b3c8c5b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:27:25.597 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:25.596+0000 7f195d400640 1 --2- 172.21.15.47:0/424055146 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560b3c8c6dd0 0x560b3c8c5b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:27:25.597 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:25.596+0000 7f195d400640 1 -- 172.21.15.47:0/424055146 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560b3c8c60f0 msgr2=0x560b3c8c6570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:27:25.597 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:25.596+0000 7f195d400640 1 --2- 172.21.15.47:0/424055146 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560b3c8c60f0 0x560b3c8c6570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:27:25.597 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:25.596+0000 7f195d400640 1 -- 172.21.15.47:0/424055146 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560b3c7bd690 con 0x560b3c8cf460 2024-03-21T10:27:25.597 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:25.596+0000 7f195d400640 1 --2- 172.21.15.47:0/424055146 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x560b3c8cf460 0x560b3c8d1850 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7f194c00bc50 tx=0x7f194c00f730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fafbe4f572fda30c server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:27:25.598 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:25.597+0000 7f194b000640 1 -- 172.21.15.47:0/424055146 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f194c015070 con 0x560b3c8cf460 2024-03-21T10:27:25.598 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:25.597+0000 7f1956a00640 1 --2- 172.21.15.47:0/4253981533 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f19440a3090 0x7f19440a5550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:27:25.598 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:25.597+0000 7f194b000640 1 -- 172.21.15.47:0/424055146 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f194c01a910 con 0x560b3c8cf460 2024-03-21T10:27:25.598 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:25.597+0000 7f194b000640 1 -- 172.21.15.47:0/424055146 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f194c011920 con 0x560b3c8cf460 2024-03-21T10:27:25.598 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:25.597+0000 7f1956a00640 1 -- 172.21.15.47:0/4253981533 <== mon.2 v2:172.21.15.100:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 9474+0+0 (secure 0 0 0) 0x7f19500f1e10 con 0x7f1960101520 2024-03-21T10:27:25.598 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:25.597+0000 7f195e800640 1 --2- 172.21.15.47:0/4253981533 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f19440a3090 0x7f19440a5550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:27:25.598 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:25.598+0000 7f196778c740 1 -- 172.21.15.47:0/424055146 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x560b3c8cf460 msgr2=0x560b3c8d1850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:27:25.598 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:25.598+0000 7f196778c740 1 --2- 172.21.15.47:0/424055146 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x560b3c8cf460 0x560b3c8d1850 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7f194c00bc50 tx=0x7f194c00f730 comp rx=0 tx=0).stop 2024-03-21T10:27:25.598 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:25.598+0000 7f195e800640 1 --2- 172.21.15.47:0/4253981533 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f19440a3090 0x7f19440a5550 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f1958005fd0 tx=0x7f1958015040 comp rx=0 tx=0).ready entity=mgr.15687 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:27:25.599 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:25.598+0000 7f196778c740 1 -- 172.21.15.47:0/424055146 shutdown_connections 2024-03-21T10:27:25.599 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:25.598+0000 7f196778c740 1 --2- 172.21.15.47:0/424055146 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x560b3c8cf460 0x560b3c8d1850 secure :-1 s=CLOSED pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7f194c00bc50 tx=0x7f194c00f730 comp rx=0 tx=0).stop 2024-03-21T10:27:25.599 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:25.598+0000 7f196778c740 1 --2- 172.21.15.47:0/424055146 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560b3c8c60f0 0x560b3c8c6570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:27:25.599 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:25.598+0000 7f196778c740 1 --2- 172.21.15.47:0/424055146 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560b3c8c6dd0 0x560b3c8c5b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:27:25.599 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:25.598+0000 7f196778c740 1 -- 172.21.15.47:0/424055146 >> 172.21.15.47:0/424055146 conn(0x560b3c88d200 msgr2=0x560b3c8c0b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:27:25.599 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:25.599+0000 7f196778c740 1 -- 172.21.15.47:0/424055146 shutdown_connections 2024-03-21T10:27:25.600 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:25.599+0000 7f196778c740 1 -- 172.21.15.47:0/424055146 wait complete. 2024-03-21T10:27:25.600 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:25.599+0000 7f196778c740 1 -- start start 2024-03-21T10:27:25.600 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:25.600+0000 7f196778c740 1 Processor -- start 2024-03-21T10:27:25.601 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:25.600+0000 7f196778c740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560b3c8c60f0 0x560b3c8ced90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:27:25.601 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:25.600+0000 7f196778c740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560b3c8c6dd0 0x560b3c8cc080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:27:25.601 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:25.600+0000 7f196778c740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x560b3c8cc610 0x560b3c8cca70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:27:25.601 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:25.600+0000 7f196778c740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x560b3c8d22d0 con 0x560b3c8cc610 2024-03-21T10:27:25.601 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:25.600+0000 7f196778c740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x560b3c7e5e80 con 0x560b3c8c6dd0 2024-03-21T10:27:25.601 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:25.600+0000 7f196778c740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x560b3c8d2450 con 0x560b3c8c60f0 2024-03-21T10:27:25.601 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:25.601+0000 7f196778c740 10 client.0 _ll_register_callbacks cb 0x560b3c7f8130 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T10:27:25.602 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:25.601+0000 7f195d400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560b3c8c6dd0 0x560b3c8cc080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:27:25.602 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:25.601+0000 7f195d400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560b3c8c6dd0 0x560b3c8cc080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:45394/0 (socket says 172.21.15.47:45394) 2024-03-21T10:27:25.602 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:25.601+0000 7f195d400640 1 -- 172.21.15.47:0/904829268 learned_addr learned my addr 172.21.15.47:0/904829268 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:27:25.602 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:25.601+0000 7f195e800640 1 --2- 172.21.15.47:0/904829268 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x560b3c8cc610 0x560b3c8cca70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:27:25.602 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:25.601+0000 7f195de00640 1 --2- 172.21.15.47:0/904829268 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560b3c8c60f0 0x560b3c8ced90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:27:25.602 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:25.601+0000 7f195d400640 1 -- 172.21.15.47:0/904829268 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560b3c8c60f0 msgr2=0x560b3c8ced90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:27:25.602 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:25.601+0000 7f195d400640 1 --2- 172.21.15.47:0/904829268 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560b3c8c60f0 0x560b3c8ced90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:27:25.602 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:25.601+0000 7f195d400640 1 -- 172.21.15.47:0/904829268 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x560b3c8cc610 msgr2=0x560b3c8cca70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:27:25.602 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:25.601+0000 7f195d400640 1 --2- 172.21.15.47:0/904829268 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x560b3c8cc610 0x560b3c8cca70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:27:25.602 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:25.601+0000 7f195d400640 1 -- 172.21.15.47:0/904829268 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x560b3c6087d0 con 0x560b3c8c6dd0 2024-03-21T10:27:25.603 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:25.602+0000 7f195d400640 1 --2- 172.21.15.47:0/904829268 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560b3c8c6dd0 0x560b3c8cc080 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f194c00bf70 tx=0x7f194c00bfa0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1e9f2cfa4e0790f9 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:27:25.604 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:25.602+0000 7f193f400640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-21T10:27:25.604 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:25.602+0000 7f193f400640 1 -- 172.21.15.47:0/904829268 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f194c028060 con 0x560b3c8c6dd0 2024-03-21T10:27:25.604 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:25.602+0000 7f196778c740 1 -- 172.21.15.47:0/904829268 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f194c01a910 con 0x560b3c8c6dd0 2024-03-21T10:27:25.604 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:25.602+0000 7f193f400640 1 -- 172.21.15.47:0/904829268 <== mon.2 v2:172.21.15.100:3300/0 2 ==== osd_map(97..97 src has 1..97) v4 ==== 9474+0+0 (secure 0 0 0) 0x7f194c01aac0 con 0x560b3c8c6dd0 2024-03-21T10:27:25.604 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:25.603+0000 7f193f400640 1 -- 172.21.15.47:0/904829268 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f194c02f050 con 0x560b3c8c6dd0 2024-03-21T10:27:25.604 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:25.603+0000 7f196778c740 10 client.36308 fetch_fsmap learned FSMap version 32 2024-03-21T10:27:25.604 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:25.603+0000 7f196778c740 1 -- 172.21.15.47:0/904829268 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x560b3c53e3f0 con 0x560b3c8c6dd0 2024-03-21T10:27:25.604 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:25.603+0000 7f193f400640 1 -- 172.21.15.47:0/904829268 <== mon.2 v2:172.21.15.100:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f194c01a910 con 0x560b3c8c6dd0 2024-03-21T10:27:25.604 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:25.603+0000 7f193f400640 1 -- 172.21.15.47:0/904829268 <== mon.2 v2:172.21.15.100:3300/0 5 ==== fsmap(e 32) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f194c022030 con 0x560b3c8c6dd0 2024-03-21T10:27:25.604 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:25.603+0000 7f196778c740 10 client.36308 fetch_fsmap finished waiting for FSMap version 32 2024-03-21T10:27:25.604 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:25.603+0000 7f196778c740 10 client.36308 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-21T10:27:25.604 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:25.603+0000 7f196778c740 20 client.36308 populate_metadata read hostname 'smithi047' 2024-03-21T10:27:25.604 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:25.603+0000 7f196778c740 1 --2- 172.21.15.47:0/904829268 >> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] conn(0x560b3c8df850 0x560b3c8e1c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:27:25.605 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:25.603+0000 7f196778c740 4 client.36308 mds_command: new command op to 34535 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T10:27:25.605 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:25.603+0000 7f196778c740 1 -- 172.21.15.47:0/904829268 --> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x560b3c837790 con 0x560b3c8df850 2024-03-21T10:27:25.605 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:25.604+0000 7f195de00640 1 --2- 172.21.15.47:0/904829268 >> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] conn(0x560b3c8df850 0x560b3c8e1c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:27:25.605 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:25.604+0000 7f195de00640 1 --2- 172.21.15.47:0/904829268 >> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] conn(0x560b3c8df850 0x560b3c8e1c40 crc :-1 s=READY pgs=93 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9b729ab0944d80c server_cookie=98810bc49e559dec in_seq=0 out_seq=0 2024-03-21T10:27:25.606 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:25.604+0000 7f193f400640 10 client.36308 ms_handle_connect on v2:172.21.15.100:6838/4173168440 2024-03-21T10:27:25.606 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:25.605+0000 7f193f400640 1 -- 172.21.15.47:0/904829268 <== mds.0 v2:172.21.15.100:6838/4173168440 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x560b3c837790 con 0x560b3c8df850 2024-03-21T10:27:25.606 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:25.605+0000 7f193f400640 10 client.36308 handle_command_reply: tid=0 2024-03-21T10:27:25.606 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:25.605+0000 7f196778c740 1 client.36308 shutdown 2024-03-21T10:27:25.606 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:25.606+0000 7f196778c740 1 -- 172.21.15.47:0/904829268 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560b3c8c6dd0 msgr2=0x560b3c8cc080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:27:25.606 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:25.606+0000 7f196778c740 1 --2- 172.21.15.47:0/904829268 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560b3c8c6dd0 0x560b3c8cc080 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f194c00bf70 tx=0x7f194c00bfa0 comp rx=0 tx=0).stop 2024-03-21T10:27:25.606 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:25.606+0000 7f196778c740 1 -- 172.21.15.47:0/904829268 shutdown_connections 2024-03-21T10:27:25.606 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:25.606+0000 7f196778c740 1 --2- 172.21.15.47:0/904829268 >> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] conn(0x560b3c8df850 0x560b3c8e1c40 crc :-1 s=READY pgs=93 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:27:25.606 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:25.606+0000 7f196778c740 1 --2- 172.21.15.47:0/904829268 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x560b3c8cc610 0x560b3c8cca70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:27:25.606 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:25.606+0000 7f196778c740 1 --2- 172.21.15.47:0/904829268 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560b3c8c6dd0 0x560b3c8cc080 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:27:25.606 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:25.606+0000 7f193f400640 0 client.36308 ms_handle_reset on v2:172.21.15.100:6838/4173168440 2024-03-21T10:27:25.606 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:25.606+0000 7f196778c740 1 --2- 172.21.15.47:0/904829268 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560b3c8c60f0 0x560b3c8ced90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:27:25.606 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:25.606+0000 7f196778c740 1 -- 172.21.15.47:0/904829268 >> 172.21.15.47:0/904829268 conn(0x560b3c88d200 msgr2=0x560b3c8c2890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:27:25.607 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:25.606+0000 7f196778c740 1 -- 172.21.15.47:0/904829268 shutdown_connections 2024-03-21T10:27:25.607 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:25.606+0000 7f196778c740 1 -- 172.21.15.47:0/904829268 wait complete. 2024-03-21T10:27:25.607 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:25.606+0000 7f196778c740 20 client.36308 trim_cache size 0 max 16384 2024-03-21T10:27:25.630 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:25.628+0000 7f196778c740 1 Processor -- start 2024-03-21T10:27:25.630 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:25.628+0000 7f196778c740 1 -- start start 2024-03-21T10:27:25.630 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:25.629+0000 7f196778c740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560b3c8c60f0 0x560b3c8c0670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:27:25.630 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:25.629+0000 7f196778c740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560b3c8c6dd0 0x560b3c8c26a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:27:25.630 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:25.629+0000 7f196778c740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x560b3c8cc610 0x560b3c8ce8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:27:25.630 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:25.629+0000 7f196778c740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19601ac3c0 con 0x560b3c8cc610 2024-03-21T10:27:25.631 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:25.629+0000 7f196778c740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19601add40 con 0x560b3c8c60f0 2024-03-21T10:27:25.631 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:25.629+0000 7f196778c740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19601aca20 con 0x560b3c8c6dd0 2024-03-21T10:27:25.631 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:25.629+0000 7f195e800640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x560b3c8cc610 0x560b3c8ce8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:27:25.631 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:25.629+0000 7f195e800640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x560b3c8cc610 0x560b3c8ce8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:35040/0 (socket says 172.21.15.47:35040) 2024-03-21T10:27:25.631 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:25.629+0000 7f195e800640 1 -- 172.21.15.47:0/88292519 learned_addr learned my addr 172.21.15.47:0/88292519 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:27:25.631 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:25.629+0000 7f195de00640 1 --2- 172.21.15.47:0/88292519 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560b3c8c60f0 0x560b3c8c0670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:27:25.631 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:25.629+0000 7f195d400640 1 --2- 172.21.15.47:0/88292519 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560b3c8c6dd0 0x560b3c8c26a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:27:25.631 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:25.630+0000 7f195e800640 1 -- 172.21.15.47:0/88292519 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560b3c8c6dd0 msgr2=0x560b3c8c26a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:27:25.631 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:25.630+0000 7f195e800640 1 --2- 172.21.15.47:0/88292519 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560b3c8c6dd0 0x560b3c8c26a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:27:25.631 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:25.630+0000 7f195e800640 1 -- 172.21.15.47:0/88292519 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560b3c8c60f0 msgr2=0x560b3c8c0670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:27:25.632 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:25.630+0000 7f195e800640 1 --2- 172.21.15.47:0/88292519 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560b3c8c60f0 0x560b3c8c0670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:27:25.632 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:25.630+0000 7f195e800640 1 -- 172.21.15.47:0/88292519 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560b3c8cb030 con 0x560b3c8cc610 2024-03-21T10:27:25.632 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:25.630+0000 7f195de00640 1 --2- 172.21.15.47:0/88292519 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560b3c8c60f0 0x560b3c8c0670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T10:27:25.635 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:25.630+0000 7f195e800640 1 --2- 172.21.15.47:0/88292519 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x560b3c8cc610 0x560b3c8ce8c0 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7f1958013d80 tx=0x7f1958017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=33683920315c9b47 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:27:25.635 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:25.631+0000 7f194a600640 1 -- 172.21.15.47:0/88292519 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1958011e00 con 0x560b3c8cc610 2024-03-21T10:27:25.635 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:25.631+0000 7f194a600640 1 -- 172.21.15.47:0/88292519 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f195802e070 con 0x560b3c8cc610 2024-03-21T10:27:25.635 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:25.631+0000 7f194a600640 1 -- 172.21.15.47:0/88292519 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1958029c90 con 0x560b3c8cc610 2024-03-21T10:27:25.635 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:25.631+0000 7f196778c740 1 -- 172.21.15.47:0/88292519 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x560b3c8cc610 msgr2=0x560b3c8ce8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:27:25.636 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:25.631+0000 7f196778c740 1 --2- 172.21.15.47:0/88292519 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x560b3c8cc610 0x560b3c8ce8c0 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7f1958013d80 tx=0x7f1958017000 comp rx=0 tx=0).stop 2024-03-21T10:27:25.636 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:25.632+0000 7f196778c740 1 -- 172.21.15.47:0/88292519 shutdown_connections 2024-03-21T10:27:25.636 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:25.632+0000 7f196778c740 1 --2- 172.21.15.47:0/88292519 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x560b3c8cc610 0x560b3c8ce8c0 unknown :-1 s=CLOSED pgs=713 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:27:25.636 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:25.632+0000 7f196778c740 1 --2- 172.21.15.47:0/88292519 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560b3c8c6dd0 0x560b3c8c26a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:27:25.636 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:25.632+0000 7f196778c740 1 --2- 172.21.15.47:0/88292519 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560b3c8c60f0 0x560b3c8c0670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:27:25.636 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:25.632+0000 7f196778c740 1 -- 172.21.15.47:0/88292519 >> 172.21.15.47:0/88292519 conn(0x560b3c88d200 msgr2=0x560b3c8c1f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:27:25.636 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:25.632+0000 7f196778c740 1 -- 172.21.15.47:0/88292519 shutdown_connections 2024-03-21T10:27:25.636 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:25.632+0000 7f196778c740 1 -- 172.21.15.47:0/88292519 wait complete. 2024-03-21T10:27:25.636 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:25.632+0000 7f196778c740 1 -- start start 2024-03-21T10:27:25.636 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:25.632+0000 7f196778c740 1 Processor -- start 2024-03-21T10:27:25.636 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:25.633+0000 7f196778c740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560b3c8c60f0 0x560b3c8d46b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:27:25.637 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:25.633+0000 7f196778c740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560b3c8c6dd0 0x560b3c8d4bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:27:25.637 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:25.633+0000 7f196778c740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x560b3c8cc610 0x560b3c8d5130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:27:25.637 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:25.633+0000 7f196778c740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19601ad6e0 con 0x560b3c8cc610 2024-03-21T10:27:25.637 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:25.633+0000 7f196778c740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19601ad080 con 0x560b3c8c6dd0 2024-03-21T10:27:25.637 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:25.633+0000 7f196778c740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19601aa580 con 0x560b3c8c60f0 2024-03-21T10:27:25.637 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:25.633+0000 7f196778c740 10 client.0 _ll_register_callbacks cb 0x7f19500f5790 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T10:27:25.637 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:25.633+0000 7f195d400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560b3c8c6dd0 0x560b3c8d4bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:27:25.637 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:25.633+0000 7f195e800640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x560b3c8cc610 0x560b3c8d5130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:27:25.637 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:25.633+0000 7f195d400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560b3c8c6dd0 0x560b3c8d4bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:45408/0 (socket says 172.21.15.47:45408) 2024-03-21T10:27:25.637 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:25.634+0000 7f195d400640 1 -- 172.21.15.47:0/3624805485 learned_addr learned my addr 172.21.15.47:0/3624805485 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:27:25.637 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:25.634+0000 7f195de00640 1 --2- 172.21.15.47:0/3624805485 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560b3c8c60f0 0x560b3c8d46b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:27:25.637 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:25.634+0000 7f195e800640 1 -- 172.21.15.47:0/3624805485 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560b3c8c60f0 msgr2=0x560b3c8d46b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:27:25.638 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:25.634+0000 7f195e800640 1 --2- 172.21.15.47:0/3624805485 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560b3c8c60f0 0x560b3c8d46b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:27:25.638 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:25.634+0000 7f195e800640 1 -- 172.21.15.47:0/3624805485 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560b3c8c6dd0 msgr2=0x560b3c8d4bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:27:25.638 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:25.634+0000 7f195e800640 1 --2- 172.21.15.47:0/3624805485 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560b3c8c6dd0 0x560b3c8d4bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:27:25.638 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:25.634+0000 7f195e800640 1 -- 172.21.15.47:0/3624805485 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x560b3c4fe0f0 con 0x560b3c8cc610 2024-03-21T10:27:25.638 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:25.634+0000 7f195e800640 1 --2- 172.21.15.47:0/3624805485 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x560b3c8cc610 0x560b3c8d5130 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7f1958029850 tx=0x7f1958004c70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e81a49008af81bdc server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:27:25.638 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:25.634+0000 7f193f400640 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-03-21T10:27:25.638 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:25.635+0000 7f193f400640 1 -- 172.21.15.47:0/3624805485 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1958036070 con 0x560b3c8cc610 2024-03-21T10:27:25.638 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:25.635+0000 7f193f400640 1 -- 172.21.15.47:0/3624805485 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(97..97 src has 1..97) v4 ==== 9474+0+0 (secure 0 0 0) 0x7f1958029da0 con 0x560b3c8cc610 2024-03-21T10:27:25.638 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:25.635+0000 7f196778c740 1 -- 172.21.15.47:0/3624805485 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f195802e070 con 0x560b3c8cc610 2024-03-21T10:27:25.638 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:25.635+0000 7f193f400640 1 -- 172.21.15.47:0/3624805485 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1958011e70 con 0x560b3c8cc610 2024-03-21T10:27:25.638 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:25.635+0000 7f196778c740 10 client.16524 fetch_fsmap learned FSMap version 32 2024-03-21T10:27:25.638 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:25.635+0000 7f196778c740 1 -- 172.21.15.47:0/3624805485 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x560b3c93be20 con 0x560b3c8cc610 2024-03-21T10:27:25.638 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:25.635+0000 7f193f400640 1 -- 172.21.15.47:0/3624805485 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f195802e070 con 0x560b3c8cc610 2024-03-21T10:27:25.640 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:25.636+0000 7f193f400640 1 -- 172.21.15.47:0/3624805485 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 32) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f195801c070 con 0x560b3c8cc610 2024-03-21T10:27:25.640 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:25.636+0000 7f196778c740 10 client.16524 fetch_fsmap finished waiting for FSMap version 32 2024-03-21T10:27:25.641 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:25.636+0000 7f196778c740 10 client.16524 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-21T10:27:25.641 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:25.636+0000 7f196778c740 20 client.16524 populate_metadata read hostname 'smithi047' 2024-03-21T10:27:25.641 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:25.636+0000 7f196778c740 1 --2- 172.21.15.47:0/3624805485 >> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] conn(0x560b3c8df850 0x560b3c93e4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:27:25.641 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:25.636+0000 7f196778c740 4 client.16524 mds_command: new command op to 34535 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-21T10:27:25.641 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:25.636+0000 7f196778c740 1 -- 172.21.15.47:0/3624805485 --> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x560b3c831ae0 con 0x560b3c8df850 2024-03-21T10:27:25.641 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:25.636+0000 7f195de00640 1 --2- 172.21.15.47:0/3624805485 >> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] conn(0x560b3c8df850 0x560b3c93e4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:27:25.641 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:25.637+0000 7f195de00640 1 --2- 172.21.15.47:0/3624805485 >> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] conn(0x560b3c8df850 0x560b3c93e4b0 crc :-1 s=READY pgs=94 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=be34521541238b02 server_cookie=8dcd23a0f68bd82 in_seq=0 out_seq=0 2024-03-21T10:27:25.642 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:25.637+0000 7f193f400640 10 client.16524 ms_handle_connect on v2:172.21.15.100:6838/4173168440 2024-03-21T10:27:25.642 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:25.637+0000 7f193f400640 1 -- 172.21.15.47:0/3624805485 <== mds.0 v2:172.21.15.100:6838/4173168440 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x560b3c831ae0 con 0x560b3c8df850 2024-03-21T10:27:25.642 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:25.637+0000 7f193f400640 10 client.16524 handle_command_reply: tid=0 2024-03-21T10:27:25.642 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:25.637+0000 7f196778c740 1 client.16524 shutdown 2024-03-21T10:27:25.642 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:25.638+0000 7f196778c740 1 -- 172.21.15.47:0/3624805485 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x560b3c8cc610 msgr2=0x560b3c8d5130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:27:25.642 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:25.638+0000 7f196778c740 1 --2- 172.21.15.47:0/3624805485 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x560b3c8cc610 0x560b3c8d5130 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7f1958029850 tx=0x7f1958004c70 comp rx=0 tx=0).stop 2024-03-21T10:27:25.642 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:25.638+0000 7f196778c740 1 -- 172.21.15.47:0/3624805485 shutdown_connections 2024-03-21T10:27:25.642 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:25.638+0000 7f196778c740 1 --2- 172.21.15.47:0/3624805485 >> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] conn(0x560b3c8df850 0x560b3c93e4b0 crc :-1 s=READY pgs=94 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:27:25.642 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:25.638+0000 7f196778c740 1 --2- 172.21.15.47:0/3624805485 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x560b3c8cc610 0x560b3c8d5130 unknown :-1 s=CLOSED pgs=714 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:27:25.642 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:25.638+0000 7f196778c740 1 --2- 172.21.15.47:0/3624805485 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560b3c8c6dd0 0x560b3c8d4bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:27:25.643 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:25.638+0000 7f196778c740 1 --2- 172.21.15.47:0/3624805485 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560b3c8c60f0 0x560b3c8d46b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:27:25.643 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:25.638+0000 7f196778c740 1 -- 172.21.15.47:0/3624805485 >> 172.21.15.47:0/3624805485 conn(0x560b3c88d200 msgr2=0x560b3c8bfbc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:27:25.643 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:25.638+0000 7f193f400640 0 client.16524 ms_handle_reset on v2:172.21.15.100:6838/4173168440 2024-03-21T10:27:25.643 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:25.638+0000 7f196778c740 1 -- 172.21.15.47:0/3624805485 shutdown_connections 2024-03-21T10:27:25.643 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:25.638+0000 7f196778c740 1 -- 172.21.15.47:0/3624805485 wait complete. 2024-03-21T10:27:25.643 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:25.638+0000 7f196778c740 20 client.16524 trim_cache size 0 max 16384 2024-03-21T10:27:25.643 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-03-21T10:27:25.643 INFO:teuthology.orchestra.run.smithi047.stdout: "status": "IDLE (waiting for more scrubs)", 2024-03-21T10:27:25.643 INFO:teuthology.orchestra.run.smithi047.stdout: "scrubs": { 2024-03-21T10:27:25.643 INFO:teuthology.orchestra.run.smithi047.stdout: "72616ced-3c4d-450c-8c3e-77e93553bb86": { 2024-03-21T10:27:25.644 INFO:teuthology.orchestra.run.smithi047.stdout: "path": "/", 2024-03-21T10:27:25.644 INFO:teuthology.orchestra.run.smithi047.stdout: "tag": "72616ced-3c4d-450c-8c3e-77e93553bb86", 2024-03-21T10:27:25.644 INFO:teuthology.orchestra.run.smithi047.stdout: "options": "recursive,force" 2024-03-21T10:27:25.644 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-21T10:27:25.644 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-21T10:27:25.644 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-03-21T10:27:25.645 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:25.639+0000 7f195f200640 1 -- 172.21.15.47:0/4253981533 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f19440a3090 msgr2=0x7f19440a5550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:27:25.646 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:25.639+0000 7f195f200640 1 --2- 172.21.15.47:0/4253981533 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f19440a3090 0x7f19440a5550 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f1958005fd0 tx=0x7f1958015040 comp rx=0 tx=0).stop 2024-03-21T10:27:25.646 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:25.639+0000 7f195f200640 1 -- 172.21.15.47:0/4253981533 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1960101520 msgr2=0x7f196019d910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:27:25.646 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:25.639+0000 7f195f200640 1 --2- 172.21.15.47:0/4253981533 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1960101520 0x7f196019d910 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f1950000c00 tx=0x7f19500045e0 comp rx=0 tx=0).stop 2024-03-21T10:27:25.646 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:25.639+0000 7f195f200640 1 -- 172.21.15.47:0/4253981533 shutdown_connections 2024-03-21T10:27:25.646 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:25.639+0000 7f195f200640 1 --2- 172.21.15.47:0/4253981533 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f19601075e0 0x7f1960197ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:27:25.646 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:25.639+0000 7f195f200640 1 --2- 172.21.15.47:0/4253981533 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1960101ee0 0x7f196019de50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:27:25.647 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:25.639+0000 7f195f200640 1 --2- 172.21.15.47:0/4253981533 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f19440a3090 0x7f19440a5550 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:27:25.647 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:25.639+0000 7f195f200640 1 --2- 172.21.15.47:0/4253981533 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1960101520 0x7f196019d910 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:27:25.647 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:25.639+0000 7f195f200640 1 -- 172.21.15.47:0/4253981533 >> 172.21.15.47:0/4253981533 conn(0x7f19600fd010 msgr2=0x7f19600fe790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:27:25.647 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:25.639+0000 7f195f200640 1 -- 172.21.15.47:0/4253981533 shutdown_connections 2024-03-21T10:27:25.647 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:25.639+0000 7f195f200640 1 -- 172.21.15.47:0/4253981533 wait complete. 2024-03-21T10:27:25.656 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:27:25 smithi047 ceph-mon[31547]: pgmap v66: 129 pgs: 129 active+clean; 142 MiB data, 667 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 4.7 MiB/s wr, 9 op/s 2024-03-21T10:27:25.659 INFO:tasks.cephfs.filesystem:scrub status for tag:72616ced-3c4d-450c-8c3e-77e93553bb86 - {'path': '/', 'tag': '72616ced-3c4d-450c-8c3e-77e93553bb86', 'options': 'recursive,force'} 2024-03-21T10:27:25.728 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:27:25 smithi100 ceph-mon[35193]: pgmap v66: 129 pgs: 129 active+clean; 142 MiB data, 667 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 4.7 MiB/s wr, 9 op/s 2024-03-21T10:27:25.738 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:27:25 smithi157 ceph-mon[35581]: pgmap v66: 129 pgs: 129 active+clean; 142 MiB data, 667 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 4.7 MiB/s wr, 9 op/s 2024-03-21T10:27:26.713 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:27:26 smithi047 ceph-mon[31547]: scrub queued for path: / 2024-03-21T10:27:26.713 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:27:26 smithi047 ceph-mon[31547]: scrub summary: idle+waiting paths [/] 2024-03-21T10:27:26.713 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:27:26 smithi047 ceph-mon[31547]: scrub summary: active paths [/] 2024-03-21T10:27:26.713 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:27:26 smithi047 ceph-mon[31547]: scrub summary: idle+waiting paths [/] 2024-03-21T10:27:26.728 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:27:26 smithi100 ceph-mon[35193]: scrub queued for path: / 2024-03-21T10:27:26.728 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:27:26 smithi100 ceph-mon[35193]: scrub summary: idle+waiting paths [/] 2024-03-21T10:27:26.728 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:27:26 smithi100 ceph-mon[35193]: scrub summary: active paths [/] 2024-03-21T10:27:26.729 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:27:26 smithi100 ceph-mon[35193]: scrub summary: idle+waiting paths [/] 2024-03-21T10:27:26.738 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:27:26 smithi157 ceph-mon[35581]: scrub queued for path: / 2024-03-21T10:27:26.739 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:27:26 smithi157 ceph-mon[35581]: scrub summary: idle+waiting paths [/] 2024-03-21T10:27:26.739 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:27:26 smithi157 ceph-mon[35581]: scrub summary: active paths [/] 2024-03-21T10:27:26.739 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:27:26 smithi157 ceph-mon[35581]: scrub summary: idle+waiting paths [/] 2024-03-21T10:27:27.713 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:27:27 smithi047 ceph-mon[31547]: pgmap v67: 129 pgs: 129 active+clean; 186 MiB data, 753 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 8.3 MiB/s wr, 18 op/s 2024-03-21T10:27:27.730 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:27:27 smithi100 ceph-mon[35193]: pgmap v67: 129 pgs: 129 active+clean; 186 MiB data, 753 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 8.3 MiB/s wr, 18 op/s 2024-03-21T10:27:27.738 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:27:27 smithi157 ceph-mon[35581]: pgmap v67: 129 pgs: 129 active+clean; 186 MiB data, 753 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 8.3 MiB/s wr, 18 op/s 2024-03-21T10:27:29.713 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:27:29 smithi047 ceph-mon[31547]: pgmap v68: 129 pgs: 129 active+clean; 186 MiB data, 753 MiB used, 1.0 TiB / 1.0 TiB avail; 8.0 KiB/s rd, 6.0 MiB/s wr, 14 op/s 2024-03-21T10:27:29.713 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:27:29 smithi047 ceph-mon[31547]: scrub summary: idle 2024-03-21T10:27:29.728 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:27:29 smithi100 ceph-mon[35193]: pgmap v68: 129 pgs: 129 active+clean; 186 MiB data, 753 MiB used, 1.0 TiB / 1.0 TiB avail; 8.0 KiB/s rd, 6.0 MiB/s wr, 14 op/s 2024-03-21T10:27:29.728 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:27:29 smithi100 ceph-mon[35193]: scrub summary: idle 2024-03-21T10:27:29.738 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:27:29 smithi157 ceph-mon[35581]: pgmap v68: 129 pgs: 129 active+clean; 186 MiB data, 753 MiB used, 1.0 TiB / 1.0 TiB avail; 8.0 KiB/s rd, 6.0 MiB/s wr, 14 op/s 2024-03-21T10:27:29.739 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:27:29 smithi157 ceph-mon[35581]: scrub summary: idle 2024-03-21T10:27:31.712 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:27:31 smithi047 ceph-mon[31547]: pgmap v69: 129 pgs: 129 active+clean; 186 MiB data, 753 MiB used, 1.0 TiB / 1.0 TiB avail; 8.0 KiB/s rd, 6.0 MiB/s wr, 14 op/s 2024-03-21T10:27:31.728 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:27:31 smithi100 ceph-mon[35193]: pgmap v69: 129 pgs: 129 active+clean; 186 MiB data, 753 MiB used, 1.0 TiB / 1.0 TiB avail; 8.0 KiB/s rd, 6.0 MiB/s wr, 14 op/s 2024-03-21T10:27:31.738 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:27:31 smithi157 ceph-mon[35581]: pgmap v69: 129 pgs: 129 active+clean; 186 MiB data, 753 MiB used, 1.0 TiB / 1.0 TiB avail; 8.0 KiB/s rd, 6.0 MiB/s wr, 14 op/s 2024-03-21T10:27:32.712 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:27:32 smithi047 ceph-mon[31547]: from='mgr.15687 172.21.15.47:0/8730661' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T10:27:32.728 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:27:32 smithi100 ceph-mon[35193]: from='mgr.15687 172.21.15.47:0/8730661' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T10:27:32.738 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:27:32 smithi157 ceph-mon[35581]: from='mgr.15687 172.21.15.47:0/8730661' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T10:27:33.712 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:27:33 smithi047 ceph-mon[31547]: pgmap v70: 129 pgs: 129 active+clean; 214 MiB data, 813 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 8.3 MiB/s wr, 18 op/s 2024-03-21T10:27:33.713 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:27:33 smithi047 ceph-mon[31547]: from='mgr.15687 172.21.15.47:0/8730661' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T10:27:33.713 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:27:33 smithi047 ceph-mon[31547]: from='mgr.15687 172.21.15.47:0/8730661' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T10:27:33.713 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:27:33 smithi047 ceph-mon[31547]: from='mgr.15687 172.21.15.47:0/8730661' entity='mgr.x' 2024-03-21T10:27:33.728 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:27:33 smithi100 ceph-mon[35193]: pgmap v70: 129 pgs: 129 active+clean; 214 MiB data, 813 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 8.3 MiB/s wr, 18 op/s 2024-03-21T10:27:33.728 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:27:33 smithi100 ceph-mon[35193]: from='mgr.15687 172.21.15.47:0/8730661' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T10:27:33.728 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:27:33 smithi100 ceph-mon[35193]: from='mgr.15687 172.21.15.47:0/8730661' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T10:27:33.729 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:27:33 smithi100 ceph-mon[35193]: from='mgr.15687 172.21.15.47:0/8730661' entity='mgr.x' 2024-03-21T10:27:33.738 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:27:33 smithi157 ceph-mon[35581]: pgmap v70: 129 pgs: 129 active+clean; 214 MiB data, 813 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 8.3 MiB/s wr, 18 op/s 2024-03-21T10:27:33.738 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:27:33 smithi157 ceph-mon[35581]: from='mgr.15687 172.21.15.47:0/8730661' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T10:27:33.739 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:27:33 smithi157 ceph-mon[35581]: from='mgr.15687 172.21.15.47:0/8730661' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T10:27:33.739 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:27:33 smithi157 ceph-mon[35581]: from='mgr.15687 172.21.15.47:0/8730661' entity='mgr.x' 2024-03-21T10:27:35.712 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:27:35 smithi047 ceph-mon[31547]: pgmap v71: 129 pgs: 129 active+clean; 214 MiB data, 813 MiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 6.0 MiB/s wr, 13 op/s 2024-03-21T10:27:35.728 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:27:35 smithi100 ceph-mon[35193]: pgmap v71: 129 pgs: 129 active+clean; 214 MiB data, 813 MiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 6.0 MiB/s wr, 13 op/s 2024-03-21T10:27:35.738 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:27:35 smithi157 ceph-mon[35581]: pgmap v71: 129 pgs: 129 active+clean; 214 MiB data, 813 MiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 6.0 MiB/s wr, 13 op/s 2024-03-21T10:27:36.198 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:36.198+0000 7f0052c00640 1 -- 172.21.15.47:0/3649488983 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f004c109890 msgr2=0x7f004c109c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:27:36.198 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:36.198+0000 7f0052c00640 1 --2- 172.21.15.47:0/3649488983 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f004c109890 0x7f004c109c70 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7f003c009930 tx=0x7f003c02bc30 comp rx=0 tx=0).stop 2024-03-21T10:27:36.199 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:36.198+0000 7f0052c00640 1 -- 172.21.15.47:0/3649488983 shutdown_connections 2024-03-21T10:27:36.199 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:36.198+0000 7f0052c00640 1 --2- 172.21.15.47:0/3649488983 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f004c104240 0x7f004c110750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:27:36.199 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:36.198+0000 7f0052c00640 1 --2- 172.21.15.47:0/3649488983 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f004c103880 0x7f004c103d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:27:36.199 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:36.198+0000 7f0052c00640 1 --2- 172.21.15.47:0/3649488983 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f004c109890 0x7f004c109c70 unknown :-1 s=CLOSED pgs=715 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:27:36.199 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:36.198+0000 7f0052c00640 1 -- 172.21.15.47:0/3649488983 >> 172.21.15.47:0/3649488983 conn(0x7f004c0ff3b0 msgr2=0x7f004c1017d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:27:36.199 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:36.198+0000 7f0052c00640 1 -- 172.21.15.47:0/3649488983 shutdown_connections 2024-03-21T10:27:36.199 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:36.198+0000 7f0052c00640 1 -- 172.21.15.47:0/3649488983 wait complete. 2024-03-21T10:27:36.200 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:36.199+0000 7f0052c00640 1 Processor -- start 2024-03-21T10:27:36.201 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:36.199+0000 7f0052c00640 1 -- start start 2024-03-21T10:27:36.201 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:36.200+0000 7f0052c00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f004c103880 0x7f004c19a0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:27:36.201 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:36.200+0000 7f0052c00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f004c104240 0x7f004c19a600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:27:36.201 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:36.200+0000 7f0052c00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f004c109890 0x7f004c19e930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:27:36.201 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:36.200+0000 7f0052c00640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f004c113ba0 con 0x7f004c104240 2024-03-21T10:27:36.201 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:36.200+0000 7f0052c00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f004c113a20 con 0x7f004c109890 2024-03-21T10:27:36.201 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:36.200+0000 7f0052c00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f004c113d20 con 0x7f004c103880 2024-03-21T10:27:36.201 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:36.200+0000 7f004b400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f004c104240 0x7f004c19a600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:27:36.201 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:36.200+0000 7f004b400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f004c104240 0x7f004c19a600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:37362/0 (socket says 172.21.15.47:37362) 2024-03-21T10:27:36.201 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:36.201+0000 7f004b400640 1 -- 172.21.15.47:0/3296989976 learned_addr learned my addr 172.21.15.47:0/3296989976 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:27:36.201 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:36.201+0000 7f004be00640 1 --2- 172.21.15.47:0/3296989976 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f004c103880 0x7f004c19a0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:27:36.201 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:36.201+0000 7f0050c00640 1 --2- 172.21.15.47:0/3296989976 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f004c109890 0x7f004c19e930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:27:36.202 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:36.201+0000 7f004b400640 1 -- 172.21.15.47:0/3296989976 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f004c103880 msgr2=0x7f004c19a0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:27:36.202 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:36.201+0000 7f004b400640 1 --2- 172.21.15.47:0/3296989976 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f004c103880 0x7f004c19a0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:27:36.203 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:36.201+0000 7f004b400640 1 -- 172.21.15.47:0/3296989976 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f004c109890 msgr2=0x7f004c19e930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:27:36.203 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:36.201+0000 7f004b400640 1 --2- 172.21.15.47:0/3296989976 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f004c109890 0x7f004c19e930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:27:36.203 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:36.201+0000 7f004b400640 1 -- 172.21.15.47:0/3296989976 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f004c19f060 con 0x7f004c104240 2024-03-21T10:27:36.203 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:36.202+0000 7f004b400640 1 --2- 172.21.15.47:0/3296989976 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f004c104240 0x7f004c19a600 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7f003800e9f0 tx=0x7f003800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:27:36.203 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:36.202+0000 7f0048c00640 1 -- 172.21.15.47:0/3296989976 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f003800cd80 con 0x7f004c104240 2024-03-21T10:27:36.203 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:36.202+0000 7f0052c00640 1 -- 172.21.15.47:0/3296989976 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f004c19f350 con 0x7f004c104240 2024-03-21T10:27:36.203 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:36.202+0000 7f0048c00640 1 -- 172.21.15.47:0/3296989976 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f00380108e0 con 0x7f004c104240 2024-03-21T10:27:36.203 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:36.202+0000 7f0048c00640 1 -- 172.21.15.47:0/3296989976 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0038018920 con 0x7f004c104240 2024-03-21T10:27:36.203 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:36.202+0000 7f0052c00640 1 -- 172.21.15.47:0/3296989976 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f004c1a6be0 con 0x7f004c104240 2024-03-21T10:27:36.206 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:36.206+0000 7f0055819740 1 Processor -- start 2024-03-21T10:27:36.206 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:36.207+0000 7f0055819740 1 -- start start 2024-03-21T10:27:36.206 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:36.207+0000 7f0048c00640 1 -- 172.21.15.47:0/3296989976 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 27) v1 ==== 137985+0+0 (secure 0 0 0) 0x7f0038010430 con 0x7f004c104240 2024-03-21T10:27:36.208 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:36.207+0000 7f0055819740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55813abd2fd0 0x55813abd33b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:27:36.208 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:36.207+0000 7f0055819740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55813abd38f0 0x55813abdc180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:27:36.208 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:36.207+0000 7f0055819740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55813abdc6c0 0x55813abdeab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:27:36.208 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:36.207+0000 7f0055819740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x55813ab953b0 con 0x55813abd2fd0 2024-03-21T10:27:36.208 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:36.207+0000 7f0055819740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55813ab37830 con 0x55813abdc6c0 2024-03-21T10:27:36.208 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:36.207+0000 7f0055819740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55813aae05d0 con 0x55813abd38f0 2024-03-21T10:27:36.208 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:36.207+0000 7f0050c00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55813abd2fd0 0x55813abd33b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:27:36.208 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:36.207+0000 7f0050c00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55813abd2fd0 0x55813abd33b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:37370/0 (socket says 172.21.15.47:37370) 2024-03-21T10:27:36.208 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:36.207+0000 7f0050c00640 1 -- 172.21.15.47:0/2472572515 learned_addr learned my addr 172.21.15.47:0/2472572515 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:27:36.208 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:36.208+0000 7f004b400640 1 --2- 172.21.15.47:0/2472572515 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55813abdc6c0 0x55813abdeab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:27:36.208 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:36.208+0000 7f004be00640 1 --2- 172.21.15.47:0/2472572515 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55813abd38f0 0x55813abdc180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:27:36.208 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:36.208+0000 7f0050c00640 1 -- 172.21.15.47:0/2472572515 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55813abd38f0 msgr2=0x55813abdc180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:27:36.209 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:36.208+0000 7f0050c00640 1 --2- 172.21.15.47:0/2472572515 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55813abd38f0 0x55813abdc180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:27:36.209 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:36.208+0000 7f0050c00640 1 -- 172.21.15.47:0/2472572515 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55813abdc6c0 msgr2=0x55813abdeab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:27:36.209 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:36.208+0000 7f0050c00640 1 --2- 172.21.15.47:0/2472572515 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55813abdc6c0 0x55813abdeab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:27:36.209 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:36.208+0000 7f0050c00640 1 -- 172.21.15.47:0/2472572515 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55813aaf82c0 con 0x55813abd2fd0 2024-03-21T10:27:36.209 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:36.208+0000 7f004b400640 1 --2- 172.21.15.47:0/2472572515 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55813abdc6c0 0x55813abdeab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T10:27:36.210 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:36.208+0000 7f0050c00640 1 --2- 172.21.15.47:0/2472572515 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55813abd2fd0 0x55813abd33b0 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7f0044009f50 tx=0x7f00440118d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4306cae0d9c8d1be server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:27:36.210 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:36.209+0000 7f0048c00640 1 --2- 172.21.15.47:0/3296989976 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f00300a31b0 0x7f00300a5670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:27:36.210 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:36.209+0000 7f0048c00640 1 -- 172.21.15.47:0/3296989976 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 9474+0+0 (secure 0 0 0) 0x7f00380cfd00 con 0x7f004c104240 2024-03-21T10:27:36.210 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:36.209+0000 7f0041200640 1 -- 172.21.15.47:0/2472572515 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0044004d20 con 0x55813abd2fd0 2024-03-21T10:27:36.211 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:36.209+0000 7f004be00640 1 --2- 172.21.15.47:0/3296989976 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f00300a31b0 0x7f00300a5670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:27:36.211 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:36.209+0000 7f0041200640 1 -- 172.21.15.47:0/2472572515 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f0044004050 con 0x55813abd2fd0 2024-03-21T10:27:36.211 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:36.209+0000 7f0041200640 1 -- 172.21.15.47:0/2472572515 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0044005280 con 0x55813abd2fd0 2024-03-21T10:27:36.211 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:36.210+0000 7f0055819740 1 -- 172.21.15.47:0/2472572515 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55813abd2fd0 msgr2=0x55813abd33b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:27:36.211 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:36.210+0000 7f004be00640 1 --2- 172.21.15.47:0/3296989976 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f00300a31b0 0x7f00300a5670 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f003c009900 tx=0x7f003c005a20 comp rx=0 tx=0).ready entity=mgr.15687 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:27:36.211 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:36.210+0000 7f0055819740 1 --2- 172.21.15.47:0/2472572515 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55813abd2fd0 0x55813abd33b0 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7f0044009f50 tx=0x7f00440118d0 comp rx=0 tx=0).stop 2024-03-21T10:27:36.211 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:36.210+0000 7f0055819740 1 -- 172.21.15.47:0/2472572515 shutdown_connections 2024-03-21T10:27:36.211 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:36.210+0000 7f0055819740 1 --2- 172.21.15.47:0/2472572515 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55813abdc6c0 0x55813abdeab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:27:36.211 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:36.210+0000 7f0055819740 1 --2- 172.21.15.47:0/2472572515 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55813abd38f0 0x55813abdc180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:27:36.211 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:36.210+0000 7f0055819740 1 --2- 172.21.15.47:0/2472572515 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55813abd2fd0 0x55813abd33b0 unknown :-1 s=CLOSED pgs=717 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:27:36.211 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:36.210+0000 7f0055819740 1 -- 172.21.15.47:0/2472572515 >> 172.21.15.47:0/2472572515 conn(0x55813abcd8f0 msgr2=0x55813abcdcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:27:36.211 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:36.210+0000 7f0055819740 1 -- 172.21.15.47:0/2472572515 shutdown_connections 2024-03-21T10:27:36.211 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:36.210+0000 7f0055819740 1 -- 172.21.15.47:0/2472572515 wait complete. 2024-03-21T10:27:36.214 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:36.211+0000 7f0055819740 1 -- start start 2024-03-21T10:27:36.214 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:36.211+0000 7f0055819740 1 Processor -- start 2024-03-21T10:27:36.214 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:36.212+0000 7f0055819740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55813abd2fd0 0x55813abd9680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:27:36.214 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:36.212+0000 7f0055819740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55813abd38f0 0x55813abd9bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:27:36.214 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:36.212+0000 7f0055819740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55813abdc6c0 0x55813abda100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:27:36.214 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:36.212+0000 7f0055819740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x55813aaf3a20 con 0x55813abd2fd0 2024-03-21T10:27:36.214 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:36.212+0000 7f0055819740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55813ab338d0 con 0x55813abdc6c0 2024-03-21T10:27:36.215 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:36.212+0000 7f0055819740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55813abdf2f0 con 0x55813abd38f0 2024-03-21T10:27:36.215 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:36.212+0000 7f0055819740 10 client.0 _ll_register_callbacks cb 0x55813ab301c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T10:27:36.215 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:36.212+0000 7f004be00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55813abd2fd0 0x55813abd9680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:27:36.215 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:36.212+0000 7f004be00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55813abd2fd0 0x55813abd9680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:37374/0 (socket says 172.21.15.47:37374) 2024-03-21T10:27:36.215 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:36.212+0000 7f004be00640 1 -- 172.21.15.47:0/3345610676 learned_addr learned my addr 172.21.15.47:0/3345610676 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:27:36.215 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:36.213+0000 7f0050c00640 1 --2- 172.21.15.47:0/3345610676 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55813abdc6c0 0x55813abda100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:27:36.215 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:36.213+0000 7f004b400640 1 --2- 172.21.15.47:0/3345610676 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55813abd38f0 0x55813abd9bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:27:36.215 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:36.213+0000 7f004be00640 1 -- 172.21.15.47:0/3345610676 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55813abd38f0 msgr2=0x55813abd9bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:27:36.215 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:36.213+0000 7f004be00640 1 --2- 172.21.15.47:0/3345610676 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55813abd38f0 0x55813abd9bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:27:36.215 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:36.213+0000 7f004be00640 1 -- 172.21.15.47:0/3345610676 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55813abdc6c0 msgr2=0x55813abda100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:27:36.215 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:36.213+0000 7f004be00640 1 --2- 172.21.15.47:0/3345610676 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55813abdc6c0 0x55813abda100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:27:36.215 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:36.213+0000 7f004be00640 1 -- 172.21.15.47:0/3345610676 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55813a84c210 con 0x55813abd2fd0 2024-03-21T10:27:36.215 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:36.213+0000 7f004be00640 1 --2- 172.21.15.47:0/3345610676 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55813abd2fd0 0x55813abd9680 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7f003c006010 tx=0x7f003c02bc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6725e916373d12d server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:27:36.215 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:36.213+0000 7f0035600640 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-03-21T10:27:36.216 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:36.214+0000 7f0035600640 1 -- 172.21.15.47:0/3345610676 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f003c04f030 con 0x55813abd2fd0 2024-03-21T10:27:36.216 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:36.214+0000 7f0035600640 1 -- 172.21.15.47:0/3345610676 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(97..97 src has 1..97) v4 ==== 9474+0+0 (secure 0 0 0) 0x7f003c03a040 con 0x55813abd2fd0 2024-03-21T10:27:36.216 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:36.214+0000 7f0055819740 1 -- 172.21.15.47:0/3345610676 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f0044004050 con 0x55813abd2fd0 2024-03-21T10:27:36.216 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:36.214+0000 7f0035600640 1 -- 172.21.15.47:0/3345610676 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f003c005040 con 0x55813abd2fd0 2024-03-21T10:27:36.216 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:36.214+0000 7f0055819740 10 client.16536 fetch_fsmap learned FSMap version 32 2024-03-21T10:27:36.216 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:36.214+0000 7f0055819740 1 -- 172.21.15.47:0/3345610676 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x55813a918620 con 0x55813abd2fd0 2024-03-21T10:27:36.216 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:36.214+0000 7f0035600640 1 -- 172.21.15.47:0/3345610676 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f0044004050 con 0x55813abd2fd0 2024-03-21T10:27:36.217 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:36.215+0000 7f0035600640 1 -- 172.21.15.47:0/3345610676 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 32) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f003c0035f0 con 0x55813abd2fd0 2024-03-21T10:27:36.217 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:36.215+0000 7f0055819740 10 client.16536 fetch_fsmap finished waiting for FSMap version 32 2024-03-21T10:27:36.217 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:36.215+0000 7f0055819740 10 client.16536 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-21T10:27:36.217 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:36.215+0000 7f0055819740 20 client.16536 populate_metadata read hostname 'smithi047' 2024-03-21T10:27:36.217 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:36.215+0000 7f0055819740 1 --2- 172.21.15.47:0/3345610676 >> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] conn(0x55813abec7c0 0x55813abeec10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:27:36.217 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:36.215+0000 7f0055819740 4 client.16536 mds_command: new command op to 34535 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T10:27:36.217 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:36.215+0000 7f0055819740 1 -- 172.21.15.47:0/3345610676 --> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55813ab44d70 con 0x55813abec7c0 2024-03-21T10:27:36.217 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:36.216+0000 7f004b400640 1 --2- 172.21.15.47:0/3345610676 >> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] conn(0x55813abec7c0 0x55813abeec10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:27:36.217 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:36.216+0000 7f004b400640 1 --2- 172.21.15.47:0/3345610676 >> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] conn(0x55813abec7c0 0x55813abeec10 crc :-1 s=READY pgs=95 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=85093d042ee70329 server_cookie=1ef156edf13f4c30 in_seq=0 out_seq=0 2024-03-21T10:27:36.217 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:36.216+0000 7f0035600640 10 client.16536 ms_handle_connect on v2:172.21.15.100:6838/4173168440 2024-03-21T10:27:36.218 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:36.218+0000 7f0035600640 1 -- 172.21.15.47:0/3345610676 <== mds.0 v2:172.21.15.100:6838/4173168440 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55813ab44d70 con 0x55813abec7c0 2024-03-21T10:27:36.218 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:36.218+0000 7f0035600640 10 client.16536 handle_command_reply: tid=0 2024-03-21T10:27:36.218 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:36.218+0000 7f0055819740 1 client.16536 shutdown 2024-03-21T10:27:36.218 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:36.218+0000 7f0055819740 1 -- 172.21.15.47:0/3345610676 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55813abd2fd0 msgr2=0x55813abd9680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:27:36.218 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:36.218+0000 7f0055819740 1 --2- 172.21.15.47:0/3345610676 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55813abd2fd0 0x55813abd9680 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7f003c006010 tx=0x7f003c02bc30 comp rx=0 tx=0).stop 2024-03-21T10:27:36.219 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:36.218+0000 7f0055819740 1 -- 172.21.15.47:0/3345610676 shutdown_connections 2024-03-21T10:27:36.219 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:36.218+0000 7f0055819740 1 --2- 172.21.15.47:0/3345610676 >> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] conn(0x55813abec7c0 0x55813abeec10 crc :-1 s=READY pgs=95 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:27:36.219 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:36.218+0000 7f0055819740 1 --2- 172.21.15.47:0/3345610676 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55813abdc6c0 0x55813abda100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:27:36.219 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:36.218+0000 7f0055819740 1 --2- 172.21.15.47:0/3345610676 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55813abd38f0 0x55813abd9bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:27:36.219 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:36.218+0000 7f0055819740 1 --2- 172.21.15.47:0/3345610676 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55813abd2fd0 0x55813abd9680 unknown :-1 s=CLOSED pgs=718 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:27:36.219 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:36.218+0000 7f0035600640 0 client.16536 ms_handle_reset on v2:172.21.15.100:6838/4173168440 2024-03-21T10:27:36.219 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:36.218+0000 7f0055819740 1 -- 172.21.15.47:0/3345610676 >> 172.21.15.47:0/3345610676 conn(0x55813abcd8f0 msgr2=0x55813abcf640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:27:36.219 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:36.219+0000 7f0055819740 1 -- 172.21.15.47:0/3345610676 shutdown_connections 2024-03-21T10:27:36.219 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:36.219+0000 7f0055819740 1 -- 172.21.15.47:0/3345610676 wait complete. 2024-03-21T10:27:36.219 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:36.219+0000 7f0055819740 20 client.16536 trim_cache size 0 max 16384 2024-03-21T10:27:36.243 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:36.243+0000 7f0055819740 1 Processor -- start 2024-03-21T10:27:36.243 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:36.243+0000 7f0055819740 1 -- start start 2024-03-21T10:27:36.244 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:36.243+0000 7f0055819740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55813abd2fd0 0x55813abdbdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:27:36.244 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:36.243+0000 7f0055819740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55813abd38f0 0x55813abcc7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:27:36.244 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:36.243+0000 7f0055819740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55813abdc6c0 0x55813abccd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:27:36.244 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:36.243+0000 7f0055819740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f002c00fd40 con 0x55813abd38f0 2024-03-21T10:27:36.244 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:36.243+0000 7f0055819740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f002c0116c0 con 0x55813abdc6c0 2024-03-21T10:27:36.244 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:36.243+0000 7f0055819740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f002c0103a0 con 0x55813abd2fd0 2024-03-21T10:27:36.244 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:36.244+0000 7f004b400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55813abd38f0 0x55813abcc7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:27:36.244 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:36.244+0000 7f004b400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55813abd38f0 0x55813abcc7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:37384/0 (socket says 172.21.15.47:37384) 2024-03-21T10:27:36.244 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:36.244+0000 7f0050c00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55813abdc6c0 0x55813abccd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:27:36.244 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:36.244+0000 7f004b400640 1 -- 172.21.15.47:0/3197357626 learned_addr learned my addr 172.21.15.47:0/3197357626 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:27:36.245 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:36.244+0000 7f004be00640 1 --2- 172.21.15.47:0/3197357626 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55813abd2fd0 0x55813abdbdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:27:36.245 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:36.244+0000 7f004b400640 1 -- 172.21.15.47:0/3197357626 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55813abd2fd0 msgr2=0x55813abdbdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:27:36.245 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:36.244+0000 7f004b400640 1 --2- 172.21.15.47:0/3197357626 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55813abd2fd0 0x55813abdbdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:27:36.245 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:36.244+0000 7f004b400640 1 -- 172.21.15.47:0/3197357626 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55813abdc6c0 msgr2=0x55813abccd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:27:36.245 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:36.244+0000 7f004b400640 1 --2- 172.21.15.47:0/3197357626 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55813abdc6c0 0x55813abccd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:27:36.246 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:36.244+0000 7f004b400640 1 -- 172.21.15.47:0/3197357626 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55813abd3f80 con 0x55813abd38f0 2024-03-21T10:27:36.246 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:36.245+0000 7f004b400640 1 --2- 172.21.15.47:0/3197357626 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55813abd38f0 0x55813abcc7d0 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7f003809bd10 tx=0x7f003801f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aa7e3602de21e1f1 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:27:36.246 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:36.245+0000 7f0037e00640 1 -- 172.21.15.47:0/3197357626 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f003809bd80 con 0x55813abd38f0 2024-03-21T10:27:36.246 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:36.245+0000 7f0037e00640 1 -- 172.21.15.47:0/3197357626 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f003809fe20 con 0x55813abd38f0 2024-03-21T10:27:36.246 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:36.245+0000 7f0037e00640 1 -- 172.21.15.47:0/3197357626 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0038097ad0 con 0x55813abd38f0 2024-03-21T10:27:36.247 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:36.246+0000 7f0055819740 1 -- 172.21.15.47:0/3197357626 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55813abd38f0 msgr2=0x55813abcc7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:27:36.247 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:36.246+0000 7f0055819740 1 --2- 172.21.15.47:0/3197357626 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55813abd38f0 0x55813abcc7d0 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7f003809bd10 tx=0x7f003801f000 comp rx=0 tx=0).stop 2024-03-21T10:27:36.247 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:36.246+0000 7f0055819740 1 -- 172.21.15.47:0/3197357626 shutdown_connections 2024-03-21T10:27:36.247 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:36.246+0000 7f0055819740 1 --2- 172.21.15.47:0/3197357626 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55813abdc6c0 0x55813abccd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:27:36.247 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:36.246+0000 7f0055819740 1 --2- 172.21.15.47:0/3197357626 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55813abd38f0 0x55813abcc7d0 unknown :-1 s=CLOSED pgs=719 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:27:36.247 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:36.246+0000 7f0055819740 1 --2- 172.21.15.47:0/3197357626 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55813abd2fd0 0x55813abdbdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:27:36.247 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:36.246+0000 7f0055819740 1 -- 172.21.15.47:0/3197357626 >> 172.21.15.47:0/3197357626 conn(0x55813abcd8f0 msgr2=0x55813abcc0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:27:36.247 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:36.246+0000 7f0055819740 1 -- 172.21.15.47:0/3197357626 shutdown_connections 2024-03-21T10:27:36.247 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:36.246+0000 7f0055819740 1 -- 172.21.15.47:0/3197357626 wait complete. 2024-03-21T10:27:36.247 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:36.247+0000 7f0055819740 1 -- start start 2024-03-21T10:27:36.248 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:36.247+0000 7f0055819740 1 Processor -- start 2024-03-21T10:27:36.249 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:36.248+0000 7f0055819740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55813abd2fd0 0x55813abd7330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:27:36.249 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:36.248+0000 7f0055819740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55813abd38f0 0x55813abe1540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:27:36.249 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:36.248+0000 7f0055819740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55813abdc6c0 0x55813abe1a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:27:36.249 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:36.248+0000 7f0055819740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f002c011060 con 0x55813abd2fd0 2024-03-21T10:27:36.249 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:36.248+0000 7f0055819740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f002c010a00 con 0x55813abdc6c0 2024-03-21T10:27:36.249 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:36.248+0000 7f0055819740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f002c00df00 con 0x55813abd38f0 2024-03-21T10:27:36.249 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:36.248+0000 7f0055819740 10 client.0 _ll_register_callbacks cb 0x7f003c043050 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T10:27:36.250 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:36.248+0000 7f004be00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55813abd2fd0 0x55813abd7330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:27:36.250 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:36.248+0000 7f0050c00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55813abdc6c0 0x55813abe1a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:27:36.250 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:36.248+0000 7f004be00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55813abd2fd0 0x55813abd7330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:37398/0 (socket says 172.21.15.47:37398) 2024-03-21T10:27:36.250 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:36.248+0000 7f004be00640 1 -- 172.21.15.47:0/1835707688 learned_addr learned my addr 172.21.15.47:0/1835707688 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:27:36.250 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:36.248+0000 7f004b400640 1 --2- 172.21.15.47:0/1835707688 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55813abd38f0 0x55813abe1540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:27:36.250 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:36.249+0000 7f0050c00640 1 -- 172.21.15.47:0/1835707688 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55813abd38f0 msgr2=0x55813abe1540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:27:36.250 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:36.249+0000 7f0050c00640 1 --2- 172.21.15.47:0/1835707688 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55813abd38f0 0x55813abe1540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:27:36.250 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:36.249+0000 7f0050c00640 1 -- 172.21.15.47:0/1835707688 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55813abd2fd0 msgr2=0x55813abd7330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:27:36.250 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:36.249+0000 7f0050c00640 1 --2- 172.21.15.47:0/1835707688 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55813abd2fd0 0x55813abd7330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:27:36.250 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:36.249+0000 7f0050c00640 1 -- 172.21.15.47:0/1835707688 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55813abe21c0 con 0x55813abdc6c0 2024-03-21T10:27:36.250 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:36.249+0000 7f0050c00640 1 --2- 172.21.15.47:0/1835707688 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55813abdc6c0 0x55813abe1a80 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f0044004740 tx=0x7f0044003f20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5f6e16a86b6c2eb1 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:27:36.250 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:36.249+0000 7f0035600640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-21T10:27:36.252 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:36.250+0000 7f0035600640 1 -- 172.21.15.47:0/1835707688 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0044004770 con 0x55813abdc6c0 2024-03-21T10:27:36.252 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:36.250+0000 7f0055819740 1 -- 172.21.15.47:0/1835707688 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f003809fe20 con 0x55813abdc6c0 2024-03-21T10:27:36.252 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:36.250+0000 7f0035600640 1 -- 172.21.15.47:0/1835707688 <== mon.2 v2:172.21.15.100:3300/0 2 ==== osd_map(97..97 src has 1..97) v4 ==== 9474+0+0 (secure 0 0 0) 0x7f0044004910 con 0x55813abdc6c0 2024-03-21T10:27:36.252 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:36.250+0000 7f0035600640 1 -- 172.21.15.47:0/1835707688 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0044004b10 con 0x55813abdc6c0 2024-03-21T10:27:36.252 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:36.250+0000 7f0055819740 10 client.36332 fetch_fsmap learned FSMap version 32 2024-03-21T10:27:36.252 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:36.250+0000 7f0055819740 1 -- 172.21.15.47:0/1835707688 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x55813ac49030 con 0x55813abdc6c0 2024-03-21T10:27:36.252 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:36.251+0000 7f0035600640 1 -- 172.21.15.47:0/1835707688 <== mon.2 v2:172.21.15.100:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f003809fe20 con 0x55813abdc6c0 2024-03-21T10:27:36.252 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:36.251+0000 7f0035600640 1 -- 172.21.15.47:0/1835707688 <== mon.2 v2:172.21.15.100:3300/0 5 ==== fsmap(e 32) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f004400f480 con 0x55813abdc6c0 2024-03-21T10:27:36.253 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:36.251+0000 7f0055819740 10 client.36332 fetch_fsmap finished waiting for FSMap version 32 2024-03-21T10:27:36.253 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:36.251+0000 7f0055819740 10 client.36332 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-21T10:27:36.253 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:36.251+0000 7f0055819740 20 client.36332 populate_metadata read hostname 'smithi047' 2024-03-21T10:27:36.253 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:36.251+0000 7f0055819740 1 --2- 172.21.15.47:0/1835707688 >> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] conn(0x55813ac49520 0x55813ac4b940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:27:36.253 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:36.251+0000 7f0055819740 4 client.36332 mds_command: new command op to 34535 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-21T10:27:36.253 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:36.251+0000 7f0055819740 1 -- 172.21.15.47:0/1835707688 --> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55813ab3f0c0 con 0x55813ac49520 2024-03-21T10:27:36.253 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:36.252+0000 7f004be00640 1 --2- 172.21.15.47:0/1835707688 >> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] conn(0x55813ac49520 0x55813ac4b940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:27:36.253 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:36.252+0000 7f004be00640 1 --2- 172.21.15.47:0/1835707688 >> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] conn(0x55813ac49520 0x55813ac4b940 crc :-1 s=READY pgs=96 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d8c90d988c166670 server_cookie=8cd9d62c2f92c8bd in_seq=0 out_seq=0 2024-03-21T10:27:36.253 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:36.252+0000 7f0035600640 10 client.36332 ms_handle_connect on v2:172.21.15.100:6838/4173168440 2024-03-21T10:27:36.269 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:36.269+0000 7f0035600640 1 -- 172.21.15.47:0/1835707688 <== mds.0 v2:172.21.15.100:6838/4173168440 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55813ab3f0c0 con 0x55813ac49520 2024-03-21T10:27:36.269 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:36.269+0000 7f0035600640 10 client.36332 handle_command_reply: tid=0 2024-03-21T10:27:36.269 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:36.269+0000 7f0055819740 1 client.36332 shutdown 2024-03-21T10:27:36.269 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:36.269+0000 7f0055819740 1 -- 172.21.15.47:0/1835707688 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55813abdc6c0 msgr2=0x55813abe1a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:27:36.270 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:36.269+0000 7f0055819740 1 --2- 172.21.15.47:0/1835707688 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55813abdc6c0 0x55813abe1a80 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f0044004740 tx=0x7f0044003f20 comp rx=0 tx=0).stop 2024-03-21T10:27:36.270 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:36.269+0000 7f0055819740 1 -- 172.21.15.47:0/1835707688 shutdown_connections 2024-03-21T10:27:36.270 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:36.269+0000 7f0055819740 1 --2- 172.21.15.47:0/1835707688 >> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] conn(0x55813ac49520 0x55813ac4b940 crc :-1 s=READY pgs=96 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:27:36.270 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:36.269+0000 7f0055819740 1 --2- 172.21.15.47:0/1835707688 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55813abdc6c0 0x55813abe1a80 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:27:36.270 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:36.269+0000 7f0055819740 1 --2- 172.21.15.47:0/1835707688 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55813abd38f0 0x55813abe1540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:27:36.270 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:36.269+0000 7f0055819740 1 --2- 172.21.15.47:0/1835707688 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55813abd2fd0 0x55813abd7330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:27:36.270 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:36.269+0000 7f0035600640 0 client.36332 ms_handle_reset on v2:172.21.15.100:6838/4173168440 2024-03-21T10:27:36.270 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:36.269+0000 7f0055819740 1 -- 172.21.15.47:0/1835707688 >> 172.21.15.47:0/1835707688 conn(0x55813abcd8f0 msgr2=0x55813abcf340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:27:36.270 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:36.269+0000 7f0055819740 1 -- 172.21.15.47:0/1835707688 shutdown_connections 2024-03-21T10:27:36.270 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:36.269+0000 7f0055819740 1 -- 172.21.15.47:0/1835707688 wait complete. 2024-03-21T10:27:36.270 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:36.270+0000 7f0055819740 20 client.36332 trim_cache size 0 max 16384 2024-03-21T10:27:36.271 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout:{ 2024-03-21T10:27:36.271 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout: "message": "", 2024-03-21T10:27:36.271 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout: "return_code": 0 2024-03-21T10:27:36.271 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout:} 2024-03-21T10:27:36.271 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:36.270+0000 7f0052c00640 1 -- 172.21.15.47:0/3296989976 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f00300a31b0 msgr2=0x7f00300a5670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:27:36.271 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:36.270+0000 7f0052c00640 1 --2- 172.21.15.47:0/3296989976 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f00300a31b0 0x7f00300a5670 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f003c009900 tx=0x7f003c005a20 comp rx=0 tx=0).stop 2024-03-21T10:27:36.272 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:36.270+0000 7f0052c00640 1 -- 172.21.15.47:0/3296989976 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f004c104240 msgr2=0x7f004c19a600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:27:36.272 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:36.270+0000 7f0052c00640 1 --2- 172.21.15.47:0/3296989976 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f004c104240 0x7f004c19a600 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7f003800e9f0 tx=0x7f003800eeb0 comp rx=0 tx=0).stop 2024-03-21T10:27:36.272 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:36.271+0000 7f0052c00640 1 -- 172.21.15.47:0/3296989976 shutdown_connections 2024-03-21T10:27:36.272 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:36.271+0000 7f0052c00640 1 --2- 172.21.15.47:0/3296989976 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f00300a31b0 0x7f00300a5670 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:27:36.272 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:36.271+0000 7f0052c00640 1 --2- 172.21.15.47:0/3296989976 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f004c109890 0x7f004c19e930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:27:36.272 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:36.271+0000 7f0052c00640 1 --2- 172.21.15.47:0/3296989976 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f004c104240 0x7f004c19a600 unknown :-1 s=CLOSED pgs=716 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:27:36.272 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:36.271+0000 7f0052c00640 1 --2- 172.21.15.47:0/3296989976 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f004c103880 0x7f004c19a0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:27:36.272 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:36.271+0000 7f0052c00640 1 -- 172.21.15.47:0/3296989976 >> 172.21.15.47:0/3296989976 conn(0x7f004c0ff3b0 msgr2=0x7f004c0ffb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:27:36.272 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:36.271+0000 7f0052c00640 1 -- 172.21.15.47:0/3296989976 shutdown_connections 2024-03-21T10:27:36.272 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:36.271+0000 7f0052c00640 1 -- 172.21.15.47:0/3296989976 wait complete. 2024-03-21T10:27:37.712 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:27:37 smithi047 ceph-mon[31547]: pgmap v72: 129 pgs: 129 active+clean; 246 MiB data, 877 MiB used, 1.0 TiB / 1.0 TiB avail; 9.4 KiB/s rd, 8.7 MiB/s wr, 18 op/s 2024-03-21T10:27:37.728 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:27:37 smithi100 ceph-mon[35193]: pgmap v72: 129 pgs: 129 active+clean; 246 MiB data, 877 MiB used, 1.0 TiB / 1.0 TiB avail; 9.4 KiB/s rd, 8.7 MiB/s wr, 18 op/s 2024-03-21T10:27:37.738 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:27:37 smithi157 ceph-mon[35581]: pgmap v72: 129 pgs: 129 active+clean; 246 MiB data, 877 MiB used, 1.0 TiB / 1.0 TiB avail; 9.4 KiB/s rd, 8.7 MiB/s wr, 18 op/s 2024-03-21T10:27:39.712 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:27:39 smithi047 ceph-mon[31547]: pgmap v73: 129 pgs: 129 active+clean; 246 MiB data, 877 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 5.0 MiB/s wr, 9 op/s 2024-03-21T10:27:39.728 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:27:39 smithi100 ceph-mon[35193]: pgmap v73: 129 pgs: 129 active+clean; 246 MiB data, 877 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 5.0 MiB/s wr, 9 op/s 2024-03-21T10:27:39.738 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:27:39 smithi157 ceph-mon[35581]: pgmap v73: 129 pgs: 129 active+clean; 246 MiB data, 877 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 5.0 MiB/s wr, 9 op/s 2024-03-21T10:27:41.728 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:27:41 smithi100 ceph-mon[35193]: pgmap v74: 129 pgs: 129 active+clean; 246 MiB data, 877 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 5.0 MiB/s wr, 9 op/s 2024-03-21T10:27:41.738 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:27:41 smithi157 ceph-mon[35581]: pgmap v74: 129 pgs: 129 active+clean; 246 MiB data, 877 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 5.0 MiB/s wr, 9 op/s 2024-03-21T10:27:41.962 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:27:41 smithi047 ceph-mon[31547]: pgmap v74: 129 pgs: 129 active+clean; 246 MiB data, 877 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 5.0 MiB/s wr, 9 op/s 2024-03-21T10:27:43.728 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:27:43 smithi100 ceph-mon[35193]: pgmap v75: 129 pgs: 129 active+clean; 260 MiB data, 939 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 7.5 MiB/s wr, 291 op/s 2024-03-21T10:27:43.738 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:27:43 smithi157 ceph-mon[35581]: pgmap v75: 129 pgs: 129 active+clean; 260 MiB data, 939 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 7.5 MiB/s wr, 291 op/s 2024-03-21T10:27:43.962 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:27:43 smithi047 ceph-mon[31547]: pgmap v75: 129 pgs: 129 active+clean; 260 MiB data, 939 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 7.5 MiB/s wr, 291 op/s 2024-03-21T10:27:45.738 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:27:45 smithi157 ceph-mon[35581]: pgmap v76: 129 pgs: 129 active+clean; 260 MiB data, 939 MiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 5.1 MiB/s wr, 286 op/s 2024-03-21T10:27:45.962 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:27:45 smithi047 ceph-mon[31547]: pgmap v76: 129 pgs: 129 active+clean; 260 MiB data, 939 MiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 5.1 MiB/s wr, 286 op/s 2024-03-21T10:27:45.978 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:27:45 smithi100 ceph-mon[35193]: pgmap v76: 129 pgs: 129 active+clean; 260 MiB data, 939 MiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 5.1 MiB/s wr, 286 op/s 2024-03-21T10:27:47.738 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:27:47 smithi157 ceph-mon[35581]: pgmap v77: 129 pgs: 129 active+clean; 266 MiB data, 965 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 8.9 MiB/s wr, 1.11k op/s 2024-03-21T10:27:47.962 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:27:47 smithi047 ceph-mon[31547]: pgmap v77: 129 pgs: 129 active+clean; 266 MiB data, 965 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 8.9 MiB/s wr, 1.11k op/s 2024-03-21T10:27:47.978 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:27:47 smithi100 ceph-mon[35193]: pgmap v77: 129 pgs: 129 active+clean; 266 MiB data, 965 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 8.9 MiB/s wr, 1.11k op/s 2024-03-21T10:27:49.474 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:49.474+0000 7f588ca00640 1 -- 172.21.15.47:0/931981862 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f588810fde0 msgr2=0x7f588810da00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:27:49.474 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:49.474+0000 7f588ca00640 1 --2- 172.21.15.47:0/931981862 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f588810fde0 0x7f588810da00 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f5878009860 tx=0x7f587802bb70 comp rx=0 tx=0).stop 2024-03-21T10:27:49.474 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:49.474+0000 7f588ca00640 1 -- 172.21.15.47:0/931981862 shutdown_connections 2024-03-21T10:27:49.474 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:49.474+0000 7f588ca00640 1 --2- 172.21.15.47:0/931981862 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f588811ae80 0x7f588811d270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:27:49.475 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:49.474+0000 7f588ca00640 1 --2- 172.21.15.47:0/931981862 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5888110250 0x7f588810df40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:27:49.475 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:49.474+0000 7f588ca00640 1 --2- 172.21.15.47:0/931981862 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f588810fde0 0x7f588810da00 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:27:49.475 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:49.474+0000 7f588ca00640 1 -- 172.21.15.47:0/931981862 >> 172.21.15.47:0/931981862 conn(0x7f588806c740 msgr2=0x7f588806cb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:27:49.476 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:49.474+0000 7f588ca00640 1 -- 172.21.15.47:0/931981862 shutdown_connections 2024-03-21T10:27:49.476 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:49.475+0000 7f588ca00640 1 -- 172.21.15.47:0/931981862 wait complete. 2024-03-21T10:27:49.477 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:49.476+0000 7f588ca00640 1 Processor -- start 2024-03-21T10:27:49.477 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:49.477+0000 7f588ca00640 1 -- start start 2024-03-21T10:27:49.479 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:49.479+0000 7f588ca00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f588810fde0 0x7f58881a6d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:27:49.479 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:49.479+0000 7f588ca00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5888110250 0x7f58881a72c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:27:49.479 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:49.479+0000 7f588ca00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f588811ae80 0x7f58881ab640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:27:49.482 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:49.480+0000 7f588ca00640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58881207e0 con 0x7f588810fde0 2024-03-21T10:27:49.483 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:49.480+0000 7f588ca00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5888120660 con 0x7f588811ae80 2024-03-21T10:27:49.483 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:49.480+0000 7f588ca00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5888120960 con 0x7f5888110250 2024-03-21T10:27:49.483 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:49.480+0000 7f587fe00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5888110250 0x7f58881a72c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:27:49.483 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:49.480+0000 7f587fe00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5888110250 0x7f58881a72c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.47:45922/0 (socket says 172.21.15.47:45922) 2024-03-21T10:27:49.483 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:49.480+0000 7f5885e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f588810fde0 0x7f58881a6d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:27:49.483 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:49.480+0000 7f587fe00640 1 -- 172.21.15.47:0/1306351155 learned_addr learned my addr 172.21.15.47:0/1306351155 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:27:49.483 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:49.480+0000 7f5885e00640 1 -- 172.21.15.47:0/1306351155 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5888110250 msgr2=0x7f58881a72c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:27:49.483 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:49.480+0000 7f5885e00640 1 --2- 172.21.15.47:0/1306351155 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5888110250 0x7f58881a72c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:27:49.483 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:49.480+0000 7f5885e00640 1 -- 172.21.15.47:0/1306351155 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f588811ae80 msgr2=0x7f58881ab640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:27:49.483 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:49.480+0000 7f5885e00640 1 --2- 172.21.15.47:0/1306351155 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f588811ae80 0x7f58881ab640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:27:49.483 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:49.480+0000 7f5885e00640 1 -- 172.21.15.47:0/1306351155 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58881abd70 con 0x7f588810fde0 2024-03-21T10:27:49.483 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:49.481+0000 7f5885e00640 1 --2- 172.21.15.47:0/1306351155 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f588810fde0 0x7f58881a6d80 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7f5878037040 tx=0x7f5878035bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:27:49.484 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:49.481+0000 7f587ea00640 1 -- 172.21.15.47:0/1306351155 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5878041020 con 0x7f588810fde0 2024-03-21T10:27:49.484 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:49.481+0000 7f587ea00640 1 -- 172.21.15.47:0/1306351155 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f587802c9c0 con 0x7f588810fde0 2024-03-21T10:27:49.484 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:49.481+0000 7f587ea00640 1 -- 172.21.15.47:0/1306351155 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5878039040 con 0x7f588810fde0 2024-03-21T10:27:49.484 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:49.482+0000 7f588ca00640 1 -- 172.21.15.47:0/1306351155 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f58881ac000 con 0x7f588810fde0 2024-03-21T10:27:49.484 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:49.482+0000 7f588ca00640 1 -- 172.21.15.47:0/1306351155 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f58881b38f0 con 0x7f588810fde0 2024-03-21T10:27:49.485 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:49.485+0000 7f587ea00640 1 -- 172.21.15.47:0/1306351155 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 27) v1 ==== 137985+0+0 (secure 0 0 0) 0x7f5878044020 con 0x7f588810fde0 2024-03-21T10:27:49.489 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:49.489+0000 7f587ea00640 1 --2- 172.21.15.47:0/1306351155 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f586c0a3200 0x7f586c0a56c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:27:49.489 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:49.489+0000 7f587ea00640 1 -- 172.21.15.47:0/1306351155 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 9474+0+0 (secure 0 0 0) 0x7f58780f13b0 con 0x7f588810fde0 2024-03-21T10:27:49.491 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:49.490+0000 7f587fe00640 1 --2- 172.21.15.47:0/1306351155 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f586c0a3200 0x7f586c0a56c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:27:49.491 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:49.491+0000 7f588f637740 1 Processor -- start 2024-03-21T10:27:49.492 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:49.492+0000 7f588f637740 1 -- start start 2024-03-21T10:27:49.492 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:49.493+0000 7f587fe00640 1 --2- 172.21.15.47:0/1306351155 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f586c0a3200 0x7f586c0a56c0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f58880731c0 tx=0x7f5870009260 comp rx=0 tx=0).ready entity=mgr.15687 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:27:49.493 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:49.493+0000 7f588f637740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fc24979fd0 0x55fc2497a3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:27:49.493 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:49.493+0000 7f588f637740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55fc2497a8f0 0x55fc24983180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:27:49.493 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:49.493+0000 7f588f637740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fc249836c0 0x55fc24985ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:27:49.493 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:49.493+0000 7f588f637740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fc2493c3b0 con 0x55fc2497a8f0 2024-03-21T10:27:49.493 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:49.493+0000 7f588f637740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fc248de830 con 0x55fc24979fd0 2024-03-21T10:27:49.493 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:49.493+0000 7f588f637740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fc248875d0 con 0x55fc249836c0 2024-03-21T10:27:49.494 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:49.494+0000 7f5885e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fc24979fd0 0x55fc2497a3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:27:49.494 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:49.494+0000 7f5885e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fc24979fd0 0x55fc2497a3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:48908/0 (socket says 172.21.15.47:48908) 2024-03-21T10:27:49.494 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:49.494+0000 7f5885e00640 1 -- 172.21.15.47:0/137163299 learned_addr learned my addr 172.21.15.47:0/137163299 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:27:49.495 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:49.494+0000 7f587fe00640 1 --2- 172.21.15.47:0/137163299 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55fc2497a8f0 0x55fc24983180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:27:49.495 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:49.495+0000 7f5886800640 1 --2- 172.21.15.47:0/137163299 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fc249836c0 0x55fc24985ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:27:49.495 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:49.495+0000 7f5885e00640 1 -- 172.21.15.47:0/137163299 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fc249836c0 msgr2=0x55fc24985ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:27:49.495 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:49.495+0000 7f5885e00640 1 --2- 172.21.15.47:0/137163299 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fc249836c0 0x55fc24985ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:27:49.495 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:49.495+0000 7f5885e00640 1 -- 172.21.15.47:0/137163299 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55fc2497a8f0 msgr2=0x55fc24983180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:27:49.495 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:49.495+0000 7f5885e00640 1 --2- 172.21.15.47:0/137163299 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55fc2497a8f0 0x55fc24983180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:27:49.495 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:49.495+0000 7f5885e00640 1 -- 172.21.15.47:0/137163299 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55fc2489f2c0 con 0x55fc24979fd0 2024-03-21T10:27:49.496 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:49.496+0000 7f5885e00640 1 --2- 172.21.15.47:0/137163299 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fc24979fd0 0x55fc2497a3b0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f58780bd480 tx=0x7f587802bb70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1af88ebbfce3d3a9 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:27:49.499 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:49.499+0000 7f5877000640 1 -- 172.21.15.47:0/137163299 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58780bd4f0 con 0x55fc24979fd0 2024-03-21T10:27:49.499 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:49.500+0000 7f5877000640 1 -- 172.21.15.47:0/137163299 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f58780c1640 con 0x55fc24979fd0 2024-03-21T10:27:49.500 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:49.500+0000 7f5877000640 1 -- 172.21.15.47:0/137163299 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58780b9b60 con 0x55fc24979fd0 2024-03-21T10:27:49.500 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:49.501+0000 7f588f637740 1 -- 172.21.15.47:0/137163299 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fc24979fd0 msgr2=0x55fc2497a3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:27:49.500 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:49.501+0000 7f588f637740 1 --2- 172.21.15.47:0/137163299 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fc24979fd0 0x55fc2497a3b0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f58780bd480 tx=0x7f587802bb70 comp rx=0 tx=0).stop 2024-03-21T10:27:49.501 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:49.501+0000 7f588f637740 1 -- 172.21.15.47:0/137163299 shutdown_connections 2024-03-21T10:27:49.501 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:49.501+0000 7f588f637740 1 --2- 172.21.15.47:0/137163299 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fc249836c0 0x55fc24985ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:27:49.501 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:49.501+0000 7f588f637740 1 --2- 172.21.15.47:0/137163299 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55fc2497a8f0 0x55fc24983180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:27:49.501 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:49.501+0000 7f588f637740 1 --2- 172.21.15.47:0/137163299 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fc24979fd0 0x55fc2497a3b0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:27:49.501 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:49.501+0000 7f588f637740 1 -- 172.21.15.47:0/137163299 >> 172.21.15.47:0/137163299 conn(0x55fc249748f0 msgr2=0x55fc24974cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:27:49.501 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:49.501+0000 7f588f637740 1 -- 172.21.15.47:0/137163299 shutdown_connections 2024-03-21T10:27:49.502 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:49.501+0000 7f588f637740 1 -- 172.21.15.47:0/137163299 wait complete. 2024-03-21T10:27:49.502 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:49.502+0000 7f588f637740 1 -- start start 2024-03-21T10:27:49.502 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:49.502+0000 7f588f637740 1 Processor -- start 2024-03-21T10:27:49.502 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:49.503+0000 7f588f637740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fc24979fd0 0x55fc249807c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:27:49.502 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:49.503+0000 7f588f637740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fc2497a8f0 0x55fc24980d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:27:49.503 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:49.503+0000 7f588f637740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55fc249836c0 0x55fc24981240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:27:49.503 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:49.503+0000 7f588f637740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fc2489aa20 con 0x55fc249836c0 2024-03-21T10:27:49.503 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:49.503+0000 7f588f637740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fc248da8d0 con 0x55fc2497a8f0 2024-03-21T10:27:49.503 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:49.503+0000 7f588f637740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fc249860b0 con 0x55fc24979fd0 2024-03-21T10:27:49.503 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:49.503+0000 7f588f637740 10 client.0 _ll_register_callbacks cb 0x55fc248d71c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T10:27:49.503 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:49.503+0000 7f587fe00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fc2497a8f0 0x55fc24980d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:27:49.504 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:49.503+0000 7f5885e00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fc24979fd0 0x55fc249807c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:27:49.504 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:49.503+0000 7f5886800640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55fc249836c0 0x55fc24981240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:27:49.504 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:49.503+0000 7f5885e00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fc24979fd0 0x55fc249807c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.47:45946/0 (socket says 172.21.15.47:45946) 2024-03-21T10:27:49.504 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:49.503+0000 7f5885e00640 1 -- 172.21.15.47:0/1957129609 learned_addr learned my addr 172.21.15.47:0/1957129609 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:27:49.504 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:49.503+0000 7f587fe00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fc2497a8f0 0x55fc24980d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:48916/0 (socket says 172.21.15.47:48916) 2024-03-21T10:27:49.505 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:49.504+0000 7f5885e00640 1 -- 172.21.15.47:0/1957129609 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fc2497a8f0 msgr2=0x55fc24980d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:27:49.505 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:49.504+0000 7f5885e00640 1 --2- 172.21.15.47:0/1957129609 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fc2497a8f0 0x55fc24980d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:27:49.505 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:49.504+0000 7f5885e00640 1 -- 172.21.15.47:0/1957129609 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55fc249836c0 msgr2=0x55fc24981240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:27:49.505 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:49.504+0000 7f5885e00640 1 --2- 172.21.15.47:0/1957129609 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55fc249836c0 0x55fc24981240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:27:49.505 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:49.504+0000 7f5885e00640 1 -- 172.21.15.47:0/1957129609 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55fc245f3210 con 0x55fc24979fd0 2024-03-21T10:27:49.505 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:49.505+0000 7f5885e00640 1 --2- 172.21.15.47:0/1957129609 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fc24979fd0 0x55fc249807c0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f58780b9950 tx=0x7f58780b9d90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=780ed939cd753c29 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:27:49.505 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:49.505+0000 7f586b400640 10 client.0 ms_handle_connect on v2:172.21.15.157:3300/0 2024-03-21T10:27:49.505 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:49.505+0000 7f586b400640 1 -- 172.21.15.47:0/1957129609 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58780d6070 con 0x55fc24979fd0 2024-03-21T10:27:49.505 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:49.505+0000 7f586b400640 1 -- 172.21.15.47:0/1957129609 <== mon.1 v2:172.21.15.157:3300/0 2 ==== osd_map(97..97 src has 1..97) v4 ==== 9474+0+0 (secure 0 0 0) 0x7f587804d300 con 0x55fc24979fd0 2024-03-21T10:27:49.505 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:49.505+0000 7f588f637740 1 -- 172.21.15.47:0/1957129609 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f58780c1640 con 0x55fc24979fd0 2024-03-21T10:27:49.506 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:49.506+0000 7f586b400640 1 -- 172.21.15.47:0/1957129609 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f587804d500 con 0x55fc24979fd0 2024-03-21T10:27:49.506 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:49.506+0000 7f588f637740 10 client.26098 fetch_fsmap learned FSMap version 32 2024-03-21T10:27:49.506 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:49.506+0000 7f588f637740 1 -- 172.21.15.47:0/1957129609 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x55fc246bf620 con 0x55fc24979fd0 2024-03-21T10:27:49.506 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:49.506+0000 7f586b400640 1 -- 172.21.15.47:0/1957129609 <== mon.1 v2:172.21.15.157:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f58780c1640 con 0x55fc24979fd0 2024-03-21T10:27:49.507 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:49.506+0000 7f586b400640 1 -- 172.21.15.47:0/1957129609 <== mon.1 v2:172.21.15.157:3300/0 5 ==== fsmap(e 32) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f587803c030 con 0x55fc24979fd0 2024-03-21T10:27:49.507 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:49.506+0000 7f588f637740 10 client.26098 fetch_fsmap finished waiting for FSMap version 32 2024-03-21T10:27:49.507 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:49.506+0000 7f588f637740 10 client.26098 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-21T10:27:49.507 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:49.506+0000 7f588f637740 20 client.26098 populate_metadata read hostname 'smithi047' 2024-03-21T10:27:49.507 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:49.507+0000 7f588f637740 1 --2- 172.21.15.47:0/1957129609 >> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] conn(0x55fc24993360 0x55fc24995750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:27:49.507 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:49.507+0000 7f588f637740 4 client.26098 mds_command: new command op to 34535 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T10:27:49.507 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:49.507+0000 7f588f637740 1 -- 172.21.15.47:0/1957129609 --> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55fc248ebd70 con 0x55fc24993360 2024-03-21T10:27:49.508 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:49.507+0000 7f587fe00640 1 --2- 172.21.15.47:0/1957129609 >> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] conn(0x55fc24993360 0x55fc24995750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:27:49.508 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:49.508+0000 7f586b400640 10 client.26098 ms_handle_connect on v2:172.21.15.100:6838/4173168440 2024-03-21T10:27:49.508 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:49.508+0000 7f587fe00640 1 --2- 172.21.15.47:0/1957129609 >> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] conn(0x55fc24993360 0x55fc24995750 crc :-1 s=READY pgs=97 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=43eba09235486c78 server_cookie=16f0768aae2bbe5d in_seq=0 out_seq=0 2024-03-21T10:27:49.509 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:49.509+0000 7f586b400640 1 -- 172.21.15.47:0/1957129609 <== mds.0 v2:172.21.15.100:6838/4173168440 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55fc248ebd70 con 0x55fc24993360 2024-03-21T10:27:49.509 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:49.509+0000 7f586b400640 10 client.26098 handle_command_reply: tid=0 2024-03-21T10:27:49.509 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:49.509+0000 7f588f637740 1 client.26098 shutdown 2024-03-21T10:27:49.510 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:49.511+0000 7f588f637740 1 -- 172.21.15.47:0/1957129609 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fc24979fd0 msgr2=0x55fc249807c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:27:49.510 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:49.511+0000 7f588f637740 1 --2- 172.21.15.47:0/1957129609 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fc24979fd0 0x55fc249807c0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f58780b9950 tx=0x7f58780b9d90 comp rx=0 tx=0).stop 2024-03-21T10:27:49.511 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:49.511+0000 7f588f637740 1 -- 172.21.15.47:0/1957129609 shutdown_connections 2024-03-21T10:27:49.511 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:49.511+0000 7f588f637740 1 --2- 172.21.15.47:0/1957129609 >> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] conn(0x55fc24993360 0x55fc24995750 crc :-1 s=READY pgs=97 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:27:49.511 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:49.511+0000 7f588f637740 1 --2- 172.21.15.47:0/1957129609 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55fc249836c0 0x55fc24981240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:27:49.512 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:49.511+0000 7f588f637740 1 --2- 172.21.15.47:0/1957129609 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fc2497a8f0 0x55fc24980d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:27:49.512 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:49.511+0000 7f588f637740 1 --2- 172.21.15.47:0/1957129609 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fc24979fd0 0x55fc249807c0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:27:49.512 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:49.511+0000 7f588f637740 1 -- 172.21.15.47:0/1957129609 >> 172.21.15.47:0/1957129609 conn(0x55fc249748f0 msgr2=0x55fc24976640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:27:49.512 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:49.511+0000 7f586b400640 0 client.26098 ms_handle_reset on v2:172.21.15.100:6838/4173168440 2024-03-21T10:27:49.512 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:49.512+0000 7f588f637740 1 -- 172.21.15.47:0/1957129609 shutdown_connections 2024-03-21T10:27:49.512 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:49.512+0000 7f588f637740 1 -- 172.21.15.47:0/1957129609 wait complete. 2024-03-21T10:27:49.512 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:49.512+0000 7f588f637740 20 client.26098 trim_cache size 0 max 16384 2024-03-21T10:27:49.542 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:49.543+0000 7f588f637740 1 Processor -- start 2024-03-21T10:27:49.543 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:49.543+0000 7f588f637740 1 -- start start 2024-03-21T10:27:49.543 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:49.543+0000 7f588f637740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fc24979fd0 0x55fc24982e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:27:49.543 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:49.543+0000 7f588f637740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55fc2497a8f0 0x55fc24976450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:27:49.543 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:49.543+0000 7f588f637740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fc249836c0 0x55fc2498e710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:27:49.543 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:49.543+0000 7f588f637740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58881bd7e0 con 0x55fc2497a8f0 2024-03-21T10:27:49.544 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:49.543+0000 7f588f637740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58881bf160 con 0x55fc24979fd0 2024-03-21T10:27:49.544 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:49.543+0000 7f588f637740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58881bde40 con 0x55fc249836c0 2024-03-21T10:27:49.544 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:49.544+0000 7f5885e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fc24979fd0 0x55fc24982e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:27:49.544 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:49.544+0000 7f587fe00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55fc2497a8f0 0x55fc24976450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:27:49.544 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:49.544+0000 7f587fe00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55fc2497a8f0 0x55fc24976450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:44444/0 (socket says 172.21.15.47:44444) 2024-03-21T10:27:49.544 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:49.544+0000 7f587fe00640 1 -- 172.21.15.47:0/676711309 learned_addr learned my addr 172.21.15.47:0/676711309 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:27:49.544 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:49.544+0000 7f5886800640 1 --2- 172.21.15.47:0/676711309 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fc249836c0 0x55fc2498e710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:27:49.545 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:49.545+0000 7f587fe00640 1 -- 172.21.15.47:0/676711309 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fc249836c0 msgr2=0x55fc2498e710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:27:49.545 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:49.545+0000 7f587fe00640 1 --2- 172.21.15.47:0/676711309 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fc249836c0 0x55fc2498e710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:27:49.545 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:49.545+0000 7f587fe00640 1 -- 172.21.15.47:0/676711309 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fc24979fd0 msgr2=0x55fc24982e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:27:49.545 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:49.545+0000 7f587fe00640 1 --2- 172.21.15.47:0/676711309 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fc24979fd0 0x55fc24982e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:27:49.545 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:49.545+0000 7f587fe00640 1 -- 172.21.15.47:0/676711309 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55fc2497ed30 con 0x55fc2497a8f0 2024-03-21T10:27:49.546 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:49.546+0000 7f587fe00640 1 --2- 172.21.15.47:0/676711309 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55fc2497a8f0 0x55fc24976450 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7f5870009a10 tx=0x7f5870014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b92aa9dddb447f4c server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:27:49.546 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:49.546+0000 7f5876600640 1 -- 172.21.15.47:0/676711309 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58700297e0 con 0x55fc2497a8f0 2024-03-21T10:27:49.549 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:49.546+0000 7f5876600640 1 -- 172.21.15.47:0/676711309 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f587002f070 con 0x55fc2497a8f0 2024-03-21T10:27:49.549 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:49.546+0000 7f5876600640 1 -- 172.21.15.47:0/676711309 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5870020c40 con 0x55fc2497a8f0 2024-03-21T10:27:49.549 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:49.548+0000 7f588f637740 1 -- 172.21.15.47:0/676711309 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55fc2497a8f0 msgr2=0x55fc24976450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:27:49.549 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:49.548+0000 7f588f637740 1 --2- 172.21.15.47:0/676711309 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55fc2497a8f0 0x55fc24976450 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7f5870009a10 tx=0x7f5870014000 comp rx=0 tx=0).stop 2024-03-21T10:27:49.549 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:49.548+0000 7f588f637740 1 -- 172.21.15.47:0/676711309 shutdown_connections 2024-03-21T10:27:49.550 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:49.548+0000 7f588f637740 1 --2- 172.21.15.47:0/676711309 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fc249836c0 0x55fc2498e710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:27:49.550 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:49.548+0000 7f588f637740 1 --2- 172.21.15.47:0/676711309 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55fc2497a8f0 0x55fc24976450 unknown :-1 s=CLOSED pgs=721 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:27:49.550 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:49.548+0000 7f588f637740 1 --2- 172.21.15.47:0/676711309 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fc24979fd0 0x55fc24982e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:27:49.550 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:49.548+0000 7f588f637740 1 -- 172.21.15.47:0/676711309 >> 172.21.15.47:0/676711309 conn(0x55fc249748f0 msgr2=0x55fc24975d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:27:49.550 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:49.550+0000 7f588f637740 1 -- 172.21.15.47:0/676711309 shutdown_connections 2024-03-21T10:27:49.550 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:49.550+0000 7f588f637740 1 -- 172.21.15.47:0/676711309 wait complete. 2024-03-21T10:27:49.551 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:49.551+0000 7f588f637740 1 -- start start 2024-03-21T10:27:49.551 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:49.551+0000 7f588f637740 1 Processor -- start 2024-03-21T10:27:49.552 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:49.552+0000 7f588f637740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fc24979fd0 0x55fc2498ac60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:27:49.552 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:49.552+0000 7f588f637740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fc2497a8f0 0x55fc2498b1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:27:49.552 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:49.552+0000 7f588f637740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55fc249836c0 0x55fc2497da20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:27:49.552 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:49.552+0000 7f588f637740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58881beb00 con 0x55fc249836c0 2024-03-21T10:27:49.552 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:49.552+0000 7f588f637740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58881be4a0 con 0x55fc2497a8f0 2024-03-21T10:27:49.552 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:49.552+0000 7f588f637740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58881bb9a0 con 0x55fc24979fd0 2024-03-21T10:27:49.552 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:49.552+0000 7f588f637740 10 client.0 _ll_register_callbacks cb 0x7f58780c1070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T10:27:49.553 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:49.552+0000 7f587fe00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fc2497a8f0 0x55fc2498b1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:27:49.553 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:49.552+0000 7f5885e00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fc24979fd0 0x55fc2498ac60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:27:49.553 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:49.552+0000 7f5886800640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55fc249836c0 0x55fc2497da20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:27:49.553 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:49.553+0000 7f587fe00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fc2497a8f0 0x55fc2498b1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:48934/0 (socket says 172.21.15.47:48934) 2024-03-21T10:27:49.553 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:49.553+0000 7f587fe00640 1 -- 172.21.15.47:0/292778806 learned_addr learned my addr 172.21.15.47:0/292778806 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:27:49.553 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:49.554+0000 7f5886800640 1 -- 172.21.15.47:0/292778806 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fc24979fd0 msgr2=0x55fc2498ac60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:27:49.553 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:49.554+0000 7f5886800640 1 --2- 172.21.15.47:0/292778806 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fc24979fd0 0x55fc2498ac60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:27:49.554 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:49.554+0000 7f5886800640 1 -- 172.21.15.47:0/292778806 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fc2497a8f0 msgr2=0x55fc2498b1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:27:49.554 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:49.554+0000 7f5886800640 1 --2- 172.21.15.47:0/292778806 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fc2497a8f0 0x55fc2498b1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:27:49.554 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:49.554+0000 7f5886800640 1 -- 172.21.15.47:0/292778806 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55fc2497e200 con 0x55fc249836c0 2024-03-21T10:27:49.555 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:49.556+0000 7f5886800640 1 --2- 172.21.15.47:0/292778806 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55fc249836c0 0x55fc2497da20 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x55fc2496f4c0 tx=0x7f588000ae70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ac5f8497347856a6 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:27:49.556 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:49.556+0000 7f586b400640 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-03-21T10:27:49.557 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:49.557+0000 7f586b400640 1 -- 172.21.15.47:0/292778806 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5880007c50 con 0x55fc249836c0 2024-03-21T10:27:49.557 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:49.557+0000 7f586b400640 1 -- 172.21.15.47:0/292778806 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(97..97 src has 1..97) v4 ==== 9474+0+0 (secure 0 0 0) 0x7f5880007df0 con 0x55fc249836c0 2024-03-21T10:27:49.557 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:49.557+0000 7f588f637740 1 -- 172.21.15.47:0/292778806 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f587002f070 con 0x55fc249836c0 2024-03-21T10:27:49.558 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:49.559+0000 7f586b400640 1 -- 172.21.15.47:0/292778806 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5880018470 con 0x55fc249836c0 2024-03-21T10:27:49.559 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:49.559+0000 7f588f637740 10 client.16557 fetch_fsmap learned FSMap version 32 2024-03-21T10:27:49.559 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:49.559+0000 7f588f637740 1 -- 172.21.15.47:0/292778806 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x55fc24a006f0 con 0x55fc249836c0 2024-03-21T10:27:49.560 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:49.560+0000 7f586b400640 1 -- 172.21.15.47:0/292778806 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f587002f070 con 0x55fc249836c0 2024-03-21T10:27:49.560 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:49.560+0000 7f586b400640 1 -- 172.21.15.47:0/292778806 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 32) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f58800178e0 con 0x55fc249836c0 2024-03-21T10:27:49.560 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:49.560+0000 7f588f637740 10 client.16557 fetch_fsmap finished waiting for FSMap version 32 2024-03-21T10:27:49.560 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:49.560+0000 7f588f637740 10 client.16557 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-21T10:27:49.560 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:49.560+0000 7f588f637740 20 client.16557 populate_metadata read hostname 'smithi047' 2024-03-21T10:27:49.560 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:49.560+0000 7f588f637740 1 --2- 172.21.15.47:0/292778806 >> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] conn(0x55fc24a00ea0 0x55fc24a032c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:27:49.560 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:49.560+0000 7f588f637740 4 client.16557 mds_command: new command op to 34535 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-21T10:27:49.560 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:49.560+0000 7f588f637740 1 -- 172.21.15.47:0/292778806 --> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55fc248e60c0 con 0x55fc24a00ea0 2024-03-21T10:27:49.561 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:49.561+0000 7f5885e00640 1 --2- 172.21.15.47:0/292778806 >> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] conn(0x55fc24a00ea0 0x55fc24a032c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:27:49.563 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:49.564+0000 7f5885e00640 1 --2- 172.21.15.47:0/292778806 >> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] conn(0x55fc24a00ea0 0x55fc24a032c0 crc :-1 s=READY pgs=98 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2182a1ec889f7989 server_cookie=e087f9e4fc31e1ff in_seq=0 out_seq=0 2024-03-21T10:27:49.563 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:49.564+0000 7f586b400640 10 client.16557 ms_handle_connect on v2:172.21.15.100:6838/4173168440 2024-03-21T10:27:49.619 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:49.618+0000 7f586b400640 1 -- 172.21.15.47:0/292778806 <== mds.0 v2:172.21.15.100:6838/4173168440 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55fc248e60c0 con 0x55fc24a00ea0 2024-03-21T10:27:49.619 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:49.619+0000 7f586b400640 10 client.16557 handle_command_reply: tid=0 2024-03-21T10:27:49.619 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:49.619+0000 7f588f637740 1 client.16557 shutdown 2024-03-21T10:27:49.619 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:49.619+0000 7f588f637740 1 -- 172.21.15.47:0/292778806 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55fc249836c0 msgr2=0x55fc2497da20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:27:49.620 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:49.619+0000 7f588f637740 1 --2- 172.21.15.47:0/292778806 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55fc249836c0 0x55fc2497da20 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x55fc2496f4c0 tx=0x7f588000ae70 comp rx=0 tx=0).stop 2024-03-21T10:27:49.620 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:49.620+0000 7f588f637740 1 -- 172.21.15.47:0/292778806 shutdown_connections 2024-03-21T10:27:49.620 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:49.620+0000 7f588f637740 1 --2- 172.21.15.47:0/292778806 >> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] conn(0x55fc24a00ea0 0x55fc24a032c0 crc :-1 s=READY pgs=98 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:27:49.620 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:49.620+0000 7f586b400640 0 client.16557 ms_handle_reset on v2:172.21.15.100:6838/4173168440 2024-03-21T10:27:49.620 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:49.620+0000 7f588f637740 1 --2- 172.21.15.47:0/292778806 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55fc249836c0 0x55fc2497da20 unknown :-1 s=CLOSED pgs=722 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:27:49.620 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:49.620+0000 7f588f637740 1 --2- 172.21.15.47:0/292778806 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fc2497a8f0 0x55fc2498b1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:27:49.620 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:49.620+0000 7f588f637740 1 --2- 172.21.15.47:0/292778806 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fc24979fd0 0x55fc2498ac60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:27:49.620 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:49.620+0000 7f588f637740 1 -- 172.21.15.47:0/292778806 >> 172.21.15.47:0/292778806 conn(0x55fc249748f0 msgr2=0x55fc249765d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:27:49.621 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:49.620+0000 7f588f637740 1 -- 172.21.15.47:0/292778806 shutdown_connections 2024-03-21T10:27:49.621 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:49.620+0000 7f588f637740 1 -- 172.21.15.47:0/292778806 wait complete. 2024-03-21T10:27:49.621 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:49.620+0000 7f588f637740 20 client.16557 trim_cache size 0 max 16384 2024-03-21T10:27:49.621 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout:{ 2024-03-21T10:27:49.621 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout: "message": "", 2024-03-21T10:27:49.622 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout: "return_code": 0 2024-03-21T10:27:49.622 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout:} 2024-03-21T10:27:49.622 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:49.622+0000 7f588ca00640 1 -- 172.21.15.47:0/1306351155 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f586c0a3200 msgr2=0x7f586c0a56c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:27:49.622 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:49.622+0000 7f588ca00640 1 --2- 172.21.15.47:0/1306351155 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f586c0a3200 0x7f586c0a56c0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f58880731c0 tx=0x7f5870009260 comp rx=0 tx=0).stop 2024-03-21T10:27:49.622 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:49.622+0000 7f588ca00640 1 -- 172.21.15.47:0/1306351155 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f588810fde0 msgr2=0x7f58881a6d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:27:49.622 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:49.622+0000 7f588ca00640 1 --2- 172.21.15.47:0/1306351155 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f588810fde0 0x7f58881a6d80 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7f5878037040 tx=0x7f5878035bc0 comp rx=0 tx=0).stop 2024-03-21T10:27:49.624 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:49.624+0000 7f588ca00640 1 -- 172.21.15.47:0/1306351155 shutdown_connections 2024-03-21T10:27:49.624 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:49.624+0000 7f588ca00640 1 --2- 172.21.15.47:0/1306351155 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f586c0a3200 0x7f586c0a56c0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:27:49.624 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:49.624+0000 7f588ca00640 1 --2- 172.21.15.47:0/1306351155 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f588811ae80 0x7f58881ab640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:27:49.624 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:49.624+0000 7f588ca00640 1 --2- 172.21.15.47:0/1306351155 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5888110250 0x7f58881a72c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:27:49.624 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:49.624+0000 7f588ca00640 1 --2- 172.21.15.47:0/1306351155 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f588810fde0 0x7f58881a6d80 unknown :-1 s=CLOSED pgs=720 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:27:49.624 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:49.624+0000 7f588ca00640 1 -- 172.21.15.47:0/1306351155 >> 172.21.15.47:0/1306351155 conn(0x7f588806c740 msgr2=0x7f5888072160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:27:49.625 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:49.625+0000 7f588ca00640 1 -- 172.21.15.47:0/1306351155 shutdown_connections 2024-03-21T10:27:49.625 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:27:49.625+0000 7f588ca00640 1 -- 172.21.15.47:0/1306351155 wait complete. 2024-03-21T10:27:49.780 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:27:49 smithi100 ceph-mon[35193]: pgmap v78: 129 pgs: 129 active+clean; 266 MiB data, 965 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 6.3 MiB/s wr, 1.10k op/s 2024-03-21T10:27:49.963 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:27:49 smithi047 ceph-mon[31547]: pgmap v78: 129 pgs: 129 active+clean; 266 MiB data, 965 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 6.3 MiB/s wr, 1.10k op/s 2024-03-21T10:27:49.988 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:27:49 smithi157 ceph-mon[35581]: pgmap v78: 129 pgs: 129 active+clean; 266 MiB data, 965 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 6.3 MiB/s wr, 1.10k op/s 2024-03-21T10:27:51.962 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:27:51 smithi047 ceph-mon[31547]: pgmap v79: 129 pgs: 129 active+clean; 266 MiB data, 965 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 6.3 MiB/s wr, 1.10k op/s 2024-03-21T10:27:51.979 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:27:51 smithi100 ceph-mon[35193]: pgmap v79: 129 pgs: 129 active+clean; 266 MiB data, 965 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 6.3 MiB/s wr, 1.10k op/s 2024-03-21T10:27:52.045 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:27:51 smithi157 ceph-mon[35581]: pgmap v79: 129 pgs: 129 active+clean; 266 MiB data, 965 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 6.3 MiB/s wr, 1.10k op/s 2024-03-21T10:27:53.765 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:27:53 smithi047 ceph-mon[31547]: pgmap v80: 129 pgs: 129 active+clean; 270 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 10 MiB/s wr, 1.91k op/s 2024-03-21T10:27:53.978 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:27:53 smithi100 ceph-mon[35193]: pgmap v80: 129 pgs: 129 active+clean; 270 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 10 MiB/s wr, 1.91k op/s 2024-03-21T10:27:53.988 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:27:53 smithi157 ceph-mon[35581]: pgmap v80: 129 pgs: 129 active+clean; 270 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 10 MiB/s wr, 1.91k op/s 2024-03-21T10:27:55.660 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-21T10:27:55.660 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-21T10:27:55.782 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:27:55 smithi047 ceph-mon[31547]: pgmap v81: 129 pgs: 129 active+clean; 270 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 7.5 MiB/s wr, 1.62k op/s 2024-03-21T10:27:55.971 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:55.970+0000 7f01b0800640 1 -- 172.21.15.47:0/1227632842 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f01a810cf80 msgr2=0x7f01a8114b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:27:55.972 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:55.970+0000 7f01b0800640 1 --2- 172.21.15.47:0/1227632842 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f01a810cf80 0x7f01a8114b50 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f01a4009860 tx=0x7f01a402bb70 comp rx=0 tx=0).stop 2024-03-21T10:27:55.972 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:55.970+0000 7f01b0800640 1 -- 172.21.15.47:0/1227632842 shutdown_connections 2024-03-21T10:27:55.972 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:55.970+0000 7f01b0800640 1 --2- 172.21.15.47:0/1227632842 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f01a810cf80 0x7f01a8114b50 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:27:55.972 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:55.970+0000 7f01b0800640 1 --2- 172.21.15.47:0/1227632842 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f01a8108860 0x7f01a810c8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:27:55.972 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:55.970+0000 7f01b0800640 1 --2- 172.21.15.47:0/1227632842 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f01a8107eb0 0x7f01a8108290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:27:55.972 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:55.970+0000 7f01b0800640 1 -- 172.21.15.47:0/1227632842 >> 172.21.15.47:0/1227632842 conn(0x7f01a8075be0 msgr2=0x7f01a8078000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:27:55.972 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:55.971+0000 7f01b0800640 1 -- 172.21.15.47:0/1227632842 shutdown_connections 2024-03-21T10:27:55.973 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:55.971+0000 7f01b0800640 1 -- 172.21.15.47:0/1227632842 wait complete. 2024-03-21T10:27:55.973 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:55.972+0000 7f01b0800640 1 Processor -- start 2024-03-21T10:27:55.973 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:55.972+0000 7f01b0800640 1 -- start start 2024-03-21T10:27:55.974 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:55.973+0000 7f01b0800640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f01a8107eb0 0x7f01a819e490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:27:55.974 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:55.973+0000 7f01b0800640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f01a810cf80 0x7f01a819e9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:27:55.974 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:55.973+0000 7f01b0800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f01a81a2d00 0x7f01a81a31b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:27:55.974 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:55.973+0000 7f01b0800640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01a8117f30 con 0x7f01a810cf80 2024-03-21T10:27:55.974 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:55.973+0000 7f01b0800640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01a8117db0 con 0x7f01a81a2d00 2024-03-21T10:27:55.974 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:55.973+0000 7f01b0800640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01a81180b0 con 0x7f01a8107eb0 2024-03-21T10:27:55.975 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:55.973+0000 7f01ad400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f01a810cf80 0x7f01a819e9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:27:55.975 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:55.974+0000 7f01ad400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f01a810cf80 0x7f01a819e9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:57540/0 (socket says 172.21.15.47:57540) 2024-03-21T10:27:55.975 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:55.974+0000 7f01ad400640 1 -- 172.21.15.47:0/3351279439 learned_addr learned my addr 172.21.15.47:0/3351279439 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:27:55.975 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:55.974+0000 7f01ae800640 1 --2- 172.21.15.47:0/3351279439 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f01a81a2d00 0x7f01a81a31b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:27:55.975 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:55.974+0000 7f01ade00640 1 --2- 172.21.15.47:0/3351279439 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f01a8107eb0 0x7f01a819e490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:27:55.975 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:55.974+0000 7f01ad400640 1 -- 172.21.15.47:0/3351279439 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f01a8107eb0 msgr2=0x7f01a819e490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:27:55.976 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:55.974+0000 7f01ad400640 1 --2- 172.21.15.47:0/3351279439 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f01a8107eb0 0x7f01a819e490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:27:55.976 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:55.974+0000 7f01ad400640 1 -- 172.21.15.47:0/3351279439 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f01a81a2d00 msgr2=0x7f01a81a31b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:27:55.976 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:55.974+0000 7f01ad400640 1 --2- 172.21.15.47:0/3351279439 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f01a81a2d00 0x7f01a81a31b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:27:55.976 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:55.974+0000 7f01ad400640 1 -- 172.21.15.47:0/3351279439 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01a81a37f0 con 0x7f01a810cf80 2024-03-21T10:27:55.976 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:55.974+0000 7f01ade00640 1 --2- 172.21.15.47:0/3351279439 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f01a8107eb0 0x7f01a819e490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T10:27:55.976 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:55.975+0000 7f01ad400640 1 --2- 172.21.15.47:0/3351279439 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f01a810cf80 0x7f01a819e9d0 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7f019800e9c0 tx=0x7f019800ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:27:55.977 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:55.975+0000 7f019ea00640 1 -- 172.21.15.47:0/3351279439 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f019800ccc0 con 0x7f01a810cf80 2024-03-21T10:27:55.977 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:55.975+0000 7f019ea00640 1 -- 172.21.15.47:0/3351279439 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f01980054d0 con 0x7f01a810cf80 2024-03-21T10:27:55.977 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:55.975+0000 7f019ea00640 1 -- 172.21.15.47:0/3351279439 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0198010430 con 0x7f01a810cf80 2024-03-21T10:27:55.977 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:55.975+0000 7f01b0800640 1 -- 172.21.15.47:0/3351279439 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f01a81a3ae0 con 0x7f01a810cf80 2024-03-21T10:27:55.977 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:55.976+0000 7f01b0800640 1 -- 172.21.15.47:0/3351279439 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f01a81ab220 con 0x7f01a810cf80 2024-03-21T10:27:55.978 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:27:55 smithi100 ceph-mon[35193]: pgmap v81: 129 pgs: 129 active+clean; 270 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 7.5 MiB/s wr, 1.62k op/s 2024-03-21T10:27:55.984 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:55.982+0000 7f019ea00640 1 -- 172.21.15.47:0/3351279439 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 27) v1 ==== 137985+0+0 (secure 0 0 0) 0x7f01980046c0 con 0x7f01a810cf80 2024-03-21T10:27:55.985 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:55.984+0000 7f019ea00640 1 --2- 172.21.15.47:0/3351279439 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f01940a3150 0x7f01940a5610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:27:55.985 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:55.984+0000 7f019ea00640 1 -- 172.21.15.47:0/3351279439 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 9474+0+0 (secure 0 0 0) 0x7f01980cfec0 con 0x7f01a810cf80 2024-03-21T10:27:55.987 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:55.986+0000 7f01ade00640 1 --2- 172.21.15.47:0/3351279439 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f01940a3150 0x7f01940a5610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:27:55.988 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:27:55 smithi157 ceph-mon[35581]: pgmap v81: 129 pgs: 129 active+clean; 270 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 7.5 MiB/s wr, 1.62k op/s 2024-03-21T10:27:55.992 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:55.992+0000 7f01b35ce740 1 Processor -- start 2024-03-21T10:27:55.994 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:55.993+0000 7f01b35ce740 1 -- start start 2024-03-21T10:27:55.995 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:55.994+0000 7f01b35ce740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a5987a6c00 0x55a5987a5950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:27:55.996 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:55.994+0000 7f01b35ce740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a5987a5f20 0x55a5987a63a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:27:55.996 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:55.994+0000 7f01b35ce740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a5987af290 0x55a5987b1680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:27:55.996 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:55.995+0000 7f01b35ce740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a59870a250 con 0x55a5987af290 2024-03-21T10:27:55.996 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:55.995+0000 7f01b35ce740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a5986b2a30 con 0x55a5987a6c00 2024-03-21T10:27:55.996 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:55.995+0000 7f01b35ce740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a5987062f0 con 0x55a5987a5f20 2024-03-21T10:27:55.996 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:55.995+0000 7f01ade00640 1 --2- 172.21.15.47:0/3351279439 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f01940a3150 0x7f01940a5610 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f01a0005fd0 tx=0x7f01a0009500 comp rx=0 tx=0).ready entity=mgr.15687 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:27:55.997 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:55.995+0000 7f01ad400640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a5987a5f20 0x55a5987a63a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:27:55.997 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:55.995+0000 7f01ad400640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a5987a5f20 0x55a5987a63a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.47:40022/0 (socket says 172.21.15.47:40022) 2024-03-21T10:27:55.997 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:55.995+0000 7f01ad400640 1 -- 172.21.15.47:0/1256382359 learned_addr learned my addr 172.21.15.47:0/1256382359 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:27:55.997 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:55.995+0000 7f01ade00640 1 --2- 172.21.15.47:0/1256382359 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a5987a6c00 0x55a5987a5950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:27:55.997 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:55.995+0000 7f01ad400640 1 -- 172.21.15.47:0/1256382359 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a5987a6c00 msgr2=0x55a5987a5950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:27:55.997 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:55.995+0000 7f01ad400640 1 --2- 172.21.15.47:0/1256382359 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a5987a6c00 0x55a5987a5950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:27:55.997 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:55.995+0000 7f01ad400640 1 -- 172.21.15.47:0/1256382359 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a5987af290 msgr2=0x55a5987b1680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:27:55.997 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:55.995+0000 7f01ad400640 1 --2- 172.21.15.47:0/1256382359 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a5987af290 0x55a5987b1680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:27:55.997 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:55.996+0000 7f01ad400640 1 -- 172.21.15.47:0/1256382359 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a59869d690 con 0x55a5987a5f20 2024-03-21T10:27:55.997 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:55.996+0000 7f01ad400640 1 --2- 172.21.15.47:0/1256382359 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a5987a5f20 0x55a5987a63a0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f0198098350 tx=0x7f019809b940 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f59105f8b1fdf301 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:27:55.998 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:55.996+0000 7f0193000640 1 -- 172.21.15.47:0/1256382359 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01980a0040 con 0x55a5987a5f20 2024-03-21T10:27:55.998 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:55.996+0000 7f0193000640 1 -- 172.21.15.47:0/1256382359 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f01980a4900 con 0x55a5987a5f20 2024-03-21T10:27:55.998 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:55.996+0000 7f0193000640 1 -- 172.21.15.47:0/1256382359 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0198097c40 con 0x55a5987a5f20 2024-03-21T10:27:55.998 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:55.997+0000 7f01b35ce740 1 -- 172.21.15.47:0/1256382359 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a5987a5f20 msgr2=0x55a5987a63a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:27:55.998 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:55.998+0000 7f01b35ce740 1 --2- 172.21.15.47:0/1256382359 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a5987a5f20 0x55a5987a63a0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f0198098350 tx=0x7f019809b940 comp rx=0 tx=0).stop 2024-03-21T10:27:55.999 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:55.998+0000 7f01b35ce740 1 -- 172.21.15.47:0/1256382359 shutdown_connections 2024-03-21T10:27:55.999 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:55.998+0000 7f01b35ce740 1 --2- 172.21.15.47:0/1256382359 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a5987af290 0x55a5987b1680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:27:55.999 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:55.998+0000 7f01b35ce740 1 --2- 172.21.15.47:0/1256382359 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a5987a5f20 0x55a5987a63a0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:27:55.999 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:55.998+0000 7f01b35ce740 1 --2- 172.21.15.47:0/1256382359 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a5987a6c00 0x55a5987a5950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:27:55.999 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:55.998+0000 7f01b35ce740 1 -- 172.21.15.47:0/1256382359 >> 172.21.15.47:0/1256382359 conn(0x55a59876d200 msgr2=0x55a5987a0970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:27:56.000 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:55.998+0000 7f01b35ce740 1 -- 172.21.15.47:0/1256382359 shutdown_connections 2024-03-21T10:27:56.000 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:55.998+0000 7f01b35ce740 1 -- 172.21.15.47:0/1256382359 wait complete. 2024-03-21T10:27:56.000 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:55.999+0000 7f01b35ce740 1 -- start start 2024-03-21T10:27:56.000 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:55.999+0000 7f01b35ce740 1 Processor -- start 2024-03-21T10:27:56.000 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:56.000+0000 7f01b35ce740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a5987a5f20 0x55a5987aea00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:27:56.001 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:56.000+0000 7f01b35ce740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a5987a6c00 0x55a5987abd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:27:56.001 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:56.000+0000 7f01b35ce740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a5987af290 0x55a5987ac2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:27:56.002 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:56.000+0000 7f01ad400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a5987a6c00 0x55a5987abd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:27:56.002 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:56.000+0000 7f01ad400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a5987a6c00 0x55a5987abd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:57560/0 (socket says 172.21.15.47:57560) 2024-03-21T10:27:56.002 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:56.000+0000 7f01ad400640 1 -- 172.21.15.47:0/2946139277 learned_addr learned my addr 172.21.15.47:0/2946139277 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:27:56.002 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:56.001+0000 7f01ae800640 1 --2- 172.21.15.47:0/2946139277 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a5987af290 0x55a5987ac2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:27:56.002 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:56.001+0000 7f01b35ce740 1 -- 172.21.15.47:0/2946139277 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a5987b2100 con 0x55a5987a6c00 2024-03-21T10:27:56.002 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:56.001+0000 7f01b35ce740 1 -- 172.21.15.47:0/2946139277 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a5986c5e80 con 0x55a5987a5f20 2024-03-21T10:27:56.002 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:56.001+0000 7f01b35ce740 1 -- 172.21.15.47:0/2946139277 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a5987b2280 con 0x55a5987af290 2024-03-21T10:27:56.002 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:56.001+0000 7f01b35ce740 10 client.0 _ll_register_callbacks cb 0x55a5986d8130 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T10:27:56.002 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:56.001+0000 7f01ade00640 1 --2- 172.21.15.47:0/2946139277 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a5987a5f20 0x55a5987aea00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:27:56.003 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:56.001+0000 7f01ad400640 1 -- 172.21.15.47:0/2946139277 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a5987af290 msgr2=0x55a5987ac2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:27:56.003 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:56.001+0000 7f01ad400640 1 --2- 172.21.15.47:0/2946139277 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a5987af290 0x55a5987ac2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:27:56.003 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:56.001+0000 7f01ad400640 1 -- 172.21.15.47:0/2946139277 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a5987a5f20 msgr2=0x55a5987aea00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:27:56.003 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:56.001+0000 7f01ad400640 1 --2- 172.21.15.47:0/2946139277 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a5987a5f20 0x55a5987aea00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:27:56.003 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:56.001+0000 7f01ad400640 1 -- 172.21.15.47:0/2946139277 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55a5984e87d0 con 0x55a5987a6c00 2024-03-21T10:27:56.003 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:56.002+0000 7f01ad400640 1 --2- 172.21.15.47:0/2946139277 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a5987a6c00 0x55a5987abd90 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7f019800c6a0 tx=0x7f0198009f50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=882347a667df40d1 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:27:56.003 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:56.002+0000 7f0187400640 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-03-21T10:27:56.004 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:56.002+0000 7f0187400640 1 -- 172.21.15.47:0/2946139277 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01980a8020 con 0x55a5987a6c00 2024-03-21T10:27:56.004 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:56.003+0000 7f0187400640 1 -- 172.21.15.47:0/2946139277 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(97..97 src has 1..97) v4 ==== 9474+0+0 (secure 0 0 0) 0x7f01980a4b50 con 0x55a5987a6c00 2024-03-21T10:27:56.004 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:56.003+0000 7f01b35ce740 1 -- 172.21.15.47:0/2946139277 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f01980a4900 con 0x55a5987a6c00 2024-03-21T10:27:56.004 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:56.004+0000 7f0187400640 1 -- 172.21.15.47:0/2946139277 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f01980a4d50 con 0x55a5987a6c00 2024-03-21T10:27:56.005 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:56.004+0000 7f01b35ce740 10 client.16566 fetch_fsmap learned FSMap version 32 2024-03-21T10:27:56.005 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:56.004+0000 7f01b35ce740 1 -- 172.21.15.47:0/2946139277 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x55a59841e3f0 con 0x55a5987a6c00 2024-03-21T10:27:56.007 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:56.005+0000 7f0187400640 1 -- 172.21.15.47:0/2946139277 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f01980a4900 con 0x55a5987a6c00 2024-03-21T10:27:56.007 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:56.006+0000 7f0187400640 1 -- 172.21.15.47:0/2946139277 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 32) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f0198097c40 con 0x55a5987a6c00 2024-03-21T10:27:56.007 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:56.007+0000 7f01b35ce740 10 client.16566 fetch_fsmap finished waiting for FSMap version 32 2024-03-21T10:27:56.007 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:56.007+0000 7f01b35ce740 10 client.16566 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-21T10:27:56.008 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:56.007+0000 7f01b35ce740 20 client.16566 populate_metadata read hostname 'smithi047' 2024-03-21T10:27:56.008 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:56.007+0000 7f01b35ce740 1 --2- 172.21.15.47:0/2946139277 >> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] conn(0x55a5987bf0b0 0x55a5987c14a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:27:56.008 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:56.007+0000 7f01b35ce740 4 client.16566 mds_command: new command op to 34535 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T10:27:56.008 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:56.007+0000 7f01b35ce740 1 -- 172.21.15.47:0/2946139277 --> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55a598717790 con 0x55a5987bf0b0 2024-03-21T10:27:56.008 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:56.007+0000 7f01ade00640 1 --2- 172.21.15.47:0/2946139277 >> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] conn(0x55a5987bf0b0 0x55a5987c14a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:27:56.009 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:56.008+0000 7f01ade00640 1 --2- 172.21.15.47:0/2946139277 >> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] conn(0x55a5987bf0b0 0x55a5987c14a0 crc :-1 s=READY pgs=99 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a3642c17fd87bbba server_cookie=7f1a3c61aa7d64f3 in_seq=0 out_seq=0 2024-03-21T10:27:56.009 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:56.008+0000 7f0187400640 10 client.16566 ms_handle_connect on v2:172.21.15.100:6838/4173168440 2024-03-21T10:27:56.010 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:56.009+0000 7f0187400640 1 -- 172.21.15.47:0/2946139277 <== mds.0 v2:172.21.15.100:6838/4173168440 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55a598717790 con 0x55a5987bf0b0 2024-03-21T10:27:56.010 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:56.009+0000 7f0187400640 10 client.16566 handle_command_reply: tid=0 2024-03-21T10:27:56.010 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:56.010+0000 7f01b35ce740 1 client.16566 shutdown 2024-03-21T10:27:56.011 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:56.010+0000 7f01b35ce740 1 -- 172.21.15.47:0/2946139277 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a5987a6c00 msgr2=0x55a5987abd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:27:56.012 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:56.010+0000 7f01b35ce740 1 --2- 172.21.15.47:0/2946139277 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a5987a6c00 0x55a5987abd90 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7f019800c6a0 tx=0x7f0198009f50 comp rx=0 tx=0).stop 2024-03-21T10:27:56.012 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:56.011+0000 7f01b35ce740 1 -- 172.21.15.47:0/2946139277 shutdown_connections 2024-03-21T10:27:56.012 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:56.011+0000 7f01b35ce740 1 --2- 172.21.15.47:0/2946139277 >> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] conn(0x55a5987bf0b0 0x55a5987c14a0 crc :-1 s=READY pgs=99 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:27:56.012 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:56.011+0000 7f01b35ce740 1 --2- 172.21.15.47:0/2946139277 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a5987af290 0x55a5987ac2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:27:56.012 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:56.011+0000 7f01b35ce740 1 --2- 172.21.15.47:0/2946139277 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a5987a6c00 0x55a5987abd90 unknown :-1 s=CLOSED pgs=724 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:27:56.012 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:56.011+0000 7f0187400640 0 client.16566 ms_handle_reset on v2:172.21.15.100:6838/4173168440 2024-03-21T10:27:56.012 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:56.011+0000 7f01b35ce740 1 --2- 172.21.15.47:0/2946139277 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a5987a5f20 0x55a5987aea00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:27:56.012 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:56.011+0000 7f01b35ce740 1 -- 172.21.15.47:0/2946139277 >> 172.21.15.47:0/2946139277 conn(0x55a59876d200 msgr2=0x55a5987a26c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:27:56.013 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:56.011+0000 7f01b35ce740 1 -- 172.21.15.47:0/2946139277 shutdown_connections 2024-03-21T10:27:56.013 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:56.012+0000 7f01b35ce740 1 -- 172.21.15.47:0/2946139277 wait complete. 2024-03-21T10:27:56.013 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:56.012+0000 7f01b35ce740 20 client.16566 trim_cache size 0 max 16384 2024-03-21T10:27:56.064 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:56.064+0000 7f01b35ce740 1 Processor -- start 2024-03-21T10:27:56.065 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:56.064+0000 7f01b35ce740 1 -- start start 2024-03-21T10:27:56.065 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:56.064+0000 7f01b35ce740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a5987a5f20 0x55a5987ae4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:27:56.065 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:56.064+0000 7f01b35ce740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a5987a6c00 0x55a5987aea00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:27:56.065 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:56.064+0000 7f01b35ce740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a5987af290 0x55a5987b0680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:27:56.065 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:56.064+0000 7f01b35ce740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01a81b4fb0 con 0x55a5987af290 2024-03-21T10:27:56.065 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:56.064+0000 7f01b35ce740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01a81b6930 con 0x55a5987a6c00 2024-03-21T10:27:56.065 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:56.064+0000 7f01b35ce740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01a81b5610 con 0x55a5987a5f20 2024-03-21T10:27:56.066 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:56.065+0000 7f01ad400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a5987a6c00 0x55a5987aea00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:27:56.067 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:56.065+0000 7f01ad400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a5987a6c00 0x55a5987aea00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:60906/0 (socket says 172.21.15.47:60906) 2024-03-21T10:27:56.067 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:56.065+0000 7f01ad400640 1 -- 172.21.15.47:0/156710527 learned_addr learned my addr 172.21.15.47:0/156710527 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:27:56.067 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:56.065+0000 7f01ade00640 1 --2- 172.21.15.47:0/156710527 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a5987a5f20 0x55a5987ae4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:27:56.067 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:56.065+0000 7f01ae800640 1 --2- 172.21.15.47:0/156710527 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a5987af290 0x55a5987b0680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:27:56.067 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:56.065+0000 7f01ad400640 1 -- 172.21.15.47:0/156710527 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a5987a5f20 msgr2=0x55a5987ae4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:27:56.067 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:56.065+0000 7f01ad400640 1 --2- 172.21.15.47:0/156710527 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a5987a5f20 0x55a5987ae4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:27:56.067 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:56.065+0000 7f01ad400640 1 -- 172.21.15.47:0/156710527 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a5987af290 msgr2=0x55a5987b0680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:27:56.067 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:56.065+0000 7f01ad400640 1 --2- 172.21.15.47:0/156710527 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a5987af290 0x55a5987b0680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:27:56.067 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:56.065+0000 7f01ad400640 1 -- 172.21.15.47:0/156710527 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a5983de0f0 con 0x55a5987a6c00 2024-03-21T10:27:56.067 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:56.066+0000 7f01ae800640 1 --2- 172.21.15.47:0/156710527 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a5987af290 0x55a5987b0680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T10:27:56.067 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:56.066+0000 7f01ad400640 1 --2- 172.21.15.47:0/156710527 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a5987a6c00 0x55a5987aea00 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f019809a1a0 tx=0x7f01980cd6d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a61c2d506f59cdf server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:27:56.068 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:56.066+0000 7f0192600640 1 -- 172.21.15.47:0/156710527 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01980a8070 con 0x55a5987a6c00 2024-03-21T10:27:56.068 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:56.066+0000 7f0192600640 1 -- 172.21.15.47:0/156710527 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f01980cd4d0 con 0x55a5987a6c00 2024-03-21T10:27:56.068 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:56.066+0000 7f0192600640 1 -- 172.21.15.47:0/156710527 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01980b0120 con 0x55a5987a6c00 2024-03-21T10:27:56.068 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:56.068+0000 7f01b35ce740 1 -- 172.21.15.47:0/156710527 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a5987a6c00 msgr2=0x55a5987aea00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:27:56.068 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:56.068+0000 7f01b35ce740 1 --2- 172.21.15.47:0/156710527 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a5987a6c00 0x55a5987aea00 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f019809a1a0 tx=0x7f01980cd6d0 comp rx=0 tx=0).stop 2024-03-21T10:27:56.069 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:56.068+0000 7f01b35ce740 1 -- 172.21.15.47:0/156710527 shutdown_connections 2024-03-21T10:27:56.069 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:56.068+0000 7f01b35ce740 1 --2- 172.21.15.47:0/156710527 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a5987af290 0x55a5987b0680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:27:56.069 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:56.068+0000 7f01b35ce740 1 --2- 172.21.15.47:0/156710527 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a5987a6c00 0x55a5987aea00 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:27:56.069 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:56.068+0000 7f01b35ce740 1 --2- 172.21.15.47:0/156710527 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a5987a5f20 0x55a5987ae4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:27:56.070 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:56.068+0000 7f01b35ce740 1 -- 172.21.15.47:0/156710527 >> 172.21.15.47:0/156710527 conn(0x55a59876d200 msgr2=0x55a5987a04a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:27:56.070 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:56.069+0000 7f01b35ce740 1 -- 172.21.15.47:0/156710527 shutdown_connections 2024-03-21T10:27:56.070 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:56.069+0000 7f01b35ce740 1 -- 172.21.15.47:0/156710527 wait complete. 2024-03-21T10:27:56.070 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:56.069+0000 7f01b35ce740 1 -- start start 2024-03-21T10:27:56.071 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:56.070+0000 7f01b35ce740 1 Processor -- start 2024-03-21T10:27:56.072 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:56.071+0000 7f01b35ce740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a5987a5f20 0x55a598815710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:27:56.072 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:56.071+0000 7f01b35ce740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a5987a6c00 0x55a598815c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:27:56.072 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:56.071+0000 7f01b35ce740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a5987af290 0x55a598812c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:27:56.073 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:56.072+0000 7f01b35ce740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01a81b62d0 con 0x55a5987af290 2024-03-21T10:27:56.073 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:56.072+0000 7f01b35ce740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01a81b5c70 con 0x55a5987a6c00 2024-03-21T10:27:56.073 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:56.072+0000 7f01b35ce740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01a81b3170 con 0x55a5987a5f20 2024-03-21T10:27:56.073 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:56.072+0000 7f01b35ce740 10 client.0 _ll_register_callbacks cb 0x7f019809bb20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T10:27:56.073 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:56.073+0000 7f01ae800640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a5987af290 0x55a598812c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:27:56.074 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:56.073+0000 7f01ae800640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a5987af290 0x55a598812c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:57586/0 (socket says 172.21.15.47:57586) 2024-03-21T10:27:56.074 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:56.073+0000 7f01ae800640 1 -- 172.21.15.47:0/1528056007 learned_addr learned my addr 172.21.15.47:0/1528056007 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:27:56.074 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:56.073+0000 7f01ade00640 1 --2- 172.21.15.47:0/1528056007 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a5987a5f20 0x55a598815710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:27:56.074 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:56.073+0000 7f01ad400640 1 --2- 172.21.15.47:0/1528056007 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a5987a6c00 0x55a598815c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:27:56.075 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:56.074+0000 7f01ae800640 1 -- 172.21.15.47:0/1528056007 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a5987a5f20 msgr2=0x55a598815710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:27:56.075 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:56.074+0000 7f01ae800640 1 --2- 172.21.15.47:0/1528056007 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a5987a5f20 0x55a598815710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:27:56.075 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:56.074+0000 7f01ae800640 1 -- 172.21.15.47:0/1528056007 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a5987a6c00 msgr2=0x55a598815c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:27:56.075 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:56.074+0000 7f01ae800640 1 --2- 172.21.15.47:0/1528056007 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a5987a6c00 0x55a598815c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:27:56.076 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:56.074+0000 7f01ae800640 1 -- 172.21.15.47:0/1528056007 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55a5987c46d0 con 0x55a5987af290 2024-03-21T10:27:56.076 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:56.075+0000 7f01ae800640 1 --2- 172.21.15.47:0/1528056007 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a5987af290 0x55a598812c10 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7f01a4009860 tx=0x7f01a40348b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4dbbff89ac1c9094 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:27:56.076 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:56.075+0000 7f0187400640 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-03-21T10:27:56.077 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:56.076+0000 7f0187400640 1 -- 172.21.15.47:0/1528056007 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01a4044070 con 0x55a5987af290 2024-03-21T10:27:56.077 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:56.076+0000 7f0187400640 1 -- 172.21.15.47:0/1528056007 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(97..97 src has 1..97) v4 ==== 9474+0+0 (secure 0 0 0) 0x7f01a4004040 con 0x55a5987af290 2024-03-21T10:27:56.077 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:56.076+0000 7f01b35ce740 1 -- 172.21.15.47:0/1528056007 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f01980cd4d0 con 0x55a5987af290 2024-03-21T10:27:56.077 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:56.076+0000 7f0187400640 1 -- 172.21.15.47:0/1528056007 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f01a4050050 con 0x55a5987af290 2024-03-21T10:27:56.077 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:56.076+0000 7f01b35ce740 10 client.16569 fetch_fsmap learned FSMap version 32 2024-03-21T10:27:56.077 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:56.076+0000 7f01b35ce740 1 -- 172.21.15.47:0/1528056007 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x55a59882bad0 con 0x55a5987af290 2024-03-21T10:27:56.079 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:56.077+0000 7f0187400640 1 -- 172.21.15.47:0/1528056007 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f01980cd4d0 con 0x55a5987af290 2024-03-21T10:27:56.079 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:56.077+0000 7f0187400640 1 -- 172.21.15.47:0/1528056007 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 32) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f01a40035f0 con 0x55a5987af290 2024-03-21T10:27:56.079 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:56.077+0000 7f01b35ce740 10 client.16569 fetch_fsmap finished waiting for FSMap version 32 2024-03-21T10:27:56.079 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:56.077+0000 7f01b35ce740 10 client.16569 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-21T10:27:56.079 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:56.077+0000 7f01b35ce740 20 client.16569 populate_metadata read hostname 'smithi047' 2024-03-21T10:27:56.079 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:56.077+0000 7f01b35ce740 1 --2- 172.21.15.47:0/1528056007 >> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] conn(0x55a59882c1d0 0x55a59882e5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:27:56.079 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:56.078+0000 7f01b35ce740 4 client.16569 mds_command: new command op to 34535 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-21T10:27:56.079 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:56.078+0000 7f01b35ce740 1 -- 172.21.15.47:0/1528056007 --> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55a598711ae0 con 0x55a59882c1d0 2024-03-21T10:27:56.079 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:56.078+0000 7f01ade00640 1 --2- 172.21.15.47:0/1528056007 >> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] conn(0x55a59882c1d0 0x55a59882e5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:27:56.080 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:56.079+0000 7f0187400640 10 client.16569 ms_handle_connect on v2:172.21.15.100:6838/4173168440 2024-03-21T10:27:56.080 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:56.079+0000 7f01ade00640 1 --2- 172.21.15.47:0/1528056007 >> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] conn(0x55a59882c1d0 0x55a59882e5f0 crc :-1 s=READY pgs=100 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8bd3da4853f28d6e server_cookie=d9b2479aa8307399 in_seq=0 out_seq=0 2024-03-21T10:27:56.080 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:56.079+0000 7f0187400640 1 -- 172.21.15.47:0/1528056007 <== mds.0 v2:172.21.15.100:6838/4173168440 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x55a598711ae0 con 0x55a59882c1d0 2024-03-21T10:27:56.080 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:56.079+0000 7f0187400640 10 client.16569 handle_command_reply: tid=0 2024-03-21T10:27:56.081 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:56.079+0000 7f01b35ce740 1 client.16569 shutdown 2024-03-21T10:27:56.081 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:56.080+0000 7f01b35ce740 1 -- 172.21.15.47:0/1528056007 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a5987af290 msgr2=0x55a598812c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:27:56.081 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:56.080+0000 7f01b35ce740 1 --2- 172.21.15.47:0/1528056007 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a5987af290 0x55a598812c10 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7f01a4009860 tx=0x7f01a40348b0 comp rx=0 tx=0).stop 2024-03-21T10:27:56.081 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:56.081+0000 7f01b35ce740 1 -- 172.21.15.47:0/1528056007 shutdown_connections 2024-03-21T10:27:56.081 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:56.081+0000 7f01b35ce740 1 --2- 172.21.15.47:0/1528056007 >> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] conn(0x55a59882c1d0 0x55a59882e5f0 crc :-1 s=READY pgs=100 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:27:56.082 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:56.081+0000 7f0187400640 0 client.16569 ms_handle_reset on v2:172.21.15.100:6838/4173168440 2024-03-21T10:27:56.082 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:56.081+0000 7f01b35ce740 1 --2- 172.21.15.47:0/1528056007 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a5987af290 0x55a598812c10 unknown :-1 s=CLOSED pgs=725 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:27:56.082 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:56.081+0000 7f01b35ce740 1 --2- 172.21.15.47:0/1528056007 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a5987a6c00 0x55a598815c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:27:56.082 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:56.081+0000 7f01b35ce740 1 --2- 172.21.15.47:0/1528056007 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a5987a5f20 0x55a598815710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:27:56.082 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:56.081+0000 7f01b35ce740 1 -- 172.21.15.47:0/1528056007 >> 172.21.15.47:0/1528056007 conn(0x55a59876d200 msgr2=0x55a5987ba510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:27:56.083 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:56.081+0000 7f01b35ce740 1 -- 172.21.15.47:0/1528056007 shutdown_connections 2024-03-21T10:27:56.083 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:56.082+0000 7f01b35ce740 1 -- 172.21.15.47:0/1528056007 wait complete. 2024-03-21T10:27:56.083 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:56.082+0000 7f01b35ce740 20 client.16569 trim_cache size 0 max 16384 2024-03-21T10:27:56.083 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-03-21T10:27:56.083 INFO:teuthology.orchestra.run.smithi047.stdout: "status": "no active scrubs running", 2024-03-21T10:27:56.083 INFO:teuthology.orchestra.run.smithi047.stdout: "scrubs": {} 2024-03-21T10:27:56.084 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-03-21T10:27:56.084 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:56.083+0000 7f01b0800640 1 -- 172.21.15.47:0/3351279439 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f01940a3150 msgr2=0x7f01940a5610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:27:56.084 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:56.083+0000 7f01b0800640 1 --2- 172.21.15.47:0/3351279439 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f01940a3150 0x7f01940a5610 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f01a0005fd0 tx=0x7f01a0009500 comp rx=0 tx=0).stop 2024-03-21T10:27:56.084 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:56.083+0000 7f01b0800640 1 -- 172.21.15.47:0/3351279439 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f01a810cf80 msgr2=0x7f01a819e9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:27:56.084 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:56.083+0000 7f01b0800640 1 --2- 172.21.15.47:0/3351279439 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f01a810cf80 0x7f01a819e9d0 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7f019800e9c0 tx=0x7f019800ee80 comp rx=0 tx=0).stop 2024-03-21T10:27:56.085 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:56.084+0000 7f01b0800640 1 -- 172.21.15.47:0/3351279439 shutdown_connections 2024-03-21T10:27:56.085 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:56.084+0000 7f01b0800640 1 --2- 172.21.15.47:0/3351279439 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f01940a3150 0x7f01940a5610 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:27:56.085 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:56.084+0000 7f01b0800640 1 --2- 172.21.15.47:0/3351279439 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f01a81a2d00 0x7f01a81a31b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:27:56.085 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:56.084+0000 7f01b0800640 1 --2- 172.21.15.47:0/3351279439 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f01a810cf80 0x7f01a819e9d0 unknown :-1 s=CLOSED pgs=723 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:27:56.085 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:56.084+0000 7f01b0800640 1 --2- 172.21.15.47:0/3351279439 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f01a8107eb0 0x7f01a819e490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:27:56.085 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:56.084+0000 7f01b0800640 1 -- 172.21.15.47:0/3351279439 >> 172.21.15.47:0/3351279439 conn(0x7f01a8075be0 msgr2=0x7f01a8076760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:27:56.085 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:56.084+0000 7f01b0800640 1 -- 172.21.15.47:0/3351279439 shutdown_connections 2024-03-21T10:27:56.085 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:56.084+0000 7f01b0800640 1 -- 172.21.15.47:0/3351279439 wait complete. 2024-03-21T10:27:56.110 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-03-21T10:27:56.110 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-21T10:27:56.411 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:56.409+0000 7fded1c00640 1 -- 172.21.15.47:0/4056811304 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdecc108910 msgr2=0x7fdecc108cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:27:56.411 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:56.409+0000 7fded1c00640 1 --2- 172.21.15.47:0/4056811304 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdecc108910 0x7fdecc108cf0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fdebc00b9f0 tx=0x7fdebc02d8f0 comp rx=0 tx=0).stop 2024-03-21T10:27:56.411 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:56.410+0000 7fded1c00640 1 -- 172.21.15.47:0/4056811304 shutdown_connections 2024-03-21T10:27:56.411 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:56.410+0000 7fded1c00640 1 --2- 172.21.15.47:0/4056811304 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdecc10d3d0 0x7fdecc114fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:27:56.411 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:56.410+0000 7fded1c00640 1 --2- 172.21.15.47:0/4056811304 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdecc1092c0 0x7fdecc10ccf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:27:56.411 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:56.410+0000 7fded1c00640 1 --2- 172.21.15.47:0/4056811304 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdecc108910 0x7fdecc108cf0 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:27:56.411 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:56.410+0000 7fded1c00640 1 -- 172.21.15.47:0/4056811304 >> 172.21.15.47:0/4056811304 conn(0x7fdecc06c790 msgr2=0x7fdecc06cba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:27:56.411 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:56.410+0000 7fded1c00640 1 -- 172.21.15.47:0/4056811304 shutdown_connections 2024-03-21T10:27:56.412 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:56.412+0000 7fded1c00640 1 -- 172.21.15.47:0/4056811304 wait complete. 2024-03-21T10:27:56.414 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:56.413+0000 7fded1c00640 1 Processor -- start 2024-03-21T10:27:56.415 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:56.414+0000 7fded1c00640 1 -- start start 2024-03-21T10:27:56.416 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:56.415+0000 7fded1c00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdecc108910 0x7fdecc1aca70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:27:56.416 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:56.415+0000 7fded1c00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdecc10d3d0 0x7fdecc1acfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:27:56.416 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:56.415+0000 7fded1c00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdecc1a6b30 0x7fdecc1a6fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:27:56.416 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:56.415+0000 7fded1c00640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdecc117f40 con 0x7fdecc10d3d0 2024-03-21T10:27:56.416 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:56.415+0000 7fded1c00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdecc117dc0 con 0x7fdecc1a6b30 2024-03-21T10:27:56.416 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:56.415+0000 7fded1c00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdecc1180c0 con 0x7fdecc108910 2024-03-21T10:27:56.416 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:56.416+0000 7fdecaa00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdecc108910 0x7fdecc1aca70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:27:56.416 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:56.416+0000 7fdecb400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdecc1a6b30 0x7fdecc1a6fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:27:56.417 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:56.416+0000 7fdeca000640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdecc10d3d0 0x7fdecc1acfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:27:56.417 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:56.416+0000 7fdeca000640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdecc10d3d0 0x7fdecc1acfb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:57612/0 (socket says 172.21.15.47:57612) 2024-03-21T10:27:56.417 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:56.416+0000 7fdeca000640 1 -- 172.21.15.47:0/2430338055 learned_addr learned my addr 172.21.15.47:0/2430338055 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:27:56.418 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:56.417+0000 7fdecaa00640 1 -- 172.21.15.47:0/2430338055 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdecc1a6b30 msgr2=0x7fdecc1a6fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:27:56.418 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:56.417+0000 7fdecaa00640 1 --2- 172.21.15.47:0/2430338055 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdecc1a6b30 0x7fdecc1a6fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:27:56.418 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:56.417+0000 7fdecaa00640 1 -- 172.21.15.47:0/2430338055 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdecc10d3d0 msgr2=0x7fdecc1acfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:27:56.418 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:56.417+0000 7fdecaa00640 1 --2- 172.21.15.47:0/2430338055 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdecc10d3d0 0x7fdecc1acfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:27:56.418 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:56.417+0000 7fdecaa00640 1 -- 172.21.15.47:0/2430338055 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdecc1a7700 con 0x7fdecc108910 2024-03-21T10:27:56.419 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:56.418+0000 7fdecaa00640 1 --2- 172.21.15.47:0/2430338055 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdecc108910 0x7fdecc1aca70 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fdebc00b9f0 tx=0x7fdebc0042f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:27:56.420 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:56.419+0000 7fdec3400640 1 -- 172.21.15.47:0/2430338055 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdebc037040 con 0x7fdecc108910 2024-03-21T10:27:56.420 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:56.419+0000 7fdec3400640 1 -- 172.21.15.47:0/2430338055 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fdebc009d60 con 0x7fdecc108910 2024-03-21T10:27:56.420 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:56.419+0000 7fdec3400640 1 -- 172.21.15.47:0/2430338055 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdebc003960 con 0x7fdecc108910 2024-03-21T10:27:56.420 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:56.419+0000 7fded1c00640 1 -- 172.21.15.47:0/2430338055 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdecc1a7990 con 0x7fdecc108910 2024-03-21T10:27:56.420 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:56.420+0000 7fded1c00640 1 -- 172.21.15.47:0/2430338055 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdecc19d6b0 con 0x7fdecc108910 2024-03-21T10:27:56.423 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:56.422+0000 7fded1c00640 1 -- 172.21.15.47:0/2430338055 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdecc1098c0 con 0x7fdecc108910 2024-03-21T10:27:56.427 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:56.427+0000 7fdec3400640 1 -- 172.21.15.47:0/2430338055 <== mon.1 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 27) v1 ==== 137985+0+0 (secure 0 0 0) 0x7fdebc044030 con 0x7fdecc108910 2024-03-21T10:27:56.430 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:56.429+0000 7fdec3400640 1 --2- 172.21.15.47:0/2430338055 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7fdeb00a3190 0x7fdeb00a5650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:27:56.430 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:56.430+0000 7fdec3400640 1 -- 172.21.15.47:0/2430338055 <== mon.1 v2:172.21.15.157:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 9474+0+0 (secure 0 0 0) 0x7fdebc0f0f50 con 0x7fdecc108910 2024-03-21T10:27:56.430 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:56.430+0000 7fdeca000640 1 --2- 172.21.15.47:0/2430338055 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7fdeb00a3190 0x7fdeb00a5650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:27:56.433 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:56.432+0000 7fdec3400640 1 -- 172.21.15.47:0/2430338055 <== mon.1 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fdebc04b070 con 0x7fdecc108910 2024-03-21T10:27:56.433 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:56.433+0000 7fdeca000640 1 --2- 172.21.15.47:0/2430338055 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7fdeb00a3190 0x7fdeb00a5650 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fdecc1a8310 tx=0x7fdeb8005f90 comp rx=0 tx=0).ready entity=mgr.15687 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:27:56.945 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:56.944+0000 7fded1c00640 1 -- 172.21.15.47:0/2430338055 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fdecc1105d0 con 0x7fdecc108910 2024-03-21T10:27:56.945 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:56.945+0000 7fdec3400640 1 -- 172.21.15.47:0/2430338055 <== mon.1 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 32 v32) v1 ==== 94+0+11471 (secure 0 0 0) 0x7fdebc0bcfd0 con 0x7fdecc108910 2024-03-21T10:27:56.949 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-21T10:27:56.949 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":32,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14511,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6835/735724037","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6834","nonce":735724037},{"type":"v1","addr":"172.21.15.47:6835","nonce":735724037}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14529,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6837/2248191724","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6836","nonce":2248191724},{"type":"v1","addr":"172.21.15.47:6837","nonce":2248191724}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":32,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-21T10:21:58.329063+0000","modified":"2024-03-21T10:26:17.169892+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":34535,"mds_1":34493,"mds_2":34454,"mds_3":34427,"mds_4":14619},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14604":{"gid":14604,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.47:6839/2881335356","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6838","nonce":2881335356},{"type":"v1","addr":"172.21.15.47:6839","nonce":2881335356}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_14619":{"gid":14619,"name":"a","rank":4,"incarnation":27,"state":"up:active","state_seq":11,"addr":"172.21.15.47:6841/3401578978","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6840","nonce":3401578978},{"type":"v1","addr":"172.21.15.47:6841","nonce":3401578978}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24406":{"gid":24406,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.157:6833/4000010136","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":4000010136},{"type":"v1","addr":"172.21.15.157:6833","nonce":4000010136}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24457":{"gid":24457,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.157:6835/3902868651","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":3902868651},{"type":"v1","addr":"172.21.15.157:6835","nonce":3902868651}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24475":{"gid":24475,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.157:6837/1908087204","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6836","nonce":1908087204},{"type":"v1","addr":"172.21.15.157:6837","nonce":1908087204}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_25729":{"gid":25729,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.157:6839/1998408416","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6838","nonce":1998408416},{"type":"v1","addr":"172.21.15.157:6839","nonce":1998408416}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_34427":{"gid":34427,"name":"k","rank":3,"incarnation":25,"state":"up:active","state_seq":20,"addr":"172.21.15.100:6833/2723302038","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6832","nonce":2723302038},{"type":"v1","addr":"172.21.15.100:6833","nonce":2723302038}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_34454":{"gid":34454,"name":"e","rank":2,"incarnation":22,"state":"up:active","state_seq":17,"addr":"172.21.15.100:6835/2597849957","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":2597849957},{"type":"v1","addr":"172.21.15.100:6835","nonce":2597849957}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_34493":{"gid":34493,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":13,"addr":"172.21.15.100:6837/3497644162","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6836","nonce":3497644162},{"type":"v1","addr":"172.21.15.100:6837","nonce":3497644162}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_34535":{"gid":34535,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.100:6839/4173168440","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6838","nonce":4173168440},{"type":"v1","addr":"172.21.15.100:6839","nonce":4173168440}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":34535,"qdb_cluster":[34427,14619,34454,34535,34493]},"id":1}]} 2024-03-21T10:27:56.949 INFO:teuthology.orchestra.run.smithi047.stderr:dumped fsmap epoch 32 2024-03-21T10:27:56.950 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:56.949+0000 7fded1c00640 1 -- 172.21.15.47:0/2430338055 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7fdeb00a3190 msgr2=0x7fdeb00a5650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:27:56.950 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:56.949+0000 7fded1c00640 1 --2- 172.21.15.47:0/2430338055 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7fdeb00a3190 0x7fdeb00a5650 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fdecc1a8310 tx=0x7fdeb8005f90 comp rx=0 tx=0).stop 2024-03-21T10:27:56.950 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:56.950+0000 7fded1c00640 1 -- 172.21.15.47:0/2430338055 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdecc108910 msgr2=0x7fdecc1aca70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:27:56.950 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:56.950+0000 7fded1c00640 1 --2- 172.21.15.47:0/2430338055 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdecc108910 0x7fdecc1aca70 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fdebc00b9f0 tx=0x7fdebc0042f0 comp rx=0 tx=0).stop 2024-03-21T10:27:56.951 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:56.950+0000 7fded1c00640 1 -- 172.21.15.47:0/2430338055 shutdown_connections 2024-03-21T10:27:56.951 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:56.950+0000 7fded1c00640 1 --2- 172.21.15.47:0/2430338055 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7fdeb00a3190 0x7fdeb00a5650 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:27:56.951 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:56.950+0000 7fded1c00640 1 --2- 172.21.15.47:0/2430338055 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdecc1a6b30 0x7fdecc1a6fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:27:56.951 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:56.950+0000 7fded1c00640 1 --2- 172.21.15.47:0/2430338055 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdecc10d3d0 0x7fdecc1acfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:27:56.951 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:56.950+0000 7fded1c00640 1 --2- 172.21.15.47:0/2430338055 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdecc108910 0x7fdecc1aca70 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:27:56.951 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:56.950+0000 7fded1c00640 1 -- 172.21.15.47:0/2430338055 >> 172.21.15.47:0/2430338055 conn(0x7fdecc06c790 msgr2=0x7fdecc071390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:27:56.952 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:56.951+0000 7fded1c00640 1 -- 172.21.15.47:0/2430338055 shutdown_connections 2024-03-21T10:27:56.952 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:56.951+0000 7fded1c00640 1 -- 172.21.15.47:0/2430338055 wait complete. 2024-03-21T10:27:56.982 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-21T10:27:56.982 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-03-21T10:27:57.315 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:57.315+0000 7f5803200640 1 -- 172.21.15.47:0/3631358372 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f57fc10dee0 msgr2=0x7f57fc1102f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:27:57.316 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:57.315+0000 7f5803200640 1 --2- 172.21.15.47:0/3631358372 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f57fc10dee0 0x7f57fc1102f0 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7f57ec009900 tx=0x7f57ec02bcd0 comp rx=0 tx=0).stop 2024-03-21T10:27:57.316 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:57.315+0000 7f5803200640 1 -- 172.21.15.47:0/3631358372 shutdown_connections 2024-03-21T10:27:57.316 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:57.315+0000 7f5803200640 1 --2- 172.21.15.47:0/3631358372 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f57fc10dee0 0x7f57fc1102f0 unknown :-1 s=CLOSED pgs=726 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:27:57.316 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:57.315+0000 7f5803200640 1 --2- 172.21.15.47:0/3631358372 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f57fc074390 0x7f57fc10d9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:27:57.316 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:57.315+0000 7f5803200640 1 --2- 172.21.15.47:0/3631358372 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f57fc073a70 0x7f57fc073e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:27:57.317 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:57.315+0000 7f5803200640 1 -- 172.21.15.47:0/3631358372 >> 172.21.15.47:0/3631358372 conn(0x7f57fc0fd0d0 msgr2=0x7f57fc0ff4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:27:57.317 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:57.315+0000 7f5803200640 1 -- 172.21.15.47:0/3631358372 shutdown_connections 2024-03-21T10:27:57.317 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:57.315+0000 7f5803200640 1 -- 172.21.15.47:0/3631358372 wait complete. 2024-03-21T10:27:57.317 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:57.317+0000 7f5803200640 1 Processor -- start 2024-03-21T10:27:57.317 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:57.317+0000 7f5803200640 1 -- start start 2024-03-21T10:27:57.319 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:57.317+0000 7f5803200640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f57fc073a70 0x7f57fc19a160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:27:57.319 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:57.317+0000 7f5803200640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f57fc074390 0x7f57fc19a6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:27:57.319 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:57.317+0000 7f5803200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f57fc10dee0 0x7f57fc19e9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:27:57.319 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:57.317+0000 7f5803200640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57fc113af0 con 0x7f57fc073a70 2024-03-21T10:27:57.319 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:57.317+0000 7f5803200640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57fc113970 con 0x7f57fc10dee0 2024-03-21T10:27:57.320 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:57.317+0000 7f5803200640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57fc113c70 con 0x7f57fc074390 2024-03-21T10:27:57.320 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:57.318+0000 7f57fb400640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f57fc074390 0x7f57fc19a6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:27:57.320 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:57.318+0000 7f57fb400640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f57fc074390 0x7f57fc19a6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.47:40096/0 (socket says 172.21.15.47:40096) 2024-03-21T10:27:57.320 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:57.318+0000 7f57fb400640 1 -- 172.21.15.47:0/93528726 learned_addr learned my addr 172.21.15.47:0/93528726 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:27:57.320 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:57.318+0000 7f5801200640 1 --2- 172.21.15.47:0/93528726 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f57fc10dee0 0x7f57fc19e9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:27:57.320 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:57.319+0000 7f57fbe00640 1 --2- 172.21.15.47:0/93528726 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f57fc073a70 0x7f57fc19a160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:27:57.320 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:57.319+0000 7f57fb400640 1 -- 172.21.15.47:0/93528726 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f57fc10dee0 msgr2=0x7f57fc19e9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:27:57.321 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:57.319+0000 7f57fb400640 1 --2- 172.21.15.47:0/93528726 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f57fc10dee0 0x7f57fc19e9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:27:57.321 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:57.319+0000 7f57fb400640 1 -- 172.21.15.47:0/93528726 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f57fc073a70 msgr2=0x7f57fc19a160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:27:57.321 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:57.319+0000 7f57fb400640 1 --2- 172.21.15.47:0/93528726 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f57fc073a70 0x7f57fc19a160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:27:57.321 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:57.319+0000 7f57fb400640 1 -- 172.21.15.47:0/93528726 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57fc19f100 con 0x7f57fc074390 2024-03-21T10:27:57.321 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:57.321+0000 7f57fb400640 1 --2- 172.21.15.47:0/93528726 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f57fc074390 0x7f57fc19a6a0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f57e800d950 tx=0x7f57e800de10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:27:57.322 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:57.321+0000 7f57f8c00640 1 -- 172.21.15.47:0/93528726 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57e8017070 con 0x7f57fc074390 2024-03-21T10:27:57.322 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:57.322+0000 7f57f8c00640 1 -- 172.21.15.47:0/93528726 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f57e8009c30 con 0x7f57fc074390 2024-03-21T10:27:57.322 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:57.322+0000 7f5803200640 1 -- 172.21.15.47:0/93528726 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f57fc19f3f0 con 0x7f57fc074390 2024-03-21T10:27:57.323 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:57.322+0000 7f5803200640 1 -- 172.21.15.47:0/93528726 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f57fc101530 con 0x7f57fc074390 2024-03-21T10:27:57.323 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:57.322+0000 7f57f8c00640 1 -- 172.21.15.47:0/93528726 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57e800f620 con 0x7f57fc074390 2024-03-21T10:27:57.329 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:57.329+0000 7f5806037740 1 Processor -- start 2024-03-21T10:27:57.329 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:57.329+0000 7f5806037740 1 -- start start 2024-03-21T10:27:57.330 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:57.329+0000 7f5806037740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f3a1deb6d0 0x55f3a1debab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:27:57.330 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:57.329+0000 7f5806037740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f3a1dec080 0x55f3a1df4630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:27:57.332 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:57.330+0000 7f5801200640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f3a1deb6d0 0x55f3a1debab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:27:57.332 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:57.330+0000 7f5801200640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f3a1deb6d0 0x55f3a1debab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.47:40104/0 (socket says 172.21.15.47:40104) 2024-03-21T10:27:57.333 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:57.330+0000 7f57fbe00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f3a1dec080 0x55f3a1df4630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:27:57.333 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:57.330+0000 7f57fbe00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f3a1dec080 0x55f3a1df4630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:57630/0 (socket says 172.21.15.47:57630) 2024-03-21T10:27:57.333 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:57.331+0000 7f5806037740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f3a1df4c00 0x55f3a1df7050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:27:57.333 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:57.331+0000 7f5806037740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f3a1d4f2c0 con 0x55f3a1dec080 2024-03-21T10:27:57.333 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:57.331+0000 7f5806037740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f3a1cf7c90 con 0x55f3a1df4c00 2024-03-21T10:27:57.333 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:57.331+0000 7f5806037740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f3a1d4b360 con 0x55f3a1deb6d0 2024-03-21T10:27:57.333 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:57.331+0000 7f5801200640 1 -- 172.21.15.47:0/279493612 learned_addr learned my addr 172.21.15.47:0/279493612 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:27:57.333 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:57.332+0000 7f57fbe00640 1 -- 172.21.15.47:0/279493612 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f3a1deb6d0 msgr2=0x55f3a1debab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:27:57.333 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:57.332+0000 7f57fbe00640 1 --2- 172.21.15.47:0/279493612 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f3a1deb6d0 0x55f3a1debab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:27:57.333 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:57.332+0000 7f57fbe00640 1 -- 172.21.15.47:0/279493612 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f3a1df4c00 msgr2=0x55f3a1df7050 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-21T10:27:57.334 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:57.332+0000 7f57fbe00640 1 --2- 172.21.15.47:0/279493612 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f3a1df4c00 0x55f3a1df7050 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:27:57.334 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:57.332+0000 7f57fbe00640 1 -- 172.21.15.47:0/279493612 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f3a1d1c080 con 0x55f3a1dec080 2024-03-21T10:27:57.335 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:57.332+0000 7f57fbe00640 1 --2- 172.21.15.47:0/279493612 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f3a1dec080 0x55f3a1df4630 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7f57f4009ec0 tx=0x7f57f4010740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f2b7a0b22053548e server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:27:57.335 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:57.333+0000 7f57f1600640 1 -- 172.21.15.47:0/279493612 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57f4016070 con 0x55f3a1dec080 2024-03-21T10:27:57.335 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:57.333+0000 7f57f1600640 1 -- 172.21.15.47:0/279493612 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f57f401bb10 con 0x55f3a1dec080 2024-03-21T10:27:57.335 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:57.333+0000 7f57f1600640 1 -- 172.21.15.47:0/279493612 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57f4012c80 con 0x55f3a1dec080 2024-03-21T10:27:57.335 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:57.333+0000 7f5806037740 1 -- 172.21.15.47:0/279493612 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f3a1dec080 msgr2=0x55f3a1df4630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:27:57.335 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:57.333+0000 7f5806037740 1 --2- 172.21.15.47:0/279493612 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f3a1dec080 0x55f3a1df4630 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7f57f4009ec0 tx=0x7f57f4010740 comp rx=0 tx=0).stop 2024-03-21T10:27:57.335 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:57.334+0000 7f5806037740 1 -- 172.21.15.47:0/279493612 shutdown_connections 2024-03-21T10:27:57.336 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:57.334+0000 7f5806037740 1 --2- 172.21.15.47:0/279493612 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f3a1df4c00 0x55f3a1df7050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:27:57.336 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:57.334+0000 7f5806037740 1 --2- 172.21.15.47:0/279493612 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f3a1dec080 0x55f3a1df4630 unknown :-1 s=CLOSED pgs=727 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:27:57.336 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:57.334+0000 7f5806037740 1 --2- 172.21.15.47:0/279493612 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f3a1deb6d0 0x55f3a1debab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:27:57.336 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:57.334+0000 7f5806037740 1 -- 172.21.15.47:0/279493612 >> 172.21.15.47:0/279493612 conn(0x55f3a1de5da0 msgr2=0x55f3a1de6180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:27:57.347 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:57.342+0000 7f57f8c00640 1 -- 172.21.15.47:0/93528726 <== mon.1 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 27) v1 ==== 137985+0+0 (secure 0 0 0) 0x7f57e800f7c0 con 0x7f57fc074390 2024-03-21T10:27:57.347 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:57.344+0000 7f5806037740 1 -- 172.21.15.47:0/279493612 shutdown_connections 2024-03-21T10:27:57.347 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:57.344+0000 7f5806037740 1 -- 172.21.15.47:0/279493612 wait complete. 2024-03-21T10:27:57.347 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:57.344+0000 7f5806037740 1 -- start start 2024-03-21T10:27:57.348 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:57.345+0000 7f5806037740 1 Processor -- start 2024-03-21T10:27:57.349 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:57.348+0000 7f57f8c00640 1 --2- 172.21.15.47:0/93528726 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f57e00a2fd0 0x7f57e00a5490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:27:57.350 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:57.349+0000 7f57f8c00640 1 -- 172.21.15.47:0/93528726 <== mon.1 v2:172.21.15.157:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 9474+0+0 (secure 0 0 0) 0x7f57e80cf5b0 con 0x7f57fc074390 2024-03-21T10:27:57.351 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:57.350+0000 7f5801200640 1 --2- 172.21.15.47:0/93528726 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f57e00a2fd0 0x7f57e00a5490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:27:57.352 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:57.351+0000 7f5801200640 1 --2- 172.21.15.47:0/93528726 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f57e00a2fd0 0x7f57e00a5490 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f57ec005c40 tx=0x7f57ec005a20 comp rx=0 tx=0).ready entity=mgr.15687 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:27:57.352 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:57.351+0000 7f5806037740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f3a1deb6d0 0x55f3a1df4570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:27:57.354 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:57.352+0000 7f5806037740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f3a1dec080 0x55f3a1df1860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:27:57.354 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:57.352+0000 7f5806037740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f3a1df1da0 0x55f3a1df2200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:27:57.354 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:57.352+0000 7f57fbe00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f3a1deb6d0 0x55f3a1df4570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:27:57.354 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:57.352+0000 7f57fbe00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f3a1deb6d0 0x55f3a1df4570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:57644/0 (socket says 172.21.15.47:57644) 2024-03-21T10:27:57.354 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:57.353+0000 7f57fbe00640 1 -- 172.21.15.47:0/4227723134 learned_addr learned my addr 172.21.15.47:0/4227723134 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:27:57.354 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:57.353+0000 7f57fb400640 1 --2- 172.21.15.47:0/4227723134 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f3a1dec080 0x55f3a1df1860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:27:57.354 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:57.353+0000 7f5806037740 1 -- 172.21.15.47:0/4227723134 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f3a1df7bc0 con 0x55f3a1deb6d0 2024-03-21T10:27:57.354 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:57.353+0000 7f5806037740 1 -- 172.21.15.47:0/4227723134 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f3a1d0b0d0 con 0x55f3a1df1da0 2024-03-21T10:27:57.354 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:57.353+0000 7f5801200640 1 --2- 172.21.15.47:0/4227723134 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f3a1df1da0 0x55f3a1df2200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:27:57.355 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:57.353+0000 7f5806037740 1 -- 172.21.15.47:0/4227723134 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f3a1df7d40 con 0x55f3a1dec080 2024-03-21T10:27:57.355 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:57.354+0000 7f5801200640 1 -- 172.21.15.47:0/4227723134 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f3a1dec080 msgr2=0x55f3a1df1860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:27:57.355 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:57.354+0000 7f5801200640 1 --2- 172.21.15.47:0/4227723134 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f3a1dec080 0x55f3a1df1860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:27:57.355 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:57.354+0000 7f5801200640 1 -- 172.21.15.47:0/4227723134 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f3a1deb6d0 msgr2=0x55f3a1df4570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:27:57.355 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:57.354+0000 7f5801200640 1 --2- 172.21.15.47:0/4227723134 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f3a1deb6d0 0x55f3a1df4570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:27:57.355 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:57.354+0000 7f5801200640 1 -- 172.21.15.47:0/4227723134 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55f3a1a633f0 con 0x55f3a1df1da0 2024-03-21T10:27:57.356 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:57.355+0000 7f57e6000640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-21T10:27:57.356 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:57.355+0000 7f5801200640 1 --2- 172.21.15.47:0/4227723134 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f3a1df1da0 0x55f3a1df2200 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f57ec005940 tx=0x7f57ec02bcd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=251dc3c875cadad4 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:27:57.356 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:57.355+0000 7f5806037740 10 client.0 _ll_register_callbacks cb 0x55f3a1d0fe80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T10:27:57.356 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:57.356+0000 7f5806037740 1 -- 172.21.15.47:0/4227723134 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f57f401bb10 con 0x55f3a1df1da0 2024-03-21T10:27:57.357 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:57.356+0000 7f57e6000640 1 -- 172.21.15.47:0/4227723134 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57ec05a050 con 0x55f3a1df1da0 2024-03-21T10:27:57.357 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:57.356+0000 7f57e6000640 1 -- 172.21.15.47:0/4227723134 <== mon.2 v2:172.21.15.100:3300/0 2 ==== osd_map(97..97 src has 1..97) v4 ==== 9474+0+0 (secure 0 0 0) 0x7f57ec03a040 con 0x55f3a1df1da0 2024-03-21T10:27:57.357 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:57.356+0000 7f57e6000640 1 -- 172.21.15.47:0/4227723134 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f57ec050070 con 0x55f3a1df1da0 2024-03-21T10:27:57.357 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:57.356+0000 7f5806037740 10 client.36377 fetch_fsmap learned FSMap version 32 2024-03-21T10:27:57.357 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:57.356+0000 7f5806037740 1 -- 172.21.15.47:0/4227723134 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({fsmap=32}) v3 -- 0x55f3a1b2d7d0 con 0x55f3a1df1da0 2024-03-21T10:27:57.359 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:57.358+0000 7f57e6000640 1 -- 172.21.15.47:0/4227723134 <== mon.2 v2:172.21.15.100:3300/0 4 ==== fsmap(e 32) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f57ec0046c0 con 0x55f3a1df1da0 2024-03-21T10:27:57.359 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:57.359+0000 7f5806037740 10 client.36377 fetch_fsmap finished waiting for FSMap version 32 2024-03-21T10:27:57.359 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:57.359+0000 7f5806037740 10 client.36377 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.a 2024-03-21T10:27:57.360 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:57.359+0000 7f5806037740 20 client.36377 populate_metadata read hostname 'smithi047' 2024-03-21T10:27:57.360 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:57.359+0000 7f5806037740 1 --2- 172.21.15.47:0/4227723134 >> [v2:172.21.15.47:6840/3401578978,v1:172.21.15.47:6841/3401578978] conn(0x55f3a1e04cc0 0x55f3a1e07130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:27:57.360 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:57.359+0000 7f5806037740 4 client.36377 mds_command: new command op to 14619 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T10:27:57.360 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:57.359+0000 7f5806037740 1 -- 172.21.15.47:0/4227723134 --> [v2:172.21.15.47:6840/3401578978,v1:172.21.15.47:6841/3401578978] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55f3a1d5c800 con 0x55f3a1e04cc0 2024-03-21T10:27:57.361 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:57.361+0000 7f57fbe00640 1 --2- 172.21.15.47:0/4227723134 >> [v2:172.21.15.47:6840/3401578978,v1:172.21.15.47:6841/3401578978] conn(0x55f3a1e04cc0 0x55f3a1e07130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:27:57.362 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:57.362+0000 7f57fbe00640 1 --2- 172.21.15.47:0/4227723134 >> [v2:172.21.15.47:6840/3401578978,v1:172.21.15.47:6841/3401578978] conn(0x55f3a1e04cc0 0x55f3a1e07130 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=9c3fcfb9d1c1ede server_cookie=df3946f2d5e050c in_seq=0 out_seq=0 2024-03-21T10:27:57.363 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:57.362+0000 7f57e6000640 10 client.36377 ms_handle_connect on v2:172.21.15.47:6840/3401578978 2024-03-21T10:27:57.364 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:57.363+0000 7f57e6000640 1 -- 172.21.15.47:0/4227723134 <== mds.4 v2:172.21.15.47:6840/3401578978 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55f3a1d5c800 con 0x55f3a1e04cc0 2024-03-21T10:27:57.364 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:57.363+0000 7f57e6000640 10 client.36377 handle_command_reply: tid=0 2024-03-21T10:27:57.364 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:57.364+0000 7f5806037740 1 client.36377 shutdown 2024-03-21T10:27:57.365 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:57.364+0000 7f5806037740 1 -- 172.21.15.47:0/4227723134 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f3a1df1da0 msgr2=0x55f3a1df2200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:27:57.365 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:57.364+0000 7f5806037740 1 --2- 172.21.15.47:0/4227723134 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f3a1df1da0 0x55f3a1df2200 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f57ec005940 tx=0x7f57ec02bcd0 comp rx=0 tx=0).stop 2024-03-21T10:27:57.365 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:57.364+0000 7f5806037740 1 -- 172.21.15.47:0/4227723134 shutdown_connections 2024-03-21T10:27:57.365 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:57.364+0000 7f5806037740 1 --2- 172.21.15.47:0/4227723134 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f3a1df1da0 0x55f3a1df2200 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:27:57.365 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:57.364+0000 7f5806037740 1 --2- 172.21.15.47:0/4227723134 >> [v2:172.21.15.47:6840/3401578978,v1:172.21.15.47:6841/3401578978] conn(0x55f3a1e04cc0 0x55f3a1e07130 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:27:57.366 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:57.365+0000 7f5806037740 1 --2- 172.21.15.47:0/4227723134 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f3a1dec080 0x55f3a1df1860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:27:57.366 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:57.365+0000 7f5806037740 1 --2- 172.21.15.47:0/4227723134 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f3a1deb6d0 0x55f3a1df4570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:27:57.366 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:57.365+0000 7f5806037740 1 -- 172.21.15.47:0/4227723134 >> 172.21.15.47:0/4227723134 conn(0x55f3a1de5da0 msgr2=0x55f3a1de7af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:27:57.366 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:57.366+0000 7f57e6000640 0 client.36377 ms_handle_reset on v2:172.21.15.47:6840/3401578978 2024-03-21T10:27:57.366 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:57.366+0000 7f5806037740 1 -- 172.21.15.47:0/4227723134 shutdown_connections 2024-03-21T10:27:57.367 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:57.366+0000 7f5806037740 1 -- 172.21.15.47:0/4227723134 wait complete. 2024-03-21T10:27:57.367 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:57.366+0000 7f5806037740 20 client.36377 trim_cache size 0 max 16384 2024-03-21T10:27:57.401 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:57.401+0000 7f5806037740 1 Processor -- start 2024-03-21T10:27:57.402 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:57.401+0000 7f5806037740 1 -- start start 2024-03-21T10:27:57.402 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:57.401+0000 7f5806037740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f3a1deb6d0 0x55f3a1de7900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:27:57.402 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:57.401+0000 7f5806037740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f3a1dec080 0x55f3a1df40a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:27:57.403 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:57.401+0000 7f5806037740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f3a1df1da0 0x55f3a1e00530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:27:57.403 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:57.401+0000 7f5806037740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57dc00fda0 con 0x55f3a1df1da0 2024-03-21T10:27:57.403 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:57.401+0000 7f5806037740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57dc011720 con 0x55f3a1deb6d0 2024-03-21T10:27:57.403 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:57.401+0000 7f5806037740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57dc010400 con 0x55f3a1dec080 2024-03-21T10:27:57.403 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:57.402+0000 7f57fbe00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f3a1deb6d0 0x55f3a1de7900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:27:57.403 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:57.402+0000 7f57fbe00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f3a1deb6d0 0x55f3a1de7900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:60958/0 (socket says 172.21.15.47:60958) 2024-03-21T10:27:57.403 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:57.402+0000 7f57fbe00640 1 -- 172.21.15.47:0/3806679030 learned_addr learned my addr 172.21.15.47:0/3806679030 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:27:57.404 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:57.402+0000 7f5801200640 1 --2- 172.21.15.47:0/3806679030 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f3a1df1da0 0x55f3a1e00530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:27:57.404 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:57.403+0000 7f57fb400640 1 --2- 172.21.15.47:0/3806679030 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f3a1dec080 0x55f3a1df40a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:27:57.404 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:57.403+0000 7f57fbe00640 1 -- 172.21.15.47:0/3806679030 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f3a1dec080 msgr2=0x55f3a1df40a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:27:57.404 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:57.403+0000 7f57fbe00640 1 --2- 172.21.15.47:0/3806679030 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f3a1dec080 0x55f3a1df40a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:27:57.404 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:57.403+0000 7f57fbe00640 1 -- 172.21.15.47:0/3806679030 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f3a1df1da0 msgr2=0x55f3a1e00530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:27:57.404 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:57.403+0000 7f57fbe00640 1 --2- 172.21.15.47:0/3806679030 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f3a1df1da0 0x55f3a1e00530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:27:57.404 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:57.403+0000 7f57fbe00640 1 -- 172.21.15.47:0/3806679030 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f3a1debe20 con 0x55f3a1deb6d0 2024-03-21T10:27:57.404 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:57.404+0000 7f57fbe00640 1 --2- 172.21.15.47:0/3806679030 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f3a1deb6d0 0x55f3a1de7900 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f57f4010c40 tx=0x7f57f4012450 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c697a2117fc729c1 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:27:57.405 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:57.404+0000 7f57f0c00640 1 -- 172.21.15.47:0/3806679030 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57f4016040 con 0x55f3a1deb6d0 2024-03-21T10:27:57.406 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:57.405+0000 7f57f0c00640 1 -- 172.21.15.47:0/3806679030 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f57f4012e40 con 0x55f3a1deb6d0 2024-03-21T10:27:57.406 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:57.405+0000 7f57f0c00640 1 -- 172.21.15.47:0/3806679030 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57f401b460 con 0x55f3a1deb6d0 2024-03-21T10:27:57.407 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:57.406+0000 7f5806037740 1 -- 172.21.15.47:0/3806679030 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f3a1deb6d0 msgr2=0x55f3a1de7900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:27:57.410 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:57.410+0000 7f5806037740 1 --2- 172.21.15.47:0/3806679030 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f3a1deb6d0 0x55f3a1de7900 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f57f4010c40 tx=0x7f57f4012450 comp rx=0 tx=0).stop 2024-03-21T10:27:57.411 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:57.410+0000 7f5806037740 1 -- 172.21.15.47:0/3806679030 shutdown_connections 2024-03-21T10:27:57.411 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:57.410+0000 7f5806037740 1 --2- 172.21.15.47:0/3806679030 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f3a1df1da0 0x55f3a1e00530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:27:57.411 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:57.410+0000 7f5806037740 1 --2- 172.21.15.47:0/3806679030 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f3a1dec080 0x55f3a1df40a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:27:57.411 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:57.410+0000 7f5806037740 1 --2- 172.21.15.47:0/3806679030 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f3a1deb6d0 0x55f3a1de7900 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:27:57.411 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:57.410+0000 7f5806037740 1 -- 172.21.15.47:0/3806679030 >> 172.21.15.47:0/3806679030 conn(0x55f3a1de5da0 msgr2=0x55f3a1de71f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:27:57.411 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:57.410+0000 7f5806037740 1 -- 172.21.15.47:0/3806679030 shutdown_connections 2024-03-21T10:27:57.412 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:57.410+0000 7f5806037740 1 -- 172.21.15.47:0/3806679030 wait complete. 2024-03-21T10:27:57.412 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:57.411+0000 7f5806037740 1 -- start start 2024-03-21T10:27:57.412 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:57.411+0000 7f5806037740 1 Processor -- start 2024-03-21T10:27:57.415 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:57.412+0000 7f5806037740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f3a1deb6d0 0x55f3a1df8b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:27:57.415 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:57.412+0000 7f5806037740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f3a1dec080 0x55f3a1df90b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:27:57.415 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:57.412+0000 7f5806037740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f3a1df1da0 0x55f3a1e5f600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:27:57.415 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:57.412+0000 7f5806037740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57dc0110c0 con 0x55f3a1deb6d0 2024-03-21T10:27:57.415 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:57.412+0000 7f5806037740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57dc010a60 con 0x55f3a1dec080 2024-03-21T10:27:57.415 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:57.412+0000 7f5806037740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57dc00df60 con 0x55f3a1df1da0 2024-03-21T10:27:57.415 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:57.412+0000 7f5806037740 10 client.0 _ll_register_callbacks cb 0x7f57ec044070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T10:27:57.415 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:57.413+0000 7f57fb400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f3a1dec080 0x55f3a1df90b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:27:57.415 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:57.413+0000 7f57fb400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f3a1dec080 0x55f3a1df90b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:60962/0 (socket says 172.21.15.47:60962) 2024-03-21T10:27:57.415 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:57.413+0000 7f57fb400640 1 -- 172.21.15.47:0/1480497982 learned_addr learned my addr 172.21.15.47:0/1480497982 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:27:57.415 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:57.413+0000 7f57fbe00640 1 --2- 172.21.15.47:0/1480497982 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f3a1deb6d0 0x55f3a1df8b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:27:57.415 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:57.413+0000 7f5801200640 1 --2- 172.21.15.47:0/1480497982 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f3a1df1da0 0x55f3a1e5f600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:27:57.415 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:57.413+0000 7f57fb400640 1 -- 172.21.15.47:0/1480497982 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f3a1df1da0 msgr2=0x55f3a1e5f600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:27:57.416 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:57.413+0000 7f57fb400640 1 --2- 172.21.15.47:0/1480497982 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f3a1df1da0 0x55f3a1e5f600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:27:57.416 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:57.413+0000 7f57fb400640 1 -- 172.21.15.47:0/1480497982 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f3a1deb6d0 msgr2=0x55f3a1df8b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:27:57.416 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:57.413+0000 7f57fb400640 1 --2- 172.21.15.47:0/1480497982 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f3a1deb6d0 0x55f3a1df8b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:27:57.416 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:57.413+0000 7f57fb400640 1 -- 172.21.15.47:0/1480497982 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55f3a1e5fb40 con 0x55f3a1dec080 2024-03-21T10:27:57.416 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:57.413+0000 7f5801200640 1 --2- 172.21.15.47:0/1480497982 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f3a1df1da0 0x55f3a1e5f600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T10:27:57.416 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:57.413+0000 7f57fb400640 1 --2- 172.21.15.47:0/1480497982 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f3a1dec080 0x55f3a1df90b0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f57e80ccf40 tx=0x7f57e809afc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=98975f1a116e3394 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:27:57.416 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:57.413+0000 7f57e6000640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-21T10:27:57.416 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:57.414+0000 7f57e6000640 1 -- 172.21.15.47:0/1480497982 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57e80b4060 con 0x55f3a1dec080 2024-03-21T10:27:57.416 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:57.414+0000 7f5806037740 1 -- 172.21.15.47:0/1480497982 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55f3a1d56b50 con 0x55f3a1dec080 2024-03-21T10:27:57.416 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:57.414+0000 7f57e6000640 1 -- 172.21.15.47:0/1480497982 <== mon.2 v2:172.21.15.100:3300/0 2 ==== osd_map(97..97 src has 1..97) v4 ==== 9474+0+0 (secure 0 0 0) 0x7f57e80cf180 con 0x55f3a1dec080 2024-03-21T10:27:57.416 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:57.414+0000 7f57e6000640 1 -- 172.21.15.47:0/1480497982 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f57e8020040 con 0x55f3a1dec080 2024-03-21T10:27:57.416 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:57.414+0000 7f5806037740 10 client.36383 fetch_fsmap learned FSMap version 32 2024-03-21T10:27:57.416 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:57.414+0000 7f5806037740 1 -- 172.21.15.47:0/1480497982 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x55f3a1e60e30 con 0x55f3a1dec080 2024-03-21T10:27:57.418 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:57.414+0000 7f57e6000640 1 -- 172.21.15.47:0/1480497982 <== mon.2 v2:172.21.15.100:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x55f3a1d56b50 con 0x55f3a1dec080 2024-03-21T10:27:57.418 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:57.415+0000 7f57e6000640 1 -- 172.21.15.47:0/1480497982 <== mon.2 v2:172.21.15.100:3300/0 5 ==== fsmap(e 32) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f57e8084b20 con 0x55f3a1dec080 2024-03-21T10:27:57.418 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:57.415+0000 7f5806037740 10 client.36383 fetch_fsmap finished waiting for FSMap version 32 2024-03-21T10:27:57.418 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:57.415+0000 7f5806037740 10 client.36383 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.a 2024-03-21T10:27:57.418 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:57.415+0000 7f5806037740 20 client.36383 populate_metadata read hostname 'smithi047' 2024-03-21T10:27:57.418 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:57.415+0000 7f5806037740 1 --2- 172.21.15.47:0/1480497982 >> [v2:172.21.15.47:6840/3401578978,v1:172.21.15.47:6841/3401578978] conn(0x55f3a1e61510 0x55f3a1e63930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:27:57.419 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:57.415+0000 7f5806037740 4 client.36383 mds_command: new command op to 14619 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-21T10:27:57.419 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:57.415+0000 7f5806037740 1 -- 172.21.15.47:0/1480497982 --> [v2:172.21.15.47:6840/3401578978,v1:172.21.15.47:6841/3401578978] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55f3a1cd4500 con 0x55f3a1e61510 2024-03-21T10:27:57.419 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:57.415+0000 7f57fbe00640 1 --2- 172.21.15.47:0/1480497982 >> [v2:172.21.15.47:6840/3401578978,v1:172.21.15.47:6841/3401578978] conn(0x55f3a1e61510 0x55f3a1e63930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:27:57.419 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:57.416+0000 7f57fbe00640 1 --2- 172.21.15.47:0/1480497982 >> [v2:172.21.15.47:6840/3401578978,v1:172.21.15.47:6841/3401578978] conn(0x55f3a1e61510 0x55f3a1e63930 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=5b93b5980ea0aa78 server_cookie=b172f0e71c829afb in_seq=0 out_seq=0 2024-03-21T10:27:57.419 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:57.416+0000 7f57e6000640 10 client.36383 ms_handle_connect on v2:172.21.15.47:6840/3401578978 2024-03-21T10:27:57.419 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:57.416+0000 7f57e6000640 1 -- 172.21.15.47:0/1480497982 <== mds.4 v2:172.21.15.47:6840/3401578978 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55f3a1cd4500 con 0x55f3a1e61510 2024-03-21T10:27:57.419 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:57.416+0000 7f57e6000640 10 client.36383 handle_command_reply: tid=0 2024-03-21T10:27:57.419 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:57.416+0000 7f5806037740 1 client.36383 shutdown 2024-03-21T10:27:57.419 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:57.416+0000 7f5806037740 1 -- 172.21.15.47:0/1480497982 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f3a1dec080 msgr2=0x55f3a1df90b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:27:57.419 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:57.416+0000 7f5806037740 1 --2- 172.21.15.47:0/1480497982 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f3a1dec080 0x55f3a1df90b0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f57e80ccf40 tx=0x7f57e809afc0 comp rx=0 tx=0).stop 2024-03-21T10:27:57.419 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:57.417+0000 7f5806037740 1 -- 172.21.15.47:0/1480497982 shutdown_connections 2024-03-21T10:27:57.419 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:57.417+0000 7f5806037740 1 --2- 172.21.15.47:0/1480497982 >> [v2:172.21.15.47:6840/3401578978,v1:172.21.15.47:6841/3401578978] conn(0x55f3a1e61510 0x55f3a1e63930 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:27:57.419 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:57.417+0000 7f5806037740 1 --2- 172.21.15.47:0/1480497982 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f3a1df1da0 0x55f3a1e5f600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:27:57.419 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:57.417+0000 7f5806037740 1 --2- 172.21.15.47:0/1480497982 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f3a1dec080 0x55f3a1df90b0 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:27:57.420 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:57.417+0000 7f5806037740 1 --2- 172.21.15.47:0/1480497982 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f3a1deb6d0 0x55f3a1df8b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:27:57.420 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:57.417+0000 7f5806037740 1 -- 172.21.15.47:0/1480497982 >> 172.21.15.47:0/1480497982 conn(0x55f3a1de5da0 msgr2=0x55f3a1de7a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:27:57.420 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:57.417+0000 7f57e6000640 0 client.36383 ms_handle_reset on v2:172.21.15.47:6840/3401578978 2024-03-21T10:27:57.420 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:57.417+0000 7f5806037740 1 -- 172.21.15.47:0/1480497982 shutdown_connections 2024-03-21T10:27:57.420 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:57.417+0000 7f5806037740 1 -- 172.21.15.47:0/1480497982 wait complete. 2024-03-21T10:27:57.420 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:57.417+0000 7f5806037740 20 client.36383 trim_cache size 0 max 16384 2024-03-21T10:27:57.420 INFO:teuthology.orchestra.run.smithi047.stdout:[] 2024-03-21T10:27:57.421 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:57.418+0000 7f5803200640 1 -- 172.21.15.47:0/93528726 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f57e00a2fd0 msgr2=0x7f57e00a5490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:27:57.421 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:57.418+0000 7f5803200640 1 --2- 172.21.15.47:0/93528726 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f57e00a2fd0 0x7f57e00a5490 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f57ec005c40 tx=0x7f57ec005a20 comp rx=0 tx=0).stop 2024-03-21T10:27:57.421 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:57.418+0000 7f5803200640 1 -- 172.21.15.47:0/93528726 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f57fc074390 msgr2=0x7f57fc19a6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:27:57.421 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:57.418+0000 7f5803200640 1 --2- 172.21.15.47:0/93528726 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f57fc074390 0x7f57fc19a6a0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f57e800d950 tx=0x7f57e800de10 comp rx=0 tx=0).stop 2024-03-21T10:27:57.421 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:57.418+0000 7f5803200640 1 -- 172.21.15.47:0/93528726 shutdown_connections 2024-03-21T10:27:57.421 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:57.418+0000 7f5803200640 1 --2- 172.21.15.47:0/93528726 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f57e00a2fd0 0x7f57e00a5490 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:27:57.421 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:57.418+0000 7f5803200640 1 --2- 172.21.15.47:0/93528726 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f57fc10dee0 0x7f57fc19e9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:27:57.421 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:57.418+0000 7f5803200640 1 --2- 172.21.15.47:0/93528726 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f57fc074390 0x7f57fc19a6a0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:27:57.421 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:57.418+0000 7f5803200640 1 --2- 172.21.15.47:0/93528726 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f57fc073a70 0x7f57fc19a160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:27:57.422 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:57.418+0000 7f5803200640 1 -- 172.21.15.47:0/93528726 >> 172.21.15.47:0/93528726 conn(0x7f57fc0fd0d0 msgr2=0x7f57fc0fdfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:27:57.422 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:57.418+0000 7f5803200640 1 -- 172.21.15.47:0/93528726 shutdown_connections 2024-03-21T10:27:57.422 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:57.418+0000 7f5803200640 1 -- 172.21.15.47:0/93528726 wait complete. 2024-03-21T10:27:57.438 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-21T10:27:57.438 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-03-21T10:27:57.712 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:27:57 smithi047 ceph-mon[31547]: pgmap v82: 129 pgs: 129 active+clean; 273 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 10 MiB/s wr, 2.26k op/s 2024-03-21T10:27:57.712 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:27:57 smithi047 ceph-mon[31547]: from='client.? 172.21.15.47:0/2430338055' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-21T10:27:57.754 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:57.753+0000 7f6608200640 1 -- 172.21.15.47:0/3161538090 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f66001124e0 msgr2=0x7f66001148d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:27:57.754 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:57.753+0000 7f6608200640 1 --2- 172.21.15.47:0/3161538090 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f66001124e0 0x7f66001148d0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f65f8009c10 tx=0x7f65f802bf10 comp rx=0 tx=0).stop 2024-03-21T10:27:57.754 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:57.753+0000 7f6608200640 1 -- 172.21.15.47:0/3161538090 shutdown_connections 2024-03-21T10:27:57.754 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:57.753+0000 7f6608200640 1 --2- 172.21.15.47:0/3161538090 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f66001124e0 0x7f66001148d0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:27:57.754 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:57.753+0000 7f6608200640 1 --2- 172.21.15.47:0/3161538090 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f66000764f0 0x7f6600111fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:27:57.754 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:57.753+0000 7f6608200640 1 --2- 172.21.15.47:0/3161538090 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6600075bd0 0x7f6600075fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:27:57.755 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:57.753+0000 7f6608200640 1 -- 172.21.15.47:0/3161538090 >> 172.21.15.47:0/3161538090 conn(0x7f66000ff310 msgr2=0x7f6600101730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:27:57.755 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:57.753+0000 7f6608200640 1 -- 172.21.15.47:0/3161538090 shutdown_connections 2024-03-21T10:27:57.755 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:57.754+0000 7f6608200640 1 -- 172.21.15.47:0/3161538090 wait complete. 2024-03-21T10:27:57.756 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:57.756+0000 7f6608200640 1 Processor -- start 2024-03-21T10:27:57.757 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:57.756+0000 7f6608200640 1 -- start start 2024-03-21T10:27:57.758 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:57.757+0000 7f6608200640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6600075bd0 0x7f660019e480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:27:57.758 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:57.757+0000 7f6608200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f66000764f0 0x7f660019e9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:27:57.758 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:57.757+0000 7f6608200640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f66001124e0 0x7f66001a2d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:27:57.758 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:57.757+0000 7f6608200640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6600117f40 con 0x7f66001124e0 2024-03-21T10:27:57.758 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:57.757+0000 7f6608200640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6600117dc0 con 0x7f66000764f0 2024-03-21T10:27:57.758 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:57.757+0000 7f6608200640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66001180c0 con 0x7f6600075bd0 2024-03-21T10:27:57.759 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:57.758+0000 7f6605800640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6600075bd0 0x7f660019e480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:27:57.759 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:57.758+0000 7f6605800640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6600075bd0 0x7f660019e480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.47:40150/0 (socket says 172.21.15.47:40150) 2024-03-21T10:27:57.759 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:57.758+0000 7f6605800640 1 -- 172.21.15.47:0/1320937196 learned_addr learned my addr 172.21.15.47:0/1320937196 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:27:57.759 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:57.758+0000 7f6604e00640 1 --2- 172.21.15.47:0/1320937196 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f66000764f0 0x7f660019e9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:27:57.759 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:57.758+0000 7f6606200640 1 --2- 172.21.15.47:0/1320937196 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f66001124e0 0x7f66001a2d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:27:57.760 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:57.759+0000 7f6605800640 1 -- 172.21.15.47:0/1320937196 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f66000764f0 msgr2=0x7f660019e9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:27:57.760 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:57.759+0000 7f6605800640 1 --2- 172.21.15.47:0/1320937196 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f66000764f0 0x7f660019e9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:27:57.760 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:57.759+0000 7f6605800640 1 -- 172.21.15.47:0/1320937196 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f66001124e0 msgr2=0x7f66001a2d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:27:57.760 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:57.759+0000 7f6605800640 1 --2- 172.21.15.47:0/1320937196 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f66001124e0 0x7f66001a2d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:27:57.760 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:57.759+0000 7f6605800640 1 -- 172.21.15.47:0/1320937196 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f66001a3470 con 0x7f6600075bd0 2024-03-21T10:27:57.760 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:57.760+0000 7f6605800640 1 --2- 172.21.15.47:0/1320937196 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6600075bd0 0x7f660019e480 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f65f400e9f0 tx=0x7f65f400eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:27:57.762 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:57.761+0000 7f65fe000640 1 -- 172.21.15.47:0/1320937196 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65f400cd80 con 0x7f6600075bd0 2024-03-21T10:27:57.763 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:57.762+0000 7f65fe000640 1 -- 172.21.15.47:0/1320937196 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f65f40108e0 con 0x7f6600075bd0 2024-03-21T10:27:57.763 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:57.762+0000 7f65fe000640 1 -- 172.21.15.47:0/1320937196 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65f4018920 con 0x7f6600075bd0 2024-03-21T10:27:57.763 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:57.762+0000 7f6608200640 1 -- 172.21.15.47:0/1320937196 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f66001a3760 con 0x7f6600075bd0 2024-03-21T10:27:57.763 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:57.762+0000 7f6608200640 1 -- 172.21.15.47:0/1320937196 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f66001aaff0 con 0x7f6600075bd0 2024-03-21T10:27:57.773 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:57.773+0000 7f660ae47740 1 Processor -- start 2024-03-21T10:27:57.775 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:57.774+0000 7f65fe000640 1 -- 172.21.15.47:0/1320937196 <== mon.1 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 27) v1 ==== 137985+0+0 (secure 0 0 0) 0x7f65f4010a90 con 0x7f6600075bd0 2024-03-21T10:27:57.775 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:57.774+0000 7f660ae47740 1 -- start start 2024-03-21T10:27:57.776 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:57.775+0000 7f660ae47740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560d37cde6d0 0x560d37cdeab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:27:57.776 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:57.775+0000 7f660ae47740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560d37cdf080 0x560d37ce7630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:27:57.776 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:57.775+0000 7f660ae47740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x560d37ce7c00 0x560d37cea050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:27:57.776 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:57.775+0000 7f660ae47740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x560d37c422c0 con 0x560d37ce7c00 2024-03-21T10:27:57.776 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:57.775+0000 7f660ae47740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x560d37beac90 con 0x560d37cde6d0 2024-03-21T10:27:57.776 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:57.775+0000 7f660ae47740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x560d37c3e360 con 0x560d37cdf080 2024-03-21T10:27:57.776 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:57.776+0000 7f6604e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x560d37ce7c00 0x560d37cea050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:27:57.777 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:57.776+0000 7f6604e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x560d37ce7c00 0x560d37cea050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:57674/0 (socket says 172.21.15.47:57674) 2024-03-21T10:27:57.777 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:57.776+0000 7f6604e00640 1 -- 172.21.15.47:0/3857884699 learned_addr learned my addr 172.21.15.47:0/3857884699 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:27:57.777 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:57.776+0000 7f6606200640 1 --2- 172.21.15.47:0/3857884699 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560d37cde6d0 0x560d37cdeab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:27:57.778 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:57.777+0000 7f6605800640 1 --2- 172.21.15.47:0/3857884699 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560d37cdf080 0x560d37ce7630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:27:57.782 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:57.782+0000 7f65fe000640 1 --2- 172.21.15.47:0/1320937196 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f65e80a3110 0x7f65e80a55d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:27:57.783 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:57.782+0000 7f6606200640 1 -- 172.21.15.47:0/3857884699 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560d37cdf080 msgr2=0x560d37ce7630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:27:57.783 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:57.782+0000 7f6606200640 1 --2- 172.21.15.47:0/3857884699 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560d37cdf080 0x560d37ce7630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:27:57.783 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:57.782+0000 7f6606200640 1 -- 172.21.15.47:0/3857884699 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x560d37ce7c00 msgr2=0x560d37cea050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:27:57.784 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:57.782+0000 7f6606200640 1 --2- 172.21.15.47:0/3857884699 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x560d37ce7c00 0x560d37cea050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:27:57.784 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:57.782+0000 7f6606200640 1 -- 172.21.15.47:0/3857884699 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560d37c0f080 con 0x560d37cde6d0 2024-03-21T10:27:57.784 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:57.783+0000 7f6604e00640 1 --2- 172.21.15.47:0/3857884699 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x560d37ce7c00 0x560d37cea050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T10:27:57.785 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:57.783+0000 7f6606200640 1 --2- 172.21.15.47:0/3857884699 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560d37cde6d0 0x560d37cdeab0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f65f802c520 tx=0x7f65f8005ec0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=86c7764a8732a6d4 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:27:57.785 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:57.783+0000 7f65fe000640 1 -- 172.21.15.47:0/1320937196 <== mon.1 v2:172.21.15.157:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 9474+0+0 (secure 0 0 0) 0x7f65f40cfec0 con 0x7f6600075bd0 2024-03-21T10:27:57.785 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:57.783+0000 7f65f2a00640 1 -- 172.21.15.47:0/3857884699 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65f8045070 con 0x560d37cde6d0 2024-03-21T10:27:57.785 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:57.784+0000 7f65f2a00640 1 -- 172.21.15.47:0/3857884699 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f65f8003970 con 0x560d37cde6d0 2024-03-21T10:27:57.785 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:57.784+0000 7f65f2a00640 1 -- 172.21.15.47:0/3857884699 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65f8004410 con 0x560d37cde6d0 2024-03-21T10:27:57.785 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:57.784+0000 7f6606200640 1 --2- 172.21.15.47:0/1320937196 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f65e80a3110 0x7f65e80a55d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:27:57.788 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:57.787+0000 7f660ae47740 1 -- 172.21.15.47:0/3857884699 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560d37cde6d0 msgr2=0x560d37cdeab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:27:57.789 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:57.787+0000 7f6606200640 1 --2- 172.21.15.47:0/1320937196 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f65e80a3110 0x7f65e80a55d0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f65f802e010 tx=0x7f65f804b040 comp rx=0 tx=0).ready entity=mgr.15687 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:27:57.789 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:57.787+0000 7f660ae47740 1 --2- 172.21.15.47:0/3857884699 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560d37cde6d0 0x560d37cdeab0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f65f802c520 tx=0x7f65f8005ec0 comp rx=0 tx=0).stop 2024-03-21T10:27:57.789 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:57.788+0000 7f660ae47740 1 -- 172.21.15.47:0/3857884699 shutdown_connections 2024-03-21T10:27:57.789 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:57.788+0000 7f660ae47740 1 --2- 172.21.15.47:0/3857884699 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x560d37ce7c00 0x560d37cea050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:27:57.789 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:57.788+0000 7f660ae47740 1 --2- 172.21.15.47:0/3857884699 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560d37cdf080 0x560d37ce7630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:27:57.789 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:57.788+0000 7f660ae47740 1 --2- 172.21.15.47:0/3857884699 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560d37cde6d0 0x560d37cdeab0 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:27:57.789 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:57.788+0000 7f660ae47740 1 -- 172.21.15.47:0/3857884699 >> 172.21.15.47:0/3857884699 conn(0x560d37cd8da0 msgr2=0x560d37cd9180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:27:57.789 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:57.789+0000 7f660ae47740 1 -- 172.21.15.47:0/3857884699 shutdown_connections 2024-03-21T10:27:57.791 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:57.789+0000 7f660ae47740 1 -- 172.21.15.47:0/3857884699 wait complete. 2024-03-21T10:27:57.791 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:57.790+0000 7f660ae47740 1 -- start start 2024-03-21T10:27:57.791 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:57.790+0000 7f660ae47740 1 Processor -- start 2024-03-21T10:27:57.792 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:57.792+0000 7f660ae47740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560d37cde6d0 0x560d37ce7590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:27:57.792 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:57.792+0000 7f660ae47740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560d37cdf080 0x560d37ce4920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:27:57.793 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:57.792+0000 7f660ae47740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x560d37ce7c00 0x560d37ce4f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:27:57.793 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:57.793+0000 7f6606200640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x560d37ce7c00 0x560d37ce4f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:27:57.794 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:57.793+0000 7f6605800640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560d37cde6d0 0x560d37ce7590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:27:57.794 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:57.793+0000 7f6605800640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560d37cde6d0 0x560d37ce7590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.47:40168/0 (socket says 172.21.15.47:40168) 2024-03-21T10:27:57.794 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:57.793+0000 7f6604e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560d37cdf080 0x560d37ce4920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:27:57.794 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:57.793+0000 7f6604e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560d37cdf080 0x560d37ce4920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:60990/0 (socket says 172.21.15.47:60990) 2024-03-21T10:27:57.794 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:57.793+0000 7f6606200640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x560d37ce7c00 0x560d37ce4f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:57678/0 (socket says 172.21.15.47:57678) 2024-03-21T10:27:57.794 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:57.793+0000 7f6605800640 1 -- 172.21.15.47:0/1910183767 learned_addr learned my addr 172.21.15.47:0/1910183767 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:27:57.794 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:57.793+0000 7f660ae47740 1 -- 172.21.15.47:0/1910183767 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x560d37ceab10 con 0x560d37ce7c00 2024-03-21T10:27:57.795 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:57.793+0000 7f660ae47740 1 -- 172.21.15.47:0/1910183767 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x560d37bfe0d0 con 0x560d37cdf080 2024-03-21T10:27:57.795 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:57.794+0000 7f660ae47740 1 -- 172.21.15.47:0/1910183767 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x560d37ceac90 con 0x560d37cde6d0 2024-03-21T10:27:57.795 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:57.794+0000 7f660ae47740 10 client.0 _ll_register_callbacks cb 0x560d37c02e80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T10:27:57.796 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:57.795+0000 7f6605800640 1 -- 172.21.15.47:0/1910183767 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560d37cdf080 msgr2=0x560d37ce4920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:27:57.796 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:57.795+0000 7f6605800640 1 --2- 172.21.15.47:0/1910183767 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560d37cdf080 0x560d37ce4920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:27:57.796 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:57.795+0000 7f6605800640 1 -- 172.21.15.47:0/1910183767 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x560d37ce7c00 msgr2=0x560d37ce4f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:27:57.796 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:57.795+0000 7f6605800640 1 --2- 172.21.15.47:0/1910183767 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x560d37ce7c00 0x560d37ce4f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:27:57.796 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:57.795+0000 7f6605800640 1 -- 172.21.15.47:0/1910183767 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x560d379563f0 con 0x560d37cde6d0 2024-03-21T10:27:57.797 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:57.796+0000 7f6605800640 1 --2- 172.21.15.47:0/1910183767 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560d37cde6d0 0x560d37ce7590 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f65f400e9c0 tx=0x7f65f40d3600 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bedb9f7d0e1c23be server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:27:57.798 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:57.797+0000 7f65e7400640 10 client.0 ms_handle_connect on v2:172.21.15.157:3300/0 2024-03-21T10:27:57.798 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:57.797+0000 7f65e7400640 1 -- 172.21.15.47:0/1910183767 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65f409b850 con 0x560d37cde6d0 2024-03-21T10:27:57.798 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:57.797+0000 7f65e7400640 1 -- 172.21.15.47:0/1910183767 <== mon.1 v2:172.21.15.157:3300/0 2 ==== osd_map(97..97 src has 1..97) v4 ==== 9474+0+0 (secure 0 0 0) 0x7f65f40d3dc0 con 0x560d37cde6d0 2024-03-21T10:27:57.798 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:57.797+0000 7f660ae47740 1 -- 172.21.15.47:0/1910183767 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f65f8003970 con 0x560d37cde6d0 2024-03-21T10:27:57.801 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:57.800+0000 7f65e7400640 1 -- 172.21.15.47:0/1910183767 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f65f4010de0 con 0x560d37cde6d0 2024-03-21T10:27:57.801 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:57.800+0000 7f660ae47740 10 client.26152 fetch_fsmap learned FSMap version 32 2024-03-21T10:27:57.801 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:57.800+0000 7f660ae47740 1 -- 172.21.15.47:0/1910183767 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x560d37a207d0 con 0x560d37cde6d0 2024-03-21T10:27:57.802 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:57.801+0000 7f65e7400640 1 -- 172.21.15.47:0/1910183767 <== mon.1 v2:172.21.15.157:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f65f8003970 con 0x560d37cde6d0 2024-03-21T10:27:57.802 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:57.801+0000 7f65e7400640 1 -- 172.21.15.47:0/1910183767 <== mon.1 v2:172.21.15.157:3300/0 5 ==== fsmap(e 32) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f65f409b340 con 0x560d37cde6d0 2024-03-21T10:27:57.802 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:57.801+0000 7f660ae47740 10 client.26152 fetch_fsmap finished waiting for FSMap version 32 2024-03-21T10:27:57.803 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:57.801+0000 7f660ae47740 10 client.26152 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.k 2024-03-21T10:27:57.803 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:57.801+0000 7f660ae47740 20 client.26152 populate_metadata read hostname 'smithi047' 2024-03-21T10:27:57.803 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:57.801+0000 7f660ae47740 1 --2- 172.21.15.47:0/1910183767 >> [v2:172.21.15.100:6832/2723302038,v1:172.21.15.100:6833/2723302038] conn(0x560d37cf7a40 0x560d37cf9e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:27:57.803 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:57.801+0000 7f660ae47740 4 client.26152 mds_command: new command op to 34427 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T10:27:57.803 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:57.801+0000 7f660ae47740 1 -- 172.21.15.47:0/1910183767 --> [v2:172.21.15.100:6832/2723302038,v1:172.21.15.100:6833/2723302038] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x560d37c4f800 con 0x560d37cf7a40 2024-03-21T10:27:57.803 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:57.802+0000 7f6604e00640 1 --2- 172.21.15.47:0/1910183767 >> [v2:172.21.15.100:6832/2723302038,v1:172.21.15.100:6833/2723302038] conn(0x560d37cf7a40 0x560d37cf9e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:27:57.805 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:57.804+0000 7f6604e00640 1 --2- 172.21.15.47:0/1910183767 >> [v2:172.21.15.100:6832/2723302038,v1:172.21.15.100:6833/2723302038] conn(0x560d37cf7a40 0x560d37cf9e30 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=5f4b22f3fe9c5ee0 server_cookie=da04c0f97422e4be in_seq=0 out_seq=0 2024-03-21T10:27:57.805 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:57.804+0000 7f65e7400640 10 client.26152 ms_handle_connect on v2:172.21.15.100:6832/2723302038 2024-03-21T10:27:57.806 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:57.806+0000 7f65e7400640 1 -- 172.21.15.47:0/1910183767 <== mds.3 v2:172.21.15.100:6832/2723302038 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x560d37c4f800 con 0x560d37cf7a40 2024-03-21T10:27:57.807 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:57.806+0000 7f65e7400640 10 client.26152 handle_command_reply: tid=0 2024-03-21T10:27:57.807 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:57.806+0000 7f660ae47740 1 client.26152 shutdown 2024-03-21T10:27:57.808 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:57.807+0000 7f660ae47740 1 -- 172.21.15.47:0/1910183767 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560d37cde6d0 msgr2=0x560d37ce7590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:27:57.808 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:57.807+0000 7f660ae47740 1 --2- 172.21.15.47:0/1910183767 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560d37cde6d0 0x560d37ce7590 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f65f400e9c0 tx=0x7f65f40d3600 comp rx=0 tx=0).stop 2024-03-21T10:27:57.808 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:57.808+0000 7f660ae47740 1 -- 172.21.15.47:0/1910183767 shutdown_connections 2024-03-21T10:27:57.808 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:57.808+0000 7f660ae47740 1 --2- 172.21.15.47:0/1910183767 >> [v2:172.21.15.100:6832/2723302038,v1:172.21.15.100:6833/2723302038] conn(0x560d37cf7a40 0x560d37cf9e30 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:27:57.809 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:57.808+0000 7f660ae47740 1 --2- 172.21.15.47:0/1910183767 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x560d37ce7c00 0x560d37ce4f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:27:57.809 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:57.808+0000 7f660ae47740 1 --2- 172.21.15.47:0/1910183767 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560d37cdf080 0x560d37ce4920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:27:57.809 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:57.808+0000 7f65e7400640 0 client.26152 ms_handle_reset on v2:172.21.15.100:6832/2723302038 2024-03-21T10:27:57.809 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:57.808+0000 7f660ae47740 1 --2- 172.21.15.47:0/1910183767 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560d37cde6d0 0x560d37ce7590 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:27:57.809 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:57.808+0000 7f660ae47740 1 -- 172.21.15.47:0/1910183767 >> 172.21.15.47:0/1910183767 conn(0x560d37cd8da0 msgr2=0x560d37cdaaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:27:57.810 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:57.810+0000 7f660ae47740 1 -- 172.21.15.47:0/1910183767 shutdown_connections 2024-03-21T10:27:57.811 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:57.810+0000 7f660ae47740 1 -- 172.21.15.47:0/1910183767 wait complete. 2024-03-21T10:27:57.811 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:57.810+0000 7f660ae47740 20 client.26152 trim_cache size 0 max 16384 2024-03-21T10:27:57.845 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:57.844+0000 7f660ae47740 1 Processor -- start 2024-03-21T10:27:57.845 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:57.844+0000 7f660ae47740 1 -- start start 2024-03-21T10:27:57.846 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:57.845+0000 7f660ae47740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x560d37cde6d0 0x560d37cd7620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:27:57.846 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:57.845+0000 7f660ae47740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560d37cdf080 0x560d37ce7050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:27:57.846 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:57.845+0000 7f660ae47740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560d37ce7c00 0x560d37ce7590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:27:57.847 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:57.846+0000 7f6604e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560d37cdf080 0x560d37ce7050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:27:57.847 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:57.846+0000 7f6604e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560d37cdf080 0x560d37ce7050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:32774/0 (socket says 172.21.15.47:32774) 2024-03-21T10:27:57.847 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:57.846+0000 7f6604e00640 1 -- 172.21.15.47:0/2442296160 learned_addr learned my addr 172.21.15.47:0/2442296160 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:27:57.847 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:57.846+0000 7f660ae47740 1 -- 172.21.15.47:0/2442296160 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66001b4ec0 con 0x560d37cde6d0 2024-03-21T10:27:57.847 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:57.846+0000 7f6606200640 1 --2- 172.21.15.47:0/2442296160 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560d37ce7c00 0x560d37ce7590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:27:57.847 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:57.846+0000 7f660ae47740 1 -- 172.21.15.47:0/2442296160 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66001b6840 con 0x560d37cdf080 2024-03-21T10:27:57.847 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:57.846+0000 7f660ae47740 1 -- 172.21.15.47:0/2442296160 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66001b5520 con 0x560d37ce7c00 2024-03-21T10:27:57.848 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:57.847+0000 7f6605800640 1 --2- 172.21.15.47:0/2442296160 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x560d37cde6d0 0x560d37cd7620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:27:57.848 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:57.847+0000 7f6604e00640 1 -- 172.21.15.47:0/2442296160 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560d37ce7c00 msgr2=0x560d37ce7590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:27:57.848 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:57.847+0000 7f6604e00640 1 --2- 172.21.15.47:0/2442296160 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560d37ce7c00 0x560d37ce7590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:27:57.848 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:57.847+0000 7f6604e00640 1 -- 172.21.15.47:0/2442296160 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x560d37cde6d0 msgr2=0x560d37cd7620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:27:57.848 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:57.847+0000 7f6604e00640 1 --2- 172.21.15.47:0/2442296160 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x560d37cde6d0 0x560d37cd7620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:27:57.848 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:57.847+0000 7f6604e00640 1 -- 172.21.15.47:0/2442296160 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560d37ce52a0 con 0x560d37cdf080 2024-03-21T10:27:57.849 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:57.848+0000 7f6604e00640 1 --2- 172.21.15.47:0/2442296160 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560d37cdf080 0x560d37ce7050 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f65ec005fd0 tx=0x7f65ec01fb40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7d1a5638a3bde00c server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:27:57.849 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:57.849+0000 7f65f2000640 1 -- 172.21.15.47:0/2442296160 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65ec01f0f0 con 0x560d37cdf080 2024-03-21T10:27:57.850 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:57.849+0000 7f65f2000640 1 -- 172.21.15.47:0/2442296160 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f65ec023070 con 0x560d37cdf080 2024-03-21T10:27:57.850 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:57.849+0000 7f65f2000640 1 -- 172.21.15.47:0/2442296160 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65ec01da90 con 0x560d37cdf080 2024-03-21T10:27:57.851 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:57.850+0000 7f660ae47740 1 -- 172.21.15.47:0/2442296160 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560d37cdf080 msgr2=0x560d37ce7050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:27:57.851 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:57.850+0000 7f660ae47740 1 --2- 172.21.15.47:0/2442296160 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560d37cdf080 0x560d37ce7050 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f65ec005fd0 tx=0x7f65ec01fb40 comp rx=0 tx=0).stop 2024-03-21T10:27:57.851 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:57.851+0000 7f660ae47740 1 -- 172.21.15.47:0/2442296160 shutdown_connections 2024-03-21T10:27:57.851 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:57.851+0000 7f660ae47740 1 --2- 172.21.15.47:0/2442296160 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560d37ce7c00 0x560d37ce7590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:27:57.852 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:57.851+0000 7f660ae47740 1 --2- 172.21.15.47:0/2442296160 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560d37cdf080 0x560d37ce7050 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:27:57.852 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:57.851+0000 7f660ae47740 1 --2- 172.21.15.47:0/2442296160 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x560d37cde6d0 0x560d37cd7620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:27:57.852 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:57.851+0000 7f660ae47740 1 -- 172.21.15.47:0/2442296160 >> 172.21.15.47:0/2442296160 conn(0x560d37cd8da0 msgr2=0x560d37cdfcf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:27:57.852 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:57.851+0000 7f660ae47740 1 -- 172.21.15.47:0/2442296160 shutdown_connections 2024-03-21T10:27:57.853 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:57.852+0000 7f660ae47740 1 -- 172.21.15.47:0/2442296160 wait complete. 2024-03-21T10:27:57.853 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:57.852+0000 7f660ae47740 1 -- start start 2024-03-21T10:27:57.853 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:57.853+0000 7f660ae47740 1 Processor -- start 2024-03-21T10:27:57.855 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:57.854+0000 7f660ae47740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x560d37cde6d0 0x560d37cefbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:27:57.855 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:57.854+0000 7f660ae47740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560d37cdf080 0x560d37cedf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:27:57.856 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:57.855+0000 7f660ae47740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560d37ce7c00 0x560d37cee4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:27:57.856 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:57.855+0000 7f660ae47740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66001b61e0 con 0x560d37cde6d0 2024-03-21T10:27:57.856 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:57.855+0000 7f660ae47740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66001b5b80 con 0x560d37cdf080 2024-03-21T10:27:57.856 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:57.855+0000 7f660ae47740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66001b3080 con 0x560d37ce7c00 2024-03-21T10:27:57.856 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:57.855+0000 7f660ae47740 10 client.0 _ll_register_callbacks cb 0x7f65f40d3700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T10:27:57.856 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:57.855+0000 7f6604e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560d37cdf080 0x560d37cedf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:27:57.857 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:57.855+0000 7f6606200640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560d37ce7c00 0x560d37cee4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:27:57.857 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:57.856+0000 7f6606200640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560d37ce7c00 0x560d37cee4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.47:40174/0 (socket says 172.21.15.47:40174) 2024-03-21T10:27:57.857 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:57.856+0000 7f6606200640 1 -- 172.21.15.47:0/1389009393 learned_addr learned my addr 172.21.15.47:0/1389009393 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:27:57.857 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:57.856+0000 7f6605800640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x560d37cde6d0 0x560d37cefbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:27:57.858 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:57.857+0000 7f6604e00640 1 -- 172.21.15.47:0/1389009393 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560d37ce7c00 msgr2=0x560d37cee4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:27:57.858 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:57.857+0000 7f6604e00640 1 --2- 172.21.15.47:0/1389009393 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560d37ce7c00 0x560d37cee4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:27:57.858 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:57.857+0000 7f6604e00640 1 -- 172.21.15.47:0/1389009393 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x560d37cde6d0 msgr2=0x560d37cefbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:27:57.858 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:57.857+0000 7f6604e00640 1 --2- 172.21.15.47:0/1389009393 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x560d37cde6d0 0x560d37cefbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:27:57.858 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:57.857+0000 7f6604e00640 1 -- 172.21.15.47:0/1389009393 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x560d37ceec80 con 0x560d37cdf080 2024-03-21T10:27:57.859 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:57.858+0000 7f65e7400640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-21T10:27:57.859 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:57.858+0000 7f6604e00640 1 --2- 172.21.15.47:0/1389009393 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560d37cdf080 0x560d37cedf90 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f65ec000c00 tx=0x7f65ec009e80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cccb7f62d505de2d server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:27:57.859 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:57.858+0000 7f65e7400640 1 -- 172.21.15.47:0/1389009393 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65ec02c070 con 0x560d37cdf080 2024-03-21T10:27:57.859 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:57.859+0000 7f65e7400640 1 -- 172.21.15.47:0/1389009393 <== mon.2 v2:172.21.15.100:3300/0 2 ==== osd_map(97..97 src has 1..97) v4 ==== 9474+0+0 (secure 0 0 0) 0x7f65ec004220 con 0x560d37cdf080 2024-03-21T10:27:57.859 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:57.859+0000 7f660ae47740 1 -- 172.21.15.47:0/1389009393 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f65ec023070 con 0x560d37cdf080 2024-03-21T10:27:57.860 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:57.859+0000 7f65e7400640 1 -- 172.21.15.47:0/1389009393 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f65ec035050 con 0x560d37cdf080 2024-03-21T10:27:57.860 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:57.859+0000 7f660ae47740 10 client.36401 fetch_fsmap learned FSMap version 32 2024-03-21T10:27:57.860 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:57.860+0000 7f660ae47740 1 -- 172.21.15.47:0/1389009393 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x560d37d54200 con 0x560d37cdf080 2024-03-21T10:27:57.861 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:57.860+0000 7f65e7400640 1 -- 172.21.15.47:0/1389009393 <== mon.2 v2:172.21.15.100:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f65ec023070 con 0x560d37cdf080 2024-03-21T10:27:57.861 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:57.860+0000 7f65e7400640 1 -- 172.21.15.47:0/1389009393 <== mon.2 v2:172.21.15.100:3300/0 5 ==== fsmap(e 32) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f65ec0052d0 con 0x560d37cdf080 2024-03-21T10:27:57.862 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:57.861+0000 7f660ae47740 10 client.36401 fetch_fsmap finished waiting for FSMap version 32 2024-03-21T10:27:57.862 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:57.861+0000 7f660ae47740 10 client.36401 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.k 2024-03-21T10:27:57.862 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:57.861+0000 7f660ae47740 20 client.36401 populate_metadata read hostname 'smithi047' 2024-03-21T10:27:57.862 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:57.861+0000 7f660ae47740 1 --2- 172.21.15.47:0/1389009393 >> [v2:172.21.15.100:6832/2723302038,v1:172.21.15.100:6833/2723302038] conn(0x560d37d548c0 0x560d37d56ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:27:57.862 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:57.861+0000 7f660ae47740 4 client.36401 mds_command: new command op to 34427 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-21T10:27:57.862 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:57.861+0000 7f660ae47740 1 -- 172.21.15.47:0/1389009393 --> [v2:172.21.15.100:6832/2723302038,v1:172.21.15.100:6833/2723302038] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x560d37c49b50 con 0x560d37d548c0 2024-03-21T10:27:57.862 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:57.861+0000 7f6605800640 1 --2- 172.21.15.47:0/1389009393 >> [v2:172.21.15.100:6832/2723302038,v1:172.21.15.100:6833/2723302038] conn(0x560d37d548c0 0x560d37d56ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:27:57.865 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:57.864+0000 7f6605800640 1 --2- 172.21.15.47:0/1389009393 >> [v2:172.21.15.100:6832/2723302038,v1:172.21.15.100:6833/2723302038] conn(0x560d37d548c0 0x560d37d56ce0 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=7ec62fa40fad40b server_cookie=ca5c7fa854133e9f in_seq=0 out_seq=0 2024-03-21T10:27:57.865 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:57.864+0000 7f65e7400640 10 client.36401 ms_handle_connect on v2:172.21.15.100:6832/2723302038 2024-03-21T10:27:57.865 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:57.865+0000 7f65e7400640 1 -- 172.21.15.47:0/1389009393 <== mds.3 v2:172.21.15.100:6832/2723302038 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x560d37c49b50 con 0x560d37d548c0 2024-03-21T10:27:57.865 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:57.865+0000 7f65e7400640 10 client.36401 handle_command_reply: tid=0 2024-03-21T10:27:57.866 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:57.865+0000 7f660ae47740 1 client.36401 shutdown 2024-03-21T10:27:57.868 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:57.867+0000 7f660ae47740 1 -- 172.21.15.47:0/1389009393 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560d37cdf080 msgr2=0x560d37cedf90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:27:57.868 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:57.867+0000 7f660ae47740 1 --2- 172.21.15.47:0/1389009393 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560d37cdf080 0x560d37cedf90 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f65ec000c00 tx=0x7f65ec009e80 comp rx=0 tx=0).stop 2024-03-21T10:27:57.868 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:57.867+0000 7f660ae47740 1 -- 172.21.15.47:0/1389009393 shutdown_connections 2024-03-21T10:27:57.868 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:57.867+0000 7f660ae47740 1 --2- 172.21.15.47:0/1389009393 >> [v2:172.21.15.100:6832/2723302038,v1:172.21.15.100:6833/2723302038] conn(0x560d37d548c0 0x560d37d56ce0 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:27:57.868 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:57.867+0000 7f65e7400640 0 client.36401 ms_handle_reset on v2:172.21.15.100:6832/2723302038 2024-03-21T10:27:57.868 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:57.867+0000 7f660ae47740 1 --2- 172.21.15.47:0/1389009393 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560d37ce7c00 0x560d37cee4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:27:57.868 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:57.867+0000 7f660ae47740 1 --2- 172.21.15.47:0/1389009393 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560d37cdf080 0x560d37cedf90 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:27:57.868 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:57.867+0000 7f660ae47740 1 --2- 172.21.15.47:0/1389009393 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x560d37cde6d0 0x560d37cefbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:27:57.869 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:57.867+0000 7f660ae47740 1 -- 172.21.15.47:0/1389009393 >> 172.21.15.47:0/1389009393 conn(0x560d37cd8da0 msgr2=0x560d37cd9f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:27:57.869 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:57.868+0000 7f660ae47740 1 -- 172.21.15.47:0/1389009393 shutdown_connections 2024-03-21T10:27:57.869 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:57.868+0000 7f660ae47740 1 -- 172.21.15.47:0/1389009393 wait complete. 2024-03-21T10:27:57.869 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:57.868+0000 7f660ae47740 20 client.36401 trim_cache size 0 max 16384 2024-03-21T10:27:57.870 INFO:teuthology.orchestra.run.smithi047.stdout:[] 2024-03-21T10:27:57.871 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:57.870+0000 7f6608200640 1 -- 172.21.15.47:0/1320937196 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f65e80a3110 msgr2=0x7f65e80a55d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:27:57.871 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:57.870+0000 7f6608200640 1 --2- 172.21.15.47:0/1320937196 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f65e80a3110 0x7f65e80a55d0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f65f802e010 tx=0x7f65f804b040 comp rx=0 tx=0).stop 2024-03-21T10:27:57.871 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:57.870+0000 7f6608200640 1 -- 172.21.15.47:0/1320937196 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6600075bd0 msgr2=0x7f660019e480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:27:57.871 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:57.870+0000 7f6608200640 1 --2- 172.21.15.47:0/1320937196 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6600075bd0 0x7f660019e480 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f65f400e9f0 tx=0x7f65f400eeb0 comp rx=0 tx=0).stop 2024-03-21T10:27:57.871 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:57.870+0000 7f6608200640 1 -- 172.21.15.47:0/1320937196 shutdown_connections 2024-03-21T10:27:57.871 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:57.870+0000 7f6608200640 1 --2- 172.21.15.47:0/1320937196 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f66001124e0 0x7f66001a2d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:27:57.871 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:57.870+0000 7f6608200640 1 --2- 172.21.15.47:0/1320937196 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f65e80a3110 0x7f65e80a55d0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:27:57.871 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:57.870+0000 7f6608200640 1 --2- 172.21.15.47:0/1320937196 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f66000764f0 0x7f660019e9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:27:57.871 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:57.870+0000 7f6608200640 1 --2- 172.21.15.47:0/1320937196 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6600075bd0 0x7f660019e480 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:27:57.872 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:57.870+0000 7f6608200640 1 -- 172.21.15.47:0/1320937196 >> 172.21.15.47:0/1320937196 conn(0x7f66000ff310 msgr2=0x7f6600113c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:27:57.872 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:57.871+0000 7f6608200640 1 -- 172.21.15.47:0/1320937196 shutdown_connections 2024-03-21T10:27:57.872 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:57.871+0000 7f6608200640 1 -- 172.21.15.47:0/1320937196 wait complete. 2024-03-21T10:27:57.903 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-21T10:27:57.903 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-03-21T10:27:57.979 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:27:57 smithi100 ceph-mon[35193]: pgmap v82: 129 pgs: 129 active+clean; 273 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 10 MiB/s wr, 2.26k op/s 2024-03-21T10:27:57.979 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:27:57 smithi100 ceph-mon[35193]: from='client.? 172.21.15.47:0/2430338055' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-21T10:27:57.988 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:27:57 smithi157 ceph-mon[35581]: pgmap v82: 129 pgs: 129 active+clean; 273 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 10 MiB/s wr, 2.26k op/s 2024-03-21T10:27:57.988 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:27:57 smithi157 ceph-mon[35581]: from='client.? 172.21.15.47:0/2430338055' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-21T10:27:58.196 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:58.195+0000 7f6f0a400640 1 -- 172.21.15.47:0/2818939956 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6f04107640 msgr2=0x7f6f04107a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:27:58.196 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:58.195+0000 7f6f0a400640 1 --2- 172.21.15.47:0/2818939956 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6f04107640 0x7f6f04107a20 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f6ef4009930 tx=0x7f6ef402bc30 comp rx=0 tx=0).stop 2024-03-21T10:27:58.196 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:58.195+0000 7f6f0a400640 1 -- 172.21.15.47:0/2818939956 shutdown_connections 2024-03-21T10:27:58.196 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:58.195+0000 7f6f0a400640 1 --2- 172.21.15.47:0/2818939956 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6f04101ff0 0x7f6f0410e500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:27:58.196 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:58.195+0000 7f6f0a400640 1 --2- 172.21.15.47:0/2818939956 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6f04101630 0x7f6f04101ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:27:58.197 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:58.195+0000 7f6f0a400640 1 --2- 172.21.15.47:0/2818939956 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6f04107640 0x7f6f04107a20 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:27:58.197 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:58.195+0000 7f6f0a400640 1 -- 172.21.15.47:0/2818939956 >> 172.21.15.47:0/2818939956 conn(0x7f6f040fd100 msgr2=0x7f6f040ff520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:27:58.197 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:58.195+0000 7f6f0a400640 1 -- 172.21.15.47:0/2818939956 shutdown_connections 2024-03-21T10:27:58.197 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:58.196+0000 7f6f0a400640 1 -- 172.21.15.47:0/2818939956 wait complete. 2024-03-21T10:27:58.197 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:58.197+0000 7f6f0a400640 1 Processor -- start 2024-03-21T10:27:58.197 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:58.197+0000 7f6f0a400640 1 -- start start 2024-03-21T10:27:58.198 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:58.198+0000 7f6f0a400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6f04101630 0x7f6f04199f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:27:58.198 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:58.198+0000 7f6f0a400640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6f04101ff0 0x7f6f0419a490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:27:58.199 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:58.198+0000 7f6f0a400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6f04107640 0x7f6f0419e810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:27:58.199 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:58.198+0000 7f6f03400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6f04101630 0x7f6f04199f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:27:58.199 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:58.198+0000 7f6f0a400640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f04113b90 con 0x7f6f04107640 2024-03-21T10:27:58.199 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:58.198+0000 7f6f0a400640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f04113a10 con 0x7f6f04101630 2024-03-21T10:27:58.199 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:58.198+0000 7f6f0a400640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f04113d10 con 0x7f6f04101ff0 2024-03-21T10:27:58.200 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:58.198+0000 7f6f03400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6f04101630 0x7f6f04199f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:32794/0 (socket says 172.21.15.47:32794) 2024-03-21T10:27:58.200 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:58.198+0000 7f6f03400640 1 -- 172.21.15.47:0/789743469 learned_addr learned my addr 172.21.15.47:0/789743469 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:27:58.200 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:58.198+0000 7f6f03e00640 1 --2- 172.21.15.47:0/789743469 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6f04107640 0x7f6f0419e810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:27:58.200 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:58.199+0000 7f6f02a00640 1 --2- 172.21.15.47:0/789743469 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6f04101ff0 0x7f6f0419a490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:27:58.200 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:58.199+0000 7f6f03400640 1 -- 172.21.15.47:0/789743469 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6f04101ff0 msgr2=0x7f6f0419a490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:27:58.201 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:58.199+0000 7f6f03400640 1 --2- 172.21.15.47:0/789743469 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6f04101ff0 0x7f6f0419a490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:27:58.201 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:58.199+0000 7f6f03400640 1 -- 172.21.15.47:0/789743469 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6f04107640 msgr2=0x7f6f0419e810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:27:58.201 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:58.199+0000 7f6f03400640 1 --2- 172.21.15.47:0/789743469 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6f04107640 0x7f6f0419e810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:27:58.201 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:58.199+0000 7f6f03400640 1 -- 172.21.15.47:0/789743469 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f0419ef40 con 0x7f6f04101630 2024-03-21T10:27:58.201 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:58.199+0000 7f6f03e00640 1 --2- 172.21.15.47:0/789743469 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6f04107640 0x7f6f0419e810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T10:27:58.201 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:58.200+0000 7f6f03400640 1 --2- 172.21.15.47:0/789743469 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6f04101630 0x7f6f04199f50 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f6ef402c170 tx=0x7f6ef402ca90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:27:58.203 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:58.201+0000 7f6efbe00640 1 -- 172.21.15.47:0/789743469 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ef4042020 con 0x7f6f04101630 2024-03-21T10:27:58.204 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:58.201+0000 7f6efbe00640 1 -- 172.21.15.47:0/789743469 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f6ef402cc80 con 0x7f6f04101630 2024-03-21T10:27:58.204 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:58.201+0000 7f6efbe00640 1 -- 172.21.15.47:0/789743469 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ef4003d40 con 0x7f6f04101630 2024-03-21T10:27:58.204 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:58.202+0000 7f6f0a400640 1 -- 172.21.15.47:0/789743469 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6f0419f1d0 con 0x7f6f04101630 2024-03-21T10:27:58.204 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:58.202+0000 7f6f0a400640 1 -- 172.21.15.47:0/789743469 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6f041a6ac0 con 0x7f6f04101630 2024-03-21T10:27:58.208 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:58.207+0000 7f6f0d220740 1 Processor -- start 2024-03-21T10:27:58.209 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:58.209+0000 7f6efbe00640 1 -- 172.21.15.47:0/789743469 <== mon.2 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 27) v1 ==== 137985+0+0 (secure 0 0 0) 0x7f6ef4045020 con 0x7f6f04101630 2024-03-21T10:27:58.210 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:58.209+0000 7f6f0d220740 1 -- start start 2024-03-21T10:27:58.212 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:58.211+0000 7f6f0d220740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b4142bcee0 0x55b4142bd2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:27:58.213 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:58.211+0000 7f6f0d220740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b4142bd890 0x55b4142c5e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:27:58.214 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:58.211+0000 7f6f0d220740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b4142c6410 0x55b4142c8860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:27:58.214 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:58.211+0000 7f6f0d220740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b4142182c0 con 0x55b4142bcee0 2024-03-21T10:27:58.214 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:58.211+0000 7f6f0d220740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b4141c0c90 con 0x55b4142bd890 2024-03-21T10:27:58.214 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:58.211+0000 7f6f0d220740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b414214360 con 0x55b4142c6410 2024-03-21T10:27:58.214 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:58.212+0000 7f6f03400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b4142bd890 0x55b4142c5e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:27:58.214 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:58.213+0000 7f6f03400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b4142bd890 0x55b4142c5e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:32798/0 (socket says 172.21.15.47:32798) 2024-03-21T10:27:58.214 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:58.213+0000 7f6f03400640 1 -- 172.21.15.47:0/3286588975 learned_addr learned my addr 172.21.15.47:0/3286588975 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:27:58.214 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:58.213+0000 7f6f02a00640 1 --2- 172.21.15.47:0/3286588975 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b4142c6410 0x55b4142c8860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:27:58.214 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:58.213+0000 7f6efbe00640 1 --2- 172.21.15.47:0/789743469 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f6ee80a3110 0x7f6ee80a55d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:27:58.215 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:58.213+0000 7f6efbe00640 1 -- 172.21.15.47:0/789743469 <== mon.2 v2:172.21.15.100:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 9474+0+0 (secure 0 0 0) 0x7f6ef40f1a10 con 0x7f6f04101630 2024-03-21T10:27:58.215 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:58.214+0000 7f6f03400640 1 -- 172.21.15.47:0/3286588975 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b4142c6410 msgr2=0x55b4142c8860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:27:58.215 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:58.214+0000 7f6f03400640 1 --2- 172.21.15.47:0/3286588975 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b4142c6410 0x55b4142c8860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:27:58.215 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:58.214+0000 7f6f03400640 1 -- 172.21.15.47:0/3286588975 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b4142bcee0 msgr2=0x55b4142bd2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:27:58.215 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:58.214+0000 7f6f03400640 1 --2- 172.21.15.47:0/3286588975 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b4142bcee0 0x55b4142bd2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:27:58.215 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:58.214+0000 7f6f03400640 1 -- 172.21.15.47:0/3286588975 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b4141e5080 con 0x55b4142bd890 2024-03-21T10:27:58.215 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:58.214+0000 7f6f03e00640 1 --2- 172.21.15.47:0/789743469 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f6ee80a3110 0x7f6ee80a55d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:27:58.215 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:58.215+0000 7f6f03400640 1 --2- 172.21.15.47:0/3286588975 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b4142bd890 0x55b4142c5e40 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f6ef40eee20 tx=0x7f6ef4005fd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4bfc1a20783f836f server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:27:58.217 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:58.216+0000 7f6ef8c00640 1 -- 172.21.15.47:0/3286588975 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ef40fa070 con 0x55b4142bd890 2024-03-21T10:27:58.217 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:58.216+0000 7f6ef8c00640 1 -- 172.21.15.47:0/3286588975 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f6ef40fb040 con 0x55b4142bd890 2024-03-21T10:27:58.217 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:58.216+0000 7f6ef8c00640 1 -- 172.21.15.47:0/3286588975 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ef40bdb50 con 0x55b4142bd890 2024-03-21T10:27:58.217 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:58.216+0000 7f6f03e00640 1 --2- 172.21.15.47:0/789743469 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f6ee80a3110 0x7f6ee80a55d0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f6efc00b8f0 tx=0x7f6efc009d20 comp rx=0 tx=0).ready entity=mgr.15687 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:27:58.219 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:58.218+0000 7f6f0d220740 1 -- 172.21.15.47:0/3286588975 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b4142bd890 msgr2=0x55b4142c5e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:27:58.219 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:58.218+0000 7f6f0d220740 1 --2- 172.21.15.47:0/3286588975 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b4142bd890 0x55b4142c5e40 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f6ef40eee20 tx=0x7f6ef4005fd0 comp rx=0 tx=0).stop 2024-03-21T10:27:58.220 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:58.219+0000 7f6f0d220740 1 -- 172.21.15.47:0/3286588975 shutdown_connections 2024-03-21T10:27:58.220 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:58.219+0000 7f6f0d220740 1 --2- 172.21.15.47:0/3286588975 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b4142c6410 0x55b4142c8860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:27:58.220 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:58.219+0000 7f6f0d220740 1 --2- 172.21.15.47:0/3286588975 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b4142bd890 0x55b4142c5e40 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:27:58.220 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:58.219+0000 7f6f0d220740 1 --2- 172.21.15.47:0/3286588975 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b4142bcee0 0x55b4142bd2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:27:58.220 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:58.219+0000 7f6f0d220740 1 -- 172.21.15.47:0/3286588975 >> 172.21.15.47:0/3286588975 conn(0x55b4142b75b0 msgr2=0x55b4142b7990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:27:58.220 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:58.220+0000 7f6f0d220740 1 -- 172.21.15.47:0/3286588975 shutdown_connections 2024-03-21T10:27:58.220 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:58.220+0000 7f6f0d220740 1 -- 172.21.15.47:0/3286588975 wait complete. 2024-03-21T10:27:58.221 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:58.220+0000 7f6f0d220740 1 -- start start 2024-03-21T10:27:58.221 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:58.221+0000 7f6f0d220740 1 Processor -- start 2024-03-21T10:27:58.223 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:58.222+0000 7f6f0d220740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b4142bcee0 0x55b4142c5d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:27:58.223 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:58.222+0000 7f6f0d220740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b4142bd890 0x55b4142c30d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:27:58.223 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:58.222+0000 7f6f0d220740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b4142c6410 0x55b4142c36e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:27:58.223 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:58.222+0000 7f6f0d220740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b4141c0c90 con 0x55b4142bd890 2024-03-21T10:27:58.224 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:58.222+0000 7f6f0d220740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b4141d40d0 con 0x55b4142bcee0 2024-03-21T10:27:58.224 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:58.222+0000 7f6f0d220740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b4142c9140 con 0x55b4142c6410 2024-03-21T10:27:58.224 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:58.222+0000 7f6f03400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b4142bcee0 0x55b4142c5d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:27:58.224 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:58.222+0000 7f6f03400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b4142bcee0 0x55b4142c5d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:32802/0 (socket says 172.21.15.47:32802) 2024-03-21T10:27:58.224 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:58.222+0000 7f6f03400640 1 -- 172.21.15.47:0/1715934986 learned_addr learned my addr 172.21.15.47:0/1715934986 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:27:58.225 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:58.223+0000 7f6f03e00640 1 --2- 172.21.15.47:0/1715934986 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b4142c6410 0x55b4142c36e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:27:58.225 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:58.223+0000 7f6f0d220740 10 client.0 _ll_register_callbacks cb 0x55b4141d8e80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T10:27:58.225 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:58.223+0000 7f6f02a00640 1 --2- 172.21.15.47:0/1715934986 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b4142bd890 0x55b4142c30d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:27:58.225 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:58.223+0000 7f6f03400640 1 -- 172.21.15.47:0/1715934986 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b4142c6410 msgr2=0x55b4142c36e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:27:58.225 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:58.223+0000 7f6f03400640 1 --2- 172.21.15.47:0/1715934986 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b4142c6410 0x55b4142c36e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:27:58.225 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:58.223+0000 7f6f03400640 1 -- 172.21.15.47:0/1715934986 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b4142bd890 msgr2=0x55b4142c30d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:27:58.225 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:58.223+0000 7f6f03400640 1 --2- 172.21.15.47:0/1715934986 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b4142bd890 0x55b4142c30d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:27:58.225 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:58.223+0000 7f6f03400640 1 -- 172.21.15.47:0/1715934986 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55b413f2c3f0 con 0x55b4142bcee0 2024-03-21T10:27:58.225 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:58.223+0000 7f6f02a00640 1 --2- 172.21.15.47:0/1715934986 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b4142bd890 0x55b4142c30d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T10:27:58.225 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:58.225+0000 7f6f03400640 1 --2- 172.21.15.47:0/1715934986 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b4142bcee0 0x55b4142c5d40 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f6ef40eef60 tx=0x7f6ef40eee90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bea8035716498e8d server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:27:58.226 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:58.225+0000 7f6eed600640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-21T10:27:58.226 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:58.225+0000 7f6eed600640 1 -- 172.21.15.47:0/1715934986 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ef40fa070 con 0x55b4142bcee0 2024-03-21T10:27:58.226 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:58.225+0000 7f6eed600640 1 -- 172.21.15.47:0/1715934986 <== mon.2 v2:172.21.15.100:3300/0 2 ==== osd_map(97..97 src has 1..97) v4 ==== 9474+0+0 (secure 0 0 0) 0x7f6ef40ba1c0 con 0x55b4142bcee0 2024-03-21T10:27:58.226 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:58.225+0000 7f6f0d220740 1 -- 172.21.15.47:0/1715934986 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f6ef40fb040 con 0x55b4142bcee0 2024-03-21T10:27:58.226 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:58.226+0000 7f6eed600640 1 -- 172.21.15.47:0/1715934986 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6ef40bd4b0 con 0x55b4142bcee0 2024-03-21T10:27:58.227 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:58.226+0000 7f6f0d220740 10 client.36410 fetch_fsmap learned FSMap version 32 2024-03-21T10:27:58.227 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:58.226+0000 7f6f0d220740 1 -- 172.21.15.47:0/1715934986 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x55b413ff67d0 con 0x55b4142bcee0 2024-03-21T10:27:58.227 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:58.226+0000 7f6eed600640 1 -- 172.21.15.47:0/1715934986 <== mon.2 v2:172.21.15.100:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f6ef40fb040 con 0x55b4142bcee0 2024-03-21T10:27:58.228 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:58.227+0000 7f6eed600640 1 -- 172.21.15.47:0/1715934986 <== mon.2 v2:172.21.15.100:3300/0 5 ==== fsmap(e 32) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f6ef404f060 con 0x55b4142bcee0 2024-03-21T10:27:58.228 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:58.227+0000 7f6f0d220740 10 client.36410 fetch_fsmap finished waiting for FSMap version 32 2024-03-21T10:27:58.228 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:58.227+0000 7f6f0d220740 10 client.36410 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.e 2024-03-21T10:27:58.228 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:58.227+0000 7f6f0d220740 20 client.36410 populate_metadata read hostname 'smithi047' 2024-03-21T10:27:58.228 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:58.227+0000 7f6f0d220740 1 --2- 172.21.15.47:0/1715934986 >> [v2:172.21.15.100:6834/2597849957,v1:172.21.15.100:6835/2597849957] conn(0x55b4142d5cb0 0x55b4142d80a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:27:58.228 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:58.227+0000 7f6f0d220740 4 client.36410 mds_command: new command op to 34454 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T10:27:58.228 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:58.227+0000 7f6f0d220740 1 -- 172.21.15.47:0/1715934986 --> [v2:172.21.15.100:6834/2597849957,v1:172.21.15.100:6835/2597849957] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55b414225800 con 0x55b4142d5cb0 2024-03-21T10:27:58.230 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:58.228+0000 7f6f02a00640 1 --2- 172.21.15.47:0/1715934986 >> [v2:172.21.15.100:6834/2597849957,v1:172.21.15.100:6835/2597849957] conn(0x55b4142d5cb0 0x55b4142d80a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:27:58.230 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:58.229+0000 7f6eed600640 10 client.36410 ms_handle_connect on v2:172.21.15.100:6834/2597849957 2024-03-21T10:27:58.230 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:58.229+0000 7f6f02a00640 1 --2- 172.21.15.47:0/1715934986 >> [v2:172.21.15.100:6834/2597849957,v1:172.21.15.100:6835/2597849957] conn(0x55b4142d5cb0 0x55b4142d80a0 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=d51e154fff9d8438 server_cookie=9162a3a7004423f2 in_seq=0 out_seq=0 2024-03-21T10:27:58.232 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:58.231+0000 7f6eed600640 1 -- 172.21.15.47:0/1715934986 <== mds.2 v2:172.21.15.100:6834/2597849957 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55b414225800 con 0x55b4142d5cb0 2024-03-21T10:27:58.232 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:58.231+0000 7f6eed600640 10 client.36410 handle_command_reply: tid=0 2024-03-21T10:27:58.232 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:58.231+0000 7f6f0d220740 1 client.36410 shutdown 2024-03-21T10:27:58.233 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:58.232+0000 7f6f0d220740 1 -- 172.21.15.47:0/1715934986 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b4142bcee0 msgr2=0x55b4142c5d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:27:58.233 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:58.232+0000 7f6f0d220740 1 --2- 172.21.15.47:0/1715934986 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b4142bcee0 0x55b4142c5d40 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f6ef40eef60 tx=0x7f6ef40eee90 comp rx=0 tx=0).stop 2024-03-21T10:27:58.234 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:58.232+0000 7f6f0d220740 1 -- 172.21.15.47:0/1715934986 shutdown_connections 2024-03-21T10:27:58.234 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:58.232+0000 7f6f0d220740 1 --2- 172.21.15.47:0/1715934986 >> [v2:172.21.15.100:6834/2597849957,v1:172.21.15.100:6835/2597849957] conn(0x55b4142d5cb0 0x55b4142d80a0 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:27:58.234 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:58.232+0000 7f6f0d220740 1 --2- 172.21.15.47:0/1715934986 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b4142c6410 0x55b4142c36e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:27:58.234 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:58.232+0000 7f6f0d220740 1 --2- 172.21.15.47:0/1715934986 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b4142bd890 0x55b4142c30d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:27:58.234 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:58.232+0000 7f6f0d220740 1 --2- 172.21.15.47:0/1715934986 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b4142bcee0 0x55b4142c5d40 secure :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f6ef40eef60 tx=0x7f6ef40eee90 comp rx=0 tx=0).stop 2024-03-21T10:27:58.234 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:58.232+0000 7f6f0d220740 1 -- 172.21.15.47:0/1715934986 >> 172.21.15.47:0/1715934986 conn(0x55b4142b75b0 msgr2=0x55b4142b9300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:27:58.234 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:58.232+0000 7f6eed600640 0 client.36410 ms_handle_reset on v2:172.21.15.100:6834/2597849957 2024-03-21T10:27:58.234 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:58.233+0000 7f6f0d220740 1 -- 172.21.15.47:0/1715934986 shutdown_connections 2024-03-21T10:27:58.234 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:58.233+0000 7f6f0d220740 1 -- 172.21.15.47:0/1715934986 wait complete. 2024-03-21T10:27:58.234 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:58.233+0000 7f6f0d220740 20 client.36410 trim_cache size 0 max 16384 2024-03-21T10:27:58.260 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:58.259+0000 7f6f0d220740 1 Processor -- start 2024-03-21T10:27:58.260 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:58.259+0000 7f6f0d220740 1 -- start start 2024-03-21T10:27:58.260 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:58.260+0000 7f6f0d220740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b4142bd890 0x55b4142d2250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:27:58.260 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:58.260+0000 7f6f0d220740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b4142c6410 0x55b4142c5800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:27:58.260 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:58.260+0000 7f6f0d220740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b4142c24d0 0x55b4142c5d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:27:58.261 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:58.260+0000 7f6f0d220740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f04070900 con 0x55b4142c24d0 2024-03-21T10:27:58.261 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:58.260+0000 7f6f0d220740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f04072280 con 0x55b4142bd890 2024-03-21T10:27:58.261 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:58.260+0000 7f6f0d220740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f04070f60 con 0x55b4142c6410 2024-03-21T10:27:58.261 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:58.260+0000 7f6f02a00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b4142c6410 0x55b4142c5800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:27:58.262 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:58.260+0000 7f6f03e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b4142c24d0 0x55b4142c5d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:27:58.262 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:58.260+0000 7f6f03400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b4142bd890 0x55b4142d2250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:27:58.262 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:58.260+0000 7f6f03400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b4142bd890 0x55b4142d2250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:32808/0 (socket says 172.21.15.47:32808) 2024-03-21T10:27:58.262 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:58.261+0000 7f6f03400640 1 -- 172.21.15.47:0/3640937974 learned_addr learned my addr 172.21.15.47:0/3640937974 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:27:58.263 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:58.261+0000 7f6f02a00640 1 -- 172.21.15.47:0/3640937974 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b4142bd890 msgr2=0x55b4142d2250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:27:58.263 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:58.262+0000 7f6f02a00640 1 --2- 172.21.15.47:0/3640937974 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b4142bd890 0x55b4142d2250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:27:58.263 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:58.262+0000 7f6f02a00640 1 -- 172.21.15.47:0/3640937974 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b4142c24d0 msgr2=0x55b4142c5d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:27:58.263 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:58.262+0000 7f6f02a00640 1 --2- 172.21.15.47:0/3640937974 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b4142c24d0 0x55b4142c5d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:27:58.263 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:58.262+0000 7f6f02a00640 1 -- 172.21.15.47:0/3640937974 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b4142c1680 con 0x55b4142c6410 2024-03-21T10:27:58.263 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:58.263+0000 7f6f03400640 1 --2- 172.21.15.47:0/3640937974 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b4142bd890 0x55b4142d2250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T10:27:58.264 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:58.263+0000 7f6f03e00640 1 --2- 172.21.15.47:0/3640937974 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b4142c24d0 0x55b4142c5d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T10:27:58.264 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:58.263+0000 7f6f02a00640 1 --2- 172.21.15.47:0/3640937974 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b4142c6410 0x55b4142c5800 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x55b413ff6910 tx=0x7f6ef001eac0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c708cc1588fd5b5d server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:27:58.265 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:58.264+0000 7f6eefe00640 1 -- 172.21.15.47:0/3640937974 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ef0004050 con 0x55b4142c6410 2024-03-21T10:27:58.265 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:58.264+0000 7f6eefe00640 1 -- 172.21.15.47:0/3640937974 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f6ef00041f0 con 0x55b4142c6410 2024-03-21T10:27:58.265 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:58.264+0000 7f6eefe00640 1 -- 172.21.15.47:0/3640937974 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ef0012450 con 0x55b4142c6410 2024-03-21T10:27:58.266 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:58.265+0000 7f6f0d220740 1 -- 172.21.15.47:0/3640937974 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b4142c6410 msgr2=0x55b4142c5800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:27:58.266 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:58.265+0000 7f6f0d220740 1 --2- 172.21.15.47:0/3640937974 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b4142c6410 0x55b4142c5800 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x55b413ff6910 tx=0x7f6ef001eac0 comp rx=0 tx=0).stop 2024-03-21T10:27:58.267 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:58.265+0000 7f6f0d220740 1 -- 172.21.15.47:0/3640937974 shutdown_connections 2024-03-21T10:27:58.267 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:58.266+0000 7f6f0d220740 1 --2- 172.21.15.47:0/3640937974 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b4142c24d0 0x55b4142c5d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:27:58.267 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:58.266+0000 7f6f0d220740 1 --2- 172.21.15.47:0/3640937974 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b4142c6410 0x55b4142c5800 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:27:58.267 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:58.266+0000 7f6f0d220740 1 --2- 172.21.15.47:0/3640937974 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b4142bd890 0x55b4142d2250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:27:58.267 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:58.266+0000 7f6f0d220740 1 -- 172.21.15.47:0/3640937974 >> 172.21.15.47:0/3640937974 conn(0x55b4142b75b0 msgr2=0x55b4142adeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:27:58.267 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:58.266+0000 7f6f0d220740 1 -- 172.21.15.47:0/3640937974 shutdown_connections 2024-03-21T10:27:58.267 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:58.266+0000 7f6f0d220740 1 -- 172.21.15.47:0/3640937974 wait complete. 2024-03-21T10:27:58.268 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:58.267+0000 7f6f0d220740 1 -- start start 2024-03-21T10:27:58.268 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:58.267+0000 7f6f0d220740 1 Processor -- start 2024-03-21T10:27:58.269 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:58.269+0000 7f6f0d220740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b4142bd890 0x55b4142da910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:27:58.270 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:58.269+0000 7f6f0d220740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b4142c24d0 0x55b4142d8cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:27:58.270 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:58.270+0000 7f6f0d220740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b4142c6410 0x55b4142d91f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:27:58.270 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:58.270+0000 7f6f0d220740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f04071c20 con 0x55b4142bd890 2024-03-21T10:27:58.270 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:58.270+0000 7f6f0d220740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f040715c0 con 0x55b4142c6410 2024-03-21T10:27:58.270 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:58.270+0000 7f6f0d220740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f0406eae0 con 0x55b4142c24d0 2024-03-21T10:27:58.271 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:58.270+0000 7f6f0d220740 10 client.0 _ll_register_callbacks cb 0x7f6ef4005b50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T10:27:58.271 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:58.270+0000 7f6f03400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b4142bd890 0x55b4142da910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:27:58.271 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:58.270+0000 7f6f03e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b4142c6410 0x55b4142d91f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:27:58.271 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:58.271+0000 7f6f03400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b4142bd890 0x55b4142da910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:57740/0 (socket says 172.21.15.47:57740) 2024-03-21T10:27:58.272 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:58.271+0000 7f6f03400640 1 -- 172.21.15.47:0/3585407662 learned_addr learned my addr 172.21.15.47:0/3585407662 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:27:58.272 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:58.271+0000 7f6f02a00640 1 --2- 172.21.15.47:0/3585407662 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b4142c24d0 0x55b4142d8cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:27:58.272 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:58.271+0000 7f6f02a00640 1 -- 172.21.15.47:0/3585407662 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b4142c6410 msgr2=0x55b4142d91f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:27:58.272 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:58.271+0000 7f6f02a00640 1 --2- 172.21.15.47:0/3585407662 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b4142c6410 0x55b4142d91f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:27:58.272 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:58.271+0000 7f6f02a00640 1 -- 172.21.15.47:0/3585407662 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b4142bd890 msgr2=0x55b4142da910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:27:58.273 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:58.272+0000 7f6f02a00640 1 --2- 172.21.15.47:0/3585407662 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b4142bd890 0x55b4142da910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:27:58.273 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:58.272+0000 7f6f02a00640 1 -- 172.21.15.47:0/3585407662 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55b4142d98b0 con 0x55b4142c24d0 2024-03-21T10:27:58.273 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:58.272+0000 7f6f03e00640 1 --2- 172.21.15.47:0/3585407662 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b4142c6410 0x55b4142d91f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T10:27:58.273 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:58.272+0000 7f6f03400640 1 --2- 172.21.15.47:0/3585407662 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b4142bd890 0x55b4142da910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T10:27:58.273 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:58.272+0000 7f6f02a00640 1 --2- 172.21.15.47:0/3585407662 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b4142c24d0 0x55b4142d8cb0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x55b4142c4750 tx=0x7f6ef0004160 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8d5905ab69a4c29b server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:27:58.273 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:58.272+0000 7f6eed600640 10 client.0 ms_handle_connect on v2:172.21.15.157:3300/0 2024-03-21T10:27:58.275 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:58.274+0000 7f6eed600640 1 -- 172.21.15.47:0/3585407662 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ef00037f0 con 0x55b4142c24d0 2024-03-21T10:27:58.275 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:58.274+0000 7f6eed600640 1 -- 172.21.15.47:0/3585407662 <== mon.1 v2:172.21.15.157:3300/0 2 ==== osd_map(97..97 src has 1..97) v4 ==== 9474+0+0 (secure 0 0 0) 0x7f6ef0003990 con 0x55b4142c24d0 2024-03-21T10:27:58.275 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:58.274+0000 7f6f0d220740 1 -- 172.21.15.47:0/3585407662 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f6ef00041f0 con 0x55b4142c24d0 2024-03-21T10:27:58.275 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:58.275+0000 7f6eed600640 1 -- 172.21.15.47:0/3585407662 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6ef0037050 con 0x55b4142c24d0 2024-03-21T10:27:58.276 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:58.275+0000 7f6f0d220740 10 client.26176 fetch_fsmap learned FSMap version 32 2024-03-21T10:27:58.276 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:58.275+0000 7f6f0d220740 1 -- 172.21.15.47:0/3585407662 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x55b4143320e0 con 0x55b4142c24d0 2024-03-21T10:27:58.276 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:58.276+0000 7f6eed600640 1 -- 172.21.15.47:0/3585407662 <== mon.1 v2:172.21.15.157:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f6ef00041f0 con 0x55b4142c24d0 2024-03-21T10:27:58.277 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:58.276+0000 7f6eed600640 1 -- 172.21.15.47:0/3585407662 <== mon.1 v2:172.21.15.157:3300/0 5 ==== fsmap(e 32) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f6ef0012ae0 con 0x55b4142c24d0 2024-03-21T10:27:58.277 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:58.276+0000 7f6f0d220740 10 client.26176 fetch_fsmap finished waiting for FSMap version 32 2024-03-21T10:27:58.277 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:58.277+0000 7f6f0d220740 10 client.26176 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.e 2024-03-21T10:27:58.277 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:58.277+0000 7f6f0d220740 20 client.26176 populate_metadata read hostname 'smithi047' 2024-03-21T10:27:58.277 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:58.277+0000 7f6f0d220740 1 --2- 172.21.15.47:0/3585407662 >> [v2:172.21.15.100:6834/2597849957,v1:172.21.15.100:6835/2597849957] conn(0x55b414332770 0x55b414334b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:27:58.278 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:58.277+0000 7f6f0d220740 4 client.26176 mds_command: new command op to 34454 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-21T10:27:58.278 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:58.277+0000 7f6f0d220740 1 -- 172.21.15.47:0/3585407662 --> [v2:172.21.15.100:6834/2597849957,v1:172.21.15.100:6835/2597849957] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55b41421fb50 con 0x55b414332770 2024-03-21T10:27:58.278 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:58.277+0000 7f6f03400640 1 --2- 172.21.15.47:0/3585407662 >> [v2:172.21.15.100:6834/2597849957,v1:172.21.15.100:6835/2597849957] conn(0x55b414332770 0x55b414334b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:27:58.281 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:58.280+0000 7f6f03400640 1 --2- 172.21.15.47:0/3585407662 >> [v2:172.21.15.100:6834/2597849957,v1:172.21.15.100:6835/2597849957] conn(0x55b414332770 0x55b414334b90 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=60280f804b87781 server_cookie=1f4f639f2cedcc60 in_seq=0 out_seq=0 2024-03-21T10:27:58.281 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:58.280+0000 7f6eed600640 10 client.26176 ms_handle_connect on v2:172.21.15.100:6834/2597849957 2024-03-21T10:27:58.282 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:58.282+0000 7f6eed600640 1 -- 172.21.15.47:0/3585407662 <== mds.2 v2:172.21.15.100:6834/2597849957 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55b41421fb50 con 0x55b414332770 2024-03-21T10:27:58.282 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:58.282+0000 7f6eed600640 10 client.26176 handle_command_reply: tid=0 2024-03-21T10:27:58.283 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:58.282+0000 7f6f0d220740 1 client.26176 shutdown 2024-03-21T10:27:58.283 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:58.283+0000 7f6f0d220740 1 -- 172.21.15.47:0/3585407662 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b4142c24d0 msgr2=0x55b4142d8cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:27:58.283 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:58.283+0000 7f6f0d220740 1 --2- 172.21.15.47:0/3585407662 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b4142c24d0 0x55b4142d8cb0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x55b4142c4750 tx=0x7f6ef0004160 comp rx=0 tx=0).stop 2024-03-21T10:27:58.284 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:58.283+0000 7f6f0d220740 1 -- 172.21.15.47:0/3585407662 shutdown_connections 2024-03-21T10:27:58.284 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:58.283+0000 7f6f0d220740 1 --2- 172.21.15.47:0/3585407662 >> [v2:172.21.15.100:6834/2597849957,v1:172.21.15.100:6835/2597849957] conn(0x55b414332770 0x55b414334b90 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:27:58.284 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:58.284+0000 7f6eed600640 0 client.26176 ms_handle_reset on v2:172.21.15.100:6834/2597849957 2024-03-21T10:27:58.285 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:58.284+0000 7f6f0d220740 1 --2- 172.21.15.47:0/3585407662 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b4142c6410 0x55b4142d91f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:27:58.285 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:58.284+0000 7f6f0d220740 1 --2- 172.21.15.47:0/3585407662 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b4142c24d0 0x55b4142d8cb0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:27:58.285 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:58.284+0000 7f6f0d220740 1 --2- 172.21.15.47:0/3585407662 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b4142bd890 0x55b4142da910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:27:58.285 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:58.284+0000 7f6f0d220740 1 -- 172.21.15.47:0/3585407662 >> 172.21.15.47:0/3585407662 conn(0x55b4142b75b0 msgr2=0x55b4142b8980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:27:58.285 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:58.284+0000 7f6f0d220740 1 -- 172.21.15.47:0/3585407662 shutdown_connections 2024-03-21T10:27:58.286 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:58.285+0000 7f6f0d220740 1 -- 172.21.15.47:0/3585407662 wait complete. 2024-03-21T10:27:58.287 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:58.287+0000 7f6f0d220740 20 client.26176 trim_cache size 0 max 16384 2024-03-21T10:27:58.288 INFO:teuthology.orchestra.run.smithi047.stdout:[] 2024-03-21T10:27:58.289 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:58.288+0000 7f6f0a400640 1 -- 172.21.15.47:0/789743469 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f6ee80a3110 msgr2=0x7f6ee80a55d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:27:58.289 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:58.288+0000 7f6f0a400640 1 --2- 172.21.15.47:0/789743469 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f6ee80a3110 0x7f6ee80a55d0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f6efc00b8f0 tx=0x7f6efc009d20 comp rx=0 tx=0).stop 2024-03-21T10:27:58.289 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:58.288+0000 7f6f0a400640 1 -- 172.21.15.47:0/789743469 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6f04101630 msgr2=0x7f6f04199f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:27:58.289 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:58.288+0000 7f6f0a400640 1 --2- 172.21.15.47:0/789743469 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6f04101630 0x7f6f04199f50 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f6ef402c170 tx=0x7f6ef402ca90 comp rx=0 tx=0).stop 2024-03-21T10:27:58.290 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:58.289+0000 7f6f0a400640 1 -- 172.21.15.47:0/789743469 shutdown_connections 2024-03-21T10:27:58.290 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:58.289+0000 7f6f0a400640 1 --2- 172.21.15.47:0/789743469 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6f04107640 0x7f6f0419e810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:27:58.290 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:58.289+0000 7f6f0a400640 1 --2- 172.21.15.47:0/789743469 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6f04101ff0 0x7f6f0419a490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:27:58.290 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:58.289+0000 7f6f0a400640 1 --2- 172.21.15.47:0/789743469 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f6ee80a3110 0x7f6ee80a55d0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:27:58.290 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:58.289+0000 7f6f0a400640 1 --2- 172.21.15.47:0/789743469 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6f04101630 0x7f6f04199f50 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:27:58.290 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:58.289+0000 7f6f0a400640 1 -- 172.21.15.47:0/789743469 >> 172.21.15.47:0/789743469 conn(0x7f6f040fd100 msgr2=0x7f6f040fd5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:27:58.291 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:58.291+0000 7f6f0a400640 1 -- 172.21.15.47:0/789743469 shutdown_connections 2024-03-21T10:27:58.292 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:58.291+0000 7f6f0a400640 1 -- 172.21.15.47:0/789743469 wait complete. 2024-03-21T10:27:58.321 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-21T10:27:58.321 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-03-21T10:27:58.619 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:58.618+0000 7f9523000640 1 -- 172.21.15.47:0/1385423270 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f951409d230 msgr2=0x7f95140a4e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:27:58.620 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:58.618+0000 7f9523000640 1 --2- 172.21.15.47:0/1385423270 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f951409d230 0x7f95140a4e00 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f951c067370 tx=0x7f951c0972f0 comp rx=0 tx=0).stop 2024-03-21T10:27:58.620 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:58.619+0000 7f9523000640 1 -- 172.21.15.47:0/1385423270 shutdown_connections 2024-03-21T10:27:58.620 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:58.619+0000 7f9523000640 1 --2- 172.21.15.47:0/1385423270 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f951409d230 0x7f95140a4e00 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:27:58.620 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:58.619+0000 7f9523000640 1 --2- 172.21.15.47:0/1385423270 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9514098b10 0x7f951409cb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:27:58.620 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:58.619+0000 7f9523000640 1 --2- 172.21.15.47:0/1385423270 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9514098160 0x7f9514098540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:27:58.620 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:58.619+0000 7f9523000640 1 -- 172.21.15.47:0/1385423270 >> 172.21.15.47:0/1385423270 conn(0x7f95140939c0 msgr2=0x7f9514095de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:27:58.620 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:58.619+0000 7f9523000640 1 -- 172.21.15.47:0/1385423270 shutdown_connections 2024-03-21T10:27:58.620 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:58.619+0000 7f9523000640 1 -- 172.21.15.47:0/1385423270 wait complete. 2024-03-21T10:27:58.623 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:58.622+0000 7f9523000640 1 Processor -- start 2024-03-21T10:27:58.623 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:58.622+0000 7f9523000640 1 -- start start 2024-03-21T10:27:58.624 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:58.623+0000 7f9523000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9514098160 0x7f951412e6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:27:58.624 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:58.623+0000 7f9523000640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f951409d230 0x7f951412ec20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:27:58.624 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:58.623+0000 7f9523000640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9514132fa0 0x7f9514133450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:27:58.624 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:58.623+0000 7f9523000640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95140a80b0 con 0x7f9514132fa0 2024-03-21T10:27:58.624 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:58.623+0000 7f9523000640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95140a7f30 con 0x7f9514098160 2024-03-21T10:27:58.624 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:58.623+0000 7f9523000640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95140a8230 con 0x7f951409d230 2024-03-21T10:27:58.625 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:58.624+0000 7f9521200640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f951409d230 0x7f951412ec20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:27:58.625 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:58.624+0000 7f9521200640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f951409d230 0x7f951412ec20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.47:40264/0 (socket says 172.21.15.47:40264) 2024-03-21T10:27:58.625 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:58.624+0000 7f9521200640 1 -- 172.21.15.47:0/16815650 learned_addr learned my addr 172.21.15.47:0/16815650 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:27:58.625 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:58.624+0000 7f9522600640 1 --2- 172.21.15.47:0/16815650 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9514132fa0 0x7f9514133450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:27:58.626 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:58.625+0000 7f9521c00640 1 --2- 172.21.15.47:0/16815650 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9514098160 0x7f951412e6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:27:58.626 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:58.626+0000 7f9521200640 1 -- 172.21.15.47:0/16815650 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9514098160 msgr2=0x7f951412e6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:27:58.627 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:58.626+0000 7f9521200640 1 --2- 172.21.15.47:0/16815650 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9514098160 0x7f951412e6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:27:58.627 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:58.626+0000 7f9521200640 1 -- 172.21.15.47:0/16815650 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9514132fa0 msgr2=0x7f9514133450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:27:58.627 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:58.626+0000 7f9521200640 1 --2- 172.21.15.47:0/16815650 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9514132fa0 0x7f9514133450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:27:58.627 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:58.626+0000 7f9521200640 1 -- 172.21.15.47:0/16815650 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9514133a90 con 0x7f951409d230 2024-03-21T10:27:58.627 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:58.627+0000 7f9521200640 1 --2- 172.21.15.47:0/16815650 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f951409d230 0x7f951412ec20 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f950800e9c0 tx=0x7f950800ee80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:27:58.629 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:58.628+0000 7f951a000640 1 -- 172.21.15.47:0/16815650 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f950800cd80 con 0x7f951409d230 2024-03-21T10:27:58.629 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:58.628+0000 7f951a000640 1 -- 172.21.15.47:0/16815650 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f95080054d0 con 0x7f951409d230 2024-03-21T10:27:58.629 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:58.628+0000 7f951a000640 1 -- 172.21.15.47:0/16815650 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9508010430 con 0x7f951409d230 2024-03-21T10:27:58.629 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:58.628+0000 7f9523000640 1 -- 172.21.15.47:0/16815650 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9514133d80 con 0x7f951409d230 2024-03-21T10:27:58.629 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:58.628+0000 7f9523000640 1 -- 172.21.15.47:0/16815650 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f951413b520 con 0x7f951409d230 2024-03-21T10:27:58.637 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:58.636+0000 7f95273fb740 1 Processor -- start 2024-03-21T10:27:58.637 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:58.636+0000 7f95273fb740 1 -- start start 2024-03-21T10:27:58.638 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:58.637+0000 7f95273fb740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x560094aae8b0 0x560094aaec90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:27:58.640 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:58.637+0000 7f95273fb740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560094aaf260 0x560094ab7810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:27:58.640 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:58.637+0000 7f95273fb740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560094ab7de0 0x560094aba230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:27:58.640 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:58.637+0000 7f95273fb740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x560094a122c0 con 0x560094aae8b0 2024-03-21T10:27:58.640 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:58.637+0000 7f95273fb740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x5600949bac90 con 0x560094aaf260 2024-03-21T10:27:58.640 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:58.637+0000 7f95273fb740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x560094a0e360 con 0x560094ab7de0 2024-03-21T10:27:58.640 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:58.637+0000 7f951a000640 1 -- 172.21.15.47:0/16815650 <== mon.1 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 27) v1 ==== 137985+0+0 (secure 0 0 0) 0x7f9508004f90 con 0x7f951409d230 2024-03-21T10:27:58.640 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:58.638+0000 7f9521c00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560094aaf260 0x560094ab7810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:27:58.640 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:58.638+0000 7f9521200640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560094ab7de0 0x560094aba230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:27:58.640 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:58.638+0000 7f9521c00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560094aaf260 0x560094ab7810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:32834/0 (socket says 172.21.15.47:32834) 2024-03-21T10:27:58.640 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:58.638+0000 7f9521c00640 1 -- 172.21.15.47:0/4189919692 learned_addr learned my addr 172.21.15.47:0/4189919692 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:27:58.640 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:58.639+0000 7f9522600640 1 --2- 172.21.15.47:0/4189919692 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x560094aae8b0 0x560094aaec90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:27:58.641 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:58.640+0000 7f9521200640 1 -- 172.21.15.47:0/4189919692 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560094aaf260 msgr2=0x560094ab7810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:27:58.641 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:58.640+0000 7f9521200640 1 --2- 172.21.15.47:0/4189919692 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560094aaf260 0x560094ab7810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:27:58.641 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:58.640+0000 7f9521200640 1 -- 172.21.15.47:0/4189919692 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x560094aae8b0 msgr2=0x560094aaec90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:27:58.641 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:58.640+0000 7f9521200640 1 --2- 172.21.15.47:0/4189919692 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x560094aae8b0 0x560094aaec90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:27:58.641 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:58.640+0000 7f9521200640 1 -- 172.21.15.47:0/4189919692 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5600949df080 con 0x560094ab7de0 2024-03-21T10:27:58.642 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:58.640+0000 7f951a000640 1 --2- 172.21.15.47:0/16815650 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f95040a3260 0x7f95040a5720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:27:58.642 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:58.641+0000 7f9521200640 1 --2- 172.21.15.47:0/4189919692 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560094ab7de0 0x560094aba230 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f95080cd360 tx=0x7f95080d84b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3d1a240e8a0299aa server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:27:58.642 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:58.641+0000 7f951a000640 1 -- 172.21.15.47:0/16815650 <== mon.1 v2:172.21.15.157:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 9474+0+0 (secure 0 0 0) 0x7f95080d0030 con 0x7f951409d230 2024-03-21T10:27:58.642 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:58.642+0000 7f950ea00640 1 -- 172.21.15.47:0/4189919692 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95080982a0 con 0x560094ab7de0 2024-03-21T10:27:58.643 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:58.642+0000 7f9522600640 1 --2- 172.21.15.47:0/16815650 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f95040a3260 0x7f95040a5720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:27:58.643 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:58.642+0000 7f950ea00640 1 -- 172.21.15.47:0/4189919692 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f950809c700 con 0x560094ab7de0 2024-03-21T10:27:58.643 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:58.642+0000 7f950ea00640 1 -- 172.21.15.47:0/4189919692 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9508005280 con 0x560094ab7de0 2024-03-21T10:27:58.644 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:58.644+0000 7f95273fb740 1 -- 172.21.15.47:0/4189919692 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560094ab7de0 msgr2=0x560094aba230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:27:58.644 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:58.644+0000 7f95273fb740 1 --2- 172.21.15.47:0/4189919692 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560094ab7de0 0x560094aba230 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f95080cd360 tx=0x7f95080d84b0 comp rx=0 tx=0).stop 2024-03-21T10:27:58.645 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:58.644+0000 7f9522600640 1 --2- 172.21.15.47:0/16815650 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f95040a3260 0x7f95040a5720 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f951c0671e0 tx=0x7f951c0a1040 comp rx=0 tx=0).ready entity=mgr.15687 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:27:58.645 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:58.645+0000 7f95273fb740 1 -- 172.21.15.47:0/4189919692 shutdown_connections 2024-03-21T10:27:58.645 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:58.645+0000 7f95273fb740 1 --2- 172.21.15.47:0/4189919692 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560094ab7de0 0x560094aba230 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:27:58.646 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:58.645+0000 7f95273fb740 1 --2- 172.21.15.47:0/4189919692 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560094aaf260 0x560094ab7810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:27:58.646 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:58.645+0000 7f95273fb740 1 --2- 172.21.15.47:0/4189919692 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x560094aae8b0 0x560094aaec90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:27:58.646 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:58.645+0000 7f95273fb740 1 -- 172.21.15.47:0/4189919692 >> 172.21.15.47:0/4189919692 conn(0x560094aa8f80 msgr2=0x560094aa9360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:27:58.646 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:58.645+0000 7f95273fb740 1 -- 172.21.15.47:0/4189919692 shutdown_connections 2024-03-21T10:27:58.646 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:58.645+0000 7f95273fb740 1 -- 172.21.15.47:0/4189919692 wait complete. 2024-03-21T10:27:58.646 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:58.646+0000 7f95273fb740 1 -- start start 2024-03-21T10:27:58.647 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:58.646+0000 7f95273fb740 1 Processor -- start 2024-03-21T10:27:58.648 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:58.647+0000 7f95273fb740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x560094aae8b0 0x560094ab7760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:27:58.649 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:58.648+0000 7f95273fb740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560094aaf260 0x560094ab4af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:27:58.649 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:58.648+0000 7f95273fb740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560094ab7de0 0x560094ab5100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:27:58.649 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:58.648+0000 7f95273fb740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x560094abad50 con 0x560094aae8b0 2024-03-21T10:27:58.649 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:58.648+0000 7f95273fb740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x5600949ce0d0 con 0x560094ab7de0 2024-03-21T10:27:58.649 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:58.648+0000 7f95273fb740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x560094abaed0 con 0x560094aaf260 2024-03-21T10:27:58.649 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:58.648+0000 7f95273fb740 10 client.0 _ll_register_callbacks cb 0x5600949d2e80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T10:27:58.649 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:58.649+0000 7f9522600640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560094ab7de0 0x560094ab5100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:27:58.650 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:58.649+0000 7f9522600640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560094ab7de0 0x560094ab5100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:32850/0 (socket says 172.21.15.47:32850) 2024-03-21T10:27:58.650 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:58.649+0000 7f9522600640 1 -- 172.21.15.47:0/1193436405 learned_addr learned my addr 172.21.15.47:0/1193436405 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:27:58.650 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:58.649+0000 7f9521c00640 1 --2- 172.21.15.47:0/1193436405 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x560094aae8b0 0x560094ab7760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:27:58.650 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:58.649+0000 7f9521200640 1 --2- 172.21.15.47:0/1193436405 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560094aaf260 0x560094ab4af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:27:58.651 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:58.650+0000 7f9522600640 1 -- 172.21.15.47:0/1193436405 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560094aaf260 msgr2=0x560094ab4af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:27:58.651 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:58.650+0000 7f9522600640 1 --2- 172.21.15.47:0/1193436405 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560094aaf260 0x560094ab4af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:27:58.651 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:58.650+0000 7f9522600640 1 -- 172.21.15.47:0/1193436405 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x560094aae8b0 msgr2=0x560094ab7760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:27:58.651 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:58.650+0000 7f9522600640 1 --2- 172.21.15.47:0/1193436405 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x560094aae8b0 0x560094ab7760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:27:58.651 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:58.650+0000 7f9522600640 1 -- 172.21.15.47:0/1193436405 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5600947263f0 con 0x560094ab7de0 2024-03-21T10:27:58.652 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:58.650+0000 7f9521c00640 1 --2- 172.21.15.47:0/1193436405 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x560094aae8b0 0x560094ab7760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T10:27:58.652 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:58.651+0000 7f9522600640 1 --2- 172.21.15.47:0/1193436405 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560094ab7de0 0x560094ab5100 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f951c078090 tx=0x7f951c0972f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ad55b280752019a2 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:27:58.652 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:58.651+0000 7f9503400640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-21T10:27:58.653 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:58.652+0000 7f9503400640 1 -- 172.21.15.47:0/1193436405 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f951c0b9060 con 0x560094ab7de0 2024-03-21T10:27:58.653 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:58.652+0000 7f9503400640 1 -- 172.21.15.47:0/1193436405 <== mon.2 v2:172.21.15.100:3300/0 2 ==== osd_map(97..97 src has 1..97) v4 ==== 9474+0+0 (secure 0 0 0) 0x7f951c069390 con 0x560094ab7de0 2024-03-21T10:27:58.653 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:58.652+0000 7f95273fb740 1 -- 172.21.15.47:0/1193436405 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f950809c700 con 0x560094ab7de0 2024-03-21T10:27:58.654 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:58.653+0000 7f9503400640 1 -- 172.21.15.47:0/1193436405 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f951c0b1040 con 0x560094ab7de0 2024-03-21T10:27:58.654 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:58.653+0000 7f95273fb740 10 client.36428 fetch_fsmap learned FSMap version 32 2024-03-21T10:27:58.654 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:58.653+0000 7f95273fb740 1 -- 172.21.15.47:0/1193436405 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x5600947f07d0 con 0x560094ab7de0 2024-03-21T10:27:58.655 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:58.654+0000 7f9503400640 1 -- 172.21.15.47:0/1193436405 <== mon.2 v2:172.21.15.100:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f950809c700 con 0x560094ab7de0 2024-03-21T10:27:58.656 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:58.655+0000 7f9503400640 1 -- 172.21.15.47:0/1193436405 <== mon.2 v2:172.21.15.100:3300/0 5 ==== fsmap(e 32) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f951c0ab3f0 con 0x560094ab7de0 2024-03-21T10:27:58.656 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:58.655+0000 7f95273fb740 10 client.36428 fetch_fsmap finished waiting for FSMap version 32 2024-03-21T10:27:58.656 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:58.655+0000 7f95273fb740 10 client.36428 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-03-21T10:27:58.656 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:58.655+0000 7f95273fb740 20 client.36428 populate_metadata read hostname 'smithi047' 2024-03-21T10:27:58.656 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:58.655+0000 7f95273fb740 1 --2- 172.21.15.47:0/1193436405 >> [v2:172.21.15.100:6836/3497644162,v1:172.21.15.100:6837/3497644162] conn(0x560094ad0550 0x560094ad2940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:27:58.656 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:58.655+0000 7f95273fb740 4 client.36428 mds_command: new command op to 34493 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T10:27:58.656 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:58.655+0000 7f95273fb740 1 -- 172.21.15.47:0/1193436405 --> [v2:172.21.15.100:6836/3497644162,v1:172.21.15.100:6837/3497644162] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x560094a1f800 con 0x560094ad0550 2024-03-21T10:27:58.657 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:58.655+0000 7f9521c00640 1 --2- 172.21.15.47:0/1193436405 >> [v2:172.21.15.100:6836/3497644162,v1:172.21.15.100:6837/3497644162] conn(0x560094ad0550 0x560094ad2940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:27:58.659 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:58.658+0000 7f9521c00640 1 --2- 172.21.15.47:0/1193436405 >> [v2:172.21.15.100:6836/3497644162,v1:172.21.15.100:6837/3497644162] conn(0x560094ad0550 0x560094ad2940 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=aac7e486a7bb9ebf server_cookie=995b5f14161320a1 in_seq=0 out_seq=0 2024-03-21T10:27:58.659 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:58.658+0000 7f9503400640 10 client.36428 ms_handle_connect on v2:172.21.15.100:6836/3497644162 2024-03-21T10:27:58.660 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:58.660+0000 7f9503400640 1 -- 172.21.15.47:0/1193436405 <== mds.1 v2:172.21.15.100:6836/3497644162 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x560094a1f800 con 0x560094ad0550 2024-03-21T10:27:58.660 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:58.660+0000 7f9503400640 10 client.36428 handle_command_reply: tid=0 2024-03-21T10:27:58.661 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:58.660+0000 7f95273fb740 1 client.36428 shutdown 2024-03-21T10:27:58.662 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:58.661+0000 7f95273fb740 1 -- 172.21.15.47:0/1193436405 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560094ab7de0 msgr2=0x560094ab5100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:27:58.662 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:58.661+0000 7f95273fb740 1 --2- 172.21.15.47:0/1193436405 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560094ab7de0 0x560094ab5100 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f951c078090 tx=0x7f951c0972f0 comp rx=0 tx=0).stop 2024-03-21T10:27:58.663 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:58.662+0000 7f95273fb740 1 -- 172.21.15.47:0/1193436405 shutdown_connections 2024-03-21T10:27:58.663 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:58.662+0000 7f95273fb740 1 --2- 172.21.15.47:0/1193436405 >> [v2:172.21.15.100:6836/3497644162,v1:172.21.15.100:6837/3497644162] conn(0x560094ad0550 0x560094ad2940 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:27:58.663 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:58.662+0000 7f9503400640 0 client.36428 ms_handle_reset on v2:172.21.15.100:6836/3497644162 2024-03-21T10:27:58.663 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:58.662+0000 7f95273fb740 1 --2- 172.21.15.47:0/1193436405 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560094ab7de0 0x560094ab5100 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:27:58.663 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:58.662+0000 7f95273fb740 1 --2- 172.21.15.47:0/1193436405 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560094aaf260 0x560094ab4af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:27:58.663 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:58.662+0000 7f95273fb740 1 --2- 172.21.15.47:0/1193436405 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x560094aae8b0 0x560094ab7760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:27:58.663 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:58.662+0000 7f95273fb740 1 -- 172.21.15.47:0/1193436405 >> 172.21.15.47:0/1193436405 conn(0x560094aa8f80 msgr2=0x560094aaacd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:27:58.663 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:58.662+0000 7f95273fb740 1 -- 172.21.15.47:0/1193436405 shutdown_connections 2024-03-21T10:27:58.663 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:58.662+0000 7f95273fb740 1 -- 172.21.15.47:0/1193436405 wait complete. 2024-03-21T10:27:58.663 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:58.662+0000 7f95273fb740 20 client.36428 trim_cache size 0 max 16384 2024-03-21T10:27:58.688 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:58.686+0000 7f95273fb740 1 Processor -- start 2024-03-21T10:27:58.688 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:58.687+0000 7f95273fb740 1 -- start start 2024-03-21T10:27:58.688 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:58.687+0000 7f95273fb740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x560094aae8b0 0x560094ab7220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:27:58.688 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:58.687+0000 7f95273fb740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560094aaf260 0x560094ab7760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:27:58.688 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:58.687+0000 7f95273fb740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560094ab7de0 0x560094ac3640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:27:58.688 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:58.687+0000 7f95273fb740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9514140d70 con 0x560094aae8b0 2024-03-21T10:27:58.688 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:58.687+0000 7f95273fb740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95141426f0 con 0x560094ab7de0 2024-03-21T10:27:58.688 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:58.687+0000 7f95273fb740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95141413d0 con 0x560094aaf260 2024-03-21T10:27:58.689 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:58.687+0000 7f9521200640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560094aaf260 0x560094ab7760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:27:58.689 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:58.688+0000 7f9522600640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560094ab7de0 0x560094ac3640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:27:58.689 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:58.688+0000 7f9521200640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560094aaf260 0x560094ab7760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.47:40278/0 (socket says 172.21.15.47:40278) 2024-03-21T10:27:58.689 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:58.688+0000 7f9521200640 1 -- 172.21.15.47:0/1414163353 learned_addr learned my addr 172.21.15.47:0/1414163353 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:27:58.689 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:58.688+0000 7f9521c00640 1 --2- 172.21.15.47:0/1414163353 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x560094aae8b0 0x560094ab7220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:27:58.689 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:58.688+0000 7f9521200640 1 -- 172.21.15.47:0/1414163353 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560094ab7de0 msgr2=0x560094ac3640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:27:58.690 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:58.688+0000 7f9521200640 1 --2- 172.21.15.47:0/1414163353 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560094ab7de0 0x560094ac3640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:27:58.690 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:58.688+0000 7f9521200640 1 -- 172.21.15.47:0/1414163353 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x560094aae8b0 msgr2=0x560094ab7220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:27:58.690 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:58.688+0000 7f9521200640 1 --2- 172.21.15.47:0/1414163353 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x560094aae8b0 0x560094ab7220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:27:58.690 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:58.688+0000 7f9521200640 1 -- 172.21.15.47:0/1414163353 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560094aaf000 con 0x560094aaf260 2024-03-21T10:27:58.690 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:58.689+0000 7f9521200640 1 --2- 172.21.15.47:0/1414163353 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560094aaf260 0x560094ab7760 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f950800f010 tx=0x7f9508004840 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e3ae6ef21335a3a2 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:27:58.691 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:58.690+0000 7f950e000640 1 -- 172.21.15.47:0/1414163353 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9508067020 con 0x560094aaf260 2024-03-21T10:27:58.691 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:58.690+0000 7f950e000640 1 -- 172.21.15.47:0/1414163353 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f950809c700 con 0x560094aaf260 2024-03-21T10:27:58.691 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:58.690+0000 7f950e000640 1 -- 172.21.15.47:0/1414163353 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f950809b960 con 0x560094aaf260 2024-03-21T10:27:58.691 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:58.691+0000 7f95273fb740 1 -- 172.21.15.47:0/1414163353 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560094aaf260 msgr2=0x560094ab7760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:27:58.691 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:58.691+0000 7f95273fb740 1 --2- 172.21.15.47:0/1414163353 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560094aaf260 0x560094ab7760 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f950800f010 tx=0x7f9508004840 comp rx=0 tx=0).stop 2024-03-21T10:27:58.692 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:58.691+0000 7f95273fb740 1 -- 172.21.15.47:0/1414163353 shutdown_connections 2024-03-21T10:27:58.693 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:58.691+0000 7f95273fb740 1 --2- 172.21.15.47:0/1414163353 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560094ab7de0 0x560094ac3640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:27:58.693 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:58.691+0000 7f95273fb740 1 --2- 172.21.15.47:0/1414163353 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560094aaf260 0x560094ab7760 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:27:58.693 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:58.691+0000 7f95273fb740 1 --2- 172.21.15.47:0/1414163353 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x560094aae8b0 0x560094ab7220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:27:58.693 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:58.691+0000 7f95273fb740 1 -- 172.21.15.47:0/1414163353 >> 172.21.15.47:0/1414163353 conn(0x560094aa8f80 msgr2=0x560094aa8160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:27:58.693 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:58.692+0000 7f95273fb740 1 -- 172.21.15.47:0/1414163353 shutdown_connections 2024-03-21T10:27:58.693 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:58.692+0000 7f95273fb740 1 -- 172.21.15.47:0/1414163353 wait complete. 2024-03-21T10:27:58.693 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:58.693+0000 7f95273fb740 1 -- start start 2024-03-21T10:27:58.695 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:58.694+0000 7f95273fb740 1 Processor -- start 2024-03-21T10:27:58.697 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:58.695+0000 7f95273fb740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560094aae8b0 0x560094b25100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:27:58.697 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:58.695+0000 7f95273fb740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x560094aaf260 0x560094b25640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:27:58.697 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:58.695+0000 7f95273fb740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560094ab7de0 0x560094abbd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:27:58.697 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:58.696+0000 7f9521200640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x560094aaf260 0x560094b25640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:27:58.697 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:58.696+0000 7f9521200640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x560094aaf260 0x560094b25640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:57782/0 (socket says 172.21.15.47:57782) 2024-03-21T10:27:58.697 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:58.696+0000 7f9521c00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560094aae8b0 0x560094b25100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:27:58.697 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:58.696+0000 7f9521c00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560094aae8b0 0x560094b25100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.47:40282/0 (socket says 172.21.15.47:40282) 2024-03-21T10:27:58.698 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:58.697+0000 7f9521200640 1 -- 172.21.15.47:0/3678977813 learned_addr learned my addr 172.21.15.47:0/3678977813 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:27:58.698 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:58.697+0000 7f95273fb740 1 -- 172.21.15.47:0/3678977813 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9514142090 con 0x560094aaf260 2024-03-21T10:27:58.698 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:58.697+0000 7f95273fb740 1 -- 172.21.15.47:0/3678977813 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9514141a30 con 0x560094ab7de0 2024-03-21T10:27:58.698 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:58.697+0000 7f95273fb740 1 -- 172.21.15.47:0/3678977813 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f951413ef50 con 0x560094aae8b0 2024-03-21T10:27:58.698 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:58.697+0000 7f95273fb740 10 client.0 _ll_register_callbacks cb 0x7f951c0ae040 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T10:27:58.699 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:58.698+0000 7f9521200640 1 -- 172.21.15.47:0/3678977813 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560094aae8b0 msgr2=0x560094b25100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:27:58.699 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:58.698+0000 7f9521200640 1 --2- 172.21.15.47:0/3678977813 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560094aae8b0 0x560094b25100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:27:58.699 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:58.698+0000 7f9521200640 1 -- 172.21.15.47:0/3678977813 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560094ab7de0 msgr2=0x560094abbd60 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-21T10:27:58.699 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:58.698+0000 7f9521200640 1 --2- 172.21.15.47:0/3678977813 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560094ab7de0 0x560094abbd60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:27:58.699 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:58.698+0000 7f9521200640 1 -- 172.21.15.47:0/3678977813 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x560094abc450 con 0x560094aaf260 2024-03-21T10:27:58.700 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:58.698+0000 7f9521200640 1 --2- 172.21.15.47:0/3678977813 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x560094aaf260 0x560094b25640 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7f9508000c00 tx=0x7f9508009770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ed1128ba4ed0fab8 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:27:58.701 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:58.699+0000 7f9503400640 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-03-21T10:27:58.701 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:58.699+0000 7f9503400640 1 -- 172.21.15.47:0/3678977813 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9508067020 con 0x560094aaf260 2024-03-21T10:27:58.701 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:58.699+0000 7f95273fb740 1 -- 172.21.15.47:0/3678977813 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f950809c700 con 0x560094aaf260 2024-03-21T10:27:58.701 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:58.699+0000 7f9503400640 1 -- 172.21.15.47:0/3678977813 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(97..97 src has 1..97) v4 ==== 9474+0+0 (secure 0 0 0) 0x7f95080d8cb0 con 0x560094aaf260 2024-03-21T10:27:58.701 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:58.700+0000 7f9503400640 1 -- 172.21.15.47:0/3678977813 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f95080b4070 con 0x560094aaf260 2024-03-21T10:27:58.701 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:58.700+0000 7f95273fb740 10 client.16638 fetch_fsmap learned FSMap version 32 2024-03-21T10:27:58.701 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:58.700+0000 7f95273fb740 1 -- 172.21.15.47:0/3678977813 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x560094b2c830 con 0x560094aaf260 2024-03-21T10:27:58.702 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:58.700+0000 7f9503400640 1 -- 172.21.15.47:0/3678977813 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f950809c700 con 0x560094aaf260 2024-03-21T10:27:58.702 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:58.701+0000 7f9503400640 1 -- 172.21.15.47:0/3678977813 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 32) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f95080cd5e0 con 0x560094aaf260 2024-03-21T10:27:58.702 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:58.701+0000 7f95273fb740 10 client.16638 fetch_fsmap finished waiting for FSMap version 32 2024-03-21T10:27:58.702 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:58.701+0000 7f95273fb740 10 client.16638 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-03-21T10:27:58.702 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:58.701+0000 7f95273fb740 20 client.16638 populate_metadata read hostname 'smithi047' 2024-03-21T10:27:58.702 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:58.701+0000 7f95273fb740 1 --2- 172.21.15.47:0/3678977813 >> [v2:172.21.15.100:6836/3497644162,v1:172.21.15.100:6837/3497644162] conn(0x560094ad0550 0x560094b2ea10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:27:58.703 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:58.701+0000 7f95273fb740 4 client.16638 mds_command: new command op to 34493 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-21T10:27:58.703 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:58.701+0000 7f95273fb740 1 -- 172.21.15.47:0/3678977813 --> [v2:172.21.15.100:6836/3497644162,v1:172.21.15.100:6837/3497644162] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x560094a19b50 con 0x560094ad0550 2024-03-21T10:27:58.703 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:58.702+0000 7f9521c00640 1 --2- 172.21.15.47:0/3678977813 >> [v2:172.21.15.100:6836/3497644162,v1:172.21.15.100:6837/3497644162] conn(0x560094ad0550 0x560094b2ea10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:27:58.704 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:58.703+0000 7f9521c00640 1 --2- 172.21.15.47:0/3678977813 >> [v2:172.21.15.100:6836/3497644162,v1:172.21.15.100:6837/3497644162] conn(0x560094ad0550 0x560094b2ea10 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=8fc081d3901720a2 server_cookie=a0047bf18cbb19a3 in_seq=0 out_seq=0 2024-03-21T10:27:58.704 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:58.703+0000 7f9503400640 10 client.16638 ms_handle_connect on v2:172.21.15.100:6836/3497644162 2024-03-21T10:27:58.704 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:58.703+0000 7f9503400640 1 -- 172.21.15.47:0/3678977813 <== mds.1 v2:172.21.15.100:6836/3497644162 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x560094a19b50 con 0x560094ad0550 2024-03-21T10:27:58.704 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:58.704+0000 7f9503400640 10 client.16638 handle_command_reply: tid=0 2024-03-21T10:27:58.705 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:58.704+0000 7f95273fb740 1 client.16638 shutdown 2024-03-21T10:27:58.705 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:58.704+0000 7f95273fb740 1 -- 172.21.15.47:0/3678977813 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x560094aaf260 msgr2=0x560094b25640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:27:58.705 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:58.704+0000 7f95273fb740 1 --2- 172.21.15.47:0/3678977813 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x560094aaf260 0x560094b25640 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7f9508000c00 tx=0x7f9508009770 comp rx=0 tx=0).stop 2024-03-21T10:27:58.706 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:58.705+0000 7f95273fb740 1 -- 172.21.15.47:0/3678977813 shutdown_connections 2024-03-21T10:27:58.706 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:58.705+0000 7f95273fb740 1 --2- 172.21.15.47:0/3678977813 >> [v2:172.21.15.100:6836/3497644162,v1:172.21.15.100:6837/3497644162] conn(0x560094ad0550 0x560094b2ea10 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:27:58.706 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:58.705+0000 7f95273fb740 1 --2- 172.21.15.47:0/3678977813 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560094ab7de0 0x560094abbd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:27:58.706 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:58.705+0000 7f95273fb740 1 --2- 172.21.15.47:0/3678977813 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x560094aaf260 0x560094b25640 unknown :-1 s=CLOSED pgs=728 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:27:58.707 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:58.705+0000 7f95273fb740 1 --2- 172.21.15.47:0/3678977813 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560094aae8b0 0x560094b25100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:27:58.707 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:58.705+0000 7f95273fb740 1 -- 172.21.15.47:0/3678977813 >> 172.21.15.47:0/3678977813 conn(0x560094aa8f80 msgr2=0x560094aaa290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:27:58.707 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:58.705+0000 7f9503400640 0 client.16638 ms_handle_reset on v2:172.21.15.100:6836/3497644162 2024-03-21T10:27:58.708 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:58.706+0000 7f95273fb740 1 -- 172.21.15.47:0/3678977813 shutdown_connections 2024-03-21T10:27:58.708 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:58.706+0000 7f95273fb740 1 -- 172.21.15.47:0/3678977813 wait complete. 2024-03-21T10:27:58.708 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:58.706+0000 7f95273fb740 20 client.16638 trim_cache size 0 max 16384 2024-03-21T10:27:58.708 INFO:teuthology.orchestra.run.smithi047.stdout:[] 2024-03-21T10:27:58.709 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:58.708+0000 7f9523000640 1 -- 172.21.15.47:0/16815650 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f95040a3260 msgr2=0x7f95040a5720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:27:58.709 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:58.708+0000 7f9523000640 1 --2- 172.21.15.47:0/16815650 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f95040a3260 0x7f95040a5720 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f951c0671e0 tx=0x7f951c0a1040 comp rx=0 tx=0).stop 2024-03-21T10:27:58.709 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:58.708+0000 7f9523000640 1 -- 172.21.15.47:0/16815650 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f951409d230 msgr2=0x7f951412ec20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:27:58.710 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:58.708+0000 7f9523000640 1 --2- 172.21.15.47:0/16815650 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f951409d230 0x7f951412ec20 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f950800e9c0 tx=0x7f950800ee80 comp rx=0 tx=0).stop 2024-03-21T10:27:58.710 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:58.709+0000 7f9523000640 1 -- 172.21.15.47:0/16815650 shutdown_connections 2024-03-21T10:27:58.710 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:58.709+0000 7f9523000640 1 --2- 172.21.15.47:0/16815650 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9514132fa0 0x7f9514133450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:27:58.710 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:58.709+0000 7f9523000640 1 --2- 172.21.15.47:0/16815650 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f951409d230 0x7f951412ec20 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:27:58.710 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:58.709+0000 7f9523000640 1 --2- 172.21.15.47:0/16815650 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f95040a3260 0x7f95040a5720 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:27:58.710 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:58.709+0000 7f9523000640 1 --2- 172.21.15.47:0/16815650 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9514098160 0x7f951412e6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:27:58.710 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:58.709+0000 7f9523000640 1 -- 172.21.15.47:0/16815650 >> 172.21.15.47:0/16815650 conn(0x7f95140939c0 msgr2=0x7f9514095ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:27:58.711 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:58.710+0000 7f9523000640 1 -- 172.21.15.47:0/16815650 shutdown_connections 2024-03-21T10:27:58.711 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:58.710+0000 7f9523000640 1 -- 172.21.15.47:0/16815650 wait complete. 2024-03-21T10:27:58.740 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-21T10:27:58.741 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-03-21T10:27:59.082 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:59.081+0000 7fe039e00640 1 -- 172.21.15.47:0/2404432046 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe034109850 msgr2=0x7fe034109c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:27:59.082 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:59.081+0000 7fe039e00640 1 --2- 172.21.15.47:0/2404432046 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe034109850 0x7fe034109c30 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fe024009900 tx=0x7fe02402bc30 comp rx=0 tx=0).stop 2024-03-21T10:27:59.082 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:59.081+0000 7fe039e00640 1 -- 172.21.15.47:0/2404432046 shutdown_connections 2024-03-21T10:27:59.082 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:59.081+0000 7fe039e00640 1 --2- 172.21.15.47:0/2404432046 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe034104200 0x7fe034110710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:27:59.082 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:59.081+0000 7fe039e00640 1 --2- 172.21.15.47:0/2404432046 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe034103840 0x7fe034103cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:27:59.082 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:59.081+0000 7fe039e00640 1 --2- 172.21.15.47:0/2404432046 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe034109850 0x7fe034109c30 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:27:59.083 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:59.081+0000 7fe039e00640 1 -- 172.21.15.47:0/2404432046 >> 172.21.15.47:0/2404432046 conn(0x7fe0340ff2f0 msgr2=0x7fe034101710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:27:59.083 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:59.082+0000 7fe039e00640 1 -- 172.21.15.47:0/2404432046 shutdown_connections 2024-03-21T10:27:59.083 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:59.082+0000 7fe039e00640 1 -- 172.21.15.47:0/2404432046 wait complete. 2024-03-21T10:27:59.084 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:59.083+0000 7fe039e00640 1 Processor -- start 2024-03-21T10:27:59.085 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:59.084+0000 7fe039e00640 1 -- start start 2024-03-21T10:27:59.085 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:59.085+0000 7fe039e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe034103840 0x7fe034110620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:27:59.086 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:59.085+0000 7fe039e00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe034104200 0x7fe03410e720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:27:59.086 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:59.085+0000 7fe039e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe034109850 0x7fe03410eda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:27:59.086 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:59.085+0000 7fe039e00640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0340763a0 con 0x7fe034109850 2024-03-21T10:27:59.086 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:59.085+0000 7fe039e00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe034076220 con 0x7fe034103840 2024-03-21T10:27:59.086 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:59.085+0000 7fe039e00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe034076520 con 0x7fe034104200 2024-03-21T10:27:59.086 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:59.085+0000 7fe032a00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe034103840 0x7fe034110620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:27:59.086 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:59.085+0000 7fe032a00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe034103840 0x7fe034110620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:32888/0 (socket says 172.21.15.47:32888) 2024-03-21T10:27:59.086 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:59.085+0000 7fe032a00640 1 -- 172.21.15.47:0/2797488963 learned_addr learned my addr 172.21.15.47:0/2797488963 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:27:59.087 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:59.086+0000 7fe033400640 1 --2- 172.21.15.47:0/2797488963 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe034109850 0x7fe03410eda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:27:59.087 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:59.087+0000 7fe032a00640 1 -- 172.21.15.47:0/2797488963 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe034104200 msgr2=0x7fe03410e720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:27:59.088 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:59.087+0000 7fe032000640 1 --2- 172.21.15.47:0/2797488963 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe034104200 0x7fe03410e720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:27:59.088 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:59.087+0000 7fe032a00640 1 --2- 172.21.15.47:0/2797488963 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe034104200 0x7fe03410e720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:27:59.088 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:59.087+0000 7fe032a00640 1 -- 172.21.15.47:0/2797488963 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe034109850 msgr2=0x7fe03410eda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:27:59.088 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:59.087+0000 7fe032a00640 1 --2- 172.21.15.47:0/2797488963 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe034109850 0x7fe03410eda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:27:59.088 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:59.087+0000 7fe032a00640 1 -- 172.21.15.47:0/2797488963 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe0341aa830 con 0x7fe034103840 2024-03-21T10:27:59.089 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:59.087+0000 7fe033400640 1 --2- 172.21.15.47:0/2797488963 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe034109850 0x7fe03410eda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T10:27:59.089 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:59.088+0000 7fe032000640 1 --2- 172.21.15.47:0/2797488963 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe034104200 0x7fe03410e720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T10:27:59.089 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:59.088+0000 7fe032a00640 1 --2- 172.21.15.47:0/2797488963 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe034103840 0x7fe034110620 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fe024005ab0 tx=0x7fe024004600 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:27:59.089 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:59.088+0000 7fe038a00640 1 -- 172.21.15.47:0/2797488963 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe024043020 con 0x7fe034103840 2024-03-21T10:27:59.089 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:59.088+0000 7fe038a00640 1 -- 172.21.15.47:0/2797488963 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fe02402ca90 con 0x7fe034103840 2024-03-21T10:27:59.089 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:59.089+0000 7fe038a00640 1 -- 172.21.15.47:0/2797488963 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe024003a80 con 0x7fe034103840 2024-03-21T10:27:59.089 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:59.089+0000 7fe039e00640 1 -- 172.21.15.47:0/2797488963 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe0341aaa00 con 0x7fe034103840 2024-03-21T10:27:59.090 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:59.089+0000 7fe039e00640 1 -- 172.21.15.47:0/2797488963 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe0341aaed0 con 0x7fe034103840 2024-03-21T10:27:59.095 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:59.095+0000 7fe03ca55740 1 Processor -- start 2024-03-21T10:27:59.096 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:59.095+0000 7fe03ca55740 1 -- start start 2024-03-21T10:27:59.096 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:59.095+0000 7fe03ca55740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ab56bf76d0 0x55ab56bf7ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:27:59.097 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:59.096+0000 7fe038a00640 1 -- 172.21.15.47:0/2797488963 <== mon.2 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 27) v1 ==== 137985+0+0 (secure 0 0 0) 0x7fe024046020 con 0x7fe034103840 2024-03-21T10:27:59.097 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:59.096+0000 7fe03ca55740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ab56bf8080 0x55ab56c00630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:27:59.097 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:59.096+0000 7fe03ca55740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ab56c00c00 0x55ab56c03050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:27:59.097 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:59.096+0000 7fe03ca55740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ab56b5b2c0 con 0x55ab56bf8080 2024-03-21T10:27:59.097 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:59.096+0000 7fe03ca55740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ab56b03c90 con 0x55ab56bf76d0 2024-03-21T10:27:59.097 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:59.096+0000 7fe03ca55740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ab56b57360 con 0x55ab56c00c00 2024-03-21T10:27:59.097 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:59.096+0000 7fe033400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ab56bf76d0 0x55ab56bf7ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:27:59.097 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:59.096+0000 7fe033400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ab56bf76d0 0x55ab56bf7ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:32900/0 (socket says 172.21.15.47:32900) 2024-03-21T10:27:59.097 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:59.096+0000 7fe033400640 1 -- 172.21.15.47:0/2852753962 learned_addr learned my addr 172.21.15.47:0/2852753962 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:27:59.098 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:59.097+0000 7fe032a00640 1 --2- 172.21.15.47:0/2852753962 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ab56bf8080 0x55ab56c00630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:27:59.098 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:59.097+0000 7fe032000640 1 --2- 172.21.15.47:0/2852753962 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ab56c00c00 0x55ab56c03050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:27:59.098 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:59.098+0000 7fe033400640 1 -- 172.21.15.47:0/2852753962 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ab56c00c00 msgr2=0x55ab56c03050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:27:59.098 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:59.098+0000 7fe033400640 1 --2- 172.21.15.47:0/2852753962 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ab56c00c00 0x55ab56c03050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:27:59.098 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:59.098+0000 7fe033400640 1 -- 172.21.15.47:0/2852753962 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ab56bf8080 msgr2=0x55ab56c00630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:27:59.098 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:59.098+0000 7fe033400640 1 --2- 172.21.15.47:0/2852753962 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ab56bf8080 0x55ab56c00630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:27:59.099 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:59.098+0000 7fe033400640 1 -- 172.21.15.47:0/2852753962 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ab56b28080 con 0x55ab56bf76d0 2024-03-21T10:27:59.099 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:59.099+0000 7fe033400640 1 --2- 172.21.15.47:0/2852753962 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ab56bf76d0 0x55ab56bf7ab0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fe02c011790 tx=0x7fe02c011c50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a4e90772ea2f39f server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:27:59.100 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:59.099+0000 7fe01fe00640 1 -- 172.21.15.47:0/2852753962 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe02c003c30 con 0x55ab56bf76d0 2024-03-21T10:27:59.100 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:59.099+0000 7fe038a00640 1 --2- 172.21.15.47:0/2797488963 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7fe0180a3150 0x7fe0180a5610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:27:59.100 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:59.099+0000 7fe01fe00640 1 -- 172.21.15.47:0/2852753962 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fe02c00fe50 con 0x55ab56bf76d0 2024-03-21T10:27:59.100 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:59.099+0000 7fe01fe00640 1 -- 172.21.15.47:0/2852753962 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe02c004820 con 0x55ab56bf76d0 2024-03-21T10:27:59.100 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:59.100+0000 7fe038a00640 1 -- 172.21.15.47:0/2797488963 <== mon.2 v2:172.21.15.100:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 9474+0+0 (secure 0 0 0) 0x7fe0240f1c30 con 0x7fe034103840 2024-03-21T10:27:59.101 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:59.101+0000 7fe032a00640 1 --2- 172.21.15.47:0/2797488963 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7fe0180a3150 0x7fe0180a5610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:27:59.102 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:59.101+0000 7fe03ca55740 1 -- 172.21.15.47:0/2852753962 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ab56bf76d0 msgr2=0x55ab56bf7ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:27:59.102 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:59.101+0000 7fe03ca55740 1 --2- 172.21.15.47:0/2852753962 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ab56bf76d0 0x55ab56bf7ab0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fe02c011790 tx=0x7fe02c011c50 comp rx=0 tx=0).stop 2024-03-21T10:27:59.103 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:59.102+0000 7fe03ca55740 1 -- 172.21.15.47:0/2852753962 shutdown_connections 2024-03-21T10:27:59.103 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:59.102+0000 7fe03ca55740 1 --2- 172.21.15.47:0/2852753962 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ab56c00c00 0x55ab56c03050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:27:59.103 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:59.102+0000 7fe03ca55740 1 --2- 172.21.15.47:0/2852753962 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ab56bf8080 0x55ab56c00630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:27:59.103 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:59.102+0000 7fe03ca55740 1 --2- 172.21.15.47:0/2852753962 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ab56bf76d0 0x55ab56bf7ab0 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:27:59.103 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:59.102+0000 7fe03ca55740 1 -- 172.21.15.47:0/2852753962 >> 172.21.15.47:0/2852753962 conn(0x55ab56bf1da0 msgr2=0x55ab56bf2180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:27:59.104 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:59.103+0000 7fe03ca55740 1 -- 172.21.15.47:0/2852753962 shutdown_connections 2024-03-21T10:27:59.104 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:59.103+0000 7fe03ca55740 1 -- 172.21.15.47:0/2852753962 wait complete. 2024-03-21T10:27:59.104 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:59.103+0000 7fe032a00640 1 --2- 172.21.15.47:0/2797488963 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7fe0180a3150 0x7fe0180a5610 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x55ab56bf71a0 tx=0x7fe0240de040 comp rx=0 tx=0).ready entity=mgr.15687 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:27:59.104 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:59.103+0000 7fe03ca55740 1 -- start start 2024-03-21T10:27:59.105 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:59.104+0000 7fe03ca55740 1 Processor -- start 2024-03-21T10:27:59.106 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:59.105+0000 7fe03ca55740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ab56bf76d0 0x55ab56c00520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:27:59.106 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:59.106+0000 7fe03ca55740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ab56bf8080 0x55ab56bfd8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:27:59.108 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:59.106+0000 7fe033400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ab56bf76d0 0x55ab56c00520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:27:59.108 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:59.107+0000 7fe033400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ab56bf76d0 0x55ab56c00520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:32912/0 (socket says 172.21.15.47:32912) 2024-03-21T10:27:59.108 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:59.107+0000 7fe032000640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ab56bf8080 0x55ab56bfd8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:27:59.108 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:59.107+0000 7fe032000640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ab56bf8080 0x55ab56bfd8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.47:40320/0 (socket says 172.21.15.47:40320) 2024-03-21T10:27:59.109 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:59.108+0000 7fe03ca55740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ab56c00c00 0x55ab56bfde50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:27:59.110 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:59.108+0000 7fe03ca55740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ab56c03b70 con 0x55ab56c00c00 2024-03-21T10:27:59.110 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:59.108+0000 7fe03ca55740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ab56b170d0 con 0x55ab56bf76d0 2024-03-21T10:27:59.110 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:59.108+0000 7fe03ca55740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ab56c03cf0 con 0x55ab56bf8080 2024-03-21T10:27:59.110 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:59.108+0000 7fe03ca55740 10 client.0 _ll_register_callbacks cb 0x55ab56b1be80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T10:27:59.110 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:59.108+0000 7fe033400640 1 -- 172.21.15.47:0/46585511 learned_addr learned my addr 172.21.15.47:0/46585511 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:27:59.110 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:59.109+0000 7fe033400640 1 -- 172.21.15.47:0/46585511 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ab56bf8080 msgr2=0x55ab56bfd8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:27:59.111 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:59.109+0000 7fe033400640 1 --2- 172.21.15.47:0/46585511 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ab56bf8080 0x55ab56bfd8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:27:59.111 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:59.109+0000 7fe033400640 1 -- 172.21.15.47:0/46585511 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ab56c00c00 msgr2=0x55ab56bfde50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:27:59.111 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:59.109+0000 7fe033400640 1 --2- 172.21.15.47:0/46585511 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ab56c00c00 0x55ab56bfde50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:27:59.111 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:59.109+0000 7fe033400640 1 -- 172.21.15.47:0/46585511 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55ab5686f3f0 con 0x55ab56bf76d0 2024-03-21T10:27:59.111 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:59.109+0000 7fe033400640 1 --2- 172.21.15.47:0/46585511 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ab56bf76d0 0x55ab56c00520 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fe02c005c10 tx=0x7fe02c00fbd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a3ec4478c95829aa server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:27:59.111 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:59.109+0000 7fe01cc00640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-21T10:27:59.114 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:59.111+0000 7fe01cc00640 1 -- 172.21.15.47:0/46585511 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe02c02a020 con 0x55ab56bf76d0 2024-03-21T10:27:59.114 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:59.111+0000 7fe01cc00640 1 -- 172.21.15.47:0/46585511 <== mon.2 v2:172.21.15.100:3300/0 2 ==== osd_map(97..97 src has 1..97) v4 ==== 9474+0+0 (secure 0 0 0) 0x7fe02c011240 con 0x55ab56bf76d0 2024-03-21T10:27:59.114 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:59.114+0000 7fe03ca55740 1 -- 172.21.15.47:0/46585511 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fe02c00fe50 con 0x55ab56bf76d0 2024-03-21T10:27:59.116 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:59.116+0000 7fe01cc00640 1 -- 172.21.15.47:0/46585511 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe02c01f040 con 0x55ab56bf76d0 2024-03-21T10:27:59.118 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:59.117+0000 7fe03ca55740 10 client.36443 fetch_fsmap learned FSMap version 32 2024-03-21T10:27:59.118 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:59.118+0000 7fe03ca55740 1 -- 172.21.15.47:0/46585511 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x55ab569397d0 con 0x55ab56bf76d0 2024-03-21T10:27:59.120 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:59.119+0000 7fe01cc00640 1 -- 172.21.15.47:0/46585511 <== mon.2 v2:172.21.15.100:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fe02c00fe50 con 0x55ab56bf76d0 2024-03-21T10:27:59.120 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:59.119+0000 7fe01cc00640 1 -- 172.21.15.47:0/46585511 <== mon.2 v2:172.21.15.100:3300/0 5 ==== fsmap(e 32) v1 ==== 7942+0+0 (secure 0 0 0) 0x7fe02c018070 con 0x55ab56bf76d0 2024-03-21T10:27:59.120 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:59.119+0000 7fe03ca55740 10 client.36443 fetch_fsmap finished waiting for FSMap version 32 2024-03-21T10:27:59.120 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:59.119+0000 7fe03ca55740 10 client.36443 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-21T10:27:59.120 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:59.119+0000 7fe03ca55740 20 client.36443 populate_metadata read hostname 'smithi047' 2024-03-21T10:27:59.120 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:59.119+0000 7fe03ca55740 1 --2- 172.21.15.47:0/46585511 >> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] conn(0x55ab56c10910 0x55ab56c12d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:27:59.120 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:59.119+0000 7fe03ca55740 4 client.36443 mds_command: new command op to 34535 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T10:27:59.120 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:59.119+0000 7fe03ca55740 1 -- 172.21.15.47:0/46585511 --> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55ab56b68800 con 0x55ab56c10910 2024-03-21T10:27:59.121 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:59.121+0000 7fe032000640 1 --2- 172.21.15.47:0/46585511 >> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] conn(0x55ab56c10910 0x55ab56c12d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:27:59.122 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:59.122+0000 7fe032000640 1 --2- 172.21.15.47:0/46585511 >> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] conn(0x55ab56c10910 0x55ab56c12d00 crc :-1 s=READY pgs=101 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=645b572e40056c7b server_cookie=681c4f3114b6aa33 in_seq=0 out_seq=0 2024-03-21T10:27:59.122 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:59.122+0000 7fe01cc00640 10 client.36443 ms_handle_connect on v2:172.21.15.100:6838/4173168440 2024-03-21T10:27:59.124 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:59.123+0000 7fe01cc00640 1 -- 172.21.15.47:0/46585511 <== mds.0 v2:172.21.15.100:6838/4173168440 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55ab56b68800 con 0x55ab56c10910 2024-03-21T10:27:59.124 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:59.123+0000 7fe01cc00640 10 client.36443 handle_command_reply: tid=0 2024-03-21T10:27:59.124 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:59.123+0000 7fe03ca55740 1 client.36443 shutdown 2024-03-21T10:27:59.124 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:59.124+0000 7fe03ca55740 1 -- 172.21.15.47:0/46585511 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ab56bf76d0 msgr2=0x55ab56c00520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:27:59.124 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:59.124+0000 7fe03ca55740 1 --2- 172.21.15.47:0/46585511 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ab56bf76d0 0x55ab56c00520 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fe02c005c10 tx=0x7fe02c00fbd0 comp rx=0 tx=0).stop 2024-03-21T10:27:59.125 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:59.124+0000 7fe03ca55740 1 -- 172.21.15.47:0/46585511 shutdown_connections 2024-03-21T10:27:59.125 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:59.124+0000 7fe03ca55740 1 --2- 172.21.15.47:0/46585511 >> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] conn(0x55ab56c10910 0x55ab56c12d00 crc :-1 s=READY pgs=101 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:27:59.125 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:59.124+0000 7fe03ca55740 1 --2- 172.21.15.47:0/46585511 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ab56c00c00 0x55ab56bfde50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:27:59.125 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:59.124+0000 7fe03ca55740 1 --2- 172.21.15.47:0/46585511 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ab56bf8080 0x55ab56bfd8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:27:59.125 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:59.124+0000 7fe03ca55740 1 --2- 172.21.15.47:0/46585511 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ab56bf76d0 0x55ab56c00520 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:27:59.125 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:59.124+0000 7fe03ca55740 1 -- 172.21.15.47:0/46585511 >> 172.21.15.47:0/46585511 conn(0x55ab56bf1da0 msgr2=0x55ab56bf3af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:27:59.125 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:59.124+0000 7fe01cc00640 0 client.36443 ms_handle_reset on v2:172.21.15.100:6838/4173168440 2024-03-21T10:27:59.126 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:59.125+0000 7fe03ca55740 1 -- 172.21.15.47:0/46585511 shutdown_connections 2024-03-21T10:27:59.126 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:59.125+0000 7fe03ca55740 1 -- 172.21.15.47:0/46585511 wait complete. 2024-03-21T10:27:59.126 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:59.125+0000 7fe03ca55740 20 client.36443 trim_cache size 0 max 16384 2024-03-21T10:27:59.155 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:59.154+0000 7fe03ca55740 1 Processor -- start 2024-03-21T10:27:59.155 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:59.154+0000 7fe03ca55740 1 -- start start 2024-03-21T10:27:59.155 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:59.155+0000 7fe03ca55740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ab56bf76d0 0x55ab56bfffe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:27:59.156 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:59.155+0000 7fe03ca55740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ab56bf8080 0x55ab56c00520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:27:59.156 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:59.155+0000 7fe03ca55740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ab56c00c00 0x55ab56bfab30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:27:59.156 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:59.155+0000 7fe03ca55740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0341b5070 con 0x55ab56c00c00 2024-03-21T10:27:59.156 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:59.155+0000 7fe03ca55740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0341b69f0 con 0x55ab56bf76d0 2024-03-21T10:27:59.156 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:59.155+0000 7fe03ca55740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0341b56d0 con 0x55ab56bf8080 2024-03-21T10:27:59.156 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:59.155+0000 7fe032000640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ab56bf8080 0x55ab56c00520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:27:59.156 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:59.156+0000 7fe032000640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ab56bf8080 0x55ab56c00520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.47:40324/0 (socket says 172.21.15.47:40324) 2024-03-21T10:27:59.157 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:59.156+0000 7fe032000640 1 -- 172.21.15.47:0/533278964 learned_addr learned my addr 172.21.15.47:0/533278964 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:27:59.158 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:59.156+0000 7fe032000640 1 -- 172.21.15.47:0/533278964 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ab56bf76d0 msgr2=0x55ab56bfffe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:27:59.158 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:59.156+0000 7fe032000640 1 --2- 172.21.15.47:0/533278964 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ab56bf76d0 0x55ab56bfffe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:27:59.158 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:59.156+0000 7fe032000640 1 -- 172.21.15.47:0/533278964 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ab56c00c00 msgr2=0x55ab56bfab30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:27:59.158 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:59.156+0000 7fe032000640 1 --2- 172.21.15.47:0/533278964 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ab56c00c00 0x55ab56bfab30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:27:59.158 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:59.156+0000 7fe032000640 1 -- 172.21.15.47:0/533278964 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ab56c03a50 con 0x55ab56bf8080 2024-03-21T10:27:59.159 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:59.158+0000 7fe032000640 1 --2- 172.21.15.47:0/533278964 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ab56bf8080 0x55ab56c00520 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x55ab56bfd110 tx=0x7fe020005f90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1eb7c5c66b54200a server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:27:59.159 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:59.158+0000 7fe01f400640 1 -- 172.21.15.47:0/533278964 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe02001e250 con 0x55ab56bf8080 2024-03-21T10:27:59.159 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:59.158+0000 7fe01f400640 1 -- 172.21.15.47:0/533278964 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fe020023070 con 0x55ab56bf8080 2024-03-21T10:27:59.159 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:59.158+0000 7fe01f400640 1 -- 172.21.15.47:0/533278964 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0200126b0 con 0x55ab56bf8080 2024-03-21T10:27:59.161 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:59.160+0000 7fe03ca55740 1 -- 172.21.15.47:0/533278964 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ab56bf8080 msgr2=0x55ab56c00520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:27:59.161 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:59.160+0000 7fe03ca55740 1 --2- 172.21.15.47:0/533278964 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ab56bf8080 0x55ab56c00520 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x55ab56bfd110 tx=0x7fe020005f90 comp rx=0 tx=0).stop 2024-03-21T10:27:59.161 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:59.161+0000 7fe03ca55740 1 -- 172.21.15.47:0/533278964 shutdown_connections 2024-03-21T10:27:59.161 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:59.161+0000 7fe03ca55740 1 --2- 172.21.15.47:0/533278964 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ab56c00c00 0x55ab56bfab30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:27:59.162 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:59.161+0000 7fe03ca55740 1 --2- 172.21.15.47:0/533278964 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ab56bf8080 0x55ab56c00520 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:27:59.162 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:59.161+0000 7fe03ca55740 1 --2- 172.21.15.47:0/533278964 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ab56bf76d0 0x55ab56bfffe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:27:59.162 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:59.161+0000 7fe03ca55740 1 -- 172.21.15.47:0/533278964 >> 172.21.15.47:0/533278964 conn(0x55ab56bf1da0 msgr2=0x55ab56bf0eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:27:59.162 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:59.162+0000 7fe03ca55740 1 -- 172.21.15.47:0/533278964 shutdown_connections 2024-03-21T10:27:59.163 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:59.162+0000 7fe03ca55740 1 -- 172.21.15.47:0/533278964 wait complete. 2024-03-21T10:27:59.163 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:59.163+0000 7fe03ca55740 1 -- start start 2024-03-21T10:27:59.164 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:59.163+0000 7fe03ca55740 1 Processor -- start 2024-03-21T10:27:59.165 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:59.164+0000 7fe03ca55740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ab56bf76d0 0x55ab56c08720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:27:59.165 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:59.164+0000 7fe03ca55740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ab56bf8080 0x55ab56c08c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:27:59.165 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:59.164+0000 7fe03ca55740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ab56c00c00 0x55ab56c64b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:27:59.165 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:59.164+0000 7fe03ca55740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0341b6390 con 0x55ab56bf8080 2024-03-21T10:27:59.166 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:59.164+0000 7fe03ca55740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0341b5d30 con 0x55ab56c00c00 2024-03-21T10:27:59.166 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:59.164+0000 7fe03ca55740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0341b3230 con 0x55ab56bf76d0 2024-03-21T10:27:59.166 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:59.164+0000 7fe03ca55740 10 client.0 _ll_register_callbacks cb 0x7fe0240f55b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T10:27:59.166 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:59.165+0000 7fe032000640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ab56bf8080 0x55ab56c08c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:27:59.167 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:59.165+0000 7fe032000640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ab56bf8080 0x55ab56c08c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:57828/0 (socket says 172.21.15.47:57828) 2024-03-21T10:27:59.167 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:59.165+0000 7fe032000640 1 -- 172.21.15.47:0/3463044220 learned_addr learned my addr 172.21.15.47:0/3463044220 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:27:59.167 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:59.165+0000 7fe033400640 1 --2- 172.21.15.47:0/3463044220 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ab56c00c00 0x55ab56c64b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:27:59.167 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:59.165+0000 7fe033400640 1 --2- 172.21.15.47:0/3463044220 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ab56bf76d0 0x55ab56c08720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:27:59.167 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:59.165+0000 7fe032000640 1 -- 172.21.15.47:0/3463044220 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ab56bf76d0 msgr2=0x55ab56c08720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:27:59.167 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:59.165+0000 7fe032000640 1 --2- 172.21.15.47:0/3463044220 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ab56bf76d0 0x55ab56c08720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:27:59.167 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:59.165+0000 7fe032000640 1 -- 172.21.15.47:0/3463044220 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ab56c00c00 msgr2=0x55ab56c64b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:27:59.167 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:59.165+0000 7fe032000640 1 --2- 172.21.15.47:0/3463044220 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ab56c00c00 0x55ab56c64b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:27:59.167 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:59.165+0000 7fe032000640 1 -- 172.21.15.47:0/3463044220 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55ab56c65330 con 0x55ab56bf8080 2024-03-21T10:27:59.167 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:59.166+0000 7fe032000640 1 --2- 172.21.15.47:0/3463044220 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ab56bf8080 0x55ab56c08c60 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7fe020009e80 tx=0x7fe020020000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8a27aa7d3bb79863 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:27:59.167 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:59.166+0000 7fe01cc00640 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-03-21T10:27:59.168 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:59.167+0000 7fe01cc00640 1 -- 172.21.15.47:0/3463044220 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe020003d20 con 0x55ab56bf8080 2024-03-21T10:27:59.168 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:59.167+0000 7fe01cc00640 1 -- 172.21.15.47:0/3463044220 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(97..97 src has 1..97) v4 ==== 9474+0+0 (secure 0 0 0) 0x7fe02001e6f0 con 0x55ab56bf8080 2024-03-21T10:27:59.168 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:59.167+0000 7fe03ca55740 1 -- 172.21.15.47:0/3463044220 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fe020023070 con 0x55ab56bf8080 2024-03-21T10:27:59.168 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:59.167+0000 7fe01cc00640 1 -- 172.21.15.47:0/3463044220 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe020038050 con 0x55ab56bf8080 2024-03-21T10:27:59.169 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:59.167+0000 7fe03ca55740 10 client.16650 fetch_fsmap learned FSMap version 32 2024-03-21T10:27:59.169 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:59.167+0000 7fe03ca55740 1 -- 172.21.15.47:0/3463044220 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x55ab56c7da30 con 0x55ab56bf8080 2024-03-21T10:27:59.169 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:59.168+0000 7fe01cc00640 1 -- 172.21.15.47:0/3463044220 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fe020023070 con 0x55ab56bf8080 2024-03-21T10:27:59.169 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:59.168+0000 7fe01cc00640 1 -- 172.21.15.47:0/3463044220 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 32) v1 ==== 7942+0+0 (secure 0 0 0) 0x7fe02000f040 con 0x55ab56bf8080 2024-03-21T10:27:59.169 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:59.168+0000 7fe03ca55740 10 client.16650 fetch_fsmap finished waiting for FSMap version 32 2024-03-21T10:27:59.169 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:59.168+0000 7fe03ca55740 10 client.16650 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-21T10:27:59.169 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:59.168+0000 7fe03ca55740 20 client.16650 populate_metadata read hostname 'smithi047' 2024-03-21T10:27:59.169 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:59.168+0000 7fe03ca55740 1 --2- 172.21.15.47:0/3463044220 >> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] conn(0x55ab56c7e0e0 0x55ab56c80500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:27:59.169 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:59.168+0000 7fe03ca55740 4 client.16650 mds_command: new command op to 34535 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-21T10:27:59.169 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:59.168+0000 7fe03ca55740 1 -- 172.21.15.47:0/3463044220 --> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55ab56b62b50 con 0x55ab56c7e0e0 2024-03-21T10:27:59.170 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:59.170+0000 7fe033400640 1 --2- 172.21.15.47:0/3463044220 >> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] conn(0x55ab56c7e0e0 0x55ab56c80500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:27:59.172 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:59.171+0000 7fe033400640 1 --2- 172.21.15.47:0/3463044220 >> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] conn(0x55ab56c7e0e0 0x55ab56c80500 crc :-1 s=READY pgs=102 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f87ae38235f55c3d server_cookie=c1b0dd17fe46881b in_seq=0 out_seq=0 2024-03-21T10:27:59.172 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:59.171+0000 7fe01cc00640 10 client.16650 ms_handle_connect on v2:172.21.15.100:6838/4173168440 2024-03-21T10:27:59.173 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:59.173+0000 7fe01cc00640 1 -- 172.21.15.47:0/3463044220 <== mds.0 v2:172.21.15.100:6838/4173168440 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55ab56b62b50 con 0x55ab56c7e0e0 2024-03-21T10:27:59.173 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:59.173+0000 7fe01cc00640 10 client.16650 handle_command_reply: tid=0 2024-03-21T10:27:59.173 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:59.173+0000 7fe03ca55740 1 client.16650 shutdown 2024-03-21T10:27:59.174 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:59.174+0000 7fe03ca55740 1 -- 172.21.15.47:0/3463044220 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ab56bf8080 msgr2=0x55ab56c08c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:27:59.175 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:59.174+0000 7fe03ca55740 1 --2- 172.21.15.47:0/3463044220 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ab56bf8080 0x55ab56c08c60 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7fe020009e80 tx=0x7fe020020000 comp rx=0 tx=0).stop 2024-03-21T10:27:59.175 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:59.174+0000 7fe03ca55740 1 -- 172.21.15.47:0/3463044220 shutdown_connections 2024-03-21T10:27:59.175 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:59.174+0000 7fe03ca55740 1 --2- 172.21.15.47:0/3463044220 >> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] conn(0x55ab56c7e0e0 0x55ab56c80500 crc :-1 s=READY pgs=102 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:27:59.175 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:59.174+0000 7fe03ca55740 1 --2- 172.21.15.47:0/3463044220 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ab56c00c00 0x55ab56c64b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:27:59.175 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:59.174+0000 7fe03ca55740 1 --2- 172.21.15.47:0/3463044220 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ab56bf8080 0x55ab56c08c60 unknown :-1 s=CLOSED pgs=729 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:27:59.175 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:59.174+0000 7fe03ca55740 1 --2- 172.21.15.47:0/3463044220 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ab56bf76d0 0x55ab56c08720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:27:59.175 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:59.174+0000 7fe03ca55740 1 -- 172.21.15.47:0/3463044220 >> 172.21.15.47:0/3463044220 conn(0x55ab56bf1da0 msgr2=0x55ab56bf30a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:27:59.175 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:59.174+0000 7fe01cc00640 0 client.16650 ms_handle_reset on v2:172.21.15.100:6838/4173168440 2024-03-21T10:27:59.176 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:59.174+0000 7fe03ca55740 1 -- 172.21.15.47:0/3463044220 shutdown_connections 2024-03-21T10:27:59.176 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:59.175+0000 7fe03ca55740 1 -- 172.21.15.47:0/3463044220 wait complete. 2024-03-21T10:27:59.176 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:59.175+0000 7fe03ca55740 20 client.16650 trim_cache size 0 max 16384 2024-03-21T10:27:59.176 INFO:teuthology.orchestra.run.smithi047.stdout:[] 2024-03-21T10:27:59.177 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:59.175+0000 7fe039e00640 1 -- 172.21.15.47:0/2797488963 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7fe0180a3150 msgr2=0x7fe0180a5610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:27:59.177 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:59.175+0000 7fe039e00640 1 --2- 172.21.15.47:0/2797488963 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7fe0180a3150 0x7fe0180a5610 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x55ab56bf71a0 tx=0x7fe0240de040 comp rx=0 tx=0).stop 2024-03-21T10:27:59.177 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:59.175+0000 7fe039e00640 1 -- 172.21.15.47:0/2797488963 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe034103840 msgr2=0x7fe034110620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:27:59.177 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:59.175+0000 7fe039e00640 1 --2- 172.21.15.47:0/2797488963 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe034103840 0x7fe034110620 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fe024005ab0 tx=0x7fe024004600 comp rx=0 tx=0).stop 2024-03-21T10:27:59.177 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:59.176+0000 7fe039e00640 1 -- 172.21.15.47:0/2797488963 shutdown_connections 2024-03-21T10:27:59.177 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:59.176+0000 7fe039e00640 1 --2- 172.21.15.47:0/2797488963 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe034109850 0x7fe03410eda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:27:59.177 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:59.176+0000 7fe039e00640 1 --2- 172.21.15.47:0/2797488963 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe034104200 0x7fe03410e720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:27:59.177 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:59.176+0000 7fe039e00640 1 --2- 172.21.15.47:0/2797488963 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7fe0180a3150 0x7fe0180a5610 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:27:59.178 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:59.176+0000 7fe039e00640 1 --2- 172.21.15.47:0/2797488963 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe034103840 0x7fe034110620 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:27:59.178 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:59.176+0000 7fe039e00640 1 -- 172.21.15.47:0/2797488963 >> 172.21.15.47:0/2797488963 conn(0x7fe0340ff2f0 msgr2=0x7fe0340ffb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:27:59.178 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:59.176+0000 7fe039e00640 1 -- 172.21.15.47:0/2797488963 shutdown_connections 2024-03-21T10:27:59.178 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:27:59.176+0000 7fe039e00640 1 -- 172.21.15.47:0/2797488963 wait complete. 2024-03-21T10:27:59.964 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:27:59 smithi047 ceph-mon[31547]: pgmap v83: 129 pgs: 129 active+clean; 273 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 7.2 MiB/s wr, 1.55k op/s 2024-03-21T10:27:59.978 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:27:59 smithi100 ceph-mon[35193]: pgmap v83: 129 pgs: 129 active+clean; 273 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 7.2 MiB/s wr, 1.55k op/s 2024-03-21T10:27:59.988 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:27:59 smithi157 ceph-mon[35581]: pgmap v83: 129 pgs: 129 active+clean; 273 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 7.2 MiB/s wr, 1.55k op/s 2024-03-21T10:28:00.213 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-03-21T10:28:00.213 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-21T10:28:00.213 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-03-21T10:28:00.524 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:00.523+0000 7f1b04c00640 1 -- 172.21.15.47:0/2447465079 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1b00073a70 msgr2=0x7f1b00073e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:28:00.524 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:00.523+0000 7f1b04c00640 1 --2- 172.21.15.47:0/2447465079 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1b00073a70 0x7f1b00073e50 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f1af0009900 tx=0x7f1af002bcc0 comp rx=0 tx=0).stop 2024-03-21T10:28:00.524 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:00.523+0000 7f1b04c00640 1 -- 172.21.15.47:0/2447465079 shutdown_connections 2024-03-21T10:28:00.525 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:00.523+0000 7f1b04c00640 1 --2- 172.21.15.47:0/2447465079 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1b0010ded0 0x7f1b001102e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:28:00.525 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:00.523+0000 7f1b04c00640 1 --2- 172.21.15.47:0/2447465079 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1b00074390 0x7f1b0010d990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:28:00.525 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:00.523+0000 7f1b04c00640 1 --2- 172.21.15.47:0/2447465079 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1b00073a70 0x7f1b00073e50 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:28:00.525 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:00.523+0000 7f1b04c00640 1 -- 172.21.15.47:0/2447465079 >> 172.21.15.47:0/2447465079 conn(0x7f1b00078aa0 msgr2=0x7f1b00078eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:28:00.525 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:00.523+0000 7f1b04c00640 1 -- 172.21.15.47:0/2447465079 shutdown_connections 2024-03-21T10:28:00.525 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:00.524+0000 7f1b04c00640 1 -- 172.21.15.47:0/2447465079 wait complete. 2024-03-21T10:28:00.525 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:00.525+0000 7f1b04c00640 1 Processor -- start 2024-03-21T10:28:00.526 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:00.525+0000 7f1b04c00640 1 -- start start 2024-03-21T10:28:00.526 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:00.525+0000 7f1b04c00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1b00073a70 0x7f1b001015f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:28:00.527 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:00.525+0000 7f1b04c00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1b00074390 0x7f1b00101b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:28:00.527 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:00.525+0000 7f1b04c00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1b0010ded0 0x7f1b00102070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:28:00.527 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:00.525+0000 7f1b04c00640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b00113b50 con 0x7f1b00074390 2024-03-21T10:28:00.527 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:00.525+0000 7f1b04c00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b001139d0 con 0x7f1b00073a70 2024-03-21T10:28:00.527 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:00.525+0000 7f1b04c00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b00113cd0 con 0x7f1b0010ded0 2024-03-21T10:28:00.527 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:00.526+0000 7f1afde00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1b00073a70 0x7f1b001015f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:28:00.527 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:00.526+0000 7f1afde00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1b00073a70 0x7f1b001015f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:32946/0 (socket says 172.21.15.47:32946) 2024-03-21T10:28:00.527 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:00.526+0000 7f1afde00640 1 -- 172.21.15.47:0/3246186942 learned_addr learned my addr 172.21.15.47:0/3246186942 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:28:00.527 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:00.526+0000 7f1afe800640 1 --2- 172.21.15.47:0/3246186942 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1b0010ded0 0x7f1b00102070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:28:00.527 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:00.526+0000 7f1afde00640 1 -- 172.21.15.47:0/3246186942 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1b0010ded0 msgr2=0x7f1b00102070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:28:00.527 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:00.526+0000 7f1afde00640 1 --2- 172.21.15.47:0/3246186942 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1b0010ded0 0x7f1b00102070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:28:00.528 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:00.526+0000 7f1afde00640 1 -- 172.21.15.47:0/3246186942 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1b00074390 msgr2=0x7f1b00101b30 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-21T10:28:00.529 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:00.527+0000 7f1afd400640 1 --2- 172.21.15.47:0/3246186942 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1b00074390 0x7f1b00101b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:28:00.529 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:00.527+0000 7f1afde00640 1 --2- 172.21.15.47:0/3246186942 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1b00074390 0x7f1b00101b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:28:00.529 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:00.527+0000 7f1afde00640 1 -- 172.21.15.47:0/3246186942 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b001aa7a0 con 0x7f1b00073a70 2024-03-21T10:28:00.529 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:00.527+0000 7f1afe800640 1 --2- 172.21.15.47:0/3246186942 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1b0010ded0 0x7f1b00102070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T10:28:00.529 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:00.527+0000 7f1afde00640 1 --2- 172.21.15.47:0/3246186942 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1b00073a70 0x7f1b001015f0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f1af002c2d0 tx=0x7f1af002c750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:28:00.529 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:00.527+0000 7f1afd400640 1 --2- 172.21.15.47:0/3246186942 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1b00074390 0x7f1b00101b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T10:28:00.529 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:00.527+0000 7f1af6a00640 1 -- 172.21.15.47:0/3246186942 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1af0042020 con 0x7f1b00073a70 2024-03-21T10:28:00.529 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:00.527+0000 7f1b04c00640 1 -- 172.21.15.47:0/3246186942 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1b001aaa30 con 0x7f1b00073a70 2024-03-21T10:28:00.529 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:00.528+0000 7f1af6a00640 1 -- 172.21.15.47:0/3246186942 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f1af002dc50 con 0x7f1b00073a70 2024-03-21T10:28:00.529 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:00.528+0000 7f1af6a00640 1 -- 172.21.15.47:0/3246186942 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1af00039e0 con 0x7f1b00073a70 2024-03-21T10:28:00.529 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:00.528+0000 7f1b04c00640 1 -- 172.21.15.47:0/3246186942 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1b001aafd0 con 0x7f1b00073a70 2024-03-21T10:28:00.533 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:00.531+0000 7f1af6a00640 1 -- 172.21.15.47:0/3246186942 <== mon.2 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 27) v1 ==== 137985+0+0 (secure 0 0 0) 0x7f1af0045020 con 0x7f1b00073a70 2024-03-21T10:28:00.533 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:00.531+0000 7f1b07999740 1 Processor -- start 2024-03-21T10:28:00.533 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:00.531+0000 7f1b07999740 1 -- start start 2024-03-21T10:28:00.533 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:00.532+0000 7f1b07999740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557b9273fd70 0x557b92740150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:28:00.533 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:00.532+0000 7f1b07999740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557b92740690 0x557b9275a050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:28:00.533 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:00.532+0000 7f1b07999740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x557b9275a590 0x557b9275c980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:28:00.534 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:00.532+0000 7f1b07999740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x557b926a4eb0 con 0x557b9275a590 2024-03-21T10:28:00.534 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:00.532+0000 7f1b07999740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x557b9264da60 con 0x557b92740690 2024-03-21T10:28:00.534 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:00.532+0000 7f1b07999740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x557b926a0f50 con 0x557b9273fd70 2024-03-21T10:28:00.534 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:00.532+0000 7f1afde00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557b92740690 0x557b9275a050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:28:00.534 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:00.533+0000 7f1afe800640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557b9273fd70 0x557b92740150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:28:00.534 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:00.533+0000 7f1afe800640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557b9273fd70 0x557b92740150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.47:40368/0 (socket says 172.21.15.47:40368) 2024-03-21T10:28:00.534 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:00.533+0000 7f1afe800640 1 -- 172.21.15.47:0/2273821711 learned_addr learned my addr 172.21.15.47:0/2273821711 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:28:00.534 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:00.533+0000 7f1afd400640 1 --2- 172.21.15.47:0/2273821711 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x557b9275a590 0x557b9275c980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:28:00.535 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:00.533+0000 7f1af6a00640 1 --2- 172.21.15.47:0/3246186942 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f1ae40a3110 0x7f1ae40a55d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:28:00.535 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:00.533+0000 7f1afde00640 1 -- 172.21.15.47:0/2273821711 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557b9273fd70 msgr2=0x557b92740150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:28:00.536 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:00.533+0000 7f1afde00640 1 --2- 172.21.15.47:0/2273821711 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557b9273fd70 0x557b92740150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:28:00.536 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:00.533+0000 7f1afde00640 1 -- 172.21.15.47:0/2273821711 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x557b9275a590 msgr2=0x557b9275c980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:28:00.536 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:00.533+0000 7f1afde00640 1 --2- 172.21.15.47:0/2273821711 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x557b9275a590 0x557b9275c980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:28:00.536 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:00.534+0000 7f1afde00640 1 -- 172.21.15.47:0/2273821711 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557b924817e0 con 0x557b92740690 2024-03-21T10:28:00.536 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:00.534+0000 7f1afe800640 1 --2- 172.21.15.47:0/3246186942 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f1ae40a3110 0x7f1ae40a55d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:28:00.536 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:00.534+0000 7f1af6a00640 1 -- 172.21.15.47:0/3246186942 <== mon.2 v2:172.21.15.100:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 9474+0+0 (secure 0 0 0) 0x7f1af00f17c0 con 0x7f1b00073a70 2024-03-21T10:28:00.536 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:00.534+0000 7f1afde00640 1 --2- 172.21.15.47:0/2273821711 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557b92740690 0x557b9275a050 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f1af00eec60 tx=0x7f1af004e000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7db117bcf4585cce server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:28:00.536 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:00.535+0000 7f1afe800640 1 --2- 172.21.15.47:0/3246186942 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f1ae40a3110 0x7f1ae40a55d0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f1af80099a0 tx=0x7f1af8004a40 comp rx=0 tx=0).ready entity=mgr.15687 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:28:00.536 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:00.535+0000 7f1aeb000640 1 -- 172.21.15.47:0/2273821711 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1af00bd6d0 con 0x557b92740690 2024-03-21T10:28:00.536 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:00.535+0000 7f1aeb000640 1 -- 172.21.15.47:0/2273821711 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f1af002de00 con 0x557b92740690 2024-03-21T10:28:00.536 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:00.535+0000 7f1aeb000640 1 -- 172.21.15.47:0/2273821711 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1af003a240 con 0x557b92740690 2024-03-21T10:28:00.537 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:00.535+0000 7f1b07999740 1 -- 172.21.15.47:0/2273821711 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557b92740690 msgr2=0x557b9275a050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:28:00.537 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:00.535+0000 7f1b07999740 1 --2- 172.21.15.47:0/2273821711 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557b92740690 0x557b9275a050 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f1af00eec60 tx=0x7f1af004e000 comp rx=0 tx=0).stop 2024-03-21T10:28:00.537 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:00.536+0000 7f1b07999740 1 -- 172.21.15.47:0/2273821711 shutdown_connections 2024-03-21T10:28:00.537 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:00.536+0000 7f1b07999740 1 --2- 172.21.15.47:0/2273821711 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x557b9275a590 0x557b9275c980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:28:00.537 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:00.536+0000 7f1b07999740 1 --2- 172.21.15.47:0/2273821711 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557b92740690 0x557b9275a050 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:28:00.537 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:00.536+0000 7f1b07999740 1 --2- 172.21.15.47:0/2273821711 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557b9273fd70 0x557b92740150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:28:00.537 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:00.536+0000 7f1b07999740 1 -- 172.21.15.47:0/2273821711 >> 172.21.15.47:0/2273821711 conn(0x557b9273a7b0 msgr2=0x557b9273ab90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:28:00.538 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:00.536+0000 7f1b07999740 1 -- 172.21.15.47:0/2273821711 shutdown_connections 2024-03-21T10:28:00.538 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:00.536+0000 7f1b07999740 1 -- 172.21.15.47:0/2273821711 wait complete. 2024-03-21T10:28:00.538 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:00.536+0000 7f1b07999740 1 -- start start 2024-03-21T10:28:00.538 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:00.537+0000 7f1b07999740 1 Processor -- start 2024-03-21T10:28:00.539 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:00.537+0000 7f1b07999740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x557b9273fd70 0x557b92759d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:28:00.539 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:00.537+0000 7f1b07999740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557b92740690 0x557b92757120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:28:00.540 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:00.537+0000 7f1b07999740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557b9275a590 0x557b92757660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:28:00.540 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:00.537+0000 7f1b07999740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x557b9275d140 con 0x557b9273fd70 2024-03-21T10:28:00.540 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:00.537+0000 7f1b07999740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x557b92660eb0 con 0x557b92740690 2024-03-21T10:28:00.540 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:00.537+0000 7f1b07999740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x557b9275d2c0 con 0x557b9275a590 2024-03-21T10:28:00.540 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:00.537+0000 7f1b07999740 10 client.0 _ll_register_callbacks cb 0x557b92655150 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T10:28:00.540 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:00.538+0000 7f1afde00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x557b9273fd70 0x557b92759d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:28:00.540 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:00.538+0000 7f1afde00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x557b9273fd70 0x557b92759d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:57878/0 (socket says 172.21.15.47:57878) 2024-03-21T10:28:00.540 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:00.538+0000 7f1afde00640 1 -- 172.21.15.47:0/2191689172 learned_addr learned my addr 172.21.15.47:0/2191689172 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:28:00.540 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:00.538+0000 7f1afe800640 1 --2- 172.21.15.47:0/2191689172 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557b9275a590 0x557b92757660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:28:00.540 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:00.538+0000 7f1afd400640 1 --2- 172.21.15.47:0/2191689172 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557b92740690 0x557b92757120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:28:00.540 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:00.538+0000 7f1afde00640 1 -- 172.21.15.47:0/2191689172 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557b9275a590 msgr2=0x557b92757660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:28:00.540 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:00.538+0000 7f1afde00640 1 --2- 172.21.15.47:0/2191689172 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557b9275a590 0x557b92757660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:28:00.540 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:00.538+0000 7f1afde00640 1 -- 172.21.15.47:0/2191689172 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557b92740690 msgr2=0x557b92757120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:28:00.540 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:00.538+0000 7f1afde00640 1 --2- 172.21.15.47:0/2191689172 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557b92740690 0x557b92757120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:28:00.540 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:00.538+0000 7f1afde00640 1 -- 172.21.15.47:0/2191689172 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x557b924e1f70 con 0x557b9273fd70 2024-03-21T10:28:00.541 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:00.538+0000 7f1afde00640 1 --2- 172.21.15.47:0/2191689172 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x557b9273fd70 0x557b92759d90 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7f1af00ef150 tx=0x7f1af002bcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5c9618ee1226a603 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:28:00.541 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:00.539+0000 7f1adf400640 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-03-21T10:28:00.541 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:00.539+0000 7f1adf400640 1 -- 172.21.15.47:0/2191689172 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1af003adf0 con 0x557b9273fd70 2024-03-21T10:28:00.541 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:00.539+0000 7f1b07999740 1 -- 172.21.15.47:0/2191689172 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x557b926b23f0 con 0x557b9273fd70 2024-03-21T10:28:00.541 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:00.541+0000 7f1adf400640 1 -- 172.21.15.47:0/2191689172 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(97..97 src has 1..97) v4 ==== 9474+0+0 (secure 0 0 0) 0x7f1af00f7180 con 0x557b9273fd70 2024-03-21T10:28:00.544 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:00.543+0000 7f1adf400640 1 -- 172.21.15.47:0/2191689172 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1af0031cf0 con 0x557b9273fd70 2024-03-21T10:28:00.544 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:00.543+0000 7f1b07999740 10 client.16659 fetch_fsmap learned FSMap version 32 2024-03-21T10:28:00.544 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:00.543+0000 7f1b07999740 1 -- 172.21.15.47:0/2191689172 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x557b92379170 con 0x557b9273fd70 2024-03-21T10:28:00.545 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:00.544+0000 7f1adf400640 1 -- 172.21.15.47:0/2191689172 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x557b926b23f0 con 0x557b9273fd70 2024-03-21T10:28:00.545 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:00.544+0000 7f1adf400640 1 -- 172.21.15.47:0/2191689172 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 32) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f1af00b9f70 con 0x557b9273fd70 2024-03-21T10:28:00.545 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:00.544+0000 7f1b07999740 10 client.16659 fetch_fsmap finished waiting for FSMap version 32 2024-03-21T10:28:00.545 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:00.544+0000 7f1b07999740 10 client.16659 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-21T10:28:00.545 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:00.544+0000 7f1b07999740 20 client.16659 populate_metadata read hostname 'smithi047' 2024-03-21T10:28:00.545 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:00.544+0000 7f1b07999740 1 --2- 172.21.15.47:0/2191689172 >> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] conn(0x557b9276a030 0x557b9276c420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:28:00.545 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:00.544+0000 7f1b07999740 4 client.16659 mds_command: new command op to 34535 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T10:28:00.545 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:00.544+0000 7f1b07999740 1 -- 172.21.15.47:0/2191689172 --> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x557b926ac740 con 0x557b9276a030 2024-03-21T10:28:00.546 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:00.545+0000 7f1afd400640 1 --2- 172.21.15.47:0/2191689172 >> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] conn(0x557b9276a030 0x557b9276c420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:28:00.546 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:00.545+0000 7f1afd400640 1 --2- 172.21.15.47:0/2191689172 >> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] conn(0x557b9276a030 0x557b9276c420 crc :-1 s=READY pgs=103 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9da64fe72cf8fcac server_cookie=c5907fcf8d2f7b9f in_seq=0 out_seq=0 2024-03-21T10:28:00.546 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:00.545+0000 7f1adf400640 10 client.16659 ms_handle_connect on v2:172.21.15.100:6838/4173168440 2024-03-21T10:28:00.548 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:00.546+0000 7f1adf400640 1 -- 172.21.15.47:0/2191689172 <== mds.0 v2:172.21.15.100:6838/4173168440 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x557b926ac740 con 0x557b9276a030 2024-03-21T10:28:00.548 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:00.546+0000 7f1adf400640 10 client.16659 handle_command_reply: tid=0 2024-03-21T10:28:00.548 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:00.547+0000 7f1b07999740 1 client.16659 shutdown 2024-03-21T10:28:00.548 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:00.547+0000 7f1b07999740 1 -- 172.21.15.47:0/2191689172 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x557b9273fd70 msgr2=0x557b92759d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:28:00.548 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:00.547+0000 7f1b07999740 1 --2- 172.21.15.47:0/2191689172 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x557b9273fd70 0x557b92759d90 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7f1af00ef150 tx=0x7f1af002bcc0 comp rx=0 tx=0).stop 2024-03-21T10:28:00.548 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:00.547+0000 7f1b07999740 1 -- 172.21.15.47:0/2191689172 shutdown_connections 2024-03-21T10:28:00.548 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:00.547+0000 7f1b07999740 1 --2- 172.21.15.47:0/2191689172 >> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] conn(0x557b9276a030 0x557b9276c420 crc :-1 s=READY pgs=103 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:28:00.548 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:00.547+0000 7f1b07999740 1 --2- 172.21.15.47:0/2191689172 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557b9275a590 0x557b92757660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:28:00.548 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:00.547+0000 7f1b07999740 1 --2- 172.21.15.47:0/2191689172 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557b92740690 0x557b92757120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:28:00.549 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:00.547+0000 7f1b07999740 1 --2- 172.21.15.47:0/2191689172 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x557b9273fd70 0x557b92759d90 unknown :-1 s=CLOSED pgs=730 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:28:00.549 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:00.547+0000 7f1b07999740 1 -- 172.21.15.47:0/2191689172 >> 172.21.15.47:0/2191689172 conn(0x557b9273a7b0 msgr2=0x557b9273c500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:28:00.549 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:00.547+0000 7f1adf400640 0 client.16659 ms_handle_reset on v2:172.21.15.100:6838/4173168440 2024-03-21T10:28:00.549 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:00.547+0000 7f1b07999740 1 -- 172.21.15.47:0/2191689172 shutdown_connections 2024-03-21T10:28:00.549 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:00.547+0000 7f1b07999740 1 -- 172.21.15.47:0/2191689172 wait complete. 2024-03-21T10:28:00.549 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:00.547+0000 7f1b07999740 20 client.16659 trim_cache size 0 max 16384 2024-03-21T10:28:00.574 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:00.573+0000 7f1b07999740 1 Processor -- start 2024-03-21T10:28:00.574 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:00.573+0000 7f1b07999740 1 -- start start 2024-03-21T10:28:00.574 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:00.573+0000 7f1b07999740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x557b9273fd70 0x557b92752660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:28:00.574 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:00.573+0000 7f1b07999740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557b92740690 0x557b927396b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:28:00.574 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:00.573+0000 7f1b07999740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557b9275a590 0x557b927659d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:28:00.574 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:00.573+0000 7f1b07999740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b001ac790 con 0x557b9273fd70 2024-03-21T10:28:00.574 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:00.573+0000 7f1b07999740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b001ae110 con 0x557b92740690 2024-03-21T10:28:00.574 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:00.573+0000 7f1b07999740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b001acdf0 con 0x557b9275a590 2024-03-21T10:28:00.575 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:00.574+0000 7f1afd400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557b92740690 0x557b927396b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:28:00.575 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:00.574+0000 7f1afd400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557b92740690 0x557b927396b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:32968/0 (socket says 172.21.15.47:32968) 2024-03-21T10:28:00.575 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:00.574+0000 7f1afd400640 1 -- 172.21.15.47:0/2091355428 learned_addr learned my addr 172.21.15.47:0/2091355428 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:28:00.575 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:00.574+0000 7f1afde00640 1 --2- 172.21.15.47:0/2091355428 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x557b9273fd70 0x557b92752660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:28:00.575 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:00.574+0000 7f1afd400640 1 -- 172.21.15.47:0/2091355428 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557b9275a590 msgr2=0x557b927659d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:28:00.575 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:00.574+0000 7f1afd400640 1 --2- 172.21.15.47:0/2091355428 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557b9275a590 0x557b927659d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:28:00.576 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:00.574+0000 7f1afd400640 1 -- 172.21.15.47:0/2091355428 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x557b9273fd70 msgr2=0x557b92752660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:28:00.576 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:00.574+0000 7f1afd400640 1 --2- 172.21.15.47:0/2091355428 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x557b9273fd70 0x557b92752660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:28:00.576 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:00.574+0000 7f1afd400640 1 -- 172.21.15.47:0/2091355428 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557b9275d0e0 con 0x557b92740690 2024-03-21T10:28:00.576 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:00.575+0000 7f1afd400640 1 --2- 172.21.15.47:0/2091355428 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557b92740690 0x557b927396b0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f1b00102db0 tx=0x7f1aec0094a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9e0f63b11079137c server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:28:00.577 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:00.575+0000 7f1aea600640 1 -- 172.21.15.47:0/2091355428 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1aec005230 con 0x557b92740690 2024-03-21T10:28:00.577 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:00.575+0000 7f1aea600640 1 -- 172.21.15.47:0/2091355428 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f1aec0053d0 con 0x557b92740690 2024-03-21T10:28:00.577 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:00.575+0000 7f1aea600640 1 -- 172.21.15.47:0/2091355428 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1aec012910 con 0x557b92740690 2024-03-21T10:28:00.577 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:00.576+0000 7f1b07999740 1 -- 172.21.15.47:0/2091355428 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557b92740690 msgr2=0x557b927396b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:28:00.577 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:00.576+0000 7f1b07999740 1 --2- 172.21.15.47:0/2091355428 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557b92740690 0x557b927396b0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f1b00102db0 tx=0x7f1aec0094a0 comp rx=0 tx=0).stop 2024-03-21T10:28:00.577 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:00.576+0000 7f1b07999740 1 -- 172.21.15.47:0/2091355428 shutdown_connections 2024-03-21T10:28:00.577 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:00.576+0000 7f1b07999740 1 --2- 172.21.15.47:0/2091355428 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557b9275a590 0x557b927659d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:28:00.577 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:00.576+0000 7f1b07999740 1 --2- 172.21.15.47:0/2091355428 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557b92740690 0x557b927396b0 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:28:00.577 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:00.576+0000 7f1b07999740 1 --2- 172.21.15.47:0/2091355428 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x557b9273fd70 0x557b92752660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:28:00.577 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:00.576+0000 7f1b07999740 1 -- 172.21.15.47:0/2091355428 >> 172.21.15.47:0/2091355428 conn(0x557b9273a7b0 msgr2=0x557b92738fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:28:00.578 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:00.576+0000 7f1b07999740 1 -- 172.21.15.47:0/2091355428 shutdown_connections 2024-03-21T10:28:00.579 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:00.576+0000 7f1b07999740 1 -- 172.21.15.47:0/2091355428 wait complete. 2024-03-21T10:28:00.579 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:00.577+0000 7f1b07999740 1 -- start start 2024-03-21T10:28:00.579 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:00.577+0000 7f1b07999740 1 Processor -- start 2024-03-21T10:28:00.579 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:00.577+0000 7f1b07999740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557b9273fd70 0x557b92754da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:28:00.579 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:00.578+0000 7f1b07999740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557b92740690 0x557b9275f4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:28:00.579 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:00.578+0000 7f1b07999740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x557b9275a590 0x557b9275fa20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:28:00.579 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:00.578+0000 7f1b07999740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b001adab0 con 0x557b9275a590 2024-03-21T10:28:00.579 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:00.578+0000 7f1b07999740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b001ad450 con 0x557b92740690 2024-03-21T10:28:00.579 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:00.578+0000 7f1b07999740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b001a1d60 con 0x557b9273fd70 2024-03-21T10:28:00.579 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:00.578+0000 7f1b07999740 10 client.0 _ll_register_callbacks cb 0x7f1aec009340 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T10:28:00.579 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:00.578+0000 7f1afd400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557b92740690 0x557b9275f4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:28:00.579 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:00.578+0000 7f1afd400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557b92740690 0x557b9275f4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:32984/0 (socket says 172.21.15.47:32984) 2024-03-21T10:28:00.579 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:00.578+0000 7f1afd400640 1 -- 172.21.15.47:0/1522639698 learned_addr learned my addr 172.21.15.47:0/1522639698 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:28:00.579 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:00.578+0000 7f1afe800640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x557b9275a590 0x557b9275fa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:28:00.580 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:00.578+0000 7f1afde00640 1 --2- 172.21.15.47:0/1522639698 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557b9273fd70 0x557b92754da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:28:00.581 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:00.578+0000 7f1afd400640 1 -- 172.21.15.47:0/1522639698 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557b9273fd70 msgr2=0x557b92754da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:28:00.581 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:00.578+0000 7f1afd400640 1 --2- 172.21.15.47:0/1522639698 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557b9273fd70 0x557b92754da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:28:00.581 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:00.578+0000 7f1afd400640 1 -- 172.21.15.47:0/1522639698 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x557b9275a590 msgr2=0x557b9275fa20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:28:00.581 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:00.578+0000 7f1afd400640 1 --2- 172.21.15.47:0/1522639698 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x557b9275a590 0x557b9275fa20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:28:00.581 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:00.578+0000 7f1afd400640 1 -- 172.21.15.47:0/1522639698 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x557b92760160 con 0x557b92740690 2024-03-21T10:28:00.581 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:00.578+0000 7f1afe800640 1 --2- 172.21.15.47:0/1522639698 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x557b9275a590 0x557b9275fa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T10:28:00.581 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:00.579+0000 7f1afd400640 1 --2- 172.21.15.47:0/1522639698 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557b92740690 0x557b9275f4e0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f1aec005c20 tx=0x7f1aec0092a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1bc7dc18cd66c860 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:28:00.581 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:00.579+0000 7f1adf400640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-21T10:28:00.581 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:00.579+0000 7f1adf400640 1 -- 172.21.15.47:0/1522639698 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1aec0050b0 con 0x557b92740690 2024-03-21T10:28:00.581 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:00.579+0000 7f1adf400640 1 -- 172.21.15.47:0/1522639698 <== mon.2 v2:172.21.15.100:3300/0 2 ==== osd_map(97..97 src has 1..97) v4 ==== 9474+0+0 (secure 0 0 0) 0x7f1aec004730 con 0x557b92740690 2024-03-21T10:28:00.581 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:00.579+0000 7f1b07999740 1 -- 172.21.15.47:0/1522639698 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f1aec0053d0 con 0x557b92740690 2024-03-21T10:28:00.582 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:00.579+0000 7f1adf400640 1 -- 172.21.15.47:0/1522639698 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1aec037050 con 0x557b92740690 2024-03-21T10:28:00.582 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:00.580+0000 7f1b07999740 10 client.36464 fetch_fsmap learned FSMap version 32 2024-03-21T10:28:00.582 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:00.580+0000 7f1b07999740 1 -- 172.21.15.47:0/1522639698 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x557b927c6830 con 0x557b92740690 2024-03-21T10:28:00.582 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:00.580+0000 7f1adf400640 1 -- 172.21.15.47:0/1522639698 <== mon.2 v2:172.21.15.100:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f1aec0053d0 con 0x557b92740690 2024-03-21T10:28:00.582 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:00.581+0000 7f1adf400640 1 -- 172.21.15.47:0/1522639698 <== mon.2 v2:172.21.15.100:3300/0 5 ==== fsmap(e 32) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f1aec02c020 con 0x557b92740690 2024-03-21T10:28:00.583 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:00.581+0000 7f1b07999740 10 client.36464 fetch_fsmap finished waiting for FSMap version 32 2024-03-21T10:28:00.583 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:00.581+0000 7f1b07999740 10 client.36464 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-21T10:28:00.583 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:00.581+0000 7f1b07999740 20 client.36464 populate_metadata read hostname 'smithi047' 2024-03-21T10:28:00.583 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:00.581+0000 7f1b07999740 1 --2- 172.21.15.47:0/1522639698 >> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] conn(0x557b927c6e00 0x557b927c9220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:28:00.583 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:00.581+0000 7f1b07999740 4 client.36464 mds_command: new command op to 34535 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-03-21T10:28:00.583 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:00.581+0000 7f1b07999740 1 -- 172.21.15.47:0/1522639698 --> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x557b9262aa40 con 0x557b927c6e00 2024-03-21T10:28:00.583 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:00.582+0000 7f1afde00640 1 --2- 172.21.15.47:0/1522639698 >> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] conn(0x557b927c6e00 0x557b927c9220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:28:00.583 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:00.582+0000 7f1afde00640 1 --2- 172.21.15.47:0/1522639698 >> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] conn(0x557b927c6e00 0x557b927c9220 crc :-1 s=READY pgs=104 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=de69139b0f6bb214 server_cookie=71afd1dd5538bf63 in_seq=0 out_seq=0 2024-03-21T10:28:00.583 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:00.582+0000 7f1adf400640 10 client.36464 ms_handle_connect on v2:172.21.15.100:6838/4173168440 2024-03-21T10:28:00.585 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:00.584+0000 7f1adf400640 1 -- 172.21.15.47:0/1522639698 <== mds.0 v2:172.21.15.100:6838/4173168440 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x557b9262aa40 con 0x557b927c6e00 2024-03-21T10:28:00.585 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:00.584+0000 7f1adf400640 10 client.36464 handle_command_reply: tid=0 2024-03-21T10:28:00.586 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:00.584+0000 7f1b07999740 1 client.36464 shutdown 2024-03-21T10:28:00.586 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:00.584+0000 7f1b07999740 1 -- 172.21.15.47:0/1522639698 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557b92740690 msgr2=0x557b9275f4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:28:00.586 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:00.584+0000 7f1b07999740 1 --2- 172.21.15.47:0/1522639698 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557b92740690 0x557b9275f4e0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f1aec005c20 tx=0x7f1aec0092a0 comp rx=0 tx=0).stop 2024-03-21T10:28:00.586 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:00.584+0000 7f1b07999740 1 -- 172.21.15.47:0/1522639698 shutdown_connections 2024-03-21T10:28:00.586 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:00.584+0000 7f1b07999740 1 --2- 172.21.15.47:0/1522639698 >> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] conn(0x557b927c6e00 0x557b927c9220 crc :-1 s=READY pgs=104 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:28:00.586 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:00.584+0000 7f1adf400640 0 client.36464 ms_handle_reset on v2:172.21.15.100:6838/4173168440 2024-03-21T10:28:00.586 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:00.584+0000 7f1b07999740 1 --2- 172.21.15.47:0/1522639698 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x557b9275a590 0x557b9275fa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:28:00.586 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:00.584+0000 7f1b07999740 1 --2- 172.21.15.47:0/1522639698 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557b92740690 0x557b9275f4e0 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:28:00.586 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:00.584+0000 7f1b07999740 1 --2- 172.21.15.47:0/1522639698 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557b9273fd70 0x557b92754da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:28:00.586 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:00.584+0000 7f1b07999740 1 -- 172.21.15.47:0/1522639698 >> 172.21.15.47:0/1522639698 conn(0x557b9273a7b0 msgr2=0x557b92739830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:28:00.586 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:00.585+0000 7f1b07999740 1 -- 172.21.15.47:0/1522639698 shutdown_connections 2024-03-21T10:28:00.587 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:00.585+0000 7f1b07999740 1 -- 172.21.15.47:0/1522639698 wait complete. 2024-03-21T10:28:00.587 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:00.585+0000 7f1b07999740 20 client.36464 trim_cache size 0 max 16384 2024-03-21T10:28:00.587 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-03-21T10:28:00.587 INFO:teuthology.orchestra.run.smithi047.stdout: "return_code": 0, 2024-03-21T10:28:00.587 INFO:teuthology.orchestra.run.smithi047.stdout: "scrub_tag": "889f24b1-253a-4150-9658-163b715897dc", 2024-03-21T10:28:00.587 INFO:teuthology.orchestra.run.smithi047.stdout: "mode": "asynchronous" 2024-03-21T10:28:00.587 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-03-21T10:28:00.588 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:00.586+0000 7f1b04c00640 1 -- 172.21.15.47:0/3246186942 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f1ae40a3110 msgr2=0x7f1ae40a55d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:28:00.588 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:00.586+0000 7f1b04c00640 1 --2- 172.21.15.47:0/3246186942 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f1ae40a3110 0x7f1ae40a55d0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f1af80099a0 tx=0x7f1af8004a40 comp rx=0 tx=0).stop 2024-03-21T10:28:00.588 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:00.586+0000 7f1b04c00640 1 -- 172.21.15.47:0/3246186942 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1b00073a70 msgr2=0x7f1b001015f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:28:00.588 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:00.586+0000 7f1b04c00640 1 --2- 172.21.15.47:0/3246186942 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1b00073a70 0x7f1b001015f0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f1af002c2d0 tx=0x7f1af002c750 comp rx=0 tx=0).stop 2024-03-21T10:28:00.588 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:00.586+0000 7f1b04c00640 1 -- 172.21.15.47:0/3246186942 shutdown_connections 2024-03-21T10:28:00.588 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:00.586+0000 7f1b04c00640 1 --2- 172.21.15.47:0/3246186942 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1b0010ded0 0x7f1b00102070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:28:00.588 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:00.586+0000 7f1b04c00640 1 --2- 172.21.15.47:0/3246186942 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1b00074390 0x7f1b00101b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:28:00.589 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:00.586+0000 7f1b04c00640 1 --2- 172.21.15.47:0/3246186942 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f1ae40a3110 0x7f1ae40a55d0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:28:00.589 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:00.586+0000 7f1b04c00640 1 --2- 172.21.15.47:0/3246186942 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1b00073a70 0x7f1b001015f0 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:28:00.589 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:00.586+0000 7f1b04c00640 1 -- 172.21.15.47:0/3246186942 >> 172.21.15.47:0/3246186942 conn(0x7f1b00078aa0 msgr2=0x7f1b000fe620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:28:00.589 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:00.587+0000 7f1b04c00640 1 -- 172.21.15.47:0/3246186942 shutdown_connections 2024-03-21T10:28:00.589 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:00.587+0000 7f1b04c00640 1 -- 172.21.15.47:0/3246186942 wait complete. 2024-03-21T10:28:00.608 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-21T10:28:00.608 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-21T10:28:00.815 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:00.814+0000 7fef19c00640 1 -- 172.21.15.47:0/2543146724 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fef1410cf80 msgr2=0x7fef14114b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:28:00.816 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:00.814+0000 7fef19c00640 1 --2- 172.21.15.47:0/2543146724 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fef1410cf80 0x7fef14114b50 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7fef0c009ca0 tx=0x7fef0c02c0f0 comp rx=0 tx=0).stop 2024-03-21T10:28:00.816 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:00.814+0000 7fef19c00640 1 -- 172.21.15.47:0/2543146724 shutdown_connections 2024-03-21T10:28:00.816 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:00.814+0000 7fef19c00640 1 --2- 172.21.15.47:0/2543146724 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fef1410cf80 0x7fef14114b50 unknown :-1 s=CLOSED pgs=731 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:28:00.816 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:00.814+0000 7fef19c00640 1 --2- 172.21.15.47:0/2543146724 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fef14108860 0x7fef1410c8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:28:00.816 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:00.814+0000 7fef19c00640 1 --2- 172.21.15.47:0/2543146724 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fef14107eb0 0x7fef14108290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:28:00.816 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:00.814+0000 7fef19c00640 1 -- 172.21.15.47:0/2543146724 >> 172.21.15.47:0/2543146724 conn(0x7fef14075be0 msgr2=0x7fef14078000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:28:00.816 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:00.814+0000 7fef19c00640 1 -- 172.21.15.47:0/2543146724 shutdown_connections 2024-03-21T10:28:00.816 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:00.814+0000 7fef19c00640 1 -- 172.21.15.47:0/2543146724 wait complete. 2024-03-21T10:28:00.818 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:00.815+0000 7fef19c00640 1 Processor -- start 2024-03-21T10:28:00.818 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:00.815+0000 7fef19c00640 1 -- start start 2024-03-21T10:28:00.818 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:00.816+0000 7fef19c00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fef14107eb0 0x7fef1419e460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:28:00.818 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:00.816+0000 7fef19c00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fef1410cf80 0x7fef1419e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:28:00.818 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:00.816+0000 7fef19c00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fef141a2d20 0x7fef141a31d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:28:00.818 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:00.816+0000 7fef19c00640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef14117ee0 con 0x7fef14107eb0 2024-03-21T10:28:00.818 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:00.816+0000 7fef19c00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef14117d60 con 0x7fef1410cf80 2024-03-21T10:28:00.819 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:00.816+0000 7fef19c00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef14118060 con 0x7fef141a2d20 2024-03-21T10:28:00.819 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:00.816+0000 7fef12a00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fef14107eb0 0x7fef1419e460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:28:00.819 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:00.816+0000 7fef12a00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fef14107eb0 0x7fef1419e460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:57930/0 (socket says 172.21.15.47:57930) 2024-03-21T10:28:00.819 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:00.816+0000 7fef12a00640 1 -- 172.21.15.47:0/44124501 learned_addr learned my addr 172.21.15.47:0/44124501 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:28:00.819 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:00.816+0000 7fef0be00640 1 --2- 172.21.15.47:0/44124501 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fef1410cf80 0x7fef1419e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:28:00.819 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:00.816+0000 7fef13400640 1 --2- 172.21.15.47:0/44124501 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fef141a2d20 0x7fef141a31d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:28:00.819 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:00.817+0000 7fef12a00640 1 -- 172.21.15.47:0/44124501 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fef141a2d20 msgr2=0x7fef141a31d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:28:00.819 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:00.817+0000 7fef12a00640 1 --2- 172.21.15.47:0/44124501 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fef141a2d20 0x7fef141a31d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:28:00.819 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:00.817+0000 7fef12a00640 1 -- 172.21.15.47:0/44124501 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fef1410cf80 msgr2=0x7fef1419e9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:28:00.819 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:00.817+0000 7fef12a00640 1 --2- 172.21.15.47:0/44124501 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fef1410cf80 0x7fef1419e9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:28:00.819 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:00.817+0000 7fef12a00640 1 -- 172.21.15.47:0/44124501 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef141a3810 con 0x7fef14107eb0 2024-03-21T10:28:00.819 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:00.817+0000 7fef0be00640 1 --2- 172.21.15.47:0/44124501 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fef1410cf80 0x7fef1419e9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T10:28:00.819 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:00.817+0000 7fef12a00640 1 --2- 172.21.15.47:0/44124501 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fef14107eb0 0x7fef1419e460 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7fef0400ea60 tx=0x7fef0400ef20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:28:00.820 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:00.817+0000 7fef0b400640 1 -- 172.21.15.47:0/44124501 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef0400cde0 con 0x7fef14107eb0 2024-03-21T10:28:00.820 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:00.817+0000 7fef19c00640 1 -- 172.21.15.47:0/44124501 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fef141a3b00 con 0x7fef14107eb0 2024-03-21T10:28:00.820 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:00.817+0000 7fef0b400640 1 -- 172.21.15.47:0/44124501 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fef040035f0 con 0x7fef14107eb0 2024-03-21T10:28:00.820 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:00.817+0000 7fef0b400640 1 -- 172.21.15.47:0/44124501 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef04010690 con 0x7fef14107eb0 2024-03-21T10:28:00.820 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:00.817+0000 7fef19c00640 1 -- 172.21.15.47:0/44124501 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fef141ab260 con 0x7fef14107eb0 2024-03-21T10:28:00.822 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:00.821+0000 7fef0b400640 1 -- 172.21.15.47:0/44124501 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 27) v1 ==== 137985+0+0 (secure 0 0 0) 0x7fef040037a0 con 0x7fef14107eb0 2024-03-21T10:28:00.822 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:00.821+0000 7fef1c97f740 1 Processor -- start 2024-03-21T10:28:00.822 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:00.822+0000 7fef1c97f740 1 -- start start 2024-03-21T10:28:00.823 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:00.822+0000 7fef1c97f740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c100290c00 0x55c10028f950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:28:00.823 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:00.822+0000 7fef1c97f740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c10028ff20 0x55c1002903a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:28:00.823 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:00.822+0000 7fef1c97f740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c100299290 0x55c10029b680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:28:00.823 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:00.822+0000 7fef1c97f740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c1001f4250 con 0x55c10028ff20 2024-03-21T10:28:00.824 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:00.822+0000 7fef1c97f740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c10019ca30 con 0x55c100290c00 2024-03-21T10:28:00.824 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:00.822+0000 7fef1c97f740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c1001f02f0 con 0x55c100299290 2024-03-21T10:28:00.824 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:00.822+0000 7fef12a00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c10028ff20 0x55c1002903a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:28:00.824 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:00.822+0000 7fef13400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c100290c00 0x55c10028f950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:28:00.824 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:00.822+0000 7fef12a00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c10028ff20 0x55c1002903a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:57942/0 (socket says 172.21.15.47:57942) 2024-03-21T10:28:00.824 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:00.822+0000 7fef12a00640 1 -- 172.21.15.47:0/4193563124 learned_addr learned my addr 172.21.15.47:0/4193563124 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:28:00.824 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:00.823+0000 7fef0be00640 1 --2- 172.21.15.47:0/4193563124 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c100299290 0x55c10029b680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:28:00.825 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:00.823+0000 7fef12a00640 1 -- 172.21.15.47:0/4193563124 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c100299290 msgr2=0x55c10029b680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:28:00.825 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:00.823+0000 7fef12a00640 1 --2- 172.21.15.47:0/4193563124 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c100299290 0x55c10029b680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:28:00.825 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:00.823+0000 7fef12a00640 1 -- 172.21.15.47:0/4193563124 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c100290c00 msgr2=0x55c10028f950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:28:00.825 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:00.823+0000 7fef12a00640 1 --2- 172.21.15.47:0/4193563124 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c100290c00 0x55c10028f950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:28:00.825 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:00.823+0000 7fef12a00640 1 -- 172.21.15.47:0/4193563124 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c100187690 con 0x55c10028ff20 2024-03-21T10:28:00.825 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:00.823+0000 7fef0b400640 1 --2- 172.21.15.47:0/44124501 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7feef80a3090 0x7feef80a5550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:28:00.825 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:00.823+0000 7fef12a00640 1 --2- 172.21.15.47:0/4193563124 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c10028ff20 0x55c1002903a0 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7fef040cd1a0 tx=0x7fef040d84b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=46324bd307583298 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:28:00.825 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:00.824+0000 7fef0b400640 1 -- 172.21.15.47:0/44124501 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 9474+0+0 (secure 0 0 0) 0x7fef040cff30 con 0x7fef14107eb0 2024-03-21T10:28:00.826 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:00.824+0000 7fef13400640 1 --2- 172.21.15.47:0/44124501 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7feef80a3090 0x7feef80a5550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:28:00.826 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:00.824+0000 7fef03e00640 1 -- 172.21.15.47:0/4193563124 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef040981a0 con 0x55c10028ff20 2024-03-21T10:28:00.826 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:00.824+0000 7fef03e00640 1 -- 172.21.15.47:0/4193563124 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fef04003b30 con 0x55c10028ff20 2024-03-21T10:28:00.826 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:00.824+0000 7fef03e00640 1 -- 172.21.15.47:0/4193563124 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef0409b860 con 0x55c10028ff20 2024-03-21T10:28:00.827 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:00.825+0000 7fef1c97f740 1 -- 172.21.15.47:0/4193563124 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c10028ff20 msgr2=0x55c1002903a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:28:00.827 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:00.825+0000 7fef13400640 1 --2- 172.21.15.47:0/44124501 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7feef80a3090 0x7feef80a5550 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fef0c006010 tx=0x7fef0c004a40 comp rx=0 tx=0).ready entity=mgr.15687 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:28:00.827 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:00.825+0000 7fef1c97f740 1 --2- 172.21.15.47:0/4193563124 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c10028ff20 0x55c1002903a0 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7fef040cd1a0 tx=0x7fef040d84b0 comp rx=0 tx=0).stop 2024-03-21T10:28:00.827 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:00.825+0000 7fef1c97f740 1 -- 172.21.15.47:0/4193563124 shutdown_connections 2024-03-21T10:28:00.827 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:00.825+0000 7fef1c97f740 1 --2- 172.21.15.47:0/4193563124 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c100299290 0x55c10029b680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:28:00.827 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:00.825+0000 7fef1c97f740 1 --2- 172.21.15.47:0/4193563124 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c10028ff20 0x55c1002903a0 unknown :-1 s=CLOSED pgs=733 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:28:00.827 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:00.825+0000 7fef1c97f740 1 --2- 172.21.15.47:0/4193563124 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c100290c00 0x55c10028f950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:28:00.827 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:00.825+0000 7fef1c97f740 1 -- 172.21.15.47:0/4193563124 >> 172.21.15.47:0/4193563124 conn(0x55c100257200 msgr2=0x55c10028a970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:28:00.827 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:00.825+0000 7fef1c97f740 1 -- 172.21.15.47:0/4193563124 shutdown_connections 2024-03-21T10:28:00.827 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:00.825+0000 7fef1c97f740 1 -- 172.21.15.47:0/4193563124 wait complete. 2024-03-21T10:28:00.827 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:00.826+0000 7fef1c97f740 1 -- start start 2024-03-21T10:28:00.827 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:00.826+0000 7fef1c97f740 1 Processor -- start 2024-03-21T10:28:00.829 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:00.826+0000 7fef1c97f740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c100290c00 0x55c100298a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:28:00.829 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:00.827+0000 7fef1c97f740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c100299290 0x55c100295d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:28:00.829 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:00.827+0000 7fef1c97f740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c1002962c0 0x55c100296740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:28:00.829 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:00.827+0000 7fef1c97f740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c10029c100 con 0x55c1002962c0 2024-03-21T10:28:00.830 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:00.827+0000 7fef1c97f740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c1001afe80 con 0x55c100290c00 2024-03-21T10:28:00.830 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:00.827+0000 7fef1c97f740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c10029c280 con 0x55c100299290 2024-03-21T10:28:00.830 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:00.827+0000 7fef1c97f740 10 client.0 _ll_register_callbacks cb 0x55c1001c2130 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T10:28:00.830 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:00.827+0000 7fef12a00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c100290c00 0x55c100298a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:28:00.830 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:00.827+0000 7fef13400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c1002962c0 0x55c100296740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:28:00.830 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:00.827+0000 7fef12a00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c100290c00 0x55c100298a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:33024/0 (socket says 172.21.15.47:33024) 2024-03-21T10:28:00.830 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:00.827+0000 7fef12a00640 1 -- 172.21.15.47:0/356466884 learned_addr learned my addr 172.21.15.47:0/356466884 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:28:00.830 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:00.827+0000 7fef12a00640 1 -- 172.21.15.47:0/356466884 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c100299290 msgr2=0x55c100295d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:28:00.830 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:00.827+0000 7fef12a00640 1 --2- 172.21.15.47:0/356466884 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c100299290 0x55c100295d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:28:00.830 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:00.827+0000 7fef12a00640 1 -- 172.21.15.47:0/356466884 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c1002962c0 msgr2=0x55c100296740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:28:00.830 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:00.828+0000 7fef12a00640 1 --2- 172.21.15.47:0/356466884 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c1002962c0 0x55c100296740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:28:00.830 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:00.828+0000 7fef12a00640 1 -- 172.21.15.47:0/356466884 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55c0fffd27d0 con 0x55c100290c00 2024-03-21T10:28:00.830 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:00.828+0000 7fef13400640 1 --2- 172.21.15.47:0/356466884 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c1002962c0 0x55c100296740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T10:28:00.830 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:00.828+0000 7fef12a00640 1 --2- 172.21.15.47:0/356466884 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c100290c00 0x55c100298a90 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fef0400efd0 tx=0x7fef0409c670 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bdc220ba30dfb53f server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:28:00.831 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:00.828+0000 7fef00c00640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-21T10:28:00.831 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:00.828+0000 7fef00c00640 1 -- 172.21.15.47:0/356466884 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef0409ba40 con 0x55c100290c00 2024-03-21T10:28:00.831 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:00.828+0000 7fef00c00640 1 -- 172.21.15.47:0/356466884 <== mon.2 v2:172.21.15.100:3300/0 2 ==== osd_map(97..97 src has 1..97) v4 ==== 9474+0+0 (secure 0 0 0) 0x7fef040cd4e0 con 0x55c100290c00 2024-03-21T10:28:00.831 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:00.828+0000 7fef1c97f740 1 -- 172.21.15.47:0/356466884 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55c100201790 con 0x55c100290c00 2024-03-21T10:28:00.831 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:00.828+0000 7fef00c00640 1 -- 172.21.15.47:0/356466884 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7fef0409bcd0 con 0x55c100290c00 2024-03-21T10:28:00.831 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:00.828+0000 7fef1c97f740 10 client.36476 fetch_fsmap learned FSMap version 32 2024-03-21T10:28:00.831 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:00.828+0000 7fef1c97f740 1 -- 172.21.15.47:0/356466884 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x55c0fff083f0 con 0x55c100290c00 2024-03-21T10:28:00.831 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:00.829+0000 7fef00c00640 1 -- 172.21.15.47:0/356466884 <== mon.2 v2:172.21.15.100:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x55c100201790 con 0x55c100290c00 2024-03-21T10:28:00.833 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:00.829+0000 7fef00c00640 1 -- 172.21.15.47:0/356466884 <== mon.2 v2:172.21.15.100:3300/0 5 ==== fsmap(e 32) v1 ==== 7942+0+0 (secure 0 0 0) 0x7fef040d8c20 con 0x55c100290c00 2024-03-21T10:28:00.833 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:00.829+0000 7fef1c97f740 10 client.36476 fetch_fsmap finished waiting for FSMap version 32 2024-03-21T10:28:00.833 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:00.829+0000 7fef1c97f740 10 client.36476 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-21T10:28:00.833 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:00.829+0000 7fef1c97f740 20 client.36476 populate_metadata read hostname 'smithi047' 2024-03-21T10:28:00.833 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:00.829+0000 7fef1c97f740 1 --2- 172.21.15.47:0/356466884 >> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] conn(0x55c1002a9540 0x55c1002ab930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:28:00.833 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:00.829+0000 7fef1c97f740 4 client.36476 mds_command: new command op to 34535 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T10:28:00.833 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:00.829+0000 7fef1c97f740 1 -- 172.21.15.47:0/356466884 --> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55c1001fbae0 con 0x55c1002a9540 2024-03-21T10:28:00.833 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:00.829+0000 7fef0be00640 1 --2- 172.21.15.47:0/356466884 >> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] conn(0x55c1002a9540 0x55c1002ab930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:28:00.833 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:00.830+0000 7fef0be00640 1 --2- 172.21.15.47:0/356466884 >> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] conn(0x55c1002a9540 0x55c1002ab930 crc :-1 s=READY pgs=105 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a5323d857603cfbb server_cookie=9c126727081d31e8 in_seq=0 out_seq=0 2024-03-21T10:28:00.833 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:00.830+0000 7fef00c00640 10 client.36476 ms_handle_connect on v2:172.21.15.100:6838/4173168440 2024-03-21T10:28:00.834 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:00.831+0000 7fef00c00640 1 -- 172.21.15.47:0/356466884 <== mds.0 v2:172.21.15.100:6838/4173168440 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55c1001fbae0 con 0x55c1002a9540 2024-03-21T10:28:00.834 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:00.831+0000 7fef00c00640 10 client.36476 handle_command_reply: tid=0 2024-03-21T10:28:00.834 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:00.831+0000 7fef1c97f740 1 client.36476 shutdown 2024-03-21T10:28:00.834 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:00.831+0000 7fef1c97f740 1 -- 172.21.15.47:0/356466884 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c100290c00 msgr2=0x55c100298a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:28:00.834 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:00.831+0000 7fef1c97f740 1 --2- 172.21.15.47:0/356466884 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c100290c00 0x55c100298a90 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fef0400efd0 tx=0x7fef0409c670 comp rx=0 tx=0).stop 2024-03-21T10:28:00.834 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:00.831+0000 7fef1c97f740 1 -- 172.21.15.47:0/356466884 shutdown_connections 2024-03-21T10:28:00.834 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:00.831+0000 7fef1c97f740 1 --2- 172.21.15.47:0/356466884 >> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] conn(0x55c1002a9540 0x55c1002ab930 crc :-1 s=READY pgs=105 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:28:00.834 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:00.831+0000 7fef1c97f740 1 --2- 172.21.15.47:0/356466884 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c1002962c0 0x55c100296740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:28:00.834 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:00.831+0000 7fef1c97f740 1 --2- 172.21.15.47:0/356466884 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c100299290 0x55c100295d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:28:00.834 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:00.831+0000 7fef1c97f740 1 --2- 172.21.15.47:0/356466884 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c100290c00 0x55c100298a90 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:28:00.834 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:00.831+0000 7fef00c00640 0 client.36476 ms_handle_reset on v2:172.21.15.100:6838/4173168440 2024-03-21T10:28:00.834 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:00.831+0000 7fef1c97f740 1 -- 172.21.15.47:0/356466884 >> 172.21.15.47:0/356466884 conn(0x55c100257200 msgr2=0x55c10028c6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:28:00.834 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:00.832+0000 7fef1c97f740 1 -- 172.21.15.47:0/356466884 shutdown_connections 2024-03-21T10:28:00.835 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:00.832+0000 7fef1c97f740 1 -- 172.21.15.47:0/356466884 wait complete. 2024-03-21T10:28:00.835 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:00.832+0000 7fef1c97f740 20 client.36476 trim_cache size 0 max 16384 2024-03-21T10:28:00.861 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:00.860+0000 7fef1c97f740 1 Processor -- start 2024-03-21T10:28:00.861 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:00.860+0000 7fef1c97f740 1 -- start start 2024-03-21T10:28:00.861 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:00.860+0000 7fef1c97f740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c100290c00 0x55c100293020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:28:00.861 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:00.860+0000 7fef1c97f740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c1002962c0 0x55c100293560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:28:00.862 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:00.860+0000 7fef1c97f740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c100299290 0x55c1002975b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:28:00.862 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:00.860+0000 7fef1c97f740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef141b4f80 con 0x55c100290c00 2024-03-21T10:28:00.862 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:00.860+0000 7fef1c97f740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef141b6900 con 0x55c1002962c0 2024-03-21T10:28:00.862 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:00.860+0000 7fef1c97f740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef141b55e0 con 0x55c100299290 2024-03-21T10:28:00.862 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:00.861+0000 7fef12a00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c100290c00 0x55c100293020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:28:00.862 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:00.861+0000 7fef12a00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c100290c00 0x55c100293020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:57964/0 (socket says 172.21.15.47:57964) 2024-03-21T10:28:00.862 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:00.861+0000 7fef12a00640 1 -- 172.21.15.47:0/1962392789 learned_addr learned my addr 172.21.15.47:0/1962392789 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:28:00.866 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:00.861+0000 7fef0be00640 1 --2- 172.21.15.47:0/1962392789 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c1002962c0 0x55c100293560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:28:00.867 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:00.861+0000 7fef13400640 1 --2- 172.21.15.47:0/1962392789 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c100299290 0x55c1002975b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:28:00.867 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:00.861+0000 7fef12a00640 1 -- 172.21.15.47:0/1962392789 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c100299290 msgr2=0x55c1002975b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:28:00.867 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:00.861+0000 7fef12a00640 1 --2- 172.21.15.47:0/1962392789 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c100299290 0x55c1002975b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:28:00.867 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:00.861+0000 7fef12a00640 1 -- 172.21.15.47:0/1962392789 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c1002962c0 msgr2=0x55c100293560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:28:00.867 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:00.861+0000 7fef12a00640 1 --2- 172.21.15.47:0/1962392789 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c1002962c0 0x55c100293560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:28:00.867 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:00.861+0000 7fef12a00640 1 -- 172.21.15.47:0/1962392789 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c10028fb00 con 0x55c100290c00 2024-03-21T10:28:00.867 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:00.861+0000 7fef13400640 1 --2- 172.21.15.47:0/1962392789 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c100299290 0x55c1002975b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T10:28:00.867 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:00.862+0000 7fef12a00640 1 --2- 172.21.15.47:0/1962392789 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c100290c00 0x55c100293020 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7fef04000c00 tx=0x7fef040d84b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e3fda99bb5215572 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:28:00.867 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:00.862+0000 7fef03400640 1 -- 172.21.15.47:0/1962392789 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef0409ba40 con 0x55c100290c00 2024-03-21T10:28:00.867 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:00.862+0000 7fef03400640 1 -- 172.21.15.47:0/1962392789 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fef040a0050 con 0x55c100290c00 2024-03-21T10:28:00.867 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:00.862+0000 7fef03400640 1 -- 172.21.15.47:0/1962392789 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef040a4960 con 0x55c100290c00 2024-03-21T10:28:00.868 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:00.863+0000 7fef1c97f740 1 -- 172.21.15.47:0/1962392789 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c100290c00 msgr2=0x55c100293020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:28:00.868 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:00.863+0000 7fef1c97f740 1 --2- 172.21.15.47:0/1962392789 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c100290c00 0x55c100293020 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7fef04000c00 tx=0x7fef040d84b0 comp rx=0 tx=0).stop 2024-03-21T10:28:00.868 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:00.863+0000 7fef1c97f740 1 -- 172.21.15.47:0/1962392789 shutdown_connections 2024-03-21T10:28:00.868 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:00.863+0000 7fef1c97f740 1 --2- 172.21.15.47:0/1962392789 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c100299290 0x55c1002975b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:28:00.868 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:00.863+0000 7fef1c97f740 1 --2- 172.21.15.47:0/1962392789 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c1002962c0 0x55c100293560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:28:00.868 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:00.863+0000 7fef1c97f740 1 --2- 172.21.15.47:0/1962392789 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c100290c00 0x55c100293020 unknown :-1 s=CLOSED pgs=734 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:28:00.868 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:00.863+0000 7fef1c97f740 1 -- 172.21.15.47:0/1962392789 >> 172.21.15.47:0/1962392789 conn(0x55c100257200 msgr2=0x55c100294670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:28:00.868 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:00.863+0000 7fef1c97f740 1 -- 172.21.15.47:0/1962392789 shutdown_connections 2024-03-21T10:28:00.868 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:00.863+0000 7fef1c97f740 1 -- 172.21.15.47:0/1962392789 wait complete. 2024-03-21T10:28:00.868 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:00.863+0000 7fef1c97f740 1 -- start start 2024-03-21T10:28:00.868 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:00.864+0000 7fef1c97f740 1 Processor -- start 2024-03-21T10:28:00.868 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:00.864+0000 7fef1c97f740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c100290c00 0x55c1002ae160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:28:00.868 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:00.864+0000 7fef1c97f740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c1002962c0 0x55c1002ae6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:28:00.868 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:00.864+0000 7fef1c97f740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c100299290 0x55c1002ac590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:28:00.869 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:00.864+0000 7fef1c97f740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef141b62a0 con 0x55c1002962c0 2024-03-21T10:28:00.869 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:00.864+0000 7fef1c97f740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef141b5c40 con 0x55c100299290 2024-03-21T10:28:00.869 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:00.864+0000 7fef1c97f740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef141b3140 con 0x55c100290c00 2024-03-21T10:28:00.869 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:00.864+0000 7fef1c97f740 10 client.0 _ll_register_callbacks cb 0x7fef0c037070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T10:28:00.869 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:00.865+0000 7fef0be00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c1002962c0 0x55c1002ae6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:28:00.869 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:00.865+0000 7fef0be00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c1002962c0 0x55c1002ae6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:57976/0 (socket says 172.21.15.47:57976) 2024-03-21T10:28:00.869 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:00.865+0000 7fef0be00640 1 -- 172.21.15.47:0/58154610 learned_addr learned my addr 172.21.15.47:0/58154610 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:28:00.869 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:00.865+0000 7fef13400640 1 --2- 172.21.15.47:0/58154610 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c100299290 0x55c1002ac590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:28:00.869 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:00.865+0000 7fef12a00640 1 --2- 172.21.15.47:0/58154610 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c100290c00 0x55c1002ae160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:28:00.869 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:00.865+0000 7fef0be00640 1 -- 172.21.15.47:0/58154610 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c100290c00 msgr2=0x55c1002ae160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:28:00.869 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:00.865+0000 7fef0be00640 1 --2- 172.21.15.47:0/58154610 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c100290c00 0x55c1002ae160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:28:00.869 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:00.865+0000 7fef0be00640 1 -- 172.21.15.47:0/58154610 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c100299290 msgr2=0x55c1002ac590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:28:00.869 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:00.865+0000 7fef0be00640 1 --2- 172.21.15.47:0/58154610 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c100299290 0x55c1002ac590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:28:00.869 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:00.865+0000 7fef0be00640 1 -- 172.21.15.47:0/58154610 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55c0ffec80f0 con 0x55c1002962c0 2024-03-21T10:28:00.870 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:00.865+0000 7fef12a00640 1 --2- 172.21.15.47:0/58154610 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c100290c00 0x55c1002ae160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T10:28:00.870 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:00.865+0000 7fef0be00640 1 --2- 172.21.15.47:0/58154610 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c1002962c0 0x55c1002ae6a0 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7feefc000c00 tx=0x7feefc0055c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=402e75335368422c server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:28:00.870 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:00.865+0000 7fef00c00640 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-03-21T10:28:00.870 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:00.866+0000 7fef00c00640 1 -- 172.21.15.47:0/58154610 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feefc004220 con 0x55c1002962c0 2024-03-21T10:28:00.870 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:00.866+0000 7fef00c00640 1 -- 172.21.15.47:0/58154610 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(97..97 src has 1..97) v4 ==== 9474+0+0 (secure 0 0 0) 0x7feefc0043c0 con 0x55c1002962c0 2024-03-21T10:28:00.870 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:00.866+0000 7fef1c97f740 1 -- 172.21.15.47:0/58154610 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fef040a0050 con 0x55c1002962c0 2024-03-21T10:28:00.870 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:00.866+0000 7fef00c00640 1 -- 172.21.15.47:0/58154610 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7feefc004e10 con 0x55c1002962c0 2024-03-21T10:28:00.870 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:00.866+0000 7fef1c97f740 10 client.16683 fetch_fsmap learned FSMap version 32 2024-03-21T10:28:00.870 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:00.866+0000 7fef1c97f740 1 -- 172.21.15.47:0/58154610 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x55c1003053c0 con 0x55c1002962c0 2024-03-21T10:28:00.872 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:00.866+0000 7fef00c00640 1 -- 172.21.15.47:0/58154610 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fef040a0050 con 0x55c1002962c0 2024-03-21T10:28:00.872 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:00.867+0000 7fef00c00640 1 -- 172.21.15.47:0/58154610 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 32) v1 ==== 7942+0+0 (secure 0 0 0) 0x7feefc01a790 con 0x55c1002962c0 2024-03-21T10:28:00.872 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:00.867+0000 7fef1c97f740 10 client.16683 fetch_fsmap finished waiting for FSMap version 32 2024-03-21T10:28:00.872 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:00.867+0000 7fef1c97f740 10 client.16683 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-21T10:28:00.872 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:00.867+0000 7fef1c97f740 20 client.16683 populate_metadata read hostname 'smithi047' 2024-03-21T10:28:00.872 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:00.867+0000 7fef1c97f740 1 --2- 172.21.15.47:0/58154610 >> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] conn(0x55c100305ad0 0x55c100307ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:28:00.872 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:00.867+0000 7fef1c97f740 4 client.16683 mds_command: new command op to 34535 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-21T10:28:00.872 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:00.867+0000 7fef1c97f740 1 -- 172.21.15.47:0/58154610 --> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55c1001793a0 con 0x55c100305ad0 2024-03-21T10:28:00.872 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:00.867+0000 7fef12a00640 1 --2- 172.21.15.47:0/58154610 >> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] conn(0x55c100305ad0 0x55c100307ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:28:00.872 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:00.868+0000 7fef00c00640 10 client.16683 ms_handle_connect on v2:172.21.15.100:6838/4173168440 2024-03-21T10:28:00.872 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:00.868+0000 7fef12a00640 1 --2- 172.21.15.47:0/58154610 >> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] conn(0x55c100305ad0 0x55c100307ef0 crc :-1 s=READY pgs=106 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7e595d37408a2264 server_cookie=4e3943756a13f74d in_seq=0 out_seq=0 2024-03-21T10:28:00.873 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:00.868+0000 7fef00c00640 1 -- 172.21.15.47:0/58154610 <== mds.0 v2:172.21.15.100:6838/4173168440 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x55c1001793a0 con 0x55c100305ad0 2024-03-21T10:28:00.873 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:00.868+0000 7fef00c00640 10 client.16683 handle_command_reply: tid=0 2024-03-21T10:28:00.873 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:00.868+0000 7fef1c97f740 1 client.16683 shutdown 2024-03-21T10:28:00.873 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:00.869+0000 7fef1c97f740 1 -- 172.21.15.47:0/58154610 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c1002962c0 msgr2=0x55c1002ae6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:28:00.873 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:00.869+0000 7fef1c97f740 1 --2- 172.21.15.47:0/58154610 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c1002962c0 0x55c1002ae6a0 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7feefc000c00 tx=0x7feefc0055c0 comp rx=0 tx=0).stop 2024-03-21T10:28:00.873 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:00.869+0000 7fef1c97f740 1 -- 172.21.15.47:0/58154610 shutdown_connections 2024-03-21T10:28:00.873 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:00.869+0000 7fef1c97f740 1 --2- 172.21.15.47:0/58154610 >> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] conn(0x55c100305ad0 0x55c100307ef0 crc :-1 s=READY pgs=106 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:28:00.873 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:00.869+0000 7fef1c97f740 1 --2- 172.21.15.47:0/58154610 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c100299290 0x55c1002ac590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:28:00.873 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:00.869+0000 7fef1c97f740 1 --2- 172.21.15.47:0/58154610 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c1002962c0 0x55c1002ae6a0 unknown :-1 s=CLOSED pgs=735 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:28:00.873 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:00.869+0000 7fef1c97f740 1 --2- 172.21.15.47:0/58154610 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c100290c00 0x55c1002ae160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:28:00.873 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:00.869+0000 7fef1c97f740 1 -- 172.21.15.47:0/58154610 >> 172.21.15.47:0/58154610 conn(0x55c100257200 msgr2=0x55c1002a4510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:28:00.873 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:00.869+0000 7fef00c00640 0 client.16683 ms_handle_reset on v2:172.21.15.100:6838/4173168440 2024-03-21T10:28:00.873 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:00.869+0000 7fef1c97f740 1 -- 172.21.15.47:0/58154610 shutdown_connections 2024-03-21T10:28:00.873 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:00.869+0000 7fef1c97f740 1 -- 172.21.15.47:0/58154610 wait complete. 2024-03-21T10:28:00.874 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:00.869+0000 7fef1c97f740 20 client.16683 trim_cache size 0 max 16384 2024-03-21T10:28:00.874 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-03-21T10:28:00.874 INFO:teuthology.orchestra.run.smithi047.stdout: "status": "IDLE (waiting for more scrubs)", 2024-03-21T10:28:00.874 INFO:teuthology.orchestra.run.smithi047.stdout: "scrubs": { 2024-03-21T10:28:00.874 INFO:teuthology.orchestra.run.smithi047.stdout: "889f24b1-253a-4150-9658-163b715897dc": { 2024-03-21T10:28:00.874 INFO:teuthology.orchestra.run.smithi047.stdout: "path": "/", 2024-03-21T10:28:00.874 INFO:teuthology.orchestra.run.smithi047.stdout: "tag": "889f24b1-253a-4150-9658-163b715897dc", 2024-03-21T10:28:00.874 INFO:teuthology.orchestra.run.smithi047.stdout: "options": "recursive,force" 2024-03-21T10:28:00.874 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-21T10:28:00.874 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-21T10:28:00.875 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-03-21T10:28:00.875 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:00.870+0000 7fef19c00640 1 -- 172.21.15.47:0/44124501 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7feef80a3090 msgr2=0x7feef80a5550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:28:00.875 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:00.870+0000 7fef19c00640 1 --2- 172.21.15.47:0/44124501 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7feef80a3090 0x7feef80a5550 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fef0c006010 tx=0x7fef0c004a40 comp rx=0 tx=0).stop 2024-03-21T10:28:00.875 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:00.870+0000 7fef19c00640 1 -- 172.21.15.47:0/44124501 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fef14107eb0 msgr2=0x7fef1419e460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:28:00.875 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:00.870+0000 7fef19c00640 1 --2- 172.21.15.47:0/44124501 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fef14107eb0 0x7fef1419e460 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7fef0400ea60 tx=0x7fef0400ef20 comp rx=0 tx=0).stop 2024-03-21T10:28:00.876 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:00.870+0000 7fef19c00640 1 -- 172.21.15.47:0/44124501 shutdown_connections 2024-03-21T10:28:00.876 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:00.870+0000 7fef19c00640 1 --2- 172.21.15.47:0/44124501 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fef141a2d20 0x7fef141a31d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:28:00.876 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:00.870+0000 7fef19c00640 1 --2- 172.21.15.47:0/44124501 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7feef80a3090 0x7feef80a5550 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:28:00.876 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:00.870+0000 7fef19c00640 1 --2- 172.21.15.47:0/44124501 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fef1410cf80 0x7fef1419e9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:28:00.876 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:00.870+0000 7fef19c00640 1 --2- 172.21.15.47:0/44124501 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fef14107eb0 0x7fef1419e460 unknown :-1 s=CLOSED pgs=732 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:28:00.876 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:00.870+0000 7fef19c00640 1 -- 172.21.15.47:0/44124501 >> 172.21.15.47:0/44124501 conn(0x7fef14075be0 msgr2=0x7fef14076ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:28:00.876 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:00.870+0000 7fef19c00640 1 -- 172.21.15.47:0/44124501 shutdown_connections 2024-03-21T10:28:00.876 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:00.870+0000 7fef19c00640 1 -- 172.21.15.47:0/44124501 wait complete. 2024-03-21T10:28:00.888 INFO:tasks.cephfs.filesystem:scrub status for tag:889f24b1-253a-4150-9658-163b715897dc - {'path': '/', 'tag': '889f24b1-253a-4150-9658-163b715897dc', 'options': 'recursive,force'} 2024-03-21T10:28:01.962 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:28:01 smithi047 ceph-mon[31547]: pgmap v84: 129 pgs: 129 active+clean; 273 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 7.2 MiB/s wr, 1.55k op/s 2024-03-21T10:28:01.962 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:28:01 smithi047 ceph-mon[31547]: scrub queued for path: / 2024-03-21T10:28:01.962 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:28:01 smithi047 ceph-mon[31547]: scrub summary: idle+waiting paths [/] 2024-03-21T10:28:01.962 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:28:01 smithi047 ceph-mon[31547]: scrub summary: active paths [/] 2024-03-21T10:28:01.963 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:28:01 smithi047 ceph-mon[31547]: scrub summary: idle+waiting paths [/] 2024-03-21T10:28:01.978 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:28:01 smithi100 ceph-mon[35193]: pgmap v84: 129 pgs: 129 active+clean; 273 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 7.2 MiB/s wr, 1.55k op/s 2024-03-21T10:28:01.978 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:28:01 smithi100 ceph-mon[35193]: scrub queued for path: / 2024-03-21T10:28:01.978 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:28:01 smithi100 ceph-mon[35193]: scrub summary: idle+waiting paths [/] 2024-03-21T10:28:01.978 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:28:01 smithi100 ceph-mon[35193]: scrub summary: active paths [/] 2024-03-21T10:28:01.978 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:28:01 smithi100 ceph-mon[35193]: scrub summary: idle+waiting paths [/] 2024-03-21T10:28:01.988 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:28:01 smithi157 ceph-mon[35581]: pgmap v84: 129 pgs: 129 active+clean; 273 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 7.2 MiB/s wr, 1.55k op/s 2024-03-21T10:28:01.989 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:28:01 smithi157 ceph-mon[35581]: scrub queued for path: / 2024-03-21T10:28:01.989 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:28:01 smithi157 ceph-mon[35581]: scrub summary: idle+waiting paths [/] 2024-03-21T10:28:01.989 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:28:01 smithi157 ceph-mon[35581]: scrub summary: active paths [/] 2024-03-21T10:28:01.989 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:28:01 smithi157 ceph-mon[35581]: scrub summary: idle+waiting paths [/] 2024-03-21T10:28:02.846 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:28:02.846+0000 7f8131600640 1 -- 172.21.15.47:0/3821450353 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f812c10aa80 msgr2=0x7f812c10eac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:28:02.846 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:28:02.846+0000 7f8131600640 1 --2- 172.21.15.47:0/3821450353 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f812c10aa80 0x7f812c10eac0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f811c009900 tx=0x7f811c02bc30 comp rx=0 tx=0).stop 2024-03-21T10:28:02.847 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:28:02.847+0000 7f8131600640 1 -- 172.21.15.47:0/3821450353 shutdown_connections 2024-03-21T10:28:02.847 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:28:02.847+0000 7f8131600640 1 --2- 172.21.15.47:0/3821450353 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f812c10f1a0 0x7f812c116d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:28:02.847 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:28:02.847+0000 7f8131600640 1 --2- 172.21.15.47:0/3821450353 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f812c10aa80 0x7f812c10eac0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:28:02.847 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:28:02.847+0000 7f8131600640 1 --2- 172.21.15.47:0/3821450353 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f812c10a0d0 0x7f812c10a4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:28:02.847 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:28:02.847+0000 7f8131600640 1 -- 172.21.15.47:0/3821450353 >> 172.21.15.47:0/3821450353 conn(0x7f812c06e830 msgr2=0x7f812c1086a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:28:02.848 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:28:02.848+0000 7f8131600640 1 -- 172.21.15.47:0/3821450353 shutdown_connections 2024-03-21T10:28:02.848 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:28:02.848+0000 7f8131600640 1 -- 172.21.15.47:0/3821450353 wait complete. 2024-03-21T10:28:02.850 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:28:02.850+0000 7f8131600640 1 Processor -- start 2024-03-21T10:28:02.850 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:28:02.850+0000 7f8131600640 1 -- start start 2024-03-21T10:28:02.851 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:28:02.851+0000 7f8131600640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f812c10a0d0 0x7f812c115090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:28:02.851 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:28:02.851+0000 7f8131600640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f812c10f1a0 0x7f812c1155d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:28:02.851 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:28:02.851+0000 7f8131600640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f812c115b10 0x7f812c1b2ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:28:02.851 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:28:02.851+0000 7f8131600640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f812c117ef0 con 0x7f812c10f1a0 2024-03-21T10:28:02.851 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:28:02.851+0000 7f8131600640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f812c117d70 con 0x7f812c115b10 2024-03-21T10:28:02.851 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:28:02.851+0000 7f8131600640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f812c118070 con 0x7f812c10a0d0 2024-03-21T10:28:02.851 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:28:02.851+0000 7f812aa00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f812c10a0d0 0x7f812c115090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:28:02.852 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:28:02.851+0000 7f812b400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f812c115b10 0x7f812c1b2ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:28:02.852 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:28:02.851+0000 7f812b400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f812c115b10 0x7f812c1b2ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:39194/0 (socket says 172.21.15.47:39194) 2024-03-21T10:28:02.852 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:28:02.851+0000 7f812b400640 1 -- 172.21.15.47:0/4031400535 learned_addr learned my addr 172.21.15.47:0/4031400535 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:28:02.852 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:28:02.852+0000 7f812a000640 1 --2- 172.21.15.47:0/4031400535 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f812c10f1a0 0x7f812c1155d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:28:02.852 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:28:02.852+0000 7f812a000640 1 -- 172.21.15.47:0/4031400535 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f812c10a0d0 msgr2=0x7f812c115090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:28:02.852 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:28:02.852+0000 7f812a000640 1 --2- 172.21.15.47:0/4031400535 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f812c10a0d0 0x7f812c115090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:28:02.852 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:28:02.852+0000 7f812a000640 1 -- 172.21.15.47:0/4031400535 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f812c115b10 msgr2=0x7f812c1b2ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:28:02.852 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:28:02.852+0000 7f812a000640 1 --2- 172.21.15.47:0/4031400535 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f812c115b10 0x7f812c1b2ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:28:02.852 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:28:02.852+0000 7f812a000640 1 -- 172.21.15.47:0/4031400535 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f812c1b32f0 con 0x7f812c10f1a0 2024-03-21T10:28:02.853 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:28:02.853+0000 7f812a000640 1 --2- 172.21.15.47:0/4031400535 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f812c10f1a0 0x7f812c1155d0 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7f811c036040 tx=0x7f811c02ca10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:28:02.855 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:28:02.854+0000 7f8123400640 1 -- 172.21.15.47:0/4031400535 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f811c043020 con 0x7f812c10f1a0 2024-03-21T10:28:02.856 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:28:02.854+0000 7f8131600640 1 -- 172.21.15.47:0/4031400535 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f812c1b3580 con 0x7f812c10f1a0 2024-03-21T10:28:02.856 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:28:02.854+0000 7f8123400640 1 -- 172.21.15.47:0/4031400535 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f811c0052b0 con 0x7f812c10f1a0 2024-03-21T10:28:02.856 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:28:02.854+0000 7f8123400640 1 -- 172.21.15.47:0/4031400535 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f811c003b50 con 0x7f812c10f1a0 2024-03-21T10:28:02.856 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:28:02.855+0000 7f8131600640 1 -- 172.21.15.47:0/4031400535 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f812c1b3af0 con 0x7f812c10f1a0 2024-03-21T10:28:02.859 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:28:02.860+0000 7f813425e740 1 Processor -- start 2024-03-21T10:28:02.861 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:28:02.862+0000 7f8123400640 1 -- 172.21.15.47:0/4031400535 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 27) v1 ==== 137985+0+0 (secure 0 0 0) 0x7f811c046020 con 0x7f812c10f1a0 2024-03-21T10:28:02.862 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:28:02.862+0000 7f813425e740 1 -- start start 2024-03-21T10:28:02.863 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:28:02.863+0000 7f813425e740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ca46894fd0 0x55ca468953b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:28:02.863 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:28:02.864+0000 7f813425e740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ca468958f0 0x55ca4689e180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:28:02.864 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:28:02.864+0000 7f813425e740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ca4689e6c0 0x55ca468a0ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:28:02.864 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:28:02.864+0000 7f813425e740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ca468573b0 con 0x55ca46894fd0 2024-03-21T10:28:02.864 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:28:02.864+0000 7f813425e740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ca467f9830 con 0x55ca468958f0 2024-03-21T10:28:02.864 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:28:02.864+0000 7f813425e740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ca467a25d0 con 0x55ca4689e6c0 2024-03-21T10:28:02.864 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:28:02.864+0000 7f812aa00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ca468958f0 0x55ca4689e180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:28:02.864 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:28:02.864+0000 7f812aa00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ca468958f0 0x55ca4689e180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:39210/0 (socket says 172.21.15.47:39210) 2024-03-21T10:28:02.864 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:28:02.864+0000 7f812a000640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ca4689e6c0 0x55ca468a0ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:28:02.864 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:28:02.864+0000 7f812aa00640 1 -- 172.21.15.47:0/1829758035 learned_addr learned my addr 172.21.15.47:0/1829758035 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:28:02.864 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:28:02.864+0000 7f812b400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ca46894fd0 0x55ca468953b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:28:02.865 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:28:02.865+0000 7f8123400640 1 --2- 172.21.15.47:0/4031400535 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f81100a3180 0x7f81100a5640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:28:02.865 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:28:02.865+0000 7f812b400640 1 -- 172.21.15.47:0/1829758035 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ca4689e6c0 msgr2=0x55ca468a0ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:28:02.865 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:28:02.865+0000 7f812b400640 1 --2- 172.21.15.47:0/1829758035 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ca4689e6c0 0x55ca468a0ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:28:02.865 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:28:02.865+0000 7f812b400640 1 -- 172.21.15.47:0/1829758035 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ca468958f0 msgr2=0x55ca4689e180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:28:02.865 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:28:02.865+0000 7f812b400640 1 --2- 172.21.15.47:0/1829758035 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ca468958f0 0x55ca4689e180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:28:02.866 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:28:02.865+0000 7f812b400640 1 -- 172.21.15.47:0/1829758035 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ca467ba2c0 con 0x55ca46894fd0 2024-03-21T10:28:02.866 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:28:02.865+0000 7f8123400640 1 -- 172.21.15.47:0/4031400535 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 9474+0+0 (secure 0 0 0) 0x7f811c0f1490 con 0x7f812c10f1a0 2024-03-21T10:28:02.866 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:28:02.866+0000 7f812b400640 1 --2- 172.21.15.47:0/4031400535 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f81100a3180 0x7f81100a5640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:28:02.866 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:28:02.866+0000 7f812b400640 1 --2- 172.21.15.47:0/1829758035 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ca46894fd0 0x55ca468953b0 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7f811800bc30 tx=0x7f81180107a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=778537e1381c4e87 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:28:02.867 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:28:02.867+0000 7f812b400640 1 --2- 172.21.15.47:0/4031400535 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f81100a3180 0x7f81100a5640 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f811801f000 tx=0x7f8118020000 comp rx=0 tx=0).ready entity=mgr.15687 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:28:02.868 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:28:02.867+0000 7f8117400640 1 -- 172.21.15.47:0/1829758035 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8118022070 con 0x55ca46894fd0 2024-03-21T10:28:02.868 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:28:02.868+0000 7f8117400640 1 -- 172.21.15.47:0/1829758035 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f811801cad0 con 0x55ca46894fd0 2024-03-21T10:28:02.868 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:28:02.869+0000 7f8117400640 1 -- 172.21.15.47:0/1829758035 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f811801b960 con 0x55ca46894fd0 2024-03-21T10:28:02.869 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:28:02.869+0000 7f813425e740 1 -- 172.21.15.47:0/1829758035 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ca46894fd0 msgr2=0x55ca468953b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:28:02.869 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:28:02.869+0000 7f813425e740 1 --2- 172.21.15.47:0/1829758035 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ca46894fd0 0x55ca468953b0 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7f811800bc30 tx=0x7f81180107a0 comp rx=0 tx=0).stop 2024-03-21T10:28:02.870 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:28:02.870+0000 7f813425e740 1 -- 172.21.15.47:0/1829758035 shutdown_connections 2024-03-21T10:28:02.870 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:28:02.870+0000 7f813425e740 1 --2- 172.21.15.47:0/1829758035 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ca4689e6c0 0x55ca468a0ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:28:02.870 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:28:02.870+0000 7f813425e740 1 --2- 172.21.15.47:0/1829758035 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ca468958f0 0x55ca4689e180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:28:02.870 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:28:02.870+0000 7f813425e740 1 --2- 172.21.15.47:0/1829758035 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ca46894fd0 0x55ca468953b0 unknown :-1 s=CLOSED pgs=737 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:28:02.870 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:28:02.870+0000 7f813425e740 1 -- 172.21.15.47:0/1829758035 >> 172.21.15.47:0/1829758035 conn(0x55ca4688f8f0 msgr2=0x55ca4688fcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:28:02.871 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:28:02.871+0000 7f813425e740 1 -- 172.21.15.47:0/1829758035 shutdown_connections 2024-03-21T10:28:02.871 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:28:02.871+0000 7f813425e740 1 -- 172.21.15.47:0/1829758035 wait complete. 2024-03-21T10:28:02.872 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:28:02.872+0000 7f813425e740 1 -- start start 2024-03-21T10:28:02.872 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:28:02.872+0000 7f813425e740 1 Processor -- start 2024-03-21T10:28:02.874 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:28:02.873+0000 7f813425e740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ca46894fd0 0x55ca4689b610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:28:02.874 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:28:02.873+0000 7f813425e740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ca468958f0 0x55ca4689bb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:28:02.874 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:28:02.873+0000 7f813425e740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ca4689e6c0 0x55ca4689c090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:28:02.874 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:28:02.873+0000 7f813425e740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ca467b5a20 con 0x55ca46894fd0 2024-03-21T10:28:02.874 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:28:02.874+0000 7f813425e740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ca467f58d0 con 0x55ca4689e6c0 2024-03-21T10:28:02.875 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:28:02.874+0000 7f813425e740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ca468a1340 con 0x55ca468958f0 2024-03-21T10:28:02.875 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:28:02.874+0000 7f813425e740 10 client.0 _ll_register_callbacks cb 0x55ca467f21c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T10:28:02.875 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:28:02.874+0000 7f812a000640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ca468958f0 0x55ca4689bb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:28:02.875 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:28:02.874+0000 7f812a000640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ca468958f0 0x55ca4689bb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.47:54906/0 (socket says 172.21.15.47:54906) 2024-03-21T10:28:02.875 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:28:02.874+0000 7f812a000640 1 -- 172.21.15.47:0/4287776182 learned_addr learned my addr 172.21.15.47:0/4287776182 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:28:02.875 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:28:02.874+0000 7f812b400640 1 --2- 172.21.15.47:0/4287776182 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ca4689e6c0 0x55ca4689c090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:28:02.875 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:28:02.875+0000 7f812a000640 1 -- 172.21.15.47:0/4287776182 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ca4689e6c0 msgr2=0x55ca4689c090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:28:02.875 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:28:02.875+0000 7f812a000640 1 --2- 172.21.15.47:0/4287776182 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ca4689e6c0 0x55ca4689c090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:28:02.875 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:28:02.875+0000 7f812a000640 1 -- 172.21.15.47:0/4287776182 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ca46894fd0 msgr2=0x55ca4689b610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:28:02.875 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:28:02.875+0000 7f812a000640 1 --2- 172.21.15.47:0/4287776182 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ca46894fd0 0x55ca4689b610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:28:02.875 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:28:02.875+0000 7f812a000640 1 -- 172.21.15.47:0/4287776182 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55ca4650e210 con 0x55ca468958f0 2024-03-21T10:28:02.876 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:28:02.875+0000 7f812a000640 1 --2- 172.21.15.47:0/4287776182 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ca468958f0 0x55ca4689bb50 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f811c009900 tx=0x7f811c0f4bd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7dc9fb61b807999a server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:28:02.876 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:28:02.875+0000 7f810be00640 10 client.0 ms_handle_connect on v2:172.21.15.157:3300/0 2024-03-21T10:28:02.876 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:28:02.876+0000 7f810be00640 1 -- 172.21.15.47:0/4287776182 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f811c0bd3a0 con 0x55ca468958f0 2024-03-21T10:28:02.876 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:28:02.876+0000 7f810be00640 1 -- 172.21.15.47:0/4287776182 <== mon.1 v2:172.21.15.157:3300/0 2 ==== osd_map(97..97 src has 1..97) v4 ==== 9474+0+0 (secure 0 0 0) 0x7f811c0b9c40 con 0x55ca468958f0 2024-03-21T10:28:02.876 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:28:02.877+0000 7f813425e740 1 -- 172.21.15.47:0/4287776182 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f811801cad0 con 0x55ca468958f0 2024-03-21T10:28:02.877 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:28:02.877+0000 7f810be00640 1 -- 172.21.15.47:0/4287776182 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f811c0bcf30 con 0x55ca468958f0 2024-03-21T10:28:02.877 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:28:02.878+0000 7f813425e740 10 client.26245 fetch_fsmap learned FSMap version 32 2024-03-21T10:28:02.878 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:28:02.878+0000 7f813425e740 1 -- 172.21.15.47:0/4287776182 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x55ca465da620 con 0x55ca468958f0 2024-03-21T10:28:02.880 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:28:02.880+0000 7f810be00640 1 -- 172.21.15.47:0/4287776182 <== mon.1 v2:172.21.15.157:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f811801cad0 con 0x55ca468958f0 2024-03-21T10:28:02.880 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:28:02.880+0000 7f810be00640 1 -- 172.21.15.47:0/4287776182 <== mon.1 v2:172.21.15.157:3300/0 5 ==== fsmap(e 32) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f811c0bd830 con 0x55ca468958f0 2024-03-21T10:28:02.881 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:28:02.881+0000 7f813425e740 10 client.26245 fetch_fsmap finished waiting for FSMap version 32 2024-03-21T10:28:02.881 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:28:02.881+0000 7f813425e740 10 client.26245 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-21T10:28:02.881 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:28:02.881+0000 7f813425e740 20 client.26245 populate_metadata read hostname 'smithi047' 2024-03-21T10:28:02.881 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:28:02.881+0000 7f813425e740 1 --2- 172.21.15.47:0/4287776182 >> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] conn(0x55ca468bf6f0 0x55ca468c1b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:28:02.881 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:28:02.881+0000 7f813425e740 4 client.26245 mds_command: new command op to 34535 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T10:28:02.881 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:28:02.881+0000 7f813425e740 1 -- 172.21.15.47:0/4287776182 --> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55ca46806d70 con 0x55ca468bf6f0 2024-03-21T10:28:02.881 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:28:02.881+0000 7f812aa00640 1 --2- 172.21.15.47:0/4287776182 >> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] conn(0x55ca468bf6f0 0x55ca468c1b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:28:02.882 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:28:02.883+0000 7f810be00640 10 client.26245 ms_handle_connect on v2:172.21.15.100:6838/4173168440 2024-03-21T10:28:02.882 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:28:02.883+0000 7f812aa00640 1 --2- 172.21.15.47:0/4287776182 >> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] conn(0x55ca468bf6f0 0x55ca468c1b70 crc :-1 s=READY pgs=107 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ab965a26099a3e2e server_cookie=d662c652840d397 in_seq=0 out_seq=0 2024-03-21T10:28:02.888 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:28:02.884+0000 7f810be00640 1 -- 172.21.15.47:0/4287776182 <== mds.0 v2:172.21.15.100:6838/4173168440 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55ca46806d70 con 0x55ca468bf6f0 2024-03-21T10:28:02.888 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:28:02.884+0000 7f810be00640 10 client.26245 handle_command_reply: tid=0 2024-03-21T10:28:02.888 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:28:02.884+0000 7f813425e740 1 client.26245 shutdown 2024-03-21T10:28:02.888 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:28:02.885+0000 7f813425e740 1 -- 172.21.15.47:0/4287776182 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ca468958f0 msgr2=0x55ca4689bb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:28:02.889 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:28:02.885+0000 7f813425e740 1 --2- 172.21.15.47:0/4287776182 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ca468958f0 0x55ca4689bb50 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f811c009900 tx=0x7f811c0f4bd0 comp rx=0 tx=0).stop 2024-03-21T10:28:02.889 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:28:02.885+0000 7f813425e740 1 -- 172.21.15.47:0/4287776182 shutdown_connections 2024-03-21T10:28:02.889 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:28:02.885+0000 7f813425e740 1 --2- 172.21.15.47:0/4287776182 >> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] conn(0x55ca468bf6f0 0x55ca468c1b70 crc :-1 s=READY pgs=107 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:28:02.889 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:28:02.885+0000 7f813425e740 1 --2- 172.21.15.47:0/4287776182 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ca4689e6c0 0x55ca4689c090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:28:02.889 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:28:02.885+0000 7f813425e740 1 --2- 172.21.15.47:0/4287776182 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ca468958f0 0x55ca4689bb50 secure :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f811c009900 tx=0x7f811c0f4bd0 comp rx=0 tx=0).stop 2024-03-21T10:28:02.889 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:28:02.885+0000 7f813425e740 1 --2- 172.21.15.47:0/4287776182 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ca46894fd0 0x55ca4689b610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:28:02.889 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:28:02.885+0000 7f813425e740 1 -- 172.21.15.47:0/4287776182 >> 172.21.15.47:0/4287776182 conn(0x55ca4688f8f0 msgr2=0x55ca46891640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:28:02.889 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:28:02.885+0000 7f810be00640 0 client.26245 ms_handle_reset on v2:172.21.15.100:6838/4173168440 2024-03-21T10:28:02.889 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:28:02.886+0000 7f813425e740 1 -- 172.21.15.47:0/4287776182 shutdown_connections 2024-03-21T10:28:02.889 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:28:02.886+0000 7f813425e740 1 -- 172.21.15.47:0/4287776182 wait complete. 2024-03-21T10:28:02.889 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:28:02.886+0000 7f813425e740 20 client.26245 trim_cache size 0 max 16384 2024-03-21T10:28:02.920 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:28:02.920+0000 7f813425e740 1 Processor -- start 2024-03-21T10:28:02.921 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:28:02.921+0000 7f813425e740 1 -- start start 2024-03-21T10:28:02.921 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:28:02.921+0000 7f813425e740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ca46894fd0 0x55ca4689dd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:28:02.921 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:28:02.921+0000 7f813425e740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ca4689e6c0 0x55ca4689a770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:28:02.922 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:28:02.921+0000 7f813425e740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ca46896f70 0x55ca4689beb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:28:02.922 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:28:02.921+0000 7f813425e740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f810c00fce0 con 0x55ca46896f70 2024-03-21T10:28:02.922 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:28:02.921+0000 7f813425e740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f810c011660 con 0x55ca46894fd0 2024-03-21T10:28:02.922 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:28:02.921+0000 7f813425e740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f810c010340 con 0x55ca4689e6c0 2024-03-21T10:28:02.922 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:28:02.922+0000 7f812b400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ca46896f70 0x55ca4689beb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:28:02.922 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:28:02.922+0000 7f812b400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ca46896f70 0x55ca4689beb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:45860/0 (socket says 172.21.15.47:45860) 2024-03-21T10:28:02.922 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:28:02.922+0000 7f812a000640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ca4689e6c0 0x55ca4689a770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:28:02.922 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:28:02.922+0000 7f812b400640 1 -- 172.21.15.47:0/4002266573 learned_addr learned my addr 172.21.15.47:0/4002266573 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:28:02.923 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:28:02.923+0000 7f812aa00640 1 --2- 172.21.15.47:0/4002266573 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ca46894fd0 0x55ca4689dd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:28:02.923 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:28:02.923+0000 7f812aa00640 1 -- 172.21.15.47:0/4002266573 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ca4689e6c0 msgr2=0x55ca4689a770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:28:02.923 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:28:02.923+0000 7f812aa00640 1 --2- 172.21.15.47:0/4002266573 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ca4689e6c0 0x55ca4689a770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:28:02.923 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:28:02.923+0000 7f812aa00640 1 -- 172.21.15.47:0/4002266573 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ca46896f70 msgr2=0x55ca4689beb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:28:02.923 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:28:02.923+0000 7f812aa00640 1 --2- 172.21.15.47:0/4002266573 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ca46896f70 0x55ca4689beb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:28:02.924 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:28:02.923+0000 7f812aa00640 1 -- 172.21.15.47:0/4002266573 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ca46890270 con 0x55ca46894fd0 2024-03-21T10:28:02.924 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:28:02.924+0000 7f812a000640 1 --2- 172.21.15.47:0/4002266573 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ca4689e6c0 0x55ca4689a770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T10:28:02.924 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:28:02.924+0000 7f812aa00640 1 --2- 172.21.15.47:0/4002266573 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ca46894fd0 0x55ca4689dd40 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f8124004a80 tx=0x7f812401bc80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b0cc48d8a4569a9f server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:28:02.925 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:28:02.925+0000 7f8116a00640 1 -- 172.21.15.47:0/4002266573 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f812400aa00 con 0x55ca46894fd0 2024-03-21T10:28:02.925 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:28:02.925+0000 7f8116a00640 1 -- 172.21.15.47:0/4002266573 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f812400aba0 con 0x55ca46894fd0 2024-03-21T10:28:02.925 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:28:02.925+0000 7f8116a00640 1 -- 172.21.15.47:0/4002266573 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8124017390 con 0x55ca46894fd0 2024-03-21T10:28:02.927 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:28:02.927+0000 7f813425e740 1 -- 172.21.15.47:0/4002266573 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ca46894fd0 msgr2=0x55ca4689dd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:28:02.927 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:28:02.927+0000 7f813425e740 1 --2- 172.21.15.47:0/4002266573 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ca46894fd0 0x55ca4689dd40 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f8124004a80 tx=0x7f812401bc80 comp rx=0 tx=0).stop 2024-03-21T10:28:02.927 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:28:02.927+0000 7f813425e740 1 -- 172.21.15.47:0/4002266573 shutdown_connections 2024-03-21T10:28:02.927 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:28:02.927+0000 7f813425e740 1 --2- 172.21.15.47:0/4002266573 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ca46896f70 0x55ca4689beb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:28:02.927 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:28:02.927+0000 7f813425e740 1 --2- 172.21.15.47:0/4002266573 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ca4689e6c0 0x55ca4689a770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:28:02.928 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:28:02.927+0000 7f813425e740 1 --2- 172.21.15.47:0/4002266573 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ca46894fd0 0x55ca4689dd40 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:28:02.928 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:28:02.927+0000 7f813425e740 1 -- 172.21.15.47:0/4002266573 >> 172.21.15.47:0/4002266573 conn(0x55ca4688f8f0 msgr2=0x55ca46890c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:28:02.928 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:28:02.928+0000 7f813425e740 1 -- 172.21.15.47:0/4002266573 shutdown_connections 2024-03-21T10:28:02.928 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:28:02.928+0000 7f813425e740 1 -- 172.21.15.47:0/4002266573 wait complete. 2024-03-21T10:28:02.928 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:28:02.929+0000 7f813425e740 1 -- start start 2024-03-21T10:28:02.929 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:28:02.929+0000 7f813425e740 1 Processor -- start 2024-03-21T10:28:02.930 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:28:02.929+0000 7f813425e740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ca46894fd0 0x55ca468a1e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:28:02.930 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:28:02.929+0000 7f813425e740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ca46896f70 0x55ca468a2390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:28:02.930 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:28:02.929+0000 7f813425e740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ca4689e6c0 0x55ca4691a000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:28:02.930 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:28:02.929+0000 7f813425e740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f810c011000 con 0x55ca46894fd0 2024-03-21T10:28:02.930 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:28:02.929+0000 7f813425e740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f810c0109a0 con 0x55ca46896f70 2024-03-21T10:28:02.930 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:28:02.929+0000 7f813425e740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f810c00dea0 con 0x55ca4689e6c0 2024-03-21T10:28:02.930 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:28:02.930+0000 7f812b400640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ca4689e6c0 0x55ca4691a000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:28:02.930 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:28:02.930+0000 7f812b400640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ca4689e6c0 0x55ca4691a000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.47:54926/0 (socket says 172.21.15.47:54926) 2024-03-21T10:28:02.930 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:28:02.930+0000 7f812b400640 1 -- 172.21.15.47:0/767153951 learned_addr learned my addr 172.21.15.47:0/767153951 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:28:02.931 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:28:02.931+0000 7f812a000640 1 --2- 172.21.15.47:0/767153951 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ca46896f70 0x55ca468a2390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:28:02.931 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:28:02.931+0000 7f813425e740 10 client.0 _ll_register_callbacks cb 0x7f811c0f4cd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T10:28:02.931 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:28:02.931+0000 7f812aa00640 1 --2- 172.21.15.47:0/767153951 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ca46894fd0 0x55ca468a1e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:28:02.932 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:28:02.932+0000 7f812b400640 1 -- 172.21.15.47:0/767153951 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ca46896f70 msgr2=0x55ca468a2390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:28:02.932 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:28:02.932+0000 7f812b400640 1 --2- 172.21.15.47:0/767153951 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ca46896f70 0x55ca468a2390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:28:02.932 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:28:02.932+0000 7f812b400640 1 -- 172.21.15.47:0/767153951 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ca46894fd0 msgr2=0x55ca468a1e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:28:02.932 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:28:02.932+0000 7f812b400640 1 --2- 172.21.15.47:0/767153951 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ca46894fd0 0x55ca468a1e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:28:02.933 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:28:02.933+0000 7f812b400640 1 -- 172.21.15.47:0/767153951 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55ca4691a540 con 0x55ca4689e6c0 2024-03-21T10:28:02.933 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:28:02.933+0000 7f812aa00640 1 --2- 172.21.15.47:0/767153951 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ca46894fd0 0x55ca468a1e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T10:28:02.933 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:28:02.933+0000 7f812a000640 1 --2- 172.21.15.47:0/767153951 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ca46896f70 0x55ca468a2390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T10:28:02.934 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:28:02.934+0000 7f812b400640 1 --2- 172.21.15.47:0/767153951 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ca4689e6c0 0x55ca4691a000 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f8118011010 tx=0x7f81180058e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b34eab651674ff4f server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:28:02.934 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:28:02.934+0000 7f810be00640 10 client.0 ms_handle_connect on v2:172.21.15.157:3300/0 2024-03-21T10:28:02.935 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:28:02.935+0000 7f810be00640 1 -- 172.21.15.47:0/767153951 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8118037060 con 0x55ca4689e6c0 2024-03-21T10:28:02.935 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:28:02.935+0000 7f810be00640 1 -- 172.21.15.47:0/767153951 <== mon.1 v2:172.21.15.157:3300/0 2 ==== osd_map(97..97 src has 1..97) v4 ==== 9474+0+0 (secure 0 0 0) 0x7f8118005450 con 0x55ca4689e6c0 2024-03-21T10:28:02.935 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:28:02.935+0000 7f813425e740 1 -- 172.21.15.47:0/767153951 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f812400aba0 con 0x55ca4689e6c0 2024-03-21T10:28:02.935 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:28:02.935+0000 7f810be00640 1 -- 172.21.15.47:0/767153951 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f811803e050 con 0x55ca4689e6c0 2024-03-21T10:28:02.935 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:28:02.935+0000 7f813425e740 10 client.26251 fetch_fsmap learned FSMap version 32 2024-03-21T10:28:02.935 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:28:02.935+0000 7f813425e740 1 -- 172.21.15.47:0/767153951 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x55ca4691b6f0 con 0x55ca4689e6c0 2024-03-21T10:28:02.938 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:28:02.936+0000 7f810be00640 1 -- 172.21.15.47:0/767153951 <== mon.1 v2:172.21.15.157:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f812400aba0 con 0x55ca4689e6c0 2024-03-21T10:28:02.938 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:28:02.936+0000 7f810be00640 1 -- 172.21.15.47:0/767153951 <== mon.1 v2:172.21.15.157:3300/0 5 ==== fsmap(e 32) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f81180326a0 con 0x55ca4689e6c0 2024-03-21T10:28:02.938 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:28:02.936+0000 7f813425e740 10 client.26251 fetch_fsmap finished waiting for FSMap version 32 2024-03-21T10:28:02.938 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:28:02.936+0000 7f813425e740 10 client.26251 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-21T10:28:02.938 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:28:02.936+0000 7f813425e740 20 client.26251 populate_metadata read hostname 'smithi047' 2024-03-21T10:28:02.939 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:28:02.937+0000 7f813425e740 1 --2- 172.21.15.47:0/767153951 >> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] conn(0x55ca4691bc10 0x55ca4691e030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:28:02.939 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:28:02.937+0000 7f813425e740 4 client.26251 mds_command: new command op to 34535 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-21T10:28:02.939 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:28:02.937+0000 7f813425e740 1 -- 172.21.15.47:0/767153951 --> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55ca468010c0 con 0x55ca4691bc10 2024-03-21T10:28:02.939 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:28:02.939+0000 7f812aa00640 1 --2- 172.21.15.47:0/767153951 >> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] conn(0x55ca4691bc10 0x55ca4691e030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:28:02.940 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:28:02.940+0000 7f812aa00640 1 --2- 172.21.15.47:0/767153951 >> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] conn(0x55ca4691bc10 0x55ca4691e030 crc :-1 s=READY pgs=108 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2954525c18859049 server_cookie=943015eef12207a2 in_seq=0 out_seq=0 2024-03-21T10:28:02.940 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:28:02.940+0000 7f810be00640 10 client.26251 ms_handle_connect on v2:172.21.15.100:6838/4173168440 2024-03-21T10:28:02.999 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:28:02.999+0000 7f810be00640 1 -- 172.21.15.47:0/767153951 <== mds.0 v2:172.21.15.100:6838/4173168440 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55ca468010c0 con 0x55ca4691bc10 2024-03-21T10:28:02.999 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:28:02.999+0000 7f810be00640 10 client.26251 handle_command_reply: tid=0 2024-03-21T10:28:02.999 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:28:02.999+0000 7f813425e740 1 client.26251 shutdown 2024-03-21T10:28:02.999 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:28:02.999+0000 7f813425e740 1 -- 172.21.15.47:0/767153951 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ca4689e6c0 msgr2=0x55ca4691a000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:28:03.000 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:28:02.999+0000 7f813425e740 1 --2- 172.21.15.47:0/767153951 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ca4689e6c0 0x55ca4691a000 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f8118011010 tx=0x7f81180058e0 comp rx=0 tx=0).stop 2024-03-21T10:28:03.000 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:28:03.000+0000 7f813425e740 1 -- 172.21.15.47:0/767153951 shutdown_connections 2024-03-21T10:28:03.000 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:28:03.000+0000 7f813425e740 1 --2- 172.21.15.47:0/767153951 >> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] conn(0x55ca4691bc10 0x55ca4691e030 crc :-1 s=READY pgs=108 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:28:03.000 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:28:03.000+0000 7f813425e740 1 --2- 172.21.15.47:0/767153951 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ca4689e6c0 0x55ca4691a000 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:28:03.000 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:28:03.000+0000 7f813425e740 1 --2- 172.21.15.47:0/767153951 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ca46896f70 0x55ca468a2390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:28:03.000 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:28:03.000+0000 7f813425e740 1 --2- 172.21.15.47:0/767153951 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ca46894fd0 0x55ca468a1e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:28:03.000 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:28:03.000+0000 7f813425e740 1 -- 172.21.15.47:0/767153951 >> 172.21.15.47:0/767153951 conn(0x55ca4688f8f0 msgr2=0x55ca4688e950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:28:03.000 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:28:03.000+0000 7f810be00640 0 client.26251 ms_handle_reset on v2:172.21.15.100:6838/4173168440 2024-03-21T10:28:03.001 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:28:03.000+0000 7f813425e740 1 -- 172.21.15.47:0/767153951 shutdown_connections 2024-03-21T10:28:03.001 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:28:03.000+0000 7f813425e740 1 -- 172.21.15.47:0/767153951 wait complete. 2024-03-21T10:28:03.001 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:28:03.000+0000 7f813425e740 20 client.26251 trim_cache size 0 max 16384 2024-03-21T10:28:03.001 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout:{ 2024-03-21T10:28:03.001 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout: "message": "", 2024-03-21T10:28:03.001 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout: "return_code": 0 2024-03-21T10:28:03.001 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout:} 2024-03-21T10:28:03.002 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:28:03.002+0000 7f8131600640 1 -- 172.21.15.47:0/4031400535 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f81100a3180 msgr2=0x7f81100a5640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:28:03.002 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:28:03.002+0000 7f8131600640 1 --2- 172.21.15.47:0/4031400535 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f81100a3180 0x7f81100a5640 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f811801f000 tx=0x7f8118020000 comp rx=0 tx=0).stop 2024-03-21T10:28:03.002 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:28:03.002+0000 7f8131600640 1 -- 172.21.15.47:0/4031400535 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f812c10f1a0 msgr2=0x7f812c1155d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:28:03.002 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:28:03.002+0000 7f8131600640 1 --2- 172.21.15.47:0/4031400535 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f812c10f1a0 0x7f812c1155d0 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7f811c036040 tx=0x7f811c02ca10 comp rx=0 tx=0).stop 2024-03-21T10:28:03.003 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:28:03.002+0000 7f8131600640 1 -- 172.21.15.47:0/4031400535 shutdown_connections 2024-03-21T10:28:03.003 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:28:03.002+0000 7f8131600640 1 --2- 172.21.15.47:0/4031400535 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f81100a3180 0x7f81100a5640 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:28:03.003 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:28:03.002+0000 7f8131600640 1 --2- 172.21.15.47:0/4031400535 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f812c115b10 0x7f812c1b2ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:28:03.003 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:28:03.002+0000 7f8131600640 1 --2- 172.21.15.47:0/4031400535 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f812c10f1a0 0x7f812c1155d0 unknown :-1 s=CLOSED pgs=736 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:28:03.003 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:28:03.002+0000 7f8131600640 1 --2- 172.21.15.47:0/4031400535 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f812c10a0d0 0x7f812c115090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:28:03.003 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:28:03.002+0000 7f8131600640 1 -- 172.21.15.47:0/4031400535 >> 172.21.15.47:0/4031400535 conn(0x7f812c06e830 msgr2=0x7f812c06edf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:28:03.003 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:28:03.003+0000 7f8131600640 1 -- 172.21.15.47:0/4031400535 shutdown_connections 2024-03-21T10:28:03.003 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:28:03.003+0000 7f8131600640 1 -- 172.21.15.47:0/4031400535 wait complete. 2024-03-21T10:28:03.962 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:28:03 smithi047 ceph-mon[31547]: pgmap v85: 129 pgs: 129 active+clean; 277 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 11 MiB/s wr, 2.37k op/s 2024-03-21T10:28:03.978 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:28:03 smithi100 ceph-mon[35193]: pgmap v85: 129 pgs: 129 active+clean; 277 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 11 MiB/s wr, 2.37k op/s 2024-03-21T10:28:03.988 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:28:03 smithi157 ceph-mon[35581]: pgmap v85: 129 pgs: 129 active+clean; 277 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 11 MiB/s wr, 2.37k op/s 2024-03-21T10:28:04.963 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:28:04 smithi047 ceph-mon[31547]: scrub summary: idle 2024-03-21T10:28:04.978 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:28:04 smithi100 ceph-mon[35193]: scrub summary: idle 2024-03-21T10:28:04.988 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:28:04 smithi157 ceph-mon[35581]: scrub summary: idle 2024-03-21T10:28:05.963 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:28:05 smithi047 ceph-mon[31547]: pgmap v86: 129 pgs: 129 active+clean; 277 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 7.2 MiB/s wr, 1.57k op/s 2024-03-21T10:28:05.978 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:28:05 smithi100 ceph-mon[35193]: pgmap v86: 129 pgs: 129 active+clean; 277 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 7.2 MiB/s wr, 1.57k op/s 2024-03-21T10:28:05.988 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:28:05 smithi157 ceph-mon[35581]: pgmap v86: 129 pgs: 129 active+clean; 277 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 7.2 MiB/s wr, 1.57k op/s 2024-03-21T10:28:07.962 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:28:07 smithi047 ceph-mon[31547]: pgmap v87: 129 pgs: 129 active+clean; 301 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 9.7 MiB/s wr, 2.08k op/s 2024-03-21T10:28:07.978 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:28:07 smithi100 ceph-mon[35193]: pgmap v87: 129 pgs: 129 active+clean; 301 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 9.7 MiB/s wr, 2.08k op/s 2024-03-21T10:28:07.988 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:28:07 smithi157 ceph-mon[35581]: pgmap v87: 129 pgs: 129 active+clean; 301 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 9.7 MiB/s wr, 2.08k op/s 2024-03-21T10:28:09.962 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:28:09 smithi047 ceph-mon[31547]: pgmap v88: 129 pgs: 129 active+clean; 311 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 7.6 MiB/s wr, 1.66k op/s 2024-03-21T10:28:09.978 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:28:09 smithi100 ceph-mon[35193]: pgmap v88: 129 pgs: 129 active+clean; 311 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 7.6 MiB/s wr, 1.66k op/s 2024-03-21T10:28:09.988 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:28:09 smithi157 ceph-mon[35581]: pgmap v88: 129 pgs: 129 active+clean; 311 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 7.6 MiB/s wr, 1.66k op/s 2024-03-21T10:28:11.962 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:28:11 smithi047 ceph-mon[31547]: pgmap v89: 129 pgs: 129 active+clean; 311 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 7.1 MiB/s wr, 1.54k op/s 2024-03-21T10:28:11.979 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:28:11 smithi100 ceph-mon[35193]: pgmap v89: 129 pgs: 129 active+clean; 311 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 7.1 MiB/s wr, 1.54k op/s 2024-03-21T10:28:11.989 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:28:11 smithi157 ceph-mon[35581]: pgmap v89: 129 pgs: 129 active+clean; 311 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 7.1 MiB/s wr, 1.54k op/s 2024-03-21T10:28:13.962 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:28:13 smithi047 ceph-mon[31547]: pgmap v90: 129 pgs: 129 active+clean; 344 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 11 MiB/s wr, 2.35k op/s 2024-03-21T10:28:13.978 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:28:13 smithi100 ceph-mon[35193]: pgmap v90: 129 pgs: 129 active+clean; 344 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 11 MiB/s wr, 2.35k op/s 2024-03-21T10:28:13.988 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:28:13 smithi157 ceph-mon[35581]: pgmap v90: 129 pgs: 129 active+clean; 344 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 11 MiB/s wr, 2.35k op/s 2024-03-21T10:28:15.962 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:28:15 smithi047 ceph-mon[31547]: pgmap v91: 129 pgs: 129 active+clean; 344 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 7.1 MiB/s wr, 1.53k op/s 2024-03-21T10:28:15.977 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:28:15 smithi100 ceph-mon[35193]: pgmap v91: 129 pgs: 129 active+clean; 344 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 7.1 MiB/s wr, 1.53k op/s 2024-03-21T10:28:15.988 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:28:15 smithi157 ceph-mon[35581]: pgmap v91: 129 pgs: 129 active+clean; 344 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 7.1 MiB/s wr, 1.53k op/s 2024-03-21T10:28:16.206 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:28:16.205+0000 7f3b2cc00640 1 -- 172.21.15.47:0/4223708695 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3b28108c70 msgr2=0x7f3b281090d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:28:16.206 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:28:16.205+0000 7f3b2cc00640 1 --2- 172.21.15.47:0/4223708695 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3b28108c70 0x7f3b281090d0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f3b14009860 tx=0x7f3b1402bb70 comp rx=0 tx=0).stop 2024-03-21T10:28:16.206 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:28:16.205+0000 7f3b2cc00640 1 -- 172.21.15.47:0/4223708695 shutdown_connections 2024-03-21T10:28:16.206 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:28:16.205+0000 7f3b2cc00640 1 --2- 172.21.15.47:0/4223708695 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3b28114340 0x7f3b28116750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:28:16.206 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:28:16.205+0000 7f3b2cc00640 1 --2- 172.21.15.47:0/4223708695 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3b28108c70 0x7f3b281090d0 secure :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f3b14009860 tx=0x7f3b1402bb70 comp rx=0 tx=0).stop 2024-03-21T10:28:16.206 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:28:16.205+0000 7f3b2cc00640 1 --2- 172.21.15.47:0/4223708695 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3b2806d980 0x7f3b281086a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:28:16.206 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:28:16.205+0000 7f3b2cc00640 1 -- 172.21.15.47:0/4223708695 >> 172.21.15.47:0/4223708695 conn(0x7f3b2806c5b0 msgr2=0x7f3b2806c9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:28:16.206 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:28:16.206+0000 7f3b2cc00640 1 -- 172.21.15.47:0/4223708695 shutdown_connections 2024-03-21T10:28:16.207 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:28:16.206+0000 7f3b2cc00640 1 -- 172.21.15.47:0/4223708695 wait complete. 2024-03-21T10:28:16.207 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:28:16.207+0000 7f3b2cc00640 1 Processor -- start 2024-03-21T10:28:16.207 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:28:16.207+0000 7f3b2cc00640 1 -- start start 2024-03-21T10:28:16.207 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:28:16.208+0000 7f3b2cc00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3b2806d980 0x7f3b281a8680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:28:16.208 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:28:16.208+0000 7f3b2cc00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3b28114340 0x7f3b281a8bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:28:16.208 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:28:16.208+0000 7f3b2cc00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3b281a2740 0x7f3b281a2bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:28:16.208 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:28:16.208+0000 7f3b2cc00640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b28119f00 con 0x7f3b28114340 2024-03-21T10:28:16.208 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:28:16.208+0000 7f3b2cc00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b28119d80 con 0x7f3b2806d980 2024-03-21T10:28:16.208 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:28:16.208+0000 7f3b2cc00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b2811a080 con 0x7f3b281a2740 2024-03-21T10:28:16.208 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:28:16.208+0000 7f3b27400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3b2806d980 0x7f3b281a8680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:28:16.208 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:28:16.208+0000 7f3b27400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3b2806d980 0x7f3b281a8680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:33482/0 (socket says 172.21.15.47:33482) 2024-03-21T10:28:16.208 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:28:16.208+0000 7f3b27400640 1 -- 172.21.15.47:0/593080153 learned_addr learned my addr 172.21.15.47:0/593080153 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:28:16.209 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:28:16.209+0000 7f3b26a00640 1 --2- 172.21.15.47:0/593080153 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3b28114340 0x7f3b281a8bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:28:16.209 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:28:16.209+0000 7f3b27e00640 1 --2- 172.21.15.47:0/593080153 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3b281a2740 0x7f3b281a2bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:28:16.210 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:28:16.210+0000 7f3b26a00640 1 -- 172.21.15.47:0/593080153 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3b281a2740 msgr2=0x7f3b281a2bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:28:16.210 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:28:16.210+0000 7f3b26a00640 1 --2- 172.21.15.47:0/593080153 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3b281a2740 0x7f3b281a2bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:28:16.210 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:28:16.210+0000 7f3b26a00640 1 -- 172.21.15.47:0/593080153 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3b2806d980 msgr2=0x7f3b281a8680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:28:16.210 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:28:16.210+0000 7f3b26a00640 1 --2- 172.21.15.47:0/593080153 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3b2806d980 0x7f3b281a8680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:28:16.210 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:28:16.210+0000 7f3b26a00640 1 -- 172.21.15.47:0/593080153 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3b281a3340 con 0x7f3b28114340 2024-03-21T10:28:16.211 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:28:16.211+0000 7f3b26a00640 1 --2- 172.21.15.47:0/593080153 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3b28114340 0x7f3b281a8bc0 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7f3b281097c0 tx=0x7f3b140042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:28:16.211 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:28:16.212+0000 7f3b1fe00640 1 -- 172.21.15.47:0/593080153 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b14039040 con 0x7f3b28114340 2024-03-21T10:28:16.211 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:28:16.212+0000 7f3b1fe00640 1 -- 172.21.15.47:0/593080153 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f3b14034b80 con 0x7f3b28114340 2024-03-21T10:28:16.211 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:28:16.212+0000 7f3b1fe00640 1 -- 172.21.15.47:0/593080153 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b1403c390 con 0x7f3b28114340 2024-03-21T10:28:16.213 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:28:16.213+0000 7f3b2cc00640 1 -- 172.21.15.47:0/593080153 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3b281a35d0 con 0x7f3b28114340 2024-03-21T10:28:16.213 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:28:16.213+0000 7f3b2cc00640 1 -- 172.21.15.47:0/593080153 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3b281af660 con 0x7f3b28114340 2024-03-21T10:28:16.218 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:28:16.219+0000 7f3b30fc2740 1 Processor -- start 2024-03-21T10:28:16.219 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:28:16.219+0000 7f3b30fc2740 1 -- start start 2024-03-21T10:28:16.220 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:28:16.221+0000 7f3b30fc2740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c421eee1a0 0x55c421eee580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:28:16.221 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:28:16.221+0000 7f3b30fc2740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c421eeeac0 0x55c421ef7350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:28:16.221 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:28:16.221+0000 7f3b30fc2740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c421ef7890 0x55c421ef9c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:28:16.221 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:28:16.221+0000 7f3b30fc2740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c421eb03b0 con 0x55c421eee1a0 2024-03-21T10:28:16.221 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:28:16.221+0000 7f3b30fc2740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c421e52830 con 0x55c421eeeac0 2024-03-21T10:28:16.221 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:28:16.221+0000 7f3b30fc2740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c421dfb5d0 con 0x55c421ef7890 2024-03-21T10:28:16.222 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:28:16.222+0000 7f3b27400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c421eeeac0 0x55c421ef7350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:28:16.223 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:28:16.222+0000 7f3b27400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c421eeeac0 0x55c421ef7350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:33486/0 (socket says 172.21.15.47:33486) 2024-03-21T10:28:16.223 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:28:16.222+0000 7f3b27400640 1 -- 172.21.15.47:0/4099342686 learned_addr learned my addr 172.21.15.47:0/4099342686 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:28:16.223 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:28:16.222+0000 7f3b27e00640 1 --2- 172.21.15.47:0/4099342686 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c421eee1a0 0x55c421eee580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:28:16.223 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:28:16.222+0000 7f3b1fe00640 1 -- 172.21.15.47:0/593080153 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 27) v1 ==== 137985+0+0 (secure 0 0 0) 0x7f3b140035f0 con 0x7f3b28114340 2024-03-21T10:28:16.223 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:28:16.223+0000 7f3b26a00640 1 --2- 172.21.15.47:0/4099342686 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c421ef7890 0x55c421ef9c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:28:16.223 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:28:16.223+0000 7f3b27400640 1 -- 172.21.15.47:0/4099342686 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c421ef7890 msgr2=0x55c421ef9c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:28:16.223 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:28:16.223+0000 7f3b27400640 1 --2- 172.21.15.47:0/4099342686 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c421ef7890 0x55c421ef9c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:28:16.223 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:28:16.223+0000 7f3b27400640 1 -- 172.21.15.47:0/4099342686 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c421eee1a0 msgr2=0x55c421eee580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:28:16.223 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:28:16.223+0000 7f3b27400640 1 --2- 172.21.15.47:0/4099342686 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c421eee1a0 0x55c421eee580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:28:16.223 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:28:16.223+0000 7f3b27400640 1 -- 172.21.15.47:0/4099342686 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c421e132c0 con 0x55c421eeeac0 2024-03-21T10:28:16.224 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:28:16.223+0000 7f3b27400640 1 --2- 172.21.15.47:0/4099342686 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c421eeeac0 0x55c421ef7350 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f3b1800b7e0 tx=0x7f3b1800bca0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=eadf7a0b4010ab3c server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:28:16.224 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:28:16.224+0000 7f3b13e00640 1 -- 172.21.15.47:0/4099342686 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b18003dd0 con 0x55c421eeeac0 2024-03-21T10:28:16.224 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:28:16.224+0000 7f3b13e00640 1 -- 172.21.15.47:0/4099342686 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f3b18015070 con 0x55c421eeeac0 2024-03-21T10:28:16.224 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:28:16.224+0000 7f3b13e00640 1 -- 172.21.15.47:0/4099342686 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b18019730 con 0x55c421eeeac0 2024-03-21T10:28:16.226 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:28:16.225+0000 7f3b1fe00640 1 --2- 172.21.15.47:0/593080153 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f3b0c0a3250 0x7f3b0c0a5710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:28:16.226 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:28:16.225+0000 7f3b1fe00640 1 -- 172.21.15.47:0/593080153 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 9474+0+0 (secure 0 0 0) 0x7f3b140f07c0 con 0x7f3b28114340 2024-03-21T10:28:16.226 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:28:16.226+0000 7f3b27e00640 1 --2- 172.21.15.47:0/593080153 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f3b0c0a3250 0x7f3b0c0a5710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:28:16.226 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:28:16.226+0000 7f3b30fc2740 1 -- 172.21.15.47:0/4099342686 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c421eeeac0 msgr2=0x55c421ef7350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:28:16.226 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:28:16.226+0000 7f3b30fc2740 1 --2- 172.21.15.47:0/4099342686 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c421eeeac0 0x55c421ef7350 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f3b1800b7e0 tx=0x7f3b1800bca0 comp rx=0 tx=0).stop 2024-03-21T10:28:16.227 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:28:16.227+0000 7f3b30fc2740 1 -- 172.21.15.47:0/4099342686 shutdown_connections 2024-03-21T10:28:16.227 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:28:16.227+0000 7f3b30fc2740 1 --2- 172.21.15.47:0/4099342686 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c421ef7890 0x55c421ef9c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:28:16.227 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:28:16.227+0000 7f3b30fc2740 1 --2- 172.21.15.47:0/4099342686 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c421eeeac0 0x55c421ef7350 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:28:16.227 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:28:16.227+0000 7f3b30fc2740 1 --2- 172.21.15.47:0/4099342686 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c421eee1a0 0x55c421eee580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:28:16.228 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:28:16.227+0000 7f3b30fc2740 1 -- 172.21.15.47:0/4099342686 >> 172.21.15.47:0/4099342686 conn(0x55c421ee8ac0 msgr2=0x55c421ee8ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:28:16.228 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:28:16.229+0000 7f3b27e00640 1 --2- 172.21.15.47:0/593080153 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f3b0c0a3250 0x7f3b0c0a5710 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f3b200092f0 tx=0x7f3b20009280 comp rx=0 tx=0).ready entity=mgr.15687 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:28:16.228 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:28:16.229+0000 7f3b30fc2740 1 -- 172.21.15.47:0/4099342686 shutdown_connections 2024-03-21T10:28:16.229 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:28:16.229+0000 7f3b30fc2740 1 -- 172.21.15.47:0/4099342686 wait complete. 2024-03-21T10:28:16.229 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:28:16.230+0000 7f3b30fc2740 1 -- start start 2024-03-21T10:28:16.230 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:28:16.230+0000 7f3b30fc2740 1 Processor -- start 2024-03-21T10:28:16.234 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:28:16.233+0000 7f3b30fc2740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c421eee1a0 0x55c421ef4880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:28:16.234 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:28:16.233+0000 7f3b30fc2740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c421eeeac0 0x55c421ef4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:28:16.234 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:28:16.233+0000 7f3b30fc2740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c421ef7890 0x55c421ef5300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:28:16.234 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:28:16.233+0000 7f3b30fc2740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c421e0ea20 con 0x55c421eeeac0 2024-03-21T10:28:16.234 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:28:16.233+0000 7f3b30fc2740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c421e4e8d0 con 0x55c421eee1a0 2024-03-21T10:28:16.235 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:28:16.233+0000 7f3b30fc2740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c421efa4c0 con 0x55c421ef7890 2024-03-21T10:28:16.235 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:28:16.233+0000 7f3b30fc2740 10 client.0 _ll_register_callbacks cb 0x55c421e4b1c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T10:28:16.235 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:28:16.235+0000 7f3b27400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c421eee1a0 0x55c421ef4880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:28:16.235 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:28:16.235+0000 7f3b27400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c421eee1a0 0x55c421ef4880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:33502/0 (socket says 172.21.15.47:33502) 2024-03-21T10:28:16.235 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:28:16.235+0000 7f3b27400640 1 -- 172.21.15.47:0/4242609630 learned_addr learned my addr 172.21.15.47:0/4242609630 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:28:16.237 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:28:16.237+0000 7f3b27400640 1 -- 172.21.15.47:0/4242609630 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c421ef7890 msgr2=0x55c421ef5300 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-21T10:28:16.237 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:28:16.237+0000 7f3b27400640 1 --2- 172.21.15.47:0/4242609630 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c421ef7890 0x55c421ef5300 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:28:16.237 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:28:16.237+0000 7f3b27400640 1 -- 172.21.15.47:0/4242609630 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c421eeeac0 msgr2=0x55c421ef4dc0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-21T10:28:16.237 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:28:16.237+0000 7f3b27400640 1 --2- 172.21.15.47:0/4242609630 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c421eeeac0 0x55c421ef4dc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:28:16.237 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:28:16.237+0000 7f3b27400640 1 -- 172.21.15.47:0/4242609630 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55c421b67210 con 0x55c421eee1a0 2024-03-21T10:28:16.237 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:28:16.237+0000 7f3b27400640 1 --2- 172.21.15.47:0/4242609630 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c421eee1a0 0x55c421ef4880 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f3b18003e00 tx=0x7f3b18003e30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2a59e0843bdf20e5 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:28:16.237 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:28:16.238+0000 7f3b10c00640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-21T10:28:16.238 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:28:16.238+0000 7f3b10c00640 1 -- 172.21.15.47:0/4242609630 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b18011650 con 0x55c421eee1a0 2024-03-21T10:28:16.238 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:28:16.238+0000 7f3b10c00640 1 -- 172.21.15.47:0/4242609630 <== mon.2 v2:172.21.15.100:3300/0 2 ==== osd_map(97..97 src has 1..97) v4 ==== 9474+0+0 (secure 0 0 0) 0x7f3b180117f0 con 0x55c421eee1a0 2024-03-21T10:28:16.239 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:28:16.239+0000 7f3b30fc2740 1 -- 172.21.15.47:0/4242609630 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f3b18015070 con 0x55c421eee1a0 2024-03-21T10:28:16.239 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:28:16.240+0000 7f3b10c00640 1 -- 172.21.15.47:0/4242609630 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3b180119f0 con 0x55c421eee1a0 2024-03-21T10:28:16.240 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:28:16.240+0000 7f3b30fc2740 10 client.36512 fetch_fsmap learned FSMap version 32 2024-03-21T10:28:16.240 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:28:16.240+0000 7f3b30fc2740 1 -- 172.21.15.47:0/4242609630 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x55c421c33620 con 0x55c421eee1a0 2024-03-21T10:28:16.241 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:28:16.241+0000 7f3b10c00640 1 -- 172.21.15.47:0/4242609630 <== mon.2 v2:172.21.15.100:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f3b18015070 con 0x55c421eee1a0 2024-03-21T10:28:16.244 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:28:16.241+0000 7f3b10c00640 1 -- 172.21.15.47:0/4242609630 <== mon.2 v2:172.21.15.100:3300/0 5 ==== fsmap(e 32) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f3b18025430 con 0x55c421eee1a0 2024-03-21T10:28:16.244 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:28:16.241+0000 7f3b30fc2740 10 client.36512 fetch_fsmap finished waiting for FSMap version 32 2024-03-21T10:28:16.244 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:28:16.241+0000 7f3b30fc2740 10 client.36512 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-21T10:28:16.244 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:28:16.241+0000 7f3b30fc2740 20 client.36512 populate_metadata read hostname 'smithi047' 2024-03-21T10:28:16.244 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:28:16.242+0000 7f3b30fc2740 1 --2- 172.21.15.47:0/4242609630 >> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] conn(0x55c421f07820 0x55c421f09c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:28:16.245 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:28:16.242+0000 7f3b30fc2740 4 client.36512 mds_command: new command op to 34535 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T10:28:16.245 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:28:16.242+0000 7f3b30fc2740 1 -- 172.21.15.47:0/4242609630 --> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55c421e5fd70 con 0x55c421f07820 2024-03-21T10:28:16.245 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:28:16.245+0000 7f3b26a00640 1 --2- 172.21.15.47:0/4242609630 >> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] conn(0x55c421f07820 0x55c421f09c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:28:16.246 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:28:16.247+0000 7f3b26a00640 1 --2- 172.21.15.47:0/4242609630 >> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] conn(0x55c421f07820 0x55c421f09c10 crc :-1 s=READY pgs=109 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e722d75a24490d3d server_cookie=c79192992e1d0f1 in_seq=0 out_seq=0 2024-03-21T10:28:16.246 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:28:16.247+0000 7f3b10c00640 10 client.36512 ms_handle_connect on v2:172.21.15.100:6838/4173168440 2024-03-21T10:28:16.247 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:28:16.248+0000 7f3b10c00640 1 -- 172.21.15.47:0/4242609630 <== mds.0 v2:172.21.15.100:6838/4173168440 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55c421e5fd70 con 0x55c421f07820 2024-03-21T10:28:16.247 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:28:16.248+0000 7f3b10c00640 10 client.36512 handle_command_reply: tid=0 2024-03-21T10:28:16.248 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:28:16.248+0000 7f3b30fc2740 1 client.36512 shutdown 2024-03-21T10:28:16.249 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:28:16.249+0000 7f3b30fc2740 1 -- 172.21.15.47:0/4242609630 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c421eee1a0 msgr2=0x55c421ef4880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:28:16.249 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:28:16.249+0000 7f3b30fc2740 1 --2- 172.21.15.47:0/4242609630 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c421eee1a0 0x55c421ef4880 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f3b18003e00 tx=0x7f3b18003e30 comp rx=0 tx=0).stop 2024-03-21T10:28:16.250 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:28:16.250+0000 7f3b30fc2740 1 -- 172.21.15.47:0/4242609630 shutdown_connections 2024-03-21T10:28:16.250 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:28:16.250+0000 7f3b30fc2740 1 --2- 172.21.15.47:0/4242609630 >> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] conn(0x55c421f07820 0x55c421f09c10 crc :-1 s=READY pgs=109 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:28:16.250 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:28:16.250+0000 7f3b30fc2740 1 --2- 172.21.15.47:0/4242609630 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c421ef7890 0x55c421ef5300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:28:16.251 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:28:16.250+0000 7f3b30fc2740 1 --2- 172.21.15.47:0/4242609630 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c421eeeac0 0x55c421ef4dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:28:16.251 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:28:16.250+0000 7f3b30fc2740 1 --2- 172.21.15.47:0/4242609630 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c421eee1a0 0x55c421ef4880 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:28:16.251 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:28:16.250+0000 7f3b30fc2740 1 -- 172.21.15.47:0/4242609630 >> 172.21.15.47:0/4242609630 conn(0x55c421ee8ac0 msgr2=0x55c421eea810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:28:16.251 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:28:16.250+0000 7f3b10c00640 0 client.36512 ms_handle_reset on v2:172.21.15.100:6838/4173168440 2024-03-21T10:28:16.251 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:28:16.250+0000 7f3b30fc2740 1 -- 172.21.15.47:0/4242609630 shutdown_connections 2024-03-21T10:28:16.251 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:28:16.250+0000 7f3b30fc2740 1 -- 172.21.15.47:0/4242609630 wait complete. 2024-03-21T10:28:16.252 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:28:16.252+0000 7f3b30fc2740 20 client.36512 trim_cache size 0 max 16384 2024-03-21T10:28:16.292 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:28:16.292+0000 7f3b30fc2740 1 Processor -- start 2024-03-21T10:28:16.292 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:28:16.292+0000 7f3b30fc2740 1 -- start start 2024-03-21T10:28:16.293 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:28:16.293+0000 7f3b30fc2740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c421eee1a0 0x55c421ef6f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:28:16.293 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:28:16.293+0000 7f3b30fc2740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c421eeeac0 0x55c421eea620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:28:16.293 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:28:16.293+0000 7f3b30fc2740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c421ef7890 0x55c421f02bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:28:16.293 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:28:16.293+0000 7f3b30fc2740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b281b9760 con 0x55c421eee1a0 2024-03-21T10:28:16.293 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:28:16.293+0000 7f3b30fc2740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b281bb0e0 con 0x55c421ef7890 2024-03-21T10:28:16.293 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:28:16.293+0000 7f3b30fc2740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b281b9dc0 con 0x55c421eeeac0 2024-03-21T10:28:16.294 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:28:16.294+0000 7f3b27e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c421ef7890 0x55c421f02bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:28:16.294 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:28:16.294+0000 7f3b27e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c421ef7890 0x55c421f02bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:33508/0 (socket says 172.21.15.47:33508) 2024-03-21T10:28:16.294 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:28:16.294+0000 7f3b26a00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c421eeeac0 0x55c421eea620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:28:16.294 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:28:16.294+0000 7f3b26a00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c421eeeac0 0x55c421eea620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.47:55070/0 (socket says 172.21.15.47:55070) 2024-03-21T10:28:16.294 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:28:16.294+0000 7f3b26a00640 1 -- 172.21.15.47:0/1570299142 learned_addr learned my addr 172.21.15.47:0/1570299142 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:28:16.294 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:28:16.294+0000 7f3b27400640 1 --2- 172.21.15.47:0/1570299142 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c421eee1a0 0x55c421ef6f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:28:16.295 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:28:16.295+0000 7f3b27e00640 1 -- 172.21.15.47:0/1570299142 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c421eeeac0 msgr2=0x55c421eea620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:28:16.295 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:28:16.295+0000 7f3b27e00640 1 --2- 172.21.15.47:0/1570299142 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c421eeeac0 0x55c421eea620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:28:16.295 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:28:16.295+0000 7f3b27e00640 1 -- 172.21.15.47:0/1570299142 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c421eee1a0 msgr2=0x55c421ef6f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:28:16.296 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:28:16.295+0000 7f3b27e00640 1 --2- 172.21.15.47:0/1570299142 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c421eee1a0 0x55c421ef6f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:28:16.296 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:28:16.295+0000 7f3b27e00640 1 -- 172.21.15.47:0/1570299142 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c421ee9690 con 0x55c421ef7890 2024-03-21T10:28:16.296 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:28:16.296+0000 7f3b27e00640 1 --2- 172.21.15.47:0/1570299142 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c421ef7890 0x55c421f02bd0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f3b20005fd0 tx=0x7f3b20014000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=832720beef1344a3 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:28:16.297 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:28:16.297+0000 7f3b13400640 1 -- 172.21.15.47:0/1570299142 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b2001f6e0 con 0x55c421ef7890 2024-03-21T10:28:16.300 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:28:16.298+0000 7f3b13400640 1 -- 172.21.15.47:0/1570299142 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f3b2001fd30 con 0x55c421ef7890 2024-03-21T10:28:16.300 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:28:16.298+0000 7f3b13400640 1 -- 172.21.15.47:0/1570299142 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b20027d20 con 0x55c421ef7890 2024-03-21T10:28:16.300 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:28:16.299+0000 7f3b30fc2740 1 -- 172.21.15.47:0/1570299142 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c421ef7890 msgr2=0x55c421f02bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:28:16.300 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:28:16.299+0000 7f3b30fc2740 1 --2- 172.21.15.47:0/1570299142 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c421ef7890 0x55c421f02bd0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f3b20005fd0 tx=0x7f3b20014000 comp rx=0 tx=0).stop 2024-03-21T10:28:16.300 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:28:16.300+0000 7f3b30fc2740 1 -- 172.21.15.47:0/1570299142 shutdown_connections 2024-03-21T10:28:16.300 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:28:16.300+0000 7f3b30fc2740 1 --2- 172.21.15.47:0/1570299142 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c421ef7890 0x55c421f02bd0 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:28:16.301 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:28:16.300+0000 7f3b30fc2740 1 --2- 172.21.15.47:0/1570299142 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c421eeeac0 0x55c421eea620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:28:16.301 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:28:16.300+0000 7f3b30fc2740 1 --2- 172.21.15.47:0/1570299142 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c421eee1a0 0x55c421ef6f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:28:16.301 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:28:16.300+0000 7f3b30fc2740 1 -- 172.21.15.47:0/1570299142 >> 172.21.15.47:0/1570299142 conn(0x55c421ee8ac0 msgr2=0x55c421ee9f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:28:16.301 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:28:16.301+0000 7f3b30fc2740 1 -- 172.21.15.47:0/1570299142 shutdown_connections 2024-03-21T10:28:16.301 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:28:16.301+0000 7f3b30fc2740 1 -- 172.21.15.47:0/1570299142 wait complete. 2024-03-21T10:28:16.301 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:28:16.301+0000 7f3b30fc2740 1 -- start start 2024-03-21T10:28:16.301 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:28:16.301+0000 7f3b30fc2740 1 Processor -- start 2024-03-21T10:28:16.303 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:28:16.303+0000 7f3b30fc2740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c421eee1a0 0x55c421ef1720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:28:16.303 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:28:16.303+0000 7f3b30fc2740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c421eeeac0 0x55c421ef1c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:28:16.303 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:28:16.303+0000 7f3b30fc2740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c421ef7890 0x55c421f6a6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:28:16.303 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:28:16.303+0000 7f3b30fc2740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b281baa80 con 0x55c421eee1a0 2024-03-21T10:28:16.303 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:28:16.303+0000 7f3b30fc2740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b281ba420 con 0x55c421eeeac0 2024-03-21T10:28:16.303 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:28:16.303+0000 7f3b30fc2740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b281b7920 con 0x55c421ef7890 2024-03-21T10:28:16.303 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:28:16.303+0000 7f3b30fc2740 10 client.0 _ll_register_callbacks cb 0x7f3b180044b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T10:28:16.304 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:28:16.304+0000 7f3b27e00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c421ef7890 0x55c421f6a6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:28:16.304 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:28:16.304+0000 7f3b27e00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c421ef7890 0x55c421f6a6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.47:55086/0 (socket says 172.21.15.47:55086) 2024-03-21T10:28:16.304 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:28:16.304+0000 7f3b27e00640 1 -- 172.21.15.47:0/3912362366 learned_addr learned my addr 172.21.15.47:0/3912362366 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:28:16.304 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:28:16.305+0000 7f3b27400640 1 --2- 172.21.15.47:0/3912362366 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c421eee1a0 0x55c421ef1720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:28:16.305 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:28:16.305+0000 7f3b26a00640 1 --2- 172.21.15.47:0/3912362366 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c421eeeac0 0x55c421ef1c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:28:16.305 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:28:16.305+0000 7f3b27e00640 1 -- 172.21.15.47:0/3912362366 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c421eeeac0 msgr2=0x55c421ef1c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:28:16.305 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:28:16.305+0000 7f3b27e00640 1 --2- 172.21.15.47:0/3912362366 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c421eeeac0 0x55c421ef1c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:28:16.305 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:28:16.305+0000 7f3b27e00640 1 -- 172.21.15.47:0/3912362366 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c421eee1a0 msgr2=0x55c421ef1720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:28:16.305 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:28:16.305+0000 7f3b27e00640 1 --2- 172.21.15.47:0/3912362366 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c421eee1a0 0x55c421ef1720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:28:16.305 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:28:16.305+0000 7f3b27e00640 1 -- 172.21.15.47:0/3912362366 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55c421f5af50 con 0x55c421ef7890 2024-03-21T10:28:16.305 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:28:16.305+0000 7f3b27400640 1 --2- 172.21.15.47:0/3912362366 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c421eee1a0 0x55c421ef1720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T10:28:16.306 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:28:16.305+0000 7f3b27e00640 1 --2- 172.21.15.47:0/3912362366 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c421ef7890 0x55c421f6a6a0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f3b20027a00 tx=0x7f3b20027630 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a1ba60b69f600b56 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:28:16.306 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:28:16.305+0000 7f3b10c00640 10 client.0 ms_handle_connect on v2:172.21.15.157:3300/0 2024-03-21T10:28:16.306 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:28:16.306+0000 7f3b10c00640 1 -- 172.21.15.47:0/3912362366 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b20037060 con 0x55c421ef7890 2024-03-21T10:28:16.306 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:28:16.306+0000 7f3b10c00640 1 -- 172.21.15.47:0/3912362366 <== mon.1 v2:172.21.15.157:3300/0 2 ==== osd_map(97..97 src has 1..97) v4 ==== 9474+0+0 (secure 0 0 0) 0x7f3b20027cb0 con 0x55c421ef7890 2024-03-21T10:28:16.306 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:28:16.306+0000 7f3b30fc2740 1 -- 172.21.15.47:0/3912362366 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f3b2001fd30 con 0x55c421ef7890 2024-03-21T10:28:16.306 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:28:16.306+0000 7f3b10c00640 1 -- 172.21.15.47:0/3912362366 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3b2003e050 con 0x55c421ef7890 2024-03-21T10:28:16.306 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:28:16.306+0000 7f3b30fc2740 10 client.26263 fetch_fsmap learned FSMap version 32 2024-03-21T10:28:16.306 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:28:16.306+0000 7f3b30fc2740 1 -- 172.21.15.47:0/3912362366 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x55c421f5c220 con 0x55c421ef7890 2024-03-21T10:28:16.307 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:28:16.307+0000 7f3b10c00640 1 -- 172.21.15.47:0/3912362366 <== mon.1 v2:172.21.15.157:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f3b2001fd30 con 0x55c421ef7890 2024-03-21T10:28:16.307 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:28:16.307+0000 7f3b10c00640 1 -- 172.21.15.47:0/3912362366 <== mon.1 v2:172.21.15.157:3300/0 5 ==== fsmap(e 32) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f3b2000f040 con 0x55c421ef7890 2024-03-21T10:28:16.307 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:28:16.307+0000 7f3b30fc2740 10 client.26263 fetch_fsmap finished waiting for FSMap version 32 2024-03-21T10:28:16.307 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:28:16.307+0000 7f3b30fc2740 10 client.26263 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-21T10:28:16.307 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:28:16.307+0000 7f3b30fc2740 20 client.26263 populate_metadata read hostname 'smithi047' 2024-03-21T10:28:16.308 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:28:16.307+0000 7f3b30fc2740 1 --2- 172.21.15.47:0/3912362366 >> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] conn(0x55c421f5c950 0x55c421f5ed70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:28:16.308 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:28:16.307+0000 7f3b30fc2740 4 client.26263 mds_command: new command op to 34535 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-21T10:28:16.308 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:28:16.307+0000 7f3b30fc2740 1 -- 172.21.15.47:0/3912362366 --> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55c421e5a0c0 con 0x55c421f5c950 2024-03-21T10:28:16.310 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:28:16.310+0000 7f3b27400640 1 --2- 172.21.15.47:0/3912362366 >> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] conn(0x55c421f5c950 0x55c421f5ed70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:28:16.312 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:28:16.312+0000 7f3b27400640 1 --2- 172.21.15.47:0/3912362366 >> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] conn(0x55c421f5c950 0x55c421f5ed70 crc :-1 s=READY pgs=110 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b7cc979952ba2898 server_cookie=7601456fc2e823aa in_seq=0 out_seq=0 2024-03-21T10:28:16.312 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:28:16.312+0000 7f3b10c00640 10 client.26263 ms_handle_connect on v2:172.21.15.100:6838/4173168440 2024-03-21T10:28:16.391 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:28:16.392+0000 7f3b10c00640 1 -- 172.21.15.47:0/3912362366 <== mds.0 v2:172.21.15.100:6838/4173168440 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55c421e5a0c0 con 0x55c421f5c950 2024-03-21T10:28:16.392 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:28:16.392+0000 7f3b10c00640 10 client.26263 handle_command_reply: tid=0 2024-03-21T10:28:16.392 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:28:16.392+0000 7f3b30fc2740 1 client.26263 shutdown 2024-03-21T10:28:16.392 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:28:16.393+0000 7f3b30fc2740 1 -- 172.21.15.47:0/3912362366 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c421ef7890 msgr2=0x55c421f6a6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:28:16.392 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:28:16.393+0000 7f3b30fc2740 1 --2- 172.21.15.47:0/3912362366 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c421ef7890 0x55c421f6a6a0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f3b20027a00 tx=0x7f3b20027630 comp rx=0 tx=0).stop 2024-03-21T10:28:16.393 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:28:16.393+0000 7f3b30fc2740 1 -- 172.21.15.47:0/3912362366 shutdown_connections 2024-03-21T10:28:16.393 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:28:16.393+0000 7f3b30fc2740 1 --2- 172.21.15.47:0/3912362366 >> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] conn(0x55c421f5c950 0x55c421f5ed70 crc :-1 s=READY pgs=110 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:28:16.393 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:28:16.394+0000 7f3b30fc2740 1 --2- 172.21.15.47:0/3912362366 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c421ef7890 0x55c421f6a6a0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:28:16.393 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:28:16.394+0000 7f3b30fc2740 1 --2- 172.21.15.47:0/3912362366 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c421eeeac0 0x55c421ef1c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:28:16.393 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:28:16.394+0000 7f3b30fc2740 1 --2- 172.21.15.47:0/3912362366 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c421eee1a0 0x55c421ef1720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:28:16.394 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:28:16.394+0000 7f3b30fc2740 1 -- 172.21.15.47:0/3912362366 >> 172.21.15.47:0/3912362366 conn(0x55c421ee8ac0 msgr2=0x55c421ef50d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:28:16.394 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:28:16.394+0000 7f3b10c00640 0 client.26263 ms_handle_reset on v2:172.21.15.100:6838/4173168440 2024-03-21T10:28:16.394 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:28:16.394+0000 7f3b30fc2740 1 -- 172.21.15.47:0/3912362366 shutdown_connections 2024-03-21T10:28:16.394 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:28:16.394+0000 7f3b30fc2740 1 -- 172.21.15.47:0/3912362366 wait complete. 2024-03-21T10:28:16.394 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:28:16.395+0000 7f3b30fc2740 20 client.26263 trim_cache size 0 max 16384 2024-03-21T10:28:16.395 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout:{ 2024-03-21T10:28:16.395 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout: "message": "", 2024-03-21T10:28:16.395 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout: "return_code": 0 2024-03-21T10:28:16.395 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout:} 2024-03-21T10:28:16.395 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:28:16.395+0000 7f3b2cc00640 1 -- 172.21.15.47:0/593080153 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f3b0c0a3250 msgr2=0x7f3b0c0a5710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:28:16.396 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:28:16.395+0000 7f3b2cc00640 1 --2- 172.21.15.47:0/593080153 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f3b0c0a3250 0x7f3b0c0a5710 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f3b200092f0 tx=0x7f3b20009280 comp rx=0 tx=0).stop 2024-03-21T10:28:16.396 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:28:16.395+0000 7f3b2cc00640 1 -- 172.21.15.47:0/593080153 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3b28114340 msgr2=0x7f3b281a8bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:28:16.396 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:28:16.395+0000 7f3b2cc00640 1 --2- 172.21.15.47:0/593080153 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3b28114340 0x7f3b281a8bc0 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7f3b281097c0 tx=0x7f3b140042f0 comp rx=0 tx=0).stop 2024-03-21T10:28:16.396 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:28:16.396+0000 7f3b2cc00640 1 -- 172.21.15.47:0/593080153 shutdown_connections 2024-03-21T10:28:16.396 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:28:16.396+0000 7f3b2cc00640 1 --2- 172.21.15.47:0/593080153 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3b281a2740 0x7f3b281a2bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:28:16.396 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:28:16.396+0000 7f3b2cc00640 1 --2- 172.21.15.47:0/593080153 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3b28114340 0x7f3b281a8bc0 unknown :-1 s=CLOSED pgs=738 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:28:16.396 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:28:16.396+0000 7f3b2cc00640 1 --2- 172.21.15.47:0/593080153 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f3b0c0a3250 0x7f3b0c0a5710 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:28:16.396 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:28:16.396+0000 7f3b2cc00640 1 --2- 172.21.15.47:0/593080153 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3b2806d980 0x7f3b281a8680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:28:16.396 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:28:16.396+0000 7f3b2cc00640 1 -- 172.21.15.47:0/593080153 >> 172.21.15.47:0/593080153 conn(0x7f3b2806c5b0 msgr2=0x7f3b2810a040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:28:16.397 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:28:16.397+0000 7f3b2cc00640 1 -- 172.21.15.47:0/593080153 shutdown_connections 2024-03-21T10:28:16.397 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:28:16.397+0000 7f3b2cc00640 1 -- 172.21.15.47:0/593080153 wait complete. 2024-03-21T10:28:16.964 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:28:16 smithi047 ceph-mon[31547]: pgmap v92: 129 pgs: 129 active+clean; 364 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 9.8 MiB/s wr, 2.08k op/s 2024-03-21T10:28:16.978 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:28:16 smithi100 ceph-mon[35193]: pgmap v92: 129 pgs: 129 active+clean; 364 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 9.8 MiB/s wr, 2.08k op/s 2024-03-21T10:28:16.988 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:28:16 smithi157 ceph-mon[35581]: pgmap v92: 129 pgs: 129 active+clean; 364 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 9.8 MiB/s wr, 2.08k op/s 2024-03-21T10:28:19.462 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:28:19 smithi047 ceph-mon[31547]: pgmap v93: 129 pgs: 129 active+clean; 372 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 8.2 MiB/s wr, 1.80k op/s 2024-03-21T10:28:19.478 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:28:19 smithi100 ceph-mon[35193]: pgmap v93: 129 pgs: 129 active+clean; 372 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 8.2 MiB/s wr, 1.80k op/s 2024-03-21T10:28:19.488 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:28:19 smithi157 ceph-mon[35581]: pgmap v93: 129 pgs: 129 active+clean; 372 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 8.2 MiB/s wr, 1.80k op/s 2024-03-21T10:28:21.462 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:28:21 smithi047 ceph-mon[31547]: pgmap v94: 129 pgs: 129 active+clean; 372 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 7.3 MiB/s wr, 1.59k op/s 2024-03-21T10:28:21.478 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:28:21 smithi100 ceph-mon[35193]: pgmap v94: 129 pgs: 129 active+clean; 372 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 7.3 MiB/s wr, 1.59k op/s 2024-03-21T10:28:21.488 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:28:21 smithi157 ceph-mon[35581]: pgmap v94: 129 pgs: 129 active+clean; 372 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 7.3 MiB/s wr, 1.59k op/s 2024-03-21T10:28:23.464 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:28:23 smithi047 ceph-mon[31547]: pgmap v95: 129 pgs: 129 active+clean; 395 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 11 MiB/s wr, 2.37k op/s 2024-03-21T10:28:23.478 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:28:23 smithi100 ceph-mon[35193]: pgmap v95: 129 pgs: 129 active+clean; 395 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 11 MiB/s wr, 2.37k op/s 2024-03-21T10:28:23.488 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:28:23 smithi157 ceph-mon[35581]: pgmap v95: 129 pgs: 129 active+clean; 395 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 11 MiB/s wr, 2.37k op/s 2024-03-21T10:28:25.462 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:28:25 smithi047 ceph-mon[31547]: pgmap v96: 129 pgs: 129 active+clean; 395 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 7.3 MiB/s wr, 1.56k op/s 2024-03-21T10:28:25.478 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:28:25 smithi100 ceph-mon[35193]: pgmap v96: 129 pgs: 129 active+clean; 395 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 7.3 MiB/s wr, 1.56k op/s 2024-03-21T10:28:25.488 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:28:25 smithi157 ceph-mon[35581]: pgmap v96: 129 pgs: 129 active+clean; 395 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 7.3 MiB/s wr, 1.56k op/s 2024-03-21T10:28:27.463 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:28:27 smithi047 ceph-mon[31547]: pgmap v97: 129 pgs: 129 active+clean; 410 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 9.9 MiB/s wr, 2.12k op/s 2024-03-21T10:28:27.478 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:28:27 smithi100 ceph-mon[35193]: pgmap v97: 129 pgs: 129 active+clean; 410 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 9.9 MiB/s wr, 2.12k op/s 2024-03-21T10:28:27.488 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:28:27 smithi157 ceph-mon[35581]: pgmap v97: 129 pgs: 129 active+clean; 410 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 9.9 MiB/s wr, 2.12k op/s 2024-03-21T10:28:29.462 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:28:29 smithi047 ceph-mon[31547]: pgmap v98: 129 pgs: 129 active+clean; 416 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 8.2 MiB/s wr, 1.80k op/s 2024-03-21T10:28:29.477 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:28:29 smithi100 ceph-mon[35193]: pgmap v98: 129 pgs: 129 active+clean; 416 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 8.2 MiB/s wr, 1.80k op/s 2024-03-21T10:28:29.488 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:28:29 smithi157 ceph-mon[35581]: pgmap v98: 129 pgs: 129 active+clean; 416 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 8.2 MiB/s wr, 1.80k op/s 2024-03-21T10:28:29.614 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:28:29.614+0000 7f3fe4200640 1 -- 172.21.15.47:0/680434390 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3fdc10c2c0 msgr2=0x7f3fdc10c6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:28:29.614 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:28:29.614+0000 7f3fe4200640 1 --2- 172.21.15.47:0/680434390 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3fdc10c2c0 0x7f3fdc10c6a0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f3fd0009900 tx=0x7f3fd002bc30 comp rx=0 tx=0).stop 2024-03-21T10:28:29.616 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:28:29.615+0000 7f3fe4200640 1 -- 172.21.15.47:0/680434390 shutdown_connections 2024-03-21T10:28:29.616 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:28:29.615+0000 7f3fe4200640 1 --2- 172.21.15.47:0/680434390 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3fdc111390 0x7f3fdc118f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:28:29.616 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:28:29.615+0000 7f3fe4200640 1 --2- 172.21.15.47:0/680434390 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3fdc10cc70 0x7f3fdc110cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:28:29.616 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:28:29.615+0000 7f3fe4200640 1 --2- 172.21.15.47:0/680434390 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3fdc10c2c0 0x7f3fdc10c6a0 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:28:29.617 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:28:29.615+0000 7f3fe4200640 1 -- 172.21.15.47:0/680434390 >> 172.21.15.47:0/680434390 conn(0x7f3fdc06c8d0 msgr2=0x7f3fdc10a6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:28:29.617 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:28:29.616+0000 7f3fe4200640 1 -- 172.21.15.47:0/680434390 shutdown_connections 2024-03-21T10:28:29.617 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:28:29.616+0000 7f3fe4200640 1 -- 172.21.15.47:0/680434390 wait complete. 2024-03-21T10:28:29.617 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:28:29.617+0000 7f3fe4200640 1 Processor -- start 2024-03-21T10:28:29.618 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:28:29.617+0000 7f3fe4200640 1 -- start start 2024-03-21T10:28:29.618 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:28:29.617+0000 7f3fe4200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3fdc10c2c0 0x7f3fdc118e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:28:29.618 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:28:29.617+0000 7f3fe4200640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3fdc111390 0x7f3fdc115ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:28:29.618 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:28:29.618+0000 7f3fe4200640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3fdc116430 0x7f3fdc1168b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:28:29.618 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:28:29.618+0000 7f3fe4200640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3fdc11c320 con 0x7f3fdc111390 2024-03-21T10:28:29.618 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:28:29.618+0000 7f3fe4200640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3fdc11c1a0 con 0x7f3fdc10c2c0 2024-03-21T10:28:29.618 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:28:29.618+0000 7f3fe4200640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3fdc11c4a0 con 0x7f3fdc116430 2024-03-21T10:28:29.618 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:28:29.618+0000 7f3fe1800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3fdc10c2c0 0x7f3fdc118e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:28:29.618 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:28:29.618+0000 7f3fe1800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3fdc10c2c0 0x7f3fdc118e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:40468/0 (socket says 172.21.15.47:40468) 2024-03-21T10:28:29.618 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:28:29.618+0000 7f3fe1800640 1 -- 172.21.15.47:0/3740472457 learned_addr learned my addr 172.21.15.47:0/3740472457 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:28:29.618 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:28:29.618+0000 7f3fe2200640 1 --2- 172.21.15.47:0/3740472457 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3fdc116430 0x7f3fdc1168b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:28:29.619 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:28:29.619+0000 7f3fe2200640 1 -- 172.21.15.47:0/3740472457 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3fdc10c2c0 msgr2=0x7f3fdc118e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:28:29.619 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:28:29.619+0000 7f3fe2200640 1 --2- 172.21.15.47:0/3740472457 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3fdc10c2c0 0x7f3fdc118e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:28:29.619 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:28:29.619+0000 7f3fe2200640 1 -- 172.21.15.47:0/3740472457 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3fdc111390 msgr2=0x7f3fdc115ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:28:29.620 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:28:29.619+0000 7f3fe2200640 1 --2- 172.21.15.47:0/3740472457 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3fdc111390 0x7f3fdc115ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:28:29.620 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:28:29.619+0000 7f3fe2200640 1 -- 172.21.15.47:0/3740472457 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3fdc1b32a0 con 0x7f3fdc116430 2024-03-21T10:28:29.620 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:28:29.619+0000 7f3fe2200640 1 --2- 172.21.15.47:0/3740472457 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3fdc116430 0x7f3fdc1168b0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f3fd400ec50 tx=0x7f3fd400c6a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:28:29.620 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:28:29.619+0000 7f3fda000640 1 -- 172.21.15.47:0/3740472457 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fd4003b70 con 0x7f3fdc116430 2024-03-21T10:28:29.621 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:28:29.620+0000 7f3fda000640 1 -- 172.21.15.47:0/3740472457 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f3fd4003d10 con 0x7f3fdc116430 2024-03-21T10:28:29.621 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:28:29.620+0000 7f3fda000640 1 -- 172.21.15.47:0/3740472457 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fd4010760 con 0x7f3fdc116430 2024-03-21T10:28:29.621 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:28:29.620+0000 7f3fe4200640 1 -- 172.21.15.47:0/3740472457 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3fdc1b3590 con 0x7f3fdc116430 2024-03-21T10:28:29.621 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:28:29.620+0000 7f3fe4200640 1 -- 172.21.15.47:0/3740472457 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3fdc1b3a90 con 0x7f3fdc116430 2024-03-21T10:28:29.624 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:28:29.625+0000 7f3fe6e63740 1 Processor -- start 2024-03-21T10:28:29.627 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:28:29.628+0000 7f3fda000640 1 -- 172.21.15.47:0/3740472457 <== mon.1 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 27) v1 ==== 137985+0+0 (secure 0 0 0) 0x7f3fd4003ec0 con 0x7f3fdc116430 2024-03-21T10:28:29.631 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:28:29.630+0000 7f3fe6e63740 1 -- start start 2024-03-21T10:28:29.631 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:28:29.630+0000 7f3fe6e63740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5599bb9bffd0 0x5599bb9c03b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:28:29.631 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:28:29.630+0000 7f3fe6e63740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5599bb9c08f0 0x5599bb9c9180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:28:29.631 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:28:29.630+0000 7f3fe6e63740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5599bb9c96c0 0x5599bb9cbab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:28:29.631 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:28:29.630+0000 7f3fe6e63740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x5599bb9823b0 con 0x5599bb9c08f0 2024-03-21T10:28:29.631 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:28:29.630+0000 7f3fe6e63740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x5599bb924830 con 0x5599bb9bffd0 2024-03-21T10:28:29.631 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:28:29.631+0000 7f3fe6e63740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x5599bb8cd5d0 con 0x5599bb9c96c0 2024-03-21T10:28:29.632 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:28:29.632+0000 7f3fda000640 1 --2- 172.21.15.47:0/3740472457 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f3fc40a32e0 0x7f3fc40a57a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:28:29.632 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:28:29.632+0000 7f3fe0e00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5599bb9c96c0 0x5599bb9cbab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:28:29.632 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:28:29.632+0000 7f3fe0e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5599bb9bffd0 0x5599bb9c03b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:28:29.632 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:28:29.632+0000 7f3fe0e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5599bb9bffd0 0x5599bb9c03b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:40472/0 (socket says 172.21.15.47:40472) 2024-03-21T10:28:29.633 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:28:29.632+0000 7f3fe0e00640 1 -- 172.21.15.47:0/3176853140 learned_addr learned my addr 172.21.15.47:0/3176853140 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:28:29.633 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:28:29.633+0000 7f3fe1800640 1 --2- 172.21.15.47:0/3176853140 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5599bb9c08f0 0x5599bb9c9180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:28:29.633 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:28:29.633+0000 7f3fe2200640 1 --2- 172.21.15.47:0/3740472457 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f3fc40a32e0 0x7f3fc40a57a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:28:29.633 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:28:29.633+0000 7f3fe1800640 1 -- 172.21.15.47:0/3176853140 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5599bb9c96c0 msgr2=0x5599bb9cbab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:28:29.634 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:28:29.633+0000 7f3fe1800640 1 --2- 172.21.15.47:0/3176853140 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5599bb9c96c0 0x5599bb9cbab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:28:29.634 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:28:29.633+0000 7f3fe1800640 1 -- 172.21.15.47:0/3176853140 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5599bb9bffd0 msgr2=0x5599bb9c03b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:28:29.634 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:28:29.633+0000 7f3fe1800640 1 --2- 172.21.15.47:0/3176853140 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5599bb9bffd0 0x5599bb9c03b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:28:29.634 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:28:29.633+0000 7f3fe1800640 1 -- 172.21.15.47:0/3176853140 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5599bb8e52c0 con 0x5599bb9c08f0 2024-03-21T10:28:29.634 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:28:29.633+0000 7f3fe0e00640 1 --2- 172.21.15.47:0/3176853140 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5599bb9bffd0 0x5599bb9c03b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T10:28:29.634 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:28:29.634+0000 7f3fda000640 1 -- 172.21.15.47:0/3740472457 <== mon.1 v2:172.21.15.157:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 9474+0+0 (secure 0 0 0) 0x7f3fd40d00e0 con 0x7f3fdc116430 2024-03-21T10:28:29.634 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:28:29.634+0000 7f3fe2200640 1 --2- 172.21.15.47:0/3740472457 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f3fc40a32e0 0x7f3fc40a57a0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f3fd4098ae0 tx=0x7f3fd40c6dd0 comp rx=0 tx=0).ready entity=mgr.15687 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:28:29.634 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:28:29.635+0000 7f3fe1800640 1 --2- 172.21.15.47:0/3176853140 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5599bb9c08f0 0x5599bb9c9180 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7f3fd0005ab0 tx=0x7f3fd0009610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5c4ab01065b0bb89 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:28:29.635 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:28:29.635+0000 7f3fcea00640 1 -- 172.21.15.47:0/3176853140 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fd0045020 con 0x5599bb9c08f0 2024-03-21T10:28:29.635 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:28:29.635+0000 7f3fcea00640 1 -- 172.21.15.47:0/3176853140 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f3fd002c7e0 con 0x5599bb9c08f0 2024-03-21T10:28:29.635 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:28:29.635+0000 7f3fcea00640 1 -- 172.21.15.47:0/3176853140 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fd0005580 con 0x5599bb9c08f0 2024-03-21T10:28:29.636 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:28:29.636+0000 7f3fe6e63740 1 -- 172.21.15.47:0/3176853140 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5599bb9c08f0 msgr2=0x5599bb9c9180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:28:29.636 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:28:29.636+0000 7f3fe6e63740 1 --2- 172.21.15.47:0/3176853140 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5599bb9c08f0 0x5599bb9c9180 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7f3fd0005ab0 tx=0x7f3fd0009610 comp rx=0 tx=0).stop 2024-03-21T10:28:29.636 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:28:29.636+0000 7f3fe6e63740 1 -- 172.21.15.47:0/3176853140 shutdown_connections 2024-03-21T10:28:29.636 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:28:29.636+0000 7f3fe6e63740 1 --2- 172.21.15.47:0/3176853140 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5599bb9c96c0 0x5599bb9cbab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:28:29.636 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:28:29.636+0000 7f3fe6e63740 1 --2- 172.21.15.47:0/3176853140 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5599bb9c08f0 0x5599bb9c9180 unknown :-1 s=CLOSED pgs=739 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:28:29.636 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:28:29.636+0000 7f3fe6e63740 1 --2- 172.21.15.47:0/3176853140 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5599bb9bffd0 0x5599bb9c03b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:28:29.636 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:28:29.636+0000 7f3fe6e63740 1 -- 172.21.15.47:0/3176853140 >> 172.21.15.47:0/3176853140 conn(0x5599bb9ba8f0 msgr2=0x5599bb9bacd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:28:29.637 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:28:29.637+0000 7f3fe6e63740 1 -- 172.21.15.47:0/3176853140 shutdown_connections 2024-03-21T10:28:29.637 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:28:29.637+0000 7f3fe6e63740 1 -- 172.21.15.47:0/3176853140 wait complete. 2024-03-21T10:28:29.637 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:28:29.637+0000 7f3fe6e63740 1 -- start start 2024-03-21T10:28:29.637 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:28:29.638+0000 7f3fe6e63740 1 Processor -- start 2024-03-21T10:28:29.638 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:28:29.638+0000 7f3fe6e63740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5599bb9bffd0 0x5599bb9c66d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:28:29.638 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:28:29.638+0000 7f3fe6e63740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5599bb9c08f0 0x5599bb9c6c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:28:29.638 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:28:29.638+0000 7f3fe6e63740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5599bb9c96c0 0x5599bb9c7150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:28:29.638 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:28:29.638+0000 7f3fe6e63740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x5599bb8e0a20 con 0x5599bb9c08f0 2024-03-21T10:28:29.638 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:28:29.639+0000 7f3fe6e63740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x5599bb9208d0 con 0x5599bb9bffd0 2024-03-21T10:28:29.638 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:28:29.639+0000 7f3fe6e63740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x5599bb9dd2a0 con 0x5599bb9c96c0 2024-03-21T10:28:29.638 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:28:29.639+0000 7f3fe6e63740 10 client.0 _ll_register_callbacks cb 0x5599bb91d1c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T10:28:29.639 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:28:29.640+0000 7f3fe1800640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5599bb9c08f0 0x5599bb9c6c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:28:29.639 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:28:29.640+0000 7f3fe1800640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5599bb9c08f0 0x5599bb9c6c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:36056/0 (socket says 172.21.15.47:36056) 2024-03-21T10:28:29.639 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:28:29.640+0000 7f3fe1800640 1 -- 172.21.15.47:0/320376728 learned_addr learned my addr 172.21.15.47:0/320376728 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:28:29.640 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:28:29.640+0000 7f3fe1800640 1 -- 172.21.15.47:0/320376728 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5599bb9c96c0 msgr2=0x5599bb9c7150 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-03-21T10:28:29.640 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:28:29.640+0000 7f3fe1800640 1 --2- 172.21.15.47:0/320376728 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5599bb9c96c0 0x5599bb9c7150 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:28:29.640 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:28:29.640+0000 7f3fe1800640 1 -- 172.21.15.47:0/320376728 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5599bb9bffd0 msgr2=0x5599bb9c66d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:28:29.640 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:28:29.640+0000 7f3fe1800640 1 --2- 172.21.15.47:0/320376728 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5599bb9bffd0 0x5599bb9c66d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:28:29.640 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:28:29.640+0000 7f3fe1800640 1 -- 172.21.15.47:0/320376728 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5599bb639210 con 0x5599bb9c08f0 2024-03-21T10:28:29.640 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:28:29.641+0000 7f3fe1800640 1 --2- 172.21.15.47:0/320376728 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5599bb9c08f0 0x5599bb9c6c10 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7f3fd0005570 tx=0x7f3fd00055a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6eed9734a50f4cb4 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:28:29.641 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:28:29.641+0000 7f3fc3400640 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-03-21T10:28:29.642 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:28:29.641+0000 7f3fc3400640 1 -- 172.21.15.47:0/320376728 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fd0045070 con 0x5599bb9c08f0 2024-03-21T10:28:29.642 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:28:29.641+0000 7f3fc3400640 1 -- 172.21.15.47:0/320376728 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(97..97 src has 1..97) v4 ==== 9474+0+0 (secure 0 0 0) 0x7f3fd00035f0 con 0x5599bb9c08f0 2024-03-21T10:28:29.642 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:28:29.641+0000 7f3fe6e63740 1 -- 172.21.15.47:0/320376728 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f3fd002c7e0 con 0x5599bb9c08f0 2024-03-21T10:28:29.642 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:28:29.641+0000 7f3fc3400640 1 -- 172.21.15.47:0/320376728 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3fd00037f0 con 0x5599bb9c08f0 2024-03-21T10:28:29.642 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:28:29.641+0000 7f3fe6e63740 10 client.16719 fetch_fsmap learned FSMap version 32 2024-03-21T10:28:29.642 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:28:29.641+0000 7f3fe6e63740 1 -- 172.21.15.47:0/320376728 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x5599bb705620 con 0x5599bb9c08f0 2024-03-21T10:28:29.642 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:28:29.642+0000 7f3fc3400640 1 -- 172.21.15.47:0/320376728 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f3fd002c7e0 con 0x5599bb9c08f0 2024-03-21T10:28:29.642 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:28:29.642+0000 7f3fc3400640 1 -- 172.21.15.47:0/320376728 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 32) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f3fd00049e0 con 0x5599bb9c08f0 2024-03-21T10:28:29.642 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:28:29.642+0000 7f3fe6e63740 10 client.16719 fetch_fsmap finished waiting for FSMap version 32 2024-03-21T10:28:29.642 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:28:29.642+0000 7f3fe6e63740 10 client.16719 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-21T10:28:29.642 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:28:29.642+0000 7f3fe6e63740 20 client.16719 populate_metadata read hostname 'smithi047' 2024-03-21T10:28:29.642 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:28:29.642+0000 7f3fe6e63740 1 --2- 172.21.15.47:0/320376728 >> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] conn(0x5599bb9ea4e0 0x5599bb9ec8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:28:29.643 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:28:29.642+0000 7f3fe6e63740 4 client.16719 mds_command: new command op to 34535 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T10:28:29.643 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:28:29.642+0000 7f3fe6e63740 1 -- 172.21.15.47:0/320376728 --> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5599bb931d70 con 0x5599bb9ea4e0 2024-03-21T10:28:29.643 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:28:29.643+0000 7f3fe0e00640 1 --2- 172.21.15.47:0/320376728 >> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] conn(0x5599bb9ea4e0 0x5599bb9ec8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:28:29.644 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:28:29.645+0000 7f3fe0e00640 1 --2- 172.21.15.47:0/320376728 >> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] conn(0x5599bb9ea4e0 0x5599bb9ec8d0 crc :-1 s=READY pgs=111 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3bb966d72338c15b server_cookie=8fc0954b9a132952 in_seq=0 out_seq=0 2024-03-21T10:28:29.645 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:28:29.646+0000 7f3fc3400640 10 client.16719 ms_handle_connect on v2:172.21.15.100:6838/4173168440 2024-03-21T10:28:29.646 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:28:29.647+0000 7f3fc3400640 1 -- 172.21.15.47:0/320376728 <== mds.0 v2:172.21.15.100:6838/4173168440 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x5599bb931d70 con 0x5599bb9ea4e0 2024-03-21T10:28:29.646 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:28:29.647+0000 7f3fc3400640 10 client.16719 handle_command_reply: tid=0 2024-03-21T10:28:29.647 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:28:29.647+0000 7f3fe6e63740 1 client.16719 shutdown 2024-03-21T10:28:29.647 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:28:29.648+0000 7f3fe6e63740 1 -- 172.21.15.47:0/320376728 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5599bb9c08f0 msgr2=0x5599bb9c6c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:28:29.647 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:28:29.648+0000 7f3fe6e63740 1 --2- 172.21.15.47:0/320376728 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5599bb9c08f0 0x5599bb9c6c10 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7f3fd0005570 tx=0x7f3fd00055a0 comp rx=0 tx=0).stop 2024-03-21T10:28:29.648 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:28:29.648+0000 7f3fe6e63740 1 -- 172.21.15.47:0/320376728 shutdown_connections 2024-03-21T10:28:29.648 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:28:29.648+0000 7f3fe6e63740 1 --2- 172.21.15.47:0/320376728 >> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] conn(0x5599bb9ea4e0 0x5599bb9ec8d0 crc :-1 s=READY pgs=111 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:28:29.648 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:28:29.648+0000 7f3fe6e63740 1 --2- 172.21.15.47:0/320376728 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5599bb9c96c0 0x5599bb9c7150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:28:29.648 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:28:29.648+0000 7f3fe6e63740 1 --2- 172.21.15.47:0/320376728 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5599bb9c08f0 0x5599bb9c6c10 unknown :-1 s=CLOSED pgs=740 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:28:29.648 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:28:29.648+0000 7f3fe6e63740 1 --2- 172.21.15.47:0/320376728 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5599bb9bffd0 0x5599bb9c66d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:28:29.648 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:28:29.649+0000 7f3fe6e63740 1 -- 172.21.15.47:0/320376728 >> 172.21.15.47:0/320376728 conn(0x5599bb9ba8f0 msgr2=0x5599bb9bc640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:28:29.649 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:28:29.649+0000 7f3fc3400640 0 client.16719 ms_handle_reset on v2:172.21.15.100:6838/4173168440 2024-03-21T10:28:29.649 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:28:29.650+0000 7f3fe6e63740 1 -- 172.21.15.47:0/320376728 shutdown_connections 2024-03-21T10:28:29.650 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:28:29.650+0000 7f3fe6e63740 1 -- 172.21.15.47:0/320376728 wait complete. 2024-03-21T10:28:29.650 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:28:29.650+0000 7f3fe6e63740 20 client.16719 trim_cache size 0 max 16384 2024-03-21T10:28:29.682 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:28:29.682+0000 7f3fe6e63740 1 Processor -- start 2024-03-21T10:28:29.682 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:28:29.682+0000 7f3fe6e63740 1 -- start start 2024-03-21T10:28:29.682 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:28:29.683+0000 7f3fe6e63740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5599bb9bffd0 0x5599bb9c8e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:28:29.682 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:28:29.683+0000 7f3fe6e63740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5599bb9c08f0 0x5599bb9c6f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:28:29.682 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:28:29.683+0000 7f3fe6e63740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5599bb9c96c0 0x5599bb9e59b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:28:29.683 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:28:29.683+0000 7f3fe6e63740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3fdc1b4d20 con 0x5599bb9c96c0 2024-03-21T10:28:29.683 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:28:29.683+0000 7f3fe6e63740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3fdc1b66a0 con 0x5599bb9c08f0 2024-03-21T10:28:29.683 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:28:29.683+0000 7f3fe6e63740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3fdc1b5380 con 0x5599bb9bffd0 2024-03-21T10:28:29.683 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:28:29.683+0000 7f3fe1800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5599bb9c08f0 0x5599bb9c6f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:28:29.683 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:28:29.684+0000 7f3fe1800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5599bb9c08f0 0x5599bb9c6f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:40486/0 (socket says 172.21.15.47:40486) 2024-03-21T10:28:29.684 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:28:29.684+0000 7f3fe1800640 1 -- 172.21.15.47:0/2064507751 learned_addr learned my addr 172.21.15.47:0/2064507751 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:28:29.684 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:28:29.683+0000 7f3fe0e00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5599bb9bffd0 0x5599bb9c8e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:28:29.685 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:28:29.685+0000 7f3fe0e00640 1 --2- 172.21.15.47:0/2064507751 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5599bb9c96c0 0x5599bb9e59b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:28:29.685 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:28:29.685+0000 7f3fe1800640 1 -- 172.21.15.47:0/2064507751 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5599bb9bffd0 msgr2=0x5599bb9c8e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:28:29.685 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:28:29.685+0000 7f3fe1800640 1 --2- 172.21.15.47:0/2064507751 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5599bb9bffd0 0x5599bb9c8e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:28:29.685 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:28:29.685+0000 7f3fe1800640 1 -- 172.21.15.47:0/2064507751 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5599bb9c96c0 msgr2=0x5599bb9e59b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:28:29.685 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:28:29.685+0000 7f3fe1800640 1 --2- 172.21.15.47:0/2064507751 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5599bb9c96c0 0x5599bb9e59b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:28:29.685 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:28:29.685+0000 7f3fe1800640 1 -- 172.21.15.47:0/2064507751 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5599bb9c4c20 con 0x5599bb9c08f0 2024-03-21T10:28:29.686 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:28:29.687+0000 7f3fe1800640 1 --2- 172.21.15.47:0/2064507751 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5599bb9c08f0 0x5599bb9c6f70 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f3fd002c240 tx=0x7f3fd0005be0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b38e16df9cc9c079 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:28:29.687 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:28:29.688+0000 7f3fce000640 1 -- 172.21.15.47:0/2064507751 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fd0045040 con 0x5599bb9c08f0 2024-03-21T10:28:29.687 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:28:29.688+0000 7f3fce000640 1 -- 172.21.15.47:0/2064507751 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f3fd002c7e0 con 0x5599bb9c08f0 2024-03-21T10:28:29.687 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:28:29.688+0000 7f3fce000640 1 -- 172.21.15.47:0/2064507751 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fd00042d0 con 0x5599bb9c08f0 2024-03-21T10:28:29.688 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:28:29.689+0000 7f3fe6e63740 1 -- 172.21.15.47:0/2064507751 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5599bb9c08f0 msgr2=0x5599bb9c6f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:28:29.689 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:28:29.689+0000 7f3fe6e63740 1 --2- 172.21.15.47:0/2064507751 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5599bb9c08f0 0x5599bb9c6f70 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f3fd002c240 tx=0x7f3fd0005be0 comp rx=0 tx=0).stop 2024-03-21T10:28:29.689 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:28:29.689+0000 7f3fe6e63740 1 -- 172.21.15.47:0/2064507751 shutdown_connections 2024-03-21T10:28:29.689 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:28:29.689+0000 7f3fe6e63740 1 --2- 172.21.15.47:0/2064507751 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5599bb9c96c0 0x5599bb9e59b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:28:29.689 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:28:29.689+0000 7f3fe6e63740 1 --2- 172.21.15.47:0/2064507751 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5599bb9c08f0 0x5599bb9c6f70 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:28:29.689 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:28:29.689+0000 7f3fe6e63740 1 --2- 172.21.15.47:0/2064507751 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5599bb9bffd0 0x5599bb9c8e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:28:29.689 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:28:29.689+0000 7f3fe6e63740 1 -- 172.21.15.47:0/2064507751 >> 172.21.15.47:0/2064507751 conn(0x5599bb9ba8f0 msgr2=0x5599bb9bc5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:28:29.690 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:28:29.689+0000 7f3fe6e63740 1 -- 172.21.15.47:0/2064507751 shutdown_connections 2024-03-21T10:28:29.690 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:28:29.689+0000 7f3fe6e63740 1 -- 172.21.15.47:0/2064507751 wait complete. 2024-03-21T10:28:29.690 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:28:29.690+0000 7f3fe6e63740 1 -- start start 2024-03-21T10:28:29.690 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:28:29.690+0000 7f3fe6e63740 1 Processor -- start 2024-03-21T10:28:29.691 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:28:29.691+0000 7f3fe6e63740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5599bb9bffd0 0x5599bb9ddb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:28:29.691 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:28:29.691+0000 7f3fe6e63740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5599bb9c08f0 0x5599bb9de090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:28:29.691 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:28:29.691+0000 7f3fe6e63740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5599bb9c96c0 0x5599bb9de5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:28:29.692 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:28:29.691+0000 7f3fe6e63740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3fdc1b6040 con 0x5599bb9c96c0 2024-03-21T10:28:29.692 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:28:29.691+0000 7f3fe6e63740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3fdc1b59e0 con 0x5599bb9bffd0 2024-03-21T10:28:29.692 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:28:29.691+0000 7f3fe6e63740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3fdc1a9130 con 0x5599bb9c08f0 2024-03-21T10:28:29.692 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:28:29.691+0000 7f3fe6e63740 10 client.0 _ll_register_callbacks cb 0x7f3fc800c940 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T10:28:29.692 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:28:29.692+0000 7f3fe1800640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5599bb9c08f0 0x5599bb9de090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:28:29.692 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:28:29.692+0000 7f3fe1800640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5599bb9c08f0 0x5599bb9de090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.47:40056/0 (socket says 172.21.15.47:40056) 2024-03-21T10:28:29.692 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:28:29.692+0000 7f3fe1800640 1 -- 172.21.15.47:0/1792862158 learned_addr learned my addr 172.21.15.47:0/1792862158 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:28:29.692 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:28:29.692+0000 7f3fe1800640 1 -- 172.21.15.47:0/1792862158 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5599bb9bffd0 msgr2=0x5599bb9ddb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:28:29.692 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:28:29.692+0000 7f3fe1800640 1 --2- 172.21.15.47:0/1792862158 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5599bb9bffd0 0x5599bb9ddb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:28:29.692 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:28:29.693+0000 7f3fe1800640 1 -- 172.21.15.47:0/1792862158 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5599bb9c96c0 msgr2=0x5599bb9de5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:28:29.693 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:28:29.693+0000 7f3fe1800640 1 --2- 172.21.15.47:0/1792862158 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5599bb9c96c0 0x5599bb9de5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:28:29.693 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:28:29.693+0000 7f3fe1800640 1 -- 172.21.15.47:0/1792862158 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5599bba44cd0 con 0x5599bb9c08f0 2024-03-21T10:28:29.694 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:28:29.694+0000 7f3fe1800640 1 --2- 172.21.15.47:0/1792862158 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5599bb9c08f0 0x5599bb9de090 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f3fd0004fc0 tx=0x7f3fd00095f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d9254f46dc5daec0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:28:29.694 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:28:29.694+0000 7f3fc3400640 10 client.0 ms_handle_connect on v2:172.21.15.157:3300/0 2024-03-21T10:28:29.696 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:28:29.694+0000 7f3fc3400640 1 -- 172.21.15.47:0/1792862158 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fd0045070 con 0x5599bb9c08f0 2024-03-21T10:28:29.696 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:28:29.695+0000 7f3fe6e63740 1 -- 172.21.15.47:0/1792862158 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f3fd002c7e0 con 0x5599bb9c08f0 2024-03-21T10:28:29.696 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:28:29.696+0000 7f3fc3400640 1 -- 172.21.15.47:0/1792862158 <== mon.1 v2:172.21.15.157:3300/0 2 ==== osd_map(97..97 src has 1..97) v4 ==== 9474+0+0 (secure 0 0 0) 0x7f3fd00046c0 con 0x5599bb9c08f0 2024-03-21T10:28:29.696 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:28:29.697+0000 7f3fc3400640 1 -- 172.21.15.47:0/1792862158 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3fd00048c0 con 0x5599bb9c08f0 2024-03-21T10:28:29.696 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:28:29.697+0000 7f3fe6e63740 10 client.26278 fetch_fsmap learned FSMap version 32 2024-03-21T10:28:29.696 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:28:29.697+0000 7f3fe6e63740 1 -- 172.21.15.47:0/1792862158 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x5599bba45fc0 con 0x5599bb9c08f0 2024-03-21T10:28:29.697 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:28:29.697+0000 7f3fc3400640 1 -- 172.21.15.47:0/1792862158 <== mon.1 v2:172.21.15.157:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f3fd002c7e0 con 0x5599bb9c08f0 2024-03-21T10:28:29.697 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:28:29.698+0000 7f3fc3400640 1 -- 172.21.15.47:0/1792862158 <== mon.1 v2:172.21.15.157:3300/0 5 ==== fsmap(e 32) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f3fd00416b0 con 0x5599bb9c08f0 2024-03-21T10:28:29.698 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:28:29.698+0000 7f3fe6e63740 10 client.26278 fetch_fsmap finished waiting for FSMap version 32 2024-03-21T10:28:29.698 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:28:29.698+0000 7f3fe6e63740 10 client.26278 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-21T10:28:29.698 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:28:29.698+0000 7f3fe6e63740 20 client.26278 populate_metadata read hostname 'smithi047' 2024-03-21T10:28:29.698 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:28:29.698+0000 7f3fe6e63740 1 --2- 172.21.15.47:0/1792862158 >> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] conn(0x5599bba46300 0x5599bba486f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:28:29.698 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:28:29.698+0000 7f3fe6e63740 4 client.26278 mds_command: new command op to 34535 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-21T10:28:29.698 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:28:29.698+0000 7f3fe6e63740 1 -- 172.21.15.47:0/1792862158 --> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x5599bb92c0c0 con 0x5599bba46300 2024-03-21T10:28:29.698 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:28:29.699+0000 7f3fe0e00640 1 --2- 172.21.15.47:0/1792862158 >> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] conn(0x5599bba46300 0x5599bba486f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:28:29.701 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:28:29.701+0000 7f3fe0e00640 1 --2- 172.21.15.47:0/1792862158 >> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] conn(0x5599bba46300 0x5599bba486f0 crc :-1 s=READY pgs=112 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ee937761ecbb6f2f server_cookie=6a75b669026279b in_seq=0 out_seq=0 2024-03-21T10:28:29.701 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:28:29.701+0000 7f3fc3400640 10 client.26278 ms_handle_connect on v2:172.21.15.100:6838/4173168440 2024-03-21T10:28:29.751 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:28:29.749+0000 7f3fc3400640 1 -- 172.21.15.47:0/1792862158 <== mds.0 v2:172.21.15.100:6838/4173168440 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x5599bb92c0c0 con 0x5599bba46300 2024-03-21T10:28:29.751 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:28:29.749+0000 7f3fc3400640 10 client.26278 handle_command_reply: tid=0 2024-03-21T10:28:29.751 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:28:29.750+0000 7f3fe6e63740 1 client.26278 shutdown 2024-03-21T10:28:29.751 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:28:29.751+0000 7f3fe6e63740 1 -- 172.21.15.47:0/1792862158 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5599bb9c08f0 msgr2=0x5599bb9de090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:28:29.751 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:28:29.751+0000 7f3fe6e63740 1 --2- 172.21.15.47:0/1792862158 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5599bb9c08f0 0x5599bb9de090 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f3fd0004fc0 tx=0x7f3fd00095f0 comp rx=0 tx=0).stop 2024-03-21T10:28:29.751 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:28:29.751+0000 7f3fe6e63740 1 -- 172.21.15.47:0/1792862158 shutdown_connections 2024-03-21T10:28:29.751 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:28:29.751+0000 7f3fe6e63740 1 --2- 172.21.15.47:0/1792862158 >> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] conn(0x5599bba46300 0x5599bba486f0 crc :-1 s=READY pgs=112 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:28:29.751 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:28:29.751+0000 7f3fe6e63740 1 --2- 172.21.15.47:0/1792862158 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5599bb9c96c0 0x5599bb9de5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:28:29.751 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:28:29.751+0000 7f3fe6e63740 1 --2- 172.21.15.47:0/1792862158 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5599bb9c08f0 0x5599bb9de090 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:28:29.751 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:28:29.751+0000 7f3fe6e63740 1 --2- 172.21.15.47:0/1792862158 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5599bb9bffd0 0x5599bb9ddb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:28:29.752 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:28:29.751+0000 7f3fe6e63740 1 -- 172.21.15.47:0/1792862158 >> 172.21.15.47:0/1792862158 conn(0x5599bb9ba8f0 msgr2=0x5599bb9b90c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:28:29.752 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:28:29.751+0000 7f3fc3400640 0 client.26278 ms_handle_reset on v2:172.21.15.100:6838/4173168440 2024-03-21T10:28:29.752 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:28:29.752+0000 7f3fe6e63740 1 -- 172.21.15.47:0/1792862158 shutdown_connections 2024-03-21T10:28:29.752 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:28:29.752+0000 7f3fe6e63740 1 -- 172.21.15.47:0/1792862158 wait complete. 2024-03-21T10:28:29.752 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:28:29.752+0000 7f3fe6e63740 20 client.26278 trim_cache size 0 max 16384 2024-03-21T10:28:29.753 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout:{ 2024-03-21T10:28:29.753 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout: "message": "", 2024-03-21T10:28:29.753 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout: "return_code": 0 2024-03-21T10:28:29.753 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stdout:} 2024-03-21T10:28:29.754 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:28:29.753+0000 7f3fe4200640 1 -- 172.21.15.47:0/3740472457 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f3fc40a32e0 msgr2=0x7f3fc40a57a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:28:29.754 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:28:29.753+0000 7f3fe4200640 1 --2- 172.21.15.47:0/3740472457 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f3fc40a32e0 0x7f3fc40a57a0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f3fd4098ae0 tx=0x7f3fd40c6dd0 comp rx=0 tx=0).stop 2024-03-21T10:28:29.754 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:28:29.754+0000 7f3fe4200640 1 -- 172.21.15.47:0/3740472457 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3fdc116430 msgr2=0x7f3fdc1168b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:28:29.754 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:28:29.754+0000 7f3fe4200640 1 --2- 172.21.15.47:0/3740472457 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3fdc116430 0x7f3fdc1168b0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f3fd400ec50 tx=0x7f3fd400c6a0 comp rx=0 tx=0).stop 2024-03-21T10:28:29.754 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:28:29.754+0000 7f3fe4200640 1 -- 172.21.15.47:0/3740472457 shutdown_connections 2024-03-21T10:28:29.754 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:28:29.754+0000 7f3fe4200640 1 --2- 172.21.15.47:0/3740472457 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3fdc116430 0x7f3fdc1168b0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:28:29.754 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:28:29.754+0000 7f3fe4200640 1 --2- 172.21.15.47:0/3740472457 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3fdc111390 0x7f3fdc115ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:28:29.754 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:28:29.754+0000 7f3fe4200640 1 --2- 172.21.15.47:0/3740472457 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f3fc40a32e0 0x7f3fc40a57a0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:28:29.754 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:28:29.754+0000 7f3fe4200640 1 --2- 172.21.15.47:0/3740472457 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3fdc10c2c0 0x7f3fdc118e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:28:29.754 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:28:29.754+0000 7f3fe4200640 1 -- 172.21.15.47:0/3740472457 >> 172.21.15.47:0/3740472457 conn(0x7f3fdc06c8d0 msgr2=0x7f3fdc10f450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:28:29.754 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:28:29.754+0000 7f3fe4200640 1 -- 172.21.15.47:0/3740472457 shutdown_connections 2024-03-21T10:28:29.754 INFO:teuthology.task.background_exec.ubuntu@smithi047.front.sepia.ceph.com.smithi047.stderr:2024-03-21T10:28:29.754+0000 7f3fe4200640 1 -- 172.21.15.47:0/3740472457 wait complete. 2024-03-21T10:28:30.889 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-21T10:28:30.890 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-21T10:28:31.265 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:31.265+0000 7fc173800640 1 -- 172.21.15.47:0/543147561 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc16c100dd0 msgr2=0x7fc16c10db20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:28:31.265 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:31.265+0000 7fc173800640 1 --2- 172.21.15.47:0/543147561 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc16c100dd0 0x7fc16c10db20 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fc158009900 tx=0x7fc15802bc30 comp rx=0 tx=0).stop 2024-03-21T10:28:31.265 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:31.265+0000 7fc173800640 1 -- 172.21.15.47:0/543147561 shutdown_connections 2024-03-21T10:28:31.265 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:31.265+0000 7fc173800640 1 --2- 172.21.15.47:0/543147561 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc16c10e060 0x7fc16c110450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:28:31.265 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:31.265+0000 7fc173800640 1 --2- 172.21.15.47:0/543147561 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc16c100dd0 0x7fc16c10db20 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:28:31.265 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:31.265+0000 7fc173800640 1 --2- 172.21.15.47:0/543147561 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc16c1004b0 0x7fc16c100890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:28:31.265 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:31.265+0000 7fc173800640 1 -- 172.21.15.47:0/543147561 >> 172.21.15.47:0/543147561 conn(0x7fc16c077700 msgr2=0x7fc16c077b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:28:31.265 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:31.265+0000 7fc173800640 1 -- 172.21.15.47:0/543147561 shutdown_connections 2024-03-21T10:28:31.265 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:31.265+0000 7fc173800640 1 -- 172.21.15.47:0/543147561 wait complete. 2024-03-21T10:28:31.266 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:31.267+0000 7fc173800640 1 Processor -- start 2024-03-21T10:28:31.267 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:31.267+0000 7fc173800640 1 -- start start 2024-03-21T10:28:31.267 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:31.268+0000 7fc173800640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc16c1004b0 0x7fc16c1028a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:28:31.268 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:31.268+0000 7fc173800640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc16c100dd0 0x7fc16c102de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:28:31.269 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:31.269+0000 7fc173800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc16c10e060 0x7fc16c106c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:28:31.269 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:31.269+0000 7fc173800640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc16c113bf0 con 0x7fc16c1004b0 2024-03-21T10:28:31.269 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:31.269+0000 7fc173800640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc16c113a70 con 0x7fc16c10e060 2024-03-21T10:28:31.269 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:31.269+0000 7fc173800640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc16c113d70 con 0x7fc16c100dd0 2024-03-21T10:28:31.269 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:31.269+0000 7fc170e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc16c1004b0 0x7fc16c1028a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:28:31.269 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:31.269+0000 7fc16be00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc16c100dd0 0x7fc16c102de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:28:31.269 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:31.269+0000 7fc16be00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc16c100dd0 0x7fc16c102de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.47:40070/0 (socket says 172.21.15.47:40070) 2024-03-21T10:28:31.269 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:31.269+0000 7fc16be00640 1 -- 172.21.15.47:0/1107227945 learned_addr learned my addr 172.21.15.47:0/1107227945 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:28:31.270 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:31.270+0000 7fc16be00640 1 -- 172.21.15.47:0/1107227945 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc16c10e060 msgr2=0x7fc16c106c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:28:31.270 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:31.270+0000 7fc16be00640 1 --2- 172.21.15.47:0/1107227945 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc16c10e060 0x7fc16c106c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:28:31.270 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:31.270+0000 7fc16be00640 1 -- 172.21.15.47:0/1107227945 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc16c1004b0 msgr2=0x7fc16c1028a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:28:31.270 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:31.270+0000 7fc16be00640 1 --2- 172.21.15.47:0/1107227945 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc16c1004b0 0x7fc16c1028a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:28:31.270 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:31.270+0000 7fc16be00640 1 -- 172.21.15.47:0/1107227945 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc16c107370 con 0x7fc16c100dd0 2024-03-21T10:28:31.270 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:31.270+0000 7fc170e00640 1 --2- 172.21.15.47:0/1107227945 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc16c1004b0 0x7fc16c1028a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T10:28:31.271 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:31.271+0000 7fc16be00640 1 --2- 172.21.15.47:0/1107227945 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc16c100dd0 0x7fc16c102de0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fc15802c170 tx=0x7fc15802ca10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:28:31.271 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:31.271+0000 7fc169600640 1 -- 172.21.15.47:0/1107227945 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc158042020 con 0x7fc16c100dd0 2024-03-21T10:28:31.271 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:31.271+0000 7fc173800640 1 -- 172.21.15.47:0/1107227945 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc16c107600 con 0x7fc16c100dd0 2024-03-21T10:28:31.271 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:31.272+0000 7fc169600640 1 -- 172.21.15.47:0/1107227945 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fc1580052b0 con 0x7fc16c100dd0 2024-03-21T10:28:31.271 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:31.272+0000 7fc169600640 1 -- 172.21.15.47:0/1107227945 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc158003e20 con 0x7fc16c100dd0 2024-03-21T10:28:31.272 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:31.272+0000 7fc173800640 1 -- 172.21.15.47:0/1107227945 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc16c10ce50 con 0x7fc16c100dd0 2024-03-21T10:28:31.276 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:31.276+0000 7fc17644a740 1 Processor -- start 2024-03-21T10:28:31.276 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:31.277+0000 7fc17644a740 1 -- start start 2024-03-21T10:28:31.277 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:31.277+0000 7fc17644a740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d50004dc00 0x55d50004c950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:28:31.277 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:31.277+0000 7fc17644a740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d50004cf20 0x55d50004d3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:28:31.277 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:31.277+0000 7fc17644a740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d500056290 0x55d500058680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:28:31.277 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:31.277+0000 7fc17644a740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d4fffb1250 con 0x55d50004dc00 2024-03-21T10:28:31.278 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:31.277+0000 7fc17644a740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d4fff59a30 con 0x55d50004cf20 2024-03-21T10:28:31.278 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:31.277+0000 7fc17644a740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d4fffad2f0 con 0x55d500056290 2024-03-21T10:28:31.278 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:31.278+0000 7fc171800640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d50004dc00 0x55d50004c950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:28:31.279 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:31.278+0000 7fc16be00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d500056290 0x55d500058680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:28:31.279 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:31.279+0000 7fc171800640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d50004dc00 0x55d50004c950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:36108/0 (socket says 172.21.15.47:36108) 2024-03-21T10:28:31.279 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:31.279+0000 7fc171800640 1 -- 172.21.15.47:0/3210316361 learned_addr learned my addr 172.21.15.47:0/3210316361 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:28:31.279 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:31.279+0000 7fc169600640 1 -- 172.21.15.47:0/1107227945 <== mon.1 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 27) v1 ==== 137985+0+0 (secure 0 0 0) 0x7fc158045020 con 0x7fc16c100dd0 2024-03-21T10:28:31.280 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:31.280+0000 7fc170e00640 1 --2- 172.21.15.47:0/3210316361 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d50004cf20 0x55d50004d3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:28:31.280 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:31.280+0000 7fc171800640 1 -- 172.21.15.47:0/3210316361 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d500056290 msgr2=0x55d500058680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:28:31.280 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:31.280+0000 7fc171800640 1 --2- 172.21.15.47:0/3210316361 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d500056290 0x55d500058680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:28:31.280 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:31.280+0000 7fc171800640 1 -- 172.21.15.47:0/3210316361 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d50004cf20 msgr2=0x55d50004d3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:28:31.280 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:31.280+0000 7fc171800640 1 --2- 172.21.15.47:0/3210316361 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d50004cf20 0x55d50004d3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:28:31.280 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:31.280+0000 7fc171800640 1 -- 172.21.15.47:0/3210316361 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d4fff44690 con 0x55d50004dc00 2024-03-21T10:28:31.280 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:31.280+0000 7fc170e00640 1 --2- 172.21.15.47:0/3210316361 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d50004cf20 0x55d50004d3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T10:28:31.281 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:31.281+0000 7fc171800640 1 --2- 172.21.15.47:0/3210316361 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d50004dc00 0x55d50004c950 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7fc164009da0 tx=0x7fc16400e6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5200972801868446 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:28:31.281 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:31.281+0000 7fc169600640 1 --2- 172.21.15.47:0/1107227945 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7fc1540a3190 0x7fc1540a5650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:28:31.282 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:31.282+0000 7fc169600640 1 -- 172.21.15.47:0/1107227945 <== mon.1 v2:172.21.15.157:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 9474+0+0 (secure 0 0 0) 0x7fc1580f1d30 con 0x7fc16c100dd0 2024-03-21T10:28:31.282 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:31.282+0000 7fc15e000640 1 -- 172.21.15.47:0/3210316361 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc164016070 con 0x55d50004dc00 2024-03-21T10:28:31.282 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:31.283+0000 7fc15e000640 1 -- 172.21.15.47:0/3210316361 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fc164005020 con 0x55d50004dc00 2024-03-21T10:28:31.282 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:31.283+0000 7fc15e000640 1 -- 172.21.15.47:0/3210316361 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc164016070 con 0x55d50004dc00 2024-03-21T10:28:31.285 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:31.285+0000 7fc17644a740 1 -- 172.21.15.47:0/3210316361 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d50004dc00 msgr2=0x55d50004c950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:28:31.285 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:31.285+0000 7fc17644a740 1 --2- 172.21.15.47:0/3210316361 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d50004dc00 0x55d50004c950 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7fc164009da0 tx=0x7fc16400e6c0 comp rx=0 tx=0).stop 2024-03-21T10:28:31.285 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:31.285+0000 7fc170e00640 1 --2- 172.21.15.47:0/1107227945 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7fc1540a3190 0x7fc1540a5650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:28:31.285 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:31.285+0000 7fc17644a740 1 -- 172.21.15.47:0/3210316361 shutdown_connections 2024-03-21T10:28:31.285 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:31.285+0000 7fc17644a740 1 --2- 172.21.15.47:0/3210316361 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d500056290 0x55d500058680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:28:31.285 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:31.285+0000 7fc17644a740 1 --2- 172.21.15.47:0/3210316361 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d50004cf20 0x55d50004d3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:28:31.285 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:31.285+0000 7fc17644a740 1 --2- 172.21.15.47:0/3210316361 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d50004dc00 0x55d50004c950 unknown :-1 s=CLOSED pgs=741 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:28:31.285 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:31.285+0000 7fc17644a740 1 -- 172.21.15.47:0/3210316361 >> 172.21.15.47:0/3210316361 conn(0x55d500014200 msgr2=0x55d500047970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:28:31.286 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:31.285+0000 7fc17644a740 1 -- 172.21.15.47:0/3210316361 shutdown_connections 2024-03-21T10:28:31.297 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:31.291+0000 7fc17644a740 1 -- 172.21.15.47:0/3210316361 wait complete. 2024-03-21T10:28:31.297 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:31.292+0000 7fc17644a740 1 -- start start 2024-03-21T10:28:31.297 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:31.292+0000 7fc170e00640 1 --2- 172.21.15.47:0/1107227945 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7fc1540a3190 0x7fc1540a5650 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x55d50004e150 tx=0x7fc160008040 comp rx=0 tx=0).ready entity=mgr.15687 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:28:31.298 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:31.292+0000 7fc17644a740 1 Processor -- start 2024-03-21T10:28:31.298 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:31.293+0000 7fc17644a740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d50004cf20 0x55d500055900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:28:31.298 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:31.293+0000 7fc17644a740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d50004dc00 0x55d500052c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:28:31.298 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:31.293+0000 7fc17644a740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d500056290 0x55d5000531d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:28:31.298 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:31.294+0000 7fc16be00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d50004dc00 0x55d500052c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:28:31.298 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:31.294+0000 7fc171800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d500056290 0x55d5000531d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:28:31.298 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:31.294+0000 7fc171800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d500056290 0x55d5000531d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:40536/0 (socket says 172.21.15.47:40536) 2024-03-21T10:28:31.298 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:31.294+0000 7fc16be00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d50004dc00 0x55d500052c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.47:40098/0 (socket says 172.21.15.47:40098) 2024-03-21T10:28:31.298 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:31.294+0000 7fc171800640 1 -- 172.21.15.47:0/2221734273 learned_addr learned my addr 172.21.15.47:0/2221734273 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:28:31.298 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:31.294+0000 7fc17644a740 1 -- 172.21.15.47:0/2221734273 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d500059100 con 0x55d50004cf20 2024-03-21T10:28:31.298 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:31.294+0000 7fc17644a740 1 -- 172.21.15.47:0/2221734273 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d4fff6ce80 con 0x55d500056290 2024-03-21T10:28:31.299 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:31.294+0000 7fc17644a740 1 -- 172.21.15.47:0/2221734273 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d500059280 con 0x55d50004dc00 2024-03-21T10:28:31.299 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:31.294+0000 7fc17644a740 10 client.0 _ll_register_callbacks cb 0x55d4fff7f130 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T10:28:31.299 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:31.295+0000 7fc16be00640 1 -- 172.21.15.47:0/2221734273 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d500056290 msgr2=0x55d5000531d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:28:31.299 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:31.295+0000 7fc16be00640 1 --2- 172.21.15.47:0/2221734273 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d500056290 0x55d5000531d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:28:31.299 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:31.295+0000 7fc16be00640 1 -- 172.21.15.47:0/2221734273 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d50004cf20 msgr2=0x55d500055900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:28:31.299 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:31.295+0000 7fc16be00640 1 --2- 172.21.15.47:0/2221734273 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d50004cf20 0x55d500055900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:28:31.299 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:31.295+0000 7fc16be00640 1 -- 172.21.15.47:0/2221734273 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55d4ffd8f7d0 con 0x55d50004dc00 2024-03-21T10:28:31.299 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:31.296+0000 7fc16be00640 1 --2- 172.21.15.47:0/2221734273 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d50004dc00 0x55d500052c90 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fc1580ef170 tx=0x7fc1580ba170 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3705a106c907dcc3 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:28:31.299 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:31.296+0000 7fc152a00640 10 client.0 ms_handle_connect on v2:172.21.15.157:3300/0 2024-03-21T10:28:31.299 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:31.296+0000 7fc152a00640 1 -- 172.21.15.47:0/2221734273 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1580be510 con 0x55d50004dc00 2024-03-21T10:28:31.299 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:31.296+0000 7fc152a00640 1 -- 172.21.15.47:0/2221734273 <== mon.1 v2:172.21.15.157:3300/0 2 ==== osd_map(97..97 src has 1..97) v4 ==== 9474+0+0 (secure 0 0 0) 0x7fc1580035f0 con 0x55d50004dc00 2024-03-21T10:28:31.299 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:31.296+0000 7fc17644a740 1 -- 172.21.15.47:0/2221734273 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fc164005020 con 0x55d50004dc00 2024-03-21T10:28:31.299 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:31.296+0000 7fc152a00640 1 -- 172.21.15.47:0/2221734273 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc1580037f0 con 0x55d50004dc00 2024-03-21T10:28:31.300 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:31.296+0000 7fc17644a740 10 client.26290 fetch_fsmap learned FSMap version 32 2024-03-21T10:28:31.300 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:31.296+0000 7fc17644a740 1 -- 172.21.15.47:0/2221734273 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x55d4ffcc53f0 con 0x55d50004dc00 2024-03-21T10:28:31.300 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:31.297+0000 7fc152a00640 1 -- 172.21.15.47:0/2221734273 <== mon.1 v2:172.21.15.157:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fc164005020 con 0x55d50004dc00 2024-03-21T10:28:31.300 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:31.297+0000 7fc152a00640 1 -- 172.21.15.47:0/2221734273 <== mon.1 v2:172.21.15.157:3300/0 5 ==== fsmap(e 32) v1 ==== 7942+0+0 (secure 0 0 0) 0x7fc1580c6390 con 0x55d50004dc00 2024-03-21T10:28:31.302 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:31.297+0000 7fc17644a740 10 client.26290 fetch_fsmap finished waiting for FSMap version 32 2024-03-21T10:28:31.302 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:31.297+0000 7fc17644a740 10 client.26290 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-21T10:28:31.302 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:31.297+0000 7fc17644a740 20 client.26290 populate_metadata read hostname 'smithi047' 2024-03-21T10:28:31.302 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:31.297+0000 7fc17644a740 1 --2- 172.21.15.47:0/2221734273 >> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] conn(0x55d500066190 0x55d5000685f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:28:31.302 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:31.297+0000 7fc17644a740 4 client.26290 mds_command: new command op to 34535 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T10:28:31.302 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:31.297+0000 7fc17644a740 1 -- 172.21.15.47:0/2221734273 --> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55d4fffbe790 con 0x55d500066190 2024-03-21T10:28:31.303 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:31.299+0000 7fc170e00640 1 --2- 172.21.15.47:0/2221734273 >> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] conn(0x55d500066190 0x55d5000685f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:28:31.303 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:31.302+0000 7fc170e00640 1 --2- 172.21.15.47:0/2221734273 >> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] conn(0x55d500066190 0x55d5000685f0 crc :-1 s=READY pgs=113 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ede55f92a4395e01 server_cookie=10edbf1c8d8526c1 in_seq=0 out_seq=0 2024-03-21T10:28:31.303 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:31.302+0000 7fc152a00640 10 client.26290 ms_handle_connect on v2:172.21.15.100:6838/4173168440 2024-03-21T10:28:31.305 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:31.306+0000 7fc152a00640 1 -- 172.21.15.47:0/2221734273 <== mds.0 v2:172.21.15.100:6838/4173168440 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55d4fffbe790 con 0x55d500066190 2024-03-21T10:28:31.305 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:31.306+0000 7fc152a00640 10 client.26290 handle_command_reply: tid=0 2024-03-21T10:28:31.305 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:31.306+0000 7fc17644a740 1 client.26290 shutdown 2024-03-21T10:28:31.306 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:31.306+0000 7fc17644a740 1 -- 172.21.15.47:0/2221734273 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d50004dc00 msgr2=0x55d500052c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:28:31.306 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:31.306+0000 7fc17644a740 1 --2- 172.21.15.47:0/2221734273 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d50004dc00 0x55d500052c90 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fc1580ef170 tx=0x7fc1580ba170 comp rx=0 tx=0).stop 2024-03-21T10:28:31.306 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:31.307+0000 7fc17644a740 1 -- 172.21.15.47:0/2221734273 shutdown_connections 2024-03-21T10:28:31.306 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:31.307+0000 7fc17644a740 1 --2- 172.21.15.47:0/2221734273 >> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] conn(0x55d500066190 0x55d5000685f0 crc :-1 s=READY pgs=113 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:28:31.306 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:31.307+0000 7fc17644a740 1 --2- 172.21.15.47:0/2221734273 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d500056290 0x55d5000531d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:28:31.306 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:31.307+0000 7fc17644a740 1 --2- 172.21.15.47:0/2221734273 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d50004dc00 0x55d500052c90 secure :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fc1580ef170 tx=0x7fc1580ba170 comp rx=0 tx=0).stop 2024-03-21T10:28:31.306 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:31.307+0000 7fc17644a740 1 --2- 172.21.15.47:0/2221734273 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d50004cf20 0x55d500055900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:28:31.306 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:31.307+0000 7fc17644a740 1 -- 172.21.15.47:0/2221734273 >> 172.21.15.47:0/2221734273 conn(0x55d500014200 msgr2=0x55d5000496c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:28:31.307 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:31.307+0000 7fc152a00640 0 client.26290 ms_handle_reset on v2:172.21.15.100:6838/4173168440 2024-03-21T10:28:31.307 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:31.307+0000 7fc17644a740 1 -- 172.21.15.47:0/2221734273 shutdown_connections 2024-03-21T10:28:31.307 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:31.307+0000 7fc17644a740 1 -- 172.21.15.47:0/2221734273 wait complete. 2024-03-21T10:28:31.307 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:31.308+0000 7fc17644a740 20 client.26290 trim_cache size 0 max 16384 2024-03-21T10:28:31.369 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:31.368+0000 7fc17644a740 1 Processor -- start 2024-03-21T10:28:31.369 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:31.369+0000 7fc17644a740 1 -- start start 2024-03-21T10:28:31.369 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:31.369+0000 7fc17644a740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d50004cf20 0x55d500049440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:28:31.369 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:31.369+0000 7fc17644a740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d500056290 0x55d5000553c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:28:31.369 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:31.369+0000 7fc17644a740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d500049980 0x55d500055900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:28:31.369 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:31.369+0000 7fc17644a740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc14c00fce0 con 0x55d50004cf20 2024-03-21T10:28:31.369 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:31.369+0000 7fc17644a740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc14c011660 con 0x55d500056290 2024-03-21T10:28:31.369 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:31.369+0000 7fc17644a740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc14c010340 con 0x55d500049980 2024-03-21T10:28:31.370 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:31.371+0000 7fc171800640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d500049980 0x55d500055900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:28:31.370 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:31.371+0000 7fc16be00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d500056290 0x55d5000553c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:28:31.371 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:31.371+0000 7fc16be00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d500056290 0x55d5000553c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:40552/0 (socket says 172.21.15.47:40552) 2024-03-21T10:28:31.371 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:31.371+0000 7fc16be00640 1 -- 172.21.15.47:0/3443357747 learned_addr learned my addr 172.21.15.47:0/3443357747 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:28:31.371 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:31.371+0000 7fc170e00640 1 --2- 172.21.15.47:0/3443357747 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d50004cf20 0x55d500049440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:28:31.371 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:31.371+0000 7fc170e00640 1 -- 172.21.15.47:0/3443357747 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d500049980 msgr2=0x55d500055900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:28:31.371 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:31.371+0000 7fc170e00640 1 --2- 172.21.15.47:0/3443357747 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d500049980 0x55d500055900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:28:31.371 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:31.371+0000 7fc170e00640 1 -- 172.21.15.47:0/3443357747 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d500056290 msgr2=0x55d5000553c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:28:31.372 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:31.371+0000 7fc170e00640 1 --2- 172.21.15.47:0/3443357747 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d500056290 0x55d5000553c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:28:31.372 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:31.371+0000 7fc170e00640 1 -- 172.21.15.47:0/3443357747 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d4ffc850f0 con 0x55d50004cf20 2024-03-21T10:28:31.372 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:31.372+0000 7fc170e00640 1 --2- 172.21.15.47:0/3443357747 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d50004cf20 0x55d500049440 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x55d500066570 tx=0x7fc1600047d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=67383576999cc9e0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:28:31.372 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:31.373+0000 7fc15d600640 1 -- 172.21.15.47:0/3443357747 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc16000f040 con 0x55d50004cf20 2024-03-21T10:28:31.373 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:31.373+0000 7fc15d600640 1 -- 172.21.15.47:0/3443357747 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fc1600242f0 con 0x55d50004cf20 2024-03-21T10:28:31.373 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:31.373+0000 7fc15d600640 1 -- 172.21.15.47:0/3443357747 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc16000f040 con 0x55d50004cf20 2024-03-21T10:28:31.374 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:31.374+0000 7fc17644a740 1 -- 172.21.15.47:0/3443357747 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d50004cf20 msgr2=0x55d500049440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:28:31.374 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:31.375+0000 7fc17644a740 1 --2- 172.21.15.47:0/3443357747 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d50004cf20 0x55d500049440 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x55d500066570 tx=0x7fc1600047d0 comp rx=0 tx=0).stop 2024-03-21T10:28:31.375 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:31.375+0000 7fc17644a740 1 -- 172.21.15.47:0/3443357747 shutdown_connections 2024-03-21T10:28:31.375 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:31.376+0000 7fc17644a740 1 --2- 172.21.15.47:0/3443357747 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d500049980 0x55d500055900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:28:31.376 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:31.376+0000 7fc17644a740 1 --2- 172.21.15.47:0/3443357747 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d500056290 0x55d5000553c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:28:31.376 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:31.376+0000 7fc17644a740 1 --2- 172.21.15.47:0/3443357747 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d50004cf20 0x55d500049440 unknown :-1 s=CLOSED pgs=742 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:28:31.376 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:31.376+0000 7fc17644a740 1 -- 172.21.15.47:0/3443357747 >> 172.21.15.47:0/3443357747 conn(0x55d500014200 msgr2=0x55d500048d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:28:31.380 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:31.380+0000 7fc17644a740 1 -- 172.21.15.47:0/3443357747 shutdown_connections 2024-03-21T10:28:31.382 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:31.383+0000 7fc17644a740 1 -- 172.21.15.47:0/3443357747 wait complete. 2024-03-21T10:28:31.383 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:31.384+0000 7fc17644a740 1 -- start start 2024-03-21T10:28:31.383 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:31.384+0000 7fc17644a740 1 Processor -- start 2024-03-21T10:28:31.384 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:31.385+0000 7fc17644a740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d500049980 0x55d50005b570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:28:31.385 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:31.386+0000 7fc17644a740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d50004cf20 0x55d50005bab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:28:31.385 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:31.386+0000 7fc17644a740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d500056290 0x55d50005bff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:28:31.386 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:31.386+0000 7fc170e00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d500049980 0x55d50005b570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:28:31.387 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:31.386+0000 7fc17644a740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc14c011000 con 0x55d50004cf20 2024-03-21T10:28:31.387 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:31.386+0000 7fc17644a740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc14c0109a0 con 0x55d500056290 2024-03-21T10:28:31.387 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:31.386+0000 7fc17644a740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc14c00dea0 con 0x55d500049980 2024-03-21T10:28:31.387 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:31.386+0000 7fc17644a740 10 client.0 _ll_register_callbacks cb 0x7fc1580092a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T10:28:31.387 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:31.387+0000 7fc16be00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d50004cf20 0x55d50005bab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:28:31.387 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:31.387+0000 7fc171800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d500056290 0x55d50005bff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:28:31.387 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:31.387+0000 7fc171800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d500056290 0x55d50005bff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:40566/0 (socket says 172.21.15.47:40566) 2024-03-21T10:28:31.387 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:31.387+0000 7fc171800640 1 -- 172.21.15.47:0/1487517709 learned_addr learned my addr 172.21.15.47:0/1487517709 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:28:31.387 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:31.387+0000 7fc16be00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d50004cf20 0x55d50005bab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:36140/0 (socket says 172.21.15.47:36140) 2024-03-21T10:28:31.387 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:31.388+0000 7fc171800640 1 -- 172.21.15.47:0/1487517709 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d500049980 msgr2=0x55d50005b570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:28:31.388 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:31.388+0000 7fc171800640 1 --2- 172.21.15.47:0/1487517709 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d500049980 0x55d50005b570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:28:31.388 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:31.388+0000 7fc171800640 1 -- 172.21.15.47:0/1487517709 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d50004cf20 msgr2=0x55d50005bab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:28:31.388 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:31.388+0000 7fc171800640 1 --2- 172.21.15.47:0/1487517709 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d50004cf20 0x55d50005bab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:28:31.388 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:31.388+0000 7fc171800640 1 -- 172.21.15.47:0/1487517709 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55d500050c60 con 0x55d500056290 2024-03-21T10:28:31.388 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:31.389+0000 7fc152a00640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-21T10:28:31.388 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:31.389+0000 7fc171800640 1 --2- 172.21.15.47:0/1487517709 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d500056290 0x55d50005bff0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fc164009550 tx=0x7fc164011bc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e27a76eda245a399 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:28:31.389 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:31.389+0000 7fc152a00640 1 -- 172.21.15.47:0/1487517709 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc164016040 con 0x55d500056290 2024-03-21T10:28:31.389 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:31.389+0000 7fc17644a740 1 -- 172.21.15.47:0/1487517709 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55d4fffb8ae0 con 0x55d500056290 2024-03-21T10:28:31.389 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:31.390+0000 7fc152a00640 1 -- 172.21.15.47:0/1487517709 <== mon.2 v2:172.21.15.100:3300/0 2 ==== osd_map(97..97 src has 1..97) v4 ==== 9474+0+0 (secure 0 0 0) 0x7fc164011d30 con 0x55d500056290 2024-03-21T10:28:31.389 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:31.390+0000 7fc152a00640 1 -- 172.21.15.47:0/1487517709 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc164016040 con 0x55d500056290 2024-03-21T10:28:31.390 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:31.391+0000 7fc17644a740 10 client.36542 fetch_fsmap learned FSMap version 32 2024-03-21T10:28:31.390 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:31.391+0000 7fc17644a740 1 -- 172.21.15.47:0/1487517709 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x55d5000c2b60 con 0x55d500056290 2024-03-21T10:28:31.391 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:31.391+0000 7fc152a00640 1 -- 172.21.15.47:0/1487517709 <== mon.2 v2:172.21.15.100:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x55d4fffb8ae0 con 0x55d500056290 2024-03-21T10:28:31.392 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:31.392+0000 7fc152a00640 1 -- 172.21.15.47:0/1487517709 <== mon.2 v2:172.21.15.100:3300/0 5 ==== fsmap(e 32) v1 ==== 7942+0+0 (secure 0 0 0) 0x7fc16400f040 con 0x55d500056290 2024-03-21T10:28:31.392 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:31.392+0000 7fc17644a740 10 client.36542 fetch_fsmap finished waiting for FSMap version 32 2024-03-21T10:28:31.392 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:31.392+0000 7fc17644a740 10 client.36542 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-21T10:28:31.392 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:31.392+0000 7fc17644a740 20 client.36542 populate_metadata read hostname 'smithi047' 2024-03-21T10:28:31.392 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:31.392+0000 7fc17644a740 1 --2- 172.21.15.47:0/1487517709 >> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] conn(0x55d5000c30f0 0x55d5000c5510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:28:31.392 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:31.392+0000 7fc17644a740 4 client.36542 mds_command: new command op to 34535 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-21T10:28:31.392 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:31.392+0000 7fc17644a740 1 -- 172.21.15.47:0/1487517709 --> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55d4fff363a0 con 0x55d5000c30f0 2024-03-21T10:28:31.393 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:31.393+0000 7fc170e00640 1 --2- 172.21.15.47:0/1487517709 >> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] conn(0x55d5000c30f0 0x55d5000c5510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:28:31.394 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:31.395+0000 7fc170e00640 1 --2- 172.21.15.47:0/1487517709 >> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] conn(0x55d5000c30f0 0x55d5000c5510 crc :-1 s=READY pgs=114 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6b4e59f91cb2234a server_cookie=a07415a948a66fa0 in_seq=0 out_seq=0 2024-03-21T10:28:31.394 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:31.395+0000 7fc152a00640 10 client.36542 ms_handle_connect on v2:172.21.15.100:6838/4173168440 2024-03-21T10:28:31.394 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:31.395+0000 7fc152a00640 1 -- 172.21.15.47:0/1487517709 <== mds.0 v2:172.21.15.100:6838/4173168440 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x55d4fff363a0 con 0x55d5000c30f0 2024-03-21T10:28:31.395 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:31.395+0000 7fc152a00640 10 client.36542 handle_command_reply: tid=0 2024-03-21T10:28:31.395 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:31.395+0000 7fc17644a740 1 client.36542 shutdown 2024-03-21T10:28:31.395 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:31.395+0000 7fc17644a740 1 -- 172.21.15.47:0/1487517709 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d500056290 msgr2=0x55d50005bff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:28:31.395 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:31.395+0000 7fc17644a740 1 --2- 172.21.15.47:0/1487517709 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d500056290 0x55d50005bff0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fc164009550 tx=0x7fc164011bc0 comp rx=0 tx=0).stop 2024-03-21T10:28:31.396 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:31.397+0000 7fc17644a740 1 -- 172.21.15.47:0/1487517709 shutdown_connections 2024-03-21T10:28:31.396 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:31.397+0000 7fc17644a740 1 --2- 172.21.15.47:0/1487517709 >> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] conn(0x55d5000c30f0 0x55d5000c5510 crc :-1 s=READY pgs=114 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:28:31.396 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:31.397+0000 7fc17644a740 1 --2- 172.21.15.47:0/1487517709 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d500056290 0x55d50005bff0 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:28:31.396 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:31.397+0000 7fc152a00640 0 client.36542 ms_handle_reset on v2:172.21.15.100:6838/4173168440 2024-03-21T10:28:31.396 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:31.397+0000 7fc17644a740 1 --2- 172.21.15.47:0/1487517709 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d50004cf20 0x55d50005bab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:28:31.396 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:31.397+0000 7fc17644a740 1 --2- 172.21.15.47:0/1487517709 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d500049980 0x55d50005b570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:28:31.396 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:31.397+0000 7fc17644a740 1 -- 172.21.15.47:0/1487517709 >> 172.21.15.47:0/1487517709 conn(0x55d500014200 msgr2=0x55d500046200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:28:31.397 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:31.397+0000 7fc17644a740 1 -- 172.21.15.47:0/1487517709 shutdown_connections 2024-03-21T10:28:31.397 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:31.397+0000 7fc17644a740 1 -- 172.21.15.47:0/1487517709 wait complete. 2024-03-21T10:28:31.397 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:31.397+0000 7fc17644a740 20 client.36542 trim_cache size 0 max 16384 2024-03-21T10:28:31.397 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-03-21T10:28:31.398 INFO:teuthology.orchestra.run.smithi047.stdout: "status": "no active scrubs running", 2024-03-21T10:28:31.398 INFO:teuthology.orchestra.run.smithi047.stdout: "scrubs": {} 2024-03-21T10:28:31.398 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-03-21T10:28:31.398 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:31.398+0000 7fc173800640 1 -- 172.21.15.47:0/1107227945 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7fc1540a3190 msgr2=0x7fc1540a5650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:28:31.398 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:31.398+0000 7fc173800640 1 --2- 172.21.15.47:0/1107227945 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7fc1540a3190 0x7fc1540a5650 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x55d50004e150 tx=0x7fc160008040 comp rx=0 tx=0).stop 2024-03-21T10:28:31.399 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:31.398+0000 7fc173800640 1 -- 172.21.15.47:0/1107227945 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc16c100dd0 msgr2=0x7fc16c102de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:28:31.399 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:31.398+0000 7fc173800640 1 --2- 172.21.15.47:0/1107227945 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc16c100dd0 0x7fc16c102de0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fc15802c170 tx=0x7fc15802ca10 comp rx=0 tx=0).stop 2024-03-21T10:28:31.399 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:31.399+0000 7fc173800640 1 -- 172.21.15.47:0/1107227945 shutdown_connections 2024-03-21T10:28:31.399 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:31.399+0000 7fc173800640 1 --2- 172.21.15.47:0/1107227945 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7fc1540a3190 0x7fc1540a5650 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:28:31.399 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:31.399+0000 7fc173800640 1 --2- 172.21.15.47:0/1107227945 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc16c10e060 0x7fc16c106c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:28:31.399 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:31.399+0000 7fc173800640 1 --2- 172.21.15.47:0/1107227945 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc16c100dd0 0x7fc16c102de0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:28:31.399 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:31.399+0000 7fc173800640 1 --2- 172.21.15.47:0/1107227945 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc16c1004b0 0x7fc16c1028a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:28:31.399 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:31.399+0000 7fc173800640 1 -- 172.21.15.47:0/1107227945 >> 172.21.15.47:0/1107227945 conn(0x7fc16c077700 msgr2=0x7fc16c10f2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:28:31.399 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:31.399+0000 7fc173800640 1 -- 172.21.15.47:0/1107227945 shutdown_connections 2024-03-21T10:28:31.399 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:31.399+0000 7fc173800640 1 -- 172.21.15.47:0/1107227945 wait complete. 2024-03-21T10:28:31.425 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:28:31 smithi047 ceph-mon[31547]: pgmap v99: 129 pgs: 129 active+clean; 416 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 7.3 MiB/s wr, 1.58k op/s 2024-03-21T10:28:31.432 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-03-21T10:28:31.432 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-21T10:28:31.478 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:28:31 smithi100 ceph-mon[35193]: pgmap v99: 129 pgs: 129 active+clean; 416 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 7.3 MiB/s wr, 1.58k op/s 2024-03-21T10:28:31.488 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:28:31 smithi157 ceph-mon[35581]: pgmap v99: 129 pgs: 129 active+clean; 416 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 7.3 MiB/s wr, 1.58k op/s 2024-03-21T10:28:31.725 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:31.725+0000 7f5845400640 1 -- 172.21.15.47:0/2768927922 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5840074390 msgr2=0x7f584010d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:28:31.725 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:31.725+0000 7f5845400640 1 --2- 172.21.15.47:0/2768927922 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5840074390 0x7f584010d990 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f5828009900 tx=0x7f582802bcc0 comp rx=0 tx=0).stop 2024-03-21T10:28:31.729 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:31.729+0000 7f5845400640 1 -- 172.21.15.47:0/2768927922 shutdown_connections 2024-03-21T10:28:31.729 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:31.729+0000 7f5845400640 1 --2- 172.21.15.47:0/2768927922 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f584010ded0 0x7f58401102e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:28:31.729 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:31.729+0000 7f5845400640 1 --2- 172.21.15.47:0/2768927922 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5840074390 0x7f584010d990 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:28:31.729 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:31.729+0000 7f5845400640 1 --2- 172.21.15.47:0/2768927922 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5840073a70 0x7f5840073e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:28:31.729 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:31.729+0000 7f5845400640 1 -- 172.21.15.47:0/2768927922 >> 172.21.15.47:0/2768927922 conn(0x7f5840078aa0 msgr2=0x7f5840078eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:28:31.730 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:31.730+0000 7f5845400640 1 -- 172.21.15.47:0/2768927922 shutdown_connections 2024-03-21T10:28:31.730 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:31.731+0000 7f5845400640 1 -- 172.21.15.47:0/2768927922 wait complete. 2024-03-21T10:28:31.733 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:31.734+0000 7f5845400640 1 Processor -- start 2024-03-21T10:28:31.733 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:31.734+0000 7f5845400640 1 -- start start 2024-03-21T10:28:31.735 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:31.734+0000 7f5845400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5840073a70 0x7f5840199f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:28:31.735 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:31.734+0000 7f5845400640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5840074390 0x7f584019a450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:28:31.735 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:31.735+0000 7f583de00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5840074390 0x7f584019a450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:28:31.735 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:31.735+0000 7f583de00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5840074390 0x7f584019a450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.47:40124/0 (socket says 172.21.15.47:40124) 2024-03-21T10:28:31.735 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:31.735+0000 7f583e800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5840073a70 0x7f5840199f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:28:31.735 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:31.735+0000 7f583e800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5840073a70 0x7f5840199f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:40580/0 (socket says 172.21.15.47:40580) 2024-03-21T10:28:31.736 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:31.736+0000 7f5845400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f584010ded0 0x7f584019e780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:28:31.736 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:31.736+0000 7f583de00640 1 -- 172.21.15.47:0/2847510263 learned_addr learned my addr 172.21.15.47:0/2847510263 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:28:31.736 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:31.736+0000 7f5845400640 1 -- 172.21.15.47:0/2847510263 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5840113b30 con 0x7f584010ded0 2024-03-21T10:28:31.736 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:31.736+0000 7f5845400640 1 -- 172.21.15.47:0/2847510263 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58401139b0 con 0x7f5840073a70 2024-03-21T10:28:31.736 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:31.736+0000 7f5845400640 1 -- 172.21.15.47:0/2847510263 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5840113cb0 con 0x7f5840074390 2024-03-21T10:28:31.737 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:31.738+0000 7f583de00640 1 -- 172.21.15.47:0/2847510263 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5840073a70 msgr2=0x7f5840199f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:28:31.738 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:31.738+0000 7f583de00640 1 --2- 172.21.15.47:0/2847510263 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5840073a70 0x7f5840199f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:28:31.738 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:31.738+0000 7f583de00640 1 -- 172.21.15.47:0/2847510263 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f584010ded0 msgr2=0x7f584019e780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:28:31.738 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:31.738+0000 7f583de00640 1 --2- 172.21.15.47:0/2847510263 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f584010ded0 0x7f584019e780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:28:31.738 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:31.738+0000 7f583de00640 1 -- 172.21.15.47:0/2847510263 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f584019eeb0 con 0x7f5840074390 2024-03-21T10:28:31.741 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:31.740+0000 7f583de00640 1 --2- 172.21.15.47:0/2847510263 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5840074390 0x7f584019a450 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f582802c200 tx=0x7f582802dab0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:28:31.741 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:31.741+0000 7f5837400640 1 -- 172.21.15.47:0/2847510263 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5828042020 con 0x7f5840074390 2024-03-21T10:28:31.741 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:31.741+0000 7f5837400640 1 -- 172.21.15.47:0/2847510263 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f58280052b0 con 0x7f5840074390 2024-03-21T10:28:31.741 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:31.741+0000 7f5837400640 1 -- 172.21.15.47:0/2847510263 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5828003d00 con 0x7f5840074390 2024-03-21T10:28:31.741 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:31.741+0000 7f5845400640 1 -- 172.21.15.47:0/2847510263 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f584019f140 con 0x7f5840074390 2024-03-21T10:28:31.741 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:31.742+0000 7f5845400640 1 -- 172.21.15.47:0/2847510263 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f58401a6a30 con 0x7f5840074390 2024-03-21T10:28:31.746 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:31.747+0000 7f5845400640 1 -- 172.21.15.47:0/2847510263 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f58400688e0 con 0x7f5840074390 2024-03-21T10:28:31.749 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:31.748+0000 7f5837400640 1 -- 172.21.15.47:0/2847510263 <== mon.1 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 27) v1 ==== 137985+0+0 (secure 0 0 0) 0x7f5828045020 con 0x7f5840074390 2024-03-21T10:28:31.751 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:31.751+0000 7f5837400640 1 --2- 172.21.15.47:0/2847510263 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f58240a3110 0x7f58240a55d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:28:31.751 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:31.752+0000 7f5837400640 1 -- 172.21.15.47:0/2847510263 <== mon.1 v2:172.21.15.157:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 9474+0+0 (secure 0 0 0) 0x7f58280f1c60 con 0x7f5840074390 2024-03-21T10:28:31.751 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:31.752+0000 7f583e800640 1 --2- 172.21.15.47:0/2847510263 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f58240a3110 0x7f58240a55d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:28:31.754 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:31.754+0000 7f583e800640 1 --2- 172.21.15.47:0/2847510263 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f58240a3110 0x7f58240a55d0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f583000bbe0 tx=0x7f583000b390 comp rx=0 tx=0).ready entity=mgr.15687 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:28:31.759 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:31.759+0000 7f5837400640 1 -- 172.21.15.47:0/2847510263 <== mon.1 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f582804c020 con 0x7f5840074390 2024-03-21T10:28:32.300 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:32.301+0000 7f5845400640 1 -- 172.21.15.47:0/2847510263 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f58401a6cc0 con 0x7f5840074390 2024-03-21T10:28:32.302 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:32.302+0000 7f5837400640 1 -- 172.21.15.47:0/2847510263 <== mon.1 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 32 v32) v1 ==== 94+0+11471 (secure 0 0 0) 0x7f58280be4a0 con 0x7f5840074390 2024-03-21T10:28:32.302 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-21T10:28:32.303 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":32,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14511,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6835/735724037","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6834","nonce":735724037},{"type":"v1","addr":"172.21.15.47:6835","nonce":735724037}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14529,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6837/2248191724","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6836","nonce":2248191724},{"type":"v1","addr":"172.21.15.47:6837","nonce":2248191724}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":32,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-21T10:21:58.329063+0000","modified":"2024-03-21T10:26:17.169892+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":34535,"mds_1":34493,"mds_2":34454,"mds_3":34427,"mds_4":14619},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14604":{"gid":14604,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.47:6839/2881335356","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6838","nonce":2881335356},{"type":"v1","addr":"172.21.15.47:6839","nonce":2881335356}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_14619":{"gid":14619,"name":"a","rank":4,"incarnation":27,"state":"up:active","state_seq":11,"addr":"172.21.15.47:6841/3401578978","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6840","nonce":3401578978},{"type":"v1","addr":"172.21.15.47:6841","nonce":3401578978}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24406":{"gid":24406,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.157:6833/4000010136","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":4000010136},{"type":"v1","addr":"172.21.15.157:6833","nonce":4000010136}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24457":{"gid":24457,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.157:6835/3902868651","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":3902868651},{"type":"v1","addr":"172.21.15.157:6835","nonce":3902868651}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24475":{"gid":24475,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.157:6837/1908087204","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6836","nonce":1908087204},{"type":"v1","addr":"172.21.15.157:6837","nonce":1908087204}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_25729":{"gid":25729,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.157:6839/1998408416","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6838","nonce":1998408416},{"type":"v1","addr":"172.21.15.157:6839","nonce":1998408416}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_34427":{"gid":34427,"name":"k","rank":3,"incarnation":25,"state":"up:active","state_seq":20,"addr":"172.21.15.100:6833/2723302038","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6832","nonce":2723302038},{"type":"v1","addr":"172.21.15.100:6833","nonce":2723302038}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_34454":{"gid":34454,"name":"e","rank":2,"incarnation":22,"state":"up:active","state_seq":17,"addr":"172.21.15.100:6835/2597849957","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":2597849957},{"type":"v1","addr":"172.21.15.100:6835","nonce":2597849957}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_34493":{"gid":34493,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":13,"addr":"172.21.15.100:6837/3497644162","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6836","nonce":3497644162},{"type":"v1","addr":"172.21.15.100:6837","nonce":3497644162}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_34535":{"gid":34535,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.100:6839/4173168440","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6838","nonce":4173168440},{"type":"v1","addr":"172.21.15.100:6839","nonce":4173168440}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":34535,"qdb_cluster":[34427,14619,34454,34535,34493]},"id":1}]} 2024-03-21T10:28:32.303 INFO:teuthology.orchestra.run.smithi047.stderr:dumped fsmap epoch 32 2024-03-21T10:28:32.307 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:32.307+0000 7f5845400640 1 -- 172.21.15.47:0/2847510263 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f58240a3110 msgr2=0x7f58240a55d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:28:32.307 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:32.307+0000 7f5845400640 1 --2- 172.21.15.47:0/2847510263 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f58240a3110 0x7f58240a55d0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f583000bbe0 tx=0x7f583000b390 comp rx=0 tx=0).stop 2024-03-21T10:28:32.307 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:32.307+0000 7f5845400640 1 -- 172.21.15.47:0/2847510263 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5840074390 msgr2=0x7f584019a450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:28:32.308 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:32.307+0000 7f5845400640 1 --2- 172.21.15.47:0/2847510263 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5840074390 0x7f584019a450 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f582802c200 tx=0x7f582802dab0 comp rx=0 tx=0).stop 2024-03-21T10:28:32.308 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:32.308+0000 7f5845400640 1 -- 172.21.15.47:0/2847510263 shutdown_connections 2024-03-21T10:28:32.308 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:32.308+0000 7f5845400640 1 --2- 172.21.15.47:0/2847510263 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f584010ded0 0x7f584019e780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:28:32.308 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:32.308+0000 7f5845400640 1 --2- 172.21.15.47:0/2847510263 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5840074390 0x7f584019a450 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:28:32.309 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:32.308+0000 7f5845400640 1 --2- 172.21.15.47:0/2847510263 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f58240a3110 0x7f58240a55d0 secure :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f583000bbe0 tx=0x7f583000b390 comp rx=0 tx=0).stop 2024-03-21T10:28:32.309 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:32.308+0000 7f5845400640 1 --2- 172.21.15.47:0/2847510263 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5840073a70 0x7f5840199f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:28:32.309 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:32.308+0000 7f5845400640 1 -- 172.21.15.47:0/2847510263 >> 172.21.15.47:0/2847510263 conn(0x7f5840078aa0 msgr2=0x7f58400fe420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:28:32.309 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:32.308+0000 7f5845400640 1 -- 172.21.15.47:0/2847510263 shutdown_connections 2024-03-21T10:28:32.309 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:32.308+0000 7f5845400640 1 -- 172.21.15.47:0/2847510263 wait complete. 2024-03-21T10:28:32.339 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-21T10:28:32.339 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-03-21T10:28:32.657 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:32.658+0000 7f6ec1000640 1 -- 172.21.15.47:0/3440571905 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6ebc101ff0 msgr2=0x7f6ebc10e500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:28:32.658 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:32.658+0000 7f6ec1000640 1 --2- 172.21.15.47:0/3440571905 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6ebc101ff0 0x7f6ebc10e500 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f6eb4009ca0 tx=0x7f6eb402c100 comp rx=0 tx=0).stop 2024-03-21T10:28:32.658 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:32.659+0000 7f6ec1000640 1 -- 172.21.15.47:0/3440571905 shutdown_connections 2024-03-21T10:28:32.658 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:32.659+0000 7f6ec1000640 1 --2- 172.21.15.47:0/3440571905 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6ebc101ff0 0x7f6ebc10e500 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:28:32.658 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:32.659+0000 7f6ec1000640 1 --2- 172.21.15.47:0/3440571905 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6ebc101630 0x7f6ebc101ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:28:32.658 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:32.659+0000 7f6ec1000640 1 --2- 172.21.15.47:0/3440571905 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6ebc107640 0x7f6ebc107a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:28:32.658 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:32.659+0000 7f6ec1000640 1 -- 172.21.15.47:0/3440571905 >> 172.21.15.47:0/3440571905 conn(0x7f6ebc078a60 msgr2=0x7f6ebc078e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:28:32.659 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:32.660+0000 7f6ec1000640 1 -- 172.21.15.47:0/3440571905 shutdown_connections 2024-03-21T10:28:32.659 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:32.660+0000 7f6ec1000640 1 -- 172.21.15.47:0/3440571905 wait complete. 2024-03-21T10:28:32.661 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:32.661+0000 7f6ec1000640 1 Processor -- start 2024-03-21T10:28:32.662 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:32.661+0000 7f6ec1000640 1 -- start start 2024-03-21T10:28:32.662 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:32.662+0000 7f6ec1000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6ebc101630 0x7f6ebc10e120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:28:32.662 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:32.662+0000 7f6ec1000640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6ebc101ff0 0x7f6ebc10a1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:28:32.662 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:32.662+0000 7f6ec1000640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6ebc107640 0x7f6ebc10a6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:28:32.662 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:32.662+0000 7f6ec1000640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ebc073ff0 con 0x7f6ebc107640 2024-03-21T10:28:32.662 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:32.662+0000 7f6ec1000640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ebc073e70 con 0x7f6ebc101630 2024-03-21T10:28:32.662 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:32.662+0000 7f6ec1000640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ebc074170 con 0x7f6ebc101ff0 2024-03-21T10:28:32.662 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:32.663+0000 7f6eb9e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6ebc101630 0x7f6ebc10e120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:28:32.663 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:32.663+0000 7f6eb9e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6ebc101630 0x7f6ebc10e120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:53834/0 (socket says 172.21.15.47:53834) 2024-03-21T10:28:32.663 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:32.663+0000 7f6eb9e00640 1 -- 172.21.15.47:0/2650488189 learned_addr learned my addr 172.21.15.47:0/2650488189 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:28:32.663 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:32.664+0000 7f6eba800640 1 --2- 172.21.15.47:0/2650488189 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6ebc107640 0x7f6ebc10a6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:28:32.664 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:32.664+0000 7f6eb9e00640 1 -- 172.21.15.47:0/2650488189 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6ebc101ff0 msgr2=0x7f6ebc10a1b0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-03-21T10:28:32.664 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:32.664+0000 7f6eb9e00640 1 --2- 172.21.15.47:0/2650488189 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6ebc101ff0 0x7f6ebc10a1b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:28:32.664 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:32.664+0000 7f6eb9e00640 1 -- 172.21.15.47:0/2650488189 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6ebc107640 msgr2=0x7f6ebc10a6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:28:32.664 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:32.664+0000 7f6eb9e00640 1 --2- 172.21.15.47:0/2650488189 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6ebc107640 0x7f6ebc10a6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:28:32.664 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:32.664+0000 7f6eb9e00640 1 -- 172.21.15.47:0/2650488189 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ebc10ad30 con 0x7f6ebc101630 2024-03-21T10:28:32.667 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:32.667+0000 7f6eb9e00640 1 --2- 172.21.15.47:0/2650488189 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6ebc101630 0x7f6ebc10e120 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f6ea800b4d0 tx=0x7f6ea800b990 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:28:32.669 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:32.669+0000 7f6eb2a00640 1 -- 172.21.15.47:0/2650488189 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ea8014070 con 0x7f6ebc101630 2024-03-21T10:28:32.669 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:32.669+0000 7f6eb2a00640 1 -- 172.21.15.47:0/2650488189 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f6ea8010d20 con 0x7f6ebc101630 2024-03-21T10:28:32.669 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:32.670+0000 7f6eb2a00640 1 -- 172.21.15.47:0/2650488189 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ea800fe10 con 0x7f6ebc101630 2024-03-21T10:28:32.669 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:32.670+0000 7f6ec1000640 1 -- 172.21.15.47:0/2650488189 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6ebc19c310 con 0x7f6ebc101630 2024-03-21T10:28:32.669 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:32.670+0000 7f6ec1000640 1 -- 172.21.15.47:0/2650488189 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6ebc19c810 con 0x7f6ebc101630 2024-03-21T10:28:32.675 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:32.676+0000 7f6ec3d64740 1 Processor -- start 2024-03-21T10:28:32.676 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:32.676+0000 7f6ec3d64740 1 -- start start 2024-03-21T10:28:32.678 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:32.678+0000 7f6ec3d64740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56547924f6d0 0x56547924fab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:28:32.678 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:32.678+0000 7f6ec3d64740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x565479250080 0x565479258630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:28:32.678 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:32.678+0000 7f6eb2a00640 1 -- 172.21.15.47:0/2650488189 <== mon.2 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 27) v1 ==== 137985+0+0 (secure 0 0 0) 0x7f6ea8017070 con 0x7f6ebc101630 2024-03-21T10:28:32.678 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:32.678+0000 7f6ec3d64740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x565479258c00 0x56547925b050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:28:32.678 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:32.678+0000 7f6ec3d64740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x5654791b32c0 con 0x565479258c00 2024-03-21T10:28:32.678 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:32.678+0000 7f6ec3d64740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x56547915bc90 con 0x56547924f6d0 2024-03-21T10:28:32.678 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:32.678+0000 7f6ec3d64740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x5654791af360 con 0x565479250080 2024-03-21T10:28:32.679 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:32.678+0000 7f6eba800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56547924f6d0 0x56547924fab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:28:32.679 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:32.679+0000 7f6eba800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56547924f6d0 0x56547924fab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:53842/0 (socket says 172.21.15.47:53842) 2024-03-21T10:28:32.679 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:32.679+0000 7f6eba800640 1 -- 172.21.15.47:0/411580178 learned_addr learned my addr 172.21.15.47:0/411580178 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:28:32.679 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:32.679+0000 7f6eb9e00640 1 --2- 172.21.15.47:0/411580178 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x565479250080 0x565479258630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:28:32.679 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:32.679+0000 7f6eba800640 1 -- 172.21.15.47:0/411580178 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x565479250080 msgr2=0x565479258630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:28:32.679 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:32.679+0000 7f6eba800640 1 --2- 172.21.15.47:0/411580178 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x565479250080 0x565479258630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:28:32.679 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:32.679+0000 7f6eba800640 1 -- 172.21.15.47:0/411580178 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x565479258c00 msgr2=0x56547925b050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:28:32.680 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:32.679+0000 7f6eb9400640 1 --2- 172.21.15.47:0/411580178 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x565479258c00 0x56547925b050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:28:32.680 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:32.680+0000 7f6eba800640 1 --2- 172.21.15.47:0/411580178 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x565479258c00 0x56547925b050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:28:32.680 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:32.680+0000 7f6eba800640 1 -- 172.21.15.47:0/411580178 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x565479180080 con 0x56547924f6d0 2024-03-21T10:28:32.680 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:32.680+0000 7f6eb9400640 1 --2- 172.21.15.47:0/411580178 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x565479258c00 0x56547925b050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T10:28:32.680 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:32.681+0000 7f6eba800640 1 --2- 172.21.15.47:0/411580178 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56547924f6d0 0x56547924fab0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f6eb4009ca0 tx=0x7f6eb4009800 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6082524c95ca9ada server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:28:32.681 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:32.682+0000 7f6ea7400640 1 -- 172.21.15.47:0/411580178 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6eb4043020 con 0x56547924f6d0 2024-03-21T10:28:32.681 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:32.682+0000 7f6ea7400640 1 -- 172.21.15.47:0/411580178 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f6eb4003b70 con 0x56547924f6d0 2024-03-21T10:28:32.681 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:32.682+0000 7f6ea7400640 1 -- 172.21.15.47:0/411580178 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6eb4043020 con 0x56547924f6d0 2024-03-21T10:28:32.687 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:32.684+0000 7f6ec3d64740 1 -- 172.21.15.47:0/411580178 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56547924f6d0 msgr2=0x56547924fab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:28:32.687 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:32.684+0000 7f6ec3d64740 1 --2- 172.21.15.47:0/411580178 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56547924f6d0 0x56547924fab0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f6eb4009ca0 tx=0x7f6eb4009800 comp rx=0 tx=0).stop 2024-03-21T10:28:32.687 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:32.684+0000 7f6ec3d64740 1 -- 172.21.15.47:0/411580178 shutdown_connections 2024-03-21T10:28:32.687 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:32.684+0000 7f6ec3d64740 1 --2- 172.21.15.47:0/411580178 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x565479258c00 0x56547925b050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:28:32.687 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:32.684+0000 7f6ec3d64740 1 --2- 172.21.15.47:0/411580178 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x565479250080 0x565479258630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:28:32.687 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:32.684+0000 7f6ec3d64740 1 --2- 172.21.15.47:0/411580178 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56547924f6d0 0x56547924fab0 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:28:32.687 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:32.684+0000 7f6ec3d64740 1 -- 172.21.15.47:0/411580178 >> 172.21.15.47:0/411580178 conn(0x565479249da0 msgr2=0x56547924a180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:28:32.687 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:32.684+0000 7f6ec3d64740 1 -- 172.21.15.47:0/411580178 shutdown_connections 2024-03-21T10:28:32.688 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:32.684+0000 7f6ec3d64740 1 -- 172.21.15.47:0/411580178 wait complete. 2024-03-21T10:28:32.688 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:32.685+0000 7f6ec3d64740 1 -- start start 2024-03-21T10:28:32.688 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:32.686+0000 7f6ec3d64740 1 Processor -- start 2024-03-21T10:28:32.688 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:32.686+0000 7f6eb2a00640 1 --2- 172.21.15.47:0/2650488189 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f6ea00a3170 0x7f6ea00a5630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:28:32.688 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:32.687+0000 7f6eb2a00640 1 -- 172.21.15.47:0/2650488189 <== mon.2 v2:172.21.15.100:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 9474+0+0 (secure 0 0 0) 0x7f6ea80cf530 con 0x7f6ebc101630 2024-03-21T10:28:32.688 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:32.687+0000 7f6eba800640 1 --2- 172.21.15.47:0/2650488189 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f6ea00a3170 0x7f6ea00a5630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:28:32.690 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:32.690+0000 7f6ec3d64740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56547924f6d0 0x565479258500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:28:32.690 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:32.690+0000 7f6ec3d64740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x565479250080 0x565479255890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:28:32.690 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:32.690+0000 7f6ec3d64740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x565479258c00 0x565479255dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:28:32.690 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:32.690+0000 7f6ec3d64740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x56547925bb10 con 0x56547924f6d0 2024-03-21T10:28:32.691 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:32.690+0000 7f6ec3d64740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x56547916f0d0 con 0x565479258c00 2024-03-21T10:28:32.691 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:32.690+0000 7f6ec3d64740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x56547925bc90 con 0x565479250080 2024-03-21T10:28:32.691 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:32.690+0000 7f6ec3d64740 10 client.0 _ll_register_callbacks cb 0x565479173e80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T10:28:32.691 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:32.691+0000 7f6eba800640 1 --2- 172.21.15.47:0/2650488189 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f6ea00a3170 0x7f6ea00a5630 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f6eb402f020 tx=0x7f6eb4036040 comp rx=0 tx=0).ready entity=mgr.15687 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:28:32.692 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:32.691+0000 7f6eb9400640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x565479250080 0x565479255890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:28:32.692 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:32.691+0000 7f6eb9e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56547924f6d0 0x565479258500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:28:32.692 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:32.691+0000 7f6eb9e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56547924f6d0 0x565479258500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:60428/0 (socket says 172.21.15.47:60428) 2024-03-21T10:28:32.692 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:32.691+0000 7f6eb9e00640 1 -- 172.21.15.47:0/967926510 learned_addr learned my addr 172.21.15.47:0/967926510 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:28:32.692 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:32.691+0000 7f6eba800640 1 --2- 172.21.15.47:0/967926510 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x565479258c00 0x565479255dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:28:32.692 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:32.692+0000 7f6eb9e00640 1 -- 172.21.15.47:0/967926510 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x565479250080 msgr2=0x565479255890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:28:32.692 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:32.692+0000 7f6eb9e00640 1 --2- 172.21.15.47:0/967926510 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x565479250080 0x565479255890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:28:32.692 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:32.692+0000 7f6eb9e00640 1 -- 172.21.15.47:0/967926510 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x565479258c00 msgr2=0x565479255dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:28:32.692 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:32.692+0000 7f6eb9e00640 1 --2- 172.21.15.47:0/967926510 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x565479258c00 0x565479255dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:28:32.692 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:32.692+0000 7f6eb9e00640 1 -- 172.21.15.47:0/967926510 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x565478ec73f0 con 0x56547924f6d0 2024-03-21T10:28:32.693 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:32.693+0000 7f6eb9e00640 1 --2- 172.21.15.47:0/967926510 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56547924f6d0 0x565479258500 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7f6ea800b4a0 tx=0x7f6ea809b3e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=60ff5e67b721ec28 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:28:32.693 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:32.693+0000 7f6e9be00640 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-03-21T10:28:32.694 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:32.695+0000 7f6e9be00640 1 -- 172.21.15.47:0/967926510 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ea801e070 con 0x56547924f6d0 2024-03-21T10:28:32.695 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:32.695+0000 7f6e9be00640 1 -- 172.21.15.47:0/967926510 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(97..97 src has 1..97) v4 ==== 9474+0+0 (secure 0 0 0) 0x7f6ea80a3b90 con 0x56547924f6d0 2024-03-21T10:28:32.695 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:32.695+0000 7f6ec3d64740 1 -- 172.21.15.47:0/967926510 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f6eb4003b70 con 0x56547924f6d0 2024-03-21T10:28:32.698 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:32.698+0000 7f6e9be00640 1 -- 172.21.15.47:0/967926510 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6ea80092b0 con 0x56547924f6d0 2024-03-21T10:28:32.698 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:32.699+0000 7f6ec3d64740 10 client.16743 fetch_fsmap learned FSMap version 32 2024-03-21T10:28:32.698 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:32.699+0000 7f6ec3d64740 1 -- 172.21.15.47:0/967926510 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x565478f917d0 con 0x56547924f6d0 2024-03-21T10:28:32.700 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:32.700+0000 7f6e9be00640 1 -- 172.21.15.47:0/967926510 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f6eb4003b70 con 0x56547924f6d0 2024-03-21T10:28:32.700 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:32.700+0000 7f6e9be00640 1 -- 172.21.15.47:0/967926510 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 32) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f6ea809aa60 con 0x56547924f6d0 2024-03-21T10:28:32.700 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:32.701+0000 7f6ec3d64740 10 client.16743 fetch_fsmap finished waiting for FSMap version 32 2024-03-21T10:28:32.701 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:32.701+0000 7f6ec3d64740 10 client.16743 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.a 2024-03-21T10:28:32.701 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:32.701+0000 7f6ec3d64740 20 client.16743 populate_metadata read hostname 'smithi047' 2024-03-21T10:28:32.701 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:32.701+0000 7f6ec3d64740 1 --2- 172.21.15.47:0/967926510 >> [v2:172.21.15.47:6840/3401578978,v1:172.21.15.47:6841/3401578978] conn(0x5654792689e0 0x56547926add0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:28:32.701 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:32.701+0000 7f6ec3d64740 4 client.16743 mds_command: new command op to 14619 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T10:28:32.701 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:32.702+0000 7f6ec3d64740 1 -- 172.21.15.47:0/967926510 --> [v2:172.21.15.47:6840/3401578978,v1:172.21.15.47:6841/3401578978] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5654791c0800 con 0x5654792689e0 2024-03-21T10:28:32.702 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:32.703+0000 7f6eb9400640 1 --2- 172.21.15.47:0/967926510 >> [v2:172.21.15.47:6840/3401578978,v1:172.21.15.47:6841/3401578978] conn(0x5654792689e0 0x56547926add0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:28:32.705 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:32.706+0000 7f6eb9400640 1 --2- 172.21.15.47:0/967926510 >> [v2:172.21.15.47:6840/3401578978,v1:172.21.15.47:6841/3401578978] conn(0x5654792689e0 0x56547926add0 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=c6ddcc0e7b8bc42b server_cookie=25df1d7c1e972df in_seq=0 out_seq=0 2024-03-21T10:28:32.705 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:32.706+0000 7f6e9be00640 10 client.16743 ms_handle_connect on v2:172.21.15.47:6840/3401578978 2024-03-21T10:28:32.707 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:32.708+0000 7f6e9be00640 1 -- 172.21.15.47:0/967926510 <== mds.4 v2:172.21.15.47:6840/3401578978 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x5654791c0800 con 0x5654792689e0 2024-03-21T10:28:32.708 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:32.708+0000 7f6e9be00640 10 client.16743 handle_command_reply: tid=0 2024-03-21T10:28:32.708 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:32.709+0000 7f6ec3d64740 1 client.16743 shutdown 2024-03-21T10:28:32.709 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:32.710+0000 7f6ec3d64740 1 -- 172.21.15.47:0/967926510 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56547924f6d0 msgr2=0x565479258500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:28:32.709 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:32.710+0000 7f6ec3d64740 1 --2- 172.21.15.47:0/967926510 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56547924f6d0 0x565479258500 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7f6ea800b4a0 tx=0x7f6ea809b3e0 comp rx=0 tx=0).stop 2024-03-21T10:28:32.710 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:32.710+0000 7f6ec3d64740 1 -- 172.21.15.47:0/967926510 shutdown_connections 2024-03-21T10:28:32.710 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:32.710+0000 7f6ec3d64740 1 --2- 172.21.15.47:0/967926510 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x565479258c00 0x565479255dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:28:32.710 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:32.710+0000 7f6ec3d64740 1 --2- 172.21.15.47:0/967926510 >> [v2:172.21.15.47:6840/3401578978,v1:172.21.15.47:6841/3401578978] conn(0x5654792689e0 0x56547926add0 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:28:32.710 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:32.710+0000 7f6ec3d64740 1 --2- 172.21.15.47:0/967926510 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x565479250080 0x565479255890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:28:32.710 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:32.710+0000 7f6ec3d64740 1 --2- 172.21.15.47:0/967926510 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56547924f6d0 0x565479258500 secure :-1 s=CLOSED pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7f6ea800b4a0 tx=0x7f6ea809b3e0 comp rx=0 tx=0).stop 2024-03-21T10:28:32.710 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:32.710+0000 7f6ec3d64740 1 -- 172.21.15.47:0/967926510 >> 172.21.15.47:0/967926510 conn(0x565479249da0 msgr2=0x56547924baf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:28:32.710 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:32.710+0000 7f6e9be00640 0 client.16743 ms_handle_reset on v2:172.21.15.47:6840/3401578978 2024-03-21T10:28:32.711 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:32.711+0000 7f6ec3d64740 1 -- 172.21.15.47:0/967926510 shutdown_connections 2024-03-21T10:28:32.711 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:32.711+0000 7f6ec3d64740 1 -- 172.21.15.47:0/967926510 wait complete. 2024-03-21T10:28:32.711 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:32.712+0000 7f6ec3d64740 20 client.16743 trim_cache size 0 max 16384 2024-03-21T10:28:32.739 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:32.740+0000 7f6ec3d64740 1 Processor -- start 2024-03-21T10:28:32.740 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:32.740+0000 7f6ec3d64740 1 -- start start 2024-03-21T10:28:32.740 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:32.741+0000 7f6ec3d64740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x565479249da0 0x565479257fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:28:32.741 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:32.741+0000 7f6ec3d64740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x565479250080 0x565479258500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:28:32.741 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:32.741+0000 7f6ec3d64740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x565479258c00 0x565479252af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:28:32.741 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:32.741+0000 7f6ec3d64740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ebc1bd870 con 0x565479250080 2024-03-21T10:28:32.741 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:32.741+0000 7f6ec3d64740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ebc1bf1f0 con 0x565479249da0 2024-03-21T10:28:32.741 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:32.741+0000 7f6ec3d64740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ebc1bded0 con 0x565479258c00 2024-03-21T10:28:32.742 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:32.742+0000 7f6eba800640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x565479258c00 0x565479252af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:28:32.742 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:32.742+0000 7f6eba800640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x565479258c00 0x565479252af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.47:33094/0 (socket says 172.21.15.47:33094) 2024-03-21T10:28:32.742 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:32.742+0000 7f6eba800640 1 -- 172.21.15.47:0/1490479781 learned_addr learned my addr 172.21.15.47:0/1490479781 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:28:32.742 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:32.742+0000 7f6eb9400640 1 --2- 172.21.15.47:0/1490479781 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x565479250080 0x565479258500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:28:32.742 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:32.742+0000 7f6eb9e00640 1 --2- 172.21.15.47:0/1490479781 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x565479249da0 0x565479257fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:28:32.742 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:32.742+0000 7f6eba800640 1 -- 172.21.15.47:0/1490479781 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x565479249da0 msgr2=0x565479257fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:28:32.742 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:32.742+0000 7f6eba800640 1 --2- 172.21.15.47:0/1490479781 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x565479249da0 0x565479257fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:28:32.742 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:32.742+0000 7f6eba800640 1 -- 172.21.15.47:0/1490479781 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x565479250080 msgr2=0x565479258500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:28:32.742 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:32.742+0000 7f6eba800640 1 --2- 172.21.15.47:0/1490479781 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x565479250080 0x565479258500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:28:32.743 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:32.742+0000 7f6eba800640 1 -- 172.21.15.47:0/1490479781 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5654792495c0 con 0x565479258c00 2024-03-21T10:28:32.743 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:32.742+0000 7f6eb9400640 1 --2- 172.21.15.47:0/1490479781 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x565479250080 0x565479258500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T10:28:32.743 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:32.743+0000 7f6eba800640 1 --2- 172.21.15.47:0/1490479781 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x565479258c00 0x565479252af0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f6eb4009ca0 tx=0x7f6eb4004960 comp rx=0 tx=0).ready entity=mon.1 client_cookie=86e340c5c7d81f7e server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:28:32.743 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:32.743+0000 7f6ea6a00640 1 -- 172.21.15.47:0/1490479781 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6eb4057070 con 0x565479258c00 2024-03-21T10:28:32.743 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:32.743+0000 7f6ea6a00640 1 -- 172.21.15.47:0/1490479781 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f6eb40329a0 con 0x565479258c00 2024-03-21T10:28:32.743 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:32.744+0000 7f6ea6a00640 1 -- 172.21.15.47:0/1490479781 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6eb4043040 con 0x565479258c00 2024-03-21T10:28:32.744 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:32.745+0000 7f6ec3d64740 1 -- 172.21.15.47:0/1490479781 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x565479258c00 msgr2=0x565479252af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:28:32.744 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:32.745+0000 7f6ec3d64740 1 --2- 172.21.15.47:0/1490479781 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x565479258c00 0x565479252af0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f6eb4009ca0 tx=0x7f6eb4004960 comp rx=0 tx=0).stop 2024-03-21T10:28:32.745 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:32.745+0000 7f6ec3d64740 1 -- 172.21.15.47:0/1490479781 shutdown_connections 2024-03-21T10:28:32.745 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:32.745+0000 7f6ec3d64740 1 --2- 172.21.15.47:0/1490479781 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x565479258c00 0x565479252af0 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:28:32.745 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:32.745+0000 7f6ec3d64740 1 --2- 172.21.15.47:0/1490479781 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x565479250080 0x565479258500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:28:32.745 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:32.745+0000 7f6ec3d64740 1 --2- 172.21.15.47:0/1490479781 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x565479249da0 0x565479257fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:28:32.745 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:32.745+0000 7f6ec3d64740 1 -- 172.21.15.47:0/1490479781 >> 172.21.15.47:0/1490479781 conn(0x565479255f30 msgr2=0x565479248eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:28:32.747 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:32.746+0000 7f6ec3d64740 1 -- 172.21.15.47:0/1490479781 shutdown_connections 2024-03-21T10:28:32.747 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:32.746+0000 7f6ec3d64740 1 -- 172.21.15.47:0/1490479781 wait complete. 2024-03-21T10:28:32.747 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:32.747+0000 7f6ec3d64740 1 -- start start 2024-03-21T10:28:32.747 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:32.747+0000 7f6ec3d64740 1 Processor -- start 2024-03-21T10:28:32.749 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:32.750+0000 7f6ec3d64740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x565479249da0 0x56547925d270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:28:32.749 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:32.750+0000 7f6ec3d64740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x565479250080 0x56547925d7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:28:32.749 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:32.750+0000 7f6ec3d64740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x565479258c00 0x56547925dcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:28:32.751 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:32.751+0000 7f6eb9400640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x565479250080 0x56547925d7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:28:32.752 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:32.751+0000 7f6eba800640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x565479258c00 0x56547925dcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:28:32.752 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:32.751+0000 7f6eb9e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x565479249da0 0x56547925d270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:28:32.752 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:32.751+0000 7f6eba800640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x565479258c00 0x56547925dcf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:60446/0 (socket says 172.21.15.47:60446) 2024-03-21T10:28:32.752 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:32.752+0000 7f6eb9400640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x565479250080 0x56547925d7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.47:33110/0 (socket says 172.21.15.47:33110) 2024-03-21T10:28:32.752 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:32.752+0000 7f6eb9e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x565479249da0 0x56547925d270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:53874/0 (socket says 172.21.15.47:53874) 2024-03-21T10:28:32.752 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:32.752+0000 7f6eba800640 1 -- 172.21.15.47:0/1795358650 learned_addr learned my addr 172.21.15.47:0/1795358650 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:28:32.753 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:32.753+0000 7f6ec3d64740 1 -- 172.21.15.47:0/1795358650 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ebc1beb90 con 0x565479258c00 2024-03-21T10:28:32.753 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:32.753+0000 7f6ec3d64740 1 -- 172.21.15.47:0/1795358650 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ebc1be530 con 0x565479249da0 2024-03-21T10:28:32.753 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:32.753+0000 7f6ec3d64740 1 -- 172.21.15.47:0/1795358650 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ebc10b6e0 con 0x565479250080 2024-03-21T10:28:32.755 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:32.753+0000 7f6ec3d64740 10 client.0 _ll_register_callbacks cb 0x7f6ea80978b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T10:28:32.755 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:32.754+0000 7f6eb9400640 1 -- 172.21.15.47:0/1795358650 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x565479249da0 msgr2=0x56547925d270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:28:32.755 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:32.754+0000 7f6eb9400640 1 --2- 172.21.15.47:0/1795358650 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x565479249da0 0x56547925d270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:28:32.755 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:32.754+0000 7f6eb9400640 1 -- 172.21.15.47:0/1795358650 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x565479258c00 msgr2=0x56547925dcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:28:32.755 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:32.754+0000 7f6eb9400640 1 --2- 172.21.15.47:0/1795358650 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x565479258c00 0x56547925dcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:28:32.755 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:32.754+0000 7f6eb9400640 1 -- 172.21.15.47:0/1795358650 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5654792c4130 con 0x565479250080 2024-03-21T10:28:32.755 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:32.756+0000 7f6eb9400640 1 --2- 172.21.15.47:0/1795358650 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x565479250080 0x56547925d7b0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x56547926b500 tx=0x7f6eac004410 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d3103527cde63e8d server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:28:32.756 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:32.756+0000 7f6e9be00640 10 client.0 ms_handle_connect on v2:172.21.15.157:3300/0 2024-03-21T10:28:32.756 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:32.756+0000 7f6e9be00640 1 -- 172.21.15.47:0/1795358650 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6eac003aa0 con 0x565479250080 2024-03-21T10:28:32.756 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:32.756+0000 7f6e9be00640 1 -- 172.21.15.47:0/1795358650 <== mon.1 v2:172.21.15.157:3300/0 2 ==== osd_map(97..97 src has 1..97) v4 ==== 9474+0+0 (secure 0 0 0) 0x7f6eac003c40 con 0x565479250080 2024-03-21T10:28:32.756 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:32.757+0000 7f6ec3d64740 1 -- 172.21.15.47:0/1795358650 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f6eb40329a0 con 0x565479250080 2024-03-21T10:28:32.757 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:32.758+0000 7f6e9be00640 1 -- 172.21.15.47:0/1795358650 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6eac027050 con 0x565479250080 2024-03-21T10:28:32.757 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:32.758+0000 7f6ec3d64740 10 client.26317 fetch_fsmap learned FSMap version 32 2024-03-21T10:28:32.757 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:32.758+0000 7f6ec3d64740 1 -- 172.21.15.47:0/1795358650 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x5654792c5410 con 0x565479250080 2024-03-21T10:28:32.759 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:32.759+0000 7f6e9be00640 1 -- 172.21.15.47:0/1795358650 <== mon.1 v2:172.21.15.157:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f6eb40329a0 con 0x565479250080 2024-03-21T10:28:32.759 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:32.759+0000 7f6e9be00640 1 -- 172.21.15.47:0/1795358650 <== mon.1 v2:172.21.15.157:3300/0 5 ==== fsmap(e 32) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f6eac00f040 con 0x565479250080 2024-03-21T10:28:32.759 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:32.760+0000 7f6ec3d64740 10 client.26317 fetch_fsmap finished waiting for FSMap version 32 2024-03-21T10:28:32.759 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:32.760+0000 7f6ec3d64740 10 client.26317 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.a 2024-03-21T10:28:32.759 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:32.760+0000 7f6ec3d64740 20 client.26317 populate_metadata read hostname 'smithi047' 2024-03-21T10:28:32.760 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:32.760+0000 7f6ec3d64740 1 --2- 172.21.15.47:0/1795358650 >> [v2:172.21.15.47:6840/3401578978,v1:172.21.15.47:6841/3401578978] conn(0x5654792c5af0 0x5654792c7f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:28:32.760 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:32.761+0000 7f6ec3d64740 4 client.26317 mds_command: new command op to 14619 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-21T10:28:32.760 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:32.761+0000 7f6ec3d64740 1 -- 172.21.15.47:0/1795358650 --> [v2:172.21.15.47:6840/3401578978,v1:172.21.15.47:6841/3401578978] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x5654791bab50 con 0x5654792c5af0 2024-03-21T10:28:32.761 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:32.762+0000 7f6eb9e00640 1 --2- 172.21.15.47:0/1795358650 >> [v2:172.21.15.47:6840/3401578978,v1:172.21.15.47:6841/3401578978] conn(0x5654792c5af0 0x5654792c7f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:28:32.763 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:32.763+0000 7f6eb9e00640 1 --2- 172.21.15.47:0/1795358650 >> [v2:172.21.15.47:6840/3401578978,v1:172.21.15.47:6841/3401578978] conn(0x5654792c5af0 0x5654792c7f10 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=3fde09b1152838d server_cookie=c350faf5e5d7844c in_seq=0 out_seq=0 2024-03-21T10:28:32.764 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:32.764+0000 7f6e9be00640 10 client.26317 ms_handle_connect on v2:172.21.15.47:6840/3401578978 2024-03-21T10:28:32.766 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:32.766+0000 7f6e9be00640 1 -- 172.21.15.47:0/1795358650 <== mds.4 v2:172.21.15.47:6840/3401578978 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x5654791bab50 con 0x5654792c5af0 2024-03-21T10:28:32.766 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:32.766+0000 7f6e9be00640 10 client.26317 handle_command_reply: tid=0 2024-03-21T10:28:32.766 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:32.766+0000 7f6ec3d64740 1 client.26317 shutdown 2024-03-21T10:28:32.766 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:32.766+0000 7f6ec3d64740 1 -- 172.21.15.47:0/1795358650 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x565479250080 msgr2=0x56547925d7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:28:32.766 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:32.766+0000 7f6ec3d64740 1 --2- 172.21.15.47:0/1795358650 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x565479250080 0x56547925d7b0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x56547926b500 tx=0x7f6eac004410 comp rx=0 tx=0).stop 2024-03-21T10:28:32.767 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:32.767+0000 7f6ec3d64740 1 -- 172.21.15.47:0/1795358650 shutdown_connections 2024-03-21T10:28:32.767 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:32.767+0000 7f6ec3d64740 1 --2- 172.21.15.47:0/1795358650 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x565479258c00 0x56547925dcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:28:32.767 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:32.767+0000 7f6ec3d64740 1 --2- 172.21.15.47:0/1795358650 >> [v2:172.21.15.47:6840/3401578978,v1:172.21.15.47:6841/3401578978] conn(0x5654792c5af0 0x5654792c7f10 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:28:32.767 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:32.767+0000 7f6ec3d64740 1 --2- 172.21.15.47:0/1795358650 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x565479250080 0x56547925d7b0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:28:32.767 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:32.767+0000 7f6ec3d64740 1 --2- 172.21.15.47:0/1795358650 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x565479249da0 0x56547925d270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:28:32.767 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:32.767+0000 7f6ec3d64740 1 -- 172.21.15.47:0/1795358650 >> 172.21.15.47:0/1795358650 conn(0x565479255f30 msgr2=0x56547924b0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:28:32.767 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:32.767+0000 7f6e9be00640 0 client.26317 ms_handle_reset on v2:172.21.15.47:6840/3401578978 2024-03-21T10:28:32.768 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:32.768+0000 7f6ec3d64740 1 -- 172.21.15.47:0/1795358650 shutdown_connections 2024-03-21T10:28:32.768 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:32.768+0000 7f6ec3d64740 1 -- 172.21.15.47:0/1795358650 wait complete. 2024-03-21T10:28:32.768 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:32.768+0000 7f6ec3d64740 20 client.26317 trim_cache size 0 max 16384 2024-03-21T10:28:32.769 INFO:teuthology.orchestra.run.smithi047.stdout:[] 2024-03-21T10:28:32.769 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:32.769+0000 7f6ec1000640 1 -- 172.21.15.47:0/2650488189 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f6ea00a3170 msgr2=0x7f6ea00a5630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:28:32.769 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:32.769+0000 7f6ec1000640 1 --2- 172.21.15.47:0/2650488189 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f6ea00a3170 0x7f6ea00a5630 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f6eb402f020 tx=0x7f6eb4036040 comp rx=0 tx=0).stop 2024-03-21T10:28:32.769 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:32.769+0000 7f6ec1000640 1 -- 172.21.15.47:0/2650488189 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6ebc101630 msgr2=0x7f6ebc10e120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:28:32.769 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:32.769+0000 7f6ec1000640 1 --2- 172.21.15.47:0/2650488189 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6ebc101630 0x7f6ebc10e120 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f6ea800b4d0 tx=0x7f6ea800b990 comp rx=0 tx=0).stop 2024-03-21T10:28:32.770 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:32.770+0000 7f6ec1000640 1 -- 172.21.15.47:0/2650488189 shutdown_connections 2024-03-21T10:28:32.770 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:32.770+0000 7f6ec1000640 1 --2- 172.21.15.47:0/2650488189 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6ebc107640 0x7f6ebc10a6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:28:32.770 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:32.770+0000 7f6ec1000640 1 --2- 172.21.15.47:0/2650488189 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6ebc101ff0 0x7f6ebc10a1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:28:32.770 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:32.770+0000 7f6ec1000640 1 --2- 172.21.15.47:0/2650488189 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f6ea00a3170 0x7f6ea00a5630 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:28:32.770 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:32.770+0000 7f6ec1000640 1 --2- 172.21.15.47:0/2650488189 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6ebc101630 0x7f6ebc10e120 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:28:32.770 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:32.770+0000 7f6ec1000640 1 -- 172.21.15.47:0/2650488189 >> 172.21.15.47:0/2650488189 conn(0x7f6ebc078a60 msgr2=0x7f6ebc103130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:28:32.770 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:32.770+0000 7f6ec1000640 1 -- 172.21.15.47:0/2650488189 shutdown_connections 2024-03-21T10:28:32.771 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:32.770+0000 7f6ec1000640 1 -- 172.21.15.47:0/2650488189 wait complete. 2024-03-21T10:28:32.799 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-21T10:28:32.799 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-03-21T10:28:33.173 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:28:33 smithi047 ceph-mon[31547]: pgmap v100: 129 pgs: 129 active+clean; 432 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 11 MiB/s wr, 2.34k op/s 2024-03-21T10:28:33.183 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:33.183+0000 7f1070200640 1 -- 172.21.15.47:0/848667303 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1068067c00 msgr2=0x7f10680ff880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:28:33.183 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:33.183+0000 7f1070200640 1 --2- 172.21.15.47:0/848667303 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1068067c00 0x7f10680ff880 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f1054009930 tx=0x7f105402bc40 comp rx=0 tx=0).stop 2024-03-21T10:28:33.183 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:33.184+0000 7f1070200640 1 -- 172.21.15.47:0/848667303 shutdown_connections 2024-03-21T10:28:33.183 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:33.184+0000 7f1070200640 1 --2- 172.21.15.47:0/848667303 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f10680ffdc0 0x7f1068100270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:28:33.184 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:33.184+0000 7f1070200640 1 --2- 172.21.15.47:0/848667303 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1068067c00 0x7f10680ff880 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:28:33.184 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:33.184+0000 7f1070200640 1 --2- 172.21.15.47:0/848667303 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1068105890 0x7f1068105c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:28:33.184 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:33.184+0000 7f1070200640 1 -- 172.21.15.47:0/848667303 >> 172.21.15.47:0/848667303 conn(0x7f10680766f0 msgr2=0x7f10680fda80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:28:33.185 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:33.186+0000 7f1070200640 1 -- 172.21.15.47:0/848667303 shutdown_connections 2024-03-21T10:28:33.186 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:33.186+0000 7f1070200640 1 -- 172.21.15.47:0/848667303 wait complete. 2024-03-21T10:28:33.187 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:33.188+0000 7f1070200640 1 Processor -- start 2024-03-21T10:28:33.189 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:33.188+0000 7f1070200640 1 -- start start 2024-03-21T10:28:33.189 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:33.189+0000 7f1070200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1068067c00 0x7f106819dbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:28:33.189 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:33.189+0000 7f1070200640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f10680ffdc0 0x7f1068195d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:28:33.189 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:33.189+0000 7f1070200640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1068105890 0x7f10681962b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:28:33.189 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:33.189+0000 7f106d800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1068067c00 0x7f106819dbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:28:33.189 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:33.189+0000 7f106d800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1068067c00 0x7f106819dbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:53884/0 (socket says 172.21.15.47:53884) 2024-03-21T10:28:33.189 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:33.189+0000 7f106ce00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f10680ffdc0 0x7f1068195d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:28:33.190 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:33.190+0000 7f106ce00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f10680ffdc0 0x7f1068195d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.47:33134/0 (socket says 172.21.15.47:33134) 2024-03-21T10:28:33.190 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:33.190+0000 7f1070200640 1 -- 172.21.15.47:0/780936011 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10681108a0 con 0x7f1068105890 2024-03-21T10:28:33.190 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:33.190+0000 7f1070200640 1 -- 172.21.15.47:0/780936011 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1068110720 con 0x7f1068067c00 2024-03-21T10:28:33.190 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:33.190+0000 7f1070200640 1 -- 172.21.15.47:0/780936011 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1068110a20 con 0x7f10680ffdc0 2024-03-21T10:28:33.190 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:33.191+0000 7f106e200640 1 --2- 172.21.15.47:0/780936011 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1068105890 0x7f10681962b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:28:33.191 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:33.191+0000 7f106ce00640 1 -- 172.21.15.47:0/780936011 learned_addr learned my addr 172.21.15.47:0/780936011 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:28:33.191 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:33.192+0000 7f106ce00640 1 -- 172.21.15.47:0/780936011 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1068067c00 msgr2=0x7f106819dbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:28:33.192 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:33.192+0000 7f106ce00640 1 --2- 172.21.15.47:0/780936011 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1068067c00 0x7f106819dbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:28:33.193 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:33.193+0000 7f106ce00640 1 -- 172.21.15.47:0/780936011 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1068105890 msgr2=0x7f10681962b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:28:33.193 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:33.193+0000 7f106ce00640 1 --2- 172.21.15.47:0/780936011 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1068105890 0x7f10681962b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:28:33.193 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:33.193+0000 7f106ce00640 1 -- 172.21.15.47:0/780936011 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f10681968a0 con 0x7f10680ffdc0 2024-03-21T10:28:33.193 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:33.193+0000 7f106e200640 1 --2- 172.21.15.47:0/780936011 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1068105890 0x7f10681962b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T10:28:33.195 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:33.195+0000 7f106ce00640 1 --2- 172.21.15.47:0/780936011 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f10680ffdc0 0x7f1068195d70 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f1054009900 tx=0x7f105402ca20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:28:33.196 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:33.196+0000 7f1066000640 1 -- 172.21.15.47:0/780936011 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1054043020 con 0x7f10680ffdc0 2024-03-21T10:28:33.196 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:33.196+0000 7f1066000640 1 -- 172.21.15.47:0/780936011 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f10540052b0 con 0x7f10680ffdc0 2024-03-21T10:28:33.196 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:33.196+0000 7f1066000640 1 -- 172.21.15.47:0/780936011 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1054003ac0 con 0x7f10680ffdc0 2024-03-21T10:28:33.196 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:33.196+0000 7f1070200640 1 -- 172.21.15.47:0/780936011 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1068196b30 con 0x7f10680ffdc0 2024-03-21T10:28:33.197 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:33.196+0000 7f1070200640 1 -- 172.21.15.47:0/780936011 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f10681b3850 con 0x7f10680ffdc0 2024-03-21T10:28:33.200 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:33.201+0000 7f1073006740 1 Processor -- start 2024-03-21T10:28:33.200 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:33.201+0000 7f1073006740 1 -- start start 2024-03-21T10:28:33.201 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:33.202+0000 7f1073006740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5559cf3636d0 0x5559cf363ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:28:33.201 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:33.202+0000 7f1073006740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5559cf364080 0x5559cf36c630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:28:33.201 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:33.202+0000 7f1073006740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5559cf36cc00 0x5559cf36f050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:28:33.201 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:33.202+0000 7f1073006740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x5559cf2c72c0 con 0x5559cf36cc00 2024-03-21T10:28:33.201 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:33.202+0000 7f1073006740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x5559cf26fc90 con 0x5559cf364080 2024-03-21T10:28:33.202 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:33.202+0000 7f1073006740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x5559cf2c3360 con 0x5559cf3636d0 2024-03-21T10:28:33.202 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:33.202+0000 7f106e200640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5559cf3636d0 0x5559cf363ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:28:33.202 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:33.202+0000 7f106e200640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5559cf3636d0 0x5559cf363ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.47:33146/0 (socket says 172.21.15.47:33146) 2024-03-21T10:28:33.202 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:33.202+0000 7f106e200640 1 -- 172.21.15.47:0/426682573 learned_addr learned my addr 172.21.15.47:0/426682573 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:28:33.202 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:33.203+0000 7f106e200640 1 -- 172.21.15.47:0/426682573 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5559cf364080 msgr2=0x5559cf36c630 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-21T10:28:33.202 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:33.203+0000 7f106e200640 1 --2- 172.21.15.47:0/426682573 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5559cf364080 0x5559cf36c630 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:28:33.203 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:33.203+0000 7f106e200640 1 -- 172.21.15.47:0/426682573 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5559cf36cc00 msgr2=0x5559cf36f050 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-21T10:28:33.203 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:33.203+0000 7f106e200640 1 --2- 172.21.15.47:0/426682573 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5559cf36cc00 0x5559cf36f050 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:28:33.203 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:33.203+0000 7f106e200640 1 -- 172.21.15.47:0/426682573 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5559cf294080 con 0x5559cf3636d0 2024-03-21T10:28:33.203 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:33.203+0000 7f106e200640 1 --2- 172.21.15.47:0/426682573 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5559cf3636d0 0x5559cf363ab0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f106000a870 tx=0x7f1060009650 comp rx=0 tx=0).ready entity=mon.1 client_cookie=dc02ffcfa1c8d44b server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:28:33.203 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:33.204+0000 7f105aa00640 1 -- 172.21.15.47:0/426682573 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1060018020 con 0x5559cf3636d0 2024-03-21T10:28:33.203 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:33.204+0000 7f105aa00640 1 -- 172.21.15.47:0/426682573 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f1060014400 con 0x5559cf3636d0 2024-03-21T10:28:33.204 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:33.204+0000 7f105aa00640 1 -- 172.21.15.47:0/426682573 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10600135c0 con 0x5559cf3636d0 2024-03-21T10:28:33.204 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:33.205+0000 7f1073006740 1 -- 172.21.15.47:0/426682573 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5559cf3636d0 msgr2=0x5559cf363ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:28:33.205 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:33.205+0000 7f1073006740 1 --2- 172.21.15.47:0/426682573 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5559cf3636d0 0x5559cf363ab0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f106000a870 tx=0x7f1060009650 comp rx=0 tx=0).stop 2024-03-21T10:28:33.205 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:33.205+0000 7f1073006740 1 -- 172.21.15.47:0/426682573 shutdown_connections 2024-03-21T10:28:33.205 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:33.206+0000 7f1073006740 1 --2- 172.21.15.47:0/426682573 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5559cf36cc00 0x5559cf36f050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:28:33.205 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:33.206+0000 7f1073006740 1 --2- 172.21.15.47:0/426682573 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5559cf364080 0x5559cf36c630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:28:33.206 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:33.206+0000 7f1073006740 1 --2- 172.21.15.47:0/426682573 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5559cf3636d0 0x5559cf363ab0 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:28:33.206 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:33.206+0000 7f1073006740 1 -- 172.21.15.47:0/426682573 >> 172.21.15.47:0/426682573 conn(0x5559cf35dda0 msgr2=0x5559cf35e180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:28:33.214 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:33.213+0000 7f1066000640 1 -- 172.21.15.47:0/780936011 <== mon.1 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 27) v1 ==== 137985+0+0 (secure 0 0 0) 0x7f1054046020 con 0x7f10680ffdc0 2024-03-21T10:28:33.214 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:33.215+0000 7f1066000640 1 --2- 172.21.15.47:0/780936011 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f10500a3150 0x7f10500a5610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:28:33.215 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:33.215+0000 7f1066000640 1 -- 172.21.15.47:0/780936011 <== mon.1 v2:172.21.15.157:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 9474+0+0 (secure 0 0 0) 0x7f10540f1bb0 con 0x7f10680ffdc0 2024-03-21T10:28:33.217 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:33.217+0000 7f1073006740 1 -- 172.21.15.47:0/426682573 shutdown_connections 2024-03-21T10:28:33.221 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:33.219+0000 7f1073006740 1 -- 172.21.15.47:0/426682573 wait complete. 2024-03-21T10:28:33.221 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:33.219+0000 7f1073006740 1 -- start start 2024-03-21T10:28:33.221 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:33.220+0000 7f1073006740 1 Processor -- start 2024-03-21T10:28:33.222 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:33.223+0000 7f1073006740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5559cf3636d0 0x5559cf369a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:28:33.222 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:33.223+0000 7f1073006740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5559cf36c720 0x5559cf369f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:28:33.223 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:33.223+0000 7f1073006740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5559cf36a5f0 0x5559cf379050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:28:33.224 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:33.224+0000 7f1073006740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x5559cf36fb70 con 0x5559cf3636d0 2024-03-21T10:28:33.224 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:33.224+0000 7f1073006740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x5559cf2830d0 con 0x5559cf36c720 2024-03-21T10:28:33.224 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:33.224+0000 7f1073006740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x5559cf36fcf0 con 0x5559cf36a5f0 2024-03-21T10:28:33.224 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:33.224+0000 7f1073006740 10 client.0 _ll_register_callbacks cb 0x5559cf287e80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T10:28:33.224 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:33.224+0000 7f106e200640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5559cf36a5f0 0x5559cf379050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:28:33.224 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:33.224+0000 7f106e200640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5559cf36a5f0 0x5559cf379050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.47:33152/0 (socket says 172.21.15.47:33152) 2024-03-21T10:28:33.224 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:33.224+0000 7f106e200640 1 -- 172.21.15.47:0/1530277430 learned_addr learned my addr 172.21.15.47:0/1530277430 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:28:33.224 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:33.225+0000 7f106ce00640 1 --2- 172.21.15.47:0/1530277430 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5559cf36c720 0x5559cf369f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:28:33.225 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:33.226+0000 7f106e200640 1 -- 172.21.15.47:0/1530277430 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5559cf36c720 msgr2=0x5559cf369f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:28:33.225 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:33.226+0000 7f106e200640 1 --2- 172.21.15.47:0/1530277430 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5559cf36c720 0x5559cf369f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:28:33.225 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:33.226+0000 7f106e200640 1 -- 172.21.15.47:0/1530277430 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5559cf3636d0 msgr2=0x5559cf369a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:28:33.226 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:33.227+0000 7f106d800640 1 --2- 172.21.15.47:0/1530277430 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5559cf3636d0 0x5559cf369a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:28:33.226 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:33.227+0000 7f106d800640 1 --2- 172.21.15.47:0/780936011 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f10500a3150 0x7f10500a5610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:28:33.226 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:33.227+0000 7f106e200640 1 --2- 172.21.15.47:0/1530277430 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5559cf3636d0 0x5559cf369a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:28:33.226 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:33.227+0000 7f106e200640 1 -- 172.21.15.47:0/1530277430 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5559cefdb3f0 con 0x5559cf36a5f0 2024-03-21T10:28:33.227 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:33.228+0000 7f106e200640 1 --2- 172.21.15.47:0/1530277430 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5559cf36a5f0 0x5559cf379050 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f10600071a0 tx=0x7f106000eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ab5db08a7a6ea481 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:28:33.228 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:33.228+0000 7f104f400640 10 client.0 ms_handle_connect on v2:172.21.15.157:3300/0 2024-03-21T10:28:33.228 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:33.229+0000 7f104f400640 1 -- 172.21.15.47:0/1530277430 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1060018040 con 0x5559cf36a5f0 2024-03-21T10:28:33.228 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:33.229+0000 7f104f400640 1 -- 172.21.15.47:0/1530277430 <== mon.1 v2:172.21.15.157:3300/0 2 ==== osd_map(97..97 src has 1..97) v4 ==== 9474+0+0 (secure 0 0 0) 0x7f1060014ac0 con 0x5559cf36a5f0 2024-03-21T10:28:33.228 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:33.229+0000 7f1073006740 1 -- 172.21.15.47:0/1530277430 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f1060014400 con 0x5559cf36a5f0 2024-03-21T10:28:33.229 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:33.230+0000 7f104f400640 1 -- 172.21.15.47:0/1530277430 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1060028480 con 0x5559cf36a5f0 2024-03-21T10:28:33.229 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:33.230+0000 7f1073006740 10 client.26329 fetch_fsmap learned FSMap version 32 2024-03-21T10:28:33.229 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:33.230+0000 7f1073006740 1 -- 172.21.15.47:0/1530277430 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x5559cf0a57d0 con 0x5559cf36a5f0 2024-03-21T10:28:33.231 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:33.230+0000 7f104f400640 1 -- 172.21.15.47:0/1530277430 <== mon.1 v2:172.21.15.157:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f1060014400 con 0x5559cf36a5f0 2024-03-21T10:28:33.231 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:33.231+0000 7f104f400640 1 -- 172.21.15.47:0/1530277430 <== mon.1 v2:172.21.15.157:3300/0 5 ==== fsmap(e 32) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f10600326e0 con 0x5559cf36a5f0 2024-03-21T10:28:33.231 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:33.231+0000 7f1073006740 10 client.26329 fetch_fsmap finished waiting for FSMap version 32 2024-03-21T10:28:33.231 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:33.231+0000 7f1073006740 10 client.26329 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.k 2024-03-21T10:28:33.231 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:33.231+0000 7f1073006740 20 client.26329 populate_metadata read hostname 'smithi047' 2024-03-21T10:28:33.232 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:33.231+0000 7f1073006740 1 --2- 172.21.15.47:0/1530277430 >> [v2:172.21.15.100:6832/2723302038,v1:172.21.15.100:6833/2723302038] conn(0x5559cf37d420 0x5559cf37f810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:28:33.232 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:33.231+0000 7f1073006740 4 client.26329 mds_command: new command op to 34427 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T10:28:33.232 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:33.231+0000 7f1073006740 1 -- 172.21.15.47:0/1530277430 --> [v2:172.21.15.100:6832/2723302038,v1:172.21.15.100:6833/2723302038] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5559cf2d4800 con 0x5559cf37d420 2024-03-21T10:28:33.232 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:33.232+0000 7f106d800640 1 --2- 172.21.15.47:0/780936011 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f10500a3150 0x7f10500a5610 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f105c005d80 tx=0x7f105c00b390 comp rx=0 tx=0).ready entity=mgr.15687 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:28:33.234 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:33.234+0000 7f106d800640 1 --2- 172.21.15.47:0/1530277430 >> [v2:172.21.15.100:6832/2723302038,v1:172.21.15.100:6833/2723302038] conn(0x5559cf37d420 0x5559cf37f810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:28:33.237 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:33.237+0000 7f106d800640 1 --2- 172.21.15.47:0/1530277430 >> [v2:172.21.15.100:6832/2723302038,v1:172.21.15.100:6833/2723302038] conn(0x5559cf37d420 0x5559cf37f810 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=93ba617e13c60b7f server_cookie=abc336e48b8788ed in_seq=0 out_seq=0 2024-03-21T10:28:33.237 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:33.237+0000 7f104f400640 10 client.26329 ms_handle_connect on v2:172.21.15.100:6832/2723302038 2024-03-21T10:28:33.240 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:33.241+0000 7f104f400640 1 -- 172.21.15.47:0/1530277430 <== mds.3 v2:172.21.15.100:6832/2723302038 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x5559cf2d4800 con 0x5559cf37d420 2024-03-21T10:28:33.240 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:33.241+0000 7f104f400640 10 client.26329 handle_command_reply: tid=0 2024-03-21T10:28:33.240 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:33.241+0000 7f1073006740 1 client.26329 shutdown 2024-03-21T10:28:33.241 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:33.241+0000 7f1073006740 1 -- 172.21.15.47:0/1530277430 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5559cf36a5f0 msgr2=0x5559cf379050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:28:33.241 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:33.241+0000 7f1073006740 1 --2- 172.21.15.47:0/1530277430 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5559cf36a5f0 0x5559cf379050 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f10600071a0 tx=0x7f106000eeb0 comp rx=0 tx=0).stop 2024-03-21T10:28:33.241 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:33.241+0000 7f1073006740 1 -- 172.21.15.47:0/1530277430 shutdown_connections 2024-03-21T10:28:33.241 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:33.241+0000 7f1073006740 1 --2- 172.21.15.47:0/1530277430 >> [v2:172.21.15.100:6832/2723302038,v1:172.21.15.100:6833/2723302038] conn(0x5559cf37d420 0x5559cf37f810 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:28:33.241 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:33.241+0000 7f1073006740 1 --2- 172.21.15.47:0/1530277430 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5559cf36a5f0 0x5559cf379050 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:28:33.241 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:33.241+0000 7f1073006740 1 --2- 172.21.15.47:0/1530277430 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5559cf36c720 0x5559cf369f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:28:33.241 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:33.241+0000 7f104f400640 0 client.26329 ms_handle_reset on v2:172.21.15.100:6832/2723302038 2024-03-21T10:28:33.241 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:33.241+0000 7f1073006740 1 --2- 172.21.15.47:0/1530277430 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5559cf3636d0 0x5559cf369a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:28:33.242 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:33.241+0000 7f1073006740 1 -- 172.21.15.47:0/1530277430 >> 172.21.15.47:0/1530277430 conn(0x5559cf35dda0 msgr2=0x5559cf35faf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:28:33.242 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:33.242+0000 7f1073006740 1 -- 172.21.15.47:0/1530277430 shutdown_connections 2024-03-21T10:28:33.242 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:33.243+0000 7f1073006740 1 -- 172.21.15.47:0/1530277430 wait complete. 2024-03-21T10:28:33.242 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:33.243+0000 7f1073006740 20 client.26329 trim_cache size 0 max 16384 2024-03-21T10:28:33.282 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:33.283+0000 7f1073006740 1 Processor -- start 2024-03-21T10:28:33.283 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:33.283+0000 7f1073006740 1 -- start start 2024-03-21T10:28:33.283 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:33.284+0000 7f1073006740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5559cf3636d0 0x5559cf378fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:28:33.283 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:33.284+0000 7f1073006740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5559cf36a5f0 0x5559cf366cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:28:33.283 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:33.284+0000 7f1073006740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5559cf36c720 0x5559cf367210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:28:33.283 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:33.284+0000 7f1073006740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10681bd8e0 con 0x5559cf36c720 2024-03-21T10:28:33.283 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:33.284+0000 7f1073006740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10681bf260 con 0x5559cf3636d0 2024-03-21T10:28:33.283 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:33.284+0000 7f1073006740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10681bdf40 con 0x5559cf36a5f0 2024-03-21T10:28:33.284 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:33.284+0000 7f106ce00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5559cf36a5f0 0x5559cf366cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:28:33.284 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:33.284+0000 7f106ce00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5559cf36a5f0 0x5559cf366cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.47:33154/0 (socket says 172.21.15.47:33154) 2024-03-21T10:28:33.284 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:33.284+0000 7f106e200640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5559cf36c720 0x5559cf367210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:28:33.284 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:33.284+0000 7f106ce00640 1 -- 172.21.15.47:0/1546334731 learned_addr learned my addr 172.21.15.47:0/1546334731 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:28:33.285 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:33.285+0000 7f106ce00640 1 -- 172.21.15.47:0/1546334731 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5559cf3636d0 msgr2=0x5559cf378fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:28:33.285 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:33.285+0000 7f106ce00640 1 --2- 172.21.15.47:0/1546334731 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5559cf3636d0 0x5559cf378fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:28:33.285 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:33.285+0000 7f106ce00640 1 -- 172.21.15.47:0/1546334731 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5559cf36c720 msgr2=0x5559cf367210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:28:33.285 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:33.285+0000 7f106ce00640 1 --2- 172.21.15.47:0/1546334731 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5559cf36c720 0x5559cf367210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:28:33.285 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:33.286+0000 7f106ce00640 1 -- 172.21.15.47:0/1546334731 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5559cf379630 con 0x5559cf36a5f0 2024-03-21T10:28:33.286 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:33.286+0000 7f106ce00640 1 --2- 172.21.15.47:0/1546334731 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5559cf36a5f0 0x5559cf366cd0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f1054009850 tx=0x7f105402c5e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=744c756d3e32b9b1 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:28:33.288 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:33.288+0000 7f105a000640 1 -- 172.21.15.47:0/1546334731 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10540bd5f0 con 0x5559cf36a5f0 2024-03-21T10:28:33.288 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:33.288+0000 7f105a000640 1 -- 172.21.15.47:0/1546334731 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f10540be390 con 0x5559cf36a5f0 2024-03-21T10:28:33.288 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:33.288+0000 7f105a000640 1 -- 172.21.15.47:0/1546334731 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10540ef160 con 0x5559cf36a5f0 2024-03-21T10:28:33.289 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:33.289+0000 7f1073006740 1 -- 172.21.15.47:0/1546334731 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5559cf36a5f0 msgr2=0x5559cf366cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:28:33.289 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:33.289+0000 7f1073006740 1 --2- 172.21.15.47:0/1546334731 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5559cf36a5f0 0x5559cf366cd0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f1054009850 tx=0x7f105402c5e0 comp rx=0 tx=0).stop 2024-03-21T10:28:33.289 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:33.290+0000 7f1073006740 1 -- 172.21.15.47:0/1546334731 shutdown_connections 2024-03-21T10:28:33.290 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:33.290+0000 7f1073006740 1 --2- 172.21.15.47:0/1546334731 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5559cf36c720 0x5559cf367210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:28:33.290 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:33.290+0000 7f1073006740 1 --2- 172.21.15.47:0/1546334731 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5559cf36a5f0 0x5559cf366cd0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:28:33.290 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:33.290+0000 7f1073006740 1 --2- 172.21.15.47:0/1546334731 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5559cf3636d0 0x5559cf378fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:28:33.290 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:33.290+0000 7f1073006740 1 -- 172.21.15.47:0/1546334731 >> 172.21.15.47:0/1546334731 conn(0x5559cf35dda0 msgr2=0x5559cf35fa80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:28:33.290 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:33.290+0000 7f1073006740 1 -- 172.21.15.47:0/1546334731 shutdown_connections 2024-03-21T10:28:33.290 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:33.290+0000 7f1073006740 1 -- 172.21.15.47:0/1546334731 wait complete. 2024-03-21T10:28:33.290 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:33.290+0000 7f1073006740 1 -- start start 2024-03-21T10:28:33.291 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:33.291+0000 7f1073006740 1 Processor -- start 2024-03-21T10:28:33.291 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:33.291+0000 7f1073006740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5559cf3636d0 0x5559cf382330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:28:33.291 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:33.291+0000 7f1073006740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5559cf36a5f0 0x5559cf380720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:28:33.291 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:33.291+0000 7f1073006740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5559cf36c720 0x5559cf380c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:28:33.291 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:33.291+0000 7f1073006740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10681bec00 con 0x5559cf36a5f0 2024-03-21T10:28:33.291 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:33.291+0000 7f1073006740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10681be5a0 con 0x5559cf3636d0 2024-03-21T10:28:33.291 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:33.291+0000 7f1073006740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10681bbaa0 con 0x5559cf36c720 2024-03-21T10:28:33.291 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:33.292+0000 7f1073006740 10 client.0 _ll_register_callbacks cb 0x7f106000a6c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T10:28:33.292 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:33.292+0000 7f106e200640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5559cf36c720 0x5559cf380c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:28:33.292 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:33.292+0000 7f106ce00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5559cf36a5f0 0x5559cf380720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:28:33.292 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:33.292+0000 7f106e200640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5559cf36c720 0x5559cf380c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.47:33158/0 (socket says 172.21.15.47:33158) 2024-03-21T10:28:33.292 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:33.292+0000 7f106e200640 1 -- 172.21.15.47:0/3196700101 learned_addr learned my addr 172.21.15.47:0/3196700101 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:28:33.292 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:33.293+0000 7f106d800640 1 --2- 172.21.15.47:0/3196700101 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5559cf3636d0 0x5559cf382330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:28:33.293 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:33.293+0000 7f106ce00640 1 -- 172.21.15.47:0/3196700101 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5559cf36c720 msgr2=0x5559cf380c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:28:33.293 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:33.293+0000 7f106ce00640 1 --2- 172.21.15.47:0/3196700101 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5559cf36c720 0x5559cf380c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:28:33.293 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:33.293+0000 7f106ce00640 1 -- 172.21.15.47:0/3196700101 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5559cf3636d0 msgr2=0x5559cf382330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:28:33.293 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:33.293+0000 7f106ce00640 1 --2- 172.21.15.47:0/3196700101 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5559cf3636d0 0x5559cf382330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:28:33.293 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:33.293+0000 7f106ce00640 1 -- 172.21.15.47:0/3196700101 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5559cf3631d0 con 0x5559cf36a5f0 2024-03-21T10:28:33.293 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:33.293+0000 7f106e200640 1 --2- 172.21.15.47:0/3196700101 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5559cf36c720 0x5559cf380c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T10:28:33.293 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:33.293+0000 7f106d800640 1 --2- 172.21.15.47:0/3196700101 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5559cf3636d0 0x5559cf382330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T10:28:33.294 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:33.293+0000 7f106ce00640 1 --2- 172.21.15.47:0/3196700101 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5559cf36a5f0 0x5559cf380720 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x5559cf35f1a0 tx=0x7f10540bd5f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d1d41387ad4f6363 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:28:33.294 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:33.294+0000 7f104f400640 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-03-21T10:28:33.294 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:33.294+0000 7f104f400640 1 -- 172.21.15.47:0/3196700101 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10540da020 con 0x5559cf36a5f0 2024-03-21T10:28:33.294 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:33.294+0000 7f104f400640 1 -- 172.21.15.47:0/3196700101 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(97..97 src has 1..97) v4 ==== 9474+0+0 (secure 0 0 0) 0x7f10540ba300 con 0x5559cf36a5f0 2024-03-21T10:28:33.294 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:33.294+0000 7f1073006740 1 -- 172.21.15.47:0/3196700101 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f10540be390 con 0x5559cf36a5f0 2024-03-21T10:28:33.295 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:33.294+0000 7f104f400640 1 -- 172.21.15.47:0/3196700101 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f10540c66a0 con 0x5559cf36a5f0 2024-03-21T10:28:33.295 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:33.295+0000 7f1073006740 10 client.16761 fetch_fsmap learned FSMap version 32 2024-03-21T10:28:33.295 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:33.295+0000 7f1073006740 1 -- 172.21.15.47:0/3196700101 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x5559cf3d9560 con 0x5559cf36a5f0 2024-03-21T10:28:33.295 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:33.296+0000 7f104f400640 1 -- 172.21.15.47:0/3196700101 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f10540be390 con 0x5559cf36a5f0 2024-03-21T10:28:33.296 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:33.296+0000 7f104f400640 1 -- 172.21.15.47:0/3196700101 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 32) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f10540c6210 con 0x5559cf36a5f0 2024-03-21T10:28:33.296 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:33.296+0000 7f1073006740 10 client.16761 fetch_fsmap finished waiting for FSMap version 32 2024-03-21T10:28:33.296 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:33.296+0000 7f1073006740 10 client.16761 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.k 2024-03-21T10:28:33.296 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:33.296+0000 7f1073006740 20 client.16761 populate_metadata read hostname 'smithi047' 2024-03-21T10:28:33.296 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:33.296+0000 7f1073006740 1 --2- 172.21.15.47:0/3196700101 >> [v2:172.21.15.100:6832/2723302038,v1:172.21.15.100:6833/2723302038] conn(0x5559cf3d9930 0x5559cf3dbd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:28:33.296 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:33.296+0000 7f1073006740 4 client.16761 mds_command: new command op to 34427 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-21T10:28:33.296 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:33.296+0000 7f1073006740 1 -- 172.21.15.47:0/3196700101 --> [v2:172.21.15.100:6832/2723302038,v1:172.21.15.100:6833/2723302038] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x5559cf36a2e0 con 0x5559cf3d9930 2024-03-21T10:28:33.296 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:33.297+0000 7f106d800640 1 --2- 172.21.15.47:0/3196700101 >> [v2:172.21.15.100:6832/2723302038,v1:172.21.15.100:6833/2723302038] conn(0x5559cf3d9930 0x5559cf3dbd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:28:33.302 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:33.300+0000 7f106d800640 1 --2- 172.21.15.47:0/3196700101 >> [v2:172.21.15.100:6832/2723302038,v1:172.21.15.100:6833/2723302038] conn(0x5559cf3d9930 0x5559cf3dbd50 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=1ed6fb336cb058ac server_cookie=f35a2346781f11a2 in_seq=0 out_seq=0 2024-03-21T10:28:33.303 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:33.300+0000 7f104f400640 10 client.16761 ms_handle_connect on v2:172.21.15.100:6832/2723302038 2024-03-21T10:28:33.303 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:33.301+0000 7f104f400640 1 -- 172.21.15.47:0/3196700101 <== mds.3 v2:172.21.15.100:6832/2723302038 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x5559cf36a2e0 con 0x5559cf3d9930 2024-03-21T10:28:33.303 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:33.301+0000 7f104f400640 10 client.16761 handle_command_reply: tid=0 2024-03-21T10:28:33.303 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:33.301+0000 7f1073006740 1 client.16761 shutdown 2024-03-21T10:28:33.303 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:33.301+0000 7f1073006740 1 -- 172.21.15.47:0/3196700101 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5559cf36a5f0 msgr2=0x5559cf380720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:28:33.303 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:33.301+0000 7f1073006740 1 --2- 172.21.15.47:0/3196700101 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5559cf36a5f0 0x5559cf380720 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x5559cf35f1a0 tx=0x7f10540bd5f0 comp rx=0 tx=0).stop 2024-03-21T10:28:33.303 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:33.302+0000 7f1073006740 1 -- 172.21.15.47:0/3196700101 shutdown_connections 2024-03-21T10:28:33.303 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:33.302+0000 7f1073006740 1 --2- 172.21.15.47:0/3196700101 >> [v2:172.21.15.100:6832/2723302038,v1:172.21.15.100:6833/2723302038] conn(0x5559cf3d9930 0x5559cf3dbd50 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:28:33.303 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:33.302+0000 7f1073006740 1 --2- 172.21.15.47:0/3196700101 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5559cf36c720 0x5559cf380c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:28:33.303 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:33.302+0000 7f1073006740 1 --2- 172.21.15.47:0/3196700101 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5559cf36a5f0 0x5559cf380720 unknown :-1 s=CLOSED pgs=744 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:28:33.303 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:33.302+0000 7f1073006740 1 --2- 172.21.15.47:0/3196700101 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5559cf3636d0 0x5559cf382330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:28:33.303 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:33.302+0000 7f1073006740 1 -- 172.21.15.47:0/3196700101 >> 172.21.15.47:0/3196700101 conn(0x5559cf35dda0 msgr2=0x5559cf3645d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:28:33.304 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:33.302+0000 7f104f400640 0 client.16761 ms_handle_reset on v2:172.21.15.100:6832/2723302038 2024-03-21T10:28:33.304 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:33.303+0000 7f1073006740 1 -- 172.21.15.47:0/3196700101 shutdown_connections 2024-03-21T10:28:33.304 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:33.303+0000 7f1073006740 1 -- 172.21.15.47:0/3196700101 wait complete. 2024-03-21T10:28:33.304 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:33.303+0000 7f1073006740 20 client.16761 trim_cache size 0 max 16384 2024-03-21T10:28:33.304 INFO:teuthology.orchestra.run.smithi047.stdout:[] 2024-03-21T10:28:33.305 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:33.304+0000 7f1070200640 1 -- 172.21.15.47:0/780936011 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f10500a3150 msgr2=0x7f10500a5610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:28:33.305 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:33.304+0000 7f1070200640 1 --2- 172.21.15.47:0/780936011 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f10500a3150 0x7f10500a5610 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f105c005d80 tx=0x7f105c00b390 comp rx=0 tx=0).stop 2024-03-21T10:28:33.305 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:33.305+0000 7f1070200640 1 -- 172.21.15.47:0/780936011 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f10680ffdc0 msgr2=0x7f1068195d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:28:33.305 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:33.305+0000 7f1070200640 1 --2- 172.21.15.47:0/780936011 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f10680ffdc0 0x7f1068195d70 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f1054009900 tx=0x7f105402ca20 comp rx=0 tx=0).stop 2024-03-21T10:28:33.305 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:33.305+0000 7f1070200640 1 -- 172.21.15.47:0/780936011 shutdown_connections 2024-03-21T10:28:33.305 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:33.305+0000 7f1070200640 1 --2- 172.21.15.47:0/780936011 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1068105890 0x7f10681962b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:28:33.305 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:33.305+0000 7f1070200640 1 --2- 172.21.15.47:0/780936011 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f10680ffdc0 0x7f1068195d70 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:28:33.305 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:33.305+0000 7f1070200640 1 --2- 172.21.15.47:0/780936011 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f10500a3150 0x7f10500a5610 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:28:33.305 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:33.305+0000 7f1070200640 1 --2- 172.21.15.47:0/780936011 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1068067c00 0x7f106819dbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:28:33.305 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:33.305+0000 7f1070200640 1 -- 172.21.15.47:0/780936011 >> 172.21.15.47:0/780936011 conn(0x7f10680766f0 msgr2=0x7f1068102420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:28:33.306 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:33.306+0000 7f1070200640 1 -- 172.21.15.47:0/780936011 shutdown_connections 2024-03-21T10:28:33.306 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:33.306+0000 7f1070200640 1 -- 172.21.15.47:0/780936011 wait complete. 2024-03-21T10:28:33.336 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-21T10:28:33.336 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-03-21T10:28:33.465 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:28:33 smithi047 ceph-mon[31547]: from='client.? 172.21.15.47:0/2847510263' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-21T10:28:33.478 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:28:33 smithi100 ceph-mon[35193]: pgmap v100: 129 pgs: 129 active+clean; 432 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 11 MiB/s wr, 2.34k op/s 2024-03-21T10:28:33.478 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:28:33 smithi100 ceph-mon[35193]: from='client.? 172.21.15.47:0/2847510263' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-21T10:28:33.488 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:28:33 smithi157 ceph-mon[35581]: pgmap v100: 129 pgs: 129 active+clean; 432 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 11 MiB/s wr, 2.34k op/s 2024-03-21T10:28:33.488 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:28:33 smithi157 ceph-mon[35581]: from='client.? 172.21.15.47:0/2847510263' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-21T10:28:33.722 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:33.722+0000 7faf13800640 1 -- 172.21.15.47:0/1758982362 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7faf0c0743f0 msgr2=0x7faf0c10d9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:28:33.723 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:33.722+0000 7faf13800640 1 --2- 172.21.15.47:0/1758982362 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7faf0c0743f0 0x7faf0c10d9a0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7faf00009900 tx=0x7faf0002bcc0 comp rx=0 tx=0).stop 2024-03-21T10:28:33.723 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:33.722+0000 7faf13800640 1 -- 172.21.15.47:0/1758982362 shutdown_connections 2024-03-21T10:28:33.723 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:33.722+0000 7faf13800640 1 --2- 172.21.15.47:0/1758982362 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7faf0c10dee0 0x7faf0c110330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:28:33.723 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:33.723+0000 7faf13800640 1 --2- 172.21.15.47:0/1758982362 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7faf0c0743f0 0x7faf0c10d9a0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:28:33.723 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:33.723+0000 7faf13800640 1 --2- 172.21.15.47:0/1758982362 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7faf0c073ad0 0x7faf0c073eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:28:33.723 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:33.723+0000 7faf13800640 1 -- 172.21.15.47:0/1758982362 >> 172.21.15.47:0/1758982362 conn(0x7faf0c0fd1d0 msgr2=0x7faf0c0ff5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:28:33.723 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:33.723+0000 7faf13800640 1 -- 172.21.15.47:0/1758982362 shutdown_connections 2024-03-21T10:28:33.723 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:33.723+0000 7faf13800640 1 -- 172.21.15.47:0/1758982362 wait complete. 2024-03-21T10:28:33.724 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:33.724+0000 7faf13800640 1 Processor -- start 2024-03-21T10:28:33.724 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:33.724+0000 7faf13800640 1 -- start start 2024-03-21T10:28:33.724 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:33.725+0000 7faf13800640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7faf0c073ad0 0x7faf0c19a090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:28:33.724 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:33.725+0000 7faf13800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7faf0c0743f0 0x7faf0c19a5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:28:33.724 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:33.725+0000 7faf13800640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7faf0c10dee0 0x7faf0c19e950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:28:33.725 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:33.725+0000 7faf13800640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf0c113af0 con 0x7faf0c10dee0 2024-03-21T10:28:33.725 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:33.725+0000 7faf13800640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf0c113970 con 0x7faf0c0743f0 2024-03-21T10:28:33.725 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:33.725+0000 7faf13800640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf0c113c70 con 0x7faf0c073ad0 2024-03-21T10:28:33.726 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:33.725+0000 7faf0be00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7faf0c0743f0 0x7faf0c19a5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:28:33.726 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:33.725+0000 7faf0be00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7faf0c0743f0 0x7faf0c19a5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:53930/0 (socket says 172.21.15.47:53930) 2024-03-21T10:28:33.726 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:33.725+0000 7faf0be00640 1 -- 172.21.15.47:0/2278846240 learned_addr learned my addr 172.21.15.47:0/2278846240 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:28:33.726 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:33.725+0000 7faf11800640 1 --2- 172.21.15.47:0/2278846240 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7faf0c10dee0 0x7faf0c19e950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:28:33.726 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:33.725+0000 7faf10e00640 1 --2- 172.21.15.47:0/2278846240 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7faf0c073ad0 0x7faf0c19a090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:28:33.726 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:33.726+0000 7faf0be00640 1 -- 172.21.15.47:0/2278846240 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7faf0c073ad0 msgr2=0x7faf0c19a090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:28:33.726 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:33.726+0000 7faf0be00640 1 --2- 172.21.15.47:0/2278846240 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7faf0c073ad0 0x7faf0c19a090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:28:33.726 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:33.726+0000 7faf0be00640 1 -- 172.21.15.47:0/2278846240 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7faf0c10dee0 msgr2=0x7faf0c19e950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:28:33.726 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:33.726+0000 7faf0be00640 1 --2- 172.21.15.47:0/2278846240 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7faf0c10dee0 0x7faf0c19e950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:28:33.727 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:33.726+0000 7faf0be00640 1 -- 172.21.15.47:0/2278846240 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf0c19f080 con 0x7faf0c0743f0 2024-03-21T10:28:33.727 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:33.726+0000 7faf0be00640 1 --2- 172.21.15.47:0/2278846240 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7faf0c0743f0 0x7faf0c19a5d0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7faf00005940 tx=0x7faf000045e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:28:33.727 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:33.726+0000 7faf09600640 1 -- 172.21.15.47:0/2278846240 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf00043020 con 0x7faf0c0743f0 2024-03-21T10:28:33.727 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:33.726+0000 7faf13800640 1 -- 172.21.15.47:0/2278846240 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faf0c19f310 con 0x7faf0c0743f0 2024-03-21T10:28:33.727 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:33.726+0000 7faf09600640 1 -- 172.21.15.47:0/2278846240 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7faf0002db90 con 0x7faf0c0743f0 2024-03-21T10:28:33.727 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:33.726+0000 7faf09600640 1 -- 172.21.15.47:0/2278846240 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf00003850 con 0x7faf0c0743f0 2024-03-21T10:28:33.727 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:33.727+0000 7faf13800640 1 -- 172.21.15.47:0/2278846240 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faf0c1a6c00 con 0x7faf0c0743f0 2024-03-21T10:28:33.729 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:33.730+0000 7faf09600640 1 -- 172.21.15.47:0/2278846240 <== mon.2 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 27) v1 ==== 137985+0+0 (secure 0 0 0) 0x7faf00046020 con 0x7faf0c0743f0 2024-03-21T10:28:33.730 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:33.731+0000 7faf16122740 1 Processor -- start 2024-03-21T10:28:33.731 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:33.731+0000 7faf16122740 1 -- start start 2024-03-21T10:28:33.731 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:33.732+0000 7faf16122740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5607027666d0 0x560702766ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:28:33.731 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:33.732+0000 7faf16122740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560702767080 0x56070276f630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:28:33.732 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:33.732+0000 7faf16122740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56070276fc00 0x560702772050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:28:33.732 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:33.732+0000 7faf16122740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x5607026ca2c0 con 0x5607027666d0 2024-03-21T10:28:33.732 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:33.732+0000 7faf16122740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x560702672c90 con 0x56070276fc00 2024-03-21T10:28:33.732 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:33.732+0000 7faf16122740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x5607026c6360 con 0x560702767080 2024-03-21T10:28:33.732 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:33.732+0000 7faf09600640 1 --2- 172.21.15.47:0/2278846240 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7faef40a3090 0x7faef40a5550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:28:33.733 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:33.732+0000 7faf0be00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56070276fc00 0x560702772050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:28:33.733 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:33.732+0000 7faf0be00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56070276fc00 0x560702772050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:53944/0 (socket says 172.21.15.47:53944) 2024-03-21T10:28:33.733 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:33.732+0000 7faf0be00640 1 -- 172.21.15.47:0/3297291023 learned_addr learned my addr 172.21.15.47:0/3297291023 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:28:33.734 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:33.732+0000 7faf11800640 1 --2- 172.21.15.47:0/3297291023 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5607027666d0 0x560702766ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:28:33.734 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:33.733+0000 7faf09600640 1 -- 172.21.15.47:0/2278846240 <== mon.2 v2:172.21.15.100:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 9474+0+0 (secure 0 0 0) 0x7faf000f1b00 con 0x7faf0c0743f0 2024-03-21T10:28:33.734 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:33.733+0000 7faf10e00640 1 --2- 172.21.15.47:0/3297291023 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560702767080 0x56070276f630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:28:33.734 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:33.733+0000 7faf0be00640 1 -- 172.21.15.47:0/3297291023 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560702767080 msgr2=0x56070276f630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:28:33.734 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:33.733+0000 7faf11800640 1 --2- 172.21.15.47:0/2278846240 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7faef40a3090 0x7faef40a5550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:28:33.734 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:33.733+0000 7faf0be00640 1 --2- 172.21.15.47:0/3297291023 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560702767080 0x56070276f630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:28:33.734 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:33.733+0000 7faf0be00640 1 -- 172.21.15.47:0/3297291023 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5607027666d0 msgr2=0x560702766ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:28:33.734 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:33.733+0000 7faf0be00640 1 --2- 172.21.15.47:0/3297291023 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5607027666d0 0x560702766ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:28:33.734 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:33.733+0000 7faf0be00640 1 -- 172.21.15.47:0/3297291023 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560702697080 con 0x56070276fc00 2024-03-21T10:28:33.734 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:33.733+0000 7faf10e00640 1 --2- 172.21.15.47:0/3297291023 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560702767080 0x56070276f630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T10:28:33.734 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:33.733+0000 7faf0be00640 1 --2- 172.21.15.47:0/3297291023 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56070276fc00 0x560702772050 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7faf000ba5b0 tx=0x7faf0002bcc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=79a22a8024a1cfa8 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:28:33.735 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:33.734+0000 7faefe000640 1 -- 172.21.15.47:0/3297291023 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf000c1d30 con 0x56070276fc00 2024-03-21T10:28:33.735 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:33.734+0000 7faefe000640 1 -- 172.21.15.47:0/3297291023 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7faf000c65f0 con 0x56070276fc00 2024-03-21T10:28:33.735 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:33.734+0000 7faefe000640 1 -- 172.21.15.47:0/3297291023 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf000bde40 con 0x56070276fc00 2024-03-21T10:28:33.736 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:33.735+0000 7faf16122740 1 -- 172.21.15.47:0/3297291023 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56070276fc00 msgr2=0x560702772050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:28:33.736 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:33.735+0000 7faf16122740 1 --2- 172.21.15.47:0/3297291023 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56070276fc00 0x560702772050 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7faf000ba5b0 tx=0x7faf0002bcc0 comp rx=0 tx=0).stop 2024-03-21T10:28:33.736 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:33.735+0000 7faf11800640 1 --2- 172.21.15.47:0/2278846240 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7faef40a3090 0x7faef40a5550 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7faf04005fd0 tx=0x7faf04009b30 comp rx=0 tx=0).ready entity=mgr.15687 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:28:33.736 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:33.735+0000 7faf16122740 1 -- 172.21.15.47:0/3297291023 shutdown_connections 2024-03-21T10:28:33.736 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:33.735+0000 7faf16122740 1 --2- 172.21.15.47:0/3297291023 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56070276fc00 0x560702772050 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:28:33.736 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:33.735+0000 7faf16122740 1 --2- 172.21.15.47:0/3297291023 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560702767080 0x56070276f630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:28:33.736 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:33.735+0000 7faf16122740 1 --2- 172.21.15.47:0/3297291023 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5607027666d0 0x560702766ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:28:33.736 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:33.735+0000 7faf16122740 1 -- 172.21.15.47:0/3297291023 >> 172.21.15.47:0/3297291023 conn(0x560702760da0 msgr2=0x560702761180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:28:33.737 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:33.735+0000 7faf16122740 1 -- 172.21.15.47:0/3297291023 shutdown_connections 2024-03-21T10:28:33.737 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:33.735+0000 7faf16122740 1 -- 172.21.15.47:0/3297291023 wait complete. 2024-03-21T10:28:33.737 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:33.736+0000 7faf16122740 1 -- start start 2024-03-21T10:28:33.737 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:33.736+0000 7faf16122740 1 Processor -- start 2024-03-21T10:28:33.737 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:33.737+0000 7faf16122740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5607027666d0 0x56070276f500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:28:33.737 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:33.737+0000 7faf16122740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560702767080 0x56070276c890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:28:33.737 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:33.737+0000 7faf16122740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56070276fc00 0x56070276ce10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:28:33.738 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:33.737+0000 7faf16122740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x560702772930 con 0x56070276fc00 2024-03-21T10:28:33.738 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:33.737+0000 7faf16122740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x5607026860d0 con 0x560702767080 2024-03-21T10:28:33.738 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:33.737+0000 7faf16122740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x560702772ab0 con 0x5607027666d0 2024-03-21T10:28:33.738 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:33.737+0000 7faf16122740 10 client.0 _ll_register_callbacks cb 0x56070268ae80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T10:28:33.738 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:33.738+0000 7faf10e00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5607027666d0 0x56070276f500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:28:33.738 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:33.738+0000 7faf10e00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5607027666d0 0x56070276f500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.47:33178/0 (socket says 172.21.15.47:33178) 2024-03-21T10:28:33.738 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:33.738+0000 7faf10e00640 1 -- 172.21.15.47:0/3254443738 learned_addr learned my addr 172.21.15.47:0/3254443738 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:28:33.738 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:33.738+0000 7faf11800640 1 --2- 172.21.15.47:0/3254443738 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56070276fc00 0x56070276ce10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:28:33.739 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:33.739+0000 7faf11800640 1 -- 172.21.15.47:0/3254443738 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5607027666d0 msgr2=0x56070276f500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:28:33.739 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:33.739+0000 7faf11800640 1 --2- 172.21.15.47:0/3254443738 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5607027666d0 0x56070276f500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:28:33.739 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:33.739+0000 7faf11800640 1 -- 172.21.15.47:0/3254443738 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560702767080 msgr2=0x56070276c890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:28:33.739 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:33.739+0000 7faf11800640 1 --2- 172.21.15.47:0/3254443738 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560702767080 0x56070276c890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:28:33.739 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:33.739+0000 7faf11800640 1 -- 172.21.15.47:0/3254443738 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5607023de3f0 con 0x56070276fc00 2024-03-21T10:28:33.739 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:33.739+0000 7faf10e00640 1 --2- 172.21.15.47:0/3254443738 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5607027666d0 0x56070276f500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T10:28:33.741 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:33.740+0000 7faf11800640 1 --2- 172.21.15.47:0/3254443738 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56070276fc00 0x56070276ce10 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7faf0400fc70 tx=0x7faf0401b000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=812cb1dbb5f10bfe server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:28:33.741 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:33.740+0000 7faef2a00640 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-03-21T10:28:33.741 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:33.740+0000 7faef2a00640 1 -- 172.21.15.47:0/3254443738 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf04003b70 con 0x56070276fc00 2024-03-21T10:28:33.741 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:33.740+0000 7faef2a00640 1 -- 172.21.15.47:0/3254443738 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(97..97 src has 1..97) v4 ==== 9474+0+0 (secure 0 0 0) 0x7faf04003d10 con 0x56070276fc00 2024-03-21T10:28:33.741 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:33.740+0000 7faf16122740 1 -- 172.21.15.47:0/3254443738 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7faf000c65f0 con 0x56070276fc00 2024-03-21T10:28:33.741 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:33.741+0000 7faef2a00640 1 -- 172.21.15.47:0/3254443738 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7faf04010930 con 0x56070276fc00 2024-03-21T10:28:33.741 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:33.741+0000 7faf16122740 10 client.16770 fetch_fsmap learned FSMap version 32 2024-03-21T10:28:33.741 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:33.741+0000 7faf16122740 1 -- 172.21.15.47:0/3254443738 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x5607024a87d0 con 0x56070276fc00 2024-03-21T10:28:33.742 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:33.742+0000 7faef2a00640 1 -- 172.21.15.47:0/3254443738 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7faf000c65f0 con 0x56070276fc00 2024-03-21T10:28:33.742 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:33.742+0000 7faef2a00640 1 -- 172.21.15.47:0/3254443738 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 32) v1 ==== 7942+0+0 (secure 0 0 0) 0x7faf04026070 con 0x56070276fc00 2024-03-21T10:28:33.742 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:33.742+0000 7faf16122740 10 client.16770 fetch_fsmap finished waiting for FSMap version 32 2024-03-21T10:28:33.744 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:33.742+0000 7faf16122740 10 client.16770 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.e 2024-03-21T10:28:33.744 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:33.742+0000 7faf16122740 20 client.16770 populate_metadata read hostname 'smithi047' 2024-03-21T10:28:33.744 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:33.742+0000 7faf16122740 1 --2- 172.21.15.47:0/3254443738 >> [v2:172.21.15.100:6834/2597849957,v1:172.21.15.100:6835/2597849957] conn(0x560702787f10 0x56070278a340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:28:33.744 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:33.742+0000 7faf16122740 4 client.16770 mds_command: new command op to 34454 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T10:28:33.744 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:33.742+0000 7faf16122740 1 -- 172.21.15.47:0/3254443738 --> [v2:172.21.15.100:6834/2597849957,v1:172.21.15.100:6835/2597849957] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5607026d7800 con 0x560702787f10 2024-03-21T10:28:33.745 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:33.742+0000 7faf10e00640 1 --2- 172.21.15.47:0/3254443738 >> [v2:172.21.15.100:6834/2597849957,v1:172.21.15.100:6835/2597849957] conn(0x560702787f10 0x56070278a340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:28:33.745 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:33.743+0000 7faf10e00640 1 --2- 172.21.15.47:0/3254443738 >> [v2:172.21.15.100:6834/2597849957,v1:172.21.15.100:6835/2597849957] conn(0x560702787f10 0x56070278a340 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=c155aeecbb4aecaa server_cookie=30436b1ef34e88c4 in_seq=0 out_seq=0 2024-03-21T10:28:33.745 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:33.743+0000 7faef2a00640 10 client.16770 ms_handle_connect on v2:172.21.15.100:6834/2597849957 2024-03-21T10:28:33.745 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:33.744+0000 7faef2a00640 1 -- 172.21.15.47:0/3254443738 <== mds.2 v2:172.21.15.100:6834/2597849957 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x5607026d7800 con 0x560702787f10 2024-03-21T10:28:33.746 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:33.744+0000 7faef2a00640 10 client.16770 handle_command_reply: tid=0 2024-03-21T10:28:33.746 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:33.745+0000 7faf16122740 1 client.16770 shutdown 2024-03-21T10:28:33.746 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:33.746+0000 7faf16122740 1 -- 172.21.15.47:0/3254443738 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56070276fc00 msgr2=0x56070276ce10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:28:33.746 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:33.746+0000 7faf16122740 1 --2- 172.21.15.47:0/3254443738 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56070276fc00 0x56070276ce10 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7faf0400fc70 tx=0x7faf0401b000 comp rx=0 tx=0).stop 2024-03-21T10:28:33.747 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:33.747+0000 7faf16122740 1 -- 172.21.15.47:0/3254443738 shutdown_connections 2024-03-21T10:28:33.747 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:33.747+0000 7faf16122740 1 --2- 172.21.15.47:0/3254443738 >> [v2:172.21.15.100:6834/2597849957,v1:172.21.15.100:6835/2597849957] conn(0x560702787f10 0x56070278a340 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:28:33.747 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:33.747+0000 7faf16122740 1 --2- 172.21.15.47:0/3254443738 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56070276fc00 0x56070276ce10 unknown :-1 s=CLOSED pgs=745 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:28:33.747 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:33.747+0000 7faf16122740 1 --2- 172.21.15.47:0/3254443738 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560702767080 0x56070276c890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:28:33.747 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:33.747+0000 7faf16122740 1 --2- 172.21.15.47:0/3254443738 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5607027666d0 0x56070276f500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:28:33.747 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:33.747+0000 7faf16122740 1 -- 172.21.15.47:0/3254443738 >> 172.21.15.47:0/3254443738 conn(0x560702760da0 msgr2=0x560702762af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:28:33.747 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:33.747+0000 7faef2a00640 0 client.16770 ms_handle_reset on v2:172.21.15.100:6834/2597849957 2024-03-21T10:28:33.749 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:33.749+0000 7faf16122740 1 -- 172.21.15.47:0/3254443738 shutdown_connections 2024-03-21T10:28:33.749 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:33.749+0000 7faf16122740 1 -- 172.21.15.47:0/3254443738 wait complete. 2024-03-21T10:28:33.749 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:33.750+0000 7faf16122740 20 client.16770 trim_cache size 0 max 16384 2024-03-21T10:28:33.784 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:33.784+0000 7faf16122740 1 Processor -- start 2024-03-21T10:28:33.784 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:33.785+0000 7faf16122740 1 -- start start 2024-03-21T10:28:33.787 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:33.785+0000 7faf16122740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5607027666d0 0x56070276efc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:28:33.787 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:33.785+0000 7faf16122740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x560702767080 0x56070276f500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:28:33.787 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:33.785+0000 7faf16122740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56070276fc00 0x560702769af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:28:33.787 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:33.785+0000 7faf16122740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf0c1ac660 con 0x560702767080 2024-03-21T10:28:33.787 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:33.785+0000 7faf16122740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf0c1adfe0 con 0x5607027666d0 2024-03-21T10:28:33.787 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:33.785+0000 7faf16122740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf0c1accc0 con 0x56070276fc00 2024-03-21T10:28:33.787 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:33.786+0000 7faf0be00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x560702767080 0x56070276f500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:28:33.787 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:33.786+0000 7faf10e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5607027666d0 0x56070276efc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:28:33.787 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:33.786+0000 7faf0be00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x560702767080 0x56070276f500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:60554/0 (socket says 172.21.15.47:60554) 2024-03-21T10:28:33.787 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:33.786+0000 7faf0be00640 1 -- 172.21.15.47:0/2040715783 learned_addr learned my addr 172.21.15.47:0/2040715783 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:28:33.787 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:33.786+0000 7faf0be00640 1 -- 172.21.15.47:0/2040715783 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56070276fc00 msgr2=0x560702769af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:28:33.787 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:33.786+0000 7faf0be00640 1 --2- 172.21.15.47:0/2040715783 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56070276fc00 0x560702769af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:28:33.788 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:33.786+0000 7faf0be00640 1 -- 172.21.15.47:0/2040715783 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5607027666d0 msgr2=0x56070276efc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:28:33.788 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:33.786+0000 7faf0be00640 1 --2- 172.21.15.47:0/2040715783 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5607027666d0 0x56070276efc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:28:33.788 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:33.786+0000 7faf0be00640 1 -- 172.21.15.47:0/2040715783 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5607027605c0 con 0x560702767080 2024-03-21T10:28:33.788 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:33.787+0000 7faf0be00640 1 --2- 172.21.15.47:0/2040715783 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x560702767080 0x56070276f500 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7faf000ba040 tx=0x7faf000bda60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8c98c1204efdd0a8 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:28:33.788 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:33.788+0000 7faefd600640 1 -- 172.21.15.47:0/2040715783 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf000bbee0 con 0x560702767080 2024-03-21T10:28:33.788 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:33.788+0000 7faefd600640 1 -- 172.21.15.47:0/2040715783 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7faf0002dd40 con 0x560702767080 2024-03-21T10:28:33.789 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:33.788+0000 7faefd600640 1 -- 172.21.15.47:0/2040715783 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf000bbee0 con 0x560702767080 2024-03-21T10:28:33.789 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:33.790+0000 7faf16122740 1 -- 172.21.15.47:0/2040715783 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x560702767080 msgr2=0x56070276f500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:28:33.789 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:33.790+0000 7faf16122740 1 --2- 172.21.15.47:0/2040715783 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x560702767080 0x56070276f500 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7faf000ba040 tx=0x7faf000bda60 comp rx=0 tx=0).stop 2024-03-21T10:28:33.790 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:33.790+0000 7faf16122740 1 -- 172.21.15.47:0/2040715783 shutdown_connections 2024-03-21T10:28:33.790 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:33.790+0000 7faf16122740 1 --2- 172.21.15.47:0/2040715783 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56070276fc00 0x560702769af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:28:33.790 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:33.790+0000 7faf16122740 1 --2- 172.21.15.47:0/2040715783 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x560702767080 0x56070276f500 unknown :-1 s=CLOSED pgs=746 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:28:33.790 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:33.790+0000 7faf16122740 1 --2- 172.21.15.47:0/2040715783 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5607027666d0 0x56070276efc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:28:33.790 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:33.790+0000 7faf16122740 1 -- 172.21.15.47:0/2040715783 >> 172.21.15.47:0/2040715783 conn(0x560702760da0 msgr2=0x56070275feb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:28:33.791 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:33.791+0000 7faf16122740 1 -- 172.21.15.47:0/2040715783 shutdown_connections 2024-03-21T10:28:33.791 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:33.791+0000 7faf16122740 1 -- 172.21.15.47:0/2040715783 wait complete. 2024-03-21T10:28:33.792 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:33.792+0000 7faf16122740 1 -- start start 2024-03-21T10:28:33.792 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:33.792+0000 7faf16122740 1 Processor -- start 2024-03-21T10:28:33.793 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:33.793+0000 7faf16122740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5607027666d0 0x56070277fe70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:28:33.793 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:33.793+0000 7faf16122740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56070276fc00 0x5607027803b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:28:33.793 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:33.793+0000 7faf16122740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56070277c1a0 0x56070277c600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:28:33.794 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:33.793+0000 7faf16122740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf0c1ad980 con 0x56070276fc00 2024-03-21T10:28:33.794 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:33.793+0000 7faf16122740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf0c1ad320 con 0x56070277c1a0 2024-03-21T10:28:33.794 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:33.793+0000 7faf16122740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf0c1aa840 con 0x5607027666d0 2024-03-21T10:28:33.794 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:33.793+0000 7faf16122740 10 client.0 _ll_register_callbacks cb 0x7faf04009d80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T10:28:33.794 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:33.794+0000 7faf10e00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5607027666d0 0x56070277fe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:28:33.794 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:33.794+0000 7faf10e00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5607027666d0 0x56070277fe70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.47:33190/0 (socket says 172.21.15.47:33190) 2024-03-21T10:28:33.794 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:33.794+0000 7faf10e00640 1 -- 172.21.15.47:0/2504290825 learned_addr learned my addr 172.21.15.47:0/2504290825 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:28:33.794 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:33.794+0000 7faf11800640 1 --2- 172.21.15.47:0/2504290825 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56070277c1a0 0x56070277c600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:28:33.794 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:33.794+0000 7faf10e00640 1 -- 172.21.15.47:0/2504290825 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56070277c1a0 msgr2=0x56070277c600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:28:33.795 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:33.794+0000 7faf10e00640 1 --2- 172.21.15.47:0/2504290825 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56070277c1a0 0x56070277c600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:28:33.795 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:33.794+0000 7faf10e00640 1 -- 172.21.15.47:0/2504290825 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56070276fc00 msgr2=0x5607027803b0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-21T10:28:33.795 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:33.796+0000 7faf0be00640 1 --2- 172.21.15.47:0/2504290825 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56070276fc00 0x5607027803b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:28:33.797 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:33.798+0000 7faf10e00640 1 --2- 172.21.15.47:0/2504290825 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56070276fc00 0x5607027803b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:28:33.799 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:33.798+0000 7faf10e00640 1 -- 172.21.15.47:0/2504290825 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x560702768470 con 0x5607027666d0 2024-03-21T10:28:33.799 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:33.799+0000 7faf10e00640 1 --2- 172.21.15.47:0/2504290825 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5607027666d0 0x56070277fe70 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7faef80124c0 tx=0x7faef801edb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=387115092e74ae71 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:28:33.799 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:33.799+0000 7faef2a00640 10 client.0 ms_handle_connect on v2:172.21.15.157:3300/0 2024-03-21T10:28:33.799 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:33.800+0000 7faf0be00640 1 --2- 172.21.15.47:0/2504290825 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56070276fc00 0x5607027803b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T10:28:33.801 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:33.800+0000 7faef2a00640 1 -- 172.21.15.47:0/2504290825 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faef8005020 con 0x5607027666d0 2024-03-21T10:28:33.801 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:33.800+0000 7faef2a00640 1 -- 172.21.15.47:0/2504290825 <== mon.1 v2:172.21.15.157:3300/0 2 ==== osd_map(97..97 src has 1..97) v4 ==== 9474+0+0 (secure 0 0 0) 0x7faef80051c0 con 0x5607027666d0 2024-03-21T10:28:33.801 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:33.801+0000 7faf16122740 1 -- 172.21.15.47:0/2504290825 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7faf0002dd40 con 0x5607027666d0 2024-03-21T10:28:33.802 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:33.802+0000 7faef2a00640 1 -- 172.21.15.47:0/2504290825 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7faef80053c0 con 0x5607027666d0 2024-03-21T10:28:33.803 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:33.802+0000 7faf16122740 10 client.26347 fetch_fsmap learned FSMap version 32 2024-03-21T10:28:33.803 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:33.802+0000 7faf16122740 1 -- 172.21.15.47:0/2504290825 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x5607027e4640 con 0x5607027666d0 2024-03-21T10:28:33.804 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:33.805+0000 7faef2a00640 1 -- 172.21.15.47:0/2504290825 <== mon.1 v2:172.21.15.157:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7faf0002dd40 con 0x5607027666d0 2024-03-21T10:28:33.804 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:33.805+0000 7faef2a00640 1 -- 172.21.15.47:0/2504290825 <== mon.1 v2:172.21.15.157:3300/0 5 ==== fsmap(e 32) v1 ==== 7942+0+0 (secure 0 0 0) 0x7faef8037760 con 0x5607027666d0 2024-03-21T10:28:33.805 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:33.805+0000 7faf16122740 10 client.26347 fetch_fsmap finished waiting for FSMap version 32 2024-03-21T10:28:33.805 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:33.805+0000 7faf16122740 10 client.26347 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.e 2024-03-21T10:28:33.805 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:33.805+0000 7faf16122740 20 client.26347 populate_metadata read hostname 'smithi047' 2024-03-21T10:28:33.805 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:33.805+0000 7faf16122740 1 --2- 172.21.15.47:0/2504290825 >> [v2:172.21.15.100:6834/2597849957,v1:172.21.15.100:6835/2597849957] conn(0x5607027e4cf0 0x5607027e7110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:28:33.805 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:33.805+0000 7faf16122740 4 client.26347 mds_command: new command op to 34454 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-21T10:28:33.805 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:33.805+0000 7faf16122740 1 -- 172.21.15.47:0/2504290825 --> [v2:172.21.15.100:6834/2597849957,v1:172.21.15.100:6835/2597849957] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x5607026d1b50 con 0x5607027e4cf0 2024-03-21T10:28:33.806 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:33.807+0000 7faf0be00640 1 --2- 172.21.15.47:0/2504290825 >> [v2:172.21.15.100:6834/2597849957,v1:172.21.15.100:6835/2597849957] conn(0x5607027e4cf0 0x5607027e7110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:28:33.809 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:33.809+0000 7faf0be00640 1 --2- 172.21.15.47:0/2504290825 >> [v2:172.21.15.100:6834/2597849957,v1:172.21.15.100:6835/2597849957] conn(0x5607027e4cf0 0x5607027e7110 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=ea6e92e75926b006 server_cookie=2d77200699d34750 in_seq=0 out_seq=0 2024-03-21T10:28:33.809 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:33.809+0000 7faef2a00640 10 client.26347 ms_handle_connect on v2:172.21.15.100:6834/2597849957 2024-03-21T10:28:33.811 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:33.811+0000 7faef2a00640 1 -- 172.21.15.47:0/2504290825 <== mds.2 v2:172.21.15.100:6834/2597849957 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x5607026d1b50 con 0x5607027e4cf0 2024-03-21T10:28:33.811 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:33.811+0000 7faef2a00640 10 client.26347 handle_command_reply: tid=0 2024-03-21T10:28:33.811 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:33.811+0000 7faf16122740 1 client.26347 shutdown 2024-03-21T10:28:33.811 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:33.812+0000 7faf16122740 1 -- 172.21.15.47:0/2504290825 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5607027666d0 msgr2=0x56070277fe70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:28:33.812 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:33.812+0000 7faf16122740 1 --2- 172.21.15.47:0/2504290825 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5607027666d0 0x56070277fe70 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7faef80124c0 tx=0x7faef801edb0 comp rx=0 tx=0).stop 2024-03-21T10:28:33.812 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:33.813+0000 7faf16122740 1 -- 172.21.15.47:0/2504290825 shutdown_connections 2024-03-21T10:28:33.812 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:33.813+0000 7faf16122740 1 --2- 172.21.15.47:0/2504290825 >> [v2:172.21.15.100:6834/2597849957,v1:172.21.15.100:6835/2597849957] conn(0x5607027e4cf0 0x5607027e7110 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:28:33.812 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:33.813+0000 7faf16122740 1 --2- 172.21.15.47:0/2504290825 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56070277c1a0 0x56070277c600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:28:33.812 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:33.813+0000 7faf16122740 1 --2- 172.21.15.47:0/2504290825 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56070276fc00 0x5607027803b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:28:33.812 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:33.813+0000 7faf16122740 1 --2- 172.21.15.47:0/2504290825 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5607027666d0 0x56070277fe70 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:28:33.812 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:33.813+0000 7faf16122740 1 -- 172.21.15.47:0/2504290825 >> 172.21.15.47:0/2504290825 conn(0x560702760da0 msgr2=0x560702767460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:28:33.813 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:33.813+0000 7faef2a00640 0 client.26347 ms_handle_reset on v2:172.21.15.100:6834/2597849957 2024-03-21T10:28:33.814 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:33.815+0000 7faf16122740 1 -- 172.21.15.47:0/2504290825 shutdown_connections 2024-03-21T10:28:33.814 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:33.815+0000 7faf16122740 1 -- 172.21.15.47:0/2504290825 wait complete. 2024-03-21T10:28:33.814 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:33.815+0000 7faf16122740 20 client.26347 trim_cache size 0 max 16384 2024-03-21T10:28:33.815 INFO:teuthology.orchestra.run.smithi047.stdout:[] 2024-03-21T10:28:33.816 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:33.816+0000 7faf13800640 1 -- 172.21.15.47:0/2278846240 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7faef40a3090 msgr2=0x7faef40a5550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:28:33.816 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:33.816+0000 7faf13800640 1 --2- 172.21.15.47:0/2278846240 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7faef40a3090 0x7faef40a5550 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7faf04005fd0 tx=0x7faf04009b30 comp rx=0 tx=0).stop 2024-03-21T10:28:33.816 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:33.816+0000 7faf13800640 1 -- 172.21.15.47:0/2278846240 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7faf0c0743f0 msgr2=0x7faf0c19a5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:28:33.816 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:33.816+0000 7faf13800640 1 --2- 172.21.15.47:0/2278846240 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7faf0c0743f0 0x7faf0c19a5d0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7faf00005940 tx=0x7faf000045e0 comp rx=0 tx=0).stop 2024-03-21T10:28:33.816 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:33.817+0000 7faf13800640 1 -- 172.21.15.47:0/2278846240 shutdown_connections 2024-03-21T10:28:33.816 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:33.817+0000 7faf13800640 1 --2- 172.21.15.47:0/2278846240 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7faf0c10dee0 0x7faf0c19e950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:28:33.816 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:33.817+0000 7faf13800640 1 --2- 172.21.15.47:0/2278846240 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7faef40a3090 0x7faef40a5550 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:28:33.816 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:33.817+0000 7faf13800640 1 --2- 172.21.15.47:0/2278846240 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7faf0c0743f0 0x7faf0c19a5d0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:28:33.817 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:33.817+0000 7faf13800640 1 --2- 172.21.15.47:0/2278846240 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7faf0c073ad0 0x7faf0c19a090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:28:33.817 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:33.817+0000 7faf13800640 1 -- 172.21.15.47:0/2278846240 >> 172.21.15.47:0/2278846240 conn(0x7faf0c0fd1d0 msgr2=0x7faf0c0fea30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:28:33.817 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:33.817+0000 7faf13800640 1 -- 172.21.15.47:0/2278846240 shutdown_connections 2024-03-21T10:28:33.817 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:33.817+0000 7faf13800640 1 -- 172.21.15.47:0/2278846240 wait complete. 2024-03-21T10:28:33.852 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-21T10:28:33.852 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-03-21T10:28:34.175 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:28:34 smithi047 ceph-mon[31547]: from='mgr.15687 172.21.15.47:0/8730661' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T10:28:34.273 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:34.273+0000 7ffa7d400640 1 -- 172.21.15.47:0/4098083090 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ffa781008d0 msgr2=0x7ffa78104910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:28:34.273 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:34.273+0000 7ffa7d400640 1 --2- 172.21.15.47:0/4098083090 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ffa781008d0 0x7ffa78104910 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7ffa64009900 tx=0x7ffa6402bc80 comp rx=0 tx=0).stop 2024-03-21T10:28:34.274 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:34.273+0000 7ffa7d400640 1 -- 172.21.15.47:0/4098083090 shutdown_connections 2024-03-21T10:28:34.274 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:34.273+0000 7ffa7d400640 1 --2- 172.21.15.47:0/4098083090 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ffa78104ff0 0x7ffa7810cbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:28:34.274 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:34.273+0000 7ffa7d400640 1 --2- 172.21.15.47:0/4098083090 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ffa781008d0 0x7ffa78104910 secure :-1 s=CLOSED pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7ffa64009900 tx=0x7ffa6402bc80 comp rx=0 tx=0).stop 2024-03-21T10:28:34.274 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:34.273+0000 7ffa7d400640 1 --2- 172.21.15.47:0/4098083090 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ffa780fff20 0x7ffa78100300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:28:34.274 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:34.273+0000 7ffa7d400640 1 -- 172.21.15.47:0/4098083090 >> 172.21.15.47:0/4098083090 conn(0x7ffa78076ac0 msgr2=0x7ffa780fde50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:28:34.274 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:34.274+0000 7ffa7d400640 1 -- 172.21.15.47:0/4098083090 shutdown_connections 2024-03-21T10:28:34.274 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:34.274+0000 7ffa7d400640 1 -- 172.21.15.47:0/4098083090 wait complete. 2024-03-21T10:28:34.274 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:34.275+0000 7ffa7d400640 1 Processor -- start 2024-03-21T10:28:34.275 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:34.275+0000 7ffa7d400640 1 -- start start 2024-03-21T10:28:34.275 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:34.276+0000 7ffa7d400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ffa780fff20 0x7ffa78106c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:28:34.275 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:34.276+0000 7ffa7d400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ffa78104ff0 0x7ffa78107180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:28:34.275 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:34.276+0000 7ffa7d400640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ffa781076c0 0x7ffa781aaa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:28:34.276 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:34.276+0000 7ffa7d400640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa78111a30 con 0x7ffa780fff20 2024-03-21T10:28:34.276 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:34.276+0000 7ffa7d400640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa781118b0 con 0x7ffa78104ff0 2024-03-21T10:28:34.276 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:34.276+0000 7ffa7d400640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa78111bb0 con 0x7ffa781076c0 2024-03-21T10:28:34.276 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:34.277+0000 7ffa76800640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ffa780fff20 0x7ffa78106c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:28:34.276 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:34.277+0000 7ffa77200640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ffa781076c0 0x7ffa781aaa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:28:34.276 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:34.277+0000 7ffa77200640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ffa781076c0 0x7ffa781aaa00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.47:33212/0 (socket says 172.21.15.47:33212) 2024-03-21T10:28:34.277 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:34.277+0000 7ffa77200640 1 -- 172.21.15.47:0/2721693983 learned_addr learned my addr 172.21.15.47:0/2721693983 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:28:34.278 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:34.278+0000 7ffa76800640 1 -- 172.21.15.47:0/2721693983 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ffa781076c0 msgr2=0x7ffa781aaa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:28:34.278 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:34.278+0000 7ffa76800640 1 --2- 172.21.15.47:0/2721693983 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ffa781076c0 0x7ffa781aaa00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:28:34.278 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:34.278+0000 7ffa76800640 1 -- 172.21.15.47:0/2721693983 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ffa78104ff0 msgr2=0x7ffa78107180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:28:34.278 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:34.278+0000 7ffa76800640 1 --2- 172.21.15.47:0/2721693983 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ffa78104ff0 0x7ffa78107180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:28:34.278 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:34.278+0000 7ffa76800640 1 -- 172.21.15.47:0/2721693983 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffa781aaf40 con 0x7ffa780fff20 2024-03-21T10:28:34.278 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:34.278+0000 7ffa76800640 1 --2- 172.21.15.47:0/2721693983 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ffa780fff20 0x7ffa78106c40 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7ffa6800be20 tx=0x7ffa68009f70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:28:34.279 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:34.279+0000 7ffa6f400640 1 -- 172.21.15.47:0/2721693983 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa68019070 con 0x7ffa780fff20 2024-03-21T10:28:34.279 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:34.279+0000 7ffa6f400640 1 -- 172.21.15.47:0/2721693983 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7ffa680148f0 con 0x7ffa780fff20 2024-03-21T10:28:34.279 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:34.279+0000 7ffa7d400640 1 -- 172.21.15.47:0/2721693983 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffa781ab1d0 con 0x7ffa780fff20 2024-03-21T10:28:34.280 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:34.279+0000 7ffa6f400640 1 -- 172.21.15.47:0/2721693983 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa680128d0 con 0x7ffa780fff20 2024-03-21T10:28:34.280 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:34.279+0000 7ffa7d400640 1 -- 172.21.15.47:0/2721693983 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffa781ab720 con 0x7ffa780fff20 2024-03-21T10:28:34.285 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:34.286+0000 7ffa6f400640 1 -- 172.21.15.47:0/2721693983 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 27) v1 ==== 137985+0+0 (secure 0 0 0) 0x7ffa6801c070 con 0x7ffa780fff20 2024-03-21T10:28:34.288 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:34.288+0000 7ffa6f400640 1 --2- 172.21.15.47:0/2721693983 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7ffa5c0a3110 0x7ffa5c0a55d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:28:34.288 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:34.289+0000 7ffa6f400640 1 -- 172.21.15.47:0/2721693983 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 9474+0+0 (secure 0 0 0) 0x7ffa680cf310 con 0x7ffa780fff20 2024-03-21T10:28:34.289 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:34.290+0000 7ffa75e00640 1 --2- 172.21.15.47:0/2721693983 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7ffa5c0a3110 0x7ffa5c0a55d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:28:34.291 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:34.291+0000 7ffa75e00640 1 --2- 172.21.15.47:0/2721693983 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7ffa5c0a3110 0x7ffa5c0a55d0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7ffa78108450 tx=0x7ffa64000bc0 comp rx=0 tx=0).ready entity=mgr.15687 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:28:34.299 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:34.299+0000 7ffa801c9740 1 Processor -- start 2024-03-21T10:28:34.299 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:34.300+0000 7ffa801c9740 1 -- start start 2024-03-21T10:28:34.300 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:34.300+0000 7ffa801c9740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5558b0a5a6d0 0x5558b0a5aab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:28:34.301 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:34.300+0000 7ffa801c9740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5558b0a5b080 0x5558b0a63630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:28:34.302 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:34.300+0000 7ffa801c9740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5558b0a63c00 0x5558b0a66050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:28:34.302 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:34.300+0000 7ffa801c9740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x5558b09be2c0 con 0x5558b0a5b080 2024-03-21T10:28:34.302 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:34.300+0000 7ffa801c9740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x5558b0966c90 con 0x5558b0a5a6d0 2024-03-21T10:28:34.302 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:34.300+0000 7ffa801c9740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x5558b09ba360 con 0x5558b0a63c00 2024-03-21T10:28:34.302 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:34.302+0000 7ffa77200640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5558b0a63c00 0x5558b0a66050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:28:34.302 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:34.302+0000 7ffa77200640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5558b0a63c00 0x5558b0a66050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.47:33224/0 (socket says 172.21.15.47:33224) 2024-03-21T10:28:34.302 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:34.302+0000 7ffa77200640 1 -- 172.21.15.47:0/630896476 learned_addr learned my addr 172.21.15.47:0/630896476 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:28:34.302 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:34.302+0000 7ffa75e00640 1 --2- 172.21.15.47:0/630896476 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5558b0a5b080 0x5558b0a63630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:28:34.302 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:34.302+0000 7ffa76800640 1 --2- 172.21.15.47:0/630896476 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5558b0a5a6d0 0x5558b0a5aab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:28:34.303 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:34.303+0000 7ffa77200640 1 -- 172.21.15.47:0/630896476 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5558b0a5a6d0 msgr2=0x5558b0a5aab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:28:34.303 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:34.303+0000 7ffa77200640 1 --2- 172.21.15.47:0/630896476 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5558b0a5a6d0 0x5558b0a5aab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:28:34.303 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:34.303+0000 7ffa77200640 1 -- 172.21.15.47:0/630896476 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5558b0a5b080 msgr2=0x5558b0a63630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:28:34.303 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:34.303+0000 7ffa77200640 1 --2- 172.21.15.47:0/630896476 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5558b0a5b080 0x5558b0a63630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:28:34.303 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:34.303+0000 7ffa77200640 1 -- 172.21.15.47:0/630896476 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5558b098b080 con 0x5558b0a63c00 2024-03-21T10:28:34.303 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:34.303+0000 7ffa77200640 1 --2- 172.21.15.47:0/630896476 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5558b0a63c00 0x5558b0a66050 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7ffa70009a80 tx=0x7ffa70009f40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b321e8d821f69b5f server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:28:34.304 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:34.304+0000 7ffa63400640 1 -- 172.21.15.47:0/630896476 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa70017070 con 0x5558b0a63c00 2024-03-21T10:28:34.304 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:34.304+0000 7ffa63400640 1 -- 172.21.15.47:0/630896476 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7ffa7000eba0 con 0x5558b0a63c00 2024-03-21T10:28:34.304 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:34.304+0000 7ffa63400640 1 -- 172.21.15.47:0/630896476 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa70012630 con 0x5558b0a63c00 2024-03-21T10:28:34.305 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:34.305+0000 7ffa801c9740 1 -- 172.21.15.47:0/630896476 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5558b0a63c00 msgr2=0x5558b0a66050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:28:34.305 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:34.305+0000 7ffa801c9740 1 --2- 172.21.15.47:0/630896476 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5558b0a63c00 0x5558b0a66050 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7ffa70009a80 tx=0x7ffa70009f40 comp rx=0 tx=0).stop 2024-03-21T10:28:34.305 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:34.306+0000 7ffa801c9740 1 -- 172.21.15.47:0/630896476 shutdown_connections 2024-03-21T10:28:34.305 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:34.306+0000 7ffa801c9740 1 --2- 172.21.15.47:0/630896476 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5558b0a63c00 0x5558b0a66050 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:28:34.305 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:34.306+0000 7ffa801c9740 1 --2- 172.21.15.47:0/630896476 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5558b0a5b080 0x5558b0a63630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:28:34.305 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:34.306+0000 7ffa801c9740 1 --2- 172.21.15.47:0/630896476 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5558b0a5a6d0 0x5558b0a5aab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:28:34.305 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:34.306+0000 7ffa801c9740 1 -- 172.21.15.47:0/630896476 >> 172.21.15.47:0/630896476 conn(0x5558b0a54da0 msgr2=0x5558b0a55180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:28:34.307 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:34.308+0000 7ffa801c9740 1 -- 172.21.15.47:0/630896476 shutdown_connections 2024-03-21T10:28:34.307 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:34.308+0000 7ffa801c9740 1 -- 172.21.15.47:0/630896476 wait complete. 2024-03-21T10:28:34.308 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:34.308+0000 7ffa801c9740 1 -- start start 2024-03-21T10:28:34.308 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:34.309+0000 7ffa801c9740 1 Processor -- start 2024-03-21T10:28:34.309 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:34.309+0000 7ffa801c9740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5558b0a5a6d0 0x5558b0a63500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:28:34.309 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:34.309+0000 7ffa801c9740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5558b0a5b080 0x5558b0a60890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:28:34.309 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:34.309+0000 7ffa801c9740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5558b0a63c00 0x5558b0a60e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:28:34.310 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:34.309+0000 7ffa801c9740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x5558b0a66b70 con 0x5558b0a5a6d0 2024-03-21T10:28:34.310 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:34.309+0000 7ffa801c9740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x5558b097a0d0 con 0x5558b0a63c00 2024-03-21T10:28:34.310 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:34.310+0000 7ffa801c9740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x5558b0a66cf0 con 0x5558b0a5b080 2024-03-21T10:28:34.310 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:34.310+0000 7ffa801c9740 10 client.0 _ll_register_callbacks cb 0x5558b097ee80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T10:28:34.310 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:34.310+0000 7ffa75e00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5558b0a5b080 0x5558b0a60890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:28:34.310 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:34.310+0000 7ffa75e00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5558b0a5b080 0x5558b0a60890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.47:33230/0 (socket says 172.21.15.47:33230) 2024-03-21T10:28:34.310 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:34.310+0000 7ffa75e00640 1 -- 172.21.15.47:0/2666454076 learned_addr learned my addr 172.21.15.47:0/2666454076 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:28:34.310 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:34.310+0000 7ffa75e00640 1 -- 172.21.15.47:0/2666454076 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5558b0a63c00 msgr2=0x5558b0a60e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:28:34.310 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:34.310+0000 7ffa75e00640 1 --2- 172.21.15.47:0/2666454076 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5558b0a63c00 0x5558b0a60e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:28:34.311 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:34.310+0000 7ffa75e00640 1 -- 172.21.15.47:0/2666454076 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5558b0a5a6d0 msgr2=0x5558b0a63500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:28:34.311 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:34.310+0000 7ffa75e00640 1 --2- 172.21.15.47:0/2666454076 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5558b0a5a6d0 0x5558b0a63500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:28:34.311 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:34.311+0000 7ffa75e00640 1 -- 172.21.15.47:0/2666454076 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5558b06d23f0 con 0x5558b0a5b080 2024-03-21T10:28:34.311 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:34.312+0000 7ffa75e00640 1 --2- 172.21.15.47:0/2666454076 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5558b0a5b080 0x5558b0a60890 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7ffa78073dc0 tx=0x7ffa64009260 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5f104e7252dcacb5 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:28:34.311 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:34.312+0000 7ffa57e00640 10 client.0 ms_handle_connect on v2:172.21.15.157:3300/0 2024-03-21T10:28:34.313 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:34.314+0000 7ffa57e00640 1 -- 172.21.15.47:0/2666454076 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa6404e020 con 0x5558b0a5b080 2024-03-21T10:28:34.313 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:34.314+0000 7ffa57e00640 1 -- 172.21.15.47:0/2666454076 <== mon.1 v2:172.21.15.157:3300/0 2 ==== osd_map(97..97 src has 1..97) v4 ==== 9474+0+0 (secure 0 0 0) 0x7ffa64005460 con 0x5558b0a5b080 2024-03-21T10:28:34.314 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:34.314+0000 7ffa801c9740 1 -- 172.21.15.47:0/2666454076 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ffa7000eba0 con 0x5558b0a5b080 2024-03-21T10:28:34.315 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:34.316+0000 7ffa57e00640 1 -- 172.21.15.47:0/2666454076 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7ffa64044030 con 0x5558b0a5b080 2024-03-21T10:28:34.315 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:34.316+0000 7ffa801c9740 10 client.26356 fetch_fsmap learned FSMap version 32 2024-03-21T10:28:34.316 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:34.316+0000 7ffa801c9740 1 -- 172.21.15.47:0/2666454076 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x5558b079c7d0 con 0x5558b0a5b080 2024-03-21T10:28:34.317 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:34.317+0000 7ffa57e00640 1 -- 172.21.15.47:0/2666454076 <== mon.1 v2:172.21.15.157:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7ffa7000eba0 con 0x5558b0a5b080 2024-03-21T10:28:34.318 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:34.319+0000 7ffa57e00640 1 -- 172.21.15.47:0/2666454076 <== mon.1 v2:172.21.15.157:3300/0 5 ==== fsmap(e 32) v1 ==== 7942+0+0 (secure 0 0 0) 0x7ffa640046c0 con 0x5558b0a5b080 2024-03-21T10:28:34.318 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:34.319+0000 7ffa801c9740 10 client.26356 fetch_fsmap finished waiting for FSMap version 32 2024-03-21T10:28:34.319 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:34.319+0000 7ffa801c9740 10 client.26356 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-03-21T10:28:34.319 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:34.319+0000 7ffa801c9740 20 client.26356 populate_metadata read hostname 'smithi047' 2024-03-21T10:28:34.319 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:34.319+0000 7ffa801c9740 1 --2- 172.21.15.47:0/2666454076 >> [v2:172.21.15.100:6836/3497644162,v1:172.21.15.100:6837/3497644162] conn(0x5558b0a849f0 0x5558b0a86e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:28:34.319 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:34.319+0000 7ffa801c9740 4 client.26356 mds_command: new command op to 34493 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T10:28:34.319 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:34.319+0000 7ffa801c9740 1 -- 172.21.15.47:0/2666454076 --> [v2:172.21.15.100:6836/3497644162,v1:172.21.15.100:6837/3497644162] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5558b09cb800 con 0x5558b0a849f0 2024-03-21T10:28:34.320 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:34.320+0000 7ffa76800640 1 --2- 172.21.15.47:0/2666454076 >> [v2:172.21.15.100:6836/3497644162,v1:172.21.15.100:6837/3497644162] conn(0x5558b0a849f0 0x5558b0a86e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:28:34.320 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:34.321+0000 7ffa76800640 1 --2- 172.21.15.47:0/2666454076 >> [v2:172.21.15.100:6836/3497644162,v1:172.21.15.100:6837/3497644162] conn(0x5558b0a849f0 0x5558b0a86e20 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=fc5d1e6e19812c71 server_cookie=3c217a25032efeac in_seq=0 out_seq=0 2024-03-21T10:28:34.320 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:34.321+0000 7ffa57e00640 10 client.26356 ms_handle_connect on v2:172.21.15.100:6836/3497644162 2024-03-21T10:28:34.327 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:34.323+0000 7ffa57e00640 1 -- 172.21.15.47:0/2666454076 <== mds.1 v2:172.21.15.100:6836/3497644162 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x5558b09cb800 con 0x5558b0a849f0 2024-03-21T10:28:34.327 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:34.323+0000 7ffa57e00640 10 client.26356 handle_command_reply: tid=0 2024-03-21T10:28:34.327 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:34.323+0000 7ffa801c9740 1 client.26356 shutdown 2024-03-21T10:28:34.327 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:34.324+0000 7ffa801c9740 1 -- 172.21.15.47:0/2666454076 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5558b0a5b080 msgr2=0x5558b0a60890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:28:34.327 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:34.324+0000 7ffa801c9740 1 --2- 172.21.15.47:0/2666454076 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5558b0a5b080 0x5558b0a60890 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7ffa78073dc0 tx=0x7ffa64009260 comp rx=0 tx=0).stop 2024-03-21T10:28:34.327 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:34.324+0000 7ffa801c9740 1 -- 172.21.15.47:0/2666454076 shutdown_connections 2024-03-21T10:28:34.327 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:34.324+0000 7ffa801c9740 1 --2- 172.21.15.47:0/2666454076 >> [v2:172.21.15.100:6836/3497644162,v1:172.21.15.100:6837/3497644162] conn(0x5558b0a849f0 0x5558b0a86e20 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:28:34.327 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:34.324+0000 7ffa801c9740 1 --2- 172.21.15.47:0/2666454076 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5558b0a63c00 0x5558b0a60e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:28:34.327 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:34.324+0000 7ffa801c9740 1 --2- 172.21.15.47:0/2666454076 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5558b0a5b080 0x5558b0a60890 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:28:34.327 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:34.324+0000 7ffa801c9740 1 --2- 172.21.15.47:0/2666454076 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5558b0a5a6d0 0x5558b0a63500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:28:34.327 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:34.324+0000 7ffa801c9740 1 -- 172.21.15.47:0/2666454076 >> 172.21.15.47:0/2666454076 conn(0x5558b0a54da0 msgr2=0x5558b0a56af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:28:34.327 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:34.324+0000 7ffa57e00640 0 client.26356 ms_handle_reset on v2:172.21.15.100:6836/3497644162 2024-03-21T10:28:34.328 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:34.325+0000 7ffa801c9740 1 -- 172.21.15.47:0/2666454076 shutdown_connections 2024-03-21T10:28:34.328 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:34.325+0000 7ffa801c9740 1 -- 172.21.15.47:0/2666454076 wait complete. 2024-03-21T10:28:34.328 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:34.325+0000 7ffa801c9740 20 client.26356 trim_cache size 0 max 16384 2024-03-21T10:28:34.356 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:34.357+0000 7ffa801c9740 1 Processor -- start 2024-03-21T10:28:34.357 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:34.357+0000 7ffa801c9740 1 -- start start 2024-03-21T10:28:34.358 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:34.358+0000 7ffa801c9740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5558b0a5a6d0 0x5558b0a62fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:28:34.358 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:34.358+0000 7ffa801c9740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5558b0a5b080 0x5558b0a63500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:28:34.358 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:34.358+0000 7ffa801c9740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5558b0a63c00 0x5558b0a6f400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:28:34.358 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:34.358+0000 7ffa801c9740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa5800fe10 con 0x5558b0a63c00 2024-03-21T10:28:34.358 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:34.358+0000 7ffa801c9740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa58011790 con 0x5558b0a5a6d0 2024-03-21T10:28:34.358 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:34.358+0000 7ffa801c9740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa58010470 con 0x5558b0a5b080 2024-03-21T10:28:34.358 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:34.358+0000 7ffa77200640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5558b0a63c00 0x5558b0a6f400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:28:34.358 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:34.358+0000 7ffa77200640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5558b0a63c00 0x5558b0a6f400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:60622/0 (socket says 172.21.15.47:60622) 2024-03-21T10:28:34.359 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:34.358+0000 7ffa77200640 1 -- 172.21.15.47:0/2488153155 learned_addr learned my addr 172.21.15.47:0/2488153155 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:28:34.359 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:34.359+0000 7ffa76800640 1 --2- 172.21.15.47:0/2488153155 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5558b0a5a6d0 0x5558b0a62fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:28:34.359 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:34.359+0000 7ffa77200640 1 -- 172.21.15.47:0/2488153155 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5558b0a5b080 msgr2=0x5558b0a63500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:28:34.359 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:34.359+0000 7ffa77200640 1 --2- 172.21.15.47:0/2488153155 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5558b0a5b080 0x5558b0a63500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:28:34.359 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:34.359+0000 7ffa77200640 1 -- 172.21.15.47:0/2488153155 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5558b0a5a6d0 msgr2=0x5558b0a62fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:28:34.360 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:34.359+0000 7ffa77200640 1 --2- 172.21.15.47:0/2488153155 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5558b0a5a6d0 0x5558b0a62fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:28:34.360 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:34.359+0000 7ffa77200640 1 -- 172.21.15.47:0/2488153155 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5558b0a5c470 con 0x5558b0a63c00 2024-03-21T10:28:34.360 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:34.359+0000 7ffa76800640 1 --2- 172.21.15.47:0/2488153155 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5558b0a5a6d0 0x5558b0a62fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T10:28:34.360 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:34.360+0000 7ffa77200640 1 --2- 172.21.15.47:0/2488153155 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5558b0a63c00 0x5558b0a6f400 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7ffa7000e660 tx=0x7ffa70009490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7f2a46916456a53e server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:28:34.360 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:34.360+0000 7ffa62a00640 1 -- 172.21.15.47:0/2488153155 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa70017050 con 0x5558b0a63c00 2024-03-21T10:28:34.360 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:34.360+0000 7ffa62a00640 1 -- 172.21.15.47:0/2488153155 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7ffa7000eda0 con 0x5558b0a63c00 2024-03-21T10:28:34.360 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:34.360+0000 7ffa62a00640 1 -- 172.21.15.47:0/2488153155 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa70013470 con 0x5558b0a63c00 2024-03-21T10:28:34.364 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:34.362+0000 7ffa801c9740 1 -- 172.21.15.47:0/2488153155 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5558b0a63c00 msgr2=0x5558b0a6f400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:28:34.364 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:34.362+0000 7ffa801c9740 1 --2- 172.21.15.47:0/2488153155 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5558b0a63c00 0x5558b0a6f400 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7ffa7000e660 tx=0x7ffa70009490 comp rx=0 tx=0).stop 2024-03-21T10:28:34.364 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:34.364+0000 7ffa801c9740 1 -- 172.21.15.47:0/2488153155 shutdown_connections 2024-03-21T10:28:34.364 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:34.364+0000 7ffa801c9740 1 --2- 172.21.15.47:0/2488153155 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5558b0a63c00 0x5558b0a6f400 unknown :-1 s=CLOSED pgs=749 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:28:34.364 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:34.364+0000 7ffa801c9740 1 --2- 172.21.15.47:0/2488153155 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5558b0a5b080 0x5558b0a63500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:28:34.364 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:34.364+0000 7ffa801c9740 1 --2- 172.21.15.47:0/2488153155 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5558b0a5a6d0 0x5558b0a62fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:28:34.364 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:34.364+0000 7ffa801c9740 1 -- 172.21.15.47:0/2488153155 >> 172.21.15.47:0/2488153155 conn(0x5558b0a54da0 msgr2=0x5558b0a53eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:28:34.370 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:34.368+0000 7ffa801c9740 1 -- 172.21.15.47:0/2488153155 shutdown_connections 2024-03-21T10:28:34.370 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:34.368+0000 7ffa801c9740 1 -- 172.21.15.47:0/2488153155 wait complete. 2024-03-21T10:28:34.370 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:34.368+0000 7ffa801c9740 1 -- start start 2024-03-21T10:28:34.370 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:34.369+0000 7ffa801c9740 1 Processor -- start 2024-03-21T10:28:34.372 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:34.373+0000 7ffa801c9740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5558b0a5a6d0 0x5558b0a6be40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:28:34.373 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:34.373+0000 7ffa801c9740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5558b0a5b080 0x5558b0a681c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:28:34.374 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:34.375+0000 7ffa75e00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5558b0a5b080 0x5558b0a681c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:28:34.374 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:34.375+0000 7ffa75e00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5558b0a5b080 0x5558b0a681c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.47:33256/0 (socket says 172.21.15.47:33256) 2024-03-21T10:28:34.374 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:34.375+0000 7ffa75e00640 1 -- 172.21.15.47:0/2812174669 learned_addr learned my addr 172.21.15.47:0/2812174669 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:28:34.374 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:34.375+0000 7ffa76800640 1 --2- 172.21.15.47:0/2812174669 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5558b0a5a6d0 0x5558b0a6be40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:28:34.375 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:34.375+0000 7ffa801c9740 1 --2- 172.21.15.47:0/2812174669 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5558b0a63c00 0x5558b0a688e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:28:34.375 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:34.375+0000 7ffa77200640 1 --2- 172.21.15.47:0/2812174669 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5558b0a63c00 0x5558b0a688e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:28:34.375 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:34.376+0000 7ffa801c9740 1 -- 172.21.15.47:0/2812174669 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa58011130 con 0x5558b0a63c00 2024-03-21T10:28:34.375 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:34.376+0000 7ffa801c9740 1 -- 172.21.15.47:0/2812174669 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa58010ad0 con 0x5558b0a5a6d0 2024-03-21T10:28:34.375 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:34.376+0000 7ffa801c9740 1 -- 172.21.15.47:0/2812174669 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa5800dfd0 con 0x5558b0a5b080 2024-03-21T10:28:34.376 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:34.376+0000 7ffa801c9740 10 client.0 _ll_register_callbacks cb 0x7ffa64009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T10:28:34.376 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:34.376+0000 7ffa75e00640 1 -- 172.21.15.47:0/2812174669 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5558b0a5a6d0 msgr2=0x5558b0a6be40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:28:34.376 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:34.376+0000 7ffa75e00640 1 --2- 172.21.15.47:0/2812174669 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5558b0a5a6d0 0x5558b0a6be40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:28:34.376 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:34.376+0000 7ffa75e00640 1 -- 172.21.15.47:0/2812174669 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5558b0a63c00 msgr2=0x5558b0a688e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:28:34.376 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:34.376+0000 7ffa75e00640 1 --2- 172.21.15.47:0/2812174669 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5558b0a63c00 0x5558b0a688e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:28:34.376 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:34.376+0000 7ffa75e00640 1 -- 172.21.15.47:0/2812174669 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5558b0ae0300 con 0x5558b0a5b080 2024-03-21T10:28:34.377 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:34.377+0000 7ffa77200640 1 --2- 172.21.15.47:0/2812174669 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5558b0a63c00 0x5558b0a688e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T10:28:34.377 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:34.377+0000 7ffa75e00640 1 --2- 172.21.15.47:0/2812174669 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5558b0a5b080 0x5558b0a681c0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7ffa64004ac0 tx=0x7ffa64004970 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5118286acdabc9b6 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:28:34.377 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:34.377+0000 7ffa76800640 1 --2- 172.21.15.47:0/2812174669 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5558b0a5a6d0 0x5558b0a6be40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T10:28:34.377 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:34.378+0000 7ffa57e00640 10 client.0 ms_handle_connect on v2:172.21.15.157:3300/0 2024-03-21T10:28:34.378 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:34.378+0000 7ffa57e00640 1 -- 172.21.15.47:0/2812174669 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa6404e070 con 0x5558b0a5b080 2024-03-21T10:28:34.378 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:34.378+0000 7ffa57e00640 1 -- 172.21.15.47:0/2812174669 <== mon.1 v2:172.21.15.157:3300/0 2 ==== osd_map(97..97 src has 1..97) v4 ==== 9474+0+0 (secure 0 0 0) 0x7ffa64005470 con 0x5558b0a5b080 2024-03-21T10:28:34.378 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:34.378+0000 7ffa801c9740 1 -- 172.21.15.47:0/2812174669 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ffa7000eda0 con 0x5558b0a5b080 2024-03-21T10:28:34.379 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:34.380+0000 7ffa57e00640 1 -- 172.21.15.47:0/2812174669 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7ffa6402c300 con 0x5558b0a5b080 2024-03-21T10:28:34.380 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:34.381+0000 7ffa801c9740 10 client.26359 fetch_fsmap learned FSMap version 32 2024-03-21T10:28:34.380 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:34.381+0000 7ffa801c9740 1 -- 172.21.15.47:0/2812174669 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x5558b0ae1400 con 0x5558b0a5b080 2024-03-21T10:28:34.381 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:34.381+0000 7ffa57e00640 1 -- 172.21.15.47:0/2812174669 <== mon.1 v2:172.21.15.157:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7ffa7000eda0 con 0x5558b0a5b080 2024-03-21T10:28:34.381 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:34.381+0000 7ffa57e00640 1 -- 172.21.15.47:0/2812174669 <== mon.1 v2:172.21.15.157:3300/0 5 ==== fsmap(e 32) v1 ==== 7942+0+0 (secure 0 0 0) 0x7ffa64003ca0 con 0x5558b0a5b080 2024-03-21T10:28:34.381 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:34.381+0000 7ffa801c9740 10 client.26359 fetch_fsmap finished waiting for FSMap version 32 2024-03-21T10:28:34.381 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:34.381+0000 7ffa801c9740 10 client.26359 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-03-21T10:28:34.381 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:34.381+0000 7ffa801c9740 20 client.26359 populate_metadata read hostname 'smithi047' 2024-03-21T10:28:34.381 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:34.381+0000 7ffa801c9740 1 --2- 172.21.15.47:0/2812174669 >> [v2:172.21.15.100:6836/3497644162,v1:172.21.15.100:6837/3497644162] conn(0x5558b0ae1b50 0x5558b0ae3f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:28:34.382 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:34.381+0000 7ffa801c9740 4 client.26359 mds_command: new command op to 34493 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-21T10:28:34.382 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:34.381+0000 7ffa801c9740 1 -- 172.21.15.47:0/2812174669 --> [v2:172.21.15.100:6836/3497644162,v1:172.21.15.100:6837/3497644162] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x5558b09c5b50 con 0x5558b0ae1b50 2024-03-21T10:28:34.386 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:34.386+0000 7ffa76800640 1 --2- 172.21.15.47:0/2812174669 >> [v2:172.21.15.100:6836/3497644162,v1:172.21.15.100:6837/3497644162] conn(0x5558b0ae1b50 0x5558b0ae3f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:28:34.388 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:34.389+0000 7ffa76800640 1 --2- 172.21.15.47:0/2812174669 >> [v2:172.21.15.100:6836/3497644162,v1:172.21.15.100:6837/3497644162] conn(0x5558b0ae1b50 0x5558b0ae3f70 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=3f00b91514ec44a4 server_cookie=10a0553559f354ac in_seq=0 out_seq=0 2024-03-21T10:28:34.389 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:34.390+0000 7ffa57e00640 10 client.26359 ms_handle_connect on v2:172.21.15.100:6836/3497644162 2024-03-21T10:28:34.389 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:34.390+0000 7ffa57e00640 1 -- 172.21.15.47:0/2812174669 <== mds.1 v2:172.21.15.100:6836/3497644162 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x5558b09c5b50 con 0x5558b0ae1b50 2024-03-21T10:28:34.389 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:34.390+0000 7ffa57e00640 10 client.26359 handle_command_reply: tid=0 2024-03-21T10:28:34.389 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:34.390+0000 7ffa801c9740 1 client.26359 shutdown 2024-03-21T10:28:34.390 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:34.391+0000 7ffa801c9740 1 -- 172.21.15.47:0/2812174669 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5558b0a5b080 msgr2=0x5558b0a681c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:28:34.390 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:34.391+0000 7ffa801c9740 1 --2- 172.21.15.47:0/2812174669 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5558b0a5b080 0x5558b0a681c0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7ffa64004ac0 tx=0x7ffa64004970 comp rx=0 tx=0).stop 2024-03-21T10:28:34.392 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:34.393+0000 7ffa801c9740 1 -- 172.21.15.47:0/2812174669 shutdown_connections 2024-03-21T10:28:34.392 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:34.393+0000 7ffa801c9740 1 --2- 172.21.15.47:0/2812174669 >> [v2:172.21.15.100:6836/3497644162,v1:172.21.15.100:6837/3497644162] conn(0x5558b0ae1b50 0x5558b0ae3f70 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:28:34.392 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:34.393+0000 7ffa801c9740 1 --2- 172.21.15.47:0/2812174669 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5558b0a63c00 0x5558b0a688e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:28:34.392 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:34.393+0000 7ffa801c9740 1 --2- 172.21.15.47:0/2812174669 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5558b0a5b080 0x5558b0a681c0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:28:34.392 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:34.393+0000 7ffa801c9740 1 --2- 172.21.15.47:0/2812174669 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5558b0a5a6d0 0x5558b0a6be40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:28:34.393 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:34.393+0000 7ffa801c9740 1 -- 172.21.15.47:0/2812174669 >> 172.21.15.47:0/2812174669 conn(0x5558b0a54da0 msgr2=0x5558b0a53620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:28:34.393 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:34.393+0000 7ffa57e00640 0 client.26359 ms_handle_reset on v2:172.21.15.100:6836/3497644162 2024-03-21T10:28:34.393 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:34.394+0000 7ffa801c9740 1 -- 172.21.15.47:0/2812174669 shutdown_connections 2024-03-21T10:28:34.393 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:34.394+0000 7ffa801c9740 1 -- 172.21.15.47:0/2812174669 wait complete. 2024-03-21T10:28:34.394 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:34.394+0000 7ffa801c9740 20 client.26359 trim_cache size 0 max 16384 2024-03-21T10:28:34.394 INFO:teuthology.orchestra.run.smithi047.stdout:[] 2024-03-21T10:28:34.397 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:34.398+0000 7ffa7d400640 1 -- 172.21.15.47:0/2721693983 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7ffa5c0a3110 msgr2=0x7ffa5c0a55d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:28:34.398 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:34.398+0000 7ffa7d400640 1 --2- 172.21.15.47:0/2721693983 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7ffa5c0a3110 0x7ffa5c0a55d0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7ffa78108450 tx=0x7ffa64000bc0 comp rx=0 tx=0).stop 2024-03-21T10:28:34.398 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:34.398+0000 7ffa7d400640 1 -- 172.21.15.47:0/2721693983 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ffa780fff20 msgr2=0x7ffa78106c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:28:34.398 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:34.398+0000 7ffa7d400640 1 --2- 172.21.15.47:0/2721693983 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ffa780fff20 0x7ffa78106c40 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7ffa6800be20 tx=0x7ffa68009f70 comp rx=0 tx=0).stop 2024-03-21T10:28:34.398 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:34.398+0000 7ffa7d400640 1 -- 172.21.15.47:0/2721693983 shutdown_connections 2024-03-21T10:28:34.398 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:34.398+0000 7ffa7d400640 1 --2- 172.21.15.47:0/2721693983 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ffa781076c0 0x7ffa781aaa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:28:34.398 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:34.398+0000 7ffa7d400640 1 --2- 172.21.15.47:0/2721693983 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7ffa5c0a3110 0x7ffa5c0a55d0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:28:34.398 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:34.398+0000 7ffa7d400640 1 --2- 172.21.15.47:0/2721693983 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ffa78104ff0 0x7ffa78107180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:28:34.399 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:34.398+0000 7ffa7d400640 1 --2- 172.21.15.47:0/2721693983 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ffa780fff20 0x7ffa78106c40 secure :-1 s=CLOSED pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7ffa6800be20 tx=0x7ffa68009f70 comp rx=0 tx=0).stop 2024-03-21T10:28:34.399 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:34.399+0000 7ffa7d400640 1 -- 172.21.15.47:0/2721693983 >> 172.21.15.47:0/2721693983 conn(0x7ffa78076ac0 msgr2=0x7ffa7810c680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:28:34.399 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:34.399+0000 7ffa7d400640 1 -- 172.21.15.47:0/2721693983 shutdown_connections 2024-03-21T10:28:34.399 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:34.400+0000 7ffa7d400640 1 -- 172.21.15.47:0/2721693983 wait complete. 2024-03-21T10:28:34.444 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-21T10:28:34.445 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-03-21T10:28:34.479 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:28:34 smithi100 ceph-mon[35193]: from='mgr.15687 172.21.15.47:0/8730661' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T10:28:34.488 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:28:34 smithi157 ceph-mon[35581]: from='mgr.15687 172.21.15.47:0/8730661' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T10:28:34.816 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:34.817+0000 7efcb4a00640 1 -- 172.21.15.47:0/1120025833 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efca802cb90 con 0x7efcb8101630 2024-03-21T10:28:34.817 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:34.817+0000 7efcbcc00640 1 -- 172.21.15.47:0/1120025833 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7efcb8101630 msgr2=0x7efcb8101ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:28:34.817 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:34.817+0000 7efcbcc00640 1 --2- 172.21.15.47:0/1120025833 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7efcb8101630 0x7efcb8101ab0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7efca8009900 tx=0x7efca802bc40 comp rx=0 tx=0).stop 2024-03-21T10:28:34.817 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:34.818+0000 7efcbcc00640 1 -- 172.21.15.47:0/1120025833 shutdown_connections 2024-03-21T10:28:34.818 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:34.818+0000 7efcbcc00640 1 --2- 172.21.15.47:0/1120025833 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7efcb8101ff0 0x7efcb810e500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:28:34.818 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:34.818+0000 7efcbcc00640 1 --2- 172.21.15.47:0/1120025833 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7efcb8101630 0x7efcb8101ab0 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:28:34.818 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:34.818+0000 7efcbcc00640 1 --2- 172.21.15.47:0/1120025833 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7efcb8107640 0x7efcb8107a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:28:34.818 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:34.818+0000 7efcbcc00640 1 -- 172.21.15.47:0/1120025833 >> 172.21.15.47:0/1120025833 conn(0x7efcb8078aa0 msgr2=0x7efcb8078eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:28:34.818 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:34.819+0000 7efcbcc00640 1 -- 172.21.15.47:0/1120025833 shutdown_connections 2024-03-21T10:28:34.820 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:34.819+0000 7efcbcc00640 1 -- 172.21.15.47:0/1120025833 wait complete. 2024-03-21T10:28:34.821 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:34.821+0000 7efcbcc00640 1 Processor -- start 2024-03-21T10:28:34.822 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:34.822+0000 7efcbcc00640 1 -- start start 2024-03-21T10:28:34.822 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:34.822+0000 7efcbcc00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7efcb8101630 0x7efcb810e210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:28:34.822 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:34.822+0000 7efcbcc00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7efcb8101ff0 0x7efcb810b300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:28:34.822 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:34.823+0000 7efcbcc00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7efcb8107640 0x7efcb810b840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:28:34.823 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:34.824+0000 7efcbcc00640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efcb80752e0 con 0x7efcb8107640 2024-03-21T10:28:34.823 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:34.824+0000 7efcbcc00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efcb8075160 con 0x7efcb8101ff0 2024-03-21T10:28:34.824 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:34.824+0000 7efcbcc00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efcb8075460 con 0x7efcb8101630 2024-03-21T10:28:34.824 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:34.823+0000 7efcb5e00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7efcb8101630 0x7efcb810e210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:28:34.824 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:34.824+0000 7efcb5400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7efcb8101ff0 0x7efcb810b300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:28:34.824 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:34.824+0000 7efcb5e00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7efcb8101630 0x7efcb810e210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.47:33262/0 (socket says 172.21.15.47:33262) 2024-03-21T10:28:34.824 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:34.824+0000 7efcb5400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7efcb8101ff0 0x7efcb810b300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:54042/0 (socket says 172.21.15.47:54042) 2024-03-21T10:28:34.824 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:34.824+0000 7efcb5e00640 1 -- 172.21.15.47:0/3982878353 learned_addr learned my addr 172.21.15.47:0/3982878353 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:28:34.824 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:34.825+0000 7efcb6800640 1 --2- 172.21.15.47:0/3982878353 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7efcb8107640 0x7efcb810b840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:28:34.825 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:34.825+0000 7efcb5e00640 1 -- 172.21.15.47:0/3982878353 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7efcb8101ff0 msgr2=0x7efcb810b300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:28:34.825 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:34.825+0000 7efcb5e00640 1 --2- 172.21.15.47:0/3982878353 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7efcb8101ff0 0x7efcb810b300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:28:34.825 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:34.825+0000 7efcb5e00640 1 -- 172.21.15.47:0/3982878353 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7efcb8107640 msgr2=0x7efcb810b840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:28:34.825 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:34.825+0000 7efcb5e00640 1 --2- 172.21.15.47:0/3982878353 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7efcb8107640 0x7efcb810b840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:28:34.825 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:34.825+0000 7efcb5e00640 1 -- 172.21.15.47:0/3982878353 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efcb810bfc0 con 0x7efcb8101630 2024-03-21T10:28:34.826 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:34.827+0000 7efcb5e00640 1 --2- 172.21.15.47:0/3982878353 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7efcb8101630 0x7efcb810e210 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7efca000c920 tx=0x7efca000cde0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:28:34.828 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:34.827+0000 7efcaea00640 1 -- 172.21.15.47:0/3982878353 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efca0007c70 con 0x7efcb8101630 2024-03-21T10:28:34.830 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:34.827+0000 7efcbcc00640 1 -- 172.21.15.47:0/3982878353 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efcb8072110 con 0x7efcb8101630 2024-03-21T10:28:34.830 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:34.827+0000 7efcbcc00640 1 -- 172.21.15.47:0/3982878353 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efcb8072650 con 0x7efcb8101630 2024-03-21T10:28:34.830 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:34.830+0000 7efcaea00640 1 -- 172.21.15.47:0/3982878353 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7efca0007e10 con 0x7efcb8101630 2024-03-21T10:28:34.830 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:34.830+0000 7efcaea00640 1 -- 172.21.15.47:0/3982878353 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efca00187c0 con 0x7efcb8101630 2024-03-21T10:28:34.835 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:34.835+0000 7efcbf86e740 1 Processor -- start 2024-03-21T10:28:34.835 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:34.836+0000 7efcbf86e740 1 -- start start 2024-03-21T10:28:34.835 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:34.836+0000 7efcbf86e740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f7eb7dc6d0 0x55f7eb7dcab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:28:34.836 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:34.836+0000 7efcbf86e740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f7eb7dd080 0x55f7eb7e5630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:28:34.836 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:34.836+0000 7efcbf86e740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f7eb7e5c00 0x55f7eb7e8050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:28:34.836 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:34.836+0000 7efcbf86e740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f7eb7402c0 con 0x55f7eb7dd080 2024-03-21T10:28:34.836 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:34.836+0000 7efcbf86e740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f7eb6e8c90 con 0x55f7eb7dc6d0 2024-03-21T10:28:34.836 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:34.836+0000 7efcbf86e740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f7eb73c360 con 0x55f7eb7e5c00 2024-03-21T10:28:34.836 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:34.837+0000 7efcb5400640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f7eb7e5c00 0x55f7eb7e8050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:28:34.837 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:34.837+0000 7efcb6800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f7eb7dc6d0 0x55f7eb7dcab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:28:34.837 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:34.837+0000 7efcb6800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f7eb7dc6d0 0x55f7eb7dcab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:54044/0 (socket says 172.21.15.47:54044) 2024-03-21T10:28:34.837 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:34.837+0000 7efcb6800640 1 -- 172.21.15.47:0/690493152 learned_addr learned my addr 172.21.15.47:0/690493152 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:28:34.837 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:34.837+0000 7efcaea00640 1 -- 172.21.15.47:0/3982878353 <== mon.1 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 27) v1 ==== 137985+0+0 (secure 0 0 0) 0x7efca0018960 con 0x7efcb8101630 2024-03-21T10:28:34.838 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:34.838+0000 7efcb6800640 1 -- 172.21.15.47:0/690493152 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f7eb7e5c00 msgr2=0x55f7eb7e8050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:28:34.838 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:34.838+0000 7efcb6800640 1 --2- 172.21.15.47:0/690493152 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f7eb7e5c00 0x55f7eb7e8050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:28:34.838 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:34.838+0000 7efcb6800640 1 -- 172.21.15.47:0/690493152 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f7eb7dd080 msgr2=0x55f7eb7e5630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:28:34.838 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:34.838+0000 7efcb6800640 1 --2- 172.21.15.47:0/690493152 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f7eb7dd080 0x55f7eb7e5630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:28:34.838 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:34.838+0000 7efcb6800640 1 -- 172.21.15.47:0/690493152 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f7eb70d080 con 0x55f7eb7dc6d0 2024-03-21T10:28:34.839 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:34.839+0000 7efcb6800640 1 --2- 172.21.15.47:0/690493152 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f7eb7dc6d0 0x55f7eb7dcab0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7efcb0008dc0 tx=0x7efcb000e720 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d3df319c74ead68e server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:28:34.839 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:34.840+0000 7efcaea00640 1 --2- 172.21.15.47:0/3982878353 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7efc9c0a3190 0x7efc9c0a5650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:28:34.840 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:34.840+0000 7efcaea00640 1 -- 172.21.15.47:0/3982878353 <== mon.1 v2:172.21.15.157:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 9474+0+0 (secure 0 0 0) 0x7efca00cfba0 con 0x7efcb8101630 2024-03-21T10:28:34.840 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:34.841+0000 7efca7000640 1 -- 172.21.15.47:0/690493152 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efcb0014070 con 0x55f7eb7dc6d0 2024-03-21T10:28:34.840 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:34.841+0000 7efca7000640 1 -- 172.21.15.47:0/690493152 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7efcb0003b70 con 0x55f7eb7dc6d0 2024-03-21T10:28:34.840 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:34.841+0000 7efca7000640 1 -- 172.21.15.47:0/690493152 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efcb00107f0 con 0x55f7eb7dc6d0 2024-03-21T10:28:34.841 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:34.842+0000 7efcbf86e740 1 -- 172.21.15.47:0/690493152 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f7eb7dc6d0 msgr2=0x55f7eb7dcab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:28:34.841 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:34.842+0000 7efcbf86e740 1 --2- 172.21.15.47:0/690493152 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f7eb7dc6d0 0x55f7eb7dcab0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7efcb0008dc0 tx=0x7efcb000e720 comp rx=0 tx=0).stop 2024-03-21T10:28:34.841 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:34.842+0000 7efcb5e00640 1 --2- 172.21.15.47:0/3982878353 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7efc9c0a3190 0x7efc9c0a5650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:28:34.842 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:34.842+0000 7efcbf86e740 1 -- 172.21.15.47:0/690493152 shutdown_connections 2024-03-21T10:28:34.843 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:34.843+0000 7efcbf86e740 1 --2- 172.21.15.47:0/690493152 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f7eb7e5c00 0x55f7eb7e8050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:28:34.843 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:34.843+0000 7efcbf86e740 1 --2- 172.21.15.47:0/690493152 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f7eb7dd080 0x55f7eb7e5630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:28:34.843 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:34.843+0000 7efcbf86e740 1 --2- 172.21.15.47:0/690493152 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f7eb7dc6d0 0x55f7eb7dcab0 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:28:34.843 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:34.843+0000 7efcbf86e740 1 -- 172.21.15.47:0/690493152 >> 172.21.15.47:0/690493152 conn(0x55f7eb7d6da0 msgr2=0x55f7eb7d7180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:28:34.843 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:34.843+0000 7efcb5e00640 1 --2- 172.21.15.47:0/3982878353 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7efc9c0a3190 0x7efc9c0a5650 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x55f7eb7dc1a0 tx=0x7efca00cfdf0 comp rx=0 tx=0).ready entity=mgr.15687 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:28:34.844 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:34.845+0000 7efcbf86e740 1 -- 172.21.15.47:0/690493152 shutdown_connections 2024-03-21T10:28:34.845 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:34.845+0000 7efcbf86e740 1 -- 172.21.15.47:0/690493152 wait complete. 2024-03-21T10:28:34.845 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:34.845+0000 7efcbf86e740 1 -- start start 2024-03-21T10:28:34.845 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:34.846+0000 7efcbf86e740 1 Processor -- start 2024-03-21T10:28:34.847 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:34.847+0000 7efcbf86e740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f7eb7dc6d0 0x55f7eb7e5490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:28:34.847 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:34.848+0000 7efcbf86e740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f7eb7dd080 0x55f7eb7e2820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:28:34.848 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:34.848+0000 7efcbf86e740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f7eb7e5c00 0x55f7eb7e2d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:28:34.848 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:34.848+0000 7efcbf86e740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f7eb7e8b70 con 0x55f7eb7dc6d0 2024-03-21T10:28:34.849 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:34.848+0000 7efcbf86e740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f7eb6fc0d0 con 0x55f7eb7dd080 2024-03-21T10:28:34.849 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:34.848+0000 7efcbf86e740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f7eb7e8cf0 con 0x55f7eb7e5c00 2024-03-21T10:28:34.849 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:34.848+0000 7efcbf86e740 10 client.0 _ll_register_callbacks cb 0x55f7eb700e80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T10:28:34.849 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:34.849+0000 7efcb5400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f7eb7dd080 0x55f7eb7e2820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:28:34.849 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:34.849+0000 7efcb5400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f7eb7dd080 0x55f7eb7e2820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:54056/0 (socket says 172.21.15.47:54056) 2024-03-21T10:28:34.849 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:34.849+0000 7efcb5400640 1 -- 172.21.15.47:0/4246705989 learned_addr learned my addr 172.21.15.47:0/4246705989 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:28:34.849 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:34.849+0000 7efcb6800640 1 --2- 172.21.15.47:0/4246705989 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f7eb7dc6d0 0x55f7eb7e5490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:28:34.850 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:34.850+0000 7efcb6800640 1 --2- 172.21.15.47:0/4246705989 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f7eb7e5c00 0x55f7eb7e2d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:28:34.850 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:34.851+0000 7efcb5400640 1 -- 172.21.15.47:0/4246705989 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f7eb7e5c00 msgr2=0x55f7eb7e2d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:28:34.850 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:34.851+0000 7efcb5400640 1 --2- 172.21.15.47:0/4246705989 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f7eb7e5c00 0x55f7eb7e2d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:28:34.851 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:34.851+0000 7efcb5400640 1 -- 172.21.15.47:0/4246705989 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f7eb7dc6d0 msgr2=0x55f7eb7e5490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:28:34.851 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:34.851+0000 7efcb5400640 1 --2- 172.21.15.47:0/4246705989 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f7eb7dc6d0 0x55f7eb7e5490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:28:34.851 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:34.851+0000 7efcb5400640 1 -- 172.21.15.47:0/4246705989 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55f7eb4543f0 con 0x55f7eb7dd080 2024-03-21T10:28:34.851 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:34.851+0000 7efcb6800640 1 --2- 172.21.15.47:0/4246705989 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f7eb7e5c00 0x55f7eb7e2d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T10:28:34.851 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:34.852+0000 7efcb5400640 1 --2- 172.21.15.47:0/4246705989 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f7eb7dd080 0x55f7eb7e2820 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7efca8035000 tx=0x7efca80058b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=29b87f8678975e79 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:28:34.851 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:34.852+0000 7efc9b400640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-21T10:28:34.852 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:34.853+0000 7efc9b400640 1 -- 172.21.15.47:0/4246705989 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efca8044070 con 0x55f7eb7dd080 2024-03-21T10:28:34.853 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:34.853+0000 7efc9b400640 1 -- 172.21.15.47:0/4246705989 <== mon.2 v2:172.21.15.100:3300/0 2 ==== osd_map(97..97 src has 1..97) v4 ==== 9474+0+0 (secure 0 0 0) 0x7efca8003800 con 0x55f7eb7dd080 2024-03-21T10:28:34.853 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:34.853+0000 7efcbf86e740 1 -- 172.21.15.47:0/4246705989 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7efcb0003b70 con 0x55f7eb7dd080 2024-03-21T10:28:34.853 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:34.854+0000 7efc9b400640 1 -- 172.21.15.47:0/4246705989 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7efca8050050 con 0x55f7eb7dd080 2024-03-21T10:28:34.853 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:34.854+0000 7efcbf86e740 10 client.36599 fetch_fsmap learned FSMap version 32 2024-03-21T10:28:34.854 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:34.854+0000 7efcbf86e740 1 -- 172.21.15.47:0/4246705989 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x55f7eb51e7d0 con 0x55f7eb7dd080 2024-03-21T10:28:34.854 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:34.855+0000 7efc9b400640 1 -- 172.21.15.47:0/4246705989 <== mon.2 v2:172.21.15.100:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7efcb0003b70 con 0x55f7eb7dd080 2024-03-21T10:28:34.855 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:34.856+0000 7efc9b400640 1 -- 172.21.15.47:0/4246705989 <== mon.2 v2:172.21.15.100:3300/0 5 ==== fsmap(e 32) v1 ==== 7942+0+0 (secure 0 0 0) 0x7efca8034900 con 0x55f7eb7dd080 2024-03-21T10:28:34.856 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:34.856+0000 7efcbf86e740 10 client.36599 fetch_fsmap finished waiting for FSMap version 32 2024-03-21T10:28:34.856 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:34.856+0000 7efcbf86e740 10 client.36599 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-21T10:28:34.856 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:34.856+0000 7efcbf86e740 20 client.36599 populate_metadata read hostname 'smithi047' 2024-03-21T10:28:34.856 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:34.856+0000 7efcbf86e740 1 --2- 172.21.15.47:0/4246705989 >> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] conn(0x55f7eb7f5a40 0x55f7eb7f7e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:28:34.856 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:34.857+0000 7efcbf86e740 4 client.36599 mds_command: new command op to 34535 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T10:28:34.857 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:34.857+0000 7efcbf86e740 1 -- 172.21.15.47:0/4246705989 --> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55f7eb74d800 con 0x55f7eb7f5a40 2024-03-21T10:28:34.857 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:34.857+0000 7efcb6800640 1 --2- 172.21.15.47:0/4246705989 >> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] conn(0x55f7eb7f5a40 0x55f7eb7f7e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:28:34.858 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:34.859+0000 7efcb6800640 1 --2- 172.21.15.47:0/4246705989 >> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] conn(0x55f7eb7f5a40 0x55f7eb7f7e80 crc :-1 s=READY pgs=115 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=867e7eeb1b85466e server_cookie=98deea85c274c516 in_seq=0 out_seq=0 2024-03-21T10:28:34.858 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:34.859+0000 7efc9b400640 10 client.36599 ms_handle_connect on v2:172.21.15.100:6838/4173168440 2024-03-21T10:28:34.862 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:34.862+0000 7efc9b400640 1 -- 172.21.15.47:0/4246705989 <== mds.0 v2:172.21.15.100:6838/4173168440 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55f7eb74d800 con 0x55f7eb7f5a40 2024-03-21T10:28:34.862 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:34.862+0000 7efc9b400640 10 client.36599 handle_command_reply: tid=0 2024-03-21T10:28:34.862 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:34.862+0000 7efcbf86e740 1 client.36599 shutdown 2024-03-21T10:28:34.862 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:34.863+0000 7efcbf86e740 1 -- 172.21.15.47:0/4246705989 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f7eb7dd080 msgr2=0x55f7eb7e2820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:28:34.862 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:34.863+0000 7efcbf86e740 1 --2- 172.21.15.47:0/4246705989 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f7eb7dd080 0x55f7eb7e2820 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7efca8035000 tx=0x7efca80058b0 comp rx=0 tx=0).stop 2024-03-21T10:28:34.864 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:34.865+0000 7efcbf86e740 1 -- 172.21.15.47:0/4246705989 shutdown_connections 2024-03-21T10:28:34.865 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:34.865+0000 7efcbf86e740 1 --2- 172.21.15.47:0/4246705989 >> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] conn(0x55f7eb7f5a40 0x55f7eb7f7e80 crc :-1 s=READY pgs=115 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:28:34.865 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:34.865+0000 7efcbf86e740 1 --2- 172.21.15.47:0/4246705989 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f7eb7e5c00 0x55f7eb7e2d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:28:34.865 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:34.865+0000 7efcbf86e740 1 --2- 172.21.15.47:0/4246705989 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f7eb7dd080 0x55f7eb7e2820 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:28:34.865 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:34.865+0000 7efcbf86e740 1 --2- 172.21.15.47:0/4246705989 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f7eb7dc6d0 0x55f7eb7e5490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:28:34.865 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:34.865+0000 7efcbf86e740 1 -- 172.21.15.47:0/4246705989 >> 172.21.15.47:0/4246705989 conn(0x55f7eb7d6da0 msgr2=0x55f7eb7d8af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:28:34.865 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:34.865+0000 7efc9b400640 0 client.36599 ms_handle_reset on v2:172.21.15.100:6838/4173168440 2024-03-21T10:28:34.865 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:34.866+0000 7efcbf86e740 1 -- 172.21.15.47:0/4246705989 shutdown_connections 2024-03-21T10:28:34.866 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:34.866+0000 7efcbf86e740 1 -- 172.21.15.47:0/4246705989 wait complete. 2024-03-21T10:28:34.866 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:34.866+0000 7efcbf86e740 20 client.36599 trim_cache size 0 max 16384 2024-03-21T10:28:34.898 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:34.898+0000 7efcbf86e740 1 Processor -- start 2024-03-21T10:28:34.898 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:34.898+0000 7efcbf86e740 1 -- start start 2024-03-21T10:28:34.898 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:34.899+0000 7efcbf86e740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f7eb7dc6d0 0x55f7eb7e4f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:28:34.898 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:34.899+0000 7efcbf86e740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f7eb7dd080 0x55f7eb7e5490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:28:34.898 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:34.899+0000 7efcbf86e740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f7eb7e5c00 0x55f7eb7dfac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:28:34.898 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:34.899+0000 7efcbf86e740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc9400fc10 con 0x55f7eb7dd080 2024-03-21T10:28:34.899 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:34.899+0000 7efcbf86e740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc94011590 con 0x55f7eb7dc6d0 2024-03-21T10:28:34.899 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:34.899+0000 7efcbf86e740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc94010270 con 0x55f7eb7e5c00 2024-03-21T10:28:34.899 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:34.899+0000 7efcb6800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f7eb7dc6d0 0x55f7eb7e4f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:28:34.899 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:34.899+0000 7efcb5400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f7eb7dd080 0x55f7eb7e5490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:28:34.899 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:34.899+0000 7efcb5400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f7eb7dd080 0x55f7eb7e5490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:60654/0 (socket says 172.21.15.47:60654) 2024-03-21T10:28:34.900 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:34.900+0000 7efcb5400640 1 -- 172.21.15.47:0/2149210830 learned_addr learned my addr 172.21.15.47:0/2149210830 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:28:34.900 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:34.900+0000 7efcb6800640 1 --2- 172.21.15.47:0/2149210830 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f7eb7e5c00 0x55f7eb7dfac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:28:34.900 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:34.900+0000 7efcb5400640 1 -- 172.21.15.47:0/2149210830 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f7eb7e5c00 msgr2=0x55f7eb7dfac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:28:34.900 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:34.900+0000 7efcb5400640 1 --2- 172.21.15.47:0/2149210830 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f7eb7e5c00 0x55f7eb7dfac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:28:34.900 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:34.900+0000 7efcb5400640 1 -- 172.21.15.47:0/2149210830 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f7eb7dc6d0 msgr2=0x55f7eb7e4f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:28:34.900 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:34.900+0000 7efcb5400640 1 --2- 172.21.15.47:0/2149210830 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f7eb7dc6d0 0x55f7eb7e4f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:28:34.900 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:34.900+0000 7efcb5400640 1 -- 172.21.15.47:0/2149210830 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f7eb7d65c0 con 0x55f7eb7dd080 2024-03-21T10:28:34.901 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:34.901+0000 7efcb5400640 1 --2- 172.21.15.47:0/2149210830 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f7eb7dd080 0x55f7eb7e5490 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7efca8036040 tx=0x7efca802c360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4d0c9cb4b9d6ff4e server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:28:34.901 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:34.901+0000 7efca6600640 1 -- 172.21.15.47:0/2149210830 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efca8044070 con 0x55f7eb7dd080 2024-03-21T10:28:34.901 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:34.901+0000 7efca6600640 1 -- 172.21.15.47:0/2149210830 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7efca804d070 con 0x55f7eb7dd080 2024-03-21T10:28:34.901 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:34.901+0000 7efca6600640 1 -- 172.21.15.47:0/2149210830 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efca80346a0 con 0x55f7eb7dd080 2024-03-21T10:28:34.905 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:34.902+0000 7efcbf86e740 1 -- 172.21.15.47:0/2149210830 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f7eb7dd080 msgr2=0x55f7eb7e5490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:28:34.905 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:34.902+0000 7efcbf86e740 1 --2- 172.21.15.47:0/2149210830 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f7eb7dd080 0x55f7eb7e5490 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7efca8036040 tx=0x7efca802c360 comp rx=0 tx=0).stop 2024-03-21T10:28:34.905 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:34.902+0000 7efcbf86e740 1 -- 172.21.15.47:0/2149210830 shutdown_connections 2024-03-21T10:28:34.905 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:34.902+0000 7efcbf86e740 1 --2- 172.21.15.47:0/2149210830 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f7eb7e5c00 0x55f7eb7dfac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:28:34.905 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:34.902+0000 7efcbf86e740 1 --2- 172.21.15.47:0/2149210830 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f7eb7dd080 0x55f7eb7e5490 secure :-1 s=CLOSED pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7efca8036040 tx=0x7efca802c360 comp rx=0 tx=0).stop 2024-03-21T10:28:34.905 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:34.902+0000 7efcbf86e740 1 --2- 172.21.15.47:0/2149210830 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f7eb7dc6d0 0x55f7eb7e4f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:28:34.905 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:34.902+0000 7efcbf86e740 1 -- 172.21.15.47:0/2149210830 >> 172.21.15.47:0/2149210830 conn(0x55f7eb7d6da0 msgr2=0x55f7eb7d5eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:28:34.905 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:34.902+0000 7efcbf86e740 1 -- 172.21.15.47:0/2149210830 shutdown_connections 2024-03-21T10:28:34.905 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:34.903+0000 7efcbf86e740 1 -- 172.21.15.47:0/2149210830 wait complete. 2024-03-21T10:28:34.905 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:34.903+0000 7efcbf86e740 1 -- start start 2024-03-21T10:28:34.906 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:34.903+0000 7efcbf86e740 1 Processor -- start 2024-03-21T10:28:34.906 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:34.904+0000 7efcbf86e740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f7eb7dc6d0 0x55f7eb7edd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:28:34.906 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:34.904+0000 7efcbf86e740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f7eb7e5c00 0x55f7eb7ea060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:28:34.906 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:34.904+0000 7efcbf86e740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f7eb7ea5a0 0x55f7eb7eaa00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:28:34.906 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:34.904+0000 7efcbf86e740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc94010f30 con 0x55f7eb7dc6d0 2024-03-21T10:28:34.906 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:34.904+0000 7efcbf86e740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc940108d0 con 0x55f7eb7ea5a0 2024-03-21T10:28:34.906 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:34.904+0000 7efcbf86e740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc9400de80 con 0x55f7eb7e5c00 2024-03-21T10:28:34.906 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:34.904+0000 7efcbf86e740 10 client.0 _ll_register_callbacks cb 0x7efca8009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T10:28:34.906 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:34.905+0000 7efcb6800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f7eb7ea5a0 0x55f7eb7eaa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:28:34.906 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:34.905+0000 7efcb6800640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f7eb7dc6d0 0x55f7eb7edd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:28:34.906 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:34.905+0000 7efcb6800640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f7eb7dc6d0 0x55f7eb7edd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:60662/0 (socket says 172.21.15.47:60662) 2024-03-21T10:28:34.906 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:34.905+0000 7efcb6800640 1 -- 172.21.15.47:0/811679101 learned_addr learned my addr 172.21.15.47:0/811679101 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:28:34.907 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:34.906+0000 7efcb5400640 1 --2- 172.21.15.47:0/811679101 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f7eb7e5c00 0x55f7eb7ea060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:28:34.907 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:34.906+0000 7efcb6800640 1 -- 172.21.15.47:0/811679101 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f7eb7e5c00 msgr2=0x55f7eb7ea060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:28:34.907 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:34.907+0000 7efcb6800640 1 --2- 172.21.15.47:0/811679101 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f7eb7e5c00 0x55f7eb7ea060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:28:34.907 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:34.907+0000 7efcb6800640 1 -- 172.21.15.47:0/811679101 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f7eb7ea5a0 msgr2=0x55f7eb7eaa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:28:34.907 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:34.907+0000 7efcb6800640 1 --2- 172.21.15.47:0/811679101 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f7eb7ea5a0 0x55f7eb7eaa00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:28:34.908 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:34.907+0000 7efcb6800640 1 -- 172.21.15.47:0/811679101 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55f7eb8734b0 con 0x55f7eb7dc6d0 2024-03-21T10:28:34.908 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:34.908+0000 7efcb6800640 1 --2- 172.21.15.47:0/811679101 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f7eb7dc6d0 0x55f7eb7edd80 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7efcb0000f80 tx=0x7efcb0005bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3b8fb9fd43b93c2c server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:28:34.908 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:34.908+0000 7efc9b400640 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-03-21T10:28:34.909 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:34.910+0000 7efc9b400640 1 -- 172.21.15.47:0/811679101 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efcb0014040 con 0x55f7eb7dc6d0 2024-03-21T10:28:34.909 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:34.910+0000 7efc9b400640 1 -- 172.21.15.47:0/811679101 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(97..97 src has 1..97) v4 ==== 9474+0+0 (secure 0 0 0) 0x7efcb0020040 con 0x55f7eb7dc6d0 2024-03-21T10:28:34.910 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:34.910+0000 7efcbf86e740 1 -- 172.21.15.47:0/811679101 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7efca804d070 con 0x55f7eb7dc6d0 2024-03-21T10:28:34.916 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:34.914+0000 7efc9b400640 1 -- 172.21.15.47:0/811679101 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7efcb0029ab0 con 0x55f7eb7dc6d0 2024-03-21T10:28:34.917 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:34.917+0000 7efcbf86e740 10 client.16797 fetch_fsmap learned FSMap version 32 2024-03-21T10:28:34.917 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:34.917+0000 7efcbf86e740 1 -- 172.21.15.47:0/811679101 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x55f7eb874790 con 0x55f7eb7dc6d0 2024-03-21T10:28:34.919 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:34.919+0000 7efc9b400640 1 -- 172.21.15.47:0/811679101 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7efca804d070 con 0x55f7eb7dc6d0 2024-03-21T10:28:34.919 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:34.919+0000 7efc9b400640 1 -- 172.21.15.47:0/811679101 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 32) v1 ==== 7942+0+0 (secure 0 0 0) 0x7efcb00052a0 con 0x55f7eb7dc6d0 2024-03-21T10:28:34.919 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:34.919+0000 7efcbf86e740 10 client.16797 fetch_fsmap finished waiting for FSMap version 32 2024-03-21T10:28:34.919 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:34.919+0000 7efcbf86e740 10 client.16797 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-21T10:28:34.919 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:34.919+0000 7efcbf86e740 20 client.16797 populate_metadata read hostname 'smithi047' 2024-03-21T10:28:34.919 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:34.919+0000 7efcbf86e740 1 --2- 172.21.15.47:0/811679101 >> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] conn(0x55f7eb874c10 0x55f7eb877030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:28:34.919 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:34.919+0000 7efcbf86e740 4 client.16797 mds_command: new command op to 34535 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-21T10:28:34.919 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:34.919+0000 7efcbf86e740 1 -- 172.21.15.47:0/811679101 --> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55f7eb747b50 con 0x55f7eb874c10 2024-03-21T10:28:34.920 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:34.920+0000 7efcb5400640 1 --2- 172.21.15.47:0/811679101 >> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] conn(0x55f7eb874c10 0x55f7eb877030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:28:34.922 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:34.923+0000 7efcb5400640 1 --2- 172.21.15.47:0/811679101 >> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] conn(0x55f7eb874c10 0x55f7eb877030 crc :-1 s=READY pgs=116 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2273ba242db4e10b server_cookie=86563d92f6725ad5 in_seq=0 out_seq=0 2024-03-21T10:28:34.922 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:34.923+0000 7efc9b400640 10 client.16797 ms_handle_connect on v2:172.21.15.100:6838/4173168440 2024-03-21T10:28:34.923 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:34.923+0000 7efc9b400640 1 -- 172.21.15.47:0/811679101 <== mds.0 v2:172.21.15.100:6838/4173168440 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55f7eb747b50 con 0x55f7eb874c10 2024-03-21T10:28:34.923 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:34.923+0000 7efc9b400640 10 client.16797 handle_command_reply: tid=0 2024-03-21T10:28:34.923 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:34.923+0000 7efcbf86e740 1 client.16797 shutdown 2024-03-21T10:28:34.923 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:34.924+0000 7efcbf86e740 1 -- 172.21.15.47:0/811679101 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f7eb7dc6d0 msgr2=0x55f7eb7edd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:28:34.924 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:34.924+0000 7efcbf86e740 1 --2- 172.21.15.47:0/811679101 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f7eb7dc6d0 0x55f7eb7edd80 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7efcb0000f80 tx=0x7efcb0005bd0 comp rx=0 tx=0).stop 2024-03-21T10:28:34.927 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:34.925+0000 7efcbf86e740 1 -- 172.21.15.47:0/811679101 shutdown_connections 2024-03-21T10:28:34.927 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:34.925+0000 7efcbf86e740 1 --2- 172.21.15.47:0/811679101 >> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] conn(0x55f7eb874c10 0x55f7eb877030 crc :-1 s=READY pgs=116 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:28:34.927 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:34.925+0000 7efcbf86e740 1 --2- 172.21.15.47:0/811679101 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f7eb7ea5a0 0x55f7eb7eaa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:28:34.927 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:34.925+0000 7efcbf86e740 1 --2- 172.21.15.47:0/811679101 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f7eb7e5c00 0x55f7eb7ea060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:28:34.927 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:34.925+0000 7efcbf86e740 1 --2- 172.21.15.47:0/811679101 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f7eb7dc6d0 0x55f7eb7edd80 unknown :-1 s=CLOSED pgs=751 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:28:34.927 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:34.925+0000 7efc9b400640 0 client.16797 ms_handle_reset on v2:172.21.15.100:6838/4173168440 2024-03-21T10:28:34.927 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:34.925+0000 7efcbf86e740 1 -- 172.21.15.47:0/811679101 >> 172.21.15.47:0/811679101 conn(0x55f7eb7d6da0 msgr2=0x55f7eb7dd460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:28:34.928 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:34.928+0000 7efcbf86e740 1 -- 172.21.15.47:0/811679101 shutdown_connections 2024-03-21T10:28:34.928 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:34.928+0000 7efcbf86e740 1 -- 172.21.15.47:0/811679101 wait complete. 2024-03-21T10:28:34.928 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:34.929+0000 7efcbf86e740 20 client.16797 trim_cache size 0 max 16384 2024-03-21T10:28:34.929 INFO:teuthology.orchestra.run.smithi047.stdout:[] 2024-03-21T10:28:34.931 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:34.932+0000 7efcbcc00640 1 -- 172.21.15.47:0/3982878353 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7efc9c0a3190 msgr2=0x7efc9c0a5650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:28:34.932 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:34.932+0000 7efcbcc00640 1 --2- 172.21.15.47:0/3982878353 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7efc9c0a3190 0x7efc9c0a5650 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x55f7eb7dc1a0 tx=0x7efca00cfdf0 comp rx=0 tx=0).stop 2024-03-21T10:28:34.932 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:34.932+0000 7efcbcc00640 1 -- 172.21.15.47:0/3982878353 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7efcb8101630 msgr2=0x7efcb810e210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:28:34.932 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:34.932+0000 7efcbcc00640 1 --2- 172.21.15.47:0/3982878353 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7efcb8101630 0x7efcb810e210 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7efca000c920 tx=0x7efca000cde0 comp rx=0 tx=0).stop 2024-03-21T10:28:34.932 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:34.932+0000 7efcbcc00640 1 -- 172.21.15.47:0/3982878353 shutdown_connections 2024-03-21T10:28:34.932 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:34.932+0000 7efcbcc00640 1 --2- 172.21.15.47:0/3982878353 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7efcb8107640 0x7efcb810b840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:28:34.932 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:34.932+0000 7efcbcc00640 1 --2- 172.21.15.47:0/3982878353 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7efc9c0a3190 0x7efc9c0a5650 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:28:34.932 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:34.932+0000 7efcbcc00640 1 --2- 172.21.15.47:0/3982878353 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7efcb8101ff0 0x7efcb810b300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:28:34.932 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:34.932+0000 7efcbcc00640 1 --2- 172.21.15.47:0/3982878353 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7efcb8101630 0x7efcb810e210 secure :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7efca000c920 tx=0x7efca000cde0 comp rx=0 tx=0).stop 2024-03-21T10:28:34.932 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:34.932+0000 7efcbcc00640 1 -- 172.21.15.47:0/3982878353 >> 172.21.15.47:0/3982878353 conn(0x7efcb8078aa0 msgr2=0x7efcb8103b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:28:34.933 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:34.933+0000 7efcbcc00640 1 -- 172.21.15.47:0/3982878353 shutdown_connections 2024-03-21T10:28:34.933 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:34.933+0000 7efcbcc00640 1 -- 172.21.15.47:0/3982878353 wait complete. 2024-03-21T10:28:35.213 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:28:35 smithi047 ceph-mon[31547]: pgmap v101: 129 pgs: 129 active+clean; 432 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 7.2 MiB/s wr, 1.56k op/s 2024-03-21T10:28:35.478 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:28:35 smithi100 ceph-mon[35193]: pgmap v101: 129 pgs: 129 active+clean; 432 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 7.2 MiB/s wr, 1.56k op/s 2024-03-21T10:28:35.488 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:28:35 smithi157 ceph-mon[35581]: pgmap v101: 129 pgs: 129 active+clean; 432 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 7.2 MiB/s wr, 1.56k op/s 2024-03-21T10:28:35.969 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-03-21T10:28:35.969 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-21T10:28:35.970 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-03-21T10:28:36.325 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:36.325+0000 7f7f51400640 1 -- 172.21.15.47:0/1867114200 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7f4c075bd0 msgr2=0x7f7f4c075fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:28:36.326 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:36.325+0000 7f7f51400640 1 --2- 172.21.15.47:0/1867114200 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7f4c075bd0 0x7f7f4c075fb0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f7f3c009930 tx=0x7f7f3c02bc60 comp rx=0 tx=0).stop 2024-03-21T10:28:36.326 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:36.325+0000 7f7f51400640 1 -- 172.21.15.47:0/1867114200 shutdown_connections 2024-03-21T10:28:36.326 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:36.325+0000 7f7f51400640 1 --2- 172.21.15.47:0/1867114200 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7f4c1123c0 0x7f7f4c1147b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:28:36.326 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:36.325+0000 7f7f51400640 1 --2- 172.21.15.47:0/1867114200 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7f4c0764f0 0x7f7f4c111e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:28:36.326 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:36.325+0000 7f7f51400640 1 --2- 172.21.15.47:0/1867114200 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7f4c075bd0 0x7f7f4c075fb0 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:28:36.326 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:36.325+0000 7f7f51400640 1 -- 172.21.15.47:0/1867114200 >> 172.21.15.47:0/1867114200 conn(0x7f7f4c0ff1b0 msgr2=0x7f7f4c1015d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:28:36.327 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:36.326+0000 7f7f51400640 1 -- 172.21.15.47:0/1867114200 shutdown_connections 2024-03-21T10:28:36.327 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:36.327+0000 7f7f51400640 1 -- 172.21.15.47:0/1867114200 wait complete. 2024-03-21T10:28:36.329 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:36.328+0000 7f7f51400640 1 Processor -- start 2024-03-21T10:28:36.329 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:36.329+0000 7f7f51400640 1 -- start start 2024-03-21T10:28:36.330 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:36.330+0000 7f7f51400640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7f4c075bd0 0x7f7f4c072f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:28:36.330 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:36.330+0000 7f7f51400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7f4c0764f0 0x7f7f4c06cff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:28:36.331 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:36.330+0000 7f7f51400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7f4c1123c0 0x7f7f4c06d530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:28:36.331 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:36.330+0000 7f7f51400640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f4c06b980 con 0x7f7f4c0764f0 2024-03-21T10:28:36.331 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:36.330+0000 7f7f51400640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f4c06b800 con 0x7f7f4c1123c0 2024-03-21T10:28:36.331 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:36.330+0000 7f7f51400640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f4c06bb00 con 0x7f7f4c075bd0 2024-03-21T10:28:36.331 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:36.331+0000 7f7f50a00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7f4c1123c0 0x7f7f4c06d530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:28:36.332 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:36.331+0000 7f7f50a00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7f4c1123c0 0x7f7f4c06d530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:54082/0 (socket says 172.21.15.47:54082) 2024-03-21T10:28:36.332 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:36.331+0000 7f7f50a00640 1 -- 172.21.15.47:0/1906740216 learned_addr learned my addr 172.21.15.47:0/1906740216 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:28:36.332 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:36.332+0000 7f7f4be00640 1 --2- 172.21.15.47:0/1906740216 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7f4c075bd0 0x7f7f4c072f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:28:36.332 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:36.332+0000 7f7f50a00640 1 -- 172.21.15.47:0/1906740216 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7f4c075bd0 msgr2=0x7f7f4c072f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:28:36.332 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:36.332+0000 7f7f50a00640 1 --2- 172.21.15.47:0/1906740216 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7f4c075bd0 0x7f7f4c072f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:28:36.333 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:36.332+0000 7f7f50a00640 1 -- 172.21.15.47:0/1906740216 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7f4c0764f0 msgr2=0x7f7f4c06cff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:28:36.333 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:36.332+0000 7f7f50a00640 1 --2- 172.21.15.47:0/1906740216 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7f4c0764f0 0x7f7f4c06cff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:28:36.333 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:36.332+0000 7f7f50a00640 1 -- 172.21.15.47:0/1906740216 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f4c06db70 con 0x7f7f4c1123c0 2024-03-21T10:28:36.333 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:36.332+0000 7f7f4be00640 1 --2- 172.21.15.47:0/1906740216 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7f4c075bd0 0x7f7f4c072f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T10:28:36.334 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:36.334+0000 7f7f50a00640 1 --2- 172.21.15.47:0/1906740216 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7f4c1123c0 0x7f7f4c06d530 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f7f4400daf0 tx=0x7f7f4400b260 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:28:36.335 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:36.334+0000 7f7f48c00640 1 -- 172.21.15.47:0/1906740216 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f44017070 con 0x7f7f4c1123c0 2024-03-21T10:28:36.336 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:36.335+0000 7f7f48c00640 1 -- 172.21.15.47:0/1906740216 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f7f44003e20 con 0x7f7f4c1123c0 2024-03-21T10:28:36.336 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:36.335+0000 7f7f48c00640 1 -- 172.21.15.47:0/1906740216 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f44010460 con 0x7f7f4c1123c0 2024-03-21T10:28:36.336 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:36.335+0000 7f7f51400640 1 -- 172.21.15.47:0/1906740216 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7f4c1b32b0 con 0x7f7f4c1123c0 2024-03-21T10:28:36.337 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:36.335+0000 7f7f51400640 1 -- 172.21.15.47:0/1906740216 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7f4c1b3660 con 0x7f7f4c1123c0 2024-03-21T10:28:36.345 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:36.343+0000 7f7f48c00640 1 -- 172.21.15.47:0/1906740216 <== mon.2 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 27) v1 ==== 137985+0+0 (secure 0 0 0) 0x7f7f44003970 con 0x7f7f4c1123c0 2024-03-21T10:28:36.345 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:36.345+0000 7f7f55824740 1 Processor -- start 2024-03-21T10:28:36.346 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:36.345+0000 7f7f55824740 1 -- start start 2024-03-21T10:28:36.346 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:36.346+0000 7f7f55824740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c0d253d290 0x55c0d253d670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:28:36.347 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:36.346+0000 7f7f55824740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c0d253dbb0 0x55c0d2546250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:28:36.347 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:36.346+0000 7f7f55824740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c0d2546790 0x55c0d2548b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:28:36.347 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:36.346+0000 7f7f55824740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c0d24a1eb0 con 0x55c0d253d290 2024-03-21T10:28:36.347 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:36.346+0000 7f7f55824740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c0d244aa60 con 0x55c0d2546790 2024-03-21T10:28:36.347 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:36.346+0000 7f7f55824740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c0d249df50 con 0x55c0d253dbb0 2024-03-21T10:28:36.347 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:36.347+0000 7f7f48c00640 1 --2- 172.21.15.47:0/1906740216 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f7f300a3190 0x7f7f300a5650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:28:36.348 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:36.347+0000 7f7f4b400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c0d253d290 0x55c0d253d670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:28:36.348 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:36.347+0000 7f7f4b400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c0d253d290 0x55c0d253d670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:60702/0 (socket says 172.21.15.47:60702) 2024-03-21T10:28:36.348 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:36.347+0000 7f7f4b400640 1 -- 172.21.15.47:0/1458840738 learned_addr learned my addr 172.21.15.47:0/1458840738 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:28:36.348 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:36.348+0000 7f7f4be00640 1 --2- 172.21.15.47:0/1458840738 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c0d253dbb0 0x55c0d2546250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:28:36.349 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:36.348+0000 7f7f48c00640 1 -- 172.21.15.47:0/1906740216 <== mon.2 v2:172.21.15.100:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 9474+0+0 (secure 0 0 0) 0x7f7f440cf9c0 con 0x7f7f4c1123c0 2024-03-21T10:28:36.349 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:36.348+0000 7f7f4b400640 1 -- 172.21.15.47:0/1458840738 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c0d253dbb0 msgr2=0x55c0d2546250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:28:36.349 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:36.348+0000 7f7f4b400640 1 --2- 172.21.15.47:0/1458840738 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c0d253dbb0 0x55c0d2546250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:28:36.349 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:36.348+0000 7f7f4b400640 1 -- 172.21.15.47:0/1458840738 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c0d2546790 msgr2=0x55c0d2548b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:28:36.349 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:36.348+0000 7f7f4b400640 1 --2- 172.21.15.47:0/1458840738 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c0d2546790 0x55c0d2548b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:28:36.350 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:36.348+0000 7f7f4b400640 1 -- 172.21.15.47:0/1458840738 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c0d227e7e0 con 0x55c0d253d290 2024-03-21T10:28:36.350 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:36.349+0000 7f7f50a00640 1 --2- 172.21.15.47:0/1906740216 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f7f300a3190 0x7f7f300a5650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:28:36.351 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:36.350+0000 7f7f4b400640 1 --2- 172.21.15.47:0/1458840738 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c0d253d290 0x55c0d253d670 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7f7f3800cb40 tx=0x7f7f3800e2f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=75f856deae3efab7 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:28:36.351 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:36.350+0000 7f7f50a00640 1 --2- 172.21.15.47:0/1906740216 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f7f300a3190 0x7f7f300a5650 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f7f44099a10 tx=0x7f7f440999a0 comp rx=0 tx=0).ready entity=mgr.15687 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:28:36.352 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:36.351+0000 7f7f41200640 1 -- 172.21.15.47:0/1458840738 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f38015070 con 0x55c0d253d290 2024-03-21T10:28:36.353 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:36.352+0000 7f7f41200640 1 -- 172.21.15.47:0/1458840738 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f7f3800ed20 con 0x55c0d253d290 2024-03-21T10:28:36.353 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:36.352+0000 7f7f41200640 1 -- 172.21.15.47:0/1458840738 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f380107f0 con 0x55c0d253d290 2024-03-21T10:28:36.354 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:36.354+0000 7f7f55824740 1 -- 172.21.15.47:0/1458840738 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c0d253d290 msgr2=0x55c0d253d670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:28:36.354 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:36.354+0000 7f7f55824740 1 --2- 172.21.15.47:0/1458840738 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c0d253d290 0x55c0d253d670 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7f7f3800cb40 tx=0x7f7f3800e2f0 comp rx=0 tx=0).stop 2024-03-21T10:28:36.355 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:36.354+0000 7f7f55824740 1 -- 172.21.15.47:0/1458840738 shutdown_connections 2024-03-21T10:28:36.355 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:36.354+0000 7f7f55824740 1 --2- 172.21.15.47:0/1458840738 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c0d2546790 0x55c0d2548b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:28:36.355 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:36.354+0000 7f7f55824740 1 --2- 172.21.15.47:0/1458840738 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c0d253dbb0 0x55c0d2546250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:28:36.355 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:36.355+0000 7f7f55824740 1 --2- 172.21.15.47:0/1458840738 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c0d253d290 0x55c0d253d670 unknown :-1 s=CLOSED pgs=752 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:28:36.355 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:36.355+0000 7f7f55824740 1 -- 172.21.15.47:0/1458840738 >> 172.21.15.47:0/1458840738 conn(0x55c0d25379c0 msgr2=0x55c0d2537da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:28:36.357 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:36.356+0000 7f7f55824740 1 -- 172.21.15.47:0/1458840738 shutdown_connections 2024-03-21T10:28:36.357 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:36.356+0000 7f7f55824740 1 -- 172.21.15.47:0/1458840738 wait complete. 2024-03-21T10:28:36.357 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:36.356+0000 7f7f55824740 1 -- start start 2024-03-21T10:28:36.357 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:36.357+0000 7f7f55824740 1 Processor -- start 2024-03-21T10:28:36.358 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:36.357+0000 7f7f55824740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c0d253d290 0x55c0d25460f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:28:36.358 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:36.357+0000 7f7f55824740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c0d253dbb0 0x55c0d2543480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:28:36.358 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:36.357+0000 7f7f55824740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c0d2546790 0x55c0d25439e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:28:36.358 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:36.357+0000 7f7f55824740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c0d255a590 con 0x55c0d253d290 2024-03-21T10:28:36.358 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:36.357+0000 7f7f55824740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c0d245deb0 con 0x55c0d253dbb0 2024-03-21T10:28:36.359 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:36.357+0000 7f7f55824740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c0d255a710 con 0x55c0d2546790 2024-03-21T10:28:36.359 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:36.357+0000 7f7f55824740 10 client.0 _ll_register_callbacks cb 0x55c0d2452150 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T10:28:36.359 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:36.359+0000 7f7f4be00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c0d253dbb0 0x55c0d2543480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:28:36.359 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:36.359+0000 7f7f4be00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c0d253dbb0 0x55c0d2543480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:54094/0 (socket says 172.21.15.47:54094) 2024-03-21T10:28:36.359 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:36.359+0000 7f7f4be00640 1 -- 172.21.15.47:0/2816804556 learned_addr learned my addr 172.21.15.47:0/2816804556 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:28:36.360 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:36.359+0000 7f7f4b400640 1 --2- 172.21.15.47:0/2816804556 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c0d2546790 0x55c0d25439e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:28:36.360 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:36.359+0000 7f7f4b400640 1 --2- 172.21.15.47:0/2816804556 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c0d253d290 0x55c0d25460f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:28:36.361 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:36.360+0000 7f7f4b400640 1 -- 172.21.15.47:0/2816804556 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c0d253dbb0 msgr2=0x55c0d2543480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:28:36.361 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:36.360+0000 7f7f4b400640 1 --2- 172.21.15.47:0/2816804556 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c0d253dbb0 0x55c0d2543480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:28:36.361 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:36.360+0000 7f7f4b400640 1 -- 172.21.15.47:0/2816804556 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c0d253d290 msgr2=0x55c0d25460f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:28:36.361 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:36.360+0000 7f7f4b400640 1 --2- 172.21.15.47:0/2816804556 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c0d253d290 0x55c0d25460f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:28:36.361 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:36.360+0000 7f7f4b400640 1 -- 172.21.15.47:0/2816804556 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55c0d22def70 con 0x55c0d2546790 2024-03-21T10:28:36.362 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:36.361+0000 7f7f35600640 10 client.0 ms_handle_connect on v2:172.21.15.157:3300/0 2024-03-21T10:28:36.362 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:36.361+0000 7f7f4b400640 1 --2- 172.21.15.47:0/2816804556 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c0d2546790 0x55c0d25439e0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f7f38005db0 tx=0x7f7f38005d40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d285e77f60305d6 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:28:36.362 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:36.362+0000 7f7f35600640 1 -- 172.21.15.47:0/2816804556 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f38015070 con 0x55c0d2546790 2024-03-21T10:28:36.363 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:36.362+0000 7f7f35600640 1 -- 172.21.15.47:0/2816804556 <== mon.1 v2:172.21.15.157:3300/0 2 ==== osd_map(97..97 src has 1..97) v4 ==== 9474+0+0 (secure 0 0 0) 0x7f7f3800eb20 con 0x55c0d2546790 2024-03-21T10:28:36.363 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:36.363+0000 7f7f55824740 1 -- 172.21.15.47:0/2816804556 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f7f3800ed20 con 0x55c0d2546790 2024-03-21T10:28:36.364 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:36.363+0000 7f7f35600640 1 -- 172.21.15.47:0/2816804556 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7f380056a0 con 0x55c0d2546790 2024-03-21T10:28:36.364 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:36.363+0000 7f7f55824740 10 client.26380 fetch_fsmap learned FSMap version 32 2024-03-21T10:28:36.364 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:36.363+0000 7f7f55824740 1 -- 172.21.15.47:0/2816804556 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x55c0d2176170 con 0x55c0d2546790 2024-03-21T10:28:36.364 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:36.364+0000 7f7f35600640 1 -- 172.21.15.47:0/2816804556 <== mon.1 v2:172.21.15.157:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f7f380056a0 con 0x55c0d2546790 2024-03-21T10:28:36.366 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:36.365+0000 7f7f35600640 1 -- 172.21.15.47:0/2816804556 <== mon.1 v2:172.21.15.157:3300/0 5 ==== fsmap(e 32) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f7f38027020 con 0x55c0d2546790 2024-03-21T10:28:36.366 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:36.365+0000 7f7f55824740 10 client.26380 fetch_fsmap finished waiting for FSMap version 32 2024-03-21T10:28:36.366 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:36.365+0000 7f7f55824740 10 client.26380 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-21T10:28:36.366 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:36.365+0000 7f7f55824740 20 client.26380 populate_metadata read hostname 'smithi047' 2024-03-21T10:28:36.366 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:36.366+0000 7f7f55824740 1 --2- 172.21.15.47:0/2816804556 >> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] conn(0x55c0d2567c30 0x55c0d256a020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:28:36.366 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:36.366+0000 7f7f55824740 4 client.26380 mds_command: new command op to 34535 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T10:28:36.366 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:36.366+0000 7f7f55824740 1 -- 172.21.15.47:0/2816804556 --> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55c0d24af3f0 con 0x55c0d2567c30 2024-03-21T10:28:36.370 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:36.366+0000 7f7f4be00640 1 --2- 172.21.15.47:0/2816804556 >> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] conn(0x55c0d2567c30 0x55c0d256a020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:28:36.370 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:36.369+0000 7f7f35600640 10 client.26380 ms_handle_connect on v2:172.21.15.100:6838/4173168440 2024-03-21T10:28:36.370 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:36.369+0000 7f7f4be00640 1 --2- 172.21.15.47:0/2816804556 >> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] conn(0x55c0d2567c30 0x55c0d256a020 crc :-1 s=READY pgs=117 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a3f499db0f723fac server_cookie=f02fd220665d2767 in_seq=0 out_seq=0 2024-03-21T10:28:36.371 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:36.371+0000 7f7f35600640 1 -- 172.21.15.47:0/2816804556 <== mds.0 v2:172.21.15.100:6838/4173168440 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55c0d24af3f0 con 0x55c0d2567c30 2024-03-21T10:28:36.371 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:36.371+0000 7f7f35600640 10 client.26380 handle_command_reply: tid=0 2024-03-21T10:28:36.372 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:36.371+0000 7f7f55824740 1 client.26380 shutdown 2024-03-21T10:28:36.373 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:36.373+0000 7f7f55824740 1 -- 172.21.15.47:0/2816804556 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c0d2546790 msgr2=0x55c0d25439e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:28:36.374 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:36.373+0000 7f7f55824740 1 --2- 172.21.15.47:0/2816804556 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c0d2546790 0x55c0d25439e0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f7f38005db0 tx=0x7f7f38005d40 comp rx=0 tx=0).stop 2024-03-21T10:28:36.374 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:36.374+0000 7f7f55824740 1 -- 172.21.15.47:0/2816804556 shutdown_connections 2024-03-21T10:28:36.374 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:36.374+0000 7f7f55824740 1 --2- 172.21.15.47:0/2816804556 >> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] conn(0x55c0d2567c30 0x55c0d256a020 crc :-1 s=READY pgs=117 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:28:36.374 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:36.374+0000 7f7f55824740 1 --2- 172.21.15.47:0/2816804556 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c0d2546790 0x55c0d25439e0 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:28:36.374 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:36.374+0000 7f7f55824740 1 --2- 172.21.15.47:0/2816804556 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c0d253dbb0 0x55c0d2543480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:28:36.374 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:36.374+0000 7f7f55824740 1 --2- 172.21.15.47:0/2816804556 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c0d253d290 0x55c0d25460f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:28:36.375 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:36.374+0000 7f7f55824740 1 -- 172.21.15.47:0/2816804556 >> 172.21.15.47:0/2816804556 conn(0x55c0d25379c0 msgr2=0x55c0d2539710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:28:36.375 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:36.374+0000 7f7f35600640 0 client.26380 ms_handle_reset on v2:172.21.15.100:6838/4173168440 2024-03-21T10:28:36.375 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:36.374+0000 7f7f55824740 1 -- 172.21.15.47:0/2816804556 shutdown_connections 2024-03-21T10:28:36.376 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:36.375+0000 7f7f55824740 1 -- 172.21.15.47:0/2816804556 wait complete. 2024-03-21T10:28:36.376 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:36.375+0000 7f7f55824740 20 client.26380 trim_cache size 0 max 16384 2024-03-21T10:28:36.421 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:36.421+0000 7f7f55824740 1 Processor -- start 2024-03-21T10:28:36.422 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:36.421+0000 7f7f55824740 1 -- start start 2024-03-21T10:28:36.422 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:36.421+0000 7f7f55824740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c0d253d290 0x55c0d2545bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:28:36.422 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:36.421+0000 7f7f55824740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c0d253dbb0 0x55c0d25460f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:28:36.422 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:36.421+0000 7f7f55824740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c0d2546790 0x55c0d2562e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:28:36.422 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:36.421+0000 7f7f55824740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f4c1b4db0 con 0x55c0d2546790 2024-03-21T10:28:36.423 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:36.421+0000 7f7f55824740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f4c1b6730 con 0x55c0d253d290 2024-03-21T10:28:36.423 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:36.421+0000 7f7f55824740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f4c1b5410 con 0x55c0d253dbb0 2024-03-21T10:28:36.423 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:36.422+0000 7f7f4be00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c0d253dbb0 0x55c0d25460f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:28:36.423 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:36.422+0000 7f7f4b400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c0d253d290 0x55c0d2545bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:28:36.423 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:36.422+0000 7f7f4b400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c0d253d290 0x55c0d2545bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:54108/0 (socket says 172.21.15.47:54108) 2024-03-21T10:28:36.423 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:36.422+0000 7f7f4b400640 1 -- 172.21.15.47:0/1686116065 learned_addr learned my addr 172.21.15.47:0/1686116065 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:28:36.424 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:36.423+0000 7f7f4b400640 1 --2- 172.21.15.47:0/1686116065 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c0d2546790 0x55c0d2562e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:28:36.424 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:36.424+0000 7f7f4b400640 1 -- 172.21.15.47:0/1686116065 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c0d253dbb0 msgr2=0x55c0d25460f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:28:36.424 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:36.424+0000 7f7f4b400640 1 --2- 172.21.15.47:0/1686116065 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c0d253dbb0 0x55c0d25460f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:28:36.425 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:36.424+0000 7f7f4b400640 1 -- 172.21.15.47:0/1686116065 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c0d2546790 msgr2=0x55c0d2562e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:28:36.425 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:36.424+0000 7f7f4b400640 1 --2- 172.21.15.47:0/1686116065 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c0d2546790 0x55c0d2562e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:28:36.425 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:36.424+0000 7f7f4b400640 1 -- 172.21.15.47:0/1686116065 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c0d2538590 con 0x55c0d253d290 2024-03-21T10:28:36.425 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:36.424+0000 7f7f4be00640 1 --2- 172.21.15.47:0/1686116065 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c0d253dbb0 0x55c0d25460f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T10:28:36.426 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:36.425+0000 7f7f4b400640 1 --2- 172.21.15.47:0/1686116065 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c0d253d290 0x55c0d2545bb0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f7f3800f790 tx=0x7f7f38011850 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cfb0f429961b3606 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:28:36.427 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:36.426+0000 7f7f37e00640 1 -- 172.21.15.47:0/1686116065 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f3800e7f0 con 0x55c0d253d290 2024-03-21T10:28:36.427 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:36.426+0000 7f7f37e00640 1 -- 172.21.15.47:0/1686116065 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f7f380056a0 con 0x55c0d253d290 2024-03-21T10:28:36.427 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:36.426+0000 7f7f37e00640 1 -- 172.21.15.47:0/1686116065 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f38028e00 con 0x55c0d253d290 2024-03-21T10:28:36.428 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:36.427+0000 7f7f55824740 1 -- 172.21.15.47:0/1686116065 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c0d253d290 msgr2=0x55c0d2545bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:28:36.429 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:36.427+0000 7f7f55824740 1 --2- 172.21.15.47:0/1686116065 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c0d253d290 0x55c0d2545bb0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f7f3800f790 tx=0x7f7f38011850 comp rx=0 tx=0).stop 2024-03-21T10:28:36.429 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:36.427+0000 7f7f55824740 1 -- 172.21.15.47:0/1686116065 shutdown_connections 2024-03-21T10:28:36.429 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:36.427+0000 7f7f55824740 1 --2- 172.21.15.47:0/1686116065 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c0d2546790 0x55c0d2562e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:28:36.430 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:36.427+0000 7f7f55824740 1 --2- 172.21.15.47:0/1686116065 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c0d253dbb0 0x55c0d25460f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:28:36.430 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:36.427+0000 7f7f55824740 1 --2- 172.21.15.47:0/1686116065 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c0d253d290 0x55c0d2545bb0 secure :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f7f3800f790 tx=0x7f7f38011850 comp rx=0 tx=0).stop 2024-03-21T10:28:36.430 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:36.427+0000 7f7f55824740 1 -- 172.21.15.47:0/1686116065 >> 172.21.15.47:0/1686116065 conn(0x55c0d25379c0 msgr2=0x55c0d2536a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:28:36.430 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:36.427+0000 7f7f55824740 1 -- 172.21.15.47:0/1686116065 shutdown_connections 2024-03-21T10:28:36.430 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:36.428+0000 7f7f55824740 1 -- 172.21.15.47:0/1686116065 wait complete. 2024-03-21T10:28:36.430 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:36.428+0000 7f7f55824740 1 -- start start 2024-03-21T10:28:36.430 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:36.428+0000 7f7f55824740 1 Processor -- start 2024-03-21T10:28:36.431 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:36.429+0000 7f7f55824740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c0d253dbb0 0x55c0d255f000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:28:36.431 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:36.429+0000 7f7f55824740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c0d2546790 0x55c0d255f540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:28:36.432 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:36.429+0000 7f7f55824740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c0d255b400 0x55c0d255b880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:28:36.432 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:36.429+0000 7f7f55824740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f4c1b60d0 con 0x55c0d2546790 2024-03-21T10:28:36.432 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:36.429+0000 7f7f55824740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f4c1b5a70 con 0x55c0d253dbb0 2024-03-21T10:28:36.432 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:36.429+0000 7f7f55824740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f4c10cd50 con 0x55c0d255b400 2024-03-21T10:28:36.432 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:36.429+0000 7f7f55824740 10 client.0 _ll_register_callbacks cb 0x7f7f3800ca10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T10:28:36.432 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:36.429+0000 7f7f4b400640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c0d255b400 0x55c0d255b880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:28:36.432 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:36.429+0000 7f7f4b400640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c0d255b400 0x55c0d255b880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.47:33356/0 (socket says 172.21.15.47:33356) 2024-03-21T10:28:36.432 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:36.429+0000 7f7f4b400640 1 -- 172.21.15.47:0/2616637432 learned_addr learned my addr 172.21.15.47:0/2616637432 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:28:36.432 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:36.429+0000 7f7f4be00640 1 --2- 172.21.15.47:0/2616637432 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c0d2546790 0x55c0d255f540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:28:36.432 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:36.429+0000 7f7f4b400640 1 --2- 172.21.15.47:0/2616637432 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c0d253dbb0 0x55c0d255f000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:28:36.432 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:36.430+0000 7f7f4b400640 1 -- 172.21.15.47:0/2616637432 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c0d253dbb0 msgr2=0x55c0d255f000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:28:36.432 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:36.430+0000 7f7f4b400640 1 --2- 172.21.15.47:0/2616637432 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c0d253dbb0 0x55c0d255f000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:28:36.432 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:36.430+0000 7f7f4b400640 1 -- 172.21.15.47:0/2616637432 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c0d2546790 msgr2=0x55c0d255f540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:28:36.432 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:36.430+0000 7f7f4b400640 1 --2- 172.21.15.47:0/2616637432 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c0d2546790 0x55c0d255f540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:28:36.433 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:36.430+0000 7f7f4b400640 1 -- 172.21.15.47:0/2616637432 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55c0d255bf70 con 0x55c0d255b400 2024-03-21T10:28:36.433 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:36.430+0000 7f7f4b400640 1 --2- 172.21.15.47:0/2616637432 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c0d255b400 0x55c0d255b880 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f7f3800ef50 tx=0x7f7f38011f90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1431ff186bfd9cf3 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:28:36.433 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:36.430+0000 7f7f35600640 10 client.0 ms_handle_connect on v2:172.21.15.157:3300/0 2024-03-21T10:28:36.433 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:36.431+0000 7f7f35600640 1 -- 172.21.15.47:0/2616637432 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f3800e7f0 con 0x55c0d255b400 2024-03-21T10:28:36.433 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:36.431+0000 7f7f35600640 1 -- 172.21.15.47:0/2616637432 <== mon.1 v2:172.21.15.157:3300/0 2 ==== osd_map(97..97 src has 1..97) v4 ==== 9474+0+0 (secure 0 0 0) 0x7f7f38028980 con 0x55c0d255b400 2024-03-21T10:28:36.433 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:36.431+0000 7f7f55824740 1 -- 172.21.15.47:0/2616637432 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f7f380056a0 con 0x55c0d255b400 2024-03-21T10:28:36.433 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:36.431+0000 7f7f35600640 1 -- 172.21.15.47:0/2616637432 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7f38005c30 con 0x55c0d255b400 2024-03-21T10:28:36.434 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:36.432+0000 7f7f55824740 10 client.26386 fetch_fsmap learned FSMap version 32 2024-03-21T10:28:36.434 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:36.432+0000 7f7f55824740 1 -- 172.21.15.47:0/2616637432 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x55c0d25c4340 con 0x55c0d255b400 2024-03-21T10:28:36.434 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:36.433+0000 7f7f35600640 1 -- 172.21.15.47:0/2616637432 <== mon.1 v2:172.21.15.157:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f7f380056a0 con 0x55c0d255b400 2024-03-21T10:28:36.434 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:36.433+0000 7f7f35600640 1 -- 172.21.15.47:0/2616637432 <== mon.1 v2:172.21.15.157:3300/0 5 ==== fsmap(e 32) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f7f38027020 con 0x55c0d255b400 2024-03-21T10:28:36.434 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:36.433+0000 7f7f55824740 10 client.26386 fetch_fsmap finished waiting for FSMap version 32 2024-03-21T10:28:36.434 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:36.433+0000 7f7f55824740 10 client.26386 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-21T10:28:36.434 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:36.433+0000 7f7f55824740 20 client.26386 populate_metadata read hostname 'smithi047' 2024-03-21T10:28:36.434 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:36.434+0000 7f7f55824740 1 --2- 172.21.15.47:0/2616637432 >> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] conn(0x55c0d25c45d0 0x55c0d25c69c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:28:36.435 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:36.434+0000 7f7f55824740 4 client.26386 mds_command: new command op to 34535 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-03-21T10:28:36.435 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:36.434+0000 7f7f55824740 1 -- 172.21.15.47:0/2616637432 --> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x55c0d24a9740 con 0x55c0d25c45d0 2024-03-21T10:28:36.435 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:36.435+0000 7f7f4be00640 1 --2- 172.21.15.47:0/2616637432 >> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] conn(0x55c0d25c45d0 0x55c0d25c69c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:28:36.438 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:36.438+0000 7f7f4be00640 1 --2- 172.21.15.47:0/2616637432 >> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] conn(0x55c0d25c45d0 0x55c0d25c69c0 crc :-1 s=READY pgs=118 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=854065907898a17f server_cookie=a861308467ee8db5 in_seq=0 out_seq=0 2024-03-21T10:28:36.438 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:36.438+0000 7f7f35600640 10 client.26386 ms_handle_connect on v2:172.21.15.100:6838/4173168440 2024-03-21T10:28:36.441 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:36.440+0000 7f7f35600640 1 -- 172.21.15.47:0/2616637432 <== mds.0 v2:172.21.15.100:6838/4173168440 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x55c0d24a9740 con 0x55c0d25c45d0 2024-03-21T10:28:36.441 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:36.440+0000 7f7f35600640 10 client.26386 handle_command_reply: tid=0 2024-03-21T10:28:36.441 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:36.440+0000 7f7f55824740 1 client.26386 shutdown 2024-03-21T10:28:36.442 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:36.441+0000 7f7f55824740 1 -- 172.21.15.47:0/2616637432 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c0d255b400 msgr2=0x55c0d255b880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:28:36.442 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:36.441+0000 7f7f55824740 1 --2- 172.21.15.47:0/2616637432 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c0d255b400 0x55c0d255b880 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f7f3800ef50 tx=0x7f7f38011f90 comp rx=0 tx=0).stop 2024-03-21T10:28:36.442 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:36.442+0000 7f7f55824740 1 -- 172.21.15.47:0/2616637432 shutdown_connections 2024-03-21T10:28:36.442 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:36.442+0000 7f7f55824740 1 --2- 172.21.15.47:0/2616637432 >> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] conn(0x55c0d25c45d0 0x55c0d25c69c0 crc :-1 s=READY pgs=118 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:28:36.443 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:36.442+0000 7f7f55824740 1 --2- 172.21.15.47:0/2616637432 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c0d255b400 0x55c0d255b880 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:28:36.444 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:36.442+0000 7f7f55824740 1 --2- 172.21.15.47:0/2616637432 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c0d2546790 0x55c0d255f540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:28:36.444 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:36.442+0000 7f7f35600640 0 client.26386 ms_handle_reset on v2:172.21.15.100:6838/4173168440 2024-03-21T10:28:36.444 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:36.443+0000 7f7f55824740 1 --2- 172.21.15.47:0/2616637432 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c0d253dbb0 0x55c0d255f000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:28:36.444 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:36.443+0000 7f7f55824740 1 -- 172.21.15.47:0/2616637432 >> 172.21.15.47:0/2616637432 conn(0x55c0d25379c0 msgr2=0x55c0d25361f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:28:36.444 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:36.444+0000 7f7f55824740 1 -- 172.21.15.47:0/2616637432 shutdown_connections 2024-03-21T10:28:36.445 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:36.444+0000 7f7f55824740 1 -- 172.21.15.47:0/2616637432 wait complete. 2024-03-21T10:28:36.445 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:36.444+0000 7f7f55824740 20 client.26386 trim_cache size 0 max 16384 2024-03-21T10:28:36.447 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-03-21T10:28:36.447 INFO:teuthology.orchestra.run.smithi047.stdout: "return_code": 0, 2024-03-21T10:28:36.447 INFO:teuthology.orchestra.run.smithi047.stdout: "scrub_tag": "9db5396b-62a3-497a-afb3-c986901483c9", 2024-03-21T10:28:36.447 INFO:teuthology.orchestra.run.smithi047.stdout: "mode": "asynchronous" 2024-03-21T10:28:36.447 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-03-21T10:28:36.448 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:36.447+0000 7f7f51400640 1 -- 172.21.15.47:0/1906740216 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f7f300a3190 msgr2=0x7f7f300a5650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:28:36.448 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:36.447+0000 7f7f51400640 1 --2- 172.21.15.47:0/1906740216 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f7f300a3190 0x7f7f300a5650 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f7f44099a10 tx=0x7f7f440999a0 comp rx=0 tx=0).stop 2024-03-21T10:28:36.448 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:36.447+0000 7f7f51400640 1 -- 172.21.15.47:0/1906740216 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7f4c1123c0 msgr2=0x7f7f4c06d530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:28:36.448 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:36.447+0000 7f7f51400640 1 --2- 172.21.15.47:0/1906740216 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7f4c1123c0 0x7f7f4c06d530 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f7f4400daf0 tx=0x7f7f4400b260 comp rx=0 tx=0).stop 2024-03-21T10:28:36.448 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:36.448+0000 7f7f51400640 1 -- 172.21.15.47:0/1906740216 shutdown_connections 2024-03-21T10:28:36.448 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:36.448+0000 7f7f51400640 1 --2- 172.21.15.47:0/1906740216 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f7f300a3190 0x7f7f300a5650 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:28:36.449 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:36.448+0000 7f7f51400640 1 --2- 172.21.15.47:0/1906740216 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7f4c1123c0 0x7f7f4c06d530 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:28:36.449 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:36.448+0000 7f7f51400640 1 --2- 172.21.15.47:0/1906740216 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7f4c0764f0 0x7f7f4c06cff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:28:36.449 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:36.448+0000 7f7f51400640 1 --2- 172.21.15.47:0/1906740216 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7f4c075bd0 0x7f7f4c072f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:28:36.449 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:36.448+0000 7f7f51400640 1 -- 172.21.15.47:0/1906740216 >> 172.21.15.47:0/1906740216 conn(0x7f7f4c0ff1b0 msgr2=0x7f7f4c113030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:28:36.449 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:36.448+0000 7f7f51400640 1 -- 172.21.15.47:0/1906740216 shutdown_connections 2024-03-21T10:28:36.449 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:36.449+0000 7f7f51400640 1 -- 172.21.15.47:0/1906740216 wait complete. 2024-03-21T10:28:36.462 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:28:36 smithi047 ceph-mon[31547]: from='mgr.15687 172.21.15.47:0/8730661' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T10:28:36.462 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:28:36 smithi047 ceph-mon[31547]: from='mgr.15687 172.21.15.47:0/8730661' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T10:28:36.463 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:28:36 smithi047 ceph-mon[31547]: from='mgr.15687 172.21.15.47:0/8730661' entity='mgr.x' 2024-03-21T10:28:36.478 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:28:36 smithi100 ceph-mon[35193]: from='mgr.15687 172.21.15.47:0/8730661' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T10:28:36.478 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:28:36 smithi100 ceph-mon[35193]: from='mgr.15687 172.21.15.47:0/8730661' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T10:28:36.478 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:28:36 smithi100 ceph-mon[35193]: from='mgr.15687 172.21.15.47:0/8730661' entity='mgr.x' 2024-03-21T10:28:36.484 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-21T10:28:36.484 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-21T10:28:36.487 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:28:36 smithi157 ceph-mon[35581]: from='mgr.15687 172.21.15.47:0/8730661' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T10:28:36.488 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:28:36 smithi157 ceph-mon[35581]: from='mgr.15687 172.21.15.47:0/8730661' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T10:28:36.488 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:28:36 smithi157 ceph-mon[35581]: from='mgr.15687 172.21.15.47:0/8730661' entity='mgr.x' 2024-03-21T10:28:36.820 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:36.819+0000 7f7a2fe00640 1 -- 172.21.15.47:0/3887477019 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7a301040e0 msgr2=0x7f7a301105f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:28:36.820 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:36.819+0000 7f7a2fe00640 1 --2- 172.21.15.47:0/3887477019 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7a301040e0 0x7f7a301105f0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f7a28009be0 tx=0x7f7a2802bef0 comp rx=0 tx=0).stop 2024-03-21T10:28:36.821 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:36.820+0000 7f7a2fe00640 1 -- 172.21.15.47:0/3887477019 shutdown_connections 2024-03-21T10:28:36.821 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:36.820+0000 7f7a2fe00640 1 --2- 172.21.15.47:0/3887477019 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7a301040e0 0x7f7a301105f0 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:28:36.821 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:36.820+0000 7f7a2fe00640 1 --2- 172.21.15.47:0/3887477019 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7a30103720 0x7f7a30103ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:28:36.821 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:36.820+0000 7f7a2fe00640 1 --2- 172.21.15.47:0/3887477019 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7a30109730 0x7f7a30109b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:28:36.821 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:36.820+0000 7f7a2fe00640 1 -- 172.21.15.47:0/3887477019 >> 172.21.15.47:0/3887477019 conn(0x7f7a300ff1b0 msgr2=0x7f7a301015d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:28:36.821 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:36.820+0000 7f7a2fe00640 1 -- 172.21.15.47:0/3887477019 shutdown_connections 2024-03-21T10:28:36.822 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:36.820+0000 7f7a2fe00640 1 -- 172.21.15.47:0/3887477019 wait complete. 2024-03-21T10:28:36.822 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:36.822+0000 7f7a2fe00640 1 Processor -- start 2024-03-21T10:28:36.823 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:36.822+0000 7f7a2fe00640 1 -- start start 2024-03-21T10:28:36.823 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:36.823+0000 7f7a2fe00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7a30103720 0x7f7a30075bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:28:36.824 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:36.823+0000 7f7a2fe00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7a301040e0 0x7f7a300760f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:28:36.824 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:36.823+0000 7f7a2fe00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7a30109730 0x7f7a30076630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:28:36.824 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:36.823+0000 7f7a2fe00640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a301139f0 con 0x7f7a30103720 2024-03-21T10:28:36.824 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:36.823+0000 7f7a2fe00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a30113870 con 0x7f7a301040e0 2024-03-21T10:28:36.824 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:36.823+0000 7f7a2fe00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a30113b70 con 0x7f7a30109730 2024-03-21T10:28:36.824 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:36.824+0000 7f7a2f400640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7a30109730 0x7f7a30076630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:28:36.825 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:36.824+0000 7f7a2f400640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7a30109730 0x7f7a30076630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.47:33378/0 (socket says 172.21.15.47:33378) 2024-03-21T10:28:36.825 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:36.824+0000 7f7a2f400640 1 -- 172.21.15.47:0/286050139 learned_addr learned my addr 172.21.15.47:0/286050139 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:28:36.825 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:36.824+0000 7f7a2ea00640 1 --2- 172.21.15.47:0/286050139 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7a30103720 0x7f7a30075bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:28:36.825 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:36.824+0000 7f7a2f400640 1 -- 172.21.15.47:0/286050139 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7a301040e0 msgr2=0x7f7a300760f0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-21T10:28:36.825 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:36.824+0000 7f7a2f400640 1 --2- 172.21.15.47:0/286050139 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7a301040e0 0x7f7a300760f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:28:36.825 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:36.824+0000 7f7a2f400640 1 -- 172.21.15.47:0/286050139 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7a30103720 msgr2=0x7f7a30075bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:28:36.825 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:36.824+0000 7f7a2f400640 1 --2- 172.21.15.47:0/286050139 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7a30103720 0x7f7a30075bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:28:36.826 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:36.824+0000 7f7a2f400640 1 -- 172.21.15.47:0/286050139 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a300720d0 con 0x7f7a30109730 2024-03-21T10:28:36.826 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:36.825+0000 7f7a2ea00640 1 --2- 172.21.15.47:0/286050139 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7a30103720 0x7f7a30075bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T10:28:36.826 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:36.825+0000 7f7a2f400640 1 --2- 172.21.15.47:0/286050139 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7a30109730 0x7f7a30076630 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f7a28036040 tx=0x7f7a28033c40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:28:36.826 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:36.826+0000 7f7a34a00640 1 -- 172.21.15.47:0/286050139 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a28043020 con 0x7f7a30109730 2024-03-21T10:28:36.827 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:36.826+0000 7f7a34a00640 1 -- 172.21.15.47:0/286050139 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f7a28003e20 con 0x7f7a30109730 2024-03-21T10:28:36.827 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:36.826+0000 7f7a34a00640 1 -- 172.21.15.47:0/286050139 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a28034430 con 0x7f7a30109730 2024-03-21T10:28:36.827 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:36.826+0000 7f7a2fe00640 1 -- 172.21.15.47:0/286050139 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7a30072300 con 0x7f7a30109730 2024-03-21T10:28:36.827 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:36.826+0000 7f7a2fe00640 1 -- 172.21.15.47:0/286050139 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7a300728a0 con 0x7f7a30109730 2024-03-21T10:28:36.836 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:36.835+0000 7f7a389f1740 1 Processor -- start 2024-03-21T10:28:36.836 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:36.836+0000 7f7a34a00640 1 -- 172.21.15.47:0/286050139 <== mon.1 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 27) v1 ==== 137985+0+0 (secure 0 0 0) 0x7f7a28046020 con 0x7f7a30109730 2024-03-21T10:28:36.837 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:36.837+0000 7f7a389f1740 1 -- start start 2024-03-21T10:28:36.838 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:36.837+0000 7f7a389f1740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55cc72e4bdd0 0x55cc72e4ab20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:28:36.838 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:36.837+0000 7f7a389f1740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55cc72e4b0f0 0x55cc72e4b570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:28:36.838 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:36.837+0000 7f7a389f1740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55cc72e54460 0x55cc72e56850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:28:36.838 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:36.837+0000 7f7a389f1740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cc72daf250 con 0x55cc72e54460 2024-03-21T10:28:36.838 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:36.837+0000 7f7a389f1740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cc72d57a30 con 0x55cc72e4bdd0 2024-03-21T10:28:36.839 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:36.837+0000 7f7a389f1740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cc72dab2f0 con 0x55cc72e4b0f0 2024-03-21T10:28:36.841 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:36.839+0000 7f7a2e000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55cc72e4bdd0 0x55cc72e4ab20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:28:36.841 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:36.839+0000 7f7a2e000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55cc72e4bdd0 0x55cc72e4ab20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:54130/0 (socket says 172.21.15.47:54130) 2024-03-21T10:28:36.841 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:36.839+0000 7f7a2e000640 1 -- 172.21.15.47:0/3921239128 learned_addr learned my addr 172.21.15.47:0/3921239128 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:28:36.841 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:36.839+0000 7f7a2ea00640 1 --2- 172.21.15.47:0/3921239128 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55cc72e4b0f0 0x55cc72e4b570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:28:36.841 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:36.840+0000 7f7a2ea00640 1 -- 172.21.15.47:0/3921239128 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55cc72e4bdd0 msgr2=0x55cc72e4ab20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:28:36.841 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:36.840+0000 7f7a2ea00640 1 --2- 172.21.15.47:0/3921239128 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55cc72e4bdd0 0x55cc72e4ab20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:28:36.841 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:36.840+0000 7f7a34a00640 1 --2- 172.21.15.47:0/286050139 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f7a140a31d0 0x7f7a140a5690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:28:36.842 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:36.840+0000 7f7a2ea00640 1 -- 172.21.15.47:0/3921239128 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55cc72e54460 msgr2=0x55cc72e56850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:28:36.842 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:36.840+0000 7f7a2ea00640 1 --2- 172.21.15.47:0/3921239128 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55cc72e54460 0x55cc72e56850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:28:36.842 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:36.840+0000 7f7a2ea00640 1 -- 172.21.15.47:0/3921239128 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55cc72d42690 con 0x55cc72e4b0f0 2024-03-21T10:28:36.842 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:36.840+0000 7f7a34a00640 1 -- 172.21.15.47:0/286050139 <== mon.1 v2:172.21.15.157:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 9474+0+0 (secure 0 0 0) 0x7f7a280f22e0 con 0x7f7a30109730 2024-03-21T10:28:36.842 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:36.841+0000 7f7a2f400640 1 --2- 172.21.15.47:0/286050139 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f7a140a31d0 0x7f7a140a5690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:28:36.842 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:36.841+0000 7f7a2ea00640 1 --2- 172.21.15.47:0/3921239128 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55cc72e4b0f0 0x55cc72e4b570 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f7a20009bc0 tx=0x7f7a20010740 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4e42f51ae2ca8ba server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:28:36.843 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:36.842+0000 7f7a1be00640 1 -- 172.21.15.47:0/3921239128 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a20004030 con 0x55cc72e4b0f0 2024-03-21T10:28:36.843 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:36.842+0000 7f7a1be00640 1 -- 172.21.15.47:0/3921239128 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f7a200035f0 con 0x55cc72e4b0f0 2024-03-21T10:28:36.843 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:36.842+0000 7f7a1be00640 1 -- 172.21.15.47:0/3921239128 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a200126b0 con 0x55cc72e4b0f0 2024-03-21T10:28:36.843 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:36.842+0000 7f7a2f400640 1 --2- 172.21.15.47:0/286050139 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f7a140a31d0 0x7f7a140a5690 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f7a280bad90 tx=0x7f7a2802bef0 comp rx=0 tx=0).ready entity=mgr.15687 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:28:36.845 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:36.843+0000 7f7a389f1740 1 -- 172.21.15.47:0/3921239128 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55cc72e4b0f0 msgr2=0x55cc72e4b570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:28:36.845 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:36.844+0000 7f7a389f1740 1 --2- 172.21.15.47:0/3921239128 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55cc72e4b0f0 0x55cc72e4b570 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f7a20009bc0 tx=0x7f7a20010740 comp rx=0 tx=0).stop 2024-03-21T10:28:36.845 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:36.844+0000 7f7a389f1740 1 -- 172.21.15.47:0/3921239128 shutdown_connections 2024-03-21T10:28:36.845 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:36.844+0000 7f7a389f1740 1 --2- 172.21.15.47:0/3921239128 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55cc72e54460 0x55cc72e56850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:28:36.845 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:36.844+0000 7f7a389f1740 1 --2- 172.21.15.47:0/3921239128 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55cc72e4b0f0 0x55cc72e4b570 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:28:36.845 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:36.844+0000 7f7a389f1740 1 --2- 172.21.15.47:0/3921239128 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55cc72e4bdd0 0x55cc72e4ab20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:28:36.845 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:36.844+0000 7f7a389f1740 1 -- 172.21.15.47:0/3921239128 >> 172.21.15.47:0/3921239128 conn(0x55cc72e12200 msgr2=0x55cc72e45b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:28:36.845 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:36.844+0000 7f7a389f1740 1 -- 172.21.15.47:0/3921239128 shutdown_connections 2024-03-21T10:28:36.846 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:36.844+0000 7f7a389f1740 1 -- 172.21.15.47:0/3921239128 wait complete. 2024-03-21T10:28:36.846 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:36.845+0000 7f7a389f1740 1 -- start start 2024-03-21T10:28:36.846 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:36.846+0000 7f7a389f1740 1 Processor -- start 2024-03-21T10:28:36.850 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:36.849+0000 7f7a389f1740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55cc72e4b0f0 0x55cc72e53b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:28:36.850 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:36.849+0000 7f7a389f1740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55cc72e4bdd0 0x55cc72e50f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:28:36.850 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:36.849+0000 7f7a389f1740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55cc72e54460 0x55cc72e51450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:28:36.850 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:36.849+0000 7f7a389f1740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cc72e572d0 con 0x55cc72e54460 2024-03-21T10:28:36.850 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:36.849+0000 7f7a389f1740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cc72d6ae80 con 0x55cc72e4b0f0 2024-03-21T10:28:36.851 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:36.849+0000 7f7a389f1740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cc72e57450 con 0x55cc72e4bdd0 2024-03-21T10:28:36.851 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:36.850+0000 7f7a389f1740 10 client.0 _ll_register_callbacks cb 0x55cc72d7d130 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T10:28:36.851 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:36.851+0000 7f7a2ea00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55cc72e4bdd0 0x55cc72e50f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:28:36.851 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:36.851+0000 7f7a2e000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55cc72e4b0f0 0x55cc72e53b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:28:36.851 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:36.851+0000 7f7a2e000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55cc72e4b0f0 0x55cc72e53b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:54132/0 (socket says 172.21.15.47:54132) 2024-03-21T10:28:36.851 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:36.851+0000 7f7a2e000640 1 -- 172.21.15.47:0/820620729 learned_addr learned my addr 172.21.15.47:0/820620729 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:28:36.852 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:36.851+0000 7f7a2e000640 1 --2- 172.21.15.47:0/820620729 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55cc72e54460 0x55cc72e51450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:28:36.852 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:36.851+0000 7f7a2e000640 1 -- 172.21.15.47:0/820620729 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55cc72e4bdd0 msgr2=0x55cc72e50f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:28:36.853 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:36.852+0000 7f7a2e000640 1 --2- 172.21.15.47:0/820620729 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55cc72e4bdd0 0x55cc72e50f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:28:36.853 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:36.852+0000 7f7a2e000640 1 -- 172.21.15.47:0/820620729 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55cc72e54460 msgr2=0x55cc72e51450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:28:36.853 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:36.852+0000 7f7a2e000640 1 --2- 172.21.15.47:0/820620729 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55cc72e54460 0x55cc72e51450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:28:36.853 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:36.852+0000 7f7a2e000640 1 -- 172.21.15.47:0/820620729 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55cc72b8d7d0 con 0x55cc72e4b0f0 2024-03-21T10:28:36.853 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:36.852+0000 7f7a2ea00640 1 --2- 172.21.15.47:0/820620729 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55cc72e4bdd0 0x55cc72e50f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T10:28:36.853 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:36.853+0000 7f7a2e000640 1 --2- 172.21.15.47:0/820620729 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55cc72e4b0f0 0x55cc72e53b80 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f7a1c012840 tx=0x7f7a1c00d820 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f7d91f303fbca5e3 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:28:36.853 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:36.853+0000 7f7a18c00640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-21T10:28:36.854 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:36.854+0000 7f7a18c00640 1 -- 172.21.15.47:0/820620729 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a1c010ce0 con 0x55cc72e4b0f0 2024-03-21T10:28:36.855 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:36.854+0000 7f7a18c00640 1 -- 172.21.15.47:0/820620729 <== mon.2 v2:172.21.15.100:3300/0 2 ==== osd_map(97..97 src has 1..97) v4 ==== 9474+0+0 (secure 0 0 0) 0x7f7a1c01e3f0 con 0x55cc72e4b0f0 2024-03-21T10:28:36.855 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:36.854+0000 7f7a389f1740 1 -- 172.21.15.47:0/820620729 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f7a200035f0 con 0x55cc72e4b0f0 2024-03-21T10:28:36.855 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:36.855+0000 7f7a18c00640 1 -- 172.21.15.47:0/820620729 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7a1c026050 con 0x55cc72e4b0f0 2024-03-21T10:28:36.855 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:36.855+0000 7f7a389f1740 10 client.36626 fetch_fsmap learned FSMap version 32 2024-03-21T10:28:36.855 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:36.855+0000 7f7a389f1740 1 -- 172.21.15.47:0/820620729 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x55cc72ac33f0 con 0x55cc72e4b0f0 2024-03-21T10:28:36.856 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:36.856+0000 7f7a18c00640 1 -- 172.21.15.47:0/820620729 <== mon.2 v2:172.21.15.100:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f7a200035f0 con 0x55cc72e4b0f0 2024-03-21T10:28:36.856 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:36.856+0000 7f7a18c00640 1 -- 172.21.15.47:0/820620729 <== mon.2 v2:172.21.15.100:3300/0 5 ==== fsmap(e 32) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f7a1c01d3a0 con 0x55cc72e4b0f0 2024-03-21T10:28:36.857 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:36.856+0000 7f7a389f1740 10 client.36626 fetch_fsmap finished waiting for FSMap version 32 2024-03-21T10:28:36.857 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:36.856+0000 7f7a389f1740 10 client.36626 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-21T10:28:36.858 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:36.856+0000 7f7a389f1740 20 client.36626 populate_metadata read hostname 'smithi047' 2024-03-21T10:28:36.858 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:36.856+0000 7f7a389f1740 1 --2- 172.21.15.47:0/820620729 >> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] conn(0x55cc72e643a0 0x55cc72e66790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:28:36.858 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:36.856+0000 7f7a389f1740 4 client.36626 mds_command: new command op to 34535 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T10:28:36.858 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:36.856+0000 7f7a389f1740 1 -- 172.21.15.47:0/820620729 --> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55cc72dbc790 con 0x55cc72e643a0 2024-03-21T10:28:36.859 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:36.857+0000 7f7a2ea00640 1 --2- 172.21.15.47:0/820620729 >> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] conn(0x55cc72e643a0 0x55cc72e66790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:28:36.860 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:36.859+0000 7f7a2ea00640 1 --2- 172.21.15.47:0/820620729 >> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] conn(0x55cc72e643a0 0x55cc72e66790 crc :-1 s=READY pgs=119 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3300c523a31ade60 server_cookie=28830825430bf68c in_seq=0 out_seq=0 2024-03-21T10:28:36.861 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:36.860+0000 7f7a18c00640 10 client.36626 ms_handle_connect on v2:172.21.15.100:6838/4173168440 2024-03-21T10:28:36.862 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:36.861+0000 7f7a18c00640 1 -- 172.21.15.47:0/820620729 <== mds.0 v2:172.21.15.100:6838/4173168440 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55cc72dbc790 con 0x55cc72e643a0 2024-03-21T10:28:36.862 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:36.861+0000 7f7a18c00640 10 client.36626 handle_command_reply: tid=0 2024-03-21T10:28:36.862 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:36.861+0000 7f7a389f1740 1 client.36626 shutdown 2024-03-21T10:28:36.862 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:36.862+0000 7f7a389f1740 1 -- 172.21.15.47:0/820620729 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55cc72e4b0f0 msgr2=0x55cc72e53b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:28:36.863 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:36.862+0000 7f7a389f1740 1 --2- 172.21.15.47:0/820620729 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55cc72e4b0f0 0x55cc72e53b80 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f7a1c012840 tx=0x7f7a1c00d820 comp rx=0 tx=0).stop 2024-03-21T10:28:36.863 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:36.862+0000 7f7a389f1740 1 -- 172.21.15.47:0/820620729 shutdown_connections 2024-03-21T10:28:36.863 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:36.862+0000 7f7a389f1740 1 --2- 172.21.15.47:0/820620729 >> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] conn(0x55cc72e643a0 0x55cc72e66790 crc :-1 s=READY pgs=119 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:28:36.863 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:36.862+0000 7f7a389f1740 1 --2- 172.21.15.47:0/820620729 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55cc72e54460 0x55cc72e51450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:28:36.863 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:36.862+0000 7f7a389f1740 1 --2- 172.21.15.47:0/820620729 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55cc72e4bdd0 0x55cc72e50f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:28:36.863 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:36.862+0000 7f7a389f1740 1 --2- 172.21.15.47:0/820620729 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55cc72e4b0f0 0x55cc72e53b80 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:28:36.863 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:36.862+0000 7f7a18c00640 0 client.36626 ms_handle_reset on v2:172.21.15.100:6838/4173168440 2024-03-21T10:28:36.863 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:36.862+0000 7f7a389f1740 1 -- 172.21.15.47:0/820620729 >> 172.21.15.47:0/820620729 conn(0x55cc72e12200 msgr2=0x55cc72e47890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:28:36.863 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:36.862+0000 7f7a389f1740 1 -- 172.21.15.47:0/820620729 shutdown_connections 2024-03-21T10:28:36.863 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:36.863+0000 7f7a389f1740 1 -- 172.21.15.47:0/820620729 wait complete. 2024-03-21T10:28:36.864 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:36.864+0000 7f7a389f1740 20 client.36626 trim_cache size 0 max 16384 2024-03-21T10:28:36.906 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:36.905+0000 7f7a389f1740 1 Processor -- start 2024-03-21T10:28:36.906 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:36.905+0000 7f7a389f1740 1 -- start start 2024-03-21T10:28:36.906 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:36.906+0000 7f7a389f1740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55cc72e4b0f0 0x55cc72e53640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:28:36.906 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:36.906+0000 7f7a389f1740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55cc72e4bdd0 0x55cc72e53b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:28:36.906 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:36.906+0000 7f7a389f1740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55cc72e54460 0x55cc72e55850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:28:36.906 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:36.906+0000 7f7a389f1740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a301bd450 con 0x55cc72e4b0f0 2024-03-21T10:28:36.907 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:36.906+0000 7f7a389f1740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a301bedd0 con 0x55cc72e54460 2024-03-21T10:28:36.907 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:36.906+0000 7f7a389f1740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a301bdab0 con 0x55cc72e4bdd0 2024-03-21T10:28:36.908 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:36.906+0000 7f7a2ea00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55cc72e4bdd0 0x55cc72e53b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:28:36.908 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:36.906+0000 7f7a2e000640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55cc72e4b0f0 0x55cc72e53640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:28:36.908 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:36.906+0000 7f7a2ea00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55cc72e4bdd0 0x55cc72e53b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.47:33408/0 (socket says 172.21.15.47:33408) 2024-03-21T10:28:36.908 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:36.906+0000 7f7a2ea00640 1 -- 172.21.15.47:0/1246319471 learned_addr learned my addr 172.21.15.47:0/1246319471 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:28:36.908 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:36.906+0000 7f7a2e000640 1 --2- 172.21.15.47:0/1246319471 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55cc72e54460 0x55cc72e55850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:28:36.908 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:36.907+0000 7f7a2e000640 1 -- 172.21.15.47:0/1246319471 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55cc72e4bdd0 msgr2=0x55cc72e53b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:28:36.908 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:36.907+0000 7f7a2e000640 1 --2- 172.21.15.47:0/1246319471 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55cc72e4bdd0 0x55cc72e53b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:28:36.908 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:36.907+0000 7f7a2e000640 1 -- 172.21.15.47:0/1246319471 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55cc72e54460 msgr2=0x55cc72e55850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:28:36.908 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:36.907+0000 7f7a2e000640 1 --2- 172.21.15.47:0/1246319471 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55cc72e54460 0x55cc72e55850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:28:36.908 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:36.907+0000 7f7a2e000640 1 -- 172.21.15.47:0/1246319471 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55cc72a830f0 con 0x55cc72e4b0f0 2024-03-21T10:28:36.908 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:36.907+0000 7f7a2e000640 1 --2- 172.21.15.47:0/1246319471 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55cc72e4b0f0 0x55cc72e53640 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7f7a1c00df40 tx=0x7f7a1c00dc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=272cd7cf4565fa51 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:28:36.909 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:36.907+0000 7f7a1b400640 1 -- 172.21.15.47:0/1246319471 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a1c01dd10 con 0x55cc72e4b0f0 2024-03-21T10:28:36.909 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:36.907+0000 7f7a1b400640 1 -- 172.21.15.47:0/1246319471 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f7a1c01d3a0 con 0x55cc72e4b0f0 2024-03-21T10:28:36.909 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:36.908+0000 7f7a1b400640 1 -- 172.21.15.47:0/1246319471 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a1c005600 con 0x55cc72e4b0f0 2024-03-21T10:28:36.909 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:36.908+0000 7f7a389f1740 1 -- 172.21.15.47:0/1246319471 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55cc72e4b0f0 msgr2=0x55cc72e53640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:28:36.910 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:36.908+0000 7f7a389f1740 1 --2- 172.21.15.47:0/1246319471 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55cc72e4b0f0 0x55cc72e53640 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7f7a1c00df40 tx=0x7f7a1c00dc70 comp rx=0 tx=0).stop 2024-03-21T10:28:36.910 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:36.908+0000 7f7a389f1740 1 -- 172.21.15.47:0/1246319471 shutdown_connections 2024-03-21T10:28:36.910 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:36.909+0000 7f7a389f1740 1 --2- 172.21.15.47:0/1246319471 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55cc72e54460 0x55cc72e55850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:28:36.910 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:36.909+0000 7f7a389f1740 1 --2- 172.21.15.47:0/1246319471 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55cc72e4bdd0 0x55cc72e53b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:28:36.910 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:36.909+0000 7f7a389f1740 1 --2- 172.21.15.47:0/1246319471 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55cc72e4b0f0 0x55cc72e53640 unknown :-1 s=CLOSED pgs=753 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:28:36.910 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:36.909+0000 7f7a389f1740 1 -- 172.21.15.47:0/1246319471 >> 172.21.15.47:0/1246319471 conn(0x55cc72e12200 msgr2=0x55cc72e45670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:28:36.910 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:36.909+0000 7f7a389f1740 1 -- 172.21.15.47:0/1246319471 shutdown_connections 2024-03-21T10:28:36.910 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:36.909+0000 7f7a389f1740 1 -- 172.21.15.47:0/1246319471 wait complete. 2024-03-21T10:28:36.910 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:36.909+0000 7f7a389f1740 1 -- start start 2024-03-21T10:28:36.910 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:36.909+0000 7f7a389f1740 1 Processor -- start 2024-03-21T10:28:36.913 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:36.910+0000 7f7a389f1740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55cc72e4b0f0 0x55cc72e5fea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:28:36.913 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:36.910+0000 7f7a389f1740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55cc72e4bdd0 0x55cc72e4e190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:28:36.913 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:36.910+0000 7f7a389f1740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55cc72e54460 0x55cc72e4e6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:28:36.913 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:36.910+0000 7f7a389f1740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a301be770 con 0x55cc72e4bdd0 2024-03-21T10:28:36.913 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:36.910+0000 7f7a389f1740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a301be110 con 0x55cc72e4b0f0 2024-03-21T10:28:36.913 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:36.910+0000 7f7a389f1740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a301bb610 con 0x55cc72e54460 2024-03-21T10:28:36.913 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:36.910+0000 7f7a389f1740 10 client.0 _ll_register_callbacks cb 0x7f7a1c00db50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T10:28:36.913 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:36.910+0000 7f7a2e000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55cc72e4b0f0 0x55cc72e5fea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:28:36.913 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:36.910+0000 7f7a2ea00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55cc72e4bdd0 0x55cc72e4e190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:28:36.913 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:36.910+0000 7f7a2ea00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55cc72e4bdd0 0x55cc72e4e190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:60780/0 (socket says 172.21.15.47:60780) 2024-03-21T10:28:36.913 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:36.910+0000 7f7a2ea00640 1 -- 172.21.15.47:0/365058872 learned_addr learned my addr 172.21.15.47:0/365058872 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:28:36.913 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:36.911+0000 7f7a2ea00640 1 -- 172.21.15.47:0/365058872 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55cc72e54460 msgr2=0x55cc72e4e6d0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-03-21T10:28:36.914 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:36.911+0000 7f7a2ea00640 1 --2- 172.21.15.47:0/365058872 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55cc72e54460 0x55cc72e4e6d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:28:36.914 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:36.911+0000 7f7a2ea00640 1 -- 172.21.15.47:0/365058872 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55cc72e4b0f0 msgr2=0x55cc72e5fea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:28:36.914 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:36.911+0000 7f7a2ea00640 1 --2- 172.21.15.47:0/365058872 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55cc72e4b0f0 0x55cc72e5fea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:28:36.914 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:36.911+0000 7f7a2ea00640 1 -- 172.21.15.47:0/365058872 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55cc72e4ed10 con 0x55cc72e4bdd0 2024-03-21T10:28:36.914 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:36.911+0000 7f7a2e000640 1 --2- 172.21.15.47:0/365058872 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55cc72e4b0f0 0x55cc72e5fea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T10:28:36.914 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:36.911+0000 7f7a2ea00640 1 --2- 172.21.15.47:0/365058872 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55cc72e4bdd0 0x55cc72e4e190 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7f7a20009bc0 tx=0x7f7a200097c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7eb5060fc26fbad7 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:28:36.914 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:36.911+0000 7f7a18c00640 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-03-21T10:28:36.914 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:36.911+0000 7f7a18c00640 1 -- 172.21.15.47:0/365058872 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a20003a00 con 0x55cc72e4bdd0 2024-03-21T10:28:36.914 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:36.911+0000 7f7a18c00640 1 -- 172.21.15.47:0/365058872 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(97..97 src has 1..97) v4 ==== 9474+0+0 (secure 0 0 0) 0x7f7a20005040 con 0x55cc72e4bdd0 2024-03-21T10:28:36.914 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:36.911+0000 7f7a389f1740 1 -- 172.21.15.47:0/365058872 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f7a1c01d3a0 con 0x55cc72e4bdd0 2024-03-21T10:28:36.914 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:36.912+0000 7f7a18c00640 1 -- 172.21.15.47:0/365058872 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7a20009e70 con 0x55cc72e4bdd0 2024-03-21T10:28:36.915 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:36.912+0000 7f7a389f1740 10 client.16821 fetch_fsmap learned FSMap version 32 2024-03-21T10:28:36.915 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:36.912+0000 7f7a389f1740 1 -- 172.21.15.47:0/365058872 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x55cc72ebfc20 con 0x55cc72e4bdd0 2024-03-21T10:28:36.915 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:36.912+0000 7f7a18c00640 1 -- 172.21.15.47:0/365058872 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f7a1c01d3a0 con 0x55cc72e4bdd0 2024-03-21T10:28:36.915 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:36.912+0000 7f7a18c00640 1 -- 172.21.15.47:0/365058872 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 32) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f7a2000f040 con 0x55cc72e4bdd0 2024-03-21T10:28:36.915 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:36.912+0000 7f7a389f1740 10 client.16821 fetch_fsmap finished waiting for FSMap version 32 2024-03-21T10:28:36.915 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:36.912+0000 7f7a389f1740 10 client.16821 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-21T10:28:36.915 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:36.912+0000 7f7a389f1740 20 client.16821 populate_metadata read hostname 'smithi047' 2024-03-21T10:28:36.915 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:36.912+0000 7f7a389f1740 1 --2- 172.21.15.47:0/365058872 >> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] conn(0x55cc72ebffb0 0x55cc72ec23d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:28:36.915 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:36.912+0000 7f7a389f1740 4 client.16821 mds_command: new command op to 34535 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-21T10:28:36.915 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:36.912+0000 7f7a389f1740 1 -- 172.21.15.47:0/365058872 --> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55cc72db6ae0 con 0x55cc72ebffb0 2024-03-21T10:28:36.917 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:36.913+0000 7f7a2e000640 1 --2- 172.21.15.47:0/365058872 >> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] conn(0x55cc72ebffb0 0x55cc72ec23d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:28:36.917 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:36.913+0000 7f7a2e000640 1 --2- 172.21.15.47:0/365058872 >> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] conn(0x55cc72ebffb0 0x55cc72ec23d0 crc :-1 s=READY pgs=120 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=56ba5198699dff24 server_cookie=1251d85cad6034ae in_seq=0 out_seq=0 2024-03-21T10:28:36.917 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:36.913+0000 7f7a18c00640 10 client.16821 ms_handle_connect on v2:172.21.15.100:6838/4173168440 2024-03-21T10:28:36.918 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:36.914+0000 7f7a18c00640 1 -- 172.21.15.47:0/365058872 <== mds.0 v2:172.21.15.100:6838/4173168440 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x55cc72db6ae0 con 0x55cc72ebffb0 2024-03-21T10:28:36.918 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:36.914+0000 7f7a18c00640 10 client.16821 handle_command_reply: tid=0 2024-03-21T10:28:36.918 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:36.914+0000 7f7a389f1740 1 client.16821 shutdown 2024-03-21T10:28:36.918 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:36.914+0000 7f7a389f1740 1 -- 172.21.15.47:0/365058872 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55cc72e4bdd0 msgr2=0x55cc72e4e190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:28:36.918 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:36.914+0000 7f7a389f1740 1 --2- 172.21.15.47:0/365058872 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55cc72e4bdd0 0x55cc72e4e190 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7f7a20009bc0 tx=0x7f7a200097c0 comp rx=0 tx=0).stop 2024-03-21T10:28:36.918 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:36.914+0000 7f7a389f1740 1 -- 172.21.15.47:0/365058872 shutdown_connections 2024-03-21T10:28:36.918 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:36.914+0000 7f7a389f1740 1 --2- 172.21.15.47:0/365058872 >> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] conn(0x55cc72ebffb0 0x55cc72ec23d0 crc :-1 s=READY pgs=120 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:28:36.918 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:36.914+0000 7f7a389f1740 1 --2- 172.21.15.47:0/365058872 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55cc72e54460 0x55cc72e4e6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:28:36.919 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:36.914+0000 7f7a389f1740 1 --2- 172.21.15.47:0/365058872 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55cc72e4bdd0 0x55cc72e4e190 unknown :-1 s=CLOSED pgs=754 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:28:36.919 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:36.914+0000 7f7a389f1740 1 --2- 172.21.15.47:0/365058872 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55cc72e4b0f0 0x55cc72e5fea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:28:36.919 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:36.914+0000 7f7a389f1740 1 -- 172.21.15.47:0/365058872 >> 172.21.15.47:0/365058872 conn(0x55cc72e12200 msgr2=0x55cc72e4f7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:28:36.919 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:36.914+0000 7f7a18c00640 0 client.16821 ms_handle_reset on v2:172.21.15.100:6838/4173168440 2024-03-21T10:28:36.919 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:36.914+0000 7f7a389f1740 1 -- 172.21.15.47:0/365058872 shutdown_connections 2024-03-21T10:28:36.919 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:36.914+0000 7f7a389f1740 1 -- 172.21.15.47:0/365058872 wait complete. 2024-03-21T10:28:36.919 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:36.914+0000 7f7a389f1740 20 client.16821 trim_cache size 0 max 16384 2024-03-21T10:28:36.919 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-03-21T10:28:36.919 INFO:teuthology.orchestra.run.smithi047.stdout: "status": "IDLE (waiting for more scrubs)", 2024-03-21T10:28:36.919 INFO:teuthology.orchestra.run.smithi047.stdout: "scrubs": { 2024-03-21T10:28:36.920 INFO:teuthology.orchestra.run.smithi047.stdout: "9db5396b-62a3-497a-afb3-c986901483c9": { 2024-03-21T10:28:36.920 INFO:teuthology.orchestra.run.smithi047.stdout: "path": "/", 2024-03-21T10:28:36.920 INFO:teuthology.orchestra.run.smithi047.stdout: "tag": "9db5396b-62a3-497a-afb3-c986901483c9", 2024-03-21T10:28:36.920 INFO:teuthology.orchestra.run.smithi047.stdout: "options": "recursive,force" 2024-03-21T10:28:36.920 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-21T10:28:36.920 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-21T10:28:36.920 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-03-21T10:28:36.921 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:36.915+0000 7f7a2fe00640 1 -- 172.21.15.47:0/286050139 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f7a140a31d0 msgr2=0x7f7a140a5690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:28:36.921 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:36.915+0000 7f7a2fe00640 1 --2- 172.21.15.47:0/286050139 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f7a140a31d0 0x7f7a140a5690 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f7a280bad90 tx=0x7f7a2802bef0 comp rx=0 tx=0).stop 2024-03-21T10:28:36.921 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:36.915+0000 7f7a2fe00640 1 -- 172.21.15.47:0/286050139 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7a30109730 msgr2=0x7f7a30076630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:28:36.921 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:36.915+0000 7f7a2fe00640 1 --2- 172.21.15.47:0/286050139 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7a30109730 0x7f7a30076630 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f7a28036040 tx=0x7f7a28033c40 comp rx=0 tx=0).stop 2024-03-21T10:28:36.921 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:36.915+0000 7f7a2fe00640 1 -- 172.21.15.47:0/286050139 shutdown_connections 2024-03-21T10:28:36.921 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:36.915+0000 7f7a2fe00640 1 --2- 172.21.15.47:0/286050139 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7a30109730 0x7f7a30076630 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:28:36.921 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:36.915+0000 7f7a2fe00640 1 --2- 172.21.15.47:0/286050139 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f7a140a31d0 0x7f7a140a5690 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:28:36.921 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:36.915+0000 7f7a2fe00640 1 --2- 172.21.15.47:0/286050139 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7a301040e0 0x7f7a300760f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:28:36.921 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:36.915+0000 7f7a2fe00640 1 --2- 172.21.15.47:0/286050139 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7a30103720 0x7f7a30075bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:28:36.921 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:36.915+0000 7f7a2fe00640 1 -- 172.21.15.47:0/286050139 >> 172.21.15.47:0/286050139 conn(0x7f7a300ff1b0 msgr2=0x7f7a300ffa50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:28:36.921 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:36.915+0000 7f7a2fe00640 1 -- 172.21.15.47:0/286050139 shutdown_connections 2024-03-21T10:28:36.921 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:28:36.915+0000 7f7a2fe00640 1 -- 172.21.15.47:0/286050139 wait complete. 2024-03-21T10:28:36.932 INFO:tasks.cephfs.filesystem:scrub status for tag:9db5396b-62a3-497a-afb3-c986901483c9 - {'path': '/', 'tag': '9db5396b-62a3-497a-afb3-c986901483c9', 'options': 'recursive,force'} 2024-03-21T10:28:37.712 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:28:37 smithi047 ceph-mon[31547]: pgmap v102: 129 pgs: 129 active+clean; 441 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 9.1 MiB/s wr, 2.02k op/s 2024-03-21T10:28:37.713 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:28:37 smithi047 ceph-mon[31547]: scrub queued for path: / 2024-03-21T10:28:37.713 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:28:37 smithi047 ceph-mon[31547]: scrub summary: idle+waiting paths [/] 2024-03-21T10:28:37.713 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:28:37 smithi047 ceph-mon[31547]: scrub summary: active paths [/] 2024-03-21T10:28:37.713 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:28:37 smithi047 ceph-mon[31547]: scrub summary: idle+waiting paths [/] 2024-03-21T10:28:37.728 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:28:37 smithi100 ceph-mon[35193]: pgmap v102: 129 pgs: 129 active+clean; 441 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 9.1 MiB/s wr, 2.02k op/s 2024-03-21T10:28:37.728 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:28:37 smithi100 ceph-mon[35193]: scrub queued for path: / 2024-03-21T10:28:37.728 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:28:37 smithi100 ceph-mon[35193]: scrub summary: idle+waiting paths [/] 2024-03-21T10:28:37.728 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:28:37 smithi100 ceph-mon[35193]: scrub summary: active paths [/] 2024-03-21T10:28:37.728 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:28:37 smithi100 ceph-mon[35193]: scrub summary: idle+waiting paths [/] 2024-03-21T10:28:37.738 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:28:37 smithi157 ceph-mon[35581]: pgmap v102: 129 pgs: 129 active+clean; 441 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 9.1 MiB/s wr, 2.02k op/s 2024-03-21T10:28:37.738 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:28:37 smithi157 ceph-mon[35581]: scrub queued for path: / 2024-03-21T10:28:37.738 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:28:37 smithi157 ceph-mon[35581]: scrub summary: idle+waiting paths [/] 2024-03-21T10:28:37.738 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:28:37 smithi157 ceph-mon[35581]: scrub summary: active paths [/] 2024-03-21T10:28:37.738 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:28:37 smithi157 ceph-mon[35581]: scrub summary: idle+waiting paths [/] 2024-03-21T10:28:37.947 INFO:tasks.workunit.client.0.smithi047.stdout:setting up random write file 2024-03-21T10:28:37.947 INFO:tasks.workunit.client.0.smithi047.stdout:done setting up random write file 2024-03-21T10:28:37.948 INFO:tasks.workunit.client.0.smithi047.stdout:starting fsync run 2024-03-21T10:28:37.948 INFO:tasks.workunit.client.0.smithi047.stdout:write time: 0.0204s fsync time: 0.0285s 2024-03-21T10:28:37.948 INFO:tasks.workunit.client.0.smithi047.stdout:write time: 0.0050s fsync time: 0.0549s 2024-03-21T10:28:37.948 INFO:tasks.workunit.client.0.smithi047.stdout:write time: 0.0058s fsync time: 0.0615s 2024-03-21T10:28:37.948 INFO:tasks.workunit.client.0.smithi047.stdout:write time: 0.0052s fsync time: 0.0752s 2024-03-21T10:28:37.948 INFO:tasks.workunit.client.0.smithi047.stdout:write time: 0.0035s fsync time: 0.0234s 2024-03-21T10:28:37.948 INFO:tasks.workunit.client.0.smithi047.stdout:write time: 0.0047s fsync time: 0.0239s 2024-03-21T10:28:37.948 INFO:tasks.workunit.client.0.smithi047.stdout:write time: 0.0046s fsync time: 0.0198s 2024-03-21T10:28:37.948 INFO:tasks.workunit.client.0.smithi047.stdout:write time: 0.0035s fsync time: 0.0815s 2024-03-21T10:28:37.948 INFO:tasks.workunit.client.0.smithi047.stdout:write time: 0.0055s fsync time: 0.0337s 2024-03-21T10:28:37.948 INFO:tasks.workunit.client.0.smithi047.stdout:write time: 0.0046s fsync time: 0.0332s 2024-03-21T10:28:37.948 INFO:tasks.workunit.client.0.smithi047.stdout:write time: 0.0032s fsync time: 0.0199s 2024-03-21T10:28:37.948 INFO:tasks.workunit.client.0.smithi047.stdout:write time: 0.0058s fsync time: 0.0597s 2024-03-21T10:28:37.948 INFO:tasks.workunit.client.0.smithi047.stdout:write time: 0.0050s fsync time: 0.0414s 2024-03-21T10:28:37.949 INFO:tasks.workunit.client.0.smithi047.stdout:write time: 0.0060s fsync time: 0.0480s 2024-03-21T10:28:37.949 INFO:tasks.workunit.client.0.smithi047.stdout:write time: 0.0044s fsync time: 0.2342s 2024-03-21T10:28:37.949 INFO:tasks.workunit.client.0.smithi047.stdout:write time: 0.0032s fsync time: 0.0317s 2024-03-21T10:28:37.949 INFO:tasks.workunit.client.0.smithi047.stdout:write time: 0.0045s fsync time: 0.0754s 2024-03-21T10:28:37.949 INFO:tasks.workunit.client.0.smithi047.stdout:write time: 0.0064s fsync time: 0.0371s 2024-03-21T10:28:37.949 INFO:tasks.workunit.client.0.smithi047.stdout:write time: 0.0049s fsync time: 0.0592s 2024-03-21T10:28:37.949 INFO:tasks.workunit.client.0.smithi047.stdout:write time: 0.0052s fsync time: 0.0336s 2024-03-21T10:28:37.949 INFO:tasks.workunit.client.0.smithi047.stdout:write time: 0.0064s fsync time: 0.0265s 2024-03-21T10:28:37.949 INFO:tasks.workunit.client.0.smithi047.stdout:write time: 0.0036s fsync time: 0.0542s 2024-03-21T10:28:37.949 INFO:tasks.workunit.client.0.smithi047.stdout:write time: 0.0260s fsync time: 0.0214s 2024-03-21T10:28:37.949 INFO:tasks.workunit.client.0.smithi047.stdout:write time: 0.0037s fsync time: 0.0254s 2024-03-21T10:28:37.949 INFO:tasks.workunit.client.0.smithi047.stdout:write time: 0.0042s fsync time: 0.0230s 2024-03-21T10:28:37.949 INFO:tasks.workunit.client.0.smithi047.stdout:write time: 0.0034s fsync time: 0.0619s 2024-03-21T10:28:37.949 INFO:tasks.workunit.client.0.smithi047.stdout:write time: 0.0040s fsync time: 0.0829s 2024-03-21T10:28:37.950 INFO:tasks.workunit.client.0.smithi047.stdout:write time: 0.0035s fsync time: 0.0478s 2024-03-21T10:28:37.950 INFO:tasks.workunit.client.0.smithi047.stdout:write time: 0.0030s fsync time: 0.0593s 2024-03-21T10:28:37.950 INFO:tasks.workunit.client.0.smithi047.stdout:write time: 0.0039s fsync time: 0.0799s 2024-03-21T10:28:37.950 INFO:tasks.workunit.client.0.smithi047.stdout:write time: 0.0051s fsync time: 0.0414s 2024-03-21T10:28:37.950 INFO:tasks.workunit.client.0.smithi047.stdout:write time: 0.0038s fsync time: 0.0624s 2024-03-21T10:28:37.950 INFO:tasks.workunit.client.0.smithi047.stdout:write time: 0.0038s fsync time: 0.0280s 2024-03-21T10:28:37.950 INFO:tasks.workunit.client.0.smithi047.stdout:write time: 0.0042s fsync time: 0.0540s 2024-03-21T10:28:37.950 INFO:tasks.workunit.client.0.smithi047.stdout:write time: 0.0024s fsync time: 0.2210s 2024-03-21T10:28:37.950 INFO:tasks.workunit.client.0.smithi047.stdout:write time: 0.0058s fsync time: 0.0204s 2024-03-21T10:28:37.950 INFO:tasks.workunit.client.0.smithi047.stdout:write time: 0.0045s fsync time: 0.0407s 2024-03-21T10:28:37.950 INFO:tasks.workunit.client.0.smithi047.stdout:write time: 0.0045s fsync time: 0.0793s 2024-03-21T10:28:37.950 INFO:tasks.workunit.client.0.smithi047.stdout:write time: 0.0063s fsync time: 0.0636s 2024-03-21T10:28:37.950 INFO:tasks.workunit.client.0.smithi047.stdout:write time: 0.0040s fsync time: 0.0318s 2024-03-21T10:28:37.951 INFO:tasks.workunit.client.0.smithi047.stdout:write time: 0.0034s fsync time: 0.0411s 2024-03-21T10:28:37.951 INFO:tasks.workunit.client.0.smithi047.stdout:write time: 0.0033s fsync time: 0.0871s 2024-03-21T10:28:37.951 INFO:tasks.workunit.client.0.smithi047.stdout:write time: 0.0059s fsync time: 0.0511s 2024-03-21T10:28:37.951 INFO:tasks.workunit.client.0.smithi047.stdout:write time: 0.0053s fsync time: 0.0741s 2024-03-21T10:28:37.951 INFO:tasks.workunit.client.0.smithi047.stdout:write time: 0.0038s fsync time: 0.1803s 2024-03-21T10:28:37.951 INFO:tasks.workunit.client.0.smithi047.stdout:write time: 0.0057s fsync time: 0.1188s 2024-03-21T10:28:37.951 INFO:tasks.workunit.client.0.smithi047.stdout:write time: 0.0050s fsync time: 0.0524s 2024-03-21T10:28:37.951 INFO:tasks.workunit.client.0.smithi047.stdout:write time: 0.0063s fsync time: 0.0421s 2024-03-21T10:28:37.951 INFO:tasks.workunit.client.0.smithi047.stdout:write time: 0.0025s fsync time: 0.0601s 2024-03-21T10:28:37.951 INFO:tasks.workunit.client.0.smithi047.stdout:write time: 0.0038s fsync time: 0.0508s 2024-03-21T10:28:37.951 INFO:tasks.workunit.client.0.smithi047.stdout:write time: 0.0040s fsync time: 0.0539s 2024-03-21T10:28:37.951 INFO:tasks.workunit.client.0.smithi047.stdout:write time: 0.0058s fsync time: 0.0623s 2024-03-21T10:28:37.951 INFO:tasks.workunit.client.0.smithi047.stdout:write time: 0.0051s fsync time: 0.0768s 2024-03-21T10:28:37.951 INFO:tasks.workunit.client.0.smithi047.stdout:write time: 0.0056s fsync time: 0.0366s 2024-03-21T10:28:37.952 INFO:tasks.workunit.client.0.smithi047.stdout:write time: 0.0051s fsync time: 0.0912s 2024-03-21T10:28:37.952 INFO:tasks.workunit.client.0.smithi047.stdout:write time: 0.0037s fsync time: 0.0974s 2024-03-21T10:28:37.952 INFO:tasks.workunit.client.0.smithi047.stdout:write time: 0.0062s fsync time: 0.0525s 2024-03-21T10:28:37.952 INFO:tasks.workunit.client.0.smithi047.stdout:write time: 0.0046s fsync time: 0.0665s 2024-03-21T10:28:37.952 INFO:tasks.workunit.client.0.smithi047.stdout:run done 58 fsyncs total, killing random writer 2024-03-21T10:28:37.958 INFO:tasks.workunit.client.0.smithi047.stdout:/home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin 2024-03-21T10:28:37.958 INFO:tasks.workunit.client.0.smithi047.stderr:+ echo /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin 2024-03-21T10:28:37.958 INFO:tasks.workunit.client.0.smithi047.stderr:+ whereis lsof 2024-03-21T10:28:38.002 INFO:tasks.workunit.client.0.smithi047.stdout:lsof: /usr/bin/lsof /usr/share/man/man1/lsof.1.gz 2024-03-21T10:28:38.002 INFO:tasks.workunit.client.0.smithi047.stderr:+ lsof 2024-03-21T10:28:38.033 INFO:tasks.workunit.client.0.smithi047.stderr:lsof: WARNING: can't stat() overlay file system /var/lib/containers/storage/overlay/10ab6f66d550c87573f945f803c4f92650c76c2142b200e308dd5734436cc6e8/merged 2024-03-21T10:28:38.034 INFO:tasks.workunit.client.0.smithi047.stderr: Output information may be incomplete. 2024-03-21T10:28:38.034 INFO:tasks.workunit.client.0.smithi047.stderr:lsof: WARNING: can't stat() overlay file system /var/lib/containers/storage/overlay/4193f3bc34f41e5fb90c176b8210c5c797208467c6f372b377075907f20e100c/merged 2024-03-21T10:28:38.034 INFO:tasks.workunit.client.0.smithi047.stderr: Output information may be incomplete. 2024-03-21T10:28:38.036 INFO:tasks.workunit.client.0.smithi047.stderr:lsof: WARNING: can't stat() overlay file system /var/lib/containers/storage/overlay/fa63c9447d222c5abaf50fb70c1cc34ea356854abf3426eb0c8a7f015a95e543/merged 2024-03-21T10:28:38.036 INFO:tasks.workunit.client.0.smithi047.stderr: Output information may be incomplete. 2024-03-21T10:28:38.037 INFO:tasks.workunit.client.0.smithi047.stderr:lsof: WARNING: can't stat() overlay file system /var/lib/containers/storage/overlay/cab8c1c11ddc44b3f7251050e29e61c724a07774891943f3f169428cf41f19ce/merged 2024-03-21T10:28:38.037 INFO:tasks.workunit.client.0.smithi047.stderr: Output information may be incomplete. 2024-03-21T10:28:38.038 INFO:tasks.workunit.client.0.smithi047.stderr:lsof: WARNING: can't stat() overlay file system /var/lib/containers/storage/overlay/8bc14524d7f02db40a42ab65f069f5682514962833182f9de83d3508887ef2e7/merged 2024-03-21T10:28:38.038 INFO:tasks.workunit.client.0.smithi047.stderr: Output information may be incomplete. 2024-03-21T10:28:38.039 INFO:tasks.workunit.client.0.smithi047.stderr:lsof: WARNING: can't stat() overlay file system /var/lib/containers/storage/overlay/225ab22923163649720ee3c99e303986cfdc6df3ece1f25ae6bea1c7386ea563/merged 2024-03-21T10:28:38.039 INFO:tasks.workunit.client.0.smithi047.stderr: Output information may be incomplete. 2024-03-21T10:28:38.041 INFO:tasks.workunit.client.0.smithi047.stderr:lsof: WARNING: can't stat() overlay file system /var/lib/containers/storage/overlay/a670d6b1797f7b111d0422282a9f2fc87507a063f306170c368d80948f2e4034/merged 2024-03-21T10:28:38.041 INFO:tasks.workunit.client.0.smithi047.stderr: Output information may be incomplete. 2024-03-21T10:28:38.042 INFO:tasks.workunit.client.0.smithi047.stderr:lsof: WARNING: can't stat() overlay file system /var/lib/containers/storage/overlay/d41f794868e6e4c013a6f5180d396f28874ea4f7b53dc71022d9b672a7f62644/merged 2024-03-21T10:28:38.042 INFO:tasks.workunit.client.0.smithi047.stderr: Output information may be incomplete. 2024-03-21T10:28:38.043 INFO:tasks.workunit.client.0.smithi047.stderr:lsof: WARNING: can't stat() overlay file system /var/lib/containers/storage/overlay/d2f5e9cf3dce6353de30b11e940b05e7d1241191f5cbdf874907eb57db79323c/merged 2024-03-21T10:28:38.043 INFO:tasks.workunit.client.0.smithi047.stderr: Output information may be incomplete. 2024-03-21T10:28:38.044 INFO:tasks.workunit.client.0.smithi047.stderr:lsof: WARNING: can't stat() overlay file system /var/lib/containers/storage/overlay/decd4140254122fd73d80c489f7134415c0f4da62e417409ccd425e24cd125cd/merged 2024-03-21T10:28:38.044 INFO:tasks.workunit.client.0.smithi047.stderr: Output information may be incomplete. 2024-03-21T10:28:38.277 INFO:tasks.workunit.client.0.smithi047.stdout:COMMAND PID TID TASKCMD USER FD TYPE DEVICE SIZE/OFF NODE NAME 2024-03-21T10:28:38.277 INFO:tasks.workunit.client.0.smithi047.stdout:systemd 1 root cwd unknown /proc/1/cwd (readlink: Permission denied) 2024-03-21T10:28:38.277 INFO:tasks.workunit.client.0.smithi047.stdout:systemd 1 root rtd unknown /proc/1/root (readlink: Permission denied) 2024-03-21T10:28:38.278 INFO:tasks.workunit.client.0.smithi047.stdout:systemd 1 root txt unknown /proc/1/exe (readlink: Permission denied) 2024-03-21T10:28:38.278 INFO:tasks.workunit.client.0.smithi047.stdout:systemd 1 root NOFD /proc/1/fd (opendir: Permission denied) 2024-03-21T10:28:38.278 INFO:tasks.workunit.client.0.smithi047.stdout:kthreadd 2 root cwd unknown /proc/2/cwd (readlink: Permission denied) 2024-03-21T10:28:38.278 INFO:tasks.workunit.client.0.smithi047.stdout:kthreadd 2 root rtd unknown /proc/2/root (readlink: Permission denied) 2024-03-21T10:28:38.278 INFO:tasks.workunit.client.0.smithi047.stdout:kthreadd 2 root txt unknown /proc/2/exe (readlink: Permission denied) 2024-03-21T10:28:38.278 INFO:tasks.workunit.client.0.smithi047.stdout:kthreadd 2 root NOFD /proc/2/fd (opendir: Permission denied) 2024-03-21T10:28:38.278 INFO:tasks.workunit.client.0.smithi047.stdout:pool_work 3 root cwd unknown /proc/3/cwd (readlink: Permission denied) 2024-03-21T10:28:38.278 INFO:tasks.workunit.client.0.smithi047.stdout:pool_work 3 root rtd unknown /proc/3/root (readlink: Permission denied) 2024-03-21T10:28:38.278 INFO:tasks.workunit.client.0.smithi047.stdout:pool_work 3 root txt unknown /proc/3/exe (readlink: Permission denied) 2024-03-21T10:28:38.278 INFO:tasks.workunit.client.0.smithi047.stdout:pool_work 3 root NOFD /proc/3/fd (opendir: Permission denied) 2024-03-21T10:28:38.278 INFO:tasks.workunit.client.0.smithi047.stdout:kworker/R 4 root cwd unknown /proc/4/cwd (readlink: Permission denied) 2024-03-21T10:28:38.278 INFO:tasks.workunit.client.0.smithi047.stdout:kworker/R 4 root rtd unknown /proc/4/root (readlink: Permission denied) 2024-03-21T10:28:38.278 INFO:tasks.workunit.client.0.smithi047.stdout:kworker/R 4 root txt unknown /proc/4/exe (readlink: Permission denied) 2024-03-21T10:28:38.279 INFO:tasks.workunit.client.0.smithi047.stdout:kworker/R 4 root NOFD /proc/4/fd (opendir: Permission denied) 2024-03-21T10:28:38.279 INFO:tasks.workunit.client.0.smithi047.stdout:kworker/R 5 root cwd unknown /proc/5/cwd (readlink: Permission denied) 2024-03-21T10:28:38.279 INFO:tasks.workunit.client.0.smithi047.stdout:kworker/R 5 root rtd unknown /proc/5/root (readlink: Permission denied) 2024-03-21T10:28:38.279 INFO:tasks.workunit.client.0.smithi047.stdout:kworker/R 5 root txt unknown /proc/5/exe (readlink: Permission denied) 2024-03-21T10:28:38.279 INFO:tasks.workunit.client.0.smithi047.stdout:kworker/R 5 root NOFD /proc/5/fd (opendir: Permission denied) 2024-03-21T10:28:38.279 INFO:tasks.workunit.client.0.smithi047.stdout:kworker/R 6 root cwd unknown /proc/6/cwd (readlink: Permission denied) 2024-03-21T10:28:38.279 INFO:tasks.workunit.client.0.smithi047.stdout:kworker/R 6 root rtd unknown /proc/6/root (readlink: Permission denied) 2024-03-21T10:28:38.279 INFO:tasks.workunit.client.0.smithi047.stdout:kworker/R 6 root txt unknown /proc/6/exe (readlink: Permission denied) 2024-03-21T10:28:38.279 INFO:tasks.workunit.client.0.smithi047.stdout:kworker/R 6 root NOFD /proc/6/fd (opendir: Permission denied) 2024-03-21T10:28:38.279 INFO:tasks.workunit.client.0.smithi047.stdout:kworker/R 7 root cwd unknown /proc/7/cwd (readlink: Permission denied) 2024-03-21T10:28:38.279 INFO:tasks.workunit.client.0.smithi047.stdout:kworker/R 7 root rtd unknown /proc/7/root (readlink: Permission denied) 2024-03-21T10:28:38.279 INFO:tasks.workunit.client.0.smithi047.stdout:kworker/R 7 root txt unknown /proc/7/exe (readlink: Permission denied) 2024-03-21T10:28:38.282 INFO:tasks.workunit.client.0.smithi047.stdout:kworker/R 7 root NOFD /proc/7/fd (opendir: Permission denied) 2024-03-21T10:28:38.282 INFO:tasks.workunit.client.0.smithi047.stdout:kworker/0 10 root cwd unknown /proc/10/cwd (readlink: Permission denied) 2024-03-21T10:28:38.283 INFO:tasks.workunit.client.0.smithi047.stdout:kworker/0 10 root rtd unknown /proc/10/root (readlink: Permission denied) 2024-03-21T10:28:38.283 INFO:tasks.workunit.client.0.smithi047.stdout:kworker/0 10 root txt unknown /proc/10/exe (readlink: Permission denied) 2024-03-21T10:28:38.283 INFO:tasks.workunit.client.0.smithi047.stdout:kworker/0 10 root NOFD /proc/10/fd (opendir: Permission denied) 2024-03-21T10:28:38.283 INFO:tasks.workunit.client.0.smithi047.stdout:kworker/u 11 root cwd unknown /proc/11/cwd (readlink: Permission denied) 2024-03-21T10:28:38.283 INFO:tasks.workunit.client.0.smithi047.stdout:kworker/u 11 root rtd unknown /proc/11/root (readlink: Permission denied) 2024-03-21T10:28:38.283 INFO:tasks.workunit.client.0.smithi047.stdout:kworker/u 11 root txt unknown /proc/11/exe (readlink: Permission denied) 2024-03-21T10:28:38.283 INFO:tasks.workunit.client.0.smithi047.stdout:kworker/u 11 root NOFD /proc/11/fd (opendir: Permission denied) 2024-03-21T10:28:38.283 INFO:tasks.workunit.client.0.smithi047.stdout:kworker/R 12 root cwd unknown /proc/12/cwd (readlink: Permission denied) 2024-03-21T10:28:38.283 INFO:tasks.workunit.client.0.smithi047.stdout:kworker/R 12 root rtd unknown /proc/12/root (readlink: Permission denied) 2024-03-21T10:28:38.283 INFO:tasks.workunit.client.0.smithi047.stdout:kworker/R 12 root txt unknown /proc/12/exe (readlink: Permission denied) 2024-03-21T10:28:38.283 INFO:tasks.workunit.client.0.smithi047.stdout:kworker/R 12 root NOFD /proc/12/fd (opendir: Permission denied) 2024-03-21T10:28:38.283 INFO:tasks.workunit.client.0.smithi047.stdout:rcu_tasks 13 root cwd unknown /proc/13/cwd (readlink: Permission denied) 2024-03-21T10:28:38.283 INFO:tasks.workunit.client.0.smithi047.stdout:rcu_tasks 13 root rtd unknown /proc/13/root (readlink: Permission denied) 2024-03-21T10:28:38.283 INFO:tasks.workunit.client.0.smithi047.stdout:rcu_tasks 13 root txt unknown /proc/13/exe (readlink: Permission denied) 2024-03-21T10:28:38.284 INFO:tasks.workunit.client.0.smithi047.stdout:rcu_tasks 13 root NOFD /proc/13/fd (opendir: Permission denied) 2024-03-21T10:28:38.284 INFO:tasks.workunit.client.0.smithi047.stdout:rcu_tasks 14 root cwd unknown /proc/14/cwd (readlink: Permission denied) 2024-03-21T10:28:38.284 INFO:tasks.workunit.client.0.smithi047.stdout:rcu_tasks 14 root rtd unknown /proc/14/root (readlink: Permission denied) 2024-03-21T10:28:38.284 INFO:tasks.workunit.client.0.smithi047.stdout:rcu_tasks 14 root txt unknown /proc/14/exe (readlink: Permission denied) 2024-03-21T10:28:38.284 INFO:tasks.workunit.client.0.smithi047.stdout:rcu_tasks 14 root NOFD /proc/14/fd (opendir: Permission denied) 2024-03-21T10:28:38.284 INFO:tasks.workunit.client.0.smithi047.stdout:rcu_tasks 15 root cwd unknown /proc/15/cwd (readlink: Permission denied) 2024-03-21T10:28:38.284 INFO:tasks.workunit.client.0.smithi047.stdout:rcu_tasks 15 root rtd unknown /proc/15/root (readlink: Permission denied) 2024-03-21T10:28:38.284 INFO:tasks.workunit.client.0.smithi047.stdout:rcu_tasks 15 root txt unknown /proc/15/exe (readlink: Permission denied) 2024-03-21T10:28:38.284 INFO:tasks.workunit.client.0.smithi047.stdout:rcu_tasks 15 root NOFD /proc/15/fd (opendir: Permission denied) 2024-03-21T10:28:38.284 INFO:tasks.workunit.client.0.smithi047.stdout:ksoftirqd 16 root cwd unknown /proc/16/cwd (readlink: Permission denied) 2024-03-21T10:28:38.284 INFO:tasks.workunit.client.0.smithi047.stdout:ksoftirqd 16 root rtd unknown /proc/16/root (readlink: Permission denied) 2024-03-21T10:28:38.284 INFO:tasks.workunit.client.0.smithi047.stdout:ksoftirqd 16 root txt unknown /proc/16/exe (readlink: Permission denied) 2024-03-21T10:28:38.284 INFO:tasks.workunit.client.0.smithi047.stdout:ksoftirqd 16 root NOFD /proc/16/fd (opendir: Permission denied) 2024-03-21T10:28:38.285 INFO:tasks.workunit.client.0.smithi047.stdout:rcu_preem 17 root cwd unknown /proc/17/cwd (readlink: Permission denied) 2024-03-21T10:28:38.285 INFO:tasks.workunit.client.0.smithi047.stdout:rcu_preem 17 root rtd unknown /proc/17/root (readlink: Permission denied) 2024-03-21T10:28:38.285 INFO:tasks.workunit.client.0.smithi047.stdout:rcu_preem 17 root txt unknown /proc/17/exe (readlink: Permission denied) 2024-03-21T10:28:38.285 INFO:tasks.workunit.client.0.smithi047.stdout:rcu_preem 17 root NOFD /proc/17/fd (opendir: Permission denied) 2024-03-21T10:28:38.285 INFO:tasks.workunit.client.0.smithi047.stdout:migration 18 root cwd unknown /proc/18/cwd (readlink: Permission denied) 2024-03-21T10:28:38.285 INFO:tasks.workunit.client.0.smithi047.stdout:migration 18 root rtd unknown /proc/18/root (readlink: Permission denied) 2024-03-21T10:28:38.285 INFO:tasks.workunit.client.0.smithi047.stdout:migration 18 root txt unknown /proc/18/exe (readlink: Permission denied) 2024-03-21T10:28:38.285 INFO:tasks.workunit.client.0.smithi047.stdout:migration 18 root NOFD /proc/18/fd (opendir: Permission denied) 2024-03-21T10:28:38.285 INFO:tasks.workunit.client.0.smithi047.stdout:cpuhp/0 19 root cwd unknown /proc/19/cwd (readlink: Permission denied) 2024-03-21T10:28:38.285 INFO:tasks.workunit.client.0.smithi047.stdout:cpuhp/0 19 root rtd unknown /proc/19/root (readlink: Permission denied) 2024-03-21T10:28:38.285 INFO:tasks.workunit.client.0.smithi047.stdout:cpuhp/0 19 root txt unknown /proc/19/exe (readlink: Permission denied) 2024-03-21T10:28:38.285 INFO:tasks.workunit.client.0.smithi047.stdout:cpuhp/0 19 root NOFD /proc/19/fd (opendir: Permission denied) 2024-03-21T10:28:38.285 INFO:tasks.workunit.client.0.smithi047.stdout:cpuhp/1 20 root cwd unknown /proc/20/cwd (readlink: Permission denied) 2024-03-21T10:28:38.285 INFO:tasks.workunit.client.0.smithi047.stdout:cpuhp/1 20 root rtd unknown /proc/20/root (readlink: Permission denied) 2024-03-21T10:28:38.286 INFO:tasks.workunit.client.0.smithi047.stdout:cpuhp/1 20 root txt unknown /proc/20/exe (readlink: Permission denied) 2024-03-21T10:28:38.286 INFO:tasks.workunit.client.0.smithi047.stdout:cpuhp/1 20 root NOFD /proc/20/fd (opendir: Permission denied) 2024-03-21T10:28:38.286 INFO:tasks.workunit.client.0.smithi047.stdout:migration 21 root cwd unknown /proc/21/cwd (readlink: Permission denied) 2024-03-21T10:28:38.286 INFO:tasks.workunit.client.0.smithi047.stdout:migration 21 root rtd unknown /proc/21/root (readlink: Permission denied) 2024-03-21T10:28:38.286 INFO:tasks.workunit.client.0.smithi047.stdout:migration 21 root txt unknown /proc/21/exe (readlink: Permission denied) 2024-03-21T10:28:38.286 INFO:tasks.workunit.client.0.smithi047.stdout:migration 21 root NOFD /proc/21/fd (opendir: Permission denied) 2024-03-21T10:28:38.286 INFO:tasks.workunit.client.0.smithi047.stdout:ksoftirqd 22 root cwd unknown /proc/22/cwd (readlink: Permission denied) 2024-03-21T10:28:38.286 INFO:tasks.workunit.client.0.smithi047.stdout:ksoftirqd 22 root rtd unknown /proc/22/root (readlink: Permission denied) 2024-03-21T10:28:38.286 INFO:tasks.workunit.client.0.smithi047.stdout:ksoftirqd 22 root txt unknown /proc/22/exe (readlink: Permission denied) 2024-03-21T10:28:38.286 INFO:tasks.workunit.client.0.smithi047.stdout:ksoftirqd 22 root NOFD /proc/22/fd (opendir: Permission denied) 2024-03-21T10:28:38.286 INFO:tasks.workunit.client.0.smithi047.stdout:kworker/1 24 root cwd unknown /proc/24/cwd (readlink: Permission denied) 2024-03-21T10:28:38.286 INFO:tasks.workunit.client.0.smithi047.stdout:kworker/1 24 root rtd unknown /proc/24/root (readlink: Permission denied) 2024-03-21T10:28:38.286 INFO:tasks.workunit.client.0.smithi047.stdout:kworker/1 24 root txt unknown /proc/24/exe (readlink: Permission denied) 2024-03-21T10:28:38.287 INFO:tasks.workunit.client.0.smithi047.stdout:kworker/1 24 root NOFD /proc/24/fd (opendir: Permission denied) 2024-03-21T10:28:38.287 INFO:tasks.workunit.client.0.smithi047.stdout:cpuhp/2 25 root cwd unknown /proc/25/cwd (readlink: Permission denied) 2024-03-21T10:28:38.287 INFO:tasks.workunit.client.0.smithi047.stdout:cpuhp/2 25 root rtd unknown /proc/25/root (readlink: Permission denied) 2024-03-21T10:28:38.287 INFO:tasks.workunit.client.0.smithi047.stdout:cpuhp/2 25 root txt unknown /proc/25/exe (readlink: Permission denied) 2024-03-21T10:28:38.287 INFO:tasks.workunit.client.0.smithi047.stdout:cpuhp/2 25 root NOFD /proc/25/fd (opendir: Permission denied) 2024-03-21T10:28:38.287 INFO:tasks.workunit.client.0.smithi047.stdout:migration 26 root cwd unknown /proc/26/cwd (readlink: Permission denied) 2024-03-21T10:28:38.287 INFO:tasks.workunit.client.0.smithi047.stdout:migration 26 root rtd unknown /proc/26/root (readlink: Permission denied) 2024-03-21T10:28:38.287 INFO:tasks.workunit.client.0.smithi047.stdout:migration 26 root txt unknown /proc/26/exe (readlink: Permission denied) 2024-03-21T10:28:38.287 INFO:tasks.workunit.client.0.smithi047.stdout:migration 26 root NOFD /proc/26/fd (opendir: Permission denied) 2024-03-21T10:28:38.287 INFO:tasks.workunit.client.0.smithi047.stdout:ksoftirqd 27 root cwd unknown /proc/27/cwd (readlink: Permission denied) 2024-03-21T10:28:38.287 INFO:tasks.workunit.client.0.smithi047.stdout:ksoftirqd 27 root rtd unknown /proc/27/root (readlink: Permission denied) 2024-03-21T10:28:38.287 INFO:tasks.workunit.client.0.smithi047.stdout:ksoftirqd 27 root txt unknown /proc/27/exe (readlink: Permission denied) 2024-03-21T10:28:38.287 INFO:tasks.workunit.client.0.smithi047.stdout:ksoftirqd 27 root NOFD /proc/27/fd (opendir: Permission denied) 2024-03-21T10:28:38.288 INFO:tasks.workunit.client.0.smithi047.stdout:kworker/2 29 root cwd unknown /proc/29/cwd (readlink: Permission denied) 2024-03-21T10:28:38.288 INFO:tasks.workunit.client.0.smithi047.stdout:kworker/2 29 root rtd unknown /proc/29/root (readlink: Permission denied) 2024-03-21T10:28:38.288 INFO:tasks.workunit.client.0.smithi047.stdout:kworker/2 29 root txt unknown /proc/29/exe (readlink: Permission denied) 2024-03-21T10:28:38.288 INFO:tasks.workunit.client.0.smithi047.stdout:kworker/2 29 root NOFD /proc/29/fd (opendir: Permission denied) 2024-03-21T10:28:38.288 INFO:tasks.workunit.client.0.smithi047.stdout:cpuhp/3 30 root cwd unknown /proc/30/cwd (readlink: Permission denied) 2024-03-21T10:28:38.288 INFO:tasks.workunit.client.0.smithi047.stdout:cpuhp/3 30 root rtd unknown /proc/30/root (readlink: Permission denied) 2024-03-21T10:28:38.288 INFO:tasks.workunit.client.0.smithi047.stdout:cpuhp/3 30 root txt unknown /proc/30/exe (readlink: Permission denied) 2024-03-21T10:28:38.288 INFO:tasks.workunit.client.0.smithi047.stdout:cpuhp/3 30 root NOFD /proc/30/fd (opendir: Permission denied) 2024-03-21T10:28:38.288 INFO:tasks.workunit.client.0.smithi047.stdout:migration 31 root cwd unknown /proc/31/cwd (readlink: Permission denied) 2024-03-21T10:28:38.288 INFO:tasks.workunit.client.0.smithi047.stdout:migration 31 root rtd unknown /proc/31/root (readlink: Permission denied) 2024-03-21T10:28:38.288 INFO:tasks.workunit.client.0.smithi047.stdout:migration 31 root txt unknown /proc/31/exe (readlink: Permission denied) 2024-03-21T10:28:38.288 INFO:tasks.workunit.client.0.smithi047.stdout:migration 31 root NOFD /proc/31/fd (opendir: Permission denied) 2024-03-21T10:28:38.289 INFO:tasks.workunit.client.0.smithi047.stdout:ksoftirqd 32 root cwd unknown /proc/32/cwd (readlink: Permission denied) 2024-03-21T10:28:38.289 INFO:tasks.workunit.client.0.smithi047.stdout:ksoftirqd 32 root rtd unknown /proc/32/root (readlink: Permission denied) 2024-03-21T10:28:38.289 INFO:tasks.workunit.client.0.smithi047.stdout:ksoftirqd 32 root txt unknown /proc/32/exe (readlink: Permission denied) 2024-03-21T10:28:38.289 INFO:tasks.workunit.client.0.smithi047.stdout:ksoftirqd 32 root NOFD /proc/32/fd (opendir: Permission denied) 2024-03-21T10:28:38.289 INFO:tasks.workunit.client.0.smithi047.stdout:kworker/3 34 root cwd unknown /proc/34/cwd (readlink: Permission denied) 2024-03-21T10:28:38.289 INFO:tasks.workunit.client.0.smithi047.stdout:kworker/3 34 root rtd unknown /proc/34/root (readlink: Permission denied) 2024-03-21T10:28:38.289 INFO:tasks.workunit.client.0.smithi047.stdout:kworker/3 34 root txt unknown /proc/34/exe (readlink: Permission denied) 2024-03-21T10:28:38.289 INFO:tasks.workunit.client.0.smithi047.stdout:kworker/3 34 root NOFD /proc/34/fd (opendir: Permission denied) 2024-03-21T10:28:38.289 INFO:tasks.workunit.client.0.smithi047.stdout:cpuhp/4 35 root cwd unknown /proc/35/cwd (readlink: Permission denied) 2024-03-21T10:28:38.289 INFO:tasks.workunit.client.0.smithi047.stdout:cpuhp/4 35 root rtd unknown /proc/35/root (readlink: Permission denied) 2024-03-21T10:28:38.289 INFO:tasks.workunit.client.0.smithi047.stdout:cpuhp/4 35 root txt unknown /proc/35/exe (readlink: Permission denied) 2024-03-21T10:28:38.289 INFO:tasks.workunit.client.0.smithi047.stdout:cpuhp/4 35 root NOFD /proc/35/fd (opendir: Permission denied) 2024-03-21T10:28:38.289 INFO:tasks.workunit.client.0.smithi047.stdout:migration 36 root cwd unknown /proc/36/cwd (readlink: Permission denied) 2024-03-21T10:28:38.289 INFO:tasks.workunit.client.0.smithi047.stdout:migration 36 root rtd unknown /proc/36/root (readlink: Permission denied) 2024-03-21T10:28:38.290 INFO:tasks.workunit.client.0.smithi047.stdout:migration 36 root txt unknown /proc/36/exe (readlink: Permission denied) 2024-03-21T10:28:38.290 INFO:tasks.workunit.client.0.smithi047.stdout:migration 36 root NOFD /proc/36/fd (opendir: Permission denied) 2024-03-21T10:28:38.290 INFO:tasks.workunit.client.0.smithi047.stdout:ksoftirqd 37 root cwd unknown /proc/37/cwd (readlink: Permission denied) 2024-03-21T10:28:38.290 INFO:tasks.workunit.client.0.smithi047.stdout:ksoftirqd 37 root rtd unknown /proc/37/root (readlink: Permission denied) 2024-03-21T10:28:38.290 INFO:tasks.workunit.client.0.smithi047.stdout:ksoftirqd 37 root txt unknown /proc/37/exe (readlink: Permission denied) 2024-03-21T10:28:38.290 INFO:tasks.workunit.client.0.smithi047.stdout:ksoftirqd 37 root NOFD /proc/37/fd (opendir: Permission denied) 2024-03-21T10:28:38.290 INFO:tasks.workunit.client.0.smithi047.stdout:kworker/4 39 root cwd unknown /proc/39/cwd (readlink: Permission denied) 2024-03-21T10:28:38.290 INFO:tasks.workunit.client.0.smithi047.stdout:kworker/4 39 root rtd unknown /proc/39/root (readlink: Permission denied) 2024-03-21T10:28:38.290 INFO:tasks.workunit.client.0.smithi047.stdout:kworker/4 39 root txt unknown /proc/39/exe (readlink: Permission denied) 2024-03-21T10:28:38.290 INFO:tasks.workunit.client.0.smithi047.stdout:kworker/4 39 root NOFD /proc/39/fd (opendir: Permission denied) 2024-03-21T10:28:38.290 INFO:tasks.workunit.client.0.smithi047.stdout:cpuhp/5 40 root cwd unknown /proc/40/cwd (readlink: Permission denied) 2024-03-21T10:28:38.290 INFO:tasks.workunit.client.0.smithi047.stdout:cpuhp/5 40 root rtd unknown /proc/40/root (readlink: Permission denied) 2024-03-21T10:28:38.291 INFO:tasks.workunit.client.0.smithi047.stdout:cpuhp/5 40 root txt unknown /proc/40/exe (readlink: Permission denied) 2024-03-21T10:28:38.291 INFO:tasks.workunit.client.0.smithi047.stdout:cpuhp/5 40 root NOFD /proc/40/fd (opendir: Permission denied) 2024-03-21T10:28:38.291 INFO:tasks.workunit.client.0.smithi047.stdout:migration 41 root cwd unknown /proc/41/cwd (readlink: Permission denied) 2024-03-21T10:28:38.291 INFO:tasks.workunit.client.0.smithi047.stdout:migration 41 root rtd unknown /proc/41/root (readlink: Permission denied) 2024-03-21T10:28:38.291 INFO:tasks.workunit.client.0.smithi047.stdout:migration 41 root txt unknown /proc/41/exe (readlink: Permission denied) 2024-03-21T10:28:38.291 INFO:tasks.workunit.client.0.smithi047.stdout:migration 41 root NOFD /proc/41/fd (opendir: Permission denied) 2024-03-21T10:28:38.291 INFO:tasks.workunit.client.0.smithi047.stdout:ksoftirqd 42 root cwd unknown /proc/42/cwd (readlink: Permission denied) 2024-03-21T10:28:38.291 INFO:tasks.workunit.client.0.smithi047.stdout:ksoftirqd 42 root rtd unknown /proc/42/root (readlink: Permission denied) 2024-03-21T10:28:38.291 INFO:tasks.workunit.client.0.smithi047.stdout:ksoftirqd 42 root txt unknown /proc/42/exe (readlink: Permission denied) 2024-03-21T10:28:38.291 INFO:tasks.workunit.client.0.smithi047.stdout:ksoftirqd 42 root NOFD /proc/42/fd (opendir: Permission denied) 2024-03-21T10:28:38.291 INFO:tasks.workunit.client.0.smithi047.stdout:kworker/5 44 root cwd unknown /proc/44/cwd (readlink: Permission denied) 2024-03-21T10:28:38.292 INFO:tasks.workunit.client.0.smithi047.stdout:kworker/5 44 root rtd unknown /proc/44/root (readlink: Permission denied) 2024-03-21T10:28:38.292 INFO:tasks.workunit.client.0.smithi047.stdout:kworker/5 44 root txt unknown /proc/44/exe (readlink: Permission denied) 2024-03-21T10:28:38.292 INFO:tasks.workunit.client.0.smithi047.stdout:kworker/5 44 root NOFD /proc/44/fd (opendir: Permission denied) 2024-03-21T10:28:38.292 INFO:tasks.workunit.client.0.smithi047.stdout:cpuhp/6 45 root cwd unknown /proc/45/cwd (readlink: Permission denied) 2024-03-21T10:28:38.292 INFO:tasks.workunit.client.0.smithi047.stdout:cpuhp/6 45 root rtd unknown /proc/45/root (readlink: Permission denied) 2024-03-21T10:28:38.292 INFO:tasks.workunit.client.0.smithi047.stdout:cpuhp/6 45 root txt unknown /proc/45/exe (readlink: Permission denied) 2024-03-21T10:28:38.292 INFO:tasks.workunit.client.0.smithi047.stdout:cpuhp/6 45 root NOFD /proc/45/fd (opendir: Permission denied) 2024-03-21T10:28:38.292 INFO:tasks.workunit.client.0.smithi047.stdout:migration 46 root cwd unknown /proc/46/cwd (readlink: Permission denied) 2024-03-21T10:28:38.292 INFO:tasks.workunit.client.0.smithi047.stdout:migration 46 root rtd unknown /proc/46/root (readlink: Permission denied) 2024-03-21T10:28:38.292 INFO:tasks.workunit.client.0.smithi047.stdout:migration 46 root txt unknown /proc/46/exe (readlink: Permission denied) 2024-03-21T10:28:38.292 INFO:tasks.workunit.client.0.smithi047.stdout:migration 46 root NOFD /proc/46/fd (opendir: Permission denied) 2024-03-21T10:28:38.292 INFO:tasks.workunit.client.0.smithi047.stdout:ksoftirqd 47 root cwd unknown /proc/47/cwd (readlink: Permission denied) 2024-03-21T10:28:38.293 INFO:tasks.workunit.client.0.smithi047.stdout:ksoftirqd 47 root rtd unknown /proc/47/root (readlink: Permission denied) 2024-03-21T10:28:38.293 INFO:tasks.workunit.client.0.smithi047.stdout:ksoftirqd 47 root txt unknown /proc/47/exe (readlink: Permission denied) 2024-03-21T10:28:38.293 INFO:tasks.workunit.client.0.smithi047.stdout:ksoftirqd 47 root NOFD /proc/47/fd (opendir: Permission denied) 2024-03-21T10:28:38.293 INFO:tasks.workunit.client.0.smithi047.stdout:kworker/6 49 root cwd unknown /proc/49/cwd (readlink: Permission denied) 2024-03-21T10:28:38.293 INFO:tasks.workunit.client.0.smithi047.stdout:kworker/6 49 root rtd unknown /proc/49/root (readlink: Permission denied) 2024-03-21T10:28:38.293 INFO:tasks.workunit.client.0.smithi047.stdout:kworker/6 49 root txt unknown /proc/49/exe (readlink: Permission denied) 2024-03-21T10:28:38.293 INFO:tasks.workunit.client.0.smithi047.stdout:kworker/6 49 root NOFD /proc/49/fd (opendir: Permission denied) 2024-03-21T10:28:38.293 INFO:tasks.workunit.client.0.smithi047.stdout:cpuhp/7 50 root cwd unknown /proc/50/cwd (readlink: Permission denied) 2024-03-21T10:28:38.293 INFO:tasks.workunit.client.0.smithi047.stdout:cpuhp/7 50 root rtd unknown /proc/50/root (readlink: Permission denied) 2024-03-21T10:28:38.294 INFO:tasks.workunit.client.0.smithi047.stdout:cpuhp/7 50 root txt unknown /proc/50/exe (readlink: Permission denied) 2024-03-21T10:28:38.294 INFO:tasks.workunit.client.0.smithi047.stdout:cpuhp/7 50 root NOFD /proc/50/fd (opendir: Permission denied) 2024-03-21T10:28:38.294 INFO:tasks.workunit.client.0.smithi047.stdout:migration 51 root cwd unknown /proc/51/cwd (readlink: Permission denied) 2024-03-21T10:28:38.294 INFO:tasks.workunit.client.0.smithi047.stdout:migration 51 root rtd unknown /proc/51/root (readlink: Permission denied) 2024-03-21T10:28:38.294 INFO:tasks.workunit.client.0.smithi047.stdout:migration 51 root txt unknown /proc/51/exe (readlink: Permission denied) 2024-03-21T10:28:38.294 INFO:tasks.workunit.client.0.smithi047.stdout:migration 51 root NOFD /proc/51/fd (opendir: Permission denied) 2024-03-21T10:28:38.294 INFO:tasks.workunit.client.0.smithi047.stdout:ksoftirqd 52 root cwd unknown /proc/52/cwd (readlink: Permission denied) 2024-03-21T10:28:38.294 INFO:tasks.workunit.client.0.smithi047.stdout:ksoftirqd 52 root rtd unknown /proc/52/root (readlink: Permission denied) 2024-03-21T10:28:38.294 INFO:tasks.workunit.client.0.smithi047.stdout:ksoftirqd 52 root txt unknown /proc/52/exe (readlink: Permission denied) 2024-03-21T10:28:38.294 INFO:tasks.workunit.client.0.smithi047.stdout:ksoftirqd 52 root NOFD /proc/52/fd (opendir: Permission denied) 2024-03-21T10:28:38.294 INFO:tasks.workunit.client.0.smithi047.stdout:kworker/7 54 root cwd unknown /proc/54/cwd (readlink: Permission denied) 2024-03-21T10:28:38.295 INFO:tasks.workunit.client.0.smithi047.stdout:kworker/7 54 root rtd unknown /proc/54/root (readlink: Permission denied) 2024-03-21T10:28:38.295 INFO:tasks.workunit.client.0.smithi047.stdout:kworker/7 54 root txt unknown /proc/54/exe (readlink: Permission denied) 2024-03-21T10:28:38.295 INFO:tasks.workunit.client.0.smithi047.stdout:kworker/7 54 root NOFD /proc/54/fd (opendir: Permission denied) 2024-03-21T10:28:38.295 INFO:tasks.workunit.client.0.smithi047.stdout:kdevtmpfs 55 root cwd unknown /proc/55/cwd (readlink: Permission denied) 2024-03-21T10:28:38.295 INFO:tasks.workunit.client.0.smithi047.stdout:kdevtmpfs 55 root rtd unknown /proc/55/root (readlink: Permission denied) 2024-03-21T10:28:38.295 INFO:tasks.workunit.client.0.smithi047.stdout:kdevtmpfs 55 root txt unknown /proc/55/exe (readlink: Permission denied) 2024-03-21T10:28:38.295 INFO:tasks.workunit.client.0.smithi047.stdout:kdevtmpfs 55 root NOFD /proc/55/fd (opendir: Permission denied) 2024-03-21T10:28:38.295 INFO:tasks.workunit.client.0.smithi047.stdout:kworker/R 56 root cwd unknown /proc/56/cwd (readlink: Permission denied) 2024-03-21T10:28:38.295 INFO:tasks.workunit.client.0.smithi047.stdout:kworker/R 56 root rtd unknown /proc/56/root (readlink: Permission denied) 2024-03-21T10:28:38.295 INFO:tasks.workunit.client.0.smithi047.stdout:kworker/R 56 root txt unknown /proc/56/exe (readlink: Permission denied) 2024-03-21T10:28:38.295 INFO:tasks.workunit.client.0.smithi047.stdout:kworker/R 56 root NOFD /proc/56/fd (opendir: Permission denied) 2024-03-21T10:28:38.295 INFO:tasks.workunit.client.0.smithi047.stdout:kauditd 57 root cwd unknown /proc/57/cwd (readlink: Permission denied) 2024-03-21T10:28:38.296 INFO:tasks.workunit.client.0.smithi047.stdout:kauditd 57 root rtd unknown /proc/57/root (readlink: Permission denied) 2024-03-21T10:28:38.296 INFO:tasks.workunit.client.0.smithi047.stdout:kauditd 57 root txt unknown /proc/57/exe (readlink: Permission denied) 2024-03-21T10:28:38.296 INFO:tasks.workunit.client.0.smithi047.stdout:kauditd 57 root NOFD /proc/57/fd (opendir: Permission denied) 2024-03-21T10:28:38.296 INFO:tasks.workunit.client.0.smithi047.stdout:khungtask 58 root cwd unknown /proc/58/cwd (readlink: Permission denied) 2024-03-21T10:28:38.296 INFO:tasks.workunit.client.0.smithi047.stdout:khungtask 58 root rtd unknown /proc/58/root (readlink: Permission denied) 2024-03-21T10:28:38.296 INFO:tasks.workunit.client.0.smithi047.stdout:khungtask 58 root txt unknown /proc/58/exe (readlink: Permission denied) 2024-03-21T10:28:38.296 INFO:tasks.workunit.client.0.smithi047.stdout:khungtask 58 root NOFD /proc/58/fd (opendir: Permission denied) 2024-03-21T10:28:38.296 INFO:tasks.workunit.client.0.smithi047.stdout:oom_reape 60 root cwd unknown /proc/60/cwd (readlink: Permission denied) 2024-03-21T10:28:38.296 INFO:tasks.workunit.client.0.smithi047.stdout:oom_reape 60 root rtd unknown /proc/60/root (readlink: Permission denied) 2024-03-21T10:28:38.296 INFO:tasks.workunit.client.0.smithi047.stdout:oom_reape 60 root txt unknown /proc/60/exe (readlink: Permission denied) 2024-03-21T10:28:38.296 INFO:tasks.workunit.client.0.smithi047.stdout:oom_reape 60 root NOFD /proc/60/fd (opendir: Permission denied) 2024-03-21T10:28:38.296 INFO:tasks.workunit.client.0.smithi047.stdout:kworker/R 62 root cwd unknown /proc/62/cwd (readlink: Permission denied) 2024-03-21T10:28:38.297 INFO:tasks.workunit.client.0.smithi047.stdout:kworker/R 62 root rtd unknown /proc/62/root (readlink: Permission denied) 2024-03-21T10:28:38.297 INFO:tasks.workunit.client.0.smithi047.stdout:kworker/R 62 root txt unknown /proc/62/exe (readlink: Permission denied) 2024-03-21T10:28:38.297 INFO:tasks.workunit.client.0.smithi047.stdout:kworker/R 62 root NOFD /proc/62/fd (opendir: Permission denied) 2024-03-21T10:28:38.297 INFO:tasks.workunit.client.0.smithi047.stdout:kcompactd 63 root cwd unknown /proc/63/cwd (readlink: Permission denied) 2024-03-21T10:28:38.297 INFO:tasks.workunit.client.0.smithi047.stdout:kcompactd 63 root rtd unknown /proc/63/root (readlink: Permission denied) 2024-03-21T10:28:38.297 INFO:tasks.workunit.client.0.smithi047.stdout:kcompactd 63 root txt unknown /proc/63/exe (readlink: Permission denied) 2024-03-21T10:28:38.297 INFO:tasks.workunit.client.0.smithi047.stdout:kcompactd 63 root NOFD /proc/63/fd (opendir: Permission denied) 2024-03-21T10:28:38.297 INFO:tasks.workunit.client.0.smithi047.stdout:ksmd 64 root cwd unknown /proc/64/cwd (readlink: Permission denied) 2024-03-21T10:28:38.297 INFO:tasks.workunit.client.0.smithi047.stdout:ksmd 64 root rtd unknown /proc/64/root (readlink: Permission denied) 2024-03-21T10:28:38.297 INFO:tasks.workunit.client.0.smithi047.stdout:ksmd 64 root txt unknown /proc/64/exe (readlink: Permission denied) 2024-03-21T10:28:38.297 INFO:tasks.workunit.client.0.smithi047.stdout:ksmd 64 root NOFD /proc/64/fd (opendir: Permission denied) 2024-03-21T10:28:38.298 INFO:tasks.workunit.client.0.smithi047.stdout:khugepage 65 root cwd unknown /proc/65/cwd (readlink: Permission denied) 2024-03-21T10:28:38.298 INFO:tasks.workunit.client.0.smithi047.stdout:khugepage 65 root rtd unknown /proc/65/root (readlink: Permission denied) 2024-03-21T10:28:38.298 INFO:tasks.workunit.client.0.smithi047.stdout:khugepage 65 root txt unknown /proc/65/exe (readlink: Permission denied) 2024-03-21T10:28:38.298 INFO:tasks.workunit.client.0.smithi047.stdout:khugepage 65 root NOFD /proc/65/fd (opendir: Permission denied) 2024-03-21T10:28:38.298 INFO:tasks.workunit.client.0.smithi047.stdout:kworker/R 66 root cwd unknown /proc/66/cwd (readlink: Permission denied) 2024-03-21T10:28:38.298 INFO:tasks.workunit.client.0.smithi047.stdout:kworker/R 66 root rtd unknown /proc/66/root (readlink: Permission denied) 2024-03-21T10:28:38.298 INFO:tasks.workunit.client.0.smithi047.stdout:kworker/R 66 root txt unknown /proc/66/exe (readlink: Permission denied) 2024-03-21T10:28:38.298 INFO:tasks.workunit.client.0.smithi047.stdout:kworker/R 66 root NOFD /proc/66/fd (opendir: Permission denied) 2024-03-21T10:28:38.298 INFO:tasks.workunit.client.0.smithi047.stdout:kworker/R 67 root cwd unknown /proc/67/cwd (readlink: Permission denied) 2024-03-21T10:28:38.298 INFO:tasks.workunit.client.0.smithi047.stdout:kworker/R 67 root rtd unknown /proc/67/root (readlink: Permission denied) 2024-03-21T10:28:38.298 INFO:tasks.workunit.client.0.smithi047.stdout:kworker/R 67 root txt unknown /proc/67/exe (readlink: Permission denied) 2024-03-21T10:28:38.298 INFO:tasks.workunit.client.0.smithi047.stdout:kworker/R 67 root NOFD /proc/67/fd (opendir: Permission denied) 2024-03-21T10:28:38.298 INFO:tasks.workunit.client.0.smithi047.stdout:kworker/R 68 root cwd unknown /proc/68/cwd (readlink: Permission denied) 2024-03-21T10:28:38.299 INFO:tasks.workunit.client.0.smithi047.stdout:kworker/R 68 root rtd unknown /proc/68/root (readlink: Permission denied) 2024-03-21T10:28:38.299 INFO:tasks.workunit.client.0.smithi047.stdout:kworker/R 68 root txt unknown /proc/68/exe (readlink: Permission denied) 2024-03-21T10:28:38.299 INFO:tasks.workunit.client.0.smithi047.stdout:kworker/R 68 root NOFD /proc/68/fd (opendir: Permission denied) 2024-03-21T10:28:38.299 INFO:tasks.workunit.client.0.smithi047.stdout:kworker/R 69 root cwd unknown /proc/69/cwd (readlink: Permission denied) 2024-03-21T10:28:38.299 INFO:tasks.workunit.client.0.smithi047.stdout:kworker/R 69 root rtd unknown /proc/69/root (readlink: Permission denied) 2024-03-21T10:28:38.299 INFO:tasks.workunit.client.0.smithi047.stdout:kworker/R 69 root txt unknown /proc/69/exe (readlink: Permission denied) 2024-03-21T10:28:38.299 INFO:tasks.workunit.client.0.smithi047.stdout:kworker/R 69 root NOFD /proc/69/fd (opendir: Permission denied) 2024-03-21T10:28:38.299 INFO:tasks.workunit.client.0.smithi047.stdout:kworker/3 71 root cwd unknown /proc/71/cwd (readlink: Permission denied) 2024-03-21T10:28:38.299 INFO:tasks.workunit.client.0.smithi047.stdout:kworker/3 71 root rtd unknown /proc/71/root (readlink: Permission denied) 2024-03-21T10:28:38.299 INFO:tasks.workunit.client.0.smithi047.stdout:kworker/3 71 root txt unknown /proc/71/exe (readlink: Permission denied) 2024-03-21T10:28:38.299 INFO:tasks.workunit.client.0.smithi047.stdout:kworker/3 71 root NOFD /proc/71/fd (opendir: Permission denied) 2024-03-21T10:28:38.299 INFO:tasks.workunit.client.0.smithi047.stdout:irq/9-acp 77 root cwd unknown /proc/77/cwd (readlink: Permission denied) 2024-03-21T10:28:38.300 INFO:tasks.workunit.client.0.smithi047.stdout:irq/9-acp 77 root rtd unknown /proc/77/root (readlink: Permission denied) 2024-03-21T10:28:38.300 INFO:tasks.workunit.client.0.smithi047.stdout:irq/9-acp 77 root txt unknown /proc/77/exe (readlink: Permission denied) 2024-03-21T10:28:38.300 INFO:tasks.workunit.client.0.smithi047.stdout:irq/9-acp 77 root NOFD /proc/77/fd (opendir: Permission denied) 2024-03-21T10:28:38.300 INFO:tasks.workunit.client.0.smithi047.stdout:kworker/R 78 root cwd unknown /proc/78/cwd (readlink: Permission denied) 2024-03-21T10:28:38.300 INFO:tasks.workunit.client.0.smithi047.stdout:kworker/R 78 root rtd unknown /proc/78/root (readlink: Permission denied) 2024-03-21T10:28:38.300 INFO:tasks.workunit.client.0.smithi047.stdout:kworker/R 78 root txt unknown /proc/78/exe (readlink: Permission denied) 2024-03-21T10:28:38.300 INFO:tasks.workunit.client.0.smithi047.stdout:kworker/R 78 root NOFD /proc/78/fd (opendir: Permission denied) 2024-03-21T10:28:38.300 INFO:tasks.workunit.client.0.smithi047.stdout:kworker/R 79 root cwd unknown /proc/79/cwd (readlink: Permission denied) 2024-03-21T10:28:38.300 INFO:tasks.workunit.client.0.smithi047.stdout:kworker/R 79 root rtd unknown /proc/79/root (readlink: Permission denied) 2024-03-21T10:28:38.300 INFO:tasks.workunit.client.0.smithi047.stdout:kworker/R 79 root txt unknown /proc/79/exe (readlink: Permission denied) 2024-03-21T10:28:38.300 INFO:tasks.workunit.client.0.smithi047.stdout:kworker/R 79 root NOFD /proc/79/fd (opendir: Permission denied) 2024-03-21T10:28:38.300 INFO:tasks.workunit.client.0.smithi047.stdout:kworker/R 80 root cwd unknown /proc/80/cwd (readlink: Permission denied) 2024-03-21T10:28:38.301 INFO:tasks.workunit.client.0.smithi047.stdout:kworker/R 80 root rtd unknown /proc/80/root (readlink: Permission denied) 2024-03-21T10:28:38.301 INFO:tasks.workunit.client.0.smithi047.stdout:kworker/R 80 root txt unknown /proc/80/exe (readlink: Permission denied) 2024-03-21T10:28:38.301 INFO:tasks.workunit.client.0.smithi047.stdout:kworker/R 80 root NOFD /proc/80/fd (opendir: Permission denied) 2024-03-21T10:28:38.301 INFO:tasks.workunit.client.0.smithi047.stdout:kworker/R 81 root cwd unknown /proc/81/cwd (readlink: Permission denied) 2024-03-21T10:28:38.301 INFO:tasks.workunit.client.0.smithi047.stdout:kworker/R 81 root rtd unknown /proc/81/root (readlink: Permission denied) 2024-03-21T10:28:38.301 INFO:tasks.workunit.client.0.smithi047.stdout:kworker/R 81 root txt unknown /proc/81/exe (readlink: Permission denied) 2024-03-21T10:28:38.302 INFO:tasks.workunit.client.0.smithi047.stdout:kworker/R 81 root NOFD /proc/81/fd (opendir: Permission denied) 2024-03-21T10:28:38.302 INFO:tasks.workunit.client.0.smithi047.stdout:kworker/R 82 root cwd unknown /proc/82/cwd (readlink: Permission denied) 2024-03-21T10:28:38.302 INFO:tasks.workunit.client.0.smithi047.stdout:kworker/R 82 root rtd unknown /proc/82/root (readlink: Permission denied) 2024-03-21T10:28:38.302 INFO:tasks.workunit.client.0.smithi047.stdout:kworker/R 82 root txt unknown /proc/82/exe (readlink: Permission denied) 2024-03-21T10:28:38.302 INFO:tasks.workunit.client.0.smithi047.stdout:kworker/R 82 root NOFD /proc/82/fd (opendir: Permission denied) 2024-03-21T10:28:38.302 INFO:tasks.workunit.client.0.smithi047.stdout:watchdogd 83 root cwd unknown /proc/83/cwd (readlink: Permission denied) 2024-03-21T10:28:38.302 INFO:tasks.workunit.client.0.smithi047.stdout:watchdogd 83 root rtd unknown /proc/83/root (readlink: Permission denied) 2024-03-21T10:28:38.302 INFO:tasks.workunit.client.0.smithi047.stdout:watchdogd 83 root txt unknown /proc/83/exe (readlink: Permission denied) 2024-03-21T10:28:38.302 INFO:tasks.workunit.client.0.smithi047.stdout:watchdogd 83 root NOFD /proc/83/fd (opendir: Permission denied) 2024-03-21T10:28:38.302 INFO:tasks.workunit.client.0.smithi047.stdout:kworker/0 84 root cwd unknown /proc/84/cwd (readlink: Permission denied) 2024-03-21T10:28:38.303 INFO:tasks.workunit.client.0.smithi047.stdout:kworker/0 84 root rtd unknown /proc/84/root (readlink: Permission denied) 2024-03-21T10:28:38.303 INFO:tasks.workunit.client.0.smithi047.stdout:kworker/0 84 root txt unknown /proc/84/exe (readlink: Permission denied) 2024-03-21T10:28:38.303 INFO:tasks.workunit.client.0.smithi047.stdout:kworker/0 84 root NOFD /proc/84/fd (opendir: Permission denied) 2024-03-21T10:28:38.303 INFO:tasks.workunit.client.0.smithi047.stdout:kswapd0 85 root cwd unknown /proc/85/cwd (readlink: Permission denied) 2024-03-21T10:28:38.303 INFO:tasks.workunit.client.0.smithi047.stdout:kswapd0 85 root rtd unknown /proc/85/root (readlink: Permission denied) 2024-03-21T10:28:38.303 INFO:tasks.workunit.client.0.smithi047.stdout:kswapd0 85 root txt unknown /proc/85/exe (readlink: Permission denied) 2024-03-21T10:28:38.303 INFO:tasks.workunit.client.0.smithi047.stdout:kswapd0 85 root NOFD /proc/85/fd (opendir: Permission denied) 2024-03-21T10:28:38.303 INFO:tasks.workunit.client.0.smithi047.stdout:kworker/R 87 root cwd unknown /proc/87/cwd (readlink: Permission denied) 2024-03-21T10:28:38.303 INFO:tasks.workunit.client.0.smithi047.stdout:kworker/R 87 root rtd unknown /proc/87/root (readlink: Permission denied) 2024-03-21T10:28:38.303 INFO:tasks.workunit.client.0.smithi047.stdout:kworker/R 87 root txt unknown /proc/87/exe (readlink: Permission denied) 2024-03-21T10:28:38.303 INFO:tasks.workunit.client.0.smithi047.stdout:kworker/R 87 root NOFD /proc/87/fd (opendir: Permission denied) 2024-03-21T10:28:38.303 INFO:tasks.workunit.client.0.smithi047.stdout:kworker/R 92 root cwd unknown /proc/92/cwd (readlink: Permission denied) 2024-03-21T10:28:38.303 INFO:tasks.workunit.client.0.smithi047.stdout:kworker/R 92 root rtd unknown /proc/92/root (readlink: Permission denied) 2024-03-21T10:28:38.304 INFO:tasks.workunit.client.0.smithi047.stdout:kworker/R 92 root txt unknown /proc/92/exe (readlink: Permission denied) 2024-03-21T10:28:38.304 INFO:tasks.workunit.client.0.smithi047.stdout:kworker/R 92 root NOFD /proc/92/fd (opendir: Permission denied) 2024-03-21T10:28:38.304 INFO:tasks.workunit.client.0.smithi047.stdout:scsi_eh_0 94 root cwd unknown /proc/94/cwd (readlink: Permission denied) 2024-03-21T10:28:38.304 INFO:tasks.workunit.client.0.smithi047.stdout:scsi_eh_0 94 root rtd unknown /proc/94/root (readlink: Permission denied) 2024-03-21T10:28:38.304 INFO:tasks.workunit.client.0.smithi047.stdout:scsi_eh_0 94 root txt unknown /proc/94/exe (readlink: Permission denied) 2024-03-21T10:28:38.304 INFO:tasks.workunit.client.0.smithi047.stdout:scsi_eh_0 94 root NOFD /proc/94/fd (opendir: Permission denied) 2024-03-21T10:28:38.304 INFO:tasks.workunit.client.0.smithi047.stdout:kworker/R 95 root cwd unknown /proc/95/cwd (readlink: Permission denied) 2024-03-21T10:28:38.304 INFO:tasks.workunit.client.0.smithi047.stdout:kworker/R 95 root rtd unknown /proc/95/root (readlink: Permission denied) 2024-03-21T10:28:38.304 INFO:tasks.workunit.client.0.smithi047.stdout:kworker/R 95 root txt unknown /proc/95/exe (readlink: Permission denied) 2024-03-21T10:28:38.304 INFO:tasks.workunit.client.0.smithi047.stdout:kworker/R 95 root NOFD /proc/95/fd (opendir: Permission denied) 2024-03-21T10:28:38.304 INFO:tasks.workunit.client.0.smithi047.stdout:scsi_eh_1 96 root cwd unknown /proc/96/cwd (readlink: Permission denied) 2024-03-21T10:28:38.304 INFO:tasks.workunit.client.0.smithi047.stdout:scsi_eh_1 96 root rtd unknown /proc/96/root (readlink: Permission denied) 2024-03-21T10:28:38.304 INFO:tasks.workunit.client.0.smithi047.stdout:scsi_eh_1 96 root txt unknown /proc/96/exe (readlink: Permission denied) 2024-03-21T10:28:38.305 INFO:tasks.workunit.client.0.smithi047.stdout:scsi_eh_1 96 root NOFD /proc/96/fd (opendir: Permission denied) 2024-03-21T10:28:38.305 INFO:tasks.workunit.client.0.smithi047.stdout:kworker/R 97 root cwd unknown /proc/97/cwd (readlink: Permission denied) 2024-03-21T10:28:38.305 INFO:tasks.workunit.client.0.smithi047.stdout:kworker/R 97 root rtd unknown /proc/97/root (readlink: Permission denied) 2024-03-21T10:28:38.305 INFO:tasks.workunit.client.0.smithi047.stdout:kworker/R 97 root txt unknown /proc/97/exe (readlink: Permission denied) 2024-03-21T10:28:38.305 INFO:tasks.workunit.client.0.smithi047.stdout:kworker/R 97 root NOFD /proc/97/fd (opendir: Permission denied) 2024-03-21T10:28:38.305 INFO:tasks.workunit.client.0.smithi047.stdout:scsi_eh_2 98 root cwd unknown /proc/98/cwd (readlink: Permission denied) 2024-03-21T10:28:38.305 INFO:tasks.workunit.client.0.smithi047.stdout:scsi_eh_2 98 root rtd unknown /proc/98/root (readlink: Permission denied) 2024-03-21T10:28:38.305 INFO:tasks.workunit.client.0.smithi047.stdout:scsi_eh_2 98 root txt unknown /proc/98/exe (readlink: Permission denied) 2024-03-21T10:28:38.305 INFO:tasks.workunit.client.0.smithi047.stdout:scsi_eh_2 98 root NOFD /proc/98/fd (opendir: Permission denied) 2024-03-21T10:28:38.306 INFO:tasks.workunit.client.0.smithi047.stdout:kworker/R 99 root cwd unknown /proc/99/cwd (readlink: Permission denied) 2024-03-21T10:28:38.306 INFO:tasks.workunit.client.0.smithi047.stdout:kworker/R 99 root rtd unknown /proc/99/root (readlink: Permission denied) 2024-03-21T10:28:38.306 INFO:tasks.workunit.client.0.smithi047.stdout:kworker/R 99 root txt unknown /proc/99/exe (readlink: Permission denied) 2024-03-21T10:28:38.306 INFO:tasks.workunit.client.0.smithi047.stdout:kworker/R 99 root NOFD /proc/99/fd (opendir: Permission denied) 2024-03-21T10:28:38.306 INFO:tasks.workunit.client.0.smithi047.stdout:scsi_eh_3 100 root cwd unknown /proc/100/cwd (readlink: Permission denied) 2024-03-21T10:28:38.306 INFO:tasks.workunit.client.0.smithi047.stdout:scsi_eh_3 100 root rtd unknown /proc/100/root (readlink: Permission denied) 2024-03-21T10:28:38.306 INFO:tasks.workunit.client.0.smithi047.stdout:scsi_eh_3 100 root txt unknown /proc/100/exe (readlink: Permission denied) 2024-03-21T10:28:38.306 INFO:tasks.workunit.client.0.smithi047.stdout:scsi_eh_3 100 root NOFD /proc/100/fd (opendir: Permission denied) 2024-03-21T10:28:38.306 INFO:tasks.workunit.client.0.smithi047.stdout:kworker/R 101 root cwd unknown /proc/101/cwd (readlink: Permission denied) 2024-03-21T10:28:38.306 INFO:tasks.workunit.client.0.smithi047.stdout:kworker/R 101 root rtd unknown /proc/101/root (readlink: Permission denied) 2024-03-21T10:28:38.306 INFO:tasks.workunit.client.0.smithi047.stdout:kworker/R 101 root txt unknown /proc/101/exe (readlink: Permission denied) 2024-03-21T10:28:38.306 INFO:tasks.workunit.client.0.smithi047.stdout:kworker/R 101 root NOFD /proc/101/fd (opendir: Permission denied) 2024-03-21T10:28:38.307 INFO:tasks.workunit.client.0.smithi047.stdout:kworker/u 104 root cwd unknown /proc/104/cwd (readlink: Permission denied) 2024-03-21T10:28:38.307 INFO:tasks.workunit.client.0.smithi047.stdout:kworker/u 104 root rtd unknown /proc/104/root (readlink: Permission denied) 2024-03-21T10:28:38.307 INFO:tasks.workunit.client.0.smithi047.stdout:kworker/u 104 root txt unknown /proc/104/exe (readlink: Permission denied) 2024-03-21T10:28:38.307 INFO:tasks.workunit.client.0.smithi047.stdout:kworker/u 104 root NOFD /proc/104/fd (opendir: Permission denied) 2024-03-21T10:28:38.307 INFO:tasks.workunit.client.0.smithi047.stdout:scsi_eh_4 105 root cwd unknown /proc/105/cwd (readlink: Permission denied) 2024-03-21T10:28:38.307 INFO:tasks.workunit.client.0.smithi047.stdout:scsi_eh_4 105 root rtd unknown /proc/105/root (readlink: Permission denied) 2024-03-21T10:28:38.307 INFO:tasks.workunit.client.0.smithi047.stdout:scsi_eh_4 105 root txt unknown /proc/105/exe (readlink: Permission denied) 2024-03-21T10:28:38.307 INFO:tasks.workunit.client.0.smithi047.stdout:scsi_eh_4 105 root NOFD /proc/105/fd (opendir: Permission denied) 2024-03-21T10:28:38.307 INFO:tasks.workunit.client.0.smithi047.stdout:kworker/R 106 root cwd unknown /proc/106/cwd (readlink: Permission denied) 2024-03-21T10:28:38.307 INFO:tasks.workunit.client.0.smithi047.stdout:kworker/R 106 root rtd unknown /proc/106/root (readlink: Permission denied) 2024-03-21T10:28:38.307 INFO:tasks.workunit.client.0.smithi047.stdout:kworker/R 106 root txt unknown /proc/106/exe (readlink: Permission denied) 2024-03-21T10:28:38.307 INFO:tasks.workunit.client.0.smithi047.stdout:kworker/R 106 root NOFD /proc/106/fd (opendir: Permission denied) 2024-03-21T10:28:38.307 INFO:tasks.workunit.client.0.smithi047.stdout:scsi_eh_5 107 root cwd unknown /proc/107/cwd (readlink: Permission denied) 2024-03-21T10:28:38.307 INFO:tasks.workunit.client.0.smithi047.stdout:scsi_eh_5 107 root rtd unknown /proc/107/root (readlink: Permission denied) 2024-03-21T10:28:38.308 INFO:tasks.workunit.client.0.smithi047.stdout:scsi_eh_5 107 root txt unknown /proc/107/exe (readlink: Permission denied) 2024-03-21T10:28:38.308 INFO:tasks.workunit.client.0.smithi047.stdout:scsi_eh_5 107 root NOFD /proc/107/fd (opendir: Permission denied) 2024-03-21T10:28:38.308 INFO:tasks.workunit.client.0.smithi047.stdout:kworker/R 108 root cwd unknown /proc/108/cwd (readlink: Permission denied) 2024-03-21T10:28:38.308 INFO:tasks.workunit.client.0.smithi047.stdout:kworker/R 108 root rtd unknown /proc/108/root (readlink: Permission denied) 2024-03-21T10:28:38.308 INFO:tasks.workunit.client.0.smithi047.stdout:kworker/R 108 root txt unknown /proc/108/exe (readlink: Permission denied) 2024-03-21T10:28:38.308 INFO:tasks.workunit.client.0.smithi047.stdout:kworker/R 108 root NOFD /proc/108/fd (opendir: Permission denied) 2024-03-21T10:28:38.308 INFO:tasks.workunit.client.0.smithi047.stdout:scsi_eh_6 109 root cwd unknown /proc/109/cwd (readlink: Permission denied) 2024-03-21T10:28:38.308 INFO:tasks.workunit.client.0.smithi047.stdout:scsi_eh_6 109 root rtd unknown /proc/109/root (readlink: Permission denied) 2024-03-21T10:28:38.308 INFO:tasks.workunit.client.0.smithi047.stdout:scsi_eh_6 109 root txt unknown /proc/109/exe (readlink: Permission denied) 2024-03-21T10:28:38.308 INFO:tasks.workunit.client.0.smithi047.stdout:scsi_eh_6 109 root NOFD /proc/109/fd (opendir: Permission denied) 2024-03-21T10:28:38.308 INFO:tasks.workunit.client.0.smithi047.stdout:kworker/R 110 root cwd unknown /proc/110/cwd (readlink: Permission denied) 2024-03-21T10:28:38.308 INFO:tasks.workunit.client.0.smithi047.stdout:kworker/R 110 root rtd unknown /proc/110/root (readlink: Permission denied) 2024-03-21T10:28:38.308 INFO:tasks.workunit.client.0.smithi047.stdout:kworker/R 110 root txt unknown /proc/110/exe (readlink: Permission denied) 2024-03-21T10:28:38.308 INFO:tasks.workunit.client.0.smithi047.stdout:kworker/R 110 root NOFD /proc/110/fd (opendir: Permission denied) 2024-03-21T10:28:38.309 INFO:tasks.workunit.client.0.smithi047.stdout:scsi_eh_7 111 root cwd unknown /proc/111/cwd (readlink: Permission denied) 2024-03-21T10:28:38.309 INFO:tasks.workunit.client.0.smithi047.stdout:scsi_eh_7 111 root rtd unknown /proc/111/root (readlink: Permission denied) 2024-03-21T10:28:38.309 INFO:tasks.workunit.client.0.smithi047.stdout:scsi_eh_7 111 root txt unknown /proc/111/exe (readlink: Permission denied) 2024-03-21T10:28:38.309 INFO:tasks.workunit.client.0.smithi047.stdout:scsi_eh_7 111 root NOFD /proc/111/fd (opendir: Permission denied) 2024-03-21T10:28:38.309 INFO:tasks.workunit.client.0.smithi047.stdout:kworker/R 112 root cwd unknown /proc/112/cwd (readlink: Permission denied) 2024-03-21T10:28:38.309 INFO:tasks.workunit.client.0.smithi047.stdout:kworker/R 112 root rtd unknown /proc/112/root (readlink: Permission denied) 2024-03-21T10:28:38.309 INFO:tasks.workunit.client.0.smithi047.stdout:kworker/R 112 root txt unknown /proc/112/exe (readlink: Permission denied) 2024-03-21T10:28:38.309 INFO:tasks.workunit.client.0.smithi047.stdout:kworker/R 112 root NOFD /proc/112/fd (opendir: Permission denied) 2024-03-21T10:28:38.309 INFO:tasks.workunit.client.0.smithi047.stdout:scsi_eh_8 113 root cwd unknown /proc/113/cwd (readlink: Permission denied) 2024-03-21T10:28:38.309 INFO:tasks.workunit.client.0.smithi047.stdout:scsi_eh_8 113 root rtd unknown /proc/113/root (readlink: Permission denied) 2024-03-21T10:28:38.309 INFO:tasks.workunit.client.0.smithi047.stdout:scsi_eh_8 113 root txt unknown /proc/113/exe (readlink: Permission denied) 2024-03-21T10:28:38.309 INFO:tasks.workunit.client.0.smithi047.stdout:scsi_eh_8 113 root NOFD /proc/113/fd (opendir: Permission denied) 2024-03-21T10:28:38.309 INFO:tasks.workunit.client.0.smithi047.stdout:kworker/R 114 root cwd unknown /proc/114/cwd (readlink: Permission denied) 2024-03-21T10:28:38.310 INFO:tasks.workunit.client.0.smithi047.stdout:kworker/R 114 root rtd unknown /proc/114/root (readlink: Permission denied) 2024-03-21T10:28:38.310 INFO:tasks.workunit.client.0.smithi047.stdout:kworker/R 114 root txt unknown /proc/114/exe (readlink: Permission denied) 2024-03-21T10:28:38.310 INFO:tasks.workunit.client.0.smithi047.stdout:kworker/R 114 root NOFD /proc/114/fd (opendir: Permission denied) 2024-03-21T10:28:38.310 INFO:tasks.workunit.client.0.smithi047.stdout:scsi_eh_9 115 root cwd unknown /proc/115/cwd (readlink: Permission denied) 2024-03-21T10:28:38.310 INFO:tasks.workunit.client.0.smithi047.stdout:scsi_eh_9 115 root rtd unknown /proc/115/root (readlink: Permission denied) 2024-03-21T10:28:38.310 INFO:tasks.workunit.client.0.smithi047.stdout:scsi_eh_9 115 root txt unknown /proc/115/exe (readlink: Permission denied) 2024-03-21T10:28:38.310 INFO:tasks.workunit.client.0.smithi047.stdout:scsi_eh_9 115 root NOFD /proc/115/fd (opendir: Permission denied) 2024-03-21T10:28:38.310 INFO:tasks.workunit.client.0.smithi047.stdout:kworker/R 116 root cwd unknown /proc/116/cwd (readlink: Permission denied) 2024-03-21T10:28:38.310 INFO:tasks.workunit.client.0.smithi047.stdout:kworker/R 116 root rtd unknown /proc/116/root (readlink: Permission denied) 2024-03-21T10:28:38.310 INFO:tasks.workunit.client.0.smithi047.stdout:kworker/R 116 root txt unknown /proc/116/exe (readlink: Permission denied) 2024-03-21T10:28:38.310 INFO:tasks.workunit.client.0.smithi047.stdout:kworker/R 116 root NOFD /proc/116/fd (opendir: Permission denied) 2024-03-21T10:28:38.310 INFO:tasks.workunit.client.0.smithi047.stdout:kworker/u 121 root cwd unknown /proc/121/cwd (readlink: Permission denied) 2024-03-21T10:28:38.311 INFO:tasks.workunit.client.0.smithi047.stdout:kworker/u 121 root rtd unknown /proc/121/root (readlink: Permission denied) 2024-03-21T10:28:38.311 INFO:tasks.workunit.client.0.smithi047.stdout:kworker/u 121 root txt unknown /proc/121/exe (readlink: Permission denied) 2024-03-21T10:28:38.311 INFO:tasks.workunit.client.0.smithi047.stdout:kworker/u 121 root NOFD /proc/121/fd (opendir: Permission denied) 2024-03-21T10:28:38.311 INFO:tasks.workunit.client.0.smithi047.stdout:kworker/u 123 root cwd unknown /proc/123/cwd (readlink: Permission denied) 2024-03-21T10:28:38.311 INFO:tasks.workunit.client.0.smithi047.stdout:kworker/u 123 root rtd unknown /proc/123/root (readlink: Permission denied) 2024-03-21T10:28:38.311 INFO:tasks.workunit.client.0.smithi047.stdout:kworker/u 123 root txt unknown /proc/123/exe (readlink: Permission denied) 2024-03-21T10:28:38.311 INFO:tasks.workunit.client.0.smithi047.stdout:kworker/u 123 root NOFD /proc/123/fd (opendir: Permission denied) 2024-03-21T10:28:38.311 INFO:tasks.workunit.client.0.smithi047.stdout:kworker/R 127 root cwd unknown /proc/127/cwd (readlink: Permission denied) 2024-03-21T10:28:38.311 INFO:tasks.workunit.client.0.smithi047.stdout:kworker/R 127 root rtd unknown /proc/127/root (readlink: Permission denied) 2024-03-21T10:28:38.311 INFO:tasks.workunit.client.0.smithi047.stdout:kworker/R 127 root txt unknown /proc/127/exe (readlink: Permission denied) 2024-03-21T10:28:38.311 INFO:tasks.workunit.client.0.smithi047.stdout:kworker/R 127 root NOFD /proc/127/fd (opendir: Permission denied) 2024-03-21T10:28:38.311 INFO:tasks.workunit.client.0.smithi047.stdout:kworker/5 128 root cwd unknown /proc/128/cwd (readlink: Permission denied) 2024-03-21T10:28:38.311 INFO:tasks.workunit.client.0.smithi047.stdout:kworker/5 128 root rtd unknown /proc/128/root (readlink: Permission denied) 2024-03-21T10:28:38.311 INFO:tasks.workunit.client.0.smithi047.stdout:kworker/5 128 root txt unknown /proc/128/exe (readlink: Permission denied) 2024-03-21T10:28:38.312 INFO:tasks.workunit.client.0.smithi047.stdout:kworker/5 128 root NOFD /proc/128/fd (opendir: Permission denied) 2024-03-21T10:28:38.312 INFO:tasks.workunit.client.0.smithi047.stdout:kworker/R 129 root cwd unknown /proc/129/cwd (readlink: Permission denied) 2024-03-21T10:28:38.312 INFO:tasks.workunit.client.0.smithi047.stdout:kworker/R 129 root rtd unknown /proc/129/root (readlink: Permission denied) 2024-03-21T10:28:38.312 INFO:tasks.workunit.client.0.smithi047.stdout:kworker/R 129 root txt unknown /proc/129/exe (readlink: Permission denied) 2024-03-21T10:28:38.312 INFO:tasks.workunit.client.0.smithi047.stdout:kworker/R 129 root NOFD /proc/129/fd (opendir: Permission denied) 2024-03-21T10:28:38.312 INFO:tasks.workunit.client.0.smithi047.stdout:kworker/2 130 root cwd unknown /proc/130/cwd (readlink: Permission denied) 2024-03-21T10:28:38.312 INFO:tasks.workunit.client.0.smithi047.stdout:kworker/2 130 root rtd unknown /proc/130/root (readlink: Permission denied) 2024-03-21T10:28:38.312 INFO:tasks.workunit.client.0.smithi047.stdout:kworker/2 130 root txt unknown /proc/130/exe (readlink: Permission denied) 2024-03-21T10:28:38.312 INFO:tasks.workunit.client.0.smithi047.stdout:kworker/2 130 root NOFD /proc/130/fd (opendir: Permission denied) 2024-03-21T10:28:38.312 INFO:tasks.workunit.client.0.smithi047.stdout:kworker/R 131 root cwd unknown /proc/131/cwd (readlink: Permission denied) 2024-03-21T10:28:38.312 INFO:tasks.workunit.client.0.smithi047.stdout:kworker/R 131 root rtd unknown /proc/131/root (readlink: Permission denied) 2024-03-21T10:28:38.312 INFO:tasks.workunit.client.0.smithi047.stdout:kworker/R 131 root txt unknown /proc/131/exe (readlink: Permission denied) 2024-03-21T10:28:38.312 INFO:tasks.workunit.client.0.smithi047.stdout:kworker/R 131 root NOFD /proc/131/fd (opendir: Permission denied) 2024-03-21T10:28:38.312 INFO:tasks.workunit.client.0.smithi047.stdout:kworker/u 139 root cwd unknown /proc/139/cwd (readlink: Permission denied) 2024-03-21T10:28:38.313 INFO:tasks.workunit.client.0.smithi047.stdout:kworker/u 139 root rtd unknown /proc/139/root (readlink: Permission denied) 2024-03-21T10:28:38.313 INFO:tasks.workunit.client.0.smithi047.stdout:kworker/u 139 root txt unknown /proc/139/exe (readlink: Permission denied) 2024-03-21T10:28:38.313 INFO:tasks.workunit.client.0.smithi047.stdout:kworker/u 139 root NOFD /proc/139/fd (opendir: Permission denied) 2024-03-21T10:28:38.313 INFO:tasks.workunit.client.0.smithi047.stdout:kworker/6 247 root cwd unknown /proc/247/cwd (readlink: Permission denied) 2024-03-21T10:28:38.313 INFO:tasks.workunit.client.0.smithi047.stdout:kworker/6 247 root rtd unknown /proc/247/root (readlink: Permission denied) 2024-03-21T10:28:38.313 INFO:tasks.workunit.client.0.smithi047.stdout:kworker/6 247 root txt unknown /proc/247/exe (readlink: Permission denied) 2024-03-21T10:28:38.313 INFO:tasks.workunit.client.0.smithi047.stdout:kworker/6 247 root NOFD /proc/247/fd (opendir: Permission denied) 2024-03-21T10:28:38.313 INFO:tasks.workunit.client.0.smithi047.stdout:kworker/3 249 root cwd unknown /proc/249/cwd (readlink: Permission denied) 2024-03-21T10:28:38.313 INFO:tasks.workunit.client.0.smithi047.stdout:kworker/3 249 root rtd unknown /proc/249/root (readlink: Permission denied) 2024-03-21T10:28:38.313 INFO:tasks.workunit.client.0.smithi047.stdout:kworker/3 249 root txt unknown /proc/249/exe (readlink: Permission denied) 2024-03-21T10:28:38.313 INFO:tasks.workunit.client.0.smithi047.stdout:kworker/3 249 root NOFD /proc/249/fd (opendir: Permission denied) 2024-03-21T10:28:38.313 INFO:tasks.workunit.client.0.smithi047.stdout:kworker/1 251 root cwd unknown /proc/251/cwd (readlink: Permission denied) 2024-03-21T10:28:38.314 INFO:tasks.workunit.client.0.smithi047.stdout:kworker/1 251 root rtd unknown /proc/251/root (readlink: Permission denied) 2024-03-21T10:28:38.314 INFO:tasks.workunit.client.0.smithi047.stdout:kworker/1 251 root txt unknown /proc/251/exe (readlink: Permission denied) 2024-03-21T10:28:38.314 INFO:tasks.workunit.client.0.smithi047.stdout:kworker/1 251 root NOFD /proc/251/fd (opendir: Permission denied) 2024-03-21T10:28:38.314 INFO:tasks.workunit.client.0.smithi047.stdout:kworker/2 261 root cwd unknown /proc/261/cwd (readlink: Permission denied) 2024-03-21T10:28:38.314 INFO:tasks.workunit.client.0.smithi047.stdout:kworker/2 261 root rtd unknown /proc/261/root (readlink: Permission denied) 2024-03-21T10:28:38.314 INFO:tasks.workunit.client.0.smithi047.stdout:kworker/2 261 root txt unknown /proc/261/exe (readlink: Permission denied) 2024-03-21T10:28:38.314 INFO:tasks.workunit.client.0.smithi047.stdout:kworker/2 261 root NOFD /proc/261/fd (opendir: Permission denied) 2024-03-21T10:28:38.314 INFO:tasks.workunit.client.0.smithi047.stdout:kworker/4 262 root cwd unknown /proc/262/cwd (readlink: Permission denied) 2024-03-21T10:28:38.314 INFO:tasks.workunit.client.0.smithi047.stdout:kworker/4 262 root rtd unknown /proc/262/root (readlink: Permission denied) 2024-03-21T10:28:38.314 INFO:tasks.workunit.client.0.smithi047.stdout:kworker/4 262 root txt unknown /proc/262/exe (readlink: Permission denied) 2024-03-21T10:28:38.314 INFO:tasks.workunit.client.0.smithi047.stdout:kworker/4 262 root NOFD /proc/262/fd (opendir: Permission denied) 2024-03-21T10:28:38.314 INFO:tasks.workunit.client.0.smithi047.stdout:kworker/6 267 root cwd unknown /proc/267/cwd (readlink: Permission denied) 2024-03-21T10:28:38.314 INFO:tasks.workunit.client.0.smithi047.stdout:kworker/6 267 root rtd unknown /proc/267/root (readlink: Permission denied) 2024-03-21T10:28:38.315 INFO:tasks.workunit.client.0.smithi047.stdout:kworker/6 267 root txt unknown /proc/267/exe (readlink: Permission denied) 2024-03-21T10:28:38.315 INFO:tasks.workunit.client.0.smithi047.stdout:kworker/6 267 root NOFD /proc/267/fd (opendir: Permission denied) 2024-03-21T10:28:38.315 INFO:tasks.workunit.client.0.smithi047.stdout:kworker/7 360 root cwd unknown /proc/360/cwd (readlink: Permission denied) 2024-03-21T10:28:38.315 INFO:tasks.workunit.client.0.smithi047.stdout:kworker/7 360 root rtd unknown /proc/360/root (readlink: Permission denied) 2024-03-21T10:28:38.315 INFO:tasks.workunit.client.0.smithi047.stdout:kworker/7 360 root txt unknown /proc/360/exe (readlink: Permission denied) 2024-03-21T10:28:38.315 INFO:tasks.workunit.client.0.smithi047.stdout:kworker/7 360 root NOFD /proc/360/fd (opendir: Permission denied) 2024-03-21T10:28:38.315 INFO:tasks.workunit.client.0.smithi047.stdout:kworker/R 402 root cwd unknown /proc/402/cwd (readlink: Permission denied) 2024-03-21T10:28:38.315 INFO:tasks.workunit.client.0.smithi047.stdout:kworker/R 402 root rtd unknown /proc/402/root (readlink: Permission denied) 2024-03-21T10:28:38.315 INFO:tasks.workunit.client.0.smithi047.stdout:kworker/R 402 root txt unknown /proc/402/exe (readlink: Permission denied) 2024-03-21T10:28:38.315 INFO:tasks.workunit.client.0.smithi047.stdout:kworker/R 402 root NOFD /proc/402/fd (opendir: Permission denied) 2024-03-21T10:28:38.315 INFO:tasks.workunit.client.0.smithi047.stdout:kworker/R 409 root cwd unknown /proc/409/cwd (readlink: Permission denied) 2024-03-21T10:28:38.315 INFO:tasks.workunit.client.0.smithi047.stdout:kworker/R 409 root rtd unknown /proc/409/root (readlink: Permission denied) 2024-03-21T10:28:38.315 INFO:tasks.workunit.client.0.smithi047.stdout:kworker/R 409 root txt unknown /proc/409/exe (readlink: Permission denied) 2024-03-21T10:28:38.315 INFO:tasks.workunit.client.0.smithi047.stdout:kworker/R 409 root NOFD /proc/409/fd (opendir: Permission denied) 2024-03-21T10:28:38.316 INFO:tasks.workunit.client.0.smithi047.stdout:kworker/R 410 root cwd unknown /proc/410/cwd (readlink: Permission denied) 2024-03-21T10:28:38.316 INFO:tasks.workunit.client.0.smithi047.stdout:kworker/R 410 root rtd unknown /proc/410/root (readlink: Permission denied) 2024-03-21T10:28:38.316 INFO:tasks.workunit.client.0.smithi047.stdout:kworker/R 410 root txt unknown /proc/410/exe (readlink: Permission denied) 2024-03-21T10:28:38.316 INFO:tasks.workunit.client.0.smithi047.stdout:kworker/R 410 root NOFD /proc/410/fd (opendir: Permission denied) 2024-03-21T10:28:38.316 INFO:tasks.workunit.client.0.smithi047.stdout:kworker/R 432 root cwd unknown /proc/432/cwd (readlink: Permission denied) 2024-03-21T10:28:38.316 INFO:tasks.workunit.client.0.smithi047.stdout:kworker/R 432 root rtd unknown /proc/432/root (readlink: Permission denied) 2024-03-21T10:28:38.316 INFO:tasks.workunit.client.0.smithi047.stdout:kworker/R 432 root txt unknown /proc/432/exe (readlink: Permission denied) 2024-03-21T10:28:38.316 INFO:tasks.workunit.client.0.smithi047.stdout:kworker/R 432 root NOFD /proc/432/fd (opendir: Permission denied) 2024-03-21T10:28:38.316 INFO:tasks.workunit.client.0.smithi047.stdout:jbd2/sda1 444 root cwd unknown /proc/444/cwd (readlink: Permission denied) 2024-03-21T10:28:38.316 INFO:tasks.workunit.client.0.smithi047.stdout:jbd2/sda1 444 root rtd unknown /proc/444/root (readlink: Permission denied) 2024-03-21T10:28:38.316 INFO:tasks.workunit.client.0.smithi047.stdout:jbd2/sda1 444 root txt unknown /proc/444/exe (readlink: Permission denied) 2024-03-21T10:28:38.316 INFO:tasks.workunit.client.0.smithi047.stdout:jbd2/sda1 444 root NOFD /proc/444/fd (opendir: Permission denied) 2024-03-21T10:28:38.316 INFO:tasks.workunit.client.0.smithi047.stdout:kworker/R 445 root cwd unknown /proc/445/cwd (readlink: Permission denied) 2024-03-21T10:28:38.317 INFO:tasks.workunit.client.0.smithi047.stdout:kworker/R 445 root rtd unknown /proc/445/root (readlink: Permission denied) 2024-03-21T10:28:38.317 INFO:tasks.workunit.client.0.smithi047.stdout:kworker/R 445 root txt unknown /proc/445/exe (readlink: Permission denied) 2024-03-21T10:28:38.317 INFO:tasks.workunit.client.0.smithi047.stdout:kworker/R 445 root NOFD /proc/445/fd (opendir: Permission denied) 2024-03-21T10:28:38.317 INFO:tasks.workunit.client.0.smithi047.stdout:kworker/7 476 root cwd unknown /proc/476/cwd (readlink: Permission denied) 2024-03-21T10:28:38.317 INFO:tasks.workunit.client.0.smithi047.stdout:kworker/7 476 root rtd unknown /proc/476/root (readlink: Permission denied) 2024-03-21T10:28:38.317 INFO:tasks.workunit.client.0.smithi047.stdout:kworker/7 476 root txt unknown /proc/476/exe (readlink: Permission denied) 2024-03-21T10:28:38.317 INFO:tasks.workunit.client.0.smithi047.stdout:kworker/7 476 root NOFD /proc/476/fd (opendir: Permission denied) 2024-03-21T10:28:38.317 INFO:tasks.workunit.client.0.smithi047.stdout:systemd-j 523 root cwd unknown /proc/523/cwd (readlink: Permission denied) 2024-03-21T10:28:38.317 INFO:tasks.workunit.client.0.smithi047.stdout:systemd-j 523 root rtd unknown /proc/523/root (readlink: Permission denied) 2024-03-21T10:28:38.317 INFO:tasks.workunit.client.0.smithi047.stdout:systemd-j 523 root txt unknown /proc/523/exe (readlink: Permission denied) 2024-03-21T10:28:38.317 INFO:tasks.workunit.client.0.smithi047.stdout:systemd-j 523 root NOFD /proc/523/fd (opendir: Permission denied) 2024-03-21T10:28:38.317 INFO:tasks.workunit.client.0.smithi047.stdout:ext4lazyi 530 root cwd unknown /proc/530/cwd (readlink: Permission denied) 2024-03-21T10:28:38.317 INFO:tasks.workunit.client.0.smithi047.stdout:ext4lazyi 530 root rtd unknown /proc/530/root (readlink: Permission denied) 2024-03-21T10:28:38.317 INFO:tasks.workunit.client.0.smithi047.stdout:ext4lazyi 530 root txt unknown /proc/530/exe (readlink: Permission denied) 2024-03-21T10:28:38.318 INFO:tasks.workunit.client.0.smithi047.stdout:ext4lazyi 530 root NOFD /proc/530/fd (opendir: Permission denied) 2024-03-21T10:28:38.318 INFO:tasks.workunit.client.0.smithi047.stdout:systemd-u 543 root cwd unknown /proc/543/cwd (readlink: Permission denied) 2024-03-21T10:28:38.318 INFO:tasks.workunit.client.0.smithi047.stdout:systemd-u 543 root rtd unknown /proc/543/root (readlink: Permission denied) 2024-03-21T10:28:38.318 INFO:tasks.workunit.client.0.smithi047.stdout:systemd-u 543 root txt unknown /proc/543/exe (readlink: Permission denied) 2024-03-21T10:28:38.318 INFO:tasks.workunit.client.0.smithi047.stdout:systemd-u 543 root NOFD /proc/543/fd (opendir: Permission denied) 2024-03-21T10:28:38.318 INFO:tasks.workunit.client.0.smithi047.stdout:kworker/R 594 root cwd unknown /proc/594/cwd (readlink: Permission denied) 2024-03-21T10:28:38.318 INFO:tasks.workunit.client.0.smithi047.stdout:kworker/R 594 root rtd unknown /proc/594/root (readlink: Permission denied) 2024-03-21T10:28:38.318 INFO:tasks.workunit.client.0.smithi047.stdout:kworker/R 594 root txt unknown /proc/594/exe (readlink: Permission denied) 2024-03-21T10:28:38.318 INFO:tasks.workunit.client.0.smithi047.stdout:kworker/R 594 root NOFD /proc/594/fd (opendir: Permission denied) 2024-03-21T10:28:38.318 INFO:tasks.workunit.client.0.smithi047.stdout:kipmi0 613 root cwd unknown /proc/613/cwd (readlink: Permission denied) 2024-03-21T10:28:38.318 INFO:tasks.workunit.client.0.smithi047.stdout:kipmi0 613 root rtd unknown /proc/613/root (readlink: Permission denied) 2024-03-21T10:28:38.318 INFO:tasks.workunit.client.0.smithi047.stdout:kipmi0 613 root txt unknown /proc/613/exe (readlink: Permission denied) 2024-03-21T10:28:38.318 INFO:tasks.workunit.client.0.smithi047.stdout:kipmi0 613 root NOFD /proc/613/fd (opendir: Permission denied) 2024-03-21T10:28:38.319 INFO:tasks.workunit.client.0.smithi047.stdout:rpcbind 616 rpc cwd unknown /proc/616/cwd (readlink: Permission denied) 2024-03-21T10:28:38.319 INFO:tasks.workunit.client.0.smithi047.stdout:rpcbind 616 rpc rtd unknown /proc/616/root (readlink: Permission denied) 2024-03-21T10:28:38.319 INFO:tasks.workunit.client.0.smithi047.stdout:rpcbind 616 rpc txt unknown /proc/616/exe (readlink: Permission denied) 2024-03-21T10:28:38.319 INFO:tasks.workunit.client.0.smithi047.stdout:rpcbind 616 rpc NOFD /proc/616/fd (opendir: Permission denied) 2024-03-21T10:28:38.319 INFO:tasks.workunit.client.0.smithi047.stdout:kworker/R 619 root cwd unknown /proc/619/cwd (readlink: Permission denied) 2024-03-21T10:28:38.319 INFO:tasks.workunit.client.0.smithi047.stdout:kworker/R 619 root rtd unknown /proc/619/root (readlink: Permission denied) 2024-03-21T10:28:38.319 INFO:tasks.workunit.client.0.smithi047.stdout:kworker/R 619 root txt unknown /proc/619/exe (readlink: Permission denied) 2024-03-21T10:28:38.319 INFO:tasks.workunit.client.0.smithi047.stdout:kworker/R 619 root NOFD /proc/619/fd (opendir: Permission denied) 2024-03-21T10:28:38.319 INFO:tasks.workunit.client.0.smithi047.stdout:kworker/R 620 root cwd unknown /proc/620/cwd (readlink: Permission denied) 2024-03-21T10:28:38.319 INFO:tasks.workunit.client.0.smithi047.stdout:kworker/R 620 root rtd unknown /proc/620/root (readlink: Permission denied) 2024-03-21T10:28:38.319 INFO:tasks.workunit.client.0.smithi047.stdout:kworker/R 620 root txt unknown /proc/620/exe (readlink: Permission denied) 2024-03-21T10:28:38.319 INFO:tasks.workunit.client.0.smithi047.stdout:kworker/R 620 root NOFD /proc/620/fd (opendir: Permission denied) 2024-03-21T10:28:38.319 INFO:tasks.workunit.client.0.smithi047.stdout:auditd 621 root cwd unknown /proc/621/cwd (readlink: Permission denied) 2024-03-21T10:28:38.320 INFO:tasks.workunit.client.0.smithi047.stdout:auditd 621 root rtd unknown /proc/621/root (readlink: Permission denied) 2024-03-21T10:28:38.320 INFO:tasks.workunit.client.0.smithi047.stdout:auditd 621 root txt unknown /proc/621/exe (readlink: Permission denied) 2024-03-21T10:28:38.320 INFO:tasks.workunit.client.0.smithi047.stdout:auditd 621 root NOFD /proc/621/fd (opendir: Permission denied) 2024-03-21T10:28:38.320 INFO:tasks.workunit.client.0.smithi047.stdout:auditd 621 622 auditd root cwd unknown /proc/621/task/622/cwd (readlink: Permission denied) 2024-03-21T10:28:38.320 INFO:tasks.workunit.client.0.smithi047.stdout:auditd 621 622 auditd root rtd unknown /proc/621/task/622/root (readlink: Permission denied) 2024-03-21T10:28:38.320 INFO:tasks.workunit.client.0.smithi047.stdout:auditd 621 622 auditd root txt unknown /proc/621/task/622/exe (readlink: Permission denied) 2024-03-21T10:28:38.320 INFO:tasks.workunit.client.0.smithi047.stdout:auditd 621 622 auditd root NOFD /proc/621/task/622/fd (opendir: Permission denied) 2024-03-21T10:28:38.320 INFO:tasks.workunit.client.0.smithi047.stdout:auditd 621 624 auditd root cwd unknown /proc/621/task/624/cwd (readlink: Permission denied) 2024-03-21T10:28:38.320 INFO:tasks.workunit.client.0.smithi047.stdout:auditd 621 624 auditd root rtd unknown /proc/621/task/624/root (readlink: Permission denied) 2024-03-21T10:28:38.320 INFO:tasks.workunit.client.0.smithi047.stdout:auditd 621 624 auditd root txt unknown /proc/621/task/624/exe (readlink: Permission denied) 2024-03-21T10:28:38.320 INFO:tasks.workunit.client.0.smithi047.stdout:auditd 621 624 auditd root NOFD /proc/621/task/624/fd (opendir: Permission denied) 2024-03-21T10:28:38.320 INFO:tasks.workunit.client.0.smithi047.stdout:sedispatc 623 root cwd unknown /proc/623/cwd (readlink: Permission denied) 2024-03-21T10:28:38.321 INFO:tasks.workunit.client.0.smithi047.stdout:sedispatc 623 root rtd unknown /proc/623/root (readlink: Permission denied) 2024-03-21T10:28:38.321 INFO:tasks.workunit.client.0.smithi047.stdout:sedispatc 623 root txt unknown /proc/623/exe (readlink: Permission denied) 2024-03-21T10:28:38.321 INFO:tasks.workunit.client.0.smithi047.stdout:sedispatc 623 root NOFD /proc/623/fd (opendir: Permission denied) 2024-03-21T10:28:38.321 INFO:tasks.workunit.client.0.smithi047.stdout:dbus-brok 660 dbus cwd unknown /proc/660/cwd (readlink: Permission denied) 2024-03-21T10:28:38.321 INFO:tasks.workunit.client.0.smithi047.stdout:dbus-brok 660 dbus rtd unknown /proc/660/root (readlink: Permission denied) 2024-03-21T10:28:38.321 INFO:tasks.workunit.client.0.smithi047.stdout:dbus-brok 660 dbus txt unknown /proc/660/exe (readlink: Permission denied) 2024-03-21T10:28:38.321 INFO:tasks.workunit.client.0.smithi047.stdout:dbus-brok 660 dbus NOFD /proc/660/fd (opendir: Permission denied) 2024-03-21T10:28:38.321 INFO:tasks.workunit.client.0.smithi047.stdout:dbus-brok 661 dbus cwd unknown /proc/661/cwd (readlink: Permission denied) 2024-03-21T10:28:38.321 INFO:tasks.workunit.client.0.smithi047.stdout:dbus-brok 661 dbus rtd unknown /proc/661/root (readlink: Permission denied) 2024-03-21T10:28:38.321 INFO:tasks.workunit.client.0.smithi047.stdout:dbus-brok 661 dbus txt unknown /proc/661/exe (readlink: Permission denied) 2024-03-21T10:28:38.321 INFO:tasks.workunit.client.0.smithi047.stdout:dbus-brok 661 dbus NOFD /proc/661/fd (opendir: Permission denied) 2024-03-21T10:28:38.321 INFO:tasks.workunit.client.0.smithi047.stdout:NetworkMa 662 root cwd unknown /proc/662/cwd (readlink: Permission denied) 2024-03-21T10:28:38.321 INFO:tasks.workunit.client.0.smithi047.stdout:NetworkMa 662 root rtd unknown /proc/662/root (readlink: Permission denied) 2024-03-21T10:28:38.321 INFO:tasks.workunit.client.0.smithi047.stdout:NetworkMa 662 root txt unknown /proc/662/exe (readlink: Permission denied) 2024-03-21T10:28:38.322 INFO:tasks.workunit.client.0.smithi047.stdout:NetworkMa 662 root NOFD /proc/662/fd (opendir: Permission denied) 2024-03-21T10:28:38.322 INFO:tasks.workunit.client.0.smithi047.stdout:NetworkMa 662 681 gmain root cwd unknown /proc/662/task/681/cwd (readlink: Permission denied) 2024-03-21T10:28:38.322 INFO:tasks.workunit.client.0.smithi047.stdout:NetworkMa 662 681 gmain root rtd unknown /proc/662/task/681/root (readlink: Permission denied) 2024-03-21T10:28:38.322 INFO:tasks.workunit.client.0.smithi047.stdout:NetworkMa 662 681 gmain root txt unknown /proc/662/task/681/exe (readlink: Permission denied) 2024-03-21T10:28:38.322 INFO:tasks.workunit.client.0.smithi047.stdout:NetworkMa 662 681 gmain root NOFD /proc/662/task/681/fd (opendir: Permission denied) 2024-03-21T10:28:38.322 INFO:tasks.workunit.client.0.smithi047.stdout:NetworkMa 662 690 gdbus root cwd unknown /proc/662/task/690/cwd (readlink: Permission denied) 2024-03-21T10:28:38.322 INFO:tasks.workunit.client.0.smithi047.stdout:NetworkMa 662 690 gdbus root rtd unknown /proc/662/task/690/root (readlink: Permission denied) 2024-03-21T10:28:38.322 INFO:tasks.workunit.client.0.smithi047.stdout:NetworkMa 662 690 gdbus root txt unknown /proc/662/task/690/exe (readlink: Permission denied) 2024-03-21T10:28:38.322 INFO:tasks.workunit.client.0.smithi047.stdout:NetworkMa 662 690 gdbus root NOFD /proc/662/task/690/fd (opendir: Permission denied) 2024-03-21T10:28:38.322 INFO:tasks.workunit.client.0.smithi047.stdout:irqbalanc 665 root cwd unknown /proc/665/cwd (readlink: Permission denied) 2024-03-21T10:28:38.322 INFO:tasks.workunit.client.0.smithi047.stdout:irqbalanc 665 root rtd unknown /proc/665/root (readlink: Permission denied) 2024-03-21T10:28:38.322 INFO:tasks.workunit.client.0.smithi047.stdout:irqbalanc 665 root txt unknown /proc/665/exe (readlink: Permission denied) 2024-03-21T10:28:38.322 INFO:tasks.workunit.client.0.smithi047.stdout:irqbalanc 665 root NOFD /proc/665/fd (opendir: Permission denied) 2024-03-21T10:28:38.323 INFO:tasks.workunit.client.0.smithi047.stdout:irqbalanc 665 671 gmain root cwd unknown /proc/665/task/671/cwd (readlink: Permission denied) 2024-03-21T10:28:38.323 INFO:tasks.workunit.client.0.smithi047.stdout:irqbalanc 665 671 gmain root rtd unknown /proc/665/task/671/root (readlink: Permission denied) 2024-03-21T10:28:38.323 INFO:tasks.workunit.client.0.smithi047.stdout:irqbalanc 665 671 gmain root txt unknown /proc/665/task/671/exe (readlink: Permission denied) 2024-03-21T10:28:38.323 INFO:tasks.workunit.client.0.smithi047.stdout:irqbalanc 665 671 gmain root NOFD /proc/665/task/671/fd (opendir: Permission denied) 2024-03-21T10:28:38.323 INFO:tasks.workunit.client.0.smithi047.stdout:lsmd 666 libstoragemgmt cwd unknown /proc/666/cwd (readlink: Permission denied) 2024-03-21T10:28:38.323 INFO:tasks.workunit.client.0.smithi047.stdout:lsmd 666 libstoragemgmt rtd unknown /proc/666/root (readlink: Permission denied) 2024-03-21T10:28:38.323 INFO:tasks.workunit.client.0.smithi047.stdout:lsmd 666 libstoragemgmt txt unknown /proc/666/exe (readlink: Permission denied) 2024-03-21T10:28:38.323 INFO:tasks.workunit.client.0.smithi047.stdout:lsmd 666 libstoragemgmt NOFD /proc/666/fd (opendir: Permission denied) 2024-03-21T10:28:38.323 INFO:tasks.workunit.client.0.smithi047.stdout:polkitd 667 polkitd cwd unknown /proc/667/cwd (readlink: Permission denied) 2024-03-21T10:28:38.323 INFO:tasks.workunit.client.0.smithi047.stdout:polkitd 667 polkitd rtd unknown /proc/667/root (readlink: Permission denied) 2024-03-21T10:28:38.323 INFO:tasks.workunit.client.0.smithi047.stdout:polkitd 667 polkitd txt unknown /proc/667/exe (readlink: Permission denied) 2024-03-21T10:28:38.323 INFO:tasks.workunit.client.0.smithi047.stdout:polkitd 667 polkitd NOFD /proc/667/fd (opendir: Permission denied) 2024-03-21T10:28:38.323 INFO:tasks.workunit.client.0.smithi047.stdout:polkitd 667 679 gmain polkitd cwd unknown /proc/667/task/679/cwd (readlink: Permission denied) 2024-03-21T10:28:38.323 INFO:tasks.workunit.client.0.smithi047.stdout:polkitd 667 679 gmain polkitd rtd unknown /proc/667/task/679/root (readlink: Permission denied) 2024-03-21T10:28:38.324 INFO:tasks.workunit.client.0.smithi047.stdout:polkitd 667 679 gmain polkitd txt unknown /proc/667/task/679/exe (readlink: Permission denied) 2024-03-21T10:28:38.324 INFO:tasks.workunit.client.0.smithi047.stdout:polkitd 667 679 gmain polkitd NOFD /proc/667/task/679/fd (opendir: Permission denied) 2024-03-21T10:28:38.324 INFO:tasks.workunit.client.0.smithi047.stdout:polkitd 667 680 gdbus polkitd cwd unknown /proc/667/task/680/cwd (readlink: Permission denied) 2024-03-21T10:28:38.324 INFO:tasks.workunit.client.0.smithi047.stdout:polkitd 667 680 gdbus polkitd rtd unknown /proc/667/task/680/root (readlink: Permission denied) 2024-03-21T10:28:38.324 INFO:tasks.workunit.client.0.smithi047.stdout:polkitd 667 680 gdbus polkitd txt unknown /proc/667/task/680/exe (readlink: Permission denied) 2024-03-21T10:28:38.324 INFO:tasks.workunit.client.0.smithi047.stdout:polkitd 667 680 gdbus polkitd NOFD /proc/667/task/680/fd (opendir: Permission denied) 2024-03-21T10:28:38.324 INFO:tasks.workunit.client.0.smithi047.stdout:polkitd 667 682 JS\x20Hel polkitd cwd unknown /proc/667/task/682/cwd (readlink: Permission denied) 2024-03-21T10:28:38.324 INFO:tasks.workunit.client.0.smithi047.stdout:polkitd 667 682 JS\x20Hel polkitd rtd unknown /proc/667/task/682/root (readlink: Permission denied) 2024-03-21T10:28:38.324 INFO:tasks.workunit.client.0.smithi047.stdout:polkitd 667 682 JS\x20Hel polkitd txt unknown /proc/667/task/682/exe (readlink: Permission denied) 2024-03-21T10:28:38.324 INFO:tasks.workunit.client.0.smithi047.stdout:polkitd 667 682 JS\x20Hel polkitd NOFD /proc/667/task/682/fd (opendir: Permission denied) 2024-03-21T10:28:38.324 INFO:tasks.workunit.client.0.smithi047.stdout:polkitd 667 683 JS\x20Hel polkitd cwd unknown /proc/667/task/683/cwd (readlink: Permission denied) 2024-03-21T10:28:38.324 INFO:tasks.workunit.client.0.smithi047.stdout:polkitd 667 683 JS\x20Hel polkitd rtd unknown /proc/667/task/683/root (readlink: Permission denied) 2024-03-21T10:28:38.324 INFO:tasks.workunit.client.0.smithi047.stdout:polkitd 667 683 JS\x20Hel polkitd txt unknown /proc/667/task/683/exe (readlink: Permission denied) 2024-03-21T10:28:38.325 INFO:tasks.workunit.client.0.smithi047.stdout:polkitd 667 683 JS\x20Hel polkitd NOFD /proc/667/task/683/fd (opendir: Permission denied) 2024-03-21T10:28:38.325 INFO:tasks.workunit.client.0.smithi047.stdout:polkitd 667 684 JS\x20Hel polkitd cwd unknown /proc/667/task/684/cwd (readlink: Permission denied) 2024-03-21T10:28:38.325 INFO:tasks.workunit.client.0.smithi047.stdout:polkitd 667 684 JS\x20Hel polkitd rtd unknown /proc/667/task/684/root (readlink: Permission denied) 2024-03-21T10:28:38.325 INFO:tasks.workunit.client.0.smithi047.stdout:polkitd 667 684 JS\x20Hel polkitd txt unknown /proc/667/task/684/exe (readlink: Permission denied) 2024-03-21T10:28:38.325 INFO:tasks.workunit.client.0.smithi047.stdout:polkitd 667 684 JS\x20Hel polkitd NOFD /proc/667/task/684/fd (opendir: Permission denied) 2024-03-21T10:28:38.325 INFO:tasks.workunit.client.0.smithi047.stdout:polkitd 667 685 JS\x20Hel polkitd cwd unknown /proc/667/task/685/cwd (readlink: Permission denied) 2024-03-21T10:28:38.325 INFO:tasks.workunit.client.0.smithi047.stdout:polkitd 667 685 JS\x20Hel polkitd rtd unknown /proc/667/task/685/root (readlink: Permission denied) 2024-03-21T10:28:38.325 INFO:tasks.workunit.client.0.smithi047.stdout:polkitd 667 685 JS\x20Hel polkitd txt unknown /proc/667/task/685/exe (readlink: Permission denied) 2024-03-21T10:28:38.325 INFO:tasks.workunit.client.0.smithi047.stdout:polkitd 667 685 JS\x20Hel polkitd NOFD /proc/667/task/685/fd (opendir: Permission denied) 2024-03-21T10:28:38.325 INFO:tasks.workunit.client.0.smithi047.stdout:polkitd 667 686 JS\x20Hel polkitd cwd unknown /proc/667/task/686/cwd (readlink: Permission denied) 2024-03-21T10:28:38.325 INFO:tasks.workunit.client.0.smithi047.stdout:polkitd 667 686 JS\x20Hel polkitd rtd unknown /proc/667/task/686/root (readlink: Permission denied) 2024-03-21T10:28:38.325 INFO:tasks.workunit.client.0.smithi047.stdout:polkitd 667 686 JS\x20Hel polkitd txt unknown /proc/667/task/686/exe (readlink: Permission denied) 2024-03-21T10:28:38.325 INFO:tasks.workunit.client.0.smithi047.stdout:polkitd 667 686 JS\x20Hel polkitd NOFD /proc/667/task/686/fd (opendir: Permission denied) 2024-03-21T10:28:38.325 INFO:tasks.workunit.client.0.smithi047.stdout:polkitd 667 687 JS\x20Hel polkitd cwd unknown /proc/667/task/687/cwd (readlink: Permission denied) 2024-03-21T10:28:38.325 INFO:tasks.workunit.client.0.smithi047.stdout:polkitd 667 687 JS\x20Hel polkitd rtd unknown /proc/667/task/687/root (readlink: Permission denied) 2024-03-21T10:28:38.326 INFO:tasks.workunit.client.0.smithi047.stdout:polkitd 667 687 JS\x20Hel polkitd txt unknown /proc/667/task/687/exe (readlink: Permission denied) 2024-03-21T10:28:38.326 INFO:tasks.workunit.client.0.smithi047.stdout:polkitd 667 687 JS\x20Hel polkitd NOFD /proc/667/task/687/fd (opendir: Permission denied) 2024-03-21T10:28:38.326 INFO:tasks.workunit.client.0.smithi047.stdout:polkitd 667 688 JS\x20Hel polkitd cwd unknown /proc/667/task/688/cwd (readlink: Permission denied) 2024-03-21T10:28:38.326 INFO:tasks.workunit.client.0.smithi047.stdout:polkitd 667 688 JS\x20Hel polkitd rtd unknown /proc/667/task/688/root (readlink: Permission denied) 2024-03-21T10:28:38.326 INFO:tasks.workunit.client.0.smithi047.stdout:polkitd 667 688 JS\x20Hel polkitd txt unknown /proc/667/task/688/exe (readlink: Permission denied) 2024-03-21T10:28:38.326 INFO:tasks.workunit.client.0.smithi047.stdout:polkitd 667 688 JS\x20Hel polkitd NOFD /proc/667/task/688/fd (opendir: Permission denied) 2024-03-21T10:28:38.326 INFO:tasks.workunit.client.0.smithi047.stdout:polkitd 667 689 JS\x20Hel polkitd cwd unknown /proc/667/task/689/cwd (readlink: Permission denied) 2024-03-21T10:28:38.326 INFO:tasks.workunit.client.0.smithi047.stdout:polkitd 667 689 JS\x20Hel polkitd rtd unknown /proc/667/task/689/root (readlink: Permission denied) 2024-03-21T10:28:38.326 INFO:tasks.workunit.client.0.smithi047.stdout:polkitd 667 689 JS\x20Hel polkitd txt unknown /proc/667/task/689/exe (readlink: Permission denied) 2024-03-21T10:28:38.326 INFO:tasks.workunit.client.0.smithi047.stdout:polkitd 667 689 JS\x20Hel polkitd NOFD /proc/667/task/689/fd (opendir: Permission denied) 2024-03-21T10:28:38.326 INFO:tasks.workunit.client.0.smithi047.stdout:polkitd 667 693 polkitd polkitd cwd unknown /proc/667/task/693/cwd (readlink: Permission denied) 2024-03-21T10:28:38.327 INFO:tasks.workunit.client.0.smithi047.stdout:polkitd 667 693 polkitd polkitd rtd unknown /proc/667/task/693/root (readlink: Permission denied) 2024-03-21T10:28:38.327 INFO:tasks.workunit.client.0.smithi047.stdout:polkitd 667 693 polkitd polkitd txt unknown /proc/667/task/693/exe (readlink: Permission denied) 2024-03-21T10:28:38.327 INFO:tasks.workunit.client.0.smithi047.stdout:polkitd 667 693 polkitd polkitd NOFD /proc/667/task/693/fd (opendir: Permission denied) 2024-03-21T10:28:38.327 INFO:tasks.workunit.client.0.smithi047.stdout:systemd-l 670 root cwd unknown /proc/670/cwd (readlink: Permission denied) 2024-03-21T10:28:38.327 INFO:tasks.workunit.client.0.smithi047.stdout:systemd-l 670 root rtd unknown /proc/670/root (readlink: Permission denied) 2024-03-21T10:28:38.327 INFO:tasks.workunit.client.0.smithi047.stdout:systemd-l 670 root txt unknown /proc/670/exe (readlink: Permission denied) 2024-03-21T10:28:38.327 INFO:tasks.workunit.client.0.smithi047.stdout:systemd-l 670 root NOFD /proc/670/fd (opendir: Permission denied) 2024-03-21T10:28:38.327 INFO:tasks.workunit.client.0.smithi047.stdout:sshd 730 root cwd unknown /proc/730/cwd (readlink: Permission denied) 2024-03-21T10:28:38.327 INFO:tasks.workunit.client.0.smithi047.stdout:sshd 730 root rtd unknown /proc/730/root (readlink: Permission denied) 2024-03-21T10:28:38.327 INFO:tasks.workunit.client.0.smithi047.stdout:sshd 730 root txt unknown /proc/730/exe (readlink: Permission denied) 2024-03-21T10:28:38.327 INFO:tasks.workunit.client.0.smithi047.stdout:sshd 730 root NOFD /proc/730/fd (opendir: Permission denied) 2024-03-21T10:28:38.328 INFO:tasks.workunit.client.0.smithi047.stdout:tuned 731 root cwd unknown /proc/731/cwd (readlink: Permission denied) 2024-03-21T10:28:38.328 INFO:tasks.workunit.client.0.smithi047.stdout:tuned 731 root rtd unknown /proc/731/root (readlink: Permission denied) 2024-03-21T10:28:38.328 INFO:tasks.workunit.client.0.smithi047.stdout:tuned 731 root txt unknown /proc/731/exe (readlink: Permission denied) 2024-03-21T10:28:38.328 INFO:tasks.workunit.client.0.smithi047.stdout:tuned 731 root NOFD /proc/731/fd (opendir: Permission denied) 2024-03-21T10:28:38.328 INFO:tasks.workunit.client.0.smithi047.stdout:tuned 731 830 tuned root cwd unknown /proc/731/task/830/cwd (readlink: Permission denied) 2024-03-21T10:28:38.328 INFO:tasks.workunit.client.0.smithi047.stdout:tuned 731 830 tuned root rtd unknown /proc/731/task/830/root (readlink: Permission denied) 2024-03-21T10:28:38.328 INFO:tasks.workunit.client.0.smithi047.stdout:tuned 731 830 tuned root txt unknown /proc/731/task/830/exe (readlink: Permission denied) 2024-03-21T10:28:38.328 INFO:tasks.workunit.client.0.smithi047.stdout:tuned 731 830 tuned root NOFD /proc/731/task/830/fd (opendir: Permission denied) 2024-03-21T10:28:38.328 INFO:tasks.workunit.client.0.smithi047.stdout:gssproxy 749 root cwd unknown /proc/749/cwd (readlink: Permission denied) 2024-03-21T10:28:38.328 INFO:tasks.workunit.client.0.smithi047.stdout:gssproxy 749 root rtd unknown /proc/749/root (readlink: Permission denied) 2024-03-21T10:28:38.328 INFO:tasks.workunit.client.0.smithi047.stdout:gssproxy 749 root txt unknown /proc/749/exe (readlink: Permission denied) 2024-03-21T10:28:38.328 INFO:tasks.workunit.client.0.smithi047.stdout:gssproxy 749 root NOFD /proc/749/fd (opendir: Permission denied) 2024-03-21T10:28:38.329 INFO:tasks.workunit.client.0.smithi047.stdout:gssproxy 749 750 gssproxy root cwd unknown /proc/749/task/750/cwd (readlink: Permission denied) 2024-03-21T10:28:38.329 INFO:tasks.workunit.client.0.smithi047.stdout:gssproxy 749 750 gssproxy root rtd unknown /proc/749/task/750/root (readlink: Permission denied) 2024-03-21T10:28:38.329 INFO:tasks.workunit.client.0.smithi047.stdout:gssproxy 749 750 gssproxy root txt unknown /proc/749/task/750/exe (readlink: Permission denied) 2024-03-21T10:28:38.329 INFO:tasks.workunit.client.0.smithi047.stdout:gssproxy 749 750 gssproxy root NOFD /proc/749/task/750/fd (opendir: Permission denied) 2024-03-21T10:28:38.329 INFO:tasks.workunit.client.0.smithi047.stdout:gssproxy 749 751 gssproxy root cwd unknown /proc/749/task/751/cwd (readlink: Permission denied) 2024-03-21T10:28:38.329 INFO:tasks.workunit.client.0.smithi047.stdout:gssproxy 749 751 gssproxy root rtd unknown /proc/749/task/751/root (readlink: Permission denied) 2024-03-21T10:28:38.329 INFO:tasks.workunit.client.0.smithi047.stdout:gssproxy 749 751 gssproxy root txt unknown /proc/749/task/751/exe (readlink: Permission denied) 2024-03-21T10:28:38.329 INFO:tasks.workunit.client.0.smithi047.stdout:gssproxy 749 751 gssproxy root NOFD /proc/749/task/751/fd (opendir: Permission denied) 2024-03-21T10:28:38.329 INFO:tasks.workunit.client.0.smithi047.stdout:gssproxy 749 752 gssproxy root cwd unknown /proc/749/task/752/cwd (readlink: Permission denied) 2024-03-21T10:28:38.329 INFO:tasks.workunit.client.0.smithi047.stdout:gssproxy 749 752 gssproxy root rtd unknown /proc/749/task/752/root (readlink: Permission denied) 2024-03-21T10:28:38.329 INFO:tasks.workunit.client.0.smithi047.stdout:gssproxy 749 752 gssproxy root txt unknown /proc/749/task/752/exe (readlink: Permission denied) 2024-03-21T10:28:38.329 INFO:tasks.workunit.client.0.smithi047.stdout:gssproxy 749 752 gssproxy root NOFD /proc/749/task/752/fd (opendir: Permission denied) 2024-03-21T10:28:38.329 INFO:tasks.workunit.client.0.smithi047.stdout:gssproxy 749 753 gssproxy root cwd unknown /proc/749/task/753/cwd (readlink: Permission denied) 2024-03-21T10:28:38.329 INFO:tasks.workunit.client.0.smithi047.stdout:gssproxy 749 753 gssproxy root rtd unknown /proc/749/task/753/root (readlink: Permission denied) 2024-03-21T10:28:38.330 INFO:tasks.workunit.client.0.smithi047.stdout:gssproxy 749 753 gssproxy root txt unknown /proc/749/task/753/exe (readlink: Permission denied) 2024-03-21T10:28:38.330 INFO:tasks.workunit.client.0.smithi047.stdout:gssproxy 749 753 gssproxy root NOFD /proc/749/task/753/fd (opendir: Permission denied) 2024-03-21T10:28:38.330 INFO:tasks.workunit.client.0.smithi047.stdout:gssproxy 749 754 gssproxy root cwd unknown /proc/749/task/754/cwd (readlink: Permission denied) 2024-03-21T10:28:38.330 INFO:tasks.workunit.client.0.smithi047.stdout:gssproxy 749 754 gssproxy root rtd unknown /proc/749/task/754/root (readlink: Permission denied) 2024-03-21T10:28:38.330 INFO:tasks.workunit.client.0.smithi047.stdout:gssproxy 749 754 gssproxy root txt unknown /proc/749/task/754/exe (readlink: Permission denied) 2024-03-21T10:28:38.330 INFO:tasks.workunit.client.0.smithi047.stdout:gssproxy 749 754 gssproxy root NOFD /proc/749/task/754/fd (opendir: Permission denied) 2024-03-21T10:28:38.330 INFO:tasks.workunit.client.0.smithi047.stdout:atd 756 root cwd unknown /proc/756/cwd (readlink: Permission denied) 2024-03-21T10:28:38.330 INFO:tasks.workunit.client.0.smithi047.stdout:atd 756 root rtd unknown /proc/756/root (readlink: Permission denied) 2024-03-21T10:28:38.330 INFO:tasks.workunit.client.0.smithi047.stdout:atd 756 root txt unknown /proc/756/exe (readlink: Permission denied) 2024-03-21T10:28:38.330 INFO:tasks.workunit.client.0.smithi047.stdout:atd 756 root NOFD /proc/756/fd (opendir: Permission denied) 2024-03-21T10:28:38.330 INFO:tasks.workunit.client.0.smithi047.stdout:crond 757 root cwd unknown /proc/757/cwd (readlink: Permission denied) 2024-03-21T10:28:38.330 INFO:tasks.workunit.client.0.smithi047.stdout:crond 757 root rtd unknown /proc/757/root (readlink: Permission denied) 2024-03-21T10:28:38.330 INFO:tasks.workunit.client.0.smithi047.stdout:crond 757 root txt unknown /proc/757/exe (readlink: Permission denied) 2024-03-21T10:28:38.330 INFO:tasks.workunit.client.0.smithi047.stdout:crond 757 root NOFD /proc/757/fd (opendir: Permission denied) 2024-03-21T10:28:38.331 INFO:tasks.workunit.client.0.smithi047.stdout:nrpe 854 nrpe cwd unknown /proc/854/cwd (readlink: Permission denied) 2024-03-21T10:28:38.331 INFO:tasks.workunit.client.0.smithi047.stdout:nrpe 854 nrpe rtd unknown /proc/854/root (readlink: Permission denied) 2024-03-21T10:28:38.331 INFO:tasks.workunit.client.0.smithi047.stdout:nrpe 854 nrpe txt unknown /proc/854/exe (readlink: Permission denied) 2024-03-21T10:28:38.331 INFO:tasks.workunit.client.0.smithi047.stdout:nrpe 854 nrpe NOFD /proc/854/fd (opendir: Permission denied) 2024-03-21T10:28:38.331 INFO:tasks.workunit.client.0.smithi047.stdout:agetty 1175 root cwd unknown /proc/1175/cwd (readlink: Permission denied) 2024-03-21T10:28:38.331 INFO:tasks.workunit.client.0.smithi047.stdout:agetty 1175 root rtd unknown /proc/1175/root (readlink: Permission denied) 2024-03-21T10:28:38.331 INFO:tasks.workunit.client.0.smithi047.stdout:agetty 1175 root txt unknown /proc/1175/exe (readlink: Permission denied) 2024-03-21T10:28:38.331 INFO:tasks.workunit.client.0.smithi047.stdout:agetty 1175 root NOFD /proc/1175/fd (opendir: Permission denied) 2024-03-21T10:28:38.331 INFO:tasks.workunit.client.0.smithi047.stdout:agetty 1177 root cwd unknown /proc/1177/cwd (readlink: Permission denied) 2024-03-21T10:28:38.331 INFO:tasks.workunit.client.0.smithi047.stdout:agetty 1177 root rtd unknown /proc/1177/root (readlink: Permission denied) 2024-03-21T10:28:38.331 INFO:tasks.workunit.client.0.smithi047.stdout:agetty 1177 root txt unknown /proc/1177/exe (readlink: Permission denied) 2024-03-21T10:28:38.331 INFO:tasks.workunit.client.0.smithi047.stdout:agetty 1177 root NOFD /proc/1177/fd (opendir: Permission denied) 2024-03-21T10:28:38.331 INFO:tasks.workunit.client.0.smithi047.stdout:systemd 1426 ubuntu cwd DIR 8,1 4096 2 / 2024-03-21T10:28:38.331 INFO:tasks.workunit.client.0.smithi047.stdout:systemd 1426 ubuntu rtd DIR 8,1 4096 2 / 2024-03-21T10:28:38.332 INFO:tasks.workunit.client.0.smithi047.stdout:systemd 1426 ubuntu txt REG 8,1 102120 20353 /usr/lib/systemd/systemd 2024-03-21T10:28:38.332 INFO:tasks.workunit.client.0.smithi047.stdout:systemd 1426 ubuntu DEL REG 8,1 1174 /etc/selinux/targeted/contexts/files/file_contexts.homedirs.bin 2024-03-21T10:28:38.332 INFO:tasks.workunit.client.0.smithi047.stdout:systemd 1426 ubuntu DEL REG 8,1 1167 /etc/selinux/targeted/contexts/files/file_contexts.bin 2024-03-21T10:28:38.332 INFO:tasks.workunit.client.0.smithi047.stdout:systemd 1426 ubuntu mem REG 8,1 637872 9740 /usr/lib64/libpcre2-8.so.0.11.0 2024-03-21T10:28:38.332 INFO:tasks.workunit.client.0.smithi047.stdout:systemd 1426 ubuntu mem REG 8,1 904672 9627 /usr/lib64/libm.so.6 2024-03-21T10:28:38.332 INFO:tasks.workunit.client.0.smithi047.stdout:systemd 1426 ubuntu mem REG 8,1 1293840 9789 /usr/lib64/libp11-kit.so.0.3.0 2024-03-21T10:28:38.332 INFO:tasks.workunit.client.0.smithi047.stdout:systemd 1426 ubuntu DEL REG 8,1 10007 /usr/lib64/libcrypto.so.3.0.7 2024-03-21T10:28:38.332 INFO:tasks.workunit.client.0.smithi047.stdout:systemd 1426 ubuntu mem REG 8,1 44784 9785 /usr/lib64/libffi.so.8.1.0 2024-03-21T10:28:38.332 INFO:tasks.workunit.client.0.smithi047.stdout:systemd 1426 ubuntu mem REG 8,1 153600 9809 /usr/lib64/libgpg-error.so.0.32.0 2024-03-21T10:28:38.332 INFO:tasks.workunit.client.0.smithi047.stdout:systemd 1426 ubuntu mem REG 8,1 28552 9804 /usr/lib64/libattr.so.1.1.2501 2024-03-21T10:28:38.333 INFO:tasks.workunit.client.0.smithi047.stdout:systemd 1426 ubuntu mem REG 8,1 102552 9686 /usr/lib64/libz.so.1.2.11 2024-03-21T10:28:38.333 INFO:tasks.workunit.client.0.smithi047.stdout:systemd 1426 ubuntu mem REG 8,1 32528 9723 /usr/lib64/libcap-ng.so.0.0.0 2024-03-21T10:28:38.333 INFO:tasks.workunit.client.0.smithi047.stdout:systemd 1426 ubuntu mem REG 8,1 178728 9694 /usr/lib64/liblzma.so.5.2.5 2024-03-21T10:28:38.333 INFO:tasks.workunit.client.0.smithi047.stdout:systemd 1426 ubuntu mem REG 8,1 1288504 9883 /usr/lib64/libgcrypt.so.20.4.0 2024-03-21T10:28:38.333 INFO:tasks.workunit.client.0.smithi047.stdout:systemd 1426 ubuntu mem REG 8,1 2387016 9624 /usr/lib64/libc.so.6 2024-03-21T10:28:38.333 INFO:tasks.workunit.client.0.smithi047.stdout:systemd 1426 ubuntu mem REG 8,1 40424 10956 /usr/lib64/libeconf.so.0.4.1 2024-03-21T10:28:38.333 INFO:tasks.workunit.client.0.smithi047.stdout:systemd 1426 ubuntu mem REG 8,1 882376 9699 /usr/lib64/libzstd.so.1.5.1 2024-03-21T10:28:38.333 INFO:tasks.workunit.client.0.smithi047.stdout:systemd 1426 ubuntu mem REG 8,1 144120 9867 /usr/lib64/liblz4.so.1.9.3 2024-03-21T10:28:38.333 INFO:tasks.workunit.client.0.smithi047.stdout:systemd 1426 ubuntu mem REG 8,1 201808 9696 /usr/lib64/libcrypt.so.2.0.0 2024-03-21T10:28:38.333 INFO:tasks.workunit.client.0.smithi047.stdout:systemd 1426 ubuntu mem REG 8,1 36296 9702 /usr/lib64/libcap.so.2.48 2024-03-21T10:28:38.333 INFO:tasks.workunit.client.0.smithi047.stdout:systemd 1426 ubuntu DEL REG 8,1 10126 /usr/lib64/libblkid.so.1.1.0 2024-03-21T10:28:38.333 INFO:tasks.workunit.client.0.smithi047.stdout:systemd 1426 ubuntu DEL REG 8,1 10156 /usr/lib64/libmount.so.1.1.0 2024-03-21T10:28:38.334 INFO:tasks.workunit.client.0.smithi047.stdout:systemd 1426 ubuntu mem REG 8,1 175552 9824 /usr/lib64/libselinux.so.1 2024-03-21T10:28:38.334 INFO:tasks.workunit.client.0.smithi047.stdout:systemd 1426 ubuntu mem REG 8,1 3645472 49253 /usr/lib64/systemd/libsystemd-shared-252.so 2024-03-21T10:28:38.334 INFO:tasks.workunit.client.0.smithi047.stdout:systemd 1426 ubuntu DEL REG 8,1 9806 /usr/lib64/libacl.so.1.1.2301 2024-03-21T10:28:38.334 INFO:tasks.workunit.client.0.smithi047.stdout:systemd 1426 ubuntu mem REG 8,1 107424 10180 /usr/lib64/libkmod.so.2.3.6 2024-03-21T10:28:38.334 INFO:tasks.workunit.client.0.smithi047.stdout:systemd 1426 ubuntu mem REG 8,1 134624 9727 /usr/lib64/libaudit.so.1.0.0 2024-03-21T10:28:38.334 INFO:tasks.workunit.client.0.smithi047.stdout:systemd 1426 ubuntu mem REG 8,1 127680 9972 /usr/lib64/libseccomp.so.2.5.2 2024-03-21T10:28:38.334 INFO:tasks.workunit.client.0.smithi047.stdout:systemd 1426 ubuntu mem REG 8,1 108136 13107 /usr/lib64/libgcc_s-11-20230605.so.1 2024-03-21T10:28:38.334 INFO:tasks.workunit.client.0.smithi047.stdout:systemd 1426 ubuntu mem REG 8,1 2174088 49251 /usr/lib64/systemd/libsystemd-core-252.so 2024-03-21T10:28:38.334 INFO:tasks.workunit.client.0.smithi047.stdout:systemd 1426 ubuntu mem REG 8,1 69472 10965 /usr/lib64/libpam.so.0.85.1 2024-03-21T10:28:38.334 INFO:tasks.workunit.client.0.smithi047.stdout:systemd 1426 ubuntu mem REG 8,1 844808 9620 /usr/lib64/ld-linux-x86-64.so.2 2024-03-21T10:28:38.334 INFO:tasks.workunit.client.0.smithi047.stdout:systemd 1426 ubuntu mem REG 0,19 0 19 /sys/fs/selinux/status 2024-03-21T10:28:38.334 INFO:tasks.workunit.client.0.smithi047.stdout:systemd 1426 ubuntu 0r CHR 1,3 0t0 4 /dev/null 2024-03-21T10:28:38.335 INFO:tasks.workunit.client.0.smithi047.stdout:systemd 1426 ubuntu 1u unix 0x0000000000000000 0t0 3593 type=STREAM (CONNECTED) 2024-03-21T10:28:38.335 INFO:tasks.workunit.client.0.smithi047.stdout:systemd 1426 ubuntu 2u unix 0x0000000000000000 0t0 3593 type=STREAM (CONNECTED) 2024-03-21T10:28:38.335 INFO:tasks.workunit.client.0.smithi047.stdout:systemd 1426 ubuntu 3u unix 0x0000000000000000 0t0 3623 type=DGRAM (CONNECTED) 2024-03-21T10:28:38.335 INFO:tasks.workunit.client.0.smithi047.stdout:systemd 1426 ubuntu 4u a_inode 0,14 0 50 [eventpoll:5,6,8,9,11,12,14,15,16,17,19,20,21] 2024-03-21T10:28:38.335 INFO:tasks.workunit.client.0.smithi047.stdout:systemd 1426 ubuntu 5u a_inode 0,14 0 50 [signalfd] 2024-03-21T10:28:38.335 INFO:tasks.workunit.client.0.smithi047.stdout:systemd 1426 ubuntu 6r a_inode 0,14 0 50 inotify 2024-03-21T10:28:38.335 INFO:tasks.workunit.client.0.smithi047.stdout:systemd 1426 ubuntu 7r DIR 0,25 0 3543 /sys/fs/cgroup/user.slice/user-1000.slice/user@1000.service 2024-03-21T10:28:38.335 INFO:tasks.workunit.client.0.smithi047.stdout:systemd 1426 ubuntu 8u a_inode 0,14 0 50 [timerfd] 2024-03-21T10:28:38.335 INFO:tasks.workunit.client.0.smithi047.stdout:systemd 1426 ubuntu 9u a_inode 0,14 0 50 [eventpoll:10,13] 2024-03-21T10:28:38.335 INFO:tasks.workunit.client.0.smithi047.stdout:systemd 1426 ubuntu 10r REG 0,20 0 3612 /proc/1426/mountinfo 2024-03-21T10:28:38.335 INFO:tasks.workunit.client.0.smithi047.stdout:systemd 1426 ubuntu 11r a_inode 0,14 0 50 inotify 2024-03-21T10:28:38.335 INFO:tasks.workunit.client.0.smithi047.stdout:systemd 1426 ubuntu 12u unix 0x0000000000000000 0t0 3652 /run/user/1000/bus type=STREAM (LISTEN) 2024-03-21T10:28:38.335 INFO:tasks.workunit.client.0.smithi047.stdout:systemd 1426 ubuntu 13r a_inode 0,14 0 50 inotify 2024-03-21T10:28:38.336 INFO:tasks.workunit.client.0.smithi047.stdout:systemd 1426 ubuntu 14r REG 0,20 0 4026532078 /proc/swaps 2024-03-21T10:28:38.336 INFO:tasks.workunit.client.0.smithi047.stdout:systemd 1426 ubuntu 15u netlink 0t0 3634 KOBJECT_UEVENT 2024-03-21T10:28:38.336 INFO:tasks.workunit.client.0.smithi047.stdout:systemd 1426 ubuntu 16u unix 0x0000000000000000 0t0 3639 /run/user/1000/systemd/notify type=DGRAM (UNCONNECTED) 2024-03-21T10:28:38.336 INFO:tasks.workunit.client.0.smithi047.stdout:systemd 1426 ubuntu 17u unix 0x0000000000000000 0t0 3640 type=DGRAM (CONNECTED) 2024-03-21T10:28:38.336 INFO:tasks.workunit.client.0.smithi047.stdout:systemd 1426 ubuntu 18u unix 0x0000000000000000 0t0 3641 type=DGRAM (CONNECTED) 2024-03-21T10:28:38.336 INFO:tasks.workunit.client.0.smithi047.stdout:systemd 1426 ubuntu 19u unix 0x0000000000000000 0t0 3642 /run/user/1000/systemd/private type=STREAM (LISTEN) 2024-03-21T10:28:38.336 INFO:tasks.workunit.client.0.smithi047.stdout:systemd 1426 ubuntu 20u unix 0x0000000000000000 0t0 3644 type=STREAM (CONNECTED) 2024-03-21T10:28:38.336 INFO:tasks.workunit.client.0.smithi047.stdout:systemd 1426 ubuntu 21u a_inode 0,14 0 50 [timerfd] 2024-03-21T10:28:38.336 INFO:tasks.workunit.client.0.smithi047.stdout:(sd-pam) 1450 ubuntu cwd unknown /proc/1450/cwd (readlink: Permission denied) 2024-03-21T10:28:38.336 INFO:tasks.workunit.client.0.smithi047.stdout:(sd-pam) 1450 ubuntu rtd unknown /proc/1450/root (readlink: Permission denied) 2024-03-21T10:28:38.336 INFO:tasks.workunit.client.0.smithi047.stdout:(sd-pam) 1450 ubuntu txt unknown /proc/1450/exe (readlink: Permission denied) 2024-03-21T10:28:38.336 INFO:tasks.workunit.client.0.smithi047.stdout:(sd-pam) 1450 ubuntu NOFD /proc/1450/fd (opendir: Permission denied) 2024-03-21T10:28:38.337 INFO:tasks.workunit.client.0.smithi047.stdout:anacron 4352 root cwd unknown /proc/4352/cwd (readlink: Permission denied) 2024-03-21T10:28:38.337 INFO:tasks.workunit.client.0.smithi047.stdout:anacron 4352 root rtd unknown /proc/4352/root (readlink: Permission denied) 2024-03-21T10:28:38.337 INFO:tasks.workunit.client.0.smithi047.stdout:anacron 4352 root txt unknown /proc/4352/exe (readlink: Permission denied) 2024-03-21T10:28:38.337 INFO:tasks.workunit.client.0.smithi047.stdout:anacron 4352 root NOFD /proc/4352/fd (opendir: Permission denied) 2024-03-21T10:28:38.337 INFO:tasks.workunit.client.0.smithi047.stdout:rsyslogd 4580 root cwd unknown /proc/4580/cwd (readlink: Permission denied) 2024-03-21T10:28:38.337 INFO:tasks.workunit.client.0.smithi047.stdout:rsyslogd 4580 root rtd unknown /proc/4580/root (readlink: Permission denied) 2024-03-21T10:28:38.337 INFO:tasks.workunit.client.0.smithi047.stdout:rsyslogd 4580 root txt unknown /proc/4580/exe (readlink: Permission denied) 2024-03-21T10:28:38.337 INFO:tasks.workunit.client.0.smithi047.stdout:rsyslogd 4580 root NOFD /proc/4580/fd (opendir: Permission denied) 2024-03-21T10:28:38.337 INFO:tasks.workunit.client.0.smithi047.stdout:rsyslogd 4580 4582 in:imjour root cwd unknown /proc/4580/task/4582/cwd (readlink: Permission denied) 2024-03-21T10:28:38.337 INFO:tasks.workunit.client.0.smithi047.stdout:rsyslogd 4580 4582 in:imjour root rtd unknown /proc/4580/task/4582/root (readlink: Permission denied) 2024-03-21T10:28:38.337 INFO:tasks.workunit.client.0.smithi047.stdout:rsyslogd 4580 4582 in:imjour root txt unknown /proc/4580/task/4582/exe (readlink: Permission denied) 2024-03-21T10:28:38.337 INFO:tasks.workunit.client.0.smithi047.stdout:rsyslogd 4580 4582 in:imjour root NOFD /proc/4580/task/4582/fd (opendir: Permission denied) 2024-03-21T10:28:38.337 INFO:tasks.workunit.client.0.smithi047.stdout:rsyslogd 4580 4583 rs:main root cwd unknown /proc/4580/task/4583/cwd (readlink: Permission denied) 2024-03-21T10:28:38.338 INFO:tasks.workunit.client.0.smithi047.stdout:rsyslogd 4580 4583 rs:main root rtd unknown /proc/4580/task/4583/root (readlink: Permission denied) 2024-03-21T10:28:38.338 INFO:tasks.workunit.client.0.smithi047.stdout:rsyslogd 4580 4583 rs:main root txt unknown /proc/4580/task/4583/exe (readlink: Permission denied) 2024-03-21T10:28:38.338 INFO:tasks.workunit.client.0.smithi047.stdout:rsyslogd 4580 4583 rs:main root NOFD /proc/4580/task/4583/fd (opendir: Permission denied) 2024-03-21T10:28:38.338 INFO:tasks.workunit.client.0.smithi047.stdout:smartd 15374 root cwd unknown /proc/15374/cwd (readlink: Permission denied) 2024-03-21T10:28:38.338 INFO:tasks.workunit.client.0.smithi047.stdout:smartd 15374 root rtd unknown /proc/15374/root (readlink: Permission denied) 2024-03-21T10:28:38.338 INFO:tasks.workunit.client.0.smithi047.stdout:smartd 15374 root txt unknown /proc/15374/exe (readlink: Permission denied) 2024-03-21T10:28:38.338 INFO:tasks.workunit.client.0.smithi047.stdout:smartd 15374 root NOFD /proc/15374/fd (opendir: Permission denied) 2024-03-21T10:28:38.338 INFO:tasks.workunit.client.0.smithi047.stdout:kworker/R 20731 root cwd unknown /proc/20731/cwd (readlink: Permission denied) 2024-03-21T10:28:38.338 INFO:tasks.workunit.client.0.smithi047.stdout:kworker/R 20731 root rtd unknown /proc/20731/root (readlink: Permission denied) 2024-03-21T10:28:38.338 INFO:tasks.workunit.client.0.smithi047.stdout:kworker/R 20731 root txt unknown /proc/20731/exe (readlink: Permission denied) 2024-03-21T10:28:38.338 INFO:tasks.workunit.client.0.smithi047.stdout:kworker/R 20731 root NOFD /proc/20731/fd (opendir: Permission denied) 2024-03-21T10:28:38.338 INFO:tasks.workunit.client.0.smithi047.stdout:kworker/R 20761 root cwd unknown /proc/20761/cwd (readlink: Permission denied) 2024-03-21T10:28:38.339 INFO:tasks.workunit.client.0.smithi047.stdout:kworker/R 20761 root rtd unknown /proc/20761/root (readlink: Permission denied) 2024-03-21T10:28:38.339 INFO:tasks.workunit.client.0.smithi047.stdout:kworker/R 20761 root txt unknown /proc/20761/exe (readlink: Permission denied) 2024-03-21T10:28:38.339 INFO:tasks.workunit.client.0.smithi047.stdout:kworker/R 20761 root NOFD /proc/20761/fd (opendir: Permission denied) 2024-03-21T10:28:38.339 INFO:tasks.workunit.client.0.smithi047.stdout:kworker/R 20790 root cwd unknown /proc/20790/cwd (readlink: Permission denied) 2024-03-21T10:28:38.339 INFO:tasks.workunit.client.0.smithi047.stdout:kworker/R 20790 root rtd unknown /proc/20790/root (readlink: Permission denied) 2024-03-21T10:28:38.339 INFO:tasks.workunit.client.0.smithi047.stdout:kworker/R 20790 root txt unknown /proc/20790/exe (readlink: Permission denied) 2024-03-21T10:28:38.339 INFO:tasks.workunit.client.0.smithi047.stdout:kworker/R 20790 root NOFD /proc/20790/fd (opendir: Permission denied) 2024-03-21T10:28:38.339 INFO:tasks.workunit.client.0.smithi047.stdout:kworker/R 20819 root cwd unknown /proc/20819/cwd (readlink: Permission denied) 2024-03-21T10:28:38.339 INFO:tasks.workunit.client.0.smithi047.stdout:kworker/R 20819 root rtd unknown /proc/20819/root (readlink: Permission denied) 2024-03-21T10:28:38.340 INFO:tasks.workunit.client.0.smithi047.stdout:kworker/R 20819 root txt unknown /proc/20819/exe (readlink: Permission denied) 2024-03-21T10:28:38.340 INFO:tasks.workunit.client.0.smithi047.stdout:kworker/R 20819 root NOFD /proc/20819/fd (opendir: Permission denied) 2024-03-21T10:28:38.340 INFO:tasks.workunit.client.0.smithi047.stdout:kworker/R 20848 root cwd unknown /proc/20848/cwd (readlink: Permission denied) 2024-03-21T10:28:38.340 INFO:tasks.workunit.client.0.smithi047.stdout:kworker/R 20848 root rtd unknown /proc/20848/root (readlink: Permission denied) 2024-03-21T10:28:38.340 INFO:tasks.workunit.client.0.smithi047.stdout:kworker/R 20848 root txt unknown /proc/20848/exe (readlink: Permission denied) 2024-03-21T10:28:38.340 INFO:tasks.workunit.client.0.smithi047.stdout:kworker/R 20848 root NOFD /proc/20848/fd (opendir: Permission denied) 2024-03-21T10:28:38.340 INFO:tasks.workunit.client.0.smithi047.stdout:kworker/R 21012 root cwd unknown /proc/21012/cwd (readlink: Permission denied) 2024-03-21T10:28:38.340 INFO:tasks.workunit.client.0.smithi047.stdout:kworker/R 21012 root rtd unknown /proc/21012/root (readlink: Permission denied) 2024-03-21T10:28:38.340 INFO:tasks.workunit.client.0.smithi047.stdout:kworker/R 21012 root txt unknown /proc/21012/exe (readlink: Permission denied) 2024-03-21T10:28:38.340 INFO:tasks.workunit.client.0.smithi047.stdout:kworker/R 21012 root NOFD /proc/21012/fd (opendir: Permission denied) 2024-03-21T10:28:38.341 INFO:tasks.workunit.client.0.smithi047.stdout:kworker/R 21013 root cwd unknown /proc/21013/cwd (readlink: Permission denied) 2024-03-21T10:28:38.341 INFO:tasks.workunit.client.0.smithi047.stdout:kworker/R 21013 root rtd unknown /proc/21013/root (readlink: Permission denied) 2024-03-21T10:28:38.341 INFO:tasks.workunit.client.0.smithi047.stdout:kworker/R 21013 root txt unknown /proc/21013/exe (readlink: Permission denied) 2024-03-21T10:28:38.341 INFO:tasks.workunit.client.0.smithi047.stdout:kworker/R 21013 root NOFD /proc/21013/fd (opendir: Permission denied) 2024-03-21T10:28:38.341 INFO:tasks.workunit.client.0.smithi047.stdout:kworker/R 21014 root cwd unknown /proc/21014/cwd (readlink: Permission denied) 2024-03-21T10:28:38.341 INFO:tasks.workunit.client.0.smithi047.stdout:kworker/R 21014 root rtd unknown /proc/21014/root (readlink: Permission denied) 2024-03-21T10:28:38.341 INFO:tasks.workunit.client.0.smithi047.stdout:kworker/R 21014 root txt unknown /proc/21014/exe (readlink: Permission denied) 2024-03-21T10:28:38.341 INFO:tasks.workunit.client.0.smithi047.stdout:kworker/R 21014 root NOFD /proc/21014/fd (opendir: Permission denied) 2024-03-21T10:28:38.341 INFO:tasks.workunit.client.0.smithi047.stdout:kworker/R 21015 root cwd unknown /proc/21015/cwd (readlink: Permission denied) 2024-03-21T10:28:38.341 INFO:tasks.workunit.client.0.smithi047.stdout:kworker/R 21015 root rtd unknown /proc/21015/root (readlink: Permission denied) 2024-03-21T10:28:38.341 INFO:tasks.workunit.client.0.smithi047.stdout:kworker/R 21015 root txt unknown /proc/21015/exe (readlink: Permission denied) 2024-03-21T10:28:38.341 INFO:tasks.workunit.client.0.smithi047.stdout:kworker/R 21015 root NOFD /proc/21015/fd (opendir: Permission denied) 2024-03-21T10:28:38.341 INFO:tasks.workunit.client.0.smithi047.stdout:kworker/R 21016 root cwd unknown /proc/21016/cwd (readlink: Permission denied) 2024-03-21T10:28:38.342 INFO:tasks.workunit.client.0.smithi047.stdout:kworker/R 21016 root rtd unknown /proc/21016/root (readlink: Permission denied) 2024-03-21T10:28:38.342 INFO:tasks.workunit.client.0.smithi047.stdout:kworker/R 21016 root txt unknown /proc/21016/exe (readlink: Permission denied) 2024-03-21T10:28:38.342 INFO:tasks.workunit.client.0.smithi047.stdout:kworker/R 21016 root NOFD /proc/21016/fd (opendir: Permission denied) 2024-03-21T10:28:38.342 INFO:tasks.workunit.client.0.smithi047.stdout:kworker/R 21017 root cwd unknown /proc/21017/cwd (readlink: Permission denied) 2024-03-21T10:28:38.342 INFO:tasks.workunit.client.0.smithi047.stdout:kworker/R 21017 root rtd unknown /proc/21017/root (readlink: Permission denied) 2024-03-21T10:28:38.342 INFO:tasks.workunit.client.0.smithi047.stdout:kworker/R 21017 root txt unknown /proc/21017/exe (readlink: Permission denied) 2024-03-21T10:28:38.342 INFO:tasks.workunit.client.0.smithi047.stdout:kworker/R 21017 root NOFD /proc/21017/fd (opendir: Permission denied) 2024-03-21T10:28:38.342 INFO:tasks.workunit.client.0.smithi047.stdout:kworker/R 21018 root cwd unknown /proc/21018/cwd (readlink: Permission denied) 2024-03-21T10:28:38.342 INFO:tasks.workunit.client.0.smithi047.stdout:kworker/R 21018 root rtd unknown /proc/21018/root (readlink: Permission denied) 2024-03-21T10:28:38.342 INFO:tasks.workunit.client.0.smithi047.stdout:kworker/R 21018 root txt unknown /proc/21018/exe (readlink: Permission denied) 2024-03-21T10:28:38.342 INFO:tasks.workunit.client.0.smithi047.stdout:kworker/R 21018 root NOFD /proc/21018/fd (opendir: Permission denied) 2024-03-21T10:28:38.342 INFO:tasks.workunit.client.0.smithi047.stdout:kworker/R 21019 root cwd unknown /proc/21019/cwd (readlink: Permission denied) 2024-03-21T10:28:38.342 INFO:tasks.workunit.client.0.smithi047.stdout:kworker/R 21019 root rtd unknown /proc/21019/root (readlink: Permission denied) 2024-03-21T10:28:38.350 INFO:tasks.workunit.client.0.smithi047.stdout:kworker/R 21019 root txt unknown /proc/21019/exe (readlink: Permission denied) 2024-03-21T10:28:38.350 INFO:tasks.workunit.client.0.smithi047.stdout:kworker/R 21019 root NOFD /proc/21019/fd (opendir: Permission denied) 2024-03-21T10:28:38.350 INFO:tasks.workunit.client.0.smithi047.stdout:kworker/R 21020 root cwd unknown /proc/21020/cwd (readlink: Permission denied) 2024-03-21T10:28:38.351 INFO:tasks.workunit.client.0.smithi047.stdout:kworker/R 21020 root rtd unknown /proc/21020/root (readlink: Permission denied) 2024-03-21T10:28:38.351 INFO:tasks.workunit.client.0.smithi047.stdout:kworker/R 21020 root txt unknown /proc/21020/exe (readlink: Permission denied) 2024-03-21T10:28:38.351 INFO:tasks.workunit.client.0.smithi047.stdout:kworker/R 21020 root NOFD /proc/21020/fd (opendir: Permission denied) 2024-03-21T10:28:38.351 INFO:tasks.workunit.client.0.smithi047.stdout:xfsaild/d 21021 root cwd unknown /proc/21021/cwd (readlink: Permission denied) 2024-03-21T10:28:38.351 INFO:tasks.workunit.client.0.smithi047.stdout:xfsaild/d 21021 root rtd unknown /proc/21021/root (readlink: Permission denied) 2024-03-21T10:28:38.351 INFO:tasks.workunit.client.0.smithi047.stdout:xfsaild/d 21021 root txt unknown /proc/21021/exe (readlink: Permission denied) 2024-03-21T10:28:38.351 INFO:tasks.workunit.client.0.smithi047.stdout:xfsaild/d 21021 root NOFD /proc/21021/fd (opendir: Permission denied) 2024-03-21T10:28:38.351 INFO:tasks.workunit.client.0.smithi047.stdout:kworker/1 22647 root cwd unknown /proc/22647/cwd (readlink: Permission denied) 2024-03-21T10:28:38.351 INFO:tasks.workunit.client.0.smithi047.stdout:kworker/1 22647 root rtd unknown /proc/22647/root (readlink: Permission denied) 2024-03-21T10:28:38.351 INFO:tasks.workunit.client.0.smithi047.stdout:kworker/1 22647 root txt unknown /proc/22647/exe (readlink: Permission denied) 2024-03-21T10:28:38.351 INFO:tasks.workunit.client.0.smithi047.stdout:kworker/1 22647 root NOFD /proc/22647/fd (opendir: Permission denied) 2024-03-21T10:28:38.351 INFO:tasks.workunit.client.0.smithi047.stdout:sshd 22956 root cwd unknown /proc/22956/cwd (readlink: Permission denied) 2024-03-21T10:28:38.351 INFO:tasks.workunit.client.0.smithi047.stdout:sshd 22956 root rtd unknown /proc/22956/root (readlink: Permission denied) 2024-03-21T10:28:38.351 INFO:tasks.workunit.client.0.smithi047.stdout:sshd 22956 root txt unknown /proc/22956/exe (readlink: Permission denied) 2024-03-21T10:28:38.352 INFO:tasks.workunit.client.0.smithi047.stdout:sshd 22956 root NOFD /proc/22956/fd (opendir: Permission denied) 2024-03-21T10:28:38.352 INFO:tasks.workunit.client.0.smithi047.stdout:sshd 22959 ubuntu cwd unknown /proc/22959/cwd (readlink: Permission denied) 2024-03-21T10:28:38.352 INFO:tasks.workunit.client.0.smithi047.stdout:sshd 22959 ubuntu rtd unknown /proc/22959/root (readlink: Permission denied) 2024-03-21T10:28:38.352 INFO:tasks.workunit.client.0.smithi047.stdout:sshd 22959 ubuntu txt unknown /proc/22959/exe (readlink: Permission denied) 2024-03-21T10:28:38.352 INFO:tasks.workunit.client.0.smithi047.stdout:sshd 22959 ubuntu NOFD /proc/22959/fd (opendir: Permission denied) 2024-03-21T10:28:38.352 INFO:tasks.workunit.client.0.smithi047.stdout:chronyd 23019 chrony cwd unknown /proc/23019/cwd (readlink: Permission denied) 2024-03-21T10:28:38.352 INFO:tasks.workunit.client.0.smithi047.stdout:chronyd 23019 chrony rtd unknown /proc/23019/root (readlink: Permission denied) 2024-03-21T10:28:38.352 INFO:tasks.workunit.client.0.smithi047.stdout:chronyd 23019 chrony txt unknown /proc/23019/exe (readlink: Permission denied) 2024-03-21T10:28:38.352 INFO:tasks.workunit.client.0.smithi047.stdout:chronyd 23019 chrony NOFD /proc/23019/fd (opendir: Permission denied) 2024-03-21T10:28:38.352 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-cras 23350 ceph cwd unknown /proc/23350/cwd (readlink: Permission denied) 2024-03-21T10:28:38.352 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-cras 23350 ceph rtd unknown /proc/23350/root (readlink: Permission denied) 2024-03-21T10:28:38.352 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-cras 23350 ceph txt unknown /proc/23350/exe (readlink: Permission denied) 2024-03-21T10:28:38.352 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-cras 23350 ceph NOFD /proc/23350/fd (opendir: Permission denied) 2024-03-21T10:28:38.352 INFO:tasks.workunit.client.0.smithi047.stdout:kworker/3 24237 root cwd unknown /proc/24237/cwd (readlink: Permission denied) 2024-03-21T10:28:38.352 INFO:tasks.workunit.client.0.smithi047.stdout:kworker/3 24237 root rtd unknown /proc/24237/root (readlink: Permission denied) 2024-03-21T10:28:38.353 INFO:tasks.workunit.client.0.smithi047.stdout:kworker/3 24237 root txt unknown /proc/24237/exe (readlink: Permission denied) 2024-03-21T10:28:38.353 INFO:tasks.workunit.client.0.smithi047.stdout:kworker/3 24237 root NOFD /proc/24237/fd (opendir: Permission denied) 2024-03-21T10:28:38.353 INFO:tasks.workunit.client.0.smithi047.stdout:kworker/0 24896 root cwd unknown /proc/24896/cwd (readlink: Permission denied) 2024-03-21T10:28:38.353 INFO:tasks.workunit.client.0.smithi047.stdout:kworker/0 24896 root rtd unknown /proc/24896/root (readlink: Permission denied) 2024-03-21T10:28:38.353 INFO:tasks.workunit.client.0.smithi047.stdout:kworker/0 24896 root txt unknown /proc/24896/exe (readlink: Permission denied) 2024-03-21T10:28:38.353 INFO:tasks.workunit.client.0.smithi047.stdout:kworker/0 24896 root NOFD /proc/24896/fd (opendir: Permission denied) 2024-03-21T10:28:38.353 INFO:tasks.workunit.client.0.smithi047.stdout:sudo 30548 root cwd unknown /proc/30548/cwd (readlink: Permission denied) 2024-03-21T10:28:38.353 INFO:tasks.workunit.client.0.smithi047.stdout:sudo 30548 root rtd unknown /proc/30548/root (readlink: Permission denied) 2024-03-21T10:28:38.353 INFO:tasks.workunit.client.0.smithi047.stdout:sudo 30548 root txt unknown /proc/30548/exe (readlink: Permission denied) 2024-03-21T10:28:38.353 INFO:tasks.workunit.client.0.smithi047.stdout:sudo 30548 root NOFD /proc/30548/fd (opendir: Permission denied) 2024-03-21T10:28:38.353 INFO:tasks.workunit.client.0.smithi047.stdout:sudo 30564 root cwd unknown /proc/30564/cwd (readlink: Permission denied) 2024-03-21T10:28:38.353 INFO:tasks.workunit.client.0.smithi047.stdout:sudo 30564 root rtd unknown /proc/30564/root (readlink: Permission denied) 2024-03-21T10:28:38.353 INFO:tasks.workunit.client.0.smithi047.stdout:sudo 30564 root txt unknown /proc/30564/exe (readlink: Permission denied) 2024-03-21T10:28:38.353 INFO:tasks.workunit.client.0.smithi047.stdout:sudo 30564 root NOFD /proc/30564/fd (opendir: Permission denied) 2024-03-21T10:28:38.353 INFO:tasks.workunit.client.0.smithi047.stdout:journalct 30578 root cwd unknown /proc/30578/cwd (readlink: Permission denied) 2024-03-21T10:28:38.354 INFO:tasks.workunit.client.0.smithi047.stdout:journalct 30578 root rtd unknown /proc/30578/root (readlink: Permission denied) 2024-03-21T10:28:38.354 INFO:tasks.workunit.client.0.smithi047.stdout:journalct 30578 root txt unknown /proc/30578/exe (readlink: Permission denied) 2024-03-21T10:28:38.354 INFO:tasks.workunit.client.0.smithi047.stdout:journalct 30578 root NOFD /proc/30578/fd (opendir: Permission denied) 2024-03-21T10:28:38.354 INFO:tasks.workunit.client.0.smithi047.stdout:journalct 30594 root cwd unknown /proc/30594/cwd (readlink: Permission denied) 2024-03-21T10:28:38.354 INFO:tasks.workunit.client.0.smithi047.stdout:journalct 30594 root rtd unknown /proc/30594/root (readlink: Permission denied) 2024-03-21T10:28:38.354 INFO:tasks.workunit.client.0.smithi047.stdout:journalct 30594 root txt unknown /proc/30594/exe (readlink: Permission denied) 2024-03-21T10:28:38.354 INFO:tasks.workunit.client.0.smithi047.stdout:journalct 30594 root NOFD /proc/30594/fd (opendir: Permission denied) 2024-03-21T10:28:38.354 INFO:tasks.workunit.client.0.smithi047.stdout:kworker/4 30845 root cwd unknown /proc/30845/cwd (readlink: Permission denied) 2024-03-21T10:28:38.354 INFO:tasks.workunit.client.0.smithi047.stdout:kworker/4 30845 root rtd unknown /proc/30845/root (readlink: Permission denied) 2024-03-21T10:28:38.354 INFO:tasks.workunit.client.0.smithi047.stdout:kworker/4 30845 root txt unknown /proc/30845/exe (readlink: Permission denied) 2024-03-21T10:28:38.354 INFO:tasks.workunit.client.0.smithi047.stdout:kworker/4 30845 root NOFD /proc/30845/fd (opendir: Permission denied) 2024-03-21T10:28:38.354 INFO:tasks.workunit.client.0.smithi047.stdout:conmon 31543 root cwd unknown /proc/31543/cwd (readlink: Permission denied) 2024-03-21T10:28:38.354 INFO:tasks.workunit.client.0.smithi047.stdout:conmon 31543 root rtd unknown /proc/31543/root (readlink: Permission denied) 2024-03-21T10:28:38.354 INFO:tasks.workunit.client.0.smithi047.stdout:conmon 31543 root txt unknown /proc/31543/exe (readlink: Permission denied) 2024-03-21T10:28:38.355 INFO:tasks.workunit.client.0.smithi047.stdout:conmon 31543 root NOFD /proc/31543/fd (opendir: Permission denied) 2024-03-21T10:28:38.355 INFO:tasks.workunit.client.0.smithi047.stdout:podman-in 31545 root cwd unknown /proc/31545/cwd (readlink: Permission denied) 2024-03-21T10:28:38.355 INFO:tasks.workunit.client.0.smithi047.stdout:podman-in 31545 root rtd unknown /proc/31545/root (readlink: Permission denied) 2024-03-21T10:28:38.355 INFO:tasks.workunit.client.0.smithi047.stdout:podman-in 31545 root txt unknown /proc/31545/exe (readlink: Permission denied) 2024-03-21T10:28:38.355 INFO:tasks.workunit.client.0.smithi047.stdout:podman-in 31545 root NOFD /proc/31545/fd (opendir: Permission denied) 2024-03-21T10:28:38.355 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mon 31547 ceph cwd unknown /proc/31547/cwd (readlink: Permission denied) 2024-03-21T10:28:38.355 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mon 31547 ceph rtd unknown /proc/31547/root (readlink: Permission denied) 2024-03-21T10:28:38.355 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mon 31547 ceph txt unknown /proc/31547/exe (readlink: Permission denied) 2024-03-21T10:28:38.355 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mon 31547 ceph NOFD /proc/31547/fd (opendir: Permission denied) 2024-03-21T10:28:38.355 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mon 31547 31548 log ceph cwd unknown /proc/31547/task/31548/cwd (readlink: Permission denied) 2024-03-21T10:28:38.355 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mon 31547 31548 log ceph rtd unknown /proc/31547/task/31548/root (readlink: Permission denied) 2024-03-21T10:28:38.355 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mon 31547 31548 log ceph txt unknown /proc/31547/task/31548/exe (readlink: Permission denied) 2024-03-21T10:28:38.355 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mon 31547 31548 log ceph NOFD /proc/31547/task/31548/fd (opendir: Permission denied) 2024-03-21T10:28:38.356 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mon 31547 31549 service ceph cwd unknown /proc/31547/task/31549/cwd (readlink: Permission denied) 2024-03-21T10:28:38.356 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mon 31547 31549 service ceph rtd unknown /proc/31547/task/31549/root (readlink: Permission denied) 2024-03-21T10:28:38.356 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mon 31547 31549 service ceph txt unknown /proc/31547/task/31549/exe (readlink: Permission denied) 2024-03-21T10:28:38.356 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mon 31547 31549 service ceph NOFD /proc/31547/task/31549/fd (opendir: Permission denied) 2024-03-21T10:28:38.356 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mon 31547 31550 admin_soc ceph cwd unknown /proc/31547/task/31550/cwd (readlink: Permission denied) 2024-03-21T10:28:38.356 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mon 31547 31550 admin_soc ceph rtd unknown /proc/31547/task/31550/root (readlink: Permission denied) 2024-03-21T10:28:38.356 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mon 31547 31550 admin_soc ceph txt unknown /proc/31547/task/31550/exe (readlink: Permission denied) 2024-03-21T10:28:38.356 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mon 31547 31550 admin_soc ceph NOFD /proc/31547/task/31550/fd (opendir: Permission denied) 2024-03-21T10:28:38.356 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mon 31547 31551 signal_ha ceph cwd unknown /proc/31547/task/31551/cwd (readlink: Permission denied) 2024-03-21T10:28:38.356 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mon 31547 31551 signal_ha ceph rtd unknown /proc/31547/task/31551/root (readlink: Permission denied) 2024-03-21T10:28:38.356 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mon 31547 31551 signal_ha ceph txt unknown /proc/31547/task/31551/exe (readlink: Permission denied) 2024-03-21T10:28:38.356 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mon 31547 31551 signal_ha ceph NOFD /proc/31547/task/31551/fd (opendir: Permission denied) 2024-03-21T10:28:38.356 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mon 31547 31552 rocksdb:l ceph cwd unknown /proc/31547/task/31552/cwd (readlink: Permission denied) 2024-03-21T10:28:38.356 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mon 31547 31552 rocksdb:l ceph rtd unknown /proc/31547/task/31552/root (readlink: Permission denied) 2024-03-21T10:28:38.357 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mon 31547 31552 rocksdb:l ceph txt unknown /proc/31547/task/31552/exe (readlink: Permission denied) 2024-03-21T10:28:38.357 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mon 31547 31552 rocksdb:l ceph NOFD /proc/31547/task/31552/fd (opendir: Permission denied) 2024-03-21T10:28:38.357 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mon 31547 31553 rocksdb:h ceph cwd unknown /proc/31547/task/31553/cwd (readlink: Permission denied) 2024-03-21T10:28:38.357 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mon 31547 31553 rocksdb:h ceph rtd unknown /proc/31547/task/31553/root (readlink: Permission denied) 2024-03-21T10:28:38.357 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mon 31547 31553 rocksdb:h ceph txt unknown /proc/31547/task/31553/exe (readlink: Permission denied) 2024-03-21T10:28:38.357 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mon 31547 31553 rocksdb:h ceph NOFD /proc/31547/task/31553/fd (opendir: Permission denied) 2024-03-21T10:28:38.357 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mon 31547 31569 ceph-mon ceph cwd unknown /proc/31547/task/31569/cwd (readlink: Permission denied) 2024-03-21T10:28:38.357 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mon 31547 31569 ceph-mon ceph rtd unknown /proc/31547/task/31569/root (readlink: Permission denied) 2024-03-21T10:28:38.357 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mon 31547 31569 ceph-mon ceph txt unknown /proc/31547/task/31569/exe (readlink: Permission denied) 2024-03-21T10:28:38.357 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mon 31547 31569 ceph-mon ceph NOFD /proc/31547/task/31569/fd (opendir: Permission denied) 2024-03-21T10:28:38.357 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mon 31547 31570 fn_monsto ceph cwd unknown /proc/31547/task/31570/cwd (readlink: Permission denied) 2024-03-21T10:28:38.357 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mon 31547 31570 fn_monsto ceph rtd unknown /proc/31547/task/31570/root (readlink: Permission denied) 2024-03-21T10:28:38.357 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mon 31547 31570 fn_monsto ceph txt unknown /proc/31547/task/31570/exe (readlink: Permission denied) 2024-03-21T10:28:38.357 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mon 31547 31570 fn_monsto ceph NOFD /proc/31547/task/31570/fd (opendir: Permission denied) 2024-03-21T10:28:38.357 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mon 31547 31571 msgr-work ceph cwd unknown /proc/31547/task/31571/cwd (readlink: Permission denied) 2024-03-21T10:28:38.358 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mon 31547 31571 msgr-work ceph rtd unknown /proc/31547/task/31571/root (readlink: Permission denied) 2024-03-21T10:28:38.358 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mon 31547 31571 msgr-work ceph txt unknown /proc/31547/task/31571/exe (readlink: Permission denied) 2024-03-21T10:28:38.358 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mon 31547 31571 msgr-work ceph NOFD /proc/31547/task/31571/fd (opendir: Permission denied) 2024-03-21T10:28:38.358 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mon 31547 31572 msgr-work ceph cwd unknown /proc/31547/task/31572/cwd (readlink: Permission denied) 2024-03-21T10:28:38.358 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mon 31547 31572 msgr-work ceph rtd unknown /proc/31547/task/31572/root (readlink: Permission denied) 2024-03-21T10:28:38.358 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mon 31547 31572 msgr-work ceph txt unknown /proc/31547/task/31572/exe (readlink: Permission denied) 2024-03-21T10:28:38.358 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mon 31547 31572 msgr-work ceph NOFD /proc/31547/task/31572/fd (opendir: Permission denied) 2024-03-21T10:28:38.358 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mon 31547 31573 msgr-work ceph cwd unknown /proc/31547/task/31573/cwd (readlink: Permission denied) 2024-03-21T10:28:38.358 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mon 31547 31573 msgr-work ceph rtd unknown /proc/31547/task/31573/root (readlink: Permission denied) 2024-03-21T10:28:38.358 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mon 31547 31573 msgr-work ceph txt unknown /proc/31547/task/31573/exe (readlink: Permission denied) 2024-03-21T10:28:38.358 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mon 31547 31573 msgr-work ceph NOFD /proc/31547/task/31573/fd (opendir: Permission denied) 2024-03-21T10:28:38.358 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mon 31547 31574 OpHistory ceph cwd unknown /proc/31547/task/31574/cwd (readlink: Permission denied) 2024-03-21T10:28:38.359 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mon 31547 31574 OpHistory ceph rtd unknown /proc/31547/task/31574/root (readlink: Permission denied) 2024-03-21T10:28:38.359 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mon 31547 31574 OpHistory ceph txt unknown /proc/31547/task/31574/exe (readlink: Permission denied) 2024-03-21T10:28:38.359 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mon 31547 31574 OpHistory ceph NOFD /proc/31547/task/31574/fd (opendir: Permission denied) 2024-03-21T10:28:38.359 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mon 31547 31575 fin ceph cwd unknown /proc/31547/task/31575/cwd (readlink: Permission denied) 2024-03-21T10:28:38.359 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mon 31547 31575 fin ceph rtd unknown /proc/31547/task/31575/root (readlink: Permission denied) 2024-03-21T10:28:38.359 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mon 31547 31575 fin ceph txt unknown /proc/31547/task/31575/exe (readlink: Permission denied) 2024-03-21T10:28:38.359 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mon 31547 31575 fin ceph NOFD /proc/31547/task/31575/fd (opendir: Permission denied) 2024-03-21T10:28:38.359 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mon 31547 31576 safe_time ceph cwd unknown /proc/31547/task/31576/cwd (readlink: Permission denied) 2024-03-21T10:28:38.359 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mon 31547 31576 safe_time ceph rtd unknown /proc/31547/task/31576/root (readlink: Permission denied) 2024-03-21T10:28:38.359 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mon 31547 31576 safe_time ceph txt unknown /proc/31547/task/31576/exe (readlink: Permission denied) 2024-03-21T10:28:38.359 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mon 31547 31576 safe_time ceph NOFD /proc/31547/task/31576/fd (opendir: Permission denied) 2024-03-21T10:28:38.359 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mon 31547 31577 cpu_tp ceph cwd unknown /proc/31547/task/31577/cwd (readlink: Permission denied) 2024-03-21T10:28:38.359 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mon 31547 31577 cpu_tp ceph rtd unknown /proc/31547/task/31577/root (readlink: Permission denied) 2024-03-21T10:28:38.359 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mon 31547 31577 cpu_tp ceph txt unknown /proc/31547/task/31577/exe (readlink: Permission denied) 2024-03-21T10:28:38.359 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mon 31547 31577 cpu_tp ceph NOFD /proc/31547/task/31577/fd (opendir: Permission denied) 2024-03-21T10:28:38.360 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mon 31547 31578 cpu_tp ceph cwd unknown /proc/31547/task/31578/cwd (readlink: Permission denied) 2024-03-21T10:28:38.360 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mon 31547 31578 cpu_tp ceph rtd unknown /proc/31547/task/31578/root (readlink: Permission denied) 2024-03-21T10:28:38.360 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mon 31547 31578 cpu_tp ceph txt unknown /proc/31547/task/31578/exe (readlink: Permission denied) 2024-03-21T10:28:38.360 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mon 31547 31578 cpu_tp ceph NOFD /proc/31547/task/31578/fd (opendir: Permission denied) 2024-03-21T10:28:38.360 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mon 31547 31579 cpu_tp ceph cwd unknown /proc/31547/task/31579/cwd (readlink: Permission denied) 2024-03-21T10:28:38.360 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mon 31547 31579 cpu_tp ceph rtd unknown /proc/31547/task/31579/root (readlink: Permission denied) 2024-03-21T10:28:38.360 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mon 31547 31579 cpu_tp ceph txt unknown /proc/31547/task/31579/exe (readlink: Permission denied) 2024-03-21T10:28:38.360 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mon 31547 31579 cpu_tp ceph NOFD /proc/31547/task/31579/fd (opendir: Permission denied) 2024-03-21T10:28:38.360 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mon 31547 31580 cpu_tp ceph cwd unknown /proc/31547/task/31580/cwd (readlink: Permission denied) 2024-03-21T10:28:38.360 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mon 31547 31580 cpu_tp ceph rtd unknown /proc/31547/task/31580/root (readlink: Permission denied) 2024-03-21T10:28:38.360 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mon 31547 31580 cpu_tp ceph txt unknown /proc/31547/task/31580/exe (readlink: Permission denied) 2024-03-21T10:28:38.360 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mon 31547 31580 cpu_tp ceph NOFD /proc/31547/task/31580/fd (opendir: Permission denied) 2024-03-21T10:28:38.361 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mon 31547 31581 ms_dispat ceph cwd unknown /proc/31547/task/31581/cwd (readlink: Permission denied) 2024-03-21T10:28:38.361 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mon 31547 31581 ms_dispat ceph rtd unknown /proc/31547/task/31581/root (readlink: Permission denied) 2024-03-21T10:28:38.361 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mon 31547 31581 ms_dispat ceph txt unknown /proc/31547/task/31581/exe (readlink: Permission denied) 2024-03-21T10:28:38.361 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mon 31547 31581 ms_dispat ceph NOFD /proc/31547/task/31581/fd (opendir: Permission denied) 2024-03-21T10:28:38.361 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mon 31547 31582 ms_local ceph cwd unknown /proc/31547/task/31582/cwd (readlink: Permission denied) 2024-03-21T10:28:38.361 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mon 31547 31582 ms_local ceph rtd unknown /proc/31547/task/31582/root (readlink: Permission denied) 2024-03-21T10:28:38.361 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mon 31547 31582 ms_local ceph txt unknown /proc/31547/task/31582/exe (readlink: Permission denied) 2024-03-21T10:28:38.361 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mon 31547 31582 ms_local ceph NOFD /proc/31547/task/31582/fd (opendir: Permission denied) 2024-03-21T10:28:38.361 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mon 31547 31583 safe_time ceph cwd unknown /proc/31547/task/31583/cwd (readlink: Permission denied) 2024-03-21T10:28:38.361 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mon 31547 31583 safe_time ceph rtd unknown /proc/31547/task/31583/root (readlink: Permission denied) 2024-03-21T10:28:38.361 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mon 31547 31583 safe_time ceph txt unknown /proc/31547/task/31583/exe (readlink: Permission denied) 2024-03-21T10:28:38.361 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mon 31547 31583 safe_time ceph NOFD /proc/31547/task/31583/fd (opendir: Permission denied) 2024-03-21T10:28:38.361 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mon 31547 31584 ms_dispat ceph cwd unknown /proc/31547/task/31584/cwd (readlink: Permission denied) 2024-03-21T10:28:38.362 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mon 31547 31584 ms_dispat ceph rtd unknown /proc/31547/task/31584/root (readlink: Permission denied) 2024-03-21T10:28:38.362 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mon 31547 31584 ms_dispat ceph txt unknown /proc/31547/task/31584/exe (readlink: Permission denied) 2024-03-21T10:28:38.362 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mon 31547 31584 ms_dispat ceph NOFD /proc/31547/task/31584/fd (opendir: Permission denied) 2024-03-21T10:28:38.362 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mon 31547 31585 ms_local ceph cwd unknown /proc/31547/task/31585/cwd (readlink: Permission denied) 2024-03-21T10:28:38.362 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mon 31547 31585 ms_local ceph rtd unknown /proc/31547/task/31585/root (readlink: Permission denied) 2024-03-21T10:28:38.362 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mon 31547 31585 ms_local ceph txt unknown /proc/31547/task/31585/exe (readlink: Permission denied) 2024-03-21T10:28:38.362 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mon 31547 31585 ms_local ceph NOFD /proc/31547/task/31585/fd (opendir: Permission denied) 2024-03-21T10:28:38.362 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mon 31547 61434 rstore_co ceph cwd unknown /proc/31547/task/61434/cwd (readlink: Permission denied) 2024-03-21T10:28:38.362 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mon 31547 61434 rstore_co ceph rtd unknown /proc/31547/task/61434/root (readlink: Permission denied) 2024-03-21T10:28:38.362 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mon 31547 61434 rstore_co ceph txt unknown /proc/31547/task/61434/exe (readlink: Permission denied) 2024-03-21T10:28:38.362 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mon 31547 61434 rstore_co ceph NOFD /proc/31547/task/61434/fd (opendir: Permission denied) 2024-03-21T10:28:38.362 INFO:tasks.workunit.client.0.smithi047.stdout:conmon 31773 root cwd unknown /proc/31773/cwd (readlink: Permission denied) 2024-03-21T10:28:38.363 INFO:tasks.workunit.client.0.smithi047.stdout:conmon 31773 root rtd unknown /proc/31773/root (readlink: Permission denied) 2024-03-21T10:28:38.363 INFO:tasks.workunit.client.0.smithi047.stdout:conmon 31773 root txt unknown /proc/31773/exe (readlink: Permission denied) 2024-03-21T10:28:38.363 INFO:tasks.workunit.client.0.smithi047.stdout:conmon 31773 root NOFD /proc/31773/fd (opendir: Permission denied) 2024-03-21T10:28:38.363 INFO:tasks.workunit.client.0.smithi047.stdout:podman-in 31775 root cwd unknown /proc/31775/cwd (readlink: Permission denied) 2024-03-21T10:28:38.363 INFO:tasks.workunit.client.0.smithi047.stdout:podman-in 31775 root rtd unknown /proc/31775/root (readlink: Permission denied) 2024-03-21T10:28:38.363 INFO:tasks.workunit.client.0.smithi047.stdout:podman-in 31775 root txt unknown /proc/31775/exe (readlink: Permission denied) 2024-03-21T10:28:38.363 INFO:tasks.workunit.client.0.smithi047.stdout:podman-in 31775 root NOFD /proc/31775/fd (opendir: Permission denied) 2024-03-21T10:28:38.363 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mgr 31777 ceph cwd unknown /proc/31777/cwd (readlink: Permission denied) 2024-03-21T10:28:38.363 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mgr 31777 ceph rtd unknown /proc/31777/root (readlink: Permission denied) 2024-03-21T10:28:38.363 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mgr 31777 ceph txt unknown /proc/31777/exe (readlink: Permission denied) 2024-03-21T10:28:38.363 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mgr 31777 ceph NOFD /proc/31777/fd (opendir: Permission denied) 2024-03-21T10:28:38.363 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mgr 31777 85214 log ceph cwd unknown /proc/31777/task/85214/cwd (readlink: Permission denied) 2024-03-21T10:28:38.363 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mgr 31777 85214 log ceph rtd unknown /proc/31777/task/85214/root (readlink: Permission denied) 2024-03-21T10:28:38.363 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mgr 31777 85214 log ceph txt unknown /proc/31777/task/85214/exe (readlink: Permission denied) 2024-03-21T10:28:38.364 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mgr 31777 85214 log ceph NOFD /proc/31777/task/85214/fd (opendir: Permission denied) 2024-03-21T10:28:38.364 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mgr 31777 85216 msgr-work ceph cwd unknown /proc/31777/task/85216/cwd (readlink: Permission denied) 2024-03-21T10:28:38.364 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mgr 31777 85216 msgr-work ceph rtd unknown /proc/31777/task/85216/root (readlink: Permission denied) 2024-03-21T10:28:38.364 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mgr 31777 85216 msgr-work ceph txt unknown /proc/31777/task/85216/exe (readlink: Permission denied) 2024-03-21T10:28:38.364 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mgr 31777 85216 msgr-work ceph NOFD /proc/31777/task/85216/fd (opendir: Permission denied) 2024-03-21T10:28:38.364 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mgr 31777 85217 msgr-work ceph cwd unknown /proc/31777/task/85217/cwd (readlink: Permission denied) 2024-03-21T10:28:38.364 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mgr 31777 85217 msgr-work ceph rtd unknown /proc/31777/task/85217/root (readlink: Permission denied) 2024-03-21T10:28:38.364 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mgr 31777 85217 msgr-work ceph txt unknown /proc/31777/task/85217/exe (readlink: Permission denied) 2024-03-21T10:28:38.364 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mgr 31777 85217 msgr-work ceph NOFD /proc/31777/task/85217/fd (opendir: Permission denied) 2024-03-21T10:28:38.364 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mgr 31777 85218 msgr-work ceph cwd unknown /proc/31777/task/85218/cwd (readlink: Permission denied) 2024-03-21T10:28:38.364 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mgr 31777 85218 msgr-work ceph rtd unknown /proc/31777/task/85218/root (readlink: Permission denied) 2024-03-21T10:28:38.364 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mgr 31777 85218 msgr-work ceph txt unknown /proc/31777/task/85218/exe (readlink: Permission denied) 2024-03-21T10:28:38.364 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mgr 31777 85218 msgr-work ceph NOFD /proc/31777/task/85218/fd (opendir: Permission denied) 2024-03-21T10:28:38.365 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mgr 31777 85222 service ceph cwd unknown /proc/31777/task/85222/cwd (readlink: Permission denied) 2024-03-21T10:28:38.365 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mgr 31777 85222 service ceph rtd unknown /proc/31777/task/85222/root (readlink: Permission denied) 2024-03-21T10:28:38.365 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mgr 31777 85222 service ceph txt unknown /proc/31777/task/85222/exe (readlink: Permission denied) 2024-03-21T10:28:38.365 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mgr 31777 85222 service ceph NOFD /proc/31777/task/85222/fd (opendir: Permission denied) 2024-03-21T10:28:38.365 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mgr 31777 85223 admin_soc ceph cwd unknown /proc/31777/task/85223/cwd (readlink: Permission denied) 2024-03-21T10:28:38.365 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mgr 31777 85223 admin_soc ceph rtd unknown /proc/31777/task/85223/root (readlink: Permission denied) 2024-03-21T10:28:38.365 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mgr 31777 85223 admin_soc ceph txt unknown /proc/31777/task/85223/exe (readlink: Permission denied) 2024-03-21T10:28:38.365 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mgr 31777 85223 admin_soc ceph NOFD /proc/31777/task/85223/fd (opendir: Permission denied) 2024-03-21T10:28:38.365 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mgr 31777 85224 ceph_time ceph cwd unknown /proc/31777/task/85224/cwd (readlink: Permission denied) 2024-03-21T10:28:38.365 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mgr 31777 85224 ceph_time ceph rtd unknown /proc/31777/task/85224/root (readlink: Permission denied) 2024-03-21T10:28:38.365 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mgr 31777 85224 ceph_time ceph txt unknown /proc/31777/task/85224/exe (readlink: Permission denied) 2024-03-21T10:28:38.365 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mgr 31777 85224 ceph_time ceph NOFD /proc/31777/task/85224/fd (opendir: Permission denied) 2024-03-21T10:28:38.365 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mgr 31777 85225 fn_anonym ceph cwd unknown /proc/31777/task/85225/cwd (readlink: Permission denied) 2024-03-21T10:28:38.365 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mgr 31777 85225 fn_anonym ceph rtd unknown /proc/31777/task/85225/root (readlink: Permission denied) 2024-03-21T10:28:38.366 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mgr 31777 85225 fn_anonym ceph txt unknown /proc/31777/task/85225/exe (readlink: Permission denied) 2024-03-21T10:28:38.366 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mgr 31777 85225 fn_anonym ceph NOFD /proc/31777/task/85225/fd (opendir: Permission denied) 2024-03-21T10:28:38.366 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mgr 31777 85226 signal_ha ceph cwd unknown /proc/31777/task/85226/cwd (readlink: Permission denied) 2024-03-21T10:28:38.366 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mgr 31777 85226 signal_ha ceph rtd unknown /proc/31777/task/85226/root (readlink: Permission denied) 2024-03-21T10:28:38.366 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mgr 31777 85226 signal_ha ceph txt unknown /proc/31777/task/85226/exe (readlink: Permission denied) 2024-03-21T10:28:38.366 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mgr 31777 85226 signal_ha ceph NOFD /proc/31777/task/85226/fd (opendir: Permission denied) 2024-03-21T10:28:38.366 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mgr 31777 85227 mgrsb-fin ceph cwd unknown /proc/31777/task/85227/cwd (readlink: Permission denied) 2024-03-21T10:28:38.366 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mgr 31777 85227 mgrsb-fin ceph rtd unknown /proc/31777/task/85227/root (readlink: Permission denied) 2024-03-21T10:28:38.366 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mgr 31777 85227 mgrsb-fin ceph txt unknown /proc/31777/task/85227/exe (readlink: Permission denied) 2024-03-21T10:28:38.366 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mgr 31777 85227 mgrsb-fin ceph NOFD /proc/31777/task/85227/fd (opendir: Permission denied) 2024-03-21T10:28:38.366 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mgr 31777 85228 ms_dispat ceph cwd unknown /proc/31777/task/85228/cwd (readlink: Permission denied) 2024-03-21T10:28:38.367 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mgr 31777 85228 ms_dispat ceph rtd unknown /proc/31777/task/85228/root (readlink: Permission denied) 2024-03-21T10:28:38.367 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mgr 31777 85228 ms_dispat ceph txt unknown /proc/31777/task/85228/exe (readlink: Permission denied) 2024-03-21T10:28:38.367 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mgr 31777 85228 ms_dispat ceph NOFD /proc/31777/task/85228/fd (opendir: Permission denied) 2024-03-21T10:28:38.367 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mgr 31777 85229 ms_local ceph cwd unknown /proc/31777/task/85229/cwd (readlink: Permission denied) 2024-03-21T10:28:38.367 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mgr 31777 85229 ms_local ceph rtd unknown /proc/31777/task/85229/root (readlink: Permission denied) 2024-03-21T10:28:38.367 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mgr 31777 85229 ms_local ceph txt unknown /proc/31777/task/85229/exe (readlink: Permission denied) 2024-03-21T10:28:38.367 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mgr 31777 85229 ms_local ceph NOFD /proc/31777/task/85229/fd (opendir: Permission denied) 2024-03-21T10:28:38.367 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mgr 31777 85230 io_contex ceph cwd unknown /proc/31777/task/85230/cwd (readlink: Permission denied) 2024-03-21T10:28:38.367 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mgr 31777 85230 io_contex ceph rtd unknown /proc/31777/task/85230/root (readlink: Permission denied) 2024-03-21T10:28:38.367 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mgr 31777 85230 io_contex ceph txt unknown /proc/31777/task/85230/exe (readlink: Permission denied) 2024-03-21T10:28:38.367 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mgr 31777 85230 io_contex ceph NOFD /proc/31777/task/85230/fd (opendir: Permission denied) 2024-03-21T10:28:38.367 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mgr 31777 85231 io_contex ceph cwd unknown /proc/31777/task/85231/cwd (readlink: Permission denied) 2024-03-21T10:28:38.368 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mgr 31777 85231 io_contex ceph rtd unknown /proc/31777/task/85231/root (readlink: Permission denied) 2024-03-21T10:28:38.368 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mgr 31777 85231 io_contex ceph txt unknown /proc/31777/task/85231/exe (readlink: Permission denied) 2024-03-21T10:28:38.368 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mgr 31777 85231 io_contex ceph NOFD /proc/31777/task/85231/fd (opendir: Permission denied) 2024-03-21T10:28:38.368 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mgr 31777 85232 safe_time ceph cwd unknown /proc/31777/task/85232/cwd (readlink: Permission denied) 2024-03-21T10:28:38.368 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mgr 31777 85232 safe_time ceph rtd unknown /proc/31777/task/85232/root (readlink: Permission denied) 2024-03-21T10:28:38.368 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mgr 31777 85232 safe_time ceph txt unknown /proc/31777/task/85232/exe (readlink: Permission denied) 2024-03-21T10:28:38.368 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mgr 31777 85232 safe_time ceph NOFD /proc/31777/task/85232/fd (opendir: Permission denied) 2024-03-21T10:28:38.368 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mgr 31777 85233 safe_time ceph cwd unknown /proc/31777/task/85233/cwd (readlink: Permission denied) 2024-03-21T10:28:38.368 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mgr 31777 85233 safe_time ceph rtd unknown /proc/31777/task/85233/root (readlink: Permission denied) 2024-03-21T10:28:38.368 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mgr 31777 85233 safe_time ceph txt unknown /proc/31777/task/85233/exe (readlink: Permission denied) 2024-03-21T10:28:38.368 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mgr 31777 85233 safe_time ceph NOFD /proc/31777/task/85233/fd (opendir: Permission denied) 2024-03-21T10:28:38.368 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mgr 31777 85234 safe_time ceph cwd unknown /proc/31777/task/85234/cwd (readlink: Permission denied) 2024-03-21T10:28:38.368 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mgr 31777 85234 safe_time ceph rtd unknown /proc/31777/task/85234/root (readlink: Permission denied) 2024-03-21T10:28:38.368 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mgr 31777 85234 safe_time ceph txt unknown /proc/31777/task/85234/exe (readlink: Permission denied) 2024-03-21T10:28:38.368 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mgr 31777 85234 safe_time ceph NOFD /proc/31777/task/85234/fd (opendir: Permission denied) 2024-03-21T10:28:38.368 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mgr 31777 85235 fn_anonym ceph cwd unknown /proc/31777/task/85235/cwd (readlink: Permission denied) 2024-03-21T10:28:38.369 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mgr 31777 85235 fn_anonym ceph rtd unknown /proc/31777/task/85235/root (readlink: Permission denied) 2024-03-21T10:28:38.369 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mgr 31777 85235 fn_anonym ceph txt unknown /proc/31777/task/85235/exe (readlink: Permission denied) 2024-03-21T10:28:38.369 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mgr 31777 85235 fn_anonym ceph NOFD /proc/31777/task/85235/fd (opendir: Permission denied) 2024-03-21T10:28:38.369 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mgr 31777 85236 flusher ceph cwd unknown /proc/31777/task/85236/cwd (readlink: Permission denied) 2024-03-21T10:28:38.369 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mgr 31777 85236 flusher ceph rtd unknown /proc/31777/task/85236/root (readlink: Permission denied) 2024-03-21T10:28:38.369 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mgr 31777 85236 flusher ceph txt unknown /proc/31777/task/85236/exe (readlink: Permission denied) 2024-03-21T10:28:38.369 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mgr 31777 85236 flusher ceph NOFD /proc/31777/task/85236/fd (opendir: Permission denied) 2024-03-21T10:28:38.369 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mgr 31777 85237 safe_time ceph cwd unknown /proc/31777/task/85237/cwd (readlink: Permission denied) 2024-03-21T10:28:38.369 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mgr 31777 85237 safe_time ceph rtd unknown /proc/31777/task/85237/root (readlink: Permission denied) 2024-03-21T10:28:38.369 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mgr 31777 85237 safe_time ceph txt unknown /proc/31777/task/85237/exe (readlink: Permission denied) 2024-03-21T10:28:38.369 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mgr 31777 85237 safe_time ceph NOFD /proc/31777/task/85237/fd (opendir: Permission denied) 2024-03-21T10:28:38.369 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mgr 31777 85400 mgr-fin ceph cwd unknown /proc/31777/task/85400/cwd (readlink: Permission denied) 2024-03-21T10:28:38.369 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mgr 31777 85400 mgr-fin ceph rtd unknown /proc/31777/task/85400/root (readlink: Permission denied) 2024-03-21T10:28:38.370 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mgr 31777 85400 mgr-fin ceph txt unknown /proc/31777/task/85400/exe (readlink: Permission denied) 2024-03-21T10:28:38.370 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mgr 31777 85400 mgr-fin ceph NOFD /proc/31777/task/85400/fd (opendir: Permission denied) 2024-03-21T10:28:38.370 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mgr 31777 85401 ms_dispat ceph cwd unknown /proc/31777/task/85401/cwd (readlink: Permission denied) 2024-03-21T10:28:38.370 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mgr 31777 85401 ms_dispat ceph rtd unknown /proc/31777/task/85401/root (readlink: Permission denied) 2024-03-21T10:28:38.370 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mgr 31777 85401 ms_dispat ceph txt unknown /proc/31777/task/85401/exe (readlink: Permission denied) 2024-03-21T10:28:38.370 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mgr 31777 85401 ms_dispat ceph NOFD /proc/31777/task/85401/fd (opendir: Permission denied) 2024-03-21T10:28:38.370 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mgr 31777 85402 ms_local ceph cwd unknown /proc/31777/task/85402/cwd (readlink: Permission denied) 2024-03-21T10:28:38.370 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mgr 31777 85402 ms_local ceph rtd unknown /proc/31777/task/85402/root (readlink: Permission denied) 2024-03-21T10:28:38.370 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mgr 31777 85402 ms_local ceph txt unknown /proc/31777/task/85402/exe (readlink: Permission denied) 2024-03-21T10:28:38.370 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mgr 31777 85402 ms_local ceph NOFD /proc/31777/task/85402/fd (opendir: Permission denied) 2024-03-21T10:28:38.370 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mgr 31777 85403 safe_time ceph cwd unknown /proc/31777/task/85403/cwd (readlink: Permission denied) 2024-03-21T10:28:38.371 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mgr 31777 85403 safe_time ceph rtd unknown /proc/31777/task/85403/root (readlink: Permission denied) 2024-03-21T10:28:38.371 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mgr 31777 85403 safe_time ceph txt unknown /proc/31777/task/85403/exe (readlink: Permission denied) 2024-03-21T10:28:38.371 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mgr 31777 85403 safe_time ceph NOFD /proc/31777/task/85403/fd (opendir: Permission denied) 2024-03-21T10:28:38.371 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mgr 31777 85404 cmdfin ceph cwd unknown /proc/31777/task/85404/cwd (readlink: Permission denied) 2024-03-21T10:28:38.371 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mgr 31777 85404 cmdfin ceph rtd unknown /proc/31777/task/85404/root (readlink: Permission denied) 2024-03-21T10:28:38.371 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mgr 31777 85404 cmdfin ceph txt unknown /proc/31777/task/85404/exe (readlink: Permission denied) 2024-03-21T10:28:38.371 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mgr 31777 85404 cmdfin ceph NOFD /proc/31777/task/85404/fd (opendir: Permission denied) 2024-03-21T10:28:38.371 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mgr 31777 85408 io_contex ceph cwd unknown /proc/31777/task/85408/cwd (readlink: Permission denied) 2024-03-21T10:28:38.371 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mgr 31777 85408 io_contex ceph rtd unknown /proc/31777/task/85408/root (readlink: Permission denied) 2024-03-21T10:28:38.371 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mgr 31777 85408 io_contex ceph txt unknown /proc/31777/task/85408/exe (readlink: Permission denied) 2024-03-21T10:28:38.371 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mgr 31777 85408 io_contex ceph NOFD /proc/31777/task/85408/fd (opendir: Permission denied) 2024-03-21T10:28:38.371 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mgr 31777 85409 io_contex ceph cwd unknown /proc/31777/task/85409/cwd (readlink: Permission denied) 2024-03-21T10:28:38.371 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mgr 31777 85409 io_contex ceph rtd unknown /proc/31777/task/85409/root (readlink: Permission denied) 2024-03-21T10:28:38.371 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mgr 31777 85409 io_contex ceph txt unknown /proc/31777/task/85409/exe (readlink: Permission denied) 2024-03-21T10:28:38.372 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mgr 31777 85409 io_contex ceph NOFD /proc/31777/task/85409/fd (opendir: Permission denied) 2024-03-21T10:28:38.372 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mgr 31777 85410 ceph_time ceph cwd unknown /proc/31777/task/85410/cwd (readlink: Permission denied) 2024-03-21T10:28:38.372 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mgr 31777 85410 ceph_time ceph rtd unknown /proc/31777/task/85410/root (readlink: Permission denied) 2024-03-21T10:28:38.372 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mgr 31777 85410 ceph_time ceph txt unknown /proc/31777/task/85410/exe (readlink: Permission denied) 2024-03-21T10:28:38.372 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mgr 31777 85410 ceph_time ceph NOFD /proc/31777/task/85410/fd (opendir: Permission denied) 2024-03-21T10:28:38.372 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mgr 31777 85411 ms_dispat ceph cwd unknown /proc/31777/task/85411/cwd (readlink: Permission denied) 2024-03-21T10:28:38.372 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mgr 31777 85411 ms_dispat ceph rtd unknown /proc/31777/task/85411/root (readlink: Permission denied) 2024-03-21T10:28:38.372 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mgr 31777 85411 ms_dispat ceph txt unknown /proc/31777/task/85411/exe (readlink: Permission denied) 2024-03-21T10:28:38.372 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mgr 31777 85411 ms_dispat ceph NOFD /proc/31777/task/85411/fd (opendir: Permission denied) 2024-03-21T10:28:38.372 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mgr 31777 85412 ms_local ceph cwd unknown /proc/31777/task/85412/cwd (readlink: Permission denied) 2024-03-21T10:28:38.372 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mgr 31777 85412 ms_local ceph rtd unknown /proc/31777/task/85412/root (readlink: Permission denied) 2024-03-21T10:28:38.372 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mgr 31777 85412 ms_local ceph txt unknown /proc/31777/task/85412/exe (readlink: Permission denied) 2024-03-21T10:28:38.372 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mgr 31777 85412 ms_local ceph NOFD /proc/31777/task/85412/fd (opendir: Permission denied) 2024-03-21T10:28:38.372 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mgr 31777 85413 safe_time ceph cwd unknown /proc/31777/task/85413/cwd (readlink: Permission denied) 2024-03-21T10:28:38.373 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mgr 31777 85413 safe_time ceph rtd unknown /proc/31777/task/85413/root (readlink: Permission denied) 2024-03-21T10:28:38.373 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mgr 31777 85413 safe_time ceph txt unknown /proc/31777/task/85413/exe (readlink: Permission denied) 2024-03-21T10:28:38.373 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mgr 31777 85413 safe_time ceph NOFD /proc/31777/task/85413/fd (opendir: Permission denied) 2024-03-21T10:28:38.373 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mgr 31777 85414 safe_time ceph cwd unknown /proc/31777/task/85414/cwd (readlink: Permission denied) 2024-03-21T10:28:38.373 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mgr 31777 85414 safe_time ceph rtd unknown /proc/31777/task/85414/root (readlink: Permission denied) 2024-03-21T10:28:38.373 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mgr 31777 85414 safe_time ceph txt unknown /proc/31777/task/85414/exe (readlink: Permission denied) 2024-03-21T10:28:38.373 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mgr 31777 85414 safe_time ceph NOFD /proc/31777/task/85414/fd (opendir: Permission denied) 2024-03-21T10:28:38.373 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mgr 31777 85415 balancer ceph cwd unknown /proc/31777/task/85415/cwd (readlink: Permission denied) 2024-03-21T10:28:38.373 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mgr 31777 85415 balancer ceph rtd unknown /proc/31777/task/85415/root (readlink: Permission denied) 2024-03-21T10:28:38.373 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mgr 31777 85415 balancer ceph txt unknown /proc/31777/task/85415/exe (readlink: Permission denied) 2024-03-21T10:28:38.373 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mgr 31777 85415 balancer ceph NOFD /proc/31777/task/85415/fd (opendir: Permission denied) 2024-03-21T10:28:38.374 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mgr 31777 85416 m-fin-bal ceph cwd unknown /proc/31777/task/85416/cwd (readlink: Permission denied) 2024-03-21T10:28:38.374 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mgr 31777 85416 m-fin-bal ceph rtd unknown /proc/31777/task/85416/root (readlink: Permission denied) 2024-03-21T10:28:38.374 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mgr 31777 85416 m-fin-bal ceph txt unknown /proc/31777/task/85416/exe (readlink: Permission denied) 2024-03-21T10:28:38.374 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mgr 31777 85416 m-fin-bal ceph NOFD /proc/31777/task/85416/fd (opendir: Permission denied) 2024-03-21T10:28:38.374 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mgr 31777 85417 mgr-fin ceph cwd unknown /proc/31777/task/85417/cwd (readlink: Permission denied) 2024-03-21T10:28:38.374 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mgr 31777 85417 mgr-fin ceph rtd unknown /proc/31777/task/85417/root (readlink: Permission denied) 2024-03-21T10:28:38.374 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mgr 31777 85417 mgr-fin ceph txt unknown /proc/31777/task/85417/exe (readlink: Permission denied) 2024-03-21T10:28:38.374 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mgr 31777 85417 mgr-fin ceph NOFD /proc/31777/task/85417/fd (opendir: Permission denied) 2024-03-21T10:28:38.374 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mgr 31777 85418 mgr-fin ceph cwd unknown /proc/31777/task/85418/cwd (readlink: Permission denied) 2024-03-21T10:28:38.374 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mgr 31777 85418 mgr-fin ceph rtd unknown /proc/31777/task/85418/root (readlink: Permission denied) 2024-03-21T10:28:38.374 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mgr 31777 85418 mgr-fin ceph txt unknown /proc/31777/task/85418/exe (readlink: Permission denied) 2024-03-21T10:28:38.374 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mgr 31777 85418 mgr-fin ceph NOFD /proc/31777/task/85418/fd (opendir: Permission denied) 2024-03-21T10:28:38.374 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mgr 31777 85419 mgr-fin ceph cwd unknown /proc/31777/task/85419/cwd (readlink: Permission denied) 2024-03-21T10:28:38.375 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mgr 31777 85419 mgr-fin ceph rtd unknown /proc/31777/task/85419/root (readlink: Permission denied) 2024-03-21T10:28:38.375 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mgr 31777 85419 mgr-fin ceph txt unknown /proc/31777/task/85419/exe (readlink: Permission denied) 2024-03-21T10:28:38.375 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mgr 31777 85419 mgr-fin ceph NOFD /proc/31777/task/85419/fd (opendir: Permission denied) 2024-03-21T10:28:38.375 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mgr 31777 85420 mgr-fin ceph cwd unknown /proc/31777/task/85420/cwd (readlink: Permission denied) 2024-03-21T10:28:38.375 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mgr 31777 85420 mgr-fin ceph rtd unknown /proc/31777/task/85420/root (readlink: Permission denied) 2024-03-21T10:28:38.375 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mgr 31777 85420 mgr-fin ceph txt unknown /proc/31777/task/85420/exe (readlink: Permission denied) 2024-03-21T10:28:38.375 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mgr 31777 85420 mgr-fin ceph NOFD /proc/31777/task/85420/fd (opendir: Permission denied) 2024-03-21T10:28:38.375 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mgr 31777 85421 mgr-fin ceph cwd unknown /proc/31777/task/85421/cwd (readlink: Permission denied) 2024-03-21T10:28:38.375 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mgr 31777 85421 mgr-fin ceph rtd unknown /proc/31777/task/85421/root (readlink: Permission denied) 2024-03-21T10:28:38.375 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mgr 31777 85421 mgr-fin ceph txt unknown /proc/31777/task/85421/exe (readlink: Permission denied) 2024-03-21T10:28:38.375 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mgr 31777 85421 mgr-fin ceph NOFD /proc/31777/task/85421/fd (opendir: Permission denied) 2024-03-21T10:28:38.375 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mgr 31777 85422 mgr-fin ceph cwd unknown /proc/31777/task/85422/cwd (readlink: Permission denied) 2024-03-21T10:28:38.375 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mgr 31777 85422 mgr-fin ceph rtd unknown /proc/31777/task/85422/root (readlink: Permission denied) 2024-03-21T10:28:38.376 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mgr 31777 85422 mgr-fin ceph txt unknown /proc/31777/task/85422/exe (readlink: Permission denied) 2024-03-21T10:28:38.376 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mgr 31777 85422 mgr-fin ceph NOFD /proc/31777/task/85422/fd (opendir: Permission denied) 2024-03-21T10:28:38.376 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mgr 31777 85423 mgr-fin ceph cwd unknown /proc/31777/task/85423/cwd (readlink: Permission denied) 2024-03-21T10:28:38.376 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mgr 31777 85423 mgr-fin ceph rtd unknown /proc/31777/task/85423/root (readlink: Permission denied) 2024-03-21T10:28:38.376 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mgr 31777 85423 mgr-fin ceph txt unknown /proc/31777/task/85423/exe (readlink: Permission denied) 2024-03-21T10:28:38.376 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mgr 31777 85423 mgr-fin ceph NOFD /proc/31777/task/85423/fd (opendir: Permission denied) 2024-03-21T10:28:38.376 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mgr 31777 85424 mgr-fin ceph cwd unknown /proc/31777/task/85424/cwd (readlink: Permission denied) 2024-03-21T10:28:38.376 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mgr 31777 85424 mgr-fin ceph rtd unknown /proc/31777/task/85424/root (readlink: Permission denied) 2024-03-21T10:28:38.376 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mgr 31777 85424 mgr-fin ceph txt unknown /proc/31777/task/85424/exe (readlink: Permission denied) 2024-03-21T10:28:38.376 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mgr 31777 85424 mgr-fin ceph NOFD /proc/31777/task/85424/fd (opendir: Permission denied) 2024-03-21T10:28:38.376 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mgr 31777 85425 mgr-fin ceph cwd unknown /proc/31777/task/85425/cwd (readlink: Permission denied) 2024-03-21T10:28:38.376 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mgr 31777 85425 mgr-fin ceph rtd unknown /proc/31777/task/85425/root (readlink: Permission denied) 2024-03-21T10:28:38.376 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mgr 31777 85425 mgr-fin ceph txt unknown /proc/31777/task/85425/exe (readlink: Permission denied) 2024-03-21T10:28:38.376 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mgr 31777 85425 mgr-fin ceph NOFD /proc/31777/task/85425/fd (opendir: Permission denied) 2024-03-21T10:28:38.376 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mgr 31777 85426 mgr-fin ceph cwd unknown /proc/31777/task/85426/cwd (readlink: Permission denied) 2024-03-21T10:28:38.377 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mgr 31777 85426 mgr-fin ceph rtd unknown /proc/31777/task/85426/root (readlink: Permission denied) 2024-03-21T10:28:38.377 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mgr 31777 85426 mgr-fin ceph txt unknown /proc/31777/task/85426/exe (readlink: Permission denied) 2024-03-21T10:28:38.377 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mgr 31777 85426 mgr-fin ceph NOFD /proc/31777/task/85426/fd (opendir: Permission denied) 2024-03-21T10:28:38.377 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mgr 31777 85427 mgr-fin ceph cwd unknown /proc/31777/task/85427/cwd (readlink: Permission denied) 2024-03-21T10:28:38.377 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mgr 31777 85427 mgr-fin ceph rtd unknown /proc/31777/task/85427/root (readlink: Permission denied) 2024-03-21T10:28:38.377 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mgr 31777 85427 mgr-fin ceph txt unknown /proc/31777/task/85427/exe (readlink: Permission denied) 2024-03-21T10:28:38.377 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mgr 31777 85427 mgr-fin ceph NOFD /proc/31777/task/85427/fd (opendir: Permission denied) 2024-03-21T10:28:38.377 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mgr 31777 85428 mgr-fin ceph cwd unknown /proc/31777/task/85428/cwd (readlink: Permission denied) 2024-03-21T10:28:38.377 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mgr 31777 85428 mgr-fin ceph rtd unknown /proc/31777/task/85428/root (readlink: Permission denied) 2024-03-21T10:28:38.377 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mgr 31777 85428 mgr-fin ceph txt unknown /proc/31777/task/85428/exe (readlink: Permission denied) 2024-03-21T10:28:38.377 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mgr 31777 85428 mgr-fin ceph NOFD /proc/31777/task/85428/fd (opendir: Permission denied) 2024-03-21T10:28:38.377 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mgr 31777 85429 mgr-fin ceph cwd unknown /proc/31777/task/85429/cwd (readlink: Permission denied) 2024-03-21T10:28:38.378 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mgr 31777 85429 mgr-fin ceph rtd unknown /proc/31777/task/85429/root (readlink: Permission denied) 2024-03-21T10:28:38.378 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mgr 31777 85429 mgr-fin ceph txt unknown /proc/31777/task/85429/exe (readlink: Permission denied) 2024-03-21T10:28:38.378 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mgr 31777 85429 mgr-fin ceph NOFD /proc/31777/task/85429/fd (opendir: Permission denied) 2024-03-21T10:28:38.378 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mgr 31777 85430 mgr-fin ceph cwd unknown /proc/31777/task/85430/cwd (readlink: Permission denied) 2024-03-21T10:28:38.378 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mgr 31777 85430 mgr-fin ceph rtd unknown /proc/31777/task/85430/root (readlink: Permission denied) 2024-03-21T10:28:38.378 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mgr 31777 85430 mgr-fin ceph txt unknown /proc/31777/task/85430/exe (readlink: Permission denied) 2024-03-21T10:28:38.378 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mgr 31777 85430 mgr-fin ceph NOFD /proc/31777/task/85430/fd (opendir: Permission denied) 2024-03-21T10:28:38.378 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mgr 31777 85431 mgr-fin ceph cwd unknown /proc/31777/task/85431/cwd (readlink: Permission denied) 2024-03-21T10:28:38.378 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mgr 31777 85431 mgr-fin ceph rtd unknown /proc/31777/task/85431/root (readlink: Permission denied) 2024-03-21T10:28:38.378 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mgr 31777 85431 mgr-fin ceph txt unknown /proc/31777/task/85431/exe (readlink: Permission denied) 2024-03-21T10:28:38.378 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mgr 31777 85431 mgr-fin ceph NOFD /proc/31777/task/85431/fd (opendir: Permission denied) 2024-03-21T10:28:38.378 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mgr 31777 85432 mgr-fin ceph cwd unknown /proc/31777/task/85432/cwd (readlink: Permission denied) 2024-03-21T10:28:38.378 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mgr 31777 85432 mgr-fin ceph rtd unknown /proc/31777/task/85432/root (readlink: Permission denied) 2024-03-21T10:28:38.379 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mgr 31777 85432 mgr-fin ceph txt unknown /proc/31777/task/85432/exe (readlink: Permission denied) 2024-03-21T10:28:38.379 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mgr 31777 85432 mgr-fin ceph NOFD /proc/31777/task/85432/fd (opendir: Permission denied) 2024-03-21T10:28:38.379 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mgr 31777 85433 cephadm ceph cwd unknown /proc/31777/task/85433/cwd (readlink: Permission denied) 2024-03-21T10:28:38.379 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mgr 31777 85433 cephadm ceph rtd unknown /proc/31777/task/85433/root (readlink: Permission denied) 2024-03-21T10:28:38.379 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mgr 31777 85433 cephadm ceph txt unknown /proc/31777/task/85433/exe (readlink: Permission denied) 2024-03-21T10:28:38.379 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mgr 31777 85433 cephadm ceph NOFD /proc/31777/task/85433/fd (opendir: Permission denied) 2024-03-21T10:28:38.379 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mgr 31777 85434 mgr-fin ceph cwd unknown /proc/31777/task/85434/cwd (readlink: Permission denied) 2024-03-21T10:28:38.379 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mgr 31777 85434 mgr-fin ceph rtd unknown /proc/31777/task/85434/root (readlink: Permission denied) 2024-03-21T10:28:38.379 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mgr 31777 85434 mgr-fin ceph txt unknown /proc/31777/task/85434/exe (readlink: Permission denied) 2024-03-21T10:28:38.379 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mgr 31777 85434 mgr-fin ceph NOFD /proc/31777/task/85434/fd (opendir: Permission denied) 2024-03-21T10:28:38.380 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mgr 31777 85435 m-fin-cep ceph cwd unknown /proc/31777/task/85435/cwd (readlink: Permission denied) 2024-03-21T10:28:38.380 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mgr 31777 85435 m-fin-cep ceph rtd unknown /proc/31777/task/85435/root (readlink: Permission denied) 2024-03-21T10:28:38.380 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mgr 31777 85435 m-fin-cep ceph txt unknown /proc/31777/task/85435/exe (readlink: Permission denied) 2024-03-21T10:28:38.380 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mgr 31777 85435 m-fin-cep ceph NOFD /proc/31777/task/85435/fd (opendir: Permission denied) 2024-03-21T10:28:38.380 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mgr 31777 85436 crash ceph cwd unknown /proc/31777/task/85436/cwd (readlink: Permission denied) 2024-03-21T10:28:38.380 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mgr 31777 85436 crash ceph rtd unknown /proc/31777/task/85436/root (readlink: Permission denied) 2024-03-21T10:28:38.380 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mgr 31777 85436 crash ceph txt unknown /proc/31777/task/85436/exe (readlink: Permission denied) 2024-03-21T10:28:38.380 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mgr 31777 85436 crash ceph NOFD /proc/31777/task/85436/fd (opendir: Permission denied) 2024-03-21T10:28:38.380 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mgr 31777 85437 m-fin-cra ceph cwd unknown /proc/31777/task/85437/cwd (readlink: Permission denied) 2024-03-21T10:28:38.380 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mgr 31777 85437 m-fin-cra ceph rtd unknown /proc/31777/task/85437/root (readlink: Permission denied) 2024-03-21T10:28:38.380 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mgr 31777 85437 m-fin-cra ceph txt unknown /proc/31777/task/85437/exe (readlink: Permission denied) 2024-03-21T10:28:38.380 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mgr 31777 85437 m-fin-cra ceph NOFD /proc/31777/task/85437/fd (opendir: Permission denied) 2024-03-21T10:28:38.380 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mgr 31777 85438 cephadm ceph cwd unknown /proc/31777/task/85438/cwd (readlink: Permission denied) 2024-03-21T10:28:38.381 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mgr 31777 85438 cephadm ceph rtd unknown /proc/31777/task/85438/root (readlink: Permission denied) 2024-03-21T10:28:38.381 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mgr 31777 85438 cephadm ceph txt unknown /proc/31777/task/85438/exe (readlink: Permission denied) 2024-03-21T10:28:38.381 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mgr 31777 85438 cephadm ceph NOFD /proc/31777/task/85438/fd (opendir: Permission denied) 2024-03-21T10:28:38.381 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mgr 31777 85439 mgr-fin ceph cwd unknown /proc/31777/task/85439/cwd (readlink: Permission denied) 2024-03-21T10:28:38.381 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mgr 31777 85439 mgr-fin ceph rtd unknown /proc/31777/task/85439/root (readlink: Permission denied) 2024-03-21T10:28:38.381 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mgr 31777 85439 mgr-fin ceph txt unknown /proc/31777/task/85439/exe (readlink: Permission denied) 2024-03-21T10:28:38.381 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mgr 31777 85439 mgr-fin ceph NOFD /proc/31777/task/85439/fd (opendir: Permission denied) 2024-03-21T10:28:38.381 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mgr 31777 85440 dashboard ceph cwd unknown /proc/31777/task/85440/cwd (readlink: Permission denied) 2024-03-21T10:28:38.381 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mgr 31777 85440 dashboard ceph rtd unknown /proc/31777/task/85440/root (readlink: Permission denied) 2024-03-21T10:28:38.381 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mgr 31777 85440 dashboard ceph txt unknown /proc/31777/task/85440/exe (readlink: Permission denied) 2024-03-21T10:28:38.381 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mgr 31777 85440 dashboard ceph NOFD /proc/31777/task/85440/fd (opendir: Permission denied) 2024-03-21T10:28:38.381 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mgr 31777 85441 m-fin-das ceph cwd unknown /proc/31777/task/85441/cwd (readlink: Permission denied) 2024-03-21T10:28:38.381 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mgr 31777 85441 m-fin-das ceph rtd unknown /proc/31777/task/85441/root (readlink: Permission denied) 2024-03-21T10:28:38.381 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mgr 31777 85441 m-fin-das ceph txt unknown /proc/31777/task/85441/exe (readlink: Permission denied) 2024-03-21T10:28:38.382 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mgr 31777 85441 m-fin-das ceph NOFD /proc/31777/task/85441/fd (opendir: Permission denied) 2024-03-21T10:28:38.382 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mgr 31777 85442 mgr-fin ceph cwd unknown /proc/31777/task/85442/cwd (readlink: Permission denied) 2024-03-21T10:28:38.382 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mgr 31777 85442 mgr-fin ceph rtd unknown /proc/31777/task/85442/root (readlink: Permission denied) 2024-03-21T10:28:38.382 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mgr 31777 85442 mgr-fin ceph txt unknown /proc/31777/task/85442/exe (readlink: Permission denied) 2024-03-21T10:28:38.382 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mgr 31777 85442 mgr-fin ceph NOFD /proc/31777/task/85442/fd (opendir: Permission denied) 2024-03-21T10:28:38.382 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mgr 31777 85443 devicehea ceph cwd unknown /proc/31777/task/85443/cwd (readlink: Permission denied) 2024-03-21T10:28:38.382 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mgr 31777 85443 devicehea ceph rtd unknown /proc/31777/task/85443/root (readlink: Permission denied) 2024-03-21T10:28:38.382 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mgr 31777 85443 devicehea ceph txt unknown /proc/31777/task/85443/exe (readlink: Permission denied) 2024-03-21T10:28:38.382 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mgr 31777 85443 devicehea ceph NOFD /proc/31777/task/85443/fd (opendir: Permission denied) 2024-03-21T10:28:38.382 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mgr 31777 85444 m-fin-dev ceph cwd unknown /proc/31777/task/85444/cwd (readlink: Permission denied) 2024-03-21T10:28:38.382 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mgr 31777 85444 m-fin-dev ceph rtd unknown /proc/31777/task/85444/root (readlink: Permission denied) 2024-03-21T10:28:38.382 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mgr 31777 85444 m-fin-dev ceph txt unknown /proc/31777/task/85444/exe (readlink: Permission denied) 2024-03-21T10:28:38.383 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mgr 31777 85444 m-fin-dev ceph NOFD /proc/31777/task/85444/fd (opendir: Permission denied) 2024-03-21T10:28:38.383 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mgr 31777 85445 mgr-fin ceph cwd unknown /proc/31777/task/85445/cwd (readlink: Permission denied) 2024-03-21T10:28:38.383 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mgr 31777 85445 mgr-fin ceph rtd unknown /proc/31777/task/85445/root (readlink: Permission denied) 2024-03-21T10:28:38.383 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mgr 31777 85445 mgr-fin ceph txt unknown /proc/31777/task/85445/exe (readlink: Permission denied) 2024-03-21T10:28:38.383 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mgr 31777 85445 mgr-fin ceph NOFD /proc/31777/task/85445/fd (opendir: Permission denied) 2024-03-21T10:28:38.383 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mgr 31777 85447 m-fin-ios ceph cwd unknown /proc/31777/task/85447/cwd (readlink: Permission denied) 2024-03-21T10:28:38.383 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mgr 31777 85447 m-fin-ios ceph rtd unknown /proc/31777/task/85447/root (readlink: Permission denied) 2024-03-21T10:28:38.383 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mgr 31777 85447 m-fin-ios ceph txt unknown /proc/31777/task/85447/exe (readlink: Permission denied) 2024-03-21T10:28:38.383 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mgr 31777 85447 m-fin-ios ceph NOFD /proc/31777/task/85447/fd (opendir: Permission denied) 2024-03-21T10:28:38.383 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mgr 31777 85448 mgr-fin ceph cwd unknown /proc/31777/task/85448/cwd (readlink: Permission denied) 2024-03-21T10:28:38.383 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mgr 31777 85448 mgr-fin ceph rtd unknown /proc/31777/task/85448/root (readlink: Permission denied) 2024-03-21T10:28:38.383 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mgr 31777 85448 mgr-fin ceph txt unknown /proc/31777/task/85448/exe (readlink: Permission denied) 2024-03-21T10:28:38.383 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mgr 31777 85448 mgr-fin ceph NOFD /proc/31777/task/85448/fd (opendir: Permission denied) 2024-03-21T10:28:38.383 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mgr 31777 85449 mgr-fin ceph cwd unknown /proc/31777/task/85449/cwd (readlink: Permission denied) 2024-03-21T10:28:38.384 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mgr 31777 85449 mgr-fin ceph rtd unknown /proc/31777/task/85449/root (readlink: Permission denied) 2024-03-21T10:28:38.384 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mgr 31777 85449 mgr-fin ceph txt unknown /proc/31777/task/85449/exe (readlink: Permission denied) 2024-03-21T10:28:38.384 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mgr 31777 85449 mgr-fin ceph NOFD /proc/31777/task/85449/fd (opendir: Permission denied) 2024-03-21T10:28:38.384 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mgr 31777 85451 m-fin-nfs ceph cwd unknown /proc/31777/task/85451/cwd (readlink: Permission denied) 2024-03-21T10:28:38.384 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mgr 31777 85451 m-fin-nfs ceph rtd unknown /proc/31777/task/85451/root (readlink: Permission denied) 2024-03-21T10:28:38.384 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mgr 31777 85451 m-fin-nfs ceph txt unknown /proc/31777/task/85451/exe (readlink: Permission denied) 2024-03-21T10:28:38.384 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mgr 31777 85451 m-fin-nfs ceph NOFD /proc/31777/task/85451/fd (opendir: Permission denied) 2024-03-21T10:28:38.384 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mgr 31777 85452 mgr-fin ceph cwd unknown /proc/31777/task/85452/cwd (readlink: Permission denied) 2024-03-21T10:28:38.384 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mgr 31777 85452 mgr-fin ceph rtd unknown /proc/31777/task/85452/root (readlink: Permission denied) 2024-03-21T10:28:38.384 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mgr 31777 85452 mgr-fin ceph txt unknown /proc/31777/task/85452/exe (readlink: Permission denied) 2024-03-21T10:28:38.384 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mgr 31777 85452 mgr-fin ceph NOFD /proc/31777/task/85452/fd (opendir: Permission denied) 2024-03-21T10:28:38.384 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mgr 31777 85454 m-fin-orc ceph cwd unknown /proc/31777/task/85454/cwd (readlink: Permission denied) 2024-03-21T10:28:38.384 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mgr 31777 85454 m-fin-orc ceph rtd unknown /proc/31777/task/85454/root (readlink: Permission denied) 2024-03-21T10:28:38.385 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mgr 31777 85454 m-fin-orc ceph txt unknown /proc/31777/task/85454/exe (readlink: Permission denied) 2024-03-21T10:28:38.385 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mgr 31777 85454 m-fin-orc ceph NOFD /proc/31777/task/85454/fd (opendir: Permission denied) 2024-03-21T10:28:38.385 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mgr 31777 85455 mgr-fin ceph cwd unknown /proc/31777/task/85455/cwd (readlink: Permission denied) 2024-03-21T10:28:38.385 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mgr 31777 85455 mgr-fin ceph rtd unknown /proc/31777/task/85455/root (readlink: Permission denied) 2024-03-21T10:28:38.385 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mgr 31777 85455 mgr-fin ceph txt unknown /proc/31777/task/85455/exe (readlink: Permission denied) 2024-03-21T10:28:38.385 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mgr 31777 85455 mgr-fin ceph NOFD /proc/31777/task/85455/fd (opendir: Permission denied) 2024-03-21T10:28:38.385 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mgr 31777 85456 pg_autosc ceph cwd unknown /proc/31777/task/85456/cwd (readlink: Permission denied) 2024-03-21T10:28:38.385 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mgr 31777 85456 pg_autosc ceph rtd unknown /proc/31777/task/85456/root (readlink: Permission denied) 2024-03-21T10:28:38.385 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mgr 31777 85456 pg_autosc ceph txt unknown /proc/31777/task/85456/exe (readlink: Permission denied) 2024-03-21T10:28:38.385 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mgr 31777 85456 pg_autosc ceph NOFD /proc/31777/task/85456/fd (opendir: Permission denied) 2024-03-21T10:28:38.385 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mgr 31777 85457 m-fin-pg_ ceph cwd unknown /proc/31777/task/85457/cwd (readlink: Permission denied) 2024-03-21T10:28:38.385 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mgr 31777 85457 m-fin-pg_ ceph rtd unknown /proc/31777/task/85457/root (readlink: Permission denied) 2024-03-21T10:28:38.386 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mgr 31777 85457 m-fin-pg_ ceph txt unknown /proc/31777/task/85457/exe (readlink: Permission denied) 2024-03-21T10:28:38.386 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mgr 31777 85457 m-fin-pg_ ceph NOFD /proc/31777/task/85457/fd (opendir: Permission denied) 2024-03-21T10:28:38.386 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mgr 31777 85458 mgr-fin ceph cwd unknown /proc/31777/task/85458/cwd (readlink: Permission denied) 2024-03-21T10:28:38.386 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mgr 31777 85458 mgr-fin ceph rtd unknown /proc/31777/task/85458/root (readlink: Permission denied) 2024-03-21T10:28:38.386 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mgr 31777 85458 mgr-fin ceph txt unknown /proc/31777/task/85458/exe (readlink: Permission denied) 2024-03-21T10:28:38.386 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mgr 31777 85458 mgr-fin ceph NOFD /proc/31777/task/85458/fd (opendir: Permission denied) 2024-03-21T10:28:38.386 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mgr 31777 85459 mgr-fin ceph cwd unknown /proc/31777/task/85459/cwd (readlink: Permission denied) 2024-03-21T10:28:38.386 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mgr 31777 85459 mgr-fin ceph rtd unknown /proc/31777/task/85459/root (readlink: Permission denied) 2024-03-21T10:28:38.386 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mgr 31777 85459 mgr-fin ceph txt unknown /proc/31777/task/85459/exe (readlink: Permission denied) 2024-03-21T10:28:38.386 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mgr 31777 85459 mgr-fin ceph NOFD /proc/31777/task/85459/fd (opendir: Permission denied) 2024-03-21T10:28:38.386 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mgr 31777 85460 mgr-fin ceph cwd unknown /proc/31777/task/85460/cwd (readlink: Permission denied) 2024-03-21T10:28:38.386 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mgr 31777 85460 mgr-fin ceph rtd unknown /proc/31777/task/85460/root (readlink: Permission denied) 2024-03-21T10:28:38.386 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mgr 31777 85460 mgr-fin ceph txt unknown /proc/31777/task/85460/exe (readlink: Permission denied) 2024-03-21T10:28:38.387 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mgr 31777 85460 mgr-fin ceph NOFD /proc/31777/task/85460/fd (opendir: Permission denied) 2024-03-21T10:28:38.387 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mgr 31777 85461 progress ceph cwd unknown /proc/31777/task/85461/cwd (readlink: Permission denied) 2024-03-21T10:28:38.387 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mgr 31777 85461 progress ceph rtd unknown /proc/31777/task/85461/root (readlink: Permission denied) 2024-03-21T10:28:38.387 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mgr 31777 85461 progress ceph txt unknown /proc/31777/task/85461/exe (readlink: Permission denied) 2024-03-21T10:28:38.387 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mgr 31777 85461 progress ceph NOFD /proc/31777/task/85461/fd (opendir: Permission denied) 2024-03-21T10:28:38.387 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mgr 31777 85462 m-fin-pro ceph cwd unknown /proc/31777/task/85462/cwd (readlink: Permission denied) 2024-03-21T10:28:38.387 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mgr 31777 85462 m-fin-pro ceph rtd unknown /proc/31777/task/85462/root (readlink: Permission denied) 2024-03-21T10:28:38.387 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mgr 31777 85462 m-fin-pro ceph txt unknown /proc/31777/task/85462/exe (readlink: Permission denied) 2024-03-21T10:28:38.387 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mgr 31777 85462 m-fin-pro ceph NOFD /proc/31777/task/85462/fd (opendir: Permission denied) 2024-03-21T10:28:38.387 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mgr 31777 85463 mgr-fin ceph cwd unknown /proc/31777/task/85463/cwd (readlink: Permission denied) 2024-03-21T10:28:38.387 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mgr 31777 85463 mgr-fin ceph rtd unknown /proc/31777/task/85463/root (readlink: Permission denied) 2024-03-21T10:28:38.387 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mgr 31777 85463 mgr-fin ceph txt unknown /proc/31777/task/85463/exe (readlink: Permission denied) 2024-03-21T10:28:38.387 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mgr 31777 85463 mgr-fin ceph NOFD /proc/31777/task/85463/fd (opendir: Permission denied) 2024-03-21T10:28:38.387 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mgr 31777 85467 io_contex ceph cwd unknown /proc/31777/task/85467/cwd (readlink: Permission denied) 2024-03-21T10:28:38.388 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mgr 31777 85467 io_contex ceph rtd unknown /proc/31777/task/85467/root (readlink: Permission denied) 2024-03-21T10:28:38.388 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mgr 31777 85467 io_contex ceph txt unknown /proc/31777/task/85467/exe (readlink: Permission denied) 2024-03-21T10:28:38.388 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mgr 31777 85467 io_contex ceph NOFD /proc/31777/task/85467/fd (opendir: Permission denied) 2024-03-21T10:28:38.388 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mgr 31777 85468 io_contex ceph cwd unknown /proc/31777/task/85468/cwd (readlink: Permission denied) 2024-03-21T10:28:38.388 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mgr 31777 85468 io_contex ceph rtd unknown /proc/31777/task/85468/root (readlink: Permission denied) 2024-03-21T10:28:38.388 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mgr 31777 85468 io_contex ceph txt unknown /proc/31777/task/85468/exe (readlink: Permission denied) 2024-03-21T10:28:38.388 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mgr 31777 85468 io_contex ceph NOFD /proc/31777/task/85468/fd (opendir: Permission denied) 2024-03-21T10:28:38.388 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mgr 31777 85469 ceph_time ceph cwd unknown /proc/31777/task/85469/cwd (readlink: Permission denied) 2024-03-21T10:28:38.388 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mgr 31777 85469 ceph_time ceph rtd unknown /proc/31777/task/85469/root (readlink: Permission denied) 2024-03-21T10:28:38.388 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mgr 31777 85469 ceph_time ceph txt unknown /proc/31777/task/85469/exe (readlink: Permission denied) 2024-03-21T10:28:38.388 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mgr 31777 85469 ceph_time ceph NOFD /proc/31777/task/85469/fd (opendir: Permission denied) 2024-03-21T10:28:38.388 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mgr 31777 85470 ms_dispat ceph cwd unknown /proc/31777/task/85470/cwd (readlink: Permission denied) 2024-03-21T10:28:38.388 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mgr 31777 85470 ms_dispat ceph rtd unknown /proc/31777/task/85470/root (readlink: Permission denied) 2024-03-21T10:28:38.389 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mgr 31777 85470 ms_dispat ceph txt unknown /proc/31777/task/85470/exe (readlink: Permission denied) 2024-03-21T10:28:38.389 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mgr 31777 85470 ms_dispat ceph NOFD /proc/31777/task/85470/fd (opendir: Permission denied) 2024-03-21T10:28:38.389 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mgr 31777 85471 ms_local ceph cwd unknown /proc/31777/task/85471/cwd (readlink: Permission denied) 2024-03-21T10:28:38.389 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mgr 31777 85471 ms_local ceph rtd unknown /proc/31777/task/85471/root (readlink: Permission denied) 2024-03-21T10:28:38.389 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mgr 31777 85471 ms_local ceph txt unknown /proc/31777/task/85471/exe (readlink: Permission denied) 2024-03-21T10:28:38.389 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mgr 31777 85471 ms_local ceph NOFD /proc/31777/task/85471/fd (opendir: Permission denied) 2024-03-21T10:28:38.389 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mgr 31777 85472 safe_time ceph cwd unknown /proc/31777/task/85472/cwd (readlink: Permission denied) 2024-03-21T10:28:38.389 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mgr 31777 85472 safe_time ceph rtd unknown /proc/31777/task/85472/root (readlink: Permission denied) 2024-03-21T10:28:38.389 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mgr 31777 85472 safe_time ceph txt unknown /proc/31777/task/85472/exe (readlink: Permission denied) 2024-03-21T10:28:38.389 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mgr 31777 85472 safe_time ceph NOFD /proc/31777/task/85472/fd (opendir: Permission denied) 2024-03-21T10:28:38.389 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mgr 31777 85473 safe_time ceph cwd unknown /proc/31777/task/85473/cwd (readlink: Permission denied) 2024-03-21T10:28:38.389 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mgr 31777 85473 safe_time ceph rtd unknown /proc/31777/task/85473/root (readlink: Permission denied) 2024-03-21T10:28:38.389 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mgr 31777 85473 safe_time ceph txt unknown /proc/31777/task/85473/exe (readlink: Permission denied) 2024-03-21T10:28:38.389 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mgr 31777 85473 safe_time ceph NOFD /proc/31777/task/85473/fd (opendir: Permission denied) 2024-03-21T10:28:38.390 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mgr 31777 85474 mgr-fin ceph cwd unknown /proc/31777/task/85474/cwd (readlink: Permission denied) 2024-03-21T10:28:38.390 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mgr 31777 85474 mgr-fin ceph rtd unknown /proc/31777/task/85474/root (readlink: Permission denied) 2024-03-21T10:28:38.390 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mgr 31777 85474 mgr-fin ceph txt unknown /proc/31777/task/85474/exe (readlink: Permission denied) 2024-03-21T10:28:38.390 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mgr 31777 85474 mgr-fin ceph NOFD /proc/31777/task/85474/fd (opendir: Permission denied) 2024-03-21T10:28:38.390 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mgr 31777 85476 m-fin-rbd ceph cwd unknown /proc/31777/task/85476/cwd (readlink: Permission denied) 2024-03-21T10:28:38.390 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mgr 31777 85476 m-fin-rbd ceph rtd unknown /proc/31777/task/85476/root (readlink: Permission denied) 2024-03-21T10:28:38.390 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mgr 31777 85476 m-fin-rbd ceph txt unknown /proc/31777/task/85476/exe (readlink: Permission denied) 2024-03-21T10:28:38.390 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mgr 31777 85476 m-fin-rbd ceph NOFD /proc/31777/task/85476/fd (opendir: Permission denied) 2024-03-21T10:28:38.390 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mgr 31777 85477 restful ceph cwd unknown /proc/31777/task/85477/cwd (readlink: Permission denied) 2024-03-21T10:28:38.390 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mgr 31777 85477 restful ceph rtd unknown /proc/31777/task/85477/root (readlink: Permission denied) 2024-03-21T10:28:38.390 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mgr 31777 85477 restful ceph txt unknown /proc/31777/task/85477/exe (readlink: Permission denied) 2024-03-21T10:28:38.390 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mgr 31777 85477 restful ceph NOFD /proc/31777/task/85477/fd (opendir: Permission denied) 2024-03-21T10:28:38.390 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mgr 31777 85478 m-fin-res ceph cwd unknown /proc/31777/task/85478/cwd (readlink: Permission denied) 2024-03-21T10:28:38.390 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mgr 31777 85478 m-fin-res ceph rtd unknown /proc/31777/task/85478/root (readlink: Permission denied) 2024-03-21T10:28:38.391 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mgr 31777 85478 m-fin-res ceph txt unknown /proc/31777/task/85478/exe (readlink: Permission denied) 2024-03-21T10:28:38.391 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mgr 31777 85478 m-fin-res ceph NOFD /proc/31777/task/85478/fd (opendir: Permission denied) 2024-03-21T10:28:38.391 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mgr 31777 85479 mgr-fin ceph cwd unknown /proc/31777/task/85479/cwd (readlink: Permission denied) 2024-03-21T10:28:38.391 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mgr 31777 85479 mgr-fin ceph rtd unknown /proc/31777/task/85479/root (readlink: Permission denied) 2024-03-21T10:28:38.391 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mgr 31777 85479 mgr-fin ceph txt unknown /proc/31777/task/85479/exe (readlink: Permission denied) 2024-03-21T10:28:38.391 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mgr 31777 85479 mgr-fin ceph NOFD /proc/31777/task/85479/fd (opendir: Permission denied) 2024-03-21T10:28:38.391 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mgr 31777 85480 mgr-fin ceph cwd unknown /proc/31777/task/85480/cwd (readlink: Permission denied) 2024-03-21T10:28:38.391 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mgr 31777 85480 mgr-fin ceph rtd unknown /proc/31777/task/85480/root (readlink: Permission denied) 2024-03-21T10:28:38.391 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mgr 31777 85480 mgr-fin ceph txt unknown /proc/31777/task/85480/exe (readlink: Permission denied) 2024-03-21T10:28:38.391 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mgr 31777 85480 mgr-fin ceph NOFD /proc/31777/task/85480/fd (opendir: Permission denied) 2024-03-21T10:28:38.391 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mgr 31777 85481 mgr-fin ceph cwd unknown /proc/31777/task/85481/cwd (readlink: Permission denied) 2024-03-21T10:28:38.391 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mgr 31777 85481 mgr-fin ceph rtd unknown /proc/31777/task/85481/root (readlink: Permission denied) 2024-03-21T10:28:38.391 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mgr 31777 85481 mgr-fin ceph txt unknown /proc/31777/task/85481/exe (readlink: Permission denied) 2024-03-21T10:28:38.392 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mgr 31777 85481 mgr-fin ceph NOFD /proc/31777/task/85481/fd (opendir: Permission denied) 2024-03-21T10:28:38.392 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mgr 31777 85482 mgr-fin ceph cwd unknown /proc/31777/task/85482/cwd (readlink: Permission denied) 2024-03-21T10:28:38.392 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mgr 31777 85482 mgr-fin ceph rtd unknown /proc/31777/task/85482/root (readlink: Permission denied) 2024-03-21T10:28:38.392 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mgr 31777 85482 mgr-fin ceph txt unknown /proc/31777/task/85482/exe (readlink: Permission denied) 2024-03-21T10:28:38.392 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mgr 31777 85482 mgr-fin ceph NOFD /proc/31777/task/85482/fd (opendir: Permission denied) 2024-03-21T10:28:38.392 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mgr 31777 85483 mgr-fin ceph cwd unknown /proc/31777/task/85483/cwd (readlink: Permission denied) 2024-03-21T10:28:38.392 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mgr 31777 85483 mgr-fin ceph rtd unknown /proc/31777/task/85483/root (readlink: Permission denied) 2024-03-21T10:28:38.392 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mgr 31777 85483 mgr-fin ceph txt unknown /proc/31777/task/85483/exe (readlink: Permission denied) 2024-03-21T10:28:38.392 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mgr 31777 85483 mgr-fin ceph NOFD /proc/31777/task/85483/fd (opendir: Permission denied) 2024-03-21T10:28:38.392 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mgr 31777 85484 mgr-fin ceph cwd unknown /proc/31777/task/85484/cwd (readlink: Permission denied) 2024-03-21T10:28:38.392 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mgr 31777 85484 mgr-fin ceph rtd unknown /proc/31777/task/85484/root (readlink: Permission denied) 2024-03-21T10:28:38.392 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mgr 31777 85484 mgr-fin ceph txt unknown /proc/31777/task/85484/exe (readlink: Permission denied) 2024-03-21T10:28:38.392 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mgr 31777 85484 mgr-fin ceph NOFD /proc/31777/task/85484/fd (opendir: Permission denied) 2024-03-21T10:28:38.392 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mgr 31777 85488 io_contex ceph cwd unknown /proc/31777/task/85488/cwd (readlink: Permission denied) 2024-03-21T10:28:38.393 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mgr 31777 85488 io_contex ceph rtd unknown /proc/31777/task/85488/root (readlink: Permission denied) 2024-03-21T10:28:38.393 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mgr 31777 85488 io_contex ceph txt unknown /proc/31777/task/85488/exe (readlink: Permission denied) 2024-03-21T10:28:38.393 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mgr 31777 85488 io_contex ceph NOFD /proc/31777/task/85488/fd (opendir: Permission denied) 2024-03-21T10:28:38.393 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mgr 31777 85489 io_contex ceph cwd unknown /proc/31777/task/85489/cwd (readlink: Permission denied) 2024-03-21T10:28:38.393 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mgr 31777 85489 io_contex ceph rtd unknown /proc/31777/task/85489/root (readlink: Permission denied) 2024-03-21T10:28:38.393 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mgr 31777 85489 io_contex ceph txt unknown /proc/31777/task/85489/exe (readlink: Permission denied) 2024-03-21T10:28:38.393 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mgr 31777 85489 io_contex ceph NOFD /proc/31777/task/85489/fd (opendir: Permission denied) 2024-03-21T10:28:38.393 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mgr 31777 85490 ceph_time ceph cwd unknown /proc/31777/task/85490/cwd (readlink: Permission denied) 2024-03-21T10:28:38.393 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mgr 31777 85490 ceph_time ceph rtd unknown /proc/31777/task/85490/root (readlink: Permission denied) 2024-03-21T10:28:38.393 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mgr 31777 85490 ceph_time ceph txt unknown /proc/31777/task/85490/exe (readlink: Permission denied) 2024-03-21T10:28:38.394 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mgr 31777 85490 ceph_time ceph NOFD /proc/31777/task/85490/fd (opendir: Permission denied) 2024-03-21T10:28:38.394 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mgr 31777 85491 ms_dispat ceph cwd unknown /proc/31777/task/85491/cwd (readlink: Permission denied) 2024-03-21T10:28:38.394 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mgr 31777 85491 ms_dispat ceph rtd unknown /proc/31777/task/85491/root (readlink: Permission denied) 2024-03-21T10:28:38.394 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mgr 31777 85491 ms_dispat ceph txt unknown /proc/31777/task/85491/exe (readlink: Permission denied) 2024-03-21T10:28:38.394 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mgr 31777 85491 ms_dispat ceph NOFD /proc/31777/task/85491/fd (opendir: Permission denied) 2024-03-21T10:28:38.394 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mgr 31777 85492 ms_local ceph cwd unknown /proc/31777/task/85492/cwd (readlink: Permission denied) 2024-03-21T10:28:38.394 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mgr 31777 85492 ms_local ceph rtd unknown /proc/31777/task/85492/root (readlink: Permission denied) 2024-03-21T10:28:38.394 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mgr 31777 85492 ms_local ceph txt unknown /proc/31777/task/85492/exe (readlink: Permission denied) 2024-03-21T10:28:38.394 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mgr 31777 85492 ms_local ceph NOFD /proc/31777/task/85492/fd (opendir: Permission denied) 2024-03-21T10:28:38.394 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mgr 31777 85493 safe_time ceph cwd unknown /proc/31777/task/85493/cwd (readlink: Permission denied) 2024-03-21T10:28:38.394 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mgr 31777 85493 safe_time ceph rtd unknown /proc/31777/task/85493/root (readlink: Permission denied) 2024-03-21T10:28:38.394 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mgr 31777 85493 safe_time ceph txt unknown /proc/31777/task/85493/exe (readlink: Permission denied) 2024-03-21T10:28:38.394 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mgr 31777 85493 safe_time ceph NOFD /proc/31777/task/85493/fd (opendir: Permission denied) 2024-03-21T10:28:38.395 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mgr 31777 85494 safe_time ceph cwd unknown /proc/31777/task/85494/cwd (readlink: Permission denied) 2024-03-21T10:28:38.395 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mgr 31777 85494 safe_time ceph rtd unknown /proc/31777/task/85494/root (readlink: Permission denied) 2024-03-21T10:28:38.395 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mgr 31777 85494 safe_time ceph txt unknown /proc/31777/task/85494/exe (readlink: Permission denied) 2024-03-21T10:28:38.395 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mgr 31777 85494 safe_time ceph NOFD /proc/31777/task/85494/fd (opendir: Permission denied) 2024-03-21T10:28:38.395 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mgr 31777 85498 m-fin-sna ceph cwd unknown /proc/31777/task/85498/cwd (readlink: Permission denied) 2024-03-21T10:28:38.395 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mgr 31777 85498 m-fin-sna ceph rtd unknown /proc/31777/task/85498/root (readlink: Permission denied) 2024-03-21T10:28:38.395 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mgr 31777 85498 m-fin-sna ceph txt unknown /proc/31777/task/85498/exe (readlink: Permission denied) 2024-03-21T10:28:38.395 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mgr 31777 85498 m-fin-sna ceph NOFD /proc/31777/task/85498/fd (opendir: Permission denied) 2024-03-21T10:28:38.395 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mgr 31777 85500 m-fin-sta ceph cwd unknown /proc/31777/task/85500/cwd (readlink: Permission denied) 2024-03-21T10:28:38.395 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mgr 31777 85500 m-fin-sta ceph rtd unknown /proc/31777/task/85500/root (readlink: Permission denied) 2024-03-21T10:28:38.395 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mgr 31777 85500 m-fin-sta ceph txt unknown /proc/31777/task/85500/exe (readlink: Permission denied) 2024-03-21T10:28:38.395 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mgr 31777 85500 m-fin-sta ceph NOFD /proc/31777/task/85500/fd (opendir: Permission denied) 2024-03-21T10:28:38.395 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mgr 31777 85501 telemetry ceph cwd unknown /proc/31777/task/85501/cwd (readlink: Permission denied) 2024-03-21T10:28:38.395 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mgr 31777 85501 telemetry ceph rtd unknown /proc/31777/task/85501/root (readlink: Permission denied) 2024-03-21T10:28:38.395 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mgr 31777 85501 telemetry ceph txt unknown /proc/31777/task/85501/exe (readlink: Permission denied) 2024-03-21T10:28:38.396 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mgr 31777 85501 telemetry ceph NOFD /proc/31777/task/85501/fd (opendir: Permission denied) 2024-03-21T10:28:38.396 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mgr 31777 85502 m-fin-tel ceph cwd unknown /proc/31777/task/85502/cwd (readlink: Permission denied) 2024-03-21T10:28:38.396 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mgr 31777 85502 m-fin-tel ceph rtd unknown /proc/31777/task/85502/root (readlink: Permission denied) 2024-03-21T10:28:38.396 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mgr 31777 85502 m-fin-tel ceph txt unknown /proc/31777/task/85502/exe (readlink: Permission denied) 2024-03-21T10:28:38.396 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mgr 31777 85502 m-fin-tel ceph NOFD /proc/31777/task/85502/fd (opendir: Permission denied) 2024-03-21T10:28:38.396 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mgr 31777 85503 mgr-fin ceph cwd unknown /proc/31777/task/85503/cwd (readlink: Permission denied) 2024-03-21T10:28:38.396 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mgr 31777 85503 mgr-fin ceph rtd unknown /proc/31777/task/85503/root (readlink: Permission denied) 2024-03-21T10:28:38.396 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mgr 31777 85503 mgr-fin ceph txt unknown /proc/31777/task/85503/exe (readlink: Permission denied) 2024-03-21T10:28:38.396 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mgr 31777 85503 mgr-fin ceph NOFD /proc/31777/task/85503/fd (opendir: Permission denied) 2024-03-21T10:28:38.396 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mgr 31777 85508 io_contex ceph cwd unknown /proc/31777/task/85508/cwd (readlink: Permission denied) 2024-03-21T10:28:38.396 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mgr 31777 85508 io_contex ceph rtd unknown /proc/31777/task/85508/root (readlink: Permission denied) 2024-03-21T10:28:38.396 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mgr 31777 85508 io_contex ceph txt unknown /proc/31777/task/85508/exe (readlink: Permission denied) 2024-03-21T10:28:38.397 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mgr 31777 85508 io_contex ceph NOFD /proc/31777/task/85508/fd (opendir: Permission denied) 2024-03-21T10:28:38.397 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mgr 31777 85509 io_contex ceph cwd unknown /proc/31777/task/85509/cwd (readlink: Permission denied) 2024-03-21T10:28:38.397 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mgr 31777 85509 io_contex ceph rtd unknown /proc/31777/task/85509/root (readlink: Permission denied) 2024-03-21T10:28:38.397 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mgr 31777 85509 io_contex ceph txt unknown /proc/31777/task/85509/exe (readlink: Permission denied) 2024-03-21T10:28:38.397 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mgr 31777 85509 io_contex ceph NOFD /proc/31777/task/85509/fd (opendir: Permission denied) 2024-03-21T10:28:38.397 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mgr 31777 85510 ceph_time ceph cwd unknown /proc/31777/task/85510/cwd (readlink: Permission denied) 2024-03-21T10:28:38.397 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mgr 31777 85510 ceph_time ceph rtd unknown /proc/31777/task/85510/root (readlink: Permission denied) 2024-03-21T10:28:38.397 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mgr 31777 85510 ceph_time ceph txt unknown /proc/31777/task/85510/exe (readlink: Permission denied) 2024-03-21T10:28:38.397 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mgr 31777 85510 ceph_time ceph NOFD /proc/31777/task/85510/fd (opendir: Permission denied) 2024-03-21T10:28:38.398 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mgr 31777 85511 ms_dispat ceph cwd unknown /proc/31777/task/85511/cwd (readlink: Permission denied) 2024-03-21T10:28:38.398 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mgr 31777 85511 ms_dispat ceph rtd unknown /proc/31777/task/85511/root (readlink: Permission denied) 2024-03-21T10:28:38.398 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mgr 31777 85511 ms_dispat ceph txt unknown /proc/31777/task/85511/exe (readlink: Permission denied) 2024-03-21T10:28:38.398 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mgr 31777 85511 ms_dispat ceph NOFD /proc/31777/task/85511/fd (opendir: Permission denied) 2024-03-21T10:28:38.398 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mgr 31777 85512 ms_local ceph cwd unknown /proc/31777/task/85512/cwd (readlink: Permission denied) 2024-03-21T10:28:38.398 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mgr 31777 85512 ms_local ceph rtd unknown /proc/31777/task/85512/root (readlink: Permission denied) 2024-03-21T10:28:38.398 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mgr 31777 85512 ms_local ceph txt unknown /proc/31777/task/85512/exe (readlink: Permission denied) 2024-03-21T10:28:38.398 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mgr 31777 85512 ms_local ceph NOFD /proc/31777/task/85512/fd (opendir: Permission denied) 2024-03-21T10:28:38.398 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mgr 31777 85513 safe_time ceph cwd unknown /proc/31777/task/85513/cwd (readlink: Permission denied) 2024-03-21T10:28:38.398 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mgr 31777 85513 safe_time ceph rtd unknown /proc/31777/task/85513/root (readlink: Permission denied) 2024-03-21T10:28:38.398 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mgr 31777 85513 safe_time ceph txt unknown /proc/31777/task/85513/exe (readlink: Permission denied) 2024-03-21T10:28:38.398 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mgr 31777 85513 safe_time ceph NOFD /proc/31777/task/85513/fd (opendir: Permission denied) 2024-03-21T10:28:38.398 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mgr 31777 85514 safe_time ceph cwd unknown /proc/31777/task/85514/cwd (readlink: Permission denied) 2024-03-21T10:28:38.399 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mgr 31777 85514 safe_time ceph rtd unknown /proc/31777/task/85514/root (readlink: Permission denied) 2024-03-21T10:28:38.399 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mgr 31777 85514 safe_time ceph txt unknown /proc/31777/task/85514/exe (readlink: Permission denied) 2024-03-21T10:28:38.399 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mgr 31777 85514 safe_time ceph NOFD /proc/31777/task/85514/fd (opendir: Permission denied) 2024-03-21T10:28:38.399 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mgr 31777 85515 mgr-fin ceph cwd unknown /proc/31777/task/85515/cwd (readlink: Permission denied) 2024-03-21T10:28:38.399 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mgr 31777 85515 mgr-fin ceph rtd unknown /proc/31777/task/85515/root (readlink: Permission denied) 2024-03-21T10:28:38.399 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mgr 31777 85515 mgr-fin ceph txt unknown /proc/31777/task/85515/exe (readlink: Permission denied) 2024-03-21T10:28:38.399 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mgr 31777 85515 mgr-fin ceph NOFD /proc/31777/task/85515/fd (opendir: Permission denied) 2024-03-21T10:28:38.399 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mgr 31777 85516 mgr-fin ceph cwd unknown /proc/31777/task/85516/cwd (readlink: Permission denied) 2024-03-21T10:28:38.399 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mgr 31777 85516 mgr-fin ceph rtd unknown /proc/31777/task/85516/root (readlink: Permission denied) 2024-03-21T10:28:38.399 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mgr 31777 85516 mgr-fin ceph txt unknown /proc/31777/task/85516/exe (readlink: Permission denied) 2024-03-21T10:28:38.399 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mgr 31777 85516 mgr-fin ceph NOFD /proc/31777/task/85516/fd (opendir: Permission denied) 2024-03-21T10:28:38.399 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mgr 31777 85517 mgr-fin ceph cwd unknown /proc/31777/task/85517/cwd (readlink: Permission denied) 2024-03-21T10:28:38.399 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mgr 31777 85517 mgr-fin ceph rtd unknown /proc/31777/task/85517/root (readlink: Permission denied) 2024-03-21T10:28:38.400 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mgr 31777 85517 mgr-fin ceph txt unknown /proc/31777/task/85517/exe (readlink: Permission denied) 2024-03-21T10:28:38.400 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mgr 31777 85517 mgr-fin ceph NOFD /proc/31777/task/85517/fd (opendir: Permission denied) 2024-03-21T10:28:38.400 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mgr 31777 85519 mgr-fin ceph cwd unknown /proc/31777/task/85519/cwd (readlink: Permission denied) 2024-03-21T10:28:38.400 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mgr 31777 85519 mgr-fin ceph rtd unknown /proc/31777/task/85519/root (readlink: Permission denied) 2024-03-21T10:28:38.400 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mgr 31777 85519 mgr-fin ceph txt unknown /proc/31777/task/85519/exe (readlink: Permission denied) 2024-03-21T10:28:38.400 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mgr 31777 85519 mgr-fin ceph NOFD /proc/31777/task/85519/fd (opendir: Permission denied) 2024-03-21T10:28:38.400 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mgr 31777 85520 mgr-fin ceph cwd unknown /proc/31777/task/85520/cwd (readlink: Permission denied) 2024-03-21T10:28:38.400 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mgr 31777 85520 mgr-fin ceph rtd unknown /proc/31777/task/85520/root (readlink: Permission denied) 2024-03-21T10:28:38.400 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mgr 31777 85520 mgr-fin ceph txt unknown /proc/31777/task/85520/exe (readlink: Permission denied) 2024-03-21T10:28:38.400 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mgr 31777 85520 mgr-fin ceph NOFD /proc/31777/task/85520/fd (opendir: Permission denied) 2024-03-21T10:28:38.400 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mgr 31777 85521 mgr-fin ceph cwd unknown /proc/31777/task/85521/cwd (readlink: Permission denied) 2024-03-21T10:28:38.400 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mgr 31777 85521 mgr-fin ceph rtd unknown /proc/31777/task/85521/root (readlink: Permission denied) 2024-03-21T10:28:38.400 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mgr 31777 85521 mgr-fin ceph txt unknown /proc/31777/task/85521/exe (readlink: Permission denied) 2024-03-21T10:28:38.400 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mgr 31777 85521 mgr-fin ceph NOFD /proc/31777/task/85521/fd (opendir: Permission denied) 2024-03-21T10:28:38.401 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mgr 31777 85522 mgr-fin ceph cwd unknown /proc/31777/task/85522/cwd (readlink: Permission denied) 2024-03-21T10:28:38.401 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mgr 31777 85522 mgr-fin ceph rtd unknown /proc/31777/task/85522/root (readlink: Permission denied) 2024-03-21T10:28:38.401 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mgr 31777 85522 mgr-fin ceph txt unknown /proc/31777/task/85522/exe (readlink: Permission denied) 2024-03-21T10:28:38.401 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mgr 31777 85522 mgr-fin ceph NOFD /proc/31777/task/85522/fd (opendir: Permission denied) 2024-03-21T10:28:38.401 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mgr 31777 85523 mgr-fin ceph cwd unknown /proc/31777/task/85523/cwd (readlink: Permission denied) 2024-03-21T10:28:38.401 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mgr 31777 85523 mgr-fin ceph rtd unknown /proc/31777/task/85523/root (readlink: Permission denied) 2024-03-21T10:28:38.401 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mgr 31777 85523 mgr-fin ceph txt unknown /proc/31777/task/85523/exe (readlink: Permission denied) 2024-03-21T10:28:38.401 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mgr 31777 85523 mgr-fin ceph NOFD /proc/31777/task/85523/fd (opendir: Permission denied) 2024-03-21T10:28:38.401 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mgr 31777 85524 mgr-fin ceph cwd unknown /proc/31777/task/85524/cwd (readlink: Permission denied) 2024-03-21T10:28:38.401 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mgr 31777 85524 mgr-fin ceph rtd unknown /proc/31777/task/85524/root (readlink: Permission denied) 2024-03-21T10:28:38.401 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mgr 31777 85524 mgr-fin ceph txt unknown /proc/31777/task/85524/exe (readlink: Permission denied) 2024-03-21T10:28:38.401 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mgr 31777 85524 mgr-fin ceph NOFD /proc/31777/task/85524/fd (opendir: Permission denied) 2024-03-21T10:28:38.401 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mgr 31777 85525 mgr-fin ceph cwd unknown /proc/31777/task/85525/cwd (readlink: Permission denied) 2024-03-21T10:28:38.402 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mgr 31777 85525 mgr-fin ceph rtd unknown /proc/31777/task/85525/root (readlink: Permission denied) 2024-03-21T10:28:38.402 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mgr 31777 85525 mgr-fin ceph txt unknown /proc/31777/task/85525/exe (readlink: Permission denied) 2024-03-21T10:28:38.402 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mgr 31777 85525 mgr-fin ceph NOFD /proc/31777/task/85525/fd (opendir: Permission denied) 2024-03-21T10:28:38.402 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mgr 31777 85526 mgr-fin ceph cwd unknown /proc/31777/task/85526/cwd (readlink: Permission denied) 2024-03-21T10:28:38.402 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mgr 31777 85526 mgr-fin ceph rtd unknown /proc/31777/task/85526/root (readlink: Permission denied) 2024-03-21T10:28:38.402 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mgr 31777 85526 mgr-fin ceph txt unknown /proc/31777/task/85526/exe (readlink: Permission denied) 2024-03-21T10:28:38.402 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mgr 31777 85526 mgr-fin ceph NOFD /proc/31777/task/85526/fd (opendir: Permission denied) 2024-03-21T10:28:38.402 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mgr 31777 85527 mgr-fin ceph cwd unknown /proc/31777/task/85527/cwd (readlink: Permission denied) 2024-03-21T10:28:38.402 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mgr 31777 85527 mgr-fin ceph rtd unknown /proc/31777/task/85527/root (readlink: Permission denied) 2024-03-21T10:28:38.402 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mgr 31777 85527 mgr-fin ceph txt unknown /proc/31777/task/85527/exe (readlink: Permission denied) 2024-03-21T10:28:38.402 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mgr 31777 85527 mgr-fin ceph NOFD /proc/31777/task/85527/fd (opendir: Permission denied) 2024-03-21T10:28:38.402 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mgr 31777 85529 m-fin-vol ceph cwd unknown /proc/31777/task/85529/cwd (readlink: Permission denied) 2024-03-21T10:28:38.402 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mgr 31777 85529 m-fin-vol ceph rtd unknown /proc/31777/task/85529/root (readlink: Permission denied) 2024-03-21T10:28:38.403 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mgr 31777 85529 m-fin-vol ceph txt unknown /proc/31777/task/85529/exe (readlink: Permission denied) 2024-03-21T10:28:38.403 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mgr 31777 85529 m-fin-vol ceph NOFD /proc/31777/task/85529/fd (opendir: Permission denied) 2024-03-21T10:28:38.403 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mgr 31777 85530 io_contex ceph cwd unknown /proc/31777/task/85530/cwd (readlink: Permission denied) 2024-03-21T10:28:38.403 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mgr 31777 85530 io_contex ceph rtd unknown /proc/31777/task/85530/root (readlink: Permission denied) 2024-03-21T10:28:38.403 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mgr 31777 85530 io_contex ceph txt unknown /proc/31777/task/85530/exe (readlink: Permission denied) 2024-03-21T10:28:38.403 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mgr 31777 85530 io_contex ceph NOFD /proc/31777/task/85530/fd (opendir: Permission denied) 2024-03-21T10:28:38.403 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mgr 31777 85531 io_contex ceph cwd unknown /proc/31777/task/85531/cwd (readlink: Permission denied) 2024-03-21T10:28:38.403 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mgr 31777 85531 io_contex ceph rtd unknown /proc/31777/task/85531/root (readlink: Permission denied) 2024-03-21T10:28:38.403 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mgr 31777 85531 io_contex ceph txt unknown /proc/31777/task/85531/exe (readlink: Permission denied) 2024-03-21T10:28:38.403 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mgr 31777 85531 io_contex ceph NOFD /proc/31777/task/85531/fd (opendir: Permission denied) 2024-03-21T10:28:38.403 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mgr 31777 85557 dashboard ceph cwd unknown /proc/31777/task/85557/cwd (readlink: Permission denied) 2024-03-21T10:28:38.403 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mgr 31777 85557 dashboard ceph rtd unknown /proc/31777/task/85557/root (readlink: Permission denied) 2024-03-21T10:28:38.403 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mgr 31777 85557 dashboard ceph txt unknown /proc/31777/task/85557/exe (readlink: Permission denied) 2024-03-21T10:28:38.403 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mgr 31777 85557 dashboard ceph NOFD /proc/31777/task/85557/fd (opendir: Permission denied) 2024-03-21T10:28:38.403 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mgr 31777 85558 dashboard ceph cwd unknown /proc/31777/task/85558/cwd (readlink: Permission denied) 2024-03-21T10:28:38.404 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mgr 31777 85558 dashboard ceph rtd unknown /proc/31777/task/85558/root (readlink: Permission denied) 2024-03-21T10:28:38.404 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mgr 31777 85558 dashboard ceph txt unknown /proc/31777/task/85558/exe (readlink: Permission denied) 2024-03-21T10:28:38.404 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mgr 31777 85558 dashboard ceph NOFD /proc/31777/task/85558/fd (opendir: Permission denied) 2024-03-21T10:28:38.404 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mgr 31777 85559 dashboard ceph cwd unknown /proc/31777/task/85559/cwd (readlink: Permission denied) 2024-03-21T10:28:38.404 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mgr 31777 85559 dashboard ceph rtd unknown /proc/31777/task/85559/root (readlink: Permission denied) 2024-03-21T10:28:38.404 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mgr 31777 85559 dashboard ceph txt unknown /proc/31777/task/85559/exe (readlink: Permission denied) 2024-03-21T10:28:38.404 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mgr 31777 85559 dashboard ceph NOFD /proc/31777/task/85559/fd (opendir: Permission denied) 2024-03-21T10:28:38.404 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mgr 31777 85560 dashboard ceph cwd unknown /proc/31777/task/85560/cwd (readlink: Permission denied) 2024-03-21T10:28:38.404 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mgr 31777 85560 dashboard ceph rtd unknown /proc/31777/task/85560/root (readlink: Permission denied) 2024-03-21T10:28:38.404 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mgr 31777 85560 dashboard ceph txt unknown /proc/31777/task/85560/exe (readlink: Permission denied) 2024-03-21T10:28:38.404 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mgr 31777 85560 dashboard ceph NOFD /proc/31777/task/85560/fd (opendir: Permission denied) 2024-03-21T10:28:38.404 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mgr 31777 85561 dashboard ceph cwd unknown /proc/31777/task/85561/cwd (readlink: Permission denied) 2024-03-21T10:28:38.404 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mgr 31777 85561 dashboard ceph rtd unknown /proc/31777/task/85561/root (readlink: Permission denied) 2024-03-21T10:28:38.404 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mgr 31777 85561 dashboard ceph txt unknown /proc/31777/task/85561/exe (readlink: Permission denied) 2024-03-21T10:28:38.404 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mgr 31777 85561 dashboard ceph NOFD /proc/31777/task/85561/fd (opendir: Permission denied) 2024-03-21T10:28:38.405 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mgr 31777 85562 dashboard ceph cwd unknown /proc/31777/task/85562/cwd (readlink: Permission denied) 2024-03-21T10:28:38.405 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mgr 31777 85562 dashboard ceph rtd unknown /proc/31777/task/85562/root (readlink: Permission denied) 2024-03-21T10:28:38.405 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mgr 31777 85562 dashboard ceph txt unknown /proc/31777/task/85562/exe (readlink: Permission denied) 2024-03-21T10:28:38.405 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mgr 31777 85562 dashboard ceph NOFD /proc/31777/task/85562/fd (opendir: Permission denied) 2024-03-21T10:28:38.405 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mgr 31777 85563 dashboard ceph cwd unknown /proc/31777/task/85563/cwd (readlink: Permission denied) 2024-03-21T10:28:38.405 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mgr 31777 85563 dashboard ceph rtd unknown /proc/31777/task/85563/root (readlink: Permission denied) 2024-03-21T10:28:38.405 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mgr 31777 85563 dashboard ceph txt unknown /proc/31777/task/85563/exe (readlink: Permission denied) 2024-03-21T10:28:38.405 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mgr 31777 85563 dashboard ceph NOFD /proc/31777/task/85563/fd (opendir: Permission denied) 2024-03-21T10:28:38.405 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mgr 31777 85564 dashboard ceph cwd unknown /proc/31777/task/85564/cwd (readlink: Permission denied) 2024-03-21T10:28:38.405 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mgr 31777 85564 dashboard ceph rtd unknown /proc/31777/task/85564/root (readlink: Permission denied) 2024-03-21T10:28:38.406 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mgr 31777 85564 dashboard ceph txt unknown /proc/31777/task/85564/exe (readlink: Permission denied) 2024-03-21T10:28:38.406 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mgr 31777 85564 dashboard ceph NOFD /proc/31777/task/85564/fd (opendir: Permission denied) 2024-03-21T10:28:38.406 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mgr 31777 85565 dashboard ceph cwd unknown /proc/31777/task/85565/cwd (readlink: Permission denied) 2024-03-21T10:28:38.406 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mgr 31777 85565 dashboard ceph rtd unknown /proc/31777/task/85565/root (readlink: Permission denied) 2024-03-21T10:28:38.406 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mgr 31777 85565 dashboard ceph txt unknown /proc/31777/task/85565/exe (readlink: Permission denied) 2024-03-21T10:28:38.406 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mgr 31777 85565 dashboard ceph NOFD /proc/31777/task/85565/fd (opendir: Permission denied) 2024-03-21T10:28:38.406 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mgr 31777 85566 dashboard ceph cwd unknown /proc/31777/task/85566/cwd (readlink: Permission denied) 2024-03-21T10:28:38.406 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mgr 31777 85566 dashboard ceph rtd unknown /proc/31777/task/85566/root (readlink: Permission denied) 2024-03-21T10:28:38.406 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mgr 31777 85566 dashboard ceph txt unknown /proc/31777/task/85566/exe (readlink: Permission denied) 2024-03-21T10:28:38.406 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mgr 31777 85566 dashboard ceph NOFD /proc/31777/task/85566/fd (opendir: Permission denied) 2024-03-21T10:28:38.406 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mgr 31777 85567 dashboard ceph cwd unknown /proc/31777/task/85567/cwd (readlink: Permission denied) 2024-03-21T10:28:38.406 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mgr 31777 85567 dashboard ceph rtd unknown /proc/31777/task/85567/root (readlink: Permission denied) 2024-03-21T10:28:38.406 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mgr 31777 85567 dashboard ceph txt unknown /proc/31777/task/85567/exe (readlink: Permission denied) 2024-03-21T10:28:38.407 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mgr 31777 85567 dashboard ceph NOFD /proc/31777/task/85567/fd (opendir: Permission denied) 2024-03-21T10:28:38.407 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mgr 31777 85604 mgr-fin ceph cwd unknown /proc/31777/task/85604/cwd (readlink: Permission denied) 2024-03-21T10:28:38.407 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mgr 31777 85604 mgr-fin ceph rtd unknown /proc/31777/task/85604/root (readlink: Permission denied) 2024-03-21T10:28:38.407 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mgr 31777 85604 mgr-fin ceph txt unknown /proc/31777/task/85604/exe (readlink: Permission denied) 2024-03-21T10:28:38.407 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mgr 31777 85604 mgr-fin ceph NOFD /proc/31777/task/85604/fd (opendir: Permission denied) 2024-03-21T10:28:38.407 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mgr 31777 85606 mgr-fin ceph cwd unknown /proc/31777/task/85606/cwd (readlink: Permission denied) 2024-03-21T10:28:38.407 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mgr 31777 85606 mgr-fin ceph rtd unknown /proc/31777/task/85606/root (readlink: Permission denied) 2024-03-21T10:28:38.407 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mgr 31777 85606 mgr-fin ceph txt unknown /proc/31777/task/85606/exe (readlink: Permission denied) 2024-03-21T10:28:38.407 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mgr 31777 85606 mgr-fin ceph NOFD /proc/31777/task/85606/fd (opendir: Permission denied) 2024-03-21T10:28:38.407 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mgr 31777 85607 mgr-fin ceph cwd unknown /proc/31777/task/85607/cwd (readlink: Permission denied) 2024-03-21T10:28:38.407 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mgr 31777 85607 mgr-fin ceph rtd unknown /proc/31777/task/85607/root (readlink: Permission denied) 2024-03-21T10:28:38.407 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mgr 31777 85607 mgr-fin ceph txt unknown /proc/31777/task/85607/exe (readlink: Permission denied) 2024-03-21T10:28:38.407 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mgr 31777 85607 mgr-fin ceph NOFD /proc/31777/task/85607/fd (opendir: Permission denied) 2024-03-21T10:28:38.407 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mgr 31777 85608 mgr-fin ceph cwd unknown /proc/31777/task/85608/cwd (readlink: Permission denied) 2024-03-21T10:28:38.407 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mgr 31777 85608 mgr-fin ceph rtd unknown /proc/31777/task/85608/root (readlink: Permission denied) 2024-03-21T10:28:38.408 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mgr 31777 85608 mgr-fin ceph txt unknown /proc/31777/task/85608/exe (readlink: Permission denied) 2024-03-21T10:28:38.408 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mgr 31777 85608 mgr-fin ceph NOFD /proc/31777/task/85608/fd (opendir: Permission denied) 2024-03-21T10:28:38.408 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mgr 31777 85610 mgr-fin ceph cwd unknown /proc/31777/task/85610/cwd (readlink: Permission denied) 2024-03-21T10:28:38.408 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mgr 31777 85610 mgr-fin ceph rtd unknown /proc/31777/task/85610/root (readlink: Permission denied) 2024-03-21T10:28:38.408 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mgr 31777 85610 mgr-fin ceph txt unknown /proc/31777/task/85610/exe (readlink: Permission denied) 2024-03-21T10:28:38.408 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mgr 31777 85610 mgr-fin ceph NOFD /proc/31777/task/85610/fd (opendir: Permission denied) 2024-03-21T10:28:38.408 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mgr 31777 85611 mgr-fin ceph cwd unknown /proc/31777/task/85611/cwd (readlink: Permission denied) 2024-03-21T10:28:38.408 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mgr 31777 85611 mgr-fin ceph rtd unknown /proc/31777/task/85611/root (readlink: Permission denied) 2024-03-21T10:28:38.408 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mgr 31777 85611 mgr-fin ceph txt unknown /proc/31777/task/85611/exe (readlink: Permission denied) 2024-03-21T10:28:38.408 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mgr 31777 85611 mgr-fin ceph NOFD /proc/31777/task/85611/fd (opendir: Permission denied) 2024-03-21T10:28:38.408 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mgr 31777 85612 mgr-fin ceph cwd unknown /proc/31777/task/85612/cwd (readlink: Permission denied) 2024-03-21T10:28:38.408 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mgr 31777 85612 mgr-fin ceph rtd unknown /proc/31777/task/85612/root (readlink: Permission denied) 2024-03-21T10:28:38.408 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mgr 31777 85612 mgr-fin ceph txt unknown /proc/31777/task/85612/exe (readlink: Permission denied) 2024-03-21T10:28:38.409 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mgr 31777 85612 mgr-fin ceph NOFD /proc/31777/task/85612/fd (opendir: Permission denied) 2024-03-21T10:28:38.409 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mgr 31777 85613 mgr-fin ceph cwd unknown /proc/31777/task/85613/cwd (readlink: Permission denied) 2024-03-21T10:28:38.409 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mgr 31777 85613 mgr-fin ceph rtd unknown /proc/31777/task/85613/root (readlink: Permission denied) 2024-03-21T10:28:38.409 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mgr 31777 85613 mgr-fin ceph txt unknown /proc/31777/task/85613/exe (readlink: Permission denied) 2024-03-21T10:28:38.409 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mgr 31777 85613 mgr-fin ceph NOFD /proc/31777/task/85613/fd (opendir: Permission denied) 2024-03-21T10:28:38.409 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mgr 31777 85614 mgr-fin ceph cwd unknown /proc/31777/task/85614/cwd (readlink: Permission denied) 2024-03-21T10:28:38.409 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mgr 31777 85614 mgr-fin ceph rtd unknown /proc/31777/task/85614/root (readlink: Permission denied) 2024-03-21T10:28:38.409 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mgr 31777 85614 mgr-fin ceph txt unknown /proc/31777/task/85614/exe (readlink: Permission denied) 2024-03-21T10:28:38.409 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mgr 31777 85614 mgr-fin ceph NOFD /proc/31777/task/85614/fd (opendir: Permission denied) 2024-03-21T10:28:38.409 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mgr 31777 85615 mgr-fin ceph cwd unknown /proc/31777/task/85615/cwd (readlink: Permission denied) 2024-03-21T10:28:38.409 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mgr 31777 85615 mgr-fin ceph rtd unknown /proc/31777/task/85615/root (readlink: Permission denied) 2024-03-21T10:28:38.409 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mgr 31777 85615 mgr-fin ceph txt unknown /proc/31777/task/85615/exe (readlink: Permission denied) 2024-03-21T10:28:38.410 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mgr 31777 85615 mgr-fin ceph NOFD /proc/31777/task/85615/fd (opendir: Permission denied) 2024-03-21T10:28:38.410 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mgr 31777 85616 mgr-fin ceph cwd unknown /proc/31777/task/85616/cwd (readlink: Permission denied) 2024-03-21T10:28:38.410 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mgr 31777 85616 mgr-fin ceph rtd unknown /proc/31777/task/85616/root (readlink: Permission denied) 2024-03-21T10:28:38.410 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mgr 31777 85616 mgr-fin ceph txt unknown /proc/31777/task/85616/exe (readlink: Permission denied) 2024-03-21T10:28:38.410 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mgr 31777 85616 mgr-fin ceph NOFD /proc/31777/task/85616/fd (opendir: Permission denied) 2024-03-21T10:28:38.410 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mgr 31777 85618 dashboard ceph cwd unknown /proc/31777/task/85618/cwd (readlink: Permission denied) 2024-03-21T10:28:38.410 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mgr 31777 85618 dashboard ceph rtd unknown /proc/31777/task/85618/root (readlink: Permission denied) 2024-03-21T10:28:38.410 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mgr 31777 85618 dashboard ceph txt unknown /proc/31777/task/85618/exe (readlink: Permission denied) 2024-03-21T10:28:38.410 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mgr 31777 85618 dashboard ceph NOFD /proc/31777/task/85618/fd (opendir: Permission denied) 2024-03-21T10:28:38.410 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mgr 31777 85647 mgr-fin ceph cwd unknown /proc/31777/task/85647/cwd (readlink: Permission denied) 2024-03-21T10:28:38.410 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mgr 31777 85647 mgr-fin ceph rtd unknown /proc/31777/task/85647/root (readlink: Permission denied) 2024-03-21T10:28:38.410 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mgr 31777 85647 mgr-fin ceph txt unknown /proc/31777/task/85647/exe (readlink: Permission denied) 2024-03-21T10:28:38.410 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mgr 31777 85647 mgr-fin ceph NOFD /proc/31777/task/85647/fd (opendir: Permission denied) 2024-03-21T10:28:38.410 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mgr 31777 85648 mgr-fin ceph cwd unknown /proc/31777/task/85648/cwd (readlink: Permission denied) 2024-03-21T10:28:38.411 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mgr 31777 85648 mgr-fin ceph rtd unknown /proc/31777/task/85648/root (readlink: Permission denied) 2024-03-21T10:28:38.411 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mgr 31777 85648 mgr-fin ceph txt unknown /proc/31777/task/85648/exe (readlink: Permission denied) 2024-03-21T10:28:38.411 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mgr 31777 85648 mgr-fin ceph NOFD /proc/31777/task/85648/fd (opendir: Permission denied) 2024-03-21T10:28:38.411 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mgr 31777 85649 mgr-fin ceph cwd unknown /proc/31777/task/85649/cwd (readlink: Permission denied) 2024-03-21T10:28:38.411 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mgr 31777 85649 mgr-fin ceph rtd unknown /proc/31777/task/85649/root (readlink: Permission denied) 2024-03-21T10:28:38.411 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mgr 31777 85649 mgr-fin ceph txt unknown /proc/31777/task/85649/exe (readlink: Permission denied) 2024-03-21T10:28:38.411 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mgr 31777 85649 mgr-fin ceph NOFD /proc/31777/task/85649/fd (opendir: Permission denied) 2024-03-21T10:28:38.411 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mgr 31777 85650 mgr-fin ceph cwd unknown /proc/31777/task/85650/cwd (readlink: Permission denied) 2024-03-21T10:28:38.411 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mgr 31777 85650 mgr-fin ceph rtd unknown /proc/31777/task/85650/root (readlink: Permission denied) 2024-03-21T10:28:38.411 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mgr 31777 85650 mgr-fin ceph txt unknown /proc/31777/task/85650/exe (readlink: Permission denied) 2024-03-21T10:28:38.411 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mgr 31777 85650 mgr-fin ceph NOFD /proc/31777/task/85650/fd (opendir: Permission denied) 2024-03-21T10:28:38.411 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mgr 31777 85651 mgr-fin ceph cwd unknown /proc/31777/task/85651/cwd (readlink: Permission denied) 2024-03-21T10:28:38.411 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mgr 31777 85651 mgr-fin ceph rtd unknown /proc/31777/task/85651/root (readlink: Permission denied) 2024-03-21T10:28:38.411 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mgr 31777 85651 mgr-fin ceph txt unknown /proc/31777/task/85651/exe (readlink: Permission denied) 2024-03-21T10:28:38.411 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mgr 31777 85651 mgr-fin ceph NOFD /proc/31777/task/85651/fd (opendir: Permission denied) 2024-03-21T10:28:38.412 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mgr 31777 85652 mgr-fin ceph cwd unknown /proc/31777/task/85652/cwd (readlink: Permission denied) 2024-03-21T10:28:38.412 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mgr 31777 85652 mgr-fin ceph rtd unknown /proc/31777/task/85652/root (readlink: Permission denied) 2024-03-21T10:28:38.412 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mgr 31777 85652 mgr-fin ceph txt unknown /proc/31777/task/85652/exe (readlink: Permission denied) 2024-03-21T10:28:38.412 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mgr 31777 85652 mgr-fin ceph NOFD /proc/31777/task/85652/fd (opendir: Permission denied) 2024-03-21T10:28:38.412 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mgr 31777 85653 mgr-fin ceph cwd unknown /proc/31777/task/85653/cwd (readlink: Permission denied) 2024-03-21T10:28:38.412 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mgr 31777 85653 mgr-fin ceph rtd unknown /proc/31777/task/85653/root (readlink: Permission denied) 2024-03-21T10:28:38.412 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mgr 31777 85653 mgr-fin ceph txt unknown /proc/31777/task/85653/exe (readlink: Permission denied) 2024-03-21T10:28:38.412 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mgr 31777 85653 mgr-fin ceph NOFD /proc/31777/task/85653/fd (opendir: Permission denied) 2024-03-21T10:28:38.412 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mgr 31777 85654 mgr-fin ceph cwd unknown /proc/31777/task/85654/cwd (readlink: Permission denied) 2024-03-21T10:28:38.412 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mgr 31777 85654 mgr-fin ceph rtd unknown /proc/31777/task/85654/root (readlink: Permission denied) 2024-03-21T10:28:38.412 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mgr 31777 85654 mgr-fin ceph txt unknown /proc/31777/task/85654/exe (readlink: Permission denied) 2024-03-21T10:28:38.412 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mgr 31777 85654 mgr-fin ceph NOFD /proc/31777/task/85654/fd (opendir: Permission denied) 2024-03-21T10:28:38.412 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mgr 31777 85655 mgr-fin ceph cwd unknown /proc/31777/task/85655/cwd (readlink: Permission denied) 2024-03-21T10:28:38.412 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mgr 31777 85655 mgr-fin ceph rtd unknown /proc/31777/task/85655/root (readlink: Permission denied) 2024-03-21T10:28:38.413 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mgr 31777 85655 mgr-fin ceph txt unknown /proc/31777/task/85655/exe (readlink: Permission denied) 2024-03-21T10:28:38.413 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mgr 31777 85655 mgr-fin ceph NOFD /proc/31777/task/85655/fd (opendir: Permission denied) 2024-03-21T10:28:38.413 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mgr 31777 85656 mgr-fin ceph cwd unknown /proc/31777/task/85656/cwd (readlink: Permission denied) 2024-03-21T10:28:38.413 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mgr 31777 85656 mgr-fin ceph rtd unknown /proc/31777/task/85656/root (readlink: Permission denied) 2024-03-21T10:28:38.413 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mgr 31777 85656 mgr-fin ceph txt unknown /proc/31777/task/85656/exe (readlink: Permission denied) 2024-03-21T10:28:38.413 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mgr 31777 85656 mgr-fin ceph NOFD /proc/31777/task/85656/fd (opendir: Permission denied) 2024-03-21T10:28:38.413 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mgr 31777 85657 mgr-fin ceph cwd unknown /proc/31777/task/85657/cwd (readlink: Permission denied) 2024-03-21T10:28:38.413 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mgr 31777 85657 mgr-fin ceph rtd unknown /proc/31777/task/85657/root (readlink: Permission denied) 2024-03-21T10:28:38.413 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mgr 31777 85657 mgr-fin ceph txt unknown /proc/31777/task/85657/exe (readlink: Permission denied) 2024-03-21T10:28:38.413 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mgr 31777 85657 mgr-fin ceph NOFD /proc/31777/task/85657/fd (opendir: Permission denied) 2024-03-21T10:28:38.413 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mgr 31777 85837 devicehea ceph cwd unknown /proc/31777/task/85837/cwd (readlink: Permission denied) 2024-03-21T10:28:38.413 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mgr 31777 85837 devicehea ceph rtd unknown /proc/31777/task/85837/root (readlink: Permission denied) 2024-03-21T10:28:38.413 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mgr 31777 85837 devicehea ceph txt unknown /proc/31777/task/85837/exe (readlink: Permission denied) 2024-03-21T10:28:38.414 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mgr 31777 85837 devicehea ceph NOFD /proc/31777/task/85837/fd (opendir: Permission denied) 2024-03-21T10:28:38.414 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mgr 31777 85841 io_contex ceph cwd unknown /proc/31777/task/85841/cwd (readlink: Permission denied) 2024-03-21T10:28:38.414 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mgr 31777 85841 io_contex ceph rtd unknown /proc/31777/task/85841/root (readlink: Permission denied) 2024-03-21T10:28:38.414 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mgr 31777 85841 io_contex ceph txt unknown /proc/31777/task/85841/exe (readlink: Permission denied) 2024-03-21T10:28:38.414 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mgr 31777 85841 io_contex ceph NOFD /proc/31777/task/85841/fd (opendir: Permission denied) 2024-03-21T10:28:38.414 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mgr 31777 85842 io_contex ceph cwd unknown /proc/31777/task/85842/cwd (readlink: Permission denied) 2024-03-21T10:28:38.414 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mgr 31777 85842 io_contex ceph rtd unknown /proc/31777/task/85842/root (readlink: Permission denied) 2024-03-21T10:28:38.414 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mgr 31777 85842 io_contex ceph txt unknown /proc/31777/task/85842/exe (readlink: Permission denied) 2024-03-21T10:28:38.414 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mgr 31777 85842 io_contex ceph NOFD /proc/31777/task/85842/fd (opendir: Permission denied) 2024-03-21T10:28:38.414 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mgr 31777 85843 ceph_time ceph cwd unknown /proc/31777/task/85843/cwd (readlink: Permission denied) 2024-03-21T10:28:38.414 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mgr 31777 85843 ceph_time ceph rtd unknown /proc/31777/task/85843/root (readlink: Permission denied) 2024-03-21T10:28:38.414 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mgr 31777 85843 ceph_time ceph txt unknown /proc/31777/task/85843/exe (readlink: Permission denied) 2024-03-21T10:28:38.414 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mgr 31777 85843 ceph_time ceph NOFD /proc/31777/task/85843/fd (opendir: Permission denied) 2024-03-21T10:28:38.415 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mgr 31777 85844 ms_dispat ceph cwd unknown /proc/31777/task/85844/cwd (readlink: Permission denied) 2024-03-21T10:28:38.415 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mgr 31777 85844 ms_dispat ceph rtd unknown /proc/31777/task/85844/root (readlink: Permission denied) 2024-03-21T10:28:38.415 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mgr 31777 85844 ms_dispat ceph txt unknown /proc/31777/task/85844/exe (readlink: Permission denied) 2024-03-21T10:28:38.415 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mgr 31777 85844 ms_dispat ceph NOFD /proc/31777/task/85844/fd (opendir: Permission denied) 2024-03-21T10:28:38.415 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mgr 31777 85845 ms_local ceph cwd unknown /proc/31777/task/85845/cwd (readlink: Permission denied) 2024-03-21T10:28:38.415 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mgr 31777 85845 ms_local ceph rtd unknown /proc/31777/task/85845/root (readlink: Permission denied) 2024-03-21T10:28:38.415 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mgr 31777 85845 ms_local ceph txt unknown /proc/31777/task/85845/exe (readlink: Permission denied) 2024-03-21T10:28:38.415 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mgr 31777 85845 ms_local ceph NOFD /proc/31777/task/85845/fd (opendir: Permission denied) 2024-03-21T10:28:38.415 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mgr 31777 85846 safe_time ceph cwd unknown /proc/31777/task/85846/cwd (readlink: Permission denied) 2024-03-21T10:28:38.415 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mgr 31777 85846 safe_time ceph rtd unknown /proc/31777/task/85846/root (readlink: Permission denied) 2024-03-21T10:28:38.415 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mgr 31777 85846 safe_time ceph txt unknown /proc/31777/task/85846/exe (readlink: Permission denied) 2024-03-21T10:28:38.415 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mgr 31777 85846 safe_time ceph NOFD /proc/31777/task/85846/fd (opendir: Permission denied) 2024-03-21T10:28:38.415 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mgr 31777 85847 safe_time ceph cwd unknown /proc/31777/task/85847/cwd (readlink: Permission denied) 2024-03-21T10:28:38.415 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mgr 31777 85847 safe_time ceph rtd unknown /proc/31777/task/85847/root (readlink: Permission denied) 2024-03-21T10:28:38.416 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mgr 31777 85847 safe_time ceph txt unknown /proc/31777/task/85847/exe (readlink: Permission denied) 2024-03-21T10:28:38.416 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mgr 31777 85847 safe_time ceph NOFD /proc/31777/task/85847/fd (opendir: Permission denied) 2024-03-21T10:28:38.416 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mgr 31777 87889 ceph_time ceph cwd unknown /proc/31777/task/87889/cwd (readlink: Permission denied) 2024-03-21T10:28:38.416 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mgr 31777 87889 ceph_time ceph rtd unknown /proc/31777/task/87889/root (readlink: Permission denied) 2024-03-21T10:28:38.416 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mgr 31777 87889 ceph_time ceph txt unknown /proc/31777/task/87889/exe (readlink: Permission denied) 2024-03-21T10:28:38.416 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mgr 31777 87889 ceph_time ceph NOFD /proc/31777/task/87889/fd (opendir: Permission denied) 2024-03-21T10:28:38.416 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mgr 31777 87890 fn_anonym ceph cwd unknown /proc/31777/task/87890/cwd (readlink: Permission denied) 2024-03-21T10:28:38.416 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mgr 31777 87890 fn_anonym ceph rtd unknown /proc/31777/task/87890/root (readlink: Permission denied) 2024-03-21T10:28:38.416 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mgr 31777 87890 fn_anonym ceph txt unknown /proc/31777/task/87890/exe (readlink: Permission denied) 2024-03-21T10:28:38.416 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mgr 31777 87890 fn_anonym ceph NOFD /proc/31777/task/87890/fd (opendir: Permission denied) 2024-03-21T10:28:38.416 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mgr 31777 87891 safe_time ceph cwd unknown /proc/31777/task/87891/cwd (readlink: Permission denied) 2024-03-21T10:28:38.416 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mgr 31777 87891 safe_time ceph rtd unknown /proc/31777/task/87891/root (readlink: Permission denied) 2024-03-21T10:28:38.417 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mgr 31777 87891 safe_time ceph txt unknown /proc/31777/task/87891/exe (readlink: Permission denied) 2024-03-21T10:28:38.417 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mgr 31777 87891 safe_time ceph NOFD /proc/31777/task/87891/fd (opendir: Permission denied) 2024-03-21T10:28:38.417 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mgr 31777 87892 fn_anonym ceph cwd unknown /proc/31777/task/87892/cwd (readlink: Permission denied) 2024-03-21T10:28:38.417 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mgr 31777 87892 fn_anonym ceph rtd unknown /proc/31777/task/87892/root (readlink: Permission denied) 2024-03-21T10:28:38.417 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mgr 31777 87892 fn_anonym ceph txt unknown /proc/31777/task/87892/exe (readlink: Permission denied) 2024-03-21T10:28:38.417 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mgr 31777 87892 fn_anonym ceph NOFD /proc/31777/task/87892/fd (opendir: Permission denied) 2024-03-21T10:28:38.417 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mgr 31777 87893 flusher ceph cwd unknown /proc/31777/task/87893/cwd (readlink: Permission denied) 2024-03-21T10:28:38.417 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mgr 31777 87893 flusher ceph rtd unknown /proc/31777/task/87893/root (readlink: Permission denied) 2024-03-21T10:28:38.417 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mgr 31777 87893 flusher ceph txt unknown /proc/31777/task/87893/exe (readlink: Permission denied) 2024-03-21T10:28:38.417 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mgr 31777 87893 flusher ceph NOFD /proc/31777/task/87893/fd (opendir: Permission denied) 2024-03-21T10:28:38.417 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mgr 31777 87894 ms_dispat ceph cwd unknown /proc/31777/task/87894/cwd (readlink: Permission denied) 2024-03-21T10:28:38.417 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mgr 31777 87894 ms_dispat ceph rtd unknown /proc/31777/task/87894/root (readlink: Permission denied) 2024-03-21T10:28:38.418 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mgr 31777 87894 ms_dispat ceph txt unknown /proc/31777/task/87894/exe (readlink: Permission denied) 2024-03-21T10:28:38.418 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mgr 31777 87894 ms_dispat ceph NOFD /proc/31777/task/87894/fd (opendir: Permission denied) 2024-03-21T10:28:38.418 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mgr 31777 87895 ms_local ceph cwd unknown /proc/31777/task/87895/cwd (readlink: Permission denied) 2024-03-21T10:28:38.418 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mgr 31777 87895 ms_local ceph rtd unknown /proc/31777/task/87895/root (readlink: Permission denied) 2024-03-21T10:28:38.418 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mgr 31777 87895 ms_local ceph txt unknown /proc/31777/task/87895/exe (readlink: Permission denied) 2024-03-21T10:28:38.418 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mgr 31777 87895 ms_local ceph NOFD /proc/31777/task/87895/fd (opendir: Permission denied) 2024-03-21T10:28:38.418 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mgr 31777 87896 safe_time ceph cwd unknown /proc/31777/task/87896/cwd (readlink: Permission denied) 2024-03-21T10:28:38.418 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mgr 31777 87896 safe_time ceph rtd unknown /proc/31777/task/87896/root (readlink: Permission denied) 2024-03-21T10:28:38.418 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mgr 31777 87896 safe_time ceph txt unknown /proc/31777/task/87896/exe (readlink: Permission denied) 2024-03-21T10:28:38.418 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mgr 31777 87896 safe_time ceph NOFD /proc/31777/task/87896/fd (opendir: Permission denied) 2024-03-21T10:28:38.418 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mgr 31777 87897 m-fin-sna ceph cwd unknown /proc/31777/task/87897/cwd (readlink: Permission denied) 2024-03-21T10:28:38.418 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mgr 31777 87897 m-fin-sna ceph rtd unknown /proc/31777/task/87897/root (readlink: Permission denied) 2024-03-21T10:28:38.418 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mgr 31777 87897 m-fin-sna ceph txt unknown /proc/31777/task/87897/exe (readlink: Permission denied) 2024-03-21T10:28:38.419 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mgr 31777 87897 m-fin-sna ceph NOFD /proc/31777/task/87897/fd (opendir: Permission denied) 2024-03-21T10:28:38.419 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mgr 31777 90882 m-fin-sna ceph cwd unknown /proc/31777/task/90882/cwd (readlink: Permission denied) 2024-03-21T10:28:38.419 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mgr 31777 90882 m-fin-sna ceph rtd unknown /proc/31777/task/90882/root (readlink: Permission denied) 2024-03-21T10:28:38.419 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mgr 31777 90882 m-fin-sna ceph txt unknown /proc/31777/task/90882/exe (readlink: Permission denied) 2024-03-21T10:28:38.422 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mgr 31777 90882 m-fin-sna ceph NOFD /proc/31777/task/90882/fd (opendir: Permission denied) 2024-03-21T10:28:38.422 INFO:tasks.workunit.client.0.smithi047.stdout:systemd 32914 root cwd unknown /proc/32914/cwd (readlink: Permission denied) 2024-03-21T10:28:38.422 INFO:tasks.workunit.client.0.smithi047.stdout:systemd 32914 root rtd unknown /proc/32914/root (readlink: Permission denied) 2024-03-21T10:28:38.422 INFO:tasks.workunit.client.0.smithi047.stdout:systemd 32914 root txt unknown /proc/32914/exe (readlink: Permission denied) 2024-03-21T10:28:38.422 INFO:tasks.workunit.client.0.smithi047.stdout:systemd 32914 root NOFD /proc/32914/fd (opendir: Permission denied) 2024-03-21T10:28:38.422 INFO:tasks.workunit.client.0.smithi047.stdout:(sd-pam) 32917 root cwd unknown /proc/32917/cwd (readlink: Permission denied) 2024-03-21T10:28:38.423 INFO:tasks.workunit.client.0.smithi047.stdout:(sd-pam) 32917 root rtd unknown /proc/32917/root (readlink: Permission denied) 2024-03-21T10:28:38.423 INFO:tasks.workunit.client.0.smithi047.stdout:(sd-pam) 32917 root txt unknown /proc/32917/exe (readlink: Permission denied) 2024-03-21T10:28:38.423 INFO:tasks.workunit.client.0.smithi047.stdout:(sd-pam) 32917 root NOFD /proc/32917/fd (opendir: Permission denied) 2024-03-21T10:28:38.423 INFO:tasks.workunit.client.0.smithi047.stdout:kworker/2 36196 root cwd unknown /proc/36196/cwd (readlink: Permission denied) 2024-03-21T10:28:38.423 INFO:tasks.workunit.client.0.smithi047.stdout:kworker/2 36196 root rtd unknown /proc/36196/root (readlink: Permission denied) 2024-03-21T10:28:38.423 INFO:tasks.workunit.client.0.smithi047.stdout:kworker/2 36196 root txt unknown /proc/36196/exe (readlink: Permission denied) 2024-03-21T10:28:38.423 INFO:tasks.workunit.client.0.smithi047.stdout:kworker/2 36196 root NOFD /proc/36196/fd (opendir: Permission denied) 2024-03-21T10:28:38.423 INFO:tasks.workunit.client.0.smithi047.stdout:kworker/5 39689 root cwd unknown /proc/39689/cwd (readlink: Permission denied) 2024-03-21T10:28:38.423 INFO:tasks.workunit.client.0.smithi047.stdout:kworker/5 39689 root rtd unknown /proc/39689/root (readlink: Permission denied) 2024-03-21T10:28:38.423 INFO:tasks.workunit.client.0.smithi047.stdout:kworker/5 39689 root txt unknown /proc/39689/exe (readlink: Permission denied) 2024-03-21T10:28:38.423 INFO:tasks.workunit.client.0.smithi047.stdout:kworker/5 39689 root NOFD /proc/39689/fd (opendir: Permission denied) 2024-03-21T10:28:38.423 INFO:tasks.workunit.client.0.smithi047.stdout:conmon 44057 root cwd unknown /proc/44057/cwd (readlink: Permission denied) 2024-03-21T10:28:38.424 INFO:tasks.workunit.client.0.smithi047.stdout:conmon 44057 root rtd unknown /proc/44057/root (readlink: Permission denied) 2024-03-21T10:28:38.424 INFO:tasks.workunit.client.0.smithi047.stdout:conmon 44057 root txt unknown /proc/44057/exe (readlink: Permission denied) 2024-03-21T10:28:38.424 INFO:tasks.workunit.client.0.smithi047.stdout:conmon 44057 root NOFD /proc/44057/fd (opendir: Permission denied) 2024-03-21T10:28:38.424 INFO:tasks.workunit.client.0.smithi047.stdout:podman-in 44059 root cwd unknown /proc/44059/cwd (readlink: Permission denied) 2024-03-21T10:28:38.424 INFO:tasks.workunit.client.0.smithi047.stdout:podman-in 44059 root rtd unknown /proc/44059/root (readlink: Permission denied) 2024-03-21T10:28:38.424 INFO:tasks.workunit.client.0.smithi047.stdout:podman-in 44059 root txt unknown /proc/44059/exe (readlink: Permission denied) 2024-03-21T10:28:38.424 INFO:tasks.workunit.client.0.smithi047.stdout:podman-in 44059 root NOFD /proc/44059/fd (opendir: Permission denied) 2024-03-21T10:28:38.424 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 44061 ceph cwd unknown /proc/44061/cwd (readlink: Permission denied) 2024-03-21T10:28:38.424 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 44061 ceph rtd unknown /proc/44061/root (readlink: Permission denied) 2024-03-21T10:28:38.424 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 44061 ceph txt unknown /proc/44061/exe (readlink: Permission denied) 2024-03-21T10:28:38.424 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 44061 ceph NOFD /proc/44061/fd (opendir: Permission denied) 2024-03-21T10:28:38.425 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 44061 44062 log ceph cwd unknown /proc/44061/task/44062/cwd (readlink: Permission denied) 2024-03-21T10:28:38.425 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 44061 44062 log ceph rtd unknown /proc/44061/task/44062/root (readlink: Permission denied) 2024-03-21T10:28:38.425 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 44061 44062 log ceph txt unknown /proc/44061/task/44062/exe (readlink: Permission denied) 2024-03-21T10:28:38.425 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 44061 44062 log ceph NOFD /proc/44061/task/44062/fd (opendir: Permission denied) 2024-03-21T10:28:38.425 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 44061 44064 msgr-work ceph cwd unknown /proc/44061/task/44064/cwd (readlink: Permission denied) 2024-03-21T10:28:38.425 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 44061 44064 msgr-work ceph rtd unknown /proc/44061/task/44064/root (readlink: Permission denied) 2024-03-21T10:28:38.425 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 44061 44064 msgr-work ceph txt unknown /proc/44061/task/44064/exe (readlink: Permission denied) 2024-03-21T10:28:38.425 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 44061 44064 msgr-work ceph NOFD /proc/44061/task/44064/fd (opendir: Permission denied) 2024-03-21T10:28:38.425 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 44061 44065 msgr-work ceph cwd unknown /proc/44061/task/44065/cwd (readlink: Permission denied) 2024-03-21T10:28:38.425 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 44061 44065 msgr-work ceph rtd unknown /proc/44061/task/44065/root (readlink: Permission denied) 2024-03-21T10:28:38.425 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 44061 44065 msgr-work ceph txt unknown /proc/44061/task/44065/exe (readlink: Permission denied) 2024-03-21T10:28:38.425 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 44061 44065 msgr-work ceph NOFD /proc/44061/task/44065/fd (opendir: Permission denied) 2024-03-21T10:28:38.426 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 44061 44066 msgr-work ceph cwd unknown /proc/44061/task/44066/cwd (readlink: Permission denied) 2024-03-21T10:28:38.426 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 44061 44066 msgr-work ceph rtd unknown /proc/44061/task/44066/root (readlink: Permission denied) 2024-03-21T10:28:38.426 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 44061 44066 msgr-work ceph txt unknown /proc/44061/task/44066/exe (readlink: Permission denied) 2024-03-21T10:28:38.426 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 44061 44066 msgr-work ceph NOFD /proc/44061/task/44066/fd (opendir: Permission denied) 2024-03-21T10:28:38.426 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 44061 44070 service ceph cwd unknown /proc/44061/task/44070/cwd (readlink: Permission denied) 2024-03-21T10:28:38.426 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 44061 44070 service ceph rtd unknown /proc/44061/task/44070/root (readlink: Permission denied) 2024-03-21T10:28:38.426 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 44061 44070 service ceph txt unknown /proc/44061/task/44070/exe (readlink: Permission denied) 2024-03-21T10:28:38.426 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 44061 44070 service ceph NOFD /proc/44061/task/44070/fd (opendir: Permission denied) 2024-03-21T10:28:38.426 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 44061 44071 admin_soc ceph cwd unknown /proc/44061/task/44071/cwd (readlink: Permission denied) 2024-03-21T10:28:38.426 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 44061 44071 admin_soc ceph rtd unknown /proc/44061/task/44071/root (readlink: Permission denied) 2024-03-21T10:28:38.426 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 44061 44071 admin_soc ceph txt unknown /proc/44061/task/44071/exe (readlink: Permission denied) 2024-03-21T10:28:38.426 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 44061 44071 admin_soc ceph NOFD /proc/44061/task/44071/fd (opendir: Permission denied) 2024-03-21T10:28:38.426 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 44061 44214 signal_ha ceph cwd unknown /proc/44061/task/44214/cwd (readlink: Permission denied) 2024-03-21T10:28:38.427 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 44061 44214 signal_ha ceph rtd unknown /proc/44061/task/44214/root (readlink: Permission denied) 2024-03-21T10:28:38.427 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 44061 44214 signal_ha ceph txt unknown /proc/44061/task/44214/exe (readlink: Permission denied) 2024-03-21T10:28:38.427 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 44061 44214 signal_ha ceph NOFD /proc/44061/task/44214/fd (opendir: Permission denied) 2024-03-21T10:28:38.427 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 44061 44215 io_contex ceph cwd unknown /proc/44061/task/44215/cwd (readlink: Permission denied) 2024-03-21T10:28:38.427 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 44061 44215 io_contex ceph rtd unknown /proc/44061/task/44215/root (readlink: Permission denied) 2024-03-21T10:28:38.427 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 44061 44215 io_contex ceph txt unknown /proc/44061/task/44215/exe (readlink: Permission denied) 2024-03-21T10:28:38.427 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 44061 44215 io_contex ceph NOFD /proc/44061/task/44215/fd (opendir: Permission denied) 2024-03-21T10:28:38.427 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 44061 44216 io_contex ceph cwd unknown /proc/44061/task/44216/cwd (readlink: Permission denied) 2024-03-21T10:28:38.427 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 44061 44216 io_contex ceph rtd unknown /proc/44061/task/44216/root (readlink: Permission denied) 2024-03-21T10:28:38.427 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 44061 44216 io_contex ceph txt unknown /proc/44061/task/44216/exe (readlink: Permission denied) 2024-03-21T10:28:38.427 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 44061 44216 io_contex ceph NOFD /proc/44061/task/44216/fd (opendir: Permission denied) 2024-03-21T10:28:38.427 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 44061 44242 OpHistory ceph cwd unknown /proc/44061/task/44242/cwd (readlink: Permission denied) 2024-03-21T10:28:38.427 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 44061 44242 OpHistory ceph rtd unknown /proc/44061/task/44242/root (readlink: Permission denied) 2024-03-21T10:28:38.428 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 44061 44242 OpHistory ceph txt unknown /proc/44061/task/44242/exe (readlink: Permission denied) 2024-03-21T10:28:38.428 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 44061 44242 OpHistory ceph NOFD /proc/44061/task/44242/fd (opendir: Permission denied) 2024-03-21T10:28:38.428 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 44061 44243 ceph_time ceph cwd unknown /proc/44061/task/44243/cwd (readlink: Permission denied) 2024-03-21T10:28:38.428 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 44061 44243 ceph_time ceph rtd unknown /proc/44061/task/44243/root (readlink: Permission denied) 2024-03-21T10:28:38.428 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 44061 44243 ceph_time ceph txt unknown /proc/44061/task/44243/exe (readlink: Permission denied) 2024-03-21T10:28:38.428 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 44061 44243 ceph_time ceph NOFD /proc/44061/task/44243/fd (opendir: Permission denied) 2024-03-21T10:28:38.428 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 44061 44479 ceph-osd ceph cwd unknown /proc/44061/task/44479/cwd (readlink: Permission denied) 2024-03-21T10:28:38.428 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 44061 44479 ceph-osd ceph rtd unknown /proc/44061/task/44479/root (readlink: Permission denied) 2024-03-21T10:28:38.428 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 44061 44479 ceph-osd ceph txt unknown /proc/44061/task/44479/exe (readlink: Permission denied) 2024-03-21T10:28:38.428 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 44061 44479 ceph-osd ceph NOFD /proc/44061/task/44479/fd (opendir: Permission denied) 2024-03-21T10:28:38.428 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 44061 44480 safe_time ceph cwd unknown /proc/44061/task/44480/cwd (readlink: Permission denied) 2024-03-21T10:28:38.428 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 44061 44480 safe_time ceph rtd unknown /proc/44061/task/44480/root (readlink: Permission denied) 2024-03-21T10:28:38.428 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 44061 44480 safe_time ceph txt unknown /proc/44061/task/44480/exe (readlink: Permission denied) 2024-03-21T10:28:38.429 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 44061 44480 safe_time ceph NOFD /proc/44061/task/44480/fd (opendir: Permission denied) 2024-03-21T10:28:38.429 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 44061 44481 safe_time ceph cwd unknown /proc/44061/task/44481/cwd (readlink: Permission denied) 2024-03-21T10:28:38.429 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 44061 44481 safe_time ceph rtd unknown /proc/44061/task/44481/root (readlink: Permission denied) 2024-03-21T10:28:38.429 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 44061 44481 safe_time ceph txt unknown /proc/44061/task/44481/exe (readlink: Permission denied) 2024-03-21T10:28:38.429 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 44061 44481 safe_time ceph NOFD /proc/44061/task/44481/fd (opendir: Permission denied) 2024-03-21T10:28:38.429 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 44061 44482 safe_time ceph cwd unknown /proc/44061/task/44482/cwd (readlink: Permission denied) 2024-03-21T10:28:38.429 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 44061 44482 safe_time ceph rtd unknown /proc/44061/task/44482/root (readlink: Permission denied) 2024-03-21T10:28:38.429 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 44061 44482 safe_time ceph txt unknown /proc/44061/task/44482/exe (readlink: Permission denied) 2024-03-21T10:28:38.429 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 44061 44482 safe_time ceph NOFD /proc/44061/task/44482/fd (opendir: Permission denied) 2024-03-21T10:28:38.429 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 44061 44483 safe_time ceph cwd unknown /proc/44061/task/44483/cwd (readlink: Permission denied) 2024-03-21T10:28:38.429 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 44061 44483 safe_time ceph rtd unknown /proc/44061/task/44483/root (readlink: Permission denied) 2024-03-21T10:28:38.430 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 44061 44483 safe_time ceph txt unknown /proc/44061/task/44483/exe (readlink: Permission denied) 2024-03-21T10:28:38.430 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 44061 44483 safe_time ceph NOFD /proc/44061/task/44483/fd (opendir: Permission denied) 2024-03-21T10:28:38.430 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 44061 44484 fn_anonym ceph cwd unknown /proc/44061/task/44484/cwd (readlink: Permission denied) 2024-03-21T10:28:38.430 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 44061 44484 fn_anonym ceph rtd unknown /proc/44061/task/44484/root (readlink: Permission denied) 2024-03-21T10:28:38.430 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 44061 44484 fn_anonym ceph txt unknown /proc/44061/task/44484/exe (readlink: Permission denied) 2024-03-21T10:28:38.430 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 44061 44484 fn_anonym ceph NOFD /proc/44061/task/44484/fd (opendir: Permission denied) 2024-03-21T10:28:38.430 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 44061 44489 rocksdb:l ceph cwd unknown /proc/44061/task/44489/cwd (readlink: Permission denied) 2024-03-21T10:28:38.430 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 44061 44489 rocksdb:l ceph rtd unknown /proc/44061/task/44489/root (readlink: Permission denied) 2024-03-21T10:28:38.430 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 44061 44489 rocksdb:l ceph txt unknown /proc/44061/task/44489/exe (readlink: Permission denied) 2024-03-21T10:28:38.430 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 44061 44489 rocksdb:l ceph NOFD /proc/44061/task/44489/fd (opendir: Permission denied) 2024-03-21T10:28:38.430 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 44061 44490 rocksdb:l ceph cwd unknown /proc/44061/task/44490/cwd (readlink: Permission denied) 2024-03-21T10:28:38.431 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 44061 44490 rocksdb:l ceph rtd unknown /proc/44061/task/44490/root (readlink: Permission denied) 2024-03-21T10:28:38.431 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 44061 44490 rocksdb:l ceph txt unknown /proc/44061/task/44490/exe (readlink: Permission denied) 2024-03-21T10:28:38.431 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 44061 44490 rocksdb:l ceph NOFD /proc/44061/task/44490/fd (opendir: Permission denied) 2024-03-21T10:28:38.431 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 44061 44491 rocksdb:l ceph cwd unknown /proc/44061/task/44491/cwd (readlink: Permission denied) 2024-03-21T10:28:38.431 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 44061 44491 rocksdb:l ceph rtd unknown /proc/44061/task/44491/root (readlink: Permission denied) 2024-03-21T10:28:38.431 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 44061 44491 rocksdb:l ceph txt unknown /proc/44061/task/44491/exe (readlink: Permission denied) 2024-03-21T10:28:38.431 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 44061 44491 rocksdb:l ceph NOFD /proc/44061/task/44491/fd (opendir: Permission denied) 2024-03-21T10:28:38.431 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 44061 44492 rocksdb:h ceph cwd unknown /proc/44061/task/44492/cwd (readlink: Permission denied) 2024-03-21T10:28:38.431 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 44061 44492 rocksdb:h ceph rtd unknown /proc/44061/task/44492/root (readlink: Permission denied) 2024-03-21T10:28:38.431 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 44061 44492 rocksdb:h ceph txt unknown /proc/44061/task/44492/exe (readlink: Permission denied) 2024-03-21T10:28:38.431 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 44061 44492 rocksdb:h ceph NOFD /proc/44061/task/44492/fd (opendir: Permission denied) 2024-03-21T10:28:38.431 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 44061 44902 bstore_ai ceph cwd unknown /proc/44061/task/44902/cwd (readlink: Permission denied) 2024-03-21T10:28:38.431 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 44061 44902 bstore_ai ceph rtd unknown /proc/44061/task/44902/root (readlink: Permission denied) 2024-03-21T10:28:38.432 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 44061 44902 bstore_ai ceph txt unknown /proc/44061/task/44902/exe (readlink: Permission denied) 2024-03-21T10:28:38.432 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 44061 44902 bstore_ai ceph NOFD /proc/44061/task/44902/fd (opendir: Permission denied) 2024-03-21T10:28:38.432 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 44061 45102 bstore_ai ceph cwd unknown /proc/44061/task/45102/cwd (readlink: Permission denied) 2024-03-21T10:28:38.432 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 44061 45102 bstore_ai ceph rtd unknown /proc/44061/task/45102/root (readlink: Permission denied) 2024-03-21T10:28:38.432 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 44061 45102 bstore_ai ceph txt unknown /proc/44061/task/45102/exe (readlink: Permission denied) 2024-03-21T10:28:38.432 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 44061 45102 bstore_ai ceph NOFD /proc/44061/task/45102/fd (opendir: Permission denied) 2024-03-21T10:28:38.432 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 44061 45288 ceph-osd ceph cwd unknown /proc/44061/task/45288/cwd (readlink: Permission denied) 2024-03-21T10:28:38.432 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 44061 45288 ceph-osd ceph rtd unknown /proc/44061/task/45288/root (readlink: Permission denied) 2024-03-21T10:28:38.432 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 44061 45288 ceph-osd ceph txt unknown /proc/44061/task/45288/exe (readlink: Permission denied) 2024-03-21T10:28:38.432 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 44061 45288 ceph-osd ceph NOFD /proc/44061/task/45288/fd (opendir: Permission denied) 2024-03-21T10:28:38.432 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 44061 45289 cfin ceph cwd unknown /proc/44061/task/45289/cwd (readlink: Permission denied) 2024-03-21T10:28:38.433 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 44061 45289 cfin ceph rtd unknown /proc/44061/task/45289/root (readlink: Permission denied) 2024-03-21T10:28:38.433 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 44061 45289 cfin ceph txt unknown /proc/44061/task/45289/exe (readlink: Permission denied) 2024-03-21T10:28:38.433 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 44061 45289 cfin ceph NOFD /proc/44061/task/45289/fd (opendir: Permission denied) 2024-03-21T10:28:38.433 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 44061 45290 bstore_kv ceph cwd unknown /proc/44061/task/45290/cwd (readlink: Permission denied) 2024-03-21T10:28:38.433 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 44061 45290 bstore_kv ceph rtd unknown /proc/44061/task/45290/root (readlink: Permission denied) 2024-03-21T10:28:38.433 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 44061 45290 bstore_kv ceph txt unknown /proc/44061/task/45290/exe (readlink: Permission denied) 2024-03-21T10:28:38.433 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 44061 45290 bstore_kv ceph NOFD /proc/44061/task/45290/fd (opendir: Permission denied) 2024-03-21T10:28:38.433 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 44061 45291 bstore_kv ceph cwd unknown /proc/44061/task/45291/cwd (readlink: Permission denied) 2024-03-21T10:28:38.433 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 44061 45291 bstore_kv ceph rtd unknown /proc/44061/task/45291/root (readlink: Permission denied) 2024-03-21T10:28:38.433 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 44061 45291 bstore_kv ceph txt unknown /proc/44061/task/45291/exe (readlink: Permission denied) 2024-03-21T10:28:38.433 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 44061 45291 bstore_kv ceph NOFD /proc/44061/task/45291/fd (opendir: Permission denied) 2024-03-21T10:28:38.433 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 44061 45292 bstore_me ceph cwd unknown /proc/44061/task/45292/cwd (readlink: Permission denied) 2024-03-21T10:28:38.433 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 44061 45292 bstore_me ceph rtd unknown /proc/44061/task/45292/root (readlink: Permission denied) 2024-03-21T10:28:38.434 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 44061 45292 bstore_me ceph txt unknown /proc/44061/task/45292/exe (readlink: Permission denied) 2024-03-21T10:28:38.434 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 44061 45292 bstore_me ceph NOFD /proc/44061/task/45292/fd (opendir: Permission denied) 2024-03-21T10:28:38.434 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 44061 45296 ms_dispat ceph cwd unknown /proc/44061/task/45296/cwd (readlink: Permission denied) 2024-03-21T10:28:38.434 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 44061 45296 ms_dispat ceph rtd unknown /proc/44061/task/45296/root (readlink: Permission denied) 2024-03-21T10:28:38.434 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 44061 45296 ms_dispat ceph txt unknown /proc/44061/task/45296/exe (readlink: Permission denied) 2024-03-21T10:28:38.434 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 44061 45296 ms_dispat ceph NOFD /proc/44061/task/45296/fd (opendir: Permission denied) 2024-03-21T10:28:38.434 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 44061 45297 ms_local ceph cwd unknown /proc/44061/task/45297/cwd (readlink: Permission denied) 2024-03-21T10:28:38.434 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 44061 45297 ms_local ceph rtd unknown /proc/44061/task/45297/root (readlink: Permission denied) 2024-03-21T10:28:38.434 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 44061 45297 ms_local ceph txt unknown /proc/44061/task/45297/exe (readlink: Permission denied) 2024-03-21T10:28:38.434 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 44061 45297 ms_local ceph NOFD /proc/44061/task/45297/fd (opendir: Permission denied) 2024-03-21T10:28:38.434 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 44061 45298 safe_time ceph cwd unknown /proc/44061/task/45298/cwd (readlink: Permission denied) 2024-03-21T10:28:38.434 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 44061 45298 safe_time ceph rtd unknown /proc/44061/task/45298/root (readlink: Permission denied) 2024-03-21T10:28:38.435 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 44061 45298 safe_time ceph txt unknown /proc/44061/task/45298/exe (readlink: Permission denied) 2024-03-21T10:28:38.435 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 44061 45298 safe_time ceph NOFD /proc/44061/task/45298/fd (opendir: Permission denied) 2024-03-21T10:28:38.435 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 44061 45299 safe_time ceph cwd unknown /proc/44061/task/45299/cwd (readlink: Permission denied) 2024-03-21T10:28:38.435 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 44061 45299 safe_time ceph rtd unknown /proc/44061/task/45299/root (readlink: Permission denied) 2024-03-21T10:28:38.435 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 44061 45299 safe_time ceph txt unknown /proc/44061/task/45299/exe (readlink: Permission denied) 2024-03-21T10:28:38.435 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 44061 45299 safe_time ceph NOFD /proc/44061/task/45299/fd (opendir: Permission denied) 2024-03-21T10:28:38.435 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 44061 45300 ms_dispat ceph cwd unknown /proc/44061/task/45300/cwd (readlink: Permission denied) 2024-03-21T10:28:38.435 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 44061 45300 ms_dispat ceph rtd unknown /proc/44061/task/45300/root (readlink: Permission denied) 2024-03-21T10:28:38.435 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 44061 45300 ms_dispat ceph txt unknown /proc/44061/task/45300/exe (readlink: Permission denied) 2024-03-21T10:28:38.435 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 44061 45300 ms_dispat ceph NOFD /proc/44061/task/45300/fd (opendir: Permission denied) 2024-03-21T10:28:38.435 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 44061 45301 ms_local ceph cwd unknown /proc/44061/task/45301/cwd (readlink: Permission denied) 2024-03-21T10:28:38.435 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 44061 45301 ms_local ceph rtd unknown /proc/44061/task/45301/root (readlink: Permission denied) 2024-03-21T10:28:38.435 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 44061 45301 ms_local ceph txt unknown /proc/44061/task/45301/exe (readlink: Permission denied) 2024-03-21T10:28:38.436 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 44061 45301 ms_local ceph NOFD /proc/44061/task/45301/fd (opendir: Permission denied) 2024-03-21T10:28:38.436 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 44061 45302 ms_dispat ceph cwd unknown /proc/44061/task/45302/cwd (readlink: Permission denied) 2024-03-21T10:28:38.436 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 44061 45302 ms_dispat ceph rtd unknown /proc/44061/task/45302/root (readlink: Permission denied) 2024-03-21T10:28:38.436 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 44061 45302 ms_dispat ceph txt unknown /proc/44061/task/45302/exe (readlink: Permission denied) 2024-03-21T10:28:38.436 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 44061 45302 ms_dispat ceph NOFD /proc/44061/task/45302/fd (opendir: Permission denied) 2024-03-21T10:28:38.436 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 44061 45303 ms_local ceph cwd unknown /proc/44061/task/45303/cwd (readlink: Permission denied) 2024-03-21T10:28:38.436 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 44061 45303 ms_local ceph rtd unknown /proc/44061/task/45303/root (readlink: Permission denied) 2024-03-21T10:28:38.436 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 44061 45303 ms_local ceph txt unknown /proc/44061/task/45303/exe (readlink: Permission denied) 2024-03-21T10:28:38.436 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 44061 45303 ms_local ceph NOFD /proc/44061/task/45303/fd (opendir: Permission denied) 2024-03-21T10:28:38.436 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 44061 45304 ms_dispat ceph cwd unknown /proc/44061/task/45304/cwd (readlink: Permission denied) 2024-03-21T10:28:38.436 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 44061 45304 ms_dispat ceph rtd unknown /proc/44061/task/45304/root (readlink: Permission denied) 2024-03-21T10:28:38.436 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 44061 45304 ms_dispat ceph txt unknown /proc/44061/task/45304/exe (readlink: Permission denied) 2024-03-21T10:28:38.436 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 44061 45304 ms_dispat ceph NOFD /proc/44061/task/45304/fd (opendir: Permission denied) 2024-03-21T10:28:38.436 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 44061 45305 ms_local ceph cwd unknown /proc/44061/task/45305/cwd (readlink: Permission denied) 2024-03-21T10:28:38.437 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 44061 45305 ms_local ceph rtd unknown /proc/44061/task/45305/root (readlink: Permission denied) 2024-03-21T10:28:38.437 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 44061 45305 ms_local ceph txt unknown /proc/44061/task/45305/exe (readlink: Permission denied) 2024-03-21T10:28:38.437 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 44061 45305 ms_local ceph NOFD /proc/44061/task/45305/fd (opendir: Permission denied) 2024-03-21T10:28:38.437 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 44061 45306 ms_dispat ceph cwd unknown /proc/44061/task/45306/cwd (readlink: Permission denied) 2024-03-21T10:28:38.437 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 44061 45306 ms_dispat ceph rtd unknown /proc/44061/task/45306/root (readlink: Permission denied) 2024-03-21T10:28:38.437 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 44061 45306 ms_dispat ceph txt unknown /proc/44061/task/45306/exe (readlink: Permission denied) 2024-03-21T10:28:38.437 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 44061 45306 ms_dispat ceph NOFD /proc/44061/task/45306/fd (opendir: Permission denied) 2024-03-21T10:28:38.437 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 44061 45307 ms_local ceph cwd unknown /proc/44061/task/45307/cwd (readlink: Permission denied) 2024-03-21T10:28:38.437 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 44061 45307 ms_local ceph rtd unknown /proc/44061/task/45307/root (readlink: Permission denied) 2024-03-21T10:28:38.437 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 44061 45307 ms_local ceph txt unknown /proc/44061/task/45307/exe (readlink: Permission denied) 2024-03-21T10:28:38.437 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 44061 45307 ms_local ceph NOFD /proc/44061/task/45307/fd (opendir: Permission denied) 2024-03-21T10:28:38.437 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 44061 45308 ms_dispat ceph cwd unknown /proc/44061/task/45308/cwd (readlink: Permission denied) 2024-03-21T10:28:38.437 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 44061 45308 ms_dispat ceph rtd unknown /proc/44061/task/45308/root (readlink: Permission denied) 2024-03-21T10:28:38.437 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 44061 45308 ms_dispat ceph txt unknown /proc/44061/task/45308/exe (readlink: Permission denied) 2024-03-21T10:28:38.438 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 44061 45308 ms_dispat ceph NOFD /proc/44061/task/45308/fd (opendir: Permission denied) 2024-03-21T10:28:38.438 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 44061 45309 ms_local ceph cwd unknown /proc/44061/task/45309/cwd (readlink: Permission denied) 2024-03-21T10:28:38.438 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 44061 45309 ms_local ceph rtd unknown /proc/44061/task/45309/root (readlink: Permission denied) 2024-03-21T10:28:38.438 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 44061 45309 ms_local ceph txt unknown /proc/44061/task/45309/exe (readlink: Permission denied) 2024-03-21T10:28:38.438 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 44061 45309 ms_local ceph NOFD /proc/44061/task/45309/fd (opendir: Permission denied) 2024-03-21T10:28:38.438 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 44061 45310 ms_dispat ceph cwd unknown /proc/44061/task/45310/cwd (readlink: Permission denied) 2024-03-21T10:28:38.438 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 44061 45310 ms_dispat ceph rtd unknown /proc/44061/task/45310/root (readlink: Permission denied) 2024-03-21T10:28:38.438 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 44061 45310 ms_dispat ceph txt unknown /proc/44061/task/45310/exe (readlink: Permission denied) 2024-03-21T10:28:38.438 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 44061 45310 ms_dispat ceph NOFD /proc/44061/task/45310/fd (opendir: Permission denied) 2024-03-21T10:28:38.438 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 44061 45311 ms_local ceph cwd unknown /proc/44061/task/45311/cwd (readlink: Permission denied) 2024-03-21T10:28:38.438 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 44061 45311 ms_local ceph rtd unknown /proc/44061/task/45311/root (readlink: Permission denied) 2024-03-21T10:28:38.438 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 44061 45311 ms_local ceph txt unknown /proc/44061/task/45311/exe (readlink: Permission denied) 2024-03-21T10:28:38.438 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 44061 45311 ms_local ceph NOFD /proc/44061/task/45311/fd (opendir: Permission denied) 2024-03-21T10:28:38.438 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 44061 45312 fn_anonym ceph cwd unknown /proc/44061/task/45312/cwd (readlink: Permission denied) 2024-03-21T10:28:38.439 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 44061 45312 fn_anonym ceph rtd unknown /proc/44061/task/45312/root (readlink: Permission denied) 2024-03-21T10:28:38.439 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 44061 45312 fn_anonym ceph txt unknown /proc/44061/task/45312/exe (readlink: Permission denied) 2024-03-21T10:28:38.439 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 44061 45312 fn_anonym ceph NOFD /proc/44061/task/45312/fd (opendir: Permission denied) 2024-03-21T10:28:38.439 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 44061 45313 finisher ceph cwd unknown /proc/44061/task/45313/cwd (readlink: Permission denied) 2024-03-21T10:28:38.439 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 44061 45313 finisher ceph rtd unknown /proc/44061/task/45313/root (readlink: Permission denied) 2024-03-21T10:28:38.439 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 44061 45313 finisher ceph txt unknown /proc/44061/task/45313/exe (readlink: Permission denied) 2024-03-21T10:28:38.439 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 44061 45313 finisher ceph NOFD /proc/44061/task/45313/fd (opendir: Permission denied) 2024-03-21T10:28:38.439 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 44061 45314 safe_time ceph cwd unknown /proc/44061/task/45314/cwd (readlink: Permission denied) 2024-03-21T10:28:38.439 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 44061 45314 safe_time ceph rtd unknown /proc/44061/task/45314/root (readlink: Permission denied) 2024-03-21T10:28:38.439 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 44061 45314 safe_time ceph txt unknown /proc/44061/task/45314/exe (readlink: Permission denied) 2024-03-21T10:28:38.439 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 44061 45314 safe_time ceph NOFD /proc/44061/task/45314/fd (opendir: Permission denied) 2024-03-21T10:28:38.439 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 44061 45315 safe_time ceph cwd unknown /proc/44061/task/45315/cwd (readlink: Permission denied) 2024-03-21T10:28:38.439 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 44061 45315 safe_time ceph rtd unknown /proc/44061/task/45315/root (readlink: Permission denied) 2024-03-21T10:28:38.439 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 44061 45315 safe_time ceph txt unknown /proc/44061/task/45315/exe (readlink: Permission denied) 2024-03-21T10:28:38.439 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 44061 45315 safe_time ceph NOFD /proc/44061/task/45315/fd (opendir: Permission denied) 2024-03-21T10:28:38.440 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 44061 45316 ceph-osd ceph cwd unknown /proc/44061/task/45316/cwd (readlink: Permission denied) 2024-03-21T10:28:38.440 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 44061 45316 ceph-osd ceph rtd unknown /proc/44061/task/45316/root (readlink: Permission denied) 2024-03-21T10:28:38.440 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 44061 45316 ceph-osd ceph txt unknown /proc/44061/task/45316/exe (readlink: Permission denied) 2024-03-21T10:28:38.440 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 44061 45316 ceph-osd ceph NOFD /proc/44061/task/45316/fd (opendir: Permission denied) 2024-03-21T10:28:38.440 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 44061 45317 osd_srv_a ceph cwd unknown /proc/44061/task/45317/cwd (readlink: Permission denied) 2024-03-21T10:28:38.440 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 44061 45317 osd_srv_a ceph rtd unknown /proc/44061/task/45317/root (readlink: Permission denied) 2024-03-21T10:28:38.440 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 44061 45317 osd_srv_a ceph txt unknown /proc/44061/task/45317/exe (readlink: Permission denied) 2024-03-21T10:28:38.440 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 44061 45317 osd_srv_a ceph NOFD /proc/44061/task/45317/fd (opendir: Permission denied) 2024-03-21T10:28:38.440 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 44061 45318 tp_osd_tp ceph cwd unknown /proc/44061/task/45318/cwd (readlink: Permission denied) 2024-03-21T10:28:38.440 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 44061 45318 tp_osd_tp ceph rtd unknown /proc/44061/task/45318/root (readlink: Permission denied) 2024-03-21T10:28:38.440 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 44061 45318 tp_osd_tp ceph txt unknown /proc/44061/task/45318/exe (readlink: Permission denied) 2024-03-21T10:28:38.440 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 44061 45318 tp_osd_tp ceph NOFD /proc/44061/task/45318/fd (opendir: Permission denied) 2024-03-21T10:28:38.440 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 44061 45319 tp_osd_tp ceph cwd unknown /proc/44061/task/45319/cwd (readlink: Permission denied) 2024-03-21T10:28:38.440 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 44061 45319 tp_osd_tp ceph rtd unknown /proc/44061/task/45319/root (readlink: Permission denied) 2024-03-21T10:28:38.441 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 44061 45319 tp_osd_tp ceph txt unknown /proc/44061/task/45319/exe (readlink: Permission denied) 2024-03-21T10:28:38.441 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 44061 45319 tp_osd_tp ceph NOFD /proc/44061/task/45319/fd (opendir: Permission denied) 2024-03-21T10:28:38.441 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 44061 45320 tp_osd_tp ceph cwd unknown /proc/44061/task/45320/cwd (readlink: Permission denied) 2024-03-21T10:28:38.441 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 44061 45320 tp_osd_tp ceph rtd unknown /proc/44061/task/45320/root (readlink: Permission denied) 2024-03-21T10:28:38.441 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 44061 45320 tp_osd_tp ceph txt unknown /proc/44061/task/45320/exe (readlink: Permission denied) 2024-03-21T10:28:38.441 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 44061 45320 tp_osd_tp ceph NOFD /proc/44061/task/45320/fd (opendir: Permission denied) 2024-03-21T10:28:38.441 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 44061 45321 tp_osd_tp ceph cwd unknown /proc/44061/task/45321/cwd (readlink: Permission denied) 2024-03-21T10:28:38.441 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 44061 45321 tp_osd_tp ceph rtd unknown /proc/44061/task/45321/root (readlink: Permission denied) 2024-03-21T10:28:38.441 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 44061 45321 tp_osd_tp ceph txt unknown /proc/44061/task/45321/exe (readlink: Permission denied) 2024-03-21T10:28:38.441 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 44061 45321 tp_osd_tp ceph NOFD /proc/44061/task/45321/fd (opendir: Permission denied) 2024-03-21T10:28:38.441 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 44061 45322 tp_osd_tp ceph cwd unknown /proc/44061/task/45322/cwd (readlink: Permission denied) 2024-03-21T10:28:38.441 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 44061 45322 tp_osd_tp ceph rtd unknown /proc/44061/task/45322/root (readlink: Permission denied) 2024-03-21T10:28:38.441 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 44061 45322 tp_osd_tp ceph txt unknown /proc/44061/task/45322/exe (readlink: Permission denied) 2024-03-21T10:28:38.441 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 44061 45322 tp_osd_tp ceph NOFD /proc/44061/task/45322/fd (opendir: Permission denied) 2024-03-21T10:28:38.442 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 44061 45323 tp_osd_tp ceph cwd unknown /proc/44061/task/45323/cwd (readlink: Permission denied) 2024-03-21T10:28:38.442 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 44061 45323 tp_osd_tp ceph rtd unknown /proc/44061/task/45323/root (readlink: Permission denied) 2024-03-21T10:28:38.442 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 44061 45323 tp_osd_tp ceph txt unknown /proc/44061/task/45323/exe (readlink: Permission denied) 2024-03-21T10:28:38.442 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 44061 45323 tp_osd_tp ceph NOFD /proc/44061/task/45323/fd (opendir: Permission denied) 2024-03-21T10:28:38.442 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 44061 45324 tp_osd_tp ceph cwd unknown /proc/44061/task/45324/cwd (readlink: Permission denied) 2024-03-21T10:28:38.442 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 44061 45324 tp_osd_tp ceph rtd unknown /proc/44061/task/45324/root (readlink: Permission denied) 2024-03-21T10:28:38.442 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 44061 45324 tp_osd_tp ceph txt unknown /proc/44061/task/45324/exe (readlink: Permission denied) 2024-03-21T10:28:38.442 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 44061 45324 tp_osd_tp ceph NOFD /proc/44061/task/45324/fd (opendir: Permission denied) 2024-03-21T10:28:38.442 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 44061 45325 tp_osd_tp ceph cwd unknown /proc/44061/task/45325/cwd (readlink: Permission denied) 2024-03-21T10:28:38.442 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 44061 45325 tp_osd_tp ceph rtd unknown /proc/44061/task/45325/root (readlink: Permission denied) 2024-03-21T10:28:38.442 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 44061 45325 tp_osd_tp ceph txt unknown /proc/44061/task/45325/exe (readlink: Permission denied) 2024-03-21T10:28:38.442 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 44061 45325 tp_osd_tp ceph NOFD /proc/44061/task/45325/fd (opendir: Permission denied) 2024-03-21T10:28:38.442 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 44061 45326 tp_osd_tp ceph cwd unknown /proc/44061/task/45326/cwd (readlink: Permission denied) 2024-03-21T10:28:38.442 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 44061 45326 tp_osd_tp ceph rtd unknown /proc/44061/task/45326/root (readlink: Permission denied) 2024-03-21T10:28:38.443 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 44061 45326 tp_osd_tp ceph txt unknown /proc/44061/task/45326/exe (readlink: Permission denied) 2024-03-21T10:28:38.443 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 44061 45326 tp_osd_tp ceph NOFD /proc/44061/task/45326/fd (opendir: Permission denied) 2024-03-21T10:28:38.443 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 44061 45327 tp_osd_tp ceph cwd unknown /proc/44061/task/45327/cwd (readlink: Permission denied) 2024-03-21T10:28:38.443 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 44061 45327 tp_osd_tp ceph rtd unknown /proc/44061/task/45327/root (readlink: Permission denied) 2024-03-21T10:28:38.443 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 44061 45327 tp_osd_tp ceph txt unknown /proc/44061/task/45327/exe (readlink: Permission denied) 2024-03-21T10:28:38.443 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 44061 45327 tp_osd_tp ceph NOFD /proc/44061/task/45327/fd (opendir: Permission denied) 2024-03-21T10:28:38.443 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 44061 45328 tp_osd_tp ceph cwd unknown /proc/44061/task/45328/cwd (readlink: Permission denied) 2024-03-21T10:28:38.443 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 44061 45328 tp_osd_tp ceph rtd unknown /proc/44061/task/45328/root (readlink: Permission denied) 2024-03-21T10:28:38.443 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 44061 45328 tp_osd_tp ceph txt unknown /proc/44061/task/45328/exe (readlink: Permission denied) 2024-03-21T10:28:38.443 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 44061 45328 tp_osd_tp ceph NOFD /proc/44061/task/45328/fd (opendir: Permission denied) 2024-03-21T10:28:38.443 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 44061 45329 tp_osd_tp ceph cwd unknown /proc/44061/task/45329/cwd (readlink: Permission denied) 2024-03-21T10:28:38.443 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 44061 45329 tp_osd_tp ceph rtd unknown /proc/44061/task/45329/root (readlink: Permission denied) 2024-03-21T10:28:38.443 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 44061 45329 tp_osd_tp ceph txt unknown /proc/44061/task/45329/exe (readlink: Permission denied) 2024-03-21T10:28:38.444 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 44061 45329 tp_osd_tp ceph NOFD /proc/44061/task/45329/fd (opendir: Permission denied) 2024-03-21T10:28:38.444 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 44061 45330 tp_osd_tp ceph cwd unknown /proc/44061/task/45330/cwd (readlink: Permission denied) 2024-03-21T10:28:38.444 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 44061 45330 tp_osd_tp ceph rtd unknown /proc/44061/task/45330/root (readlink: Permission denied) 2024-03-21T10:28:38.444 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 44061 45330 tp_osd_tp ceph txt unknown /proc/44061/task/45330/exe (readlink: Permission denied) 2024-03-21T10:28:38.444 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 44061 45330 tp_osd_tp ceph NOFD /proc/44061/task/45330/fd (opendir: Permission denied) 2024-03-21T10:28:38.444 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 44061 45331 tp_osd_tp ceph cwd unknown /proc/44061/task/45331/cwd (readlink: Permission denied) 2024-03-21T10:28:38.444 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 44061 45331 tp_osd_tp ceph rtd unknown /proc/44061/task/45331/root (readlink: Permission denied) 2024-03-21T10:28:38.444 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 44061 45331 tp_osd_tp ceph txt unknown /proc/44061/task/45331/exe (readlink: Permission denied) 2024-03-21T10:28:38.444 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 44061 45331 tp_osd_tp ceph NOFD /proc/44061/task/45331/fd (opendir: Permission denied) 2024-03-21T10:28:38.444 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 44061 45332 tp_osd_tp ceph cwd unknown /proc/44061/task/45332/cwd (readlink: Permission denied) 2024-03-21T10:28:38.444 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 44061 45332 tp_osd_tp ceph rtd unknown /proc/44061/task/45332/root (readlink: Permission denied) 2024-03-21T10:28:38.444 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 44061 45332 tp_osd_tp ceph txt unknown /proc/44061/task/45332/exe (readlink: Permission denied) 2024-03-21T10:28:38.445 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 44061 45332 tp_osd_tp ceph NOFD /proc/44061/task/45332/fd (opendir: Permission denied) 2024-03-21T10:28:38.445 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 44061 45333 tp_osd_tp ceph cwd unknown /proc/44061/task/45333/cwd (readlink: Permission denied) 2024-03-21T10:28:38.445 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 44061 45333 tp_osd_tp ceph rtd unknown /proc/44061/task/45333/root (readlink: Permission denied) 2024-03-21T10:28:38.445 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 44061 45333 tp_osd_tp ceph txt unknown /proc/44061/task/45333/exe (readlink: Permission denied) 2024-03-21T10:28:38.445 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 44061 45333 tp_osd_tp ceph NOFD /proc/44061/task/45333/fd (opendir: Permission denied) 2024-03-21T10:28:38.445 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 44061 45334 osd_srv_h ceph cwd unknown /proc/44061/task/45334/cwd (readlink: Permission denied) 2024-03-21T10:28:38.445 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 44061 45334 osd_srv_h ceph rtd unknown /proc/44061/task/45334/root (readlink: Permission denied) 2024-03-21T10:28:38.445 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 44061 45334 osd_srv_h ceph txt unknown /proc/44061/task/45334/exe (readlink: Permission denied) 2024-03-21T10:28:38.445 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 44061 45334 osd_srv_h ceph NOFD /proc/44061/task/45334/fd (opendir: Permission denied) 2024-03-21T10:28:38.445 INFO:tasks.workunit.client.0.smithi047.stdout:kworker/1 45346 root cwd unknown /proc/45346/cwd (readlink: Permission denied) 2024-03-21T10:28:38.445 INFO:tasks.workunit.client.0.smithi047.stdout:kworker/1 45346 root rtd unknown /proc/45346/root (readlink: Permission denied) 2024-03-21T10:28:38.445 INFO:tasks.workunit.client.0.smithi047.stdout:kworker/1 45346 root txt unknown /proc/45346/exe (readlink: Permission denied) 2024-03-21T10:28:38.445 INFO:tasks.workunit.client.0.smithi047.stdout:kworker/1 45346 root NOFD /proc/45346/fd (opendir: Permission denied) 2024-03-21T10:28:38.446 INFO:tasks.workunit.client.0.smithi047.stdout:sudo 45455 root cwd unknown /proc/45455/cwd (readlink: Permission denied) 2024-03-21T10:28:38.446 INFO:tasks.workunit.client.0.smithi047.stdout:sudo 45455 root rtd unknown /proc/45455/root (readlink: Permission denied) 2024-03-21T10:28:38.446 INFO:tasks.workunit.client.0.smithi047.stdout:sudo 45455 root txt unknown /proc/45455/exe (readlink: Permission denied) 2024-03-21T10:28:38.446 INFO:tasks.workunit.client.0.smithi047.stdout:sudo 45455 root NOFD /proc/45455/fd (opendir: Permission denied) 2024-03-21T10:28:38.446 INFO:tasks.workunit.client.0.smithi047.stdout:journalct 45487 root cwd unknown /proc/45487/cwd (readlink: Permission denied) 2024-03-21T10:28:38.446 INFO:tasks.workunit.client.0.smithi047.stdout:journalct 45487 root rtd unknown /proc/45487/root (readlink: Permission denied) 2024-03-21T10:28:38.446 INFO:tasks.workunit.client.0.smithi047.stdout:journalct 45487 root txt unknown /proc/45487/exe (readlink: Permission denied) 2024-03-21T10:28:38.446 INFO:tasks.workunit.client.0.smithi047.stdout:journalct 45487 root NOFD /proc/45487/fd (opendir: Permission denied) 2024-03-21T10:28:38.446 INFO:tasks.workunit.client.0.smithi047.stdout:kworker/u 48418 root cwd unknown /proc/48418/cwd (readlink: Permission denied) 2024-03-21T10:28:38.446 INFO:tasks.workunit.client.0.smithi047.stdout:kworker/u 48418 root rtd unknown /proc/48418/root (readlink: Permission denied) 2024-03-21T10:28:38.446 INFO:tasks.workunit.client.0.smithi047.stdout:kworker/u 48418 root txt unknown /proc/48418/exe (readlink: Permission denied) 2024-03-21T10:28:38.446 INFO:tasks.workunit.client.0.smithi047.stdout:kworker/u 48418 root NOFD /proc/48418/fd (opendir: Permission denied) 2024-03-21T10:28:38.446 INFO:tasks.workunit.client.0.smithi047.stdout:conmon 50720 root cwd unknown /proc/50720/cwd (readlink: Permission denied) 2024-03-21T10:28:38.446 INFO:tasks.workunit.client.0.smithi047.stdout:conmon 50720 root rtd unknown /proc/50720/root (readlink: Permission denied) 2024-03-21T10:28:38.447 INFO:tasks.workunit.client.0.smithi047.stdout:conmon 50720 root txt unknown /proc/50720/exe (readlink: Permission denied) 2024-03-21T10:28:38.447 INFO:tasks.workunit.client.0.smithi047.stdout:conmon 50720 root NOFD /proc/50720/fd (opendir: Permission denied) 2024-03-21T10:28:38.447 INFO:tasks.workunit.client.0.smithi047.stdout:podman-in 50722 root cwd unknown /proc/50722/cwd (readlink: Permission denied) 2024-03-21T10:28:38.447 INFO:tasks.workunit.client.0.smithi047.stdout:podman-in 50722 root rtd unknown /proc/50722/root (readlink: Permission denied) 2024-03-21T10:28:38.447 INFO:tasks.workunit.client.0.smithi047.stdout:podman-in 50722 root txt unknown /proc/50722/exe (readlink: Permission denied) 2024-03-21T10:28:38.448 INFO:tasks.workunit.client.0.smithi047.stdout:podman-in 50722 root NOFD /proc/50722/fd (opendir: Permission denied) 2024-03-21T10:28:38.448 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 50724 ceph cwd unknown /proc/50724/cwd (readlink: Permission denied) 2024-03-21T10:28:38.448 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 50724 ceph rtd unknown /proc/50724/root (readlink: Permission denied) 2024-03-21T10:28:38.448 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 50724 ceph txt unknown /proc/50724/exe (readlink: Permission denied) 2024-03-21T10:28:38.448 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 50724 ceph NOFD /proc/50724/fd (opendir: Permission denied) 2024-03-21T10:28:38.448 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 50724 50725 log ceph cwd unknown /proc/50724/task/50725/cwd (readlink: Permission denied) 2024-03-21T10:28:38.448 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 50724 50725 log ceph rtd unknown /proc/50724/task/50725/root (readlink: Permission denied) 2024-03-21T10:28:38.448 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 50724 50725 log ceph txt unknown /proc/50724/task/50725/exe (readlink: Permission denied) 2024-03-21T10:28:38.449 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 50724 50725 log ceph NOFD /proc/50724/task/50725/fd (opendir: Permission denied) 2024-03-21T10:28:38.449 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 50724 50727 msgr-work ceph cwd unknown /proc/50724/task/50727/cwd (readlink: Permission denied) 2024-03-21T10:28:38.449 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 50724 50727 msgr-work ceph rtd unknown /proc/50724/task/50727/root (readlink: Permission denied) 2024-03-21T10:28:38.449 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 50724 50727 msgr-work ceph txt unknown /proc/50724/task/50727/exe (readlink: Permission denied) 2024-03-21T10:28:38.449 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 50724 50727 msgr-work ceph NOFD /proc/50724/task/50727/fd (opendir: Permission denied) 2024-03-21T10:28:38.449 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 50724 50728 msgr-work ceph cwd unknown /proc/50724/task/50728/cwd (readlink: Permission denied) 2024-03-21T10:28:38.449 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 50724 50728 msgr-work ceph rtd unknown /proc/50724/task/50728/root (readlink: Permission denied) 2024-03-21T10:28:38.449 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 50724 50728 msgr-work ceph txt unknown /proc/50724/task/50728/exe (readlink: Permission denied) 2024-03-21T10:28:38.449 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 50724 50728 msgr-work ceph NOFD /proc/50724/task/50728/fd (opendir: Permission denied) 2024-03-21T10:28:38.449 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 50724 50729 msgr-work ceph cwd unknown /proc/50724/task/50729/cwd (readlink: Permission denied) 2024-03-21T10:28:38.449 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 50724 50729 msgr-work ceph rtd unknown /proc/50724/task/50729/root (readlink: Permission denied) 2024-03-21T10:28:38.449 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 50724 50729 msgr-work ceph txt unknown /proc/50724/task/50729/exe (readlink: Permission denied) 2024-03-21T10:28:38.449 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 50724 50729 msgr-work ceph NOFD /proc/50724/task/50729/fd (opendir: Permission denied) 2024-03-21T10:28:38.449 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 50724 50733 service ceph cwd unknown /proc/50724/task/50733/cwd (readlink: Permission denied) 2024-03-21T10:28:38.450 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 50724 50733 service ceph rtd unknown /proc/50724/task/50733/root (readlink: Permission denied) 2024-03-21T10:28:38.450 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 50724 50733 service ceph txt unknown /proc/50724/task/50733/exe (readlink: Permission denied) 2024-03-21T10:28:38.450 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 50724 50733 service ceph NOFD /proc/50724/task/50733/fd (opendir: Permission denied) 2024-03-21T10:28:38.450 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 50724 50734 admin_soc ceph cwd unknown /proc/50724/task/50734/cwd (readlink: Permission denied) 2024-03-21T10:28:38.450 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 50724 50734 admin_soc ceph rtd unknown /proc/50724/task/50734/root (readlink: Permission denied) 2024-03-21T10:28:38.450 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 50724 50734 admin_soc ceph txt unknown /proc/50724/task/50734/exe (readlink: Permission denied) 2024-03-21T10:28:38.450 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 50724 50734 admin_soc ceph NOFD /proc/50724/task/50734/fd (opendir: Permission denied) 2024-03-21T10:28:38.450 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 50724 50878 signal_ha ceph cwd unknown /proc/50724/task/50878/cwd (readlink: Permission denied) 2024-03-21T10:28:38.450 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 50724 50878 signal_ha ceph rtd unknown /proc/50724/task/50878/root (readlink: Permission denied) 2024-03-21T10:28:38.450 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 50724 50878 signal_ha ceph txt unknown /proc/50724/task/50878/exe (readlink: Permission denied) 2024-03-21T10:28:38.450 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 50724 50878 signal_ha ceph NOFD /proc/50724/task/50878/fd (opendir: Permission denied) 2024-03-21T10:28:38.450 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 50724 50879 io_contex ceph cwd unknown /proc/50724/task/50879/cwd (readlink: Permission denied) 2024-03-21T10:28:38.450 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 50724 50879 io_contex ceph rtd unknown /proc/50724/task/50879/root (readlink: Permission denied) 2024-03-21T10:28:38.451 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 50724 50879 io_contex ceph txt unknown /proc/50724/task/50879/exe (readlink: Permission denied) 2024-03-21T10:28:38.451 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 50724 50879 io_contex ceph NOFD /proc/50724/task/50879/fd (opendir: Permission denied) 2024-03-21T10:28:38.451 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 50724 50880 io_contex ceph cwd unknown /proc/50724/task/50880/cwd (readlink: Permission denied) 2024-03-21T10:28:38.451 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 50724 50880 io_contex ceph rtd unknown /proc/50724/task/50880/root (readlink: Permission denied) 2024-03-21T10:28:38.451 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 50724 50880 io_contex ceph txt unknown /proc/50724/task/50880/exe (readlink: Permission denied) 2024-03-21T10:28:38.451 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 50724 50880 io_contex ceph NOFD /proc/50724/task/50880/fd (opendir: Permission denied) 2024-03-21T10:28:38.451 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 50724 50884 OpHistory ceph cwd unknown /proc/50724/task/50884/cwd (readlink: Permission denied) 2024-03-21T10:28:38.451 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 50724 50884 OpHistory ceph rtd unknown /proc/50724/task/50884/root (readlink: Permission denied) 2024-03-21T10:28:38.451 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 50724 50884 OpHistory ceph txt unknown /proc/50724/task/50884/exe (readlink: Permission denied) 2024-03-21T10:28:38.451 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 50724 50884 OpHistory ceph NOFD /proc/50724/task/50884/fd (opendir: Permission denied) 2024-03-21T10:28:38.451 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 50724 50885 ceph_time ceph cwd unknown /proc/50724/task/50885/cwd (readlink: Permission denied) 2024-03-21T10:28:38.451 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 50724 50885 ceph_time ceph rtd unknown /proc/50724/task/50885/root (readlink: Permission denied) 2024-03-21T10:28:38.451 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 50724 50885 ceph_time ceph txt unknown /proc/50724/task/50885/exe (readlink: Permission denied) 2024-03-21T10:28:38.451 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 50724 50885 ceph_time ceph NOFD /proc/50724/task/50885/fd (opendir: Permission denied) 2024-03-21T10:28:38.452 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 50724 51000 ceph-osd ceph cwd unknown /proc/50724/task/51000/cwd (readlink: Permission denied) 2024-03-21T10:28:38.452 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 50724 51000 ceph-osd ceph rtd unknown /proc/50724/task/51000/root (readlink: Permission denied) 2024-03-21T10:28:38.452 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 50724 51000 ceph-osd ceph txt unknown /proc/50724/task/51000/exe (readlink: Permission denied) 2024-03-21T10:28:38.452 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 50724 51000 ceph-osd ceph NOFD /proc/50724/task/51000/fd (opendir: Permission denied) 2024-03-21T10:28:38.452 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 50724 51012 ceph-osd ceph cwd unknown /proc/50724/task/51012/cwd (readlink: Permission denied) 2024-03-21T10:28:38.452 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 50724 51012 ceph-osd ceph rtd unknown /proc/50724/task/51012/root (readlink: Permission denied) 2024-03-21T10:28:38.452 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 50724 51012 ceph-osd ceph txt unknown /proc/50724/task/51012/exe (readlink: Permission denied) 2024-03-21T10:28:38.452 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 50724 51012 ceph-osd ceph NOFD /proc/50724/task/51012/fd (opendir: Permission denied) 2024-03-21T10:28:38.452 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 50724 51053 ceph-osd ceph cwd unknown /proc/50724/task/51053/cwd (readlink: Permission denied) 2024-03-21T10:28:38.452 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 50724 51053 ceph-osd ceph rtd unknown /proc/50724/task/51053/root (readlink: Permission denied) 2024-03-21T10:28:38.452 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 50724 51053 ceph-osd ceph txt unknown /proc/50724/task/51053/exe (readlink: Permission denied) 2024-03-21T10:28:38.452 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 50724 51053 ceph-osd ceph NOFD /proc/50724/task/51053/fd (opendir: Permission denied) 2024-03-21T10:28:38.452 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 50724 51062 ceph-osd ceph cwd unknown /proc/50724/task/51062/cwd (readlink: Permission denied) 2024-03-21T10:28:38.452 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 50724 51062 ceph-osd ceph rtd unknown /proc/50724/task/51062/root (readlink: Permission denied) 2024-03-21T10:28:38.453 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 50724 51062 ceph-osd ceph txt unknown /proc/50724/task/51062/exe (readlink: Permission denied) 2024-03-21T10:28:38.453 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 50724 51062 ceph-osd ceph NOFD /proc/50724/task/51062/fd (opendir: Permission denied) 2024-03-21T10:28:38.453 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 50724 51072 ceph-osd ceph cwd unknown /proc/50724/task/51072/cwd (readlink: Permission denied) 2024-03-21T10:28:38.453 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 50724 51072 ceph-osd ceph rtd unknown /proc/50724/task/51072/root (readlink: Permission denied) 2024-03-21T10:28:38.453 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 50724 51072 ceph-osd ceph txt unknown /proc/50724/task/51072/exe (readlink: Permission denied) 2024-03-21T10:28:38.453 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 50724 51072 ceph-osd ceph NOFD /proc/50724/task/51072/fd (opendir: Permission denied) 2024-03-21T10:28:38.453 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 50724 51087 ceph-osd ceph cwd unknown /proc/50724/task/51087/cwd (readlink: Permission denied) 2024-03-21T10:28:38.453 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 50724 51087 ceph-osd ceph rtd unknown /proc/50724/task/51087/root (readlink: Permission denied) 2024-03-21T10:28:38.453 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 50724 51087 ceph-osd ceph txt unknown /proc/50724/task/51087/exe (readlink: Permission denied) 2024-03-21T10:28:38.453 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 50724 51087 ceph-osd ceph NOFD /proc/50724/task/51087/fd (opendir: Permission denied) 2024-03-21T10:28:38.453 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 50724 51092 ceph-osd ceph cwd unknown /proc/50724/task/51092/cwd (readlink: Permission denied) 2024-03-21T10:28:38.453 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 50724 51092 ceph-osd ceph rtd unknown /proc/50724/task/51092/root (readlink: Permission denied) 2024-03-21T10:28:38.453 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 50724 51092 ceph-osd ceph txt unknown /proc/50724/task/51092/exe (readlink: Permission denied) 2024-03-21T10:28:38.453 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 50724 51092 ceph-osd ceph NOFD /proc/50724/task/51092/fd (opendir: Permission denied) 2024-03-21T10:28:38.454 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 50724 51109 ceph-osd ceph cwd unknown /proc/50724/task/51109/cwd (readlink: Permission denied) 2024-03-21T10:28:38.454 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 50724 51109 ceph-osd ceph rtd unknown /proc/50724/task/51109/root (readlink: Permission denied) 2024-03-21T10:28:38.454 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 50724 51109 ceph-osd ceph txt unknown /proc/50724/task/51109/exe (readlink: Permission denied) 2024-03-21T10:28:38.454 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 50724 51109 ceph-osd ceph NOFD /proc/50724/task/51109/fd (opendir: Permission denied) 2024-03-21T10:28:38.454 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 50724 51110 ceph-osd ceph cwd unknown /proc/50724/task/51110/cwd (readlink: Permission denied) 2024-03-21T10:28:38.454 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 50724 51110 ceph-osd ceph rtd unknown /proc/50724/task/51110/root (readlink: Permission denied) 2024-03-21T10:28:38.454 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 50724 51110 ceph-osd ceph txt unknown /proc/50724/task/51110/exe (readlink: Permission denied) 2024-03-21T10:28:38.454 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 50724 51110 ceph-osd ceph NOFD /proc/50724/task/51110/fd (opendir: Permission denied) 2024-03-21T10:28:38.454 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 50724 51111 safe_time ceph cwd unknown /proc/50724/task/51111/cwd (readlink: Permission denied) 2024-03-21T10:28:38.454 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 50724 51111 safe_time ceph rtd unknown /proc/50724/task/51111/root (readlink: Permission denied) 2024-03-21T10:28:38.454 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 50724 51111 safe_time ceph txt unknown /proc/50724/task/51111/exe (readlink: Permission denied) 2024-03-21T10:28:38.454 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 50724 51111 safe_time ceph NOFD /proc/50724/task/51111/fd (opendir: Permission denied) 2024-03-21T10:28:38.454 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 50724 51112 safe_time ceph cwd unknown /proc/50724/task/51112/cwd (readlink: Permission denied) 2024-03-21T10:28:38.454 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 50724 51112 safe_time ceph rtd unknown /proc/50724/task/51112/root (readlink: Permission denied) 2024-03-21T10:28:38.455 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 50724 51112 safe_time ceph txt unknown /proc/50724/task/51112/exe (readlink: Permission denied) 2024-03-21T10:28:38.455 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 50724 51112 safe_time ceph NOFD /proc/50724/task/51112/fd (opendir: Permission denied) 2024-03-21T10:28:38.455 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 50724 51113 safe_time ceph cwd unknown /proc/50724/task/51113/cwd (readlink: Permission denied) 2024-03-21T10:28:38.455 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 50724 51113 safe_time ceph rtd unknown /proc/50724/task/51113/root (readlink: Permission denied) 2024-03-21T10:28:38.455 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 50724 51113 safe_time ceph txt unknown /proc/50724/task/51113/exe (readlink: Permission denied) 2024-03-21T10:28:38.455 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 50724 51113 safe_time ceph NOFD /proc/50724/task/51113/fd (opendir: Permission denied) 2024-03-21T10:28:38.455 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 50724 51114 safe_time ceph cwd unknown /proc/50724/task/51114/cwd (readlink: Permission denied) 2024-03-21T10:28:38.455 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 50724 51114 safe_time ceph rtd unknown /proc/50724/task/51114/root (readlink: Permission denied) 2024-03-21T10:28:38.455 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 50724 51114 safe_time ceph txt unknown /proc/50724/task/51114/exe (readlink: Permission denied) 2024-03-21T10:28:38.455 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 50724 51114 safe_time ceph NOFD /proc/50724/task/51114/fd (opendir: Permission denied) 2024-03-21T10:28:38.455 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 50724 51115 fn_anonym ceph cwd unknown /proc/50724/task/51115/cwd (readlink: Permission denied) 2024-03-21T10:28:38.455 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 50724 51115 fn_anonym ceph rtd unknown /proc/50724/task/51115/root (readlink: Permission denied) 2024-03-21T10:28:38.455 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 50724 51115 fn_anonym ceph txt unknown /proc/50724/task/51115/exe (readlink: Permission denied) 2024-03-21T10:28:38.455 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 50724 51115 fn_anonym ceph NOFD /proc/50724/task/51115/fd (opendir: Permission denied) 2024-03-21T10:28:38.455 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 50724 51125 rocksdb:l ceph cwd unknown /proc/50724/task/51125/cwd (readlink: Permission denied) 2024-03-21T10:28:38.456 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 50724 51125 rocksdb:l ceph rtd unknown /proc/50724/task/51125/root (readlink: Permission denied) 2024-03-21T10:28:38.456 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 50724 51125 rocksdb:l ceph txt unknown /proc/50724/task/51125/exe (readlink: Permission denied) 2024-03-21T10:28:38.456 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 50724 51125 rocksdb:l ceph NOFD /proc/50724/task/51125/fd (opendir: Permission denied) 2024-03-21T10:28:38.456 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 50724 51126 rocksdb:l ceph cwd unknown /proc/50724/task/51126/cwd (readlink: Permission denied) 2024-03-21T10:28:38.456 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 50724 51126 rocksdb:l ceph rtd unknown /proc/50724/task/51126/root (readlink: Permission denied) 2024-03-21T10:28:38.456 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 50724 51126 rocksdb:l ceph txt unknown /proc/50724/task/51126/exe (readlink: Permission denied) 2024-03-21T10:28:38.456 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 50724 51126 rocksdb:l ceph NOFD /proc/50724/task/51126/fd (opendir: Permission denied) 2024-03-21T10:28:38.456 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 50724 51127 rocksdb:l ceph cwd unknown /proc/50724/task/51127/cwd (readlink: Permission denied) 2024-03-21T10:28:38.456 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 50724 51127 rocksdb:l ceph rtd unknown /proc/50724/task/51127/root (readlink: Permission denied) 2024-03-21T10:28:38.456 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 50724 51127 rocksdb:l ceph txt unknown /proc/50724/task/51127/exe (readlink: Permission denied) 2024-03-21T10:28:38.456 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 50724 51127 rocksdb:l ceph NOFD /proc/50724/task/51127/fd (opendir: Permission denied) 2024-03-21T10:28:38.456 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 50724 51128 rocksdb:h ceph cwd unknown /proc/50724/task/51128/cwd (readlink: Permission denied) 2024-03-21T10:28:38.456 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 50724 51128 rocksdb:h ceph rtd unknown /proc/50724/task/51128/root (readlink: Permission denied) 2024-03-21T10:28:38.457 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 50724 51128 rocksdb:h ceph txt unknown /proc/50724/task/51128/exe (readlink: Permission denied) 2024-03-21T10:28:38.457 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 50724 51128 rocksdb:h ceph NOFD /proc/50724/task/51128/fd (opendir: Permission denied) 2024-03-21T10:28:38.457 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 50724 51549 bstore_ai ceph cwd unknown /proc/50724/task/51549/cwd (readlink: Permission denied) 2024-03-21T10:28:38.457 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 50724 51549 bstore_ai ceph rtd unknown /proc/50724/task/51549/root (readlink: Permission denied) 2024-03-21T10:28:38.457 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 50724 51549 bstore_ai ceph txt unknown /proc/50724/task/51549/exe (readlink: Permission denied) 2024-03-21T10:28:38.457 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 50724 51549 bstore_ai ceph NOFD /proc/50724/task/51549/fd (opendir: Permission denied) 2024-03-21T10:28:38.457 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 50724 51770 bstore_ai ceph cwd unknown /proc/50724/task/51770/cwd (readlink: Permission denied) 2024-03-21T10:28:38.457 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 50724 51770 bstore_ai ceph rtd unknown /proc/50724/task/51770/root (readlink: Permission denied) 2024-03-21T10:28:38.457 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 50724 51770 bstore_ai ceph txt unknown /proc/50724/task/51770/exe (readlink: Permission denied) 2024-03-21T10:28:38.457 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 50724 51770 bstore_ai ceph NOFD /proc/50724/task/51770/fd (opendir: Permission denied) 2024-03-21T10:28:38.457 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 50724 51966 ceph-osd ceph cwd unknown /proc/50724/task/51966/cwd (readlink: Permission denied) 2024-03-21T10:28:38.457 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 50724 51966 ceph-osd ceph rtd unknown /proc/50724/task/51966/root (readlink: Permission denied) 2024-03-21T10:28:38.457 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 50724 51966 ceph-osd ceph txt unknown /proc/50724/task/51966/exe (readlink: Permission denied) 2024-03-21T10:28:38.458 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 50724 51966 ceph-osd ceph NOFD /proc/50724/task/51966/fd (opendir: Permission denied) 2024-03-21T10:28:38.458 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 50724 51967 cfin ceph cwd unknown /proc/50724/task/51967/cwd (readlink: Permission denied) 2024-03-21T10:28:38.458 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 50724 51967 cfin ceph rtd unknown /proc/50724/task/51967/root (readlink: Permission denied) 2024-03-21T10:28:38.458 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 50724 51967 cfin ceph txt unknown /proc/50724/task/51967/exe (readlink: Permission denied) 2024-03-21T10:28:38.458 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 50724 51967 cfin ceph NOFD /proc/50724/task/51967/fd (opendir: Permission denied) 2024-03-21T10:28:38.458 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 50724 51968 bstore_kv ceph cwd unknown /proc/50724/task/51968/cwd (readlink: Permission denied) 2024-03-21T10:28:38.458 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 50724 51968 bstore_kv ceph rtd unknown /proc/50724/task/51968/root (readlink: Permission denied) 2024-03-21T10:28:38.458 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 50724 51968 bstore_kv ceph txt unknown /proc/50724/task/51968/exe (readlink: Permission denied) 2024-03-21T10:28:38.458 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 50724 51968 bstore_kv ceph NOFD /proc/50724/task/51968/fd (opendir: Permission denied) 2024-03-21T10:28:38.458 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 50724 51969 bstore_kv ceph cwd unknown /proc/50724/task/51969/cwd (readlink: Permission denied) 2024-03-21T10:28:38.458 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 50724 51969 bstore_kv ceph rtd unknown /proc/50724/task/51969/root (readlink: Permission denied) 2024-03-21T10:28:38.458 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 50724 51969 bstore_kv ceph txt unknown /proc/50724/task/51969/exe (readlink: Permission denied) 2024-03-21T10:28:38.458 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 50724 51969 bstore_kv ceph NOFD /proc/50724/task/51969/fd (opendir: Permission denied) 2024-03-21T10:28:38.459 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 50724 51970 bstore_me ceph cwd unknown /proc/50724/task/51970/cwd (readlink: Permission denied) 2024-03-21T10:28:38.459 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 50724 51970 bstore_me ceph rtd unknown /proc/50724/task/51970/root (readlink: Permission denied) 2024-03-21T10:28:38.459 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 50724 51970 bstore_me ceph txt unknown /proc/50724/task/51970/exe (readlink: Permission denied) 2024-03-21T10:28:38.459 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 50724 51970 bstore_me ceph NOFD /proc/50724/task/51970/fd (opendir: Permission denied) 2024-03-21T10:28:38.459 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 50724 51984 ms_dispat ceph cwd unknown /proc/50724/task/51984/cwd (readlink: Permission denied) 2024-03-21T10:28:38.459 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 50724 51984 ms_dispat ceph rtd unknown /proc/50724/task/51984/root (readlink: Permission denied) 2024-03-21T10:28:38.459 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 50724 51984 ms_dispat ceph txt unknown /proc/50724/task/51984/exe (readlink: Permission denied) 2024-03-21T10:28:38.459 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 50724 51984 ms_dispat ceph NOFD /proc/50724/task/51984/fd (opendir: Permission denied) 2024-03-21T10:28:38.459 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 50724 51985 ms_local ceph cwd unknown /proc/50724/task/51985/cwd (readlink: Permission denied) 2024-03-21T10:28:38.459 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 50724 51985 ms_local ceph rtd unknown /proc/50724/task/51985/root (readlink: Permission denied) 2024-03-21T10:28:38.459 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 50724 51985 ms_local ceph txt unknown /proc/50724/task/51985/exe (readlink: Permission denied) 2024-03-21T10:28:38.459 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 50724 51985 ms_local ceph NOFD /proc/50724/task/51985/fd (opendir: Permission denied) 2024-03-21T10:28:38.460 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 50724 51986 safe_time ceph cwd unknown /proc/50724/task/51986/cwd (readlink: Permission denied) 2024-03-21T10:28:38.460 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 50724 51986 safe_time ceph rtd unknown /proc/50724/task/51986/root (readlink: Permission denied) 2024-03-21T10:28:38.460 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 50724 51986 safe_time ceph txt unknown /proc/50724/task/51986/exe (readlink: Permission denied) 2024-03-21T10:28:38.460 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 50724 51986 safe_time ceph NOFD /proc/50724/task/51986/fd (opendir: Permission denied) 2024-03-21T10:28:38.460 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 50724 51987 safe_time ceph cwd unknown /proc/50724/task/51987/cwd (readlink: Permission denied) 2024-03-21T10:28:38.460 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 50724 51987 safe_time ceph rtd unknown /proc/50724/task/51987/root (readlink: Permission denied) 2024-03-21T10:28:38.460 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 50724 51987 safe_time ceph txt unknown /proc/50724/task/51987/exe (readlink: Permission denied) 2024-03-21T10:28:38.460 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 50724 51987 safe_time ceph NOFD /proc/50724/task/51987/fd (opendir: Permission denied) 2024-03-21T10:28:38.460 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 50724 51988 ms_dispat ceph cwd unknown /proc/50724/task/51988/cwd (readlink: Permission denied) 2024-03-21T10:28:38.460 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 50724 51988 ms_dispat ceph rtd unknown /proc/50724/task/51988/root (readlink: Permission denied) 2024-03-21T10:28:38.460 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 50724 51988 ms_dispat ceph txt unknown /proc/50724/task/51988/exe (readlink: Permission denied) 2024-03-21T10:28:38.460 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 50724 51988 ms_dispat ceph NOFD /proc/50724/task/51988/fd (opendir: Permission denied) 2024-03-21T10:28:38.460 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 50724 51989 ms_local ceph cwd unknown /proc/50724/task/51989/cwd (readlink: Permission denied) 2024-03-21T10:28:38.461 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 50724 51989 ms_local ceph rtd unknown /proc/50724/task/51989/root (readlink: Permission denied) 2024-03-21T10:28:38.461 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 50724 51989 ms_local ceph txt unknown /proc/50724/task/51989/exe (readlink: Permission denied) 2024-03-21T10:28:38.461 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 50724 51989 ms_local ceph NOFD /proc/50724/task/51989/fd (opendir: Permission denied) 2024-03-21T10:28:38.461 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 50724 51990 ms_dispat ceph cwd unknown /proc/50724/task/51990/cwd (readlink: Permission denied) 2024-03-21T10:28:38.461 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 50724 51990 ms_dispat ceph rtd unknown /proc/50724/task/51990/root (readlink: Permission denied) 2024-03-21T10:28:38.461 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 50724 51990 ms_dispat ceph txt unknown /proc/50724/task/51990/exe (readlink: Permission denied) 2024-03-21T10:28:38.461 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 50724 51990 ms_dispat ceph NOFD /proc/50724/task/51990/fd (opendir: Permission denied) 2024-03-21T10:28:38.461 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 50724 51991 ms_local ceph cwd unknown /proc/50724/task/51991/cwd (readlink: Permission denied) 2024-03-21T10:28:38.461 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 50724 51991 ms_local ceph rtd unknown /proc/50724/task/51991/root (readlink: Permission denied) 2024-03-21T10:28:38.461 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 50724 51991 ms_local ceph txt unknown /proc/50724/task/51991/exe (readlink: Permission denied) 2024-03-21T10:28:38.461 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 50724 51991 ms_local ceph NOFD /proc/50724/task/51991/fd (opendir: Permission denied) 2024-03-21T10:28:38.461 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 50724 51992 ms_dispat ceph cwd unknown /proc/50724/task/51992/cwd (readlink: Permission denied) 2024-03-21T10:28:38.462 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 50724 51992 ms_dispat ceph rtd unknown /proc/50724/task/51992/root (readlink: Permission denied) 2024-03-21T10:28:38.462 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 50724 51992 ms_dispat ceph txt unknown /proc/50724/task/51992/exe (readlink: Permission denied) 2024-03-21T10:28:38.462 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 50724 51992 ms_dispat ceph NOFD /proc/50724/task/51992/fd (opendir: Permission denied) 2024-03-21T10:28:38.462 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 50724 51993 ms_local ceph cwd unknown /proc/50724/task/51993/cwd (readlink: Permission denied) 2024-03-21T10:28:38.462 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 50724 51993 ms_local ceph rtd unknown /proc/50724/task/51993/root (readlink: Permission denied) 2024-03-21T10:28:38.462 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 50724 51993 ms_local ceph txt unknown /proc/50724/task/51993/exe (readlink: Permission denied) 2024-03-21T10:28:38.462 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 50724 51993 ms_local ceph NOFD /proc/50724/task/51993/fd (opendir: Permission denied) 2024-03-21T10:28:38.462 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 50724 51994 ms_dispat ceph cwd unknown /proc/50724/task/51994/cwd (readlink: Permission denied) 2024-03-21T10:28:38.462 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 50724 51994 ms_dispat ceph rtd unknown /proc/50724/task/51994/root (readlink: Permission denied) 2024-03-21T10:28:38.462 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 50724 51994 ms_dispat ceph txt unknown /proc/50724/task/51994/exe (readlink: Permission denied) 2024-03-21T10:28:38.462 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 50724 51994 ms_dispat ceph NOFD /proc/50724/task/51994/fd (opendir: Permission denied) 2024-03-21T10:28:38.462 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 50724 51995 ms_local ceph cwd unknown /proc/50724/task/51995/cwd (readlink: Permission denied) 2024-03-21T10:28:38.462 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 50724 51995 ms_local ceph rtd unknown /proc/50724/task/51995/root (readlink: Permission denied) 2024-03-21T10:28:38.462 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 50724 51995 ms_local ceph txt unknown /proc/50724/task/51995/exe (readlink: Permission denied) 2024-03-21T10:28:38.463 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 50724 51995 ms_local ceph NOFD /proc/50724/task/51995/fd (opendir: Permission denied) 2024-03-21T10:28:38.463 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 50724 51996 ms_dispat ceph cwd unknown /proc/50724/task/51996/cwd (readlink: Permission denied) 2024-03-21T10:28:38.463 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 50724 51996 ms_dispat ceph rtd unknown /proc/50724/task/51996/root (readlink: Permission denied) 2024-03-21T10:28:38.463 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 50724 51996 ms_dispat ceph txt unknown /proc/50724/task/51996/exe (readlink: Permission denied) 2024-03-21T10:28:38.463 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 50724 51996 ms_dispat ceph NOFD /proc/50724/task/51996/fd (opendir: Permission denied) 2024-03-21T10:28:38.463 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 50724 51997 ms_local ceph cwd unknown /proc/50724/task/51997/cwd (readlink: Permission denied) 2024-03-21T10:28:38.463 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 50724 51997 ms_local ceph rtd unknown /proc/50724/task/51997/root (readlink: Permission denied) 2024-03-21T10:28:38.463 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 50724 51997 ms_local ceph txt unknown /proc/50724/task/51997/exe (readlink: Permission denied) 2024-03-21T10:28:38.463 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 50724 51997 ms_local ceph NOFD /proc/50724/task/51997/fd (opendir: Permission denied) 2024-03-21T10:28:38.463 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 50724 51998 ms_dispat ceph cwd unknown /proc/50724/task/51998/cwd (readlink: Permission denied) 2024-03-21T10:28:38.463 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 50724 51998 ms_dispat ceph rtd unknown /proc/50724/task/51998/root (readlink: Permission denied) 2024-03-21T10:28:38.463 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 50724 51998 ms_dispat ceph txt unknown /proc/50724/task/51998/exe (readlink: Permission denied) 2024-03-21T10:28:38.463 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 50724 51998 ms_dispat ceph NOFD /proc/50724/task/51998/fd (opendir: Permission denied) 2024-03-21T10:28:38.464 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 50724 51999 ms_local ceph cwd unknown /proc/50724/task/51999/cwd (readlink: Permission denied) 2024-03-21T10:28:38.464 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 50724 51999 ms_local ceph rtd unknown /proc/50724/task/51999/root (readlink: Permission denied) 2024-03-21T10:28:38.464 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 50724 51999 ms_local ceph txt unknown /proc/50724/task/51999/exe (readlink: Permission denied) 2024-03-21T10:28:38.464 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 50724 51999 ms_local ceph NOFD /proc/50724/task/51999/fd (opendir: Permission denied) 2024-03-21T10:28:38.464 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 50724 52000 fn_anonym ceph cwd unknown /proc/50724/task/52000/cwd (readlink: Permission denied) 2024-03-21T10:28:38.464 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 50724 52000 fn_anonym ceph rtd unknown /proc/50724/task/52000/root (readlink: Permission denied) 2024-03-21T10:28:38.464 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 50724 52000 fn_anonym ceph txt unknown /proc/50724/task/52000/exe (readlink: Permission denied) 2024-03-21T10:28:38.464 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 50724 52000 fn_anonym ceph NOFD /proc/50724/task/52000/fd (opendir: Permission denied) 2024-03-21T10:28:38.464 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 50724 52001 finisher ceph cwd unknown /proc/50724/task/52001/cwd (readlink: Permission denied) 2024-03-21T10:28:38.464 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 50724 52001 finisher ceph rtd unknown /proc/50724/task/52001/root (readlink: Permission denied) 2024-03-21T10:28:38.464 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 50724 52001 finisher ceph txt unknown /proc/50724/task/52001/exe (readlink: Permission denied) 2024-03-21T10:28:38.464 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 50724 52001 finisher ceph NOFD /proc/50724/task/52001/fd (opendir: Permission denied) 2024-03-21T10:28:38.464 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 50724 52002 safe_time ceph cwd unknown /proc/50724/task/52002/cwd (readlink: Permission denied) 2024-03-21T10:28:38.464 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 50724 52002 safe_time ceph rtd unknown /proc/50724/task/52002/root (readlink: Permission denied) 2024-03-21T10:28:38.465 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 50724 52002 safe_time ceph txt unknown /proc/50724/task/52002/exe (readlink: Permission denied) 2024-03-21T10:28:38.465 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 50724 52002 safe_time ceph NOFD /proc/50724/task/52002/fd (opendir: Permission denied) 2024-03-21T10:28:38.465 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 50724 52003 safe_time ceph cwd unknown /proc/50724/task/52003/cwd (readlink: Permission denied) 2024-03-21T10:28:38.465 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 50724 52003 safe_time ceph rtd unknown /proc/50724/task/52003/root (readlink: Permission denied) 2024-03-21T10:28:38.465 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 50724 52003 safe_time ceph txt unknown /proc/50724/task/52003/exe (readlink: Permission denied) 2024-03-21T10:28:38.465 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 50724 52003 safe_time ceph NOFD /proc/50724/task/52003/fd (opendir: Permission denied) 2024-03-21T10:28:38.465 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 50724 52004 ceph-osd ceph cwd unknown /proc/50724/task/52004/cwd (readlink: Permission denied) 2024-03-21T10:28:38.465 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 50724 52004 ceph-osd ceph rtd unknown /proc/50724/task/52004/root (readlink: Permission denied) 2024-03-21T10:28:38.465 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 50724 52004 ceph-osd ceph txt unknown /proc/50724/task/52004/exe (readlink: Permission denied) 2024-03-21T10:28:38.465 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 50724 52004 ceph-osd ceph NOFD /proc/50724/task/52004/fd (opendir: Permission denied) 2024-03-21T10:28:38.465 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 50724 52005 osd_srv_a ceph cwd unknown /proc/50724/task/52005/cwd (readlink: Permission denied) 2024-03-21T10:28:38.465 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 50724 52005 osd_srv_a ceph rtd unknown /proc/50724/task/52005/root (readlink: Permission denied) 2024-03-21T10:28:38.465 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 50724 52005 osd_srv_a ceph txt unknown /proc/50724/task/52005/exe (readlink: Permission denied) 2024-03-21T10:28:38.466 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 50724 52005 osd_srv_a ceph NOFD /proc/50724/task/52005/fd (opendir: Permission denied) 2024-03-21T10:28:38.466 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 50724 52006 tp_osd_tp ceph cwd unknown /proc/50724/task/52006/cwd (readlink: Permission denied) 2024-03-21T10:28:38.466 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 50724 52006 tp_osd_tp ceph rtd unknown /proc/50724/task/52006/root (readlink: Permission denied) 2024-03-21T10:28:38.466 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 50724 52006 tp_osd_tp ceph txt unknown /proc/50724/task/52006/exe (readlink: Permission denied) 2024-03-21T10:28:38.466 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 50724 52006 tp_osd_tp ceph NOFD /proc/50724/task/52006/fd (opendir: Permission denied) 2024-03-21T10:28:38.466 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 50724 52007 tp_osd_tp ceph cwd unknown /proc/50724/task/52007/cwd (readlink: Permission denied) 2024-03-21T10:28:38.466 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 50724 52007 tp_osd_tp ceph rtd unknown /proc/50724/task/52007/root (readlink: Permission denied) 2024-03-21T10:28:38.466 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 50724 52007 tp_osd_tp ceph txt unknown /proc/50724/task/52007/exe (readlink: Permission denied) 2024-03-21T10:28:38.466 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 50724 52007 tp_osd_tp ceph NOFD /proc/50724/task/52007/fd (opendir: Permission denied) 2024-03-21T10:28:38.466 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 50724 52008 tp_osd_tp ceph cwd unknown /proc/50724/task/52008/cwd (readlink: Permission denied) 2024-03-21T10:28:38.466 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 50724 52008 tp_osd_tp ceph rtd unknown /proc/50724/task/52008/root (readlink: Permission denied) 2024-03-21T10:28:38.466 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 50724 52008 tp_osd_tp ceph txt unknown /proc/50724/task/52008/exe (readlink: Permission denied) 2024-03-21T10:28:38.467 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 50724 52008 tp_osd_tp ceph NOFD /proc/50724/task/52008/fd (opendir: Permission denied) 2024-03-21T10:28:38.467 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 50724 52009 tp_osd_tp ceph cwd unknown /proc/50724/task/52009/cwd (readlink: Permission denied) 2024-03-21T10:28:38.467 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 50724 52009 tp_osd_tp ceph rtd unknown /proc/50724/task/52009/root (readlink: Permission denied) 2024-03-21T10:28:38.467 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 50724 52009 tp_osd_tp ceph txt unknown /proc/50724/task/52009/exe (readlink: Permission denied) 2024-03-21T10:28:38.467 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 50724 52009 tp_osd_tp ceph NOFD /proc/50724/task/52009/fd (opendir: Permission denied) 2024-03-21T10:28:38.467 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 50724 52010 tp_osd_tp ceph cwd unknown /proc/50724/task/52010/cwd (readlink: Permission denied) 2024-03-21T10:28:38.467 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 50724 52010 tp_osd_tp ceph rtd unknown /proc/50724/task/52010/root (readlink: Permission denied) 2024-03-21T10:28:38.467 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 50724 52010 tp_osd_tp ceph txt unknown /proc/50724/task/52010/exe (readlink: Permission denied) 2024-03-21T10:28:38.467 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 50724 52010 tp_osd_tp ceph NOFD /proc/50724/task/52010/fd (opendir: Permission denied) 2024-03-21T10:28:38.467 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 50724 52011 tp_osd_tp ceph cwd unknown /proc/50724/task/52011/cwd (readlink: Permission denied) 2024-03-21T10:28:38.467 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 50724 52011 tp_osd_tp ceph rtd unknown /proc/50724/task/52011/root (readlink: Permission denied) 2024-03-21T10:28:38.467 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 50724 52011 tp_osd_tp ceph txt unknown /proc/50724/task/52011/exe (readlink: Permission denied) 2024-03-21T10:28:38.467 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 50724 52011 tp_osd_tp ceph NOFD /proc/50724/task/52011/fd (opendir: Permission denied) 2024-03-21T10:28:38.467 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 50724 52012 tp_osd_tp ceph cwd unknown /proc/50724/task/52012/cwd (readlink: Permission denied) 2024-03-21T10:28:38.468 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 50724 52012 tp_osd_tp ceph rtd unknown /proc/50724/task/52012/root (readlink: Permission denied) 2024-03-21T10:28:38.468 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 50724 52012 tp_osd_tp ceph txt unknown /proc/50724/task/52012/exe (readlink: Permission denied) 2024-03-21T10:28:38.468 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 50724 52012 tp_osd_tp ceph NOFD /proc/50724/task/52012/fd (opendir: Permission denied) 2024-03-21T10:28:38.468 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 50724 52013 tp_osd_tp ceph cwd unknown /proc/50724/task/52013/cwd (readlink: Permission denied) 2024-03-21T10:28:38.468 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 50724 52013 tp_osd_tp ceph rtd unknown /proc/50724/task/52013/root (readlink: Permission denied) 2024-03-21T10:28:38.468 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 50724 52013 tp_osd_tp ceph txt unknown /proc/50724/task/52013/exe (readlink: Permission denied) 2024-03-21T10:28:38.468 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 50724 52013 tp_osd_tp ceph NOFD /proc/50724/task/52013/fd (opendir: Permission denied) 2024-03-21T10:28:38.468 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 50724 52014 tp_osd_tp ceph cwd unknown /proc/50724/task/52014/cwd (readlink: Permission denied) 2024-03-21T10:28:38.468 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 50724 52014 tp_osd_tp ceph rtd unknown /proc/50724/task/52014/root (readlink: Permission denied) 2024-03-21T10:28:38.468 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 50724 52014 tp_osd_tp ceph txt unknown /proc/50724/task/52014/exe (readlink: Permission denied) 2024-03-21T10:28:38.468 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 50724 52014 tp_osd_tp ceph NOFD /proc/50724/task/52014/fd (opendir: Permission denied) 2024-03-21T10:28:38.468 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 50724 52015 tp_osd_tp ceph cwd unknown /proc/50724/task/52015/cwd (readlink: Permission denied) 2024-03-21T10:28:38.468 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 50724 52015 tp_osd_tp ceph rtd unknown /proc/50724/task/52015/root (readlink: Permission denied) 2024-03-21T10:28:38.468 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 50724 52015 tp_osd_tp ceph txt unknown /proc/50724/task/52015/exe (readlink: Permission denied) 2024-03-21T10:28:38.469 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 50724 52015 tp_osd_tp ceph NOFD /proc/50724/task/52015/fd (opendir: Permission denied) 2024-03-21T10:28:38.469 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 50724 52016 tp_osd_tp ceph cwd unknown /proc/50724/task/52016/cwd (readlink: Permission denied) 2024-03-21T10:28:38.469 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 50724 52016 tp_osd_tp ceph rtd unknown /proc/50724/task/52016/root (readlink: Permission denied) 2024-03-21T10:28:38.469 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 50724 52016 tp_osd_tp ceph txt unknown /proc/50724/task/52016/exe (readlink: Permission denied) 2024-03-21T10:28:38.469 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 50724 52016 tp_osd_tp ceph NOFD /proc/50724/task/52016/fd (opendir: Permission denied) 2024-03-21T10:28:38.469 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 50724 52017 tp_osd_tp ceph cwd unknown /proc/50724/task/52017/cwd (readlink: Permission denied) 2024-03-21T10:28:38.469 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 50724 52017 tp_osd_tp ceph rtd unknown /proc/50724/task/52017/root (readlink: Permission denied) 2024-03-21T10:28:38.469 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 50724 52017 tp_osd_tp ceph txt unknown /proc/50724/task/52017/exe (readlink: Permission denied) 2024-03-21T10:28:38.469 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 50724 52017 tp_osd_tp ceph NOFD /proc/50724/task/52017/fd (opendir: Permission denied) 2024-03-21T10:28:38.469 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 50724 52018 tp_osd_tp ceph cwd unknown /proc/50724/task/52018/cwd (readlink: Permission denied) 2024-03-21T10:28:38.469 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 50724 52018 tp_osd_tp ceph rtd unknown /proc/50724/task/52018/root (readlink: Permission denied) 2024-03-21T10:28:38.469 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 50724 52018 tp_osd_tp ceph txt unknown /proc/50724/task/52018/exe (readlink: Permission denied) 2024-03-21T10:28:38.469 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 50724 52018 tp_osd_tp ceph NOFD /proc/50724/task/52018/fd (opendir: Permission denied) 2024-03-21T10:28:38.469 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 50724 52019 tp_osd_tp ceph cwd unknown /proc/50724/task/52019/cwd (readlink: Permission denied) 2024-03-21T10:28:38.470 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 50724 52019 tp_osd_tp ceph rtd unknown /proc/50724/task/52019/root (readlink: Permission denied) 2024-03-21T10:28:38.470 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 50724 52019 tp_osd_tp ceph txt unknown /proc/50724/task/52019/exe (readlink: Permission denied) 2024-03-21T10:28:38.470 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 50724 52019 tp_osd_tp ceph NOFD /proc/50724/task/52019/fd (opendir: Permission denied) 2024-03-21T10:28:38.470 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 50724 52020 tp_osd_tp ceph cwd unknown /proc/50724/task/52020/cwd (readlink: Permission denied) 2024-03-21T10:28:38.470 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 50724 52020 tp_osd_tp ceph rtd unknown /proc/50724/task/52020/root (readlink: Permission denied) 2024-03-21T10:28:38.470 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 50724 52020 tp_osd_tp ceph txt unknown /proc/50724/task/52020/exe (readlink: Permission denied) 2024-03-21T10:28:38.470 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 50724 52020 tp_osd_tp ceph NOFD /proc/50724/task/52020/fd (opendir: Permission denied) 2024-03-21T10:28:38.470 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 50724 52021 tp_osd_tp ceph cwd unknown /proc/50724/task/52021/cwd (readlink: Permission denied) 2024-03-21T10:28:38.470 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 50724 52021 tp_osd_tp ceph rtd unknown /proc/50724/task/52021/root (readlink: Permission denied) 2024-03-21T10:28:38.470 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 50724 52021 tp_osd_tp ceph txt unknown /proc/50724/task/52021/exe (readlink: Permission denied) 2024-03-21T10:28:38.470 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 50724 52021 tp_osd_tp ceph NOFD /proc/50724/task/52021/fd (opendir: Permission denied) 2024-03-21T10:28:38.470 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 50724 52022 osd_srv_h ceph cwd unknown /proc/50724/task/52022/cwd (readlink: Permission denied) 2024-03-21T10:28:38.470 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 50724 52022 osd_srv_h ceph rtd unknown /proc/50724/task/52022/root (readlink: Permission denied) 2024-03-21T10:28:38.471 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 50724 52022 osd_srv_h ceph txt unknown /proc/50724/task/52022/exe (readlink: Permission denied) 2024-03-21T10:28:38.471 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 50724 52022 osd_srv_h ceph NOFD /proc/50724/task/52022/fd (opendir: Permission denied) 2024-03-21T10:28:38.471 INFO:tasks.workunit.client.0.smithi047.stdout:sudo 52262 root cwd unknown /proc/52262/cwd (readlink: Permission denied) 2024-03-21T10:28:38.471 INFO:tasks.workunit.client.0.smithi047.stdout:sudo 52262 root rtd unknown /proc/52262/root (readlink: Permission denied) 2024-03-21T10:28:38.471 INFO:tasks.workunit.client.0.smithi047.stdout:sudo 52262 root txt unknown /proc/52262/exe (readlink: Permission denied) 2024-03-21T10:28:38.471 INFO:tasks.workunit.client.0.smithi047.stdout:sudo 52262 root NOFD /proc/52262/fd (opendir: Permission denied) 2024-03-21T10:28:38.471 INFO:tasks.workunit.client.0.smithi047.stdout:journalct 52290 root cwd unknown /proc/52290/cwd (readlink: Permission denied) 2024-03-21T10:28:38.471 INFO:tasks.workunit.client.0.smithi047.stdout:journalct 52290 root rtd unknown /proc/52290/root (readlink: Permission denied) 2024-03-21T10:28:38.471 INFO:tasks.workunit.client.0.smithi047.stdout:journalct 52290 root txt unknown /proc/52290/exe (readlink: Permission denied) 2024-03-21T10:28:38.471 INFO:tasks.workunit.client.0.smithi047.stdout:journalct 52290 root NOFD /proc/52290/fd (opendir: Permission denied) 2024-03-21T10:28:38.471 INFO:tasks.workunit.client.0.smithi047.stdout:kworker/6 54773 root cwd unknown /proc/54773/cwd (readlink: Permission denied) 2024-03-21T10:28:38.471 INFO:tasks.workunit.client.0.smithi047.stdout:kworker/6 54773 root rtd unknown /proc/54773/root (readlink: Permission denied) 2024-03-21T10:28:38.471 INFO:tasks.workunit.client.0.smithi047.stdout:kworker/6 54773 root txt unknown /proc/54773/exe (readlink: Permission denied) 2024-03-21T10:28:38.471 INFO:tasks.workunit.client.0.smithi047.stdout:kworker/6 54773 root NOFD /proc/54773/fd (opendir: Permission denied) 2024-03-21T10:28:38.471 INFO:tasks.workunit.client.0.smithi047.stdout:conmon 57372 root cwd unknown /proc/57372/cwd (readlink: Permission denied) 2024-03-21T10:28:38.472 INFO:tasks.workunit.client.0.smithi047.stdout:conmon 57372 root rtd unknown /proc/57372/root (readlink: Permission denied) 2024-03-21T10:28:38.472 INFO:tasks.workunit.client.0.smithi047.stdout:conmon 57372 root txt unknown /proc/57372/exe (readlink: Permission denied) 2024-03-21T10:28:38.472 INFO:tasks.workunit.client.0.smithi047.stdout:conmon 57372 root NOFD /proc/57372/fd (opendir: Permission denied) 2024-03-21T10:28:38.472 INFO:tasks.workunit.client.0.smithi047.stdout:podman-in 57374 root cwd unknown /proc/57374/cwd (readlink: Permission denied) 2024-03-21T10:28:38.472 INFO:tasks.workunit.client.0.smithi047.stdout:podman-in 57374 root rtd unknown /proc/57374/root (readlink: Permission denied) 2024-03-21T10:28:38.472 INFO:tasks.workunit.client.0.smithi047.stdout:podman-in 57374 root txt unknown /proc/57374/exe (readlink: Permission denied) 2024-03-21T10:28:38.472 INFO:tasks.workunit.client.0.smithi047.stdout:podman-in 57374 root NOFD /proc/57374/fd (opendir: Permission denied) 2024-03-21T10:28:38.472 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 57376 ceph cwd unknown /proc/57376/cwd (readlink: Permission denied) 2024-03-21T10:28:38.472 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 57376 ceph rtd unknown /proc/57376/root (readlink: Permission denied) 2024-03-21T10:28:38.472 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 57376 ceph txt unknown /proc/57376/exe (readlink: Permission denied) 2024-03-21T10:28:38.472 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 57376 ceph NOFD /proc/57376/fd (opendir: Permission denied) 2024-03-21T10:28:38.472 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 57376 57377 log ceph cwd unknown /proc/57376/task/57377/cwd (readlink: Permission denied) 2024-03-21T10:28:38.472 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 57376 57377 log ceph rtd unknown /proc/57376/task/57377/root (readlink: Permission denied) 2024-03-21T10:28:38.472 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 57376 57377 log ceph txt unknown /proc/57376/task/57377/exe (readlink: Permission denied) 2024-03-21T10:28:38.473 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 57376 57377 log ceph NOFD /proc/57376/task/57377/fd (opendir: Permission denied) 2024-03-21T10:28:38.473 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 57376 57379 msgr-work ceph cwd unknown /proc/57376/task/57379/cwd (readlink: Permission denied) 2024-03-21T10:28:38.473 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 57376 57379 msgr-work ceph rtd unknown /proc/57376/task/57379/root (readlink: Permission denied) 2024-03-21T10:28:38.473 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 57376 57379 msgr-work ceph txt unknown /proc/57376/task/57379/exe (readlink: Permission denied) 2024-03-21T10:28:38.473 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 57376 57379 msgr-work ceph NOFD /proc/57376/task/57379/fd (opendir: Permission denied) 2024-03-21T10:28:38.473 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 57376 57380 msgr-work ceph cwd unknown /proc/57376/task/57380/cwd (readlink: Permission denied) 2024-03-21T10:28:38.473 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 57376 57380 msgr-work ceph rtd unknown /proc/57376/task/57380/root (readlink: Permission denied) 2024-03-21T10:28:38.473 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 57376 57380 msgr-work ceph txt unknown /proc/57376/task/57380/exe (readlink: Permission denied) 2024-03-21T10:28:38.473 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 57376 57380 msgr-work ceph NOFD /proc/57376/task/57380/fd (opendir: Permission denied) 2024-03-21T10:28:38.473 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 57376 57381 msgr-work ceph cwd unknown /proc/57376/task/57381/cwd (readlink: Permission denied) 2024-03-21T10:28:38.473 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 57376 57381 msgr-work ceph rtd unknown /proc/57376/task/57381/root (readlink: Permission denied) 2024-03-21T10:28:38.473 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 57376 57381 msgr-work ceph txt unknown /proc/57376/task/57381/exe (readlink: Permission denied) 2024-03-21T10:28:38.473 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 57376 57381 msgr-work ceph NOFD /proc/57376/task/57381/fd (opendir: Permission denied) 2024-03-21T10:28:38.473 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 57376 57385 service ceph cwd unknown /proc/57376/task/57385/cwd (readlink: Permission denied) 2024-03-21T10:28:38.473 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 57376 57385 service ceph rtd unknown /proc/57376/task/57385/root (readlink: Permission denied) 2024-03-21T10:28:38.474 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 57376 57385 service ceph txt unknown /proc/57376/task/57385/exe (readlink: Permission denied) 2024-03-21T10:28:38.474 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 57376 57385 service ceph NOFD /proc/57376/task/57385/fd (opendir: Permission denied) 2024-03-21T10:28:38.474 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 57376 57386 admin_soc ceph cwd unknown /proc/57376/task/57386/cwd (readlink: Permission denied) 2024-03-21T10:28:38.474 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 57376 57386 admin_soc ceph rtd unknown /proc/57376/task/57386/root (readlink: Permission denied) 2024-03-21T10:28:38.474 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 57376 57386 admin_soc ceph txt unknown /proc/57376/task/57386/exe (readlink: Permission denied) 2024-03-21T10:28:38.474 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 57376 57386 admin_soc ceph NOFD /proc/57376/task/57386/fd (opendir: Permission denied) 2024-03-21T10:28:38.474 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 57376 57531 signal_ha ceph cwd unknown /proc/57376/task/57531/cwd (readlink: Permission denied) 2024-03-21T10:28:38.474 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 57376 57531 signal_ha ceph rtd unknown /proc/57376/task/57531/root (readlink: Permission denied) 2024-03-21T10:28:38.474 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 57376 57531 signal_ha ceph txt unknown /proc/57376/task/57531/exe (readlink: Permission denied) 2024-03-21T10:28:38.474 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 57376 57531 signal_ha ceph NOFD /proc/57376/task/57531/fd (opendir: Permission denied) 2024-03-21T10:28:38.474 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 57376 57532 io_contex ceph cwd unknown /proc/57376/task/57532/cwd (readlink: Permission denied) 2024-03-21T10:28:38.474 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 57376 57532 io_contex ceph rtd unknown /proc/57376/task/57532/root (readlink: Permission denied) 2024-03-21T10:28:38.474 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 57376 57532 io_contex ceph txt unknown /proc/57376/task/57532/exe (readlink: Permission denied) 2024-03-21T10:28:38.474 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 57376 57532 io_contex ceph NOFD /proc/57376/task/57532/fd (opendir: Permission denied) 2024-03-21T10:28:38.475 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 57376 57533 io_contex ceph cwd unknown /proc/57376/task/57533/cwd (readlink: Permission denied) 2024-03-21T10:28:38.475 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 57376 57533 io_contex ceph rtd unknown /proc/57376/task/57533/root (readlink: Permission denied) 2024-03-21T10:28:38.475 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 57376 57533 io_contex ceph txt unknown /proc/57376/task/57533/exe (readlink: Permission denied) 2024-03-21T10:28:38.475 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 57376 57533 io_contex ceph NOFD /proc/57376/task/57533/fd (opendir: Permission denied) 2024-03-21T10:28:38.475 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 57376 57541 OpHistory ceph cwd unknown /proc/57376/task/57541/cwd (readlink: Permission denied) 2024-03-21T10:28:38.475 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 57376 57541 OpHistory ceph rtd unknown /proc/57376/task/57541/root (readlink: Permission denied) 2024-03-21T10:28:38.475 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 57376 57541 OpHistory ceph txt unknown /proc/57376/task/57541/exe (readlink: Permission denied) 2024-03-21T10:28:38.475 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 57376 57541 OpHistory ceph NOFD /proc/57376/task/57541/fd (opendir: Permission denied) 2024-03-21T10:28:38.475 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 57376 57542 ceph_time ceph cwd unknown /proc/57376/task/57542/cwd (readlink: Permission denied) 2024-03-21T10:28:38.475 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 57376 57542 ceph_time ceph rtd unknown /proc/57376/task/57542/root (readlink: Permission denied) 2024-03-21T10:28:38.475 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 57376 57542 ceph_time ceph txt unknown /proc/57376/task/57542/exe (readlink: Permission denied) 2024-03-21T10:28:38.475 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 57376 57542 ceph_time ceph NOFD /proc/57376/task/57542/fd (opendir: Permission denied) 2024-03-21T10:28:38.475 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 57376 57650 ceph-osd ceph cwd unknown /proc/57376/task/57650/cwd (readlink: Permission denied) 2024-03-21T10:28:38.475 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 57376 57650 ceph-osd ceph rtd unknown /proc/57376/task/57650/root (readlink: Permission denied) 2024-03-21T10:28:38.476 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 57376 57650 ceph-osd ceph txt unknown /proc/57376/task/57650/exe (readlink: Permission denied) 2024-03-21T10:28:38.476 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 57376 57650 ceph-osd ceph NOFD /proc/57376/task/57650/fd (opendir: Permission denied) 2024-03-21T10:28:38.476 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 57376 57671 ceph-osd ceph cwd unknown /proc/57376/task/57671/cwd (readlink: Permission denied) 2024-03-21T10:28:38.476 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 57376 57671 ceph-osd ceph rtd unknown /proc/57376/task/57671/root (readlink: Permission denied) 2024-03-21T10:28:38.476 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 57376 57671 ceph-osd ceph txt unknown /proc/57376/task/57671/exe (readlink: Permission denied) 2024-03-21T10:28:38.476 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 57376 57671 ceph-osd ceph NOFD /proc/57376/task/57671/fd (opendir: Permission denied) 2024-03-21T10:28:38.476 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 57376 57701 ceph-osd ceph cwd unknown /proc/57376/task/57701/cwd (readlink: Permission denied) 2024-03-21T10:28:38.476 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 57376 57701 ceph-osd ceph rtd unknown /proc/57376/task/57701/root (readlink: Permission denied) 2024-03-21T10:28:38.476 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 57376 57701 ceph-osd ceph txt unknown /proc/57376/task/57701/exe (readlink: Permission denied) 2024-03-21T10:28:38.476 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 57376 57701 ceph-osd ceph NOFD /proc/57376/task/57701/fd (opendir: Permission denied) 2024-03-21T10:28:38.476 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 57376 57714 ceph-osd ceph cwd unknown /proc/57376/task/57714/cwd (readlink: Permission denied) 2024-03-21T10:28:38.476 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 57376 57714 ceph-osd ceph rtd unknown /proc/57376/task/57714/root (readlink: Permission denied) 2024-03-21T10:28:38.476 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 57376 57714 ceph-osd ceph txt unknown /proc/57376/task/57714/exe (readlink: Permission denied) 2024-03-21T10:28:38.477 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 57376 57714 ceph-osd ceph NOFD /proc/57376/task/57714/fd (opendir: Permission denied) 2024-03-21T10:28:38.477 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 57376 57724 ceph-osd ceph cwd unknown /proc/57376/task/57724/cwd (readlink: Permission denied) 2024-03-21T10:28:38.477 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 57376 57724 ceph-osd ceph rtd unknown /proc/57376/task/57724/root (readlink: Permission denied) 2024-03-21T10:28:38.477 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 57376 57724 ceph-osd ceph txt unknown /proc/57376/task/57724/exe (readlink: Permission denied) 2024-03-21T10:28:38.477 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 57376 57724 ceph-osd ceph NOFD /proc/57376/task/57724/fd (opendir: Permission denied) 2024-03-21T10:28:38.477 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 57376 57742 ceph-osd ceph cwd unknown /proc/57376/task/57742/cwd (readlink: Permission denied) 2024-03-21T10:28:38.477 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 57376 57742 ceph-osd ceph rtd unknown /proc/57376/task/57742/root (readlink: Permission denied) 2024-03-21T10:28:38.477 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 57376 57742 ceph-osd ceph txt unknown /proc/57376/task/57742/exe (readlink: Permission denied) 2024-03-21T10:28:38.477 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 57376 57742 ceph-osd ceph NOFD /proc/57376/task/57742/fd (opendir: Permission denied) 2024-03-21T10:28:38.477 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 57376 57748 ceph-osd ceph cwd unknown /proc/57376/task/57748/cwd (readlink: Permission denied) 2024-03-21T10:28:38.477 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 57376 57748 ceph-osd ceph rtd unknown /proc/57376/task/57748/root (readlink: Permission denied) 2024-03-21T10:28:38.477 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 57376 57748 ceph-osd ceph txt unknown /proc/57376/task/57748/exe (readlink: Permission denied) 2024-03-21T10:28:38.477 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 57376 57748 ceph-osd ceph NOFD /proc/57376/task/57748/fd (opendir: Permission denied) 2024-03-21T10:28:38.477 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 57376 57766 ceph-osd ceph cwd unknown /proc/57376/task/57766/cwd (readlink: Permission denied) 2024-03-21T10:28:38.478 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 57376 57766 ceph-osd ceph rtd unknown /proc/57376/task/57766/root (readlink: Permission denied) 2024-03-21T10:28:38.478 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 57376 57766 ceph-osd ceph txt unknown /proc/57376/task/57766/exe (readlink: Permission denied) 2024-03-21T10:28:38.478 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 57376 57766 ceph-osd ceph NOFD /proc/57376/task/57766/fd (opendir: Permission denied) 2024-03-21T10:28:38.478 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 57376 57767 ceph-osd ceph cwd unknown /proc/57376/task/57767/cwd (readlink: Permission denied) 2024-03-21T10:28:38.478 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 57376 57767 ceph-osd ceph rtd unknown /proc/57376/task/57767/root (readlink: Permission denied) 2024-03-21T10:28:38.478 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 57376 57767 ceph-osd ceph txt unknown /proc/57376/task/57767/exe (readlink: Permission denied) 2024-03-21T10:28:38.478 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 57376 57767 ceph-osd ceph NOFD /proc/57376/task/57767/fd (opendir: Permission denied) 2024-03-21T10:28:38.478 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 57376 57768 safe_time ceph cwd unknown /proc/57376/task/57768/cwd (readlink: Permission denied) 2024-03-21T10:28:38.478 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 57376 57768 safe_time ceph rtd unknown /proc/57376/task/57768/root (readlink: Permission denied) 2024-03-21T10:28:38.478 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 57376 57768 safe_time ceph txt unknown /proc/57376/task/57768/exe (readlink: Permission denied) 2024-03-21T10:28:38.478 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 57376 57768 safe_time ceph NOFD /proc/57376/task/57768/fd (opendir: Permission denied) 2024-03-21T10:28:38.478 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 57376 57769 safe_time ceph cwd unknown /proc/57376/task/57769/cwd (readlink: Permission denied) 2024-03-21T10:28:38.478 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 57376 57769 safe_time ceph rtd unknown /proc/57376/task/57769/root (readlink: Permission denied) 2024-03-21T10:28:38.479 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 57376 57769 safe_time ceph txt unknown /proc/57376/task/57769/exe (readlink: Permission denied) 2024-03-21T10:28:38.479 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 57376 57769 safe_time ceph NOFD /proc/57376/task/57769/fd (opendir: Permission denied) 2024-03-21T10:28:38.479 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 57376 57771 safe_time ceph cwd unknown /proc/57376/task/57771/cwd (readlink: Permission denied) 2024-03-21T10:28:38.479 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 57376 57771 safe_time ceph rtd unknown /proc/57376/task/57771/root (readlink: Permission denied) 2024-03-21T10:28:38.479 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 57376 57771 safe_time ceph txt unknown /proc/57376/task/57771/exe (readlink: Permission denied) 2024-03-21T10:28:38.479 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 57376 57771 safe_time ceph NOFD /proc/57376/task/57771/fd (opendir: Permission denied) 2024-03-21T10:28:38.479 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 57376 57772 safe_time ceph cwd unknown /proc/57376/task/57772/cwd (readlink: Permission denied) 2024-03-21T10:28:38.479 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 57376 57772 safe_time ceph rtd unknown /proc/57376/task/57772/root (readlink: Permission denied) 2024-03-21T10:28:38.479 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 57376 57772 safe_time ceph txt unknown /proc/57376/task/57772/exe (readlink: Permission denied) 2024-03-21T10:28:38.479 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 57376 57772 safe_time ceph NOFD /proc/57376/task/57772/fd (opendir: Permission denied) 2024-03-21T10:28:38.479 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 57376 57773 fn_anonym ceph cwd unknown /proc/57376/task/57773/cwd (readlink: Permission denied) 2024-03-21T10:28:38.479 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 57376 57773 fn_anonym ceph rtd unknown /proc/57376/task/57773/root (readlink: Permission denied) 2024-03-21T10:28:38.479 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 57376 57773 fn_anonym ceph txt unknown /proc/57376/task/57773/exe (readlink: Permission denied) 2024-03-21T10:28:38.479 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 57376 57773 fn_anonym ceph NOFD /proc/57376/task/57773/fd (opendir: Permission denied) 2024-03-21T10:28:38.480 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 57376 57777 rocksdb:l ceph cwd unknown /proc/57376/task/57777/cwd (readlink: Permission denied) 2024-03-21T10:28:38.480 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 57376 57777 rocksdb:l ceph rtd unknown /proc/57376/task/57777/root (readlink: Permission denied) 2024-03-21T10:28:38.480 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 57376 57777 rocksdb:l ceph txt unknown /proc/57376/task/57777/exe (readlink: Permission denied) 2024-03-21T10:28:38.480 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 57376 57777 rocksdb:l ceph NOFD /proc/57376/task/57777/fd (opendir: Permission denied) 2024-03-21T10:28:38.480 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 57376 57778 rocksdb:l ceph cwd unknown /proc/57376/task/57778/cwd (readlink: Permission denied) 2024-03-21T10:28:38.480 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 57376 57778 rocksdb:l ceph rtd unknown /proc/57376/task/57778/root (readlink: Permission denied) 2024-03-21T10:28:38.480 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 57376 57778 rocksdb:l ceph txt unknown /proc/57376/task/57778/exe (readlink: Permission denied) 2024-03-21T10:28:38.480 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 57376 57778 rocksdb:l ceph NOFD /proc/57376/task/57778/fd (opendir: Permission denied) 2024-03-21T10:28:38.480 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 57376 57779 rocksdb:l ceph cwd unknown /proc/57376/task/57779/cwd (readlink: Permission denied) 2024-03-21T10:28:38.480 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 57376 57779 rocksdb:l ceph rtd unknown /proc/57376/task/57779/root (readlink: Permission denied) 2024-03-21T10:28:38.480 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 57376 57779 rocksdb:l ceph txt unknown /proc/57376/task/57779/exe (readlink: Permission denied) 2024-03-21T10:28:38.480 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 57376 57779 rocksdb:l ceph NOFD /proc/57376/task/57779/fd (opendir: Permission denied) 2024-03-21T10:28:38.480 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 57376 57780 rocksdb:h ceph cwd unknown /proc/57376/task/57780/cwd (readlink: Permission denied) 2024-03-21T10:28:38.480 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 57376 57780 rocksdb:h ceph rtd unknown /proc/57376/task/57780/root (readlink: Permission denied) 2024-03-21T10:28:38.480 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 57376 57780 rocksdb:h ceph txt unknown /proc/57376/task/57780/exe (readlink: Permission denied) 2024-03-21T10:28:38.480 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 57376 57780 rocksdb:h ceph NOFD /proc/57376/task/57780/fd (opendir: Permission denied) 2024-03-21T10:28:38.480 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 57376 58213 bstore_ai ceph cwd unknown /proc/57376/task/58213/cwd (readlink: Permission denied) 2024-03-21T10:28:38.481 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 57376 58213 bstore_ai ceph rtd unknown /proc/57376/task/58213/root (readlink: Permission denied) 2024-03-21T10:28:38.481 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 57376 58213 bstore_ai ceph txt unknown /proc/57376/task/58213/exe (readlink: Permission denied) 2024-03-21T10:28:38.481 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 57376 58213 bstore_ai ceph NOFD /proc/57376/task/58213/fd (opendir: Permission denied) 2024-03-21T10:28:38.481 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 57376 58420 bstore_ai ceph cwd unknown /proc/57376/task/58420/cwd (readlink: Permission denied) 2024-03-21T10:28:38.481 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 57376 58420 bstore_ai ceph rtd unknown /proc/57376/task/58420/root (readlink: Permission denied) 2024-03-21T10:28:38.481 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 57376 58420 bstore_ai ceph txt unknown /proc/57376/task/58420/exe (readlink: Permission denied) 2024-03-21T10:28:38.481 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 57376 58420 bstore_ai ceph NOFD /proc/57376/task/58420/fd (opendir: Permission denied) 2024-03-21T10:28:38.481 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 57376 58604 ceph-osd ceph cwd unknown /proc/57376/task/58604/cwd (readlink: Permission denied) 2024-03-21T10:28:38.481 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 57376 58604 ceph-osd ceph rtd unknown /proc/57376/task/58604/root (readlink: Permission denied) 2024-03-21T10:28:38.481 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 57376 58604 ceph-osd ceph txt unknown /proc/57376/task/58604/exe (readlink: Permission denied) 2024-03-21T10:28:38.481 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 57376 58604 ceph-osd ceph NOFD /proc/57376/task/58604/fd (opendir: Permission denied) 2024-03-21T10:28:38.481 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 57376 58605 cfin ceph cwd unknown /proc/57376/task/58605/cwd (readlink: Permission denied) 2024-03-21T10:28:38.481 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 57376 58605 cfin ceph rtd unknown /proc/57376/task/58605/root (readlink: Permission denied) 2024-03-21T10:28:38.481 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 57376 58605 cfin ceph txt unknown /proc/57376/task/58605/exe (readlink: Permission denied) 2024-03-21T10:28:38.481 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 57376 58605 cfin ceph NOFD /proc/57376/task/58605/fd (opendir: Permission denied) 2024-03-21T10:28:38.482 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 57376 58606 bstore_kv ceph cwd unknown /proc/57376/task/58606/cwd (readlink: Permission denied) 2024-03-21T10:28:38.482 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 57376 58606 bstore_kv ceph rtd unknown /proc/57376/task/58606/root (readlink: Permission denied) 2024-03-21T10:28:38.482 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 57376 58606 bstore_kv ceph txt unknown /proc/57376/task/58606/exe (readlink: Permission denied) 2024-03-21T10:28:38.482 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 57376 58606 bstore_kv ceph NOFD /proc/57376/task/58606/fd (opendir: Permission denied) 2024-03-21T10:28:38.482 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 57376 58607 bstore_kv ceph cwd unknown /proc/57376/task/58607/cwd (readlink: Permission denied) 2024-03-21T10:28:38.482 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 57376 58607 bstore_kv ceph rtd unknown /proc/57376/task/58607/root (readlink: Permission denied) 2024-03-21T10:28:38.482 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 57376 58607 bstore_kv ceph txt unknown /proc/57376/task/58607/exe (readlink: Permission denied) 2024-03-21T10:28:38.482 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 57376 58607 bstore_kv ceph NOFD /proc/57376/task/58607/fd (opendir: Permission denied) 2024-03-21T10:28:38.482 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 57376 58608 bstore_me ceph cwd unknown /proc/57376/task/58608/cwd (readlink: Permission denied) 2024-03-21T10:28:38.482 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 57376 58608 bstore_me ceph rtd unknown /proc/57376/task/58608/root (readlink: Permission denied) 2024-03-21T10:28:38.482 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 57376 58608 bstore_me ceph txt unknown /proc/57376/task/58608/exe (readlink: Permission denied) 2024-03-21T10:28:38.482 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 57376 58608 bstore_me ceph NOFD /proc/57376/task/58608/fd (opendir: Permission denied) 2024-03-21T10:28:38.482 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 57376 58623 ms_dispat ceph cwd unknown /proc/57376/task/58623/cwd (readlink: Permission denied) 2024-03-21T10:28:38.483 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 57376 58623 ms_dispat ceph rtd unknown /proc/57376/task/58623/root (readlink: Permission denied) 2024-03-21T10:28:38.483 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 57376 58623 ms_dispat ceph txt unknown /proc/57376/task/58623/exe (readlink: Permission denied) 2024-03-21T10:28:38.483 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 57376 58623 ms_dispat ceph NOFD /proc/57376/task/58623/fd (opendir: Permission denied) 2024-03-21T10:28:38.483 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 57376 58624 ms_local ceph cwd unknown /proc/57376/task/58624/cwd (readlink: Permission denied) 2024-03-21T10:28:38.483 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 57376 58624 ms_local ceph rtd unknown /proc/57376/task/58624/root (readlink: Permission denied) 2024-03-21T10:28:38.483 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 57376 58624 ms_local ceph txt unknown /proc/57376/task/58624/exe (readlink: Permission denied) 2024-03-21T10:28:38.483 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 57376 58624 ms_local ceph NOFD /proc/57376/task/58624/fd (opendir: Permission denied) 2024-03-21T10:28:38.483 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 57376 58625 safe_time ceph cwd unknown /proc/57376/task/58625/cwd (readlink: Permission denied) 2024-03-21T10:28:38.483 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 57376 58625 safe_time ceph rtd unknown /proc/57376/task/58625/root (readlink: Permission denied) 2024-03-21T10:28:38.483 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 57376 58625 safe_time ceph txt unknown /proc/57376/task/58625/exe (readlink: Permission denied) 2024-03-21T10:28:38.483 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 57376 58625 safe_time ceph NOFD /proc/57376/task/58625/fd (opendir: Permission denied) 2024-03-21T10:28:38.483 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 57376 58626 safe_time ceph cwd unknown /proc/57376/task/58626/cwd (readlink: Permission denied) 2024-03-21T10:28:38.484 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 57376 58626 safe_time ceph rtd unknown /proc/57376/task/58626/root (readlink: Permission denied) 2024-03-21T10:28:38.484 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 57376 58626 safe_time ceph txt unknown /proc/57376/task/58626/exe (readlink: Permission denied) 2024-03-21T10:28:38.484 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 57376 58626 safe_time ceph NOFD /proc/57376/task/58626/fd (opendir: Permission denied) 2024-03-21T10:28:38.484 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 57376 58627 ms_dispat ceph cwd unknown /proc/57376/task/58627/cwd (readlink: Permission denied) 2024-03-21T10:28:38.484 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 57376 58627 ms_dispat ceph rtd unknown /proc/57376/task/58627/root (readlink: Permission denied) 2024-03-21T10:28:38.484 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 57376 58627 ms_dispat ceph txt unknown /proc/57376/task/58627/exe (readlink: Permission denied) 2024-03-21T10:28:38.484 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 57376 58627 ms_dispat ceph NOFD /proc/57376/task/58627/fd (opendir: Permission denied) 2024-03-21T10:28:38.484 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 57376 58628 ms_local ceph cwd unknown /proc/57376/task/58628/cwd (readlink: Permission denied) 2024-03-21T10:28:38.484 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 57376 58628 ms_local ceph rtd unknown /proc/57376/task/58628/root (readlink: Permission denied) 2024-03-21T10:28:38.484 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 57376 58628 ms_local ceph txt unknown /proc/57376/task/58628/exe (readlink: Permission denied) 2024-03-21T10:28:38.484 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 57376 58628 ms_local ceph NOFD /proc/57376/task/58628/fd (opendir: Permission denied) 2024-03-21T10:28:38.484 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 57376 58629 ms_dispat ceph cwd unknown /proc/57376/task/58629/cwd (readlink: Permission denied) 2024-03-21T10:28:38.484 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 57376 58629 ms_dispat ceph rtd unknown /proc/57376/task/58629/root (readlink: Permission denied) 2024-03-21T10:28:38.484 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 57376 58629 ms_dispat ceph txt unknown /proc/57376/task/58629/exe (readlink: Permission denied) 2024-03-21T10:28:38.485 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 57376 58629 ms_dispat ceph NOFD /proc/57376/task/58629/fd (opendir: Permission denied) 2024-03-21T10:28:38.485 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 57376 58630 ms_local ceph cwd unknown /proc/57376/task/58630/cwd (readlink: Permission denied) 2024-03-21T10:28:38.485 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 57376 58630 ms_local ceph rtd unknown /proc/57376/task/58630/root (readlink: Permission denied) 2024-03-21T10:28:38.485 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 57376 58630 ms_local ceph txt unknown /proc/57376/task/58630/exe (readlink: Permission denied) 2024-03-21T10:28:38.485 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 57376 58630 ms_local ceph NOFD /proc/57376/task/58630/fd (opendir: Permission denied) 2024-03-21T10:28:38.485 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 57376 58631 ms_dispat ceph cwd unknown /proc/57376/task/58631/cwd (readlink: Permission denied) 2024-03-21T10:28:38.485 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 57376 58631 ms_dispat ceph rtd unknown /proc/57376/task/58631/root (readlink: Permission denied) 2024-03-21T10:28:38.485 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 57376 58631 ms_dispat ceph txt unknown /proc/57376/task/58631/exe (readlink: Permission denied) 2024-03-21T10:28:38.485 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 57376 58631 ms_dispat ceph NOFD /proc/57376/task/58631/fd (opendir: Permission denied) 2024-03-21T10:28:38.485 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 57376 58632 ms_local ceph cwd unknown /proc/57376/task/58632/cwd (readlink: Permission denied) 2024-03-21T10:28:38.485 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 57376 58632 ms_local ceph rtd unknown /proc/57376/task/58632/root (readlink: Permission denied) 2024-03-21T10:28:38.485 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 57376 58632 ms_local ceph txt unknown /proc/57376/task/58632/exe (readlink: Permission denied) 2024-03-21T10:28:38.485 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 57376 58632 ms_local ceph NOFD /proc/57376/task/58632/fd (opendir: Permission denied) 2024-03-21T10:28:38.485 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 57376 58633 ms_dispat ceph cwd unknown /proc/57376/task/58633/cwd (readlink: Permission denied) 2024-03-21T10:28:38.486 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 57376 58633 ms_dispat ceph rtd unknown /proc/57376/task/58633/root (readlink: Permission denied) 2024-03-21T10:28:38.486 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 57376 58633 ms_dispat ceph txt unknown /proc/57376/task/58633/exe (readlink: Permission denied) 2024-03-21T10:28:38.486 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 57376 58633 ms_dispat ceph NOFD /proc/57376/task/58633/fd (opendir: Permission denied) 2024-03-21T10:28:38.486 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 57376 58634 ms_local ceph cwd unknown /proc/57376/task/58634/cwd (readlink: Permission denied) 2024-03-21T10:28:38.486 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 57376 58634 ms_local ceph rtd unknown /proc/57376/task/58634/root (readlink: Permission denied) 2024-03-21T10:28:38.486 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 57376 58634 ms_local ceph txt unknown /proc/57376/task/58634/exe (readlink: Permission denied) 2024-03-21T10:28:38.486 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 57376 58634 ms_local ceph NOFD /proc/57376/task/58634/fd (opendir: Permission denied) 2024-03-21T10:28:38.486 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 57376 58635 ms_dispat ceph cwd unknown /proc/57376/task/58635/cwd (readlink: Permission denied) 2024-03-21T10:28:38.486 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 57376 58635 ms_dispat ceph rtd unknown /proc/57376/task/58635/root (readlink: Permission denied) 2024-03-21T10:28:38.486 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 57376 58635 ms_dispat ceph txt unknown /proc/57376/task/58635/exe (readlink: Permission denied) 2024-03-21T10:28:38.486 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 57376 58635 ms_dispat ceph NOFD /proc/57376/task/58635/fd (opendir: Permission denied) 2024-03-21T10:28:38.486 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 57376 58636 ms_local ceph cwd unknown /proc/57376/task/58636/cwd (readlink: Permission denied) 2024-03-21T10:28:38.487 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 57376 58636 ms_local ceph rtd unknown /proc/57376/task/58636/root (readlink: Permission denied) 2024-03-21T10:28:38.487 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 57376 58636 ms_local ceph txt unknown /proc/57376/task/58636/exe (readlink: Permission denied) 2024-03-21T10:28:38.487 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 57376 58636 ms_local ceph NOFD /proc/57376/task/58636/fd (opendir: Permission denied) 2024-03-21T10:28:38.487 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 57376 58637 ms_dispat ceph cwd unknown /proc/57376/task/58637/cwd (readlink: Permission denied) 2024-03-21T10:28:38.487 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 57376 58637 ms_dispat ceph rtd unknown /proc/57376/task/58637/root (readlink: Permission denied) 2024-03-21T10:28:38.487 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 57376 58637 ms_dispat ceph txt unknown /proc/57376/task/58637/exe (readlink: Permission denied) 2024-03-21T10:28:38.487 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 57376 58637 ms_dispat ceph NOFD /proc/57376/task/58637/fd (opendir: Permission denied) 2024-03-21T10:28:38.487 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 57376 58639 ms_local ceph cwd unknown /proc/57376/task/58639/cwd (readlink: Permission denied) 2024-03-21T10:28:38.487 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 57376 58639 ms_local ceph rtd unknown /proc/57376/task/58639/root (readlink: Permission denied) 2024-03-21T10:28:38.487 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 57376 58639 ms_local ceph txt unknown /proc/57376/task/58639/exe (readlink: Permission denied) 2024-03-21T10:28:38.487 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 57376 58639 ms_local ceph NOFD /proc/57376/task/58639/fd (opendir: Permission denied) 2024-03-21T10:28:38.487 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 57376 58640 fn_anonym ceph cwd unknown /proc/57376/task/58640/cwd (readlink: Permission denied) 2024-03-21T10:28:38.487 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 57376 58640 fn_anonym ceph rtd unknown /proc/57376/task/58640/root (readlink: Permission denied) 2024-03-21T10:28:38.488 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 57376 58640 fn_anonym ceph txt unknown /proc/57376/task/58640/exe (readlink: Permission denied) 2024-03-21T10:28:38.488 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 57376 58640 fn_anonym ceph NOFD /proc/57376/task/58640/fd (opendir: Permission denied) 2024-03-21T10:28:38.488 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 57376 58641 finisher ceph cwd unknown /proc/57376/task/58641/cwd (readlink: Permission denied) 2024-03-21T10:28:38.488 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 57376 58641 finisher ceph rtd unknown /proc/57376/task/58641/root (readlink: Permission denied) 2024-03-21T10:28:38.488 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 57376 58641 finisher ceph txt unknown /proc/57376/task/58641/exe (readlink: Permission denied) 2024-03-21T10:28:38.488 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 57376 58641 finisher ceph NOFD /proc/57376/task/58641/fd (opendir: Permission denied) 2024-03-21T10:28:38.488 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 57376 58642 safe_time ceph cwd unknown /proc/57376/task/58642/cwd (readlink: Permission denied) 2024-03-21T10:28:38.488 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 57376 58642 safe_time ceph rtd unknown /proc/57376/task/58642/root (readlink: Permission denied) 2024-03-21T10:28:38.488 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 57376 58642 safe_time ceph txt unknown /proc/57376/task/58642/exe (readlink: Permission denied) 2024-03-21T10:28:38.488 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 57376 58642 safe_time ceph NOFD /proc/57376/task/58642/fd (opendir: Permission denied) 2024-03-21T10:28:38.488 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 57376 58643 safe_time ceph cwd unknown /proc/57376/task/58643/cwd (readlink: Permission denied) 2024-03-21T10:28:38.488 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 57376 58643 safe_time ceph rtd unknown /proc/57376/task/58643/root (readlink: Permission denied) 2024-03-21T10:28:38.488 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 57376 58643 safe_time ceph txt unknown /proc/57376/task/58643/exe (readlink: Permission denied) 2024-03-21T10:28:38.488 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 57376 58643 safe_time ceph NOFD /proc/57376/task/58643/fd (opendir: Permission denied) 2024-03-21T10:28:38.489 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 57376 58644 ceph-osd ceph cwd unknown /proc/57376/task/58644/cwd (readlink: Permission denied) 2024-03-21T10:28:38.489 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 57376 58644 ceph-osd ceph rtd unknown /proc/57376/task/58644/root (readlink: Permission denied) 2024-03-21T10:28:38.489 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 57376 58644 ceph-osd ceph txt unknown /proc/57376/task/58644/exe (readlink: Permission denied) 2024-03-21T10:28:38.489 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 57376 58644 ceph-osd ceph NOFD /proc/57376/task/58644/fd (opendir: Permission denied) 2024-03-21T10:28:38.489 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 57376 58645 osd_srv_a ceph cwd unknown /proc/57376/task/58645/cwd (readlink: Permission denied) 2024-03-21T10:28:38.489 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 57376 58645 osd_srv_a ceph rtd unknown /proc/57376/task/58645/root (readlink: Permission denied) 2024-03-21T10:28:38.489 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 57376 58645 osd_srv_a ceph txt unknown /proc/57376/task/58645/exe (readlink: Permission denied) 2024-03-21T10:28:38.489 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 57376 58645 osd_srv_a ceph NOFD /proc/57376/task/58645/fd (opendir: Permission denied) 2024-03-21T10:28:38.489 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 57376 58646 tp_osd_tp ceph cwd unknown /proc/57376/task/58646/cwd (readlink: Permission denied) 2024-03-21T10:28:38.489 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 57376 58646 tp_osd_tp ceph rtd unknown /proc/57376/task/58646/root (readlink: Permission denied) 2024-03-21T10:28:38.489 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 57376 58646 tp_osd_tp ceph txt unknown /proc/57376/task/58646/exe (readlink: Permission denied) 2024-03-21T10:28:38.489 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 57376 58646 tp_osd_tp ceph NOFD /proc/57376/task/58646/fd (opendir: Permission denied) 2024-03-21T10:28:38.489 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 57376 58647 tp_osd_tp ceph cwd unknown /proc/57376/task/58647/cwd (readlink: Permission denied) 2024-03-21T10:28:38.489 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 57376 58647 tp_osd_tp ceph rtd unknown /proc/57376/task/58647/root (readlink: Permission denied) 2024-03-21T10:28:38.490 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 57376 58647 tp_osd_tp ceph txt unknown /proc/57376/task/58647/exe (readlink: Permission denied) 2024-03-21T10:28:38.490 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 57376 58647 tp_osd_tp ceph NOFD /proc/57376/task/58647/fd (opendir: Permission denied) 2024-03-21T10:28:38.490 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 57376 58648 tp_osd_tp ceph cwd unknown /proc/57376/task/58648/cwd (readlink: Permission denied) 2024-03-21T10:28:38.490 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 57376 58648 tp_osd_tp ceph rtd unknown /proc/57376/task/58648/root (readlink: Permission denied) 2024-03-21T10:28:38.490 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 57376 58648 tp_osd_tp ceph txt unknown /proc/57376/task/58648/exe (readlink: Permission denied) 2024-03-21T10:28:38.490 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 57376 58648 tp_osd_tp ceph NOFD /proc/57376/task/58648/fd (opendir: Permission denied) 2024-03-21T10:28:38.490 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 57376 58649 tp_osd_tp ceph cwd unknown /proc/57376/task/58649/cwd (readlink: Permission denied) 2024-03-21T10:28:38.490 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 57376 58649 tp_osd_tp ceph rtd unknown /proc/57376/task/58649/root (readlink: Permission denied) 2024-03-21T10:28:38.490 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 57376 58649 tp_osd_tp ceph txt unknown /proc/57376/task/58649/exe (readlink: Permission denied) 2024-03-21T10:28:38.490 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 57376 58649 tp_osd_tp ceph NOFD /proc/57376/task/58649/fd (opendir: Permission denied) 2024-03-21T10:28:38.490 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 57376 58650 tp_osd_tp ceph cwd unknown /proc/57376/task/58650/cwd (readlink: Permission denied) 2024-03-21T10:28:38.490 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 57376 58650 tp_osd_tp ceph rtd unknown /proc/57376/task/58650/root (readlink: Permission denied) 2024-03-21T10:28:38.491 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 57376 58650 tp_osd_tp ceph txt unknown /proc/57376/task/58650/exe (readlink: Permission denied) 2024-03-21T10:28:38.491 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 57376 58650 tp_osd_tp ceph NOFD /proc/57376/task/58650/fd (opendir: Permission denied) 2024-03-21T10:28:38.491 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 57376 58651 tp_osd_tp ceph cwd unknown /proc/57376/task/58651/cwd (readlink: Permission denied) 2024-03-21T10:28:38.491 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 57376 58651 tp_osd_tp ceph rtd unknown /proc/57376/task/58651/root (readlink: Permission denied) 2024-03-21T10:28:38.491 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 57376 58651 tp_osd_tp ceph txt unknown /proc/57376/task/58651/exe (readlink: Permission denied) 2024-03-21T10:28:38.491 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 57376 58651 tp_osd_tp ceph NOFD /proc/57376/task/58651/fd (opendir: Permission denied) 2024-03-21T10:28:38.491 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 57376 58652 tp_osd_tp ceph cwd unknown /proc/57376/task/58652/cwd (readlink: Permission denied) 2024-03-21T10:28:38.491 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 57376 58652 tp_osd_tp ceph rtd unknown /proc/57376/task/58652/root (readlink: Permission denied) 2024-03-21T10:28:38.491 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 57376 58652 tp_osd_tp ceph txt unknown /proc/57376/task/58652/exe (readlink: Permission denied) 2024-03-21T10:28:38.491 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 57376 58652 tp_osd_tp ceph NOFD /proc/57376/task/58652/fd (opendir: Permission denied) 2024-03-21T10:28:38.491 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 57376 58653 tp_osd_tp ceph cwd unknown /proc/57376/task/58653/cwd (readlink: Permission denied) 2024-03-21T10:28:38.491 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 57376 58653 tp_osd_tp ceph rtd unknown /proc/57376/task/58653/root (readlink: Permission denied) 2024-03-21T10:28:38.491 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 57376 58653 tp_osd_tp ceph txt unknown /proc/57376/task/58653/exe (readlink: Permission denied) 2024-03-21T10:28:38.491 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 57376 58653 tp_osd_tp ceph NOFD /proc/57376/task/58653/fd (opendir: Permission denied) 2024-03-21T10:28:38.491 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 57376 58654 tp_osd_tp ceph cwd unknown /proc/57376/task/58654/cwd (readlink: Permission denied) 2024-03-21T10:28:38.492 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 57376 58654 tp_osd_tp ceph rtd unknown /proc/57376/task/58654/root (readlink: Permission denied) 2024-03-21T10:28:38.492 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 57376 58654 tp_osd_tp ceph txt unknown /proc/57376/task/58654/exe (readlink: Permission denied) 2024-03-21T10:28:38.492 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 57376 58654 tp_osd_tp ceph NOFD /proc/57376/task/58654/fd (opendir: Permission denied) 2024-03-21T10:28:38.492 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 57376 58655 tp_osd_tp ceph cwd unknown /proc/57376/task/58655/cwd (readlink: Permission denied) 2024-03-21T10:28:38.492 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 57376 58655 tp_osd_tp ceph rtd unknown /proc/57376/task/58655/root (readlink: Permission denied) 2024-03-21T10:28:38.492 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 57376 58655 tp_osd_tp ceph txt unknown /proc/57376/task/58655/exe (readlink: Permission denied) 2024-03-21T10:28:38.492 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 57376 58655 tp_osd_tp ceph NOFD /proc/57376/task/58655/fd (opendir: Permission denied) 2024-03-21T10:28:38.492 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 57376 58656 tp_osd_tp ceph cwd unknown /proc/57376/task/58656/cwd (readlink: Permission denied) 2024-03-21T10:28:38.492 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 57376 58656 tp_osd_tp ceph rtd unknown /proc/57376/task/58656/root (readlink: Permission denied) 2024-03-21T10:28:38.492 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 57376 58656 tp_osd_tp ceph txt unknown /proc/57376/task/58656/exe (readlink: Permission denied) 2024-03-21T10:28:38.492 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 57376 58656 tp_osd_tp ceph NOFD /proc/57376/task/58656/fd (opendir: Permission denied) 2024-03-21T10:28:38.492 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 57376 58657 tp_osd_tp ceph cwd unknown /proc/57376/task/58657/cwd (readlink: Permission denied) 2024-03-21T10:28:38.492 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 57376 58657 tp_osd_tp ceph rtd unknown /proc/57376/task/58657/root (readlink: Permission denied) 2024-03-21T10:28:38.492 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 57376 58657 tp_osd_tp ceph txt unknown /proc/57376/task/58657/exe (readlink: Permission denied) 2024-03-21T10:28:38.493 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 57376 58657 tp_osd_tp ceph NOFD /proc/57376/task/58657/fd (opendir: Permission denied) 2024-03-21T10:28:38.493 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 57376 58658 tp_osd_tp ceph cwd unknown /proc/57376/task/58658/cwd (readlink: Permission denied) 2024-03-21T10:28:38.493 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 57376 58658 tp_osd_tp ceph rtd unknown /proc/57376/task/58658/root (readlink: Permission denied) 2024-03-21T10:28:38.493 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 57376 58658 tp_osd_tp ceph txt unknown /proc/57376/task/58658/exe (readlink: Permission denied) 2024-03-21T10:28:38.493 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 57376 58658 tp_osd_tp ceph NOFD /proc/57376/task/58658/fd (opendir: Permission denied) 2024-03-21T10:28:38.493 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 57376 58659 tp_osd_tp ceph cwd unknown /proc/57376/task/58659/cwd (readlink: Permission denied) 2024-03-21T10:28:38.493 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 57376 58659 tp_osd_tp ceph rtd unknown /proc/57376/task/58659/root (readlink: Permission denied) 2024-03-21T10:28:38.493 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 57376 58659 tp_osd_tp ceph txt unknown /proc/57376/task/58659/exe (readlink: Permission denied) 2024-03-21T10:28:38.493 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 57376 58659 tp_osd_tp ceph NOFD /proc/57376/task/58659/fd (opendir: Permission denied) 2024-03-21T10:28:38.493 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 57376 58660 tp_osd_tp ceph cwd unknown /proc/57376/task/58660/cwd (readlink: Permission denied) 2024-03-21T10:28:38.493 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 57376 58660 tp_osd_tp ceph rtd unknown /proc/57376/task/58660/root (readlink: Permission denied) 2024-03-21T10:28:38.493 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 57376 58660 tp_osd_tp ceph txt unknown /proc/57376/task/58660/exe (readlink: Permission denied) 2024-03-21T10:28:38.494 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 57376 58660 tp_osd_tp ceph NOFD /proc/57376/task/58660/fd (opendir: Permission denied) 2024-03-21T10:28:38.494 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 57376 58661 tp_osd_tp ceph cwd unknown /proc/57376/task/58661/cwd (readlink: Permission denied) 2024-03-21T10:28:38.494 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 57376 58661 tp_osd_tp ceph rtd unknown /proc/57376/task/58661/root (readlink: Permission denied) 2024-03-21T10:28:38.494 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 57376 58661 tp_osd_tp ceph txt unknown /proc/57376/task/58661/exe (readlink: Permission denied) 2024-03-21T10:28:38.494 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 57376 58661 tp_osd_tp ceph NOFD /proc/57376/task/58661/fd (opendir: Permission denied) 2024-03-21T10:28:38.494 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 57376 58662 osd_srv_h ceph cwd unknown /proc/57376/task/58662/cwd (readlink: Permission denied) 2024-03-21T10:28:38.494 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 57376 58662 osd_srv_h ceph rtd unknown /proc/57376/task/58662/root (readlink: Permission denied) 2024-03-21T10:28:38.494 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 57376 58662 osd_srv_h ceph txt unknown /proc/57376/task/58662/exe (readlink: Permission denied) 2024-03-21T10:28:38.494 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 57376 58662 osd_srv_h ceph NOFD /proc/57376/task/58662/fd (opendir: Permission denied) 2024-03-21T10:28:38.494 INFO:tasks.workunit.client.0.smithi047.stdout:sudo 58820 root cwd unknown /proc/58820/cwd (readlink: Permission denied) 2024-03-21T10:28:38.494 INFO:tasks.workunit.client.0.smithi047.stdout:sudo 58820 root rtd unknown /proc/58820/root (readlink: Permission denied) 2024-03-21T10:28:38.494 INFO:tasks.workunit.client.0.smithi047.stdout:sudo 58820 root txt unknown /proc/58820/exe (readlink: Permission denied) 2024-03-21T10:28:38.494 INFO:tasks.workunit.client.0.smithi047.stdout:sudo 58820 root NOFD /proc/58820/fd (opendir: Permission denied) 2024-03-21T10:28:38.495 INFO:tasks.workunit.client.0.smithi047.stdout:journalct 58850 root cwd unknown /proc/58850/cwd (readlink: Permission denied) 2024-03-21T10:28:38.495 INFO:tasks.workunit.client.0.smithi047.stdout:journalct 58850 root rtd unknown /proc/58850/root (readlink: Permission denied) 2024-03-21T10:28:38.495 INFO:tasks.workunit.client.0.smithi047.stdout:journalct 58850 root txt unknown /proc/58850/exe (readlink: Permission denied) 2024-03-21T10:28:38.495 INFO:tasks.workunit.client.0.smithi047.stdout:journalct 58850 root NOFD /proc/58850/fd (opendir: Permission denied) 2024-03-21T10:28:38.495 INFO:tasks.workunit.client.0.smithi047.stdout:kworker/5 61435 root cwd unknown /proc/61435/cwd (readlink: Permission denied) 2024-03-21T10:28:38.495 INFO:tasks.workunit.client.0.smithi047.stdout:kworker/5 61435 root rtd unknown /proc/61435/root (readlink: Permission denied) 2024-03-21T10:28:38.495 INFO:tasks.workunit.client.0.smithi047.stdout:kworker/5 61435 root txt unknown /proc/61435/exe (readlink: Permission denied) 2024-03-21T10:28:38.495 INFO:tasks.workunit.client.0.smithi047.stdout:kworker/5 61435 root NOFD /proc/61435/fd (opendir: Permission denied) 2024-03-21T10:28:38.495 INFO:tasks.workunit.client.0.smithi047.stdout:kworker/7 61657 root cwd unknown /proc/61657/cwd (readlink: Permission denied) 2024-03-21T10:28:38.495 INFO:tasks.workunit.client.0.smithi047.stdout:kworker/7 61657 root rtd unknown /proc/61657/root (readlink: Permission denied) 2024-03-21T10:28:38.495 INFO:tasks.workunit.client.0.smithi047.stdout:kworker/7 61657 root txt unknown /proc/61657/exe (readlink: Permission denied) 2024-03-21T10:28:38.495 INFO:tasks.workunit.client.0.smithi047.stdout:kworker/7 61657 root NOFD /proc/61657/fd (opendir: Permission denied) 2024-03-21T10:28:38.495 INFO:tasks.workunit.client.0.smithi047.stdout:kworker/4 63731 root cwd unknown /proc/63731/cwd (readlink: Permission denied) 2024-03-21T10:28:38.495 INFO:tasks.workunit.client.0.smithi047.stdout:kworker/4 63731 root rtd unknown /proc/63731/root (readlink: Permission denied) 2024-03-21T10:28:38.495 INFO:tasks.workunit.client.0.smithi047.stdout:kworker/4 63731 root txt unknown /proc/63731/exe (readlink: Permission denied) 2024-03-21T10:28:38.496 INFO:tasks.workunit.client.0.smithi047.stdout:kworker/4 63731 root NOFD /proc/63731/fd (opendir: Permission denied) 2024-03-21T10:28:38.496 INFO:tasks.workunit.client.0.smithi047.stdout:conmon 64137 root cwd unknown /proc/64137/cwd (readlink: Permission denied) 2024-03-21T10:28:38.496 INFO:tasks.workunit.client.0.smithi047.stdout:conmon 64137 root rtd unknown /proc/64137/root (readlink: Permission denied) 2024-03-21T10:28:38.496 INFO:tasks.workunit.client.0.smithi047.stdout:conmon 64137 root txt unknown /proc/64137/exe (readlink: Permission denied) 2024-03-21T10:28:38.496 INFO:tasks.workunit.client.0.smithi047.stdout:conmon 64137 root NOFD /proc/64137/fd (opendir: Permission denied) 2024-03-21T10:28:38.496 INFO:tasks.workunit.client.0.smithi047.stdout:podman-in 64139 root cwd unknown /proc/64139/cwd (readlink: Permission denied) 2024-03-21T10:28:38.496 INFO:tasks.workunit.client.0.smithi047.stdout:podman-in 64139 root rtd unknown /proc/64139/root (readlink: Permission denied) 2024-03-21T10:28:38.496 INFO:tasks.workunit.client.0.smithi047.stdout:podman-in 64139 root txt unknown /proc/64139/exe (readlink: Permission denied) 2024-03-21T10:28:38.496 INFO:tasks.workunit.client.0.smithi047.stdout:podman-in 64139 root NOFD /proc/64139/fd (opendir: Permission denied) 2024-03-21T10:28:38.496 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 64141 ceph cwd unknown /proc/64141/cwd (readlink: Permission denied) 2024-03-21T10:28:38.496 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 64141 ceph rtd unknown /proc/64141/root (readlink: Permission denied) 2024-03-21T10:28:38.496 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 64141 ceph txt unknown /proc/64141/exe (readlink: Permission denied) 2024-03-21T10:28:38.496 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 64141 ceph NOFD /proc/64141/fd (opendir: Permission denied) 2024-03-21T10:28:38.496 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 64141 64142 log ceph cwd unknown /proc/64141/task/64142/cwd (readlink: Permission denied) 2024-03-21T10:28:38.497 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 64141 64142 log ceph rtd unknown /proc/64141/task/64142/root (readlink: Permission denied) 2024-03-21T10:28:38.497 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 64141 64142 log ceph txt unknown /proc/64141/task/64142/exe (readlink: Permission denied) 2024-03-21T10:28:38.497 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 64141 64142 log ceph NOFD /proc/64141/task/64142/fd (opendir: Permission denied) 2024-03-21T10:28:38.497 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 64141 64144 msgr-work ceph cwd unknown /proc/64141/task/64144/cwd (readlink: Permission denied) 2024-03-21T10:28:38.497 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 64141 64144 msgr-work ceph rtd unknown /proc/64141/task/64144/root (readlink: Permission denied) 2024-03-21T10:28:38.497 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 64141 64144 msgr-work ceph txt unknown /proc/64141/task/64144/exe (readlink: Permission denied) 2024-03-21T10:28:38.497 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 64141 64144 msgr-work ceph NOFD /proc/64141/task/64144/fd (opendir: Permission denied) 2024-03-21T10:28:38.497 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 64141 64145 msgr-work ceph cwd unknown /proc/64141/task/64145/cwd (readlink: Permission denied) 2024-03-21T10:28:38.497 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 64141 64145 msgr-work ceph rtd unknown /proc/64141/task/64145/root (readlink: Permission denied) 2024-03-21T10:28:38.497 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 64141 64145 msgr-work ceph txt unknown /proc/64141/task/64145/exe (readlink: Permission denied) 2024-03-21T10:28:38.497 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 64141 64145 msgr-work ceph NOFD /proc/64141/task/64145/fd (opendir: Permission denied) 2024-03-21T10:28:38.497 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 64141 64146 msgr-work ceph cwd unknown /proc/64141/task/64146/cwd (readlink: Permission denied) 2024-03-21T10:28:38.497 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 64141 64146 msgr-work ceph rtd unknown /proc/64141/task/64146/root (readlink: Permission denied) 2024-03-21T10:28:38.498 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 64141 64146 msgr-work ceph txt unknown /proc/64141/task/64146/exe (readlink: Permission denied) 2024-03-21T10:28:38.498 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 64141 64146 msgr-work ceph NOFD /proc/64141/task/64146/fd (opendir: Permission denied) 2024-03-21T10:28:38.498 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 64141 64150 service ceph cwd unknown /proc/64141/task/64150/cwd (readlink: Permission denied) 2024-03-21T10:28:38.498 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 64141 64150 service ceph rtd unknown /proc/64141/task/64150/root (readlink: Permission denied) 2024-03-21T10:28:38.498 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 64141 64150 service ceph txt unknown /proc/64141/task/64150/exe (readlink: Permission denied) 2024-03-21T10:28:38.498 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 64141 64150 service ceph NOFD /proc/64141/task/64150/fd (opendir: Permission denied) 2024-03-21T10:28:38.498 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 64141 64151 admin_soc ceph cwd unknown /proc/64141/task/64151/cwd (readlink: Permission denied) 2024-03-21T10:28:38.498 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 64141 64151 admin_soc ceph rtd unknown /proc/64141/task/64151/root (readlink: Permission denied) 2024-03-21T10:28:38.498 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 64141 64151 admin_soc ceph txt unknown /proc/64141/task/64151/exe (readlink: Permission denied) 2024-03-21T10:28:38.498 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 64141 64151 admin_soc ceph NOFD /proc/64141/task/64151/fd (opendir: Permission denied) 2024-03-21T10:28:38.498 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 64141 64295 signal_ha ceph cwd unknown /proc/64141/task/64295/cwd (readlink: Permission denied) 2024-03-21T10:28:38.498 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 64141 64295 signal_ha ceph rtd unknown /proc/64141/task/64295/root (readlink: Permission denied) 2024-03-21T10:28:38.498 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 64141 64295 signal_ha ceph txt unknown /proc/64141/task/64295/exe (readlink: Permission denied) 2024-03-21T10:28:38.498 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 64141 64295 signal_ha ceph NOFD /proc/64141/task/64295/fd (opendir: Permission denied) 2024-03-21T10:28:38.499 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 64141 64296 io_contex ceph cwd unknown /proc/64141/task/64296/cwd (readlink: Permission denied) 2024-03-21T10:28:38.499 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 64141 64296 io_contex ceph rtd unknown /proc/64141/task/64296/root (readlink: Permission denied) 2024-03-21T10:28:38.499 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 64141 64296 io_contex ceph txt unknown /proc/64141/task/64296/exe (readlink: Permission denied) 2024-03-21T10:28:38.499 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 64141 64296 io_contex ceph NOFD /proc/64141/task/64296/fd (opendir: Permission denied) 2024-03-21T10:28:38.499 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 64141 64297 io_contex ceph cwd unknown /proc/64141/task/64297/cwd (readlink: Permission denied) 2024-03-21T10:28:38.499 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 64141 64297 io_contex ceph rtd unknown /proc/64141/task/64297/root (readlink: Permission denied) 2024-03-21T10:28:38.499 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 64141 64297 io_contex ceph txt unknown /proc/64141/task/64297/exe (readlink: Permission denied) 2024-03-21T10:28:38.499 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 64141 64297 io_contex ceph NOFD /proc/64141/task/64297/fd (opendir: Permission denied) 2024-03-21T10:28:38.499 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 64141 64301 OpHistory ceph cwd unknown /proc/64141/task/64301/cwd (readlink: Permission denied) 2024-03-21T10:28:38.499 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 64141 64301 OpHistory ceph rtd unknown /proc/64141/task/64301/root (readlink: Permission denied) 2024-03-21T10:28:38.499 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 64141 64301 OpHistory ceph txt unknown /proc/64141/task/64301/exe (readlink: Permission denied) 2024-03-21T10:28:38.499 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 64141 64301 OpHistory ceph NOFD /proc/64141/task/64301/fd (opendir: Permission denied) 2024-03-21T10:28:38.499 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 64141 64302 ceph_time ceph cwd unknown /proc/64141/task/64302/cwd (readlink: Permission denied) 2024-03-21T10:28:38.499 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 64141 64302 ceph_time ceph rtd unknown /proc/64141/task/64302/root (readlink: Permission denied) 2024-03-21T10:28:38.499 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 64141 64302 ceph_time ceph txt unknown /proc/64141/task/64302/exe (readlink: Permission denied) 2024-03-21T10:28:38.500 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 64141 64302 ceph_time ceph NOFD /proc/64141/task/64302/fd (opendir: Permission denied) 2024-03-21T10:28:38.500 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 64141 64523 ceph-osd ceph cwd unknown /proc/64141/task/64523/cwd (readlink: Permission denied) 2024-03-21T10:28:38.500 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 64141 64523 ceph-osd ceph rtd unknown /proc/64141/task/64523/root (readlink: Permission denied) 2024-03-21T10:28:38.500 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 64141 64523 ceph-osd ceph txt unknown /proc/64141/task/64523/exe (readlink: Permission denied) 2024-03-21T10:28:38.500 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 64141 64523 ceph-osd ceph NOFD /proc/64141/task/64523/fd (opendir: Permission denied) 2024-03-21T10:28:38.500 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 64141 64524 safe_time ceph cwd unknown /proc/64141/task/64524/cwd (readlink: Permission denied) 2024-03-21T10:28:38.500 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 64141 64524 safe_time ceph rtd unknown /proc/64141/task/64524/root (readlink: Permission denied) 2024-03-21T10:28:38.500 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 64141 64524 safe_time ceph txt unknown /proc/64141/task/64524/exe (readlink: Permission denied) 2024-03-21T10:28:38.500 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 64141 64524 safe_time ceph NOFD /proc/64141/task/64524/fd (opendir: Permission denied) 2024-03-21T10:28:38.500 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 64141 64525 safe_time ceph cwd unknown /proc/64141/task/64525/cwd (readlink: Permission denied) 2024-03-21T10:28:38.500 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 64141 64525 safe_time ceph rtd unknown /proc/64141/task/64525/root (readlink: Permission denied) 2024-03-21T10:28:38.500 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 64141 64525 safe_time ceph txt unknown /proc/64141/task/64525/exe (readlink: Permission denied) 2024-03-21T10:28:38.500 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 64141 64525 safe_time ceph NOFD /proc/64141/task/64525/fd (opendir: Permission denied) 2024-03-21T10:28:38.500 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 64141 64526 safe_time ceph cwd unknown /proc/64141/task/64526/cwd (readlink: Permission denied) 2024-03-21T10:28:38.501 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 64141 64526 safe_time ceph rtd unknown /proc/64141/task/64526/root (readlink: Permission denied) 2024-03-21T10:28:38.501 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 64141 64526 safe_time ceph txt unknown /proc/64141/task/64526/exe (readlink: Permission denied) 2024-03-21T10:28:38.501 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 64141 64526 safe_time ceph NOFD /proc/64141/task/64526/fd (opendir: Permission denied) 2024-03-21T10:28:38.501 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 64141 64527 safe_time ceph cwd unknown /proc/64141/task/64527/cwd (readlink: Permission denied) 2024-03-21T10:28:38.501 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 64141 64527 safe_time ceph rtd unknown /proc/64141/task/64527/root (readlink: Permission denied) 2024-03-21T10:28:38.501 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 64141 64527 safe_time ceph txt unknown /proc/64141/task/64527/exe (readlink: Permission denied) 2024-03-21T10:28:38.501 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 64141 64527 safe_time ceph NOFD /proc/64141/task/64527/fd (opendir: Permission denied) 2024-03-21T10:28:38.501 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 64141 64528 fn_anonym ceph cwd unknown /proc/64141/task/64528/cwd (readlink: Permission denied) 2024-03-21T10:28:38.501 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 64141 64528 fn_anonym ceph rtd unknown /proc/64141/task/64528/root (readlink: Permission denied) 2024-03-21T10:28:38.501 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 64141 64528 fn_anonym ceph txt unknown /proc/64141/task/64528/exe (readlink: Permission denied) 2024-03-21T10:28:38.501 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 64141 64528 fn_anonym ceph NOFD /proc/64141/task/64528/fd (opendir: Permission denied) 2024-03-21T10:28:38.501 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 64141 64533 rocksdb:l ceph cwd unknown /proc/64141/task/64533/cwd (readlink: Permission denied) 2024-03-21T10:28:38.501 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 64141 64533 rocksdb:l ceph rtd unknown /proc/64141/task/64533/root (readlink: Permission denied) 2024-03-21T10:28:38.501 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 64141 64533 rocksdb:l ceph txt unknown /proc/64141/task/64533/exe (readlink: Permission denied) 2024-03-21T10:28:38.501 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 64141 64533 rocksdb:l ceph NOFD /proc/64141/task/64533/fd (opendir: Permission denied) 2024-03-21T10:28:38.502 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 64141 64534 rocksdb:l ceph cwd unknown /proc/64141/task/64534/cwd (readlink: Permission denied) 2024-03-21T10:28:38.502 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 64141 64534 rocksdb:l ceph rtd unknown /proc/64141/task/64534/root (readlink: Permission denied) 2024-03-21T10:28:38.502 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 64141 64534 rocksdb:l ceph txt unknown /proc/64141/task/64534/exe (readlink: Permission denied) 2024-03-21T10:28:38.502 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 64141 64534 rocksdb:l ceph NOFD /proc/64141/task/64534/fd (opendir: Permission denied) 2024-03-21T10:28:38.502 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 64141 64535 rocksdb:l ceph cwd unknown /proc/64141/task/64535/cwd (readlink: Permission denied) 2024-03-21T10:28:38.502 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 64141 64535 rocksdb:l ceph rtd unknown /proc/64141/task/64535/root (readlink: Permission denied) 2024-03-21T10:28:38.502 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 64141 64535 rocksdb:l ceph txt unknown /proc/64141/task/64535/exe (readlink: Permission denied) 2024-03-21T10:28:38.502 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 64141 64535 rocksdb:l ceph NOFD /proc/64141/task/64535/fd (opendir: Permission denied) 2024-03-21T10:28:38.502 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 64141 64536 rocksdb:h ceph cwd unknown /proc/64141/task/64536/cwd (readlink: Permission denied) 2024-03-21T10:28:38.502 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 64141 64536 rocksdb:h ceph rtd unknown /proc/64141/task/64536/root (readlink: Permission denied) 2024-03-21T10:28:38.502 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 64141 64536 rocksdb:h ceph txt unknown /proc/64141/task/64536/exe (readlink: Permission denied) 2024-03-21T10:28:38.502 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 64141 64536 rocksdb:h ceph NOFD /proc/64141/task/64536/fd (opendir: Permission denied) 2024-03-21T10:28:38.502 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 64141 64955 bstore_ai ceph cwd unknown /proc/64141/task/64955/cwd (readlink: Permission denied) 2024-03-21T10:28:38.503 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 64141 64955 bstore_ai ceph rtd unknown /proc/64141/task/64955/root (readlink: Permission denied) 2024-03-21T10:28:38.503 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 64141 64955 bstore_ai ceph txt unknown /proc/64141/task/64955/exe (readlink: Permission denied) 2024-03-21T10:28:38.503 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 64141 64955 bstore_ai ceph NOFD /proc/64141/task/64955/fd (opendir: Permission denied) 2024-03-21T10:28:38.503 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 64141 65193 bstore_ai ceph cwd unknown /proc/64141/task/65193/cwd (readlink: Permission denied) 2024-03-21T10:28:38.503 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 64141 65193 bstore_ai ceph rtd unknown /proc/64141/task/65193/root (readlink: Permission denied) 2024-03-21T10:28:38.503 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 64141 65193 bstore_ai ceph txt unknown /proc/64141/task/65193/exe (readlink: Permission denied) 2024-03-21T10:28:38.503 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 64141 65193 bstore_ai ceph NOFD /proc/64141/task/65193/fd (opendir: Permission denied) 2024-03-21T10:28:38.503 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 64141 65381 ceph-osd ceph cwd unknown /proc/64141/task/65381/cwd (readlink: Permission denied) 2024-03-21T10:28:38.503 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 64141 65381 ceph-osd ceph rtd unknown /proc/64141/task/65381/root (readlink: Permission denied) 2024-03-21T10:28:38.503 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 64141 65381 ceph-osd ceph txt unknown /proc/64141/task/65381/exe (readlink: Permission denied) 2024-03-21T10:28:38.503 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 64141 65381 ceph-osd ceph NOFD /proc/64141/task/65381/fd (opendir: Permission denied) 2024-03-21T10:28:38.503 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 64141 65382 cfin ceph cwd unknown /proc/64141/task/65382/cwd (readlink: Permission denied) 2024-03-21T10:28:38.503 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 64141 65382 cfin ceph rtd unknown /proc/64141/task/65382/root (readlink: Permission denied) 2024-03-21T10:28:38.503 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 64141 65382 cfin ceph txt unknown /proc/64141/task/65382/exe (readlink: Permission denied) 2024-03-21T10:28:38.504 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 64141 65382 cfin ceph NOFD /proc/64141/task/65382/fd (opendir: Permission denied) 2024-03-21T10:28:38.504 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 64141 65383 bstore_kv ceph cwd unknown /proc/64141/task/65383/cwd (readlink: Permission denied) 2024-03-21T10:28:38.504 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 64141 65383 bstore_kv ceph rtd unknown /proc/64141/task/65383/root (readlink: Permission denied) 2024-03-21T10:28:38.504 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 64141 65383 bstore_kv ceph txt unknown /proc/64141/task/65383/exe (readlink: Permission denied) 2024-03-21T10:28:38.504 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 64141 65383 bstore_kv ceph NOFD /proc/64141/task/65383/fd (opendir: Permission denied) 2024-03-21T10:28:38.504 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 64141 65384 bstore_kv ceph cwd unknown /proc/64141/task/65384/cwd (readlink: Permission denied) 2024-03-21T10:28:38.504 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 64141 65384 bstore_kv ceph rtd unknown /proc/64141/task/65384/root (readlink: Permission denied) 2024-03-21T10:28:38.504 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 64141 65384 bstore_kv ceph txt unknown /proc/64141/task/65384/exe (readlink: Permission denied) 2024-03-21T10:28:38.504 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 64141 65384 bstore_kv ceph NOFD /proc/64141/task/65384/fd (opendir: Permission denied) 2024-03-21T10:28:38.504 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 64141 65385 bstore_me ceph cwd unknown /proc/64141/task/65385/cwd (readlink: Permission denied) 2024-03-21T10:28:38.504 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 64141 65385 bstore_me ceph rtd unknown /proc/64141/task/65385/root (readlink: Permission denied) 2024-03-21T10:28:38.504 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 64141 65385 bstore_me ceph txt unknown /proc/64141/task/65385/exe (readlink: Permission denied) 2024-03-21T10:28:38.504 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 64141 65385 bstore_me ceph NOFD /proc/64141/task/65385/fd (opendir: Permission denied) 2024-03-21T10:28:38.505 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 64141 65389 ms_dispat ceph cwd unknown /proc/64141/task/65389/cwd (readlink: Permission denied) 2024-03-21T10:28:38.505 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 64141 65389 ms_dispat ceph rtd unknown /proc/64141/task/65389/root (readlink: Permission denied) 2024-03-21T10:28:38.505 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 64141 65389 ms_dispat ceph txt unknown /proc/64141/task/65389/exe (readlink: Permission denied) 2024-03-21T10:28:38.505 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 64141 65389 ms_dispat ceph NOFD /proc/64141/task/65389/fd (opendir: Permission denied) 2024-03-21T10:28:38.505 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 64141 65390 ms_local ceph cwd unknown /proc/64141/task/65390/cwd (readlink: Permission denied) 2024-03-21T10:28:38.505 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 64141 65390 ms_local ceph rtd unknown /proc/64141/task/65390/root (readlink: Permission denied) 2024-03-21T10:28:38.505 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 64141 65390 ms_local ceph txt unknown /proc/64141/task/65390/exe (readlink: Permission denied) 2024-03-21T10:28:38.505 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 64141 65390 ms_local ceph NOFD /proc/64141/task/65390/fd (opendir: Permission denied) 2024-03-21T10:28:38.505 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 64141 65391 safe_time ceph cwd unknown /proc/64141/task/65391/cwd (readlink: Permission denied) 2024-03-21T10:28:38.505 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 64141 65391 safe_time ceph rtd unknown /proc/64141/task/65391/root (readlink: Permission denied) 2024-03-21T10:28:38.505 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 64141 65391 safe_time ceph txt unknown /proc/64141/task/65391/exe (readlink: Permission denied) 2024-03-21T10:28:38.505 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 64141 65391 safe_time ceph NOFD /proc/64141/task/65391/fd (opendir: Permission denied) 2024-03-21T10:28:38.505 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 64141 65392 safe_time ceph cwd unknown /proc/64141/task/65392/cwd (readlink: Permission denied) 2024-03-21T10:28:38.505 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 64141 65392 safe_time ceph rtd unknown /proc/64141/task/65392/root (readlink: Permission denied) 2024-03-21T10:28:38.506 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 64141 65392 safe_time ceph txt unknown /proc/64141/task/65392/exe (readlink: Permission denied) 2024-03-21T10:28:38.506 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 64141 65392 safe_time ceph NOFD /proc/64141/task/65392/fd (opendir: Permission denied) 2024-03-21T10:28:38.506 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 64141 65393 ms_dispat ceph cwd unknown /proc/64141/task/65393/cwd (readlink: Permission denied) 2024-03-21T10:28:38.506 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 64141 65393 ms_dispat ceph rtd unknown /proc/64141/task/65393/root (readlink: Permission denied) 2024-03-21T10:28:38.506 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 64141 65393 ms_dispat ceph txt unknown /proc/64141/task/65393/exe (readlink: Permission denied) 2024-03-21T10:28:38.506 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 64141 65393 ms_dispat ceph NOFD /proc/64141/task/65393/fd (opendir: Permission denied) 2024-03-21T10:28:38.506 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 64141 65394 ms_local ceph cwd unknown /proc/64141/task/65394/cwd (readlink: Permission denied) 2024-03-21T10:28:38.506 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 64141 65394 ms_local ceph rtd unknown /proc/64141/task/65394/root (readlink: Permission denied) 2024-03-21T10:28:38.506 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 64141 65394 ms_local ceph txt unknown /proc/64141/task/65394/exe (readlink: Permission denied) 2024-03-21T10:28:38.506 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 64141 65394 ms_local ceph NOFD /proc/64141/task/65394/fd (opendir: Permission denied) 2024-03-21T10:28:38.506 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 64141 65395 ms_dispat ceph cwd unknown /proc/64141/task/65395/cwd (readlink: Permission denied) 2024-03-21T10:28:38.506 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 64141 65395 ms_dispat ceph rtd unknown /proc/64141/task/65395/root (readlink: Permission denied) 2024-03-21T10:28:38.506 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 64141 65395 ms_dispat ceph txt unknown /proc/64141/task/65395/exe (readlink: Permission denied) 2024-03-21T10:28:38.507 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 64141 65395 ms_dispat ceph NOFD /proc/64141/task/65395/fd (opendir: Permission denied) 2024-03-21T10:28:38.507 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 64141 65396 ms_local ceph cwd unknown /proc/64141/task/65396/cwd (readlink: Permission denied) 2024-03-21T10:28:38.507 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 64141 65396 ms_local ceph rtd unknown /proc/64141/task/65396/root (readlink: Permission denied) 2024-03-21T10:28:38.507 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 64141 65396 ms_local ceph txt unknown /proc/64141/task/65396/exe (readlink: Permission denied) 2024-03-21T10:28:38.507 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 64141 65396 ms_local ceph NOFD /proc/64141/task/65396/fd (opendir: Permission denied) 2024-03-21T10:28:38.507 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 64141 65397 ms_dispat ceph cwd unknown /proc/64141/task/65397/cwd (readlink: Permission denied) 2024-03-21T10:28:38.507 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 64141 65397 ms_dispat ceph rtd unknown /proc/64141/task/65397/root (readlink: Permission denied) 2024-03-21T10:28:38.507 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 64141 65397 ms_dispat ceph txt unknown /proc/64141/task/65397/exe (readlink: Permission denied) 2024-03-21T10:28:38.507 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 64141 65397 ms_dispat ceph NOFD /proc/64141/task/65397/fd (opendir: Permission denied) 2024-03-21T10:28:38.507 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 64141 65398 ms_local ceph cwd unknown /proc/64141/task/65398/cwd (readlink: Permission denied) 2024-03-21T10:28:38.507 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 64141 65398 ms_local ceph rtd unknown /proc/64141/task/65398/root (readlink: Permission denied) 2024-03-21T10:28:38.507 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 64141 65398 ms_local ceph txt unknown /proc/64141/task/65398/exe (readlink: Permission denied) 2024-03-21T10:28:38.507 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 64141 65398 ms_local ceph NOFD /proc/64141/task/65398/fd (opendir: Permission denied) 2024-03-21T10:28:38.507 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 64141 65399 ms_dispat ceph cwd unknown /proc/64141/task/65399/cwd (readlink: Permission denied) 2024-03-21T10:28:38.508 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 64141 65399 ms_dispat ceph rtd unknown /proc/64141/task/65399/root (readlink: Permission denied) 2024-03-21T10:28:38.508 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 64141 65399 ms_dispat ceph txt unknown /proc/64141/task/65399/exe (readlink: Permission denied) 2024-03-21T10:28:38.508 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 64141 65399 ms_dispat ceph NOFD /proc/64141/task/65399/fd (opendir: Permission denied) 2024-03-21T10:28:38.508 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 64141 65400 ms_local ceph cwd unknown /proc/64141/task/65400/cwd (readlink: Permission denied) 2024-03-21T10:28:38.508 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 64141 65400 ms_local ceph rtd unknown /proc/64141/task/65400/root (readlink: Permission denied) 2024-03-21T10:28:38.508 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 64141 65400 ms_local ceph txt unknown /proc/64141/task/65400/exe (readlink: Permission denied) 2024-03-21T10:28:38.508 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 64141 65400 ms_local ceph NOFD /proc/64141/task/65400/fd (opendir: Permission denied) 2024-03-21T10:28:38.508 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 64141 65401 ms_dispat ceph cwd unknown /proc/64141/task/65401/cwd (readlink: Permission denied) 2024-03-21T10:28:38.508 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 64141 65401 ms_dispat ceph rtd unknown /proc/64141/task/65401/root (readlink: Permission denied) 2024-03-21T10:28:38.508 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 64141 65401 ms_dispat ceph txt unknown /proc/64141/task/65401/exe (readlink: Permission denied) 2024-03-21T10:28:38.508 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 64141 65401 ms_dispat ceph NOFD /proc/64141/task/65401/fd (opendir: Permission denied) 2024-03-21T10:28:38.508 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 64141 65402 ms_local ceph cwd unknown /proc/64141/task/65402/cwd (readlink: Permission denied) 2024-03-21T10:28:38.508 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 64141 65402 ms_local ceph rtd unknown /proc/64141/task/65402/root (readlink: Permission denied) 2024-03-21T10:28:38.508 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 64141 65402 ms_local ceph txt unknown /proc/64141/task/65402/exe (readlink: Permission denied) 2024-03-21T10:28:38.509 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 64141 65402 ms_local ceph NOFD /proc/64141/task/65402/fd (opendir: Permission denied) 2024-03-21T10:28:38.509 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 64141 65403 ms_dispat ceph cwd unknown /proc/64141/task/65403/cwd (readlink: Permission denied) 2024-03-21T10:28:38.509 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 64141 65403 ms_dispat ceph rtd unknown /proc/64141/task/65403/root (readlink: Permission denied) 2024-03-21T10:28:38.509 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 64141 65403 ms_dispat ceph txt unknown /proc/64141/task/65403/exe (readlink: Permission denied) 2024-03-21T10:28:38.509 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 64141 65403 ms_dispat ceph NOFD /proc/64141/task/65403/fd (opendir: Permission denied) 2024-03-21T10:28:38.509 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 64141 65404 ms_local ceph cwd unknown /proc/64141/task/65404/cwd (readlink: Permission denied) 2024-03-21T10:28:38.509 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 64141 65404 ms_local ceph rtd unknown /proc/64141/task/65404/root (readlink: Permission denied) 2024-03-21T10:28:38.509 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 64141 65404 ms_local ceph txt unknown /proc/64141/task/65404/exe (readlink: Permission denied) 2024-03-21T10:28:38.509 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 64141 65404 ms_local ceph NOFD /proc/64141/task/65404/fd (opendir: Permission denied) 2024-03-21T10:28:38.509 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 64141 65405 fn_anonym ceph cwd unknown /proc/64141/task/65405/cwd (readlink: Permission denied) 2024-03-21T10:28:38.509 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 64141 65405 fn_anonym ceph rtd unknown /proc/64141/task/65405/root (readlink: Permission denied) 2024-03-21T10:28:38.509 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 64141 65405 fn_anonym ceph txt unknown /proc/64141/task/65405/exe (readlink: Permission denied) 2024-03-21T10:28:38.509 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 64141 65405 fn_anonym ceph NOFD /proc/64141/task/65405/fd (opendir: Permission denied) 2024-03-21T10:28:38.509 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 64141 65406 finisher ceph cwd unknown /proc/64141/task/65406/cwd (readlink: Permission denied) 2024-03-21T10:28:38.510 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 64141 65406 finisher ceph rtd unknown /proc/64141/task/65406/root (readlink: Permission denied) 2024-03-21T10:28:38.510 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 64141 65406 finisher ceph txt unknown /proc/64141/task/65406/exe (readlink: Permission denied) 2024-03-21T10:28:38.510 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 64141 65406 finisher ceph NOFD /proc/64141/task/65406/fd (opendir: Permission denied) 2024-03-21T10:28:38.510 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 64141 65407 safe_time ceph cwd unknown /proc/64141/task/65407/cwd (readlink: Permission denied) 2024-03-21T10:28:38.510 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 64141 65407 safe_time ceph rtd unknown /proc/64141/task/65407/root (readlink: Permission denied) 2024-03-21T10:28:38.510 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 64141 65407 safe_time ceph txt unknown /proc/64141/task/65407/exe (readlink: Permission denied) 2024-03-21T10:28:38.510 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 64141 65407 safe_time ceph NOFD /proc/64141/task/65407/fd (opendir: Permission denied) 2024-03-21T10:28:38.510 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 64141 65408 safe_time ceph cwd unknown /proc/64141/task/65408/cwd (readlink: Permission denied) 2024-03-21T10:28:38.510 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 64141 65408 safe_time ceph rtd unknown /proc/64141/task/65408/root (readlink: Permission denied) 2024-03-21T10:28:38.510 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 64141 65408 safe_time ceph txt unknown /proc/64141/task/65408/exe (readlink: Permission denied) 2024-03-21T10:28:38.510 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 64141 65408 safe_time ceph NOFD /proc/64141/task/65408/fd (opendir: Permission denied) 2024-03-21T10:28:38.510 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 64141 65409 ceph-osd ceph cwd unknown /proc/64141/task/65409/cwd (readlink: Permission denied) 2024-03-21T10:28:38.511 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 64141 65409 ceph-osd ceph rtd unknown /proc/64141/task/65409/root (readlink: Permission denied) 2024-03-21T10:28:38.511 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 64141 65409 ceph-osd ceph txt unknown /proc/64141/task/65409/exe (readlink: Permission denied) 2024-03-21T10:28:38.511 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 64141 65409 ceph-osd ceph NOFD /proc/64141/task/65409/fd (opendir: Permission denied) 2024-03-21T10:28:38.511 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 64141 65410 osd_srv_a ceph cwd unknown /proc/64141/task/65410/cwd (readlink: Permission denied) 2024-03-21T10:28:38.511 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 64141 65410 osd_srv_a ceph rtd unknown /proc/64141/task/65410/root (readlink: Permission denied) 2024-03-21T10:28:38.511 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 64141 65410 osd_srv_a ceph txt unknown /proc/64141/task/65410/exe (readlink: Permission denied) 2024-03-21T10:28:38.511 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 64141 65410 osd_srv_a ceph NOFD /proc/64141/task/65410/fd (opendir: Permission denied) 2024-03-21T10:28:38.511 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 64141 65411 tp_osd_tp ceph cwd unknown /proc/64141/task/65411/cwd (readlink: Permission denied) 2024-03-21T10:28:38.511 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 64141 65411 tp_osd_tp ceph rtd unknown /proc/64141/task/65411/root (readlink: Permission denied) 2024-03-21T10:28:38.511 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 64141 65411 tp_osd_tp ceph txt unknown /proc/64141/task/65411/exe (readlink: Permission denied) 2024-03-21T10:28:38.511 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 64141 65411 tp_osd_tp ceph NOFD /proc/64141/task/65411/fd (opendir: Permission denied) 2024-03-21T10:28:38.511 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 64141 65412 tp_osd_tp ceph cwd unknown /proc/64141/task/65412/cwd (readlink: Permission denied) 2024-03-21T10:28:38.511 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 64141 65412 tp_osd_tp ceph rtd unknown /proc/64141/task/65412/root (readlink: Permission denied) 2024-03-21T10:28:38.511 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 64141 65412 tp_osd_tp ceph txt unknown /proc/64141/task/65412/exe (readlink: Permission denied) 2024-03-21T10:28:38.512 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 64141 65412 tp_osd_tp ceph NOFD /proc/64141/task/65412/fd (opendir: Permission denied) 2024-03-21T10:28:38.512 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 64141 65413 tp_osd_tp ceph cwd unknown /proc/64141/task/65413/cwd (readlink: Permission denied) 2024-03-21T10:28:38.512 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 64141 65413 tp_osd_tp ceph rtd unknown /proc/64141/task/65413/root (readlink: Permission denied) 2024-03-21T10:28:38.512 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 64141 65413 tp_osd_tp ceph txt unknown /proc/64141/task/65413/exe (readlink: Permission denied) 2024-03-21T10:28:38.512 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 64141 65413 tp_osd_tp ceph NOFD /proc/64141/task/65413/fd (opendir: Permission denied) 2024-03-21T10:28:38.512 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 64141 65414 tp_osd_tp ceph cwd unknown /proc/64141/task/65414/cwd (readlink: Permission denied) 2024-03-21T10:28:38.512 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 64141 65414 tp_osd_tp ceph rtd unknown /proc/64141/task/65414/root (readlink: Permission denied) 2024-03-21T10:28:38.512 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 64141 65414 tp_osd_tp ceph txt unknown /proc/64141/task/65414/exe (readlink: Permission denied) 2024-03-21T10:28:38.512 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 64141 65414 tp_osd_tp ceph NOFD /proc/64141/task/65414/fd (opendir: Permission denied) 2024-03-21T10:28:38.512 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 64141 65415 tp_osd_tp ceph cwd unknown /proc/64141/task/65415/cwd (readlink: Permission denied) 2024-03-21T10:28:38.512 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 64141 65415 tp_osd_tp ceph rtd unknown /proc/64141/task/65415/root (readlink: Permission denied) 2024-03-21T10:28:38.512 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 64141 65415 tp_osd_tp ceph txt unknown /proc/64141/task/65415/exe (readlink: Permission denied) 2024-03-21T10:28:38.512 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 64141 65415 tp_osd_tp ceph NOFD /proc/64141/task/65415/fd (opendir: Permission denied) 2024-03-21T10:28:38.512 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 64141 65416 tp_osd_tp ceph cwd unknown /proc/64141/task/65416/cwd (readlink: Permission denied) 2024-03-21T10:28:38.513 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 64141 65416 tp_osd_tp ceph rtd unknown /proc/64141/task/65416/root (readlink: Permission denied) 2024-03-21T10:28:38.513 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 64141 65416 tp_osd_tp ceph txt unknown /proc/64141/task/65416/exe (readlink: Permission denied) 2024-03-21T10:28:38.513 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 64141 65416 tp_osd_tp ceph NOFD /proc/64141/task/65416/fd (opendir: Permission denied) 2024-03-21T10:28:38.513 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 64141 65417 tp_osd_tp ceph cwd unknown /proc/64141/task/65417/cwd (readlink: Permission denied) 2024-03-21T10:28:38.513 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 64141 65417 tp_osd_tp ceph rtd unknown /proc/64141/task/65417/root (readlink: Permission denied) 2024-03-21T10:28:38.513 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 64141 65417 tp_osd_tp ceph txt unknown /proc/64141/task/65417/exe (readlink: Permission denied) 2024-03-21T10:28:38.513 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 64141 65417 tp_osd_tp ceph NOFD /proc/64141/task/65417/fd (opendir: Permission denied) 2024-03-21T10:28:38.513 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 64141 65418 tp_osd_tp ceph cwd unknown /proc/64141/task/65418/cwd (readlink: Permission denied) 2024-03-21T10:28:38.513 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 64141 65418 tp_osd_tp ceph rtd unknown /proc/64141/task/65418/root (readlink: Permission denied) 2024-03-21T10:28:38.513 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 64141 65418 tp_osd_tp ceph txt unknown /proc/64141/task/65418/exe (readlink: Permission denied) 2024-03-21T10:28:38.513 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 64141 65418 tp_osd_tp ceph NOFD /proc/64141/task/65418/fd (opendir: Permission denied) 2024-03-21T10:28:38.513 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 64141 65419 tp_osd_tp ceph cwd unknown /proc/64141/task/65419/cwd (readlink: Permission denied) 2024-03-21T10:28:38.513 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 64141 65419 tp_osd_tp ceph rtd unknown /proc/64141/task/65419/root (readlink: Permission denied) 2024-03-21T10:28:38.513 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 64141 65419 tp_osd_tp ceph txt unknown /proc/64141/task/65419/exe (readlink: Permission denied) 2024-03-21T10:28:38.514 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 64141 65419 tp_osd_tp ceph NOFD /proc/64141/task/65419/fd (opendir: Permission denied) 2024-03-21T10:28:38.514 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 64141 65420 tp_osd_tp ceph cwd unknown /proc/64141/task/65420/cwd (readlink: Permission denied) 2024-03-21T10:28:38.514 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 64141 65420 tp_osd_tp ceph rtd unknown /proc/64141/task/65420/root (readlink: Permission denied) 2024-03-21T10:28:38.514 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 64141 65420 tp_osd_tp ceph txt unknown /proc/64141/task/65420/exe (readlink: Permission denied) 2024-03-21T10:28:38.514 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 64141 65420 tp_osd_tp ceph NOFD /proc/64141/task/65420/fd (opendir: Permission denied) 2024-03-21T10:28:38.514 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 64141 65421 tp_osd_tp ceph cwd unknown /proc/64141/task/65421/cwd (readlink: Permission denied) 2024-03-21T10:28:38.514 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 64141 65421 tp_osd_tp ceph rtd unknown /proc/64141/task/65421/root (readlink: Permission denied) 2024-03-21T10:28:38.514 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 64141 65421 tp_osd_tp ceph txt unknown /proc/64141/task/65421/exe (readlink: Permission denied) 2024-03-21T10:28:38.514 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 64141 65421 tp_osd_tp ceph NOFD /proc/64141/task/65421/fd (opendir: Permission denied) 2024-03-21T10:28:38.514 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 64141 65422 tp_osd_tp ceph cwd unknown /proc/64141/task/65422/cwd (readlink: Permission denied) 2024-03-21T10:28:38.514 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 64141 65422 tp_osd_tp ceph rtd unknown /proc/64141/task/65422/root (readlink: Permission denied) 2024-03-21T10:28:38.514 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 64141 65422 tp_osd_tp ceph txt unknown /proc/64141/task/65422/exe (readlink: Permission denied) 2024-03-21T10:28:38.514 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 64141 65422 tp_osd_tp ceph NOFD /proc/64141/task/65422/fd (opendir: Permission denied) 2024-03-21T10:28:38.515 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 64141 65423 tp_osd_tp ceph cwd unknown /proc/64141/task/65423/cwd (readlink: Permission denied) 2024-03-21T10:28:38.515 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 64141 65423 tp_osd_tp ceph rtd unknown /proc/64141/task/65423/root (readlink: Permission denied) 2024-03-21T10:28:38.515 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 64141 65423 tp_osd_tp ceph txt unknown /proc/64141/task/65423/exe (readlink: Permission denied) 2024-03-21T10:28:38.515 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 64141 65423 tp_osd_tp ceph NOFD /proc/64141/task/65423/fd (opendir: Permission denied) 2024-03-21T10:28:38.515 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 64141 65424 tp_osd_tp ceph cwd unknown /proc/64141/task/65424/cwd (readlink: Permission denied) 2024-03-21T10:28:38.515 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 64141 65424 tp_osd_tp ceph rtd unknown /proc/64141/task/65424/root (readlink: Permission denied) 2024-03-21T10:28:38.515 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 64141 65424 tp_osd_tp ceph txt unknown /proc/64141/task/65424/exe (readlink: Permission denied) 2024-03-21T10:28:38.515 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 64141 65424 tp_osd_tp ceph NOFD /proc/64141/task/65424/fd (opendir: Permission denied) 2024-03-21T10:28:38.515 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 64141 65425 tp_osd_tp ceph cwd unknown /proc/64141/task/65425/cwd (readlink: Permission denied) 2024-03-21T10:28:38.515 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 64141 65425 tp_osd_tp ceph rtd unknown /proc/64141/task/65425/root (readlink: Permission denied) 2024-03-21T10:28:38.515 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 64141 65425 tp_osd_tp ceph txt unknown /proc/64141/task/65425/exe (readlink: Permission denied) 2024-03-21T10:28:38.515 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 64141 65425 tp_osd_tp ceph NOFD /proc/64141/task/65425/fd (opendir: Permission denied) 2024-03-21T10:28:38.515 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 64141 65426 tp_osd_tp ceph cwd unknown /proc/64141/task/65426/cwd (readlink: Permission denied) 2024-03-21T10:28:38.515 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 64141 65426 tp_osd_tp ceph rtd unknown /proc/64141/task/65426/root (readlink: Permission denied) 2024-03-21T10:28:38.516 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 64141 65426 tp_osd_tp ceph txt unknown /proc/64141/task/65426/exe (readlink: Permission denied) 2024-03-21T10:28:38.516 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 64141 65426 tp_osd_tp ceph NOFD /proc/64141/task/65426/fd (opendir: Permission denied) 2024-03-21T10:28:38.516 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 64141 65427 osd_srv_h ceph cwd unknown /proc/64141/task/65427/cwd (readlink: Permission denied) 2024-03-21T10:28:38.516 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 64141 65427 osd_srv_h ceph rtd unknown /proc/64141/task/65427/root (readlink: Permission denied) 2024-03-21T10:28:38.516 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 64141 65427 osd_srv_h ceph txt unknown /proc/64141/task/65427/exe (readlink: Permission denied) 2024-03-21T10:28:38.516 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-osd 64141 65427 osd_srv_h ceph NOFD /proc/64141/task/65427/fd (opendir: Permission denied) 2024-03-21T10:28:38.516 INFO:tasks.workunit.client.0.smithi047.stdout:sudo 65640 root cwd unknown /proc/65640/cwd (readlink: Permission denied) 2024-03-21T10:28:38.516 INFO:tasks.workunit.client.0.smithi047.stdout:sudo 65640 root rtd unknown /proc/65640/root (readlink: Permission denied) 2024-03-21T10:28:38.516 INFO:tasks.workunit.client.0.smithi047.stdout:sudo 65640 root txt unknown /proc/65640/exe (readlink: Permission denied) 2024-03-21T10:28:38.516 INFO:tasks.workunit.client.0.smithi047.stdout:sudo 65640 root NOFD /proc/65640/fd (opendir: Permission denied) 2024-03-21T10:28:38.516 INFO:tasks.workunit.client.0.smithi047.stdout:journalct 65657 root cwd unknown /proc/65657/cwd (readlink: Permission denied) 2024-03-21T10:28:38.516 INFO:tasks.workunit.client.0.smithi047.stdout:journalct 65657 root rtd unknown /proc/65657/root (readlink: Permission denied) 2024-03-21T10:28:38.516 INFO:tasks.workunit.client.0.smithi047.stdout:journalct 65657 root txt unknown /proc/65657/exe (readlink: Permission denied) 2024-03-21T10:28:38.517 INFO:tasks.workunit.client.0.smithi047.stdout:journalct 65657 root NOFD /proc/65657/fd (opendir: Permission denied) 2024-03-21T10:28:38.517 INFO:tasks.workunit.client.0.smithi047.stdout:kworker/1 68409 root cwd unknown /proc/68409/cwd (readlink: Permission denied) 2024-03-21T10:28:38.517 INFO:tasks.workunit.client.0.smithi047.stdout:kworker/1 68409 root rtd unknown /proc/68409/root (readlink: Permission denied) 2024-03-21T10:28:38.517 INFO:tasks.workunit.client.0.smithi047.stdout:kworker/1 68409 root txt unknown /proc/68409/exe (readlink: Permission denied) 2024-03-21T10:28:38.517 INFO:tasks.workunit.client.0.smithi047.stdout:kworker/1 68409 root NOFD /proc/68409/fd (opendir: Permission denied) 2024-03-21T10:28:38.517 INFO:tasks.workunit.client.0.smithi047.stdout:sudo 69127 root cwd unknown /proc/69127/cwd (readlink: Permission denied) 2024-03-21T10:28:38.517 INFO:tasks.workunit.client.0.smithi047.stdout:sudo 69127 root rtd unknown /proc/69127/root (readlink: Permission denied) 2024-03-21T10:28:38.517 INFO:tasks.workunit.client.0.smithi047.stdout:sudo 69127 root txt unknown /proc/69127/exe (readlink: Permission denied) 2024-03-21T10:28:38.517 INFO:tasks.workunit.client.0.smithi047.stdout:sudo 69127 root NOFD /proc/69127/fd (opendir: Permission denied) 2024-03-21T10:28:38.517 INFO:tasks.workunit.client.0.smithi047.stdout:sudo 69143 root cwd unknown /proc/69143/cwd (readlink: Permission denied) 2024-03-21T10:28:38.517 INFO:tasks.workunit.client.0.smithi047.stdout:sudo 69143 root rtd unknown /proc/69143/root (readlink: Permission denied) 2024-03-21T10:28:38.517 INFO:tasks.workunit.client.0.smithi047.stdout:sudo 69143 root txt unknown /proc/69143/exe (readlink: Permission denied) 2024-03-21T10:28:38.517 INFO:tasks.workunit.client.0.smithi047.stdout:sudo 69143 root NOFD /proc/69143/fd (opendir: Permission denied) 2024-03-21T10:28:38.517 INFO:tasks.workunit.client.0.smithi047.stdout:journalct 69159 root cwd unknown /proc/69159/cwd (readlink: Permission denied) 2024-03-21T10:28:38.518 INFO:tasks.workunit.client.0.smithi047.stdout:journalct 69159 root rtd unknown /proc/69159/root (readlink: Permission denied) 2024-03-21T10:28:38.518 INFO:tasks.workunit.client.0.smithi047.stdout:journalct 69159 root txt unknown /proc/69159/exe (readlink: Permission denied) 2024-03-21T10:28:38.518 INFO:tasks.workunit.client.0.smithi047.stdout:journalct 69159 root NOFD /proc/69159/fd (opendir: Permission denied) 2024-03-21T10:28:38.518 INFO:tasks.workunit.client.0.smithi047.stdout:sudo 69161 root cwd unknown /proc/69161/cwd (readlink: Permission denied) 2024-03-21T10:28:38.518 INFO:tasks.workunit.client.0.smithi047.stdout:sudo 69161 root rtd unknown /proc/69161/root (readlink: Permission denied) 2024-03-21T10:28:38.518 INFO:tasks.workunit.client.0.smithi047.stdout:sudo 69161 root txt unknown /proc/69161/exe (readlink: Permission denied) 2024-03-21T10:28:38.518 INFO:tasks.workunit.client.0.smithi047.stdout:sudo 69161 root NOFD /proc/69161/fd (opendir: Permission denied) 2024-03-21T10:28:38.518 INFO:tasks.workunit.client.0.smithi047.stdout:journalct 69175 root cwd unknown /proc/69175/cwd (readlink: Permission denied) 2024-03-21T10:28:38.518 INFO:tasks.workunit.client.0.smithi047.stdout:journalct 69175 root rtd unknown /proc/69175/root (readlink: Permission denied) 2024-03-21T10:28:38.518 INFO:tasks.workunit.client.0.smithi047.stdout:journalct 69175 root txt unknown /proc/69175/exe (readlink: Permission denied) 2024-03-21T10:28:38.518 INFO:tasks.workunit.client.0.smithi047.stdout:journalct 69175 root NOFD /proc/69175/fd (opendir: Permission denied) 2024-03-21T10:28:38.518 INFO:tasks.workunit.client.0.smithi047.stdout:sudo 69179 root cwd unknown /proc/69179/cwd (readlink: Permission denied) 2024-03-21T10:28:38.518 INFO:tasks.workunit.client.0.smithi047.stdout:sudo 69179 root rtd unknown /proc/69179/root (readlink: Permission denied) 2024-03-21T10:28:38.519 INFO:tasks.workunit.client.0.smithi047.stdout:sudo 69179 root txt unknown /proc/69179/exe (readlink: Permission denied) 2024-03-21T10:28:38.519 INFO:tasks.workunit.client.0.smithi047.stdout:sudo 69179 root NOFD /proc/69179/fd (opendir: Permission denied) 2024-03-21T10:28:38.519 INFO:tasks.workunit.client.0.smithi047.stdout:journalct 69193 root cwd unknown /proc/69193/cwd (readlink: Permission denied) 2024-03-21T10:28:38.519 INFO:tasks.workunit.client.0.smithi047.stdout:journalct 69193 root rtd unknown /proc/69193/root (readlink: Permission denied) 2024-03-21T10:28:38.519 INFO:tasks.workunit.client.0.smithi047.stdout:journalct 69193 root txt unknown /proc/69193/exe (readlink: Permission denied) 2024-03-21T10:28:38.519 INFO:tasks.workunit.client.0.smithi047.stdout:journalct 69193 root NOFD /proc/69193/fd (opendir: Permission denied) 2024-03-21T10:28:38.519 INFO:tasks.workunit.client.0.smithi047.stdout:journalct 69198 root cwd unknown /proc/69198/cwd (readlink: Permission denied) 2024-03-21T10:28:38.519 INFO:tasks.workunit.client.0.smithi047.stdout:journalct 69198 root rtd unknown /proc/69198/root (readlink: Permission denied) 2024-03-21T10:28:38.519 INFO:tasks.workunit.client.0.smithi047.stdout:journalct 69198 root txt unknown /proc/69198/exe (readlink: Permission denied) 2024-03-21T10:28:38.519 INFO:tasks.workunit.client.0.smithi047.stdout:journalct 69198 root NOFD /proc/69198/fd (opendir: Permission denied) 2024-03-21T10:28:38.519 INFO:tasks.workunit.client.0.smithi047.stdout:conmon 69717 root cwd unknown /proc/69717/cwd (readlink: Permission denied) 2024-03-21T10:28:38.519 INFO:tasks.workunit.client.0.smithi047.stdout:conmon 69717 root rtd unknown /proc/69717/root (readlink: Permission denied) 2024-03-21T10:28:38.519 INFO:tasks.workunit.client.0.smithi047.stdout:conmon 69717 root txt unknown /proc/69717/exe (readlink: Permission denied) 2024-03-21T10:28:38.519 INFO:tasks.workunit.client.0.smithi047.stdout:conmon 69717 root NOFD /proc/69717/fd (opendir: Permission denied) 2024-03-21T10:28:38.520 INFO:tasks.workunit.client.0.smithi047.stdout:podman-in 69719 root cwd unknown /proc/69719/cwd (readlink: Permission denied) 2024-03-21T10:28:38.520 INFO:tasks.workunit.client.0.smithi047.stdout:podman-in 69719 root rtd unknown /proc/69719/root (readlink: Permission denied) 2024-03-21T10:28:38.520 INFO:tasks.workunit.client.0.smithi047.stdout:podman-in 69719 root txt unknown /proc/69719/exe (readlink: Permission denied) 2024-03-21T10:28:38.520 INFO:tasks.workunit.client.0.smithi047.stdout:podman-in 69719 root NOFD /proc/69719/fd (opendir: Permission denied) 2024-03-21T10:28:38.520 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mds 69721 ceph cwd unknown /proc/69721/cwd (readlink: Permission denied) 2024-03-21T10:28:38.520 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mds 69721 ceph rtd unknown /proc/69721/root (readlink: Permission denied) 2024-03-21T10:28:38.520 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mds 69721 ceph txt unknown /proc/69721/exe (readlink: Permission denied) 2024-03-21T10:28:38.520 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mds 69721 ceph NOFD /proc/69721/fd (opendir: Permission denied) 2024-03-21T10:28:38.520 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mds 69721 69722 log ceph cwd unknown /proc/69721/task/69722/cwd (readlink: Permission denied) 2024-03-21T10:28:38.520 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mds 69721 69722 log ceph rtd unknown /proc/69721/task/69722/root (readlink: Permission denied) 2024-03-21T10:28:38.520 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mds 69721 69722 log ceph txt unknown /proc/69721/task/69722/exe (readlink: Permission denied) 2024-03-21T10:28:38.520 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mds 69721 69722 log ceph NOFD /proc/69721/task/69722/fd (opendir: Permission denied) 2024-03-21T10:28:38.520 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mds 69721 69724 msgr-work ceph cwd unknown /proc/69721/task/69724/cwd (readlink: Permission denied) 2024-03-21T10:28:38.520 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mds 69721 69724 msgr-work ceph rtd unknown /proc/69721/task/69724/root (readlink: Permission denied) 2024-03-21T10:28:38.521 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mds 69721 69724 msgr-work ceph txt unknown /proc/69721/task/69724/exe (readlink: Permission denied) 2024-03-21T10:28:38.521 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mds 69721 69724 msgr-work ceph NOFD /proc/69721/task/69724/fd (opendir: Permission denied) 2024-03-21T10:28:38.521 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mds 69721 69725 msgr-work ceph cwd unknown /proc/69721/task/69725/cwd (readlink: Permission denied) 2024-03-21T10:28:38.521 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mds 69721 69725 msgr-work ceph rtd unknown /proc/69721/task/69725/root (readlink: Permission denied) 2024-03-21T10:28:38.521 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mds 69721 69725 msgr-work ceph txt unknown /proc/69721/task/69725/exe (readlink: Permission denied) 2024-03-21T10:28:38.521 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mds 69721 69725 msgr-work ceph NOFD /proc/69721/task/69725/fd (opendir: Permission denied) 2024-03-21T10:28:38.521 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mds 69721 69726 msgr-work ceph cwd unknown /proc/69721/task/69726/cwd (readlink: Permission denied) 2024-03-21T10:28:38.521 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mds 69721 69726 msgr-work ceph rtd unknown /proc/69721/task/69726/root (readlink: Permission denied) 2024-03-21T10:28:38.521 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mds 69721 69726 msgr-work ceph txt unknown /proc/69721/task/69726/exe (readlink: Permission denied) 2024-03-21T10:28:38.521 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mds 69721 69726 msgr-work ceph NOFD /proc/69721/task/69726/fd (opendir: Permission denied) 2024-03-21T10:28:38.521 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mds 69721 69730 service ceph cwd unknown /proc/69721/task/69730/cwd (readlink: Permission denied) 2024-03-21T10:28:38.521 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mds 69721 69730 service ceph rtd unknown /proc/69721/task/69730/root (readlink: Permission denied) 2024-03-21T10:28:38.521 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mds 69721 69730 service ceph txt unknown /proc/69721/task/69730/exe (readlink: Permission denied) 2024-03-21T10:28:38.521 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mds 69721 69730 service ceph NOFD /proc/69721/task/69730/fd (opendir: Permission denied) 2024-03-21T10:28:38.521 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mds 69721 69731 admin_soc ceph cwd unknown /proc/69721/task/69731/cwd (readlink: Permission denied) 2024-03-21T10:28:38.522 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mds 69721 69731 admin_soc ceph rtd unknown /proc/69721/task/69731/root (readlink: Permission denied) 2024-03-21T10:28:38.522 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mds 69721 69731 admin_soc ceph txt unknown /proc/69721/task/69731/exe (readlink: Permission denied) 2024-03-21T10:28:38.522 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mds 69721 69731 admin_soc ceph NOFD /proc/69721/task/69731/fd (opendir: Permission denied) 2024-03-21T10:28:38.522 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mds 69721 69732 signal_ha ceph cwd unknown /proc/69721/task/69732/cwd (readlink: Permission denied) 2024-03-21T10:28:38.522 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mds 69721 69732 signal_ha ceph rtd unknown /proc/69721/task/69732/root (readlink: Permission denied) 2024-03-21T10:28:38.522 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mds 69721 69732 signal_ha ceph txt unknown /proc/69721/task/69732/exe (readlink: Permission denied) 2024-03-21T10:28:38.522 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mds 69721 69732 signal_ha ceph NOFD /proc/69721/task/69732/fd (opendir: Permission denied) 2024-03-21T10:28:38.522 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mds 69721 69733 io_contex ceph cwd unknown /proc/69721/task/69733/cwd (readlink: Permission denied) 2024-03-21T10:28:38.522 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mds 69721 69733 io_contex ceph rtd unknown /proc/69721/task/69733/root (readlink: Permission denied) 2024-03-21T10:28:38.522 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mds 69721 69733 io_contex ceph txt unknown /proc/69721/task/69733/exe (readlink: Permission denied) 2024-03-21T10:28:38.522 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mds 69721 69733 io_contex ceph NOFD /proc/69721/task/69733/fd (opendir: Permission denied) 2024-03-21T10:28:38.522 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mds 69721 69734 io_contex ceph cwd unknown /proc/69721/task/69734/cwd (readlink: Permission denied) 2024-03-21T10:28:38.523 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mds 69721 69734 io_contex ceph rtd unknown /proc/69721/task/69734/root (readlink: Permission denied) 2024-03-21T10:28:38.523 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mds 69721 69734 io_contex ceph txt unknown /proc/69721/task/69734/exe (readlink: Permission denied) 2024-03-21T10:28:38.523 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mds 69721 69734 io_contex ceph NOFD /proc/69721/task/69734/fd (opendir: Permission denied) 2024-03-21T10:28:38.523 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mds 69721 69735 ms_dispat ceph cwd unknown /proc/69721/task/69735/cwd (readlink: Permission denied) 2024-03-21T10:28:38.523 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mds 69721 69735 ms_dispat ceph rtd unknown /proc/69721/task/69735/root (readlink: Permission denied) 2024-03-21T10:28:38.523 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mds 69721 69735 ms_dispat ceph txt unknown /proc/69721/task/69735/exe (readlink: Permission denied) 2024-03-21T10:28:38.523 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mds 69721 69735 ms_dispat ceph NOFD /proc/69721/task/69735/fd (opendir: Permission denied) 2024-03-21T10:28:38.523 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mds 69721 69736 ms_local ceph cwd unknown /proc/69721/task/69736/cwd (readlink: Permission denied) 2024-03-21T10:28:38.523 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mds 69721 69736 ms_local ceph rtd unknown /proc/69721/task/69736/root (readlink: Permission denied) 2024-03-21T10:28:38.523 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mds 69721 69736 ms_local ceph txt unknown /proc/69721/task/69736/exe (readlink: Permission denied) 2024-03-21T10:28:38.523 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mds 69721 69736 ms_local ceph NOFD /proc/69721/task/69736/fd (opendir: Permission denied) 2024-03-21T10:28:38.523 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mds 69721 69737 safe_time ceph cwd unknown /proc/69721/task/69737/cwd (readlink: Permission denied) 2024-03-21T10:28:38.523 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mds 69721 69737 safe_time ceph rtd unknown /proc/69721/task/69737/root (readlink: Permission denied) 2024-03-21T10:28:38.523 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mds 69721 69737 safe_time ceph txt unknown /proc/69721/task/69737/exe (readlink: Permission denied) 2024-03-21T10:28:38.524 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mds 69721 69737 safe_time ceph NOFD /proc/69721/task/69737/fd (opendir: Permission denied) 2024-03-21T10:28:38.524 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mds 69721 69738 safe_time ceph cwd unknown /proc/69721/task/69738/cwd (readlink: Permission denied) 2024-03-21T10:28:38.524 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mds 69721 69738 safe_time ceph rtd unknown /proc/69721/task/69738/root (readlink: Permission denied) 2024-03-21T10:28:38.524 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mds 69721 69738 safe_time ceph txt unknown /proc/69721/task/69738/exe (readlink: Permission denied) 2024-03-21T10:28:38.524 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mds 69721 69738 safe_time ceph NOFD /proc/69721/task/69738/fd (opendir: Permission denied) 2024-03-21T10:28:38.524 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mds 69721 69739 ceph-mds ceph cwd unknown /proc/69721/task/69739/cwd (readlink: Permission denied) 2024-03-21T10:28:38.524 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mds 69721 69739 ceph-mds ceph rtd unknown /proc/69721/task/69739/root (readlink: Permission denied) 2024-03-21T10:28:38.524 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mds 69721 69739 ceph-mds ceph txt unknown /proc/69721/task/69739/exe (readlink: Permission denied) 2024-03-21T10:28:38.524 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mds 69721 69739 ceph-mds ceph NOFD /proc/69721/task/69739/fd (opendir: Permission denied) 2024-03-21T10:28:38.524 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mds 69721 69804 safe_time ceph cwd unknown /proc/69721/task/69804/cwd (readlink: Permission denied) 2024-03-21T10:28:38.524 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mds 69721 69804 safe_time ceph rtd unknown /proc/69721/task/69804/root (readlink: Permission denied) 2024-03-21T10:28:38.524 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mds 69721 69804 safe_time ceph txt unknown /proc/69721/task/69804/exe (readlink: Permission denied) 2024-03-21T10:28:38.524 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mds 69721 69804 safe_time ceph NOFD /proc/69721/task/69804/fd (opendir: Permission denied) 2024-03-21T10:28:38.524 INFO:tasks.workunit.client.0.smithi047.stdout:conmon 70102 root cwd unknown /proc/70102/cwd (readlink: Permission denied) 2024-03-21T10:28:38.524 INFO:tasks.workunit.client.0.smithi047.stdout:conmon 70102 root rtd unknown /proc/70102/root (readlink: Permission denied) 2024-03-21T10:28:38.525 INFO:tasks.workunit.client.0.smithi047.stdout:conmon 70102 root txt unknown /proc/70102/exe (readlink: Permission denied) 2024-03-21T10:28:38.525 INFO:tasks.workunit.client.0.smithi047.stdout:conmon 70102 root NOFD /proc/70102/fd (opendir: Permission denied) 2024-03-21T10:28:38.525 INFO:tasks.workunit.client.0.smithi047.stdout:podman-in 70104 root cwd unknown /proc/70104/cwd (readlink: Permission denied) 2024-03-21T10:28:38.525 INFO:tasks.workunit.client.0.smithi047.stdout:podman-in 70104 root rtd unknown /proc/70104/root (readlink: Permission denied) 2024-03-21T10:28:38.525 INFO:tasks.workunit.client.0.smithi047.stdout:podman-in 70104 root txt unknown /proc/70104/exe (readlink: Permission denied) 2024-03-21T10:28:38.525 INFO:tasks.workunit.client.0.smithi047.stdout:podman-in 70104 root NOFD /proc/70104/fd (opendir: Permission denied) 2024-03-21T10:28:38.525 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mds 70106 ceph cwd unknown /proc/70106/cwd (readlink: Permission denied) 2024-03-21T10:28:38.525 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mds 70106 ceph rtd unknown /proc/70106/root (readlink: Permission denied) 2024-03-21T10:28:38.525 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mds 70106 ceph txt unknown /proc/70106/exe (readlink: Permission denied) 2024-03-21T10:28:38.525 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mds 70106 ceph NOFD /proc/70106/fd (opendir: Permission denied) 2024-03-21T10:28:38.525 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mds 70106 70107 log ceph cwd unknown /proc/70106/task/70107/cwd (readlink: Permission denied) 2024-03-21T10:28:38.525 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mds 70106 70107 log ceph rtd unknown /proc/70106/task/70107/root (readlink: Permission denied) 2024-03-21T10:28:38.525 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mds 70106 70107 log ceph txt unknown /proc/70106/task/70107/exe (readlink: Permission denied) 2024-03-21T10:28:38.525 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mds 70106 70107 log ceph NOFD /proc/70106/task/70107/fd (opendir: Permission denied) 2024-03-21T10:28:38.526 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mds 70106 70109 msgr-work ceph cwd unknown /proc/70106/task/70109/cwd (readlink: Permission denied) 2024-03-21T10:28:38.526 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mds 70106 70109 msgr-work ceph rtd unknown /proc/70106/task/70109/root (readlink: Permission denied) 2024-03-21T10:28:38.526 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mds 70106 70109 msgr-work ceph txt unknown /proc/70106/task/70109/exe (readlink: Permission denied) 2024-03-21T10:28:38.526 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mds 70106 70109 msgr-work ceph NOFD /proc/70106/task/70109/fd (opendir: Permission denied) 2024-03-21T10:28:38.526 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mds 70106 70110 msgr-work ceph cwd unknown /proc/70106/task/70110/cwd (readlink: Permission denied) 2024-03-21T10:28:38.526 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mds 70106 70110 msgr-work ceph rtd unknown /proc/70106/task/70110/root (readlink: Permission denied) 2024-03-21T10:28:38.526 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mds 70106 70110 msgr-work ceph txt unknown /proc/70106/task/70110/exe (readlink: Permission denied) 2024-03-21T10:28:38.526 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mds 70106 70110 msgr-work ceph NOFD /proc/70106/task/70110/fd (opendir: Permission denied) 2024-03-21T10:28:38.526 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mds 70106 70111 msgr-work ceph cwd unknown /proc/70106/task/70111/cwd (readlink: Permission denied) 2024-03-21T10:28:38.526 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mds 70106 70111 msgr-work ceph rtd unknown /proc/70106/task/70111/root (readlink: Permission denied) 2024-03-21T10:28:38.526 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mds 70106 70111 msgr-work ceph txt unknown /proc/70106/task/70111/exe (readlink: Permission denied) 2024-03-21T10:28:38.526 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mds 70106 70111 msgr-work ceph NOFD /proc/70106/task/70111/fd (opendir: Permission denied) 2024-03-21T10:28:38.526 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mds 70106 70115 service ceph cwd unknown /proc/70106/task/70115/cwd (readlink: Permission denied) 2024-03-21T10:28:38.527 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mds 70106 70115 service ceph rtd unknown /proc/70106/task/70115/root (readlink: Permission denied) 2024-03-21T10:28:38.527 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mds 70106 70115 service ceph txt unknown /proc/70106/task/70115/exe (readlink: Permission denied) 2024-03-21T10:28:38.527 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mds 70106 70115 service ceph NOFD /proc/70106/task/70115/fd (opendir: Permission denied) 2024-03-21T10:28:38.527 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mds 70106 70116 admin_soc ceph cwd unknown /proc/70106/task/70116/cwd (readlink: Permission denied) 2024-03-21T10:28:38.527 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mds 70106 70116 admin_soc ceph rtd unknown /proc/70106/task/70116/root (readlink: Permission denied) 2024-03-21T10:28:38.527 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mds 70106 70116 admin_soc ceph txt unknown /proc/70106/task/70116/exe (readlink: Permission denied) 2024-03-21T10:28:38.527 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mds 70106 70116 admin_soc ceph NOFD /proc/70106/task/70116/fd (opendir: Permission denied) 2024-03-21T10:28:38.527 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mds 70106 70117 signal_ha ceph cwd unknown /proc/70106/task/70117/cwd (readlink: Permission denied) 2024-03-21T10:28:38.527 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mds 70106 70117 signal_ha ceph rtd unknown /proc/70106/task/70117/root (readlink: Permission denied) 2024-03-21T10:28:38.527 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mds 70106 70117 signal_ha ceph txt unknown /proc/70106/task/70117/exe (readlink: Permission denied) 2024-03-21T10:28:38.527 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mds 70106 70117 signal_ha ceph NOFD /proc/70106/task/70117/fd (opendir: Permission denied) 2024-03-21T10:28:38.527 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mds 70106 70118 io_contex ceph cwd unknown /proc/70106/task/70118/cwd (readlink: Permission denied) 2024-03-21T10:28:38.527 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mds 70106 70118 io_contex ceph rtd unknown /proc/70106/task/70118/root (readlink: Permission denied) 2024-03-21T10:28:38.527 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mds 70106 70118 io_contex ceph txt unknown /proc/70106/task/70118/exe (readlink: Permission denied) 2024-03-21T10:28:38.528 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mds 70106 70118 io_contex ceph NOFD /proc/70106/task/70118/fd (opendir: Permission denied) 2024-03-21T10:28:38.528 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mds 70106 70119 io_contex ceph cwd unknown /proc/70106/task/70119/cwd (readlink: Permission denied) 2024-03-21T10:28:38.528 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mds 70106 70119 io_contex ceph rtd unknown /proc/70106/task/70119/root (readlink: Permission denied) 2024-03-21T10:28:38.528 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mds 70106 70119 io_contex ceph txt unknown /proc/70106/task/70119/exe (readlink: Permission denied) 2024-03-21T10:28:38.528 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mds 70106 70119 io_contex ceph NOFD /proc/70106/task/70119/fd (opendir: Permission denied) 2024-03-21T10:28:38.528 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mds 70106 70120 ms_dispat ceph cwd unknown /proc/70106/task/70120/cwd (readlink: Permission denied) 2024-03-21T10:28:38.528 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mds 70106 70120 ms_dispat ceph rtd unknown /proc/70106/task/70120/root (readlink: Permission denied) 2024-03-21T10:28:38.528 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mds 70106 70120 ms_dispat ceph txt unknown /proc/70106/task/70120/exe (readlink: Permission denied) 2024-03-21T10:28:38.528 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mds 70106 70120 ms_dispat ceph NOFD /proc/70106/task/70120/fd (opendir: Permission denied) 2024-03-21T10:28:38.528 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mds 70106 70121 ms_local ceph cwd unknown /proc/70106/task/70121/cwd (readlink: Permission denied) 2024-03-21T10:28:38.528 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mds 70106 70121 ms_local ceph rtd unknown /proc/70106/task/70121/root (readlink: Permission denied) 2024-03-21T10:28:38.528 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mds 70106 70121 ms_local ceph txt unknown /proc/70106/task/70121/exe (readlink: Permission denied) 2024-03-21T10:28:38.528 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mds 70106 70121 ms_local ceph NOFD /proc/70106/task/70121/fd (opendir: Permission denied) 2024-03-21T10:28:38.528 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mds 70106 70122 safe_time ceph cwd unknown /proc/70106/task/70122/cwd (readlink: Permission denied) 2024-03-21T10:28:38.529 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mds 70106 70122 safe_time ceph rtd unknown /proc/70106/task/70122/root (readlink: Permission denied) 2024-03-21T10:28:38.529 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mds 70106 70122 safe_time ceph txt unknown /proc/70106/task/70122/exe (readlink: Permission denied) 2024-03-21T10:28:38.529 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mds 70106 70122 safe_time ceph NOFD /proc/70106/task/70122/fd (opendir: Permission denied) 2024-03-21T10:28:38.529 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mds 70106 70123 safe_time ceph cwd unknown /proc/70106/task/70123/cwd (readlink: Permission denied) 2024-03-21T10:28:38.529 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mds 70106 70123 safe_time ceph rtd unknown /proc/70106/task/70123/root (readlink: Permission denied) 2024-03-21T10:28:38.529 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mds 70106 70123 safe_time ceph txt unknown /proc/70106/task/70123/exe (readlink: Permission denied) 2024-03-21T10:28:38.529 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mds 70106 70123 safe_time ceph NOFD /proc/70106/task/70123/fd (opendir: Permission denied) 2024-03-21T10:28:38.529 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mds 70106 70124 ceph-mds ceph cwd unknown /proc/70106/task/70124/cwd (readlink: Permission denied) 2024-03-21T10:28:38.529 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mds 70106 70124 ceph-mds ceph rtd unknown /proc/70106/task/70124/root (readlink: Permission denied) 2024-03-21T10:28:38.529 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mds 70106 70124 ceph-mds ceph txt unknown /proc/70106/task/70124/exe (readlink: Permission denied) 2024-03-21T10:28:38.529 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mds 70106 70124 ceph-mds ceph NOFD /proc/70106/task/70124/fd (opendir: Permission denied) 2024-03-21T10:28:38.529 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mds 70106 70129 safe_time ceph cwd unknown /proc/70106/task/70129/cwd (readlink: Permission denied) 2024-03-21T10:28:38.529 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mds 70106 70129 safe_time ceph rtd unknown /proc/70106/task/70129/root (readlink: Permission denied) 2024-03-21T10:28:38.529 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mds 70106 70129 safe_time ceph txt unknown /proc/70106/task/70129/exe (readlink: Permission denied) 2024-03-21T10:28:38.529 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mds 70106 70129 safe_time ceph NOFD /proc/70106/task/70129/fd (opendir: Permission denied) 2024-03-21T10:28:38.530 INFO:tasks.workunit.client.0.smithi047.stdout:kworker/u 70130 root cwd unknown /proc/70130/cwd (readlink: Permission denied) 2024-03-21T10:28:38.530 INFO:tasks.workunit.client.0.smithi047.stdout:kworker/u 70130 root rtd unknown /proc/70130/root (readlink: Permission denied) 2024-03-21T10:28:38.530 INFO:tasks.workunit.client.0.smithi047.stdout:kworker/u 70130 root txt unknown /proc/70130/exe (readlink: Permission denied) 2024-03-21T10:28:38.530 INFO:tasks.workunit.client.0.smithi047.stdout:kworker/u 70130 root NOFD /proc/70130/fd (opendir: Permission denied) 2024-03-21T10:28:38.530 INFO:tasks.workunit.client.0.smithi047.stdout:kworker/0 70628 root cwd unknown /proc/70628/cwd (readlink: Permission denied) 2024-03-21T10:28:38.530 INFO:tasks.workunit.client.0.smithi047.stdout:kworker/0 70628 root rtd unknown /proc/70628/root (readlink: Permission denied) 2024-03-21T10:28:38.530 INFO:tasks.workunit.client.0.smithi047.stdout:kworker/0 70628 root txt unknown /proc/70628/exe (readlink: Permission denied) 2024-03-21T10:28:38.530 INFO:tasks.workunit.client.0.smithi047.stdout:kworker/0 70628 root NOFD /proc/70628/fd (opendir: Permission denied) 2024-03-21T10:28:38.530 INFO:tasks.workunit.client.0.smithi047.stdout:conmon 70784 root cwd unknown /proc/70784/cwd (readlink: Permission denied) 2024-03-21T10:28:38.530 INFO:tasks.workunit.client.0.smithi047.stdout:conmon 70784 root rtd unknown /proc/70784/root (readlink: Permission denied) 2024-03-21T10:28:38.530 INFO:tasks.workunit.client.0.smithi047.stdout:conmon 70784 root txt unknown /proc/70784/exe (readlink: Permission denied) 2024-03-21T10:28:38.530 INFO:tasks.workunit.client.0.smithi047.stdout:conmon 70784 root NOFD /proc/70784/fd (opendir: Permission denied) 2024-03-21T10:28:38.530 INFO:tasks.workunit.client.0.smithi047.stdout:podman-in 70786 root cwd unknown /proc/70786/cwd (readlink: Permission denied) 2024-03-21T10:28:38.530 INFO:tasks.workunit.client.0.smithi047.stdout:podman-in 70786 root rtd unknown /proc/70786/root (readlink: Permission denied) 2024-03-21T10:28:38.531 INFO:tasks.workunit.client.0.smithi047.stdout:podman-in 70786 root txt unknown /proc/70786/exe (readlink: Permission denied) 2024-03-21T10:28:38.531 INFO:tasks.workunit.client.0.smithi047.stdout:podman-in 70786 root NOFD /proc/70786/fd (opendir: Permission denied) 2024-03-21T10:28:38.531 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mds 70808 ceph cwd unknown /proc/70808/cwd (readlink: Permission denied) 2024-03-21T10:28:38.531 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mds 70808 ceph rtd unknown /proc/70808/root (readlink: Permission denied) 2024-03-21T10:28:38.531 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mds 70808 ceph txt unknown /proc/70808/exe (readlink: Permission denied) 2024-03-21T10:28:38.531 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mds 70808 ceph NOFD /proc/70808/fd (opendir: Permission denied) 2024-03-21T10:28:38.531 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mds 70808 70810 log ceph cwd unknown /proc/70808/task/70810/cwd (readlink: Permission denied) 2024-03-21T10:28:38.531 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mds 70808 70810 log ceph rtd unknown /proc/70808/task/70810/root (readlink: Permission denied) 2024-03-21T10:28:38.531 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mds 70808 70810 log ceph txt unknown /proc/70808/task/70810/exe (readlink: Permission denied) 2024-03-21T10:28:38.531 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mds 70808 70810 log ceph NOFD /proc/70808/task/70810/fd (opendir: Permission denied) 2024-03-21T10:28:38.531 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mds 70808 70812 msgr-work ceph cwd unknown /proc/70808/task/70812/cwd (readlink: Permission denied) 2024-03-21T10:28:38.531 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mds 70808 70812 msgr-work ceph rtd unknown /proc/70808/task/70812/root (readlink: Permission denied) 2024-03-21T10:28:38.531 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mds 70808 70812 msgr-work ceph txt unknown /proc/70808/task/70812/exe (readlink: Permission denied) 2024-03-21T10:28:38.532 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mds 70808 70812 msgr-work ceph NOFD /proc/70808/task/70812/fd (opendir: Permission denied) 2024-03-21T10:28:38.532 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mds 70808 70813 msgr-work ceph cwd unknown /proc/70808/task/70813/cwd (readlink: Permission denied) 2024-03-21T10:28:38.532 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mds 70808 70813 msgr-work ceph rtd unknown /proc/70808/task/70813/root (readlink: Permission denied) 2024-03-21T10:28:38.532 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mds 70808 70813 msgr-work ceph txt unknown /proc/70808/task/70813/exe (readlink: Permission denied) 2024-03-21T10:28:38.532 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mds 70808 70813 msgr-work ceph NOFD /proc/70808/task/70813/fd (opendir: Permission denied) 2024-03-21T10:28:38.532 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mds 70808 70814 msgr-work ceph cwd unknown /proc/70808/task/70814/cwd (readlink: Permission denied) 2024-03-21T10:28:38.532 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mds 70808 70814 msgr-work ceph rtd unknown /proc/70808/task/70814/root (readlink: Permission denied) 2024-03-21T10:28:38.532 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mds 70808 70814 msgr-work ceph txt unknown /proc/70808/task/70814/exe (readlink: Permission denied) 2024-03-21T10:28:38.532 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mds 70808 70814 msgr-work ceph NOFD /proc/70808/task/70814/fd (opendir: Permission denied) 2024-03-21T10:28:38.532 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mds 70808 70818 service ceph cwd unknown /proc/70808/task/70818/cwd (readlink: Permission denied) 2024-03-21T10:28:38.532 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mds 70808 70818 service ceph rtd unknown /proc/70808/task/70818/root (readlink: Permission denied) 2024-03-21T10:28:38.532 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mds 70808 70818 service ceph txt unknown /proc/70808/task/70818/exe (readlink: Permission denied) 2024-03-21T10:28:38.532 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mds 70808 70818 service ceph NOFD /proc/70808/task/70818/fd (opendir: Permission denied) 2024-03-21T10:28:38.532 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mds 70808 70819 admin_soc ceph cwd unknown /proc/70808/task/70819/cwd (readlink: Permission denied) 2024-03-21T10:28:38.533 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mds 70808 70819 admin_soc ceph rtd unknown /proc/70808/task/70819/root (readlink: Permission denied) 2024-03-21T10:28:38.533 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mds 70808 70819 admin_soc ceph txt unknown /proc/70808/task/70819/exe (readlink: Permission denied) 2024-03-21T10:28:38.533 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mds 70808 70819 admin_soc ceph NOFD /proc/70808/task/70819/fd (opendir: Permission denied) 2024-03-21T10:28:38.533 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mds 70808 70820 signal_ha ceph cwd unknown /proc/70808/task/70820/cwd (readlink: Permission denied) 2024-03-21T10:28:38.533 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mds 70808 70820 signal_ha ceph rtd unknown /proc/70808/task/70820/root (readlink: Permission denied) 2024-03-21T10:28:38.533 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mds 70808 70820 signal_ha ceph txt unknown /proc/70808/task/70820/exe (readlink: Permission denied) 2024-03-21T10:28:38.533 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mds 70808 70820 signal_ha ceph NOFD /proc/70808/task/70820/fd (opendir: Permission denied) 2024-03-21T10:28:38.533 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mds 70808 70821 io_contex ceph cwd unknown /proc/70808/task/70821/cwd (readlink: Permission denied) 2024-03-21T10:28:38.533 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mds 70808 70821 io_contex ceph rtd unknown /proc/70808/task/70821/root (readlink: Permission denied) 2024-03-21T10:28:38.533 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mds 70808 70821 io_contex ceph txt unknown /proc/70808/task/70821/exe (readlink: Permission denied) 2024-03-21T10:28:38.533 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mds 70808 70821 io_contex ceph NOFD /proc/70808/task/70821/fd (opendir: Permission denied) 2024-03-21T10:28:38.533 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mds 70808 70822 io_contex ceph cwd unknown /proc/70808/task/70822/cwd (readlink: Permission denied) 2024-03-21T10:28:38.533 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mds 70808 70822 io_contex ceph rtd unknown /proc/70808/task/70822/root (readlink: Permission denied) 2024-03-21T10:28:38.533 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mds 70808 70822 io_contex ceph txt unknown /proc/70808/task/70822/exe (readlink: Permission denied) 2024-03-21T10:28:38.533 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mds 70808 70822 io_contex ceph NOFD /proc/70808/task/70822/fd (opendir: Permission denied) 2024-03-21T10:28:38.533 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mds 70808 70823 ms_dispat ceph cwd unknown /proc/70808/task/70823/cwd (readlink: Permission denied) 2024-03-21T10:28:38.534 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mds 70808 70823 ms_dispat ceph rtd unknown /proc/70808/task/70823/root (readlink: Permission denied) 2024-03-21T10:28:38.534 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mds 70808 70823 ms_dispat ceph txt unknown /proc/70808/task/70823/exe (readlink: Permission denied) 2024-03-21T10:28:38.534 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mds 70808 70823 ms_dispat ceph NOFD /proc/70808/task/70823/fd (opendir: Permission denied) 2024-03-21T10:28:38.534 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mds 70808 70824 ms_local ceph cwd unknown /proc/70808/task/70824/cwd (readlink: Permission denied) 2024-03-21T10:28:38.534 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mds 70808 70824 ms_local ceph rtd unknown /proc/70808/task/70824/root (readlink: Permission denied) 2024-03-21T10:28:38.534 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mds 70808 70824 ms_local ceph txt unknown /proc/70808/task/70824/exe (readlink: Permission denied) 2024-03-21T10:28:38.534 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mds 70808 70824 ms_local ceph NOFD /proc/70808/task/70824/fd (opendir: Permission denied) 2024-03-21T10:28:38.534 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mds 70808 70825 safe_time ceph cwd unknown /proc/70808/task/70825/cwd (readlink: Permission denied) 2024-03-21T10:28:38.534 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mds 70808 70825 safe_time ceph rtd unknown /proc/70808/task/70825/root (readlink: Permission denied) 2024-03-21T10:28:38.534 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mds 70808 70825 safe_time ceph txt unknown /proc/70808/task/70825/exe (readlink: Permission denied) 2024-03-21T10:28:38.534 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mds 70808 70825 safe_time ceph NOFD /proc/70808/task/70825/fd (opendir: Permission denied) 2024-03-21T10:28:38.534 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mds 70808 70826 safe_time ceph cwd unknown /proc/70808/task/70826/cwd (readlink: Permission denied) 2024-03-21T10:28:38.534 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mds 70808 70826 safe_time ceph rtd unknown /proc/70808/task/70826/root (readlink: Permission denied) 2024-03-21T10:28:38.534 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mds 70808 70826 safe_time ceph txt unknown /proc/70808/task/70826/exe (readlink: Permission denied) 2024-03-21T10:28:38.535 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mds 70808 70826 safe_time ceph NOFD /proc/70808/task/70826/fd (opendir: Permission denied) 2024-03-21T10:28:38.535 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mds 70808 70827 ceph-mds ceph cwd unknown /proc/70808/task/70827/cwd (readlink: Permission denied) 2024-03-21T10:28:38.535 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mds 70808 70827 ceph-mds ceph rtd unknown /proc/70808/task/70827/root (readlink: Permission denied) 2024-03-21T10:28:38.535 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mds 70808 70827 ceph-mds ceph txt unknown /proc/70808/task/70827/exe (readlink: Permission denied) 2024-03-21T10:28:38.535 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mds 70808 70827 ceph-mds ceph NOFD /proc/70808/task/70827/fd (opendir: Permission denied) 2024-03-21T10:28:38.535 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mds 70808 70851 safe_time ceph cwd unknown /proc/70808/task/70851/cwd (readlink: Permission denied) 2024-03-21T10:28:38.535 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mds 70808 70851 safe_time ceph rtd unknown /proc/70808/task/70851/root (readlink: Permission denied) 2024-03-21T10:28:38.535 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mds 70808 70851 safe_time ceph txt unknown /proc/70808/task/70851/exe (readlink: Permission denied) 2024-03-21T10:28:38.535 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mds 70808 70851 safe_time ceph NOFD /proc/70808/task/70851/fd (opendir: Permission denied) 2024-03-21T10:28:38.535 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mds 70808 74590 ceph_time ceph cwd unknown /proc/70808/task/74590/cwd (readlink: Permission denied) 2024-03-21T10:28:38.535 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mds 70808 74590 ceph_time ceph rtd unknown /proc/70808/task/74590/root (readlink: Permission denied) 2024-03-21T10:28:38.535 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mds 70808 74590 ceph_time ceph txt unknown /proc/70808/task/74590/exe (readlink: Permission denied) 2024-03-21T10:28:38.535 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mds 70808 74590 ceph_time ceph NOFD /proc/70808/task/74590/fd (opendir: Permission denied) 2024-03-21T10:28:38.535 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mds 70808 74591 OpHistory ceph cwd unknown /proc/70808/task/74591/cwd (readlink: Permission denied) 2024-03-21T10:28:38.536 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mds 70808 74591 OpHistory ceph rtd unknown /proc/70808/task/74591/root (readlink: Permission denied) 2024-03-21T10:28:38.536 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mds 70808 74591 OpHistory ceph txt unknown /proc/70808/task/74591/exe (readlink: Permission denied) 2024-03-21T10:28:38.536 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mds 70808 74591 OpHistory ceph NOFD /proc/70808/task/74591/fd (opendir: Permission denied) 2024-03-21T10:28:38.536 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mds 70808 74592 ms_dispat ceph cwd unknown /proc/70808/task/74592/cwd (readlink: Permission denied) 2024-03-21T10:28:38.536 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mds 70808 74592 ms_dispat ceph rtd unknown /proc/70808/task/74592/root (readlink: Permission denied) 2024-03-21T10:28:38.536 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mds 70808 74592 ms_dispat ceph txt unknown /proc/70808/task/74592/exe (readlink: Permission denied) 2024-03-21T10:28:38.536 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mds 70808 74592 ms_dispat ceph NOFD /proc/70808/task/74592/fd (opendir: Permission denied) 2024-03-21T10:28:38.536 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mds 70808 74593 ms_dispat ceph cwd unknown /proc/70808/task/74593/cwd (readlink: Permission denied) 2024-03-21T10:28:38.536 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mds 70808 74593 ms_dispat ceph rtd unknown /proc/70808/task/74593/root (readlink: Permission denied) 2024-03-21T10:28:38.536 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mds 70808 74593 ms_dispat ceph txt unknown /proc/70808/task/74593/exe (readlink: Permission denied) 2024-03-21T10:28:38.536 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mds 70808 74593 ms_dispat ceph NOFD /proc/70808/task/74593/fd (opendir: Permission denied) 2024-03-21T10:28:38.536 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mds 70808 74594 mds_rank_ ceph cwd unknown /proc/70808/task/74594/cwd (readlink: Permission denied) 2024-03-21T10:28:38.536 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mds 70808 74594 mds_rank_ ceph rtd unknown /proc/70808/task/74594/root (readlink: Permission denied) 2024-03-21T10:28:38.536 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mds 70808 74594 mds_rank_ ceph txt unknown /proc/70808/task/74594/exe (readlink: Permission denied) 2024-03-21T10:28:38.536 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mds 70808 74594 mds_rank_ ceph NOFD /proc/70808/task/74594/fd (opendir: Permission denied) 2024-03-21T10:28:38.537 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mds 70808 74595 PQ_Finish ceph cwd unknown /proc/70808/task/74595/cwd (readlink: Permission denied) 2024-03-21T10:28:38.537 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mds 70808 74595 PQ_Finish ceph rtd unknown /proc/70808/task/74595/root (readlink: Permission denied) 2024-03-21T10:28:38.537 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mds 70808 74595 PQ_Finish ceph txt unknown /proc/70808/task/74595/exe (readlink: Permission denied) 2024-03-21T10:28:38.537 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mds 70808 74595 PQ_Finish ceph NOFD /proc/70808/task/74595/fd (opendir: Permission denied) 2024-03-21T10:28:38.537 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mds 70808 74596 safe_time ceph cwd unknown /proc/70808/task/74596/cwd (readlink: Permission denied) 2024-03-21T10:28:38.537 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mds 70808 74596 safe_time ceph rtd unknown /proc/70808/task/74596/root (readlink: Permission denied) 2024-03-21T10:28:38.537 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mds 70808 74596 safe_time ceph txt unknown /proc/70808/task/74596/exe (readlink: Permission denied) 2024-03-21T10:28:38.537 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mds 70808 74596 safe_time ceph NOFD /proc/70808/task/74596/fd (opendir: Permission denied) 2024-03-21T10:28:38.537 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mds 70808 74597 MR_Finish ceph cwd unknown /proc/70808/task/74597/cwd (readlink: Permission denied) 2024-03-21T10:28:38.537 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mds 70808 74597 MR_Finish ceph rtd unknown /proc/70808/task/74597/root (readlink: Permission denied) 2024-03-21T10:28:38.537 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mds 70808 74597 MR_Finish ceph txt unknown /proc/70808/task/74597/exe (readlink: Permission denied) 2024-03-21T10:28:38.537 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mds 70808 74597 MR_Finish ceph NOFD /proc/70808/task/74597/fd (opendir: Permission denied) 2024-03-21T10:28:38.537 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mds 70808 74599 md_submit ceph cwd unknown /proc/70808/task/74599/cwd (readlink: Permission denied) 2024-03-21T10:28:38.537 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mds 70808 74599 md_submit ceph rtd unknown /proc/70808/task/74599/root (readlink: Permission denied) 2024-03-21T10:28:38.538 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mds 70808 74599 md_submit ceph txt unknown /proc/70808/task/74599/exe (readlink: Permission denied) 2024-03-21T10:28:38.538 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mds 70808 74599 md_submit ceph NOFD /proc/70808/task/74599/fd (opendir: Permission denied) 2024-03-21T10:28:38.538 INFO:tasks.workunit.client.0.smithi047.stdout:kworker/2 71124 root cwd unknown /proc/71124/cwd (readlink: Permission denied) 2024-03-21T10:28:38.538 INFO:tasks.workunit.client.0.smithi047.stdout:kworker/2 71124 root rtd unknown /proc/71124/root (readlink: Permission denied) 2024-03-21T10:28:38.538 INFO:tasks.workunit.client.0.smithi047.stdout:kworker/2 71124 root txt unknown /proc/71124/exe (readlink: Permission denied) 2024-03-21T10:28:38.538 INFO:tasks.workunit.client.0.smithi047.stdout:kworker/2 71124 root NOFD /proc/71124/fd (opendir: Permission denied) 2024-03-21T10:28:38.538 INFO:tasks.workunit.client.0.smithi047.stdout:conmon 71185 root cwd unknown /proc/71185/cwd (readlink: Permission denied) 2024-03-21T10:28:38.538 INFO:tasks.workunit.client.0.smithi047.stdout:conmon 71185 root rtd unknown /proc/71185/root (readlink: Permission denied) 2024-03-21T10:28:38.538 INFO:tasks.workunit.client.0.smithi047.stdout:conmon 71185 root txt unknown /proc/71185/exe (readlink: Permission denied) 2024-03-21T10:28:38.538 INFO:tasks.workunit.client.0.smithi047.stdout:conmon 71185 root NOFD /proc/71185/fd (opendir: Permission denied) 2024-03-21T10:28:38.538 INFO:tasks.workunit.client.0.smithi047.stdout:podman-in 71187 root cwd unknown /proc/71187/cwd (readlink: Permission denied) 2024-03-21T10:28:38.538 INFO:tasks.workunit.client.0.smithi047.stdout:podman-in 71187 root rtd unknown /proc/71187/root (readlink: Permission denied) 2024-03-21T10:28:38.538 INFO:tasks.workunit.client.0.smithi047.stdout:podman-in 71187 root txt unknown /proc/71187/exe (readlink: Permission denied) 2024-03-21T10:28:38.539 INFO:tasks.workunit.client.0.smithi047.stdout:podman-in 71187 root NOFD /proc/71187/fd (opendir: Permission denied) 2024-03-21T10:28:38.539 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mds 71189 ceph cwd unknown /proc/71189/cwd (readlink: Permission denied) 2024-03-21T10:28:38.539 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mds 71189 ceph rtd unknown /proc/71189/root (readlink: Permission denied) 2024-03-21T10:28:38.539 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mds 71189 ceph txt unknown /proc/71189/exe (readlink: Permission denied) 2024-03-21T10:28:38.539 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mds 71189 ceph NOFD /proc/71189/fd (opendir: Permission denied) 2024-03-21T10:28:38.539 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mds 71189 71190 log ceph cwd unknown /proc/71189/task/71190/cwd (readlink: Permission denied) 2024-03-21T10:28:38.539 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mds 71189 71190 log ceph rtd unknown /proc/71189/task/71190/root (readlink: Permission denied) 2024-03-21T10:28:38.539 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mds 71189 71190 log ceph txt unknown /proc/71189/task/71190/exe (readlink: Permission denied) 2024-03-21T10:28:38.539 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mds 71189 71190 log ceph NOFD /proc/71189/task/71190/fd (opendir: Permission denied) 2024-03-21T10:28:38.539 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mds 71189 71192 msgr-work ceph cwd unknown /proc/71189/task/71192/cwd (readlink: Permission denied) 2024-03-21T10:28:38.539 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mds 71189 71192 msgr-work ceph rtd unknown /proc/71189/task/71192/root (readlink: Permission denied) 2024-03-21T10:28:38.539 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mds 71189 71192 msgr-work ceph txt unknown /proc/71189/task/71192/exe (readlink: Permission denied) 2024-03-21T10:28:38.539 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mds 71189 71192 msgr-work ceph NOFD /proc/71189/task/71192/fd (opendir: Permission denied) 2024-03-21T10:28:38.539 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mds 71189 71193 msgr-work ceph cwd unknown /proc/71189/task/71193/cwd (readlink: Permission denied) 2024-03-21T10:28:38.539 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mds 71189 71193 msgr-work ceph rtd unknown /proc/71189/task/71193/root (readlink: Permission denied) 2024-03-21T10:28:38.540 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mds 71189 71193 msgr-work ceph txt unknown /proc/71189/task/71193/exe (readlink: Permission denied) 2024-03-21T10:28:38.540 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mds 71189 71193 msgr-work ceph NOFD /proc/71189/task/71193/fd (opendir: Permission denied) 2024-03-21T10:28:38.540 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mds 71189 71194 msgr-work ceph cwd unknown /proc/71189/task/71194/cwd (readlink: Permission denied) 2024-03-21T10:28:38.540 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mds 71189 71194 msgr-work ceph rtd unknown /proc/71189/task/71194/root (readlink: Permission denied) 2024-03-21T10:28:38.540 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mds 71189 71194 msgr-work ceph txt unknown /proc/71189/task/71194/exe (readlink: Permission denied) 2024-03-21T10:28:38.540 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mds 71189 71194 msgr-work ceph NOFD /proc/71189/task/71194/fd (opendir: Permission denied) 2024-03-21T10:28:38.540 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mds 71189 71198 service ceph cwd unknown /proc/71189/task/71198/cwd (readlink: Permission denied) 2024-03-21T10:28:38.540 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mds 71189 71198 service ceph rtd unknown /proc/71189/task/71198/root (readlink: Permission denied) 2024-03-21T10:28:38.540 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mds 71189 71198 service ceph txt unknown /proc/71189/task/71198/exe (readlink: Permission denied) 2024-03-21T10:28:38.540 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mds 71189 71198 service ceph NOFD /proc/71189/task/71198/fd (opendir: Permission denied) 2024-03-21T10:28:38.540 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mds 71189 71199 admin_soc ceph cwd unknown /proc/71189/task/71199/cwd (readlink: Permission denied) 2024-03-21T10:28:38.540 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mds 71189 71199 admin_soc ceph rtd unknown /proc/71189/task/71199/root (readlink: Permission denied) 2024-03-21T10:28:38.540 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mds 71189 71199 admin_soc ceph txt unknown /proc/71189/task/71199/exe (readlink: Permission denied) 2024-03-21T10:28:38.540 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mds 71189 71199 admin_soc ceph NOFD /proc/71189/task/71199/fd (opendir: Permission denied) 2024-03-21T10:28:38.540 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mds 71189 71200 signal_ha ceph cwd unknown /proc/71189/task/71200/cwd (readlink: Permission denied) 2024-03-21T10:28:38.540 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mds 71189 71200 signal_ha ceph rtd unknown /proc/71189/task/71200/root (readlink: Permission denied) 2024-03-21T10:28:38.541 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mds 71189 71200 signal_ha ceph txt unknown /proc/71189/task/71200/exe (readlink: Permission denied) 2024-03-21T10:28:38.541 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mds 71189 71200 signal_ha ceph NOFD /proc/71189/task/71200/fd (opendir: Permission denied) 2024-03-21T10:28:38.541 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mds 71189 71201 io_contex ceph cwd unknown /proc/71189/task/71201/cwd (readlink: Permission denied) 2024-03-21T10:28:38.541 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mds 71189 71201 io_contex ceph rtd unknown /proc/71189/task/71201/root (readlink: Permission denied) 2024-03-21T10:28:38.541 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mds 71189 71201 io_contex ceph txt unknown /proc/71189/task/71201/exe (readlink: Permission denied) 2024-03-21T10:28:38.541 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mds 71189 71201 io_contex ceph NOFD /proc/71189/task/71201/fd (opendir: Permission denied) 2024-03-21T10:28:38.541 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mds 71189 71202 io_contex ceph cwd unknown /proc/71189/task/71202/cwd (readlink: Permission denied) 2024-03-21T10:28:38.541 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mds 71189 71202 io_contex ceph rtd unknown /proc/71189/task/71202/root (readlink: Permission denied) 2024-03-21T10:28:38.541 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mds 71189 71202 io_contex ceph txt unknown /proc/71189/task/71202/exe (readlink: Permission denied) 2024-03-21T10:28:38.541 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mds 71189 71202 io_contex ceph NOFD /proc/71189/task/71202/fd (opendir: Permission denied) 2024-03-21T10:28:38.541 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mds 71189 71203 ms_dispat ceph cwd unknown /proc/71189/task/71203/cwd (readlink: Permission denied) 2024-03-21T10:28:38.541 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mds 71189 71203 ms_dispat ceph rtd unknown /proc/71189/task/71203/root (readlink: Permission denied) 2024-03-21T10:28:38.541 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mds 71189 71203 ms_dispat ceph txt unknown /proc/71189/task/71203/exe (readlink: Permission denied) 2024-03-21T10:28:38.541 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mds 71189 71203 ms_dispat ceph NOFD /proc/71189/task/71203/fd (opendir: Permission denied) 2024-03-21T10:28:38.542 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mds 71189 71204 ms_local ceph cwd unknown /proc/71189/task/71204/cwd (readlink: Permission denied) 2024-03-21T10:28:38.542 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mds 71189 71204 ms_local ceph rtd unknown /proc/71189/task/71204/root (readlink: Permission denied) 2024-03-21T10:28:38.542 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mds 71189 71204 ms_local ceph txt unknown /proc/71189/task/71204/exe (readlink: Permission denied) 2024-03-21T10:28:38.542 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mds 71189 71204 ms_local ceph NOFD /proc/71189/task/71204/fd (opendir: Permission denied) 2024-03-21T10:28:38.542 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mds 71189 71205 safe_time ceph cwd unknown /proc/71189/task/71205/cwd (readlink: Permission denied) 2024-03-21T10:28:38.542 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mds 71189 71205 safe_time ceph rtd unknown /proc/71189/task/71205/root (readlink: Permission denied) 2024-03-21T10:28:38.542 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mds 71189 71205 safe_time ceph txt unknown /proc/71189/task/71205/exe (readlink: Permission denied) 2024-03-21T10:28:38.542 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mds 71189 71205 safe_time ceph NOFD /proc/71189/task/71205/fd (opendir: Permission denied) 2024-03-21T10:28:38.542 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mds 71189 71206 safe_time ceph cwd unknown /proc/71189/task/71206/cwd (readlink: Permission denied) 2024-03-21T10:28:38.542 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mds 71189 71206 safe_time ceph rtd unknown /proc/71189/task/71206/root (readlink: Permission denied) 2024-03-21T10:28:38.542 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mds 71189 71206 safe_time ceph txt unknown /proc/71189/task/71206/exe (readlink: Permission denied) 2024-03-21T10:28:38.542 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mds 71189 71206 safe_time ceph NOFD /proc/71189/task/71206/fd (opendir: Permission denied) 2024-03-21T10:28:38.542 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mds 71189 71207 ceph-mds ceph cwd unknown /proc/71189/task/71207/cwd (readlink: Permission denied) 2024-03-21T10:28:38.542 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mds 71189 71207 ceph-mds ceph rtd unknown /proc/71189/task/71207/root (readlink: Permission denied) 2024-03-21T10:28:38.543 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mds 71189 71207 ceph-mds ceph txt unknown /proc/71189/task/71207/exe (readlink: Permission denied) 2024-03-21T10:28:38.543 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mds 71189 71207 ceph-mds ceph NOFD /proc/71189/task/71207/fd (opendir: Permission denied) 2024-03-21T10:28:38.543 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mds 71189 71212 safe_time ceph cwd unknown /proc/71189/task/71212/cwd (readlink: Permission denied) 2024-03-21T10:28:38.543 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mds 71189 71212 safe_time ceph rtd unknown /proc/71189/task/71212/root (readlink: Permission denied) 2024-03-21T10:28:38.543 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mds 71189 71212 safe_time ceph txt unknown /proc/71189/task/71212/exe (readlink: Permission denied) 2024-03-21T10:28:38.543 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mds 71189 71212 safe_time ceph NOFD /proc/71189/task/71212/fd (opendir: Permission denied) 2024-03-21T10:28:38.543 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mds 71189 74496 ceph_time ceph cwd unknown /proc/71189/task/74496/cwd (readlink: Permission denied) 2024-03-21T10:28:38.543 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mds 71189 74496 ceph_time ceph rtd unknown /proc/71189/task/74496/root (readlink: Permission denied) 2024-03-21T10:28:38.543 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mds 71189 74496 ceph_time ceph txt unknown /proc/71189/task/74496/exe (readlink: Permission denied) 2024-03-21T10:28:38.543 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mds 71189 74496 ceph_time ceph NOFD /proc/71189/task/74496/fd (opendir: Permission denied) 2024-03-21T10:28:38.543 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mds 71189 74497 OpHistory ceph cwd unknown /proc/71189/task/74497/cwd (readlink: Permission denied) 2024-03-21T10:28:38.543 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mds 71189 74497 OpHistory ceph rtd unknown /proc/71189/task/74497/root (readlink: Permission denied) 2024-03-21T10:28:38.543 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mds 71189 74497 OpHistory ceph txt unknown /proc/71189/task/74497/exe (readlink: Permission denied) 2024-03-21T10:28:38.543 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mds 71189 74497 OpHistory ceph NOFD /proc/71189/task/74497/fd (opendir: Permission denied) 2024-03-21T10:28:38.544 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mds 71189 74498 ms_dispat ceph cwd unknown /proc/71189/task/74498/cwd (readlink: Permission denied) 2024-03-21T10:28:38.544 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mds 71189 74498 ms_dispat ceph rtd unknown /proc/71189/task/74498/root (readlink: Permission denied) 2024-03-21T10:28:38.544 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mds 71189 74498 ms_dispat ceph txt unknown /proc/71189/task/74498/exe (readlink: Permission denied) 2024-03-21T10:28:38.544 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mds 71189 74498 ms_dispat ceph NOFD /proc/71189/task/74498/fd (opendir: Permission denied) 2024-03-21T10:28:38.544 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mds 71189 74499 ms_dispat ceph cwd unknown /proc/71189/task/74499/cwd (readlink: Permission denied) 2024-03-21T10:28:38.544 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mds 71189 74499 ms_dispat ceph rtd unknown /proc/71189/task/74499/root (readlink: Permission denied) 2024-03-21T10:28:38.544 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mds 71189 74499 ms_dispat ceph txt unknown /proc/71189/task/74499/exe (readlink: Permission denied) 2024-03-21T10:28:38.544 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mds 71189 74499 ms_dispat ceph NOFD /proc/71189/task/74499/fd (opendir: Permission denied) 2024-03-21T10:28:38.544 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mds 71189 74500 mds_rank_ ceph cwd unknown /proc/71189/task/74500/cwd (readlink: Permission denied) 2024-03-21T10:28:38.544 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mds 71189 74500 mds_rank_ ceph rtd unknown /proc/71189/task/74500/root (readlink: Permission denied) 2024-03-21T10:28:38.544 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mds 71189 74500 mds_rank_ ceph txt unknown /proc/71189/task/74500/exe (readlink: Permission denied) 2024-03-21T10:28:38.544 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mds 71189 74500 mds_rank_ ceph NOFD /proc/71189/task/74500/fd (opendir: Permission denied) 2024-03-21T10:28:38.544 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mds 71189 74501 PQ_Finish ceph cwd unknown /proc/71189/task/74501/cwd (readlink: Permission denied) 2024-03-21T10:28:38.544 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mds 71189 74501 PQ_Finish ceph rtd unknown /proc/71189/task/74501/root (readlink: Permission denied) 2024-03-21T10:28:38.544 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mds 71189 74501 PQ_Finish ceph txt unknown /proc/71189/task/74501/exe (readlink: Permission denied) 2024-03-21T10:28:38.545 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mds 71189 74501 PQ_Finish ceph NOFD /proc/71189/task/74501/fd (opendir: Permission denied) 2024-03-21T10:28:38.545 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mds 71189 74502 safe_time ceph cwd unknown /proc/71189/task/74502/cwd (readlink: Permission denied) 2024-03-21T10:28:38.545 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mds 71189 74502 safe_time ceph rtd unknown /proc/71189/task/74502/root (readlink: Permission denied) 2024-03-21T10:28:38.545 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mds 71189 74502 safe_time ceph txt unknown /proc/71189/task/74502/exe (readlink: Permission denied) 2024-03-21T10:28:38.545 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mds 71189 74502 safe_time ceph NOFD /proc/71189/task/74502/fd (opendir: Permission denied) 2024-03-21T10:28:38.545 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mds 71189 74503 MR_Finish ceph cwd unknown /proc/71189/task/74503/cwd (readlink: Permission denied) 2024-03-21T10:28:38.545 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mds 71189 74503 MR_Finish ceph rtd unknown /proc/71189/task/74503/root (readlink: Permission denied) 2024-03-21T10:28:38.545 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mds 71189 74503 MR_Finish ceph txt unknown /proc/71189/task/74503/exe (readlink: Permission denied) 2024-03-21T10:28:38.545 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mds 71189 74503 MR_Finish ceph NOFD /proc/71189/task/74503/fd (opendir: Permission denied) 2024-03-21T10:28:38.545 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mds 71189 74504 md_submit ceph cwd unknown /proc/71189/task/74504/cwd (readlink: Permission denied) 2024-03-21T10:28:38.545 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mds 71189 74504 md_submit ceph rtd unknown /proc/71189/task/74504/root (readlink: Permission denied) 2024-03-21T10:28:38.545 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mds 71189 74504 md_submit ceph txt unknown /proc/71189/task/74504/exe (readlink: Permission denied) 2024-03-21T10:28:38.545 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mds 71189 74504 md_submit ceph NOFD /proc/71189/task/74504/fd (opendir: Permission denied) 2024-03-21T10:28:38.546 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mds 71189 74587 ms_dispat ceph cwd unknown /proc/71189/task/74587/cwd (readlink: Permission denied) 2024-03-21T10:28:38.546 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mds 71189 74587 ms_dispat ceph rtd unknown /proc/71189/task/74587/root (readlink: Permission denied) 2024-03-21T10:28:38.546 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mds 71189 74587 ms_dispat ceph txt unknown /proc/71189/task/74587/exe (readlink: Permission denied) 2024-03-21T10:28:38.546 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mds 71189 74587 ms_dispat ceph NOFD /proc/71189/task/74587/fd (opendir: Permission denied) 2024-03-21T10:28:38.546 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mds 71189 74588 quiesce.a ceph cwd unknown /proc/71189/task/74588/cwd (readlink: Permission denied) 2024-03-21T10:28:38.546 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mds 71189 74588 quiesce.a ceph rtd unknown /proc/71189/task/74588/root (readlink: Permission denied) 2024-03-21T10:28:38.546 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mds 71189 74588 quiesce.a ceph txt unknown /proc/71189/task/74588/exe (readlink: Permission denied) 2024-03-21T10:28:38.546 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mds 71189 74588 quiesce.a ceph NOFD /proc/71189/task/74588/fd (opendir: Permission denied) 2024-03-21T10:28:38.546 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mds 71189 74589 quiesce_d ceph cwd unknown /proc/71189/task/74589/cwd (readlink: Permission denied) 2024-03-21T10:28:38.546 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mds 71189 74589 quiesce_d ceph rtd unknown /proc/71189/task/74589/root (readlink: Permission denied) 2024-03-21T10:28:38.546 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mds 71189 74589 quiesce_d ceph txt unknown /proc/71189/task/74589/exe (readlink: Permission denied) 2024-03-21T10:28:38.546 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-mds 71189 74589 quiesce_d ceph NOFD /proc/71189/task/74589/fd (opendir: Permission denied) 2024-03-21T10:28:38.546 INFO:tasks.workunit.client.0.smithi047.stdout:kworker/6 78209 root cwd unknown /proc/78209/cwd (readlink: Permission denied) 2024-03-21T10:28:38.546 INFO:tasks.workunit.client.0.smithi047.stdout:kworker/6 78209 root rtd unknown /proc/78209/root (readlink: Permission denied) 2024-03-21T10:28:38.547 INFO:tasks.workunit.client.0.smithi047.stdout:kworker/6 78209 root txt unknown /proc/78209/exe (readlink: Permission denied) 2024-03-21T10:28:38.547 INFO:tasks.workunit.client.0.smithi047.stdout:kworker/6 78209 root NOFD /proc/78209/fd (opendir: Permission denied) 2024-03-21T10:28:38.547 INFO:tasks.workunit.client.0.smithi047.stdout:kworker/2 79988 root cwd unknown /proc/79988/cwd (readlink: Permission denied) 2024-03-21T10:28:38.547 INFO:tasks.workunit.client.0.smithi047.stdout:kworker/2 79988 root rtd unknown /proc/79988/root (readlink: Permission denied) 2024-03-21T10:28:38.547 INFO:tasks.workunit.client.0.smithi047.stdout:kworker/2 79988 root txt unknown /proc/79988/exe (readlink: Permission denied) 2024-03-21T10:28:38.547 INFO:tasks.workunit.client.0.smithi047.stdout:kworker/2 79988 root NOFD /proc/79988/fd (opendir: Permission denied) 2024-03-21T10:28:38.547 INFO:tasks.workunit.client.0.smithi047.stdout:kworker/5 81405 root cwd unknown /proc/81405/cwd (readlink: Permission denied) 2024-03-21T10:28:38.547 INFO:tasks.workunit.client.0.smithi047.stdout:kworker/5 81405 root rtd unknown /proc/81405/root (readlink: Permission denied) 2024-03-21T10:28:38.547 INFO:tasks.workunit.client.0.smithi047.stdout:kworker/5 81405 root txt unknown /proc/81405/exe (readlink: Permission denied) 2024-03-21T10:28:38.547 INFO:tasks.workunit.client.0.smithi047.stdout:kworker/5 81405 root NOFD /proc/81405/fd (opendir: Permission denied) 2024-03-21T10:28:38.547 INFO:tasks.workunit.client.0.smithi047.stdout:kworker/3 83761 root cwd unknown /proc/83761/cwd (readlink: Permission denied) 2024-03-21T10:28:38.547 INFO:tasks.workunit.client.0.smithi047.stdout:kworker/3 83761 root rtd unknown /proc/83761/root (readlink: Permission denied) 2024-03-21T10:28:38.547 INFO:tasks.workunit.client.0.smithi047.stdout:kworker/3 83761 root txt unknown /proc/83761/exe (readlink: Permission denied) 2024-03-21T10:28:38.547 INFO:tasks.workunit.client.0.smithi047.stdout:kworker/3 83761 root NOFD /proc/83761/fd (opendir: Permission denied) 2024-03-21T10:28:38.547 INFO:tasks.workunit.client.0.smithi047.stdout:bash 83896 ubuntu cwd DIR 8,1 4096 1026 /home/ubuntu 2024-03-21T10:28:38.547 INFO:tasks.workunit.client.0.smithi047.stdout:bash 83896 ubuntu rtd DIR 8,1 4096 2 / 2024-03-21T10:28:38.548 INFO:tasks.workunit.client.0.smithi047.stdout:bash 83896 ubuntu txt REG 8,1 1388920 9667 /usr/bin/bash 2024-03-21T10:28:38.548 INFO:tasks.workunit.client.0.smithi047.stdout:bash 83896 ubuntu mem REG 8,1 2586930 17848 /usr/lib/locale/en_US.utf8/LC_COLLATE 2024-03-21T10:28:38.548 INFO:tasks.workunit.client.0.smithi047.stdout:bash 83896 ubuntu mem REG 8,1 2387016 9624 /usr/lib64/libc.so.6 2024-03-21T10:28:38.548 INFO:tasks.workunit.client.0.smithi047.stdout:bash 83896 ubuntu mem REG 8,1 346132 30861 /usr/lib/locale/C.utf8/LC_CTYPE 2024-03-21T10:28:38.548 INFO:tasks.workunit.client.0.smithi047.stdout:bash 83896 ubuntu mem REG 8,1 54 17851 /usr/lib/locale/en_US.utf8/LC_NUMERIC 2024-03-21T10:28:38.548 INFO:tasks.workunit.client.0.smithi047.stdout:bash 83896 ubuntu mem REG 8,1 3284 30787 /usr/lib/locale/en_US.utf8/LC_TIME 2024-03-21T10:28:38.548 INFO:tasks.workunit.client.0.smithi047.stdout:bash 83896 ubuntu mem REG 8,1 286 30784 /usr/lib/locale/en_US.utf8/LC_MONETARY 2024-03-21T10:28:38.548 INFO:tasks.workunit.client.0.smithi047.stdout:bash 83896 ubuntu mem REG 8,1 57 28351 /usr/lib/locale/en_US.utf8/LC_MESSAGES/SYS_LC_MESSAGES 2024-03-21T10:28:38.548 INFO:tasks.workunit.client.0.smithi047.stdout:bash 83896 ubuntu mem REG 8,1 34 30837 /usr/lib/locale/en_US.utf8/LC_PAPER 2024-03-21T10:28:38.548 INFO:tasks.workunit.client.0.smithi047.stdout:bash 83896 ubuntu mem REG 8,1 77 17850 /usr/lib/locale/en_US.utf8/LC_NAME 2024-03-21T10:28:38.548 INFO:tasks.workunit.client.0.smithi047.stdout:bash 83896 ubuntu mem REG 8,1 167 30779 /usr/lib/locale/en_US.utf8/LC_ADDRESS 2024-03-21T10:28:38.548 INFO:tasks.workunit.client.0.smithi047.stdout:bash 83896 ubuntu mem REG 8,1 59 30786 /usr/lib/locale/en_US.utf8/LC_TELEPHONE 2024-03-21T10:28:38.548 INFO:tasks.workunit.client.0.smithi047.stdout:bash 83896 ubuntu mem REG 8,1 195080 9664 /usr/lib64/libtinfo.so.6.2 2024-03-21T10:28:38.548 INFO:tasks.workunit.client.0.smithi047.stdout:bash 83896 ubuntu mem REG 8,1 23 30783 /usr/lib/locale/en_US.utf8/LC_MEASUREMENT 2024-03-21T10:28:38.549 INFO:tasks.workunit.client.0.smithi047.stdout:bash 83896 ubuntu mem REG 8,1 26988 29960 /usr/lib64/gconv/gconv-modules.cache 2024-03-21T10:28:38.549 INFO:tasks.workunit.client.0.smithi047.stdout:bash 83896 ubuntu mem REG 8,1 844808 9620 /usr/lib64/ld-linux-x86-64.so.2 2024-03-21T10:28:38.549 INFO:tasks.workunit.client.0.smithi047.stdout:bash 83896 ubuntu mem REG 8,1 369 30782 /usr/lib/locale/en_US.utf8/LC_IDENTIFICATION 2024-03-21T10:28:38.549 INFO:tasks.workunit.client.0.smithi047.stdout:bash 83896 ubuntu 0r FIFO 0,13 0t0 214450 pipe 2024-03-21T10:28:38.549 INFO:tasks.workunit.client.0.smithi047.stdout:bash 83896 ubuntu 1w FIFO 0,13 0t0 214451 pipe 2024-03-21T10:28:38.549 INFO:tasks.workunit.client.0.smithi047.stdout:bash 83896 ubuntu 2w FIFO 0,13 0t0 214452 pipe 2024-03-21T10:28:38.549 INFO:tasks.workunit.client.0.smithi047.stdout:sudo 83928 root cwd unknown /proc/83928/cwd (readlink: Permission denied) 2024-03-21T10:28:38.549 INFO:tasks.workunit.client.0.smithi047.stdout:sudo 83928 root rtd unknown /proc/83928/root (readlink: Permission denied) 2024-03-21T10:28:38.549 INFO:tasks.workunit.client.0.smithi047.stdout:sudo 83928 root txt unknown /proc/83928/exe (readlink: Permission denied) 2024-03-21T10:28:38.549 INFO:tasks.workunit.client.0.smithi047.stdout:sudo 83928 root NOFD /proc/83928/fd (opendir: Permission denied) 2024-03-21T10:28:38.549 INFO:tasks.workunit.client.0.smithi047.stdout:sudo 83931 root cwd unknown /proc/83931/cwd (readlink: Permission denied) 2024-03-21T10:28:38.549 INFO:tasks.workunit.client.0.smithi047.stdout:sudo 83931 root rtd unknown /proc/83931/root (readlink: Permission denied) 2024-03-21T10:28:38.549 INFO:tasks.workunit.client.0.smithi047.stdout:sudo 83931 root txt unknown /proc/83931/exe (readlink: Permission denied) 2024-03-21T10:28:38.549 INFO:tasks.workunit.client.0.smithi047.stdout:sudo 83931 root NOFD /proc/83931/fd (opendir: Permission denied) 2024-03-21T10:28:38.549 INFO:tasks.workunit.client.0.smithi047.stdout:daemon-he 83942 root cwd unknown /proc/83942/cwd (readlink: Permission denied) 2024-03-21T10:28:38.550 INFO:tasks.workunit.client.0.smithi047.stdout:daemon-he 83942 root rtd unknown /proc/83942/root (readlink: Permission denied) 2024-03-21T10:28:38.550 INFO:tasks.workunit.client.0.smithi047.stdout:daemon-he 83942 root txt unknown /proc/83942/exe (readlink: Permission denied) 2024-03-21T10:28:38.550 INFO:tasks.workunit.client.0.smithi047.stdout:daemon-he 83942 root NOFD /proc/83942/fd (opendir: Permission denied) 2024-03-21T10:28:38.550 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-fuse 83961 root cwd unknown /proc/83961/cwd (readlink: Permission denied) 2024-03-21T10:28:38.550 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-fuse 83961 root rtd unknown /proc/83961/root (readlink: Permission denied) 2024-03-21T10:28:38.550 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-fuse 83961 root txt unknown /proc/83961/exe (readlink: Permission denied) 2024-03-21T10:28:38.550 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-fuse 83961 root NOFD /proc/83961/fd (opendir: Permission denied) 2024-03-21T10:28:38.550 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-fuse 83961 83971 log root cwd unknown /proc/83961/task/83971/cwd (readlink: Permission denied) 2024-03-21T10:28:38.550 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-fuse 83961 83971 log root rtd unknown /proc/83961/task/83971/root (readlink: Permission denied) 2024-03-21T10:28:38.550 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-fuse 83961 83971 log root txt unknown /proc/83961/task/83971/exe (readlink: Permission denied) 2024-03-21T10:28:38.550 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-fuse 83961 83971 log root NOFD /proc/83961/task/83971/fd (opendir: Permission denied) 2024-03-21T10:28:38.550 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-fuse 83961 83973 msgr-work root cwd unknown /proc/83961/task/83973/cwd (readlink: Permission denied) 2024-03-21T10:28:38.550 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-fuse 83961 83973 msgr-work root rtd unknown /proc/83961/task/83973/root (readlink: Permission denied) 2024-03-21T10:28:38.550 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-fuse 83961 83973 msgr-work root txt unknown /proc/83961/task/83973/exe (readlink: Permission denied) 2024-03-21T10:28:38.551 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-fuse 83961 83973 msgr-work root NOFD /proc/83961/task/83973/fd (opendir: Permission denied) 2024-03-21T10:28:38.551 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-fuse 83961 83974 msgr-work root cwd unknown /proc/83961/task/83974/cwd (readlink: Permission denied) 2024-03-21T10:28:38.551 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-fuse 83961 83974 msgr-work root rtd unknown /proc/83961/task/83974/root (readlink: Permission denied) 2024-03-21T10:28:38.551 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-fuse 83961 83974 msgr-work root txt unknown /proc/83961/task/83974/exe (readlink: Permission denied) 2024-03-21T10:28:38.551 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-fuse 83961 83974 msgr-work root NOFD /proc/83961/task/83974/fd (opendir: Permission denied) 2024-03-21T10:28:38.551 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-fuse 83961 83975 msgr-work root cwd unknown /proc/83961/task/83975/cwd (readlink: Permission denied) 2024-03-21T10:28:38.551 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-fuse 83961 83975 msgr-work root rtd unknown /proc/83961/task/83975/root (readlink: Permission denied) 2024-03-21T10:28:38.551 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-fuse 83961 83975 msgr-work root txt unknown /proc/83961/task/83975/exe (readlink: Permission denied) 2024-03-21T10:28:38.551 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-fuse 83961 83975 msgr-work root NOFD /proc/83961/task/83975/fd (opendir: Permission denied) 2024-03-21T10:28:38.551 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-fuse 83961 83979 service root cwd unknown /proc/83961/task/83979/cwd (readlink: Permission denied) 2024-03-21T10:28:38.551 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-fuse 83961 83979 service root rtd unknown /proc/83961/task/83979/root (readlink: Permission denied) 2024-03-21T10:28:38.551 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-fuse 83961 83979 service root txt unknown /proc/83961/task/83979/exe (readlink: Permission denied) 2024-03-21T10:28:38.551 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-fuse 83961 83979 service root NOFD /proc/83961/task/83979/fd (opendir: Permission denied) 2024-03-21T10:28:38.551 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-fuse 83961 83980 admin_soc root cwd unknown /proc/83961/task/83980/cwd (readlink: Permission denied) 2024-03-21T10:28:38.551 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-fuse 83961 83980 admin_soc root rtd unknown /proc/83961/task/83980/root (readlink: Permission denied) 2024-03-21T10:28:38.552 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-fuse 83961 83980 admin_soc root txt unknown /proc/83961/task/83980/exe (readlink: Permission denied) 2024-03-21T10:28:38.552 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-fuse 83961 83980 admin_soc root NOFD /proc/83961/task/83980/fd (opendir: Permission denied) 2024-03-21T10:28:38.552 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-fuse 83961 83981 signal_ha root cwd unknown /proc/83961/task/83981/cwd (readlink: Permission denied) 2024-03-21T10:28:38.552 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-fuse 83961 83981 signal_ha root rtd unknown /proc/83961/task/83981/root (readlink: Permission denied) 2024-03-21T10:28:38.552 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-fuse 83961 83981 signal_ha root txt unknown /proc/83961/task/83981/exe (readlink: Permission denied) 2024-03-21T10:28:38.552 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-fuse 83961 83981 signal_ha root NOFD /proc/83961/task/83981/fd (opendir: Permission denied) 2024-03-21T10:28:38.552 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-fuse 83961 83982 io_contex root cwd unknown /proc/83961/task/83982/cwd (readlink: Permission denied) 2024-03-21T10:28:38.552 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-fuse 83961 83982 io_contex root rtd unknown /proc/83961/task/83982/root (readlink: Permission denied) 2024-03-21T10:28:38.552 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-fuse 83961 83982 io_contex root txt unknown /proc/83961/task/83982/exe (readlink: Permission denied) 2024-03-21T10:28:38.552 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-fuse 83961 83982 io_contex root NOFD /proc/83961/task/83982/fd (opendir: Permission denied) 2024-03-21T10:28:38.552 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-fuse 83961 83983 io_contex root cwd unknown /proc/83961/task/83983/cwd (readlink: Permission denied) 2024-03-21T10:28:38.552 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-fuse 83961 83983 io_contex root rtd unknown /proc/83961/task/83983/root (readlink: Permission denied) 2024-03-21T10:28:38.552 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-fuse 83961 83983 io_contex root txt unknown /proc/83961/task/83983/exe (readlink: Permission denied) 2024-03-21T10:28:38.553 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-fuse 83961 83983 io_contex root NOFD /proc/83961/task/83983/fd (opendir: Permission denied) 2024-03-21T10:28:38.553 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-fuse 83961 83984 ceph_time root cwd unknown /proc/83961/task/83984/cwd (readlink: Permission denied) 2024-03-21T10:28:38.553 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-fuse 83961 83984 ceph_time root rtd unknown /proc/83961/task/83984/root (readlink: Permission denied) 2024-03-21T10:28:38.553 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-fuse 83961 83984 ceph_time root txt unknown /proc/83961/task/83984/exe (readlink: Permission denied) 2024-03-21T10:28:38.553 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-fuse 83961 83984 ceph_time root NOFD /proc/83961/task/83984/fd (opendir: Permission denied) 2024-03-21T10:28:38.553 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-fuse 83961 83985 fn_anonym root cwd unknown /proc/83961/task/83985/cwd (readlink: Permission denied) 2024-03-21T10:28:38.553 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-fuse 83961 83985 fn_anonym root rtd unknown /proc/83961/task/83985/root (readlink: Permission denied) 2024-03-21T10:28:38.553 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-fuse 83961 83985 fn_anonym root txt unknown /proc/83961/task/83985/exe (readlink: Permission denied) 2024-03-21T10:28:38.553 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-fuse 83961 83985 fn_anonym root NOFD /proc/83961/task/83985/fd (opendir: Permission denied) 2024-03-21T10:28:38.553 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-fuse 83961 83986 fn_anonym root cwd unknown /proc/83961/task/83986/cwd (readlink: Permission denied) 2024-03-21T10:28:38.553 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-fuse 83961 83986 fn_anonym root rtd unknown /proc/83961/task/83986/root (readlink: Permission denied) 2024-03-21T10:28:38.553 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-fuse 83961 83986 fn_anonym root txt unknown /proc/83961/task/83986/exe (readlink: Permission denied) 2024-03-21T10:28:38.553 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-fuse 83961 83986 fn_anonym root NOFD /proc/83961/task/83986/fd (opendir: Permission denied) 2024-03-21T10:28:38.553 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-fuse 83961 83987 fn_anonym root cwd unknown /proc/83961/task/83987/cwd (readlink: Permission denied) 2024-03-21T10:28:38.553 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-fuse 83961 83987 fn_anonym root rtd unknown /proc/83961/task/83987/root (readlink: Permission denied) 2024-03-21T10:28:38.554 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-fuse 83961 83987 fn_anonym root txt unknown /proc/83961/task/83987/exe (readlink: Permission denied) 2024-03-21T10:28:38.554 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-fuse 83961 83987 fn_anonym root NOFD /proc/83961/task/83987/fd (opendir: Permission denied) 2024-03-21T10:28:38.554 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-fuse 83961 83988 fn_anonym root cwd unknown /proc/83961/task/83988/cwd (readlink: Permission denied) 2024-03-21T10:28:38.554 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-fuse 83961 83988 fn_anonym root rtd unknown /proc/83961/task/83988/root (readlink: Permission denied) 2024-03-21T10:28:38.554 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-fuse 83961 83988 fn_anonym root txt unknown /proc/83961/task/83988/exe (readlink: Permission denied) 2024-03-21T10:28:38.554 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-fuse 83961 83988 fn_anonym root NOFD /proc/83961/task/83988/fd (opendir: Permission denied) 2024-03-21T10:28:38.554 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-fuse 83961 83989 fn_anonym root cwd unknown /proc/83961/task/83989/cwd (readlink: Permission denied) 2024-03-21T10:28:38.554 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-fuse 83961 83989 fn_anonym root rtd unknown /proc/83961/task/83989/root (readlink: Permission denied) 2024-03-21T10:28:38.554 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-fuse 83961 83989 fn_anonym root txt unknown /proc/83961/task/83989/exe (readlink: Permission denied) 2024-03-21T10:28:38.554 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-fuse 83961 83989 fn_anonym root NOFD /proc/83961/task/83989/fd (opendir: Permission denied) 2024-03-21T10:28:38.554 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-fuse 83961 83990 safe_time root cwd unknown /proc/83961/task/83990/cwd (readlink: Permission denied) 2024-03-21T10:28:38.554 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-fuse 83961 83990 safe_time root rtd unknown /proc/83961/task/83990/root (readlink: Permission denied) 2024-03-21T10:28:38.554 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-fuse 83961 83990 safe_time root txt unknown /proc/83961/task/83990/exe (readlink: Permission denied) 2024-03-21T10:28:38.554 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-fuse 83961 83990 safe_time root NOFD /proc/83961/task/83990/fd (opendir: Permission denied) 2024-03-21T10:28:38.555 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-fuse 83961 83991 fn_anonym root cwd unknown /proc/83961/task/83991/cwd (readlink: Permission denied) 2024-03-21T10:28:38.555 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-fuse 83961 83991 fn_anonym root rtd unknown /proc/83961/task/83991/root (readlink: Permission denied) 2024-03-21T10:28:38.555 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-fuse 83961 83991 fn_anonym root txt unknown /proc/83961/task/83991/exe (readlink: Permission denied) 2024-03-21T10:28:38.555 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-fuse 83961 83991 fn_anonym root NOFD /proc/83961/task/83991/fd (opendir: Permission denied) 2024-03-21T10:28:38.555 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-fuse 83961 83992 flusher root cwd unknown /proc/83961/task/83992/cwd (readlink: Permission denied) 2024-03-21T10:28:38.555 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-fuse 83961 83992 flusher root rtd unknown /proc/83961/task/83992/root (readlink: Permission denied) 2024-03-21T10:28:38.555 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-fuse 83961 83992 flusher root txt unknown /proc/83961/task/83992/exe (readlink: Permission denied) 2024-03-21T10:28:38.555 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-fuse 83961 83992 flusher root NOFD /proc/83961/task/83992/fd (opendir: Permission denied) 2024-03-21T10:28:38.555 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-fuse 83961 83993 ms_dispat root cwd unknown /proc/83961/task/83993/cwd (readlink: Permission denied) 2024-03-21T10:28:38.555 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-fuse 83961 83993 ms_dispat root rtd unknown /proc/83961/task/83993/root (readlink: Permission denied) 2024-03-21T10:28:38.555 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-fuse 83961 83993 ms_dispat root txt unknown /proc/83961/task/83993/exe (readlink: Permission denied) 2024-03-21T10:28:38.555 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-fuse 83961 83993 ms_dispat root NOFD /proc/83961/task/83993/fd (opendir: Permission denied) 2024-03-21T10:28:38.555 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-fuse 83961 83994 ms_local root cwd unknown /proc/83961/task/83994/cwd (readlink: Permission denied) 2024-03-21T10:28:38.555 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-fuse 83961 83994 ms_local root rtd unknown /proc/83961/task/83994/root (readlink: Permission denied) 2024-03-21T10:28:38.555 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-fuse 83961 83994 ms_local root txt unknown /proc/83961/task/83994/exe (readlink: Permission denied) 2024-03-21T10:28:38.556 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-fuse 83961 83994 ms_local root NOFD /proc/83961/task/83994/fd (opendir: Permission denied) 2024-03-21T10:28:38.556 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-fuse 83961 83995 safe_time root cwd unknown /proc/83961/task/83995/cwd (readlink: Permission denied) 2024-03-21T10:28:38.556 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-fuse 83961 83995 safe_time root rtd unknown /proc/83961/task/83995/root (readlink: Permission denied) 2024-03-21T10:28:38.556 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-fuse 83961 83995 safe_time root txt unknown /proc/83961/task/83995/exe (readlink: Permission denied) 2024-03-21T10:28:38.556 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-fuse 83961 83995 safe_time root NOFD /proc/83961/task/83995/fd (opendir: Permission denied) 2024-03-21T10:28:38.556 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-fuse 83961 83996 ceph-fuse root cwd unknown /proc/83961/task/83996/cwd (readlink: Permission denied) 2024-03-21T10:28:38.556 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-fuse 83961 83996 ceph-fuse root rtd unknown /proc/83961/task/83996/root (readlink: Permission denied) 2024-03-21T10:28:38.556 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-fuse 83961 83996 ceph-fuse root txt unknown /proc/83961/task/83996/exe (readlink: Permission denied) 2024-03-21T10:28:38.556 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-fuse 83961 83996 ceph-fuse root NOFD /proc/83961/task/83996/fd (opendir: Permission denied) 2024-03-21T10:28:38.556 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-fuse 83961 83998 ceph-fuse root cwd unknown /proc/83961/task/83998/cwd (readlink: Permission denied) 2024-03-21T10:28:38.556 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-fuse 83961 83998 ceph-fuse root rtd unknown /proc/83961/task/83998/root (readlink: Permission denied) 2024-03-21T10:28:38.556 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-fuse 83961 83998 ceph-fuse root txt unknown /proc/83961/task/83998/exe (readlink: Permission denied) 2024-03-21T10:28:38.556 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-fuse 83961 83998 ceph-fuse root NOFD /proc/83961/task/83998/fd (opendir: Permission denied) 2024-03-21T10:28:38.556 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-fuse 83961 84000 ceph-fuse root cwd unknown /proc/83961/task/84000/cwd (readlink: Permission denied) 2024-03-21T10:28:38.557 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-fuse 83961 84000 ceph-fuse root rtd unknown /proc/83961/task/84000/root (readlink: Permission denied) 2024-03-21T10:28:38.557 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-fuse 83961 84000 ceph-fuse root txt unknown /proc/83961/task/84000/exe (readlink: Permission denied) 2024-03-21T10:28:38.557 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-fuse 83961 84000 ceph-fuse root NOFD /proc/83961/task/84000/fd (opendir: Permission denied) 2024-03-21T10:28:38.557 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-fuse 83961 89462 ceph-fuse root cwd unknown /proc/83961/task/89462/cwd (readlink: Permission denied) 2024-03-21T10:28:38.557 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-fuse 83961 89462 ceph-fuse root rtd unknown /proc/83961/task/89462/root (readlink: Permission denied) 2024-03-21T10:28:38.557 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-fuse 83961 89462 ceph-fuse root txt unknown /proc/83961/task/89462/exe (readlink: Permission denied) 2024-03-21T10:28:38.557 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-fuse 83961 89462 ceph-fuse root NOFD /proc/83961/task/89462/fd (opendir: Permission denied) 2024-03-21T10:28:38.557 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-fuse 83961 90094 ceph-fuse root cwd unknown /proc/83961/task/90094/cwd (readlink: Permission denied) 2024-03-21T10:28:38.557 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-fuse 83961 90094 ceph-fuse root rtd unknown /proc/83961/task/90094/root (readlink: Permission denied) 2024-03-21T10:28:38.557 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-fuse 83961 90094 ceph-fuse root txt unknown /proc/83961/task/90094/exe (readlink: Permission denied) 2024-03-21T10:28:38.557 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-fuse 83961 90094 ceph-fuse root NOFD /proc/83961/task/90094/fd (opendir: Permission denied) 2024-03-21T10:28:38.557 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-fuse 83961 90350 ceph-fuse root cwd unknown /proc/83961/task/90350/cwd (readlink: Permission denied) 2024-03-21T10:28:38.557 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-fuse 83961 90350 ceph-fuse root rtd unknown /proc/83961/task/90350/root (readlink: Permission denied) 2024-03-21T10:28:38.557 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-fuse 83961 90350 ceph-fuse root txt unknown /proc/83961/task/90350/exe (readlink: Permission denied) 2024-03-21T10:28:38.558 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-fuse 83961 90350 ceph-fuse root NOFD /proc/83961/task/90350/fd (opendir: Permission denied) 2024-03-21T10:28:38.558 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-fuse 83961 90398 ceph-fuse root cwd unknown /proc/83961/task/90398/cwd (readlink: Permission denied) 2024-03-21T10:28:38.558 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-fuse 83961 90398 ceph-fuse root rtd unknown /proc/83961/task/90398/root (readlink: Permission denied) 2024-03-21T10:28:38.558 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-fuse 83961 90398 ceph-fuse root txt unknown /proc/83961/task/90398/exe (readlink: Permission denied) 2024-03-21T10:28:38.558 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-fuse 83961 90398 ceph-fuse root NOFD /proc/83961/task/90398/fd (opendir: Permission denied) 2024-03-21T10:28:38.558 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-fuse 83961 91146 ceph-fuse root cwd unknown /proc/83961/task/91146/cwd (readlink: Permission denied) 2024-03-21T10:28:38.558 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-fuse 83961 91146 ceph-fuse root rtd unknown /proc/83961/task/91146/root (readlink: Permission denied) 2024-03-21T10:28:38.558 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-fuse 83961 91146 ceph-fuse root txt unknown /proc/83961/task/91146/exe (readlink: Permission denied) 2024-03-21T10:28:38.558 INFO:tasks.workunit.client.0.smithi047.stdout:ceph-fuse 83961 91146 ceph-fuse root NOFD /proc/83961/task/91146/fd (opendir: Permission denied) 2024-03-21T10:28:38.558 INFO:tasks.workunit.client.0.smithi047.stdout:kworker/4 84595 root cwd unknown /proc/84595/cwd (readlink: Permission denied) 2024-03-21T10:28:38.558 INFO:tasks.workunit.client.0.smithi047.stdout:kworker/4 84595 root rtd unknown /proc/84595/root (readlink: Permission denied) 2024-03-21T10:28:38.558 INFO:tasks.workunit.client.0.smithi047.stdout:kworker/4 84595 root txt unknown /proc/84595/exe (readlink: Permission denied) 2024-03-21T10:28:38.558 INFO:tasks.workunit.client.0.smithi047.stdout:kworker/4 84595 root NOFD /proc/84595/fd (opendir: Permission denied) 2024-03-21T10:28:38.558 INFO:tasks.workunit.client.0.smithi047.stdout:sshd 85495 root cwd unknown /proc/85495/cwd (readlink: Permission denied) 2024-03-21T10:28:38.558 INFO:tasks.workunit.client.0.smithi047.stdout:sshd 85495 root rtd unknown /proc/85495/root (readlink: Permission denied) 2024-03-21T10:28:38.559 INFO:tasks.workunit.client.0.smithi047.stdout:sshd 85495 root txt unknown /proc/85495/exe (readlink: Permission denied) 2024-03-21T10:28:38.559 INFO:tasks.workunit.client.0.smithi047.stdout:sshd 85495 root NOFD /proc/85495/fd (opendir: Permission denied) 2024-03-21T10:28:38.559 INFO:tasks.workunit.client.0.smithi047.stdout:sshd 85518 root cwd unknown /proc/85518/cwd (readlink: Permission denied) 2024-03-21T10:28:38.559 INFO:tasks.workunit.client.0.smithi047.stdout:sshd 85518 root rtd unknown /proc/85518/root (readlink: Permission denied) 2024-03-21T10:28:38.559 INFO:tasks.workunit.client.0.smithi047.stdout:sshd 85518 root txt unknown /proc/85518/exe (readlink: Permission denied) 2024-03-21T10:28:38.559 INFO:tasks.workunit.client.0.smithi047.stdout:sshd 85518 root NOFD /proc/85518/fd (opendir: Permission denied) 2024-03-21T10:28:38.559 INFO:tasks.workunit.client.0.smithi047.stdout:sudo 86007 root cwd unknown /proc/86007/cwd (readlink: Permission denied) 2024-03-21T10:28:38.559 INFO:tasks.workunit.client.0.smithi047.stdout:sudo 86007 root rtd unknown /proc/86007/root (readlink: Permission denied) 2024-03-21T10:28:38.559 INFO:tasks.workunit.client.0.smithi047.stdout:sudo 86007 root txt unknown /proc/86007/exe (readlink: Permission denied) 2024-03-21T10:28:38.559 INFO:tasks.workunit.client.0.smithi047.stdout:sudo 86007 root NOFD /proc/86007/fd (opendir: Permission denied) 2024-03-21T10:28:38.559 INFO:tasks.workunit.client.0.smithi047.stdout:daemon-he 86051 root cwd unknown /proc/86051/cwd (readlink: Permission denied) 2024-03-21T10:28:38.559 INFO:tasks.workunit.client.0.smithi047.stdout:daemon-he 86051 root rtd unknown /proc/86051/root (readlink: Permission denied) 2024-03-21T10:28:38.559 INFO:tasks.workunit.client.0.smithi047.stdout:daemon-he 86051 root txt unknown /proc/86051/exe (readlink: Permission denied) 2024-03-21T10:28:38.560 INFO:tasks.workunit.client.0.smithi047.stdout:daemon-he 86051 root NOFD /proc/86051/fd (opendir: Permission denied) 2024-03-21T10:28:38.560 INFO:tasks.workunit.client.0.smithi047.stdout:bash 86073 root cwd unknown /proc/86073/cwd (readlink: Permission denied) 2024-03-21T10:28:38.560 INFO:tasks.workunit.client.0.smithi047.stdout:bash 86073 root rtd unknown /proc/86073/root (readlink: Permission denied) 2024-03-21T10:28:38.560 INFO:tasks.workunit.client.0.smithi047.stdout:bash 86073 root txt unknown /proc/86073/exe (readlink: Permission denied) 2024-03-21T10:28:38.560 INFO:tasks.workunit.client.0.smithi047.stdout:bash 86073 root NOFD /proc/86073/fd (opendir: Permission denied) 2024-03-21T10:28:38.560 INFO:tasks.workunit.client.0.smithi047.stdout:kworker/u 87229 root cwd unknown /proc/87229/cwd (readlink: Permission denied) 2024-03-21T10:28:38.560 INFO:tasks.workunit.client.0.smithi047.stdout:kworker/u 87229 root rtd unknown /proc/87229/root (readlink: Permission denied) 2024-03-21T10:28:38.560 INFO:tasks.workunit.client.0.smithi047.stdout:kworker/u 87229 root txt unknown /proc/87229/exe (readlink: Permission denied) 2024-03-21T10:28:38.560 INFO:tasks.workunit.client.0.smithi047.stdout:kworker/u 87229 root NOFD /proc/87229/fd (opendir: Permission denied) 2024-03-21T10:28:38.560 INFO:tasks.workunit.client.0.smithi047.stdout:timeout 89442 ubuntu cwd DIR 0,118 194248488 1099511627782 /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-03-21T10:28:38.560 INFO:tasks.workunit.client.0.smithi047.stdout:timeout 89442 ubuntu rtd DIR 8,1 4096 2 / 2024-03-21T10:28:38.560 INFO:tasks.workunit.client.0.smithi047.stdout:timeout 89442 ubuntu txt REG 8,1 40944 10098 /usr/bin/timeout 2024-03-21T10:28:38.560 INFO:tasks.workunit.client.0.smithi047.stdout:timeout 89442 ubuntu mem REG 8,1 2586930 17848 /usr/lib/locale/en_US.utf8/LC_COLLATE 2024-03-21T10:28:38.560 INFO:tasks.workunit.client.0.smithi047.stdout:timeout 89442 ubuntu mem REG 8,1 2387016 9624 /usr/lib64/libc.so.6 2024-03-21T10:28:38.560 INFO:tasks.workunit.client.0.smithi047.stdout:timeout 89442 ubuntu mem REG 8,1 346132 30861 /usr/lib/locale/C.utf8/LC_CTYPE 2024-03-21T10:28:38.561 INFO:tasks.workunit.client.0.smithi047.stdout:timeout 89442 ubuntu mem REG 8,1 54 17851 /usr/lib/locale/en_US.utf8/LC_NUMERIC 2024-03-21T10:28:38.561 INFO:tasks.workunit.client.0.smithi047.stdout:timeout 89442 ubuntu mem REG 8,1 3284 30787 /usr/lib/locale/en_US.utf8/LC_TIME 2024-03-21T10:28:38.561 INFO:tasks.workunit.client.0.smithi047.stdout:timeout 89442 ubuntu mem REG 8,1 286 30784 /usr/lib/locale/en_US.utf8/LC_MONETARY 2024-03-21T10:28:38.561 INFO:tasks.workunit.client.0.smithi047.stdout:timeout 89442 ubuntu mem REG 8,1 57 28351 /usr/lib/locale/en_US.utf8/LC_MESSAGES/SYS_LC_MESSAGES 2024-03-21T10:28:38.561 INFO:tasks.workunit.client.0.smithi047.stdout:timeout 89442 ubuntu mem REG 8,1 34 30837 /usr/lib/locale/en_US.utf8/LC_PAPER 2024-03-21T10:28:38.561 INFO:tasks.workunit.client.0.smithi047.stdout:timeout 89442 ubuntu mem REG 8,1 77 17850 /usr/lib/locale/en_US.utf8/LC_NAME 2024-03-21T10:28:38.561 INFO:tasks.workunit.client.0.smithi047.stdout:timeout 89442 ubuntu mem REG 8,1 167 30779 /usr/lib/locale/en_US.utf8/LC_ADDRESS 2024-03-21T10:28:38.561 INFO:tasks.workunit.client.0.smithi047.stdout:timeout 89442 ubuntu mem REG 8,1 59 30786 /usr/lib/locale/en_US.utf8/LC_TELEPHONE 2024-03-21T10:28:38.561 INFO:tasks.workunit.client.0.smithi047.stdout:timeout 89442 ubuntu mem REG 8,1 23 30783 /usr/lib/locale/en_US.utf8/LC_MEASUREMENT 2024-03-21T10:28:38.561 INFO:tasks.workunit.client.0.smithi047.stdout:timeout 89442 ubuntu mem REG 8,1 26988 29960 /usr/lib64/gconv/gconv-modules.cache 2024-03-21T10:28:38.561 INFO:tasks.workunit.client.0.smithi047.stdout:timeout 89442 ubuntu mem REG 8,1 844808 9620 /usr/lib64/ld-linux-x86-64.so.2 2024-03-21T10:28:38.561 INFO:tasks.workunit.client.0.smithi047.stdout:timeout 89442 ubuntu mem REG 8,1 369 30782 /usr/lib/locale/en_US.utf8/LC_IDENTIFICATION 2024-03-21T10:28:38.561 INFO:tasks.workunit.client.0.smithi047.stdout:timeout 89442 ubuntu 0r FIFO 0,13 0t0 229501 pipe 2024-03-21T10:28:38.562 INFO:tasks.workunit.client.0.smithi047.stdout:timeout 89442 ubuntu 1w FIFO 0,13 0t0 229502 pipe 2024-03-21T10:28:38.562 INFO:tasks.workunit.client.0.smithi047.stdout:timeout 89442 ubuntu 2w FIFO 0,13 0t0 229503 pipe 2024-03-21T10:28:38.562 INFO:tasks.workunit.client.0.smithi047.stdout:fsync-tes 89460 ubuntu cwd DIR 0,118 194248488 1099511627782 /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-03-21T10:28:38.562 INFO:tasks.workunit.client.0.smithi047.stdout:fsync-tes 89460 ubuntu rtd DIR 8,1 4096 2 / 2024-03-21T10:28:38.562 INFO:tasks.workunit.client.0.smithi047.stdout:fsync-tes 89460 ubuntu txt REG 8,1 1388920 9667 /usr/bin/bash 2024-03-21T10:28:38.562 INFO:tasks.workunit.client.0.smithi047.stdout:fsync-tes 89460 ubuntu mem REG 8,1 2586930 17848 /usr/lib/locale/en_US.utf8/LC_COLLATE 2024-03-21T10:28:38.562 INFO:tasks.workunit.client.0.smithi047.stdout:fsync-tes 89460 ubuntu mem REG 8,1 2387016 9624 /usr/lib64/libc.so.6 2024-03-21T10:28:38.562 INFO:tasks.workunit.client.0.smithi047.stdout:fsync-tes 89460 ubuntu mem REG 8,1 346132 30861 /usr/lib/locale/C.utf8/LC_CTYPE 2024-03-21T10:28:38.562 INFO:tasks.workunit.client.0.smithi047.stdout:fsync-tes 89460 ubuntu mem REG 8,1 54 17851 /usr/lib/locale/en_US.utf8/LC_NUMERIC 2024-03-21T10:28:38.562 INFO:tasks.workunit.client.0.smithi047.stdout:fsync-tes 89460 ubuntu mem REG 8,1 3284 30787 /usr/lib/locale/en_US.utf8/LC_TIME 2024-03-21T10:28:38.562 INFO:tasks.workunit.client.0.smithi047.stdout:fsync-tes 89460 ubuntu mem REG 8,1 286 30784 /usr/lib/locale/en_US.utf8/LC_MONETARY 2024-03-21T10:28:38.562 INFO:tasks.workunit.client.0.smithi047.stdout:fsync-tes 89460 ubuntu mem REG 8,1 57 28351 /usr/lib/locale/en_US.utf8/LC_MESSAGES/SYS_LC_MESSAGES 2024-03-21T10:28:38.562 INFO:tasks.workunit.client.0.smithi047.stdout:fsync-tes 89460 ubuntu mem REG 8,1 34 30837 /usr/lib/locale/en_US.utf8/LC_PAPER 2024-03-21T10:28:38.562 INFO:tasks.workunit.client.0.smithi047.stdout:fsync-tes 89460 ubuntu mem REG 8,1 77 17850 /usr/lib/locale/en_US.utf8/LC_NAME 2024-03-21T10:28:38.563 INFO:tasks.workunit.client.0.smithi047.stdout:fsync-tes 89460 ubuntu mem REG 8,1 167 30779 /usr/lib/locale/en_US.utf8/LC_ADDRESS 2024-03-21T10:28:38.563 INFO:tasks.workunit.client.0.smithi047.stdout:fsync-tes 89460 ubuntu mem REG 8,1 59 30786 /usr/lib/locale/en_US.utf8/LC_TELEPHONE 2024-03-21T10:28:38.563 INFO:tasks.workunit.client.0.smithi047.stdout:fsync-tes 89460 ubuntu mem REG 8,1 195080 9664 /usr/lib64/libtinfo.so.6.2 2024-03-21T10:28:38.563 INFO:tasks.workunit.client.0.smithi047.stdout:fsync-tes 89460 ubuntu mem REG 8,1 23 30783 /usr/lib/locale/en_US.utf8/LC_MEASUREMENT 2024-03-21T10:28:38.563 INFO:tasks.workunit.client.0.smithi047.stdout:fsync-tes 89460 ubuntu mem REG 8,1 26988 29960 /usr/lib64/gconv/gconv-modules.cache 2024-03-21T10:28:38.563 INFO:tasks.workunit.client.0.smithi047.stdout:fsync-tes 89460 ubuntu mem REG 8,1 844808 9620 /usr/lib64/ld-linux-x86-64.so.2 2024-03-21T10:28:38.563 INFO:tasks.workunit.client.0.smithi047.stdout:fsync-tes 89460 ubuntu mem REG 8,1 369 30782 /usr/lib/locale/en_US.utf8/LC_IDENTIFICATION 2024-03-21T10:28:38.563 INFO:tasks.workunit.client.0.smithi047.stdout:fsync-tes 89460 ubuntu 0r FIFO 0,13 0t0 229501 pipe 2024-03-21T10:28:38.563 INFO:tasks.workunit.client.0.smithi047.stdout:fsync-tes 89460 ubuntu 1w FIFO 0,13 0t0 229502 pipe 2024-03-21T10:28:38.563 INFO:tasks.workunit.client.0.smithi047.stdout:fsync-tes 89460 ubuntu 2w FIFO 0,13 0t0 229503 pipe 2024-03-21T10:28:38.563 INFO:tasks.workunit.client.0.smithi047.stdout:fsync-tes 89460 ubuntu 255r REG 8,1 207 661773 /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/fsync-tester.sh 2024-03-21T10:28:38.563 INFO:tasks.workunit.client.0.smithi047.stdout:kworker/0 89668 root cwd unknown /proc/89668/cwd (readlink: Permission denied) 2024-03-21T10:28:38.563 INFO:tasks.workunit.client.0.smithi047.stdout:kworker/0 89668 root rtd unknown /proc/89668/root (readlink: Permission denied) 2024-03-21T10:28:38.563 INFO:tasks.workunit.client.0.smithi047.stdout:kworker/0 89668 root txt unknown /proc/89668/exe (readlink: Permission denied) 2024-03-21T10:28:38.564 INFO:tasks.workunit.client.0.smithi047.stdout:kworker/0 89668 root NOFD /proc/89668/fd (opendir: Permission denied) 2024-03-21T10:28:38.564 INFO:tasks.workunit.client.0.smithi047.stdout:kworker/u 90399 root cwd unknown /proc/90399/cwd (readlink: Permission denied) 2024-03-21T10:28:38.564 INFO:tasks.workunit.client.0.smithi047.stdout:kworker/u 90399 root rtd unknown /proc/90399/root (readlink: Permission denied) 2024-03-21T10:28:38.564 INFO:tasks.workunit.client.0.smithi047.stdout:kworker/u 90399 root txt unknown /proc/90399/exe (readlink: Permission denied) 2024-03-21T10:28:38.564 INFO:tasks.workunit.client.0.smithi047.stdout:kworker/u 90399 root NOFD /proc/90399/fd (opendir: Permission denied) 2024-03-21T10:28:38.564 INFO:tasks.workunit.client.0.smithi047.stdout:sleep 91102 root cwd unknown /proc/91102/cwd (readlink: Permission denied) 2024-03-21T10:28:38.564 INFO:tasks.workunit.client.0.smithi047.stdout:sleep 91102 root rtd unknown /proc/91102/root (readlink: Permission denied) 2024-03-21T10:28:38.564 INFO:tasks.workunit.client.0.smithi047.stdout:sleep 91102 root txt unknown /proc/91102/exe (readlink: Permission denied) 2024-03-21T10:28:38.564 INFO:tasks.workunit.client.0.smithi047.stdout:sleep 91102 root NOFD /proc/91102/fd (opendir: Permission denied) 2024-03-21T10:28:38.564 INFO:tasks.workunit.client.0.smithi047.stdout:lsof 91793 ubuntu cwd DIR 0,118 194248488 1099511627782 /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-03-21T10:28:38.564 INFO:tasks.workunit.client.0.smithi047.stdout:lsof 91793 ubuntu rtd DIR 8,1 4096 2 / 2024-03-21T10:28:38.564 INFO:tasks.workunit.client.0.smithi047.stdout:lsof 91793 ubuntu txt REG 8,1 179232 12343 /usr/bin/lsof 2024-03-21T10:28:38.564 INFO:tasks.workunit.client.0.smithi047.stdout:lsof 91793 ubuntu mem REG 8,1 4487176 2249 /usr/lib64/libcrypto.so.3.0.7 2024-03-21T10:28:38.564 INFO:tasks.workunit.client.0.smithi047.stdout:lsof 91793 ubuntu mem REG 8,1 346132 30861 /usr/lib/locale/C.utf8/LC_CTYPE 2024-03-21T10:28:38.565 INFO:tasks.workunit.client.0.smithi047.stdout:lsof 91793 ubuntu mem REG 8,1 102552 9686 /usr/lib64/libz.so.1.2.11 2024-03-21T10:28:38.565 INFO:tasks.workunit.client.0.smithi047.stdout:lsof 91793 ubuntu mem REG 8,1 70512 9633 /usr/lib64/libresolv.so.2 2024-03-21T10:28:38.565 INFO:tasks.workunit.client.0.smithi047.stdout:lsof 91793 ubuntu mem REG 8,1 66192 11405 /usr/lib64/libkrb5support.so.0.1 2024-03-21T10:28:38.565 INFO:tasks.workunit.client.0.smithi047.stdout:lsof 91793 ubuntu mem REG 8,1 98992 11397 /usr/lib64/libk5crypto.so.3.1 2024-03-21T10:28:38.565 INFO:tasks.workunit.client.0.smithi047.stdout:lsof 91793 ubuntu mem REG 8,1 906256 11403 /usr/lib64/libkrb5.so.3.3 2024-03-21T10:28:38.565 INFO:tasks.workunit.client.0.smithi047.stdout:lsof 91793 ubuntu mem REG 8,1 2387016 9624 /usr/lib64/libc.so.6 2024-03-21T10:28:38.565 INFO:tasks.workunit.client.0.smithi047.stdout:lsof 91793 ubuntu mem REG 8,1 23992 9760 /usr/lib64/libkeyutils.so.1.10 2024-03-21T10:28:38.565 INFO:tasks.workunit.client.0.smithi047.stdout:lsof 91793 ubuntu mem REG 8,1 24400 9711 /usr/lib64/libcom_err.so.2.1 2024-03-21T10:28:38.565 INFO:tasks.workunit.client.0.smithi047.stdout:lsof 91793 ubuntu mem REG 8,1 358448 11393 /usr/lib64/libgssapi_krb5.so.2.2 2024-03-21T10:28:38.565 INFO:tasks.workunit.client.0.smithi047.stdout:lsof 91793 ubuntu mem REG 8,1 637872 9740 /usr/lib64/libpcre2-8.so.0.11.0 2024-03-21T10:28:38.565 INFO:tasks.workunit.client.0.smithi047.stdout:lsof 91793 ubuntu mem REG 8,1 196112 11431 /usr/lib64/libtirpc.so.3.0.0 2024-03-21T10:28:38.565 INFO:tasks.workunit.client.0.smithi047.stdout:lsof 91793 ubuntu mem REG 8,1 175552 9824 /usr/lib64/libselinux.so.1 2024-03-21T10:28:38.565 INFO:tasks.workunit.client.0.smithi047.stdout:lsof 91793 ubuntu mem REG 8,1 26988 29960 /usr/lib64/gconv/gconv-modules.cache 2024-03-21T10:28:38.565 INFO:tasks.workunit.client.0.smithi047.stdout:lsof 91793 ubuntu mem REG 8,1 844808 9620 /usr/lib64/ld-linux-x86-64.so.2 2024-03-21T10:28:38.565 INFO:tasks.workunit.client.0.smithi047.stdout:lsof 91793 ubuntu 0r FIFO 0,13 0t0 229501 pipe 2024-03-21T10:28:38.566 INFO:tasks.workunit.client.0.smithi047.stdout:lsof 91793 ubuntu 1w FIFO 0,13 0t0 229502 pipe 2024-03-21T10:28:38.566 INFO:tasks.workunit.client.0.smithi047.stdout:lsof 91793 ubuntu 2w FIFO 0,13 0t0 229503 pipe 2024-03-21T10:28:38.566 INFO:tasks.workunit.client.0.smithi047.stdout:lsof 91793 ubuntu 3r DIR 0,20 0 1 /proc 2024-03-21T10:28:38.566 INFO:tasks.workunit.client.0.smithi047.stdout:lsof 91793 ubuntu 4r DIR 0,20 7 230098 /proc/91793/fd 2024-03-21T10:28:38.566 INFO:tasks.workunit.client.0.smithi047.stdout:lsof 91793 ubuntu 5w FIFO 0,13 0t0 230103 pipe 2024-03-21T10:28:38.566 INFO:tasks.workunit.client.0.smithi047.stdout:lsof 91793 ubuntu 6r FIFO 0,13 0t0 230104 pipe 2024-03-21T10:28:38.566 INFO:tasks.workunit.client.0.smithi047.stdout:lsof 91794 ubuntu cwd DIR 0,118 194248488 1099511627782 /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-03-21T10:28:38.566 INFO:tasks.workunit.client.0.smithi047.stdout:lsof 91794 ubuntu rtd DIR 8,1 4096 2 / 2024-03-21T10:28:38.566 INFO:tasks.workunit.client.0.smithi047.stdout:lsof 91794 ubuntu txt REG 8,1 179232 12343 /usr/bin/lsof 2024-03-21T10:28:38.566 INFO:tasks.workunit.client.0.smithi047.stdout:lsof 91794 ubuntu mem REG 8,1 4487176 2249 /usr/lib64/libcrypto.so.3.0.7 2024-03-21T10:28:38.566 INFO:tasks.workunit.client.0.smithi047.stdout:lsof 91794 ubuntu mem REG 8,1 346132 30861 /usr/lib/locale/C.utf8/LC_CTYPE 2024-03-21T10:28:38.566 INFO:tasks.workunit.client.0.smithi047.stdout:lsof 91794 ubuntu mem REG 8,1 102552 9686 /usr/lib64/libz.so.1.2.11 2024-03-21T10:28:38.566 INFO:tasks.workunit.client.0.smithi047.stdout:lsof 91794 ubuntu mem REG 8,1 70512 9633 /usr/lib64/libresolv.so.2 2024-03-21T10:28:38.566 INFO:tasks.workunit.client.0.smithi047.stdout:lsof 91794 ubuntu mem REG 8,1 66192 11405 /usr/lib64/libkrb5support.so.0.1 2024-03-21T10:28:38.566 INFO:tasks.workunit.client.0.smithi047.stdout:lsof 91794 ubuntu mem REG 8,1 98992 11397 /usr/lib64/libk5crypto.so.3.1 2024-03-21T10:28:38.567 INFO:tasks.workunit.client.0.smithi047.stdout:lsof 91794 ubuntu mem REG 8,1 906256 11403 /usr/lib64/libkrb5.so.3.3 2024-03-21T10:28:38.567 INFO:tasks.workunit.client.0.smithi047.stdout:lsof 91794 ubuntu mem REG 8,1 2387016 9624 /usr/lib64/libc.so.6 2024-03-21T10:28:38.567 INFO:tasks.workunit.client.0.smithi047.stdout:lsof 91794 ubuntu mem REG 8,1 23992 9760 /usr/lib64/libkeyutils.so.1.10 2024-03-21T10:28:38.567 INFO:tasks.workunit.client.0.smithi047.stdout:lsof 91794 ubuntu mem REG 8,1 24400 9711 /usr/lib64/libcom_err.so.2.1 2024-03-21T10:28:38.567 INFO:tasks.workunit.client.0.smithi047.stdout:lsof 91794 ubuntu mem REG 8,1 358448 11393 /usr/lib64/libgssapi_krb5.so.2.2 2024-03-21T10:28:38.567 INFO:tasks.workunit.client.0.smithi047.stdout:lsof 91794 ubuntu mem REG 8,1 637872 9740 /usr/lib64/libpcre2-8.so.0.11.0 2024-03-21T10:28:38.567 INFO:tasks.workunit.client.0.smithi047.stdout:lsof 91794 ubuntu mem REG 8,1 196112 11431 /usr/lib64/libtirpc.so.3.0.0 2024-03-21T10:28:38.567 INFO:tasks.workunit.client.0.smithi047.stdout:lsof 91794 ubuntu mem REG 8,1 175552 9824 /usr/lib64/libselinux.so.1 2024-03-21T10:28:38.567 INFO:tasks.workunit.client.0.smithi047.stdout:lsof 91794 ubuntu mem REG 8,1 26988 29960 /usr/lib64/gconv/gconv-modules.cache 2024-03-21T10:28:38.567 INFO:tasks.workunit.client.0.smithi047.stdout:lsof 91794 ubuntu mem REG 8,1 844808 9620 /usr/lib64/ld-linux-x86-64.so.2 2024-03-21T10:28:38.567 INFO:tasks.workunit.client.0.smithi047.stdout:lsof 91794 ubuntu 4r FIFO 0,13 0t0 230103 pipe 2024-03-21T10:28:38.567 INFO:tasks.workunit.client.0.smithi047.stdout:lsof 91794 ubuntu 7w FIFO 0,13 0t0 230104 pipe 2024-03-21T10:28:38.568 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-03-21T10:28:38.568 DEBUG:teuthology.orchestra.run.smithi047:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-03-21T10:28:38.767 INFO:tasks.workunit:Stopping ['suites/fsync-tester.sh'] on client.0... 2024-03-21T10:28:38.767 DEBUG:teuthology.orchestra.run.smithi047:> sudo rm -rf -- /home/ubuntu/cephtest/workunits.list.client.0 /home/ubuntu/cephtest/clone.client.0 2024-03-21T10:28:39.712 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:28:39 smithi047 ceph-mon[31547]: pgmap v103: 129 pgs: 129 active+clean; 446 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 7.9 MiB/s wr, 1.70k op/s 2024-03-21T10:28:39.712 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:28:39 smithi047 ceph-mon[31547]: scrub summary: idle 2024-03-21T10:28:39.728 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:28:39 smithi100 ceph-mon[35193]: pgmap v103: 129 pgs: 129 active+clean; 446 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 7.9 MiB/s wr, 1.70k op/s 2024-03-21T10:28:39.728 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:28:39 smithi100 ceph-mon[35193]: scrub summary: idle 2024-03-21T10:28:39.738 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:28:39 smithi157 ceph-mon[35581]: pgmap v103: 129 pgs: 129 active+clean; 446 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 7.9 MiB/s wr, 1.70k op/s 2024-03-21T10:28:39.738 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:28:39 smithi157 ceph-mon[35581]: scrub summary: idle 2024-03-21T10:28:39.844 DEBUG:teuthology.parallel:result is None 2024-03-21T10:28:39.844 DEBUG:teuthology.orchestra.run.smithi047:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0 2024-03-21T10:28:39.958 INFO:tasks.workunit:Deleted dir /home/ubuntu/cephtest/mnt.0/client.0 2024-03-21T10:28:39.958 DEBUG:teuthology.run_tasks:Unwinding manager background_exec 2024-03-21T10:28:39.969 INFO:teuthology.task.background_exec:Stopping background command on ubuntu@smithi047.front.sepia.ceph.com 2024-03-21T10:28:39.988 DEBUG:teuthology.run_tasks:Unwinding manager full_sequential_finally 2024-03-21T10:28:39.999 INFO:teuthology.task.full_sequential_finally:In full_sequential_finally, running task exec... 2024-03-21T10:28:39.999 INFO:teuthology.task.exec:Executing custom commands... 2024-03-21T10:28:39.999 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi047.front.sepia.ceph.com 2024-03-21T10:28:40.000 DEBUG:teuthology.orchestra.run.smithi047:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'date +%s > END_TIME' 2024-03-21T10:28:40.137 DEBUG:teuthology.orchestra.run.smithi047:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'START_TIME=$(cat START_TIME); END_TIME=$(cat END_TIME); DIFF_TIME=$((600-(END_TIME-START_TIME))); if [ "$DIFF_TIME" -gt 0 ]; then sleep "$DIFF_TIME"; fi' 2024-03-21T10:28:41.712 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:28:41 smithi047 ceph-mon[31547]: pgmap v104: 129 pgs: 129 active+clean; 446 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 6.9 MiB/s wr, 1.46k op/s 2024-03-21T10:28:41.728 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:28:41 smithi100 ceph-mon[35193]: pgmap v104: 129 pgs: 129 active+clean; 446 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 6.9 MiB/s wr, 1.46k op/s 2024-03-21T10:28:41.738 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:28:41 smithi157 ceph-mon[35581]: pgmap v104: 129 pgs: 129 active+clean; 446 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 6.9 MiB/s wr, 1.46k op/s 2024-03-21T10:28:43.712 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:28:43 smithi047 ceph-mon[31547]: pgmap v105: 129 pgs: 129 active+clean; 277 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 8.9 MiB/s wr, 1.91k op/s 2024-03-21T10:28:43.728 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:28:43 smithi100 ceph-mon[35193]: pgmap v105: 129 pgs: 129 active+clean; 277 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 8.9 MiB/s wr, 1.91k op/s 2024-03-21T10:28:43.738 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:28:43 smithi157 ceph-mon[35581]: pgmap v105: 129 pgs: 129 active+clean; 277 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 8.9 MiB/s wr, 1.91k op/s 2024-03-21T10:28:45.712 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:28:45 smithi047 ceph-mon[31547]: pgmap v106: 129 pgs: 129 active+clean; 277 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 5.3 MiB/s wr, 1.15k op/s 2024-03-21T10:28:45.728 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:28:45 smithi100 ceph-mon[35193]: pgmap v106: 129 pgs: 129 active+clean; 277 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 5.3 MiB/s wr, 1.15k op/s 2024-03-21T10:28:45.738 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:28:45 smithi157 ceph-mon[35581]: pgmap v106: 129 pgs: 129 active+clean; 277 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 5.3 MiB/s wr, 1.15k op/s 2024-03-21T10:28:47.712 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:28:47 smithi047 ceph-mon[31547]: pgmap v107: 129 pgs: 129 active+clean; 277 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 5.3 MiB/s wr, 1.15k op/s 2024-03-21T10:28:47.728 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:28:47 smithi100 ceph-mon[35193]: pgmap v107: 129 pgs: 129 active+clean; 277 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 5.3 MiB/s wr, 1.15k op/s 2024-03-21T10:28:47.738 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:28:47 smithi157 ceph-mon[35581]: pgmap v107: 129 pgs: 129 active+clean; 277 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 5.3 MiB/s wr, 1.15k op/s 2024-03-21T10:28:49.712 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:28:49 smithi047 ceph-mon[31547]: pgmap v108: 129 pgs: 129 active+clean; 277 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 3.4 MiB/s wr, 699 op/s 2024-03-21T10:28:49.727 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:28:49 smithi100 ceph-mon[35193]: pgmap v108: 129 pgs: 129 active+clean; 277 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 3.4 MiB/s wr, 699 op/s 2024-03-21T10:28:49.738 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:28:49 smithi157 ceph-mon[35581]: pgmap v108: 129 pgs: 129 active+clean; 277 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 3.4 MiB/s wr, 699 op/s 2024-03-21T10:28:51.712 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:28:51 smithi047 ceph-mon[31547]: pgmap v109: 129 pgs: 129 active+clean; 277 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 2.0 MiB/s wr, 460 op/s 2024-03-21T10:28:51.728 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:28:51 smithi100 ceph-mon[35193]: pgmap v109: 129 pgs: 129 active+clean; 277 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 2.0 MiB/s wr, 460 op/s 2024-03-21T10:28:51.738 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:28:51 smithi157 ceph-mon[35581]: pgmap v109: 129 pgs: 129 active+clean; 277 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 2.0 MiB/s wr, 460 op/s 2024-03-21T10:28:53.728 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:28:53 smithi100 ceph-mon[35193]: pgmap v110: 129 pgs: 129 active+clean; 277 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 2.0 MiB/s wr, 465 op/s 2024-03-21T10:28:53.738 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:28:53 smithi157 ceph-mon[35581]: pgmap v110: 129 pgs: 129 active+clean; 277 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 2.0 MiB/s wr, 465 op/s 2024-03-21T10:28:53.921 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:28:53 smithi047 ceph-mon[31547]: pgmap v110: 129 pgs: 129 active+clean; 277 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 2.0 MiB/s wr, 465 op/s 2024-03-21T10:28:55.712 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:28:55 smithi047 ceph-mon[31547]: pgmap v111: 129 pgs: 129 active+clean; 277 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 7.8 KiB/s rd, 4.2 KiB/s wr, 9 op/s 2024-03-21T10:28:55.727 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:28:55 smithi100 ceph-mon[35193]: pgmap v111: 129 pgs: 129 active+clean; 277 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 7.8 KiB/s rd, 4.2 KiB/s wr, 9 op/s 2024-03-21T10:28:55.738 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:28:55 smithi157 ceph-mon[35581]: pgmap v111: 129 pgs: 129 active+clean; 277 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 7.8 KiB/s rd, 4.2 KiB/s wr, 9 op/s 2024-03-21T10:28:57.738 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:28:57 smithi157 ceph-mon[35581]: pgmap v112: 129 pgs: 129 active+clean; 277 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 9.5 KiB/s rd, 4.2 KiB/s wr, 13 op/s 2024-03-21T10:28:57.962 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:28:57 smithi047 ceph-mon[31547]: pgmap v112: 129 pgs: 129 active+clean; 277 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 9.5 KiB/s rd, 4.2 KiB/s wr, 13 op/s 2024-03-21T10:28:57.978 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:28:57 smithi100 ceph-mon[35193]: pgmap v112: 129 pgs: 129 active+clean; 277 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 9.5 KiB/s rd, 4.2 KiB/s wr, 13 op/s 2024-03-21T10:28:59.738 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:28:59 smithi157 ceph-mon[35581]: pgmap v113: 129 pgs: 129 active+clean; 277 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 2.1 KiB/s wr, 10 op/s 2024-03-21T10:28:59.962 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:28:59 smithi047 ceph-mon[31547]: pgmap v113: 129 pgs: 129 active+clean; 277 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 2.1 KiB/s wr, 10 op/s 2024-03-21T10:28:59.977 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:28:59 smithi100 ceph-mon[35193]: pgmap v113: 129 pgs: 129 active+clean; 277 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 2.1 KiB/s wr, 10 op/s 2024-03-21T10:29:01.962 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:29:01 smithi047 ceph-mon[31547]: pgmap v114: 129 pgs: 129 active+clean; 277 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 1.9 KiB/s wr, 8 op/s 2024-03-21T10:29:01.977 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:29:01 smithi100 ceph-mon[35193]: pgmap v114: 129 pgs: 129 active+clean; 277 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 1.9 KiB/s wr, 8 op/s 2024-03-21T10:29:01.988 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:29:01 smithi157 ceph-mon[35581]: pgmap v114: 129 pgs: 129 active+clean; 277 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 1.9 KiB/s wr, 8 op/s 2024-03-21T10:29:03.962 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:29:03 smithi047 ceph-mon[31547]: pgmap v115: 129 pgs: 129 active+clean; 277 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 24 KiB/s wr, 27 op/s 2024-03-21T10:29:03.977 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:29:03 smithi100 ceph-mon[35193]: pgmap v115: 129 pgs: 129 active+clean; 277 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 24 KiB/s wr, 27 op/s 2024-03-21T10:29:03.988 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:29:03 smithi157 ceph-mon[35581]: pgmap v115: 129 pgs: 129 active+clean; 277 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 24 KiB/s wr, 27 op/s 2024-03-21T10:29:05.962 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:29:05 smithi047 ceph-mon[31547]: pgmap v116: 129 pgs: 129 active+clean; 277 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-03-21T10:29:05.977 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:29:05 smithi100 ceph-mon[35193]: pgmap v116: 129 pgs: 129 active+clean; 277 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-03-21T10:29:05.988 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:29:05 smithi157 ceph-mon[35581]: pgmap v116: 129 pgs: 129 active+clean; 277 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-03-21T10:29:06.933 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-21T10:29:06.933 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-21T10:29:07.188 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:07.188+0000 7f88ab200640 1 -- 172.21.15.47:0/2102167462 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f88ac103720 msgr2=0x7f88ac103ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:29:07.188 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:07.188+0000 7f88ab200640 1 --2- 172.21.15.47:0/2102167462 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f88ac103720 0x7f88ac103ba0 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7f8898009930 tx=0x7f889802bc30 comp rx=0 tx=0).stop 2024-03-21T10:29:07.188 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:07.188+0000 7f88ab200640 1 -- 172.21.15.47:0/2102167462 shutdown_connections 2024-03-21T10:29:07.188 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:07.188+0000 7f88ab200640 1 --2- 172.21.15.47:0/2102167462 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f88ac1040e0 0x7f88ac1105f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:29:07.188 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:07.188+0000 7f88ab200640 1 --2- 172.21.15.47:0/2102167462 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f88ac103720 0x7f88ac103ba0 unknown :-1 s=CLOSED pgs=755 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:29:07.189 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:07.188+0000 7f88ab200640 1 --2- 172.21.15.47:0/2102167462 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f88ac109730 0x7f88ac109b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:29:07.189 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:07.188+0000 7f88ab200640 1 -- 172.21.15.47:0/2102167462 >> 172.21.15.47:0/2102167462 conn(0x7f88ac0ff1d0 msgr2=0x7f88ac1015f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:29:07.189 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:07.188+0000 7f88ab200640 1 -- 172.21.15.47:0/2102167462 shutdown_connections 2024-03-21T10:29:07.189 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:07.188+0000 7f88ab200640 1 -- 172.21.15.47:0/2102167462 wait complete. 2024-03-21T10:29:07.191 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:07.189+0000 7f88ab200640 1 Processor -- start 2024-03-21T10:29:07.191 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:07.189+0000 7f88ab200640 1 -- start start 2024-03-21T10:29:07.191 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:07.190+0000 7f88ab200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f88ac103720 0x7f88ac199f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:29:07.191 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:07.190+0000 7f88ab200640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f88ac1040e0 0x7f88ac19a4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:29:07.191 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:07.190+0000 7f88ab200640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f88ac109730 0x7f88ac19e840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:29:07.191 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:07.190+0000 7f88ab200640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88ac113a00 con 0x7f88ac109730 2024-03-21T10:29:07.192 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:07.190+0000 7f88ab200640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88ac113880 con 0x7f88ac103720 2024-03-21T10:29:07.192 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:07.190+0000 7f88ab200640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88ac113b80 con 0x7f88ac1040e0 2024-03-21T10:29:07.192 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:07.190+0000 7f88aa800640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f88ac109730 0x7f88ac19e840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:29:07.192 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:07.190+0000 7f88aa800640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f88ac109730 0x7f88ac19e840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:52674/0 (socket says 172.21.15.47:52674) 2024-03-21T10:29:07.192 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:07.190+0000 7f88aa800640 1 -- 172.21.15.47:0/355643268 learned_addr learned my addr 172.21.15.47:0/355643268 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:29:07.192 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:07.190+0000 7f88a9e00640 1 --2- 172.21.15.47:0/355643268 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f88ac103720 0x7f88ac199f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:29:07.192 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:07.190+0000 7f88a9400640 1 --2- 172.21.15.47:0/355643268 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f88ac1040e0 0x7f88ac19a4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:29:07.192 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:07.191+0000 7f88aa800640 1 -- 172.21.15.47:0/355643268 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f88ac1040e0 msgr2=0x7f88ac19a4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:29:07.192 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:07.191+0000 7f88aa800640 1 --2- 172.21.15.47:0/355643268 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f88ac1040e0 0x7f88ac19a4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:29:07.192 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:07.191+0000 7f88aa800640 1 -- 172.21.15.47:0/355643268 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f88ac103720 msgr2=0x7f88ac199f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:29:07.192 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:07.191+0000 7f88aa800640 1 --2- 172.21.15.47:0/355643268 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f88ac103720 0x7f88ac199f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:29:07.192 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:07.191+0000 7f88aa800640 1 -- 172.21.15.47:0/355643268 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f88ac19ef70 con 0x7f88ac109730 2024-03-21T10:29:07.193 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:07.191+0000 7f88aa800640 1 --2- 172.21.15.47:0/355643268 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f88ac109730 0x7f88ac19e840 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7f88a400eb10 tx=0x7f88a400c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:29:07.193 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:07.191+0000 7f88a2a00640 1 -- 172.21.15.47:0/355643268 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88a4004b20 con 0x7f88ac109730 2024-03-21T10:29:07.193 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:07.191+0000 7f88a2a00640 1 -- 172.21.15.47:0/355643268 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f88a4004cc0 con 0x7f88ac109730 2024-03-21T10:29:07.193 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:07.191+0000 7f88a2a00640 1 -- 172.21.15.47:0/355643268 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88a4010690 con 0x7f88ac109730 2024-03-21T10:29:07.193 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:07.191+0000 7f88ab200640 1 -- 172.21.15.47:0/355643268 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f88ac19f260 con 0x7f88ac109730 2024-03-21T10:29:07.193 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:07.192+0000 7f88ab200640 1 -- 172.21.15.47:0/355643268 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f88ac1a6af0 con 0x7f88ac109730 2024-03-21T10:29:07.194 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:07.195+0000 7f88a2a00640 1 -- 172.21.15.47:0/355643268 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 27) v1 ==== 137985+0+0 (secure 0 0 0) 0x7f88a4005020 con 0x7f88ac109730 2024-03-21T10:29:07.195 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:07.196+0000 7f88b37aa740 1 Processor -- start 2024-03-21T10:29:07.195 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:07.196+0000 7f88b37aa740 1 -- start start 2024-03-21T10:29:07.197 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:07.196+0000 7f88b37aa740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563c2163cdd0 0x563c2163bb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:29:07.197 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:07.196+0000 7f88b37aa740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x563c2163c0f0 0x563c2163c570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:29:07.198 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:07.196+0000 7f88b37aa740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563c21645460 0x563c21647850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:29:07.198 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:07.196+0000 7f88b37aa740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x563c215a0250 con 0x563c2163c0f0 2024-03-21T10:29:07.198 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:07.196+0000 7f88b37aa740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x563c21548a30 con 0x563c21645460 2024-03-21T10:29:07.198 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:07.196+0000 7f88b37aa740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x563c2159c2f0 con 0x563c2163cdd0 2024-03-21T10:29:07.198 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:07.197+0000 7f88a9e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x563c2163c0f0 0x563c2163c570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:29:07.198 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:07.197+0000 7f88a9e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x563c2163c0f0 0x563c2163c570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:52680/0 (socket says 172.21.15.47:52680) 2024-03-21T10:29:07.198 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:07.197+0000 7f88a9e00640 1 -- 172.21.15.47:0/2120867354 learned_addr learned my addr 172.21.15.47:0/2120867354 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:29:07.198 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:07.197+0000 7f88a9400640 1 --2- 172.21.15.47:0/2120867354 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563c21645460 0x563c21647850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:29:07.198 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:07.197+0000 7f88a9400640 1 --2- 172.21.15.47:0/2120867354 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563c2163cdd0 0x563c2163bb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:29:07.199 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:07.197+0000 7f88a9e00640 1 -- 172.21.15.47:0/2120867354 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563c2163cdd0 msgr2=0x563c2163bb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:29:07.199 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:07.197+0000 7f88a9e00640 1 --2- 172.21.15.47:0/2120867354 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563c2163cdd0 0x563c2163bb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:29:07.199 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:07.197+0000 7f88a9e00640 1 -- 172.21.15.47:0/2120867354 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563c21645460 msgr2=0x563c21647850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:29:07.199 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:07.197+0000 7f88a9e00640 1 --2- 172.21.15.47:0/2120867354 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563c21645460 0x563c21647850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:29:07.199 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:07.197+0000 7f88a9e00640 1 -- 172.21.15.47:0/2120867354 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563c21533690 con 0x563c2163c0f0 2024-03-21T10:29:07.199 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:07.197+0000 7f88a9400640 1 --2- 172.21.15.47:0/2120867354 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563c2163cdd0 0x563c2163bb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T10:29:07.199 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:07.197+0000 7f88a9e00640 1 --2- 172.21.15.47:0/2120867354 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x563c2163c0f0 0x563c2163c570 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7f889c00bc90 tx=0x7f889c0107a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6dd883f28c1cf484 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:29:07.199 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:07.198+0000 7f88a2a00640 1 --2- 172.21.15.47:0/355643268 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f88900a3110 0x7f88900a55d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:29:07.199 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:07.198+0000 7f8897000640 1 -- 172.21.15.47:0/2120867354 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f889c01c480 con 0x563c2163c0f0 2024-03-21T10:29:07.200 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:07.198+0000 7f88a2a00640 1 -- 172.21.15.47:0/355643268 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 9474+0+0 (secure 0 0 0) 0x7f88a40d0150 con 0x7f88ac109730 2024-03-21T10:29:07.200 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:07.198+0000 7f8897000640 1 -- 172.21.15.47:0/2120867354 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f889c01cad0 con 0x563c2163c0f0 2024-03-21T10:29:07.200 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:07.198+0000 7f88a9400640 1 --2- 172.21.15.47:0/355643268 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f88900a3110 0x7f88900a55d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:29:07.200 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:07.198+0000 7f8897000640 1 -- 172.21.15.47:0/2120867354 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f889c011ae0 con 0x563c2163c0f0 2024-03-21T10:29:07.203 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:07.199+0000 7f88a9400640 1 --2- 172.21.15.47:0/355643268 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f88900a3110 0x7f88900a55d0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f8898009900 tx=0x7f8898005fd0 comp rx=0 tx=0).ready entity=mgr.15687 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:29:07.203 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:07.199+0000 7f88b37aa740 1 -- 172.21.15.47:0/2120867354 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x563c2163c0f0 msgr2=0x563c2163c570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:29:07.203 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:07.199+0000 7f88b37aa740 1 --2- 172.21.15.47:0/2120867354 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x563c2163c0f0 0x563c2163c570 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7f889c00bc90 tx=0x7f889c0107a0 comp rx=0 tx=0).stop 2024-03-21T10:29:07.203 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:07.199+0000 7f88b37aa740 1 -- 172.21.15.47:0/2120867354 shutdown_connections 2024-03-21T10:29:07.203 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:07.199+0000 7f88b37aa740 1 --2- 172.21.15.47:0/2120867354 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563c21645460 0x563c21647850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:29:07.204 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:07.199+0000 7f88b37aa740 1 --2- 172.21.15.47:0/2120867354 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x563c2163c0f0 0x563c2163c570 unknown :-1 s=CLOSED pgs=757 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:29:07.204 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:07.199+0000 7f88b37aa740 1 --2- 172.21.15.47:0/2120867354 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563c2163cdd0 0x563c2163bb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:29:07.204 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:07.199+0000 7f88b37aa740 1 -- 172.21.15.47:0/2120867354 >> 172.21.15.47:0/2120867354 conn(0x563c21603200 msgr2=0x563c21636b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:29:07.204 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:07.199+0000 7f88b37aa740 1 -- 172.21.15.47:0/2120867354 shutdown_connections 2024-03-21T10:29:07.204 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:07.199+0000 7f88b37aa740 1 -- 172.21.15.47:0/2120867354 wait complete. 2024-03-21T10:29:07.204 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:07.199+0000 7f88b37aa740 1 -- start start 2024-03-21T10:29:07.204 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:07.200+0000 7f88b37aa740 1 Processor -- start 2024-03-21T10:29:07.204 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:07.200+0000 7f88b37aa740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x563c2163c0f0 0x563c21644af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:29:07.204 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:07.200+0000 7f88b37aa740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563c2163cdd0 0x563c21641e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:29:07.204 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:07.200+0000 7f88b37aa740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563c21645460 0x563c216423c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:29:07.204 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:07.200+0000 7f88b37aa740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x563c21648320 con 0x563c2163c0f0 2024-03-21T10:29:07.204 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:07.200+0000 7f88b37aa740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x563c2155be80 con 0x563c2163cdd0 2024-03-21T10:29:07.204 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:07.200+0000 7f88b37aa740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x563c216484a0 con 0x563c21645460 2024-03-21T10:29:07.204 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:07.200+0000 7f88b37aa740 10 client.0 _ll_register_callbacks cb 0x563c2156e130 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T10:29:07.205 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:07.200+0000 7f88a9e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x563c2163c0f0 0x563c21644af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:29:07.205 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:07.201+0000 7f88a9e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x563c2163c0f0 0x563c21644af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:52686/0 (socket says 172.21.15.47:52686) 2024-03-21T10:29:07.205 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:07.201+0000 7f88a9e00640 1 -- 172.21.15.47:0/134991551 learned_addr learned my addr 172.21.15.47:0/134991551 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:29:07.205 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:07.201+0000 7f88a9400640 1 --2- 172.21.15.47:0/134991551 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563c2163cdd0 0x563c21641e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:29:07.205 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:07.201+0000 7f88aa800640 1 --2- 172.21.15.47:0/134991551 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563c21645460 0x563c216423c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:29:07.205 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:07.201+0000 7f88a9e00640 1 -- 172.21.15.47:0/134991551 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563c21645460 msgr2=0x563c216423c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:29:07.205 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:07.201+0000 7f88a9e00640 1 --2- 172.21.15.47:0/134991551 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563c21645460 0x563c216423c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:29:07.205 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:07.201+0000 7f88a9e00640 1 -- 172.21.15.47:0/134991551 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563c2163cdd0 msgr2=0x563c21641e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:29:07.205 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:07.201+0000 7f88a9e00640 1 --2- 172.21.15.47:0/134991551 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563c2163cdd0 0x563c21641e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:29:07.205 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:07.201+0000 7f88a9e00640 1 -- 172.21.15.47:0/134991551 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x563c2137e7d0 con 0x563c2163c0f0 2024-03-21T10:29:07.205 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:07.201+0000 7f88a9e00640 1 --2- 172.21.15.47:0/134991551 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x563c2163c0f0 0x563c21644af0 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7f889c011010 tx=0x7f889c01c6b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ad50f82708666f3e server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:29:07.205 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:07.201+0000 7f888b400640 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-03-21T10:29:07.206 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:07.202+0000 7f888b400640 1 -- 172.21.15.47:0/134991551 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f889c005040 con 0x563c2163c0f0 2024-03-21T10:29:07.206 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:07.202+0000 7f888b400640 1 -- 172.21.15.47:0/134991551 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(97..97 src has 1..97) v4 ==== 9474+0+0 (secure 0 0 0) 0x7f889c0051e0 con 0x563c2163c0f0 2024-03-21T10:29:07.206 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:07.202+0000 7f88b37aa740 1 -- 172.21.15.47:0/134991551 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f889c01cad0 con 0x563c2163c0f0 2024-03-21T10:29:07.206 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:07.202+0000 7f888b400640 1 -- 172.21.15.47:0/134991551 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f889c032050 con 0x563c2163c0f0 2024-03-21T10:29:07.206 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:07.202+0000 7f88b37aa740 10 client.16833 fetch_fsmap learned FSMap version 32 2024-03-21T10:29:07.206 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:07.202+0000 7f88b37aa740 1 -- 172.21.15.47:0/134991551 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x563c212b43f0 con 0x563c2163c0f0 2024-03-21T10:29:07.206 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:07.202+0000 7f888b400640 1 -- 172.21.15.47:0/134991551 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f889c01cad0 con 0x563c2163c0f0 2024-03-21T10:29:07.206 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:07.203+0000 7f888b400640 1 -- 172.21.15.47:0/134991551 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 32) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f889c0047e0 con 0x563c2163c0f0 2024-03-21T10:29:07.206 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:07.203+0000 7f88b37aa740 10 client.16833 fetch_fsmap finished waiting for FSMap version 32 2024-03-21T10:29:07.206 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:07.203+0000 7f88b37aa740 10 client.16833 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-21T10:29:07.206 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:07.203+0000 7f88b37aa740 20 client.16833 populate_metadata read hostname 'smithi047' 2024-03-21T10:29:07.206 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:07.203+0000 7f88b37aa740 1 --2- 172.21.15.47:0/134991551 >> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] conn(0x563c21655250 0x563c21657640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:29:07.206 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:07.203+0000 7f88b37aa740 4 client.16833 mds_command: new command op to 34535 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T10:29:07.206 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:07.203+0000 7f88b37aa740 1 -- 172.21.15.47:0/134991551 --> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x563c215ad790 con 0x563c21655250 2024-03-21T10:29:07.207 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:07.203+0000 7f88a9400640 1 --2- 172.21.15.47:0/134991551 >> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] conn(0x563c21655250 0x563c21657640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:29:07.208 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:07.204+0000 7f888b400640 10 client.16833 ms_handle_connect on v2:172.21.15.100:6838/4173168440 2024-03-21T10:29:07.208 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:07.204+0000 7f88a9400640 1 --2- 172.21.15.47:0/134991551 >> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] conn(0x563c21655250 0x563c21657640 crc :-1 s=READY pgs=121 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=99352ca1ff383586 server_cookie=f6c9f2bc2d9a6f45 in_seq=0 out_seq=0 2024-03-21T10:29:07.208 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:07.206+0000 7f888b400640 1 -- 172.21.15.47:0/134991551 <== mds.0 v2:172.21.15.100:6838/4173168440 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x563c215ad790 con 0x563c21655250 2024-03-21T10:29:07.208 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:07.206+0000 7f888b400640 10 client.16833 handle_command_reply: tid=0 2024-03-21T10:29:07.208 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:07.206+0000 7f88b37aa740 1 client.16833 shutdown 2024-03-21T10:29:07.208 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:07.207+0000 7f88b37aa740 1 -- 172.21.15.47:0/134991551 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x563c2163c0f0 msgr2=0x563c21644af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:29:07.208 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:07.207+0000 7f88b37aa740 1 --2- 172.21.15.47:0/134991551 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x563c2163c0f0 0x563c21644af0 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7f889c011010 tx=0x7f889c01c6b0 comp rx=0 tx=0).stop 2024-03-21T10:29:07.209 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:07.207+0000 7f88b37aa740 1 -- 172.21.15.47:0/134991551 shutdown_connections 2024-03-21T10:29:07.209 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:07.207+0000 7f88b37aa740 1 --2- 172.21.15.47:0/134991551 >> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] conn(0x563c21655250 0x563c21657640 crc :-1 s=READY pgs=121 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:29:07.209 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:07.207+0000 7f88b37aa740 1 --2- 172.21.15.47:0/134991551 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563c21645460 0x563c216423c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:29:07.209 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:07.207+0000 7f88b37aa740 1 --2- 172.21.15.47:0/134991551 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563c2163cdd0 0x563c21641e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:29:07.209 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:07.207+0000 7f88b37aa740 1 --2- 172.21.15.47:0/134991551 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x563c2163c0f0 0x563c21644af0 unknown :-1 s=CLOSED pgs=758 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:29:07.209 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:07.207+0000 7f88b37aa740 1 -- 172.21.15.47:0/134991551 >> 172.21.15.47:0/134991551 conn(0x563c21603200 msgr2=0x563c21638890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:29:07.209 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:07.207+0000 7f888b400640 0 client.16833 ms_handle_reset on v2:172.21.15.100:6838/4173168440 2024-03-21T10:29:07.209 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:07.207+0000 7f88b37aa740 1 -- 172.21.15.47:0/134991551 shutdown_connections 2024-03-21T10:29:07.210 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:07.207+0000 7f88b37aa740 1 -- 172.21.15.47:0/134991551 wait complete. 2024-03-21T10:29:07.210 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:07.207+0000 7f88b37aa740 20 client.16833 trim_cache size 0 max 16384 2024-03-21T10:29:07.228 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:07.228+0000 7f88b37aa740 1 Processor -- start 2024-03-21T10:29:07.228 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:07.228+0000 7f88b37aa740 1 -- start start 2024-03-21T10:29:07.228 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:07.228+0000 7f88b37aa740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563c2163c0f0 0x563c216445b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:29:07.228 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:07.228+0000 7f88b37aa740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563c2163cdd0 0x563c21644af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:29:07.229 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:07.228+0000 7f88b37aa740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x563c21645460 0x563c21646850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:29:07.229 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:07.228+0000 7f88b37aa740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88ac1b0980 con 0x563c21645460 2024-03-21T10:29:07.229 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:07.228+0000 7f88b37aa740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88ac1b2300 con 0x563c2163cdd0 2024-03-21T10:29:07.229 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:07.228+0000 7f88b37aa740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88ac1b0fe0 con 0x563c2163c0f0 2024-03-21T10:29:07.229 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:07.228+0000 7f88aa800640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x563c21645460 0x563c21646850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:29:07.229 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:07.228+0000 7f88aa800640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x563c21645460 0x563c21646850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:52698/0 (socket says 172.21.15.47:52698) 2024-03-21T10:29:07.229 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:07.229+0000 7f88aa800640 1 -- 172.21.15.47:0/1307679358 learned_addr learned my addr 172.21.15.47:0/1307679358 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:29:07.229 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:07.229+0000 7f88a9400640 1 --2- 172.21.15.47:0/1307679358 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563c2163cdd0 0x563c21644af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:29:07.229 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:07.229+0000 7f88a9e00640 1 --2- 172.21.15.47:0/1307679358 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563c2163c0f0 0x563c216445b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:29:07.229 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:07.229+0000 7f88aa800640 1 -- 172.21.15.47:0/1307679358 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563c2163c0f0 msgr2=0x563c216445b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:29:07.229 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:07.229+0000 7f88aa800640 1 --2- 172.21.15.47:0/1307679358 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563c2163c0f0 0x563c216445b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:29:07.229 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:07.229+0000 7f88aa800640 1 -- 172.21.15.47:0/1307679358 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563c2163cdd0 msgr2=0x563c21644af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:29:07.229 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:07.229+0000 7f88aa800640 1 --2- 172.21.15.47:0/1307679358 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563c2163cdd0 0x563c21644af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:29:07.229 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:07.229+0000 7f88aa800640 1 -- 172.21.15.47:0/1307679358 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563c212740f0 con 0x563c21645460 2024-03-21T10:29:07.232 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:07.229+0000 7f88aa800640 1 --2- 172.21.15.47:0/1307679358 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x563c21645460 0x563c21646850 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7f88a40049d0 tx=0x7f88a409bcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e43ad14f921febb9 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:29:07.232 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:07.229+0000 7f8896600640 1 -- 172.21.15.47:0/1307679358 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88a409c180 con 0x563c21645460 2024-03-21T10:29:07.233 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:07.230+0000 7f8896600640 1 -- 172.21.15.47:0/1307679358 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f88a40a02d0 con 0x563c21645460 2024-03-21T10:29:07.233 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:07.230+0000 7f8896600640 1 -- 172.21.15.47:0/1307679358 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88a40a4700 con 0x563c21645460 2024-03-21T10:29:07.233 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:07.230+0000 7f88b37aa740 1 -- 172.21.15.47:0/1307679358 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x563c21645460 msgr2=0x563c21646850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:29:07.233 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:07.230+0000 7f88b37aa740 1 --2- 172.21.15.47:0/1307679358 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x563c21645460 0x563c21646850 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7f88a40049d0 tx=0x7f88a409bcc0 comp rx=0 tx=0).stop 2024-03-21T10:29:07.233 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:07.230+0000 7f88b37aa740 1 -- 172.21.15.47:0/1307679358 shutdown_connections 2024-03-21T10:29:07.233 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:07.230+0000 7f88b37aa740 1 --2- 172.21.15.47:0/1307679358 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x563c21645460 0x563c21646850 unknown :-1 s=CLOSED pgs=759 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:29:07.233 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:07.230+0000 7f88b37aa740 1 --2- 172.21.15.47:0/1307679358 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563c2163cdd0 0x563c21644af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:29:07.233 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:07.230+0000 7f88b37aa740 1 --2- 172.21.15.47:0/1307679358 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563c2163c0f0 0x563c216445b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:29:07.233 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:07.230+0000 7f88b37aa740 1 -- 172.21.15.47:0/1307679358 >> 172.21.15.47:0/1307679358 conn(0x563c21603200 msgr2=0x563c21640790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:29:07.233 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:07.230+0000 7f88b37aa740 1 -- 172.21.15.47:0/1307679358 shutdown_connections 2024-03-21T10:29:07.233 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:07.230+0000 7f88b37aa740 1 -- 172.21.15.47:0/1307679358 wait complete. 2024-03-21T10:29:07.233 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:07.231+0000 7f88b37aa740 1 -- start start 2024-03-21T10:29:07.233 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:07.231+0000 7f88b37aa740 1 Processor -- start 2024-03-21T10:29:07.233 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:07.232+0000 7f88b37aa740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x563c2163c0f0 0x563c2163ecd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:29:07.234 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:07.232+0000 7f88b37aa740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563c2163cdd0 0x563c2163f210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:29:07.234 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:07.232+0000 7f88b37aa740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563c21645460 0x563c216af990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:29:07.234 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:07.232+0000 7f88b37aa740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88ac1b1ca0 con 0x563c2163c0f0 2024-03-21T10:29:07.234 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:07.232+0000 7f88b37aa740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88ac1b1640 con 0x563c2163cdd0 2024-03-21T10:29:07.234 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:07.232+0000 7f88b37aa740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88ac1aeb40 con 0x563c21645460 2024-03-21T10:29:07.234 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:07.232+0000 7f88b37aa740 10 client.0 _ll_register_callbacks cb 0x7f889c00bae0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T10:29:07.234 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:07.232+0000 7f88a9e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x563c2163c0f0 0x563c2163ecd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:29:07.234 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:07.232+0000 7f88a9e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x563c2163c0f0 0x563c2163ecd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:52700/0 (socket says 172.21.15.47:52700) 2024-03-21T10:29:07.234 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:07.232+0000 7f88a9e00640 1 -- 172.21.15.47:0/2304960737 learned_addr learned my addr 172.21.15.47:0/2304960737 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:29:07.234 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:07.232+0000 7f88a9400640 1 --2- 172.21.15.47:0/2304960737 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563c2163cdd0 0x563c2163f210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:29:07.234 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:07.233+0000 7f88aa800640 1 --2- 172.21.15.47:0/2304960737 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563c21645460 0x563c216af990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:29:07.234 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:07.233+0000 7f88a9e00640 1 -- 172.21.15.47:0/2304960737 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563c21645460 msgr2=0x563c216af990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:29:07.234 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:07.233+0000 7f88a9e00640 1 --2- 172.21.15.47:0/2304960737 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563c21645460 0x563c216af990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:29:07.234 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:07.233+0000 7f88a9e00640 1 -- 172.21.15.47:0/2304960737 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563c2163cdd0 msgr2=0x563c2163f210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:29:07.235 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:07.233+0000 7f88a9e00640 1 --2- 172.21.15.47:0/2304960737 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563c2163cdd0 0x563c2163f210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:29:07.235 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:07.233+0000 7f88a9e00640 1 -- 172.21.15.47:0/2304960737 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x563c216aff20 con 0x563c2163c0f0 2024-03-21T10:29:07.235 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:07.233+0000 7f88a9e00640 1 --2- 172.21.15.47:0/2304960737 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x563c2163c0f0 0x563c2163ecd0 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7f889c011010 tx=0x7f889c005e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ffc6ad311a123d4f server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:29:07.235 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:07.233+0000 7f888b400640 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-03-21T10:29:07.237 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:07.233+0000 7f888b400640 1 -- 172.21.15.47:0/2304960737 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f889c01ccb0 con 0x563c2163c0f0 2024-03-21T10:29:07.237 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:07.234+0000 7f888b400640 1 -- 172.21.15.47:0/2304960737 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(97..97 src has 1..97) v4 ==== 9474+0+0 (secure 0 0 0) 0x7f889c0117b0 con 0x563c2163c0f0 2024-03-21T10:29:07.237 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:07.234+0000 7f88b37aa740 1 -- 172.21.15.47:0/2304960737 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f88a40a02d0 con 0x563c2163c0f0 2024-03-21T10:29:07.237 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:07.234+0000 7f888b400640 1 -- 172.21.15.47:0/2304960737 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f889c044050 con 0x563c2163c0f0 2024-03-21T10:29:07.237 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:07.234+0000 7f88b37aa740 10 client.16839 fetch_fsmap learned FSMap version 32 2024-03-21T10:29:07.237 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:07.234+0000 7f88b37aa740 1 -- 172.21.15.47:0/2304960737 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x563c216b1210 con 0x563c2163c0f0 2024-03-21T10:29:07.237 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:07.234+0000 7f888b400640 1 -- 172.21.15.47:0/2304960737 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f88a40a02d0 con 0x563c2163c0f0 2024-03-21T10:29:07.238 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:07.234+0000 7f888b400640 1 -- 172.21.15.47:0/2304960737 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 32) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f889c0035f0 con 0x563c2163c0f0 2024-03-21T10:29:07.238 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:07.234+0000 7f88b37aa740 10 client.16839 fetch_fsmap finished waiting for FSMap version 32 2024-03-21T10:29:07.238 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:07.234+0000 7f88b37aa740 10 client.16839 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-21T10:29:07.238 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:07.234+0000 7f88b37aa740 20 client.16839 populate_metadata read hostname 'smithi047' 2024-03-21T10:29:07.238 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:07.234+0000 7f88b37aa740 1 --2- 172.21.15.47:0/2304960737 >> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] conn(0x563c216b1960 0x563c216b3d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:29:07.238 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:07.235+0000 7f88b37aa740 4 client.16839 mds_command: new command op to 34535 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-21T10:29:07.238 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:07.235+0000 7f88b37aa740 1 -- 172.21.15.47:0/2304960737 --> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x563c215a7ae0 con 0x563c216b1960 2024-03-21T10:29:07.238 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:07.235+0000 7f88a9400640 1 --2- 172.21.15.47:0/2304960737 >> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] conn(0x563c216b1960 0x563c216b3d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:29:07.238 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:07.236+0000 7f88a9400640 1 --2- 172.21.15.47:0/2304960737 >> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] conn(0x563c216b1960 0x563c216b3d80 crc :-1 s=READY pgs=122 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7e485f5109b40bd7 server_cookie=2370b67eebf823b7 in_seq=0 out_seq=0 2024-03-21T10:29:07.238 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:07.236+0000 7f888b400640 10 client.16839 ms_handle_connect on v2:172.21.15.100:6838/4173168440 2024-03-21T10:29:07.238 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:07.236+0000 7f888b400640 1 -- 172.21.15.47:0/2304960737 <== mds.0 v2:172.21.15.100:6838/4173168440 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x563c215a7ae0 con 0x563c216b1960 2024-03-21T10:29:07.238 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:07.236+0000 7f888b400640 10 client.16839 handle_command_reply: tid=0 2024-03-21T10:29:07.238 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:07.236+0000 7f88b37aa740 1 client.16839 shutdown 2024-03-21T10:29:07.238 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:07.236+0000 7f88b37aa740 1 -- 172.21.15.47:0/2304960737 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x563c2163c0f0 msgr2=0x563c2163ecd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:29:07.239 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:07.236+0000 7f88b37aa740 1 --2- 172.21.15.47:0/2304960737 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x563c2163c0f0 0x563c2163ecd0 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7f889c011010 tx=0x7f889c005e50 comp rx=0 tx=0).stop 2024-03-21T10:29:07.239 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:07.237+0000 7f88b37aa740 1 -- 172.21.15.47:0/2304960737 shutdown_connections 2024-03-21T10:29:07.239 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:07.237+0000 7f88b37aa740 1 --2- 172.21.15.47:0/2304960737 >> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] conn(0x563c216b1960 0x563c216b3d80 crc :-1 s=READY pgs=122 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:29:07.239 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:07.237+0000 7f88b37aa740 1 --2- 172.21.15.47:0/2304960737 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563c21645460 0x563c216af990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:29:07.239 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:07.237+0000 7f88b37aa740 1 --2- 172.21.15.47:0/2304960737 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563c2163cdd0 0x563c2163f210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:29:07.239 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:07.237+0000 7f88b37aa740 1 --2- 172.21.15.47:0/2304960737 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x563c2163c0f0 0x563c2163ecd0 unknown :-1 s=CLOSED pgs=760 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:29:07.239 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:07.237+0000 7f88b37aa740 1 -- 172.21.15.47:0/2304960737 >> 172.21.15.47:0/2304960737 conn(0x563c21603200 msgr2=0x563c21635330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:29:07.239 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:07.237+0000 7f888b400640 0 client.16839 ms_handle_reset on v2:172.21.15.100:6838/4173168440 2024-03-21T10:29:07.239 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:07.237+0000 7f88b37aa740 1 -- 172.21.15.47:0/2304960737 shutdown_connections 2024-03-21T10:29:07.239 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:07.237+0000 7f88b37aa740 1 -- 172.21.15.47:0/2304960737 wait complete. 2024-03-21T10:29:07.239 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:07.237+0000 7f88b37aa740 20 client.16839 trim_cache size 0 max 16384 2024-03-21T10:29:07.240 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-03-21T10:29:07.240 INFO:teuthology.orchestra.run.smithi047.stdout: "status": "no active scrubs running", 2024-03-21T10:29:07.240 INFO:teuthology.orchestra.run.smithi047.stdout: "scrubs": {} 2024-03-21T10:29:07.240 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-03-21T10:29:07.240 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:07.238+0000 7f88ab200640 1 -- 172.21.15.47:0/355643268 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f88900a3110 msgr2=0x7f88900a55d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:29:07.240 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:07.238+0000 7f88ab200640 1 --2- 172.21.15.47:0/355643268 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f88900a3110 0x7f88900a55d0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f8898009900 tx=0x7f8898005fd0 comp rx=0 tx=0).stop 2024-03-21T10:29:07.241 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:07.238+0000 7f88ab200640 1 -- 172.21.15.47:0/355643268 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f88ac109730 msgr2=0x7f88ac19e840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:29:07.241 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:07.238+0000 7f88ab200640 1 --2- 172.21.15.47:0/355643268 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f88ac109730 0x7f88ac19e840 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7f88a400eb10 tx=0x7f88a400c2d0 comp rx=0 tx=0).stop 2024-03-21T10:29:07.241 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:07.238+0000 7f88ab200640 1 -- 172.21.15.47:0/355643268 shutdown_connections 2024-03-21T10:29:07.241 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:07.238+0000 7f88ab200640 1 --2- 172.21.15.47:0/355643268 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f88ac109730 0x7f88ac19e840 unknown :-1 s=CLOSED pgs=756 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:29:07.241 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:07.238+0000 7f88ab200640 1 --2- 172.21.15.47:0/355643268 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f88ac1040e0 0x7f88ac19a4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:29:07.241 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:07.238+0000 7f88ab200640 1 --2- 172.21.15.47:0/355643268 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f88900a3110 0x7f88900a55d0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:29:07.241 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:07.238+0000 7f88ab200640 1 --2- 172.21.15.47:0/355643268 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f88ac103720 0x7f88ac199f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:29:07.242 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:07.238+0000 7f88ab200640 1 -- 172.21.15.47:0/355643268 >> 172.21.15.47:0/355643268 conn(0x7f88ac0ff1d0 msgr2=0x7f88ac0ffa60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:29:07.242 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:07.238+0000 7f88ab200640 1 -- 172.21.15.47:0/355643268 shutdown_connections 2024-03-21T10:29:07.242 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:07.238+0000 7f88ab200640 1 -- 172.21.15.47:0/355643268 wait complete. 2024-03-21T10:29:07.256 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-03-21T10:29:07.257 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-21T10:29:07.460 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:07.460+0000 7f459e800640 1 -- 172.21.15.47:0/2558827455 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4598107a60 msgr2=0x7f4598107ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:29:07.461 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:07.460+0000 7f459e800640 1 --2- 172.21.15.47:0/2558827455 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4598107a60 0x7f4598107ee0 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7f4584009930 tx=0x7f458402bc20 comp rx=0 tx=0).stop 2024-03-21T10:29:07.461 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:07.461+0000 7f459e800640 1 -- 172.21.15.47:0/2558827455 shutdown_connections 2024-03-21T10:29:07.461 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:07.461+0000 7f459e800640 1 --2- 172.21.15.47:0/2558827455 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4598108420 0x7f4598114930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:29:07.461 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:07.461+0000 7f459e800640 1 --2- 172.21.15.47:0/2558827455 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4598107a60 0x7f4598107ee0 unknown :-1 s=CLOSED pgs=761 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:29:07.461 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:07.461+0000 7f459e800640 1 --2- 172.21.15.47:0/2558827455 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f459810da70 0x7f459810de50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:29:07.461 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:07.461+0000 7f459e800640 1 -- 172.21.15.47:0/2558827455 >> 172.21.15.47:0/2558827455 conn(0x7f45980771c0 msgr2=0x7f45980795e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:29:07.461 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:07.461+0000 7f459e800640 1 -- 172.21.15.47:0/2558827455 shutdown_connections 2024-03-21T10:29:07.461 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:07.461+0000 7f459e800640 1 -- 172.21.15.47:0/2558827455 wait complete. 2024-03-21T10:29:07.463 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:07.462+0000 7f459e800640 1 Processor -- start 2024-03-21T10:29:07.463 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:07.462+0000 7f459e800640 1 -- start start 2024-03-21T10:29:07.463 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:07.462+0000 7f459e800640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4598107a60 0x7f459819e2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:29:07.463 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:07.462+0000 7f459e800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4598108420 0x7f459819e7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:29:07.463 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:07.462+0000 7f459e800640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f459810da70 0x7f45981a2b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:29:07.463 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:07.462+0000 7f459e800640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4598117d30 con 0x7f459810da70 2024-03-21T10:29:07.464 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:07.462+0000 7f459e800640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4598117bb0 con 0x7f4598108420 2024-03-21T10:29:07.464 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:07.462+0000 7f459e800640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4598117eb0 con 0x7f4598107a60 2024-03-21T10:29:07.464 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:07.463+0000 7f459de00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f459810da70 0x7f45981a2b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:29:07.464 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:07.463+0000 7f459de00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f459810da70 0x7f45981a2b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:52724/0 (socket says 172.21.15.47:52724) 2024-03-21T10:29:07.464 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:07.463+0000 7f459de00640 1 -- 172.21.15.47:0/2764959579 learned_addr learned my addr 172.21.15.47:0/2764959579 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:29:07.464 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:07.463+0000 7f459ca00640 1 --2- 172.21.15.47:0/2764959579 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4598108420 0x7f459819e7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:29:07.464 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:07.463+0000 7f459d400640 1 --2- 172.21.15.47:0/2764959579 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4598107a60 0x7f459819e2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:29:07.464 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:07.463+0000 7f459de00640 1 -- 172.21.15.47:0/2764959579 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4598107a60 msgr2=0x7f459819e2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:29:07.464 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:07.463+0000 7f459de00640 1 --2- 172.21.15.47:0/2764959579 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4598107a60 0x7f459819e2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:29:07.464 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:07.463+0000 7f459de00640 1 -- 172.21.15.47:0/2764959579 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4598108420 msgr2=0x7f459819e7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:29:07.464 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:07.463+0000 7f459de00640 1 --2- 172.21.15.47:0/2764959579 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4598108420 0x7f459819e7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:29:07.464 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:07.463+0000 7f459de00640 1 -- 172.21.15.47:0/2764959579 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f45981a3290 con 0x7f459810da70 2024-03-21T10:29:07.464 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:07.463+0000 7f459de00640 1 --2- 172.21.15.47:0/2764959579 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f459810da70 0x7f45981a2b60 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7f459000efd0 tx=0x7f459000c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:29:07.464 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:07.464+0000 7f4596000640 1 -- 172.21.15.47:0/2764959579 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4590003b70 con 0x7f459810da70 2024-03-21T10:29:07.464 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:07.464+0000 7f4596000640 1 -- 172.21.15.47:0/2764959579 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f4590003d10 con 0x7f459810da70 2024-03-21T10:29:07.465 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:07.464+0000 7f459e800640 1 -- 172.21.15.47:0/2764959579 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f45981a3580 con 0x7f459810da70 2024-03-21T10:29:07.465 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:07.464+0000 7f4596000640 1 -- 172.21.15.47:0/2764959579 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4590010980 con 0x7f459810da70 2024-03-21T10:29:07.465 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:07.464+0000 7f459e800640 1 -- 172.21.15.47:0/2764959579 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f45981aae10 con 0x7f459810da70 2024-03-21T10:29:07.465 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:07.465+0000 7f459e800640 1 -- 172.21.15.47:0/2764959579 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4598068800 con 0x7f459810da70 2024-03-21T10:29:07.466 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:07.467+0000 7f4596000640 1 -- 172.21.15.47:0/2764959579 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 27) v1 ==== 137985+0+0 (secure 0 0 0) 0x7f4590004020 con 0x7f459810da70 2024-03-21T10:29:07.468 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:07.469+0000 7f4596000640 1 --2- 172.21.15.47:0/2764959579 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f45800a3090 0x7f45800a5550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:29:07.469 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:07.469+0000 7f4596000640 1 -- 172.21.15.47:0/2764959579 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 9474+0+0 (secure 0 0 0) 0x7f45900d00e0 con 0x7f459810da70 2024-03-21T10:29:07.469 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:07.470+0000 7f459d400640 1 --2- 172.21.15.47:0/2764959579 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f45800a3090 0x7f45800a5550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:29:07.469 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:07.470+0000 7f459d400640 1 --2- 172.21.15.47:0/2764959579 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f45800a3090 0x7f45800a5550 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f458c005fd0 tx=0x7f458c00b510 comp rx=0 tx=0).ready entity=mgr.15687 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:29:07.473 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:07.474+0000 7f4596000640 1 -- 172.21.15.47:0/2764959579 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f459009c7b0 con 0x7f459810da70 2024-03-21T10:29:07.712 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:29:07 smithi047 ceph-mon[31547]: pgmap v117: 129 pgs: 129 active+clean; 277 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 KiB/s wr, 26 op/s 2024-03-21T10:29:07.725 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:07.726+0000 7f459e800640 1 -- 172.21.15.47:0/2764959579 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f459802d070 con 0x7f459810da70 2024-03-21T10:29:07.726 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:07.727+0000 7f4596000640 1 -- 172.21.15.47:0/2764959579 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 32 v32) v1 ==== 94+0+11471 (secure 0 0 0) 0x7f45900bae20 con 0x7f459810da70 2024-03-21T10:29:07.726 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-21T10:29:07.727 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":32,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14511,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6835/735724037","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6834","nonce":735724037},{"type":"v1","addr":"172.21.15.47:6835","nonce":735724037}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14529,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6837/2248191724","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6836","nonce":2248191724},{"type":"v1","addr":"172.21.15.47:6837","nonce":2248191724}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":32,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-21T10:21:58.329063+0000","modified":"2024-03-21T10:26:17.169892+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":34535,"mds_1":34493,"mds_2":34454,"mds_3":34427,"mds_4":14619},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14604":{"gid":14604,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.47:6839/2881335356","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6838","nonce":2881335356},{"type":"v1","addr":"172.21.15.47:6839","nonce":2881335356}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_14619":{"gid":14619,"name":"a","rank":4,"incarnation":27,"state":"up:active","state_seq":11,"addr":"172.21.15.47:6841/3401578978","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6840","nonce":3401578978},{"type":"v1","addr":"172.21.15.47:6841","nonce":3401578978}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24406":{"gid":24406,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.157:6833/4000010136","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":4000010136},{"type":"v1","addr":"172.21.15.157:6833","nonce":4000010136}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24457":{"gid":24457,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.157:6835/3902868651","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":3902868651},{"type":"v1","addr":"172.21.15.157:6835","nonce":3902868651}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24475":{"gid":24475,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.157:6837/1908087204","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6836","nonce":1908087204},{"type":"v1","addr":"172.21.15.157:6837","nonce":1908087204}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_25729":{"gid":25729,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.157:6839/1998408416","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6838","nonce":1998408416},{"type":"v1","addr":"172.21.15.157:6839","nonce":1998408416}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_34427":{"gid":34427,"name":"k","rank":3,"incarnation":25,"state":"up:active","state_seq":20,"addr":"172.21.15.100:6833/2723302038","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6832","nonce":2723302038},{"type":"v1","addr":"172.21.15.100:6833","nonce":2723302038}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_34454":{"gid":34454,"name":"e","rank":2,"incarnation":22,"state":"up:active","state_seq":17,"addr":"172.21.15.100:6835/2597849957","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":2597849957},{"type":"v1","addr":"172.21.15.100:6835","nonce":2597849957}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_34493":{"gid":34493,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":13,"addr":"172.21.15.100:6837/3497644162","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6836","nonce":3497644162},{"type":"v1","addr":"172.21.15.100:6837","nonce":3497644162}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_34535":{"gid":34535,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.100:6839/4173168440","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6838","nonce":4173168440},{"type":"v1","addr":"172.21.15.100:6839","nonce":4173168440}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":34535,"qdb_cluster":[34427,14619,34454,34535,34493]},"id":1}]} 2024-03-21T10:29:07.727 INFO:teuthology.orchestra.run.smithi047.stderr:dumped fsmap epoch 32 2024-03-21T10:29:07.730 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:07.730+0000 7f459e800640 1 -- 172.21.15.47:0/2764959579 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f45800a3090 msgr2=0x7f45800a5550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:29:07.730 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:07.730+0000 7f459e800640 1 --2- 172.21.15.47:0/2764959579 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f45800a3090 0x7f45800a5550 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f458c005fd0 tx=0x7f458c00b510 comp rx=0 tx=0).stop 2024-03-21T10:29:07.730 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:07.730+0000 7f459e800640 1 -- 172.21.15.47:0/2764959579 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f459810da70 msgr2=0x7f45981a2b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:29:07.731 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:07.730+0000 7f459e800640 1 --2- 172.21.15.47:0/2764959579 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f459810da70 0x7f45981a2b60 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7f459000efd0 tx=0x7f459000c6a0 comp rx=0 tx=0).stop 2024-03-21T10:29:07.731 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:07.730+0000 7f459e800640 1 -- 172.21.15.47:0/2764959579 shutdown_connections 2024-03-21T10:29:07.731 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:07.730+0000 7f459e800640 1 --2- 172.21.15.47:0/2764959579 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f459810da70 0x7f45981a2b60 unknown :-1 s=CLOSED pgs=762 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:29:07.731 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:07.730+0000 7f459e800640 1 --2- 172.21.15.47:0/2764959579 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f45800a3090 0x7f45800a5550 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:29:07.731 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:07.730+0000 7f459e800640 1 --2- 172.21.15.47:0/2764959579 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4598108420 0x7f459819e7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:29:07.731 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:07.730+0000 7f459e800640 1 --2- 172.21.15.47:0/2764959579 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4598107a60 0x7f459819e2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:29:07.731 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:07.730+0000 7f459e800640 1 -- 172.21.15.47:0/2764959579 >> 172.21.15.47:0/2764959579 conn(0x7f45980771c0 msgr2=0x7f4598077b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:29:07.731 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:07.730+0000 7f459e800640 1 -- 172.21.15.47:0/2764959579 shutdown_connections 2024-03-21T10:29:07.731 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:07.730+0000 7f459e800640 1 -- 172.21.15.47:0/2764959579 wait complete. 2024-03-21T10:29:07.750 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-21T10:29:07.750 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-03-21T10:29:07.977 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:07.977+0000 7ff573e00640 1 -- 172.21.15.47:0/3455651111 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff574107e50 msgr2=0x7ff574108230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:29:07.977 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:07.977+0000 7ff573e00640 1 --2- 172.21.15.47:0/3455651111 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff574107e50 0x7ff574108230 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7ff564009900 tx=0x7ff56402bc30 comp rx=0 tx=0).stop 2024-03-21T10:29:07.977 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:07.977+0000 7ff573e00640 1 -- 172.21.15.47:0/3455651111 shutdown_connections 2024-03-21T10:29:07.977 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:07.977+0000 7ff573e00640 1 --2- 172.21.15.47:0/3455651111 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff57410cea0 0x7ff574114a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:29:07.977 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:07.977+0000 7ff573e00640 1 --2- 172.21.15.47:0/3455651111 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff574108800 0x7ff57410c7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:29:07.977 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:07.977+0000 7ff573e00640 1 --2- 172.21.15.47:0/3455651111 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff574107e50 0x7ff574108230 unknown :-1 s=CLOSED pgs=763 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:29:07.977 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:07.977+0000 7ff573e00640 1 -- 172.21.15.47:0/3455651111 >> 172.21.15.47:0/3455651111 conn(0x7ff5740774c0 msgr2=0x7ff5741064c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:29:07.977 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:07.977+0000 7ff573e00640 1 -- 172.21.15.47:0/3455651111 shutdown_connections 2024-03-21T10:29:07.977 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:07.977+0000 7ff573e00640 1 -- 172.21.15.47:0/3455651111 wait complete. 2024-03-21T10:29:07.978 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:29:07 smithi100 ceph-mon[35193]: pgmap v117: 129 pgs: 129 active+clean; 277 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 KiB/s wr, 26 op/s 2024-03-21T10:29:07.980 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:07.978+0000 7ff573e00640 1 Processor -- start 2024-03-21T10:29:07.980 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:07.978+0000 7ff573e00640 1 -- start start 2024-03-21T10:29:07.980 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:07.979+0000 7ff573e00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff574107e50 0x7ff57419e310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:29:07.980 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:07.979+0000 7ff573e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff57410cea0 0x7ff57419e850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:29:07.980 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:07.979+0000 7ff573e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff5741a2bd0 0x7ff5741a3080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:29:07.981 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:07.979+0000 7ff573e00640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff574117e50 con 0x7ff57410cea0 2024-03-21T10:29:07.981 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:07.979+0000 7ff573e00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff574117cd0 con 0x7ff5741a2bd0 2024-03-21T10:29:07.981 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:07.979+0000 7ff573e00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff574117fd0 con 0x7ff574107e50 2024-03-21T10:29:07.981 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:07.979+0000 7ff56be00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff57410cea0 0x7ff57419e850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:29:07.981 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:07.979+0000 7ff56be00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff57410cea0 0x7ff57419e850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:52736/0 (socket says 172.21.15.47:52736) 2024-03-21T10:29:07.981 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:07.979+0000 7ff56be00640 1 -- 172.21.15.47:0/3885057748 learned_addr learned my addr 172.21.15.47:0/3885057748 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:29:07.981 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:07.979+0000 7ff573400640 1 --2- 172.21.15.47:0/3885057748 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff5741a2bd0 0x7ff5741a3080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:29:07.981 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:07.979+0000 7ff572a00640 1 --2- 172.21.15.47:0/3885057748 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff574107e50 0x7ff57419e310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:29:07.981 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:07.980+0000 7ff56be00640 1 -- 172.21.15.47:0/3885057748 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff574107e50 msgr2=0x7ff57419e310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:29:07.981 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:07.980+0000 7ff56be00640 1 --2- 172.21.15.47:0/3885057748 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff574107e50 0x7ff57419e310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:29:07.981 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:07.980+0000 7ff56be00640 1 -- 172.21.15.47:0/3885057748 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff5741a2bd0 msgr2=0x7ff5741a3080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:29:07.981 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:07.980+0000 7ff56be00640 1 --2- 172.21.15.47:0/3885057748 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff5741a2bd0 0x7ff5741a3080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:29:07.981 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:07.980+0000 7ff56be00640 1 -- 172.21.15.47:0/3885057748 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5741a36c0 con 0x7ff57410cea0 2024-03-21T10:29:07.982 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:07.980+0000 7ff572a00640 1 --2- 172.21.15.47:0/3885057748 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff574107e50 0x7ff57419e310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T10:29:07.982 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:07.980+0000 7ff56be00640 1 --2- 172.21.15.47:0/3885057748 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff57410cea0 0x7ff57419e850 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7ff55c00d740 tx=0x7ff55c00dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:29:07.982 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:07.980+0000 7ff56b400640 1 -- 172.21.15.47:0/3885057748 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff55c017070 con 0x7ff57410cea0 2024-03-21T10:29:07.982 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:07.980+0000 7ff56b400640 1 -- 172.21.15.47:0/3885057748 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7ff55c00de50 con 0x7ff57410cea0 2024-03-21T10:29:07.982 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:07.980+0000 7ff573e00640 1 -- 172.21.15.47:0/3885057748 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff5741a39b0 con 0x7ff57410cea0 2024-03-21T10:29:07.982 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:07.980+0000 7ff56b400640 1 -- 172.21.15.47:0/3885057748 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff55c00f620 con 0x7ff57410cea0 2024-03-21T10:29:07.982 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:07.980+0000 7ff573e00640 1 -- 172.21.15.47:0/3885057748 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff5741ab1a0 con 0x7ff57410cea0 2024-03-21T10:29:07.983 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:07.984+0000 7ff57c3f7740 1 Processor -- start 2024-03-21T10:29:07.983 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:07.984+0000 7ff57c3f7740 1 -- start start 2024-03-21T10:29:07.984 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:07.984+0000 7ff57c3f7740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ec633638b0 0x55ec63363c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:29:07.984 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:07.984+0000 7ff57c3f7740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ec63364260 0x55ec6336c810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:29:07.984 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:07.984+0000 7ff57c3f7740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ec6336cde0 0x55ec6336f230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:29:07.985 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:07.984+0000 7ff57c3f7740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ec632c72c0 con 0x55ec633638b0 2024-03-21T10:29:07.985 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:07.984+0000 7ff57c3f7740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ec6326fc90 con 0x55ec6336cde0 2024-03-21T10:29:07.985 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:07.984+0000 7ff57c3f7740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ec632c3360 con 0x55ec63364260 2024-03-21T10:29:07.985 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:07.985+0000 7ff573400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ec633638b0 0x55ec63363c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:29:07.985 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:07.985+0000 7ff573400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ec633638b0 0x55ec63363c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:52742/0 (socket says 172.21.15.47:52742) 2024-03-21T10:29:07.985 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:07.985+0000 7ff573400640 1 -- 172.21.15.47:0/1419122973 learned_addr learned my addr 172.21.15.47:0/1419122973 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:29:07.985 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:07.985+0000 7ff572a00640 1 --2- 172.21.15.47:0/1419122973 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ec63364260 0x55ec6336c810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:29:07.985 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:07.985+0000 7ff573400640 1 -- 172.21.15.47:0/1419122973 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ec63364260 msgr2=0x55ec6336c810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:29:07.985 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:07.985+0000 7ff573400640 1 --2- 172.21.15.47:0/1419122973 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ec63364260 0x55ec6336c810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:29:07.985 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:07.985+0000 7ff573400640 1 -- 172.21.15.47:0/1419122973 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ec6336cde0 msgr2=0x55ec6336f230 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-21T10:29:07.985 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:07.985+0000 7ff573400640 1 --2- 172.21.15.47:0/1419122973 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ec6336cde0 0x55ec6336f230 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:29:07.985 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:07.985+0000 7ff573400640 1 -- 172.21.15.47:0/1419122973 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ec63294080 con 0x55ec633638b0 2024-03-21T10:29:07.985 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:07.985+0000 7ff573400640 1 --2- 172.21.15.47:0/1419122973 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ec633638b0 0x55ec63363c90 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7ff56c009f40 tx=0x7ff56c0118d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ecb0d87de9ead358 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:29:07.990 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:07.985+0000 7ff56b400640 1 -- 172.21.15.47:0/3885057748 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 27) v1 ==== 137985+0+0 (secure 0 0 0) 0x7ff55c01a070 con 0x7ff57410cea0 2024-03-21T10:29:07.990 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:07.986+0000 7ff563e00640 1 -- 172.21.15.47:0/1419122973 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff56c004d20 con 0x55ec633638b0 2024-03-21T10:29:07.990 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:07.986+0000 7ff563e00640 1 -- 172.21.15.47:0/1419122973 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7ff56c004050 con 0x55ec633638b0 2024-03-21T10:29:07.990 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:07.986+0000 7ff563e00640 1 -- 172.21.15.47:0/1419122973 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff56c005220 con 0x55ec633638b0 2024-03-21T10:29:07.990 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:07.986+0000 7ff57c3f7740 1 -- 172.21.15.47:0/1419122973 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ec633638b0 msgr2=0x55ec63363c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:29:07.990 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:07.986+0000 7ff57c3f7740 1 --2- 172.21.15.47:0/1419122973 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ec633638b0 0x55ec63363c90 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7ff56c009f40 tx=0x7ff56c0118d0 comp rx=0 tx=0).stop 2024-03-21T10:29:07.990 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:07.987+0000 7ff57c3f7740 1 -- 172.21.15.47:0/1419122973 shutdown_connections 2024-03-21T10:29:07.990 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:07.987+0000 7ff57c3f7740 1 --2- 172.21.15.47:0/1419122973 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ec6336cde0 0x55ec6336f230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:29:07.990 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:07.987+0000 7ff57c3f7740 1 --2- 172.21.15.47:0/1419122973 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ec63364260 0x55ec6336c810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:29:07.990 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:07.987+0000 7ff57c3f7740 1 --2- 172.21.15.47:0/1419122973 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ec633638b0 0x55ec63363c90 unknown :-1 s=CLOSED pgs=765 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:29:07.990 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:07.987+0000 7ff57c3f7740 1 -- 172.21.15.47:0/1419122973 >> 172.21.15.47:0/1419122973 conn(0x55ec6335df80 msgr2=0x55ec6335e360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:29:07.991 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:07.987+0000 7ff57c3f7740 1 -- 172.21.15.47:0/1419122973 shutdown_connections 2024-03-21T10:29:07.991 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:07.987+0000 7ff57c3f7740 1 -- 172.21.15.47:0/1419122973 wait complete. 2024-03-21T10:29:07.991 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:07.987+0000 7ff57c3f7740 1 -- start start 2024-03-21T10:29:07.991 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:07.987+0000 7ff57c3f7740 1 Processor -- start 2024-03-21T10:29:07.991 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:07.987+0000 7ff56b400640 1 --2- 172.21.15.47:0/3885057748 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7ff5580a2fb0 0x7ff5580a5470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:29:07.991 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:07.988+0000 7ff56b400640 1 -- 172.21.15.47:0/3885057748 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 9474+0+0 (secure 0 0 0) 0x7ff55c0cf5b0 con 0x7ff57410cea0 2024-03-21T10:29:07.991 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:07.988+0000 7ff573400640 1 --2- 172.21.15.47:0/3885057748 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7ff5580a2fb0 0x7ff5580a5470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:29:07.991 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:07.988+0000 7ff57c3f7740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ec633638b0 0x55ec6336c770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:29:07.991 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:07.988+0000 7ff57c3f7740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ec63364260 0x55ec63369b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:29:07.991 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:07.988+0000 7ff57c3f7740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ec6336cde0 0x55ec6336a110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:29:07.991 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:07.988+0000 7ff57c3f7740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ec6336fd50 con 0x55ec63364260 2024-03-21T10:29:07.991 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:07.988+0000 7ff57c3f7740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ec632830d0 con 0x55ec6336cde0 2024-03-21T10:29:07.991 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:07.988+0000 7ff57c3f7740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ec6336fed0 con 0x55ec633638b0 2024-03-21T10:29:07.991 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:07.988+0000 7ff57c3f7740 10 client.0 _ll_register_callbacks cb 0x55ec63287e80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T10:29:07.992 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:07.988+0000 7ff56be00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ec63364260 0x55ec63369b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:29:07.992 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:07.988+0000 7ff56be00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ec63364260 0x55ec63369b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:52744/0 (socket says 172.21.15.47:52744) 2024-03-21T10:29:07.992 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:07.988+0000 7ff56be00640 1 -- 172.21.15.47:0/1582325646 learned_addr learned my addr 172.21.15.47:0/1582325646 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:29:07.992 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:07.989+0000 7ff573400640 1 --2- 172.21.15.47:0/1582325646 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ec6336cde0 0x55ec6336a110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:29:07.992 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:07.989+0000 7ff572a00640 1 --2- 172.21.15.47:0/1582325646 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ec633638b0 0x55ec6336c770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:29:07.992 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:07.989+0000 7ff56be00640 1 -- 172.21.15.47:0/1582325646 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ec633638b0 msgr2=0x55ec6336c770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:29:07.992 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:07.989+0000 7ff56be00640 1 --2- 172.21.15.47:0/1582325646 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ec633638b0 0x55ec6336c770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:29:07.992 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:07.989+0000 7ff56be00640 1 -- 172.21.15.47:0/1582325646 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ec6336cde0 msgr2=0x55ec6336a110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:29:07.992 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:07.989+0000 7ff56be00640 1 --2- 172.21.15.47:0/1582325646 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ec6336cde0 0x55ec6336a110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:29:07.992 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:07.989+0000 7ff56be00640 1 -- 172.21.15.47:0/1582325646 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55ec62fdb3f0 con 0x55ec63364260 2024-03-21T10:29:07.992 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:07.989+0000 7ff572a00640 1 --2- 172.21.15.47:0/1582325646 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ec633638b0 0x55ec6336c770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T10:29:07.992 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:07.989+0000 7ff573400640 1 --2- 172.21.15.47:0/3885057748 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7ff5580a2fb0 0x7ff5580a5470 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7ff56c01b040 tx=0x7ff56c000f40 comp rx=0 tx=0).ready entity=mgr.15687 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:29:07.992 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:07.989+0000 7ff56be00640 1 --2- 172.21.15.47:0/1582325646 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ec63364260 0x55ec63369b00 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7ff55c0cc9a0 tx=0x7ff55c009c30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5e2f4f456f3f238 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:29:07.992 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:07.989+0000 7ff560c00640 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-03-21T10:29:07.993 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:07.989+0000 7ff560c00640 1 -- 172.21.15.47:0/1582325646 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff55c021050 con 0x55ec63364260 2024-03-21T10:29:07.993 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:07.990+0000 7ff560c00640 1 -- 172.21.15.47:0/1582325646 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(97..97 src has 1..97) v4 ==== 9474+0+0 (secure 0 0 0) 0x7ff55c0cf1d0 con 0x55ec63364260 2024-03-21T10:29:07.993 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:07.990+0000 7ff57c3f7740 1 -- 172.21.15.47:0/1582325646 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ff56c004050 con 0x55ec63364260 2024-03-21T10:29:07.993 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:07.990+0000 7ff560c00640 1 -- 172.21.15.47:0/1582325646 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff55c0cf410 con 0x55ec63364260 2024-03-21T10:29:07.993 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:07.990+0000 7ff57c3f7740 10 client.16857 fetch_fsmap learned FSMap version 32 2024-03-21T10:29:07.993 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:07.990+0000 7ff57c3f7740 1 -- 172.21.15.47:0/1582325646 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x55ec630a57d0 con 0x55ec63364260 2024-03-21T10:29:07.993 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:07.990+0000 7ff560c00640 1 -- 172.21.15.47:0/1582325646 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7ff56c004050 con 0x55ec63364260 2024-03-21T10:29:07.993 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:07.990+0000 7ff560c00640 1 -- 172.21.15.47:0/1582325646 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 32) v1 ==== 7942+0+0 (secure 0 0 0) 0x7ff55c084bb0 con 0x55ec63364260 2024-03-21T10:29:07.993 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:07.990+0000 7ff57c3f7740 10 client.16857 fetch_fsmap finished waiting for FSMap version 32 2024-03-21T10:29:07.993 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:07.990+0000 7ff57c3f7740 10 client.16857 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.a 2024-03-21T10:29:07.993 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:07.991+0000 7ff57c3f7740 20 client.16857 populate_metadata read hostname 'smithi047' 2024-03-21T10:29:07.993 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:07.991+0000 7ff57c3f7740 1 --2- 172.21.15.47:0/1582325646 >> [v2:172.21.15.47:6840/3401578978,v1:172.21.15.47:6841/3401578978] conn(0x55ec6337cca0 0x55ec6337f090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:29:07.993 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:07.991+0000 7ff57c3f7740 4 client.16857 mds_command: new command op to 14619 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T10:29:07.993 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:07.991+0000 7ff57c3f7740 1 -- 172.21.15.47:0/1582325646 --> [v2:172.21.15.47:6840/3401578978,v1:172.21.15.47:6841/3401578978] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55ec632d4800 con 0x55ec6337cca0 2024-03-21T10:29:07.994 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:29:07 smithi157 ceph-mon[35581]: pgmap v117: 129 pgs: 129 active+clean; 277 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 KiB/s wr, 26 op/s 2024-03-21T10:29:07.995 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:07.991+0000 7ff572a00640 1 --2- 172.21.15.47:0/1582325646 >> [v2:172.21.15.47:6840/3401578978,v1:172.21.15.47:6841/3401578978] conn(0x55ec6337cca0 0x55ec6337f090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:29:07.995 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:07.991+0000 7ff572a00640 1 --2- 172.21.15.47:0/1582325646 >> [v2:172.21.15.47:6840/3401578978,v1:172.21.15.47:6841/3401578978] conn(0x55ec6337cca0 0x55ec6337f090 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=32ae8343739e334c server_cookie=24eb8e1935ce8765 in_seq=0 out_seq=0 2024-03-21T10:29:07.995 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:07.991+0000 7ff560c00640 10 client.16857 ms_handle_connect on v2:172.21.15.47:6840/3401578978 2024-03-21T10:29:07.995 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:07.992+0000 7ff560c00640 1 -- 172.21.15.47:0/1582325646 <== mds.4 v2:172.21.15.47:6840/3401578978 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55ec632d4800 con 0x55ec6337cca0 2024-03-21T10:29:07.996 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:07.992+0000 7ff560c00640 10 client.16857 handle_command_reply: tid=0 2024-03-21T10:29:07.996 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:07.992+0000 7ff57c3f7740 1 client.16857 shutdown 2024-03-21T10:29:07.996 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:07.992+0000 7ff57c3f7740 1 -- 172.21.15.47:0/1582325646 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ec63364260 msgr2=0x55ec63369b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:29:07.996 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:07.992+0000 7ff57c3f7740 1 --2- 172.21.15.47:0/1582325646 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ec63364260 0x55ec63369b00 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7ff55c0cc9a0 tx=0x7ff55c009c30 comp rx=0 tx=0).stop 2024-03-21T10:29:07.996 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:07.993+0000 7ff57c3f7740 1 -- 172.21.15.47:0/1582325646 shutdown_connections 2024-03-21T10:29:07.996 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:07.993+0000 7ff57c3f7740 1 --2- 172.21.15.47:0/1582325646 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ec6336cde0 0x55ec6336a110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:29:07.996 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:07.993+0000 7ff57c3f7740 1 --2- 172.21.15.47:0/1582325646 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ec63364260 0x55ec63369b00 unknown :-1 s=CLOSED pgs=766 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:29:07.996 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:07.993+0000 7ff57c3f7740 1 --2- 172.21.15.47:0/1582325646 >> [v2:172.21.15.47:6840/3401578978,v1:172.21.15.47:6841/3401578978] conn(0x55ec6337cca0 0x55ec6337f090 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:29:07.996 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:07.993+0000 7ff57c3f7740 1 --2- 172.21.15.47:0/1582325646 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ec633638b0 0x55ec6336c770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:29:07.996 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:07.993+0000 7ff57c3f7740 1 -- 172.21.15.47:0/1582325646 >> 172.21.15.47:0/1582325646 conn(0x55ec6335df80 msgr2=0x55ec6335fcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:29:07.996 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:07.993+0000 7ff560c00640 0 client.16857 ms_handle_reset on v2:172.21.15.47:6840/3401578978 2024-03-21T10:29:07.996 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:07.993+0000 7ff57c3f7740 1 -- 172.21.15.47:0/1582325646 shutdown_connections 2024-03-21T10:29:07.996 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:07.993+0000 7ff57c3f7740 1 -- 172.21.15.47:0/1582325646 wait complete. 2024-03-21T10:29:07.996 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:07.993+0000 7ff57c3f7740 20 client.16857 trim_cache size 0 max 16384 2024-03-21T10:29:08.007 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:08.007+0000 7ff57c3f7740 1 Processor -- start 2024-03-21T10:29:08.007 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:08.007+0000 7ff57c3f7740 1 -- start start 2024-03-21T10:29:08.007 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:08.007+0000 7ff57c3f7740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ec633638b0 0x55ec6336c230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:29:08.007 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:08.007+0000 7ff57c3f7740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ec63364260 0x55ec6336c770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:29:08.007 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:08.007+0000 7ff57c3f7740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ec6336cde0 0x55ec63366d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:29:08.007 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:08.007+0000 7ff57c3f7740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5741b4d80 con 0x55ec6336cde0 2024-03-21T10:29:08.007 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:08.007+0000 7ff57c3f7740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5741b6700 con 0x55ec63364260 2024-03-21T10:29:08.007 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:08.007+0000 7ff57c3f7740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5741b53e0 con 0x55ec633638b0 2024-03-21T10:29:08.008 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:08.007+0000 7ff573400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ec6336cde0 0x55ec63366d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:29:08.008 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:08.007+0000 7ff573400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ec6336cde0 0x55ec63366d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:52750/0 (socket says 172.21.15.47:52750) 2024-03-21T10:29:08.008 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:08.007+0000 7ff573400640 1 -- 172.21.15.47:0/63698997 learned_addr learned my addr 172.21.15.47:0/63698997 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:29:08.008 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:08.007+0000 7ff56be00640 1 --2- 172.21.15.47:0/63698997 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ec63364260 0x55ec6336c770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:29:08.008 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:08.008+0000 7ff572a00640 1 --2- 172.21.15.47:0/63698997 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ec633638b0 0x55ec6336c230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:29:08.008 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:08.008+0000 7ff573400640 1 -- 172.21.15.47:0/63698997 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ec633638b0 msgr2=0x55ec6336c230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:29:08.008 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:08.008+0000 7ff573400640 1 --2- 172.21.15.47:0/63698997 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ec633638b0 0x55ec6336c230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:29:08.008 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:08.008+0000 7ff573400640 1 -- 172.21.15.47:0/63698997 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ec63364260 msgr2=0x55ec6336c770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:29:08.008 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:08.008+0000 7ff573400640 1 --2- 172.21.15.47:0/63698997 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ec63364260 0x55ec6336c770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:29:08.008 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:08.008+0000 7ff573400640 1 -- 172.21.15.47:0/63698997 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ec6337f400 con 0x55ec6336cde0 2024-03-21T10:29:08.010 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:08.008+0000 7ff573400640 1 --2- 172.21.15.47:0/63698997 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ec6336cde0 0x55ec63366d80 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7ff56c009f40 tx=0x7ff56c027000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ca634a49cfba75d5 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:29:08.011 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:08.009+0000 7ff563400640 1 -- 172.21.15.47:0/63698997 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff56c003ca0 con 0x55ec6336cde0 2024-03-21T10:29:08.011 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:08.009+0000 7ff563400640 1 -- 172.21.15.47:0/63698997 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7ff56c025900 con 0x55ec6336cde0 2024-03-21T10:29:08.011 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:08.009+0000 7ff563400640 1 -- 172.21.15.47:0/63698997 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff56c00f7e0 con 0x55ec6336cde0 2024-03-21T10:29:08.011 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:08.009+0000 7ff57c3f7740 1 -- 172.21.15.47:0/63698997 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ec6336cde0 msgr2=0x55ec63366d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:29:08.011 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:08.009+0000 7ff57c3f7740 1 --2- 172.21.15.47:0/63698997 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ec6336cde0 0x55ec63366d80 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7ff56c009f40 tx=0x7ff56c027000 comp rx=0 tx=0).stop 2024-03-21T10:29:08.011 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:08.009+0000 7ff57c3f7740 1 -- 172.21.15.47:0/63698997 shutdown_connections 2024-03-21T10:29:08.011 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:08.009+0000 7ff57c3f7740 1 --2- 172.21.15.47:0/63698997 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ec6336cde0 0x55ec63366d80 unknown :-1 s=CLOSED pgs=767 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:29:08.011 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:08.009+0000 7ff57c3f7740 1 --2- 172.21.15.47:0/63698997 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ec63364260 0x55ec6336c770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:29:08.011 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:08.009+0000 7ff57c3f7740 1 --2- 172.21.15.47:0/63698997 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ec633638b0 0x55ec6336c230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:29:08.011 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:08.009+0000 7ff57c3f7740 1 -- 172.21.15.47:0/63698997 >> 172.21.15.47:0/63698997 conn(0x55ec6335df80 msgr2=0x55ec6335d160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:29:08.011 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:08.009+0000 7ff57c3f7740 1 -- 172.21.15.47:0/63698997 shutdown_connections 2024-03-21T10:29:08.011 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:08.009+0000 7ff57c3f7740 1 -- 172.21.15.47:0/63698997 wait complete. 2024-03-21T10:29:08.012 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:08.010+0000 7ff57c3f7740 1 -- start start 2024-03-21T10:29:08.012 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:08.010+0000 7ff57c3f7740 1 Processor -- start 2024-03-21T10:29:08.012 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:08.010+0000 7ff57c3f7740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ec633638b0 0x55ec633738d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:29:08.012 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:08.010+0000 7ff57c3f7740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ec63364260 0x55ec63373e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:29:08.012 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:08.011+0000 7ff57c3f7740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ec6336cde0 0x55ec633d8480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:29:08.012 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:08.011+0000 7ff57c3f7740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5741b60a0 con 0x55ec6336cde0 2024-03-21T10:29:08.012 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:08.011+0000 7ff57c3f7740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5741b5a40 con 0x55ec63364260 2024-03-21T10:29:08.012 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:08.011+0000 7ff57c3f7740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5741b2f40 con 0x55ec633638b0 2024-03-21T10:29:08.012 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:08.011+0000 7ff57c3f7740 10 client.0 _ll_register_callbacks cb 0x7ff55c010b00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T10:29:08.012 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:08.011+0000 7ff573400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ec6336cde0 0x55ec633d8480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:29:08.012 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:08.011+0000 7ff573400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ec6336cde0 0x55ec633d8480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:52758/0 (socket says 172.21.15.47:52758) 2024-03-21T10:29:08.012 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:08.011+0000 7ff573400640 1 -- 172.21.15.47:0/3522892841 learned_addr learned my addr 172.21.15.47:0/3522892841 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:29:08.012 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:08.011+0000 7ff56be00640 1 --2- 172.21.15.47:0/3522892841 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ec63364260 0x55ec63373e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:29:08.012 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:08.011+0000 7ff572a00640 1 --2- 172.21.15.47:0/3522892841 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ec633638b0 0x55ec633738d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:29:08.013 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:08.011+0000 7ff573400640 1 -- 172.21.15.47:0/3522892841 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ec633638b0 msgr2=0x55ec633738d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:29:08.013 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:08.011+0000 7ff573400640 1 --2- 172.21.15.47:0/3522892841 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ec633638b0 0x55ec633738d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:29:08.013 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:08.011+0000 7ff573400640 1 -- 172.21.15.47:0/3522892841 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ec63364260 msgr2=0x55ec63373e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:29:08.013 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:08.011+0000 7ff573400640 1 --2- 172.21.15.47:0/3522892841 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ec63364260 0x55ec63373e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:29:08.013 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:08.011+0000 7ff573400640 1 -- 172.21.15.47:0/3522892841 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55ec633d89c0 con 0x55ec6336cde0 2024-03-21T10:29:08.015 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:08.012+0000 7ff573400640 1 --2- 172.21.15.47:0/3522892841 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ec6336cde0 0x55ec633d8480 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7ff56c003e50 tx=0x7ff56c009480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ecbdd7b91447cb73 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:29:08.015 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:08.012+0000 7ff560c00640 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-03-21T10:29:08.015 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:08.012+0000 7ff560c00640 1 -- 172.21.15.47:0/3522892841 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff56c03e030 con 0x55ec6336cde0 2024-03-21T10:29:08.015 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:08.012+0000 7ff57c3f7740 1 -- 172.21.15.47:0/3522892841 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ff56c025900 con 0x55ec6336cde0 2024-03-21T10:29:08.015 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:08.012+0000 7ff560c00640 1 -- 172.21.15.47:0/3522892841 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(97..97 src has 1..97) v4 ==== 9474+0+0 (secure 0 0 0) 0x7ff56c025ab0 con 0x55ec6336cde0 2024-03-21T10:29:08.016 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:08.012+0000 7ff560c00640 1 -- 172.21.15.47:0/3522892841 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff56c04b050 con 0x55ec6336cde0 2024-03-21T10:29:08.016 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:08.013+0000 7ff57c3f7740 10 client.16863 fetch_fsmap learned FSMap version 32 2024-03-21T10:29:08.016 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:08.013+0000 7ff57c3f7740 1 -- 172.21.15.47:0/3522892841 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x55ec633d9af0 con 0x55ec6336cde0 2024-03-21T10:29:08.016 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:08.013+0000 7ff560c00640 1 -- 172.21.15.47:0/3522892841 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7ff56c025900 con 0x55ec6336cde0 2024-03-21T10:29:08.016 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:08.013+0000 7ff560c00640 1 -- 172.21.15.47:0/3522892841 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 32) v1 ==== 7942+0+0 (secure 0 0 0) 0x7ff56c023430 con 0x55ec6336cde0 2024-03-21T10:29:08.016 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:08.013+0000 7ff57c3f7740 10 client.16863 fetch_fsmap finished waiting for FSMap version 32 2024-03-21T10:29:08.016 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:08.013+0000 7ff57c3f7740 10 client.16863 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.a 2024-03-21T10:29:08.016 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:08.013+0000 7ff57c3f7740 20 client.16863 populate_metadata read hostname 'smithi047' 2024-03-21T10:29:08.016 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:08.013+0000 7ff57c3f7740 1 --2- 172.21.15.47:0/3522892841 >> [v2:172.21.15.47:6840/3401578978,v1:172.21.15.47:6841/3401578978] conn(0x55ec633d9f80 0x55ec633dc3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:29:08.016 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:08.013+0000 7ff57c3f7740 4 client.16863 mds_command: new command op to 14619 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-21T10:29:08.016 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:08.013+0000 7ff57c3f7740 1 -- 172.21.15.47:0/3522892841 --> [v2:172.21.15.47:6840/3401578978,v1:172.21.15.47:6841/3401578978] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55ec632ceb50 con 0x55ec633d9f80 2024-03-21T10:29:08.016 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:08.013+0000 7ff572a00640 1 --2- 172.21.15.47:0/3522892841 >> [v2:172.21.15.47:6840/3401578978,v1:172.21.15.47:6841/3401578978] conn(0x55ec633d9f80 0x55ec633dc3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:29:08.016 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:08.014+0000 7ff572a00640 1 --2- 172.21.15.47:0/3522892841 >> [v2:172.21.15.47:6840/3401578978,v1:172.21.15.47:6841/3401578978] conn(0x55ec633d9f80 0x55ec633dc3a0 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=226267734b6266cf server_cookie=7fa0913a0ebcb592 in_seq=0 out_seq=0 2024-03-21T10:29:08.017 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:08.014+0000 7ff560c00640 10 client.16863 ms_handle_connect on v2:172.21.15.47:6840/3401578978 2024-03-21T10:29:08.017 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:08.014+0000 7ff560c00640 1 -- 172.21.15.47:0/3522892841 <== mds.4 v2:172.21.15.47:6840/3401578978 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55ec632ceb50 con 0x55ec633d9f80 2024-03-21T10:29:08.017 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:08.014+0000 7ff560c00640 10 client.16863 handle_command_reply: tid=0 2024-03-21T10:29:08.017 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:08.014+0000 7ff57c3f7740 1 client.16863 shutdown 2024-03-21T10:29:08.017 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:08.014+0000 7ff57c3f7740 1 -- 172.21.15.47:0/3522892841 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ec6336cde0 msgr2=0x55ec633d8480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:29:08.017 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:08.014+0000 7ff57c3f7740 1 --2- 172.21.15.47:0/3522892841 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ec6336cde0 0x55ec633d8480 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7ff56c003e50 tx=0x7ff56c009480 comp rx=0 tx=0).stop 2024-03-21T10:29:08.017 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:08.015+0000 7ff57c3f7740 1 -- 172.21.15.47:0/3522892841 shutdown_connections 2024-03-21T10:29:08.017 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:08.015+0000 7ff57c3f7740 1 --2- 172.21.15.47:0/3522892841 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ec6336cde0 0x55ec633d8480 unknown :-1 s=CLOSED pgs=768 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:29:08.017 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:08.015+0000 7ff57c3f7740 1 --2- 172.21.15.47:0/3522892841 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ec63364260 0x55ec63373e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:29:08.017 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:08.015+0000 7ff57c3f7740 1 --2- 172.21.15.47:0/3522892841 >> [v2:172.21.15.47:6840/3401578978,v1:172.21.15.47:6841/3401578978] conn(0x55ec633d9f80 0x55ec633dc3a0 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:29:08.017 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:08.015+0000 7ff57c3f7740 1 --2- 172.21.15.47:0/3522892841 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ec633638b0 0x55ec633738d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:29:08.017 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:08.015+0000 7ff57c3f7740 1 -- 172.21.15.47:0/3522892841 >> 172.21.15.47:0/3522892841 conn(0x55ec6335df80 msgr2=0x55ec6335f3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:29:08.017 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:08.015+0000 7ff560c00640 0 client.16863 ms_handle_reset on v2:172.21.15.47:6840/3401578978 2024-03-21T10:29:08.017 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:08.015+0000 7ff57c3f7740 1 -- 172.21.15.47:0/3522892841 shutdown_connections 2024-03-21T10:29:08.018 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:08.015+0000 7ff57c3f7740 1 -- 172.21.15.47:0/3522892841 wait complete. 2024-03-21T10:29:08.018 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:08.015+0000 7ff57c3f7740 20 client.16863 trim_cache size 0 max 16384 2024-03-21T10:29:08.018 INFO:teuthology.orchestra.run.smithi047.stdout:[] 2024-03-21T10:29:08.019 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:08.015+0000 7ff573e00640 1 -- 172.21.15.47:0/3885057748 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7ff5580a2fb0 msgr2=0x7ff5580a5470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:29:08.019 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:08.015+0000 7ff573e00640 1 --2- 172.21.15.47:0/3885057748 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7ff5580a2fb0 0x7ff5580a5470 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7ff56c01b040 tx=0x7ff56c000f40 comp rx=0 tx=0).stop 2024-03-21T10:29:08.019 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:08.015+0000 7ff573e00640 1 -- 172.21.15.47:0/3885057748 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff57410cea0 msgr2=0x7ff57419e850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:29:08.019 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:08.015+0000 7ff573e00640 1 --2- 172.21.15.47:0/3885057748 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff57410cea0 0x7ff57419e850 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7ff55c00d740 tx=0x7ff55c00dc00 comp rx=0 tx=0).stop 2024-03-21T10:29:08.019 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:08.016+0000 7ff573e00640 1 -- 172.21.15.47:0/3885057748 shutdown_connections 2024-03-21T10:29:08.019 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:08.016+0000 7ff573e00640 1 --2- 172.21.15.47:0/3885057748 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7ff5580a2fb0 0x7ff5580a5470 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:29:08.019 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:08.016+0000 7ff573e00640 1 --2- 172.21.15.47:0/3885057748 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff5741a2bd0 0x7ff5741a3080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:29:08.019 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:08.016+0000 7ff573e00640 1 --2- 172.21.15.47:0/3885057748 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff57410cea0 0x7ff57419e850 unknown :-1 s=CLOSED pgs=764 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:29:08.019 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:08.016+0000 7ff573e00640 1 --2- 172.21.15.47:0/3885057748 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff574107e50 0x7ff57419e310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:29:08.020 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:08.016+0000 7ff573e00640 1 -- 172.21.15.47:0/3885057748 >> 172.21.15.47:0/3885057748 conn(0x7ff5740774c0 msgr2=0x7ff574077b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:29:08.020 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:08.016+0000 7ff573e00640 1 -- 172.21.15.47:0/3885057748 shutdown_connections 2024-03-21T10:29:08.020 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:08.016+0000 7ff573e00640 1 -- 172.21.15.47:0/3885057748 wait complete. 2024-03-21T10:29:08.032 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-21T10:29:08.033 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-03-21T10:29:08.285 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:08.285+0000 7faafe200640 1 -- 172.21.15.47:0/1964021428 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7faaf8102ff0 msgr2=0x7faaf810b4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:29:08.285 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:08.285+0000 7faafe200640 1 --2- 172.21.15.47:0/1964021428 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7faaf8102ff0 0x7faaf810b4d0 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7faae4009900 tx=0x7faae402bcd0 comp rx=0 tx=0).stop 2024-03-21T10:29:08.285 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:08.285+0000 7faafe200640 1 -- 172.21.15.47:0/1964021428 shutdown_connections 2024-03-21T10:29:08.285 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:08.285+0000 7faafe200640 1 --2- 172.21.15.47:0/1964021428 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7faaf810baf0 0x7faaf810df90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:29:08.285 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:08.285+0000 7faafe200640 1 --2- 172.21.15.47:0/1964021428 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7faaf8102ff0 0x7faaf810b4d0 unknown :-1 s=CLOSED pgs=769 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:29:08.285 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:08.285+0000 7faafe200640 1 --2- 172.21.15.47:0/1964021428 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7faaf81026d0 0x7faaf8102ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:29:08.285 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:08.285+0000 7faafe200640 1 -- 172.21.15.47:0/1964021428 >> 172.21.15.47:0/1964021428 conn(0x7faaf80fbef0 msgr2=0x7faaf80fe310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:29:08.285 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:08.285+0000 7faafe200640 1 -- 172.21.15.47:0/1964021428 shutdown_connections 2024-03-21T10:29:08.286 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:08.285+0000 7faafe200640 1 -- 172.21.15.47:0/1964021428 wait complete. 2024-03-21T10:29:08.288 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:08.286+0000 7faafe200640 1 Processor -- start 2024-03-21T10:29:08.288 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:08.286+0000 7faafe200640 1 -- start start 2024-03-21T10:29:08.288 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:08.286+0000 7faafe200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7faaf81026d0 0x7faaf8100320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:29:08.288 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:08.286+0000 7faafe200640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7faaf8102ff0 0x7faaf8100860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:29:08.288 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:08.286+0000 7faafe200640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7faaf810baf0 0x7faaf8100da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:29:08.288 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:08.286+0000 7faafe200640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faaf8111760 con 0x7faaf8102ff0 2024-03-21T10:29:08.288 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:08.286+0000 7faafe200640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faaf81115e0 con 0x7faaf81026d0 2024-03-21T10:29:08.288 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:08.286+0000 7faafe200640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faaf81118e0 con 0x7faaf810baf0 2024-03-21T10:29:08.288 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:08.287+0000 7faaf7e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7faaf8102ff0 0x7faaf8100860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:29:08.288 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:08.287+0000 7faaf7e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7faaf8102ff0 0x7faaf8100860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:52768/0 (socket says 172.21.15.47:52768) 2024-03-21T10:29:08.288 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:08.287+0000 7faaf7e00640 1 -- 172.21.15.47:0/1267510802 learned_addr learned my addr 172.21.15.47:0/1267510802 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:29:08.288 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:08.287+0000 7faafce00640 1 --2- 172.21.15.47:0/1267510802 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7faaf81026d0 0x7faaf8100320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:29:08.289 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:08.287+0000 7faafd800640 1 --2- 172.21.15.47:0/1267510802 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7faaf810baf0 0x7faaf8100da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:29:08.289 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:08.287+0000 7faaf7e00640 1 -- 172.21.15.47:0/1267510802 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7faaf810baf0 msgr2=0x7faaf8100da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:29:08.289 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:08.287+0000 7faaf7e00640 1 --2- 172.21.15.47:0/1267510802 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7faaf810baf0 0x7faaf8100da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:29:08.289 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:08.287+0000 7faaf7e00640 1 -- 172.21.15.47:0/1267510802 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7faaf81026d0 msgr2=0x7faaf8100320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:29:08.289 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:08.287+0000 7faaf7e00640 1 --2- 172.21.15.47:0/1267510802 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7faaf81026d0 0x7faaf8100320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:29:08.289 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:08.287+0000 7faaf7e00640 1 -- 172.21.15.47:0/1267510802 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faaf81a6200 con 0x7faaf8102ff0 2024-03-21T10:29:08.289 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:08.287+0000 7faafd800640 1 --2- 172.21.15.47:0/1267510802 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7faaf810baf0 0x7faaf8100da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T10:29:08.289 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:08.288+0000 7faaf7e00640 1 --2- 172.21.15.47:0/1267510802 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7faaf8102ff0 0x7faaf8100860 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7faae4005940 tx=0x7faae402c690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:29:08.289 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:08.288+0000 7faaf5600640 1 -- 172.21.15.47:0/1267510802 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faae4043020 con 0x7faaf8102ff0 2024-03-21T10:29:08.289 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:08.288+0000 7faaf5600640 1 -- 172.21.15.47:0/1267510802 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7faae4004e00 con 0x7faaf8102ff0 2024-03-21T10:29:08.289 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:08.288+0000 7faafe200640 1 -- 172.21.15.47:0/1267510802 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faaf81a6490 con 0x7faaf8102ff0 2024-03-21T10:29:08.289 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:08.288+0000 7faaf5600640 1 -- 172.21.15.47:0/1267510802 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faae4003970 con 0x7faaf8102ff0 2024-03-21T10:29:08.289 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:08.288+0000 7faafe200640 1 -- 172.21.15.47:0/1267510802 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faaf81a6a30 con 0x7faaf8102ff0 2024-03-21T10:29:08.291 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:08.292+0000 7fab02578740 1 Processor -- start 2024-03-21T10:29:08.291 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:08.292+0000 7fab02578740 1 -- start start 2024-03-21T10:29:08.292 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:08.292+0000 7fab02578740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d8623ff8b0 0x55d8623ffc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:29:08.292 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:08.292+0000 7fab02578740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d862400260 0x55d862408810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:29:08.293 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:08.292+0000 7fab02578740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d862408de0 0x55d86240b230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:29:08.293 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:08.292+0000 7fab02578740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d8623632c0 con 0x55d862408de0 2024-03-21T10:29:08.293 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:08.292+0000 7fab02578740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d86230bc90 con 0x55d862400260 2024-03-21T10:29:08.293 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:08.292+0000 7fab02578740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d86235f360 con 0x55d8623ff8b0 2024-03-21T10:29:08.293 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:08.293+0000 7faafce00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d862400260 0x55d862408810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:29:08.293 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:08.293+0000 7faafce00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d862400260 0x55d862408810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:33108/0 (socket says 172.21.15.47:33108) 2024-03-21T10:29:08.293 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:08.293+0000 7faafce00640 1 -- 172.21.15.47:0/3594221530 learned_addr learned my addr 172.21.15.47:0/3594221530 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:29:08.293 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:08.293+0000 7faafd800640 1 --2- 172.21.15.47:0/3594221530 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d8623ff8b0 0x55d8623ffc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:29:08.293 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:08.293+0000 7faafce00640 1 -- 172.21.15.47:0/3594221530 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d8623ff8b0 msgr2=0x55d8623ffc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:29:08.293 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:08.293+0000 7faafce00640 1 --2- 172.21.15.47:0/3594221530 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d8623ff8b0 0x55d8623ffc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:29:08.293 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:08.293+0000 7faafce00640 1 -- 172.21.15.47:0/3594221530 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d862408de0 msgr2=0x55d86240b230 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-21T10:29:08.293 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:08.293+0000 7faafce00640 1 --2- 172.21.15.47:0/3594221530 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d862408de0 0x55d86240b230 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:29:08.293 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:08.293+0000 7faafce00640 1 -- 172.21.15.47:0/3594221530 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d862330080 con 0x55d862400260 2024-03-21T10:29:08.295 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:08.294+0000 7faaf5600640 1 -- 172.21.15.47:0/1267510802 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 27) v1 ==== 137985+0+0 (secure 0 0 0) 0x7faae4046020 con 0x7faaf8102ff0 2024-03-21T10:29:08.295 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:08.294+0000 7faafce00640 1 --2- 172.21.15.47:0/3594221530 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d862400260 0x55d862408810 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7faaec00bd10 tx=0x7faaec0107a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b29856179419adad server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:29:08.295 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:08.294+0000 7faaea000640 1 -- 172.21.15.47:0/3594221530 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faaec004030 con 0x55d862400260 2024-03-21T10:29:08.295 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:08.294+0000 7faaea000640 1 -- 172.21.15.47:0/3594221530 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7faaec0035f0 con 0x55d862400260 2024-03-21T10:29:08.295 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:08.294+0000 7faaea000640 1 -- 172.21.15.47:0/3594221530 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faaec011770 con 0x55d862400260 2024-03-21T10:29:08.295 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:08.295+0000 7faaf5600640 1 --2- 172.21.15.47:0/1267510802 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7faae00a3110 0x7faae00a55d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:29:08.295 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:08.295+0000 7fab02578740 1 -- 172.21.15.47:0/3594221530 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d862400260 msgr2=0x55d862408810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:29:08.296 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:08.295+0000 7fab02578740 1 --2- 172.21.15.47:0/3594221530 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d862400260 0x55d862408810 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7faaec00bd10 tx=0x7faaec0107a0 comp rx=0 tx=0).stop 2024-03-21T10:29:08.296 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:08.295+0000 7faaf5600640 1 -- 172.21.15.47:0/1267510802 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 9474+0+0 (secure 0 0 0) 0x7faae40f1a20 con 0x7faaf8102ff0 2024-03-21T10:29:08.296 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:08.295+0000 7fab02578740 1 -- 172.21.15.47:0/3594221530 shutdown_connections 2024-03-21T10:29:08.296 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:08.295+0000 7fab02578740 1 --2- 172.21.15.47:0/3594221530 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d862408de0 0x55d86240b230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:29:08.296 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:08.295+0000 7fab02578740 1 --2- 172.21.15.47:0/3594221530 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d862400260 0x55d862408810 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:29:08.296 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:08.295+0000 7fab02578740 1 --2- 172.21.15.47:0/3594221530 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d8623ff8b0 0x55d8623ffc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:29:08.296 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:08.295+0000 7fab02578740 1 -- 172.21.15.47:0/3594221530 >> 172.21.15.47:0/3594221530 conn(0x55d8623f9f80 msgr2=0x55d8623fa360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:29:08.296 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:08.295+0000 7faafd800640 1 --2- 172.21.15.47:0/1267510802 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7faae00a3110 0x7faae00a55d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:29:08.296 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:08.296+0000 7fab02578740 1 -- 172.21.15.47:0/3594221530 shutdown_connections 2024-03-21T10:29:08.296 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:08.296+0000 7fab02578740 1 -- 172.21.15.47:0/3594221530 wait complete. 2024-03-21T10:29:08.301 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:08.296+0000 7faafd800640 1 --2- 172.21.15.47:0/1267510802 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7faae00a3110 0x7faae00a55d0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7faaf0005fd0 tx=0x7faaf0009500 comp rx=0 tx=0).ready entity=mgr.15687 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:29:08.301 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:08.296+0000 7fab02578740 1 -- start start 2024-03-21T10:29:08.302 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:08.297+0000 7fab02578740 1 Processor -- start 2024-03-21T10:29:08.302 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:08.297+0000 7fab02578740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d8623ff8b0 0x55d8624086f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:29:08.302 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:08.297+0000 7fab02578740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d862400260 0x55d862405a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:29:08.302 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:08.297+0000 7fab02578740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d862408de0 0x55d862406020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:29:08.302 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:08.297+0000 7fab02578740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d86240bd50 con 0x55d862400260 2024-03-21T10:29:08.302 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:08.297+0000 7fab02578740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d86231f0d0 con 0x55d8623ff8b0 2024-03-21T10:29:08.302 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:08.297+0000 7fab02578740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d86240bed0 con 0x55d862408de0 2024-03-21T10:29:08.302 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:08.298+0000 7fab02578740 10 client.0 _ll_register_callbacks cb 0x55d862323e80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T10:29:08.302 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:08.298+0000 7faaf7e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d862400260 0x55d862405a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:29:08.302 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:08.298+0000 7faaf7e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d862400260 0x55d862405a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:52784/0 (socket says 172.21.15.47:52784) 2024-03-21T10:29:08.302 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:08.298+0000 7faaf7e00640 1 -- 172.21.15.47:0/4041339723 learned_addr learned my addr 172.21.15.47:0/4041339723 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:29:08.302 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:08.298+0000 7faafce00640 1 --2- 172.21.15.47:0/4041339723 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d8623ff8b0 0x55d8624086f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:29:08.303 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:08.298+0000 7faafd800640 1 --2- 172.21.15.47:0/4041339723 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d862408de0 0x55d862406020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:29:08.303 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:08.298+0000 7faaf7e00640 1 -- 172.21.15.47:0/4041339723 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d862408de0 msgr2=0x55d862406020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:29:08.303 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:08.298+0000 7faaf7e00640 1 --2- 172.21.15.47:0/4041339723 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d862408de0 0x55d862406020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:29:08.303 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:08.298+0000 7faaf7e00640 1 -- 172.21.15.47:0/4041339723 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d8623ff8b0 msgr2=0x55d8624086f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:29:08.303 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:08.298+0000 7faaf7e00640 1 --2- 172.21.15.47:0/4041339723 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d8623ff8b0 0x55d8624086f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:29:08.303 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:08.298+0000 7faaf7e00640 1 -- 172.21.15.47:0/4041339723 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55d8620773f0 con 0x55d862400260 2024-03-21T10:29:08.303 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:08.298+0000 7faafce00640 1 --2- 172.21.15.47:0/4041339723 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d8623ff8b0 0x55d8624086f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T10:29:08.303 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:08.298+0000 7faaf7e00640 1 --2- 172.21.15.47:0/4041339723 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d862400260 0x55d862405a80 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7faae40bdaf0 tx=0x7faae40be320 comp rx=0 tx=0).ready entity=mon.0 client_cookie=832b6a2ba1e3452a server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:29:08.303 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:08.298+0000 7faadea00640 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-03-21T10:29:08.303 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:08.299+0000 7faadea00640 1 -- 172.21.15.47:0/4041339723 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faae40c6570 con 0x55d862400260 2024-03-21T10:29:08.303 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:08.299+0000 7faadea00640 1 -- 172.21.15.47:0/4041339723 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(97..97 src has 1..97) v4 ==== 9474+0+0 (secure 0 0 0) 0x7faae4003b10 con 0x55d862400260 2024-03-21T10:29:08.303 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:08.299+0000 7fab02578740 1 -- 172.21.15.47:0/4041339723 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7faaec0035f0 con 0x55d862400260 2024-03-21T10:29:08.303 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:08.299+0000 7faadea00640 1 -- 172.21.15.47:0/4041339723 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7faae4004fb0 con 0x55d862400260 2024-03-21T10:29:08.303 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:08.299+0000 7fab02578740 10 client.16872 fetch_fsmap learned FSMap version 32 2024-03-21T10:29:08.304 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:08.299+0000 7fab02578740 1 -- 172.21.15.47:0/4041339723 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x55d8621417d0 con 0x55d862400260 2024-03-21T10:29:08.304 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:08.299+0000 7faadea00640 1 -- 172.21.15.47:0/4041339723 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7faaec0035f0 con 0x55d862400260 2024-03-21T10:29:08.304 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:08.299+0000 7faadea00640 1 -- 172.21.15.47:0/4041339723 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 32) v1 ==== 7942+0+0 (secure 0 0 0) 0x7faae40bcfb0 con 0x55d862400260 2024-03-21T10:29:08.304 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:08.300+0000 7fab02578740 10 client.16872 fetch_fsmap finished waiting for FSMap version 32 2024-03-21T10:29:08.304 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:08.300+0000 7fab02578740 10 client.16872 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.k 2024-03-21T10:29:08.304 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:08.300+0000 7fab02578740 20 client.16872 populate_metadata read hostname 'smithi047' 2024-03-21T10:29:08.304 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:08.300+0000 7fab02578740 1 --2- 172.21.15.47:0/4041339723 >> [v2:172.21.15.100:6832/2723302038,v1:172.21.15.100:6833/2723302038] conn(0x55d862418cb0 0x55d86241b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:29:08.304 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:08.300+0000 7fab02578740 4 client.16872 mds_command: new command op to 34427 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T10:29:08.304 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:08.300+0000 7fab02578740 1 -- 172.21.15.47:0/4041339723 --> [v2:172.21.15.100:6832/2723302038,v1:172.21.15.100:6833/2723302038] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55d862370800 con 0x55d862418cb0 2024-03-21T10:29:08.304 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:08.300+0000 7faafce00640 1 --2- 172.21.15.47:0/4041339723 >> [v2:172.21.15.100:6832/2723302038,v1:172.21.15.100:6833/2723302038] conn(0x55d862418cb0 0x55d86241b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:29:08.305 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:08.301+0000 7faafce00640 1 --2- 172.21.15.47:0/4041339723 >> [v2:172.21.15.100:6832/2723302038,v1:172.21.15.100:6833/2723302038] conn(0x55d862418cb0 0x55d86241b0a0 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=e1967ed4b86b2075 server_cookie=8a53c95ddc45a89b in_seq=0 out_seq=0 2024-03-21T10:29:08.305 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:08.301+0000 7faadea00640 10 client.16872 ms_handle_connect on v2:172.21.15.100:6832/2723302038 2024-03-21T10:29:08.306 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:08.302+0000 7faadea00640 1 -- 172.21.15.47:0/4041339723 <== mds.3 v2:172.21.15.100:6832/2723302038 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55d862370800 con 0x55d862418cb0 2024-03-21T10:29:08.306 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:08.302+0000 7faadea00640 10 client.16872 handle_command_reply: tid=0 2024-03-21T10:29:08.306 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:08.302+0000 7fab02578740 1 client.16872 shutdown 2024-03-21T10:29:08.306 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:08.303+0000 7fab02578740 1 -- 172.21.15.47:0/4041339723 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d862400260 msgr2=0x55d862405a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:29:08.306 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:08.303+0000 7fab02578740 1 --2- 172.21.15.47:0/4041339723 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d862400260 0x55d862405a80 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7faae40bdaf0 tx=0x7faae40be320 comp rx=0 tx=0).stop 2024-03-21T10:29:08.306 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:08.303+0000 7fab02578740 1 -- 172.21.15.47:0/4041339723 shutdown_connections 2024-03-21T10:29:08.306 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:08.303+0000 7fab02578740 1 --2- 172.21.15.47:0/4041339723 >> [v2:172.21.15.100:6832/2723302038,v1:172.21.15.100:6833/2723302038] conn(0x55d862418cb0 0x55d86241b0a0 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:29:08.306 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:08.303+0000 7fab02578740 1 --2- 172.21.15.47:0/4041339723 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d862408de0 0x55d862406020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:29:08.306 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:08.303+0000 7fab02578740 1 --2- 172.21.15.47:0/4041339723 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d862400260 0x55d862405a80 unknown :-1 s=CLOSED pgs=771 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:29:08.306 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:08.303+0000 7fab02578740 1 --2- 172.21.15.47:0/4041339723 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d8623ff8b0 0x55d8624086f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:29:08.306 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:08.303+0000 7faadea00640 0 client.16872 ms_handle_reset on v2:172.21.15.100:6832/2723302038 2024-03-21T10:29:08.306 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:08.303+0000 7fab02578740 1 -- 172.21.15.47:0/4041339723 >> 172.21.15.47:0/4041339723 conn(0x55d8623f9f80 msgr2=0x55d8623fbcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:29:08.306 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:08.303+0000 7fab02578740 1 -- 172.21.15.47:0/4041339723 shutdown_connections 2024-03-21T10:29:08.307 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:08.303+0000 7fab02578740 1 -- 172.21.15.47:0/4041339723 wait complete. 2024-03-21T10:29:08.307 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:08.303+0000 7fab02578740 20 client.16872 trim_cache size 0 max 16384 2024-03-21T10:29:08.317 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:08.317+0000 7fab02578740 1 Processor -- start 2024-03-21T10:29:08.317 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:08.317+0000 7fab02578740 1 -- start start 2024-03-21T10:29:08.317 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:08.318+0000 7fab02578740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d8623ff8b0 0x55d8623f4ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:29:08.317 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:08.318+0000 7fab02578740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d862400260 0x55d8624081b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:29:08.317 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:08.318+0000 7fab02578740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d862408de0 0x55d8624086f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:29:08.317 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:08.318+0000 7fab02578740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faaf81b08e0 con 0x55d862400260 2024-03-21T10:29:08.317 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:08.318+0000 7fab02578740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faaf81b2260 con 0x55d8623ff8b0 2024-03-21T10:29:08.317 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:08.318+0000 7fab02578740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faaf81b0f40 con 0x55d862408de0 2024-03-21T10:29:08.319 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:08.318+0000 7faaf7e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d862400260 0x55d8624081b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:29:08.319 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:08.318+0000 7faafce00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d8623ff8b0 0x55d8623f4ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:29:08.319 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:08.318+0000 7faaf7e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d862400260 0x55d8624081b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:52800/0 (socket says 172.21.15.47:52800) 2024-03-21T10:29:08.319 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:08.318+0000 7faaf7e00640 1 -- 172.21.15.47:0/3022188484 learned_addr learned my addr 172.21.15.47:0/3022188484 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:29:08.320 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:08.318+0000 7faafd800640 1 --2- 172.21.15.47:0/3022188484 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d862408de0 0x55d8624086f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:29:08.320 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:08.318+0000 7faaf7e00640 1 -- 172.21.15.47:0/3022188484 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d862408de0 msgr2=0x55d8624086f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:29:08.320 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:08.318+0000 7faaf7e00640 1 --2- 172.21.15.47:0/3022188484 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d862408de0 0x55d8624086f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:29:08.320 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:08.318+0000 7faaf7e00640 1 -- 172.21.15.47:0/3022188484 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d8623ff8b0 msgr2=0x55d8623f4ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:29:08.320 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:08.318+0000 7faaf7e00640 1 --2- 172.21.15.47:0/3022188484 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d8623ff8b0 0x55d8623f4ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:29:08.320 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:08.318+0000 7faaf7e00640 1 -- 172.21.15.47:0/3022188484 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d8623f5530 con 0x55d862400260 2024-03-21T10:29:08.320 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:08.318+0000 7faafce00640 1 --2- 172.21.15.47:0/3022188484 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d8623ff8b0 0x55d8623f4ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T10:29:08.320 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:08.319+0000 7faaf7e00640 1 --2- 172.21.15.47:0/3022188484 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d862400260 0x55d8624081b0 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7faae4006010 tx=0x7faae40c1e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=831e7c4100677fd4 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:29:08.320 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:08.319+0000 7faae9600640 1 -- 172.21.15.47:0/3022188484 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faae40c6570 con 0x55d862400260 2024-03-21T10:29:08.320 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:08.319+0000 7faae9600640 1 -- 172.21.15.47:0/3022188484 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7faae4005150 con 0x55d862400260 2024-03-21T10:29:08.320 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:08.319+0000 7faae9600640 1 -- 172.21.15.47:0/3022188484 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faae40bcfb0 con 0x55d862400260 2024-03-21T10:29:08.320 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:08.320+0000 7fab02578740 1 -- 172.21.15.47:0/3022188484 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d862400260 msgr2=0x55d8624081b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:29:08.320 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:08.320+0000 7fab02578740 1 --2- 172.21.15.47:0/3022188484 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d862400260 0x55d8624081b0 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7faae4006010 tx=0x7faae40c1e30 comp rx=0 tx=0).stop 2024-03-21T10:29:08.321 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:08.320+0000 7fab02578740 1 -- 172.21.15.47:0/3022188484 shutdown_connections 2024-03-21T10:29:08.321 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:08.320+0000 7fab02578740 1 --2- 172.21.15.47:0/3022188484 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d862408de0 0x55d8624086f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:29:08.321 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:08.320+0000 7fab02578740 1 --2- 172.21.15.47:0/3022188484 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d862400260 0x55d8624081b0 unknown :-1 s=CLOSED pgs=772 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:29:08.321 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:08.320+0000 7fab02578740 1 --2- 172.21.15.47:0/3022188484 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d8623ff8b0 0x55d8623f4ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:29:08.321 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:08.320+0000 7fab02578740 1 -- 172.21.15.47:0/3022188484 >> 172.21.15.47:0/3022188484 conn(0x55d8623f9f80 msgr2=0x55d8623fbc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:29:08.321 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:08.320+0000 7fab02578740 1 -- 172.21.15.47:0/3022188484 shutdown_connections 2024-03-21T10:29:08.321 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:08.320+0000 7fab02578740 1 -- 172.21.15.47:0/3022188484 wait complete. 2024-03-21T10:29:08.325 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:08.320+0000 7fab02578740 1 -- start start 2024-03-21T10:29:08.325 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:08.321+0000 7fab02578740 1 Processor -- start 2024-03-21T10:29:08.325 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:08.321+0000 7fab02578740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d8623ff8b0 0x55d862410f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:29:08.325 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:08.321+0000 7fab02578740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d862400260 0x55d86240f330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:29:08.325 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:08.321+0000 7fab02578740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d862408de0 0x55d86240f890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:29:08.325 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:08.321+0000 7fab02578740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faaf81b1c00 con 0x55d8623ff8b0 2024-03-21T10:29:08.325 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:08.321+0000 7fab02578740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faaf81b15a0 con 0x55d862400260 2024-03-21T10:29:08.325 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:08.321+0000 7fab02578740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faaf81aeaa0 con 0x55d862408de0 2024-03-21T10:29:08.325 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:08.321+0000 7fab02578740 10 client.0 _ll_register_callbacks cb 0x7faae40bd4c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T10:29:08.325 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:08.321+0000 7faafce00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d8623ff8b0 0x55d862410f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:29:08.325 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:08.321+0000 7faafce00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d8623ff8b0 0x55d862410f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:52812/0 (socket says 172.21.15.47:52812) 2024-03-21T10:29:08.325 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:08.321+0000 7faafce00640 1 -- 172.21.15.47:0/1975949224 learned_addr learned my addr 172.21.15.47:0/1975949224 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:29:08.326 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:08.322+0000 7faaf7e00640 1 --2- 172.21.15.47:0/1975949224 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d862400260 0x55d86240f330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:29:08.326 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:08.322+0000 7faafce00640 1 -- 172.21.15.47:0/1975949224 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d862408de0 msgr2=0x55d86240f890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:29:08.326 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:08.322+0000 7faafd800640 1 --2- 172.21.15.47:0/1975949224 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d862408de0 0x55d86240f890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:29:08.326 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:08.322+0000 7faafce00640 1 --2- 172.21.15.47:0/1975949224 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d862408de0 0x55d86240f890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:29:08.326 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:08.322+0000 7faafce00640 1 -- 172.21.15.47:0/1975949224 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d862400260 msgr2=0x55d86240f330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:29:08.326 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:08.322+0000 7faafce00640 1 --2- 172.21.15.47:0/1975949224 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d862400260 0x55d86240f330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:29:08.326 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:08.322+0000 7faafce00640 1 -- 172.21.15.47:0/1975949224 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55d8624100c0 con 0x55d8623ff8b0 2024-03-21T10:29:08.326 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:08.322+0000 7faafce00640 1 --2- 172.21.15.47:0/1975949224 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d8623ff8b0 0x55d862410f90 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x55d8623ff380 tx=0x7faaec00b700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=83ec30c7ff3ec4ad server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:29:08.326 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:08.322+0000 7faadea00640 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-03-21T10:29:08.326 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:08.322+0000 7faadea00640 1 -- 172.21.15.47:0/1975949224 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faaec02f060 con 0x55d8623ff8b0 2024-03-21T10:29:08.326 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:08.322+0000 7faadea00640 1 -- 172.21.15.47:0/1975949224 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(97..97 src has 1..97) v4 ==== 9474+0+0 (secure 0 0 0) 0x7faaec0035f0 con 0x55d8623ff8b0 2024-03-21T10:29:08.326 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:08.323+0000 7fab02578740 1 -- 172.21.15.47:0/1975949224 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7faae4005150 con 0x55d8623ff8b0 2024-03-21T10:29:08.326 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:08.323+0000 7faadea00640 1 -- 172.21.15.47:0/1975949224 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7faaec036050 con 0x55d8623ff8b0 2024-03-21T10:29:08.327 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:08.323+0000 7fab02578740 10 client.16878 fetch_fsmap learned FSMap version 32 2024-03-21T10:29:08.327 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:08.323+0000 7fab02578740 1 -- 172.21.15.47:0/1975949224 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x55d862475620 con 0x55d8623ff8b0 2024-03-21T10:29:08.327 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:08.323+0000 7faadea00640 1 -- 172.21.15.47:0/1975949224 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7faae4005150 con 0x55d8623ff8b0 2024-03-21T10:29:08.327 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:08.323+0000 7faadea00640 1 -- 172.21.15.47:0/1975949224 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 32) v1 ==== 7942+0+0 (secure 0 0 0) 0x7faaec029770 con 0x55d8623ff8b0 2024-03-21T10:29:08.327 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:08.323+0000 7fab02578740 10 client.16878 fetch_fsmap finished waiting for FSMap version 32 2024-03-21T10:29:08.327 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:08.323+0000 7fab02578740 10 client.16878 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.k 2024-03-21T10:29:08.327 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:08.323+0000 7fab02578740 20 client.16878 populate_metadata read hostname 'smithi047' 2024-03-21T10:29:08.327 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:08.323+0000 7fab02578740 1 --2- 172.21.15.47:0/1975949224 >> [v2:172.21.15.100:6832/2723302038,v1:172.21.15.100:6833/2723302038] conn(0x55d862475a20 0x55d862477e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:29:08.327 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:08.323+0000 7fab02578740 4 client.16878 mds_command: new command op to 34427 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-21T10:29:08.327 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:08.323+0000 7fab02578740 1 -- 172.21.15.47:0/1975949224 --> [v2:172.21.15.100:6832/2723302038,v1:172.21.15.100:6833/2723302038] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55d86236ab50 con 0x55d862475a20 2024-03-21T10:29:08.327 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:08.324+0000 7faaf7e00640 1 --2- 172.21.15.47:0/1975949224 >> [v2:172.21.15.100:6832/2723302038,v1:172.21.15.100:6833/2723302038] conn(0x55d862475a20 0x55d862477e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:29:08.327 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:08.324+0000 7faaf7e00640 1 --2- 172.21.15.47:0/1975949224 >> [v2:172.21.15.100:6832/2723302038,v1:172.21.15.100:6833/2723302038] conn(0x55d862475a20 0x55d862477e40 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=a9f41b388a80835a server_cookie=5edcb35fdb9761a7 in_seq=0 out_seq=0 2024-03-21T10:29:08.327 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:08.324+0000 7faadea00640 10 client.16878 ms_handle_connect on v2:172.21.15.100:6832/2723302038 2024-03-21T10:29:08.327 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:08.325+0000 7faadea00640 1 -- 172.21.15.47:0/1975949224 <== mds.3 v2:172.21.15.100:6832/2723302038 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55d86236ab50 con 0x55d862475a20 2024-03-21T10:29:08.328 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:08.325+0000 7faadea00640 10 client.16878 handle_command_reply: tid=0 2024-03-21T10:29:08.328 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:08.325+0000 7fab02578740 1 client.16878 shutdown 2024-03-21T10:29:08.328 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:08.325+0000 7fab02578740 1 -- 172.21.15.47:0/1975949224 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d8623ff8b0 msgr2=0x55d862410f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:29:08.328 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:08.325+0000 7fab02578740 1 --2- 172.21.15.47:0/1975949224 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d8623ff8b0 0x55d862410f90 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x55d8623ff380 tx=0x7faaec00b700 comp rx=0 tx=0).stop 2024-03-21T10:29:08.328 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:08.325+0000 7fab02578740 1 -- 172.21.15.47:0/1975949224 shutdown_connections 2024-03-21T10:29:08.328 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:08.325+0000 7fab02578740 1 --2- 172.21.15.47:0/1975949224 >> [v2:172.21.15.100:6832/2723302038,v1:172.21.15.100:6833/2723302038] conn(0x55d862475a20 0x55d862477e40 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:29:08.328 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:08.325+0000 7fab02578740 1 --2- 172.21.15.47:0/1975949224 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d862408de0 0x55d86240f890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:29:08.328 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:08.325+0000 7fab02578740 1 --2- 172.21.15.47:0/1975949224 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d862400260 0x55d86240f330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:29:08.328 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:08.325+0000 7fab02578740 1 --2- 172.21.15.47:0/1975949224 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d8623ff8b0 0x55d862410f90 unknown :-1 s=CLOSED pgs=773 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:29:08.328 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:08.325+0000 7faadea00640 0 client.16878 ms_handle_reset on v2:172.21.15.100:6832/2723302038 2024-03-21T10:29:08.328 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:08.325+0000 7fab02578740 1 -- 172.21.15.47:0/1975949224 >> 172.21.15.47:0/1975949224 conn(0x55d8623f9f80 msgr2=0x55d8623fb370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:29:08.328 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:08.325+0000 7fab02578740 1 -- 172.21.15.47:0/1975949224 shutdown_connections 2024-03-21T10:29:08.328 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:08.325+0000 7fab02578740 1 -- 172.21.15.47:0/1975949224 wait complete. 2024-03-21T10:29:08.329 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:08.325+0000 7fab02578740 20 client.16878 trim_cache size 0 max 16384 2024-03-21T10:29:08.329 INFO:teuthology.orchestra.run.smithi047.stdout:[] 2024-03-21T10:29:08.330 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:08.326+0000 7faafe200640 1 -- 172.21.15.47:0/1267510802 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7faae00a3110 msgr2=0x7faae00a55d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:29:08.330 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:08.326+0000 7faafe200640 1 --2- 172.21.15.47:0/1267510802 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7faae00a3110 0x7faae00a55d0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7faaf0005fd0 tx=0x7faaf0009500 comp rx=0 tx=0).stop 2024-03-21T10:29:08.330 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:08.326+0000 7faafe200640 1 -- 172.21.15.47:0/1267510802 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7faaf8102ff0 msgr2=0x7faaf8100860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:29:08.330 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:08.326+0000 7faafe200640 1 --2- 172.21.15.47:0/1267510802 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7faaf8102ff0 0x7faaf8100860 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7faae4005940 tx=0x7faae402c690 comp rx=0 tx=0).stop 2024-03-21T10:29:08.330 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:08.326+0000 7faafe200640 1 -- 172.21.15.47:0/1267510802 shutdown_connections 2024-03-21T10:29:08.330 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:08.326+0000 7faafe200640 1 --2- 172.21.15.47:0/1267510802 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7faaf810baf0 0x7faaf8100da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:29:08.330 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:08.326+0000 7faafe200640 1 --2- 172.21.15.47:0/1267510802 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7faaf8102ff0 0x7faaf8100860 unknown :-1 s=CLOSED pgs=770 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:29:08.330 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:08.326+0000 7faafe200640 1 --2- 172.21.15.47:0/1267510802 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7faae00a3110 0x7faae00a55d0 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:29:08.330 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:08.326+0000 7faafe200640 1 --2- 172.21.15.47:0/1267510802 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7faaf81026d0 0x7faaf8100320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:29:08.330 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:08.326+0000 7faafe200640 1 -- 172.21.15.47:0/1267510802 >> 172.21.15.47:0/1267510802 conn(0x7faaf80fbef0 msgr2=0x7faaf80fcc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:29:08.330 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:08.326+0000 7faafe200640 1 -- 172.21.15.47:0/1267510802 shutdown_connections 2024-03-21T10:29:08.330 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:08.326+0000 7faafe200640 1 -- 172.21.15.47:0/1267510802 wait complete. 2024-03-21T10:29:08.345 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-21T10:29:08.345 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-03-21T10:29:08.552 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:08.552+0000 7f0d49400640 1 -- 172.21.15.47:0/1212834908 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0d44102770 msgr2=0x7f0d44102b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:29:08.552 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:08.552+0000 7f0d49400640 1 --2- 172.21.15.47:0/1212834908 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0d44102770 0x7f0d44102b50 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f0d34009ca0 tx=0x7f0d3402c190 comp rx=0 tx=0).stop 2024-03-21T10:29:08.552 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:08.552+0000 7f0d49400640 1 -- 172.21.15.47:0/1212834908 shutdown_connections 2024-03-21T10:29:08.552 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:08.552+0000 7f0d49400640 1 --2- 172.21.15.47:0/1212834908 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0d4410bb40 0x7f0d4410df90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:29:08.553 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:08.552+0000 7f0d49400640 1 --2- 172.21.15.47:0/1212834908 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0d44103090 0x7f0d4410b570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:29:08.553 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:08.552+0000 7f0d49400640 1 --2- 172.21.15.47:0/1212834908 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0d44102770 0x7f0d44102b50 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:29:08.553 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:08.552+0000 7f0d49400640 1 -- 172.21.15.47:0/1212834908 >> 172.21.15.47:0/1212834908 conn(0x7f0d440fbef0 msgr2=0x7f0d440fe310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:29:08.553 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:08.552+0000 7f0d49400640 1 -- 172.21.15.47:0/1212834908 shutdown_connections 2024-03-21T10:29:08.553 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:08.553+0000 7f0d49400640 1 -- 172.21.15.47:0/1212834908 wait complete. 2024-03-21T10:29:08.555 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:08.553+0000 7f0d49400640 1 Processor -- start 2024-03-21T10:29:08.555 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:08.554+0000 7f0d49400640 1 -- start start 2024-03-21T10:29:08.555 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:08.554+0000 7f0d49400640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0d44102770 0x7f0d44199f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:29:08.555 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:08.554+0000 7f0d49400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0d44103090 0x7f0d4419a470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:29:08.556 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:08.554+0000 7f0d49400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0d4410bb40 0x7f0d4419e7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:29:08.556 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:08.554+0000 7f0d49400640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d44111770 con 0x7f0d44103090 2024-03-21T10:29:08.556 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:08.554+0000 7f0d49400640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d441115f0 con 0x7f0d4410bb40 2024-03-21T10:29:08.556 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:08.554+0000 7f0d49400640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d441118f0 con 0x7f0d44102770 2024-03-21T10:29:08.556 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:08.554+0000 7f0d43400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0d44103090 0x7f0d4419a470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:29:08.556 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:08.554+0000 7f0d43400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0d44103090 0x7f0d4419a470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:52826/0 (socket says 172.21.15.47:52826) 2024-03-21T10:29:08.556 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:08.554+0000 7f0d43400640 1 -- 172.21.15.47:0/299544392 learned_addr learned my addr 172.21.15.47:0/299544392 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:29:08.556 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:08.555+0000 7f0d48a00640 1 --2- 172.21.15.47:0/299544392 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0d4410bb40 0x7f0d4419e7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:29:08.556 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:08.555+0000 7f0d43e00640 1 --2- 172.21.15.47:0/299544392 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0d44102770 0x7f0d44199f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:29:08.556 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:08.555+0000 7f0d43400640 1 -- 172.21.15.47:0/299544392 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0d44102770 msgr2=0x7f0d44199f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:29:08.556 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:08.555+0000 7f0d43400640 1 --2- 172.21.15.47:0/299544392 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0d44102770 0x7f0d44199f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:29:08.556 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:08.555+0000 7f0d43400640 1 -- 172.21.15.47:0/299544392 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0d4410bb40 msgr2=0x7f0d4419e7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:29:08.557 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:08.555+0000 7f0d43400640 1 --2- 172.21.15.47:0/299544392 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0d4410bb40 0x7f0d4419e7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:29:08.557 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:08.555+0000 7f0d43400640 1 -- 172.21.15.47:0/299544392 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d4419ef20 con 0x7f0d44103090 2024-03-21T10:29:08.557 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:08.555+0000 7f0d48a00640 1 --2- 172.21.15.47:0/299544392 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0d4410bb40 0x7f0d4419e7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T10:29:08.557 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:08.555+0000 7f0d43400640 1 --2- 172.21.15.47:0/299544392 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0d44103090 0x7f0d4419a470 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7f0d2c00d950 tx=0x7f0d2c00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:29:08.557 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:08.555+0000 7f0d40c00640 1 -- 172.21.15.47:0/299544392 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d2c017030 con 0x7f0d44103090 2024-03-21T10:29:08.557 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:08.556+0000 7f0d49400640 1 -- 172.21.15.47:0/299544392 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0d4419f210 con 0x7f0d44103090 2024-03-21T10:29:08.557 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:08.556+0000 7f0d49400640 1 -- 172.21.15.47:0/299544392 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0d441a6aa0 con 0x7f0d44103090 2024-03-21T10:29:08.557 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:08.556+0000 7f0d40c00640 1 -- 172.21.15.47:0/299544392 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f0d2c004020 con 0x7f0d44103090 2024-03-21T10:29:08.557 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:08.556+0000 7f0d40c00640 1 -- 172.21.15.47:0/299544392 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d2c005640 con 0x7f0d44103090 2024-03-21T10:29:08.558 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:08.559+0000 7f0d4d7c0740 1 Processor -- start 2024-03-21T10:29:08.560 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:08.559+0000 7f0d40c00640 1 -- 172.21.15.47:0/299544392 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 27) v1 ==== 137985+0+0 (secure 0 0 0) 0x7f0d2c0041d0 con 0x7f0d44103090 2024-03-21T10:29:08.560 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:08.559+0000 7f0d4d7c0740 1 -- start start 2024-03-21T10:29:08.560 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:08.560+0000 7f0d4d7c0740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x557cdceb18b0 0x557cdceb1c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:29:08.560 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:08.560+0000 7f0d4d7c0740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557cdceb2260 0x557cdceba810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:29:08.560 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:08.560+0000 7f0d4d7c0740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557cdcebade0 0x557cdcebd230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:29:08.560 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:08.560+0000 7f0d4d7c0740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x557cdce152c0 con 0x557cdceb18b0 2024-03-21T10:29:08.560 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:08.560+0000 7f0d4d7c0740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x557cdcdbdc90 con 0x557cdceb2260 2024-03-21T10:29:08.560 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:08.560+0000 7f0d4d7c0740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x557cdce11360 con 0x557cdcebade0 2024-03-21T10:29:08.560 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:08.560+0000 7f0d48a00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x557cdceb18b0 0x557cdceb1c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:29:08.560 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:08.560+0000 7f0d48a00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x557cdceb18b0 0x557cdceb1c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:52842/0 (socket says 172.21.15.47:52842) 2024-03-21T10:29:08.560 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:08.560+0000 7f0d48a00640 1 -- 172.21.15.47:0/3481117140 learned_addr learned my addr 172.21.15.47:0/3481117140 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:29:08.560 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:08.560+0000 7f0d43e00640 1 --2- 172.21.15.47:0/3481117140 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557cdceb2260 0x557cdceba810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:29:08.561 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:08.561+0000 7f0d43400640 1 --2- 172.21.15.47:0/3481117140 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557cdcebade0 0x557cdcebd230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:29:08.562 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:08.561+0000 7f0d48a00640 1 -- 172.21.15.47:0/3481117140 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557cdcebade0 msgr2=0x557cdcebd230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:29:08.562 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:08.561+0000 7f0d48a00640 1 --2- 172.21.15.47:0/3481117140 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557cdcebade0 0x557cdcebd230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:29:08.562 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:08.561+0000 7f0d48a00640 1 -- 172.21.15.47:0/3481117140 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557cdceb2260 msgr2=0x557cdceba810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:29:08.563 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:08.561+0000 7f0d48a00640 1 --2- 172.21.15.47:0/3481117140 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557cdceb2260 0x557cdceba810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:29:08.563 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:08.561+0000 7f0d48a00640 1 -- 172.21.15.47:0/3481117140 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557cdcde2080 con 0x557cdceb18b0 2024-03-21T10:29:08.563 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:08.561+0000 7f0d40c00640 1 --2- 172.21.15.47:0/299544392 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f0d280a3230 0x7f0d280a56f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:29:08.563 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:08.561+0000 7f0d48a00640 1 --2- 172.21.15.47:0/3481117140 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x557cdceb18b0 0x557cdceb1c90 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7f0d3c009c10 tx=0x7f0d3c0117b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=37066dcbb20bdcb6 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:29:08.563 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:08.561+0000 7f0d40c00640 1 -- 172.21.15.47:0/299544392 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 9474+0+0 (secure 0 0 0) 0x7f0d2c0cf660 con 0x7f0d44103090 2024-03-21T10:29:08.563 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:08.562+0000 7f0d43e00640 1 --2- 172.21.15.47:0/299544392 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f0d280a3230 0x7f0d280a56f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:29:08.563 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:08.562+0000 7f0d39200640 1 -- 172.21.15.47:0/3481117140 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d3c012450 con 0x557cdceb18b0 2024-03-21T10:29:08.564 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:08.562+0000 7f0d39200640 1 -- 172.21.15.47:0/3481117140 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f0d3c012aa0 con 0x557cdceb18b0 2024-03-21T10:29:08.564 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:08.562+0000 7f0d39200640 1 -- 172.21.15.47:0/3481117140 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d3c00fb30 con 0x557cdceb18b0 2024-03-21T10:29:08.564 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:08.562+0000 7f0d4d7c0740 1 -- 172.21.15.47:0/3481117140 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x557cdceb18b0 msgr2=0x557cdceb1c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:29:08.564 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:08.562+0000 7f0d4d7c0740 1 --2- 172.21.15.47:0/3481117140 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x557cdceb18b0 0x557cdceb1c90 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7f0d3c009c10 tx=0x7f0d3c0117b0 comp rx=0 tx=0).stop 2024-03-21T10:29:08.565 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:08.562+0000 7f0d43e00640 1 --2- 172.21.15.47:0/299544392 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f0d280a3230 0x7f0d280a56f0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f0d34000f80 tx=0x7f0d34005d80 comp rx=0 tx=0).ready entity=mgr.15687 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:29:08.565 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:08.563+0000 7f0d4d7c0740 1 -- 172.21.15.47:0/3481117140 shutdown_connections 2024-03-21T10:29:08.565 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:08.563+0000 7f0d4d7c0740 1 --2- 172.21.15.47:0/3481117140 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557cdcebade0 0x557cdcebd230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:29:08.565 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:08.563+0000 7f0d4d7c0740 1 --2- 172.21.15.47:0/3481117140 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557cdceb2260 0x557cdceba810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:29:08.565 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:08.563+0000 7f0d4d7c0740 1 --2- 172.21.15.47:0/3481117140 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x557cdceb18b0 0x557cdceb1c90 unknown :-1 s=CLOSED pgs=775 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:29:08.565 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:08.563+0000 7f0d4d7c0740 1 -- 172.21.15.47:0/3481117140 >> 172.21.15.47:0/3481117140 conn(0x557cdceabf80 msgr2=0x557cdceac360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:29:08.565 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:08.563+0000 7f0d4d7c0740 1 -- 172.21.15.47:0/3481117140 shutdown_connections 2024-03-21T10:29:08.565 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:08.563+0000 7f0d4d7c0740 1 -- 172.21.15.47:0/3481117140 wait complete. 2024-03-21T10:29:08.565 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:08.563+0000 7f0d4d7c0740 1 -- start start 2024-03-21T10:29:08.568 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:08.564+0000 7f0d4d7c0740 1 Processor -- start 2024-03-21T10:29:08.568 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:08.564+0000 7f0d4d7c0740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557cdceb18b0 0x557cdceba6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:29:08.568 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:08.564+0000 7f0d4d7c0740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x557cdceb2260 0x557cdceb7a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:29:08.568 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:08.564+0000 7f0d4d7c0740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557cdcebade0 0x557cdceb7ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:29:08.568 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:08.564+0000 7f0d4d7c0740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x557cdcebdd50 con 0x557cdceb2260 2024-03-21T10:29:08.568 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:08.564+0000 7f0d4d7c0740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x557cdcdd10d0 con 0x557cdcebade0 2024-03-21T10:29:08.568 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:08.564+0000 7f0d4d7c0740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x557cdcebded0 con 0x557cdceb18b0 2024-03-21T10:29:08.569 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:08.564+0000 7f0d4d7c0740 10 client.0 _ll_register_callbacks cb 0x557cdcdd5e80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T10:29:08.569 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:08.564+0000 7f0d43400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x557cdceb2260 0x557cdceb7a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:29:08.569 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:08.565+0000 7f0d43400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x557cdceb2260 0x557cdceb7a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:52844/0 (socket says 172.21.15.47:52844) 2024-03-21T10:29:08.569 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:08.565+0000 7f0d43400640 1 -- 172.21.15.47:0/1887885819 learned_addr learned my addr 172.21.15.47:0/1887885819 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:29:08.569 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:08.565+0000 7f0d43e00640 1 --2- 172.21.15.47:0/1887885819 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557cdceb18b0 0x557cdceba6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:29:08.569 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:08.565+0000 7f0d48a00640 1 --2- 172.21.15.47:0/1887885819 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557cdcebade0 0x557cdceb7ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:29:08.569 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:08.565+0000 7f0d43400640 1 -- 172.21.15.47:0/1887885819 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557cdceb18b0 msgr2=0x557cdceba6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:29:08.569 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:08.565+0000 7f0d43400640 1 --2- 172.21.15.47:0/1887885819 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557cdceb18b0 0x557cdceba6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:29:08.569 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:08.565+0000 7f0d43400640 1 -- 172.21.15.47:0/1887885819 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557cdcebade0 msgr2=0x557cdceb7ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:29:08.569 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:08.565+0000 7f0d43400640 1 --2- 172.21.15.47:0/1887885819 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557cdcebade0 0x557cdceb7ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:29:08.569 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:08.565+0000 7f0d43400640 1 -- 172.21.15.47:0/1887885819 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x557cdcb293f0 con 0x557cdceb2260 2024-03-21T10:29:08.569 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:08.565+0000 7f0d43400640 1 --2- 172.21.15.47:0/1887885819 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x557cdceb2260 0x557cdceb7a70 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7f0d2c000c00 tx=0x7f0d2c09af90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f797fc23d7f91e83 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:29:08.569 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:08.565+0000 7f0d31600640 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-03-21T10:29:08.570 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:08.565+0000 7f0d31600640 1 -- 172.21.15.47:0/1887885819 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d2c0cce10 con 0x557cdceb2260 2024-03-21T10:29:08.570 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:08.565+0000 7f0d31600640 1 -- 172.21.15.47:0/1887885819 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(97..97 src has 1..97) v4 ==== 9474+0+0 (secure 0 0 0) 0x7f0d2c0cf1e0 con 0x557cdceb2260 2024-03-21T10:29:08.570 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:08.566+0000 7f0d4d7c0740 1 -- 172.21.15.47:0/1887885819 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f0d3c012aa0 con 0x557cdceb2260 2024-03-21T10:29:08.570 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:08.566+0000 7f0d31600640 1 -- 172.21.15.47:0/1887885819 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0d2c09b630 con 0x557cdceb2260 2024-03-21T10:29:08.570 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:08.566+0000 7f0d4d7c0740 10 client.16890 fetch_fsmap learned FSMap version 32 2024-03-21T10:29:08.570 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:08.566+0000 7f0d4d7c0740 1 -- 172.21.15.47:0/1887885819 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x557cdcbf37d0 con 0x557cdceb2260 2024-03-21T10:29:08.570 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:08.566+0000 7f0d31600640 1 -- 172.21.15.47:0/1887885819 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f0d3c012aa0 con 0x557cdceb2260 2024-03-21T10:29:08.570 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:08.566+0000 7f0d31600640 1 -- 172.21.15.47:0/1887885819 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 32) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f0d2c0a3bb0 con 0x557cdceb2260 2024-03-21T10:29:08.570 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:08.566+0000 7f0d4d7c0740 10 client.16890 fetch_fsmap finished waiting for FSMap version 32 2024-03-21T10:29:08.570 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:08.566+0000 7f0d4d7c0740 10 client.16890 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.e 2024-03-21T10:29:08.570 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:08.566+0000 7f0d4d7c0740 20 client.16890 populate_metadata read hostname 'smithi047' 2024-03-21T10:29:08.570 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:08.566+0000 7f0d4d7c0740 1 --2- 172.21.15.47:0/1887885819 >> [v2:172.21.15.100:6834/2597849957,v1:172.21.15.100:6835/2597849957] conn(0x557cdcecac70 0x557cdcecd060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:29:08.571 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:08.566+0000 7f0d4d7c0740 4 client.16890 mds_command: new command op to 34454 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T10:29:08.571 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:08.566+0000 7f0d4d7c0740 1 -- 172.21.15.47:0/1887885819 --> [v2:172.21.15.100:6834/2597849957,v1:172.21.15.100:6835/2597849957] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x557cdce22800 con 0x557cdcecac70 2024-03-21T10:29:08.571 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:08.567+0000 7f0d43e00640 1 --2- 172.21.15.47:0/1887885819 >> [v2:172.21.15.100:6834/2597849957,v1:172.21.15.100:6835/2597849957] conn(0x557cdcecac70 0x557cdcecd060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:29:08.571 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:08.568+0000 7f0d43e00640 1 --2- 172.21.15.47:0/1887885819 >> [v2:172.21.15.100:6834/2597849957,v1:172.21.15.100:6835/2597849957] conn(0x557cdcecac70 0x557cdcecd060 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=9bed66a88e2a51b1 server_cookie=6b364613438db0b5 in_seq=0 out_seq=0 2024-03-21T10:29:08.571 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:08.568+0000 7f0d31600640 10 client.16890 ms_handle_connect on v2:172.21.15.100:6834/2597849957 2024-03-21T10:29:08.572 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:08.570+0000 7f0d31600640 1 -- 172.21.15.47:0/1887885819 <== mds.2 v2:172.21.15.100:6834/2597849957 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x557cdce22800 con 0x557cdcecac70 2024-03-21T10:29:08.572 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:08.570+0000 7f0d31600640 10 client.16890 handle_command_reply: tid=0 2024-03-21T10:29:08.572 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:08.570+0000 7f0d4d7c0740 1 client.16890 shutdown 2024-03-21T10:29:08.572 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:08.570+0000 7f0d4d7c0740 1 -- 172.21.15.47:0/1887885819 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x557cdceb2260 msgr2=0x557cdceb7a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:29:08.572 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:08.570+0000 7f0d4d7c0740 1 --2- 172.21.15.47:0/1887885819 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x557cdceb2260 0x557cdceb7a70 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7f0d2c000c00 tx=0x7f0d2c09af90 comp rx=0 tx=0).stop 2024-03-21T10:29:08.572 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:08.570+0000 7f0d4d7c0740 1 -- 172.21.15.47:0/1887885819 shutdown_connections 2024-03-21T10:29:08.573 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:08.570+0000 7f0d4d7c0740 1 --2- 172.21.15.47:0/1887885819 >> [v2:172.21.15.100:6834/2597849957,v1:172.21.15.100:6835/2597849957] conn(0x557cdcecac70 0x557cdcecd060 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:29:08.573 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:08.570+0000 7f0d4d7c0740 1 --2- 172.21.15.47:0/1887885819 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557cdcebade0 0x557cdceb7ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:29:08.573 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:08.570+0000 7f0d4d7c0740 1 --2- 172.21.15.47:0/1887885819 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x557cdceb2260 0x557cdceb7a70 unknown :-1 s=CLOSED pgs=776 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:29:08.573 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:08.570+0000 7f0d31600640 0 client.16890 ms_handle_reset on v2:172.21.15.100:6834/2597849957 2024-03-21T10:29:08.573 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:08.570+0000 7f0d4d7c0740 1 --2- 172.21.15.47:0/1887885819 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557cdceb18b0 0x557cdceba6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:29:08.573 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:08.570+0000 7f0d4d7c0740 1 -- 172.21.15.47:0/1887885819 >> 172.21.15.47:0/1887885819 conn(0x557cdceabf80 msgr2=0x557cdceadcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:29:08.573 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:08.570+0000 7f0d4d7c0740 1 -- 172.21.15.47:0/1887885819 shutdown_connections 2024-03-21T10:29:08.573 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:08.570+0000 7f0d4d7c0740 1 -- 172.21.15.47:0/1887885819 wait complete. 2024-03-21T10:29:08.573 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:08.570+0000 7f0d4d7c0740 20 client.16890 trim_cache size 0 max 16384 2024-03-21T10:29:08.584 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:08.584+0000 7f0d4d7c0740 1 Processor -- start 2024-03-21T10:29:08.584 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:08.585+0000 7f0d4d7c0740 1 -- start start 2024-03-21T10:29:08.586 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:08.585+0000 7f0d4d7c0740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557cdceb18b0 0x557cdceba1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:29:08.586 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:08.585+0000 7f0d4d7c0740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557cdceb2260 0x557cdceba6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:29:08.586 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:08.585+0000 7f0d4d7c0740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x557cdcebade0 0x557cdceb4d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:29:08.586 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:08.585+0000 7f0d4d7c0740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d2000fd40 con 0x557cdcebade0 2024-03-21T10:29:08.586 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:08.585+0000 7f0d4d7c0740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d200116c0 con 0x557cdceb2260 2024-03-21T10:29:08.586 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:08.585+0000 7f0d4d7c0740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d200103a0 con 0x557cdceb18b0 2024-03-21T10:29:08.586 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:08.585+0000 7f0d48a00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x557cdcebade0 0x557cdceb4d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:29:08.586 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:08.585+0000 7f0d48a00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x557cdcebade0 0x557cdceb4d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:52852/0 (socket says 172.21.15.47:52852) 2024-03-21T10:29:08.586 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:08.585+0000 7f0d48a00640 1 -- 172.21.15.47:0/4036193824 learned_addr learned my addr 172.21.15.47:0/4036193824 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:29:08.586 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:08.585+0000 7f0d43400640 1 --2- 172.21.15.47:0/4036193824 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557cdceb2260 0x557cdceba6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:29:08.587 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:08.586+0000 7f0d43e00640 1 --2- 172.21.15.47:0/4036193824 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557cdceb18b0 0x557cdceba1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:29:08.587 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:08.586+0000 7f0d48a00640 1 -- 172.21.15.47:0/4036193824 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557cdceb18b0 msgr2=0x557cdceba1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:29:08.587 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:08.586+0000 7f0d48a00640 1 --2- 172.21.15.47:0/4036193824 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557cdceb18b0 0x557cdceba1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:29:08.587 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:08.586+0000 7f0d48a00640 1 -- 172.21.15.47:0/4036193824 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557cdceb2260 msgr2=0x557cdceba6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:29:08.587 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:08.586+0000 7f0d48a00640 1 --2- 172.21.15.47:0/4036193824 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557cdceb2260 0x557cdceba6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:29:08.587 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:08.586+0000 7f0d48a00640 1 -- 172.21.15.47:0/4036193824 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557cdcea78b0 con 0x557cdcebade0 2024-03-21T10:29:08.587 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:08.586+0000 7f0d48a00640 1 --2- 172.21.15.47:0/4036193824 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x557cdcebade0 0x557cdceb4d10 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7f0d3c005c50 tx=0x7f0d3c005c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8160b482e2554579 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:29:08.587 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:08.586+0000 7f0d33e00640 1 -- 172.21.15.47:0/4036193824 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d3c02a070 con 0x557cdcebade0 2024-03-21T10:29:08.587 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:08.587+0000 7f0d33e00640 1 -- 172.21.15.47:0/4036193824 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f0d3c0096c0 con 0x557cdcebade0 2024-03-21T10:29:08.587 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:08.587+0000 7f0d33e00640 1 -- 172.21.15.47:0/4036193824 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d3c005510 con 0x557cdcebade0 2024-03-21T10:29:08.590 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:08.587+0000 7f0d4d7c0740 1 -- 172.21.15.47:0/4036193824 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x557cdcebade0 msgr2=0x557cdceb4d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:29:08.590 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:08.587+0000 7f0d4d7c0740 1 --2- 172.21.15.47:0/4036193824 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x557cdcebade0 0x557cdceb4d10 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7f0d3c005c50 tx=0x7f0d3c005c80 comp rx=0 tx=0).stop 2024-03-21T10:29:08.590 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:08.587+0000 7f0d4d7c0740 1 -- 172.21.15.47:0/4036193824 shutdown_connections 2024-03-21T10:29:08.591 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:08.587+0000 7f0d4d7c0740 1 --2- 172.21.15.47:0/4036193824 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x557cdcebade0 0x557cdceb4d10 unknown :-1 s=CLOSED pgs=777 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:29:08.591 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:08.587+0000 7f0d4d7c0740 1 --2- 172.21.15.47:0/4036193824 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557cdceb2260 0x557cdceba6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:29:08.591 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:08.587+0000 7f0d4d7c0740 1 --2- 172.21.15.47:0/4036193824 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557cdceb18b0 0x557cdceba1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:29:08.591 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:08.587+0000 7f0d4d7c0740 1 -- 172.21.15.47:0/4036193824 >> 172.21.15.47:0/4036193824 conn(0x557cdceabf80 msgr2=0x557cdceab160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:29:08.591 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:08.587+0000 7f0d4d7c0740 1 -- 172.21.15.47:0/4036193824 shutdown_connections 2024-03-21T10:29:08.591 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:08.588+0000 7f0d4d7c0740 1 -- 172.21.15.47:0/4036193824 wait complete. 2024-03-21T10:29:08.591 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:08.588+0000 7f0d4d7c0740 1 -- start start 2024-03-21T10:29:08.591 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:08.588+0000 7f0d4d7c0740 1 Processor -- start 2024-03-21T10:29:08.591 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:08.588+0000 7f0d4d7c0740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x557cdceb18b0 0x557cdcec2cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:29:08.591 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:08.589+0000 7f0d4d7c0740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557cdceb2260 0x557cdcec1090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:29:08.591 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:08.589+0000 7f0d4d7c0740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557cdcebade0 0x557cdcec15d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:29:08.591 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:08.589+0000 7f0d4d7c0740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d20011060 con 0x557cdceb18b0 2024-03-21T10:29:08.592 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:08.589+0000 7f0d4d7c0740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d20010a00 con 0x557cdcebade0 2024-03-21T10:29:08.592 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:08.589+0000 7f0d4d7c0740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d2000df00 con 0x557cdceb2260 2024-03-21T10:29:08.592 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:08.589+0000 7f0d4d7c0740 10 client.0 _ll_register_callbacks cb 0x7f0d2c09b400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T10:29:08.592 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:08.589+0000 7f0d43e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x557cdceb18b0 0x557cdcec2cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:29:08.592 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:08.589+0000 7f0d43e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x557cdceb18b0 0x557cdcec2cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:52864/0 (socket says 172.21.15.47:52864) 2024-03-21T10:29:08.592 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:08.589+0000 7f0d43e00640 1 -- 172.21.15.47:0/1054507690 learned_addr learned my addr 172.21.15.47:0/1054507690 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:29:08.592 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:08.589+0000 7f0d48a00640 1 --2- 172.21.15.47:0/1054507690 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557cdcebade0 0x557cdcec15d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:29:08.592 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:08.589+0000 7f0d43400640 1 --2- 172.21.15.47:0/1054507690 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557cdceb2260 0x557cdcec1090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:29:08.592 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:08.589+0000 7f0d43e00640 1 -- 172.21.15.47:0/1054507690 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557cdceb2260 msgr2=0x557cdcec1090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:29:08.592 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:08.589+0000 7f0d43e00640 1 --2- 172.21.15.47:0/1054507690 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557cdceb2260 0x557cdcec1090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:29:08.592 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:08.589+0000 7f0d43e00640 1 -- 172.21.15.47:0/1054507690 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557cdcebade0 msgr2=0x557cdcec15d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:29:08.592 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:08.589+0000 7f0d43e00640 1 --2- 172.21.15.47:0/1054507690 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557cdcebade0 0x557cdcec15d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:29:08.592 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:08.589+0000 7f0d43e00640 1 -- 172.21.15.47:0/1054507690 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x557cdcec1c40 con 0x557cdceb18b0 2024-03-21T10:29:08.592 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:08.590+0000 7f0d43e00640 1 --2- 172.21.15.47:0/1054507690 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x557cdceb18b0 0x557cdcec2cf0 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7f0d34009ca0 tx=0x7f0d340095c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=57de6e8738dcaa66 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:29:08.593 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:08.590+0000 7f0d31600640 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-03-21T10:29:08.593 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:08.590+0000 7f0d31600640 1 -- 172.21.15.47:0/1054507690 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d34039040 con 0x557cdceb18b0 2024-03-21T10:29:08.593 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:08.590+0000 7f0d31600640 1 -- 172.21.15.47:0/1054507690 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(97..97 src has 1..97) v4 ==== 9474+0+0 (secure 0 0 0) 0x7f0d340511b0 con 0x557cdceb18b0 2024-03-21T10:29:08.593 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:08.590+0000 7f0d4d7c0740 1 -- 172.21.15.47:0/1054507690 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f0d3c0096c0 con 0x557cdceb18b0 2024-03-21T10:29:08.593 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:08.590+0000 7f0d31600640 1 -- 172.21.15.47:0/1054507690 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0d3405f050 con 0x557cdceb18b0 2024-03-21T10:29:08.593 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:08.590+0000 7f0d4d7c0740 10 client.16896 fetch_fsmap learned FSMap version 32 2024-03-21T10:29:08.593 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:08.590+0000 7f0d4d7c0740 1 -- 172.21.15.47:0/1054507690 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x557cdcf271b0 con 0x557cdceb18b0 2024-03-21T10:29:08.593 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:08.591+0000 7f0d31600640 1 -- 172.21.15.47:0/1054507690 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f0d3c0096c0 con 0x557cdceb18b0 2024-03-21T10:29:08.593 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:08.591+0000 7f0d31600640 1 -- 172.21.15.47:0/1054507690 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 32) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f0d34040690 con 0x557cdceb18b0 2024-03-21T10:29:08.593 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:08.591+0000 7f0d4d7c0740 10 client.16896 fetch_fsmap finished waiting for FSMap version 32 2024-03-21T10:29:08.593 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:08.591+0000 7f0d4d7c0740 10 client.16896 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.e 2024-03-21T10:29:08.593 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:08.591+0000 7f0d4d7c0740 20 client.16896 populate_metadata read hostname 'smithi047' 2024-03-21T10:29:08.593 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:08.591+0000 7f0d4d7c0740 1 --2- 172.21.15.47:0/1054507690 >> [v2:172.21.15.100:6834/2597849957,v1:172.21.15.100:6835/2597849957] conn(0x557cdcf278e0 0x557cdcf29d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:29:08.594 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:08.591+0000 7f0d4d7c0740 4 client.16896 mds_command: new command op to 34454 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-21T10:29:08.594 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:08.591+0000 7f0d4d7c0740 1 -- 172.21.15.47:0/1054507690 --> [v2:172.21.15.100:6834/2597849957,v1:172.21.15.100:6835/2597849957] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x557cdce1cb50 con 0x557cdcf278e0 2024-03-21T10:29:08.594 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:08.591+0000 7f0d43400640 1 --2- 172.21.15.47:0/1054507690 >> [v2:172.21.15.100:6834/2597849957,v1:172.21.15.100:6835/2597849957] conn(0x557cdcf278e0 0x557cdcf29d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:29:08.595 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:08.592+0000 7f0d43400640 1 --2- 172.21.15.47:0/1054507690 >> [v2:172.21.15.100:6834/2597849957,v1:172.21.15.100:6835/2597849957] conn(0x557cdcf278e0 0x557cdcf29d00 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=a0067f0194e2749 server_cookie=7521c22da2212a0b in_seq=0 out_seq=0 2024-03-21T10:29:08.595 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:08.592+0000 7f0d31600640 10 client.16896 ms_handle_connect on v2:172.21.15.100:6834/2597849957 2024-03-21T10:29:08.595 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:08.592+0000 7f0d31600640 1 -- 172.21.15.47:0/1054507690 <== mds.2 v2:172.21.15.100:6834/2597849957 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x557cdce1cb50 con 0x557cdcf278e0 2024-03-21T10:29:08.595 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:08.592+0000 7f0d31600640 10 client.16896 handle_command_reply: tid=0 2024-03-21T10:29:08.595 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:08.592+0000 7f0d4d7c0740 1 client.16896 shutdown 2024-03-21T10:29:08.595 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:08.592+0000 7f0d4d7c0740 1 -- 172.21.15.47:0/1054507690 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x557cdceb18b0 msgr2=0x557cdcec2cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:29:08.596 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:08.592+0000 7f0d4d7c0740 1 --2- 172.21.15.47:0/1054507690 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x557cdceb18b0 0x557cdcec2cf0 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7f0d34009ca0 tx=0x7f0d340095c0 comp rx=0 tx=0).stop 2024-03-21T10:29:08.596 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:08.592+0000 7f0d4d7c0740 1 -- 172.21.15.47:0/1054507690 shutdown_connections 2024-03-21T10:29:08.596 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:08.592+0000 7f0d4d7c0740 1 --2- 172.21.15.47:0/1054507690 >> [v2:172.21.15.100:6834/2597849957,v1:172.21.15.100:6835/2597849957] conn(0x557cdcf278e0 0x557cdcf29d00 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:29:08.596 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:08.592+0000 7f0d4d7c0740 1 --2- 172.21.15.47:0/1054507690 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557cdcebade0 0x557cdcec15d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:29:08.596 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:08.593+0000 7f0d31600640 0 client.16896 ms_handle_reset on v2:172.21.15.100:6834/2597849957 2024-03-21T10:29:08.596 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:08.593+0000 7f0d4d7c0740 1 --2- 172.21.15.47:0/1054507690 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557cdceb2260 0x557cdcec1090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:29:08.596 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:08.593+0000 7f0d4d7c0740 1 --2- 172.21.15.47:0/1054507690 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x557cdceb18b0 0x557cdcec2cf0 unknown :-1 s=CLOSED pgs=778 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:29:08.596 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:08.593+0000 7f0d4d7c0740 1 -- 172.21.15.47:0/1054507690 >> 172.21.15.47:0/1054507690 conn(0x557cdceabf80 msgr2=0x557cdcead260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:29:08.596 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:08.593+0000 7f0d4d7c0740 1 -- 172.21.15.47:0/1054507690 shutdown_connections 2024-03-21T10:29:08.596 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:08.593+0000 7f0d4d7c0740 1 -- 172.21.15.47:0/1054507690 wait complete. 2024-03-21T10:29:08.596 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:08.593+0000 7f0d4d7c0740 20 client.16896 trim_cache size 0 max 16384 2024-03-21T10:29:08.597 INFO:teuthology.orchestra.run.smithi047.stdout:[] 2024-03-21T10:29:08.597 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:08.593+0000 7f0d49400640 1 -- 172.21.15.47:0/299544392 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f0d280a3230 msgr2=0x7f0d280a56f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:29:08.597 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:08.593+0000 7f0d49400640 1 --2- 172.21.15.47:0/299544392 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f0d280a3230 0x7f0d280a56f0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f0d34000f80 tx=0x7f0d34005d80 comp rx=0 tx=0).stop 2024-03-21T10:29:08.598 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:08.593+0000 7f0d49400640 1 -- 172.21.15.47:0/299544392 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0d44103090 msgr2=0x7f0d4419a470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:29:08.598 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:08.593+0000 7f0d49400640 1 --2- 172.21.15.47:0/299544392 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0d44103090 0x7f0d4419a470 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7f0d2c00d950 tx=0x7f0d2c00de10 comp rx=0 tx=0).stop 2024-03-21T10:29:08.598 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:08.594+0000 7f0d49400640 1 -- 172.21.15.47:0/299544392 shutdown_connections 2024-03-21T10:29:08.598 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:08.594+0000 7f0d49400640 1 --2- 172.21.15.47:0/299544392 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f0d280a3230 0x7f0d280a56f0 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:29:08.598 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:08.594+0000 7f0d49400640 1 --2- 172.21.15.47:0/299544392 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0d4410bb40 0x7f0d4419e7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:29:08.598 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:08.594+0000 7f0d49400640 1 --2- 172.21.15.47:0/299544392 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0d44103090 0x7f0d4419a470 unknown :-1 s=CLOSED pgs=774 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:29:08.598 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:08.594+0000 7f0d49400640 1 --2- 172.21.15.47:0/299544392 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0d44102770 0x7f0d44199f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:29:08.598 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:08.594+0000 7f0d49400640 1 -- 172.21.15.47:0/299544392 >> 172.21.15.47:0/299544392 conn(0x7f0d440fbef0 msgr2=0x7f0d4410ce10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:29:08.598 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:08.594+0000 7f0d49400640 1 -- 172.21.15.47:0/299544392 shutdown_connections 2024-03-21T10:29:08.598 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:08.594+0000 7f0d49400640 1 -- 172.21.15.47:0/299544392 wait complete. 2024-03-21T10:29:08.608 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:29:08 smithi047 ceph-mon[31547]: from='client.? 172.21.15.47:0/2764959579' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-21T10:29:08.611 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-21T10:29:08.611 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-03-21T10:29:08.821 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:08.821+0000 7fa5a1c00640 1 -- 172.21.15.47:0/2165512965 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa59c100490 msgr2=0x7fa59c100870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:29:08.821 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:08.821+0000 7fa5a1c00640 1 --2- 172.21.15.47:0/2165512965 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa59c100490 0x7fa59c100870 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7fa58c009ca0 tx=0x7fa58c02c100 comp rx=0 tx=0).stop 2024-03-21T10:29:08.821 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:08.821+0000 7fa5a1c00640 1 -- 172.21.15.47:0/2165512965 shutdown_connections 2024-03-21T10:29:08.821 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:08.821+0000 7fa5a1c00640 1 --2- 172.21.15.47:0/2165512965 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa59c10e040 0x7fa59c110430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:29:08.821 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:08.821+0000 7fa5a1c00640 1 --2- 172.21.15.47:0/2165512965 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa59c100e40 0x7fa59c10db00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:29:08.821 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:08.821+0000 7fa5a1c00640 1 --2- 172.21.15.47:0/2165512965 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa59c100490 0x7fa59c100870 unknown :-1 s=CLOSED pgs=779 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:29:08.821 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:08.821+0000 7fa5a1c00640 1 -- 172.21.15.47:0/2165512965 >> 172.21.15.47:0/2165512965 conn(0x7fa59c077700 msgr2=0x7fa59c077b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:29:08.821 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:08.821+0000 7fa5a1c00640 1 -- 172.21.15.47:0/2165512965 shutdown_connections 2024-03-21T10:29:08.822 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:08.821+0000 7fa5a1c00640 1 -- 172.21.15.47:0/2165512965 wait complete. 2024-03-21T10:29:08.822 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:08.823+0000 7fa5a1c00640 1 Processor -- start 2024-03-21T10:29:08.822 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:08.823+0000 7fa5a1c00640 1 -- start start 2024-03-21T10:29:08.823 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:08.823+0000 7fa5a1c00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa59c100490 0x7fa59c19a130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:29:08.823 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:08.823+0000 7fa5a1c00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa59c100e40 0x7fa59c19a670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:29:08.823 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:08.823+0000 7fa5a1c00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa59c10e040 0x7fa59c19e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:29:08.823 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:08.823+0000 7fa5a1c00640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa59c113bb0 con 0x7fa59c10e040 2024-03-21T10:29:08.823 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:08.823+0000 7fa5a1c00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa59c113a30 con 0x7fa59c100e40 2024-03-21T10:29:08.823 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:08.823+0000 7fa5a1c00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa59c113d30 con 0x7fa59c100490 2024-03-21T10:29:08.824 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:08.824+0000 7fa59b400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa59c10e040 0x7fa59c19e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:29:08.825 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:08.824+0000 7fa59b400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa59c10e040 0x7fa59c19e9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:52878/0 (socket says 172.21.15.47:52878) 2024-03-21T10:29:08.825 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:08.824+0000 7fa592000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa59c100e40 0x7fa59c19a670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:29:08.825 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:08.824+0000 7fa59b400640 1 -- 172.21.15.47:0/649113193 learned_addr learned my addr 172.21.15.47:0/649113193 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:29:08.825 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:08.824+0000 7fa59aa00640 1 --2- 172.21.15.47:0/649113193 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa59c100490 0x7fa59c19a130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:29:08.825 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:08.824+0000 7fa59b400640 1 -- 172.21.15.47:0/649113193 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa59c100490 msgr2=0x7fa59c19a130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:29:08.825 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:08.824+0000 7fa59b400640 1 --2- 172.21.15.47:0/649113193 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa59c100490 0x7fa59c19a130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:29:08.825 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:08.824+0000 7fa59b400640 1 -- 172.21.15.47:0/649113193 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa59c100e40 msgr2=0x7fa59c19a670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:29:08.825 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:08.824+0000 7fa59b400640 1 --2- 172.21.15.47:0/649113193 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa59c100e40 0x7fa59c19a670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:29:08.825 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:08.824+0000 7fa59b400640 1 -- 172.21.15.47:0/649113193 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa59c19f120 con 0x7fa59c10e040 2024-03-21T10:29:08.825 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:08.824+0000 7fa59aa00640 1 --2- 172.21.15.47:0/649113193 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa59c100490 0x7fa59c19a130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T10:29:08.825 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:08.825+0000 7fa59b400640 1 --2- 172.21.15.47:0/649113193 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa59c10e040 0x7fa59c19e9f0 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7fa594005fd0 tx=0x7fa59400dbd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:29:08.825 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:08.825+0000 7fa593e00640 1 -- 172.21.15.47:0/649113193 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa594017070 con 0x7fa59c10e040 2024-03-21T10:29:08.825 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:08.825+0000 7fa5a1c00640 1 -- 172.21.15.47:0/649113193 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa59c19f410 con 0x7fa59c10e040 2024-03-21T10:29:08.825 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:08.825+0000 7fa593e00640 1 -- 172.21.15.47:0/649113193 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fa59400de20 con 0x7fa59c10e040 2024-03-21T10:29:08.826 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:08.825+0000 7fa593e00640 1 -- 172.21.15.47:0/649113193 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa594003bc0 con 0x7fa59c10e040 2024-03-21T10:29:08.826 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:08.825+0000 7fa5a1c00640 1 -- 172.21.15.47:0/649113193 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa59c1a6ca0 con 0x7fa59c10e040 2024-03-21T10:29:08.828 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:08.829+0000 7fa593e00640 1 -- 172.21.15.47:0/649113193 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 27) v1 ==== 137985+0+0 (secure 0 0 0) 0x7fa594003de0 con 0x7fa59c10e040 2024-03-21T10:29:08.829 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:08.830+0000 7fa5a498a740 1 Processor -- start 2024-03-21T10:29:08.830 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:08.830+0000 7fa593e00640 1 --2- 172.21.15.47:0/649113193 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7fa5800a3110 0x7fa5800a55d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:29:08.830 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:08.830+0000 7fa5a498a740 1 -- start start 2024-03-21T10:29:08.830 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:08.830+0000 7fa593e00640 1 -- 172.21.15.47:0/649113193 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 9474+0+0 (secure 0 0 0) 0x7fa5940cf450 con 0x7fa59c10e040 2024-03-21T10:29:08.830 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:08.830+0000 7fa592000640 1 --2- 172.21.15.47:0/649113193 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7fa5800a3110 0x7fa5800a55d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:29:08.832 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:08.831+0000 7fa5a498a740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559cf4c576d0 0x559cf4c57ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:29:08.832 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:08.831+0000 7fa5a498a740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x559cf4c58080 0x559cf4c60630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:29:08.832 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:08.831+0000 7fa5a498a740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559cf4c60c00 0x559cf4c63050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:29:08.832 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:08.831+0000 7fa5a498a740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x559cf4bbb2c0 con 0x559cf4c58080 2024-03-21T10:29:08.833 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:08.831+0000 7fa5a498a740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x559cf4b63c90 con 0x559cf4c60c00 2024-03-21T10:29:08.833 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:08.831+0000 7fa5a498a740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x559cf4bb7360 con 0x559cf4c576d0 2024-03-21T10:29:08.833 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:08.831+0000 7fa592000640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x559cf4c58080 0x559cf4c60630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:29:08.833 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:08.831+0000 7fa592000640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x559cf4c58080 0x559cf4c60630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:52880/0 (socket says 172.21.15.47:52880) 2024-03-21T10:29:08.833 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:08.831+0000 7fa592000640 1 -- 172.21.15.47:0/3927661187 learned_addr learned my addr 172.21.15.47:0/3927661187 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:29:08.833 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:08.831+0000 7fa59aa00640 1 --2- 172.21.15.47:0/3927661187 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559cf4c576d0 0x559cf4c57ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:29:08.833 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:08.831+0000 7fa59b400640 1 --2- 172.21.15.47:0/3927661187 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559cf4c60c00 0x559cf4c63050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:29:08.833 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:08.831+0000 7fa592000640 1 --2- 172.21.15.47:0/649113193 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7fa5800a3110 0x7fa5800a55d0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fa59c100070 tx=0x7fa58400bab0 comp rx=0 tx=0).ready entity=mgr.15687 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:29:08.833 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:08.832+0000 7fa592000640 1 -- 172.21.15.47:0/3927661187 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559cf4c576d0 msgr2=0x559cf4c57ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:29:08.833 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:08.832+0000 7fa592000640 1 --2- 172.21.15.47:0/3927661187 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559cf4c576d0 0x559cf4c57ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:29:08.833 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:08.832+0000 7fa592000640 1 -- 172.21.15.47:0/3927661187 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559cf4c60c00 msgr2=0x559cf4c63050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:29:08.833 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:08.832+0000 7fa592000640 1 --2- 172.21.15.47:0/3927661187 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559cf4c60c00 0x559cf4c63050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:29:08.833 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:08.832+0000 7fa592000640 1 -- 172.21.15.47:0/3927661187 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559cf4b88080 con 0x559cf4c58080 2024-03-21T10:29:08.833 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:08.832+0000 7fa59b400640 1 --2- 172.21.15.47:0/3927661187 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559cf4c60c00 0x559cf4c63050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T10:29:08.834 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:08.832+0000 7fa59aa00640 1 --2- 172.21.15.47:0/3927661187 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559cf4c576d0 0x559cf4c57ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T10:29:08.834 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:08.832+0000 7fa592000640 1 --2- 172.21.15.47:0/3927661187 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x559cf4c58080 0x559cf4c60630 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7fa58400b600 tx=0x7fa58401b000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=26ee902354b83e59 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:29:08.834 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:08.832+0000 7fa58be00640 1 -- 172.21.15.47:0/3927661187 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5840130d0 con 0x559cf4c58080 2024-03-21T10:29:08.834 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:08.832+0000 7fa58be00640 1 -- 172.21.15.47:0/3927661187 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fa584013270 con 0x559cf4c58080 2024-03-21T10:29:08.834 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:08.832+0000 7fa58be00640 1 -- 172.21.15.47:0/3927661187 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa58401e680 con 0x559cf4c58080 2024-03-21T10:29:08.834 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:08.833+0000 7fa5a498a740 1 -- 172.21.15.47:0/3927661187 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x559cf4c58080 msgr2=0x559cf4c60630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:29:08.834 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:08.833+0000 7fa5a498a740 1 --2- 172.21.15.47:0/3927661187 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x559cf4c58080 0x559cf4c60630 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7fa58400b600 tx=0x7fa58401b000 comp rx=0 tx=0).stop 2024-03-21T10:29:08.837 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:08.833+0000 7fa5a498a740 1 -- 172.21.15.47:0/3927661187 shutdown_connections 2024-03-21T10:29:08.837 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:08.833+0000 7fa5a498a740 1 --2- 172.21.15.47:0/3927661187 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559cf4c60c00 0x559cf4c63050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:29:08.838 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:08.833+0000 7fa5a498a740 1 --2- 172.21.15.47:0/3927661187 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x559cf4c58080 0x559cf4c60630 unknown :-1 s=CLOSED pgs=781 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:29:08.838 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:08.833+0000 7fa5a498a740 1 --2- 172.21.15.47:0/3927661187 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559cf4c576d0 0x559cf4c57ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:29:08.838 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:08.833+0000 7fa5a498a740 1 -- 172.21.15.47:0/3927661187 >> 172.21.15.47:0/3927661187 conn(0x559cf4c51da0 msgr2=0x559cf4c52180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:29:08.838 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:08.833+0000 7fa5a498a740 1 -- 172.21.15.47:0/3927661187 shutdown_connections 2024-03-21T10:29:08.838 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:08.833+0000 7fa5a498a740 1 -- 172.21.15.47:0/3927661187 wait complete. 2024-03-21T10:29:08.838 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:08.834+0000 7fa5a498a740 1 -- start start 2024-03-21T10:29:08.838 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:08.834+0000 7fa5a498a740 1 Processor -- start 2024-03-21T10:29:08.838 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:08.834+0000 7fa5a498a740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x559cf4c576d0 0x559cf4c60550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:29:08.838 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:08.835+0000 7fa5a498a740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559cf4c58080 0x559cf4c5d8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:29:08.838 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:08.835+0000 7fa5a498a740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559cf4c60c00 0x559cf4c5def0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:29:08.838 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:08.835+0000 7fa5a498a740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x559cf4c63b70 con 0x559cf4c576d0 2024-03-21T10:29:08.838 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:08.835+0000 7fa5a498a740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x559cf4b770d0 con 0x559cf4c58080 2024-03-21T10:29:08.838 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:08.835+0000 7fa5a498a740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x559cf4c63cf0 con 0x559cf4c60c00 2024-03-21T10:29:08.839 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:08.835+0000 7fa5a498a740 10 client.0 _ll_register_callbacks cb 0x559cf4b7be80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T10:29:08.839 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:08.835+0000 7fa59aa00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x559cf4c576d0 0x559cf4c60550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:29:08.839 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:08.835+0000 7fa59aa00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x559cf4c576d0 0x559cf4c60550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:52892/0 (socket says 172.21.15.47:52892) 2024-03-21T10:29:08.839 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:08.835+0000 7fa59aa00640 1 -- 172.21.15.47:0/3454866947 learned_addr learned my addr 172.21.15.47:0/3454866947 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:29:08.839 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:08.835+0000 7fa592000640 1 --2- 172.21.15.47:0/3454866947 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559cf4c58080 0x559cf4c5d8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:29:08.839 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:08.835+0000 7fa59b400640 1 --2- 172.21.15.47:0/3454866947 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559cf4c60c00 0x559cf4c5def0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:29:08.839 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:08.835+0000 7fa59aa00640 1 -- 172.21.15.47:0/3454866947 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559cf4c60c00 msgr2=0x559cf4c5def0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:29:08.839 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:08.835+0000 7fa59aa00640 1 --2- 172.21.15.47:0/3454866947 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559cf4c60c00 0x559cf4c5def0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:29:08.839 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:08.835+0000 7fa59aa00640 1 -- 172.21.15.47:0/3454866947 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559cf4c58080 msgr2=0x559cf4c5d8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:29:08.839 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:08.835+0000 7fa59aa00640 1 --2- 172.21.15.47:0/3454866947 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559cf4c58080 0x559cf4c5d8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:29:08.839 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:08.835+0000 7fa59aa00640 1 -- 172.21.15.47:0/3454866947 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x559cf48cf3f0 con 0x559cf4c576d0 2024-03-21T10:29:08.839 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:08.835+0000 7fa592000640 1 --2- 172.21.15.47:0/3454866947 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559cf4c58080 0x559cf4c5d8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T10:29:08.839 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:08.836+0000 7fa59aa00640 1 --2- 172.21.15.47:0/3454866947 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x559cf4c576d0 0x559cf4c60550 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7fa58c037040 tx=0x7fa58c032540 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8b71d88810b889e4 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:29:08.839 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:08.836+0000 7fa588c00640 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-03-21T10:29:08.840 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:08.836+0000 7fa588c00640 1 -- 172.21.15.47:0/3454866947 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa58c043070 con 0x559cf4c576d0 2024-03-21T10:29:08.840 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:08.836+0000 7fa588c00640 1 -- 172.21.15.47:0/3454866947 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(97..97 src has 1..97) v4 ==== 9474+0+0 (secure 0 0 0) 0x7fa58c003970 con 0x559cf4c576d0 2024-03-21T10:29:08.840 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:08.836+0000 7fa5a498a740 1 -- 172.21.15.47:0/3454866947 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fa584013270 con 0x559cf4c576d0 2024-03-21T10:29:08.840 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:08.836+0000 7fa588c00640 1 -- 172.21.15.47:0/3454866947 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa58c04f050 con 0x559cf4c576d0 2024-03-21T10:29:08.840 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:08.836+0000 7fa5a498a740 10 client.16908 fetch_fsmap learned FSMap version 32 2024-03-21T10:29:08.840 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:08.836+0000 7fa5a498a740 1 -- 172.21.15.47:0/3454866947 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x559cf49997d0 con 0x559cf4c576d0 2024-03-21T10:29:08.840 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:08.837+0000 7fa588c00640 1 -- 172.21.15.47:0/3454866947 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fa584013270 con 0x559cf4c576d0 2024-03-21T10:29:08.840 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:08.837+0000 7fa588c00640 1 -- 172.21.15.47:0/3454866947 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 32) v1 ==== 7942+0+0 (secure 0 0 0) 0x7fa58c034440 con 0x559cf4c576d0 2024-03-21T10:29:08.840 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:08.837+0000 7fa5a498a740 10 client.16908 fetch_fsmap finished waiting for FSMap version 32 2024-03-21T10:29:08.840 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:08.837+0000 7fa5a498a740 10 client.16908 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-03-21T10:29:08.840 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:08.837+0000 7fa5a498a740 20 client.16908 populate_metadata read hostname 'smithi047' 2024-03-21T10:29:08.840 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:08.837+0000 7fa5a498a740 1 --2- 172.21.15.47:0/3454866947 >> [v2:172.21.15.100:6836/3497644162,v1:172.21.15.100:6837/3497644162] conn(0x559cf4c70a90 0x559cf4c72e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:29:08.840 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:08.837+0000 7fa5a498a740 4 client.16908 mds_command: new command op to 34493 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T10:29:08.840 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:08.837+0000 7fa5a498a740 1 -- 172.21.15.47:0/3454866947 --> [v2:172.21.15.100:6836/3497644162,v1:172.21.15.100:6837/3497644162] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x559cf4bc8800 con 0x559cf4c70a90 2024-03-21T10:29:08.841 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:08.838+0000 7fa592000640 1 --2- 172.21.15.47:0/3454866947 >> [v2:172.21.15.100:6836/3497644162,v1:172.21.15.100:6837/3497644162] conn(0x559cf4c70a90 0x559cf4c72e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:29:08.842 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:08.839+0000 7fa592000640 1 --2- 172.21.15.47:0/3454866947 >> [v2:172.21.15.100:6836/3497644162,v1:172.21.15.100:6837/3497644162] conn(0x559cf4c70a90 0x559cf4c72e80 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=d7c675deedb29d48 server_cookie=47529e5964eae221 in_seq=0 out_seq=0 2024-03-21T10:29:08.842 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:08.839+0000 7fa588c00640 10 client.16908 ms_handle_connect on v2:172.21.15.100:6836/3497644162 2024-03-21T10:29:08.842 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:08.840+0000 7fa588c00640 1 -- 172.21.15.47:0/3454866947 <== mds.1 v2:172.21.15.100:6836/3497644162 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x559cf4bc8800 con 0x559cf4c70a90 2024-03-21T10:29:08.842 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:08.840+0000 7fa588c00640 10 client.16908 handle_command_reply: tid=0 2024-03-21T10:29:08.842 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:08.840+0000 7fa5a498a740 1 client.16908 shutdown 2024-03-21T10:29:08.842 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:08.840+0000 7fa5a498a740 1 -- 172.21.15.47:0/3454866947 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x559cf4c576d0 msgr2=0x559cf4c60550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:29:08.842 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:08.840+0000 7fa5a498a740 1 --2- 172.21.15.47:0/3454866947 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x559cf4c576d0 0x559cf4c60550 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7fa58c037040 tx=0x7fa58c032540 comp rx=0 tx=0).stop 2024-03-21T10:29:08.842 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:08.840+0000 7fa5a498a740 1 -- 172.21.15.47:0/3454866947 shutdown_connections 2024-03-21T10:29:08.842 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:08.840+0000 7fa5a498a740 1 --2- 172.21.15.47:0/3454866947 >> [v2:172.21.15.100:6836/3497644162,v1:172.21.15.100:6837/3497644162] conn(0x559cf4c70a90 0x559cf4c72e80 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:29:08.842 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:08.840+0000 7fa588c00640 0 client.16908 ms_handle_reset on v2:172.21.15.100:6836/3497644162 2024-03-21T10:29:08.842 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:08.840+0000 7fa5a498a740 1 --2- 172.21.15.47:0/3454866947 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559cf4c60c00 0x559cf4c5def0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:29:08.842 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:08.840+0000 7fa5a498a740 1 --2- 172.21.15.47:0/3454866947 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559cf4c58080 0x559cf4c5d8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:29:08.843 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:08.840+0000 7fa5a498a740 1 --2- 172.21.15.47:0/3454866947 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x559cf4c576d0 0x559cf4c60550 unknown :-1 s=CLOSED pgs=782 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:29:08.843 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:08.840+0000 7fa5a498a740 1 -- 172.21.15.47:0/3454866947 >> 172.21.15.47:0/3454866947 conn(0x559cf4c51da0 msgr2=0x559cf4c53af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:29:08.843 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:08.840+0000 7fa5a498a740 1 -- 172.21.15.47:0/3454866947 shutdown_connections 2024-03-21T10:29:08.843 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:08.840+0000 7fa5a498a740 1 -- 172.21.15.47:0/3454866947 wait complete. 2024-03-21T10:29:08.843 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:08.840+0000 7fa5a498a740 20 client.16908 trim_cache size 0 max 16384 2024-03-21T10:29:08.854 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:08.854+0000 7fa5a498a740 1 Processor -- start 2024-03-21T10:29:08.854 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:08.854+0000 7fa5a498a740 1 -- start start 2024-03-21T10:29:08.854 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:08.854+0000 7fa5a498a740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559cf4c576d0 0x559cf4c60010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:29:08.854 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:08.854+0000 7fa5a498a740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559cf4c58080 0x559cf4c60550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:29:08.854 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:08.854+0000 7fa5a498a740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x559cf4c60c00 0x559cf4c5aba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:29:08.854 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:08.854+0000 7fa5a498a740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa59c1b0a50 con 0x559cf4c60c00 2024-03-21T10:29:08.854 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:08.854+0000 7fa5a498a740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa59c1b23d0 con 0x559cf4c58080 2024-03-21T10:29:08.854 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:08.854+0000 7fa5a498a740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa59c1b10b0 con 0x559cf4c576d0 2024-03-21T10:29:08.856 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:08.855+0000 7fa59b400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x559cf4c60c00 0x559cf4c5aba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:29:08.856 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:08.855+0000 7fa59b400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x559cf4c60c00 0x559cf4c5aba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:52900/0 (socket says 172.21.15.47:52900) 2024-03-21T10:29:08.856 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:08.855+0000 7fa59b400640 1 -- 172.21.15.47:0/2255783998 learned_addr learned my addr 172.21.15.47:0/2255783998 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:29:08.856 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:08.855+0000 7fa592000640 1 --2- 172.21.15.47:0/2255783998 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559cf4c58080 0x559cf4c60550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:29:08.856 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:08.855+0000 7fa59aa00640 1 --2- 172.21.15.47:0/2255783998 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559cf4c576d0 0x559cf4c60010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:29:08.857 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:08.855+0000 7fa59b400640 1 -- 172.21.15.47:0/2255783998 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559cf4c576d0 msgr2=0x559cf4c60010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:29:08.857 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:08.855+0000 7fa59b400640 1 --2- 172.21.15.47:0/2255783998 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559cf4c576d0 0x559cf4c60010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:29:08.857 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:08.855+0000 7fa59b400640 1 -- 172.21.15.47:0/2255783998 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559cf4c58080 msgr2=0x559cf4c60550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:29:08.857 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:08.855+0000 7fa59b400640 1 --2- 172.21.15.47:0/2255783998 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559cf4c58080 0x559cf4c60550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:29:08.857 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:08.855+0000 7fa59b400640 1 -- 172.21.15.47:0/2255783998 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559cf4c5e260 con 0x559cf4c60c00 2024-03-21T10:29:08.857 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:08.855+0000 7fa59aa00640 1 --2- 172.21.15.47:0/2255783998 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559cf4c576d0 0x559cf4c60010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T10:29:08.857 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:08.855+0000 7fa59b400640 1 --2- 172.21.15.47:0/2255783998 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x559cf4c60c00 0x559cf4c5aba0 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x7fa594097880 tx=0x7fa594095f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e078cfc6b5425b44 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:29:08.857 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:08.856+0000 7fa58b400640 1 -- 172.21.15.47:0/2255783998 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa594020070 con 0x559cf4c60c00 2024-03-21T10:29:08.857 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:08.856+0000 7fa58b400640 1 -- 172.21.15.47:0/2255783998 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fa5940a3e30 con 0x559cf4c60c00 2024-03-21T10:29:08.857 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:08.856+0000 7fa58b400640 1 -- 172.21.15.47:0/2255783998 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa594097bf0 con 0x559cf4c60c00 2024-03-21T10:29:08.857 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:08.856+0000 7fa5a498a740 1 -- 172.21.15.47:0/2255783998 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x559cf4c60c00 msgr2=0x559cf4c5aba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:29:08.857 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:08.856+0000 7fa5a498a740 1 --2- 172.21.15.47:0/2255783998 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x559cf4c60c00 0x559cf4c5aba0 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x7fa594097880 tx=0x7fa594095f90 comp rx=0 tx=0).stop 2024-03-21T10:29:08.857 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:08.856+0000 7fa5a498a740 1 -- 172.21.15.47:0/2255783998 shutdown_connections 2024-03-21T10:29:08.858 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:08.856+0000 7fa5a498a740 1 --2- 172.21.15.47:0/2255783998 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x559cf4c60c00 0x559cf4c5aba0 unknown :-1 s=CLOSED pgs=783 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:29:08.858 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:08.856+0000 7fa5a498a740 1 --2- 172.21.15.47:0/2255783998 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559cf4c58080 0x559cf4c60550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:29:08.858 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:08.856+0000 7fa5a498a740 1 --2- 172.21.15.47:0/2255783998 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559cf4c576d0 0x559cf4c60010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:29:08.858 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:08.856+0000 7fa5a498a740 1 -- 172.21.15.47:0/2255783998 >> 172.21.15.47:0/2255783998 conn(0x559cf4c51da0 msgr2=0x559cf4c50eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:29:08.858 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:08.857+0000 7fa5a498a740 1 -- 172.21.15.47:0/2255783998 shutdown_connections 2024-03-21T10:29:08.858 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:08.857+0000 7fa5a498a740 1 -- 172.21.15.47:0/2255783998 wait complete. 2024-03-21T10:29:08.858 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:08.857+0000 7fa5a498a740 1 -- start start 2024-03-21T10:29:08.858 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:08.857+0000 7fa5a498a740 1 Processor -- start 2024-03-21T10:29:08.862 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:08.858+0000 7fa5a498a740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559cf4c576d0 0x559cf4c66b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:29:08.862 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:08.858+0000 7fa5a498a740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559cf4c58080 0x559cf4c67070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:29:08.862 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:08.858+0000 7fa5a498a740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x559cf4c60c00 0x559cf4ccb300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:29:08.862 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:08.858+0000 7fa5a498a740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa59c1b1d70 con 0x559cf4c60c00 2024-03-21T10:29:08.862 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:08.858+0000 7fa5a498a740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa59c1b1710 con 0x559cf4c576d0 2024-03-21T10:29:08.863 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:08.858+0000 7fa5a498a740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa59c1aec10 con 0x559cf4c58080 2024-03-21T10:29:08.863 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:08.858+0000 7fa5a498a740 10 client.0 _ll_register_callbacks cb 0x7fa58c037070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T10:29:08.863 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:08.858+0000 7fa59b400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x559cf4c60c00 0x559cf4ccb300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:29:08.863 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:08.858+0000 7fa59b400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x559cf4c60c00 0x559cf4ccb300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:52912/0 (socket says 172.21.15.47:52912) 2024-03-21T10:29:08.863 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:08.858+0000 7fa59b400640 1 -- 172.21.15.47:0/3267407195 learned_addr learned my addr 172.21.15.47:0/3267407195 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:29:08.863 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:08.858+0000 7fa59aa00640 1 --2- 172.21.15.47:0/3267407195 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559cf4c576d0 0x559cf4c66b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:29:08.863 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:08.858+0000 7fa592000640 1 --2- 172.21.15.47:0/3267407195 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559cf4c58080 0x559cf4c67070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:29:08.863 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:08.858+0000 7fa59b400640 1 -- 172.21.15.47:0/3267407195 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559cf4c58080 msgr2=0x559cf4c67070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:29:08.863 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:08.858+0000 7fa59b400640 1 --2- 172.21.15.47:0/3267407195 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559cf4c58080 0x559cf4c67070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:29:08.863 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:08.858+0000 7fa59b400640 1 -- 172.21.15.47:0/3267407195 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559cf4c576d0 msgr2=0x559cf4c66b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:29:08.863 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:08.858+0000 7fa59b400640 1 --2- 172.21.15.47:0/3267407195 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559cf4c576d0 0x559cf4c66b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:29:08.863 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:08.858+0000 7fa59b400640 1 -- 172.21.15.47:0/3267407195 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x559cf4ccb8f0 con 0x559cf4c60c00 2024-03-21T10:29:08.863 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:08.858+0000 7fa592000640 1 --2- 172.21.15.47:0/3267407195 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559cf4c58080 0x559cf4c67070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T10:29:08.863 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:08.858+0000 7fa59aa00640 1 --2- 172.21.15.47:0/3267407195 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559cf4c576d0 0x559cf4c66b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T10:29:08.864 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:08.859+0000 7fa588c00640 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-03-21T10:29:08.864 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:08.859+0000 7fa59b400640 1 --2- 172.21.15.47:0/3267407195 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x559cf4c60c00 0x559cf4ccb300 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7fa59409b3b0 tx=0x7fa59409f6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1f44822a1869c183 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:29:08.864 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:08.859+0000 7fa588c00640 1 -- 172.21.15.47:0/3267407195 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa594020040 con 0x559cf4c60c00 2024-03-21T10:29:08.864 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:08.859+0000 7fa588c00640 1 -- 172.21.15.47:0/3267407195 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(97..97 src has 1..97) v4 ==== 9474+0+0 (secure 0 0 0) 0x7fa59409f120 con 0x559cf4c60c00 2024-03-21T10:29:08.864 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:08.859+0000 7fa5a498a740 1 -- 172.21.15.47:0/3267407195 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fa5940a3e30 con 0x559cf4c60c00 2024-03-21T10:29:08.864 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:08.859+0000 7fa588c00640 1 -- 172.21.15.47:0/3267407195 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa594009d80 con 0x559cf4c60c00 2024-03-21T10:29:08.864 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:08.859+0000 7fa5a498a740 10 client.16914 fetch_fsmap learned FSMap version 32 2024-03-21T10:29:08.864 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:08.859+0000 7fa5a498a740 1 -- 172.21.15.47:0/3267407195 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x559cf4cccb40 con 0x559cf4c60c00 2024-03-21T10:29:08.864 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:08.860+0000 7fa588c00640 1 -- 172.21.15.47:0/3267407195 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fa5940a3e30 con 0x559cf4c60c00 2024-03-21T10:29:08.864 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:08.860+0000 7fa588c00640 1 -- 172.21.15.47:0/3267407195 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 32) v1 ==== 7942+0+0 (secure 0 0 0) 0x7fa594097170 con 0x559cf4c60c00 2024-03-21T10:29:08.864 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:08.860+0000 7fa5a498a740 10 client.16914 fetch_fsmap finished waiting for FSMap version 32 2024-03-21T10:29:08.864 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:08.860+0000 7fa5a498a740 10 client.16914 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-03-21T10:29:08.864 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:08.860+0000 7fa5a498a740 20 client.16914 populate_metadata read hostname 'smithi047' 2024-03-21T10:29:08.864 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:08.860+0000 7fa5a498a740 1 --2- 172.21.15.47:0/3267407195 >> [v2:172.21.15.100:6836/3497644162,v1:172.21.15.100:6837/3497644162] conn(0x559cf4c70a90 0x559cf4ccf120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:29:08.865 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:08.860+0000 7fa5a498a740 4 client.16914 mds_command: new command op to 34493 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-21T10:29:08.865 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:08.860+0000 7fa5a498a740 1 -- 172.21.15.47:0/3267407195 --> [v2:172.21.15.100:6836/3497644162,v1:172.21.15.100:6837/3497644162] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x559cf4bc2b50 con 0x559cf4c70a90 2024-03-21T10:29:08.865 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:08.861+0000 7fa59aa00640 1 --2- 172.21.15.47:0/3267407195 >> [v2:172.21.15.100:6836/3497644162,v1:172.21.15.100:6837/3497644162] conn(0x559cf4c70a90 0x559cf4ccf120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:29:08.865 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:08.861+0000 7fa59aa00640 1 --2- 172.21.15.47:0/3267407195 >> [v2:172.21.15.100:6836/3497644162,v1:172.21.15.100:6837/3497644162] conn(0x559cf4c70a90 0x559cf4ccf120 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=b61443b48083b7a2 server_cookie=8aaaeaf48ba561f5 in_seq=0 out_seq=0 2024-03-21T10:29:08.865 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:08.861+0000 7fa588c00640 10 client.16914 ms_handle_connect on v2:172.21.15.100:6836/3497644162 2024-03-21T10:29:08.865 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:08.862+0000 7fa588c00640 1 -- 172.21.15.47:0/3267407195 <== mds.1 v2:172.21.15.100:6836/3497644162 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x559cf4bc2b50 con 0x559cf4c70a90 2024-03-21T10:29:08.865 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:08.862+0000 7fa588c00640 10 client.16914 handle_command_reply: tid=0 2024-03-21T10:29:08.865 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:08.862+0000 7fa5a498a740 1 client.16914 shutdown 2024-03-21T10:29:08.865 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:08.862+0000 7fa5a498a740 1 -- 172.21.15.47:0/3267407195 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x559cf4c60c00 msgr2=0x559cf4ccb300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:29:08.865 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:08.862+0000 7fa5a498a740 1 --2- 172.21.15.47:0/3267407195 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x559cf4c60c00 0x559cf4ccb300 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7fa59409b3b0 tx=0x7fa59409f6f0 comp rx=0 tx=0).stop 2024-03-21T10:29:08.865 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:08.862+0000 7fa5a498a740 1 -- 172.21.15.47:0/3267407195 shutdown_connections 2024-03-21T10:29:08.865 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:08.862+0000 7fa5a498a740 1 --2- 172.21.15.47:0/3267407195 >> [v2:172.21.15.100:6836/3497644162,v1:172.21.15.100:6837/3497644162] conn(0x559cf4c70a90 0x559cf4ccf120 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:29:08.865 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:08.862+0000 7fa5a498a740 1 --2- 172.21.15.47:0/3267407195 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x559cf4c60c00 0x559cf4ccb300 unknown :-1 s=CLOSED pgs=784 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:29:08.865 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:08.862+0000 7fa5a498a740 1 --2- 172.21.15.47:0/3267407195 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559cf4c58080 0x559cf4c67070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:29:08.865 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:08.862+0000 7fa5a498a740 1 --2- 172.21.15.47:0/3267407195 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559cf4c576d0 0x559cf4c66b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:29:08.866 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:08.862+0000 7fa5a498a740 1 -- 172.21.15.47:0/3267407195 >> 172.21.15.47:0/3267407195 conn(0x559cf4c51da0 msgr2=0x559cf4c584b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:29:08.866 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:08.862+0000 7fa588c00640 0 client.16914 ms_handle_reset on v2:172.21.15.100:6836/3497644162 2024-03-21T10:29:08.866 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:08.862+0000 7fa5a498a740 1 -- 172.21.15.47:0/3267407195 shutdown_connections 2024-03-21T10:29:08.866 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:08.862+0000 7fa5a498a740 1 -- 172.21.15.47:0/3267407195 wait complete. 2024-03-21T10:29:08.866 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:08.862+0000 7fa5a498a740 20 client.16914 trim_cache size 0 max 16384 2024-03-21T10:29:08.866 INFO:teuthology.orchestra.run.smithi047.stdout:[] 2024-03-21T10:29:08.867 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:08.863+0000 7fa5a1c00640 1 -- 172.21.15.47:0/649113193 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7fa5800a3110 msgr2=0x7fa5800a55d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:29:08.867 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:08.863+0000 7fa5a1c00640 1 --2- 172.21.15.47:0/649113193 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7fa5800a3110 0x7fa5800a55d0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fa59c100070 tx=0x7fa58400bab0 comp rx=0 tx=0).stop 2024-03-21T10:29:08.867 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:08.863+0000 7fa5a1c00640 1 -- 172.21.15.47:0/649113193 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa59c10e040 msgr2=0x7fa59c19e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:29:08.867 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:08.863+0000 7fa5a1c00640 1 --2- 172.21.15.47:0/649113193 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa59c10e040 0x7fa59c19e9f0 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7fa594005fd0 tx=0x7fa59400dbd0 comp rx=0 tx=0).stop 2024-03-21T10:29:08.867 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:08.863+0000 7fa5a1c00640 1 -- 172.21.15.47:0/649113193 shutdown_connections 2024-03-21T10:29:08.867 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:08.863+0000 7fa5a1c00640 1 --2- 172.21.15.47:0/649113193 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa59c10e040 0x7fa59c19e9f0 unknown :-1 s=CLOSED pgs=780 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:29:08.867 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:08.863+0000 7fa5a1c00640 1 --2- 172.21.15.47:0/649113193 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7fa5800a3110 0x7fa5800a55d0 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:29:08.867 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:08.863+0000 7fa5a1c00640 1 --2- 172.21.15.47:0/649113193 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa59c100e40 0x7fa59c19a670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:29:08.867 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:08.863+0000 7fa5a1c00640 1 --2- 172.21.15.47:0/649113193 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa59c100490 0x7fa59c19a130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:29:08.868 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:08.863+0000 7fa5a1c00640 1 -- 172.21.15.47:0/649113193 >> 172.21.15.47:0/649113193 conn(0x7fa59c077700 msgr2=0x7fa59c10f6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:29:08.868 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:08.863+0000 7fa5a1c00640 1 -- 172.21.15.47:0/649113193 shutdown_connections 2024-03-21T10:29:08.868 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:08.863+0000 7fa5a1c00640 1 -- 172.21.15.47:0/649113193 wait complete. 2024-03-21T10:29:08.883 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-21T10:29:08.883 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-03-21T10:29:08.978 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:29:08 smithi100 ceph-mon[35193]: from='client.? 172.21.15.47:0/2764959579' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-21T10:29:08.988 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:29:08 smithi157 ceph-mon[35581]: from='client.? 172.21.15.47:0/2764959579' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-21T10:29:09.104 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:09.104+0000 7fa957c00640 1 -- 172.21.15.47:0/575563756 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa950104200 msgr2=0x7fa950110710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:29:09.104 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:09.104+0000 7fa957c00640 1 --2- 172.21.15.47:0/575563756 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa950104200 0x7fa950110710 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7fa948009cd0 tx=0x7fa94802c0f0 comp rx=0 tx=0).stop 2024-03-21T10:29:09.104 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:09.104+0000 7fa957c00640 1 -- 172.21.15.47:0/575563756 shutdown_connections 2024-03-21T10:29:09.104 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:09.104+0000 7fa957c00640 1 --2- 172.21.15.47:0/575563756 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa950104200 0x7fa950110710 unknown :-1 s=CLOSED pgs=785 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:29:09.104 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:09.104+0000 7fa957c00640 1 --2- 172.21.15.47:0/575563756 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa950103840 0x7fa950103cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:29:09.104 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:09.104+0000 7fa957c00640 1 --2- 172.21.15.47:0/575563756 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa950109850 0x7fa950109c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:29:09.104 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:09.104+0000 7fa957c00640 1 -- 172.21.15.47:0/575563756 >> 172.21.15.47:0/575563756 conn(0x7fa9500ff2f0 msgr2=0x7fa950101710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:29:09.104 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:09.104+0000 7fa957c00640 1 -- 172.21.15.47:0/575563756 shutdown_connections 2024-03-21T10:29:09.104 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:09.104+0000 7fa957c00640 1 -- 172.21.15.47:0/575563756 wait complete. 2024-03-21T10:29:09.106 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:09.105+0000 7fa957c00640 1 Processor -- start 2024-03-21T10:29:09.106 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:09.105+0000 7fa957c00640 1 -- start start 2024-03-21T10:29:09.106 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:09.106+0000 7fa957c00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa950103840 0x7fa95019a090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:29:09.106 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:09.106+0000 7fa957c00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa950104200 0x7fa95019a5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:29:09.106 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:09.106+0000 7fa957c00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa950109850 0x7fa95019e900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:29:09.106 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:09.106+0000 7fa957c00640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa950113b10 con 0x7fa950103840 2024-03-21T10:29:09.106 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:09.106+0000 7fa957c00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa950113990 con 0x7fa950104200 2024-03-21T10:29:09.106 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:09.106+0000 7fa957c00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa950113c90 con 0x7fa950109850 2024-03-21T10:29:09.106 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:09.106+0000 7fa955200640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa950103840 0x7fa95019a090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:29:09.106 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:09.106+0000 7fa955200640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa950103840 0x7fa95019a090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:52930/0 (socket says 172.21.15.47:52930) 2024-03-21T10:29:09.106 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:09.106+0000 7fa955200640 1 -- 172.21.15.47:0/1112690482 learned_addr learned my addr 172.21.15.47:0/1112690482 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:29:09.106 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:09.106+0000 7fa94fe00640 1 --2- 172.21.15.47:0/1112690482 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa950104200 0x7fa95019a5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:29:09.106 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:09.107+0000 7fa955c00640 1 --2- 172.21.15.47:0/1112690482 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa950109850 0x7fa95019e900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:29:09.108 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:09.107+0000 7fa955200640 1 -- 172.21.15.47:0/1112690482 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa950109850 msgr2=0x7fa95019e900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:29:09.108 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:09.107+0000 7fa955200640 1 --2- 172.21.15.47:0/1112690482 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa950109850 0x7fa95019e900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:29:09.108 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:09.107+0000 7fa955200640 1 -- 172.21.15.47:0/1112690482 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa950104200 msgr2=0x7fa95019a5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:29:09.108 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:09.107+0000 7fa955200640 1 --2- 172.21.15.47:0/1112690482 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa950104200 0x7fa95019a5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:29:09.108 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:09.107+0000 7fa955200640 1 -- 172.21.15.47:0/1112690482 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa95019f030 con 0x7fa950103840 2024-03-21T10:29:09.108 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:09.107+0000 7fa94fe00640 1 --2- 172.21.15.47:0/1112690482 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa950104200 0x7fa95019a5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T10:29:09.108 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:09.107+0000 7fa955200640 1 --2- 172.21.15.47:0/1112690482 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa950103840 0x7fa95019a090 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7fa94400e9f0 tx=0x7fa94400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:29:09.108 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:09.107+0000 7fa94d600640 1 -- 172.21.15.47:0/1112690482 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa94400cd80 con 0x7fa950103840 2024-03-21T10:29:09.108 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:09.108+0000 7fa957c00640 1 -- 172.21.15.47:0/1112690482 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa95019f320 con 0x7fa950103840 2024-03-21T10:29:09.108 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:09.108+0000 7fa94d600640 1 -- 172.21.15.47:0/1112690482 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fa9440108e0 con 0x7fa950103840 2024-03-21T10:29:09.108 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:09.108+0000 7fa957c00640 1 -- 172.21.15.47:0/1112690482 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa9501a6bb0 con 0x7fa950103840 2024-03-21T10:29:09.108 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:09.108+0000 7fa94d600640 1 -- 172.21.15.47:0/1112690482 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9440188c0 con 0x7fa950103840 2024-03-21T10:29:09.111 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:09.112+0000 7fa95a989740 1 Processor -- start 2024-03-21T10:29:09.112 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:09.112+0000 7fa95a989740 1 -- start start 2024-03-21T10:29:09.112 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:09.112+0000 7fa94d600640 1 -- 172.21.15.47:0/1112690482 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 27) v1 ==== 137985+0+0 (secure 0 0 0) 0x7fa944010a90 con 0x7fa950103840 2024-03-21T10:29:09.112 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:09.113+0000 7fa95a989740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559f9fd856d0 0x559f9fd85ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:29:09.112 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:09.113+0000 7fa95a989740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559f9fd86080 0x559f9fd8e630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:29:09.113 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:09.113+0000 7fa95a989740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x559f9fd8ec00 0x559f9fd91050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:29:09.113 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:09.113+0000 7fa95a989740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x559f9fce92c0 con 0x559f9fd8ec00 2024-03-21T10:29:09.113 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:09.113+0000 7fa95a989740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x559f9fc91c90 con 0x559f9fd86080 2024-03-21T10:29:09.113 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:09.113+0000 7fa95a989740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x559f9fce5360 con 0x559f9fd856d0 2024-03-21T10:29:09.113 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:09.113+0000 7fa94fe00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x559f9fd8ec00 0x559f9fd91050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:29:09.113 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:09.113+0000 7fa955200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559f9fd86080 0x559f9fd8e630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:29:09.113 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:09.113+0000 7fa94fe00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x559f9fd8ec00 0x559f9fd91050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:52942/0 (socket says 172.21.15.47:52942) 2024-03-21T10:29:09.113 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:09.113+0000 7fa94fe00640 1 -- 172.21.15.47:0/1071679925 learned_addr learned my addr 172.21.15.47:0/1071679925 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:29:09.115 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:09.113+0000 7fa955c00640 1 --2- 172.21.15.47:0/1071679925 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559f9fd856d0 0x559f9fd85ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:29:09.115 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:09.114+0000 7fa94fe00640 1 -- 172.21.15.47:0/1071679925 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559f9fd856d0 msgr2=0x559f9fd85ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:29:09.115 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:09.114+0000 7fa94fe00640 1 --2- 172.21.15.47:0/1071679925 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559f9fd856d0 0x559f9fd85ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:29:09.115 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:09.114+0000 7fa94fe00640 1 -- 172.21.15.47:0/1071679925 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559f9fd86080 msgr2=0x559f9fd8e630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:29:09.115 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:09.114+0000 7fa94fe00640 1 --2- 172.21.15.47:0/1071679925 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559f9fd86080 0x559f9fd8e630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:29:09.115 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:09.114+0000 7fa94fe00640 1 -- 172.21.15.47:0/1071679925 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559f9fcb6080 con 0x559f9fd8ec00 2024-03-21T10:29:09.115 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:09.114+0000 7fa94fe00640 1 --2- 172.21.15.47:0/1071679925 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x559f9fd8ec00 0x559f9fd91050 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7fa93c010450 tx=0x7fa93c010910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1030d599ab6b9d21 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:29:09.115 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:09.114+0000 7fa94d600640 1 --2- 172.21.15.47:0/1112690482 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7fa9380a3090 0x7fa9380a5550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:29:09.115 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:09.114+0000 7fa942000640 1 -- 172.21.15.47:0/1071679925 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa93c011760 con 0x559f9fd8ec00 2024-03-21T10:29:09.116 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:09.114+0000 7fa942000640 1 -- 172.21.15.47:0/1071679925 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fa93c011db0 con 0x559f9fd8ec00 2024-03-21T10:29:09.116 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:09.114+0000 7fa94d600640 1 -- 172.21.15.47:0/1112690482 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 9474+0+0 (secure 0 0 0) 0x7fa9440cff30 con 0x7fa950103840 2024-03-21T10:29:09.116 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:09.115+0000 7fa955c00640 1 --2- 172.21.15.47:0/1112690482 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7fa9380a3090 0x7fa9380a5550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:29:09.116 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:09.115+0000 7fa95a989740 1 -- 172.21.15.47:0/1071679925 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x559f9fd8ec00 msgr2=0x559f9fd91050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:29:09.116 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:09.115+0000 7fa95a989740 1 --2- 172.21.15.47:0/1071679925 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x559f9fd8ec00 0x559f9fd91050 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7fa93c010450 tx=0x7fa93c010910 comp rx=0 tx=0).stop 2024-03-21T10:29:09.116 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:09.115+0000 7fa955c00640 1 --2- 172.21.15.47:0/1112690482 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7fa9380a3090 0x7fa9380a5550 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fa94802c630 tx=0x7fa94802e040 comp rx=0 tx=0).ready entity=mgr.15687 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:29:09.116 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:09.115+0000 7fa95a989740 1 -- 172.21.15.47:0/1071679925 shutdown_connections 2024-03-21T10:29:09.116 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:09.115+0000 7fa95a989740 1 --2- 172.21.15.47:0/1071679925 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x559f9fd8ec00 0x559f9fd91050 unknown :-1 s=CLOSED pgs=787 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:29:09.116 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:09.115+0000 7fa95a989740 1 --2- 172.21.15.47:0/1071679925 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559f9fd86080 0x559f9fd8e630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:29:09.116 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:09.115+0000 7fa95a989740 1 --2- 172.21.15.47:0/1071679925 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559f9fd856d0 0x559f9fd85ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:29:09.116 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:09.115+0000 7fa95a989740 1 -- 172.21.15.47:0/1071679925 >> 172.21.15.47:0/1071679925 conn(0x559f9fd7fda0 msgr2=0x559f9fd80180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:29:09.116 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:09.116+0000 7fa95a989740 1 -- 172.21.15.47:0/1071679925 shutdown_connections 2024-03-21T10:29:09.116 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:09.116+0000 7fa95a989740 1 -- 172.21.15.47:0/1071679925 wait complete. 2024-03-21T10:29:09.119 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:09.116+0000 7fa95a989740 1 -- start start 2024-03-21T10:29:09.119 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:09.116+0000 7fa95a989740 1 Processor -- start 2024-03-21T10:29:09.119 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:09.117+0000 7fa95a989740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559f9fd856d0 0x559f9fd8e430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:29:09.119 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:09.117+0000 7fa95a989740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559f9fd86080 0x559f9fd8b7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:29:09.119 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:09.117+0000 7fa95a989740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x559f9fd8ec00 0x559f9fd8bd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:29:09.120 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:09.117+0000 7fa95a989740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x559f9fd91bc0 con 0x559f9fd8ec00 2024-03-21T10:29:09.120 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:09.117+0000 7fa95a989740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x559f9fca50d0 con 0x559f9fd856d0 2024-03-21T10:29:09.120 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:09.117+0000 7fa95a989740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x559f9fd91d40 con 0x559f9fd86080 2024-03-21T10:29:09.120 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:09.117+0000 7fa95a989740 10 client.0 _ll_register_callbacks cb 0x559f9fca9e80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T10:29:09.120 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:09.117+0000 7fa955c00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x559f9fd8ec00 0x559f9fd8bd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:29:09.120 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:09.117+0000 7fa955c00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x559f9fd8ec00 0x559f9fd8bd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:52948/0 (socket says 172.21.15.47:52948) 2024-03-21T10:29:09.120 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:09.117+0000 7fa955200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559f9fd856d0 0x559f9fd8e430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:29:09.120 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:09.117+0000 7fa955c00640 1 -- 172.21.15.47:0/995403401 learned_addr learned my addr 172.21.15.47:0/995403401 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:29:09.120 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:09.117+0000 7fa94fe00640 1 --2- 172.21.15.47:0/995403401 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559f9fd86080 0x559f9fd8b7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:29:09.120 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:09.118+0000 7fa955c00640 1 -- 172.21.15.47:0/995403401 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559f9fd86080 msgr2=0x559f9fd8b7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:29:09.120 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:09.118+0000 7fa955c00640 1 --2- 172.21.15.47:0/995403401 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559f9fd86080 0x559f9fd8b7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:29:09.120 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:09.118+0000 7fa955c00640 1 -- 172.21.15.47:0/995403401 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559f9fd856d0 msgr2=0x559f9fd8e430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:29:09.120 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:09.118+0000 7fa955c00640 1 --2- 172.21.15.47:0/995403401 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559f9fd856d0 0x559f9fd8e430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:29:09.120 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:09.118+0000 7fa955c00640 1 -- 172.21.15.47:0/995403401 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x559f9f9fd3f0 con 0x559f9fd8ec00 2024-03-21T10:29:09.121 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:09.118+0000 7fa955c00640 1 --2- 172.21.15.47:0/995403401 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x559f9fd8ec00 0x559f9fd8bd00 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7fa948009cd0 tx=0x7fa94802c0f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=746e3c14909d9b19 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:29:09.121 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:09.118+0000 7fa936a00640 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-03-21T10:29:09.121 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:09.118+0000 7fa936a00640 1 -- 172.21.15.47:0/995403401 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa948045070 con 0x559f9fd8ec00 2024-03-21T10:29:09.121 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:09.118+0000 7fa936a00640 1 -- 172.21.15.47:0/995403401 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(97..97 src has 1..97) v4 ==== 9474+0+0 (secure 0 0 0) 0x7fa948003b70 con 0x559f9fd8ec00 2024-03-21T10:29:09.121 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:09.118+0000 7fa95a989740 1 -- 172.21.15.47:0/995403401 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fa93c011db0 con 0x559f9fd8ec00 2024-03-21T10:29:09.121 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:09.119+0000 7fa936a00640 1 -- 172.21.15.47:0/995403401 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa948059050 con 0x559f9fd8ec00 2024-03-21T10:29:09.121 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:09.119+0000 7fa95a989740 10 client.16926 fetch_fsmap learned FSMap version 32 2024-03-21T10:29:09.121 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:09.119+0000 7fa95a989740 1 -- 172.21.15.47:0/995403401 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x559f9fac77d0 con 0x559f9fd8ec00 2024-03-21T10:29:09.121 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:09.119+0000 7fa936a00640 1 -- 172.21.15.47:0/995403401 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fa93c011db0 con 0x559f9fd8ec00 2024-03-21T10:29:09.121 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:09.119+0000 7fa936a00640 1 -- 172.21.15.47:0/995403401 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 32) v1 ==== 7942+0+0 (secure 0 0 0) 0x7fa948054030 con 0x559f9fd8ec00 2024-03-21T10:29:09.121 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:09.119+0000 7fa95a989740 10 client.16926 fetch_fsmap finished waiting for FSMap version 32 2024-03-21T10:29:09.121 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:09.119+0000 7fa95a989740 10 client.16926 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-21T10:29:09.121 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:09.119+0000 7fa95a989740 20 client.16926 populate_metadata read hostname 'smithi047' 2024-03-21T10:29:09.121 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:09.119+0000 7fa95a989740 1 --2- 172.21.15.47:0/995403401 >> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] conn(0x559f9fd9ea40 0x559f9fda0ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:29:09.121 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:09.119+0000 7fa95a989740 4 client.16926 mds_command: new command op to 34535 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T10:29:09.122 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:09.119+0000 7fa95a989740 1 -- 172.21.15.47:0/995403401 --> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x559f9fcf6800 con 0x559f9fd9ea40 2024-03-21T10:29:09.122 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:09.120+0000 7fa955200640 1 --2- 172.21.15.47:0/995403401 >> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] conn(0x559f9fd9ea40 0x559f9fda0ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:29:09.126 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:09.121+0000 7fa955200640 1 --2- 172.21.15.47:0/995403401 >> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] conn(0x559f9fd9ea40 0x559f9fda0ea0 crc :-1 s=READY pgs=123 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7c4228bfc5fbcf34 server_cookie=e5143fec94943cfa in_seq=0 out_seq=0 2024-03-21T10:29:09.126 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:09.121+0000 7fa936a00640 10 client.16926 ms_handle_connect on v2:172.21.15.100:6838/4173168440 2024-03-21T10:29:09.126 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:09.122+0000 7fa936a00640 1 -- 172.21.15.47:0/995403401 <== mds.0 v2:172.21.15.100:6838/4173168440 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x559f9fcf6800 con 0x559f9fd9ea40 2024-03-21T10:29:09.126 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:09.122+0000 7fa936a00640 10 client.16926 handle_command_reply: tid=0 2024-03-21T10:29:09.126 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:09.122+0000 7fa95a989740 1 client.16926 shutdown 2024-03-21T10:29:09.126 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:09.122+0000 7fa95a989740 1 -- 172.21.15.47:0/995403401 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x559f9fd8ec00 msgr2=0x559f9fd8bd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:29:09.126 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:09.122+0000 7fa95a989740 1 --2- 172.21.15.47:0/995403401 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x559f9fd8ec00 0x559f9fd8bd00 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7fa948009cd0 tx=0x7fa94802c0f0 comp rx=0 tx=0).stop 2024-03-21T10:29:09.126 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:09.122+0000 7fa95a989740 1 -- 172.21.15.47:0/995403401 shutdown_connections 2024-03-21T10:29:09.126 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:09.122+0000 7fa95a989740 1 --2- 172.21.15.47:0/995403401 >> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] conn(0x559f9fd9ea40 0x559f9fda0ea0 crc :-1 s=READY pgs=123 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:29:09.126 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:09.122+0000 7fa95a989740 1 --2- 172.21.15.47:0/995403401 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x559f9fd8ec00 0x559f9fd8bd00 unknown :-1 s=CLOSED pgs=788 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:29:09.126 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:09.122+0000 7fa95a989740 1 --2- 172.21.15.47:0/995403401 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559f9fd86080 0x559f9fd8b7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:29:09.126 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:09.122+0000 7fa95a989740 1 --2- 172.21.15.47:0/995403401 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559f9fd856d0 0x559f9fd8e430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:29:09.127 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:09.122+0000 7fa936a00640 0 client.16926 ms_handle_reset on v2:172.21.15.100:6838/4173168440 2024-03-21T10:29:09.127 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:09.122+0000 7fa95a989740 1 -- 172.21.15.47:0/995403401 >> 172.21.15.47:0/995403401 conn(0x559f9fd7fda0 msgr2=0x559f9fd81af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:29:09.127 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:09.123+0000 7fa95a989740 1 -- 172.21.15.47:0/995403401 shutdown_connections 2024-03-21T10:29:09.127 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:09.123+0000 7fa95a989740 1 -- 172.21.15.47:0/995403401 wait complete. 2024-03-21T10:29:09.127 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:09.123+0000 7fa95a989740 20 client.16926 trim_cache size 0 max 16384 2024-03-21T10:29:09.136 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:09.136+0000 7fa95a989740 1 Processor -- start 2024-03-21T10:29:09.136 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:09.136+0000 7fa95a989740 1 -- start start 2024-03-21T10:29:09.136 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:09.137+0000 7fa95a989740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559f9fd856d0 0x559f9fd8def0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:29:09.137 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:09.137+0000 7fa95a989740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559f9fd86080 0x559f9fd8e430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:29:09.137 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:09.137+0000 7fa95a989740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x559f9fd8ec00 0x559f9fd88a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:29:09.137 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:09.137+0000 7fa95a989740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9501b0b70 con 0x559f9fd8ec00 2024-03-21T10:29:09.137 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:09.137+0000 7fa95a989740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9501b24f0 con 0x559f9fd856d0 2024-03-21T10:29:09.137 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:09.137+0000 7fa95a989740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9501b11d0 con 0x559f9fd86080 2024-03-21T10:29:09.137 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:09.137+0000 7fa955c00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x559f9fd8ec00 0x559f9fd88a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:29:09.137 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:09.137+0000 7fa955200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559f9fd856d0 0x559f9fd8def0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:29:09.137 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:09.137+0000 7fa955200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559f9fd856d0 0x559f9fd8def0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:33296/0 (socket says 172.21.15.47:33296) 2024-03-21T10:29:09.137 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:09.137+0000 7fa955200640 1 -- 172.21.15.47:0/413513079 learned_addr learned my addr 172.21.15.47:0/413513079 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:29:09.138 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:09.137+0000 7fa94fe00640 1 --2- 172.21.15.47:0/413513079 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559f9fd86080 0x559f9fd8e430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:29:09.139 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:09.137+0000 7fa955200640 1 -- 172.21.15.47:0/413513079 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559f9fd86080 msgr2=0x559f9fd8e430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:29:09.139 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:09.137+0000 7fa955200640 1 --2- 172.21.15.47:0/413513079 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559f9fd86080 0x559f9fd8e430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:29:09.139 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:09.137+0000 7fa955200640 1 -- 172.21.15.47:0/413513079 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x559f9fd8ec00 msgr2=0x559f9fd88a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:29:09.139 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:09.138+0000 7fa955200640 1 --2- 172.21.15.47:0/413513079 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x559f9fd8ec00 0x559f9fd88a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:29:09.139 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:09.138+0000 7fa955200640 1 -- 172.21.15.47:0/413513079 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559f9fd85e20 con 0x559f9fd856d0 2024-03-21T10:29:09.139 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:09.138+0000 7fa955c00640 1 --2- 172.21.15.47:0/413513079 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x559f9fd8ec00 0x559f9fd88a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T10:29:09.139 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:09.138+0000 7fa955200640 1 --2- 172.21.15.47:0/413513079 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559f9fd856d0 0x559f9fd8def0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fa94400c6a0 tx=0x7fa9440d39e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8029995755802727 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:29:09.139 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:09.138+0000 7fa941600640 1 -- 172.21.15.47:0/413513079 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa944010de0 con 0x559f9fd856d0 2024-03-21T10:29:09.139 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:09.138+0000 7fa941600640 1 -- 172.21.15.47:0/413513079 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fa94409c600 con 0x559f9fd856d0 2024-03-21T10:29:09.139 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:09.138+0000 7fa941600640 1 -- 172.21.15.47:0/413513079 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa944098570 con 0x559f9fd856d0 2024-03-21T10:29:09.139 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:09.139+0000 7fa95a989740 1 -- 172.21.15.47:0/413513079 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559f9fd856d0 msgr2=0x559f9fd8def0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:29:09.139 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:09.139+0000 7fa95a989740 1 --2- 172.21.15.47:0/413513079 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559f9fd856d0 0x559f9fd8def0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fa94400c6a0 tx=0x7fa9440d39e0 comp rx=0 tx=0).stop 2024-03-21T10:29:09.139 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:09.139+0000 7fa95a989740 1 -- 172.21.15.47:0/413513079 shutdown_connections 2024-03-21T10:29:09.139 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:09.139+0000 7fa95a989740 1 --2- 172.21.15.47:0/413513079 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x559f9fd8ec00 0x559f9fd88a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:29:09.140 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:09.139+0000 7fa95a989740 1 --2- 172.21.15.47:0/413513079 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559f9fd86080 0x559f9fd8e430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:29:09.140 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:09.139+0000 7fa95a989740 1 --2- 172.21.15.47:0/413513079 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559f9fd856d0 0x559f9fd8def0 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:29:09.140 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:09.139+0000 7fa95a989740 1 -- 172.21.15.47:0/413513079 >> 172.21.15.47:0/413513079 conn(0x559f9fd7fda0 msgr2=0x559f9fd86cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:29:09.140 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:09.139+0000 7fa95a989740 1 -- 172.21.15.47:0/413513079 shutdown_connections 2024-03-21T10:29:09.140 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:09.139+0000 7fa95a989740 1 -- 172.21.15.47:0/413513079 wait complete. 2024-03-21T10:29:09.140 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:09.139+0000 7fa95a989740 1 -- start start 2024-03-21T10:29:09.142 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:09.140+0000 7fa95a989740 1 Processor -- start 2024-03-21T10:29:09.142 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:09.140+0000 7fa95a989740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559f9fd856d0 0x559f9fdf2670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:29:09.142 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:09.140+0000 7fa95a989740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559f9fd86080 0x559f9fdf2bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:29:09.142 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:09.140+0000 7fa95a989740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x559f9fd8ec00 0x559f9fd92ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:29:09.142 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:09.140+0000 7fa95a989740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9501b1e90 con 0x559f9fd8ec00 2024-03-21T10:29:09.143 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:09.140+0000 7fa95a989740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9501b1830 con 0x559f9fd86080 2024-03-21T10:29:09.143 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:09.140+0000 7fa95a989740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9501aed30 con 0x559f9fd856d0 2024-03-21T10:29:09.143 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:09.140+0000 7fa95a989740 10 client.0 _ll_register_callbacks cb 0x7fa948037070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T10:29:09.143 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:09.141+0000 7fa955c00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x559f9fd8ec00 0x559f9fd92ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:29:09.143 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:09.141+0000 7fa955c00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x559f9fd8ec00 0x559f9fd92ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:52962/0 (socket says 172.21.15.47:52962) 2024-03-21T10:29:09.143 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:09.141+0000 7fa955c00640 1 -- 172.21.15.47:0/1732280574 learned_addr learned my addr 172.21.15.47:0/1732280574 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:29:09.143 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:09.141+0000 7fa94fe00640 1 --2- 172.21.15.47:0/1732280574 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559f9fd86080 0x559f9fdf2bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:29:09.143 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:09.141+0000 7fa955c00640 1 -- 172.21.15.47:0/1732280574 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559f9fd856d0 msgr2=0x559f9fdf2670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:29:09.143 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:09.141+0000 7fa955c00640 1 --2- 172.21.15.47:0/1732280574 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559f9fd856d0 0x559f9fdf2670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:29:09.143 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:09.141+0000 7fa955c00640 1 -- 172.21.15.47:0/1732280574 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559f9fd86080 msgr2=0x559f9fdf2bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:29:09.143 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:09.141+0000 7fa955c00640 1 --2- 172.21.15.47:0/1732280574 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559f9fd86080 0x559f9fdf2bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:29:09.143 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:09.141+0000 7fa955c00640 1 -- 172.21.15.47:0/1732280574 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x559f9fd934a0 con 0x559f9fd8ec00 2024-03-21T10:29:09.143 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:09.141+0000 7fa955c00640 1 --2- 172.21.15.47:0/1732280574 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x559f9fd8ec00 0x559f9fd92ca0 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7fa948009990 tx=0x7fa94804ce00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c8ce37a21217594b server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:29:09.143 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:09.141+0000 7fa936a00640 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-03-21T10:29:09.143 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:09.142+0000 7fa936a00640 1 -- 172.21.15.47:0/1732280574 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa948045040 con 0x559f9fd8ec00 2024-03-21T10:29:09.144 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:09.142+0000 7fa936a00640 1 -- 172.21.15.47:0/1732280574 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(97..97 src has 1..97) v4 ==== 9474+0+0 (secure 0 0 0) 0x7fa948034440 con 0x559f9fd8ec00 2024-03-21T10:29:09.144 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:09.142+0000 7fa95a989740 1 -- 172.21.15.47:0/1732280574 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fa94409c600 con 0x559f9fd8ec00 2024-03-21T10:29:09.144 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:09.142+0000 7fa936a00640 1 -- 172.21.15.47:0/1732280574 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa948039040 con 0x559f9fd8ec00 2024-03-21T10:29:09.144 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:09.142+0000 7fa95a989740 10 client.16932 fetch_fsmap learned FSMap version 32 2024-03-21T10:29:09.144 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:09.142+0000 7fa95a989740 1 -- 172.21.15.47:0/1732280574 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x559f9fdfaf00 con 0x559f9fd8ec00 2024-03-21T10:29:09.144 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:09.142+0000 7fa936a00640 1 -- 172.21.15.47:0/1732280574 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fa94409c600 con 0x559f9fd8ec00 2024-03-21T10:29:09.144 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:09.143+0000 7fa936a00640 1 -- 172.21.15.47:0/1732280574 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 32) v1 ==== 7942+0+0 (secure 0 0 0) 0x7fa948054040 con 0x559f9fd8ec00 2024-03-21T10:29:09.144 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:09.143+0000 7fa95a989740 10 client.16932 fetch_fsmap finished waiting for FSMap version 32 2024-03-21T10:29:09.144 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:09.143+0000 7fa95a989740 10 client.16932 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-21T10:29:09.144 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:09.143+0000 7fa95a989740 20 client.16932 populate_metadata read hostname 'smithi047' 2024-03-21T10:29:09.144 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:09.143+0000 7fa95a989740 1 --2- 172.21.15.47:0/1732280574 >> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] conn(0x559f9fdfb4e0 0x559f9fdfd900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:29:09.144 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:09.143+0000 7fa95a989740 4 client.16932 mds_command: new command op to 34535 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-21T10:29:09.144 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:09.143+0000 7fa95a989740 1 -- 172.21.15.47:0/1732280574 --> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x559f9fcf0b50 con 0x559f9fdfb4e0 2024-03-21T10:29:09.144 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:09.143+0000 7fa955200640 1 --2- 172.21.15.47:0/1732280574 >> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] conn(0x559f9fdfb4e0 0x559f9fdfd900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:29:09.146 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:09.144+0000 7fa955200640 1 --2- 172.21.15.47:0/1732280574 >> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] conn(0x559f9fdfb4e0 0x559f9fdfd900 crc :-1 s=READY pgs=124 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ba59f2926523e230 server_cookie=ec36b45cc61f3cab in_seq=0 out_seq=0 2024-03-21T10:29:09.146 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:09.144+0000 7fa936a00640 10 client.16932 ms_handle_connect on v2:172.21.15.100:6838/4173168440 2024-03-21T10:29:09.146 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:09.144+0000 7fa936a00640 1 -- 172.21.15.47:0/1732280574 <== mds.0 v2:172.21.15.100:6838/4173168440 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x559f9fcf0b50 con 0x559f9fdfb4e0 2024-03-21T10:29:09.146 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:09.144+0000 7fa936a00640 10 client.16932 handle_command_reply: tid=0 2024-03-21T10:29:09.146 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:09.144+0000 7fa95a989740 1 client.16932 shutdown 2024-03-21T10:29:09.146 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:09.144+0000 7fa95a989740 1 -- 172.21.15.47:0/1732280574 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x559f9fd8ec00 msgr2=0x559f9fd92ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:29:09.146 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:09.144+0000 7fa95a989740 1 --2- 172.21.15.47:0/1732280574 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x559f9fd8ec00 0x559f9fd92ca0 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7fa948009990 tx=0x7fa94804ce00 comp rx=0 tx=0).stop 2024-03-21T10:29:09.146 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:09.144+0000 7fa95a989740 1 -- 172.21.15.47:0/1732280574 shutdown_connections 2024-03-21T10:29:09.146 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:09.144+0000 7fa95a989740 1 --2- 172.21.15.47:0/1732280574 >> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] conn(0x559f9fdfb4e0 0x559f9fdfd900 crc :-1 s=READY pgs=124 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:29:09.146 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:09.144+0000 7fa95a989740 1 --2- 172.21.15.47:0/1732280574 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x559f9fd8ec00 0x559f9fd92ca0 unknown :-1 s=CLOSED pgs=789 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:29:09.146 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:09.144+0000 7fa95a989740 1 --2- 172.21.15.47:0/1732280574 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559f9fd86080 0x559f9fdf2bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:29:09.147 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:09.144+0000 7fa95a989740 1 --2- 172.21.15.47:0/1732280574 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559f9fd856d0 0x559f9fdf2670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:29:09.147 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:09.144+0000 7fa95a989740 1 -- 172.21.15.47:0/1732280574 >> 172.21.15.47:0/1732280574 conn(0x559f9fd7fda0 msgr2=0x559f9fd7e620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:29:09.147 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:09.144+0000 7fa936a00640 0 client.16932 ms_handle_reset on v2:172.21.15.100:6838/4173168440 2024-03-21T10:29:09.147 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:09.144+0000 7fa95a989740 1 -- 172.21.15.47:0/1732280574 shutdown_connections 2024-03-21T10:29:09.147 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:09.144+0000 7fa95a989740 1 -- 172.21.15.47:0/1732280574 wait complete. 2024-03-21T10:29:09.147 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:09.145+0000 7fa95a989740 20 client.16932 trim_cache size 0 max 16384 2024-03-21T10:29:09.147 INFO:teuthology.orchestra.run.smithi047.stdout:[] 2024-03-21T10:29:09.148 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:09.145+0000 7fa957c00640 1 -- 172.21.15.47:0/1112690482 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7fa9380a3090 msgr2=0x7fa9380a5550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:29:09.148 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:09.145+0000 7fa957c00640 1 --2- 172.21.15.47:0/1112690482 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7fa9380a3090 0x7fa9380a5550 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fa94802c630 tx=0x7fa94802e040 comp rx=0 tx=0).stop 2024-03-21T10:29:09.148 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:09.145+0000 7fa957c00640 1 -- 172.21.15.47:0/1112690482 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa950103840 msgr2=0x7fa95019a090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:29:09.148 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:09.145+0000 7fa957c00640 1 --2- 172.21.15.47:0/1112690482 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa950103840 0x7fa95019a090 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7fa94400e9f0 tx=0x7fa94400eeb0 comp rx=0 tx=0).stop 2024-03-21T10:29:09.148 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:09.145+0000 7fa957c00640 1 -- 172.21.15.47:0/1112690482 shutdown_connections 2024-03-21T10:29:09.149 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:09.145+0000 7fa957c00640 1 --2- 172.21.15.47:0/1112690482 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa950109850 0x7fa95019e900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:29:09.149 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:09.145+0000 7fa957c00640 1 --2- 172.21.15.47:0/1112690482 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7fa9380a3090 0x7fa9380a5550 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:29:09.149 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:09.145+0000 7fa957c00640 1 --2- 172.21.15.47:0/1112690482 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa950104200 0x7fa95019a5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:29:09.149 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:09.145+0000 7fa957c00640 1 --2- 172.21.15.47:0/1112690482 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa950103840 0x7fa95019a090 unknown :-1 s=CLOSED pgs=786 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:29:09.149 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:09.145+0000 7fa957c00640 1 -- 172.21.15.47:0/1112690482 >> 172.21.15.47:0/1112690482 conn(0x7fa9500ff2f0 msgr2=0x7fa9500ffb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:29:09.149 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:09.146+0000 7fa957c00640 1 -- 172.21.15.47:0/1112690482 shutdown_connections 2024-03-21T10:29:09.149 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:09.146+0000 7fa957c00640 1 -- 172.21.15.47:0/1112690482 wait complete. 2024-03-21T10:29:09.962 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:29:09 smithi047 ceph-mon[31547]: pgmap v118: 129 pgs: 129 active+clean; 277 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 24 op/s 2024-03-21T10:29:09.977 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:29:09 smithi100 ceph-mon[35193]: pgmap v118: 129 pgs: 129 active+clean; 277 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 24 op/s 2024-03-21T10:29:09.988 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:29:09 smithi157 ceph-mon[35581]: pgmap v118: 129 pgs: 129 active+clean; 277 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 24 op/s 2024-03-21T10:29:10.165 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-03-21T10:29:10.165 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-21T10:29:10.166 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-03-21T10:29:10.418 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:10.418+0000 7f74d4000640 1 -- 172.21.15.47:0/2576179831 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f74cc107eb0 msgr2=0x7f74cc108290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:29:10.418 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:10.418+0000 7f74d4000640 1 --2- 172.21.15.47:0/2576179831 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f74cc107eb0 0x7f74cc108290 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7f74c0009900 tx=0x7f74c002bc30 comp rx=0 tx=0).stop 2024-03-21T10:29:10.418 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:10.418+0000 7f74d4000640 1 -- 172.21.15.47:0/2576179831 shutdown_connections 2024-03-21T10:29:10.418 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:10.418+0000 7f74d4000640 1 --2- 172.21.15.47:0/2576179831 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f74cc10cf80 0x7f74cc114b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:29:10.418 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:10.418+0000 7f74d4000640 1 --2- 172.21.15.47:0/2576179831 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f74cc108860 0x7f74cc10c8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:29:10.418 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:10.418+0000 7f74d4000640 1 --2- 172.21.15.47:0/2576179831 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f74cc107eb0 0x7f74cc108290 unknown :-1 s=CLOSED pgs=790 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:29:10.418 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:10.418+0000 7f74d4000640 1 -- 172.21.15.47:0/2576179831 >> 172.21.15.47:0/2576179831 conn(0x7f74cc075be0 msgr2=0x7f74cc078000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:29:10.418 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:10.418+0000 7f74d4000640 1 -- 172.21.15.47:0/2576179831 shutdown_connections 2024-03-21T10:29:10.418 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:10.418+0000 7f74d4000640 1 -- 172.21.15.47:0/2576179831 wait complete. 2024-03-21T10:29:10.421 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:10.419+0000 7f74d4000640 1 Processor -- start 2024-03-21T10:29:10.421 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:10.419+0000 7f74d4000640 1 -- start start 2024-03-21T10:29:10.421 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:10.420+0000 7f74d4000640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f74cc107eb0 0x7f74cc19e5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:29:10.421 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:10.420+0000 7f74d4000640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f74cc10cf80 0x7f74cc19eb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:29:10.421 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:10.420+0000 7f74d4000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f74cc1a2e30 0x7f74cc1a32e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:29:10.421 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:10.420+0000 7f74d4000640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74cc117ee0 con 0x7f74cc10cf80 2024-03-21T10:29:10.421 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:10.420+0000 7f74d4000640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74cc117d60 con 0x7f74cc1a2e30 2024-03-21T10:29:10.421 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:10.420+0000 7f74d4000640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74cc118060 con 0x7f74cc107eb0 2024-03-21T10:29:10.421 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:10.420+0000 7f74d0c00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f74cc10cf80 0x7f74cc19eb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:29:10.421 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:10.420+0000 7f74d0c00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f74cc10cf80 0x7f74cc19eb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:52974/0 (socket says 172.21.15.47:52974) 2024-03-21T10:29:10.421 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:10.420+0000 7f74d0c00640 1 -- 172.21.15.47:0/360428039 learned_addr learned my addr 172.21.15.47:0/360428039 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:29:10.421 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:10.420+0000 7f74d2000640 1 --2- 172.21.15.47:0/360428039 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f74cc1a2e30 0x7f74cc1a32e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:29:10.421 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:10.420+0000 7f74d1600640 1 --2- 172.21.15.47:0/360428039 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f74cc107eb0 0x7f74cc19e5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:29:10.422 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:10.421+0000 7f74d0c00640 1 -- 172.21.15.47:0/360428039 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f74cc107eb0 msgr2=0x7f74cc19e5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:29:10.422 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:10.421+0000 7f74d0c00640 1 --2- 172.21.15.47:0/360428039 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f74cc107eb0 0x7f74cc19e5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:29:10.422 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:10.421+0000 7f74d0c00640 1 -- 172.21.15.47:0/360428039 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f74cc1a2e30 msgr2=0x7f74cc1a32e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:29:10.422 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:10.421+0000 7f74d0c00640 1 --2- 172.21.15.47:0/360428039 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f74cc1a2e30 0x7f74cc1a32e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:29:10.422 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:10.421+0000 7f74d0c00640 1 -- 172.21.15.47:0/360428039 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f74cc1a3920 con 0x7f74cc10cf80 2024-03-21T10:29:10.422 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:10.421+0000 7f74d0c00640 1 --2- 172.21.15.47:0/360428039 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f74cc10cf80 0x7f74cc19eb00 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7f74b800d740 tx=0x7f74b800dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:29:10.422 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:10.421+0000 7f74ca000640 1 -- 172.21.15.47:0/360428039 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74b8017070 con 0x7f74cc10cf80 2024-03-21T10:29:10.422 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:10.421+0000 7f74d4000640 1 -- 172.21.15.47:0/360428039 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f74cc1a3c10 con 0x7f74cc10cf80 2024-03-21T10:29:10.422 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:10.421+0000 7f74ca000640 1 -- 172.21.15.47:0/360428039 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f74b800de50 con 0x7f74cc10cf80 2024-03-21T10:29:10.422 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:10.421+0000 7f74ca000640 1 -- 172.21.15.47:0/360428039 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74b8004d60 con 0x7f74cc10cf80 2024-03-21T10:29:10.422 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:10.421+0000 7f74d4000640 1 -- 172.21.15.47:0/360428039 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f74cc1ab3b0 con 0x7f74cc10cf80 2024-03-21T10:29:10.424 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:10.425+0000 7f74d6d8e740 1 Processor -- start 2024-03-21T10:29:10.424 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:10.425+0000 7f74ca000640 1 -- 172.21.15.47:0/360428039 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 27) v1 ==== 137985+0+0 (secure 0 0 0) 0x7f74b8004f00 con 0x7f74cc10cf80 2024-03-21T10:29:10.424 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:10.425+0000 7f74d6d8e740 1 -- start start 2024-03-21T10:29:10.426 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:10.425+0000 7f74d6d8e740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563f39108080 0x563f39108460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:29:10.426 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:10.425+0000 7f74d6d8e740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563f391089a0 0x563f39111040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:29:10.426 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:10.425+0000 7f74d6d8e740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x563f39111580 0x563f39113970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:29:10.426 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:10.425+0000 7f74d6d8e740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x563f3906ceb0 con 0x563f39111580 2024-03-21T10:29:10.426 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:10.425+0000 7f74d6d8e740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x563f39015a60 con 0x563f39108080 2024-03-21T10:29:10.426 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:10.425+0000 7f74d6d8e740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x563f39068f50 con 0x563f391089a0 2024-03-21T10:29:10.426 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:10.426+0000 7f74d0c00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x563f39111580 0x563f39113970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:29:10.426 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:10.426+0000 7f74d2000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563f39108080 0x563f39108460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:29:10.427 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:10.426+0000 7f74d2000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563f39108080 0x563f39108460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:33330/0 (socket says 172.21.15.47:33330) 2024-03-21T10:29:10.427 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:10.426+0000 7f74d2000640 1 -- 172.21.15.47:0/4221993761 learned_addr learned my addr 172.21.15.47:0/4221993761 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:29:10.427 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:10.426+0000 7f74d1600640 1 --2- 172.21.15.47:0/4221993761 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563f391089a0 0x563f39111040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:29:10.427 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:10.426+0000 7f74d2000640 1 -- 172.21.15.47:0/4221993761 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563f391089a0 msgr2=0x563f39111040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:29:10.427 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:10.426+0000 7f74d2000640 1 --2- 172.21.15.47:0/4221993761 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563f391089a0 0x563f39111040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:29:10.427 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:10.426+0000 7f74d2000640 1 -- 172.21.15.47:0/4221993761 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x563f39111580 msgr2=0x563f39113970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:29:10.427 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:10.426+0000 7f74d2000640 1 --2- 172.21.15.47:0/4221993761 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x563f39111580 0x563f39113970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:29:10.427 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:10.426+0000 7f74d2000640 1 -- 172.21.15.47:0/4221993761 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563f38e497e0 con 0x563f39108080 2024-03-21T10:29:10.427 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:10.426+0000 7f74d0c00640 1 --2- 172.21.15.47:0/4221993761 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x563f39111580 0x563f39113970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T10:29:10.427 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:10.427+0000 7f74ca000640 1 --2- 172.21.15.47:0/360428039 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f74b40a3090 0x7f74b40a5550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:29:10.427 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:10.427+0000 7f74d2000640 1 --2- 172.21.15.47:0/4221993761 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563f39108080 0x563f39108460 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f74c4009f50 tx=0x7f74c40118d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=50cb40955cce8530 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:29:10.427 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:10.427+0000 7f74ca000640 1 -- 172.21.15.47:0/360428039 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 9474+0+0 (secure 0 0 0) 0x7f74b80cf5f0 con 0x7f74cc10cf80 2024-03-21T10:29:10.430 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:10.427+0000 7f74d1600640 1 --2- 172.21.15.47:0/360428039 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f74b40a3090 0x7f74b40a5550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:29:10.430 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:10.427+0000 7f74be600640 1 -- 172.21.15.47:0/4221993761 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74c4004d20 con 0x563f39108080 2024-03-21T10:29:10.431 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:10.427+0000 7f74be600640 1 -- 172.21.15.47:0/4221993761 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f74c4004050 con 0x563f39108080 2024-03-21T10:29:10.431 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:10.427+0000 7f74be600640 1 -- 172.21.15.47:0/4221993761 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74c4005280 con 0x563f39108080 2024-03-21T10:29:10.431 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:10.428+0000 7f74d6d8e740 1 -- 172.21.15.47:0/4221993761 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563f39108080 msgr2=0x563f39108460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:29:10.431 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:10.428+0000 7f74d6d8e740 1 --2- 172.21.15.47:0/4221993761 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563f39108080 0x563f39108460 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f74c4009f50 tx=0x7f74c40118d0 comp rx=0 tx=0).stop 2024-03-21T10:29:10.431 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:10.428+0000 7f74d6d8e740 1 -- 172.21.15.47:0/4221993761 shutdown_connections 2024-03-21T10:29:10.431 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:10.428+0000 7f74d6d8e740 1 --2- 172.21.15.47:0/4221993761 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x563f39111580 0x563f39113970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:29:10.431 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:10.428+0000 7f74d6d8e740 1 --2- 172.21.15.47:0/4221993761 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563f391089a0 0x563f39111040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:29:10.431 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:10.428+0000 7f74d6d8e740 1 --2- 172.21.15.47:0/4221993761 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563f39108080 0x563f39108460 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:29:10.431 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:10.428+0000 7f74d6d8e740 1 -- 172.21.15.47:0/4221993761 >> 172.21.15.47:0/4221993761 conn(0x563f391027b0 msgr2=0x563f39102b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:29:10.431 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:10.428+0000 7f74d1600640 1 --2- 172.21.15.47:0/360428039 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f74b40a3090 0x7f74b40a5550 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f74c0000c00 tx=0x7f74c00058b0 comp rx=0 tx=0).ready entity=mgr.15687 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:29:10.431 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:10.428+0000 7f74d6d8e740 1 -- 172.21.15.47:0/4221993761 shutdown_connections 2024-03-21T10:29:10.431 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:10.428+0000 7f74d6d8e740 1 -- 172.21.15.47:0/4221993761 wait complete. 2024-03-21T10:29:10.432 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:10.429+0000 7f74d6d8e740 1 -- start start 2024-03-21T10:29:10.432 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:10.429+0000 7f74d6d8e740 1 Processor -- start 2024-03-21T10:29:10.432 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:10.429+0000 7f74d6d8e740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563f39108080 0x563f39110e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:29:10.432 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:10.429+0000 7f74d6d8e740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x563f391089a0 0x563f3910e1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:29:10.432 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:10.429+0000 7f74d6d8e740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563f39111580 0x563f3910e700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:29:10.432 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:10.430+0000 7f74d6d8e740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x563f39114370 con 0x563f391089a0 2024-03-21T10:29:10.432 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:10.430+0000 7f74d6d8e740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x563f39028eb0 con 0x563f39108080 2024-03-21T10:29:10.432 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:10.430+0000 7f74d6d8e740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x563f391144f0 con 0x563f39111580 2024-03-21T10:29:10.432 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:10.430+0000 7f74d6d8e740 10 client.0 _ll_register_callbacks cb 0x563f3901d150 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T10:29:10.432 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:10.430+0000 7f74d1600640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563f39108080 0x563f39110e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:29:10.432 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:10.430+0000 7f74d0c00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x563f391089a0 0x563f3910e1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:29:10.432 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:10.430+0000 7f74d1600640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563f39108080 0x563f39110e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:33340/0 (socket says 172.21.15.47:33340) 2024-03-21T10:29:10.432 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:10.430+0000 7f74d0c00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x563f391089a0 0x563f3910e1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:52994/0 (socket says 172.21.15.47:52994) 2024-03-21T10:29:10.433 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:10.430+0000 7f74d1600640 1 -- 172.21.15.47:0/3858143050 learned_addr learned my addr 172.21.15.47:0/3858143050 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:29:10.433 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:10.430+0000 7f74d2000640 1 --2- 172.21.15.47:0/3858143050 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563f39111580 0x563f3910e700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:29:10.433 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:10.430+0000 7f74d0c00640 1 -- 172.21.15.47:0/3858143050 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563f39111580 msgr2=0x563f3910e700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:29:10.433 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:10.430+0000 7f74d0c00640 1 --2- 172.21.15.47:0/3858143050 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563f39111580 0x563f3910e700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:29:10.433 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:10.430+0000 7f74d0c00640 1 -- 172.21.15.47:0/3858143050 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563f39108080 msgr2=0x563f39110e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:29:10.433 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:10.430+0000 7f74d0c00640 1 --2- 172.21.15.47:0/3858143050 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563f39108080 0x563f39110e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:29:10.433 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:10.430+0000 7f74d0c00640 1 -- 172.21.15.47:0/3858143050 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x563f38ea9f70 con 0x563f391089a0 2024-03-21T10:29:10.433 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:10.431+0000 7f74d0c00640 1 --2- 172.21.15.47:0/3858143050 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x563f391089a0 0x563f3910e1c0 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7f74b80cc920 tx=0x7f74b80971a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8796cff85e065c4f server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:29:10.433 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:10.431+0000 7f74b2a00640 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-03-21T10:29:10.433 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:10.431+0000 7f74b2a00640 1 -- 172.21.15.47:0/3858143050 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74b8020070 con 0x563f391089a0 2024-03-21T10:29:10.433 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:10.431+0000 7f74b2a00640 1 -- 172.21.15.47:0/3858143050 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(97..97 src has 1..97) v4 ==== 9474+0+0 (secure 0 0 0) 0x7f74b80cf150 con 0x563f391089a0 2024-03-21T10:29:10.433 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:10.431+0000 7f74d6d8e740 1 -- 172.21.15.47:0/3858143050 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f74c4004050 con 0x563f391089a0 2024-03-21T10:29:10.433 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:10.431+0000 7f74b2a00640 1 -- 172.21.15.47:0/3858143050 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f74b809b100 con 0x563f391089a0 2024-03-21T10:29:10.434 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:10.431+0000 7f74d6d8e740 10 client.16944 fetch_fsmap learned FSMap version 32 2024-03-21T10:29:10.434 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:10.431+0000 7f74d6d8e740 1 -- 172.21.15.47:0/3858143050 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x563f38d41170 con 0x563f391089a0 2024-03-21T10:29:10.436 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:10.432+0000 7f74b2a00640 1 -- 172.21.15.47:0/3858143050 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f74c4004050 con 0x563f391089a0 2024-03-21T10:29:10.436 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:10.432+0000 7f74b2a00640 1 -- 172.21.15.47:0/3858143050 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 32) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f74b80d1b20 con 0x563f391089a0 2024-03-21T10:29:10.436 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:10.432+0000 7f74d6d8e740 10 client.16944 fetch_fsmap finished waiting for FSMap version 32 2024-03-21T10:29:10.436 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:10.432+0000 7f74d6d8e740 10 client.16944 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-21T10:29:10.436 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:10.432+0000 7f74d6d8e740 20 client.16944 populate_metadata read hostname 'smithi047' 2024-03-21T10:29:10.436 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:10.432+0000 7f74d6d8e740 1 --2- 172.21.15.47:0/3858143050 >> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] conn(0x563f39121590 0x563f39123980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:29:10.437 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:10.432+0000 7f74d6d8e740 4 client.16944 mds_command: new command op to 34535 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T10:29:10.437 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:10.432+0000 7f74d6d8e740 1 -- 172.21.15.47:0/3858143050 --> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x563f3907a3f0 con 0x563f39121590 2024-03-21T10:29:10.437 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:10.432+0000 7f74d1600640 1 --2- 172.21.15.47:0/3858143050 >> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] conn(0x563f39121590 0x563f39123980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:29:10.437 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:10.433+0000 7f74d1600640 1 --2- 172.21.15.47:0/3858143050 >> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] conn(0x563f39121590 0x563f39123980 crc :-1 s=READY pgs=125 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8111b3d05e23ba0f server_cookie=601abb89f5b33d11 in_seq=0 out_seq=0 2024-03-21T10:29:10.437 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:10.433+0000 7f74b2a00640 10 client.16944 ms_handle_connect on v2:172.21.15.100:6838/4173168440 2024-03-21T10:29:10.437 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:10.434+0000 7f74b2a00640 1 -- 172.21.15.47:0/3858143050 <== mds.0 v2:172.21.15.100:6838/4173168440 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x563f3907a3f0 con 0x563f39121590 2024-03-21T10:29:10.437 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:10.434+0000 7f74b2a00640 10 client.16944 handle_command_reply: tid=0 2024-03-21T10:29:10.437 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:10.434+0000 7f74d6d8e740 1 client.16944 shutdown 2024-03-21T10:29:10.437 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:10.434+0000 7f74d6d8e740 1 -- 172.21.15.47:0/3858143050 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x563f391089a0 msgr2=0x563f3910e1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:29:10.437 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:10.434+0000 7f74d6d8e740 1 --2- 172.21.15.47:0/3858143050 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x563f391089a0 0x563f3910e1c0 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7f74b80cc920 tx=0x7f74b80971a0 comp rx=0 tx=0).stop 2024-03-21T10:29:10.437 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:10.434+0000 7f74d6d8e740 1 -- 172.21.15.47:0/3858143050 shutdown_connections 2024-03-21T10:29:10.438 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:10.434+0000 7f74d6d8e740 1 --2- 172.21.15.47:0/3858143050 >> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] conn(0x563f39121590 0x563f39123980 crc :-1 s=READY pgs=125 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:29:10.438 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:10.434+0000 7f74d6d8e740 1 --2- 172.21.15.47:0/3858143050 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563f39111580 0x563f3910e700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:29:10.438 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:10.434+0000 7f74d6d8e740 1 --2- 172.21.15.47:0/3858143050 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x563f391089a0 0x563f3910e1c0 unknown :-1 s=CLOSED pgs=792 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:29:10.438 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:10.434+0000 7f74d6d8e740 1 --2- 172.21.15.47:0/3858143050 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563f39108080 0x563f39110e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:29:10.438 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:10.434+0000 7f74d6d8e740 1 -- 172.21.15.47:0/3858143050 >> 172.21.15.47:0/3858143050 conn(0x563f391027b0 msgr2=0x563f39104500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:29:10.438 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:10.434+0000 7f74b2a00640 0 client.16944 ms_handle_reset on v2:172.21.15.100:6838/4173168440 2024-03-21T10:29:10.438 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:10.435+0000 7f74d6d8e740 1 -- 172.21.15.47:0/3858143050 shutdown_connections 2024-03-21T10:29:10.438 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:10.435+0000 7f74d6d8e740 1 -- 172.21.15.47:0/3858143050 wait complete. 2024-03-21T10:29:10.438 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:10.435+0000 7f74d6d8e740 20 client.16944 trim_cache size 0 max 16384 2024-03-21T10:29:10.453 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:10.452+0000 7f74d6d8e740 1 Processor -- start 2024-03-21T10:29:10.453 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:10.453+0000 7f74d6d8e740 1 -- start start 2024-03-21T10:29:10.453 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:10.453+0000 7f74d6d8e740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563f39108080 0x563f391108f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:29:10.454 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:10.453+0000 7f74d6d8e740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563f391089a0 0x563f39110e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:29:10.454 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:10.453+0000 7f74d6d8e740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x563f39111580 0x563f39109490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:29:10.454 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:10.453+0000 7f74d6d8e740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74cc1b4e10 con 0x563f39111580 2024-03-21T10:29:10.454 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:10.453+0000 7f74d6d8e740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74cc1b6790 con 0x563f39108080 2024-03-21T10:29:10.454 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:10.453+0000 7f74d6d8e740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74cc1b5470 con 0x563f391089a0 2024-03-21T10:29:10.454 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:10.453+0000 7f74d2000640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x563f39111580 0x563f39109490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:29:10.454 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:10.453+0000 7f74d2000640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x563f39111580 0x563f39109490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:53010/0 (socket says 172.21.15.47:53010) 2024-03-21T10:29:10.454 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:10.453+0000 7f74d2000640 1 -- 172.21.15.47:0/2272294643 learned_addr learned my addr 172.21.15.47:0/2272294643 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:29:10.454 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:10.453+0000 7f74d1600640 1 --2- 172.21.15.47:0/2272294643 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563f39108080 0x563f391108f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:29:10.455 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:10.454+0000 7f74d2000640 1 -- 172.21.15.47:0/2272294643 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563f391089a0 msgr2=0x563f39110e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:29:10.455 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:10.454+0000 7f74d0c00640 1 --2- 172.21.15.47:0/2272294643 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563f391089a0 0x563f39110e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:29:10.455 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:10.454+0000 7f74d2000640 1 --2- 172.21.15.47:0/2272294643 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563f391089a0 0x563f39110e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:29:10.455 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:10.454+0000 7f74d2000640 1 -- 172.21.15.47:0/2272294643 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563f39108080 msgr2=0x563f391108f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:29:10.455 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:10.454+0000 7f74d2000640 1 --2- 172.21.15.47:0/2272294643 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563f39108080 0x563f391108f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:29:10.455 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:10.454+0000 7f74d2000640 1 -- 172.21.15.47:0/2272294643 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563f3910ea30 con 0x563f39111580 2024-03-21T10:29:10.455 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:10.454+0000 7f74d1600640 1 --2- 172.21.15.47:0/2272294643 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563f39108080 0x563f391108f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T10:29:10.455 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:10.454+0000 7f74d0c00640 1 --2- 172.21.15.47:0/2272294643 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563f391089a0 0x563f39110e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T10:29:10.455 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:10.454+0000 7f74d2000640 1 --2- 172.21.15.47:0/2272294643 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x563f39111580 0x563f39109490 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7f74c4006010 tx=0x7f74c4003970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5b09431a0e05177e server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:29:10.460 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:10.454+0000 7f74bdc00640 1 -- 172.21.15.47:0/2272294643 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74c4003d60 con 0x563f39111580 2024-03-21T10:29:10.460 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:10.454+0000 7f74bdc00640 1 -- 172.21.15.47:0/2272294643 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f74c4005560 con 0x563f39111580 2024-03-21T10:29:10.460 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:10.454+0000 7f74bdc00640 1 -- 172.21.15.47:0/2272294643 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74c4025b20 con 0x563f39111580 2024-03-21T10:29:10.460 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:10.455+0000 7f74d6d8e740 1 -- 172.21.15.47:0/2272294643 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x563f39111580 msgr2=0x563f39109490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:29:10.460 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:10.455+0000 7f74d6d8e740 1 --2- 172.21.15.47:0/2272294643 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x563f39111580 0x563f39109490 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7f74c4006010 tx=0x7f74c4003970 comp rx=0 tx=0).stop 2024-03-21T10:29:10.460 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:10.455+0000 7f74d6d8e740 1 -- 172.21.15.47:0/2272294643 shutdown_connections 2024-03-21T10:29:10.460 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:10.455+0000 7f74d6d8e740 1 --2- 172.21.15.47:0/2272294643 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x563f39111580 0x563f39109490 unknown :-1 s=CLOSED pgs=793 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:29:10.460 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:10.455+0000 7f74d6d8e740 1 --2- 172.21.15.47:0/2272294643 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563f391089a0 0x563f39110e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:29:10.460 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:10.455+0000 7f74d6d8e740 1 --2- 172.21.15.47:0/2272294643 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563f39108080 0x563f391108f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:29:10.461 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:10.455+0000 7f74d6d8e740 1 -- 172.21.15.47:0/2272294643 >> 172.21.15.47:0/2272294643 conn(0x563f391027b0 msgr2=0x563f39108d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:29:10.461 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:10.455+0000 7f74d6d8e740 1 -- 172.21.15.47:0/2272294643 shutdown_connections 2024-03-21T10:29:10.461 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:10.455+0000 7f74d6d8e740 1 -- 172.21.15.47:0/2272294643 wait complete. 2024-03-21T10:29:10.461 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:10.456+0000 7f74d6d8e740 1 -- start start 2024-03-21T10:29:10.461 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:10.456+0000 7f74d6d8e740 1 Processor -- start 2024-03-21T10:29:10.461 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:10.457+0000 7f74d6d8e740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x563f39108080 0x563f39116740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:29:10.461 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:10.457+0000 7f74d6d8e740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563f391089a0 0x563f39116c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:29:10.461 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:10.457+0000 7f74d6d8e740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563f39111580 0x563f391171c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:29:10.461 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:10.457+0000 7f74d6d8e740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74cc1b6130 con 0x563f39108080 2024-03-21T10:29:10.461 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:10.457+0000 7f74d6d8e740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74cc1b5ad0 con 0x563f39111580 2024-03-21T10:29:10.461 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:10.457+0000 7f74d6d8e740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74cc1b2fd0 con 0x563f391089a0 2024-03-21T10:29:10.462 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:10.457+0000 7f74d6d8e740 10 client.0 _ll_register_callbacks cb 0x7f74b80d2d90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T10:29:10.462 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:10.457+0000 7f74d1600640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x563f39108080 0x563f39116740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:29:10.462 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:10.457+0000 7f74d1600640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x563f39108080 0x563f39116740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:53012/0 (socket says 172.21.15.47:53012) 2024-03-21T10:29:10.462 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:10.457+0000 7f74d1600640 1 -- 172.21.15.47:0/2312529479 learned_addr learned my addr 172.21.15.47:0/2312529479 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:29:10.462 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:10.457+0000 7f74d2000640 1 --2- 172.21.15.47:0/2312529479 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563f39111580 0x563f391171c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:29:10.462 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:10.458+0000 7f74d1600640 1 -- 172.21.15.47:0/2312529479 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563f391089a0 msgr2=0x563f39116c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:29:10.462 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:10.458+0000 7f74d1600640 1 --2- 172.21.15.47:0/2312529479 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563f391089a0 0x563f39116c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:29:10.462 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:10.458+0000 7f74d1600640 1 -- 172.21.15.47:0/2312529479 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563f39111580 msgr2=0x563f391171c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:29:10.462 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:10.458+0000 7f74d1600640 1 --2- 172.21.15.47:0/2312529479 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563f39111580 0x563f391171c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:29:10.462 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:10.458+0000 7f74d1600640 1 -- 172.21.15.47:0/2312529479 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x563f3910c160 con 0x563f39108080 2024-03-21T10:29:10.462 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:10.458+0000 7f74d1600640 1 --2- 172.21.15.47:0/2312529479 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x563f39108080 0x563f39116740 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7f74c0005940 tx=0x7f74c003f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=de1580e9b33275ef server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:29:10.463 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:10.458+0000 7f74b2a00640 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-03-21T10:29:10.463 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:10.458+0000 7f74b2a00640 1 -- 172.21.15.47:0/2312529479 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74c0005450 con 0x563f39108080 2024-03-21T10:29:10.463 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:10.458+0000 7f74b2a00640 1 -- 172.21.15.47:0/2312529479 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(97..97 src has 1..97) v4 ==== 9474+0+0 (secure 0 0 0) 0x7f74c00046c0 con 0x563f39108080 2024-03-21T10:29:10.463 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:10.458+0000 7f74d6d8e740 1 -- 172.21.15.47:0/2312529479 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f74c4005560 con 0x563f39108080 2024-03-21T10:29:10.463 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:10.459+0000 7f74b2a00640 1 -- 172.21.15.47:0/2312529479 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f74c0063050 con 0x563f39108080 2024-03-21T10:29:10.463 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:10.459+0000 7f74d6d8e740 10 client.16950 fetch_fsmap learned FSMap version 32 2024-03-21T10:29:10.463 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:10.459+0000 7f74d6d8e740 1 -- 172.21.15.47:0/2312529479 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x563f3917de10 con 0x563f39108080 2024-03-21T10:29:10.463 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:10.459+0000 7f74b2a00640 1 -- 172.21.15.47:0/2312529479 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f74c4005560 con 0x563f39108080 2024-03-21T10:29:10.463 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:10.459+0000 7f74b2a00640 1 -- 172.21.15.47:0/2312529479 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 32) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f74c0054a30 con 0x563f39108080 2024-03-21T10:29:10.463 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:10.459+0000 7f74d6d8e740 10 client.16950 fetch_fsmap finished waiting for FSMap version 32 2024-03-21T10:29:10.463 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:10.459+0000 7f74d6d8e740 10 client.16950 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-21T10:29:10.464 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:10.459+0000 7f74d6d8e740 20 client.16950 populate_metadata read hostname 'smithi047' 2024-03-21T10:29:10.464 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:10.459+0000 7f74d6d8e740 1 --2- 172.21.15.47:0/2312529479 >> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] conn(0x563f3917e4c0 0x563f39180900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:29:10.464 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:10.459+0000 7f74d6d8e740 4 client.16950 mds_command: new command op to 34535 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-03-21T10:29:10.464 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:10.459+0000 7f74d6d8e740 1 -- 172.21.15.47:0/2312529479 --> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x563f39074740 con 0x563f3917e4c0 2024-03-21T10:29:10.464 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:10.460+0000 7f74d0c00640 1 --2- 172.21.15.47:0/2312529479 >> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] conn(0x563f3917e4c0 0x563f39180900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:29:10.464 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:10.460+0000 7f74d0c00640 1 --2- 172.21.15.47:0/2312529479 >> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] conn(0x563f3917e4c0 0x563f39180900 crc :-1 s=READY pgs=126 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4ea2aed17a7d1451 server_cookie=865cfe4fed06c87e in_seq=0 out_seq=0 2024-03-21T10:29:10.464 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:10.460+0000 7f74b2a00640 10 client.16950 ms_handle_connect on v2:172.21.15.100:6838/4173168440 2024-03-21T10:29:10.466 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:10.462+0000 7f74b2a00640 1 -- 172.21.15.47:0/2312529479 <== mds.0 v2:172.21.15.100:6838/4173168440 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x563f39074740 con 0x563f3917e4c0 2024-03-21T10:29:10.466 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:10.462+0000 7f74b2a00640 10 client.16950 handle_command_reply: tid=0 2024-03-21T10:29:10.466 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:10.462+0000 7f74d6d8e740 1 client.16950 shutdown 2024-03-21T10:29:10.466 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:10.462+0000 7f74d6d8e740 1 -- 172.21.15.47:0/2312529479 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x563f39108080 msgr2=0x563f39116740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:29:10.466 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:10.462+0000 7f74d6d8e740 1 --2- 172.21.15.47:0/2312529479 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x563f39108080 0x563f39116740 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7f74c0005940 tx=0x7f74c003f000 comp rx=0 tx=0).stop 2024-03-21T10:29:10.466 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:10.463+0000 7f74d6d8e740 1 -- 172.21.15.47:0/2312529479 shutdown_connections 2024-03-21T10:29:10.466 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:10.463+0000 7f74d6d8e740 1 --2- 172.21.15.47:0/2312529479 >> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] conn(0x563f3917e4c0 0x563f39180900 crc :-1 s=READY pgs=126 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:29:10.467 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:10.463+0000 7f74d6d8e740 1 --2- 172.21.15.47:0/2312529479 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563f39111580 0x563f391171c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:29:10.467 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:10.463+0000 7f74d6d8e740 1 --2- 172.21.15.47:0/2312529479 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563f391089a0 0x563f39116c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:29:10.467 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:10.463+0000 7f74d6d8e740 1 --2- 172.21.15.47:0/2312529479 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x563f39108080 0x563f39116740 unknown :-1 s=CLOSED pgs=794 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:29:10.467 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:10.463+0000 7f74d6d8e740 1 -- 172.21.15.47:0/2312529479 >> 172.21.15.47:0/2312529479 conn(0x563f391027b0 msgr2=0x563f39109610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:29:10.467 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:10.463+0000 7f74b2a00640 0 client.16950 ms_handle_reset on v2:172.21.15.100:6838/4173168440 2024-03-21T10:29:10.467 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:10.463+0000 7f74d6d8e740 1 -- 172.21.15.47:0/2312529479 shutdown_connections 2024-03-21T10:29:10.467 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:10.463+0000 7f74d6d8e740 1 -- 172.21.15.47:0/2312529479 wait complete. 2024-03-21T10:29:10.467 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:10.463+0000 7f74d6d8e740 20 client.16950 trim_cache size 0 max 16384 2024-03-21T10:29:10.468 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-03-21T10:29:10.468 INFO:teuthology.orchestra.run.smithi047.stdout: "return_code": 0, 2024-03-21T10:29:10.468 INFO:teuthology.orchestra.run.smithi047.stdout: "scrub_tag": "a0e78fc3-5774-44c1-9533-c92e85b8e4df", 2024-03-21T10:29:10.468 INFO:teuthology.orchestra.run.smithi047.stdout: "mode": "asynchronous" 2024-03-21T10:29:10.468 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-03-21T10:29:10.469 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:10.464+0000 7f74d4000640 1 -- 172.21.15.47:0/360428039 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f74b40a3090 msgr2=0x7f74b40a5550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:29:10.469 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:10.464+0000 7f74d4000640 1 --2- 172.21.15.47:0/360428039 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f74b40a3090 0x7f74b40a5550 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f74c0000c00 tx=0x7f74c00058b0 comp rx=0 tx=0).stop 2024-03-21T10:29:10.469 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:10.464+0000 7f74d4000640 1 -- 172.21.15.47:0/360428039 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f74cc10cf80 msgr2=0x7f74cc19eb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:29:10.469 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:10.464+0000 7f74d4000640 1 --2- 172.21.15.47:0/360428039 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f74cc10cf80 0x7f74cc19eb00 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7f74b800d740 tx=0x7f74b800dc00 comp rx=0 tx=0).stop 2024-03-21T10:29:10.469 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:10.464+0000 7f74d4000640 1 -- 172.21.15.47:0/360428039 shutdown_connections 2024-03-21T10:29:10.469 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:10.464+0000 7f74d4000640 1 --2- 172.21.15.47:0/360428039 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f74b40a3090 0x7f74b40a5550 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:29:10.470 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:10.464+0000 7f74d4000640 1 --2- 172.21.15.47:0/360428039 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f74cc1a2e30 0x7f74cc1a32e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:29:10.470 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:10.464+0000 7f74d4000640 1 --2- 172.21.15.47:0/360428039 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f74cc10cf80 0x7f74cc19eb00 unknown :-1 s=CLOSED pgs=791 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:29:10.470 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:10.464+0000 7f74d4000640 1 --2- 172.21.15.47:0/360428039 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f74cc107eb0 0x7f74cc19e5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:29:10.470 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:10.464+0000 7f74d4000640 1 -- 172.21.15.47:0/360428039 >> 172.21.15.47:0/360428039 conn(0x7f74cc075be0 msgr2=0x7f74cc077070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:29:10.470 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:10.464+0000 7f74d4000640 1 -- 172.21.15.47:0/360428039 shutdown_connections 2024-03-21T10:29:10.470 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:10.464+0000 7f74d4000640 1 -- 172.21.15.47:0/360428039 wait complete. 2024-03-21T10:29:10.481 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-21T10:29:10.481 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-21T10:29:10.678 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:10.678+0000 7fa0ad000640 1 -- 172.21.15.47:0/192222818 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa0a8103200 msgr2=0x7fa0a810b6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:29:10.678 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:10.678+0000 7fa0ad000640 1 --2- 172.21.15.47:0/192222818 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa0a8103200 0x7fa0a810b6e0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fa090009900 tx=0x7fa09002bcc0 comp rx=0 tx=0).stop 2024-03-21T10:29:10.678 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:10.678+0000 7fa0ad000640 1 -- 172.21.15.47:0/192222818 shutdown_connections 2024-03-21T10:29:10.678 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:10.678+0000 7fa0ad000640 1 --2- 172.21.15.47:0/192222818 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa0a810bcb0 0x7fa0a810e0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:29:10.678 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:10.678+0000 7fa0ad000640 1 --2- 172.21.15.47:0/192222818 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa0a8103200 0x7fa0a810b6e0 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:29:10.678 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:10.678+0000 7fa0ad000640 1 --2- 172.21.15.47:0/192222818 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa0a81028e0 0x7fa0a8102cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:29:10.678 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:10.678+0000 7fa0ad000640 1 -- 172.21.15.47:0/192222818 >> 172.21.15.47:0/192222818 conn(0x7fa0a8076920 msgr2=0x7fa0a8076d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:29:10.679 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:10.678+0000 7fa0ad000640 1 -- 172.21.15.47:0/192222818 shutdown_connections 2024-03-21T10:29:10.679 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:10.678+0000 7fa0ad000640 1 -- 172.21.15.47:0/192222818 wait complete. 2024-03-21T10:29:10.681 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:10.680+0000 7fa0ad000640 1 Processor -- start 2024-03-21T10:29:10.681 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:10.680+0000 7fa0ad000640 1 -- start start 2024-03-21T10:29:10.681 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:10.681+0000 7fa0ad000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa0a81028e0 0x7fa0a8100530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:29:10.681 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:10.681+0000 7fa0ad000640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa0a8103200 0x7fa0a8100a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:29:10.681 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:10.681+0000 7fa0ad000640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa0a810bcb0 0x7fa0a8100fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:29:10.681 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:10.681+0000 7fa0ad000640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0a81118f0 con 0x7fa0a8103200 2024-03-21T10:29:10.681 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:10.681+0000 7fa0ad000640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0a8111770 con 0x7fa0a81028e0 2024-03-21T10:29:10.681 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:10.681+0000 7fa0ad000640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0a8111a70 con 0x7fa0a810bcb0 2024-03-21T10:29:10.681 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:10.681+0000 7fa09d400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa0a8103200 0x7fa0a8100a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:29:10.681 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:10.681+0000 7fa09d400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa0a8103200 0x7fa0a8100a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:53028/0 (socket says 172.21.15.47:53028) 2024-03-21T10:29:10.681 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:10.681+0000 7fa09d400640 1 -- 172.21.15.47:0/1609866976 learned_addr learned my addr 172.21.15.47:0/1609866976 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:29:10.681 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:10.681+0000 7fa0a5e00640 1 --2- 172.21.15.47:0/1609866976 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa0a81028e0 0x7fa0a8100530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:29:10.681 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:10.681+0000 7fa09d400640 1 -- 172.21.15.47:0/1609866976 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa0a810bcb0 msgr2=0x7fa0a8100fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:29:10.682 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:10.681+0000 7fa09d400640 1 --2- 172.21.15.47:0/1609866976 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa0a810bcb0 0x7fa0a8100fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:29:10.682 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:10.681+0000 7fa09d400640 1 -- 172.21.15.47:0/1609866976 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa0a81028e0 msgr2=0x7fa0a8100530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:29:10.682 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:10.681+0000 7fa09d400640 1 --2- 172.21.15.47:0/1609866976 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa0a81028e0 0x7fa0a8100530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:29:10.682 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:10.681+0000 7fa09d400640 1 -- 172.21.15.47:0/1609866976 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0a81a6370 con 0x7fa0a8103200 2024-03-21T10:29:10.683 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:10.682+0000 7fa09d400640 1 --2- 172.21.15.47:0/1609866976 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa0a8103200 0x7fa0a8100a70 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7fa090005940 tx=0x7fa0900045e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:29:10.683 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:10.682+0000 7fa09f400640 1 -- 172.21.15.47:0/1609866976 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa090043020 con 0x7fa0a8103200 2024-03-21T10:29:10.683 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:10.682+0000 7fa0ad000640 1 -- 172.21.15.47:0/1609866976 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa0a81a6600 con 0x7fa0a8103200 2024-03-21T10:29:10.683 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:10.682+0000 7fa09f400640 1 -- 172.21.15.47:0/1609866976 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fa09002db90 con 0x7fa0a8103200 2024-03-21T10:29:10.683 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:10.682+0000 7fa09f400640 1 -- 172.21.15.47:0/1609866976 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0900039b0 con 0x7fa0a8103200 2024-03-21T10:29:10.683 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:10.682+0000 7fa0ad000640 1 -- 172.21.15.47:0/1609866976 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa0a81a6b50 con 0x7fa0a8103200 2024-03-21T10:29:10.685 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:10.686+0000 7fa0afc02740 1 Processor -- start 2024-03-21T10:29:10.685 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:10.686+0000 7fa09f400640 1 -- 172.21.15.47:0/1609866976 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 27) v1 ==== 137985+0+0 (secure 0 0 0) 0x7fa090046020 con 0x7fa0a8103200 2024-03-21T10:29:10.685 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:10.686+0000 7fa0afc02740 1 -- start start 2024-03-21T10:29:10.687 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:10.686+0000 7fa0afc02740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55dd0e73ec00 0x55dd0e73d950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:29:10.687 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:10.686+0000 7fa0afc02740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55dd0e73df20 0x55dd0e73e3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:29:10.687 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:10.686+0000 7fa0afc02740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55dd0e747290 0x55dd0e749680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:29:10.687 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:10.686+0000 7fa0afc02740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dd0e6a2250 con 0x55dd0e73ec00 2024-03-21T10:29:10.687 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:10.686+0000 7fa0afc02740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dd0e64aa30 con 0x55dd0e747290 2024-03-21T10:29:10.687 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:10.686+0000 7fa0afc02740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dd0e69e2f0 con 0x55dd0e73df20 2024-03-21T10:29:10.687 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:10.687+0000 7fa0a6800640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55dd0e73ec00 0x55dd0e73d950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:29:10.687 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:10.687+0000 7fa0a6800640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55dd0e73ec00 0x55dd0e73d950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:53038/0 (socket says 172.21.15.47:53038) 2024-03-21T10:29:10.687 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:10.687+0000 7fa0a6800640 1 -- 172.21.15.47:0/4235315446 learned_addr learned my addr 172.21.15.47:0/4235315446 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:29:10.687 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:10.687+0000 7fa09d400640 1 --2- 172.21.15.47:0/4235315446 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55dd0e747290 0x55dd0e749680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:29:10.687 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:10.687+0000 7fa0a5e00640 1 --2- 172.21.15.47:0/4235315446 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55dd0e73df20 0x55dd0e73e3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:29:10.688 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:10.687+0000 7fa0a6800640 1 -- 172.21.15.47:0/4235315446 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55dd0e73df20 msgr2=0x55dd0e73e3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:29:10.688 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:10.687+0000 7fa0a6800640 1 --2- 172.21.15.47:0/4235315446 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55dd0e73df20 0x55dd0e73e3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:29:10.688 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:10.687+0000 7fa0a6800640 1 -- 172.21.15.47:0/4235315446 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55dd0e747290 msgr2=0x55dd0e749680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:29:10.688 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:10.687+0000 7fa0a6800640 1 --2- 172.21.15.47:0/4235315446 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55dd0e747290 0x55dd0e749680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:29:10.688 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:10.687+0000 7fa0a6800640 1 -- 172.21.15.47:0/4235315446 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55dd0e635690 con 0x55dd0e73ec00 2024-03-21T10:29:10.688 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:10.687+0000 7fa09f400640 1 --2- 172.21.15.47:0/1609866976 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7fa08c0a3090 0x7fa08c0a5550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:29:10.688 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:10.688+0000 7fa0a6800640 1 --2- 172.21.15.47:0/4235315446 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55dd0e73ec00 0x55dd0e73d950 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 crypto rx=0x7fa0a000b9e0 tx=0x7fa0a000bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8438c7fdc6c2daf2 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:29:10.692 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:10.688+0000 7fa09f400640 1 -- 172.21.15.47:0/1609866976 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 9474+0+0 (secure 0 0 0) 0x7fa0900f1b70 con 0x7fa0a8103200 2024-03-21T10:29:10.692 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:10.688+0000 7fa0a5e00640 1 --2- 172.21.15.47:0/1609866976 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7fa08c0a3090 0x7fa08c0a5550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:29:10.692 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:10.688+0000 7fa097400640 1 -- 172.21.15.47:0/4235315446 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0a0016070 con 0x55dd0e73ec00 2024-03-21T10:29:10.692 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:10.688+0000 7fa097400640 1 -- 172.21.15.47:0/4235315446 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fa0a00054d0 con 0x55dd0e73ec00 2024-03-21T10:29:10.692 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:10.688+0000 7fa097400640 1 -- 172.21.15.47:0/4235315446 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0a0004d60 con 0x55dd0e73ec00 2024-03-21T10:29:10.692 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:10.689+0000 7fa0afc02740 1 -- 172.21.15.47:0/4235315446 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55dd0e73ec00 msgr2=0x55dd0e73d950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:29:10.692 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:10.689+0000 7fa0afc02740 1 --2- 172.21.15.47:0/4235315446 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55dd0e73ec00 0x55dd0e73d950 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 crypto rx=0x7fa0a000b9e0 tx=0x7fa0a000bea0 comp rx=0 tx=0).stop 2024-03-21T10:29:10.692 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:10.689+0000 7fa0a5e00640 1 --2- 172.21.15.47:0/1609866976 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7fa08c0a3090 0x7fa08c0a5550 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x55dd0e73f150 tx=0x7fa098009500 comp rx=0 tx=0).ready entity=mgr.15687 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:29:10.692 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:10.689+0000 7fa0afc02740 1 -- 172.21.15.47:0/4235315446 shutdown_connections 2024-03-21T10:29:10.693 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:10.689+0000 7fa0afc02740 1 --2- 172.21.15.47:0/4235315446 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55dd0e747290 0x55dd0e749680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:29:10.693 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:10.689+0000 7fa0afc02740 1 --2- 172.21.15.47:0/4235315446 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55dd0e73df20 0x55dd0e73e3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:29:10.693 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:10.689+0000 7fa0afc02740 1 --2- 172.21.15.47:0/4235315446 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55dd0e73ec00 0x55dd0e73d950 unknown :-1 s=CLOSED pgs=796 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:29:10.693 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:10.689+0000 7fa0afc02740 1 -- 172.21.15.47:0/4235315446 >> 172.21.15.47:0/4235315446 conn(0x55dd0e705200 msgr2=0x55dd0e738970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:29:10.693 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:10.689+0000 7fa0afc02740 1 -- 172.21.15.47:0/4235315446 shutdown_connections 2024-03-21T10:29:10.693 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:10.689+0000 7fa0afc02740 1 -- 172.21.15.47:0/4235315446 wait complete. 2024-03-21T10:29:10.693 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:10.689+0000 7fa0afc02740 1 -- start start 2024-03-21T10:29:10.693 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:10.690+0000 7fa0afc02740 1 Processor -- start 2024-03-21T10:29:10.693 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:10.690+0000 7fa0afc02740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55dd0e73df20 0x55dd0e7469b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:29:10.693 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:10.690+0000 7fa0afc02740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55dd0e73ec00 0x55dd0e743d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:29:10.693 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:10.690+0000 7fa0afc02740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55dd0e747290 0x55dd0e744280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:29:10.693 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:10.690+0000 7fa0afc02740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dd0e74a100 con 0x55dd0e747290 2024-03-21T10:29:10.693 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:10.690+0000 7fa0afc02740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dd0e65de80 con 0x55dd0e73ec00 2024-03-21T10:29:10.694 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:10.690+0000 7fa0afc02740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dd0e74a280 con 0x55dd0e73df20 2024-03-21T10:29:10.694 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:10.690+0000 7fa0afc02740 10 client.0 _ll_register_callbacks cb 0x55dd0e670130 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T10:29:10.694 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:10.690+0000 7fa0a6800640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55dd0e747290 0x55dd0e744280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:29:10.694 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:10.690+0000 7fa0a6800640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55dd0e747290 0x55dd0e744280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:53048/0 (socket says 172.21.15.47:53048) 2024-03-21T10:29:10.694 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:10.690+0000 7fa0a6800640 1 -- 172.21.15.47:0/2469626736 learned_addr learned my addr 172.21.15.47:0/2469626736 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:29:10.694 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:10.690+0000 7fa09d400640 1 --2- 172.21.15.47:0/2469626736 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55dd0e73ec00 0x55dd0e743d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:29:10.694 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:10.691+0000 7fa0a5e00640 1 --2- 172.21.15.47:0/2469626736 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55dd0e73df20 0x55dd0e7469b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:29:10.694 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:10.691+0000 7fa0a6800640 1 -- 172.21.15.47:0/2469626736 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55dd0e73df20 msgr2=0x55dd0e7469b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:29:10.694 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:10.691+0000 7fa0a6800640 1 --2- 172.21.15.47:0/2469626736 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55dd0e73df20 0x55dd0e7469b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:29:10.694 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:10.691+0000 7fa0a6800640 1 -- 172.21.15.47:0/2469626736 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55dd0e73ec00 msgr2=0x55dd0e743d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:29:10.694 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:10.691+0000 7fa0a6800640 1 --2- 172.21.15.47:0/2469626736 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55dd0e73ec00 0x55dd0e743d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:29:10.694 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:10.691+0000 7fa0a6800640 1 -- 172.21.15.47:0/2469626736 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55dd0e4807d0 con 0x55dd0e747290 2024-03-21T10:29:10.694 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:10.691+0000 7fa0a6800640 1 --2- 172.21.15.47:0/2469626736 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55dd0e747290 0x55dd0e744280 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7fa0a0005c90 tx=0x7fa0a0005250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4fa69b2e4792ff3e server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:29:10.694 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:10.691+0000 7fa08be00640 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-03-21T10:29:10.695 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:10.692+0000 7fa08be00640 1 -- 172.21.15.47:0/2469626736 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0a0016030 con 0x55dd0e747290 2024-03-21T10:29:10.695 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:10.692+0000 7fa08be00640 1 -- 172.21.15.47:0/2469626736 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(97..97 src has 1..97) v4 ==== 9474+0+0 (secure 0 0 0) 0x7fa0a000d9b0 con 0x55dd0e747290 2024-03-21T10:29:10.695 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:10.692+0000 7fa0afc02740 1 -- 172.21.15.47:0/2469626736 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fa0a00054d0 con 0x55dd0e747290 2024-03-21T10:29:10.695 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:10.692+0000 7fa08be00640 1 -- 172.21.15.47:0/2469626736 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa0a000dbb0 con 0x55dd0e747290 2024-03-21T10:29:10.695 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:10.692+0000 7fa0afc02740 10 client.16962 fetch_fsmap learned FSMap version 32 2024-03-21T10:29:10.695 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:10.692+0000 7fa0afc02740 1 -- 172.21.15.47:0/2469626736 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x55dd0e3b63f0 con 0x55dd0e747290 2024-03-21T10:29:10.695 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:10.692+0000 7fa08be00640 1 -- 172.21.15.47:0/2469626736 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fa0a00054d0 con 0x55dd0e747290 2024-03-21T10:29:10.695 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:10.692+0000 7fa08be00640 1 -- 172.21.15.47:0/2469626736 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 32) v1 ==== 7942+0+0 (secure 0 0 0) 0x7fa0a0022070 con 0x55dd0e747290 2024-03-21T10:29:10.695 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:10.693+0000 7fa0afc02740 10 client.16962 fetch_fsmap finished waiting for FSMap version 32 2024-03-21T10:29:10.695 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:10.693+0000 7fa0afc02740 10 client.16962 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-21T10:29:10.695 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:10.693+0000 7fa0afc02740 20 client.16962 populate_metadata read hostname 'smithi047' 2024-03-21T10:29:10.695 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:10.693+0000 7fa0afc02740 1 --2- 172.21.15.47:0/2469626736 >> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] conn(0x55dd0e7570c0 0x55dd0e7594b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:29:10.695 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:10.693+0000 7fa0afc02740 4 client.16962 mds_command: new command op to 34535 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T10:29:10.695 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:10.693+0000 7fa0afc02740 1 -- 172.21.15.47:0/2469626736 --> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55dd0e6af790 con 0x55dd0e7570c0 2024-03-21T10:29:10.697 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:10.693+0000 7fa0a5e00640 1 --2- 172.21.15.47:0/2469626736 >> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] conn(0x55dd0e7570c0 0x55dd0e7594b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:29:10.697 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:10.694+0000 7fa0a5e00640 1 --2- 172.21.15.47:0/2469626736 >> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] conn(0x55dd0e7570c0 0x55dd0e7594b0 crc :-1 s=READY pgs=127 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=faab32b610739dde server_cookie=e2eabbe43603b6cd in_seq=0 out_seq=0 2024-03-21T10:29:10.697 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:10.694+0000 7fa08be00640 10 client.16962 ms_handle_connect on v2:172.21.15.100:6838/4173168440 2024-03-21T10:29:10.697 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:10.695+0000 7fa08be00640 1 -- 172.21.15.47:0/2469626736 <== mds.0 v2:172.21.15.100:6838/4173168440 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55dd0e6af790 con 0x55dd0e7570c0 2024-03-21T10:29:10.697 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:10.695+0000 7fa08be00640 10 client.16962 handle_command_reply: tid=0 2024-03-21T10:29:10.697 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:10.695+0000 7fa0afc02740 1 client.16962 shutdown 2024-03-21T10:29:10.698 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:10.695+0000 7fa0afc02740 1 -- 172.21.15.47:0/2469626736 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55dd0e747290 msgr2=0x55dd0e744280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:29:10.698 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:10.695+0000 7fa0afc02740 1 --2- 172.21.15.47:0/2469626736 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55dd0e747290 0x55dd0e744280 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7fa0a0005c90 tx=0x7fa0a0005250 comp rx=0 tx=0).stop 2024-03-21T10:29:10.698 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:10.695+0000 7fa0afc02740 1 -- 172.21.15.47:0/2469626736 shutdown_connections 2024-03-21T10:29:10.698 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:10.695+0000 7fa0afc02740 1 --2- 172.21.15.47:0/2469626736 >> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] conn(0x55dd0e7570c0 0x55dd0e7594b0 crc :-1 s=READY pgs=127 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:29:10.698 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:10.695+0000 7fa0afc02740 1 --2- 172.21.15.47:0/2469626736 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55dd0e747290 0x55dd0e744280 unknown :-1 s=CLOSED pgs=797 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:29:10.698 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:10.695+0000 7fa0afc02740 1 --2- 172.21.15.47:0/2469626736 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55dd0e73ec00 0x55dd0e743d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:29:10.698 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:10.695+0000 7fa08be00640 0 client.16962 ms_handle_reset on v2:172.21.15.100:6838/4173168440 2024-03-21T10:29:10.698 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:10.695+0000 7fa0afc02740 1 --2- 172.21.15.47:0/2469626736 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55dd0e73df20 0x55dd0e7469b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:29:10.698 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:10.695+0000 7fa0afc02740 1 -- 172.21.15.47:0/2469626736 >> 172.21.15.47:0/2469626736 conn(0x55dd0e705200 msgr2=0x55dd0e73a6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:29:10.698 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:10.695+0000 7fa0afc02740 1 -- 172.21.15.47:0/2469626736 shutdown_connections 2024-03-21T10:29:10.698 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:10.695+0000 7fa0afc02740 1 -- 172.21.15.47:0/2469626736 wait complete. 2024-03-21T10:29:10.698 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:10.695+0000 7fa0afc02740 20 client.16962 trim_cache size 0 max 16384 2024-03-21T10:29:10.730 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:10.730+0000 7fa0afc02740 1 Processor -- start 2024-03-21T10:29:10.730 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:10.730+0000 7fa0afc02740 1 -- start start 2024-03-21T10:29:10.730 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:10.730+0000 7fa0afc02740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55dd0e73df20 0x55dd0e746470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:29:10.730 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:10.730+0000 7fa0afc02740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55dd0e73ec00 0x55dd0e7469b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:29:10.730 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:10.730+0000 7fa0afc02740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55dd0e747290 0x55dd0e748680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:29:10.730 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:10.730+0000 7fa0afc02740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0a81b0ad0 con 0x55dd0e73ec00 2024-03-21T10:29:10.730 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:10.730+0000 7fa0afc02740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0a81b2450 con 0x55dd0e747290 2024-03-21T10:29:10.730 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:10.730+0000 7fa0afc02740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0a81b1130 con 0x55dd0e73df20 2024-03-21T10:29:10.731 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:10.730+0000 7fa09d400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55dd0e73ec00 0x55dd0e7469b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:29:10.731 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:10.730+0000 7fa09d400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55dd0e73ec00 0x55dd0e7469b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:53064/0 (socket says 172.21.15.47:53064) 2024-03-21T10:29:10.731 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:10.730+0000 7fa09d400640 1 -- 172.21.15.47:0/49641957 learned_addr learned my addr 172.21.15.47:0/49641957 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:29:10.731 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:10.730+0000 7fa0a6800640 1 --2- 172.21.15.47:0/49641957 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55dd0e747290 0x55dd0e748680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:29:10.731 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:10.731+0000 7fa0a5e00640 1 --2- 172.21.15.47:0/49641957 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55dd0e73df20 0x55dd0e746470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:29:10.731 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:10.731+0000 7fa09d400640 1 -- 172.21.15.47:0/49641957 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55dd0e73df20 msgr2=0x55dd0e746470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:29:10.731 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:10.731+0000 7fa09d400640 1 --2- 172.21.15.47:0/49641957 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55dd0e73df20 0x55dd0e746470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:29:10.731 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:10.731+0000 7fa09d400640 1 -- 172.21.15.47:0/49641957 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55dd0e747290 msgr2=0x55dd0e748680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:29:10.731 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:10.731+0000 7fa09d400640 1 --2- 172.21.15.47:0/49641957 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55dd0e747290 0x55dd0e748680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:29:10.731 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:10.731+0000 7fa09d400640 1 -- 172.21.15.47:0/49641957 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55dd0e3760f0 con 0x55dd0e73ec00 2024-03-21T10:29:10.731 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:10.731+0000 7fa0a6800640 1 --2- 172.21.15.47:0/49641957 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55dd0e747290 0x55dd0e748680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T10:29:10.734 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:10.731+0000 7fa09d400640 1 --2- 172.21.15.47:0/49641957 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55dd0e73ec00 0x55dd0e7469b0 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7fa0900ef010 tx=0x7fa090000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ad9f2b16a311b749 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:29:10.734 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:10.731+0000 7fa096a00640 1 -- 172.21.15.47:0/49641957 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0900bdc50 con 0x55dd0e73ec00 2024-03-21T10:29:10.734 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:10.731+0000 7fa096a00640 1 -- 172.21.15.47:0/49641957 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fa0900c1da0 con 0x55dd0e73ec00 2024-03-21T10:29:10.735 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:10.731+0000 7fa096a00640 1 -- 172.21.15.47:0/49641957 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa090005340 con 0x55dd0e73ec00 2024-03-21T10:29:10.735 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:10.732+0000 7fa0afc02740 1 -- 172.21.15.47:0/49641957 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55dd0e73ec00 msgr2=0x55dd0e7469b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:29:10.735 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:10.732+0000 7fa0afc02740 1 --2- 172.21.15.47:0/49641957 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55dd0e73ec00 0x55dd0e7469b0 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7fa0900ef010 tx=0x7fa090000bc0 comp rx=0 tx=0).stop 2024-03-21T10:29:10.735 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:10.732+0000 7fa0afc02740 1 -- 172.21.15.47:0/49641957 shutdown_connections 2024-03-21T10:29:10.735 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:10.732+0000 7fa0afc02740 1 --2- 172.21.15.47:0/49641957 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55dd0e747290 0x55dd0e748680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:29:10.735 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:10.732+0000 7fa0afc02740 1 --2- 172.21.15.47:0/49641957 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55dd0e73ec00 0x55dd0e7469b0 unknown :-1 s=CLOSED pgs=798 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:29:10.735 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:10.732+0000 7fa0afc02740 1 --2- 172.21.15.47:0/49641957 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55dd0e73df20 0x55dd0e746470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:29:10.735 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:10.732+0000 7fa0afc02740 1 -- 172.21.15.47:0/49641957 >> 172.21.15.47:0/49641957 conn(0x55dd0e705200 msgr2=0x55dd0e7384a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:29:10.735 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:10.732+0000 7fa0afc02740 1 -- 172.21.15.47:0/49641957 shutdown_connections 2024-03-21T10:29:10.735 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:10.732+0000 7fa0afc02740 1 -- 172.21.15.47:0/49641957 wait complete. 2024-03-21T10:29:10.735 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:10.732+0000 7fa0afc02740 1 -- start start 2024-03-21T10:29:10.735 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:10.733+0000 7fa0afc02740 1 Processor -- start 2024-03-21T10:29:10.735 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:10.733+0000 7fa0afc02740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55dd0e73df20 0x55dd0e746830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:29:10.736 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:10.733+0000 7fa0afc02740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55dd0e73ec00 0x55dd0e740fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:29:10.736 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:10.733+0000 7fa0afc02740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55dd0e747290 0x55dd0e741540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:29:10.736 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:10.733+0000 7fa0afc02740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0a81b1df0 con 0x55dd0e73ec00 2024-03-21T10:29:10.736 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:10.733+0000 7fa0afc02740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0a81b1790 con 0x55dd0e73df20 2024-03-21T10:29:10.736 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:10.733+0000 7fa0afc02740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0a81aec90 con 0x55dd0e747290 2024-03-21T10:29:10.736 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:10.733+0000 7fa0afc02740 10 client.0 _ll_register_callbacks cb 0x7fa0a000b830 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T10:29:10.736 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:10.733+0000 7fa09d400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55dd0e73ec00 0x55dd0e740fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:29:10.736 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:10.733+0000 7fa0a5e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55dd0e73df20 0x55dd0e746830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:29:10.736 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:10.734+0000 7fa09d400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55dd0e73ec00 0x55dd0e740fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:53074/0 (socket says 172.21.15.47:53074) 2024-03-21T10:29:10.736 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:10.734+0000 7fa0a5e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55dd0e73df20 0x55dd0e746830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:33422/0 (socket says 172.21.15.47:33422) 2024-03-21T10:29:10.736 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:10.734+0000 7fa09d400640 1 -- 172.21.15.47:0/1786453793 learned_addr learned my addr 172.21.15.47:0/1786453793 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:29:10.736 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:10.734+0000 7fa0a5e00640 1 -- 172.21.15.47:0/1786453793 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55dd0e747290 msgr2=0x55dd0e741540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:29:10.736 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:10.734+0000 7fa0a5e00640 1 --2- 172.21.15.47:0/1786453793 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55dd0e747290 0x55dd0e741540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:29:10.736 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:10.734+0000 7fa0a5e00640 1 -- 172.21.15.47:0/1786453793 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55dd0e73ec00 msgr2=0x55dd0e740fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:29:10.737 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:10.734+0000 7fa0a5e00640 1 --2- 172.21.15.47:0/1786453793 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55dd0e73ec00 0x55dd0e740fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:29:10.737 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:10.734+0000 7fa0a5e00640 1 -- 172.21.15.47:0/1786453793 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55dd0e741d70 con 0x55dd0e73df20 2024-03-21T10:29:10.737 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:10.734+0000 7fa09d400640 1 --2- 172.21.15.47:0/1786453793 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55dd0e73ec00 0x55dd0e740fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T10:29:10.737 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:10.734+0000 7fa0a5e00640 1 --2- 172.21.15.47:0/1786453793 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55dd0e73df20 0x55dd0e746830 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x55dd0e759a50 tx=0x7fa098016000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8389fd762298f894 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:29:10.737 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:10.734+0000 7fa08be00640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-21T10:29:10.737 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:10.735+0000 7fa08be00640 1 -- 172.21.15.47:0/1786453793 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa09802f060 con 0x55dd0e73df20 2024-03-21T10:29:10.737 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:10.735+0000 7fa0afc02740 1 -- 172.21.15.47:0/1786453793 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fa0900c1da0 con 0x55dd0e73df20 2024-03-21T10:29:10.737 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:10.735+0000 7fa08be00640 1 -- 172.21.15.47:0/1786453793 <== mon.2 v2:172.21.15.100:3300/0 2 ==== osd_map(97..97 src has 1..97) v4 ==== 9474+0+0 (secure 0 0 0) 0x7fa09800fc20 con 0x55dd0e73df20 2024-03-21T10:29:10.737 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:10.735+0000 7fa08be00640 1 -- 172.21.15.47:0/1786453793 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa098036050 con 0x55dd0e73df20 2024-03-21T10:29:10.737 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:10.735+0000 7fa0afc02740 10 client.36773 fetch_fsmap learned FSMap version 32 2024-03-21T10:29:10.737 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:10.735+0000 7fa0afc02740 1 -- 172.21.15.47:0/1786453793 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x55dd0e7bb030 con 0x55dd0e73df20 2024-03-21T10:29:10.737 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:10.735+0000 7fa08be00640 1 -- 172.21.15.47:0/1786453793 <== mon.2 v2:172.21.15.100:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fa0900c1da0 con 0x55dd0e73df20 2024-03-21T10:29:10.740 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:10.736+0000 7fa08be00640 1 -- 172.21.15.47:0/1786453793 <== mon.2 v2:172.21.15.100:3300/0 5 ==== fsmap(e 32) v1 ==== 7942+0+0 (secure 0 0 0) 0x7fa09801c550 con 0x55dd0e73df20 2024-03-21T10:29:10.740 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:10.736+0000 7fa0afc02740 10 client.36773 fetch_fsmap finished waiting for FSMap version 32 2024-03-21T10:29:10.740 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:10.736+0000 7fa0afc02740 10 client.36773 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-21T10:29:10.740 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:10.736+0000 7fa0afc02740 20 client.36773 populate_metadata read hostname 'smithi047' 2024-03-21T10:29:10.740 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:10.736+0000 7fa0afc02740 1 --2- 172.21.15.47:0/1786453793 >> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] conn(0x55dd0e7bb710 0x55dd0e7bdb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:29:10.740 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:10.736+0000 7fa0afc02740 4 client.36773 mds_command: new command op to 34535 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-21T10:29:10.740 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:10.736+0000 7fa0afc02740 1 -- 172.21.15.47:0/1786453793 --> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55dd0e6a9ae0 con 0x55dd0e7bb710 2024-03-21T10:29:10.740 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:10.736+0000 7fa09d400640 1 --2- 172.21.15.47:0/1786453793 >> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] conn(0x55dd0e7bb710 0x55dd0e7bdb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:29:10.740 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:10.737+0000 7fa09d400640 1 --2- 172.21.15.47:0/1786453793 >> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] conn(0x55dd0e7bb710 0x55dd0e7bdb30 crc :-1 s=READY pgs=128 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7db15b569703b553 server_cookie=5e9a55dfd28fda63 in_seq=0 out_seq=0 2024-03-21T10:29:10.740 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:10.737+0000 7fa08be00640 10 client.36773 ms_handle_connect on v2:172.21.15.100:6838/4173168440 2024-03-21T10:29:10.740 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:10.737+0000 7fa08be00640 1 -- 172.21.15.47:0/1786453793 <== mds.0 v2:172.21.15.100:6838/4173168440 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x55dd0e6a9ae0 con 0x55dd0e7bb710 2024-03-21T10:29:10.741 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:10.737+0000 7fa08be00640 10 client.36773 handle_command_reply: tid=0 2024-03-21T10:29:10.741 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:10.737+0000 7fa0afc02740 1 client.36773 shutdown 2024-03-21T10:29:10.741 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:10.737+0000 7fa0afc02740 1 -- 172.21.15.47:0/1786453793 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55dd0e73df20 msgr2=0x55dd0e746830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:29:10.741 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:10.737+0000 7fa0afc02740 1 --2- 172.21.15.47:0/1786453793 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55dd0e73df20 0x55dd0e746830 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x55dd0e759a50 tx=0x7fa098016000 comp rx=0 tx=0).stop 2024-03-21T10:29:10.741 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:10.737+0000 7fa0afc02740 1 -- 172.21.15.47:0/1786453793 shutdown_connections 2024-03-21T10:29:10.741 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:10.737+0000 7fa0afc02740 1 --2- 172.21.15.47:0/1786453793 >> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] conn(0x55dd0e7bb710 0x55dd0e7bdb30 crc :-1 s=READY pgs=128 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:29:10.741 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:10.737+0000 7fa0afc02740 1 --2- 172.21.15.47:0/1786453793 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55dd0e747290 0x55dd0e741540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:29:10.741 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:10.737+0000 7fa0afc02740 1 --2- 172.21.15.47:0/1786453793 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55dd0e73ec00 0x55dd0e740fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:29:10.741 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:10.737+0000 7fa08be00640 0 client.36773 ms_handle_reset on v2:172.21.15.100:6838/4173168440 2024-03-21T10:29:10.741 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:10.737+0000 7fa0afc02740 1 --2- 172.21.15.47:0/1786453793 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55dd0e73df20 0x55dd0e746830 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:29:10.741 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:10.737+0000 7fa0afc02740 1 -- 172.21.15.47:0/1786453793 >> 172.21.15.47:0/1786453793 conn(0x55dd0e705200 msgr2=0x55dd0e742610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:29:10.741 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:10.737+0000 7fa0afc02740 1 -- 172.21.15.47:0/1786453793 shutdown_connections 2024-03-21T10:29:10.741 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:10.738+0000 7fa0afc02740 1 -- 172.21.15.47:0/1786453793 wait complete. 2024-03-21T10:29:10.741 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:10.738+0000 7fa0afc02740 20 client.36773 trim_cache size 0 max 16384 2024-03-21T10:29:10.742 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-03-21T10:29:10.742 INFO:teuthology.orchestra.run.smithi047.stdout: "status": "IDLE (waiting for more scrubs)", 2024-03-21T10:29:10.742 INFO:teuthology.orchestra.run.smithi047.stdout: "scrubs": { 2024-03-21T10:29:10.742 INFO:teuthology.orchestra.run.smithi047.stdout: "a0e78fc3-5774-44c1-9533-c92e85b8e4df": { 2024-03-21T10:29:10.742 INFO:teuthology.orchestra.run.smithi047.stdout: "path": "/", 2024-03-21T10:29:10.742 INFO:teuthology.orchestra.run.smithi047.stdout: "tag": "a0e78fc3-5774-44c1-9533-c92e85b8e4df", 2024-03-21T10:29:10.742 INFO:teuthology.orchestra.run.smithi047.stdout: "options": "recursive,force" 2024-03-21T10:29:10.742 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-21T10:29:10.742 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-21T10:29:10.742 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-03-21T10:29:10.743 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:10.738+0000 7fa0ad000640 1 -- 172.21.15.47:0/1609866976 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7fa08c0a3090 msgr2=0x7fa08c0a5550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:29:10.743 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:10.738+0000 7fa0ad000640 1 --2- 172.21.15.47:0/1609866976 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7fa08c0a3090 0x7fa08c0a5550 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x55dd0e73f150 tx=0x7fa098009500 comp rx=0 tx=0).stop 2024-03-21T10:29:10.743 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:10.738+0000 7fa0ad000640 1 -- 172.21.15.47:0/1609866976 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa0a8103200 msgr2=0x7fa0a8100a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:29:10.743 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:10.738+0000 7fa0ad000640 1 --2- 172.21.15.47:0/1609866976 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa0a8103200 0x7fa0a8100a70 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7fa090005940 tx=0x7fa0900045e0 comp rx=0 tx=0).stop 2024-03-21T10:29:10.743 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:10.739+0000 7fa0ad000640 1 -- 172.21.15.47:0/1609866976 shutdown_connections 2024-03-21T10:29:10.743 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:10.739+0000 7fa0ad000640 1 --2- 172.21.15.47:0/1609866976 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa0a810bcb0 0x7fa0a8100fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:29:10.744 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:10.739+0000 7fa0ad000640 1 --2- 172.21.15.47:0/1609866976 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa0a8103200 0x7fa0a8100a70 unknown :-1 s=CLOSED pgs=795 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:29:10.744 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:10.739+0000 7fa0ad000640 1 --2- 172.21.15.47:0/1609866976 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7fa08c0a3090 0x7fa08c0a5550 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:29:10.744 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:10.739+0000 7fa0ad000640 1 --2- 172.21.15.47:0/1609866976 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa0a81028e0 0x7fa0a8100530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:29:10.744 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:10.739+0000 7fa0ad000640 1 -- 172.21.15.47:0/1609866976 >> 172.21.15.47:0/1609866976 conn(0x7fa0a8076920 msgr2=0x7fa0a80fd540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:29:10.744 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:10.739+0000 7fa0ad000640 1 -- 172.21.15.47:0/1609866976 shutdown_connections 2024-03-21T10:29:10.744 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:10.739+0000 7fa0ad000640 1 -- 172.21.15.47:0/1609866976 wait complete. 2024-03-21T10:29:10.761 INFO:tasks.cephfs.filesystem:scrub status for tag:a0e78fc3-5774-44c1-9533-c92e85b8e4df - {'path': '/', 'tag': 'a0e78fc3-5774-44c1-9533-c92e85b8e4df', 'options': 'recursive,force'} 2024-03-21T10:29:11.790 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:29:11 smithi157 ceph-mon[35581]: pgmap v119: 129 pgs: 129 active+clean; 277 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-03-21T10:29:11.790 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:29:11 smithi157 ceph-mon[35581]: scrub queued for path: / 2024-03-21T10:29:11.791 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:29:11 smithi157 ceph-mon[35581]: scrub summary: idle+waiting paths [/] 2024-03-21T10:29:11.791 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:29:11 smithi157 ceph-mon[35581]: scrub summary: active paths [/] 2024-03-21T10:29:11.791 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:29:11 smithi157 ceph-mon[35581]: scrub summary: idle+waiting paths [/] 2024-03-21T10:29:11.962 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:29:11 smithi047 ceph-mon[31547]: pgmap v119: 129 pgs: 129 active+clean; 277 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-03-21T10:29:11.962 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:29:11 smithi047 ceph-mon[31547]: scrub queued for path: / 2024-03-21T10:29:11.963 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:29:11 smithi047 ceph-mon[31547]: scrub summary: idle+waiting paths [/] 2024-03-21T10:29:11.963 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:29:11 smithi047 ceph-mon[31547]: scrub summary: active paths [/] 2024-03-21T10:29:11.963 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:29:11 smithi047 ceph-mon[31547]: scrub summary: idle+waiting paths [/] 2024-03-21T10:29:11.977 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:29:11 smithi100 ceph-mon[35193]: pgmap v119: 129 pgs: 129 active+clean; 277 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-03-21T10:29:11.978 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:29:11 smithi100 ceph-mon[35193]: scrub queued for path: / 2024-03-21T10:29:11.978 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:29:11 smithi100 ceph-mon[35193]: scrub summary: idle+waiting paths [/] 2024-03-21T10:29:11.978 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:29:11 smithi100 ceph-mon[35193]: scrub summary: active paths [/] 2024-03-21T10:29:11.978 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:29:11 smithi100 ceph-mon[35193]: scrub summary: idle+waiting paths [/] 2024-03-21T10:29:13.962 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:29:13 smithi047 ceph-mon[31547]: pgmap v120: 129 pgs: 129 active+clean; 277 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 23 KiB/s wr, 28 op/s 2024-03-21T10:29:13.978 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:29:13 smithi100 ceph-mon[35193]: pgmap v120: 129 pgs: 129 active+clean; 277 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 23 KiB/s wr, 28 op/s 2024-03-21T10:29:13.988 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:29:13 smithi157 ceph-mon[35581]: pgmap v120: 129 pgs: 129 active+clean; 277 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 23 KiB/s wr, 28 op/s 2024-03-21T10:29:14.962 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:29:14 smithi047 ceph-mon[31547]: scrub summary: idle 2024-03-21T10:29:14.977 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:29:14 smithi100 ceph-mon[35193]: scrub summary: idle 2024-03-21T10:29:14.988 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:29:14 smithi157 ceph-mon[35581]: scrub summary: idle 2024-03-21T10:29:15.962 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:29:15 smithi047 ceph-mon[31547]: pgmap v121: 129 pgs: 129 active+clean; 277 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 255 B/s wr, 9 op/s 2024-03-21T10:29:15.977 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:29:15 smithi100 ceph-mon[35193]: pgmap v121: 129 pgs: 129 active+clean; 277 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 255 B/s wr, 9 op/s 2024-03-21T10:29:15.988 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:29:15 smithi157 ceph-mon[35581]: pgmap v121: 129 pgs: 129 active+clean; 277 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 255 B/s wr, 9 op/s 2024-03-21T10:29:17.962 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:29:17 smithi047 ceph-mon[31547]: pgmap v122: 129 pgs: 129 active+clean; 277 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 255 B/s wr, 13 op/s 2024-03-21T10:29:17.977 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:29:17 smithi100 ceph-mon[35193]: pgmap v122: 129 pgs: 129 active+clean; 277 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 255 B/s wr, 13 op/s 2024-03-21T10:29:17.988 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:29:17 smithi157 ceph-mon[35581]: pgmap v122: 129 pgs: 129 active+clean; 277 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 255 B/s wr, 13 op/s 2024-03-21T10:29:19.962 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:29:19 smithi047 ceph-mon[31547]: pgmap v123: 129 pgs: 129 active+clean; 277 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 170 B/s wr, 10 op/s 2024-03-21T10:29:19.977 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:29:19 smithi100 ceph-mon[35193]: pgmap v123: 129 pgs: 129 active+clean; 277 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 170 B/s wr, 10 op/s 2024-03-21T10:29:19.988 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:29:19 smithi157 ceph-mon[35581]: pgmap v123: 129 pgs: 129 active+clean; 277 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 170 B/s wr, 10 op/s 2024-03-21T10:29:21.962 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:29:21 smithi047 ceph-mon[31547]: pgmap v124: 129 pgs: 129 active+clean; 277 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 170 B/s wr, 9 op/s 2024-03-21T10:29:21.978 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:29:21 smithi100 ceph-mon[35193]: pgmap v124: 129 pgs: 129 active+clean; 277 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 170 B/s wr, 9 op/s 2024-03-21T10:29:21.990 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:29:21 smithi157 ceph-mon[35581]: pgmap v124: 129 pgs: 129 active+clean; 277 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 170 B/s wr, 9 op/s 2024-03-21T10:29:23.962 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:29:23 smithi047 ceph-mon[31547]: pgmap v125: 129 pgs: 129 active+clean; 277 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 KiB/s rd, 170 B/s wr, 13 op/s 2024-03-21T10:29:23.977 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:29:23 smithi100 ceph-mon[35193]: pgmap v125: 129 pgs: 129 active+clean; 277 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 KiB/s rd, 170 B/s wr, 13 op/s 2024-03-21T10:29:23.988 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:29:23 smithi157 ceph-mon[35581]: pgmap v125: 129 pgs: 129 active+clean; 277 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 KiB/s rd, 170 B/s wr, 13 op/s 2024-03-21T10:29:25.962 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:29:25 smithi047 ceph-mon[31547]: pgmap v126: 129 pgs: 129 active+clean; 277 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-21T10:29:25.977 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:29:25 smithi100 ceph-mon[35193]: pgmap v126: 129 pgs: 129 active+clean; 277 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-21T10:29:25.988 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:29:25 smithi157 ceph-mon[35581]: pgmap v126: 129 pgs: 129 active+clean; 277 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-21T10:29:27.962 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:29:27 smithi047 ceph-mon[31547]: pgmap v127: 129 pgs: 129 active+clean; 277 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 12 op/s 2024-03-21T10:29:27.977 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:29:27 smithi100 ceph-mon[35193]: pgmap v127: 129 pgs: 129 active+clean; 277 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 12 op/s 2024-03-21T10:29:27.988 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:29:27 smithi157 ceph-mon[35581]: pgmap v127: 129 pgs: 129 active+clean; 277 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 12 op/s 2024-03-21T10:29:29.962 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:29:29 smithi047 ceph-mon[31547]: pgmap v128: 129 pgs: 129 active+clean; 277 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-03-21T10:29:29.977 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:29:29 smithi100 ceph-mon[35193]: pgmap v128: 129 pgs: 129 active+clean; 277 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-03-21T10:29:29.988 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:29:29 smithi157 ceph-mon[35581]: pgmap v128: 129 pgs: 129 active+clean; 277 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-03-21T10:29:31.962 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:29:31 smithi047 ceph-mon[31547]: pgmap v129: 129 pgs: 129 active+clean; 277 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-21T10:29:31.977 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:29:31 smithi100 ceph-mon[35193]: pgmap v129: 129 pgs: 129 active+clean; 277 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-21T10:29:31.988 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:29:31 smithi157 ceph-mon[35581]: pgmap v129: 129 pgs: 129 active+clean; 277 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-21T10:29:33.962 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:29:33 smithi047 ceph-mon[31547]: pgmap v130: 129 pgs: 129 active+clean; 277 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-03-21T10:29:33.977 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:29:33 smithi100 ceph-mon[35193]: pgmap v130: 129 pgs: 129 active+clean; 277 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-03-21T10:29:33.988 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:29:33 smithi157 ceph-mon[35581]: pgmap v130: 129 pgs: 129 active+clean; 277 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-03-21T10:29:35.962 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:29:35 smithi047 ceph-mon[31547]: pgmap v131: 129 pgs: 129 active+clean; 277 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-21T10:29:35.977 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:29:35 smithi100 ceph-mon[35193]: pgmap v131: 129 pgs: 129 active+clean; 277 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-21T10:29:35.988 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:29:35 smithi157 ceph-mon[35581]: pgmap v131: 129 pgs: 129 active+clean; 277 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-21T10:29:36.962 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:29:36 smithi047 ceph-mon[31547]: from='mgr.15687 172.21.15.47:0/8730661' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T10:29:36.977 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:29:36 smithi100 ceph-mon[35193]: from='mgr.15687 172.21.15.47:0/8730661' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T10:29:36.988 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:29:36 smithi157 ceph-mon[35581]: from='mgr.15687 172.21.15.47:0/8730661' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T10:29:37.962 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:29:37 smithi047 ceph-mon[31547]: pgmap v132: 129 pgs: 129 active+clean; 277 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 12 op/s 2024-03-21T10:29:37.962 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:29:37 smithi047 ceph-mon[31547]: from='mgr.15687 172.21.15.47:0/8730661' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T10:29:37.962 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:29:37 smithi047 ceph-mon[31547]: from='mgr.15687 172.21.15.47:0/8730661' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T10:29:37.962 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:29:37 smithi047 ceph-mon[31547]: from='mgr.15687 172.21.15.47:0/8730661' entity='mgr.x' 2024-03-21T10:29:37.977 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:29:37 smithi100 ceph-mon[35193]: pgmap v132: 129 pgs: 129 active+clean; 277 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 12 op/s 2024-03-21T10:29:37.978 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:29:37 smithi100 ceph-mon[35193]: from='mgr.15687 172.21.15.47:0/8730661' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T10:29:37.978 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:29:37 smithi100 ceph-mon[35193]: from='mgr.15687 172.21.15.47:0/8730661' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T10:29:37.978 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:29:37 smithi100 ceph-mon[35193]: from='mgr.15687 172.21.15.47:0/8730661' entity='mgr.x' 2024-03-21T10:29:37.988 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:29:37 smithi157 ceph-mon[35581]: pgmap v132: 129 pgs: 129 active+clean; 277 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 12 op/s 2024-03-21T10:29:37.988 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:29:37 smithi157 ceph-mon[35581]: from='mgr.15687 172.21.15.47:0/8730661' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T10:29:37.988 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:29:37 smithi157 ceph-mon[35581]: from='mgr.15687 172.21.15.47:0/8730661' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T10:29:37.988 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:29:37 smithi157 ceph-mon[35581]: from='mgr.15687 172.21.15.47:0/8730661' entity='mgr.x' 2024-03-21T10:29:39.962 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:29:39 smithi047 ceph-mon[31547]: pgmap v133: 129 pgs: 129 active+clean; 277 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-03-21T10:29:39.977 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:29:39 smithi100 ceph-mon[35193]: pgmap v133: 129 pgs: 129 active+clean; 277 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-03-21T10:29:39.988 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:29:39 smithi157 ceph-mon[35581]: pgmap v133: 129 pgs: 129 active+clean; 277 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-03-21T10:29:40.761 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-21T10:29:40.762 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-21T10:29:41.004 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:41.005+0000 7f98fbe00640 1 -- 172.21.15.47:0/3259453478 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f98fc10baf0 msgr2=0x7f98fc10df90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:29:41.005 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:41.005+0000 7f98fbe00640 1 --2- 172.21.15.47:0/3259453478 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f98fc10baf0 0x7f98fc10df90 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 crypto rx=0x7f98f4009ca0 tx=0x7f98f402c190 comp rx=0 tx=0).stop 2024-03-21T10:29:41.005 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:41.005+0000 7f98fbe00640 1 -- 172.21.15.47:0/3259453478 shutdown_connections 2024-03-21T10:29:41.005 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:41.005+0000 7f98fbe00640 1 --2- 172.21.15.47:0/3259453478 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f98fc10baf0 0x7f98fc10df90 unknown :-1 s=CLOSED pgs=799 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:29:41.005 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:41.005+0000 7f98fbe00640 1 --2- 172.21.15.47:0/3259453478 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f98fc102ff0 0x7f98fc10b4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:29:41.005 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:41.005+0000 7f98fbe00640 1 --2- 172.21.15.47:0/3259453478 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f98fc1026d0 0x7f98fc102ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:29:41.005 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:41.005+0000 7f98fbe00640 1 -- 172.21.15.47:0/3259453478 >> 172.21.15.47:0/3259453478 conn(0x7f98fc0fbef0 msgr2=0x7f98fc0fe310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:29:41.005 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:41.005+0000 7f98fbe00640 1 -- 172.21.15.47:0/3259453478 shutdown_connections 2024-03-21T10:29:41.005 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:41.005+0000 7f98fbe00640 1 -- 172.21.15.47:0/3259453478 wait complete. 2024-03-21T10:29:41.007 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:41.006+0000 7f98fbe00640 1 Processor -- start 2024-03-21T10:29:41.007 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:41.006+0000 7f98fbe00640 1 -- start start 2024-03-21T10:29:41.007 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:41.006+0000 7f98fbe00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f98fc1026d0 0x7f98fc19da50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:29:41.008 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:41.006+0000 7f98fbe00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f98fc102ff0 0x7f98fc19df90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:29:41.008 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:41.006+0000 7f98fbe00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f98fc10baf0 0x7f98fc197b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:29:41.008 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:41.006+0000 7f98fbe00640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98fc111780 con 0x7f98fc102ff0 2024-03-21T10:29:41.008 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:41.006+0000 7f98fbe00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98fc111600 con 0x7f98fc10baf0 2024-03-21T10:29:41.008 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:41.006+0000 7f98fbe00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98fc111900 con 0x7f98fc1026d0 2024-03-21T10:29:41.008 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:41.007+0000 7f98f3e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f98fc102ff0 0x7f98fc19df90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:29:41.008 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:41.007+0000 7f98f3e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f98fc102ff0 0x7f98fc19df90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:54148/0 (socket says 172.21.15.47:54148) 2024-03-21T10:29:41.008 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:41.007+0000 7f98f3e00640 1 -- 172.21.15.47:0/2897481531 learned_addr learned my addr 172.21.15.47:0/2897481531 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:29:41.008 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:41.007+0000 7f98fb400640 1 --2- 172.21.15.47:0/2897481531 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f98fc10baf0 0x7f98fc197b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:29:41.008 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:41.007+0000 7f98faa00640 1 --2- 172.21.15.47:0/2897481531 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f98fc1026d0 0x7f98fc19da50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:29:41.008 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:41.007+0000 7f98fb400640 1 -- 172.21.15.47:0/2897481531 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f98fc1026d0 msgr2=0x7f98fc19da50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:29:41.008 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:41.007+0000 7f98fb400640 1 --2- 172.21.15.47:0/2897481531 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f98fc1026d0 0x7f98fc19da50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:29:41.008 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:41.007+0000 7f98fb400640 1 -- 172.21.15.47:0/2897481531 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f98fc102ff0 msgr2=0x7f98fc19df90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:29:41.009 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:41.007+0000 7f98fb400640 1 --2- 172.21.15.47:0/2897481531 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f98fc102ff0 0x7f98fc19df90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:29:41.009 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:41.007+0000 7f98fb400640 1 -- 172.21.15.47:0/2897481531 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98fc198420 con 0x7f98fc10baf0 2024-03-21T10:29:41.009 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:41.007+0000 7f98f3e00640 1 --2- 172.21.15.47:0/2897481531 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f98fc102ff0 0x7f98fc19df90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T10:29:41.009 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:41.008+0000 7f98fb400640 1 --2- 172.21.15.47:0/2897481531 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f98fc10baf0 0x7f98fc197b60 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f98f402c7a0 tx=0x7f98f4035ea0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:29:41.009 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:41.008+0000 7f98f3400640 1 -- 172.21.15.47:0/2897481531 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98f4039040 con 0x7f98fc10baf0 2024-03-21T10:29:41.009 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:41.008+0000 7f98fbe00640 1 -- 172.21.15.47:0/2897481531 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f98fc1986b0 con 0x7f98fc10baf0 2024-03-21T10:29:41.009 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:41.008+0000 7f98f3400640 1 -- 172.21.15.47:0/2897481531 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f98f4044070 con 0x7f98fc10baf0 2024-03-21T10:29:41.009 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:41.008+0000 7f98f3400640 1 -- 172.21.15.47:0/2897481531 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98f4032850 con 0x7f98fc10baf0 2024-03-21T10:29:41.009 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:41.008+0000 7f98fbe00640 1 -- 172.21.15.47:0/2897481531 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f98fc1a47f0 con 0x7f98fc10baf0 2024-03-21T10:29:41.011 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:41.012+0000 7f98f3400640 1 -- 172.21.15.47:0/2897481531 <== mon.2 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 27) v1 ==== 137985+0+0 (secure 0 0 0) 0x7f98f40329f0 con 0x7f98fc10baf0 2024-03-21T10:29:41.011 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:41.012+0000 7f99045c2740 1 Processor -- start 2024-03-21T10:29:41.013 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:41.012+0000 7f99045c2740 1 -- start start 2024-03-21T10:29:41.017 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:41.013+0000 7f99045c2740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563f206a6dd0 0x563f206a5b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:29:41.017 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:41.013+0000 7f99045c2740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563f206a60f0 0x563f206a6570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:29:41.017 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:41.013+0000 7f99045c2740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x563f206af460 0x563f206b1850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:29:41.017 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:41.013+0000 7f99045c2740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x563f2060a250 con 0x563f206af460 2024-03-21T10:29:41.017 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:41.013+0000 7f99045c2740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x563f205b2a30 con 0x563f206a6dd0 2024-03-21T10:29:41.017 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:41.013+0000 7f99045c2740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x563f206062f0 con 0x563f206a60f0 2024-03-21T10:29:41.017 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:41.013+0000 7f98f3e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x563f206af460 0x563f206b1850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:29:41.017 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:41.013+0000 7f98f3e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563f206a6dd0 0x563f206a5b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:29:41.017 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:41.013+0000 7f98f3e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563f206a6dd0 0x563f206a5b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:48078/0 (socket says 172.21.15.47:48078) 2024-03-21T10:29:41.018 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:41.013+0000 7f98f3e00640 1 -- 172.21.15.47:0/2440128900 learned_addr learned my addr 172.21.15.47:0/2440128900 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:29:41.018 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:41.013+0000 7f98faa00640 1 --2- 172.21.15.47:0/2440128900 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563f206a60f0 0x563f206a6570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:29:41.018 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:41.014+0000 7f98f3e00640 1 -- 172.21.15.47:0/2440128900 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563f206a60f0 msgr2=0x563f206a6570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:29:41.018 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:41.014+0000 7f98f3e00640 1 --2- 172.21.15.47:0/2440128900 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563f206a60f0 0x563f206a6570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:29:41.018 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:41.014+0000 7f98f3e00640 1 -- 172.21.15.47:0/2440128900 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563f206a6dd0 msgr2=0x563f206a5b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:29:41.018 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:41.014+0000 7f98f3e00640 1 --2- 172.21.15.47:0/2440128900 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563f206a6dd0 0x563f206a5b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:29:41.018 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:41.014+0000 7f98f3e00640 1 -- 172.21.15.47:0/2440128900 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563f2059d690 con 0x563f206af460 2024-03-21T10:29:41.018 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:41.014+0000 7f98f3400640 1 --2- 172.21.15.47:0/2897481531 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f98e00a3090 0x7f98e00a5550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:29:41.018 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:41.014+0000 7f98f3400640 1 -- 172.21.15.47:0/2897481531 <== mon.2 v2:172.21.15.100:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 9474+0+0 (secure 0 0 0) 0x7f98f40f1ff0 con 0x7f98fc10baf0 2024-03-21T10:29:41.018 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:41.014+0000 7f98f3e00640 1 --2- 172.21.15.47:0/2440128900 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x563f206af460 0x563f206b1850 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7f98e4012bb0 tx=0x7f98e4004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=34bc7950308ffc2a server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:29:41.018 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:41.014+0000 7f98faa00640 1 --2- 172.21.15.47:0/2897481531 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f98e00a3090 0x7f98e00a5550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:29:41.018 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:41.015+0000 7f98ebe00640 1 -- 172.21.15.47:0/2440128900 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98e4012d60 con 0x563f206af460 2024-03-21T10:29:41.018 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:41.015+0000 7f98ebe00640 1 -- 172.21.15.47:0/2440128900 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f98e4005020 con 0x563f206af460 2024-03-21T10:29:41.019 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:41.015+0000 7f98ebe00640 1 -- 172.21.15.47:0/2440128900 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98e40049a0 con 0x563f206af460 2024-03-21T10:29:41.019 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:41.015+0000 7f99045c2740 1 -- 172.21.15.47:0/2440128900 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x563f206af460 msgr2=0x563f206b1850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:29:41.019 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:41.015+0000 7f99045c2740 1 --2- 172.21.15.47:0/2440128900 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x563f206af460 0x563f206b1850 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7f98e4012bb0 tx=0x7f98e4004730 comp rx=0 tx=0).stop 2024-03-21T10:29:41.019 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:41.015+0000 7f98faa00640 1 --2- 172.21.15.47:0/2897481531 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f98e00a3090 0x7f98e00a5550 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x563f206a7320 tx=0x7f98ec005f90 comp rx=0 tx=0).ready entity=mgr.15687 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:29:41.019 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:41.015+0000 7f99045c2740 1 -- 172.21.15.47:0/2440128900 shutdown_connections 2024-03-21T10:29:41.019 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:41.015+0000 7f99045c2740 1 --2- 172.21.15.47:0/2440128900 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x563f206af460 0x563f206b1850 unknown :-1 s=CLOSED pgs=800 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:29:41.019 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:41.015+0000 7f99045c2740 1 --2- 172.21.15.47:0/2440128900 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563f206a60f0 0x563f206a6570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:29:41.019 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:41.015+0000 7f99045c2740 1 --2- 172.21.15.47:0/2440128900 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563f206a6dd0 0x563f206a5b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:29:41.019 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:41.015+0000 7f99045c2740 1 -- 172.21.15.47:0/2440128900 >> 172.21.15.47:0/2440128900 conn(0x563f2066d200 msgr2=0x563f206a0b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:29:41.019 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:41.016+0000 7f99045c2740 1 -- 172.21.15.47:0/2440128900 shutdown_connections 2024-03-21T10:29:41.019 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:41.016+0000 7f99045c2740 1 -- 172.21.15.47:0/2440128900 wait complete. 2024-03-21T10:29:41.019 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:41.016+0000 7f99045c2740 1 -- start start 2024-03-21T10:29:41.019 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:41.016+0000 7f99045c2740 1 Processor -- start 2024-03-21T10:29:41.019 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:41.017+0000 7f99045c2740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x563f206a60f0 0x563f206aebb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:29:41.020 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:41.017+0000 7f99045c2740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563f206a6dd0 0x563f206abf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:29:41.020 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:41.017+0000 7f99045c2740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563f206af460 0x563f206ac480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:29:41.020 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:41.017+0000 7f99045c2740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x563f206b22d0 con 0x563f206a60f0 2024-03-21T10:29:41.020 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:41.017+0000 7f99045c2740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x563f205c5e80 con 0x563f206a6dd0 2024-03-21T10:29:41.020 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:41.017+0000 7f99045c2740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x563f206b2450 con 0x563f206af460 2024-03-21T10:29:41.020 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:41.017+0000 7f99045c2740 10 client.0 _ll_register_callbacks cb 0x563f205d8130 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T10:29:41.020 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:41.017+0000 7f98faa00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x563f206a60f0 0x563f206aebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:29:41.020 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:41.017+0000 7f98faa00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x563f206a60f0 0x563f206aebb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:54168/0 (socket says 172.21.15.47:54168) 2024-03-21T10:29:41.020 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:41.017+0000 7f98faa00640 1 -- 172.21.15.47:0/3038895628 learned_addr learned my addr 172.21.15.47:0/3038895628 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:29:41.020 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:41.017+0000 7f98f3e00640 1 --2- 172.21.15.47:0/3038895628 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563f206a6dd0 0x563f206abf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:29:41.020 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:41.017+0000 7f98fb400640 1 --2- 172.21.15.47:0/3038895628 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563f206af460 0x563f206ac480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:29:41.020 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:41.018+0000 7f98faa00640 1 -- 172.21.15.47:0/3038895628 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563f206af460 msgr2=0x563f206ac480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:29:41.020 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:41.018+0000 7f98faa00640 1 --2- 172.21.15.47:0/3038895628 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563f206af460 0x563f206ac480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:29:41.020 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:41.018+0000 7f98faa00640 1 -- 172.21.15.47:0/3038895628 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563f206a6dd0 msgr2=0x563f206abf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:29:41.020 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:41.018+0000 7f98faa00640 1 --2- 172.21.15.47:0/3038895628 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563f206a6dd0 0x563f206abf40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:29:41.021 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:41.018+0000 7f98faa00640 1 -- 172.21.15.47:0/3038895628 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x563f203e87d0 con 0x563f206a60f0 2024-03-21T10:29:41.023 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:41.018+0000 7f98faa00640 1 --2- 172.21.15.47:0/3038895628 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x563f206a60f0 0x563f206aebb0 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x7f98ec0045e0 tx=0x7f98ec004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2614909f87f416f7 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:29:41.023 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:41.018+0000 7f98e8c00640 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-03-21T10:29:41.023 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:41.018+0000 7f98e8c00640 1 -- 172.21.15.47:0/3038895628 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98ec003dc0 con 0x563f206a60f0 2024-03-21T10:29:41.023 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:41.019+0000 7f98e8c00640 1 -- 172.21.15.47:0/3038895628 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(97..97 src has 1..97) v4 ==== 9474+0+0 (secure 0 0 0) 0x7f98ec02a420 con 0x563f206a60f0 2024-03-21T10:29:41.024 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:41.019+0000 7f99045c2740 1 -- 172.21.15.47:0/3038895628 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f98e4005020 con 0x563f206a60f0 2024-03-21T10:29:41.024 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:41.019+0000 7f98e8c00640 1 -- 172.21.15.47:0/3038895628 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f98ec032050 con 0x563f206a60f0 2024-03-21T10:29:41.024 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:41.019+0000 7f99045c2740 10 client.16980 fetch_fsmap learned FSMap version 32 2024-03-21T10:29:41.024 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:41.019+0000 7f99045c2740 1 -- 172.21.15.47:0/3038895628 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x563f2031e3f0 con 0x563f206a60f0 2024-03-21T10:29:41.024 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:41.019+0000 7f98e8c00640 1 -- 172.21.15.47:0/3038895628 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f98e4005020 con 0x563f206a60f0 2024-03-21T10:29:41.024 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:41.019+0000 7f98e8c00640 1 -- 172.21.15.47:0/3038895628 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 32) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f98ec0206f0 con 0x563f206a60f0 2024-03-21T10:29:41.024 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:41.020+0000 7f99045c2740 10 client.16980 fetch_fsmap finished waiting for FSMap version 32 2024-03-21T10:29:41.024 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:41.020+0000 7f99045c2740 10 client.16980 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-21T10:29:41.024 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:41.020+0000 7f99045c2740 20 client.16980 populate_metadata read hostname 'smithi047' 2024-03-21T10:29:41.024 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:41.020+0000 7f99045c2740 1 --2- 172.21.15.47:0/3038895628 >> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] conn(0x563f206c3890 0x563f206c5ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:29:41.024 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:41.020+0000 7f99045c2740 4 client.16980 mds_command: new command op to 34535 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T10:29:41.024 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:41.020+0000 7f99045c2740 1 -- 172.21.15.47:0/3038895628 --> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x563f20617790 con 0x563f206c3890 2024-03-21T10:29:41.024 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:41.020+0000 7f98f3e00640 1 --2- 172.21.15.47:0/3038895628 >> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] conn(0x563f206c3890 0x563f206c5ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:29:41.025 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:41.021+0000 7f98f3e00640 1 --2- 172.21.15.47:0/3038895628 >> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] conn(0x563f206c3890 0x563f206c5ce0 crc :-1 s=READY pgs=129 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7df8bb2b31af2689 server_cookie=486ce717a9bd242e in_seq=0 out_seq=0 2024-03-21T10:29:41.025 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:41.021+0000 7f98e8c00640 10 client.16980 ms_handle_connect on v2:172.21.15.100:6838/4173168440 2024-03-21T10:29:41.025 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:41.022+0000 7f98e8c00640 1 -- 172.21.15.47:0/3038895628 <== mds.0 v2:172.21.15.100:6838/4173168440 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x563f20617790 con 0x563f206c3890 2024-03-21T10:29:41.025 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:41.022+0000 7f98e8c00640 10 client.16980 handle_command_reply: tid=0 2024-03-21T10:29:41.025 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:41.022+0000 7f99045c2740 1 client.16980 shutdown 2024-03-21T10:29:41.025 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:41.023+0000 7f99045c2740 1 -- 172.21.15.47:0/3038895628 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x563f206a60f0 msgr2=0x563f206aebb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:29:41.025 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:41.023+0000 7f99045c2740 1 --2- 172.21.15.47:0/3038895628 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x563f206a60f0 0x563f206aebb0 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x7f98ec0045e0 tx=0x7f98ec004730 comp rx=0 tx=0).stop 2024-03-21T10:29:41.025 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:41.023+0000 7f99045c2740 1 -- 172.21.15.47:0/3038895628 shutdown_connections 2024-03-21T10:29:41.025 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:41.023+0000 7f99045c2740 1 --2- 172.21.15.47:0/3038895628 >> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] conn(0x563f206c3890 0x563f206c5ce0 crc :-1 s=READY pgs=129 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:29:41.025 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:41.023+0000 7f99045c2740 1 --2- 172.21.15.47:0/3038895628 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563f206af460 0x563f206ac480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:29:41.025 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:41.023+0000 7f99045c2740 1 --2- 172.21.15.47:0/3038895628 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563f206a6dd0 0x563f206abf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:29:41.025 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:41.023+0000 7f99045c2740 1 --2- 172.21.15.47:0/3038895628 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x563f206a60f0 0x563f206aebb0 unknown :-1 s=CLOSED pgs=801 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:29:41.025 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:41.023+0000 7f99045c2740 1 -- 172.21.15.47:0/3038895628 >> 172.21.15.47:0/3038895628 conn(0x563f2066d200 msgr2=0x563f206a2890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:29:41.025 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:41.023+0000 7f98e8c00640 0 client.16980 ms_handle_reset on v2:172.21.15.100:6838/4173168440 2024-03-21T10:29:41.026 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:41.023+0000 7f99045c2740 1 -- 172.21.15.47:0/3038895628 shutdown_connections 2024-03-21T10:29:41.026 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:41.023+0000 7f99045c2740 1 -- 172.21.15.47:0/3038895628 wait complete. 2024-03-21T10:29:41.026 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:41.023+0000 7f99045c2740 20 client.16980 trim_cache size 0 max 16384 2024-03-21T10:29:41.043 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:41.044+0000 7f99045c2740 1 Processor -- start 2024-03-21T10:29:41.043 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:41.044+0000 7f99045c2740 1 -- start start 2024-03-21T10:29:41.044 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:41.045+0000 7f99045c2740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x563f206a60f0 0x563f206ae670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:29:41.045 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:41.045+0000 7f99045c2740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563f206a6dd0 0x563f206aebb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:29:41.045 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:41.045+0000 7f99045c2740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563f206af460 0x563f206bf060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:29:41.045 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:41.045+0000 7f99045c2740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98fc1ac450 con 0x563f206a60f0 2024-03-21T10:29:41.045 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:41.045+0000 7f99045c2740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98fc1addd0 con 0x563f206af460 2024-03-21T10:29:41.045 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:41.045+0000 7f99045c2740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98fc1acab0 con 0x563f206a6dd0 2024-03-21T10:29:41.045 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:41.045+0000 7f98faa00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x563f206a60f0 0x563f206ae670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:29:41.045 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:41.045+0000 7f98faa00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x563f206a60f0 0x563f206ae670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:54178/0 (socket says 172.21.15.47:54178) 2024-03-21T10:29:41.045 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:41.045+0000 7f98faa00640 1 -- 172.21.15.47:0/1766907855 learned_addr learned my addr 172.21.15.47:0/1766907855 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:29:41.045 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:41.045+0000 7f98fb400640 1 --2- 172.21.15.47:0/1766907855 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563f206af460 0x563f206bf060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:29:41.045 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:41.046+0000 7f98f3e00640 1 --2- 172.21.15.47:0/1766907855 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563f206a6dd0 0x563f206aebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:29:41.046 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:41.046+0000 7f98faa00640 1 -- 172.21.15.47:0/1766907855 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563f206a6dd0 msgr2=0x563f206aebb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:29:41.046 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:41.046+0000 7f98faa00640 1 --2- 172.21.15.47:0/1766907855 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563f206a6dd0 0x563f206aebb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:29:41.046 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:41.046+0000 7f98faa00640 1 -- 172.21.15.47:0/1766907855 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563f206af460 msgr2=0x563f206bf060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:29:41.047 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:41.046+0000 7f98faa00640 1 --2- 172.21.15.47:0/1766907855 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563f206af460 0x563f206bf060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:29:41.047 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:41.046+0000 7f98faa00640 1 -- 172.21.15.47:0/1766907855 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563f202de0f0 con 0x563f206a60f0 2024-03-21T10:29:41.047 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:41.046+0000 7f98faa00640 1 --2- 172.21.15.47:0/1766907855 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x563f206a60f0 0x563f206ae670 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7f98ec021430 tx=0x7f98ec004610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a5e998a97ea794df server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:29:41.047 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:41.047+0000 7f98eb400640 1 -- 172.21.15.47:0/1766907855 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98ec020d00 con 0x563f206a60f0 2024-03-21T10:29:41.047 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:41.047+0000 7f98eb400640 1 -- 172.21.15.47:0/1766907855 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f98ec025030 con 0x563f206a60f0 2024-03-21T10:29:41.047 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:41.047+0000 7f98eb400640 1 -- 172.21.15.47:0/1766907855 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98ec021a30 con 0x563f206a60f0 2024-03-21T10:29:41.048 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:41.048+0000 7f99045c2740 1 -- 172.21.15.47:0/1766907855 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x563f206a60f0 msgr2=0x563f206ae670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:29:41.048 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:41.048+0000 7f99045c2740 1 --2- 172.21.15.47:0/1766907855 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x563f206a60f0 0x563f206ae670 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7f98ec021430 tx=0x7f98ec004610 comp rx=0 tx=0).stop 2024-03-21T10:29:41.048 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:41.048+0000 7f99045c2740 1 -- 172.21.15.47:0/1766907855 shutdown_connections 2024-03-21T10:29:41.048 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:41.048+0000 7f99045c2740 1 --2- 172.21.15.47:0/1766907855 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563f206af460 0x563f206bf060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:29:41.048 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:41.048+0000 7f99045c2740 1 --2- 172.21.15.47:0/1766907855 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563f206a6dd0 0x563f206aebb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:29:41.048 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:41.048+0000 7f99045c2740 1 --2- 172.21.15.47:0/1766907855 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x563f206a60f0 0x563f206ae670 unknown :-1 s=CLOSED pgs=802 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:29:41.048 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:41.048+0000 7f99045c2740 1 -- 172.21.15.47:0/1766907855 >> 172.21.15.47:0/1766907855 conn(0x563f2066d200 msgr2=0x563f206a0670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:29:41.048 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:41.048+0000 7f99045c2740 1 -- 172.21.15.47:0/1766907855 shutdown_connections 2024-03-21T10:29:41.049 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:41.048+0000 7f99045c2740 1 -- 172.21.15.47:0/1766907855 wait complete. 2024-03-21T10:29:41.049 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:41.049+0000 7f99045c2740 1 -- start start 2024-03-21T10:29:41.049 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:41.049+0000 7f99045c2740 1 Processor -- start 2024-03-21T10:29:41.051 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:41.049+0000 7f99045c2740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563f206a60f0 0x563f206a91c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:29:41.051 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:41.049+0000 7f99045c2740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x563f206a6dd0 0x563f206a9700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:29:41.051 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:41.049+0000 7f99045c2740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563f206af460 0x563f206a9c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:29:41.051 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:41.049+0000 7f99045c2740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98fc1ad770 con 0x563f206a6dd0 2024-03-21T10:29:41.051 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:41.049+0000 7f99045c2740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98fc1ad110 con 0x563f206a60f0 2024-03-21T10:29:41.051 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:41.049+0000 7f99045c2740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98fc1aa610 con 0x563f206af460 2024-03-21T10:29:41.052 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:41.050+0000 7f99045c2740 10 client.0 _ll_register_callbacks cb 0x7f98ec0044a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T10:29:41.052 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:41.050+0000 7f98f3e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x563f206a6dd0 0x563f206a9700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:29:41.052 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:41.050+0000 7f98f3e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x563f206a6dd0 0x563f206a9700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:54192/0 (socket says 172.21.15.47:54192) 2024-03-21T10:29:41.052 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:41.050+0000 7f98f3e00640 1 -- 172.21.15.47:0/1095553449 learned_addr learned my addr 172.21.15.47:0/1095553449 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:29:41.052 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:41.050+0000 7f98faa00640 1 --2- 172.21.15.47:0/1095553449 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563f206a60f0 0x563f206a91c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:29:41.052 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:41.050+0000 7f98fb400640 1 --2- 172.21.15.47:0/1095553449 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563f206af460 0x563f206a9c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:29:41.052 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:41.050+0000 7f98f3e00640 1 -- 172.21.15.47:0/1095553449 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563f206af460 msgr2=0x563f206a9c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:29:41.052 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:41.050+0000 7f98f3e00640 1 --2- 172.21.15.47:0/1095553449 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563f206af460 0x563f206a9c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:29:41.052 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:41.050+0000 7f98f3e00640 1 -- 172.21.15.47:0/1095553449 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563f206a60f0 msgr2=0x563f206a91c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:29:41.052 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:41.050+0000 7f98f3e00640 1 --2- 172.21.15.47:0/1095553449 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563f206a60f0 0x563f206a91c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:29:41.052 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:41.050+0000 7f98f3e00640 1 -- 172.21.15.47:0/1095553449 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x563f206afa90 con 0x563f206a6dd0 2024-03-21T10:29:41.052 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:41.050+0000 7f98faa00640 1 --2- 172.21.15.47:0/1095553449 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563f206a60f0 0x563f206a91c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T10:29:41.053 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:41.051+0000 7f98f3e00640 1 --2- 172.21.15.47:0/1095553449 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x563f206a6dd0 0x563f206a9700 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x7f98e4014010 tx=0x7f98e4004660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=40608fd90586e85c server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:29:41.053 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:41.051+0000 7f98e8c00640 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-03-21T10:29:41.053 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:41.051+0000 7f98e8c00640 1 -- 172.21.15.47:0/1095553449 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98e402c2d0 con 0x563f206a6dd0 2024-03-21T10:29:41.053 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:41.051+0000 7f98e8c00640 1 -- 172.21.15.47:0/1095553449 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(97..97 src has 1..97) v4 ==== 9474+0+0 (secure 0 0 0) 0x7f98e4012040 con 0x563f206a6dd0 2024-03-21T10:29:41.053 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:41.051+0000 7f99045c2740 1 -- 172.21.15.47:0/1095553449 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f98ec025030 con 0x563f206a6dd0 2024-03-21T10:29:41.053 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:41.051+0000 7f98e8c00640 1 -- 172.21.15.47:0/1095553449 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f98e403c050 con 0x563f206a6dd0 2024-03-21T10:29:41.053 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:41.051+0000 7f99045c2740 10 client.16986 fetch_fsmap learned FSMap version 32 2024-03-21T10:29:41.053 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:41.051+0000 7f99045c2740 1 -- 172.21.15.47:0/1095553449 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x563f2071f590 con 0x563f206a6dd0 2024-03-21T10:29:41.053 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:41.052+0000 7f98e8c00640 1 -- 172.21.15.47:0/1095553449 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f98ec025030 con 0x563f206a6dd0 2024-03-21T10:29:41.053 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:41.052+0000 7f98e8c00640 1 -- 172.21.15.47:0/1095553449 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 32) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f98e4011430 con 0x563f206a6dd0 2024-03-21T10:29:41.053 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:41.052+0000 7f99045c2740 10 client.16986 fetch_fsmap finished waiting for FSMap version 32 2024-03-21T10:29:41.053 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:41.052+0000 7f99045c2740 10 client.16986 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-21T10:29:41.053 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:41.052+0000 7f99045c2740 20 client.16986 populate_metadata read hostname 'smithi047' 2024-03-21T10:29:41.053 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:41.052+0000 7f99045c2740 1 --2- 172.21.15.47:0/1095553449 >> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] conn(0x563f2071fa10 0x563f20721e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:29:41.053 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:41.052+0000 7f99045c2740 4 client.16986 mds_command: new command op to 34535 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-21T10:29:41.054 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:41.052+0000 7f99045c2740 1 -- 172.21.15.47:0/1095553449 --> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x563f20611ae0 con 0x563f2071fa10 2024-03-21T10:29:41.055 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:41.053+0000 7f98faa00640 1 --2- 172.21.15.47:0/1095553449 >> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] conn(0x563f2071fa10 0x563f20721e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:29:41.055 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:41.053+0000 7f98faa00640 1 --2- 172.21.15.47:0/1095553449 >> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] conn(0x563f2071fa10 0x563f20721e00 crc :-1 s=READY pgs=130 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bce80274d45563c8 server_cookie=c9892169edc045db in_seq=0 out_seq=0 2024-03-21T10:29:41.055 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:41.053+0000 7f98e8c00640 10 client.16986 ms_handle_connect on v2:172.21.15.100:6838/4173168440 2024-03-21T10:29:41.055 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:41.053+0000 7f98e8c00640 1 -- 172.21.15.47:0/1095553449 <== mds.0 v2:172.21.15.100:6838/4173168440 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x563f20611ae0 con 0x563f2071fa10 2024-03-21T10:29:41.056 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:41.053+0000 7f98e8c00640 10 client.16986 handle_command_reply: tid=0 2024-03-21T10:29:41.056 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:41.053+0000 7f99045c2740 1 client.16986 shutdown 2024-03-21T10:29:41.056 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:41.054+0000 7f99045c2740 1 -- 172.21.15.47:0/1095553449 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x563f206a6dd0 msgr2=0x563f206a9700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:29:41.056 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:41.054+0000 7f99045c2740 1 --2- 172.21.15.47:0/1095553449 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x563f206a6dd0 0x563f206a9700 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x7f98e4014010 tx=0x7f98e4004660 comp rx=0 tx=0).stop 2024-03-21T10:29:41.056 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:41.054+0000 7f99045c2740 1 -- 172.21.15.47:0/1095553449 shutdown_connections 2024-03-21T10:29:41.056 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:41.054+0000 7f99045c2740 1 --2- 172.21.15.47:0/1095553449 >> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] conn(0x563f2071fa10 0x563f20721e00 crc :-1 s=READY pgs=130 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:29:41.056 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:41.054+0000 7f99045c2740 1 --2- 172.21.15.47:0/1095553449 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563f206af460 0x563f206a9c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:29:41.056 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:41.054+0000 7f99045c2740 1 --2- 172.21.15.47:0/1095553449 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x563f206a6dd0 0x563f206a9700 unknown :-1 s=CLOSED pgs=803 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:29:41.056 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:41.054+0000 7f99045c2740 1 --2- 172.21.15.47:0/1095553449 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563f206a60f0 0x563f206a91c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:29:41.056 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:41.054+0000 7f99045c2740 1 -- 172.21.15.47:0/1095553449 >> 172.21.15.47:0/1095553449 conn(0x563f2066d200 msgr2=0x563f206abee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:29:41.056 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:41.054+0000 7f98e8c00640 0 client.16986 ms_handle_reset on v2:172.21.15.100:6838/4173168440 2024-03-21T10:29:41.056 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:41.054+0000 7f99045c2740 1 -- 172.21.15.47:0/1095553449 shutdown_connections 2024-03-21T10:29:41.056 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:41.054+0000 7f99045c2740 1 -- 172.21.15.47:0/1095553449 wait complete. 2024-03-21T10:29:41.057 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:41.054+0000 7f99045c2740 20 client.16986 trim_cache size 0 max 16384 2024-03-21T10:29:41.057 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-03-21T10:29:41.057 INFO:teuthology.orchestra.run.smithi047.stdout: "status": "no active scrubs running", 2024-03-21T10:29:41.057 INFO:teuthology.orchestra.run.smithi047.stdout: "scrubs": {} 2024-03-21T10:29:41.057 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-03-21T10:29:41.058 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:41.055+0000 7f98fbe00640 1 -- 172.21.15.47:0/2897481531 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f98e00a3090 msgr2=0x7f98e00a5550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:29:41.058 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:41.055+0000 7f98fbe00640 1 --2- 172.21.15.47:0/2897481531 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f98e00a3090 0x7f98e00a5550 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x563f206a7320 tx=0x7f98ec005f90 comp rx=0 tx=0).stop 2024-03-21T10:29:41.058 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:41.055+0000 7f98fbe00640 1 -- 172.21.15.47:0/2897481531 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f98fc10baf0 msgr2=0x7f98fc197b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:29:41.058 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:41.055+0000 7f98fbe00640 1 --2- 172.21.15.47:0/2897481531 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f98fc10baf0 0x7f98fc197b60 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f98f402c7a0 tx=0x7f98f4035ea0 comp rx=0 tx=0).stop 2024-03-21T10:29:41.058 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:41.055+0000 7f98fbe00640 1 -- 172.21.15.47:0/2897481531 shutdown_connections 2024-03-21T10:29:41.058 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:41.055+0000 7f98fbe00640 1 --2- 172.21.15.47:0/2897481531 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f98e00a3090 0x7f98e00a5550 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:29:41.058 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:41.055+0000 7f98fbe00640 1 --2- 172.21.15.47:0/2897481531 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f98fc10baf0 0x7f98fc197b60 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:29:41.058 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:41.055+0000 7f98fbe00640 1 --2- 172.21.15.47:0/2897481531 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f98fc102ff0 0x7f98fc19df90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:29:41.059 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:41.055+0000 7f98fbe00640 1 --2- 172.21.15.47:0/2897481531 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f98fc1026d0 0x7f98fc19da50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:29:41.059 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:41.055+0000 7f98fbe00640 1 -- 172.21.15.47:0/2897481531 >> 172.21.15.47:0/2897481531 conn(0x7f98fc0fbef0 msgr2=0x7f98fc0fcca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:29:41.059 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:41.055+0000 7f98fbe00640 1 -- 172.21.15.47:0/2897481531 shutdown_connections 2024-03-21T10:29:41.059 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:41.055+0000 7f98fbe00640 1 -- 172.21.15.47:0/2897481531 wait complete. 2024-03-21T10:29:41.073 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-03-21T10:29:41.074 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-21T10:29:41.286 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:41.286+0000 7fec05a00640 1 -- 172.21.15.47:0/1984467081 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fec001014b0 msgr2=0x7fec00101930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:29:41.286 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:41.286+0000 7fec05a00640 1 --2- 172.21.15.47:0/1984467081 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fec001014b0 0x7fec00101930 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x7febec009900 tx=0x7febec02bc30 comp rx=0 tx=0).stop 2024-03-21T10:29:41.286 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:41.287+0000 7fec05a00640 1 -- 172.21.15.47:0/1984467081 shutdown_connections 2024-03-21T10:29:41.287 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:41.287+0000 7fec05a00640 1 --2- 172.21.15.47:0/1984467081 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fec00101e70 0x7fec0010e330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:29:41.287 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:41.287+0000 7fec05a00640 1 --2- 172.21.15.47:0/1984467081 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fec001014b0 0x7fec00101930 unknown :-1 s=CLOSED pgs=804 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:29:41.287 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:41.287+0000 7fec05a00640 1 --2- 172.21.15.47:0/1984467081 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fec00107570 0x7fec00107950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:29:41.287 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:41.287+0000 7fec05a00640 1 -- 172.21.15.47:0/1984467081 >> 172.21.15.47:0/1984467081 conn(0x7fec000fcf80 msgr2=0x7fec000ff3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:29:41.287 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:41.287+0000 7fec05a00640 1 -- 172.21.15.47:0/1984467081 shutdown_connections 2024-03-21T10:29:41.287 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:41.287+0000 7fec05a00640 1 -- 172.21.15.47:0/1984467081 wait complete. 2024-03-21T10:29:41.289 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:41.288+0000 7fec05a00640 1 Processor -- start 2024-03-21T10:29:41.289 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:41.288+0000 7fec05a00640 1 -- start start 2024-03-21T10:29:41.289 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:41.288+0000 7fec05a00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fec001014b0 0x7fec00199e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:29:41.290 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:41.288+0000 7fec05a00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fec00101e70 0x7fec0019a390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:29:41.290 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:41.288+0000 7fec05a00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fec00107570 0x7fec0019e710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:29:41.290 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:41.288+0000 7fec05a00640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec00111750 con 0x7fec001014b0 2024-03-21T10:29:41.290 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:41.288+0000 7fec05a00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec001115d0 con 0x7fec00107570 2024-03-21T10:29:41.290 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:41.288+0000 7fec05a00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec001118d0 con 0x7fec00101e70 2024-03-21T10:29:41.290 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:41.288+0000 7febffe00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fec001014b0 0x7fec00199e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:29:41.290 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:41.289+0000 7febffe00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fec001014b0 0x7fec00199e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:54204/0 (socket says 172.21.15.47:54204) 2024-03-21T10:29:41.290 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:41.289+0000 7febffe00640 1 -- 172.21.15.47:0/1316904535 learned_addr learned my addr 172.21.15.47:0/1316904535 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:29:41.290 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:41.289+0000 7fec05000640 1 --2- 172.21.15.47:0/1316904535 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fec00107570 0x7fec0019e710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:29:41.291 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:41.289+0000 7febff400640 1 --2- 172.21.15.47:0/1316904535 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fec00101e70 0x7fec0019a390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:29:41.291 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:41.289+0000 7febffe00640 1 -- 172.21.15.47:0/1316904535 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fec00101e70 msgr2=0x7fec0019a390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:29:41.291 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:41.289+0000 7febffe00640 1 --2- 172.21.15.47:0/1316904535 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fec00101e70 0x7fec0019a390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:29:41.291 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:41.289+0000 7febffe00640 1 -- 172.21.15.47:0/1316904535 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fec00107570 msgr2=0x7fec0019e710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:29:41.291 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:41.289+0000 7febffe00640 1 --2- 172.21.15.47:0/1316904535 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fec00107570 0x7fec0019e710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:29:41.291 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:41.289+0000 7febffe00640 1 -- 172.21.15.47:0/1316904535 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec0019ee40 con 0x7fec001014b0 2024-03-21T10:29:41.291 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:41.289+0000 7febffe00640 1 --2- 172.21.15.47:0/1316904535 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fec001014b0 0x7fec00199e50 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7febf000d770 tx=0x7febf000dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:29:41.291 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:41.290+0000 7febfcc00640 1 -- 172.21.15.47:0/1316904535 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febf0017070 con 0x7fec001014b0 2024-03-21T10:29:41.291 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:41.290+0000 7febfcc00640 1 -- 172.21.15.47:0/1316904535 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7febf00054d0 con 0x7fec001014b0 2024-03-21T10:29:41.291 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:41.290+0000 7fec05a00640 1 -- 172.21.15.47:0/1316904535 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fec0019f130 con 0x7fec001014b0 2024-03-21T10:29:41.291 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:41.290+0000 7febfcc00640 1 -- 172.21.15.47:0/1316904535 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febf0004cf0 con 0x7fec001014b0 2024-03-21T10:29:41.291 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:41.290+0000 7fec05a00640 1 -- 172.21.15.47:0/1316904535 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fec001a69c0 con 0x7fec001014b0 2024-03-21T10:29:41.292 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:41.291+0000 7fec05a00640 1 -- 172.21.15.47:0/1316904535 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fec00102bd0 con 0x7fec001014b0 2024-03-21T10:29:41.292 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:41.293+0000 7febfcc00640 1 -- 172.21.15.47:0/1316904535 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 27) v1 ==== 137985+0+0 (secure 0 0 0) 0x7febf0004e90 con 0x7fec001014b0 2024-03-21T10:29:41.293 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:41.294+0000 7febfcc00640 1 --2- 172.21.15.47:0/1316904535 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7febe40a3090 0x7febe40a5550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:29:41.293 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:41.294+0000 7febfcc00640 1 -- 172.21.15.47:0/1316904535 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 9474+0+0 (secure 0 0 0) 0x7febf00cf5e0 con 0x7fec001014b0 2024-03-21T10:29:41.293 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:41.294+0000 7febff400640 1 --2- 172.21.15.47:0/1316904535 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7febe40a3090 0x7febe40a5550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:29:41.294 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:41.295+0000 7febff400640 1 --2- 172.21.15.47:0/1316904535 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7febe40a3090 0x7febe40a5550 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7febec02c720 tx=0x7febec005fd0 comp rx=0 tx=0).ready entity=mgr.15687 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:29:41.298 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:41.299+0000 7febfcc00640 1 -- 172.21.15.47:0/1316904535 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7febf009bcb0 con 0x7fec001014b0 2024-03-21T10:29:41.599 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:41.600+0000 7fec05a00640 1 -- 172.21.15.47:0/1316904535 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fec001a6c50 con 0x7fec001014b0 2024-03-21T10:29:41.600 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:41.601+0000 7febfcc00640 1 -- 172.21.15.47:0/1316904535 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 32 v32) v1 ==== 94+0+11471 (secure 0 0 0) 0x7febf009b5b0 con 0x7fec001014b0 2024-03-21T10:29:41.601 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-21T10:29:41.601 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":32,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14511,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6835/735724037","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6834","nonce":735724037},{"type":"v1","addr":"172.21.15.47:6835","nonce":735724037}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14529,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6837/2248191724","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6836","nonce":2248191724},{"type":"v1","addr":"172.21.15.47:6837","nonce":2248191724}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":32,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-21T10:21:58.329063+0000","modified":"2024-03-21T10:26:17.169892+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":34535,"mds_1":34493,"mds_2":34454,"mds_3":34427,"mds_4":14619},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14604":{"gid":14604,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.47:6839/2881335356","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6838","nonce":2881335356},{"type":"v1","addr":"172.21.15.47:6839","nonce":2881335356}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_14619":{"gid":14619,"name":"a","rank":4,"incarnation":27,"state":"up:active","state_seq":11,"addr":"172.21.15.47:6841/3401578978","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6840","nonce":3401578978},{"type":"v1","addr":"172.21.15.47:6841","nonce":3401578978}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24406":{"gid":24406,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.157:6833/4000010136","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":4000010136},{"type":"v1","addr":"172.21.15.157:6833","nonce":4000010136}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24457":{"gid":24457,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.157:6835/3902868651","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":3902868651},{"type":"v1","addr":"172.21.15.157:6835","nonce":3902868651}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24475":{"gid":24475,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.157:6837/1908087204","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6836","nonce":1908087204},{"type":"v1","addr":"172.21.15.157:6837","nonce":1908087204}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_25729":{"gid":25729,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.157:6839/1998408416","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6838","nonce":1998408416},{"type":"v1","addr":"172.21.15.157:6839","nonce":1998408416}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_34427":{"gid":34427,"name":"k","rank":3,"incarnation":25,"state":"up:active","state_seq":20,"addr":"172.21.15.100:6833/2723302038","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6832","nonce":2723302038},{"type":"v1","addr":"172.21.15.100:6833","nonce":2723302038}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_34454":{"gid":34454,"name":"e","rank":2,"incarnation":22,"state":"up:active","state_seq":17,"addr":"172.21.15.100:6835/2597849957","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":2597849957},{"type":"v1","addr":"172.21.15.100:6835","nonce":2597849957}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_34493":{"gid":34493,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":13,"addr":"172.21.15.100:6837/3497644162","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6836","nonce":3497644162},{"type":"v1","addr":"172.21.15.100:6837","nonce":3497644162}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_34535":{"gid":34535,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.100:6839/4173168440","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6838","nonce":4173168440},{"type":"v1","addr":"172.21.15.100:6839","nonce":4173168440}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":34535,"qdb_cluster":[34427,14619,34454,34535,34493]},"id":1}]} 2024-03-21T10:29:41.601 INFO:teuthology.orchestra.run.smithi047.stderr:dumped fsmap epoch 32 2024-03-21T10:29:41.603 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:41.604+0000 7fec05a00640 1 -- 172.21.15.47:0/1316904535 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7febe40a3090 msgr2=0x7febe40a5550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:29:41.603 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:41.604+0000 7fec05a00640 1 --2- 172.21.15.47:0/1316904535 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7febe40a3090 0x7febe40a5550 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7febec02c720 tx=0x7febec005fd0 comp rx=0 tx=0).stop 2024-03-21T10:29:41.603 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:41.604+0000 7fec05a00640 1 -- 172.21.15.47:0/1316904535 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fec001014b0 msgr2=0x7fec00199e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:29:41.604 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:41.604+0000 7fec05a00640 1 --2- 172.21.15.47:0/1316904535 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fec001014b0 0x7fec00199e50 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7febf000d770 tx=0x7febf000dc30 comp rx=0 tx=0).stop 2024-03-21T10:29:41.604 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:41.604+0000 7fec05a00640 1 -- 172.21.15.47:0/1316904535 shutdown_connections 2024-03-21T10:29:41.604 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:41.604+0000 7fec05a00640 1 --2- 172.21.15.47:0/1316904535 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7febe40a3090 0x7febe40a5550 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:29:41.604 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:41.604+0000 7fec05a00640 1 --2- 172.21.15.47:0/1316904535 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fec00107570 0x7fec0019e710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:29:41.604 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:41.604+0000 7fec05a00640 1 --2- 172.21.15.47:0/1316904535 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fec00101e70 0x7fec0019a390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:29:41.604 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:41.604+0000 7fec05a00640 1 --2- 172.21.15.47:0/1316904535 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fec001014b0 0x7fec00199e50 unknown :-1 s=CLOSED pgs=805 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:29:41.604 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:41.604+0000 7fec05a00640 1 -- 172.21.15.47:0/1316904535 >> 172.21.15.47:0/1316904535 conn(0x7fec000fcf80 msgr2=0x7fec000fe450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:29:41.604 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:41.604+0000 7fec05a00640 1 -- 172.21.15.47:0/1316904535 shutdown_connections 2024-03-21T10:29:41.604 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:41.604+0000 7fec05a00640 1 -- 172.21.15.47:0/1316904535 wait complete. 2024-03-21T10:29:41.623 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-21T10:29:41.623 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-03-21T10:29:41.834 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:41.835+0000 7f029fe00640 1 -- 172.21.15.47:0/1438445080 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f02a0101ef0 msgr2=0x7f02a010e3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:29:41.834 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:41.835+0000 7f029fe00640 1 --2- 172.21.15.47:0/1438445080 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f02a0101ef0 0x7f02a010e3b0 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 crypto rx=0x7f0298009ca0 tx=0x7f029802c100 comp rx=0 tx=0).stop 2024-03-21T10:29:41.834 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:41.835+0000 7f029fe00640 1 -- 172.21.15.47:0/1438445080 shutdown_connections 2024-03-21T10:29:41.835 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:41.835+0000 7f029fe00640 1 --2- 172.21.15.47:0/1438445080 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f02a0101ef0 0x7f02a010e3b0 unknown :-1 s=CLOSED pgs=806 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:29:41.835 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:41.835+0000 7f029fe00640 1 --2- 172.21.15.47:0/1438445080 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f02a0101530 0x7f02a01019b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:29:41.835 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:41.835+0000 7f029fe00640 1 --2- 172.21.15.47:0/1438445080 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f02a01075f0 0x7f02a01079d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:29:41.835 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:41.835+0000 7f029fe00640 1 -- 172.21.15.47:0/1438445080 >> 172.21.15.47:0/1438445080 conn(0x7f02a00fd000 msgr2=0x7f02a00ff420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:29:41.835 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:41.835+0000 7f029fe00640 1 -- 172.21.15.47:0/1438445080 shutdown_connections 2024-03-21T10:29:41.835 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:41.835+0000 7f029fe00640 1 -- 172.21.15.47:0/1438445080 wait complete. 2024-03-21T10:29:41.837 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:41.836+0000 7f029fe00640 1 Processor -- start 2024-03-21T10:29:41.837 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:41.836+0000 7f029fe00640 1 -- start start 2024-03-21T10:29:41.837 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:41.836+0000 7f029fe00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f02a0101530 0x7f02a00739a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:29:41.837 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:41.836+0000 7f029fe00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f02a0101ef0 0x7f02a0073ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:29:41.837 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:41.836+0000 7f029fe00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f02a01075f0 0x7f02a0074420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:29:41.837 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:41.836+0000 7f029fe00640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02a0111760 con 0x7f02a0101530 2024-03-21T10:29:41.838 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:41.836+0000 7f029fe00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02a01115e0 con 0x7f02a0101ef0 2024-03-21T10:29:41.838 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:41.836+0000 7f029fe00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02a01118e0 con 0x7f02a01075f0 2024-03-21T10:29:41.838 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:41.837+0000 7f029ea00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f02a0101530 0x7f02a00739a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:29:41.838 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:41.837+0000 7f029ea00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f02a0101530 0x7f02a00739a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:54216/0 (socket says 172.21.15.47:54216) 2024-03-21T10:29:41.838 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:41.837+0000 7f029ea00640 1 -- 172.21.15.47:0/3338877164 learned_addr learned my addr 172.21.15.47:0/3338877164 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:29:41.838 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:41.837+0000 7f029e000640 1 --2- 172.21.15.47:0/3338877164 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f02a0101ef0 0x7f02a0073ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:29:41.838 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:41.837+0000 7f029f400640 1 --2- 172.21.15.47:0/3338877164 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f02a01075f0 0x7f02a0074420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:29:41.838 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:41.837+0000 7f029ea00640 1 -- 172.21.15.47:0/3338877164 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f02a01075f0 msgr2=0x7f02a0074420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:29:41.838 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:41.837+0000 7f029ea00640 1 --2- 172.21.15.47:0/3338877164 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f02a01075f0 0x7f02a0074420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:29:41.838 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:41.837+0000 7f029ea00640 1 -- 172.21.15.47:0/3338877164 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f02a0101ef0 msgr2=0x7f02a0073ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:29:41.838 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:41.837+0000 7f029ea00640 1 --2- 172.21.15.47:0/3338877164 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f02a0101ef0 0x7f02a0073ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:29:41.838 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:41.837+0000 7f029ea00640 1 -- 172.21.15.47:0/3338877164 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f02a01a60c0 con 0x7f02a0101530 2024-03-21T10:29:41.838 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:41.837+0000 7f029e000640 1 --2- 172.21.15.47:0/3338877164 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f02a0101ef0 0x7f02a0073ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T10:29:41.839 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:41.838+0000 7f029ea00640 1 --2- 172.21.15.47:0/3338877164 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f02a0101530 0x7f02a00739a0 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x7f029000d950 tx=0x7f029000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:29:41.839 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:41.838+0000 7f02a4a00640 1 -- 172.21.15.47:0/3338877164 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0290017070 con 0x7f02a0101530 2024-03-21T10:29:41.839 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:41.838+0000 7f02a4a00640 1 -- 172.21.15.47:0/3338877164 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f0290009c30 con 0x7f02a0101530 2024-03-21T10:29:41.839 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:41.838+0000 7f029fe00640 1 -- 172.21.15.47:0/3338877164 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f02a01a63b0 con 0x7f02a0101530 2024-03-21T10:29:41.839 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:41.838+0000 7f02a4a00640 1 -- 172.21.15.47:0/3338877164 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f029000f620 con 0x7f02a0101530 2024-03-21T10:29:41.839 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:41.838+0000 7f029fe00640 1 -- 172.21.15.47:0/3338877164 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f02a01a6950 con 0x7f02a0101530 2024-03-21T10:29:41.841 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:41.841+0000 7f02a8ba0740 1 Processor -- start 2024-03-21T10:29:41.841 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:41.841+0000 7f02a8ba0740 1 -- start start 2024-03-21T10:29:41.841 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:41.841+0000 7f02a4a00640 1 -- 172.21.15.47:0/3338877164 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 27) v1 ==== 137985+0+0 (secure 0 0 0) 0x7f02900109a0 con 0x7f02a0101530 2024-03-21T10:29:41.841 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:41.842+0000 7f02a8ba0740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556a84ea88b0 0x556a84ea8c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:29:41.841 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:41.842+0000 7f02a8ba0740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x556a84ea9260 0x556a84eb1810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:29:41.841 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:41.842+0000 7f02a8ba0740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556a84eb1de0 0x556a84eb4230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:29:41.841 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:41.842+0000 7f02a8ba0740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x556a84e0c2c0 con 0x556a84ea9260 2024-03-21T10:29:41.841 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:41.842+0000 7f02a8ba0740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x556a84db4c90 con 0x556a84eb1de0 2024-03-21T10:29:41.841 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:41.842+0000 7f02a8ba0740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x556a84e08360 con 0x556a84ea88b0 2024-03-21T10:29:41.843 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:41.842+0000 7f029ea00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x556a84ea9260 0x556a84eb1810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:29:41.843 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:41.842+0000 7f029e000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556a84eb1de0 0x556a84eb4230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:29:41.843 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:41.842+0000 7f029ea00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x556a84ea9260 0x556a84eb1810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:54218/0 (socket says 172.21.15.47:54218) 2024-03-21T10:29:41.843 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:41.842+0000 7f029ea00640 1 -- 172.21.15.47:0/3543519096 learned_addr learned my addr 172.21.15.47:0/3543519096 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:29:41.843 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:41.842+0000 7f029e000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556a84eb1de0 0x556a84eb4230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:48162/0 (socket says 172.21.15.47:48162) 2024-03-21T10:29:41.843 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:41.842+0000 7f029f400640 1 --2- 172.21.15.47:0/3543519096 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556a84ea88b0 0x556a84ea8c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:29:41.843 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:41.843+0000 7f029e000640 1 -- 172.21.15.47:0/3543519096 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556a84ea88b0 msgr2=0x556a84ea8c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:29:41.843 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:41.843+0000 7f029e000640 1 --2- 172.21.15.47:0/3543519096 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556a84ea88b0 0x556a84ea8c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:29:41.843 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:41.843+0000 7f029e000640 1 -- 172.21.15.47:0/3543519096 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x556a84ea9260 msgr2=0x556a84eb1810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:29:41.843 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:41.843+0000 7f029e000640 1 --2- 172.21.15.47:0/3543519096 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x556a84ea9260 0x556a84eb1810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:29:41.843 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:41.843+0000 7f029e000640 1 -- 172.21.15.47:0/3543519096 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556a84dd9080 con 0x556a84eb1de0 2024-03-21T10:29:41.844 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:41.843+0000 7f029e000640 1 --2- 172.21.15.47:0/3543519096 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556a84eb1de0 0x556a84eb4230 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f028c009fa0 tx=0x7f028c0118d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b4313a944737b273 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:29:41.844 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:41.843+0000 7f028be00640 1 -- 172.21.15.47:0/3543519096 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f028c004030 con 0x556a84eb1de0 2024-03-21T10:29:41.844 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:41.843+0000 7f028be00640 1 -- 172.21.15.47:0/3543519096 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f028c0041d0 con 0x556a84eb1de0 2024-03-21T10:29:41.844 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:41.843+0000 7f028be00640 1 -- 172.21.15.47:0/3543519096 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f028c00f800 con 0x556a84eb1de0 2024-03-21T10:29:41.844 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:41.843+0000 7f02a4a00640 1 --2- 172.21.15.47:0/3338877164 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f02840a3110 0x7f02840a55d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:29:41.844 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:41.844+0000 7f02a4a00640 1 -- 172.21.15.47:0/3338877164 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 9474+0+0 (secure 0 0 0) 0x7f02900cf7d0 con 0x7f02a0101530 2024-03-21T10:29:41.844 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:41.844+0000 7f029f400640 1 --2- 172.21.15.47:0/3338877164 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f02840a3110 0x7f02840a55d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:29:41.847 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:41.844+0000 7f02a8ba0740 1 -- 172.21.15.47:0/3543519096 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556a84eb1de0 msgr2=0x556a84eb4230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:29:41.847 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:41.844+0000 7f02a8ba0740 1 --2- 172.21.15.47:0/3543519096 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556a84eb1de0 0x556a84eb4230 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f028c009fa0 tx=0x7f028c0118d0 comp rx=0 tx=0).stop 2024-03-21T10:29:41.848 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:41.844+0000 7f02a8ba0740 1 -- 172.21.15.47:0/3543519096 shutdown_connections 2024-03-21T10:29:41.848 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:41.844+0000 7f02a8ba0740 1 --2- 172.21.15.47:0/3543519096 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556a84eb1de0 0x556a84eb4230 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:29:41.848 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:41.844+0000 7f02a8ba0740 1 --2- 172.21.15.47:0/3543519096 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x556a84ea9260 0x556a84eb1810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:29:41.848 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:41.844+0000 7f02a8ba0740 1 --2- 172.21.15.47:0/3543519096 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556a84ea88b0 0x556a84ea8c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:29:41.848 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:41.844+0000 7f02a8ba0740 1 -- 172.21.15.47:0/3543519096 >> 172.21.15.47:0/3543519096 conn(0x556a84ea2f80 msgr2=0x556a84ea3360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:29:41.848 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:41.845+0000 7f02a8ba0740 1 -- 172.21.15.47:0/3543519096 shutdown_connections 2024-03-21T10:29:41.848 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:41.845+0000 7f029f400640 1 --2- 172.21.15.47:0/3338877164 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f02840a3110 0x7f02840a55d0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f0298006010 tx=0x7f029802e040 comp rx=0 tx=0).ready entity=mgr.15687 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:29:41.848 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:41.845+0000 7f02a8ba0740 1 -- 172.21.15.47:0/3543519096 wait complete. 2024-03-21T10:29:41.848 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:41.845+0000 7f02a8ba0740 1 -- start start 2024-03-21T10:29:41.848 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:41.845+0000 7f02a8ba0740 1 Processor -- start 2024-03-21T10:29:41.848 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:41.846+0000 7f02a8ba0740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556a84ea88b0 0x556a84eb1640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:29:41.848 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:41.846+0000 7f02a8ba0740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556a84ea9260 0x556a84eae9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:29:41.848 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:41.846+0000 7f02a8ba0740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x556a84eb1de0 0x556a84eaef10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:29:41.849 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:41.846+0000 7f02a8ba0740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x556a84eb4d50 con 0x556a84eb1de0 2024-03-21T10:29:41.849 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:41.846+0000 7f02a8ba0740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x556a84dc80d0 con 0x556a84ea9260 2024-03-21T10:29:41.849 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:41.846+0000 7f02a8ba0740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x556a84eb4ed0 con 0x556a84ea88b0 2024-03-21T10:29:41.849 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:41.846+0000 7f02a8ba0740 10 client.0 _ll_register_callbacks cb 0x556a84dcce80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T10:29:41.849 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:41.846+0000 7f029f400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x556a84eb1de0 0x556a84eaef10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:29:41.849 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:41.846+0000 7f029f400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x556a84eb1de0 0x556a84eaef10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:54220/0 (socket says 172.21.15.47:54220) 2024-03-21T10:29:41.849 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:41.846+0000 7f029f400640 1 -- 172.21.15.47:0/1939721148 learned_addr learned my addr 172.21.15.47:0/1939721148 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:29:41.849 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:41.846+0000 7f029e000640 1 --2- 172.21.15.47:0/1939721148 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556a84ea9260 0x556a84eae9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:29:41.849 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:41.846+0000 7f029ea00640 1 --2- 172.21.15.47:0/1939721148 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556a84ea88b0 0x556a84eb1640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:29:41.849 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:41.846+0000 7f029f400640 1 -- 172.21.15.47:0/1939721148 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556a84ea88b0 msgr2=0x556a84eb1640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:29:41.849 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:41.846+0000 7f029f400640 1 --2- 172.21.15.47:0/1939721148 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556a84ea88b0 0x556a84eb1640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:29:41.849 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:41.846+0000 7f029f400640 1 -- 172.21.15.47:0/1939721148 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556a84ea9260 msgr2=0x556a84eae9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:29:41.849 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:41.846+0000 7f029f400640 1 --2- 172.21.15.47:0/1939721148 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556a84ea9260 0x556a84eae9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:29:41.849 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:41.846+0000 7f029f400640 1 -- 172.21.15.47:0/1939721148 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x556a84b203f0 con 0x556a84eb1de0 2024-03-21T10:29:41.849 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:41.847+0000 7f029f400640 1 --2- 172.21.15.47:0/1939721148 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x556a84eb1de0 0x556a84eaef10 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 crypto rx=0x7f0298036000 tx=0x7f029802c100 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fa9fd6acbd126839 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:29:41.850 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:41.847+0000 7f0288c00640 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-03-21T10:29:41.850 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:41.847+0000 7f0288c00640 1 -- 172.21.15.47:0/1939721148 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0298039040 con 0x556a84eb1de0 2024-03-21T10:29:41.850 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:41.847+0000 7f0288c00640 1 -- 172.21.15.47:0/1939721148 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(97..97 src has 1..97) v4 ==== 9474+0+0 (secure 0 0 0) 0x7f029804b660 con 0x556a84eb1de0 2024-03-21T10:29:41.850 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:41.847+0000 7f02a8ba0740 1 -- 172.21.15.47:0/1939721148 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f028c0041d0 con 0x556a84eb1de0 2024-03-21T10:29:41.850 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:41.847+0000 7f0288c00640 1 -- 172.21.15.47:0/1939721148 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0298059050 con 0x556a84eb1de0 2024-03-21T10:29:41.850 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:41.847+0000 7f02a8ba0740 10 client.17004 fetch_fsmap learned FSMap version 32 2024-03-21T10:29:41.850 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:41.847+0000 7f02a8ba0740 1 -- 172.21.15.47:0/1939721148 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x556a84bea7d0 con 0x556a84eb1de0 2024-03-21T10:29:41.850 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:41.848+0000 7f0288c00640 1 -- 172.21.15.47:0/1939721148 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f028c0041d0 con 0x556a84eb1de0 2024-03-21T10:29:41.850 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:41.848+0000 7f0288c00640 1 -- 172.21.15.47:0/1939721148 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 32) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f0298034790 con 0x556a84eb1de0 2024-03-21T10:29:41.850 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:41.848+0000 7f02a8ba0740 10 client.17004 fetch_fsmap finished waiting for FSMap version 32 2024-03-21T10:29:41.851 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:41.848+0000 7f02a8ba0740 10 client.17004 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.a 2024-03-21T10:29:41.851 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:41.848+0000 7f02a8ba0740 20 client.17004 populate_metadata read hostname 'smithi047' 2024-03-21T10:29:41.851 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:41.848+0000 7f02a8ba0740 1 --2- 172.21.15.47:0/1939721148 >> [v2:172.21.15.47:6840/3401578978,v1:172.21.15.47:6841/3401578978] conn(0x556a84ec1b70 0x556a84ec3fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:29:41.851 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:41.848+0000 7f02a8ba0740 4 client.17004 mds_command: new command op to 14619 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T10:29:41.851 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:41.848+0000 7f02a8ba0740 1 -- 172.21.15.47:0/1939721148 --> [v2:172.21.15.47:6840/3401578978,v1:172.21.15.47:6841/3401578978] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x556a84e19800 con 0x556a84ec1b70 2024-03-21T10:29:41.851 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:41.848+0000 7f029ea00640 1 --2- 172.21.15.47:0/1939721148 >> [v2:172.21.15.47:6840/3401578978,v1:172.21.15.47:6841/3401578978] conn(0x556a84ec1b70 0x556a84ec3fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:29:41.852 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:41.849+0000 7f029ea00640 1 --2- 172.21.15.47:0/1939721148 >> [v2:172.21.15.47:6840/3401578978,v1:172.21.15.47:6841/3401578978] conn(0x556a84ec1b70 0x556a84ec3fd0 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=7388f8eb7cfa3554 server_cookie=63baa98858b747da in_seq=0 out_seq=0 2024-03-21T10:29:41.852 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:41.849+0000 7f0288c00640 10 client.17004 ms_handle_connect on v2:172.21.15.47:6840/3401578978 2024-03-21T10:29:41.852 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:41.849+0000 7f0288c00640 1 -- 172.21.15.47:0/1939721148 <== mds.4 v2:172.21.15.47:6840/3401578978 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x556a84e19800 con 0x556a84ec1b70 2024-03-21T10:29:41.852 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:41.849+0000 7f0288c00640 10 client.17004 handle_command_reply: tid=0 2024-03-21T10:29:41.852 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:41.850+0000 7f02a8ba0740 1 client.17004 shutdown 2024-03-21T10:29:41.852 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:41.850+0000 7f02a8ba0740 1 -- 172.21.15.47:0/1939721148 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x556a84eb1de0 msgr2=0x556a84eaef10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:29:41.852 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:41.850+0000 7f02a8ba0740 1 --2- 172.21.15.47:0/1939721148 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x556a84eb1de0 0x556a84eaef10 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 crypto rx=0x7f0298036000 tx=0x7f029802c100 comp rx=0 tx=0).stop 2024-03-21T10:29:41.853 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:41.850+0000 7f02a8ba0740 1 -- 172.21.15.47:0/1939721148 shutdown_connections 2024-03-21T10:29:41.853 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:41.850+0000 7f02a8ba0740 1 --2- 172.21.15.47:0/1939721148 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x556a84eb1de0 0x556a84eaef10 unknown :-1 s=CLOSED pgs=808 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:29:41.853 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:41.850+0000 7f02a8ba0740 1 --2- 172.21.15.47:0/1939721148 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556a84ea9260 0x556a84eae9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:29:41.853 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:41.850+0000 7f02a8ba0740 1 --2- 172.21.15.47:0/1939721148 >> [v2:172.21.15.47:6840/3401578978,v1:172.21.15.47:6841/3401578978] conn(0x556a84ec1b70 0x556a84ec3fd0 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:29:41.853 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:41.850+0000 7f02a8ba0740 1 --2- 172.21.15.47:0/1939721148 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556a84ea88b0 0x556a84eb1640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:29:41.853 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:41.850+0000 7f02a8ba0740 1 -- 172.21.15.47:0/1939721148 >> 172.21.15.47:0/1939721148 conn(0x556a84ea2f80 msgr2=0x556a84ea4cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:29:41.853 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:41.850+0000 7f0288c00640 0 client.17004 ms_handle_reset on v2:172.21.15.47:6840/3401578978 2024-03-21T10:29:41.853 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:41.850+0000 7f02a8ba0740 1 -- 172.21.15.47:0/1939721148 shutdown_connections 2024-03-21T10:29:41.853 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:41.850+0000 7f02a8ba0740 1 -- 172.21.15.47:0/1939721148 wait complete. 2024-03-21T10:29:41.853 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:41.850+0000 7f02a8ba0740 20 client.17004 trim_cache size 0 max 16384 2024-03-21T10:29:41.863 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:41.864+0000 7f02a8ba0740 1 Processor -- start 2024-03-21T10:29:41.863 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:41.864+0000 7f02a8ba0740 1 -- start start 2024-03-21T10:29:41.864 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:41.864+0000 7f02a8ba0740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556a84ea88b0 0x556a84ea18d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:29:41.864 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:41.864+0000 7f02a8ba0740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x556a84ea9260 0x556a84eb1100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:29:41.864 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:41.864+0000 7f02a8ba0740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556a84eb1de0 0x556a84eb1640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:29:41.864 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:41.864+0000 7f02a8ba0740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02a01b0810 con 0x556a84ea9260 2024-03-21T10:29:41.865 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:41.864+0000 7f02a8ba0740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02a01b2190 con 0x556a84ea88b0 2024-03-21T10:29:41.865 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:41.864+0000 7f02a8ba0740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02a01b0e70 con 0x556a84eb1de0 2024-03-21T10:29:41.865 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:41.865+0000 7f029e000640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x556a84ea9260 0x556a84eb1100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:29:41.865 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:41.865+0000 7f029ea00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556a84ea88b0 0x556a84ea18d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:29:41.865 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:41.865+0000 7f029e000640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x556a84ea9260 0x556a84eb1100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:55024/0 (socket says 172.21.15.47:55024) 2024-03-21T10:29:41.865 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:41.865+0000 7f029e000640 1 -- 172.21.15.47:0/1532922097 learned_addr learned my addr 172.21.15.47:0/1532922097 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:29:41.865 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:41.865+0000 7f029e000640 1 -- 172.21.15.47:0/1532922097 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556a84eb1de0 msgr2=0x556a84eb1640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:29:41.865 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:41.865+0000 7f029e000640 1 --2- 172.21.15.47:0/1532922097 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556a84eb1de0 0x556a84eb1640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:29:41.865 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:41.865+0000 7f029e000640 1 -- 172.21.15.47:0/1532922097 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556a84ea88b0 msgr2=0x556a84ea18d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:29:41.865 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:41.865+0000 7f029e000640 1 --2- 172.21.15.47:0/1532922097 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556a84ea88b0 0x556a84ea18d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:29:41.865 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:41.865+0000 7f029e000640 1 -- 172.21.15.47:0/1532922097 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556a84ea9000 con 0x556a84ea9260 2024-03-21T10:29:41.865 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:41.865+0000 7f029ea00640 1 --2- 172.21.15.47:0/1532922097 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556a84ea88b0 0x556a84ea18d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T10:29:41.865 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:41.865+0000 7f029e000640 1 --2- 172.21.15.47:0/1532922097 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x556a84ea9260 0x556a84eb1100 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x7f028c01b040 tx=0x7f028c009310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=86cd40c1ff7a53e5 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:29:41.869 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:41.866+0000 7f028b400640 1 -- 172.21.15.47:0/1532922097 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f028c026020 con 0x556a84ea9260 2024-03-21T10:29:41.869 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:41.866+0000 7f028b400640 1 -- 172.21.15.47:0/1532922097 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f028c01e040 con 0x556a84ea9260 2024-03-21T10:29:41.869 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:41.866+0000 7f028b400640 1 -- 172.21.15.47:0/1532922097 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f028c01d050 con 0x556a84ea9260 2024-03-21T10:29:41.869 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:41.866+0000 7f02a8ba0740 1 -- 172.21.15.47:0/1532922097 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x556a84ea9260 msgr2=0x556a84eb1100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:29:41.870 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:41.866+0000 7f02a8ba0740 1 --2- 172.21.15.47:0/1532922097 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x556a84ea9260 0x556a84eb1100 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x7f028c01b040 tx=0x7f028c009310 comp rx=0 tx=0).stop 2024-03-21T10:29:41.870 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:41.866+0000 7f02a8ba0740 1 -- 172.21.15.47:0/1532922097 shutdown_connections 2024-03-21T10:29:41.870 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:41.866+0000 7f02a8ba0740 1 --2- 172.21.15.47:0/1532922097 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556a84eb1de0 0x556a84eb1640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:29:41.870 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:41.866+0000 7f02a8ba0740 1 --2- 172.21.15.47:0/1532922097 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x556a84ea9260 0x556a84eb1100 unknown :-1 s=CLOSED pgs=809 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:29:41.870 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:41.866+0000 7f02a8ba0740 1 --2- 172.21.15.47:0/1532922097 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556a84ea88b0 0x556a84ea18d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:29:41.870 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:41.866+0000 7f02a8ba0740 1 -- 172.21.15.47:0/1532922097 >> 172.21.15.47:0/1532922097 conn(0x556a84ea2f80 msgr2=0x556a84ea9f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:29:41.870 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:41.867+0000 7f02a8ba0740 1 -- 172.21.15.47:0/1532922097 shutdown_connections 2024-03-21T10:29:41.870 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:41.867+0000 7f02a8ba0740 1 -- 172.21.15.47:0/1532922097 wait complete. 2024-03-21T10:29:41.870 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:41.867+0000 7f02a8ba0740 1 -- start start 2024-03-21T10:29:41.870 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:41.867+0000 7f02a8ba0740 1 Processor -- start 2024-03-21T10:29:41.870 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:41.868+0000 7f02a8ba0740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x556a84ea88b0 0x556a84eb9da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:29:41.870 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:41.868+0000 7f02a8ba0740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556a84ea9260 0x556a84eb61c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:29:41.870 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:41.868+0000 7f02a8ba0740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556a84eb1de0 0x556a84eb6700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:29:41.871 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:41.868+0000 7f02a8ba0740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02a01b1b30 con 0x556a84ea88b0 2024-03-21T10:29:41.871 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:41.868+0000 7f02a8ba0740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02a01b14d0 con 0x556a84eb1de0 2024-03-21T10:29:41.871 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:41.868+0000 7f02a8ba0740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02a01ae9d0 con 0x556a84ea9260 2024-03-21T10:29:41.871 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:41.868+0000 7f02a8ba0740 10 client.0 _ll_register_callbacks cb 0x7f0298037070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T10:29:41.871 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:41.868+0000 7f029ea00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x556a84ea88b0 0x556a84eb9da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:29:41.871 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:41.868+0000 7f029ea00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x556a84ea88b0 0x556a84eb9da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:55028/0 (socket says 172.21.15.47:55028) 2024-03-21T10:29:41.871 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:41.868+0000 7f029ea00640 1 -- 172.21.15.47:0/3729652997 learned_addr learned my addr 172.21.15.47:0/3729652997 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:29:41.871 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:41.868+0000 7f029f400640 1 --2- 172.21.15.47:0/3729652997 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556a84eb1de0 0x556a84eb6700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:29:41.871 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:41.868+0000 7f029ea00640 1 -- 172.21.15.47:0/3729652997 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556a84ea9260 msgr2=0x556a84eb61c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:29:41.871 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:41.868+0000 7f029ea00640 1 --2- 172.21.15.47:0/3729652997 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556a84ea9260 0x556a84eb61c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:29:41.871 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:41.868+0000 7f029ea00640 1 -- 172.21.15.47:0/3729652997 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556a84eb1de0 msgr2=0x556a84eb6700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:29:41.871 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:41.868+0000 7f029ea00640 1 --2- 172.21.15.47:0/3729652997 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556a84eb1de0 0x556a84eb6700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:29:41.871 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:41.868+0000 7f029ea00640 1 -- 172.21.15.47:0/3729652997 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x556a84eb6d70 con 0x556a84ea88b0 2024-03-21T10:29:41.871 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:41.869+0000 7f029ea00640 1 --2- 172.21.15.47:0/3729652997 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x556a84ea88b0 0x556a84eb9da0 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 crypto rx=0x7f029009b4a0 tx=0x7f029000d8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aef26036994e73e5 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:29:41.871 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:41.869+0000 7f0288c00640 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-03-21T10:29:41.872 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:41.869+0000 7f0288c00640 1 -- 172.21.15.47:0/3729652997 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0290020040 con 0x556a84ea88b0 2024-03-21T10:29:41.872 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:41.869+0000 7f0288c00640 1 -- 172.21.15.47:0/3729652997 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(97..97 src has 1..97) v4 ==== 9474+0+0 (secure 0 0 0) 0x7f0290097e10 con 0x556a84ea88b0 2024-03-21T10:29:41.872 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:41.869+0000 7f02a8ba0740 1 -- 172.21.15.47:0/3729652997 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f028c01e040 con 0x556a84ea88b0 2024-03-21T10:29:41.872 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:41.869+0000 7f0288c00640 1 -- 172.21.15.47:0/3729652997 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0290009de0 con 0x556a84ea88b0 2024-03-21T10:29:41.872 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:41.869+0000 7f02a8ba0740 10 client.17010 fetch_fsmap learned FSMap version 32 2024-03-21T10:29:41.872 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:41.869+0000 7f02a8ba0740 1 -- 172.21.15.47:0/3729652997 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x556a84f1e360 con 0x556a84ea88b0 2024-03-21T10:29:41.872 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:41.869+0000 7f0288c00640 1 -- 172.21.15.47:0/3729652997 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f028c01e040 con 0x556a84ea88b0 2024-03-21T10:29:41.872 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:41.869+0000 7f0288c00640 1 -- 172.21.15.47:0/3729652997 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 32) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f02900cf030 con 0x556a84ea88b0 2024-03-21T10:29:41.872 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:41.869+0000 7f02a8ba0740 10 client.17010 fetch_fsmap finished waiting for FSMap version 32 2024-03-21T10:29:41.872 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:41.869+0000 7f02a8ba0740 10 client.17010 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.a 2024-03-21T10:29:41.872 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:41.869+0000 7f02a8ba0740 20 client.17010 populate_metadata read hostname 'smithi047' 2024-03-21T10:29:41.872 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:41.869+0000 7f02a8ba0740 1 --2- 172.21.15.47:0/3729652997 >> [v2:172.21.15.47:6840/3401578978,v1:172.21.15.47:6841/3401578978] conn(0x556a84f1ea40 0x556a84f20e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:29:41.872 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:41.870+0000 7f02a8ba0740 4 client.17010 mds_command: new command op to 14619 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-21T10:29:41.872 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:41.870+0000 7f02a8ba0740 1 -- 172.21.15.47:0/3729652997 --> [v2:172.21.15.47:6840/3401578978,v1:172.21.15.47:6841/3401578978] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x556a84e13b50 con 0x556a84f1ea40 2024-03-21T10:29:41.873 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:41.870+0000 7f029e000640 1 --2- 172.21.15.47:0/3729652997 >> [v2:172.21.15.47:6840/3401578978,v1:172.21.15.47:6841/3401578978] conn(0x556a84f1ea40 0x556a84f20e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:29:41.873 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:41.870+0000 7f029e000640 1 --2- 172.21.15.47:0/3729652997 >> [v2:172.21.15.47:6840/3401578978,v1:172.21.15.47:6841/3401578978] conn(0x556a84f1ea40 0x556a84f20e60 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=1126f908e6b3ead9 server_cookie=a07f5f14127b699a in_seq=0 out_seq=0 2024-03-21T10:29:41.873 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:41.870+0000 7f0288c00640 10 client.17010 ms_handle_connect on v2:172.21.15.47:6840/3401578978 2024-03-21T10:29:41.874 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:41.871+0000 7f0288c00640 1 -- 172.21.15.47:0/3729652997 <== mds.4 v2:172.21.15.47:6840/3401578978 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x556a84e13b50 con 0x556a84f1ea40 2024-03-21T10:29:41.874 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:41.871+0000 7f0288c00640 10 client.17010 handle_command_reply: tid=0 2024-03-21T10:29:41.874 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:41.871+0000 7f02a8ba0740 1 client.17010 shutdown 2024-03-21T10:29:41.874 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:41.871+0000 7f02a8ba0740 1 -- 172.21.15.47:0/3729652997 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x556a84ea88b0 msgr2=0x556a84eb9da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:29:41.874 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:41.871+0000 7f02a8ba0740 1 --2- 172.21.15.47:0/3729652997 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x556a84ea88b0 0x556a84eb9da0 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 crypto rx=0x7f029009b4a0 tx=0x7f029000d8d0 comp rx=0 tx=0).stop 2024-03-21T10:29:41.874 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:41.871+0000 7f02a8ba0740 1 -- 172.21.15.47:0/3729652997 shutdown_connections 2024-03-21T10:29:41.874 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:41.871+0000 7f02a8ba0740 1 --2- 172.21.15.47:0/3729652997 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556a84eb1de0 0x556a84eb6700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:29:41.874 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:41.871+0000 7f02a8ba0740 1 --2- 172.21.15.47:0/3729652997 >> [v2:172.21.15.47:6840/3401578978,v1:172.21.15.47:6841/3401578978] conn(0x556a84f1ea40 0x556a84f20e60 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:29:41.874 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:41.871+0000 7f02a8ba0740 1 --2- 172.21.15.47:0/3729652997 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556a84ea9260 0x556a84eb61c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:29:41.874 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:41.871+0000 7f02a8ba0740 1 --2- 172.21.15.47:0/3729652997 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x556a84ea88b0 0x556a84eb9da0 unknown :-1 s=CLOSED pgs=810 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:29:41.874 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:41.871+0000 7f0288c00640 0 client.17010 ms_handle_reset on v2:172.21.15.47:6840/3401578978 2024-03-21T10:29:41.875 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:41.871+0000 7f02a8ba0740 1 -- 172.21.15.47:0/3729652997 >> 172.21.15.47:0/3729652997 conn(0x556a84ea2f80 msgr2=0x556a84ea4020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:29:41.875 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:41.871+0000 7f02a8ba0740 1 -- 172.21.15.47:0/3729652997 shutdown_connections 2024-03-21T10:29:41.875 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:41.871+0000 7f02a8ba0740 1 -- 172.21.15.47:0/3729652997 wait complete. 2024-03-21T10:29:41.875 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:41.871+0000 7f02a8ba0740 20 client.17010 trim_cache size 0 max 16384 2024-03-21T10:29:41.875 INFO:teuthology.orchestra.run.smithi047.stdout:[] 2024-03-21T10:29:41.876 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:41.872+0000 7f029fe00640 1 -- 172.21.15.47:0/3338877164 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f02840a3110 msgr2=0x7f02840a55d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:29:41.876 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:41.872+0000 7f029fe00640 1 --2- 172.21.15.47:0/3338877164 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f02840a3110 0x7f02840a55d0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f0298006010 tx=0x7f029802e040 comp rx=0 tx=0).stop 2024-03-21T10:29:41.876 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:41.872+0000 7f029fe00640 1 -- 172.21.15.47:0/3338877164 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f02a0101530 msgr2=0x7f02a00739a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:29:41.876 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:41.872+0000 7f029fe00640 1 --2- 172.21.15.47:0/3338877164 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f02a0101530 0x7f02a00739a0 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x7f029000d950 tx=0x7f029000de10 comp rx=0 tx=0).stop 2024-03-21T10:29:41.876 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:41.872+0000 7f029fe00640 1 -- 172.21.15.47:0/3338877164 shutdown_connections 2024-03-21T10:29:41.876 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:41.872+0000 7f029fe00640 1 --2- 172.21.15.47:0/3338877164 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f02a01075f0 0x7f02a0074420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:29:41.876 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:41.872+0000 7f029fe00640 1 --2- 172.21.15.47:0/3338877164 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f02840a3110 0x7f02840a55d0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:29:41.876 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:41.872+0000 7f029fe00640 1 --2- 172.21.15.47:0/3338877164 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f02a0101ef0 0x7f02a0073ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:29:41.876 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:41.872+0000 7f029fe00640 1 --2- 172.21.15.47:0/3338877164 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f02a0101530 0x7f02a00739a0 unknown :-1 s=CLOSED pgs=807 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:29:41.876 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:41.872+0000 7f029fe00640 1 -- 172.21.15.47:0/3338877164 >> 172.21.15.47:0/3338877164 conn(0x7f02a00fd000 msgr2=0x7f02a00fe7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:29:41.876 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:41.872+0000 7f029fe00640 1 -- 172.21.15.47:0/3338877164 shutdown_connections 2024-03-21T10:29:41.877 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:41.872+0000 7f029fe00640 1 -- 172.21.15.47:0/3338877164 wait complete. 2024-03-21T10:29:41.886 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:29:41 smithi047 ceph-mon[31547]: pgmap v134: 129 pgs: 129 active+clean; 277 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-21T10:29:41.887 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:29:41 smithi047 ceph-mon[31547]: from='client.? 172.21.15.47:0/1316904535' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-21T10:29:41.891 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-21T10:29:41.891 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-03-21T10:29:41.977 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:29:41 smithi100 ceph-mon[35193]: pgmap v134: 129 pgs: 129 active+clean; 277 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-21T10:29:41.977 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:29:41 smithi100 ceph-mon[35193]: from='client.? 172.21.15.47:0/1316904535' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-21T10:29:41.988 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:29:41 smithi157 ceph-mon[35581]: pgmap v134: 129 pgs: 129 active+clean; 277 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-21T10:29:41.988 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:29:41 smithi157 ceph-mon[35581]: from='client.? 172.21.15.47:0/1316904535' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-21T10:29:42.106 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.107+0000 7faa49800640 1 -- 172.21.15.47:0/791821010 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7faa44102ff0 msgr2=0x7faa4410b4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:29:42.106 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.107+0000 7faa49800640 1 --2- 172.21.15.47:0/791821010 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7faa44102ff0 0x7faa4410b4d0 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x7faa2c009900 tx=0x7faa2c02bcc0 comp rx=0 tx=0).stop 2024-03-21T10:29:42.107 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.107+0000 7faa49800640 1 -- 172.21.15.47:0/791821010 shutdown_connections 2024-03-21T10:29:42.107 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.107+0000 7faa49800640 1 --2- 172.21.15.47:0/791821010 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7faa4410baf0 0x7faa4410df90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:29:42.107 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.107+0000 7faa49800640 1 --2- 172.21.15.47:0/791821010 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7faa44102ff0 0x7faa4410b4d0 unknown :-1 s=CLOSED pgs=811 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:29:42.107 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.107+0000 7faa49800640 1 --2- 172.21.15.47:0/791821010 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7faa441026d0 0x7faa44102ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:29:42.107 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.107+0000 7faa49800640 1 -- 172.21.15.47:0/791821010 >> 172.21.15.47:0/791821010 conn(0x7faa440fbef0 msgr2=0x7faa440fe310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:29:42.107 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.107+0000 7faa49800640 1 -- 172.21.15.47:0/791821010 shutdown_connections 2024-03-21T10:29:42.107 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.107+0000 7faa49800640 1 -- 172.21.15.47:0/791821010 wait complete. 2024-03-21T10:29:42.109 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.108+0000 7faa49800640 1 Processor -- start 2024-03-21T10:29:42.109 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.108+0000 7faa49800640 1 -- start start 2024-03-21T10:29:42.109 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.109+0000 7faa49800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7faa441026d0 0x7faa44199ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:29:42.109 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.109+0000 7faa49800640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7faa44102ff0 0x7faa4419a3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:29:42.109 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.109+0000 7faa49800640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7faa4410baf0 0x7faa4419e760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:29:42.109 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.109+0000 7faa49800640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa441117b0 con 0x7faa44102ff0 2024-03-21T10:29:42.109 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.109+0000 7faa49800640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa44111630 con 0x7faa441026d0 2024-03-21T10:29:42.109 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.109+0000 7faa49800640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa44111930 con 0x7faa4410baf0 2024-03-21T10:29:42.109 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.109+0000 7faa3be00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7faa44102ff0 0x7faa4419a3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:29:42.109 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.109+0000 7faa3be00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7faa44102ff0 0x7faa4419a3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:55046/0 (socket says 172.21.15.47:55046) 2024-03-21T10:29:42.109 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.109+0000 7faa3be00640 1 -- 172.21.15.47:0/1296693612 learned_addr learned my addr 172.21.15.47:0/1296693612 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:29:42.110 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.109+0000 7faa43e00640 1 --2- 172.21.15.47:0/1296693612 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7faa441026d0 0x7faa44199ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:29:42.110 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.109+0000 7faa48e00640 1 --2- 172.21.15.47:0/1296693612 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7faa4410baf0 0x7faa4419e760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:29:42.110 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.109+0000 7faa3be00640 1 -- 172.21.15.47:0/1296693612 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7faa4410baf0 msgr2=0x7faa4419e760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:29:42.110 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.109+0000 7faa3be00640 1 --2- 172.21.15.47:0/1296693612 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7faa4410baf0 0x7faa4419e760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:29:42.110 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.110+0000 7faa3be00640 1 -- 172.21.15.47:0/1296693612 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7faa441026d0 msgr2=0x7faa44199ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:29:42.110 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.110+0000 7faa3be00640 1 --2- 172.21.15.47:0/1296693612 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7faa441026d0 0x7faa44199ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:29:42.110 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.110+0000 7faa3be00640 1 -- 172.21.15.47:0/1296693612 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa4419ee90 con 0x7faa44102ff0 2024-03-21T10:29:42.110 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.110+0000 7faa3be00640 1 --2- 172.21.15.47:0/1296693612 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7faa44102ff0 0x7faa4419a3e0 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x7faa2c000c00 tx=0x7faa2c02c6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:29:42.111 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.110+0000 7faa41600640 1 -- 172.21.15.47:0/1296693612 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa2c043020 con 0x7faa44102ff0 2024-03-21T10:29:42.111 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.110+0000 7faa41600640 1 -- 172.21.15.47:0/1296693612 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7faa2c004e00 con 0x7faa44102ff0 2024-03-21T10:29:42.111 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.110+0000 7faa49800640 1 -- 172.21.15.47:0/1296693612 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faa4419f120 con 0x7faa44102ff0 2024-03-21T10:29:42.111 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.110+0000 7faa41600640 1 -- 172.21.15.47:0/1296693612 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa2c0038e0 con 0x7faa44102ff0 2024-03-21T10:29:42.112 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.110+0000 7faa49800640 1 -- 172.21.15.47:0/1296693612 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faa441a6a10 con 0x7faa44102ff0 2024-03-21T10:29:42.113 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.114+0000 7faa4dba6740 1 Processor -- start 2024-03-21T10:29:42.113 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.114+0000 7faa41600640 1 -- 172.21.15.47:0/1296693612 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 27) v1 ==== 137985+0+0 (secure 0 0 0) 0x7faa2c046020 con 0x7faa44102ff0 2024-03-21T10:29:42.113 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.114+0000 7faa4dba6740 1 -- start start 2024-03-21T10:29:42.115 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.114+0000 7faa4dba6740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561412c7d8b0 0x561412c7dc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:29:42.115 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.114+0000 7faa4dba6740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561412c7e260 0x561412c86810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:29:42.115 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.114+0000 7faa4dba6740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561412c86de0 0x561412c89230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:29:42.115 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.114+0000 7faa4dba6740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x561412be12c0 con 0x561412c7e260 2024-03-21T10:29:42.115 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.114+0000 7faa4dba6740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x561412b89c90 con 0x561412c86de0 2024-03-21T10:29:42.115 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.114+0000 7faa4dba6740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x561412bdd360 con 0x561412c7d8b0 2024-03-21T10:29:42.115 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.115+0000 7faa43e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561412c7e260 0x561412c86810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:29:42.115 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.115+0000 7faa43e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561412c7e260 0x561412c86810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:55056/0 (socket says 172.21.15.47:55056) 2024-03-21T10:29:42.115 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.115+0000 7faa43e00640 1 -- 172.21.15.47:0/1579378726 learned_addr learned my addr 172.21.15.47:0/1579378726 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:29:42.115 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.115+0000 7faa48e00640 1 --2- 172.21.15.47:0/1579378726 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561412c7d8b0 0x561412c7dc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:29:42.116 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.115+0000 7faa3be00640 1 --2- 172.21.15.47:0/1579378726 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561412c86de0 0x561412c89230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:29:42.116 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.115+0000 7faa43e00640 1 -- 172.21.15.47:0/1579378726 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561412c7d8b0 msgr2=0x561412c7dc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:29:42.116 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.115+0000 7faa43e00640 1 --2- 172.21.15.47:0/1579378726 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561412c7d8b0 0x561412c7dc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:29:42.116 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.115+0000 7faa43e00640 1 -- 172.21.15.47:0/1579378726 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561412c86de0 msgr2=0x561412c89230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:29:42.116 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.115+0000 7faa43e00640 1 --2- 172.21.15.47:0/1579378726 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561412c86de0 0x561412c89230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:29:42.116 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.115+0000 7faa43e00640 1 -- 172.21.15.47:0/1579378726 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561412bae080 con 0x561412c7e260 2024-03-21T10:29:42.116 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.116+0000 7faa43e00640 1 --2- 172.21.15.47:0/1579378726 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561412c7e260 0x561412c86810 secure :-1 s=READY pgs=813 cs=0 l=1 rev1=1 crypto rx=0x7faa3400ad00 tx=0x7faa3400f790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=11eb8c4cb6e207a5 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:29:42.116 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.116+0000 7faa41600640 1 --2- 172.21.15.47:0/1296693612 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7faa280a3090 0x7faa280a5550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:29:42.120 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.116+0000 7faa41600640 1 -- 172.21.15.47:0/1296693612 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 9474+0+0 (secure 0 0 0) 0x7faa2c0f1a50 con 0x7faa44102ff0 2024-03-21T10:29:42.120 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.116+0000 7faa39600640 1 -- 172.21.15.47:0/1579378726 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa3401d070 con 0x561412c7e260 2024-03-21T10:29:42.120 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.116+0000 7faa39600640 1 -- 172.21.15.47:0/1579378726 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7faa34018af0 con 0x561412c7e260 2024-03-21T10:29:42.120 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.116+0000 7faa48e00640 1 --2- 172.21.15.47:0/1296693612 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7faa280a3090 0x7faa280a5550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:29:42.120 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.116+0000 7faa39600640 1 -- 172.21.15.47:0/1579378726 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa34010b40 con 0x561412c7e260 2024-03-21T10:29:42.120 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.117+0000 7faa4dba6740 1 -- 172.21.15.47:0/1579378726 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561412c7e260 msgr2=0x561412c86810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:29:42.121 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.117+0000 7faa4dba6740 1 --2- 172.21.15.47:0/1579378726 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561412c7e260 0x561412c86810 secure :-1 s=READY pgs=813 cs=0 l=1 rev1=1 crypto rx=0x7faa3400ad00 tx=0x7faa3400f790 comp rx=0 tx=0).stop 2024-03-21T10:29:42.121 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.117+0000 7faa48e00640 1 --2- 172.21.15.47:0/1296693612 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7faa280a3090 0x7faa280a5550 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7faa3c005fd0 tx=0x7faa3c009500 comp rx=0 tx=0).ready entity=mgr.15687 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:29:42.121 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.117+0000 7faa4dba6740 1 -- 172.21.15.47:0/1579378726 shutdown_connections 2024-03-21T10:29:42.121 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.117+0000 7faa4dba6740 1 --2- 172.21.15.47:0/1579378726 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561412c86de0 0x561412c89230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:29:42.121 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.117+0000 7faa4dba6740 1 --2- 172.21.15.47:0/1579378726 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561412c7e260 0x561412c86810 unknown :-1 s=CLOSED pgs=813 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:29:42.121 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.117+0000 7faa4dba6740 1 --2- 172.21.15.47:0/1579378726 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561412c7d8b0 0x561412c7dc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:29:42.121 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.117+0000 7faa4dba6740 1 -- 172.21.15.47:0/1579378726 >> 172.21.15.47:0/1579378726 conn(0x561412c77f80 msgr2=0x561412c78360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:29:42.121 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.117+0000 7faa4dba6740 1 -- 172.21.15.47:0/1579378726 shutdown_connections 2024-03-21T10:29:42.121 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.117+0000 7faa4dba6740 1 -- 172.21.15.47:0/1579378726 wait complete. 2024-03-21T10:29:42.121 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.118+0000 7faa4dba6740 1 -- start start 2024-03-21T10:29:42.121 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.118+0000 7faa4dba6740 1 Processor -- start 2024-03-21T10:29:42.121 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.118+0000 7faa4dba6740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561412c7d8b0 0x561412c866b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:29:42.122 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.118+0000 7faa4dba6740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561412c7e260 0x561412c83a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:29:42.122 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.118+0000 7faa4dba6740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561412c86de0 0x561412c83fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:29:42.122 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.118+0000 7faa4dba6740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x561412c89d50 con 0x561412c86de0 2024-03-21T10:29:42.122 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.118+0000 7faa4dba6740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x561412b9d0d0 con 0x561412c7d8b0 2024-03-21T10:29:42.122 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.118+0000 7faa4dba6740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x561412c89ed0 con 0x561412c7e260 2024-03-21T10:29:42.122 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.118+0000 7faa4dba6740 10 client.0 _ll_register_callbacks cb 0x561412ba1e80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T10:29:42.122 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.119+0000 7faa48e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561412c86de0 0x561412c83fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:29:42.122 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.119+0000 7faa48e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561412c86de0 0x561412c83fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:55062/0 (socket says 172.21.15.47:55062) 2024-03-21T10:29:42.122 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.119+0000 7faa48e00640 1 -- 172.21.15.47:0/1035018263 learned_addr learned my addr 172.21.15.47:0/1035018263 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:29:42.123 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.119+0000 7faa43e00640 1 --2- 172.21.15.47:0/1035018263 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561412c7d8b0 0x561412c866b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:29:42.123 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.119+0000 7faa3be00640 1 --2- 172.21.15.47:0/1035018263 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561412c7e260 0x561412c83a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:29:42.123 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.119+0000 7faa48e00640 1 -- 172.21.15.47:0/1035018263 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561412c7e260 msgr2=0x561412c83a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:29:42.123 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.119+0000 7faa48e00640 1 --2- 172.21.15.47:0/1035018263 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561412c7e260 0x561412c83a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:29:42.123 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.119+0000 7faa48e00640 1 -- 172.21.15.47:0/1035018263 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561412c7d8b0 msgr2=0x561412c866b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:29:42.123 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.119+0000 7faa48e00640 1 --2- 172.21.15.47:0/1035018263 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561412c7d8b0 0x561412c866b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:29:42.123 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.119+0000 7faa48e00640 1 -- 172.21.15.47:0/1035018263 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5614128f53f0 con 0x561412c86de0 2024-03-21T10:29:42.123 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.119+0000 7faa3be00640 1 --2- 172.21.15.47:0/1035018263 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561412c7e260 0x561412c83a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T10:29:42.123 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.119+0000 7faa43e00640 1 --2- 172.21.15.47:0/1035018263 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561412c7d8b0 0x561412c866b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T10:29:42.123 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.120+0000 7faa48e00640 1 --2- 172.21.15.47:0/1035018263 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561412c86de0 0x561412c83fa0 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7faa3c012830 tx=0x7faa3c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b1a2c293a2a07c89 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:29:42.123 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.120+0000 7faa32000640 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-03-21T10:29:42.123 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.120+0000 7faa32000640 1 -- 172.21.15.47:0/1035018263 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa3c010be0 con 0x561412c86de0 2024-03-21T10:29:42.123 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.120+0000 7faa32000640 1 -- 172.21.15.47:0/1035018263 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(97..97 src has 1..97) v4 ==== 9474+0+0 (secure 0 0 0) 0x7faa3c010d80 con 0x561412c86de0 2024-03-21T10:29:42.123 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.120+0000 7faa4dba6740 1 -- 172.21.15.47:0/1035018263 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7faa34018af0 con 0x561412c86de0 2024-03-21T10:29:42.124 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.120+0000 7faa32000640 1 -- 172.21.15.47:0/1035018263 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7faa3c01b460 con 0x561412c86de0 2024-03-21T10:29:42.124 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.120+0000 7faa4dba6740 10 client.17022 fetch_fsmap learned FSMap version 32 2024-03-21T10:29:42.124 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.120+0000 7faa4dba6740 1 -- 172.21.15.47:0/1035018263 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x5614129bf7d0 con 0x561412c86de0 2024-03-21T10:29:42.124 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.121+0000 7faa32000640 1 -- 172.21.15.47:0/1035018263 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7faa34018af0 con 0x561412c86de0 2024-03-21T10:29:42.124 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.121+0000 7faa32000640 1 -- 172.21.15.47:0/1035018263 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 32) v1 ==== 7942+0+0 (secure 0 0 0) 0x7faa3c024070 con 0x561412c86de0 2024-03-21T10:29:42.124 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.121+0000 7faa4dba6740 10 client.17022 fetch_fsmap finished waiting for FSMap version 32 2024-03-21T10:29:42.124 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.121+0000 7faa4dba6740 10 client.17022 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.k 2024-03-21T10:29:42.124 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.121+0000 7faa4dba6740 20 client.17022 populate_metadata read hostname 'smithi047' 2024-03-21T10:29:42.124 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.121+0000 7faa4dba6740 1 --2- 172.21.15.47:0/1035018263 >> [v2:172.21.15.100:6832/2723302038,v1:172.21.15.100:6833/2723302038] conn(0x561412c96ce0 0x561412c99160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:29:42.124 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.121+0000 7faa4dba6740 4 client.17022 mds_command: new command op to 34427 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T10:29:42.124 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.121+0000 7faa4dba6740 1 -- 172.21.15.47:0/1035018263 --> [v2:172.21.15.100:6832/2723302038,v1:172.21.15.100:6833/2723302038] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x561412bee800 con 0x561412c96ce0 2024-03-21T10:29:42.126 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.122+0000 7faa43e00640 1 --2- 172.21.15.47:0/1035018263 >> [v2:172.21.15.100:6832/2723302038,v1:172.21.15.100:6833/2723302038] conn(0x561412c96ce0 0x561412c99160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:29:42.126 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.122+0000 7faa43e00640 1 --2- 172.21.15.47:0/1035018263 >> [v2:172.21.15.100:6832/2723302038,v1:172.21.15.100:6833/2723302038] conn(0x561412c96ce0 0x561412c99160 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=6b83097121b74430 server_cookie=f4558c36677e3804 in_seq=0 out_seq=0 2024-03-21T10:29:42.126 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.122+0000 7faa32000640 10 client.17022 ms_handle_connect on v2:172.21.15.100:6832/2723302038 2024-03-21T10:29:42.126 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.123+0000 7faa32000640 1 -- 172.21.15.47:0/1035018263 <== mds.3 v2:172.21.15.100:6832/2723302038 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x561412bee800 con 0x561412c96ce0 2024-03-21T10:29:42.126 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.123+0000 7faa32000640 10 client.17022 handle_command_reply: tid=0 2024-03-21T10:29:42.126 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.123+0000 7faa4dba6740 1 client.17022 shutdown 2024-03-21T10:29:42.126 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.124+0000 7faa4dba6740 1 -- 172.21.15.47:0/1035018263 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561412c86de0 msgr2=0x561412c83fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:29:42.126 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.124+0000 7faa4dba6740 1 --2- 172.21.15.47:0/1035018263 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561412c86de0 0x561412c83fa0 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7faa3c012830 tx=0x7faa3c004a40 comp rx=0 tx=0).stop 2024-03-21T10:29:42.126 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.124+0000 7faa4dba6740 1 -- 172.21.15.47:0/1035018263 shutdown_connections 2024-03-21T10:29:42.126 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.124+0000 7faa4dba6740 1 --2- 172.21.15.47:0/1035018263 >> [v2:172.21.15.100:6832/2723302038,v1:172.21.15.100:6833/2723302038] conn(0x561412c96ce0 0x561412c99160 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:29:42.126 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.124+0000 7faa4dba6740 1 --2- 172.21.15.47:0/1035018263 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561412c86de0 0x561412c83fa0 unknown :-1 s=CLOSED pgs=814 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:29:42.126 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.124+0000 7faa4dba6740 1 --2- 172.21.15.47:0/1035018263 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561412c7e260 0x561412c83a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:29:42.126 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.124+0000 7faa4dba6740 1 --2- 172.21.15.47:0/1035018263 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561412c7d8b0 0x561412c866b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:29:42.127 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.124+0000 7faa32000640 0 client.17022 ms_handle_reset on v2:172.21.15.100:6832/2723302038 2024-03-21T10:29:42.127 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.124+0000 7faa4dba6740 1 -- 172.21.15.47:0/1035018263 >> 172.21.15.47:0/1035018263 conn(0x561412c77f80 msgr2=0x561412c79cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:29:42.127 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.124+0000 7faa4dba6740 1 -- 172.21.15.47:0/1035018263 shutdown_connections 2024-03-21T10:29:42.127 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.124+0000 7faa4dba6740 1 -- 172.21.15.47:0/1035018263 wait complete. 2024-03-21T10:29:42.127 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.124+0000 7faa4dba6740 20 client.17022 trim_cache size 0 max 16384 2024-03-21T10:29:42.139 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.138+0000 7faa4dba6740 1 Processor -- start 2024-03-21T10:29:42.139 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.138+0000 7faa4dba6740 1 -- start start 2024-03-21T10:29:42.139 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.138+0000 7faa4dba6740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561412c7d8b0 0x561412c76fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:29:42.139 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.138+0000 7faa4dba6740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561412c7e260 0x561412c86170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:29:42.139 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.138+0000 7faa4dba6740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561412c86de0 0x561412c866b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:29:42.139 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.138+0000 7faa4dba6740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa441b08c0 con 0x561412c7d8b0 2024-03-21T10:29:42.139 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.138+0000 7faa4dba6740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa441b2240 con 0x561412c86de0 2024-03-21T10:29:42.139 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.138+0000 7faa4dba6740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa441b0f20 con 0x561412c7e260 2024-03-21T10:29:42.139 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.138+0000 7faa43e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561412c7d8b0 0x561412c76fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:29:42.139 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.139+0000 7faa43e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561412c7d8b0 0x561412c76fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:55072/0 (socket says 172.21.15.47:55072) 2024-03-21T10:29:42.139 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.139+0000 7faa43e00640 1 -- 172.21.15.47:0/3394415730 learned_addr learned my addr 172.21.15.47:0/3394415730 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:29:42.140 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.139+0000 7faa48e00640 1 --2- 172.21.15.47:0/3394415730 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561412c86de0 0x561412c866b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:29:42.140 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.139+0000 7faa43e00640 1 -- 172.21.15.47:0/3394415730 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561412c7e260 msgr2=0x561412c86170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:29:42.140 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.139+0000 7faa3be00640 1 --2- 172.21.15.47:0/3394415730 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561412c7e260 0x561412c86170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:29:42.140 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.139+0000 7faa43e00640 1 --2- 172.21.15.47:0/3394415730 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561412c7e260 0x561412c86170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:29:42.140 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.139+0000 7faa43e00640 1 -- 172.21.15.47:0/3394415730 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561412c86de0 msgr2=0x561412c866b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:29:42.140 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.139+0000 7faa43e00640 1 --2- 172.21.15.47:0/3394415730 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561412c86de0 0x561412c866b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:29:42.140 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.139+0000 7faa43e00640 1 -- 172.21.15.47:0/3394415730 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561412c7e000 con 0x561412c7d8b0 2024-03-21T10:29:42.140 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.139+0000 7faa48e00640 1 --2- 172.21.15.47:0/3394415730 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561412c86de0 0x561412c866b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T10:29:42.140 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.139+0000 7faa43e00640 1 --2- 172.21.15.47:0/3394415730 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561412c7d8b0 0x561412c76fe0 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x7faa34010770 tx=0x7faa3400a360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=51a65b43bacb8802 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:29:42.140 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.140+0000 7faa38c00640 1 -- 172.21.15.47:0/3394415730 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa3401d040 con 0x561412c7d8b0 2024-03-21T10:29:42.140 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.140+0000 7faa38c00640 1 -- 172.21.15.47:0/3394415730 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7faa3401e040 con 0x561412c7d8b0 2024-03-21T10:29:42.140 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.140+0000 7faa38c00640 1 -- 172.21.15.47:0/3394415730 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa340052a0 con 0x561412c7d8b0 2024-03-21T10:29:42.144 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.140+0000 7faa4dba6740 1 -- 172.21.15.47:0/3394415730 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561412c7d8b0 msgr2=0x561412c76fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:29:42.144 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.140+0000 7faa4dba6740 1 --2- 172.21.15.47:0/3394415730 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561412c7d8b0 0x561412c76fe0 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x7faa34010770 tx=0x7faa3400a360 comp rx=0 tx=0).stop 2024-03-21T10:29:42.144 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.140+0000 7faa4dba6740 1 -- 172.21.15.47:0/3394415730 shutdown_connections 2024-03-21T10:29:42.144 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.140+0000 7faa4dba6740 1 --2- 172.21.15.47:0/3394415730 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561412c86de0 0x561412c866b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:29:42.144 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.140+0000 7faa4dba6740 1 --2- 172.21.15.47:0/3394415730 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561412c7e260 0x561412c86170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:29:42.144 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.140+0000 7faa4dba6740 1 --2- 172.21.15.47:0/3394415730 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561412c7d8b0 0x561412c76fe0 unknown :-1 s=CLOSED pgs=815 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:29:42.144 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.140+0000 7faa4dba6740 1 -- 172.21.15.47:0/3394415730 >> 172.21.15.47:0/3394415730 conn(0x561412c77f80 msgr2=0x561412c768d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:29:42.144 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.141+0000 7faa4dba6740 1 -- 172.21.15.47:0/3394415730 shutdown_connections 2024-03-21T10:29:42.144 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.141+0000 7faa4dba6740 1 -- 172.21.15.47:0/3394415730 wait complete. 2024-03-21T10:29:42.144 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.141+0000 7faa4dba6740 1 -- start start 2024-03-21T10:29:42.144 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.141+0000 7faa4dba6740 1 Processor -- start 2024-03-21T10:29:42.144 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.142+0000 7faa4dba6740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561412c7e260 0x561412c8cd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:29:42.144 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.142+0000 7faa4dba6740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561412c86de0 0x561412c8d260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:29:42.145 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.142+0000 7faa4dba6740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561412ceb610 0x561412ceba90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:29:42.145 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.142+0000 7faa4dba6740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa441b1be0 con 0x561412c7e260 2024-03-21T10:29:42.145 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.142+0000 7faa4dba6740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa441b1580 con 0x561412ceb610 2024-03-21T10:29:42.145 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.142+0000 7faa4dba6740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa441aea80 con 0x561412c86de0 2024-03-21T10:29:42.145 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.142+0000 7faa4dba6740 10 client.0 _ll_register_callbacks cb 0x7faa3c009ca0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T10:29:42.145 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.142+0000 7faa43e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561412c7e260 0x561412c8cd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:29:42.145 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.142+0000 7faa43e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561412c7e260 0x561412c8cd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:55074/0 (socket says 172.21.15.47:55074) 2024-03-21T10:29:42.145 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.142+0000 7faa43e00640 1 -- 172.21.15.47:0/2712862513 learned_addr learned my addr 172.21.15.47:0/2712862513 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:29:42.145 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.142+0000 7faa48e00640 1 --2- 172.21.15.47:0/2712862513 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561412ceb610 0x561412ceba90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:29:42.145 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.142+0000 7faa3be00640 1 --2- 172.21.15.47:0/2712862513 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561412c86de0 0x561412c8d260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:29:42.145 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.142+0000 7faa43e00640 1 -- 172.21.15.47:0/2712862513 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561412c86de0 msgr2=0x561412c8d260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:29:42.145 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.142+0000 7faa43e00640 1 --2- 172.21.15.47:0/2712862513 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561412c86de0 0x561412c8d260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:29:42.145 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.142+0000 7faa43e00640 1 -- 172.21.15.47:0/2712862513 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561412ceb610 msgr2=0x561412ceba90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:29:42.146 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.142+0000 7faa43e00640 1 --2- 172.21.15.47:0/2712862513 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561412ceb610 0x561412ceba90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:29:42.146 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.142+0000 7faa43e00640 1 -- 172.21.15.47:0/2712862513 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x561412cebfd0 con 0x561412c7e260 2024-03-21T10:29:42.146 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.142+0000 7faa3be00640 1 --2- 172.21.15.47:0/2712862513 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561412c86de0 0x561412c8d260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T10:29:42.146 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.143+0000 7faa43e00640 1 --2- 172.21.15.47:0/2712862513 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561412c7e260 0x561412c8cd20 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 crypto rx=0x7faa3401b040 tx=0x7faa340292e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9135adfdf18fa5ea server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:29:42.146 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.143+0000 7faa32000640 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-03-21T10:29:42.146 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.143+0000 7faa32000640 1 -- 172.21.15.47:0/2712862513 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa34037060 con 0x561412c7e260 2024-03-21T10:29:42.146 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.143+0000 7faa4dba6740 1 -- 172.21.15.47:0/2712862513 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7faa3401e040 con 0x561412c7e260 2024-03-21T10:29:42.146 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.143+0000 7faa32000640 1 -- 172.21.15.47:0/2712862513 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(97..97 src has 1..97) v4 ==== 9474+0+0 (secure 0 0 0) 0x7faa340298a0 con 0x561412c7e260 2024-03-21T10:29:42.146 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.143+0000 7faa32000640 1 -- 172.21.15.47:0/2712862513 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7faa3401d070 con 0x561412c7e260 2024-03-21T10:29:42.146 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.144+0000 7faa4dba6740 10 client.17028 fetch_fsmap learned FSMap version 32 2024-03-21T10:29:42.146 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.144+0000 7faa4dba6740 1 -- 172.21.15.47:0/2712862513 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x561412cf3350 con 0x561412c7e260 2024-03-21T10:29:42.146 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.144+0000 7faa32000640 1 -- 172.21.15.47:0/2712862513 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7faa3401e040 con 0x561412c7e260 2024-03-21T10:29:42.146 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.144+0000 7faa32000640 1 -- 172.21.15.47:0/2712862513 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 32) v1 ==== 7942+0+0 (secure 0 0 0) 0x7faa340184c0 con 0x561412c7e260 2024-03-21T10:29:42.147 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.144+0000 7faa4dba6740 10 client.17028 fetch_fsmap finished waiting for FSMap version 32 2024-03-21T10:29:42.147 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.144+0000 7faa4dba6740 10 client.17028 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.k 2024-03-21T10:29:42.147 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.144+0000 7faa4dba6740 20 client.17028 populate_metadata read hostname 'smithi047' 2024-03-21T10:29:42.147 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.144+0000 7faa4dba6740 1 --2- 172.21.15.47:0/2712862513 >> [v2:172.21.15.100:6832/2723302038,v1:172.21.15.100:6833/2723302038] conn(0x561412cf3770 0x561412cf5b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:29:42.147 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.144+0000 7faa4dba6740 4 client.17028 mds_command: new command op to 34427 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-21T10:29:42.147 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.144+0000 7faa4dba6740 1 -- 172.21.15.47:0/2712862513 --> [v2:172.21.15.100:6832/2723302038,v1:172.21.15.100:6833/2723302038] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x561412be8b50 con 0x561412cf3770 2024-03-21T10:29:42.147 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.145+0000 7faa3be00640 1 --2- 172.21.15.47:0/2712862513 >> [v2:172.21.15.100:6832/2723302038,v1:172.21.15.100:6833/2723302038] conn(0x561412cf3770 0x561412cf5b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:29:42.148 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.145+0000 7faa3be00640 1 --2- 172.21.15.47:0/2712862513 >> [v2:172.21.15.100:6832/2723302038,v1:172.21.15.100:6833/2723302038] conn(0x561412cf3770 0x561412cf5b60 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=7517e3c39276a15d server_cookie=e3ef44955348a03 in_seq=0 out_seq=0 2024-03-21T10:29:42.148 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.145+0000 7faa32000640 10 client.17028 ms_handle_connect on v2:172.21.15.100:6832/2723302038 2024-03-21T10:29:42.148 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.146+0000 7faa32000640 1 -- 172.21.15.47:0/2712862513 <== mds.3 v2:172.21.15.100:6832/2723302038 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x561412be8b50 con 0x561412cf3770 2024-03-21T10:29:42.148 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.146+0000 7faa32000640 10 client.17028 handle_command_reply: tid=0 2024-03-21T10:29:42.148 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.146+0000 7faa4dba6740 1 client.17028 shutdown 2024-03-21T10:29:42.148 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.146+0000 7faa4dba6740 1 -- 172.21.15.47:0/2712862513 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561412c7e260 msgr2=0x561412c8cd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:29:42.148 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.146+0000 7faa4dba6740 1 --2- 172.21.15.47:0/2712862513 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561412c7e260 0x561412c8cd20 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 crypto rx=0x7faa3401b040 tx=0x7faa340292e0 comp rx=0 tx=0).stop 2024-03-21T10:29:42.148 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.146+0000 7faa4dba6740 1 -- 172.21.15.47:0/2712862513 shutdown_connections 2024-03-21T10:29:42.149 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.146+0000 7faa4dba6740 1 --2- 172.21.15.47:0/2712862513 >> [v2:172.21.15.100:6832/2723302038,v1:172.21.15.100:6833/2723302038] conn(0x561412cf3770 0x561412cf5b60 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:29:42.149 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.146+0000 7faa4dba6740 1 --2- 172.21.15.47:0/2712862513 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561412ceb610 0x561412ceba90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:29:42.149 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.146+0000 7faa4dba6740 1 --2- 172.21.15.47:0/2712862513 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561412c86de0 0x561412c8d260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:29:42.149 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.146+0000 7faa4dba6740 1 --2- 172.21.15.47:0/2712862513 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561412c7e260 0x561412c8cd20 unknown :-1 s=CLOSED pgs=816 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:29:42.149 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.146+0000 7faa32000640 0 client.17028 ms_handle_reset on v2:172.21.15.100:6832/2723302038 2024-03-21T10:29:42.149 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.146+0000 7faa4dba6740 1 -- 172.21.15.47:0/2712862513 >> 172.21.15.47:0/2712862513 conn(0x561412c77f80 msgr2=0x561412c79c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:29:42.149 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.146+0000 7faa4dba6740 1 -- 172.21.15.47:0/2712862513 shutdown_connections 2024-03-21T10:29:42.149 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.146+0000 7faa4dba6740 1 -- 172.21.15.47:0/2712862513 wait complete. 2024-03-21T10:29:42.149 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.146+0000 7faa4dba6740 20 client.17028 trim_cache size 0 max 16384 2024-03-21T10:29:42.149 INFO:teuthology.orchestra.run.smithi047.stdout:[] 2024-03-21T10:29:42.150 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.147+0000 7faa49800640 1 -- 172.21.15.47:0/1296693612 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7faa280a3090 msgr2=0x7faa280a5550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:29:42.150 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.147+0000 7faa49800640 1 --2- 172.21.15.47:0/1296693612 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7faa280a3090 0x7faa280a5550 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7faa3c005fd0 tx=0x7faa3c009500 comp rx=0 tx=0).stop 2024-03-21T10:29:42.150 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.147+0000 7faa49800640 1 -- 172.21.15.47:0/1296693612 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7faa44102ff0 msgr2=0x7faa4419a3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:29:42.150 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.147+0000 7faa49800640 1 --2- 172.21.15.47:0/1296693612 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7faa44102ff0 0x7faa4419a3e0 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x7faa2c000c00 tx=0x7faa2c02c6c0 comp rx=0 tx=0).stop 2024-03-21T10:29:42.150 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.147+0000 7faa49800640 1 -- 172.21.15.47:0/1296693612 shutdown_connections 2024-03-21T10:29:42.151 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.147+0000 7faa49800640 1 --2- 172.21.15.47:0/1296693612 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7faa4410baf0 0x7faa4419e760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:29:42.151 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.147+0000 7faa49800640 1 --2- 172.21.15.47:0/1296693612 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7faa44102ff0 0x7faa4419a3e0 unknown :-1 s=CLOSED pgs=812 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:29:42.151 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.147+0000 7faa49800640 1 --2- 172.21.15.47:0/1296693612 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7faa280a3090 0x7faa280a5550 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:29:42.151 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.147+0000 7faa49800640 1 --2- 172.21.15.47:0/1296693612 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7faa441026d0 0x7faa44199ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:29:42.151 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.147+0000 7faa49800640 1 -- 172.21.15.47:0/1296693612 >> 172.21.15.47:0/1296693612 conn(0x7faa440fbef0 msgr2=0x7faa440fe2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:29:42.151 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.147+0000 7faa49800640 1 -- 172.21.15.47:0/1296693612 shutdown_connections 2024-03-21T10:29:42.151 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.147+0000 7faa49800640 1 -- 172.21.15.47:0/1296693612 wait complete. 2024-03-21T10:29:42.165 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-21T10:29:42.165 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-03-21T10:29:42.384 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.384+0000 7fa87d400640 1 -- 172.21.15.47:0/162430320 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa8781026d0 msgr2=0x7fa878102ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:29:42.384 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.384+0000 7fa87d400640 1 --2- 172.21.15.47:0/162430320 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa8781026d0 0x7fa878102ab0 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 crypto rx=0x7fa868009900 tx=0x7fa86802bcc0 comp rx=0 tx=0).stop 2024-03-21T10:29:42.384 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.384+0000 7fa87d400640 1 -- 172.21.15.47:0/162430320 shutdown_connections 2024-03-21T10:29:42.384 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.384+0000 7fa87d400640 1 --2- 172.21.15.47:0/162430320 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa87810baf0 0x7fa87810df90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:29:42.384 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.384+0000 7fa87d400640 1 --2- 172.21.15.47:0/162430320 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa878102ff0 0x7fa87810b4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:29:42.384 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.384+0000 7fa87d400640 1 --2- 172.21.15.47:0/162430320 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa8781026d0 0x7fa878102ab0 unknown :-1 s=CLOSED pgs=817 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:29:42.385 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.385+0000 7fa87d400640 1 -- 172.21.15.47:0/162430320 >> 172.21.15.47:0/162430320 conn(0x7fa8780fbef0 msgr2=0x7fa8780fe310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:29:42.385 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.385+0000 7fa87d400640 1 -- 172.21.15.47:0/162430320 shutdown_connections 2024-03-21T10:29:42.385 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.385+0000 7fa87d400640 1 -- 172.21.15.47:0/162430320 wait complete. 2024-03-21T10:29:42.385 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.386+0000 7fa87d400640 1 Processor -- start 2024-03-21T10:29:42.385 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.386+0000 7fa87d400640 1 -- start start 2024-03-21T10:29:42.387 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.387+0000 7fa87d400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa8781026d0 0x7fa878197d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:29:42.387 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.387+0000 7fa87d400640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa878102ff0 0x7fa878198280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:29:42.387 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.387+0000 7fa87d400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa87810baf0 0x7fa87819c600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:29:42.387 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.387+0000 7fa87d400640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa878111780 con 0x7fa8781026d0 2024-03-21T10:29:42.387 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.387+0000 7fa87d400640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa878111600 con 0x7fa87810baf0 2024-03-21T10:29:42.387 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.387+0000 7fa87d400640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa878111900 con 0x7fa878102ff0 2024-03-21T10:29:42.387 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.387+0000 7fa877e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa8781026d0 0x7fa878197d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:29:42.387 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.387+0000 7fa877e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa8781026d0 0x7fa878197d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:55096/0 (socket says 172.21.15.47:55096) 2024-03-21T10:29:42.387 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.387+0000 7fa877e00640 1 -- 172.21.15.47:0/3793812704 learned_addr learned my addr 172.21.15.47:0/3793812704 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:29:42.387 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.387+0000 7fa87ca00640 1 --2- 172.21.15.47:0/3793812704 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa87810baf0 0x7fa87819c600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:29:42.388 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.387+0000 7fa877400640 1 --2- 172.21.15.47:0/3793812704 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa878102ff0 0x7fa878198280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:29:42.388 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.387+0000 7fa877e00640 1 -- 172.21.15.47:0/3793812704 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa878102ff0 msgr2=0x7fa878198280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:29:42.388 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.387+0000 7fa877e00640 1 --2- 172.21.15.47:0/3793812704 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa878102ff0 0x7fa878198280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:29:42.388 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.388+0000 7fa877e00640 1 -- 172.21.15.47:0/3793812704 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa87810baf0 msgr2=0x7fa87819c600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:29:42.388 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.388+0000 7fa877e00640 1 --2- 172.21.15.47:0/3793812704 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa87810baf0 0x7fa87819c600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:29:42.388 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.388+0000 7fa877e00640 1 -- 172.21.15.47:0/3793812704 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa87819cd30 con 0x7fa8781026d0 2024-03-21T10:29:42.388 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.388+0000 7fa87ca00640 1 --2- 172.21.15.47:0/3793812704 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa87810baf0 0x7fa87819c600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T10:29:42.388 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.388+0000 7fa877400640 1 --2- 172.21.15.47:0/3793812704 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa878102ff0 0x7fa878198280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T10:29:42.388 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.388+0000 7fa877e00640 1 --2- 172.21.15.47:0/3793812704 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa8781026d0 0x7fa878197d40 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 crypto rx=0x7fa868005940 tx=0x7fa86802d880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:29:42.389 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.388+0000 7fa874c00640 1 -- 172.21.15.47:0/3793812704 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa868043020 con 0x7fa8781026d0 2024-03-21T10:29:42.389 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.388+0000 7fa87d400640 1 -- 172.21.15.47:0/3793812704 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa87819cfc0 con 0x7fa8781026d0 2024-03-21T10:29:42.389 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.388+0000 7fa874c00640 1 -- 172.21.15.47:0/3793812704 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fa86802dc80 con 0x7fa8781026d0 2024-03-21T10:29:42.389 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.388+0000 7fa874c00640 1 -- 172.21.15.47:0/3793812704 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa868003ab0 con 0x7fa8781026d0 2024-03-21T10:29:42.389 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.388+0000 7fa87d400640 1 -- 172.21.15.47:0/3793812704 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa8781a48b0 con 0x7fa8781026d0 2024-03-21T10:29:42.390 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.392+0000 7fa881647740 1 Processor -- start 2024-03-21T10:29:42.393 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.392+0000 7fa881647740 1 -- start start 2024-03-21T10:29:42.393 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.392+0000 7fa881647740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5608a419f8b0 0x5608a419fc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:29:42.393 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.392+0000 7fa881647740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5608a41a0260 0x5608a41a8810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:29:42.393 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.392+0000 7fa881647740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5608a41a8de0 0x5608a41ab230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:29:42.393 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.392+0000 7fa881647740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x5608a41032c0 con 0x5608a419f8b0 2024-03-21T10:29:42.393 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.392+0000 7fa881647740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x5608a40abc90 con 0x5608a41a8de0 2024-03-21T10:29:42.393 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.392+0000 7fa881647740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x5608a40ff360 con 0x5608a41a0260 2024-03-21T10:29:42.393 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.392+0000 7fa87ca00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5608a419f8b0 0x5608a419fc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:29:42.393 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.393+0000 7fa87ca00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5608a419f8b0 0x5608a419fc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:55108/0 (socket says 172.21.15.47:55108) 2024-03-21T10:29:42.393 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.393+0000 7fa87ca00640 1 -- 172.21.15.47:0/2185458808 learned_addr learned my addr 172.21.15.47:0/2185458808 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:29:42.394 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.393+0000 7fa874c00640 1 -- 172.21.15.47:0/3793812704 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 27) v1 ==== 137985+0+0 (secure 0 0 0) 0x7fa868046020 con 0x7fa8781026d0 2024-03-21T10:29:42.394 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.393+0000 7fa877400640 1 --2- 172.21.15.47:0/2185458808 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5608a41a8de0 0x5608a41ab230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:29:42.394 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.393+0000 7fa87ca00640 1 -- 172.21.15.47:0/2185458808 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5608a41a0260 msgr2=0x5608a41a8810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:29:42.394 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.393+0000 7fa87ca00640 1 --2- 172.21.15.47:0/2185458808 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5608a41a0260 0x5608a41a8810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:29:42.394 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.393+0000 7fa87ca00640 1 -- 172.21.15.47:0/2185458808 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5608a41a8de0 msgr2=0x5608a41ab230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:29:42.394 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.393+0000 7fa87ca00640 1 --2- 172.21.15.47:0/2185458808 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5608a41a8de0 0x5608a41ab230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:29:42.394 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.393+0000 7fa87ca00640 1 -- 172.21.15.47:0/2185458808 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5608a40d0080 con 0x5608a419f8b0 2024-03-21T10:29:42.394 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.393+0000 7fa87ca00640 1 --2- 172.21.15.47:0/2185458808 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5608a419f8b0 0x5608a419fc90 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 crypto rx=0x7fa870008de0 tx=0x7fa87000d710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8dd9be50ed56d376 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:29:42.394 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.394+0000 7fa86d200640 1 -- 172.21.15.47:0/2185458808 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa87000f830 con 0x5608a419f8b0 2024-03-21T10:29:42.394 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.394+0000 7fa86d200640 1 -- 172.21.15.47:0/2185458808 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fa87000de20 con 0x5608a419f8b0 2024-03-21T10:29:42.394 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.394+0000 7fa86d200640 1 -- 172.21.15.47:0/2185458808 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa87000e3b0 con 0x5608a419f8b0 2024-03-21T10:29:42.397 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.394+0000 7fa874c00640 1 --2- 172.21.15.47:0/3793812704 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7fa85c0a3090 0x7fa85c0a5550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:29:42.397 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.394+0000 7fa874c00640 1 -- 172.21.15.47:0/3793812704 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 9474+0+0 (secure 0 0 0) 0x7fa8680f1df0 con 0x7fa8781026d0 2024-03-21T10:29:42.397 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.395+0000 7fa881647740 1 -- 172.21.15.47:0/2185458808 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5608a419f8b0 msgr2=0x5608a419fc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:29:42.397 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.395+0000 7fa881647740 1 --2- 172.21.15.47:0/2185458808 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5608a419f8b0 0x5608a419fc90 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 crypto rx=0x7fa870008de0 tx=0x7fa87000d710 comp rx=0 tx=0).stop 2024-03-21T10:29:42.397 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.395+0000 7fa877e00640 1 --2- 172.21.15.47:0/3793812704 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7fa85c0a3090 0x7fa85c0a5550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:29:42.397 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.395+0000 7fa881647740 1 -- 172.21.15.47:0/2185458808 shutdown_connections 2024-03-21T10:29:42.397 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.395+0000 7fa881647740 1 --2- 172.21.15.47:0/2185458808 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5608a41a8de0 0x5608a41ab230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:29:42.397 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.395+0000 7fa881647740 1 --2- 172.21.15.47:0/2185458808 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5608a41a0260 0x5608a41a8810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:29:42.397 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.395+0000 7fa881647740 1 --2- 172.21.15.47:0/2185458808 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5608a419f8b0 0x5608a419fc90 unknown :-1 s=CLOSED pgs=819 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:29:42.398 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.395+0000 7fa881647740 1 -- 172.21.15.47:0/2185458808 >> 172.21.15.47:0/2185458808 conn(0x5608a4199f80 msgr2=0x5608a419a360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:29:42.398 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.395+0000 7fa881647740 1 -- 172.21.15.47:0/2185458808 shutdown_connections 2024-03-21T10:29:42.398 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.395+0000 7fa881647740 1 -- 172.21.15.47:0/2185458808 wait complete. 2024-03-21T10:29:42.398 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.395+0000 7fa877e00640 1 --2- 172.21.15.47:0/3793812704 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7fa85c0a3090 0x7fa85c0a5550 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x5608a419f380 tx=0x7fa8680de040 comp rx=0 tx=0).ready entity=mgr.15687 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:29:42.398 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.396+0000 7fa881647740 1 -- start start 2024-03-21T10:29:42.398 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.396+0000 7fa881647740 1 Processor -- start 2024-03-21T10:29:42.398 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.396+0000 7fa881647740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5608a419f8b0 0x5608a41b2e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:29:42.398 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.396+0000 7fa881647740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5608a41a0260 0x5608a41b3380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:29:42.398 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.396+0000 7fa881647740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5608a41a8de0 0x5608a41ad250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:29:42.398 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.396+0000 7fa881647740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x5608a41abd50 con 0x5608a419f8b0 2024-03-21T10:29:42.398 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.396+0000 7fa881647740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x5608a40bf0d0 con 0x5608a41a8de0 2024-03-21T10:29:42.398 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.396+0000 7fa881647740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x5608a41abed0 con 0x5608a41a0260 2024-03-21T10:29:42.398 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.396+0000 7fa881647740 10 client.0 _ll_register_callbacks cb 0x5608a40c3e80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T10:29:42.398 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.397+0000 7fa87ca00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5608a419f8b0 0x5608a41b2e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:29:42.399 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.397+0000 7fa87ca00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5608a419f8b0 0x5608a41b2e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:55118/0 (socket says 172.21.15.47:55118) 2024-03-21T10:29:42.399 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.397+0000 7fa87ca00640 1 -- 172.21.15.47:0/395864460 learned_addr learned my addr 172.21.15.47:0/395864460 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:29:42.399 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.397+0000 7fa877400640 1 --2- 172.21.15.47:0/395864460 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5608a41a0260 0x5608a41b3380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:29:42.399 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.397+0000 7fa87ca00640 1 -- 172.21.15.47:0/395864460 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5608a41a0260 msgr2=0x5608a41b3380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:29:42.399 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.397+0000 7fa87ca00640 1 --2- 172.21.15.47:0/395864460 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5608a41a0260 0x5608a41b3380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:29:42.399 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.397+0000 7fa87ca00640 1 -- 172.21.15.47:0/395864460 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5608a41a8de0 msgr2=0x5608a41ad250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:29:42.399 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.397+0000 7fa87ca00640 1 --2- 172.21.15.47:0/395864460 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5608a41a8de0 0x5608a41ad250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:29:42.399 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.397+0000 7fa87ca00640 1 -- 172.21.15.47:0/395864460 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5608a3e173f0 con 0x5608a419f8b0 2024-03-21T10:29:42.399 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.397+0000 7fa877400640 1 --2- 172.21.15.47:0/395864460 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5608a41a0260 0x5608a41b3380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T10:29:42.399 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.398+0000 7fa87ca00640 1 --2- 172.21.15.47:0/395864460 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5608a419f8b0 0x5608a41b2e40 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 crypto rx=0x7fa870018be0 tx=0x7fa870008ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=913ae14b9c5122d6 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:29:42.399 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.398+0000 7fa861600640 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-03-21T10:29:42.399 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.398+0000 7fa861600640 1 -- 172.21.15.47:0/395864460 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa87000f830 con 0x5608a419f8b0 2024-03-21T10:29:42.399 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.398+0000 7fa881647740 1 -- 172.21.15.47:0/395864460 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fa87000de20 con 0x5608a419f8b0 2024-03-21T10:29:42.399 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.398+0000 7fa861600640 1 -- 172.21.15.47:0/395864460 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(97..97 src has 1..97) v4 ==== 9474+0+0 (secure 0 0 0) 0x7fa87000f9d0 con 0x5608a419f8b0 2024-03-21T10:29:42.402 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.398+0000 7fa861600640 1 -- 172.21.15.47:0/395864460 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa87002f050 con 0x5608a419f8b0 2024-03-21T10:29:42.402 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.398+0000 7fa881647740 10 client.17040 fetch_fsmap learned FSMap version 32 2024-03-21T10:29:42.402 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.398+0000 7fa881647740 1 -- 172.21.15.47:0/395864460 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x5608a3ee17d0 con 0x5608a419f8b0 2024-03-21T10:29:42.402 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.399+0000 7fa861600640 1 -- 172.21.15.47:0/395864460 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fa87000de20 con 0x5608a419f8b0 2024-03-21T10:29:42.402 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.399+0000 7fa861600640 1 -- 172.21.15.47:0/395864460 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 32) v1 ==== 7942+0+0 (secure 0 0 0) 0x7fa870026470 con 0x5608a419f8b0 2024-03-21T10:29:42.402 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.399+0000 7fa881647740 10 client.17040 fetch_fsmap finished waiting for FSMap version 32 2024-03-21T10:29:42.402 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.399+0000 7fa881647740 10 client.17040 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.e 2024-03-21T10:29:42.402 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.399+0000 7fa881647740 20 client.17040 populate_metadata read hostname 'smithi047' 2024-03-21T10:29:42.403 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.399+0000 7fa881647740 1 --2- 172.21.15.47:0/395864460 >> [v2:172.21.15.100:6834/2597849957,v1:172.21.15.100:6835/2597849957] conn(0x5608a41bcf70 0x5608a41bf360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:29:42.403 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.399+0000 7fa881647740 4 client.17040 mds_command: new command op to 34454 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T10:29:42.403 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.399+0000 7fa881647740 1 -- 172.21.15.47:0/395864460 --> [v2:172.21.15.100:6834/2597849957,v1:172.21.15.100:6835/2597849957] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5608a4110800 con 0x5608a41bcf70 2024-03-21T10:29:42.403 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.400+0000 7fa877400640 1 --2- 172.21.15.47:0/395864460 >> [v2:172.21.15.100:6834/2597849957,v1:172.21.15.100:6835/2597849957] conn(0x5608a41bcf70 0x5608a41bf360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:29:42.403 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.400+0000 7fa877400640 1 --2- 172.21.15.47:0/395864460 >> [v2:172.21.15.100:6834/2597849957,v1:172.21.15.100:6835/2597849957] conn(0x5608a41bcf70 0x5608a41bf360 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=1be4751077f1a688 server_cookie=60e740c158fd266e in_seq=0 out_seq=0 2024-03-21T10:29:42.403 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.400+0000 7fa861600640 10 client.17040 ms_handle_connect on v2:172.21.15.100:6834/2597849957 2024-03-21T10:29:42.403 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.401+0000 7fa861600640 1 -- 172.21.15.47:0/395864460 <== mds.2 v2:172.21.15.100:6834/2597849957 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x5608a4110800 con 0x5608a41bcf70 2024-03-21T10:29:42.403 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.401+0000 7fa861600640 10 client.17040 handle_command_reply: tid=0 2024-03-21T10:29:42.403 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.401+0000 7fa881647740 1 client.17040 shutdown 2024-03-21T10:29:42.403 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.401+0000 7fa881647740 1 -- 172.21.15.47:0/395864460 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5608a419f8b0 msgr2=0x5608a41b2e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:29:42.403 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.401+0000 7fa881647740 1 --2- 172.21.15.47:0/395864460 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5608a419f8b0 0x5608a41b2e40 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 crypto rx=0x7fa870018be0 tx=0x7fa870008ad0 comp rx=0 tx=0).stop 2024-03-21T10:29:42.403 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.402+0000 7fa881647740 1 -- 172.21.15.47:0/395864460 shutdown_connections 2024-03-21T10:29:42.403 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.402+0000 7fa881647740 1 --2- 172.21.15.47:0/395864460 >> [v2:172.21.15.100:6834/2597849957,v1:172.21.15.100:6835/2597849957] conn(0x5608a41bcf70 0x5608a41bf360 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:29:42.403 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.402+0000 7fa881647740 1 --2- 172.21.15.47:0/395864460 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5608a41a8de0 0x5608a41ad250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:29:42.403 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.402+0000 7fa881647740 1 --2- 172.21.15.47:0/395864460 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5608a41a0260 0x5608a41b3380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:29:42.404 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.402+0000 7fa861600640 0 client.17040 ms_handle_reset on v2:172.21.15.100:6834/2597849957 2024-03-21T10:29:42.404 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.402+0000 7fa881647740 1 --2- 172.21.15.47:0/395864460 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5608a419f8b0 0x5608a41b2e40 unknown :-1 s=CLOSED pgs=820 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:29:42.404 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.402+0000 7fa881647740 1 -- 172.21.15.47:0/395864460 >> 172.21.15.47:0/395864460 conn(0x5608a4199f80 msgr2=0x5608a419bcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:29:42.404 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.402+0000 7fa881647740 1 -- 172.21.15.47:0/395864460 shutdown_connections 2024-03-21T10:29:42.404 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.402+0000 7fa881647740 1 -- 172.21.15.47:0/395864460 wait complete. 2024-03-21T10:29:42.404 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.402+0000 7fa881647740 20 client.17040 trim_cache size 0 max 16384 2024-03-21T10:29:42.422 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.422+0000 7fa881647740 1 Processor -- start 2024-03-21T10:29:42.422 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.422+0000 7fa881647740 1 -- start start 2024-03-21T10:29:42.422 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.422+0000 7fa881647740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5608a419f8b0 0x5608a41c4390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:29:42.422 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.422+0000 7fa881647740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5608a41a0260 0x5608a41a70f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:29:42.422 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.423+0000 7fa881647740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5608a41a8de0 0x5608a41a7630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:29:42.422 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.423+0000 7fa881647740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8781ac520 con 0x5608a419f8b0 2024-03-21T10:29:42.423 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.423+0000 7fa881647740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8781adea0 con 0x5608a41a8de0 2024-03-21T10:29:42.423 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.423+0000 7fa881647740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8781acb80 con 0x5608a41a0260 2024-03-21T10:29:42.423 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.423+0000 7fa87ca00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5608a419f8b0 0x5608a41c4390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:29:42.423 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.423+0000 7fa87ca00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5608a419f8b0 0x5608a41c4390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:55134/0 (socket says 172.21.15.47:55134) 2024-03-21T10:29:42.423 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.423+0000 7fa87ca00640 1 -- 172.21.15.47:0/3041240135 learned_addr learned my addr 172.21.15.47:0/3041240135 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:29:42.424 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.423+0000 7fa877400640 1 --2- 172.21.15.47:0/3041240135 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5608a41a0260 0x5608a41a70f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:29:42.424 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.423+0000 7fa87ca00640 1 -- 172.21.15.47:0/3041240135 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5608a41a0260 msgr2=0x5608a41a70f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:29:42.425 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.423+0000 7fa87ca00640 1 --2- 172.21.15.47:0/3041240135 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5608a41a0260 0x5608a41a70f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:29:42.425 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.423+0000 7fa87ca00640 1 -- 172.21.15.47:0/3041240135 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5608a41a8de0 msgr2=0x5608a41a7630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:29:42.425 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.423+0000 7fa87ca00640 1 --2- 172.21.15.47:0/3041240135 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5608a41a8de0 0x5608a41a7630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:29:42.425 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.423+0000 7fa87ca00640 1 -- 172.21.15.47:0/3041240135 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5608a41ab650 con 0x5608a419f8b0 2024-03-21T10:29:42.425 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.423+0000 7fa877400640 1 --2- 172.21.15.47:0/3041240135 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5608a41a0260 0x5608a41a70f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T10:29:42.425 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.424+0000 7fa87ca00640 1 --2- 172.21.15.47:0/3041240135 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5608a419f8b0 0x5608a41c4390 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 crypto rx=0x7fa87002d040 tx=0x7fa8700082a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2583f3fe5e002b33 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:29:42.425 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.424+0000 7fa863e00640 1 -- 172.21.15.47:0/3041240135 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa870018a40 con 0x5608a419f8b0 2024-03-21T10:29:42.425 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.424+0000 7fa863e00640 1 -- 172.21.15.47:0/3041240135 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fa870021070 con 0x5608a419f8b0 2024-03-21T10:29:42.425 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.424+0000 7fa863e00640 1 -- 172.21.15.47:0/3041240135 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa87000fae0 con 0x5608a419f8b0 2024-03-21T10:29:42.425 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.425+0000 7fa881647740 1 -- 172.21.15.47:0/3041240135 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5608a419f8b0 msgr2=0x5608a41c4390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:29:42.425 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.425+0000 7fa881647740 1 --2- 172.21.15.47:0/3041240135 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5608a419f8b0 0x5608a41c4390 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 crypto rx=0x7fa87002d040 tx=0x7fa8700082a0 comp rx=0 tx=0).stop 2024-03-21T10:29:42.425 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.425+0000 7fa881647740 1 -- 172.21.15.47:0/3041240135 shutdown_connections 2024-03-21T10:29:42.425 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.425+0000 7fa881647740 1 --2- 172.21.15.47:0/3041240135 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5608a41a8de0 0x5608a41a7630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:29:42.425 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.425+0000 7fa881647740 1 --2- 172.21.15.47:0/3041240135 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5608a41a0260 0x5608a41a70f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:29:42.426 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.425+0000 7fa881647740 1 --2- 172.21.15.47:0/3041240135 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5608a419f8b0 0x5608a41c4390 unknown :-1 s=CLOSED pgs=821 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:29:42.426 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.425+0000 7fa881647740 1 -- 172.21.15.47:0/3041240135 >> 172.21.15.47:0/3041240135 conn(0x5608a4199f80 msgr2=0x5608a41a5450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:29:42.426 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.425+0000 7fa881647740 1 -- 172.21.15.47:0/3041240135 shutdown_connections 2024-03-21T10:29:42.426 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.425+0000 7fa881647740 1 -- 172.21.15.47:0/3041240135 wait complete. 2024-03-21T10:29:42.426 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.425+0000 7fa881647740 1 -- start start 2024-03-21T10:29:42.426 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.425+0000 7fa881647740 1 Processor -- start 2024-03-21T10:29:42.428 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.426+0000 7fa881647740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5608a419f8b0 0x5608a41b9530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:29:42.428 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.426+0000 7fa881647740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5608a41a0260 0x5608a41b78d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:29:42.428 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.426+0000 7fa881647740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5608a41a8de0 0x5608a41b7e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:29:42.429 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.426+0000 7fa881647740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8781ad840 con 0x5608a41a8de0 2024-03-21T10:29:42.429 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.426+0000 7fa881647740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8781ad1e0 con 0x5608a419f8b0 2024-03-21T10:29:42.429 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.426+0000 7fa881647740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8781aa6e0 con 0x5608a41a0260 2024-03-21T10:29:42.429 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.426+0000 7fa881647740 10 client.0 _ll_register_callbacks cb 0x7fa870008c30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T10:29:42.429 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.426+0000 7fa87ca00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5608a41a8de0 0x5608a41b7e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:29:42.429 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.426+0000 7fa87ca00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5608a419f8b0 0x5608a41b9530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:29:42.429 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.426+0000 7fa87ca00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5608a41a8de0 0x5608a41b7e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:55150/0 (socket says 172.21.15.47:55150) 2024-03-21T10:29:42.429 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.426+0000 7fa87ca00640 1 -- 172.21.15.47:0/366195280 learned_addr learned my addr 172.21.15.47:0/366195280 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:29:42.429 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.427+0000 7fa877400640 1 --2- 172.21.15.47:0/366195280 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5608a41a0260 0x5608a41b78d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:29:42.429 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.427+0000 7fa87ca00640 1 -- 172.21.15.47:0/366195280 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5608a41a0260 msgr2=0x5608a41b78d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:29:42.429 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.427+0000 7fa87ca00640 1 --2- 172.21.15.47:0/366195280 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5608a41a0260 0x5608a41b78d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:29:42.429 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.427+0000 7fa87ca00640 1 -- 172.21.15.47:0/366195280 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5608a419f8b0 msgr2=0x5608a41b9530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:29:42.429 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.427+0000 7fa87ca00640 1 --2- 172.21.15.47:0/366195280 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5608a419f8b0 0x5608a41b9530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:29:42.430 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.427+0000 7fa87ca00640 1 -- 172.21.15.47:0/366195280 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5608a41b8350 con 0x5608a41a8de0 2024-03-21T10:29:42.430 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.427+0000 7fa87ca00640 1 --2- 172.21.15.47:0/366195280 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5608a41a8de0 0x5608a41b7e10 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x7fa870008410 tx=0x7fa87001a040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a9dd9b028063d1a7 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:29:42.430 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.427+0000 7fa861600640 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-03-21T10:29:42.430 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.427+0000 7fa861600640 1 -- 172.21.15.47:0/366195280 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa870026470 con 0x5608a41a8de0 2024-03-21T10:29:42.430 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.427+0000 7fa861600640 1 -- 172.21.15.47:0/366195280 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(97..97 src has 1..97) v4 ==== 9474+0+0 (secure 0 0 0) 0x7fa870026610 con 0x5608a41a8de0 2024-03-21T10:29:42.430 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.427+0000 7fa881647740 1 -- 172.21.15.47:0/366195280 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fa870021070 con 0x5608a41a8de0 2024-03-21T10:29:42.430 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.428+0000 7fa861600640 1 -- 172.21.15.47:0/366195280 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa87000f630 con 0x5608a41a8de0 2024-03-21T10:29:42.430 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.428+0000 7fa881647740 10 client.17046 fetch_fsmap learned FSMap version 32 2024-03-21T10:29:42.430 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.428+0000 7fa881647740 1 -- 172.21.15.47:0/366195280 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x5608a4219190 con 0x5608a41a8de0 2024-03-21T10:29:42.430 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.428+0000 7fa861600640 1 -- 172.21.15.47:0/366195280 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fa870021070 con 0x5608a41a8de0 2024-03-21T10:29:42.430 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.428+0000 7fa861600640 1 -- 172.21.15.47:0/366195280 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 32) v1 ==== 7942+0+0 (secure 0 0 0) 0x7fa870005530 con 0x5608a41a8de0 2024-03-21T10:29:42.430 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.428+0000 7fa881647740 10 client.17046 fetch_fsmap finished waiting for FSMap version 32 2024-03-21T10:29:42.430 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.428+0000 7fa881647740 10 client.17046 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.e 2024-03-21T10:29:42.431 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.428+0000 7fa881647740 20 client.17046 populate_metadata read hostname 'smithi047' 2024-03-21T10:29:42.431 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.429+0000 7fa881647740 1 --2- 172.21.15.47:0/366195280 >> [v2:172.21.15.100:6834/2597849957,v1:172.21.15.100:6835/2597849957] conn(0x5608a42197a0 0x5608a421bb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:29:42.431 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.429+0000 7fa881647740 4 client.17046 mds_command: new command op to 34454 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-21T10:29:42.431 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.429+0000 7fa881647740 1 -- 172.21.15.47:0/366195280 --> [v2:172.21.15.100:6834/2597849957,v1:172.21.15.100:6835/2597849957] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x5608a410ab50 con 0x5608a42197a0 2024-03-21T10:29:42.431 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.429+0000 7fa877400640 1 --2- 172.21.15.47:0/366195280 >> [v2:172.21.15.100:6834/2597849957,v1:172.21.15.100:6835/2597849957] conn(0x5608a42197a0 0x5608a421bb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:29:42.432 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.430+0000 7fa877400640 1 --2- 172.21.15.47:0/366195280 >> [v2:172.21.15.100:6834/2597849957,v1:172.21.15.100:6835/2597849957] conn(0x5608a42197a0 0x5608a421bb90 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=d1e66be695d58bec server_cookie=28823dc35b415037 in_seq=0 out_seq=0 2024-03-21T10:29:42.432 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.430+0000 7fa861600640 10 client.17046 ms_handle_connect on v2:172.21.15.100:6834/2597849957 2024-03-21T10:29:42.432 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.430+0000 7fa861600640 1 -- 172.21.15.47:0/366195280 <== mds.2 v2:172.21.15.100:6834/2597849957 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x5608a410ab50 con 0x5608a42197a0 2024-03-21T10:29:42.432 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.430+0000 7fa861600640 10 client.17046 handle_command_reply: tid=0 2024-03-21T10:29:42.432 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.430+0000 7fa881647740 1 client.17046 shutdown 2024-03-21T10:29:42.432 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.430+0000 7fa881647740 1 -- 172.21.15.47:0/366195280 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5608a41a8de0 msgr2=0x5608a41b7e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:29:42.432 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.430+0000 7fa881647740 1 --2- 172.21.15.47:0/366195280 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5608a41a8de0 0x5608a41b7e10 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x7fa870008410 tx=0x7fa87001a040 comp rx=0 tx=0).stop 2024-03-21T10:29:42.432 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.430+0000 7fa881647740 1 -- 172.21.15.47:0/366195280 shutdown_connections 2024-03-21T10:29:42.433 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.430+0000 7fa881647740 1 --2- 172.21.15.47:0/366195280 >> [v2:172.21.15.100:6834/2597849957,v1:172.21.15.100:6835/2597849957] conn(0x5608a42197a0 0x5608a421bb90 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:29:42.433 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.430+0000 7fa881647740 1 --2- 172.21.15.47:0/366195280 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5608a41a8de0 0x5608a41b7e10 unknown :-1 s=CLOSED pgs=822 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:29:42.433 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.430+0000 7fa881647740 1 --2- 172.21.15.47:0/366195280 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5608a41a0260 0x5608a41b78d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:29:42.433 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.430+0000 7fa881647740 1 --2- 172.21.15.47:0/366195280 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5608a419f8b0 0x5608a41b9530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:29:42.433 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.430+0000 7fa881647740 1 -- 172.21.15.47:0/366195280 >> 172.21.15.47:0/366195280 conn(0x5608a4199f80 msgr2=0x5608a41988d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:29:42.433 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.430+0000 7fa861600640 0 client.17046 ms_handle_reset on v2:172.21.15.100:6834/2597849957 2024-03-21T10:29:42.433 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.431+0000 7fa881647740 1 -- 172.21.15.47:0/366195280 shutdown_connections 2024-03-21T10:29:42.433 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.431+0000 7fa881647740 1 -- 172.21.15.47:0/366195280 wait complete. 2024-03-21T10:29:42.433 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.431+0000 7fa881647740 20 client.17046 trim_cache size 0 max 16384 2024-03-21T10:29:42.433 INFO:teuthology.orchestra.run.smithi047.stdout:[] 2024-03-21T10:29:42.434 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.431+0000 7fa87d400640 1 -- 172.21.15.47:0/3793812704 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7fa85c0a3090 msgr2=0x7fa85c0a5550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:29:42.434 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.431+0000 7fa87d400640 1 --2- 172.21.15.47:0/3793812704 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7fa85c0a3090 0x7fa85c0a5550 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x5608a419f380 tx=0x7fa8680de040 comp rx=0 tx=0).stop 2024-03-21T10:29:42.434 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.431+0000 7fa87d400640 1 -- 172.21.15.47:0/3793812704 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa8781026d0 msgr2=0x7fa878197d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:29:42.434 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.431+0000 7fa87d400640 1 --2- 172.21.15.47:0/3793812704 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa8781026d0 0x7fa878197d40 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 crypto rx=0x7fa868005940 tx=0x7fa86802d880 comp rx=0 tx=0).stop 2024-03-21T10:29:42.434 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.432+0000 7fa87d400640 1 -- 172.21.15.47:0/3793812704 shutdown_connections 2024-03-21T10:29:42.434 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.432+0000 7fa87d400640 1 --2- 172.21.15.47:0/3793812704 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7fa85c0a3090 0x7fa85c0a5550 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:29:42.435 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.432+0000 7fa87d400640 1 --2- 172.21.15.47:0/3793812704 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa87810baf0 0x7fa87819c600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:29:42.435 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.432+0000 7fa87d400640 1 --2- 172.21.15.47:0/3793812704 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa878102ff0 0x7fa878198280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:29:42.435 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.432+0000 7fa87d400640 1 --2- 172.21.15.47:0/3793812704 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa8781026d0 0x7fa878197d40 unknown :-1 s=CLOSED pgs=818 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:29:42.435 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.432+0000 7fa87d400640 1 -- 172.21.15.47:0/3793812704 >> 172.21.15.47:0/3793812704 conn(0x7fa8780fbef0 msgr2=0x7fa8780fcd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:29:42.435 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.432+0000 7fa87d400640 1 -- 172.21.15.47:0/3793812704 shutdown_connections 2024-03-21T10:29:42.435 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.432+0000 7fa87d400640 1 -- 172.21.15.47:0/3793812704 wait complete. 2024-03-21T10:29:42.448 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-21T10:29:42.448 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-03-21T10:29:42.676 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.676+0000 7f1178c00640 1 -- 172.21.15.47:0/2004393163 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1174103840 msgr2=0x7f1174103cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:29:42.677 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.676+0000 7f1178c00640 1 --2- 172.21.15.47:0/2004393163 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1174103840 0x7f1174103cc0 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 crypto rx=0x7f1160009930 tx=0x7f116002bc30 comp rx=0 tx=0).stop 2024-03-21T10:29:42.677 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.677+0000 7f1178c00640 1 -- 172.21.15.47:0/2004393163 shutdown_connections 2024-03-21T10:29:42.677 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.677+0000 7f1178c00640 1 --2- 172.21.15.47:0/2004393163 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1174104200 0x7f1174110710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:29:42.677 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.677+0000 7f1178c00640 1 --2- 172.21.15.47:0/2004393163 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1174103840 0x7f1174103cc0 unknown :-1 s=CLOSED pgs=823 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:29:42.677 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.677+0000 7f1178c00640 1 --2- 172.21.15.47:0/2004393163 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1174109850 0x7f1174109c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:29:42.677 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.677+0000 7f1178c00640 1 -- 172.21.15.47:0/2004393163 >> 172.21.15.47:0/2004393163 conn(0x7f11740ff2f0 msgr2=0x7f1174101710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:29:42.677 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.677+0000 7f1178c00640 1 -- 172.21.15.47:0/2004393163 shutdown_connections 2024-03-21T10:29:42.677 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.677+0000 7f1178c00640 1 -- 172.21.15.47:0/2004393163 wait complete. 2024-03-21T10:29:42.679 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.678+0000 7f1178c00640 1 Processor -- start 2024-03-21T10:29:42.679 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.678+0000 7f1178c00640 1 -- start start 2024-03-21T10:29:42.679 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.678+0000 7f1178c00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1174103840 0x7f117419a0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:29:42.679 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.678+0000 7f1178c00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1174104200 0x7f117419a600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:29:42.679 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.678+0000 7f1178c00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1174109850 0x7f117419e980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:29:42.680 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.678+0000 7f1178c00640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1174113b10 con 0x7f1174103840 2024-03-21T10:29:42.680 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.678+0000 7f1178c00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1174113990 con 0x7f1174104200 2024-03-21T10:29:42.680 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.678+0000 7f1178c00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1174113c90 con 0x7f1174109850 2024-03-21T10:29:42.680 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.679+0000 7f1171e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1174103840 0x7f117419a0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:29:42.680 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.679+0000 7f1171e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1174103840 0x7f117419a0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:55172/0 (socket says 172.21.15.47:55172) 2024-03-21T10:29:42.680 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.679+0000 7f1171e00640 1 -- 172.21.15.47:0/382431472 learned_addr learned my addr 172.21.15.47:0/382431472 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:29:42.680 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.679+0000 7f1171400640 1 --2- 172.21.15.47:0/382431472 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1174104200 0x7f117419a600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:29:42.680 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.679+0000 7f1172800640 1 --2- 172.21.15.47:0/382431472 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1174109850 0x7f117419e980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:29:42.680 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.679+0000 7f1171e00640 1 -- 172.21.15.47:0/382431472 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1174109850 msgr2=0x7f117419e980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:29:42.680 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.679+0000 7f1171e00640 1 --2- 172.21.15.47:0/382431472 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1174109850 0x7f117419e980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:29:42.680 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.679+0000 7f1171e00640 1 -- 172.21.15.47:0/382431472 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1174104200 msgr2=0x7f117419a600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:29:42.680 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.679+0000 7f1171e00640 1 --2- 172.21.15.47:0/382431472 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1174104200 0x7f117419a600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:29:42.680 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.679+0000 7f1171e00640 1 -- 172.21.15.47:0/382431472 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f117419f0b0 con 0x7f1174103840 2024-03-21T10:29:42.680 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.680+0000 7f1171e00640 1 --2- 172.21.15.47:0/382431472 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1174103840 0x7f117419a0c0 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 crypto rx=0x7f116400d950 tx=0x7f116400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:29:42.681 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.680+0000 7f116aa00640 1 -- 172.21.15.47:0/382431472 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1164017070 con 0x7f1174103840 2024-03-21T10:29:42.681 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.680+0000 7f116aa00640 1 -- 172.21.15.47:0/382431472 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f1164009c30 con 0x7f1174103840 2024-03-21T10:29:42.681 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.680+0000 7f1178c00640 1 -- 172.21.15.47:0/382431472 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f117419f3a0 con 0x7f1174103840 2024-03-21T10:29:42.681 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.680+0000 7f116aa00640 1 -- 172.21.15.47:0/382431472 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f116400f620 con 0x7f1174103840 2024-03-21T10:29:42.681 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.680+0000 7f1178c00640 1 -- 172.21.15.47:0/382431472 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f11741a6c30 con 0x7f1174103840 2024-03-21T10:29:42.682 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.683+0000 7f117b853740 1 Processor -- start 2024-03-21T10:29:42.682 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.683+0000 7f117b853740 1 -- start start 2024-03-21T10:29:42.683 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.684+0000 7f117b853740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fa6f26a6d0 0x55fa6f26aab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:29:42.684 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.684+0000 7f117b853740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fa6f26b080 0x55fa6f273630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:29:42.684 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.684+0000 7f117b853740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55fa6f273c00 0x55fa6f276050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:29:42.684 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.684+0000 7f117b853740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fa6f1ce2c0 con 0x55fa6f273c00 2024-03-21T10:29:42.684 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.684+0000 7f117b853740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fa6f176c90 con 0x55fa6f26b080 2024-03-21T10:29:42.684 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.684+0000 7f117b853740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fa6f1ca360 con 0x55fa6f26a6d0 2024-03-21T10:29:42.684 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.684+0000 7f1171400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55fa6f273c00 0x55fa6f276050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:29:42.684 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.684+0000 7f1171400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55fa6f273c00 0x55fa6f276050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:55184/0 (socket says 172.21.15.47:55184) 2024-03-21T10:29:42.684 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.684+0000 7f1171400640 1 -- 172.21.15.47:0/1511196582 learned_addr learned my addr 172.21.15.47:0/1511196582 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:29:42.684 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.684+0000 7f1172800640 1 --2- 172.21.15.47:0/1511196582 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fa6f26a6d0 0x55fa6f26aab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:29:42.684 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.684+0000 7f1171400640 1 -- 172.21.15.47:0/1511196582 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fa6f26a6d0 msgr2=0x55fa6f26aab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:29:42.684 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.684+0000 7f1171400640 1 --2- 172.21.15.47:0/1511196582 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fa6f26a6d0 0x55fa6f26aab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:29:42.684 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.684+0000 7f116aa00640 1 -- 172.21.15.47:0/382431472 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 27) v1 ==== 137985+0+0 (secure 0 0 0) 0x7f11640109a0 con 0x7f1174103840 2024-03-21T10:29:42.684 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.684+0000 7f1171400640 1 -- 172.21.15.47:0/1511196582 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fa6f26b080 msgr2=0x55fa6f273630 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-21T10:29:42.684 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.684+0000 7f1171400640 1 --2- 172.21.15.47:0/1511196582 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fa6f26b080 0x55fa6f273630 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:29:42.684 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.684+0000 7f1171400640 1 -- 172.21.15.47:0/1511196582 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55fa6f19b080 con 0x55fa6f273c00 2024-03-21T10:29:42.686 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.685+0000 7f1171400640 1 --2- 172.21.15.47:0/1511196582 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55fa6f273c00 0x55fa6f276050 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 crypto rx=0x7f1160036040 tx=0x7f1160005c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c39776d90b9cbed8 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:29:42.686 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.685+0000 7f115f000640 1 -- 172.21.15.47:0/1511196582 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1160045020 con 0x55fa6f273c00 2024-03-21T10:29:42.686 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.685+0000 7f115f000640 1 -- 172.21.15.47:0/1511196582 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f116002c7e0 con 0x55fa6f273c00 2024-03-21T10:29:42.687 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.685+0000 7f115f000640 1 -- 172.21.15.47:0/1511196582 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11600046c0 con 0x55fa6f273c00 2024-03-21T10:29:42.687 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.686+0000 7f116aa00640 1 --2- 172.21.15.47:0/382431472 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f11580a3090 0x7f11580a5550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:29:42.687 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.686+0000 7f117b853740 1 -- 172.21.15.47:0/1511196582 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55fa6f273c00 msgr2=0x55fa6f276050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:29:42.687 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.686+0000 7f117b853740 1 --2- 172.21.15.47:0/1511196582 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55fa6f273c00 0x55fa6f276050 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 crypto rx=0x7f1160036040 tx=0x7f1160005c00 comp rx=0 tx=0).stop 2024-03-21T10:29:42.687 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.686+0000 7f116aa00640 1 -- 172.21.15.47:0/382431472 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 9474+0+0 (secure 0 0 0) 0x7f11640cf750 con 0x7f1174103840 2024-03-21T10:29:42.687 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.686+0000 7f117b853740 1 -- 172.21.15.47:0/1511196582 shutdown_connections 2024-03-21T10:29:42.687 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.686+0000 7f117b853740 1 --2- 172.21.15.47:0/1511196582 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55fa6f273c00 0x55fa6f276050 unknown :-1 s=CLOSED pgs=825 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:29:42.687 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.686+0000 7f117b853740 1 --2- 172.21.15.47:0/1511196582 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fa6f26b080 0x55fa6f273630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:29:42.687 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.686+0000 7f117b853740 1 --2- 172.21.15.47:0/1511196582 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fa6f26a6d0 0x55fa6f26aab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:29:42.687 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.686+0000 7f117b853740 1 -- 172.21.15.47:0/1511196582 >> 172.21.15.47:0/1511196582 conn(0x55fa6f264da0 msgr2=0x55fa6f265180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:29:42.687 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.686+0000 7f1172800640 1 --2- 172.21.15.47:0/382431472 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f11580a3090 0x7f11580a5550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:29:42.687 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.686+0000 7f117b853740 1 -- 172.21.15.47:0/1511196582 shutdown_connections 2024-03-21T10:29:42.687 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.687+0000 7f117b853740 1 -- 172.21.15.47:0/1511196582 wait complete. 2024-03-21T10:29:42.688 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.687+0000 7f117b853740 1 -- start start 2024-03-21T10:29:42.688 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.687+0000 7f117b853740 1 Processor -- start 2024-03-21T10:29:42.688 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.687+0000 7f1172800640 1 --2- 172.21.15.47:0/382431472 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f11580a3090 0x7f11580a5550 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f116c005ee0 tx=0x7f116c005df0 comp rx=0 tx=0).ready entity=mgr.15687 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:29:42.690 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.688+0000 7f117b853740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fa6f26a6d0 0x55fa6f273580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:29:42.690 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.688+0000 7f117b853740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fa6f26b080 0x55fa6f270980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:29:42.690 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.688+0000 7f117b853740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55fa6f273c00 0x55fa6f271000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:29:42.690 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.688+0000 7f117b853740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fa6f276b10 con 0x55fa6f273c00 2024-03-21T10:29:42.690 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.688+0000 7f117b853740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fa6f18a0d0 con 0x55fa6f26a6d0 2024-03-21T10:29:42.691 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.688+0000 7f117b853740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fa6f276c90 con 0x55fa6f26b080 2024-03-21T10:29:42.691 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.688+0000 7f117b853740 10 client.0 _ll_register_callbacks cb 0x55fa6f18ee80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T10:29:42.691 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.688+0000 7f1172800640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55fa6f273c00 0x55fa6f271000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:29:42.691 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.688+0000 7f1171e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fa6f26a6d0 0x55fa6f273580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:29:42.691 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.688+0000 7f1171e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fa6f26a6d0 0x55fa6f273580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:40446/0 (socket says 172.21.15.47:40446) 2024-03-21T10:29:42.691 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.688+0000 7f1171e00640 1 -- 172.21.15.47:0/3717148837 learned_addr learned my addr 172.21.15.47:0/3717148837 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:29:42.691 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.688+0000 7f1171400640 1 --2- 172.21.15.47:0/3717148837 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fa6f26b080 0x55fa6f270980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:29:42.691 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.689+0000 7f1171e00640 1 -- 172.21.15.47:0/3717148837 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fa6f26b080 msgr2=0x55fa6f270980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:29:42.691 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.689+0000 7f1171e00640 1 --2- 172.21.15.47:0/3717148837 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fa6f26b080 0x55fa6f270980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:29:42.691 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.689+0000 7f1171e00640 1 -- 172.21.15.47:0/3717148837 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55fa6f273c00 msgr2=0x55fa6f271000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:29:42.691 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.689+0000 7f1171e00640 1 --2- 172.21.15.47:0/3717148837 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55fa6f273c00 0x55fa6f271000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:29:42.691 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.689+0000 7f1171e00640 1 -- 172.21.15.47:0/3717148837 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55fa6eee23f0 con 0x55fa6f26a6d0 2024-03-21T10:29:42.691 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.689+0000 7f1172800640 1 --2- 172.21.15.47:0/3717148837 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55fa6f273c00 0x55fa6f271000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T10:29:42.691 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.689+0000 7f1171e00640 1 --2- 172.21.15.47:0/3717148837 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fa6f26a6d0 0x55fa6f273580 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f11640cd0e0 tx=0x7f1164009de0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d73fca13dcdb8fd server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:29:42.691 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.689+0000 7f1153400640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-21T10:29:42.692 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.689+0000 7f1153400640 1 -- 172.21.15.47:0/3717148837 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1164020070 con 0x55fa6f26a6d0 2024-03-21T10:29:42.692 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.689+0000 7f117b853740 1 -- 172.21.15.47:0/3717148837 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f116002c7e0 con 0x55fa6f26a6d0 2024-03-21T10:29:42.692 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.689+0000 7f1153400640 1 -- 172.21.15.47:0/3717148837 <== mon.2 v2:172.21.15.100:3300/0 2 ==== osd_map(97..97 src has 1..97) v4 ==== 9474+0+0 (secure 0 0 0) 0x7f1164097df0 con 0x55fa6f26a6d0 2024-03-21T10:29:42.692 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.690+0000 7f1153400640 1 -- 172.21.15.47:0/3717148837 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f116409f8d0 con 0x55fa6f26a6d0 2024-03-21T10:29:42.692 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.690+0000 7f117b853740 10 client.36848 fetch_fsmap learned FSMap version 32 2024-03-21T10:29:42.692 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.690+0000 7f117b853740 1 -- 172.21.15.47:0/3717148837 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x55fa6efac7d0 con 0x55fa6f26a6d0 2024-03-21T10:29:42.692 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.690+0000 7f1153400640 1 -- 172.21.15.47:0/3717148837 <== mon.2 v2:172.21.15.100:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f116002c7e0 con 0x55fa6f26a6d0 2024-03-21T10:29:42.692 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.690+0000 7f1153400640 1 -- 172.21.15.47:0/3717148837 <== mon.2 v2:172.21.15.100:3300/0 5 ==== fsmap(e 32) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f1164097040 con 0x55fa6f26a6d0 2024-03-21T10:29:42.692 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.690+0000 7f117b853740 10 client.36848 fetch_fsmap finished waiting for FSMap version 32 2024-03-21T10:29:42.692 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.690+0000 7f117b853740 10 client.36848 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-03-21T10:29:42.692 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.691+0000 7f117b853740 20 client.36848 populate_metadata read hostname 'smithi047' 2024-03-21T10:29:42.692 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.691+0000 7f117b853740 1 --2- 172.21.15.47:0/3717148837 >> [v2:172.21.15.100:6836/3497644162,v1:172.21.15.100:6837/3497644162] conn(0x55fa6f283a50 0x55fa6f285eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:29:42.692 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.691+0000 7f117b853740 4 client.36848 mds_command: new command op to 34493 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T10:29:42.692 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.691+0000 7f117b853740 1 -- 172.21.15.47:0/3717148837 --> [v2:172.21.15.100:6836/3497644162,v1:172.21.15.100:6837/3497644162] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55fa6f1db800 con 0x55fa6f283a50 2024-03-21T10:29:42.693 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.691+0000 7f1171400640 1 --2- 172.21.15.47:0/3717148837 >> [v2:172.21.15.100:6836/3497644162,v1:172.21.15.100:6837/3497644162] conn(0x55fa6f283a50 0x55fa6f285eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:29:42.694 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.692+0000 7f1171400640 1 --2- 172.21.15.47:0/3717148837 >> [v2:172.21.15.100:6836/3497644162,v1:172.21.15.100:6837/3497644162] conn(0x55fa6f283a50 0x55fa6f285eb0 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=90d37e5d16ce7dfb server_cookie=cef80f24c3a3d1 in_seq=0 out_seq=0 2024-03-21T10:29:42.694 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.692+0000 7f1153400640 10 client.36848 ms_handle_connect on v2:172.21.15.100:6836/3497644162 2024-03-21T10:29:42.694 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.693+0000 7f1153400640 1 -- 172.21.15.47:0/3717148837 <== mds.1 v2:172.21.15.100:6836/3497644162 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55fa6f1db800 con 0x55fa6f283a50 2024-03-21T10:29:42.694 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.693+0000 7f1153400640 10 client.36848 handle_command_reply: tid=0 2024-03-21T10:29:42.694 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.693+0000 7f117b853740 1 client.36848 shutdown 2024-03-21T10:29:42.694 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.693+0000 7f117b853740 1 -- 172.21.15.47:0/3717148837 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fa6f26a6d0 msgr2=0x55fa6f273580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:29:42.695 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.693+0000 7f117b853740 1 --2- 172.21.15.47:0/3717148837 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fa6f26a6d0 0x55fa6f273580 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f11640cd0e0 tx=0x7f1164009de0 comp rx=0 tx=0).stop 2024-03-21T10:29:42.695 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.694+0000 7f117b853740 1 -- 172.21.15.47:0/3717148837 shutdown_connections 2024-03-21T10:29:42.695 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.694+0000 7f117b853740 1 --2- 172.21.15.47:0/3717148837 >> [v2:172.21.15.100:6836/3497644162,v1:172.21.15.100:6837/3497644162] conn(0x55fa6f283a50 0x55fa6f285eb0 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:29:42.695 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.694+0000 7f117b853740 1 --2- 172.21.15.47:0/3717148837 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55fa6f273c00 0x55fa6f271000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:29:42.695 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.694+0000 7f117b853740 1 --2- 172.21.15.47:0/3717148837 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fa6f26b080 0x55fa6f270980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:29:42.695 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.694+0000 7f1153400640 0 client.36848 ms_handle_reset on v2:172.21.15.100:6836/3497644162 2024-03-21T10:29:42.695 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.694+0000 7f117b853740 1 --2- 172.21.15.47:0/3717148837 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fa6f26a6d0 0x55fa6f273580 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:29:42.695 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.694+0000 7f117b853740 1 -- 172.21.15.47:0/3717148837 >> 172.21.15.47:0/3717148837 conn(0x55fa6f264da0 msgr2=0x55fa6f266af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:29:42.695 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.694+0000 7f117b853740 1 -- 172.21.15.47:0/3717148837 shutdown_connections 2024-03-21T10:29:42.695 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.694+0000 7f117b853740 1 -- 172.21.15.47:0/3717148837 wait complete. 2024-03-21T10:29:42.695 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.694+0000 7f117b853740 20 client.36848 trim_cache size 0 max 16384 2024-03-21T10:29:42.707 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.708+0000 7f117b853740 1 Processor -- start 2024-03-21T10:29:42.707 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.708+0000 7f117b853740 1 -- start start 2024-03-21T10:29:42.708 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.708+0000 7f117b853740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fa6f26a6d0 0x55fa6f2730b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:29:42.708 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.708+0000 7f117b853740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fa6f26b080 0x55fa6f266670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:29:42.708 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.708+0000 7f117b853740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55fa6f273c00 0x55fa6f266bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:29:42.708 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.708+0000 7f117b853740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11741b0ac0 con 0x55fa6f273c00 2024-03-21T10:29:42.708 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.708+0000 7f117b853740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11741b2440 con 0x55fa6f26a6d0 2024-03-21T10:29:42.708 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.708+0000 7f117b853740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11741b1120 con 0x55fa6f26b080 2024-03-21T10:29:42.709 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.708+0000 7f1172800640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55fa6f273c00 0x55fa6f266bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:29:42.709 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.708+0000 7f1172800640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55fa6f273c00 0x55fa6f266bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:55194/0 (socket says 172.21.15.47:55194) 2024-03-21T10:29:42.709 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.708+0000 7f1172800640 1 -- 172.21.15.47:0/1156619440 learned_addr learned my addr 172.21.15.47:0/1156619440 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:29:42.709 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.708+0000 7f1171e00640 1 --2- 172.21.15.47:0/1156619440 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fa6f26a6d0 0x55fa6f2730b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:29:42.709 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.709+0000 7f1171400640 1 --2- 172.21.15.47:0/1156619440 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fa6f26b080 0x55fa6f266670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:29:42.709 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.709+0000 7f1172800640 1 -- 172.21.15.47:0/1156619440 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fa6f26b080 msgr2=0x55fa6f266670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:29:42.709 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.709+0000 7f1172800640 1 --2- 172.21.15.47:0/1156619440 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fa6f26b080 0x55fa6f266670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:29:42.709 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.709+0000 7f1172800640 1 -- 172.21.15.47:0/1156619440 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fa6f26a6d0 msgr2=0x55fa6f2730b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:29:42.709 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.709+0000 7f1172800640 1 --2- 172.21.15.47:0/1156619440 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fa6f26a6d0 0x55fa6f2730b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:29:42.709 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.709+0000 7f1172800640 1 -- 172.21.15.47:0/1156619440 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55fa6f26eed0 con 0x55fa6f273c00 2024-03-21T10:29:42.709 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.709+0000 7f1172800640 1 --2- 172.21.15.47:0/1156619440 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55fa6f273c00 0x55fa6f266bb0 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 crypto rx=0x7f116c011790 tx=0x7f116c013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ad3c5d39772a1e02 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:29:42.711 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.709+0000 7f115e600640 1 -- 172.21.15.47:0/1156619440 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f116c01f9b0 con 0x55fa6f273c00 2024-03-21T10:29:42.711 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.710+0000 7f115e600640 1 -- 172.21.15.47:0/1156619440 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f116c0273e0 con 0x55fa6f273c00 2024-03-21T10:29:42.711 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.710+0000 7f115e600640 1 -- 172.21.15.47:0/1156619440 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f116c01a600 con 0x55fa6f273c00 2024-03-21T10:29:42.711 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.710+0000 7f117b853740 1 -- 172.21.15.47:0/1156619440 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55fa6f273c00 msgr2=0x55fa6f266bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:29:42.711 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.710+0000 7f117b853740 1 --2- 172.21.15.47:0/1156619440 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55fa6f273c00 0x55fa6f266bb0 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 crypto rx=0x7f116c011790 tx=0x7f116c013000 comp rx=0 tx=0).stop 2024-03-21T10:29:42.711 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.710+0000 7f117b853740 1 -- 172.21.15.47:0/1156619440 shutdown_connections 2024-03-21T10:29:42.711 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.710+0000 7f117b853740 1 --2- 172.21.15.47:0/1156619440 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55fa6f273c00 0x55fa6f266bb0 unknown :-1 s=CLOSED pgs=826 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:29:42.711 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.710+0000 7f117b853740 1 --2- 172.21.15.47:0/1156619440 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fa6f26b080 0x55fa6f266670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:29:42.711 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.710+0000 7f117b853740 1 --2- 172.21.15.47:0/1156619440 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fa6f26a6d0 0x55fa6f2730b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:29:42.711 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.710+0000 7f117b853740 1 -- 172.21.15.47:0/1156619440 >> 172.21.15.47:0/1156619440 conn(0x55fa6f264da0 msgr2=0x55fa6f265f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:29:42.711 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.711+0000 7f117b853740 1 -- 172.21.15.47:0/1156619440 shutdown_connections 2024-03-21T10:29:42.712 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.711+0000 7f117b853740 1 -- 172.21.15.47:0/1156619440 wait complete. 2024-03-21T10:29:42.712 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.711+0000 7f117b853740 1 -- start start 2024-03-21T10:29:42.712 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.711+0000 7f117b853740 1 Processor -- start 2024-03-21T10:29:42.714 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.712+0000 7f117b853740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fa6f26a6d0 0x55fa6f27b5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:29:42.714 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.712+0000 7f117b853740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55fa6f26b080 0x55fa6f27bb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:29:42.715 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.712+0000 7f117b853740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fa6f273c00 0x55fa6f279a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:29:42.715 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.712+0000 7f117b853740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11741b1de0 con 0x55fa6f26b080 2024-03-21T10:29:42.715 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.712+0000 7f117b853740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11741b1780 con 0x55fa6f273c00 2024-03-21T10:29:42.715 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.712+0000 7f117b853740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11741aec80 con 0x55fa6f26a6d0 2024-03-21T10:29:42.715 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.712+0000 7f117b853740 10 client.0 _ll_register_callbacks cb 0x7f116400df40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T10:29:42.715 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.713+0000 7f1171400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55fa6f26b080 0x55fa6f27bb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:29:42.715 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.713+0000 7f1171400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55fa6f26b080 0x55fa6f27bb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:55198/0 (socket says 172.21.15.47:55198) 2024-03-21T10:29:42.715 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.713+0000 7f1171400640 1 -- 172.21.15.47:0/658376307 learned_addr learned my addr 172.21.15.47:0/658376307 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:29:42.715 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.713+0000 7f1171e00640 1 --2- 172.21.15.47:0/658376307 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fa6f26a6d0 0x55fa6f27b5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:29:42.715 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.713+0000 7f1172800640 1 --2- 172.21.15.47:0/658376307 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fa6f273c00 0x55fa6f279a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:29:42.715 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.713+0000 7f1171400640 1 -- 172.21.15.47:0/658376307 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fa6f26a6d0 msgr2=0x55fa6f27b5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:29:42.715 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.713+0000 7f1171400640 1 --2- 172.21.15.47:0/658376307 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fa6f26a6d0 0x55fa6f27b5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:29:42.715 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.713+0000 7f1171400640 1 -- 172.21.15.47:0/658376307 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fa6f273c00 msgr2=0x55fa6f279a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:29:42.716 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.713+0000 7f1171400640 1 --2- 172.21.15.47:0/658376307 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fa6f273c00 0x55fa6f279a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:29:42.716 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.713+0000 7f1171400640 1 -- 172.21.15.47:0/658376307 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55fa6f27a1f0 con 0x55fa6f26b080 2024-03-21T10:29:42.716 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.713+0000 7f1172800640 1 --2- 172.21.15.47:0/658376307 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fa6f273c00 0x55fa6f279a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T10:29:42.716 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.713+0000 7f1171400640 1 --2- 172.21.15.47:0/658376307 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55fa6f26b080 0x55fa6f27bb00 secure :-1 s=READY pgs=827 cs=0 l=1 rev1=1 crypto rx=0x7f1160036040 tx=0x7f116002bc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a2d1ed787d360289 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:29:42.716 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.713+0000 7f1153400640 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-03-21T10:29:42.716 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.714+0000 7f1153400640 1 -- 172.21.15.47:0/658376307 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1160045040 con 0x55fa6f26b080 2024-03-21T10:29:42.716 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.714+0000 7f117b853740 1 -- 172.21.15.47:0/658376307 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f116c0273e0 con 0x55fa6f26b080 2024-03-21T10:29:42.716 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.714+0000 7f1153400640 1 -- 172.21.15.47:0/658376307 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(97..97 src has 1..97) v4 ==== 9474+0+0 (secure 0 0 0) 0x7f1160004300 con 0x55fa6f26b080 2024-03-21T10:29:42.716 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.714+0000 7f1153400640 1 -- 172.21.15.47:0/658376307 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f116005f050 con 0x55fa6f26b080 2024-03-21T10:29:42.716 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.714+0000 7f117b853740 10 client.17064 fetch_fsmap learned FSMap version 32 2024-03-21T10:29:42.716 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.714+0000 7f117b853740 1 -- 172.21.15.47:0/658376307 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x55fa6f2dfda0 con 0x55fa6f26b080 2024-03-21T10:29:42.716 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.714+0000 7f1153400640 1 -- 172.21.15.47:0/658376307 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f116c0273e0 con 0x55fa6f26b080 2024-03-21T10:29:42.716 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.715+0000 7f1153400640 1 -- 172.21.15.47:0/658376307 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 32) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f1160003b10 con 0x55fa6f26b080 2024-03-21T10:29:42.716 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.715+0000 7f117b853740 10 client.17064 fetch_fsmap finished waiting for FSMap version 32 2024-03-21T10:29:42.716 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.715+0000 7f117b853740 10 client.17064 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-03-21T10:29:42.717 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.715+0000 7f117b853740 20 client.17064 populate_metadata read hostname 'smithi047' 2024-03-21T10:29:42.717 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.715+0000 7f117b853740 1 --2- 172.21.15.47:0/658376307 >> [v2:172.21.15.100:6836/3497644162,v1:172.21.15.100:6837/3497644162] conn(0x55fa6f2e0170 0x55fa6f2e2560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:29:42.717 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.715+0000 7f117b853740 4 client.17064 mds_command: new command op to 34493 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-21T10:29:42.717 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.715+0000 7f117b853740 1 -- 172.21.15.47:0/658376307 --> [v2:172.21.15.100:6836/3497644162,v1:172.21.15.100:6837/3497644162] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55fa6f1d5b50 con 0x55fa6f2e0170 2024-03-21T10:29:42.717 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.715+0000 7f1171e00640 1 --2- 172.21.15.47:0/658376307 >> [v2:172.21.15.100:6836/3497644162,v1:172.21.15.100:6837/3497644162] conn(0x55fa6f2e0170 0x55fa6f2e2560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:29:42.718 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.716+0000 7f1171e00640 1 --2- 172.21.15.47:0/658376307 >> [v2:172.21.15.100:6836/3497644162,v1:172.21.15.100:6837/3497644162] conn(0x55fa6f2e0170 0x55fa6f2e2560 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=7f67aa8f604086e9 server_cookie=b19122ac18b4cd0c in_seq=0 out_seq=0 2024-03-21T10:29:42.718 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.716+0000 7f1153400640 10 client.17064 ms_handle_connect on v2:172.21.15.100:6836/3497644162 2024-03-21T10:29:42.718 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.716+0000 7f1153400640 1 -- 172.21.15.47:0/658376307 <== mds.1 v2:172.21.15.100:6836/3497644162 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55fa6f1d5b50 con 0x55fa6f2e0170 2024-03-21T10:29:42.718 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.716+0000 7f1153400640 10 client.17064 handle_command_reply: tid=0 2024-03-21T10:29:42.718 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.716+0000 7f117b853740 1 client.17064 shutdown 2024-03-21T10:29:42.718 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.716+0000 7f117b853740 1 -- 172.21.15.47:0/658376307 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55fa6f26b080 msgr2=0x55fa6f27bb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:29:42.718 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.716+0000 7f117b853740 1 --2- 172.21.15.47:0/658376307 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55fa6f26b080 0x55fa6f27bb00 secure :-1 s=READY pgs=827 cs=0 l=1 rev1=1 crypto rx=0x7f1160036040 tx=0x7f116002bc30 comp rx=0 tx=0).stop 2024-03-21T10:29:42.719 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.716+0000 7f117b853740 1 -- 172.21.15.47:0/658376307 shutdown_connections 2024-03-21T10:29:42.719 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.716+0000 7f117b853740 1 --2- 172.21.15.47:0/658376307 >> [v2:172.21.15.100:6836/3497644162,v1:172.21.15.100:6837/3497644162] conn(0x55fa6f2e0170 0x55fa6f2e2560 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:29:42.719 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.716+0000 7f117b853740 1 --2- 172.21.15.47:0/658376307 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fa6f273c00 0x55fa6f279a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:29:42.719 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.716+0000 7f117b853740 1 --2- 172.21.15.47:0/658376307 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55fa6f26b080 0x55fa6f27bb00 unknown :-1 s=CLOSED pgs=827 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:29:42.719 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.716+0000 7f117b853740 1 --2- 172.21.15.47:0/658376307 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fa6f26a6d0 0x55fa6f27b5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:29:42.719 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.716+0000 7f117b853740 1 -- 172.21.15.47:0/658376307 >> 172.21.15.47:0/658376307 conn(0x55fa6f264da0 msgr2=0x55fa6f266780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:29:42.719 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.716+0000 7f1153400640 0 client.17064 ms_handle_reset on v2:172.21.15.100:6836/3497644162 2024-03-21T10:29:42.719 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.717+0000 7f117b853740 1 -- 172.21.15.47:0/658376307 shutdown_connections 2024-03-21T10:29:42.719 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.717+0000 7f117b853740 1 -- 172.21.15.47:0/658376307 wait complete. 2024-03-21T10:29:42.719 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.717+0000 7f117b853740 20 client.17064 trim_cache size 0 max 16384 2024-03-21T10:29:42.719 INFO:teuthology.orchestra.run.smithi047.stdout:[] 2024-03-21T10:29:42.720 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.717+0000 7f1178c00640 1 -- 172.21.15.47:0/382431472 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f11580a3090 msgr2=0x7f11580a5550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:29:42.720 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.717+0000 7f1178c00640 1 --2- 172.21.15.47:0/382431472 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f11580a3090 0x7f11580a5550 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f116c005ee0 tx=0x7f116c005df0 comp rx=0 tx=0).stop 2024-03-21T10:29:42.720 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.717+0000 7f1178c00640 1 -- 172.21.15.47:0/382431472 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1174103840 msgr2=0x7f117419a0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:29:42.721 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.717+0000 7f1178c00640 1 --2- 172.21.15.47:0/382431472 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1174103840 0x7f117419a0c0 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 crypto rx=0x7f116400d950 tx=0x7f116400de10 comp rx=0 tx=0).stop 2024-03-21T10:29:42.721 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.718+0000 7f1178c00640 1 -- 172.21.15.47:0/382431472 shutdown_connections 2024-03-21T10:29:42.721 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.718+0000 7f1178c00640 1 --2- 172.21.15.47:0/382431472 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1174109850 0x7f117419e980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:29:42.721 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.718+0000 7f1178c00640 1 --2- 172.21.15.47:0/382431472 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f11580a3090 0x7f11580a5550 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:29:42.721 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.718+0000 7f1178c00640 1 --2- 172.21.15.47:0/382431472 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1174104200 0x7f117419a600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:29:42.721 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.718+0000 7f1178c00640 1 --2- 172.21.15.47:0/382431472 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1174103840 0x7f117419a0c0 unknown :-1 s=CLOSED pgs=824 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:29:42.721 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.718+0000 7f1178c00640 1 -- 172.21.15.47:0/382431472 >> 172.21.15.47:0/382431472 conn(0x7f11740ff2f0 msgr2=0x7f11740ffc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:29:42.721 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.718+0000 7f1178c00640 1 -- 172.21.15.47:0/382431472 shutdown_connections 2024-03-21T10:29:42.721 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.718+0000 7f1178c00640 1 -- 172.21.15.47:0/382431472 wait complete. 2024-03-21T10:29:42.734 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-21T10:29:42.734 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-03-21T10:29:42.963 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.963+0000 7f7b68800640 1 -- 172.21.15.47:0/5973932 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7b60102910 msgr2=0x7f7b60102cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:29:42.963 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.963+0000 7f7b68800640 1 --2- 172.21.15.47:0/5973932 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7b60102910 0x7f7b60102cf0 secure :-1 s=READY pgs=828 cs=0 l=1 rev1=1 crypto rx=0x7f7b54009900 tx=0x7f7b5402bcd0 comp rx=0 tx=0).stop 2024-03-21T10:29:42.963 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.963+0000 7f7b68800640 1 -- 172.21.15.47:0/5973932 shutdown_connections 2024-03-21T10:29:42.963 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.963+0000 7f7b68800640 1 --2- 172.21.15.47:0/5973932 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7b6010bce0 0x7f7b6010e150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:29:42.963 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.963+0000 7f7b68800640 1 --2- 172.21.15.47:0/5973932 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7b60103230 0x7f7b6010b710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:29:42.963 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.963+0000 7f7b68800640 1 --2- 172.21.15.47:0/5973932 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7b60102910 0x7f7b60102cf0 unknown :-1 s=CLOSED pgs=828 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:29:42.963 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.963+0000 7f7b68800640 1 -- 172.21.15.47:0/5973932 >> 172.21.15.47:0/5973932 conn(0x7f7b60077700 msgr2=0x7f7b60077b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:29:42.964 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.963+0000 7f7b68800640 1 -- 172.21.15.47:0/5973932 shutdown_connections 2024-03-21T10:29:42.964 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.963+0000 7f7b68800640 1 -- 172.21.15.47:0/5973932 wait complete. 2024-03-21T10:29:42.965 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.964+0000 7f7b68800640 1 Processor -- start 2024-03-21T10:29:42.966 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.964+0000 7f7b68800640 1 -- start start 2024-03-21T10:29:42.966 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.965+0000 7f7b68800640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7b60102910 0x7f7b6019a100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:29:42.966 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.965+0000 7f7b68800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7b60103230 0x7f7b6019a640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:29:42.966 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.965+0000 7f7b68800640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7b6010bce0 0x7f7b6019e970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:29:42.966 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.965+0000 7f7b68800640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b60111960 con 0x7f7b60102910 2024-03-21T10:29:42.966 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.965+0000 7f7b68800640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b601117e0 con 0x7f7b60103230 2024-03-21T10:29:42.966 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.965+0000 7f7b68800640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b60111ae0 con 0x7f7b6010bce0 2024-03-21T10:29:42.966 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.965+0000 7f7b65e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7b60102910 0x7f7b6019a100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:29:42.966 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.965+0000 7f7b65e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7b60102910 0x7f7b6019a100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:55212/0 (socket says 172.21.15.47:55212) 2024-03-21T10:29:42.966 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.965+0000 7f7b65e00640 1 -- 172.21.15.47:0/4275128922 learned_addr learned my addr 172.21.15.47:0/4275128922 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:29:42.966 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.965+0000 7f7b65400640 1 --2- 172.21.15.47:0/4275128922 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7b60103230 0x7f7b6019a640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:29:42.967 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.965+0000 7f7b66800640 1 --2- 172.21.15.47:0/4275128922 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7b6010bce0 0x7f7b6019e970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:29:42.967 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.966+0000 7f7b65e00640 1 -- 172.21.15.47:0/4275128922 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7b6010bce0 msgr2=0x7f7b6019e970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:29:42.967 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.966+0000 7f7b65e00640 1 --2- 172.21.15.47:0/4275128922 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7b6010bce0 0x7f7b6019e970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:29:42.967 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.966+0000 7f7b65e00640 1 -- 172.21.15.47:0/4275128922 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7b60103230 msgr2=0x7f7b6019a640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:29:42.967 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.966+0000 7f7b65e00640 1 --2- 172.21.15.47:0/4275128922 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7b60103230 0x7f7b6019a640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:29:42.967 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.966+0000 7f7b65e00640 1 -- 172.21.15.47:0/4275128922 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b6019f0a0 con 0x7f7b60102910 2024-03-21T10:29:42.967 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.966+0000 7f7b65e00640 1 --2- 172.21.15.47:0/4275128922 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7b60102910 0x7f7b6019a100 secure :-1 s=READY pgs=829 cs=0 l=1 rev1=1 crypto rx=0x7f7b54005940 tx=0x7f7b540045e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:29:42.967 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.966+0000 7f7b5aa00640 1 -- 172.21.15.47:0/4275128922 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b54043020 con 0x7f7b60102910 2024-03-21T10:29:42.967 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.967+0000 7f7b68800640 1 -- 172.21.15.47:0/4275128922 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7b6019f330 con 0x7f7b60102910 2024-03-21T10:29:42.967 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.967+0000 7f7b5aa00640 1 -- 172.21.15.47:0/4275128922 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f7b5402dba0 con 0x7f7b60102910 2024-03-21T10:29:42.967 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.967+0000 7f7b68800640 1 -- 172.21.15.47:0/4275128922 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7b601a6c20 con 0x7f7b60102910 2024-03-21T10:29:42.968 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.967+0000 7f7b5aa00640 1 -- 172.21.15.47:0/4275128922 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b540039b0 con 0x7f7b60102910 2024-03-21T10:29:42.969 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.970+0000 7f7b6b409740 1 Processor -- start 2024-03-21T10:29:42.969 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.970+0000 7f7b6b409740 1 -- start start 2024-03-21T10:29:42.969 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.970+0000 7f7b6b409740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55924eff06d0 0x55924eff0ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:29:42.970 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.970+0000 7f7b6b409740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55924eff1080 0x55924eff9630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:29:42.970 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.970+0000 7f7b6b409740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55924eff9c00 0x55924effc050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:29:42.970 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.970+0000 7f7b6b409740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x55924ef542c0 con 0x55924eff1080 2024-03-21T10:29:42.970 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.970+0000 7f7b6b409740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55924eefcc90 con 0x55924eff06d0 2024-03-21T10:29:42.970 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.970+0000 7f7b6b409740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55924ef50360 con 0x55924eff9c00 2024-03-21T10:29:42.973 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.971+0000 7f7b5aa00640 1 -- 172.21.15.47:0/4275128922 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 27) v1 ==== 137985+0+0 (secure 0 0 0) 0x7f7b54046020 con 0x7f7b60102910 2024-03-21T10:29:42.973 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.971+0000 7f7b66800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55924eff06d0 0x55924eff0ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:29:42.973 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.971+0000 7f7b66800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55924eff06d0 0x55924eff0ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:40484/0 (socket says 172.21.15.47:40484) 2024-03-21T10:29:42.973 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.971+0000 7f7b66800640 1 -- 172.21.15.47:0/3838714152 learned_addr learned my addr 172.21.15.47:0/3838714152 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:29:42.973 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.971+0000 7f7b65e00640 1 --2- 172.21.15.47:0/3838714152 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55924eff1080 0x55924eff9630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:29:42.973 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.971+0000 7f7b65400640 1 --2- 172.21.15.47:0/3838714152 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55924eff9c00 0x55924effc050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:29:42.973 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.971+0000 7f7b66800640 1 -- 172.21.15.47:0/3838714152 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55924eff9c00 msgr2=0x55924effc050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:29:42.973 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.971+0000 7f7b66800640 1 --2- 172.21.15.47:0/3838714152 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55924eff9c00 0x55924effc050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:29:42.973 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.971+0000 7f7b66800640 1 -- 172.21.15.47:0/3838714152 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55924eff1080 msgr2=0x55924eff9630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:29:42.973 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.971+0000 7f7b66800640 1 --2- 172.21.15.47:0/3838714152 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55924eff1080 0x55924eff9630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:29:42.973 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.971+0000 7f7b66800640 1 -- 172.21.15.47:0/3838714152 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55924ef21080 con 0x55924eff06d0 2024-03-21T10:29:42.974 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.971+0000 7f7b65e00640 1 --2- 172.21.15.47:0/3838714152 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55924eff1080 0x55924eff9630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T10:29:42.974 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.972+0000 7f7b66800640 1 --2- 172.21.15.47:0/3838714152 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55924eff06d0 0x55924eff0ab0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f7b5c009f40 tx=0x7f7b5c010860 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a5323b8074fc76e server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:29:42.974 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.972+0000 7f7b4f000640 1 -- 172.21.15.47:0/3838714152 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b5c003b70 con 0x55924eff06d0 2024-03-21T10:29:42.974 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.972+0000 7f7b4f000640 1 -- 172.21.15.47:0/3838714152 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f7b5c003d10 con 0x55924eff06d0 2024-03-21T10:29:42.974 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.972+0000 7f7b4f000640 1 -- 172.21.15.47:0/3838714152 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b5c0118e0 con 0x55924eff06d0 2024-03-21T10:29:42.974 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.972+0000 7f7b5aa00640 1 --2- 172.21.15.47:0/4275128922 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f7b480a3090 0x7f7b480a5550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:29:42.974 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.973+0000 7f7b5aa00640 1 -- 172.21.15.47:0/4275128922 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 9474+0+0 (secure 0 0 0) 0x7f7b540f1be0 con 0x7f7b60102910 2024-03-21T10:29:42.974 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.973+0000 7f7b65e00640 1 --2- 172.21.15.47:0/4275128922 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f7b480a3090 0x7f7b480a5550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:29:42.974 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.973+0000 7f7b6b409740 1 -- 172.21.15.47:0/3838714152 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55924eff06d0 msgr2=0x55924eff0ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:29:42.974 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.973+0000 7f7b6b409740 1 --2- 172.21.15.47:0/3838714152 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55924eff06d0 0x55924eff0ab0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f7b5c009f40 tx=0x7f7b5c010860 comp rx=0 tx=0).stop 2024-03-21T10:29:42.974 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.973+0000 7f7b6b409740 1 -- 172.21.15.47:0/3838714152 shutdown_connections 2024-03-21T10:29:42.974 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.973+0000 7f7b6b409740 1 --2- 172.21.15.47:0/3838714152 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55924eff9c00 0x55924effc050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:29:42.974 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.973+0000 7f7b6b409740 1 --2- 172.21.15.47:0/3838714152 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55924eff1080 0x55924eff9630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:29:42.974 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.973+0000 7f7b6b409740 1 --2- 172.21.15.47:0/3838714152 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55924eff06d0 0x55924eff0ab0 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:29:42.975 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.973+0000 7f7b6b409740 1 -- 172.21.15.47:0/3838714152 >> 172.21.15.47:0/3838714152 conn(0x55924efeada0 msgr2=0x55924efeb180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:29:42.975 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.973+0000 7f7b6b409740 1 -- 172.21.15.47:0/3838714152 shutdown_connections 2024-03-21T10:29:42.975 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.973+0000 7f7b6b409740 1 -- 172.21.15.47:0/3838714152 wait complete. 2024-03-21T10:29:42.975 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.974+0000 7f7b65e00640 1 --2- 172.21.15.47:0/4275128922 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f7b480a3090 0x7f7b480a5550 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x55924eff01a0 tx=0x7f7b54000bc0 comp rx=0 tx=0).ready entity=mgr.15687 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:29:42.975 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.974+0000 7f7b6b409740 1 -- start start 2024-03-21T10:29:42.978 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.974+0000 7f7b6b409740 1 Processor -- start 2024-03-21T10:29:42.978 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.974+0000 7f7b6b409740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55924eff06d0 0x55924eff9530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:29:42.978 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.975+0000 7f7b6b409740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55924eff1080 0x55924eff68c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:29:42.978 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.975+0000 7f7b6b409740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55924eff9c00 0x55924eff6ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:29:42.978 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.975+0000 7f7b6b409740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x55924effcb70 con 0x55924eff9c00 2024-03-21T10:29:42.978 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.975+0000 7f7b6b409740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55924ef100d0 con 0x55924eff06d0 2024-03-21T10:29:42.978 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.975+0000 7f7b6b409740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55924effccf0 con 0x55924eff1080 2024-03-21T10:29:42.978 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.975+0000 7f7b6b409740 10 client.0 _ll_register_callbacks cb 0x55924ef14e80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T10:29:42.978 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.975+0000 7f7b66800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55924eff06d0 0x55924eff9530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:29:42.978 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.975+0000 7f7b66800640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55924eff9c00 0x55924eff6ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:29:42.978 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.975+0000 7f7b66800640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55924eff9c00 0x55924eff6ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:55234/0 (socket says 172.21.15.47:55234) 2024-03-21T10:29:42.979 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.975+0000 7f7b66800640 1 -- 172.21.15.47:0/2941575400 learned_addr learned my addr 172.21.15.47:0/2941575400 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:29:42.979 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.975+0000 7f7b65400640 1 --2- 172.21.15.47:0/2941575400 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55924eff1080 0x55924eff68c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:29:42.979 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.975+0000 7f7b66800640 1 -- 172.21.15.47:0/2941575400 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55924eff1080 msgr2=0x55924eff68c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:29:42.979 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.975+0000 7f7b66800640 1 --2- 172.21.15.47:0/2941575400 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55924eff1080 0x55924eff68c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:29:42.979 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.975+0000 7f7b66800640 1 -- 172.21.15.47:0/2941575400 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55924eff06d0 msgr2=0x55924eff9530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:29:42.979 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.975+0000 7f7b66800640 1 --2- 172.21.15.47:0/2941575400 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55924eff06d0 0x55924eff9530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:29:42.979 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.975+0000 7f7b66800640 1 -- 172.21.15.47:0/2941575400 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55924ec683f0 con 0x55924eff9c00 2024-03-21T10:29:42.979 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.975+0000 7f7b65400640 1 --2- 172.21.15.47:0/2941575400 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55924eff1080 0x55924eff68c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T10:29:42.979 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.976+0000 7f7b66800640 1 --2- 172.21.15.47:0/2941575400 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55924eff9c00 0x55924eff6ed0 secure :-1 s=READY pgs=830 cs=0 l=1 rev1=1 crypto rx=0x7f7b5c0048f0 tx=0x7f7b5c005fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=64e367ce87dc5cec server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:29:42.979 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.976+0000 7f7b43400640 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-03-21T10:29:42.979 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.976+0000 7f7b43400640 1 -- 172.21.15.47:0/2941575400 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b5c004530 con 0x55924eff9c00 2024-03-21T10:29:42.979 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.976+0000 7f7b43400640 1 -- 172.21.15.47:0/2941575400 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(97..97 src has 1..97) v4 ==== 9474+0+0 (secure 0 0 0) 0x7f7b5c004a40 con 0x55924eff9c00 2024-03-21T10:29:42.979 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.976+0000 7f7b6b409740 1 -- 172.21.15.47:0/2941575400 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f7b5c003d10 con 0x55924eff9c00 2024-03-21T10:29:42.979 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.976+0000 7f7b43400640 1 -- 172.21.15.47:0/2941575400 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7b5c0115e0 con 0x55924eff9c00 2024-03-21T10:29:42.979 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.976+0000 7f7b6b409740 10 client.17076 fetch_fsmap learned FSMap version 32 2024-03-21T10:29:42.980 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.976+0000 7f7b6b409740 1 -- 172.21.15.47:0/2941575400 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x55924ed327d0 con 0x55924eff9c00 2024-03-21T10:29:42.980 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.977+0000 7f7b43400640 1 -- 172.21.15.47:0/2941575400 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f7b5c003d10 con 0x55924eff9c00 2024-03-21T10:29:42.980 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.977+0000 7f7b43400640 1 -- 172.21.15.47:0/2941575400 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 32) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f7b5c029040 con 0x55924eff9c00 2024-03-21T10:29:42.980 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.977+0000 7f7b6b409740 10 client.17076 fetch_fsmap finished waiting for FSMap version 32 2024-03-21T10:29:42.980 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.977+0000 7f7b6b409740 10 client.17076 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-21T10:29:42.980 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.977+0000 7f7b6b409740 20 client.17076 populate_metadata read hostname 'smithi047' 2024-03-21T10:29:42.980 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.977+0000 7f7b6b409740 1 --2- 172.21.15.47:0/2941575400 >> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] conn(0x55924f009970 0x55924f00bd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:29:42.980 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.977+0000 7f7b6b409740 4 client.17076 mds_command: new command op to 34535 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T10:29:42.980 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.977+0000 7f7b6b409740 1 -- 172.21.15.47:0/2941575400 --> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55924ef61800 con 0x55924f009970 2024-03-21T10:29:42.980 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.978+0000 7f7b65400640 1 --2- 172.21.15.47:0/2941575400 >> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] conn(0x55924f009970 0x55924f00bd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:29:42.980 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.978+0000 7f7b65400640 1 --2- 172.21.15.47:0/2941575400 >> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] conn(0x55924f009970 0x55924f00bd60 crc :-1 s=READY pgs=131 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f9760f4185f2bf74 server_cookie=1654f65e6ea1eb60 in_seq=0 out_seq=0 2024-03-21T10:29:42.980 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.979+0000 7f7b43400640 10 client.17076 ms_handle_connect on v2:172.21.15.100:6838/4173168440 2024-03-21T10:29:42.981 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.980+0000 7f7b43400640 1 -- 172.21.15.47:0/2941575400 <== mds.0 v2:172.21.15.100:6838/4173168440 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55924ef61800 con 0x55924f009970 2024-03-21T10:29:42.981 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.980+0000 7f7b43400640 10 client.17076 handle_command_reply: tid=0 2024-03-21T10:29:42.981 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.980+0000 7f7b6b409740 1 client.17076 shutdown 2024-03-21T10:29:42.982 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.980+0000 7f7b6b409740 1 -- 172.21.15.47:0/2941575400 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55924eff9c00 msgr2=0x55924eff6ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:29:42.982 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.980+0000 7f7b6b409740 1 --2- 172.21.15.47:0/2941575400 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55924eff9c00 0x55924eff6ed0 secure :-1 s=READY pgs=830 cs=0 l=1 rev1=1 crypto rx=0x7f7b5c0048f0 tx=0x7f7b5c005fd0 comp rx=0 tx=0).stop 2024-03-21T10:29:42.982 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.980+0000 7f7b6b409740 1 -- 172.21.15.47:0/2941575400 shutdown_connections 2024-03-21T10:29:42.982 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.980+0000 7f7b6b409740 1 --2- 172.21.15.47:0/2941575400 >> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] conn(0x55924f009970 0x55924f00bd60 crc :-1 s=READY pgs=131 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:29:42.982 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.980+0000 7f7b6b409740 1 --2- 172.21.15.47:0/2941575400 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55924eff9c00 0x55924eff6ed0 unknown :-1 s=CLOSED pgs=830 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:29:42.982 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.980+0000 7f7b6b409740 1 --2- 172.21.15.47:0/2941575400 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55924eff1080 0x55924eff68c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:29:42.982 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.980+0000 7f7b6b409740 1 --2- 172.21.15.47:0/2941575400 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55924eff06d0 0x55924eff9530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:29:42.982 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.980+0000 7f7b43400640 0 client.17076 ms_handle_reset on v2:172.21.15.100:6838/4173168440 2024-03-21T10:29:42.982 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.980+0000 7f7b6b409740 1 -- 172.21.15.47:0/2941575400 >> 172.21.15.47:0/2941575400 conn(0x55924efeada0 msgr2=0x55924efecaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:29:42.982 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.981+0000 7f7b6b409740 1 -- 172.21.15.47:0/2941575400 shutdown_connections 2024-03-21T10:29:42.982 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.981+0000 7f7b6b409740 1 -- 172.21.15.47:0/2941575400 wait complete. 2024-03-21T10:29:42.982 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.981+0000 7f7b6b409740 20 client.17076 trim_cache size 0 max 16384 2024-03-21T10:29:42.994 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.995+0000 7f7b6b409740 1 Processor -- start 2024-03-21T10:29:42.994 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.995+0000 7f7b6b409740 1 -- start start 2024-03-21T10:29:42.994 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.995+0000 7f7b6b409740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55924eff06d0 0x55924eff5060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:29:42.994 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.995+0000 7f7b6b409740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55924eff1080 0x55924eff8ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:29:42.995 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.995+0000 7f7b6b409740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55924eff9c00 0x55924eff9530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:29:42.995 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.995+0000 7f7b6b409740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b601b0b30 con 0x55924eff06d0 2024-03-21T10:29:42.995 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.995+0000 7f7b6b409740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b601b24b0 con 0x55924eff9c00 2024-03-21T10:29:42.995 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.995+0000 7f7b6b409740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b601b1190 con 0x55924eff1080 2024-03-21T10:29:42.996 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.995+0000 7f7b66800640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55924eff06d0 0x55924eff5060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:29:42.996 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.996+0000 7f7b66800640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55924eff06d0 0x55924eff5060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:55240/0 (socket says 172.21.15.47:55240) 2024-03-21T10:29:42.996 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.996+0000 7f7b66800640 1 -- 172.21.15.47:0/3459956468 learned_addr learned my addr 172.21.15.47:0/3459956468 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:29:42.996 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.996+0000 7f7b66800640 1 --2- 172.21.15.47:0/3459956468 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55924eff9c00 0x55924eff9530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:29:42.996 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.996+0000 7f7b65400640 1 --2- 172.21.15.47:0/3459956468 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55924eff1080 0x55924eff8ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:29:42.996 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.996+0000 7f7b66800640 1 -- 172.21.15.47:0/3459956468 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55924eff1080 msgr2=0x55924eff8ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:29:42.996 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.996+0000 7f7b66800640 1 --2- 172.21.15.47:0/3459956468 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55924eff1080 0x55924eff8ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:29:42.997 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.996+0000 7f7b66800640 1 -- 172.21.15.47:0/3459956468 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55924eff9c00 msgr2=0x55924eff9530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:29:42.997 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.996+0000 7f7b66800640 1 --2- 172.21.15.47:0/3459956468 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55924eff9c00 0x55924eff9530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:29:42.997 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.996+0000 7f7b66800640 1 -- 172.21.15.47:0/3459956468 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55924eff02b0 con 0x55924eff06d0 2024-03-21T10:29:42.997 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.996+0000 7f7b65400640 1 --2- 172.21.15.47:0/3459956468 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55924eff1080 0x55924eff8ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T10:29:42.997 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.996+0000 7f7b66800640 1 --2- 172.21.15.47:0/3459956468 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55924eff06d0 0x55924eff5060 secure :-1 s=READY pgs=831 cs=0 l=1 rev1=1 crypto rx=0x7f7b5c00a010 tx=0x7f7b5c01b9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=990e3b13065a3508 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:29:42.997 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.997+0000 7f7b4e600640 1 -- 172.21.15.47:0/3459956468 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b5c003cb0 con 0x55924eff06d0 2024-03-21T10:29:42.997 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.997+0000 7f7b4e600640 1 -- 172.21.15.47:0/3459956468 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f7b5c004dd0 con 0x55924eff06d0 2024-03-21T10:29:42.997 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.997+0000 7f7b4e600640 1 -- 172.21.15.47:0/3459956468 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b5c0098e0 con 0x55924eff06d0 2024-03-21T10:29:43.000 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.997+0000 7f7b6b409740 1 -- 172.21.15.47:0/3459956468 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55924eff06d0 msgr2=0x55924eff5060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:29:43.000 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.997+0000 7f7b6b409740 1 --2- 172.21.15.47:0/3459956468 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55924eff06d0 0x55924eff5060 secure :-1 s=READY pgs=831 cs=0 l=1 rev1=1 crypto rx=0x7f7b5c00a010 tx=0x7f7b5c01b9f0 comp rx=0 tx=0).stop 2024-03-21T10:29:43.001 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.998+0000 7f7b6b409740 1 -- 172.21.15.47:0/3459956468 shutdown_connections 2024-03-21T10:29:43.001 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.998+0000 7f7b6b409740 1 --2- 172.21.15.47:0/3459956468 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55924eff9c00 0x55924eff9530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:29:43.001 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.998+0000 7f7b6b409740 1 --2- 172.21.15.47:0/3459956468 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55924eff1080 0x55924eff8ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:29:43.001 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.998+0000 7f7b6b409740 1 --2- 172.21.15.47:0/3459956468 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55924eff06d0 0x55924eff5060 unknown :-1 s=CLOSED pgs=831 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:29:43.001 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.998+0000 7f7b6b409740 1 -- 172.21.15.47:0/3459956468 >> 172.21.15.47:0/3459956468 conn(0x55924efeada0 msgr2=0x55924efe9eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:29:43.001 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.998+0000 7f7b6b409740 1 -- 172.21.15.47:0/3459956468 shutdown_connections 2024-03-21T10:29:43.001 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.998+0000 7f7b6b409740 1 -- 172.21.15.47:0/3459956468 wait complete. 2024-03-21T10:29:43.001 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.998+0000 7f7b6b409740 1 -- start start 2024-03-21T10:29:43.001 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.998+0000 7f7b6b409740 1 Processor -- start 2024-03-21T10:29:43.001 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.999+0000 7f7b6b409740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55924eff06d0 0x55924f00cd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:29:43.001 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.999+0000 7f7b6b409740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55924eff1080 0x55924f00d2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:29:43.001 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.999+0000 7f7b6b409740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55924eff9c00 0x55924f00e220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:29:43.001 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.999+0000 7f7b6b409740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b601b1e50 con 0x55924eff9c00 2024-03-21T10:29:43.002 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.999+0000 7f7b6b409740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b601b17f0 con 0x55924eff06d0 2024-03-21T10:29:43.002 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.999+0000 7f7b6b409740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b601aecf0 con 0x55924eff1080 2024-03-21T10:29:43.002 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.999+0000 7f7b6b409740 10 client.0 _ll_register_callbacks cb 0x7f7b5c009e10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T10:29:43.002 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.999+0000 7f7b66800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55924eff06d0 0x55924f00cd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:29:43.002 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.999+0000 7f7b66800640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55924eff9c00 0x55924f00e220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:29:43.002 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.999+0000 7f7b66800640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55924eff9c00 0x55924f00e220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:55256/0 (socket says 172.21.15.47:55256) 2024-03-21T10:29:43.002 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.999+0000 7f7b66800640 1 -- 172.21.15.47:0/3113527758 learned_addr learned my addr 172.21.15.47:0/3113527758 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:29:43.002 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:42.999+0000 7f7b65400640 1 --2- 172.21.15.47:0/3113527758 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55924eff1080 0x55924f00d2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:29:43.002 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:43.000+0000 7f7b66800640 1 -- 172.21.15.47:0/3113527758 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55924eff1080 msgr2=0x55924f00d2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:29:43.002 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:43.000+0000 7f7b66800640 1 --2- 172.21.15.47:0/3113527758 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55924eff1080 0x55924f00d2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:29:43.002 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:43.000+0000 7f7b66800640 1 -- 172.21.15.47:0/3113527758 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55924eff06d0 msgr2=0x55924f00cd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:29:43.002 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:43.000+0000 7f7b66800640 1 --2- 172.21.15.47:0/3113527758 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55924eff06d0 0x55924f00cd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:29:43.002 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:43.000+0000 7f7b66800640 1 -- 172.21.15.47:0/3113527758 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55924f00e910 con 0x55924eff9c00 2024-03-21T10:29:43.003 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:43.000+0000 7f7b65400640 1 --2- 172.21.15.47:0/3113527758 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55924eff1080 0x55924f00d2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T10:29:43.003 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:43.000+0000 7f7b66800640 1 --2- 172.21.15.47:0/3113527758 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55924eff9c00 0x55924f00e220 secure :-1 s=READY pgs=832 cs=0 l=1 rev1=1 crypto rx=0x7f7b5c0115b0 tx=0x7f7b5c005cf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f8e671975ec12dbd server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:29:43.003 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:43.000+0000 7f7b43400640 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-03-21T10:29:43.003 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:43.000+0000 7f7b43400640 1 -- 172.21.15.47:0/3113527758 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b5c011780 con 0x55924eff9c00 2024-03-21T10:29:43.003 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:43.000+0000 7f7b43400640 1 -- 172.21.15.47:0/3113527758 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(97..97 src has 1..97) v4 ==== 9474+0+0 (secure 0 0 0) 0x7f7b5c011920 con 0x55924eff9c00 2024-03-21T10:29:43.003 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:43.000+0000 7f7b6b409740 1 -- 172.21.15.47:0/3113527758 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f7b5c004dd0 con 0x55924eff9c00 2024-03-21T10:29:43.003 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:43.000+0000 7f7b43400640 1 -- 172.21.15.47:0/3113527758 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7b5c011b20 con 0x55924eff9c00 2024-03-21T10:29:43.003 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:43.001+0000 7f7b6b409740 10 client.17082 fetch_fsmap learned FSMap version 32 2024-03-21T10:29:43.003 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:43.001+0000 7f7b6b409740 1 -- 172.21.15.47:0/3113527758 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x55924f06e210 con 0x55924eff9c00 2024-03-21T10:29:43.003 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:43.001+0000 7f7b43400640 1 -- 172.21.15.47:0/3113527758 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f7b5c004dd0 con 0x55924eff9c00 2024-03-21T10:29:43.003 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:43.001+0000 7f7b43400640 1 -- 172.21.15.47:0/3113527758 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 32) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f7b5c005160 con 0x55924eff9c00 2024-03-21T10:29:43.003 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:43.001+0000 7f7b6b409740 10 client.17082 fetch_fsmap finished waiting for FSMap version 32 2024-03-21T10:29:43.003 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:43.001+0000 7f7b6b409740 10 client.17082 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-21T10:29:43.004 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:43.001+0000 7f7b6b409740 20 client.17082 populate_metadata read hostname 'smithi047' 2024-03-21T10:29:43.004 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:43.001+0000 7f7b6b409740 1 --2- 172.21.15.47:0/3113527758 >> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] conn(0x55924f06e770 0x55924f070b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:29:43.004 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:43.001+0000 7f7b6b409740 4 client.17082 mds_command: new command op to 34535 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-21T10:29:43.004 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:43.001+0000 7f7b6b409740 1 -- 172.21.15.47:0/3113527758 --> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55924ef5bb50 con 0x55924f06e770 2024-03-21T10:29:43.004 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:43.002+0000 7f7b65400640 1 --2- 172.21.15.47:0/3113527758 >> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] conn(0x55924f06e770 0x55924f070b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:29:43.005 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:43.002+0000 7f7b65400640 1 --2- 172.21.15.47:0/3113527758 >> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] conn(0x55924f06e770 0x55924f070b60 crc :-1 s=READY pgs=132 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cf962a46590db14d server_cookie=d2556ee804762f4d in_seq=0 out_seq=0 2024-03-21T10:29:43.005 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:43.002+0000 7f7b43400640 10 client.17082 ms_handle_connect on v2:172.21.15.100:6838/4173168440 2024-03-21T10:29:43.005 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:43.003+0000 7f7b43400640 1 -- 172.21.15.47:0/3113527758 <== mds.0 v2:172.21.15.100:6838/4173168440 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55924ef5bb50 con 0x55924f06e770 2024-03-21T10:29:43.005 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:43.003+0000 7f7b43400640 10 client.17082 handle_command_reply: tid=0 2024-03-21T10:29:43.005 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:43.003+0000 7f7b6b409740 1 client.17082 shutdown 2024-03-21T10:29:43.006 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:43.003+0000 7f7b6b409740 1 -- 172.21.15.47:0/3113527758 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55924eff9c00 msgr2=0x55924f00e220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:29:43.006 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:43.003+0000 7f7b6b409740 1 --2- 172.21.15.47:0/3113527758 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55924eff9c00 0x55924f00e220 secure :-1 s=READY pgs=832 cs=0 l=1 rev1=1 crypto rx=0x7f7b5c0115b0 tx=0x7f7b5c005cf0 comp rx=0 tx=0).stop 2024-03-21T10:29:43.006 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:43.003+0000 7f7b6b409740 1 -- 172.21.15.47:0/3113527758 shutdown_connections 2024-03-21T10:29:43.006 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:43.003+0000 7f7b6b409740 1 --2- 172.21.15.47:0/3113527758 >> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] conn(0x55924f06e770 0x55924f070b60 crc :-1 s=READY pgs=132 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:29:43.006 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:43.003+0000 7f7b6b409740 1 --2- 172.21.15.47:0/3113527758 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55924eff9c00 0x55924f00e220 unknown :-1 s=CLOSED pgs=832 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:29:43.006 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:43.003+0000 7f7b6b409740 1 --2- 172.21.15.47:0/3113527758 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55924eff1080 0x55924f00d2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:29:43.006 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:43.003+0000 7f7b6b409740 1 --2- 172.21.15.47:0/3113527758 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55924eff06d0 0x55924f00cd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:29:43.006 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:43.003+0000 7f7b43400640 0 client.17082 ms_handle_reset on v2:172.21.15.100:6838/4173168440 2024-03-21T10:29:43.006 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:43.003+0000 7f7b6b409740 1 -- 172.21.15.47:0/3113527758 >> 172.21.15.47:0/3113527758 conn(0x55924efeada0 msgr2=0x55924efec080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:29:43.006 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:43.003+0000 7f7b6b409740 1 -- 172.21.15.47:0/3113527758 shutdown_connections 2024-03-21T10:29:43.006 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:43.003+0000 7f7b6b409740 1 -- 172.21.15.47:0/3113527758 wait complete. 2024-03-21T10:29:43.006 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:43.003+0000 7f7b6b409740 20 client.17082 trim_cache size 0 max 16384 2024-03-21T10:29:43.007 INFO:teuthology.orchestra.run.smithi047.stdout:[] 2024-03-21T10:29:43.007 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:43.004+0000 7f7b68800640 1 -- 172.21.15.47:0/4275128922 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f7b480a3090 msgr2=0x7f7b480a5550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:29:43.007 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:43.004+0000 7f7b68800640 1 --2- 172.21.15.47:0/4275128922 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f7b480a3090 0x7f7b480a5550 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x55924eff01a0 tx=0x7f7b54000bc0 comp rx=0 tx=0).stop 2024-03-21T10:29:43.008 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:43.004+0000 7f7b68800640 1 -- 172.21.15.47:0/4275128922 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7b60102910 msgr2=0x7f7b6019a100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:29:43.008 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:43.004+0000 7f7b68800640 1 --2- 172.21.15.47:0/4275128922 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7b60102910 0x7f7b6019a100 secure :-1 s=READY pgs=829 cs=0 l=1 rev1=1 crypto rx=0x7f7b54005940 tx=0x7f7b540045e0 comp rx=0 tx=0).stop 2024-03-21T10:29:43.008 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:43.004+0000 7f7b68800640 1 -- 172.21.15.47:0/4275128922 shutdown_connections 2024-03-21T10:29:43.008 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:43.004+0000 7f7b68800640 1 --2- 172.21.15.47:0/4275128922 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7b6010bce0 0x7f7b6019e970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:29:43.008 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:43.004+0000 7f7b68800640 1 --2- 172.21.15.47:0/4275128922 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f7b480a3090 0x7f7b480a5550 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:29:43.008 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:43.004+0000 7f7b68800640 1 --2- 172.21.15.47:0/4275128922 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7b60103230 0x7f7b6019a640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:29:43.008 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:43.004+0000 7f7b68800640 1 --2- 172.21.15.47:0/4275128922 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7b60102910 0x7f7b6019a100 unknown :-1 s=CLOSED pgs=829 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:29:43.008 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:43.004+0000 7f7b68800640 1 -- 172.21.15.47:0/4275128922 >> 172.21.15.47:0/4275128922 conn(0x7f7b60077700 msgr2=0x7f7b60107c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:29:43.008 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:43.004+0000 7f7b68800640 1 -- 172.21.15.47:0/4275128922 shutdown_connections 2024-03-21T10:29:43.008 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:43.004+0000 7f7b68800640 1 -- 172.21.15.47:0/4275128922 wait complete. 2024-03-21T10:29:43.962 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:29:43 smithi047 ceph-mon[31547]: pgmap v135: 129 pgs: 129 active+clean; 277 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-03-21T10:29:43.977 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:29:43 smithi100 ceph-mon[35193]: pgmap v135: 129 pgs: 129 active+clean; 277 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-03-21T10:29:43.988 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:29:43 smithi157 ceph-mon[35581]: pgmap v135: 129 pgs: 129 active+clean; 277 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-03-21T10:29:44.023 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-03-21T10:29:44.023 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-21T10:29:44.023 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-03-21T10:29:44.254 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:44.255+0000 7f9024600640 1 -- 172.21.15.47:0/596930956 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f901c1076f0 msgr2=0x7f901c107ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:29:44.254 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:44.255+0000 7f9024600640 1 --2- 172.21.15.47:0/596930956 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f901c1076f0 0x7f901c107ad0 secure :-1 s=READY pgs=833 cs=0 l=1 rev1=1 crypto rx=0x7f9010009930 tx=0x7f901002bc20 comp rx=0 tx=0).stop 2024-03-21T10:29:44.255 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:44.255+0000 7f9024600640 1 -- 172.21.15.47:0/596930956 shutdown_connections 2024-03-21T10:29:44.255 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:44.255+0000 7f9024600640 1 --2- 172.21.15.47:0/596930956 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f901c069260 0x7f901c10e500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:29:44.255 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:44.255+0000 7f9024600640 1 --2- 172.21.15.47:0/596930956 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f901c0688a0 0x7f901c068d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:29:44.255 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:44.255+0000 7f9024600640 1 --2- 172.21.15.47:0/596930956 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f901c1076f0 0x7f901c107ad0 unknown :-1 s=CLOSED pgs=833 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:29:44.255 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:44.255+0000 7f9024600640 1 -- 172.21.15.47:0/596930956 >> 172.21.15.47:0/596930956 conn(0x7f901c0776a0 msgr2=0x7f901c077ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:29:44.255 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:44.255+0000 7f9024600640 1 -- 172.21.15.47:0/596930956 shutdown_connections 2024-03-21T10:29:44.255 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:44.255+0000 7f9024600640 1 -- 172.21.15.47:0/596930956 wait complete. 2024-03-21T10:29:44.257 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:44.256+0000 7f9024600640 1 Processor -- start 2024-03-21T10:29:44.257 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:44.256+0000 7f9024600640 1 -- start start 2024-03-21T10:29:44.257 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:44.256+0000 7f9024600640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f901c0688a0 0x7f901c19a070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:29:44.258 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:44.256+0000 7f9024600640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f901c069260 0x7f901c19a5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:29:44.258 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:44.256+0000 7f9024600640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f901c1076f0 0x7f901c19e930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:29:44.258 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:44.256+0000 7f9024600640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f901c1118f0 con 0x7f901c1076f0 2024-03-21T10:29:44.258 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:44.256+0000 7f9024600640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f901c111770 con 0x7f901c0688a0 2024-03-21T10:29:44.258 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:44.256+0000 7f9024600640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f901c111a70 con 0x7f901c069260 2024-03-21T10:29:44.258 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:44.257+0000 7f9022600640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f901c1076f0 0x7f901c19e930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:29:44.258 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:44.257+0000 7f9022600640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f901c1076f0 0x7f901c19e930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:55286/0 (socket says 172.21.15.47:55286) 2024-03-21T10:29:44.258 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:44.257+0000 7f9022600640 1 -- 172.21.15.47:0/53495233 learned_addr learned my addr 172.21.15.47:0/53495233 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:29:44.258 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:44.257+0000 7f9021c00640 1 --2- 172.21.15.47:0/53495233 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f901c0688a0 0x7f901c19a070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:29:44.258 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:44.257+0000 7f9021200640 1 --2- 172.21.15.47:0/53495233 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f901c069260 0x7f901c19a5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:29:44.258 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:44.257+0000 7f9022600640 1 -- 172.21.15.47:0/53495233 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f901c069260 msgr2=0x7f901c19a5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:29:44.258 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:44.257+0000 7f9022600640 1 --2- 172.21.15.47:0/53495233 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f901c069260 0x7f901c19a5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:29:44.258 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:44.257+0000 7f9022600640 1 -- 172.21.15.47:0/53495233 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f901c0688a0 msgr2=0x7f901c19a070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:29:44.259 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:44.257+0000 7f9022600640 1 --2- 172.21.15.47:0/53495233 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f901c0688a0 0x7f901c19a070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:29:44.259 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:44.257+0000 7f9022600640 1 -- 172.21.15.47:0/53495233 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f901c19f060 con 0x7f901c1076f0 2024-03-21T10:29:44.259 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:44.258+0000 7f9022600640 1 --2- 172.21.15.47:0/53495233 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f901c1076f0 0x7f901c19e930 secure :-1 s=READY pgs=834 cs=0 l=1 rev1=1 crypto rx=0x7f901400eaa0 tx=0x7f901400ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:29:44.259 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:44.258+0000 7f901a000640 1 -- 172.21.15.47:0/53495233 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f901400cdb0 con 0x7f901c1076f0 2024-03-21T10:29:44.259 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:44.258+0000 7f901a000640 1 -- 172.21.15.47:0/53495233 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f9014005020 con 0x7f901c1076f0 2024-03-21T10:29:44.259 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:44.258+0000 7f9024600640 1 -- 172.21.15.47:0/53495233 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f901c19f350 con 0x7f901c1076f0 2024-03-21T10:29:44.259 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:44.258+0000 7f901a000640 1 -- 172.21.15.47:0/53495233 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90140116d0 con 0x7f901c1076f0 2024-03-21T10:29:44.259 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:44.258+0000 7f9024600640 1 -- 172.21.15.47:0/53495233 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f901c1a6be0 con 0x7f901c1076f0 2024-03-21T10:29:44.260 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:44.261+0000 7f90273c9740 1 Processor -- start 2024-03-21T10:29:44.261 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:44.261+0000 7f901a000640 1 -- 172.21.15.47:0/53495233 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 27) v1 ==== 137985+0+0 (secure 0 0 0) 0x7f90140051d0 con 0x7f901c1076f0 2024-03-21T10:29:44.261 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:44.261+0000 7f90273c9740 1 -- start start 2024-03-21T10:29:44.263 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:44.262+0000 7f90273c9740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55cbc9b36080 0x55cbc9b36460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:29:44.263 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:44.262+0000 7f90273c9740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55cbc9b369a0 0x55cbc9b3f040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:29:44.263 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:44.262+0000 7f90273c9740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55cbc9b3f580 0x55cbc9b41970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:29:44.263 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:44.262+0000 7f90273c9740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cbc9a9aeb0 con 0x55cbc9b36080 2024-03-21T10:29:44.263 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:44.262+0000 7f90273c9740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cbc9a43a60 con 0x55cbc9b3f580 2024-03-21T10:29:44.263 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:44.262+0000 7f90273c9740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cbc9a96f50 con 0x55cbc9b369a0 2024-03-21T10:29:44.263 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:44.262+0000 7f9021200640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55cbc9b36080 0x55cbc9b36460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:29:44.263 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:44.262+0000 7f9021200640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55cbc9b36080 0x55cbc9b36460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:55296/0 (socket says 172.21.15.47:55296) 2024-03-21T10:29:44.264 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:44.262+0000 7f9021200640 1 -- 172.21.15.47:0/4119112600 learned_addr learned my addr 172.21.15.47:0/4119112600 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:29:44.264 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:44.263+0000 7f9021c00640 1 --2- 172.21.15.47:0/4119112600 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55cbc9b369a0 0x55cbc9b3f040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:29:44.264 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:44.263+0000 7f9021200640 1 --2- 172.21.15.47:0/4119112600 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55cbc9b3f580 0x55cbc9b41970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:29:44.264 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:44.263+0000 7f9021200640 1 -- 172.21.15.47:0/4119112600 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55cbc9b369a0 msgr2=0x55cbc9b3f040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:29:44.264 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:44.263+0000 7f9021200640 1 --2- 172.21.15.47:0/4119112600 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55cbc9b369a0 0x55cbc9b3f040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:29:44.264 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:44.263+0000 7f9021200640 1 -- 172.21.15.47:0/4119112600 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55cbc9b3f580 msgr2=0x55cbc9b41970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:29:44.264 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:44.263+0000 7f9021200640 1 --2- 172.21.15.47:0/4119112600 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55cbc9b3f580 0x55cbc9b41970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:29:44.264 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:44.263+0000 7f9021200640 1 -- 172.21.15.47:0/4119112600 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55cbc98777e0 con 0x55cbc9b36080 2024-03-21T10:29:44.264 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:44.263+0000 7f901a000640 1 --2- 172.21.15.47:0/53495233 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f90040a3090 0x7f90040a5550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:29:44.264 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:44.263+0000 7f9021200640 1 --2- 172.21.15.47:0/4119112600 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55cbc9b36080 0x55cbc9b36460 secure :-1 s=READY pgs=835 cs=0 l=1 rev1=1 crypto rx=0x7f9008013580 tx=0x7f900800b6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=30b44122d1707260 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:29:44.264 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:44.263+0000 7f901a000640 1 -- 172.21.15.47:0/53495233 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 9474+0+0 (secure 0 0 0) 0x7f90140cfd90 con 0x7f901c1076f0 2024-03-21T10:29:44.264 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:44.263+0000 7f9021c00640 1 --2- 172.21.15.47:0/53495233 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f90040a3090 0x7f90040a5550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:29:44.264 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:44.264+0000 7f900ea00640 1 -- 172.21.15.47:0/4119112600 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9008019030 con 0x55cbc9b36080 2024-03-21T10:29:44.265 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:44.264+0000 7f900ea00640 1 -- 172.21.15.47:0/4119112600 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f90080118f0 con 0x55cbc9b36080 2024-03-21T10:29:44.265 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:44.264+0000 7f900ea00640 1 -- 172.21.15.47:0/4119112600 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f900800e750 con 0x55cbc9b36080 2024-03-21T10:29:44.268 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:44.264+0000 7f90273c9740 1 -- 172.21.15.47:0/4119112600 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55cbc9b36080 msgr2=0x55cbc9b36460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:29:44.269 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:44.264+0000 7f90273c9740 1 --2- 172.21.15.47:0/4119112600 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55cbc9b36080 0x55cbc9b36460 secure :-1 s=READY pgs=835 cs=0 l=1 rev1=1 crypto rx=0x7f9008013580 tx=0x7f900800b6a0 comp rx=0 tx=0).stop 2024-03-21T10:29:44.269 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:44.264+0000 7f9021c00640 1 --2- 172.21.15.47:0/53495233 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f90040a3090 0x7f90040a5550 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f9010005ab0 tx=0x7f9010005fd0 comp rx=0 tx=0).ready entity=mgr.15687 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:29:44.269 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:44.265+0000 7f90273c9740 1 -- 172.21.15.47:0/4119112600 shutdown_connections 2024-03-21T10:29:44.269 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:44.265+0000 7f90273c9740 1 --2- 172.21.15.47:0/4119112600 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55cbc9b3f580 0x55cbc9b41970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:29:44.269 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:44.265+0000 7f90273c9740 1 --2- 172.21.15.47:0/4119112600 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55cbc9b369a0 0x55cbc9b3f040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:29:44.269 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:44.265+0000 7f90273c9740 1 --2- 172.21.15.47:0/4119112600 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55cbc9b36080 0x55cbc9b36460 unknown :-1 s=CLOSED pgs=835 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:29:44.269 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:44.265+0000 7f90273c9740 1 -- 172.21.15.47:0/4119112600 >> 172.21.15.47:0/4119112600 conn(0x55cbc9b307b0 msgr2=0x55cbc9b30b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:29:44.269 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:44.265+0000 7f90273c9740 1 -- 172.21.15.47:0/4119112600 shutdown_connections 2024-03-21T10:29:44.269 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:44.265+0000 7f90273c9740 1 -- 172.21.15.47:0/4119112600 wait complete. 2024-03-21T10:29:44.269 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:44.265+0000 7f90273c9740 1 -- start start 2024-03-21T10:29:44.269 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:44.265+0000 7f90273c9740 1 Processor -- start 2024-03-21T10:29:44.269 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:44.266+0000 7f90273c9740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55cbc9b36080 0x55cbc9b3ee90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:29:44.270 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:44.266+0000 7f90273c9740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55cbc9b369a0 0x55cbc9b3c220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:29:44.270 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:44.266+0000 7f90273c9740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55cbc9b3f580 0x55cbc9b3c760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:29:44.270 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:44.266+0000 7f90273c9740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cbc9b42370 con 0x55cbc9b369a0 2024-03-21T10:29:44.270 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:44.266+0000 7f90273c9740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cbc9a56eb0 con 0x55cbc9b3f580 2024-03-21T10:29:44.270 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:44.266+0000 7f90273c9740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cbc9b424f0 con 0x55cbc9b36080 2024-03-21T10:29:44.270 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:44.266+0000 7f90273c9740 10 client.0 _ll_register_callbacks cb 0x55cbc9a4b150 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T10:29:44.270 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:44.266+0000 7f9021200640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55cbc9b369a0 0x55cbc9b3c220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:29:44.270 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:44.266+0000 7f9021200640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55cbc9b369a0 0x55cbc9b3c220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:55308/0 (socket says 172.21.15.47:55308) 2024-03-21T10:29:44.270 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:44.266+0000 7f9021200640 1 -- 172.21.15.47:0/1555997519 learned_addr learned my addr 172.21.15.47:0/1555997519 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:29:44.270 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:44.266+0000 7f9022600640 1 --2- 172.21.15.47:0/1555997519 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55cbc9b3f580 0x55cbc9b3c760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:29:44.270 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:44.267+0000 7f9021c00640 1 --2- 172.21.15.47:0/1555997519 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55cbc9b36080 0x55cbc9b3ee90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:29:44.270 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:44.267+0000 7f9021200640 1 -- 172.21.15.47:0/1555997519 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55cbc9b36080 msgr2=0x55cbc9b3ee90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:29:44.271 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:44.267+0000 7f9021200640 1 --2- 172.21.15.47:0/1555997519 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55cbc9b36080 0x55cbc9b3ee90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:29:44.271 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:44.267+0000 7f9021200640 1 -- 172.21.15.47:0/1555997519 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55cbc9b3f580 msgr2=0x55cbc9b3c760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:29:44.271 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:44.267+0000 7f9021200640 1 --2- 172.21.15.47:0/1555997519 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55cbc9b3f580 0x55cbc9b3c760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:29:44.271 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:44.267+0000 7f9021200640 1 -- 172.21.15.47:0/1555997519 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55cbc98d7f70 con 0x55cbc9b369a0 2024-03-21T10:29:44.271 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:44.267+0000 7f9022600640 1 --2- 172.21.15.47:0/1555997519 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55cbc9b3f580 0x55cbc9b3c760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T10:29:44.271 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:44.267+0000 7f9021200640 1 --2- 172.21.15.47:0/1555997519 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55cbc9b369a0 0x55cbc9b3c220 secure :-1 s=READY pgs=836 cs=0 l=1 rev1=1 crypto rx=0x7f9008005cd0 tx=0x7f9008011440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9bd224f513aabad8 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:29:44.271 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:44.267+0000 7f9003400640 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-03-21T10:29:44.271 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:44.267+0000 7f9003400640 1 -- 172.21.15.47:0/1555997519 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9008019040 con 0x55cbc9b369a0 2024-03-21T10:29:44.271 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:44.267+0000 7f90273c9740 1 -- 172.21.15.47:0/1555997519 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f90080118f0 con 0x55cbc9b369a0 2024-03-21T10:29:44.271 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:44.267+0000 7f9003400640 1 -- 172.21.15.47:0/1555997519 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(97..97 src has 1..97) v4 ==== 9474+0+0 (secure 0 0 0) 0x7f9008011cb0 con 0x55cbc9b369a0 2024-03-21T10:29:44.271 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:44.268+0000 7f9003400640 1 -- 172.21.15.47:0/1555997519 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9008031050 con 0x55cbc9b369a0 2024-03-21T10:29:44.271 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:44.268+0000 7f90273c9740 10 client.17094 fetch_fsmap learned FSMap version 32 2024-03-21T10:29:44.272 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:44.268+0000 7f90273c9740 1 -- 172.21.15.47:0/1555997519 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x55cbc976f170 con 0x55cbc9b369a0 2024-03-21T10:29:44.272 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:44.268+0000 7f9003400640 1 -- 172.21.15.47:0/1555997519 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f90080118f0 con 0x55cbc9b369a0 2024-03-21T10:29:44.272 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:44.268+0000 7f9003400640 1 -- 172.21.15.47:0/1555997519 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 32) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f9008004ad0 con 0x55cbc9b369a0 2024-03-21T10:29:44.272 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:44.268+0000 7f90273c9740 10 client.17094 fetch_fsmap finished waiting for FSMap version 32 2024-03-21T10:29:44.272 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:44.268+0000 7f90273c9740 10 client.17094 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-21T10:29:44.272 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:44.268+0000 7f90273c9740 20 client.17094 populate_metadata read hostname 'smithi047' 2024-03-21T10:29:44.272 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:44.268+0000 7f90273c9740 1 --2- 172.21.15.47:0/1555997519 >> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] conn(0x55cbc9b4f4c0 0x55cbc9b518b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:29:44.272 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:44.268+0000 7f90273c9740 4 client.17094 mds_command: new command op to 34535 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T10:29:44.272 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:44.268+0000 7f90273c9740 1 -- 172.21.15.47:0/1555997519 --> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55cbc9aa83f0 con 0x55cbc9b4f4c0 2024-03-21T10:29:44.272 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:44.269+0000 7f9021c00640 1 --2- 172.21.15.47:0/1555997519 >> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] conn(0x55cbc9b4f4c0 0x55cbc9b518b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:29:44.273 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:44.270+0000 7f9021c00640 1 --2- 172.21.15.47:0/1555997519 >> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] conn(0x55cbc9b4f4c0 0x55cbc9b518b0 crc :-1 s=READY pgs=133 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=30f608570d63a3ab server_cookie=3b4b284219546ad8 in_seq=0 out_seq=0 2024-03-21T10:29:44.274 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:44.270+0000 7f9003400640 10 client.17094 ms_handle_connect on v2:172.21.15.100:6838/4173168440 2024-03-21T10:29:44.274 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:44.271+0000 7f9003400640 1 -- 172.21.15.47:0/1555997519 <== mds.0 v2:172.21.15.100:6838/4173168440 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55cbc9aa83f0 con 0x55cbc9b4f4c0 2024-03-21T10:29:44.274 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:44.271+0000 7f9003400640 10 client.17094 handle_command_reply: tid=0 2024-03-21T10:29:44.274 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:44.272+0000 7f90273c9740 1 client.17094 shutdown 2024-03-21T10:29:44.274 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:44.272+0000 7f90273c9740 1 -- 172.21.15.47:0/1555997519 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55cbc9b369a0 msgr2=0x55cbc9b3c220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:29:44.274 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:44.272+0000 7f90273c9740 1 --2- 172.21.15.47:0/1555997519 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55cbc9b369a0 0x55cbc9b3c220 secure :-1 s=READY pgs=836 cs=0 l=1 rev1=1 crypto rx=0x7f9008005cd0 tx=0x7f9008011440 comp rx=0 tx=0).stop 2024-03-21T10:29:44.274 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:44.272+0000 7f90273c9740 1 -- 172.21.15.47:0/1555997519 shutdown_connections 2024-03-21T10:29:44.274 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:44.272+0000 7f90273c9740 1 --2- 172.21.15.47:0/1555997519 >> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] conn(0x55cbc9b4f4c0 0x55cbc9b518b0 crc :-1 s=READY pgs=133 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:29:44.274 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:44.272+0000 7f90273c9740 1 --2- 172.21.15.47:0/1555997519 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55cbc9b3f580 0x55cbc9b3c760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:29:44.274 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:44.272+0000 7f90273c9740 1 --2- 172.21.15.47:0/1555997519 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55cbc9b369a0 0x55cbc9b3c220 unknown :-1 s=CLOSED pgs=836 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:29:44.274 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:44.272+0000 7f90273c9740 1 --2- 172.21.15.47:0/1555997519 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55cbc9b36080 0x55cbc9b3ee90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:29:44.274 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:44.272+0000 7f90273c9740 1 -- 172.21.15.47:0/1555997519 >> 172.21.15.47:0/1555997519 conn(0x55cbc9b307b0 msgr2=0x55cbc9b32500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:29:44.274 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:44.272+0000 7f9003400640 0 client.17094 ms_handle_reset on v2:172.21.15.100:6838/4173168440 2024-03-21T10:29:44.274 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:44.272+0000 7f90273c9740 1 -- 172.21.15.47:0/1555997519 shutdown_connections 2024-03-21T10:29:44.275 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:44.272+0000 7f90273c9740 1 -- 172.21.15.47:0/1555997519 wait complete. 2024-03-21T10:29:44.275 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:44.272+0000 7f90273c9740 20 client.17094 trim_cache size 0 max 16384 2024-03-21T10:29:44.300 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:44.301+0000 7f90273c9740 1 Processor -- start 2024-03-21T10:29:44.300 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:44.301+0000 7f90273c9740 1 -- start start 2024-03-21T10:29:44.302 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:44.301+0000 7f90273c9740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55cbc9b36080 0x55cbc9b3e950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:29:44.302 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:44.301+0000 7f90273c9740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55cbc9b369a0 0x55cbc9b3ee90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:29:44.302 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:44.301+0000 7f90273c9740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55cbc9b3f580 0x55cbc9b2f6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:29:44.302 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:44.301+0000 7f90273c9740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f901c1b0b50 con 0x55cbc9b3f580 2024-03-21T10:29:44.302 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:44.301+0000 7f90273c9740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f901c1b24d0 con 0x55cbc9b36080 2024-03-21T10:29:44.302 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:44.301+0000 7f90273c9740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f901c1b11b0 con 0x55cbc9b369a0 2024-03-21T10:29:44.302 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:44.302+0000 7f9021c00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55cbc9b36080 0x55cbc9b3e950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:29:44.302 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:44.302+0000 7f9022600640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55cbc9b3f580 0x55cbc9b2f6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:29:44.302 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:44.302+0000 7f9022600640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55cbc9b3f580 0x55cbc9b2f6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:55324/0 (socket says 172.21.15.47:55324) 2024-03-21T10:29:44.302 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:44.302+0000 7f9022600640 1 -- 172.21.15.47:0/2362646110 learned_addr learned my addr 172.21.15.47:0/2362646110 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:29:44.303 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:44.302+0000 7f9021c00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55cbc9b36080 0x55cbc9b3e950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:40548/0 (socket says 172.21.15.47:40548) 2024-03-21T10:29:44.303 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:44.302+0000 7f9021200640 1 --2- 172.21.15.47:0/2362646110 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55cbc9b369a0 0x55cbc9b3ee90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:29:44.303 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:44.302+0000 7f9022600640 1 -- 172.21.15.47:0/2362646110 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55cbc9b369a0 msgr2=0x55cbc9b3ee90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:29:44.303 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:44.302+0000 7f9022600640 1 --2- 172.21.15.47:0/2362646110 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55cbc9b369a0 0x55cbc9b3ee90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:29:44.303 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:44.302+0000 7f9022600640 1 -- 172.21.15.47:0/2362646110 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55cbc9b36080 msgr2=0x55cbc9b3e950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:29:44.303 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:44.302+0000 7f9022600640 1 --2- 172.21.15.47:0/2362646110 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55cbc9b36080 0x55cbc9b3e950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:29:44.303 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:44.302+0000 7f9022600640 1 -- 172.21.15.47:0/2362646110 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55cbc9b41ce0 con 0x55cbc9b3f580 2024-03-21T10:29:44.303 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:44.303+0000 7f9022600640 1 --2- 172.21.15.47:0/2362646110 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55cbc9b3f580 0x55cbc9b2f6b0 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 crypto rx=0x7f9014098790 tx=0x7f901409c630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a22e874eb0b2fbbd server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:29:44.307 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:44.303+0000 7f900e000640 1 -- 172.21.15.47:0/2362646110 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90140b4030 con 0x55cbc9b3f580 2024-03-21T10:29:44.307 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:44.303+0000 7f900e000640 1 -- 172.21.15.47:0/2362646110 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f90140a47d0 con 0x55cbc9b3f580 2024-03-21T10:29:44.307 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:44.303+0000 7f900e000640 1 -- 172.21.15.47:0/2362646110 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f901401f040 con 0x55cbc9b3f580 2024-03-21T10:29:44.307 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:44.304+0000 7f90273c9740 1 -- 172.21.15.47:0/2362646110 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55cbc9b3f580 msgr2=0x55cbc9b2f6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:29:44.307 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:44.304+0000 7f90273c9740 1 --2- 172.21.15.47:0/2362646110 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55cbc9b3f580 0x55cbc9b2f6b0 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 crypto rx=0x7f9014098790 tx=0x7f901409c630 comp rx=0 tx=0).stop 2024-03-21T10:29:44.307 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:44.304+0000 7f90273c9740 1 -- 172.21.15.47:0/2362646110 shutdown_connections 2024-03-21T10:29:44.307 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:44.304+0000 7f90273c9740 1 --2- 172.21.15.47:0/2362646110 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55cbc9b3f580 0x55cbc9b2f6b0 unknown :-1 s=CLOSED pgs=837 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:29:44.308 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:44.304+0000 7f90273c9740 1 --2- 172.21.15.47:0/2362646110 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55cbc9b369a0 0x55cbc9b3ee90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:29:44.308 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:44.304+0000 7f90273c9740 1 --2- 172.21.15.47:0/2362646110 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55cbc9b36080 0x55cbc9b3e950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:29:44.308 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:44.304+0000 7f90273c9740 1 -- 172.21.15.47:0/2362646110 >> 172.21.15.47:0/2362646110 conn(0x55cbc9b307b0 msgr2=0x55cbc9b2efa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:29:44.308 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:44.304+0000 7f90273c9740 1 -- 172.21.15.47:0/2362646110 shutdown_connections 2024-03-21T10:29:44.308 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:44.304+0000 7f90273c9740 1 -- 172.21.15.47:0/2362646110 wait complete. 2024-03-21T10:29:44.308 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:44.304+0000 7f90273c9740 1 -- start start 2024-03-21T10:29:44.308 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:44.304+0000 7f90273c9740 1 Processor -- start 2024-03-21T10:29:44.308 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:44.305+0000 7f90273c9740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55cbc9b36080 0x55cbc9b47150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:29:44.308 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:44.305+0000 7f90273c9740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55cbc9b369a0 0x55cbc9b47690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:29:44.308 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:44.305+0000 7f90273c9740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55cbc9b3f580 0x55cbc9b39410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:29:44.308 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:44.305+0000 7f90273c9740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f901c1b1e70 con 0x55cbc9b369a0 2024-03-21T10:29:44.308 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:44.305+0000 7f90273c9740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f901c1b1810 con 0x55cbc9b3f580 2024-03-21T10:29:44.308 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:44.305+0000 7f90273c9740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f901c1aed10 con 0x55cbc9b36080 2024-03-21T10:29:44.309 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:44.305+0000 7f90273c9740 10 client.0 _ll_register_callbacks cb 0x7f9010005b50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T10:29:44.309 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:44.305+0000 7f9021200640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55cbc9b369a0 0x55cbc9b47690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:29:44.309 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:44.305+0000 7f9021200640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55cbc9b369a0 0x55cbc9b47690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:55334/0 (socket says 172.21.15.47:55334) 2024-03-21T10:29:44.309 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:44.305+0000 7f9021200640 1 -- 172.21.15.47:0/1338609657 learned_addr learned my addr 172.21.15.47:0/1338609657 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:29:44.309 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:44.306+0000 7f9022600640 1 --2- 172.21.15.47:0/1338609657 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55cbc9b3f580 0x55cbc9b39410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:29:44.309 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:44.306+0000 7f9021c00640 1 --2- 172.21.15.47:0/1338609657 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55cbc9b36080 0x55cbc9b47150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:29:44.309 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:44.306+0000 7f9021200640 1 -- 172.21.15.47:0/1338609657 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55cbc9b36080 msgr2=0x55cbc9b47150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:29:44.309 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:44.306+0000 7f9021200640 1 --2- 172.21.15.47:0/1338609657 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55cbc9b36080 0x55cbc9b47150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:29:44.309 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:44.306+0000 7f9021200640 1 -- 172.21.15.47:0/1338609657 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55cbc9b3f580 msgr2=0x55cbc9b39410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:29:44.309 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:44.306+0000 7f9021200640 1 --2- 172.21.15.47:0/1338609657 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55cbc9b3f580 0x55cbc9b39410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:29:44.309 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:44.306+0000 7f9021200640 1 -- 172.21.15.47:0/1338609657 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55cbc9b39c40 con 0x55cbc9b369a0 2024-03-21T10:29:44.309 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:44.306+0000 7f9021200640 1 --2- 172.21.15.47:0/1338609657 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55cbc9b369a0 0x55cbc9b47690 secure :-1 s=READY pgs=838 cs=0 l=1 rev1=1 crypto rx=0x7f9008004e70 tx=0x7f90080114b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ab9d3cc89190e8ae server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:29:44.310 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:44.306+0000 7f9003400640 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-03-21T10:29:44.310 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:44.306+0000 7f9003400640 1 -- 172.21.15.47:0/1338609657 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9008039020 con 0x55cbc9b369a0 2024-03-21T10:29:44.310 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:44.307+0000 7f90273c9740 1 -- 172.21.15.47:0/1338609657 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f90140a47d0 con 0x55cbc9b369a0 2024-03-21T10:29:44.310 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:44.307+0000 7f9003400640 1 -- 172.21.15.47:0/1338609657 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(97..97 src has 1..97) v4 ==== 9474+0+0 (secure 0 0 0) 0x7f90080116f0 con 0x55cbc9b369a0 2024-03-21T10:29:44.310 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:44.307+0000 7f9003400640 1 -- 172.21.15.47:0/1338609657 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9008044050 con 0x55cbc9b369a0 2024-03-21T10:29:44.310 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:44.307+0000 7f90273c9740 10 client.17100 fetch_fsmap learned FSMap version 32 2024-03-21T10:29:44.310 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:44.307+0000 7f90273c9740 1 -- 172.21.15.47:0/1338609657 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x55cbc9bab6c0 con 0x55cbc9b369a0 2024-03-21T10:29:44.310 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:44.307+0000 7f9003400640 1 -- 172.21.15.47:0/1338609657 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f90140a47d0 con 0x55cbc9b369a0 2024-03-21T10:29:44.310 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:44.307+0000 7f9003400640 1 -- 172.21.15.47:0/1338609657 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 32) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f9008013040 con 0x55cbc9b369a0 2024-03-21T10:29:44.310 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:44.307+0000 7f90273c9740 10 client.17100 fetch_fsmap finished waiting for FSMap version 32 2024-03-21T10:29:44.310 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:44.307+0000 7f90273c9740 10 client.17100 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-21T10:29:44.310 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:44.307+0000 7f90273c9740 20 client.17100 populate_metadata read hostname 'smithi047' 2024-03-21T10:29:44.311 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:44.307+0000 7f90273c9740 1 --2- 172.21.15.47:0/1338609657 >> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] conn(0x55cbc9babd50 0x55cbc9bae170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:29:44.311 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:44.307+0000 7f90273c9740 4 client.17100 mds_command: new command op to 34535 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-03-21T10:29:44.311 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:44.307+0000 7f90273c9740 1 -- 172.21.15.47:0/1338609657 --> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x55cbc9aa2740 con 0x55cbc9babd50 2024-03-21T10:29:44.311 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:44.308+0000 7f9021c00640 1 --2- 172.21.15.47:0/1338609657 >> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] conn(0x55cbc9babd50 0x55cbc9bae170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:29:44.312 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:44.309+0000 7f9021c00640 1 --2- 172.21.15.47:0/1338609657 >> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] conn(0x55cbc9babd50 0x55cbc9bae170 crc :-1 s=READY pgs=134 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6019e5c808136ea2 server_cookie=6d06eef2104a6ba in_seq=0 out_seq=0 2024-03-21T10:29:44.312 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:44.309+0000 7f9003400640 10 client.17100 ms_handle_connect on v2:172.21.15.100:6838/4173168440 2024-03-21T10:29:44.313 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:44.311+0000 7f9003400640 1 -- 172.21.15.47:0/1338609657 <== mds.0 v2:172.21.15.100:6838/4173168440 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x55cbc9aa2740 con 0x55cbc9babd50 2024-03-21T10:29:44.313 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:44.311+0000 7f9003400640 10 client.17100 handle_command_reply: tid=0 2024-03-21T10:29:44.313 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:44.311+0000 7f90273c9740 1 client.17100 shutdown 2024-03-21T10:29:44.313 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:44.311+0000 7f90273c9740 1 -- 172.21.15.47:0/1338609657 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55cbc9b369a0 msgr2=0x55cbc9b47690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:29:44.313 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:44.311+0000 7f90273c9740 1 --2- 172.21.15.47:0/1338609657 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55cbc9b369a0 0x55cbc9b47690 secure :-1 s=READY pgs=838 cs=0 l=1 rev1=1 crypto rx=0x7f9008004e70 tx=0x7f90080114b0 comp rx=0 tx=0).stop 2024-03-21T10:29:44.313 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:44.311+0000 7f90273c9740 1 -- 172.21.15.47:0/1338609657 shutdown_connections 2024-03-21T10:29:44.313 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:44.311+0000 7f90273c9740 1 --2- 172.21.15.47:0/1338609657 >> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] conn(0x55cbc9babd50 0x55cbc9bae170 crc :-1 s=READY pgs=134 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:29:44.313 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:44.311+0000 7f90273c9740 1 --2- 172.21.15.47:0/1338609657 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55cbc9b3f580 0x55cbc9b39410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:29:44.313 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:44.311+0000 7f90273c9740 1 --2- 172.21.15.47:0/1338609657 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55cbc9b369a0 0x55cbc9b47690 unknown :-1 s=CLOSED pgs=838 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:29:44.313 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:44.311+0000 7f90273c9740 1 --2- 172.21.15.47:0/1338609657 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55cbc9b36080 0x55cbc9b47150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:29:44.313 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:44.311+0000 7f90273c9740 1 -- 172.21.15.47:0/1338609657 >> 172.21.15.47:0/1338609657 conn(0x55cbc9b307b0 msgr2=0x55cbc9b2f830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:29:44.313 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:44.311+0000 7f9003400640 0 client.17100 ms_handle_reset on v2:172.21.15.100:6838/4173168440 2024-03-21T10:29:44.313 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:44.312+0000 7f90273c9740 1 -- 172.21.15.47:0/1338609657 shutdown_connections 2024-03-21T10:29:44.313 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:44.312+0000 7f90273c9740 1 -- 172.21.15.47:0/1338609657 wait complete. 2024-03-21T10:29:44.314 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:44.312+0000 7f90273c9740 20 client.17100 trim_cache size 0 max 16384 2024-03-21T10:29:44.314 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-03-21T10:29:44.314 INFO:teuthology.orchestra.run.smithi047.stdout: "return_code": 0, 2024-03-21T10:29:44.314 INFO:teuthology.orchestra.run.smithi047.stdout: "scrub_tag": "a7eff57c-820c-4374-b2aa-54e9fbfd6dc7", 2024-03-21T10:29:44.314 INFO:teuthology.orchestra.run.smithi047.stdout: "mode": "asynchronous" 2024-03-21T10:29:44.314 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-03-21T10:29:44.315 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:44.312+0000 7f9024600640 1 -- 172.21.15.47:0/53495233 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f90040a3090 msgr2=0x7f90040a5550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:29:44.315 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:44.312+0000 7f9024600640 1 --2- 172.21.15.47:0/53495233 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f90040a3090 0x7f90040a5550 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f9010005ab0 tx=0x7f9010005fd0 comp rx=0 tx=0).stop 2024-03-21T10:29:44.315 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:44.312+0000 7f9024600640 1 -- 172.21.15.47:0/53495233 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f901c1076f0 msgr2=0x7f901c19e930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:29:44.315 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:44.312+0000 7f9024600640 1 --2- 172.21.15.47:0/53495233 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f901c1076f0 0x7f901c19e930 secure :-1 s=READY pgs=834 cs=0 l=1 rev1=1 crypto rx=0x7f901400eaa0 tx=0x7f901400ef60 comp rx=0 tx=0).stop 2024-03-21T10:29:44.315 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:44.313+0000 7f9024600640 1 -- 172.21.15.47:0/53495233 shutdown_connections 2024-03-21T10:29:44.315 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:44.313+0000 7f9024600640 1 --2- 172.21.15.47:0/53495233 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f901c1076f0 0x7f901c19e930 unknown :-1 s=CLOSED pgs=834 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:29:44.315 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:44.313+0000 7f9024600640 1 --2- 172.21.15.47:0/53495233 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f901c069260 0x7f901c19a5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:29:44.315 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:44.313+0000 7f9024600640 1 --2- 172.21.15.47:0/53495233 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f90040a3090 0x7f90040a5550 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:29:44.316 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:44.313+0000 7f9024600640 1 --2- 172.21.15.47:0/53495233 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f901c0688a0 0x7f901c19a070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:29:44.316 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:44.313+0000 7f9024600640 1 -- 172.21.15.47:0/53495233 >> 172.21.15.47:0/53495233 conn(0x7f901c0776a0 msgr2=0x7f901c10cde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:29:44.316 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:44.313+0000 7f9024600640 1 -- 172.21.15.47:0/53495233 shutdown_connections 2024-03-21T10:29:44.316 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:44.313+0000 7f9024600640 1 -- 172.21.15.47:0/53495233 wait complete. 2024-03-21T10:29:44.333 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-21T10:29:44.333 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-21T10:29:44.561 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:44.562+0000 7ff7fbe00640 1 -- 172.21.15.47:0/2660217873 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff7fc1026d0 msgr2=0x7ff7fc102ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:29:44.562 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:44.562+0000 7ff7fbe00640 1 --2- 172.21.15.47:0/2660217873 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff7fc1026d0 0x7ff7fc102ab0 secure :-1 s=READY pgs=839 cs=0 l=1 rev1=1 crypto rx=0x7ff7ec009900 tx=0x7ff7ec02bcd0 comp rx=0 tx=0).stop 2024-03-21T10:29:44.562 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:44.562+0000 7ff7fbe00640 1 -- 172.21.15.47:0/2660217873 shutdown_connections 2024-03-21T10:29:44.562 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:44.562+0000 7ff7fbe00640 1 --2- 172.21.15.47:0/2660217873 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff7fc10baf0 0x7ff7fc10df90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:29:44.562 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:44.562+0000 7ff7fbe00640 1 --2- 172.21.15.47:0/2660217873 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff7fc102ff0 0x7ff7fc10b4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:29:44.562 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:44.562+0000 7ff7fbe00640 1 --2- 172.21.15.47:0/2660217873 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff7fc1026d0 0x7ff7fc102ab0 unknown :-1 s=CLOSED pgs=839 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:29:44.562 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:44.562+0000 7ff7fbe00640 1 -- 172.21.15.47:0/2660217873 >> 172.21.15.47:0/2660217873 conn(0x7ff7fc0fbef0 msgr2=0x7ff7fc0fe310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:29:44.562 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:44.562+0000 7ff7fbe00640 1 -- 172.21.15.47:0/2660217873 shutdown_connections 2024-03-21T10:29:44.562 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:44.562+0000 7ff7fbe00640 1 -- 172.21.15.47:0/2660217873 wait complete. 2024-03-21T10:29:44.564 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:44.563+0000 7ff7fbe00640 1 Processor -- start 2024-03-21T10:29:44.564 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:44.563+0000 7ff7fbe00640 1 -- start start 2024-03-21T10:29:44.564 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:44.564+0000 7ff7fbe00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff7fc1026d0 0x7ff7fc199fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:29:44.565 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:44.564+0000 7ff7fbe00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff7fc102ff0 0x7ff7fc19a4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:29:44.565 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:44.564+0000 7ff7fbe00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff7fc10baf0 0x7ff7fc19e870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:29:44.565 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:44.564+0000 7ff7fbe00640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7fc1117d0 con 0x7ff7fc102ff0 2024-03-21T10:29:44.565 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:44.564+0000 7ff7fbe00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7fc111650 con 0x7ff7fc10baf0 2024-03-21T10:29:44.565 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:44.564+0000 7ff7fbe00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7fc111950 con 0x7ff7fc1026d0 2024-03-21T10:29:44.565 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:44.564+0000 7ff7fa000640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff7fc102ff0 0x7ff7fc19a4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:29:44.565 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:44.564+0000 7ff7fa000640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff7fc102ff0 0x7ff7fc19a4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:55354/0 (socket says 172.21.15.47:55354) 2024-03-21T10:29:44.565 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:44.564+0000 7ff7fa000640 1 -- 172.21.15.47:0/3705466811 learned_addr learned my addr 172.21.15.47:0/3705466811 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:29:44.565 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:44.564+0000 7ff7fb400640 1 --2- 172.21.15.47:0/3705466811 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff7fc10baf0 0x7ff7fc19e870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:29:44.565 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:44.564+0000 7ff7faa00640 1 --2- 172.21.15.47:0/3705466811 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff7fc1026d0 0x7ff7fc199fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:29:44.565 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:44.564+0000 7ff7fa000640 1 -- 172.21.15.47:0/3705466811 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff7fc1026d0 msgr2=0x7ff7fc199fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:29:44.565 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:44.564+0000 7ff7fa000640 1 --2- 172.21.15.47:0/3705466811 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff7fc1026d0 0x7ff7fc199fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:29:44.565 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:44.564+0000 7ff7fa000640 1 -- 172.21.15.47:0/3705466811 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff7fc10baf0 msgr2=0x7ff7fc19e870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:29:44.565 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:44.564+0000 7ff7fa000640 1 --2- 172.21.15.47:0/3705466811 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff7fc10baf0 0x7ff7fc19e870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:29:44.566 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:44.564+0000 7ff7fa000640 1 -- 172.21.15.47:0/3705466811 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7fc19efa0 con 0x7ff7fc102ff0 2024-03-21T10:29:44.566 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:44.564+0000 7ff7fb400640 1 --2- 172.21.15.47:0/3705466811 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff7fc10baf0 0x7ff7fc19e870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T10:29:44.566 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:44.565+0000 7ff7fa000640 1 --2- 172.21.15.47:0/3705466811 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff7fc102ff0 0x7ff7fc19a4f0 secure :-1 s=READY pgs=840 cs=0 l=1 rev1=1 crypto rx=0x7ff7e800e9f0 tx=0x7ff7e800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:29:44.566 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:44.565+0000 7ff7f3400640 1 -- 172.21.15.47:0/3705466811 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7e800ccc0 con 0x7ff7fc102ff0 2024-03-21T10:29:44.566 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:44.565+0000 7ff7fbe00640 1 -- 172.21.15.47:0/3705466811 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff7fc19f290 con 0x7ff7fc102ff0 2024-03-21T10:29:44.566 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:44.565+0000 7ff7f3400640 1 -- 172.21.15.47:0/3705466811 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7ff7e80108e0 con 0x7ff7fc102ff0 2024-03-21T10:29:44.566 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:44.565+0000 7ff7fbe00640 1 -- 172.21.15.47:0/3705466811 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff7fc1a6b20 con 0x7ff7fc102ff0 2024-03-21T10:29:44.566 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:44.565+0000 7ff7f3400640 1 -- 172.21.15.47:0/3705466811 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7e8018760 con 0x7ff7fc102ff0 2024-03-21T10:29:44.569 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:44.569+0000 7ff804473740 1 Processor -- start 2024-03-21T10:29:44.569 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:44.569+0000 7ff804473740 1 -- start start 2024-03-21T10:29:44.569 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:44.569+0000 7ff7f3400640 1 -- 172.21.15.47:0/3705466811 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 27) v1 ==== 137985+0+0 (secure 0 0 0) 0x7ff7e8010430 con 0x7ff7fc102ff0 2024-03-21T10:29:44.569 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:44.570+0000 7ff804473740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b557b4edd0 0x55b557b4db20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:29:44.569 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:44.570+0000 7ff804473740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b557b4e0f0 0x55b557b4e570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:29:44.569 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:44.570+0000 7ff804473740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b557b57460 0x55b557b59850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:29:44.569 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:44.570+0000 7ff804473740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b557ab2250 con 0x55b557b57460 2024-03-21T10:29:44.569 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:44.570+0000 7ff804473740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b557a5aa30 con 0x55b557b4e0f0 2024-03-21T10:29:44.569 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:44.570+0000 7ff804473740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b557aae2f0 con 0x55b557b4edd0 2024-03-21T10:29:44.571 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:44.570+0000 7ff7fa000640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b557b57460 0x55b557b59850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:29:44.571 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:44.570+0000 7ff7fa000640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b557b57460 0x55b557b59850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:55366/0 (socket says 172.21.15.47:55366) 2024-03-21T10:29:44.571 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:44.570+0000 7ff7fa000640 1 -- 172.21.15.47:0/3883204476 learned_addr learned my addr 172.21.15.47:0/3883204476 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:29:44.571 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:44.570+0000 7ff7faa00640 1 --2- 172.21.15.47:0/3883204476 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b557b4e0f0 0x55b557b4e570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:29:44.571 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:44.570+0000 7ff7fb400640 1 --2- 172.21.15.47:0/3883204476 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b557b4edd0 0x55b557b4db20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:29:44.571 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:44.571+0000 7ff7fa000640 1 -- 172.21.15.47:0/3883204476 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b557b4edd0 msgr2=0x55b557b4db20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:29:44.571 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:44.571+0000 7ff7fa000640 1 --2- 172.21.15.47:0/3883204476 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b557b4edd0 0x55b557b4db20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:29:44.571 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:44.571+0000 7ff7fa000640 1 -- 172.21.15.47:0/3883204476 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b557b4e0f0 msgr2=0x55b557b4e570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:29:44.571 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:44.571+0000 7ff7fa000640 1 --2- 172.21.15.47:0/3883204476 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b557b4e0f0 0x55b557b4e570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:29:44.572 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:44.571+0000 7ff7fa000640 1 -- 172.21.15.47:0/3883204476 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b557a45690 con 0x55b557b57460 2024-03-21T10:29:44.572 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:44.571+0000 7ff7fa000640 1 --2- 172.21.15.47:0/3883204476 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b557b57460 0x55b557b59850 secure :-1 s=READY pgs=841 cs=0 l=1 rev1=1 crypto rx=0x7ff7e80cd2a0 tx=0x7ff7e80d86f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e9730ad659d06574 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:29:44.572 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:44.571+0000 7ff7e7e00640 1 -- 172.21.15.47:0/3883204476 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7e80e3070 con 0x55b557b57460 2024-03-21T10:29:44.572 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:44.572+0000 7ff7f3400640 1 --2- 172.21.15.47:0/3705466811 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7ff7e00a3090 0x7ff7e00a5550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:29:44.572 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:44.572+0000 7ff7e7e00640 1 -- 172.21.15.47:0/3883204476 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7ff7e80a49e0 con 0x55b557b57460 2024-03-21T10:29:44.572 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:44.572+0000 7ff7e7e00640 1 -- 172.21.15.47:0/3883204476 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7e809bb10 con 0x55b557b57460 2024-03-21T10:29:44.572 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:44.572+0000 7ff7f3400640 1 -- 172.21.15.47:0/3705466811 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 9474+0+0 (secure 0 0 0) 0x7ff7e80cffa0 con 0x7ff7fc102ff0 2024-03-21T10:29:44.572 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:44.572+0000 7ff7fb400640 1 --2- 172.21.15.47:0/3705466811 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7ff7e00a3090 0x7ff7e00a5550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:29:44.574 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:44.572+0000 7ff804473740 1 -- 172.21.15.47:0/3883204476 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b557b57460 msgr2=0x55b557b59850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:29:44.574 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:44.572+0000 7ff804473740 1 --2- 172.21.15.47:0/3883204476 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b557b57460 0x55b557b59850 secure :-1 s=READY pgs=841 cs=0 l=1 rev1=1 crypto rx=0x7ff7e80cd2a0 tx=0x7ff7e80d86f0 comp rx=0 tx=0).stop 2024-03-21T10:29:44.575 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:44.573+0000 7ff804473740 1 -- 172.21.15.47:0/3883204476 shutdown_connections 2024-03-21T10:29:44.575 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:44.573+0000 7ff804473740 1 --2- 172.21.15.47:0/3883204476 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b557b57460 0x55b557b59850 unknown :-1 s=CLOSED pgs=841 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:29:44.575 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:44.573+0000 7ff804473740 1 --2- 172.21.15.47:0/3883204476 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b557b4e0f0 0x55b557b4e570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:29:44.575 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:44.573+0000 7ff804473740 1 --2- 172.21.15.47:0/3883204476 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b557b4edd0 0x55b557b4db20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:29:44.575 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:44.573+0000 7ff804473740 1 -- 172.21.15.47:0/3883204476 >> 172.21.15.47:0/3883204476 conn(0x55b557b15200 msgr2=0x55b557b48b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:29:44.575 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:44.573+0000 7ff7fb400640 1 --2- 172.21.15.47:0/3705466811 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7ff7e00a3090 0x7ff7e00a5550 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7ff7f4005fd0 tx=0x7ff7f4009500 comp rx=0 tx=0).ready entity=mgr.15687 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:29:44.575 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:44.573+0000 7ff804473740 1 -- 172.21.15.47:0/3883204476 shutdown_connections 2024-03-21T10:29:44.575 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:44.573+0000 7ff804473740 1 -- 172.21.15.47:0/3883204476 wait complete. 2024-03-21T10:29:44.575 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:44.573+0000 7ff804473740 1 -- start start 2024-03-21T10:29:44.575 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:44.574+0000 7ff804473740 1 Processor -- start 2024-03-21T10:29:44.575 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:44.574+0000 7ff804473740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b557b4e0f0 0x55b557b56b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:29:44.575 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:44.574+0000 7ff804473740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b557b4edd0 0x55b557b53ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:29:44.575 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:44.574+0000 7ff804473740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b557b57460 0x55b557b54430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:29:44.576 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:44.574+0000 7ff804473740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b557b5a2d0 con 0x55b557b57460 2024-03-21T10:29:44.576 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:44.574+0000 7ff804473740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b557a6de80 con 0x55b557b4e0f0 2024-03-21T10:29:44.576 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:44.574+0000 7ff804473740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b557b5a450 con 0x55b557b4edd0 2024-03-21T10:29:44.576 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:44.575+0000 7ff804473740 10 client.0 _ll_register_callbacks cb 0x55b557a80130 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T10:29:44.576 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:44.575+0000 7ff7fb400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b557b57460 0x55b557b54430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:29:44.576 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:44.575+0000 7ff7fb400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b557b57460 0x55b557b54430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:55372/0 (socket says 172.21.15.47:55372) 2024-03-21T10:29:44.576 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:44.575+0000 7ff7fb400640 1 -- 172.21.15.47:0/4231314185 learned_addr learned my addr 172.21.15.47:0/4231314185 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:29:44.576 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:44.575+0000 7ff7faa00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b557b4e0f0 0x55b557b56b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:29:44.576 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:44.575+0000 7ff7fa000640 1 --2- 172.21.15.47:0/4231314185 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b557b4edd0 0x55b557b53ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:29:44.576 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:44.575+0000 7ff7fb400640 1 -- 172.21.15.47:0/4231314185 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b557b4edd0 msgr2=0x55b557b53ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:29:44.576 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:44.575+0000 7ff7fb400640 1 --2- 172.21.15.47:0/4231314185 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b557b4edd0 0x55b557b53ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:29:44.576 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:44.575+0000 7ff7fb400640 1 -- 172.21.15.47:0/4231314185 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b557b4e0f0 msgr2=0x55b557b56b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:29:44.576 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:44.575+0000 7ff7fb400640 1 --2- 172.21.15.47:0/4231314185 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b557b4e0f0 0x55b557b56b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:29:44.576 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:44.575+0000 7ff7fb400640 1 -- 172.21.15.47:0/4231314185 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55b5578907d0 con 0x55b557b57460 2024-03-21T10:29:44.578 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:44.576+0000 7ff7fb400640 1 --2- 172.21.15.47:0/4231314185 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b557b57460 0x55b557b54430 secure :-1 s=READY pgs=842 cs=0 l=1 rev1=1 crypto rx=0x7ff7f4010b00 tx=0x7ff7f4004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=470a619f58f5e46d server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:29:44.578 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:44.576+0000 7ff7e4c00640 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-03-21T10:29:44.578 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:44.576+0000 7ff7e4c00640 1 -- 172.21.15.47:0/4231314185 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7f402c060 con 0x55b557b57460 2024-03-21T10:29:44.579 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:44.576+0000 7ff804473740 1 -- 172.21.15.47:0/4231314185 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ff7e80a49e0 con 0x55b557b57460 2024-03-21T10:29:44.579 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:44.576+0000 7ff7e4c00640 1 -- 172.21.15.47:0/4231314185 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(97..97 src has 1..97) v4 ==== 9474+0+0 (secure 0 0 0) 0x7ff7f4017030 con 0x55b557b57460 2024-03-21T10:29:44.579 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:44.576+0000 7ff7e4c00640 1 -- 172.21.15.47:0/4231314185 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff7f4033050 con 0x55b557b57460 2024-03-21T10:29:44.579 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:44.576+0000 7ff804473740 10 client.17112 fetch_fsmap learned FSMap version 32 2024-03-21T10:29:44.579 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:44.576+0000 7ff804473740 1 -- 172.21.15.47:0/4231314185 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x55b5577c63f0 con 0x55b557b57460 2024-03-21T10:29:44.579 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:44.577+0000 7ff7e4c00640 1 -- 172.21.15.47:0/4231314185 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7ff7e80a49e0 con 0x55b557b57460 2024-03-21T10:29:44.579 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:44.577+0000 7ff7e4c00640 1 -- 172.21.15.47:0/4231314185 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 32) v1 ==== 7942+0+0 (secure 0 0 0) 0x7ff7f401c3f0 con 0x55b557b57460 2024-03-21T10:29:44.579 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:44.577+0000 7ff804473740 10 client.17112 fetch_fsmap finished waiting for FSMap version 32 2024-03-21T10:29:44.579 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:44.577+0000 7ff804473740 10 client.17112 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-21T10:29:44.579 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:44.577+0000 7ff804473740 20 client.17112 populate_metadata read hostname 'smithi047' 2024-03-21T10:29:44.579 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:44.577+0000 7ff804473740 1 --2- 172.21.15.47:0/4231314185 >> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] conn(0x55b557b67380 0x55b557b69790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:29:44.579 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:44.577+0000 7ff804473740 4 client.17112 mds_command: new command op to 34535 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T10:29:44.579 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:44.577+0000 7ff804473740 1 -- 172.21.15.47:0/4231314185 --> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55b557abf790 con 0x55b557b67380 2024-03-21T10:29:44.579 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:44.578+0000 7ff7faa00640 1 --2- 172.21.15.47:0/4231314185 >> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] conn(0x55b557b67380 0x55b557b69790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:29:44.580 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:44.578+0000 7ff7e4c00640 10 client.17112 ms_handle_connect on v2:172.21.15.100:6838/4173168440 2024-03-21T10:29:44.580 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:44.578+0000 7ff7faa00640 1 --2- 172.21.15.47:0/4231314185 >> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] conn(0x55b557b67380 0x55b557b69790 crc :-1 s=READY pgs=135 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f9f6bf1a85219fb2 server_cookie=73918d65e36fd046 in_seq=0 out_seq=0 2024-03-21T10:29:44.581 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:44.580+0000 7ff7e4c00640 1 -- 172.21.15.47:0/4231314185 <== mds.0 v2:172.21.15.100:6838/4173168440 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55b557abf790 con 0x55b557b67380 2024-03-21T10:29:44.581 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:44.580+0000 7ff7e4c00640 10 client.17112 handle_command_reply: tid=0 2024-03-21T10:29:44.581 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:44.580+0000 7ff804473740 1 client.17112 shutdown 2024-03-21T10:29:44.581 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:44.580+0000 7ff804473740 1 -- 172.21.15.47:0/4231314185 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b557b57460 msgr2=0x55b557b54430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:29:44.581 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:44.580+0000 7ff804473740 1 --2- 172.21.15.47:0/4231314185 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b557b57460 0x55b557b54430 secure :-1 s=READY pgs=842 cs=0 l=1 rev1=1 crypto rx=0x7ff7f4010b00 tx=0x7ff7f4004ab0 comp rx=0 tx=0).stop 2024-03-21T10:29:44.581 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:44.580+0000 7ff804473740 1 -- 172.21.15.47:0/4231314185 shutdown_connections 2024-03-21T10:29:44.581 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:44.580+0000 7ff804473740 1 --2- 172.21.15.47:0/4231314185 >> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] conn(0x55b557b67380 0x55b557b69790 crc :-1 s=READY pgs=135 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:29:44.582 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:44.580+0000 7ff804473740 1 --2- 172.21.15.47:0/4231314185 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b557b57460 0x55b557b54430 unknown :-1 s=CLOSED pgs=842 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:29:44.582 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:44.580+0000 7ff804473740 1 --2- 172.21.15.47:0/4231314185 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b557b4edd0 0x55b557b53ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:29:44.582 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:44.580+0000 7ff804473740 1 --2- 172.21.15.47:0/4231314185 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b557b4e0f0 0x55b557b56b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:29:44.582 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:44.580+0000 7ff804473740 1 -- 172.21.15.47:0/4231314185 >> 172.21.15.47:0/4231314185 conn(0x55b557b15200 msgr2=0x55b557b4a890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:29:44.582 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:44.580+0000 7ff7e4c00640 0 client.17112 ms_handle_reset on v2:172.21.15.100:6838/4173168440 2024-03-21T10:29:44.582 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:44.580+0000 7ff804473740 1 -- 172.21.15.47:0/4231314185 shutdown_connections 2024-03-21T10:29:44.582 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:44.580+0000 7ff804473740 1 -- 172.21.15.47:0/4231314185 wait complete. 2024-03-21T10:29:44.582 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:44.581+0000 7ff804473740 20 client.17112 trim_cache size 0 max 16384 2024-03-21T10:29:44.598 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:44.598+0000 7ff804473740 1 Processor -- start 2024-03-21T10:29:44.598 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:44.598+0000 7ff804473740 1 -- start start 2024-03-21T10:29:44.598 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:44.598+0000 7ff804473740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b557b4e0f0 0x55b557b52ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:29:44.598 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:44.598+0000 7ff804473740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b557b4edd0 0x55b557b56620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:29:44.598 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:44.598+0000 7ff804473740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b557b57460 0x55b557b56b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:29:44.598 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:44.599+0000 7ff804473740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7fc1b0a10 con 0x55b557b57460 2024-03-21T10:29:44.599 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:44.599+0000 7ff804473740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7fc1b2390 con 0x55b557b4e0f0 2024-03-21T10:29:44.599 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:44.599+0000 7ff804473740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7fc1b1070 con 0x55b557b4edd0 2024-03-21T10:29:44.599 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:44.599+0000 7ff7fb400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b557b57460 0x55b557b56b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:29:44.599 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:44.599+0000 7ff7fb400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b557b57460 0x55b557b56b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:55384/0 (socket says 172.21.15.47:55384) 2024-03-21T10:29:44.599 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:44.599+0000 7ff7fb400640 1 -- 172.21.15.47:0/781068637 learned_addr learned my addr 172.21.15.47:0/781068637 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:29:44.599 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:44.599+0000 7ff7faa00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b557b4e0f0 0x55b557b52ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:29:44.599 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:44.599+0000 7ff7fa000640 1 --2- 172.21.15.47:0/781068637 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b557b4edd0 0x55b557b56620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:29:44.602 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:44.599+0000 7ff7fb400640 1 -- 172.21.15.47:0/781068637 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b557b4edd0 msgr2=0x55b557b56620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:29:44.602 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:44.599+0000 7ff7fb400640 1 --2- 172.21.15.47:0/781068637 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b557b4edd0 0x55b557b56620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:29:44.602 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:44.599+0000 7ff7fb400640 1 -- 172.21.15.47:0/781068637 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b557b4e0f0 msgr2=0x55b557b52ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:29:44.602 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:44.599+0000 7ff7fb400640 1 --2- 172.21.15.47:0/781068637 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b557b4e0f0 0x55b557b52ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:29:44.602 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:44.599+0000 7ff7fb400640 1 -- 172.21.15.47:0/781068637 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b5577860f0 con 0x55b557b57460 2024-03-21T10:29:44.602 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:44.600+0000 7ff7fb400640 1 --2- 172.21.15.47:0/781068637 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b557b57460 0x55b557b56b60 secure :-1 s=READY pgs=843 cs=0 l=1 rev1=1 crypto rx=0x7ff7f4018040 tx=0x7ff7f4009af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4c93730a167eb409 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:29:44.602 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:44.600+0000 7ff7e7400640 1 -- 172.21.15.47:0/781068637 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7f403b020 con 0x55b557b57460 2024-03-21T10:29:44.602 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:44.600+0000 7ff7e7400640 1 -- 172.21.15.47:0/781068637 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7ff7f40108f0 con 0x55b557b57460 2024-03-21T10:29:44.602 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:44.600+0000 7ff7e7400640 1 -- 172.21.15.47:0/781068637 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7f4005400 con 0x55b557b57460 2024-03-21T10:29:44.603 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:44.601+0000 7ff804473740 1 -- 172.21.15.47:0/781068637 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b557b57460 msgr2=0x55b557b56b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:29:44.603 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:44.601+0000 7ff804473740 1 --2- 172.21.15.47:0/781068637 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b557b57460 0x55b557b56b60 secure :-1 s=READY pgs=843 cs=0 l=1 rev1=1 crypto rx=0x7ff7f4018040 tx=0x7ff7f4009af0 comp rx=0 tx=0).stop 2024-03-21T10:29:44.603 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:44.601+0000 7ff804473740 1 -- 172.21.15.47:0/781068637 shutdown_connections 2024-03-21T10:29:44.603 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:44.601+0000 7ff804473740 1 --2- 172.21.15.47:0/781068637 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b557b57460 0x55b557b56b60 unknown :-1 s=CLOSED pgs=843 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:29:44.603 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:44.601+0000 7ff804473740 1 --2- 172.21.15.47:0/781068637 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b557b4edd0 0x55b557b56620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:29:44.603 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:44.601+0000 7ff804473740 1 --2- 172.21.15.47:0/781068637 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b557b4e0f0 0x55b557b52ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:29:44.603 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:44.601+0000 7ff804473740 1 -- 172.21.15.47:0/781068637 >> 172.21.15.47:0/781068637 conn(0x55b557b15200 msgr2=0x55b557b527e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:29:44.603 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:44.601+0000 7ff804473740 1 -- 172.21.15.47:0/781068637 shutdown_connections 2024-03-21T10:29:44.603 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:44.601+0000 7ff804473740 1 -- 172.21.15.47:0/781068637 wait complete. 2024-03-21T10:29:44.603 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:44.601+0000 7ff804473740 1 -- start start 2024-03-21T10:29:44.603 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:44.601+0000 7ff804473740 1 Processor -- start 2024-03-21T10:29:44.603 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:44.602+0000 7ff804473740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b557b4e0f0 0x55b557b51880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:29:44.603 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:44.602+0000 7ff804473740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b557b4edd0 0x55b557b55e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:29:44.603 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:44.602+0000 7ff804473740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b557b57460 0x55b557b56350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:29:44.604 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:44.602+0000 7ff804473740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7fc1b1d30 con 0x55b557b4e0f0 2024-03-21T10:29:44.604 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:44.602+0000 7ff804473740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7fc1b16d0 con 0x55b557b4edd0 2024-03-21T10:29:44.604 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:44.602+0000 7ff804473740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7fc1aebd0 con 0x55b557b57460 2024-03-21T10:29:44.604 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:44.602+0000 7ff804473740 10 client.0 _ll_register_callbacks cb 0x7ff7f4009d70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T10:29:44.604 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:44.602+0000 7ff7faa00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b557b4e0f0 0x55b557b51880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:29:44.604 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:44.602+0000 7ff7faa00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b557b4e0f0 0x55b557b51880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:55394/0 (socket says 172.21.15.47:55394) 2024-03-21T10:29:44.604 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:44.602+0000 7ff7faa00640 1 -- 172.21.15.47:0/727554237 learned_addr learned my addr 172.21.15.47:0/727554237 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:29:44.604 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:44.602+0000 7ff7fa000640 1 --2- 172.21.15.47:0/727554237 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b557b4edd0 0x55b557b55e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:29:44.604 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:44.603+0000 7ff7fb400640 1 --2- 172.21.15.47:0/727554237 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b557b57460 0x55b557b56350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:29:44.604 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:44.603+0000 7ff7faa00640 1 -- 172.21.15.47:0/727554237 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b557b57460 msgr2=0x55b557b56350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:29:44.604 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:44.603+0000 7ff7faa00640 1 --2- 172.21.15.47:0/727554237 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b557b57460 0x55b557b56350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:29:44.604 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:44.603+0000 7ff7faa00640 1 -- 172.21.15.47:0/727554237 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b557b4edd0 msgr2=0x55b557b55e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:29:44.604 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:44.603+0000 7ff7faa00640 1 --2- 172.21.15.47:0/727554237 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b557b4edd0 0x55b557b55e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:29:44.605 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:44.603+0000 7ff7faa00640 1 -- 172.21.15.47:0/727554237 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55b557b56890 con 0x55b557b4e0f0 2024-03-21T10:29:44.607 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:44.603+0000 7ff7faa00640 1 --2- 172.21.15.47:0/727554237 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b557b4e0f0 0x55b557b51880 secure :-1 s=READY pgs=844 cs=0 l=1 rev1=1 crypto rx=0x7ff7ec009230 tx=0x7ff7ec004460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4dd7948251510bab server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:29:44.607 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:44.603+0000 7ff7e4c00640 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-03-21T10:29:44.607 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:44.603+0000 7ff7e4c00640 1 -- 172.21.15.47:0/727554237 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7ec02cc90 con 0x55b557b4e0f0 2024-03-21T10:29:44.607 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:44.603+0000 7ff7e4c00640 1 -- 172.21.15.47:0/727554237 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(97..97 src has 1..97) v4 ==== 9474+0+0 (secure 0 0 0) 0x7ff7ec005040 con 0x55b557b4e0f0 2024-03-21T10:29:44.607 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:44.603+0000 7ff804473740 1 -- 172.21.15.47:0/727554237 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55b557ab9ae0 con 0x55b557b4e0f0 2024-03-21T10:29:44.607 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:44.604+0000 7ff7e4c00640 1 -- 172.21.15.47:0/727554237 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff7ec058050 con 0x55b557b4e0f0 2024-03-21T10:29:44.607 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:44.604+0000 7ff804473740 10 client.17118 fetch_fsmap learned FSMap version 32 2024-03-21T10:29:44.607 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:44.604+0000 7ff804473740 1 -- 172.21.15.47:0/727554237 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x55b557bc3870 con 0x55b557b4e0f0 2024-03-21T10:29:44.607 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:44.604+0000 7ff7e4c00640 1 -- 172.21.15.47:0/727554237 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x55b557ab9ae0 con 0x55b557b4e0f0 2024-03-21T10:29:44.607 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:44.604+0000 7ff7e4c00640 1 -- 172.21.15.47:0/727554237 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 32) v1 ==== 7942+0+0 (secure 0 0 0) 0x7ff7ec004cc0 con 0x55b557b4e0f0 2024-03-21T10:29:44.607 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:44.604+0000 7ff804473740 10 client.17118 fetch_fsmap finished waiting for FSMap version 32 2024-03-21T10:29:44.608 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:44.604+0000 7ff804473740 10 client.17118 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-21T10:29:44.608 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:44.604+0000 7ff804473740 20 client.17118 populate_metadata read hostname 'smithi047' 2024-03-21T10:29:44.608 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:44.604+0000 7ff804473740 1 --2- 172.21.15.47:0/727554237 >> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] conn(0x55b557bc3f70 0x55b557bc6390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:29:44.608 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:44.604+0000 7ff804473740 4 client.17118 mds_command: new command op to 34535 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-21T10:29:44.608 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:44.604+0000 7ff804473740 1 -- 172.21.15.47:0/727554237 --> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55b557a373a0 con 0x55b557bc3f70 2024-03-21T10:29:44.608 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:44.605+0000 7ff7fa000640 1 --2- 172.21.15.47:0/727554237 >> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] conn(0x55b557bc3f70 0x55b557bc6390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:29:44.608 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:44.605+0000 7ff7fa000640 1 --2- 172.21.15.47:0/727554237 >> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] conn(0x55b557bc3f70 0x55b557bc6390 crc :-1 s=READY pgs=136 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b6a5068abe0fcdb8 server_cookie=5e6993c85125b8e6 in_seq=0 out_seq=0 2024-03-21T10:29:44.608 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:44.605+0000 7ff7e4c00640 10 client.17118 ms_handle_connect on v2:172.21.15.100:6838/4173168440 2024-03-21T10:29:44.608 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:44.606+0000 7ff7e4c00640 1 -- 172.21.15.47:0/727554237 <== mds.0 v2:172.21.15.100:6838/4173168440 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x55b557a373a0 con 0x55b557bc3f70 2024-03-21T10:29:44.608 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:44.606+0000 7ff7e4c00640 10 client.17118 handle_command_reply: tid=0 2024-03-21T10:29:44.608 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:44.606+0000 7ff804473740 1 client.17118 shutdown 2024-03-21T10:29:44.608 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:44.606+0000 7ff804473740 1 -- 172.21.15.47:0/727554237 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b557b4e0f0 msgr2=0x55b557b51880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:29:44.608 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:44.606+0000 7ff804473740 1 --2- 172.21.15.47:0/727554237 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b557b4e0f0 0x55b557b51880 secure :-1 s=READY pgs=844 cs=0 l=1 rev1=1 crypto rx=0x7ff7ec009230 tx=0x7ff7ec004460 comp rx=0 tx=0).stop 2024-03-21T10:29:44.608 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:44.606+0000 7ff804473740 1 -- 172.21.15.47:0/727554237 shutdown_connections 2024-03-21T10:29:44.609 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:44.606+0000 7ff804473740 1 --2- 172.21.15.47:0/727554237 >> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] conn(0x55b557bc3f70 0x55b557bc6390 crc :-1 s=READY pgs=136 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:29:44.609 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:44.606+0000 7ff804473740 1 --2- 172.21.15.47:0/727554237 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b557b57460 0x55b557b56350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:29:44.609 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:44.606+0000 7ff804473740 1 --2- 172.21.15.47:0/727554237 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b557b4edd0 0x55b557b55e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:29:44.609 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:44.606+0000 7ff804473740 1 --2- 172.21.15.47:0/727554237 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b557b4e0f0 0x55b557b51880 unknown :-1 s=CLOSED pgs=844 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:29:44.609 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:44.606+0000 7ff7e4c00640 0 client.17118 ms_handle_reset on v2:172.21.15.100:6838/4173168440 2024-03-21T10:29:44.609 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:44.606+0000 7ff804473740 1 -- 172.21.15.47:0/727554237 >> 172.21.15.47:0/727554237 conn(0x55b557b15200 msgr2=0x55b557b53070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:29:44.609 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:44.606+0000 7ff804473740 1 -- 172.21.15.47:0/727554237 shutdown_connections 2024-03-21T10:29:44.609 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:44.606+0000 7ff804473740 1 -- 172.21.15.47:0/727554237 wait complete. 2024-03-21T10:29:44.609 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:44.606+0000 7ff804473740 20 client.17118 trim_cache size 0 max 16384 2024-03-21T10:29:44.609 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-03-21T10:29:44.609 INFO:teuthology.orchestra.run.smithi047.stdout: "status": "IDLE (waiting for more scrubs)", 2024-03-21T10:29:44.610 INFO:teuthology.orchestra.run.smithi047.stdout: "scrubs": { 2024-03-21T10:29:44.610 INFO:teuthology.orchestra.run.smithi047.stdout: "a7eff57c-820c-4374-b2aa-54e9fbfd6dc7": { 2024-03-21T10:29:44.610 INFO:teuthology.orchestra.run.smithi047.stdout: "path": "/", 2024-03-21T10:29:44.610 INFO:teuthology.orchestra.run.smithi047.stdout: "tag": "a7eff57c-820c-4374-b2aa-54e9fbfd6dc7", 2024-03-21T10:29:44.610 INFO:teuthology.orchestra.run.smithi047.stdout: "options": "recursive,force" 2024-03-21T10:29:44.610 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-21T10:29:44.610 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-21T10:29:44.610 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-03-21T10:29:44.611 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:44.607+0000 7ff7fbe00640 1 -- 172.21.15.47:0/3705466811 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7ff7e00a3090 msgr2=0x7ff7e00a5550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:29:44.611 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:44.607+0000 7ff7fbe00640 1 --2- 172.21.15.47:0/3705466811 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7ff7e00a3090 0x7ff7e00a5550 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7ff7f4005fd0 tx=0x7ff7f4009500 comp rx=0 tx=0).stop 2024-03-21T10:29:44.611 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:44.607+0000 7ff7fbe00640 1 -- 172.21.15.47:0/3705466811 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff7fc102ff0 msgr2=0x7ff7fc19a4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:29:44.611 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:44.607+0000 7ff7fbe00640 1 --2- 172.21.15.47:0/3705466811 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff7fc102ff0 0x7ff7fc19a4f0 secure :-1 s=READY pgs=840 cs=0 l=1 rev1=1 crypto rx=0x7ff7e800e9f0 tx=0x7ff7e800eeb0 comp rx=0 tx=0).stop 2024-03-21T10:29:44.611 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:44.607+0000 7ff7fbe00640 1 -- 172.21.15.47:0/3705466811 shutdown_connections 2024-03-21T10:29:44.611 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:44.607+0000 7ff7fbe00640 1 --2- 172.21.15.47:0/3705466811 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7ff7e00a3090 0x7ff7e00a5550 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:29:44.611 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:44.607+0000 7ff7fbe00640 1 --2- 172.21.15.47:0/3705466811 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff7fc10baf0 0x7ff7fc19e870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:29:44.611 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:44.607+0000 7ff7fbe00640 1 --2- 172.21.15.47:0/3705466811 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff7fc102ff0 0x7ff7fc19a4f0 unknown :-1 s=CLOSED pgs=840 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:29:44.611 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:44.607+0000 7ff7fbe00640 1 --2- 172.21.15.47:0/3705466811 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff7fc1026d0 0x7ff7fc199fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:29:44.611 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:44.607+0000 7ff7fbe00640 1 -- 172.21.15.47:0/3705466811 >> 172.21.15.47:0/3705466811 conn(0x7ff7fc0fbef0 msgr2=0x7ff7fc0fcb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:29:44.612 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:44.607+0000 7ff7fbe00640 1 -- 172.21.15.47:0/3705466811 shutdown_connections 2024-03-21T10:29:44.612 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:29:44.607+0000 7ff7fbe00640 1 -- 172.21.15.47:0/3705466811 wait complete. 2024-03-21T10:29:44.626 INFO:tasks.cephfs.filesystem:scrub status for tag:a7eff57c-820c-4374-b2aa-54e9fbfd6dc7 - {'path': '/', 'tag': 'a7eff57c-820c-4374-b2aa-54e9fbfd6dc7', 'options': 'recursive,force'} 2024-03-21T10:29:45.962 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:29:45 smithi047 ceph-mon[31547]: pgmap v136: 129 pgs: 129 active+clean; 277 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-21T10:29:45.962 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:29:45 smithi047 ceph-mon[31547]: scrub queued for path: / 2024-03-21T10:29:45.963 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:29:45 smithi047 ceph-mon[31547]: scrub summary: idle+waiting paths [/] 2024-03-21T10:29:45.963 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:29:45 smithi047 ceph-mon[31547]: scrub summary: active paths [/] 2024-03-21T10:29:45.963 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:29:45 smithi047 ceph-mon[31547]: scrub summary: idle+waiting paths [/] 2024-03-21T10:29:45.977 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:29:45 smithi100 ceph-mon[35193]: pgmap v136: 129 pgs: 129 active+clean; 277 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-21T10:29:45.978 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:29:45 smithi100 ceph-mon[35193]: scrub queued for path: / 2024-03-21T10:29:45.978 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:29:45 smithi100 ceph-mon[35193]: scrub summary: idle+waiting paths [/] 2024-03-21T10:29:45.978 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:29:45 smithi100 ceph-mon[35193]: scrub summary: active paths [/] 2024-03-21T10:29:45.978 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:29:45 smithi100 ceph-mon[35193]: scrub summary: idle+waiting paths [/] 2024-03-21T10:29:45.988 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:29:45 smithi157 ceph-mon[35581]: pgmap v136: 129 pgs: 129 active+clean; 277 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-21T10:29:45.988 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:29:45 smithi157 ceph-mon[35581]: scrub queued for path: / 2024-03-21T10:29:45.988 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:29:45 smithi157 ceph-mon[35581]: scrub summary: idle+waiting paths [/] 2024-03-21T10:29:45.988 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:29:45 smithi157 ceph-mon[35581]: scrub summary: active paths [/] 2024-03-21T10:29:45.988 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:29:45 smithi157 ceph-mon[35581]: scrub summary: idle+waiting paths [/] 2024-03-21T10:29:46.962 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:29:46 smithi047 ceph-mon[31547]: pgmap v137: 129 pgs: 129 active+clean; 277 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 12 op/s 2024-03-21T10:29:46.978 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:29:46 smithi100 ceph-mon[35193]: pgmap v137: 129 pgs: 129 active+clean; 277 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 12 op/s 2024-03-21T10:29:46.988 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:29:46 smithi157 ceph-mon[35581]: pgmap v137: 129 pgs: 129 active+clean; 277 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 12 op/s 2024-03-21T10:29:49.462 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:29:49 smithi047 ceph-mon[31547]: pgmap v138: 129 pgs: 129 active+clean; 277 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 9 op/s 2024-03-21T10:29:49.462 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:29:49 smithi047 ceph-mon[31547]: scrub summary: idle 2024-03-21T10:29:49.477 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:29:49 smithi100 ceph-mon[35193]: pgmap v138: 129 pgs: 129 active+clean; 277 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 9 op/s 2024-03-21T10:29:49.477 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:29:49 smithi100 ceph-mon[35193]: scrub summary: idle 2024-03-21T10:29:49.488 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:29:49 smithi157 ceph-mon[35581]: pgmap v138: 129 pgs: 129 active+clean; 277 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 9 op/s 2024-03-21T10:29:49.488 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:29:49 smithi157 ceph-mon[35581]: scrub summary: idle 2024-03-21T10:29:51.462 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:29:51 smithi047 ceph-mon[31547]: pgmap v139: 129 pgs: 129 active+clean; 277 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 9 op/s 2024-03-21T10:29:51.477 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:29:51 smithi100 ceph-mon[35193]: pgmap v139: 129 pgs: 129 active+clean; 277 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 9 op/s 2024-03-21T10:29:51.488 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:29:51 smithi157 ceph-mon[35581]: pgmap v139: 129 pgs: 129 active+clean; 277 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 9 op/s 2024-03-21T10:29:53.462 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:29:53 smithi047 ceph-mon[31547]: pgmap v140: 129 pgs: 129 active+clean; 277 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 13 op/s 2024-03-21T10:29:53.477 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:29:53 smithi100 ceph-mon[35193]: pgmap v140: 129 pgs: 129 active+clean; 277 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 13 op/s 2024-03-21T10:29:53.487 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:29:53 smithi157 ceph-mon[35581]: pgmap v140: 129 pgs: 129 active+clean; 277 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 13 op/s 2024-03-21T10:29:55.462 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:29:55 smithi047 ceph-mon[31547]: pgmap v141: 129 pgs: 129 active+clean; 277 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 9 op/s 2024-03-21T10:29:55.477 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:29:55 smithi100 ceph-mon[35193]: pgmap v141: 129 pgs: 129 active+clean; 277 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 9 op/s 2024-03-21T10:29:55.488 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:29:55 smithi157 ceph-mon[35581]: pgmap v141: 129 pgs: 129 active+clean; 277 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 9 op/s 2024-03-21T10:29:57.462 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:29:57 smithi047 ceph-mon[31547]: pgmap v142: 129 pgs: 129 active+clean; 277 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 13 op/s 2024-03-21T10:29:57.477 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:29:57 smithi100 ceph-mon[35193]: pgmap v142: 129 pgs: 129 active+clean; 277 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 13 op/s 2024-03-21T10:29:57.488 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:29:57 smithi157 ceph-mon[35581]: pgmap v142: 129 pgs: 129 active+clean; 277 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 13 op/s 2024-03-21T10:29:59.462 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:29:59 smithi047 ceph-mon[31547]: pgmap v143: 129 pgs: 129 active+clean; 277 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 8 op/s 2024-03-21T10:29:59.477 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:29:59 smithi100 ceph-mon[35193]: pgmap v143: 129 pgs: 129 active+clean; 277 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 8 op/s 2024-03-21T10:29:59.487 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:29:59 smithi157 ceph-mon[35581]: pgmap v143: 129 pgs: 129 active+clean; 277 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 8 op/s 2024-03-21T10:30:00.462 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:30:00 smithi047 ceph-mon[31547]: overall HEALTH_OK 2024-03-21T10:30:00.477 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:30:00 smithi100 ceph-mon[35193]: overall HEALTH_OK 2024-03-21T10:30:00.487 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:30:00 smithi157 ceph-mon[35581]: overall HEALTH_OK 2024-03-21T10:30:01.462 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:30:01 smithi047 ceph-mon[31547]: pgmap v144: 129 pgs: 129 active+clean; 277 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-21T10:30:01.477 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:30:01 smithi100 ceph-mon[35193]: pgmap v144: 129 pgs: 129 active+clean; 277 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-21T10:30:01.487 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:30:01 smithi157 ceph-mon[35581]: pgmap v144: 129 pgs: 129 active+clean; 277 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-21T10:30:03.462 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:30:03 smithi047 ceph-mon[31547]: pgmap v145: 129 pgs: 129 active+clean; 277 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 22 KiB/s wr, 27 op/s 2024-03-21T10:30:03.477 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:30:03 smithi100 ceph-mon[35193]: pgmap v145: 129 pgs: 129 active+clean; 277 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 22 KiB/s wr, 27 op/s 2024-03-21T10:30:03.487 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:30:03 smithi157 ceph-mon[35581]: pgmap v145: 129 pgs: 129 active+clean; 277 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 22 KiB/s wr, 27 op/s 2024-03-21T10:30:05.462 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:30:05 smithi047 ceph-mon[31547]: pgmap v146: 129 pgs: 129 active+clean; 277 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-03-21T10:30:05.477 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:30:05 smithi100 ceph-mon[35193]: pgmap v146: 129 pgs: 129 active+clean; 277 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-03-21T10:30:05.487 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:30:05 smithi157 ceph-mon[35581]: pgmap v146: 129 pgs: 129 active+clean; 277 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-03-21T10:30:07.462 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:30:07 smithi047 ceph-mon[31547]: pgmap v147: 129 pgs: 129 active+clean; 277 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 22 KiB/s wr, 27 op/s 2024-03-21T10:30:07.477 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:30:07 smithi100 ceph-mon[35193]: pgmap v147: 129 pgs: 129 active+clean; 277 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 22 KiB/s wr, 27 op/s 2024-03-21T10:30:07.488 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:30:07 smithi157 ceph-mon[35581]: pgmap v147: 129 pgs: 129 active+clean; 277 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 22 KiB/s wr, 27 op/s 2024-03-21T10:30:09.462 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:30:09 smithi047 ceph-mon[31547]: pgmap v148: 129 pgs: 129 active+clean; 277 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-03-21T10:30:09.477 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:30:09 smithi100 ceph-mon[35193]: pgmap v148: 129 pgs: 129 active+clean; 277 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-03-21T10:30:09.487 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:30:09 smithi157 ceph-mon[35581]: pgmap v148: 129 pgs: 129 active+clean; 277 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-03-21T10:30:11.462 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:30:11 smithi047 ceph-mon[31547]: pgmap v149: 129 pgs: 129 active+clean; 277 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-03-21T10:30:11.477 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:30:11 smithi100 ceph-mon[35193]: pgmap v149: 129 pgs: 129 active+clean; 277 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-03-21T10:30:11.487 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:30:11 smithi157 ceph-mon[35581]: pgmap v149: 129 pgs: 129 active+clean; 277 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-03-21T10:30:13.462 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:30:13 smithi047 ceph-mon[31547]: pgmap v150: 129 pgs: 129 active+clean; 277 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 KiB/s wr, 27 op/s 2024-03-21T10:30:13.477 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:30:13 smithi100 ceph-mon[35193]: pgmap v150: 129 pgs: 129 active+clean; 277 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 KiB/s wr, 27 op/s 2024-03-21T10:30:13.487 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:30:13 smithi157 ceph-mon[35581]: pgmap v150: 129 pgs: 129 active+clean; 277 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 KiB/s wr, 27 op/s 2024-03-21T10:30:14.627 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-21T10:30:14.627 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-21T10:30:14.859 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:14.858+0000 7fc48c200640 1 -- 172.21.15.47:0/44792323 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc484073a40 msgr2=0x7fc484073e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:30:14.859 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:14.858+0000 7fc48c200640 1 --2- 172.21.15.47:0/44792323 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc484073a40 0x7fc484073e20 secure :-1 s=READY pgs=845 cs=0 l=1 rev1=1 crypto rx=0x7fc478009900 tx=0x7fc47802bcc0 comp rx=0 tx=0).stop 2024-03-21T10:30:14.859 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:14.858+0000 7fc48c200640 1 -- 172.21.15.47:0/44792323 shutdown_connections 2024-03-21T10:30:14.859 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:14.858+0000 7fc48c200640 1 --2- 172.21.15.47:0/44792323 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc484112620 0x7fc484114a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:30:14.859 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:14.858+0000 7fc48c200640 1 --2- 172.21.15.47:0/44792323 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc484074360 0x7fc4841120e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:30:14.859 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:14.858+0000 7fc48c200640 1 --2- 172.21.15.47:0/44792323 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc484073a40 0x7fc484073e20 unknown :-1 s=CLOSED pgs=845 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:30:14.859 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:14.858+0000 7fc48c200640 1 -- 172.21.15.47:0/44792323 >> 172.21.15.47:0/44792323 conn(0x7fc4840fd100 msgr2=0x7fc4840ff520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:30:14.859 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:14.858+0000 7fc48c200640 1 -- 172.21.15.47:0/44792323 shutdown_connections 2024-03-21T10:30:14.859 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:14.858+0000 7fc48c200640 1 -- 172.21.15.47:0/44792323 wait complete. 2024-03-21T10:30:14.860 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:14.860+0000 7fc48c200640 1 Processor -- start 2024-03-21T10:30:14.860 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:14.860+0000 7fc48c200640 1 -- start start 2024-03-21T10:30:14.862 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:14.861+0000 7fc48c200640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc484073a40 0x7fc48419e4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:30:14.862 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:14.861+0000 7fc48c200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc484074360 0x7fc48419ea20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:30:14.863 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:14.861+0000 7fc48c200640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc484112620 0x7fc4841a2da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:30:14.863 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:14.861+0000 7fc48c200640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc484117ed0 con 0x7fc484112620 2024-03-21T10:30:14.863 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:14.861+0000 7fc48c200640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc484117d50 con 0x7fc484074360 2024-03-21T10:30:14.863 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:14.861+0000 7fc48c200640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc484118050 con 0x7fc484073a40 2024-03-21T10:30:14.863 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:14.861+0000 7fc48a200640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc484112620 0x7fc4841a2da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:30:14.863 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:14.861+0000 7fc48a200640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc484112620 0x7fc4841a2da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:38090/0 (socket says 172.21.15.47:38090) 2024-03-21T10:30:14.863 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:14.861+0000 7fc488e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc484074360 0x7fc48419ea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:30:14.863 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:14.861+0000 7fc48a200640 1 -- 172.21.15.47:0/2305977590 learned_addr learned my addr 172.21.15.47:0/2305977590 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:30:14.863 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:14.861+0000 7fc489800640 1 --2- 172.21.15.47:0/2305977590 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc484073a40 0x7fc48419e4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:30:14.863 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:14.862+0000 7fc48a200640 1 -- 172.21.15.47:0/2305977590 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc484073a40 msgr2=0x7fc48419e4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:30:14.863 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:14.862+0000 7fc48a200640 1 --2- 172.21.15.47:0/2305977590 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc484073a40 0x7fc48419e4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:30:14.863 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:14.862+0000 7fc48a200640 1 -- 172.21.15.47:0/2305977590 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc484074360 msgr2=0x7fc48419ea20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:30:14.863 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:14.862+0000 7fc48a200640 1 --2- 172.21.15.47:0/2305977590 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc484074360 0x7fc48419ea20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:30:14.864 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:14.862+0000 7fc48a200640 1 -- 172.21.15.47:0/2305977590 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc4841a34d0 con 0x7fc484112620 2024-03-21T10:30:14.864 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:14.862+0000 7fc489800640 1 --2- 172.21.15.47:0/2305977590 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc484073a40 0x7fc48419e4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T10:30:14.864 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:14.862+0000 7fc488e00640 1 --2- 172.21.15.47:0/2305977590 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc484074360 0x7fc48419ea20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T10:30:14.864 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:14.862+0000 7fc48a200640 1 --2- 172.21.15.47:0/2305977590 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc484112620 0x7fc4841a2da0 secure :-1 s=READY pgs=846 cs=0 l=1 rev1=1 crypto rx=0x7fc47c00e9d0 tx=0x7fc47c00ee90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:30:14.864 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:14.862+0000 7fc482000640 1 -- 172.21.15.47:0/2305977590 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc47c00cd80 con 0x7fc484112620 2024-03-21T10:30:14.864 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:14.862+0000 7fc482000640 1 -- 172.21.15.47:0/2305977590 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fc47c005020 con 0x7fc484112620 2024-03-21T10:30:14.864 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:14.862+0000 7fc48c200640 1 -- 172.21.15.47:0/2305977590 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc4841a37c0 con 0x7fc484112620 2024-03-21T10:30:14.864 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:14.862+0000 7fc482000640 1 -- 172.21.15.47:0/2305977590 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc47c0116d0 con 0x7fc484112620 2024-03-21T10:30:14.864 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:14.863+0000 7fc48c200640 1 -- 172.21.15.47:0/2305977590 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc4841ab050 con 0x7fc484112620 2024-03-21T10:30:14.866 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:14.866+0000 7fc482000640 1 -- 172.21.15.47:0/2305977590 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 27) v1 ==== 137985+0+0 (secure 0 0 0) 0x7fc47c0051d0 con 0x7fc484112620 2024-03-21T10:30:14.866 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:14.866+0000 7fc48ef76740 1 Processor -- start 2024-03-21T10:30:14.866 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:14.866+0000 7fc48ef76740 1 -- start start 2024-03-21T10:30:14.868 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:14.867+0000 7fc48ef76740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e9488a7c00 0x55e9488a6950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:30:14.868 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:14.867+0000 7fc48ef76740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e9488a6f20 0x55e9488a73a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:30:14.868 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:14.867+0000 7fc48ef76740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e9488b0290 0x55e9488b2680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:30:14.868 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:14.867+0000 7fc48ef76740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e94880b250 con 0x55e9488a6f20 2024-03-21T10:30:14.868 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:14.867+0000 7fc48ef76740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e9487b3a30 con 0x55e9488b0290 2024-03-21T10:30:14.869 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:14.867+0000 7fc48ef76740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e9488072f0 con 0x55e9488a7c00 2024-03-21T10:30:14.869 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:14.867+0000 7fc489800640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e9488a6f20 0x55e9488a73a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:30:14.869 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:14.867+0000 7fc489800640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e9488a6f20 0x55e9488a73a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:38102/0 (socket says 172.21.15.47:38102) 2024-03-21T10:30:14.869 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:14.867+0000 7fc489800640 1 -- 172.21.15.47:0/508089465 learned_addr learned my addr 172.21.15.47:0/508089465 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:30:14.869 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:14.867+0000 7fc488e00640 1 --2- 172.21.15.47:0/508089465 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e9488b0290 0x55e9488b2680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:30:14.869 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:14.867+0000 7fc488e00640 1 --2- 172.21.15.47:0/508089465 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e9488a7c00 0x55e9488a6950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:30:14.869 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:14.867+0000 7fc489800640 1 -- 172.21.15.47:0/508089465 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e9488a7c00 msgr2=0x55e9488a6950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:30:14.869 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:14.868+0000 7fc489800640 1 --2- 172.21.15.47:0/508089465 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e9488a7c00 0x55e9488a6950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:30:14.869 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:14.868+0000 7fc489800640 1 -- 172.21.15.47:0/508089465 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e9488b0290 msgr2=0x55e9488b2680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:30:14.869 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:14.868+0000 7fc489800640 1 --2- 172.21.15.47:0/508089465 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e9488b0290 0x55e9488b2680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:30:14.869 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:14.868+0000 7fc489800640 1 -- 172.21.15.47:0/508089465 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e94879e690 con 0x55e9488a6f20 2024-03-21T10:30:14.869 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:14.868+0000 7fc488e00640 1 --2- 172.21.15.47:0/508089465 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e9488a7c00 0x55e9488a6950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T10:30:14.869 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:14.868+0000 7fc482000640 1 --2- 172.21.15.47:0/2305977590 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7fc46c0a3090 0x7fc46c0a5550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:30:14.869 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:14.868+0000 7fc489800640 1 --2- 172.21.15.47:0/508089465 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e9488a6f20 0x55e9488a73a0 secure :-1 s=READY pgs=847 cs=0 l=1 rev1=1 crypto rx=0x7fc478009900 tx=0x7fc478005e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6825a28fdd121f0d server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:30:14.870 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:14.868+0000 7fc482000640 1 -- 172.21.15.47:0/2305977590 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 9474+0+0 (secure 0 0 0) 0x7fc47c0d0070 con 0x7fc484112620 2024-03-21T10:30:14.870 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:14.868+0000 7fc488e00640 1 --2- 172.21.15.47:0/2305977590 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7fc46c0a3090 0x7fc46c0a5550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:30:14.870 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:14.868+0000 7fc476a00640 1 -- 172.21.15.47:0/508089465 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc478004240 con 0x55e9488a6f20 2024-03-21T10:30:14.870 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:14.868+0000 7fc476a00640 1 -- 172.21.15.47:0/508089465 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fc4780043e0 con 0x55e9488a6f20 2024-03-21T10:30:14.870 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:14.868+0000 7fc476a00640 1 -- 172.21.15.47:0/508089465 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4780048c0 con 0x55e9488a6f20 2024-03-21T10:30:14.872 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:14.869+0000 7fc48ef76740 1 -- 172.21.15.47:0/508089465 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e9488a6f20 msgr2=0x55e9488a73a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:30:14.872 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:14.869+0000 7fc488e00640 1 --2- 172.21.15.47:0/2305977590 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7fc46c0a3090 0x7fc46c0a5550 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fc4840ff8d0 tx=0x7fc470005850 comp rx=0 tx=0).ready entity=mgr.15687 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:30:14.872 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:14.869+0000 7fc48ef76740 1 --2- 172.21.15.47:0/508089465 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e9488a6f20 0x55e9488a73a0 secure :-1 s=READY pgs=847 cs=0 l=1 rev1=1 crypto rx=0x7fc478009900 tx=0x7fc478005e50 comp rx=0 tx=0).stop 2024-03-21T10:30:14.872 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:14.869+0000 7fc48ef76740 1 -- 172.21.15.47:0/508089465 shutdown_connections 2024-03-21T10:30:14.872 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:14.869+0000 7fc48ef76740 1 --2- 172.21.15.47:0/508089465 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e9488b0290 0x55e9488b2680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:30:14.872 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:14.869+0000 7fc48ef76740 1 --2- 172.21.15.47:0/508089465 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e9488a6f20 0x55e9488a73a0 unknown :-1 s=CLOSED pgs=847 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:30:14.873 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:14.869+0000 7fc48ef76740 1 --2- 172.21.15.47:0/508089465 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e9488a7c00 0x55e9488a6950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:30:14.873 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:14.869+0000 7fc48ef76740 1 -- 172.21.15.47:0/508089465 >> 172.21.15.47:0/508089465 conn(0x55e94886e200 msgr2=0x55e9488a1970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:30:14.873 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:14.869+0000 7fc48ef76740 1 -- 172.21.15.47:0/508089465 shutdown_connections 2024-03-21T10:30:14.873 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:14.870+0000 7fc48ef76740 1 -- 172.21.15.47:0/508089465 wait complete. 2024-03-21T10:30:14.873 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:14.870+0000 7fc48ef76740 1 -- start start 2024-03-21T10:30:14.873 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:14.870+0000 7fc48ef76740 1 Processor -- start 2024-03-21T10:30:14.873 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:14.871+0000 7fc48ef76740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e9488a6f20 0x55e9488af8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:30:14.873 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:14.871+0000 7fc48ef76740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e9488a7c00 0x55e9488acc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:30:14.873 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:14.871+0000 7fc48ef76740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e9488b0290 0x55e9488ad1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:30:14.873 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:14.871+0000 7fc48ef76740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e9488b30a0 con 0x55e9488a7c00 2024-03-21T10:30:14.873 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:14.871+0000 7fc48ef76740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e9487c6e80 con 0x55e9488a6f20 2024-03-21T10:30:14.873 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:14.871+0000 7fc48ef76740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e9488b3220 con 0x55e9488b0290 2024-03-21T10:30:14.873 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:14.871+0000 7fc48ef76740 10 client.0 _ll_register_callbacks cb 0x55e9487d9130 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T10:30:14.874 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:14.872+0000 7fc489800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e9488a6f20 0x55e9488af8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:30:14.874 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:14.872+0000 7fc488e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e9488a7c00 0x55e9488acc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:30:14.874 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:14.872+0000 7fc489800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e9488a6f20 0x55e9488af8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:42092/0 (socket says 172.21.15.47:42092) 2024-03-21T10:30:14.874 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:14.872+0000 7fc489800640 1 -- 172.21.15.47:0/797994469 learned_addr learned my addr 172.21.15.47:0/797994469 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:30:14.874 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:14.872+0000 7fc48a200640 1 --2- 172.21.15.47:0/797994469 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e9488b0290 0x55e9488ad1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:30:14.874 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:14.872+0000 7fc489800640 1 -- 172.21.15.47:0/797994469 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e9488b0290 msgr2=0x55e9488ad1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:30:14.874 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:14.872+0000 7fc489800640 1 --2- 172.21.15.47:0/797994469 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e9488b0290 0x55e9488ad1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:30:14.874 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:14.872+0000 7fc489800640 1 -- 172.21.15.47:0/797994469 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e9488a7c00 msgr2=0x55e9488acc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:30:14.874 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:14.872+0000 7fc489800640 1 --2- 172.21.15.47:0/797994469 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e9488a7c00 0x55e9488acc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:30:14.874 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:14.872+0000 7fc489800640 1 -- 172.21.15.47:0/797994469 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55e9485e97d0 con 0x55e9488a6f20 2024-03-21T10:30:14.876 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:14.873+0000 7fc489800640 1 --2- 172.21.15.47:0/797994469 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e9488a6f20 0x55e9488af8d0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fc47802bd00 tx=0x7fc478034bb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c81125601b184b75 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:30:14.876 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:14.873+0000 7fc46b400640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-21T10:30:14.876 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:14.873+0000 7fc46b400640 1 -- 172.21.15.47:0/797994469 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc47804e020 con 0x55e9488a6f20 2024-03-21T10:30:14.876 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:14.873+0000 7fc48ef76740 1 -- 172.21.15.47:0/797994469 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fc4780043e0 con 0x55e9488a6f20 2024-03-21T10:30:14.876 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:14.873+0000 7fc46b400640 1 -- 172.21.15.47:0/797994469 <== mon.2 v2:172.21.15.100:3300/0 2 ==== osd_map(97..97 src has 1..97) v4 ==== 9474+0+0 (secure 0 0 0) 0x7fc478005050 con 0x55e9488a6f20 2024-03-21T10:30:14.876 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:14.873+0000 7fc46b400640 1 -- 172.21.15.47:0/797994469 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc478005250 con 0x55e9488a6f20 2024-03-21T10:30:14.877 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:14.873+0000 7fc48ef76740 10 client.36920 fetch_fsmap learned FSMap version 32 2024-03-21T10:30:14.877 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:14.873+0000 7fc48ef76740 1 -- 172.21.15.47:0/797994469 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x55e94851f3f0 con 0x55e9488a6f20 2024-03-21T10:30:14.877 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:14.874+0000 7fc46b400640 1 -- 172.21.15.47:0/797994469 <== mon.2 v2:172.21.15.100:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fc4780043e0 con 0x55e9488a6f20 2024-03-21T10:30:14.877 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:14.874+0000 7fc46b400640 1 -- 172.21.15.47:0/797994469 <== mon.2 v2:172.21.15.100:3300/0 5 ==== fsmap(e 32) v1 ==== 7942+0+0 (secure 0 0 0) 0x7fc478004b70 con 0x55e9488a6f20 2024-03-21T10:30:14.877 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:14.874+0000 7fc48ef76740 10 client.36920 fetch_fsmap finished waiting for FSMap version 32 2024-03-21T10:30:14.877 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:14.874+0000 7fc48ef76740 10 client.36920 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-21T10:30:14.877 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:14.874+0000 7fc48ef76740 20 client.36920 populate_metadata read hostname 'smithi047' 2024-03-21T10:30:14.877 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:14.874+0000 7fc48ef76740 1 --2- 172.21.15.47:0/797994469 >> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] conn(0x55e9488bfec0 0x55e9488c22b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:30:14.877 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:14.874+0000 7fc48ef76740 4 client.36920 mds_command: new command op to 34535 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T10:30:14.877 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:14.874+0000 7fc48ef76740 1 -- 172.21.15.47:0/797994469 --> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55e948818790 con 0x55e9488bfec0 2024-03-21T10:30:14.877 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:14.875+0000 7fc488e00640 1 --2- 172.21.15.47:0/797994469 >> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] conn(0x55e9488bfec0 0x55e9488c22b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:30:14.877 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:14.876+0000 7fc488e00640 1 --2- 172.21.15.47:0/797994469 >> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] conn(0x55e9488bfec0 0x55e9488c22b0 crc :-1 s=READY pgs=137 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9a888a7e0a58ec7a server_cookie=13e27deda5e94533 in_seq=0 out_seq=0 2024-03-21T10:30:14.877 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:14.876+0000 7fc46b400640 10 client.36920 ms_handle_connect on v2:172.21.15.100:6838/4173168440 2024-03-21T10:30:14.879 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:14.877+0000 7fc46b400640 1 -- 172.21.15.47:0/797994469 <== mds.0 v2:172.21.15.100:6838/4173168440 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55e948818790 con 0x55e9488bfec0 2024-03-21T10:30:14.879 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:14.877+0000 7fc46b400640 10 client.36920 handle_command_reply: tid=0 2024-03-21T10:30:14.879 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:14.877+0000 7fc48ef76740 1 client.36920 shutdown 2024-03-21T10:30:14.879 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:14.878+0000 7fc48ef76740 1 -- 172.21.15.47:0/797994469 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e9488a6f20 msgr2=0x55e9488af8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:30:14.879 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:14.878+0000 7fc48ef76740 1 --2- 172.21.15.47:0/797994469 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e9488a6f20 0x55e9488af8d0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fc47802bd00 tx=0x7fc478034bb0 comp rx=0 tx=0).stop 2024-03-21T10:30:14.879 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:14.878+0000 7fc48ef76740 1 -- 172.21.15.47:0/797994469 shutdown_connections 2024-03-21T10:30:14.879 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:14.878+0000 7fc48ef76740 1 --2- 172.21.15.47:0/797994469 >> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] conn(0x55e9488bfec0 0x55e9488c22b0 crc :-1 s=READY pgs=137 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:30:14.879 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:14.878+0000 7fc48ef76740 1 --2- 172.21.15.47:0/797994469 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e9488b0290 0x55e9488ad1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:30:14.879 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:14.878+0000 7fc48ef76740 1 --2- 172.21.15.47:0/797994469 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e9488a7c00 0x55e9488acc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:30:14.879 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:14.878+0000 7fc48ef76740 1 --2- 172.21.15.47:0/797994469 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e9488a6f20 0x55e9488af8d0 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:30:14.879 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:14.878+0000 7fc46b400640 0 client.36920 ms_handle_reset on v2:172.21.15.100:6838/4173168440 2024-03-21T10:30:14.879 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:14.878+0000 7fc48ef76740 1 -- 172.21.15.47:0/797994469 >> 172.21.15.47:0/797994469 conn(0x55e94886e200 msgr2=0x55e9488a36c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:30:14.879 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:14.878+0000 7fc48ef76740 1 -- 172.21.15.47:0/797994469 shutdown_connections 2024-03-21T10:30:14.880 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:14.878+0000 7fc48ef76740 1 -- 172.21.15.47:0/797994469 wait complete. 2024-03-21T10:30:14.880 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:14.878+0000 7fc48ef76740 20 client.36920 trim_cache size 0 max 16384 2024-03-21T10:30:14.904 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:14.901+0000 7fc48ef76740 1 Processor -- start 2024-03-21T10:30:14.904 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:14.901+0000 7fc48ef76740 1 -- start start 2024-03-21T10:30:14.904 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:14.901+0000 7fc48ef76740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e9488a6f20 0x55e9488bc4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:30:14.904 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:14.901+0000 7fc48ef76740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e9488a7c00 0x55e9488abb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:30:14.904 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:14.901+0000 7fc48ef76740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e9488b0290 0x55e9488b1680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:30:14.904 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:14.901+0000 7fc48ef76740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4841b4f60 con 0x55e9488a7c00 2024-03-21T10:30:14.904 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:14.901+0000 7fc48ef76740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4841b68e0 con 0x55e9488a6f20 2024-03-21T10:30:14.904 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:14.901+0000 7fc48ef76740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4841b55c0 con 0x55e9488b0290 2024-03-21T10:30:14.904 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:14.902+0000 7fc489800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e9488a6f20 0x55e9488bc4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:30:14.905 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:14.902+0000 7fc488e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e9488a7c00 0x55e9488abb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:30:14.905 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:14.902+0000 7fc489800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e9488a6f20 0x55e9488bc4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:42096/0 (socket says 172.21.15.47:42096) 2024-03-21T10:30:14.905 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:14.902+0000 7fc489800640 1 -- 172.21.15.47:0/1562965123 learned_addr learned my addr 172.21.15.47:0/1562965123 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:30:14.905 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:14.902+0000 7fc48a200640 1 --2- 172.21.15.47:0/1562965123 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e9488b0290 0x55e9488b1680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:30:14.905 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:14.902+0000 7fc489800640 1 -- 172.21.15.47:0/1562965123 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e9488b0290 msgr2=0x55e9488b1680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:30:14.905 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:14.902+0000 7fc489800640 1 --2- 172.21.15.47:0/1562965123 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e9488b0290 0x55e9488b1680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:30:14.905 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:14.902+0000 7fc489800640 1 -- 172.21.15.47:0/1562965123 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e9488a7c00 msgr2=0x55e9488abb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:30:14.905 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:14.902+0000 7fc489800640 1 --2- 172.21.15.47:0/1562965123 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e9488a7c00 0x55e9488abb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:30:14.905 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:14.902+0000 7fc489800640 1 -- 172.21.15.47:0/1562965123 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e9484df0f0 con 0x55e9488a6f20 2024-03-21T10:30:14.905 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:14.902+0000 7fc488e00640 1 --2- 172.21.15.47:0/1562965123 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e9488a7c00 0x55e9488abb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T10:30:14.905 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:14.902+0000 7fc48a200640 1 --2- 172.21.15.47:0/1562965123 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e9488b0290 0x55e9488b1680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T10:30:14.905 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:14.903+0000 7fc489800640 1 --2- 172.21.15.47:0/1562965123 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e9488a6f20 0x55e9488bc4f0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fc4780057c0 tx=0x7fc478005c90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3f69afd86989626d server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:30:14.905 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:14.903+0000 7fc476000640 1 -- 172.21.15.47:0/1562965123 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc47804e040 con 0x55e9488a6f20 2024-03-21T10:30:14.905 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:14.903+0000 7fc476000640 1 -- 172.21.15.47:0/1562965123 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fc47802ca80 con 0x55e9488a6f20 2024-03-21T10:30:14.906 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:14.903+0000 7fc476000640 1 -- 172.21.15.47:0/1562965123 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc47803e770 con 0x55e9488a6f20 2024-03-21T10:30:14.906 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:14.904+0000 7fc48ef76740 1 -- 172.21.15.47:0/1562965123 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e9488a6f20 msgr2=0x55e9488bc4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:30:14.906 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:14.904+0000 7fc48ef76740 1 --2- 172.21.15.47:0/1562965123 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e9488a6f20 0x55e9488bc4f0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fc4780057c0 tx=0x7fc478005c90 comp rx=0 tx=0).stop 2024-03-21T10:30:14.906 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:14.904+0000 7fc48ef76740 1 -- 172.21.15.47:0/1562965123 shutdown_connections 2024-03-21T10:30:14.906 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:14.904+0000 7fc48ef76740 1 --2- 172.21.15.47:0/1562965123 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e9488b0290 0x55e9488b1680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:30:14.906 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:14.904+0000 7fc48ef76740 1 --2- 172.21.15.47:0/1562965123 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e9488a7c00 0x55e9488abb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:30:14.906 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:14.904+0000 7fc48ef76740 1 --2- 172.21.15.47:0/1562965123 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e9488a6f20 0x55e9488bc4f0 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:30:14.906 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:14.904+0000 7fc48ef76740 1 -- 172.21.15.47:0/1562965123 >> 172.21.15.47:0/1562965123 conn(0x55e94886e200 msgr2=0x55e9488ab400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:30:14.906 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:14.904+0000 7fc48ef76740 1 -- 172.21.15.47:0/1562965123 shutdown_connections 2024-03-21T10:30:14.906 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:14.904+0000 7fc48ef76740 1 -- 172.21.15.47:0/1562965123 wait complete. 2024-03-21T10:30:14.910 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:14.904+0000 7fc48ef76740 1 -- start start 2024-03-21T10:30:14.910 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:14.904+0000 7fc48ef76740 1 Processor -- start 2024-03-21T10:30:14.910 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:14.905+0000 7fc48ef76740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e9488a6f20 0x55e948914750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:30:14.910 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:14.905+0000 7fc48ef76740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e9488a7c00 0x55e948914c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:30:14.910 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:14.905+0000 7fc48ef76740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e9488b0290 0x55e9488a9a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:30:14.910 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:14.905+0000 7fc48ef76740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4841b6280 con 0x55e9488a7c00 2024-03-21T10:30:14.910 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:14.905+0000 7fc48ef76740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4841b5c20 con 0x55e9488b0290 2024-03-21T10:30:14.910 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:14.905+0000 7fc48ef76740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4841b3120 con 0x55e9488a6f20 2024-03-21T10:30:14.911 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:14.905+0000 7fc48ef76740 10 client.0 _ll_register_callbacks cb 0x7fc478009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T10:30:14.911 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:14.905+0000 7fc488e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e9488a7c00 0x55e948914c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:30:14.911 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:14.905+0000 7fc488e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e9488a7c00 0x55e948914c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:38130/0 (socket says 172.21.15.47:38130) 2024-03-21T10:30:14.911 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:14.905+0000 7fc488e00640 1 -- 172.21.15.47:0/2313266760 learned_addr learned my addr 172.21.15.47:0/2313266760 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:30:14.911 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:14.905+0000 7fc48a200640 1 --2- 172.21.15.47:0/2313266760 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e9488b0290 0x55e9488a9a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:30:14.911 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:14.906+0000 7fc489800640 1 --2- 172.21.15.47:0/2313266760 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e9488a6f20 0x55e948914750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:30:14.911 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:14.906+0000 7fc488e00640 1 -- 172.21.15.47:0/2313266760 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e9488a6f20 msgr2=0x55e948914750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:30:14.911 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:14.906+0000 7fc488e00640 1 --2- 172.21.15.47:0/2313266760 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e9488a6f20 0x55e948914750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:30:14.911 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:14.906+0000 7fc488e00640 1 -- 172.21.15.47:0/2313266760 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e9488b0290 msgr2=0x55e9488a9a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:30:14.911 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:14.906+0000 7fc488e00640 1 --2- 172.21.15.47:0/2313266760 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e9488b0290 0x55e9488a9a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:30:14.911 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:14.906+0000 7fc488e00640 1 -- 172.21.15.47:0/2313266760 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55e9488aa170 con 0x55e9488a7c00 2024-03-21T10:30:14.911 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:14.906+0000 7fc488e00640 1 --2- 172.21.15.47:0/2313266760 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e9488a7c00 0x55e948914c90 secure :-1 s=READY pgs=848 cs=0 l=1 rev1=1 crypto rx=0x7fc470009fd0 tx=0x7fc470005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e60fdbb31e05d8dd server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:30:14.911 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:14.906+0000 7fc46b400640 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-03-21T10:30:14.912 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:14.906+0000 7fc46b400640 1 -- 172.21.15.47:0/2313266760 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc47002a110 con 0x55e9488a7c00 2024-03-21T10:30:14.912 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:14.906+0000 7fc48ef76740 1 -- 172.21.15.47:0/2313266760 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fc47802ca80 con 0x55e9488a7c00 2024-03-21T10:30:14.912 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:14.906+0000 7fc46b400640 1 -- 172.21.15.47:0/2313266760 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(97..97 src has 1..97) v4 ==== 9474+0+0 (secure 0 0 0) 0x7fc470017030 con 0x55e9488a7c00 2024-03-21T10:30:14.912 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:14.907+0000 7fc46b400640 1 -- 172.21.15.47:0/2313266760 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc470038050 con 0x55e9488a7c00 2024-03-21T10:30:14.912 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:14.907+0000 7fc48ef76740 10 client.17136 fetch_fsmap learned FSMap version 32 2024-03-21T10:30:14.912 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:14.907+0000 7fc48ef76740 1 -- 172.21.15.47:0/2313266760 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x55e94891be40 con 0x55e9488a7c00 2024-03-21T10:30:14.912 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:14.907+0000 7fc46b400640 1 -- 172.21.15.47:0/2313266760 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fc47802ca80 con 0x55e9488a7c00 2024-03-21T10:30:14.912 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:14.907+0000 7fc46b400640 1 -- 172.21.15.47:0/2313266760 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 32) v1 ==== 7942+0+0 (secure 0 0 0) 0x7fc4700047a0 con 0x55e9488a7c00 2024-03-21T10:30:14.912 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:14.907+0000 7fc48ef76740 10 client.17136 fetch_fsmap finished waiting for FSMap version 32 2024-03-21T10:30:14.912 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:14.907+0000 7fc48ef76740 10 client.17136 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-21T10:30:14.912 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:14.907+0000 7fc48ef76740 20 client.17136 populate_metadata read hostname 'smithi047' 2024-03-21T10:30:14.912 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:14.907+0000 7fc48ef76740 1 --2- 172.21.15.47:0/2313266760 >> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] conn(0x55e94891c5a0 0x55e94891e9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:30:14.912 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:14.907+0000 7fc48ef76740 4 client.17136 mds_command: new command op to 34535 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-21T10:30:14.912 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:14.907+0000 7fc48ef76740 1 -- 172.21.15.47:0/2313266760 --> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55e948812ae0 con 0x55e94891c5a0 2024-03-21T10:30:14.913 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:14.908+0000 7fc489800640 1 --2- 172.21.15.47:0/2313266760 >> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] conn(0x55e94891c5a0 0x55e94891e9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:30:14.913 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:14.908+0000 7fc489800640 1 --2- 172.21.15.47:0/2313266760 >> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] conn(0x55e94891c5a0 0x55e94891e9c0 crc :-1 s=READY pgs=138 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3720f3fa4c6071c2 server_cookie=dbc10dd3cfe4ed2f in_seq=0 out_seq=0 2024-03-21T10:30:14.913 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:14.908+0000 7fc46b400640 10 client.17136 ms_handle_connect on v2:172.21.15.100:6838/4173168440 2024-03-21T10:30:14.913 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:14.909+0000 7fc46b400640 1 -- 172.21.15.47:0/2313266760 <== mds.0 v2:172.21.15.100:6838/4173168440 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x55e948812ae0 con 0x55e94891c5a0 2024-03-21T10:30:14.913 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:14.909+0000 7fc46b400640 10 client.17136 handle_command_reply: tid=0 2024-03-21T10:30:14.913 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:14.909+0000 7fc48ef76740 1 client.17136 shutdown 2024-03-21T10:30:14.913 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:14.909+0000 7fc48ef76740 1 -- 172.21.15.47:0/2313266760 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e9488a7c00 msgr2=0x55e948914c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:30:14.913 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:14.909+0000 7fc48ef76740 1 --2- 172.21.15.47:0/2313266760 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e9488a7c00 0x55e948914c90 secure :-1 s=READY pgs=848 cs=0 l=1 rev1=1 crypto rx=0x7fc470009fd0 tx=0x7fc470005f90 comp rx=0 tx=0).stop 2024-03-21T10:30:14.913 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:14.909+0000 7fc48ef76740 1 -- 172.21.15.47:0/2313266760 shutdown_connections 2024-03-21T10:30:14.913 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:14.909+0000 7fc48ef76740 1 --2- 172.21.15.47:0/2313266760 >> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] conn(0x55e94891c5a0 0x55e94891e9c0 crc :-1 s=READY pgs=138 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:30:14.913 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:14.909+0000 7fc48ef76740 1 --2- 172.21.15.47:0/2313266760 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e9488b0290 0x55e9488a9a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:30:14.913 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:14.909+0000 7fc48ef76740 1 --2- 172.21.15.47:0/2313266760 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e9488a7c00 0x55e948914c90 unknown :-1 s=CLOSED pgs=848 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:30:14.913 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:14.909+0000 7fc48ef76740 1 --2- 172.21.15.47:0/2313266760 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e9488a6f20 0x55e948914750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:30:14.913 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:14.909+0000 7fc48ef76740 1 -- 172.21.15.47:0/2313266760 >> 172.21.15.47:0/2313266760 conn(0x55e94886e200 msgr2=0x55e9488a0a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:30:14.914 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:14.909+0000 7fc46b400640 0 client.17136 ms_handle_reset on v2:172.21.15.100:6838/4173168440 2024-03-21T10:30:14.914 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:14.909+0000 7fc48ef76740 1 -- 172.21.15.47:0/2313266760 shutdown_connections 2024-03-21T10:30:14.914 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:14.909+0000 7fc48ef76740 1 -- 172.21.15.47:0/2313266760 wait complete. 2024-03-21T10:30:14.914 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:14.909+0000 7fc48ef76740 20 client.17136 trim_cache size 0 max 16384 2024-03-21T10:30:14.914 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-03-21T10:30:14.914 INFO:teuthology.orchestra.run.smithi047.stdout: "status": "no active scrubs running", 2024-03-21T10:30:14.914 INFO:teuthology.orchestra.run.smithi047.stdout: "scrubs": {} 2024-03-21T10:30:14.914 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-03-21T10:30:14.915 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:14.910+0000 7fc48c200640 1 -- 172.21.15.47:0/2305977590 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7fc46c0a3090 msgr2=0x7fc46c0a5550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:30:14.915 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:14.910+0000 7fc48c200640 1 --2- 172.21.15.47:0/2305977590 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7fc46c0a3090 0x7fc46c0a5550 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fc4840ff8d0 tx=0x7fc470005850 comp rx=0 tx=0).stop 2024-03-21T10:30:14.915 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:14.910+0000 7fc48c200640 1 -- 172.21.15.47:0/2305977590 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc484112620 msgr2=0x7fc4841a2da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:30:14.915 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:14.910+0000 7fc48c200640 1 --2- 172.21.15.47:0/2305977590 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc484112620 0x7fc4841a2da0 secure :-1 s=READY pgs=846 cs=0 l=1 rev1=1 crypto rx=0x7fc47c00e9d0 tx=0x7fc47c00ee90 comp rx=0 tx=0).stop 2024-03-21T10:30:14.915 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:14.910+0000 7fc48c200640 1 -- 172.21.15.47:0/2305977590 shutdown_connections 2024-03-21T10:30:14.915 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:14.910+0000 7fc48c200640 1 --2- 172.21.15.47:0/2305977590 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc484112620 0x7fc4841a2da0 unknown :-1 s=CLOSED pgs=846 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:30:14.916 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:14.910+0000 7fc48c200640 1 --2- 172.21.15.47:0/2305977590 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7fc46c0a3090 0x7fc46c0a5550 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:30:14.916 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:14.910+0000 7fc48c200640 1 --2- 172.21.15.47:0/2305977590 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc484074360 0x7fc48419ea20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:30:14.916 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:14.910+0000 7fc48c200640 1 --2- 172.21.15.47:0/2305977590 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc484073a40 0x7fc48419e4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:30:14.916 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:14.910+0000 7fc48c200640 1 -- 172.21.15.47:0/2305977590 >> 172.21.15.47:0/2305977590 conn(0x7fc4840fd100 msgr2=0x7fc4840fead0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:30:14.916 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:14.910+0000 7fc48c200640 1 -- 172.21.15.47:0/2305977590 shutdown_connections 2024-03-21T10:30:14.916 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:14.910+0000 7fc48c200640 1 -- 172.21.15.47:0/2305977590 wait complete. 2024-03-21T10:30:14.928 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-03-21T10:30:14.928 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-21T10:30:15.140 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:15.139+0000 7f986f000640 1 -- 172.21.15.47:0/862565043 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f98681008c0 msgr2=0x7f9868100ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:30:15.140 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:15.139+0000 7f986f000640 1 --2- 172.21.15.47:0/862565043 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f98681008c0 0x7f9868100ca0 secure :-1 s=READY pgs=849 cs=0 l=1 rev1=1 crypto rx=0x7f985c009900 tx=0x7f985c02bc50 comp rx=0 tx=0).stop 2024-03-21T10:30:15.140 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:15.140+0000 7f986f000640 1 -- 172.21.15.47:0/862565043 shutdown_connections 2024-03-21T10:30:15.140 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:15.140+0000 7f986f000640 1 --2- 172.21.15.47:0/862565043 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f98681056e0 0x7f986810d2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:30:15.140 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:15.140+0000 7f986f000640 1 --2- 172.21.15.47:0/862565043 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9868101270 0x7f9868105000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:30:15.141 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:15.140+0000 7f986f000640 1 --2- 172.21.15.47:0/862565043 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f98681008c0 0x7f9868100ca0 unknown :-1 s=CLOSED pgs=849 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:30:15.141 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:15.140+0000 7f986f000640 1 -- 172.21.15.47:0/862565043 >> 172.21.15.47:0/862565043 conn(0x7f9868075fb0 msgr2=0x7f98680763c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:30:15.141 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:15.140+0000 7f986f000640 1 -- 172.21.15.47:0/862565043 shutdown_connections 2024-03-21T10:30:15.141 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:15.140+0000 7f986f000640 1 -- 172.21.15.47:0/862565043 wait complete. 2024-03-21T10:30:15.142 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:15.141+0000 7f986f000640 1 Processor -- start 2024-03-21T10:30:15.142 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:15.141+0000 7f986f000640 1 -- start start 2024-03-21T10:30:15.142 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:15.141+0000 7f986f000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f98681008c0 0x7f986819d920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:30:15.142 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:15.141+0000 7f986f000640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f98681056e0 0x7f986819de60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:30:15.142 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:15.141+0000 7f986f000640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9868197b20 0x7f9868197fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:30:15.142 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:15.141+0000 7f986f000640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9868110690 con 0x7f98681056e0 2024-03-21T10:30:15.142 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:15.141+0000 7f986f000640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9868110510 con 0x7f98681008c0 2024-03-21T10:30:15.143 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:15.141+0000 7f986f000640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9868110810 con 0x7f9868197b20 2024-03-21T10:30:15.143 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:15.142+0000 7f986d200640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f98681056e0 0x7f986819de60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:30:15.143 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:15.142+0000 7f986dc00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f98681008c0 0x7f986819d920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:30:15.143 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:15.142+0000 7f986d200640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f98681056e0 0x7f986819de60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:38152/0 (socket says 172.21.15.47:38152) 2024-03-21T10:30:15.143 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:15.142+0000 7f986d200640 1 -- 172.21.15.47:0/1495842099 learned_addr learned my addr 172.21.15.47:0/1495842099 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:30:15.143 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:15.142+0000 7f986e600640 1 --2- 172.21.15.47:0/1495842099 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9868197b20 0x7f9868197fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:30:15.143 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:15.142+0000 7f986d200640 1 -- 172.21.15.47:0/1495842099 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9868197b20 msgr2=0x7f9868197fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:30:15.143 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:15.142+0000 7f986d200640 1 --2- 172.21.15.47:0/1495842099 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9868197b20 0x7f9868197fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:30:15.143 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:15.142+0000 7f986d200640 1 -- 172.21.15.47:0/1495842099 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f98681008c0 msgr2=0x7f986819d920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:30:15.143 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:15.142+0000 7f986d200640 1 --2- 172.21.15.47:0/1495842099 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f98681008c0 0x7f986819d920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:30:15.143 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:15.142+0000 7f986d200640 1 -- 172.21.15.47:0/1495842099 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9868198640 con 0x7f98681056e0 2024-03-21T10:30:15.143 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:15.142+0000 7f986e600640 1 --2- 172.21.15.47:0/1495842099 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9868197b20 0x7f9868197fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T10:30:15.144 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:15.142+0000 7f986d200640 1 --2- 172.21.15.47:0/1495842099 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f98681056e0 0x7f986819de60 secure :-1 s=READY pgs=850 cs=0 l=1 rev1=1 crypto rx=0x7f985400d740 tx=0x7f985400dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:30:15.144 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:15.143+0000 7f9866000640 1 -- 172.21.15.47:0/1495842099 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9854017070 con 0x7f98681056e0 2024-03-21T10:30:15.144 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:15.143+0000 7f9866000640 1 -- 172.21.15.47:0/1495842099 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f985400de50 con 0x7f98681056e0 2024-03-21T10:30:15.145 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:15.143+0000 7f9866000640 1 -- 172.21.15.47:0/1495842099 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9854004d60 con 0x7f98681056e0 2024-03-21T10:30:15.145 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:15.143+0000 7f986f000640 1 -- 172.21.15.47:0/1495842099 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9868198930 con 0x7f98681056e0 2024-03-21T10:30:15.145 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:15.143+0000 7f986f000640 1 -- 172.21.15.47:0/1495842099 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f98681022d0 con 0x7f98681056e0 2024-03-21T10:30:15.145 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:15.144+0000 7f986f000640 1 -- 172.21.15.47:0/1495842099 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f98680683b0 con 0x7f98681056e0 2024-03-21T10:30:15.147 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:15.148+0000 7f9866000640 1 -- 172.21.15.47:0/1495842099 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 27) v1 ==== 137985+0+0 (secure 0 0 0) 0x7f9854004f00 con 0x7f98681056e0 2024-03-21T10:30:15.149 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:15.149+0000 7f9866000640 1 --2- 172.21.15.47:0/1495842099 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f98500a3090 0x7f98500a5550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:30:15.149 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:15.149+0000 7f9866000640 1 -- 172.21.15.47:0/1495842099 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 9474+0+0 (secure 0 0 0) 0x7f98540cfa10 con 0x7f98681056e0 2024-03-21T10:30:15.149 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:15.149+0000 7f986dc00640 1 --2- 172.21.15.47:0/1495842099 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f98500a3090 0x7f98500a5550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:30:15.150 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:15.150+0000 7f986dc00640 1 --2- 172.21.15.47:0/1495842099 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f98500a3090 0x7f98500a5550 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f985c005ab0 tx=0x7f985c005a20 comp rx=0 tx=0).ready entity=mgr.15687 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:30:15.153 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:15.153+0000 7f9866000640 1 -- 172.21.15.47:0/1495842099 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f985409c0e0 con 0x7f98681056e0 2024-03-21T10:30:15.402 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:15.402+0000 7f986f000640 1 -- 172.21.15.47:0/1495842099 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f9868101270 con 0x7f98681056e0 2024-03-21T10:30:15.404 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:15.404+0000 7f9866000640 1 -- 172.21.15.47:0/1495842099 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 32 v32) v1 ==== 94+0+11471 (secure 0 0 0) 0x7f985409b9e0 con 0x7f98681056e0 2024-03-21T10:30:15.405 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-21T10:30:15.405 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":32,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14511,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6835/735724037","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6834","nonce":735724037},{"type":"v1","addr":"172.21.15.47:6835","nonce":735724037}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14529,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6837/2248191724","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6836","nonce":2248191724},{"type":"v1","addr":"172.21.15.47:6837","nonce":2248191724}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":32,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-21T10:21:58.329063+0000","modified":"2024-03-21T10:26:17.169892+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":34535,"mds_1":34493,"mds_2":34454,"mds_3":34427,"mds_4":14619},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14604":{"gid":14604,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.47:6839/2881335356","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6838","nonce":2881335356},{"type":"v1","addr":"172.21.15.47:6839","nonce":2881335356}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_14619":{"gid":14619,"name":"a","rank":4,"incarnation":27,"state":"up:active","state_seq":11,"addr":"172.21.15.47:6841/3401578978","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6840","nonce":3401578978},{"type":"v1","addr":"172.21.15.47:6841","nonce":3401578978}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24406":{"gid":24406,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.157:6833/4000010136","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":4000010136},{"type":"v1","addr":"172.21.15.157:6833","nonce":4000010136}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24457":{"gid":24457,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.157:6835/3902868651","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":3902868651},{"type":"v1","addr":"172.21.15.157:6835","nonce":3902868651}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24475":{"gid":24475,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.157:6837/1908087204","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6836","nonce":1908087204},{"type":"v1","addr":"172.21.15.157:6837","nonce":1908087204}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_25729":{"gid":25729,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.157:6839/1998408416","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6838","nonce":1998408416},{"type":"v1","addr":"172.21.15.157:6839","nonce":1998408416}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_34427":{"gid":34427,"name":"k","rank":3,"incarnation":25,"state":"up:active","state_seq":20,"addr":"172.21.15.100:6833/2723302038","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6832","nonce":2723302038},{"type":"v1","addr":"172.21.15.100:6833","nonce":2723302038}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_34454":{"gid":34454,"name":"e","rank":2,"incarnation":22,"state":"up:active","state_seq":17,"addr":"172.21.15.100:6835/2597849957","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":2597849957},{"type":"v1","addr":"172.21.15.100:6835","nonce":2597849957}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_34493":{"gid":34493,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":13,"addr":"172.21.15.100:6837/3497644162","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6836","nonce":3497644162},{"type":"v1","addr":"172.21.15.100:6837","nonce":3497644162}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_34535":{"gid":34535,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.100:6839/4173168440","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6838","nonce":4173168440},{"type":"v1","addr":"172.21.15.100:6839","nonce":4173168440}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":34535,"qdb_cluster":[34427,14619,34454,34535,34493]},"id":1}]} 2024-03-21T10:30:15.405 INFO:teuthology.orchestra.run.smithi047.stderr:dumped fsmap epoch 32 2024-03-21T10:30:15.407 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:15.407+0000 7f986f000640 1 -- 172.21.15.47:0/1495842099 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f98500a3090 msgr2=0x7f98500a5550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:30:15.407 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:15.407+0000 7f986f000640 1 --2- 172.21.15.47:0/1495842099 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f98500a3090 0x7f98500a5550 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f985c005ab0 tx=0x7f985c005a20 comp rx=0 tx=0).stop 2024-03-21T10:30:15.407 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:15.407+0000 7f986f000640 1 -- 172.21.15.47:0/1495842099 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f98681056e0 msgr2=0x7f986819de60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:30:15.407 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:15.407+0000 7f986f000640 1 --2- 172.21.15.47:0/1495842099 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f98681056e0 0x7f986819de60 secure :-1 s=READY pgs=850 cs=0 l=1 rev1=1 crypto rx=0x7f985400d740 tx=0x7f985400dc00 comp rx=0 tx=0).stop 2024-03-21T10:30:15.408 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:15.407+0000 7f986f000640 1 -- 172.21.15.47:0/1495842099 shutdown_connections 2024-03-21T10:30:15.408 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:15.407+0000 7f986f000640 1 --2- 172.21.15.47:0/1495842099 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9868197b20 0x7f9868197fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:30:15.408 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:15.407+0000 7f986f000640 1 --2- 172.21.15.47:0/1495842099 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f98681056e0 0x7f986819de60 unknown :-1 s=CLOSED pgs=850 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:30:15.408 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:15.407+0000 7f986f000640 1 --2- 172.21.15.47:0/1495842099 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f98500a3090 0x7f98500a5550 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:30:15.408 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:15.407+0000 7f986f000640 1 --2- 172.21.15.47:0/1495842099 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f98681008c0 0x7f986819d920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:30:15.408 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:15.407+0000 7f986f000640 1 -- 172.21.15.47:0/1495842099 >> 172.21.15.47:0/1495842099 conn(0x7f9868075fb0 msgr2=0x7f9868068be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:30:15.408 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:15.408+0000 7f986f000640 1 -- 172.21.15.47:0/1495842099 shutdown_connections 2024-03-21T10:30:15.408 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:15.408+0000 7f986f000640 1 -- 172.21.15.47:0/1495842099 wait complete. 2024-03-21T10:30:15.424 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:30:15 smithi047 ceph-mon[31547]: pgmap v151: 129 pgs: 129 active+clean; 277 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 170 B/s wr, 8 op/s 2024-03-21T10:30:15.428 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-21T10:30:15.428 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-03-21T10:30:15.477 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:30:15 smithi100 ceph-mon[35193]: pgmap v151: 129 pgs: 129 active+clean; 277 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 170 B/s wr, 8 op/s 2024-03-21T10:30:15.487 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:30:15 smithi157 ceph-mon[35581]: pgmap v151: 129 pgs: 129 active+clean; 277 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 170 B/s wr, 8 op/s 2024-03-21T10:30:15.648 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:15.648+0000 7f6290200640 1 -- 172.21.15.47:0/1380203880 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6288104200 msgr2=0x7f6288110710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:30:15.649 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:15.648+0000 7f6290200640 1 --2- 172.21.15.47:0/1380203880 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6288104200 0x7f6288110710 secure :-1 s=READY pgs=851 cs=0 l=1 rev1=1 crypto rx=0x7f6280009cd0 tx=0x7f628002c0e0 comp rx=0 tx=0).stop 2024-03-21T10:30:15.649 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:15.648+0000 7f6290200640 1 -- 172.21.15.47:0/1380203880 shutdown_connections 2024-03-21T10:30:15.649 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:15.648+0000 7f6290200640 1 --2- 172.21.15.47:0/1380203880 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6288104200 0x7f6288110710 unknown :-1 s=CLOSED pgs=851 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:30:15.649 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:15.648+0000 7f6290200640 1 --2- 172.21.15.47:0/1380203880 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6288103840 0x7f6288103cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:30:15.649 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:15.648+0000 7f6290200640 1 --2- 172.21.15.47:0/1380203880 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6288109850 0x7f6288109c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:30:15.649 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:15.648+0000 7f6290200640 1 -- 172.21.15.47:0/1380203880 >> 172.21.15.47:0/1380203880 conn(0x7f62880ff2f0 msgr2=0x7f6288101710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:30:15.649 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:15.648+0000 7f6290200640 1 -- 172.21.15.47:0/1380203880 shutdown_connections 2024-03-21T10:30:15.649 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:15.648+0000 7f6290200640 1 -- 172.21.15.47:0/1380203880 wait complete. 2024-03-21T10:30:15.651 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:15.649+0000 7f6290200640 1 Processor -- start 2024-03-21T10:30:15.652 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:15.649+0000 7f6290200640 1 -- start start 2024-03-21T10:30:15.652 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:15.650+0000 7f6290200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6288103840 0x7f628819a030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:30:15.652 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:15.650+0000 7f6290200640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6288104200 0x7f628819a570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:30:15.652 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:15.650+0000 7f6290200640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6288109850 0x7f628819e8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:30:15.652 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:15.650+0000 7f6290200640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6288113b10 con 0x7f6288109850 2024-03-21T10:30:15.652 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:15.650+0000 7f6290200640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6288113990 con 0x7f6288103840 2024-03-21T10:30:15.652 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:15.650+0000 7f6290200640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6288113c90 con 0x7f6288104200 2024-03-21T10:30:15.652 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:15.650+0000 7f628e200640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6288109850 0x7f628819e8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:30:15.652 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:15.650+0000 7f628e200640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6288109850 0x7f628819e8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:38168/0 (socket says 172.21.15.47:38168) 2024-03-21T10:30:15.652 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:15.650+0000 7f628d800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6288103840 0x7f628819a030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:30:15.652 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:15.650+0000 7f628e200640 1 -- 172.21.15.47:0/3833902664 learned_addr learned my addr 172.21.15.47:0/3833902664 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:30:15.652 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:15.650+0000 7f628ce00640 1 --2- 172.21.15.47:0/3833902664 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6288104200 0x7f628819a570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:30:15.652 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:15.651+0000 7f628e200640 1 -- 172.21.15.47:0/3833902664 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6288104200 msgr2=0x7f628819a570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:30:15.652 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:15.651+0000 7f628e200640 1 --2- 172.21.15.47:0/3833902664 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6288104200 0x7f628819a570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:30:15.653 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:15.651+0000 7f628e200640 1 -- 172.21.15.47:0/3833902664 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6288103840 msgr2=0x7f628819a030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:30:15.653 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:15.651+0000 7f628e200640 1 --2- 172.21.15.47:0/3833902664 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6288103840 0x7f628819a030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:30:15.653 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:15.651+0000 7f628e200640 1 -- 172.21.15.47:0/3833902664 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f628819f020 con 0x7f6288109850 2024-03-21T10:30:15.653 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:15.651+0000 7f628e200640 1 --2- 172.21.15.47:0/3833902664 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6288109850 0x7f628819e8f0 secure :-1 s=READY pgs=852 cs=0 l=1 rev1=1 crypto rx=0x7f6280004af0 tx=0x7f6280035ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:30:15.653 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:15.651+0000 7f6286000640 1 -- 172.21.15.47:0/3833902664 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6280039040 con 0x7f6288109850 2024-03-21T10:30:15.653 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:15.651+0000 7f6290200640 1 -- 172.21.15.47:0/3833902664 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f628819f2b0 con 0x7f6288109850 2024-03-21T10:30:15.653 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:15.651+0000 7f6286000640 1 -- 172.21.15.47:0/3833902664 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f6280044070 con 0x7f6288109850 2024-03-21T10:30:15.653 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:15.651+0000 7f6286000640 1 -- 172.21.15.47:0/3833902664 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62800043c0 con 0x7f6288109850 2024-03-21T10:30:15.653 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:15.651+0000 7f6290200640 1 -- 172.21.15.47:0/3833902664 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f62881a6ba0 con 0x7f6288109850 2024-03-21T10:30:15.654 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:15.655+0000 7f6292e16740 1 Processor -- start 2024-03-21T10:30:15.655 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:15.655+0000 7f6286000640 1 -- 172.21.15.47:0/3833902664 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 27) v1 ==== 137985+0+0 (secure 0 0 0) 0x7f6280004560 con 0x7f6288109850 2024-03-21T10:30:15.655 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:15.655+0000 7f6292e16740 1 -- start start 2024-03-21T10:30:15.655 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:15.655+0000 7f6292e16740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56076c20c6d0 0x56076c20cab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:30:15.656 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:15.655+0000 7f6292e16740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56076c20d080 0x56076c215630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:30:15.656 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:15.655+0000 7f6292e16740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56076c215c00 0x56076c218050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:30:15.656 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:15.655+0000 7f6292e16740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x56076c1702c0 con 0x56076c20c6d0 2024-03-21T10:30:15.656 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:15.655+0000 7f6292e16740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x56076c118c90 con 0x56076c215c00 2024-03-21T10:30:15.656 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:15.655+0000 7f6292e16740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x56076c16c360 con 0x56076c20d080 2024-03-21T10:30:15.657 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:15.656+0000 7f628ce00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56076c20c6d0 0x56076c20cab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:30:15.657 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:15.656+0000 7f628ce00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56076c20c6d0 0x56076c20cab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:38182/0 (socket says 172.21.15.47:38182) 2024-03-21T10:30:15.657 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:15.656+0000 7f628ce00640 1 -- 172.21.15.47:0/3049142593 learned_addr learned my addr 172.21.15.47:0/3049142593 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:30:15.657 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:15.656+0000 7f628d800640 1 --2- 172.21.15.47:0/3049142593 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56076c20d080 0x56076c215630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:30:15.657 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:15.656+0000 7f628ce00640 1 --2- 172.21.15.47:0/3049142593 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56076c215c00 0x56076c218050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:30:15.658 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:15.656+0000 7f628ce00640 1 -- 172.21.15.47:0/3049142593 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56076c20d080 msgr2=0x56076c215630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:30:15.658 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:15.656+0000 7f628ce00640 1 --2- 172.21.15.47:0/3049142593 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56076c20d080 0x56076c215630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:30:15.658 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:15.656+0000 7f628ce00640 1 -- 172.21.15.47:0/3049142593 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56076c215c00 msgr2=0x56076c218050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:30:15.658 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:15.656+0000 7f628ce00640 1 --2- 172.21.15.47:0/3049142593 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56076c215c00 0x56076c218050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:30:15.658 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:15.656+0000 7f628ce00640 1 -- 172.21.15.47:0/3049142593 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56076c13d080 con 0x56076c20c6d0 2024-03-21T10:30:15.658 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:15.657+0000 7f628ce00640 1 --2- 172.21.15.47:0/3049142593 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56076c20c6d0 0x56076c20cab0 secure :-1 s=READY pgs=853 cs=0 l=1 rev1=1 crypto rx=0x7f62740135d0 tx=0x7f627400b670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6d5a6eb162ca8c39 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:30:15.658 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:15.657+0000 7f627aa00640 1 -- 172.21.15.47:0/3049142593 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6274019030 con 0x56076c20c6d0 2024-03-21T10:30:15.658 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:15.657+0000 7f627aa00640 1 -- 172.21.15.47:0/3049142593 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f6274011970 con 0x56076c20c6d0 2024-03-21T10:30:15.658 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:15.657+0000 7f627aa00640 1 -- 172.21.15.47:0/3049142593 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f627401d880 con 0x56076c20c6d0 2024-03-21T10:30:15.658 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:15.657+0000 7f6286000640 1 --2- 172.21.15.47:0/3833902664 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f62700a3110 0x7f62700a55d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:30:15.662 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:15.658+0000 7f6286000640 1 -- 172.21.15.47:0/3833902664 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 9474+0+0 (secure 0 0 0) 0x7f62800f1e00 con 0x7f6288109850 2024-03-21T10:30:15.662 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:15.658+0000 7f628d800640 1 --2- 172.21.15.47:0/3833902664 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f62700a3110 0x7f62700a55d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:30:15.662 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:15.658+0000 7f6292e16740 1 -- 172.21.15.47:0/3049142593 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56076c20c6d0 msgr2=0x56076c20cab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:30:15.662 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:15.658+0000 7f6292e16740 1 --2- 172.21.15.47:0/3049142593 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56076c20c6d0 0x56076c20cab0 secure :-1 s=READY pgs=853 cs=0 l=1 rev1=1 crypto rx=0x7f62740135d0 tx=0x7f627400b670 comp rx=0 tx=0).stop 2024-03-21T10:30:15.662 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:15.658+0000 7f6292e16740 1 -- 172.21.15.47:0/3049142593 shutdown_connections 2024-03-21T10:30:15.662 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:15.658+0000 7f6292e16740 1 --2- 172.21.15.47:0/3049142593 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56076c215c00 0x56076c218050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:30:15.662 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:15.658+0000 7f6292e16740 1 --2- 172.21.15.47:0/3049142593 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56076c20d080 0x56076c215630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:30:15.662 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:15.658+0000 7f6292e16740 1 --2- 172.21.15.47:0/3049142593 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56076c20c6d0 0x56076c20cab0 unknown :-1 s=CLOSED pgs=853 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:30:15.663 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:15.658+0000 7f6292e16740 1 -- 172.21.15.47:0/3049142593 >> 172.21.15.47:0/3049142593 conn(0x56076c206da0 msgr2=0x56076c207180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:30:15.663 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:15.658+0000 7f6292e16740 1 -- 172.21.15.47:0/3049142593 shutdown_connections 2024-03-21T10:30:15.663 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:15.658+0000 7f6292e16740 1 -- 172.21.15.47:0/3049142593 wait complete. 2024-03-21T10:30:15.663 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:15.659+0000 7f628d800640 1 --2- 172.21.15.47:0/3833902664 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f62700a3110 0x7f62700a55d0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x56076c207e10 tx=0x7f627c00b390 comp rx=0 tx=0).ready entity=mgr.15687 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:30:15.663 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:15.659+0000 7f6292e16740 1 -- start start 2024-03-21T10:30:15.663 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:15.659+0000 7f6292e16740 1 Processor -- start 2024-03-21T10:30:15.663 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:15.659+0000 7f6292e16740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56076c20c6d0 0x56076c215540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:30:15.663 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:15.659+0000 7f6292e16740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56076c20d080 0x56076c2128d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:30:15.663 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:15.659+0000 7f6292e16740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56076c215c00 0x56076c212ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:30:15.663 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:15.659+0000 7f6292e16740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x56076c218b70 con 0x56076c215c00 2024-03-21T10:30:15.663 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:15.659+0000 7f6292e16740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x56076c12c0d0 con 0x56076c20d080 2024-03-21T10:30:15.663 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:15.659+0000 7f6292e16740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x56076c218cf0 con 0x56076c20c6d0 2024-03-21T10:30:15.664 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:15.659+0000 7f6292e16740 10 client.0 _ll_register_callbacks cb 0x56076c130e80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T10:30:15.664 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:15.660+0000 7f628e200640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56076c215c00 0x56076c212ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:30:15.664 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:15.660+0000 7f628ce00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56076c20d080 0x56076c2128d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:30:15.664 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:15.660+0000 7f628e200640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56076c215c00 0x56076c212ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:38184/0 (socket says 172.21.15.47:38184) 2024-03-21T10:30:15.664 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:15.660+0000 7f628e200640 1 -- 172.21.15.47:0/2342272282 learned_addr learned my addr 172.21.15.47:0/2342272282 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:30:15.664 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:15.660+0000 7f628d800640 1 --2- 172.21.15.47:0/2342272282 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56076c20c6d0 0x56076c215540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:30:15.664 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:15.660+0000 7f628e200640 1 -- 172.21.15.47:0/2342272282 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56076c20c6d0 msgr2=0x56076c215540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:30:15.664 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:15.660+0000 7f628e200640 1 --2- 172.21.15.47:0/2342272282 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56076c20c6d0 0x56076c215540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:30:15.664 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:15.660+0000 7f628e200640 1 -- 172.21.15.47:0/2342272282 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56076c20d080 msgr2=0x56076c2128d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:30:15.664 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:15.660+0000 7f628e200640 1 --2- 172.21.15.47:0/2342272282 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56076c20d080 0x56076c2128d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:30:15.664 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:15.660+0000 7f628e200640 1 -- 172.21.15.47:0/2342272282 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56076be843f0 con 0x56076c215c00 2024-03-21T10:30:15.665 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:15.660+0000 7f628e200640 1 --2- 172.21.15.47:0/2342272282 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56076c215c00 0x56076c212ee0 secure :-1 s=READY pgs=854 cs=0 l=1 rev1=1 crypto rx=0x7f62800ba8b0 tx=0x7f62800098d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c1759a6c9a077fcb server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:30:15.665 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:15.660+0000 7f626f400640 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-03-21T10:30:15.665 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:15.661+0000 7f626f400640 1 -- 172.21.15.47:0/2342272282 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62800bdee0 con 0x56076c215c00 2024-03-21T10:30:15.665 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:15.661+0000 7f626f400640 1 -- 172.21.15.47:0/2342272282 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(97..97 src has 1..97) v4 ==== 9474+0+0 (secure 0 0 0) 0x7f62800ba550 con 0x56076c215c00 2024-03-21T10:30:15.665 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:15.661+0000 7f6292e16740 1 -- 172.21.15.47:0/2342272282 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f6274011970 con 0x56076c215c00 2024-03-21T10:30:15.665 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:15.661+0000 7f626f400640 1 -- 172.21.15.47:0/2342272282 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f62800c2030 con 0x56076c215c00 2024-03-21T10:30:15.665 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:15.661+0000 7f6292e16740 10 client.17154 fetch_fsmap learned FSMap version 32 2024-03-21T10:30:15.665 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:15.661+0000 7f6292e16740 1 -- 172.21.15.47:0/2342272282 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x56076bf4e7d0 con 0x56076c215c00 2024-03-21T10:30:15.665 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:15.661+0000 7f626f400640 1 -- 172.21.15.47:0/2342272282 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f6274011970 con 0x56076c215c00 2024-03-21T10:30:15.665 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:15.662+0000 7f626f400640 1 -- 172.21.15.47:0/2342272282 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 32) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f628003c030 con 0x56076c215c00 2024-03-21T10:30:15.665 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:15.662+0000 7f6292e16740 10 client.17154 fetch_fsmap finished waiting for FSMap version 32 2024-03-21T10:30:15.665 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:15.662+0000 7f6292e16740 10 client.17154 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.a 2024-03-21T10:30:15.665 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:15.662+0000 7f6292e16740 20 client.17154 populate_metadata read hostname 'smithi047' 2024-03-21T10:30:15.665 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:15.662+0000 7f6292e16740 1 --2- 172.21.15.47:0/2342272282 >> [v2:172.21.15.47:6840/3401578978,v1:172.21.15.47:6841/3401578978] conn(0x56076c225a00 0x56076c227df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:30:15.666 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:15.662+0000 7f6292e16740 4 client.17154 mds_command: new command op to 14619 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T10:30:15.666 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:15.662+0000 7f6292e16740 1 -- 172.21.15.47:0/2342272282 --> [v2:172.21.15.47:6840/3401578978,v1:172.21.15.47:6841/3401578978] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56076c17d800 con 0x56076c225a00 2024-03-21T10:30:15.666 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:15.662+0000 7f628d800640 1 --2- 172.21.15.47:0/2342272282 >> [v2:172.21.15.47:6840/3401578978,v1:172.21.15.47:6841/3401578978] conn(0x56076c225a00 0x56076c227df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:30:15.667 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:15.663+0000 7f628d800640 1 --2- 172.21.15.47:0/2342272282 >> [v2:172.21.15.47:6840/3401578978,v1:172.21.15.47:6841/3401578978] conn(0x56076c225a00 0x56076c227df0 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=a223be0c2897c6f9 server_cookie=1349917f38b8a2cb in_seq=0 out_seq=0 2024-03-21T10:30:15.667 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:15.663+0000 7f626f400640 10 client.17154 ms_handle_connect on v2:172.21.15.47:6840/3401578978 2024-03-21T10:30:15.667 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:15.664+0000 7f626f400640 1 -- 172.21.15.47:0/2342272282 <== mds.4 v2:172.21.15.47:6840/3401578978 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x56076c17d800 con 0x56076c225a00 2024-03-21T10:30:15.667 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:15.664+0000 7f626f400640 10 client.17154 handle_command_reply: tid=0 2024-03-21T10:30:15.667 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:15.664+0000 7f6292e16740 1 client.17154 shutdown 2024-03-21T10:30:15.667 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:15.664+0000 7f6292e16740 1 -- 172.21.15.47:0/2342272282 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56076c215c00 msgr2=0x56076c212ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:30:15.667 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:15.664+0000 7f6292e16740 1 --2- 172.21.15.47:0/2342272282 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56076c215c00 0x56076c212ee0 secure :-1 s=READY pgs=854 cs=0 l=1 rev1=1 crypto rx=0x7f62800ba8b0 tx=0x7f62800098d0 comp rx=0 tx=0).stop 2024-03-21T10:30:15.668 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:15.664+0000 7f6292e16740 1 -- 172.21.15.47:0/2342272282 shutdown_connections 2024-03-21T10:30:15.668 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:15.664+0000 7f6292e16740 1 --2- 172.21.15.47:0/2342272282 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56076c215c00 0x56076c212ee0 unknown :-1 s=CLOSED pgs=854 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:30:15.668 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:15.664+0000 7f6292e16740 1 --2- 172.21.15.47:0/2342272282 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56076c20d080 0x56076c2128d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:30:15.668 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:15.664+0000 7f6292e16740 1 --2- 172.21.15.47:0/2342272282 >> [v2:172.21.15.47:6840/3401578978,v1:172.21.15.47:6841/3401578978] conn(0x56076c225a00 0x56076c227df0 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:30:15.668 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:15.664+0000 7f6292e16740 1 --2- 172.21.15.47:0/2342272282 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56076c20c6d0 0x56076c215540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:30:15.668 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:15.664+0000 7f6292e16740 1 -- 172.21.15.47:0/2342272282 >> 172.21.15.47:0/2342272282 conn(0x56076c206da0 msgr2=0x56076c208af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:30:15.668 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:15.664+0000 7f626f400640 0 client.17154 ms_handle_reset on v2:172.21.15.47:6840/3401578978 2024-03-21T10:30:15.668 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:15.664+0000 7f6292e16740 1 -- 172.21.15.47:0/2342272282 shutdown_connections 2024-03-21T10:30:15.668 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:15.664+0000 7f6292e16740 1 -- 172.21.15.47:0/2342272282 wait complete. 2024-03-21T10:30:15.668 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:15.664+0000 7f6292e16740 20 client.17154 trim_cache size 0 max 16384 2024-03-21T10:30:15.679 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:15.678+0000 7f6292e16740 1 Processor -- start 2024-03-21T10:30:15.680 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:15.678+0000 7f6292e16740 1 -- start start 2024-03-21T10:30:15.680 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:15.678+0000 7f6292e16740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56076c20c6d0 0x56076c215000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:30:15.680 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:15.679+0000 7f6292e16740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56076c20d080 0x56076c215540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:30:15.680 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:15.679+0000 7f6292e16740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56076c215c00 0x56076c20fb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:30:15.680 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:15.679+0000 7f6292e16740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62881b0a60 con 0x56076c215c00 2024-03-21T10:30:15.680 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:15.679+0000 7f6292e16740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62881b23e0 con 0x56076c20c6d0 2024-03-21T10:30:15.680 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:15.679+0000 7f6292e16740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62881b10c0 con 0x56076c20d080 2024-03-21T10:30:15.680 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:15.679+0000 7f628e200640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56076c215c00 0x56076c20fb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:30:15.680 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:15.679+0000 7f628e200640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56076c215c00 0x56076c20fb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:38196/0 (socket says 172.21.15.47:38196) 2024-03-21T10:30:15.680 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:15.679+0000 7f628e200640 1 -- 172.21.15.47:0/3764651600 learned_addr learned my addr 172.21.15.47:0/3764651600 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:30:15.680 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:15.679+0000 7f628d800640 1 --2- 172.21.15.47:0/3764651600 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56076c20c6d0 0x56076c215000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:30:15.680 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:15.679+0000 7f628ce00640 1 --2- 172.21.15.47:0/3764651600 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56076c20d080 0x56076c215540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:30:15.680 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:15.679+0000 7f628e200640 1 -- 172.21.15.47:0/3764651600 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56076c20d080 msgr2=0x56076c215540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:30:15.681 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:15.679+0000 7f628e200640 1 --2- 172.21.15.47:0/3764651600 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56076c20d080 0x56076c215540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:30:15.681 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:15.679+0000 7f628e200640 1 -- 172.21.15.47:0/3764651600 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56076c20c6d0 msgr2=0x56076c215000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:30:15.681 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:15.679+0000 7f628e200640 1 --2- 172.21.15.47:0/3764651600 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56076c20c6d0 0x56076c215000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:30:15.681 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:15.679+0000 7f628e200640 1 -- 172.21.15.47:0/3764651600 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56076c228450 con 0x56076c215c00 2024-03-21T10:30:15.681 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:15.679+0000 7f628d800640 1 --2- 172.21.15.47:0/3764651600 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56076c20c6d0 0x56076c215000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T10:30:15.685 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:15.680+0000 7f628e200640 1 --2- 172.21.15.47:0/3764651600 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56076c215c00 0x56076c20fb50 secure :-1 s=READY pgs=855 cs=0 l=1 rev1=1 crypto rx=0x7f62800bdbe0 tx=0x7f6280005f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fc6a659b68c3c680 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:30:15.685 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:15.680+0000 7f627a000640 1 -- 172.21.15.47:0/3764651600 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62800bdee0 con 0x56076c215c00 2024-03-21T10:30:15.685 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:15.680+0000 7f627a000640 1 -- 172.21.15.47:0/3764651600 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f62800c68f0 con 0x56076c215c00 2024-03-21T10:30:15.685 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:15.680+0000 7f627a000640 1 -- 172.21.15.47:0/3764651600 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62800bd530 con 0x56076c215c00 2024-03-21T10:30:15.685 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:15.681+0000 7f6292e16740 1 -- 172.21.15.47:0/3764651600 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56076c215c00 msgr2=0x56076c20fb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:30:15.685 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:15.681+0000 7f6292e16740 1 --2- 172.21.15.47:0/3764651600 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56076c215c00 0x56076c20fb50 secure :-1 s=READY pgs=855 cs=0 l=1 rev1=1 crypto rx=0x7f62800bdbe0 tx=0x7f6280005f60 comp rx=0 tx=0).stop 2024-03-21T10:30:15.685 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:15.681+0000 7f6292e16740 1 -- 172.21.15.47:0/3764651600 shutdown_connections 2024-03-21T10:30:15.685 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:15.681+0000 7f6292e16740 1 --2- 172.21.15.47:0/3764651600 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56076c215c00 0x56076c20fb50 unknown :-1 s=CLOSED pgs=855 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:30:15.686 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:15.681+0000 7f6292e16740 1 --2- 172.21.15.47:0/3764651600 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56076c20d080 0x56076c215540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:30:15.686 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:15.681+0000 7f6292e16740 1 --2- 172.21.15.47:0/3764651600 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56076c20c6d0 0x56076c215000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:30:15.686 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:15.681+0000 7f6292e16740 1 -- 172.21.15.47:0/3764651600 >> 172.21.15.47:0/3764651600 conn(0x56076c206da0 msgr2=0x56076c205eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:30:15.686 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:15.681+0000 7f6292e16740 1 -- 172.21.15.47:0/3764651600 shutdown_connections 2024-03-21T10:30:15.686 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:15.681+0000 7f6292e16740 1 -- 172.21.15.47:0/3764651600 wait complete. 2024-03-21T10:30:15.686 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:15.681+0000 7f6292e16740 1 -- start start 2024-03-21T10:30:15.686 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:15.681+0000 7f6292e16740 1 Processor -- start 2024-03-21T10:30:15.686 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:15.682+0000 7f6292e16740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56076c20c6d0 0x56076c21bb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:30:15.686 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:15.682+0000 7f6292e16740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56076c20d080 0x56076c21c0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:30:15.686 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:15.682+0000 7f6292e16740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56076c215c00 0x56076c2802b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:30:15.686 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:15.682+0000 7f6292e16740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62881b1d80 con 0x56076c215c00 2024-03-21T10:30:15.686 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:15.682+0000 7f6292e16740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62881b1720 con 0x56076c20c6d0 2024-03-21T10:30:15.686 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:15.682+0000 7f6292e16740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62881aec20 con 0x56076c20d080 2024-03-21T10:30:15.687 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:15.682+0000 7f6292e16740 10 client.0 _ll_register_callbacks cb 0x7f62800bda20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T10:30:15.687 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:15.682+0000 7f628e200640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56076c215c00 0x56076c2802b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:30:15.687 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:15.682+0000 7f628e200640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56076c215c00 0x56076c2802b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:38200/0 (socket says 172.21.15.47:38200) 2024-03-21T10:30:15.687 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:15.682+0000 7f628e200640 1 -- 172.21.15.47:0/1795226577 learned_addr learned my addr 172.21.15.47:0/1795226577 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:30:15.687 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:15.682+0000 7f628d800640 1 --2- 172.21.15.47:0/1795226577 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56076c20c6d0 0x56076c21bb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:30:15.687 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:15.683+0000 7f628ce00640 1 --2- 172.21.15.47:0/1795226577 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56076c20d080 0x56076c21c0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:30:15.687 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:15.683+0000 7f628e200640 1 -- 172.21.15.47:0/1795226577 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56076c20d080 msgr2=0x56076c21c0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:30:15.687 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:15.683+0000 7f628e200640 1 --2- 172.21.15.47:0/1795226577 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56076c20d080 0x56076c21c0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:30:15.687 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:15.683+0000 7f628e200640 1 -- 172.21.15.47:0/1795226577 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56076c20c6d0 msgr2=0x56076c21bb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:30:15.687 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:15.683+0000 7f628e200640 1 --2- 172.21.15.47:0/1795226577 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56076c20c6d0 0x56076c21bb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:30:15.687 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:15.683+0000 7f628e200640 1 -- 172.21.15.47:0/1795226577 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56076c213250 con 0x56076c215c00 2024-03-21T10:30:15.687 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:15.683+0000 7f628d800640 1 --2- 172.21.15.47:0/1795226577 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56076c20c6d0 0x56076c21bb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T10:30:15.687 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:15.683+0000 7f628e200640 1 --2- 172.21.15.47:0/1795226577 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56076c215c00 0x56076c2802b0 secure :-1 s=READY pgs=856 cs=0 l=1 rev1=1 crypto rx=0x7f62800c6670 tx=0x7f62800098d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1983a514bd87d2e5 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:30:15.687 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:15.683+0000 7f626f400640 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-03-21T10:30:15.687 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:15.683+0000 7f626f400640 1 -- 172.21.15.47:0/1795226577 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62800de0e0 con 0x56076c215c00 2024-03-21T10:30:15.688 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:15.683+0000 7f626f400640 1 -- 172.21.15.47:0/1795226577 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(97..97 src has 1..97) v4 ==== 9474+0+0 (secure 0 0 0) 0x7f62800dc8c0 con 0x56076c215c00 2024-03-21T10:30:15.688 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:15.683+0000 7f6292e16740 1 -- 172.21.15.47:0/1795226577 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f62800c68f0 con 0x56076c215c00 2024-03-21T10:30:15.688 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:15.684+0000 7f626f400640 1 -- 172.21.15.47:0/1795226577 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f62800bd330 con 0x56076c215c00 2024-03-21T10:30:15.688 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:15.684+0000 7f6292e16740 10 client.17160 fetch_fsmap learned FSMap version 32 2024-03-21T10:30:15.688 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:15.684+0000 7f6292e16740 1 -- 172.21.15.47:0/1795226577 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x56076c213250 con 0x56076c215c00 2024-03-21T10:30:15.688 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:15.684+0000 7f626f400640 1 -- 172.21.15.47:0/1795226577 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f62800c68f0 con 0x56076c215c00 2024-03-21T10:30:15.688 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:15.684+0000 7f626f400640 1 -- 172.21.15.47:0/1795226577 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 32) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f62800bcf60 con 0x56076c215c00 2024-03-21T10:30:15.688 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:15.684+0000 7f6292e16740 10 client.17160 fetch_fsmap finished waiting for FSMap version 32 2024-03-21T10:30:15.688 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:15.684+0000 7f6292e16740 10 client.17160 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.a 2024-03-21T10:30:15.688 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:15.684+0000 7f6292e16740 20 client.17160 populate_metadata read hostname 'smithi047' 2024-03-21T10:30:15.688 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:15.684+0000 7f6292e16740 1 --2- 172.21.15.47:0/1795226577 >> [v2:172.21.15.47:6840/3401578978,v1:172.21.15.47:6841/3401578978] conn(0x56076c281ca0 0x56076c284090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:30:15.688 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:15.684+0000 7f6292e16740 4 client.17160 mds_command: new command op to 14619 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-21T10:30:15.688 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:15.684+0000 7f6292e16740 1 -- 172.21.15.47:0/1795226577 --> [v2:172.21.15.47:6840/3401578978,v1:172.21.15.47:6841/3401578978] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x56076c177b50 con 0x56076c281ca0 2024-03-21T10:30:15.688 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:15.685+0000 7f628d800640 1 --2- 172.21.15.47:0/1795226577 >> [v2:172.21.15.47:6840/3401578978,v1:172.21.15.47:6841/3401578978] conn(0x56076c281ca0 0x56076c284090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:30:15.689 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:15.685+0000 7f628d800640 1 --2- 172.21.15.47:0/1795226577 >> [v2:172.21.15.47:6840/3401578978,v1:172.21.15.47:6841/3401578978] conn(0x56076c281ca0 0x56076c284090 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=5a75eb38fd758cdd server_cookie=624f61c309fc00c in_seq=0 out_seq=0 2024-03-21T10:30:15.689 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:15.685+0000 7f626f400640 10 client.17160 ms_handle_connect on v2:172.21.15.47:6840/3401578978 2024-03-21T10:30:15.690 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:15.685+0000 7f626f400640 1 -- 172.21.15.47:0/1795226577 <== mds.4 v2:172.21.15.47:6840/3401578978 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x56076c177b50 con 0x56076c281ca0 2024-03-21T10:30:15.690 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:15.685+0000 7f626f400640 10 client.17160 handle_command_reply: tid=0 2024-03-21T10:30:15.690 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:15.685+0000 7f6292e16740 1 client.17160 shutdown 2024-03-21T10:30:15.690 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:15.686+0000 7f6292e16740 1 -- 172.21.15.47:0/1795226577 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56076c215c00 msgr2=0x56076c2802b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:30:15.690 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:15.686+0000 7f6292e16740 1 --2- 172.21.15.47:0/1795226577 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56076c215c00 0x56076c2802b0 secure :-1 s=READY pgs=856 cs=0 l=1 rev1=1 crypto rx=0x7f62800c6670 tx=0x7f62800098d0 comp rx=0 tx=0).stop 2024-03-21T10:30:15.690 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:15.686+0000 7f6292e16740 1 -- 172.21.15.47:0/1795226577 shutdown_connections 2024-03-21T10:30:15.690 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:15.686+0000 7f6292e16740 1 --2- 172.21.15.47:0/1795226577 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56076c215c00 0x56076c2802b0 unknown :-1 s=CLOSED pgs=856 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:30:15.690 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:15.686+0000 7f6292e16740 1 --2- 172.21.15.47:0/1795226577 >> [v2:172.21.15.47:6840/3401578978,v1:172.21.15.47:6841/3401578978] conn(0x56076c281ca0 0x56076c284090 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:30:15.690 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:15.686+0000 7f6292e16740 1 --2- 172.21.15.47:0/1795226577 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56076c20d080 0x56076c21c0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:30:15.690 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:15.686+0000 7f6292e16740 1 --2- 172.21.15.47:0/1795226577 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56076c20c6d0 0x56076c21bb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:30:15.691 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:15.686+0000 7f6292e16740 1 -- 172.21.15.47:0/1795226577 >> 172.21.15.47:0/1795226577 conn(0x56076c206da0 msgr2=0x56076c208060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:30:15.691 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:15.686+0000 7f626f400640 0 client.17160 ms_handle_reset on v2:172.21.15.47:6840/3401578978 2024-03-21T10:30:15.691 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:15.686+0000 7f6292e16740 1 -- 172.21.15.47:0/1795226577 shutdown_connections 2024-03-21T10:30:15.691 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:15.686+0000 7f6292e16740 1 -- 172.21.15.47:0/1795226577 wait complete. 2024-03-21T10:30:15.691 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:15.686+0000 7f6292e16740 20 client.17160 trim_cache size 0 max 16384 2024-03-21T10:30:15.691 INFO:teuthology.orchestra.run.smithi047.stdout:[] 2024-03-21T10:30:15.692 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:15.687+0000 7f6290200640 1 -- 172.21.15.47:0/3833902664 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f62700a3110 msgr2=0x7f62700a55d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:30:15.692 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:15.687+0000 7f6290200640 1 --2- 172.21.15.47:0/3833902664 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f62700a3110 0x7f62700a55d0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x56076c207e10 tx=0x7f627c00b390 comp rx=0 tx=0).stop 2024-03-21T10:30:15.692 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:15.687+0000 7f6290200640 1 -- 172.21.15.47:0/3833902664 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6288109850 msgr2=0x7f628819e8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:30:15.692 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:15.687+0000 7f6290200640 1 --2- 172.21.15.47:0/3833902664 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6288109850 0x7f628819e8f0 secure :-1 s=READY pgs=852 cs=0 l=1 rev1=1 crypto rx=0x7f6280004af0 tx=0x7f6280035ce0 comp rx=0 tx=0).stop 2024-03-21T10:30:15.692 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:15.687+0000 7f6290200640 1 -- 172.21.15.47:0/3833902664 shutdown_connections 2024-03-21T10:30:15.692 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:15.687+0000 7f6290200640 1 --2- 172.21.15.47:0/3833902664 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6288109850 0x7f628819e8f0 unknown :-1 s=CLOSED pgs=852 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:30:15.692 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:15.687+0000 7f6290200640 1 --2- 172.21.15.47:0/3833902664 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6288104200 0x7f628819a570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:30:15.693 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:15.687+0000 7f6290200640 1 --2- 172.21.15.47:0/3833902664 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f62700a3110 0x7f62700a55d0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:30:15.693 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:15.687+0000 7f6290200640 1 --2- 172.21.15.47:0/3833902664 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6288103840 0x7f628819a030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:30:15.693 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:15.687+0000 7f6290200640 1 -- 172.21.15.47:0/3833902664 >> 172.21.15.47:0/3833902664 conn(0x7f62880ff2f0 msgr2=0x7f62880ffbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:30:15.693 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:15.687+0000 7f6290200640 1 -- 172.21.15.47:0/3833902664 shutdown_connections 2024-03-21T10:30:15.693 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:15.687+0000 7f6290200640 1 -- 172.21.15.47:0/3833902664 wait complete. 2024-03-21T10:30:15.705 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-21T10:30:15.705 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-03-21T10:30:15.923 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:15.923+0000 7fa004a00640 1 -- 172.21.15.47:0/487383281 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa000102790 msgr2=0x7fa000102b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:30:15.923 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:15.923+0000 7fa004a00640 1 --2- 172.21.15.47:0/487383281 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa000102790 0x7fa000102b70 secure :-1 s=READY pgs=857 cs=0 l=1 rev1=1 crypto rx=0x7f9ff0009900 tx=0x7f9ff002bcd0 comp rx=0 tx=0).stop 2024-03-21T10:30:15.924 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:15.923+0000 7fa004a00640 1 -- 172.21.15.47:0/487383281 shutdown_connections 2024-03-21T10:30:15.924 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:15.923+0000 7fa004a00640 1 --2- 172.21.15.47:0/487383281 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa00010bb60 0x7fa00010df90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:30:15.924 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:15.923+0000 7fa004a00640 1 --2- 172.21.15.47:0/487383281 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa0001030b0 0x7fa00010b590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:30:15.924 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:15.923+0000 7fa004a00640 1 --2- 172.21.15.47:0/487383281 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa000102790 0x7fa000102b70 unknown :-1 s=CLOSED pgs=857 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:30:15.924 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:15.923+0000 7fa004a00640 1 -- 172.21.15.47:0/487383281 >> 172.21.15.47:0/487383281 conn(0x7fa0000fbef0 msgr2=0x7fa0000fe310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:30:15.924 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:15.923+0000 7fa004a00640 1 -- 172.21.15.47:0/487383281 shutdown_connections 2024-03-21T10:30:15.924 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:15.923+0000 7fa004a00640 1 -- 172.21.15.47:0/487383281 wait complete. 2024-03-21T10:30:15.927 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:15.924+0000 7fa004a00640 1 Processor -- start 2024-03-21T10:30:15.927 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:15.924+0000 7fa004a00640 1 -- start start 2024-03-21T10:30:15.927 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:15.925+0000 7fa004a00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa000102790 0x7fa0001003e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:30:15.927 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:15.925+0000 7fa004a00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa0001030b0 0x7fa000100920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:30:15.927 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:15.925+0000 7fa004a00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa00010bb60 0x7fa000100e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:30:15.927 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:15.925+0000 7fa004a00640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0001117b0 con 0x7fa000102790 2024-03-21T10:30:15.927 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:15.925+0000 7fa004a00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa000111630 con 0x7fa0001030b0 2024-03-21T10:30:15.927 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:15.925+0000 7fa004a00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa000111930 con 0x7fa00010bb60 2024-03-21T10:30:15.927 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:15.925+0000 7f9ffe800640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa000102790 0x7fa0001003e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:30:15.927 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:15.925+0000 7f9ffe800640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa000102790 0x7fa0001003e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:38208/0 (socket says 172.21.15.47:38208) 2024-03-21T10:30:15.927 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:15.925+0000 7f9ffe800640 1 -- 172.21.15.47:0/3208346820 learned_addr learned my addr 172.21.15.47:0/3208346820 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:30:15.927 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:15.925+0000 7f9ffde00640 1 --2- 172.21.15.47:0/3208346820 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa0001030b0 0x7fa000100920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:30:15.927 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:15.925+0000 7f9fff200640 1 --2- 172.21.15.47:0/3208346820 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa00010bb60 0x7fa000100e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:30:15.927 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:15.925+0000 7f9ffe800640 1 -- 172.21.15.47:0/3208346820 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa00010bb60 msgr2=0x7fa000100e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:30:15.928 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:15.925+0000 7f9ffe800640 1 --2- 172.21.15.47:0/3208346820 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa00010bb60 0x7fa000100e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:30:15.928 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:15.925+0000 7f9ffe800640 1 -- 172.21.15.47:0/3208346820 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa0001030b0 msgr2=0x7fa000100920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:30:15.928 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:15.925+0000 7f9ffe800640 1 --2- 172.21.15.47:0/3208346820 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa0001030b0 0x7fa000100920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:30:15.928 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:15.925+0000 7f9ffe800640 1 -- 172.21.15.47:0/3208346820 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0001a6270 con 0x7fa000102790 2024-03-21T10:30:15.928 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:15.926+0000 7f9ffde00640 1 --2- 172.21.15.47:0/3208346820 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa0001030b0 0x7fa000100920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T10:30:15.928 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:15.926+0000 7f9ffe800640 1 --2- 172.21.15.47:0/3208346820 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa000102790 0x7fa0001003e0 secure :-1 s=READY pgs=858 cs=0 l=1 rev1=1 crypto rx=0x7f9ff0005940 tx=0x7f9ff002c690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:30:15.928 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:15.926+0000 7f9ff7400640 1 -- 172.21.15.47:0/3208346820 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ff0043020 con 0x7fa000102790 2024-03-21T10:30:15.928 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:15.926+0000 7f9ff7400640 1 -- 172.21.15.47:0/3208346820 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f9ff0004e00 con 0x7fa000102790 2024-03-21T10:30:15.928 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:15.926+0000 7fa004a00640 1 -- 172.21.15.47:0/3208346820 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa0001a6500 con 0x7fa000102790 2024-03-21T10:30:15.928 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:15.926+0000 7f9ff7400640 1 -- 172.21.15.47:0/3208346820 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ff0003970 con 0x7fa000102790 2024-03-21T10:30:15.928 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:15.926+0000 7fa004a00640 1 -- 172.21.15.47:0/3208346820 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa0001a6a50 con 0x7fa000102790 2024-03-21T10:30:15.931 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:15.930+0000 7fa008163740 1 Processor -- start 2024-03-21T10:30:15.931 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:15.930+0000 7fa008163740 1 -- start start 2024-03-21T10:30:15.931 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:15.930+0000 7f9ff7400640 1 -- 172.21.15.47:0/3208346820 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 27) v1 ==== 137985+0+0 (secure 0 0 0) 0x7f9ff0046020 con 0x7fa000102790 2024-03-21T10:30:15.931 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:15.930+0000 7fa008163740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c481dec8b0 0x55c481decc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:30:15.931 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:15.930+0000 7fa008163740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c481ded260 0x55c481df5810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:30:15.931 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:15.930+0000 7fa008163740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c481df5de0 0x55c481df8230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:30:15.931 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:15.930+0000 7fa008163740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c481d502c0 con 0x55c481ded260 2024-03-21T10:30:15.931 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:15.930+0000 7fa008163740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c481cf8c90 con 0x55c481df5de0 2024-03-21T10:30:15.932 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:15.930+0000 7fa008163740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c481d4c360 con 0x55c481dec8b0 2024-03-21T10:30:15.932 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:15.931+0000 7f9ffe800640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c481ded260 0x55c481df5810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:30:15.932 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:15.931+0000 7f9ffe800640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c481ded260 0x55c481df5810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:38224/0 (socket says 172.21.15.47:38224) 2024-03-21T10:30:15.932 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:15.931+0000 7f9ffe800640 1 -- 172.21.15.47:0/3227102458 learned_addr learned my addr 172.21.15.47:0/3227102458 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:30:15.932 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:15.931+0000 7f9ffde00640 1 --2- 172.21.15.47:0/3227102458 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c481df5de0 0x55c481df8230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:30:15.932 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:15.931+0000 7f9fff200640 1 --2- 172.21.15.47:0/3227102458 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c481dec8b0 0x55c481decc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:30:15.932 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:15.931+0000 7f9ffe800640 1 -- 172.21.15.47:0/3227102458 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c481dec8b0 msgr2=0x55c481decc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:30:15.932 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:15.931+0000 7f9ffe800640 1 --2- 172.21.15.47:0/3227102458 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c481dec8b0 0x55c481decc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:30:15.932 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:15.931+0000 7f9ffe800640 1 -- 172.21.15.47:0/3227102458 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c481df5de0 msgr2=0x55c481df8230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:30:15.932 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:15.931+0000 7f9ffe800640 1 --2- 172.21.15.47:0/3227102458 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c481df5de0 0x55c481df8230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:30:15.932 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:15.931+0000 7f9ffe800640 1 -- 172.21.15.47:0/3227102458 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c481d1d080 con 0x55c481ded260 2024-03-21T10:30:15.934 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:15.932+0000 7f9ffe800640 1 --2- 172.21.15.47:0/3227102458 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c481ded260 0x55c481df5810 secure :-1 s=READY pgs=859 cs=0 l=1 rev1=1 crypto rx=0x7f9ff00eef70 tx=0x7f9ff00f5c90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c2dc682e9d09414f server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:30:15.934 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:15.932+0000 7f9ff7400640 1 --2- 172.21.15.47:0/3208346820 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f9fe40a3090 0x7f9fe40a5550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:30:15.934 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:15.932+0000 7f9feb400640 1 -- 172.21.15.47:0/3227102458 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ff0101020 con 0x55c481ded260 2024-03-21T10:30:15.934 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:15.932+0000 7f9feb400640 1 -- 172.21.15.47:0/3227102458 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f9ff00be3d0 con 0x55c481ded260 2024-03-21T10:30:15.934 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:15.932+0000 7f9feb400640 1 -- 172.21.15.47:0/3227102458 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ff009fe40 con 0x55c481ded260 2024-03-21T10:30:15.934 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:15.932+0000 7f9ff7400640 1 -- 172.21.15.47:0/3208346820 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 9474+0+0 (secure 0 0 0) 0x7f9ff00f1bf0 con 0x7fa000102790 2024-03-21T10:30:15.934 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:15.932+0000 7f9fff200640 1 --2- 172.21.15.47:0/3208346820 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f9fe40a3090 0x7f9fe40a5550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:30:15.934 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:15.933+0000 7fa008163740 1 -- 172.21.15.47:0/3227102458 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c481ded260 msgr2=0x55c481df5810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:30:15.934 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:15.933+0000 7fa008163740 1 --2- 172.21.15.47:0/3227102458 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c481ded260 0x55c481df5810 secure :-1 s=READY pgs=859 cs=0 l=1 rev1=1 crypto rx=0x7f9ff00eef70 tx=0x7f9ff00f5c90 comp rx=0 tx=0).stop 2024-03-21T10:30:15.935 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:15.933+0000 7f9fff200640 1 --2- 172.21.15.47:0/3208346820 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f9fe40a3090 0x7f9fe40a5550 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fa00006acd0 tx=0x7f9ff8009500 comp rx=0 tx=0).ready entity=mgr.15687 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:30:15.935 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:15.933+0000 7fa008163740 1 -- 172.21.15.47:0/3227102458 shutdown_connections 2024-03-21T10:30:15.935 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:15.933+0000 7fa008163740 1 --2- 172.21.15.47:0/3227102458 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c481df5de0 0x55c481df8230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:30:15.935 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:15.933+0000 7fa008163740 1 --2- 172.21.15.47:0/3227102458 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c481ded260 0x55c481df5810 unknown :-1 s=CLOSED pgs=859 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:30:15.935 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:15.933+0000 7fa008163740 1 --2- 172.21.15.47:0/3227102458 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c481dec8b0 0x55c481decc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:30:15.935 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:15.933+0000 7fa008163740 1 -- 172.21.15.47:0/3227102458 >> 172.21.15.47:0/3227102458 conn(0x55c481de6f80 msgr2=0x55c481de7360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:30:15.935 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:15.933+0000 7fa008163740 1 -- 172.21.15.47:0/3227102458 shutdown_connections 2024-03-21T10:30:15.935 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:15.933+0000 7fa008163740 1 -- 172.21.15.47:0/3227102458 wait complete. 2024-03-21T10:30:15.935 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:15.934+0000 7fa008163740 1 -- start start 2024-03-21T10:30:15.935 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:15.934+0000 7fa008163740 1 Processor -- start 2024-03-21T10:30:15.941 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:15.934+0000 7fa008163740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c481dec8b0 0x55c481df55d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:30:15.941 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:15.934+0000 7fa008163740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c481ded260 0x55c481df2960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:30:15.941 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:15.934+0000 7fa008163740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c481df5de0 0x55c481df2ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:30:15.941 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:15.934+0000 7fa008163740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c481df8b10 con 0x55c481df5de0 2024-03-21T10:30:15.941 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:15.934+0000 7fa008163740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c481d0c0d0 con 0x55c481dec8b0 2024-03-21T10:30:15.941 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:15.934+0000 7fa008163740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c481df8c90 con 0x55c481ded260 2024-03-21T10:30:15.941 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:15.934+0000 7fa008163740 10 client.0 _ll_register_callbacks cb 0x55c481d10e80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T10:30:15.941 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:15.935+0000 7f9fff200640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c481df5de0 0x55c481df2ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:30:15.941 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:15.935+0000 7f9fff200640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c481df5de0 0x55c481df2ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:38232/0 (socket says 172.21.15.47:38232) 2024-03-21T10:30:15.941 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:15.935+0000 7f9fff200640 1 -- 172.21.15.47:0/2147143822 learned_addr learned my addr 172.21.15.47:0/2147143822 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:30:15.941 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:15.935+0000 7f9ffe800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c481dec8b0 0x55c481df55d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:30:15.941 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:15.935+0000 7f9fff200640 1 -- 172.21.15.47:0/2147143822 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c481ded260 msgr2=0x55c481df2960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:30:15.942 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:15.935+0000 7f9fff200640 1 --2- 172.21.15.47:0/2147143822 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c481ded260 0x55c481df2960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:30:15.942 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:15.935+0000 7f9fff200640 1 -- 172.21.15.47:0/2147143822 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c481dec8b0 msgr2=0x55c481df55d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:30:15.942 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:15.935+0000 7f9fff200640 1 --2- 172.21.15.47:0/2147143822 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c481dec8b0 0x55c481df55d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:30:15.942 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:15.935+0000 7f9fff200640 1 -- 172.21.15.47:0/2147143822 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55c481a643f0 con 0x55c481df5de0 2024-03-21T10:30:15.942 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:15.935+0000 7f9fff200640 1 --2- 172.21.15.47:0/2147143822 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c481df5de0 0x55c481df2ea0 secure :-1 s=READY pgs=860 cs=0 l=1 rev1=1 crypto rx=0x7f9ff8012850 tx=0x7f9ff8004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=26d8c147258fdeab server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:30:15.942 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:15.935+0000 7f9fdfe00640 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-03-21T10:30:15.942 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:15.936+0000 7f9fdfe00640 1 -- 172.21.15.47:0/2147143822 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ff8010be0 con 0x55c481df5de0 2024-03-21T10:30:15.942 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:15.936+0000 7f9fdfe00640 1 -- 172.21.15.47:0/2147143822 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(97..97 src has 1..97) v4 ==== 9474+0+0 (secure 0 0 0) 0x7f9ff8010d80 con 0x55c481df5de0 2024-03-21T10:30:15.942 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:15.936+0000 7fa008163740 1 -- 172.21.15.47:0/2147143822 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f9ff00be3d0 con 0x55c481df5de0 2024-03-21T10:30:15.942 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:15.936+0000 7f9fdfe00640 1 -- 172.21.15.47:0/2147143822 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9ff801b460 con 0x55c481df5de0 2024-03-21T10:30:15.942 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:15.936+0000 7fa008163740 10 client.17172 fetch_fsmap learned FSMap version 32 2024-03-21T10:30:15.942 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:15.936+0000 7fa008163740 1 -- 172.21.15.47:0/2147143822 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x55c481b2e7d0 con 0x55c481df5de0 2024-03-21T10:30:15.942 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:15.936+0000 7f9fdfe00640 1 -- 172.21.15.47:0/2147143822 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f9ff00be3d0 con 0x55c481df5de0 2024-03-21T10:30:15.942 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:15.936+0000 7f9fdfe00640 1 -- 172.21.15.47:0/2147143822 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 32) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f9ff8024070 con 0x55c481df5de0 2024-03-21T10:30:15.943 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:15.937+0000 7fa008163740 10 client.17172 fetch_fsmap finished waiting for FSMap version 32 2024-03-21T10:30:15.943 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:15.937+0000 7fa008163740 10 client.17172 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.k 2024-03-21T10:30:15.943 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:15.937+0000 7fa008163740 20 client.17172 populate_metadata read hostname 'smithi047' 2024-03-21T10:30:15.943 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:15.937+0000 7fa008163740 1 --2- 172.21.15.47:0/2147143822 >> [v2:172.21.15.100:6832/2723302038,v1:172.21.15.100:6833/2723302038] conn(0x55c481e05910 0x55c481e07db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:30:15.943 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:15.937+0000 7fa008163740 4 client.17172 mds_command: new command op to 34427 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T10:30:15.943 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:15.937+0000 7fa008163740 1 -- 172.21.15.47:0/2147143822 --> [v2:172.21.15.100:6832/2723302038,v1:172.21.15.100:6833/2723302038] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55c481d5d800 con 0x55c481e05910 2024-03-21T10:30:15.943 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:15.937+0000 7f9ffe800640 1 --2- 172.21.15.47:0/2147143822 >> [v2:172.21.15.100:6832/2723302038,v1:172.21.15.100:6833/2723302038] conn(0x55c481e05910 0x55c481e07db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:30:15.943 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:15.938+0000 7f9ffe800640 1 --2- 172.21.15.47:0/2147143822 >> [v2:172.21.15.100:6832/2723302038,v1:172.21.15.100:6833/2723302038] conn(0x55c481e05910 0x55c481e07db0 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=2acce96b3aade921 server_cookie=b28732eeca2a5af in_seq=0 out_seq=0 2024-03-21T10:30:15.943 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:15.938+0000 7f9fdfe00640 10 client.17172 ms_handle_connect on v2:172.21.15.100:6832/2723302038 2024-03-21T10:30:15.943 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:15.939+0000 7f9fdfe00640 1 -- 172.21.15.47:0/2147143822 <== mds.3 v2:172.21.15.100:6832/2723302038 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55c481d5d800 con 0x55c481e05910 2024-03-21T10:30:15.943 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:15.939+0000 7f9fdfe00640 10 client.17172 handle_command_reply: tid=0 2024-03-21T10:30:15.943 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:15.939+0000 7fa008163740 1 client.17172 shutdown 2024-03-21T10:30:15.943 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:15.939+0000 7fa008163740 1 -- 172.21.15.47:0/2147143822 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c481df5de0 msgr2=0x55c481df2ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:30:15.943 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:15.939+0000 7fa008163740 1 --2- 172.21.15.47:0/2147143822 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c481df5de0 0x55c481df2ea0 secure :-1 s=READY pgs=860 cs=0 l=1 rev1=1 crypto rx=0x7f9ff8012850 tx=0x7f9ff8004a40 comp rx=0 tx=0).stop 2024-03-21T10:30:15.943 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:15.939+0000 7fa008163740 1 -- 172.21.15.47:0/2147143822 shutdown_connections 2024-03-21T10:30:15.944 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:15.939+0000 7fa008163740 1 --2- 172.21.15.47:0/2147143822 >> [v2:172.21.15.100:6832/2723302038,v1:172.21.15.100:6833/2723302038] conn(0x55c481e05910 0x55c481e07db0 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:30:15.944 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:15.939+0000 7fa008163740 1 --2- 172.21.15.47:0/2147143822 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c481df5de0 0x55c481df2ea0 unknown :-1 s=CLOSED pgs=860 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:30:15.944 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:15.939+0000 7fa008163740 1 --2- 172.21.15.47:0/2147143822 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c481ded260 0x55c481df2960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:30:15.944 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:15.939+0000 7fa008163740 1 --2- 172.21.15.47:0/2147143822 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c481dec8b0 0x55c481df55d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:30:15.944 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:15.939+0000 7fa008163740 1 -- 172.21.15.47:0/2147143822 >> 172.21.15.47:0/2147143822 conn(0x55c481de6f80 msgr2=0x55c481de8cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:30:15.944 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:15.939+0000 7f9fdfe00640 0 client.17172 ms_handle_reset on v2:172.21.15.100:6832/2723302038 2024-03-21T10:30:15.944 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:15.939+0000 7fa008163740 1 -- 172.21.15.47:0/2147143822 shutdown_connections 2024-03-21T10:30:15.944 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:15.940+0000 7fa008163740 1 -- 172.21.15.47:0/2147143822 wait complete. 2024-03-21T10:30:15.944 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:15.940+0000 7fa008163740 20 client.17172 trim_cache size 0 max 16384 2024-03-21T10:30:15.954 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:15.953+0000 7fa008163740 1 Processor -- start 2024-03-21T10:30:15.954 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:15.953+0000 7fa008163740 1 -- start start 2024-03-21T10:30:15.954 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:15.953+0000 7fa008163740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c481dec8b0 0x55c481de5fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:30:15.954 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:15.954+0000 7fa008163740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c481ded260 0x55c481df5090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:30:15.954 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:15.954+0000 7fa008163740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c481df5de0 0x55c481df55d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:30:15.954 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:15.954+0000 7fa008163740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0001b0950 con 0x55c481df5de0 2024-03-21T10:30:15.954 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:15.954+0000 7fa008163740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0001b22d0 con 0x55c481dec8b0 2024-03-21T10:30:15.954 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:15.954+0000 7fa008163740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0001b0fb0 con 0x55c481ded260 2024-03-21T10:30:15.956 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:15.954+0000 7f9fff200640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c481df5de0 0x55c481df55d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:30:15.956 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:15.954+0000 7f9fff200640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c481df5de0 0x55c481df55d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:38236/0 (socket says 172.21.15.47:38236) 2024-03-21T10:30:15.957 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:15.954+0000 7f9fff200640 1 -- 172.21.15.47:0/187486581 learned_addr learned my addr 172.21.15.47:0/187486581 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:30:15.957 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:15.954+0000 7f9ffe800640 1 --2- 172.21.15.47:0/187486581 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c481dec8b0 0x55c481de5fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:30:15.957 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:15.954+0000 7f9ffde00640 1 --2- 172.21.15.47:0/187486581 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c481ded260 0x55c481df5090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:30:15.957 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:15.954+0000 7f9fff200640 1 -- 172.21.15.47:0/187486581 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c481ded260 msgr2=0x55c481df5090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:30:15.957 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:15.954+0000 7f9fff200640 1 --2- 172.21.15.47:0/187486581 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c481ded260 0x55c481df5090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:30:15.957 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:15.955+0000 7f9fff200640 1 -- 172.21.15.47:0/187486581 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c481dec8b0 msgr2=0x55c481de5fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:30:15.957 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:15.955+0000 7f9fff200640 1 --2- 172.21.15.47:0/187486581 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c481dec8b0 0x55c481de5fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:30:15.957 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:15.955+0000 7f9fff200640 1 -- 172.21.15.47:0/187486581 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c481df0eb0 con 0x55c481df5de0 2024-03-21T10:30:15.957 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:15.955+0000 7f9ffde00640 1 --2- 172.21.15.47:0/187486581 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c481ded260 0x55c481df5090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T10:30:15.957 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:15.955+0000 7f9fff200640 1 --2- 172.21.15.47:0/187486581 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c481df5de0 0x55c481df55d0 secure :-1 s=READY pgs=861 cs=0 l=1 rev1=1 crypto rx=0x7f9ff8005c40 tx=0x7f9ff8005fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9e1fcf11fc2a7ef1 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:30:15.957 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:15.955+0000 7f9feaa00640 1 -- 172.21.15.47:0/187486581 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ff8039070 con 0x55c481df5de0 2024-03-21T10:30:15.957 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:15.955+0000 7f9feaa00640 1 -- 172.21.15.47:0/187486581 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f9ff8026440 con 0x55c481df5de0 2024-03-21T10:30:15.957 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:15.955+0000 7f9feaa00640 1 -- 172.21.15.47:0/187486581 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ff801b860 con 0x55c481df5de0 2024-03-21T10:30:15.958 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:15.956+0000 7fa008163740 1 -- 172.21.15.47:0/187486581 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c481df5de0 msgr2=0x55c481df55d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:30:15.958 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:15.956+0000 7fa008163740 1 --2- 172.21.15.47:0/187486581 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c481df5de0 0x55c481df55d0 secure :-1 s=READY pgs=861 cs=0 l=1 rev1=1 crypto rx=0x7f9ff8005c40 tx=0x7f9ff8005fd0 comp rx=0 tx=0).stop 2024-03-21T10:30:15.958 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:15.956+0000 7fa008163740 1 -- 172.21.15.47:0/187486581 shutdown_connections 2024-03-21T10:30:15.958 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:15.956+0000 7fa008163740 1 --2- 172.21.15.47:0/187486581 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c481df5de0 0x55c481df55d0 unknown :-1 s=CLOSED pgs=861 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:30:15.958 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:15.956+0000 7fa008163740 1 --2- 172.21.15.47:0/187486581 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c481ded260 0x55c481df5090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:30:15.958 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:15.956+0000 7fa008163740 1 --2- 172.21.15.47:0/187486581 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c481dec8b0 0x55c481de5fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:30:15.958 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:15.956+0000 7fa008163740 1 -- 172.21.15.47:0/187486581 >> 172.21.15.47:0/187486581 conn(0x55c481de6f80 msgr2=0x55c481de58d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:30:15.958 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:15.956+0000 7fa008163740 1 -- 172.21.15.47:0/187486581 shutdown_connections 2024-03-21T10:30:15.958 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:15.956+0000 7fa008163740 1 -- 172.21.15.47:0/187486581 wait complete. 2024-03-21T10:30:15.958 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:15.956+0000 7fa008163740 1 -- start start 2024-03-21T10:30:15.958 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:15.957+0000 7fa008163740 1 Processor -- start 2024-03-21T10:30:15.961 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:15.957+0000 7fa008163740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c481dec8b0 0x55c481df9a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:30:15.961 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:15.957+0000 7fa008163740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c481ded260 0x55c481df9f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:30:15.961 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:15.957+0000 7fa008163740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c481df5de0 0x55c481e60240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:30:15.961 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:15.957+0000 7fa008163740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0001b1c70 con 0x55c481df5de0 2024-03-21T10:30:15.962 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:15.957+0000 7fa008163740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0001b1610 con 0x55c481dec8b0 2024-03-21T10:30:15.962 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:15.957+0000 7fa008163740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0001aeb10 con 0x55c481ded260 2024-03-21T10:30:15.962 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:15.957+0000 7fa008163740 10 client.0 _ll_register_callbacks cb 0x7f9ff8009ca0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T10:30:15.962 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:15.958+0000 7f9fff200640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c481df5de0 0x55c481e60240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:30:15.962 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:15.958+0000 7f9fff200640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c481df5de0 0x55c481e60240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:38242/0 (socket says 172.21.15.47:38242) 2024-03-21T10:30:15.962 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:15.958+0000 7f9fff200640 1 -- 172.21.15.47:0/1319885264 learned_addr learned my addr 172.21.15.47:0/1319885264 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:30:15.962 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:15.958+0000 7f9ffe800640 1 --2- 172.21.15.47:0/1319885264 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c481dec8b0 0x55c481df9a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:30:15.962 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:15.958+0000 7f9ffde00640 1 --2- 172.21.15.47:0/1319885264 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c481ded260 0x55c481df9f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:30:15.962 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:15.958+0000 7f9fff200640 1 -- 172.21.15.47:0/1319885264 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c481ded260 msgr2=0x55c481df9f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:30:15.962 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:15.958+0000 7f9fff200640 1 --2- 172.21.15.47:0/1319885264 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c481ded260 0x55c481df9f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:30:15.962 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:15.958+0000 7f9fff200640 1 -- 172.21.15.47:0/1319885264 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c481dec8b0 msgr2=0x55c481df9a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:30:15.962 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:15.958+0000 7f9fff200640 1 --2- 172.21.15.47:0/1319885264 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c481dec8b0 0x55c481df9a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:30:15.962 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:15.958+0000 7f9fff200640 1 -- 172.21.15.47:0/1319885264 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55c481e607d0 con 0x55c481df5de0 2024-03-21T10:30:15.962 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:15.958+0000 7f9ffe800640 1 --2- 172.21.15.47:0/1319885264 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c481dec8b0 0x55c481df9a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T10:30:15.963 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:15.958+0000 7f9fff200640 1 --2- 172.21.15.47:0/1319885264 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c481df5de0 0x55c481e60240 secure :-1 s=READY pgs=862 cs=0 l=1 rev1=1 crypto rx=0x7f9ff8026e50 tx=0x7f9ff801bf70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a46b0933cca8b93a server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:30:15.963 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:15.958+0000 7f9fdfe00640 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-03-21T10:30:15.963 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:15.959+0000 7f9fdfe00640 1 -- 172.21.15.47:0/1319885264 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ff8039040 con 0x55c481df5de0 2024-03-21T10:30:15.963 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:15.959+0000 7fa008163740 1 -- 172.21.15.47:0/1319885264 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f9ff8026440 con 0x55c481df5de0 2024-03-21T10:30:15.963 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:15.959+0000 7f9fdfe00640 1 -- 172.21.15.47:0/1319885264 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(97..97 src has 1..97) v4 ==== 9474+0+0 (secure 0 0 0) 0x7f9ff8004c40 con 0x55c481df5de0 2024-03-21T10:30:15.963 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:15.959+0000 7f9fdfe00640 1 -- 172.21.15.47:0/1319885264 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9ff804b050 con 0x55c481df5de0 2024-03-21T10:30:15.963 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:15.959+0000 7fa008163740 10 client.17178 fetch_fsmap learned FSMap version 32 2024-03-21T10:30:15.963 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:15.959+0000 7fa008163740 1 -- 172.21.15.47:0/1319885264 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x55c481e61ad0 con 0x55c481df5de0 2024-03-21T10:30:15.963 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:15.959+0000 7f9fdfe00640 1 -- 172.21.15.47:0/1319885264 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f9ff8026440 con 0x55c481df5de0 2024-03-21T10:30:15.963 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:15.960+0000 7f9fdfe00640 1 -- 172.21.15.47:0/1319885264 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 32) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f9ff803c050 con 0x55c481df5de0 2024-03-21T10:30:15.963 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:15.960+0000 7fa008163740 10 client.17178 fetch_fsmap finished waiting for FSMap version 32 2024-03-21T10:30:15.963 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:15.960+0000 7fa008163740 10 client.17178 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.k 2024-03-21T10:30:15.963 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:15.960+0000 7fa008163740 20 client.17178 populate_metadata read hostname 'smithi047' 2024-03-21T10:30:15.963 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:15.960+0000 7fa008163740 1 --2- 172.21.15.47:0/1319885264 >> [v2:172.21.15.100:6832/2723302038,v1:172.21.15.100:6833/2723302038] conn(0x55c481e621c0 0x55c481e645e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:30:15.964 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:15.960+0000 7fa008163740 4 client.17178 mds_command: new command op to 34427 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-21T10:30:15.964 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:15.960+0000 7fa008163740 1 -- 172.21.15.47:0/1319885264 --> [v2:172.21.15.100:6832/2723302038,v1:172.21.15.100:6833/2723302038] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55c481d57b50 con 0x55c481e621c0 2024-03-21T10:30:15.964 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:15.960+0000 7f9ffe800640 1 --2- 172.21.15.47:0/1319885264 >> [v2:172.21.15.100:6832/2723302038,v1:172.21.15.100:6833/2723302038] conn(0x55c481e621c0 0x55c481e645e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:30:15.964 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:15.961+0000 7f9ffe800640 1 --2- 172.21.15.47:0/1319885264 >> [v2:172.21.15.100:6832/2723302038,v1:172.21.15.100:6833/2723302038] conn(0x55c481e621c0 0x55c481e645e0 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=c0c3722c3beedaef server_cookie=d21066869d37de57 in_seq=0 out_seq=0 2024-03-21T10:30:15.964 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:15.961+0000 7f9fdfe00640 10 client.17178 ms_handle_connect on v2:172.21.15.100:6832/2723302038 2024-03-21T10:30:15.964 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:15.961+0000 7f9fdfe00640 1 -- 172.21.15.47:0/1319885264 <== mds.3 v2:172.21.15.100:6832/2723302038 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55c481d57b50 con 0x55c481e621c0 2024-03-21T10:30:15.964 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:15.961+0000 7f9fdfe00640 10 client.17178 handle_command_reply: tid=0 2024-03-21T10:30:15.964 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:15.961+0000 7fa008163740 1 client.17178 shutdown 2024-03-21T10:30:15.965 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:15.962+0000 7fa008163740 1 -- 172.21.15.47:0/1319885264 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c481df5de0 msgr2=0x55c481e60240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:30:15.965 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:15.962+0000 7fa008163740 1 --2- 172.21.15.47:0/1319885264 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c481df5de0 0x55c481e60240 secure :-1 s=READY pgs=862 cs=0 l=1 rev1=1 crypto rx=0x7f9ff8026e50 tx=0x7f9ff801bf70 comp rx=0 tx=0).stop 2024-03-21T10:30:15.965 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:15.962+0000 7fa008163740 1 -- 172.21.15.47:0/1319885264 shutdown_connections 2024-03-21T10:30:15.965 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:15.962+0000 7fa008163740 1 --2- 172.21.15.47:0/1319885264 >> [v2:172.21.15.100:6832/2723302038,v1:172.21.15.100:6833/2723302038] conn(0x55c481e621c0 0x55c481e645e0 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:30:15.965 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:15.962+0000 7fa008163740 1 --2- 172.21.15.47:0/1319885264 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c481df5de0 0x55c481e60240 unknown :-1 s=CLOSED pgs=862 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:30:15.965 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:15.962+0000 7fa008163740 1 --2- 172.21.15.47:0/1319885264 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c481ded260 0x55c481df9f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:30:15.965 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:15.962+0000 7fa008163740 1 --2- 172.21.15.47:0/1319885264 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c481dec8b0 0x55c481df9a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:30:15.965 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:15.962+0000 7f9fdfe00640 0 client.17178 ms_handle_reset on v2:172.21.15.100:6832/2723302038 2024-03-21T10:30:15.966 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:15.962+0000 7fa008163740 1 -- 172.21.15.47:0/1319885264 >> 172.21.15.47:0/1319885264 conn(0x55c481de6f80 msgr2=0x55c481de6160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:30:15.966 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:15.962+0000 7fa008163740 1 -- 172.21.15.47:0/1319885264 shutdown_connections 2024-03-21T10:30:15.966 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:15.962+0000 7fa008163740 1 -- 172.21.15.47:0/1319885264 wait complete. 2024-03-21T10:30:15.966 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:15.962+0000 7fa008163740 20 client.17178 trim_cache size 0 max 16384 2024-03-21T10:30:15.966 INFO:teuthology.orchestra.run.smithi047.stdout:[] 2024-03-21T10:30:15.967 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:15.963+0000 7fa004a00640 1 -- 172.21.15.47:0/3208346820 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f9fe40a3090 msgr2=0x7f9fe40a5550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:30:15.967 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:15.963+0000 7fa004a00640 1 --2- 172.21.15.47:0/3208346820 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f9fe40a3090 0x7f9fe40a5550 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fa00006acd0 tx=0x7f9ff8009500 comp rx=0 tx=0).stop 2024-03-21T10:30:15.967 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:15.963+0000 7fa004a00640 1 -- 172.21.15.47:0/3208346820 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa000102790 msgr2=0x7fa0001003e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:30:15.967 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:15.963+0000 7fa004a00640 1 --2- 172.21.15.47:0/3208346820 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa000102790 0x7fa0001003e0 secure :-1 s=READY pgs=858 cs=0 l=1 rev1=1 crypto rx=0x7f9ff0005940 tx=0x7f9ff002c690 comp rx=0 tx=0).stop 2024-03-21T10:30:15.967 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:15.963+0000 7fa004a00640 1 -- 172.21.15.47:0/3208346820 shutdown_connections 2024-03-21T10:30:15.967 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:15.963+0000 7fa004a00640 1 --2- 172.21.15.47:0/3208346820 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa00010bb60 0x7fa000100e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:30:15.967 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:15.963+0000 7fa004a00640 1 --2- 172.21.15.47:0/3208346820 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f9fe40a3090 0x7f9fe40a5550 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:30:15.967 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:15.963+0000 7fa004a00640 1 --2- 172.21.15.47:0/3208346820 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa0001030b0 0x7fa000100920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:30:15.967 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:15.963+0000 7fa004a00640 1 --2- 172.21.15.47:0/3208346820 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa000102790 0x7fa0001003e0 unknown :-1 s=CLOSED pgs=858 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:30:15.967 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:15.963+0000 7fa004a00640 1 -- 172.21.15.47:0/3208346820 >> 172.21.15.47:0/3208346820 conn(0x7fa0000fbef0 msgr2=0x7fa0000fd870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:30:15.968 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:15.963+0000 7fa004a00640 1 -- 172.21.15.47:0/3208346820 shutdown_connections 2024-03-21T10:30:15.968 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:15.963+0000 7fa004a00640 1 -- 172.21.15.47:0/3208346820 wait complete. 2024-03-21T10:30:15.983 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-21T10:30:15.984 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-03-21T10:30:16.199 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:16.199+0000 7fb176a00640 1 -- 172.21.15.47:0/2830811497 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb1701026d0 msgr2=0x7fb170102ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:30:16.199 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:16.199+0000 7fb176a00640 1 --2- 172.21.15.47:0/2830811497 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb1701026d0 0x7fb170102ab0 secure :-1 s=READY pgs=863 cs=0 l=1 rev1=1 crypto rx=0x7fb164009be0 tx=0x7fb16402bf80 comp rx=0 tx=0).stop 2024-03-21T10:30:16.199 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:16.199+0000 7fb176a00640 1 -- 172.21.15.47:0/2830811497 shutdown_connections 2024-03-21T10:30:16.199 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:16.199+0000 7fb176a00640 1 --2- 172.21.15.47:0/2830811497 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb17010baf0 0x7fb17010df90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:30:16.200 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:16.199+0000 7fb176a00640 1 --2- 172.21.15.47:0/2830811497 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb170102ff0 0x7fb17010b4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:30:16.200 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:16.199+0000 7fb176a00640 1 --2- 172.21.15.47:0/2830811497 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb1701026d0 0x7fb170102ab0 unknown :-1 s=CLOSED pgs=863 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:30:16.200 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:16.199+0000 7fb176a00640 1 -- 172.21.15.47:0/2830811497 >> 172.21.15.47:0/2830811497 conn(0x7fb1700fbef0 msgr2=0x7fb1700fe310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:30:16.200 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:16.199+0000 7fb176a00640 1 -- 172.21.15.47:0/2830811497 shutdown_connections 2024-03-21T10:30:16.200 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:16.199+0000 7fb176a00640 1 -- 172.21.15.47:0/2830811497 wait complete. 2024-03-21T10:30:16.201 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:16.200+0000 7fb176a00640 1 Processor -- start 2024-03-21T10:30:16.202 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:16.200+0000 7fb176a00640 1 -- start start 2024-03-21T10:30:16.202 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:16.201+0000 7fb176a00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb1701026d0 0x7fb170100320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:30:16.202 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:16.201+0000 7fb176a00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb170102ff0 0x7fb170100860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:30:16.202 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:16.201+0000 7fb176a00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb17010baf0 0x7fb170100da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:30:16.202 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:16.201+0000 7fb176a00640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1701117b0 con 0x7fb17010baf0 2024-03-21T10:30:16.202 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:16.201+0000 7fb176a00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb170111630 con 0x7fb1701026d0 2024-03-21T10:30:16.202 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:16.201+0000 7fb176a00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb170111930 con 0x7fb170102ff0 2024-03-21T10:30:16.202 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:16.201+0000 7fb176000640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb17010baf0 0x7fb170100da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:30:16.202 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:16.201+0000 7fb176000640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb17010baf0 0x7fb170100da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:38260/0 (socket says 172.21.15.47:38260) 2024-03-21T10:30:16.202 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:16.201+0000 7fb176000640 1 -- 172.21.15.47:0/518978189 learned_addr learned my addr 172.21.15.47:0/518978189 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:30:16.202 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:16.201+0000 7fb175600640 1 --2- 172.21.15.47:0/518978189 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb1701026d0 0x7fb170100320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:30:16.202 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:16.201+0000 7fb174c00640 1 --2- 172.21.15.47:0/518978189 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb170102ff0 0x7fb170100860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:30:16.203 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:16.201+0000 7fb176000640 1 -- 172.21.15.47:0/518978189 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb170102ff0 msgr2=0x7fb170100860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:30:16.203 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:16.201+0000 7fb176000640 1 --2- 172.21.15.47:0/518978189 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb170102ff0 0x7fb170100860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:30:16.203 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:16.201+0000 7fb176000640 1 -- 172.21.15.47:0/518978189 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb1701026d0 msgr2=0x7fb170100320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:30:16.203 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:16.201+0000 7fb176000640 1 --2- 172.21.15.47:0/518978189 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb1701026d0 0x7fb170100320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:30:16.203 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:16.201+0000 7fb176000640 1 -- 172.21.15.47:0/518978189 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1701a6210 con 0x7fb17010baf0 2024-03-21T10:30:16.203 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:16.202+0000 7fb176000640 1 --2- 172.21.15.47:0/518978189 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb17010baf0 0x7fb170100da0 secure :-1 s=READY pgs=864 cs=0 l=1 rev1=1 crypto rx=0x7fb16800d750 tx=0x7fb16800dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:30:16.204 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:16.202+0000 7fb16e000640 1 -- 172.21.15.47:0/518978189 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb168017070 con 0x7fb17010baf0 2024-03-21T10:30:16.204 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:16.202+0000 7fb16e000640 1 -- 172.21.15.47:0/518978189 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fb1680054d0 con 0x7fb17010baf0 2024-03-21T10:30:16.204 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:16.202+0000 7fb176a00640 1 -- 172.21.15.47:0/518978189 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb1701a6500 con 0x7fb17010baf0 2024-03-21T10:30:16.204 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:16.202+0000 7fb16e000640 1 -- 172.21.15.47:0/518978189 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb168004c60 con 0x7fb17010baf0 2024-03-21T10:30:16.204 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:16.202+0000 7fb176a00640 1 -- 172.21.15.47:0/518978189 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb1701a6a50 con 0x7fb17010baf0 2024-03-21T10:30:16.205 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:16.206+0000 7fb16e000640 1 -- 172.21.15.47:0/518978189 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 27) v1 ==== 137985+0+0 (secure 0 0 0) 0x7fb168004e00 con 0x7fb17010baf0 2024-03-21T10:30:16.207 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:16.207+0000 7fb16e000640 1 --2- 172.21.15.47:0/518978189 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7fb1580a3140 0x7fb1580a5600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:30:16.207 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:16.207+0000 7fb17ada2740 1 Processor -- start 2024-03-21T10:30:16.207 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:16.207+0000 7fb16e000640 1 -- 172.21.15.47:0/518978189 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 9474+0+0 (secure 0 0 0) 0x7fb1680cf4e0 con 0x7fb17010baf0 2024-03-21T10:30:16.207 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:16.207+0000 7fb175600640 1 --2- 172.21.15.47:0/518978189 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7fb1580a3140 0x7fb1580a5600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:30:16.207 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:16.207+0000 7fb17ada2740 1 -- start start 2024-03-21T10:30:16.210 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:16.208+0000 7fb17ada2740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55578ff338b0 0x55578ff33c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:30:16.210 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:16.208+0000 7fb17ada2740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55578ff34260 0x55578ff3c810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:30:16.210 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:16.208+0000 7fb17ada2740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55578ff3cde0 0x55578ff3f230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:30:16.210 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:16.208+0000 7fb17ada2740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x55578fe972c0 con 0x55578ff3cde0 2024-03-21T10:30:16.210 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:16.208+0000 7fb17ada2740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55578fe3fc90 con 0x55578ff34260 2024-03-21T10:30:16.210 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:16.208+0000 7fb17ada2740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55578fe93360 con 0x55578ff338b0 2024-03-21T10:30:16.210 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:16.208+0000 7fb174c00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55578ff34260 0x55578ff3c810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:30:16.210 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:16.208+0000 7fb174c00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55578ff34260 0x55578ff3c810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:42248/0 (socket says 172.21.15.47:42248) 2024-03-21T10:30:16.210 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:16.208+0000 7fb174c00640 1 -- 172.21.15.47:0/3583625217 learned_addr learned my addr 172.21.15.47:0/3583625217 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:30:16.210 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:16.208+0000 7fb176000640 1 --2- 172.21.15.47:0/3583625217 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55578ff3cde0 0x55578ff3f230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:30:16.210 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:16.208+0000 7fb175600640 1 --2- 172.21.15.47:0/518978189 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7fb1580a3140 0x7fb1580a5600 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fb164004af0 tx=0x7fb164000f40 comp rx=0 tx=0).ready entity=mgr.15687 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:30:16.210 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:16.208+0000 7fb175600640 1 --2- 172.21.15.47:0/3583625217 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55578ff338b0 0x55578ff33c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:30:16.210 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:16.209+0000 7fb174c00640 1 -- 172.21.15.47:0/3583625217 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55578ff338b0 msgr2=0x55578ff33c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:30:16.211 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:16.209+0000 7fb174c00640 1 --2- 172.21.15.47:0/3583625217 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55578ff338b0 0x55578ff33c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:30:16.211 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:16.209+0000 7fb174c00640 1 -- 172.21.15.47:0/3583625217 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55578ff3cde0 msgr2=0x55578ff3f230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:30:16.211 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:16.209+0000 7fb174c00640 1 --2- 172.21.15.47:0/3583625217 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55578ff3cde0 0x55578ff3f230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:30:16.211 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:16.209+0000 7fb174c00640 1 -- 172.21.15.47:0/3583625217 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55578fe64080 con 0x55578ff34260 2024-03-21T10:30:16.211 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:16.209+0000 7fb175600640 1 --2- 172.21.15.47:0/3583625217 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55578ff338b0 0x55578ff33c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T10:30:16.211 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:16.209+0000 7fb174c00640 1 --2- 172.21.15.47:0/3583625217 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55578ff34260 0x55578ff3c810 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fb15c009c10 tx=0x7fb15c0117b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=46f9628b2184ec94 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:30:16.211 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:16.209+0000 7fb162600640 1 -- 172.21.15.47:0/3583625217 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb15c004030 con 0x55578ff34260 2024-03-21T10:30:16.211 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:16.210+0000 7fb162600640 1 -- 172.21.15.47:0/3583625217 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fb15c0035f0 con 0x55578ff34260 2024-03-21T10:30:16.211 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:16.210+0000 7fb162600640 1 -- 172.21.15.47:0/3583625217 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb15c012690 con 0x55578ff34260 2024-03-21T10:30:16.215 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:16.210+0000 7fb17ada2740 1 -- 172.21.15.47:0/3583625217 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55578ff34260 msgr2=0x55578ff3c810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:30:16.215 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:16.210+0000 7fb17ada2740 1 --2- 172.21.15.47:0/3583625217 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55578ff34260 0x55578ff3c810 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fb15c009c10 tx=0x7fb15c0117b0 comp rx=0 tx=0).stop 2024-03-21T10:30:16.215 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:16.210+0000 7fb17ada2740 1 -- 172.21.15.47:0/3583625217 shutdown_connections 2024-03-21T10:30:16.215 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:16.210+0000 7fb17ada2740 1 --2- 172.21.15.47:0/3583625217 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55578ff3cde0 0x55578ff3f230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:30:16.215 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:16.210+0000 7fb17ada2740 1 --2- 172.21.15.47:0/3583625217 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55578ff34260 0x55578ff3c810 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:30:16.215 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:16.210+0000 7fb17ada2740 1 --2- 172.21.15.47:0/3583625217 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55578ff338b0 0x55578ff33c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:30:16.215 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:16.210+0000 7fb17ada2740 1 -- 172.21.15.47:0/3583625217 >> 172.21.15.47:0/3583625217 conn(0x55578ff2df80 msgr2=0x55578ff2e360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:30:16.215 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:16.210+0000 7fb17ada2740 1 -- 172.21.15.47:0/3583625217 shutdown_connections 2024-03-21T10:30:16.215 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:16.211+0000 7fb17ada2740 1 -- 172.21.15.47:0/3583625217 wait complete. 2024-03-21T10:30:16.215 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:16.211+0000 7fb17ada2740 1 -- start start 2024-03-21T10:30:16.215 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:16.211+0000 7fb17ada2740 1 Processor -- start 2024-03-21T10:30:16.216 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:16.212+0000 7fb17ada2740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55578ff338b0 0x55578ff3c680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:30:16.216 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:16.212+0000 7fb17ada2740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55578ff34260 0x55578ff39a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:30:16.216 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:16.212+0000 7fb17ada2740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55578ff3cde0 0x55578ff39f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:30:16.216 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:16.212+0000 7fb17ada2740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x55578ff3fda0 con 0x55578ff3cde0 2024-03-21T10:30:16.216 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:16.212+0000 7fb17ada2740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55578fe530d0 con 0x55578ff34260 2024-03-21T10:30:16.216 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:16.212+0000 7fb17ada2740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55578ff3ff20 con 0x55578ff338b0 2024-03-21T10:30:16.216 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:16.212+0000 7fb17ada2740 10 client.0 _ll_register_callbacks cb 0x55578fe57e80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T10:30:16.216 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:16.212+0000 7fb176000640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55578ff3cde0 0x55578ff39f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:30:16.216 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:16.212+0000 7fb176000640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55578ff3cde0 0x55578ff39f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:38276/0 (socket says 172.21.15.47:38276) 2024-03-21T10:30:16.216 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:16.212+0000 7fb174c00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55578ff34260 0x55578ff39a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:30:16.216 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:16.212+0000 7fb176000640 1 -- 172.21.15.47:0/298621257 learned_addr learned my addr 172.21.15.47:0/298621257 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:30:16.216 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:16.212+0000 7fb175600640 1 --2- 172.21.15.47:0/298621257 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55578ff338b0 0x55578ff3c680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:30:16.216 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:16.212+0000 7fb176000640 1 -- 172.21.15.47:0/298621257 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55578ff338b0 msgr2=0x55578ff3c680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:30:16.216 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:16.212+0000 7fb176000640 1 --2- 172.21.15.47:0/298621257 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55578ff338b0 0x55578ff3c680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:30:16.216 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:16.212+0000 7fb176000640 1 -- 172.21.15.47:0/298621257 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55578ff34260 msgr2=0x55578ff39a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:30:16.217 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:16.213+0000 7fb176000640 1 --2- 172.21.15.47:0/298621257 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55578ff34260 0x55578ff39a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:30:16.217 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:16.213+0000 7fb176000640 1 -- 172.21.15.47:0/298621257 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55578fbab3f0 con 0x55578ff3cde0 2024-03-21T10:30:16.217 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:16.213+0000 7fb174c00640 1 --2- 172.21.15.47:0/298621257 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55578ff34260 0x55578ff39a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T10:30:16.217 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:16.213+0000 7fb176000640 1 --2- 172.21.15.47:0/298621257 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55578ff3cde0 0x55578ff39f50 secure :-1 s=READY pgs=865 cs=0 l=1 rev1=1 crypto rx=0x7fb1680cc870 tx=0x7fb16809aed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d13cddf92131fc17 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:30:16.217 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:16.213+0000 7fb156a00640 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-03-21T10:30:16.217 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:16.213+0000 7fb156a00640 1 -- 172.21.15.47:0/298621257 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb168020040 con 0x55578ff3cde0 2024-03-21T10:30:16.217 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:16.213+0000 7fb156a00640 1 -- 172.21.15.47:0/298621257 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(97..97 src has 1..97) v4 ==== 9474+0+0 (secure 0 0 0) 0x7fb1680cf120 con 0x55578ff3cde0 2024-03-21T10:30:16.217 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:16.213+0000 7fb17ada2740 1 -- 172.21.15.47:0/298621257 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fb15c0035f0 con 0x55578ff3cde0 2024-03-21T10:30:16.217 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:16.213+0000 7fb156a00640 1 -- 172.21.15.47:0/298621257 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb16809b4b0 con 0x55578ff3cde0 2024-03-21T10:30:16.217 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:16.213+0000 7fb17ada2740 10 client.17190 fetch_fsmap learned FSMap version 32 2024-03-21T10:30:16.217 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:16.214+0000 7fb17ada2740 1 -- 172.21.15.47:0/298621257 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x55578fc757d0 con 0x55578ff3cde0 2024-03-21T10:30:16.217 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:16.214+0000 7fb156a00640 1 -- 172.21.15.47:0/298621257 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fb15c0035f0 con 0x55578ff3cde0 2024-03-21T10:30:16.217 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:16.214+0000 7fb156a00640 1 -- 172.21.15.47:0/298621257 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 32) v1 ==== 7942+0+0 (secure 0 0 0) 0x7fb168011d80 con 0x55578ff3cde0 2024-03-21T10:30:16.217 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:16.214+0000 7fb17ada2740 10 client.17190 fetch_fsmap finished waiting for FSMap version 32 2024-03-21T10:30:16.218 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:16.214+0000 7fb17ada2740 10 client.17190 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.e 2024-03-21T10:30:16.218 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:16.214+0000 7fb17ada2740 20 client.17190 populate_metadata read hostname 'smithi047' 2024-03-21T10:30:16.218 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:16.214+0000 7fb17ada2740 1 --2- 172.21.15.47:0/298621257 >> [v2:172.21.15.100:6834/2597849957,v1:172.21.15.100:6835/2597849957] conn(0x55578ff4cc00 0x55578ff4eff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:30:16.218 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:16.214+0000 7fb17ada2740 4 client.17190 mds_command: new command op to 34454 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T10:30:16.218 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:16.214+0000 7fb17ada2740 1 -- 172.21.15.47:0/298621257 --> [v2:172.21.15.100:6834/2597849957,v1:172.21.15.100:6835/2597849957] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55578fea4800 con 0x55578ff4cc00 2024-03-21T10:30:16.218 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:16.215+0000 7fb175600640 1 --2- 172.21.15.47:0/298621257 >> [v2:172.21.15.100:6834/2597849957,v1:172.21.15.100:6835/2597849957] conn(0x55578ff4cc00 0x55578ff4eff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:30:16.219 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:16.216+0000 7fb175600640 1 --2- 172.21.15.47:0/298621257 >> [v2:172.21.15.100:6834/2597849957,v1:172.21.15.100:6835/2597849957] conn(0x55578ff4cc00 0x55578ff4eff0 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=74289121d6001939 server_cookie=1d9dda5e565fdad9 in_seq=0 out_seq=0 2024-03-21T10:30:16.219 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:16.216+0000 7fb156a00640 10 client.17190 ms_handle_connect on v2:172.21.15.100:6834/2597849957 2024-03-21T10:30:16.219 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:16.217+0000 7fb156a00640 1 -- 172.21.15.47:0/298621257 <== mds.2 v2:172.21.15.100:6834/2597849957 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55578fea4800 con 0x55578ff4cc00 2024-03-21T10:30:16.219 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:16.217+0000 7fb156a00640 10 client.17190 handle_command_reply: tid=0 2024-03-21T10:30:16.219 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:16.217+0000 7fb17ada2740 1 client.17190 shutdown 2024-03-21T10:30:16.219 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:16.217+0000 7fb17ada2740 1 -- 172.21.15.47:0/298621257 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55578ff3cde0 msgr2=0x55578ff39f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:30:16.219 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:16.217+0000 7fb17ada2740 1 --2- 172.21.15.47:0/298621257 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55578ff3cde0 0x55578ff39f50 secure :-1 s=READY pgs=865 cs=0 l=1 rev1=1 crypto rx=0x7fb1680cc870 tx=0x7fb16809aed0 comp rx=0 tx=0).stop 2024-03-21T10:30:16.220 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:16.217+0000 7fb17ada2740 1 -- 172.21.15.47:0/298621257 shutdown_connections 2024-03-21T10:30:16.220 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:16.217+0000 7fb17ada2740 1 --2- 172.21.15.47:0/298621257 >> [v2:172.21.15.100:6834/2597849957,v1:172.21.15.100:6835/2597849957] conn(0x55578ff4cc00 0x55578ff4eff0 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:30:16.220 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:16.217+0000 7fb17ada2740 1 --2- 172.21.15.47:0/298621257 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55578ff3cde0 0x55578ff39f50 unknown :-1 s=CLOSED pgs=865 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:30:16.220 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:16.217+0000 7fb17ada2740 1 --2- 172.21.15.47:0/298621257 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55578ff34260 0x55578ff39a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:30:16.220 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:16.217+0000 7fb17ada2740 1 --2- 172.21.15.47:0/298621257 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55578ff338b0 0x55578ff3c680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:30:16.220 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:16.217+0000 7fb156a00640 0 client.17190 ms_handle_reset on v2:172.21.15.100:6834/2597849957 2024-03-21T10:30:16.220 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:16.217+0000 7fb17ada2740 1 -- 172.21.15.47:0/298621257 >> 172.21.15.47:0/298621257 conn(0x55578ff2df80 msgr2=0x55578ff2fcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:30:16.220 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:16.218+0000 7fb17ada2740 1 -- 172.21.15.47:0/298621257 shutdown_connections 2024-03-21T10:30:16.220 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:16.218+0000 7fb17ada2740 1 -- 172.21.15.47:0/298621257 wait complete. 2024-03-21T10:30:16.220 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:16.218+0000 7fb17ada2740 20 client.17190 trim_cache size 0 max 16384 2024-03-21T10:30:16.236 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:16.236+0000 7fb17ada2740 1 Processor -- start 2024-03-21T10:30:16.236 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:16.237+0000 7fb17ada2740 1 -- start start 2024-03-21T10:30:16.237 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:16.237+0000 7fb17ada2740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55578ff338b0 0x55578ff3c140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:30:16.238 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:16.237+0000 7fb17ada2740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55578ff34260 0x55578ff3c680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:30:16.238 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:16.237+0000 7fb17ada2740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55578ff3cde0 0x55578ff36c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:30:16.238 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:16.237+0000 7fb17ada2740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1701b0810 con 0x55578ff34260 2024-03-21T10:30:16.238 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:16.237+0000 7fb17ada2740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1701b2190 con 0x55578ff338b0 2024-03-21T10:30:16.238 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:16.237+0000 7fb17ada2740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1701b0e70 con 0x55578ff3cde0 2024-03-21T10:30:16.238 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:16.237+0000 7fb174c00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55578ff34260 0x55578ff3c680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:30:16.238 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:16.237+0000 7fb175600640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55578ff338b0 0x55578ff3c140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:30:16.238 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:16.237+0000 7fb175600640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55578ff338b0 0x55578ff3c140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:42276/0 (socket says 172.21.15.47:42276) 2024-03-21T10:30:16.238 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:16.237+0000 7fb175600640 1 -- 172.21.15.47:0/3336583153 learned_addr learned my addr 172.21.15.47:0/3336583153 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:30:16.238 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:16.238+0000 7fb176000640 1 --2- 172.21.15.47:0/3336583153 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55578ff3cde0 0x55578ff36c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:30:16.239 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:16.238+0000 7fb175600640 1 -- 172.21.15.47:0/3336583153 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55578ff3cde0 msgr2=0x55578ff36c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:30:16.239 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:16.238+0000 7fb175600640 1 --2- 172.21.15.47:0/3336583153 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55578ff3cde0 0x55578ff36c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:30:16.239 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:16.238+0000 7fb175600640 1 -- 172.21.15.47:0/3336583153 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55578ff34260 msgr2=0x55578ff3c680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:30:16.239 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:16.238+0000 7fb175600640 1 --2- 172.21.15.47:0/3336583153 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55578ff34260 0x55578ff3c680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:30:16.239 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:16.238+0000 7fb175600640 1 -- 172.21.15.47:0/3336583153 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55578ff2ddb0 con 0x55578ff338b0 2024-03-21T10:30:16.239 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:16.238+0000 7fb174c00640 1 --2- 172.21.15.47:0/3336583153 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55578ff34260 0x55578ff3c680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T10:30:16.239 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:16.238+0000 7fb175600640 1 --2- 172.21.15.47:0/3336583153 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55578ff338b0 0x55578ff3c140 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fb16402cce0 tx=0x7fb16403f000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f1ecc8c51932a23c server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:30:16.240 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:16.239+0000 7fb161c00640 1 -- 172.21.15.47:0/3336583153 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb16405d070 con 0x55578ff338b0 2024-03-21T10:30:16.240 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:16.239+0000 7fb161c00640 1 -- 172.21.15.47:0/3336583153 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fb164045040 con 0x55578ff338b0 2024-03-21T10:30:16.240 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:16.239+0000 7fb161c00640 1 -- 172.21.15.47:0/3336583153 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb16403fd30 con 0x55578ff338b0 2024-03-21T10:30:16.243 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:16.240+0000 7fb17ada2740 1 -- 172.21.15.47:0/3336583153 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55578ff338b0 msgr2=0x55578ff3c140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:30:16.243 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:16.240+0000 7fb17ada2740 1 --2- 172.21.15.47:0/3336583153 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55578ff338b0 0x55578ff3c140 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fb16402cce0 tx=0x7fb16403f000 comp rx=0 tx=0).stop 2024-03-21T10:30:16.243 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:16.240+0000 7fb17ada2740 1 -- 172.21.15.47:0/3336583153 shutdown_connections 2024-03-21T10:30:16.243 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:16.240+0000 7fb17ada2740 1 --2- 172.21.15.47:0/3336583153 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55578ff3cde0 0x55578ff36c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:30:16.243 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:16.240+0000 7fb17ada2740 1 --2- 172.21.15.47:0/3336583153 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55578ff34260 0x55578ff3c680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:30:16.243 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:16.240+0000 7fb17ada2740 1 --2- 172.21.15.47:0/3336583153 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55578ff338b0 0x55578ff3c140 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:30:16.243 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:16.240+0000 7fb17ada2740 1 -- 172.21.15.47:0/3336583153 >> 172.21.15.47:0/3336583153 conn(0x55578ff2df80 msgr2=0x55578ff2d160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:30:16.243 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:16.240+0000 7fb17ada2740 1 -- 172.21.15.47:0/3336583153 shutdown_connections 2024-03-21T10:30:16.243 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:16.240+0000 7fb17ada2740 1 -- 172.21.15.47:0/3336583153 wait complete. 2024-03-21T10:30:16.243 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:16.240+0000 7fb17ada2740 1 -- start start 2024-03-21T10:30:16.243 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:16.240+0000 7fb17ada2740 1 Processor -- start 2024-03-21T10:30:16.243 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:16.241+0000 7fb17ada2740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55578ff338b0 0x55578ff44be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:30:16.243 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:16.241+0000 7fb17ada2740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55578ff34260 0x55578ff40f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:30:16.243 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:16.241+0000 7fb17ada2740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55578ff3cde0 0x55578ff414a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:30:16.244 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:16.241+0000 7fb17ada2740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1701b1b30 con 0x55578ff338b0 2024-03-21T10:30:16.244 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:16.241+0000 7fb17ada2740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1701b14d0 con 0x55578ff34260 2024-03-21T10:30:16.244 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:16.241+0000 7fb17ada2740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1701ae9d0 con 0x55578ff3cde0 2024-03-21T10:30:16.244 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:16.241+0000 7fb17ada2740 10 client.0 _ll_register_callbacks cb 0x7fb1680046c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T10:30:16.244 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:16.241+0000 7fb175600640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55578ff338b0 0x55578ff44be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:30:16.244 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:16.241+0000 7fb175600640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55578ff338b0 0x55578ff44be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:38302/0 (socket says 172.21.15.47:38302) 2024-03-21T10:30:16.244 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:16.241+0000 7fb175600640 1 -- 172.21.15.47:0/565646531 learned_addr learned my addr 172.21.15.47:0/565646531 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:30:16.244 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:16.241+0000 7fb174c00640 1 --2- 172.21.15.47:0/565646531 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55578ff34260 0x55578ff40f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:30:16.244 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:16.241+0000 7fb176000640 1 --2- 172.21.15.47:0/565646531 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55578ff3cde0 0x55578ff414a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:30:16.244 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:16.242+0000 7fb175600640 1 -- 172.21.15.47:0/565646531 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55578ff3cde0 msgr2=0x55578ff414a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:30:16.244 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:16.242+0000 7fb175600640 1 --2- 172.21.15.47:0/565646531 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55578ff3cde0 0x55578ff414a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:30:16.244 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:16.242+0000 7fb175600640 1 -- 172.21.15.47:0/565646531 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55578ff34260 msgr2=0x55578ff40f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:30:16.244 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:16.242+0000 7fb175600640 1 --2- 172.21.15.47:0/565646531 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55578ff34260 0x55578ff40f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:30:16.244 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:16.242+0000 7fb175600640 1 -- 172.21.15.47:0/565646531 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55578ff419e0 con 0x55578ff338b0 2024-03-21T10:30:16.244 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:16.242+0000 7fb174c00640 1 --2- 172.21.15.47:0/565646531 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55578ff34260 0x55578ff40f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T10:30:16.245 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:16.242+0000 7fb175600640 1 --2- 172.21.15.47:0/565646531 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55578ff338b0 0x55578ff44be0 secure :-1 s=READY pgs=866 cs=0 l=1 rev1=1 crypto rx=0x7fb16402cb70 tx=0x7fb1640573c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=73b10fb80c3b721a server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:30:16.245 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:16.242+0000 7fb156a00640 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-03-21T10:30:16.245 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:16.242+0000 7fb156a00640 1 -- 172.21.15.47:0/565646531 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb164066060 con 0x55578ff338b0 2024-03-21T10:30:16.245 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:16.242+0000 7fb156a00640 1 -- 172.21.15.47:0/565646531 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(97..97 src has 1..97) v4 ==== 9474+0+0 (secure 0 0 0) 0x7fb164057cd0 con 0x55578ff338b0 2024-03-21T10:30:16.245 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:16.242+0000 7fb17ada2740 1 -- 172.21.15.47:0/565646531 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fb164045040 con 0x55578ff338b0 2024-03-21T10:30:16.245 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:16.243+0000 7fb156a00640 1 -- 172.21.15.47:0/565646531 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb16406d050 con 0x55578ff338b0 2024-03-21T10:30:16.245 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:16.243+0000 7fb17ada2740 10 client.17196 fetch_fsmap learned FSMap version 32 2024-03-21T10:30:16.245 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:16.243+0000 7fb17ada2740 1 -- 172.21.15.47:0/565646531 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x55578ffa9150 con 0x55578ff338b0 2024-03-21T10:30:16.247 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:16.243+0000 7fb156a00640 1 -- 172.21.15.47:0/565646531 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fb164045040 con 0x55578ff338b0 2024-03-21T10:30:16.247 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:16.243+0000 7fb156a00640 1 -- 172.21.15.47:0/565646531 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 32) v1 ==== 7942+0+0 (secure 0 0 0) 0x7fb1640349e0 con 0x55578ff338b0 2024-03-21T10:30:16.247 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:16.243+0000 7fb17ada2740 10 client.17196 fetch_fsmap finished waiting for FSMap version 32 2024-03-21T10:30:16.247 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:16.243+0000 7fb17ada2740 10 client.17196 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.e 2024-03-21T10:30:16.247 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:16.243+0000 7fb17ada2740 20 client.17196 populate_metadata read hostname 'smithi047' 2024-03-21T10:30:16.247 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:16.243+0000 7fb17ada2740 1 --2- 172.21.15.47:0/565646531 >> [v2:172.21.15.100:6834/2597849957,v1:172.21.15.100:6835/2597849957] conn(0x55578ffa97c0 0x55578ffabbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:30:16.247 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:16.243+0000 7fb17ada2740 4 client.17196 mds_command: new command op to 34454 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-21T10:30:16.247 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:16.243+0000 7fb17ada2740 1 -- 172.21.15.47:0/565646531 --> [v2:172.21.15.100:6834/2597849957,v1:172.21.15.100:6835/2597849957] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55578fe9eb50 con 0x55578ffa97c0 2024-03-21T10:30:16.247 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:16.244+0000 7fb174c00640 1 --2- 172.21.15.47:0/565646531 >> [v2:172.21.15.100:6834/2597849957,v1:172.21.15.100:6835/2597849957] conn(0x55578ffa97c0 0x55578ffabbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:30:16.247 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:16.245+0000 7fb174c00640 1 --2- 172.21.15.47:0/565646531 >> [v2:172.21.15.100:6834/2597849957,v1:172.21.15.100:6835/2597849957] conn(0x55578ffa97c0 0x55578ffabbe0 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=b66a13be82c6f8de server_cookie=b9cfd1f58e861d40 in_seq=0 out_seq=0 2024-03-21T10:30:16.248 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:16.245+0000 7fb156a00640 10 client.17196 ms_handle_connect on v2:172.21.15.100:6834/2597849957 2024-03-21T10:30:16.248 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:16.245+0000 7fb156a00640 1 -- 172.21.15.47:0/565646531 <== mds.2 v2:172.21.15.100:6834/2597849957 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55578fe9eb50 con 0x55578ffa97c0 2024-03-21T10:30:16.248 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:16.245+0000 7fb156a00640 10 client.17196 handle_command_reply: tid=0 2024-03-21T10:30:16.248 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:16.245+0000 7fb17ada2740 1 client.17196 shutdown 2024-03-21T10:30:16.248 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:16.245+0000 7fb17ada2740 1 -- 172.21.15.47:0/565646531 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55578ff338b0 msgr2=0x55578ff44be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:30:16.248 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:16.245+0000 7fb17ada2740 1 --2- 172.21.15.47:0/565646531 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55578ff338b0 0x55578ff44be0 secure :-1 s=READY pgs=866 cs=0 l=1 rev1=1 crypto rx=0x7fb16402cb70 tx=0x7fb1640573c0 comp rx=0 tx=0).stop 2024-03-21T10:30:16.248 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:16.245+0000 7fb17ada2740 1 -- 172.21.15.47:0/565646531 shutdown_connections 2024-03-21T10:30:16.248 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:16.245+0000 7fb17ada2740 1 --2- 172.21.15.47:0/565646531 >> [v2:172.21.15.100:6834/2597849957,v1:172.21.15.100:6835/2597849957] conn(0x55578ffa97c0 0x55578ffabbe0 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:30:16.248 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:16.245+0000 7fb17ada2740 1 --2- 172.21.15.47:0/565646531 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55578ff3cde0 0x55578ff414a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:30:16.248 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:16.245+0000 7fb17ada2740 1 --2- 172.21.15.47:0/565646531 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55578ff34260 0x55578ff40f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:30:16.248 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:16.245+0000 7fb17ada2740 1 --2- 172.21.15.47:0/565646531 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55578ff338b0 0x55578ff44be0 unknown :-1 s=CLOSED pgs=866 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:30:16.248 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:16.245+0000 7fb156a00640 0 client.17196 ms_handle_reset on v2:172.21.15.100:6834/2597849957 2024-03-21T10:30:16.248 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:16.245+0000 7fb17ada2740 1 -- 172.21.15.47:0/565646531 >> 172.21.15.47:0/565646531 conn(0x55578ff2df80 msgr2=0x55578ff34690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:30:16.248 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:16.245+0000 7fb17ada2740 1 -- 172.21.15.47:0/565646531 shutdown_connections 2024-03-21T10:30:16.248 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:16.246+0000 7fb17ada2740 1 -- 172.21.15.47:0/565646531 wait complete. 2024-03-21T10:30:16.249 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:16.246+0000 7fb17ada2740 20 client.17196 trim_cache size 0 max 16384 2024-03-21T10:30:16.249 INFO:teuthology.orchestra.run.smithi047.stdout:[] 2024-03-21T10:30:16.249 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:16.246+0000 7fb176a00640 1 -- 172.21.15.47:0/518978189 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7fb1580a3140 msgr2=0x7fb1580a5600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:30:16.250 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:16.246+0000 7fb176a00640 1 --2- 172.21.15.47:0/518978189 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7fb1580a3140 0x7fb1580a5600 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fb164004af0 tx=0x7fb164000f40 comp rx=0 tx=0).stop 2024-03-21T10:30:16.250 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:16.246+0000 7fb176a00640 1 -- 172.21.15.47:0/518978189 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb17010baf0 msgr2=0x7fb170100da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:30:16.250 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:16.246+0000 7fb176a00640 1 --2- 172.21.15.47:0/518978189 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb17010baf0 0x7fb170100da0 secure :-1 s=READY pgs=864 cs=0 l=1 rev1=1 crypto rx=0x7fb16800d750 tx=0x7fb16800dc10 comp rx=0 tx=0).stop 2024-03-21T10:30:16.250 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:16.246+0000 7fb176a00640 1 -- 172.21.15.47:0/518978189 shutdown_connections 2024-03-21T10:30:16.250 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:16.246+0000 7fb176a00640 1 --2- 172.21.15.47:0/518978189 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb17010baf0 0x7fb170100da0 unknown :-1 s=CLOSED pgs=864 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:30:16.250 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:16.246+0000 7fb176a00640 1 --2- 172.21.15.47:0/518978189 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb170102ff0 0x7fb170100860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:30:16.250 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:16.246+0000 7fb176a00640 1 --2- 172.21.15.47:0/518978189 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7fb1580a3140 0x7fb1580a5600 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:30:16.250 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:16.246+0000 7fb176a00640 1 --2- 172.21.15.47:0/518978189 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb1701026d0 0x7fb170100320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:30:16.250 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:16.246+0000 7fb176a00640 1 -- 172.21.15.47:0/518978189 >> 172.21.15.47:0/518978189 conn(0x7fb1700fbef0 msgr2=0x7fb1700fcca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:30:16.250 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:16.247+0000 7fb176a00640 1 -- 172.21.15.47:0/518978189 shutdown_connections 2024-03-21T10:30:16.250 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:16.247+0000 7fb176a00640 1 -- 172.21.15.47:0/518978189 wait complete. 2024-03-21T10:30:16.261 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:30:16 smithi047 ceph-mon[31547]: from='client.? 172.21.15.47:0/1495842099' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-21T10:30:16.265 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-21T10:30:16.265 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-03-21T10:30:16.478 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:30:16 smithi100 ceph-mon[35193]: from='client.? 172.21.15.47:0/1495842099' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-21T10:30:16.484 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:16.483+0000 7f3402800640 1 -- 172.21.15.47:0/1250744463 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f33fc108860 msgr2=0x7f33fc10c8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:30:16.484 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:16.483+0000 7f3402800640 1 --2- 172.21.15.47:0/1250744463 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f33fc108860 0x7f33fc10c8a0 secure :-1 s=READY pgs=867 cs=0 l=1 rev1=1 crypto rx=0x7f33e4009900 tx=0x7f33e402bc30 comp rx=0 tx=0).stop 2024-03-21T10:30:16.484 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:16.484+0000 7f3402800640 1 -- 172.21.15.47:0/1250744463 shutdown_connections 2024-03-21T10:30:16.484 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:16.484+0000 7f3402800640 1 --2- 172.21.15.47:0/1250744463 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f33fc10cf80 0x7f33fc114b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:30:16.484 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:16.484+0000 7f3402800640 1 --2- 172.21.15.47:0/1250744463 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f33fc108860 0x7f33fc10c8a0 unknown :-1 s=CLOSED pgs=867 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:30:16.484 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:16.484+0000 7f3402800640 1 --2- 172.21.15.47:0/1250744463 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f33fc107eb0 0x7f33fc108290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:30:16.484 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:16.484+0000 7f3402800640 1 -- 172.21.15.47:0/1250744463 >> 172.21.15.47:0/1250744463 conn(0x7f33fc075be0 msgr2=0x7f33fc078000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:30:16.485 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:16.484+0000 7f3402800640 1 -- 172.21.15.47:0/1250744463 shutdown_connections 2024-03-21T10:30:16.485 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:16.484+0000 7f3402800640 1 -- 172.21.15.47:0/1250744463 wait complete. 2024-03-21T10:30:16.485 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:16.485+0000 7f3402800640 1 Processor -- start 2024-03-21T10:30:16.485 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:16.485+0000 7f3402800640 1 -- start start 2024-03-21T10:30:16.486 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:16.486+0000 7f3402800640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f33fc107eb0 0x7f33fc19e470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:30:16.486 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:16.486+0000 7f3402800640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f33fc10cf80 0x7f33fc19e9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:30:16.486 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:16.486+0000 7f3402800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f33fc1a2d30 0x7f33fc1a31e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:30:16.486 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:16.486+0000 7f3402800640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33fc117f30 con 0x7f33fc10cf80 2024-03-21T10:30:16.486 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:16.486+0000 7f3402800640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33fc117db0 con 0x7f33fc1a2d30 2024-03-21T10:30:16.486 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:16.486+0000 7f3402800640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33fc1180b0 con 0x7f33fc107eb0 2024-03-21T10:30:16.488 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:16.486+0000 7f33f3e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f33fc10cf80 0x7f33fc19e9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:30:16.488 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:16.486+0000 7f33f3e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f33fc10cf80 0x7f33fc19e9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:38320/0 (socket says 172.21.15.47:38320) 2024-03-21T10:30:16.488 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:16.486+0000 7f33f3e00640 1 -- 172.21.15.47:0/3087132670 learned_addr learned my addr 172.21.15.47:0/3087132670 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:30:16.488 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:16.487+0000 7f33fbe00640 1 --2- 172.21.15.47:0/3087132670 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f33fc1a2d30 0x7f33fc1a31e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:30:16.488 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:16.487+0000 7f33fb400640 1 --2- 172.21.15.47:0/3087132670 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f33fc107eb0 0x7f33fc19e470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:30:16.488 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:16.487+0000 7f33f3e00640 1 -- 172.21.15.47:0/3087132670 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f33fc107eb0 msgr2=0x7f33fc19e470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:30:16.488 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:16.487+0000 7f33f3e00640 1 --2- 172.21.15.47:0/3087132670 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f33fc107eb0 0x7f33fc19e470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:30:16.488 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:16.487+0000 7f33f3e00640 1 -- 172.21.15.47:0/3087132670 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f33fc1a2d30 msgr2=0x7f33fc1a31e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:30:16.488 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:16.487+0000 7f33f3e00640 1 --2- 172.21.15.47:0/3087132670 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f33fc1a2d30 0x7f33fc1a31e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:30:16.488 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:16.487+0000 7f33f3e00640 1 -- 172.21.15.47:0/3087132670 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f33fc1a3820 con 0x7f33fc10cf80 2024-03-21T10:30:16.488 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:16.487+0000 7f33f3e00640 1 --2- 172.21.15.47:0/3087132670 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f33fc10cf80 0x7f33fc19e9b0 secure :-1 s=READY pgs=868 cs=0 l=1 rev1=1 crypto rx=0x7f33e4000c00 tx=0x7f33e402c660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:30:16.489 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:16.488+0000 7f33f8c00640 1 -- 172.21.15.47:0/3087132670 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33e4043020 con 0x7f33fc10cf80 2024-03-21T10:30:16.489 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:16.488+0000 7f3402800640 1 -- 172.21.15.47:0/3087132670 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f33fc1a3ab0 con 0x7f33fc10cf80 2024-03-21T10:30:16.489 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:16.488+0000 7f33f8c00640 1 -- 172.21.15.47:0/3087132670 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f33e402caf0 con 0x7f33fc10cf80 2024-03-21T10:30:16.489 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:16.488+0000 7f33f8c00640 1 -- 172.21.15.47:0/3087132670 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33e4003b40 con 0x7f33fc10cf80 2024-03-21T10:30:16.489 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:16.488+0000 7f3402800640 1 -- 172.21.15.47:0/3087132670 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f33fc1ab220 con 0x7f33fc10cf80 2024-03-21T10:30:16.489 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:30:16 smithi157 ceph-mon[35581]: from='client.? 172.21.15.47:0/1495842099' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-21T10:30:16.491 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:16.491+0000 7f3405591740 1 Processor -- start 2024-03-21T10:30:16.491 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:16.491+0000 7f33f8c00640 1 -- 172.21.15.47:0/3087132670 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 27) v1 ==== 137985+0+0 (secure 0 0 0) 0x7f33e4046020 con 0x7f33fc10cf80 2024-03-21T10:30:16.491 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:16.491+0000 7f3405591740 1 -- start start 2024-03-21T10:30:16.492 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:16.492+0000 7f3405591740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x559d160336d0 0x559d16033ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:30:16.493 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:16.492+0000 7f3405591740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559d16034080 0x559d1603c630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:30:16.493 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:16.492+0000 7f3405591740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559d1603cc00 0x559d1603f050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:30:16.493 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:16.492+0000 7f3405591740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x559d15f972c0 con 0x559d160336d0 2024-03-21T10:30:16.493 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:16.492+0000 7f3405591740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x559d15f3fc90 con 0x559d1603cc00 2024-03-21T10:30:16.493 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:16.492+0000 7f3405591740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x559d15f93360 con 0x559d16034080 2024-03-21T10:30:16.493 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:16.492+0000 7f33fbe00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x559d160336d0 0x559d16033ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:30:16.493 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:16.492+0000 7f33fbe00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x559d160336d0 0x559d16033ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:38328/0 (socket says 172.21.15.47:38328) 2024-03-21T10:30:16.493 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:16.492+0000 7f33fbe00640 1 -- 172.21.15.47:0/3610041588 learned_addr learned my addr 172.21.15.47:0/3610041588 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:30:16.493 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:16.492+0000 7f33f3e00640 1 --2- 172.21.15.47:0/3610041588 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559d1603cc00 0x559d1603f050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:30:16.493 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:16.492+0000 7f33fb400640 1 --2- 172.21.15.47:0/3610041588 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559d16034080 0x559d1603c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:30:16.493 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:16.493+0000 7f33fbe00640 1 -- 172.21.15.47:0/3610041588 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559d16034080 msgr2=0x559d1603c630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:30:16.493 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:16.493+0000 7f33fbe00640 1 --2- 172.21.15.47:0/3610041588 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559d16034080 0x559d1603c630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:30:16.493 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:16.493+0000 7f33fbe00640 1 -- 172.21.15.47:0/3610041588 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559d1603cc00 msgr2=0x559d1603f050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:30:16.494 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:16.493+0000 7f33fbe00640 1 --2- 172.21.15.47:0/3610041588 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559d1603cc00 0x559d1603f050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:30:16.494 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:16.493+0000 7f33fbe00640 1 -- 172.21.15.47:0/3610041588 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559d15f64080 con 0x559d160336d0 2024-03-21T10:30:16.494 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:16.493+0000 7f33fb400640 1 --2- 172.21.15.47:0/3610041588 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559d16034080 0x559d1603c630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T10:30:16.494 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:16.493+0000 7f33f8c00640 1 --2- 172.21.15.47:0/3087132670 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f33e00a3090 0x7f33e00a5550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:30:16.495 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:16.493+0000 7f33fbe00640 1 --2- 172.21.15.47:0/3610041588 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x559d160336d0 0x559d16033ab0 secure :-1 s=READY pgs=869 cs=0 l=1 rev1=1 crypto rx=0x7f33f4009f50 tx=0x7f33f40118d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=48a724e28833adb server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:30:16.495 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:16.493+0000 7f33f8c00640 1 -- 172.21.15.47:0/3087132670 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 9474+0+0 (secure 0 0 0) 0x7f33e40f1be0 con 0x7f33fc10cf80 2024-03-21T10:30:16.495 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:16.493+0000 7f33fb400640 1 --2- 172.21.15.47:0/3087132670 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f33e00a3090 0x7f33e00a5550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:30:16.495 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:16.493+0000 7f33f0c00640 1 -- 172.21.15.47:0/3610041588 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33f4004d20 con 0x559d160336d0 2024-03-21T10:30:16.496 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:16.493+0000 7f33f0c00640 1 -- 172.21.15.47:0/3610041588 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f33f4004050 con 0x559d160336d0 2024-03-21T10:30:16.496 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:16.493+0000 7f33f0c00640 1 -- 172.21.15.47:0/3610041588 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33f4005280 con 0x559d160336d0 2024-03-21T10:30:16.496 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:16.494+0000 7f3405591740 1 -- 172.21.15.47:0/3610041588 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x559d160336d0 msgr2=0x559d16033ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:30:16.496 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:16.494+0000 7f33fb400640 1 --2- 172.21.15.47:0/3087132670 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f33e00a3090 0x7f33e00a5550 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f33ec0095f0 tx=0x7f33ec005f90 comp rx=0 tx=0).ready entity=mgr.15687 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:30:16.496 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:16.494+0000 7f3405591740 1 --2- 172.21.15.47:0/3610041588 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x559d160336d0 0x559d16033ab0 secure :-1 s=READY pgs=869 cs=0 l=1 rev1=1 crypto rx=0x7f33f4009f50 tx=0x7f33f40118d0 comp rx=0 tx=0).stop 2024-03-21T10:30:16.496 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:16.494+0000 7f3405591740 1 -- 172.21.15.47:0/3610041588 shutdown_connections 2024-03-21T10:30:16.496 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:16.494+0000 7f3405591740 1 --2- 172.21.15.47:0/3610041588 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559d1603cc00 0x559d1603f050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:30:16.496 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:16.494+0000 7f3405591740 1 --2- 172.21.15.47:0/3610041588 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559d16034080 0x559d1603c630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:30:16.496 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:16.494+0000 7f3405591740 1 --2- 172.21.15.47:0/3610041588 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x559d160336d0 0x559d16033ab0 unknown :-1 s=CLOSED pgs=869 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:30:16.496 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:16.494+0000 7f3405591740 1 -- 172.21.15.47:0/3610041588 >> 172.21.15.47:0/3610041588 conn(0x559d1602dda0 msgr2=0x559d1602e180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:30:16.496 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:16.494+0000 7f3405591740 1 -- 172.21.15.47:0/3610041588 shutdown_connections 2024-03-21T10:30:16.496 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:16.494+0000 7f3405591740 1 -- 172.21.15.47:0/3610041588 wait complete. 2024-03-21T10:30:16.496 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:16.495+0000 7f3405591740 1 -- start start 2024-03-21T10:30:16.496 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:16.495+0000 7f3405591740 1 Processor -- start 2024-03-21T10:30:16.499 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:16.496+0000 7f3405591740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559d160336d0 0x559d1603c530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:30:16.499 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:16.496+0000 7f3405591740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559d16034080 0x559d160398c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:30:16.500 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:16.496+0000 7f3405591740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x559d1603cc00 0x559d16039ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:30:16.500 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:16.496+0000 7f3405591740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x559d1603fb70 con 0x559d1603cc00 2024-03-21T10:30:16.500 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:16.496+0000 7f3405591740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x559d15f530d0 con 0x559d160336d0 2024-03-21T10:30:16.500 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:16.496+0000 7f3405591740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x559d1603fcf0 con 0x559d16034080 2024-03-21T10:30:16.500 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:16.496+0000 7f3405591740 10 client.0 _ll_register_callbacks cb 0x559d15f57e80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T10:30:16.500 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:16.496+0000 7f33fbe00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x559d1603cc00 0x559d16039ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:30:16.500 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:16.496+0000 7f33fb400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559d160336d0 0x559d1603c530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:30:16.500 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:16.496+0000 7f33fbe00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x559d1603cc00 0x559d16039ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:38340/0 (socket says 172.21.15.47:38340) 2024-03-21T10:30:16.500 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:16.496+0000 7f33fb400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559d160336d0 0x559d1603c530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:42314/0 (socket says 172.21.15.47:42314) 2024-03-21T10:30:16.500 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:16.496+0000 7f33fbe00640 1 -- 172.21.15.47:0/2610885088 learned_addr learned my addr 172.21.15.47:0/2610885088 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:30:16.500 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:16.496+0000 7f33f3e00640 1 --2- 172.21.15.47:0/2610885088 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559d16034080 0x559d160398c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:30:16.500 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:16.497+0000 7f33fbe00640 1 -- 172.21.15.47:0/2610885088 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559d16034080 msgr2=0x559d160398c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:30:16.500 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:16.497+0000 7f33fbe00640 1 --2- 172.21.15.47:0/2610885088 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559d16034080 0x559d160398c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:30:16.500 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:16.497+0000 7f33fbe00640 1 -- 172.21.15.47:0/2610885088 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559d160336d0 msgr2=0x559d1603c530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:30:16.501 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:16.497+0000 7f33fbe00640 1 --2- 172.21.15.47:0/2610885088 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559d160336d0 0x559d1603c530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:30:16.501 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:16.497+0000 7f33fbe00640 1 -- 172.21.15.47:0/2610885088 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x559d15cab3f0 con 0x559d1603cc00 2024-03-21T10:30:16.501 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:16.497+0000 7f33fbe00640 1 --2- 172.21.15.47:0/2610885088 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x559d1603cc00 0x559d16039ed0 secure :-1 s=READY pgs=870 cs=0 l=1 rev1=1 crypto rx=0x7f33f4009250 tx=0x7f33f40044b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3655ad940c97a3d0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:30:16.501 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:16.497+0000 7f33e9600640 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-03-21T10:30:16.501 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:16.497+0000 7f33e9600640 1 -- 172.21.15.47:0/2610885088 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33f4028070 con 0x559d1603cc00 2024-03-21T10:30:16.501 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:16.497+0000 7f33e9600640 1 -- 172.21.15.47:0/2610885088 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(97..97 src has 1..97) v4 ==== 9474+0+0 (secure 0 0 0) 0x7f33f4005040 con 0x559d1603cc00 2024-03-21T10:30:16.501 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:16.497+0000 7f3405591740 1 -- 172.21.15.47:0/2610885088 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f33f4004050 con 0x559d1603cc00 2024-03-21T10:30:16.501 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:16.498+0000 7f33e9600640 1 -- 172.21.15.47:0/2610885088 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f33f4009be0 con 0x559d1603cc00 2024-03-21T10:30:16.501 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:16.498+0000 7f3405591740 10 client.17208 fetch_fsmap learned FSMap version 32 2024-03-21T10:30:16.501 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:16.498+0000 7f3405591740 1 -- 172.21.15.47:0/2610885088 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x559d15d757d0 con 0x559d1603cc00 2024-03-21T10:30:16.501 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:16.498+0000 7f33e9600640 1 -- 172.21.15.47:0/2610885088 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f33f4004050 con 0x559d1603cc00 2024-03-21T10:30:16.501 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:16.498+0000 7f33e9600640 1 -- 172.21.15.47:0/2610885088 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 32) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f33f4024640 con 0x559d1603cc00 2024-03-21T10:30:16.501 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:16.498+0000 7f3405591740 10 client.17208 fetch_fsmap finished waiting for FSMap version 32 2024-03-21T10:30:16.501 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:16.498+0000 7f3405591740 10 client.17208 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-03-21T10:30:16.501 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:16.498+0000 7f3405591740 20 client.17208 populate_metadata read hostname 'smithi047' 2024-03-21T10:30:16.502 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:16.498+0000 7f3405591740 1 --2- 172.21.15.47:0/2610885088 >> [v2:172.21.15.100:6836/3497644162,v1:172.21.15.100:6837/3497644162] conn(0x559d1604c970 0x559d1604ed60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:30:16.502 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:16.498+0000 7f3405591740 4 client.17208 mds_command: new command op to 34493 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T10:30:16.502 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:16.498+0000 7f3405591740 1 -- 172.21.15.47:0/2610885088 --> [v2:172.21.15.100:6836/3497644162,v1:172.21.15.100:6837/3497644162] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x559d15fa4800 con 0x559d1604c970 2024-03-21T10:30:16.502 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:16.499+0000 7f33fb400640 1 --2- 172.21.15.47:0/2610885088 >> [v2:172.21.15.100:6836/3497644162,v1:172.21.15.100:6837/3497644162] conn(0x559d1604c970 0x559d1604ed60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:30:16.502 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:16.499+0000 7f33fb400640 1 --2- 172.21.15.47:0/2610885088 >> [v2:172.21.15.100:6836/3497644162,v1:172.21.15.100:6837/3497644162] conn(0x559d1604c970 0x559d1604ed60 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=e3e56bb7467383cc server_cookie=99504e77e9607f7e in_seq=0 out_seq=0 2024-03-21T10:30:16.502 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:16.499+0000 7f33e9600640 10 client.17208 ms_handle_connect on v2:172.21.15.100:6836/3497644162 2024-03-21T10:30:16.503 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:16.500+0000 7f33e9600640 1 -- 172.21.15.47:0/2610885088 <== mds.1 v2:172.21.15.100:6836/3497644162 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x559d15fa4800 con 0x559d1604c970 2024-03-21T10:30:16.503 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:16.500+0000 7f33e9600640 10 client.17208 handle_command_reply: tid=0 2024-03-21T10:30:16.503 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:16.500+0000 7f3405591740 1 client.17208 shutdown 2024-03-21T10:30:16.503 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:16.501+0000 7f3405591740 1 -- 172.21.15.47:0/2610885088 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x559d1603cc00 msgr2=0x559d16039ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:30:16.503 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:16.501+0000 7f3405591740 1 --2- 172.21.15.47:0/2610885088 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x559d1603cc00 0x559d16039ed0 secure :-1 s=READY pgs=870 cs=0 l=1 rev1=1 crypto rx=0x7f33f4009250 tx=0x7f33f40044b0 comp rx=0 tx=0).stop 2024-03-21T10:30:16.503 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:16.501+0000 7f3405591740 1 -- 172.21.15.47:0/2610885088 shutdown_connections 2024-03-21T10:30:16.503 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:16.501+0000 7f3405591740 1 --2- 172.21.15.47:0/2610885088 >> [v2:172.21.15.100:6836/3497644162,v1:172.21.15.100:6837/3497644162] conn(0x559d1604c970 0x559d1604ed60 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:30:16.504 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:16.501+0000 7f3405591740 1 --2- 172.21.15.47:0/2610885088 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x559d1603cc00 0x559d16039ed0 unknown :-1 s=CLOSED pgs=870 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:30:16.504 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:16.501+0000 7f3405591740 1 --2- 172.21.15.47:0/2610885088 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559d16034080 0x559d160398c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:30:16.504 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:16.501+0000 7f33e9600640 0 client.17208 ms_handle_reset on v2:172.21.15.100:6836/3497644162 2024-03-21T10:30:16.504 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:16.501+0000 7f3405591740 1 --2- 172.21.15.47:0/2610885088 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559d160336d0 0x559d1603c530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:30:16.504 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:16.501+0000 7f3405591740 1 -- 172.21.15.47:0/2610885088 >> 172.21.15.47:0/2610885088 conn(0x559d1602dda0 msgr2=0x559d1602faf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:30:16.504 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:16.501+0000 7f3405591740 1 -- 172.21.15.47:0/2610885088 shutdown_connections 2024-03-21T10:30:16.504 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:16.501+0000 7f3405591740 1 -- 172.21.15.47:0/2610885088 wait complete. 2024-03-21T10:30:16.504 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:16.501+0000 7f3405591740 20 client.17208 trim_cache size 0 max 16384 2024-03-21T10:30:16.515 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:16.515+0000 7f3405591740 1 Processor -- start 2024-03-21T10:30:16.515 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:16.515+0000 7f3405591740 1 -- start start 2024-03-21T10:30:16.517 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:16.516+0000 7f3405591740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x559d160336d0 0x559d1602f1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:30:16.517 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:16.516+0000 7f3405591740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559d16034080 0x559d1603bff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:30:16.517 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:16.516+0000 7f3405591740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559d1603cc00 0x559d1603c530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:30:16.517 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:16.516+0000 7f3405591740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33fc1b4eb0 con 0x559d160336d0 2024-03-21T10:30:16.517 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:16.516+0000 7f3405591740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33fc1b6830 con 0x559d16034080 2024-03-21T10:30:16.517 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:16.516+0000 7f3405591740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33fc1b5510 con 0x559d1603cc00 2024-03-21T10:30:16.518 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:16.516+0000 7f33fb400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x559d160336d0 0x559d1602f1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:30:16.518 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:16.516+0000 7f33fb400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x559d160336d0 0x559d1602f1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:38350/0 (socket says 172.21.15.47:38350) 2024-03-21T10:30:16.518 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:16.516+0000 7f33fb400640 1 -- 172.21.15.47:0/1810780602 learned_addr learned my addr 172.21.15.47:0/1810780602 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:30:16.518 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:16.516+0000 7f33f3e00640 1 --2- 172.21.15.47:0/1810780602 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559d16034080 0x559d1603bff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:30:16.518 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:16.516+0000 7f33fbe00640 1 --2- 172.21.15.47:0/1810780602 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559d1603cc00 0x559d1603c530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:30:16.518 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:16.516+0000 7f33fb400640 1 -- 172.21.15.47:0/1810780602 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559d1603cc00 msgr2=0x559d1603c530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:30:16.518 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:16.516+0000 7f33fb400640 1 --2- 172.21.15.47:0/1810780602 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559d1603cc00 0x559d1603c530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:30:16.518 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:16.516+0000 7f33fb400640 1 -- 172.21.15.47:0/1810780602 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559d16034080 msgr2=0x559d1603bff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:30:16.518 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:16.516+0000 7f33fb400640 1 --2- 172.21.15.47:0/1810780602 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559d16034080 0x559d1603bff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:30:16.518 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:16.516+0000 7f33fb400640 1 -- 172.21.15.47:0/1810780602 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559d16033e20 con 0x559d160336d0 2024-03-21T10:30:16.518 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:16.516+0000 7f33fbe00640 1 --2- 172.21.15.47:0/1810780602 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559d1603cc00 0x559d1603c530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T10:30:16.518 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:16.516+0000 7f33f3e00640 1 --2- 172.21.15.47:0/1810780602 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559d16034080 0x559d1603bff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T10:30:16.518 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:16.517+0000 7f33fb400640 1 --2- 172.21.15.47:0/1810780602 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x559d160336d0 0x559d1602f1b0 secure :-1 s=READY pgs=871 cs=0 l=1 rev1=1 crypto rx=0x559d1602ee10 tx=0x7f33ec016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bbbef4227c23073e server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:30:16.518 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:16.517+0000 7f33ebe00640 1 -- 172.21.15.47:0/1810780602 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33ec004020 con 0x559d160336d0 2024-03-21T10:30:16.518 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:16.517+0000 7f33ebe00640 1 -- 172.21.15.47:0/1810780602 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f33ec0054d0 con 0x559d160336d0 2024-03-21T10:30:16.519 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:16.517+0000 7f33ebe00640 1 -- 172.21.15.47:0/1810780602 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33ec004b20 con 0x559d160336d0 2024-03-21T10:30:16.522 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:16.518+0000 7f3405591740 1 -- 172.21.15.47:0/1810780602 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x559d160336d0 msgr2=0x559d1602f1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:30:16.522 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:16.518+0000 7f3405591740 1 --2- 172.21.15.47:0/1810780602 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x559d160336d0 0x559d1602f1b0 secure :-1 s=READY pgs=871 cs=0 l=1 rev1=1 crypto rx=0x559d1602ee10 tx=0x7f33ec016000 comp rx=0 tx=0).stop 2024-03-21T10:30:16.522 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:16.518+0000 7f3405591740 1 -- 172.21.15.47:0/1810780602 shutdown_connections 2024-03-21T10:30:16.522 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:16.518+0000 7f3405591740 1 --2- 172.21.15.47:0/1810780602 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559d1603cc00 0x559d1603c530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:30:16.522 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:16.518+0000 7f3405591740 1 --2- 172.21.15.47:0/1810780602 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559d16034080 0x559d1603bff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:30:16.522 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:16.518+0000 7f3405591740 1 --2- 172.21.15.47:0/1810780602 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x559d160336d0 0x559d1602f1b0 unknown :-1 s=CLOSED pgs=871 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:30:16.523 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:16.518+0000 7f3405591740 1 -- 172.21.15.47:0/1810780602 >> 172.21.15.47:0/1810780602 conn(0x559d1602dda0 msgr2=0x559d160388f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:30:16.523 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:16.518+0000 7f3405591740 1 -- 172.21.15.47:0/1810780602 shutdown_connections 2024-03-21T10:30:16.523 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:16.518+0000 7f3405591740 1 -- 172.21.15.47:0/1810780602 wait complete. 2024-03-21T10:30:16.523 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:16.518+0000 7f3405591740 1 -- start start 2024-03-21T10:30:16.523 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:16.519+0000 7f3405591740 1 Processor -- start 2024-03-21T10:30:16.523 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:16.519+0000 7f3405591740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559d160336d0 0x559d160426c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:30:16.523 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:16.519+0000 7f3405591740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x559d16034080 0x559d16042c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:30:16.523 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:16.519+0000 7f3405591740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559d1603cc00 0x559d160a6fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:30:16.523 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:16.519+0000 7f3405591740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33fc1b61d0 con 0x559d16034080 2024-03-21T10:30:16.523 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:16.519+0000 7f3405591740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33fc1b5b70 con 0x559d1603cc00 2024-03-21T10:30:16.523 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:16.519+0000 7f3405591740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33fc1b3070 con 0x559d160336d0 2024-03-21T10:30:16.524 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:16.519+0000 7f3405591740 10 client.0 _ll_register_callbacks cb 0x7f33e40f5560 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T10:30:16.524 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:16.519+0000 7f33f3e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x559d16034080 0x559d16042c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:30:16.524 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:16.519+0000 7f33f3e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x559d16034080 0x559d16042c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:38362/0 (socket says 172.21.15.47:38362) 2024-03-21T10:30:16.524 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:16.519+0000 7f33f3e00640 1 -- 172.21.15.47:0/4148745710 learned_addr learned my addr 172.21.15.47:0/4148745710 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:30:16.524 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:16.520+0000 7f33fb400640 1 --2- 172.21.15.47:0/4148745710 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559d160336d0 0x559d160426c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:30:16.524 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:16.520+0000 7f33fbe00640 1 --2- 172.21.15.47:0/4148745710 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559d1603cc00 0x559d160a6fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:30:16.524 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:16.520+0000 7f33f3e00640 1 -- 172.21.15.47:0/4148745710 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559d160336d0 msgr2=0x559d160426c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:30:16.524 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:16.520+0000 7f33f3e00640 1 --2- 172.21.15.47:0/4148745710 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559d160336d0 0x559d160426c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:30:16.524 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:16.520+0000 7f33f3e00640 1 -- 172.21.15.47:0/4148745710 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559d1603cc00 msgr2=0x559d160a6fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:30:16.524 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:16.520+0000 7f33f3e00640 1 --2- 172.21.15.47:0/4148745710 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559d1603cc00 0x559d160a6fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:30:16.524 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:16.520+0000 7f33f3e00640 1 -- 172.21.15.47:0/4148745710 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x559d160434d0 con 0x559d16034080 2024-03-21T10:30:16.524 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:16.520+0000 7f33f3e00640 1 --2- 172.21.15.47:0/4148745710 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x559d16034080 0x559d16042c00 secure :-1 s=READY pgs=872 cs=0 l=1 rev1=1 crypto rx=0x7f33e4005ab0 tx=0x7f33e4009620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cb35aa4733aafbb server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:30:16.524 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:16.520+0000 7f33e9600640 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-03-21T10:30:16.525 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:16.520+0000 7f33e9600640 1 -- 172.21.15.47:0/4148745710 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33e40bd620 con 0x559d16034080 2024-03-21T10:30:16.525 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:16.520+0000 7f33e9600640 1 -- 172.21.15.47:0/4148745710 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(97..97 src has 1..97) v4 ==== 9474+0+0 (secure 0 0 0) 0x7f33e4005490 con 0x559d16034080 2024-03-21T10:30:16.525 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:16.520+0000 7f3405591740 1 -- 172.21.15.47:0/4148745710 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f33ec0054d0 con 0x559d16034080 2024-03-21T10:30:16.525 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:16.521+0000 7f33e9600640 1 -- 172.21.15.47:0/4148745710 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f33e402cca0 con 0x559d16034080 2024-03-21T10:30:16.525 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:16.521+0000 7f3405591740 10 client.17214 fetch_fsmap learned FSMap version 32 2024-03-21T10:30:16.525 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:16.521+0000 7f3405591740 1 -- 172.21.15.47:0/4148745710 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x559d160a85c0 con 0x559d16034080 2024-03-21T10:30:16.525 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:16.521+0000 7f33e9600640 1 -- 172.21.15.47:0/4148745710 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f33ec0054d0 con 0x559d16034080 2024-03-21T10:30:16.525 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:16.521+0000 7f33e9600640 1 -- 172.21.15.47:0/4148745710 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 32) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f33e40c6240 con 0x559d16034080 2024-03-21T10:30:16.525 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:16.521+0000 7f3405591740 10 client.17214 fetch_fsmap finished waiting for FSMap version 32 2024-03-21T10:30:16.525 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:16.521+0000 7f3405591740 10 client.17214 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-03-21T10:30:16.526 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:16.521+0000 7f3405591740 20 client.17214 populate_metadata read hostname 'smithi047' 2024-03-21T10:30:16.526 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:16.521+0000 7f3405591740 1 --2- 172.21.15.47:0/4148745710 >> [v2:172.21.15.100:6836/3497644162,v1:172.21.15.100:6837/3497644162] conn(0x559d160a8d40 0x559d160ab160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:30:16.526 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:16.521+0000 7f3405591740 4 client.17214 mds_command: new command op to 34493 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-21T10:30:16.526 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:16.521+0000 7f3405591740 1 -- 172.21.15.47:0/4148745710 --> [v2:172.21.15.100:6836/3497644162,v1:172.21.15.100:6837/3497644162] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x559d15f9eb50 con 0x559d160a8d40 2024-03-21T10:30:16.526 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:16.522+0000 7f33fb400640 1 --2- 172.21.15.47:0/4148745710 >> [v2:172.21.15.100:6836/3497644162,v1:172.21.15.100:6837/3497644162] conn(0x559d160a8d40 0x559d160ab160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:30:16.526 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:16.522+0000 7f33fb400640 1 --2- 172.21.15.47:0/4148745710 >> [v2:172.21.15.100:6836/3497644162,v1:172.21.15.100:6837/3497644162] conn(0x559d160a8d40 0x559d160ab160 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=b648f24305111889 server_cookie=c7ee3b5eb341e262 in_seq=0 out_seq=0 2024-03-21T10:30:16.526 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:16.522+0000 7f33e9600640 10 client.17214 ms_handle_connect on v2:172.21.15.100:6836/3497644162 2024-03-21T10:30:16.527 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:16.523+0000 7f33e9600640 1 -- 172.21.15.47:0/4148745710 <== mds.1 v2:172.21.15.100:6836/3497644162 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x559d15f9eb50 con 0x559d160a8d40 2024-03-21T10:30:16.527 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:16.523+0000 7f33e9600640 10 client.17214 handle_command_reply: tid=0 2024-03-21T10:30:16.527 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:16.523+0000 7f3405591740 1 client.17214 shutdown 2024-03-21T10:30:16.527 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:16.523+0000 7f3405591740 1 -- 172.21.15.47:0/4148745710 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x559d16034080 msgr2=0x559d16042c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:30:16.527 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:16.523+0000 7f3405591740 1 --2- 172.21.15.47:0/4148745710 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x559d16034080 0x559d16042c00 secure :-1 s=READY pgs=872 cs=0 l=1 rev1=1 crypto rx=0x7f33e4005ab0 tx=0x7f33e4009620 comp rx=0 tx=0).stop 2024-03-21T10:30:16.528 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:16.523+0000 7f3405591740 1 -- 172.21.15.47:0/4148745710 shutdown_connections 2024-03-21T10:30:16.528 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:16.523+0000 7f3405591740 1 --2- 172.21.15.47:0/4148745710 >> [v2:172.21.15.100:6836/3497644162,v1:172.21.15.100:6837/3497644162] conn(0x559d160a8d40 0x559d160ab160 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:30:16.528 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:16.523+0000 7f3405591740 1 --2- 172.21.15.47:0/4148745710 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559d1603cc00 0x559d160a6fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:30:16.528 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:16.523+0000 7f3405591740 1 --2- 172.21.15.47:0/4148745710 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x559d16034080 0x559d16042c00 unknown :-1 s=CLOSED pgs=872 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:30:16.528 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:16.523+0000 7f3405591740 1 --2- 172.21.15.47:0/4148745710 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559d160336d0 0x559d160426c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:30:16.528 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:16.523+0000 7f3405591740 1 -- 172.21.15.47:0/4148745710 >> 172.21.15.47:0/4148745710 conn(0x559d1602dda0 msgr2=0x559d16038060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:30:16.528 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:16.523+0000 7f33e9600640 0 client.17214 ms_handle_reset on v2:172.21.15.100:6836/3497644162 2024-03-21T10:30:16.528 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:16.523+0000 7f3405591740 1 -- 172.21.15.47:0/4148745710 shutdown_connections 2024-03-21T10:30:16.528 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:16.523+0000 7f3405591740 1 -- 172.21.15.47:0/4148745710 wait complete. 2024-03-21T10:30:16.528 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:16.523+0000 7f3405591740 20 client.17214 trim_cache size 0 max 16384 2024-03-21T10:30:16.528 INFO:teuthology.orchestra.run.smithi047.stdout:[] 2024-03-21T10:30:16.529 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:16.524+0000 7f3402800640 1 -- 172.21.15.47:0/3087132670 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f33e00a3090 msgr2=0x7f33e00a5550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:30:16.529 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:16.524+0000 7f3402800640 1 --2- 172.21.15.47:0/3087132670 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f33e00a3090 0x7f33e00a5550 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f33ec0095f0 tx=0x7f33ec005f90 comp rx=0 tx=0).stop 2024-03-21T10:30:16.529 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:16.524+0000 7f3402800640 1 -- 172.21.15.47:0/3087132670 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f33fc10cf80 msgr2=0x7f33fc19e9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:30:16.529 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:16.524+0000 7f3402800640 1 --2- 172.21.15.47:0/3087132670 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f33fc10cf80 0x7f33fc19e9b0 secure :-1 s=READY pgs=868 cs=0 l=1 rev1=1 crypto rx=0x7f33e4000c00 tx=0x7f33e402c660 comp rx=0 tx=0).stop 2024-03-21T10:30:16.529 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:16.524+0000 7f3402800640 1 -- 172.21.15.47:0/3087132670 shutdown_connections 2024-03-21T10:30:16.529 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:16.524+0000 7f3402800640 1 --2- 172.21.15.47:0/3087132670 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f33e00a3090 0x7f33e00a5550 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:30:16.530 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:16.524+0000 7f3402800640 1 --2- 172.21.15.47:0/3087132670 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f33fc1a2d30 0x7f33fc1a31e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:30:16.530 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:16.524+0000 7f3402800640 1 --2- 172.21.15.47:0/3087132670 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f33fc10cf80 0x7f33fc19e9b0 unknown :-1 s=CLOSED pgs=868 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:30:16.530 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:16.524+0000 7f3402800640 1 --2- 172.21.15.47:0/3087132670 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f33fc107eb0 0x7f33fc19e470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:30:16.530 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:16.524+0000 7f3402800640 1 -- 172.21.15.47:0/3087132670 >> 172.21.15.47:0/3087132670 conn(0x7f33fc075be0 msgr2=0x7f33fc077f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:30:16.530 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:16.524+0000 7f3402800640 1 -- 172.21.15.47:0/3087132670 shutdown_connections 2024-03-21T10:30:16.530 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:16.524+0000 7f3402800640 1 -- 172.21.15.47:0/3087132670 wait complete. 2024-03-21T10:30:16.541 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-21T10:30:16.541 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-03-21T10:30:16.753 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:16.752+0000 7f70f4200640 1 -- 172.21.15.47:0/2521118961 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f70ec10cf80 msgr2=0x7f70ec114b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:30:16.753 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:16.752+0000 7f70f4200640 1 --2- 172.21.15.47:0/2521118961 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f70ec10cf80 0x7f70ec114b50 secure :-1 s=READY pgs=873 cs=0 l=1 rev1=1 crypto rx=0x7f70e4009ca0 tx=0x7f70e402c0f0 comp rx=0 tx=0).stop 2024-03-21T10:30:16.753 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:16.752+0000 7f70f4200640 1 -- 172.21.15.47:0/2521118961 shutdown_connections 2024-03-21T10:30:16.753 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:16.752+0000 7f70f4200640 1 --2- 172.21.15.47:0/2521118961 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f70ec10cf80 0x7f70ec114b50 unknown :-1 s=CLOSED pgs=873 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:30:16.753 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:16.752+0000 7f70f4200640 1 --2- 172.21.15.47:0/2521118961 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f70ec108860 0x7f70ec10c8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:30:16.753 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:16.752+0000 7f70f4200640 1 --2- 172.21.15.47:0/2521118961 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f70ec107eb0 0x7f70ec108290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:30:16.753 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:16.752+0000 7f70f4200640 1 -- 172.21.15.47:0/2521118961 >> 172.21.15.47:0/2521118961 conn(0x7f70ec075be0 msgr2=0x7f70ec078000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:30:16.753 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:16.752+0000 7f70f4200640 1 -- 172.21.15.47:0/2521118961 shutdown_connections 2024-03-21T10:30:16.753 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:16.752+0000 7f70f4200640 1 -- 172.21.15.47:0/2521118961 wait complete. 2024-03-21T10:30:16.756 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:16.754+0000 7f70f4200640 1 Processor -- start 2024-03-21T10:30:16.756 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:16.754+0000 7f70f4200640 1 -- start start 2024-03-21T10:30:16.756 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:16.754+0000 7f70f4200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f70ec107eb0 0x7f70ec19e440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:30:16.756 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:16.754+0000 7f70f4200640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f70ec10cf80 0x7f70ec19e980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:30:16.756 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:16.754+0000 7f70f4200640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f70ec1a2d00 0x7f70ec1a31b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:30:16.757 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:16.754+0000 7f70f4200640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70ec117ee0 con 0x7f70ec10cf80 2024-03-21T10:30:16.757 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:16.754+0000 7f70f4200640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70ec117d60 con 0x7f70ec107eb0 2024-03-21T10:30:16.757 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:16.754+0000 7f70f4200640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70ec118060 con 0x7f70ec1a2d00 2024-03-21T10:30:16.757 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:16.755+0000 7f70f0e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f70ec10cf80 0x7f70ec19e980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:30:16.757 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:16.755+0000 7f70f0e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f70ec10cf80 0x7f70ec19e980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:38394/0 (socket says 172.21.15.47:38394) 2024-03-21T10:30:16.757 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:16.755+0000 7f70f0e00640 1 -- 172.21.15.47:0/3283878643 learned_addr learned my addr 172.21.15.47:0/3283878643 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:30:16.757 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:16.755+0000 7f70f1800640 1 --2- 172.21.15.47:0/3283878643 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f70ec107eb0 0x7f70ec19e440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:30:16.757 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:16.755+0000 7f70f2200640 1 --2- 172.21.15.47:0/3283878643 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f70ec1a2d00 0x7f70ec1a31b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:30:16.757 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:16.755+0000 7f70f0e00640 1 -- 172.21.15.47:0/3283878643 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f70ec1a2d00 msgr2=0x7f70ec1a31b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:30:16.757 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:16.755+0000 7f70f0e00640 1 --2- 172.21.15.47:0/3283878643 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f70ec1a2d00 0x7f70ec1a31b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:30:16.757 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:16.755+0000 7f70f0e00640 1 -- 172.21.15.47:0/3283878643 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f70ec107eb0 msgr2=0x7f70ec19e440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:30:16.757 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:16.755+0000 7f70f0e00640 1 --2- 172.21.15.47:0/3283878643 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f70ec107eb0 0x7f70ec19e440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:30:16.757 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:16.755+0000 7f70f0e00640 1 -- 172.21.15.47:0/3283878643 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70ec1a37f0 con 0x7f70ec10cf80 2024-03-21T10:30:16.758 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:16.756+0000 7f70f0e00640 1 --2- 172.21.15.47:0/3283878643 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f70ec10cf80 0x7f70ec19e980 secure :-1 s=READY pgs=874 cs=0 l=1 rev1=1 crypto rx=0x7f70d800e9c0 tx=0x7f70d800ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:30:16.758 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:16.756+0000 7f70ea000640 1 -- 172.21.15.47:0/3283878643 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70d800cd80 con 0x7f70ec10cf80 2024-03-21T10:30:16.758 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:16.756+0000 7f70ea000640 1 -- 172.21.15.47:0/3283878643 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f70d80054d0 con 0x7f70ec10cf80 2024-03-21T10:30:16.758 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:16.756+0000 7f70ea000640 1 -- 172.21.15.47:0/3283878643 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70d8010430 con 0x7f70ec10cf80 2024-03-21T10:30:16.758 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:16.756+0000 7f70f4200640 1 -- 172.21.15.47:0/3283878643 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f70ec1a3ae0 con 0x7f70ec10cf80 2024-03-21T10:30:16.758 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:16.756+0000 7f70f4200640 1 -- 172.21.15.47:0/3283878643 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f70ec1ab220 con 0x7f70ec10cf80 2024-03-21T10:30:16.760 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:16.760+0000 7f70f6e58740 1 Processor -- start 2024-03-21T10:30:16.760 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:16.760+0000 7f70f6e58740 1 -- start start 2024-03-21T10:30:16.761 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:16.760+0000 7f70ea000640 1 -- 172.21.15.47:0/3283878643 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 27) v1 ==== 137985+0+0 (secure 0 0 0) 0x7f70d8004fd0 con 0x7f70ec10cf80 2024-03-21T10:30:16.762 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:16.760+0000 7f70f6e58740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x557d906a26d0 0x557d906a2ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:30:16.762 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:16.760+0000 7f70f6e58740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557d906a3080 0x557d906ab630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:30:16.762 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:16.761+0000 7f70f6e58740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557d906abc00 0x557d906ae050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:30:16.762 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:16.761+0000 7f70f6e58740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x557d906062c0 con 0x557d906a26d0 2024-03-21T10:30:16.762 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:16.761+0000 7f70f6e58740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x557d905aec90 con 0x557d906abc00 2024-03-21T10:30:16.762 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:16.761+0000 7f70f6e58740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x557d90602360 con 0x557d906a3080 2024-03-21T10:30:16.762 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:16.761+0000 7f70f2200640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x557d906a26d0 0x557d906a2ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:30:16.762 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:16.761+0000 7f70f2200640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x557d906a26d0 0x557d906a2ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:38402/0 (socket says 172.21.15.47:38402) 2024-03-21T10:30:16.763 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:16.761+0000 7f70f2200640 1 -- 172.21.15.47:0/2101448165 learned_addr learned my addr 172.21.15.47:0/2101448165 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:30:16.763 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:16.761+0000 7f70f0e00640 1 --2- 172.21.15.47:0/2101448165 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557d906abc00 0x557d906ae050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:30:16.763 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:16.761+0000 7f70f1800640 1 --2- 172.21.15.47:0/2101448165 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557d906a3080 0x557d906ab630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:30:16.763 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:16.761+0000 7f70f2200640 1 -- 172.21.15.47:0/2101448165 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557d906a3080 msgr2=0x557d906ab630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:30:16.763 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:16.761+0000 7f70f2200640 1 --2- 172.21.15.47:0/2101448165 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557d906a3080 0x557d906ab630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:30:16.763 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:16.761+0000 7f70f2200640 1 -- 172.21.15.47:0/2101448165 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557d906abc00 msgr2=0x557d906ae050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:30:16.763 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:16.761+0000 7f70f2200640 1 --2- 172.21.15.47:0/2101448165 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557d906abc00 0x557d906ae050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:30:16.763 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:16.761+0000 7f70f2200640 1 -- 172.21.15.47:0/2101448165 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557d905d3080 con 0x557d906a26d0 2024-03-21T10:30:16.763 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:16.762+0000 7f70f2200640 1 --2- 172.21.15.47:0/2101448165 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x557d906a26d0 0x557d906a2ab0 secure :-1 s=READY pgs=875 cs=0 l=1 rev1=1 crypto rx=0x7f70e402c700 tx=0x7f70e4032340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d6a8771441a5561a server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:30:16.767 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:16.762+0000 7f70ea000640 1 --2- 172.21.15.47:0/3283878643 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f70d40a3110 0x7f70d40a55d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:30:16.767 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:16.762+0000 7f70dea00640 1 -- 172.21.15.47:0/2101448165 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70e4043020 con 0x557d906a26d0 2024-03-21T10:30:16.767 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:16.762+0000 7f70dea00640 1 -- 172.21.15.47:0/2101448165 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f70e4032bb0 con 0x557d906a26d0 2024-03-21T10:30:16.767 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:16.762+0000 7f70dea00640 1 -- 172.21.15.47:0/2101448165 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70e4039040 con 0x557d906a26d0 2024-03-21T10:30:16.767 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:16.762+0000 7f70ea000640 1 -- 172.21.15.47:0/3283878643 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 9474+0+0 (secure 0 0 0) 0x7f70d80cff20 con 0x7f70ec10cf80 2024-03-21T10:30:16.768 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:16.762+0000 7f70f1800640 1 --2- 172.21.15.47:0/3283878643 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f70d40a3110 0x7f70d40a55d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:30:16.768 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:16.763+0000 7f70f6e58740 1 -- 172.21.15.47:0/2101448165 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x557d906a26d0 msgr2=0x557d906a2ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:30:16.768 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:16.763+0000 7f70f6e58740 1 --2- 172.21.15.47:0/2101448165 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x557d906a26d0 0x557d906a2ab0 secure :-1 s=READY pgs=875 cs=0 l=1 rev1=1 crypto rx=0x7f70e402c700 tx=0x7f70e4032340 comp rx=0 tx=0).stop 2024-03-21T10:30:16.768 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:16.763+0000 7f70f1800640 1 --2- 172.21.15.47:0/3283878643 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f70d40a3110 0x7f70d40a55d0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x557d9069de10 tx=0x7f70e0005f90 comp rx=0 tx=0).ready entity=mgr.15687 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:30:16.768 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:16.763+0000 7f70f6e58740 1 -- 172.21.15.47:0/2101448165 shutdown_connections 2024-03-21T10:30:16.768 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:16.763+0000 7f70f6e58740 1 --2- 172.21.15.47:0/2101448165 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557d906abc00 0x557d906ae050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:30:16.768 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:16.763+0000 7f70f6e58740 1 --2- 172.21.15.47:0/2101448165 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557d906a3080 0x557d906ab630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:30:16.768 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:16.763+0000 7f70f6e58740 1 --2- 172.21.15.47:0/2101448165 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x557d906a26d0 0x557d906a2ab0 unknown :-1 s=CLOSED pgs=875 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:30:16.768 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:16.763+0000 7f70f6e58740 1 -- 172.21.15.47:0/2101448165 >> 172.21.15.47:0/2101448165 conn(0x557d9069cda0 msgr2=0x557d9069d180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:30:16.768 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:16.763+0000 7f70f6e58740 1 -- 172.21.15.47:0/2101448165 shutdown_connections 2024-03-21T10:30:16.768 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:16.763+0000 7f70f6e58740 1 -- 172.21.15.47:0/2101448165 wait complete. 2024-03-21T10:30:16.768 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:16.764+0000 7f70f6e58740 1 -- start start 2024-03-21T10:30:16.769 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:16.764+0000 7f70f6e58740 1 Processor -- start 2024-03-21T10:30:16.769 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:16.764+0000 7f70f6e58740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557d906a26d0 0x557d906ab530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:30:16.769 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:16.764+0000 7f70f6e58740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x557d906a3080 0x557d906a88c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:30:16.769 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:16.764+0000 7f70f6e58740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557d906abc00 0x557d906a8ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:30:16.769 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:16.764+0000 7f70f6e58740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x557d906aeb10 con 0x557d906a3080 2024-03-21T10:30:16.769 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:16.764+0000 7f70f6e58740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x557d905c20d0 con 0x557d906a26d0 2024-03-21T10:30:16.769 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:16.764+0000 7f70f6e58740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x557d906aec90 con 0x557d906abc00 2024-03-21T10:30:16.769 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:16.764+0000 7f70f6e58740 10 client.0 _ll_register_callbacks cb 0x557d905c6e80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T10:30:16.769 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:16.765+0000 7f70f0e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x557d906a3080 0x557d906a88c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:30:16.769 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:16.765+0000 7f70f0e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x557d906a3080 0x557d906a88c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:38414/0 (socket says 172.21.15.47:38414) 2024-03-21T10:30:16.769 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:16.765+0000 7f70f0e00640 1 -- 172.21.15.47:0/2371358791 learned_addr learned my addr 172.21.15.47:0/2371358791 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:30:16.769 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:16.765+0000 7f70f1800640 1 --2- 172.21.15.47:0/2371358791 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557d906a26d0 0x557d906ab530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:30:16.769 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:16.765+0000 7f70f0e00640 1 -- 172.21.15.47:0/2371358791 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557d906abc00 msgr2=0x557d906a8ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:30:16.769 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:16.765+0000 7f70f2200640 1 --2- 172.21.15.47:0/2371358791 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557d906abc00 0x557d906a8ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:30:16.769 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:16.765+0000 7f70f0e00640 1 --2- 172.21.15.47:0/2371358791 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557d906abc00 0x557d906a8ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:30:16.770 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:16.765+0000 7f70f0e00640 1 -- 172.21.15.47:0/2371358791 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557d906a26d0 msgr2=0x557d906ab530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:30:16.770 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:16.765+0000 7f70f0e00640 1 --2- 172.21.15.47:0/2371358791 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557d906a26d0 0x557d906ab530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:30:16.770 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:16.765+0000 7f70f0e00640 1 -- 172.21.15.47:0/2371358791 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x557d9031a3f0 con 0x557d906a3080 2024-03-21T10:30:16.770 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:16.765+0000 7f70f0e00640 1 --2- 172.21.15.47:0/2371358791 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x557d906a3080 0x557d906a88c0 secure :-1 s=READY pgs=876 cs=0 l=1 rev1=1 crypto rx=0x7f70d809bee0 tx=0x7f70d80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=be36e1ce916d0324 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:30:16.770 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:16.765+0000 7f70d3400640 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-03-21T10:30:16.770 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:16.766+0000 7f70d3400640 1 -- 172.21.15.47:0/2371358791 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70d809ba90 con 0x557d906a3080 2024-03-21T10:30:16.770 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:16.766+0000 7f70d3400640 1 -- 172.21.15.47:0/2371358791 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(97..97 src has 1..97) v4 ==== 9474+0+0 (secure 0 0 0) 0x7f70d80985c0 con 0x557d906a3080 2024-03-21T10:30:16.770 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:16.766+0000 7f70f6e58740 1 -- 172.21.15.47:0/2371358791 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f70e4032bb0 con 0x557d906a3080 2024-03-21T10:30:16.770 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:16.766+0000 7f70d3400640 1 -- 172.21.15.47:0/2371358791 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f70d809b8b0 con 0x557d906a3080 2024-03-21T10:30:16.770 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:16.766+0000 7f70f6e58740 10 client.17226 fetch_fsmap learned FSMap version 32 2024-03-21T10:30:16.770 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:16.766+0000 7f70f6e58740 1 -- 172.21.15.47:0/2371358791 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x557d903e47d0 con 0x557d906a3080 2024-03-21T10:30:16.770 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:16.766+0000 7f70d3400640 1 -- 172.21.15.47:0/2371358791 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f70e4032bb0 con 0x557d906a3080 2024-03-21T10:30:16.770 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:16.766+0000 7f70d3400640 1 -- 172.21.15.47:0/2371358791 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 32) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f70d80a44d0 con 0x557d906a3080 2024-03-21T10:30:16.771 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:16.767+0000 7f70f6e58740 10 client.17226 fetch_fsmap finished waiting for FSMap version 32 2024-03-21T10:30:16.771 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:16.767+0000 7f70f6e58740 10 client.17226 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-21T10:30:16.771 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:16.767+0000 7f70f6e58740 20 client.17226 populate_metadata read hostname 'smithi047' 2024-03-21T10:30:16.771 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:16.767+0000 7f70f6e58740 1 --2- 172.21.15.47:0/2371358791 >> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] conn(0x557d906bba80 0x557d906bde70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:30:16.771 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:16.767+0000 7f70f6e58740 4 client.17226 mds_command: new command op to 34535 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T10:30:16.771 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:16.767+0000 7f70f6e58740 1 -- 172.21.15.47:0/2371358791 --> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x557d90613800 con 0x557d906bba80 2024-03-21T10:30:16.771 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:16.767+0000 7f70f1800640 1 --2- 172.21.15.47:0/2371358791 >> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] conn(0x557d906bba80 0x557d906bde70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:30:16.772 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:16.768+0000 7f70f1800640 1 --2- 172.21.15.47:0/2371358791 >> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] conn(0x557d906bba80 0x557d906bde70 crc :-1 s=READY pgs=139 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f108dbe4b51946ed server_cookie=a81daa11c69cd731 in_seq=0 out_seq=0 2024-03-21T10:30:16.772 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:16.768+0000 7f70d3400640 10 client.17226 ms_handle_connect on v2:172.21.15.100:6838/4173168440 2024-03-21T10:30:16.772 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:16.769+0000 7f70d3400640 1 -- 172.21.15.47:0/2371358791 <== mds.0 v2:172.21.15.100:6838/4173168440 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x557d90613800 con 0x557d906bba80 2024-03-21T10:30:16.772 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:16.769+0000 7f70d3400640 10 client.17226 handle_command_reply: tid=0 2024-03-21T10:30:16.773 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:16.769+0000 7f70f6e58740 1 client.17226 shutdown 2024-03-21T10:30:16.773 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:16.769+0000 7f70f6e58740 1 -- 172.21.15.47:0/2371358791 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x557d906a3080 msgr2=0x557d906a88c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:30:16.773 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:16.769+0000 7f70f6e58740 1 --2- 172.21.15.47:0/2371358791 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x557d906a3080 0x557d906a88c0 secure :-1 s=READY pgs=876 cs=0 l=1 rev1=1 crypto rx=0x7f70d809bee0 tx=0x7f70d80046c0 comp rx=0 tx=0).stop 2024-03-21T10:30:16.773 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:16.770+0000 7f70f6e58740 1 -- 172.21.15.47:0/2371358791 shutdown_connections 2024-03-21T10:30:16.773 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:16.770+0000 7f70f6e58740 1 --2- 172.21.15.47:0/2371358791 >> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] conn(0x557d906bba80 0x557d906bde70 crc :-1 s=READY pgs=139 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:30:16.773 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:16.770+0000 7f70f6e58740 1 --2- 172.21.15.47:0/2371358791 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557d906abc00 0x557d906a8ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:30:16.773 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:16.770+0000 7f70f6e58740 1 --2- 172.21.15.47:0/2371358791 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x557d906a3080 0x557d906a88c0 unknown :-1 s=CLOSED pgs=876 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:30:16.773 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:16.770+0000 7f70f6e58740 1 --2- 172.21.15.47:0/2371358791 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557d906a26d0 0x557d906ab530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:30:16.773 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:16.770+0000 7f70d3400640 0 client.17226 ms_handle_reset on v2:172.21.15.100:6838/4173168440 2024-03-21T10:30:16.773 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:16.770+0000 7f70f6e58740 1 -- 172.21.15.47:0/2371358791 >> 172.21.15.47:0/2371358791 conn(0x557d9069cda0 msgr2=0x557d9069eaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:30:16.773 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:16.770+0000 7f70f6e58740 1 -- 172.21.15.47:0/2371358791 shutdown_connections 2024-03-21T10:30:16.774 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:16.770+0000 7f70f6e58740 1 -- 172.21.15.47:0/2371358791 wait complete. 2024-03-21T10:30:16.774 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:16.770+0000 7f70f6e58740 20 client.17226 trim_cache size 0 max 16384 2024-03-21T10:30:16.784 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:16.783+0000 7f70f6e58740 1 Processor -- start 2024-03-21T10:30:16.784 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:16.784+0000 7f70f6e58740 1 -- start start 2024-03-21T10:30:16.784 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:16.784+0000 7f70f6e58740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557d906a26d0 0x557d906aaff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:30:16.784 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:16.784+0000 7f70f6e58740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x557d906a3080 0x557d906ab530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:30:16.784 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:16.784+0000 7f70f6e58740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557d906abc00 0x557d9069bd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:30:16.784 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:16.784+0000 7f70f6e58740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70ec1b4e90 con 0x557d906a3080 2024-03-21T10:30:16.784 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:16.784+0000 7f70f6e58740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70ec1b6810 con 0x557d906abc00 2024-03-21T10:30:16.785 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:16.784+0000 7f70f6e58740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70ec1b54f0 con 0x557d906a26d0 2024-03-21T10:30:16.785 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:16.784+0000 7f70f0e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x557d906a3080 0x557d906ab530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:30:16.785 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:16.784+0000 7f70f0e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x557d906a3080 0x557d906ab530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:38422/0 (socket says 172.21.15.47:38422) 2024-03-21T10:30:16.785 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:16.784+0000 7f70f0e00640 1 -- 172.21.15.47:0/1431210238 learned_addr learned my addr 172.21.15.47:0/1431210238 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:30:16.785 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:16.784+0000 7f70f2200640 1 --2- 172.21.15.47:0/1431210238 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557d906abc00 0x557d9069bd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:30:16.787 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:16.785+0000 7f70f1800640 1 --2- 172.21.15.47:0/1431210238 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557d906a26d0 0x557d906aaff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:30:16.787 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:16.785+0000 7f70f0e00640 1 -- 172.21.15.47:0/1431210238 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557d906a26d0 msgr2=0x557d906aaff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:30:16.787 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:16.785+0000 7f70f0e00640 1 --2- 172.21.15.47:0/1431210238 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557d906a26d0 0x557d906aaff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:30:16.787 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:16.785+0000 7f70f0e00640 1 -- 172.21.15.47:0/1431210238 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557d906abc00 msgr2=0x557d9069bd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:30:16.787 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:16.785+0000 7f70f0e00640 1 --2- 172.21.15.47:0/1431210238 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557d906abc00 0x557d9069bd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:30:16.787 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:16.785+0000 7f70f0e00640 1 -- 172.21.15.47:0/1431210238 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557d906a2e20 con 0x557d906a3080 2024-03-21T10:30:16.787 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:16.785+0000 7f70f0e00640 1 --2- 172.21.15.47:0/1431210238 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x557d906a3080 0x557d906ab530 secure :-1 s=READY pgs=877 cs=0 l=1 rev1=1 crypto rx=0x7f70d809bc50 tx=0x7f70d80d3760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4d44b71b7f91564e server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:30:16.787 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:16.785+0000 7f70de000640 1 -- 172.21.15.47:0/1431210238 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70d809ba90 con 0x557d906a3080 2024-03-21T10:30:16.787 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:16.785+0000 7f70de000640 1 -- 172.21.15.47:0/1431210238 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f70d8098000 con 0x557d906a3080 2024-03-21T10:30:16.787 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:16.785+0000 7f70de000640 1 -- 172.21.15.47:0/1431210238 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70d80981f0 con 0x557d906a3080 2024-03-21T10:30:16.787 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:16.786+0000 7f70f6e58740 1 -- 172.21.15.47:0/1431210238 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x557d906a3080 msgr2=0x557d906ab530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:30:16.787 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:16.786+0000 7f70f6e58740 1 --2- 172.21.15.47:0/1431210238 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x557d906a3080 0x557d906ab530 secure :-1 s=READY pgs=877 cs=0 l=1 rev1=1 crypto rx=0x7f70d809bc50 tx=0x7f70d80d3760 comp rx=0 tx=0).stop 2024-03-21T10:30:16.787 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:16.786+0000 7f70f6e58740 1 -- 172.21.15.47:0/1431210238 shutdown_connections 2024-03-21T10:30:16.788 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:16.786+0000 7f70f6e58740 1 --2- 172.21.15.47:0/1431210238 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557d906abc00 0x557d9069bd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:30:16.788 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:16.786+0000 7f70f6e58740 1 --2- 172.21.15.47:0/1431210238 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x557d906a3080 0x557d906ab530 unknown :-1 s=CLOSED pgs=877 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:30:16.788 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:16.786+0000 7f70f6e58740 1 --2- 172.21.15.47:0/1431210238 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557d906a26d0 0x557d906aaff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:30:16.788 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:16.786+0000 7f70f6e58740 1 -- 172.21.15.47:0/1431210238 >> 172.21.15.47:0/1431210238 conn(0x557d9069cda0 msgr2=0x557d9069b620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:30:16.788 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:16.786+0000 7f70f6e58740 1 -- 172.21.15.47:0/1431210238 shutdown_connections 2024-03-21T10:30:16.788 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:16.786+0000 7f70f6e58740 1 -- 172.21.15.47:0/1431210238 wait complete. 2024-03-21T10:30:16.788 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:16.786+0000 7f70f6e58740 1 -- start start 2024-03-21T10:30:16.788 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:16.787+0000 7f70f6e58740 1 Processor -- start 2024-03-21T10:30:16.791 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:16.787+0000 7f70f6e58740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557d906a26d0 0x557d906b3c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:30:16.791 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:16.787+0000 7f70f6e58740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x557d906a3080 0x557d906b1fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:30:16.791 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:16.787+0000 7f70f6e58740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557d906abc00 0x557d906b2500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:30:16.791 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:16.787+0000 7f70f6e58740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70ec1b61b0 con 0x557d906a3080 2024-03-21T10:30:16.791 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:16.787+0000 7f70f6e58740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70ec1b5b50 con 0x557d906a26d0 2024-03-21T10:30:16.791 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:16.787+0000 7f70f6e58740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70ec1b3050 con 0x557d906abc00 2024-03-21T10:30:16.791 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:16.787+0000 7f70f6e58740 10 client.0 _ll_register_callbacks cb 0x7f70d8009700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T10:30:16.791 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:16.788+0000 7f70f0e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x557d906a3080 0x557d906b1fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:30:16.791 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:16.788+0000 7f70f0e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x557d906a3080 0x557d906b1fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:38434/0 (socket says 172.21.15.47:38434) 2024-03-21T10:30:16.791 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:16.788+0000 7f70f0e00640 1 -- 172.21.15.47:0/980899592 learned_addr learned my addr 172.21.15.47:0/980899592 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:30:16.791 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:16.788+0000 7f70f1800640 1 --2- 172.21.15.47:0/980899592 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557d906a26d0 0x557d906b3c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:30:16.792 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:16.788+0000 7f70f2200640 1 --2- 172.21.15.47:0/980899592 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557d906abc00 0x557d906b2500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:30:16.792 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:16.788+0000 7f70f0e00640 1 -- 172.21.15.47:0/980899592 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557d906abc00 msgr2=0x557d906b2500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:30:16.792 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:16.788+0000 7f70f0e00640 1 --2- 172.21.15.47:0/980899592 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557d906abc00 0x557d906b2500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:30:16.792 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:16.788+0000 7f70f0e00640 1 -- 172.21.15.47:0/980899592 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557d906a26d0 msgr2=0x557d906b3c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:30:16.792 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:16.788+0000 7f70f0e00640 1 --2- 172.21.15.47:0/980899592 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557d906a26d0 0x557d906b3c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:30:16.792 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:16.788+0000 7f70f0e00640 1 -- 172.21.15.47:0/980899592 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x557d906b2d30 con 0x557d906a3080 2024-03-21T10:30:16.792 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:16.788+0000 7f70f2200640 1 --2- 172.21.15.47:0/980899592 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557d906abc00 0x557d906b2500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T10:30:16.792 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:16.788+0000 7f70f0e00640 1 --2- 172.21.15.47:0/980899592 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x557d906a3080 0x557d906b1fc0 secure :-1 s=READY pgs=878 cs=0 l=1 rev1=1 crypto rx=0x7f70d8097e30 tx=0x7f70d80a4780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f73a595d529c7d42 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:30:16.792 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:16.788+0000 7f70d3400640 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-03-21T10:30:16.792 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:16.789+0000 7f70d3400640 1 -- 172.21.15.47:0/980899592 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70d80981f0 con 0x557d906a3080 2024-03-21T10:30:16.792 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:16.789+0000 7f70d3400640 1 -- 172.21.15.47:0/980899592 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(97..97 src has 1..97) v4 ==== 9474+0+0 (secure 0 0 0) 0x7f70d80850b0 con 0x557d906a3080 2024-03-21T10:30:16.792 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:16.789+0000 7f70f6e58740 1 -- 172.21.15.47:0/980899592 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x557d9060db50 con 0x557d906a3080 2024-03-21T10:30:16.792 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:16.789+0000 7f70d3400640 1 -- 172.21.15.47:0/980899592 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f70d809ba90 con 0x557d906a3080 2024-03-21T10:30:16.793 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:16.789+0000 7f70f6e58740 10 client.17232 fetch_fsmap learned FSMap version 32 2024-03-21T10:30:16.793 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:16.789+0000 7f70f6e58740 1 -- 172.21.15.47:0/980899592 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x557d90718290 con 0x557d906a3080 2024-03-21T10:30:16.793 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:16.789+0000 7f70d3400640 1 -- 172.21.15.47:0/980899592 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x557d9060db50 con 0x557d906a3080 2024-03-21T10:30:16.793 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:16.789+0000 7f70d3400640 1 -- 172.21.15.47:0/980899592 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 32) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f70d80ad080 con 0x557d906a3080 2024-03-21T10:30:16.793 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:16.790+0000 7f70f6e58740 10 client.17232 fetch_fsmap finished waiting for FSMap version 32 2024-03-21T10:30:16.793 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:16.790+0000 7f70f6e58740 10 client.17232 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-21T10:30:16.793 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:16.790+0000 7f70f6e58740 20 client.17232 populate_metadata read hostname 'smithi047' 2024-03-21T10:30:16.793 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:16.790+0000 7f70f6e58740 1 --2- 172.21.15.47:0/980899592 >> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] conn(0x557d907185d0 0x557d9071a9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:30:16.793 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:16.790+0000 7f70f6e58740 4 client.17232 mds_command: new command op to 34535 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-21T10:30:16.793 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:16.790+0000 7f70f6e58740 1 -- 172.21.15.47:0/980899592 --> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x557d9058b500 con 0x557d907185d0 2024-03-21T10:30:16.793 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:16.790+0000 7f70f1800640 1 --2- 172.21.15.47:0/980899592 >> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] conn(0x557d907185d0 0x557d9071a9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:30:16.793 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:16.791+0000 7f70f1800640 1 --2- 172.21.15.47:0/980899592 >> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] conn(0x557d907185d0 0x557d9071a9c0 crc :-1 s=READY pgs=140 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ae78ac93af341d4a server_cookie=53d19b35fd04dc2f in_seq=0 out_seq=0 2024-03-21T10:30:16.793 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:16.791+0000 7f70d3400640 10 client.17232 ms_handle_connect on v2:172.21.15.100:6838/4173168440 2024-03-21T10:30:16.793 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:16.791+0000 7f70d3400640 1 -- 172.21.15.47:0/980899592 <== mds.0 v2:172.21.15.100:6838/4173168440 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x557d9058b500 con 0x557d907185d0 2024-03-21T10:30:16.793 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:16.791+0000 7f70d3400640 10 client.17232 handle_command_reply: tid=0 2024-03-21T10:30:16.794 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:16.791+0000 7f70f6e58740 1 client.17232 shutdown 2024-03-21T10:30:16.795 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:16.791+0000 7f70f6e58740 1 -- 172.21.15.47:0/980899592 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x557d906a3080 msgr2=0x557d906b1fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:30:16.795 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:16.791+0000 7f70f6e58740 1 --2- 172.21.15.47:0/980899592 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x557d906a3080 0x557d906b1fc0 secure :-1 s=READY pgs=878 cs=0 l=1 rev1=1 crypto rx=0x7f70d8097e30 tx=0x7f70d80a4780 comp rx=0 tx=0).stop 2024-03-21T10:30:16.795 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:16.791+0000 7f70f6e58740 1 -- 172.21.15.47:0/980899592 shutdown_connections 2024-03-21T10:30:16.795 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:16.791+0000 7f70f6e58740 1 --2- 172.21.15.47:0/980899592 >> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] conn(0x557d907185d0 0x557d9071a9c0 crc :-1 s=READY pgs=140 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:30:16.795 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:16.791+0000 7f70f6e58740 1 --2- 172.21.15.47:0/980899592 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557d906abc00 0x557d906b2500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:30:16.795 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:16.791+0000 7f70f6e58740 1 --2- 172.21.15.47:0/980899592 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x557d906a3080 0x557d906b1fc0 unknown :-1 s=CLOSED pgs=878 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:30:16.795 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:16.791+0000 7f70f6e58740 1 --2- 172.21.15.47:0/980899592 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557d906a26d0 0x557d906b3c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:30:16.795 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:16.791+0000 7f70f6e58740 1 -- 172.21.15.47:0/980899592 >> 172.21.15.47:0/980899592 conn(0x557d9069cda0 msgr2=0x557d9069beb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:30:16.795 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:16.791+0000 7f70d3400640 0 client.17232 ms_handle_reset on v2:172.21.15.100:6838/4173168440 2024-03-21T10:30:16.795 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:16.792+0000 7f70f6e58740 1 -- 172.21.15.47:0/980899592 shutdown_connections 2024-03-21T10:30:16.795 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:16.792+0000 7f70f6e58740 1 -- 172.21.15.47:0/980899592 wait complete. 2024-03-21T10:30:16.795 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:16.792+0000 7f70f6e58740 20 client.17232 trim_cache size 0 max 16384 2024-03-21T10:30:16.796 INFO:teuthology.orchestra.run.smithi047.stdout:[] 2024-03-21T10:30:16.796 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:16.792+0000 7f70f4200640 1 -- 172.21.15.47:0/3283878643 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f70d40a3110 msgr2=0x7f70d40a55d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:30:16.796 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:16.792+0000 7f70f4200640 1 --2- 172.21.15.47:0/3283878643 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f70d40a3110 0x7f70d40a55d0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x557d9069de10 tx=0x7f70e0005f90 comp rx=0 tx=0).stop 2024-03-21T10:30:16.797 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:16.792+0000 7f70f4200640 1 -- 172.21.15.47:0/3283878643 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f70ec10cf80 msgr2=0x7f70ec19e980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:30:16.797 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:16.792+0000 7f70f4200640 1 --2- 172.21.15.47:0/3283878643 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f70ec10cf80 0x7f70ec19e980 secure :-1 s=READY pgs=874 cs=0 l=1 rev1=1 crypto rx=0x7f70d800e9c0 tx=0x7f70d800ee80 comp rx=0 tx=0).stop 2024-03-21T10:30:16.797 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:16.793+0000 7f70f4200640 1 -- 172.21.15.47:0/3283878643 shutdown_connections 2024-03-21T10:30:16.797 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:16.793+0000 7f70f4200640 1 --2- 172.21.15.47:0/3283878643 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f70ec1a2d00 0x7f70ec1a31b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:30:16.797 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:16.793+0000 7f70f4200640 1 --2- 172.21.15.47:0/3283878643 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f70ec10cf80 0x7f70ec19e980 unknown :-1 s=CLOSED pgs=874 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:30:16.797 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:16.793+0000 7f70f4200640 1 --2- 172.21.15.47:0/3283878643 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f70d40a3110 0x7f70d40a55d0 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:30:16.797 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:16.793+0000 7f70f4200640 1 --2- 172.21.15.47:0/3283878643 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f70ec107eb0 0x7f70ec19e440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:30:16.797 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:16.793+0000 7f70f4200640 1 -- 172.21.15.47:0/3283878643 >> 172.21.15.47:0/3283878643 conn(0x7f70ec075be0 msgr2=0x7f70ec077f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:30:16.797 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:16.793+0000 7f70f4200640 1 -- 172.21.15.47:0/3283878643 shutdown_connections 2024-03-21T10:30:16.797 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:16.793+0000 7f70f4200640 1 -- 172.21.15.47:0/3283878643 wait complete. 2024-03-21T10:30:17.488 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:30:17 smithi157 ceph-mon[35581]: pgmap v152: 129 pgs: 129 active+clean; 277 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 170 B/s wr, 12 op/s 2024-03-21T10:30:17.712 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:30:17 smithi047 ceph-mon[31547]: pgmap v152: 129 pgs: 129 active+clean; 277 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 170 B/s wr, 12 op/s 2024-03-21T10:30:17.728 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:30:17 smithi100 ceph-mon[35193]: pgmap v152: 129 pgs: 129 active+clean; 277 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 170 B/s wr, 12 op/s 2024-03-21T10:30:17.810 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-03-21T10:30:17.811 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-21T10:30:17.811 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-03-21T10:30:18.049 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:18.049+0000 7fa8bba00640 1 -- 172.21.15.47:0/1713055786 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa8b4103840 msgr2=0x7fa8b4103cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:30:18.049 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:18.049+0000 7fa8bba00640 1 --2- 172.21.15.47:0/1713055786 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa8b4103840 0x7fa8b4103cc0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fa8a0009900 tx=0x7fa8a002bc40 comp rx=0 tx=0).stop 2024-03-21T10:30:18.049 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:18.049+0000 7fa8bba00640 1 -- 172.21.15.47:0/1713055786 shutdown_connections 2024-03-21T10:30:18.050 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:18.049+0000 7fa8bba00640 1 --2- 172.21.15.47:0/1713055786 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa8b4104200 0x7fa8b4110710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:30:18.050 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:18.049+0000 7fa8bba00640 1 --2- 172.21.15.47:0/1713055786 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa8b4103840 0x7fa8b4103cc0 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:30:18.050 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:18.049+0000 7fa8bba00640 1 --2- 172.21.15.47:0/1713055786 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa8b4109850 0x7fa8b4109c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:30:18.050 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:18.049+0000 7fa8bba00640 1 -- 172.21.15.47:0/1713055786 >> 172.21.15.47:0/1713055786 conn(0x7fa8b40ff2f0 msgr2=0x7fa8b4101710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:30:18.050 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:18.049+0000 7fa8bba00640 1 -- 172.21.15.47:0/1713055786 shutdown_connections 2024-03-21T10:30:18.050 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:18.049+0000 7fa8bba00640 1 -- 172.21.15.47:0/1713055786 wait complete. 2024-03-21T10:30:18.050 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:18.050+0000 7fa8bba00640 1 Processor -- start 2024-03-21T10:30:18.051 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:18.051+0000 7fa8bba00640 1 -- start start 2024-03-21T10:30:18.051 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:18.051+0000 7fa8bba00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa8b4103840 0x7fa8b419a080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:30:18.051 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:18.051+0000 7fa8bba00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa8b4104200 0x7fa8b419a5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:30:18.051 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:18.051+0000 7fa8bba00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa8b4109850 0x7fa8b419e940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:30:18.051 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:18.051+0000 7fa8bba00640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8b4113b10 con 0x7fa8b4109850 2024-03-21T10:30:18.051 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:18.051+0000 7fa8bba00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8b4113990 con 0x7fa8b4104200 2024-03-21T10:30:18.052 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:18.051+0000 7fa8bba00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8b4113c90 con 0x7fa8b4103840 2024-03-21T10:30:18.053 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:18.052+0000 7fa8b9a00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa8b4109850 0x7fa8b419e940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:30:18.053 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:18.052+0000 7fa8b3e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa8b4104200 0x7fa8b419a5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:30:18.053 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:18.052+0000 7fa8b9a00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa8b4109850 0x7fa8b419e940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:38446/0 (socket says 172.21.15.47:38446) 2024-03-21T10:30:18.053 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:18.052+0000 7fa8b3e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa8b4104200 0x7fa8b419a5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:42400/0 (socket says 172.21.15.47:42400) 2024-03-21T10:30:18.053 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:18.052+0000 7fa8b9a00640 1 -- 172.21.15.47:0/1781034116 learned_addr learned my addr 172.21.15.47:0/1781034116 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:30:18.053 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:18.052+0000 7fa8b9000640 1 --2- 172.21.15.47:0/1781034116 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa8b4103840 0x7fa8b419a080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:30:18.053 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:18.052+0000 7fa8b3e00640 1 -- 172.21.15.47:0/1781034116 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa8b4103840 msgr2=0x7fa8b419a080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:30:18.053 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:18.052+0000 7fa8b3e00640 1 --2- 172.21.15.47:0/1781034116 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa8b4103840 0x7fa8b419a080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:30:18.053 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:18.052+0000 7fa8b3e00640 1 -- 172.21.15.47:0/1781034116 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa8b4109850 msgr2=0x7fa8b419e940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:30:18.053 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:18.052+0000 7fa8b3e00640 1 --2- 172.21.15.47:0/1781034116 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa8b4109850 0x7fa8b419e940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:30:18.053 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:18.052+0000 7fa8b3e00640 1 -- 172.21.15.47:0/1781034116 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa8b419f070 con 0x7fa8b4104200 2024-03-21T10:30:18.053 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:18.052+0000 7fa8b9a00640 1 --2- 172.21.15.47:0/1781034116 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa8b4109850 0x7fa8b419e940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T10:30:18.053 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:18.052+0000 7fa8b9000640 1 --2- 172.21.15.47:0/1781034116 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa8b4103840 0x7fa8b419a080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T10:30:18.054 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:18.053+0000 7fa8b3e00640 1 --2- 172.21.15.47:0/1781034116 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa8b4104200 0x7fa8b419a5c0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fa8a0036040 tx=0x7fa8a002cd30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:30:18.054 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:18.053+0000 7fa8b1600640 1 -- 172.21.15.47:0/1781034116 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8a0043020 con 0x7fa8b4104200 2024-03-21T10:30:18.054 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:18.053+0000 7fa8bba00640 1 -- 172.21.15.47:0/1781034116 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa8b419f300 con 0x7fa8b4104200 2024-03-21T10:30:18.055 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:18.053+0000 7fa8b1600640 1 -- 172.21.15.47:0/1781034116 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fa8a0004490 con 0x7fa8b4104200 2024-03-21T10:30:18.055 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:18.053+0000 7fa8b1600640 1 -- 172.21.15.47:0/1781034116 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8a0003b80 con 0x7fa8b4104200 2024-03-21T10:30:18.055 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:18.053+0000 7fa8bba00640 1 -- 172.21.15.47:0/1781034116 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa8b41a6bf0 con 0x7fa8b4104200 2024-03-21T10:30:18.057 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:18.057+0000 7fa8b1600640 1 -- 172.21.15.47:0/1781034116 <== mon.2 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 27) v1 ==== 137985+0+0 (secure 0 0 0) 0x7fa8a0046020 con 0x7fa8b4104200 2024-03-21T10:30:18.057 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:18.058+0000 7fa8be77e740 1 Processor -- start 2024-03-21T10:30:18.058 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:18.058+0000 7fa8be77e740 1 -- start start 2024-03-21T10:30:18.059 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:18.058+0000 7fa8be77e740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ec3c384080 0x55ec3c384460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:30:18.059 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:18.058+0000 7fa8be77e740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ec3c3849a0 0x55ec3c38d040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:30:18.060 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:18.058+0000 7fa8be77e740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ec3c38d580 0x55ec3c38f970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:30:18.060 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:18.058+0000 7fa8be77e740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ec3c2e8eb0 con 0x55ec3c384080 2024-03-21T10:30:18.060 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:18.058+0000 7fa8be77e740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ec3c291a60 con 0x55ec3c3849a0 2024-03-21T10:30:18.060 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:18.058+0000 7fa8be77e740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ec3c2e4f50 con 0x55ec3c38d580 2024-03-21T10:30:18.060 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:18.058+0000 7fa8b1600640 1 --2- 172.21.15.47:0/1781034116 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7fa89c0a3090 0x7fa89c0a5550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:30:18.060 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:18.059+0000 7fa8b1600640 1 -- 172.21.15.47:0/1781034116 <== mon.2 v2:172.21.15.100:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 9474+0+0 (secure 0 0 0) 0x7fa8a00f16d0 con 0x7fa8b4104200 2024-03-21T10:30:18.060 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:18.059+0000 7fa8b9000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ec3c3849a0 0x55ec3c38d040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:30:18.060 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:18.059+0000 7fa8b9000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ec3c3849a0 0x55ec3c38d040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:42414/0 (socket says 172.21.15.47:42414) 2024-03-21T10:30:18.060 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:18.059+0000 7fa8b9a00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ec3c384080 0x55ec3c384460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:30:18.060 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:18.059+0000 7fa8b9000640 1 -- 172.21.15.47:0/2332269925 learned_addr learned my addr 172.21.15.47:0/2332269925 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:30:18.060 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:18.059+0000 7fa8b9a00640 1 --2- 172.21.15.47:0/1781034116 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7fa89c0a3090 0x7fa89c0a5550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:30:18.060 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:18.059+0000 7fa8b3e00640 1 --2- 172.21.15.47:0/2332269925 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ec3c38d580 0x55ec3c38f970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:30:18.060 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:18.059+0000 7fa8b9000640 1 -- 172.21.15.47:0/2332269925 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ec3c38d580 msgr2=0x55ec3c38f970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:30:18.060 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:18.059+0000 7fa8b9000640 1 --2- 172.21.15.47:0/2332269925 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ec3c38d580 0x55ec3c38f970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:30:18.061 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:18.059+0000 7fa8b9000640 1 -- 172.21.15.47:0/2332269925 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ec3c384080 msgr2=0x55ec3c384460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:30:18.061 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:18.059+0000 7fa8b9000640 1 --2- 172.21.15.47:0/2332269925 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ec3c384080 0x55ec3c384460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:30:18.061 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:18.059+0000 7fa8b9000640 1 -- 172.21.15.47:0/2332269925 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ec3c0c57e0 con 0x55ec3c3849a0 2024-03-21T10:30:18.061 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:18.060+0000 7fa8b3e00640 1 --2- 172.21.15.47:0/2332269925 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ec3c38d580 0x55ec3c38f970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T10:30:18.062 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:18.060+0000 7fa8b9000640 1 --2- 172.21.15.47:0/2332269925 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ec3c3849a0 0x55ec3c38d040 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fa8a800bcc0 tx=0x7fa8a800f730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4d4222152c8feeea server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:30:18.062 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:18.060+0000 7fa8b9a00640 1 --2- 172.21.15.47:0/1781034116 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7fa89c0a3090 0x7fa89c0a5550 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fa8ac0049d0 tx=0x7fa8ac009500 comp rx=0 tx=0).ready entity=mgr.15687 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:30:18.062 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:18.060+0000 7fa8a6000640 1 -- 172.21.15.47:0/2332269925 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8a8015070 con 0x55ec3c3849a0 2024-03-21T10:30:18.062 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:18.060+0000 7fa8a6000640 1 -- 172.21.15.47:0/2332269925 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fa8a801a910 con 0x55ec3c3849a0 2024-03-21T10:30:18.062 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:18.060+0000 7fa8a6000640 1 -- 172.21.15.47:0/2332269925 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8a8011a00 con 0x55ec3c3849a0 2024-03-21T10:30:18.062 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:18.061+0000 7fa8be77e740 1 -- 172.21.15.47:0/2332269925 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ec3c3849a0 msgr2=0x55ec3c38d040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:30:18.063 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:18.061+0000 7fa8be77e740 1 --2- 172.21.15.47:0/2332269925 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ec3c3849a0 0x55ec3c38d040 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fa8a800bcc0 tx=0x7fa8a800f730 comp rx=0 tx=0).stop 2024-03-21T10:30:18.063 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:18.061+0000 7fa8be77e740 1 -- 172.21.15.47:0/2332269925 shutdown_connections 2024-03-21T10:30:18.063 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:18.061+0000 7fa8be77e740 1 --2- 172.21.15.47:0/2332269925 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ec3c38d580 0x55ec3c38f970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:30:18.063 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:18.061+0000 7fa8be77e740 1 --2- 172.21.15.47:0/2332269925 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ec3c3849a0 0x55ec3c38d040 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:30:18.063 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:18.061+0000 7fa8be77e740 1 --2- 172.21.15.47:0/2332269925 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ec3c384080 0x55ec3c384460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:30:18.063 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:18.061+0000 7fa8be77e740 1 -- 172.21.15.47:0/2332269925 >> 172.21.15.47:0/2332269925 conn(0x55ec3c37e7b0 msgr2=0x55ec3c37eb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:30:18.063 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:18.061+0000 7fa8be77e740 1 -- 172.21.15.47:0/2332269925 shutdown_connections 2024-03-21T10:30:18.063 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:18.061+0000 7fa8be77e740 1 -- 172.21.15.47:0/2332269925 wait complete. 2024-03-21T10:30:18.063 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:18.062+0000 7fa8be77e740 1 -- start start 2024-03-21T10:30:18.063 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:18.062+0000 7fa8be77e740 1 Processor -- start 2024-03-21T10:30:18.066 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:18.063+0000 7fa8be77e740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ec3c384080 0x55ec3c38cec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:30:18.066 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:18.063+0000 7fa8be77e740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ec3c3849a0 0x55ec3c38a250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:30:18.066 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:18.063+0000 7fa8be77e740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ec3c38d580 0x55ec3c38a790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:30:18.067 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:18.063+0000 7fa8be77e740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ec3c390370 con 0x55ec3c38d580 2024-03-21T10:30:18.067 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:18.063+0000 7fa8be77e740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ec3c2a4eb0 con 0x55ec3c3849a0 2024-03-21T10:30:18.067 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:18.063+0000 7fa8be77e740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ec3c3904f0 con 0x55ec3c384080 2024-03-21T10:30:18.067 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:18.063+0000 7fa8be77e740 10 client.0 _ll_register_callbacks cb 0x55ec3c299150 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T10:30:18.067 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:18.063+0000 7fa8b9a00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ec3c38d580 0x55ec3c38a790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:30:18.067 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:18.063+0000 7fa8b3e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ec3c3849a0 0x55ec3c38a250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:30:18.067 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:18.063+0000 7fa8b3e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ec3c3849a0 0x55ec3c38a250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:42430/0 (socket says 172.21.15.47:42430) 2024-03-21T10:30:18.067 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:18.063+0000 7fa8b9a00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ec3c38d580 0x55ec3c38a790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:38468/0 (socket says 172.21.15.47:38468) 2024-03-21T10:30:18.067 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:18.063+0000 7fa8b3e00640 1 -- 172.21.15.47:0/3943153054 learned_addr learned my addr 172.21.15.47:0/3943153054 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:30:18.067 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:18.063+0000 7fa8b9000640 1 --2- 172.21.15.47:0/3943153054 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ec3c384080 0x55ec3c38cec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:30:18.067 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:18.064+0000 7fa8b3e00640 1 -- 172.21.15.47:0/3943153054 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ec3c384080 msgr2=0x55ec3c38cec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:30:18.067 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:18.064+0000 7fa8b3e00640 1 --2- 172.21.15.47:0/3943153054 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ec3c384080 0x55ec3c38cec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:30:18.067 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:18.064+0000 7fa8b3e00640 1 -- 172.21.15.47:0/3943153054 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ec3c38d580 msgr2=0x55ec3c38a790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:30:18.068 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:18.064+0000 7fa8b3e00640 1 --2- 172.21.15.47:0/3943153054 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ec3c38d580 0x55ec3c38a790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:30:18.068 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:18.064+0000 7fa8b3e00640 1 -- 172.21.15.47:0/3943153054 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55ec3c125f70 con 0x55ec3c3849a0 2024-03-21T10:30:18.068 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:18.064+0000 7fa8b9a00640 1 --2- 172.21.15.47:0/3943153054 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ec3c38d580 0x55ec3c38a790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T10:30:18.068 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:18.064+0000 7fa8b3e00640 1 --2- 172.21.15.47:0/3943153054 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ec3c3849a0 0x55ec3c38a250 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fa8a00ba180 tx=0x7fa8a00bd690 comp rx=0 tx=0).ready entity=mon.2 client_cookie=88e456566a319077 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:30:18.068 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:18.064+0000 7fa89aa00640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-21T10:30:18.068 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:18.064+0000 7fa89aa00640 1 -- 172.21.15.47:0/3943153054 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8a00d5020 con 0x55ec3c3849a0 2024-03-21T10:30:18.068 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:18.064+0000 7fa8be77e740 1 -- 172.21.15.47:0/3943153054 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fa8a801a910 con 0x55ec3c3849a0 2024-03-21T10:30:18.068 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:18.064+0000 7fa89aa00640 1 -- 172.21.15.47:0/3943153054 <== mon.2 v2:172.21.15.100:3300/0 2 ==== osd_map(97..97 src has 1..97) v4 ==== 9474+0+0 (secure 0 0 0) 0x7fa8a00a80b0 con 0x55ec3c3849a0 2024-03-21T10:30:18.068 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:18.065+0000 7fa89aa00640 1 -- 172.21.15.47:0/3943153054 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa8a00bdeb0 con 0x55ec3c3849a0 2024-03-21T10:30:18.068 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:18.065+0000 7fa8be77e740 10 client.37031 fetch_fsmap learned FSMap version 32 2024-03-21T10:30:18.068 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:18.065+0000 7fa8be77e740 1 -- 172.21.15.47:0/3943153054 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x55ec3bfbd170 con 0x55ec3c3849a0 2024-03-21T10:30:18.068 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:18.065+0000 7fa89aa00640 1 -- 172.21.15.47:0/3943153054 <== mon.2 v2:172.21.15.100:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fa8a801a910 con 0x55ec3c3849a0 2024-03-21T10:30:18.068 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:18.065+0000 7fa89aa00640 1 -- 172.21.15.47:0/3943153054 <== mon.2 v2:172.21.15.100:3300/0 5 ==== fsmap(e 32) v1 ==== 7942+0+0 (secure 0 0 0) 0x7fa8a00dc170 con 0x55ec3c3849a0 2024-03-21T10:30:18.068 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:18.065+0000 7fa8be77e740 10 client.37031 fetch_fsmap finished waiting for FSMap version 32 2024-03-21T10:30:18.069 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:18.065+0000 7fa8be77e740 10 client.37031 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-21T10:30:18.069 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:18.065+0000 7fa8be77e740 20 client.37031 populate_metadata read hostname 'smithi047' 2024-03-21T10:30:18.069 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:18.065+0000 7fa8be77e740 1 --2- 172.21.15.47:0/3943153054 >> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] conn(0x55ec3c39d5b0 0x55ec3c39f9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:30:18.069 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:18.065+0000 7fa8be77e740 4 client.37031 mds_command: new command op to 34535 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T10:30:18.069 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:18.065+0000 7fa8be77e740 1 -- 172.21.15.47:0/3943153054 --> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55ec3c2f63f0 con 0x55ec3c39d5b0 2024-03-21T10:30:18.069 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:18.066+0000 7fa8b9000640 1 --2- 172.21.15.47:0/3943153054 >> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] conn(0x55ec3c39d5b0 0x55ec3c39f9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:30:18.070 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:18.067+0000 7fa8b9000640 1 --2- 172.21.15.47:0/3943153054 >> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] conn(0x55ec3c39d5b0 0x55ec3c39f9f0 crc :-1 s=READY pgs=141 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e4c7cfea612a9472 server_cookie=d553adb77c1888da in_seq=0 out_seq=0 2024-03-21T10:30:18.070 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:18.067+0000 7fa89aa00640 10 client.37031 ms_handle_connect on v2:172.21.15.100:6838/4173168440 2024-03-21T10:30:18.070 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:18.068+0000 7fa89aa00640 1 -- 172.21.15.47:0/3943153054 <== mds.0 v2:172.21.15.100:6838/4173168440 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55ec3c2f63f0 con 0x55ec3c39d5b0 2024-03-21T10:30:18.070 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:18.068+0000 7fa89aa00640 10 client.37031 handle_command_reply: tid=0 2024-03-21T10:30:18.070 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:18.068+0000 7fa8be77e740 1 client.37031 shutdown 2024-03-21T10:30:18.071 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:18.069+0000 7fa8be77e740 1 -- 172.21.15.47:0/3943153054 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ec3c3849a0 msgr2=0x55ec3c38a250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:30:18.071 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:18.069+0000 7fa8be77e740 1 --2- 172.21.15.47:0/3943153054 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ec3c3849a0 0x55ec3c38a250 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fa8a00ba180 tx=0x7fa8a00bd690 comp rx=0 tx=0).stop 2024-03-21T10:30:18.071 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:18.069+0000 7fa8be77e740 1 -- 172.21.15.47:0/3943153054 shutdown_connections 2024-03-21T10:30:18.071 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:18.069+0000 7fa8be77e740 1 --2- 172.21.15.47:0/3943153054 >> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] conn(0x55ec3c39d5b0 0x55ec3c39f9f0 crc :-1 s=READY pgs=141 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:30:18.071 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:18.069+0000 7fa8be77e740 1 --2- 172.21.15.47:0/3943153054 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ec3c38d580 0x55ec3c38a790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:30:18.071 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:18.069+0000 7fa8be77e740 1 --2- 172.21.15.47:0/3943153054 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ec3c3849a0 0x55ec3c38a250 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:30:18.071 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:18.069+0000 7fa89aa00640 0 client.37031 ms_handle_reset on v2:172.21.15.100:6838/4173168440 2024-03-21T10:30:18.071 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:18.069+0000 7fa8be77e740 1 --2- 172.21.15.47:0/3943153054 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ec3c384080 0x55ec3c38cec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:30:18.071 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:18.069+0000 7fa8be77e740 1 -- 172.21.15.47:0/3943153054 >> 172.21.15.47:0/3943153054 conn(0x55ec3c37e7b0 msgr2=0x55ec3c380500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:30:18.071 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:18.069+0000 7fa8be77e740 1 -- 172.21.15.47:0/3943153054 shutdown_connections 2024-03-21T10:30:18.071 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:18.069+0000 7fa8be77e740 1 -- 172.21.15.47:0/3943153054 wait complete. 2024-03-21T10:30:18.071 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:18.069+0000 7fa8be77e740 20 client.37031 trim_cache size 0 max 16384 2024-03-21T10:30:18.104 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:18.104+0000 7fa8be77e740 1 Processor -- start 2024-03-21T10:30:18.104 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:18.104+0000 7fa8be77e740 1 -- start start 2024-03-21T10:30:18.106 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:18.105+0000 7fa8be77e740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ec3c384080 0x55ec3c38c980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:30:18.106 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:18.105+0000 7fa8be77e740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ec3c3849a0 0x55ec3c38cec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:30:18.106 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:18.105+0000 7fa8be77e740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ec3c38d580 0x55ec3c37d6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:30:18.106 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:18.105+0000 7fa8be77e740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8b41b0a70 con 0x55ec3c384080 2024-03-21T10:30:18.106 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:18.105+0000 7fa8be77e740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8b41b23f0 con 0x55ec3c3849a0 2024-03-21T10:30:18.106 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:18.105+0000 7fa8be77e740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8b41b10d0 con 0x55ec3c38d580 2024-03-21T10:30:18.106 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:18.105+0000 7fa8b9000640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ec3c384080 0x55ec3c38c980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:30:18.107 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:18.105+0000 7fa8b9000640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ec3c384080 0x55ec3c38c980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:38472/0 (socket says 172.21.15.47:38472) 2024-03-21T10:30:18.107 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:18.105+0000 7fa8b9000640 1 -- 172.21.15.47:0/1167302232 learned_addr learned my addr 172.21.15.47:0/1167302232 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:30:18.107 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:18.105+0000 7fa8b3e00640 1 --2- 172.21.15.47:0/1167302232 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ec3c3849a0 0x55ec3c38cec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:30:18.107 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:18.106+0000 7fa8b9a00640 1 --2- 172.21.15.47:0/1167302232 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ec3c38d580 0x55ec3c37d6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:30:18.107 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:18.106+0000 7fa8b9000640 1 -- 172.21.15.47:0/1167302232 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ec3c38d580 msgr2=0x55ec3c37d6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:30:18.107 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:18.106+0000 7fa8b9000640 1 --2- 172.21.15.47:0/1167302232 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ec3c38d580 0x55ec3c37d6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:30:18.107 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:18.106+0000 7fa8b9000640 1 -- 172.21.15.47:0/1167302232 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ec3c3849a0 msgr2=0x55ec3c38cec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:30:18.107 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:18.106+0000 7fa8b9000640 1 --2- 172.21.15.47:0/1167302232 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ec3c3849a0 0x55ec3c38cec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:30:18.107 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:18.106+0000 7fa8b9000640 1 -- 172.21.15.47:0/1167302232 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ec3c379c20 con 0x55ec3c384080 2024-03-21T10:30:18.107 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:18.106+0000 7fa8b9a00640 1 --2- 172.21.15.47:0/1167302232 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ec3c38d580 0x55ec3c37d6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T10:30:18.108 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:18.106+0000 7fa8b3e00640 1 --2- 172.21.15.47:0/1167302232 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ec3c3849a0 0x55ec3c38cec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T10:30:18.108 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:18.106+0000 7fa8b9000640 1 --2- 172.21.15.47:0/1167302232 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ec3c384080 0x55ec3c38c980 secure :-1 s=READY pgs=879 cs=0 l=1 rev1=1 crypto rx=0x7fa8a800bcc0 tx=0x7fa8a800b3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e6e15bf90f38bff0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:30:18.111 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:18.106+0000 7fa8a5600640 1 -- 172.21.15.47:0/1167302232 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8a8015040 con 0x55ec3c384080 2024-03-21T10:30:18.111 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:18.107+0000 7fa8a5600640 1 -- 172.21.15.47:0/1167302232 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fa8a8026030 con 0x55ec3c384080 2024-03-21T10:30:18.111 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:18.107+0000 7fa8a5600640 1 -- 172.21.15.47:0/1167302232 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8a80119c0 con 0x55ec3c384080 2024-03-21T10:30:18.111 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:18.107+0000 7fa8be77e740 1 -- 172.21.15.47:0/1167302232 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ec3c384080 msgr2=0x55ec3c38c980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:30:18.111 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:18.107+0000 7fa8be77e740 1 --2- 172.21.15.47:0/1167302232 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ec3c384080 0x55ec3c38c980 secure :-1 s=READY pgs=879 cs=0 l=1 rev1=1 crypto rx=0x7fa8a800bcc0 tx=0x7fa8a800b3c0 comp rx=0 tx=0).stop 2024-03-21T10:30:18.111 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:18.107+0000 7fa8be77e740 1 -- 172.21.15.47:0/1167302232 shutdown_connections 2024-03-21T10:30:18.111 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:18.107+0000 7fa8be77e740 1 --2- 172.21.15.47:0/1167302232 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ec3c38d580 0x55ec3c37d6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:30:18.112 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:18.107+0000 7fa8be77e740 1 --2- 172.21.15.47:0/1167302232 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ec3c3849a0 0x55ec3c38cec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:30:18.112 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:18.107+0000 7fa8be77e740 1 --2- 172.21.15.47:0/1167302232 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ec3c384080 0x55ec3c38c980 unknown :-1 s=CLOSED pgs=879 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:30:18.112 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:18.107+0000 7fa8be77e740 1 -- 172.21.15.47:0/1167302232 >> 172.21.15.47:0/1167302232 conn(0x55ec3c37e7b0 msgr2=0x55ec3c37cfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:30:18.112 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:18.107+0000 7fa8be77e740 1 -- 172.21.15.47:0/1167302232 shutdown_connections 2024-03-21T10:30:18.112 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:18.107+0000 7fa8be77e740 1 -- 172.21.15.47:0/1167302232 wait complete. 2024-03-21T10:30:18.112 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:18.108+0000 7fa8be77e740 1 -- start start 2024-03-21T10:30:18.112 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:18.108+0000 7fa8be77e740 1 Processor -- start 2024-03-21T10:30:18.112 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:18.108+0000 7fa8be77e740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ec3c384080 0x55ec3c392770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:30:18.112 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:18.108+0000 7fa8be77e740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ec3c3849a0 0x55ec3c392cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:30:18.112 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:18.108+0000 7fa8be77e740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ec3c38d580 0x55ec3c3f8ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:30:18.113 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:18.108+0000 7fa8be77e740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8b41b1d90 con 0x55ec3c3849a0 2024-03-21T10:30:18.113 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:18.109+0000 7fa8be77e740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8b41b1730 con 0x55ec3c384080 2024-03-21T10:30:18.113 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:18.109+0000 7fa8be77e740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8b41aec30 con 0x55ec3c38d580 2024-03-21T10:30:18.113 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:18.109+0000 7fa8be77e740 10 client.0 _ll_register_callbacks cb 0x7fa8a00035f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T10:30:18.113 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:18.109+0000 7fa8b3e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ec3c3849a0 0x55ec3c392cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:30:18.113 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:18.109+0000 7fa8b3e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ec3c3849a0 0x55ec3c392cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:38488/0 (socket says 172.21.15.47:38488) 2024-03-21T10:30:18.113 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:18.109+0000 7fa8b3e00640 1 -- 172.21.15.47:0/1761160307 learned_addr learned my addr 172.21.15.47:0/1761160307 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:30:18.113 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:18.109+0000 7fa8b9000640 1 --2- 172.21.15.47:0/1761160307 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ec3c384080 0x55ec3c392770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:30:18.113 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:18.109+0000 7fa8b9a00640 1 --2- 172.21.15.47:0/1761160307 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ec3c38d580 0x55ec3c3f8ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:30:18.113 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:18.109+0000 7fa8b3e00640 1 -- 172.21.15.47:0/1761160307 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ec3c38d580 msgr2=0x55ec3c3f8ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:30:18.113 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:18.109+0000 7fa8b3e00640 1 --2- 172.21.15.47:0/1761160307 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ec3c38d580 0x55ec3c3f8ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:30:18.113 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:18.109+0000 7fa8b3e00640 1 -- 172.21.15.47:0/1761160307 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ec3c384080 msgr2=0x55ec3c392770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:30:18.113 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:18.109+0000 7fa8b3e00640 1 --2- 172.21.15.47:0/1761160307 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ec3c384080 0x55ec3c392770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:30:18.113 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:18.109+0000 7fa8b3e00640 1 -- 172.21.15.47:0/1761160307 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55ec3c393570 con 0x55ec3c3849a0 2024-03-21T10:30:18.114 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:18.110+0000 7fa8b3e00640 1 --2- 172.21.15.47:0/1761160307 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ec3c3849a0 0x55ec3c392cb0 secure :-1 s=READY pgs=880 cs=0 l=1 rev1=1 crypto rx=0x7fa8a00036a0 tx=0x7fa8a00bd5f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=65a3127fd39c47da server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:30:18.114 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:18.110+0000 7fa89aa00640 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-03-21T10:30:18.114 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:18.110+0000 7fa89aa00640 1 -- 172.21.15.47:0/1761160307 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8a00d5040 con 0x55ec3c3849a0 2024-03-21T10:30:18.114 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:18.110+0000 7fa89aa00640 1 -- 172.21.15.47:0/1761160307 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(97..97 src has 1..97) v4 ==== 9474+0+0 (secure 0 0 0) 0x7fa8a00eec80 con 0x55ec3c3849a0 2024-03-21T10:30:18.114 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:18.110+0000 7fa8be77e740 1 -- 172.21.15.47:0/1761160307 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fa8a8026030 con 0x55ec3c3849a0 2024-03-21T10:30:18.114 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:18.110+0000 7fa89aa00640 1 -- 172.21.15.47:0/1761160307 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa8a00b9a50 con 0x55ec3c3849a0 2024-03-21T10:30:18.114 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:18.110+0000 7fa8be77e740 10 client.17250 fetch_fsmap learned FSMap version 32 2024-03-21T10:30:18.114 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:18.110+0000 7fa8be77e740 1 -- 172.21.15.47:0/1761160307 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x55ec3c3fa4c0 con 0x55ec3c3849a0 2024-03-21T10:30:18.114 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:18.110+0000 7fa89aa00640 1 -- 172.21.15.47:0/1761160307 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fa8a8026030 con 0x55ec3c3849a0 2024-03-21T10:30:18.114 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:18.111+0000 7fa89aa00640 1 -- 172.21.15.47:0/1761160307 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 32) v1 ==== 7942+0+0 (secure 0 0 0) 0x7fa8a00c5d30 con 0x55ec3c3849a0 2024-03-21T10:30:18.114 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:18.111+0000 7fa8be77e740 10 client.17250 fetch_fsmap finished waiting for FSMap version 32 2024-03-21T10:30:18.114 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:18.111+0000 7fa8be77e740 10 client.17250 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-21T10:30:18.114 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:18.111+0000 7fa8be77e740 20 client.17250 populate_metadata read hostname 'smithi047' 2024-03-21T10:30:18.115 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:18.111+0000 7fa8be77e740 1 --2- 172.21.15.47:0/1761160307 >> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] conn(0x55ec3c3fa9c0 0x55ec3c3fcde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:30:18.115 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:18.111+0000 7fa8be77e740 4 client.17250 mds_command: new command op to 34535 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-03-21T10:30:18.115 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:18.111+0000 7fa8be77e740 1 -- 172.21.15.47:0/1761160307 --> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x55ec3c2f0740 con 0x55ec3c3fa9c0 2024-03-21T10:30:18.115 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:18.111+0000 7fa8b9000640 1 --2- 172.21.15.47:0/1761160307 >> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] conn(0x55ec3c3fa9c0 0x55ec3c3fcde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:30:18.116 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:18.112+0000 7fa8b9000640 1 --2- 172.21.15.47:0/1761160307 >> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] conn(0x55ec3c3fa9c0 0x55ec3c3fcde0 crc :-1 s=READY pgs=142 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=38eb41235656bc72 server_cookie=cf994ac4f395d161 in_seq=0 out_seq=0 2024-03-21T10:30:18.116 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:18.112+0000 7fa89aa00640 10 client.17250 ms_handle_connect on v2:172.21.15.100:6838/4173168440 2024-03-21T10:30:18.116 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:18.114+0000 7fa89aa00640 1 -- 172.21.15.47:0/1761160307 <== mds.0 v2:172.21.15.100:6838/4173168440 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x55ec3c2f0740 con 0x55ec3c3fa9c0 2024-03-21T10:30:18.116 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:18.114+0000 7fa89aa00640 10 client.17250 handle_command_reply: tid=0 2024-03-21T10:30:18.116 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:18.114+0000 7fa8be77e740 1 client.17250 shutdown 2024-03-21T10:30:18.117 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:18.114+0000 7fa8be77e740 1 -- 172.21.15.47:0/1761160307 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ec3c3849a0 msgr2=0x55ec3c392cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:30:18.117 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:18.114+0000 7fa8be77e740 1 --2- 172.21.15.47:0/1761160307 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ec3c3849a0 0x55ec3c392cb0 secure :-1 s=READY pgs=880 cs=0 l=1 rev1=1 crypto rx=0x7fa8a00036a0 tx=0x7fa8a00bd5f0 comp rx=0 tx=0).stop 2024-03-21T10:30:18.117 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:18.114+0000 7fa8be77e740 1 -- 172.21.15.47:0/1761160307 shutdown_connections 2024-03-21T10:30:18.117 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:18.114+0000 7fa8be77e740 1 --2- 172.21.15.47:0/1761160307 >> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] conn(0x55ec3c3fa9c0 0x55ec3c3fcde0 crc :-1 s=READY pgs=142 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:30:18.117 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:18.114+0000 7fa8be77e740 1 --2- 172.21.15.47:0/1761160307 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ec3c38d580 0x55ec3c3f8ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:30:18.117 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:18.114+0000 7fa8be77e740 1 --2- 172.21.15.47:0/1761160307 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ec3c3849a0 0x55ec3c392cb0 unknown :-1 s=CLOSED pgs=880 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:30:18.117 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:18.114+0000 7fa8be77e740 1 --2- 172.21.15.47:0/1761160307 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ec3c384080 0x55ec3c392770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:30:18.117 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:18.114+0000 7fa8be77e740 1 -- 172.21.15.47:0/1761160307 >> 172.21.15.47:0/1761160307 conn(0x55ec3c37e7b0 msgr2=0x55ec3c37d830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:30:18.117 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:18.114+0000 7fa89aa00640 0 client.17250 ms_handle_reset on v2:172.21.15.100:6838/4173168440 2024-03-21T10:30:18.117 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:18.115+0000 7fa8be77e740 1 -- 172.21.15.47:0/1761160307 shutdown_connections 2024-03-21T10:30:18.117 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:18.115+0000 7fa8be77e740 1 -- 172.21.15.47:0/1761160307 wait complete. 2024-03-21T10:30:18.118 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:18.115+0000 7fa8be77e740 20 client.17250 trim_cache size 0 max 16384 2024-03-21T10:30:18.118 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-03-21T10:30:18.118 INFO:teuthology.orchestra.run.smithi047.stdout: "return_code": 0, 2024-03-21T10:30:18.119 INFO:teuthology.orchestra.run.smithi047.stdout: "scrub_tag": "72f8e749-8322-4803-8156-ec32d7aefe22", 2024-03-21T10:30:18.119 INFO:teuthology.orchestra.run.smithi047.stdout: "mode": "asynchronous" 2024-03-21T10:30:18.119 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-03-21T10:30:18.119 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:18.115+0000 7fa8bba00640 1 -- 172.21.15.47:0/1781034116 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7fa89c0a3090 msgr2=0x7fa89c0a5550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:30:18.120 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:18.115+0000 7fa8bba00640 1 --2- 172.21.15.47:0/1781034116 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7fa89c0a3090 0x7fa89c0a5550 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fa8ac0049d0 tx=0x7fa8ac009500 comp rx=0 tx=0).stop 2024-03-21T10:30:18.120 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:18.115+0000 7fa8bba00640 1 -- 172.21.15.47:0/1781034116 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa8b4104200 msgr2=0x7fa8b419a5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:30:18.120 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:18.115+0000 7fa8bba00640 1 --2- 172.21.15.47:0/1781034116 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa8b4104200 0x7fa8b419a5c0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fa8a0036040 tx=0x7fa8a002cd30 comp rx=0 tx=0).stop 2024-03-21T10:30:18.120 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:18.115+0000 7fa8bba00640 1 -- 172.21.15.47:0/1781034116 shutdown_connections 2024-03-21T10:30:18.120 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:18.115+0000 7fa8bba00640 1 --2- 172.21.15.47:0/1781034116 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa8b4109850 0x7fa8b419e940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:30:18.120 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:18.115+0000 7fa8bba00640 1 --2- 172.21.15.47:0/1781034116 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7fa89c0a3090 0x7fa89c0a5550 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:30:18.120 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:18.115+0000 7fa8bba00640 1 --2- 172.21.15.47:0/1781034116 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa8b4104200 0x7fa8b419a5c0 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:30:18.120 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:18.115+0000 7fa8bba00640 1 --2- 172.21.15.47:0/1781034116 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa8b4103840 0x7fa8b419a080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:30:18.120 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:18.115+0000 7fa8bba00640 1 -- 172.21.15.47:0/1781034116 >> 172.21.15.47:0/1781034116 conn(0x7fa8b40ff2f0 msgr2=0x7fa8b40ffc90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:30:18.120 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:18.116+0000 7fa8bba00640 1 -- 172.21.15.47:0/1781034116 shutdown_connections 2024-03-21T10:30:18.120 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:18.116+0000 7fa8bba00640 1 -- 172.21.15.47:0/1781034116 wait complete. 2024-03-21T10:30:18.133 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-21T10:30:18.134 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-21T10:30:18.379 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:18.379+0000 7f5c16c00640 1 -- 172.21.15.47:0/550005974 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5c101058c0 msgr2=0x7f5c10105d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:30:18.379 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:18.379+0000 7f5c16c00640 1 --2- 172.21.15.47:0/550005974 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5c101058c0 0x7f5c10105d50 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f5c08009be0 tx=0x7f5c0802bf80 comp rx=0 tx=0).stop 2024-03-21T10:30:18.379 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:18.379+0000 7f5c16c00640 1 -- 172.21.15.47:0/550005974 shutdown_connections 2024-03-21T10:30:18.379 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:18.379+0000 7f5c16c00640 1 --2- 172.21.15.47:0/550005974 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5c101058c0 0x7f5c10105d50 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:30:18.379 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:18.379+0000 7f5c16c00640 1 --2- 172.21.15.47:0/550005974 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5c10100de0 0x7f5c101052a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:30:18.379 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:18.379+0000 7f5c16c00640 1 --2- 172.21.15.47:0/550005974 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5c101004c0 0x7f5c101008a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:30:18.379 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:18.379+0000 7f5c16c00640 1 -- 172.21.15.47:0/550005974 >> 172.21.15.47:0/550005974 conn(0x7f5c100fbef0 msgr2=0x7f5c100fe310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:30:18.380 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:18.379+0000 7f5c16c00640 1 -- 172.21.15.47:0/550005974 shutdown_connections 2024-03-21T10:30:18.380 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:18.379+0000 7f5c16c00640 1 -- 172.21.15.47:0/550005974 wait complete. 2024-03-21T10:30:18.381 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:18.381+0000 7f5c16c00640 1 Processor -- start 2024-03-21T10:30:18.381 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:18.381+0000 7f5c16c00640 1 -- start start 2024-03-21T10:30:18.381 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:18.381+0000 7f5c16c00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5c101004c0 0x7f5c1019e270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:30:18.381 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:18.381+0000 7f5c16c00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5c10100de0 0x7f5c1019e7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:30:18.381 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:18.381+0000 7f5c16c00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5c101058c0 0x7f5c101a2b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:30:18.382 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:18.382+0000 7f5c16c00640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c10117cf0 con 0x7f5c101004c0 2024-03-21T10:30:18.382 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:18.382+0000 7f5c16c00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c10117b70 con 0x7f5c10100de0 2024-03-21T10:30:18.382 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:18.382+0000 7f5c16c00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c10117e70 con 0x7f5c101058c0 2024-03-21T10:30:18.382 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:18.382+0000 7f5c15800640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5c101004c0 0x7f5c1019e270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:30:18.382 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:18.382+0000 7f5c14e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5c10100de0 0x7f5c1019e7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:30:18.382 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:18.382+0000 7f5c15800640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5c101004c0 0x7f5c1019e270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:38502/0 (socket says 172.21.15.47:38502) 2024-03-21T10:30:18.382 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:18.382+0000 7f5c15800640 1 -- 172.21.15.47:0/2800061000 learned_addr learned my addr 172.21.15.47:0/2800061000 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:30:18.382 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:18.382+0000 7f5c16200640 1 --2- 172.21.15.47:0/2800061000 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5c101058c0 0x7f5c101a2b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:30:18.382 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:18.382+0000 7f5c15800640 1 -- 172.21.15.47:0/2800061000 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5c101058c0 msgr2=0x7f5c101a2b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:30:18.382 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:18.382+0000 7f5c15800640 1 --2- 172.21.15.47:0/2800061000 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5c101058c0 0x7f5c101a2b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:30:18.382 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:18.382+0000 7f5c15800640 1 -- 172.21.15.47:0/2800061000 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5c10100de0 msgr2=0x7f5c1019e7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:30:18.382 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:18.382+0000 7f5c15800640 1 --2- 172.21.15.47:0/2800061000 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5c10100de0 0x7f5c1019e7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:30:18.382 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:18.382+0000 7f5c15800640 1 -- 172.21.15.47:0/2800061000 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c101a3260 con 0x7f5c101004c0 2024-03-21T10:30:18.383 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:18.383+0000 7f5c15800640 1 --2- 172.21.15.47:0/2800061000 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5c101004c0 0x7f5c1019e270 secure :-1 s=READY pgs=881 cs=0 l=1 rev1=1 crypto rx=0x7f5c0400d770 tx=0x7f5c0400dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:30:18.383 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:18.383+0000 7f5c0e000640 1 -- 172.21.15.47:0/2800061000 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c04017070 con 0x7f5c101004c0 2024-03-21T10:30:18.383 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:18.383+0000 7f5c16c00640 1 -- 172.21.15.47:0/2800061000 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5c101a3550 con 0x7f5c101004c0 2024-03-21T10:30:18.383 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:18.383+0000 7f5c0e000640 1 -- 172.21.15.47:0/2800061000 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f5c040054d0 con 0x7f5c101004c0 2024-03-21T10:30:18.384 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:18.383+0000 7f5c16c00640 1 -- 172.21.15.47:0/2800061000 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5c101aade0 con 0x7f5c101004c0 2024-03-21T10:30:18.384 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:18.383+0000 7f5c0e000640 1 -- 172.21.15.47:0/2800061000 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c04020060 con 0x7f5c101004c0 2024-03-21T10:30:18.385 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:18.387+0000 7f5c1ae57740 1 Processor -- start 2024-03-21T10:30:18.386 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:18.387+0000 7f5c1ae57740 1 -- start start 2024-03-21T10:30:18.387 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:18.387+0000 7f5c1ae57740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b5232e8dd0 0x55b5232e7b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:30:18.387 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:18.387+0000 7f5c1ae57740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b5232e80f0 0x55b5232e8570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:30:18.387 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:18.387+0000 7f5c1ae57740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b5232f1460 0x55b5232f3850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:30:18.387 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:18.387+0000 7f5c1ae57740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b52324c250 con 0x55b5232f1460 2024-03-21T10:30:18.387 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:18.387+0000 7f5c1ae57740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b5231f4a30 con 0x55b5232e80f0 2024-03-21T10:30:18.387 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:18.387+0000 7f5c1ae57740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b5232482f0 con 0x55b5232e8dd0 2024-03-21T10:30:18.387 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:18.387+0000 7f5c14e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b5232f1460 0x55b5232f3850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:30:18.387 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:18.388+0000 7f5c14e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b5232f1460 0x55b5232f3850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:38518/0 (socket says 172.21.15.47:38518) 2024-03-21T10:30:18.388 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:18.388+0000 7f5c14e00640 1 -- 172.21.15.47:0/1881915124 learned_addr learned my addr 172.21.15.47:0/1881915124 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:30:18.388 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:18.388+0000 7f5c16200640 1 --2- 172.21.15.47:0/1881915124 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b5232e8dd0 0x55b5232e7b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:30:18.388 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:18.388+0000 7f5c14e00640 1 -- 172.21.15.47:0/1881915124 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b5232e8dd0 msgr2=0x55b5232e7b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:30:18.388 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:18.388+0000 7f5c14e00640 1 --2- 172.21.15.47:0/1881915124 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b5232e8dd0 0x55b5232e7b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:30:18.388 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:18.388+0000 7f5c14e00640 1 -- 172.21.15.47:0/1881915124 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b5232e80f0 msgr2=0x55b5232e8570 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-21T10:30:18.388 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:18.388+0000 7f5c14e00640 1 --2- 172.21.15.47:0/1881915124 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b5232e80f0 0x55b5232e8570 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:30:18.388 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:18.388+0000 7f5c14e00640 1 -- 172.21.15.47:0/1881915124 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b5231df690 con 0x55b5232f1460 2024-03-21T10:30:18.388 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:18.388+0000 7f5c0e000640 1 -- 172.21.15.47:0/2800061000 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 27) v1 ==== 137985+0+0 (secure 0 0 0) 0x7f5c04004c90 con 0x7f5c101004c0 2024-03-21T10:30:18.392 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:18.389+0000 7f5c14e00640 1 --2- 172.21.15.47:0/1881915124 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b5232f1460 0x55b5232f3850 secure :-1 s=READY pgs=882 cs=0 l=1 rev1=1 crypto rx=0x7f5bfc009c10 tx=0x7f5bfc011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3eab5aa8cced736f server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:30:18.392 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:18.389+0000 7f5c02a00640 1 -- 172.21.15.47:0/1881915124 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5bfc004030 con 0x55b5232f1460 2024-03-21T10:30:18.392 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:18.389+0000 7f5c02a00640 1 -- 172.21.15.47:0/1881915124 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f5bfc0035f0 con 0x55b5232f1460 2024-03-21T10:30:18.392 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:18.389+0000 7f5c02a00640 1 -- 172.21.15.47:0/1881915124 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5bfc0196d0 con 0x55b5232f1460 2024-03-21T10:30:18.392 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:18.389+0000 7f5c1ae57740 1 -- 172.21.15.47:0/1881915124 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b5232f1460 msgr2=0x55b5232f3850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:30:18.392 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:18.389+0000 7f5c1ae57740 1 --2- 172.21.15.47:0/1881915124 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b5232f1460 0x55b5232f3850 secure :-1 s=READY pgs=882 cs=0 l=1 rev1=1 crypto rx=0x7f5bfc009c10 tx=0x7f5bfc011810 comp rx=0 tx=0).stop 2024-03-21T10:30:18.392 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:18.390+0000 7f5c1ae57740 1 -- 172.21.15.47:0/1881915124 shutdown_connections 2024-03-21T10:30:18.392 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:18.390+0000 7f5c1ae57740 1 --2- 172.21.15.47:0/1881915124 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b5232f1460 0x55b5232f3850 unknown :-1 s=CLOSED pgs=882 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:30:18.392 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:18.390+0000 7f5c1ae57740 1 --2- 172.21.15.47:0/1881915124 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b5232e80f0 0x55b5232e8570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:30:18.392 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:18.390+0000 7f5c1ae57740 1 --2- 172.21.15.47:0/1881915124 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b5232e8dd0 0x55b5232e7b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:30:18.392 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:18.390+0000 7f5c1ae57740 1 -- 172.21.15.47:0/1881915124 >> 172.21.15.47:0/1881915124 conn(0x55b5232af200 msgr2=0x55b5232e2b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:30:18.392 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:18.390+0000 7f5c1ae57740 1 -- 172.21.15.47:0/1881915124 shutdown_connections 2024-03-21T10:30:18.393 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:18.390+0000 7f5c1ae57740 1 -- 172.21.15.47:0/1881915124 wait complete. 2024-03-21T10:30:18.393 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:18.390+0000 7f5c1ae57740 1 -- start start 2024-03-21T10:30:18.393 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:18.390+0000 7f5c1ae57740 1 Processor -- start 2024-03-21T10:30:18.393 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:18.391+0000 7f5c0e000640 1 --2- 172.21.15.47:0/2800061000 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f5bf80a2fa0 0x7f5bf80a5460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:30:18.393 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:18.391+0000 7f5c0e000640 1 -- 172.21.15.47:0/2800061000 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 9474+0+0 (secure 0 0 0) 0x7f5c040cf6c0 con 0x7f5c101004c0 2024-03-21T10:30:18.393 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:18.391+0000 7f5c1ae57740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b5232e80f0 0x55b5232f0b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:30:18.393 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:18.391+0000 7f5c16200640 1 --2- 172.21.15.47:0/2800061000 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f5bf80a2fa0 0x7f5bf80a5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:30:18.393 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:18.391+0000 7f5c1ae57740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b5232e8dd0 0x55b5232edeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:30:18.393 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:18.391+0000 7f5c1ae57740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b5232f1460 0x55b5232ee3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:30:18.393 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:18.391+0000 7f5c1ae57740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b5232f42d0 con 0x55b5232e80f0 2024-03-21T10:30:18.393 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:18.391+0000 7f5c1ae57740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b523207e80 con 0x55b5232e8dd0 2024-03-21T10:30:18.393 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:18.391+0000 7f5c1ae57740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b5232f4450 con 0x55b5232f1460 2024-03-21T10:30:18.393 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:18.391+0000 7f5c1ae57740 10 client.0 _ll_register_callbacks cb 0x55b52321a130 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T10:30:18.393 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:18.391+0000 7f5c15800640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b5232e80f0 0x55b5232f0b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:30:18.393 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:18.391+0000 7f5c15800640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b5232e80f0 0x55b5232f0b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:38526/0 (socket says 172.21.15.47:38526) 2024-03-21T10:30:18.394 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:18.391+0000 7f5c15800640 1 -- 172.21.15.47:0/2892467076 learned_addr learned my addr 172.21.15.47:0/2892467076 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:30:18.394 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:18.392+0000 7f5c14e00640 1 --2- 172.21.15.47:0/2892467076 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b5232e8dd0 0x55b5232edeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:30:18.394 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:18.392+0000 7f5c15800640 1 -- 172.21.15.47:0/2892467076 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b5232f1460 msgr2=0x55b5232ee3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:30:18.394 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:18.392+0000 7f5c15800640 1 --2- 172.21.15.47:0/2892467076 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b5232f1460 0x55b5232ee3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:30:18.394 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:18.392+0000 7f5c15800640 1 -- 172.21.15.47:0/2892467076 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b5232e8dd0 msgr2=0x55b5232edeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:30:18.394 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:18.392+0000 7f5c15800640 1 --2- 172.21.15.47:0/2892467076 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b5232e8dd0 0x55b5232edeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:30:18.394 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:18.392+0000 7f5c15800640 1 -- 172.21.15.47:0/2892467076 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55b52302a7d0 con 0x55b5232e80f0 2024-03-21T10:30:18.394 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:18.392+0000 7f5c14e00640 1 --2- 172.21.15.47:0/2892467076 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b5232e8dd0 0x55b5232edeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T10:30:18.394 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:18.392+0000 7f5c16200640 1 --2- 172.21.15.47:0/2800061000 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f5bf80a2fa0 0x7f5bf80a5460 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f5c080099a0 tx=0x7f5c08033c90 comp rx=0 tx=0).ready entity=mgr.15687 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:30:18.394 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:18.392+0000 7f5c15800640 1 --2- 172.21.15.47:0/2892467076 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b5232e80f0 0x55b5232f0b20 secure :-1 s=READY pgs=883 cs=0 l=1 rev1=1 crypto rx=0x7f5c0400d740 tx=0x7f5c0409b290 comp rx=0 tx=0).ready entity=mon.0 client_cookie=290898e542ba0e95 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:30:18.394 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:18.392+0000 7f5bf7400640 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-03-21T10:30:18.394 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:18.392+0000 7f5bf7400640 1 -- 172.21.15.47:0/2892467076 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c040b4060 con 0x55b5232e80f0 2024-03-21T10:30:18.394 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:18.393+0000 7f5bf7400640 1 -- 172.21.15.47:0/2892467076 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(97..97 src has 1..97) v4 ==== 9474+0+0 (secure 0 0 0) 0x7f5c040cf260 con 0x55b5232e80f0 2024-03-21T10:30:18.394 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:18.393+0000 7f5c1ae57740 1 -- 172.21.15.47:0/2892467076 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f5bfc0035f0 con 0x55b5232e80f0 2024-03-21T10:30:18.395 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:18.393+0000 7f5bf7400640 1 -- 172.21.15.47:0/2892467076 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5c04014070 con 0x55b5232e80f0 2024-03-21T10:30:18.395 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:18.393+0000 7f5c1ae57740 10 client.17262 fetch_fsmap learned FSMap version 32 2024-03-21T10:30:18.395 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:18.393+0000 7f5c1ae57740 1 -- 172.21.15.47:0/2892467076 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x55b522f603f0 con 0x55b5232e80f0 2024-03-21T10:30:18.397 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:18.393+0000 7f5bf7400640 1 -- 172.21.15.47:0/2892467076 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f5bfc0035f0 con 0x55b5232e80f0 2024-03-21T10:30:18.397 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:18.393+0000 7f5bf7400640 1 -- 172.21.15.47:0/2892467076 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 32) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f5c040afce0 con 0x55b5232e80f0 2024-03-21T10:30:18.397 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:18.393+0000 7f5c1ae57740 10 client.17262 fetch_fsmap finished waiting for FSMap version 32 2024-03-21T10:30:18.397 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:18.393+0000 7f5c1ae57740 10 client.17262 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-21T10:30:18.397 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:18.394+0000 7f5c1ae57740 20 client.17262 populate_metadata read hostname 'smithi047' 2024-03-21T10:30:18.397 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:18.394+0000 7f5c1ae57740 1 --2- 172.21.15.47:0/2892467076 >> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] conn(0x55b523301290 0x55b523303680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:30:18.397 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:18.394+0000 7f5c1ae57740 4 client.17262 mds_command: new command op to 34535 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T10:30:18.397 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:18.394+0000 7f5c1ae57740 1 -- 172.21.15.47:0/2892467076 --> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55b523259790 con 0x55b523301290 2024-03-21T10:30:18.397 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:18.394+0000 7f5c14e00640 1 --2- 172.21.15.47:0/2892467076 >> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] conn(0x55b523301290 0x55b523303680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:30:18.397 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:18.395+0000 7f5c14e00640 1 --2- 172.21.15.47:0/2892467076 >> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] conn(0x55b523301290 0x55b523303680 crc :-1 s=READY pgs=143 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3f0a83b41a7033a4 server_cookie=ca72facadb95bfd5 in_seq=0 out_seq=0 2024-03-21T10:30:18.397 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:18.395+0000 7f5bf7400640 10 client.17262 ms_handle_connect on v2:172.21.15.100:6838/4173168440 2024-03-21T10:30:18.397 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:18.396+0000 7f5bf7400640 1 -- 172.21.15.47:0/2892467076 <== mds.0 v2:172.21.15.100:6838/4173168440 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55b523259790 con 0x55b523301290 2024-03-21T10:30:18.398 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:18.396+0000 7f5bf7400640 10 client.17262 handle_command_reply: tid=0 2024-03-21T10:30:18.398 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:18.396+0000 7f5c1ae57740 1 client.17262 shutdown 2024-03-21T10:30:18.398 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:18.396+0000 7f5c1ae57740 1 -- 172.21.15.47:0/2892467076 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b5232e80f0 msgr2=0x55b5232f0b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:30:18.398 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:18.396+0000 7f5c1ae57740 1 --2- 172.21.15.47:0/2892467076 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b5232e80f0 0x55b5232f0b20 secure :-1 s=READY pgs=883 cs=0 l=1 rev1=1 crypto rx=0x7f5c0400d740 tx=0x7f5c0409b290 comp rx=0 tx=0).stop 2024-03-21T10:30:18.398 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:18.396+0000 7f5c1ae57740 1 -- 172.21.15.47:0/2892467076 shutdown_connections 2024-03-21T10:30:18.398 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:18.396+0000 7f5c1ae57740 1 --2- 172.21.15.47:0/2892467076 >> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] conn(0x55b523301290 0x55b523303680 crc :-1 s=READY pgs=143 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:30:18.398 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:18.396+0000 7f5c1ae57740 1 --2- 172.21.15.47:0/2892467076 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b5232f1460 0x55b5232ee3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:30:18.398 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:18.396+0000 7f5bf7400640 0 client.17262 ms_handle_reset on v2:172.21.15.100:6838/4173168440 2024-03-21T10:30:18.398 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:18.396+0000 7f5c1ae57740 1 --2- 172.21.15.47:0/2892467076 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b5232e8dd0 0x55b5232edeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:30:18.398 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:18.396+0000 7f5c1ae57740 1 --2- 172.21.15.47:0/2892467076 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b5232e80f0 0x55b5232f0b20 unknown :-1 s=CLOSED pgs=883 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:30:18.398 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:18.396+0000 7f5c1ae57740 1 -- 172.21.15.47:0/2892467076 >> 172.21.15.47:0/2892467076 conn(0x55b5232af200 msgr2=0x55b5232e4890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:30:18.398 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:18.396+0000 7f5c1ae57740 1 -- 172.21.15.47:0/2892467076 shutdown_connections 2024-03-21T10:30:18.398 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:18.396+0000 7f5c1ae57740 1 -- 172.21.15.47:0/2892467076 wait complete. 2024-03-21T10:30:18.398 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:18.397+0000 7f5c1ae57740 20 client.17262 trim_cache size 0 max 16384 2024-03-21T10:30:18.414 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:18.414+0000 7f5c1ae57740 1 Processor -- start 2024-03-21T10:30:18.414 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:18.414+0000 7f5c1ae57740 1 -- start start 2024-03-21T10:30:18.415 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:18.415+0000 7f5c1ae57740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b5232e80f0 0x55b5232f05e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:30:18.415 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:18.415+0000 7f5c1ae57740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b5232e8dd0 0x55b5232f0b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:30:18.415 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:18.415+0000 7f5c1ae57740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b5232f1460 0x55b5232f2850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:30:18.415 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:18.415+0000 7f5c1ae57740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c101b4c00 con 0x55b5232e8dd0 2024-03-21T10:30:18.415 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:18.415+0000 7f5c1ae57740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c101b6580 con 0x55b5232f1460 2024-03-21T10:30:18.415 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:18.415+0000 7f5c1ae57740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c101b5260 con 0x55b5232e80f0 2024-03-21T10:30:18.415 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:18.415+0000 7f5c14e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b5232e8dd0 0x55b5232f0b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:30:18.415 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:18.415+0000 7f5c14e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b5232e8dd0 0x55b5232f0b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:38530/0 (socket says 172.21.15.47:38530) 2024-03-21T10:30:18.415 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:18.415+0000 7f5c14e00640 1 -- 172.21.15.47:0/3839428474 learned_addr learned my addr 172.21.15.47:0/3839428474 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:30:18.415 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:18.415+0000 7f5c16200640 1 --2- 172.21.15.47:0/3839428474 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b5232f1460 0x55b5232f2850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:30:18.415 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:18.415+0000 7f5c15800640 1 --2- 172.21.15.47:0/3839428474 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b5232e80f0 0x55b5232f05e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:30:18.415 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:18.415+0000 7f5c14e00640 1 -- 172.21.15.47:0/3839428474 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b5232e80f0 msgr2=0x55b5232f05e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:30:18.415 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:18.415+0000 7f5c14e00640 1 --2- 172.21.15.47:0/3839428474 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b5232e80f0 0x55b5232f05e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:30:18.415 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:18.415+0000 7f5c14e00640 1 -- 172.21.15.47:0/3839428474 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b5232f1460 msgr2=0x55b5232f2850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:30:18.416 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:18.415+0000 7f5c14e00640 1 --2- 172.21.15.47:0/3839428474 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b5232f1460 0x55b5232f2850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:30:18.416 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:18.415+0000 7f5c14e00640 1 -- 172.21.15.47:0/3839428474 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b522f200f0 con 0x55b5232e8dd0 2024-03-21T10:30:18.419 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:18.416+0000 7f5c14e00640 1 --2- 172.21.15.47:0/3839428474 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b5232e8dd0 0x55b5232f0b20 secure :-1 s=READY pgs=884 cs=0 l=1 rev1=1 crypto rx=0x7f5bfc0099d0 tx=0x7f5bfc003d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a0376dc9ad368a47 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:30:18.419 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:18.416+0000 7f5c02000640 1 -- 172.21.15.47:0/3839428474 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5bfc01f040 con 0x55b5232e8dd0 2024-03-21T10:30:18.419 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:18.416+0000 7f5c02000640 1 -- 172.21.15.47:0/3839428474 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f5bfc0035f0 con 0x55b5232e8dd0 2024-03-21T10:30:18.419 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:18.416+0000 7f5c02000640 1 -- 172.21.15.47:0/3839428474 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5bfc005040 con 0x55b5232e8dd0 2024-03-21T10:30:18.419 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:18.417+0000 7f5c1ae57740 1 -- 172.21.15.47:0/3839428474 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b5232e8dd0 msgr2=0x55b5232f0b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:30:18.419 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:18.417+0000 7f5c1ae57740 1 --2- 172.21.15.47:0/3839428474 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b5232e8dd0 0x55b5232f0b20 secure :-1 s=READY pgs=884 cs=0 l=1 rev1=1 crypto rx=0x7f5bfc0099d0 tx=0x7f5bfc003d00 comp rx=0 tx=0).stop 2024-03-21T10:30:18.420 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:18.417+0000 7f5c1ae57740 1 -- 172.21.15.47:0/3839428474 shutdown_connections 2024-03-21T10:30:18.420 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:18.417+0000 7f5c1ae57740 1 --2- 172.21.15.47:0/3839428474 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b5232f1460 0x55b5232f2850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:30:18.420 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:18.417+0000 7f5c1ae57740 1 --2- 172.21.15.47:0/3839428474 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b5232e8dd0 0x55b5232f0b20 unknown :-1 s=CLOSED pgs=884 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:30:18.420 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:18.417+0000 7f5c1ae57740 1 --2- 172.21.15.47:0/3839428474 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b5232e80f0 0x55b5232f05e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:30:18.420 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:18.417+0000 7f5c1ae57740 1 -- 172.21.15.47:0/3839428474 >> 172.21.15.47:0/3839428474 conn(0x55b5232af200 msgr2=0x55b5232e2670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:30:18.420 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:18.417+0000 7f5c1ae57740 1 -- 172.21.15.47:0/3839428474 shutdown_connections 2024-03-21T10:30:18.420 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:18.417+0000 7f5c1ae57740 1 -- 172.21.15.47:0/3839428474 wait complete. 2024-03-21T10:30:18.420 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:18.417+0000 7f5c1ae57740 1 -- start start 2024-03-21T10:30:18.420 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:18.417+0000 7f5c1ae57740 1 Processor -- start 2024-03-21T10:30:18.420 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:18.418+0000 7f5c1ae57740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b5232e80f0 0x55b5232f9150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:30:18.420 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:18.418+0000 7f5c1ae57740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b5232e8dd0 0x55b5232eb050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:30:18.420 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:18.418+0000 7f5c1ae57740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b5232f1460 0x55b5232eb590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:30:18.420 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:18.418+0000 7f5c1ae57740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c101b5f20 con 0x55b5232e8dd0 2024-03-21T10:30:18.420 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:18.418+0000 7f5c1ae57740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c101b58c0 con 0x55b5232e80f0 2024-03-21T10:30:18.421 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:18.418+0000 7f5c1ae57740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c101b2dc0 con 0x55b5232f1460 2024-03-21T10:30:18.421 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:18.418+0000 7f5c1ae57740 10 client.0 _ll_register_callbacks cb 0x7f5c0409b460 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T10:30:18.421 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:18.418+0000 7f5c14e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b5232e8dd0 0x55b5232eb050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:30:18.421 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:18.418+0000 7f5c14e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b5232e8dd0 0x55b5232eb050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:38544/0 (socket says 172.21.15.47:38544) 2024-03-21T10:30:18.421 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:18.418+0000 7f5c14e00640 1 -- 172.21.15.47:0/1209205628 learned_addr learned my addr 172.21.15.47:0/1209205628 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:30:18.421 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:18.418+0000 7f5c15800640 1 --2- 172.21.15.47:0/1209205628 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b5232e80f0 0x55b5232f9150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:30:18.421 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:18.419+0000 7f5c16200640 1 --2- 172.21.15.47:0/1209205628 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b5232f1460 0x55b5232eb590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:30:18.421 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:18.419+0000 7f5c14e00640 1 -- 172.21.15.47:0/1209205628 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b5232f1460 msgr2=0x55b5232eb590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:30:18.421 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:18.419+0000 7f5c14e00640 1 --2- 172.21.15.47:0/1209205628 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b5232f1460 0x55b5232eb590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:30:18.421 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:18.419+0000 7f5c14e00640 1 -- 172.21.15.47:0/1209205628 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b5232e80f0 msgr2=0x55b5232f9150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:30:18.421 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:18.419+0000 7f5c14e00640 1 --2- 172.21.15.47:0/1209205628 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b5232e80f0 0x55b5232f9150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:30:18.421 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:18.419+0000 7f5c14e00640 1 -- 172.21.15.47:0/1209205628 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55b5232ebad0 con 0x55b5232e8dd0 2024-03-21T10:30:18.421 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:18.419+0000 7f5c16200640 1 --2- 172.21.15.47:0/1209205628 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b5232f1460 0x55b5232eb590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T10:30:18.421 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:18.419+0000 7f5c14e00640 1 --2- 172.21.15.47:0/1209205628 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b5232e8dd0 0x55b5232eb050 secure :-1 s=READY pgs=885 cs=0 l=1 rev1=1 crypto rx=0x7f5bfc006010 tx=0x7f5bfc005850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ceb47bfc9ba17101 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:30:18.421 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:18.419+0000 7f5bf7400640 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-03-21T10:30:18.422 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:18.419+0000 7f5bf7400640 1 -- 172.21.15.47:0/1209205628 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5bfc005d10 con 0x55b5232e8dd0 2024-03-21T10:30:18.422 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:18.419+0000 7f5c1ae57740 1 -- 172.21.15.47:0/1209205628 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f5bfc0035f0 con 0x55b5232e8dd0 2024-03-21T10:30:18.422 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:18.419+0000 7f5bf7400640 1 -- 172.21.15.47:0/1209205628 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(97..97 src has 1..97) v4 ==== 9474+0+0 (secure 0 0 0) 0x7f5bfc02b230 con 0x55b5232e8dd0 2024-03-21T10:30:18.422 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:18.420+0000 7f5bf7400640 1 -- 172.21.15.47:0/1209205628 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5bfc042050 con 0x55b5232e8dd0 2024-03-21T10:30:18.422 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:18.420+0000 7f5c1ae57740 10 client.17268 fetch_fsmap learned FSMap version 32 2024-03-21T10:30:18.422 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:18.420+0000 7f5c1ae57740 1 -- 172.21.15.47:0/1209205628 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x55b52335d790 con 0x55b5232e8dd0 2024-03-21T10:30:18.422 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:18.420+0000 7f5bf7400640 1 -- 172.21.15.47:0/1209205628 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f5bfc0035f0 con 0x55b5232e8dd0 2024-03-21T10:30:18.422 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:18.420+0000 7f5bf7400640 1 -- 172.21.15.47:0/1209205628 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 32) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f5bfc018070 con 0x55b5232e8dd0 2024-03-21T10:30:18.422 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:18.420+0000 7f5c1ae57740 10 client.17268 fetch_fsmap finished waiting for FSMap version 32 2024-03-21T10:30:18.422 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:18.420+0000 7f5c1ae57740 10 client.17268 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-21T10:30:18.422 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:18.420+0000 7f5c1ae57740 20 client.17268 populate_metadata read hostname 'smithi047' 2024-03-21T10:30:18.422 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:18.420+0000 7f5c1ae57740 1 --2- 172.21.15.47:0/1209205628 >> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] conn(0x55b52335dc00 0x55b52335fff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:30:18.422 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:18.420+0000 7f5c1ae57740 4 client.17268 mds_command: new command op to 34535 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-21T10:30:18.423 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:18.420+0000 7f5c1ae57740 1 -- 172.21.15.47:0/1209205628 --> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55b523253ae0 con 0x55b52335dc00 2024-03-21T10:30:18.424 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:18.421+0000 7f5c15800640 1 --2- 172.21.15.47:0/1209205628 >> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] conn(0x55b52335dc00 0x55b52335fff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:30:18.424 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:18.421+0000 7f5c15800640 1 --2- 172.21.15.47:0/1209205628 >> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] conn(0x55b52335dc00 0x55b52335fff0 crc :-1 s=READY pgs=144 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e74b63a4efcbfc15 server_cookie=ffa31829da86527f in_seq=0 out_seq=0 2024-03-21T10:30:18.424 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:18.421+0000 7f5bf7400640 10 client.17268 ms_handle_connect on v2:172.21.15.100:6838/4173168440 2024-03-21T10:30:18.424 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:18.421+0000 7f5bf7400640 1 -- 172.21.15.47:0/1209205628 <== mds.0 v2:172.21.15.100:6838/4173168440 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x55b523253ae0 con 0x55b52335dc00 2024-03-21T10:30:18.424 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:18.421+0000 7f5bf7400640 10 client.17268 handle_command_reply: tid=0 2024-03-21T10:30:18.424 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:18.421+0000 7f5c1ae57740 1 client.17268 shutdown 2024-03-21T10:30:18.424 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:18.422+0000 7f5c1ae57740 1 -- 172.21.15.47:0/1209205628 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b5232e8dd0 msgr2=0x55b5232eb050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:30:18.424 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:18.422+0000 7f5c1ae57740 1 --2- 172.21.15.47:0/1209205628 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b5232e8dd0 0x55b5232eb050 secure :-1 s=READY pgs=885 cs=0 l=1 rev1=1 crypto rx=0x7f5bfc006010 tx=0x7f5bfc005850 comp rx=0 tx=0).stop 2024-03-21T10:30:18.424 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:18.422+0000 7f5c1ae57740 1 -- 172.21.15.47:0/1209205628 shutdown_connections 2024-03-21T10:30:18.424 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:18.422+0000 7f5c1ae57740 1 --2- 172.21.15.47:0/1209205628 >> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] conn(0x55b52335dc00 0x55b52335fff0 crc :-1 s=READY pgs=144 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:30:18.425 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:18.422+0000 7f5c1ae57740 1 --2- 172.21.15.47:0/1209205628 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b5232f1460 0x55b5232eb590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:30:18.425 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:18.422+0000 7f5c1ae57740 1 --2- 172.21.15.47:0/1209205628 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b5232e8dd0 0x55b5232eb050 unknown :-1 s=CLOSED pgs=885 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:30:18.425 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:18.422+0000 7f5c1ae57740 1 --2- 172.21.15.47:0/1209205628 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b5232e80f0 0x55b5232f9150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:30:18.425 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:18.422+0000 7f5c1ae57740 1 -- 172.21.15.47:0/1209205628 >> 172.21.15.47:0/1209205628 conn(0x55b5232af200 msgr2=0x55b5232e1330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:30:18.425 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:18.422+0000 7f5bf7400640 0 client.17268 ms_handle_reset on v2:172.21.15.100:6838/4173168440 2024-03-21T10:30:18.425 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:18.422+0000 7f5c1ae57740 1 -- 172.21.15.47:0/1209205628 shutdown_connections 2024-03-21T10:30:18.425 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:18.422+0000 7f5c1ae57740 1 -- 172.21.15.47:0/1209205628 wait complete. 2024-03-21T10:30:18.425 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:18.422+0000 7f5c1ae57740 20 client.17268 trim_cache size 0 max 16384 2024-03-21T10:30:18.425 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-03-21T10:30:18.425 INFO:teuthology.orchestra.run.smithi047.stdout: "status": "IDLE (waiting for more scrubs)", 2024-03-21T10:30:18.426 INFO:teuthology.orchestra.run.smithi047.stdout: "scrubs": { 2024-03-21T10:30:18.426 INFO:teuthology.orchestra.run.smithi047.stdout: "72f8e749-8322-4803-8156-ec32d7aefe22": { 2024-03-21T10:30:18.426 INFO:teuthology.orchestra.run.smithi047.stdout: "path": "/", 2024-03-21T10:30:18.426 INFO:teuthology.orchestra.run.smithi047.stdout: "tag": "72f8e749-8322-4803-8156-ec32d7aefe22", 2024-03-21T10:30:18.426 INFO:teuthology.orchestra.run.smithi047.stdout: "options": "recursive,force" 2024-03-21T10:30:18.426 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-21T10:30:18.426 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-21T10:30:18.426 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-03-21T10:30:18.427 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:18.423+0000 7f5c16c00640 1 -- 172.21.15.47:0/2800061000 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f5bf80a2fa0 msgr2=0x7f5bf80a5460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:30:18.427 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:18.423+0000 7f5c16c00640 1 --2- 172.21.15.47:0/2800061000 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f5bf80a2fa0 0x7f5bf80a5460 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f5c080099a0 tx=0x7f5c08033c90 comp rx=0 tx=0).stop 2024-03-21T10:30:18.427 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:18.423+0000 7f5c16c00640 1 -- 172.21.15.47:0/2800061000 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5c101004c0 msgr2=0x7f5c1019e270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:30:18.427 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:18.423+0000 7f5c16c00640 1 --2- 172.21.15.47:0/2800061000 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5c101004c0 0x7f5c1019e270 secure :-1 s=READY pgs=881 cs=0 l=1 rev1=1 crypto rx=0x7f5c0400d770 tx=0x7f5c0400dc30 comp rx=0 tx=0).stop 2024-03-21T10:30:18.427 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:18.423+0000 7f5c16c00640 1 -- 172.21.15.47:0/2800061000 shutdown_connections 2024-03-21T10:30:18.427 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:18.423+0000 7f5c16c00640 1 --2- 172.21.15.47:0/2800061000 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5c101058c0 0x7f5c101a2b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:30:18.427 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:18.423+0000 7f5c16c00640 1 --2- 172.21.15.47:0/2800061000 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f5bf80a2fa0 0x7f5bf80a5460 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:30:18.427 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:18.423+0000 7f5c16c00640 1 --2- 172.21.15.47:0/2800061000 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5c10100de0 0x7f5c1019e7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:30:18.427 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:18.423+0000 7f5c16c00640 1 --2- 172.21.15.47:0/2800061000 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5c101004c0 0x7f5c1019e270 unknown :-1 s=CLOSED pgs=881 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:30:18.427 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:18.423+0000 7f5c16c00640 1 -- 172.21.15.47:0/2800061000 >> 172.21.15.47:0/2800061000 conn(0x7f5c100fbef0 msgr2=0x7f5c10103270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:30:18.427 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:18.423+0000 7f5c16c00640 1 -- 172.21.15.47:0/2800061000 shutdown_connections 2024-03-21T10:30:18.428 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:18.423+0000 7f5c16c00640 1 -- 172.21.15.47:0/2800061000 wait complete. 2024-03-21T10:30:18.440 INFO:tasks.cephfs.filesystem:scrub status for tag:72f8e749-8322-4803-8156-ec32d7aefe22 - {'path': '/', 'tag': '72f8e749-8322-4803-8156-ec32d7aefe22', 'options': 'recursive,force'} 2024-03-21T10:30:19.488 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:30:19 smithi157 ceph-mon[35581]: pgmap v153: 129 pgs: 129 active+clean; 277 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 85 B/s wr, 8 op/s 2024-03-21T10:30:19.488 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:30:19 smithi157 ceph-mon[35581]: scrub queued for path: / 2024-03-21T10:30:19.488 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:30:19 smithi157 ceph-mon[35581]: scrub summary: idle+waiting paths [/] 2024-03-21T10:30:19.488 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:30:19 smithi157 ceph-mon[35581]: scrub summary: active paths [/] 2024-03-21T10:30:19.488 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:30:19 smithi157 ceph-mon[35581]: scrub summary: idle+waiting paths [/] 2024-03-21T10:30:19.488 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:30:19 smithi157 ceph-mon[35581]: scrub summary: idle 2024-03-21T10:30:19.530 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:30:19 smithi100 ceph-mon[35193]: pgmap v153: 129 pgs: 129 active+clean; 277 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 85 B/s wr, 8 op/s 2024-03-21T10:30:19.530 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:30:19 smithi100 ceph-mon[35193]: scrub queued for path: / 2024-03-21T10:30:19.531 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:30:19 smithi100 ceph-mon[35193]: scrub summary: idle+waiting paths [/] 2024-03-21T10:30:19.531 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:30:19 smithi100 ceph-mon[35193]: scrub summary: active paths [/] 2024-03-21T10:30:19.531 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:30:19 smithi100 ceph-mon[35193]: scrub summary: idle+waiting paths [/] 2024-03-21T10:30:19.531 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:30:19 smithi100 ceph-mon[35193]: scrub summary: idle 2024-03-21T10:30:19.712 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:30:19 smithi047 ceph-mon[31547]: pgmap v153: 129 pgs: 129 active+clean; 277 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 85 B/s wr, 8 op/s 2024-03-21T10:30:19.712 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:30:19 smithi047 ceph-mon[31547]: scrub queued for path: / 2024-03-21T10:30:19.712 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:30:19 smithi047 ceph-mon[31547]: scrub summary: idle+waiting paths [/] 2024-03-21T10:30:19.712 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:30:19 smithi047 ceph-mon[31547]: scrub summary: active paths [/] 2024-03-21T10:30:19.712 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:30:19 smithi047 ceph-mon[31547]: scrub summary: idle+waiting paths [/] 2024-03-21T10:30:19.712 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:30:19 smithi047 ceph-mon[31547]: scrub summary: idle 2024-03-21T10:30:21.712 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:30:21 smithi047 ceph-mon[31547]: pgmap v154: 129 pgs: 129 active+clean; 277 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 8 op/s 2024-03-21T10:30:21.727 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:30:21 smithi100 ceph-mon[35193]: pgmap v154: 129 pgs: 129 active+clean; 277 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 8 op/s 2024-03-21T10:30:21.737 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:30:21 smithi157 ceph-mon[35581]: pgmap v154: 129 pgs: 129 active+clean; 277 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 8 op/s 2024-03-21T10:30:23.507 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:30:23 smithi157 ceph-mon[35581]: pgmap v155: 129 pgs: 129 active+clean; 277 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 KiB/s rd, 85 B/s wr, 13 op/s 2024-03-21T10:30:23.514 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:30:23 smithi047 ceph-mon[31547]: pgmap v155: 129 pgs: 129 active+clean; 277 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 KiB/s rd, 85 B/s wr, 13 op/s 2024-03-21T10:30:23.727 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:30:23 smithi100 ceph-mon[35193]: pgmap v155: 129 pgs: 129 active+clean; 277 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 KiB/s rd, 85 B/s wr, 13 op/s 2024-03-21T10:30:25.712 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:30:25 smithi047 ceph-mon[31547]: pgmap v156: 129 pgs: 129 active+clean; 277 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 9 op/s 2024-03-21T10:30:25.727 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:30:25 smithi100 ceph-mon[35193]: pgmap v156: 129 pgs: 129 active+clean; 277 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 9 op/s 2024-03-21T10:30:25.738 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:30:25 smithi157 ceph-mon[35581]: pgmap v156: 129 pgs: 129 active+clean; 277 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 9 op/s 2024-03-21T10:30:27.712 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:30:27 smithi047 ceph-mon[31547]: pgmap v157: 129 pgs: 129 active+clean; 277 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 13 op/s 2024-03-21T10:30:27.727 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:30:27 smithi100 ceph-mon[35193]: pgmap v157: 129 pgs: 129 active+clean; 277 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 13 op/s 2024-03-21T10:30:27.738 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:30:27 smithi157 ceph-mon[35581]: pgmap v157: 129 pgs: 129 active+clean; 277 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 13 op/s 2024-03-21T10:30:29.712 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:30:29 smithi047 ceph-mon[31547]: pgmap v158: 129 pgs: 129 active+clean; 277 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 9 op/s 2024-03-21T10:30:29.727 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:30:29 smithi100 ceph-mon[35193]: pgmap v158: 129 pgs: 129 active+clean; 277 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 9 op/s 2024-03-21T10:30:29.737 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:30:29 smithi157 ceph-mon[35581]: pgmap v158: 129 pgs: 129 active+clean; 277 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 9 op/s 2024-03-21T10:30:31.712 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:30:31 smithi047 ceph-mon[31547]: pgmap v159: 129 pgs: 129 active+clean; 277 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 9 op/s 2024-03-21T10:30:31.727 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:30:31 smithi100 ceph-mon[35193]: pgmap v159: 129 pgs: 129 active+clean; 277 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 9 op/s 2024-03-21T10:30:31.737 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:30:31 smithi157 ceph-mon[35581]: pgmap v159: 129 pgs: 129 active+clean; 277 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 9 op/s 2024-03-21T10:30:33.712 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:30:33 smithi047 ceph-mon[31547]: pgmap v160: 129 pgs: 129 active+clean; 277 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 13 op/s 2024-03-21T10:30:33.727 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:30:33 smithi100 ceph-mon[35193]: pgmap v160: 129 pgs: 129 active+clean; 277 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 13 op/s 2024-03-21T10:30:33.737 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:30:33 smithi157 ceph-mon[35581]: pgmap v160: 129 pgs: 129 active+clean; 277 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 13 op/s 2024-03-21T10:30:35.712 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:30:35 smithi047 ceph-mon[31547]: pgmap v161: 129 pgs: 129 active+clean; 277 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-03-21T10:30:35.727 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:30:35 smithi100 ceph-mon[35193]: pgmap v161: 129 pgs: 129 active+clean; 277 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-03-21T10:30:35.737 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:30:35 smithi157 ceph-mon[35581]: pgmap v161: 129 pgs: 129 active+clean; 277 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-03-21T10:30:37.712 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:30:37 smithi047 ceph-mon[31547]: pgmap v162: 129 pgs: 129 active+clean; 277 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 11 op/s 2024-03-21T10:30:37.727 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:30:37 smithi100 ceph-mon[35193]: pgmap v162: 129 pgs: 129 active+clean; 277 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 11 op/s 2024-03-21T10:30:37.737 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:30:37 smithi157 ceph-mon[35581]: pgmap v162: 129 pgs: 129 active+clean; 277 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 11 op/s 2024-03-21T10:30:38.712 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:30:38 smithi047 ceph-mon[31547]: from='mgr.15687 172.21.15.47:0/8730661' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T10:30:38.727 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:30:38 smithi100 ceph-mon[35193]: from='mgr.15687 172.21.15.47:0/8730661' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T10:30:38.737 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:30:38 smithi157 ceph-mon[35581]: from='mgr.15687 172.21.15.47:0/8730661' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T10:30:39.712 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:30:39 smithi047 ceph-mon[31547]: pgmap v163: 129 pgs: 129 active+clean; 277 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 8 op/s 2024-03-21T10:30:39.712 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:30:39 smithi047 ceph-mon[31547]: from='mgr.15687 172.21.15.47:0/8730661' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T10:30:39.712 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:30:39 smithi047 ceph-mon[31547]: from='mgr.15687 172.21.15.47:0/8730661' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T10:30:39.712 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:30:39 smithi047 ceph-mon[31547]: from='mgr.15687 172.21.15.47:0/8730661' entity='mgr.x' 2024-03-21T10:30:39.727 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:30:39 smithi100 ceph-mon[35193]: pgmap v163: 129 pgs: 129 active+clean; 277 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 8 op/s 2024-03-21T10:30:39.727 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:30:39 smithi100 ceph-mon[35193]: from='mgr.15687 172.21.15.47:0/8730661' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T10:30:39.727 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:30:39 smithi100 ceph-mon[35193]: from='mgr.15687 172.21.15.47:0/8730661' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T10:30:39.727 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:30:39 smithi100 ceph-mon[35193]: from='mgr.15687 172.21.15.47:0/8730661' entity='mgr.x' 2024-03-21T10:30:39.737 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:30:39 smithi157 ceph-mon[35581]: pgmap v163: 129 pgs: 129 active+clean; 277 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 8 op/s 2024-03-21T10:30:39.737 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:30:39 smithi157 ceph-mon[35581]: from='mgr.15687 172.21.15.47:0/8730661' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T10:30:39.738 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:30:39 smithi157 ceph-mon[35581]: from='mgr.15687 172.21.15.47:0/8730661' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T10:30:39.738 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:30:39 smithi157 ceph-mon[35581]: from='mgr.15687 172.21.15.47:0/8730661' entity='mgr.x' 2024-03-21T10:30:41.712 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:30:41 smithi047 ceph-mon[31547]: pgmap v164: 129 pgs: 129 active+clean; 277 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-03-21T10:30:41.727 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:30:41 smithi100 ceph-mon[35193]: pgmap v164: 129 pgs: 129 active+clean; 277 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-03-21T10:30:41.737 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:30:41 smithi157 ceph-mon[35581]: pgmap v164: 129 pgs: 129 active+clean; 277 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-03-21T10:30:43.712 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:30:43 smithi047 ceph-mon[31547]: pgmap v165: 129 pgs: 129 active+clean; 277 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-03-21T10:30:43.727 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:30:43 smithi100 ceph-mon[35193]: pgmap v165: 129 pgs: 129 active+clean; 277 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-03-21T10:30:43.737 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:30:43 smithi157 ceph-mon[35581]: pgmap v165: 129 pgs: 129 active+clean; 277 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-03-21T10:30:45.712 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:30:45 smithi047 ceph-mon[31547]: pgmap v166: 129 pgs: 129 active+clean; 277 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-21T10:30:45.727 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:30:45 smithi100 ceph-mon[35193]: pgmap v166: 129 pgs: 129 active+clean; 277 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-21T10:30:45.737 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:30:45 smithi157 ceph-mon[35581]: pgmap v166: 129 pgs: 129 active+clean; 277 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-21T10:30:47.712 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:30:47 smithi047 ceph-mon[31547]: pgmap v167: 129 pgs: 129 active+clean; 277 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-03-21T10:30:47.727 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:30:47 smithi100 ceph-mon[35193]: pgmap v167: 129 pgs: 129 active+clean; 277 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-03-21T10:30:47.737 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:30:47 smithi157 ceph-mon[35581]: pgmap v167: 129 pgs: 129 active+clean; 277 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-03-21T10:30:48.440 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-21T10:30:48.441 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-21T10:30:48.679 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:48.679+0000 7ff36de00640 1 -- 172.21.15.47:0/1195517248 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff368100700 msgr2=0x7ff368100ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:30:48.679 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:48.679+0000 7ff36de00640 1 --2- 172.21.15.47:0/1195517248 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff368100700 0x7ff368100ae0 secure :-1 s=READY pgs=886 cs=0 l=1 rev1=1 crypto rx=0x7ff358009960 tx=0x7ff35802bca0 comp rx=0 tx=0).stop 2024-03-21T10:30:48.679 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:48.679+0000 7ff36de00640 1 -- 172.21.15.47:0/1195517248 shutdown_connections 2024-03-21T10:30:48.679 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:48.679+0000 7ff36de00640 1 --2- 172.21.15.47:0/1195517248 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff368105b00 0x7ff368105fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:30:48.679 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:48.679+0000 7ff36de00640 1 --2- 172.21.15.47:0/1195517248 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff368101020 0x7ff3681054e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:30:48.679 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:48.679+0000 7ff36de00640 1 --2- 172.21.15.47:0/1195517248 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff368100700 0x7ff368100ae0 unknown :-1 s=CLOSED pgs=886 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:30:48.679 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:48.679+0000 7ff36de00640 1 -- 172.21.15.47:0/1195517248 >> 172.21.15.47:0/1195517248 conn(0x7ff368077700 msgr2=0x7ff368077b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:30:48.679 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:48.679+0000 7ff36de00640 1 -- 172.21.15.47:0/1195517248 shutdown_connections 2024-03-21T10:30:48.679 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:48.679+0000 7ff36de00640 1 -- 172.21.15.47:0/1195517248 wait complete. 2024-03-21T10:30:48.680 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:48.681+0000 7ff36de00640 1 Processor -- start 2024-03-21T10:30:48.680 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:48.681+0000 7ff36de00640 1 -- start start 2024-03-21T10:30:48.681 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:48.681+0000 7ff36de00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff368100700 0x7ff36819e5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:30:48.681 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:48.682+0000 7ff36de00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff368101020 0x7ff36819eaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:30:48.681 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:48.682+0000 7ff36de00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff368105b00 0x7ff3681a2e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:30:48.681 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:48.682+0000 7ff36de00640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff368117f50 con 0x7ff368100700 2024-03-21T10:30:48.681 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:48.682+0000 7ff36de00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff368117dd0 con 0x7ff368101020 2024-03-21T10:30:48.681 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:48.682+0000 7ff36de00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3681180d0 con 0x7ff368105b00 2024-03-21T10:30:48.681 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:48.682+0000 7ff366a00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff368100700 0x7ff36819e5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:30:48.681 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:48.682+0000 7ff366a00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff368100700 0x7ff36819e5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:46212/0 (socket says 172.21.15.47:46212) 2024-03-21T10:30:48.681 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:48.682+0000 7ff366a00640 1 -- 172.21.15.47:0/1736683395 learned_addr learned my addr 172.21.15.47:0/1736683395 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:30:48.681 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:48.682+0000 7ff35fe00640 1 --2- 172.21.15.47:0/1736683395 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff368101020 0x7ff36819eaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:30:48.681 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:48.682+0000 7ff367400640 1 --2- 172.21.15.47:0/1736683395 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff368105b00 0x7ff3681a2e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:30:48.682 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:48.682+0000 7ff366a00640 1 -- 172.21.15.47:0/1736683395 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff368105b00 msgr2=0x7ff3681a2e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:30:48.683 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:48.682+0000 7ff366a00640 1 --2- 172.21.15.47:0/1736683395 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff368105b00 0x7ff3681a2e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:30:48.683 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:48.682+0000 7ff366a00640 1 -- 172.21.15.47:0/1736683395 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff368101020 msgr2=0x7ff36819eaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:30:48.683 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:48.682+0000 7ff366a00640 1 --2- 172.21.15.47:0/1736683395 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff368101020 0x7ff36819eaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:30:48.683 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:48.682+0000 7ff366a00640 1 -- 172.21.15.47:0/1736683395 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff3681a35a0 con 0x7ff368100700 2024-03-21T10:30:48.683 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:48.683+0000 7ff366a00640 1 --2- 172.21.15.47:0/1736683395 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff368100700 0x7ff36819e5b0 secure :-1 s=READY pgs=887 cs=0 l=1 rev1=1 crypto rx=0x7ff358036040 tx=0x7ff3580045e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:30:48.683 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:48.683+0000 7ff36ca00640 1 -- 172.21.15.47:0/1736683395 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff358043020 con 0x7ff368100700 2024-03-21T10:30:48.683 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:48.683+0000 7ff36ca00640 1 -- 172.21.15.47:0/1736683395 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7ff35802dbf0 con 0x7ff368100700 2024-03-21T10:30:48.683 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:48.683+0000 7ff36de00640 1 -- 172.21.15.47:0/1736683395 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff3681a3830 con 0x7ff368100700 2024-03-21T10:30:48.683 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:48.683+0000 7ff36ca00640 1 -- 172.21.15.47:0/1736683395 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff358003a50 con 0x7ff368100700 2024-03-21T10:30:48.683 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:48.683+0000 7ff36de00640 1 -- 172.21.15.47:0/1736683395 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff3681ab120 con 0x7ff368100700 2024-03-21T10:30:48.685 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:48.687+0000 7ff36ca00640 1 -- 172.21.15.47:0/1736683395 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 27) v1 ==== 137985+0+0 (secure 0 0 0) 0x7ff358046020 con 0x7ff368100700 2024-03-21T10:30:48.688 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:48.688+0000 7ff36ca00640 1 --2- 172.21.15.47:0/1736683395 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7ff34c0a3110 0x7ff34c0a55d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:30:48.688 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:48.688+0000 7ff370bbe740 1 Processor -- start 2024-03-21T10:30:48.688 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:48.688+0000 7ff36ca00640 1 -- 172.21.15.47:0/1736683395 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 9474+0+0 (secure 0 0 0) 0x7ff3580f1be0 con 0x7ff368100700 2024-03-21T10:30:48.688 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:48.688+0000 7ff370bbe740 1 -- start start 2024-03-21T10:30:48.688 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:48.688+0000 7ff35fe00640 1 --2- 172.21.15.47:0/1736683395 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7ff34c0a3110 0x7ff34c0a55d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:30:48.688 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:48.689+0000 7ff370bbe740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5590f9fd5c00 0x5590f9fd4950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:30:48.688 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:48.689+0000 7ff370bbe740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5590f9fd4f20 0x5590f9fd53a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:30:48.688 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:48.689+0000 7ff370bbe740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5590f9fde290 0x5590f9fe0680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:30:48.688 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:48.689+0000 7ff370bbe740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x5590f9f39250 con 0x5590f9fd4f20 2024-03-21T10:30:48.688 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:48.689+0000 7ff370bbe740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x5590f9ee1a30 con 0x5590f9fde290 2024-03-21T10:30:48.688 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:48.689+0000 7ff370bbe740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x5590f9f352f0 con 0x5590f9fd5c00 2024-03-21T10:30:48.690 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:48.689+0000 7ff35fe00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5590f9fd4f20 0x5590f9fd53a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:30:48.690 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:48.689+0000 7ff35fe00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5590f9fd4f20 0x5590f9fd53a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:46218/0 (socket says 172.21.15.47:46218) 2024-03-21T10:30:48.690 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:48.689+0000 7ff35fe00640 1 -- 172.21.15.47:0/3236147733 learned_addr learned my addr 172.21.15.47:0/3236147733 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:30:48.690 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:48.689+0000 7ff367400640 1 --2- 172.21.15.47:0/3236147733 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5590f9fde290 0x5590f9fe0680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:30:48.691 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:48.689+0000 7ff366a00640 1 --2- 172.21.15.47:0/3236147733 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5590f9fd5c00 0x5590f9fd4950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:30:48.691 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:48.690+0000 7ff35fe00640 1 --2- 172.21.15.47:0/1736683395 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7ff34c0a3110 0x7ff34c0a55d0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7ff350005fd0 tx=0x7ff35000f730 comp rx=0 tx=0).ready entity=mgr.15687 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:30:48.691 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:48.690+0000 7ff35fe00640 1 -- 172.21.15.47:0/3236147733 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5590f9fd5c00 msgr2=0x5590f9fd4950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:30:48.691 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:48.690+0000 7ff35fe00640 1 --2- 172.21.15.47:0/3236147733 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5590f9fd5c00 0x5590f9fd4950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:30:48.691 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:48.690+0000 7ff35fe00640 1 -- 172.21.15.47:0/3236147733 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5590f9fde290 msgr2=0x5590f9fe0680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:30:48.691 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:48.690+0000 7ff35fe00640 1 --2- 172.21.15.47:0/3236147733 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5590f9fde290 0x5590f9fe0680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:30:48.691 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:48.690+0000 7ff35fe00640 1 -- 172.21.15.47:0/3236147733 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5590f9ecc690 con 0x5590f9fd4f20 2024-03-21T10:30:48.691 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:48.690+0000 7ff35fe00640 1 --2- 172.21.15.47:0/3236147733 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5590f9fd4f20 0x5590f9fd53a0 secure :-1 s=READY pgs=888 cs=0 l=1 rev1=1 crypto rx=0x7ff350019000 tx=0x7ff35001a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7db2e4a2e0b7494c server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:30:48.691 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:48.690+0000 7ff357e00640 1 -- 172.21.15.47:0/3236147733 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff35001d620 con 0x5590f9fd4f20 2024-03-21T10:30:48.691 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:48.690+0000 7ff357e00640 1 -- 172.21.15.47:0/3236147733 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7ff35001dc70 con 0x5590f9fd4f20 2024-03-21T10:30:48.691 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:48.690+0000 7ff357e00640 1 -- 172.21.15.47:0/3236147733 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff350010b90 con 0x5590f9fd4f20 2024-03-21T10:30:48.691 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:48.691+0000 7ff370bbe740 1 -- 172.21.15.47:0/3236147733 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5590f9fd4f20 msgr2=0x5590f9fd53a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:30:48.691 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:48.691+0000 7ff370bbe740 1 --2- 172.21.15.47:0/3236147733 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5590f9fd4f20 0x5590f9fd53a0 secure :-1 s=READY pgs=888 cs=0 l=1 rev1=1 crypto rx=0x7ff350019000 tx=0x7ff35001a000 comp rx=0 tx=0).stop 2024-03-21T10:30:48.691 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:48.691+0000 7ff370bbe740 1 -- 172.21.15.47:0/3236147733 shutdown_connections 2024-03-21T10:30:48.691 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:48.691+0000 7ff370bbe740 1 --2- 172.21.15.47:0/3236147733 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5590f9fde290 0x5590f9fe0680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:30:48.692 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:48.691+0000 7ff370bbe740 1 --2- 172.21.15.47:0/3236147733 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5590f9fd4f20 0x5590f9fd53a0 unknown :-1 s=CLOSED pgs=888 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:30:48.692 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:48.691+0000 7ff370bbe740 1 --2- 172.21.15.47:0/3236147733 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5590f9fd5c00 0x5590f9fd4950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:30:48.692 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:48.691+0000 7ff370bbe740 1 -- 172.21.15.47:0/3236147733 >> 172.21.15.47:0/3236147733 conn(0x5590f9f9c200 msgr2=0x5590f9fcf970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:30:48.692 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:48.691+0000 7ff370bbe740 1 -- 172.21.15.47:0/3236147733 shutdown_connections 2024-03-21T10:30:48.692 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:48.691+0000 7ff370bbe740 1 -- 172.21.15.47:0/3236147733 wait complete. 2024-03-21T10:30:48.695 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:48.692+0000 7ff370bbe740 1 -- start start 2024-03-21T10:30:48.695 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:48.692+0000 7ff370bbe740 1 Processor -- start 2024-03-21T10:30:48.695 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:48.693+0000 7ff370bbe740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5590f9fd4f20 0x5590f9fdd890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:30:48.695 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:48.693+0000 7ff370bbe740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5590f9fd5c00 0x5590f9fdac90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:30:48.695 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:48.693+0000 7ff370bbe740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5590f9fde290 0x5590f9fdb1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:30:48.695 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:48.693+0000 7ff370bbe740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x5590f9fe1100 con 0x5590f9fd4f20 2024-03-21T10:30:48.696 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:48.693+0000 7ff370bbe740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x5590f9ef4e80 con 0x5590f9fd5c00 2024-03-21T10:30:48.696 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:48.693+0000 7ff370bbe740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x5590f9fe1280 con 0x5590f9fde290 2024-03-21T10:30:48.696 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:48.693+0000 7ff370bbe740 10 client.0 _ll_register_callbacks cb 0x5590f9f07130 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T10:30:48.696 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:48.693+0000 7ff366a00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5590f9fd4f20 0x5590f9fdd890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:30:48.696 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:48.693+0000 7ff366a00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5590f9fd4f20 0x5590f9fdd890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:46222/0 (socket says 172.21.15.47:46222) 2024-03-21T10:30:48.696 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:48.693+0000 7ff366a00640 1 -- 172.21.15.47:0/948694018 learned_addr learned my addr 172.21.15.47:0/948694018 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:30:48.696 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:48.693+0000 7ff35fe00640 1 --2- 172.21.15.47:0/948694018 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5590f9fd5c00 0x5590f9fdac90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:30:48.696 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:48.693+0000 7ff367400640 1 --2- 172.21.15.47:0/948694018 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5590f9fde290 0x5590f9fdb1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:30:48.696 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:48.693+0000 7ff366a00640 1 -- 172.21.15.47:0/948694018 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5590f9fde290 msgr2=0x5590f9fdb1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:30:48.696 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:48.693+0000 7ff366a00640 1 --2- 172.21.15.47:0/948694018 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5590f9fde290 0x5590f9fdb1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:30:48.696 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:48.693+0000 7ff366a00640 1 -- 172.21.15.47:0/948694018 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5590f9fd5c00 msgr2=0x5590f9fdac90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:30:48.696 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:48.693+0000 7ff366a00640 1 --2- 172.21.15.47:0/948694018 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5590f9fd5c00 0x5590f9fdac90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:30:48.696 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:48.693+0000 7ff366a00640 1 -- 172.21.15.47:0/948694018 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5590f9d177d0 con 0x5590f9fd4f20 2024-03-21T10:30:48.696 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:48.693+0000 7ff367400640 1 --2- 172.21.15.47:0/948694018 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5590f9fde290 0x5590f9fdb1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T10:30:48.697 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:48.694+0000 7ff366a00640 1 --2- 172.21.15.47:0/948694018 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5590f9fd4f20 0x5590f9fdd890 secure :-1 s=READY pgs=889 cs=0 l=1 rev1=1 crypto rx=0x7ff358005ec0 tx=0x7ff35802dee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d79c74016299ba96 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:30:48.697 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:48.694+0000 7ff354c00640 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-03-21T10:30:48.697 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:48.694+0000 7ff354c00640 1 -- 172.21.15.47:0/948694018 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3580c66d0 con 0x5590f9fd4f20 2024-03-21T10:30:48.697 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:48.694+0000 7ff354c00640 1 -- 172.21.15.47:0/948694018 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(97..97 src has 1..97) v4 ==== 9474+0+0 (secure 0 0 0) 0x7ff3580053d0 con 0x5590f9fd4f20 2024-03-21T10:30:48.697 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:48.694+0000 7ff370bbe740 1 -- 172.21.15.47:0/948694018 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ff35001dc70 con 0x5590f9fd4f20 2024-03-21T10:30:48.697 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:48.694+0000 7ff354c00640 1 -- 172.21.15.47:0/948694018 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff3580bdf20 con 0x5590f9fd4f20 2024-03-21T10:30:48.697 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:48.694+0000 7ff370bbe740 10 client.17280 fetch_fsmap learned FSMap version 32 2024-03-21T10:30:48.697 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:48.694+0000 7ff370bbe740 1 -- 172.21.15.47:0/948694018 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x5590f9c4d3f0 con 0x5590f9fd4f20 2024-03-21T10:30:48.697 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:48.695+0000 7ff354c00640 1 -- 172.21.15.47:0/948694018 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7ff35001dc70 con 0x5590f9fd4f20 2024-03-21T10:30:48.697 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:48.695+0000 7ff354c00640 1 -- 172.21.15.47:0/948694018 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 32) v1 ==== 7942+0+0 (secure 0 0 0) 0x7ff3580b9a10 con 0x5590f9fd4f20 2024-03-21T10:30:48.697 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:48.695+0000 7ff370bbe740 10 client.17280 fetch_fsmap finished waiting for FSMap version 32 2024-03-21T10:30:48.697 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:48.695+0000 7ff370bbe740 10 client.17280 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-21T10:30:48.697 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:48.695+0000 7ff370bbe740 20 client.17280 populate_metadata read hostname 'smithi047' 2024-03-21T10:30:48.697 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:48.695+0000 7ff370bbe740 1 --2- 172.21.15.47:0/948694018 >> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] conn(0x5590f9fee0d0 0x5590f9ff0500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:30:48.698 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:48.695+0000 7ff370bbe740 4 client.17280 mds_command: new command op to 34535 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T10:30:48.698 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:48.695+0000 7ff370bbe740 1 -- 172.21.15.47:0/948694018 --> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5590f9f46790 con 0x5590f9fee0d0 2024-03-21T10:30:48.698 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:48.695+0000 7ff35fe00640 1 --2- 172.21.15.47:0/948694018 >> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] conn(0x5590f9fee0d0 0x5590f9ff0500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:30:48.698 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:48.696+0000 7ff35fe00640 1 --2- 172.21.15.47:0/948694018 >> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] conn(0x5590f9fee0d0 0x5590f9ff0500 crc :-1 s=READY pgs=145 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dbde6d85605a6b1c server_cookie=e09d10c2ccfe1841 in_seq=0 out_seq=0 2024-03-21T10:30:48.698 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:48.696+0000 7ff354c00640 10 client.17280 ms_handle_connect on v2:172.21.15.100:6838/4173168440 2024-03-21T10:30:48.699 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:48.697+0000 7ff354c00640 1 -- 172.21.15.47:0/948694018 <== mds.0 v2:172.21.15.100:6838/4173168440 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x5590f9f46790 con 0x5590f9fee0d0 2024-03-21T10:30:48.699 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:48.697+0000 7ff354c00640 10 client.17280 handle_command_reply: tid=0 2024-03-21T10:30:48.699 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:48.697+0000 7ff370bbe740 1 client.17280 shutdown 2024-03-21T10:30:48.699 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:48.697+0000 7ff370bbe740 1 -- 172.21.15.47:0/948694018 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5590f9fd4f20 msgr2=0x5590f9fdd890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:30:48.699 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:48.697+0000 7ff370bbe740 1 --2- 172.21.15.47:0/948694018 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5590f9fd4f20 0x5590f9fdd890 secure :-1 s=READY pgs=889 cs=0 l=1 rev1=1 crypto rx=0x7ff358005ec0 tx=0x7ff35802dee0 comp rx=0 tx=0).stop 2024-03-21T10:30:48.699 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:48.697+0000 7ff370bbe740 1 -- 172.21.15.47:0/948694018 shutdown_connections 2024-03-21T10:30:48.699 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:48.697+0000 7ff370bbe740 1 --2- 172.21.15.47:0/948694018 >> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] conn(0x5590f9fee0d0 0x5590f9ff0500 crc :-1 s=READY pgs=145 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:30:48.699 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:48.697+0000 7ff370bbe740 1 --2- 172.21.15.47:0/948694018 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5590f9fde290 0x5590f9fdb1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:30:48.699 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:48.697+0000 7ff370bbe740 1 --2- 172.21.15.47:0/948694018 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5590f9fd5c00 0x5590f9fdac90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:30:48.700 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:48.697+0000 7ff370bbe740 1 --2- 172.21.15.47:0/948694018 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5590f9fd4f20 0x5590f9fdd890 unknown :-1 s=CLOSED pgs=889 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:30:48.700 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:48.697+0000 7ff354c00640 0 client.17280 ms_handle_reset on v2:172.21.15.100:6838/4173168440 2024-03-21T10:30:48.700 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:48.698+0000 7ff370bbe740 1 -- 172.21.15.47:0/948694018 >> 172.21.15.47:0/948694018 conn(0x5590f9f9c200 msgr2=0x5590f9fd16c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:30:48.700 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:48.698+0000 7ff370bbe740 1 -- 172.21.15.47:0/948694018 shutdown_connections 2024-03-21T10:30:48.700 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:48.698+0000 7ff370bbe740 1 -- 172.21.15.47:0/948694018 wait complete. 2024-03-21T10:30:48.700 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:48.698+0000 7ff370bbe740 20 client.17280 trim_cache size 0 max 16384 2024-03-21T10:30:48.715 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:48.715+0000 7ff370bbe740 1 Processor -- start 2024-03-21T10:30:48.715 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:48.715+0000 7ff370bbe740 1 -- start start 2024-03-21T10:30:48.715 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:48.716+0000 7ff370bbe740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5590f9fd4f20 0x5590f9fda590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:30:48.716 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:48.716+0000 7ff370bbe740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5590f9fd5c00 0x5590f9fdd3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:30:48.716 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:48.716+0000 7ff370bbe740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5590f9fde290 0x5590f9fce910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:30:48.716 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:48.716+0000 7ff370bbe740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3681b4f10 con 0x5590f9fd5c00 2024-03-21T10:30:48.716 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:48.716+0000 7ff370bbe740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3681b6890 con 0x5590f9fde290 2024-03-21T10:30:48.716 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:48.716+0000 7ff370bbe740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3681b5570 con 0x5590f9fd4f20 2024-03-21T10:30:48.716 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:48.716+0000 7ff35fe00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5590f9fd5c00 0x5590f9fdd3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:30:48.716 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:48.716+0000 7ff35fe00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5590f9fd5c00 0x5590f9fdd3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:46238/0 (socket says 172.21.15.47:46238) 2024-03-21T10:30:48.716 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:48.716+0000 7ff35fe00640 1 -- 172.21.15.47:0/1244117664 learned_addr learned my addr 172.21.15.47:0/1244117664 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:30:48.716 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:48.716+0000 7ff367400640 1 --2- 172.21.15.47:0/1244117664 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5590f9fde290 0x5590f9fce910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:30:48.716 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:48.716+0000 7ff366a00640 1 --2- 172.21.15.47:0/1244117664 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5590f9fd4f20 0x5590f9fda590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:30:48.716 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:48.716+0000 7ff35fe00640 1 -- 172.21.15.47:0/1244117664 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5590f9fd4f20 msgr2=0x5590f9fda590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:30:48.716 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:48.716+0000 7ff35fe00640 1 --2- 172.21.15.47:0/1244117664 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5590f9fd4f20 0x5590f9fda590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:30:48.716 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:48.716+0000 7ff35fe00640 1 -- 172.21.15.47:0/1244117664 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5590f9fde290 msgr2=0x5590f9fce910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:30:48.716 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:48.716+0000 7ff35fe00640 1 --2- 172.21.15.47:0/1244117664 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5590f9fde290 0x5590f9fce910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:30:48.716 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:48.716+0000 7ff35fe00640 1 -- 172.21.15.47:0/1244117664 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5590f9c0d0f0 con 0x5590f9fd5c00 2024-03-21T10:30:48.720 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:48.717+0000 7ff35fe00640 1 --2- 172.21.15.47:0/1244117664 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5590f9fd5c00 0x5590f9fdd3c0 secure :-1 s=READY pgs=890 cs=0 l=1 rev1=1 crypto rx=0x5590f9fd6150 tx=0x7ff350005870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=98d0ec909574bb90 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:30:48.720 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:48.717+0000 7ff357400640 1 -- 172.21.15.47:0/1244117664 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff35003a070 con 0x5590f9fd5c00 2024-03-21T10:30:48.720 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:48.717+0000 7ff357400640 1 -- 172.21.15.47:0/1244117664 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7ff350034070 con 0x5590f9fd5c00 2024-03-21T10:30:48.720 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:48.717+0000 7ff357400640 1 -- 172.21.15.47:0/1244117664 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3500108a0 con 0x5590f9fd5c00 2024-03-21T10:30:48.720 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:48.718+0000 7ff370bbe740 1 -- 172.21.15.47:0/1244117664 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5590f9fd5c00 msgr2=0x5590f9fdd3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:30:48.721 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:48.718+0000 7ff370bbe740 1 --2- 172.21.15.47:0/1244117664 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5590f9fd5c00 0x5590f9fdd3c0 secure :-1 s=READY pgs=890 cs=0 l=1 rev1=1 crypto rx=0x5590f9fd6150 tx=0x7ff350005870 comp rx=0 tx=0).stop 2024-03-21T10:30:48.721 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:48.718+0000 7ff370bbe740 1 -- 172.21.15.47:0/1244117664 shutdown_connections 2024-03-21T10:30:48.721 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:48.718+0000 7ff370bbe740 1 --2- 172.21.15.47:0/1244117664 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5590f9fde290 0x5590f9fce910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:30:48.721 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:48.718+0000 7ff370bbe740 1 --2- 172.21.15.47:0/1244117664 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5590f9fd5c00 0x5590f9fdd3c0 unknown :-1 s=CLOSED pgs=890 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:30:48.721 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:48.718+0000 7ff370bbe740 1 --2- 172.21.15.47:0/1244117664 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5590f9fd4f20 0x5590f9fda590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:30:48.721 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:48.718+0000 7ff370bbe740 1 -- 172.21.15.47:0/1244117664 >> 172.21.15.47:0/1244117664 conn(0x5590f9f9c200 msgr2=0x5590f9fce200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:30:48.721 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:48.718+0000 7ff370bbe740 1 -- 172.21.15.47:0/1244117664 shutdown_connections 2024-03-21T10:30:48.721 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:48.718+0000 7ff370bbe740 1 -- 172.21.15.47:0/1244117664 wait complete. 2024-03-21T10:30:48.721 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:48.718+0000 7ff370bbe740 1 -- start start 2024-03-21T10:30:48.721 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:48.719+0000 7ff370bbe740 1 Processor -- start 2024-03-21T10:30:48.721 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:48.719+0000 7ff370bbe740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5590f9fd4f20 0x5590f9fd74a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:30:48.721 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:48.719+0000 7ff370bbe740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5590f9fd5c00 0x5590f9fd79e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:30:48.721 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:48.719+0000 7ff370bbe740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5590f9fde290 0x5590f9fd7f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:30:48.722 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:48.719+0000 7ff370bbe740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3681b6230 con 0x5590f9fd4f20 2024-03-21T10:30:48.722 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:48.719+0000 7ff370bbe740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3681b5bd0 con 0x5590f9fd5c00 2024-03-21T10:30:48.722 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:48.719+0000 7ff370bbe740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3681b30d0 con 0x5590f9fde290 2024-03-21T10:30:48.722 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:48.719+0000 7ff370bbe740 10 client.0 _ll_register_callbacks cb 0x7ff3580bdb20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T10:30:48.722 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:48.719+0000 7ff366a00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5590f9fd4f20 0x5590f9fd74a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:30:48.722 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:48.719+0000 7ff366a00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5590f9fd4f20 0x5590f9fd74a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:46246/0 (socket says 172.21.15.47:46246) 2024-03-21T10:30:48.722 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:48.719+0000 7ff366a00640 1 -- 172.21.15.47:0/141765790 learned_addr learned my addr 172.21.15.47:0/141765790 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:30:48.722 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:48.720+0000 7ff35fe00640 1 --2- 172.21.15.47:0/141765790 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5590f9fd5c00 0x5590f9fd79e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:30:48.722 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:48.720+0000 7ff367400640 1 --2- 172.21.15.47:0/141765790 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5590f9fde290 0x5590f9fd7f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:30:48.722 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:48.720+0000 7ff366a00640 1 -- 172.21.15.47:0/141765790 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5590f9fde290 msgr2=0x5590f9fd7f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:30:48.722 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:48.720+0000 7ff366a00640 1 --2- 172.21.15.47:0/141765790 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5590f9fde290 0x5590f9fd7f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:30:48.722 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:48.720+0000 7ff366a00640 1 -- 172.21.15.47:0/141765790 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5590f9fd5c00 msgr2=0x5590f9fd79e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:30:48.722 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:48.720+0000 7ff366a00640 1 --2- 172.21.15.47:0/141765790 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5590f9fd5c00 0x5590f9fd79e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:30:48.722 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:48.720+0000 7ff366a00640 1 -- 172.21.15.47:0/141765790 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5590fa048a90 con 0x5590f9fd4f20 2024-03-21T10:30:48.723 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:48.720+0000 7ff35fe00640 1 --2- 172.21.15.47:0/141765790 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5590f9fd5c00 0x5590f9fd79e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T10:30:48.723 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:48.720+0000 7ff366a00640 1 --2- 172.21.15.47:0/141765790 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5590f9fd4f20 0x5590f9fd74a0 secure :-1 s=READY pgs=891 cs=0 l=1 rev1=1 crypto rx=0x7ff3580057c0 tx=0x7ff358009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f87ec59d33c8890f server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:30:48.723 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:48.720+0000 7ff354c00640 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-03-21T10:30:48.723 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:48.720+0000 7ff354c00640 1 -- 172.21.15.47:0/141765790 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3580c66d0 con 0x5590f9fd4f20 2024-03-21T10:30:48.723 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:48.720+0000 7ff354c00640 1 -- 172.21.15.47:0/141765790 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(97..97 src has 1..97) v4 ==== 9474+0+0 (secure 0 0 0) 0x7ff3580053d0 con 0x5590f9fd4f20 2024-03-21T10:30:48.723 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:48.720+0000 7ff370bbe740 1 -- 172.21.15.47:0/141765790 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ff350034070 con 0x5590f9fd4f20 2024-03-21T10:30:48.723 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:48.721+0000 7ff354c00640 1 -- 172.21.15.47:0/141765790 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff3580bd620 con 0x5590f9fd4f20 2024-03-21T10:30:48.723 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:48.721+0000 7ff370bbe740 10 client.17286 fetch_fsmap learned FSMap version 32 2024-03-21T10:30:48.723 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:48.721+0000 7ff370bbe740 1 -- 172.21.15.47:0/141765790 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x5590fa048a90 con 0x5590f9fd4f20 2024-03-21T10:30:48.723 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:48.721+0000 7ff354c00640 1 -- 172.21.15.47:0/141765790 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7ff350034070 con 0x5590f9fd4f20 2024-03-21T10:30:48.723 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:48.721+0000 7ff354c00640 1 -- 172.21.15.47:0/141765790 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 32) v1 ==== 7942+0+0 (secure 0 0 0) 0x7ff3580a6060 con 0x5590f9fd4f20 2024-03-21T10:30:48.723 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:48.721+0000 7ff370bbe740 10 client.17286 fetch_fsmap finished waiting for FSMap version 32 2024-03-21T10:30:48.723 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:48.721+0000 7ff370bbe740 10 client.17286 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-21T10:30:48.723 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:48.721+0000 7ff370bbe740 20 client.17286 populate_metadata read hostname 'smithi047' 2024-03-21T10:30:48.724 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:48.721+0000 7ff370bbe740 1 --2- 172.21.15.47:0/141765790 >> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] conn(0x5590fa04a180 0x5590fa04c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:30:48.724 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:48.721+0000 7ff370bbe740 4 client.17286 mds_command: new command op to 34535 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-21T10:30:48.724 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:48.721+0000 7ff370bbe740 1 -- 172.21.15.47:0/141765790 --> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x5590f9f40ae0 con 0x5590fa04a180 2024-03-21T10:30:48.725 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:48.722+0000 7ff35fe00640 1 --2- 172.21.15.47:0/141765790 >> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] conn(0x5590fa04a180 0x5590fa04c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:30:48.725 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:48.723+0000 7ff35fe00640 1 --2- 172.21.15.47:0/141765790 >> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] conn(0x5590fa04a180 0x5590fa04c5a0 crc :-1 s=READY pgs=146 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=394ff3f07af1531b server_cookie=d519b5f6418f27d5 in_seq=0 out_seq=0 2024-03-21T10:30:48.725 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:48.723+0000 7ff354c00640 10 client.17286 ms_handle_connect on v2:172.21.15.100:6838/4173168440 2024-03-21T10:30:48.725 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:48.723+0000 7ff354c00640 1 -- 172.21.15.47:0/141765790 <== mds.0 v2:172.21.15.100:6838/4173168440 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x5590f9f40ae0 con 0x5590fa04a180 2024-03-21T10:30:48.725 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:48.723+0000 7ff354c00640 10 client.17286 handle_command_reply: tid=0 2024-03-21T10:30:48.725 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:48.723+0000 7ff370bbe740 1 client.17286 shutdown 2024-03-21T10:30:48.725 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:48.723+0000 7ff370bbe740 1 -- 172.21.15.47:0/141765790 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5590f9fd4f20 msgr2=0x5590f9fd74a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:30:48.725 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:48.723+0000 7ff370bbe740 1 --2- 172.21.15.47:0/141765790 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5590f9fd4f20 0x5590f9fd74a0 secure :-1 s=READY pgs=891 cs=0 l=1 rev1=1 crypto rx=0x7ff3580057c0 tx=0x7ff358009260 comp rx=0 tx=0).stop 2024-03-21T10:30:48.726 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:48.723+0000 7ff370bbe740 1 -- 172.21.15.47:0/141765790 shutdown_connections 2024-03-21T10:30:48.726 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:48.723+0000 7ff370bbe740 1 --2- 172.21.15.47:0/141765790 >> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] conn(0x5590fa04a180 0x5590fa04c5a0 crc :-1 s=READY pgs=146 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:30:48.726 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:48.723+0000 7ff370bbe740 1 --2- 172.21.15.47:0/141765790 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5590f9fde290 0x5590f9fd7f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:30:48.726 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:48.723+0000 7ff370bbe740 1 --2- 172.21.15.47:0/141765790 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5590f9fd5c00 0x5590f9fd79e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:30:48.726 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:48.723+0000 7ff370bbe740 1 --2- 172.21.15.47:0/141765790 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5590f9fd4f20 0x5590f9fd74a0 unknown :-1 s=CLOSED pgs=891 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:30:48.726 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:48.723+0000 7ff354c00640 0 client.17286 ms_handle_reset on v2:172.21.15.100:6838/4173168440 2024-03-21T10:30:48.726 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:48.723+0000 7ff370bbe740 1 -- 172.21.15.47:0/141765790 >> 172.21.15.47:0/141765790 conn(0x5590f9f9c200 msgr2=0x5590f9fcea90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:30:48.726 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:48.723+0000 7ff370bbe740 1 -- 172.21.15.47:0/141765790 shutdown_connections 2024-03-21T10:30:48.726 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:48.723+0000 7ff370bbe740 1 -- 172.21.15.47:0/141765790 wait complete. 2024-03-21T10:30:48.726 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:48.724+0000 7ff370bbe740 20 client.17286 trim_cache size 0 max 16384 2024-03-21T10:30:48.726 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-03-21T10:30:48.727 INFO:teuthology.orchestra.run.smithi047.stdout: "status": "no active scrubs running", 2024-03-21T10:30:48.727 INFO:teuthology.orchestra.run.smithi047.stdout: "scrubs": {} 2024-03-21T10:30:48.727 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-03-21T10:30:48.727 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:48.724+0000 7ff36de00640 1 -- 172.21.15.47:0/1736683395 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7ff34c0a3110 msgr2=0x7ff34c0a55d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:30:48.727 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:48.724+0000 7ff36de00640 1 --2- 172.21.15.47:0/1736683395 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7ff34c0a3110 0x7ff34c0a55d0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7ff350005fd0 tx=0x7ff35000f730 comp rx=0 tx=0).stop 2024-03-21T10:30:48.728 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:48.724+0000 7ff36de00640 1 -- 172.21.15.47:0/1736683395 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff368100700 msgr2=0x7ff36819e5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:30:48.728 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:48.724+0000 7ff36de00640 1 --2- 172.21.15.47:0/1736683395 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff368100700 0x7ff36819e5b0 secure :-1 s=READY pgs=887 cs=0 l=1 rev1=1 crypto rx=0x7ff358036040 tx=0x7ff3580045e0 comp rx=0 tx=0).stop 2024-03-21T10:30:48.728 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:48.724+0000 7ff36de00640 1 -- 172.21.15.47:0/1736683395 shutdown_connections 2024-03-21T10:30:48.728 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:48.724+0000 7ff36de00640 1 --2- 172.21.15.47:0/1736683395 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff368105b00 0x7ff3681a2e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:30:48.728 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:48.724+0000 7ff36de00640 1 --2- 172.21.15.47:0/1736683395 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7ff34c0a3110 0x7ff34c0a55d0 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:30:48.728 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:48.724+0000 7ff36de00640 1 --2- 172.21.15.47:0/1736683395 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff368101020 0x7ff36819eaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:30:48.728 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:48.724+0000 7ff36de00640 1 --2- 172.21.15.47:0/1736683395 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff368100700 0x7ff36819e5b0 unknown :-1 s=CLOSED pgs=887 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:30:48.728 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:48.724+0000 7ff36de00640 1 -- 172.21.15.47:0/1736683395 >> 172.21.15.47:0/1736683395 conn(0x7ff368077700 msgr2=0x7ff3681034b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:30:48.728 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:48.724+0000 7ff36de00640 1 -- 172.21.15.47:0/1736683395 shutdown_connections 2024-03-21T10:30:48.728 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:48.724+0000 7ff36de00640 1 -- 172.21.15.47:0/1736683395 wait complete. 2024-03-21T10:30:48.740 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-03-21T10:30:48.741 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-21T10:30:48.946 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:48.946+0000 7ff559200640 1 -- 172.21.15.47:0/4221459577 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff554103850 msgr2=0x7ff554103cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:30:48.946 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:48.946+0000 7ff559200640 1 --2- 172.21.15.47:0/4221459577 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff554103850 0x7ff554103cd0 secure :-1 s=READY pgs=892 cs=0 l=1 rev1=1 crypto rx=0x7ff53c009930 tx=0x7ff53c02bc30 comp rx=0 tx=0).stop 2024-03-21T10:30:48.946 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:48.947+0000 7ff559200640 1 -- 172.21.15.47:0/4221459577 shutdown_connections 2024-03-21T10:30:48.946 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:48.947+0000 7ff559200640 1 --2- 172.21.15.47:0/4221459577 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff554104210 0x7ff554110720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:30:48.946 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:48.947+0000 7ff559200640 1 --2- 172.21.15.47:0/4221459577 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff554103850 0x7ff554103cd0 unknown :-1 s=CLOSED pgs=892 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:30:48.946 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:48.947+0000 7ff559200640 1 --2- 172.21.15.47:0/4221459577 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff554109860 0x7ff554109c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:30:48.947 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:48.947+0000 7ff559200640 1 -- 172.21.15.47:0/4221459577 >> 172.21.15.47:0/4221459577 conn(0x7ff5540ff2e0 msgr2=0x7ff554101700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:30:48.947 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:48.947+0000 7ff559200640 1 -- 172.21.15.47:0/4221459577 shutdown_connections 2024-03-21T10:30:48.947 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:48.947+0000 7ff559200640 1 -- 172.21.15.47:0/4221459577 wait complete. 2024-03-21T10:30:48.949 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:48.948+0000 7ff559200640 1 Processor -- start 2024-03-21T10:30:48.949 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:48.948+0000 7ff559200640 1 -- start start 2024-03-21T10:30:48.949 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:48.948+0000 7ff559200640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff554103850 0x7ff55419a090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:30:48.949 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:48.948+0000 7ff559200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff554104210 0x7ff55419a5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:30:48.949 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:48.948+0000 7ff559200640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff554109860 0x7ff55419e950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:30:48.949 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:48.948+0000 7ff559200640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff554113b70 con 0x7ff554103850 2024-03-21T10:30:48.949 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:48.948+0000 7ff559200640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5541139f0 con 0x7ff554104210 2024-03-21T10:30:48.949 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:48.948+0000 7ff559200640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff554113cf0 con 0x7ff554109860 2024-03-21T10:30:48.950 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:48.949+0000 7ff551e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff554103850 0x7ff55419a090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:30:48.950 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:48.949+0000 7ff551e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff554103850 0x7ff55419a090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:46268/0 (socket says 172.21.15.47:46268) 2024-03-21T10:30:48.950 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:48.949+0000 7ff551e00640 1 -- 172.21.15.47:0/1171619226 learned_addr learned my addr 172.21.15.47:0/1171619226 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:30:48.950 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:48.949+0000 7ff549400640 1 --2- 172.21.15.47:0/1171619226 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff554104210 0x7ff55419a5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:30:48.950 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:48.949+0000 7ff552800640 1 --2- 172.21.15.47:0/1171619226 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff554109860 0x7ff55419e950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:30:48.950 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:48.949+0000 7ff551e00640 1 -- 172.21.15.47:0/1171619226 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff554109860 msgr2=0x7ff55419e950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:30:48.950 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:48.949+0000 7ff551e00640 1 --2- 172.21.15.47:0/1171619226 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff554109860 0x7ff55419e950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:30:48.950 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:48.949+0000 7ff551e00640 1 -- 172.21.15.47:0/1171619226 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff554104210 msgr2=0x7ff55419a5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:30:48.950 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:48.949+0000 7ff551e00640 1 --2- 172.21.15.47:0/1171619226 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff554104210 0x7ff55419a5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:30:48.950 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:48.949+0000 7ff551e00640 1 -- 172.21.15.47:0/1171619226 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff55419f080 con 0x7ff554103850 2024-03-21T10:30:48.950 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:48.949+0000 7ff549400640 1 --2- 172.21.15.47:0/1171619226 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff554104210 0x7ff55419a5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T10:30:48.950 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:48.949+0000 7ff551e00640 1 --2- 172.21.15.47:0/1171619226 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff554103850 0x7ff55419a090 secure :-1 s=READY pgs=893 cs=0 l=1 rev1=1 crypto rx=0x7ff54400eab0 tx=0x7ff54400ef70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:30:48.950 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:48.950+0000 7ff54b400640 1 -- 172.21.15.47:0/1171619226 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff54400ce40 con 0x7ff554103850 2024-03-21T10:30:48.951 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:48.950+0000 7ff54b400640 1 -- 172.21.15.47:0/1171619226 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7ff544011920 con 0x7ff554103850 2024-03-21T10:30:48.951 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:48.950+0000 7ff559200640 1 -- 172.21.15.47:0/1171619226 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff55419f370 con 0x7ff554103850 2024-03-21T10:30:48.951 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:48.950+0000 7ff54b400640 1 -- 172.21.15.47:0/1171619226 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff544010a00 con 0x7ff554103850 2024-03-21T10:30:48.951 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:48.950+0000 7ff559200640 1 -- 172.21.15.47:0/1171619226 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff5541a6c00 con 0x7ff554103850 2024-03-21T10:30:48.951 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:48.951+0000 7ff559200640 1 -- 172.21.15.47:0/1171619226 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff554068490 con 0x7ff554103850 2024-03-21T10:30:48.952 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:48.953+0000 7ff54b400640 1 -- 172.21.15.47:0/1171619226 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 27) v1 ==== 137985+0+0 (secure 0 0 0) 0x7ff544011ad0 con 0x7ff554103850 2024-03-21T10:30:48.954 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:48.955+0000 7ff54b400640 1 --2- 172.21.15.47:0/1171619226 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7ff5380a3110 0x7ff5380a55d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:30:48.955 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:48.956+0000 7ff54b400640 1 -- 172.21.15.47:0/1171619226 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 9474+0+0 (secure 0 0 0) 0x7ff5440d0220 con 0x7ff554103850 2024-03-21T10:30:48.955 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:48.956+0000 7ff549400640 1 --2- 172.21.15.47:0/1171619226 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7ff5380a3110 0x7ff5380a55d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:30:48.955 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:48.957+0000 7ff549400640 1 --2- 172.21.15.47:0/1171619226 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7ff5380a3110 0x7ff5380a55d0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7ff53c009900 tx=0x7ff53c005fd0 comp rx=0 tx=0).ready entity=mgr.15687 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:30:48.959 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:48.960+0000 7ff54b400640 1 -- 172.21.15.47:0/1171619226 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7ff544016030 con 0x7ff554103850 2024-03-21T10:30:49.232 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:49.233+0000 7ff559200640 1 -- 172.21.15.47:0/1171619226 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff55419b3e0 con 0x7ff554103850 2024-03-21T10:30:49.233 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:49.235+0000 7ff54b400640 1 -- 172.21.15.47:0/1171619226 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 32 v32) v1 ==== 94+0+11471 (secure 0 0 0) 0x7ff54409c8f0 con 0x7ff554103850 2024-03-21T10:30:49.234 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-21T10:30:49.234 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":32,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14511,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6835/735724037","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6834","nonce":735724037},{"type":"v1","addr":"172.21.15.47:6835","nonce":735724037}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14529,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6837/2248191724","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6836","nonce":2248191724},{"type":"v1","addr":"172.21.15.47:6837","nonce":2248191724}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":32,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-21T10:21:58.329063+0000","modified":"2024-03-21T10:26:17.169892+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":34535,"mds_1":34493,"mds_2":34454,"mds_3":34427,"mds_4":14619},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14604":{"gid":14604,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.47:6839/2881335356","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6838","nonce":2881335356},{"type":"v1","addr":"172.21.15.47:6839","nonce":2881335356}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_14619":{"gid":14619,"name":"a","rank":4,"incarnation":27,"state":"up:active","state_seq":11,"addr":"172.21.15.47:6841/3401578978","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6840","nonce":3401578978},{"type":"v1","addr":"172.21.15.47:6841","nonce":3401578978}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24406":{"gid":24406,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.157:6833/4000010136","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":4000010136},{"type":"v1","addr":"172.21.15.157:6833","nonce":4000010136}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24457":{"gid":24457,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.157:6835/3902868651","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":3902868651},{"type":"v1","addr":"172.21.15.157:6835","nonce":3902868651}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24475":{"gid":24475,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.157:6837/1908087204","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6836","nonce":1908087204},{"type":"v1","addr":"172.21.15.157:6837","nonce":1908087204}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_25729":{"gid":25729,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.157:6839/1998408416","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6838","nonce":1998408416},{"type":"v1","addr":"172.21.15.157:6839","nonce":1998408416}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_34427":{"gid":34427,"name":"k","rank":3,"incarnation":25,"state":"up:active","state_seq":20,"addr":"172.21.15.100:6833/2723302038","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6832","nonce":2723302038},{"type":"v1","addr":"172.21.15.100:6833","nonce":2723302038}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_34454":{"gid":34454,"name":"e","rank":2,"incarnation":22,"state":"up:active","state_seq":17,"addr":"172.21.15.100:6835/2597849957","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":2597849957},{"type":"v1","addr":"172.21.15.100:6835","nonce":2597849957}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_34493":{"gid":34493,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":13,"addr":"172.21.15.100:6837/3497644162","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6836","nonce":3497644162},{"type":"v1","addr":"172.21.15.100:6837","nonce":3497644162}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_34535":{"gid":34535,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.100:6839/4173168440","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6838","nonce":4173168440},{"type":"v1","addr":"172.21.15.100:6839","nonce":4173168440}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":34535,"qdb_cluster":[34427,14619,34454,34535,34493]},"id":1}]} 2024-03-21T10:30:49.234 INFO:teuthology.orchestra.run.smithi047.stderr:dumped fsmap epoch 32 2024-03-21T10:30:49.236 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:49.237+0000 7ff559200640 1 -- 172.21.15.47:0/1171619226 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7ff5380a3110 msgr2=0x7ff5380a55d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:30:49.237 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:49.237+0000 7ff559200640 1 --2- 172.21.15.47:0/1171619226 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7ff5380a3110 0x7ff5380a55d0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7ff53c009900 tx=0x7ff53c005fd0 comp rx=0 tx=0).stop 2024-03-21T10:30:49.237 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:49.237+0000 7ff559200640 1 -- 172.21.15.47:0/1171619226 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff554103850 msgr2=0x7ff55419a090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:30:49.237 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:49.237+0000 7ff559200640 1 --2- 172.21.15.47:0/1171619226 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff554103850 0x7ff55419a090 secure :-1 s=READY pgs=893 cs=0 l=1 rev1=1 crypto rx=0x7ff54400eab0 tx=0x7ff54400ef70 comp rx=0 tx=0).stop 2024-03-21T10:30:49.237 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:49.237+0000 7ff559200640 1 -- 172.21.15.47:0/1171619226 shutdown_connections 2024-03-21T10:30:49.237 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:49.237+0000 7ff559200640 1 --2- 172.21.15.47:0/1171619226 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff554109860 0x7ff55419e950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:30:49.237 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:49.237+0000 7ff559200640 1 --2- 172.21.15.47:0/1171619226 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7ff5380a3110 0x7ff5380a55d0 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:30:49.237 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:49.237+0000 7ff559200640 1 --2- 172.21.15.47:0/1171619226 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff554104210 0x7ff55419a5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:30:49.237 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:49.237+0000 7ff559200640 1 --2- 172.21.15.47:0/1171619226 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff554103850 0x7ff55419a090 unknown :-1 s=CLOSED pgs=893 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:30:49.237 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:49.237+0000 7ff559200640 1 -- 172.21.15.47:0/1171619226 >> 172.21.15.47:0/1171619226 conn(0x7ff5540ff2e0 msgr2=0x7ff5540ffaa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:30:49.237 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:49.238+0000 7ff559200640 1 -- 172.21.15.47:0/1171619226 shutdown_connections 2024-03-21T10:30:49.237 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:49.238+0000 7ff559200640 1 -- 172.21.15.47:0/1171619226 wait complete. 2024-03-21T10:30:49.258 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-21T10:30:49.258 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-03-21T10:30:49.471 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:49.471+0000 7f6f23200640 1 -- 172.21.15.47:0/1547578882 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6f24100dc0 msgr2=0x7f6f2410d2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:30:49.471 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:49.471+0000 7f6f23200640 1 --2- 172.21.15.47:0/1547578882 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6f24100dc0 0x7f6f2410d2d0 secure :-1 s=READY pgs=894 cs=0 l=1 rev1=1 crypto rx=0x7f6f1c009cd0 tx=0x7f6f1c02c0e0 comp rx=0 tx=0).stop 2024-03-21T10:30:49.471 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:49.472+0000 7f6f23200640 1 -- 172.21.15.47:0/1547578882 shutdown_connections 2024-03-21T10:30:49.471 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:49.472+0000 7f6f23200640 1 --2- 172.21.15.47:0/1547578882 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6f24100dc0 0x7f6f2410d2d0 unknown :-1 s=CLOSED pgs=894 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:30:49.471 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:49.472+0000 7f6f23200640 1 --2- 172.21.15.47:0/1547578882 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6f24100400 0x7f6f24100880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:30:49.472 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:49.472+0000 7f6f23200640 1 --2- 172.21.15.47:0/1547578882 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6f241064a0 0x7f6f24106880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:30:49.472 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:49.472+0000 7f6f23200640 1 -- 172.21.15.47:0/1547578882 >> 172.21.15.47:0/1547578882 conn(0x7f6f240fbef0 msgr2=0x7f6f240fe310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:30:49.472 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:49.472+0000 7f6f23200640 1 -- 172.21.15.47:0/1547578882 shutdown_connections 2024-03-21T10:30:49.472 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:49.472+0000 7f6f23200640 1 -- 172.21.15.47:0/1547578882 wait complete. 2024-03-21T10:30:49.474 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:49.473+0000 7f6f23200640 1 Processor -- start 2024-03-21T10:30:49.474 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:49.473+0000 7f6f23200640 1 -- start start 2024-03-21T10:30:49.474 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:49.474+0000 7f6f23200640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6f24100400 0x7f6f24199f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:30:49.474 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:49.474+0000 7f6f23200640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6f24100dc0 0x7f6f2419a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:30:49.474 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:49.474+0000 7f6f23200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6f241064a0 0x7f6f2419e7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:30:49.474 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:49.474+0000 7f6f23200640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f24110660 con 0x7f6f24100dc0 2024-03-21T10:30:49.475 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:49.474+0000 7f6f23200640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f241104e0 con 0x7f6f241064a0 2024-03-21T10:30:49.475 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:49.474+0000 7f6f23200640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f241107e0 con 0x7f6f24100400 2024-03-21T10:30:49.475 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:49.474+0000 7f6f21400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6f24100dc0 0x7f6f2419a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:30:49.475 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:49.474+0000 7f6f21400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6f24100dc0 0x7f6f2419a480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:46290/0 (socket says 172.21.15.47:46290) 2024-03-21T10:30:49.475 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:49.474+0000 7f6f21400640 1 -- 172.21.15.47:0/905721577 learned_addr learned my addr 172.21.15.47:0/905721577 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:30:49.475 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:49.474+0000 7f6f22800640 1 --2- 172.21.15.47:0/905721577 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6f241064a0 0x7f6f2419e7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:30:49.475 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:49.474+0000 7f6f21e00640 1 --2- 172.21.15.47:0/905721577 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6f24100400 0x7f6f24199f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:30:49.475 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:49.474+0000 7f6f21400640 1 -- 172.21.15.47:0/905721577 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6f24100400 msgr2=0x7f6f24199f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:30:49.475 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:49.474+0000 7f6f21400640 1 --2- 172.21.15.47:0/905721577 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6f24100400 0x7f6f24199f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:30:49.475 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:49.474+0000 7f6f21400640 1 -- 172.21.15.47:0/905721577 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6f241064a0 msgr2=0x7f6f2419e7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:30:49.475 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:49.474+0000 7f6f21400640 1 --2- 172.21.15.47:0/905721577 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6f241064a0 0x7f6f2419e7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:30:49.475 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:49.474+0000 7f6f21400640 1 -- 172.21.15.47:0/905721577 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f2419eee0 con 0x7f6f24100dc0 2024-03-21T10:30:49.475 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:49.475+0000 7f6f21400640 1 --2- 172.21.15.47:0/905721577 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6f24100dc0 0x7f6f2419a480 secure :-1 s=READY pgs=895 cs=0 l=1 rev1=1 crypto rx=0x7f6f1000c890 tx=0x7f6f1000cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:30:49.476 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:49.475+0000 7f6f1aa00640 1 -- 172.21.15.47:0/905721577 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f10004220 con 0x7f6f24100dc0 2024-03-21T10:30:49.476 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:49.475+0000 7f6f1aa00640 1 -- 172.21.15.47:0/905721577 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f6f100043c0 con 0x7f6f24100dc0 2024-03-21T10:30:49.476 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:49.475+0000 7f6f23200640 1 -- 172.21.15.47:0/905721577 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6f2419f1d0 con 0x7f6f24100dc0 2024-03-21T10:30:49.476 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:49.475+0000 7f6f1aa00640 1 -- 172.21.15.47:0/905721577 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f10004a00 con 0x7f6f24100dc0 2024-03-21T10:30:49.476 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:49.475+0000 7f6f23200640 1 -- 172.21.15.47:0/905721577 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6f241a6a60 con 0x7f6f24100dc0 2024-03-21T10:30:49.478 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:49.479+0000 7f6f2ba4f740 1 Processor -- start 2024-03-21T10:30:49.478 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:49.479+0000 7f6f2ba4f740 1 -- start start 2024-03-21T10:30:49.479 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:49.479+0000 7f6f2ba4f740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560a959ca8b0 0x560a959cac90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:30:49.479 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:49.479+0000 7f6f2ba4f740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x560a959cb260 0x560a959d3810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:30:49.479 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:49.479+0000 7f6f2ba4f740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560a959d3de0 0x560a959d6230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:30:49.479 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:49.479+0000 7f6f2ba4f740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x560a9592e2c0 con 0x560a959cb260 2024-03-21T10:30:49.479 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:49.479+0000 7f6f2ba4f740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x560a958d6c90 con 0x560a959ca8b0 2024-03-21T10:30:49.479 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:49.479+0000 7f6f2ba4f740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x560a9592a360 con 0x560a959d3de0 2024-03-21T10:30:49.479 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:49.480+0000 7f6f21e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x560a959cb260 0x560a959d3810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:30:49.480 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:49.480+0000 7f6f21e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x560a959cb260 0x560a959d3810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:46302/0 (socket says 172.21.15.47:46302) 2024-03-21T10:30:49.480 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:49.480+0000 7f6f22800640 1 --2- 172.21.15.47:0/3975996264 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560a959ca8b0 0x560a959cac90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:30:49.480 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:49.480+0000 7f6f21e00640 1 -- 172.21.15.47:0/3975996264 learned_addr learned my addr 172.21.15.47:0/3975996264 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:30:49.480 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:49.480+0000 7f6f21e00640 1 -- 172.21.15.47:0/3975996264 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560a959d3de0 msgr2=0x560a959d6230 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-21T10:30:49.480 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:49.480+0000 7f6f21e00640 1 --2- 172.21.15.47:0/3975996264 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560a959d3de0 0x560a959d6230 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:30:49.480 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:49.480+0000 7f6f21e00640 1 -- 172.21.15.47:0/3975996264 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560a959ca8b0 msgr2=0x560a959cac90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:30:49.480 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:49.480+0000 7f6f21e00640 1 --2- 172.21.15.47:0/3975996264 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560a959ca8b0 0x560a959cac90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:30:49.480 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:49.480+0000 7f6f21e00640 1 -- 172.21.15.47:0/3975996264 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560a958fb080 con 0x560a959cb260 2024-03-21T10:30:49.481 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:49.481+0000 7f6f21e00640 1 --2- 172.21.15.47:0/3975996264 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x560a959cb260 0x560a959d3810 secure :-1 s=READY pgs=896 cs=0 l=1 rev1=1 crypto rx=0x7f6f1400bc60 tx=0x7f6f1400f730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9c86ab73a8ac57a2 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:30:49.481 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:49.481+0000 7f6f1aa00640 1 -- 172.21.15.47:0/905721577 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 27) v1 ==== 137985+0+0 (secure 0 0 0) 0x7f6f10004ba0 con 0x7f6f24100dc0 2024-03-21T10:30:49.482 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:49.481+0000 7f6f0f400640 1 -- 172.21.15.47:0/3975996264 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f14015070 con 0x560a959cb260 2024-03-21T10:30:49.482 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:49.481+0000 7f6f0f400640 1 -- 172.21.15.47:0/3975996264 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f6f140118f0 con 0x560a959cb260 2024-03-21T10:30:49.482 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:49.481+0000 7f6f0f400640 1 -- 172.21.15.47:0/3975996264 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f14010830 con 0x560a959cb260 2024-03-21T10:30:49.482 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:49.482+0000 7f6f2ba4f740 1 -- 172.21.15.47:0/3975996264 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x560a959cb260 msgr2=0x560a959d3810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:30:49.482 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:49.482+0000 7f6f2ba4f740 1 --2- 172.21.15.47:0/3975996264 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x560a959cb260 0x560a959d3810 secure :-1 s=READY pgs=896 cs=0 l=1 rev1=1 crypto rx=0x7f6f1400bc60 tx=0x7f6f1400f730 comp rx=0 tx=0).stop 2024-03-21T10:30:49.482 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:49.482+0000 7f6f2ba4f740 1 -- 172.21.15.47:0/3975996264 shutdown_connections 2024-03-21T10:30:49.482 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:49.482+0000 7f6f2ba4f740 1 --2- 172.21.15.47:0/3975996264 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560a959d3de0 0x560a959d6230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:30:49.482 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:49.482+0000 7f6f2ba4f740 1 --2- 172.21.15.47:0/3975996264 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x560a959cb260 0x560a959d3810 unknown :-1 s=CLOSED pgs=896 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:30:49.482 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:49.482+0000 7f6f2ba4f740 1 --2- 172.21.15.47:0/3975996264 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560a959ca8b0 0x560a959cac90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:30:49.482 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:49.482+0000 7f6f2ba4f740 1 -- 172.21.15.47:0/3975996264 >> 172.21.15.47:0/3975996264 conn(0x560a959c4f80 msgr2=0x560a959c5360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:30:49.482 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:49.482+0000 7f6f1aa00640 1 --2- 172.21.15.47:0/905721577 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f6f080a3090 0x7f6f080a5550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:30:49.482 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:49.482+0000 7f6f2ba4f740 1 -- 172.21.15.47:0/3975996264 shutdown_connections 2024-03-21T10:30:49.483 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:49.483+0000 7f6f2ba4f740 1 -- 172.21.15.47:0/3975996264 wait complete. 2024-03-21T10:30:49.483 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:49.483+0000 7f6f1aa00640 1 -- 172.21.15.47:0/905721577 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 9474+0+0 (secure 0 0 0) 0x7f6f100cf930 con 0x7f6f24100dc0 2024-03-21T10:30:49.483 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:49.483+0000 7f6f21e00640 1 --2- 172.21.15.47:0/905721577 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f6f080a3090 0x7f6f080a5550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:30:49.486 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:49.483+0000 7f6f2ba4f740 1 -- start start 2024-03-21T10:30:49.486 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:49.483+0000 7f6f2ba4f740 1 Processor -- start 2024-03-21T10:30:49.486 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:49.484+0000 7f6f21e00640 1 --2- 172.21.15.47:0/905721577 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f6f080a3090 0x7f6f080a5550 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f6f1400bc30 tx=0x7f6f14005f30 comp rx=0 tx=0).ready entity=mgr.15687 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:30:49.486 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:49.484+0000 7f6f2ba4f740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x560a959ca8b0 0x560a959d36e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:30:49.486 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:49.484+0000 7f6f2ba4f740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560a959cb260 0x560a959d0a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:30:49.486 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:49.484+0000 7f6f2ba4f740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560a959d3de0 0x560a959d0ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:30:49.486 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:49.484+0000 7f6f2ba4f740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x560a959d6d50 con 0x560a959ca8b0 2024-03-21T10:30:49.486 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:49.484+0000 7f6f2ba4f740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x560a958ea0d0 con 0x560a959d3de0 2024-03-21T10:30:49.486 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:49.484+0000 7f6f2ba4f740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x560a959d6ed0 con 0x560a959cb260 2024-03-21T10:30:49.487 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:49.484+0000 7f6f2ba4f740 10 client.0 _ll_register_callbacks cb 0x560a958eee80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T10:30:49.487 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:49.484+0000 7f6f21e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x560a959ca8b0 0x560a959d36e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:30:49.487 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:49.484+0000 7f6f21e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x560a959ca8b0 0x560a959d36e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:46306/0 (socket says 172.21.15.47:46306) 2024-03-21T10:30:49.487 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:49.484+0000 7f6f21e00640 1 -- 172.21.15.47:0/3805101718 learned_addr learned my addr 172.21.15.47:0/3805101718 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:30:49.487 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:49.485+0000 7f6f22800640 1 --2- 172.21.15.47:0/3805101718 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560a959d3de0 0x560a959d0ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:30:49.487 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:49.485+0000 7f6f21400640 1 --2- 172.21.15.47:0/3805101718 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560a959cb260 0x560a959d0a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:30:49.487 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:49.485+0000 7f6f21e00640 1 -- 172.21.15.47:0/3805101718 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560a959cb260 msgr2=0x560a959d0a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:30:49.487 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:49.485+0000 7f6f21e00640 1 --2- 172.21.15.47:0/3805101718 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560a959cb260 0x560a959d0a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:30:49.487 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:49.485+0000 7f6f21e00640 1 -- 172.21.15.47:0/3805101718 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560a959d3de0 msgr2=0x560a959d0ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:30:49.487 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:49.485+0000 7f6f21e00640 1 --2- 172.21.15.47:0/3805101718 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560a959d3de0 0x560a959d0ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:30:49.487 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:49.485+0000 7f6f21e00640 1 -- 172.21.15.47:0/3805101718 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x560a956423f0 con 0x560a959ca8b0 2024-03-21T10:30:49.487 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:49.485+0000 7f6f22800640 1 --2- 172.21.15.47:0/3805101718 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560a959d3de0 0x560a959d0ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T10:30:49.487 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:49.485+0000 7f6f21e00640 1 --2- 172.21.15.47:0/3805101718 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x560a959ca8b0 0x560a959d36e0 secure :-1 s=READY pgs=897 cs=0 l=1 rev1=1 crypto rx=0x560a959ca380 tx=0x7f6f1400f730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b04c8fb323ace4c5 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:30:49.488 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:49.485+0000 7f6f03e00640 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-03-21T10:30:49.488 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:49.485+0000 7f6f03e00640 1 -- 172.21.15.47:0/3805101718 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f14035060 con 0x560a959ca8b0 2024-03-21T10:30:49.488 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:49.486+0000 7f6f03e00640 1 -- 172.21.15.47:0/3805101718 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(97..97 src has 1..97) v4 ==== 9474+0+0 (secure 0 0 0) 0x7f6f14010570 con 0x560a959ca8b0 2024-03-21T10:30:49.488 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:49.486+0000 7f6f2ba4f740 1 -- 172.21.15.47:0/3805101718 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f6f140118f0 con 0x560a959ca8b0 2024-03-21T10:30:49.488 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:49.486+0000 7f6f03e00640 1 -- 172.21.15.47:0/3805101718 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6f14015040 con 0x560a959ca8b0 2024-03-21T10:30:49.488 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:49.486+0000 7f6f2ba4f740 10 client.17304 fetch_fsmap learned FSMap version 32 2024-03-21T10:30:49.488 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:49.486+0000 7f6f2ba4f740 1 -- 172.21.15.47:0/3805101718 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x560a9570c7d0 con 0x560a959ca8b0 2024-03-21T10:30:49.488 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:49.486+0000 7f6f03e00640 1 -- 172.21.15.47:0/3805101718 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f6f140118f0 con 0x560a959ca8b0 2024-03-21T10:30:49.488 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:49.486+0000 7f6f03e00640 1 -- 172.21.15.47:0/3805101718 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 32) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f6f14030850 con 0x560a959ca8b0 2024-03-21T10:30:49.488 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:49.486+0000 7f6f2ba4f740 10 client.17304 fetch_fsmap finished waiting for FSMap version 32 2024-03-21T10:30:49.488 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:49.486+0000 7f6f2ba4f740 10 client.17304 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.a 2024-03-21T10:30:49.488 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:49.487+0000 7f6f2ba4f740 20 client.17304 populate_metadata read hostname 'smithi047' 2024-03-21T10:30:49.488 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:49.487+0000 7f6f2ba4f740 1 --2- 172.21.15.47:0/3805101718 >> [v2:172.21.15.47:6840/3401578978,v1:172.21.15.47:6841/3401578978] conn(0x560a959e3cd0 0x560a959e60c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:30:49.488 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:49.487+0000 7f6f2ba4f740 4 client.17304 mds_command: new command op to 14619 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T10:30:49.489 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:49.487+0000 7f6f2ba4f740 1 -- 172.21.15.47:0/3805101718 --> [v2:172.21.15.47:6840/3401578978,v1:172.21.15.47:6841/3401578978] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x560a9593b800 con 0x560a959e3cd0 2024-03-21T10:30:49.489 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:49.487+0000 7f6f21400640 1 --2- 172.21.15.47:0/3805101718 >> [v2:172.21.15.47:6840/3401578978,v1:172.21.15.47:6841/3401578978] conn(0x560a959e3cd0 0x560a959e60c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:30:49.490 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:49.488+0000 7f6f21400640 1 --2- 172.21.15.47:0/3805101718 >> [v2:172.21.15.47:6840/3401578978,v1:172.21.15.47:6841/3401578978] conn(0x560a959e3cd0 0x560a959e60c0 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=9ed3f582dfe96f54 server_cookie=edb3cddab854169e in_seq=0 out_seq=0 2024-03-21T10:30:49.490 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:49.488+0000 7f6f03e00640 10 client.17304 ms_handle_connect on v2:172.21.15.47:6840/3401578978 2024-03-21T10:30:49.490 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:49.489+0000 7f6f03e00640 1 -- 172.21.15.47:0/3805101718 <== mds.4 v2:172.21.15.47:6840/3401578978 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x560a9593b800 con 0x560a959e3cd0 2024-03-21T10:30:49.490 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:49.489+0000 7f6f03e00640 10 client.17304 handle_command_reply: tid=0 2024-03-21T10:30:49.490 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:49.489+0000 7f6f2ba4f740 1 client.17304 shutdown 2024-03-21T10:30:49.490 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:49.489+0000 7f6f2ba4f740 1 -- 172.21.15.47:0/3805101718 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x560a959ca8b0 msgr2=0x560a959d36e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:30:49.490 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:49.489+0000 7f6f2ba4f740 1 --2- 172.21.15.47:0/3805101718 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x560a959ca8b0 0x560a959d36e0 secure :-1 s=READY pgs=897 cs=0 l=1 rev1=1 crypto rx=0x560a959ca380 tx=0x7f6f1400f730 comp rx=0 tx=0).stop 2024-03-21T10:30:49.490 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:49.489+0000 7f6f2ba4f740 1 -- 172.21.15.47:0/3805101718 shutdown_connections 2024-03-21T10:30:49.490 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:49.489+0000 7f6f2ba4f740 1 --2- 172.21.15.47:0/3805101718 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560a959d3de0 0x560a959d0ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:30:49.490 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:49.489+0000 7f6f2ba4f740 1 --2- 172.21.15.47:0/3805101718 >> [v2:172.21.15.47:6840/3401578978,v1:172.21.15.47:6841/3401578978] conn(0x560a959e3cd0 0x560a959e60c0 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:30:49.491 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:49.489+0000 7f6f2ba4f740 1 --2- 172.21.15.47:0/3805101718 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560a959cb260 0x560a959d0a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:30:49.491 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:49.489+0000 7f6f2ba4f740 1 --2- 172.21.15.47:0/3805101718 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x560a959ca8b0 0x560a959d36e0 unknown :-1 s=CLOSED pgs=897 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:30:49.491 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:49.489+0000 7f6f03e00640 0 client.17304 ms_handle_reset on v2:172.21.15.47:6840/3401578978 2024-03-21T10:30:49.491 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:49.489+0000 7f6f2ba4f740 1 -- 172.21.15.47:0/3805101718 >> 172.21.15.47:0/3805101718 conn(0x560a959c4f80 msgr2=0x560a959c6cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:30:49.491 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:49.489+0000 7f6f2ba4f740 1 -- 172.21.15.47:0/3805101718 shutdown_connections 2024-03-21T10:30:49.491 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:49.489+0000 7f6f2ba4f740 1 -- 172.21.15.47:0/3805101718 wait complete. 2024-03-21T10:30:49.491 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:49.489+0000 7f6f2ba4f740 20 client.17304 trim_cache size 0 max 16384 2024-03-21T10:30:49.516 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:49.517+0000 7f6f2ba4f740 1 Processor -- start 2024-03-21T10:30:49.517 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:49.517+0000 7f6f2ba4f740 1 -- start start 2024-03-21T10:30:49.517 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:49.517+0000 7f6f2ba4f740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560a959ca8b0 0x560a959c6a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:30:49.517 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:49.517+0000 7f6f2ba4f740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x560a959cb260 0x560a959d31a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:30:49.517 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:49.518+0000 7f6f2ba4f740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560a959d3de0 0x560a959d36e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:30:49.517 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:49.518+0000 7f6f2ba4f740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f241b09a0 con 0x560a959cb260 2024-03-21T10:30:49.517 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:49.518+0000 7f6f2ba4f740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f241b2320 con 0x560a959d3de0 2024-03-21T10:30:49.517 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:49.518+0000 7f6f2ba4f740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f241b1000 con 0x560a959ca8b0 2024-03-21T10:30:49.519 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:49.518+0000 7f6f21400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x560a959cb260 0x560a959d31a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:30:49.519 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:49.518+0000 7f6f21400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x560a959cb260 0x560a959d31a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:46318/0 (socket says 172.21.15.47:46318) 2024-03-21T10:30:49.519 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:49.518+0000 7f6f21400640 1 -- 172.21.15.47:0/1126525494 learned_addr learned my addr 172.21.15.47:0/1126525494 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:30:49.519 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:49.518+0000 7f6f22800640 1 --2- 172.21.15.47:0/1126525494 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560a959d3de0 0x560a959d36e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:30:49.519 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:49.518+0000 7f6f21e00640 1 --2- 172.21.15.47:0/1126525494 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560a959ca8b0 0x560a959c6a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:30:49.519 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:49.518+0000 7f6f21400640 1 -- 172.21.15.47:0/1126525494 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560a959ca8b0 msgr2=0x560a959c6a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:30:49.519 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:49.518+0000 7f6f21400640 1 --2- 172.21.15.47:0/1126525494 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560a959ca8b0 0x560a959c6a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:30:49.519 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:49.518+0000 7f6f21400640 1 -- 172.21.15.47:0/1126525494 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560a959d3de0 msgr2=0x560a959d36e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:30:49.519 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:49.518+0000 7f6f21400640 1 --2- 172.21.15.47:0/1126525494 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560a959d3de0 0x560a959d36e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:30:49.519 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:49.518+0000 7f6f21400640 1 -- 172.21.15.47:0/1126525494 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560a959c4ab0 con 0x560a959cb260 2024-03-21T10:30:49.520 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:49.519+0000 7f6f21400640 1 --2- 172.21.15.47:0/1126525494 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x560a959cb260 0x560a959d31a0 secure :-1 s=READY pgs=898 cs=0 l=1 rev1=1 crypto rx=0x7f6f100ccd20 tx=0x7f6f10097dc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4612728331ef7157 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:30:49.520 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:49.519+0000 7f6f0ea00640 1 -- 172.21.15.47:0/1126525494 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f1001d040 con 0x560a959cb260 2024-03-21T10:30:49.520 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:49.519+0000 7f6f0ea00640 1 -- 172.21.15.47:0/1126525494 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f6f100a4370 con 0x560a959cb260 2024-03-21T10:30:49.520 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:49.519+0000 7f6f0ea00640 1 -- 172.21.15.47:0/1126525494 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f10097fd0 con 0x560a959cb260 2024-03-21T10:30:49.520 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:49.520+0000 7f6f2ba4f740 1 -- 172.21.15.47:0/1126525494 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x560a959cb260 msgr2=0x560a959d31a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:30:49.520 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:49.520+0000 7f6f2ba4f740 1 --2- 172.21.15.47:0/1126525494 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x560a959cb260 0x560a959d31a0 secure :-1 s=READY pgs=898 cs=0 l=1 rev1=1 crypto rx=0x7f6f100ccd20 tx=0x7f6f10097dc0 comp rx=0 tx=0).stop 2024-03-21T10:30:49.520 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:49.520+0000 7f6f2ba4f740 1 -- 172.21.15.47:0/1126525494 shutdown_connections 2024-03-21T10:30:49.520 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:49.520+0000 7f6f2ba4f740 1 --2- 172.21.15.47:0/1126525494 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560a959d3de0 0x560a959d36e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:30:49.520 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:49.520+0000 7f6f2ba4f740 1 --2- 172.21.15.47:0/1126525494 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x560a959cb260 0x560a959d31a0 unknown :-1 s=CLOSED pgs=898 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:30:49.520 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:49.520+0000 7f6f2ba4f740 1 --2- 172.21.15.47:0/1126525494 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560a959ca8b0 0x560a959c6a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:30:49.520 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:49.520+0000 7f6f2ba4f740 1 -- 172.21.15.47:0/1126525494 >> 172.21.15.47:0/1126525494 conn(0x560a959c4f80 msgr2=0x560a959c62f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:30:49.520 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:49.520+0000 7f6f2ba4f740 1 -- 172.21.15.47:0/1126525494 shutdown_connections 2024-03-21T10:30:49.520 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:49.520+0000 7f6f2ba4f740 1 -- 172.21.15.47:0/1126525494 wait complete. 2024-03-21T10:30:49.524 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:49.520+0000 7f6f2ba4f740 1 -- start start 2024-03-21T10:30:49.524 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:49.521+0000 7f6f2ba4f740 1 Processor -- start 2024-03-21T10:30:49.524 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:49.521+0000 7f6f2ba4f740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560a959ca8b0 0x560a959d9950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:30:49.524 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:49.521+0000 7f6f2ba4f740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x560a959cb260 0x560a959d9e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:30:49.524 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:49.521+0000 7f6f2ba4f740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560a959d3de0 0x560a95a3e400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:30:49.524 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:49.521+0000 7f6f2ba4f740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f241b1cc0 con 0x560a959cb260 2024-03-21T10:30:49.524 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:49.521+0000 7f6f2ba4f740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f241b1660 con 0x560a959d3de0 2024-03-21T10:30:49.524 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:49.521+0000 7f6f2ba4f740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f241aeb60 con 0x560a959ca8b0 2024-03-21T10:30:49.525 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:49.521+0000 7f6f2ba4f740 10 client.0 _ll_register_callbacks cb 0x7f6f1c037070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T10:30:49.525 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:49.521+0000 7f6f21400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x560a959cb260 0x560a959d9e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:30:49.525 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:49.521+0000 7f6f21400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x560a959cb260 0x560a959d9e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:46330/0 (socket says 172.21.15.47:46330) 2024-03-21T10:30:49.525 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:49.521+0000 7f6f21400640 1 -- 172.21.15.47:0/3361996719 learned_addr learned my addr 172.21.15.47:0/3361996719 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:30:49.525 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:49.522+0000 7f6f21e00640 1 --2- 172.21.15.47:0/3361996719 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560a959ca8b0 0x560a959d9950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:30:49.525 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:49.522+0000 7f6f21400640 1 -- 172.21.15.47:0/3361996719 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560a959ca8b0 msgr2=0x560a959d9950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:30:49.525 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:49.522+0000 7f6f21400640 1 --2- 172.21.15.47:0/3361996719 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560a959ca8b0 0x560a959d9950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:30:49.525 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:49.522+0000 7f6f22800640 1 --2- 172.21.15.47:0/3361996719 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560a959d3de0 0x560a95a3e400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:30:49.525 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:49.522+0000 7f6f21400640 1 -- 172.21.15.47:0/3361996719 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560a959d3de0 msgr2=0x560a95a3e400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:30:49.525 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:49.522+0000 7f6f21400640 1 --2- 172.21.15.47:0/3361996719 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560a959d3de0 0x560a95a3e400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:30:49.525 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:49.522+0000 7f6f21400640 1 -- 172.21.15.47:0/3361996719 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x560a95a3e940 con 0x560a959cb260 2024-03-21T10:30:49.525 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:49.522+0000 7f6f21400640 1 --2- 172.21.15.47:0/3361996719 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x560a959cb260 0x560a959d9e90 secure :-1 s=READY pgs=899 cs=0 l=1 rev1=1 crypto rx=0x7f6f10004770 tx=0x7f6f10004f80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f097676305af12f server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:30:49.525 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:49.522+0000 7f6f03e00640 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-03-21T10:30:49.526 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:49.522+0000 7f6f03e00640 1 -- 172.21.15.47:0/3361996719 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f1001d030 con 0x560a959cb260 2024-03-21T10:30:49.526 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:49.522+0000 7f6f03e00640 1 -- 172.21.15.47:0/3361996719 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(97..97 src has 1..97) v4 ==== 9474+0+0 (secure 0 0 0) 0x7f6f10097fd0 con 0x560a959cb260 2024-03-21T10:30:49.526 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:49.522+0000 7f6f2ba4f740 1 -- 172.21.15.47:0/3361996719 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f6f100a4370 con 0x560a959cb260 2024-03-21T10:30:49.526 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:49.523+0000 7f6f03e00640 1 -- 172.21.15.47:0/3361996719 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6f10086030 con 0x560a959cb260 2024-03-21T10:30:49.526 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:49.523+0000 7f6f2ba4f740 10 client.17310 fetch_fsmap learned FSMap version 32 2024-03-21T10:30:49.526 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:49.523+0000 7f6f2ba4f740 1 -- 172.21.15.47:0/3361996719 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x560a95a3fa70 con 0x560a959cb260 2024-03-21T10:30:49.526 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:49.523+0000 7f6f03e00640 1 -- 172.21.15.47:0/3361996719 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f6f100a4370 con 0x560a959cb260 2024-03-21T10:30:49.526 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:49.523+0000 7f6f03e00640 1 -- 172.21.15.47:0/3361996719 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 32) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f6f10097740 con 0x560a959cb260 2024-03-21T10:30:49.526 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:49.523+0000 7f6f2ba4f740 10 client.17310 fetch_fsmap finished waiting for FSMap version 32 2024-03-21T10:30:49.526 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:49.523+0000 7f6f2ba4f740 10 client.17310 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.a 2024-03-21T10:30:49.526 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:49.523+0000 7f6f2ba4f740 20 client.17310 populate_metadata read hostname 'smithi047' 2024-03-21T10:30:49.526 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:49.523+0000 7f6f2ba4f740 1 --2- 172.21.15.47:0/3361996719 >> [v2:172.21.15.47:6840/3401578978,v1:172.21.15.47:6841/3401578978] conn(0x560a95a40180 0x560a95a425a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:30:49.526 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:49.523+0000 7f6f2ba4f740 4 client.17310 mds_command: new command op to 14619 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-21T10:30:49.526 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:49.523+0000 7f6f2ba4f740 1 -- 172.21.15.47:0/3361996719 --> [v2:172.21.15.47:6840/3401578978,v1:172.21.15.47:6841/3401578978] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x560a95935b50 con 0x560a95a40180 2024-03-21T10:30:49.527 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:49.524+0000 7f6f21e00640 1 --2- 172.21.15.47:0/3361996719 >> [v2:172.21.15.47:6840/3401578978,v1:172.21.15.47:6841/3401578978] conn(0x560a95a40180 0x560a95a425a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:30:49.527 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:49.524+0000 7f6f03e00640 10 client.17310 ms_handle_connect on v2:172.21.15.47:6840/3401578978 2024-03-21T10:30:49.527 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:49.524+0000 7f6f21e00640 1 --2- 172.21.15.47:0/3361996719 >> [v2:172.21.15.47:6840/3401578978,v1:172.21.15.47:6841/3401578978] conn(0x560a95a40180 0x560a95a425a0 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=fd90f24ff7e8d15b server_cookie=e3fd84b622353a9b in_seq=0 out_seq=0 2024-03-21T10:30:49.527 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:49.524+0000 7f6f03e00640 1 -- 172.21.15.47:0/3361996719 <== mds.4 v2:172.21.15.47:6840/3401578978 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x560a95935b50 con 0x560a95a40180 2024-03-21T10:30:49.527 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:49.524+0000 7f6f03e00640 10 client.17310 handle_command_reply: tid=0 2024-03-21T10:30:49.527 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:49.524+0000 7f6f2ba4f740 1 client.17310 shutdown 2024-03-21T10:30:49.527 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:49.525+0000 7f6f2ba4f740 1 -- 172.21.15.47:0/3361996719 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x560a959cb260 msgr2=0x560a959d9e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:30:49.527 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:49.525+0000 7f6f2ba4f740 1 --2- 172.21.15.47:0/3361996719 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x560a959cb260 0x560a959d9e90 secure :-1 s=READY pgs=899 cs=0 l=1 rev1=1 crypto rx=0x7f6f10004770 tx=0x7f6f10004f80 comp rx=0 tx=0).stop 2024-03-21T10:30:49.527 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:49.525+0000 7f6f2ba4f740 1 -- 172.21.15.47:0/3361996719 shutdown_connections 2024-03-21T10:30:49.527 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:49.525+0000 7f6f2ba4f740 1 --2- 172.21.15.47:0/3361996719 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560a959d3de0 0x560a95a3e400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:30:49.527 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:49.525+0000 7f6f2ba4f740 1 --2- 172.21.15.47:0/3361996719 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x560a959cb260 0x560a959d9e90 unknown :-1 s=CLOSED pgs=899 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:30:49.527 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:49.525+0000 7f6f2ba4f740 1 --2- 172.21.15.47:0/3361996719 >> [v2:172.21.15.47:6840/3401578978,v1:172.21.15.47:6841/3401578978] conn(0x560a95a40180 0x560a95a425a0 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:30:49.527 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:49.525+0000 7f6f2ba4f740 1 --2- 172.21.15.47:0/3361996719 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560a959ca8b0 0x560a959d9950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:30:49.527 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:49.525+0000 7f6f2ba4f740 1 -- 172.21.15.47:0/3361996719 >> 172.21.15.47:0/3361996719 conn(0x560a959c4f80 msgr2=0x560a959c62f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:30:49.528 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:49.525+0000 7f6f03e00640 0 client.17310 ms_handle_reset on v2:172.21.15.47:6840/3401578978 2024-03-21T10:30:49.528 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:49.525+0000 7f6f2ba4f740 1 -- 172.21.15.47:0/3361996719 shutdown_connections 2024-03-21T10:30:49.528 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:49.525+0000 7f6f2ba4f740 1 -- 172.21.15.47:0/3361996719 wait complete. 2024-03-21T10:30:49.528 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:49.525+0000 7f6f2ba4f740 20 client.17310 trim_cache size 0 max 16384 2024-03-21T10:30:49.528 INFO:teuthology.orchestra.run.smithi047.stdout:[] 2024-03-21T10:30:49.529 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:49.526+0000 7f6f23200640 1 -- 172.21.15.47:0/905721577 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f6f080a3090 msgr2=0x7f6f080a5550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:30:49.529 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:49.526+0000 7f6f23200640 1 --2- 172.21.15.47:0/905721577 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f6f080a3090 0x7f6f080a5550 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f6f1400bc30 tx=0x7f6f14005f30 comp rx=0 tx=0).stop 2024-03-21T10:30:49.529 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:49.526+0000 7f6f23200640 1 -- 172.21.15.47:0/905721577 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6f24100dc0 msgr2=0x7f6f2419a480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:30:49.529 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:49.526+0000 7f6f23200640 1 --2- 172.21.15.47:0/905721577 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6f24100dc0 0x7f6f2419a480 secure :-1 s=READY pgs=895 cs=0 l=1 rev1=1 crypto rx=0x7f6f1000c890 tx=0x7f6f1000cd50 comp rx=0 tx=0).stop 2024-03-21T10:30:49.529 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:49.526+0000 7f6f23200640 1 -- 172.21.15.47:0/905721577 shutdown_connections 2024-03-21T10:30:49.529 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:49.526+0000 7f6f23200640 1 --2- 172.21.15.47:0/905721577 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f6f080a3090 0x7f6f080a5550 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:30:49.529 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:49.526+0000 7f6f23200640 1 --2- 172.21.15.47:0/905721577 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6f241064a0 0x7f6f2419e7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:30:49.529 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:49.526+0000 7f6f23200640 1 --2- 172.21.15.47:0/905721577 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6f24100dc0 0x7f6f2419a480 unknown :-1 s=CLOSED pgs=895 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:30:49.529 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:49.526+0000 7f6f23200640 1 --2- 172.21.15.47:0/905721577 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6f24100400 0x7f6f24199f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:30:49.529 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:49.526+0000 7f6f23200640 1 -- 172.21.15.47:0/905721577 >> 172.21.15.47:0/905721577 conn(0x7f6f240fbef0 msgr2=0x7f6f2410c170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:30:49.530 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:49.526+0000 7f6f23200640 1 -- 172.21.15.47:0/905721577 shutdown_connections 2024-03-21T10:30:49.530 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:49.526+0000 7f6f23200640 1 -- 172.21.15.47:0/905721577 wait complete. 2024-03-21T10:30:49.541 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:30:49 smithi047 ceph-mon[31547]: pgmap v168: 129 pgs: 129 active+clean; 277 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-03-21T10:30:49.541 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:30:49 smithi047 ceph-mon[31547]: from='client.? 172.21.15.47:0/1171619226' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-21T10:30:49.545 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-21T10:30:49.545 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-03-21T10:30:49.728 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:30:49 smithi100 ceph-mon[35193]: pgmap v168: 129 pgs: 129 active+clean; 277 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-03-21T10:30:49.728 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:30:49 smithi100 ceph-mon[35193]: from='client.? 172.21.15.47:0/1171619226' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-21T10:30:49.737 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:30:49 smithi157 ceph-mon[35581]: pgmap v168: 129 pgs: 129 active+clean; 277 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-03-21T10:30:49.738 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:30:49 smithi157 ceph-mon[35581]: from='client.? 172.21.15.47:0/1171619226' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-21T10:30:49.760 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:49.760+0000 7f7b2e800640 1 -- 172.21.15.47:0/2526585558 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7b281041f0 msgr2=0x7f7b28110700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:30:49.760 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:49.760+0000 7f7b2e800640 1 --2- 172.21.15.47:0/2526585558 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7b281041f0 0x7f7b28110700 secure :-1 s=READY pgs=900 cs=0 l=1 rev1=1 crypto rx=0x7f7b20009cd0 tx=0x7f7b2002c0f0 comp rx=0 tx=0).stop 2024-03-21T10:30:49.760 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:49.760+0000 7f7b2e800640 1 -- 172.21.15.47:0/2526585558 shutdown_connections 2024-03-21T10:30:49.760 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:49.761+0000 7f7b2e800640 1 --2- 172.21.15.47:0/2526585558 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7b281041f0 0x7f7b28110700 unknown :-1 s=CLOSED pgs=900 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:30:49.760 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:49.761+0000 7f7b2e800640 1 --2- 172.21.15.47:0/2526585558 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7b28103830 0x7f7b28103cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:30:49.760 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:49.761+0000 7f7b2e800640 1 --2- 172.21.15.47:0/2526585558 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7b28109840 0x7f7b28109c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:30:49.760 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:49.761+0000 7f7b2e800640 1 -- 172.21.15.47:0/2526585558 >> 172.21.15.47:0/2526585558 conn(0x7f7b280ff320 msgr2=0x7f7b28101740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:30:49.760 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:49.761+0000 7f7b2e800640 1 -- 172.21.15.47:0/2526585558 shutdown_connections 2024-03-21T10:30:49.761 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:49.761+0000 7f7b2e800640 1 -- 172.21.15.47:0/2526585558 wait complete. 2024-03-21T10:30:49.763 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:49.762+0000 7f7b2e800640 1 Processor -- start 2024-03-21T10:30:49.763 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:49.762+0000 7f7b2e800640 1 -- start start 2024-03-21T10:30:49.763 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:49.762+0000 7f7b2e800640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7b28103830 0x7f7b2819a040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:30:49.763 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:49.762+0000 7f7b2e800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7b281041f0 0x7f7b2819a580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:30:49.763 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:49.762+0000 7f7b2e800640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7b28109840 0x7f7b2819e900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:30:49.763 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:49.762+0000 7f7b2e800640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b28113b00 con 0x7f7b28103830 2024-03-21T10:30:49.763 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:49.762+0000 7f7b2e800640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b28113980 con 0x7f7b281041f0 2024-03-21T10:30:49.763 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:49.762+0000 7f7b2e800640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b28113c80 con 0x7f7b28109840 2024-03-21T10:30:49.764 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:49.763+0000 7f7b27400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7b28103830 0x7f7b2819a040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:30:49.764 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:49.763+0000 7f7b27400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7b28103830 0x7f7b2819a040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:46358/0 (socket says 172.21.15.47:46358) 2024-03-21T10:30:49.764 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:49.763+0000 7f7b27400640 1 -- 172.21.15.47:0/2779379146 learned_addr learned my addr 172.21.15.47:0/2779379146 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:30:49.764 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:49.763+0000 7f7b1fe00640 1 --2- 172.21.15.47:0/2779379146 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7b281041f0 0x7f7b2819a580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:30:49.764 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:49.763+0000 7f7b27e00640 1 --2- 172.21.15.47:0/2779379146 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7b28109840 0x7f7b2819e900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:30:49.764 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:49.763+0000 7f7b27400640 1 -- 172.21.15.47:0/2779379146 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7b28109840 msgr2=0x7f7b2819e900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:30:49.764 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:49.763+0000 7f7b27400640 1 --2- 172.21.15.47:0/2779379146 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7b28109840 0x7f7b2819e900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:30:49.764 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:49.763+0000 7f7b27400640 1 -- 172.21.15.47:0/2779379146 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7b281041f0 msgr2=0x7f7b2819a580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:30:49.764 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:49.763+0000 7f7b27400640 1 --2- 172.21.15.47:0/2779379146 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7b281041f0 0x7f7b2819a580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:30:49.764 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:49.763+0000 7f7b27400640 1 -- 172.21.15.47:0/2779379146 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b2819f030 con 0x7f7b28103830 2024-03-21T10:30:49.764 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:49.764+0000 7f7b27400640 1 --2- 172.21.15.47:0/2779379146 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7b28103830 0x7f7b2819a040 secure :-1 s=READY pgs=901 cs=0 l=1 rev1=1 crypto rx=0x7f7b1800c890 tx=0x7f7b1800cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:30:49.764 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:49.764+0000 7f7b24c00640 1 -- 172.21.15.47:0/2779379146 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b18004220 con 0x7f7b28103830 2024-03-21T10:30:49.764 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:49.764+0000 7f7b2e800640 1 -- 172.21.15.47:0/2779379146 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7b2819f320 con 0x7f7b28103830 2024-03-21T10:30:49.764 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:49.764+0000 7f7b24c00640 1 -- 172.21.15.47:0/2779379146 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f7b180043c0 con 0x7f7b28103830 2024-03-21T10:30:49.764 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:49.764+0000 7f7b24c00640 1 -- 172.21.15.47:0/2779379146 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b18004a00 con 0x7f7b28103830 2024-03-21T10:30:49.765 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:49.764+0000 7f7b2e800640 1 -- 172.21.15.47:0/2779379146 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7b281a6bb0 con 0x7f7b28103830 2024-03-21T10:30:49.767 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:49.767+0000 7f7b31596740 1 Processor -- start 2024-03-21T10:30:49.767 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:49.768+0000 7f7b31596740 1 -- start start 2024-03-21T10:30:49.767 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:49.768+0000 7f7b24c00640 1 -- 172.21.15.47:0/2779379146 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 27) v1 ==== 137985+0+0 (secure 0 0 0) 0x7f7b18004ba0 con 0x7f7b28103830 2024-03-21T10:30:49.767 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:49.768+0000 7f7b31596740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5601718896d0 0x560171889ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:30:49.767 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:49.768+0000 7f7b31596740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56017188a080 0x560171892630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:30:49.767 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:49.768+0000 7f7b31596740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560171892c00 0x560171895050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:30:49.767 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:49.768+0000 7f7b31596740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x5601717ed2c0 con 0x5601718896d0 2024-03-21T10:30:49.767 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:49.768+0000 7f7b31596740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x560171795c90 con 0x560171892c00 2024-03-21T10:30:49.767 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:49.768+0000 7f7b31596740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x5601717e9360 con 0x56017188a080 2024-03-21T10:30:49.769 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:49.768+0000 7f7b27e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5601718896d0 0x560171889ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:30:49.769 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:49.768+0000 7f7b1fe00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560171892c00 0x560171895050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:30:49.769 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:49.768+0000 7f7b27e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5601718896d0 0x560171889ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:46370/0 (socket says 172.21.15.47:46370) 2024-03-21T10:30:49.769 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:49.768+0000 7f7b27e00640 1 -- 172.21.15.47:0/3141347432 learned_addr learned my addr 172.21.15.47:0/3141347432 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:30:49.769 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:49.769+0000 7f7b27400640 1 --2- 172.21.15.47:0/3141347432 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56017188a080 0x560171892630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:30:49.769 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:49.769+0000 7f7b27e00640 1 -- 172.21.15.47:0/3141347432 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56017188a080 msgr2=0x560171892630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:30:49.769 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:49.769+0000 7f7b27e00640 1 --2- 172.21.15.47:0/3141347432 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56017188a080 0x560171892630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:30:49.770 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:49.769+0000 7f7b27e00640 1 -- 172.21.15.47:0/3141347432 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560171892c00 msgr2=0x560171895050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:30:49.770 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:49.769+0000 7f7b27e00640 1 --2- 172.21.15.47:0/3141347432 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560171892c00 0x560171895050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:30:49.770 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:49.769+0000 7f7b27e00640 1 -- 172.21.15.47:0/3141347432 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5601717ba080 con 0x5601718896d0 2024-03-21T10:30:49.770 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:49.769+0000 7f7b27e00640 1 --2- 172.21.15.47:0/3141347432 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5601718896d0 0x560171889ab0 secure :-1 s=READY pgs=902 cs=0 l=1 rev1=1 crypto rx=0x7f7b20009cd0 tx=0x7f7b20009890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b0425896668560fe server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:30:49.770 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:49.770+0000 7f7b1cc00640 1 -- 172.21.15.47:0/3141347432 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b20043020 con 0x5601718896d0 2024-03-21T10:30:49.770 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:49.770+0000 7f7b24c00640 1 --2- 172.21.15.47:0/2779379146 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f7b0c0a3090 0x7f7b0c0a5550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:30:49.770 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:49.770+0000 7f7b1cc00640 1 -- 172.21.15.47:0/3141347432 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f7b20032e30 con 0x5601718896d0 2024-03-21T10:30:49.770 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:49.770+0000 7f7b1cc00640 1 -- 172.21.15.47:0/3141347432 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b20039040 con 0x5601718896d0 2024-03-21T10:30:49.770 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:49.770+0000 7f7b24c00640 1 -- 172.21.15.47:0/2779379146 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 9474+0+0 (secure 0 0 0) 0x7f7b180cf920 con 0x7f7b28103830 2024-03-21T10:30:49.770 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:49.770+0000 7f7b27400640 1 --2- 172.21.15.47:0/2779379146 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f7b0c0a3090 0x7f7b0c0a5550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:30:49.774 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:49.771+0000 7f7b31596740 1 -- 172.21.15.47:0/3141347432 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5601718896d0 msgr2=0x560171889ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:30:49.775 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:49.771+0000 7f7b31596740 1 --2- 172.21.15.47:0/3141347432 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5601718896d0 0x560171889ab0 secure :-1 s=READY pgs=902 cs=0 l=1 rev1=1 crypto rx=0x7f7b20009cd0 tx=0x7f7b20009890 comp rx=0 tx=0).stop 2024-03-21T10:30:49.775 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:49.771+0000 7f7b31596740 1 -- 172.21.15.47:0/3141347432 shutdown_connections 2024-03-21T10:30:49.775 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:49.771+0000 7f7b31596740 1 --2- 172.21.15.47:0/3141347432 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560171892c00 0x560171895050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:30:49.775 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:49.771+0000 7f7b31596740 1 --2- 172.21.15.47:0/3141347432 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56017188a080 0x560171892630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:30:49.775 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:49.771+0000 7f7b31596740 1 --2- 172.21.15.47:0/3141347432 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5601718896d0 0x560171889ab0 unknown :-1 s=CLOSED pgs=902 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:30:49.775 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:49.771+0000 7f7b27400640 1 --2- 172.21.15.47:0/2779379146 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f7b0c0a3090 0x7f7b0c0a5550 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x5601718891a0 tx=0x7f7b18000bc0 comp rx=0 tx=0).ready entity=mgr.15687 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:30:49.775 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:49.771+0000 7f7b31596740 1 -- 172.21.15.47:0/3141347432 >> 172.21.15.47:0/3141347432 conn(0x560171883da0 msgr2=0x560171884180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:30:49.776 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:49.771+0000 7f7b31596740 1 -- 172.21.15.47:0/3141347432 shutdown_connections 2024-03-21T10:30:49.776 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:49.771+0000 7f7b31596740 1 -- 172.21.15.47:0/3141347432 wait complete. 2024-03-21T10:30:49.776 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:49.771+0000 7f7b31596740 1 -- start start 2024-03-21T10:30:49.776 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:49.772+0000 7f7b31596740 1 Processor -- start 2024-03-21T10:30:49.776 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:49.772+0000 7f7b31596740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5601718896d0 0x5601718925a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:30:49.776 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:49.772+0000 7f7b31596740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56017188a080 0x56017188f930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:30:49.776 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:49.772+0000 7f7b31596740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560171892c00 0x560171890070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:30:49.776 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:49.772+0000 7f7b31596740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x560171895b10 con 0x5601718896d0 2024-03-21T10:30:49.776 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:49.772+0000 7f7b31596740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x5601717a90d0 con 0x56017188a080 2024-03-21T10:30:49.776 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:49.772+0000 7f7b31596740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x560171895c90 con 0x560171892c00 2024-03-21T10:30:49.777 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:49.772+0000 7f7b31596740 10 client.0 _ll_register_callbacks cb 0x5601717ade80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T10:30:49.777 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:49.772+0000 7f7b27e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5601718896d0 0x5601718925a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:30:49.777 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:49.773+0000 7f7b27e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5601718896d0 0x5601718925a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:46378/0 (socket says 172.21.15.47:46378) 2024-03-21T10:30:49.777 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:49.773+0000 7f7b27e00640 1 -- 172.21.15.47:0/3221548447 learned_addr learned my addr 172.21.15.47:0/3221548447 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:30:49.777 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:49.773+0000 7f7b1fe00640 1 --2- 172.21.15.47:0/3221548447 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56017188a080 0x56017188f930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:30:49.777 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:49.773+0000 7f7b27e00640 1 -- 172.21.15.47:0/3221548447 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560171892c00 msgr2=0x560171890070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:30:49.777 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:49.773+0000 7f7b27e00640 1 --2- 172.21.15.47:0/3221548447 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560171892c00 0x560171890070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:30:49.777 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:49.773+0000 7f7b27e00640 1 -- 172.21.15.47:0/3221548447 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56017188a080 msgr2=0x56017188f930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:30:49.777 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:49.773+0000 7f7b27e00640 1 --2- 172.21.15.47:0/3221548447 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56017188a080 0x56017188f930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:30:49.777 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:49.773+0000 7f7b27e00640 1 -- 172.21.15.47:0/3221548447 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5601715013f0 con 0x5601718896d0 2024-03-21T10:30:49.777 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:49.773+0000 7f7b27e00640 1 --2- 172.21.15.47:0/3221548447 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5601718896d0 0x5601718925a0 secure :-1 s=READY pgs=903 cs=0 l=1 rev1=1 crypto rx=0x7f7b20009ca0 tx=0x7f7b20003d20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=137fbd7600e07bc3 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:30:49.777 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:49.773+0000 7f7b15600640 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-03-21T10:30:49.778 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:49.773+0000 7f7b15600640 1 -- 172.21.15.47:0/3221548447 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b2004d020 con 0x5601718896d0 2024-03-21T10:30:49.778 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:49.774+0000 7f7b15600640 1 -- 172.21.15.47:0/3221548447 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(97..97 src has 1..97) v4 ==== 9474+0+0 (secure 0 0 0) 0x7f7b20032900 con 0x5601718896d0 2024-03-21T10:30:49.778 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:49.774+0000 7f7b31596740 1 -- 172.21.15.47:0/3221548447 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f7b20032e30 con 0x5601718896d0 2024-03-21T10:30:49.778 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:49.774+0000 7f7b15600640 1 -- 172.21.15.47:0/3221548447 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7b20058050 con 0x5601718896d0 2024-03-21T10:30:49.778 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:49.774+0000 7f7b31596740 10 client.17322 fetch_fsmap learned FSMap version 32 2024-03-21T10:30:49.778 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:49.774+0000 7f7b31596740 1 -- 172.21.15.47:0/3221548447 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x5601715cb7d0 con 0x5601718896d0 2024-03-21T10:30:49.778 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:49.774+0000 7f7b15600640 1 -- 172.21.15.47:0/3221548447 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f7b20032e30 con 0x5601718896d0 2024-03-21T10:30:49.778 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:49.774+0000 7f7b15600640 1 -- 172.21.15.47:0/3221548447 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 32) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f7b2003f6a0 con 0x5601718896d0 2024-03-21T10:30:49.778 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:49.774+0000 7f7b31596740 10 client.17322 fetch_fsmap finished waiting for FSMap version 32 2024-03-21T10:30:49.778 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:49.774+0000 7f7b31596740 10 client.17322 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.k 2024-03-21T10:30:49.778 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:49.774+0000 7f7b31596740 20 client.17322 populate_metadata read hostname 'smithi047' 2024-03-21T10:30:49.779 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:49.774+0000 7f7b31596740 1 --2- 172.21.15.47:0/3221548447 >> [v2:172.21.15.100:6832/2723302038,v1:172.21.15.100:6833/2723302038] conn(0x5601718a2920 0x5601718a4d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:30:49.779 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:49.774+0000 7f7b31596740 4 client.17322 mds_command: new command op to 34427 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T10:30:49.779 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:49.774+0000 7f7b31596740 1 -- 172.21.15.47:0/3221548447 --> [v2:172.21.15.100:6832/2723302038,v1:172.21.15.100:6833/2723302038] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5601717fa800 con 0x5601718a2920 2024-03-21T10:30:49.779 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:49.775+0000 7f7b1fe00640 1 --2- 172.21.15.47:0/3221548447 >> [v2:172.21.15.100:6832/2723302038,v1:172.21.15.100:6833/2723302038] conn(0x5601718a2920 0x5601718a4d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:30:49.779 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:49.776+0000 7f7b1fe00640 1 --2- 172.21.15.47:0/3221548447 >> [v2:172.21.15.100:6832/2723302038,v1:172.21.15.100:6833/2723302038] conn(0x5601718a2920 0x5601718a4d10 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=f723cde042b73409 server_cookie=113ee245a9f74bee in_seq=0 out_seq=0 2024-03-21T10:30:49.779 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:49.776+0000 7f7b15600640 10 client.17322 ms_handle_connect on v2:172.21.15.100:6832/2723302038 2024-03-21T10:30:49.780 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:49.777+0000 7f7b15600640 1 -- 172.21.15.47:0/3221548447 <== mds.3 v2:172.21.15.100:6832/2723302038 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x5601717fa800 con 0x5601718a2920 2024-03-21T10:30:49.781 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:49.777+0000 7f7b15600640 10 client.17322 handle_command_reply: tid=0 2024-03-21T10:30:49.781 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:49.777+0000 7f7b31596740 1 client.17322 shutdown 2024-03-21T10:30:49.781 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:49.777+0000 7f7b31596740 1 -- 172.21.15.47:0/3221548447 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5601718896d0 msgr2=0x5601718925a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:30:49.781 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:49.777+0000 7f7b31596740 1 --2- 172.21.15.47:0/3221548447 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5601718896d0 0x5601718925a0 secure :-1 s=READY pgs=903 cs=0 l=1 rev1=1 crypto rx=0x7f7b20009ca0 tx=0x7f7b20003d20 comp rx=0 tx=0).stop 2024-03-21T10:30:49.781 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:49.777+0000 7f7b31596740 1 -- 172.21.15.47:0/3221548447 shutdown_connections 2024-03-21T10:30:49.781 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:49.777+0000 7f7b31596740 1 --2- 172.21.15.47:0/3221548447 >> [v2:172.21.15.100:6832/2723302038,v1:172.21.15.100:6833/2723302038] conn(0x5601718a2920 0x5601718a4d10 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:30:49.781 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:49.777+0000 7f7b31596740 1 --2- 172.21.15.47:0/3221548447 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560171892c00 0x560171890070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:30:49.781 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:49.777+0000 7f7b31596740 1 --2- 172.21.15.47:0/3221548447 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56017188a080 0x56017188f930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:30:49.781 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:49.777+0000 7f7b31596740 1 --2- 172.21.15.47:0/3221548447 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5601718896d0 0x5601718925a0 unknown :-1 s=CLOSED pgs=903 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:30:49.781 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:49.777+0000 7f7b15600640 0 client.17322 ms_handle_reset on v2:172.21.15.100:6832/2723302038 2024-03-21T10:30:49.782 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:49.777+0000 7f7b31596740 1 -- 172.21.15.47:0/3221548447 >> 172.21.15.47:0/3221548447 conn(0x560171883da0 msgr2=0x560171885af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:30:49.782 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:49.777+0000 7f7b31596740 1 -- 172.21.15.47:0/3221548447 shutdown_connections 2024-03-21T10:30:49.782 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:49.777+0000 7f7b31596740 1 -- 172.21.15.47:0/3221548447 wait complete. 2024-03-21T10:30:49.782 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:49.778+0000 7f7b31596740 20 client.17322 trim_cache size 0 max 16384 2024-03-21T10:30:49.792 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:49.791+0000 7f7b31596740 1 Processor -- start 2024-03-21T10:30:49.792 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:49.791+0000 7f7b31596740 1 -- start start 2024-03-21T10:30:49.792 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:49.792+0000 7f7b31596740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5601718896d0 0x560171885ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:30:49.792 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:49.792+0000 7f7b31596740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56017188a080 0x560171892060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:30:49.792 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:49.792+0000 7f7b31596740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560171892c00 0x5601718925a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:30:49.792 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:49.792+0000 7f7b31596740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b281b0a90 con 0x56017188a080 2024-03-21T10:30:49.792 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:49.792+0000 7f7b31596740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b281b2410 con 0x5601718896d0 2024-03-21T10:30:49.792 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:49.792+0000 7f7b31596740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b281b10f0 con 0x560171892c00 2024-03-21T10:30:49.792 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:49.792+0000 7f7b1fe00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56017188a080 0x560171892060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:30:49.792 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:49.792+0000 7f7b27e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5601718896d0 0x560171885ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:30:49.792 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:49.792+0000 7f7b1fe00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56017188a080 0x560171892060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:46392/0 (socket says 172.21.15.47:46392) 2024-03-21T10:30:49.793 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:49.792+0000 7f7b1fe00640 1 -- 172.21.15.47:0/2089730660 learned_addr learned my addr 172.21.15.47:0/2089730660 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:30:49.793 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:49.792+0000 7f7b27e00640 1 --2- 172.21.15.47:0/2089730660 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560171892c00 0x5601718925a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:30:49.793 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:49.792+0000 7f7b27e00640 1 -- 172.21.15.47:0/2089730660 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560171892c00 msgr2=0x5601718925a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:30:49.793 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:49.792+0000 7f7b27e00640 1 --2- 172.21.15.47:0/2089730660 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560171892c00 0x5601718925a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:30:49.793 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:49.792+0000 7f7b27e00640 1 -- 172.21.15.47:0/2089730660 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56017188a080 msgr2=0x560171892060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:30:49.793 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:49.793+0000 7f7b27e00640 1 --2- 172.21.15.47:0/2089730660 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56017188a080 0x560171892060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:30:49.793 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:49.793+0000 7f7b27e00640 1 -- 172.21.15.47:0/2089730660 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56017188dea0 con 0x5601718896d0 2024-03-21T10:30:49.793 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:49.793+0000 7f7b1fe00640 1 --2- 172.21.15.47:0/2089730660 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56017188a080 0x560171892060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T10:30:49.793 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:49.793+0000 7f7b27e00640 1 --2- 172.21.15.47:0/2089730660 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5601718896d0 0x560171885ca0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f7b20003cf0 tx=0x7f7b20036040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2da45d6bcd51f756 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:30:49.797 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:49.793+0000 7f7b17e00640 1 -- 172.21.15.47:0/2089730660 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b2004d070 con 0x5601718896d0 2024-03-21T10:30:49.797 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:49.793+0000 7f7b17e00640 1 -- 172.21.15.47:0/2089730660 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f7b2003fc60 con 0x5601718896d0 2024-03-21T10:30:49.797 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:49.793+0000 7f7b17e00640 1 -- 172.21.15.47:0/2089730660 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b20043030 con 0x5601718896d0 2024-03-21T10:30:49.797 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:49.794+0000 7f7b31596740 1 -- 172.21.15.47:0/2089730660 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5601718896d0 msgr2=0x560171885ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:30:49.797 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:49.794+0000 7f7b31596740 1 --2- 172.21.15.47:0/2089730660 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5601718896d0 0x560171885ca0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f7b20003cf0 tx=0x7f7b20036040 comp rx=0 tx=0).stop 2024-03-21T10:30:49.798 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:49.794+0000 7f7b31596740 1 -- 172.21.15.47:0/2089730660 shutdown_connections 2024-03-21T10:30:49.798 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:49.794+0000 7f7b31596740 1 --2- 172.21.15.47:0/2089730660 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560171892c00 0x5601718925a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:30:49.798 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:49.794+0000 7f7b31596740 1 --2- 172.21.15.47:0/2089730660 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56017188a080 0x560171892060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:30:49.798 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:49.794+0000 7f7b31596740 1 --2- 172.21.15.47:0/2089730660 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5601718896d0 0x560171885ca0 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:30:49.798 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:49.794+0000 7f7b31596740 1 -- 172.21.15.47:0/2089730660 >> 172.21.15.47:0/2089730660 conn(0x560171883da0 msgr2=0x560171882eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:30:49.798 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:49.794+0000 7f7b31596740 1 -- 172.21.15.47:0/2089730660 shutdown_connections 2024-03-21T10:30:49.798 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:49.794+0000 7f7b31596740 1 -- 172.21.15.47:0/2089730660 wait complete. 2024-03-21T10:30:49.799 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:49.794+0000 7f7b31596740 1 -- start start 2024-03-21T10:30:49.799 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:49.795+0000 7f7b31596740 1 Processor -- start 2024-03-21T10:30:49.799 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:49.795+0000 7f7b31596740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5601718896d0 0x56017189ab20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:30:49.799 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:49.795+0000 7f7b31596740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56017188a080 0x560171898ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:30:49.799 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:49.795+0000 7f7b31596740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x560171892c00 0x560171899400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:30:49.799 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:49.795+0000 7f7b31596740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b281b1db0 con 0x560171892c00 2024-03-21T10:30:49.799 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:49.795+0000 7f7b31596740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b281b1750 con 0x56017188a080 2024-03-21T10:30:49.799 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:49.795+0000 7f7b31596740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b281aec50 con 0x5601718896d0 2024-03-21T10:30:49.799 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:49.795+0000 7f7b31596740 10 client.0 _ll_register_callbacks cb 0x7f7b1000b3a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T10:30:49.799 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:49.795+0000 7f7b27e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x560171892c00 0x560171899400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:30:49.799 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:49.795+0000 7f7b1fe00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56017188a080 0x560171898ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:30:49.800 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:49.795+0000 7f7b1fe00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56017188a080 0x560171898ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:39730/0 (socket says 172.21.15.47:39730) 2024-03-21T10:30:49.800 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:49.795+0000 7f7b1fe00640 1 -- 172.21.15.47:0/298347310 learned_addr learned my addr 172.21.15.47:0/298347310 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:30:49.800 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:49.796+0000 7f7b27e00640 1 --2- 172.21.15.47:0/298347310 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5601718896d0 0x56017189ab20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:30:49.800 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:49.796+0000 7f7b1fe00640 1 -- 172.21.15.47:0/298347310 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5601718896d0 msgr2=0x56017189ab20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:30:49.800 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:49.796+0000 7f7b1fe00640 1 --2- 172.21.15.47:0/298347310 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5601718896d0 0x56017189ab20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:30:49.800 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:49.796+0000 7f7b1fe00640 1 -- 172.21.15.47:0/298347310 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x560171892c00 msgr2=0x560171899400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:30:49.800 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:49.796+0000 7f7b1fe00640 1 --2- 172.21.15.47:0/298347310 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x560171892c00 0x560171899400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:30:49.800 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:49.796+0000 7f7b1fe00640 1 -- 172.21.15.47:0/298347310 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x560171899a70 con 0x56017188a080 2024-03-21T10:30:49.800 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:49.796+0000 7f7b27e00640 1 --2- 172.21.15.47:0/298347310 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x560171892c00 0x560171899400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T10:30:49.801 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:49.796+0000 7f7b1fe00640 1 --2- 172.21.15.47:0/298347310 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56017188a080 0x560171898ec0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x56017188f190 tx=0x7f7b1000bfd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5e5fb4c0412a288c server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:30:49.801 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:49.796+0000 7f7b15600640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-21T10:30:49.801 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:49.797+0000 7f7b15600640 1 -- 172.21.15.47:0/298347310 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b10010960 con 0x56017188a080 2024-03-21T10:30:49.801 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:49.797+0000 7f7b31596740 1 -- 172.21.15.47:0/298347310 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f7b2003fc60 con 0x56017188a080 2024-03-21T10:30:49.801 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:49.797+0000 7f7b15600640 1 -- 172.21.15.47:0/298347310 <== mon.2 v2:172.21.15.100:3300/0 2 ==== osd_map(97..97 src has 1..97) v4 ==== 9474+0+0 (secure 0 0 0) 0x7f7b10010b00 con 0x56017188a080 2024-03-21T10:30:49.801 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:49.797+0000 7f7b15600640 1 -- 172.21.15.47:0/298347310 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7b1002f050 con 0x56017188a080 2024-03-21T10:30:49.801 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:49.797+0000 7f7b31596740 10 client.37109 fetch_fsmap learned FSMap version 32 2024-03-21T10:30:49.801 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:49.797+0000 7f7b31596740 1 -- 172.21.15.47:0/298347310 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x560171907610 con 0x56017188a080 2024-03-21T10:30:49.801 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:49.798+0000 7f7b15600640 1 -- 172.21.15.47:0/298347310 <== mon.2 v2:172.21.15.100:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f7b2003fc60 con 0x56017188a080 2024-03-21T10:30:49.801 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:49.798+0000 7f7b15600640 1 -- 172.21.15.47:0/298347310 <== mon.2 v2:172.21.15.100:3300/0 5 ==== fsmap(e 32) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f7b10011440 con 0x56017188a080 2024-03-21T10:30:49.801 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:49.798+0000 7f7b31596740 10 client.37109 fetch_fsmap finished waiting for FSMap version 32 2024-03-21T10:30:49.801 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:49.798+0000 7f7b31596740 10 client.37109 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.k 2024-03-21T10:30:49.801 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:49.798+0000 7f7b31596740 20 client.37109 populate_metadata read hostname 'smithi047' 2024-03-21T10:30:49.802 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:49.798+0000 7f7b31596740 1 --2- 172.21.15.47:0/298347310 >> [v2:172.21.15.100:6832/2723302038,v1:172.21.15.100:6833/2723302038] conn(0x560171907d20 0x56017190a140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:30:49.802 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:49.798+0000 7f7b31596740 4 client.37109 mds_command: new command op to 34427 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-21T10:30:49.802 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:49.798+0000 7f7b31596740 1 -- 172.21.15.47:0/298347310 --> [v2:172.21.15.100:6832/2723302038,v1:172.21.15.100:6833/2723302038] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x5601717f4b50 con 0x560171907d20 2024-03-21T10:30:49.802 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:49.799+0000 7f7b27e00640 1 --2- 172.21.15.47:0/298347310 >> [v2:172.21.15.100:6832/2723302038,v1:172.21.15.100:6833/2723302038] conn(0x560171907d20 0x56017190a140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:30:49.803 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:49.799+0000 7f7b27e00640 1 --2- 172.21.15.47:0/298347310 >> [v2:172.21.15.100:6832/2723302038,v1:172.21.15.100:6833/2723302038] conn(0x560171907d20 0x56017190a140 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=8684354b097f1e34 server_cookie=2b66be3a9f0b9c3a in_seq=0 out_seq=0 2024-03-21T10:30:49.804 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:49.799+0000 7f7b15600640 10 client.37109 ms_handle_connect on v2:172.21.15.100:6832/2723302038 2024-03-21T10:30:49.804 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:49.800+0000 7f7b15600640 1 -- 172.21.15.47:0/298347310 <== mds.3 v2:172.21.15.100:6832/2723302038 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x5601717f4b50 con 0x560171907d20 2024-03-21T10:30:49.804 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:49.800+0000 7f7b15600640 10 client.37109 handle_command_reply: tid=0 2024-03-21T10:30:49.804 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:49.800+0000 7f7b31596740 1 client.37109 shutdown 2024-03-21T10:30:49.804 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:49.800+0000 7f7b31596740 1 -- 172.21.15.47:0/298347310 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56017188a080 msgr2=0x560171898ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:30:49.804 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:49.800+0000 7f7b31596740 1 --2- 172.21.15.47:0/298347310 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56017188a080 0x560171898ec0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x56017188f190 tx=0x7f7b1000bfd0 comp rx=0 tx=0).stop 2024-03-21T10:30:49.804 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:49.800+0000 7f7b31596740 1 -- 172.21.15.47:0/298347310 shutdown_connections 2024-03-21T10:30:49.804 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:49.800+0000 7f7b31596740 1 --2- 172.21.15.47:0/298347310 >> [v2:172.21.15.100:6832/2723302038,v1:172.21.15.100:6833/2723302038] conn(0x560171907d20 0x56017190a140 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:30:49.804 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:49.800+0000 7f7b31596740 1 --2- 172.21.15.47:0/298347310 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x560171892c00 0x560171899400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:30:49.805 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:49.800+0000 7f7b31596740 1 --2- 172.21.15.47:0/298347310 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56017188a080 0x560171898ec0 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:30:49.805 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:49.800+0000 7f7b15600640 0 client.37109 ms_handle_reset on v2:172.21.15.100:6832/2723302038 2024-03-21T10:30:49.805 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:49.800+0000 7f7b31596740 1 --2- 172.21.15.47:0/298347310 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5601718896d0 0x56017189ab20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:30:49.805 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:49.800+0000 7f7b31596740 1 -- 172.21.15.47:0/298347310 >> 172.21.15.47:0/298347310 conn(0x560171883da0 msgr2=0x56017188a460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:30:49.805 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:49.800+0000 7f7b31596740 1 -- 172.21.15.47:0/298347310 shutdown_connections 2024-03-21T10:30:49.805 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:49.800+0000 7f7b31596740 1 -- 172.21.15.47:0/298347310 wait complete. 2024-03-21T10:30:49.805 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:49.800+0000 7f7b31596740 20 client.37109 trim_cache size 0 max 16384 2024-03-21T10:30:49.805 INFO:teuthology.orchestra.run.smithi047.stdout:[] 2024-03-21T10:30:49.806 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:49.801+0000 7f7b2e800640 1 -- 172.21.15.47:0/2779379146 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f7b0c0a3090 msgr2=0x7f7b0c0a5550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:30:49.806 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:49.801+0000 7f7b2e800640 1 --2- 172.21.15.47:0/2779379146 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f7b0c0a3090 0x7f7b0c0a5550 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x5601718891a0 tx=0x7f7b18000bc0 comp rx=0 tx=0).stop 2024-03-21T10:30:49.806 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:49.801+0000 7f7b2e800640 1 -- 172.21.15.47:0/2779379146 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7b28103830 msgr2=0x7f7b2819a040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:30:49.806 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:49.801+0000 7f7b2e800640 1 --2- 172.21.15.47:0/2779379146 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7b28103830 0x7f7b2819a040 secure :-1 s=READY pgs=901 cs=0 l=1 rev1=1 crypto rx=0x7f7b1800c890 tx=0x7f7b1800cd50 comp rx=0 tx=0).stop 2024-03-21T10:30:49.807 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:49.801+0000 7f7b2e800640 1 -- 172.21.15.47:0/2779379146 shutdown_connections 2024-03-21T10:30:49.807 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:49.801+0000 7f7b2e800640 1 --2- 172.21.15.47:0/2779379146 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7b28109840 0x7f7b2819e900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:30:49.807 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:49.801+0000 7f7b2e800640 1 --2- 172.21.15.47:0/2779379146 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f7b0c0a3090 0x7f7b0c0a5550 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:30:49.807 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:49.801+0000 7f7b2e800640 1 --2- 172.21.15.47:0/2779379146 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7b281041f0 0x7f7b2819a580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:30:49.807 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:49.801+0000 7f7b2e800640 1 --2- 172.21.15.47:0/2779379146 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7b28103830 0x7f7b2819a040 unknown :-1 s=CLOSED pgs=901 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:30:49.807 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:49.801+0000 7f7b2e800640 1 -- 172.21.15.47:0/2779379146 >> 172.21.15.47:0/2779379146 conn(0x7f7b280ff320 msgr2=0x7f7b280ffb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:30:49.807 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:49.801+0000 7f7b2e800640 1 -- 172.21.15.47:0/2779379146 shutdown_connections 2024-03-21T10:30:49.807 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:49.801+0000 7f7b2e800640 1 -- 172.21.15.47:0/2779379146 wait complete. 2024-03-21T10:30:49.820 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-21T10:30:49.820 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-03-21T10:30:50.035 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:50.035+0000 7fed35000640 1 -- 172.21.15.47:0/341867136 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fed30075bd0 msgr2=0x7fed30075fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:30:50.035 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:50.035+0000 7fed35000640 1 --2- 172.21.15.47:0/341867136 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fed30075bd0 0x7fed30075fb0 secure :-1 s=READY pgs=904 cs=0 l=1 rev1=1 crypto rx=0x7fed20009860 tx=0x7fed2002bb70 comp rx=0 tx=0).stop 2024-03-21T10:30:50.035 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:50.035+0000 7fed35000640 1 -- 172.21.15.47:0/341867136 shutdown_connections 2024-03-21T10:30:50.035 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:50.035+0000 7fed35000640 1 --2- 172.21.15.47:0/341867136 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fed301124e0 0x7fed301148d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:30:50.035 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:50.035+0000 7fed35000640 1 --2- 172.21.15.47:0/341867136 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fed300764f0 0x7fed30111fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:30:50.036 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:50.035+0000 7fed35000640 1 --2- 172.21.15.47:0/341867136 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fed30075bd0 0x7fed30075fb0 unknown :-1 s=CLOSED pgs=904 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:30:50.036 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:50.035+0000 7fed35000640 1 -- 172.21.15.47:0/341867136 >> 172.21.15.47:0/341867136 conn(0x7fed300ff350 msgr2=0x7fed30101770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:30:50.036 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:50.036+0000 7fed35000640 1 -- 172.21.15.47:0/341867136 shutdown_connections 2024-03-21T10:30:50.036 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:50.036+0000 7fed35000640 1 -- 172.21.15.47:0/341867136 wait complete. 2024-03-21T10:30:50.045 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:50.037+0000 7fed35000640 1 Processor -- start 2024-03-21T10:30:50.045 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:50.037+0000 7fed35000640 1 -- start start 2024-03-21T10:30:50.045 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:50.038+0000 7fed35000640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fed30075bd0 0x7fed3019e510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:30:50.045 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:50.038+0000 7fed35000640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fed300764f0 0x7fed3019ea50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:30:50.045 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:50.038+0000 7fed35000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fed301124e0 0x7fed301a2d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:30:50.045 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:50.038+0000 7fed35000640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed30117ef0 con 0x7fed300764f0 2024-03-21T10:30:50.045 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:50.038+0000 7fed35000640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed30117d70 con 0x7fed301124e0 2024-03-21T10:30:50.046 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:50.038+0000 7fed35000640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed30118070 con 0x7fed30075bd0 2024-03-21T10:30:50.046 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:50.038+0000 7fed25400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fed300764f0 0x7fed3019ea50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:30:50.046 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:50.038+0000 7fed25400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fed300764f0 0x7fed3019ea50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:46430/0 (socket says 172.21.15.47:46430) 2024-03-21T10:30:50.046 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:50.038+0000 7fed25400640 1 -- 172.21.15.47:0/22395328 learned_addr learned my addr 172.21.15.47:0/22395328 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:30:50.046 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:50.038+0000 7fed2e800640 1 --2- 172.21.15.47:0/22395328 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fed301124e0 0x7fed301a2d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:30:50.046 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:50.038+0000 7fed2de00640 1 --2- 172.21.15.47:0/22395328 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fed30075bd0 0x7fed3019e510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:30:50.046 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:50.038+0000 7fed25400640 1 -- 172.21.15.47:0/22395328 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fed30075bd0 msgr2=0x7fed3019e510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:30:50.046 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:50.038+0000 7fed25400640 1 --2- 172.21.15.47:0/22395328 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fed30075bd0 0x7fed3019e510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:30:50.047 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:50.038+0000 7fed25400640 1 -- 172.21.15.47:0/22395328 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fed301124e0 msgr2=0x7fed301a2d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:30:50.047 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:50.038+0000 7fed25400640 1 --2- 172.21.15.47:0/22395328 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fed301124e0 0x7fed301a2d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:30:50.047 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:50.038+0000 7fed25400640 1 -- 172.21.15.47:0/22395328 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed301a34b0 con 0x7fed300764f0 2024-03-21T10:30:50.047 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:50.038+0000 7fed2de00640 1 --2- 172.21.15.47:0/22395328 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fed30075bd0 0x7fed3019e510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T10:30:50.047 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:50.039+0000 7fed25400640 1 --2- 172.21.15.47:0/22395328 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fed300764f0 0x7fed3019ea50 secure :-1 s=READY pgs=905 cs=0 l=1 rev1=1 crypto rx=0x7fed1800d950 tx=0x7fed1800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:30:50.047 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:50.039+0000 7fed27400640 1 -- 172.21.15.47:0/22395328 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed18017070 con 0x7fed300764f0 2024-03-21T10:30:50.047 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:50.039+0000 7fed27400640 1 -- 172.21.15.47:0/22395328 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fed18009c30 con 0x7fed300764f0 2024-03-21T10:30:50.047 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:50.039+0000 7fed27400640 1 -- 172.21.15.47:0/22395328 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed1800f620 con 0x7fed300764f0 2024-03-21T10:30:50.047 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:50.039+0000 7fed35000640 1 -- 172.21.15.47:0/22395328 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fed301a37a0 con 0x7fed300764f0 2024-03-21T10:30:50.047 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:50.039+0000 7fed35000640 1 -- 172.21.15.47:0/22395328 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fed301ab030 con 0x7fed300764f0 2024-03-21T10:30:50.047 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:50.042+0000 7fed37d84740 1 Processor -- start 2024-03-21T10:30:50.047 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:50.042+0000 7fed27400640 1 -- 172.21.15.47:0/22395328 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 27) v1 ==== 137985+0+0 (secure 0 0 0) 0x7fed180109a0 con 0x7fed300764f0 2024-03-21T10:30:50.047 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:50.043+0000 7fed37d84740 1 -- start start 2024-03-21T10:30:50.047 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:50.043+0000 7fed37d84740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561689d336d0 0x561689d33ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:30:50.048 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:50.043+0000 7fed37d84740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561689d34080 0x561689d3c630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:30:50.048 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:50.043+0000 7fed37d84740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561689d3cc00 0x561689d3f050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:30:50.048 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:50.043+0000 7fed37d84740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x561689c972c0 con 0x561689d336d0 2024-03-21T10:30:50.048 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:50.043+0000 7fed37d84740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x561689c3fc90 con 0x561689d3cc00 2024-03-21T10:30:50.048 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:50.043+0000 7fed37d84740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x561689c93360 con 0x561689d34080 2024-03-21T10:30:50.048 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:50.043+0000 7fed2e800640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561689d336d0 0x561689d33ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:30:50.048 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:50.043+0000 7fed2e800640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561689d336d0 0x561689d33ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:46444/0 (socket says 172.21.15.47:46444) 2024-03-21T10:30:50.048 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:50.043+0000 7fed2e800640 1 -- 172.21.15.47:0/2032500621 learned_addr learned my addr 172.21.15.47:0/2032500621 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:30:50.048 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:50.043+0000 7fed25400640 1 --2- 172.21.15.47:0/2032500621 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561689d3cc00 0x561689d3f050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:30:50.048 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:50.044+0000 7fed2de00640 1 --2- 172.21.15.47:0/2032500621 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561689d34080 0x561689d3c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:30:50.048 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:50.044+0000 7fed2e800640 1 -- 172.21.15.47:0/2032500621 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561689d34080 msgr2=0x561689d3c630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:30:50.048 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:50.044+0000 7fed2e800640 1 --2- 172.21.15.47:0/2032500621 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561689d34080 0x561689d3c630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:30:50.048 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:50.044+0000 7fed2e800640 1 -- 172.21.15.47:0/2032500621 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561689d3cc00 msgr2=0x561689d3f050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:30:50.048 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:50.044+0000 7fed2e800640 1 --2- 172.21.15.47:0/2032500621 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561689d3cc00 0x561689d3f050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:30:50.049 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:50.044+0000 7fed2e800640 1 -- 172.21.15.47:0/2032500621 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561689c64080 con 0x561689d336d0 2024-03-21T10:30:50.049 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:50.044+0000 7fed27400640 1 --2- 172.21.15.47:0/22395328 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7fed140a3110 0x7fed140a55d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:30:50.049 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:50.044+0000 7fed2e800640 1 --2- 172.21.15.47:0/2032500621 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561689d336d0 0x561689d33ab0 secure :-1 s=READY pgs=906 cs=0 l=1 rev1=1 crypto rx=0x7fed2800e480 tx=0x7fed280095c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=db03fa9dd868bcd1 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:30:50.049 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:50.044+0000 7fed27400640 1 -- 172.21.15.47:0/22395328 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 9474+0+0 (secure 0 0 0) 0x7fed180cf7e0 con 0x7fed300764f0 2024-03-21T10:30:50.049 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:50.045+0000 7fed1f400640 1 -- 172.21.15.47:0/2032500621 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed28003970 con 0x561689d336d0 2024-03-21T10:30:50.049 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:50.045+0000 7fed2de00640 1 --2- 172.21.15.47:0/22395328 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7fed140a3110 0x7fed140a55d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:30:50.049 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:50.045+0000 7fed1f400640 1 -- 172.21.15.47:0/2032500621 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fed2800d7d0 con 0x561689d336d0 2024-03-21T10:30:50.049 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:50.045+0000 7fed1f400640 1 -- 172.21.15.47:0/2032500621 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed28004380 con 0x561689d336d0 2024-03-21T10:30:50.049 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:50.045+0000 7fed37d84740 1 -- 172.21.15.47:0/2032500621 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561689d336d0 msgr2=0x561689d33ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:30:50.049 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:50.045+0000 7fed37d84740 1 --2- 172.21.15.47:0/2032500621 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561689d336d0 0x561689d33ab0 secure :-1 s=READY pgs=906 cs=0 l=1 rev1=1 crypto rx=0x7fed2800e480 tx=0x7fed280095c0 comp rx=0 tx=0).stop 2024-03-21T10:30:50.049 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:50.045+0000 7fed2de00640 1 --2- 172.21.15.47:0/22395328 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7fed140a3110 0x7fed140a55d0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fed20005940 tx=0x7fed20005a90 comp rx=0 tx=0).ready entity=mgr.15687 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:30:50.049 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:50.045+0000 7fed37d84740 1 -- 172.21.15.47:0/2032500621 shutdown_connections 2024-03-21T10:30:50.049 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:50.046+0000 7fed37d84740 1 --2- 172.21.15.47:0/2032500621 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561689d3cc00 0x561689d3f050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:30:50.049 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:50.046+0000 7fed37d84740 1 --2- 172.21.15.47:0/2032500621 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561689d34080 0x561689d3c630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:30:50.050 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:50.046+0000 7fed37d84740 1 --2- 172.21.15.47:0/2032500621 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561689d336d0 0x561689d33ab0 unknown :-1 s=CLOSED pgs=906 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:30:50.050 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:50.046+0000 7fed37d84740 1 -- 172.21.15.47:0/2032500621 >> 172.21.15.47:0/2032500621 conn(0x561689d2dda0 msgr2=0x561689d2e180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:30:50.050 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:50.046+0000 7fed37d84740 1 -- 172.21.15.47:0/2032500621 shutdown_connections 2024-03-21T10:30:50.050 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:50.046+0000 7fed37d84740 1 -- 172.21.15.47:0/2032500621 wait complete. 2024-03-21T10:30:50.050 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:50.046+0000 7fed37d84740 1 -- start start 2024-03-21T10:30:50.050 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:50.046+0000 7fed37d84740 1 Processor -- start 2024-03-21T10:30:50.054 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:50.047+0000 7fed37d84740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561689d336d0 0x561689d3c470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:30:50.054 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:50.047+0000 7fed37d84740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561689d34080 0x561689d39870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:30:50.054 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:50.047+0000 7fed37d84740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561689d3cc00 0x561689d39dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:30:50.054 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:50.047+0000 7fed37d84740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x561689d3fbc0 con 0x561689d336d0 2024-03-21T10:30:50.055 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:50.047+0000 7fed37d84740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x561689c530d0 con 0x561689d3cc00 2024-03-21T10:30:50.055 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:50.047+0000 7fed37d84740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x561689d3fd40 con 0x561689d34080 2024-03-21T10:30:50.055 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:50.047+0000 7fed37d84740 10 client.0 _ll_register_callbacks cb 0x561689c57e80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T10:30:50.055 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:50.047+0000 7fed2de00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561689d336d0 0x561689d3c470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:30:50.055 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:50.047+0000 7fed2de00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561689d336d0 0x561689d3c470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:46456/0 (socket says 172.21.15.47:46456) 2024-03-21T10:30:50.055 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:50.047+0000 7fed2de00640 1 -- 172.21.15.47:0/602694666 learned_addr learned my addr 172.21.15.47:0/602694666 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:30:50.055 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:50.047+0000 7fed2e800640 1 --2- 172.21.15.47:0/602694666 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561689d3cc00 0x561689d39dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:30:50.055 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:50.047+0000 7fed25400640 1 --2- 172.21.15.47:0/602694666 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561689d34080 0x561689d39870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:30:50.055 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:50.048+0000 7fed2de00640 1 -- 172.21.15.47:0/602694666 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561689d34080 msgr2=0x561689d39870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:30:50.055 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:50.048+0000 7fed2de00640 1 --2- 172.21.15.47:0/602694666 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561689d34080 0x561689d39870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:30:50.055 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:50.048+0000 7fed2de00640 1 -- 172.21.15.47:0/602694666 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561689d3cc00 msgr2=0x561689d39dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:30:50.055 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:50.048+0000 7fed2de00640 1 --2- 172.21.15.47:0/602694666 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561689d3cc00 0x561689d39dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:30:50.055 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:50.048+0000 7fed2de00640 1 -- 172.21.15.47:0/602694666 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5616899ab3f0 con 0x561689d336d0 2024-03-21T10:30:50.056 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:50.048+0000 7fed25400640 1 --2- 172.21.15.47:0/602694666 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561689d34080 0x561689d39870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T10:30:50.056 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:50.048+0000 7fed2de00640 1 --2- 172.21.15.47:0/602694666 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561689d336d0 0x561689d3c470 secure :-1 s=READY pgs=907 cs=0 l=1 rev1=1 crypto rx=0x7fed20005e00 tx=0x7fed2002bb70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b14b4884c5e1454 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:30:50.056 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:50.048+0000 7fed13e00640 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-03-21T10:30:50.056 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:50.048+0000 7fed13e00640 1 -- 172.21.15.47:0/602694666 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed20046040 con 0x561689d336d0 2024-03-21T10:30:50.056 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:50.048+0000 7fed13e00640 1 -- 172.21.15.47:0/602694666 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(97..97 src has 1..97) v4 ==== 9474+0+0 (secure 0 0 0) 0x7fed20005310 con 0x561689d336d0 2024-03-21T10:30:50.056 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:50.048+0000 7fed37d84740 1 -- 172.21.15.47:0/602694666 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fed2800d7d0 con 0x561689d336d0 2024-03-21T10:30:50.056 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:50.049+0000 7fed13e00640 1 -- 172.21.15.47:0/602694666 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7fed20039040 con 0x561689d336d0 2024-03-21T10:30:50.057 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:50.049+0000 7fed37d84740 10 client.17340 fetch_fsmap learned FSMap version 32 2024-03-21T10:30:50.057 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:50.049+0000 7fed37d84740 1 -- 172.21.15.47:0/602694666 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x561689a757d0 con 0x561689d336d0 2024-03-21T10:30:50.057 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:50.049+0000 7fed13e00640 1 -- 172.21.15.47:0/602694666 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fed2800d7d0 con 0x561689d336d0 2024-03-21T10:30:50.057 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:50.049+0000 7fed13e00640 1 -- 172.21.15.47:0/602694666 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 32) v1 ==== 7942+0+0 (secure 0 0 0) 0x7fed20052030 con 0x561689d336d0 2024-03-21T10:30:50.057 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:50.049+0000 7fed37d84740 10 client.17340 fetch_fsmap finished waiting for FSMap version 32 2024-03-21T10:30:50.057 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:50.049+0000 7fed37d84740 10 client.17340 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.e 2024-03-21T10:30:50.057 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:50.049+0000 7fed37d84740 20 client.17340 populate_metadata read hostname 'smithi047' 2024-03-21T10:30:50.057 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:50.049+0000 7fed37d84740 1 --2- 172.21.15.47:0/602694666 >> [v2:172.21.15.100:6834/2597849957,v1:172.21.15.100:6835/2597849957] conn(0x561689d4ca90 0x561689d4eee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:30:50.057 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:50.049+0000 7fed37d84740 4 client.17340 mds_command: new command op to 34454 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T10:30:50.057 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:50.049+0000 7fed37d84740 1 -- 172.21.15.47:0/602694666 --> [v2:172.21.15.100:6834/2597849957,v1:172.21.15.100:6835/2597849957] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x561689ca4800 con 0x561689d4ca90 2024-03-21T10:30:50.057 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:50.050+0000 7fed25400640 1 --2- 172.21.15.47:0/602694666 >> [v2:172.21.15.100:6834/2597849957,v1:172.21.15.100:6835/2597849957] conn(0x561689d4ca90 0x561689d4eee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:30:50.058 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:50.050+0000 7fed25400640 1 --2- 172.21.15.47:0/602694666 >> [v2:172.21.15.100:6834/2597849957,v1:172.21.15.100:6835/2597849957] conn(0x561689d4ca90 0x561689d4eee0 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=76910101f9701271 server_cookie=4620eb1f1dd9b9a in_seq=0 out_seq=0 2024-03-21T10:30:50.058 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:50.050+0000 7fed13e00640 10 client.17340 ms_handle_connect on v2:172.21.15.100:6834/2597849957 2024-03-21T10:30:50.058 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:50.052+0000 7fed13e00640 1 -- 172.21.15.47:0/602694666 <== mds.2 v2:172.21.15.100:6834/2597849957 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x561689ca4800 con 0x561689d4ca90 2024-03-21T10:30:50.058 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:50.052+0000 7fed13e00640 10 client.17340 handle_command_reply: tid=0 2024-03-21T10:30:50.058 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:50.052+0000 7fed37d84740 1 client.17340 shutdown 2024-03-21T10:30:50.058 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:50.052+0000 7fed37d84740 1 -- 172.21.15.47:0/602694666 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561689d336d0 msgr2=0x561689d3c470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:30:50.058 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:50.052+0000 7fed37d84740 1 --2- 172.21.15.47:0/602694666 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561689d336d0 0x561689d3c470 secure :-1 s=READY pgs=907 cs=0 l=1 rev1=1 crypto rx=0x7fed20005e00 tx=0x7fed2002bb70 comp rx=0 tx=0).stop 2024-03-21T10:30:50.058 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:50.052+0000 7fed37d84740 1 -- 172.21.15.47:0/602694666 shutdown_connections 2024-03-21T10:30:50.059 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:50.052+0000 7fed37d84740 1 --2- 172.21.15.47:0/602694666 >> [v2:172.21.15.100:6834/2597849957,v1:172.21.15.100:6835/2597849957] conn(0x561689d4ca90 0x561689d4eee0 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:30:50.059 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:50.052+0000 7fed37d84740 1 --2- 172.21.15.47:0/602694666 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561689d3cc00 0x561689d39dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:30:50.059 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:50.052+0000 7fed37d84740 1 --2- 172.21.15.47:0/602694666 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561689d34080 0x561689d39870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:30:50.059 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:50.052+0000 7fed13e00640 0 client.17340 ms_handle_reset on v2:172.21.15.100:6834/2597849957 2024-03-21T10:30:50.059 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:50.052+0000 7fed37d84740 1 --2- 172.21.15.47:0/602694666 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561689d336d0 0x561689d3c470 unknown :-1 s=CLOSED pgs=907 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:30:50.059 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:50.052+0000 7fed37d84740 1 -- 172.21.15.47:0/602694666 >> 172.21.15.47:0/602694666 conn(0x561689d2dda0 msgr2=0x561689d2faf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:30:50.059 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:50.052+0000 7fed37d84740 1 -- 172.21.15.47:0/602694666 shutdown_connections 2024-03-21T10:30:50.059 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:50.052+0000 7fed37d84740 1 -- 172.21.15.47:0/602694666 wait complete. 2024-03-21T10:30:50.059 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:50.052+0000 7fed37d84740 20 client.17340 trim_cache size 0 max 16384 2024-03-21T10:30:50.066 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:50.066+0000 7fed37d84740 1 Processor -- start 2024-03-21T10:30:50.066 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:50.066+0000 7fed37d84740 1 -- start start 2024-03-21T10:30:50.066 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:50.066+0000 7fed37d84740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561689d336d0 0x561689d3bfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:30:50.066 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:50.066+0000 7fed37d84740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561689d34080 0x561689d2cd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:30:50.067 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:50.066+0000 7fed37d84740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561689d3cc00 0x561689d2d270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:30:50.067 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:50.066+0000 7fed37d84740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed301b4fa0 con 0x561689d336d0 2024-03-21T10:30:50.067 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:50.066+0000 7fed37d84740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed301b6920 con 0x561689d3cc00 2024-03-21T10:30:50.067 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:50.066+0000 7fed37d84740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed301b5600 con 0x561689d34080 2024-03-21T10:30:50.067 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:50.067+0000 7fed2de00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561689d336d0 0x561689d3bfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:30:50.067 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:50.067+0000 7fed2de00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561689d336d0 0x561689d3bfa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:46468/0 (socket says 172.21.15.47:46468) 2024-03-21T10:30:50.067 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:50.067+0000 7fed2de00640 1 -- 172.21.15.47:0/1668481617 learned_addr learned my addr 172.21.15.47:0/1668481617 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:30:50.067 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:50.067+0000 7fed2e800640 1 --2- 172.21.15.47:0/1668481617 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561689d3cc00 0x561689d2d270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:30:50.067 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:50.067+0000 7fed2de00640 1 -- 172.21.15.47:0/1668481617 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561689d34080 msgr2=0x561689d2cd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:30:50.068 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:50.067+0000 7fed2de00640 1 --2- 172.21.15.47:0/1668481617 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561689d34080 0x561689d2cd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:30:50.068 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:50.067+0000 7fed2de00640 1 -- 172.21.15.47:0/1668481617 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561689d3cc00 msgr2=0x561689d2d270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:30:50.068 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:50.067+0000 7fed2de00640 1 --2- 172.21.15.47:0/1668481617 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561689d3cc00 0x561689d2d270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:30:50.068 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:50.067+0000 7fed2de00640 1 -- 172.21.15.47:0/1668481617 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561689d33e20 con 0x561689d336d0 2024-03-21T10:30:50.068 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:50.067+0000 7fed2e800640 1 --2- 172.21.15.47:0/1668481617 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561689d3cc00 0x561689d2d270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T10:30:50.068 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:50.067+0000 7fed2de00640 1 --2- 172.21.15.47:0/1668481617 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561689d336d0 0x561689d3bfa0 secure :-1 s=READY pgs=908 cs=0 l=1 rev1=1 crypto rx=0x7fed2002c0b0 tx=0x7fed20034910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3589670949725f76 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:30:50.071 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:50.068+0000 7fed1ea00640 1 -- 172.21.15.47:0/1668481617 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed20046040 con 0x561689d336d0 2024-03-21T10:30:50.072 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:50.068+0000 7fed1ea00640 1 -- 172.21.15.47:0/1668481617 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fed200055c0 con 0x561689d336d0 2024-03-21T10:30:50.072 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:50.068+0000 7fed1ea00640 1 -- 172.21.15.47:0/1668481617 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed200049b0 con 0x561689d336d0 2024-03-21T10:30:50.072 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:50.068+0000 7fed37d84740 1 -- 172.21.15.47:0/1668481617 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561689d336d0 msgr2=0x561689d3bfa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:30:50.072 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:50.068+0000 7fed37d84740 1 --2- 172.21.15.47:0/1668481617 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561689d336d0 0x561689d3bfa0 secure :-1 s=READY pgs=908 cs=0 l=1 rev1=1 crypto rx=0x7fed2002c0b0 tx=0x7fed20034910 comp rx=0 tx=0).stop 2024-03-21T10:30:50.072 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:50.068+0000 7fed37d84740 1 -- 172.21.15.47:0/1668481617 shutdown_connections 2024-03-21T10:30:50.072 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:50.068+0000 7fed37d84740 1 --2- 172.21.15.47:0/1668481617 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561689d3cc00 0x561689d2d270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:30:50.072 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:50.068+0000 7fed37d84740 1 --2- 172.21.15.47:0/1668481617 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561689d34080 0x561689d2cd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:30:50.073 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:50.068+0000 7fed37d84740 1 --2- 172.21.15.47:0/1668481617 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561689d336d0 0x561689d3bfa0 unknown :-1 s=CLOSED pgs=908 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:30:50.073 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:50.068+0000 7fed37d84740 1 -- 172.21.15.47:0/1668481617 >> 172.21.15.47:0/1668481617 conn(0x561689d2dda0 msgr2=0x561689d2c620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:30:50.073 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:50.068+0000 7fed37d84740 1 -- 172.21.15.47:0/1668481617 shutdown_connections 2024-03-21T10:30:50.073 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:50.069+0000 7fed37d84740 1 -- 172.21.15.47:0/1668481617 wait complete. 2024-03-21T10:30:50.073 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:50.069+0000 7fed37d84740 1 -- start start 2024-03-21T10:30:50.073 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:50.069+0000 7fed37d84740 1 Processor -- start 2024-03-21T10:30:50.073 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:50.070+0000 7fed37d84740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561689d336d0 0x561689d44a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:30:50.073 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:50.070+0000 7fed37d84740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561689d34080 0x561689d42df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:30:50.073 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:50.070+0000 7fed37d84740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561689d3cc00 0x561689d43330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:30:50.073 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:50.070+0000 7fed37d84740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed301b62c0 con 0x561689d3cc00 2024-03-21T10:30:50.073 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:50.070+0000 7fed37d84740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed301b5c60 con 0x561689d34080 2024-03-21T10:30:50.073 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:50.070+0000 7fed37d84740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed301b3160 con 0x561689d336d0 2024-03-21T10:30:50.073 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:50.070+0000 7fed37d84740 10 client.0 _ll_register_callbacks cb 0x7fed20043040 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T10:30:50.074 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:50.070+0000 7fed2e800640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561689d3cc00 0x561689d43330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:30:50.074 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:50.070+0000 7fed25400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561689d34080 0x561689d42df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:30:50.074 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:50.070+0000 7fed2e800640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561689d3cc00 0x561689d43330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:46484/0 (socket says 172.21.15.47:46484) 2024-03-21T10:30:50.074 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:50.070+0000 7fed2e800640 1 -- 172.21.15.47:0/2649397414 learned_addr learned my addr 172.21.15.47:0/2649397414 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:30:50.074 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:50.070+0000 7fed2de00640 1 --2- 172.21.15.47:0/2649397414 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561689d336d0 0x561689d44a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:30:50.074 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:50.070+0000 7fed2e800640 1 -- 172.21.15.47:0/2649397414 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561689d336d0 msgr2=0x561689d44a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:30:50.074 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:50.070+0000 7fed2e800640 1 --2- 172.21.15.47:0/2649397414 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561689d336d0 0x561689d44a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:30:50.074 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:50.070+0000 7fed2e800640 1 -- 172.21.15.47:0/2649397414 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561689d34080 msgr2=0x561689d42df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:30:50.074 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:50.070+0000 7fed2e800640 1 --2- 172.21.15.47:0/2649397414 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561689d34080 0x561689d42df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:30:50.074 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:50.070+0000 7fed2e800640 1 -- 172.21.15.47:0/2649397414 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x561689d3a140 con 0x561689d3cc00 2024-03-21T10:30:50.074 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:50.071+0000 7fed2e800640 1 --2- 172.21.15.47:0/2649397414 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561689d3cc00 0x561689d43330 secure :-1 s=READY pgs=909 cs=0 l=1 rev1=1 crypto rx=0x7fed28000f80 tx=0x7fed28009450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=782dc87f2ad6f6e0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:30:50.074 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:50.071+0000 7fed13e00640 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-03-21T10:30:50.074 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:50.071+0000 7fed13e00640 1 -- 172.21.15.47:0/2649397414 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed28003b80 con 0x561689d3cc00 2024-03-21T10:30:50.074 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:50.071+0000 7fed13e00640 1 -- 172.21.15.47:0/2649397414 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(97..97 src has 1..97) v4 ==== 9474+0+0 (secure 0 0 0) 0x7fed28003d20 con 0x561689d3cc00 2024-03-21T10:30:50.074 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:50.071+0000 7fed37d84740 1 -- 172.21.15.47:0/2649397414 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fed200055c0 con 0x561689d3cc00 2024-03-21T10:30:50.075 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:50.071+0000 7fed13e00640 1 -- 172.21.15.47:0/2649397414 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7fed2800d7d0 con 0x561689d3cc00 2024-03-21T10:30:50.075 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:50.071+0000 7fed37d84740 10 client.17346 fetch_fsmap learned FSMap version 32 2024-03-21T10:30:50.075 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:50.071+0000 7fed37d84740 1 -- 172.21.15.47:0/2649397414 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x561689da8f70 con 0x561689d3cc00 2024-03-21T10:30:50.075 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:50.072+0000 7fed13e00640 1 -- 172.21.15.47:0/2649397414 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fed200055c0 con 0x561689d3cc00 2024-03-21T10:30:50.075 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:50.072+0000 7fed13e00640 1 -- 172.21.15.47:0/2649397414 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 32) v1 ==== 7942+0+0 (secure 0 0 0) 0x7fed2800b040 con 0x561689d3cc00 2024-03-21T10:30:50.075 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:50.072+0000 7fed37d84740 10 client.17346 fetch_fsmap finished waiting for FSMap version 32 2024-03-21T10:30:50.075 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:50.072+0000 7fed37d84740 10 client.17346 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.e 2024-03-21T10:30:50.075 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:50.072+0000 7fed37d84740 20 client.17346 populate_metadata read hostname 'smithi047' 2024-03-21T10:30:50.075 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:50.072+0000 7fed37d84740 1 --2- 172.21.15.47:0/2649397414 >> [v2:172.21.15.100:6834/2597849957,v1:172.21.15.100:6835/2597849957] conn(0x561689da9550 0x561689dab970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:30:50.075 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:50.072+0000 7fed37d84740 4 client.17346 mds_command: new command op to 34454 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-21T10:30:50.075 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:50.072+0000 7fed37d84740 1 -- 172.21.15.47:0/2649397414 --> [v2:172.21.15.100:6834/2597849957,v1:172.21.15.100:6835/2597849957] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x561689c9eb50 con 0x561689da9550 2024-03-21T10:30:50.075 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:50.072+0000 7fed2de00640 1 --2- 172.21.15.47:0/2649397414 >> [v2:172.21.15.100:6834/2597849957,v1:172.21.15.100:6835/2597849957] conn(0x561689da9550 0x561689dab970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:30:50.077 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:50.073+0000 7fed2de00640 1 --2- 172.21.15.47:0/2649397414 >> [v2:172.21.15.100:6834/2597849957,v1:172.21.15.100:6835/2597849957] conn(0x561689da9550 0x561689dab970 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=73a8c5e7cc10abf2 server_cookie=de7b870185cf04a in_seq=0 out_seq=0 2024-03-21T10:30:50.077 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:50.073+0000 7fed13e00640 10 client.17346 ms_handle_connect on v2:172.21.15.100:6834/2597849957 2024-03-21T10:30:50.077 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:50.073+0000 7fed13e00640 1 -- 172.21.15.47:0/2649397414 <== mds.2 v2:172.21.15.100:6834/2597849957 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x561689c9eb50 con 0x561689da9550 2024-03-21T10:30:50.077 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:50.073+0000 7fed13e00640 10 client.17346 handle_command_reply: tid=0 2024-03-21T10:30:50.077 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:50.073+0000 7fed37d84740 1 client.17346 shutdown 2024-03-21T10:30:50.077 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:50.074+0000 7fed37d84740 1 -- 172.21.15.47:0/2649397414 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561689d3cc00 msgr2=0x561689d43330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:30:50.077 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:50.074+0000 7fed37d84740 1 --2- 172.21.15.47:0/2649397414 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561689d3cc00 0x561689d43330 secure :-1 s=READY pgs=909 cs=0 l=1 rev1=1 crypto rx=0x7fed28000f80 tx=0x7fed28009450 comp rx=0 tx=0).stop 2024-03-21T10:30:50.078 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:50.074+0000 7fed37d84740 1 -- 172.21.15.47:0/2649397414 shutdown_connections 2024-03-21T10:30:50.078 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:50.074+0000 7fed37d84740 1 --2- 172.21.15.47:0/2649397414 >> [v2:172.21.15.100:6834/2597849957,v1:172.21.15.100:6835/2597849957] conn(0x561689da9550 0x561689dab970 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:30:50.078 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:50.074+0000 7fed37d84740 1 --2- 172.21.15.47:0/2649397414 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561689d3cc00 0x561689d43330 unknown :-1 s=CLOSED pgs=909 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:30:50.078 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:50.074+0000 7fed37d84740 1 --2- 172.21.15.47:0/2649397414 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561689d34080 0x561689d42df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:30:50.078 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:50.074+0000 7fed37d84740 1 --2- 172.21.15.47:0/2649397414 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561689d336d0 0x561689d44a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:30:50.078 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:50.074+0000 7fed13e00640 0 client.17346 ms_handle_reset on v2:172.21.15.100:6834/2597849957 2024-03-21T10:30:50.078 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:50.074+0000 7fed37d84740 1 -- 172.21.15.47:0/2649397414 >> 172.21.15.47:0/2649397414 conn(0x561689d2dda0 msgr2=0x561689d34cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:30:50.078 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:50.074+0000 7fed37d84740 1 -- 172.21.15.47:0/2649397414 shutdown_connections 2024-03-21T10:30:50.078 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:50.074+0000 7fed37d84740 1 -- 172.21.15.47:0/2649397414 wait complete. 2024-03-21T10:30:50.078 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:50.074+0000 7fed37d84740 20 client.17346 trim_cache size 0 max 16384 2024-03-21T10:30:50.079 INFO:teuthology.orchestra.run.smithi047.stdout:[] 2024-03-21T10:30:50.079 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:50.075+0000 7fed35000640 1 -- 172.21.15.47:0/22395328 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7fed140a3110 msgr2=0x7fed140a55d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:30:50.079 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:50.075+0000 7fed35000640 1 --2- 172.21.15.47:0/22395328 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7fed140a3110 0x7fed140a55d0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fed20005940 tx=0x7fed20005a90 comp rx=0 tx=0).stop 2024-03-21T10:30:50.079 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:50.075+0000 7fed35000640 1 -- 172.21.15.47:0/22395328 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fed300764f0 msgr2=0x7fed3019ea50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:30:50.079 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:50.075+0000 7fed35000640 1 --2- 172.21.15.47:0/22395328 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fed300764f0 0x7fed3019ea50 secure :-1 s=READY pgs=905 cs=0 l=1 rev1=1 crypto rx=0x7fed1800d950 tx=0x7fed1800de10 comp rx=0 tx=0).stop 2024-03-21T10:30:50.079 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:50.075+0000 7fed35000640 1 -- 172.21.15.47:0/22395328 shutdown_connections 2024-03-21T10:30:50.080 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:50.075+0000 7fed35000640 1 --2- 172.21.15.47:0/22395328 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7fed140a3110 0x7fed140a55d0 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:30:50.080 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:50.075+0000 7fed35000640 1 --2- 172.21.15.47:0/22395328 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fed301124e0 0x7fed301a2d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:30:50.080 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:50.075+0000 7fed35000640 1 --2- 172.21.15.47:0/22395328 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fed300764f0 0x7fed3019ea50 unknown :-1 s=CLOSED pgs=905 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:30:50.080 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:50.075+0000 7fed35000640 1 --2- 172.21.15.47:0/22395328 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fed30075bd0 0x7fed3019e510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:30:50.080 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:50.075+0000 7fed35000640 1 -- 172.21.15.47:0/22395328 >> 172.21.15.47:0/22395328 conn(0x7fed300ff350 msgr2=0x7fed30101740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:30:50.080 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:50.075+0000 7fed35000640 1 -- 172.21.15.47:0/22395328 shutdown_connections 2024-03-21T10:30:50.080 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:50.075+0000 7fed35000640 1 -- 172.21.15.47:0/22395328 wait complete. 2024-03-21T10:30:50.096 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-21T10:30:50.096 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-03-21T10:30:50.305 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:50.305+0000 7fec41a00640 1 -- 172.21.15.47:0/3741869885 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fec3c1028e0 msgr2=0x7fec3c102cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:30:50.305 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:50.305+0000 7fec41a00640 1 --2- 172.21.15.47:0/3741869885 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fec3c1028e0 0x7fec3c102cc0 secure :-1 s=READY pgs=910 cs=0 l=1 rev1=1 crypto rx=0x7fec2c009900 tx=0x7fec2c02bcd0 comp rx=0 tx=0).stop 2024-03-21T10:30:50.305 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:50.306+0000 7fec41a00640 1 -- 172.21.15.47:0/3741869885 shutdown_connections 2024-03-21T10:30:50.305 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:50.306+0000 7fec41a00640 1 --2- 172.21.15.47:0/3741869885 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fec3c10bcb0 0x7fec3c10e0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:30:50.305 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:50.306+0000 7fec41a00640 1 --2- 172.21.15.47:0/3741869885 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fec3c103200 0x7fec3c10b6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:30:50.305 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:50.306+0000 7fec41a00640 1 --2- 172.21.15.47:0/3741869885 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fec3c1028e0 0x7fec3c102cc0 unknown :-1 s=CLOSED pgs=910 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:30:50.305 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:50.306+0000 7fec41a00640 1 -- 172.21.15.47:0/3741869885 >> 172.21.15.47:0/3741869885 conn(0x7fec3c076920 msgr2=0x7fec3c076d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:30:50.305 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:50.306+0000 7fec41a00640 1 -- 172.21.15.47:0/3741869885 shutdown_connections 2024-03-21T10:30:50.306 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:50.306+0000 7fec41a00640 1 -- 172.21.15.47:0/3741869885 wait complete. 2024-03-21T10:30:50.307 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:50.307+0000 7fec41a00640 1 Processor -- start 2024-03-21T10:30:50.307 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:50.307+0000 7fec41a00640 1 -- start start 2024-03-21T10:30:50.307 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:50.307+0000 7fec41a00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fec3c1028e0 0x7fec3c100530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:30:50.307 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:50.307+0000 7fec41a00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fec3c103200 0x7fec3c100a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:30:50.307 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:50.307+0000 7fec41a00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fec3c10bcb0 0x7fec3c100fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:30:50.307 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:50.307+0000 7fec41a00640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec3c111940 con 0x7fec3c10bcb0 2024-03-21T10:30:50.307 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:50.307+0000 7fec41a00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec3c1117c0 con 0x7fec3c1028e0 2024-03-21T10:30:50.308 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:50.307+0000 7fec41a00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec3c111ac0 con 0x7fec3c103200 2024-03-21T10:30:50.308 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:50.307+0000 7fec3b400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fec3c10bcb0 0x7fec3c100fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:30:50.308 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:50.308+0000 7fec3b400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fec3c10bcb0 0x7fec3c100fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:46502/0 (socket says 172.21.15.47:46502) 2024-03-21T10:30:50.308 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:50.308+0000 7fec3b400640 1 -- 172.21.15.47:0/2954333324 learned_addr learned my addr 172.21.15.47:0/2954333324 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:30:50.308 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:50.308+0000 7fec3aa00640 1 --2- 172.21.15.47:0/2954333324 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fec3c1028e0 0x7fec3c100530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:30:50.308 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:50.308+0000 7fec3b400640 1 -- 172.21.15.47:0/2954333324 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fec3c103200 msgr2=0x7fec3c100a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:30:50.308 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:50.308+0000 7fec33e00640 1 --2- 172.21.15.47:0/2954333324 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fec3c103200 0x7fec3c100a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:30:50.308 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:50.308+0000 7fec3b400640 1 --2- 172.21.15.47:0/2954333324 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fec3c103200 0x7fec3c100a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:30:50.308 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:50.308+0000 7fec3b400640 1 -- 172.21.15.47:0/2954333324 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fec3c1028e0 msgr2=0x7fec3c100530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:30:50.308 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:50.308+0000 7fec3b400640 1 --2- 172.21.15.47:0/2954333324 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fec3c1028e0 0x7fec3c100530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:30:50.308 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:50.308+0000 7fec3b400640 1 -- 172.21.15.47:0/2954333324 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec3c1a6360 con 0x7fec3c10bcb0 2024-03-21T10:30:50.308 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:50.308+0000 7fec3b400640 1 --2- 172.21.15.47:0/2954333324 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fec3c10bcb0 0x7fec3c100fb0 secure :-1 s=READY pgs=911 cs=0 l=1 rev1=1 crypto rx=0x7fec3400eb40 tx=0x7fec3400c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:30:50.309 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:50.309+0000 7fec33400640 1 -- 172.21.15.47:0/2954333324 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec34004b20 con 0x7fec3c10bcb0 2024-03-21T10:30:50.309 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:50.309+0000 7fec41a00640 1 -- 172.21.15.47:0/2954333324 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fec3c1a6650 con 0x7fec3c10bcb0 2024-03-21T10:30:50.309 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:50.309+0000 7fec33400640 1 -- 172.21.15.47:0/2954333324 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fec34004cc0 con 0x7fec3c10bcb0 2024-03-21T10:30:50.309 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:50.309+0000 7fec33400640 1 -- 172.21.15.47:0/2954333324 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec340107b0 con 0x7fec3c10bcb0 2024-03-21T10:30:50.310 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:50.309+0000 7fec41a00640 1 -- 172.21.15.47:0/2954333324 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fec3c1a6ba0 con 0x7fec3c10bcb0 2024-03-21T10:30:50.311 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:50.312+0000 7fec44612740 1 Processor -- start 2024-03-21T10:30:50.311 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:50.312+0000 7fec33400640 1 -- 172.21.15.47:0/2954333324 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 27) v1 ==== 137985+0+0 (secure 0 0 0) 0x7fec34005020 con 0x7fec3c10bcb0 2024-03-21T10:30:50.312 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:50.313+0000 7fec44612740 1 -- start start 2024-03-21T10:30:50.312 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:50.313+0000 7fec44612740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561c7c0a06d0 0x561c7c0a0ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:30:50.312 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:50.313+0000 7fec44612740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561c7c0a1080 0x561c7c0a9630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:30:50.312 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:50.313+0000 7fec44612740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561c7c0a9c00 0x561c7c0ac050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:30:50.312 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:50.313+0000 7fec44612740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x561c7c0042c0 con 0x561c7c0a1080 2024-03-21T10:30:50.312 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:50.313+0000 7fec44612740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x561c7bfacc90 con 0x561c7c0a9c00 2024-03-21T10:30:50.313 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:50.313+0000 7fec44612740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x561c7c000360 con 0x561c7c0a06d0 2024-03-21T10:30:50.313 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:50.313+0000 7fec3aa00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561c7c0a1080 0x561c7c0a9630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:30:50.313 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:50.313+0000 7fec3aa00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561c7c0a1080 0x561c7c0a9630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:46510/0 (socket says 172.21.15.47:46510) 2024-03-21T10:30:50.313 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:50.313+0000 7fec3aa00640 1 -- 172.21.15.47:0/539833945 learned_addr learned my addr 172.21.15.47:0/539833945 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:30:50.313 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:50.313+0000 7fec33e00640 1 --2- 172.21.15.47:0/539833945 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561c7c0a9c00 0x561c7c0ac050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:30:50.315 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:50.314+0000 7fec33e00640 1 --2- 172.21.15.47:0/539833945 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561c7c0a06d0 0x561c7c0a0ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:30:50.315 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:50.314+0000 7fec3aa00640 1 -- 172.21.15.47:0/539833945 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561c7c0a06d0 msgr2=0x561c7c0a0ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:30:50.316 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:50.314+0000 7fec3aa00640 1 --2- 172.21.15.47:0/539833945 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561c7c0a06d0 0x561c7c0a0ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:30:50.316 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:50.314+0000 7fec3aa00640 1 -- 172.21.15.47:0/539833945 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561c7c0a9c00 msgr2=0x561c7c0ac050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:30:50.316 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:50.314+0000 7fec3aa00640 1 --2- 172.21.15.47:0/539833945 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561c7c0a9c00 0x561c7c0ac050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:30:50.316 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:50.314+0000 7fec3aa00640 1 -- 172.21.15.47:0/539833945 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561c7bfd1080 con 0x561c7c0a1080 2024-03-21T10:30:50.316 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:50.314+0000 7fec33400640 1 --2- 172.21.15.47:0/2954333324 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7fec200a3090 0x7fec200a5550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:30:50.316 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:50.314+0000 7fec33400640 1 -- 172.21.15.47:0/2954333324 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 9474+0+0 (secure 0 0 0) 0x7fec340d03a0 con 0x7fec3c10bcb0 2024-03-21T10:30:50.316 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:50.314+0000 7fec33e00640 1 --2- 172.21.15.47:0/2954333324 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7fec200a3090 0x7fec200a5550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:30:50.316 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:50.314+0000 7fec3aa00640 1 --2- 172.21.15.47:0/539833945 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561c7c0a1080 0x561c7c0a9630 secure :-1 s=READY pgs=912 cs=0 l=1 rev1=1 crypto rx=0x7fec2c000c00 tx=0x7fec2c02c690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=837f6eb6387f7509 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:30:50.316 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:50.315+0000 7fec2be00640 1 -- 172.21.15.47:0/539833945 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec2c0040c0 con 0x561c7c0a1080 2024-03-21T10:30:50.316 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:50.315+0000 7fec2be00640 1 -- 172.21.15.47:0/539833945 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fec2c02cd30 con 0x561c7c0a1080 2024-03-21T10:30:50.316 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:50.315+0000 7fec2be00640 1 -- 172.21.15.47:0/539833945 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec2c005600 con 0x561c7c0a1080 2024-03-21T10:30:50.316 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:50.316+0000 7fec44612740 1 -- 172.21.15.47:0/539833945 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561c7c0a1080 msgr2=0x561c7c0a9630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:30:50.316 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:50.316+0000 7fec33e00640 1 --2- 172.21.15.47:0/2954333324 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7fec200a3090 0x7fec200a5550 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fec24005d00 tx=0x7fec24015040 comp rx=0 tx=0).ready entity=mgr.15687 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:30:50.316 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:50.316+0000 7fec44612740 1 --2- 172.21.15.47:0/539833945 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561c7c0a1080 0x561c7c0a9630 secure :-1 s=READY pgs=912 cs=0 l=1 rev1=1 crypto rx=0x7fec2c000c00 tx=0x7fec2c02c690 comp rx=0 tx=0).stop 2024-03-21T10:30:50.317 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:50.316+0000 7fec44612740 1 -- 172.21.15.47:0/539833945 shutdown_connections 2024-03-21T10:30:50.317 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:50.316+0000 7fec44612740 1 --2- 172.21.15.47:0/539833945 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561c7c0a9c00 0x561c7c0ac050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:30:50.317 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:50.316+0000 7fec44612740 1 --2- 172.21.15.47:0/539833945 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561c7c0a1080 0x561c7c0a9630 unknown :-1 s=CLOSED pgs=912 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:30:50.317 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:50.316+0000 7fec44612740 1 --2- 172.21.15.47:0/539833945 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561c7c0a06d0 0x561c7c0a0ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:30:50.317 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:50.316+0000 7fec44612740 1 -- 172.21.15.47:0/539833945 >> 172.21.15.47:0/539833945 conn(0x561c7c09ada0 msgr2=0x561c7c09b180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:30:50.317 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:50.316+0000 7fec44612740 1 -- 172.21.15.47:0/539833945 shutdown_connections 2024-03-21T10:30:50.317 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:50.316+0000 7fec44612740 1 -- 172.21.15.47:0/539833945 wait complete. 2024-03-21T10:30:50.317 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:50.316+0000 7fec44612740 1 -- start start 2024-03-21T10:30:50.317 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:50.317+0000 7fec44612740 1 Processor -- start 2024-03-21T10:30:50.320 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:50.317+0000 7fec44612740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561c7c0a06d0 0x561c7c0a9520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:30:50.320 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:50.317+0000 7fec44612740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561c7c0a1080 0x561c7c0a68b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:30:50.320 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:50.317+0000 7fec44612740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561c7c0a9c00 0x561c7c0a6e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:30:50.320 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:50.317+0000 7fec44612740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x561c7c0acb10 con 0x561c7c0a9c00 2024-03-21T10:30:50.320 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:50.317+0000 7fec44612740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x561c7bfc00d0 con 0x561c7c0a06d0 2024-03-21T10:30:50.320 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:50.317+0000 7fec44612740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x561c7c0acc90 con 0x561c7c0a1080 2024-03-21T10:30:50.320 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:50.317+0000 7fec44612740 10 client.0 _ll_register_callbacks cb 0x561c7bfc4e80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T10:30:50.321 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:50.317+0000 7fec3b400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561c7c0a9c00 0x561c7c0a6e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:30:50.321 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:50.317+0000 7fec3aa00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561c7c0a06d0 0x561c7c0a9520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:30:50.321 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:50.318+0000 7fec3b400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561c7c0a9c00 0x561c7c0a6e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:46512/0 (socket says 172.21.15.47:46512) 2024-03-21T10:30:50.321 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:50.318+0000 7fec3aa00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561c7c0a06d0 0x561c7c0a9520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:39816/0 (socket says 172.21.15.47:39816) 2024-03-21T10:30:50.321 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:50.318+0000 7fec3b400640 1 -- 172.21.15.47:0/2319758189 learned_addr learned my addr 172.21.15.47:0/2319758189 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:30:50.321 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:50.318+0000 7fec33e00640 1 --2- 172.21.15.47:0/2319758189 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561c7c0a1080 0x561c7c0a68b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:30:50.321 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:50.318+0000 7fec3b400640 1 -- 172.21.15.47:0/2319758189 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561c7c0a1080 msgr2=0x561c7c0a68b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:30:50.321 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:50.318+0000 7fec3b400640 1 --2- 172.21.15.47:0/2319758189 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561c7c0a1080 0x561c7c0a68b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:30:50.321 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:50.318+0000 7fec3b400640 1 -- 172.21.15.47:0/2319758189 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561c7c0a06d0 msgr2=0x561c7c0a9520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:30:50.321 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:50.318+0000 7fec3b400640 1 --2- 172.21.15.47:0/2319758189 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561c7c0a06d0 0x561c7c0a9520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:30:50.321 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:50.318+0000 7fec3b400640 1 -- 172.21.15.47:0/2319758189 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x561c7bd183f0 con 0x561c7c0a9c00 2024-03-21T10:30:50.321 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:50.318+0000 7fec33e00640 1 --2- 172.21.15.47:0/2319758189 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561c7c0a1080 0x561c7c0a68b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T10:30:50.321 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:50.318+0000 7fec3b400640 1 --2- 172.21.15.47:0/2319758189 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561c7c0a9c00 0x561c7c0a6e50 secure :-1 s=READY pgs=913 cs=0 l=1 rev1=1 crypto rx=0x7fec34005ba0 tx=0x7fec3409c010 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a4719e6eaced0d4f server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:30:50.321 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:50.318+0000 7fec28c00640 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-03-21T10:30:50.322 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:50.318+0000 7fec28c00640 1 -- 172.21.15.47:0/2319758189 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec3409cad0 con 0x561c7c0a9c00 2024-03-21T10:30:50.322 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:50.318+0000 7fec28c00640 1 -- 172.21.15.47:0/2319758189 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(97..97 src has 1..97) v4 ==== 9474+0+0 (secure 0 0 0) 0x7fec34098a40 con 0x561c7c0a9c00 2024-03-21T10:30:50.322 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:50.318+0000 7fec44612740 1 -- 172.21.15.47:0/2319758189 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fec2c02cd30 con 0x561c7c0a9c00 2024-03-21T10:30:50.322 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:50.319+0000 7fec28c00640 1 -- 172.21.15.47:0/2319758189 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7fec3409c3d0 con 0x561c7c0a9c00 2024-03-21T10:30:50.322 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:50.319+0000 7fec44612740 10 client.17358 fetch_fsmap learned FSMap version 32 2024-03-21T10:30:50.322 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:50.319+0000 7fec44612740 1 -- 172.21.15.47:0/2319758189 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x561c7bde27d0 con 0x561c7c0a9c00 2024-03-21T10:30:50.322 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:50.319+0000 7fec28c00640 1 -- 172.21.15.47:0/2319758189 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fec2c02cd30 con 0x561c7c0a9c00 2024-03-21T10:30:50.322 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:50.319+0000 7fec28c00640 1 -- 172.21.15.47:0/2319758189 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 32) v1 ==== 7942+0+0 (secure 0 0 0) 0x7fec340b5210 con 0x561c7c0a9c00 2024-03-21T10:30:50.322 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:50.319+0000 7fec44612740 10 client.17358 fetch_fsmap finished waiting for FSMap version 32 2024-03-21T10:30:50.322 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:50.319+0000 7fec44612740 10 client.17358 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-03-21T10:30:50.322 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:50.319+0000 7fec44612740 20 client.17358 populate_metadata read hostname 'smithi047' 2024-03-21T10:30:50.322 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:50.319+0000 7fec44612740 1 --2- 172.21.15.47:0/2319758189 >> [v2:172.21.15.100:6836/3497644162,v1:172.21.15.100:6837/3497644162] conn(0x561c7c0b9a80 0x561c7c0bbec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:30:50.322 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:50.319+0000 7fec44612740 4 client.17358 mds_command: new command op to 34493 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T10:30:50.323 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:50.319+0000 7fec44612740 1 -- 172.21.15.47:0/2319758189 --> [v2:172.21.15.100:6836/3497644162,v1:172.21.15.100:6837/3497644162] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x561c7c011800 con 0x561c7c0b9a80 2024-03-21T10:30:50.323 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:50.320+0000 7fec3aa00640 1 --2- 172.21.15.47:0/2319758189 >> [v2:172.21.15.100:6836/3497644162,v1:172.21.15.100:6837/3497644162] conn(0x561c7c0b9a80 0x561c7c0bbec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:30:50.323 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:50.321+0000 7fec3aa00640 1 --2- 172.21.15.47:0/2319758189 >> [v2:172.21.15.100:6836/3497644162,v1:172.21.15.100:6837/3497644162] conn(0x561c7c0b9a80 0x561c7c0bbec0 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=befc62dadb8d7566 server_cookie=e409427027b0406b in_seq=0 out_seq=0 2024-03-21T10:30:50.323 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:50.321+0000 7fec28c00640 10 client.17358 ms_handle_connect on v2:172.21.15.100:6836/3497644162 2024-03-21T10:30:50.324 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:50.322+0000 7fec28c00640 1 -- 172.21.15.47:0/2319758189 <== mds.1 v2:172.21.15.100:6836/3497644162 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x561c7c011800 con 0x561c7c0b9a80 2024-03-21T10:30:50.324 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:50.322+0000 7fec28c00640 10 client.17358 handle_command_reply: tid=0 2024-03-21T10:30:50.324 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:50.322+0000 7fec44612740 1 client.17358 shutdown 2024-03-21T10:30:50.324 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:50.322+0000 7fec44612740 1 -- 172.21.15.47:0/2319758189 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561c7c0a9c00 msgr2=0x561c7c0a6e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:30:50.324 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:50.322+0000 7fec44612740 1 --2- 172.21.15.47:0/2319758189 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561c7c0a9c00 0x561c7c0a6e50 secure :-1 s=READY pgs=913 cs=0 l=1 rev1=1 crypto rx=0x7fec34005ba0 tx=0x7fec3409c010 comp rx=0 tx=0).stop 2024-03-21T10:30:50.324 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:50.322+0000 7fec44612740 1 -- 172.21.15.47:0/2319758189 shutdown_connections 2024-03-21T10:30:50.324 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:50.322+0000 7fec44612740 1 --2- 172.21.15.47:0/2319758189 >> [v2:172.21.15.100:6836/3497644162,v1:172.21.15.100:6837/3497644162] conn(0x561c7c0b9a80 0x561c7c0bbec0 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:30:50.324 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:50.322+0000 7fec44612740 1 --2- 172.21.15.47:0/2319758189 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561c7c0a9c00 0x561c7c0a6e50 unknown :-1 s=CLOSED pgs=913 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:30:50.325 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:50.322+0000 7fec44612740 1 --2- 172.21.15.47:0/2319758189 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561c7c0a1080 0x561c7c0a68b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:30:50.325 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:50.322+0000 7fec44612740 1 --2- 172.21.15.47:0/2319758189 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561c7c0a06d0 0x561c7c0a9520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:30:50.325 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:50.322+0000 7fec28c00640 0 client.17358 ms_handle_reset on v2:172.21.15.100:6836/3497644162 2024-03-21T10:30:50.325 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:50.322+0000 7fec44612740 1 -- 172.21.15.47:0/2319758189 >> 172.21.15.47:0/2319758189 conn(0x561c7c09ada0 msgr2=0x561c7c09caf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:30:50.325 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:50.322+0000 7fec44612740 1 -- 172.21.15.47:0/2319758189 shutdown_connections 2024-03-21T10:30:50.325 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:50.322+0000 7fec44612740 1 -- 172.21.15.47:0/2319758189 wait complete. 2024-03-21T10:30:50.325 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:50.323+0000 7fec44612740 20 client.17358 trim_cache size 0 max 16384 2024-03-21T10:30:50.335 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:50.336+0000 7fec44612740 1 Processor -- start 2024-03-21T10:30:50.336 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:50.336+0000 7fec44612740 1 -- start start 2024-03-21T10:30:50.336 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:50.337+0000 7fec44612740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561c7c0a06d0 0x561c7c0a8fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:30:50.336 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:50.337+0000 7fec44612740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561c7c0a1080 0x561c7c0a9520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:30:50.336 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:50.337+0000 7fec44612740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561c7c0a9c00 0x561c7c0a3b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:30:50.336 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:50.337+0000 7fec44612740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec3c1b0a10 con 0x561c7c0a06d0 2024-03-21T10:30:50.336 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:50.337+0000 7fec44612740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec3c1b2390 con 0x561c7c0a9c00 2024-03-21T10:30:50.336 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:50.337+0000 7fec44612740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec3c1b1070 con 0x561c7c0a1080 2024-03-21T10:30:50.336 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:50.337+0000 7fec3aa00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561c7c0a06d0 0x561c7c0a8fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:30:50.336 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:50.337+0000 7fec3aa00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561c7c0a06d0 0x561c7c0a8fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:46516/0 (socket says 172.21.15.47:46516) 2024-03-21T10:30:50.336 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:50.337+0000 7fec3aa00640 1 -- 172.21.15.47:0/3702748501 learned_addr learned my addr 172.21.15.47:0/3702748501 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:30:50.336 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:50.337+0000 7fec3b400640 1 --2- 172.21.15.47:0/3702748501 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561c7c0a9c00 0x561c7c0a3b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:30:50.338 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:50.337+0000 7fec33e00640 1 --2- 172.21.15.47:0/3702748501 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561c7c0a1080 0x561c7c0a9520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:30:50.338 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:50.337+0000 7fec3aa00640 1 -- 172.21.15.47:0/3702748501 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561c7c0a1080 msgr2=0x561c7c0a9520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:30:50.338 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:50.337+0000 7fec3aa00640 1 --2- 172.21.15.47:0/3702748501 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561c7c0a1080 0x561c7c0a9520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:30:50.338 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:50.337+0000 7fec3aa00640 1 -- 172.21.15.47:0/3702748501 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561c7c0a9c00 msgr2=0x561c7c0a3b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:30:50.338 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:50.337+0000 7fec3aa00640 1 --2- 172.21.15.47:0/3702748501 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561c7c0a9c00 0x561c7c0a3b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:30:50.338 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:50.337+0000 7fec3aa00640 1 -- 172.21.15.47:0/3702748501 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561c7c0a4e00 con 0x561c7c0a06d0 2024-03-21T10:30:50.338 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:50.337+0000 7fec3b400640 1 --2- 172.21.15.47:0/3702748501 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561c7c0a9c00 0x561c7c0a3b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T10:30:50.338 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:50.338+0000 7fec3aa00640 1 --2- 172.21.15.47:0/3702748501 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561c7c0a06d0 0x561c7c0a8fe0 secure :-1 s=READY pgs=914 cs=0 l=1 rev1=1 crypto rx=0x7fec2c009230 tx=0x7fec2c009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7b06411ec3237666 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:30:50.338 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:50.338+0000 7fec2b400640 1 -- 172.21.15.47:0/3702748501 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec2c02ca80 con 0x561c7c0a06d0 2024-03-21T10:30:50.338 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:50.338+0000 7fec2b400640 1 -- 172.21.15.47:0/3702748501 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fec2c005640 con 0x561c7c0a06d0 2024-03-21T10:30:50.338 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:50.338+0000 7fec2b400640 1 -- 172.21.15.47:0/3702748501 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec2c0346a0 con 0x561c7c0a06d0 2024-03-21T10:30:50.338 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:50.339+0000 7fec44612740 1 -- 172.21.15.47:0/3702748501 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561c7c0a06d0 msgr2=0x561c7c0a8fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:30:50.339 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:50.339+0000 7fec44612740 1 --2- 172.21.15.47:0/3702748501 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561c7c0a06d0 0x561c7c0a8fe0 secure :-1 s=READY pgs=914 cs=0 l=1 rev1=1 crypto rx=0x7fec2c009230 tx=0x7fec2c009260 comp rx=0 tx=0).stop 2024-03-21T10:30:50.339 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:50.339+0000 7fec44612740 1 -- 172.21.15.47:0/3702748501 shutdown_connections 2024-03-21T10:30:50.339 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:50.339+0000 7fec44612740 1 --2- 172.21.15.47:0/3702748501 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561c7c0a9c00 0x561c7c0a3b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:30:50.339 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:50.339+0000 7fec44612740 1 --2- 172.21.15.47:0/3702748501 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561c7c0a1080 0x561c7c0a9520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:30:50.339 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:50.339+0000 7fec44612740 1 --2- 172.21.15.47:0/3702748501 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561c7c0a06d0 0x561c7c0a8fe0 unknown :-1 s=CLOSED pgs=914 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:30:50.339 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:50.339+0000 7fec44612740 1 -- 172.21.15.47:0/3702748501 >> 172.21.15.47:0/3702748501 conn(0x561c7c09ada0 msgr2=0x561c7c099eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:30:50.339 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:50.339+0000 7fec44612740 1 -- 172.21.15.47:0/3702748501 shutdown_connections 2024-03-21T10:30:50.339 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:50.339+0000 7fec44612740 1 -- 172.21.15.47:0/3702748501 wait complete. 2024-03-21T10:30:50.341 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:50.339+0000 7fec44612740 1 -- start start 2024-03-21T10:30:50.341 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:50.339+0000 7fec44612740 1 Processor -- start 2024-03-21T10:30:50.341 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:50.340+0000 7fec44612740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561c7c0a06d0 0x561c7c0b1ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:30:50.341 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:50.340+0000 7fec44612740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561c7c0a1080 0x561c7c0b0040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:30:50.341 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:50.340+0000 7fec44612740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561c7c0a9c00 0x561c7c0b05c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:30:50.341 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:50.340+0000 7fec44612740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec3c1b1d30 con 0x561c7c0a06d0 2024-03-21T10:30:50.341 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:50.340+0000 7fec44612740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec3c1b16d0 con 0x561c7c0a1080 2024-03-21T10:30:50.341 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:50.340+0000 7fec44612740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec3c1aebd0 con 0x561c7c0a9c00 2024-03-21T10:30:50.341 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:50.340+0000 7fec44612740 10 client.0 _ll_register_callbacks cb 0x7fec34004a40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T10:30:50.341 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:50.340+0000 7fec3aa00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561c7c0a06d0 0x561c7c0b1ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:30:50.341 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:50.340+0000 7fec3aa00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561c7c0a06d0 0x561c7c0b1ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:46524/0 (socket says 172.21.15.47:46524) 2024-03-21T10:30:50.342 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:50.340+0000 7fec3aa00640 1 -- 172.21.15.47:0/2229125499 learned_addr learned my addr 172.21.15.47:0/2229125499 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:30:50.342 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:50.341+0000 7fec33e00640 1 --2- 172.21.15.47:0/2229125499 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561c7c0a1080 0x561c7c0b0040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:30:50.342 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:50.341+0000 7fec3b400640 1 --2- 172.21.15.47:0/2229125499 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561c7c0a9c00 0x561c7c0b05c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:30:50.342 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:50.341+0000 7fec3aa00640 1 -- 172.21.15.47:0/2229125499 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561c7c0a9c00 msgr2=0x561c7c0b05c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:30:50.342 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:50.341+0000 7fec3aa00640 1 --2- 172.21.15.47:0/2229125499 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561c7c0a9c00 0x561c7c0b05c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:30:50.342 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:50.341+0000 7fec3aa00640 1 -- 172.21.15.47:0/2229125499 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561c7c0a1080 msgr2=0x561c7c0b0040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:30:50.342 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:50.341+0000 7fec3aa00640 1 --2- 172.21.15.47:0/2229125499 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561c7c0a1080 0x561c7c0b0040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:30:50.342 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:50.341+0000 7fec3aa00640 1 -- 172.21.15.47:0/2229125499 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x561c7c0b0df0 con 0x561c7c0a06d0 2024-03-21T10:30:50.342 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:50.341+0000 7fec3b400640 1 --2- 172.21.15.47:0/2229125499 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561c7c0a9c00 0x561c7c0b05c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T10:30:50.342 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:50.341+0000 7fec3aa00640 1 --2- 172.21.15.47:0/2229125499 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561c7c0a06d0 0x561c7c0b1ca0 secure :-1 s=READY pgs=915 cs=0 l=1 rev1=1 crypto rx=0x7fec2c000c00 tx=0x7fec2c051780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7c125df9c367dfc7 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:30:50.342 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:50.341+0000 7fec28c00640 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-03-21T10:30:50.343 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:50.344+0000 7fec28c00640 1 -- 172.21.15.47:0/2229125499 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec2c004450 con 0x561c7c0a06d0 2024-03-21T10:30:50.344 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:50.344+0000 7fec28c00640 1 -- 172.21.15.47:0/2229125499 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(97..97 src has 1..97) v4 ==== 9474+0+0 (secure 0 0 0) 0x7fec2c02c880 con 0x561c7c0a06d0 2024-03-21T10:30:50.344 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:50.345+0000 7fec44612740 1 -- 172.21.15.47:0/2229125499 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fec2c005640 con 0x561c7c0a06d0 2024-03-21T10:30:50.344 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:50.345+0000 7fec28c00640 1 -- 172.21.15.47:0/2229125499 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7fec2c062050 con 0x561c7c0a06d0 2024-03-21T10:30:50.344 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:50.345+0000 7fec44612740 10 client.17364 fetch_fsmap learned FSMap version 32 2024-03-21T10:30:50.344 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:50.345+0000 7fec44612740 1 -- 172.21.15.47:0/2229125499 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x561c7c116410 con 0x561c7c0a06d0 2024-03-21T10:30:50.345 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:50.345+0000 7fec28c00640 1 -- 172.21.15.47:0/2229125499 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fec2c005640 con 0x561c7c0a06d0 2024-03-21T10:30:50.345 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:50.345+0000 7fec28c00640 1 -- 172.21.15.47:0/2229125499 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 32) v1 ==== 7942+0+0 (secure 0 0 0) 0x7fec2c003c90 con 0x561c7c0a06d0 2024-03-21T10:30:50.345 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:50.345+0000 7fec44612740 10 client.17364 fetch_fsmap finished waiting for FSMap version 32 2024-03-21T10:30:50.345 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:50.345+0000 7fec44612740 10 client.17364 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-03-21T10:30:50.345 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:50.345+0000 7fec44612740 20 client.17364 populate_metadata read hostname 'smithi047' 2024-03-21T10:30:50.345 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:50.345+0000 7fec44612740 1 --2- 172.21.15.47:0/2229125499 >> [v2:172.21.15.100:6836/3497644162,v1:172.21.15.100:6837/3497644162] conn(0x561c7c1166a0 0x561c7c118a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:30:50.345 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:50.345+0000 7fec44612740 4 client.17364 mds_command: new command op to 34493 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-21T10:30:50.345 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:50.345+0000 7fec44612740 1 -- 172.21.15.47:0/2229125499 --> [v2:172.21.15.100:6836/3497644162,v1:172.21.15.100:6837/3497644162] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x561c7c00bb50 con 0x561c7c1166a0 2024-03-21T10:30:50.345 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:50.346+0000 7fec33e00640 1 --2- 172.21.15.47:0/2229125499 >> [v2:172.21.15.100:6836/3497644162,v1:172.21.15.100:6837/3497644162] conn(0x561c7c1166a0 0x561c7c118a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:30:50.346 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:50.347+0000 7fec33e00640 1 --2- 172.21.15.47:0/2229125499 >> [v2:172.21.15.100:6836/3497644162,v1:172.21.15.100:6837/3497644162] conn(0x561c7c1166a0 0x561c7c118a90 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=996d3ad9beb2f9f0 server_cookie=2ccf29eaf6581d7 in_seq=0 out_seq=0 2024-03-21T10:30:50.346 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:50.347+0000 7fec28c00640 10 client.17364 ms_handle_connect on v2:172.21.15.100:6836/3497644162 2024-03-21T10:30:50.347 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:50.347+0000 7fec28c00640 1 -- 172.21.15.47:0/2229125499 <== mds.1 v2:172.21.15.100:6836/3497644162 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x561c7c00bb50 con 0x561c7c1166a0 2024-03-21T10:30:50.347 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:50.347+0000 7fec28c00640 10 client.17364 handle_command_reply: tid=0 2024-03-21T10:30:50.347 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:50.347+0000 7fec44612740 1 client.17364 shutdown 2024-03-21T10:30:50.347 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:50.347+0000 7fec44612740 1 -- 172.21.15.47:0/2229125499 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561c7c0a06d0 msgr2=0x561c7c0b1ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:30:50.347 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:50.347+0000 7fec44612740 1 --2- 172.21.15.47:0/2229125499 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561c7c0a06d0 0x561c7c0b1ca0 secure :-1 s=READY pgs=915 cs=0 l=1 rev1=1 crypto rx=0x7fec2c000c00 tx=0x7fec2c051780 comp rx=0 tx=0).stop 2024-03-21T10:30:50.347 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:50.348+0000 7fec44612740 1 -- 172.21.15.47:0/2229125499 shutdown_connections 2024-03-21T10:30:50.347 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:50.348+0000 7fec44612740 1 --2- 172.21.15.47:0/2229125499 >> [v2:172.21.15.100:6836/3497644162,v1:172.21.15.100:6837/3497644162] conn(0x561c7c1166a0 0x561c7c118a90 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:30:50.347 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:50.348+0000 7fec44612740 1 --2- 172.21.15.47:0/2229125499 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561c7c0a9c00 0x561c7c0b05c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:30:50.347 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:50.348+0000 7fec44612740 1 --2- 172.21.15.47:0/2229125499 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561c7c0a1080 0x561c7c0b0040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:30:50.347 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:50.348+0000 7fec44612740 1 --2- 172.21.15.47:0/2229125499 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561c7c0a06d0 0x561c7c0b1ca0 unknown :-1 s=CLOSED pgs=915 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:30:50.348 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:50.348+0000 7fec28c00640 0 client.17364 ms_handle_reset on v2:172.21.15.100:6836/3497644162 2024-03-21T10:30:50.348 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:50.348+0000 7fec44612740 1 -- 172.21.15.47:0/2229125499 >> 172.21.15.47:0/2229125499 conn(0x561c7c09ada0 msgr2=0x561c7c0a1460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:30:50.348 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:50.348+0000 7fec44612740 1 -- 172.21.15.47:0/2229125499 shutdown_connections 2024-03-21T10:30:50.348 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:50.348+0000 7fec44612740 1 -- 172.21.15.47:0/2229125499 wait complete. 2024-03-21T10:30:50.348 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:50.348+0000 7fec44612740 20 client.17364 trim_cache size 0 max 16384 2024-03-21T10:30:50.348 INFO:teuthology.orchestra.run.smithi047.stdout:[] 2024-03-21T10:30:50.349 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:50.349+0000 7fec41a00640 1 -- 172.21.15.47:0/2954333324 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7fec200a3090 msgr2=0x7fec200a5550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:30:50.349 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:50.349+0000 7fec41a00640 1 --2- 172.21.15.47:0/2954333324 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7fec200a3090 0x7fec200a5550 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fec24005d00 tx=0x7fec24015040 comp rx=0 tx=0).stop 2024-03-21T10:30:50.349 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:50.349+0000 7fec41a00640 1 -- 172.21.15.47:0/2954333324 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fec3c10bcb0 msgr2=0x7fec3c100fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:30:50.349 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:50.349+0000 7fec41a00640 1 --2- 172.21.15.47:0/2954333324 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fec3c10bcb0 0x7fec3c100fb0 secure :-1 s=READY pgs=911 cs=0 l=1 rev1=1 crypto rx=0x7fec3400eb40 tx=0x7fec3400c2d0 comp rx=0 tx=0).stop 2024-03-21T10:30:50.349 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:50.349+0000 7fec41a00640 1 -- 172.21.15.47:0/2954333324 shutdown_connections 2024-03-21T10:30:50.349 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:50.349+0000 7fec41a00640 1 --2- 172.21.15.47:0/2954333324 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fec3c10bcb0 0x7fec3c100fb0 unknown :-1 s=CLOSED pgs=911 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:30:50.349 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:50.349+0000 7fec41a00640 1 --2- 172.21.15.47:0/2954333324 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fec3c103200 0x7fec3c100a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:30:50.349 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:50.349+0000 7fec41a00640 1 --2- 172.21.15.47:0/2954333324 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7fec200a3090 0x7fec200a5550 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:30:50.349 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:50.349+0000 7fec41a00640 1 --2- 172.21.15.47:0/2954333324 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fec3c1028e0 0x7fec3c100530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:30:50.349 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:50.349+0000 7fec41a00640 1 -- 172.21.15.47:0/2954333324 >> 172.21.15.47:0/2954333324 conn(0x7fec3c076920 msgr2=0x7fec3c0fd4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:30:50.350 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:50.349+0000 7fec41a00640 1 -- 172.21.15.47:0/2954333324 shutdown_connections 2024-03-21T10:30:50.350 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:50.349+0000 7fec41a00640 1 -- 172.21.15.47:0/2954333324 wait complete. 2024-03-21T10:30:50.366 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-21T10:30:50.367 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-03-21T10:30:50.572 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:50.572+0000 7f6da1c00640 1 -- 172.21.15.47:0/878617392 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6d9c068880 msgr2=0x7f6d9c068d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:30:50.572 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:50.572+0000 7f6da1c00640 1 --2- 172.21.15.47:0/878617392 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6d9c068880 0x7f6d9c068d00 secure :-1 s=READY pgs=916 cs=0 l=1 rev1=1 crypto rx=0x7f6d84009930 tx=0x7f6d8402bc30 comp rx=0 tx=0).stop 2024-03-21T10:30:50.572 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:50.572+0000 7f6da1c00640 1 -- 172.21.15.47:0/878617392 shutdown_connections 2024-03-21T10:30:50.572 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:50.572+0000 7f6da1c00640 1 --2- 172.21.15.47:0/878617392 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6d9c069240 0x7f6d9c10e4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:30:50.572 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:50.572+0000 7f6da1c00640 1 --2- 172.21.15.47:0/878617392 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6d9c068880 0x7f6d9c068d00 unknown :-1 s=CLOSED pgs=916 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:30:50.572 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:50.572+0000 7f6da1c00640 1 --2- 172.21.15.47:0/878617392 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6d9c1076b0 0x7f6d9c107a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:30:50.572 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:50.572+0000 7f6da1c00640 1 -- 172.21.15.47:0/878617392 >> 172.21.15.47:0/878617392 conn(0x7f6d9c0776a0 msgr2=0x7f6d9c077ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:30:50.572 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:50.573+0000 7f6da1c00640 1 -- 172.21.15.47:0/878617392 shutdown_connections 2024-03-21T10:30:50.572 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:50.573+0000 7f6da1c00640 1 -- 172.21.15.47:0/878617392 wait complete. 2024-03-21T10:30:50.574 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:50.574+0000 7f6da1c00640 1 Processor -- start 2024-03-21T10:30:50.575 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:50.574+0000 7f6da1c00640 1 -- start start 2024-03-21T10:30:50.575 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:50.574+0000 7f6da1c00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6d9c068880 0x7f6d9c197e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:30:50.575 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:50.574+0000 7f6da1c00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6d9c069240 0x7f6d9c1983d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:30:50.575 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:50.574+0000 7f6da1c00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6d9c1076b0 0x7f6d9c19c700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:30:50.575 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:50.574+0000 7f6da1c00640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d9c111930 con 0x7f6d9c068880 2024-03-21T10:30:50.575 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:50.574+0000 7f6da1c00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d9c1117b0 con 0x7f6d9c1076b0 2024-03-21T10:30:50.575 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:50.574+0000 7f6da1c00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d9c111ab0 con 0x7f6d9c069240 2024-03-21T10:30:50.575 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:50.574+0000 7f6d9aa00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6d9c068880 0x7f6d9c197e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:30:50.575 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:50.575+0000 7f6d9aa00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6d9c068880 0x7f6d9c197e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:46552/0 (socket says 172.21.15.47:46552) 2024-03-21T10:30:50.575 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:50.575+0000 7f6d9aa00640 1 -- 172.21.15.47:0/1745291081 learned_addr learned my addr 172.21.15.47:0/1745291081 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:30:50.575 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:50.575+0000 7f6d9b400640 1 --2- 172.21.15.47:0/1745291081 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6d9c1076b0 0x7f6d9c19c700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:30:50.575 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:50.575+0000 7f6d92000640 1 --2- 172.21.15.47:0/1745291081 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6d9c069240 0x7f6d9c1983d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:30:50.575 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:50.575+0000 7f6d9aa00640 1 -- 172.21.15.47:0/1745291081 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6d9c069240 msgr2=0x7f6d9c1983d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:30:50.575 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:50.575+0000 7f6d9aa00640 1 --2- 172.21.15.47:0/1745291081 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6d9c069240 0x7f6d9c1983d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:30:50.576 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:50.575+0000 7f6d9aa00640 1 -- 172.21.15.47:0/1745291081 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6d9c1076b0 msgr2=0x7f6d9c19c700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:30:50.576 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:50.575+0000 7f6d9aa00640 1 --2- 172.21.15.47:0/1745291081 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6d9c1076b0 0x7f6d9c19c700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:30:50.576 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:50.575+0000 7f6d9aa00640 1 -- 172.21.15.47:0/1745291081 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d9c19ce30 con 0x7f6d9c068880 2024-03-21T10:30:50.576 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:50.575+0000 7f6d92000640 1 --2- 172.21.15.47:0/1745291081 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6d9c069240 0x7f6d9c1983d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T10:30:50.576 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:50.575+0000 7f6d9aa00640 1 --2- 172.21.15.47:0/1745291081 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6d9c068880 0x7f6d9c197e90 secure :-1 s=READY pgs=917 cs=0 l=1 rev1=1 crypto rx=0x7f6d8c00d950 tx=0x7f6d8c00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:30:50.576 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:50.576+0000 7f6d93e00640 1 -- 172.21.15.47:0/1745291081 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d8c017070 con 0x7f6d9c068880 2024-03-21T10:30:50.576 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:50.576+0000 7f6d93e00640 1 -- 172.21.15.47:0/1745291081 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f6d8c009c30 con 0x7f6d9c068880 2024-03-21T10:30:50.576 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:50.576+0000 7f6da1c00640 1 -- 172.21.15.47:0/1745291081 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6d9c19d120 con 0x7f6d9c068880 2024-03-21T10:30:50.576 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:50.576+0000 7f6d93e00640 1 -- 172.21.15.47:0/1745291081 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d8c00f620 con 0x7f6d9c068880 2024-03-21T10:30:50.576 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:50.576+0000 7f6da1c00640 1 -- 172.21.15.47:0/1745291081 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6d9c1a49b0 con 0x7f6d9c068880 2024-03-21T10:30:50.578 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:50.579+0000 7f6da49c6740 1 Processor -- start 2024-03-21T10:30:50.578 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:50.579+0000 7f6da49c6740 1 -- start start 2024-03-21T10:30:50.580 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:50.580+0000 7f6d93e00640 1 -- 172.21.15.47:0/1745291081 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 27) v1 ==== 137985+0+0 (secure 0 0 0) 0x7f6d8c01a070 con 0x7f6d9c068880 2024-03-21T10:30:50.580 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:50.580+0000 7f6da49c6740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55818ee80450 0x55818ee80830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:30:50.580 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:50.580+0000 7f6da49c6740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55818ee80e00 0x55818ee8da40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:30:50.580 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:50.580+0000 7f6da49c6740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55818ee8e010 0x55818ee90420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:30:50.580 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:50.580+0000 7f6da49c6740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x55818ede42c0 con 0x55818ee80450 2024-03-21T10:30:50.580 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:50.580+0000 7f6da49c6740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55818ed8cc90 con 0x55818ee8e010 2024-03-21T10:30:50.580 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:50.580+0000 7f6da49c6740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55818ede0360 con 0x55818ee80e00 2024-03-21T10:30:50.581 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:50.580+0000 7f6d9b400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55818ee80450 0x55818ee80830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:30:50.581 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:50.580+0000 7f6d9b400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55818ee80450 0x55818ee80830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:46568/0 (socket says 172.21.15.47:46568) 2024-03-21T10:30:50.581 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:50.580+0000 7f6d9b400640 1 -- 172.21.15.47:0/3466325442 learned_addr learned my addr 172.21.15.47:0/3466325442 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:30:50.581 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:50.580+0000 7f6d92000640 1 --2- 172.21.15.47:0/3466325442 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55818ee8e010 0x55818ee90420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:30:50.581 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:50.580+0000 7f6d9aa00640 1 --2- 172.21.15.47:0/3466325442 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55818ee80e00 0x55818ee8da40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:30:50.581 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:50.581+0000 7f6d9b400640 1 -- 172.21.15.47:0/3466325442 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55818ee80e00 msgr2=0x55818ee8da40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:30:50.581 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:50.581+0000 7f6d9b400640 1 --2- 172.21.15.47:0/3466325442 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55818ee80e00 0x55818ee8da40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:30:50.581 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:50.581+0000 7f6d9b400640 1 -- 172.21.15.47:0/3466325442 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55818ee8e010 msgr2=0x55818ee90420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:30:50.581 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:50.581+0000 7f6d9b400640 1 --2- 172.21.15.47:0/3466325442 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55818ee8e010 0x55818ee90420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:30:50.581 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:50.581+0000 7f6d9b400640 1 -- 172.21.15.47:0/3466325442 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55818edb1080 con 0x55818ee80450 2024-03-21T10:30:50.581 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:50.581+0000 7f6d9aa00640 1 --2- 172.21.15.47:0/3466325442 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55818ee80e00 0x55818ee8da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T10:30:50.581 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:50.581+0000 7f6d9b400640 1 --2- 172.21.15.47:0/3466325442 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55818ee80450 0x55818ee80830 secure :-1 s=READY pgs=918 cs=0 l=1 rev1=1 crypto rx=0x7f6d94009fc0 tx=0x7f6d94010860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c309678e30b8f198 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:30:50.581 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:50.581+0000 7f6d8be00640 1 -- 172.21.15.47:0/3466325442 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d94003b70 con 0x55818ee80450 2024-03-21T10:30:50.581 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:50.581+0000 7f6d8be00640 1 -- 172.21.15.47:0/3466325442 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f6d94003d10 con 0x55818ee80450 2024-03-21T10:30:50.582 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:50.581+0000 7f6d8be00640 1 -- 172.21.15.47:0/3466325442 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d94011a40 con 0x55818ee80450 2024-03-21T10:30:50.584 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:50.582+0000 7f6d93e00640 1 --2- 172.21.15.47:0/1745291081 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f6d800a3090 0x7f6d800a5550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:30:50.584 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:50.582+0000 7f6d93e00640 1 -- 172.21.15.47:0/1745291081 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 9474+0+0 (secure 0 0 0) 0x7f6d8c0cf6d0 con 0x7f6d9c068880 2024-03-21T10:30:50.584 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:50.582+0000 7f6d9aa00640 1 --2- 172.21.15.47:0/1745291081 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f6d800a3090 0x7f6d800a5550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:30:50.585 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:50.582+0000 7f6da49c6740 1 -- 172.21.15.47:0/3466325442 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55818ee80450 msgr2=0x55818ee80830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:30:50.585 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:50.582+0000 7f6da49c6740 1 --2- 172.21.15.47:0/3466325442 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55818ee80450 0x55818ee80830 secure :-1 s=READY pgs=918 cs=0 l=1 rev1=1 crypto rx=0x7f6d94009fc0 tx=0x7f6d94010860 comp rx=0 tx=0).stop 2024-03-21T10:30:50.585 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:50.582+0000 7f6da49c6740 1 -- 172.21.15.47:0/3466325442 shutdown_connections 2024-03-21T10:30:50.585 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:50.582+0000 7f6da49c6740 1 --2- 172.21.15.47:0/3466325442 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55818ee8e010 0x55818ee90420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:30:50.585 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:50.582+0000 7f6da49c6740 1 --2- 172.21.15.47:0/3466325442 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55818ee80e00 0x55818ee8da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:30:50.585 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:50.582+0000 7f6da49c6740 1 --2- 172.21.15.47:0/3466325442 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55818ee80450 0x55818ee80830 unknown :-1 s=CLOSED pgs=918 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:30:50.585 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:50.582+0000 7f6da49c6740 1 -- 172.21.15.47:0/3466325442 >> 172.21.15.47:0/3466325442 conn(0x55818ee7ada0 msgr2=0x55818ee7b180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:30:50.585 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:50.583+0000 7f6da49c6740 1 -- 172.21.15.47:0/3466325442 shutdown_connections 2024-03-21T10:30:50.585 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:50.583+0000 7f6da49c6740 1 -- 172.21.15.47:0/3466325442 wait complete. 2024-03-21T10:30:50.585 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:50.583+0000 7f6d9aa00640 1 --2- 172.21.15.47:0/1745291081 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f6d800a3090 0x7f6d800a5550 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f6d8c0ccb10 tx=0x7f6d8c005f30 comp rx=0 tx=0).ready entity=mgr.15687 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:30:50.585 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:50.583+0000 7f6da49c6740 1 -- start start 2024-03-21T10:30:50.585 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:50.583+0000 7f6da49c6740 1 Processor -- start 2024-03-21T10:30:50.585 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:50.584+0000 7f6da49c6740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55818ee80450 0x55818ee8d9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:30:50.586 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:50.584+0000 7f6da49c6740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55818ee80e00 0x55818ee8ad40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:30:50.586 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:50.584+0000 7f6da49c6740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55818ee8e010 0x55818ee8b480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:30:50.586 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:50.584+0000 7f6da49c6740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x55818ee90f40 con 0x55818ee8e010 2024-03-21T10:30:50.586 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:50.584+0000 7f6da49c6740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55818eda00d0 con 0x55818ee80450 2024-03-21T10:30:50.586 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:50.584+0000 7f6da49c6740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55818ee910c0 con 0x55818ee80e00 2024-03-21T10:30:50.586 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:50.584+0000 7f6da49c6740 10 client.0 _ll_register_callbacks cb 0x55818eda4e80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T10:30:50.586 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:50.584+0000 7f6d9b400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55818ee8e010 0x55818ee8b480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:30:50.586 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:50.584+0000 7f6d9b400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55818ee8e010 0x55818ee8b480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:46582/0 (socket says 172.21.15.47:46582) 2024-03-21T10:30:50.586 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:50.584+0000 7f6d9b400640 1 -- 172.21.15.47:0/313562059 learned_addr learned my addr 172.21.15.47:0/313562059 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:30:50.586 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:50.585+0000 7f6d92000640 1 --2- 172.21.15.47:0/313562059 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55818ee80e00 0x55818ee8ad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:30:50.586 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:50.585+0000 7f6d9b400640 1 --2- 172.21.15.47:0/313562059 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55818ee80450 0x55818ee8d9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:30:50.586 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:50.585+0000 7f6d9b400640 1 -- 172.21.15.47:0/313562059 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55818ee80e00 msgr2=0x55818ee8ad40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:30:50.586 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:50.585+0000 7f6d9b400640 1 --2- 172.21.15.47:0/313562059 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55818ee80e00 0x55818ee8ad40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:30:50.586 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:50.585+0000 7f6d9b400640 1 -- 172.21.15.47:0/313562059 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55818ee80450 msgr2=0x55818ee8d9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:30:50.587 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:50.585+0000 7f6d9b400640 1 --2- 172.21.15.47:0/313562059 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55818ee80450 0x55818ee8d9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:30:50.587 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:50.585+0000 7f6d9b400640 1 -- 172.21.15.47:0/313562059 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55818eaf83f0 con 0x55818ee8e010 2024-03-21T10:30:50.587 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:50.585+0000 7f6d9b400640 1 --2- 172.21.15.47:0/313562059 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55818ee8e010 0x55818ee8b480 secure :-1 s=READY pgs=919 cs=0 l=1 rev1=1 crypto rx=0x7f6d94010d60 tx=0x7f6d94013040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=db2994d5a13489b server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:30:50.587 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:50.585+0000 7f6d88c00640 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-03-21T10:30:50.587 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:50.586+0000 7f6d88c00640 1 -- 172.21.15.47:0/313562059 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d94004c50 con 0x55818ee8e010 2024-03-21T10:30:50.587 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:50.586+0000 7f6da49c6740 1 -- 172.21.15.47:0/313562059 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f6d94003d10 con 0x55818ee8e010 2024-03-21T10:30:50.587 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:50.586+0000 7f6d88c00640 1 -- 172.21.15.47:0/313562059 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(97..97 src has 1..97) v4 ==== 9474+0+0 (secure 0 0 0) 0x7f6d94004df0 con 0x55818ee8e010 2024-03-21T10:30:50.589 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:50.586+0000 7f6d88c00640 1 -- 172.21.15.47:0/313562059 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6d94032050 con 0x55818ee8e010 2024-03-21T10:30:50.589 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:50.586+0000 7f6da49c6740 10 client.17376 fetch_fsmap learned FSMap version 32 2024-03-21T10:30:50.589 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:50.586+0000 7f6da49c6740 1 -- 172.21.15.47:0/313562059 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x55818ebc27d0 con 0x55818ee8e010 2024-03-21T10:30:50.589 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:50.586+0000 7f6d88c00640 1 -- 172.21.15.47:0/313562059 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f6d94003d10 con 0x55818ee8e010 2024-03-21T10:30:50.589 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:50.587+0000 7f6d88c00640 1 -- 172.21.15.47:0/313562059 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 32) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f6d94026070 con 0x55818ee8e010 2024-03-21T10:30:50.590 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:50.587+0000 7f6da49c6740 10 client.17376 fetch_fsmap finished waiting for FSMap version 32 2024-03-21T10:30:50.590 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:50.587+0000 7f6da49c6740 10 client.17376 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-21T10:30:50.590 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:50.587+0000 7f6da49c6740 20 client.17376 populate_metadata read hostname 'smithi047' 2024-03-21T10:30:50.590 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:50.587+0000 7f6da49c6740 1 --2- 172.21.15.47:0/313562059 >> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] conn(0x55818ee9dd00 0x55818eea00f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:30:50.590 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:50.587+0000 7f6da49c6740 4 client.17376 mds_command: new command op to 34535 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T10:30:50.590 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:50.587+0000 7f6da49c6740 1 -- 172.21.15.47:0/313562059 --> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55818edf1800 con 0x55818ee9dd00 2024-03-21T10:30:50.590 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:50.587+0000 7f6d92000640 1 --2- 172.21.15.47:0/313562059 >> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] conn(0x55818ee9dd00 0x55818eea00f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:30:50.590 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:50.588+0000 7f6d92000640 1 --2- 172.21.15.47:0/313562059 >> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] conn(0x55818ee9dd00 0x55818eea00f0 crc :-1 s=READY pgs=147 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=763281b382fa19df server_cookie=b0ddcf2e21252fe8 in_seq=0 out_seq=0 2024-03-21T10:30:50.590 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:50.588+0000 7f6d88c00640 10 client.17376 ms_handle_connect on v2:172.21.15.100:6838/4173168440 2024-03-21T10:30:50.590 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:50.589+0000 7f6d88c00640 1 -- 172.21.15.47:0/313562059 <== mds.0 v2:172.21.15.100:6838/4173168440 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55818edf1800 con 0x55818ee9dd00 2024-03-21T10:30:50.590 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:50.589+0000 7f6d88c00640 10 client.17376 handle_command_reply: tid=0 2024-03-21T10:30:50.590 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:50.589+0000 7f6da49c6740 1 client.17376 shutdown 2024-03-21T10:30:50.590 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:50.589+0000 7f6da49c6740 1 -- 172.21.15.47:0/313562059 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55818ee8e010 msgr2=0x55818ee8b480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:30:50.590 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:50.589+0000 7f6da49c6740 1 --2- 172.21.15.47:0/313562059 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55818ee8e010 0x55818ee8b480 secure :-1 s=READY pgs=919 cs=0 l=1 rev1=1 crypto rx=0x7f6d94010d60 tx=0x7f6d94013040 comp rx=0 tx=0).stop 2024-03-21T10:30:50.591 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:50.589+0000 7f6da49c6740 1 -- 172.21.15.47:0/313562059 shutdown_connections 2024-03-21T10:30:50.591 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:50.589+0000 7f6da49c6740 1 --2- 172.21.15.47:0/313562059 >> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] conn(0x55818ee9dd00 0x55818eea00f0 crc :-1 s=READY pgs=147 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:30:50.591 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:50.589+0000 7f6da49c6740 1 --2- 172.21.15.47:0/313562059 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55818ee8e010 0x55818ee8b480 unknown :-1 s=CLOSED pgs=919 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:30:50.591 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:50.589+0000 7f6da49c6740 1 --2- 172.21.15.47:0/313562059 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55818ee80e00 0x55818ee8ad40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:30:50.591 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:50.589+0000 7f6da49c6740 1 --2- 172.21.15.47:0/313562059 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55818ee80450 0x55818ee8d9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:30:50.591 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:50.589+0000 7f6da49c6740 1 -- 172.21.15.47:0/313562059 >> 172.21.15.47:0/313562059 conn(0x55818ee7ada0 msgr2=0x55818ee7caf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:30:50.591 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:50.589+0000 7f6d88c00640 0 client.17376 ms_handle_reset on v2:172.21.15.100:6838/4173168440 2024-03-21T10:30:50.591 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:50.590+0000 7f6da49c6740 1 -- 172.21.15.47:0/313562059 shutdown_connections 2024-03-21T10:30:50.591 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:50.590+0000 7f6da49c6740 1 -- 172.21.15.47:0/313562059 wait complete. 2024-03-21T10:30:50.591 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:50.590+0000 7f6da49c6740 20 client.17376 trim_cache size 0 max 16384 2024-03-21T10:30:50.604 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:50.604+0000 7f6da49c6740 1 Processor -- start 2024-03-21T10:30:50.604 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:50.604+0000 7f6da49c6740 1 -- start start 2024-03-21T10:30:50.604 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:50.604+0000 7f6da49c6740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55818ee80450 0x55818ee8a4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:30:50.604 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:50.604+0000 7f6da49c6740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55818ee80e00 0x55818ee8d470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:30:50.604 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:50.604+0000 7f6da49c6740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55818ee8e010 0x55818ee8d9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:30:50.604 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:50.604+0000 7f6da49c6740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d9c1ac7a0 con 0x55818ee80e00 2024-03-21T10:30:50.604 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:50.604+0000 7f6da49c6740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d9c1ae120 con 0x55818ee80450 2024-03-21T10:30:50.604 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:50.604+0000 7f6da49c6740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d9c1ace00 con 0x55818ee8e010 2024-03-21T10:30:50.604 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:50.604+0000 7f6d92000640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55818ee80e00 0x55818ee8d470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:30:50.604 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:50.604+0000 7f6d92000640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55818ee80e00 0x55818ee8d470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:46598/0 (socket says 172.21.15.47:46598) 2024-03-21T10:30:50.604 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:50.604+0000 7f6d92000640 1 -- 172.21.15.47:0/1709319310 learned_addr learned my addr 172.21.15.47:0/1709319310 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:30:50.604 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:50.604+0000 7f6d9b400640 1 --2- 172.21.15.47:0/1709319310 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55818ee80450 0x55818ee8a4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:30:50.605 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:50.605+0000 7f6d92000640 1 -- 172.21.15.47:0/1709319310 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55818ee8e010 msgr2=0x55818ee8d9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:30:50.605 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:50.605+0000 7f6d92000640 1 --2- 172.21.15.47:0/1709319310 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55818ee8e010 0x55818ee8d9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:30:50.605 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:50.605+0000 7f6d92000640 1 -- 172.21.15.47:0/1709319310 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55818ee80450 msgr2=0x55818ee8a4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:30:50.605 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:50.605+0000 7f6d92000640 1 --2- 172.21.15.47:0/1709319310 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55818ee80450 0x55818ee8a4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:30:50.605 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:50.605+0000 7f6d92000640 1 -- 172.21.15.47:0/1709319310 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55818ee7ff90 con 0x55818ee80e00 2024-03-21T10:30:50.605 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:50.605+0000 7f6d92000640 1 --2- 172.21.15.47:0/1709319310 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55818ee80e00 0x55818ee8d470 secure :-1 s=READY pgs=920 cs=0 l=1 rev1=1 crypto rx=0x7f6d8402c210 tx=0x7f6d84005b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=260fb2f3891cff3a server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:30:50.608 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:50.605+0000 7f6d8b400640 1 -- 172.21.15.47:0/1709319310 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d840462d0 con 0x55818ee80e00 2024-03-21T10:30:50.608 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:50.605+0000 7f6d8b400640 1 -- 172.21.15.47:0/1709319310 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f6d8402cc90 con 0x55818ee80e00 2024-03-21T10:30:50.608 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:50.605+0000 7f6d8b400640 1 -- 172.21.15.47:0/1709319310 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d84005470 con 0x55818ee80e00 2024-03-21T10:30:50.608 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:50.606+0000 7f6da49c6740 1 -- 172.21.15.47:0/1709319310 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55818ee80e00 msgr2=0x55818ee8d470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:30:50.608 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:50.606+0000 7f6da49c6740 1 --2- 172.21.15.47:0/1709319310 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55818ee80e00 0x55818ee8d470 secure :-1 s=READY pgs=920 cs=0 l=1 rev1=1 crypto rx=0x7f6d8402c210 tx=0x7f6d84005b50 comp rx=0 tx=0).stop 2024-03-21T10:30:50.608 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:50.606+0000 7f6da49c6740 1 -- 172.21.15.47:0/1709319310 shutdown_connections 2024-03-21T10:30:50.608 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:50.606+0000 7f6da49c6740 1 --2- 172.21.15.47:0/1709319310 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55818ee8e010 0x55818ee8d9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:30:50.608 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:50.606+0000 7f6da49c6740 1 --2- 172.21.15.47:0/1709319310 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55818ee80e00 0x55818ee8d470 unknown :-1 s=CLOSED pgs=920 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:30:50.608 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:50.606+0000 7f6da49c6740 1 --2- 172.21.15.47:0/1709319310 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55818ee80450 0x55818ee8a4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:30:50.608 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:50.606+0000 7f6da49c6740 1 -- 172.21.15.47:0/1709319310 >> 172.21.15.47:0/1709319310 conn(0x55818ee7ada0 msgr2=0x55818ee86270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:30:50.608 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:50.606+0000 7f6da49c6740 1 -- 172.21.15.47:0/1709319310 shutdown_connections 2024-03-21T10:30:50.608 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:50.606+0000 7f6da49c6740 1 -- 172.21.15.47:0/1709319310 wait complete. 2024-03-21T10:30:50.609 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:50.607+0000 7f6da49c6740 1 -- start start 2024-03-21T10:30:50.609 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:50.607+0000 7f6da49c6740 1 Processor -- start 2024-03-21T10:30:50.609 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:50.607+0000 7f6da49c6740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55818ee80450 0x55818ee949d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:30:50.609 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:50.607+0000 7f6da49c6740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55818ee80e00 0x55818ee94f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:30:50.609 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:50.607+0000 7f6da49c6740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55818ee8e010 0x55818ee95450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:30:50.609 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:50.607+0000 7f6da49c6740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d9c1adac0 con 0x55818ee80e00 2024-03-21T10:30:50.609 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:50.607+0000 7f6da49c6740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d9c1ad460 con 0x55818ee80450 2024-03-21T10:30:50.609 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:50.607+0000 7f6da49c6740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d9c1aa960 con 0x55818ee8e010 2024-03-21T10:30:50.609 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:50.607+0000 7f6da49c6740 10 client.0 _ll_register_callbacks cb 0x7f6d940093a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T10:30:50.609 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:50.608+0000 7f6d92000640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55818ee80e00 0x55818ee94f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:30:50.610 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:50.608+0000 7f6d92000640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55818ee80e00 0x55818ee94f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:46612/0 (socket says 172.21.15.47:46612) 2024-03-21T10:30:50.610 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:50.608+0000 7f6d92000640 1 -- 172.21.15.47:0/978245889 learned_addr learned my addr 172.21.15.47:0/978245889 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:30:50.610 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:50.608+0000 7f6d9b400640 1 --2- 172.21.15.47:0/978245889 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55818ee80450 0x55818ee949d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:30:50.610 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:50.608+0000 7f6d92000640 1 -- 172.21.15.47:0/978245889 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55818ee8e010 msgr2=0x55818ee95450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:30:50.610 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:50.608+0000 7f6d92000640 1 --2- 172.21.15.47:0/978245889 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55818ee8e010 0x55818ee95450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:30:50.610 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:50.608+0000 7f6d92000640 1 -- 172.21.15.47:0/978245889 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55818ee80450 msgr2=0x55818ee949d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:30:50.610 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:50.608+0000 7f6d92000640 1 --2- 172.21.15.47:0/978245889 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55818ee80450 0x55818ee949d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:30:50.610 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:50.608+0000 7f6d92000640 1 -- 172.21.15.47:0/978245889 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55818ee857d0 con 0x55818ee80e00 2024-03-21T10:30:50.610 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:50.608+0000 7f6d9b400640 1 --2- 172.21.15.47:0/978245889 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55818ee80450 0x55818ee949d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T10:30:50.610 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:50.608+0000 7f6d92000640 1 --2- 172.21.15.47:0/978245889 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55818ee80e00 0x55818ee94f10 secure :-1 s=READY pgs=921 cs=0 l=1 rev1=1 crypto rx=0x7f6d84005290 tx=0x7f6d84005640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=52b87cf151450dc2 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:30:50.610 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:50.608+0000 7f6d88c00640 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-03-21T10:30:50.610 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:50.609+0000 7f6d88c00640 1 -- 172.21.15.47:0/978245889 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d8402c7e0 con 0x55818ee80e00 2024-03-21T10:30:50.610 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:50.609+0000 7f6d88c00640 1 -- 172.21.15.47:0/978245889 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(97..97 src has 1..97) v4 ==== 9474+0+0 (secure 0 0 0) 0x7f6d8404b040 con 0x55818ee80e00 2024-03-21T10:30:50.610 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:50.609+0000 7f6da49c6740 1 -- 172.21.15.47:0/978245889 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f6d8402cc90 con 0x55818ee80e00 2024-03-21T10:30:50.610 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:50.609+0000 7f6d88c00640 1 -- 172.21.15.47:0/978245889 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6d8405d050 con 0x55818ee80e00 2024-03-21T10:30:50.611 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:50.609+0000 7f6da49c6740 10 client.17382 fetch_fsmap learned FSMap version 32 2024-03-21T10:30:50.611 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:50.609+0000 7f6da49c6740 1 -- 172.21.15.47:0/978245889 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x55818eefdc10 con 0x55818ee80e00 2024-03-21T10:30:50.612 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:50.609+0000 7f6d88c00640 1 -- 172.21.15.47:0/978245889 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f6d8402cc90 con 0x55818ee80e00 2024-03-21T10:30:50.612 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:50.610+0000 7f6d88c00640 1 -- 172.21.15.47:0/978245889 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 32) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f6d840046c0 con 0x55818ee80e00 2024-03-21T10:30:50.613 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:50.610+0000 7f6da49c6740 10 client.17382 fetch_fsmap finished waiting for FSMap version 32 2024-03-21T10:30:50.613 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:50.610+0000 7f6da49c6740 10 client.17382 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-21T10:30:50.613 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:50.610+0000 7f6da49c6740 20 client.17382 populate_metadata read hostname 'smithi047' 2024-03-21T10:30:50.613 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:50.610+0000 7f6da49c6740 1 --2- 172.21.15.47:0/978245889 >> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] conn(0x55818eefe2e0 0x55818ef00700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:30:50.613 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:50.610+0000 7f6da49c6740 4 client.17382 mds_command: new command op to 34535 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-21T10:30:50.613 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:50.610+0000 7f6da49c6740 1 -- 172.21.15.47:0/978245889 --> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55818edebb50 con 0x55818eefe2e0 2024-03-21T10:30:50.613 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:50.610+0000 7f6d9b400640 1 --2- 172.21.15.47:0/978245889 >> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] conn(0x55818eefe2e0 0x55818ef00700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:30:50.613 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:50.611+0000 7f6d9b400640 1 --2- 172.21.15.47:0/978245889 >> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] conn(0x55818eefe2e0 0x55818ef00700 crc :-1 s=READY pgs=148 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bc7ff5e977d6608a server_cookie=d59c60b5c4b48236 in_seq=0 out_seq=0 2024-03-21T10:30:50.613 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:50.611+0000 7f6d88c00640 10 client.17382 ms_handle_connect on v2:172.21.15.100:6838/4173168440 2024-03-21T10:30:50.613 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:50.612+0000 7f6d88c00640 1 -- 172.21.15.47:0/978245889 <== mds.0 v2:172.21.15.100:6838/4173168440 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55818edebb50 con 0x55818eefe2e0 2024-03-21T10:30:50.613 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:50.612+0000 7f6d88c00640 10 client.17382 handle_command_reply: tid=0 2024-03-21T10:30:50.613 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:50.612+0000 7f6da49c6740 1 client.17382 shutdown 2024-03-21T10:30:50.613 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:50.612+0000 7f6da49c6740 1 -- 172.21.15.47:0/978245889 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55818ee80e00 msgr2=0x55818ee94f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:30:50.613 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:50.612+0000 7f6da49c6740 1 --2- 172.21.15.47:0/978245889 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55818ee80e00 0x55818ee94f10 secure :-1 s=READY pgs=921 cs=0 l=1 rev1=1 crypto rx=0x7f6d84005290 tx=0x7f6d84005640 comp rx=0 tx=0).stop 2024-03-21T10:30:50.613 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:50.612+0000 7f6da49c6740 1 -- 172.21.15.47:0/978245889 shutdown_connections 2024-03-21T10:30:50.614 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:50.612+0000 7f6da49c6740 1 --2- 172.21.15.47:0/978245889 >> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] conn(0x55818eefe2e0 0x55818ef00700 crc :-1 s=READY pgs=148 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:30:50.614 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:50.612+0000 7f6da49c6740 1 --2- 172.21.15.47:0/978245889 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55818ee8e010 0x55818ee95450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:30:50.614 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:50.612+0000 7f6da49c6740 1 --2- 172.21.15.47:0/978245889 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55818ee80e00 0x55818ee94f10 unknown :-1 s=CLOSED pgs=921 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:30:50.614 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:50.612+0000 7f6d88c00640 0 client.17382 ms_handle_reset on v2:172.21.15.100:6838/4173168440 2024-03-21T10:30:50.614 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:50.612+0000 7f6da49c6740 1 --2- 172.21.15.47:0/978245889 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55818ee80450 0x55818ee949d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:30:50.614 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:50.612+0000 7f6da49c6740 1 -- 172.21.15.47:0/978245889 >> 172.21.15.47:0/978245889 conn(0x55818ee7ada0 msgr2=0x55818eea4790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:30:50.614 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:50.612+0000 7f6da49c6740 1 -- 172.21.15.47:0/978245889 shutdown_connections 2024-03-21T10:30:50.614 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:50.612+0000 7f6da49c6740 1 -- 172.21.15.47:0/978245889 wait complete. 2024-03-21T10:30:50.614 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:50.612+0000 7f6da49c6740 20 client.17382 trim_cache size 0 max 16384 2024-03-21T10:30:50.614 INFO:teuthology.orchestra.run.smithi047.stdout:[] 2024-03-21T10:30:50.615 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:50.613+0000 7f6da1c00640 1 -- 172.21.15.47:0/1745291081 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f6d800a3090 msgr2=0x7f6d800a5550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:30:50.615 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:50.613+0000 7f6da1c00640 1 --2- 172.21.15.47:0/1745291081 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f6d800a3090 0x7f6d800a5550 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f6d8c0ccb10 tx=0x7f6d8c005f30 comp rx=0 tx=0).stop 2024-03-21T10:30:50.615 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:50.613+0000 7f6da1c00640 1 -- 172.21.15.47:0/1745291081 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6d9c068880 msgr2=0x7f6d9c197e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:30:50.615 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:50.613+0000 7f6da1c00640 1 --2- 172.21.15.47:0/1745291081 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6d9c068880 0x7f6d9c197e90 secure :-1 s=READY pgs=917 cs=0 l=1 rev1=1 crypto rx=0x7f6d8c00d950 tx=0x7f6d8c00de10 comp rx=0 tx=0).stop 2024-03-21T10:30:50.615 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:50.613+0000 7f6da1c00640 1 -- 172.21.15.47:0/1745291081 shutdown_connections 2024-03-21T10:30:50.615 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:50.613+0000 7f6da1c00640 1 --2- 172.21.15.47:0/1745291081 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f6d800a3090 0x7f6d800a5550 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:30:50.616 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:50.613+0000 7f6da1c00640 1 --2- 172.21.15.47:0/1745291081 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6d9c1076b0 0x7f6d9c19c700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:30:50.616 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:50.613+0000 7f6da1c00640 1 --2- 172.21.15.47:0/1745291081 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6d9c069240 0x7f6d9c1983d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:30:50.616 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:50.613+0000 7f6da1c00640 1 --2- 172.21.15.47:0/1745291081 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6d9c068880 0x7f6d9c197e90 unknown :-1 s=CLOSED pgs=917 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:30:50.616 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:50.613+0000 7f6da1c00640 1 -- 172.21.15.47:0/1745291081 >> 172.21.15.47:0/1745291081 conn(0x7f6d9c0776a0 msgr2=0x7f6d9c0ff140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:30:50.616 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:50.613+0000 7f6da1c00640 1 -- 172.21.15.47:0/1745291081 shutdown_connections 2024-03-21T10:30:50.616 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:50.613+0000 7f6da1c00640 1 -- 172.21.15.47:0/1745291081 wait complete. 2024-03-21T10:30:51.632 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-03-21T10:30:51.632 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-21T10:30:51.632 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-03-21T10:30:51.711 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:30:51 smithi047 ceph-mon[31547]: pgmap v169: 129 pgs: 129 active+clean; 277 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-21T10:30:51.727 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:30:51 smithi100 ceph-mon[35193]: pgmap v169: 129 pgs: 129 active+clean; 277 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-21T10:30:51.737 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:30:51 smithi157 ceph-mon[35581]: pgmap v169: 129 pgs: 129 active+clean; 277 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-21T10:30:51.883 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:51.883+0000 7f871f200640 1 -- 172.21.15.47:0/2595218258 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8720112510 msgr2=0x7f8720114900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:30:51.885 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:51.883+0000 7f871f200640 1 --2- 172.21.15.47:0/2595218258 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8720112510 0x7f8720114900 secure :-1 s=READY pgs=922 cs=0 l=1 rev1=1 crypto rx=0x7f8718009ca0 tx=0x7f871802c190 comp rx=0 tx=0).stop 2024-03-21T10:30:51.889 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:51.883+0000 7f871f200640 1 -- 172.21.15.47:0/2595218258 shutdown_connections 2024-03-21T10:30:51.889 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:51.883+0000 7f871f200640 1 --2- 172.21.15.47:0/2595218258 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8720112510 0x7f8720114900 unknown :-1 s=CLOSED pgs=922 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:30:51.889 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:51.883+0000 7f871f200640 1 --2- 172.21.15.47:0/2595218258 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8720074360 0x7f8720111fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:30:51.889 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:51.883+0000 7f871f200640 1 --2- 172.21.15.47:0/2595218258 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8720073a40 0x7f8720073e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:30:51.889 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:51.883+0000 7f871f200640 1 -- 172.21.15.47:0/2595218258 >> 172.21.15.47:0/2595218258 conn(0x7f87200fd010 msgr2=0x7f87200ff430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:30:51.889 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:51.884+0000 7f871f200640 1 -- 172.21.15.47:0/2595218258 shutdown_connections 2024-03-21T10:30:51.889 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:51.884+0000 7f871f200640 1 -- 172.21.15.47:0/2595218258 wait complete. 2024-03-21T10:30:51.889 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:51.884+0000 7f871f200640 1 Processor -- start 2024-03-21T10:30:51.889 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:51.885+0000 7f871f200640 1 -- start start 2024-03-21T10:30:51.889 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:51.885+0000 7f871f200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8720073a40 0x7f872019e300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:30:51.890 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:51.885+0000 7f871f200640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8720074360 0x7f872019e840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:30:51.890 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:51.885+0000 7f871f200640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8720112510 0x7f87201a2bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:30:51.890 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:51.885+0000 7f871f200640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8720117dc0 con 0x7f8720112510 2024-03-21T10:30:51.890 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:51.885+0000 7f871f200640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8720117c40 con 0x7f8720073a40 2024-03-21T10:30:51.890 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:51.885+0000 7f871f200640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8720117f40 con 0x7f8720074360 2024-03-21T10:30:51.890 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:51.885+0000 7f871e800640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8720112510 0x7f87201a2bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:30:51.890 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:51.885+0000 7f871e800640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8720112510 0x7f87201a2bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:49064/0 (socket says 172.21.15.47:49064) 2024-03-21T10:30:51.890 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:51.885+0000 7f871e800640 1 -- 172.21.15.47:0/958786270 learned_addr learned my addr 172.21.15.47:0/958786270 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:30:51.890 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:51.886+0000 7f871de00640 1 --2- 172.21.15.47:0/958786270 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8720073a40 0x7f872019e300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:30:51.890 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:51.886+0000 7f871d400640 1 --2- 172.21.15.47:0/958786270 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8720074360 0x7f872019e840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:30:51.890 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:51.886+0000 7f871e800640 1 -- 172.21.15.47:0/958786270 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8720074360 msgr2=0x7f872019e840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:30:51.891 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:51.886+0000 7f871e800640 1 --2- 172.21.15.47:0/958786270 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8720074360 0x7f872019e840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:30:51.891 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:51.886+0000 7f871e800640 1 -- 172.21.15.47:0/958786270 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8720073a40 msgr2=0x7f872019e300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:30:51.891 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:51.886+0000 7f871e800640 1 --2- 172.21.15.47:0/958786270 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8720073a40 0x7f872019e300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:30:51.891 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:51.886+0000 7f871e800640 1 -- 172.21.15.47:0/958786270 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87201a32f0 con 0x7f8720112510 2024-03-21T10:30:51.891 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:51.886+0000 7f871e800640 1 --2- 172.21.15.47:0/958786270 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8720112510 0x7f87201a2bc0 secure :-1 s=READY pgs=923 cs=0 l=1 rev1=1 crypto rx=0x7f871802c7a0 tx=0x7f8718035d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:30:51.891 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:51.887+0000 7f8716a00640 1 -- 172.21.15.47:0/958786270 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8718039040 con 0x7f8720112510 2024-03-21T10:30:51.891 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:51.887+0000 7f8716a00640 1 -- 172.21.15.47:0/958786270 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f8718044070 con 0x7f8720112510 2024-03-21T10:30:51.891 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:51.887+0000 7f871f200640 1 -- 172.21.15.47:0/958786270 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f87201a3580 con 0x7f8720112510 2024-03-21T10:30:51.891 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:51.887+0000 7f8716a00640 1 -- 172.21.15.47:0/958786270 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8718004860 con 0x7f8720112510 2024-03-21T10:30:51.891 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:51.887+0000 7f871f200640 1 -- 172.21.15.47:0/958786270 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f87201aae70 con 0x7f8720112510 2024-03-21T10:30:51.899 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:51.890+0000 7f87277c8740 1 Processor -- start 2024-03-21T10:30:51.899 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:51.890+0000 7f8716a00640 1 -- 172.21.15.47:0/958786270 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 27) v1 ==== 137985+0+0 (secure 0 0 0) 0x7f8718004040 con 0x7f8720112510 2024-03-21T10:30:51.899 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:51.890+0000 7f87277c8740 1 -- start start 2024-03-21T10:30:51.899 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:51.891+0000 7f87277c8740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55974e0c9290 0x55974e0c9670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:30:51.900 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:51.891+0000 7f87277c8740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55974e0c9bb0 0x55974e0d2250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:30:51.900 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:51.891+0000 7f87277c8740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55974e0d2790 0x55974e0d4b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:30:51.900 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:51.891+0000 7f87277c8740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x55974e02deb0 con 0x55974e0c9290 2024-03-21T10:30:51.900 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:51.891+0000 7f87277c8740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55974dfd6a60 con 0x55974e0d2790 2024-03-21T10:30:51.900 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:51.891+0000 7f87277c8740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55974e029f50 con 0x55974e0c9bb0 2024-03-21T10:30:51.900 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:51.891+0000 7f871d400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55974e0c9290 0x55974e0c9670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:30:51.900 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:51.891+0000 7f871d400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55974e0c9290 0x55974e0c9670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:49066/0 (socket says 172.21.15.47:49066) 2024-03-21T10:30:51.900 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:51.891+0000 7f871d400640 1 -- 172.21.15.47:0/3729652255 learned_addr learned my addr 172.21.15.47:0/3729652255 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:30:51.900 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:51.891+0000 7f871de00640 1 --2- 172.21.15.47:0/3729652255 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55974e0c9bb0 0x55974e0d2250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:30:51.901 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:51.891+0000 7f871d400640 1 --2- 172.21.15.47:0/3729652255 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55974e0d2790 0x55974e0d4b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:30:51.901 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:51.892+0000 7f871d400640 1 -- 172.21.15.47:0/3729652255 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55974e0c9bb0 msgr2=0x55974e0d2250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:30:51.901 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:51.892+0000 7f871d400640 1 --2- 172.21.15.47:0/3729652255 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55974e0c9bb0 0x55974e0d2250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:30:51.901 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:51.892+0000 7f871d400640 1 -- 172.21.15.47:0/3729652255 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55974e0d2790 msgr2=0x55974e0d4b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:30:51.901 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:51.892+0000 7f871d400640 1 --2- 172.21.15.47:0/3729652255 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55974e0d2790 0x55974e0d4b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:30:51.901 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:51.892+0000 7f871d400640 1 -- 172.21.15.47:0/3729652255 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55974de0a7e0 con 0x55974e0c9290 2024-03-21T10:30:51.901 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:51.892+0000 7f8716a00640 1 --2- 172.21.15.47:0/958786270 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f87040a3090 0x7f87040a5550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:30:51.901 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:51.892+0000 7f871d400640 1 --2- 172.21.15.47:0/3729652255 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55974e0c9290 0x55974e0c9670 secure :-1 s=READY pgs=924 cs=0 l=1 rev1=1 crypto rx=0x7f870c005f60 tx=0x7f870c009770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=28f0a10377f402ce server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:30:51.901 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:51.892+0000 7f8716a00640 1 -- 172.21.15.47:0/958786270 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 9474+0+0 (secure 0 0 0) 0x7f87180f1f90 con 0x7f8720112510 2024-03-21T10:30:51.901 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:51.892+0000 7f871de00640 1 --2- 172.21.15.47:0/958786270 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f87040a3090 0x7f87040a5550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:30:51.901 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:51.892+0000 7f870b000640 1 -- 172.21.15.47:0/3729652255 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f870c009450 con 0x55974e0c9290 2024-03-21T10:30:51.901 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:51.892+0000 7f870b000640 1 -- 172.21.15.47:0/3729652255 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f870c010940 con 0x55974e0c9290 2024-03-21T10:30:51.901 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:51.893+0000 7f870b000640 1 -- 172.21.15.47:0/3729652255 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f870c011980 con 0x55974e0c9290 2024-03-21T10:30:51.901 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:51.893+0000 7f87277c8740 1 -- 172.21.15.47:0/3729652255 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55974e0c9290 msgr2=0x55974e0c9670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:30:51.902 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:51.893+0000 7f871de00640 1 --2- 172.21.15.47:0/958786270 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f87040a3090 0x7f87040a5550 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x55974e0c8d60 tx=0x7f8710005f90 comp rx=0 tx=0).ready entity=mgr.15687 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:30:51.902 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:51.893+0000 7f87277c8740 1 --2- 172.21.15.47:0/3729652255 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55974e0c9290 0x55974e0c9670 secure :-1 s=READY pgs=924 cs=0 l=1 rev1=1 crypto rx=0x7f870c005f60 tx=0x7f870c009770 comp rx=0 tx=0).stop 2024-03-21T10:30:51.902 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:51.894+0000 7f87277c8740 1 -- 172.21.15.47:0/3729652255 shutdown_connections 2024-03-21T10:30:51.902 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:51.894+0000 7f87277c8740 1 --2- 172.21.15.47:0/3729652255 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55974e0d2790 0x55974e0d4b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:30:51.902 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:51.894+0000 7f87277c8740 1 --2- 172.21.15.47:0/3729652255 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55974e0c9bb0 0x55974e0d2250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:30:51.902 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:51.894+0000 7f87277c8740 1 --2- 172.21.15.47:0/3729652255 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55974e0c9290 0x55974e0c9670 unknown :-1 s=CLOSED pgs=924 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:30:51.902 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:51.894+0000 7f87277c8740 1 -- 172.21.15.47:0/3729652255 >> 172.21.15.47:0/3729652255 conn(0x55974e0c39c0 msgr2=0x55974e0c3da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:30:51.903 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:51.894+0000 7f87277c8740 1 -- 172.21.15.47:0/3729652255 shutdown_connections 2024-03-21T10:30:51.903 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:51.894+0000 7f87277c8740 1 -- 172.21.15.47:0/3729652255 wait complete. 2024-03-21T10:30:51.903 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:51.894+0000 7f87277c8740 1 -- start start 2024-03-21T10:30:51.903 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:51.894+0000 7f87277c8740 1 Processor -- start 2024-03-21T10:30:51.903 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:51.895+0000 7f87277c8740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55974e0c9290 0x55974e0d2150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:30:51.903 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:51.895+0000 7f87277c8740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55974e0c9bb0 0x55974e0cf4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:30:51.905 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:51.895+0000 7f87277c8740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55974e0d2790 0x55974e0cfaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:30:51.906 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:51.895+0000 7f87277c8740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x55974e0d5580 con 0x55974e0d2790 2024-03-21T10:30:51.906 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:51.895+0000 7f87277c8740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55974dfe9eb0 con 0x55974e0c9290 2024-03-21T10:30:51.906 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:51.895+0000 7f87277c8740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55974e0d5700 con 0x55974e0c9bb0 2024-03-21T10:30:51.906 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:51.895+0000 7f87277c8740 10 client.0 _ll_register_callbacks cb 0x55974dfde150 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T10:30:51.906 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:51.895+0000 7f871e800640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55974e0d2790 0x55974e0cfaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:30:51.906 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:51.895+0000 7f871de00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55974e0c9290 0x55974e0d2150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:30:51.906 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:51.895+0000 7f871e800640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55974e0d2790 0x55974e0cfaf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:49080/0 (socket says 172.21.15.47:49080) 2024-03-21T10:30:51.906 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:51.895+0000 7f871e800640 1 -- 172.21.15.47:0/3800755404 learned_addr learned my addr 172.21.15.47:0/3800755404 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:30:51.907 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:51.895+0000 7f871d400640 1 --2- 172.21.15.47:0/3800755404 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55974e0c9bb0 0x55974e0cf4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:30:51.907 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:51.896+0000 7f871e800640 1 -- 172.21.15.47:0/3800755404 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55974e0c9bb0 msgr2=0x55974e0cf4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:30:51.907 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:51.896+0000 7f871e800640 1 --2- 172.21.15.47:0/3800755404 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55974e0c9bb0 0x55974e0cf4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:30:51.907 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:51.896+0000 7f871e800640 1 -- 172.21.15.47:0/3800755404 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55974e0c9290 msgr2=0x55974e0d2150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:30:51.907 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:51.896+0000 7f871e800640 1 --2- 172.21.15.47:0/3800755404 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55974e0c9290 0x55974e0d2150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:30:51.907 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:51.896+0000 7f871e800640 1 -- 172.21.15.47:0/3800755404 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55974de6af70 con 0x55974e0d2790 2024-03-21T10:30:51.907 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:51.896+0000 7f871e800640 1 --2- 172.21.15.47:0/3800755404 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55974e0d2790 0x55974e0cfaf0 secure :-1 s=READY pgs=925 cs=0 l=1 rev1=1 crypto rx=0x7f8718009430 tx=0x7f87180be830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=954ed08c7d8292cf server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:30:51.907 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:51.896+0000 7f86ff400640 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-03-21T10:30:51.907 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:51.896+0000 7f86ff400640 1 -- 172.21.15.47:0/3800755404 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87180c6a80 con 0x55974e0d2790 2024-03-21T10:30:51.907 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:51.896+0000 7f86ff400640 1 -- 172.21.15.47:0/3800755404 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(97..97 src has 1..97) v4 ==== 9474+0+0 (secure 0 0 0) 0x7f87180ba6e0 con 0x55974e0d2790 2024-03-21T10:30:51.907 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:51.896+0000 7f87277c8740 1 -- 172.21.15.47:0/3800755404 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f870c010940 con 0x55974e0d2790 2024-03-21T10:30:51.908 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:51.897+0000 7f86ff400640 1 -- 172.21.15.47:0/3800755404 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f87180d70d0 con 0x55974e0d2790 2024-03-21T10:30:51.908 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:51.897+0000 7f87277c8740 10 client.17394 fetch_fsmap learned FSMap version 32 2024-03-21T10:30:51.908 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:51.897+0000 7f87277c8740 1 -- 172.21.15.47:0/3800755404 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x55974dd02170 con 0x55974e0d2790 2024-03-21T10:30:51.908 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:51.897+0000 7f86ff400640 1 -- 172.21.15.47:0/3800755404 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f870c010940 con 0x55974e0d2790 2024-03-21T10:30:51.908 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:51.897+0000 7f86ff400640 1 -- 172.21.15.47:0/3800755404 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 32) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f871803c030 con 0x55974e0d2790 2024-03-21T10:30:51.908 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:51.897+0000 7f87277c8740 10 client.17394 fetch_fsmap finished waiting for FSMap version 32 2024-03-21T10:30:51.908 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:51.897+0000 7f87277c8740 10 client.17394 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-21T10:30:51.908 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:51.897+0000 7f87277c8740 20 client.17394 populate_metadata read hostname 'smithi047' 2024-03-21T10:30:51.908 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:51.897+0000 7f87277c8740 1 --2- 172.21.15.47:0/3800755404 >> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] conn(0x55974e0e2890 0x55974e0e4cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:30:51.908 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:51.897+0000 7f87277c8740 4 client.17394 mds_command: new command op to 34535 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T10:30:51.908 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:51.897+0000 7f87277c8740 1 -- 172.21.15.47:0/3800755404 --> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55974e03b3f0 con 0x55974e0e2890 2024-03-21T10:30:51.908 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:51.897+0000 7f871de00640 1 --2- 172.21.15.47:0/3800755404 >> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] conn(0x55974e0e2890 0x55974e0e4cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:30:51.908 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:51.898+0000 7f871de00640 1 --2- 172.21.15.47:0/3800755404 >> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] conn(0x55974e0e2890 0x55974e0e4cf0 crc :-1 s=READY pgs=149 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7b2899d46075fb80 server_cookie=c9a07110623f7f94 in_seq=0 out_seq=0 2024-03-21T10:30:51.908 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:51.898+0000 7f86ff400640 10 client.17394 ms_handle_connect on v2:172.21.15.100:6838/4173168440 2024-03-21T10:30:51.909 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:51.899+0000 7f86ff400640 1 -- 172.21.15.47:0/3800755404 <== mds.0 v2:172.21.15.100:6838/4173168440 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55974e03b3f0 con 0x55974e0e2890 2024-03-21T10:30:51.909 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:51.899+0000 7f86ff400640 10 client.17394 handle_command_reply: tid=0 2024-03-21T10:30:51.909 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:51.899+0000 7f87277c8740 1 client.17394 shutdown 2024-03-21T10:30:51.909 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:51.899+0000 7f87277c8740 1 -- 172.21.15.47:0/3800755404 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55974e0d2790 msgr2=0x55974e0cfaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:30:51.909 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:51.899+0000 7f87277c8740 1 --2- 172.21.15.47:0/3800755404 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55974e0d2790 0x55974e0cfaf0 secure :-1 s=READY pgs=925 cs=0 l=1 rev1=1 crypto rx=0x7f8718009430 tx=0x7f87180be830 comp rx=0 tx=0).stop 2024-03-21T10:30:51.909 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:51.899+0000 7f87277c8740 1 -- 172.21.15.47:0/3800755404 shutdown_connections 2024-03-21T10:30:51.909 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:51.899+0000 7f87277c8740 1 --2- 172.21.15.47:0/3800755404 >> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] conn(0x55974e0e2890 0x55974e0e4cf0 crc :-1 s=READY pgs=149 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:30:51.909 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:51.899+0000 7f87277c8740 1 --2- 172.21.15.47:0/3800755404 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55974e0d2790 0x55974e0cfaf0 unknown :-1 s=CLOSED pgs=925 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:30:51.909 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:51.899+0000 7f87277c8740 1 --2- 172.21.15.47:0/3800755404 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55974e0c9bb0 0x55974e0cf4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:30:51.909 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:51.899+0000 7f87277c8740 1 --2- 172.21.15.47:0/3800755404 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55974e0c9290 0x55974e0d2150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:30:51.909 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:51.899+0000 7f87277c8740 1 -- 172.21.15.47:0/3800755404 >> 172.21.15.47:0/3800755404 conn(0x55974e0c39c0 msgr2=0x55974e0c5710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:30:51.909 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:51.899+0000 7f86ff400640 0 client.17394 ms_handle_reset on v2:172.21.15.100:6838/4173168440 2024-03-21T10:30:51.909 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:51.899+0000 7f87277c8740 1 -- 172.21.15.47:0/3800755404 shutdown_connections 2024-03-21T10:30:51.909 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:51.899+0000 7f87277c8740 1 -- 172.21.15.47:0/3800755404 wait complete. 2024-03-21T10:30:51.910 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:51.900+0000 7f87277c8740 20 client.17394 trim_cache size 0 max 16384 2024-03-21T10:30:51.920 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:51.917+0000 7f87277c8740 1 Processor -- start 2024-03-21T10:30:51.920 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:51.917+0000 7f87277c8740 1 -- start start 2024-03-21T10:30:51.920 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:51.917+0000 7f87277c8740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55974e0c9290 0x55974e0d1c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:30:51.920 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:51.917+0000 7f87277c8740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55974e0c9bb0 0x55974e0d2150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:30:51.920 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:51.918+0000 7f87277c8740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55974e0d2790 0x55974e0c2900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:30:51.921 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:51.918+0000 7f87277c8740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87201b4c60 con 0x55974e0d2790 2024-03-21T10:30:51.921 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:51.918+0000 7f87277c8740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87201b65e0 con 0x55974e0c9290 2024-03-21T10:30:51.921 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:51.918+0000 7f87277c8740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87201b52c0 con 0x55974e0c9bb0 2024-03-21T10:30:51.921 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:51.918+0000 7f871e800640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55974e0d2790 0x55974e0c2900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:30:51.921 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:51.918+0000 7f871de00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55974e0c9290 0x55974e0d1c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:30:51.921 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:51.918+0000 7f871e800640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55974e0d2790 0x55974e0c2900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:49082/0 (socket says 172.21.15.47:49082) 2024-03-21T10:30:51.921 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:51.918+0000 7f871e800640 1 -- 172.21.15.47:0/2769940059 learned_addr learned my addr 172.21.15.47:0/2769940059 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:30:51.921 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:51.918+0000 7f871d400640 1 --2- 172.21.15.47:0/2769940059 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55974e0c9bb0 0x55974e0d2150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:30:51.921 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:51.918+0000 7f871de00640 1 -- 172.21.15.47:0/2769940059 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55974e0c9bb0 msgr2=0x55974e0d2150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:30:51.922 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:51.918+0000 7f871de00640 1 --2- 172.21.15.47:0/2769940059 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55974e0c9bb0 0x55974e0d2150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:30:51.922 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:51.918+0000 7f871de00640 1 -- 172.21.15.47:0/2769940059 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55974e0d2790 msgr2=0x55974e0c2900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:30:51.922 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:51.918+0000 7f871de00640 1 --2- 172.21.15.47:0/2769940059 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55974e0d2790 0x55974e0c2900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:30:51.922 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:51.918+0000 7f871de00640 1 -- 172.21.15.47:0/2769940059 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55974e0e5370 con 0x55974e0c9290 2024-03-21T10:30:51.922 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:51.918+0000 7f871e800640 1 --2- 172.21.15.47:0/2769940059 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55974e0d2790 0x55974e0c2900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T10:30:51.922 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:51.919+0000 7f871de00640 1 --2- 172.21.15.47:0/2769940059 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55974e0c9290 0x55974e0d1c10 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x55974e0e2c70 tx=0x7f871001b000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a9fd9eedeba98a3 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:30:51.922 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:51.919+0000 7f870a600640 1 -- 172.21.15.47:0/2769940059 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8710005020 con 0x55974e0c9290 2024-03-21T10:30:51.922 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:51.919+0000 7f870a600640 1 -- 172.21.15.47:0/2769940059 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f87100047a0 con 0x55974e0c9290 2024-03-21T10:30:51.922 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:51.919+0000 7f870a600640 1 -- 172.21.15.47:0/2769940059 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f871001f620 con 0x55974e0c9290 2024-03-21T10:30:51.923 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:51.920+0000 7f87277c8740 1 -- 172.21.15.47:0/2769940059 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55974e0c9290 msgr2=0x55974e0d1c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:30:51.923 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:51.920+0000 7f87277c8740 1 --2- 172.21.15.47:0/2769940059 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55974e0c9290 0x55974e0d1c10 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x55974e0e2c70 tx=0x7f871001b000 comp rx=0 tx=0).stop 2024-03-21T10:30:51.923 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:51.920+0000 7f87277c8740 1 -- 172.21.15.47:0/2769940059 shutdown_connections 2024-03-21T10:30:51.923 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:51.920+0000 7f87277c8740 1 --2- 172.21.15.47:0/2769940059 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55974e0d2790 0x55974e0c2900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:30:51.923 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:51.920+0000 7f87277c8740 1 --2- 172.21.15.47:0/2769940059 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55974e0c9bb0 0x55974e0d2150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:30:51.923 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:51.920+0000 7f87277c8740 1 --2- 172.21.15.47:0/2769940059 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55974e0c9290 0x55974e0d1c10 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:30:51.924 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:51.920+0000 7f87277c8740 1 -- 172.21.15.47:0/2769940059 >> 172.21.15.47:0/2769940059 conn(0x55974e0c39c0 msgr2=0x55974e0c21f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:30:51.924 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:51.920+0000 7f87277c8740 1 -- 172.21.15.47:0/2769940059 shutdown_connections 2024-03-21T10:30:51.924 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:51.920+0000 7f87277c8740 1 -- 172.21.15.47:0/2769940059 wait complete. 2024-03-21T10:30:51.924 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:51.920+0000 7f87277c8740 1 -- start start 2024-03-21T10:30:51.924 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:51.920+0000 7f87277c8740 1 Processor -- start 2024-03-21T10:30:51.930 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:51.921+0000 7f87277c8740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55974e0c9290 0x55974e0cd170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:30:51.931 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:51.921+0000 7f87277c8740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55974e0c9bb0 0x55974e0d7980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:30:51.931 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:51.921+0000 7f87277c8740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55974e0d2790 0x55974e0d7ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:30:51.931 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:51.921+0000 7f87277c8740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87201b5f80 con 0x55974e0c9290 2024-03-21T10:30:51.931 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:51.921+0000 7f87277c8740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87201b5920 con 0x55974e0c9bb0 2024-03-21T10:30:51.931 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:51.921+0000 7f87277c8740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87201b2e20 con 0x55974e0d2790 2024-03-21T10:30:51.931 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:51.921+0000 7f87277c8740 10 client.0 _ll_register_callbacks cb 0x7f8718005ef0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T10:30:51.931 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:51.921+0000 7f871de00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55974e0c9290 0x55974e0cd170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:30:51.931 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:51.921+0000 7f871de00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55974e0c9290 0x55974e0cd170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:49098/0 (socket says 172.21.15.47:49098) 2024-03-21T10:30:51.931 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:51.921+0000 7f871de00640 1 -- 172.21.15.47:0/3022678833 learned_addr learned my addr 172.21.15.47:0/3022678833 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:30:51.931 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:51.921+0000 7f871d400640 1 --2- 172.21.15.47:0/3022678833 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55974e0c9bb0 0x55974e0d7980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:30:51.931 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:51.921+0000 7f871e800640 1 --2- 172.21.15.47:0/3022678833 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55974e0d2790 0x55974e0d7ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:30:51.931 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:51.922+0000 7f871de00640 1 -- 172.21.15.47:0/3022678833 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55974e0d2790 msgr2=0x55974e0d7ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:30:51.932 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:51.922+0000 7f871de00640 1 --2- 172.21.15.47:0/3022678833 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55974e0d2790 0x55974e0d7ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:30:51.932 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:51.922+0000 7f871de00640 1 -- 172.21.15.47:0/3022678833 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55974e0c9bb0 msgr2=0x55974e0d7980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:30:51.932 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:51.922+0000 7f871de00640 1 --2- 172.21.15.47:0/3022678833 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55974e0c9bb0 0x55974e0d7980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:30:51.932 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:51.922+0000 7f871de00640 1 -- 172.21.15.47:0/3022678833 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55974e0d8600 con 0x55974e0c9290 2024-03-21T10:30:51.932 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:51.922+0000 7f871de00640 1 --2- 172.21.15.47:0/3022678833 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55974e0c9290 0x55974e0cd170 secure :-1 s=READY pgs=926 cs=0 l=1 rev1=1 crypto rx=0x7f8710004240 tx=0x7f8710009f80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3c68bf6d903d6110 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:30:51.932 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:51.922+0000 7f86ff400640 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-03-21T10:30:51.932 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:51.922+0000 7f86ff400640 1 -- 172.21.15.47:0/3022678833 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8710039020 con 0x55974e0c9290 2024-03-21T10:30:51.932 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:51.922+0000 7f86ff400640 1 -- 172.21.15.47:0/3022678833 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(97..97 src has 1..97) v4 ==== 9474+0+0 (secure 0 0 0) 0x7f8710018040 con 0x55974e0c9290 2024-03-21T10:30:51.932 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:51.922+0000 7f87277c8740 1 -- 172.21.15.47:0/3022678833 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f87100047a0 con 0x55974e0c9290 2024-03-21T10:30:51.932 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:51.923+0000 7f86ff400640 1 -- 172.21.15.47:0/3022678833 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8710044050 con 0x55974e0c9290 2024-03-21T10:30:51.932 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:51.923+0000 7f87277c8740 10 client.17400 fetch_fsmap learned FSMap version 32 2024-03-21T10:30:51.932 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:51.923+0000 7f87277c8740 1 -- 172.21.15.47:0/3022678833 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x55974e13f180 con 0x55974e0c9290 2024-03-21T10:30:51.932 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:51.923+0000 7f86ff400640 1 -- 172.21.15.47:0/3022678833 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f87100047a0 con 0x55974e0c9290 2024-03-21T10:30:51.933 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:51.923+0000 7f86ff400640 1 -- 172.21.15.47:0/3022678833 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 32) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f8710035950 con 0x55974e0c9290 2024-03-21T10:30:51.933 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:51.923+0000 7f87277c8740 10 client.17400 fetch_fsmap finished waiting for FSMap version 32 2024-03-21T10:30:51.933 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:51.923+0000 7f87277c8740 10 client.17400 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-21T10:30:51.933 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:51.923+0000 7f87277c8740 20 client.17400 populate_metadata read hostname 'smithi047' 2024-03-21T10:30:51.933 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:51.923+0000 7f87277c8740 1 --2- 172.21.15.47:0/3022678833 >> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] conn(0x55974e13f900 0x55974e141d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:30:51.933 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:51.923+0000 7f87277c8740 4 client.17400 mds_command: new command op to 34535 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-03-21T10:30:51.933 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:51.923+0000 7f87277c8740 1 -- 172.21.15.47:0/3022678833 --> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x55974e035740 con 0x55974e13f900 2024-03-21T10:30:51.933 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:51.924+0000 7f871d400640 1 --2- 172.21.15.47:0/3022678833 >> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] conn(0x55974e13f900 0x55974e141d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:30:51.933 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:51.924+0000 7f871d400640 1 --2- 172.21.15.47:0/3022678833 >> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] conn(0x55974e13f900 0x55974e141d20 crc :-1 s=READY pgs=150 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3c3a58d6c454743a server_cookie=b3b275ac43d00ee1 in_seq=0 out_seq=0 2024-03-21T10:30:51.934 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:51.924+0000 7f86ff400640 10 client.17400 ms_handle_connect on v2:172.21.15.100:6838/4173168440 2024-03-21T10:30:51.934 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:51.927+0000 7f86ff400640 1 -- 172.21.15.47:0/3022678833 <== mds.0 v2:172.21.15.100:6838/4173168440 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x55974e035740 con 0x55974e13f900 2024-03-21T10:30:51.934 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:51.927+0000 7f86ff400640 10 client.17400 handle_command_reply: tid=0 2024-03-21T10:30:51.934 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:51.927+0000 7f87277c8740 1 client.17400 shutdown 2024-03-21T10:30:51.934 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:51.927+0000 7f87277c8740 1 -- 172.21.15.47:0/3022678833 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55974e0c9290 msgr2=0x55974e0cd170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:30:51.934 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:51.927+0000 7f87277c8740 1 --2- 172.21.15.47:0/3022678833 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55974e0c9290 0x55974e0cd170 secure :-1 s=READY pgs=926 cs=0 l=1 rev1=1 crypto rx=0x7f8710004240 tx=0x7f8710009f80 comp rx=0 tx=0).stop 2024-03-21T10:30:51.934 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:51.927+0000 7f87277c8740 1 -- 172.21.15.47:0/3022678833 shutdown_connections 2024-03-21T10:30:51.934 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:51.927+0000 7f87277c8740 1 --2- 172.21.15.47:0/3022678833 >> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] conn(0x55974e13f900 0x55974e141d20 crc :-1 s=READY pgs=150 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:30:51.934 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:51.927+0000 7f87277c8740 1 --2- 172.21.15.47:0/3022678833 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55974e0d2790 0x55974e0d7ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:30:51.934 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:51.927+0000 7f87277c8740 1 --2- 172.21.15.47:0/3022678833 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55974e0c9bb0 0x55974e0d7980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:30:51.934 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:51.927+0000 7f87277c8740 1 --2- 172.21.15.47:0/3022678833 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55974e0c9290 0x55974e0cd170 unknown :-1 s=CLOSED pgs=926 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:30:51.934 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:51.927+0000 7f87277c8740 1 -- 172.21.15.47:0/3022678833 >> 172.21.15.47:0/3022678833 conn(0x55974e0c39c0 msgr2=0x55974e0c2a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:30:51.935 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:51.927+0000 7f86ff400640 0 client.17400 ms_handle_reset on v2:172.21.15.100:6838/4173168440 2024-03-21T10:30:51.935 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:51.927+0000 7f87277c8740 1 -- 172.21.15.47:0/3022678833 shutdown_connections 2024-03-21T10:30:51.935 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:51.927+0000 7f87277c8740 1 -- 172.21.15.47:0/3022678833 wait complete. 2024-03-21T10:30:51.935 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:51.928+0000 7f87277c8740 20 client.17400 trim_cache size 0 max 16384 2024-03-21T10:30:51.935 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-03-21T10:30:51.935 INFO:teuthology.orchestra.run.smithi047.stdout: "return_code": 0, 2024-03-21T10:30:51.935 INFO:teuthology.orchestra.run.smithi047.stdout: "scrub_tag": "0d860191-5721-42c3-9876-0c8234119bff", 2024-03-21T10:30:51.935 INFO:teuthology.orchestra.run.smithi047.stdout: "mode": "asynchronous" 2024-03-21T10:30:51.935 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-03-21T10:30:51.936 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:51.928+0000 7f871f200640 1 -- 172.21.15.47:0/958786270 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f87040a3090 msgr2=0x7f87040a5550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:30:51.936 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:51.928+0000 7f871f200640 1 --2- 172.21.15.47:0/958786270 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f87040a3090 0x7f87040a5550 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x55974e0c8d60 tx=0x7f8710005f90 comp rx=0 tx=0).stop 2024-03-21T10:30:51.936 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:51.928+0000 7f871f200640 1 -- 172.21.15.47:0/958786270 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8720112510 msgr2=0x7f87201a2bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:30:51.936 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:51.928+0000 7f871f200640 1 --2- 172.21.15.47:0/958786270 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8720112510 0x7f87201a2bc0 secure :-1 s=READY pgs=923 cs=0 l=1 rev1=1 crypto rx=0x7f871802c7a0 tx=0x7f8718035d50 comp rx=0 tx=0).stop 2024-03-21T10:30:51.936 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:51.928+0000 7f871f200640 1 -- 172.21.15.47:0/958786270 shutdown_connections 2024-03-21T10:30:51.937 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:51.928+0000 7f871f200640 1 --2- 172.21.15.47:0/958786270 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8720112510 0x7f87201a2bc0 unknown :-1 s=CLOSED pgs=923 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:30:51.937 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:51.928+0000 7f871f200640 1 --2- 172.21.15.47:0/958786270 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8720074360 0x7f872019e840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:30:51.937 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:51.928+0000 7f871f200640 1 --2- 172.21.15.47:0/958786270 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f87040a3090 0x7f87040a5550 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:30:51.937 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:51.928+0000 7f871f200640 1 --2- 172.21.15.47:0/958786270 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8720073a40 0x7f872019e300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:30:51.937 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:51.928+0000 7f871f200640 1 -- 172.21.15.47:0/958786270 >> 172.21.15.47:0/958786270 conn(0x7f87200fd010 msgr2=0x7f87200fe8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:30:51.937 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:51.929+0000 7f871f200640 1 -- 172.21.15.47:0/958786270 shutdown_connections 2024-03-21T10:30:51.937 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:51.929+0000 7f871f200640 1 -- 172.21.15.47:0/958786270 wait complete. 2024-03-21T10:30:51.949 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-21T10:30:51.949 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-21T10:30:52.156 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:52.156+0000 7ffbcf200640 1 -- 172.21.15.47:0/4003740479 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ffbd0107e50 msgr2=0x7ffbd0108230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:30:52.156 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:52.156+0000 7ffbcf200640 1 --2- 172.21.15.47:0/4003740479 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ffbd0107e50 0x7ffbd0108230 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7ffbc0009900 tx=0x7ffbc002bc30 comp rx=0 tx=0).stop 2024-03-21T10:30:52.156 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:52.156+0000 7ffbcf200640 1 -- 172.21.15.47:0/4003740479 shutdown_connections 2024-03-21T10:30:52.156 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:52.156+0000 7ffbcf200640 1 --2- 172.21.15.47:0/4003740479 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ffbd010cea0 0x7ffbd0114a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:30:52.157 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:52.156+0000 7ffbcf200640 1 --2- 172.21.15.47:0/4003740479 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ffbd0108800 0x7ffbd010c7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:30:52.157 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:52.156+0000 7ffbcf200640 1 --2- 172.21.15.47:0/4003740479 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ffbd0107e50 0x7ffbd0108230 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:30:52.157 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:52.156+0000 7ffbcf200640 1 -- 172.21.15.47:0/4003740479 >> 172.21.15.47:0/4003740479 conn(0x7ffbd00774c0 msgr2=0x7ffbd01064c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:30:52.157 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:52.156+0000 7ffbcf200640 1 -- 172.21.15.47:0/4003740479 shutdown_connections 2024-03-21T10:30:52.157 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:52.156+0000 7ffbcf200640 1 -- 172.21.15.47:0/4003740479 wait complete. 2024-03-21T10:30:52.159 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:52.157+0000 7ffbcf200640 1 Processor -- start 2024-03-21T10:30:52.159 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:52.157+0000 7ffbcf200640 1 -- start start 2024-03-21T10:30:52.160 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:52.157+0000 7ffbcf200640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ffbd0107e50 0x7ffbd019e380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:30:52.160 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:52.158+0000 7ffbcf200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ffbd010cea0 0x7ffbd019e8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:30:52.160 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:52.158+0000 7ffbcf200640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ffbd01a2c40 0x7ffbd01a30f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:30:52.160 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:52.158+0000 7ffbcf200640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffbd0117e20 con 0x7ffbd0107e50 2024-03-21T10:30:52.160 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:52.158+0000 7ffbcf200640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffbd0117ca0 con 0x7ffbd010cea0 2024-03-21T10:30:52.160 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:52.158+0000 7ffbcf200640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffbd0117fa0 con 0x7ffbd01a2c40 2024-03-21T10:30:52.160 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:52.158+0000 7ffbcde00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ffbd0107e50 0x7ffbd019e380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:30:52.160 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:52.158+0000 7ffbcde00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ffbd0107e50 0x7ffbd019e380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:49120/0 (socket says 172.21.15.47:49120) 2024-03-21T10:30:52.160 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:52.158+0000 7ffbcde00640 1 -- 172.21.15.47:0/3117427933 learned_addr learned my addr 172.21.15.47:0/3117427933 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:30:52.160 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:52.158+0000 7ffbcd400640 1 --2- 172.21.15.47:0/3117427933 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ffbd010cea0 0x7ffbd019e8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:30:52.160 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:52.158+0000 7ffbce800640 1 --2- 172.21.15.47:0/3117427933 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ffbd01a2c40 0x7ffbd01a30f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:30:52.160 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:52.158+0000 7ffbcde00640 1 -- 172.21.15.47:0/3117427933 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ffbd01a2c40 msgr2=0x7ffbd01a30f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:30:52.160 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:52.158+0000 7ffbcde00640 1 --2- 172.21.15.47:0/3117427933 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ffbd01a2c40 0x7ffbd01a30f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:30:52.161 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:52.158+0000 7ffbcde00640 1 -- 172.21.15.47:0/3117427933 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ffbd010cea0 msgr2=0x7ffbd019e8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:30:52.161 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:52.158+0000 7ffbcde00640 1 --2- 172.21.15.47:0/3117427933 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ffbd010cea0 0x7ffbd019e8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:30:52.161 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:52.158+0000 7ffbcde00640 1 -- 172.21.15.47:0/3117427933 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffbd01a3730 con 0x7ffbd0107e50 2024-03-21T10:30:52.161 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:52.158+0000 7ffbcd400640 1 --2- 172.21.15.47:0/3117427933 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ffbd010cea0 0x7ffbd019e8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T10:30:52.161 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:52.159+0000 7ffbcde00640 1 --2- 172.21.15.47:0/3117427933 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ffbd0107e50 0x7ffbd019e380 secure :-1 s=READY pgs=927 cs=0 l=1 rev1=1 crypto rx=0x7ffbc0005940 tx=0x7ffbc00045e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:30:52.161 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:52.159+0000 7ffbc6a00640 1 -- 172.21.15.47:0/3117427933 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffbc0043020 con 0x7ffbd0107e50 2024-03-21T10:30:52.161 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:52.159+0000 7ffbc6a00640 1 -- 172.21.15.47:0/3117427933 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7ffbc002caf0 con 0x7ffbd0107e50 2024-03-21T10:30:52.161 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:52.159+0000 7ffbc6a00640 1 -- 172.21.15.47:0/3117427933 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffbc0003a50 con 0x7ffbd0107e50 2024-03-21T10:30:52.161 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:52.159+0000 7ffbcf200640 1 -- 172.21.15.47:0/3117427933 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffbd01a39c0 con 0x7ffbd0107e50 2024-03-21T10:30:52.161 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:52.159+0000 7ffbcf200640 1 -- 172.21.15.47:0/3117427933 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffbd01ab190 con 0x7ffbd0107e50 2024-03-21T10:30:52.163 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:52.163+0000 7ffbc6a00640 1 -- 172.21.15.47:0/3117427933 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 27) v1 ==== 137985+0+0 (secure 0 0 0) 0x7ffbc0046020 con 0x7ffbd0107e50 2024-03-21T10:30:52.163 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:52.163+0000 7ffbd777d740 1 Processor -- start 2024-03-21T10:30:52.163 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:52.163+0000 7ffbd777d740 1 -- start start 2024-03-21T10:30:52.165 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:52.164+0000 7ffbd777d740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x563a7e222dd0 0x563a7e221b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:30:52.166 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:52.164+0000 7ffbd777d740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563a7e2220f0 0x563a7e222570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:30:52.166 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:52.164+0000 7ffbd777d740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563a7e22b460 0x563a7e22d850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:30:52.166 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:52.164+0000 7ffbd777d740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x563a7e186250 con 0x563a7e222dd0 2024-03-21T10:30:52.166 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:52.164+0000 7ffbd777d740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x563a7e12ea30 con 0x563a7e22b460 2024-03-21T10:30:52.166 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:52.164+0000 7ffbd777d740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x563a7e1822f0 con 0x563a7e2220f0 2024-03-21T10:30:52.166 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:52.164+0000 7ffbce800640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x563a7e222dd0 0x563a7e221b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:30:52.166 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:52.164+0000 7ffbcd400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563a7e22b460 0x563a7e22d850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:30:52.166 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:52.164+0000 7ffbce800640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x563a7e222dd0 0x563a7e221b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:49122/0 (socket says 172.21.15.47:49122) 2024-03-21T10:30:52.166 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:52.164+0000 7ffbce800640 1 -- 172.21.15.47:0/3130985866 learned_addr learned my addr 172.21.15.47:0/3130985866 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:30:52.166 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:52.164+0000 7ffbcd400640 1 --2- 172.21.15.47:0/3130985866 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563a7e22b460 0x563a7e22d850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:59840/0 (socket says 172.21.15.47:59840) 2024-03-21T10:30:52.166 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:52.164+0000 7ffbcde00640 1 --2- 172.21.15.47:0/3130985866 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563a7e2220f0 0x563a7e222570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:30:52.166 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:52.165+0000 7ffbce800640 1 -- 172.21.15.47:0/3130985866 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563a7e2220f0 msgr2=0x563a7e222570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:30:52.167 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:52.165+0000 7ffbce800640 1 --2- 172.21.15.47:0/3130985866 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563a7e2220f0 0x563a7e222570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:30:52.167 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:52.165+0000 7ffbce800640 1 -- 172.21.15.47:0/3130985866 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563a7e22b460 msgr2=0x563a7e22d850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:30:52.167 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:52.165+0000 7ffbce800640 1 --2- 172.21.15.47:0/3130985866 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563a7e22b460 0x563a7e22d850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:30:52.167 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:52.165+0000 7ffbce800640 1 -- 172.21.15.47:0/3130985866 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563a7e119690 con 0x563a7e222dd0 2024-03-21T10:30:52.167 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:52.165+0000 7ffbcd400640 1 --2- 172.21.15.47:0/3130985866 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563a7e22b460 0x563a7e22d850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T10:30:52.167 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:52.165+0000 7ffbc6a00640 1 --2- 172.21.15.47:0/3117427933 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7ffbb40a3090 0x7ffbb40a5550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:30:52.167 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:52.165+0000 7ffbce800640 1 --2- 172.21.15.47:0/3130985866 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x563a7e222dd0 0x563a7e221b20 secure :-1 s=READY pgs=928 cs=0 l=1 rev1=1 crypto rx=0x7ffbc8009ef0 tx=0x7ffbc80117b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6b9b1127749ef66 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:30:52.167 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:52.165+0000 7ffbc6a00640 1 -- 172.21.15.47:0/3117427933 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 9474+0+0 (secure 0 0 0) 0x7ffbc00f1d30 con 0x7ffbd0107e50 2024-03-21T10:30:52.167 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:52.165+0000 7ffbcde00640 1 --2- 172.21.15.47:0/3117427933 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7ffbb40a3090 0x7ffbb40a5550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:30:52.167 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:52.165+0000 7ffbbb000640 1 -- 172.21.15.47:0/3130985866 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffbc8003b70 con 0x563a7e222dd0 2024-03-21T10:30:52.167 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:52.165+0000 7ffbbb000640 1 -- 172.21.15.47:0/3130985866 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7ffbc80099c0 con 0x563a7e222dd0 2024-03-21T10:30:52.167 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:52.166+0000 7ffbbb000640 1 -- 172.21.15.47:0/3130985866 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffbc80128b0 con 0x563a7e222dd0 2024-03-21T10:30:52.171 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:52.166+0000 7ffbd777d740 1 -- 172.21.15.47:0/3130985866 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x563a7e222dd0 msgr2=0x563a7e221b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:30:52.171 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:52.166+0000 7ffbd777d740 1 --2- 172.21.15.47:0/3130985866 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x563a7e222dd0 0x563a7e221b20 secure :-1 s=READY pgs=928 cs=0 l=1 rev1=1 crypto rx=0x7ffbc8009ef0 tx=0x7ffbc80117b0 comp rx=0 tx=0).stop 2024-03-21T10:30:52.171 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:52.166+0000 7ffbcde00640 1 --2- 172.21.15.47:0/3117427933 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7ffbb40a3090 0x7ffbb40a5550 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x563a7e223320 tx=0x7ffbc0000bc0 comp rx=0 tx=0).ready entity=mgr.15687 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:30:52.171 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:52.166+0000 7ffbd777d740 1 -- 172.21.15.47:0/3130985866 shutdown_connections 2024-03-21T10:30:52.171 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:52.166+0000 7ffbd777d740 1 --2- 172.21.15.47:0/3130985866 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563a7e22b460 0x563a7e22d850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:30:52.171 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:52.166+0000 7ffbd777d740 1 --2- 172.21.15.47:0/3130985866 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563a7e2220f0 0x563a7e222570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:30:52.172 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:52.166+0000 7ffbd777d740 1 --2- 172.21.15.47:0/3130985866 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x563a7e222dd0 0x563a7e221b20 unknown :-1 s=CLOSED pgs=928 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:30:52.172 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:52.166+0000 7ffbd777d740 1 -- 172.21.15.47:0/3130985866 >> 172.21.15.47:0/3130985866 conn(0x563a7e1e9200 msgr2=0x563a7e21cb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:30:52.172 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:52.167+0000 7ffbd777d740 1 -- 172.21.15.47:0/3130985866 shutdown_connections 2024-03-21T10:30:52.172 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:52.167+0000 7ffbd777d740 1 -- 172.21.15.47:0/3130985866 wait complete. 2024-03-21T10:30:52.172 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:52.167+0000 7ffbd777d740 1 -- start start 2024-03-21T10:30:52.172 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:52.167+0000 7ffbd777d740 1 Processor -- start 2024-03-21T10:30:52.172 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:52.168+0000 7ffbd777d740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563a7e2220f0 0x563a7e22ab60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:30:52.172 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:52.168+0000 7ffbd777d740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x563a7e222dd0 0x563a7e227ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:30:52.172 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:52.168+0000 7ffbd777d740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563a7e22b460 0x563a7e228430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:30:52.172 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:52.168+0000 7ffbd777d740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x563a7e22e2d0 con 0x563a7e222dd0 2024-03-21T10:30:52.173 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:52.168+0000 7ffbd777d740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x563a7e141e80 con 0x563a7e22b460 2024-03-21T10:30:52.173 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:52.168+0000 7ffbd777d740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x563a7e22e450 con 0x563a7e2220f0 2024-03-21T10:30:52.173 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:52.168+0000 7ffbd777d740 10 client.0 _ll_register_callbacks cb 0x563a7e154130 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T10:30:52.173 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:52.168+0000 7ffbcd400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x563a7e222dd0 0x563a7e227ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:30:52.173 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:52.168+0000 7ffbcd400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x563a7e222dd0 0x563a7e227ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:49132/0 (socket says 172.21.15.47:49132) 2024-03-21T10:30:52.173 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:52.168+0000 7ffbcd400640 1 -- 172.21.15.47:0/1145453168 learned_addr learned my addr 172.21.15.47:0/1145453168 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:30:52.173 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:52.168+0000 7ffbce800640 1 --2- 172.21.15.47:0/1145453168 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563a7e22b460 0x563a7e228430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:30:52.173 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:52.168+0000 7ffbce800640 1 --2- 172.21.15.47:0/1145453168 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563a7e2220f0 0x563a7e22ab60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:30:52.173 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:52.169+0000 7ffbcd400640 1 -- 172.21.15.47:0/1145453168 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563a7e2220f0 msgr2=0x563a7e22ab60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:30:52.173 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:52.169+0000 7ffbcd400640 1 --2- 172.21.15.47:0/1145453168 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563a7e2220f0 0x563a7e22ab60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:30:52.173 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:52.169+0000 7ffbcd400640 1 -- 172.21.15.47:0/1145453168 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563a7e22b460 msgr2=0x563a7e228430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:30:52.173 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:52.169+0000 7ffbcd400640 1 --2- 172.21.15.47:0/1145453168 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563a7e22b460 0x563a7e228430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:30:52.173 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:52.169+0000 7ffbcd400640 1 -- 172.21.15.47:0/1145453168 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x563a7df647d0 con 0x563a7e222dd0 2024-03-21T10:30:52.174 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:52.169+0000 7ffbce800640 1 --2- 172.21.15.47:0/1145453168 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563a7e22b460 0x563a7e228430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T10:30:52.174 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:52.169+0000 7ffbcd400640 1 --2- 172.21.15.47:0/1145453168 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x563a7e222dd0 0x563a7e227ef0 secure :-1 s=READY pgs=929 cs=0 l=1 rev1=1 crypto rx=0x7ffbbc004430 tx=0x7ffbbc0039c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bc0afcfcfaa56635 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:30:52.174 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:52.169+0000 7ffbaf400640 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-03-21T10:30:52.174 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:52.169+0000 7ffbaf400640 1 -- 172.21.15.47:0/1145453168 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffbbc011710 con 0x563a7e222dd0 2024-03-21T10:30:52.174 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:52.169+0000 7ffbaf400640 1 -- 172.21.15.47:0/1145453168 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(97..97 src has 1..97) v4 ==== 9474+0+0 (secure 0 0 0) 0x7ffbbc0118b0 con 0x563a7e222dd0 2024-03-21T10:30:52.174 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:52.169+0000 7ffbd777d740 1 -- 172.21.15.47:0/1145453168 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ffbc80099c0 con 0x563a7e222dd0 2024-03-21T10:30:52.174 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:52.170+0000 7ffbaf400640 1 -- 172.21.15.47:0/1145453168 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7ffbbc011ab0 con 0x563a7e222dd0 2024-03-21T10:30:52.174 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:52.170+0000 7ffbd777d740 10 client.17412 fetch_fsmap learned FSMap version 32 2024-03-21T10:30:52.174 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:52.170+0000 7ffbd777d740 1 -- 172.21.15.47:0/1145453168 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x563a7de9a3f0 con 0x563a7e222dd0 2024-03-21T10:30:52.174 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:52.170+0000 7ffbaf400640 1 -- 172.21.15.47:0/1145453168 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7ffbc80099c0 con 0x563a7e222dd0 2024-03-21T10:30:52.174 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:52.170+0000 7ffbaf400640 1 -- 172.21.15.47:0/1145453168 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 32) v1 ==== 7942+0+0 (secure 0 0 0) 0x7ffbbc010440 con 0x563a7e222dd0 2024-03-21T10:30:52.174 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:52.170+0000 7ffbd777d740 10 client.17412 fetch_fsmap finished waiting for FSMap version 32 2024-03-21T10:30:52.174 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:52.170+0000 7ffbd777d740 10 client.17412 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-21T10:30:52.175 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:52.170+0000 7ffbd777d740 20 client.17412 populate_metadata read hostname 'smithi047' 2024-03-21T10:30:52.175 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:52.170+0000 7ffbd777d740 1 --2- 172.21.15.47:0/1145453168 >> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] conn(0x563a7e23b480 0x563a7e23d870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:30:52.175 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:52.170+0000 7ffbd777d740 4 client.17412 mds_command: new command op to 34535 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T10:30:52.175 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:52.170+0000 7ffbd777d740 1 -- 172.21.15.47:0/1145453168 --> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x563a7e193790 con 0x563a7e23b480 2024-03-21T10:30:52.175 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:52.171+0000 7ffbce800640 1 --2- 172.21.15.47:0/1145453168 >> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] conn(0x563a7e23b480 0x563a7e23d870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:30:52.175 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:52.171+0000 7ffbce800640 1 --2- 172.21.15.47:0/1145453168 >> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] conn(0x563a7e23b480 0x563a7e23d870 crc :-1 s=READY pgs=151 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b009c873e46a0f4b server_cookie=924b8ca9f2d408bf in_seq=0 out_seq=0 2024-03-21T10:30:52.175 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:52.171+0000 7ffbaf400640 10 client.17412 ms_handle_connect on v2:172.21.15.100:6838/4173168440 2024-03-21T10:30:52.176 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:52.172+0000 7ffbaf400640 1 -- 172.21.15.47:0/1145453168 <== mds.0 v2:172.21.15.100:6838/4173168440 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x563a7e193790 con 0x563a7e23b480 2024-03-21T10:30:52.176 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:52.172+0000 7ffbaf400640 10 client.17412 handle_command_reply: tid=0 2024-03-21T10:30:52.176 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:52.173+0000 7ffbd777d740 1 client.17412 shutdown 2024-03-21T10:30:52.177 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:52.173+0000 7ffbd777d740 1 -- 172.21.15.47:0/1145453168 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x563a7e222dd0 msgr2=0x563a7e227ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:30:52.177 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:52.173+0000 7ffbd777d740 1 --2- 172.21.15.47:0/1145453168 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x563a7e222dd0 0x563a7e227ef0 secure :-1 s=READY pgs=929 cs=0 l=1 rev1=1 crypto rx=0x7ffbbc004430 tx=0x7ffbbc0039c0 comp rx=0 tx=0).stop 2024-03-21T10:30:52.177 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:52.173+0000 7ffbd777d740 1 -- 172.21.15.47:0/1145453168 shutdown_connections 2024-03-21T10:30:52.177 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:52.173+0000 7ffbd777d740 1 --2- 172.21.15.47:0/1145453168 >> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] conn(0x563a7e23b480 0x563a7e23d870 crc :-1 s=READY pgs=151 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:30:52.177 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:52.173+0000 7ffbd777d740 1 --2- 172.21.15.47:0/1145453168 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563a7e22b460 0x563a7e228430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:30:52.177 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:52.173+0000 7ffbd777d740 1 --2- 172.21.15.47:0/1145453168 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x563a7e222dd0 0x563a7e227ef0 unknown :-1 s=CLOSED pgs=929 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:30:52.177 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:52.173+0000 7ffbd777d740 1 --2- 172.21.15.47:0/1145453168 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563a7e2220f0 0x563a7e22ab60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:30:52.177 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:52.173+0000 7ffbaf400640 0 client.17412 ms_handle_reset on v2:172.21.15.100:6838/4173168440 2024-03-21T10:30:52.177 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:52.173+0000 7ffbd777d740 1 -- 172.21.15.47:0/1145453168 >> 172.21.15.47:0/1145453168 conn(0x563a7e1e9200 msgr2=0x563a7e21e890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:30:52.177 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:52.173+0000 7ffbd777d740 1 -- 172.21.15.47:0/1145453168 shutdown_connections 2024-03-21T10:30:52.177 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:52.173+0000 7ffbd777d740 1 -- 172.21.15.47:0/1145453168 wait complete. 2024-03-21T10:30:52.177 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:52.173+0000 7ffbd777d740 20 client.17412 trim_cache size 0 max 16384 2024-03-21T10:30:52.193 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:52.191+0000 7ffbd777d740 1 Processor -- start 2024-03-21T10:30:52.194 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:52.191+0000 7ffbd777d740 1 -- start start 2024-03-21T10:30:52.194 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:52.191+0000 7ffbd777d740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563a7e2220f0 0x563a7e22a620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:30:52.194 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:52.191+0000 7ffbd777d740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563a7e222dd0 0x563a7e22ab60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:30:52.194 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:52.191+0000 7ffbd777d740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x563a7e22b460 0x563a7e22c850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:30:52.194 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:52.191+0000 7ffbd777d740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffbd01b4ca0 con 0x563a7e22b460 2024-03-21T10:30:52.194 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:52.191+0000 7ffbd777d740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffbd01b6620 con 0x563a7e2220f0 2024-03-21T10:30:52.194 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:52.191+0000 7ffbd777d740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffbd01b5300 con 0x563a7e222dd0 2024-03-21T10:30:52.194 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:52.192+0000 7ffbce800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563a7e2220f0 0x563a7e22a620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:30:52.195 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:52.192+0000 7ffbce800640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x563a7e22b460 0x563a7e22c850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:30:52.195 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:52.192+0000 7ffbce800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563a7e2220f0 0x563a7e22a620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:59868/0 (socket says 172.21.15.47:59868) 2024-03-21T10:30:52.195 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:52.192+0000 7ffbce800640 1 -- 172.21.15.47:0/2936920379 learned_addr learned my addr 172.21.15.47:0/2936920379 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:30:52.195 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:52.192+0000 7ffbcd400640 1 --2- 172.21.15.47:0/2936920379 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563a7e222dd0 0x563a7e22ab60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:30:52.195 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:52.192+0000 7ffbce800640 1 -- 172.21.15.47:0/2936920379 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563a7e222dd0 msgr2=0x563a7e22ab60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:30:52.195 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:52.192+0000 7ffbce800640 1 --2- 172.21.15.47:0/2936920379 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563a7e222dd0 0x563a7e22ab60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:30:52.196 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:52.192+0000 7ffbce800640 1 -- 172.21.15.47:0/2936920379 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x563a7e22b460 msgr2=0x563a7e22c850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:30:52.196 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:52.192+0000 7ffbce800640 1 --2- 172.21.15.47:0/2936920379 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x563a7e22b460 0x563a7e22c850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:30:52.196 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:52.192+0000 7ffbce800640 1 -- 172.21.15.47:0/2936920379 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563a7e23dbe0 con 0x563a7e2220f0 2024-03-21T10:30:52.196 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:52.193+0000 7ffbce800640 1 --2- 172.21.15.47:0/2936920379 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563a7e2220f0 0x563a7e22a620 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7ffbc8006010 tx=0x7ffbc8014040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=afda85a6a7919733 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:30:52.196 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:52.193+0000 7ffbba600640 1 -- 172.21.15.47:0/2936920379 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffbc8032070 con 0x563a7e2220f0 2024-03-21T10:30:52.196 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:52.193+0000 7ffbba600640 1 -- 172.21.15.47:0/2936920379 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7ffbc802cd00 con 0x563a7e2220f0 2024-03-21T10:30:52.196 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:52.193+0000 7ffbba600640 1 -- 172.21.15.47:0/2936920379 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffbc8012870 con 0x563a7e2220f0 2024-03-21T10:30:52.201 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:52.194+0000 7ffbd777d740 1 -- 172.21.15.47:0/2936920379 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563a7e2220f0 msgr2=0x563a7e22a620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:30:52.201 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:52.194+0000 7ffbd777d740 1 --2- 172.21.15.47:0/2936920379 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563a7e2220f0 0x563a7e22a620 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7ffbc8006010 tx=0x7ffbc8014040 comp rx=0 tx=0).stop 2024-03-21T10:30:52.201 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:52.194+0000 7ffbd777d740 1 -- 172.21.15.47:0/2936920379 shutdown_connections 2024-03-21T10:30:52.201 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:52.194+0000 7ffbd777d740 1 --2- 172.21.15.47:0/2936920379 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x563a7e22b460 0x563a7e22c850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:30:52.201 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:52.194+0000 7ffbd777d740 1 --2- 172.21.15.47:0/2936920379 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563a7e222dd0 0x563a7e22ab60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:30:52.201 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:52.194+0000 7ffbd777d740 1 --2- 172.21.15.47:0/2936920379 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563a7e2220f0 0x563a7e22a620 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:30:52.202 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:52.194+0000 7ffbd777d740 1 -- 172.21.15.47:0/2936920379 >> 172.21.15.47:0/2936920379 conn(0x563a7e1e9200 msgr2=0x563a7e21c670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:30:52.202 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:52.194+0000 7ffbd777d740 1 -- 172.21.15.47:0/2936920379 shutdown_connections 2024-03-21T10:30:52.202 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:52.194+0000 7ffbd777d740 1 -- 172.21.15.47:0/2936920379 wait complete. 2024-03-21T10:30:52.202 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:52.194+0000 7ffbd777d740 1 -- start start 2024-03-21T10:30:52.202 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:52.195+0000 7ffbd777d740 1 Processor -- start 2024-03-21T10:30:52.202 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:52.195+0000 7ffbd777d740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x563a7e2220f0 0x563a7e225190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:30:52.202 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:52.195+0000 7ffbd777d740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563a7e222dd0 0x563a7e2256d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:30:52.202 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:52.195+0000 7ffbd777d740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563a7e22b460 0x563a7e225c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:30:52.202 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:52.195+0000 7ffbd777d740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffbd01b5fc0 con 0x563a7e2220f0 2024-03-21T10:30:52.203 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:52.195+0000 7ffbd777d740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffbd01b5960 con 0x563a7e22b460 2024-03-21T10:30:52.203 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:52.195+0000 7ffbd777d740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffbd01b2e60 con 0x563a7e222dd0 2024-03-21T10:30:52.203 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:52.195+0000 7ffbd777d740 10 client.0 _ll_register_callbacks cb 0x7ffbbc00bb60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T10:30:52.203 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:52.195+0000 7ffbce800640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x563a7e2220f0 0x563a7e225190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:30:52.203 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:52.195+0000 7ffbce800640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x563a7e2220f0 0x563a7e225190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:49138/0 (socket says 172.21.15.47:49138) 2024-03-21T10:30:52.203 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:52.195+0000 7ffbce800640 1 -- 172.21.15.47:0/1935686504 learned_addr learned my addr 172.21.15.47:0/1935686504 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:30:52.203 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:52.196+0000 7ffbcd400640 1 --2- 172.21.15.47:0/1935686504 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563a7e222dd0 0x563a7e2256d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:30:52.204 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:52.196+0000 7ffbce800640 1 --2- 172.21.15.47:0/1935686504 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563a7e22b460 0x563a7e225c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:30:52.204 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:52.196+0000 7ffbce800640 1 -- 172.21.15.47:0/1935686504 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563a7e222dd0 msgr2=0x563a7e2256d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:30:52.204 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:52.196+0000 7ffbce800640 1 --2- 172.21.15.47:0/1935686504 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563a7e222dd0 0x563a7e2256d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:30:52.204 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:52.196+0000 7ffbce800640 1 -- 172.21.15.47:0/1935686504 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563a7e22b460 msgr2=0x563a7e225c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:30:52.204 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:52.196+0000 7ffbce800640 1 --2- 172.21.15.47:0/1935686504 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563a7e22b460 0x563a7e225c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:30:52.204 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:52.196+0000 7ffbce800640 1 -- 172.21.15.47:0/1935686504 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x563a7de5a0f0 con 0x563a7e2220f0 2024-03-21T10:30:52.204 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:52.196+0000 7ffbce800640 1 --2- 172.21.15.47:0/1935686504 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x563a7e2220f0 0x563a7e225190 secure :-1 s=READY pgs=930 cs=0 l=1 rev1=1 crypto rx=0x7ffbc8009cd0 tx=0x7ffbc8009e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=15e95c713fc32570 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:30:52.204 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:52.196+0000 7ffbaf400640 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-03-21T10:30:52.204 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:52.196+0000 7ffbaf400640 1 -- 172.21.15.47:0/1935686504 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffbc803c060 con 0x563a7e2220f0 2024-03-21T10:30:52.204 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:52.197+0000 7ffbd777d740 1 -- 172.21.15.47:0/1935686504 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ffbc802cd00 con 0x563a7e2220f0 2024-03-21T10:30:52.204 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:52.197+0000 7ffbaf400640 1 -- 172.21.15.47:0/1935686504 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(97..97 src has 1..97) v4 ==== 9474+0+0 (secure 0 0 0) 0x7ffbc802c850 con 0x563a7e2220f0 2024-03-21T10:30:52.204 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:52.197+0000 7ffbaf400640 1 -- 172.21.15.47:0/1935686504 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7ffbc8043050 con 0x563a7e2220f0 2024-03-21T10:30:52.205 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:52.197+0000 7ffbd777d740 10 client.17418 fetch_fsmap learned FSMap version 32 2024-03-21T10:30:52.205 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:52.197+0000 7ffbd777d740 1 -- 172.21.15.47:0/1935686504 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x563a7e2970b0 con 0x563a7e2220f0 2024-03-21T10:30:52.205 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:52.197+0000 7ffbaf400640 1 -- 172.21.15.47:0/1935686504 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7ffbc802cd00 con 0x563a7e2220f0 2024-03-21T10:30:52.205 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:52.197+0000 7ffbaf400640 1 -- 172.21.15.47:0/1935686504 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 32) v1 ==== 7942+0+0 (secure 0 0 0) 0x7ffbc8025040 con 0x563a7e2220f0 2024-03-21T10:30:52.205 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:52.197+0000 7ffbd777d740 10 client.17418 fetch_fsmap finished waiting for FSMap version 32 2024-03-21T10:30:52.205 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:52.197+0000 7ffbd777d740 10 client.17418 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-21T10:30:52.205 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:52.197+0000 7ffbd777d740 20 client.17418 populate_metadata read hostname 'smithi047' 2024-03-21T10:30:52.205 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:52.197+0000 7ffbd777d740 1 --2- 172.21.15.47:0/1935686504 >> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] conn(0x563a7e297540 0x563a7e299930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:30:52.205 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:52.197+0000 7ffbd777d740 4 client.17418 mds_command: new command op to 34535 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-21T10:30:52.205 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:52.197+0000 7ffbd777d740 1 -- 172.21.15.47:0/1935686504 --> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x563a7e18dae0 con 0x563a7e297540 2024-03-21T10:30:52.205 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:52.198+0000 7ffbcd400640 1 --2- 172.21.15.47:0/1935686504 >> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] conn(0x563a7e297540 0x563a7e299930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:30:52.205 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:52.198+0000 7ffbcd400640 1 --2- 172.21.15.47:0/1935686504 >> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] conn(0x563a7e297540 0x563a7e299930 crc :-1 s=READY pgs=152 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d11a35375ad22f85 server_cookie=931fe8205b041d5a in_seq=0 out_seq=0 2024-03-21T10:30:52.205 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:52.198+0000 7ffbaf400640 10 client.17418 ms_handle_connect on v2:172.21.15.100:6838/4173168440 2024-03-21T10:30:52.205 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:52.199+0000 7ffbaf400640 1 -- 172.21.15.47:0/1935686504 <== mds.0 v2:172.21.15.100:6838/4173168440 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x563a7e18dae0 con 0x563a7e297540 2024-03-21T10:30:52.206 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:52.199+0000 7ffbaf400640 10 client.17418 handle_command_reply: tid=0 2024-03-21T10:30:52.206 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:52.199+0000 7ffbd777d740 1 client.17418 shutdown 2024-03-21T10:30:52.206 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:52.199+0000 7ffbd777d740 1 -- 172.21.15.47:0/1935686504 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x563a7e2220f0 msgr2=0x563a7e225190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:30:52.206 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:52.199+0000 7ffbd777d740 1 --2- 172.21.15.47:0/1935686504 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x563a7e2220f0 0x563a7e225190 secure :-1 s=READY pgs=930 cs=0 l=1 rev1=1 crypto rx=0x7ffbc8009cd0 tx=0x7ffbc8009e70 comp rx=0 tx=0).stop 2024-03-21T10:30:52.206 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:52.199+0000 7ffbd777d740 1 -- 172.21.15.47:0/1935686504 shutdown_connections 2024-03-21T10:30:52.206 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:52.199+0000 7ffbd777d740 1 --2- 172.21.15.47:0/1935686504 >> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] conn(0x563a7e297540 0x563a7e299930 crc :-1 s=READY pgs=152 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:30:52.206 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:52.199+0000 7ffbd777d740 1 --2- 172.21.15.47:0/1935686504 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563a7e22b460 0x563a7e225c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:30:52.206 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:52.199+0000 7ffbd777d740 1 --2- 172.21.15.47:0/1935686504 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563a7e222dd0 0x563a7e2256d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:30:52.206 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:52.199+0000 7ffbd777d740 1 --2- 172.21.15.47:0/1935686504 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x563a7e2220f0 0x563a7e225190 unknown :-1 s=CLOSED pgs=930 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:30:52.206 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:52.199+0000 7ffbd777d740 1 -- 172.21.15.47:0/1935686504 >> 172.21.15.47:0/1935686504 conn(0x563a7e1e9200 msgr2=0x563a7e2266f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:30:52.206 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:52.199+0000 7ffbaf400640 0 client.17418 ms_handle_reset on v2:172.21.15.100:6838/4173168440 2024-03-21T10:30:52.206 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:52.199+0000 7ffbd777d740 1 -- 172.21.15.47:0/1935686504 shutdown_connections 2024-03-21T10:30:52.206 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:52.199+0000 7ffbd777d740 1 -- 172.21.15.47:0/1935686504 wait complete. 2024-03-21T10:30:52.206 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:52.199+0000 7ffbd777d740 20 client.17418 trim_cache size 0 max 16384 2024-03-21T10:30:52.207 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-03-21T10:30:52.207 INFO:teuthology.orchestra.run.smithi047.stdout: "status": "IDLE (waiting for more scrubs)", 2024-03-21T10:30:52.207 INFO:teuthology.orchestra.run.smithi047.stdout: "scrubs": { 2024-03-21T10:30:52.207 INFO:teuthology.orchestra.run.smithi047.stdout: "0d860191-5721-42c3-9876-0c8234119bff": { 2024-03-21T10:30:52.207 INFO:teuthology.orchestra.run.smithi047.stdout: "path": "/", 2024-03-21T10:30:52.207 INFO:teuthology.orchestra.run.smithi047.stdout: "tag": "0d860191-5721-42c3-9876-0c8234119bff", 2024-03-21T10:30:52.207 INFO:teuthology.orchestra.run.smithi047.stdout: "options": "recursive,force" 2024-03-21T10:30:52.207 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-21T10:30:52.207 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-21T10:30:52.207 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-03-21T10:30:52.208 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:52.200+0000 7ffbcf200640 1 -- 172.21.15.47:0/3117427933 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7ffbb40a3090 msgr2=0x7ffbb40a5550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:30:52.208 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:52.200+0000 7ffbcf200640 1 --2- 172.21.15.47:0/3117427933 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7ffbb40a3090 0x7ffbb40a5550 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x563a7e223320 tx=0x7ffbc0000bc0 comp rx=0 tx=0).stop 2024-03-21T10:30:52.208 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:52.200+0000 7ffbcf200640 1 -- 172.21.15.47:0/3117427933 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ffbd0107e50 msgr2=0x7ffbd019e380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:30:52.208 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:52.200+0000 7ffbcf200640 1 --2- 172.21.15.47:0/3117427933 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ffbd0107e50 0x7ffbd019e380 secure :-1 s=READY pgs=927 cs=0 l=1 rev1=1 crypto rx=0x7ffbc0005940 tx=0x7ffbc00045e0 comp rx=0 tx=0).stop 2024-03-21T10:30:52.208 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:52.200+0000 7ffbcf200640 1 -- 172.21.15.47:0/3117427933 shutdown_connections 2024-03-21T10:30:52.208 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:52.200+0000 7ffbcf200640 1 --2- 172.21.15.47:0/3117427933 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ffbd01a2c40 0x7ffbd01a30f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:30:52.209 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:52.200+0000 7ffbcf200640 1 --2- 172.21.15.47:0/3117427933 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7ffbb40a3090 0x7ffbb40a5550 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:30:52.209 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:52.200+0000 7ffbcf200640 1 --2- 172.21.15.47:0/3117427933 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ffbd010cea0 0x7ffbd019e8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:30:52.209 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:52.200+0000 7ffbcf200640 1 --2- 172.21.15.47:0/3117427933 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ffbd0107e50 0x7ffbd019e380 unknown :-1 s=CLOSED pgs=927 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:30:52.209 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:52.200+0000 7ffbcf200640 1 -- 172.21.15.47:0/3117427933 >> 172.21.15.47:0/3117427933 conn(0x7ffbd00774c0 msgr2=0x7ffbd010e020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:30:52.209 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:52.200+0000 7ffbcf200640 1 -- 172.21.15.47:0/3117427933 shutdown_connections 2024-03-21T10:30:52.209 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:30:52.200+0000 7ffbcf200640 1 -- 172.21.15.47:0/3117427933 wait complete. 2024-03-21T10:30:52.218 INFO:tasks.cephfs.filesystem:scrub status for tag:0d860191-5721-42c3-9876-0c8234119bff - {'path': '/', 'tag': '0d860191-5721-42c3-9876-0c8234119bff', 'options': 'recursive,force'} 2024-03-21T10:30:53.712 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:30:53 smithi047 ceph-mon[31547]: scrub queued for path: / 2024-03-21T10:30:53.712 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:30:53 smithi047 ceph-mon[31547]: scrub summary: idle+waiting paths [/] 2024-03-21T10:30:53.712 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:30:53 smithi047 ceph-mon[31547]: scrub summary: active paths [/] 2024-03-21T10:30:53.712 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:30:53 smithi047 ceph-mon[31547]: scrub summary: idle+waiting paths [/] 2024-03-21T10:30:53.712 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:30:53 smithi047 ceph-mon[31547]: pgmap v170: 129 pgs: 129 active+clean; 277 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-03-21T10:30:53.727 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:30:53 smithi100 ceph-mon[35193]: scrub queued for path: / 2024-03-21T10:30:53.727 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:30:53 smithi100 ceph-mon[35193]: scrub summary: idle+waiting paths [/] 2024-03-21T10:30:53.727 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:30:53 smithi100 ceph-mon[35193]: scrub summary: active paths [/] 2024-03-21T10:30:53.727 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:30:53 smithi100 ceph-mon[35193]: scrub summary: idle+waiting paths [/] 2024-03-21T10:30:53.727 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:30:53 smithi100 ceph-mon[35193]: pgmap v170: 129 pgs: 129 active+clean; 277 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-03-21T10:30:53.737 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:30:53 smithi157 ceph-mon[35581]: scrub queued for path: / 2024-03-21T10:30:53.738 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:30:53 smithi157 ceph-mon[35581]: scrub summary: idle+waiting paths [/] 2024-03-21T10:30:53.738 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:30:53 smithi157 ceph-mon[35581]: scrub summary: active paths [/] 2024-03-21T10:30:53.738 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:30:53 smithi157 ceph-mon[35581]: scrub summary: idle+waiting paths [/] 2024-03-21T10:30:53.738 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:30:53 smithi157 ceph-mon[35581]: pgmap v170: 129 pgs: 129 active+clean; 277 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-03-21T10:30:54.711 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:30:54 smithi047 ceph-mon[31547]: scrub summary: idle 2024-03-21T10:30:54.727 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:30:54 smithi100 ceph-mon[35193]: scrub summary: idle 2024-03-21T10:30:54.737 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:30:54 smithi157 ceph-mon[35581]: scrub summary: idle 2024-03-21T10:30:55.712 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:30:55 smithi047 ceph-mon[31547]: pgmap v171: 129 pgs: 129 active+clean; 277 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-21T10:30:55.727 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:30:55 smithi100 ceph-mon[35193]: pgmap v171: 129 pgs: 129 active+clean; 277 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-21T10:30:55.737 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:30:55 smithi157 ceph-mon[35581]: pgmap v171: 129 pgs: 129 active+clean; 277 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-21T10:30:57.712 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:30:57 smithi047 ceph-mon[31547]: pgmap v172: 129 pgs: 129 active+clean; 277 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 12 op/s 2024-03-21T10:30:57.727 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:30:57 smithi100 ceph-mon[35193]: pgmap v172: 129 pgs: 129 active+clean; 277 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 12 op/s 2024-03-21T10:30:57.737 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:30:57 smithi157 ceph-mon[35581]: pgmap v172: 129 pgs: 129 active+clean; 277 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 12 op/s 2024-03-21T10:30:59.712 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:30:59 smithi047 ceph-mon[31547]: pgmap v173: 129 pgs: 129 active+clean; 277 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 9 op/s 2024-03-21T10:30:59.727 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:30:59 smithi100 ceph-mon[35193]: pgmap v173: 129 pgs: 129 active+clean; 277 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 9 op/s 2024-03-21T10:30:59.737 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:30:59 smithi157 ceph-mon[35581]: pgmap v173: 129 pgs: 129 active+clean; 277 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 9 op/s 2024-03-21T10:31:01.712 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:31:01 smithi047 ceph-mon[31547]: pgmap v174: 129 pgs: 129 active+clean; 277 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 9 op/s 2024-03-21T10:31:01.727 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:31:01 smithi100 ceph-mon[35193]: pgmap v174: 129 pgs: 129 active+clean; 277 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 9 op/s 2024-03-21T10:31:01.737 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:31:01 smithi157 ceph-mon[35581]: pgmap v174: 129 pgs: 129 active+clean; 277 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 9 op/s 2024-03-21T10:31:03.712 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:31:03 smithi047 ceph-mon[31547]: pgmap v175: 129 pgs: 129 active+clean; 277 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 KiB/s wr, 27 op/s 2024-03-21T10:31:03.727 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:31:03 smithi100 ceph-mon[35193]: pgmap v175: 129 pgs: 129 active+clean; 277 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 KiB/s wr, 27 op/s 2024-03-21T10:31:03.737 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:31:03 smithi157 ceph-mon[35581]: pgmap v175: 129 pgs: 129 active+clean; 277 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 KiB/s wr, 27 op/s 2024-03-21T10:31:05.712 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:31:05 smithi047 ceph-mon[31547]: pgmap v176: 129 pgs: 129 active+clean; 277 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-03-21T10:31:05.727 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:31:05 smithi100 ceph-mon[35193]: pgmap v176: 129 pgs: 129 active+clean; 277 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-03-21T10:31:05.737 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:31:05 smithi157 ceph-mon[35581]: pgmap v176: 129 pgs: 129 active+clean; 277 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-03-21T10:31:07.712 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:31:07 smithi047 ceph-mon[31547]: pgmap v177: 129 pgs: 129 active+clean; 277 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 KiB/s wr, 28 op/s 2024-03-21T10:31:07.727 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:31:07 smithi100 ceph-mon[35193]: pgmap v177: 129 pgs: 129 active+clean; 277 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 KiB/s wr, 28 op/s 2024-03-21T10:31:07.737 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:31:07 smithi157 ceph-mon[35581]: pgmap v177: 129 pgs: 129 active+clean; 277 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 KiB/s wr, 28 op/s 2024-03-21T10:31:09.712 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:31:09 smithi047 ceph-mon[31547]: pgmap v178: 129 pgs: 129 active+clean; 277 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 24 op/s 2024-03-21T10:31:09.727 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:31:09 smithi100 ceph-mon[35193]: pgmap v178: 129 pgs: 129 active+clean; 277 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 24 op/s 2024-03-21T10:31:09.737 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:31:09 smithi157 ceph-mon[35581]: pgmap v178: 129 pgs: 129 active+clean; 277 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 24 op/s 2024-03-21T10:31:11.712 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:31:11 smithi047 ceph-mon[31547]: pgmap v179: 129 pgs: 129 active+clean; 277 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-03-21T10:31:11.727 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:31:11 smithi100 ceph-mon[35193]: pgmap v179: 129 pgs: 129 active+clean; 277 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-03-21T10:31:11.737 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:31:11 smithi157 ceph-mon[35581]: pgmap v179: 129 pgs: 129 active+clean; 277 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-03-21T10:31:13.712 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:31:13 smithi047 ceph-mon[31547]: pgmap v180: 129 pgs: 129 active+clean; 277 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 KiB/s wr, 27 op/s 2024-03-21T10:31:13.727 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:31:13 smithi100 ceph-mon[35193]: pgmap v180: 129 pgs: 129 active+clean; 277 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 KiB/s wr, 27 op/s 2024-03-21T10:31:13.738 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:31:13 smithi157 ceph-mon[35581]: pgmap v180: 129 pgs: 129 active+clean; 277 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 KiB/s wr, 27 op/s 2024-03-21T10:31:15.712 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:31:15 smithi047 ceph-mon[31547]: pgmap v181: 129 pgs: 129 active+clean; 277 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 170 B/s wr, 9 op/s 2024-03-21T10:31:15.727 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:31:15 smithi100 ceph-mon[35193]: pgmap v181: 129 pgs: 129 active+clean; 277 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 170 B/s wr, 9 op/s 2024-03-21T10:31:15.737 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:31:15 smithi157 ceph-mon[35581]: pgmap v181: 129 pgs: 129 active+clean; 277 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 170 B/s wr, 9 op/s 2024-03-21T10:31:17.712 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:31:17 smithi047 ceph-mon[31547]: pgmap v182: 129 pgs: 129 active+clean; 277 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 170 B/s wr, 13 op/s 2024-03-21T10:31:17.727 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:31:17 smithi100 ceph-mon[35193]: pgmap v182: 129 pgs: 129 active+clean; 277 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 170 B/s wr, 13 op/s 2024-03-21T10:31:17.737 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:31:17 smithi157 ceph-mon[35581]: pgmap v182: 129 pgs: 129 active+clean; 277 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 170 B/s wr, 13 op/s 2024-03-21T10:31:19.712 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:31:19 smithi047 ceph-mon[31547]: pgmap v183: 129 pgs: 129 active+clean; 277 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 85 B/s wr, 8 op/s 2024-03-21T10:31:19.727 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:31:19 smithi100 ceph-mon[35193]: pgmap v183: 129 pgs: 129 active+clean; 277 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 85 B/s wr, 8 op/s 2024-03-21T10:31:19.737 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:31:19 smithi157 ceph-mon[35581]: pgmap v183: 129 pgs: 129 active+clean; 277 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 85 B/s wr, 8 op/s 2024-03-21T10:31:21.711 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:31:21 smithi047 ceph-mon[31547]: pgmap v184: 129 pgs: 129 active+clean; 277 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 8 op/s 2024-03-21T10:31:21.727 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:31:21 smithi100 ceph-mon[35193]: pgmap v184: 129 pgs: 129 active+clean; 277 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 8 op/s 2024-03-21T10:31:21.737 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:31:21 smithi157 ceph-mon[35581]: pgmap v184: 129 pgs: 129 active+clean; 277 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 8 op/s 2024-03-21T10:31:22.219 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-21T10:31:22.220 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-21T10:31:22.456 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:22.456+0000 7f1ea5400640 1 -- 172.21.15.47:0/2288070805 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1ea0101ee0 msgr2=0x7f1ea010e3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:31:22.456 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:22.456+0000 7f1ea5400640 1 --2- 172.21.15.47:0/2288070805 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1ea0101ee0 0x7f1ea010e3a0 secure :-1 s=READY pgs=931 cs=0 l=1 rev1=1 crypto rx=0x7f1e98009ca0 tx=0x7f1e9802c0f0 comp rx=0 tx=0).stop 2024-03-21T10:31:22.456 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:22.456+0000 7f1ea5400640 1 -- 172.21.15.47:0/2288070805 shutdown_connections 2024-03-21T10:31:22.456 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:22.456+0000 7f1ea5400640 1 --2- 172.21.15.47:0/2288070805 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1ea0101ee0 0x7f1ea010e3a0 unknown :-1 s=CLOSED pgs=931 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:31:22.456 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:22.456+0000 7f1ea5400640 1 --2- 172.21.15.47:0/2288070805 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1ea0101520 0x7f1ea01019a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:31:22.456 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:22.456+0000 7f1ea5400640 1 --2- 172.21.15.47:0/2288070805 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1ea01075e0 0x7f1ea01079c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:31:22.456 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:22.456+0000 7f1ea5400640 1 -- 172.21.15.47:0/2288070805 >> 172.21.15.47:0/2288070805 conn(0x7f1ea00fd010 msgr2=0x7f1ea00ff430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:31:22.456 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:22.456+0000 7f1ea5400640 1 -- 172.21.15.47:0/2288070805 shutdown_connections 2024-03-21T10:31:22.456 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:22.456+0000 7f1ea5400640 1 -- 172.21.15.47:0/2288070805 wait complete. 2024-03-21T10:31:22.457 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:22.458+0000 7f1ea5400640 1 Processor -- start 2024-03-21T10:31:22.457 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:22.458+0000 7f1ea5400640 1 -- start start 2024-03-21T10:31:22.458 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:22.459+0000 7f1ea5400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1ea0101520 0x7f1ea00739a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:31:22.458 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:22.459+0000 7f1ea5400640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1ea0101ee0 0x7f1ea0073ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:31:22.459 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:22.459+0000 7f1ea5400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1ea01075e0 0x7f1ea0074420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:31:22.459 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:22.459+0000 7f1ea5400640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ea0111770 con 0x7f1ea01075e0 2024-03-21T10:31:22.459 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:22.459+0000 7f1ea5400640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ea01115f0 con 0x7f1ea0101520 2024-03-21T10:31:22.459 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:22.459+0000 7f1ea5400640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ea01118f0 con 0x7f1ea0101ee0 2024-03-21T10:31:22.459 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:22.459+0000 7f1ea4a00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1ea01075e0 0x7f1ea0074420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:31:22.459 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:22.459+0000 7f1e9fe00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1ea0101520 0x7f1ea00739a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:31:22.459 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:22.459+0000 7f1e9fe00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1ea0101520 0x7f1ea00739a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:58544/0 (socket says 172.21.15.47:58544) 2024-03-21T10:31:22.459 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:22.459+0000 7f1e9fe00640 1 -- 172.21.15.47:0/1037222169 learned_addr learned my addr 172.21.15.47:0/1037222169 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:31:22.459 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:22.459+0000 7f1e9f400640 1 --2- 172.21.15.47:0/1037222169 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1ea0101ee0 0x7f1ea0073ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:31:22.459 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:22.459+0000 7f1e9fe00640 1 -- 172.21.15.47:0/1037222169 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1ea0101ee0 msgr2=0x7f1ea0073ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:31:22.459 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:22.460+0000 7f1e9fe00640 1 --2- 172.21.15.47:0/1037222169 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1ea0101ee0 0x7f1ea0073ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:31:22.459 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:22.460+0000 7f1e9fe00640 1 -- 172.21.15.47:0/1037222169 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1ea01075e0 msgr2=0x7f1ea0074420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:31:22.460 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:22.460+0000 7f1e9fe00640 1 --2- 172.21.15.47:0/1037222169 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1ea01075e0 0x7f1ea0074420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:31:22.460 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:22.460+0000 7f1e9fe00640 1 -- 172.21.15.47:0/1037222169 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ea01a6110 con 0x7f1ea0101520 2024-03-21T10:31:22.460 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:22.460+0000 7f1ea4a00640 1 --2- 172.21.15.47:0/1037222169 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1ea01075e0 0x7f1ea0074420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T10:31:22.460 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:22.460+0000 7f1e9fe00640 1 --2- 172.21.15.47:0/1037222169 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1ea0101520 0x7f1ea00739a0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f1e9000e9f0 tx=0x7f1e9000edf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:31:22.461 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:22.460+0000 7f1e9cc00640 1 -- 172.21.15.47:0/1037222169 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e9000cd80 con 0x7f1ea0101520 2024-03-21T10:31:22.461 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:22.460+0000 7f1e9cc00640 1 -- 172.21.15.47:0/1037222169 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f1e900054d0 con 0x7f1ea0101520 2024-03-21T10:31:22.461 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:22.460+0000 7f1ea5400640 1 -- 172.21.15.47:0/1037222169 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1ea01a6400 con 0x7f1ea0101520 2024-03-21T10:31:22.461 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:22.460+0000 7f1e9cc00640 1 -- 172.21.15.47:0/1037222169 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e90010490 con 0x7f1ea0101520 2024-03-21T10:31:22.461 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:22.460+0000 7f1ea5400640 1 -- 172.21.15.47:0/1037222169 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1ea01a6950 con 0x7f1ea0101520 2024-03-21T10:31:22.462 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:22.464+0000 7f1e9cc00640 1 -- 172.21.15.47:0/1037222169 <== mon.2 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 27) v1 ==== 137985+0+0 (secure 0 0 0) 0x7f1e90005020 con 0x7f1ea0101520 2024-03-21T10:31:22.463 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:22.465+0000 7f1ea960a740 1 Processor -- start 2024-03-21T10:31:22.465 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:22.465+0000 7f1ea960a740 1 -- start start 2024-03-21T10:31:22.465 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:22.465+0000 7f1e9cc00640 1 --2- 172.21.15.47:0/1037222169 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f1e840a3140 0x7f1e840a5600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:31:22.465 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:22.465+0000 7f1ea960a740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55cfbd36edd0 0x55cfbd36db20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:31:22.465 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:22.465+0000 7f1ea960a740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55cfbd36e0f0 0x55cfbd36e570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:31:22.465 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:22.466+0000 7f1ea960a740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55cfbd377460 0x55cfbd379850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:31:22.466 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:22.466+0000 7f1ea960a740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cfbd2d2250 con 0x55cfbd36e0f0 2024-03-21T10:31:22.466 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:22.466+0000 7f1ea960a740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cfbd27aa30 con 0x55cfbd377460 2024-03-21T10:31:22.466 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:22.466+0000 7f1e9cc00640 1 -- 172.21.15.47:0/1037222169 <== mon.2 v2:172.21.15.100:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 9474+0+0 (secure 0 0 0) 0x7f1e900cff10 con 0x7f1ea0101520 2024-03-21T10:31:22.466 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:22.466+0000 7f1ea960a740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cfbd2ce2f0 con 0x55cfbd36edd0 2024-03-21T10:31:22.466 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:22.466+0000 7f1ea4a00640 1 --2- 172.21.15.47:0/1037222169 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f1e840a3140 0x7f1e840a5600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:31:22.466 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:22.466+0000 7f1e9f400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55cfbd36e0f0 0x55cfbd36e570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:31:22.466 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:22.466+0000 7f1e9f400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55cfbd36e0f0 0x55cfbd36e570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:39992/0 (socket says 172.21.15.47:39992) 2024-03-21T10:31:22.466 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:22.466+0000 7f1e9f400640 1 -- 172.21.15.47:0/4189348974 learned_addr learned my addr 172.21.15.47:0/4189348974 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:31:22.466 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:22.466+0000 7f1ea4a00640 1 --2- 172.21.15.47:0/4189348974 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55cfbd377460 0x55cfbd379850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:31:22.466 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:22.466+0000 7f1e9fe00640 1 --2- 172.21.15.47:0/4189348974 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55cfbd36edd0 0x55cfbd36db20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:31:22.466 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:22.466+0000 7f1e9f400640 1 -- 172.21.15.47:0/4189348974 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55cfbd36edd0 msgr2=0x55cfbd36db20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:31:22.466 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:22.466+0000 7f1e9f400640 1 --2- 172.21.15.47:0/4189348974 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55cfbd36edd0 0x55cfbd36db20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:31:22.466 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:22.466+0000 7f1e9f400640 1 -- 172.21.15.47:0/4189348974 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55cfbd377460 msgr2=0x55cfbd379850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:31:22.467 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:22.466+0000 7f1e9f400640 1 --2- 172.21.15.47:0/4189348974 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55cfbd377460 0x55cfbd379850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:31:22.467 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:22.466+0000 7f1e9f400640 1 -- 172.21.15.47:0/4189348974 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55cfbd265690 con 0x55cfbd36e0f0 2024-03-21T10:31:22.472 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:22.467+0000 7f1e9f400640 1 --2- 172.21.15.47:0/4189348974 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55cfbd36e0f0 0x55cfbd36e570 secure :-1 s=READY pgs=932 cs=0 l=1 rev1=1 crypto rx=0x7f1e8c009c10 tx=0x7f1e8c0117b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=776551b35c1b41c server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:31:22.472 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:22.467+0000 7f1ea4a00640 1 --2- 172.21.15.47:0/1037222169 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f1e840a3140 0x7f1e840a5600 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f1e98004af0 tx=0x7f1e98032450 comp rx=0 tx=0).ready entity=mgr.15687 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:31:22.472 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:22.467+0000 7f1e95200640 1 -- 172.21.15.47:0/4189348974 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e8c012450 con 0x55cfbd36e0f0 2024-03-21T10:31:22.472 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:22.467+0000 7f1e95200640 1 -- 172.21.15.47:0/4189348974 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f1e8c012aa0 con 0x55cfbd36e0f0 2024-03-21T10:31:22.472 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:22.467+0000 7f1e95200640 1 -- 172.21.15.47:0/4189348974 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e8c00f940 con 0x55cfbd36e0f0 2024-03-21T10:31:22.472 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:22.468+0000 7f1ea960a740 1 -- 172.21.15.47:0/4189348974 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55cfbd36e0f0 msgr2=0x55cfbd36e570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:31:22.472 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:22.468+0000 7f1ea960a740 1 --2- 172.21.15.47:0/4189348974 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55cfbd36e0f0 0x55cfbd36e570 secure :-1 s=READY pgs=932 cs=0 l=1 rev1=1 crypto rx=0x7f1e8c009c10 tx=0x7f1e8c0117b0 comp rx=0 tx=0).stop 2024-03-21T10:31:22.472 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:22.468+0000 7f1ea960a740 1 -- 172.21.15.47:0/4189348974 shutdown_connections 2024-03-21T10:31:22.473 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:22.468+0000 7f1ea960a740 1 --2- 172.21.15.47:0/4189348974 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55cfbd377460 0x55cfbd379850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:31:22.473 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:22.468+0000 7f1ea960a740 1 --2- 172.21.15.47:0/4189348974 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55cfbd36e0f0 0x55cfbd36e570 unknown :-1 s=CLOSED pgs=932 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:31:22.473 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:22.468+0000 7f1ea960a740 1 --2- 172.21.15.47:0/4189348974 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55cfbd36edd0 0x55cfbd36db20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:31:22.473 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:22.468+0000 7f1ea960a740 1 -- 172.21.15.47:0/4189348974 >> 172.21.15.47:0/4189348974 conn(0x55cfbd335200 msgr2=0x55cfbd368b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:31:22.473 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:22.468+0000 7f1ea960a740 1 -- 172.21.15.47:0/4189348974 shutdown_connections 2024-03-21T10:31:22.473 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:22.468+0000 7f1ea960a740 1 -- 172.21.15.47:0/4189348974 wait complete. 2024-03-21T10:31:22.473 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:22.469+0000 7f1ea960a740 1 -- start start 2024-03-21T10:31:22.473 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:22.469+0000 7f1ea960a740 1 Processor -- start 2024-03-21T10:31:22.473 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:22.469+0000 7f1ea960a740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55cfbd36e0f0 0x55cfbd376b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:31:22.473 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:22.469+0000 7f1ea960a740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55cfbd36edd0 0x55cfbd373ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:31:22.473 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:22.469+0000 7f1ea960a740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55cfbd377460 0x55cfbd374430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:31:22.473 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:22.469+0000 7f1ea960a740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cfbd37a2d0 con 0x55cfbd377460 2024-03-21T10:31:22.473 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:22.469+0000 7f1ea960a740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cfbd28de80 con 0x55cfbd36edd0 2024-03-21T10:31:22.474 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:22.469+0000 7f1ea960a740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cfbd37a450 con 0x55cfbd36e0f0 2024-03-21T10:31:22.474 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:22.470+0000 7f1ea960a740 10 client.0 _ll_register_callbacks cb 0x55cfbd2a0130 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T10:31:22.474 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:22.470+0000 7f1ea4a00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55cfbd377460 0x55cfbd374430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:31:22.474 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:22.470+0000 7f1ea4a00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55cfbd377460 0x55cfbd374430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:39996/0 (socket says 172.21.15.47:39996) 2024-03-21T10:31:22.474 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:22.470+0000 7f1ea4a00640 1 -- 172.21.15.47:0/567538258 learned_addr learned my addr 172.21.15.47:0/567538258 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:31:22.474 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:22.470+0000 7f1e9f400640 1 --2- 172.21.15.47:0/567538258 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55cfbd36edd0 0x55cfbd373ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:31:22.474 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:22.470+0000 7f1e9fe00640 1 --2- 172.21.15.47:0/567538258 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55cfbd36e0f0 0x55cfbd376b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:31:22.474 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:22.470+0000 7f1ea4a00640 1 -- 172.21.15.47:0/567538258 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55cfbd36e0f0 msgr2=0x55cfbd376b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:31:22.474 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:22.470+0000 7f1ea4a00640 1 --2- 172.21.15.47:0/567538258 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55cfbd36e0f0 0x55cfbd376b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:31:22.474 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:22.470+0000 7f1ea4a00640 1 -- 172.21.15.47:0/567538258 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55cfbd36edd0 msgr2=0x55cfbd373ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:31:22.474 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:22.470+0000 7f1ea4a00640 1 --2- 172.21.15.47:0/567538258 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55cfbd36edd0 0x55cfbd373ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:31:22.474 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:22.470+0000 7f1ea4a00640 1 -- 172.21.15.47:0/567538258 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55cfbd0b07d0 con 0x55cfbd377460 2024-03-21T10:31:22.474 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:22.471+0000 7f1ea4a00640 1 --2- 172.21.15.47:0/567538258 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55cfbd377460 0x55cfbd374430 secure :-1 s=READY pgs=933 cs=0 l=1 rev1=1 crypto rx=0x7f1e98009ca0 tx=0x7f1e98005c50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9a1b8afb241748d2 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:31:22.475 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:22.471+0000 7f1e89600640 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-03-21T10:31:22.475 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:22.471+0000 7f1e89600640 1 -- 172.21.15.47:0/567538258 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e98039040 con 0x55cfbd377460 2024-03-21T10:31:22.475 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:22.471+0000 7f1e89600640 1 -- 172.21.15.47:0/567538258 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(97..97 src has 1..97) v4 ==== 9474+0+0 (secure 0 0 0) 0x7f1e98043810 con 0x55cfbd377460 2024-03-21T10:31:22.475 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:22.471+0000 7f1ea960a740 1 -- 172.21.15.47:0/567538258 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f1e8c012aa0 con 0x55cfbd377460 2024-03-21T10:31:22.475 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:22.471+0000 7f1e89600640 1 -- 172.21.15.47:0/567538258 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1e9805c050 con 0x55cfbd377460 2024-03-21T10:31:22.475 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:22.471+0000 7f1ea960a740 10 client.17430 fetch_fsmap learned FSMap version 32 2024-03-21T10:31:22.475 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:22.471+0000 7f1ea960a740 1 -- 172.21.15.47:0/567538258 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x55cfbcfe63f0 con 0x55cfbd377460 2024-03-21T10:31:22.475 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:22.472+0000 7f1e89600640 1 -- 172.21.15.47:0/567538258 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f1e8c012aa0 con 0x55cfbd377460 2024-03-21T10:31:22.475 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:22.472+0000 7f1e89600640 1 -- 172.21.15.47:0/567538258 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 32) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f1e9803f040 con 0x55cfbd377460 2024-03-21T10:31:22.475 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:22.472+0000 7f1ea960a740 10 client.17430 fetch_fsmap finished waiting for FSMap version 32 2024-03-21T10:31:22.475 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:22.472+0000 7f1ea960a740 10 client.17430 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-21T10:31:22.475 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:22.472+0000 7f1ea960a740 20 client.17430 populate_metadata read hostname 'smithi047' 2024-03-21T10:31:22.475 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:22.472+0000 7f1ea960a740 1 --2- 172.21.15.47:0/567538258 >> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] conn(0x55cfbd387380 0x55cfbd389790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:31:22.475 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:22.472+0000 7f1ea960a740 4 client.17430 mds_command: new command op to 34535 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T10:31:22.476 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:22.472+0000 7f1ea960a740 1 -- 172.21.15.47:0/567538258 --> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55cfbd2df790 con 0x55cfbd387380 2024-03-21T10:31:22.476 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:22.473+0000 7f1e9fe00640 1 --2- 172.21.15.47:0/567538258 >> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] conn(0x55cfbd387380 0x55cfbd389790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:31:22.477 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:22.473+0000 7f1e9fe00640 1 --2- 172.21.15.47:0/567538258 >> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] conn(0x55cfbd387380 0x55cfbd389790 crc :-1 s=READY pgs=153 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dbbc20bf4926e7bd server_cookie=86eb02e5c26c9cc6 in_seq=0 out_seq=0 2024-03-21T10:31:22.477 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:22.473+0000 7f1e89600640 10 client.17430 ms_handle_connect on v2:172.21.15.100:6838/4173168440 2024-03-21T10:31:22.477 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:22.475+0000 7f1e89600640 1 -- 172.21.15.47:0/567538258 <== mds.0 v2:172.21.15.100:6838/4173168440 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55cfbd2df790 con 0x55cfbd387380 2024-03-21T10:31:22.477 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:22.475+0000 7f1e89600640 10 client.17430 handle_command_reply: tid=0 2024-03-21T10:31:22.477 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:22.475+0000 7f1ea960a740 1 client.17430 shutdown 2024-03-21T10:31:22.477 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:22.475+0000 7f1ea960a740 1 -- 172.21.15.47:0/567538258 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55cfbd377460 msgr2=0x55cfbd374430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:31:22.477 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:22.475+0000 7f1ea960a740 1 --2- 172.21.15.47:0/567538258 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55cfbd377460 0x55cfbd374430 secure :-1 s=READY pgs=933 cs=0 l=1 rev1=1 crypto rx=0x7f1e98009ca0 tx=0x7f1e98005c50 comp rx=0 tx=0).stop 2024-03-21T10:31:22.477 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:22.476+0000 7f1ea960a740 1 -- 172.21.15.47:0/567538258 shutdown_connections 2024-03-21T10:31:22.477 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:22.476+0000 7f1ea960a740 1 --2- 172.21.15.47:0/567538258 >> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] conn(0x55cfbd387380 0x55cfbd389790 crc :-1 s=READY pgs=153 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:31:22.478 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:22.476+0000 7f1ea960a740 1 --2- 172.21.15.47:0/567538258 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55cfbd377460 0x55cfbd374430 unknown :-1 s=CLOSED pgs=933 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:31:22.478 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:22.476+0000 7f1ea960a740 1 --2- 172.21.15.47:0/567538258 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55cfbd36edd0 0x55cfbd373ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:31:22.478 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:22.476+0000 7f1ea960a740 1 --2- 172.21.15.47:0/567538258 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55cfbd36e0f0 0x55cfbd376b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:31:22.478 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:22.476+0000 7f1ea960a740 1 -- 172.21.15.47:0/567538258 >> 172.21.15.47:0/567538258 conn(0x55cfbd335200 msgr2=0x55cfbd36a890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:31:22.478 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:22.476+0000 7f1e89600640 0 client.17430 ms_handle_reset on v2:172.21.15.100:6838/4173168440 2024-03-21T10:31:22.478 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:22.476+0000 7f1ea960a740 1 -- 172.21.15.47:0/567538258 shutdown_connections 2024-03-21T10:31:22.478 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:22.476+0000 7f1ea960a740 1 -- 172.21.15.47:0/567538258 wait complete. 2024-03-21T10:31:22.478 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:22.476+0000 7f1ea960a740 20 client.17430 trim_cache size 0 max 16384 2024-03-21T10:31:22.493 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:22.493+0000 7f1ea960a740 1 Processor -- start 2024-03-21T10:31:22.493 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:22.494+0000 7f1ea960a740 1 -- start start 2024-03-21T10:31:22.493 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:22.494+0000 7f1ea960a740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55cfbd36e0f0 0x55cfbd376620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:31:22.493 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:22.494+0000 7f1ea960a740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55cfbd36edd0 0x55cfbd376b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:31:22.493 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:22.494+0000 7f1ea960a740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55cfbd377460 0x55cfbd378850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:31:22.493 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:22.494+0000 7f1ea960a740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ea01b0880 con 0x55cfbd36edd0 2024-03-21T10:31:22.493 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:22.494+0000 7f1ea960a740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ea01b2200 con 0x55cfbd36e0f0 2024-03-21T10:31:22.494 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:22.494+0000 7f1ea960a740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ea01b0ee0 con 0x55cfbd377460 2024-03-21T10:31:22.494 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:22.494+0000 7f1e9f400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55cfbd36edd0 0x55cfbd376b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:31:22.494 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:22.494+0000 7f1e9f400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55cfbd36edd0 0x55cfbd376b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:40006/0 (socket says 172.21.15.47:40006) 2024-03-21T10:31:22.494 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:22.494+0000 7f1e9f400640 1 -- 172.21.15.47:0/1536789527 learned_addr learned my addr 172.21.15.47:0/1536789527 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:31:22.494 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:22.494+0000 7f1e9fe00640 1 --2- 172.21.15.47:0/1536789527 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55cfbd36e0f0 0x55cfbd376620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:31:22.496 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:22.495+0000 7f1e9f400640 1 -- 172.21.15.47:0/1536789527 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55cfbd377460 msgr2=0x55cfbd378850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:31:22.496 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:22.495+0000 7f1e9f400640 1 --2- 172.21.15.47:0/1536789527 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55cfbd377460 0x55cfbd378850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:31:22.497 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:22.495+0000 7f1e9f400640 1 -- 172.21.15.47:0/1536789527 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55cfbd36e0f0 msgr2=0x55cfbd376620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:31:22.497 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:22.495+0000 7f1e9f400640 1 --2- 172.21.15.47:0/1536789527 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55cfbd36e0f0 0x55cfbd376620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:31:22.497 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:22.495+0000 7f1e9f400640 1 -- 172.21.15.47:0/1536789527 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55cfbcfa60f0 con 0x55cfbd36edd0 2024-03-21T10:31:22.497 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:22.495+0000 7f1e9f400640 1 --2- 172.21.15.47:0/1536789527 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55cfbd36edd0 0x55cfbd376b60 secure :-1 s=READY pgs=934 cs=0 l=1 rev1=1 crypto rx=0x55cfbd36f320 tx=0x7f1e8c009db0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d5b6e95a604c2f5e server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:31:22.497 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:22.495+0000 7f1e8be00640 1 -- 172.21.15.47:0/1536789527 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e8c02b030 con 0x55cfbd36edd0 2024-03-21T10:31:22.497 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:22.495+0000 7f1e8be00640 1 -- 172.21.15.47:0/1536789527 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f1e8c0096c0 con 0x55cfbd36edd0 2024-03-21T10:31:22.497 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:22.495+0000 7f1e8be00640 1 -- 172.21.15.47:0/1536789527 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e8c004680 con 0x55cfbd36edd0 2024-03-21T10:31:22.497 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:22.496+0000 7f1ea960a740 1 -- 172.21.15.47:0/1536789527 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55cfbd36edd0 msgr2=0x55cfbd376b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:31:22.497 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:22.496+0000 7f1ea960a740 1 --2- 172.21.15.47:0/1536789527 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55cfbd36edd0 0x55cfbd376b60 secure :-1 s=READY pgs=934 cs=0 l=1 rev1=1 crypto rx=0x55cfbd36f320 tx=0x7f1e8c009db0 comp rx=0 tx=0).stop 2024-03-21T10:31:22.497 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:22.496+0000 7f1ea960a740 1 -- 172.21.15.47:0/1536789527 shutdown_connections 2024-03-21T10:31:22.497 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:22.496+0000 7f1ea960a740 1 --2- 172.21.15.47:0/1536789527 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55cfbd377460 0x55cfbd378850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:31:22.497 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:22.496+0000 7f1ea960a740 1 --2- 172.21.15.47:0/1536789527 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55cfbd36edd0 0x55cfbd376b60 unknown :-1 s=CLOSED pgs=934 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:31:22.497 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:22.496+0000 7f1ea960a740 1 --2- 172.21.15.47:0/1536789527 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55cfbd36e0f0 0x55cfbd376620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:31:22.497 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:22.496+0000 7f1ea960a740 1 -- 172.21.15.47:0/1536789527 >> 172.21.15.47:0/1536789527 conn(0x55cfbd335200 msgr2=0x55cfbd368670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:31:22.498 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:22.496+0000 7f1ea960a740 1 -- 172.21.15.47:0/1536789527 shutdown_connections 2024-03-21T10:31:22.498 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:22.496+0000 7f1ea960a740 1 -- 172.21.15.47:0/1536789527 wait complete. 2024-03-21T10:31:22.498 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:22.496+0000 7f1ea960a740 1 -- start start 2024-03-21T10:31:22.498 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:22.497+0000 7f1ea960a740 1 Processor -- start 2024-03-21T10:31:22.498 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:22.497+0000 7f1ea960a740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55cfbd36e0f0 0x55cfbd371190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:31:22.498 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:22.497+0000 7f1ea960a740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55cfbd36edd0 0x55cfbd3716d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:31:22.498 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:22.497+0000 7f1ea960a740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55cfbd377460 0x55cfbd371c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:31:22.498 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:22.497+0000 7f1ea960a740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ea01b1ba0 con 0x55cfbd377460 2024-03-21T10:31:22.498 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:22.497+0000 7f1ea960a740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ea01b1540 con 0x55cfbd36edd0 2024-03-21T10:31:22.499 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:22.497+0000 7f1ea960a740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ea01aea40 con 0x55cfbd36e0f0 2024-03-21T10:31:22.499 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:22.497+0000 7f1ea960a740 10 client.0 _ll_register_callbacks cb 0x7f1e90005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T10:31:22.499 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:22.497+0000 7f1ea4a00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55cfbd377460 0x55cfbd371c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:31:22.499 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:22.497+0000 7f1ea4a00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55cfbd377460 0x55cfbd371c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:40016/0 (socket says 172.21.15.47:40016) 2024-03-21T10:31:22.499 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:22.497+0000 7f1ea4a00640 1 -- 172.21.15.47:0/249780706 learned_addr learned my addr 172.21.15.47:0/249780706 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:31:22.499 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:22.498+0000 7f1e9f400640 1 --2- 172.21.15.47:0/249780706 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55cfbd36edd0 0x55cfbd3716d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:31:22.499 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:22.498+0000 7f1e9fe00640 1 --2- 172.21.15.47:0/249780706 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55cfbd36e0f0 0x55cfbd371190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:31:22.499 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:22.498+0000 7f1ea4a00640 1 -- 172.21.15.47:0/249780706 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55cfbd36e0f0 msgr2=0x55cfbd371190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:31:22.499 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:22.498+0000 7f1ea4a00640 1 --2- 172.21.15.47:0/249780706 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55cfbd36e0f0 0x55cfbd371190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:31:22.499 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:22.498+0000 7f1ea4a00640 1 -- 172.21.15.47:0/249780706 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55cfbd36edd0 msgr2=0x55cfbd3716d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:31:22.499 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:22.498+0000 7f1ea4a00640 1 --2- 172.21.15.47:0/249780706 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55cfbd36edd0 0x55cfbd3716d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:31:22.499 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:22.498+0000 7f1ea4a00640 1 -- 172.21.15.47:0/249780706 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55cfbd381b90 con 0x55cfbd377460 2024-03-21T10:31:22.499 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:22.498+0000 7f1e9fe00640 1 --2- 172.21.15.47:0/249780706 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55cfbd36e0f0 0x55cfbd371190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T10:31:22.502 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:22.498+0000 7f1ea4a00640 1 --2- 172.21.15.47:0/249780706 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55cfbd377460 0x55cfbd371c10 secure :-1 s=READY pgs=935 cs=0 l=1 rev1=1 crypto rx=0x7f1e98032b20 tx=0x7f1e980039e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a19a044d06bb5836 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:31:22.502 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:22.498+0000 7f1e89600640 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-03-21T10:31:22.502 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:22.498+0000 7f1e89600640 1 -- 172.21.15.47:0/249780706 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e98039040 con 0x55cfbd377460 2024-03-21T10:31:22.502 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:22.498+0000 7f1e89600640 1 -- 172.21.15.47:0/249780706 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(97..97 src has 1..97) v4 ==== 9474+0+0 (secure 0 0 0) 0x7f1e9804e420 con 0x55cfbd377460 2024-03-21T10:31:22.502 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:22.498+0000 7f1ea960a740 1 -- 172.21.15.47:0/249780706 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f1e8c0096c0 con 0x55cfbd377460 2024-03-21T10:31:22.502 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:22.499+0000 7f1e89600640 1 -- 172.21.15.47:0/249780706 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1e9804e620 con 0x55cfbd377460 2024-03-21T10:31:22.502 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:22.499+0000 7f1ea960a740 10 client.17436 fetch_fsmap learned FSMap version 32 2024-03-21T10:31:22.502 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:22.499+0000 7f1ea960a740 1 -- 172.21.15.47:0/249780706 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x55cfbd3e2e30 con 0x55cfbd377460 2024-03-21T10:31:22.502 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:22.499+0000 7f1e89600640 1 -- 172.21.15.47:0/249780706 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f1e8c0096c0 con 0x55cfbd377460 2024-03-21T10:31:22.502 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:22.499+0000 7f1e89600640 1 -- 172.21.15.47:0/249780706 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 32) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f1e9803f070 con 0x55cfbd377460 2024-03-21T10:31:22.503 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:22.499+0000 7f1ea960a740 10 client.17436 fetch_fsmap finished waiting for FSMap version 32 2024-03-21T10:31:22.503 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:22.499+0000 7f1ea960a740 10 client.17436 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-21T10:31:22.503 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:22.499+0000 7f1ea960a740 20 client.17436 populate_metadata read hostname 'smithi047' 2024-03-21T10:31:22.503 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:22.499+0000 7f1ea960a740 1 --2- 172.21.15.47:0/249780706 >> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] conn(0x55cfbd3e3520 0x55cfbd3e5940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:31:22.503 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:22.499+0000 7f1ea960a740 4 client.17436 mds_command: new command op to 34535 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-21T10:31:22.503 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:22.499+0000 7f1ea960a740 1 -- 172.21.15.47:0/249780706 --> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55cfbd2d9ae0 con 0x55cfbd3e3520 2024-03-21T10:31:22.503 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:22.500+0000 7f1e9fe00640 1 --2- 172.21.15.47:0/249780706 >> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] conn(0x55cfbd3e3520 0x55cfbd3e5940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:31:22.503 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:22.501+0000 7f1e9fe00640 1 --2- 172.21.15.47:0/249780706 >> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] conn(0x55cfbd3e3520 0x55cfbd3e5940 crc :-1 s=READY pgs=154 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2afbb4d43a4b70ab server_cookie=5daadfdc83f3f1ea in_seq=0 out_seq=0 2024-03-21T10:31:22.503 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:22.501+0000 7f1e89600640 10 client.17436 ms_handle_connect on v2:172.21.15.100:6838/4173168440 2024-03-21T10:31:22.503 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:22.501+0000 7f1e89600640 1 -- 172.21.15.47:0/249780706 <== mds.0 v2:172.21.15.100:6838/4173168440 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x55cfbd2d9ae0 con 0x55cfbd3e3520 2024-03-21T10:31:22.503 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:22.501+0000 7f1e89600640 10 client.17436 handle_command_reply: tid=0 2024-03-21T10:31:22.503 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:22.501+0000 7f1ea960a740 1 client.17436 shutdown 2024-03-21T10:31:22.503 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:22.501+0000 7f1ea960a740 1 -- 172.21.15.47:0/249780706 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55cfbd377460 msgr2=0x55cfbd371c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:31:22.503 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:22.501+0000 7f1ea960a740 1 --2- 172.21.15.47:0/249780706 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55cfbd377460 0x55cfbd371c10 secure :-1 s=READY pgs=935 cs=0 l=1 rev1=1 crypto rx=0x7f1e98032b20 tx=0x7f1e980039e0 comp rx=0 tx=0).stop 2024-03-21T10:31:22.504 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:22.502+0000 7f1ea960a740 1 -- 172.21.15.47:0/249780706 shutdown_connections 2024-03-21T10:31:22.504 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:22.502+0000 7f1ea960a740 1 --2- 172.21.15.47:0/249780706 >> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] conn(0x55cfbd3e3520 0x55cfbd3e5940 crc :-1 s=READY pgs=154 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:31:22.504 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:22.502+0000 7f1ea960a740 1 --2- 172.21.15.47:0/249780706 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55cfbd377460 0x55cfbd371c10 unknown :-1 s=CLOSED pgs=935 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:31:22.504 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:22.502+0000 7f1ea960a740 1 --2- 172.21.15.47:0/249780706 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55cfbd36edd0 0x55cfbd3716d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:31:22.504 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:22.502+0000 7f1ea960a740 1 --2- 172.21.15.47:0/249780706 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55cfbd36e0f0 0x55cfbd371190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:31:22.504 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:22.502+0000 7f1ea960a740 1 -- 172.21.15.47:0/249780706 >> 172.21.15.47:0/249780706 conn(0x55cfbd335200 msgr2=0x55cfbd3727a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:31:22.504 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:22.502+0000 7f1e89600640 0 client.17436 ms_handle_reset on v2:172.21.15.100:6838/4173168440 2024-03-21T10:31:22.504 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:22.502+0000 7f1ea960a740 1 -- 172.21.15.47:0/249780706 shutdown_connections 2024-03-21T10:31:22.504 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:22.502+0000 7f1ea960a740 1 -- 172.21.15.47:0/249780706 wait complete. 2024-03-21T10:31:22.504 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:22.502+0000 7f1ea960a740 20 client.17436 trim_cache size 0 max 16384 2024-03-21T10:31:22.504 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-03-21T10:31:22.505 INFO:teuthology.orchestra.run.smithi047.stdout: "status": "no active scrubs running", 2024-03-21T10:31:22.505 INFO:teuthology.orchestra.run.smithi047.stdout: "scrubs": {} 2024-03-21T10:31:22.505 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-03-21T10:31:22.506 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:22.503+0000 7f1ea5400640 1 -- 172.21.15.47:0/1037222169 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f1e840a3140 msgr2=0x7f1e840a5600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:31:22.506 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:22.503+0000 7f1ea5400640 1 --2- 172.21.15.47:0/1037222169 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f1e840a3140 0x7f1e840a5600 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f1e98004af0 tx=0x7f1e98032450 comp rx=0 tx=0).stop 2024-03-21T10:31:22.506 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:22.503+0000 7f1ea5400640 1 -- 172.21.15.47:0/1037222169 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1ea0101520 msgr2=0x7f1ea00739a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:31:22.506 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:22.503+0000 7f1ea5400640 1 --2- 172.21.15.47:0/1037222169 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1ea0101520 0x7f1ea00739a0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f1e9000e9f0 tx=0x7f1e9000edf0 comp rx=0 tx=0).stop 2024-03-21T10:31:22.506 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:22.503+0000 7f1ea5400640 1 -- 172.21.15.47:0/1037222169 shutdown_connections 2024-03-21T10:31:22.506 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:22.503+0000 7f1ea5400640 1 --2- 172.21.15.47:0/1037222169 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1ea01075e0 0x7f1ea0074420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:31:22.506 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:22.503+0000 7f1ea5400640 1 --2- 172.21.15.47:0/1037222169 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1ea0101ee0 0x7f1ea0073ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:31:22.506 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:22.503+0000 7f1ea5400640 1 --2- 172.21.15.47:0/1037222169 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f1e840a3140 0x7f1e840a5600 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:31:22.506 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:22.503+0000 7f1ea5400640 1 --2- 172.21.15.47:0/1037222169 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1ea0101520 0x7f1ea00739a0 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:31:22.506 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:22.503+0000 7f1ea5400640 1 -- 172.21.15.47:0/1037222169 >> 172.21.15.47:0/1037222169 conn(0x7f1ea00fd010 msgr2=0x7f1ea00fe820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:31:22.506 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:22.503+0000 7f1ea5400640 1 -- 172.21.15.47:0/1037222169 shutdown_connections 2024-03-21T10:31:22.506 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:22.503+0000 7f1ea5400640 1 -- 172.21.15.47:0/1037222169 wait complete. 2024-03-21T10:31:22.520 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-03-21T10:31:22.520 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-21T10:31:22.777 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:22.778+0000 7f36bd200640 1 -- 172.21.15.47:0/278787581 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f36b8104040 msgr2=0x7f36b8110550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:31:22.778 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:22.778+0000 7f36bd200640 1 --2- 172.21.15.47:0/278787581 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f36b8104040 0x7f36b8110550 secure :-1 s=READY pgs=936 cs=0 l=1 rev1=1 crypto rx=0x7f36b0009930 tx=0x7f36b002bc20 comp rx=0 tx=0).stop 2024-03-21T10:31:22.778 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:22.778+0000 7f36bd200640 1 -- 172.21.15.47:0/278787581 shutdown_connections 2024-03-21T10:31:22.778 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:22.778+0000 7f36bd200640 1 --2- 172.21.15.47:0/278787581 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f36b8104040 0x7f36b8110550 unknown :-1 s=CLOSED pgs=936 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:31:22.778 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:22.778+0000 7f36bd200640 1 --2- 172.21.15.47:0/278787581 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f36b8103680 0x7f36b8103b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:31:22.778 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:22.778+0000 7f36bd200640 1 --2- 172.21.15.47:0/278787581 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f36b8109690 0x7f36b8109a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:31:22.778 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:22.778+0000 7f36bd200640 1 -- 172.21.15.47:0/278787581 >> 172.21.15.47:0/278787581 conn(0x7f36b80ff130 msgr2=0x7f36b8101550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:31:22.778 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:22.779+0000 7f36bd200640 1 -- 172.21.15.47:0/278787581 shutdown_connections 2024-03-21T10:31:22.778 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:22.779+0000 7f36bd200640 1 -- 172.21.15.47:0/278787581 wait complete. 2024-03-21T10:31:22.779 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:22.780+0000 7f36bd200640 1 Processor -- start 2024-03-21T10:31:22.779 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:22.780+0000 7f36bd200640 1 -- start start 2024-03-21T10:31:22.780 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:22.781+0000 7f36bd200640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f36b8103680 0x7f36b8199e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:31:22.780 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:22.781+0000 7f36bd200640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f36b8104040 0x7f36b819a3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:31:22.780 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:22.781+0000 7f36bd200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f36b8109690 0x7f36b819e740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:31:22.780 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:22.781+0000 7f36bd200640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36b81139a0 con 0x7f36b8104040 2024-03-21T10:31:22.780 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:22.781+0000 7f36bd200640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36b8113820 con 0x7f36b8109690 2024-03-21T10:31:22.780 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:22.781+0000 7f36bd200640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36b8113b20 con 0x7f36b8103680 2024-03-21T10:31:22.780 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:22.781+0000 7f36b6a00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f36b8104040 0x7f36b819a3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:31:22.780 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:22.781+0000 7f36b6a00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f36b8104040 0x7f36b819a3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:40024/0 (socket says 172.21.15.47:40024) 2024-03-21T10:31:22.780 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:22.781+0000 7f36b6a00640 1 -- 172.21.15.47:0/3074868012 learned_addr learned my addr 172.21.15.47:0/3074868012 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:31:22.780 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:22.781+0000 7f36b7e00640 1 --2- 172.21.15.47:0/3074868012 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f36b8109690 0x7f36b819e740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:31:22.781 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:22.781+0000 7f36b7400640 1 --2- 172.21.15.47:0/3074868012 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f36b8103680 0x7f36b8199e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:31:22.781 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:22.782+0000 7f36b6a00640 1 -- 172.21.15.47:0/3074868012 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f36b8103680 msgr2=0x7f36b8199e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:31:22.782 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:22.782+0000 7f36b6a00640 1 --2- 172.21.15.47:0/3074868012 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f36b8103680 0x7f36b8199e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:31:22.782 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:22.782+0000 7f36b6a00640 1 -- 172.21.15.47:0/3074868012 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f36b8109690 msgr2=0x7f36b819e740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:31:22.782 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:22.782+0000 7f36b6a00640 1 --2- 172.21.15.47:0/3074868012 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f36b8109690 0x7f36b819e740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:31:22.782 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:22.782+0000 7f36b6a00640 1 -- 172.21.15.47:0/3074868012 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f36b819ee70 con 0x7f36b8104040 2024-03-21T10:31:22.782 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:22.782+0000 7f36b6a00640 1 --2- 172.21.15.47:0/3074868012 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f36b8104040 0x7f36b819a3c0 secure :-1 s=READY pgs=937 cs=0 l=1 rev1=1 crypto rx=0x7f36a000e9f0 tx=0x7f36a000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:31:22.782 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:22.782+0000 7f36afe00640 1 -- 172.21.15.47:0/3074868012 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36a000cd80 con 0x7f36b8104040 2024-03-21T10:31:22.782 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:22.782+0000 7f36afe00640 1 -- 172.21.15.47:0/3074868012 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f36a00108e0 con 0x7f36b8104040 2024-03-21T10:31:22.782 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:22.782+0000 7f36afe00640 1 -- 172.21.15.47:0/3074868012 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36a0018920 con 0x7f36b8104040 2024-03-21T10:31:22.782 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:22.782+0000 7f36bd200640 1 -- 172.21.15.47:0/3074868012 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f36b819f160 con 0x7f36b8104040 2024-03-21T10:31:22.782 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:22.783+0000 7f36bd200640 1 -- 172.21.15.47:0/3074868012 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f36b81a69f0 con 0x7f36b8104040 2024-03-21T10:31:22.784 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:22.784+0000 7f36bd200640 1 -- 172.21.15.47:0/3074868012 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f36b8068800 con 0x7f36b8104040 2024-03-21T10:31:22.785 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:22.786+0000 7f36afe00640 1 -- 172.21.15.47:0/3074868012 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 27) v1 ==== 137985+0+0 (secure 0 0 0) 0x7f36a0010a90 con 0x7f36b8104040 2024-03-21T10:31:22.786 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:22.787+0000 7f36afe00640 1 --2- 172.21.15.47:0/3074868012 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f369c0a3110 0x7f369c0a55d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:31:22.786 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:22.788+0000 7f36afe00640 1 -- 172.21.15.47:0/3074868012 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 9474+0+0 (secure 0 0 0) 0x7f36a00d0310 con 0x7f36b8104040 2024-03-21T10:31:22.787 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:22.788+0000 7f36b7400640 1 --2- 172.21.15.47:0/3074868012 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f369c0a3110 0x7f369c0a55d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:31:22.787 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:22.789+0000 7f36b7400640 1 --2- 172.21.15.47:0/3074868012 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f369c0a3110 0x7f369c0a55d0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f36a8005fd0 tx=0x7f36a8009760 comp rx=0 tx=0).ready entity=mgr.15687 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:31:22.794 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:22.795+0000 7f36afe00640 1 -- 172.21.15.47:0/3074868012 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f36a009c390 con 0x7f36b8104040 2024-03-21T10:31:23.043 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:23.044+0000 7f36bd200640 1 -- 172.21.15.47:0/3074868012 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f36b819b1d0 con 0x7f36b8104040 2024-03-21T10:31:23.044 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:23.046+0000 7f36afe00640 1 -- 172.21.15.47:0/3074868012 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 32 v32) v1 ==== 94+0+11471 (secure 0 0 0) 0x7f36b819b1d0 con 0x7f36b8104040 2024-03-21T10:31:23.045 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-21T10:31:23.045 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":32,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14511,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6835/735724037","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6834","nonce":735724037},{"type":"v1","addr":"172.21.15.47:6835","nonce":735724037}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14529,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6837/2248191724","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6836","nonce":2248191724},{"type":"v1","addr":"172.21.15.47:6837","nonce":2248191724}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":32,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-21T10:21:58.329063+0000","modified":"2024-03-21T10:26:17.169892+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":34535,"mds_1":34493,"mds_2":34454,"mds_3":34427,"mds_4":14619},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14604":{"gid":14604,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.47:6839/2881335356","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6838","nonce":2881335356},{"type":"v1","addr":"172.21.15.47:6839","nonce":2881335356}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_14619":{"gid":14619,"name":"a","rank":4,"incarnation":27,"state":"up:active","state_seq":11,"addr":"172.21.15.47:6841/3401578978","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6840","nonce":3401578978},{"type":"v1","addr":"172.21.15.47:6841","nonce":3401578978}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24406":{"gid":24406,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.157:6833/4000010136","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":4000010136},{"type":"v1","addr":"172.21.15.157:6833","nonce":4000010136}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24457":{"gid":24457,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.157:6835/3902868651","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":3902868651},{"type":"v1","addr":"172.21.15.157:6835","nonce":3902868651}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24475":{"gid":24475,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.157:6837/1908087204","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6836","nonce":1908087204},{"type":"v1","addr":"172.21.15.157:6837","nonce":1908087204}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_25729":{"gid":25729,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.157:6839/1998408416","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6838","nonce":1998408416},{"type":"v1","addr":"172.21.15.157:6839","nonce":1998408416}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_34427":{"gid":34427,"name":"k","rank":3,"incarnation":25,"state":"up:active","state_seq":20,"addr":"172.21.15.100:6833/2723302038","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6832","nonce":2723302038},{"type":"v1","addr":"172.21.15.100:6833","nonce":2723302038}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_34454":{"gid":34454,"name":"e","rank":2,"incarnation":22,"state":"up:active","state_seq":17,"addr":"172.21.15.100:6835/2597849957","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":2597849957},{"type":"v1","addr":"172.21.15.100:6835","nonce":2597849957}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_34493":{"gid":34493,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":13,"addr":"172.21.15.100:6837/3497644162","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6836","nonce":3497644162},{"type":"v1","addr":"172.21.15.100:6837","nonce":3497644162}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_34535":{"gid":34535,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.100:6839/4173168440","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6838","nonce":4173168440},{"type":"v1","addr":"172.21.15.100:6839","nonce":4173168440}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":34535,"qdb_cluster":[34427,14619,34454,34535,34493]},"id":1}]} 2024-03-21T10:31:23.045 INFO:teuthology.orchestra.run.smithi047.stderr:dumped fsmap epoch 32 2024-03-21T10:31:23.047 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:23.049+0000 7f36bd200640 1 -- 172.21.15.47:0/3074868012 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f369c0a3110 msgr2=0x7f369c0a55d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:31:23.047 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:23.049+0000 7f36bd200640 1 --2- 172.21.15.47:0/3074868012 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f369c0a3110 0x7f369c0a55d0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f36a8005fd0 tx=0x7f36a8009760 comp rx=0 tx=0).stop 2024-03-21T10:31:23.047 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:23.049+0000 7f36bd200640 1 -- 172.21.15.47:0/3074868012 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f36b8104040 msgr2=0x7f36b819a3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:31:23.048 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:23.049+0000 7f36bd200640 1 --2- 172.21.15.47:0/3074868012 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f36b8104040 0x7f36b819a3c0 secure :-1 s=READY pgs=937 cs=0 l=1 rev1=1 crypto rx=0x7f36a000e9f0 tx=0x7f36a000eeb0 comp rx=0 tx=0).stop 2024-03-21T10:31:23.048 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:23.049+0000 7f36bd200640 1 -- 172.21.15.47:0/3074868012 shutdown_connections 2024-03-21T10:31:23.048 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:23.049+0000 7f36bd200640 1 --2- 172.21.15.47:0/3074868012 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f369c0a3110 0x7f369c0a55d0 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:31:23.048 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:23.049+0000 7f36bd200640 1 --2- 172.21.15.47:0/3074868012 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f36b8109690 0x7f36b819e740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:31:23.048 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:23.049+0000 7f36bd200640 1 --2- 172.21.15.47:0/3074868012 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f36b8104040 0x7f36b819a3c0 unknown :-1 s=CLOSED pgs=937 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:31:23.048 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:23.049+0000 7f36bd200640 1 --2- 172.21.15.47:0/3074868012 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f36b8103680 0x7f36b8199e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:31:23.049 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:23.049+0000 7f36bd200640 1 -- 172.21.15.47:0/3074868012 >> 172.21.15.47:0/3074868012 conn(0x7f36b80ff130 msgr2=0x7f36b80ff8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:31:23.049 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:23.049+0000 7f36bd200640 1 -- 172.21.15.47:0/3074868012 shutdown_connections 2024-03-21T10:31:23.049 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:23.049+0000 7f36bd200640 1 -- 172.21.15.47:0/3074868012 wait complete. 2024-03-21T10:31:23.076 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-21T10:31:23.077 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-03-21T10:31:23.308 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:23.309+0000 7f39e9800640 1 -- 172.21.15.47:0/148869636 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f39e4073a40 msgr2=0x7f39e4073e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:31:23.309 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:23.309+0000 7f39e9800640 1 --2- 172.21.15.47:0/148869636 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f39e4073a40 0x7f39e4073e20 secure :-1 s=READY pgs=938 cs=0 l=1 rev1=1 crypto rx=0x7f39d4009900 tx=0x7f39d402bcc0 comp rx=0 tx=0).stop 2024-03-21T10:31:23.309 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:23.309+0000 7f39e9800640 1 -- 172.21.15.47:0/148869636 shutdown_connections 2024-03-21T10:31:23.309 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:23.309+0000 7f39e9800640 1 --2- 172.21.15.47:0/148869636 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f39e4112510 0x7f39e4114900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:31:23.309 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:23.309+0000 7f39e9800640 1 --2- 172.21.15.47:0/148869636 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f39e4074360 0x7f39e4111fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:31:23.309 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:23.309+0000 7f39e9800640 1 --2- 172.21.15.47:0/148869636 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f39e4073a40 0x7f39e4073e20 unknown :-1 s=CLOSED pgs=938 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:31:23.309 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:23.309+0000 7f39e9800640 1 -- 172.21.15.47:0/148869636 >> 172.21.15.47:0/148869636 conn(0x7f39e40fd010 msgr2=0x7f39e40ff430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:31:23.309 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:23.309+0000 7f39e9800640 1 -- 172.21.15.47:0/148869636 shutdown_connections 2024-03-21T10:31:23.309 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:23.309+0000 7f39e9800640 1 -- 172.21.15.47:0/148869636 wait complete. 2024-03-21T10:31:23.312 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:23.310+0000 7f39e9800640 1 Processor -- start 2024-03-21T10:31:23.312 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:23.311+0000 7f39e9800640 1 -- start start 2024-03-21T10:31:23.312 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:23.311+0000 7f39e9800640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f39e4073a40 0x7f39e419e350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:31:23.312 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:23.311+0000 7f39e9800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f39e4074360 0x7f39e419e890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:31:23.312 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:23.311+0000 7f39e9800640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f39e4112510 0x7f39e41a2c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:31:23.312 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:23.311+0000 7f39e9800640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39e4117e10 con 0x7f39e4112510 2024-03-21T10:31:23.312 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:23.311+0000 7f39e9800640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39e4117c90 con 0x7f39e4074360 2024-03-21T10:31:23.312 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:23.311+0000 7f39e9800640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39e4117f90 con 0x7f39e4073a40 2024-03-21T10:31:23.312 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:23.311+0000 7f39e8e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f39e4112510 0x7f39e41a2c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:31:23.312 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:23.311+0000 7f39e8e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f39e4112510 0x7f39e41a2c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:40036/0 (socket says 172.21.15.47:40036) 2024-03-21T10:31:23.312 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:23.311+0000 7f39e8e00640 1 -- 172.21.15.47:0/1018743522 learned_addr learned my addr 172.21.15.47:0/1018743522 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:31:23.313 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:23.311+0000 7f39dbe00640 1 --2- 172.21.15.47:0/1018743522 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f39e4074360 0x7f39e419e890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:31:23.313 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:23.312+0000 7f39e3e00640 1 --2- 172.21.15.47:0/1018743522 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f39e4073a40 0x7f39e419e350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:31:23.313 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:23.312+0000 7f39e8e00640 1 -- 172.21.15.47:0/1018743522 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f39e4073a40 msgr2=0x7f39e419e350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:31:23.313 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:23.312+0000 7f39e8e00640 1 --2- 172.21.15.47:0/1018743522 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f39e4073a40 0x7f39e419e350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:31:23.313 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:23.312+0000 7f39e8e00640 1 -- 172.21.15.47:0/1018743522 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f39e4074360 msgr2=0x7f39e419e890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:31:23.313 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:23.312+0000 7f39e8e00640 1 --2- 172.21.15.47:0/1018743522 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f39e4074360 0x7f39e419e890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:31:23.313 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:23.312+0000 7f39e8e00640 1 -- 172.21.15.47:0/1018743522 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f39e41a3340 con 0x7f39e4112510 2024-03-21T10:31:23.313 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:23.312+0000 7f39e8e00640 1 --2- 172.21.15.47:0/1018743522 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f39e4112510 0x7f39e41a2c10 secure :-1 s=READY pgs=939 cs=0 l=1 rev1=1 crypto rx=0x7f39dc00e9d0 tx=0x7f39dc00ee90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:31:23.313 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:23.312+0000 7f39e1600640 1 -- 172.21.15.47:0/1018743522 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39dc00cd80 con 0x7f39e4112510 2024-03-21T10:31:23.313 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:23.313+0000 7f39e1600640 1 -- 172.21.15.47:0/1018743522 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f39dc005020 con 0x7f39e4112510 2024-03-21T10:31:23.313 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:23.313+0000 7f39e1600640 1 -- 172.21.15.47:0/1018743522 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39dc0116d0 con 0x7f39e4112510 2024-03-21T10:31:23.313 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:23.313+0000 7f39e9800640 1 -- 172.21.15.47:0/1018743522 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f39e41a3630 con 0x7f39e4112510 2024-03-21T10:31:23.313 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:23.313+0000 7f39e9800640 1 -- 172.21.15.47:0/1018743522 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f39e41aaec0 con 0x7f39e4112510 2024-03-21T10:31:23.314 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:23.316+0000 7f39e1600640 1 -- 172.21.15.47:0/1018743522 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 27) v1 ==== 137985+0+0 (secure 0 0 0) 0x7f39dc0051d0 con 0x7f39e4112510 2024-03-21T10:31:23.315 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:23.316+0000 7f39edbe6740 1 Processor -- start 2024-03-21T10:31:23.315 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:23.317+0000 7f39edbe6740 1 -- start start 2024-03-21T10:31:23.319 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:23.317+0000 7f39edbe6740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564741a858b0 0x564741a85c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:31:23.319 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:23.317+0000 7f39edbe6740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564741a86260 0x564741a8e810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:31:23.319 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:23.317+0000 7f39edbe6740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x564741a8ede0 0x564741a91230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:31:23.319 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:23.317+0000 7f39edbe6740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x5647419e92c0 con 0x564741a8ede0 2024-03-21T10:31:23.319 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:23.317+0000 7f39edbe6740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x564741991c90 con 0x564741a858b0 2024-03-21T10:31:23.319 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:23.317+0000 7f39edbe6740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x5647419e5360 con 0x564741a86260 2024-03-21T10:31:23.319 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:23.318+0000 7f39dbe00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564741a858b0 0x564741a85c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:31:23.319 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:23.318+0000 7f39dbe00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564741a858b0 0x564741a85c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:58624/0 (socket says 172.21.15.47:58624) 2024-03-21T10:31:23.319 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:23.318+0000 7f39dbe00640 1 -- 172.21.15.47:0/5469538 learned_addr learned my addr 172.21.15.47:0/5469538 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:31:23.320 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:23.318+0000 7f39dbe00640 1 --2- 172.21.15.47:0/5469538 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x564741a8ede0 0x564741a91230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:31:23.320 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:23.318+0000 7f39e3e00640 1 --2- 172.21.15.47:0/5469538 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564741a86260 0x564741a8e810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:31:23.320 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:23.318+0000 7f39e1600640 1 --2- 172.21.15.47:0/1018743522 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f39c80a3110 0x7f39c80a55d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:31:23.320 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:23.318+0000 7f39dbe00640 1 -- 172.21.15.47:0/5469538 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564741a86260 msgr2=0x564741a8e810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:31:23.320 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:23.318+0000 7f39dbe00640 1 --2- 172.21.15.47:0/5469538 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564741a86260 0x564741a8e810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:31:23.320 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:23.318+0000 7f39dbe00640 1 -- 172.21.15.47:0/5469538 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x564741a8ede0 msgr2=0x564741a91230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:31:23.320 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:23.318+0000 7f39dbe00640 1 --2- 172.21.15.47:0/5469538 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x564741a8ede0 0x564741a91230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:31:23.320 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:23.318+0000 7f39dbe00640 1 -- 172.21.15.47:0/5469538 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5647419b6080 con 0x564741a858b0 2024-03-21T10:31:23.320 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:23.318+0000 7f39e3e00640 1 --2- 172.21.15.47:0/5469538 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564741a86260 0x564741a8e810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T10:31:23.320 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:23.318+0000 7f39e1600640 1 -- 172.21.15.47:0/1018743522 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 9474+0+0 (secure 0 0 0) 0x7f39dc0cffb0 con 0x7f39e4112510 2024-03-21T10:31:23.320 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:23.318+0000 7f39e8e00640 1 --2- 172.21.15.47:0/1018743522 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f39c80a3110 0x7f39c80a55d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:31:23.320 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:23.319+0000 7f39dbe00640 1 --2- 172.21.15.47:0/5469538 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564741a858b0 0x564741a85c90 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f39cc0127c0 tx=0x7f39cc00b7c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fcce66f1325b2bc3 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:31:23.320 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:23.319+0000 7f39e8e00640 1 --2- 172.21.15.47:0/1018743522 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f39c80a3110 0x7f39c80a55d0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f39dc0a4460 tx=0x7f39dc01ef90 comp rx=0 tx=0).ready entity=mgr.15687 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:31:23.321 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:23.319+0000 7f39d9600640 1 -- 172.21.15.47:0/5469538 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39cc012e00 con 0x564741a858b0 2024-03-21T10:31:23.321 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:23.319+0000 7f39d9600640 1 -- 172.21.15.47:0/5469538 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f39cc010c20 con 0x564741a858b0 2024-03-21T10:31:23.321 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:23.319+0000 7f39d9600640 1 -- 172.21.15.47:0/5469538 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39cc01ea90 con 0x564741a858b0 2024-03-21T10:31:23.321 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:23.320+0000 7f39edbe6740 1 -- 172.21.15.47:0/5469538 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564741a858b0 msgr2=0x564741a85c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:31:23.321 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:23.320+0000 7f39edbe6740 1 --2- 172.21.15.47:0/5469538 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564741a858b0 0x564741a85c90 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f39cc0127c0 tx=0x7f39cc00b7c0 comp rx=0 tx=0).stop 2024-03-21T10:31:23.321 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:23.320+0000 7f39edbe6740 1 -- 172.21.15.47:0/5469538 shutdown_connections 2024-03-21T10:31:23.321 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:23.320+0000 7f39edbe6740 1 --2- 172.21.15.47:0/5469538 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x564741a8ede0 0x564741a91230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:31:23.321 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:23.320+0000 7f39edbe6740 1 --2- 172.21.15.47:0/5469538 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564741a86260 0x564741a8e810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:31:23.321 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:23.320+0000 7f39edbe6740 1 --2- 172.21.15.47:0/5469538 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564741a858b0 0x564741a85c90 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:31:23.321 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:23.320+0000 7f39edbe6740 1 -- 172.21.15.47:0/5469538 >> 172.21.15.47:0/5469538 conn(0x564741a7ff80 msgr2=0x564741a80360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:31:23.321 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:23.320+0000 7f39edbe6740 1 -- 172.21.15.47:0/5469538 shutdown_connections 2024-03-21T10:31:23.321 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:23.320+0000 7f39edbe6740 1 -- 172.21.15.47:0/5469538 wait complete. 2024-03-21T10:31:23.325 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:23.320+0000 7f39edbe6740 1 -- start start 2024-03-21T10:31:23.325 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:23.321+0000 7f39edbe6740 1 Processor -- start 2024-03-21T10:31:23.325 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:23.321+0000 7f39edbe6740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564741a858b0 0x564741a8e770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:31:23.325 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:23.321+0000 7f39edbe6740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564741a86260 0x564741a8bb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:31:23.325 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:23.321+0000 7f39edbe6740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x564741a8ede0 0x564741a8c110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:31:23.326 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:23.321+0000 7f39edbe6740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x564741a91d50 con 0x564741a8ede0 2024-03-21T10:31:23.326 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:23.321+0000 7f39edbe6740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x5647419a50d0 con 0x564741a858b0 2024-03-21T10:31:23.326 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:23.321+0000 7f39edbe6740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x564741a91ed0 con 0x564741a86260 2024-03-21T10:31:23.326 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:23.321+0000 7f39edbe6740 10 client.0 _ll_register_callbacks cb 0x5647419a9e80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T10:31:23.326 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:23.322+0000 7f39dbe00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x564741a8ede0 0x564741a8c110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:31:23.326 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:23.322+0000 7f39dbe00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x564741a8ede0 0x564741a8c110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:40048/0 (socket says 172.21.15.47:40048) 2024-03-21T10:31:23.326 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:23.322+0000 7f39dbe00640 1 -- 172.21.15.47:0/2673018787 learned_addr learned my addr 172.21.15.47:0/2673018787 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:31:23.326 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:23.322+0000 7f39dbe00640 1 --2- 172.21.15.47:0/2673018787 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564741a858b0 0x564741a8e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:31:23.326 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:23.322+0000 7f39e3e00640 1 --2- 172.21.15.47:0/2673018787 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564741a86260 0x564741a8bb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:31:23.327 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:23.322+0000 7f39dbe00640 1 -- 172.21.15.47:0/2673018787 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564741a86260 msgr2=0x564741a8bb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:31:23.327 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:23.322+0000 7f39dbe00640 1 --2- 172.21.15.47:0/2673018787 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564741a86260 0x564741a8bb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:31:23.327 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:23.322+0000 7f39dbe00640 1 -- 172.21.15.47:0/2673018787 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564741a858b0 msgr2=0x564741a8e770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:31:23.327 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:23.322+0000 7f39dbe00640 1 --2- 172.21.15.47:0/2673018787 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564741a858b0 0x564741a8e770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:31:23.327 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:23.322+0000 7f39dbe00640 1 -- 172.21.15.47:0/2673018787 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5647416fd3f0 con 0x564741a8ede0 2024-03-21T10:31:23.327 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:23.322+0000 7f39e3e00640 1 --2- 172.21.15.47:0/2673018787 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564741a86260 0x564741a8bb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T10:31:23.327 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:23.322+0000 7f39dbe00640 1 --2- 172.21.15.47:0/2673018787 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x564741a8ede0 0x564741a8c110 secure :-1 s=READY pgs=940 cs=0 l=1 rev1=1 crypto rx=0x7f39cc00b790 tx=0x7f39cc014040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=711c4600aa0039e9 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:31:23.327 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:23.323+0000 7f39d2000640 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-03-21T10:31:23.327 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:23.323+0000 7f39d2000640 1 -- 172.21.15.47:0/2673018787 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39cc024040 con 0x564741a8ede0 2024-03-21T10:31:23.327 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:23.323+0000 7f39d2000640 1 -- 172.21.15.47:0/2673018787 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(97..97 src has 1..97) v4 ==== 9474+0+0 (secure 0 0 0) 0x7f39cc01e810 con 0x564741a8ede0 2024-03-21T10:31:23.327 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:23.323+0000 7f39edbe6740 1 -- 172.21.15.47:0/2673018787 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f39cc010c20 con 0x564741a8ede0 2024-03-21T10:31:23.327 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:23.323+0000 7f39d2000640 1 -- 172.21.15.47:0/2673018787 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f39cc01ea10 con 0x564741a8ede0 2024-03-21T10:31:23.327 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:23.323+0000 7f39edbe6740 10 client.17454 fetch_fsmap learned FSMap version 32 2024-03-21T10:31:23.328 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:23.323+0000 7f39edbe6740 1 -- 172.21.15.47:0/2673018787 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x5647417c77d0 con 0x564741a8ede0 2024-03-21T10:31:23.328 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:23.324+0000 7f39d2000640 1 -- 172.21.15.47:0/2673018787 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f39cc010c20 con 0x564741a8ede0 2024-03-21T10:31:23.328 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:23.324+0000 7f39d2000640 1 -- 172.21.15.47:0/2673018787 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 32) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f39cc02a040 con 0x564741a8ede0 2024-03-21T10:31:23.328 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:23.324+0000 7f39edbe6740 10 client.17454 fetch_fsmap finished waiting for FSMap version 32 2024-03-21T10:31:23.328 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:23.324+0000 7f39edbe6740 10 client.17454 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.a 2024-03-21T10:31:23.328 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:23.324+0000 7f39edbe6740 20 client.17454 populate_metadata read hostname 'smithi047' 2024-03-21T10:31:23.328 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:23.324+0000 7f39edbe6740 1 --2- 172.21.15.47:0/2673018787 >> [v2:172.21.15.47:6840/3401578978,v1:172.21.15.47:6841/3401578978] conn(0x564741a9ebd0 0x564741aa0fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:31:23.328 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:23.324+0000 7f39edbe6740 4 client.17454 mds_command: new command op to 14619 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T10:31:23.328 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:23.324+0000 7f39edbe6740 1 -- 172.21.15.47:0/2673018787 --> [v2:172.21.15.47:6840/3401578978,v1:172.21.15.47:6841/3401578978] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5647419f6800 con 0x564741a9ebd0 2024-03-21T10:31:23.328 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:23.324+0000 7f39e3e00640 1 --2- 172.21.15.47:0/2673018787 >> [v2:172.21.15.47:6840/3401578978,v1:172.21.15.47:6841/3401578978] conn(0x564741a9ebd0 0x564741aa0fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:31:23.328 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:23.325+0000 7f39e3e00640 1 --2- 172.21.15.47:0/2673018787 >> [v2:172.21.15.47:6840/3401578978,v1:172.21.15.47:6841/3401578978] conn(0x564741a9ebd0 0x564741aa0fc0 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=5aaa9650803c085e server_cookie=b56148cab661b2ec in_seq=0 out_seq=0 2024-03-21T10:31:23.329 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:23.325+0000 7f39d2000640 10 client.17454 ms_handle_connect on v2:172.21.15.47:6840/3401578978 2024-03-21T10:31:23.329 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:23.326+0000 7f39d2000640 1 -- 172.21.15.47:0/2673018787 <== mds.4 v2:172.21.15.47:6840/3401578978 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x5647419f6800 con 0x564741a9ebd0 2024-03-21T10:31:23.329 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:23.326+0000 7f39d2000640 10 client.17454 handle_command_reply: tid=0 2024-03-21T10:31:23.329 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:23.326+0000 7f39edbe6740 1 client.17454 shutdown 2024-03-21T10:31:23.329 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:23.326+0000 7f39edbe6740 1 -- 172.21.15.47:0/2673018787 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x564741a8ede0 msgr2=0x564741a8c110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:31:23.329 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:23.326+0000 7f39edbe6740 1 --2- 172.21.15.47:0/2673018787 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x564741a8ede0 0x564741a8c110 secure :-1 s=READY pgs=940 cs=0 l=1 rev1=1 crypto rx=0x7f39cc00b790 tx=0x7f39cc014040 comp rx=0 tx=0).stop 2024-03-21T10:31:23.329 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:23.326+0000 7f39edbe6740 1 -- 172.21.15.47:0/2673018787 shutdown_connections 2024-03-21T10:31:23.329 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:23.326+0000 7f39edbe6740 1 --2- 172.21.15.47:0/2673018787 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x564741a8ede0 0x564741a8c110 unknown :-1 s=CLOSED pgs=940 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:31:23.329 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:23.326+0000 7f39edbe6740 1 --2- 172.21.15.47:0/2673018787 >> [v2:172.21.15.47:6840/3401578978,v1:172.21.15.47:6841/3401578978] conn(0x564741a9ebd0 0x564741aa0fc0 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:31:23.329 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:23.326+0000 7f39edbe6740 1 --2- 172.21.15.47:0/2673018787 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564741a86260 0x564741a8bb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:31:23.329 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:23.326+0000 7f39edbe6740 1 --2- 172.21.15.47:0/2673018787 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564741a858b0 0x564741a8e770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:31:23.329 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:23.326+0000 7f39d2000640 0 client.17454 ms_handle_reset on v2:172.21.15.47:6840/3401578978 2024-03-21T10:31:23.329 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:23.326+0000 7f39edbe6740 1 -- 172.21.15.47:0/2673018787 >> 172.21.15.47:0/2673018787 conn(0x564741a7ff80 msgr2=0x564741a81cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:31:23.330 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:23.326+0000 7f39edbe6740 1 -- 172.21.15.47:0/2673018787 shutdown_connections 2024-03-21T10:31:23.330 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:23.326+0000 7f39edbe6740 1 -- 172.21.15.47:0/2673018787 wait complete. 2024-03-21T10:31:23.330 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:23.327+0000 7f39edbe6740 20 client.17454 trim_cache size 0 max 16384 2024-03-21T10:31:23.339 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:23.340+0000 7f39edbe6740 1 Processor -- start 2024-03-21T10:31:23.339 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:23.340+0000 7f39edbe6740 1 -- start start 2024-03-21T10:31:23.339 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:23.341+0000 7f39edbe6740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564741a858b0 0x564741a7f3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:31:23.339 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:23.341+0000 7f39edbe6740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x564741a86260 0x564741a8e230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:31:23.341 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:23.341+0000 7f39edbe6740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564741a8ede0 0x564741a8e770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:31:23.341 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:23.341+0000 7f39edbe6740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39e41b4d90 con 0x564741a86260 2024-03-21T10:31:23.341 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:23.341+0000 7f39edbe6740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39e41b6710 con 0x564741a8ede0 2024-03-21T10:31:23.341 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:23.341+0000 7f39edbe6740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39e41b53f0 con 0x564741a858b0 2024-03-21T10:31:23.341 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:23.341+0000 7f39e3e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x564741a86260 0x564741a8e230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:31:23.341 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:23.341+0000 7f39e3e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x564741a86260 0x564741a8e230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:40056/0 (socket says 172.21.15.47:40056) 2024-03-21T10:31:23.341 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:23.341+0000 7f39e3e00640 1 -- 172.21.15.47:0/3211077847 learned_addr learned my addr 172.21.15.47:0/3211077847 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:31:23.341 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:23.341+0000 7f39dbe00640 1 --2- 172.21.15.47:0/3211077847 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564741a8ede0 0x564741a8e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:31:23.341 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:23.341+0000 7f39dbe00640 1 --2- 172.21.15.47:0/3211077847 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564741a858b0 0x564741a7f3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:31:23.341 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:23.342+0000 7f39e3e00640 1 -- 172.21.15.47:0/3211077847 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564741a858b0 msgr2=0x564741a7f3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:31:23.341 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:23.342+0000 7f39e3e00640 1 --2- 172.21.15.47:0/3211077847 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564741a858b0 0x564741a7f3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:31:23.341 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:23.342+0000 7f39e3e00640 1 -- 172.21.15.47:0/3211077847 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564741a8ede0 msgr2=0x564741a8e770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:31:23.341 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:23.342+0000 7f39e3e00640 1 --2- 172.21.15.47:0/3211077847 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564741a8ede0 0x564741a8e770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:31:23.342 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:23.342+0000 7f39e3e00640 1 -- 172.21.15.47:0/3211077847 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564741a91690 con 0x564741a86260 2024-03-21T10:31:23.342 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:23.342+0000 7f39dbe00640 1 --2- 172.21.15.47:0/3211077847 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564741a8ede0 0x564741a8e770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T10:31:23.342 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:23.342+0000 7f39e3e00640 1 --2- 172.21.15.47:0/3211077847 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x564741a86260 0x564741a8e230 secure :-1 s=READY pgs=941 cs=0 l=1 rev1=1 crypto rx=0x7f39d4005ab0 tx=0x7f39d4009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b910dd47a01e99d6 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:31:23.343 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:23.342+0000 7f39d8c00640 1 -- 172.21.15.47:0/3211077847 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39d4004440 con 0x564741a86260 2024-03-21T10:31:23.343 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:23.343+0000 7f39d8c00640 1 -- 172.21.15.47:0/3211077847 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f39d402cd20 con 0x564741a86260 2024-03-21T10:31:23.343 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:23.343+0000 7f39d8c00640 1 -- 172.21.15.47:0/3211077847 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39d4004920 con 0x564741a86260 2024-03-21T10:31:23.343 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:23.343+0000 7f39edbe6740 1 -- 172.21.15.47:0/3211077847 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x564741a86260 msgr2=0x564741a8e230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:31:23.343 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:23.343+0000 7f39edbe6740 1 --2- 172.21.15.47:0/3211077847 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x564741a86260 0x564741a8e230 secure :-1 s=READY pgs=941 cs=0 l=1 rev1=1 crypto rx=0x7f39d4005ab0 tx=0x7f39d4009260 comp rx=0 tx=0).stop 2024-03-21T10:31:23.343 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:23.343+0000 7f39edbe6740 1 -- 172.21.15.47:0/3211077847 shutdown_connections 2024-03-21T10:31:23.344 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:23.343+0000 7f39edbe6740 1 --2- 172.21.15.47:0/3211077847 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564741a8ede0 0x564741a8e770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:31:23.344 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:23.343+0000 7f39edbe6740 1 --2- 172.21.15.47:0/3211077847 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x564741a86260 0x564741a8e230 unknown :-1 s=CLOSED pgs=941 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:31:23.344 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:23.343+0000 7f39edbe6740 1 --2- 172.21.15.47:0/3211077847 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564741a858b0 0x564741a7f3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:31:23.344 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:23.343+0000 7f39edbe6740 1 -- 172.21.15.47:0/3211077847 >> 172.21.15.47:0/3211077847 conn(0x564741a7ff80 msgr2=0x564741a81b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:31:23.344 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:23.344+0000 7f39edbe6740 1 -- 172.21.15.47:0/3211077847 shutdown_connections 2024-03-21T10:31:23.344 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:23.344+0000 7f39edbe6740 1 -- 172.21.15.47:0/3211077847 wait complete. 2024-03-21T10:31:23.344 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:23.344+0000 7f39edbe6740 1 -- start start 2024-03-21T10:31:23.344 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:23.344+0000 7f39edbe6740 1 Processor -- start 2024-03-21T10:31:23.346 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:23.345+0000 7f39edbe6740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x564741a858b0 0x564741a96d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:31:23.347 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:23.345+0000 7f39edbe6740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564741a86260 0x564741a950b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:31:23.347 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:23.345+0000 7f39edbe6740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564741a8ede0 0x564741a955f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:31:23.347 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:23.345+0000 7f39edbe6740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39e41b60b0 con 0x564741a858b0 2024-03-21T10:31:23.347 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:23.345+0000 7f39edbe6740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39e41b5a50 con 0x564741a86260 2024-03-21T10:31:23.347 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:23.345+0000 7f39edbe6740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39e41b2f50 con 0x564741a8ede0 2024-03-21T10:31:23.347 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:23.345+0000 7f39edbe6740 10 client.0 _ll_register_callbacks cb 0x7f39cc00be90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T10:31:23.347 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:23.345+0000 7f39dbe00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x564741a858b0 0x564741a96d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:31:23.347 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:23.346+0000 7f39dbe00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x564741a858b0 0x564741a96d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:40068/0 (socket says 172.21.15.47:40068) 2024-03-21T10:31:23.347 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:23.346+0000 7f39dbe00640 1 -- 172.21.15.47:0/2380495815 learned_addr learned my addr 172.21.15.47:0/2380495815 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:31:23.348 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:23.346+0000 7f39e3e00640 1 --2- 172.21.15.47:0/2380495815 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564741a86260 0x564741a950b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:31:23.348 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:23.346+0000 7f39dbe00640 1 -- 172.21.15.47:0/2380495815 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564741a8ede0 msgr2=0x564741a955f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:31:23.348 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:23.346+0000 7f39dbe00640 1 --2- 172.21.15.47:0/2380495815 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564741a8ede0 0x564741a955f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:31:23.348 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:23.346+0000 7f39dbe00640 1 -- 172.21.15.47:0/2380495815 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564741a86260 msgr2=0x564741a950b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:31:23.348 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:23.346+0000 7f39dbe00640 1 --2- 172.21.15.47:0/2380495815 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564741a86260 0x564741a950b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:31:23.348 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:23.346+0000 7f39dbe00640 1 -- 172.21.15.47:0/2380495815 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x564741a95c30 con 0x564741a858b0 2024-03-21T10:31:23.348 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:23.346+0000 7f39dbe00640 1 --2- 172.21.15.47:0/2380495815 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x564741a858b0 0x564741a96d10 secure :-1 s=READY pgs=942 cs=0 l=1 rev1=1 crypto rx=0x7f39cc02a5f0 tx=0x7f39cc00bf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a40773fa3a8329b5 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:31:23.348 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:23.346+0000 7f39d2000640 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-03-21T10:31:23.348 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:23.347+0000 7f39d2000640 1 -- 172.21.15.47:0/2380495815 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39cc010c20 con 0x564741a858b0 2024-03-21T10:31:23.348 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:23.347+0000 7f39d2000640 1 -- 172.21.15.47:0/2380495815 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(97..97 src has 1..97) v4 ==== 9474+0+0 (secure 0 0 0) 0x7f39cc01ebb0 con 0x564741a858b0 2024-03-21T10:31:23.348 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:23.347+0000 7f39edbe6740 1 -- 172.21.15.47:0/2380495815 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f39d402cd20 con 0x564741a858b0 2024-03-21T10:31:23.348 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:23.347+0000 7f39d2000640 1 -- 172.21.15.47:0/2380495815 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f39cc047050 con 0x564741a858b0 2024-03-21T10:31:23.348 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:23.347+0000 7f39edbe6740 10 client.17460 fetch_fsmap learned FSMap version 32 2024-03-21T10:31:23.348 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:23.347+0000 7f39edbe6740 1 -- 172.21.15.47:0/2380495815 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x564741afb020 con 0x564741a858b0 2024-03-21T10:31:23.348 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:23.347+0000 7f39d2000640 1 -- 172.21.15.47:0/2380495815 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f39d402cd20 con 0x564741a858b0 2024-03-21T10:31:23.349 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:23.348+0000 7f39d2000640 1 -- 172.21.15.47:0/2380495815 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 32) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f39cc02a050 con 0x564741a858b0 2024-03-21T10:31:23.349 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:23.348+0000 7f39edbe6740 10 client.17460 fetch_fsmap finished waiting for FSMap version 32 2024-03-21T10:31:23.349 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:23.348+0000 7f39edbe6740 10 client.17460 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.a 2024-03-21T10:31:23.349 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:23.348+0000 7f39edbe6740 20 client.17460 populate_metadata read hostname 'smithi047' 2024-03-21T10:31:23.349 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:23.348+0000 7f39edbe6740 1 --2- 172.21.15.47:0/2380495815 >> [v2:172.21.15.47:6840/3401578978,v1:172.21.15.47:6841/3401578978] conn(0x564741afb6f0 0x564741afdb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:31:23.349 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:23.348+0000 7f39edbe6740 4 client.17460 mds_command: new command op to 14619 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-21T10:31:23.349 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:23.348+0000 7f39edbe6740 1 -- 172.21.15.47:0/2380495815 --> [v2:172.21.15.47:6840/3401578978,v1:172.21.15.47:6841/3401578978] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x5647419f0b50 con 0x564741afb6f0 2024-03-21T10:31:23.350 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:23.348+0000 7f39e3e00640 1 --2- 172.21.15.47:0/2380495815 >> [v2:172.21.15.47:6840/3401578978,v1:172.21.15.47:6841/3401578978] conn(0x564741afb6f0 0x564741afdb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:31:23.351 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:23.349+0000 7f39e3e00640 1 --2- 172.21.15.47:0/2380495815 >> [v2:172.21.15.47:6840/3401578978,v1:172.21.15.47:6841/3401578978] conn(0x564741afb6f0 0x564741afdb10 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=b121b8b0f544b90c server_cookie=d1680810c4366f34 in_seq=0 out_seq=0 2024-03-21T10:31:23.351 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:23.349+0000 7f39d2000640 10 client.17460 ms_handle_connect on v2:172.21.15.47:6840/3401578978 2024-03-21T10:31:23.351 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:23.349+0000 7f39d2000640 1 -- 172.21.15.47:0/2380495815 <== mds.4 v2:172.21.15.47:6840/3401578978 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x5647419f0b50 con 0x564741afb6f0 2024-03-21T10:31:23.351 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:23.349+0000 7f39d2000640 10 client.17460 handle_command_reply: tid=0 2024-03-21T10:31:23.351 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:23.349+0000 7f39edbe6740 1 client.17460 shutdown 2024-03-21T10:31:23.351 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:23.349+0000 7f39edbe6740 1 -- 172.21.15.47:0/2380495815 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x564741a858b0 msgr2=0x564741a96d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:31:23.351 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:23.349+0000 7f39edbe6740 1 --2- 172.21.15.47:0/2380495815 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x564741a858b0 0x564741a96d10 secure :-1 s=READY pgs=942 cs=0 l=1 rev1=1 crypto rx=0x7f39cc02a5f0 tx=0x7f39cc00bf60 comp rx=0 tx=0).stop 2024-03-21T10:31:23.351 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:23.349+0000 7f39edbe6740 1 -- 172.21.15.47:0/2380495815 shutdown_connections 2024-03-21T10:31:23.351 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:23.349+0000 7f39edbe6740 1 --2- 172.21.15.47:0/2380495815 >> [v2:172.21.15.47:6840/3401578978,v1:172.21.15.47:6841/3401578978] conn(0x564741afb6f0 0x564741afdb10 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:31:23.351 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:23.349+0000 7f39edbe6740 1 --2- 172.21.15.47:0/2380495815 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564741a8ede0 0x564741a955f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:31:23.351 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:23.349+0000 7f39d2000640 0 client.17460 ms_handle_reset on v2:172.21.15.47:6840/3401578978 2024-03-21T10:31:23.351 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:23.349+0000 7f39edbe6740 1 --2- 172.21.15.47:0/2380495815 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564741a86260 0x564741a950b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:31:23.351 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:23.350+0000 7f39edbe6740 1 --2- 172.21.15.47:0/2380495815 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x564741a858b0 0x564741a96d10 unknown :-1 s=CLOSED pgs=942 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:31:23.352 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:23.350+0000 7f39edbe6740 1 -- 172.21.15.47:0/2380495815 >> 172.21.15.47:0/2380495815 conn(0x564741a7ff80 msgr2=0x564741a86640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:31:23.352 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:23.350+0000 7f39edbe6740 1 -- 172.21.15.47:0/2380495815 shutdown_connections 2024-03-21T10:31:23.352 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:23.350+0000 7f39edbe6740 1 -- 172.21.15.47:0/2380495815 wait complete. 2024-03-21T10:31:23.352 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:23.350+0000 7f39edbe6740 20 client.17460 trim_cache size 0 max 16384 2024-03-21T10:31:23.352 INFO:teuthology.orchestra.run.smithi047.stdout:[] 2024-03-21T10:31:23.353 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:23.351+0000 7f39e9800640 1 -- 172.21.15.47:0/1018743522 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f39c80a3110 msgr2=0x7f39c80a55d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:31:23.353 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:23.351+0000 7f39e9800640 1 --2- 172.21.15.47:0/1018743522 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f39c80a3110 0x7f39c80a55d0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f39dc0a4460 tx=0x7f39dc01ef90 comp rx=0 tx=0).stop 2024-03-21T10:31:23.353 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:23.351+0000 7f39e9800640 1 -- 172.21.15.47:0/1018743522 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f39e4112510 msgr2=0x7f39e41a2c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:31:23.353 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:23.351+0000 7f39e9800640 1 --2- 172.21.15.47:0/1018743522 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f39e4112510 0x7f39e41a2c10 secure :-1 s=READY pgs=939 cs=0 l=1 rev1=1 crypto rx=0x7f39dc00e9d0 tx=0x7f39dc00ee90 comp rx=0 tx=0).stop 2024-03-21T10:31:23.353 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:23.351+0000 7f39e9800640 1 -- 172.21.15.47:0/1018743522 shutdown_connections 2024-03-21T10:31:23.353 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:23.351+0000 7f39e9800640 1 --2- 172.21.15.47:0/1018743522 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f39e4112510 0x7f39e41a2c10 unknown :-1 s=CLOSED pgs=939 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:31:23.353 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:23.351+0000 7f39e9800640 1 --2- 172.21.15.47:0/1018743522 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f39c80a3110 0x7f39c80a55d0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:31:23.353 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:23.351+0000 7f39e9800640 1 --2- 172.21.15.47:0/1018743522 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f39e4074360 0x7f39e419e890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:31:23.353 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:23.351+0000 7f39e9800640 1 --2- 172.21.15.47:0/1018743522 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f39e4073a40 0x7f39e419e350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:31:23.353 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:23.351+0000 7f39e9800640 1 -- 172.21.15.47:0/1018743522 >> 172.21.15.47:0/1018743522 conn(0x7f39e40fd010 msgr2=0x7f39e40fe810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:31:23.354 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:23.351+0000 7f39e9800640 1 -- 172.21.15.47:0/1018743522 shutdown_connections 2024-03-21T10:31:23.354 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:23.351+0000 7f39e9800640 1 -- 172.21.15.47:0/1018743522 wait complete. 2024-03-21T10:31:23.370 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-21T10:31:23.370 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-03-21T10:31:23.588 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:23.588+0000 7f89b9800640 1 -- 172.21.15.47:0/2737856199 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f89b4104200 msgr2=0x7f89b4110710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:31:23.588 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:23.588+0000 7f89b9800640 1 --2- 172.21.15.47:0/2737856199 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f89b4104200 0x7f89b4110710 secure :-1 s=READY pgs=943 cs=0 l=1 rev1=1 crypto rx=0x7f89ac009cd0 tx=0x7f89ac02c0f0 comp rx=0 tx=0).stop 2024-03-21T10:31:23.588 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:23.588+0000 7f89b9800640 1 -- 172.21.15.47:0/2737856199 shutdown_connections 2024-03-21T10:31:23.588 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:23.588+0000 7f89b9800640 1 --2- 172.21.15.47:0/2737856199 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f89b4104200 0x7f89b4110710 unknown :-1 s=CLOSED pgs=943 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:31:23.588 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:23.588+0000 7f89b9800640 1 --2- 172.21.15.47:0/2737856199 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f89b4103840 0x7f89b4103cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:31:23.588 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:23.588+0000 7f89b9800640 1 --2- 172.21.15.47:0/2737856199 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f89b4109850 0x7f89b4109c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:31:23.588 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:23.588+0000 7f89b9800640 1 -- 172.21.15.47:0/2737856199 >> 172.21.15.47:0/2737856199 conn(0x7f89b40ff350 msgr2=0x7f89b4101770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:31:23.588 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:23.588+0000 7f89b9800640 1 -- 172.21.15.47:0/2737856199 shutdown_connections 2024-03-21T10:31:23.588 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:23.589+0000 7f89b9800640 1 -- 172.21.15.47:0/2737856199 wait complete. 2024-03-21T10:31:23.591 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:23.590+0000 7f89b9800640 1 Processor -- start 2024-03-21T10:31:23.591 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:23.590+0000 7f89b9800640 1 -- start start 2024-03-21T10:31:23.591 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:23.590+0000 7f89b9800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f89b4103840 0x7f89b419a100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:31:23.591 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:23.590+0000 7f89b9800640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f89b4104200 0x7f89b419a640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:31:23.591 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:23.590+0000 7f89b9800640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f89b4109850 0x7f89b419e9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:31:23.591 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:23.590+0000 7f89b9800640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89b4113ad0 con 0x7f89b4109850 2024-03-21T10:31:23.591 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:23.590+0000 7f89b9800640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89b4113950 con 0x7f89b4103840 2024-03-21T10:31:23.591 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:23.590+0000 7f89b9800640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89b4113c50 con 0x7f89b4104200 2024-03-21T10:31:23.591 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:23.591+0000 7f89b3400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f89b4109850 0x7f89b419e9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:31:23.591 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:23.591+0000 7f89b3400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f89b4109850 0x7f89b419e9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:40084/0 (socket says 172.21.15.47:40084) 2024-03-21T10:31:23.591 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:23.591+0000 7f89b3400640 1 -- 172.21.15.47:0/171426656 learned_addr learned my addr 172.21.15.47:0/171426656 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:31:23.592 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:23.591+0000 7f89b2a00640 1 --2- 172.21.15.47:0/171426656 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f89b4103840 0x7f89b419a100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:31:23.592 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:23.591+0000 7f89b2000640 1 --2- 172.21.15.47:0/171426656 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f89b4104200 0x7f89b419a640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:31:23.592 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:23.591+0000 7f89b3400640 1 -- 172.21.15.47:0/171426656 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f89b4104200 msgr2=0x7f89b419a640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:31:23.592 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:23.591+0000 7f89b3400640 1 --2- 172.21.15.47:0/171426656 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f89b4104200 0x7f89b419a640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:31:23.592 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:23.591+0000 7f89b3400640 1 -- 172.21.15.47:0/171426656 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f89b4103840 msgr2=0x7f89b419a100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:31:23.592 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:23.591+0000 7f89b3400640 1 --2- 172.21.15.47:0/171426656 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f89b4103840 0x7f89b419a100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:31:23.592 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:23.591+0000 7f89b3400640 1 -- 172.21.15.47:0/171426656 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f89b419f0f0 con 0x7f89b4109850 2024-03-21T10:31:23.592 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:23.592+0000 7f89b3400640 1 --2- 172.21.15.47:0/171426656 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f89b4109850 0x7f89b419e9c0 secure :-1 s=READY pgs=944 cs=0 l=1 rev1=1 crypto rx=0x7f89ac005e30 tx=0x7f89ac035ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:31:23.592 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:23.592+0000 7f89ab400640 1 -- 172.21.15.47:0/171426656 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89ac039040 con 0x7f89b4109850 2024-03-21T10:31:23.592 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:23.592+0000 7f89ab400640 1 -- 172.21.15.47:0/171426656 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f89ac044070 con 0x7f89b4109850 2024-03-21T10:31:23.592 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:23.592+0000 7f89ab400640 1 -- 172.21.15.47:0/171426656 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89ac0042a0 con 0x7f89b4109850 2024-03-21T10:31:23.592 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:23.592+0000 7f89b9800640 1 -- 172.21.15.47:0/171426656 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f89b419f380 con 0x7f89b4109850 2024-03-21T10:31:23.593 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:23.592+0000 7f89b9800640 1 -- 172.21.15.47:0/171426656 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f89b41a6c70 con 0x7f89b4109850 2024-03-21T10:31:23.594 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:23.595+0000 7f89bc418740 1 Processor -- start 2024-03-21T10:31:23.594 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:23.596+0000 7f89ab400640 1 -- 172.21.15.47:0/171426656 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 27) v1 ==== 137985+0+0 (secure 0 0 0) 0x7f89ac004440 con 0x7f89b4109850 2024-03-21T10:31:23.594 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:23.596+0000 7f89bc418740 1 -- start start 2024-03-21T10:31:23.596 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:23.596+0000 7f89bc418740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5556450636d0 0x555645063ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:31:23.596 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:23.596+0000 7f89bc418740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555645064080 0x55564506c630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:31:23.596 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:23.596+0000 7f89bc418740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55564506cc00 0x55564506f050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:31:23.596 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:23.596+0000 7f89bc418740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x555644fc72c0 con 0x5556450636d0 2024-03-21T10:31:23.596 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:23.596+0000 7f89bc418740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x555644f6fc90 con 0x55564506cc00 2024-03-21T10:31:23.596 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:23.596+0000 7f89bc418740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x555644fc3360 con 0x555645064080 2024-03-21T10:31:23.596 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:23.596+0000 7f89b2000640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5556450636d0 0x555645063ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:31:23.597 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:23.597+0000 7f89b2000640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5556450636d0 0x555645063ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:40100/0 (socket says 172.21.15.47:40100) 2024-03-21T10:31:23.597 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:23.597+0000 7f89b2000640 1 -- 172.21.15.47:0/4224266592 learned_addr learned my addr 172.21.15.47:0/4224266592 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:31:23.597 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:23.597+0000 7f89b2a00640 1 --2- 172.21.15.47:0/4224266592 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555645064080 0x55564506c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:31:23.597 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:23.597+0000 7f89b2000640 1 --2- 172.21.15.47:0/4224266592 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55564506cc00 0x55564506f050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:31:23.597 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:23.597+0000 7f89b2000640 1 -- 172.21.15.47:0/4224266592 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555645064080 msgr2=0x55564506c630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:31:23.597 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:23.597+0000 7f89b2000640 1 --2- 172.21.15.47:0/4224266592 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555645064080 0x55564506c630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:31:23.597 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:23.597+0000 7f89b2000640 1 -- 172.21.15.47:0/4224266592 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55564506cc00 msgr2=0x55564506f050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:31:23.597 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:23.597+0000 7f89b2000640 1 --2- 172.21.15.47:0/4224266592 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55564506cc00 0x55564506f050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:31:23.597 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:23.597+0000 7f89b2000640 1 -- 172.21.15.47:0/4224266592 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555644f94080 con 0x5556450636d0 2024-03-21T10:31:23.597 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:23.597+0000 7f89ab400640 1 --2- 172.21.15.47:0/171426656 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f89980a3090 0x7f89980a5550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:31:23.597 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:23.597+0000 7f89b2000640 1 --2- 172.21.15.47:0/4224266592 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5556450636d0 0x555645063ab0 secure :-1 s=READY pgs=945 cs=0 l=1 rev1=1 crypto rx=0x7f89a00135a0 tx=0x7f89a000b6b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3d905e502bff3738 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:31:23.597 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:23.598+0000 7f89ab400640 1 -- 172.21.15.47:0/171426656 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 9474+0+0 (secure 0 0 0) 0x7f89ac0f20c0 con 0x7f89b4109850 2024-03-21T10:31:23.599 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:23.598+0000 7f899fe00640 1 -- 172.21.15.47:0/4224266592 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89a0019030 con 0x5556450636d0 2024-03-21T10:31:23.599 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:23.598+0000 7f89b2a00640 1 --2- 172.21.15.47:0/171426656 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f89980a3090 0x7f89980a5550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:31:23.599 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:23.598+0000 7f899fe00640 1 -- 172.21.15.47:0/4224266592 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f89a0011970 con 0x5556450636d0 2024-03-21T10:31:23.599 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:23.598+0000 7f899fe00640 1 -- 172.21.15.47:0/4224266592 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89a001d880 con 0x5556450636d0 2024-03-21T10:31:23.599 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:23.599+0000 7f89bc418740 1 -- 172.21.15.47:0/4224266592 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5556450636d0 msgr2=0x555645063ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:31:23.599 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:23.599+0000 7f89b2a00640 1 --2- 172.21.15.47:0/171426656 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f89980a3090 0x7f89980a5550 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x55564505ee10 tx=0x7f89a400b410 comp rx=0 tx=0).ready entity=mgr.15687 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:31:23.599 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:23.599+0000 7f89bc418740 1 --2- 172.21.15.47:0/4224266592 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5556450636d0 0x555645063ab0 secure :-1 s=READY pgs=945 cs=0 l=1 rev1=1 crypto rx=0x7f89a00135a0 tx=0x7f89a000b6b0 comp rx=0 tx=0).stop 2024-03-21T10:31:23.599 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:23.599+0000 7f89bc418740 1 -- 172.21.15.47:0/4224266592 shutdown_connections 2024-03-21T10:31:23.599 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:23.599+0000 7f89bc418740 1 --2- 172.21.15.47:0/4224266592 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55564506cc00 0x55564506f050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:31:23.599 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:23.599+0000 7f89bc418740 1 --2- 172.21.15.47:0/4224266592 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555645064080 0x55564506c630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:31:23.599 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:23.599+0000 7f89bc418740 1 --2- 172.21.15.47:0/4224266592 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5556450636d0 0x555645063ab0 unknown :-1 s=CLOSED pgs=945 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:31:23.599 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:23.599+0000 7f89bc418740 1 -- 172.21.15.47:0/4224266592 >> 172.21.15.47:0/4224266592 conn(0x55564505dda0 msgr2=0x55564505e180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:31:23.599 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:23.599+0000 7f89bc418740 1 -- 172.21.15.47:0/4224266592 shutdown_connections 2024-03-21T10:31:23.600 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:23.599+0000 7f89bc418740 1 -- 172.21.15.47:0/4224266592 wait complete. 2024-03-21T10:31:23.600 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:23.600+0000 7f89bc418740 1 -- start start 2024-03-21T10:31:23.600 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:23.600+0000 7f89bc418740 1 Processor -- start 2024-03-21T10:31:23.603 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:23.601+0000 7f89bc418740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5556450636d0 0x55564506c560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:31:23.603 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:23.601+0000 7f89bc418740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x555645064080 0x5556450698f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:31:23.604 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:23.601+0000 7f89bc418740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55564506cc00 0x555645069f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:31:23.604 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:23.601+0000 7f89bc418740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x55564506fb70 con 0x555645064080 2024-03-21T10:31:23.604 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:23.601+0000 7f89bc418740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x555644f830d0 con 0x55564506cc00 2024-03-21T10:31:23.604 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:23.601+0000 7f89bc418740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55564506fcf0 con 0x5556450636d0 2024-03-21T10:31:23.604 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:23.601+0000 7f89bc418740 10 client.0 _ll_register_callbacks cb 0x555644f87e80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T10:31:23.604 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:23.601+0000 7f89b2000640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x555645064080 0x5556450698f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:31:23.604 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:23.601+0000 7f89b2000640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x555645064080 0x5556450698f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:40102/0 (socket says 172.21.15.47:40102) 2024-03-21T10:31:23.604 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:23.601+0000 7f89b2000640 1 -- 172.21.15.47:0/2836504673 learned_addr learned my addr 172.21.15.47:0/2836504673 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:31:23.604 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:23.601+0000 7f89b2a00640 1 --2- 172.21.15.47:0/2836504673 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5556450636d0 0x55564506c560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:31:23.604 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:23.601+0000 7f89b3400640 1 --2- 172.21.15.47:0/2836504673 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55564506cc00 0x555645069f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:31:23.604 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:23.602+0000 7f89b2000640 1 -- 172.21.15.47:0/2836504673 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5556450636d0 msgr2=0x55564506c560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:31:23.604 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:23.602+0000 7f89b2000640 1 --2- 172.21.15.47:0/2836504673 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5556450636d0 0x55564506c560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:31:23.605 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:23.602+0000 7f89b2000640 1 -- 172.21.15.47:0/2836504673 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55564506cc00 msgr2=0x555645069f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:31:23.605 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:23.602+0000 7f89b2000640 1 --2- 172.21.15.47:0/2836504673 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55564506cc00 0x555645069f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:31:23.605 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:23.602+0000 7f89b2000640 1 -- 172.21.15.47:0/2836504673 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x555644cdb3f0 con 0x555645064080 2024-03-21T10:31:23.605 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:23.602+0000 7f89b2000640 1 --2- 172.21.15.47:0/2836504673 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x555645064080 0x5556450698f0 secure :-1 s=READY pgs=946 cs=0 l=1 rev1=1 crypto rx=0x7f89a0005f70 tx=0x7f89a000bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=367ef74ab93f97e1 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:31:23.605 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:23.602+0000 7f899cc00640 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-03-21T10:31:23.605 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:23.602+0000 7f899cc00640 1 -- 172.21.15.47:0/2836504673 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89a0019040 con 0x555645064080 2024-03-21T10:31:23.605 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:23.602+0000 7f899cc00640 1 -- 172.21.15.47:0/2836504673 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(97..97 src has 1..97) v4 ==== 9474+0+0 (secure 0 0 0) 0x7f89a0011b20 con 0x555645064080 2024-03-21T10:31:23.605 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:23.602+0000 7f89bc418740 1 -- 172.21.15.47:0/2836504673 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f89a0011970 con 0x555645064080 2024-03-21T10:31:23.605 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:23.603+0000 7f899cc00640 1 -- 172.21.15.47:0/2836504673 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f89a002f050 con 0x555645064080 2024-03-21T10:31:23.605 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:23.603+0000 7f89bc418740 10 client.17472 fetch_fsmap learned FSMap version 32 2024-03-21T10:31:23.605 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:23.603+0000 7f89bc418740 1 -- 172.21.15.47:0/2836504673 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x555644da57d0 con 0x555645064080 2024-03-21T10:31:23.605 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:23.603+0000 7f899cc00640 1 -- 172.21.15.47:0/2836504673 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f89a0011970 con 0x555645064080 2024-03-21T10:31:23.605 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:23.603+0000 7f899cc00640 1 -- 172.21.15.47:0/2836504673 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 32) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f89a0025030 con 0x555645064080 2024-03-21T10:31:23.605 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:23.604+0000 7f89bc418740 10 client.17472 fetch_fsmap finished waiting for FSMap version 32 2024-03-21T10:31:23.606 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:23.604+0000 7f89bc418740 10 client.17472 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.k 2024-03-21T10:31:23.606 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:23.604+0000 7f89bc418740 20 client.17472 populate_metadata read hostname 'smithi047' 2024-03-21T10:31:23.606 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:23.604+0000 7f89bc418740 1 --2- 172.21.15.47:0/2836504673 >> [v2:172.21.15.100:6832/2723302038,v1:172.21.15.100:6833/2723302038] conn(0x55564507ca90 0x55564507ee80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:31:23.606 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:23.604+0000 7f89bc418740 4 client.17472 mds_command: new command op to 34427 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T10:31:23.606 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:23.604+0000 7f89bc418740 1 -- 172.21.15.47:0/2836504673 --> [v2:172.21.15.100:6832/2723302038,v1:172.21.15.100:6833/2723302038] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x555644fd4800 con 0x55564507ca90 2024-03-21T10:31:23.606 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:23.604+0000 7f89b2a00640 1 --2- 172.21.15.47:0/2836504673 >> [v2:172.21.15.100:6832/2723302038,v1:172.21.15.100:6833/2723302038] conn(0x55564507ca90 0x55564507ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:31:23.607 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:23.605+0000 7f89b2a00640 1 --2- 172.21.15.47:0/2836504673 >> [v2:172.21.15.100:6832/2723302038,v1:172.21.15.100:6833/2723302038] conn(0x55564507ca90 0x55564507ee80 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=b2558ca253b49297 server_cookie=bf3b10968092a059 in_seq=0 out_seq=0 2024-03-21T10:31:23.607 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:23.605+0000 7f899cc00640 10 client.17472 ms_handle_connect on v2:172.21.15.100:6832/2723302038 2024-03-21T10:31:23.607 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:23.607+0000 7f899cc00640 1 -- 172.21.15.47:0/2836504673 <== mds.3 v2:172.21.15.100:6832/2723302038 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x555644fd4800 con 0x55564507ca90 2024-03-21T10:31:23.607 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:23.607+0000 7f899cc00640 10 client.17472 handle_command_reply: tid=0 2024-03-21T10:31:23.607 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:23.607+0000 7f89bc418740 1 client.17472 shutdown 2024-03-21T10:31:23.607 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:23.607+0000 7f89bc418740 1 -- 172.21.15.47:0/2836504673 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x555645064080 msgr2=0x5556450698f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:31:23.607 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:23.607+0000 7f89bc418740 1 --2- 172.21.15.47:0/2836504673 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x555645064080 0x5556450698f0 secure :-1 s=READY pgs=946 cs=0 l=1 rev1=1 crypto rx=0x7f89a0005f70 tx=0x7f89a000bed0 comp rx=0 tx=0).stop 2024-03-21T10:31:23.608 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:23.607+0000 7f89bc418740 1 -- 172.21.15.47:0/2836504673 shutdown_connections 2024-03-21T10:31:23.608 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:23.607+0000 7f89bc418740 1 --2- 172.21.15.47:0/2836504673 >> [v2:172.21.15.100:6832/2723302038,v1:172.21.15.100:6833/2723302038] conn(0x55564507ca90 0x55564507ee80 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:31:23.608 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:23.607+0000 7f89bc418740 1 --2- 172.21.15.47:0/2836504673 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55564506cc00 0x555645069f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:31:23.608 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:23.607+0000 7f89bc418740 1 --2- 172.21.15.47:0/2836504673 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x555645064080 0x5556450698f0 unknown :-1 s=CLOSED pgs=946 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:31:23.608 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:23.607+0000 7f89bc418740 1 --2- 172.21.15.47:0/2836504673 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5556450636d0 0x55564506c560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:31:23.608 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:23.607+0000 7f899cc00640 0 client.17472 ms_handle_reset on v2:172.21.15.100:6832/2723302038 2024-03-21T10:31:23.608 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:23.607+0000 7f89bc418740 1 -- 172.21.15.47:0/2836504673 >> 172.21.15.47:0/2836504673 conn(0x55564505dda0 msgr2=0x55564505faf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:31:23.608 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:23.607+0000 7f89bc418740 1 -- 172.21.15.47:0/2836504673 shutdown_connections 2024-03-21T10:31:23.608 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:23.607+0000 7f89bc418740 1 -- 172.21.15.47:0/2836504673 wait complete. 2024-03-21T10:31:23.608 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:23.607+0000 7f89bc418740 20 client.17472 trim_cache size 0 max 16384 2024-03-21T10:31:23.620 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:23.621+0000 7f89bc418740 1 Processor -- start 2024-03-21T10:31:23.620 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:23.621+0000 7f89bc418740 1 -- start start 2024-03-21T10:31:23.620 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:23.621+0000 7f89bc418740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5556450636d0 0x55564505f900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:31:23.620 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:23.621+0000 7f89bc418740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x555645064080 0x55564506c020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:31:23.620 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:23.621+0000 7f89bc418740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55564506cc00 0x55564506c560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:31:23.620 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:23.621+0000 7f89bc418740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89b41b0b00 con 0x555645064080 2024-03-21T10:31:23.620 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:23.621+0000 7f89bc418740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89b41b2480 con 0x5556450636d0 2024-03-21T10:31:23.620 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:23.621+0000 7f89bc418740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89b41b1160 con 0x55564506cc00 2024-03-21T10:31:23.623 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:23.622+0000 7f89b2a00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5556450636d0 0x55564505f900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:31:23.623 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:23.622+0000 7f89b2000640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x555645064080 0x55564506c020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:31:23.623 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:23.622+0000 7f89b2a00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5556450636d0 0x55564505f900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:58686/0 (socket says 172.21.15.47:58686) 2024-03-21T10:31:23.623 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:23.622+0000 7f89b2a00640 1 -- 172.21.15.47:0/3804069417 learned_addr learned my addr 172.21.15.47:0/3804069417 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:31:23.623 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:23.622+0000 7f89b3400640 1 --2- 172.21.15.47:0/3804069417 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55564506cc00 0x55564506c560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:31:23.623 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:23.622+0000 7f89b2a00640 1 -- 172.21.15.47:0/3804069417 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55564506cc00 msgr2=0x55564506c560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:31:23.623 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:23.622+0000 7f89b2a00640 1 --2- 172.21.15.47:0/3804069417 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55564506cc00 0x55564506c560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:31:23.623 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:23.622+0000 7f89b2a00640 1 -- 172.21.15.47:0/3804069417 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x555645064080 msgr2=0x55564506c020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:31:23.623 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:23.622+0000 7f89b2a00640 1 --2- 172.21.15.47:0/3804069417 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x555645064080 0x55564506c020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:31:23.623 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:23.622+0000 7f89b2a00640 1 -- 172.21.15.47:0/3804069417 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555645067e40 con 0x5556450636d0 2024-03-21T10:31:23.624 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:23.623+0000 7f89b2a00640 1 --2- 172.21.15.47:0/3804069417 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5556450636d0 0x55564505f900 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x5556450631a0 tx=0x7f89a4015000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=abd47d27e77a6302 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:31:23.624 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:23.623+0000 7f899f400640 1 -- 172.21.15.47:0/3804069417 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89a4004020 con 0x5556450636d0 2024-03-21T10:31:23.624 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:23.623+0000 7f899f400640 1 -- 172.21.15.47:0/3804069417 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f89a40041c0 con 0x5556450636d0 2024-03-21T10:31:23.624 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:23.623+0000 7f899f400640 1 -- 172.21.15.47:0/3804069417 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89a4010860 con 0x5556450636d0 2024-03-21T10:31:23.624 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:23.624+0000 7f89bc418740 1 -- 172.21.15.47:0/3804069417 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5556450636d0 msgr2=0x55564505f900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:31:23.624 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:23.624+0000 7f89bc418740 1 --2- 172.21.15.47:0/3804069417 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5556450636d0 0x55564505f900 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x5556450631a0 tx=0x7f89a4015000 comp rx=0 tx=0).stop 2024-03-21T10:31:23.624 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:23.624+0000 7f89bc418740 1 -- 172.21.15.47:0/3804069417 shutdown_connections 2024-03-21T10:31:23.624 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:23.624+0000 7f89bc418740 1 --2- 172.21.15.47:0/3804069417 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55564506cc00 0x55564506c560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:31:23.624 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:23.624+0000 7f89bc418740 1 --2- 172.21.15.47:0/3804069417 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x555645064080 0x55564506c020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:31:23.624 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:23.624+0000 7f89bc418740 1 --2- 172.21.15.47:0/3804069417 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5556450636d0 0x55564505f900 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:31:23.624 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:23.624+0000 7f89bc418740 1 -- 172.21.15.47:0/3804069417 >> 172.21.15.47:0/3804069417 conn(0x55564505dda0 msgr2=0x55564505f1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:31:23.624 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:23.624+0000 7f89bc418740 1 -- 172.21.15.47:0/3804069417 shutdown_connections 2024-03-21T10:31:23.625 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:23.624+0000 7f89bc418740 1 -- 172.21.15.47:0/3804069417 wait complete. 2024-03-21T10:31:23.625 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:23.624+0000 7f89bc418740 1 -- start start 2024-03-21T10:31:23.628 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:23.625+0000 7f89bc418740 1 Processor -- start 2024-03-21T10:31:23.628 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:23.625+0000 7f89bc418740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5556450636d0 0x555645073b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:31:23.628 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:23.625+0000 7f89bc418740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x555645064080 0x555645074060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:31:23.629 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:23.625+0000 7f89bc418740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55564506cc00 0x5556450745a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:31:23.629 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:23.625+0000 7f89bc418740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89b41b1e20 con 0x555645064080 2024-03-21T10:31:23.629 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:23.625+0000 7f89bc418740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89b41b17c0 con 0x5556450636d0 2024-03-21T10:31:23.629 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:23.625+0000 7f89bc418740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89b41aecc0 con 0x55564506cc00 2024-03-21T10:31:23.629 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:23.625+0000 7f89bc418740 10 client.0 _ll_register_callbacks cb 0x7f89a0013400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T10:31:23.629 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:23.626+0000 7f89b2000640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x555645064080 0x555645074060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:31:23.629 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:23.626+0000 7f89b2000640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x555645064080 0x555645074060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:40120/0 (socket says 172.21.15.47:40120) 2024-03-21T10:31:23.630 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:23.626+0000 7f89b2000640 1 -- 172.21.15.47:0/2470631361 learned_addr learned my addr 172.21.15.47:0/2470631361 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:31:23.630 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:23.626+0000 7f89b2a00640 1 --2- 172.21.15.47:0/2470631361 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5556450636d0 0x555645073b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:31:23.630 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:23.626+0000 7f89b2000640 1 -- 172.21.15.47:0/2470631361 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55564506cc00 msgr2=0x5556450745a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:31:23.630 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:23.626+0000 7f89b2000640 1 --2- 172.21.15.47:0/2470631361 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55564506cc00 0x5556450745a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:31:23.630 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:23.626+0000 7f89b2000640 1 -- 172.21.15.47:0/2470631361 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5556450636d0 msgr2=0x555645073b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:31:23.630 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:23.626+0000 7f89b2000640 1 --2- 172.21.15.47:0/2470631361 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5556450636d0 0x555645073b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:31:23.630 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:23.626+0000 7f89b2000640 1 -- 172.21.15.47:0/2470631361 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5556450d7100 con 0x555645064080 2024-03-21T10:31:23.630 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:23.627+0000 7f89b2000640 1 --2- 172.21.15.47:0/2470631361 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x555645064080 0x555645074060 secure :-1 s=READY pgs=947 cs=0 l=1 rev1=1 crypto rx=0x555645069150 tx=0x7f89a000bb80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fcca842c3430cfcd server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:31:23.630 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:23.627+0000 7f899cc00640 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-03-21T10:31:23.630 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:23.627+0000 7f899cc00640 1 -- 172.21.15.47:0/2470631361 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89a0019050 con 0x555645064080 2024-03-21T10:31:23.630 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:23.627+0000 7f899cc00640 1 -- 172.21.15.47:0/2470631361 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(97..97 src has 1..97) v4 ==== 9474+0+0 (secure 0 0 0) 0x7f89a002c070 con 0x555645064080 2024-03-21T10:31:23.630 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:23.627+0000 7f89bc418740 1 -- 172.21.15.47:0/2470631361 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f89a40041c0 con 0x555645064080 2024-03-21T10:31:23.631 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:23.627+0000 7f899cc00640 1 -- 172.21.15.47:0/2470631361 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f89a0040050 con 0x555645064080 2024-03-21T10:31:23.631 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:23.627+0000 7f89bc418740 10 client.17478 fetch_fsmap learned FSMap version 32 2024-03-21T10:31:23.631 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:23.627+0000 7f89bc418740 1 -- 172.21.15.47:0/2470631361 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x5556450d83d0 con 0x555645064080 2024-03-21T10:31:23.631 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:23.628+0000 7f899cc00640 1 -- 172.21.15.47:0/2470631361 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f89a40041c0 con 0x555645064080 2024-03-21T10:31:23.631 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:23.628+0000 7f899cc00640 1 -- 172.21.15.47:0/2470631361 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 32) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f89a0025040 con 0x555645064080 2024-03-21T10:31:23.631 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:23.628+0000 7f89bc418740 10 client.17478 fetch_fsmap finished waiting for FSMap version 32 2024-03-21T10:31:23.631 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:23.628+0000 7f89bc418740 10 client.17478 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.k 2024-03-21T10:31:23.631 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:23.628+0000 7f89bc418740 20 client.17478 populate_metadata read hostname 'smithi047' 2024-03-21T10:31:23.631 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:23.628+0000 7f89bc418740 1 --2- 172.21.15.47:0/2470631361 >> [v2:172.21.15.100:6832/2723302038,v1:172.21.15.100:6833/2723302038] conn(0x5556450d8a60 0x5556450dae80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:31:23.631 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:23.628+0000 7f89bc418740 4 client.17478 mds_command: new command op to 34427 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-21T10:31:23.631 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:23.628+0000 7f89bc418740 1 -- 172.21.15.47:0/2470631361 --> [v2:172.21.15.100:6832/2723302038,v1:172.21.15.100:6833/2723302038] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x555644fceb50 con 0x5556450d8a60 2024-03-21T10:31:23.631 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:23.628+0000 7f89b2a00640 1 --2- 172.21.15.47:0/2470631361 >> [v2:172.21.15.100:6832/2723302038,v1:172.21.15.100:6833/2723302038] conn(0x5556450d8a60 0x5556450dae80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:31:23.631 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:23.629+0000 7f89b2a00640 1 --2- 172.21.15.47:0/2470631361 >> [v2:172.21.15.100:6832/2723302038,v1:172.21.15.100:6833/2723302038] conn(0x5556450d8a60 0x5556450dae80 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=d540f163604f1c7f server_cookie=27291617d82f14c2 in_seq=0 out_seq=0 2024-03-21T10:31:23.631 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:23.629+0000 7f899cc00640 10 client.17478 ms_handle_connect on v2:172.21.15.100:6832/2723302038 2024-03-21T10:31:23.632 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:23.629+0000 7f899cc00640 1 -- 172.21.15.47:0/2470631361 <== mds.3 v2:172.21.15.100:6832/2723302038 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x555644fceb50 con 0x5556450d8a60 2024-03-21T10:31:23.632 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:23.629+0000 7f899cc00640 10 client.17478 handle_command_reply: tid=0 2024-03-21T10:31:23.632 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:23.629+0000 7f89bc418740 1 client.17478 shutdown 2024-03-21T10:31:23.632 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:23.629+0000 7f89bc418740 1 -- 172.21.15.47:0/2470631361 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x555645064080 msgr2=0x555645074060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:31:23.632 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:23.629+0000 7f89bc418740 1 --2- 172.21.15.47:0/2470631361 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x555645064080 0x555645074060 secure :-1 s=READY pgs=947 cs=0 l=1 rev1=1 crypto rx=0x555645069150 tx=0x7f89a000bb80 comp rx=0 tx=0).stop 2024-03-21T10:31:23.632 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:23.629+0000 7f89bc418740 1 -- 172.21.15.47:0/2470631361 shutdown_connections 2024-03-21T10:31:23.632 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:23.629+0000 7f89bc418740 1 --2- 172.21.15.47:0/2470631361 >> [v2:172.21.15.100:6832/2723302038,v1:172.21.15.100:6833/2723302038] conn(0x5556450d8a60 0x5556450dae80 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:31:23.632 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:23.629+0000 7f89bc418740 1 --2- 172.21.15.47:0/2470631361 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55564506cc00 0x5556450745a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:31:23.632 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:23.629+0000 7f89bc418740 1 --2- 172.21.15.47:0/2470631361 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x555645064080 0x555645074060 unknown :-1 s=CLOSED pgs=947 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:31:23.632 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:23.629+0000 7f89bc418740 1 --2- 172.21.15.47:0/2470631361 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5556450636d0 0x555645073b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:31:23.632 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:23.629+0000 7f899cc00640 0 client.17478 ms_handle_reset on v2:172.21.15.100:6832/2723302038 2024-03-21T10:31:23.633 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:23.629+0000 7f89bc418740 1 -- 172.21.15.47:0/2470631361 >> 172.21.15.47:0/2470631361 conn(0x55564505dda0 msgr2=0x55564505fa80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:31:23.633 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:23.630+0000 7f89bc418740 1 -- 172.21.15.47:0/2470631361 shutdown_connections 2024-03-21T10:31:23.633 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:23.630+0000 7f89bc418740 1 -- 172.21.15.47:0/2470631361 wait complete. 2024-03-21T10:31:23.633 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:23.630+0000 7f89bc418740 20 client.17478 trim_cache size 0 max 16384 2024-03-21T10:31:23.634 INFO:teuthology.orchestra.run.smithi047.stdout:[] 2024-03-21T10:31:23.634 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:23.630+0000 7f89b9800640 1 -- 172.21.15.47:0/171426656 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f89980a3090 msgr2=0x7f89980a5550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:31:23.634 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:23.630+0000 7f89b9800640 1 --2- 172.21.15.47:0/171426656 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f89980a3090 0x7f89980a5550 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x55564505ee10 tx=0x7f89a400b410 comp rx=0 tx=0).stop 2024-03-21T10:31:23.635 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:23.630+0000 7f89b9800640 1 -- 172.21.15.47:0/171426656 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f89b4109850 msgr2=0x7f89b419e9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:31:23.635 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:23.630+0000 7f89b9800640 1 --2- 172.21.15.47:0/171426656 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f89b4109850 0x7f89b419e9c0 secure :-1 s=READY pgs=944 cs=0 l=1 rev1=1 crypto rx=0x7f89ac005e30 tx=0x7f89ac035ce0 comp rx=0 tx=0).stop 2024-03-21T10:31:23.635 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:23.631+0000 7f89b9800640 1 -- 172.21.15.47:0/171426656 shutdown_connections 2024-03-21T10:31:23.635 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:23.631+0000 7f89b9800640 1 --2- 172.21.15.47:0/171426656 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f89b4109850 0x7f89b419e9c0 unknown :-1 s=CLOSED pgs=944 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:31:23.635 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:23.631+0000 7f89b9800640 1 --2- 172.21.15.47:0/171426656 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f89b4104200 0x7f89b419a640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:31:23.635 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:23.631+0000 7f89b9800640 1 --2- 172.21.15.47:0/171426656 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f89980a3090 0x7f89980a5550 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:31:23.635 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:23.631+0000 7f89b9800640 1 --2- 172.21.15.47:0/171426656 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f89b4103840 0x7f89b419a100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:31:23.635 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:23.631+0000 7f89b9800640 1 -- 172.21.15.47:0/171426656 >> 172.21.15.47:0/171426656 conn(0x7f89b40ff350 msgr2=0x7f89b40ffca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:31:23.635 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:23.631+0000 7f89b9800640 1 -- 172.21.15.47:0/171426656 shutdown_connections 2024-03-21T10:31:23.635 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:23.631+0000 7f89b9800640 1 -- 172.21.15.47:0/171426656 wait complete. 2024-03-21T10:31:23.647 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:31:23 smithi047 ceph-mon[31547]: pgmap v185: 129 pgs: 129 active+clean; 277 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 85 B/s wr, 12 op/s 2024-03-21T10:31:23.648 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:31:23 smithi047 ceph-mon[31547]: from='client.? 172.21.15.47:0/3074868012' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-21T10:31:23.650 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-21T10:31:23.650 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-03-21T10:31:23.727 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:31:23 smithi100 ceph-mon[35193]: pgmap v185: 129 pgs: 129 active+clean; 277 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 85 B/s wr, 12 op/s 2024-03-21T10:31:23.727 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:31:23 smithi100 ceph-mon[35193]: from='client.? 172.21.15.47:0/3074868012' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-21T10:31:23.737 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:31:23 smithi157 ceph-mon[35581]: pgmap v185: 129 pgs: 129 active+clean; 277 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 85 B/s wr, 12 op/s 2024-03-21T10:31:23.737 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:31:23 smithi157 ceph-mon[35581]: from='client.? 172.21.15.47:0/3074868012' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-21T10:31:23.860 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:23.861+0000 7f6514a00640 1 -- 172.21.15.47:0/3263283122 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f65101004c0 msgr2=0x7f65101008a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:31:23.860 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:23.861+0000 7f6514a00640 1 --2- 172.21.15.47:0/3263283122 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f65101004c0 0x7f65101008a0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f6500009900 tx=0x7f650002bcc0 comp rx=0 tx=0).stop 2024-03-21T10:31:23.860 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:23.861+0000 7f6514a00640 1 -- 172.21.15.47:0/3263283122 shutdown_connections 2024-03-21T10:31:23.860 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:23.861+0000 7f6514a00640 1 --2- 172.21.15.47:0/3263283122 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f65101058c0 0x7f6510105d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:31:23.861 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:23.861+0000 7f6514a00640 1 --2- 172.21.15.47:0/3263283122 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6510100de0 0x7f65101052a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:31:23.861 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:23.861+0000 7f6514a00640 1 --2- 172.21.15.47:0/3263283122 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f65101004c0 0x7f65101008a0 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:31:23.861 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:23.861+0000 7f6514a00640 1 -- 172.21.15.47:0/3263283122 >> 172.21.15.47:0/3263283122 conn(0x7f65100fbef0 msgr2=0x7f65100fe310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:31:23.861 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:23.861+0000 7f6514a00640 1 -- 172.21.15.47:0/3263283122 shutdown_connections 2024-03-21T10:31:23.861 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:23.861+0000 7f6514a00640 1 -- 172.21.15.47:0/3263283122 wait complete. 2024-03-21T10:31:23.861 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:23.862+0000 7f6514a00640 1 Processor -- start 2024-03-21T10:31:23.861 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:23.863+0000 7f6514a00640 1 -- start start 2024-03-21T10:31:23.863 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:23.863+0000 7f6514a00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f65101004c0 0x7f651019e250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:31:23.863 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:23.863+0000 7f6514a00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6510100de0 0x7f651019e790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:31:23.863 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:23.863+0000 7f6514a00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f65101058c0 0x7f65101a2b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:31:23.864 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:23.863+0000 7f6514a00640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6510117cf0 con 0x7f65101058c0 2024-03-21T10:31:23.864 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:23.863+0000 7f6514a00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6510117b70 con 0x7f65101004c0 2024-03-21T10:31:23.864 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:23.863+0000 7f6514a00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6510117e70 con 0x7f6510100de0 2024-03-21T10:31:23.864 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:23.864+0000 7f650f200640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f65101058c0 0x7f65101a2b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:31:23.864 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:23.864+0000 7f650e800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f65101004c0 0x7f651019e250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:31:23.864 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:23.864+0000 7f650e800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f65101004c0 0x7f651019e250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:58714/0 (socket says 172.21.15.47:58714) 2024-03-21T10:31:23.864 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:23.864+0000 7f650e800640 1 -- 172.21.15.47:0/3271018471 learned_addr learned my addr 172.21.15.47:0/3271018471 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:31:23.864 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:23.864+0000 7f650de00640 1 --2- 172.21.15.47:0/3271018471 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6510100de0 0x7f651019e790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:31:23.864 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:23.864+0000 7f650e800640 1 -- 172.21.15.47:0/3271018471 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6510100de0 msgr2=0x7f651019e790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:31:23.864 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:23.864+0000 7f650e800640 1 --2- 172.21.15.47:0/3271018471 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6510100de0 0x7f651019e790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:31:23.865 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:23.864+0000 7f650e800640 1 -- 172.21.15.47:0/3271018471 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f65101058c0 msgr2=0x7f65101a2b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:31:23.865 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:23.864+0000 7f650e800640 1 --2- 172.21.15.47:0/3271018471 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f65101058c0 0x7f65101a2b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:31:23.865 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:23.864+0000 7f650e800640 1 -- 172.21.15.47:0/3271018471 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65101a3240 con 0x7f65101004c0 2024-03-21T10:31:23.865 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:23.864+0000 7f650f200640 1 --2- 172.21.15.47:0/3271018471 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f65101058c0 0x7f65101a2b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T10:31:23.865 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:23.865+0000 7f650e800640 1 --2- 172.21.15.47:0/3271018471 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f65101004c0 0x7f651019e250 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f6500005ab0 tx=0x7f65000040c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:31:23.866 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:23.865+0000 7f6507400640 1 -- 172.21.15.47:0/3271018471 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6500043020 con 0x7f65101004c0 2024-03-21T10:31:23.866 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:23.865+0000 7f6507400640 1 -- 172.21.15.47:0/3271018471 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f6500003aa0 con 0x7f65101004c0 2024-03-21T10:31:23.867 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:23.865+0000 7f6514a00640 1 -- 172.21.15.47:0/3271018471 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f65101a34d0 con 0x7f65101004c0 2024-03-21T10:31:23.867 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:23.865+0000 7f6507400640 1 -- 172.21.15.47:0/3271018471 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6500005350 con 0x7f65101004c0 2024-03-21T10:31:23.867 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:23.865+0000 7f6514a00640 1 -- 172.21.15.47:0/3271018471 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f65101aadc0 con 0x7f65101004c0 2024-03-21T10:31:23.879 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:23.868+0000 7f651806e740 1 Processor -- start 2024-03-21T10:31:23.879 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:23.869+0000 7f651806e740 1 -- start start 2024-03-21T10:31:23.879 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:23.869+0000 7f6507400640 1 -- 172.21.15.47:0/3271018471 <== mon.2 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 27) v1 ==== 137985+0+0 (secure 0 0 0) 0x7f6500046020 con 0x7f65101004c0 2024-03-21T10:31:23.879 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:23.869+0000 7f651806e740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560293b6f8b0 0x560293b6fc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:31:23.880 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:23.869+0000 7f651806e740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x560293b70260 0x560293b78810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:31:23.880 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:23.869+0000 7f651806e740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560293b78de0 0x560293b7b230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:31:23.880 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:23.869+0000 7f651806e740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x560293ad32c0 con 0x560293b70260 2024-03-21T10:31:23.880 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:23.869+0000 7f651806e740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x560293a7bc90 con 0x560293b6f8b0 2024-03-21T10:31:23.880 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:23.869+0000 7f651806e740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x560293acf360 con 0x560293b78de0 2024-03-21T10:31:23.880 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:23.869+0000 7f650e800640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x560293b70260 0x560293b78810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:31:23.881 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:23.869+0000 7f650f200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560293b6f8b0 0x560293b6fc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:31:23.881 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:23.869+0000 7f650f200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560293b6f8b0 0x560293b6fc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:58728/0 (socket says 172.21.15.47:58728) 2024-03-21T10:31:23.881 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:23.869+0000 7f650f200640 1 -- 172.21.15.47:0/2465586738 learned_addr learned my addr 172.21.15.47:0/2465586738 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:31:23.881 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:23.869+0000 7f650e800640 1 --2- 172.21.15.47:0/2465586738 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x560293b70260 0x560293b78810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:40150/0 (socket says 172.21.15.47:40150) 2024-03-21T10:31:23.881 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:23.870+0000 7f650de00640 1 --2- 172.21.15.47:0/2465586738 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560293b78de0 0x560293b7b230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:31:23.881 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:23.870+0000 7f650f200640 1 -- 172.21.15.47:0/2465586738 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560293b78de0 msgr2=0x560293b7b230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:31:23.881 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:23.870+0000 7f650f200640 1 --2- 172.21.15.47:0/2465586738 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560293b78de0 0x560293b7b230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:31:23.881 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:23.870+0000 7f650f200640 1 -- 172.21.15.47:0/2465586738 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x560293b70260 msgr2=0x560293b78810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:31:23.882 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:23.870+0000 7f650f200640 1 --2- 172.21.15.47:0/2465586738 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x560293b70260 0x560293b78810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:31:23.882 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:23.870+0000 7f650f200640 1 -- 172.21.15.47:0/2465586738 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560293aa0080 con 0x560293b6f8b0 2024-03-21T10:31:23.882 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:23.870+0000 7f650de00640 1 --2- 172.21.15.47:0/2465586738 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560293b78de0 0x560293b7b230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T10:31:23.882 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:23.870+0000 7f650f200640 1 --2- 172.21.15.47:0/2465586738 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560293b6f8b0 0x560293b6fc90 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f6508010710 tx=0x7f6508010bd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=61b2a5faa894d65e server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:31:23.882 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:23.870+0000 7f6507400640 1 --2- 172.21.15.47:0/3271018471 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f64f40a3090 0x7f64f40a5550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:31:23.882 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:23.871+0000 7f64fb400640 1 -- 172.21.15.47:0/2465586738 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f650800e8d0 con 0x560293b6f8b0 2024-03-21T10:31:23.882 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:23.871+0000 7f6507400640 1 -- 172.21.15.47:0/3271018471 <== mon.2 v2:172.21.15.100:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 9474+0+0 (secure 0 0 0) 0x7f65000f1f10 con 0x7f65101004c0 2024-03-21T10:31:23.883 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:23.871+0000 7f650e800640 1 --2- 172.21.15.47:0/3271018471 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f64f40a3090 0x7f64f40a5550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:31:23.883 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:23.871+0000 7f64fb400640 1 -- 172.21.15.47:0/2465586738 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f6508012450 con 0x560293b6f8b0 2024-03-21T10:31:23.883 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:23.871+0000 7f64fb400640 1 -- 172.21.15.47:0/2465586738 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f650801c670 con 0x560293b6f8b0 2024-03-21T10:31:23.883 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:23.872+0000 7f651806e740 1 -- 172.21.15.47:0/2465586738 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560293b6f8b0 msgr2=0x560293b6fc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:31:23.883 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:23.872+0000 7f651806e740 1 --2- 172.21.15.47:0/2465586738 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560293b6f8b0 0x560293b6fc90 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f6508010710 tx=0x7f6508010bd0 comp rx=0 tx=0).stop 2024-03-21T10:31:23.883 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:23.872+0000 7f650e800640 1 --2- 172.21.15.47:0/3271018471 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f64f40a3090 0x7f64f40a5550 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x560293b6f380 tx=0x7f65000f59c0 comp rx=0 tx=0).ready entity=mgr.15687 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:31:23.883 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:23.872+0000 7f651806e740 1 -- 172.21.15.47:0/2465586738 shutdown_connections 2024-03-21T10:31:23.883 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:23.872+0000 7f651806e740 1 --2- 172.21.15.47:0/2465586738 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560293b78de0 0x560293b7b230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:31:23.883 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:23.872+0000 7f651806e740 1 --2- 172.21.15.47:0/2465586738 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x560293b70260 0x560293b78810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:31:23.883 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:23.872+0000 7f651806e740 1 --2- 172.21.15.47:0/2465586738 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560293b6f8b0 0x560293b6fc90 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:31:23.884 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:23.872+0000 7f651806e740 1 -- 172.21.15.47:0/2465586738 >> 172.21.15.47:0/2465586738 conn(0x560293b69f80 msgr2=0x560293b6a360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:31:23.884 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:23.872+0000 7f651806e740 1 -- 172.21.15.47:0/2465586738 shutdown_connections 2024-03-21T10:31:23.884 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:23.872+0000 7f651806e740 1 -- 172.21.15.47:0/2465586738 wait complete. 2024-03-21T10:31:23.884 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:23.873+0000 7f651806e740 1 -- start start 2024-03-21T10:31:23.884 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:23.873+0000 7f651806e740 1 Processor -- start 2024-03-21T10:31:23.884 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:23.873+0000 7f651806e740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x560293b6f8b0 0x560293b78740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:31:23.884 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:23.873+0000 7f651806e740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560293b70260 0x560293b75ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:31:23.884 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:23.873+0000 7f651806e740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560293b78de0 0x560293b760e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:31:23.884 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:23.873+0000 7f651806e740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x560293b7bd50 con 0x560293b6f8b0 2024-03-21T10:31:23.884 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:23.873+0000 7f651806e740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x560293a8f0d0 con 0x560293b70260 2024-03-21T10:31:23.884 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:23.873+0000 7f651806e740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x560293b7bed0 con 0x560293b78de0 2024-03-21T10:31:23.884 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:23.873+0000 7f651806e740 10 client.0 _ll_register_callbacks cb 0x560293a93e80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T10:31:23.884 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:23.874+0000 7f650f200640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x560293b6f8b0 0x560293b78740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:31:23.885 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:23.874+0000 7f650f200640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x560293b6f8b0 0x560293b78740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:40156/0 (socket says 172.21.15.47:40156) 2024-03-21T10:31:23.885 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:23.874+0000 7f650f200640 1 -- 172.21.15.47:0/4268803357 learned_addr learned my addr 172.21.15.47:0/4268803357 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:31:23.885 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:23.874+0000 7f650de00640 1 --2- 172.21.15.47:0/4268803357 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560293b70260 0x560293b75ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:31:23.885 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:23.874+0000 7f650f200640 1 -- 172.21.15.47:0/4268803357 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560293b78de0 msgr2=0x560293b760e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:31:23.885 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:23.874+0000 7f650f200640 1 --2- 172.21.15.47:0/4268803357 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560293b78de0 0x560293b760e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:31:23.885 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:23.874+0000 7f650f200640 1 -- 172.21.15.47:0/4268803357 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560293b70260 msgr2=0x560293b75ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:31:23.885 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:23.874+0000 7f650f200640 1 --2- 172.21.15.47:0/4268803357 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560293b70260 0x560293b75ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:31:23.885 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:23.874+0000 7f650f200640 1 -- 172.21.15.47:0/4268803357 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5602937e73f0 con 0x560293b6f8b0 2024-03-21T10:31:23.885 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:23.874+0000 7f650f200640 1 --2- 172.21.15.47:0/4268803357 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x560293b6f8b0 0x560293b78740 secure :-1 s=READY pgs=948 cs=0 l=1 rev1=1 crypto rx=0x7f6508006010 tx=0x7f6508005a70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f9b927906ec8338f server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:31:23.885 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:23.874+0000 7f64efe00640 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-03-21T10:31:23.886 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:23.875+0000 7f64efe00640 1 -- 172.21.15.47:0/4268803357 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f650800ece0 con 0x560293b6f8b0 2024-03-21T10:31:23.886 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:23.875+0000 7f64efe00640 1 -- 172.21.15.47:0/4268803357 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(97..97 src has 1..97) v4 ==== 9474+0+0 (secure 0 0 0) 0x7f650801c410 con 0x560293b6f8b0 2024-03-21T10:31:23.886 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:23.875+0000 7f651806e740 1 -- 172.21.15.47:0/4268803357 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f6508012450 con 0x560293b6f8b0 2024-03-21T10:31:23.886 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:23.875+0000 7f64efe00640 1 -- 172.21.15.47:0/4268803357 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6508033050 con 0x560293b6f8b0 2024-03-21T10:31:23.886 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:23.875+0000 7f651806e740 10 client.17487 fetch_fsmap learned FSMap version 32 2024-03-21T10:31:23.886 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:23.875+0000 7f651806e740 1 -- 172.21.15.47:0/4268803357 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x5602938b17d0 con 0x560293b6f8b0 2024-03-21T10:31:23.886 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:23.875+0000 7f64efe00640 1 -- 172.21.15.47:0/4268803357 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f6508012450 con 0x560293b6f8b0 2024-03-21T10:31:23.886 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:23.876+0000 7f64efe00640 1 -- 172.21.15.47:0/4268803357 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 32) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f650802e030 con 0x560293b6f8b0 2024-03-21T10:31:23.886 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:23.876+0000 7f651806e740 10 client.17487 fetch_fsmap finished waiting for FSMap version 32 2024-03-21T10:31:23.886 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:23.876+0000 7f651806e740 10 client.17487 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.e 2024-03-21T10:31:23.886 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:23.876+0000 7f651806e740 20 client.17487 populate_metadata read hostname 'smithi047' 2024-03-21T10:31:23.886 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:23.876+0000 7f651806e740 1 --2- 172.21.15.47:0/4268803357 >> [v2:172.21.15.100:6834/2597849957,v1:172.21.15.100:6835/2597849957] conn(0x560293b88b00 0x560293b8aef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:31:23.886 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:23.876+0000 7f651806e740 4 client.17487 mds_command: new command op to 34454 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T10:31:23.887 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:23.876+0000 7f651806e740 1 -- 172.21.15.47:0/4268803357 --> [v2:172.21.15.100:6834/2597849957,v1:172.21.15.100:6835/2597849957] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x560293ae0800 con 0x560293b88b00 2024-03-21T10:31:23.887 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:23.876+0000 7f650de00640 1 --2- 172.21.15.47:0/4268803357 >> [v2:172.21.15.100:6834/2597849957,v1:172.21.15.100:6835/2597849957] conn(0x560293b88b00 0x560293b8aef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:31:23.887 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:23.877+0000 7f650de00640 1 --2- 172.21.15.47:0/4268803357 >> [v2:172.21.15.100:6834/2597849957,v1:172.21.15.100:6835/2597849957] conn(0x560293b88b00 0x560293b8aef0 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=184b573f136ccfc8 server_cookie=4c442bbeee18b70a in_seq=0 out_seq=0 2024-03-21T10:31:23.887 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:23.877+0000 7f64efe00640 10 client.17487 ms_handle_connect on v2:172.21.15.100:6834/2597849957 2024-03-21T10:31:23.887 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:23.878+0000 7f64efe00640 1 -- 172.21.15.47:0/4268803357 <== mds.2 v2:172.21.15.100:6834/2597849957 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x560293ae0800 con 0x560293b88b00 2024-03-21T10:31:23.887 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:23.878+0000 7f64efe00640 10 client.17487 handle_command_reply: tid=0 2024-03-21T10:31:23.887 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:23.878+0000 7f651806e740 1 client.17487 shutdown 2024-03-21T10:31:23.887 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:23.878+0000 7f651806e740 1 -- 172.21.15.47:0/4268803357 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x560293b6f8b0 msgr2=0x560293b78740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:31:23.887 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:23.878+0000 7f651806e740 1 --2- 172.21.15.47:0/4268803357 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x560293b6f8b0 0x560293b78740 secure :-1 s=READY pgs=948 cs=0 l=1 rev1=1 crypto rx=0x7f6508006010 tx=0x7f6508005a70 comp rx=0 tx=0).stop 2024-03-21T10:31:23.887 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:23.878+0000 7f651806e740 1 -- 172.21.15.47:0/4268803357 shutdown_connections 2024-03-21T10:31:23.887 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:23.878+0000 7f651806e740 1 --2- 172.21.15.47:0/4268803357 >> [v2:172.21.15.100:6834/2597849957,v1:172.21.15.100:6835/2597849957] conn(0x560293b88b00 0x560293b8aef0 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:31:23.887 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:23.878+0000 7f651806e740 1 --2- 172.21.15.47:0/4268803357 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560293b78de0 0x560293b760e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:31:23.887 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:23.878+0000 7f651806e740 1 --2- 172.21.15.47:0/4268803357 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560293b70260 0x560293b75ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:31:23.887 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:23.878+0000 7f651806e740 1 --2- 172.21.15.47:0/4268803357 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x560293b6f8b0 0x560293b78740 unknown :-1 s=CLOSED pgs=948 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:31:23.888 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:23.878+0000 7f64efe00640 0 client.17487 ms_handle_reset on v2:172.21.15.100:6834/2597849957 2024-03-21T10:31:23.888 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:23.878+0000 7f651806e740 1 -- 172.21.15.47:0/4268803357 >> 172.21.15.47:0/4268803357 conn(0x560293b69f80 msgr2=0x560293b6bcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:31:23.888 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:23.879+0000 7f651806e740 1 -- 172.21.15.47:0/4268803357 shutdown_connections 2024-03-21T10:31:23.888 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:23.879+0000 7f651806e740 1 -- 172.21.15.47:0/4268803357 wait complete. 2024-03-21T10:31:23.888 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:23.879+0000 7f651806e740 20 client.17487 trim_cache size 0 max 16384 2024-03-21T10:31:23.905 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:23.905+0000 7f651806e740 1 Processor -- start 2024-03-21T10:31:23.906 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:23.905+0000 7f651806e740 1 -- start start 2024-03-21T10:31:23.906 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:23.905+0000 7f651806e740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x560293b6f8b0 0x560293b78200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:31:23.906 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:23.905+0000 7f651806e740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560293b70260 0x560293b78740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:31:23.906 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:23.905+0000 7f651806e740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560293b78de0 0x560293b72d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:31:23.906 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:23.905+0000 7f651806e740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65101b4c50 con 0x560293b6f8b0 2024-03-21T10:31:23.906 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:23.905+0000 7f651806e740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65101b65d0 con 0x560293b70260 2024-03-21T10:31:23.906 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:23.905+0000 7f651806e740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65101b52b0 con 0x560293b78de0 2024-03-21T10:31:23.906 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:23.906+0000 7f650f200640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x560293b6f8b0 0x560293b78200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:31:23.906 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:23.906+0000 7f650f200640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x560293b6f8b0 0x560293b78200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:40158/0 (socket says 172.21.15.47:40158) 2024-03-21T10:31:23.906 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:23.906+0000 7f650f200640 1 -- 172.21.15.47:0/1950735220 learned_addr learned my addr 172.21.15.47:0/1950735220 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:31:23.906 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:23.906+0000 7f650de00640 1 --2- 172.21.15.47:0/1950735220 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560293b70260 0x560293b78740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:31:23.906 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:23.906+0000 7f650f200640 1 -- 172.21.15.47:0/1950735220 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560293b78de0 msgr2=0x560293b72d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:31:23.907 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:23.906+0000 7f650f200640 1 --2- 172.21.15.47:0/1950735220 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560293b78de0 0x560293b72d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:31:23.907 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:23.906+0000 7f650f200640 1 -- 172.21.15.47:0/1950735220 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560293b70260 msgr2=0x560293b78740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:31:23.907 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:23.906+0000 7f650f200640 1 --2- 172.21.15.47:0/1950735220 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560293b70260 0x560293b78740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:31:23.907 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:23.906+0000 7f650f200640 1 -- 172.21.15.47:0/1950735220 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560293b74020 con 0x560293b6f8b0 2024-03-21T10:31:23.907 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:23.907+0000 7f650f200640 1 --2- 172.21.15.47:0/1950735220 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x560293b6f8b0 0x560293b78200 secure :-1 s=READY pgs=949 cs=0 l=1 rev1=1 crypto rx=0x7f650801f000 tx=0x7f6508026400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=908e8958397e1b42 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:31:23.907 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:23.907+0000 7f64faa00640 1 -- 172.21.15.47:0/1950735220 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6508012980 con 0x560293b6f8b0 2024-03-21T10:31:23.907 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:23.907+0000 7f64faa00640 1 -- 172.21.15.47:0/1950735220 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f6508009d40 con 0x560293b6f8b0 2024-03-21T10:31:23.907 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:23.907+0000 7f64faa00640 1 -- 172.21.15.47:0/1950735220 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6508004e30 con 0x560293b6f8b0 2024-03-21T10:31:23.910 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:23.907+0000 7f651806e740 1 -- 172.21.15.47:0/1950735220 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x560293b6f8b0 msgr2=0x560293b78200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:31:23.910 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:23.907+0000 7f651806e740 1 --2- 172.21.15.47:0/1950735220 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x560293b6f8b0 0x560293b78200 secure :-1 s=READY pgs=949 cs=0 l=1 rev1=1 crypto rx=0x7f650801f000 tx=0x7f6508026400 comp rx=0 tx=0).stop 2024-03-21T10:31:23.910 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:23.907+0000 7f651806e740 1 -- 172.21.15.47:0/1950735220 shutdown_connections 2024-03-21T10:31:23.910 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:23.907+0000 7f651806e740 1 --2- 172.21.15.47:0/1950735220 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560293b78de0 0x560293b72d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:31:23.910 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:23.907+0000 7f651806e740 1 --2- 172.21.15.47:0/1950735220 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560293b70260 0x560293b78740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:31:23.910 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:23.907+0000 7f651806e740 1 --2- 172.21.15.47:0/1950735220 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x560293b6f8b0 0x560293b78200 unknown :-1 s=CLOSED pgs=949 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:31:23.910 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:23.907+0000 7f651806e740 1 -- 172.21.15.47:0/1950735220 >> 172.21.15.47:0/1950735220 conn(0x560293b69f80 msgr2=0x560293b69160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:31:23.910 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:23.908+0000 7f651806e740 1 -- 172.21.15.47:0/1950735220 shutdown_connections 2024-03-21T10:31:23.910 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:23.908+0000 7f651806e740 1 -- 172.21.15.47:0/1950735220 wait complete. 2024-03-21T10:31:23.910 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:23.908+0000 7f651806e740 1 -- start start 2024-03-21T10:31:23.910 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:23.908+0000 7f651806e740 1 Processor -- start 2024-03-21T10:31:23.910 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:23.909+0000 7f651806e740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x560293b6f8b0 0x560293b80c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:31:23.911 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:23.909+0000 7f651806e740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560293b70260 0x560293b7f010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:31:23.911 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:23.909+0000 7f651806e740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560293b78de0 0x560293b7f550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:31:23.911 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:23.909+0000 7f651806e740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65101b5f70 con 0x560293b6f8b0 2024-03-21T10:31:23.911 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:23.909+0000 7f651806e740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65101b5910 con 0x560293b70260 2024-03-21T10:31:23.911 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:23.909+0000 7f651806e740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65101b2e10 con 0x560293b78de0 2024-03-21T10:31:23.911 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:23.909+0000 7f651806e740 10 client.0 _ll_register_callbacks cb 0x7f6508016070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T10:31:23.911 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:23.909+0000 7f650f200640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x560293b6f8b0 0x560293b80c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:31:23.911 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:23.909+0000 7f650f200640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x560293b6f8b0 0x560293b80c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:40172/0 (socket says 172.21.15.47:40172) 2024-03-21T10:31:23.911 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:23.909+0000 7f650f200640 1 -- 172.21.15.47:0/4062324149 learned_addr learned my addr 172.21.15.47:0/4062324149 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:31:23.911 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:23.910+0000 7f650de00640 1 --2- 172.21.15.47:0/4062324149 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560293b70260 0x560293b7f010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:31:23.911 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:23.910+0000 7f650f200640 1 -- 172.21.15.47:0/4062324149 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560293b78de0 msgr2=0x560293b7f550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:31:23.911 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:23.910+0000 7f650f200640 1 --2- 172.21.15.47:0/4062324149 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560293b78de0 0x560293b7f550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:31:23.911 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:23.910+0000 7f650f200640 1 -- 172.21.15.47:0/4062324149 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560293b70260 msgr2=0x560293b7f010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:31:23.911 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:23.910+0000 7f650f200640 1 --2- 172.21.15.47:0/4062324149 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560293b70260 0x560293b7f010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:31:23.912 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:23.910+0000 7f650f200640 1 -- 172.21.15.47:0/4062324149 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x560293b7fac0 con 0x560293b6f8b0 2024-03-21T10:31:23.912 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:23.910+0000 7f650de00640 1 --2- 172.21.15.47:0/4062324149 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560293b70260 0x560293b7f010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T10:31:23.912 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:23.910+0000 7f650f200640 1 --2- 172.21.15.47:0/4062324149 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x560293b6f8b0 0x560293b80c70 secure :-1 s=READY pgs=950 cs=0 l=1 rev1=1 crypto rx=0x7f65080264d0 tx=0x7f6508004a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ae3c6ed84a3393ff server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:31:23.912 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:23.910+0000 7f64efe00640 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-03-21T10:31:23.912 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:23.910+0000 7f64efe00640 1 -- 172.21.15.47:0/4062324149 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6508012de0 con 0x560293b6f8b0 2024-03-21T10:31:23.912 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:23.911+0000 7f651806e740 1 -- 172.21.15.47:0/4062324149 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f6508009d40 con 0x560293b6f8b0 2024-03-21T10:31:23.912 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:23.911+0000 7f64efe00640 1 -- 172.21.15.47:0/4062324149 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(97..97 src has 1..97) v4 ==== 9474+0+0 (secure 0 0 0) 0x7f6508004af0 con 0x560293b6f8b0 2024-03-21T10:31:23.912 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:23.911+0000 7f64efe00640 1 -- 172.21.15.47:0/4062324149 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f650804a050 con 0x560293b6f8b0 2024-03-21T10:31:23.912 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:23.911+0000 7f651806e740 10 client.17493 fetch_fsmap learned FSMap version 32 2024-03-21T10:31:23.912 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:23.911+0000 7f651806e740 1 -- 172.21.15.47:0/4062324149 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x560293be4f90 con 0x560293b6f8b0 2024-03-21T10:31:23.912 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:23.911+0000 7f64efe00640 1 -- 172.21.15.47:0/4062324149 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f6508009d40 con 0x560293b6f8b0 2024-03-21T10:31:23.914 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:23.911+0000 7f64efe00640 1 -- 172.21.15.47:0/4062324149 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 32) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f650802e070 con 0x560293b6f8b0 2024-03-21T10:31:23.914 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:23.911+0000 7f651806e740 10 client.17493 fetch_fsmap finished waiting for FSMap version 32 2024-03-21T10:31:23.914 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:23.911+0000 7f651806e740 10 client.17493 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.e 2024-03-21T10:31:23.914 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:23.911+0000 7f651806e740 20 client.17493 populate_metadata read hostname 'smithi047' 2024-03-21T10:31:23.914 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:23.911+0000 7f651806e740 1 --2- 172.21.15.47:0/4062324149 >> [v2:172.21.15.100:6834/2597849957,v1:172.21.15.100:6835/2597849957] conn(0x560293be55a0 0x560293be7990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:31:23.914 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:23.912+0000 7f651806e740 4 client.17493 mds_command: new command op to 34454 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-21T10:31:23.914 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:23.912+0000 7f651806e740 1 -- 172.21.15.47:0/4062324149 --> [v2:172.21.15.100:6834/2597849957,v1:172.21.15.100:6835/2597849957] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x560293adab50 con 0x560293be55a0 2024-03-21T10:31:23.914 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:23.912+0000 7f650de00640 1 --2- 172.21.15.47:0/4062324149 >> [v2:172.21.15.100:6834/2597849957,v1:172.21.15.100:6835/2597849957] conn(0x560293be55a0 0x560293be7990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:31:23.915 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:23.913+0000 7f650de00640 1 --2- 172.21.15.47:0/4062324149 >> [v2:172.21.15.100:6834/2597849957,v1:172.21.15.100:6835/2597849957] conn(0x560293be55a0 0x560293be7990 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=65f704bd60d5c677 server_cookie=d3c653d6b43eaade in_seq=0 out_seq=0 2024-03-21T10:31:23.915 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:23.913+0000 7f64efe00640 10 client.17493 ms_handle_connect on v2:172.21.15.100:6834/2597849957 2024-03-21T10:31:23.915 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:23.913+0000 7f64efe00640 1 -- 172.21.15.47:0/4062324149 <== mds.2 v2:172.21.15.100:6834/2597849957 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x560293adab50 con 0x560293be55a0 2024-03-21T10:31:23.915 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:23.913+0000 7f64efe00640 10 client.17493 handle_command_reply: tid=0 2024-03-21T10:31:23.915 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:23.913+0000 7f651806e740 1 client.17493 shutdown 2024-03-21T10:31:23.915 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:23.913+0000 7f651806e740 1 -- 172.21.15.47:0/4062324149 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x560293b6f8b0 msgr2=0x560293b80c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:31:23.915 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:23.913+0000 7f651806e740 1 --2- 172.21.15.47:0/4062324149 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x560293b6f8b0 0x560293b80c70 secure :-1 s=READY pgs=950 cs=0 l=1 rev1=1 crypto rx=0x7f65080264d0 tx=0x7f6508004a30 comp rx=0 tx=0).stop 2024-03-21T10:31:23.915 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:23.913+0000 7f651806e740 1 -- 172.21.15.47:0/4062324149 shutdown_connections 2024-03-21T10:31:23.915 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:23.913+0000 7f651806e740 1 --2- 172.21.15.47:0/4062324149 >> [v2:172.21.15.100:6834/2597849957,v1:172.21.15.100:6835/2597849957] conn(0x560293be55a0 0x560293be7990 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:31:23.915 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:23.913+0000 7f651806e740 1 --2- 172.21.15.47:0/4062324149 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560293b78de0 0x560293b7f550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:31:23.915 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:23.913+0000 7f651806e740 1 --2- 172.21.15.47:0/4062324149 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560293b70260 0x560293b7f010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:31:23.915 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:23.913+0000 7f651806e740 1 --2- 172.21.15.47:0/4062324149 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x560293b6f8b0 0x560293b80c70 unknown :-1 s=CLOSED pgs=950 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:31:23.915 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:23.913+0000 7f64efe00640 0 client.17493 ms_handle_reset on v2:172.21.15.100:6834/2597849957 2024-03-21T10:31:23.915 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:23.913+0000 7f651806e740 1 -- 172.21.15.47:0/4062324149 >> 172.21.15.47:0/4062324149 conn(0x560293b69f80 msgr2=0x560293b6b260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:31:23.916 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:23.914+0000 7f651806e740 1 -- 172.21.15.47:0/4062324149 shutdown_connections 2024-03-21T10:31:23.916 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:23.914+0000 7f651806e740 1 -- 172.21.15.47:0/4062324149 wait complete. 2024-03-21T10:31:23.916 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:23.914+0000 7f651806e740 20 client.17493 trim_cache size 0 max 16384 2024-03-21T10:31:23.916 INFO:teuthology.orchestra.run.smithi047.stdout:[] 2024-03-21T10:31:23.917 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:23.914+0000 7f6514a00640 1 -- 172.21.15.47:0/3271018471 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f64f40a3090 msgr2=0x7f64f40a5550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:31:23.917 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:23.914+0000 7f6514a00640 1 --2- 172.21.15.47:0/3271018471 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f64f40a3090 0x7f64f40a5550 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x560293b6f380 tx=0x7f65000f59c0 comp rx=0 tx=0).stop 2024-03-21T10:31:23.917 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:23.914+0000 7f6514a00640 1 -- 172.21.15.47:0/3271018471 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f65101004c0 msgr2=0x7f651019e250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:31:23.917 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:23.914+0000 7f6514a00640 1 --2- 172.21.15.47:0/3271018471 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f65101004c0 0x7f651019e250 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f6500005ab0 tx=0x7f65000040c0 comp rx=0 tx=0).stop 2024-03-21T10:31:23.917 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:23.915+0000 7f6514a00640 1 -- 172.21.15.47:0/3271018471 shutdown_connections 2024-03-21T10:31:23.917 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:23.915+0000 7f6514a00640 1 --2- 172.21.15.47:0/3271018471 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f65101058c0 0x7f65101a2b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:31:23.917 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:23.915+0000 7f6514a00640 1 --2- 172.21.15.47:0/3271018471 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6510100de0 0x7f651019e790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:31:23.917 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:23.915+0000 7f6514a00640 1 --2- 172.21.15.47:0/3271018471 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f64f40a3090 0x7f64f40a5550 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:31:23.917 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:23.915+0000 7f6514a00640 1 --2- 172.21.15.47:0/3271018471 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f65101004c0 0x7f651019e250 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:31:23.917 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:23.915+0000 7f6514a00640 1 -- 172.21.15.47:0/3271018471 >> 172.21.15.47:0/3271018471 conn(0x7f65100fbef0 msgr2=0x7f6510103270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:31:23.918 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:23.915+0000 7f6514a00640 1 -- 172.21.15.47:0/3271018471 shutdown_connections 2024-03-21T10:31:23.918 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:23.915+0000 7f6514a00640 1 -- 172.21.15.47:0/3271018471 wait complete. 2024-03-21T10:31:23.930 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-21T10:31:23.930 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-03-21T10:31:24.142 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:24.143+0000 7fba95e00640 1 -- 172.21.15.47:0/290114653 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fba90101ef0 msgr2=0x7fba9010e3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:31:24.142 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:24.143+0000 7fba95e00640 1 --2- 172.21.15.47:0/290114653 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fba90101ef0 0x7fba9010e3b0 secure :-1 s=READY pgs=951 cs=0 l=1 rev1=1 crypto rx=0x7fba88009ca0 tx=0x7fba8802c0f0 comp rx=0 tx=0).stop 2024-03-21T10:31:24.142 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:24.143+0000 7fba95e00640 1 -- 172.21.15.47:0/290114653 shutdown_connections 2024-03-21T10:31:24.142 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:24.143+0000 7fba95e00640 1 --2- 172.21.15.47:0/290114653 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fba90101ef0 0x7fba9010e3b0 unknown :-1 s=CLOSED pgs=951 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:31:24.143 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:24.143+0000 7fba95e00640 1 --2- 172.21.15.47:0/290114653 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fba90101530 0x7fba901019b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:31:24.143 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:24.143+0000 7fba95e00640 1 --2- 172.21.15.47:0/290114653 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fba901075f0 0x7fba901079d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:31:24.143 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:24.143+0000 7fba95e00640 1 -- 172.21.15.47:0/290114653 >> 172.21.15.47:0/290114653 conn(0x7fba900fd040 msgr2=0x7fba900ff460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:31:24.143 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:24.143+0000 7fba95e00640 1 -- 172.21.15.47:0/290114653 shutdown_connections 2024-03-21T10:31:24.143 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:24.143+0000 7fba95e00640 1 -- 172.21.15.47:0/290114653 wait complete. 2024-03-21T10:31:24.144 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:24.144+0000 7fba95e00640 1 Processor -- start 2024-03-21T10:31:24.144 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:24.144+0000 7fba95e00640 1 -- start start 2024-03-21T10:31:24.145 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:24.145+0000 7fba95e00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fba90101530 0x7fba90199e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:31:24.145 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:24.145+0000 7fba95e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fba90101ef0 0x7fba9019a380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:31:24.145 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:24.145+0000 7fba95e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fba901075f0 0x7fba9019e700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:31:24.145 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:24.145+0000 7fba95e00640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba90111780 con 0x7fba90101ef0 2024-03-21T10:31:24.145 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:24.145+0000 7fba95e00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba90111600 con 0x7fba901075f0 2024-03-21T10:31:24.145 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:24.145+0000 7fba95e00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba90111900 con 0x7fba90101530 2024-03-21T10:31:24.145 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:24.145+0000 7fba87e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fba90101ef0 0x7fba9019a380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:31:24.145 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:24.145+0000 7fba87e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fba90101ef0 0x7fba9019a380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:40198/0 (socket says 172.21.15.47:40198) 2024-03-21T10:31:24.145 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:24.145+0000 7fba87e00640 1 -- 172.21.15.47:0/162218021 learned_addr learned my addr 172.21.15.47:0/162218021 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:31:24.145 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:24.145+0000 7fba95400640 1 --2- 172.21.15.47:0/162218021 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fba901075f0 0x7fba9019e700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:31:24.145 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:24.145+0000 7fba94a00640 1 --2- 172.21.15.47:0/162218021 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fba90101530 0x7fba90199e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:31:24.145 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:24.145+0000 7fba87e00640 1 -- 172.21.15.47:0/162218021 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fba90101530 msgr2=0x7fba90199e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:31:24.145 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:24.145+0000 7fba87e00640 1 --2- 172.21.15.47:0/162218021 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fba90101530 0x7fba90199e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:31:24.145 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:24.146+0000 7fba87e00640 1 -- 172.21.15.47:0/162218021 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fba901075f0 msgr2=0x7fba9019e700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:31:24.146 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:24.146+0000 7fba87e00640 1 --2- 172.21.15.47:0/162218021 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fba901075f0 0x7fba9019e700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:31:24.146 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:24.146+0000 7fba87e00640 1 -- 172.21.15.47:0/162218021 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba9019ee30 con 0x7fba90101ef0 2024-03-21T10:31:24.146 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:24.146+0000 7fba87e00640 1 --2- 172.21.15.47:0/162218021 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fba90101ef0 0x7fba9019a380 secure :-1 s=READY pgs=952 cs=0 l=1 rev1=1 crypto rx=0x7fba8000d770 tx=0x7fba8000dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:31:24.146 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:24.146+0000 7fba8e000640 1 -- 172.21.15.47:0/162218021 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba80017070 con 0x7fba90101ef0 2024-03-21T10:31:24.146 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:24.146+0000 7fba8e000640 1 -- 172.21.15.47:0/162218021 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fba800054d0 con 0x7fba90101ef0 2024-03-21T10:31:24.147 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:24.146+0000 7fba8e000640 1 -- 172.21.15.47:0/162218021 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba80004cf0 con 0x7fba90101ef0 2024-03-21T10:31:24.147 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:24.146+0000 7fba95e00640 1 -- 172.21.15.47:0/162218021 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fba9019f120 con 0x7fba90101ef0 2024-03-21T10:31:24.147 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:24.147+0000 7fba95e00640 1 -- 172.21.15.47:0/162218021 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fba901a69b0 con 0x7fba90101ef0 2024-03-21T10:31:24.148 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:24.150+0000 7fba9a046740 1 Processor -- start 2024-03-21T10:31:24.149 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:24.150+0000 7fba9a046740 1 -- start start 2024-03-21T10:31:24.150 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:24.150+0000 7fba9a046740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5627692568b0 0x562769256c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:31:24.150 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:24.150+0000 7fba9a046740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562769257260 0x56276925f810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:31:24.150 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:24.151+0000 7fba9a046740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56276925fde0 0x562769262230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:31:24.150 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:24.151+0000 7fba9a046740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x5627691ba2c0 con 0x56276925fde0 2024-03-21T10:31:24.150 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:24.151+0000 7fba9a046740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x562769162c90 con 0x562769257260 2024-03-21T10:31:24.150 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:24.151+0000 7fba9a046740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x5627691b6360 con 0x5627692568b0 2024-03-21T10:31:24.150 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:24.151+0000 7fba94a00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562769257260 0x56276925f810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:31:24.150 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:24.151+0000 7fba94a00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562769257260 0x56276925f810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:58782/0 (socket says 172.21.15.47:58782) 2024-03-21T10:31:24.150 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:24.151+0000 7fba94a00640 1 -- 172.21.15.47:0/1265013145 learned_addr learned my addr 172.21.15.47:0/1265013145 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:31:24.150 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:24.151+0000 7fba95400640 1 --2- 172.21.15.47:0/1265013145 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5627692568b0 0x562769256c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:31:24.151 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:24.151+0000 7fba94a00640 1 -- 172.21.15.47:0/1265013145 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5627692568b0 msgr2=0x562769256c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:31:24.151 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:24.151+0000 7fba94a00640 1 --2- 172.21.15.47:0/1265013145 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5627692568b0 0x562769256c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:31:24.151 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:24.151+0000 7fba94a00640 1 -- 172.21.15.47:0/1265013145 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56276925fde0 msgr2=0x562769262230 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-21T10:31:24.151 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:24.151+0000 7fba94a00640 1 --2- 172.21.15.47:0/1265013145 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56276925fde0 0x562769262230 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:31:24.151 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:24.151+0000 7fba94a00640 1 -- 172.21.15.47:0/1265013145 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562769187080 con 0x562769257260 2024-03-21T10:31:24.153 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:24.152+0000 7fba94a00640 1 --2- 172.21.15.47:0/1265013145 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562769257260 0x56276925f810 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fba78009c80 tx=0x7fba780117b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c5c07e52aa4c586 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:31:24.153 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:24.152+0000 7fba8e000640 1 -- 172.21.15.47:0/162218021 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 27) v1 ==== 137985+0+0 (secure 0 0 0) 0x7fba80004e90 con 0x7fba90101ef0 2024-03-21T10:31:24.153 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:24.152+0000 7fba85c00640 1 -- 172.21.15.47:0/1265013145 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba78012450 con 0x562769257260 2024-03-21T10:31:24.153 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:24.152+0000 7fba85c00640 1 -- 172.21.15.47:0/1265013145 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fba78012aa0 con 0x562769257260 2024-03-21T10:31:24.153 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:24.152+0000 7fba85c00640 1 -- 172.21.15.47:0/1265013145 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba7800fb90 con 0x562769257260 2024-03-21T10:31:24.153 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:24.153+0000 7fba9a046740 1 -- 172.21.15.47:0/1265013145 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562769257260 msgr2=0x56276925f810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:31:24.153 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:24.153+0000 7fba9a046740 1 --2- 172.21.15.47:0/1265013145 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562769257260 0x56276925f810 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fba78009c80 tx=0x7fba780117b0 comp rx=0 tx=0).stop 2024-03-21T10:31:24.153 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:24.153+0000 7fba8e000640 1 --2- 172.21.15.47:0/162218021 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7fba700a3090 0x7fba700a5550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:31:24.153 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:24.153+0000 7fba8e000640 1 -- 172.21.15.47:0/162218021 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 9474+0+0 (secure 0 0 0) 0x7fba800cf610 con 0x7fba90101ef0 2024-03-21T10:31:24.153 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:24.153+0000 7fba9a046740 1 -- 172.21.15.47:0/1265013145 shutdown_connections 2024-03-21T10:31:24.153 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:24.153+0000 7fba9a046740 1 --2- 172.21.15.47:0/1265013145 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56276925fde0 0x562769262230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:31:24.153 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:24.153+0000 7fba9a046740 1 --2- 172.21.15.47:0/1265013145 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562769257260 0x56276925f810 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:31:24.153 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:24.153+0000 7fba9a046740 1 --2- 172.21.15.47:0/1265013145 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5627692568b0 0x562769256c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:31:24.154 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:24.153+0000 7fba9a046740 1 -- 172.21.15.47:0/1265013145 >> 172.21.15.47:0/1265013145 conn(0x562769250f80 msgr2=0x562769251360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:31:24.154 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:24.154+0000 7fba95400640 1 --2- 172.21.15.47:0/162218021 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7fba700a3090 0x7fba700a5550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:31:24.154 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:24.154+0000 7fba9a046740 1 -- 172.21.15.47:0/1265013145 shutdown_connections 2024-03-21T10:31:24.154 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:24.154+0000 7fba9a046740 1 -- 172.21.15.47:0/1265013145 wait complete. 2024-03-21T10:31:24.154 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:24.154+0000 7fba9a046740 1 -- start start 2024-03-21T10:31:24.157 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:24.154+0000 7fba9a046740 1 Processor -- start 2024-03-21T10:31:24.157 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:24.154+0000 7fba95400640 1 --2- 172.21.15.47:0/162218021 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7fba700a3090 0x7fba700a5550 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fba88004af0 tx=0x7fba8802e040 comp rx=0 tx=0).ready entity=mgr.15687 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:31:24.157 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:24.155+0000 7fba9a046740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5627692568b0 0x56276925f6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:31:24.157 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:24.155+0000 7fba9a046740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562769257260 0x56276925ca30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:31:24.157 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:24.155+0000 7fba9a046740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56276925fde0 0x56276925cf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:31:24.157 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:24.155+0000 7fba9a046740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x562769262d50 con 0x56276925fde0 2024-03-21T10:31:24.157 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:24.155+0000 7fba9a046740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x5627691760d0 con 0x5627692568b0 2024-03-21T10:31:24.157 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:24.155+0000 7fba9a046740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x562769262ed0 con 0x562769257260 2024-03-21T10:31:24.157 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:24.155+0000 7fba9a046740 10 client.0 _ll_register_callbacks cb 0x56276917ae80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T10:31:24.157 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:24.155+0000 7fba95400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56276925fde0 0x56276925cf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:31:24.157 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:24.155+0000 7fba94a00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5627692568b0 0x56276925f6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:31:24.157 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:24.155+0000 7fba95400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56276925fde0 0x56276925cf90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:40202/0 (socket says 172.21.15.47:40202) 2024-03-21T10:31:24.158 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:24.155+0000 7fba95400640 1 -- 172.21.15.47:0/3071337087 learned_addr learned my addr 172.21.15.47:0/3071337087 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:31:24.158 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:24.156+0000 7fba87e00640 1 --2- 172.21.15.47:0/3071337087 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562769257260 0x56276925ca30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:31:24.158 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:24.156+0000 7fba95400640 1 -- 172.21.15.47:0/3071337087 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562769257260 msgr2=0x56276925ca30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:31:24.158 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:24.156+0000 7fba95400640 1 --2- 172.21.15.47:0/3071337087 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562769257260 0x56276925ca30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:31:24.158 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:24.156+0000 7fba95400640 1 -- 172.21.15.47:0/3071337087 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5627692568b0 msgr2=0x56276925f6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:31:24.158 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:24.156+0000 7fba95400640 1 --2- 172.21.15.47:0/3071337087 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5627692568b0 0x56276925f6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:31:24.158 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:24.156+0000 7fba95400640 1 -- 172.21.15.47:0/3071337087 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x562768ece3f0 con 0x56276925fde0 2024-03-21T10:31:24.158 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:24.156+0000 7fba87e00640 1 --2- 172.21.15.47:0/3071337087 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562769257260 0x56276925ca30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T10:31:24.158 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:24.156+0000 7fba95400640 1 --2- 172.21.15.47:0/3071337087 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56276925fde0 0x56276925cf90 secure :-1 s=READY pgs=953 cs=0 l=1 rev1=1 crypto rx=0x7fba88009ca0 tx=0x7fba8802c0f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3959ee6f19027736 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:31:24.158 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:24.156+0000 7fba7e000640 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-03-21T10:31:24.158 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:24.157+0000 7fba7e000640 1 -- 172.21.15.47:0/3071337087 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba88046070 con 0x56276925fde0 2024-03-21T10:31:24.158 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:24.157+0000 7fba7e000640 1 -- 172.21.15.47:0/3071337087 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(97..97 src has 1..97) v4 ==== 9474+0+0 (secure 0 0 0) 0x7fba8804d670 con 0x56276925fde0 2024-03-21T10:31:24.158 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:24.157+0000 7fba9a046740 1 -- 172.21.15.47:0/3071337087 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fba78012aa0 con 0x56276925fde0 2024-03-21T10:31:24.158 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:24.157+0000 7fba7e000640 1 -- 172.21.15.47:0/3071337087 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7fba8805b050 con 0x56276925fde0 2024-03-21T10:31:24.158 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:24.157+0000 7fba9a046740 10 client.17502 fetch_fsmap learned FSMap version 32 2024-03-21T10:31:24.159 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:24.157+0000 7fba9a046740 1 -- 172.21.15.47:0/3071337087 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x562768f987d0 con 0x56276925fde0 2024-03-21T10:31:24.159 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:24.157+0000 7fba7e000640 1 -- 172.21.15.47:0/3071337087 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fba78012aa0 con 0x56276925fde0 2024-03-21T10:31:24.159 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:24.158+0000 7fba7e000640 1 -- 172.21.15.47:0/3071337087 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 32) v1 ==== 7942+0+0 (secure 0 0 0) 0x7fba88040940 con 0x56276925fde0 2024-03-21T10:31:24.159 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:24.158+0000 7fba9a046740 10 client.17502 fetch_fsmap finished waiting for FSMap version 32 2024-03-21T10:31:24.159 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:24.158+0000 7fba9a046740 10 client.17502 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-03-21T10:31:24.159 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:24.158+0000 7fba9a046740 20 client.17502 populate_metadata read hostname 'smithi047' 2024-03-21T10:31:24.159 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:24.158+0000 7fba9a046740 1 --2- 172.21.15.47:0/3071337087 >> [v2:172.21.15.100:6836/3497644162,v1:172.21.15.100:6837/3497644162] conn(0x56276926fc50 0x562769272080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:31:24.159 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:24.158+0000 7fba9a046740 4 client.17502 mds_command: new command op to 34493 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T10:31:24.159 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:24.158+0000 7fba9a046740 1 -- 172.21.15.47:0/3071337087 --> [v2:172.21.15.100:6836/3497644162,v1:172.21.15.100:6837/3497644162] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5627691c7800 con 0x56276926fc50 2024-03-21T10:31:24.161 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:24.158+0000 7fba94a00640 1 --2- 172.21.15.47:0/3071337087 >> [v2:172.21.15.100:6836/3497644162,v1:172.21.15.100:6837/3497644162] conn(0x56276926fc50 0x562769272080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:31:24.161 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:24.159+0000 7fba94a00640 1 --2- 172.21.15.47:0/3071337087 >> [v2:172.21.15.100:6836/3497644162,v1:172.21.15.100:6837/3497644162] conn(0x56276926fc50 0x562769272080 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=52b56573bb5b0af8 server_cookie=6483378633594e14 in_seq=0 out_seq=0 2024-03-21T10:31:24.161 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:24.159+0000 7fba7e000640 10 client.17502 ms_handle_connect on v2:172.21.15.100:6836/3497644162 2024-03-21T10:31:24.161 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:24.160+0000 7fba7e000640 1 -- 172.21.15.47:0/3071337087 <== mds.1 v2:172.21.15.100:6836/3497644162 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x5627691c7800 con 0x56276926fc50 2024-03-21T10:31:24.161 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:24.160+0000 7fba7e000640 10 client.17502 handle_command_reply: tid=0 2024-03-21T10:31:24.162 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:24.161+0000 7fba9a046740 1 client.17502 shutdown 2024-03-21T10:31:24.162 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:24.161+0000 7fba9a046740 1 -- 172.21.15.47:0/3071337087 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56276925fde0 msgr2=0x56276925cf90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:31:24.162 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:24.161+0000 7fba9a046740 1 --2- 172.21.15.47:0/3071337087 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56276925fde0 0x56276925cf90 secure :-1 s=READY pgs=953 cs=0 l=1 rev1=1 crypto rx=0x7fba88009ca0 tx=0x7fba8802c0f0 comp rx=0 tx=0).stop 2024-03-21T10:31:24.162 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:24.161+0000 7fba9a046740 1 -- 172.21.15.47:0/3071337087 shutdown_connections 2024-03-21T10:31:24.162 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:24.161+0000 7fba9a046740 1 --2- 172.21.15.47:0/3071337087 >> [v2:172.21.15.100:6836/3497644162,v1:172.21.15.100:6837/3497644162] conn(0x56276926fc50 0x562769272080 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:31:24.162 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:24.161+0000 7fba9a046740 1 --2- 172.21.15.47:0/3071337087 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56276925fde0 0x56276925cf90 unknown :-1 s=CLOSED pgs=953 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:31:24.162 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:24.161+0000 7fba9a046740 1 --2- 172.21.15.47:0/3071337087 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562769257260 0x56276925ca30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:31:24.162 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:24.161+0000 7fba9a046740 1 --2- 172.21.15.47:0/3071337087 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5627692568b0 0x56276925f6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:31:24.162 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:24.161+0000 7fba9a046740 1 -- 172.21.15.47:0/3071337087 >> 172.21.15.47:0/3071337087 conn(0x562769250f80 msgr2=0x562769252cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:31:24.162 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:24.161+0000 7fba7e000640 0 client.17502 ms_handle_reset on v2:172.21.15.100:6836/3497644162 2024-03-21T10:31:24.162 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:24.161+0000 7fba9a046740 1 -- 172.21.15.47:0/3071337087 shutdown_connections 2024-03-21T10:31:24.162 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:24.161+0000 7fba9a046740 1 -- 172.21.15.47:0/3071337087 wait complete. 2024-03-21T10:31:24.163 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:24.161+0000 7fba9a046740 20 client.17502 trim_cache size 0 max 16384 2024-03-21T10:31:24.175 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:24.175+0000 7fba9a046740 1 Processor -- start 2024-03-21T10:31:24.175 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:24.175+0000 7fba9a046740 1 -- start start 2024-03-21T10:31:24.175 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:24.175+0000 7fba9a046740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5627692568b0 0x56276925bbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:31:24.175 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:24.175+0000 7fba9a046740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562769257260 0x562769252ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:31:24.175 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:24.175+0000 7fba9a046740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56276925fde0 0x56276925f160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:31:24.175 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:24.175+0000 7fba9a046740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba901b0840 con 0x5627692568b0 2024-03-21T10:31:24.176 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:24.175+0000 7fba9a046740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba901b21c0 con 0x56276925fde0 2024-03-21T10:31:24.176 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:24.175+0000 7fba9a046740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba901b0ea0 con 0x562769257260 2024-03-21T10:31:24.176 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:24.176+0000 7fba94a00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5627692568b0 0x56276925bbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:31:24.176 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:24.176+0000 7fba94a00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5627692568b0 0x56276925bbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:40206/0 (socket says 172.21.15.47:40206) 2024-03-21T10:31:24.176 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:24.176+0000 7fba94a00640 1 -- 172.21.15.47:0/1842777131 learned_addr learned my addr 172.21.15.47:0/1842777131 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:31:24.177 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:24.176+0000 7fba95400640 1 --2- 172.21.15.47:0/1842777131 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56276925fde0 0x56276925f160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:31:24.177 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:24.176+0000 7fba87e00640 1 --2- 172.21.15.47:0/1842777131 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562769257260 0x562769252ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:31:24.177 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:24.176+0000 7fba94a00640 1 -- 172.21.15.47:0/1842777131 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562769257260 msgr2=0x562769252ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:31:24.177 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:24.176+0000 7fba94a00640 1 --2- 172.21.15.47:0/1842777131 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562769257260 0x562769252ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:31:24.177 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:24.176+0000 7fba94a00640 1 -- 172.21.15.47:0/1842777131 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56276925fde0 msgr2=0x56276925f160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:31:24.177 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:24.176+0000 7fba94a00640 1 --2- 172.21.15.47:0/1842777131 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56276925fde0 0x56276925f160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:31:24.177 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:24.176+0000 7fba94a00640 1 -- 172.21.15.47:0/1842777131 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562769255b70 con 0x5627692568b0 2024-03-21T10:31:24.177 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:24.176+0000 7fba95400640 1 --2- 172.21.15.47:0/1842777131 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56276925fde0 0x56276925f160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T10:31:24.177 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:24.176+0000 7fba94a00640 1 --2- 172.21.15.47:0/1842777131 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5627692568b0 0x56276925bbf0 secure :-1 s=READY pgs=954 cs=0 l=1 rev1=1 crypto rx=0x7fba78009d90 tx=0x7fba7800f7b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b0fddc3b9116de70 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:31:24.183 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:24.177+0000 7fba85200640 1 -- 172.21.15.47:0/1842777131 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba78032070 con 0x5627692568b0 2024-03-21T10:31:24.183 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:24.177+0000 7fba85200640 1 -- 172.21.15.47:0/1842777131 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fba780054f0 con 0x5627692568b0 2024-03-21T10:31:24.183 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:24.177+0000 7fba85200640 1 -- 172.21.15.47:0/1842777131 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba7800fd80 con 0x5627692568b0 2024-03-21T10:31:24.183 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:24.177+0000 7fba9a046740 1 -- 172.21.15.47:0/1842777131 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5627692568b0 msgr2=0x56276925bbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:31:24.183 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:24.177+0000 7fba9a046740 1 --2- 172.21.15.47:0/1842777131 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5627692568b0 0x56276925bbf0 secure :-1 s=READY pgs=954 cs=0 l=1 rev1=1 crypto rx=0x7fba78009d90 tx=0x7fba7800f7b0 comp rx=0 tx=0).stop 2024-03-21T10:31:24.183 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:24.178+0000 7fba9a046740 1 -- 172.21.15.47:0/1842777131 shutdown_connections 2024-03-21T10:31:24.183 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:24.178+0000 7fba9a046740 1 --2- 172.21.15.47:0/1842777131 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56276925fde0 0x56276925f160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:31:24.183 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:24.178+0000 7fba9a046740 1 --2- 172.21.15.47:0/1842777131 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562769257260 0x562769252ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:31:24.183 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:24.178+0000 7fba9a046740 1 --2- 172.21.15.47:0/1842777131 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5627692568b0 0x56276925bbf0 unknown :-1 s=CLOSED pgs=954 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:31:24.183 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:24.178+0000 7fba9a046740 1 -- 172.21.15.47:0/1842777131 >> 172.21.15.47:0/1842777131 conn(0x562769250f80 msgr2=0x5627692523d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:31:24.184 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:24.178+0000 7fba9a046740 1 -- 172.21.15.47:0/1842777131 shutdown_connections 2024-03-21T10:31:24.184 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:24.178+0000 7fba9a046740 1 -- 172.21.15.47:0/1842777131 wait complete. 2024-03-21T10:31:24.184 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:24.178+0000 7fba9a046740 1 -- start start 2024-03-21T10:31:24.184 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:24.178+0000 7fba9a046740 1 Processor -- start 2024-03-21T10:31:24.184 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:24.179+0000 7fba9a046740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5627692568b0 0x5627692c38f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:31:24.184 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:24.179+0000 7fba9a046740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562769257260 0x5627692c3e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:31:24.184 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:24.179+0000 7fba9a046740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56276925fde0 0x562769263110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:31:24.184 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:24.179+0000 7fba9a046740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba901b1b60 con 0x56276925fde0 2024-03-21T10:31:24.184 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:24.179+0000 7fba9a046740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba901b1500 con 0x5627692568b0 2024-03-21T10:31:24.184 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:24.179+0000 7fba9a046740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba901aea00 con 0x562769257260 2024-03-21T10:31:24.184 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:24.179+0000 7fba9a046740 10 client.0 _ll_register_callbacks cb 0x7fba78005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T10:31:24.184 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:24.179+0000 7fba95400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56276925fde0 0x562769263110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:31:24.184 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:24.179+0000 7fba95400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56276925fde0 0x562769263110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:40210/0 (socket says 172.21.15.47:40210) 2024-03-21T10:31:24.184 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:24.179+0000 7fba95400640 1 -- 172.21.15.47:0/1697482192 learned_addr learned my addr 172.21.15.47:0/1697482192 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:31:24.184 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:24.179+0000 7fba94a00640 1 --2- 172.21.15.47:0/1697482192 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5627692568b0 0x5627692c38f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:31:24.185 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:24.179+0000 7fba87e00640 1 --2- 172.21.15.47:0/1697482192 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562769257260 0x5627692c3e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:31:24.185 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:24.179+0000 7fba95400640 1 -- 172.21.15.47:0/1697482192 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562769257260 msgr2=0x5627692c3e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:31:24.185 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:24.179+0000 7fba95400640 1 --2- 172.21.15.47:0/1697482192 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562769257260 0x5627692c3e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:31:24.185 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:24.179+0000 7fba95400640 1 -- 172.21.15.47:0/1697482192 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5627692568b0 msgr2=0x5627692c38f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:31:24.185 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:24.179+0000 7fba95400640 1 --2- 172.21.15.47:0/1697482192 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5627692568b0 0x5627692c38f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:31:24.185 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:24.179+0000 7fba95400640 1 -- 172.21.15.47:0/1697482192 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x562769263800 con 0x56276925fde0 2024-03-21T10:31:24.185 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:24.180+0000 7fba87e00640 1 --2- 172.21.15.47:0/1697482192 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562769257260 0x5627692c3e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T10:31:24.185 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:24.180+0000 7fba95400640 1 --2- 172.21.15.47:0/1697482192 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56276925fde0 0x562769263110 secure :-1 s=READY pgs=955 cs=0 l=1 rev1=1 crypto rx=0x7fba8802e010 tx=0x7fba88009c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=61bb8eaf3a3c7814 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:31:24.185 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:24.180+0000 7fba7e000640 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-03-21T10:31:24.185 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:24.180+0000 7fba7e000640 1 -- 172.21.15.47:0/1697482192 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba88063070 con 0x56276925fde0 2024-03-21T10:31:24.185 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:24.180+0000 7fba7e000640 1 -- 172.21.15.47:0/1697482192 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(97..97 src has 1..97) v4 ==== 9474+0+0 (secure 0 0 0) 0x7fba88040870 con 0x56276925fde0 2024-03-21T10:31:24.185 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:24.180+0000 7fba9a046740 1 -- 172.21.15.47:0/1697482192 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fba780054f0 con 0x56276925fde0 2024-03-21T10:31:24.185 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:24.180+0000 7fba7e000640 1 -- 172.21.15.47:0/1697482192 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7fba8806f050 con 0x56276925fde0 2024-03-21T10:31:24.185 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:24.180+0000 7fba9a046740 10 client.17508 fetch_fsmap learned FSMap version 32 2024-03-21T10:31:24.185 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:24.180+0000 7fba9a046740 1 -- 172.21.15.47:0/1697482192 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x5627692cb040 con 0x56276925fde0 2024-03-21T10:31:24.186 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:24.181+0000 7fba7e000640 1 -- 172.21.15.47:0/1697482192 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fba780054f0 con 0x56276925fde0 2024-03-21T10:31:24.186 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:24.181+0000 7fba7e000640 1 -- 172.21.15.47:0/1697482192 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 32) v1 ==== 7942+0+0 (secure 0 0 0) 0x7fba8804d470 con 0x56276925fde0 2024-03-21T10:31:24.186 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:24.181+0000 7fba9a046740 10 client.17508 fetch_fsmap finished waiting for FSMap version 32 2024-03-21T10:31:24.186 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:24.181+0000 7fba9a046740 10 client.17508 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-03-21T10:31:24.186 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:24.181+0000 7fba9a046740 20 client.17508 populate_metadata read hostname 'smithi047' 2024-03-21T10:31:24.186 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:24.181+0000 7fba9a046740 1 --2- 172.21.15.47:0/1697482192 >> [v2:172.21.15.100:6836/3497644162,v1:172.21.15.100:6837/3497644162] conn(0x5627692cb650 0x5627692cda70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:31:24.186 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:24.181+0000 7fba9a046740 4 client.17508 mds_command: new command op to 34493 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-21T10:31:24.186 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:24.181+0000 7fba9a046740 1 -- 172.21.15.47:0/1697482192 --> [v2:172.21.15.100:6836/3497644162,v1:172.21.15.100:6837/3497644162] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x5627691c1b50 con 0x5627692cb650 2024-03-21T10:31:24.186 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:24.182+0000 7fba94a00640 1 --2- 172.21.15.47:0/1697482192 >> [v2:172.21.15.100:6836/3497644162,v1:172.21.15.100:6837/3497644162] conn(0x5627692cb650 0x5627692cda70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:31:24.186 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:24.182+0000 7fba94a00640 1 --2- 172.21.15.47:0/1697482192 >> [v2:172.21.15.100:6836/3497644162,v1:172.21.15.100:6837/3497644162] conn(0x5627692cb650 0x5627692cda70 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=36690b10fb07a106 server_cookie=8b0fedaad93b8e9e in_seq=0 out_seq=0 2024-03-21T10:31:24.186 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:24.182+0000 7fba7e000640 10 client.17508 ms_handle_connect on v2:172.21.15.100:6836/3497644162 2024-03-21T10:31:24.186 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:24.183+0000 7fba7e000640 1 -- 172.21.15.47:0/1697482192 <== mds.1 v2:172.21.15.100:6836/3497644162 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x5627691c1b50 con 0x5627692cb650 2024-03-21T10:31:24.186 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:24.183+0000 7fba7e000640 10 client.17508 handle_command_reply: tid=0 2024-03-21T10:31:24.186 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:24.183+0000 7fba9a046740 1 client.17508 shutdown 2024-03-21T10:31:24.187 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:24.183+0000 7fba9a046740 1 -- 172.21.15.47:0/1697482192 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56276925fde0 msgr2=0x562769263110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:31:24.187 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:24.183+0000 7fba9a046740 1 --2- 172.21.15.47:0/1697482192 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56276925fde0 0x562769263110 secure :-1 s=READY pgs=955 cs=0 l=1 rev1=1 crypto rx=0x7fba8802e010 tx=0x7fba88009c20 comp rx=0 tx=0).stop 2024-03-21T10:31:24.187 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:24.183+0000 7fba9a046740 1 -- 172.21.15.47:0/1697482192 shutdown_connections 2024-03-21T10:31:24.187 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:24.183+0000 7fba9a046740 1 --2- 172.21.15.47:0/1697482192 >> [v2:172.21.15.100:6836/3497644162,v1:172.21.15.100:6837/3497644162] conn(0x5627692cb650 0x5627692cda70 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:31:24.187 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:24.183+0000 7fba9a046740 1 --2- 172.21.15.47:0/1697482192 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56276925fde0 0x562769263110 unknown :-1 s=CLOSED pgs=955 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:31:24.187 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:24.183+0000 7fba9a046740 1 --2- 172.21.15.47:0/1697482192 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562769257260 0x5627692c3e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:31:24.187 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:24.183+0000 7fba9a046740 1 --2- 172.21.15.47:0/1697482192 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5627692568b0 0x5627692c38f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:31:24.187 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:24.183+0000 7fba9a046740 1 -- 172.21.15.47:0/1697482192 >> 172.21.15.47:0/1697482192 conn(0x562769250f80 msgr2=0x562769250160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:31:24.187 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:24.183+0000 7fba7e000640 0 client.17508 ms_handle_reset on v2:172.21.15.100:6836/3497644162 2024-03-21T10:31:24.187 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:24.183+0000 7fba9a046740 1 -- 172.21.15.47:0/1697482192 shutdown_connections 2024-03-21T10:31:24.187 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:24.183+0000 7fba9a046740 1 -- 172.21.15.47:0/1697482192 wait complete. 2024-03-21T10:31:24.187 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:24.183+0000 7fba9a046740 20 client.17508 trim_cache size 0 max 16384 2024-03-21T10:31:24.188 INFO:teuthology.orchestra.run.smithi047.stdout:[] 2024-03-21T10:31:24.188 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:24.184+0000 7fba95e00640 1 -- 172.21.15.47:0/162218021 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7fba700a3090 msgr2=0x7fba700a5550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:31:24.188 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:24.184+0000 7fba95e00640 1 --2- 172.21.15.47:0/162218021 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7fba700a3090 0x7fba700a5550 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fba88004af0 tx=0x7fba8802e040 comp rx=0 tx=0).stop 2024-03-21T10:31:24.188 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:24.184+0000 7fba95e00640 1 -- 172.21.15.47:0/162218021 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fba90101ef0 msgr2=0x7fba9019a380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:31:24.188 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:24.184+0000 7fba95e00640 1 --2- 172.21.15.47:0/162218021 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fba90101ef0 0x7fba9019a380 secure :-1 s=READY pgs=952 cs=0 l=1 rev1=1 crypto rx=0x7fba8000d770 tx=0x7fba8000dc30 comp rx=0 tx=0).stop 2024-03-21T10:31:24.188 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:24.184+0000 7fba95e00640 1 -- 172.21.15.47:0/162218021 shutdown_connections 2024-03-21T10:31:24.189 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:24.184+0000 7fba95e00640 1 --2- 172.21.15.47:0/162218021 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7fba700a3090 0x7fba700a5550 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:31:24.189 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:24.184+0000 7fba95e00640 1 --2- 172.21.15.47:0/162218021 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fba901075f0 0x7fba9019e700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:31:24.189 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:24.184+0000 7fba95e00640 1 --2- 172.21.15.47:0/162218021 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fba90101ef0 0x7fba9019a380 unknown :-1 s=CLOSED pgs=952 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:31:24.189 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:24.184+0000 7fba95e00640 1 --2- 172.21.15.47:0/162218021 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fba90101530 0x7fba90199e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:31:24.189 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:24.184+0000 7fba95e00640 1 -- 172.21.15.47:0/162218021 >> 172.21.15.47:0/162218021 conn(0x7fba900fd040 msgr2=0x7fba900fe5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:31:24.189 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:24.184+0000 7fba95e00640 1 -- 172.21.15.47:0/162218021 shutdown_connections 2024-03-21T10:31:24.189 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:24.184+0000 7fba95e00640 1 -- 172.21.15.47:0/162218021 wait complete. 2024-03-21T10:31:24.201 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-21T10:31:24.201 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-03-21T10:31:24.418 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:24.419+0000 7f33a7e00640 1 -- 172.21.15.47:0/300997880 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f33a8102ff0 msgr2=0x7f33a810b4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:31:24.419 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:24.419+0000 7f33a7e00640 1 --2- 172.21.15.47:0/300997880 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f33a8102ff0 0x7f33a810b4d0 secure :-1 s=READY pgs=956 cs=0 l=1 rev1=1 crypto rx=0x7f3390009900 tx=0x7f339002bcc0 comp rx=0 tx=0).stop 2024-03-21T10:31:24.419 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:24.419+0000 7f33a7e00640 1 -- 172.21.15.47:0/300997880 shutdown_connections 2024-03-21T10:31:24.419 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:24.419+0000 7f33a7e00640 1 --2- 172.21.15.47:0/300997880 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f33a810baf0 0x7f33a810df90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:31:24.419 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:24.419+0000 7f33a7e00640 1 --2- 172.21.15.47:0/300997880 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f33a8102ff0 0x7f33a810b4d0 unknown :-1 s=CLOSED pgs=956 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:31:24.419 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:24.419+0000 7f33a7e00640 1 --2- 172.21.15.47:0/300997880 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f33a81026d0 0x7f33a8102ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:31:24.419 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:24.419+0000 7f33a7e00640 1 -- 172.21.15.47:0/300997880 >> 172.21.15.47:0/300997880 conn(0x7f33a80fbef0 msgr2=0x7f33a80fe310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:31:24.419 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:24.420+0000 7f33a7e00640 1 -- 172.21.15.47:0/300997880 shutdown_connections 2024-03-21T10:31:24.419 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:24.420+0000 7f33a7e00640 1 -- 172.21.15.47:0/300997880 wait complete. 2024-03-21T10:31:24.420 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:24.421+0000 7f33a7e00640 1 Processor -- start 2024-03-21T10:31:24.420 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:24.421+0000 7f33a7e00640 1 -- start start 2024-03-21T10:31:24.420 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:24.421+0000 7f33a7e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f33a81026d0 0x7f33a819da70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:31:24.420 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:24.421+0000 7f33a7e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f33a8102ff0 0x7f33a819dfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:31:24.420 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:24.422+0000 7f33a7e00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f33a810baf0 0x7f33a8197b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:31:24.420 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:24.422+0000 7f33a7e00640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33a81117d0 con 0x7f33a8102ff0 2024-03-21T10:31:24.421 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:24.422+0000 7f33a7e00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33a8111650 con 0x7f33a81026d0 2024-03-21T10:31:24.421 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:24.422+0000 7f33a7e00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33a8111950 con 0x7f33a810baf0 2024-03-21T10:31:24.422 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:24.422+0000 7f33a6a00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f33a81026d0 0x7f33a819da70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:31:24.422 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:24.422+0000 7f339e000640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f33a8102ff0 0x7f33a819dfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:31:24.422 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:24.422+0000 7f33a6a00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f33a81026d0 0x7f33a819da70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:58826/0 (socket says 172.21.15.47:58826) 2024-03-21T10:31:24.422 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:24.422+0000 7f33a6a00640 1 -- 172.21.15.47:0/1114431138 learned_addr learned my addr 172.21.15.47:0/1114431138 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:31:24.422 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:24.422+0000 7f33a7400640 1 --2- 172.21.15.47:0/1114431138 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f33a810baf0 0x7f33a8197b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:31:24.422 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:24.422+0000 7f33a6a00640 1 -- 172.21.15.47:0/1114431138 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f33a810baf0 msgr2=0x7f33a8197b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:31:24.423 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:24.422+0000 7f33a6a00640 1 --2- 172.21.15.47:0/1114431138 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f33a810baf0 0x7f33a8197b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:31:24.423 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:24.422+0000 7f33a6a00640 1 -- 172.21.15.47:0/1114431138 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f33a8102ff0 msgr2=0x7f33a819dfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:31:24.423 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:24.422+0000 7f33a6a00640 1 --2- 172.21.15.47:0/1114431138 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f33a8102ff0 0x7f33a819dfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:31:24.423 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:24.422+0000 7f33a6a00640 1 -- 172.21.15.47:0/1114431138 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f33a81983f0 con 0x7f33a81026d0 2024-03-21T10:31:24.423 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:24.423+0000 7f339e000640 1 --2- 172.21.15.47:0/1114431138 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f33a8102ff0 0x7f33a819dfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T10:31:24.423 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:24.423+0000 7f33a6a00640 1 --2- 172.21.15.47:0/1114431138 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f33a81026d0 0x7f33a819da70 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f339800ea80 tx=0x7f339800ef40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:31:24.423 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:24.423+0000 7f339fe00640 1 -- 172.21.15.47:0/1114431138 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f339800ce40 con 0x7f33a81026d0 2024-03-21T10:31:24.423 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:24.423+0000 7f33a7e00640 1 -- 172.21.15.47:0/1114431138 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f33a81986e0 con 0x7f33a81026d0 2024-03-21T10:31:24.423 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:24.423+0000 7f339fe00640 1 -- 172.21.15.47:0/1114431138 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f3398005020 con 0x7f33a81026d0 2024-03-21T10:31:24.423 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:24.423+0000 7f339fe00640 1 -- 172.21.15.47:0/1114431138 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33980116d0 con 0x7f33a81026d0 2024-03-21T10:31:24.423 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:24.423+0000 7f33a7e00640 1 -- 172.21.15.47:0/1114431138 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f33a81a49f0 con 0x7f33a81026d0 2024-03-21T10:31:24.425 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:24.427+0000 7f33b07c1740 1 Processor -- start 2024-03-21T10:31:24.425 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:24.427+0000 7f33b07c1740 1 -- start start 2024-03-21T10:31:24.426 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:24.427+0000 7f33b07c1740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5591b70fb850 0x5591b70fbc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:31:24.427 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:24.427+0000 7f33b07c1740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5591b70fc200 0x5591b7108c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:31:24.427 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:24.427+0000 7f33b07c1740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5591b71091f0 0x5591b710b600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:31:24.427 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:24.427+0000 7f33b07c1740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x5591b705f2c0 con 0x5591b70fc200 2024-03-21T10:31:24.427 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:24.427+0000 7f33b07c1740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x5591b7007c90 con 0x5591b70fb850 2024-03-21T10:31:24.427 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:24.427+0000 7f33b07c1740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x5591b705b360 con 0x5591b71091f0 2024-03-21T10:31:24.427 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:24.428+0000 7f33a7400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5591b70fb850 0x5591b70fbc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:31:24.427 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:24.428+0000 7f339fe00640 1 -- 172.21.15.47:0/1114431138 <== mon.2 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 27) v1 ==== 137985+0+0 (secure 0 0 0) 0x7f33980051d0 con 0x7f33a81026d0 2024-03-21T10:31:24.427 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:24.428+0000 7f33a7400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5591b70fb850 0x5591b70fbc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:58838/0 (socket says 172.21.15.47:58838) 2024-03-21T10:31:24.427 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:24.428+0000 7f33a7400640 1 -- 172.21.15.47:0/2400016890 learned_addr learned my addr 172.21.15.47:0/2400016890 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:31:24.427 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:24.428+0000 7f339e000640 1 --2- 172.21.15.47:0/2400016890 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5591b71091f0 0x5591b710b600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:31:24.427 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:24.428+0000 7f33a6a00640 1 --2- 172.21.15.47:0/2400016890 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5591b70fc200 0x5591b7108c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:31:24.428 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:24.428+0000 7f33a7400640 1 -- 172.21.15.47:0/2400016890 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5591b71091f0 msgr2=0x5591b710b600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:31:24.428 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:24.428+0000 7f33a7400640 1 --2- 172.21.15.47:0/2400016890 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5591b71091f0 0x5591b710b600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:31:24.428 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:24.428+0000 7f33a7400640 1 -- 172.21.15.47:0/2400016890 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5591b70fc200 msgr2=0x5591b7108c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:31:24.429 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:24.428+0000 7f33a7400640 1 --2- 172.21.15.47:0/2400016890 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5591b70fc200 0x5591b7108c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:31:24.429 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:24.428+0000 7f33a7400640 1 -- 172.21.15.47:0/2400016890 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5591b702c080 con 0x5591b70fb850 2024-03-21T10:31:24.429 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:24.429+0000 7f33a7400640 1 --2- 172.21.15.47:0/2400016890 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5591b70fb850 0x5591b70fbc30 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f33a0009b90 tx=0x7f33a00117b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a4a59f92588a8c47 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:31:24.429 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:24.429+0000 7f3397e00640 1 -- 172.21.15.47:0/2400016890 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33a0012450 con 0x5591b70fb850 2024-03-21T10:31:24.429 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:24.429+0000 7f3397e00640 1 -- 172.21.15.47:0/2400016890 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f33a0012aa0 con 0x5591b70fb850 2024-03-21T10:31:24.429 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:24.429+0000 7f3397e00640 1 -- 172.21.15.47:0/2400016890 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33a000fa10 con 0x5591b70fb850 2024-03-21T10:31:24.429 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:24.429+0000 7f339fe00640 1 --2- 172.21.15.47:0/1114431138 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f338c0a3090 0x7f338c0a5550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:31:24.429 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:24.430+0000 7f339fe00640 1 -- 172.21.15.47:0/1114431138 <== mon.2 v2:172.21.15.100:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 9474+0+0 (secure 0 0 0) 0x7f33980cfec0 con 0x7f33a81026d0 2024-03-21T10:31:24.429 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:24.430+0000 7f33a6a00640 1 --2- 172.21.15.47:0/1114431138 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f338c0a3090 0x7f338c0a5550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:31:24.430 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:24.430+0000 7f33b07c1740 1 -- 172.21.15.47:0/2400016890 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5591b70fb850 msgr2=0x5591b70fbc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:31:24.430 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:24.430+0000 7f33b07c1740 1 --2- 172.21.15.47:0/2400016890 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5591b70fb850 0x5591b70fbc30 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f33a0009b90 tx=0x7f33a00117b0 comp rx=0 tx=0).stop 2024-03-21T10:31:24.430 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:24.430+0000 7f33b07c1740 1 -- 172.21.15.47:0/2400016890 shutdown_connections 2024-03-21T10:31:24.431 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:24.430+0000 7f33b07c1740 1 --2- 172.21.15.47:0/2400016890 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5591b71091f0 0x5591b710b600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:31:24.431 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:24.430+0000 7f33b07c1740 1 --2- 172.21.15.47:0/2400016890 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5591b70fc200 0x5591b7108c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:31:24.431 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:24.430+0000 7f33b07c1740 1 --2- 172.21.15.47:0/2400016890 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5591b70fb850 0x5591b70fbc30 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:31:24.431 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:24.430+0000 7f33b07c1740 1 -- 172.21.15.47:0/2400016890 >> 172.21.15.47:0/2400016890 conn(0x5591b70f5f80 msgr2=0x5591b70f6360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:31:24.431 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:24.430+0000 7f33b07c1740 1 -- 172.21.15.47:0/2400016890 shutdown_connections 2024-03-21T10:31:24.431 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:24.430+0000 7f33b07c1740 1 -- 172.21.15.47:0/2400016890 wait complete. 2024-03-21T10:31:24.431 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:24.431+0000 7f33a6a00640 1 --2- 172.21.15.47:0/1114431138 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f338c0a3090 0x7f338c0a5550 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f33980cd300 tx=0x7f3398005810 comp rx=0 tx=0).ready entity=mgr.15687 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:31:24.431 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:24.431+0000 7f33b07c1740 1 -- start start 2024-03-21T10:31:24.431 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:24.431+0000 7f33b07c1740 1 Processor -- start 2024-03-21T10:31:24.434 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:24.432+0000 7f33b07c1740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5591b70fb850 0x5591b7108b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:31:24.434 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:24.432+0000 7f33b07c1740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5591b70fc200 0x5591b7105e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:31:24.434 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:24.432+0000 7f33b07c1740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5591b71091f0 0x5591b7106430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:31:24.434 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:24.432+0000 7f33b07c1740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x5591b710c120 con 0x5591b71091f0 2024-03-21T10:31:24.434 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:24.432+0000 7f33b07c1740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x5591b701b0d0 con 0x5591b70fb850 2024-03-21T10:31:24.434 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:24.432+0000 7f33b07c1740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x5591b710c2a0 con 0x5591b70fc200 2024-03-21T10:31:24.434 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:24.432+0000 7f33b07c1740 10 client.0 _ll_register_callbacks cb 0x5591b701fe80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T10:31:24.434 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:24.432+0000 7f33a7400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5591b71091f0 0x5591b7106430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:31:24.434 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:24.433+0000 7f33a7400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5591b70fb850 0x5591b7108b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:31:24.434 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:24.433+0000 7f33a7400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5591b70fb850 0x5591b7108b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:58850/0 (socket says 172.21.15.47:58850) 2024-03-21T10:31:24.434 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:24.433+0000 7f33a7400640 1 -- 172.21.15.47:0/4049978930 learned_addr learned my addr 172.21.15.47:0/4049978930 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:31:24.435 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:24.433+0000 7f339e000640 1 --2- 172.21.15.47:0/4049978930 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5591b70fc200 0x5591b7105e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:31:24.435 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:24.433+0000 7f33a7400640 1 -- 172.21.15.47:0/4049978930 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5591b70fc200 msgr2=0x5591b7105e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:31:24.435 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:24.433+0000 7f33a7400640 1 --2- 172.21.15.47:0/4049978930 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5591b70fc200 0x5591b7105e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:31:24.435 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:24.433+0000 7f33a7400640 1 -- 172.21.15.47:0/4049978930 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5591b70fb850 msgr2=0x5591b7108b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:31:24.435 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:24.433+0000 7f33a7400640 1 --2- 172.21.15.47:0/4049978930 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5591b70fb850 0x5591b7108b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:31:24.435 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:24.433+0000 7f33a7400640 1 -- 172.21.15.47:0/4049978930 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5591b6d733f0 con 0x5591b71091f0 2024-03-21T10:31:24.435 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:24.434+0000 7f33a7400640 1 --2- 172.21.15.47:0/4049978930 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5591b71091f0 0x5591b7106430 secure :-1 s=READY pgs=957 cs=0 l=1 rev1=1 crypto rx=0x7f33a0012e90 tx=0x7f33a0014040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e1ab2623d000672 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:31:24.435 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:24.434+0000 7f3394c00640 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-03-21T10:31:24.435 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:24.434+0000 7f3394c00640 1 -- 172.21.15.47:0/4049978930 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33a00128e0 con 0x5591b71091f0 2024-03-21T10:31:24.435 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:24.434+0000 7f33b07c1740 1 -- 172.21.15.47:0/4049978930 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f33a0012aa0 con 0x5591b71091f0 2024-03-21T10:31:24.435 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:24.434+0000 7f3394c00640 1 -- 172.21.15.47:0/4049978930 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(97..97 src has 1..97) v4 ==== 9474+0+0 (secure 0 0 0) 0x7f33a0020040 con 0x5591b71091f0 2024-03-21T10:31:24.435 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:24.434+0000 7f3394c00640 1 -- 172.21.15.47:0/4049978930 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f33a0036050 con 0x5591b71091f0 2024-03-21T10:31:24.435 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:24.434+0000 7f33b07c1740 10 client.17520 fetch_fsmap learned FSMap version 32 2024-03-21T10:31:24.435 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:24.434+0000 7f33b07c1740 1 -- 172.21.15.47:0/4049978930 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x5591b6e3d7d0 con 0x5591b71091f0 2024-03-21T10:31:24.436 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:24.435+0000 7f3394c00640 1 -- 172.21.15.47:0/4049978930 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f33a0012aa0 con 0x5591b71091f0 2024-03-21T10:31:24.436 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:24.435+0000 7f3394c00640 1 -- 172.21.15.47:0/4049978930 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 32) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f33a0026040 con 0x5591b71091f0 2024-03-21T10:31:24.436 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:24.435+0000 7f33b07c1740 10 client.17520 fetch_fsmap finished waiting for FSMap version 32 2024-03-21T10:31:24.436 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:24.435+0000 7f33b07c1740 10 client.17520 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-21T10:31:24.436 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:24.435+0000 7f33b07c1740 20 client.17520 populate_metadata read hostname 'smithi047' 2024-03-21T10:31:24.436 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:24.435+0000 7f33b07c1740 1 --2- 172.21.15.47:0/4049978930 >> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] conn(0x5591b7118f00 0x5591b711b2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:31:24.436 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:24.435+0000 7f33b07c1740 4 client.17520 mds_command: new command op to 34535 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T10:31:24.436 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:24.435+0000 7f33b07c1740 1 -- 172.21.15.47:0/4049978930 --> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5591b706c800 con 0x5591b7118f00 2024-03-21T10:31:24.437 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:24.435+0000 7f339e000640 1 --2- 172.21.15.47:0/4049978930 >> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] conn(0x5591b7118f00 0x5591b711b2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:31:24.438 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:24.436+0000 7f339e000640 1 --2- 172.21.15.47:0/4049978930 >> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] conn(0x5591b7118f00 0x5591b711b2f0 crc :-1 s=READY pgs=155 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a908a09ac6b036bc server_cookie=ab1ae269b7a42bb2 in_seq=0 out_seq=0 2024-03-21T10:31:24.438 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:24.436+0000 7f3394c00640 10 client.17520 ms_handle_connect on v2:172.21.15.100:6838/4173168440 2024-03-21T10:31:24.438 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:24.437+0000 7f3394c00640 1 -- 172.21.15.47:0/4049978930 <== mds.0 v2:172.21.15.100:6838/4173168440 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x5591b706c800 con 0x5591b7118f00 2024-03-21T10:31:24.438 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:24.437+0000 7f3394c00640 10 client.17520 handle_command_reply: tid=0 2024-03-21T10:31:24.438 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:24.437+0000 7f33b07c1740 1 client.17520 shutdown 2024-03-21T10:31:24.438 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:24.438+0000 7f33b07c1740 1 -- 172.21.15.47:0/4049978930 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5591b71091f0 msgr2=0x5591b7106430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:31:24.438 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:24.438+0000 7f33b07c1740 1 --2- 172.21.15.47:0/4049978930 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5591b71091f0 0x5591b7106430 secure :-1 s=READY pgs=957 cs=0 l=1 rev1=1 crypto rx=0x7f33a0012e90 tx=0x7f33a0014040 comp rx=0 tx=0).stop 2024-03-21T10:31:24.438 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:24.438+0000 7f33b07c1740 1 -- 172.21.15.47:0/4049978930 shutdown_connections 2024-03-21T10:31:24.438 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:24.438+0000 7f33b07c1740 1 --2- 172.21.15.47:0/4049978930 >> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] conn(0x5591b7118f00 0x5591b711b2f0 crc :-1 s=READY pgs=155 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:31:24.438 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:24.438+0000 7f33b07c1740 1 --2- 172.21.15.47:0/4049978930 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5591b71091f0 0x5591b7106430 unknown :-1 s=CLOSED pgs=957 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:31:24.438 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:24.438+0000 7f33b07c1740 1 --2- 172.21.15.47:0/4049978930 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5591b70fc200 0x5591b7105e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:31:24.438 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:24.438+0000 7f33b07c1740 1 --2- 172.21.15.47:0/4049978930 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5591b70fb850 0x5591b7108b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:31:24.438 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:24.438+0000 7f33b07c1740 1 -- 172.21.15.47:0/4049978930 >> 172.21.15.47:0/4049978930 conn(0x5591b70f5f80 msgr2=0x5591b70f7cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:31:24.439 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:24.438+0000 7f3394c00640 0 client.17520 ms_handle_reset on v2:172.21.15.100:6838/4173168440 2024-03-21T10:31:24.439 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:24.438+0000 7f33b07c1740 1 -- 172.21.15.47:0/4049978930 shutdown_connections 2024-03-21T10:31:24.439 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:24.438+0000 7f33b07c1740 1 -- 172.21.15.47:0/4049978930 wait complete. 2024-03-21T10:31:24.439 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:24.438+0000 7f33b07c1740 20 client.17520 trim_cache size 0 max 16384 2024-03-21T10:31:24.452 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:24.452+0000 7f33b07c1740 1 Processor -- start 2024-03-21T10:31:24.452 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:24.452+0000 7f33b07c1740 1 -- start start 2024-03-21T10:31:24.452 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:24.453+0000 7f33b07c1740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5591b70fb850 0x5591b7101690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:31:24.452 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:24.453+0000 7f33b07c1740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5591b70fc200 0x5591b71085c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:31:24.452 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:24.453+0000 7f33b07c1740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5591b71091f0 0x5591b7108b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:31:24.453 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:24.453+0000 7f33b07c1740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33a81ac4e0 con 0x5591b71091f0 2024-03-21T10:31:24.453 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:24.453+0000 7f33b07c1740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33a81ade60 con 0x5591b70fc200 2024-03-21T10:31:24.453 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:24.453+0000 7f33b07c1740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33a81acb40 con 0x5591b70fb850 2024-03-21T10:31:24.453 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:24.453+0000 7f33a7400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5591b71091f0 0x5591b7108b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:31:24.453 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:24.453+0000 7f33a7400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5591b71091f0 0x5591b7108b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:40252/0 (socket says 172.21.15.47:40252) 2024-03-21T10:31:24.453 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:24.453+0000 7f33a7400640 1 -- 172.21.15.47:0/469186640 learned_addr learned my addr 172.21.15.47:0/469186640 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:31:24.453 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:24.453+0000 7f339e000640 1 --2- 172.21.15.47:0/469186640 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5591b70fc200 0x5591b71085c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:31:24.453 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:24.453+0000 7f33a7400640 1 --2- 172.21.15.47:0/469186640 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5591b70fb850 0x5591b7101690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:31:24.453 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:24.454+0000 7f33a7400640 1 -- 172.21.15.47:0/469186640 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5591b70fb850 msgr2=0x5591b7101690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:31:24.453 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:24.454+0000 7f33a7400640 1 --2- 172.21.15.47:0/469186640 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5591b70fb850 0x5591b7101690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:31:24.453 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:24.454+0000 7f33a7400640 1 -- 172.21.15.47:0/469186640 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5591b70fc200 msgr2=0x5591b71085c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:31:24.453 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:24.454+0000 7f33a7400640 1 --2- 172.21.15.47:0/469186640 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5591b70fc200 0x5591b71085c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:31:24.453 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:24.454+0000 7f33a7400640 1 -- 172.21.15.47:0/469186640 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5591b70fc5e0 con 0x5591b71091f0 2024-03-21T10:31:24.454 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:24.454+0000 7f33a7400640 1 --2- 172.21.15.47:0/469186640 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5591b71091f0 0x5591b7108b00 secure :-1 s=READY pgs=958 cs=0 l=1 rev1=1 crypto rx=0x7f33a0009b90 tx=0x7f33a00117b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7828d651a17bad34 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:31:24.454 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:24.455+0000 7f3397400640 1 -- 172.21.15.47:0/469186640 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33a0009570 con 0x5591b71091f0 2024-03-21T10:31:24.454 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:24.455+0000 7f3397400640 1 -- 172.21.15.47:0/469186640 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f33a0005bf0 con 0x5591b71091f0 2024-03-21T10:31:24.454 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:24.455+0000 7f3397400640 1 -- 172.21.15.47:0/469186640 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33a0005330 con 0x5591b71091f0 2024-03-21T10:31:24.455 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:24.456+0000 7f33b07c1740 1 -- 172.21.15.47:0/469186640 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5591b71091f0 msgr2=0x5591b7108b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:31:24.455 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:24.456+0000 7f33b07c1740 1 --2- 172.21.15.47:0/469186640 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5591b71091f0 0x5591b7108b00 secure :-1 s=READY pgs=958 cs=0 l=1 rev1=1 crypto rx=0x7f33a0009b90 tx=0x7f33a00117b0 comp rx=0 tx=0).stop 2024-03-21T10:31:24.455 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:24.456+0000 7f33b07c1740 1 -- 172.21.15.47:0/469186640 shutdown_connections 2024-03-21T10:31:24.455 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:24.456+0000 7f33b07c1740 1 --2- 172.21.15.47:0/469186640 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5591b71091f0 0x5591b7108b00 unknown :-1 s=CLOSED pgs=958 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:31:24.455 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:24.456+0000 7f33b07c1740 1 --2- 172.21.15.47:0/469186640 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5591b70fc200 0x5591b71085c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:31:24.455 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:24.456+0000 7f33b07c1740 1 --2- 172.21.15.47:0/469186640 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5591b70fb850 0x5591b7101690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:31:24.455 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:24.456+0000 7f33b07c1740 1 -- 172.21.15.47:0/469186640 >> 172.21.15.47:0/469186640 conn(0x5591b70f5f80 msgr2=0x5591b70f7a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:31:24.455 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:24.456+0000 7f33b07c1740 1 -- 172.21.15.47:0/469186640 shutdown_connections 2024-03-21T10:31:24.456 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:24.456+0000 7f33b07c1740 1 -- 172.21.15.47:0/469186640 wait complete. 2024-03-21T10:31:24.459 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:24.457+0000 7f33b07c1740 1 -- start start 2024-03-21T10:31:24.459 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:24.457+0000 7f33b07c1740 1 Processor -- start 2024-03-21T10:31:24.460 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:24.457+0000 7f33b07c1740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5591b70fb850 0x5591b711d5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:31:24.460 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:24.457+0000 7f33b07c1740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5591b70fc200 0x5591b711db20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:31:24.460 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:24.457+0000 7f33b07c1740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5591b71091f0 0x5591b711bb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:31:24.460 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:24.457+0000 7f33b07c1740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33a81ad800 con 0x5591b71091f0 2024-03-21T10:31:24.460 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:24.457+0000 7f33b07c1740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33a81ad1a0 con 0x5591b70fc200 2024-03-21T10:31:24.460 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:24.458+0000 7f33b07c1740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33a81aa6a0 con 0x5591b70fb850 2024-03-21T10:31:24.460 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:24.458+0000 7f33b07c1740 10 client.0 _ll_register_callbacks cb 0x7f33a0009a60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T10:31:24.460 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:24.458+0000 7f339e000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5591b70fc200 0x5591b711db20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:31:24.460 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:24.458+0000 7f33a7400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5591b71091f0 0x5591b711bb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:31:24.460 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:24.458+0000 7f339e000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5591b70fc200 0x5591b711db20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:58866/0 (socket says 172.21.15.47:58866) 2024-03-21T10:31:24.460 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:24.458+0000 7f339e000640 1 -- 172.21.15.47:0/1140742766 learned_addr learned my addr 172.21.15.47:0/1140742766 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:31:24.460 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:24.458+0000 7f33a7400640 1 --2- 172.21.15.47:0/1140742766 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5591b70fb850 0x5591b711d5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:31:24.460 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:24.458+0000 7f339e000640 1 -- 172.21.15.47:0/1140742766 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5591b70fb850 msgr2=0x5591b711d5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:31:24.460 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:24.458+0000 7f339e000640 1 --2- 172.21.15.47:0/1140742766 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5591b70fb850 0x5591b711d5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:31:24.461 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:24.458+0000 7f339e000640 1 -- 172.21.15.47:0/1140742766 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5591b71091f0 msgr2=0x5591b711bb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:31:24.461 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:24.458+0000 7f339e000640 1 --2- 172.21.15.47:0/1140742766 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5591b71091f0 0x5591b711bb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:31:24.461 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:24.458+0000 7f339e000640 1 -- 172.21.15.47:0/1140742766 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5591b711c200 con 0x5591b70fc200 2024-03-21T10:31:24.461 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:24.458+0000 7f33a7400640 1 --2- 172.21.15.47:0/1140742766 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5591b70fb850 0x5591b711d5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T10:31:24.461 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:24.459+0000 7f339e000640 1 --2- 172.21.15.47:0/1140742766 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5591b70fc200 0x5591b711db20 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f3390005ab0 tx=0x7f339002c7a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e16f9e90c86b80a5 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:31:24.461 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:24.459+0000 7f3394c00640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-21T10:31:24.461 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:24.459+0000 7f3394c00640 1 -- 172.21.15.47:0/1140742766 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3390048310 con 0x5591b70fc200 2024-03-21T10:31:24.461 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:24.459+0000 7f33b07c1740 1 -- 172.21.15.47:0/1140742766 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f33a0005bf0 con 0x5591b70fc200 2024-03-21T10:31:24.461 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:24.459+0000 7f3394c00640 1 -- 172.21.15.47:0/1140742766 <== mon.2 v2:172.21.15.100:3300/0 2 ==== osd_map(97..97 src has 1..97) v4 ==== 9474+0+0 (secure 0 0 0) 0x7f339002cb00 con 0x5591b70fc200 2024-03-21T10:31:24.461 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:24.459+0000 7f3394c00640 1 -- 172.21.15.47:0/1140742766 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3390058050 con 0x5591b70fc200 2024-03-21T10:31:24.461 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:24.459+0000 7f33b07c1740 10 client.37313 fetch_fsmap learned FSMap version 32 2024-03-21T10:31:24.461 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:24.460+0000 7f33b07c1740 1 -- 172.21.15.47:0/1140742766 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x5591b7174f50 con 0x5591b70fc200 2024-03-21T10:31:24.461 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:24.460+0000 7f3394c00640 1 -- 172.21.15.47:0/1140742766 <== mon.2 v2:172.21.15.100:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f33a0005bf0 con 0x5591b70fc200 2024-03-21T10:31:24.461 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:24.460+0000 7f3394c00640 1 -- 172.21.15.47:0/1140742766 <== mon.2 v2:172.21.15.100:3300/0 5 ==== fsmap(e 32) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f3390051460 con 0x5591b70fc200 2024-03-21T10:31:24.461 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:24.460+0000 7f33b07c1740 10 client.37313 fetch_fsmap finished waiting for FSMap version 32 2024-03-21T10:31:24.462 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:24.460+0000 7f33b07c1740 10 client.37313 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-21T10:31:24.462 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:24.460+0000 7f33b07c1740 20 client.37313 populate_metadata read hostname 'smithi047' 2024-03-21T10:31:24.462 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:24.460+0000 7f33b07c1740 1 --2- 172.21.15.47:0/1140742766 >> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] conn(0x5591b7175630 0x5591b7177a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:31:24.462 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:24.460+0000 7f33b07c1740 4 client.37313 mds_command: new command op to 34535 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-21T10:31:24.462 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:24.460+0000 7f33b07c1740 1 -- 172.21.15.47:0/1140742766 --> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x5591b7066b50 con 0x5591b7175630 2024-03-21T10:31:24.462 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:24.461+0000 7f33a7400640 1 --2- 172.21.15.47:0/1140742766 >> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] conn(0x5591b7175630 0x5591b7177a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:31:24.463 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:24.461+0000 7f33a7400640 1 --2- 172.21.15.47:0/1140742766 >> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] conn(0x5591b7175630 0x5591b7177a50 crc :-1 s=READY pgs=156 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c4b5c52be0739b56 server_cookie=877bb21df5e92d30 in_seq=0 out_seq=0 2024-03-21T10:31:24.463 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:24.461+0000 7f3394c00640 10 client.37313 ms_handle_connect on v2:172.21.15.100:6838/4173168440 2024-03-21T10:31:24.463 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:24.462+0000 7f3394c00640 1 -- 172.21.15.47:0/1140742766 <== mds.0 v2:172.21.15.100:6838/4173168440 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x5591b7066b50 con 0x5591b7175630 2024-03-21T10:31:24.463 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:24.462+0000 7f3394c00640 10 client.37313 handle_command_reply: tid=0 2024-03-21T10:31:24.463 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:24.462+0000 7f33b07c1740 1 client.37313 shutdown 2024-03-21T10:31:24.463 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:24.462+0000 7f33b07c1740 1 -- 172.21.15.47:0/1140742766 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5591b70fc200 msgr2=0x5591b711db20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:31:24.463 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:24.462+0000 7f33b07c1740 1 --2- 172.21.15.47:0/1140742766 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5591b70fc200 0x5591b711db20 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f3390005ab0 tx=0x7f339002c7a0 comp rx=0 tx=0).stop 2024-03-21T10:31:24.463 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:24.462+0000 7f33b07c1740 1 -- 172.21.15.47:0/1140742766 shutdown_connections 2024-03-21T10:31:24.463 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:24.462+0000 7f33b07c1740 1 --2- 172.21.15.47:0/1140742766 >> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] conn(0x5591b7175630 0x5591b7177a50 crc :-1 s=READY pgs=156 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:31:24.464 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:24.462+0000 7f33b07c1740 1 --2- 172.21.15.47:0/1140742766 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5591b71091f0 0x5591b711bb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:31:24.464 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:24.462+0000 7f33b07c1740 1 --2- 172.21.15.47:0/1140742766 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5591b70fc200 0x5591b711db20 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:31:24.464 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:24.462+0000 7f33b07c1740 1 --2- 172.21.15.47:0/1140742766 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5591b70fb850 0x5591b711d5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:31:24.464 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:24.462+0000 7f3394c00640 0 client.37313 ms_handle_reset on v2:172.21.15.100:6838/4173168440 2024-03-21T10:31:24.464 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:24.462+0000 7f33b07c1740 1 -- 172.21.15.47:0/1140742766 >> 172.21.15.47:0/1140742766 conn(0x5591b70f5f80 msgr2=0x5591b7114530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:31:24.464 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:24.462+0000 7f33b07c1740 1 -- 172.21.15.47:0/1140742766 shutdown_connections 2024-03-21T10:31:24.464 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:24.462+0000 7f33b07c1740 1 -- 172.21.15.47:0/1140742766 wait complete. 2024-03-21T10:31:24.464 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:24.463+0000 7f33b07c1740 20 client.37313 trim_cache size 0 max 16384 2024-03-21T10:31:24.464 INFO:teuthology.orchestra.run.smithi047.stdout:[] 2024-03-21T10:31:24.465 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:24.463+0000 7f33a7e00640 1 -- 172.21.15.47:0/1114431138 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f338c0a3090 msgr2=0x7f338c0a5550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:31:24.465 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:24.463+0000 7f33a7e00640 1 --2- 172.21.15.47:0/1114431138 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f338c0a3090 0x7f338c0a5550 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f33980cd300 tx=0x7f3398005810 comp rx=0 tx=0).stop 2024-03-21T10:31:24.465 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:24.463+0000 7f33a7e00640 1 -- 172.21.15.47:0/1114431138 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f33a81026d0 msgr2=0x7f33a819da70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:31:24.465 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:24.463+0000 7f33a7e00640 1 --2- 172.21.15.47:0/1114431138 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f33a81026d0 0x7f33a819da70 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f339800ea80 tx=0x7f339800ef40 comp rx=0 tx=0).stop 2024-03-21T10:31:24.465 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:24.463+0000 7f33a7e00640 1 -- 172.21.15.47:0/1114431138 shutdown_connections 2024-03-21T10:31:24.465 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:24.463+0000 7f33a7e00640 1 --2- 172.21.15.47:0/1114431138 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f33a810baf0 0x7f33a8197b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:31:24.465 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:24.463+0000 7f33a7e00640 1 --2- 172.21.15.47:0/1114431138 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f33a8102ff0 0x7f33a819dfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:31:24.465 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:24.463+0000 7f33a7e00640 1 --2- 172.21.15.47:0/1114431138 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f338c0a3090 0x7f338c0a5550 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:31:24.466 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:24.463+0000 7f33a7e00640 1 --2- 172.21.15.47:0/1114431138 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f33a81026d0 0x7f33a819da70 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:31:24.466 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:24.463+0000 7f33a7e00640 1 -- 172.21.15.47:0/1114431138 >> 172.21.15.47:0/1114431138 conn(0x7f33a80fbef0 msgr2=0x7f33a80fcc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:31:24.466 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:24.464+0000 7f33a7e00640 1 -- 172.21.15.47:0/1114431138 shutdown_connections 2024-03-21T10:31:24.466 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:24.464+0000 7f33a7e00640 1 -- 172.21.15.47:0/1114431138 wait complete. 2024-03-21T10:31:25.481 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-03-21T10:31:25.481 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-21T10:31:25.482 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-03-21T10:31:25.711 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:31:25 smithi047 ceph-mon[31547]: pgmap v186: 129 pgs: 129 active+clean; 277 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 8 op/s 2024-03-21T10:31:25.715 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:25.716+0000 7fa797200640 1 -- 172.21.15.47:0/291325234 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa790101620 msgr2=0x7fa790101aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:31:25.715 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:25.716+0000 7fa797200640 1 --2- 172.21.15.47:0/291325234 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa790101620 0x7fa790101aa0 secure :-1 s=READY pgs=959 cs=0 l=1 rev1=1 crypto rx=0x7fa778009930 tx=0x7fa77802bc20 comp rx=0 tx=0).stop 2024-03-21T10:31:25.715 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:25.716+0000 7fa797200640 1 -- 172.21.15.47:0/291325234 shutdown_connections 2024-03-21T10:31:25.715 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:25.716+0000 7fa797200640 1 --2- 172.21.15.47:0/291325234 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa790101fe0 0x7fa79010e4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:31:25.715 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:25.716+0000 7fa797200640 1 --2- 172.21.15.47:0/291325234 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa790101620 0x7fa790101aa0 unknown :-1 s=CLOSED pgs=959 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:31:25.715 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:25.716+0000 7fa797200640 1 --2- 172.21.15.47:0/291325234 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa790107630 0x7fa790107a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:31:25.715 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:25.716+0000 7fa797200640 1 -- 172.21.15.47:0/291325234 >> 172.21.15.47:0/291325234 conn(0x7fa7900fd130 msgr2=0x7fa7900ff550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:31:25.715 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:25.716+0000 7fa797200640 1 -- 172.21.15.47:0/291325234 shutdown_connections 2024-03-21T10:31:25.715 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:25.716+0000 7fa797200640 1 -- 172.21.15.47:0/291325234 wait complete. 2024-03-21T10:31:25.717 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:25.717+0000 7fa797200640 1 Processor -- start 2024-03-21T10:31:25.717 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:25.717+0000 7fa797200640 1 -- start start 2024-03-21T10:31:25.717 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:25.718+0000 7fa797200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa790101620 0x7fa79019a0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:31:25.717 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:25.718+0000 7fa797200640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa790101fe0 0x7fa79019a5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:31:25.717 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:25.718+0000 7fa797200640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa790107630 0x7fa79019e960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:31:25.717 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:25.718+0000 7fa797200640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7901118b0 con 0x7fa790107630 2024-03-21T10:31:25.718 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:25.718+0000 7fa797200640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa790111730 con 0x7fa790101620 2024-03-21T10:31:25.718 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:25.718+0000 7fa797200640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa790111a30 con 0x7fa790101fe0 2024-03-21T10:31:25.718 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:25.718+0000 7fa795200640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa790107630 0x7fa79019e960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:31:25.718 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:25.718+0000 7fa795200640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa790107630 0x7fa79019e960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:40282/0 (socket says 172.21.15.47:40282) 2024-03-21T10:31:25.718 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:25.718+0000 7fa795200640 1 -- 172.21.15.47:0/866990308 learned_addr learned my addr 172.21.15.47:0/866990308 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:31:25.718 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:25.718+0000 7fa78fe00640 1 --2- 172.21.15.47:0/866990308 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa790101620 0x7fa79019a0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:31:25.718 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:25.718+0000 7fa787400640 1 --2- 172.21.15.47:0/866990308 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa790101fe0 0x7fa79019a5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:31:25.718 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:25.718+0000 7fa795200640 1 -- 172.21.15.47:0/866990308 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa790101fe0 msgr2=0x7fa79019a5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:31:25.718 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:25.718+0000 7fa795200640 1 --2- 172.21.15.47:0/866990308 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa790101fe0 0x7fa79019a5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:31:25.718 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:25.718+0000 7fa795200640 1 -- 172.21.15.47:0/866990308 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa790101620 msgr2=0x7fa79019a0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:31:25.718 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:25.718+0000 7fa795200640 1 --2- 172.21.15.47:0/866990308 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa790101620 0x7fa79019a0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:31:25.718 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:25.718+0000 7fa795200640 1 -- 172.21.15.47:0/866990308 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa79019f090 con 0x7fa790107630 2024-03-21T10:31:25.718 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:25.718+0000 7fa78fe00640 1 --2- 172.21.15.47:0/866990308 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa790101620 0x7fa79019a0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T10:31:25.719 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:25.719+0000 7fa795200640 1 --2- 172.21.15.47:0/866990308 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa790107630 0x7fa79019e960 secure :-1 s=READY pgs=960 cs=0 l=1 rev1=1 crypto rx=0x7fa78800e970 tx=0x7fa78800ed70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:31:25.719 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:25.719+0000 7fa78d600640 1 -- 172.21.15.47:0/866990308 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa78800ccc0 con 0x7fa790107630 2024-03-21T10:31:25.719 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:25.719+0000 7fa797200640 1 -- 172.21.15.47:0/866990308 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa79019f380 con 0x7fa790107630 2024-03-21T10:31:25.719 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:25.719+0000 7fa78d600640 1 -- 172.21.15.47:0/866990308 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fa78800ce60 con 0x7fa790107630 2024-03-21T10:31:25.720 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:25.719+0000 7fa78d600640 1 -- 172.21.15.47:0/866990308 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa788010430 con 0x7fa790107630 2024-03-21T10:31:25.720 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:25.719+0000 7fa797200640 1 -- 172.21.15.47:0/866990308 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa7901a6c10 con 0x7fa790107630 2024-03-21T10:31:25.721 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:25.723+0000 7fa78d600640 1 -- 172.21.15.47:0/866990308 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 27) v1 ==== 137985+0+0 (secure 0 0 0) 0x7fa788004ff0 con 0x7fa790107630 2024-03-21T10:31:25.721 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:25.723+0000 7fa799e84740 1 Processor -- start 2024-03-21T10:31:25.723 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:25.723+0000 7fa799e84740 1 -- start start 2024-03-21T10:31:25.723 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:25.724+0000 7fa799e84740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x558f95890080 0x558f95890460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:31:25.723 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:25.724+0000 7fa799e84740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558f958909a0 0x558f95899040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:31:25.723 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:25.724+0000 7fa799e84740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558f95899580 0x558f9589b970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:31:25.723 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:25.724+0000 7fa799e84740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x558f957f4eb0 con 0x558f95890080 2024-03-21T10:31:25.723 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:25.724+0000 7fa799e84740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x558f9579da60 con 0x558f958909a0 2024-03-21T10:31:25.723 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:25.724+0000 7fa799e84740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x558f957f0f50 con 0x558f95899580 2024-03-21T10:31:25.723 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:25.724+0000 7fa787400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x558f95890080 0x558f95890460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:31:25.723 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:25.724+0000 7fa787400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x558f95890080 0x558f95890460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:40292/0 (socket says 172.21.15.47:40292) 2024-03-21T10:31:25.723 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:25.724+0000 7fa787400640 1 -- 172.21.15.47:0/3850735704 learned_addr learned my addr 172.21.15.47:0/3850735704 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:31:25.724 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:25.724+0000 7fa78fe00640 1 --2- 172.21.15.47:0/3850735704 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558f958909a0 0x558f95899040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:31:25.725 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:25.725+0000 7fa78d600640 1 --2- 172.21.15.47:0/866990308 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7fa7740a3090 0x7fa7740a5550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:31:25.725 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:25.725+0000 7fa787400640 1 --2- 172.21.15.47:0/3850735704 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558f95899580 0x558f9589b970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:31:25.725 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:25.725+0000 7fa787400640 1 -- 172.21.15.47:0/3850735704 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558f95899580 msgr2=0x558f9589b970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:31:25.725 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:25.725+0000 7fa787400640 1 --2- 172.21.15.47:0/3850735704 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558f95899580 0x558f9589b970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:31:25.725 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:25.725+0000 7fa787400640 1 -- 172.21.15.47:0/3850735704 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558f958909a0 msgr2=0x558f95899040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:31:25.725 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:25.725+0000 7fa787400640 1 --2- 172.21.15.47:0/3850735704 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558f958909a0 0x558f95899040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:31:25.725 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:25.725+0000 7fa787400640 1 -- 172.21.15.47:0/3850735704 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558f955d17e0 con 0x558f95890080 2024-03-21T10:31:25.725 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:25.725+0000 7fa78d600640 1 -- 172.21.15.47:0/866990308 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 9474+0+0 (secure 0 0 0) 0x7fa7880d0150 con 0x7fa790107630 2024-03-21T10:31:25.725 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:25.725+0000 7fa795200640 1 --2- 172.21.15.47:0/866990308 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7fa7740a3090 0x7fa7740a5550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:31:25.725 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:25.726+0000 7fa787400640 1 --2- 172.21.15.47:0/3850735704 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x558f95890080 0x558f95890460 secure :-1 s=READY pgs=961 cs=0 l=1 rev1=1 crypto rx=0x7fa778005940 tx=0x7fa778005a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6f04eef99db87abe server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:31:25.725 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:25.726+0000 7fa795200640 1 --2- 172.21.15.47:0/866990308 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7fa7740a3090 0x7fa7740a5550 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fa788098580 tx=0x7fa7880bc040 comp rx=0 tx=0).ready entity=mgr.15687 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:31:25.726 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:25.726+0000 7fa785600640 1 -- 172.21.15.47:0/3850735704 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7780041c0 con 0x558f95890080 2024-03-21T10:31:25.726 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:25.726+0000 7fa785600640 1 -- 172.21.15.47:0/3850735704 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fa77802cc80 con 0x558f95890080 2024-03-21T10:31:25.726 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:25.726+0000 7fa785600640 1 -- 172.21.15.47:0/3850735704 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7780046c0 con 0x558f95890080 2024-03-21T10:31:25.727 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:25.727+0000 7fa799e84740 1 -- 172.21.15.47:0/3850735704 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x558f95890080 msgr2=0x558f95890460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:31:25.727 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:25.727+0000 7fa799e84740 1 --2- 172.21.15.47:0/3850735704 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x558f95890080 0x558f95890460 secure :-1 s=READY pgs=961 cs=0 l=1 rev1=1 crypto rx=0x7fa778005940 tx=0x7fa778005a20 comp rx=0 tx=0).stop 2024-03-21T10:31:25.727 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:25.727+0000 7fa799e84740 1 -- 172.21.15.47:0/3850735704 shutdown_connections 2024-03-21T10:31:25.727 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:25.727+0000 7fa799e84740 1 --2- 172.21.15.47:0/3850735704 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558f95899580 0x558f9589b970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:31:25.727 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:25.727+0000 7fa799e84740 1 --2- 172.21.15.47:0/3850735704 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558f958909a0 0x558f95899040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:31:25.727 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:25.727+0000 7fa799e84740 1 --2- 172.21.15.47:0/3850735704 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x558f95890080 0x558f95890460 unknown :-1 s=CLOSED pgs=961 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:31:25.727 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:25.727+0000 7fa799e84740 1 -- 172.21.15.47:0/3850735704 >> 172.21.15.47:0/3850735704 conn(0x558f9588a7b0 msgr2=0x558f9588ab90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:31:25.727 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:25.727+0000 7fa799e84740 1 -- 172.21.15.47:0/3850735704 shutdown_connections 2024-03-21T10:31:25.727 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:25.727+0000 7fa799e84740 1 -- 172.21.15.47:0/3850735704 wait complete. 2024-03-21T10:31:25.727 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:25.728+0000 7fa799e84740 1 -- start start 2024-03-21T10:31:25.727 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:25.728+0000 7fa799e84740 1 Processor -- start 2024-03-21T10:31:25.729 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:31:25 smithi100 ceph-mon[35193]: pgmap v186: 129 pgs: 129 active+clean; 277 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 8 op/s 2024-03-21T10:31:25.732 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:25.729+0000 7fa799e84740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558f95890080 0x558f95898f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:31:25.733 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:25.729+0000 7fa799e84740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558f958909a0 0x558f95896310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:31:25.733 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:25.729+0000 7fa799e84740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x558f95899580 0x558f95896920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:31:25.733 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:25.729+0000 7fa799e84740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x558f9589c310 con 0x558f95899580 2024-03-21T10:31:25.733 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:25.729+0000 7fa799e84740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x558f957b0eb0 con 0x558f95890080 2024-03-21T10:31:25.733 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:25.729+0000 7fa799e84740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x558f9589c490 con 0x558f958909a0 2024-03-21T10:31:25.733 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:25.729+0000 7fa799e84740 10 client.0 _ll_register_callbacks cb 0x558f957a5150 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T10:31:25.733 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:25.729+0000 7fa787400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x558f95899580 0x558f95896920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:31:25.733 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:25.729+0000 7fa787400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558f95890080 0x558f95898f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:31:25.733 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:25.729+0000 7fa787400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558f95890080 0x558f95898f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:58894/0 (socket says 172.21.15.47:58894) 2024-03-21T10:31:25.733 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:25.729+0000 7fa787400640 1 -- 172.21.15.47:0/90241856 learned_addr learned my addr 172.21.15.47:0/90241856 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:31:25.734 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:25.729+0000 7fa78fe00640 1 --2- 172.21.15.47:0/90241856 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558f958909a0 0x558f95896310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:31:25.734 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:25.730+0000 7fa787400640 1 -- 172.21.15.47:0/90241856 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558f958909a0 msgr2=0x558f95896310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:31:25.734 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:25.730+0000 7fa787400640 1 --2- 172.21.15.47:0/90241856 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558f958909a0 0x558f95896310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:31:25.734 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:25.730+0000 7fa787400640 1 -- 172.21.15.47:0/90241856 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x558f95899580 msgr2=0x558f95896920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:31:25.734 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:25.730+0000 7fa787400640 1 --2- 172.21.15.47:0/90241856 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x558f95899580 0x558f95896920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:31:25.734 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:25.730+0000 7fa787400640 1 -- 172.21.15.47:0/90241856 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x558f95631f70 con 0x558f95890080 2024-03-21T10:31:25.734 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:25.730+0000 7fa787400640 1 --2- 172.21.15.47:0/90241856 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558f95890080 0x558f95898f80 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fa77802c410 tx=0x7fa77802bc20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=19ef382be8e51cf9 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:31:25.734 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:25.730+0000 7fa77e000640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-21T10:31:25.734 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:25.731+0000 7fa77e000640 1 -- 172.21.15.47:0/90241856 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa778055060 con 0x558f95890080 2024-03-21T10:31:25.734 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:25.731+0000 7fa799e84740 1 -- 172.21.15.47:0/90241856 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fa77802cc80 con 0x558f95890080 2024-03-21T10:31:25.734 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:25.731+0000 7fa77e000640 1 -- 172.21.15.47:0/90241856 <== mon.2 v2:172.21.15.100:3300/0 2 ==== osd_map(97..97 src has 1..97) v4 ==== 9474+0+0 (secure 0 0 0) 0x7fa7780041c0 con 0x558f95890080 2024-03-21T10:31:25.734 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:25.731+0000 7fa77e000640 1 -- 172.21.15.47:0/90241856 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa77802caa0 con 0x558f95890080 2024-03-21T10:31:25.734 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:25.731+0000 7fa799e84740 10 client.37325 fetch_fsmap learned FSMap version 32 2024-03-21T10:31:25.735 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:25.731+0000 7fa799e84740 1 -- 172.21.15.47:0/90241856 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x558f954c9170 con 0x558f95890080 2024-03-21T10:31:25.735 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:25.732+0000 7fa77e000640 1 -- 172.21.15.47:0/90241856 <== mon.2 v2:172.21.15.100:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fa77802cc80 con 0x558f95890080 2024-03-21T10:31:25.735 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:25.732+0000 7fa77e000640 1 -- 172.21.15.47:0/90241856 <== mon.2 v2:172.21.15.100:3300/0 5 ==== fsmap(e 32) v1 ==== 7942+0+0 (secure 0 0 0) 0x7fa77804a070 con 0x558f95890080 2024-03-21T10:31:25.735 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:25.732+0000 7fa799e84740 10 client.37325 fetch_fsmap finished waiting for FSMap version 32 2024-03-21T10:31:25.735 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:25.732+0000 7fa799e84740 10 client.37325 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-21T10:31:25.735 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:25.732+0000 7fa799e84740 20 client.37325 populate_metadata read hostname 'smithi047' 2024-03-21T10:31:25.735 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:25.732+0000 7fa799e84740 1 --2- 172.21.15.47:0/90241856 >> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] conn(0x558f958a9480 0x558f958ab870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:31:25.735 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:25.732+0000 7fa799e84740 4 client.37325 mds_command: new command op to 34535 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T10:31:25.735 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:25.732+0000 7fa799e84740 1 -- 172.21.15.47:0/90241856 --> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x558f958023f0 con 0x558f958a9480 2024-03-21T10:31:25.735 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:25.732+0000 7fa78fe00640 1 --2- 172.21.15.47:0/90241856 >> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] conn(0x558f958a9480 0x558f958ab870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:31:25.735 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:25.733+0000 7fa78fe00640 1 --2- 172.21.15.47:0/90241856 >> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] conn(0x558f958a9480 0x558f958ab870 crc :-1 s=READY pgs=157 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f24c72aea8cc3c9d server_cookie=bfda2fc61b8734dd in_seq=0 out_seq=0 2024-03-21T10:31:25.735 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:25.733+0000 7fa77e000640 10 client.37325 ms_handle_connect on v2:172.21.15.100:6838/4173168440 2024-03-21T10:31:25.736 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:25.734+0000 7fa77e000640 1 -- 172.21.15.47:0/90241856 <== mds.0 v2:172.21.15.100:6838/4173168440 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x558f958023f0 con 0x558f958a9480 2024-03-21T10:31:25.736 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:25.734+0000 7fa77e000640 10 client.37325 handle_command_reply: tid=0 2024-03-21T10:31:25.737 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:25.735+0000 7fa799e84740 1 client.37325 shutdown 2024-03-21T10:31:25.737 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:25.735+0000 7fa799e84740 1 -- 172.21.15.47:0/90241856 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558f95890080 msgr2=0x558f95898f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:31:25.737 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:25.735+0000 7fa799e84740 1 --2- 172.21.15.47:0/90241856 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558f95890080 0x558f95898f80 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fa77802c410 tx=0x7fa77802bc20 comp rx=0 tx=0).stop 2024-03-21T10:31:25.737 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:25.735+0000 7fa799e84740 1 -- 172.21.15.47:0/90241856 shutdown_connections 2024-03-21T10:31:25.737 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:25.735+0000 7fa799e84740 1 --2- 172.21.15.47:0/90241856 >> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] conn(0x558f958a9480 0x558f958ab870 crc :-1 s=READY pgs=157 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:31:25.737 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:25.735+0000 7fa799e84740 1 --2- 172.21.15.47:0/90241856 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x558f95899580 0x558f95896920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:31:25.737 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:25.735+0000 7fa77e000640 0 client.37325 ms_handle_reset on v2:172.21.15.100:6838/4173168440 2024-03-21T10:31:25.737 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:25.735+0000 7fa799e84740 1 --2- 172.21.15.47:0/90241856 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558f958909a0 0x558f95896310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:31:25.737 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:25.735+0000 7fa799e84740 1 --2- 172.21.15.47:0/90241856 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558f95890080 0x558f95898f80 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:31:25.737 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:25.735+0000 7fa799e84740 1 -- 172.21.15.47:0/90241856 >> 172.21.15.47:0/90241856 conn(0x558f9588a7b0 msgr2=0x558f9588c500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:31:25.737 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:25.735+0000 7fa799e84740 1 -- 172.21.15.47:0/90241856 shutdown_connections 2024-03-21T10:31:25.737 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:25.735+0000 7fa799e84740 1 -- 172.21.15.47:0/90241856 wait complete. 2024-03-21T10:31:25.737 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:25.735+0000 7fa799e84740 20 client.37325 trim_cache size 0 max 16384 2024-03-21T10:31:25.738 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:31:25 smithi157 ceph-mon[35581]: pgmap v186: 129 pgs: 129 active+clean; 277 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 8 op/s 2024-03-21T10:31:25.756 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:25.756+0000 7fa799e84740 1 Processor -- start 2024-03-21T10:31:25.756 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:25.756+0000 7fa799e84740 1 -- start start 2024-03-21T10:31:25.757 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:25.756+0000 7fa799e84740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558f95890080 0x558f95898a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:31:25.757 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:25.756+0000 7fa799e84740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x558f958909a0 0x558f95898f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:31:25.757 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:25.756+0000 7fa799e84740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558f95899580 0x558f958935b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:31:25.757 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:25.756+0000 7fa799e84740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7901b0a70 con 0x558f958909a0 2024-03-21T10:31:25.757 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:25.756+0000 7fa799e84740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7901b23f0 con 0x558f95890080 2024-03-21T10:31:25.757 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:25.756+0000 7fa799e84740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7901b10d0 con 0x558f95899580 2024-03-21T10:31:25.757 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:25.757+0000 7fa787400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558f95890080 0x558f95898a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:31:25.757 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:25.757+0000 7fa78fe00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x558f958909a0 0x558f95898f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:31:25.757 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:25.757+0000 7fa787400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558f95890080 0x558f95898a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:58902/0 (socket says 172.21.15.47:58902) 2024-03-21T10:31:25.757 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:25.757+0000 7fa787400640 1 -- 172.21.15.47:0/1837147800 learned_addr learned my addr 172.21.15.47:0/1837147800 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:31:25.757 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:25.757+0000 7fa787400640 1 --2- 172.21.15.47:0/1837147800 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558f95899580 0x558f958935b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:31:25.757 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:25.757+0000 7fa78fe00640 1 -- 172.21.15.47:0/1837147800 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558f95899580 msgr2=0x558f958935b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:31:25.757 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:25.757+0000 7fa78fe00640 1 --2- 172.21.15.47:0/1837147800 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558f95899580 0x558f958935b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:31:25.758 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:25.757+0000 7fa78fe00640 1 -- 172.21.15.47:0/1837147800 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558f95890080 msgr2=0x558f95898a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:31:25.758 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:25.757+0000 7fa78fe00640 1 --2- 172.21.15.47:0/1837147800 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558f95890080 0x558f95898a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:31:25.758 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:25.757+0000 7fa78fe00640 1 -- 172.21.15.47:0/1837147800 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558f9588c190 con 0x558f958909a0 2024-03-21T10:31:25.758 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:25.757+0000 7fa78fe00640 1 --2- 172.21.15.47:0/1837147800 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x558f958909a0 0x558f95898f80 secure :-1 s=READY pgs=962 cs=0 l=1 rev1=1 crypto rx=0x558f9589c460 tx=0x7fa780003eb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4f630f57237d4c2d server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:31:25.758 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:25.758+0000 7fa784c00640 1 -- 172.21.15.47:0/1837147800 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa780024070 con 0x558f958909a0 2024-03-21T10:31:25.758 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:25.758+0000 7fa784c00640 1 -- 172.21.15.47:0/1837147800 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fa780020690 con 0x558f958909a0 2024-03-21T10:31:25.758 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:25.758+0000 7fa784c00640 1 -- 172.21.15.47:0/1837147800 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7800042a0 con 0x558f958909a0 2024-03-21T10:31:25.763 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:25.758+0000 7fa799e84740 1 -- 172.21.15.47:0/1837147800 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x558f958909a0 msgr2=0x558f95898f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:31:25.763 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:25.758+0000 7fa799e84740 1 --2- 172.21.15.47:0/1837147800 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x558f958909a0 0x558f95898f80 secure :-1 s=READY pgs=962 cs=0 l=1 rev1=1 crypto rx=0x558f9589c460 tx=0x7fa780003eb0 comp rx=0 tx=0).stop 2024-03-21T10:31:25.763 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:25.758+0000 7fa799e84740 1 -- 172.21.15.47:0/1837147800 shutdown_connections 2024-03-21T10:31:25.763 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:25.758+0000 7fa799e84740 1 --2- 172.21.15.47:0/1837147800 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558f95899580 0x558f958935b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:31:25.763 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:25.758+0000 7fa799e84740 1 --2- 172.21.15.47:0/1837147800 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x558f958909a0 0x558f95898f80 unknown :-1 s=CLOSED pgs=962 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:31:25.763 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:25.758+0000 7fa799e84740 1 --2- 172.21.15.47:0/1837147800 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558f95890080 0x558f95898a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:31:25.763 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:25.758+0000 7fa799e84740 1 -- 172.21.15.47:0/1837147800 >> 172.21.15.47:0/1837147800 conn(0x558f9588a7b0 msgr2=0x558f958a4f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:31:25.763 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:25.759+0000 7fa799e84740 1 -- 172.21.15.47:0/1837147800 shutdown_connections 2024-03-21T10:31:25.763 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:25.759+0000 7fa799e84740 1 -- 172.21.15.47:0/1837147800 wait complete. 2024-03-21T10:31:25.763 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:25.759+0000 7fa799e84740 1 -- start start 2024-03-21T10:31:25.763 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:25.759+0000 7fa799e84740 1 Processor -- start 2024-03-21T10:31:25.763 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:25.760+0000 7fa799e84740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558f95890080 0x558f958ac590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:31:25.763 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:25.760+0000 7fa799e84740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558f958909a0 0x558f958acad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:31:25.764 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:25.760+0000 7fa799e84740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x558f95899580 0x558f95906200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:31:25.764 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:25.760+0000 7fa799e84740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7901b1d90 con 0x558f95899580 2024-03-21T10:31:25.764 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:25.760+0000 7fa799e84740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7901b1730 con 0x558f95890080 2024-03-21T10:31:25.764 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:25.760+0000 7fa799e84740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7901aec30 con 0x558f958909a0 2024-03-21T10:31:25.764 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:25.760+0000 7fa799e84740 10 client.0 _ll_register_callbacks cb 0x7fa7800096a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T10:31:25.764 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:25.760+0000 7fa787400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x558f95899580 0x558f95906200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:31:25.764 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:25.760+0000 7fa787400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x558f95899580 0x558f95906200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:40306/0 (socket says 172.21.15.47:40306) 2024-03-21T10:31:25.764 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:25.760+0000 7fa787400640 1 -- 172.21.15.47:0/3368773725 learned_addr learned my addr 172.21.15.47:0/3368773725 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:31:25.764 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:25.760+0000 7fa787400640 1 --2- 172.21.15.47:0/3368773725 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558f95890080 0x558f958ac590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:31:25.764 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:25.760+0000 7fa78fe00640 1 --2- 172.21.15.47:0/3368773725 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558f958909a0 0x558f958acad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:31:25.764 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:25.761+0000 7fa787400640 1 -- 172.21.15.47:0/3368773725 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558f958909a0 msgr2=0x558f958acad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:31:25.764 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:25.761+0000 7fa787400640 1 --2- 172.21.15.47:0/3368773725 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558f958909a0 0x558f958acad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:31:25.764 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:25.761+0000 7fa787400640 1 -- 172.21.15.47:0/3368773725 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558f95890080 msgr2=0x558f958ac590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:31:25.765 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:25.761+0000 7fa787400640 1 --2- 172.21.15.47:0/3368773725 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558f95890080 0x558f958ac590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:31:25.765 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:25.761+0000 7fa787400640 1 -- 172.21.15.47:0/3368773725 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x558f959067f0 con 0x558f95899580 2024-03-21T10:31:25.765 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:25.761+0000 7fa787400640 1 --2- 172.21.15.47:0/3368773725 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x558f95899580 0x558f95906200 secure :-1 s=READY pgs=963 cs=0 l=1 rev1=1 crypto rx=0x7fa77802c410 tx=0x7fa77802c320 comp rx=0 tx=0).ready entity=mon.0 client_cookie=699e99b230803809 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:31:25.766 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:25.761+0000 7fa77e000640 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-03-21T10:31:25.766 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:25.761+0000 7fa77e000640 1 -- 172.21.15.47:0/3368773725 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa778055040 con 0x558f95899580 2024-03-21T10:31:25.766 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:25.761+0000 7fa77e000640 1 -- 172.21.15.47:0/3368773725 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(97..97 src has 1..97) v4 ==== 9474+0+0 (secure 0 0 0) 0x7fa7780046c0 con 0x558f95899580 2024-03-21T10:31:25.766 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:25.761+0000 7fa799e84740 1 -- 172.21.15.47:0/3368773725 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fa780020690 con 0x558f95899580 2024-03-21T10:31:25.766 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:25.762+0000 7fa77e000640 1 -- 172.21.15.47:0/3368773725 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa778005650 con 0x558f95899580 2024-03-21T10:31:25.766 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:25.762+0000 7fa799e84740 10 client.17544 fetch_fsmap learned FSMap version 32 2024-03-21T10:31:25.766 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:25.762+0000 7fa799e84740 1 -- 172.21.15.47:0/3368773725 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x558f9590dbd0 con 0x558f95899580 2024-03-21T10:31:25.766 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:25.762+0000 7fa77e000640 1 -- 172.21.15.47:0/3368773725 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fa780020690 con 0x558f95899580 2024-03-21T10:31:25.766 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:25.762+0000 7fa77e000640 1 -- 172.21.15.47:0/3368773725 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 32) v1 ==== 7942+0+0 (secure 0 0 0) 0x7fa77804a070 con 0x558f95899580 2024-03-21T10:31:25.766 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:25.762+0000 7fa799e84740 10 client.17544 fetch_fsmap finished waiting for FSMap version 32 2024-03-21T10:31:25.766 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:25.762+0000 7fa799e84740 10 client.17544 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-21T10:31:25.766 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:25.762+0000 7fa799e84740 20 client.17544 populate_metadata read hostname 'smithi047' 2024-03-21T10:31:25.767 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:25.762+0000 7fa799e84740 1 --2- 172.21.15.47:0/3368773725 >> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] conn(0x558f9590e0e0 0x558f95910500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:31:25.767 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:25.762+0000 7fa799e84740 4 client.17544 mds_command: new command op to 34535 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-03-21T10:31:25.767 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:25.762+0000 7fa799e84740 1 -- 172.21.15.47:0/3368773725 --> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x558f957fc740 con 0x558f9590e0e0 2024-03-21T10:31:25.767 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:25.763+0000 7fa78fe00640 1 --2- 172.21.15.47:0/3368773725 >> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] conn(0x558f9590e0e0 0x558f95910500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:31:25.767 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:25.763+0000 7fa78fe00640 1 --2- 172.21.15.47:0/3368773725 >> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] conn(0x558f9590e0e0 0x558f95910500 crc :-1 s=READY pgs=158 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=89edb334816c03b3 server_cookie=9d95b4b428c106dd in_seq=0 out_seq=0 2024-03-21T10:31:25.767 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:25.763+0000 7fa77e000640 10 client.17544 ms_handle_connect on v2:172.21.15.100:6838/4173168440 2024-03-21T10:31:25.768 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:25.766+0000 7fa77e000640 1 -- 172.21.15.47:0/3368773725 <== mds.0 v2:172.21.15.100:6838/4173168440 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x558f957fc740 con 0x558f9590e0e0 2024-03-21T10:31:25.768 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:25.766+0000 7fa77e000640 10 client.17544 handle_command_reply: tid=0 2024-03-21T10:31:25.768 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:25.766+0000 7fa799e84740 1 client.17544 shutdown 2024-03-21T10:31:25.768 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:25.766+0000 7fa799e84740 1 -- 172.21.15.47:0/3368773725 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x558f95899580 msgr2=0x558f95906200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:31:25.769 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:25.766+0000 7fa799e84740 1 --2- 172.21.15.47:0/3368773725 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x558f95899580 0x558f95906200 secure :-1 s=READY pgs=963 cs=0 l=1 rev1=1 crypto rx=0x7fa77802c410 tx=0x7fa77802c320 comp rx=0 tx=0).stop 2024-03-21T10:31:25.769 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:25.766+0000 7fa799e84740 1 -- 172.21.15.47:0/3368773725 shutdown_connections 2024-03-21T10:31:25.769 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:25.766+0000 7fa799e84740 1 --2- 172.21.15.47:0/3368773725 >> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] conn(0x558f9590e0e0 0x558f95910500 crc :-1 s=READY pgs=158 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:31:25.769 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:25.766+0000 7fa799e84740 1 --2- 172.21.15.47:0/3368773725 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x558f95899580 0x558f95906200 unknown :-1 s=CLOSED pgs=963 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:31:25.769 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:25.766+0000 7fa799e84740 1 --2- 172.21.15.47:0/3368773725 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558f958909a0 0x558f958acad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:31:25.769 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:25.766+0000 7fa799e84740 1 --2- 172.21.15.47:0/3368773725 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558f95890080 0x558f958ac590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:31:25.769 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:25.766+0000 7fa799e84740 1 -- 172.21.15.47:0/3368773725 >> 172.21.15.47:0/3368773725 conn(0x558f9588a7b0 msgr2=0x558f95894ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:31:25.769 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:25.767+0000 7fa77e000640 0 client.17544 ms_handle_reset on v2:172.21.15.100:6838/4173168440 2024-03-21T10:31:25.769 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:25.767+0000 7fa799e84740 1 -- 172.21.15.47:0/3368773725 shutdown_connections 2024-03-21T10:31:25.769 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:25.767+0000 7fa799e84740 1 -- 172.21.15.47:0/3368773725 wait complete. 2024-03-21T10:31:25.769 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:25.767+0000 7fa799e84740 20 client.17544 trim_cache size 0 max 16384 2024-03-21T10:31:25.770 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-03-21T10:31:25.770 INFO:teuthology.orchestra.run.smithi047.stdout: "return_code": 0, 2024-03-21T10:31:25.770 INFO:teuthology.orchestra.run.smithi047.stdout: "scrub_tag": "e5476d02-3f84-4405-b253-9328e8614af9", 2024-03-21T10:31:25.770 INFO:teuthology.orchestra.run.smithi047.stdout: "mode": "asynchronous" 2024-03-21T10:31:25.770 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-03-21T10:31:25.771 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:25.768+0000 7fa797200640 1 -- 172.21.15.47:0/866990308 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7fa7740a3090 msgr2=0x7fa7740a5550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:31:25.771 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:25.768+0000 7fa797200640 1 --2- 172.21.15.47:0/866990308 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7fa7740a3090 0x7fa7740a5550 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fa788098580 tx=0x7fa7880bc040 comp rx=0 tx=0).stop 2024-03-21T10:31:25.771 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:25.768+0000 7fa797200640 1 -- 172.21.15.47:0/866990308 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa790107630 msgr2=0x7fa79019e960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:31:25.771 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:25.768+0000 7fa797200640 1 --2- 172.21.15.47:0/866990308 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa790107630 0x7fa79019e960 secure :-1 s=READY pgs=960 cs=0 l=1 rev1=1 crypto rx=0x7fa78800e970 tx=0x7fa78800ed70 comp rx=0 tx=0).stop 2024-03-21T10:31:25.771 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:25.768+0000 7fa797200640 1 -- 172.21.15.47:0/866990308 shutdown_connections 2024-03-21T10:31:25.771 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:25.768+0000 7fa797200640 1 --2- 172.21.15.47:0/866990308 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa790107630 0x7fa79019e960 unknown :-1 s=CLOSED pgs=960 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:31:25.771 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:25.768+0000 7fa797200640 1 --2- 172.21.15.47:0/866990308 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa790101fe0 0x7fa79019a5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:31:25.771 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:25.768+0000 7fa797200640 1 --2- 172.21.15.47:0/866990308 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7fa7740a3090 0x7fa7740a5550 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:31:25.771 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:25.768+0000 7fa797200640 1 --2- 172.21.15.47:0/866990308 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa790101620 0x7fa79019a0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:31:25.771 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:25.768+0000 7fa797200640 1 -- 172.21.15.47:0/866990308 >> 172.21.15.47:0/866990308 conn(0x7fa7900fd130 msgr2=0x7fa79010cdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:31:25.772 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:25.768+0000 7fa797200640 1 -- 172.21.15.47:0/866990308 shutdown_connections 2024-03-21T10:31:25.772 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:25.768+0000 7fa797200640 1 -- 172.21.15.47:0/866990308 wait complete. 2024-03-21T10:31:25.787 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-21T10:31:25.787 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-21T10:31:26.006 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:26.006+0000 7ff0ffe00640 1 -- 172.21.15.47:0/1318528732 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff100102ff0 msgr2=0x7ff10010b4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:31:26.006 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:26.006+0000 7ff0ffe00640 1 --2- 172.21.15.47:0/1318528732 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff100102ff0 0x7ff10010b4d0 secure :-1 s=READY pgs=964 cs=0 l=1 rev1=1 crypto rx=0x7ff0e8009900 tx=0x7ff0e802bcc0 comp rx=0 tx=0).stop 2024-03-21T10:31:26.006 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:26.007+0000 7ff0ffe00640 1 -- 172.21.15.47:0/1318528732 shutdown_connections 2024-03-21T10:31:26.006 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:26.007+0000 7ff0ffe00640 1 --2- 172.21.15.47:0/1318528732 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff10010baf0 0x7ff10010df90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:31:26.006 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:26.007+0000 7ff0ffe00640 1 --2- 172.21.15.47:0/1318528732 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff100102ff0 0x7ff10010b4d0 unknown :-1 s=CLOSED pgs=964 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:31:26.006 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:26.007+0000 7ff0ffe00640 1 --2- 172.21.15.47:0/1318528732 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff1001026d0 0x7ff100102ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:31:26.006 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:26.007+0000 7ff0ffe00640 1 -- 172.21.15.47:0/1318528732 >> 172.21.15.47:0/1318528732 conn(0x7ff1000fbef0 msgr2=0x7ff1000fe310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:31:26.006 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:26.007+0000 7ff0ffe00640 1 -- 172.21.15.47:0/1318528732 shutdown_connections 2024-03-21T10:31:26.006 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:26.007+0000 7ff0ffe00640 1 -- 172.21.15.47:0/1318528732 wait complete. 2024-03-21T10:31:26.008 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:26.008+0000 7ff0ffe00640 1 Processor -- start 2024-03-21T10:31:26.008 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:26.008+0000 7ff0ffe00640 1 -- start start 2024-03-21T10:31:26.008 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:26.008+0000 7ff0ffe00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff1001026d0 0x7ff10019da70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:31:26.008 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:26.008+0000 7ff0ffe00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff100102ff0 0x7ff10019dfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:31:26.008 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:26.008+0000 7ff0ffe00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff10010baf0 0x7ff100197b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:31:26.008 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:26.008+0000 7ff0ffe00640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1001117d0 con 0x7ff1001026d0 2024-03-21T10:31:26.008 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:26.008+0000 7ff0ffe00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff100111650 con 0x7ff100102ff0 2024-03-21T10:31:26.009 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:26.008+0000 7ff0ffe00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff100111950 con 0x7ff10010baf0 2024-03-21T10:31:26.009 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:26.008+0000 7ff0fea00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff1001026d0 0x7ff10019da70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:31:26.009 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:26.008+0000 7ff0fea00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff1001026d0 0x7ff10019da70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:40328/0 (socket says 172.21.15.47:40328) 2024-03-21T10:31:26.009 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:26.008+0000 7ff0fea00640 1 -- 172.21.15.47:0/117274242 learned_addr learned my addr 172.21.15.47:0/117274242 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:31:26.009 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:26.009+0000 7ff0f7e00640 1 --2- 172.21.15.47:0/117274242 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff100102ff0 0x7ff10019dfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:31:26.009 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:26.009+0000 7ff0ff400640 1 --2- 172.21.15.47:0/117274242 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff10010baf0 0x7ff100197b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:31:26.010 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:26.009+0000 7ff0fea00640 1 -- 172.21.15.47:0/117274242 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff10010baf0 msgr2=0x7ff100197b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:31:26.010 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:26.009+0000 7ff0fea00640 1 --2- 172.21.15.47:0/117274242 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff10010baf0 0x7ff100197b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:31:26.010 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:26.009+0000 7ff0fea00640 1 -- 172.21.15.47:0/117274242 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff100102ff0 msgr2=0x7ff10019dfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:31:26.010 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:26.009+0000 7ff0fea00640 1 --2- 172.21.15.47:0/117274242 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff100102ff0 0x7ff10019dfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:31:26.010 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:26.009+0000 7ff0fea00640 1 -- 172.21.15.47:0/117274242 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1001983f0 con 0x7ff1001026d0 2024-03-21T10:31:26.010 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:26.009+0000 7ff0f7e00640 1 --2- 172.21.15.47:0/117274242 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff100102ff0 0x7ff10019dfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T10:31:26.011 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:26.009+0000 7ff0fea00640 1 --2- 172.21.15.47:0/117274242 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff1001026d0 0x7ff10019da70 secure :-1 s=READY pgs=965 cs=0 l=1 rev1=1 crypto rx=0x7ff0f000d770 tx=0x7ff0f000dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:31:26.012 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:26.010+0000 7ff0f7400640 1 -- 172.21.15.47:0/117274242 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0f0017070 con 0x7ff1001026d0 2024-03-21T10:31:26.012 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:26.010+0000 7ff0f7400640 1 -- 172.21.15.47:0/117274242 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7ff0f00054d0 con 0x7ff1001026d0 2024-03-21T10:31:26.012 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:26.010+0000 7ff0ffe00640 1 -- 172.21.15.47:0/117274242 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff1001986e0 con 0x7ff1001026d0 2024-03-21T10:31:26.012 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:26.010+0000 7ff0f7400640 1 -- 172.21.15.47:0/117274242 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0f0004cf0 con 0x7ff1001026d0 2024-03-21T10:31:26.012 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:26.010+0000 7ff0ffe00640 1 -- 172.21.15.47:0/117274242 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff1001a49f0 con 0x7ff1001026d0 2024-03-21T10:31:26.012 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:26.013+0000 7ff1089df740 1 Processor -- start 2024-03-21T10:31:26.012 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:26.013+0000 7ff1089df740 1 -- start start 2024-03-21T10:31:26.022 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:26.013+0000 7ff0f7400640 1 -- 172.21.15.47:0/117274242 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 27) v1 ==== 137985+0+0 (secure 0 0 0) 0x7ff0f0004e90 con 0x7ff1001026d0 2024-03-21T10:31:26.022 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:26.013+0000 7ff1089df740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x555c864fbdd0 0x555c864fab20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:31:26.022 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:26.014+0000 7ff1089df740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555c864fb0f0 0x555c864fb570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:31:26.023 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:26.014+0000 7ff1089df740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555c86504460 0x555c86506850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:31:26.023 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:26.014+0000 7ff1089df740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x555c8645f250 con 0x555c864fbdd0 2024-03-21T10:31:26.023 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:26.014+0000 7ff1089df740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x555c86407a30 con 0x555c86504460 2024-03-21T10:31:26.023 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:26.014+0000 7ff1089df740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x555c8645b2f0 con 0x555c864fb0f0 2024-03-21T10:31:26.023 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:26.014+0000 7ff0ff400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x555c864fbdd0 0x555c864fab20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:31:26.023 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:26.014+0000 7ff0ff400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x555c864fbdd0 0x555c864fab20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:40342/0 (socket says 172.21.15.47:40342) 2024-03-21T10:31:26.023 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:26.014+0000 7ff0ff400640 1 -- 172.21.15.47:0/3804524757 learned_addr learned my addr 172.21.15.47:0/3804524757 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:31:26.024 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:26.014+0000 7ff0f7e00640 1 --2- 172.21.15.47:0/3804524757 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555c86504460 0x555c86506850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:31:26.024 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:26.014+0000 7ff0fea00640 1 --2- 172.21.15.47:0/3804524757 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555c864fb0f0 0x555c864fb570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:31:26.024 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:26.014+0000 7ff0ff400640 1 -- 172.21.15.47:0/3804524757 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555c864fb0f0 msgr2=0x555c864fb570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:31:26.024 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:26.014+0000 7ff0ff400640 1 --2- 172.21.15.47:0/3804524757 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555c864fb0f0 0x555c864fb570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:31:26.024 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:26.014+0000 7ff0ff400640 1 -- 172.21.15.47:0/3804524757 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555c86504460 msgr2=0x555c86506850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:31:26.024 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:26.014+0000 7ff0ff400640 1 --2- 172.21.15.47:0/3804524757 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555c86504460 0x555c86506850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:31:26.024 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:26.014+0000 7ff0ff400640 1 -- 172.21.15.47:0/3804524757 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555c863f2690 con 0x555c864fbdd0 2024-03-21T10:31:26.024 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:26.015+0000 7ff0ff400640 1 --2- 172.21.15.47:0/3804524757 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x555c864fbdd0 0x555c864fab20 secure :-1 s=READY pgs=966 cs=0 l=1 rev1=1 crypto rx=0x7ff0f8009f40 tx=0x7ff0f8010860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2395ddccf9d2ac0f server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:31:26.025 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:26.015+0000 7ff0efe00640 1 -- 172.21.15.47:0/3804524757 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0f8003b70 con 0x555c864fbdd0 2024-03-21T10:31:26.025 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:26.015+0000 7ff0efe00640 1 -- 172.21.15.47:0/3804524757 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7ff0f8003d10 con 0x555c864fbdd0 2024-03-21T10:31:26.025 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:26.015+0000 7ff0efe00640 1 -- 172.21.15.47:0/3804524757 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0f80119c0 con 0x555c864fbdd0 2024-03-21T10:31:26.025 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:26.015+0000 7ff0f7400640 1 --2- 172.21.15.47:0/117274242 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7ff0e40a3110 0x7ff0e40a55d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:31:26.025 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:26.016+0000 7ff0f7400640 1 -- 172.21.15.47:0/117274242 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 9474+0+0 (secure 0 0 0) 0x7ff0f00cf6a0 con 0x7ff1001026d0 2024-03-21T10:31:26.025 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:26.016+0000 7ff0fea00640 1 --2- 172.21.15.47:0/117274242 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7ff0e40a3110 0x7ff0e40a55d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:31:26.025 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:26.016+0000 7ff1089df740 1 -- 172.21.15.47:0/3804524757 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x555c864fbdd0 msgr2=0x555c864fab20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:31:26.025 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:26.016+0000 7ff1089df740 1 --2- 172.21.15.47:0/3804524757 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x555c864fbdd0 0x555c864fab20 secure :-1 s=READY pgs=966 cs=0 l=1 rev1=1 crypto rx=0x7ff0f8009f40 tx=0x7ff0f8010860 comp rx=0 tx=0).stop 2024-03-21T10:31:26.025 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:26.016+0000 7ff1089df740 1 -- 172.21.15.47:0/3804524757 shutdown_connections 2024-03-21T10:31:26.026 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:26.016+0000 7ff1089df740 1 --2- 172.21.15.47:0/3804524757 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555c86504460 0x555c86506850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:31:26.026 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:26.016+0000 7ff1089df740 1 --2- 172.21.15.47:0/3804524757 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555c864fb0f0 0x555c864fb570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:31:26.026 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:26.016+0000 7ff1089df740 1 --2- 172.21.15.47:0/3804524757 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x555c864fbdd0 0x555c864fab20 unknown :-1 s=CLOSED pgs=966 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:31:26.026 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:26.016+0000 7ff1089df740 1 -- 172.21.15.47:0/3804524757 >> 172.21.15.47:0/3804524757 conn(0x555c864c2200 msgr2=0x555c864f5b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:31:26.026 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:26.016+0000 7ff1089df740 1 -- 172.21.15.47:0/3804524757 shutdown_connections 2024-03-21T10:31:26.026 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:26.016+0000 7ff0fea00640 1 --2- 172.21.15.47:0/117274242 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7ff0e40a3110 0x7ff0e40a55d0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x555c864fc320 tx=0x7ff0f00d5260 comp rx=0 tx=0).ready entity=mgr.15687 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:31:26.026 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:26.016+0000 7ff1089df740 1 -- 172.21.15.47:0/3804524757 wait complete. 2024-03-21T10:31:26.026 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:26.017+0000 7ff1089df740 1 -- start start 2024-03-21T10:31:26.027 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:26.017+0000 7ff1089df740 1 Processor -- start 2024-03-21T10:31:26.027 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:26.018+0000 7ff1089df740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555c864fb0f0 0x555c86503b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:31:26.027 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:26.018+0000 7ff1089df740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555c864fbdd0 0x555c86500ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:31:26.027 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:26.018+0000 7ff1089df740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x555c86504460 0x555c86501400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:31:26.027 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:26.018+0000 7ff1089df740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x555c865072d0 con 0x555c86504460 2024-03-21T10:31:26.027 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:26.018+0000 7ff1089df740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x555c8641ae80 con 0x555c864fb0f0 2024-03-21T10:31:26.027 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:26.018+0000 7ff1089df740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x555c86507450 con 0x555c864fbdd0 2024-03-21T10:31:26.027 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:26.018+0000 7ff1089df740 10 client.0 _ll_register_callbacks cb 0x555c8642d130 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T10:31:26.028 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:26.018+0000 7ff0ff400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x555c86504460 0x555c86501400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:31:26.028 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:26.018+0000 7ff0ff400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x555c86504460 0x555c86501400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:40344/0 (socket says 172.21.15.47:40344) 2024-03-21T10:31:26.028 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:26.018+0000 7ff0ff400640 1 -- 172.21.15.47:0/1170699709 learned_addr learned my addr 172.21.15.47:0/1170699709 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:31:26.028 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:26.018+0000 7ff0ff400640 1 --2- 172.21.15.47:0/1170699709 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555c864fb0f0 0x555c86503b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:31:26.028 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:26.018+0000 7ff0f7e00640 1 --2- 172.21.15.47:0/1170699709 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555c864fbdd0 0x555c86500ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:31:26.028 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:26.018+0000 7ff0ff400640 1 -- 172.21.15.47:0/1170699709 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555c864fbdd0 msgr2=0x555c86500ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:31:26.028 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:26.018+0000 7ff0ff400640 1 --2- 172.21.15.47:0/1170699709 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555c864fbdd0 0x555c86500ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:31:26.028 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:26.018+0000 7ff0ff400640 1 -- 172.21.15.47:0/1170699709 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555c864fb0f0 msgr2=0x555c86503b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:31:26.029 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:26.018+0000 7ff0ff400640 1 --2- 172.21.15.47:0/1170699709 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555c864fb0f0 0x555c86503b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:31:26.029 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:26.018+0000 7ff0ff400640 1 -- 172.21.15.47:0/1170699709 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x555c8623d7d0 con 0x555c86504460 2024-03-21T10:31:26.029 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:26.019+0000 7ff0ff400640 1 --2- 172.21.15.47:0/1170699709 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x555c86504460 0x555c86501400 secure :-1 s=READY pgs=967 cs=0 l=1 rev1=1 crypto rx=0x7ff0f80048a0 tx=0x7ff0f8013040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2aa82b95e2eb0a6a server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:31:26.029 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:26.019+0000 7ff0ecc00640 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-03-21T10:31:26.029 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:26.019+0000 7ff0ecc00640 1 -- 172.21.15.47:0/1170699709 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0f8011740 con 0x555c86504460 2024-03-21T10:31:26.029 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:26.019+0000 7ff0ecc00640 1 -- 172.21.15.47:0/1170699709 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(97..97 src has 1..97) v4 ==== 9474+0+0 (secure 0 0 0) 0x7ff0f80118e0 con 0x555c86504460 2024-03-21T10:31:26.029 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:26.019+0000 7ff1089df740 1 -- 172.21.15.47:0/1170699709 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ff0f8003d10 con 0x555c86504460 2024-03-21T10:31:26.029 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:26.019+0000 7ff0ecc00640 1 -- 172.21.15.47:0/1170699709 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff0f8011ae0 con 0x555c86504460 2024-03-21T10:31:26.029 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:26.019+0000 7ff1089df740 10 client.17556 fetch_fsmap learned FSMap version 32 2024-03-21T10:31:26.030 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:26.019+0000 7ff1089df740 1 -- 172.21.15.47:0/1170699709 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x555c861733f0 con 0x555c86504460 2024-03-21T10:31:26.030 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:26.020+0000 7ff0ecc00640 1 -- 172.21.15.47:0/1170699709 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7ff0f8003d10 con 0x555c86504460 2024-03-21T10:31:26.030 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:26.020+0000 7ff0ecc00640 1 -- 172.21.15.47:0/1170699709 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 32) v1 ==== 7942+0+0 (secure 0 0 0) 0x7ff0f8029040 con 0x555c86504460 2024-03-21T10:31:26.030 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:26.020+0000 7ff1089df740 10 client.17556 fetch_fsmap finished waiting for FSMap version 32 2024-03-21T10:31:26.030 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:26.020+0000 7ff1089df740 10 client.17556 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-21T10:31:26.030 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:26.020+0000 7ff1089df740 20 client.17556 populate_metadata read hostname 'smithi047' 2024-03-21T10:31:26.030 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:26.020+0000 7ff1089df740 1 --2- 172.21.15.47:0/1170699709 >> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] conn(0x555c86514280 0x555c86516670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:31:26.031 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:26.020+0000 7ff1089df740 4 client.17556 mds_command: new command op to 34535 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T10:31:26.031 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:26.020+0000 7ff1089df740 1 -- 172.21.15.47:0/1170699709 --> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x555c8646c790 con 0x555c86514280 2024-03-21T10:31:26.031 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:26.021+0000 7ff0f7e00640 1 --2- 172.21.15.47:0/1170699709 >> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] conn(0x555c86514280 0x555c86516670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:31:26.031 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:26.021+0000 7ff0f7e00640 1 --2- 172.21.15.47:0/1170699709 >> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] conn(0x555c86514280 0x555c86516670 crc :-1 s=READY pgs=159 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3644825cbe85078 server_cookie=cfec3850fb1d202e in_seq=0 out_seq=0 2024-03-21T10:31:26.031 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:26.021+0000 7ff0ecc00640 10 client.17556 ms_handle_connect on v2:172.21.15.100:6838/4173168440 2024-03-21T10:31:26.031 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:26.022+0000 7ff0ecc00640 1 -- 172.21.15.47:0/1170699709 <== mds.0 v2:172.21.15.100:6838/4173168440 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x555c8646c790 con 0x555c86514280 2024-03-21T10:31:26.031 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:26.022+0000 7ff0ecc00640 10 client.17556 handle_command_reply: tid=0 2024-03-21T10:31:26.031 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:26.022+0000 7ff1089df740 1 client.17556 shutdown 2024-03-21T10:31:26.031 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:26.023+0000 7ff1089df740 1 -- 172.21.15.47:0/1170699709 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x555c86504460 msgr2=0x555c86501400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:31:26.031 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:26.023+0000 7ff1089df740 1 --2- 172.21.15.47:0/1170699709 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x555c86504460 0x555c86501400 secure :-1 s=READY pgs=967 cs=0 l=1 rev1=1 crypto rx=0x7ff0f80048a0 tx=0x7ff0f8013040 comp rx=0 tx=0).stop 2024-03-21T10:31:26.031 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:26.023+0000 7ff1089df740 1 -- 172.21.15.47:0/1170699709 shutdown_connections 2024-03-21T10:31:26.032 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:26.023+0000 7ff1089df740 1 --2- 172.21.15.47:0/1170699709 >> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] conn(0x555c86514280 0x555c86516670 crc :-1 s=READY pgs=159 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:31:26.032 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:26.023+0000 7ff1089df740 1 --2- 172.21.15.47:0/1170699709 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x555c86504460 0x555c86501400 unknown :-1 s=CLOSED pgs=967 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:31:26.032 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:26.023+0000 7ff1089df740 1 --2- 172.21.15.47:0/1170699709 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555c864fbdd0 0x555c86500ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:31:26.032 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:26.023+0000 7ff1089df740 1 --2- 172.21.15.47:0/1170699709 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555c864fb0f0 0x555c86503b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:31:26.032 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:26.023+0000 7ff1089df740 1 -- 172.21.15.47:0/1170699709 >> 172.21.15.47:0/1170699709 conn(0x555c864c2200 msgr2=0x555c864f7890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:31:26.032 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:26.023+0000 7ff0ecc00640 0 client.17556 ms_handle_reset on v2:172.21.15.100:6838/4173168440 2024-03-21T10:31:26.032 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:26.023+0000 7ff1089df740 1 -- 172.21.15.47:0/1170699709 shutdown_connections 2024-03-21T10:31:26.032 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:26.023+0000 7ff1089df740 1 -- 172.21.15.47:0/1170699709 wait complete. 2024-03-21T10:31:26.032 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:26.023+0000 7ff1089df740 20 client.17556 trim_cache size 0 max 16384 2024-03-21T10:31:26.046 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:26.047+0000 7ff1089df740 1 Processor -- start 2024-03-21T10:31:26.046 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:26.047+0000 7ff1089df740 1 -- start start 2024-03-21T10:31:26.047 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:26.047+0000 7ff1089df740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555c864fb0f0 0x555c864f77b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:31:26.047 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:26.047+0000 7ff1089df740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555c864fbdd0 0x555c864f4a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:31:26.047 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:26.047+0000 7ff1089df740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x555c86504460 0x555c865035f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:31:26.047 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:26.047+0000 7ff1089df740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1001ae730 con 0x555c86504460 2024-03-21T10:31:26.047 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:26.047+0000 7ff1089df740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1001b00b0 con 0x555c864fb0f0 2024-03-21T10:31:26.047 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:26.048+0000 7ff1089df740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1001aed90 con 0x555c864fbdd0 2024-03-21T10:31:26.049 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:26.048+0000 7ff0ff400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x555c86504460 0x555c865035f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:31:26.049 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:26.048+0000 7ff0ff400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555c864fb0f0 0x555c864f77b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:31:26.049 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:26.048+0000 7ff0ff400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555c864fb0f0 0x555c864f77b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:58958/0 (socket says 172.21.15.47:58958) 2024-03-21T10:31:26.049 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:26.048+0000 7ff0ff400640 1 -- 172.21.15.47:0/3223197078 learned_addr learned my addr 172.21.15.47:0/3223197078 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:31:26.049 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:26.048+0000 7ff0f7e00640 1 --2- 172.21.15.47:0/3223197078 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555c864fbdd0 0x555c864f4a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:31:26.049 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:26.049+0000 7ff0ff400640 1 -- 172.21.15.47:0/3223197078 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555c864fbdd0 msgr2=0x555c864f4a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:31:26.049 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:26.049+0000 7ff0ff400640 1 --2- 172.21.15.47:0/3223197078 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555c864fbdd0 0x555c864f4a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:31:26.049 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:26.049+0000 7ff0ff400640 1 -- 172.21.15.47:0/3223197078 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x555c86504460 msgr2=0x555c865035f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:31:26.049 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:26.049+0000 7ff0ff400640 1 --2- 172.21.15.47:0/3223197078 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x555c86504460 0x555c865035f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:31:26.049 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:26.049+0000 7ff0ff400640 1 -- 172.21.15.47:0/3223197078 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555c861330f0 con 0x555c864fb0f0 2024-03-21T10:31:26.050 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:26.049+0000 7ff0ff400640 1 --2- 172.21.15.47:0/3223197078 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555c864fb0f0 0x555c864f77b0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7ff0f8009730 tx=0x7ff0f8010860 comp rx=0 tx=0).ready entity=mon.2 client_cookie=415329cf70704c70 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:31:26.050 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:26.049+0000 7ff0ef400640 1 -- 172.21.15.47:0/3223197078 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0f8010d60 con 0x555c864fb0f0 2024-03-21T10:31:26.050 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:26.049+0000 7ff0ef400640 1 -- 172.21.15.47:0/3223197078 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7ff0f80046d0 con 0x555c864fb0f0 2024-03-21T10:31:26.050 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:26.049+0000 7ff0ef400640 1 -- 172.21.15.47:0/3223197078 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0f8004c60 con 0x555c864fb0f0 2024-03-21T10:31:26.050 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:26.050+0000 7ff1089df740 1 -- 172.21.15.47:0/3223197078 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555c864fb0f0 msgr2=0x555c864f77b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:31:26.050 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:26.050+0000 7ff1089df740 1 --2- 172.21.15.47:0/3223197078 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555c864fb0f0 0x555c864f77b0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7ff0f8009730 tx=0x7ff0f8010860 comp rx=0 tx=0).stop 2024-03-21T10:31:26.050 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:26.050+0000 7ff1089df740 1 -- 172.21.15.47:0/3223197078 shutdown_connections 2024-03-21T10:31:26.050 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:26.050+0000 7ff1089df740 1 --2- 172.21.15.47:0/3223197078 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x555c86504460 0x555c865035f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:31:26.050 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:26.050+0000 7ff1089df740 1 --2- 172.21.15.47:0/3223197078 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555c864fbdd0 0x555c864f4a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:31:26.050 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:26.050+0000 7ff1089df740 1 --2- 172.21.15.47:0/3223197078 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555c864fb0f0 0x555c864f77b0 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:31:26.050 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:26.050+0000 7ff1089df740 1 -- 172.21.15.47:0/3223197078 >> 172.21.15.47:0/3223197078 conn(0x555c864c2200 msgr2=0x555c864f4330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:31:26.050 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:26.050+0000 7ff1089df740 1 -- 172.21.15.47:0/3223197078 shutdown_connections 2024-03-21T10:31:26.051 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:26.050+0000 7ff1089df740 1 -- 172.21.15.47:0/3223197078 wait complete. 2024-03-21T10:31:26.054 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:26.051+0000 7ff1089df740 1 -- start start 2024-03-21T10:31:26.054 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:26.051+0000 7ff1089df740 1 Processor -- start 2024-03-21T10:31:26.054 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:26.051+0000 7ff1089df740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x555c864fb0f0 0x555c864fd7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:31:26.054 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:26.051+0000 7ff1089df740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555c864fbdd0 0x555c864fdd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:31:26.054 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:26.051+0000 7ff1089df740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555c86504460 0x555c864fe270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:31:26.055 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:26.051+0000 7ff1089df740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1001afa50 con 0x555c864fb0f0 2024-03-21T10:31:26.055 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:26.051+0000 7ff1089df740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1001af3f0 con 0x555c86504460 2024-03-21T10:31:26.055 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:26.051+0000 7ff1089df740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1001ac8f0 con 0x555c864fbdd0 2024-03-21T10:31:26.055 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:26.051+0000 7ff1089df740 10 client.0 _ll_register_callbacks cb 0x7ff100069860 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T10:31:26.055 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:26.052+0000 7ff0ff400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x555c864fb0f0 0x555c864fd7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:31:26.055 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:26.052+0000 7ff0ff400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x555c864fb0f0 0x555c864fd7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:40354/0 (socket says 172.21.15.47:40354) 2024-03-21T10:31:26.055 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:26.052+0000 7ff0ff400640 1 -- 172.21.15.47:0/662758774 learned_addr learned my addr 172.21.15.47:0/662758774 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:31:26.055 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:26.052+0000 7ff0ff400640 1 --2- 172.21.15.47:0/662758774 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555c86504460 0x555c864fe270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:31:26.055 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:26.052+0000 7ff0f7e00640 1 --2- 172.21.15.47:0/662758774 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555c864fbdd0 0x555c864fdd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:31:26.055 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:26.052+0000 7ff0ff400640 1 -- 172.21.15.47:0/662758774 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555c864fbdd0 msgr2=0x555c864fdd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:31:26.055 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:26.052+0000 7ff0ff400640 1 --2- 172.21.15.47:0/662758774 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555c864fbdd0 0x555c864fdd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:31:26.055 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:26.052+0000 7ff0ff400640 1 -- 172.21.15.47:0/662758774 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555c86504460 msgr2=0x555c864fe270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:31:26.055 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:26.052+0000 7ff0ff400640 1 --2- 172.21.15.47:0/662758774 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555c86504460 0x555c864fe270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:31:26.056 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:26.052+0000 7ff0ff400640 1 -- 172.21.15.47:0/662758774 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x555c8656eb60 con 0x555c864fb0f0 2024-03-21T10:31:26.056 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:26.052+0000 7ff0f7e00640 1 --2- 172.21.15.47:0/662758774 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555c864fbdd0 0x555c864fdd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T10:31:26.056 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:26.052+0000 7ff0ff400640 1 --2- 172.21.15.47:0/662758774 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x555c864fb0f0 0x555c864fd7f0 secure :-1 s=READY pgs=968 cs=0 l=1 rev1=1 crypto rx=0x7ff0f800a010 tx=0x7ff0f8009280 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3fc855f9f3e88cfd server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:31:26.056 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:26.052+0000 7ff0ecc00640 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-03-21T10:31:26.056 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:26.053+0000 7ff0ecc00640 1 -- 172.21.15.47:0/662758774 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0f8011500 con 0x555c864fb0f0 2024-03-21T10:31:26.056 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:26.053+0000 7ff0ecc00640 1 -- 172.21.15.47:0/662758774 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(97..97 src has 1..97) v4 ==== 9474+0+0 (secure 0 0 0) 0x7ff0f80116a0 con 0x555c864fb0f0 2024-03-21T10:31:26.056 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:26.053+0000 7ff1089df740 1 -- 172.21.15.47:0/662758774 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ff0f80046d0 con 0x555c864fb0f0 2024-03-21T10:31:26.056 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:26.053+0000 7ff0ecc00640 1 -- 172.21.15.47:0/662758774 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff0f80118a0 con 0x555c864fb0f0 2024-03-21T10:31:26.056 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:26.053+0000 7ff1089df740 10 client.17562 fetch_fsmap learned FSMap version 32 2024-03-21T10:31:26.056 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:26.053+0000 7ff1089df740 1 -- 172.21.15.47:0/662758774 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x555c8656fe50 con 0x555c864fb0f0 2024-03-21T10:31:26.056 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:26.053+0000 7ff0ecc00640 1 -- 172.21.15.47:0/662758774 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7ff0f80046d0 con 0x555c864fb0f0 2024-03-21T10:31:26.056 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:26.053+0000 7ff0ecc00640 1 -- 172.21.15.47:0/662758774 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 32) v1 ==== 7942+0+0 (secure 0 0 0) 0x7ff0f8005040 con 0x555c864fb0f0 2024-03-21T10:31:26.056 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:26.053+0000 7ff1089df740 10 client.17562 fetch_fsmap finished waiting for FSMap version 32 2024-03-21T10:31:26.056 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:26.053+0000 7ff1089df740 10 client.17562 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-21T10:31:26.057 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:26.053+0000 7ff1089df740 20 client.17562 populate_metadata read hostname 'smithi047' 2024-03-21T10:31:26.057 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:26.054+0000 7ff1089df740 1 --2- 172.21.15.47:0/662758774 >> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] conn(0x555c86570440 0x555c86572860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:31:26.057 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:26.054+0000 7ff1089df740 4 client.17562 mds_command: new command op to 34535 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-21T10:31:26.057 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:26.054+0000 7ff1089df740 1 -- 172.21.15.47:0/662758774 --> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x555c86466ae0 con 0x555c86570440 2024-03-21T10:31:26.057 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:26.054+0000 7ff0f7e00640 1 --2- 172.21.15.47:0/662758774 >> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] conn(0x555c86570440 0x555c86572860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:31:26.057 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:26.054+0000 7ff0f7e00640 1 --2- 172.21.15.47:0/662758774 >> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] conn(0x555c86570440 0x555c86572860 crc :-1 s=READY pgs=160 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9634c4416bb0a50d server_cookie=57ab0fa360749049 in_seq=0 out_seq=0 2024-03-21T10:31:26.057 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:26.054+0000 7ff0ecc00640 10 client.17562 ms_handle_connect on v2:172.21.15.100:6838/4173168440 2024-03-21T10:31:26.057 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:26.055+0000 7ff0ecc00640 1 -- 172.21.15.47:0/662758774 <== mds.0 v2:172.21.15.100:6838/4173168440 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x555c86466ae0 con 0x555c86570440 2024-03-21T10:31:26.058 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:26.055+0000 7ff0ecc00640 10 client.17562 handle_command_reply: tid=0 2024-03-21T10:31:26.058 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:26.055+0000 7ff1089df740 1 client.17562 shutdown 2024-03-21T10:31:26.058 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:26.055+0000 7ff1089df740 1 -- 172.21.15.47:0/662758774 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x555c864fb0f0 msgr2=0x555c864fd7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:31:26.058 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:26.055+0000 7ff1089df740 1 --2- 172.21.15.47:0/662758774 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x555c864fb0f0 0x555c864fd7f0 secure :-1 s=READY pgs=968 cs=0 l=1 rev1=1 crypto rx=0x7ff0f800a010 tx=0x7ff0f8009280 comp rx=0 tx=0).stop 2024-03-21T10:31:26.058 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:26.055+0000 7ff1089df740 1 -- 172.21.15.47:0/662758774 shutdown_connections 2024-03-21T10:31:26.058 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:26.055+0000 7ff1089df740 1 --2- 172.21.15.47:0/662758774 >> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] conn(0x555c86570440 0x555c86572860 crc :-1 s=READY pgs=160 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:31:26.058 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:26.055+0000 7ff1089df740 1 --2- 172.21.15.47:0/662758774 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555c86504460 0x555c864fe270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:31:26.058 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:26.055+0000 7ff1089df740 1 --2- 172.21.15.47:0/662758774 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555c864fbdd0 0x555c864fdd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:31:26.058 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:26.055+0000 7ff1089df740 1 --2- 172.21.15.47:0/662758774 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x555c864fb0f0 0x555c864fd7f0 unknown :-1 s=CLOSED pgs=968 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:31:26.059 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:26.055+0000 7ff0ecc00640 0 client.17562 ms_handle_reset on v2:172.21.15.100:6838/4173168440 2024-03-21T10:31:26.059 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:26.055+0000 7ff1089df740 1 -- 172.21.15.47:0/662758774 >> 172.21.15.47:0/662758774 conn(0x555c864c2200 msgr2=0x555c864f4330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:31:26.059 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:26.055+0000 7ff1089df740 1 -- 172.21.15.47:0/662758774 shutdown_connections 2024-03-21T10:31:26.059 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:26.055+0000 7ff1089df740 1 -- 172.21.15.47:0/662758774 wait complete. 2024-03-21T10:31:26.059 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:26.055+0000 7ff1089df740 20 client.17562 trim_cache size 0 max 16384 2024-03-21T10:31:26.059 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-03-21T10:31:26.060 INFO:teuthology.orchestra.run.smithi047.stdout: "status": "IDLE (waiting for more scrubs)", 2024-03-21T10:31:26.060 INFO:teuthology.orchestra.run.smithi047.stdout: "scrubs": { 2024-03-21T10:31:26.060 INFO:teuthology.orchestra.run.smithi047.stdout: "e5476d02-3f84-4405-b253-9328e8614af9": { 2024-03-21T10:31:26.060 INFO:teuthology.orchestra.run.smithi047.stdout: "path": "/", 2024-03-21T10:31:26.060 INFO:teuthology.orchestra.run.smithi047.stdout: "tag": "e5476d02-3f84-4405-b253-9328e8614af9", 2024-03-21T10:31:26.060 INFO:teuthology.orchestra.run.smithi047.stdout: "options": "recursive,force" 2024-03-21T10:31:26.060 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-21T10:31:26.060 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-21T10:31:26.060 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-03-21T10:31:26.061 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:26.056+0000 7ff0ffe00640 1 -- 172.21.15.47:0/117274242 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7ff0e40a3110 msgr2=0x7ff0e40a55d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:31:26.061 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:26.056+0000 7ff0ffe00640 1 --2- 172.21.15.47:0/117274242 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7ff0e40a3110 0x7ff0e40a55d0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x555c864fc320 tx=0x7ff0f00d5260 comp rx=0 tx=0).stop 2024-03-21T10:31:26.061 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:26.056+0000 7ff0ffe00640 1 -- 172.21.15.47:0/117274242 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff1001026d0 msgr2=0x7ff10019da70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:31:26.061 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:26.056+0000 7ff0ffe00640 1 --2- 172.21.15.47:0/117274242 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff1001026d0 0x7ff10019da70 secure :-1 s=READY pgs=965 cs=0 l=1 rev1=1 crypto rx=0x7ff0f000d770 tx=0x7ff0f000dc30 comp rx=0 tx=0).stop 2024-03-21T10:31:26.061 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:26.056+0000 7ff0ffe00640 1 -- 172.21.15.47:0/117274242 shutdown_connections 2024-03-21T10:31:26.061 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:26.056+0000 7ff0ffe00640 1 --2- 172.21.15.47:0/117274242 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff10010baf0 0x7ff100197b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:31:26.061 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:26.056+0000 7ff0ffe00640 1 --2- 172.21.15.47:0/117274242 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7ff0e40a3110 0x7ff0e40a55d0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:31:26.061 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:26.056+0000 7ff0ffe00640 1 --2- 172.21.15.47:0/117274242 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff100102ff0 0x7ff10019dfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:31:26.062 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:26.056+0000 7ff0ffe00640 1 --2- 172.21.15.47:0/117274242 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff1001026d0 0x7ff10019da70 unknown :-1 s=CLOSED pgs=965 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:31:26.062 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:26.056+0000 7ff0ffe00640 1 -- 172.21.15.47:0/117274242 >> 172.21.15.47:0/117274242 conn(0x7ff1000fbef0 msgr2=0x7ff1000fcc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:31:26.062 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:26.057+0000 7ff0ffe00640 1 -- 172.21.15.47:0/117274242 shutdown_connections 2024-03-21T10:31:26.062 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:26.057+0000 7ff0ffe00640 1 -- 172.21.15.47:0/117274242 wait complete. 2024-03-21T10:31:26.073 INFO:tasks.cephfs.filesystem:scrub status for tag:e5476d02-3f84-4405-b253-9328e8614af9 - {'path': '/', 'tag': 'e5476d02-3f84-4405-b253-9328e8614af9', 'options': 'recursive,force'} 2024-03-21T10:31:27.712 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:31:27 smithi047 ceph-mon[31547]: scrub queued for path: / 2024-03-21T10:31:27.712 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:31:27 smithi047 ceph-mon[31547]: scrub summary: idle+waiting paths [/] 2024-03-21T10:31:27.712 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:31:27 smithi047 ceph-mon[31547]: scrub summary: active paths [/] 2024-03-21T10:31:27.712 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:31:27 smithi047 ceph-mon[31547]: scrub summary: idle+waiting paths [/] 2024-03-21T10:31:27.712 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:31:27 smithi047 ceph-mon[31547]: pgmap v187: 129 pgs: 129 active+clean; 277 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 12 op/s 2024-03-21T10:31:27.727 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:31:27 smithi100 ceph-mon[35193]: scrub queued for path: / 2024-03-21T10:31:27.727 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:31:27 smithi100 ceph-mon[35193]: scrub summary: idle+waiting paths [/] 2024-03-21T10:31:27.727 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:31:27 smithi100 ceph-mon[35193]: scrub summary: active paths [/] 2024-03-21T10:31:27.727 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:31:27 smithi100 ceph-mon[35193]: scrub summary: idle+waiting paths [/] 2024-03-21T10:31:27.727 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:31:27 smithi100 ceph-mon[35193]: pgmap v187: 129 pgs: 129 active+clean; 277 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 12 op/s 2024-03-21T10:31:27.738 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:31:27 smithi157 ceph-mon[35581]: scrub queued for path: / 2024-03-21T10:31:27.738 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:31:27 smithi157 ceph-mon[35581]: scrub summary: idle+waiting paths [/] 2024-03-21T10:31:27.738 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:31:27 smithi157 ceph-mon[35581]: scrub summary: active paths [/] 2024-03-21T10:31:27.738 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:31:27 smithi157 ceph-mon[35581]: scrub summary: idle+waiting paths [/] 2024-03-21T10:31:27.738 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:31:27 smithi157 ceph-mon[35581]: pgmap v187: 129 pgs: 129 active+clean; 277 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 12 op/s 2024-03-21T10:31:29.727 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:31:29 smithi100 ceph-mon[35193]: pgmap v188: 129 pgs: 129 active+clean; 277 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 9 op/s 2024-03-21T10:31:29.727 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:31:29 smithi100 ceph-mon[35193]: scrub summary: idle 2024-03-21T10:31:29.737 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:31:29 smithi157 ceph-mon[35581]: pgmap v188: 129 pgs: 129 active+clean; 277 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 9 op/s 2024-03-21T10:31:29.737 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:31:29 smithi157 ceph-mon[35581]: scrub summary: idle 2024-03-21T10:31:29.961 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:31:29 smithi047 ceph-mon[31547]: pgmap v188: 129 pgs: 129 active+clean; 277 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 9 op/s 2024-03-21T10:31:29.962 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:31:29 smithi047 ceph-mon[31547]: scrub summary: idle 2024-03-21T10:31:31.727 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:31:31 smithi100 ceph-mon[35193]: pgmap v189: 129 pgs: 129 active+clean; 277 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 9 op/s 2024-03-21T10:31:31.737 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:31:31 smithi157 ceph-mon[35581]: pgmap v189: 129 pgs: 129 active+clean; 277 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 9 op/s 2024-03-21T10:31:31.961 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:31:31 smithi047 ceph-mon[31547]: pgmap v189: 129 pgs: 129 active+clean; 277 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 9 op/s 2024-03-21T10:31:33.737 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:31:33 smithi157 ceph-mon[35581]: pgmap v190: 129 pgs: 129 active+clean; 277 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 13 op/s 2024-03-21T10:31:33.961 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:31:33 smithi047 ceph-mon[31547]: pgmap v190: 129 pgs: 129 active+clean; 277 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 13 op/s 2024-03-21T10:31:33.977 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:31:33 smithi100 ceph-mon[35193]: pgmap v190: 129 pgs: 129 active+clean; 277 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 13 op/s 2024-03-21T10:31:35.737 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:31:35 smithi157 ceph-mon[35581]: pgmap v191: 129 pgs: 129 active+clean; 277 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 9 op/s 2024-03-21T10:31:35.961 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:31:35 smithi047 ceph-mon[31547]: pgmap v191: 129 pgs: 129 active+clean; 277 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 9 op/s 2024-03-21T10:31:35.977 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:31:35 smithi100 ceph-mon[35193]: pgmap v191: 129 pgs: 129 active+clean; 277 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 9 op/s 2024-03-21T10:31:37.961 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:31:37 smithi047 ceph-mon[31547]: pgmap v192: 129 pgs: 129 active+clean; 277 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 12 op/s 2024-03-21T10:31:37.977 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:31:37 smithi100 ceph-mon[35193]: pgmap v192: 129 pgs: 129 active+clean; 277 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 12 op/s 2024-03-21T10:31:37.987 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:31:37 smithi157 ceph-mon[35581]: pgmap v192: 129 pgs: 129 active+clean; 277 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 12 op/s 2024-03-21T10:31:39.962 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:31:39 smithi047 ceph-mon[31547]: pgmap v193: 129 pgs: 129 active+clean; 277 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 8 op/s 2024-03-21T10:31:39.977 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:31:39 smithi100 ceph-mon[35193]: pgmap v193: 129 pgs: 129 active+clean; 277 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 8 op/s 2024-03-21T10:31:39.987 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:31:39 smithi157 ceph-mon[35581]: pgmap v193: 129 pgs: 129 active+clean; 277 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 8 op/s 2024-03-21T10:31:40.961 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:31:40 smithi047 ceph-mon[31547]: from='mgr.15687 172.21.15.47:0/8730661' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T10:31:40.977 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:31:40 smithi100 ceph-mon[35193]: from='mgr.15687 172.21.15.47:0/8730661' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T10:31:40.987 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:31:40 smithi157 ceph-mon[35581]: from='mgr.15687 172.21.15.47:0/8730661' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T10:31:41.889 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:31:41 smithi157 ceph-mon[35581]: pgmap v194: 129 pgs: 129 active+clean; 277 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-21T10:31:41.889 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:31:41 smithi157 ceph-mon[35581]: from='mgr.15687 172.21.15.47:0/8730661' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T10:31:41.889 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:31:41 smithi157 ceph-mon[35581]: from='mgr.15687 172.21.15.47:0/8730661' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T10:31:41.889 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:31:41 smithi157 ceph-mon[35581]: from='mgr.15687 172.21.15.47:0/8730661' entity='mgr.x' 2024-03-21T10:31:41.962 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:31:41 smithi047 ceph-mon[31547]: pgmap v194: 129 pgs: 129 active+clean; 277 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-21T10:31:41.962 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:31:41 smithi047 ceph-mon[31547]: from='mgr.15687 172.21.15.47:0/8730661' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T10:31:41.962 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:31:41 smithi047 ceph-mon[31547]: from='mgr.15687 172.21.15.47:0/8730661' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T10:31:41.962 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:31:41 smithi047 ceph-mon[31547]: from='mgr.15687 172.21.15.47:0/8730661' entity='mgr.x' 2024-03-21T10:31:41.977 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:31:41 smithi100 ceph-mon[35193]: pgmap v194: 129 pgs: 129 active+clean; 277 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-21T10:31:41.977 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:31:41 smithi100 ceph-mon[35193]: from='mgr.15687 172.21.15.47:0/8730661' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T10:31:41.977 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:31:41 smithi100 ceph-mon[35193]: from='mgr.15687 172.21.15.47:0/8730661' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T10:31:41.977 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:31:41 smithi100 ceph-mon[35193]: from='mgr.15687 172.21.15.47:0/8730661' entity='mgr.x' 2024-03-21T10:31:43.961 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:31:43 smithi047 ceph-mon[31547]: pgmap v195: 129 pgs: 129 active+clean; 277 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-03-21T10:31:43.977 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:31:43 smithi100 ceph-mon[35193]: pgmap v195: 129 pgs: 129 active+clean; 277 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-03-21T10:31:43.987 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:31:43 smithi157 ceph-mon[35581]: pgmap v195: 129 pgs: 129 active+clean; 277 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-03-21T10:31:45.962 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:31:45 smithi047 ceph-mon[31547]: pgmap v196: 129 pgs: 129 active+clean; 277 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-21T10:31:45.977 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:31:45 smithi100 ceph-mon[35193]: pgmap v196: 129 pgs: 129 active+clean; 277 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-21T10:31:45.987 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:31:45 smithi157 ceph-mon[35581]: pgmap v196: 129 pgs: 129 active+clean; 277 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-21T10:31:47.961 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:31:47 smithi047 ceph-mon[31547]: pgmap v197: 129 pgs: 129 active+clean; 277 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 12 op/s 2024-03-21T10:31:47.977 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:31:47 smithi100 ceph-mon[35193]: pgmap v197: 129 pgs: 129 active+clean; 277 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 12 op/s 2024-03-21T10:31:47.987 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:31:47 smithi157 ceph-mon[35581]: pgmap v197: 129 pgs: 129 active+clean; 277 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 12 op/s 2024-03-21T10:31:49.961 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:31:49 smithi047 ceph-mon[31547]: pgmap v198: 129 pgs: 129 active+clean; 277 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-03-21T10:31:49.977 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:31:49 smithi100 ceph-mon[35193]: pgmap v198: 129 pgs: 129 active+clean; 277 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-03-21T10:31:49.987 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:31:49 smithi157 ceph-mon[35581]: pgmap v198: 129 pgs: 129 active+clean; 277 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-03-21T10:31:51.961 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:31:51 smithi047 ceph-mon[31547]: pgmap v199: 129 pgs: 129 active+clean; 277 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-21T10:31:51.977 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:31:51 smithi100 ceph-mon[35193]: pgmap v199: 129 pgs: 129 active+clean; 277 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-21T10:31:51.987 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:31:51 smithi157 ceph-mon[35581]: pgmap v199: 129 pgs: 129 active+clean; 277 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-21T10:31:53.961 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:31:53 smithi047 ceph-mon[31547]: pgmap v200: 129 pgs: 129 active+clean; 277 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-03-21T10:31:53.977 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:31:53 smithi100 ceph-mon[35193]: pgmap v200: 129 pgs: 129 active+clean; 277 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-03-21T10:31:53.987 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:31:53 smithi157 ceph-mon[35581]: pgmap v200: 129 pgs: 129 active+clean; 277 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-03-21T10:31:55.961 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:31:55 smithi047 ceph-mon[31547]: pgmap v201: 129 pgs: 129 active+clean; 277 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-21T10:31:55.977 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:31:55 smithi100 ceph-mon[35193]: pgmap v201: 129 pgs: 129 active+clean; 277 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-21T10:31:55.989 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:31:55 smithi157 ceph-mon[35581]: pgmap v201: 129 pgs: 129 active+clean; 277 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-21T10:31:56.073 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-21T10:31:56.074 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-21T10:31:56.295 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:56.296+0000 7f38d2800640 1 -- 172.21.15.47:0/1114507975 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f38cc100ef0 msgr2=0x7f38cc10d3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:31:56.296 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:56.296+0000 7f38d2800640 1 --2- 172.21.15.47:0/1114507975 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f38cc100ef0 0x7f38cc10d3b0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f38c4009c10 tx=0x7f38c402bef0 comp rx=0 tx=0).stop 2024-03-21T10:31:56.296 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:56.297+0000 7f38d2800640 1 -- 172.21.15.47:0/1114507975 shutdown_connections 2024-03-21T10:31:56.296 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:56.297+0000 7f38d2800640 1 --2- 172.21.15.47:0/1114507975 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f38cc100ef0 0x7f38cc10d3b0 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:31:56.296 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:56.297+0000 7f38d2800640 1 --2- 172.21.15.47:0/1114507975 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f38cc100530 0x7f38cc1009b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:31:56.296 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:56.297+0000 7f38d2800640 1 --2- 172.21.15.47:0/1114507975 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f38cc1065f0 0x7f38cc1069d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:31:56.296 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:56.297+0000 7f38d2800640 1 -- 172.21.15.47:0/1114507975 >> 172.21.15.47:0/1114507975 conn(0x7f38cc0776a0 msgr2=0x7f38cc077ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:31:56.296 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:56.297+0000 7f38d2800640 1 -- 172.21.15.47:0/1114507975 shutdown_connections 2024-03-21T10:31:56.296 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:56.297+0000 7f38d2800640 1 -- 172.21.15.47:0/1114507975 wait complete. 2024-03-21T10:31:56.298 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:56.298+0000 7f38d2800640 1 Processor -- start 2024-03-21T10:31:56.298 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:56.298+0000 7f38d2800640 1 -- start start 2024-03-21T10:31:56.298 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:56.298+0000 7f38d2800640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f38cc100530 0x7f38cc199ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:31:56.298 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:56.298+0000 7f38d2800640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f38cc100ef0 0x7f38cc19a420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:31:56.298 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:56.298+0000 7f38d2800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f38cc1065f0 0x7f38cc19e7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:31:56.298 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:56.298+0000 7f38d2800640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38cc110760 con 0x7f38cc100ef0 2024-03-21T10:31:56.298 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:56.298+0000 7f38d2800640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38cc1105e0 con 0x7f38cc1065f0 2024-03-21T10:31:56.298 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:56.298+0000 7f38d2800640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38cc1108e0 con 0x7f38cc100530 2024-03-21T10:31:56.299 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:56.299+0000 7f38cbe00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f38cc1065f0 0x7f38cc19e7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:31:56.299 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:56.299+0000 7f38cbe00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f38cc1065f0 0x7f38cc19e7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:38934/0 (socket says 172.21.15.47:38934) 2024-03-21T10:31:56.299 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:56.299+0000 7f38cbe00640 1 -- 172.21.15.47:0/439652624 learned_addr learned my addr 172.21.15.47:0/439652624 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:31:56.299 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:56.299+0000 7f38caa00640 1 --2- 172.21.15.47:0/439652624 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f38cc100ef0 0x7f38cc19a420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:31:56.299 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:56.299+0000 7f38cb400640 1 --2- 172.21.15.47:0/439652624 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f38cc100530 0x7f38cc199ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:31:56.299 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:56.299+0000 7f38cbe00640 1 -- 172.21.15.47:0/439652624 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f38cc100530 msgr2=0x7f38cc199ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:31:56.299 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:56.299+0000 7f38cbe00640 1 --2- 172.21.15.47:0/439652624 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f38cc100530 0x7f38cc199ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:31:56.299 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:56.299+0000 7f38cbe00640 1 -- 172.21.15.47:0/439652624 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f38cc100ef0 msgr2=0x7f38cc19a420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:31:56.299 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:56.299+0000 7f38cbe00640 1 --2- 172.21.15.47:0/439652624 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f38cc100ef0 0x7f38cc19a420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:31:56.299 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:56.299+0000 7f38cbe00640 1 -- 172.21.15.47:0/439652624 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38cc19eed0 con 0x7f38cc1065f0 2024-03-21T10:31:56.299 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:56.299+0000 7f38caa00640 1 --2- 172.21.15.47:0/439652624 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f38cc100ef0 0x7f38cc19a420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T10:31:56.299 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:56.300+0000 7f38cbe00640 1 --2- 172.21.15.47:0/439652624 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f38cc1065f0 0x7f38cc19e7a0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f38c4034fd0 tx=0x7f38c4033c90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:31:56.300 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:56.300+0000 7f38c3e00640 1 -- 172.21.15.47:0/439652624 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38c4043020 con 0x7f38cc1065f0 2024-03-21T10:31:56.300 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:56.300+0000 7f38d2800640 1 -- 172.21.15.47:0/439652624 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f38cc19f160 con 0x7f38cc1065f0 2024-03-21T10:31:56.300 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:56.300+0000 7f38c3e00640 1 -- 172.21.15.47:0/439652624 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f38c402ca60 con 0x7f38cc1065f0 2024-03-21T10:31:56.300 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:56.300+0000 7f38c3e00640 1 -- 172.21.15.47:0/439652624 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38c4034690 con 0x7f38cc1065f0 2024-03-21T10:31:56.301 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:56.300+0000 7f38d2800640 1 -- 172.21.15.47:0/439652624 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f38cc1a6a50 con 0x7f38cc1065f0 2024-03-21T10:31:56.302 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:56.303+0000 7f38d5609740 1 Processor -- start 2024-03-21T10:31:56.302 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:56.304+0000 7f38c3e00640 1 -- 172.21.15.47:0/439652624 <== mon.2 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 27) v1 ==== 137985+0+0 (secure 0 0 0) 0x7f38c4046020 con 0x7f38cc1065f0 2024-03-21T10:31:56.302 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:56.304+0000 7f38d5609740 1 -- start start 2024-03-21T10:31:56.304 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:56.304+0000 7f38d5609740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x556dd11f2c00 0x556dd11f1950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:31:56.304 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:56.304+0000 7f38d5609740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556dd11f1f20 0x556dd11f23a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:31:56.304 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:56.304+0000 7f38d5609740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556dd11fb290 0x556dd11fd680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:31:56.304 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:56.304+0000 7f38d5609740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x556dd1156250 con 0x556dd11f2c00 2024-03-21T10:31:56.304 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:56.304+0000 7f38d5609740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x556dd10fea30 con 0x556dd11f1f20 2024-03-21T10:31:56.304 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:56.304+0000 7f38d5609740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x556dd11522f0 con 0x556dd11fb290 2024-03-21T10:31:56.304 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:56.305+0000 7f38caa00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x556dd11f2c00 0x556dd11f1950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:31:56.304 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:56.305+0000 7f38caa00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x556dd11f2c00 0x556dd11f1950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:57604/0 (socket says 172.21.15.47:57604) 2024-03-21T10:31:56.304 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:56.305+0000 7f38caa00640 1 -- 172.21.15.47:0/540445983 learned_addr learned my addr 172.21.15.47:0/540445983 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:31:56.304 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:56.305+0000 7f38cb400640 1 --2- 172.21.15.47:0/540445983 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556dd11f1f20 0x556dd11f23a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:31:56.304 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:56.305+0000 7f38caa00640 1 -- 172.21.15.47:0/540445983 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556dd11fb290 msgr2=0x556dd11fd680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:31:56.304 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:56.305+0000 7f38caa00640 1 --2- 172.21.15.47:0/540445983 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556dd11fb290 0x556dd11fd680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:31:56.305 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:56.305+0000 7f38caa00640 1 -- 172.21.15.47:0/540445983 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556dd11f1f20 msgr2=0x556dd11f23a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:31:56.305 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:56.305+0000 7f38caa00640 1 --2- 172.21.15.47:0/540445983 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556dd11f1f20 0x556dd11f23a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:31:56.305 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:56.305+0000 7f38caa00640 1 -- 172.21.15.47:0/540445983 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556dd10e9690 con 0x556dd11f2c00 2024-03-21T10:31:56.305 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:56.305+0000 7f38c3e00640 1 --2- 172.21.15.47:0/439652624 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f38b00a3090 0x7f38b00a5550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:31:56.307 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:56.306+0000 7f38caa00640 1 --2- 172.21.15.47:0/540445983 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x556dd11f2c00 0x556dd11f1950 secure :-1 s=READY pgs=969 cs=0 l=1 rev1=1 crypto rx=0x7f38b8008cf0 tx=0x7f38b8010740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8e47662ea9efb26d server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:31:56.307 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:56.306+0000 7f38c3e00640 1 -- 172.21.15.47:0/439652624 <== mon.2 v2:172.21.15.100:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 9474+0+0 (secure 0 0 0) 0x7f38c40f2210 con 0x7f38cc1065f0 2024-03-21T10:31:56.308 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:56.306+0000 7f38cb400640 1 --2- 172.21.15.47:0/439652624 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f38b00a3090 0x7f38b00a5550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:31:56.308 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:56.306+0000 7f38c0c00640 1 -- 172.21.15.47:0/540445983 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38b8016070 con 0x556dd11f2c00 2024-03-21T10:31:56.308 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:56.306+0000 7f38c0c00640 1 -- 172.21.15.47:0/540445983 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f38b80056d0 con 0x556dd11f2c00 2024-03-21T10:31:56.308 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:56.306+0000 7f38c0c00640 1 -- 172.21.15.47:0/540445983 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38b8012680 con 0x556dd11f2c00 2024-03-21T10:31:56.308 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:56.307+0000 7f38d5609740 1 -- 172.21.15.47:0/540445983 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x556dd11f2c00 msgr2=0x556dd11f1950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:31:56.308 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:56.307+0000 7f38d5609740 1 --2- 172.21.15.47:0/540445983 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x556dd11f2c00 0x556dd11f1950 secure :-1 s=READY pgs=969 cs=0 l=1 rev1=1 crypto rx=0x7f38b8008cf0 tx=0x7f38b8010740 comp rx=0 tx=0).stop 2024-03-21T10:31:56.308 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:56.307+0000 7f38cb400640 1 --2- 172.21.15.47:0/439652624 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f38b00a3090 0x7f38b00a5550 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x556dd11f3150 tx=0x7f38bc009260 comp rx=0 tx=0).ready entity=mgr.15687 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:31:56.308 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:56.307+0000 7f38d5609740 1 -- 172.21.15.47:0/540445983 shutdown_connections 2024-03-21T10:31:56.308 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:56.307+0000 7f38d5609740 1 --2- 172.21.15.47:0/540445983 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556dd11fb290 0x556dd11fd680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:31:56.308 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:56.307+0000 7f38d5609740 1 --2- 172.21.15.47:0/540445983 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556dd11f1f20 0x556dd11f23a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:31:56.308 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:56.307+0000 7f38d5609740 1 --2- 172.21.15.47:0/540445983 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x556dd11f2c00 0x556dd11f1950 unknown :-1 s=CLOSED pgs=969 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:31:56.308 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:56.307+0000 7f38d5609740 1 -- 172.21.15.47:0/540445983 >> 172.21.15.47:0/540445983 conn(0x556dd11b9200 msgr2=0x556dd11ec970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:31:56.308 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:56.307+0000 7f38d5609740 1 -- 172.21.15.47:0/540445983 shutdown_connections 2024-03-21T10:31:56.309 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:56.307+0000 7f38d5609740 1 -- 172.21.15.47:0/540445983 wait complete. 2024-03-21T10:31:56.309 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:56.307+0000 7f38d5609740 1 -- start start 2024-03-21T10:31:56.309 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:56.308+0000 7f38d5609740 1 Processor -- start 2024-03-21T10:31:56.309 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:56.308+0000 7f38d5609740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556dd11f1f20 0x556dd11fa8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:31:56.309 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:56.308+0000 7f38d5609740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556dd11f2c00 0x556dd11f7c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:31:56.309 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:56.308+0000 7f38d5609740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x556dd11fb290 0x556dd11f81c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:31:56.309 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:56.308+0000 7f38d5609740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x556dd11fe100 con 0x556dd11fb290 2024-03-21T10:31:56.309 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:56.308+0000 7f38d5609740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x556dd1111e80 con 0x556dd11f2c00 2024-03-21T10:31:56.309 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:56.308+0000 7f38d5609740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x556dd11fe280 con 0x556dd11f1f20 2024-03-21T10:31:56.309 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:56.308+0000 7f38d5609740 10 client.0 _ll_register_callbacks cb 0x556dd1124130 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T10:31:56.309 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:56.308+0000 7f38cbe00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x556dd11fb290 0x556dd11f81c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:31:56.309 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:56.309+0000 7f38cbe00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x556dd11fb290 0x556dd11f81c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:57608/0 (socket says 172.21.15.47:57608) 2024-03-21T10:31:56.309 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:56.309+0000 7f38cbe00640 1 -- 172.21.15.47:0/769059267 learned_addr learned my addr 172.21.15.47:0/769059267 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:31:56.309 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:56.309+0000 7f38cb400640 1 --2- 172.21.15.47:0/769059267 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556dd11f1f20 0x556dd11fa8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:31:56.310 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:56.309+0000 7f38caa00640 1 --2- 172.21.15.47:0/769059267 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556dd11f2c00 0x556dd11f7c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:31:56.310 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:56.309+0000 7f38cbe00640 1 -- 172.21.15.47:0/769059267 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556dd11f1f20 msgr2=0x556dd11fa8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:31:56.310 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:56.309+0000 7f38cbe00640 1 --2- 172.21.15.47:0/769059267 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556dd11f1f20 0x556dd11fa8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:31:56.310 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:56.309+0000 7f38cbe00640 1 -- 172.21.15.47:0/769059267 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556dd11f2c00 msgr2=0x556dd11f7c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:31:56.310 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:56.309+0000 7f38cbe00640 1 --2- 172.21.15.47:0/769059267 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556dd11f2c00 0x556dd11f7c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:31:56.310 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:56.309+0000 7f38cbe00640 1 -- 172.21.15.47:0/769059267 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x556dd0f347d0 con 0x556dd11fb290 2024-03-21T10:31:56.310 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:56.309+0000 7f38cb400640 1 --2- 172.21.15.47:0/769059267 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556dd11f1f20 0x556dd11fa8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T10:31:56.311 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:56.309+0000 7f38cbe00640 1 --2- 172.21.15.47:0/769059267 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x556dd11fb290 0x556dd11f81c0 secure :-1 s=READY pgs=970 cs=0 l=1 rev1=1 crypto rx=0x7f38c4003cc0 tx=0x7f38c40bdd10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=565ea517e5d86021 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:31:56.311 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:56.309+0000 7f38b5600640 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-03-21T10:31:56.312 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:56.310+0000 7f38b5600640 1 -- 172.21.15.47:0/769059267 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38c40b8e60 con 0x556dd11fb290 2024-03-21T10:31:56.312 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:56.310+0000 7f38b5600640 1 -- 172.21.15.47:0/769059267 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(97..97 src has 1..97) v4 ==== 9474+0+0 (secure 0 0 0) 0x7f38c40ba960 con 0x556dd11fb290 2024-03-21T10:31:56.312 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:56.310+0000 7f38d5609740 1 -- 172.21.15.47:0/769059267 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f38b80056d0 con 0x556dd11fb290 2024-03-21T10:31:56.312 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:56.310+0000 7f38b5600640 1 -- 172.21.15.47:0/769059267 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f38c40be2f0 con 0x556dd11fb290 2024-03-21T10:31:56.312 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:56.310+0000 7f38d5609740 10 client.17574 fetch_fsmap learned FSMap version 32 2024-03-21T10:31:56.312 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:56.310+0000 7f38d5609740 1 -- 172.21.15.47:0/769059267 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x556dd0e6a3f0 con 0x556dd11fb290 2024-03-21T10:31:56.312 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:56.310+0000 7f38b5600640 1 -- 172.21.15.47:0/769059267 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f38b80056d0 con 0x556dd11fb290 2024-03-21T10:31:56.312 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:56.310+0000 7f38b5600640 1 -- 172.21.15.47:0/769059267 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 32) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f38c40bd740 con 0x556dd11fb290 2024-03-21T10:31:56.312 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:56.310+0000 7f38d5609740 10 client.17574 fetch_fsmap finished waiting for FSMap version 32 2024-03-21T10:31:56.312 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:56.310+0000 7f38d5609740 10 client.17574 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-21T10:31:56.312 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:56.310+0000 7f38d5609740 20 client.17574 populate_metadata read hostname 'smithi047' 2024-03-21T10:31:56.312 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:56.310+0000 7f38d5609740 1 --2- 172.21.15.47:0/769059267 >> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] conn(0x556dd120afe0 0x556dd120d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:31:56.312 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:56.310+0000 7f38d5609740 4 client.17574 mds_command: new command op to 34535 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T10:31:56.312 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:56.310+0000 7f38d5609740 1 -- 172.21.15.47:0/769059267 --> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x556dd1163790 con 0x556dd120afe0 2024-03-21T10:31:56.313 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:56.311+0000 7f38cb400640 1 --2- 172.21.15.47:0/769059267 >> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] conn(0x556dd120afe0 0x556dd120d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:31:56.313 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:56.312+0000 7f38cb400640 1 --2- 172.21.15.47:0/769059267 >> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] conn(0x556dd120afe0 0x556dd120d3f0 crc :-1 s=READY pgs=161 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b73ceaf4a3da12f8 server_cookie=100df8bbca750c in_seq=0 out_seq=0 2024-03-21T10:31:56.313 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:56.312+0000 7f38b5600640 10 client.17574 ms_handle_connect on v2:172.21.15.100:6838/4173168440 2024-03-21T10:31:56.314 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:56.313+0000 7f38b5600640 1 -- 172.21.15.47:0/769059267 <== mds.0 v2:172.21.15.100:6838/4173168440 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x556dd1163790 con 0x556dd120afe0 2024-03-21T10:31:56.314 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:56.313+0000 7f38b5600640 10 client.17574 handle_command_reply: tid=0 2024-03-21T10:31:56.314 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:56.313+0000 7f38d5609740 1 client.17574 shutdown 2024-03-21T10:31:56.314 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:56.314+0000 7f38d5609740 1 -- 172.21.15.47:0/769059267 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x556dd11fb290 msgr2=0x556dd11f81c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:31:56.314 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:56.314+0000 7f38d5609740 1 --2- 172.21.15.47:0/769059267 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x556dd11fb290 0x556dd11f81c0 secure :-1 s=READY pgs=970 cs=0 l=1 rev1=1 crypto rx=0x7f38c4003cc0 tx=0x7f38c40bdd10 comp rx=0 tx=0).stop 2024-03-21T10:31:56.314 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:56.314+0000 7f38d5609740 1 -- 172.21.15.47:0/769059267 shutdown_connections 2024-03-21T10:31:56.314 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:56.314+0000 7f38d5609740 1 --2- 172.21.15.47:0/769059267 >> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] conn(0x556dd120afe0 0x556dd120d3f0 crc :-1 s=READY pgs=161 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:31:56.314 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:56.314+0000 7f38d5609740 1 --2- 172.21.15.47:0/769059267 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x556dd11fb290 0x556dd11f81c0 unknown :-1 s=CLOSED pgs=970 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:31:56.314 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:56.314+0000 7f38d5609740 1 --2- 172.21.15.47:0/769059267 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556dd11f2c00 0x556dd11f7c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:31:56.314 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:56.314+0000 7f38d5609740 1 --2- 172.21.15.47:0/769059267 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556dd11f1f20 0x556dd11fa8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:31:56.315 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:56.314+0000 7f38d5609740 1 -- 172.21.15.47:0/769059267 >> 172.21.15.47:0/769059267 conn(0x556dd11b9200 msgr2=0x556dd11ede30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:31:56.315 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:56.314+0000 7f38b5600640 0 client.17574 ms_handle_reset on v2:172.21.15.100:6838/4173168440 2024-03-21T10:31:56.315 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:56.314+0000 7f38d5609740 1 -- 172.21.15.47:0/769059267 shutdown_connections 2024-03-21T10:31:56.315 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:56.314+0000 7f38d5609740 1 -- 172.21.15.47:0/769059267 wait complete. 2024-03-21T10:31:56.315 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:56.314+0000 7f38d5609740 20 client.17574 trim_cache size 0 max 16384 2024-03-21T10:31:56.345 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:56.345+0000 7f38d5609740 1 Processor -- start 2024-03-21T10:31:56.345 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:56.346+0000 7f38d5609740 1 -- start start 2024-03-21T10:31:56.345 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:56.346+0000 7f38d5609740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556dd11f1f20 0x556dd11f6c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:31:56.346 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:56.346+0000 7f38d5609740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556dd11f2c00 0x556dd11fa3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:31:56.346 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:56.346+0000 7f38d5609740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x556dd11fb290 0x556dd11fa8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:31:56.346 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:56.346+0000 7f38d5609740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38cc1b0950 con 0x556dd11fb290 2024-03-21T10:31:56.346 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:56.346+0000 7f38d5609740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38cc1b22d0 con 0x556dd11f2c00 2024-03-21T10:31:56.346 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:56.346+0000 7f38d5609740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38cc1b0fb0 con 0x556dd11f1f20 2024-03-21T10:31:56.346 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:56.346+0000 7f38cbe00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x556dd11fb290 0x556dd11fa8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:31:56.346 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:56.346+0000 7f38caa00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556dd11f2c00 0x556dd11fa3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:31:56.346 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:56.346+0000 7f38cbe00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x556dd11fb290 0x556dd11fa8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:57616/0 (socket says 172.21.15.47:57616) 2024-03-21T10:31:56.346 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:56.346+0000 7f38cbe00640 1 -- 172.21.15.47:0/3163255794 learned_addr learned my addr 172.21.15.47:0/3163255794 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:31:56.347 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:56.347+0000 7f38cb400640 1 --2- 172.21.15.47:0/3163255794 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556dd11f1f20 0x556dd11f6c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:31:56.352 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:56.347+0000 7f38cbe00640 1 -- 172.21.15.47:0/3163255794 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556dd11f1f20 msgr2=0x556dd11f6c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:31:56.352 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:56.347+0000 7f38cbe00640 1 --2- 172.21.15.47:0/3163255794 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556dd11f1f20 0x556dd11f6c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:31:56.352 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:56.347+0000 7f38cbe00640 1 -- 172.21.15.47:0/3163255794 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556dd11f2c00 msgr2=0x556dd11fa3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:31:56.352 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:56.347+0000 7f38cbe00640 1 --2- 172.21.15.47:0/3163255794 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556dd11f2c00 0x556dd11fa3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:31:56.352 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:56.347+0000 7f38cbe00640 1 -- 172.21.15.47:0/3163255794 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556dd0e2a0f0 con 0x556dd11fb290 2024-03-21T10:31:56.352 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:56.347+0000 7f38caa00640 1 --2- 172.21.15.47:0/3163255794 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556dd11f2c00 0x556dd11fa3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T10:31:56.353 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:56.347+0000 7f38cbe00640 1 --2- 172.21.15.47:0/3163255794 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x556dd11fb290 0x556dd11fa8f0 secure :-1 s=READY pgs=971 cs=0 l=1 rev1=1 crypto rx=0x7f38c4033950 tx=0x7f38c40bdd80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e06eba74096ef9e1 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:31:56.353 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:56.348+0000 7f38b7e00640 1 -- 172.21.15.47:0/3163255794 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38c40b8e60 con 0x556dd11fb290 2024-03-21T10:31:56.353 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:56.348+0000 7f38b7e00640 1 -- 172.21.15.47:0/3163255794 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f38c4087020 con 0x556dd11fb290 2024-03-21T10:31:56.353 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:56.348+0000 7f38b7e00640 1 -- 172.21.15.47:0/3163255794 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38c40c2120 con 0x556dd11fb290 2024-03-21T10:31:56.353 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:56.348+0000 7f38d5609740 1 -- 172.21.15.47:0/3163255794 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x556dd11fb290 msgr2=0x556dd11fa8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:31:56.353 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:56.348+0000 7f38d5609740 1 --2- 172.21.15.47:0/3163255794 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x556dd11fb290 0x556dd11fa8f0 secure :-1 s=READY pgs=971 cs=0 l=1 rev1=1 crypto rx=0x7f38c4033950 tx=0x7f38c40bdd80 comp rx=0 tx=0).stop 2024-03-21T10:31:56.353 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:56.348+0000 7f38d5609740 1 -- 172.21.15.47:0/3163255794 shutdown_connections 2024-03-21T10:31:56.353 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:56.348+0000 7f38d5609740 1 --2- 172.21.15.47:0/3163255794 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x556dd11fb290 0x556dd11fa8f0 unknown :-1 s=CLOSED pgs=971 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:31:56.353 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:56.348+0000 7f38d5609740 1 --2- 172.21.15.47:0/3163255794 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556dd11f2c00 0x556dd11fa3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:31:56.353 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:56.348+0000 7f38d5609740 1 --2- 172.21.15.47:0/3163255794 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556dd11f1f20 0x556dd11f6c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:31:56.353 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:56.348+0000 7f38d5609740 1 -- 172.21.15.47:0/3163255794 >> 172.21.15.47:0/3163255794 conn(0x556dd11b9200 msgr2=0x556dd11f6550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:31:56.354 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:56.349+0000 7f38d5609740 1 -- 172.21.15.47:0/3163255794 shutdown_connections 2024-03-21T10:31:56.354 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:56.349+0000 7f38d5609740 1 -- 172.21.15.47:0/3163255794 wait complete. 2024-03-21T10:31:56.354 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:56.349+0000 7f38d5609740 1 -- start start 2024-03-21T10:31:56.354 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:56.349+0000 7f38d5609740 1 Processor -- start 2024-03-21T10:31:56.354 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:56.350+0000 7f38d5609740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x556dd11f1f20 0x556dd1200a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:31:56.354 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:56.350+0000 7f38d5609740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556dd11f2c00 0x556dd1200fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:31:56.354 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:56.350+0000 7f38d5609740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556dd11fb290 0x556dd1201510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:31:56.354 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:56.350+0000 7f38d5609740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38cc1b1c70 con 0x556dd11f1f20 2024-03-21T10:31:56.354 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:56.350+0000 7f38d5609740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38cc1b1610 con 0x556dd11f2c00 2024-03-21T10:31:56.354 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:56.350+0000 7f38d5609740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38cc1aeb10 con 0x556dd11fb290 2024-03-21T10:31:56.355 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:56.350+0000 7f38d5609740 10 client.0 _ll_register_callbacks cb 0x7f38c4005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T10:31:56.355 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:56.350+0000 7f38cb400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x556dd11f1f20 0x556dd1200a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:31:56.355 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:56.350+0000 7f38cb400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x556dd11f1f20 0x556dd1200a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:57632/0 (socket says 172.21.15.47:57632) 2024-03-21T10:31:56.355 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:56.350+0000 7f38cb400640 1 -- 172.21.15.47:0/1864117163 learned_addr learned my addr 172.21.15.47:0/1864117163 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:31:56.355 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:56.351+0000 7f38caa00640 1 --2- 172.21.15.47:0/1864117163 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556dd11f2c00 0x556dd1200fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:31:56.355 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:56.351+0000 7f38cbe00640 1 --2- 172.21.15.47:0/1864117163 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556dd11fb290 0x556dd1201510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:31:56.356 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:56.351+0000 7f38cb400640 1 -- 172.21.15.47:0/1864117163 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556dd11fb290 msgr2=0x556dd1201510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:31:56.356 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:56.351+0000 7f38cb400640 1 --2- 172.21.15.47:0/1864117163 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556dd11fb290 0x556dd1201510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:31:56.356 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:56.351+0000 7f38cb400640 1 -- 172.21.15.47:0/1864117163 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556dd11f2c00 msgr2=0x556dd1200fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:31:56.356 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:56.351+0000 7f38cb400640 1 --2- 172.21.15.47:0/1864117163 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556dd11f2c00 0x556dd1200fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:31:56.356 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:56.351+0000 7f38cb400640 1 -- 172.21.15.47:0/1864117163 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x556dd12659f0 con 0x556dd11f1f20 2024-03-21T10:31:56.356 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:56.351+0000 7f38caa00640 1 --2- 172.21.15.47:0/1864117163 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556dd11f2c00 0x556dd1200fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T10:31:56.357 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:56.351+0000 7f38cb400640 1 --2- 172.21.15.47:0/1864117163 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x556dd11f1f20 0x556dd1200a90 secure :-1 s=READY pgs=972 cs=0 l=1 rev1=1 crypto rx=0x556dd120dc00 tx=0x7f38bc0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6da4bff87fdd600b server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:31:56.357 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:56.351+0000 7f38b5600640 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-03-21T10:31:56.357 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:56.351+0000 7f38b5600640 1 -- 172.21.15.47:0/1864117163 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38bc011a20 con 0x556dd11f1f20 2024-03-21T10:31:56.357 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:56.352+0000 7f38b5600640 1 -- 172.21.15.47:0/1864117163 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(97..97 src has 1..97) v4 ==== 9474+0+0 (secure 0 0 0) 0x7f38bc011bc0 con 0x556dd11f1f20 2024-03-21T10:31:56.357 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:56.352+0000 7f38d5609740 1 -- 172.21.15.47:0/1864117163 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f38c4087020 con 0x556dd11f1f20 2024-03-21T10:31:56.357 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:56.352+0000 7f38b5600640 1 -- 172.21.15.47:0/1864117163 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f38bc03b050 con 0x556dd11f1f20 2024-03-21T10:31:56.358 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:56.352+0000 7f38d5609740 10 client.17580 fetch_fsmap learned FSMap version 32 2024-03-21T10:31:56.358 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:56.352+0000 7f38d5609740 1 -- 172.21.15.47:0/1864117163 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x556dd1266d40 con 0x556dd11f1f20 2024-03-21T10:31:56.358 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:56.352+0000 7f38b5600640 1 -- 172.21.15.47:0/1864117163 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f38c4087020 con 0x556dd11f1f20 2024-03-21T10:31:56.358 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:56.352+0000 7f38b5600640 1 -- 172.21.15.47:0/1864117163 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 32) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f38bc01d040 con 0x556dd11f1f20 2024-03-21T10:31:56.358 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:56.352+0000 7f38d5609740 10 client.17580 fetch_fsmap finished waiting for FSMap version 32 2024-03-21T10:31:56.358 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:56.352+0000 7f38d5609740 10 client.17580 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-21T10:31:56.358 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:56.352+0000 7f38d5609740 20 client.17580 populate_metadata read hostname 'smithi047' 2024-03-21T10:31:56.358 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:56.352+0000 7f38d5609740 1 --2- 172.21.15.47:0/1864117163 >> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] conn(0x556dd1267420 0x556dd1269840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:31:56.359 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:56.352+0000 7f38d5609740 4 client.17580 mds_command: new command op to 34535 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-21T10:31:56.359 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:56.352+0000 7f38d5609740 1 -- 172.21.15.47:0/1864117163 --> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x556dd115dae0 con 0x556dd1267420 2024-03-21T10:31:56.359 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:56.353+0000 7f38caa00640 1 --2- 172.21.15.47:0/1864117163 >> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] conn(0x556dd1267420 0x556dd1269840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:31:56.361 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:56.354+0000 7f38caa00640 1 --2- 172.21.15.47:0/1864117163 >> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] conn(0x556dd1267420 0x556dd1269840 crc :-1 s=READY pgs=162 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=44bfb71bfa1f8f36 server_cookie=f40a458dcc939182 in_seq=0 out_seq=0 2024-03-21T10:31:56.362 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:56.354+0000 7f38b5600640 10 client.17580 ms_handle_connect on v2:172.21.15.100:6838/4173168440 2024-03-21T10:31:56.362 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:56.354+0000 7f38b5600640 1 -- 172.21.15.47:0/1864117163 <== mds.0 v2:172.21.15.100:6838/4173168440 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x556dd115dae0 con 0x556dd1267420 2024-03-21T10:31:56.362 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:56.354+0000 7f38b5600640 10 client.17580 handle_command_reply: tid=0 2024-03-21T10:31:56.362 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:56.354+0000 7f38d5609740 1 client.17580 shutdown 2024-03-21T10:31:56.362 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:56.354+0000 7f38d5609740 1 -- 172.21.15.47:0/1864117163 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x556dd11f1f20 msgr2=0x556dd1200a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:31:56.362 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:56.354+0000 7f38d5609740 1 --2- 172.21.15.47:0/1864117163 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x556dd11f1f20 0x556dd1200a90 secure :-1 s=READY pgs=972 cs=0 l=1 rev1=1 crypto rx=0x556dd120dc00 tx=0x7f38bc0046c0 comp rx=0 tx=0).stop 2024-03-21T10:31:56.363 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:56.354+0000 7f38d5609740 1 -- 172.21.15.47:0/1864117163 shutdown_connections 2024-03-21T10:31:56.363 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:56.354+0000 7f38d5609740 1 --2- 172.21.15.47:0/1864117163 >> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] conn(0x556dd1267420 0x556dd1269840 crc :-1 s=READY pgs=162 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:31:56.363 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:56.355+0000 7f38d5609740 1 --2- 172.21.15.47:0/1864117163 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556dd11fb290 0x556dd1201510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:31:56.363 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:56.355+0000 7f38d5609740 1 --2- 172.21.15.47:0/1864117163 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556dd11f2c00 0x556dd1200fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:31:56.363 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:56.355+0000 7f38d5609740 1 --2- 172.21.15.47:0/1864117163 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x556dd11f1f20 0x556dd1200a90 unknown :-1 s=CLOSED pgs=972 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:31:56.363 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:56.355+0000 7f38d5609740 1 -- 172.21.15.47:0/1864117163 >> 172.21.15.47:0/1864117163 conn(0x556dd11b9200 msgr2=0x556dd11edd50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:31:56.363 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:56.355+0000 7f38b5600640 0 client.17580 ms_handle_reset on v2:172.21.15.100:6838/4173168440 2024-03-21T10:31:56.364 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:56.355+0000 7f38d5609740 1 -- 172.21.15.47:0/1864117163 shutdown_connections 2024-03-21T10:31:56.364 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:56.355+0000 7f38d5609740 1 -- 172.21.15.47:0/1864117163 wait complete. 2024-03-21T10:31:56.364 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:56.355+0000 7f38d5609740 20 client.17580 trim_cache size 0 max 16384 2024-03-21T10:31:56.364 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-03-21T10:31:56.364 INFO:teuthology.orchestra.run.smithi047.stdout: "status": "no active scrubs running", 2024-03-21T10:31:56.364 INFO:teuthology.orchestra.run.smithi047.stdout: "scrubs": {} 2024-03-21T10:31:56.364 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-03-21T10:31:56.365 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:56.355+0000 7f38d2800640 1 -- 172.21.15.47:0/439652624 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f38b00a3090 msgr2=0x7f38b00a5550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:31:56.365 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:56.355+0000 7f38d2800640 1 --2- 172.21.15.47:0/439652624 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f38b00a3090 0x7f38b00a5550 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x556dd11f3150 tx=0x7f38bc009260 comp rx=0 tx=0).stop 2024-03-21T10:31:56.365 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:56.356+0000 7f38d2800640 1 -- 172.21.15.47:0/439652624 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f38cc1065f0 msgr2=0x7f38cc19e7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:31:56.365 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:56.356+0000 7f38d2800640 1 --2- 172.21.15.47:0/439652624 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f38cc1065f0 0x7f38cc19e7a0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f38c4034fd0 tx=0x7f38c4033c90 comp rx=0 tx=0).stop 2024-03-21T10:31:56.365 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:56.356+0000 7f38d2800640 1 -- 172.21.15.47:0/439652624 shutdown_connections 2024-03-21T10:31:56.365 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:56.356+0000 7f38d2800640 1 --2- 172.21.15.47:0/439652624 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f38b00a3090 0x7f38b00a5550 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:31:56.365 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:56.356+0000 7f38d2800640 1 --2- 172.21.15.47:0/439652624 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f38cc1065f0 0x7f38cc19e7a0 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:31:56.366 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:56.356+0000 7f38d2800640 1 --2- 172.21.15.47:0/439652624 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f38cc100ef0 0x7f38cc19a420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:31:56.366 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:56.356+0000 7f38d2800640 1 --2- 172.21.15.47:0/439652624 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f38cc100530 0x7f38cc199ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:31:56.366 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:56.356+0000 7f38d2800640 1 -- 172.21.15.47:0/439652624 >> 172.21.15.47:0/439652624 conn(0x7f38cc0776a0 msgr2=0x7f38cc068880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:31:56.366 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:56.356+0000 7f38d2800640 1 -- 172.21.15.47:0/439652624 shutdown_connections 2024-03-21T10:31:56.366 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:56.356+0000 7f38d2800640 1 -- 172.21.15.47:0/439652624 wait complete. 2024-03-21T10:31:56.373 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-03-21T10:31:56.374 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-21T10:31:56.599 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:56.599+0000 7f2e53200640 1 -- 172.21.15.47:0/3243874002 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2e54101e20 msgr2=0x7f2e5410e330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:31:56.600 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:56.599+0000 7f2e53200640 1 --2- 172.21.15.47:0/3243874002 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2e54101e20 0x7f2e5410e330 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f2e4c009ca0 tx=0x7f2e4c02c100 comp rx=0 tx=0).stop 2024-03-21T10:31:56.600 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:56.599+0000 7f2e53200640 1 -- 172.21.15.47:0/3243874002 shutdown_connections 2024-03-21T10:31:56.600 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:56.599+0000 7f2e53200640 1 --2- 172.21.15.47:0/3243874002 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2e54101e20 0x7f2e5410e330 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:31:56.600 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:56.599+0000 7f2e53200640 1 --2- 172.21.15.47:0/3243874002 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2e54101460 0x7f2e541018e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:31:56.600 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:56.599+0000 7f2e53200640 1 --2- 172.21.15.47:0/3243874002 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2e54107500 0x7f2e541078e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:31:56.600 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:56.599+0000 7f2e53200640 1 -- 172.21.15.47:0/3243874002 >> 172.21.15.47:0/3243874002 conn(0x7f2e540fcf70 msgr2=0x7f2e540ff390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:31:56.601 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:56.599+0000 7f2e53200640 1 -- 172.21.15.47:0/3243874002 shutdown_connections 2024-03-21T10:31:56.601 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:56.599+0000 7f2e53200640 1 -- 172.21.15.47:0/3243874002 wait complete. 2024-03-21T10:31:56.601 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:56.601+0000 7f2e53200640 1 Processor -- start 2024-03-21T10:31:56.601 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:56.601+0000 7f2e53200640 1 -- start start 2024-03-21T10:31:56.605 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:56.601+0000 7f2e53200640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2e54101460 0x7f2e540739a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:31:56.605 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:56.601+0000 7f2e53200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2e54101e20 0x7f2e54073ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:31:56.605 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:56.601+0000 7f2e53200640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2e54107500 0x7f2e54074420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:31:56.606 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:56.601+0000 7f2e53200640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e541116e0 con 0x7f2e54107500 2024-03-21T10:31:56.606 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:56.601+0000 7f2e53200640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e54111560 con 0x7f2e54101e20 2024-03-21T10:31:56.606 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:56.601+0000 7f2e53200640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e54111860 con 0x7f2e54101460 2024-03-21T10:31:56.606 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:56.602+0000 7f2e52800640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2e54107500 0x7f2e54074420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:31:56.606 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:56.602+0000 7f2e4be00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2e54101e20 0x7f2e54073ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:31:56.606 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:56.602+0000 7f2e4be00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2e54101e20 0x7f2e54073ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:38982/0 (socket says 172.21.15.47:38982) 2024-03-21T10:31:56.607 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:56.602+0000 7f2e4be00640 1 -- 172.21.15.47:0/230257554 learned_addr learned my addr 172.21.15.47:0/230257554 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:31:56.607 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:56.602+0000 7f2e51e00640 1 --2- 172.21.15.47:0/230257554 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2e54101460 0x7f2e540739a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:31:56.607 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:56.602+0000 7f2e52800640 1 -- 172.21.15.47:0/230257554 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2e54101460 msgr2=0x7f2e540739a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:31:56.607 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:56.602+0000 7f2e52800640 1 --2- 172.21.15.47:0/230257554 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2e54101460 0x7f2e540739a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:31:56.607 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:56.602+0000 7f2e52800640 1 -- 172.21.15.47:0/230257554 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2e54101e20 msgr2=0x7f2e54073ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:31:56.607 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:56.602+0000 7f2e52800640 1 --2- 172.21.15.47:0/230257554 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2e54101e20 0x7f2e54073ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:31:56.608 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:56.602+0000 7f2e52800640 1 -- 172.21.15.47:0/230257554 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2e541a6140 con 0x7f2e54107500 2024-03-21T10:31:56.608 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:56.602+0000 7f2e4be00640 1 --2- 172.21.15.47:0/230257554 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2e54101e20 0x7f2e54073ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T10:31:56.608 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:56.603+0000 7f2e52800640 1 --2- 172.21.15.47:0/230257554 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2e54107500 0x7f2e54074420 secure :-1 s=READY pgs=973 cs=0 l=1 rev1=1 crypto rx=0x7f2e4c02c710 tx=0x7f2e4c035d60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:31:56.608 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:56.603+0000 7f2e4aa00640 1 -- 172.21.15.47:0/230257554 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e4c039040 con 0x7f2e54107500 2024-03-21T10:31:56.608 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:56.603+0000 7f2e4aa00640 1 -- 172.21.15.47:0/230257554 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f2e4c044070 con 0x7f2e54107500 2024-03-21T10:31:56.609 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:56.603+0000 7f2e53200640 1 -- 172.21.15.47:0/230257554 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2e541a63d0 con 0x7f2e54107500 2024-03-21T10:31:56.609 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:56.603+0000 7f2e4aa00640 1 -- 172.21.15.47:0/230257554 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e4c004330 con 0x7f2e54107500 2024-03-21T10:31:56.609 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:56.603+0000 7f2e53200640 1 -- 172.21.15.47:0/230257554 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2e541a6890 con 0x7f2e54107500 2024-03-21T10:31:56.609 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:56.604+0000 7f2e53200640 1 -- 172.21.15.47:0/230257554 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2e54068800 con 0x7f2e54107500 2024-03-21T10:31:56.610 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:56.607+0000 7f2e4aa00640 1 -- 172.21.15.47:0/230257554 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 27) v1 ==== 137985+0+0 (secure 0 0 0) 0x7f2e4c003970 con 0x7f2e54107500 2024-03-21T10:31:56.611 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:56.608+0000 7f2e4aa00640 1 --2- 172.21.15.47:0/230257554 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f2e380a3090 0x7f2e380a5550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:31:56.612 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:56.608+0000 7f2e4aa00640 1 -- 172.21.15.47:0/230257554 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 9474+0+0 (secure 0 0 0) 0x7f2e4c0f2a70 con 0x7f2e54107500 2024-03-21T10:31:56.612 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:56.608+0000 7f2e51e00640 1 --2- 172.21.15.47:0/230257554 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f2e380a3090 0x7f2e380a5550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:31:56.612 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:56.609+0000 7f2e51e00640 1 --2- 172.21.15.47:0/230257554 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f2e380a3090 0x7f2e380a5550 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f2e44009ae0 tx=0x7f2e44005f90 comp rx=0 tx=0).ready entity=mgr.15687 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:31:56.612 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:56.611+0000 7f2e4aa00640 1 -- 172.21.15.47:0/230257554 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f2e4c0bf2b0 con 0x7f2e54107500 2024-03-21T10:31:56.873 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:56.875+0000 7f2e53200640 1 -- 172.21.15.47:0/230257554 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f2e5402d070 con 0x7f2e54107500 2024-03-21T10:31:56.874 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:56.876+0000 7f2e4aa00640 1 -- 172.21.15.47:0/230257554 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 32 v32) v1 ==== 94+0+11471 (secure 0 0 0) 0x7f2e4c0bebb0 con 0x7f2e54107500 2024-03-21T10:31:56.875 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-21T10:31:56.875 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":32,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14511,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6835/735724037","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6834","nonce":735724037},{"type":"v1","addr":"172.21.15.47:6835","nonce":735724037}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14529,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6837/2248191724","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6836","nonce":2248191724},{"type":"v1","addr":"172.21.15.47:6837","nonce":2248191724}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":32,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-21T10:21:58.329063+0000","modified":"2024-03-21T10:26:17.169892+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":34535,"mds_1":34493,"mds_2":34454,"mds_3":34427,"mds_4":14619},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14604":{"gid":14604,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.47:6839/2881335356","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6838","nonce":2881335356},{"type":"v1","addr":"172.21.15.47:6839","nonce":2881335356}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_14619":{"gid":14619,"name":"a","rank":4,"incarnation":27,"state":"up:active","state_seq":11,"addr":"172.21.15.47:6841/3401578978","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6840","nonce":3401578978},{"type":"v1","addr":"172.21.15.47:6841","nonce":3401578978}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24406":{"gid":24406,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.157:6833/4000010136","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":4000010136},{"type":"v1","addr":"172.21.15.157:6833","nonce":4000010136}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24457":{"gid":24457,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.157:6835/3902868651","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":3902868651},{"type":"v1","addr":"172.21.15.157:6835","nonce":3902868651}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24475":{"gid":24475,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.157:6837/1908087204","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6836","nonce":1908087204},{"type":"v1","addr":"172.21.15.157:6837","nonce":1908087204}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_25729":{"gid":25729,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.157:6839/1998408416","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6838","nonce":1998408416},{"type":"v1","addr":"172.21.15.157:6839","nonce":1998408416}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_34427":{"gid":34427,"name":"k","rank":3,"incarnation":25,"state":"up:active","state_seq":20,"addr":"172.21.15.100:6833/2723302038","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6832","nonce":2723302038},{"type":"v1","addr":"172.21.15.100:6833","nonce":2723302038}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_34454":{"gid":34454,"name":"e","rank":2,"incarnation":22,"state":"up:active","state_seq":17,"addr":"172.21.15.100:6835/2597849957","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":2597849957},{"type":"v1","addr":"172.21.15.100:6835","nonce":2597849957}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_34493":{"gid":34493,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":13,"addr":"172.21.15.100:6837/3497644162","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6836","nonce":3497644162},{"type":"v1","addr":"172.21.15.100:6837","nonce":3497644162}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_34535":{"gid":34535,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.100:6839/4173168440","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6838","nonce":4173168440},{"type":"v1","addr":"172.21.15.100:6839","nonce":4173168440}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":34535,"qdb_cluster":[34427,14619,34454,34535,34493]},"id":1}]} 2024-03-21T10:31:56.875 INFO:teuthology.orchestra.run.smithi047.stderr:dumped fsmap epoch 32 2024-03-21T10:31:56.877 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:56.878+0000 7f2e53200640 1 -- 172.21.15.47:0/230257554 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f2e380a3090 msgr2=0x7f2e380a5550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:31:56.877 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:56.878+0000 7f2e53200640 1 --2- 172.21.15.47:0/230257554 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f2e380a3090 0x7f2e380a5550 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f2e44009ae0 tx=0x7f2e44005f90 comp rx=0 tx=0).stop 2024-03-21T10:31:56.877 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:56.878+0000 7f2e53200640 1 -- 172.21.15.47:0/230257554 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2e54107500 msgr2=0x7f2e54074420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:31:56.877 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:56.878+0000 7f2e53200640 1 --2- 172.21.15.47:0/230257554 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2e54107500 0x7f2e54074420 secure :-1 s=READY pgs=973 cs=0 l=1 rev1=1 crypto rx=0x7f2e4c02c710 tx=0x7f2e4c035d60 comp rx=0 tx=0).stop 2024-03-21T10:31:56.878 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:56.879+0000 7f2e53200640 1 -- 172.21.15.47:0/230257554 shutdown_connections 2024-03-21T10:31:56.878 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:56.879+0000 7f2e53200640 1 --2- 172.21.15.47:0/230257554 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2e54107500 0x7f2e54074420 unknown :-1 s=CLOSED pgs=973 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:31:56.878 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:56.879+0000 7f2e53200640 1 --2- 172.21.15.47:0/230257554 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f2e380a3090 0x7f2e380a5550 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:31:56.878 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:56.879+0000 7f2e53200640 1 --2- 172.21.15.47:0/230257554 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2e54101e20 0x7f2e54073ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:31:56.878 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:56.879+0000 7f2e53200640 1 --2- 172.21.15.47:0/230257554 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2e54101460 0x7f2e540739a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:31:56.878 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:56.879+0000 7f2e53200640 1 -- 172.21.15.47:0/230257554 >> 172.21.15.47:0/230257554 conn(0x7f2e540fcf70 msgr2=0x7f2e540fd730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:31:56.878 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:56.879+0000 7f2e53200640 1 -- 172.21.15.47:0/230257554 shutdown_connections 2024-03-21T10:31:56.879 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:56.879+0000 7f2e53200640 1 -- 172.21.15.47:0/230257554 wait complete. 2024-03-21T10:31:56.899 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-21T10:31:56.899 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-03-21T10:31:57.118 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.119+0000 7fb25f400640 1 -- 172.21.15.47:0/3308817888 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb25806a590 msgr2=0x7fb25810e520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:31:57.118 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.119+0000 7fb25f400640 1 --2- 172.21.15.47:0/3308817888 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb25806a590 0x7fb25810e520 secure :-1 s=READY pgs=974 cs=0 l=1 rev1=1 crypto rx=0x7fb250009cd0 tx=0x7fb25002c0f0 comp rx=0 tx=0).stop 2024-03-21T10:31:57.118 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.119+0000 7fb25f400640 1 -- 172.21.15.47:0/3308817888 shutdown_connections 2024-03-21T10:31:57.118 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.119+0000 7fb25f400640 1 --2- 172.21.15.47:0/3308817888 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb25806a590 0x7fb25810e520 unknown :-1 s=CLOSED pgs=974 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:31:57.119 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.119+0000 7fb25f400640 1 --2- 172.21.15.47:0/3308817888 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb258069c10 0x7fb25806a050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:31:57.119 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.119+0000 7fb25f400640 1 --2- 172.21.15.47:0/3308817888 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb258107660 0x7fb258107a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:31:57.119 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.119+0000 7fb25f400640 1 -- 172.21.15.47:0/3308817888 >> 172.21.15.47:0/3308817888 conn(0x7fb258077580 msgr2=0x7fb258077990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:31:57.119 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.119+0000 7fb25f400640 1 -- 172.21.15.47:0/3308817888 shutdown_connections 2024-03-21T10:31:57.119 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.119+0000 7fb25f400640 1 -- 172.21.15.47:0/3308817888 wait complete. 2024-03-21T10:31:57.120 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.120+0000 7fb25f400640 1 Processor -- start 2024-03-21T10:31:57.121 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.120+0000 7fb25f400640 1 -- start start 2024-03-21T10:31:57.121 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.121+0000 7fb25f400640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb258069c10 0x7fb25819a010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:31:57.121 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.121+0000 7fb25f400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb25806a590 0x7fb25819a550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:31:57.121 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.121+0000 7fb25f400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb258107660 0x7fb25819e8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:31:57.121 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.121+0000 7fb25f400640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2581118d0 con 0x7fb25806a590 2024-03-21T10:31:57.121 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.121+0000 7fb25f400640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb258111750 con 0x7fb258107660 2024-03-21T10:31:57.121 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.121+0000 7fb25f400640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb258111a50 con 0x7fb258069c10 2024-03-21T10:31:57.121 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.121+0000 7fb257e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb25806a590 0x7fb25819a550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:31:57.121 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.121+0000 7fb257e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb25806a590 0x7fb25819a550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:57658/0 (socket says 172.21.15.47:57658) 2024-03-21T10:31:57.121 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.121+0000 7fb257e00640 1 -- 172.21.15.47:0/4213483118 learned_addr learned my addr 172.21.15.47:0/4213483118 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:31:57.121 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.121+0000 7fb25d400640 1 --2- 172.21.15.47:0/4213483118 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb258107660 0x7fb25819e8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:31:57.121 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.121+0000 7fb25ca00640 1 --2- 172.21.15.47:0/4213483118 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb258069c10 0x7fb25819a010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:31:57.121 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.122+0000 7fb257e00640 1 -- 172.21.15.47:0/4213483118 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb258069c10 msgr2=0x7fb25819a010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:31:57.121 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.122+0000 7fb257e00640 1 --2- 172.21.15.47:0/4213483118 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb258069c10 0x7fb25819a010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:31:57.122 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.122+0000 7fb257e00640 1 -- 172.21.15.47:0/4213483118 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb258107660 msgr2=0x7fb25819e8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:31:57.122 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.122+0000 7fb257e00640 1 --2- 172.21.15.47:0/4213483118 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb258107660 0x7fb25819e8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:31:57.122 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.122+0000 7fb257e00640 1 -- 172.21.15.47:0/4213483118 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb25819f000 con 0x7fb25806a590 2024-03-21T10:31:57.122 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.122+0000 7fb25ca00640 1 --2- 172.21.15.47:0/4213483118 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb258069c10 0x7fb25819a010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T10:31:57.122 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.122+0000 7fb257e00640 1 --2- 172.21.15.47:0/4213483118 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb25806a590 0x7fb25819a550 secure :-1 s=READY pgs=975 cs=0 l=1 rev1=1 crypto rx=0x7fb24400d950 tx=0x7fb24400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:31:57.122 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.122+0000 7fb255600640 1 -- 172.21.15.47:0/4213483118 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb244017070 con 0x7fb25806a590 2024-03-21T10:31:57.123 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.122+0000 7fb255600640 1 -- 172.21.15.47:0/4213483118 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fb244009c30 con 0x7fb25806a590 2024-03-21T10:31:57.123 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.122+0000 7fb255600640 1 -- 172.21.15.47:0/4213483118 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb24400f620 con 0x7fb25806a590 2024-03-21T10:31:57.123 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.122+0000 7fb25f400640 1 -- 172.21.15.47:0/4213483118 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb25819f2f0 con 0x7fb25806a590 2024-03-21T10:31:57.123 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.123+0000 7fb25f400640 1 -- 172.21.15.47:0/4213483118 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb2581a6b80 con 0x7fb25806a590 2024-03-21T10:31:57.124 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.126+0000 7fb255600640 1 -- 172.21.15.47:0/4213483118 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 27) v1 ==== 137985+0+0 (secure 0 0 0) 0x7fb2440109a0 con 0x7fb25806a590 2024-03-21T10:31:57.127 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.127+0000 7fb262198740 1 Processor -- start 2024-03-21T10:31:57.127 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.127+0000 7fb255600640 1 --2- 172.21.15.47:0/4213483118 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7fb2400a3090 0x7fb2400a5550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:31:57.127 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.128+0000 7fb262198740 1 -- start start 2024-03-21T10:31:57.127 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.128+0000 7fb255600640 1 -- 172.21.15.47:0/4213483118 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 9474+0+0 (secure 0 0 0) 0x7fb2440cf840 con 0x7fb25806a590 2024-03-21T10:31:57.127 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.128+0000 7fb25d400640 1 --2- 172.21.15.47:0/4213483118 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7fb2400a3090 0x7fb2400a5550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:31:57.127 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.128+0000 7fb262198740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a070ee86d0 0x55a070ee8ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:31:57.127 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.128+0000 7fb262198740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a070ee9080 0x55a070ef1630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:31:57.128 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.128+0000 7fb262198740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a070ef1c00 0x55a070ef4050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:31:57.128 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.128+0000 7fb262198740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a070e4c2c0 con 0x55a070ee9080 2024-03-21T10:31:57.128 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.128+0000 7fb262198740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a070df4c90 con 0x55a070ee86d0 2024-03-21T10:31:57.128 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.128+0000 7fb262198740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a070e48360 con 0x55a070ef1c00 2024-03-21T10:31:57.128 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.128+0000 7fb257e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a070ee9080 0x55a070ef1630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:31:57.128 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.128+0000 7fb257e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a070ee9080 0x55a070ef1630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:57672/0 (socket says 172.21.15.47:57672) 2024-03-21T10:31:57.128 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.128+0000 7fb257e00640 1 -- 172.21.15.47:0/1599463991 learned_addr learned my addr 172.21.15.47:0/1599463991 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:31:57.128 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.129+0000 7fb25ca00640 1 --2- 172.21.15.47:0/1599463991 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a070ee86d0 0x55a070ee8ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:31:57.128 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.129+0000 7fb25d400640 1 --2- 172.21.15.47:0/1599463991 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a070ef1c00 0x55a070ef4050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:31:57.130 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.129+0000 7fb257e00640 1 -- 172.21.15.47:0/1599463991 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a070ef1c00 msgr2=0x55a070ef4050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:31:57.130 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.129+0000 7fb257e00640 1 --2- 172.21.15.47:0/1599463991 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a070ef1c00 0x55a070ef4050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:31:57.130 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.129+0000 7fb257e00640 1 -- 172.21.15.47:0/1599463991 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a070ee86d0 msgr2=0x55a070ee8ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:31:57.130 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.129+0000 7fb257e00640 1 --2- 172.21.15.47:0/1599463991 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a070ee86d0 0x55a070ee8ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:31:57.130 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.129+0000 7fb257e00640 1 -- 172.21.15.47:0/1599463991 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a070e19080 con 0x55a070ee9080 2024-03-21T10:31:57.130 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.129+0000 7fb25d400640 1 --2- 172.21.15.47:0/4213483118 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7fb2400a3090 0x7fb2400a5550 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fb250005cc0 tx=0x7fb250009900 comp rx=0 tx=0).ready entity=mgr.15687 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:31:57.130 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.129+0000 7fb257e00640 1 --2- 172.21.15.47:0/1599463991 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a070ee9080 0x55a070ef1630 secure :-1 s=READY pgs=976 cs=0 l=1 rev1=1 crypto rx=0x7fb2440a3cf0 tx=0x7fb24409b710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=37fe07cd2d6d26a3 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:31:57.130 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.129+0000 7fb249c00640 1 -- 172.21.15.47:0/1599463991 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2440afe60 con 0x55a070ee9080 2024-03-21T10:31:57.131 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.130+0000 7fb249c00640 1 -- 172.21.15.47:0/1599463991 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fb24409f960 con 0x55a070ee9080 2024-03-21T10:31:57.131 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.130+0000 7fb249c00640 1 -- 172.21.15.47:0/1599463991 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb244020070 con 0x55a070ee9080 2024-03-21T10:31:57.131 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.130+0000 7fb262198740 1 -- 172.21.15.47:0/1599463991 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a070ee9080 msgr2=0x55a070ef1630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:31:57.131 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.130+0000 7fb262198740 1 --2- 172.21.15.47:0/1599463991 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a070ee9080 0x55a070ef1630 secure :-1 s=READY pgs=976 cs=0 l=1 rev1=1 crypto rx=0x7fb2440a3cf0 tx=0x7fb24409b710 comp rx=0 tx=0).stop 2024-03-21T10:31:57.131 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.130+0000 7fb262198740 1 -- 172.21.15.47:0/1599463991 shutdown_connections 2024-03-21T10:31:57.131 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.130+0000 7fb262198740 1 --2- 172.21.15.47:0/1599463991 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a070ef1c00 0x55a070ef4050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:31:57.131 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.130+0000 7fb262198740 1 --2- 172.21.15.47:0/1599463991 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a070ee9080 0x55a070ef1630 unknown :-1 s=CLOSED pgs=976 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:31:57.131 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.130+0000 7fb262198740 1 --2- 172.21.15.47:0/1599463991 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a070ee86d0 0x55a070ee8ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:31:57.131 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.130+0000 7fb262198740 1 -- 172.21.15.47:0/1599463991 >> 172.21.15.47:0/1599463991 conn(0x55a070ee2da0 msgr2=0x55a070ee3180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:31:57.131 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.130+0000 7fb262198740 1 -- 172.21.15.47:0/1599463991 shutdown_connections 2024-03-21T10:31:57.131 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.131+0000 7fb262198740 1 -- 172.21.15.47:0/1599463991 wait complete. 2024-03-21T10:31:57.131 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.131+0000 7fb262198740 1 -- start start 2024-03-21T10:31:57.131 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.131+0000 7fb262198740 1 Processor -- start 2024-03-21T10:31:57.135 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.132+0000 7fb262198740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a070ee86d0 0x55a070ef1530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:31:57.135 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.132+0000 7fb262198740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a070ee9080 0x55a070eee8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:31:57.136 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.132+0000 7fb262198740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a070ef1c00 0x55a070eeeed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:31:57.136 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.132+0000 7fb262198740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a070ef4ab0 con 0x55a070ef1c00 2024-03-21T10:31:57.136 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.132+0000 7fb262198740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a070e080d0 con 0x55a070ee9080 2024-03-21T10:31:57.136 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.132+0000 7fb262198740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a070ef4c30 con 0x55a070ee86d0 2024-03-21T10:31:57.136 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.132+0000 7fb262198740 10 client.0 _ll_register_callbacks cb 0x55a070e0ce80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T10:31:57.136 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.132+0000 7fb25d400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a070ef1c00 0x55a070eeeed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:31:57.136 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.132+0000 7fb25d400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a070ef1c00 0x55a070eeeed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:57678/0 (socket says 172.21.15.47:57678) 2024-03-21T10:31:57.136 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.132+0000 7fb25d400640 1 -- 172.21.15.47:0/4133538577 learned_addr learned my addr 172.21.15.47:0/4133538577 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:31:57.136 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.132+0000 7fb257e00640 1 --2- 172.21.15.47:0/4133538577 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a070ee9080 0x55a070eee8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:31:57.136 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.133+0000 7fb25ca00640 1 --2- 172.21.15.47:0/4133538577 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a070ee86d0 0x55a070ef1530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:31:57.137 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.133+0000 7fb25d400640 1 -- 172.21.15.47:0/4133538577 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a070ee86d0 msgr2=0x55a070ef1530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:31:57.137 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.133+0000 7fb25d400640 1 --2- 172.21.15.47:0/4133538577 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a070ee86d0 0x55a070ef1530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:31:57.137 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.133+0000 7fb25d400640 1 -- 172.21.15.47:0/4133538577 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a070ee9080 msgr2=0x55a070eee8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:31:57.137 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.133+0000 7fb25d400640 1 --2- 172.21.15.47:0/4133538577 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a070ee9080 0x55a070eee8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:31:57.137 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.133+0000 7fb25d400640 1 -- 172.21.15.47:0/4133538577 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55a070b603f0 con 0x55a070ef1c00 2024-03-21T10:31:57.137 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.133+0000 7fb25ca00640 1 --2- 172.21.15.47:0/4133538577 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a070ee86d0 0x55a070ef1530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T10:31:57.137 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.133+0000 7fb25d400640 1 --2- 172.21.15.47:0/4133538577 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a070ef1c00 0x55a070eeeed0 secure :-1 s=READY pgs=977 cs=0 l=1 rev1=1 crypto rx=0x7fb250036000 tx=0x7fb25002c0f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aff923765b2ca573 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:31:57.137 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.133+0000 7fb23e000640 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-03-21T10:31:57.137 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.133+0000 7fb23e000640 1 -- 172.21.15.47:0/4133538577 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb250039040 con 0x55a070ef1c00 2024-03-21T10:31:57.137 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.134+0000 7fb262198740 1 -- 172.21.15.47:0/4133538577 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fb24409f960 con 0x55a070ef1c00 2024-03-21T10:31:57.137 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.134+0000 7fb23e000640 1 -- 172.21.15.47:0/4133538577 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(97..97 src has 1..97) v4 ==== 9474+0+0 (secure 0 0 0) 0x7fb250003d80 con 0x55a070ef1c00 2024-03-21T10:31:57.137 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.134+0000 7fb23e000640 1 -- 172.21.15.47:0/4133538577 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb250059050 con 0x55a070ef1c00 2024-03-21T10:31:57.137 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.134+0000 7fb262198740 10 client.17598 fetch_fsmap learned FSMap version 32 2024-03-21T10:31:57.137 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.134+0000 7fb262198740 1 -- 172.21.15.47:0/4133538577 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x55a070c2a7d0 con 0x55a070ef1c00 2024-03-21T10:31:57.138 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.134+0000 7fb23e000640 1 -- 172.21.15.47:0/4133538577 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fb24409f960 con 0x55a070ef1c00 2024-03-21T10:31:57.138 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.134+0000 7fb23e000640 1 -- 172.21.15.47:0/4133538577 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 32) v1 ==== 7942+0+0 (secure 0 0 0) 0x7fb25003f040 con 0x55a070ef1c00 2024-03-21T10:31:57.138 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.134+0000 7fb262198740 10 client.17598 fetch_fsmap finished waiting for FSMap version 32 2024-03-21T10:31:57.138 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.134+0000 7fb262198740 10 client.17598 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.a 2024-03-21T10:31:57.138 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.134+0000 7fb262198740 20 client.17598 populate_metadata read hostname 'smithi047' 2024-03-21T10:31:57.138 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.135+0000 7fb262198740 1 --2- 172.21.15.47:0/4133538577 >> [v2:172.21.15.47:6840/3401578978,v1:172.21.15.47:6841/3401578978] conn(0x55a070f01960 0x55a070f03d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:31:57.138 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.135+0000 7fb262198740 4 client.17598 mds_command: new command op to 14619 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T10:31:57.138 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.135+0000 7fb262198740 1 -- 172.21.15.47:0/4133538577 --> [v2:172.21.15.47:6840/3401578978,v1:172.21.15.47:6841/3401578978] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55a070e59800 con 0x55a070f01960 2024-03-21T10:31:57.138 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.135+0000 7fb25ca00640 1 --2- 172.21.15.47:0/4133538577 >> [v2:172.21.15.47:6840/3401578978,v1:172.21.15.47:6841/3401578978] conn(0x55a070f01960 0x55a070f03d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:31:57.138 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.136+0000 7fb25ca00640 1 --2- 172.21.15.47:0/4133538577 >> [v2:172.21.15.47:6840/3401578978,v1:172.21.15.47:6841/3401578978] conn(0x55a070f01960 0x55a070f03d70 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=e9cf16830e654cd5 server_cookie=cc9226336bcec7b7 in_seq=0 out_seq=0 2024-03-21T10:31:57.138 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.136+0000 7fb23e000640 10 client.17598 ms_handle_connect on v2:172.21.15.47:6840/3401578978 2024-03-21T10:31:57.138 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.136+0000 7fb23e000640 1 -- 172.21.15.47:0/4133538577 <== mds.4 v2:172.21.15.47:6840/3401578978 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55a070e59800 con 0x55a070f01960 2024-03-21T10:31:57.138 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.136+0000 7fb23e000640 10 client.17598 handle_command_reply: tid=0 2024-03-21T10:31:57.138 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.136+0000 7fb262198740 1 client.17598 shutdown 2024-03-21T10:31:57.139 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.137+0000 7fb262198740 1 -- 172.21.15.47:0/4133538577 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a070ef1c00 msgr2=0x55a070eeeed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:31:57.139 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.137+0000 7fb262198740 1 --2- 172.21.15.47:0/4133538577 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a070ef1c00 0x55a070eeeed0 secure :-1 s=READY pgs=977 cs=0 l=1 rev1=1 crypto rx=0x7fb250036000 tx=0x7fb25002c0f0 comp rx=0 tx=0).stop 2024-03-21T10:31:57.139 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.137+0000 7fb262198740 1 -- 172.21.15.47:0/4133538577 shutdown_connections 2024-03-21T10:31:57.139 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.137+0000 7fb262198740 1 --2- 172.21.15.47:0/4133538577 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a070ef1c00 0x55a070eeeed0 unknown :-1 s=CLOSED pgs=977 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:31:57.139 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.137+0000 7fb262198740 1 --2- 172.21.15.47:0/4133538577 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a070ee9080 0x55a070eee8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:31:57.139 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.137+0000 7fb262198740 1 --2- 172.21.15.47:0/4133538577 >> [v2:172.21.15.47:6840/3401578978,v1:172.21.15.47:6841/3401578978] conn(0x55a070f01960 0x55a070f03d70 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:31:57.139 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.137+0000 7fb262198740 1 --2- 172.21.15.47:0/4133538577 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a070ee86d0 0x55a070ef1530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:31:57.139 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.137+0000 7fb262198740 1 -- 172.21.15.47:0/4133538577 >> 172.21.15.47:0/4133538577 conn(0x55a070ee2da0 msgr2=0x55a070ee4af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:31:57.139 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.137+0000 7fb23e000640 0 client.17598 ms_handle_reset on v2:172.21.15.47:6840/3401578978 2024-03-21T10:31:57.139 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.137+0000 7fb262198740 1 -- 172.21.15.47:0/4133538577 shutdown_connections 2024-03-21T10:31:57.139 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.137+0000 7fb262198740 1 -- 172.21.15.47:0/4133538577 wait complete. 2024-03-21T10:31:57.140 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.137+0000 7fb262198740 20 client.17598 trim_cache size 0 max 16384 2024-03-21T10:31:57.150 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.151+0000 7fb262198740 1 Processor -- start 2024-03-21T10:31:57.150 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.151+0000 7fb262198740 1 -- start start 2024-03-21T10:31:57.150 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.151+0000 7fb262198740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a070ee86d0 0x55a070ef0ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:31:57.150 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.151+0000 7fb262198740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a070ee9080 0x55a070ef1530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:31:57.151 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.151+0000 7fb262198740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a070ef1c00 0x55a070ee1d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:31:57.151 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.151+0000 7fb262198740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2581b0b20 con 0x55a070ef1c00 2024-03-21T10:31:57.151 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.151+0000 7fb262198740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2581b24a0 con 0x55a070ee86d0 2024-03-21T10:31:57.151 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.151+0000 7fb262198740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2581b1180 con 0x55a070ee9080 2024-03-21T10:31:57.151 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.151+0000 7fb25d400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a070ef1c00 0x55a070ee1d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:31:57.151 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.151+0000 7fb25d400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a070ef1c00 0x55a070ee1d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:57684/0 (socket says 172.21.15.47:57684) 2024-03-21T10:31:57.151 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.151+0000 7fb25d400640 1 -- 172.21.15.47:0/3749618621 learned_addr learned my addr 172.21.15.47:0/3749618621 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:31:57.151 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.151+0000 7fb25ca00640 1 --2- 172.21.15.47:0/3749618621 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a070ee86d0 0x55a070ef0ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:31:57.151 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.152+0000 7fb257e00640 1 --2- 172.21.15.47:0/3749618621 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a070ee9080 0x55a070ef1530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:31:57.151 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.152+0000 7fb25d400640 1 -- 172.21.15.47:0/3749618621 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a070ee9080 msgr2=0x55a070ef1530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:31:57.151 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.152+0000 7fb25d400640 1 --2- 172.21.15.47:0/3749618621 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a070ee9080 0x55a070ef1530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:31:57.151 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.152+0000 7fb25d400640 1 -- 172.21.15.47:0/3749618621 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a070ee86d0 msgr2=0x55a070ef0ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:31:57.151 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.152+0000 7fb25d400640 1 --2- 172.21.15.47:0/3749618621 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a070ee86d0 0x55a070ef0ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:31:57.151 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.152+0000 7fb25d400640 1 -- 172.21.15.47:0/3749618621 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a070eece10 con 0x55a070ef1c00 2024-03-21T10:31:57.155 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.152+0000 7fb25d400640 1 --2- 172.21.15.47:0/3749618621 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a070ef1c00 0x55a070ee1d30 secure :-1 s=READY pgs=978 cs=0 l=1 rev1=1 crypto rx=0x7fb25002c700 tx=0x7fb250032fc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c440f4e5781b94a3 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:31:57.155 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.152+0000 7fb249200640 1 -- 172.21.15.47:0/3749618621 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb250062070 con 0x55a070ef1c00 2024-03-21T10:31:57.155 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.152+0000 7fb249200640 1 -- 172.21.15.47:0/3749618621 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fb250050070 con 0x55a070ef1c00 2024-03-21T10:31:57.155 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.152+0000 7fb249200640 1 -- 172.21.15.47:0/3749618621 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb250039070 con 0x55a070ef1c00 2024-03-21T10:31:57.155 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.153+0000 7fb262198740 1 -- 172.21.15.47:0/3749618621 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a070ef1c00 msgr2=0x55a070ee1d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:31:57.155 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.153+0000 7fb262198740 1 --2- 172.21.15.47:0/3749618621 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a070ef1c00 0x55a070ee1d30 secure :-1 s=READY pgs=978 cs=0 l=1 rev1=1 crypto rx=0x7fb25002c700 tx=0x7fb250032fc0 comp rx=0 tx=0).stop 2024-03-21T10:31:57.155 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.153+0000 7fb262198740 1 -- 172.21.15.47:0/3749618621 shutdown_connections 2024-03-21T10:31:57.155 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.153+0000 7fb262198740 1 --2- 172.21.15.47:0/3749618621 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a070ef1c00 0x55a070ee1d30 unknown :-1 s=CLOSED pgs=978 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:31:57.155 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.153+0000 7fb262198740 1 --2- 172.21.15.47:0/3749618621 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a070ee9080 0x55a070ef1530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:31:57.155 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.153+0000 7fb262198740 1 --2- 172.21.15.47:0/3749618621 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a070ee86d0 0x55a070ef0ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:31:57.155 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.153+0000 7fb262198740 1 -- 172.21.15.47:0/3749618621 >> 172.21.15.47:0/3749618621 conn(0x55a070ee2da0 msgr2=0x55a070ee1620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:31:57.156 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.153+0000 7fb262198740 1 -- 172.21.15.47:0/3749618621 shutdown_connections 2024-03-21T10:31:57.156 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.153+0000 7fb262198740 1 -- 172.21.15.47:0/3749618621 wait complete. 2024-03-21T10:31:57.156 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.154+0000 7fb262198740 1 -- start start 2024-03-21T10:31:57.156 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.154+0000 7fb262198740 1 Processor -- start 2024-03-21T10:31:57.156 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.154+0000 7fb262198740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a070ee86d0 0x55a070ef94c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:31:57.156 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.154+0000 7fb262198740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a070ee9080 0x55a070ef9a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:31:57.156 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.154+0000 7fb262198740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a070ef1c00 0x55a070ef79f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:31:57.156 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.154+0000 7fb262198740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2581b1e40 con 0x55a070ef1c00 2024-03-21T10:31:57.156 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.154+0000 7fb262198740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2581b17e0 con 0x55a070ee86d0 2024-03-21T10:31:57.157 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.154+0000 7fb262198740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2581aece0 con 0x55a070ee9080 2024-03-21T10:31:57.157 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.154+0000 7fb262198740 10 client.0 _ll_register_callbacks cb 0x7fb250045030 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T10:31:57.157 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.155+0000 7fb25d400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a070ef1c00 0x55a070ef79f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:31:57.157 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.155+0000 7fb25d400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a070ef1c00 0x55a070ef79f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:57696/0 (socket says 172.21.15.47:57696) 2024-03-21T10:31:57.157 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.155+0000 7fb25d400640 1 -- 172.21.15.47:0/347757739 learned_addr learned my addr 172.21.15.47:0/347757739 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:31:57.157 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.155+0000 7fb25ca00640 1 --2- 172.21.15.47:0/347757739 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a070ee86d0 0x55a070ef94c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:31:57.157 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.155+0000 7fb257e00640 1 --2- 172.21.15.47:0/347757739 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a070ee9080 0x55a070ef9a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:31:57.157 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.155+0000 7fb25d400640 1 -- 172.21.15.47:0/347757739 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a070ee9080 msgr2=0x55a070ef9a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:31:57.157 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.155+0000 7fb25d400640 1 --2- 172.21.15.47:0/347757739 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a070ee9080 0x55a070ef9a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:31:57.157 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.155+0000 7fb25d400640 1 -- 172.21.15.47:0/347757739 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a070ee86d0 msgr2=0x55a070ef94c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:31:57.157 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.155+0000 7fb25d400640 1 --2- 172.21.15.47:0/347757739 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a070ee86d0 0x55a070ef94c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:31:57.157 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.155+0000 7fb25d400640 1 -- 172.21.15.47:0/347757739 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55a070ef80e0 con 0x55a070ef1c00 2024-03-21T10:31:57.158 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.155+0000 7fb257e00640 1 --2- 172.21.15.47:0/347757739 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a070ee9080 0x55a070ef9a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T10:31:57.158 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.155+0000 7fb25d400640 1 --2- 172.21.15.47:0/347757739 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a070ef1c00 0x55a070ef79f0 secure :-1 s=READY pgs=979 cs=0 l=1 rev1=1 crypto rx=0x7fb25004b780 tx=0x7fb250009c90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e30e8f3a95841173 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:31:57.158 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.155+0000 7fb23e000640 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-03-21T10:31:57.158 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.156+0000 7fb23e000640 1 -- 172.21.15.47:0/347757739 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb250062070 con 0x55a070ef1c00 2024-03-21T10:31:57.158 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.156+0000 7fb262198740 1 -- 172.21.15.47:0/347757739 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fb250050070 con 0x55a070ef1c00 2024-03-21T10:31:57.158 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.156+0000 7fb23e000640 1 -- 172.21.15.47:0/347757739 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(97..97 src has 1..97) v4 ==== 9474+0+0 (secure 0 0 0) 0x7fb250055650 con 0x55a070ef1c00 2024-03-21T10:31:57.158 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.156+0000 7fb23e000640 1 -- 172.21.15.47:0/347757739 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb25006e050 con 0x55a070ef1c00 2024-03-21T10:31:57.158 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.156+0000 7fb262198740 10 client.17604 fetch_fsmap learned FSMap version 32 2024-03-21T10:31:57.158 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.156+0000 7fb262198740 1 -- 172.21.15.47:0/347757739 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x55a070f5dc70 con 0x55a070ef1c00 2024-03-21T10:31:57.158 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.156+0000 7fb23e000640 1 -- 172.21.15.47:0/347757739 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fb250050070 con 0x55a070ef1c00 2024-03-21T10:31:57.160 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.157+0000 7fb23e000640 1 -- 172.21.15.47:0/347757739 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 32) v1 ==== 7942+0+0 (secure 0 0 0) 0x7fb250034960 con 0x55a070ef1c00 2024-03-21T10:31:57.160 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.157+0000 7fb262198740 10 client.17604 fetch_fsmap finished waiting for FSMap version 32 2024-03-21T10:31:57.160 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.157+0000 7fb262198740 10 client.17604 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.a 2024-03-21T10:31:57.160 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.157+0000 7fb262198740 20 client.17604 populate_metadata read hostname 'smithi047' 2024-03-21T10:31:57.160 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.157+0000 7fb262198740 1 --2- 172.21.15.47:0/347757739 >> [v2:172.21.15.47:6840/3401578978,v1:172.21.15.47:6841/3401578978] conn(0x55a070f5e250 0x55a070f60670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:31:57.160 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.157+0000 7fb262198740 4 client.17604 mds_command: new command op to 14619 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-21T10:31:57.161 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.157+0000 7fb262198740 1 -- 172.21.15.47:0/347757739 --> [v2:172.21.15.47:6840/3401578978,v1:172.21.15.47:6841/3401578978] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55a070e53b50 con 0x55a070f5e250 2024-03-21T10:31:57.161 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.157+0000 7fb25ca00640 1 --2- 172.21.15.47:0/347757739 >> [v2:172.21.15.47:6840/3401578978,v1:172.21.15.47:6841/3401578978] conn(0x55a070f5e250 0x55a070f60670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:31:57.161 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.158+0000 7fb25ca00640 1 --2- 172.21.15.47:0/347757739 >> [v2:172.21.15.47:6840/3401578978,v1:172.21.15.47:6841/3401578978] conn(0x55a070f5e250 0x55a070f60670 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=93bacdd2cb54db12 server_cookie=368705c05af9e660 in_seq=0 out_seq=0 2024-03-21T10:31:57.161 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.158+0000 7fb23e000640 10 client.17604 ms_handle_connect on v2:172.21.15.47:6840/3401578978 2024-03-21T10:31:57.161 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.158+0000 7fb23e000640 1 -- 172.21.15.47:0/347757739 <== mds.4 v2:172.21.15.47:6840/3401578978 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55a070e53b50 con 0x55a070f5e250 2024-03-21T10:31:57.161 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.158+0000 7fb23e000640 10 client.17604 handle_command_reply: tid=0 2024-03-21T10:31:57.161 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.158+0000 7fb262198740 1 client.17604 shutdown 2024-03-21T10:31:57.161 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.158+0000 7fb262198740 1 -- 172.21.15.47:0/347757739 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a070ef1c00 msgr2=0x55a070ef79f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:31:57.161 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.158+0000 7fb262198740 1 --2- 172.21.15.47:0/347757739 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a070ef1c00 0x55a070ef79f0 secure :-1 s=READY pgs=979 cs=0 l=1 rev1=1 crypto rx=0x7fb25004b780 tx=0x7fb250009c90 comp rx=0 tx=0).stop 2024-03-21T10:31:57.161 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.158+0000 7fb262198740 1 -- 172.21.15.47:0/347757739 shutdown_connections 2024-03-21T10:31:57.161 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.158+0000 7fb262198740 1 --2- 172.21.15.47:0/347757739 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a070ef1c00 0x55a070ef79f0 unknown :-1 s=CLOSED pgs=979 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:31:57.161 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.158+0000 7fb262198740 1 --2- 172.21.15.47:0/347757739 >> [v2:172.21.15.47:6840/3401578978,v1:172.21.15.47:6841/3401578978] conn(0x55a070f5e250 0x55a070f60670 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:31:57.161 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.158+0000 7fb262198740 1 --2- 172.21.15.47:0/347757739 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a070ee9080 0x55a070ef9a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:31:57.161 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.158+0000 7fb262198740 1 --2- 172.21.15.47:0/347757739 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a070ee86d0 0x55a070ef94c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:31:57.161 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.158+0000 7fb262198740 1 -- 172.21.15.47:0/347757739 >> 172.21.15.47:0/347757739 conn(0x55a070ee2da0 msgr2=0x55a070ee9d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:31:57.162 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.158+0000 7fb23e000640 0 client.17604 ms_handle_reset on v2:172.21.15.47:6840/3401578978 2024-03-21T10:31:57.162 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.159+0000 7fb262198740 1 -- 172.21.15.47:0/347757739 shutdown_connections 2024-03-21T10:31:57.162 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.159+0000 7fb262198740 1 -- 172.21.15.47:0/347757739 wait complete. 2024-03-21T10:31:57.162 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.159+0000 7fb262198740 20 client.17604 trim_cache size 0 max 16384 2024-03-21T10:31:57.162 INFO:teuthology.orchestra.run.smithi047.stdout:[] 2024-03-21T10:31:57.163 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.159+0000 7fb25f400640 1 -- 172.21.15.47:0/4213483118 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7fb2400a3090 msgr2=0x7fb2400a5550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:31:57.163 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.159+0000 7fb25f400640 1 --2- 172.21.15.47:0/4213483118 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7fb2400a3090 0x7fb2400a5550 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fb250005cc0 tx=0x7fb250009900 comp rx=0 tx=0).stop 2024-03-21T10:31:57.163 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.159+0000 7fb25f400640 1 -- 172.21.15.47:0/4213483118 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb25806a590 msgr2=0x7fb25819a550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:31:57.163 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.159+0000 7fb25f400640 1 --2- 172.21.15.47:0/4213483118 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb25806a590 0x7fb25819a550 secure :-1 s=READY pgs=975 cs=0 l=1 rev1=1 crypto rx=0x7fb24400d950 tx=0x7fb24400de10 comp rx=0 tx=0).stop 2024-03-21T10:31:57.163 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.159+0000 7fb25f400640 1 -- 172.21.15.47:0/4213483118 shutdown_connections 2024-03-21T10:31:57.163 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.159+0000 7fb25f400640 1 --2- 172.21.15.47:0/4213483118 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7fb2400a3090 0x7fb2400a5550 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:31:57.163 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.159+0000 7fb25f400640 1 --2- 172.21.15.47:0/4213483118 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb258107660 0x7fb25819e8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:31:57.163 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.159+0000 7fb25f400640 1 --2- 172.21.15.47:0/4213483118 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb25806a590 0x7fb25819a550 unknown :-1 s=CLOSED pgs=975 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:31:57.163 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.159+0000 7fb25f400640 1 --2- 172.21.15.47:0/4213483118 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb258069c10 0x7fb25819a010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:31:57.163 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.160+0000 7fb25f400640 1 -- 172.21.15.47:0/4213483118 >> 172.21.15.47:0/4213483118 conn(0x7fb258077580 msgr2=0x7fb25810cdb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:31:57.164 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.160+0000 7fb25f400640 1 -- 172.21.15.47:0/4213483118 shutdown_connections 2024-03-21T10:31:57.164 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.160+0000 7fb25f400640 1 -- 172.21.15.47:0/4213483118 wait complete. 2024-03-21T10:31:57.175 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-21T10:31:57.175 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-03-21T10:31:57.390 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.391+0000 7f9696200640 1 -- 172.21.15.47:0/1394605992 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9690102ff0 msgr2=0x7f969010b4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:31:57.390 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.391+0000 7f9696200640 1 --2- 172.21.15.47:0/1394605992 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9690102ff0 0x7f969010b4d0 secure :-1 s=READY pgs=980 cs=0 l=1 rev1=1 crypto rx=0x7f9678009900 tx=0x7f967802bcd0 comp rx=0 tx=0).stop 2024-03-21T10:31:57.391 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.391+0000 7f9696200640 1 -- 172.21.15.47:0/1394605992 shutdown_connections 2024-03-21T10:31:57.391 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.391+0000 7f9696200640 1 --2- 172.21.15.47:0/1394605992 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f969010baf0 0x7f969010df90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:31:57.391 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.391+0000 7f9696200640 1 --2- 172.21.15.47:0/1394605992 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9690102ff0 0x7f969010b4d0 unknown :-1 s=CLOSED pgs=980 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:31:57.391 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.391+0000 7f9696200640 1 --2- 172.21.15.47:0/1394605992 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f96901026d0 0x7f9690102ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:31:57.391 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.391+0000 7f9696200640 1 -- 172.21.15.47:0/1394605992 >> 172.21.15.47:0/1394605992 conn(0x7f96900fbef0 msgr2=0x7f96900fe310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:31:57.391 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.391+0000 7f9696200640 1 -- 172.21.15.47:0/1394605992 shutdown_connections 2024-03-21T10:31:57.391 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.392+0000 7f9696200640 1 -- 172.21.15.47:0/1394605992 wait complete. 2024-03-21T10:31:57.393 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.393+0000 7f9696200640 1 Processor -- start 2024-03-21T10:31:57.393 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.393+0000 7f9696200640 1 -- start start 2024-03-21T10:31:57.393 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.393+0000 7f9696200640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f96901026d0 0x7f9690199fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:31:57.393 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.393+0000 7f9696200640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9690102ff0 0x7f969019a510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:31:57.393 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.393+0000 7f9696200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f969010baf0 0x7f969019e890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:31:57.393 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.393+0000 7f9696200640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96901117b0 con 0x7f9690102ff0 2024-03-21T10:31:57.393 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.393+0000 7f9696200640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9690111630 con 0x7f969010baf0 2024-03-21T10:31:57.393 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.393+0000 7f9696200640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9690111930 con 0x7f96901026d0 2024-03-21T10:31:57.393 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.394+0000 7f9687e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9690102ff0 0x7f969019a510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:31:57.393 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.394+0000 7f9687e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9690102ff0 0x7f969019a510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:57708/0 (socket says 172.21.15.47:57708) 2024-03-21T10:31:57.393 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.394+0000 7f9687e00640 1 -- 172.21.15.47:0/2977812504 learned_addr learned my addr 172.21.15.47:0/2977812504 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:31:57.393 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.394+0000 7f9694e00640 1 --2- 172.21.15.47:0/2977812504 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f96901026d0 0x7f9690199fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:31:57.393 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.394+0000 7f9695800640 1 --2- 172.21.15.47:0/2977812504 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f969010baf0 0x7f969019e890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:31:57.394 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.394+0000 7f9687e00640 1 -- 172.21.15.47:0/2977812504 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f96901026d0 msgr2=0x7f9690199fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:31:57.394 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.394+0000 7f9687e00640 1 --2- 172.21.15.47:0/2977812504 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f96901026d0 0x7f9690199fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:31:57.394 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.394+0000 7f9687e00640 1 -- 172.21.15.47:0/2977812504 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f969010baf0 msgr2=0x7f969019e890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:31:57.394 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.394+0000 7f9687e00640 1 --2- 172.21.15.47:0/2977812504 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f969010baf0 0x7f969019e890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:31:57.394 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.394+0000 7f9687e00640 1 -- 172.21.15.47:0/2977812504 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f969019efc0 con 0x7f9690102ff0 2024-03-21T10:31:57.395 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.395+0000 7f9687e00640 1 --2- 172.21.15.47:0/2977812504 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9690102ff0 0x7f969019a510 secure :-1 s=READY pgs=981 cs=0 l=1 rev1=1 crypto rx=0x7f9678000c00 tx=0x7f967802c6d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:31:57.395 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.395+0000 7f968e000640 1 -- 172.21.15.47:0/2977812504 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9678043020 con 0x7f9690102ff0 2024-03-21T10:31:57.395 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.395+0000 7f968e000640 1 -- 172.21.15.47:0/2977812504 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f967802dbe0 con 0x7f9690102ff0 2024-03-21T10:31:57.395 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.395+0000 7f9696200640 1 -- 172.21.15.47:0/2977812504 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f969019f250 con 0x7f9690102ff0 2024-03-21T10:31:57.395 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.395+0000 7f968e000640 1 -- 172.21.15.47:0/2977812504 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9678003970 con 0x7f9690102ff0 2024-03-21T10:31:57.395 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.395+0000 7f9696200640 1 -- 172.21.15.47:0/2977812504 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f96901a6b40 con 0x7f9690102ff0 2024-03-21T10:31:57.409 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.398+0000 7f968e000640 1 -- 172.21.15.47:0/2977812504 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 27) v1 ==== 137985+0+0 (secure 0 0 0) 0x7f9678046020 con 0x7f9690102ff0 2024-03-21T10:31:57.409 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.398+0000 7f969a5ac740 1 Processor -- start 2024-03-21T10:31:57.409 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.399+0000 7f969a5ac740 1 -- start start 2024-03-21T10:31:57.409 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.399+0000 7f969a5ac740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x556563a758b0 0x556563a75c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:31:57.410 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.399+0000 7f969a5ac740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556563a76260 0x556563a7e810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:31:57.410 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.399+0000 7f969a5ac740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556563a7ede0 0x556563a81230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:31:57.410 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.399+0000 7f969a5ac740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x5565639d92c0 con 0x556563a758b0 2024-03-21T10:31:57.410 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.399+0000 7f969a5ac740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x556563981c90 con 0x556563a7ede0 2024-03-21T10:31:57.410 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.399+0000 7f969a5ac740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x5565639d5360 con 0x556563a76260 2024-03-21T10:31:57.410 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.399+0000 7f9695800640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x556563a758b0 0x556563a75c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:31:57.410 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.399+0000 7f9695800640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x556563a758b0 0x556563a75c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:57724/0 (socket says 172.21.15.47:57724) 2024-03-21T10:31:57.410 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.399+0000 7f9695800640 1 -- 172.21.15.47:0/190955053 learned_addr learned my addr 172.21.15.47:0/190955053 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:31:57.410 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.399+0000 7f9694e00640 1 --2- 172.21.15.47:0/190955053 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556563a76260 0x556563a7e810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:31:57.410 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.400+0000 7f9687e00640 1 --2- 172.21.15.47:0/190955053 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556563a7ede0 0x556563a81230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:31:57.411 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.400+0000 7f9695800640 1 -- 172.21.15.47:0/190955053 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556563a76260 msgr2=0x556563a7e810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:31:57.411 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.400+0000 7f9695800640 1 --2- 172.21.15.47:0/190955053 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556563a76260 0x556563a7e810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:31:57.411 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.400+0000 7f9695800640 1 -- 172.21.15.47:0/190955053 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556563a7ede0 msgr2=0x556563a81230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:31:57.411 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.400+0000 7f9695800640 1 --2- 172.21.15.47:0/190955053 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556563a7ede0 0x556563a81230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:31:57.411 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.400+0000 7f9695800640 1 -- 172.21.15.47:0/190955053 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5565639a6080 con 0x556563a758b0 2024-03-21T10:31:57.411 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.400+0000 7f9695800640 1 --2- 172.21.15.47:0/190955053 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x556563a758b0 0x556563a75c90 secure :-1 s=READY pgs=982 cs=0 l=1 rev1=1 crypto rx=0x7f9688009c20 tx=0x7f9688011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a0f77d19ac90e82c server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:31:57.411 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.400+0000 7f968e000640 1 --2- 172.21.15.47:0/2977812504 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f96740a3090 0x7f96740a5550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:31:57.411 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.400+0000 7f968e000640 1 -- 172.21.15.47:0/2977812504 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 9474+0+0 (secure 0 0 0) 0x7f96780f1aa0 con 0x7f9690102ff0 2024-03-21T10:31:57.411 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.401+0000 7f9686000640 1 -- 172.21.15.47:0/190955053 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f968801e070 con 0x556563a758b0 2024-03-21T10:31:57.411 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.401+0000 7f9694e00640 1 --2- 172.21.15.47:0/2977812504 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f96740a3090 0x7f96740a5550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:31:57.411 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.401+0000 7f9686000640 1 -- 172.21.15.47:0/190955053 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f9688019ae0 con 0x556563a758b0 2024-03-21T10:31:57.411 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.401+0000 7f9686000640 1 -- 172.21.15.47:0/190955053 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f968800e9b0 con 0x556563a758b0 2024-03-21T10:31:57.411 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.402+0000 7f969a5ac740 1 -- 172.21.15.47:0/190955053 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x556563a758b0 msgr2=0x556563a75c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:31:57.412 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.402+0000 7f969a5ac740 1 --2- 172.21.15.47:0/190955053 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x556563a758b0 0x556563a75c90 secure :-1 s=READY pgs=982 cs=0 l=1 rev1=1 crypto rx=0x7f9688009c20 tx=0x7f9688011810 comp rx=0 tx=0).stop 2024-03-21T10:31:57.412 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.402+0000 7f9694e00640 1 --2- 172.21.15.47:0/2977812504 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f96740a3090 0x7f96740a5550 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x556563a70ff0 tx=0x7f9680009580 comp rx=0 tx=0).ready entity=mgr.15687 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:31:57.412 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.402+0000 7f969a5ac740 1 -- 172.21.15.47:0/190955053 shutdown_connections 2024-03-21T10:31:57.412 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.402+0000 7f969a5ac740 1 --2- 172.21.15.47:0/190955053 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556563a7ede0 0x556563a81230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:31:57.412 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.402+0000 7f969a5ac740 1 --2- 172.21.15.47:0/190955053 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556563a76260 0x556563a7e810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:31:57.412 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.402+0000 7f969a5ac740 1 --2- 172.21.15.47:0/190955053 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x556563a758b0 0x556563a75c90 unknown :-1 s=CLOSED pgs=982 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:31:57.412 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.402+0000 7f969a5ac740 1 -- 172.21.15.47:0/190955053 >> 172.21.15.47:0/190955053 conn(0x556563a6ff80 msgr2=0x556563a70360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:31:57.412 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.402+0000 7f969a5ac740 1 -- 172.21.15.47:0/190955053 shutdown_connections 2024-03-21T10:31:57.412 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.402+0000 7f969a5ac740 1 -- 172.21.15.47:0/190955053 wait complete. 2024-03-21T10:31:57.412 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.402+0000 7f969a5ac740 1 -- start start 2024-03-21T10:31:57.412 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.403+0000 7f969a5ac740 1 Processor -- start 2024-03-21T10:31:57.412 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.403+0000 7f969a5ac740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x556563a758b0 0x556563a7e6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:31:57.412 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.403+0000 7f969a5ac740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556563a76260 0x556563a7ba70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:31:57.412 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.403+0000 7f969a5ac740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556563a7ede0 0x556563a7bff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:31:57.412 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.403+0000 7f969a5ac740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x556563a81d50 con 0x556563a758b0 2024-03-21T10:31:57.413 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.403+0000 7f969a5ac740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x5565639950d0 con 0x556563a76260 2024-03-21T10:31:57.413 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.403+0000 7f969a5ac740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x556563a81ed0 con 0x556563a7ede0 2024-03-21T10:31:57.413 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.403+0000 7f969a5ac740 10 client.0 _ll_register_callbacks cb 0x556563999e80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T10:31:57.413 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.403+0000 7f9694e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x556563a758b0 0x556563a7e6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:31:57.413 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.403+0000 7f9694e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x556563a758b0 0x556563a7e6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:57730/0 (socket says 172.21.15.47:57730) 2024-03-21T10:31:57.413 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.403+0000 7f9694e00640 1 -- 172.21.15.47:0/843677982 learned_addr learned my addr 172.21.15.47:0/843677982 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:31:57.413 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.404+0000 7f9687e00640 1 --2- 172.21.15.47:0/843677982 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556563a76260 0x556563a7ba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:31:57.413 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.404+0000 7f9694e00640 1 -- 172.21.15.47:0/843677982 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556563a7ede0 msgr2=0x556563a7bff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:31:57.413 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.404+0000 7f9695800640 1 --2- 172.21.15.47:0/843677982 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556563a7ede0 0x556563a7bff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:31:57.413 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.404+0000 7f9694e00640 1 --2- 172.21.15.47:0/843677982 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556563a7ede0 0x556563a7bff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:31:57.413 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.404+0000 7f9694e00640 1 -- 172.21.15.47:0/843677982 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556563a76260 msgr2=0x556563a7ba70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:31:57.413 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.404+0000 7f9694e00640 1 --2- 172.21.15.47:0/843677982 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556563a76260 0x556563a7ba70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:31:57.413 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.404+0000 7f9694e00640 1 -- 172.21.15.47:0/843677982 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5565636ed3f0 con 0x556563a758b0 2024-03-21T10:31:57.414 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.404+0000 7f9694e00640 1 --2- 172.21.15.47:0/843677982 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x556563a758b0 0x556563a7e6e0 secure :-1 s=READY pgs=983 cs=0 l=1 rev1=1 crypto rx=0x7f96800128a0 tx=0x7f9680004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e673f546f40bfa0c server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:31:57.414 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.404+0000 7f967ea00640 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-03-21T10:31:57.414 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.404+0000 7f967ea00640 1 -- 172.21.15.47:0/843677982 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f968000ec60 con 0x556563a758b0 2024-03-21T10:31:57.414 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.404+0000 7f969a5ac740 1 -- 172.21.15.47:0/843677982 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f9688019ae0 con 0x556563a758b0 2024-03-21T10:31:57.414 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.405+0000 7f967ea00640 1 -- 172.21.15.47:0/843677982 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(97..97 src has 1..97) v4 ==== 9474+0+0 (secure 0 0 0) 0x7f968000ee00 con 0x556563a758b0 2024-03-21T10:31:57.414 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.405+0000 7f967ea00640 1 -- 172.21.15.47:0/843677982 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9680030050 con 0x556563a758b0 2024-03-21T10:31:57.414 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.405+0000 7f969a5ac740 10 client.17616 fetch_fsmap learned FSMap version 32 2024-03-21T10:31:57.414 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.405+0000 7f969a5ac740 1 -- 172.21.15.47:0/843677982 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x5565637b77d0 con 0x556563a758b0 2024-03-21T10:31:57.414 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.405+0000 7f967ea00640 1 -- 172.21.15.47:0/843677982 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f9688019ae0 con 0x556563a758b0 2024-03-21T10:31:57.414 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.405+0000 7f967ea00640 1 -- 172.21.15.47:0/843677982 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 32) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f968000f040 con 0x556563a758b0 2024-03-21T10:31:57.414 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.406+0000 7f969a5ac740 10 client.17616 fetch_fsmap finished waiting for FSMap version 32 2024-03-21T10:31:57.414 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.406+0000 7f969a5ac740 10 client.17616 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.k 2024-03-21T10:31:57.414 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.406+0000 7f969a5ac740 20 client.17616 populate_metadata read hostname 'smithi047' 2024-03-21T10:31:57.415 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.406+0000 7f969a5ac740 1 --2- 172.21.15.47:0/843677982 >> [v2:172.21.15.100:6832/2723302038,v1:172.21.15.100:6833/2723302038] conn(0x556563a8eca0 0x556563a91090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:31:57.415 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.406+0000 7f969a5ac740 4 client.17616 mds_command: new command op to 34427 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T10:31:57.415 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.406+0000 7f969a5ac740 1 -- 172.21.15.47:0/843677982 --> [v2:172.21.15.100:6832/2723302038,v1:172.21.15.100:6833/2723302038] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5565639e6800 con 0x556563a8eca0 2024-03-21T10:31:57.415 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.406+0000 7f9687e00640 1 --2- 172.21.15.47:0/843677982 >> [v2:172.21.15.100:6832/2723302038,v1:172.21.15.100:6833/2723302038] conn(0x556563a8eca0 0x556563a91090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:31:57.415 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.407+0000 7f9687e00640 1 --2- 172.21.15.47:0/843677982 >> [v2:172.21.15.100:6832/2723302038,v1:172.21.15.100:6833/2723302038] conn(0x556563a8eca0 0x556563a91090 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=d90f3c518a5ddd server_cookie=e5c6b2ef1903c3ea in_seq=0 out_seq=0 2024-03-21T10:31:57.415 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.407+0000 7f967ea00640 10 client.17616 ms_handle_connect on v2:172.21.15.100:6832/2723302038 2024-03-21T10:31:57.415 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.408+0000 7f967ea00640 1 -- 172.21.15.47:0/843677982 <== mds.3 v2:172.21.15.100:6832/2723302038 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x5565639e6800 con 0x556563a8eca0 2024-03-21T10:31:57.415 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.408+0000 7f967ea00640 10 client.17616 handle_command_reply: tid=0 2024-03-21T10:31:57.415 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.408+0000 7f969a5ac740 1 client.17616 shutdown 2024-03-21T10:31:57.415 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.408+0000 7f969a5ac740 1 -- 172.21.15.47:0/843677982 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x556563a758b0 msgr2=0x556563a7e6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:31:57.415 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.408+0000 7f969a5ac740 1 --2- 172.21.15.47:0/843677982 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x556563a758b0 0x556563a7e6e0 secure :-1 s=READY pgs=983 cs=0 l=1 rev1=1 crypto rx=0x7f96800128a0 tx=0x7f9680004a40 comp rx=0 tx=0).stop 2024-03-21T10:31:57.415 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.408+0000 7f969a5ac740 1 -- 172.21.15.47:0/843677982 shutdown_connections 2024-03-21T10:31:57.415 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.408+0000 7f969a5ac740 1 --2- 172.21.15.47:0/843677982 >> [v2:172.21.15.100:6832/2723302038,v1:172.21.15.100:6833/2723302038] conn(0x556563a8eca0 0x556563a91090 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:31:57.415 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.409+0000 7f969a5ac740 1 --2- 172.21.15.47:0/843677982 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556563a7ede0 0x556563a7bff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:31:57.416 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.409+0000 7f967ea00640 0 client.17616 ms_handle_reset on v2:172.21.15.100:6832/2723302038 2024-03-21T10:31:57.416 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.409+0000 7f969a5ac740 1 --2- 172.21.15.47:0/843677982 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556563a76260 0x556563a7ba70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:31:57.416 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.409+0000 7f969a5ac740 1 --2- 172.21.15.47:0/843677982 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x556563a758b0 0x556563a7e6e0 unknown :-1 s=CLOSED pgs=983 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:31:57.416 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.409+0000 7f969a5ac740 1 -- 172.21.15.47:0/843677982 >> 172.21.15.47:0/843677982 conn(0x556563a6ff80 msgr2=0x556563a71cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:31:57.416 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.409+0000 7f969a5ac740 1 -- 172.21.15.47:0/843677982 shutdown_connections 2024-03-21T10:31:57.416 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.409+0000 7f969a5ac740 1 -- 172.21.15.47:0/843677982 wait complete. 2024-03-21T10:31:57.416 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.409+0000 7f969a5ac740 20 client.17616 trim_cache size 0 max 16384 2024-03-21T10:31:57.421 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.423+0000 7f969a5ac740 1 Processor -- start 2024-03-21T10:31:57.422 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.423+0000 7f969a5ac740 1 -- start start 2024-03-21T10:31:57.422 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.423+0000 7f969a5ac740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556563a758b0 0x556563a6e8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:31:57.422 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.423+0000 7f969a5ac740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x556563a76260 0x556563a7e1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:31:57.422 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.423+0000 7f969a5ac740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556563a7ede0 0x556563a7e6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:31:57.422 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.423+0000 7f969a5ac740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96901b0920 con 0x556563a76260 2024-03-21T10:31:57.422 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.423+0000 7f969a5ac740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96901b22a0 con 0x556563a758b0 2024-03-21T10:31:57.422 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.423+0000 7f969a5ac740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96901b0f80 con 0x556563a7ede0 2024-03-21T10:31:57.424 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.423+0000 7f9687e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x556563a76260 0x556563a7e1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:31:57.424 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.423+0000 7f9687e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x556563a76260 0x556563a7e1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:57746/0 (socket says 172.21.15.47:57746) 2024-03-21T10:31:57.424 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.423+0000 7f9687e00640 1 -- 172.21.15.47:0/1277959487 learned_addr learned my addr 172.21.15.47:0/1277959487 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:31:57.424 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.423+0000 7f9694e00640 1 --2- 172.21.15.47:0/1277959487 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556563a758b0 0x556563a6e8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:31:57.424 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.424+0000 7f9695800640 1 --2- 172.21.15.47:0/1277959487 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556563a7ede0 0x556563a7e6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:31:57.424 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.424+0000 7f9687e00640 1 -- 172.21.15.47:0/1277959487 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556563a7ede0 msgr2=0x556563a7e6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:31:57.424 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.424+0000 7f9687e00640 1 --2- 172.21.15.47:0/1277959487 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556563a7ede0 0x556563a7e6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:31:57.424 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.424+0000 7f9687e00640 1 -- 172.21.15.47:0/1277959487 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556563a758b0 msgr2=0x556563a6e8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:31:57.425 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.424+0000 7f9687e00640 1 --2- 172.21.15.47:0/1277959487 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556563a758b0 0x556563a6e8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:31:57.425 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.424+0000 7f9687e00640 1 -- 172.21.15.47:0/1277959487 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556563a91400 con 0x556563a76260 2024-03-21T10:31:57.425 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.424+0000 7f9687e00640 1 --2- 172.21.15.47:0/1277959487 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x556563a76260 0x556563a7e1a0 secure :-1 s=READY pgs=984 cs=0 l=1 rev1=1 crypto rx=0x7f96780ba070 tx=0x7f96780c4040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6d98192682d7b14a server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:31:57.425 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.424+0000 7f9685600640 1 -- 172.21.15.47:0/1277959487 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96780bd750 con 0x556563a76260 2024-03-21T10:31:57.425 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.425+0000 7f9685600640 1 -- 172.21.15.47:0/1277959487 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f96780be310 con 0x556563a76260 2024-03-21T10:31:57.425 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.425+0000 7f9685600640 1 -- 172.21.15.47:0/1277959487 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96780bd570 con 0x556563a76260 2024-03-21T10:31:57.425 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.425+0000 7f969a5ac740 1 -- 172.21.15.47:0/1277959487 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x556563a76260 msgr2=0x556563a7e1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:31:57.425 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.425+0000 7f969a5ac740 1 --2- 172.21.15.47:0/1277959487 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x556563a76260 0x556563a7e1a0 secure :-1 s=READY pgs=984 cs=0 l=1 rev1=1 crypto rx=0x7f96780ba070 tx=0x7f96780c4040 comp rx=0 tx=0).stop 2024-03-21T10:31:57.425 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.425+0000 7f969a5ac740 1 -- 172.21.15.47:0/1277959487 shutdown_connections 2024-03-21T10:31:57.425 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.425+0000 7f969a5ac740 1 --2- 172.21.15.47:0/1277959487 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556563a7ede0 0x556563a7e6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:31:57.425 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.425+0000 7f969a5ac740 1 --2- 172.21.15.47:0/1277959487 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x556563a76260 0x556563a7e1a0 unknown :-1 s=CLOSED pgs=984 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:31:57.425 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.425+0000 7f969a5ac740 1 --2- 172.21.15.47:0/1277959487 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556563a758b0 0x556563a6e8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:31:57.425 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.425+0000 7f969a5ac740 1 -- 172.21.15.47:0/1277959487 >> 172.21.15.47:0/1277959487 conn(0x556563a6ff80 msgr2=0x556563a71af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:31:57.425 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.425+0000 7f969a5ac740 1 -- 172.21.15.47:0/1277959487 shutdown_connections 2024-03-21T10:31:57.425 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.425+0000 7f969a5ac740 1 -- 172.21.15.47:0/1277959487 wait complete. 2024-03-21T10:31:57.426 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.426+0000 7f969a5ac740 1 -- start start 2024-03-21T10:31:57.432 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.426+0000 7f969a5ac740 1 Processor -- start 2024-03-21T10:31:57.432 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.426+0000 7f969a5ac740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x556563a758b0 0x556563a85770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:31:57.432 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.426+0000 7f969a5ac740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556563a76260 0x556563a85cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:31:57.432 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.426+0000 7f969a5ac740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556563a7ede0 0x556563a861f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:31:57.432 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.426+0000 7f969a5ac740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96901b1c40 con 0x556563a758b0 2024-03-21T10:31:57.432 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.426+0000 7f969a5ac740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96901b15e0 con 0x556563a7ede0 2024-03-21T10:31:57.433 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.426+0000 7f969a5ac740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96901aeae0 con 0x556563a76260 2024-03-21T10:31:57.433 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.427+0000 7f969a5ac740 10 client.0 _ll_register_callbacks cb 0x7f9680012520 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T10:31:57.433 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.427+0000 7f9694e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x556563a758b0 0x556563a85770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:31:57.433 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.427+0000 7f9694e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x556563a758b0 0x556563a85770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:57760/0 (socket says 172.21.15.47:57760) 2024-03-21T10:31:57.433 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.427+0000 7f9694e00640 1 -- 172.21.15.47:0/3335521809 learned_addr learned my addr 172.21.15.47:0/3335521809 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:31:57.433 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.427+0000 7f9695800640 1 --2- 172.21.15.47:0/3335521809 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556563a7ede0 0x556563a861f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:31:57.433 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.427+0000 7f9694e00640 1 -- 172.21.15.47:0/3335521809 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556563a76260 msgr2=0x556563a85cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:31:57.433 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.427+0000 7f9687e00640 1 --2- 172.21.15.47:0/3335521809 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556563a76260 0x556563a85cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:31:57.433 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.427+0000 7f9694e00640 1 --2- 172.21.15.47:0/3335521809 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556563a76260 0x556563a85cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:31:57.433 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.427+0000 7f9694e00640 1 -- 172.21.15.47:0/3335521809 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556563a7ede0 msgr2=0x556563a861f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:31:57.433 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.427+0000 7f9694e00640 1 --2- 172.21.15.47:0/3335521809 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556563a7ede0 0x556563a861f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:31:57.433 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.427+0000 7f9694e00640 1 -- 172.21.15.47:0/3335521809 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x556563aea6f0 con 0x556563a758b0 2024-03-21T10:31:57.433 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.427+0000 7f9687e00640 1 --2- 172.21.15.47:0/3335521809 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556563a76260 0x556563a85cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T10:31:57.433 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.427+0000 7f9694e00640 1 --2- 172.21.15.47:0/3335521809 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x556563a758b0 0x556563a85770 secure :-1 s=READY pgs=985 cs=0 l=1 rev1=1 crypto rx=0x7f968001d490 tx=0x7f9680000f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fff323f5e7b31fc9 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:31:57.434 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.428+0000 7f967ea00640 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-03-21T10:31:57.434 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.428+0000 7f967ea00640 1 -- 172.21.15.47:0/3335521809 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9680011c90 con 0x556563a758b0 2024-03-21T10:31:57.434 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.428+0000 7f967ea00640 1 -- 172.21.15.47:0/3335521809 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(97..97 src has 1..97) v4 ==== 9474+0+0 (secure 0 0 0) 0x7f968001dcc0 con 0x556563a758b0 2024-03-21T10:31:57.434 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.428+0000 7f969a5ac740 1 -- 172.21.15.47:0/3335521809 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f96780be310 con 0x556563a758b0 2024-03-21T10:31:57.434 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.428+0000 7f967ea00640 1 -- 172.21.15.47:0/3335521809 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9680012e20 con 0x556563a758b0 2024-03-21T10:31:57.434 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.428+0000 7f969a5ac740 10 client.17622 fetch_fsmap learned FSMap version 32 2024-03-21T10:31:57.434 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.428+0000 7f969a5ac740 1 -- 172.21.15.47:0/3335521809 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x556563aeb9d0 con 0x556563a758b0 2024-03-21T10:31:57.434 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.428+0000 7f967ea00640 1 -- 172.21.15.47:0/3335521809 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f96780be310 con 0x556563a758b0 2024-03-21T10:31:57.434 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.429+0000 7f967ea00640 1 -- 172.21.15.47:0/3335521809 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 32) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f968003c070 con 0x556563a758b0 2024-03-21T10:31:57.434 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.429+0000 7f969a5ac740 10 client.17622 fetch_fsmap finished waiting for FSMap version 32 2024-03-21T10:31:57.434 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.429+0000 7f969a5ac740 10 client.17622 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.k 2024-03-21T10:31:57.434 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.429+0000 7f969a5ac740 20 client.17622 populate_metadata read hostname 'smithi047' 2024-03-21T10:31:57.434 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.429+0000 7f969a5ac740 1 --2- 172.21.15.47:0/3335521809 >> [v2:172.21.15.100:6832/2723302038,v1:172.21.15.100:6833/2723302038] conn(0x556563aec0c0 0x556563aee4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:31:57.434 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.429+0000 7f969a5ac740 4 client.17622 mds_command: new command op to 34427 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-21T10:31:57.434 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.429+0000 7f969a5ac740 1 -- 172.21.15.47:0/3335521809 --> [v2:172.21.15.100:6832/2723302038,v1:172.21.15.100:6833/2723302038] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x5565639e0b50 con 0x556563aec0c0 2024-03-21T10:31:57.435 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.429+0000 7f9687e00640 1 --2- 172.21.15.47:0/3335521809 >> [v2:172.21.15.100:6832/2723302038,v1:172.21.15.100:6833/2723302038] conn(0x556563aec0c0 0x556563aee4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:31:57.435 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.430+0000 7f9687e00640 1 --2- 172.21.15.47:0/3335521809 >> [v2:172.21.15.100:6832/2723302038,v1:172.21.15.100:6833/2723302038] conn(0x556563aec0c0 0x556563aee4e0 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=5f336122cf91ef5 server_cookie=501035d2f03be0b7 in_seq=0 out_seq=0 2024-03-21T10:31:57.435 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.430+0000 7f967ea00640 10 client.17622 ms_handle_connect on v2:172.21.15.100:6832/2723302038 2024-03-21T10:31:57.435 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.430+0000 7f967ea00640 1 -- 172.21.15.47:0/3335521809 <== mds.3 v2:172.21.15.100:6832/2723302038 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x5565639e0b50 con 0x556563aec0c0 2024-03-21T10:31:57.435 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.430+0000 7f967ea00640 10 client.17622 handle_command_reply: tid=0 2024-03-21T10:31:57.435 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.430+0000 7f969a5ac740 1 client.17622 shutdown 2024-03-21T10:31:57.435 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.430+0000 7f969a5ac740 1 -- 172.21.15.47:0/3335521809 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x556563a758b0 msgr2=0x556563a85770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:31:57.435 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.430+0000 7f969a5ac740 1 --2- 172.21.15.47:0/3335521809 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x556563a758b0 0x556563a85770 secure :-1 s=READY pgs=985 cs=0 l=1 rev1=1 crypto rx=0x7f968001d490 tx=0x7f9680000f40 comp rx=0 tx=0).stop 2024-03-21T10:31:57.435 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.430+0000 7f969a5ac740 1 -- 172.21.15.47:0/3335521809 shutdown_connections 2024-03-21T10:31:57.435 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.430+0000 7f969a5ac740 1 --2- 172.21.15.47:0/3335521809 >> [v2:172.21.15.100:6832/2723302038,v1:172.21.15.100:6833/2723302038] conn(0x556563aec0c0 0x556563aee4e0 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:31:57.435 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.430+0000 7f969a5ac740 1 --2- 172.21.15.47:0/3335521809 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556563a7ede0 0x556563a861f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:31:57.435 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.430+0000 7f969a5ac740 1 --2- 172.21.15.47:0/3335521809 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556563a76260 0x556563a85cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:31:57.435 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.430+0000 7f969a5ac740 1 --2- 172.21.15.47:0/3335521809 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x556563a758b0 0x556563a85770 unknown :-1 s=CLOSED pgs=985 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:31:57.435 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.430+0000 7f967ea00640 0 client.17622 ms_handle_reset on v2:172.21.15.100:6832/2723302038 2024-03-21T10:31:57.436 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.430+0000 7f969a5ac740 1 -- 172.21.15.47:0/3335521809 >> 172.21.15.47:0/3335521809 conn(0x556563a6ff80 msgr2=0x556563a76640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:31:57.436 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.430+0000 7f969a5ac740 1 -- 172.21.15.47:0/3335521809 shutdown_connections 2024-03-21T10:31:57.436 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.430+0000 7f969a5ac740 1 -- 172.21.15.47:0/3335521809 wait complete. 2024-03-21T10:31:57.436 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.431+0000 7f969a5ac740 20 client.17622 trim_cache size 0 max 16384 2024-03-21T10:31:57.436 INFO:teuthology.orchestra.run.smithi047.stdout:[] 2024-03-21T10:31:57.437 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.431+0000 7f9696200640 1 -- 172.21.15.47:0/2977812504 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f96740a3090 msgr2=0x7f96740a5550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:31:57.437 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.431+0000 7f9696200640 1 --2- 172.21.15.47:0/2977812504 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f96740a3090 0x7f96740a5550 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x556563a70ff0 tx=0x7f9680009580 comp rx=0 tx=0).stop 2024-03-21T10:31:57.437 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.431+0000 7f9696200640 1 -- 172.21.15.47:0/2977812504 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9690102ff0 msgr2=0x7f969019a510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:31:57.437 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.431+0000 7f9696200640 1 --2- 172.21.15.47:0/2977812504 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9690102ff0 0x7f969019a510 secure :-1 s=READY pgs=981 cs=0 l=1 rev1=1 crypto rx=0x7f9678000c00 tx=0x7f967802c6d0 comp rx=0 tx=0).stop 2024-03-21T10:31:57.437 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.431+0000 7f9696200640 1 -- 172.21.15.47:0/2977812504 shutdown_connections 2024-03-21T10:31:57.437 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.431+0000 7f9696200640 1 --2- 172.21.15.47:0/2977812504 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f96740a3090 0x7f96740a5550 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:31:57.437 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.431+0000 7f9696200640 1 --2- 172.21.15.47:0/2977812504 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f969010baf0 0x7f969019e890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:31:57.437 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.431+0000 7f9696200640 1 --2- 172.21.15.47:0/2977812504 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9690102ff0 0x7f969019a510 unknown :-1 s=CLOSED pgs=981 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:31:57.437 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.431+0000 7f9696200640 1 --2- 172.21.15.47:0/2977812504 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f96901026d0 0x7f9690199fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:31:57.437 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.431+0000 7f9696200640 1 -- 172.21.15.47:0/2977812504 >> 172.21.15.47:0/2977812504 conn(0x7f96900fbef0 msgr2=0x7f969010cd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:31:57.437 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.432+0000 7f9696200640 1 -- 172.21.15.47:0/2977812504 shutdown_connections 2024-03-21T10:31:57.438 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.432+0000 7f9696200640 1 -- 172.21.15.47:0/2977812504 wait complete. 2024-03-21T10:31:57.447 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-21T10:31:57.448 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-03-21T10:31:57.673 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.674+0000 7f1514a00640 1 -- 172.21.15.47:0/334527086 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1510102ff0 msgr2=0x7f151010b4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:31:57.673 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.674+0000 7f1514a00640 1 --2- 172.21.15.47:0/334527086 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1510102ff0 0x7f151010b4d0 secure :-1 s=READY pgs=986 cs=0 l=1 rev1=1 crypto rx=0x7f14f8009900 tx=0x7f14f802bcc0 comp rx=0 tx=0).stop 2024-03-21T10:31:57.673 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.674+0000 7f1514a00640 1 -- 172.21.15.47:0/334527086 shutdown_connections 2024-03-21T10:31:57.674 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.674+0000 7f1514a00640 1 --2- 172.21.15.47:0/334527086 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f151010baf0 0x7f151010df90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:31:57.674 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.674+0000 7f1514a00640 1 --2- 172.21.15.47:0/334527086 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1510102ff0 0x7f151010b4d0 unknown :-1 s=CLOSED pgs=986 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:31:57.674 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.674+0000 7f1514a00640 1 --2- 172.21.15.47:0/334527086 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f15101026d0 0x7f1510102ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:31:57.674 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.674+0000 7f1514a00640 1 -- 172.21.15.47:0/334527086 >> 172.21.15.47:0/334527086 conn(0x7f15100fbef0 msgr2=0x7f15100fe310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:31:57.674 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.674+0000 7f1514a00640 1 -- 172.21.15.47:0/334527086 shutdown_connections 2024-03-21T10:31:57.674 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.674+0000 7f1514a00640 1 -- 172.21.15.47:0/334527086 wait complete. 2024-03-21T10:31:57.675 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.675+0000 7f1514a00640 1 Processor -- start 2024-03-21T10:31:57.675 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.675+0000 7f1514a00640 1 -- start start 2024-03-21T10:31:57.675 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.676+0000 7f1514a00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f15101026d0 0x7f1510197e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:31:57.676 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.676+0000 7f1514a00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1510102ff0 0x7f1510198360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:31:57.676 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.676+0000 7f1514a00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f151010baf0 0x7f151019c690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:31:57.676 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.676+0000 7f1514a00640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15101117d0 con 0x7f151010baf0 2024-03-21T10:31:57.676 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.676+0000 7f1514a00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1510111650 con 0x7f1510102ff0 2024-03-21T10:31:57.676 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.676+0000 7f1514a00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1510111950 con 0x7f15101026d0 2024-03-21T10:31:57.676 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.676+0000 7f150f200640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f151010baf0 0x7f151019c690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:31:57.676 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.676+0000 7f150f200640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f151010baf0 0x7f151019c690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:57778/0 (socket says 172.21.15.47:57778) 2024-03-21T10:31:57.676 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.676+0000 7f150f200640 1 -- 172.21.15.47:0/2338147875 learned_addr learned my addr 172.21.15.47:0/2338147875 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:31:57.676 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.676+0000 7f1505e00640 1 --2- 172.21.15.47:0/2338147875 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1510102ff0 0x7f1510198360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:31:57.676 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.676+0000 7f150e800640 1 --2- 172.21.15.47:0/2338147875 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f15101026d0 0x7f1510197e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:31:57.676 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.677+0000 7f150f200640 1 -- 172.21.15.47:0/2338147875 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f15101026d0 msgr2=0x7f1510197e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:31:57.676 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.677+0000 7f150f200640 1 --2- 172.21.15.47:0/2338147875 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f15101026d0 0x7f1510197e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:31:57.676 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.677+0000 7f150f200640 1 -- 172.21.15.47:0/2338147875 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1510102ff0 msgr2=0x7f1510198360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:31:57.676 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.677+0000 7f150f200640 1 --2- 172.21.15.47:0/2338147875 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1510102ff0 0x7f1510198360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:31:57.676 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.677+0000 7f150f200640 1 -- 172.21.15.47:0/2338147875 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f151019cdc0 con 0x7f151010baf0 2024-03-21T10:31:57.677 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.677+0000 7f150f200640 1 --2- 172.21.15.47:0/2338147875 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f151010baf0 0x7f151019c690 secure :-1 s=READY pgs=987 cs=0 l=1 rev1=1 crypto rx=0x7f150800eaa0 tx=0x7f150800ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:31:57.677 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.677+0000 7f1507e00640 1 -- 172.21.15.47:0/2338147875 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f150800cdb0 con 0x7f151010baf0 2024-03-21T10:31:57.677 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.677+0000 7f1507e00640 1 -- 172.21.15.47:0/2338147875 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f1508005020 con 0x7f151010baf0 2024-03-21T10:31:57.677 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.678+0000 7f1507e00640 1 -- 172.21.15.47:0/2338147875 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15080116d0 con 0x7f151010baf0 2024-03-21T10:31:57.678 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.678+0000 7f1514a00640 1 -- 172.21.15.47:0/2338147875 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f151019d0b0 con 0x7f151010baf0 2024-03-21T10:31:57.678 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.678+0000 7f1514a00640 1 -- 172.21.15.47:0/2338147875 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f15101a4940 con 0x7f151010baf0 2024-03-21T10:31:57.680 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.682+0000 7f15181c8740 1 Processor -- start 2024-03-21T10:31:57.681 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.682+0000 7f1507e00640 1 -- 172.21.15.47:0/2338147875 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 27) v1 ==== 137985+0+0 (secure 0 0 0) 0x7f15080051d0 con 0x7f151010baf0 2024-03-21T10:31:57.681 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.682+0000 7f15181c8740 1 -- start start 2024-03-21T10:31:57.682 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.683+0000 7f15181c8740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55922045b630 0x55922045ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:31:57.682 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.683+0000 7f15181c8740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55922045bfe0 0x559220468c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:31:57.682 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.683+0000 7f15181c8740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5592204691f0 0x55922046b600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:31:57.682 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.683+0000 7f15181c8740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x5592203bf2c0 con 0x55922045bfe0 2024-03-21T10:31:57.682 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.683+0000 7f15181c8740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x559220367c90 con 0x55922045b630 2024-03-21T10:31:57.682 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.683+0000 7f15181c8740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x5592203bb360 con 0x5592204691f0 2024-03-21T10:31:57.682 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.683+0000 7f150e800640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55922045bfe0 0x559220468c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:31:57.683 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.683+0000 7f1505e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55922045b630 0x55922045ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:31:57.683 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.683+0000 7f150e800640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55922045bfe0 0x559220468c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:57780/0 (socket says 172.21.15.47:57780) 2024-03-21T10:31:57.683 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.683+0000 7f150e800640 1 -- 172.21.15.47:0/3583764911 learned_addr learned my addr 172.21.15.47:0/3583764911 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:31:57.683 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.683+0000 7f1505e00640 1 --2- 172.21.15.47:0/3583764911 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5592204691f0 0x55922046b600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:31:57.683 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.684+0000 7f150e800640 1 -- 172.21.15.47:0/3583764911 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5592204691f0 msgr2=0x55922046b600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:31:57.683 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.684+0000 7f150e800640 1 --2- 172.21.15.47:0/3583764911 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5592204691f0 0x55922046b600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:31:57.683 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.684+0000 7f150e800640 1 -- 172.21.15.47:0/3583764911 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55922045b630 msgr2=0x55922045ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:31:57.683 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.684+0000 7f150e800640 1 --2- 172.21.15.47:0/3583764911 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55922045b630 0x55922045ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:31:57.683 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.684+0000 7f150e800640 1 -- 172.21.15.47:0/3583764911 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55922038c080 con 0x55922045bfe0 2024-03-21T10:31:57.683 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.684+0000 7f1505e00640 1 --2- 172.21.15.47:0/3583764911 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55922045b630 0x55922045ba10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T10:31:57.685 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.684+0000 7f150e800640 1 --2- 172.21.15.47:0/3583764911 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55922045bfe0 0x559220468c20 secure :-1 s=READY pgs=988 cs=0 l=1 rev1=1 crypto rx=0x7f1500009cf0 tx=0x7f1500011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d44c5f5872e28ba server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:31:57.685 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.684+0000 7f14ff400640 1 -- 172.21.15.47:0/3583764911 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1500004230 con 0x55922045bfe0 2024-03-21T10:31:57.685 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.684+0000 7f14ff400640 1 -- 172.21.15.47:0/3583764911 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f15000043d0 con 0x55922045bfe0 2024-03-21T10:31:57.685 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.684+0000 7f14ff400640 1 -- 172.21.15.47:0/3583764911 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15000197e0 con 0x55922045bfe0 2024-03-21T10:31:57.685 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.685+0000 7f1507e00640 1 --2- 172.21.15.47:0/2338147875 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f14f40a3110 0x7f14f40a55d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:31:57.685 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.685+0000 7f1507e00640 1 -- 172.21.15.47:0/2338147875 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 9474+0+0 (secure 0 0 0) 0x7f15080cfc20 con 0x7f151010baf0 2024-03-21T10:31:57.685 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.685+0000 7f1505e00640 1 --2- 172.21.15.47:0/2338147875 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f14f40a3110 0x7f14f40a55d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:31:57.685 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.685+0000 7f15181c8740 1 -- 172.21.15.47:0/3583764911 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55922045bfe0 msgr2=0x559220468c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:31:57.685 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.685+0000 7f15181c8740 1 --2- 172.21.15.47:0/3583764911 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55922045bfe0 0x559220468c20 secure :-1 s=READY pgs=988 cs=0 l=1 rev1=1 crypto rx=0x7f1500009cf0 tx=0x7f1500011810 comp rx=0 tx=0).stop 2024-03-21T10:31:57.685 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.685+0000 7f15181c8740 1 -- 172.21.15.47:0/3583764911 shutdown_connections 2024-03-21T10:31:57.686 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.685+0000 7f15181c8740 1 --2- 172.21.15.47:0/3583764911 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5592204691f0 0x55922046b600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:31:57.686 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.685+0000 7f15181c8740 1 --2- 172.21.15.47:0/3583764911 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55922045bfe0 0x559220468c20 unknown :-1 s=CLOSED pgs=988 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:31:57.686 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.685+0000 7f15181c8740 1 --2- 172.21.15.47:0/3583764911 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55922045b630 0x55922045ba10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:31:57.686 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.685+0000 7f15181c8740 1 -- 172.21.15.47:0/3583764911 >> 172.21.15.47:0/3583764911 conn(0x559220455f80 msgr2=0x559220456360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:31:57.686 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.686+0000 7f15181c8740 1 -- 172.21.15.47:0/3583764911 shutdown_connections 2024-03-21T10:31:57.686 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.686+0000 7f15181c8740 1 -- 172.21.15.47:0/3583764911 wait complete. 2024-03-21T10:31:57.686 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.686+0000 7f1505e00640 1 --2- 172.21.15.47:0/2338147875 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f14f40a3110 0x7f14f40a55d0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f14f8005ab0 tx=0x7f14f8005e20 comp rx=0 tx=0).ready entity=mgr.15687 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:31:57.686 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.686+0000 7f15181c8740 1 -- start start 2024-03-21T10:31:57.686 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.686+0000 7f15181c8740 1 Processor -- start 2024-03-21T10:31:57.689 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.687+0000 7f15181c8740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55922045b630 0x559220468a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:31:57.689 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.687+0000 7f15181c8740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55922045bfe0 0x559220465dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:31:57.689 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.687+0000 7f15181c8740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5592204691f0 0x559220466310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:31:57.689 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.687+0000 7f15181c8740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x55922046c120 con 0x5592204691f0 2024-03-21T10:31:57.689 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.687+0000 7f15181c8740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55922037b0d0 con 0x55922045bfe0 2024-03-21T10:31:57.689 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.687+0000 7f15181c8740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55922046c2a0 con 0x55922045b630 2024-03-21T10:31:57.689 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.687+0000 7f15181c8740 10 client.0 _ll_register_callbacks cb 0x55922037fe80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T10:31:57.689 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.687+0000 7f150f200640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5592204691f0 0x559220466310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:31:57.690 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.687+0000 7f150f200640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5592204691f0 0x559220466310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:57788/0 (socket says 172.21.15.47:57788) 2024-03-21T10:31:57.690 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.687+0000 7f150f200640 1 -- 172.21.15.47:0/2057659405 learned_addr learned my addr 172.21.15.47:0/2057659405 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:31:57.690 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.687+0000 7f1505e00640 1 --2- 172.21.15.47:0/2057659405 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55922045bfe0 0x559220465dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:31:57.690 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.687+0000 7f150e800640 1 --2- 172.21.15.47:0/2057659405 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55922045b630 0x559220468a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:31:57.690 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.688+0000 7f150f200640 1 -- 172.21.15.47:0/2057659405 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55922045b630 msgr2=0x559220468a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:31:57.690 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.688+0000 7f150f200640 1 --2- 172.21.15.47:0/2057659405 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55922045b630 0x559220468a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:31:57.690 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.688+0000 7f150f200640 1 -- 172.21.15.47:0/2057659405 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55922045bfe0 msgr2=0x559220465dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:31:57.690 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.688+0000 7f150f200640 1 --2- 172.21.15.47:0/2057659405 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55922045bfe0 0x559220465dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:31:57.690 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.688+0000 7f150f200640 1 -- 172.21.15.47:0/2057659405 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5592200d33f0 con 0x5592204691f0 2024-03-21T10:31:57.690 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.688+0000 7f150f200640 1 --2- 172.21.15.47:0/2057659405 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5592204691f0 0x559220466310 secure :-1 s=READY pgs=989 cs=0 l=1 rev1=1 crypto rx=0x7f1508098670 tx=0x7f150809bdd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b962abea14813afd server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:31:57.690 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.688+0000 7f14f3e00640 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-03-21T10:31:57.690 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.688+0000 7f14f3e00640 1 -- 172.21.15.47:0/2057659405 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f150801f070 con 0x5592204691f0 2024-03-21T10:31:57.690 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.688+0000 7f15181c8740 1 -- 172.21.15.47:0/2057659405 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f15000043d0 con 0x5592204691f0 2024-03-21T10:31:57.691 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.688+0000 7f14f3e00640 1 -- 172.21.15.47:0/2057659405 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(97..97 src has 1..97) v4 ==== 9474+0+0 (secure 0 0 0) 0x7f150809b0f0 con 0x5592204691f0 2024-03-21T10:31:57.691 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.689+0000 7f14f3e00640 1 -- 172.21.15.47:0/2057659405 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f150809a060 con 0x5592204691f0 2024-03-21T10:31:57.691 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.689+0000 7f15181c8740 10 client.17634 fetch_fsmap learned FSMap version 32 2024-03-21T10:31:57.691 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.689+0000 7f15181c8740 1 -- 172.21.15.47:0/2057659405 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x55922019d7d0 con 0x5592204691f0 2024-03-21T10:31:57.691 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.689+0000 7f14f3e00640 1 -- 172.21.15.47:0/2057659405 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f15000043d0 con 0x5592204691f0 2024-03-21T10:31:57.691 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.689+0000 7f14f3e00640 1 -- 172.21.15.47:0/2057659405 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 32) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f150800f040 con 0x5592204691f0 2024-03-21T10:31:57.691 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.690+0000 7f15181c8740 10 client.17634 fetch_fsmap finished waiting for FSMap version 32 2024-03-21T10:31:57.691 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.690+0000 7f15181c8740 10 client.17634 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.e 2024-03-21T10:31:57.691 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.690+0000 7f15181c8740 20 client.17634 populate_metadata read hostname 'smithi047' 2024-03-21T10:31:57.691 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.690+0000 7f15181c8740 1 --2- 172.21.15.47:0/2057659405 >> [v2:172.21.15.100:6834/2597849957,v1:172.21.15.100:6835/2597849957] conn(0x559220479010 0x55922047b440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:31:57.691 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.690+0000 7f15181c8740 4 client.17634 mds_command: new command op to 34454 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T10:31:57.691 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.690+0000 7f15181c8740 1 -- 172.21.15.47:0/2057659405 --> [v2:172.21.15.100:6834/2597849957,v1:172.21.15.100:6835/2597849957] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5592203cc800 con 0x559220479010 2024-03-21T10:31:57.691 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.690+0000 7f150e800640 1 --2- 172.21.15.47:0/2057659405 >> [v2:172.21.15.100:6834/2597849957,v1:172.21.15.100:6835/2597849957] conn(0x559220479010 0x55922047b440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:31:57.692 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.691+0000 7f150e800640 1 --2- 172.21.15.47:0/2057659405 >> [v2:172.21.15.100:6834/2597849957,v1:172.21.15.100:6835/2597849957] conn(0x559220479010 0x55922047b440 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=c12db4363a711040 server_cookie=15bd87e6c554d5ba in_seq=0 out_seq=0 2024-03-21T10:31:57.693 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.691+0000 7f14f3e00640 10 client.17634 ms_handle_connect on v2:172.21.15.100:6834/2597849957 2024-03-21T10:31:57.693 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.692+0000 7f14f3e00640 1 -- 172.21.15.47:0/2057659405 <== mds.2 v2:172.21.15.100:6834/2597849957 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x5592203cc800 con 0x559220479010 2024-03-21T10:31:57.693 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.692+0000 7f14f3e00640 10 client.17634 handle_command_reply: tid=0 2024-03-21T10:31:57.693 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.692+0000 7f15181c8740 1 client.17634 shutdown 2024-03-21T10:31:57.693 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.693+0000 7f15181c8740 1 -- 172.21.15.47:0/2057659405 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5592204691f0 msgr2=0x559220466310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:31:57.693 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.693+0000 7f15181c8740 1 --2- 172.21.15.47:0/2057659405 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5592204691f0 0x559220466310 secure :-1 s=READY pgs=989 cs=0 l=1 rev1=1 crypto rx=0x7f1508098670 tx=0x7f150809bdd0 comp rx=0 tx=0).stop 2024-03-21T10:31:57.693 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.693+0000 7f15181c8740 1 -- 172.21.15.47:0/2057659405 shutdown_connections 2024-03-21T10:31:57.693 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.693+0000 7f15181c8740 1 --2- 172.21.15.47:0/2057659405 >> [v2:172.21.15.100:6834/2597849957,v1:172.21.15.100:6835/2597849957] conn(0x559220479010 0x55922047b440 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:31:57.693 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.693+0000 7f15181c8740 1 --2- 172.21.15.47:0/2057659405 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5592204691f0 0x559220466310 unknown :-1 s=CLOSED pgs=989 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:31:57.693 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.693+0000 7f15181c8740 1 --2- 172.21.15.47:0/2057659405 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55922045bfe0 0x559220465dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:31:57.693 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.693+0000 7f14f3e00640 0 client.17634 ms_handle_reset on v2:172.21.15.100:6834/2597849957 2024-03-21T10:31:57.693 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.693+0000 7f15181c8740 1 --2- 172.21.15.47:0/2057659405 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55922045b630 0x559220468a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:31:57.693 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.693+0000 7f15181c8740 1 -- 172.21.15.47:0/2057659405 >> 172.21.15.47:0/2057659405 conn(0x559220455f80 msgr2=0x559220457cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:31:57.694 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.693+0000 7f15181c8740 1 -- 172.21.15.47:0/2057659405 shutdown_connections 2024-03-21T10:31:57.694 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.693+0000 7f15181c8740 1 -- 172.21.15.47:0/2057659405 wait complete. 2024-03-21T10:31:57.694 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.693+0000 7f15181c8740 20 client.17634 trim_cache size 0 max 16384 2024-03-21T10:31:57.706 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.707+0000 7f15181c8740 1 Processor -- start 2024-03-21T10:31:57.706 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.707+0000 7f15181c8740 1 -- start start 2024-03-21T10:31:57.706 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.708+0000 7f15181c8740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55922045b630 0x559220468500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:31:57.707 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.708+0000 7f15181c8740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55922045bfe0 0x559220468a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:31:57.707 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.708+0000 7f15181c8740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5592204691f0 0x559220463050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:31:57.707 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.708+0000 7f15181c8740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15101ac5b0 con 0x5592204691f0 2024-03-21T10:31:57.707 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.708+0000 7f15181c8740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15101adf30 con 0x55922045b630 2024-03-21T10:31:57.707 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.708+0000 7f15181c8740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15101acc10 con 0x55922045bfe0 2024-03-21T10:31:57.708 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.708+0000 7f150f200640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5592204691f0 0x559220463050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:31:57.708 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.708+0000 7f150f200640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5592204691f0 0x559220463050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:57802/0 (socket says 172.21.15.47:57802) 2024-03-21T10:31:57.708 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.708+0000 7f150e800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55922045b630 0x559220468500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:31:57.708 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.708+0000 7f150f200640 1 -- 172.21.15.47:0/2944214732 learned_addr learned my addr 172.21.15.47:0/2944214732 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:31:57.708 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.708+0000 7f1505e00640 1 --2- 172.21.15.47:0/2944214732 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55922045bfe0 0x559220468a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:31:57.708 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.708+0000 7f150f200640 1 -- 172.21.15.47:0/2944214732 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55922045bfe0 msgr2=0x559220468a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:31:57.708 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.709+0000 7f150f200640 1 --2- 172.21.15.47:0/2944214732 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55922045bfe0 0x559220468a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:31:57.708 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.709+0000 7f150f200640 1 -- 172.21.15.47:0/2944214732 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55922045b630 msgr2=0x559220468500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:31:57.709 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.709+0000 7f150f200640 1 --2- 172.21.15.47:0/2944214732 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55922045b630 0x559220468500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:31:57.709 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.709+0000 7f150f200640 1 -- 172.21.15.47:0/2944214732 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559220455ab0 con 0x5592204691f0 2024-03-21T10:31:57.709 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.709+0000 7f1505e00640 1 --2- 172.21.15.47:0/2944214732 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55922045bfe0 0x559220468a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T10:31:57.709 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.709+0000 7f150f200640 1 --2- 172.21.15.47:0/2944214732 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5592204691f0 0x559220463050 secure :-1 s=READY pgs=990 cs=0 l=1 rev1=1 crypto rx=0x7f150800efd0 tx=0x7f15080a4740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d3aa81036f517f1e server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:31:57.709 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.709+0000 7f14fea00640 1 -- 172.21.15.47:0/2944214732 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f150807f070 con 0x5592204691f0 2024-03-21T10:31:57.709 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.709+0000 7f14fea00640 1 -- 172.21.15.47:0/2944214732 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f1508098310 con 0x5592204691f0 2024-03-21T10:31:57.709 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.709+0000 7f14fea00640 1 -- 172.21.15.47:0/2944214732 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15080b8070 con 0x5592204691f0 2024-03-21T10:31:57.711 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.710+0000 7f15181c8740 1 -- 172.21.15.47:0/2944214732 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5592204691f0 msgr2=0x559220463050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:31:57.711 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.710+0000 7f15181c8740 1 --2- 172.21.15.47:0/2944214732 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5592204691f0 0x559220463050 secure :-1 s=READY pgs=990 cs=0 l=1 rev1=1 crypto rx=0x7f150800efd0 tx=0x7f15080a4740 comp rx=0 tx=0).stop 2024-03-21T10:31:57.711 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.710+0000 7f15181c8740 1 -- 172.21.15.47:0/2944214732 shutdown_connections 2024-03-21T10:31:57.711 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.710+0000 7f15181c8740 1 --2- 172.21.15.47:0/2944214732 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5592204691f0 0x559220463050 unknown :-1 s=CLOSED pgs=990 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:31:57.711 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.710+0000 7f15181c8740 1 --2- 172.21.15.47:0/2944214732 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55922045bfe0 0x559220468a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:31:57.712 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.710+0000 7f15181c8740 1 --2- 172.21.15.47:0/2944214732 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55922045b630 0x559220468500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:31:57.712 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.710+0000 7f15181c8740 1 -- 172.21.15.47:0/2944214732 >> 172.21.15.47:0/2944214732 conn(0x559220455f80 msgr2=0x559220461450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:31:57.712 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.710+0000 7f15181c8740 1 -- 172.21.15.47:0/2944214732 shutdown_connections 2024-03-21T10:31:57.712 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.710+0000 7f15181c8740 1 -- 172.21.15.47:0/2944214732 wait complete. 2024-03-21T10:31:57.712 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.711+0000 7f15181c8740 1 -- start start 2024-03-21T10:31:57.712 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.711+0000 7f15181c8740 1 Processor -- start 2024-03-21T10:31:57.712 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.711+0000 7f15181c8740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55922045b630 0x55922046d000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:31:57.712 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.711+0000 7f15181c8740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55922045bfe0 0x55922046d540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:31:57.712 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.711+0000 7f15181c8740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5592204691f0 0x5592204d38d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:31:57.712 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.711+0000 7f15181c8740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15101ad8d0 con 0x55922045b630 2024-03-21T10:31:57.712 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.711+0000 7f15181c8740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15101ad270 con 0x55922045bfe0 2024-03-21T10:31:57.712 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.711+0000 7f15181c8740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15101aa770 con 0x5592204691f0 2024-03-21T10:31:57.712 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.712+0000 7f15181c8740 10 client.0 _ll_register_callbacks cb 0x7f1508005830 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T10:31:57.712 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.712+0000 7f150e800640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55922045b630 0x55922046d000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:31:57.713 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.712+0000 7f150e800640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55922045b630 0x55922046d000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:57808/0 (socket says 172.21.15.47:57808) 2024-03-21T10:31:57.713 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.712+0000 7f150e800640 1 -- 172.21.15.47:0/1397134557 learned_addr learned my addr 172.21.15.47:0/1397134557 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:31:57.713 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.712+0000 7f1505e00640 1 --2- 172.21.15.47:0/1397134557 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55922045bfe0 0x55922046d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:31:57.713 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.712+0000 7f150f200640 1 --2- 172.21.15.47:0/1397134557 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5592204691f0 0x5592204d38d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:31:57.713 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.712+0000 7f150e800640 1 -- 172.21.15.47:0/1397134557 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5592204691f0 msgr2=0x5592204d38d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:31:57.713 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.712+0000 7f150e800640 1 --2- 172.21.15.47:0/1397134557 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5592204691f0 0x5592204d38d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:31:57.713 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.712+0000 7f150e800640 1 -- 172.21.15.47:0/1397134557 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55922045bfe0 msgr2=0x55922046d540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:31:57.713 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.712+0000 7f150e800640 1 --2- 172.21.15.47:0/1397134557 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55922045bfe0 0x55922046d540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:31:57.713 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.712+0000 7f150e800640 1 -- 172.21.15.47:0/1397134557 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5592204d3e10 con 0x55922045b630 2024-03-21T10:31:57.713 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.712+0000 7f150f200640 1 --2- 172.21.15.47:0/1397134557 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5592204691f0 0x5592204d38d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T10:31:57.713 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.712+0000 7f1505e00640 1 --2- 172.21.15.47:0/1397134557 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55922045bfe0 0x55922046d540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T10:31:57.714 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:31:57 smithi047 ceph-mon[31547]: pgmap v202: 129 pgs: 129 active+clean; 277 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 12 op/s 2024-03-21T10:31:57.714 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:31:57 smithi047 ceph-mon[31547]: from='client.? 172.21.15.47:0/230257554' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-21T10:31:57.716 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.713+0000 7f150e800640 1 --2- 172.21.15.47:0/1397134557 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55922045b630 0x55922046d000 secure :-1 s=READY pgs=991 cs=0 l=1 rev1=1 crypto rx=0x7f1500009c40 tx=0x7f1500011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5c47b99ee0b96a37 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:31:57.716 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.713+0000 7f14f3e00640 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-03-21T10:31:57.716 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.713+0000 7f14f3e00640 1 -- 172.21.15.47:0/1397134557 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15000052c0 con 0x55922045b630 2024-03-21T10:31:57.716 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.713+0000 7f15181c8740 1 -- 172.21.15.47:0/1397134557 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f1508098310 con 0x55922045b630 2024-03-21T10:31:57.716 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.713+0000 7f14f3e00640 1 -- 172.21.15.47:0/1397134557 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(97..97 src has 1..97) v4 ==== 9474+0+0 (secure 0 0 0) 0x7f1500005460 con 0x55922045b630 2024-03-21T10:31:57.716 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.713+0000 7f14f3e00640 1 -- 172.21.15.47:0/1397134557 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1500039050 con 0x55922045b630 2024-03-21T10:31:57.716 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.713+0000 7f15181c8740 10 client.17640 fetch_fsmap learned FSMap version 32 2024-03-21T10:31:57.716 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.713+0000 7f15181c8740 1 -- 172.21.15.47:0/1397134557 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x5592204d4f80 con 0x55922045b630 2024-03-21T10:31:57.717 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.714+0000 7f14f3e00640 1 -- 172.21.15.47:0/1397134557 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f1508098310 con 0x55922045b630 2024-03-21T10:31:57.717 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.714+0000 7f14f3e00640 1 -- 172.21.15.47:0/1397134557 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 32) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f150002c5b0 con 0x55922045b630 2024-03-21T10:31:57.717 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.714+0000 7f15181c8740 10 client.17640 fetch_fsmap finished waiting for FSMap version 32 2024-03-21T10:31:57.717 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.714+0000 7f15181c8740 10 client.17640 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.e 2024-03-21T10:31:57.717 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.714+0000 7f15181c8740 20 client.17640 populate_metadata read hostname 'smithi047' 2024-03-21T10:31:57.717 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.714+0000 7f15181c8740 1 --2- 172.21.15.47:0/1397134557 >> [v2:172.21.15.100:6834/2597849957,v1:172.21.15.100:6835/2597849957] conn(0x5592204d5400 0x5592204d77f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:31:57.717 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.714+0000 7f15181c8740 4 client.17640 mds_command: new command op to 34454 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-21T10:31:57.717 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.714+0000 7f15181c8740 1 -- 172.21.15.47:0/1397134557 --> [v2:172.21.15.100:6834/2597849957,v1:172.21.15.100:6835/2597849957] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x5592203c6b50 con 0x5592204d5400 2024-03-21T10:31:57.717 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.714+0000 7f1505e00640 1 --2- 172.21.15.47:0/1397134557 >> [v2:172.21.15.100:6834/2597849957,v1:172.21.15.100:6835/2597849957] conn(0x5592204d5400 0x5592204d77f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:31:57.717 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.715+0000 7f1505e00640 1 --2- 172.21.15.47:0/1397134557 >> [v2:172.21.15.100:6834/2597849957,v1:172.21.15.100:6835/2597849957] conn(0x5592204d5400 0x5592204d77f0 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=87af193c52895610 server_cookie=134a21fa82d15759 in_seq=0 out_seq=0 2024-03-21T10:31:57.717 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.715+0000 7f14f3e00640 10 client.17640 ms_handle_connect on v2:172.21.15.100:6834/2597849957 2024-03-21T10:31:57.717 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.715+0000 7f14f3e00640 1 -- 172.21.15.47:0/1397134557 <== mds.2 v2:172.21.15.100:6834/2597849957 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x5592203c6b50 con 0x5592204d5400 2024-03-21T10:31:57.717 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.715+0000 7f14f3e00640 10 client.17640 handle_command_reply: tid=0 2024-03-21T10:31:57.717 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.715+0000 7f15181c8740 1 client.17640 shutdown 2024-03-21T10:31:57.717 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.716+0000 7f15181c8740 1 -- 172.21.15.47:0/1397134557 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55922045b630 msgr2=0x55922046d000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:31:57.718 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.716+0000 7f15181c8740 1 --2- 172.21.15.47:0/1397134557 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55922045b630 0x55922046d000 secure :-1 s=READY pgs=991 cs=0 l=1 rev1=1 crypto rx=0x7f1500009c40 tx=0x7f1500011810 comp rx=0 tx=0).stop 2024-03-21T10:31:57.718 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.716+0000 7f15181c8740 1 -- 172.21.15.47:0/1397134557 shutdown_connections 2024-03-21T10:31:57.718 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.716+0000 7f15181c8740 1 --2- 172.21.15.47:0/1397134557 >> [v2:172.21.15.100:6834/2597849957,v1:172.21.15.100:6835/2597849957] conn(0x5592204d5400 0x5592204d77f0 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:31:57.718 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.716+0000 7f15181c8740 1 --2- 172.21.15.47:0/1397134557 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5592204691f0 0x5592204d38d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:31:57.718 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.716+0000 7f15181c8740 1 --2- 172.21.15.47:0/1397134557 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55922045bfe0 0x55922046d540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:31:57.718 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.716+0000 7f15181c8740 1 --2- 172.21.15.47:0/1397134557 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55922045b630 0x55922046d000 unknown :-1 s=CLOSED pgs=991 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:31:57.718 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.716+0000 7f14f3e00640 0 client.17640 ms_handle_reset on v2:172.21.15.100:6834/2597849957 2024-03-21T10:31:57.718 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.716+0000 7f15181c8740 1 -- 172.21.15.47:0/1397134557 >> 172.21.15.47:0/1397134557 conn(0x559220455f80 msgr2=0x5592204548d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:31:57.718 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.716+0000 7f15181c8740 1 -- 172.21.15.47:0/1397134557 shutdown_connections 2024-03-21T10:31:57.718 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.716+0000 7f15181c8740 1 -- 172.21.15.47:0/1397134557 wait complete. 2024-03-21T10:31:57.718 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.716+0000 7f15181c8740 20 client.17640 trim_cache size 0 max 16384 2024-03-21T10:31:57.719 INFO:teuthology.orchestra.run.smithi047.stdout:[] 2024-03-21T10:31:57.719 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.717+0000 7f1514a00640 1 -- 172.21.15.47:0/2338147875 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f14f40a3110 msgr2=0x7f14f40a55d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:31:57.719 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.717+0000 7f1514a00640 1 --2- 172.21.15.47:0/2338147875 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f14f40a3110 0x7f14f40a55d0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f14f8005ab0 tx=0x7f14f8005e20 comp rx=0 tx=0).stop 2024-03-21T10:31:57.719 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.717+0000 7f1514a00640 1 -- 172.21.15.47:0/2338147875 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f151010baf0 msgr2=0x7f151019c690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:31:57.720 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.717+0000 7f1514a00640 1 --2- 172.21.15.47:0/2338147875 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f151010baf0 0x7f151019c690 secure :-1 s=READY pgs=987 cs=0 l=1 rev1=1 crypto rx=0x7f150800eaa0 tx=0x7f150800ef60 comp rx=0 tx=0).stop 2024-03-21T10:31:57.720 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.717+0000 7f1514a00640 1 -- 172.21.15.47:0/2338147875 shutdown_connections 2024-03-21T10:31:57.720 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.717+0000 7f1514a00640 1 --2- 172.21.15.47:0/2338147875 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f151010baf0 0x7f151019c690 unknown :-1 s=CLOSED pgs=987 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:31:57.720 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.717+0000 7f1514a00640 1 --2- 172.21.15.47:0/2338147875 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f14f40a3110 0x7f14f40a55d0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:31:57.720 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.717+0000 7f1514a00640 1 --2- 172.21.15.47:0/2338147875 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1510102ff0 0x7f1510198360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:31:57.720 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.717+0000 7f1514a00640 1 --2- 172.21.15.47:0/2338147875 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f15101026d0 0x7f1510197e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:31:57.720 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.717+0000 7f1514a00640 1 -- 172.21.15.47:0/2338147875 >> 172.21.15.47:0/2338147875 conn(0x7f15100fbef0 msgr2=0x7f15100fcd40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:31:57.720 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.717+0000 7f1514a00640 1 -- 172.21.15.47:0/2338147875 shutdown_connections 2024-03-21T10:31:57.720 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.717+0000 7f1514a00640 1 -- 172.21.15.47:0/2338147875 wait complete. 2024-03-21T10:31:57.733 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-21T10:31:57.733 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-03-21T10:31:57.948 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.949+0000 7f2171000640 1 -- 172.21.15.47:0/17492828 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f216c073ad0 msgr2=0x7f216c073eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:31:57.948 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.949+0000 7f2171000640 1 --2- 172.21.15.47:0/17492828 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f216c073ad0 0x7f216c073eb0 secure :-1 s=READY pgs=992 cs=0 l=1 rev1=1 crypto rx=0x7f215c009900 tx=0x7f215c02bcd0 comp rx=0 tx=0).stop 2024-03-21T10:31:57.948 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.949+0000 7f2171000640 1 -- 172.21.15.47:0/17492828 shutdown_connections 2024-03-21T10:31:57.948 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.949+0000 7f2171000640 1 --2- 172.21.15.47:0/17492828 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f216c10de50 0x7f216c1102a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:31:57.948 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.949+0000 7f2171000640 1 --2- 172.21.15.47:0/17492828 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f216c0743f0 0x7f216c10d910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:31:57.948 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.949+0000 7f2171000640 1 --2- 172.21.15.47:0/17492828 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f216c073ad0 0x7f216c073eb0 unknown :-1 s=CLOSED pgs=992 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:31:57.948 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.949+0000 7f2171000640 1 -- 172.21.15.47:0/17492828 >> 172.21.15.47:0/17492828 conn(0x7f216c0fd100 msgr2=0x7f216c0ff520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:31:57.949 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.949+0000 7f2171000640 1 -- 172.21.15.47:0/17492828 shutdown_connections 2024-03-21T10:31:57.949 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.949+0000 7f2171000640 1 -- 172.21.15.47:0/17492828 wait complete. 2024-03-21T10:31:57.951 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.950+0000 7f2171000640 1 Processor -- start 2024-03-21T10:31:57.951 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.950+0000 7f2171000640 1 -- start start 2024-03-21T10:31:57.951 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.950+0000 7f2171000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f216c073ad0 0x7f216c19a090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:31:57.951 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.951+0000 7f2171000640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f216c0743f0 0x7f216c19a5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:31:57.951 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.951+0000 7f2171000640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f216c10de50 0x7f216c19e950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:31:57.952 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.951+0000 7f2171000640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f216c113aa0 con 0x7f216c10de50 2024-03-21T10:31:57.952 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.951+0000 7f2171000640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f216c113920 con 0x7f216c073ad0 2024-03-21T10:31:57.952 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.951+0000 7f2171000640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f216c113c20 con 0x7f216c0743f0 2024-03-21T10:31:57.952 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.951+0000 7f216a800640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f216c10de50 0x7f216c19e950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:31:57.952 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.951+0000 7f2169e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f216c073ad0 0x7f216c19a090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:31:57.952 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.951+0000 7f2169e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f216c073ad0 0x7f216c19a090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:39138/0 (socket says 172.21.15.47:39138) 2024-03-21T10:31:57.952 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.951+0000 7f2169e00640 1 -- 172.21.15.47:0/245184901 learned_addr learned my addr 172.21.15.47:0/245184901 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:31:57.952 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.951+0000 7f216a800640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f216c10de50 0x7f216c19e950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:57828/0 (socket says 172.21.15.47:57828) 2024-03-21T10:31:57.952 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.951+0000 7f2163e00640 1 --2- 172.21.15.47:0/245184901 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f216c0743f0 0x7f216c19a5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:31:57.952 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.951+0000 7f2169e00640 1 -- 172.21.15.47:0/245184901 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f216c0743f0 msgr2=0x7f216c19a5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:31:57.952 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.951+0000 7f2169e00640 1 --2- 172.21.15.47:0/245184901 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f216c0743f0 0x7f216c19a5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:31:57.952 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.951+0000 7f2169e00640 1 -- 172.21.15.47:0/245184901 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f216c10de50 msgr2=0x7f216c19e950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:31:57.952 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.951+0000 7f2169e00640 1 --2- 172.21.15.47:0/245184901 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f216c10de50 0x7f216c19e950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:31:57.952 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.951+0000 7f2169e00640 1 -- 172.21.15.47:0/245184901 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f216c19f080 con 0x7f216c073ad0 2024-03-21T10:31:57.953 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.951+0000 7f216a800640 1 --2- 172.21.15.47:0/245184901 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f216c10de50 0x7f216c19e950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T10:31:57.953 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.952+0000 7f2169e00640 1 --2- 172.21.15.47:0/245184901 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f216c073ad0 0x7f216c19a090 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f215c005940 tx=0x7f215c0045e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:31:57.953 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.952+0000 7f2162a00640 1 -- 172.21.15.47:0/245184901 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f215c043020 con 0x7f216c073ad0 2024-03-21T10:31:57.953 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.952+0000 7f2171000640 1 -- 172.21.15.47:0/245184901 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f216c19f310 con 0x7f216c073ad0 2024-03-21T10:31:57.953 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.952+0000 7f2162a00640 1 -- 172.21.15.47:0/245184901 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f215c02dba0 con 0x7f216c073ad0 2024-03-21T10:31:57.953 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.952+0000 7f2162a00640 1 -- 172.21.15.47:0/245184901 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f215c003a10 con 0x7f216c073ad0 2024-03-21T10:31:57.953 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.952+0000 7f2171000640 1 -- 172.21.15.47:0/245184901 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f216c1a6c00 con 0x7f216c073ad0 2024-03-21T10:31:57.954 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.955+0000 7f2173d84740 1 Processor -- start 2024-03-21T10:31:57.954 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.956+0000 7f2173d84740 1 -- start start 2024-03-21T10:31:57.955 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.956+0000 7f2173d84740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55762ec216d0 0x55762ec21ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:31:57.956 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.956+0000 7f2173d84740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55762ec22080 0x55762ec2a630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:31:57.956 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.956+0000 7f2173d84740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55762ec2ac00 0x55762ec2d050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:31:57.956 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.956+0000 7f2173d84740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x55762eb852c0 con 0x55762ec2ac00 2024-03-21T10:31:57.956 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.956+0000 7f2173d84740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55762eb2dc90 con 0x55762ec216d0 2024-03-21T10:31:57.956 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.956+0000 7f2173d84740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55762eb81360 con 0x55762ec22080 2024-03-21T10:31:57.956 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.956+0000 7f2163e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55762ec2ac00 0x55762ec2d050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:31:57.956 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.956+0000 7f216a800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55762ec216d0 0x55762ec21ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:31:57.956 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.956+0000 7f2163e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55762ec2ac00 0x55762ec2d050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:57834/0 (socket says 172.21.15.47:57834) 2024-03-21T10:31:57.956 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.956+0000 7f2163e00640 1 -- 172.21.15.47:0/1388321026 learned_addr learned my addr 172.21.15.47:0/1388321026 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:31:57.956 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.957+0000 7f2162a00640 1 -- 172.21.15.47:0/245184901 <== mon.2 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 27) v1 ==== 137985+0+0 (secure 0 0 0) 0x7f215c046020 con 0x7f216c073ad0 2024-03-21T10:31:57.956 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.957+0000 7f2163e00640 1 -- 172.21.15.47:0/1388321026 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55762ec22080 msgr2=0x55762ec2a630 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-21T10:31:57.956 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.957+0000 7f2163e00640 1 --2- 172.21.15.47:0/1388321026 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55762ec22080 0x55762ec2a630 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:31:57.956 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.957+0000 7f2163e00640 1 -- 172.21.15.47:0/1388321026 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55762ec216d0 msgr2=0x55762ec21ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:31:57.957 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.957+0000 7f2163e00640 1 --2- 172.21.15.47:0/1388321026 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55762ec216d0 0x55762ec21ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:31:57.957 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.957+0000 7f2163e00640 1 -- 172.21.15.47:0/1388321026 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55762eb52080 con 0x55762ec2ac00 2024-03-21T10:31:57.957 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.957+0000 7f216a800640 1 --2- 172.21.15.47:0/1388321026 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55762ec216d0 0x55762ec21ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T10:31:57.957 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.957+0000 7f2163e00640 1 --2- 172.21.15.47:0/1388321026 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55762ec2ac00 0x55762ec2d050 secure :-1 s=READY pgs=993 cs=0 l=1 rev1=1 crypto rx=0x7f2154009c90 tx=0x7f2154011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cf106d93c2d58c05 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:31:57.962 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.958+0000 7f215b400640 1 -- 172.21.15.47:0/1388321026 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f215401e070 con 0x55762ec2ac00 2024-03-21T10:31:57.962 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.958+0000 7f215b400640 1 -- 172.21.15.47:0/1388321026 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f2154019ae0 con 0x55762ec2ac00 2024-03-21T10:31:57.962 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.958+0000 7f215b400640 1 -- 172.21.15.47:0/1388321026 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f215400ea00 con 0x55762ec2ac00 2024-03-21T10:31:57.962 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.958+0000 7f2173d84740 1 -- 172.21.15.47:0/1388321026 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55762ec2ac00 msgr2=0x55762ec2d050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:31:57.962 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.958+0000 7f2173d84740 1 --2- 172.21.15.47:0/1388321026 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55762ec2ac00 0x55762ec2d050 secure :-1 s=READY pgs=993 cs=0 l=1 rev1=1 crypto rx=0x7f2154009c90 tx=0x7f2154011810 comp rx=0 tx=0).stop 2024-03-21T10:31:57.962 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.958+0000 7f2173d84740 1 -- 172.21.15.47:0/1388321026 shutdown_connections 2024-03-21T10:31:57.962 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.958+0000 7f2173d84740 1 --2- 172.21.15.47:0/1388321026 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55762ec2ac00 0x55762ec2d050 unknown :-1 s=CLOSED pgs=993 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:31:57.962 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.958+0000 7f2173d84740 1 --2- 172.21.15.47:0/1388321026 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55762ec22080 0x55762ec2a630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:31:57.962 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.958+0000 7f2173d84740 1 --2- 172.21.15.47:0/1388321026 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55762ec216d0 0x55762ec21ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:31:57.962 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.958+0000 7f2173d84740 1 -- 172.21.15.47:0/1388321026 >> 172.21.15.47:0/1388321026 conn(0x55762ec1bda0 msgr2=0x55762ec1c180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:31:57.962 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.958+0000 7f2162a00640 1 --2- 172.21.15.47:0/245184901 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f21500a3090 0x7f21500a5550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:31:57.963 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.959+0000 7f2173d84740 1 -- 172.21.15.47:0/1388321026 shutdown_connections 2024-03-21T10:31:57.963 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.959+0000 7f2173d84740 1 -- 172.21.15.47:0/1388321026 wait complete. 2024-03-21T10:31:57.963 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.959+0000 7f2162a00640 1 -- 172.21.15.47:0/245184901 <== mon.2 v2:172.21.15.100:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 9474+0+0 (secure 0 0 0) 0x7f215c0f1ad0 con 0x7f216c073ad0 2024-03-21T10:31:57.963 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.959+0000 7f2163e00640 1 --2- 172.21.15.47:0/245184901 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f21500a3090 0x7f21500a5550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:31:57.963 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.959+0000 7f2173d84740 1 -- start start 2024-03-21T10:31:57.963 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.959+0000 7f2173d84740 1 Processor -- start 2024-03-21T10:31:57.963 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.960+0000 7f2163e00640 1 --2- 172.21.15.47:0/245184901 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f21500a3090 0x7f21500a5550 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f2154011d10 tx=0x7f21540093b0 comp rx=0 tx=0).ready entity=mgr.15687 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:31:57.963 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.960+0000 7f2173d84740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55762ec216d0 0x55762ec2a550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:31:57.963 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.960+0000 7f2173d84740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55762ec22080 0x55762ec278e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:31:57.963 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.960+0000 7f2173d84740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55762ec2ac00 0x55762ec27ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:31:57.963 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.960+0000 7f2173d84740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x55762ec2db70 con 0x55762ec22080 2024-03-21T10:31:57.963 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.960+0000 7f2173d84740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55762eb410d0 con 0x55762ec2ac00 2024-03-21T10:31:57.963 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.960+0000 7f2173d84740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55762ec2dcf0 con 0x55762ec216d0 2024-03-21T10:31:57.963 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.960+0000 7f2173d84740 10 client.0 _ll_register_callbacks cb 0x55762eb45e80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T10:31:57.964 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.960+0000 7f2163e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55762ec22080 0x55762ec278e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:31:57.964 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.960+0000 7f216a800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55762ec2ac00 0x55762ec27ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:31:57.964 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.960+0000 7f2163e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55762ec22080 0x55762ec278e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:57846/0 (socket says 172.21.15.47:57846) 2024-03-21T10:31:57.964 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.960+0000 7f2163e00640 1 -- 172.21.15.47:0/3923000097 learned_addr learned my addr 172.21.15.47:0/3923000097 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:31:57.964 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.961+0000 7f2169e00640 1 --2- 172.21.15.47:0/3923000097 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55762ec216d0 0x55762ec2a550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:31:57.964 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.961+0000 7f2163e00640 1 -- 172.21.15.47:0/3923000097 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55762ec216d0 msgr2=0x55762ec2a550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:31:57.964 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.961+0000 7f2163e00640 1 --2- 172.21.15.47:0/3923000097 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55762ec216d0 0x55762ec2a550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:31:57.964 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.961+0000 7f2163e00640 1 -- 172.21.15.47:0/3923000097 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55762ec2ac00 msgr2=0x55762ec27ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:31:57.964 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.961+0000 7f2163e00640 1 --2- 172.21.15.47:0/3923000097 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55762ec2ac00 0x55762ec27ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:31:57.964 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.961+0000 7f2163e00640 1 -- 172.21.15.47:0/3923000097 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55762e8993f0 con 0x55762ec22080 2024-03-21T10:31:57.964 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.961+0000 7f2163e00640 1 --2- 172.21.15.47:0/3923000097 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55762ec22080 0x55762ec278e0 secure :-1 s=READY pgs=994 cs=0 l=1 rev1=1 crypto rx=0x7f215401b000 tx=0x7f2154011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6f132acaea71f9dc server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:31:57.964 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.961+0000 7f214fe00640 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-03-21T10:31:57.964 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.962+0000 7f214fe00640 1 -- 172.21.15.47:0/3923000097 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2154035060 con 0x55762ec22080 2024-03-21T10:31:57.964 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.962+0000 7f214fe00640 1 -- 172.21.15.47:0/3923000097 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(97..97 src has 1..97) v4 ==== 9474+0+0 (secure 0 0 0) 0x7f215400e740 con 0x55762ec22080 2024-03-21T10:31:57.965 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.962+0000 7f2173d84740 1 -- 172.21.15.47:0/3923000097 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f2154019ae0 con 0x55762ec22080 2024-03-21T10:31:57.965 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.962+0000 7f214fe00640 1 -- 172.21.15.47:0/3923000097 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f215403c050 con 0x55762ec22080 2024-03-21T10:31:57.965 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.962+0000 7f2173d84740 10 client.17652 fetch_fsmap learned FSMap version 32 2024-03-21T10:31:57.965 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.962+0000 7f2173d84740 1 -- 172.21.15.47:0/3923000097 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x55762e9637d0 con 0x55762ec22080 2024-03-21T10:31:57.965 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.962+0000 7f214fe00640 1 -- 172.21.15.47:0/3923000097 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f2154019ae0 con 0x55762ec22080 2024-03-21T10:31:57.965 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.963+0000 7f214fe00640 1 -- 172.21.15.47:0/3923000097 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 32) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f2154004b50 con 0x55762ec22080 2024-03-21T10:31:57.965 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.963+0000 7f2173d84740 10 client.17652 fetch_fsmap finished waiting for FSMap version 32 2024-03-21T10:31:57.965 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.963+0000 7f2173d84740 10 client.17652 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-03-21T10:31:57.965 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.963+0000 7f2173d84740 20 client.17652 populate_metadata read hostname 'smithi047' 2024-03-21T10:31:57.965 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.963+0000 7f2173d84740 1 --2- 172.21.15.47:0/3923000097 >> [v2:172.21.15.100:6836/3497644162,v1:172.21.15.100:6837/3497644162] conn(0x55762ec3aa60 0x55762ec3ce50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:31:57.965 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.963+0000 7f2173d84740 4 client.17652 mds_command: new command op to 34493 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T10:31:57.965 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.963+0000 7f2173d84740 1 -- 172.21.15.47:0/3923000097 --> [v2:172.21.15.100:6836/3497644162,v1:172.21.15.100:6837/3497644162] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55762eb92800 con 0x55762ec3aa60 2024-03-21T10:31:57.965 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.963+0000 7f2169e00640 1 --2- 172.21.15.47:0/3923000097 >> [v2:172.21.15.100:6836/3497644162,v1:172.21.15.100:6837/3497644162] conn(0x55762ec3aa60 0x55762ec3ce50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:31:57.967 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.964+0000 7f2169e00640 1 --2- 172.21.15.47:0/3923000097 >> [v2:172.21.15.100:6836/3497644162,v1:172.21.15.100:6837/3497644162] conn(0x55762ec3aa60 0x55762ec3ce50 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=e38699b9a59617a4 server_cookie=d502b5b675c8553d in_seq=0 out_seq=0 2024-03-21T10:31:57.967 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.964+0000 7f214fe00640 10 client.17652 ms_handle_connect on v2:172.21.15.100:6836/3497644162 2024-03-21T10:31:57.967 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.965+0000 7f214fe00640 1 -- 172.21.15.47:0/3923000097 <== mds.1 v2:172.21.15.100:6836/3497644162 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55762eb92800 con 0x55762ec3aa60 2024-03-21T10:31:57.967 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.965+0000 7f214fe00640 10 client.17652 handle_command_reply: tid=0 2024-03-21T10:31:57.967 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.965+0000 7f2173d84740 1 client.17652 shutdown 2024-03-21T10:31:57.967 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.965+0000 7f2173d84740 1 -- 172.21.15.47:0/3923000097 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55762ec22080 msgr2=0x55762ec278e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:31:57.967 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.965+0000 7f2173d84740 1 --2- 172.21.15.47:0/3923000097 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55762ec22080 0x55762ec278e0 secure :-1 s=READY pgs=994 cs=0 l=1 rev1=1 crypto rx=0x7f215401b000 tx=0x7f2154011810 comp rx=0 tx=0).stop 2024-03-21T10:31:57.967 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.965+0000 7f2173d84740 1 -- 172.21.15.47:0/3923000097 shutdown_connections 2024-03-21T10:31:57.967 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.965+0000 7f2173d84740 1 --2- 172.21.15.47:0/3923000097 >> [v2:172.21.15.100:6836/3497644162,v1:172.21.15.100:6837/3497644162] conn(0x55762ec3aa60 0x55762ec3ce50 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:31:57.967 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.965+0000 7f2173d84740 1 --2- 172.21.15.47:0/3923000097 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55762ec2ac00 0x55762ec27ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:31:57.967 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.965+0000 7f2173d84740 1 --2- 172.21.15.47:0/3923000097 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55762ec22080 0x55762ec278e0 unknown :-1 s=CLOSED pgs=994 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:31:57.967 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.965+0000 7f2173d84740 1 --2- 172.21.15.47:0/3923000097 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55762ec216d0 0x55762ec2a550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:31:57.967 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.965+0000 7f2173d84740 1 -- 172.21.15.47:0/3923000097 >> 172.21.15.47:0/3923000097 conn(0x55762ec1bda0 msgr2=0x55762ec1daf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:31:57.968 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.965+0000 7f214fe00640 0 client.17652 ms_handle_reset on v2:172.21.15.100:6836/3497644162 2024-03-21T10:31:57.968 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.966+0000 7f2173d84740 1 -- 172.21.15.47:0/3923000097 shutdown_connections 2024-03-21T10:31:57.968 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.966+0000 7f2173d84740 1 -- 172.21.15.47:0/3923000097 wait complete. 2024-03-21T10:31:57.968 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.966+0000 7f2173d84740 20 client.17652 trim_cache size 0 max 16384 2024-03-21T10:31:57.977 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:31:57 smithi100 ceph-mon[35193]: pgmap v202: 129 pgs: 129 active+clean; 277 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 12 op/s 2024-03-21T10:31:57.977 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:31:57 smithi100 ceph-mon[35193]: from='client.? 172.21.15.47:0/230257554' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-21T10:31:57.978 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.979+0000 7f2173d84740 1 Processor -- start 2024-03-21T10:31:57.978 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.980+0000 7f2173d84740 1 -- start start 2024-03-21T10:31:57.979 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.980+0000 7f2173d84740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55762ec216d0 0x55762ec1aeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:31:57.979 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.980+0000 7f2173d84740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55762ec22080 0x55762ec2a010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:31:57.979 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.980+0000 7f2173d84740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55762ec2ac00 0x55762ec2a550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:31:57.979 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.980+0000 7f2173d84740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f216c1b0b80 con 0x55762ec2ac00 2024-03-21T10:31:57.979 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.980+0000 7f2173d84740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f216c1b2500 con 0x55762ec22080 2024-03-21T10:31:57.979 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.980+0000 7f2173d84740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f216c1b11e0 con 0x55762ec216d0 2024-03-21T10:31:57.979 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.980+0000 7f2163e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55762ec22080 0x55762ec2a010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:31:57.979 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.980+0000 7f216a800640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55762ec2ac00 0x55762ec2a550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:31:57.979 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.980+0000 7f2163e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55762ec22080 0x55762ec2a010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:39158/0 (socket says 172.21.15.47:39158) 2024-03-21T10:31:57.980 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.980+0000 7f2163e00640 1 -- 172.21.15.47:0/2265280265 learned_addr learned my addr 172.21.15.47:0/2265280265 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:31:57.980 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.980+0000 7f216a800640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55762ec2ac00 0x55762ec2a550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:57860/0 (socket says 172.21.15.47:57860) 2024-03-21T10:31:57.980 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.980+0000 7f2169e00640 1 --2- 172.21.15.47:0/2265280265 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55762ec216d0 0x55762ec1aeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:31:57.981 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.981+0000 7f216a800640 1 -- 172.21.15.47:0/2265280265 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55762ec216d0 msgr2=0x55762ec1aeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:31:57.981 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.981+0000 7f216a800640 1 --2- 172.21.15.47:0/2265280265 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55762ec216d0 0x55762ec1aeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:31:57.981 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.981+0000 7f216a800640 1 -- 172.21.15.47:0/2265280265 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55762ec22080 msgr2=0x55762ec2a010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:31:57.981 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.981+0000 7f216a800640 1 --2- 172.21.15.47:0/2265280265 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55762ec22080 0x55762ec2a010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:31:57.981 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.981+0000 7f216a800640 1 -- 172.21.15.47:0/2265280265 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55762ec28880 con 0x55762ec2ac00 2024-03-21T10:31:57.982 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.981+0000 7f216a800640 1 --2- 172.21.15.47:0/2265280265 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55762ec2ac00 0x55762ec2a550 secure :-1 s=READY pgs=995 cs=0 l=1 rev1=1 crypto rx=0x7f2164011850 tx=0x7f2164011d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=677de24dd532c54d server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:31:57.982 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.981+0000 7f215aa00640 1 -- 172.21.15.47:0/2265280265 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2164021070 con 0x55762ec2ac00 2024-03-21T10:31:57.982 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.981+0000 7f215aa00640 1 -- 172.21.15.47:0/2265280265 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f216401c410 con 0x55762ec2ac00 2024-03-21T10:31:57.982 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.982+0000 7f215aa00640 1 -- 172.21.15.47:0/2265280265 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2164010630 con 0x55762ec2ac00 2024-03-21T10:31:57.982 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.982+0000 7f2173d84740 1 -- 172.21.15.47:0/2265280265 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55762ec2ac00 msgr2=0x55762ec2a550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:31:57.982 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.982+0000 7f2173d84740 1 --2- 172.21.15.47:0/2265280265 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55762ec2ac00 0x55762ec2a550 secure :-1 s=READY pgs=995 cs=0 l=1 rev1=1 crypto rx=0x7f2164011850 tx=0x7f2164011d10 comp rx=0 tx=0).stop 2024-03-21T10:31:57.982 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.982+0000 7f2173d84740 1 -- 172.21.15.47:0/2265280265 shutdown_connections 2024-03-21T10:31:57.982 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.982+0000 7f2173d84740 1 --2- 172.21.15.47:0/2265280265 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55762ec2ac00 0x55762ec2a550 unknown :-1 s=CLOSED pgs=995 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:31:57.982 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.982+0000 7f2173d84740 1 --2- 172.21.15.47:0/2265280265 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55762ec22080 0x55762ec2a010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:31:57.982 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.982+0000 7f2173d84740 1 --2- 172.21.15.47:0/2265280265 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55762ec216d0 0x55762ec1aeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:31:57.982 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.982+0000 7f2173d84740 1 -- 172.21.15.47:0/2265280265 >> 172.21.15.47:0/2265280265 conn(0x55762ec1bda0 msgr2=0x55762ec22460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:31:57.982 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.982+0000 7f2173d84740 1 -- 172.21.15.47:0/2265280265 shutdown_connections 2024-03-21T10:31:57.982 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.982+0000 7f2173d84740 1 -- 172.21.15.47:0/2265280265 wait complete. 2024-03-21T10:31:57.982 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.983+0000 7f2173d84740 1 -- start start 2024-03-21T10:31:57.983 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.983+0000 7f2173d84740 1 Processor -- start 2024-03-21T10:31:57.985 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.983+0000 7f2173d84740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55762ec216d0 0x55762ec32ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:31:57.985 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.983+0000 7f2173d84740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55762ec22080 0x55762ec30f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:31:57.985 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.983+0000 7f2173d84740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55762ec2ac00 0x55762ec314d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:31:57.985 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.983+0000 7f2173d84740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f216c1b1ea0 con 0x55762ec216d0 2024-03-21T10:31:57.985 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.983+0000 7f2173d84740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f216c1b1840 con 0x55762ec22080 2024-03-21T10:31:57.985 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.983+0000 7f2173d84740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f216c1aed40 con 0x55762ec2ac00 2024-03-21T10:31:57.985 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.983+0000 7f2173d84740 10 client.0 _ll_register_callbacks cb 0x7f215401c070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T10:31:57.986 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.984+0000 7f2169e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55762ec216d0 0x55762ec32ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:31:57.986 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.984+0000 7f2169e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55762ec216d0 0x55762ec32ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:57868/0 (socket says 172.21.15.47:57868) 2024-03-21T10:31:57.986 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.984+0000 7f2169e00640 1 -- 172.21.15.47:0/4168957313 learned_addr learned my addr 172.21.15.47:0/4168957313 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:31:57.986 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.984+0000 7f2163e00640 1 --2- 172.21.15.47:0/4168957313 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55762ec22080 0x55762ec30f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:31:57.986 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.984+0000 7f216a800640 1 --2- 172.21.15.47:0/4168957313 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55762ec2ac00 0x55762ec314d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:31:57.986 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.984+0000 7f2169e00640 1 -- 172.21.15.47:0/4168957313 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55762ec2ac00 msgr2=0x55762ec314d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:31:57.986 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.984+0000 7f2169e00640 1 --2- 172.21.15.47:0/4168957313 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55762ec2ac00 0x55762ec314d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:31:57.986 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.984+0000 7f2169e00640 1 -- 172.21.15.47:0/4168957313 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55762ec22080 msgr2=0x55762ec30f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:31:57.986 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.984+0000 7f2169e00640 1 --2- 172.21.15.47:0/4168957313 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55762ec22080 0x55762ec30f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:31:57.986 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.984+0000 7f2169e00640 1 -- 172.21.15.47:0/4168957313 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55762ec31b90 con 0x55762ec216d0 2024-03-21T10:31:57.986 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.984+0000 7f2169e00640 1 --2- 172.21.15.47:0/4168957313 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55762ec216d0 0x55762ec32ba0 secure :-1 s=READY pgs=996 cs=0 l=1 rev1=1 crypto rx=0x7f215c000c00 tx=0x7f215c0b9f20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=906064098f1026e2 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:31:57.986 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.984+0000 7f214fe00640 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-03-21T10:31:57.986 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.985+0000 7f214fe00640 1 -- 172.21.15.47:0/4168957313 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f215c0c1d00 con 0x55762ec216d0 2024-03-21T10:31:57.986 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.985+0000 7f214fe00640 1 -- 172.21.15.47:0/4168957313 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(97..97 src has 1..97) v4 ==== 9474+0+0 (secure 0 0 0) 0x7f215c0ba220 con 0x55762ec216d0 2024-03-21T10:31:57.987 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.985+0000 7f2173d84740 1 -- 172.21.15.47:0/4168957313 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f216401c410 con 0x55762ec216d0 2024-03-21T10:31:57.987 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.985+0000 7f214fe00640 1 -- 172.21.15.47:0/4168957313 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f215c0c65c0 con 0x55762ec216d0 2024-03-21T10:31:57.987 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.985+0000 7f2173d84740 10 client.17658 fetch_fsmap learned FSMap version 32 2024-03-21T10:31:57.987 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.985+0000 7f2173d84740 1 -- 172.21.15.47:0/4168957313 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x55762ec97080 con 0x55762ec216d0 2024-03-21T10:31:57.987 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.985+0000 7f214fe00640 1 -- 172.21.15.47:0/4168957313 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f216401c410 con 0x55762ec216d0 2024-03-21T10:31:57.987 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.985+0000 7f214fe00640 1 -- 172.21.15.47:0/4168957313 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 32) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f215c0c6130 con 0x55762ec216d0 2024-03-21T10:31:57.987 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.985+0000 7f2173d84740 10 client.17658 fetch_fsmap finished waiting for FSMap version 32 2024-03-21T10:31:57.987 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.985+0000 7f2173d84740 10 client.17658 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-03-21T10:31:57.987 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.985+0000 7f2173d84740 20 client.17658 populate_metadata read hostname 'smithi047' 2024-03-21T10:31:57.987 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.985+0000 7f2173d84740 1 --2- 172.21.15.47:0/4168957313 >> [v2:172.21.15.100:6836/3497644162,v1:172.21.15.100:6837/3497644162] conn(0x55762ec97700 0x55762ec99b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:31:57.987 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.985+0000 7f2173d84740 4 client.17658 mds_command: new command op to 34493 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-21T10:31:57.987 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.985+0000 7f2173d84740 1 -- 172.21.15.47:0/4168957313 --> [v2:172.21.15.100:6836/3497644162,v1:172.21.15.100:6837/3497644162] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55762ec1b5b0 con 0x55762ec97700 2024-03-21T10:31:57.987 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.986+0000 7f2163e00640 1 --2- 172.21.15.47:0/4168957313 >> [v2:172.21.15.100:6836/3497644162,v1:172.21.15.100:6837/3497644162] conn(0x55762ec97700 0x55762ec99b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:31:57.987 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.986+0000 7f2163e00640 1 --2- 172.21.15.47:0/4168957313 >> [v2:172.21.15.100:6836/3497644162,v1:172.21.15.100:6837/3497644162] conn(0x55762ec97700 0x55762ec99b20 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=428d601a950ca206 server_cookie=bb6237bc6cec5f63 in_seq=0 out_seq=0 2024-03-21T10:31:57.987 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.986+0000 7f214fe00640 10 client.17658 ms_handle_connect on v2:172.21.15.100:6836/3497644162 2024-03-21T10:31:57.988 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:31:57 smithi157 ceph-mon[35581]: pgmap v202: 129 pgs: 129 active+clean; 277 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 12 op/s 2024-03-21T10:31:57.988 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:31:57 smithi157 ceph-mon[35581]: from='client.? 172.21.15.47:0/230257554' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-21T10:31:57.989 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.987+0000 7f214fe00640 1 -- 172.21.15.47:0/4168957313 <== mds.1 v2:172.21.15.100:6836/3497644162 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55762ec1b5b0 con 0x55762ec97700 2024-03-21T10:31:57.989 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.987+0000 7f214fe00640 10 client.17658 handle_command_reply: tid=0 2024-03-21T10:31:57.989 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.987+0000 7f2173d84740 1 client.17658 shutdown 2024-03-21T10:31:57.989 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.987+0000 7f2173d84740 1 -- 172.21.15.47:0/4168957313 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55762ec216d0 msgr2=0x55762ec32ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:31:57.990 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.987+0000 7f2173d84740 1 --2- 172.21.15.47:0/4168957313 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55762ec216d0 0x55762ec32ba0 secure :-1 s=READY pgs=996 cs=0 l=1 rev1=1 crypto rx=0x7f215c000c00 tx=0x7f215c0b9f20 comp rx=0 tx=0).stop 2024-03-21T10:31:57.990 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.987+0000 7f2173d84740 1 -- 172.21.15.47:0/4168957313 shutdown_connections 2024-03-21T10:31:57.990 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.987+0000 7f2173d84740 1 --2- 172.21.15.47:0/4168957313 >> [v2:172.21.15.100:6836/3497644162,v1:172.21.15.100:6837/3497644162] conn(0x55762ec97700 0x55762ec99b20 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:31:57.990 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.987+0000 7f2173d84740 1 --2- 172.21.15.47:0/4168957313 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55762ec2ac00 0x55762ec314d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:31:57.990 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.987+0000 7f2173d84740 1 --2- 172.21.15.47:0/4168957313 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55762ec22080 0x55762ec30f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:31:57.990 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.987+0000 7f214fe00640 0 client.17658 ms_handle_reset on v2:172.21.15.100:6836/3497644162 2024-03-21T10:31:57.990 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.987+0000 7f2173d84740 1 --2- 172.21.15.47:0/4168957313 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55762ec216d0 0x55762ec32ba0 unknown :-1 s=CLOSED pgs=996 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:31:57.990 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.987+0000 7f2173d84740 1 -- 172.21.15.47:0/4168957313 >> 172.21.15.47:0/4168957313 conn(0x55762ec1bda0 msgr2=0x55762ec1ad30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:31:57.990 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.987+0000 7f2173d84740 1 -- 172.21.15.47:0/4168957313 shutdown_connections 2024-03-21T10:31:57.990 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.987+0000 7f2173d84740 1 -- 172.21.15.47:0/4168957313 wait complete. 2024-03-21T10:31:57.990 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.987+0000 7f2173d84740 20 client.17658 trim_cache size 0 max 16384 2024-03-21T10:31:57.991 INFO:teuthology.orchestra.run.smithi047.stdout:[] 2024-03-21T10:31:57.991 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.988+0000 7f2171000640 1 -- 172.21.15.47:0/245184901 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f21500a3090 msgr2=0x7f21500a5550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:31:57.991 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.988+0000 7f2171000640 1 --2- 172.21.15.47:0/245184901 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f21500a3090 0x7f21500a5550 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f2154011d10 tx=0x7f21540093b0 comp rx=0 tx=0).stop 2024-03-21T10:31:57.992 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.988+0000 7f2171000640 1 -- 172.21.15.47:0/245184901 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f216c073ad0 msgr2=0x7f216c19a090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:31:57.992 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.988+0000 7f2171000640 1 --2- 172.21.15.47:0/245184901 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f216c073ad0 0x7f216c19a090 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f215c005940 tx=0x7f215c0045e0 comp rx=0 tx=0).stop 2024-03-21T10:31:57.992 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.988+0000 7f2171000640 1 -- 172.21.15.47:0/245184901 shutdown_connections 2024-03-21T10:31:57.992 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.988+0000 7f2171000640 1 --2- 172.21.15.47:0/245184901 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f216c10de50 0x7f216c19e950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:31:57.992 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.988+0000 7f2171000640 1 --2- 172.21.15.47:0/245184901 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f216c0743f0 0x7f216c19a5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:31:57.992 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.988+0000 7f2171000640 1 --2- 172.21.15.47:0/245184901 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f21500a3090 0x7f21500a5550 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:31:57.992 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.988+0000 7f2171000640 1 --2- 172.21.15.47:0/245184901 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f216c073ad0 0x7f216c19a090 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:31:57.992 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.988+0000 7f2171000640 1 -- 172.21.15.47:0/245184901 >> 172.21.15.47:0/245184901 conn(0x7f216c0fd100 msgr2=0x7f216c0fea80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:31:57.992 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.988+0000 7f2171000640 1 -- 172.21.15.47:0/245184901 shutdown_connections 2024-03-21T10:31:57.992 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:57.988+0000 7f2171000640 1 -- 172.21.15.47:0/245184901 wait complete. 2024-03-21T10:31:58.007 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-21T10:31:58.007 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-03-21T10:31:58.242 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:58.243+0000 7f79b3200640 1 -- 172.21.15.47:0/3782823148 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f79ac10bd00 msgr2=0x7f79ac10e130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:31:58.242 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:58.243+0000 7f79b3200640 1 --2- 172.21.15.47:0/3782823148 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f79ac10bd00 0x7f79ac10e130 secure :-1 s=READY pgs=997 cs=0 l=1 rev1=1 crypto rx=0x7f79a4009ca0 tx=0x7f79a402c180 comp rx=0 tx=0).stop 2024-03-21T10:31:58.242 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:58.243+0000 7f79b3200640 1 -- 172.21.15.47:0/3782823148 shutdown_connections 2024-03-21T10:31:58.242 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:58.243+0000 7f79b3200640 1 --2- 172.21.15.47:0/3782823148 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f79ac10bd00 0x7f79ac10e130 unknown :-1 s=CLOSED pgs=997 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:31:58.243 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:58.243+0000 7f79b3200640 1 --2- 172.21.15.47:0/3782823148 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f79ac103250 0x7f79ac10b730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:31:58.243 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:58.243+0000 7f79b3200640 1 --2- 172.21.15.47:0/3782823148 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f79ac102930 0x7f79ac102d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:31:58.243 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:58.243+0000 7f79b3200640 1 -- 172.21.15.47:0/3782823148 >> 172.21.15.47:0/3782823148 conn(0x7f79ac077700 msgr2=0x7f79ac077b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:31:58.243 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:58.243+0000 7f79b3200640 1 -- 172.21.15.47:0/3782823148 shutdown_connections 2024-03-21T10:31:58.243 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:58.243+0000 7f79b3200640 1 -- 172.21.15.47:0/3782823148 wait complete. 2024-03-21T10:31:58.244 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:58.244+0000 7f79b3200640 1 Processor -- start 2024-03-21T10:31:58.244 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:58.244+0000 7f79b3200640 1 -- start start 2024-03-21T10:31:58.244 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:58.245+0000 7f79b3200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f79ac102930 0x7f79ac19a080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:31:58.244 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:58.245+0000 7f79b3200640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f79ac103250 0x7f79ac19a5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:31:58.245 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:58.245+0000 7f79b3200640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f79ac10bd00 0x7f79ac19e940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:31:58.245 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:58.245+0000 7f79b3200640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79ac111940 con 0x7f79ac10bd00 2024-03-21T10:31:58.245 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:58.245+0000 7f79b3200640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79ac1117c0 con 0x7f79ac102930 2024-03-21T10:31:58.245 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:58.245+0000 7f79b3200640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79ac111ac0 con 0x7f79ac103250 2024-03-21T10:31:58.245 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:58.245+0000 7f79b1200640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f79ac10bd00 0x7f79ac19e940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:31:58.245 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:58.245+0000 7f79b1200640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f79ac10bd00 0x7f79ac19e940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:57890/0 (socket says 172.21.15.47:57890) 2024-03-21T10:31:58.245 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:58.245+0000 7f79b1200640 1 -- 172.21.15.47:0/3552989258 learned_addr learned my addr 172.21.15.47:0/3552989258 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:31:58.245 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:58.245+0000 7f79abe00640 1 --2- 172.21.15.47:0/3552989258 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f79ac102930 0x7f79ac19a080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:31:58.245 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:58.245+0000 7f79a3400640 1 --2- 172.21.15.47:0/3552989258 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f79ac103250 0x7f79ac19a5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:31:58.245 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:58.246+0000 7f79b1200640 1 -- 172.21.15.47:0/3552989258 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f79ac103250 msgr2=0x7f79ac19a5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:31:58.245 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:58.246+0000 7f79b1200640 1 --2- 172.21.15.47:0/3552989258 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f79ac103250 0x7f79ac19a5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:31:58.245 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:58.246+0000 7f79b1200640 1 -- 172.21.15.47:0/3552989258 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f79ac102930 msgr2=0x7f79ac19a080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:31:58.245 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:58.246+0000 7f79b1200640 1 --2- 172.21.15.47:0/3552989258 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f79ac102930 0x7f79ac19a080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:31:58.245 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:58.246+0000 7f79b1200640 1 -- 172.21.15.47:0/3552989258 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79ac19f070 con 0x7f79ac10bd00 2024-03-21T10:31:58.246 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:58.246+0000 7f79b1200640 1 --2- 172.21.15.47:0/3552989258 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f79ac10bd00 0x7f79ac19e940 secure :-1 s=READY pgs=998 cs=0 l=1 rev1=1 crypto rx=0x7f79a4037040 tx=0x7f79a4035e00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:31:58.246 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:58.247+0000 7f79a9600640 1 -- 172.21.15.47:0/3552989258 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79a4039040 con 0x7f79ac10bd00 2024-03-21T10:31:58.246 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:58.247+0000 7f79b3200640 1 -- 172.21.15.47:0/3552989258 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f79ac19f300 con 0x7f79ac10bd00 2024-03-21T10:31:58.246 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:58.247+0000 7f79a9600640 1 -- 172.21.15.47:0/3552989258 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f79a4044070 con 0x7f79ac10bd00 2024-03-21T10:31:58.246 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:58.247+0000 7f79a9600640 1 -- 172.21.15.47:0/3552989258 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79a4032850 con 0x7f79ac10bd00 2024-03-21T10:31:58.247 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:58.247+0000 7f79b3200640 1 -- 172.21.15.47:0/3552989258 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f79ac1a6bf0 con 0x7f79ac10bd00 2024-03-21T10:31:58.249 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:58.250+0000 7f79a9600640 1 -- 172.21.15.47:0/3552989258 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 27) v1 ==== 137985+0+0 (secure 0 0 0) 0x7f79a40329f0 con 0x7f79ac10bd00 2024-03-21T10:31:58.249 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:58.250+0000 7f79b5f79740 1 Processor -- start 2024-03-21T10:31:58.249 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:58.251+0000 7f79b5f79740 1 -- start start 2024-03-21T10:31:58.250 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:58.251+0000 7f79b5f79740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563e9a5eb6d0 0x563e9a5ebab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:31:58.250 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:58.251+0000 7f79b5f79740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563e9a5ec080 0x563e9a5f4630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:31:58.250 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:58.251+0000 7f79b5f79740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x563e9a5f4c00 0x563e9a5f7050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:31:58.250 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:58.251+0000 7f79b5f79740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x563e9a54f2c0 con 0x563e9a5f4c00 2024-03-21T10:31:58.250 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:58.251+0000 7f79b5f79740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x563e9a4f7c90 con 0x563e9a5ec080 2024-03-21T10:31:58.250 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:58.251+0000 7f79b5f79740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x563e9a54b360 con 0x563e9a5eb6d0 2024-03-21T10:31:58.250 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:58.252+0000 7f79a3400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x563e9a5f4c00 0x563e9a5f7050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:31:58.250 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:58.252+0000 7f79abe00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563e9a5ec080 0x563e9a5f4630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:31:58.251 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:58.252+0000 7f79a3400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x563e9a5f4c00 0x563e9a5f7050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:57902/0 (socket says 172.21.15.47:57902) 2024-03-21T10:31:58.251 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:58.252+0000 7f79a3400640 1 -- 172.21.15.47:0/1112948370 learned_addr learned my addr 172.21.15.47:0/1112948370 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:31:58.254 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:58.252+0000 7f79a3400640 1 --2- 172.21.15.47:0/1112948370 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563e9a5eb6d0 0x563e9a5ebab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:31:58.254 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:58.252+0000 7f79abe00640 1 -- 172.21.15.47:0/1112948370 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563e9a5eb6d0 msgr2=0x563e9a5ebab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:31:58.254 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:58.252+0000 7f79abe00640 1 --2- 172.21.15.47:0/1112948370 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563e9a5eb6d0 0x563e9a5ebab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:31:58.254 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:58.252+0000 7f79abe00640 1 -- 172.21.15.47:0/1112948370 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x563e9a5f4c00 msgr2=0x563e9a5f7050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:31:58.254 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:58.252+0000 7f79abe00640 1 --2- 172.21.15.47:0/1112948370 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x563e9a5f4c00 0x563e9a5f7050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:31:58.254 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:58.252+0000 7f79abe00640 1 -- 172.21.15.47:0/1112948370 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563e9a51c080 con 0x563e9a5ec080 2024-03-21T10:31:58.254 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:58.252+0000 7f79a9600640 1 --2- 172.21.15.47:0/3552989258 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f79900a3090 0x7f79900a5550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:31:58.258 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:58.253+0000 7f79abe00640 1 --2- 172.21.15.47:0/1112948370 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563e9a5ec080 0x563e9a5f4630 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f799c009c20 tx=0x7f799c011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8c87a0f20692b7a5 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:31:58.258 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:58.253+0000 7f79a9600640 1 -- 172.21.15.47:0/3552989258 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 9474+0+0 (secure 0 0 0) 0x7f79a40f1cc0 con 0x7f79ac10bd00 2024-03-21T10:31:58.258 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:58.253+0000 7f79a3400640 1 --2- 172.21.15.47:0/3552989258 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f79900a3090 0x7f79900a5550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:31:58.258 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:58.253+0000 7f79a1600640 1 -- 172.21.15.47:0/1112948370 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f799c004050 con 0x563e9a5ec080 2024-03-21T10:31:58.258 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:58.253+0000 7f79a1600640 1 -- 172.21.15.47:0/1112948370 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f799c0054d0 con 0x563e9a5ec080 2024-03-21T10:31:58.258 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:58.253+0000 7f79a1600640 1 -- 172.21.15.47:0/1112948370 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f799c004cf0 con 0x563e9a5ec080 2024-03-21T10:31:58.258 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:58.254+0000 7f79b5f79740 1 -- 172.21.15.47:0/1112948370 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563e9a5ec080 msgr2=0x563e9a5f4630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:31:58.258 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:58.254+0000 7f79a3400640 1 --2- 172.21.15.47:0/3552989258 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f79900a3090 0x7f79900a5550 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f7994004200 tx=0x7f799400b6e0 comp rx=0 tx=0).ready entity=mgr.15687 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:31:58.259 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:58.254+0000 7f79b5f79740 1 --2- 172.21.15.47:0/1112948370 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563e9a5ec080 0x563e9a5f4630 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f799c009c20 tx=0x7f799c011810 comp rx=0 tx=0).stop 2024-03-21T10:31:58.259 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:58.254+0000 7f79b5f79740 1 -- 172.21.15.47:0/1112948370 shutdown_connections 2024-03-21T10:31:58.259 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:58.254+0000 7f79b5f79740 1 --2- 172.21.15.47:0/1112948370 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x563e9a5f4c00 0x563e9a5f7050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:31:58.259 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:58.254+0000 7f79b5f79740 1 --2- 172.21.15.47:0/1112948370 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563e9a5ec080 0x563e9a5f4630 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:31:58.259 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:58.254+0000 7f79b5f79740 1 --2- 172.21.15.47:0/1112948370 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563e9a5eb6d0 0x563e9a5ebab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:31:58.259 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:58.254+0000 7f79b5f79740 1 -- 172.21.15.47:0/1112948370 >> 172.21.15.47:0/1112948370 conn(0x563e9a5e5da0 msgr2=0x563e9a5e6180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:31:58.259 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:58.254+0000 7f79b5f79740 1 -- 172.21.15.47:0/1112948370 shutdown_connections 2024-03-21T10:31:58.259 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:58.254+0000 7f79b5f79740 1 -- 172.21.15.47:0/1112948370 wait complete. 2024-03-21T10:31:58.259 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:58.255+0000 7f79b5f79740 1 -- start start 2024-03-21T10:31:58.259 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:58.255+0000 7f79b5f79740 1 Processor -- start 2024-03-21T10:31:58.259 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:58.255+0000 7f79b5f79740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x563e9a5eb6d0 0x563e9a5f4410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:31:58.259 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:58.255+0000 7f79b5f79740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563e9a5ec080 0x563e9a5f17a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:31:58.260 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:58.255+0000 7f79b5f79740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563e9a5f4c00 0x563e9a5f1ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:31:58.260 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:58.255+0000 7f79b5f79740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x563e9a5f7b70 con 0x563e9a5eb6d0 2024-03-21T10:31:58.260 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:58.255+0000 7f79b5f79740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x563e9a50b0d0 con 0x563e9a5ec080 2024-03-21T10:31:58.260 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:58.255+0000 7f79b5f79740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x563e9a5f7cf0 con 0x563e9a5f4c00 2024-03-21T10:31:58.260 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:58.256+0000 7f79b5f79740 10 client.0 _ll_register_callbacks cb 0x563e9a50fe80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T10:31:58.260 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:58.256+0000 7f79abe00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x563e9a5eb6d0 0x563e9a5f4410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:31:58.260 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:58.256+0000 7f79abe00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x563e9a5eb6d0 0x563e9a5f4410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:57918/0 (socket says 172.21.15.47:57918) 2024-03-21T10:31:58.260 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:58.256+0000 7f79abe00640 1 -- 172.21.15.47:0/2409074095 learned_addr learned my addr 172.21.15.47:0/2409074095 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:31:58.260 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:58.256+0000 7f79a3400640 1 --2- 172.21.15.47:0/2409074095 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563e9a5ec080 0x563e9a5f17a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:31:58.260 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:58.256+0000 7f79b1200640 1 --2- 172.21.15.47:0/2409074095 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563e9a5f4c00 0x563e9a5f1ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:31:58.260 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:58.256+0000 7f79abe00640 1 -- 172.21.15.47:0/2409074095 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563e9a5f4c00 msgr2=0x563e9a5f1ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:31:58.260 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:58.256+0000 7f79abe00640 1 --2- 172.21.15.47:0/2409074095 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563e9a5f4c00 0x563e9a5f1ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:31:58.260 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:58.256+0000 7f79abe00640 1 -- 172.21.15.47:0/2409074095 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563e9a5ec080 msgr2=0x563e9a5f17a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:31:58.260 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:58.256+0000 7f79abe00640 1 --2- 172.21.15.47:0/2409074095 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563e9a5ec080 0x563e9a5f17a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:31:58.260 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:58.256+0000 7f79abe00640 1 -- 172.21.15.47:0/2409074095 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x563e9a2633f0 con 0x563e9a5eb6d0 2024-03-21T10:31:58.261 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:58.257+0000 7f79abe00640 1 --2- 172.21.15.47:0/2409074095 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x563e9a5eb6d0 0x563e9a5f4410 secure :-1 s=READY pgs=999 cs=0 l=1 rev1=1 crypto rx=0x7f799c0049d0 tx=0x7f799c0038a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=626f69f87770788c server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:31:58.261 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:58.257+0000 7f799a000640 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-03-21T10:31:58.261 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:58.257+0000 7f799a000640 1 -- 172.21.15.47:0/2409074095 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f799c029070 con 0x563e9a5eb6d0 2024-03-21T10:31:58.261 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:58.257+0000 7f799a000640 1 -- 172.21.15.47:0/2409074095 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(97..97 src has 1..97) v4 ==== 9474+0+0 (secure 0 0 0) 0x7f799c003ca0 con 0x563e9a5eb6d0 2024-03-21T10:31:58.261 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:58.257+0000 7f79b5f79740 1 -- 172.21.15.47:0/2409074095 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f799c0054d0 con 0x563e9a5eb6d0 2024-03-21T10:31:58.261 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:58.257+0000 7f799a000640 1 -- 172.21.15.47:0/2409074095 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f799c035050 con 0x563e9a5eb6d0 2024-03-21T10:31:58.261 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:58.257+0000 7f79b5f79740 10 client.17670 fetch_fsmap learned FSMap version 32 2024-03-21T10:31:58.261 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:58.257+0000 7f79b5f79740 1 -- 172.21.15.47:0/2409074095 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x563e9a32d7d0 con 0x563e9a5eb6d0 2024-03-21T10:31:58.261 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:58.257+0000 7f799a000640 1 -- 172.21.15.47:0/2409074095 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f799c0054d0 con 0x563e9a5eb6d0 2024-03-21T10:31:58.261 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:58.258+0000 7f799a000640 1 -- 172.21.15.47:0/2409074095 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 32) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f799c0050e0 con 0x563e9a5eb6d0 2024-03-21T10:31:58.261 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:58.258+0000 7f79b5f79740 10 client.17670 fetch_fsmap finished waiting for FSMap version 32 2024-03-21T10:31:58.261 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:58.258+0000 7f79b5f79740 10 client.17670 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-21T10:31:58.261 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:58.258+0000 7f79b5f79740 20 client.17670 populate_metadata read hostname 'smithi047' 2024-03-21T10:31:58.261 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:58.258+0000 7f79b5f79740 1 --2- 172.21.15.47:0/2409074095 >> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] conn(0x563e9a604870 0x563e9a606c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:31:58.261 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:58.258+0000 7f79b5f79740 4 client.17670 mds_command: new command op to 34535 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T10:31:58.262 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:58.258+0000 7f79b5f79740 1 -- 172.21.15.47:0/2409074095 --> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x563e9a55c800 con 0x563e9a604870 2024-03-21T10:31:58.262 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:58.258+0000 7f79a3400640 1 --2- 172.21.15.47:0/2409074095 >> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] conn(0x563e9a604870 0x563e9a606c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:31:58.262 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:58.259+0000 7f79a3400640 1 --2- 172.21.15.47:0/2409074095 >> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] conn(0x563e9a604870 0x563e9a606c60 crc :-1 s=READY pgs=163 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9e607d8da826630c server_cookie=484a787001297120 in_seq=0 out_seq=0 2024-03-21T10:31:58.262 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:58.259+0000 7f799a000640 10 client.17670 ms_handle_connect on v2:172.21.15.100:6838/4173168440 2024-03-21T10:31:58.263 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:58.261+0000 7f799a000640 1 -- 172.21.15.47:0/2409074095 <== mds.0 v2:172.21.15.100:6838/4173168440 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x563e9a55c800 con 0x563e9a604870 2024-03-21T10:31:58.263 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:58.261+0000 7f799a000640 10 client.17670 handle_command_reply: tid=0 2024-03-21T10:31:58.263 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:58.261+0000 7f79b5f79740 1 client.17670 shutdown 2024-03-21T10:31:58.263 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:58.261+0000 7f79b5f79740 1 -- 172.21.15.47:0/2409074095 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x563e9a5eb6d0 msgr2=0x563e9a5f4410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:31:58.263 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:58.261+0000 7f79b5f79740 1 --2- 172.21.15.47:0/2409074095 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x563e9a5eb6d0 0x563e9a5f4410 secure :-1 s=READY pgs=999 cs=0 l=1 rev1=1 crypto rx=0x7f799c0049d0 tx=0x7f799c0038a0 comp rx=0 tx=0).stop 2024-03-21T10:31:58.263 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:58.261+0000 7f79b5f79740 1 -- 172.21.15.47:0/2409074095 shutdown_connections 2024-03-21T10:31:58.263 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:58.261+0000 7f79b5f79740 1 --2- 172.21.15.47:0/2409074095 >> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] conn(0x563e9a604870 0x563e9a606c60 crc :-1 s=READY pgs=163 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:31:58.263 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:58.261+0000 7f79b5f79740 1 --2- 172.21.15.47:0/2409074095 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563e9a5f4c00 0x563e9a5f1ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:31:58.263 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:58.261+0000 7f79b5f79740 1 --2- 172.21.15.47:0/2409074095 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563e9a5ec080 0x563e9a5f17a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:31:58.263 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:58.261+0000 7f79b5f79740 1 --2- 172.21.15.47:0/2409074095 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x563e9a5eb6d0 0x563e9a5f4410 unknown :-1 s=CLOSED pgs=999 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:31:58.264 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:58.261+0000 7f79b5f79740 1 -- 172.21.15.47:0/2409074095 >> 172.21.15.47:0/2409074095 conn(0x563e9a5e5da0 msgr2=0x563e9a5e7af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:31:58.264 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:58.261+0000 7f799a000640 0 client.17670 ms_handle_reset on v2:172.21.15.100:6838/4173168440 2024-03-21T10:31:58.264 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:58.261+0000 7f79b5f79740 1 -- 172.21.15.47:0/2409074095 shutdown_connections 2024-03-21T10:31:58.264 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:58.261+0000 7f79b5f79740 1 -- 172.21.15.47:0/2409074095 wait complete. 2024-03-21T10:31:58.264 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:58.262+0000 7f79b5f79740 20 client.17670 trim_cache size 0 max 16384 2024-03-21T10:31:58.275 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:58.275+0000 7f79b5f79740 1 Processor -- start 2024-03-21T10:31:58.275 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:58.275+0000 7f79b5f79740 1 -- start start 2024-03-21T10:31:58.275 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:58.275+0000 7f79b5f79740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563e9a5eb6d0 0x563e9a5f3ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:31:58.275 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:58.275+0000 7f79b5f79740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x563e9a5ec080 0x563e9a5f4410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:31:58.275 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:58.275+0000 7f79b5f79740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563e9a5f4c00 0x563e9a5eea20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:31:58.275 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:58.275+0000 7f79b5f79740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79ac1b0ae0 con 0x563e9a5ec080 2024-03-21T10:31:58.275 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:58.276+0000 7f79b5f79740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79ac1b2460 con 0x563e9a5f4c00 2024-03-21T10:31:58.275 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:58.276+0000 7f79b5f79740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79ac1b1140 con 0x563e9a5eb6d0 2024-03-21T10:31:58.275 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:58.276+0000 7f79a3400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x563e9a5ec080 0x563e9a5f4410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:31:58.275 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:58.276+0000 7f79a3400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x563e9a5ec080 0x563e9a5f4410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:57922/0 (socket says 172.21.15.47:57922) 2024-03-21T10:31:58.276 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:58.276+0000 7f79a3400640 1 -- 172.21.15.47:0/1256280180 learned_addr learned my addr 172.21.15.47:0/1256280180 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:31:58.276 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:58.276+0000 7f79b1200640 1 --2- 172.21.15.47:0/1256280180 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563e9a5f4c00 0x563e9a5eea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:31:58.276 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:58.276+0000 7f79abe00640 1 --2- 172.21.15.47:0/1256280180 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563e9a5eb6d0 0x563e9a5f3ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:31:58.276 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:58.276+0000 7f79a3400640 1 -- 172.21.15.47:0/1256280180 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563e9a5eb6d0 msgr2=0x563e9a5f3ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:31:58.276 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:58.276+0000 7f79a3400640 1 --2- 172.21.15.47:0/1256280180 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563e9a5eb6d0 0x563e9a5f3ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:31:58.276 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:58.276+0000 7f79a3400640 1 -- 172.21.15.47:0/1256280180 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563e9a5f4c00 msgr2=0x563e9a5eea20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:31:58.276 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:58.276+0000 7f79a3400640 1 --2- 172.21.15.47:0/1256280180 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563e9a5f4c00 0x563e9a5eea20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:31:58.276 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:58.276+0000 7f79a3400640 1 -- 172.21.15.47:0/1256280180 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563e9a5efcf0 con 0x563e9a5ec080 2024-03-21T10:31:58.278 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:58.277+0000 7f79a3400640 1 --2- 172.21.15.47:0/1256280180 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x563e9a5ec080 0x563e9a5f4410 secure :-1 s=READY pgs=1000 cs=0 l=1 rev1=1 crypto rx=0x7f799400c010 tx=0x7f79940046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e4669756851cbeec server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:31:58.278 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:58.277+0000 7f79a0c00640 1 -- 172.21.15.47:0/1256280180 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f799402b260 con 0x563e9a5ec080 2024-03-21T10:31:58.278 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:58.277+0000 7f79a0c00640 1 -- 172.21.15.47:0/1256280180 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f7994005020 con 0x563e9a5ec080 2024-03-21T10:31:58.278 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:58.277+0000 7f79a0c00640 1 -- 172.21.15.47:0/1256280180 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79940047a0 con 0x563e9a5ec080 2024-03-21T10:31:58.278 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:58.278+0000 7f79b5f79740 1 -- 172.21.15.47:0/1256280180 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x563e9a5ec080 msgr2=0x563e9a5f4410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:31:58.278 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:58.278+0000 7f79b5f79740 1 --2- 172.21.15.47:0/1256280180 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x563e9a5ec080 0x563e9a5f4410 secure :-1 s=READY pgs=1000 cs=0 l=1 rev1=1 crypto rx=0x7f799400c010 tx=0x7f79940046c0 comp rx=0 tx=0).stop 2024-03-21T10:31:58.279 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:58.278+0000 7f79b5f79740 1 -- 172.21.15.47:0/1256280180 shutdown_connections 2024-03-21T10:31:58.279 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:58.278+0000 7f79b5f79740 1 --2- 172.21.15.47:0/1256280180 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563e9a5f4c00 0x563e9a5eea20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:31:58.279 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:58.278+0000 7f79b5f79740 1 --2- 172.21.15.47:0/1256280180 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x563e9a5ec080 0x563e9a5f4410 unknown :-1 s=CLOSED pgs=1000 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:31:58.279 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:58.278+0000 7f79b5f79740 1 --2- 172.21.15.47:0/1256280180 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563e9a5eb6d0 0x563e9a5f3ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:31:58.279 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:58.278+0000 7f79b5f79740 1 -- 172.21.15.47:0/1256280180 >> 172.21.15.47:0/1256280180 conn(0x563e9a5e5da0 msgr2=0x563e9a5f07d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:31:58.279 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:58.278+0000 7f79b5f79740 1 -- 172.21.15.47:0/1256280180 shutdown_connections 2024-03-21T10:31:58.279 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:58.278+0000 7f79b5f79740 1 -- 172.21.15.47:0/1256280180 wait complete. 2024-03-21T10:31:58.279 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:58.278+0000 7f79b5f79740 1 -- start start 2024-03-21T10:31:58.279 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:58.279+0000 7f79b5f79740 1 Processor -- start 2024-03-21T10:31:58.279 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:58.279+0000 7f79b5f79740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563e9a5eb6d0 0x563e9a5fca00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:31:58.279 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:58.279+0000 7f79b5f79740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563e9a5ec080 0x563e9a5f7f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:31:58.279 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:58.279+0000 7f79b5f79740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x563e9a5f4c00 0x563e9a5f84a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:31:58.279 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:58.279+0000 7f79b5f79740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79ac1b1e00 con 0x563e9a5f4c00 2024-03-21T10:31:58.279 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:58.279+0000 7f79b5f79740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79ac1b17a0 con 0x563e9a5ec080 2024-03-21T10:31:58.279 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:58.279+0000 7f79b5f79740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79ac1aeca0 con 0x563e9a5eb6d0 2024-03-21T10:31:58.280 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:58.279+0000 7f79b5f79740 10 client.0 _ll_register_callbacks cb 0x7f799400ba30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T10:31:58.280 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:58.279+0000 7f79b1200640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x563e9a5f4c00 0x563e9a5f84a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:31:58.280 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:58.280+0000 7f79b1200640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x563e9a5f4c00 0x563e9a5f84a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:57930/0 (socket says 172.21.15.47:57930) 2024-03-21T10:31:58.280 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:58.280+0000 7f79b1200640 1 -- 172.21.15.47:0/3575396413 learned_addr learned my addr 172.21.15.47:0/3575396413 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:31:58.280 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:58.280+0000 7f79a3400640 1 --2- 172.21.15.47:0/3575396413 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563e9a5ec080 0x563e9a5f7f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:31:58.280 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:58.280+0000 7f79abe00640 1 --2- 172.21.15.47:0/3575396413 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563e9a5eb6d0 0x563e9a5fca00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:31:58.282 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:58.280+0000 7f79b1200640 1 -- 172.21.15.47:0/3575396413 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563e9a5eb6d0 msgr2=0x563e9a5fca00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:31:58.283 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:58.280+0000 7f79b1200640 1 --2- 172.21.15.47:0/3575396413 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563e9a5eb6d0 0x563e9a5fca00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:31:58.283 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:58.280+0000 7f79b1200640 1 -- 172.21.15.47:0/3575396413 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563e9a5ec080 msgr2=0x563e9a5f7f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:31:58.283 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:58.280+0000 7f79b1200640 1 --2- 172.21.15.47:0/3575396413 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563e9a5ec080 0x563e9a5f7f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:31:58.283 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:58.280+0000 7f79b1200640 1 -- 172.21.15.47:0/3575396413 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x563e9a5f89e0 con 0x563e9a5f4c00 2024-03-21T10:31:58.283 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:58.280+0000 7f79b1200640 1 --2- 172.21.15.47:0/3575396413 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x563e9a5f4c00 0x563e9a5f84a0 secure :-1 s=READY pgs=1001 cs=0 l=1 rev1=1 crypto rx=0x7f79a4009430 tx=0x7f79a40bdd20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6bc25e341c658dcf server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:31:58.283 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:58.280+0000 7f799a000640 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-03-21T10:31:58.283 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:58.280+0000 7f799a000640 1 -- 172.21.15.47:0/3575396413 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79a40c4070 con 0x563e9a5f4c00 2024-03-21T10:31:58.283 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:58.281+0000 7f799a000640 1 -- 172.21.15.47:0/3575396413 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(97..97 src has 1..97) v4 ==== 9474+0+0 (secure 0 0 0) 0x7f79a40be0f0 con 0x563e9a5f4c00 2024-03-21T10:31:58.283 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:58.281+0000 7f79b5f79740 1 -- 172.21.15.47:0/3575396413 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f7994005020 con 0x563e9a5f4c00 2024-03-21T10:31:58.283 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:58.281+0000 7f799a000640 1 -- 172.21.15.47:0/3575396413 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f79a40be590 con 0x563e9a5f4c00 2024-03-21T10:31:58.283 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:58.281+0000 7f79b5f79740 10 client.17676 fetch_fsmap learned FSMap version 32 2024-03-21T10:31:58.283 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:58.281+0000 7f79b5f79740 1 -- 172.21.15.47:0/3575396413 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x563e9a65fca0 con 0x563e9a5f4c00 2024-03-21T10:31:58.283 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:58.281+0000 7f799a000640 1 -- 172.21.15.47:0/3575396413 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f7994005020 con 0x563e9a5f4c00 2024-03-21T10:31:58.284 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:58.281+0000 7f799a000640 1 -- 172.21.15.47:0/3575396413 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 32) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f79a403c030 con 0x563e9a5f4c00 2024-03-21T10:31:58.284 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:58.281+0000 7f79b5f79740 10 client.17676 fetch_fsmap finished waiting for FSMap version 32 2024-03-21T10:31:58.284 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:58.281+0000 7f79b5f79740 10 client.17676 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-21T10:31:58.284 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:58.281+0000 7f79b5f79740 20 client.17676 populate_metadata read hostname 'smithi047' 2024-03-21T10:31:58.284 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:58.281+0000 7f79b5f79740 1 --2- 172.21.15.47:0/3575396413 >> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] conn(0x563e9a6603a0 0x563e9a6627c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:31:58.284 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:58.281+0000 7f79b5f79740 4 client.17676 mds_command: new command op to 34535 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-21T10:31:58.284 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:58.281+0000 7f79b5f79740 1 -- 172.21.15.47:0/3575396413 --> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x563e9a5f2070 con 0x563e9a6603a0 2024-03-21T10:31:58.284 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:58.282+0000 7f79abe00640 1 --2- 172.21.15.47:0/3575396413 >> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] conn(0x563e9a6603a0 0x563e9a6627c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:31:58.284 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:58.283+0000 7f79abe00640 1 --2- 172.21.15.47:0/3575396413 >> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] conn(0x563e9a6603a0 0x563e9a6627c0 crc :-1 s=READY pgs=164 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4ad2aa86ee29b35f server_cookie=202a470dee9516b1 in_seq=0 out_seq=0 2024-03-21T10:31:58.284 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:58.283+0000 7f799a000640 10 client.17676 ms_handle_connect on v2:172.21.15.100:6838/4173168440 2024-03-21T10:31:58.284 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:58.283+0000 7f799a000640 1 -- 172.21.15.47:0/3575396413 <== mds.0 v2:172.21.15.100:6838/4173168440 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x563e9a5f2070 con 0x563e9a6603a0 2024-03-21T10:31:58.284 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:58.283+0000 7f799a000640 10 client.17676 handle_command_reply: tid=0 2024-03-21T10:31:58.284 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:58.283+0000 7f79b5f79740 1 client.17676 shutdown 2024-03-21T10:31:58.284 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:58.283+0000 7f79b5f79740 1 -- 172.21.15.47:0/3575396413 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x563e9a5f4c00 msgr2=0x563e9a5f84a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:31:58.285 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:58.283+0000 7f79b5f79740 1 --2- 172.21.15.47:0/3575396413 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x563e9a5f4c00 0x563e9a5f84a0 secure :-1 s=READY pgs=1001 cs=0 l=1 rev1=1 crypto rx=0x7f79a4009430 tx=0x7f79a40bdd20 comp rx=0 tx=0).stop 2024-03-21T10:31:58.285 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:58.283+0000 7f79b5f79740 1 -- 172.21.15.47:0/3575396413 shutdown_connections 2024-03-21T10:31:58.285 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:58.283+0000 7f79b5f79740 1 --2- 172.21.15.47:0/3575396413 >> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] conn(0x563e9a6603a0 0x563e9a6627c0 crc :-1 s=READY pgs=164 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:31:58.285 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:58.283+0000 7f79b5f79740 1 --2- 172.21.15.47:0/3575396413 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x563e9a5f4c00 0x563e9a5f84a0 unknown :-1 s=CLOSED pgs=1001 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:31:58.285 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:58.283+0000 7f79b5f79740 1 --2- 172.21.15.47:0/3575396413 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563e9a5ec080 0x563e9a5f7f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:31:58.285 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:58.283+0000 7f79b5f79740 1 --2- 172.21.15.47:0/3575396413 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563e9a5eb6d0 0x563e9a5fca00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:31:58.285 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:58.283+0000 7f79b5f79740 1 -- 172.21.15.47:0/3575396413 >> 172.21.15.47:0/3575396413 conn(0x563e9a5e5da0 msgr2=0x563e9a5e70b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:31:58.285 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:58.283+0000 7f799a000640 0 client.17676 ms_handle_reset on v2:172.21.15.100:6838/4173168440 2024-03-21T10:31:58.285 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:58.283+0000 7f79b5f79740 1 -- 172.21.15.47:0/3575396413 shutdown_connections 2024-03-21T10:31:58.285 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:58.283+0000 7f79b5f79740 1 -- 172.21.15.47:0/3575396413 wait complete. 2024-03-21T10:31:58.285 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:58.284+0000 7f79b5f79740 20 client.17676 trim_cache size 0 max 16384 2024-03-21T10:31:58.285 INFO:teuthology.orchestra.run.smithi047.stdout:[] 2024-03-21T10:31:58.286 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:58.284+0000 7f79b3200640 1 -- 172.21.15.47:0/3552989258 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f79900a3090 msgr2=0x7f79900a5550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:31:58.286 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:58.284+0000 7f79b3200640 1 --2- 172.21.15.47:0/3552989258 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f79900a3090 0x7f79900a5550 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f7994004200 tx=0x7f799400b6e0 comp rx=0 tx=0).stop 2024-03-21T10:31:58.286 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:58.284+0000 7f79b3200640 1 -- 172.21.15.47:0/3552989258 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f79ac10bd00 msgr2=0x7f79ac19e940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:31:58.286 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:58.284+0000 7f79b3200640 1 --2- 172.21.15.47:0/3552989258 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f79ac10bd00 0x7f79ac19e940 secure :-1 s=READY pgs=998 cs=0 l=1 rev1=1 crypto rx=0x7f79a4037040 tx=0x7f79a4035e00 comp rx=0 tx=0).stop 2024-03-21T10:31:58.287 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:58.284+0000 7f79b3200640 1 -- 172.21.15.47:0/3552989258 shutdown_connections 2024-03-21T10:31:58.287 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:58.284+0000 7f79b3200640 1 --2- 172.21.15.47:0/3552989258 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f79ac10bd00 0x7f79ac19e940 unknown :-1 s=CLOSED pgs=998 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:31:58.287 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:58.284+0000 7f79b3200640 1 --2- 172.21.15.47:0/3552989258 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f79ac103250 0x7f79ac19a5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:31:58.287 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:58.284+0000 7f79b3200640 1 --2- 172.21.15.47:0/3552989258 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f79900a3090 0x7f79900a5550 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:31:58.287 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:58.284+0000 7f79b3200640 1 --2- 172.21.15.47:0/3552989258 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f79ac102930 0x7f79ac19a080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:31:58.287 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:58.284+0000 7f79b3200640 1 -- 172.21.15.47:0/3552989258 >> 172.21.15.47:0/3552989258 conn(0x7f79ac077700 msgr2=0x7f79ac0fd4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:31:58.287 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:58.285+0000 7f79b3200640 1 -- 172.21.15.47:0/3552989258 shutdown_connections 2024-03-21T10:31:58.287 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:58.285+0000 7f79b3200640 1 -- 172.21.15.47:0/3552989258 wait complete. 2024-03-21T10:31:59.301 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-03-21T10:31:59.302 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-21T10:31:59.302 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-03-21T10:31:59.535 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:59.536+0000 7fef77000640 1 -- 172.21.15.47:0/2616545067 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fef7006a3e0 msgr2=0x7fef7010e4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:31:59.535 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:59.536+0000 7fef77000640 1 --2- 172.21.15.47:0/2616545067 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fef7006a3e0 0x7fef7010e4e0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fef68009be0 tx=0x7fef6802bf80 comp rx=0 tx=0).stop 2024-03-21T10:31:59.535 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:59.536+0000 7fef77000640 1 -- 172.21.15.47:0/2616545067 shutdown_connections 2024-03-21T10:31:59.535 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:59.536+0000 7fef77000640 1 --2- 172.21.15.47:0/2616545067 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fef7006a3e0 0x7fef7010e4e0 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:31:59.535 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:59.536+0000 7fef77000640 1 --2- 172.21.15.47:0/2616545067 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fef70069a40 0x7fef70069ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:31:59.535 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:59.536+0000 7fef77000640 1 --2- 172.21.15.47:0/2616545067 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fef70107690 0x7fef70107a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:31:59.536 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:59.536+0000 7fef77000640 1 -- 172.21.15.47:0/2616545067 >> 172.21.15.47:0/2616545067 conn(0x7fef70077630 msgr2=0x7fef70077a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:31:59.536 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:59.536+0000 7fef77000640 1 -- 172.21.15.47:0/2616545067 shutdown_connections 2024-03-21T10:31:59.536 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:59.536+0000 7fef77000640 1 -- 172.21.15.47:0/2616545067 wait complete. 2024-03-21T10:31:59.538 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:59.537+0000 7fef77000640 1 Processor -- start 2024-03-21T10:31:59.538 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:59.537+0000 7fef77000640 1 -- start start 2024-03-21T10:31:59.538 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:59.538+0000 7fef77000640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fef70069a40 0x7fef7019a040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:31:59.538 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:59.538+0000 7fef77000640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fef7006a3e0 0x7fef7019a580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:31:59.539 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:59.538+0000 7fef77000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fef70107690 0x7fef7019e900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:31:59.539 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:59.538+0000 7fef77000640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef70111900 con 0x7fef70069a40 2024-03-21T10:31:59.539 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:59.538+0000 7fef77000640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef70111780 con 0x7fef70107690 2024-03-21T10:31:59.539 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:59.538+0000 7fef77000640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef70111a80 con 0x7fef7006a3e0 2024-03-21T10:31:59.539 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:59.538+0000 7fef6fe00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fef70069a40 0x7fef7019a040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:31:59.539 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:59.538+0000 7fef6fe00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fef70069a40 0x7fef7019a040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:57960/0 (socket says 172.21.15.47:57960) 2024-03-21T10:31:59.539 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:59.538+0000 7fef6fe00640 1 -- 172.21.15.47:0/2312198352 learned_addr learned my addr 172.21.15.47:0/2312198352 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:31:59.539 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:59.538+0000 7fef75000640 1 --2- 172.21.15.47:0/2312198352 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fef70107690 0x7fef7019e900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:31:59.539 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:59.538+0000 7fef6f400640 1 --2- 172.21.15.47:0/2312198352 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fef7006a3e0 0x7fef7019a580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:31:59.539 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:59.539+0000 7fef6fe00640 1 -- 172.21.15.47:0/2312198352 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fef7006a3e0 msgr2=0x7fef7019a580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:31:59.539 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:59.539+0000 7fef6fe00640 1 --2- 172.21.15.47:0/2312198352 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fef7006a3e0 0x7fef7019a580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:31:59.539 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:59.539+0000 7fef6fe00640 1 -- 172.21.15.47:0/2312198352 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fef70107690 msgr2=0x7fef7019e900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:31:59.539 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:59.539+0000 7fef6fe00640 1 --2- 172.21.15.47:0/2312198352 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fef70107690 0x7fef7019e900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:31:59.540 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:59.539+0000 7fef6fe00640 1 -- 172.21.15.47:0/2312198352 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef7019f030 con 0x7fef70069a40 2024-03-21T10:31:59.540 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:59.539+0000 7fef6f400640 1 --2- 172.21.15.47:0/2312198352 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fef7006a3e0 0x7fef7019a580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T10:31:59.540 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:59.539+0000 7fef6fe00640 1 --2- 172.21.15.47:0/2312198352 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fef70069a40 0x7fef7019a040 secure :-1 s=READY pgs=1002 cs=0 l=1 rev1=1 crypto rx=0x7fef6000e9c0 tx=0x7fef6000ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:31:59.540 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:59.539+0000 7fef6cc00640 1 -- 172.21.15.47:0/2312198352 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef6000cde0 con 0x7fef70069a40 2024-03-21T10:31:59.540 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:59.539+0000 7fef77000640 1 -- 172.21.15.47:0/2312198352 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fef7019f320 con 0x7fef70069a40 2024-03-21T10:31:59.540 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:59.539+0000 7fef6cc00640 1 -- 172.21.15.47:0/2312198352 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fef600035f0 con 0x7fef70069a40 2024-03-21T10:31:59.540 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:59.539+0000 7fef6cc00640 1 -- 172.21.15.47:0/2312198352 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef60010690 con 0x7fef70069a40 2024-03-21T10:31:59.540 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:59.540+0000 7fef77000640 1 -- 172.21.15.47:0/2312198352 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fef701a6bb0 con 0x7fef70069a40 2024-03-21T10:31:59.549 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:59.543+0000 7fef79d84740 1 Processor -- start 2024-03-21T10:31:59.549 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:59.543+0000 7fef79d84740 1 -- start start 2024-03-21T10:31:59.549 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:59.543+0000 7fef6cc00640 1 -- 172.21.15.47:0/2312198352 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 27) v1 ==== 137985+0+0 (secure 0 0 0) 0x7fef600037a0 con 0x7fef70069a40 2024-03-21T10:31:59.549 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:59.543+0000 7fef79d84740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fbcbaf8080 0x55fbcbaf8460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:31:59.549 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:59.543+0000 7fef79d84740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fbcbaf89a0 0x55fbcbb01040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:31:59.550 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:59.543+0000 7fef79d84740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55fbcbb01580 0x55fbcbb03970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:31:59.550 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:59.543+0000 7fef79d84740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fbcba5ceb0 con 0x55fbcbb01580 2024-03-21T10:31:59.550 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:59.543+0000 7fef79d84740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fbcba05a60 con 0x55fbcbaf8080 2024-03-21T10:31:59.550 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:59.543+0000 7fef79d84740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fbcba58f50 con 0x55fbcbaf89a0 2024-03-21T10:31:59.550 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:59.544+0000 7fef6f400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55fbcbb01580 0x55fbcbb03970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:31:59.550 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:59.544+0000 7fef6f400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55fbcbb01580 0x55fbcbb03970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:57970/0 (socket says 172.21.15.47:57970) 2024-03-21T10:31:59.550 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:59.544+0000 7fef75000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fbcbaf8080 0x55fbcbaf8460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:31:59.550 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:59.544+0000 7fef6f400640 1 -- 172.21.15.47:0/3665850611 learned_addr learned my addr 172.21.15.47:0/3665850611 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:31:59.550 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:59.544+0000 7fef6fe00640 1 --2- 172.21.15.47:0/3665850611 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fbcbaf89a0 0x55fbcbb01040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:31:59.550 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:59.544+0000 7fef75000640 1 -- 172.21.15.47:0/3665850611 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fbcbaf89a0 msgr2=0x55fbcbb01040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:31:59.550 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:59.544+0000 7fef75000640 1 --2- 172.21.15.47:0/3665850611 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fbcbaf89a0 0x55fbcbb01040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:31:59.550 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:59.544+0000 7fef75000640 1 -- 172.21.15.47:0/3665850611 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55fbcbb01580 msgr2=0x55fbcbb03970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:31:59.550 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:59.544+0000 7fef75000640 1 --2- 172.21.15.47:0/3665850611 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55fbcbb01580 0x55fbcbb03970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:31:59.551 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:59.545+0000 7fef75000640 1 -- 172.21.15.47:0/3665850611 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55fbcb8397e0 con 0x55fbcbaf8080 2024-03-21T10:31:59.551 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:59.545+0000 7fef6f400640 1 --2- 172.21.15.47:0/3665850611 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55fbcbb01580 0x55fbcbb03970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T10:31:59.551 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:59.545+0000 7fef75000640 1 --2- 172.21.15.47:0/3665850611 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fbcbaf8080 0x55fbcbaf8460 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fef6802c4c0 tx=0x7fef680098b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e67e9cc56ad8b0b server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:31:59.551 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:59.545+0000 7fef6cc00640 1 --2- 172.21.15.47:0/2312198352 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7fef540a3090 0x7fef540a5550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:31:59.551 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:59.545+0000 7fef65600640 1 -- 172.21.15.47:0/3665850611 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef68043030 con 0x55fbcbaf8080 2024-03-21T10:31:59.551 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:59.545+0000 7fef65600640 1 -- 172.21.15.47:0/3665850611 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fef680333e0 con 0x55fbcbaf8080 2024-03-21T10:31:59.551 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:59.545+0000 7fef65600640 1 -- 172.21.15.47:0/3665850611 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef68003d20 con 0x55fbcbaf8080 2024-03-21T10:31:59.551 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:59.546+0000 7fef6cc00640 1 -- 172.21.15.47:0/2312198352 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 9474+0+0 (secure 0 0 0) 0x7fef600cff30 con 0x7fef70069a40 2024-03-21T10:31:59.551 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:59.546+0000 7fef6fe00640 1 --2- 172.21.15.47:0/2312198352 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7fef540a3090 0x7fef540a5550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:31:59.551 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:59.546+0000 7fef79d84740 1 -- 172.21.15.47:0/3665850611 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fbcbaf8080 msgr2=0x55fbcbaf8460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:31:59.552 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:59.546+0000 7fef79d84740 1 --2- 172.21.15.47:0/3665850611 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fbcbaf8080 0x55fbcbaf8460 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fef6802c4c0 tx=0x7fef680098b0 comp rx=0 tx=0).stop 2024-03-21T10:31:59.552 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:59.546+0000 7fef6fe00640 1 --2- 172.21.15.47:0/2312198352 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7fef540a3090 0x7fef540a5550 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x55fbcbaf7b50 tx=0x7fef60003e10 comp rx=0 tx=0).ready entity=mgr.15687 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:31:59.552 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:59.546+0000 7fef79d84740 1 -- 172.21.15.47:0/3665850611 shutdown_connections 2024-03-21T10:31:59.552 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:59.546+0000 7fef79d84740 1 --2- 172.21.15.47:0/3665850611 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55fbcbb01580 0x55fbcbb03970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:31:59.552 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:59.546+0000 7fef79d84740 1 --2- 172.21.15.47:0/3665850611 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fbcbaf89a0 0x55fbcbb01040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:31:59.552 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:59.546+0000 7fef79d84740 1 --2- 172.21.15.47:0/3665850611 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fbcbaf8080 0x55fbcbaf8460 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:31:59.552 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:59.546+0000 7fef79d84740 1 -- 172.21.15.47:0/3665850611 >> 172.21.15.47:0/3665850611 conn(0x55fbcbaf27b0 msgr2=0x55fbcbaf2b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:31:59.552 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:59.546+0000 7fef79d84740 1 -- 172.21.15.47:0/3665850611 shutdown_connections 2024-03-21T10:31:59.552 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:59.546+0000 7fef79d84740 1 -- 172.21.15.47:0/3665850611 wait complete. 2024-03-21T10:31:59.552 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:59.547+0000 7fef79d84740 1 -- start start 2024-03-21T10:31:59.552 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:59.547+0000 7fef79d84740 1 Processor -- start 2024-03-21T10:31:59.552 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:59.548+0000 7fef79d84740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fbcbaf8080 0x55fbcbb00ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:31:59.552 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:59.548+0000 7fef79d84740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55fbcbaf89a0 0x55fbcbafe280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:31:59.552 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:59.548+0000 7fef79d84740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fbcbb01580 0x55fbcbafe7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:31:59.553 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:59.548+0000 7fef79d84740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fbcbb04310 con 0x55fbcbaf89a0 2024-03-21T10:31:59.553 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:59.548+0000 7fef79d84740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fbcba18eb0 con 0x55fbcbaf8080 2024-03-21T10:31:59.553 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:59.548+0000 7fef79d84740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fbcbb04490 con 0x55fbcbb01580 2024-03-21T10:31:59.553 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:59.548+0000 7fef79d84740 10 client.0 _ll_register_callbacks cb 0x55fbcba0d150 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T10:31:59.553 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:59.548+0000 7fef6f400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55fbcbaf89a0 0x55fbcbafe280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:31:59.553 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:59.548+0000 7fef6f400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55fbcbaf89a0 0x55fbcbafe280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:57978/0 (socket says 172.21.15.47:57978) 2024-03-21T10:31:59.553 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:59.548+0000 7fef6f400640 1 -- 172.21.15.47:0/3670982822 learned_addr learned my addr 172.21.15.47:0/3670982822 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:31:59.553 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:59.548+0000 7fef75000640 1 --2- 172.21.15.47:0/3670982822 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fbcbaf8080 0x55fbcbb00ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:31:59.553 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:59.548+0000 7fef75000640 1 --2- 172.21.15.47:0/3670982822 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fbcbb01580 0x55fbcbafe7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:31:59.553 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:59.548+0000 7fef6f400640 1 -- 172.21.15.47:0/3670982822 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fbcbb01580 msgr2=0x55fbcbafe7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:31:59.553 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:59.548+0000 7fef6f400640 1 --2- 172.21.15.47:0/3670982822 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fbcbb01580 0x55fbcbafe7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:31:59.553 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:59.548+0000 7fef6f400640 1 -- 172.21.15.47:0/3670982822 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fbcbaf8080 msgr2=0x55fbcbb00ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:31:59.553 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:59.548+0000 7fef6f400640 1 --2- 172.21.15.47:0/3670982822 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fbcbaf8080 0x55fbcbb00ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:31:59.553 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:59.548+0000 7fef6f400640 1 -- 172.21.15.47:0/3670982822 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55fbcb899f70 con 0x55fbcbaf89a0 2024-03-21T10:31:59.554 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:59.548+0000 7fef75000640 1 --2- 172.21.15.47:0/3670982822 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fbcbaf8080 0x55fbcbb00ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T10:31:59.554 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:59.549+0000 7fef6f400640 1 --2- 172.21.15.47:0/3670982822 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55fbcbaf89a0 0x55fbcbafe280 secure :-1 s=READY pgs=1003 cs=0 l=1 rev1=1 crypto rx=0x7fef5c012700 tx=0x7fef5c012bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=828f899ab0ea27a0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:31:59.554 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:59.549+0000 7fef5a000640 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-03-21T10:31:59.554 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:59.549+0000 7fef5a000640 1 -- 172.21.15.47:0/3670982822 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef5c011bb0 con 0x55fbcbaf89a0 2024-03-21T10:31:59.554 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:59.549+0000 7fef5a000640 1 -- 172.21.15.47:0/3670982822 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(97..97 src has 1..97) v4 ==== 9474+0+0 (secure 0 0 0) 0x7fef5c011d50 con 0x55fbcbaf89a0 2024-03-21T10:31:59.554 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:59.549+0000 7fef79d84740 1 -- 172.21.15.47:0/3670982822 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fef680333e0 con 0x55fbcbaf89a0 2024-03-21T10:31:59.554 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:59.549+0000 7fef5a000640 1 -- 172.21.15.47:0/3670982822 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7fef5c025050 con 0x55fbcbaf89a0 2024-03-21T10:31:59.554 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:59.549+0000 7fef79d84740 10 client.17688 fetch_fsmap learned FSMap version 32 2024-03-21T10:31:59.555 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:59.550+0000 7fef79d84740 1 -- 172.21.15.47:0/3670982822 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x55fbcb731170 con 0x55fbcbaf89a0 2024-03-21T10:31:59.555 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:59.550+0000 7fef5a000640 1 -- 172.21.15.47:0/3670982822 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fef680333e0 con 0x55fbcbaf89a0 2024-03-21T10:31:59.555 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:59.550+0000 7fef5a000640 1 -- 172.21.15.47:0/3670982822 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 32) v1 ==== 7942+0+0 (secure 0 0 0) 0x7fef5c01c460 con 0x55fbcbaf89a0 2024-03-21T10:31:59.555 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:59.550+0000 7fef79d84740 10 client.17688 fetch_fsmap finished waiting for FSMap version 32 2024-03-21T10:31:59.555 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:59.550+0000 7fef79d84740 10 client.17688 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-21T10:31:59.555 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:59.550+0000 7fef79d84740 20 client.17688 populate_metadata read hostname 'smithi047' 2024-03-21T10:31:59.555 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:59.550+0000 7fef79d84740 1 --2- 172.21.15.47:0/3670982822 >> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] conn(0x55fbcbb115d0 0x55fbcbb139f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:31:59.555 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:59.550+0000 7fef79d84740 4 client.17688 mds_command: new command op to 34535 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T10:31:59.555 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:59.550+0000 7fef79d84740 1 -- 172.21.15.47:0/3670982822 --> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55fbcba6a3f0 con 0x55fbcbb115d0 2024-03-21T10:31:59.555 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:59.551+0000 7fef75000640 1 --2- 172.21.15.47:0/3670982822 >> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] conn(0x55fbcbb115d0 0x55fbcbb139f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:31:59.557 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:59.551+0000 7fef75000640 1 --2- 172.21.15.47:0/3670982822 >> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] conn(0x55fbcbb115d0 0x55fbcbb139f0 crc :-1 s=READY pgs=165 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9398861e36bac342 server_cookie=ff3f566eb18da10b in_seq=0 out_seq=0 2024-03-21T10:31:59.557 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:59.551+0000 7fef5a000640 10 client.17688 ms_handle_connect on v2:172.21.15.100:6838/4173168440 2024-03-21T10:31:59.557 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:59.553+0000 7fef5a000640 1 -- 172.21.15.47:0/3670982822 <== mds.0 v2:172.21.15.100:6838/4173168440 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55fbcba6a3f0 con 0x55fbcbb115d0 2024-03-21T10:31:59.557 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:59.553+0000 7fef5a000640 10 client.17688 handle_command_reply: tid=0 2024-03-21T10:31:59.557 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:59.553+0000 7fef79d84740 1 client.17688 shutdown 2024-03-21T10:31:59.557 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:59.553+0000 7fef79d84740 1 -- 172.21.15.47:0/3670982822 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55fbcbaf89a0 msgr2=0x55fbcbafe280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:31:59.557 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:59.553+0000 7fef79d84740 1 --2- 172.21.15.47:0/3670982822 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55fbcbaf89a0 0x55fbcbafe280 secure :-1 s=READY pgs=1003 cs=0 l=1 rev1=1 crypto rx=0x7fef5c012700 tx=0x7fef5c012bc0 comp rx=0 tx=0).stop 2024-03-21T10:31:59.557 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:59.553+0000 7fef79d84740 1 -- 172.21.15.47:0/3670982822 shutdown_connections 2024-03-21T10:31:59.557 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:59.553+0000 7fef79d84740 1 --2- 172.21.15.47:0/3670982822 >> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] conn(0x55fbcbb115d0 0x55fbcbb139f0 crc :-1 s=READY pgs=165 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:31:59.557 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:59.553+0000 7fef79d84740 1 --2- 172.21.15.47:0/3670982822 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fbcbb01580 0x55fbcbafe7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:31:59.557 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:59.553+0000 7fef79d84740 1 --2- 172.21.15.47:0/3670982822 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55fbcbaf89a0 0x55fbcbafe280 unknown :-1 s=CLOSED pgs=1003 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:31:59.557 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:59.553+0000 7fef79d84740 1 --2- 172.21.15.47:0/3670982822 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fbcbaf8080 0x55fbcbb00ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:31:59.558 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:59.553+0000 7fef79d84740 1 -- 172.21.15.47:0/3670982822 >> 172.21.15.47:0/3670982822 conn(0x55fbcbaf27b0 msgr2=0x55fbcbaf4500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:31:59.558 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:59.553+0000 7fef5a000640 0 client.17688 ms_handle_reset on v2:172.21.15.100:6838/4173168440 2024-03-21T10:31:59.558 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:59.553+0000 7fef79d84740 1 -- 172.21.15.47:0/3670982822 shutdown_connections 2024-03-21T10:31:59.558 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:59.553+0000 7fef79d84740 1 -- 172.21.15.47:0/3670982822 wait complete. 2024-03-21T10:31:59.558 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:59.554+0000 7fef79d84740 20 client.17688 trim_cache size 0 max 16384 2024-03-21T10:31:59.571 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:59.572+0000 7fef79d84740 1 Processor -- start 2024-03-21T10:31:59.572 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:59.572+0000 7fef79d84740 1 -- start start 2024-03-21T10:31:59.572 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:59.573+0000 7fef79d84740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55fbcbaf8080 0x55fbcbb00920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:31:59.572 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:59.573+0000 7fef79d84740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fbcbaf89a0 0x55fbcbb00e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:31:59.572 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:59.573+0000 7fef79d84740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fbcbb01580 0x55fbcbaf16b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:31:59.572 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:59.573+0000 7fef79d84740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef701b0a60 con 0x55fbcbaf8080 2024-03-21T10:31:59.572 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:59.573+0000 7fef79d84740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef701b23e0 con 0x55fbcbb01580 2024-03-21T10:31:59.572 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:59.573+0000 7fef79d84740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef701b10c0 con 0x55fbcbaf89a0 2024-03-21T10:31:59.573 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:59.573+0000 7fef75000640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55fbcbaf8080 0x55fbcbb00920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:31:59.573 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:59.573+0000 7fef75000640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55fbcbaf8080 0x55fbcbb00920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:57990/0 (socket says 172.21.15.47:57990) 2024-03-21T10:31:59.573 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:59.573+0000 7fef75000640 1 -- 172.21.15.47:0/1504323165 learned_addr learned my addr 172.21.15.47:0/1504323165 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:31:59.575 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:59.573+0000 7fef6f400640 1 --2- 172.21.15.47:0/1504323165 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fbcbaf89a0 0x55fbcbb00e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:31:59.575 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:59.573+0000 7fef75000640 1 --2- 172.21.15.47:0/1504323165 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fbcbb01580 0x55fbcbaf16b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:31:59.575 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:59.574+0000 7fef75000640 1 -- 172.21.15.47:0/1504323165 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fbcbaf89a0 msgr2=0x55fbcbb00e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:31:59.575 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:59.574+0000 7fef75000640 1 --2- 172.21.15.47:0/1504323165 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fbcbaf89a0 0x55fbcbb00e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:31:59.575 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:59.574+0000 7fef75000640 1 -- 172.21.15.47:0/1504323165 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fbcbb01580 msgr2=0x55fbcbaf16b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:31:59.575 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:59.574+0000 7fef75000640 1 --2- 172.21.15.47:0/1504323165 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fbcbb01580 0x55fbcbaf16b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:31:59.576 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:59.574+0000 7fef75000640 1 -- 172.21.15.47:0/1504323165 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55fbcbb14050 con 0x55fbcbaf8080 2024-03-21T10:31:59.576 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:59.574+0000 7fef75000640 1 --2- 172.21.15.47:0/1504323165 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55fbcbaf8080 0x55fbcbb00920 secure :-1 s=READY pgs=1004 cs=0 l=1 rev1=1 crypto rx=0x7fef68009be0 tx=0x7fef68009500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5aa8dc3e1d850929 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:31:59.576 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:59.575+0000 7fef64c00640 1 -- 172.21.15.47:0/1504323165 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef68043040 con 0x55fbcbaf8080 2024-03-21T10:31:59.576 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:59.575+0000 7fef64c00640 1 -- 172.21.15.47:0/1504323165 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fef680333e0 con 0x55fbcbaf8080 2024-03-21T10:31:59.576 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:59.575+0000 7fef64c00640 1 -- 172.21.15.47:0/1504323165 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef6802cb80 con 0x55fbcbaf8080 2024-03-21T10:31:59.576 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:59.575+0000 7fef79d84740 1 -- 172.21.15.47:0/1504323165 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55fbcbaf8080 msgr2=0x55fbcbb00920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:31:59.576 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:59.575+0000 7fef79d84740 1 --2- 172.21.15.47:0/1504323165 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55fbcbaf8080 0x55fbcbb00920 secure :-1 s=READY pgs=1004 cs=0 l=1 rev1=1 crypto rx=0x7fef68009be0 tx=0x7fef68009500 comp rx=0 tx=0).stop 2024-03-21T10:31:59.576 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:59.576+0000 7fef79d84740 1 -- 172.21.15.47:0/1504323165 shutdown_connections 2024-03-21T10:31:59.576 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:59.576+0000 7fef79d84740 1 --2- 172.21.15.47:0/1504323165 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fbcbb01580 0x55fbcbaf16b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:31:59.576 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:59.576+0000 7fef79d84740 1 --2- 172.21.15.47:0/1504323165 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fbcbaf89a0 0x55fbcbb00e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:31:59.576 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:59.576+0000 7fef79d84740 1 --2- 172.21.15.47:0/1504323165 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55fbcbaf8080 0x55fbcbb00920 unknown :-1 s=CLOSED pgs=1004 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:31:59.576 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:59.576+0000 7fef79d84740 1 -- 172.21.15.47:0/1504323165 >> 172.21.15.47:0/1504323165 conn(0x55fbcbaf27b0 msgr2=0x55fbcbaf0fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:31:59.576 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:59.576+0000 7fef79d84740 1 -- 172.21.15.47:0/1504323165 shutdown_connections 2024-03-21T10:31:59.576 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:59.576+0000 7fef79d84740 1 -- 172.21.15.47:0/1504323165 wait complete. 2024-03-21T10:31:59.577 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:59.576+0000 7fef79d84740 1 -- start start 2024-03-21T10:31:59.577 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:59.576+0000 7fef79d84740 1 Processor -- start 2024-03-21T10:31:59.579 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:59.577+0000 7fef79d84740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55fbcbaf8080 0x55fbcbb066b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:31:59.579 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:59.577+0000 7fef79d84740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fbcbaf89a0 0x55fbcbb06bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:31:59.579 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:59.577+0000 7fef79d84740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fbcbb01580 0x55fbcbb07130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:31:59.580 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:59.577+0000 7fef79d84740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef701b1d80 con 0x55fbcbaf8080 2024-03-21T10:31:59.580 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:59.577+0000 7fef79d84740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef701b1720 con 0x55fbcbb01580 2024-03-21T10:31:59.580 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:59.577+0000 7fef79d84740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef701aec20 con 0x55fbcbaf89a0 2024-03-21T10:31:59.580 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:59.577+0000 7fef79d84740 10 client.0 _ll_register_callbacks cb 0x7fef5c009820 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T10:31:59.580 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:59.577+0000 7fef75000640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55fbcbaf8080 0x55fbcbb066b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:31:59.580 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:59.577+0000 7fef75000640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55fbcbaf8080 0x55fbcbb066b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:58000/0 (socket says 172.21.15.47:58000) 2024-03-21T10:31:59.581 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:59.577+0000 7fef75000640 1 -- 172.21.15.47:0/1484141456 learned_addr learned my addr 172.21.15.47:0/1484141456 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:31:59.581 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:59.577+0000 7fef6f400640 1 --2- 172.21.15.47:0/1484141456 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fbcbaf89a0 0x55fbcbb06bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:31:59.581 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:59.577+0000 7fef75000640 1 --2- 172.21.15.47:0/1484141456 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fbcbb01580 0x55fbcbb07130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:31:59.581 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:59.578+0000 7fef75000640 1 -- 172.21.15.47:0/1484141456 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fbcbaf89a0 msgr2=0x55fbcbb06bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:31:59.581 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:59.578+0000 7fef75000640 1 --2- 172.21.15.47:0/1484141456 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fbcbaf89a0 0x55fbcbb06bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:31:59.581 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:59.578+0000 7fef75000640 1 -- 172.21.15.47:0/1484141456 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fbcbb01580 msgr2=0x55fbcbb07130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:31:59.582 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:59.578+0000 7fef75000640 1 --2- 172.21.15.47:0/1484141456 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fbcbb01580 0x55fbcbb07130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:31:59.582 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:59.578+0000 7fef75000640 1 -- 172.21.15.47:0/1484141456 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55fbcbb03ba0 con 0x55fbcbaf8080 2024-03-21T10:31:59.582 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:59.578+0000 7fef75000640 1 --2- 172.21.15.47:0/1484141456 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55fbcbaf8080 0x55fbcbb066b0 secure :-1 s=READY pgs=1005 cs=0 l=1 rev1=1 crypto rx=0x7fef68009b30 tx=0x7fef68033820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3553ab4e3ad96a31 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:31:59.582 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:59.578+0000 7fef5a000640 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-03-21T10:31:59.582 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:59.578+0000 7fef5a000640 1 -- 172.21.15.47:0/1484141456 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef68043070 con 0x55fbcbaf8080 2024-03-21T10:31:59.582 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:59.578+0000 7fef79d84740 1 -- 172.21.15.47:0/1484141456 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fef680333e0 con 0x55fbcbaf8080 2024-03-21T10:31:59.583 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:59.578+0000 7fef5a000640 1 -- 172.21.15.47:0/1484141456 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(97..97 src has 1..97) v4 ==== 9474+0+0 (secure 0 0 0) 0x7fef68033a00 con 0x55fbcbaf8080 2024-03-21T10:31:59.583 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:59.578+0000 7fef5a000640 1 -- 172.21.15.47:0/1484141456 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7fef68033c00 con 0x55fbcbaf8080 2024-03-21T10:31:59.583 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:59.578+0000 7fef79d84740 10 client.17694 fetch_fsmap learned FSMap version 32 2024-03-21T10:31:59.583 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:59.579+0000 7fef79d84740 1 -- 172.21.15.47:0/1484141456 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x55fbcbb6de90 con 0x55fbcbaf8080 2024-03-21T10:31:59.583 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:59.579+0000 7fef5a000640 1 -- 172.21.15.47:0/1484141456 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fef680333e0 con 0x55fbcbaf8080 2024-03-21T10:31:59.583 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:59.579+0000 7fef5a000640 1 -- 172.21.15.47:0/1484141456 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 32) v1 ==== 7942+0+0 (secure 0 0 0) 0x7fef6805b070 con 0x55fbcbaf8080 2024-03-21T10:31:59.583 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:59.579+0000 7fef79d84740 10 client.17694 fetch_fsmap finished waiting for FSMap version 32 2024-03-21T10:31:59.584 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:59.579+0000 7fef79d84740 10 client.17694 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-21T10:31:59.584 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:59.579+0000 7fef79d84740 20 client.17694 populate_metadata read hostname 'smithi047' 2024-03-21T10:31:59.584 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:59.579+0000 7fef79d84740 1 --2- 172.21.15.47:0/1484141456 >> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] conn(0x55fbcbb6e0b0 0x55fbcbb704a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:31:59.584 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:59.579+0000 7fef79d84740 4 client.17694 mds_command: new command op to 34535 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-03-21T10:31:59.584 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:59.579+0000 7fef79d84740 1 -- 172.21.15.47:0/1484141456 --> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x55fbcba64740 con 0x55fbcbb6e0b0 2024-03-21T10:31:59.584 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:59.580+0000 7fef6f400640 1 --2- 172.21.15.47:0/1484141456 >> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] conn(0x55fbcbb6e0b0 0x55fbcbb704a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:31:59.585 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:59.580+0000 7fef6f400640 1 --2- 172.21.15.47:0/1484141456 >> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] conn(0x55fbcbb6e0b0 0x55fbcbb704a0 crc :-1 s=READY pgs=166 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=debacfa37a4424b7 server_cookie=68ebd4d182146ced in_seq=0 out_seq=0 2024-03-21T10:31:59.585 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:59.580+0000 7fef5a000640 10 client.17694 ms_handle_connect on v2:172.21.15.100:6838/4173168440 2024-03-21T10:31:59.587 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:59.583+0000 7fef5a000640 1 -- 172.21.15.47:0/1484141456 <== mds.0 v2:172.21.15.100:6838/4173168440 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x55fbcba64740 con 0x55fbcbb6e0b0 2024-03-21T10:31:59.587 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:59.583+0000 7fef5a000640 10 client.17694 handle_command_reply: tid=0 2024-03-21T10:31:59.587 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:59.583+0000 7fef79d84740 1 client.17694 shutdown 2024-03-21T10:31:59.587 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:59.583+0000 7fef79d84740 1 -- 172.21.15.47:0/1484141456 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55fbcbaf8080 msgr2=0x55fbcbb066b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:31:59.587 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:59.583+0000 7fef79d84740 1 --2- 172.21.15.47:0/1484141456 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55fbcbaf8080 0x55fbcbb066b0 secure :-1 s=READY pgs=1005 cs=0 l=1 rev1=1 crypto rx=0x7fef68009b30 tx=0x7fef68033820 comp rx=0 tx=0).stop 2024-03-21T10:31:59.587 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:59.583+0000 7fef79d84740 1 -- 172.21.15.47:0/1484141456 shutdown_connections 2024-03-21T10:31:59.587 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:59.583+0000 7fef79d84740 1 --2- 172.21.15.47:0/1484141456 >> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] conn(0x55fbcbb6e0b0 0x55fbcbb704a0 crc :-1 s=READY pgs=166 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:31:59.587 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:59.583+0000 7fef79d84740 1 --2- 172.21.15.47:0/1484141456 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fbcbb01580 0x55fbcbb07130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:31:59.587 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:59.583+0000 7fef79d84740 1 --2- 172.21.15.47:0/1484141456 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fbcbaf89a0 0x55fbcbb06bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:31:59.587 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:59.583+0000 7fef79d84740 1 --2- 172.21.15.47:0/1484141456 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55fbcbaf8080 0x55fbcbb066b0 unknown :-1 s=CLOSED pgs=1005 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:31:59.587 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:59.583+0000 7fef79d84740 1 -- 172.21.15.47:0/1484141456 >> 172.21.15.47:0/1484141456 conn(0x55fbcbaf27b0 msgr2=0x55fbcbaf1830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:31:59.587 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:59.583+0000 7fef5a000640 0 client.17694 ms_handle_reset on v2:172.21.15.100:6838/4173168440 2024-03-21T10:31:59.588 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:59.583+0000 7fef79d84740 1 -- 172.21.15.47:0/1484141456 shutdown_connections 2024-03-21T10:31:59.588 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:59.583+0000 7fef79d84740 1 -- 172.21.15.47:0/1484141456 wait complete. 2024-03-21T10:31:59.588 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:59.583+0000 7fef79d84740 20 client.17694 trim_cache size 0 max 16384 2024-03-21T10:31:59.588 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-03-21T10:31:59.588 INFO:teuthology.orchestra.run.smithi047.stdout: "return_code": 0, 2024-03-21T10:31:59.588 INFO:teuthology.orchestra.run.smithi047.stdout: "scrub_tag": "8d39ddcd-0c12-447e-8220-733bbd4bce1e", 2024-03-21T10:31:59.588 INFO:teuthology.orchestra.run.smithi047.stdout: "mode": "asynchronous" 2024-03-21T10:31:59.588 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-03-21T10:31:59.589 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:59.584+0000 7fef77000640 1 -- 172.21.15.47:0/2312198352 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7fef540a3090 msgr2=0x7fef540a5550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:31:59.589 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:59.584+0000 7fef77000640 1 --2- 172.21.15.47:0/2312198352 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7fef540a3090 0x7fef540a5550 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x55fbcbaf7b50 tx=0x7fef60003e10 comp rx=0 tx=0).stop 2024-03-21T10:31:59.589 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:59.584+0000 7fef77000640 1 -- 172.21.15.47:0/2312198352 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fef70069a40 msgr2=0x7fef7019a040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:31:59.589 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:59.584+0000 7fef77000640 1 --2- 172.21.15.47:0/2312198352 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fef70069a40 0x7fef7019a040 secure :-1 s=READY pgs=1002 cs=0 l=1 rev1=1 crypto rx=0x7fef6000e9c0 tx=0x7fef6000ee80 comp rx=0 tx=0).stop 2024-03-21T10:31:59.589 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:59.584+0000 7fef77000640 1 -- 172.21.15.47:0/2312198352 shutdown_connections 2024-03-21T10:31:59.590 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:59.584+0000 7fef77000640 1 --2- 172.21.15.47:0/2312198352 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7fef540a3090 0x7fef540a5550 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:31:59.590 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:59.584+0000 7fef77000640 1 --2- 172.21.15.47:0/2312198352 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fef70107690 0x7fef7019e900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:31:59.590 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:59.584+0000 7fef77000640 1 --2- 172.21.15.47:0/2312198352 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fef7006a3e0 0x7fef7019a580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:31:59.590 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:59.584+0000 7fef77000640 1 --2- 172.21.15.47:0/2312198352 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fef70069a40 0x7fef7019a040 unknown :-1 s=CLOSED pgs=1002 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:31:59.590 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:59.584+0000 7fef77000640 1 -- 172.21.15.47:0/2312198352 >> 172.21.15.47:0/2312198352 conn(0x7fef70077630 msgr2=0x7fef7010c830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:31:59.590 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:59.584+0000 7fef77000640 1 -- 172.21.15.47:0/2312198352 shutdown_connections 2024-03-21T10:31:59.590 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:59.585+0000 7fef77000640 1 -- 172.21.15.47:0/2312198352 wait complete. 2024-03-21T10:31:59.605 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-21T10:31:59.606 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-21T10:31:59.670 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:31:59 smithi047 ceph-mon[31547]: pgmap v203: 129 pgs: 129 active+clean; 277 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-03-21T10:31:59.811 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:59.811+0000 7f4a40c00640 1 -- 172.21.15.47:0/2007751542 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4a3c107640 msgr2=0x7f4a3c107a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:31:59.811 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:59.811+0000 7f4a40c00640 1 --2- 172.21.15.47:0/2007751542 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4a3c107640 0x7f4a3c107a20 secure :-1 s=READY pgs=1006 cs=0 l=1 rev1=1 crypto rx=0x7f4a2c009930 tx=0x7f4a2c02bc20 comp rx=0 tx=0).stop 2024-03-21T10:31:59.811 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:59.812+0000 7f4a40c00640 1 -- 172.21.15.47:0/2007751542 shutdown_connections 2024-03-21T10:31:59.811 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:59.812+0000 7f4a40c00640 1 --2- 172.21.15.47:0/2007751542 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4a3c101ff0 0x7f4a3c10e500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:31:59.811 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:59.812+0000 7f4a40c00640 1 --2- 172.21.15.47:0/2007751542 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4a3c101630 0x7f4a3c101ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:31:59.811 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:59.812+0000 7f4a40c00640 1 --2- 172.21.15.47:0/2007751542 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4a3c107640 0x7f4a3c107a20 unknown :-1 s=CLOSED pgs=1006 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:31:59.811 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:59.812+0000 7f4a40c00640 1 -- 172.21.15.47:0/2007751542 >> 172.21.15.47:0/2007751542 conn(0x7f4a3c078aa0 msgr2=0x7f4a3c078eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:31:59.811 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:59.812+0000 7f4a40c00640 1 -- 172.21.15.47:0/2007751542 shutdown_connections 2024-03-21T10:31:59.811 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:59.812+0000 7f4a40c00640 1 -- 172.21.15.47:0/2007751542 wait complete. 2024-03-21T10:31:59.812 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:59.813+0000 7f4a40c00640 1 Processor -- start 2024-03-21T10:31:59.812 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:59.813+0000 7f4a40c00640 1 -- start start 2024-03-21T10:31:59.812 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:59.814+0000 7f4a40c00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4a3c101630 0x7f4a3c19a070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:31:59.812 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:59.814+0000 7f4a40c00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4a3c101ff0 0x7f4a3c19a5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:31:59.812 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:59.814+0000 7f4a40c00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4a3c107640 0x7f4a3c19e930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:31:59.812 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:59.814+0000 7f4a40c00640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a3c111910 con 0x7f4a3c101ff0 2024-03-21T10:31:59.813 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:59.814+0000 7f4a40c00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a3c111790 con 0x7f4a3c101630 2024-03-21T10:31:59.813 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:59.814+0000 7f4a40c00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a3c111a90 con 0x7f4a3c107640 2024-03-21T10:31:59.814 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:59.814+0000 7f4a39400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4a3c101ff0 0x7f4a3c19a5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:31:59.814 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:59.814+0000 7f4a39400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4a3c101ff0 0x7f4a3c19a5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:58018/0 (socket says 172.21.15.47:58018) 2024-03-21T10:31:59.814 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:59.814+0000 7f4a39400640 1 -- 172.21.15.47:0/3250579041 learned_addr learned my addr 172.21.15.47:0/3250579041 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:31:59.815 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:59.814+0000 7f4a39e00640 1 --2- 172.21.15.47:0/3250579041 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4a3c101630 0x7f4a3c19a070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:31:59.815 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:59.814+0000 7f4a3a800640 1 --2- 172.21.15.47:0/3250579041 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4a3c107640 0x7f4a3c19e930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:31:59.815 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:59.815+0000 7f4a39400640 1 -- 172.21.15.47:0/3250579041 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4a3c107640 msgr2=0x7f4a3c19e930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:31:59.815 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:59.815+0000 7f4a39400640 1 --2- 172.21.15.47:0/3250579041 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4a3c107640 0x7f4a3c19e930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:31:59.815 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:59.815+0000 7f4a39400640 1 -- 172.21.15.47:0/3250579041 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4a3c101630 msgr2=0x7f4a3c19a070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:31:59.815 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:59.815+0000 7f4a39400640 1 --2- 172.21.15.47:0/3250579041 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4a3c101630 0x7f4a3c19a070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:31:59.815 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:59.815+0000 7f4a39400640 1 -- 172.21.15.47:0/3250579041 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a3c19f060 con 0x7f4a3c101ff0 2024-03-21T10:31:59.815 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:59.815+0000 7f4a3a800640 1 --2- 172.21.15.47:0/3250579041 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4a3c107640 0x7f4a3c19e930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T10:31:59.815 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:59.815+0000 7f4a39400640 1 --2- 172.21.15.47:0/3250579041 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4a3c101ff0 0x7f4a3c19a5b0 secure :-1 s=READY pgs=1007 cs=0 l=1 rev1=1 crypto rx=0x7f4a2800d950 tx=0x7f4a2800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:31:59.815 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:59.815+0000 7f4a32a00640 1 -- 172.21.15.47:0/3250579041 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a28017070 con 0x7f4a3c101ff0 2024-03-21T10:31:59.815 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:59.815+0000 7f4a32a00640 1 -- 172.21.15.47:0/3250579041 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f4a28009c30 con 0x7f4a3c101ff0 2024-03-21T10:31:59.815 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:59.815+0000 7f4a32a00640 1 -- 172.21.15.47:0/3250579041 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a2800f620 con 0x7f4a3c101ff0 2024-03-21T10:31:59.815 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:59.815+0000 7f4a40c00640 1 -- 172.21.15.47:0/3250579041 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4a3c19f350 con 0x7f4a3c101ff0 2024-03-21T10:31:59.815 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:59.816+0000 7f4a40c00640 1 -- 172.21.15.47:0/3250579041 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4a3c1a6be0 con 0x7f4a3c101ff0 2024-03-21T10:31:59.819 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:59.819+0000 7f4a32a00640 1 -- 172.21.15.47:0/3250579041 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 27) v1 ==== 137985+0+0 (secure 0 0 0) 0x7f4a280109a0 con 0x7f4a3c101ff0 2024-03-21T10:31:59.820 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:59.819+0000 7f4a43861740 1 Processor -- start 2024-03-21T10:31:59.820 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:59.819+0000 7f4a43861740 1 -- start start 2024-03-21T10:31:59.820 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:59.820+0000 7f4a43861740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fc355a4c00 0x55fc355a3950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:31:59.820 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:59.820+0000 7f4a43861740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55fc355a3f20 0x55fc355a43a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:31:59.820 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:59.820+0000 7f4a43861740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fc355ad290 0x55fc355af680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:31:59.820 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:59.820+0000 7f4a43861740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fc35508250 con 0x55fc355a3f20 2024-03-21T10:31:59.820 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:59.820+0000 7f4a43861740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fc354b0a30 con 0x55fc355a4c00 2024-03-21T10:31:59.820 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:59.820+0000 7f4a43861740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fc355042f0 con 0x55fc355ad290 2024-03-21T10:31:59.820 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:59.820+0000 7f4a39e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55fc355a3f20 0x55fc355a43a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:31:59.820 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:59.820+0000 7f4a3a800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fc355a4c00 0x55fc355a3950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:31:59.820 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:59.820+0000 7f4a3a800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fc355a4c00 0x55fc355a3950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:39292/0 (socket says 172.21.15.47:39292) 2024-03-21T10:31:59.820 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:59.820+0000 7f4a3a800640 1 -- 172.21.15.47:0/257460601 learned_addr learned my addr 172.21.15.47:0/257460601 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:31:59.821 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:59.820+0000 7f4a39e00640 1 --2- 172.21.15.47:0/257460601 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55fc355a3f20 0x55fc355a43a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:58026/0 (socket says 172.21.15.47:58026) 2024-03-21T10:31:59.821 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:59.820+0000 7f4a39400640 1 --2- 172.21.15.47:0/257460601 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fc355ad290 0x55fc355af680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:31:59.821 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:59.820+0000 7f4a39e00640 1 -- 172.21.15.47:0/257460601 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fc355ad290 msgr2=0x55fc355af680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:31:59.821 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:59.820+0000 7f4a39e00640 1 --2- 172.21.15.47:0/257460601 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fc355ad290 0x55fc355af680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:31:59.821 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:59.820+0000 7f4a39e00640 1 -- 172.21.15.47:0/257460601 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fc355a4c00 msgr2=0x55fc355a3950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:31:59.821 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:59.820+0000 7f4a39e00640 1 --2- 172.21.15.47:0/257460601 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fc355a4c00 0x55fc355a3950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:31:59.821 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:59.820+0000 7f4a39e00640 1 -- 172.21.15.47:0/257460601 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55fc3549b690 con 0x55fc355a3f20 2024-03-21T10:31:59.821 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:59.820+0000 7f4a39400640 1 --2- 172.21.15.47:0/257460601 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fc355ad290 0x55fc355af680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T10:31:59.821 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:59.821+0000 7f4a32a00640 1 --2- 172.21.15.47:0/3250579041 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f4a200a3090 0x7f4a200a5550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:31:59.821 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:59.821+0000 7f4a39e00640 1 --2- 172.21.15.47:0/257460601 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55fc355a3f20 0x55fc355a43a0 secure :-1 s=READY pgs=1008 cs=0 l=1 rev1=1 crypto rx=0x7f4a2c036040 tx=0x7f4a2c02c5c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9201d1f322f86a8d server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:31:59.821 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:59.821+0000 7f4a32a00640 1 -- 172.21.15.47:0/3250579041 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 9474+0+0 (secure 0 0 0) 0x7f4a280cf840 con 0x7f4a3c101ff0 2024-03-21T10:31:59.821 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:59.821+0000 7f4a3a800640 1 --2- 172.21.15.47:0/3250579041 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f4a200a3090 0x7f4a200a5550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:31:59.826 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:59.821+0000 7f4a27000640 1 -- 172.21.15.47:0/257460601 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a2c045020 con 0x55fc355a3f20 2024-03-21T10:31:59.826 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:59.821+0000 7f4a27000640 1 -- 172.21.15.47:0/257460601 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f4a2c02c7d0 con 0x55fc355a3f20 2024-03-21T10:31:59.826 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:59.821+0000 7f4a27000640 1 -- 172.21.15.47:0/257460601 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a2c005520 con 0x55fc355a3f20 2024-03-21T10:31:59.826 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:59.822+0000 7f4a43861740 1 -- 172.21.15.47:0/257460601 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55fc355a3f20 msgr2=0x55fc355a43a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:31:59.826 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:59.822+0000 7f4a3a800640 1 --2- 172.21.15.47:0/3250579041 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f4a200a3090 0x7f4a200a5550 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f4a34009fd0 tx=0x7f4a34009500 comp rx=0 tx=0).ready entity=mgr.15687 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:31:59.826 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:59.822+0000 7f4a43861740 1 --2- 172.21.15.47:0/257460601 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55fc355a3f20 0x55fc355a43a0 secure :-1 s=READY pgs=1008 cs=0 l=1 rev1=1 crypto rx=0x7f4a2c036040 tx=0x7f4a2c02c5c0 comp rx=0 tx=0).stop 2024-03-21T10:31:59.826 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:59.822+0000 7f4a43861740 1 -- 172.21.15.47:0/257460601 shutdown_connections 2024-03-21T10:31:59.826 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:59.822+0000 7f4a43861740 1 --2- 172.21.15.47:0/257460601 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fc355ad290 0x55fc355af680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:31:59.826 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:59.822+0000 7f4a43861740 1 --2- 172.21.15.47:0/257460601 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55fc355a3f20 0x55fc355a43a0 unknown :-1 s=CLOSED pgs=1008 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:31:59.826 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:59.822+0000 7f4a43861740 1 --2- 172.21.15.47:0/257460601 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fc355a4c00 0x55fc355a3950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:31:59.826 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:59.822+0000 7f4a43861740 1 -- 172.21.15.47:0/257460601 >> 172.21.15.47:0/257460601 conn(0x55fc3556b200 msgr2=0x55fc3559e970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:31:59.827 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:59.822+0000 7f4a43861740 1 -- 172.21.15.47:0/257460601 shutdown_connections 2024-03-21T10:31:59.827 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:59.822+0000 7f4a43861740 1 -- 172.21.15.47:0/257460601 wait complete. 2024-03-21T10:31:59.827 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:59.823+0000 7f4a43861740 1 -- start start 2024-03-21T10:31:59.827 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:59.823+0000 7f4a43861740 1 Processor -- start 2024-03-21T10:31:59.827 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:59.824+0000 7f4a43861740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fc355a3f20 0x55fc355ac8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:31:59.827 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:59.824+0000 7f4a43861740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55fc355a4c00 0x55fc355a9c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:31:59.827 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:59.824+0000 7f4a43861740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fc355ad290 0x55fc355aa1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:31:59.827 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:59.824+0000 7f4a43861740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fc355b00a0 con 0x55fc355a4c00 2024-03-21T10:31:59.827 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:59.824+0000 7f4a43861740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fc354c3e80 con 0x55fc355ad290 2024-03-21T10:31:59.827 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:59.824+0000 7f4a43861740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fc355b0220 con 0x55fc355a3f20 2024-03-21T10:31:59.827 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:59.824+0000 7f4a43861740 10 client.0 _ll_register_callbacks cb 0x55fc354d6130 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T10:31:59.827 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:59.824+0000 7f4a39400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55fc355a4c00 0x55fc355a9c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:31:59.827 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:59.824+0000 7f4a39400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55fc355a4c00 0x55fc355a9c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:58036/0 (socket says 172.21.15.47:58036) 2024-03-21T10:31:59.827 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:59.824+0000 7f4a39400640 1 -- 172.21.15.47:0/1225534546 learned_addr learned my addr 172.21.15.47:0/1225534546 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:31:59.828 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:59.824+0000 7f4a3a800640 1 --2- 172.21.15.47:0/1225534546 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fc355ad290 0x55fc355aa1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:31:59.828 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:59.824+0000 7f4a39e00640 1 --2- 172.21.15.47:0/1225534546 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fc355a3f20 0x55fc355ac8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:31:59.828 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:59.824+0000 7f4a39400640 1 -- 172.21.15.47:0/1225534546 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fc355a3f20 msgr2=0x55fc355ac8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:31:59.828 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:59.824+0000 7f4a39400640 1 --2- 172.21.15.47:0/1225534546 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fc355a3f20 0x55fc355ac8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:31:59.828 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:59.824+0000 7f4a39400640 1 -- 172.21.15.47:0/1225534546 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fc355ad290 msgr2=0x55fc355aa1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:31:59.828 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:59.824+0000 7f4a39400640 1 --2- 172.21.15.47:0/1225534546 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fc355ad290 0x55fc355aa1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:31:59.828 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:59.825+0000 7f4a39400640 1 -- 172.21.15.47:0/1225534546 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55fc352e67d0 con 0x55fc355a4c00 2024-03-21T10:31:59.828 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:59.825+0000 7f4a39400640 1 --2- 172.21.15.47:0/1225534546 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55fc355a4c00 0x55fc355a9c70 secure :-1 s=READY pgs=1009 cs=0 l=1 rev1=1 crypto rx=0x7f4a2809b510 tx=0x7f4a28009de0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3f7b6b0a817414f3 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:31:59.828 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:59.825+0000 7f4a1b400640 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-03-21T10:31:59.828 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:59.825+0000 7f4a1b400640 1 -- 172.21.15.47:0/1225534546 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a28020050 con 0x55fc355a4c00 2024-03-21T10:31:59.828 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:59.825+0000 7f4a1b400640 1 -- 172.21.15.47:0/1225534546 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(97..97 src has 1..97) v4 ==== 9474+0+0 (secure 0 0 0) 0x7f4a280ccdf0 con 0x55fc355a4c00 2024-03-21T10:31:59.828 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:59.825+0000 7f4a43861740 1 -- 172.21.15.47:0/1225534546 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f4a2c02c7d0 con 0x55fc355a4c00 2024-03-21T10:31:59.828 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:59.825+0000 7f4a1b400640 1 -- 172.21.15.47:0/1225534546 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4a280ccff0 con 0x55fc355a4c00 2024-03-21T10:31:59.829 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:59.825+0000 7f4a43861740 10 client.17706 fetch_fsmap learned FSMap version 32 2024-03-21T10:31:59.829 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:59.825+0000 7f4a43861740 1 -- 172.21.15.47:0/1225534546 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x55fc3521c3f0 con 0x55fc355a4c00 2024-03-21T10:31:59.829 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:59.826+0000 7f4a1b400640 1 -- 172.21.15.47:0/1225534546 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f4a2c02c7d0 con 0x55fc355a4c00 2024-03-21T10:31:59.829 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:59.826+0000 7f4a1b400640 1 -- 172.21.15.47:0/1225534546 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 32) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f4a280ba110 con 0x55fc355a4c00 2024-03-21T10:31:59.829 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:59.826+0000 7f4a43861740 10 client.17706 fetch_fsmap finished waiting for FSMap version 32 2024-03-21T10:31:59.829 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:59.826+0000 7f4a43861740 10 client.17706 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-21T10:31:59.829 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:59.826+0000 7f4a43861740 20 client.17706 populate_metadata read hostname 'smithi047' 2024-03-21T10:31:59.829 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:59.826+0000 7f4a43861740 1 --2- 172.21.15.47:0/1225534546 >> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] conn(0x55fc355bd040 0x55fc355bf430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:31:59.829 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:59.826+0000 7f4a43861740 4 client.17706 mds_command: new command op to 34535 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T10:31:59.829 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:59.826+0000 7f4a43861740 1 -- 172.21.15.47:0/1225534546 --> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55fc35515790 con 0x55fc355bd040 2024-03-21T10:31:59.829 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:59.827+0000 7f4a39e00640 1 --2- 172.21.15.47:0/1225534546 >> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] conn(0x55fc355bd040 0x55fc355bf430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:31:59.829 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:59.827+0000 7f4a1b400640 10 client.17706 ms_handle_connect on v2:172.21.15.100:6838/4173168440 2024-03-21T10:31:59.829 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:59.827+0000 7f4a39e00640 1 --2- 172.21.15.47:0/1225534546 >> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] conn(0x55fc355bd040 0x55fc355bf430 crc :-1 s=READY pgs=167 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b224f92aea33cb5a server_cookie=d5b94626167724b3 in_seq=0 out_seq=0 2024-03-21T10:31:59.830 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:59.828+0000 7f4a1b400640 1 -- 172.21.15.47:0/1225534546 <== mds.0 v2:172.21.15.100:6838/4173168440 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55fc35515790 con 0x55fc355bd040 2024-03-21T10:31:59.831 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:59.828+0000 7f4a1b400640 10 client.17706 handle_command_reply: tid=0 2024-03-21T10:31:59.831 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:59.829+0000 7f4a43861740 1 client.17706 shutdown 2024-03-21T10:31:59.831 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:59.829+0000 7f4a43861740 1 -- 172.21.15.47:0/1225534546 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55fc355a4c00 msgr2=0x55fc355a9c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:31:59.831 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:59.829+0000 7f4a43861740 1 --2- 172.21.15.47:0/1225534546 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55fc355a4c00 0x55fc355a9c70 secure :-1 s=READY pgs=1009 cs=0 l=1 rev1=1 crypto rx=0x7f4a2809b510 tx=0x7f4a28009de0 comp rx=0 tx=0).stop 2024-03-21T10:31:59.831 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:59.829+0000 7f4a43861740 1 -- 172.21.15.47:0/1225534546 shutdown_connections 2024-03-21T10:31:59.831 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:59.829+0000 7f4a43861740 1 --2- 172.21.15.47:0/1225534546 >> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] conn(0x55fc355bd040 0x55fc355bf430 crc :-1 s=READY pgs=167 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:31:59.831 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:59.829+0000 7f4a43861740 1 --2- 172.21.15.47:0/1225534546 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fc355ad290 0x55fc355aa1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:31:59.831 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:59.829+0000 7f4a43861740 1 --2- 172.21.15.47:0/1225534546 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55fc355a4c00 0x55fc355a9c70 unknown :-1 s=CLOSED pgs=1009 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:31:59.831 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:59.829+0000 7f4a43861740 1 --2- 172.21.15.47:0/1225534546 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fc355a3f20 0x55fc355ac8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:31:59.831 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:59.829+0000 7f4a1b400640 0 client.17706 ms_handle_reset on v2:172.21.15.100:6838/4173168440 2024-03-21T10:31:59.831 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:59.829+0000 7f4a43861740 1 -- 172.21.15.47:0/1225534546 >> 172.21.15.47:0/1225534546 conn(0x55fc3556b200 msgr2=0x55fc355a06c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:31:59.831 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:59.829+0000 7f4a43861740 1 -- 172.21.15.47:0/1225534546 shutdown_connections 2024-03-21T10:31:59.831 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:59.829+0000 7f4a43861740 1 -- 172.21.15.47:0/1225534546 wait complete. 2024-03-21T10:31:59.832 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:59.829+0000 7f4a43861740 20 client.17706 trim_cache size 0 max 16384 2024-03-21T10:31:59.852 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:59.847+0000 7f4a43861740 1 Processor -- start 2024-03-21T10:31:59.853 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:59.847+0000 7f4a43861740 1 -- start start 2024-03-21T10:31:59.853 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:59.847+0000 7f4a43861740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fc355a3f20 0x55fc355ac3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:31:59.853 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:59.847+0000 7f4a43861740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55fc355a4c00 0x55fc355ac8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:31:59.853 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:59.847+0000 7f4a43861740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fc355ad290 0x55fc355ae680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:31:59.853 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:59.847+0000 7f4a43861740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a3c1b0a20 con 0x55fc355a4c00 2024-03-21T10:31:59.853 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:59.847+0000 7f4a43861740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a3c1b23a0 con 0x55fc355a3f20 2024-03-21T10:31:59.853 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:59.847+0000 7f4a43861740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a3c1b1080 con 0x55fc355ad290 2024-03-21T10:31:59.853 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:59.848+0000 7f4a39400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55fc355a4c00 0x55fc355ac8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:31:59.853 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:59.848+0000 7f4a39400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55fc355a4c00 0x55fc355ac8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:58042/0 (socket says 172.21.15.47:58042) 2024-03-21T10:31:59.853 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:59.848+0000 7f4a39400640 1 -- 172.21.15.47:0/1580265472 learned_addr learned my addr 172.21.15.47:0/1580265472 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:31:59.853 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:59.848+0000 7f4a39e00640 1 --2- 172.21.15.47:0/1580265472 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fc355a3f20 0x55fc355ac3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:31:59.853 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:59.848+0000 7f4a3a800640 1 --2- 172.21.15.47:0/1580265472 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fc355ad290 0x55fc355ae680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:31:59.854 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:59.848+0000 7f4a39400640 1 -- 172.21.15.47:0/1580265472 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fc355ad290 msgr2=0x55fc355ae680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:31:59.854 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:59.848+0000 7f4a39400640 1 --2- 172.21.15.47:0/1580265472 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fc355ad290 0x55fc355ae680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:31:59.854 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:59.848+0000 7f4a39400640 1 -- 172.21.15.47:0/1580265472 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fc355a3f20 msgr2=0x55fc355ac3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:31:59.854 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:59.848+0000 7f4a39400640 1 --2- 172.21.15.47:0/1580265472 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fc355a3f20 0x55fc355ac3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:31:59.854 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:59.848+0000 7f4a39400640 1 -- 172.21.15.47:0/1580265472 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55fc355bf7a0 con 0x55fc355a4c00 2024-03-21T10:31:59.854 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:59.848+0000 7f4a3a800640 1 --2- 172.21.15.47:0/1580265472 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fc355ad290 0x55fc355ae680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T10:31:59.854 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:59.848+0000 7f4a39400640 1 --2- 172.21.15.47:0/1580265472 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55fc355a4c00 0x55fc355ac8e0 secure :-1 s=READY pgs=1010 cs=0 l=1 rev1=1 crypto rx=0x7f4a280cfb30 tx=0x7f4a280862f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9c99a0926195ce06 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:31:59.854 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:59.849+0000 7f4a26600640 1 -- 172.21.15.47:0/1580265472 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a280afe60 con 0x55fc355a4c00 2024-03-21T10:31:59.854 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:59.849+0000 7f4a26600640 1 -- 172.21.15.47:0/1580265472 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f4a280ba290 con 0x55fc355a4c00 2024-03-21T10:31:59.854 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:59.849+0000 7f4a26600640 1 -- 172.21.15.47:0/1580265472 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a2809cb40 con 0x55fc355a4c00 2024-03-21T10:31:59.854 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:59.849+0000 7f4a43861740 1 -- 172.21.15.47:0/1580265472 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55fc355a4c00 msgr2=0x55fc355ac8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:31:59.854 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:59.849+0000 7f4a43861740 1 --2- 172.21.15.47:0/1580265472 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55fc355a4c00 0x55fc355ac8e0 secure :-1 s=READY pgs=1010 cs=0 l=1 rev1=1 crypto rx=0x7f4a280cfb30 tx=0x7f4a280862f0 comp rx=0 tx=0).stop 2024-03-21T10:31:59.854 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:59.849+0000 7f4a43861740 1 -- 172.21.15.47:0/1580265472 shutdown_connections 2024-03-21T10:31:59.855 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:59.849+0000 7f4a43861740 1 --2- 172.21.15.47:0/1580265472 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fc355ad290 0x55fc355ae680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:31:59.855 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:59.849+0000 7f4a43861740 1 --2- 172.21.15.47:0/1580265472 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55fc355a4c00 0x55fc355ac8e0 unknown :-1 s=CLOSED pgs=1010 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:31:59.855 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:59.849+0000 7f4a43861740 1 --2- 172.21.15.47:0/1580265472 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fc355a3f20 0x55fc355ac3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:31:59.855 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:59.849+0000 7f4a43861740 1 -- 172.21.15.47:0/1580265472 >> 172.21.15.47:0/1580265472 conn(0x55fc3556b200 msgr2=0x55fc355a8540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:31:59.855 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:59.849+0000 7f4a43861740 1 -- 172.21.15.47:0/1580265472 shutdown_connections 2024-03-21T10:31:59.855 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:59.849+0000 7f4a43861740 1 -- 172.21.15.47:0/1580265472 wait complete. 2024-03-21T10:31:59.855 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:59.850+0000 7f4a43861740 1 -- start start 2024-03-21T10:31:59.855 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:59.850+0000 7f4a43861740 1 Processor -- start 2024-03-21T10:31:59.855 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:59.851+0000 7f4a43861740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55fc355a3f20 0x55fc355b0850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:31:59.855 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:59.851+0000 7f4a43861740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fc355a4c00 0x55fc355b0d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:31:59.855 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:59.851+0000 7f4a43861740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fc355ad290 0x55fc355b12d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:31:59.855 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:59.851+0000 7f4a43861740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a3c1b1d40 con 0x55fc355a3f20 2024-03-21T10:31:59.855 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:59.851+0000 7f4a43861740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a3c1b16e0 con 0x55fc355a4c00 2024-03-21T10:31:59.855 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:59.851+0000 7f4a43861740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a3c1aebe0 con 0x55fc355ad290 2024-03-21T10:31:59.856 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:59.851+0000 7f4a43861740 10 client.0 _ll_register_callbacks cb 0x7f4a280973f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T10:31:59.856 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:59.851+0000 7f4a39e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55fc355a3f20 0x55fc355b0850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:31:59.856 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:59.851+0000 7f4a39e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55fc355a3f20 0x55fc355b0850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:58050/0 (socket says 172.21.15.47:58050) 2024-03-21T10:31:59.856 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:59.851+0000 7f4a39e00640 1 -- 172.21.15.47:0/4088313001 learned_addr learned my addr 172.21.15.47:0/4088313001 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:31:59.856 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:59.851+0000 7f4a39400640 1 --2- 172.21.15.47:0/4088313001 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fc355a4c00 0x55fc355b0d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:31:59.856 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:59.852+0000 7f4a39e00640 1 -- 172.21.15.47:0/4088313001 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fc355ad290 msgr2=0x55fc355b12d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:31:59.856 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:59.852+0000 7f4a3a800640 1 --2- 172.21.15.47:0/4088313001 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fc355ad290 0x55fc355b12d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:31:59.856 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:59.852+0000 7f4a39e00640 1 --2- 172.21.15.47:0/4088313001 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fc355ad290 0x55fc355b12d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:31:59.856 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:59.852+0000 7f4a39e00640 1 -- 172.21.15.47:0/4088313001 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fc355a4c00 msgr2=0x55fc355b0d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:31:59.856 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:59.852+0000 7f4a39e00640 1 --2- 172.21.15.47:0/4088313001 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fc355a4c00 0x55fc355b0d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:31:59.856 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:59.852+0000 7f4a39e00640 1 -- 172.21.15.47:0/4088313001 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55fc351dc0f0 con 0x55fc355a3f20 2024-03-21T10:31:59.856 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:59.852+0000 7f4a39400640 1 --2- 172.21.15.47:0/4088313001 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fc355a4c00 0x55fc355b0d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T10:31:59.856 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:59.852+0000 7f4a3a800640 1 --2- 172.21.15.47:0/4088313001 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fc355ad290 0x55fc355b12d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T10:31:59.857 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:59.852+0000 7f4a39e00640 1 --2- 172.21.15.47:0/4088313001 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55fc355a3f20 0x55fc355b0850 secure :-1 s=READY pgs=1011 cs=0 l=1 rev1=1 crypto rx=0x7f4a2c009930 tx=0x7f4a2c02c730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=27437605dee599a9 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:31:59.857 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:59.852+0000 7f4a1b400640 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-03-21T10:31:59.857 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:59.852+0000 7f4a1b400640 1 -- 172.21.15.47:0/4088313001 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a2c045040 con 0x55fc355a3f20 2024-03-21T10:31:59.857 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:59.852+0000 7f4a1b400640 1 -- 172.21.15.47:0/4088313001 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(97..97 src has 1..97) v4 ==== 9474+0+0 (secure 0 0 0) 0x7f4a2c004240 con 0x55fc355a3f20 2024-03-21T10:31:59.857 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:59.852+0000 7f4a43861740 1 -- 172.21.15.47:0/4088313001 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f4a280ba290 con 0x55fc355a3f20 2024-03-21T10:31:59.857 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:59.853+0000 7f4a1b400640 1 -- 172.21.15.47:0/4088313001 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4a2c004440 con 0x55fc355a3f20 2024-03-21T10:31:59.857 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:59.853+0000 7f4a43861740 10 client.17712 fetch_fsmap learned FSMap version 32 2024-03-21T10:31:59.858 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:59.853+0000 7f4a43861740 1 -- 172.21.15.47:0/4088313001 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x55fc35618890 con 0x55fc355a3f20 2024-03-21T10:31:59.858 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:59.853+0000 7f4a1b400640 1 -- 172.21.15.47:0/4088313001 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f4a280ba290 con 0x55fc355a3f20 2024-03-21T10:31:59.858 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:59.853+0000 7f4a1b400640 1 -- 172.21.15.47:0/4088313001 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 32) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f4a2c004a30 con 0x55fc355a3f20 2024-03-21T10:31:59.858 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:59.853+0000 7f4a43861740 10 client.17712 fetch_fsmap finished waiting for FSMap version 32 2024-03-21T10:31:59.858 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:59.853+0000 7f4a43861740 10 client.17712 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-21T10:31:59.858 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:59.853+0000 7f4a43861740 20 client.17712 populate_metadata read hostname 'smithi047' 2024-03-21T10:31:59.859 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:59.853+0000 7f4a43861740 1 --2- 172.21.15.47:0/4088313001 >> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] conn(0x55fc35618bd0 0x55fc3561afc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:31:59.859 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:59.853+0000 7f4a43861740 4 client.17712 mds_command: new command op to 34535 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-21T10:31:59.859 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:59.853+0000 7f4a43861740 1 -- 172.21.15.47:0/4088313001 --> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55fc3550fae0 con 0x55fc35618bd0 2024-03-21T10:31:59.859 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:59.854+0000 7f4a39400640 1 --2- 172.21.15.47:0/4088313001 >> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] conn(0x55fc35618bd0 0x55fc3561afc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:31:59.862 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:59.854+0000 7f4a39400640 1 --2- 172.21.15.47:0/4088313001 >> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] conn(0x55fc35618bd0 0x55fc3561afc0 crc :-1 s=READY pgs=168 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f7abd6ab8689a042 server_cookie=8f6c028dba89840f in_seq=0 out_seq=0 2024-03-21T10:31:59.862 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:59.854+0000 7f4a1b400640 10 client.17712 ms_handle_connect on v2:172.21.15.100:6838/4173168440 2024-03-21T10:31:59.863 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:59.855+0000 7f4a1b400640 1 -- 172.21.15.47:0/4088313001 <== mds.0 v2:172.21.15.100:6838/4173168440 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x55fc3550fae0 con 0x55fc35618bd0 2024-03-21T10:31:59.863 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:59.855+0000 7f4a1b400640 10 client.17712 handle_command_reply: tid=0 2024-03-21T10:31:59.863 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:59.855+0000 7f4a43861740 1 client.17712 shutdown 2024-03-21T10:31:59.863 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:59.855+0000 7f4a43861740 1 -- 172.21.15.47:0/4088313001 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55fc355a3f20 msgr2=0x55fc355b0850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:31:59.863 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:59.855+0000 7f4a43861740 1 --2- 172.21.15.47:0/4088313001 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55fc355a3f20 0x55fc355b0850 secure :-1 s=READY pgs=1011 cs=0 l=1 rev1=1 crypto rx=0x7f4a2c009930 tx=0x7f4a2c02c730 comp rx=0 tx=0).stop 2024-03-21T10:31:59.863 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:59.855+0000 7f4a43861740 1 -- 172.21.15.47:0/4088313001 shutdown_connections 2024-03-21T10:31:59.863 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:59.855+0000 7f4a43861740 1 --2- 172.21.15.47:0/4088313001 >> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] conn(0x55fc35618bd0 0x55fc3561afc0 crc :-1 s=READY pgs=168 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:31:59.863 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:59.855+0000 7f4a43861740 1 --2- 172.21.15.47:0/4088313001 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fc355ad290 0x55fc355b12d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:31:59.863 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:59.855+0000 7f4a43861740 1 --2- 172.21.15.47:0/4088313001 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fc355a4c00 0x55fc355b0d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:31:59.863 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:59.855+0000 7f4a43861740 1 --2- 172.21.15.47:0/4088313001 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55fc355a3f20 0x55fc355b0850 unknown :-1 s=CLOSED pgs=1011 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:31:59.864 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:59.855+0000 7f4a43861740 1 -- 172.21.15.47:0/4088313001 >> 172.21.15.47:0/4088313001 conn(0x55fc3556b200 msgr2=0x55fc3559da90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:31:59.864 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:59.855+0000 7f4a1b400640 0 client.17712 ms_handle_reset on v2:172.21.15.100:6838/4173168440 2024-03-21T10:31:59.864 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:59.855+0000 7f4a43861740 1 -- 172.21.15.47:0/4088313001 shutdown_connections 2024-03-21T10:31:59.864 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:59.855+0000 7f4a43861740 1 -- 172.21.15.47:0/4088313001 wait complete. 2024-03-21T10:31:59.864 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:59.855+0000 7f4a43861740 20 client.17712 trim_cache size 0 max 16384 2024-03-21T10:31:59.864 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-03-21T10:31:59.865 INFO:teuthology.orchestra.run.smithi047.stdout: "status": "IDLE (waiting for more scrubs)", 2024-03-21T10:31:59.865 INFO:teuthology.orchestra.run.smithi047.stdout: "scrubs": { 2024-03-21T10:31:59.865 INFO:teuthology.orchestra.run.smithi047.stdout: "8d39ddcd-0c12-447e-8220-733bbd4bce1e": { 2024-03-21T10:31:59.865 INFO:teuthology.orchestra.run.smithi047.stdout: "path": "/", 2024-03-21T10:31:59.865 INFO:teuthology.orchestra.run.smithi047.stdout: "tag": "8d39ddcd-0c12-447e-8220-733bbd4bce1e", 2024-03-21T10:31:59.865 INFO:teuthology.orchestra.run.smithi047.stdout: "options": "recursive,force" 2024-03-21T10:31:59.865 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-21T10:31:59.865 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-21T10:31:59.865 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-03-21T10:31:59.866 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:59.856+0000 7f4a40c00640 1 -- 172.21.15.47:0/3250579041 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f4a200a3090 msgr2=0x7f4a200a5550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:31:59.866 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:59.856+0000 7f4a40c00640 1 --2- 172.21.15.47:0/3250579041 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f4a200a3090 0x7f4a200a5550 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f4a34009fd0 tx=0x7f4a34009500 comp rx=0 tx=0).stop 2024-03-21T10:31:59.866 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:59.856+0000 7f4a40c00640 1 -- 172.21.15.47:0/3250579041 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4a3c101ff0 msgr2=0x7f4a3c19a5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:31:59.866 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:59.856+0000 7f4a40c00640 1 --2- 172.21.15.47:0/3250579041 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4a3c101ff0 0x7f4a3c19a5b0 secure :-1 s=READY pgs=1007 cs=0 l=1 rev1=1 crypto rx=0x7f4a2800d950 tx=0x7f4a2800de10 comp rx=0 tx=0).stop 2024-03-21T10:31:59.866 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:59.856+0000 7f4a40c00640 1 -- 172.21.15.47:0/3250579041 shutdown_connections 2024-03-21T10:31:59.866 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:59.856+0000 7f4a40c00640 1 --2- 172.21.15.47:0/3250579041 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4a3c107640 0x7f4a3c19e930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:31:59.866 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:59.856+0000 7f4a40c00640 1 --2- 172.21.15.47:0/3250579041 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4a3c101ff0 0x7f4a3c19a5b0 unknown :-1 s=CLOSED pgs=1007 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:31:59.866 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:59.856+0000 7f4a40c00640 1 --2- 172.21.15.47:0/3250579041 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f4a200a3090 0x7f4a200a5550 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:31:59.867 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:59.856+0000 7f4a40c00640 1 --2- 172.21.15.47:0/3250579041 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4a3c101630 0x7f4a3c19a070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:31:59.867 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:59.856+0000 7f4a40c00640 1 -- 172.21.15.47:0/3250579041 >> 172.21.15.47:0/3250579041 conn(0x7f4a3c078aa0 msgr2=0x7f4a3c10cde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:31:59.867 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:59.856+0000 7f4a40c00640 1 -- 172.21.15.47:0/3250579041 shutdown_connections 2024-03-21T10:31:59.867 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:31:59.856+0000 7f4a40c00640 1 -- 172.21.15.47:0/3250579041 wait complete. 2024-03-21T10:31:59.873 INFO:tasks.cephfs.filesystem:scrub status for tag:8d39ddcd-0c12-447e-8220-733bbd4bce1e - {'path': '/', 'tag': '8d39ddcd-0c12-447e-8220-733bbd4bce1e', 'options': 'recursive,force'} 2024-03-21T10:31:59.977 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:31:59 smithi100 ceph-mon[35193]: pgmap v203: 129 pgs: 129 active+clean; 277 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-03-21T10:31:59.987 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:31:59 smithi157 ceph-mon[35581]: pgmap v203: 129 pgs: 129 active+clean; 277 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-03-21T10:32:00.962 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:32:00 smithi047 ceph-mon[31547]: scrub queued for path: / 2024-03-21T10:32:00.962 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:32:00 smithi047 ceph-mon[31547]: scrub summary: idle+waiting paths [/] 2024-03-21T10:32:00.962 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:32:00 smithi047 ceph-mon[31547]: scrub summary: active paths [/] 2024-03-21T10:32:00.962 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:32:00 smithi047 ceph-mon[31547]: scrub summary: idle+waiting paths [/] 2024-03-21T10:32:00.977 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:32:00 smithi100 ceph-mon[35193]: scrub queued for path: / 2024-03-21T10:32:00.977 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:32:00 smithi100 ceph-mon[35193]: scrub summary: idle+waiting paths [/] 2024-03-21T10:32:00.977 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:32:00 smithi100 ceph-mon[35193]: scrub summary: active paths [/] 2024-03-21T10:32:00.977 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:32:00 smithi100 ceph-mon[35193]: scrub summary: idle+waiting paths [/] 2024-03-21T10:32:00.987 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:32:00 smithi157 ceph-mon[35581]: scrub queued for path: / 2024-03-21T10:32:00.988 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:32:00 smithi157 ceph-mon[35581]: scrub summary: idle+waiting paths [/] 2024-03-21T10:32:00.988 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:32:00 smithi157 ceph-mon[35581]: scrub summary: active paths [/] 2024-03-21T10:32:00.988 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:32:00 smithi157 ceph-mon[35581]: scrub summary: idle+waiting paths [/] 2024-03-21T10:32:01.961 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:32:01 smithi047 ceph-mon[31547]: pgmap v204: 129 pgs: 129 active+clean; 277 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-21T10:32:01.977 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:32:01 smithi100 ceph-mon[35193]: pgmap v204: 129 pgs: 129 active+clean; 277 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-21T10:32:01.987 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:32:01 smithi157 ceph-mon[35581]: pgmap v204: 129 pgs: 129 active+clean; 277 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-21T10:32:03.961 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:32:03 smithi047 ceph-mon[31547]: pgmap v205: 129 pgs: 129 active+clean; 277 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 44 KiB/s wr, 30 op/s 2024-03-21T10:32:03.977 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:32:03 smithi100 ceph-mon[35193]: pgmap v205: 129 pgs: 129 active+clean; 277 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 44 KiB/s wr, 30 op/s 2024-03-21T10:32:03.987 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:32:03 smithi157 ceph-mon[35581]: pgmap v205: 129 pgs: 129 active+clean; 277 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 44 KiB/s wr, 30 op/s 2024-03-21T10:32:04.961 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:32:04 smithi047 ceph-mon[31547]: scrub summary: idle 2024-03-21T10:32:04.977 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:32:04 smithi100 ceph-mon[35193]: scrub summary: idle 2024-03-21T10:32:04.987 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:32:04 smithi157 ceph-mon[35581]: scrub summary: idle 2024-03-21T10:32:05.961 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:32:05 smithi047 ceph-mon[31547]: pgmap v206: 129 pgs: 129 active+clean; 277 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 44 KiB/s wr, 26 op/s 2024-03-21T10:32:05.977 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:32:05 smithi100 ceph-mon[35193]: pgmap v206: 129 pgs: 129 active+clean; 277 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 44 KiB/s wr, 26 op/s 2024-03-21T10:32:05.987 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:32:05 smithi157 ceph-mon[35581]: pgmap v206: 129 pgs: 129 active+clean; 277 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 44 KiB/s wr, 26 op/s 2024-03-21T10:32:07.961 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:32:07 smithi047 ceph-mon[31547]: pgmap v207: 129 pgs: 129 active+clean; 277 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 44 KiB/s wr, 30 op/s 2024-03-21T10:32:07.977 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:32:07 smithi100 ceph-mon[35193]: pgmap v207: 129 pgs: 129 active+clean; 277 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 44 KiB/s wr, 30 op/s 2024-03-21T10:32:07.987 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:32:07 smithi157 ceph-mon[35581]: pgmap v207: 129 pgs: 129 active+clean; 277 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 44 KiB/s wr, 30 op/s 2024-03-21T10:32:09.961 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:32:09 smithi047 ceph-mon[31547]: pgmap v208: 129 pgs: 129 active+clean; 277 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 44 KiB/s wr, 27 op/s 2024-03-21T10:32:09.977 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:32:09 smithi100 ceph-mon[35193]: pgmap v208: 129 pgs: 129 active+clean; 277 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 44 KiB/s wr, 27 op/s 2024-03-21T10:32:09.987 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:32:09 smithi157 ceph-mon[35581]: pgmap v208: 129 pgs: 129 active+clean; 277 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 44 KiB/s wr, 27 op/s 2024-03-21T10:32:10.961 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:32:10 smithi047 ceph-mon[31547]: osdmap e98: 12 total, 12 up, 12 in 2024-03-21T10:32:10.977 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:32:10 smithi100 ceph-mon[35193]: osdmap e98: 12 total, 12 up, 12 in 2024-03-21T10:32:10.987 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:32:10 smithi157 ceph-mon[35581]: osdmap e98: 12 total, 12 up, 12 in 2024-03-21T10:32:11.889 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:32:11 smithi157 ceph-mon[35581]: pgmap v210: 129 pgs: 129 active+clean; 277 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 53 KiB/s wr, 31 op/s 2024-03-21T10:32:11.961 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:32:11 smithi047 ceph-mon[31547]: pgmap v210: 129 pgs: 129 active+clean; 277 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 53 KiB/s wr, 31 op/s 2024-03-21T10:32:11.977 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:32:11 smithi100 ceph-mon[35193]: pgmap v210: 129 pgs: 129 active+clean; 277 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 53 KiB/s wr, 31 op/s 2024-03-21T10:32:13.961 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:32:13 smithi047 ceph-mon[31547]: pgmap v211: 129 pgs: 129 active+clean; 277 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-03-21T10:32:13.977 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:32:13 smithi100 ceph-mon[35193]: pgmap v211: 129 pgs: 129 active+clean; 277 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-03-21T10:32:13.987 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:32:13 smithi157 ceph-mon[35581]: pgmap v211: 129 pgs: 129 active+clean; 277 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-03-21T10:32:15.211 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:32:14 smithi047 ceph-mon[31547]: osdmap e99: 12 total, 12 up, 12 in 2024-03-21T10:32:15.211 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:32:14 smithi047 ceph-mon[31547]: pgmap v213: 129 pgs: 129 active+clean; 277 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 127 B/s wr, 7 op/s 2024-03-21T10:32:15.227 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:32:14 smithi100 ceph-mon[35193]: osdmap e99: 12 total, 12 up, 12 in 2024-03-21T10:32:15.227 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:32:14 smithi100 ceph-mon[35193]: pgmap v213: 129 pgs: 129 active+clean; 277 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 127 B/s wr, 7 op/s 2024-03-21T10:32:15.237 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:32:14 smithi157 ceph-mon[35581]: osdmap e99: 12 total, 12 up, 12 in 2024-03-21T10:32:15.237 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:32:14 smithi157 ceph-mon[35581]: pgmap v213: 129 pgs: 129 active+clean; 277 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 127 B/s wr, 7 op/s 2024-03-21T10:32:17.461 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:32:17 smithi047 ceph-mon[31547]: pgmap v214: 129 pgs: 129 active+clean; 277 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 255 B/s wr, 12 op/s 2024-03-21T10:32:17.477 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:32:17 smithi100 ceph-mon[35193]: pgmap v214: 129 pgs: 129 active+clean; 277 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 255 B/s wr, 12 op/s 2024-03-21T10:32:17.487 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:32:17 smithi157 ceph-mon[35581]: pgmap v214: 129 pgs: 129 active+clean; 277 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 255 B/s wr, 12 op/s 2024-03-21T10:32:19.461 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:32:19 smithi047 ceph-mon[31547]: pgmap v215: 129 pgs: 129 active+clean; 277 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 240 B/s wr, 12 op/s 2024-03-21T10:32:19.477 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:32:19 smithi100 ceph-mon[35193]: pgmap v215: 129 pgs: 129 active+clean; 277 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 240 B/s wr, 12 op/s 2024-03-21T10:32:19.487 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:32:19 smithi157 ceph-mon[35581]: pgmap v215: 129 pgs: 129 active+clean; 277 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 240 B/s wr, 12 op/s 2024-03-21T10:32:21.477 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:32:21 smithi100 ceph-mon[35193]: pgmap v216: 129 pgs: 129 active+clean; 277 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-03-21T10:32:21.487 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:32:21 smithi157 ceph-mon[35581]: pgmap v216: 129 pgs: 129 active+clean; 277 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-03-21T10:32:21.711 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:32:21 smithi047 ceph-mon[31547]: pgmap v216: 129 pgs: 129 active+clean; 277 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-03-21T10:32:23.477 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:32:23 smithi100 ceph-mon[35193]: pgmap v217: 129 pgs: 129 active+clean; 277 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 10 op/s 2024-03-21T10:32:23.487 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:32:23 smithi157 ceph-mon[35581]: pgmap v217: 129 pgs: 129 active+clean; 277 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 10 op/s 2024-03-21T10:32:23.514 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:32:23 smithi047 ceph-mon[31547]: pgmap v217: 129 pgs: 129 active+clean; 277 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 10 op/s 2024-03-21T10:32:25.487 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:32:25 smithi157 ceph-mon[35581]: pgmap v218: 129 pgs: 129 active+clean; 277 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 98 B/s wr, 9 op/s 2024-03-21T10:32:25.711 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:32:25 smithi047 ceph-mon[31547]: pgmap v218: 129 pgs: 129 active+clean; 277 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 98 B/s wr, 9 op/s 2024-03-21T10:32:25.727 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:32:25 smithi100 ceph-mon[35193]: pgmap v218: 129 pgs: 129 active+clean; 277 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 98 B/s wr, 9 op/s 2024-03-21T10:32:27.487 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:32:27 smithi157 ceph-mon[35581]: pgmap v219: 129 pgs: 129 active+clean; 277 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 85 B/s wr, 12 op/s 2024-03-21T10:32:27.711 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:32:27 smithi047 ceph-mon[31547]: pgmap v219: 129 pgs: 129 active+clean; 277 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 85 B/s wr, 12 op/s 2024-03-21T10:32:27.727 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:32:27 smithi100 ceph-mon[35193]: pgmap v219: 129 pgs: 129 active+clean; 277 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 85 B/s wr, 12 op/s 2024-03-21T10:32:29.711 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:32:29 smithi047 ceph-mon[31547]: pgmap v220: 129 pgs: 129 active+clean; 277 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-03-21T10:32:29.727 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:32:29 smithi100 ceph-mon[35193]: pgmap v220: 129 pgs: 129 active+clean; 277 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-03-21T10:32:29.737 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:32:29 smithi157 ceph-mon[35581]: pgmap v220: 129 pgs: 129 active+clean; 277 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-03-21T10:32:29.874 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-21T10:32:29.874 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-21T10:32:30.105 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:30.105+0000 7f9bc2c00640 1 -- 172.21.15.47:0/2805476357 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9bbc103200 msgr2=0x7f9bbc10b6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:32:30.105 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:30.105+0000 7f9bc2c00640 1 --2- 172.21.15.47:0/2805476357 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9bbc103200 0x7f9bbc10b6e0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f9ba4009900 tx=0x7f9ba402bcc0 comp rx=0 tx=0).stop 2024-03-21T10:32:30.105 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:30.105+0000 7f9bc2c00640 1 -- 172.21.15.47:0/2805476357 shutdown_connections 2024-03-21T10:32:30.105 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:30.105+0000 7f9bc2c00640 1 --2- 172.21.15.47:0/2805476357 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9bbc10bcb0 0x7f9bbc10e0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:32:30.105 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:30.105+0000 7f9bc2c00640 1 --2- 172.21.15.47:0/2805476357 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9bbc103200 0x7f9bbc10b6e0 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:32:30.105 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:30.105+0000 7f9bc2c00640 1 --2- 172.21.15.47:0/2805476357 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9bbc1028e0 0x7f9bbc102cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:32:30.105 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:30.105+0000 7f9bc2c00640 1 -- 172.21.15.47:0/2805476357 >> 172.21.15.47:0/2805476357 conn(0x7f9bbc076920 msgr2=0x7f9bbc076d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:32:30.105 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:30.105+0000 7f9bc2c00640 1 -- 172.21.15.47:0/2805476357 shutdown_connections 2024-03-21T10:32:30.105 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:30.105+0000 7f9bc2c00640 1 -- 172.21.15.47:0/2805476357 wait complete. 2024-03-21T10:32:30.106 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:30.106+0000 7f9bc2c00640 1 Processor -- start 2024-03-21T10:32:30.106 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:30.106+0000 7f9bc2c00640 1 -- start start 2024-03-21T10:32:30.107 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:30.107+0000 7f9bc2c00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9bbc1028e0 0x7f9bbc19a110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:32:30.108 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:30.107+0000 7f9bc2c00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9bbc103200 0x7f9bbc19a650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:32:30.108 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:30.107+0000 7f9bc2c00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9bbc10bcb0 0x7f9bbc19e9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:32:30.108 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:30.107+0000 7f9bc2c00640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9bbc1118f0 con 0x7f9bbc10bcb0 2024-03-21T10:32:30.108 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:30.107+0000 7f9bc2c00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9bbc111770 con 0x7f9bbc1028e0 2024-03-21T10:32:30.108 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:30.107+0000 7f9bc2c00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9bbc111a70 con 0x7f9bbc103200 2024-03-21T10:32:30.108 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:30.107+0000 7f9bbbe00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9bbc1028e0 0x7f9bbc19a110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:32:30.108 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:30.107+0000 7f9bbbe00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9bbc1028e0 0x7f9bbc19a110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:33620/0 (socket says 172.21.15.47:33620) 2024-03-21T10:32:30.108 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:30.107+0000 7f9bbbe00640 1 -- 172.21.15.47:0/3823049562 learned_addr learned my addr 172.21.15.47:0/3823049562 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:32:30.108 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:30.107+0000 7f9bc0c00640 1 --2- 172.21.15.47:0/3823049562 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9bbc10bcb0 0x7f9bbc19e9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:32:30.108 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:30.108+0000 7f9bb3e00640 1 --2- 172.21.15.47:0/3823049562 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9bbc103200 0x7f9bbc19a650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:32:30.108 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:30.108+0000 7f9bbbe00640 1 -- 172.21.15.47:0/3823049562 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9bbc103200 msgr2=0x7f9bbc19a650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:32:30.108 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:30.108+0000 7f9bbbe00640 1 --2- 172.21.15.47:0/3823049562 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9bbc103200 0x7f9bbc19a650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:32:30.109 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:30.108+0000 7f9bbbe00640 1 -- 172.21.15.47:0/3823049562 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9bbc10bcb0 msgr2=0x7f9bbc19e9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:32:30.109 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:30.108+0000 7f9bbbe00640 1 --2- 172.21.15.47:0/3823049562 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9bbc10bcb0 0x7f9bbc19e9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:32:30.109 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:30.108+0000 7f9bbbe00640 1 -- 172.21.15.47:0/3823049562 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9bbc19f100 con 0x7f9bbc1028e0 2024-03-21T10:32:30.109 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:30.108+0000 7f9bc0c00640 1 --2- 172.21.15.47:0/3823049562 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9bbc10bcb0 0x7f9bbc19e9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T10:32:30.109 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:30.108+0000 7f9bbbe00640 1 --2- 172.21.15.47:0/3823049562 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9bbc1028e0 0x7f9bbc19a110 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f9bac00e9f0 tx=0x7f9bac00edf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:32:30.110 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:30.109+0000 7f9bb9600640 1 -- 172.21.15.47:0/3823049562 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9bac00ccc0 con 0x7f9bbc1028e0 2024-03-21T10:32:30.110 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:30.109+0000 7f9bc2c00640 1 -- 172.21.15.47:0/3823049562 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9bbc19f3f0 con 0x7f9bbc1028e0 2024-03-21T10:32:30.110 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:30.109+0000 7f9bb9600640 1 -- 172.21.15.47:0/3823049562 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f9bac00ce60 con 0x7f9bbc1028e0 2024-03-21T10:32:30.110 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:30.109+0000 7f9bb9600640 1 -- 172.21.15.47:0/3823049562 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9bac010430 con 0x7f9bbc1028e0 2024-03-21T10:32:30.110 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:30.109+0000 7f9bc2c00640 1 -- 172.21.15.47:0/3823049562 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9bbc1a6c80 con 0x7f9bbc1028e0 2024-03-21T10:32:30.112 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:30.113+0000 7f9bb9600640 1 -- 172.21.15.47:0/3823049562 <== mon.2 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 27) v1 ==== 137985+0+0 (secure 0 0 0) 0x7f9bac005020 con 0x7f9bbc1028e0 2024-03-21T10:32:30.113 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:30.113+0000 7f9bc5985740 1 Processor -- start 2024-03-21T10:32:30.113 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:30.114+0000 7f9bc5985740 1 -- start start 2024-03-21T10:32:30.115 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:30.114+0000 7f9bc5985740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562543281c00 0x562543280950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:32:30.115 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:30.114+0000 7f9bc5985740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562543280f20 0x5625432813a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:32:30.115 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:30.114+0000 7f9bc5985740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56254328a290 0x56254328c680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:32:30.115 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:30.114+0000 7f9bc5985740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x5625431e5250 con 0x562543281c00 2024-03-21T10:32:30.115 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:30.114+0000 7f9bc5985740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x56254318da30 con 0x56254328a290 2024-03-21T10:32:30.115 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:30.114+0000 7f9bc5985740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x5625431e12f0 con 0x562543280f20 2024-03-21T10:32:30.115 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:30.114+0000 7f9bb9600640 1 --2- 172.21.15.47:0/3823049562 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f9ba00a3090 0x7f9ba00a5550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:32:30.115 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:30.115+0000 7f9bb3e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56254328a290 0x56254328c680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:32:30.115 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:30.115+0000 7f9bb3e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56254328a290 0x56254328c680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:33632/0 (socket says 172.21.15.47:33632) 2024-03-21T10:32:30.115 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:30.115+0000 7f9bb3e00640 1 -- 172.21.15.47:0/3927006310 learned_addr learned my addr 172.21.15.47:0/3927006310 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:32:30.115 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:30.115+0000 7f9bc0c00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562543281c00 0x562543280950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:32:30.115 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:30.115+0000 7f9bb9600640 1 -- 172.21.15.47:0/3823049562 <== mon.2 v2:172.21.15.100:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 9562+0+0 (secure 0 0 0) 0x7f9bac0d00c0 con 0x7f9bbc1028e0 2024-03-21T10:32:30.115 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:30.115+0000 7f9bbbe00640 1 --2- 172.21.15.47:0/3927006310 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562543280f20 0x5625432813a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:32:30.116 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:30.115+0000 7f9bc0c00640 1 --2- 172.21.15.47:0/3823049562 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f9ba00a3090 0x7f9ba00a5550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:32:30.116 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:30.115+0000 7f9bb3e00640 1 -- 172.21.15.47:0/3927006310 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562543280f20 msgr2=0x5625432813a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:32:30.116 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:30.115+0000 7f9bb3e00640 1 --2- 172.21.15.47:0/3927006310 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562543280f20 0x5625432813a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:32:30.116 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:30.115+0000 7f9bb3e00640 1 -- 172.21.15.47:0/3927006310 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562543281c00 msgr2=0x562543280950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:32:30.116 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:30.115+0000 7f9bb3e00640 1 --2- 172.21.15.47:0/3927006310 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562543281c00 0x562543280950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:32:30.116 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:30.115+0000 7f9bb3e00640 1 -- 172.21.15.47:0/3927006310 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562543178690 con 0x56254328a290 2024-03-21T10:32:30.119 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:30.116+0000 7f9bb3e00640 1 --2- 172.21.15.47:0/3927006310 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56254328a290 0x56254328c680 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f9ba402c200 tx=0x7f9ba4005dc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2c3af7f975f10b4a server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:32:30.119 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:30.116+0000 7f9bc0c00640 1 --2- 172.21.15.47:0/3823049562 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f9ba00a3090 0x7f9ba00a5550 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f9bbc06ad50 tx=0x7f9bb4009500 comp rx=0 tx=0).ready entity=mgr.15687 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:32:30.119 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:30.116+0000 7f9bb1200640 1 -- 172.21.15.47:0/3927006310 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ba4004240 con 0x56254328a290 2024-03-21T10:32:30.119 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:30.116+0000 7f9bb1200640 1 -- 172.21.15.47:0/3927006310 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f9ba40043e0 con 0x56254328a290 2024-03-21T10:32:30.119 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:30.116+0000 7f9bb1200640 1 -- 172.21.15.47:0/3927006310 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ba40046c0 con 0x56254328a290 2024-03-21T10:32:30.120 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:30.117+0000 7f9bc5985740 1 -- 172.21.15.47:0/3927006310 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56254328a290 msgr2=0x56254328c680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:32:30.120 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:30.117+0000 7f9bc5985740 1 --2- 172.21.15.47:0/3927006310 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56254328a290 0x56254328c680 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f9ba402c200 tx=0x7f9ba4005dc0 comp rx=0 tx=0).stop 2024-03-21T10:32:30.120 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:30.117+0000 7f9bc5985740 1 -- 172.21.15.47:0/3927006310 shutdown_connections 2024-03-21T10:32:30.120 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:30.117+0000 7f9bc5985740 1 --2- 172.21.15.47:0/3927006310 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56254328a290 0x56254328c680 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:32:30.120 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:30.117+0000 7f9bc5985740 1 --2- 172.21.15.47:0/3927006310 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562543280f20 0x5625432813a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:32:30.120 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:30.117+0000 7f9bc5985740 1 --2- 172.21.15.47:0/3927006310 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562543281c00 0x562543280950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:32:30.120 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:30.117+0000 7f9bc5985740 1 -- 172.21.15.47:0/3927006310 >> 172.21.15.47:0/3927006310 conn(0x562543248200 msgr2=0x56254327b970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:32:30.120 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:30.117+0000 7f9bc5985740 1 -- 172.21.15.47:0/3927006310 shutdown_connections 2024-03-21T10:32:30.120 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:30.117+0000 7f9bc5985740 1 -- 172.21.15.47:0/3927006310 wait complete. 2024-03-21T10:32:30.120 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:30.117+0000 7f9bc5985740 1 -- start start 2024-03-21T10:32:30.120 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:30.118+0000 7f9bc5985740 1 Processor -- start 2024-03-21T10:32:30.120 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:30.118+0000 7f9bc5985740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562543280f20 0x5625432898f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:32:30.120 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:30.118+0000 7f9bc5985740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562543281c00 0x562543286c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:32:30.120 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:30.118+0000 7f9bc5985740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56254328a290 0x5625432871c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:32:30.120 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:30.118+0000 7f9bc5985740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x56254328d0a0 con 0x562543281c00 2024-03-21T10:32:30.121 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:30.118+0000 7f9bc5985740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x5625431a0e80 con 0x562543280f20 2024-03-21T10:32:30.121 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:30.118+0000 7f9bc5985740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x56254328d220 con 0x56254328a290 2024-03-21T10:32:30.121 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:30.118+0000 7f9bc5985740 10 client.0 _ll_register_callbacks cb 0x5625431b3130 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T10:32:30.121 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:30.119+0000 7f9bb3e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562543281c00 0x562543286c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:32:30.121 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:30.119+0000 7f9bb3e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562543281c00 0x562543286c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:51234/0 (socket says 172.21.15.47:51234) 2024-03-21T10:32:30.121 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:30.119+0000 7f9bb3e00640 1 -- 172.21.15.47:0/1431896470 learned_addr learned my addr 172.21.15.47:0/1431896470 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:32:30.121 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:30.119+0000 7f9bbbe00640 1 --2- 172.21.15.47:0/1431896470 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562543280f20 0x5625432898f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:32:30.121 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:30.119+0000 7f9bb3e00640 1 -- 172.21.15.47:0/1431896470 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56254328a290 msgr2=0x5625432871c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:32:30.121 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:30.119+0000 7f9bc0c00640 1 --2- 172.21.15.47:0/1431896470 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56254328a290 0x5625432871c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:32:30.121 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:30.119+0000 7f9bb3e00640 1 --2- 172.21.15.47:0/1431896470 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56254328a290 0x5625432871c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:32:30.121 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:30.119+0000 7f9bb3e00640 1 -- 172.21.15.47:0/1431896470 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562543280f20 msgr2=0x5625432898f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:32:30.121 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:30.119+0000 7f9bb3e00640 1 --2- 172.21.15.47:0/1431896470 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562543280f20 0x5625432898f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:32:30.121 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:30.119+0000 7f9bb3e00640 1 -- 172.21.15.47:0/1431896470 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x562542fc37d0 con 0x562543281c00 2024-03-21T10:32:30.121 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:30.119+0000 7f9bc0c00640 1 --2- 172.21.15.47:0/1431896470 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56254328a290 0x5625432871c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T10:32:30.122 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:30.119+0000 7f9bb3e00640 1 --2- 172.21.15.47:0/1431896470 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562543281c00 0x562543286c80 secure :-1 s=READY pgs=1012 cs=0 l=1 rev1=1 crypto rx=0x7f9ba4034670 tx=0x7f9ba402bcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d551c9a683da37d server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:32:30.122 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:30.120+0000 7f9ba9600640 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-03-21T10:32:30.122 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:30.120+0000 7f9ba9600640 1 -- 172.21.15.47:0/1431896470 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ba404e070 con 0x562543281c00 2024-03-21T10:32:30.122 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:30.120+0000 7f9ba9600640 1 -- 172.21.15.47:0/1431896470 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(99..99 src has 1..99) v4 ==== 9562+0+0 (secure 0 0 0) 0x7f9ba4034b60 con 0x562543281c00 2024-03-21T10:32:30.122 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:30.120+0000 7f9bc5985740 1 -- 172.21.15.47:0/1431896470 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f9ba40043e0 con 0x562543281c00 2024-03-21T10:32:30.124 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:30.120+0000 7f9ba9600640 1 -- 172.21.15.47:0/1431896470 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9ba402c870 con 0x562543281c00 2024-03-21T10:32:30.124 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:30.120+0000 7f9bc5985740 10 client.17724 fetch_fsmap learned FSMap version 32 2024-03-21T10:32:30.124 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:30.120+0000 7f9bc5985740 1 -- 172.21.15.47:0/1431896470 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x562542ef93f0 con 0x562543281c00 2024-03-21T10:32:30.124 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:30.120+0000 7f9ba9600640 1 -- 172.21.15.47:0/1431896470 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f9ba40043e0 con 0x562543281c00 2024-03-21T10:32:30.124 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:30.121+0000 7f9ba9600640 1 -- 172.21.15.47:0/1431896470 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 32) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f9ba40463f0 con 0x562543281c00 2024-03-21T10:32:30.124 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:30.121+0000 7f9bc5985740 10 client.17724 fetch_fsmap finished waiting for FSMap version 32 2024-03-21T10:32:30.124 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:30.121+0000 7f9bc5985740 10 client.17724 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-21T10:32:30.124 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:30.121+0000 7f9bc5985740 20 client.17724 populate_metadata read hostname 'smithi047' 2024-03-21T10:32:30.124 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:30.121+0000 7f9bc5985740 1 --2- 172.21.15.47:0/1431896470 >> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] conn(0x562543299f50 0x56254329c340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:32:30.124 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:30.121+0000 7f9bc5985740 4 client.17724 mds_command: new command op to 34535 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T10:32:30.125 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:30.121+0000 7f9bc5985740 1 -- 172.21.15.47:0/1431896470 --> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5625431f2790 con 0x562543299f50 2024-03-21T10:32:30.125 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:30.121+0000 7f9bbbe00640 1 --2- 172.21.15.47:0/1431896470 >> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] conn(0x562543299f50 0x56254329c340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:32:30.125 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:30.122+0000 7f9bbbe00640 1 --2- 172.21.15.47:0/1431896470 >> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] conn(0x562543299f50 0x56254329c340 crc :-1 s=READY pgs=169 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=29238d7195a8a4ce server_cookie=64d68edef1266ea in_seq=0 out_seq=0 2024-03-21T10:32:30.125 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:30.122+0000 7f9ba9600640 10 client.17724 ms_handle_connect on v2:172.21.15.100:6838/4173168440 2024-03-21T10:32:30.125 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:30.124+0000 7f9ba9600640 1 -- 172.21.15.47:0/1431896470 <== mds.0 v2:172.21.15.100:6838/4173168440 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x5625431f2790 con 0x562543299f50 2024-03-21T10:32:30.125 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:30.124+0000 7f9ba9600640 10 client.17724 handle_command_reply: tid=0 2024-03-21T10:32:30.125 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:30.124+0000 7f9bc5985740 1 client.17724 shutdown 2024-03-21T10:32:30.125 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:30.124+0000 7f9bc5985740 1 -- 172.21.15.47:0/1431896470 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562543281c00 msgr2=0x562543286c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:32:30.125 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:30.124+0000 7f9bc5985740 1 --2- 172.21.15.47:0/1431896470 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562543281c00 0x562543286c80 secure :-1 s=READY pgs=1012 cs=0 l=1 rev1=1 crypto rx=0x7f9ba4034670 tx=0x7f9ba402bcc0 comp rx=0 tx=0).stop 2024-03-21T10:32:30.125 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:30.124+0000 7f9bc5985740 1 -- 172.21.15.47:0/1431896470 shutdown_connections 2024-03-21T10:32:30.125 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:30.124+0000 7f9bc5985740 1 --2- 172.21.15.47:0/1431896470 >> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] conn(0x562543299f50 0x56254329c340 crc :-1 s=READY pgs=169 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:32:30.125 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:30.124+0000 7f9bc5985740 1 --2- 172.21.15.47:0/1431896470 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56254328a290 0x5625432871c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:32:30.125 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:30.124+0000 7f9bc5985740 1 --2- 172.21.15.47:0/1431896470 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562543281c00 0x562543286c80 unknown :-1 s=CLOSED pgs=1012 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:32:30.125 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:30.124+0000 7f9bc5985740 1 --2- 172.21.15.47:0/1431896470 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562543280f20 0x5625432898f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:32:30.125 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:30.124+0000 7f9ba9600640 0 client.17724 ms_handle_reset on v2:172.21.15.100:6838/4173168440 2024-03-21T10:32:30.126 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:30.124+0000 7f9bc5985740 1 -- 172.21.15.47:0/1431896470 >> 172.21.15.47:0/1431896470 conn(0x562543248200 msgr2=0x56254327d6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:32:30.126 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:30.124+0000 7f9bc5985740 1 -- 172.21.15.47:0/1431896470 shutdown_connections 2024-03-21T10:32:30.126 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:30.124+0000 7f9bc5985740 1 -- 172.21.15.47:0/1431896470 wait complete. 2024-03-21T10:32:30.126 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:30.124+0000 7f9bc5985740 20 client.17724 trim_cache size 0 max 16384 2024-03-21T10:32:30.142 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:30.142+0000 7f9bc5985740 1 Processor -- start 2024-03-21T10:32:30.142 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:30.142+0000 7f9bc5985740 1 -- start start 2024-03-21T10:32:30.142 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:30.142+0000 7f9bc5985740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562543280f20 0x5625432964c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:32:30.142 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:30.142+0000 7f9bc5985740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562543281c00 0x5625432893b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:32:30.142 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:30.142+0000 7f9bc5985740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56254328a290 0x5625432898f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:32:30.142 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:30.142+0000 7f9bc5985740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9bbc1b0bf0 con 0x562543280f20 2024-03-21T10:32:30.142 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:30.142+0000 7f9bc5985740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9bbc1b2570 con 0x56254328a290 2024-03-21T10:32:30.142 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:30.142+0000 7f9bc5985740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9bbc1b1250 con 0x562543281c00 2024-03-21T10:32:30.144 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:30.143+0000 7f9bbbe00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562543280f20 0x5625432964c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:32:30.144 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:30.143+0000 7f9bbbe00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562543280f20 0x5625432964c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:51240/0 (socket says 172.21.15.47:51240) 2024-03-21T10:32:30.144 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:30.143+0000 7f9bbbe00640 1 -- 172.21.15.47:0/113967142 learned_addr learned my addr 172.21.15.47:0/113967142 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:32:30.144 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:30.143+0000 7f9bb3e00640 1 --2- 172.21.15.47:0/113967142 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562543281c00 0x5625432893b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:32:30.144 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:30.143+0000 7f9bc0c00640 1 --2- 172.21.15.47:0/113967142 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56254328a290 0x5625432898f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:32:30.144 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:30.143+0000 7f9bbbe00640 1 -- 172.21.15.47:0/113967142 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562543281c00 msgr2=0x5625432893b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:32:30.144 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:30.143+0000 7f9bbbe00640 1 --2- 172.21.15.47:0/113967142 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562543281c00 0x5625432893b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:32:30.145 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:30.143+0000 7f9bbbe00640 1 -- 172.21.15.47:0/113967142 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56254328a290 msgr2=0x5625432898f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:32:30.145 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:30.143+0000 7f9bbbe00640 1 --2- 172.21.15.47:0/113967142 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56254328a290 0x5625432898f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:32:30.145 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:30.143+0000 7f9bbbe00640 1 -- 172.21.15.47:0/113967142 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562542eb90f0 con 0x562543280f20 2024-03-21T10:32:30.145 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:30.143+0000 7f9bc0c00640 1 --2- 172.21.15.47:0/113967142 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56254328a290 0x5625432898f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T10:32:30.145 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:30.144+0000 7f9bbbe00640 1 --2- 172.21.15.47:0/113967142 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562543280f20 0x5625432964c0 secure :-1 s=READY pgs=1013 cs=0 l=1 rev1=1 crypto rx=0x7f9bac09bd40 tx=0x7f9bac0bc040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ab995df01569b495 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:32:30.145 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:30.144+0000 7f9babe00640 1 -- 172.21.15.47:0/113967142 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9bac016e10 con 0x562543280f20 2024-03-21T10:32:30.145 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:30.144+0000 7f9babe00640 1 -- 172.21.15.47:0/113967142 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f9bac09a340 con 0x562543280f20 2024-03-21T10:32:30.145 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:30.144+0000 7f9babe00640 1 -- 172.21.15.47:0/113967142 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9bac0cd620 con 0x562543280f20 2024-03-21T10:32:30.145 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:30.145+0000 7f9bc5985740 1 -- 172.21.15.47:0/113967142 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562543280f20 msgr2=0x5625432964c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:32:30.145 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:30.145+0000 7f9bc5985740 1 --2- 172.21.15.47:0/113967142 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562543280f20 0x5625432964c0 secure :-1 s=READY pgs=1013 cs=0 l=1 rev1=1 crypto rx=0x7f9bac09bd40 tx=0x7f9bac0bc040 comp rx=0 tx=0).stop 2024-03-21T10:32:30.148 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:30.145+0000 7f9bc5985740 1 -- 172.21.15.47:0/113967142 shutdown_connections 2024-03-21T10:32:30.148 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:30.145+0000 7f9bc5985740 1 --2- 172.21.15.47:0/113967142 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56254328a290 0x5625432898f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:32:30.148 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:30.145+0000 7f9bc5985740 1 --2- 172.21.15.47:0/113967142 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562543281c00 0x5625432893b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:32:30.148 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:30.145+0000 7f9bc5985740 1 --2- 172.21.15.47:0/113967142 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562543280f20 0x5625432964c0 unknown :-1 s=CLOSED pgs=1013 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:32:30.148 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:30.145+0000 7f9bc5985740 1 -- 172.21.15.47:0/113967142 >> 172.21.15.47:0/113967142 conn(0x562543248200 msgr2=0x56254327b4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:32:30.149 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:30.145+0000 7f9bc5985740 1 -- 172.21.15.47:0/113967142 shutdown_connections 2024-03-21T10:32:30.149 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:30.145+0000 7f9bc5985740 1 -- 172.21.15.47:0/113967142 wait complete. 2024-03-21T10:32:30.149 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:30.146+0000 7f9bc5985740 1 -- start start 2024-03-21T10:32:30.149 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:30.146+0000 7f9bc5985740 1 Processor -- start 2024-03-21T10:32:30.149 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:30.146+0000 7f9bc5985740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562543281c00 0x562543283a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:32:30.149 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:30.146+0000 7f9bc5985740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56254328a290 0x562543283f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:32:30.149 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:30.146+0000 7f9bc5985740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5625432fac70 0x5625432fd060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:32:30.149 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:30.146+0000 7f9bc5985740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9bbc1b1f10 con 0x5625432fac70 2024-03-21T10:32:30.149 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:30.146+0000 7f9bc5985740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9bbc1b18b0 con 0x562543281c00 2024-03-21T10:32:30.149 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:30.146+0000 7f9bc5985740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9bbc1aedd0 con 0x56254328a290 2024-03-21T10:32:30.149 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:30.146+0000 7f9bc5985740 10 client.0 _ll_register_callbacks cb 0x7f9bac005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T10:32:30.149 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:30.147+0000 7f9bc0c00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5625432fac70 0x5625432fd060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:32:30.149 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:30.147+0000 7f9bc0c00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5625432fac70 0x5625432fd060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:51246/0 (socket says 172.21.15.47:51246) 2024-03-21T10:32:30.149 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:30.147+0000 7f9bc0c00640 1 -- 172.21.15.47:0/1158237711 learned_addr learned my addr 172.21.15.47:0/1158237711 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:32:30.150 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:30.147+0000 7f9bbbe00640 1 --2- 172.21.15.47:0/1158237711 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562543281c00 0x562543283a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:32:30.150 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:30.147+0000 7f9bb3e00640 1 --2- 172.21.15.47:0/1158237711 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56254328a290 0x562543283f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:32:30.150 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:30.147+0000 7f9bc0c00640 1 -- 172.21.15.47:0/1158237711 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56254328a290 msgr2=0x562543283f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:32:30.150 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:30.147+0000 7f9bc0c00640 1 --2- 172.21.15.47:0/1158237711 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56254328a290 0x562543283f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:32:30.150 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:30.147+0000 7f9bc0c00640 1 -- 172.21.15.47:0/1158237711 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562543281c00 msgr2=0x562543283a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:32:30.150 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:30.147+0000 7f9bc0c00640 1 --2- 172.21.15.47:0/1158237711 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562543281c00 0x562543283a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:32:30.150 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:30.147+0000 7f9bc0c00640 1 -- 172.21.15.47:0/1158237711 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5625432fd5d0 con 0x5625432fac70 2024-03-21T10:32:30.150 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:30.147+0000 7f9bbbe00640 1 --2- 172.21.15.47:0/1158237711 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562543281c00 0x562543283a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T10:32:30.150 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:30.147+0000 7f9bc0c00640 1 --2- 172.21.15.47:0/1158237711 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5625432fac70 0x5625432fd060 secure :-1 s=READY pgs=1014 cs=0 l=1 rev1=1 crypto rx=0x7f9bb4011530 tx=0x7f9bb4004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8c860b07b666625a server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:32:30.150 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:30.147+0000 7f9ba9600640 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-03-21T10:32:30.150 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:30.148+0000 7f9ba9600640 1 -- 172.21.15.47:0/1158237711 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9bb4010e70 con 0x5625432fac70 2024-03-21T10:32:30.150 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:30.148+0000 7f9bc5985740 1 -- 172.21.15.47:0/1158237711 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f9bac09a340 con 0x5625432fac70 2024-03-21T10:32:30.150 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:30.148+0000 7f9ba9600640 1 -- 172.21.15.47:0/1158237711 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(99..99 src has 1..99) v4 ==== 9562+0+0 (secure 0 0 0) 0x7f9bb402a3f0 con 0x5625432fac70 2024-03-21T10:32:30.150 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:30.148+0000 7f9ba9600640 1 -- 172.21.15.47:0/1158237711 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9bb4032050 con 0x5625432fac70 2024-03-21T10:32:30.151 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:30.148+0000 7f9bc5985740 10 client.17730 fetch_fsmap learned FSMap version 32 2024-03-21T10:32:30.151 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:30.148+0000 7f9bc5985740 1 -- 172.21.15.47:0/1158237711 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x5625432fe990 con 0x5625432fac70 2024-03-21T10:32:30.151 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:30.148+0000 7f9ba9600640 1 -- 172.21.15.47:0/1158237711 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f9bac09a340 con 0x5625432fac70 2024-03-21T10:32:30.151 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:30.149+0000 7f9ba9600640 1 -- 172.21.15.47:0/1158237711 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 32) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f9bb401d050 con 0x5625432fac70 2024-03-21T10:32:30.151 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:30.149+0000 7f9bc5985740 10 client.17730 fetch_fsmap finished waiting for FSMap version 32 2024-03-21T10:32:30.151 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:30.149+0000 7f9bc5985740 10 client.17730 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-21T10:32:30.151 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:30.149+0000 7f9bc5985740 20 client.17730 populate_metadata read hostname 'smithi047' 2024-03-21T10:32:30.151 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:30.149+0000 7f9bc5985740 1 --2- 172.21.15.47:0/1158237711 >> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] conn(0x5625432ff120 0x562543301540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:32:30.151 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:30.149+0000 7f9bc5985740 4 client.17730 mds_command: new command op to 34535 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-21T10:32:30.151 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:30.149+0000 7f9bc5985740 1 -- 172.21.15.47:0/1158237711 --> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x5625431ecae0 con 0x5625432ff120 2024-03-21T10:32:30.152 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:30.149+0000 7f9bbbe00640 1 --2- 172.21.15.47:0/1158237711 >> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] conn(0x5625432ff120 0x562543301540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:32:30.152 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:30.150+0000 7f9bbbe00640 1 --2- 172.21.15.47:0/1158237711 >> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] conn(0x5625432ff120 0x562543301540 crc :-1 s=READY pgs=170 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f31deba4163ed4eb server_cookie=515eca0986ca9d4f in_seq=0 out_seq=0 2024-03-21T10:32:30.152 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:30.150+0000 7f9ba9600640 10 client.17730 ms_handle_connect on v2:172.21.15.100:6838/4173168440 2024-03-21T10:32:30.153 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:30.150+0000 7f9ba9600640 1 -- 172.21.15.47:0/1158237711 <== mds.0 v2:172.21.15.100:6838/4173168440 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x5625431ecae0 con 0x5625432ff120 2024-03-21T10:32:30.153 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:30.150+0000 7f9ba9600640 10 client.17730 handle_command_reply: tid=0 2024-03-21T10:32:30.153 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:30.150+0000 7f9bc5985740 1 client.17730 shutdown 2024-03-21T10:32:30.153 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:30.150+0000 7f9bc5985740 1 -- 172.21.15.47:0/1158237711 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5625432fac70 msgr2=0x5625432fd060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:32:30.153 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:30.150+0000 7f9bc5985740 1 --2- 172.21.15.47:0/1158237711 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5625432fac70 0x5625432fd060 secure :-1 s=READY pgs=1014 cs=0 l=1 rev1=1 crypto rx=0x7f9bb4011530 tx=0x7f9bb4004a40 comp rx=0 tx=0).stop 2024-03-21T10:32:30.153 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:30.150+0000 7f9bc5985740 1 -- 172.21.15.47:0/1158237711 shutdown_connections 2024-03-21T10:32:30.153 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:30.150+0000 7f9bc5985740 1 --2- 172.21.15.47:0/1158237711 >> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] conn(0x5625432ff120 0x562543301540 crc :-1 s=READY pgs=170 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:32:30.153 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:30.150+0000 7f9bc5985740 1 --2- 172.21.15.47:0/1158237711 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5625432fac70 0x5625432fd060 unknown :-1 s=CLOSED pgs=1014 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:32:30.153 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:30.150+0000 7f9bc5985740 1 --2- 172.21.15.47:0/1158237711 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56254328a290 0x562543283f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:32:30.153 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:30.150+0000 7f9bc5985740 1 --2- 172.21.15.47:0/1158237711 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562543281c00 0x562543283a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:32:30.153 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:30.150+0000 7f9ba9600640 0 client.17730 ms_handle_reset on v2:172.21.15.100:6838/4173168440 2024-03-21T10:32:30.153 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:30.150+0000 7f9bc5985740 1 -- 172.21.15.47:0/1158237711 >> 172.21.15.47:0/1158237711 conn(0x562543248200 msgr2=0x562543285490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:32:30.153 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:30.151+0000 7f9bc5985740 1 -- 172.21.15.47:0/1158237711 shutdown_connections 2024-03-21T10:32:30.153 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:30.151+0000 7f9bc5985740 1 -- 172.21.15.47:0/1158237711 wait complete. 2024-03-21T10:32:30.153 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:30.151+0000 7f9bc5985740 20 client.17730 trim_cache size 0 max 16384 2024-03-21T10:32:30.154 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-03-21T10:32:30.154 INFO:teuthology.orchestra.run.smithi047.stdout: "status": "no active scrubs running", 2024-03-21T10:32:30.154 INFO:teuthology.orchestra.run.smithi047.stdout: "scrubs": {} 2024-03-21T10:32:30.154 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-03-21T10:32:30.155 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:30.151+0000 7f9bc2c00640 1 -- 172.21.15.47:0/3823049562 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f9ba00a3090 msgr2=0x7f9ba00a5550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:32:30.155 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:30.151+0000 7f9bc2c00640 1 --2- 172.21.15.47:0/3823049562 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f9ba00a3090 0x7f9ba00a5550 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f9bbc06ad50 tx=0x7f9bb4009500 comp rx=0 tx=0).stop 2024-03-21T10:32:30.155 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:30.151+0000 7f9bc2c00640 1 -- 172.21.15.47:0/3823049562 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9bbc1028e0 msgr2=0x7f9bbc19a110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:32:30.155 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:30.151+0000 7f9bc2c00640 1 --2- 172.21.15.47:0/3823049562 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9bbc1028e0 0x7f9bbc19a110 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f9bac00e9f0 tx=0x7f9bac00edf0 comp rx=0 tx=0).stop 2024-03-21T10:32:30.155 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:30.152+0000 7f9bc2c00640 1 -- 172.21.15.47:0/3823049562 shutdown_connections 2024-03-21T10:32:30.155 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:30.152+0000 7f9bc2c00640 1 --2- 172.21.15.47:0/3823049562 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9bbc10bcb0 0x7f9bbc19e9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:32:30.155 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:30.152+0000 7f9bc2c00640 1 --2- 172.21.15.47:0/3823049562 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9bbc103200 0x7f9bbc19a650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:32:30.155 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:30.152+0000 7f9bc2c00640 1 --2- 172.21.15.47:0/3823049562 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f9ba00a3090 0x7f9ba00a5550 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:32:30.155 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:30.152+0000 7f9bc2c00640 1 --2- 172.21.15.47:0/3823049562 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9bbc1028e0 0x7f9bbc19a110 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:32:30.155 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:30.152+0000 7f9bc2c00640 1 -- 172.21.15.47:0/3823049562 >> 172.21.15.47:0/3823049562 conn(0x7f9bbc076920 msgr2=0x7f9bbc100560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:32:30.155 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:30.152+0000 7f9bc2c00640 1 -- 172.21.15.47:0/3823049562 shutdown_connections 2024-03-21T10:32:30.156 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:30.152+0000 7f9bc2c00640 1 -- 172.21.15.47:0/3823049562 wait complete. 2024-03-21T10:32:30.168 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-03-21T10:32:30.169 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-21T10:32:30.378 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:30.378+0000 7f206b200640 1 -- 172.21.15.47:0/4093673818 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f206c1063f0 msgr2=0x7f206c068800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:32:30.378 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:30.378+0000 7f206b200640 1 --2- 172.21.15.47:0/4093673818 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f206c1063f0 0x7f206c068800 secure :-1 s=READY pgs=1015 cs=0 l=1 rev1=1 crypto rx=0x7f205c009900 tx=0x7f205c02bc30 comp rx=0 tx=0).stop 2024-03-21T10:32:30.378 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:30.378+0000 7f206b200640 1 -- 172.21.15.47:0/4093673818 shutdown_connections 2024-03-21T10:32:30.378 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:30.378+0000 7f206b200640 1 --2- 172.21.15.47:0/4093673818 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f206c103ef0 0x7f206c1043a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:32:30.378 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:30.378+0000 7f206b200640 1 --2- 172.21.15.47:0/4093673818 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f206c106860 0x7f206c068d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:32:30.379 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:30.378+0000 7f206b200640 1 --2- 172.21.15.47:0/4093673818 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f206c1063f0 0x7f206c068800 unknown :-1 s=CLOSED pgs=1015 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:32:30.379 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:30.378+0000 7f206b200640 1 -- 172.21.15.47:0/4093673818 >> 172.21.15.47:0/4093673818 conn(0x7f206c075fb0 msgr2=0x7f206c0763c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:32:30.379 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:30.378+0000 7f206b200640 1 -- 172.21.15.47:0/4093673818 shutdown_connections 2024-03-21T10:32:30.379 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:30.378+0000 7f206b200640 1 -- 172.21.15.47:0/4093673818 wait complete. 2024-03-21T10:32:30.380 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:30.379+0000 7f206b200640 1 Processor -- start 2024-03-21T10:32:30.380 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:30.380+0000 7f206b200640 1 -- start start 2024-03-21T10:32:30.380 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:30.380+0000 7f206b200640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f206c103ef0 0x7f206c19e1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:32:30.380 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:30.380+0000 7f206b200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f206c1063f0 0x7f206c19e730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:32:30.380 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:30.380+0000 7f206b200640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f206c106860 0x7f206c1a2ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:32:30.380 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:30.380+0000 7f206b200640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f206c117d10 con 0x7f206c106860 2024-03-21T10:32:30.380 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:30.380+0000 7f206b200640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f206c117b90 con 0x7f206c1063f0 2024-03-21T10:32:30.380 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:30.380+0000 7f206b200640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f206c117e90 con 0x7f206c103ef0 2024-03-21T10:32:30.381 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:30.380+0000 7f206a800640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f206c106860 0x7f206c1a2ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:32:30.381 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:30.381+0000 7f206a800640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f206c106860 0x7f206c1a2ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:51274/0 (socket says 172.21.15.47:51274) 2024-03-21T10:32:30.381 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:30.381+0000 7f206a800640 1 -- 172.21.15.47:0/4175665670 learned_addr learned my addr 172.21.15.47:0/4175665670 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:32:30.381 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:30.381+0000 7f2069400640 1 --2- 172.21.15.47:0/4175665670 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f206c1063f0 0x7f206c19e730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:32:30.381 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:30.381+0000 7f2069e00640 1 --2- 172.21.15.47:0/4175665670 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f206c103ef0 0x7f206c19e1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:32:30.382 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:30.381+0000 7f206a800640 1 -- 172.21.15.47:0/4175665670 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f206c103ef0 msgr2=0x7f206c19e1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:32:30.382 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:30.381+0000 7f206a800640 1 --2- 172.21.15.47:0/4175665670 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f206c103ef0 0x7f206c19e1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:32:30.382 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:30.381+0000 7f206a800640 1 -- 172.21.15.47:0/4175665670 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f206c1063f0 msgr2=0x7f206c19e730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:32:30.382 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:30.381+0000 7f206a800640 1 --2- 172.21.15.47:0/4175665670 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f206c1063f0 0x7f206c19e730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:32:30.382 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:30.381+0000 7f206a800640 1 -- 172.21.15.47:0/4175665670 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f206c1a31e0 con 0x7f206c106860 2024-03-21T10:32:30.382 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:30.381+0000 7f2069e00640 1 --2- 172.21.15.47:0/4175665670 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f206c103ef0 0x7f206c19e1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T10:32:30.382 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:30.381+0000 7f206a800640 1 --2- 172.21.15.47:0/4175665670 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f206c106860 0x7f206c1a2ab0 secure :-1 s=READY pgs=1016 cs=0 l=1 rev1=1 crypto rx=0x7f206400eaa0 tx=0x7f206400ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:32:30.382 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:30.382+0000 7f2062a00640 1 -- 172.21.15.47:0/4175665670 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f206400cdb0 con 0x7f206c106860 2024-03-21T10:32:30.383 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:30.382+0000 7f2062a00640 1 -- 172.21.15.47:0/4175665670 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f2064005020 con 0x7f206c106860 2024-03-21T10:32:30.383 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:30.382+0000 7f206b200640 1 -- 172.21.15.47:0/4175665670 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f206c1a34d0 con 0x7f206c106860 2024-03-21T10:32:30.383 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:30.382+0000 7f2062a00640 1 -- 172.21.15.47:0/4175665670 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20640116d0 con 0x7f206c106860 2024-03-21T10:32:30.383 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:30.382+0000 7f206b200640 1 -- 172.21.15.47:0/4175665670 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f206c1aad60 con 0x7f206c106860 2024-03-21T10:32:30.383 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:30.383+0000 7f206b200640 1 -- 172.21.15.47:0/4175665670 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f206c0683b0 con 0x7f206c106860 2024-03-21T10:32:30.385 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:30.385+0000 7f2062a00640 1 -- 172.21.15.47:0/4175665670 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 27) v1 ==== 137985+0+0 (secure 0 0 0) 0x7f20640051d0 con 0x7f206c106860 2024-03-21T10:32:30.387 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:30.388+0000 7f2062a00640 1 --2- 172.21.15.47:0/4175665670 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f20500a3090 0x7f20500a5550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:32:30.387 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:30.388+0000 7f2062a00640 1 -- 172.21.15.47:0/4175665670 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 9562+0+0 (secure 0 0 0) 0x7f20640d0150 con 0x7f206c106860 2024-03-21T10:32:30.388 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:30.388+0000 7f2069e00640 1 --2- 172.21.15.47:0/4175665670 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f20500a3090 0x7f20500a5550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:32:30.388 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:30.389+0000 7f2069e00640 1 --2- 172.21.15.47:0/4175665670 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f20500a3090 0x7f20500a5550 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f205c006010 tx=0x7f205c005a40 comp rx=0 tx=0).ready entity=mgr.15687 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:32:30.389 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:30.390+0000 7f2062a00640 1 -- 172.21.15.47:0/4175665670 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f2064016030 con 0x7f206c106860 2024-03-21T10:32:30.636 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:30.636+0000 7f206b200640 1 -- 172.21.15.47:0/4175665670 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f206c1ab000 con 0x7f206c106860 2024-03-21T10:32:30.637 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:30.637+0000 7f2062a00640 1 -- 172.21.15.47:0/4175665670 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 32 v32) v1 ==== 94+0+11471 (secure 0 0 0) 0x7f206409c830 con 0x7f206c106860 2024-03-21T10:32:30.637 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-21T10:32:30.637 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":32,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14511,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6835/735724037","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6834","nonce":735724037},{"type":"v1","addr":"172.21.15.47:6835","nonce":735724037}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14529,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6837/2248191724","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6836","nonce":2248191724},{"type":"v1","addr":"172.21.15.47:6837","nonce":2248191724}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":32,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-21T10:21:58.329063+0000","modified":"2024-03-21T10:26:17.169892+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":34535,"mds_1":34493,"mds_2":34454,"mds_3":34427,"mds_4":14619},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14604":{"gid":14604,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.47:6839/2881335356","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6838","nonce":2881335356},{"type":"v1","addr":"172.21.15.47:6839","nonce":2881335356}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_14619":{"gid":14619,"name":"a","rank":4,"incarnation":27,"state":"up:active","state_seq":11,"addr":"172.21.15.47:6841/3401578978","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6840","nonce":3401578978},{"type":"v1","addr":"172.21.15.47:6841","nonce":3401578978}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24406":{"gid":24406,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.157:6833/4000010136","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":4000010136},{"type":"v1","addr":"172.21.15.157:6833","nonce":4000010136}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24457":{"gid":24457,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.157:6835/3902868651","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":3902868651},{"type":"v1","addr":"172.21.15.157:6835","nonce":3902868651}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24475":{"gid":24475,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.157:6837/1908087204","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6836","nonce":1908087204},{"type":"v1","addr":"172.21.15.157:6837","nonce":1908087204}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_25729":{"gid":25729,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.157:6839/1998408416","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6838","nonce":1998408416},{"type":"v1","addr":"172.21.15.157:6839","nonce":1998408416}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_34427":{"gid":34427,"name":"k","rank":3,"incarnation":25,"state":"up:active","state_seq":20,"addr":"172.21.15.100:6833/2723302038","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6832","nonce":2723302038},{"type":"v1","addr":"172.21.15.100:6833","nonce":2723302038}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_34454":{"gid":34454,"name":"e","rank":2,"incarnation":22,"state":"up:active","state_seq":17,"addr":"172.21.15.100:6835/2597849957","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":2597849957},{"type":"v1","addr":"172.21.15.100:6835","nonce":2597849957}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_34493":{"gid":34493,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":13,"addr":"172.21.15.100:6837/3497644162","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6836","nonce":3497644162},{"type":"v1","addr":"172.21.15.100:6837","nonce":3497644162}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_34535":{"gid":34535,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.100:6839/4173168440","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6838","nonce":4173168440},{"type":"v1","addr":"172.21.15.100:6839","nonce":4173168440}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":34535,"qdb_cluster":[34427,14619,34454,34535,34493]},"id":1}]} 2024-03-21T10:32:30.637 INFO:teuthology.orchestra.run.smithi047.stderr:dumped fsmap epoch 32 2024-03-21T10:32:30.640 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:30.640+0000 7f206b200640 1 -- 172.21.15.47:0/4175665670 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f20500a3090 msgr2=0x7f20500a5550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:32:30.640 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:30.640+0000 7f206b200640 1 --2- 172.21.15.47:0/4175665670 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f20500a3090 0x7f20500a5550 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f205c006010 tx=0x7f205c005a40 comp rx=0 tx=0).stop 2024-03-21T10:32:30.640 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:30.640+0000 7f206b200640 1 -- 172.21.15.47:0/4175665670 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f206c106860 msgr2=0x7f206c1a2ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:32:30.640 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:30.640+0000 7f206b200640 1 --2- 172.21.15.47:0/4175665670 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f206c106860 0x7f206c1a2ab0 secure :-1 s=READY pgs=1016 cs=0 l=1 rev1=1 crypto rx=0x7f206400eaa0 tx=0x7f206400ef60 comp rx=0 tx=0).stop 2024-03-21T10:32:30.640 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:30.640+0000 7f206b200640 1 -- 172.21.15.47:0/4175665670 shutdown_connections 2024-03-21T10:32:30.640 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:30.640+0000 7f206b200640 1 --2- 172.21.15.47:0/4175665670 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f206c106860 0x7f206c1a2ab0 unknown :-1 s=CLOSED pgs=1016 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:32:30.640 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:30.640+0000 7f206b200640 1 --2- 172.21.15.47:0/4175665670 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f20500a3090 0x7f20500a5550 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:32:30.640 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:30.640+0000 7f206b200640 1 --2- 172.21.15.47:0/4175665670 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f206c1063f0 0x7f206c19e730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:32:30.640 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:30.640+0000 7f206b200640 1 --2- 172.21.15.47:0/4175665670 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f206c103ef0 0x7f206c19e1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:32:30.640 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:30.640+0000 7f206b200640 1 -- 172.21.15.47:0/4175665670 >> 172.21.15.47:0/4175665670 conn(0x7f206c075fb0 msgr2=0x7f206c101370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:32:30.640 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:30.640+0000 7f206b200640 1 -- 172.21.15.47:0/4175665670 shutdown_connections 2024-03-21T10:32:30.640 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:30.640+0000 7f206b200640 1 -- 172.21.15.47:0/4175665670 wait complete. 2024-03-21T10:32:30.660 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-21T10:32:30.660 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-03-21T10:32:30.883 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:30.883+0000 7f9406e00640 1 -- 172.21.15.47:0/2170886985 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9400102ff0 msgr2=0x7f940010b4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:32:30.884 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:30.883+0000 7f9406e00640 1 --2- 172.21.15.47:0/2170886985 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9400102ff0 0x7f940010b4d0 secure :-1 s=READY pgs=1017 cs=0 l=1 rev1=1 crypto rx=0x7f93ec009900 tx=0x7f93ec02bcd0 comp rx=0 tx=0).stop 2024-03-21T10:32:30.884 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:30.884+0000 7f9406e00640 1 -- 172.21.15.47:0/2170886985 shutdown_connections 2024-03-21T10:32:30.884 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:30.884+0000 7f9406e00640 1 --2- 172.21.15.47:0/2170886985 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f940010baf0 0x7f940010df90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:32:30.884 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:30.884+0000 7f9406e00640 1 --2- 172.21.15.47:0/2170886985 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9400102ff0 0x7f940010b4d0 unknown :-1 s=CLOSED pgs=1017 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:32:30.884 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:30.884+0000 7f9406e00640 1 --2- 172.21.15.47:0/2170886985 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f94001026d0 0x7f9400102ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:32:30.884 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:30.884+0000 7f9406e00640 1 -- 172.21.15.47:0/2170886985 >> 172.21.15.47:0/2170886985 conn(0x7f94000fbef0 msgr2=0x7f94000fe310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:32:30.884 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:30.884+0000 7f9406e00640 1 -- 172.21.15.47:0/2170886985 shutdown_connections 2024-03-21T10:32:30.884 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:30.884+0000 7f9406e00640 1 -- 172.21.15.47:0/2170886985 wait complete. 2024-03-21T10:32:30.885 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:30.885+0000 7f9406e00640 1 Processor -- start 2024-03-21T10:32:30.885 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:30.885+0000 7f9406e00640 1 -- start start 2024-03-21T10:32:30.886 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:30.886+0000 7f9406e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f94001026d0 0x7f9400100320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:32:30.886 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:30.886+0000 7f9406e00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9400102ff0 0x7f9400100860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:32:30.886 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:30.886+0000 7f9406e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f940010baf0 0x7f9400100da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:32:30.886 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:30.886+0000 7f9406e00640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9400111760 con 0x7f940010baf0 2024-03-21T10:32:30.886 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:30.886+0000 7f9406e00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94001115e0 con 0x7f94001026d0 2024-03-21T10:32:30.886 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:30.886+0000 7f9406e00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94001118e0 con 0x7f9400102ff0 2024-03-21T10:32:30.886 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:30.886+0000 7f9406400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f940010baf0 0x7f9400100da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:32:30.886 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:30.886+0000 7f9406400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f940010baf0 0x7f9400100da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:51298/0 (socket says 172.21.15.47:51298) 2024-03-21T10:32:30.887 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:30.886+0000 7f9406400640 1 -- 172.21.15.47:0/2515679930 learned_addr learned my addr 172.21.15.47:0/2515679930 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:32:30.887 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:30.887+0000 7f9405a00640 1 --2- 172.21.15.47:0/2515679930 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f94001026d0 0x7f9400100320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:32:30.887 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:30.887+0000 7f9405000640 1 --2- 172.21.15.47:0/2515679930 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9400102ff0 0x7f9400100860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:32:30.888 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:30.887+0000 7f9406400640 1 -- 172.21.15.47:0/2515679930 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9400102ff0 msgr2=0x7f9400100860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:32:30.888 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:30.887+0000 7f9406400640 1 --2- 172.21.15.47:0/2515679930 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9400102ff0 0x7f9400100860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:32:30.888 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:30.887+0000 7f9406400640 1 -- 172.21.15.47:0/2515679930 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f94001026d0 msgr2=0x7f9400100320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:32:30.888 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:30.887+0000 7f9406400640 1 --2- 172.21.15.47:0/2515679930 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f94001026d0 0x7f9400100320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:32:30.888 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:30.887+0000 7f9406400640 1 -- 172.21.15.47:0/2515679930 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94001a6200 con 0x7f940010baf0 2024-03-21T10:32:30.888 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:30.887+0000 7f9405a00640 1 --2- 172.21.15.47:0/2515679930 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f94001026d0 0x7f9400100320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T10:32:30.888 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:30.887+0000 7f9406400640 1 --2- 172.21.15.47:0/2515679930 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f940010baf0 0x7f9400100da0 secure :-1 s=READY pgs=1018 cs=0 l=1 rev1=1 crypto rx=0x7f93f800eb10 tx=0x7f93f800c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:32:30.888 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:30.888+0000 7f93fe000640 1 -- 172.21.15.47:0/2515679930 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93f8004b20 con 0x7f940010baf0 2024-03-21T10:32:30.888 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:30.888+0000 7f9406e00640 1 -- 172.21.15.47:0/2515679930 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f94001a64f0 con 0x7f940010baf0 2024-03-21T10:32:30.888 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:30.888+0000 7f9406e00640 1 -- 172.21.15.47:0/2515679930 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f94001a6a40 con 0x7f940010baf0 2024-03-21T10:32:30.888 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:30.888+0000 7f93fe000640 1 -- 172.21.15.47:0/2515679930 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f93f8004cc0 con 0x7f940010baf0 2024-03-21T10:32:30.888 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:30.888+0000 7f93fe000640 1 -- 172.21.15.47:0/2515679930 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93f8010690 con 0x7f940010baf0 2024-03-21T10:32:30.891 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:30.892+0000 7f93fe000640 1 -- 172.21.15.47:0/2515679930 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 27) v1 ==== 137985+0+0 (secure 0 0 0) 0x7f93f8005000 con 0x7f940010baf0 2024-03-21T10:32:30.892 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:30.892+0000 7f940b17b740 1 Processor -- start 2024-03-21T10:32:30.892 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:30.893+0000 7f940b17b740 1 -- start start 2024-03-21T10:32:30.893 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:30.893+0000 7f940b17b740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x557f67f488b0 0x557f67f48c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:32:30.893 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:30.893+0000 7f940b17b740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557f67f49260 0x557f67f51810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:32:30.893 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:30.893+0000 7f940b17b740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557f67f51de0 0x557f67f54230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:32:30.893 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:30.893+0000 7f940b17b740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x557f67eac2c0 con 0x557f67f488b0 2024-03-21T10:32:30.893 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:30.893+0000 7f940b17b740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x557f67e54c90 con 0x557f67f51de0 2024-03-21T10:32:30.893 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:30.893+0000 7f940b17b740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x557f67ea8360 con 0x557f67f49260 2024-03-21T10:32:30.893 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:30.894+0000 7f9405000640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x557f67f488b0 0x557f67f48c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:32:30.893 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:30.894+0000 7f9405000640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x557f67f488b0 0x557f67f48c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:51308/0 (socket says 172.21.15.47:51308) 2024-03-21T10:32:30.894 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:30.894+0000 7f9405000640 1 -- 172.21.15.47:0/2109583374 learned_addr learned my addr 172.21.15.47:0/2109583374 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:32:30.895 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:30.894+0000 7f9405000640 1 --2- 172.21.15.47:0/2109583374 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557f67f51de0 0x557f67f54230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:32:30.895 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:30.894+0000 7f9405a00640 1 --2- 172.21.15.47:0/2109583374 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557f67f49260 0x557f67f51810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:32:30.895 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:30.894+0000 7f93fe000640 1 --2- 172.21.15.47:0/2515679930 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f93e80a3110 0x7f93e80a55d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:32:30.895 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:30.895+0000 7f93fe000640 1 -- 172.21.15.47:0/2515679930 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 9562+0+0 (secure 0 0 0) 0x7f93f80d01f0 con 0x7f940010baf0 2024-03-21T10:32:30.895 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:30.895+0000 7f9405000640 1 -- 172.21.15.47:0/2109583374 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557f67f49260 msgr2=0x557f67f51810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:32:30.895 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:30.895+0000 7f9405000640 1 --2- 172.21.15.47:0/2109583374 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557f67f49260 0x557f67f51810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:32:30.895 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:30.895+0000 7f9405000640 1 -- 172.21.15.47:0/2109583374 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557f67f51de0 msgr2=0x557f67f54230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:32:30.895 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:30.895+0000 7f9405000640 1 --2- 172.21.15.47:0/2109583374 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557f67f51de0 0x557f67f54230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:32:30.895 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:30.895+0000 7f9405000640 1 -- 172.21.15.47:0/2109583374 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557f67e79080 con 0x557f67f488b0 2024-03-21T10:32:30.896 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:30.895+0000 7f9406400640 1 --2- 172.21.15.47:0/2515679930 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f93e80a3110 0x7f93e80a55d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:32:30.896 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:30.895+0000 7f9405000640 1 --2- 172.21.15.47:0/2109583374 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x557f67f488b0 0x557f67f48c90 secure :-1 s=READY pgs=1019 cs=0 l=1 rev1=1 crypto rx=0x7f93ec005ab0 tx=0x7f93ec005b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c78f7ac31b4ea28e server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:32:30.896 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:30.896+0000 7f93f2a00640 1 -- 172.21.15.47:0/2109583374 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93ec005050 con 0x557f67f488b0 2024-03-21T10:32:30.896 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:30.896+0000 7f93f2a00640 1 -- 172.21.15.47:0/2109583374 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f93ec02c880 con 0x557f67f488b0 2024-03-21T10:32:30.896 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:30.896+0000 7f93f2a00640 1 -- 172.21.15.47:0/2109583374 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93ec004920 con 0x557f67f488b0 2024-03-21T10:32:30.897 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:30.897+0000 7f940b17b740 1 -- 172.21.15.47:0/2109583374 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x557f67f488b0 msgr2=0x557f67f48c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:32:30.897 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:30.897+0000 7f9406400640 1 --2- 172.21.15.47:0/2515679930 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f93e80a3110 0x7f93e80a55d0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f93f80cd5e0 tx=0x7f93f800c950 comp rx=0 tx=0).ready entity=mgr.15687 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:32:30.897 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:30.897+0000 7f940b17b740 1 --2- 172.21.15.47:0/2109583374 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x557f67f488b0 0x557f67f48c90 secure :-1 s=READY pgs=1019 cs=0 l=1 rev1=1 crypto rx=0x7f93ec005ab0 tx=0x7f93ec005b50 comp rx=0 tx=0).stop 2024-03-21T10:32:30.897 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:30.897+0000 7f940b17b740 1 -- 172.21.15.47:0/2109583374 shutdown_connections 2024-03-21T10:32:30.897 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:30.897+0000 7f940b17b740 1 --2- 172.21.15.47:0/2109583374 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557f67f51de0 0x557f67f54230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:32:30.897 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:30.897+0000 7f940b17b740 1 --2- 172.21.15.47:0/2109583374 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557f67f49260 0x557f67f51810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:32:30.897 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:30.897+0000 7f940b17b740 1 --2- 172.21.15.47:0/2109583374 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x557f67f488b0 0x557f67f48c90 unknown :-1 s=CLOSED pgs=1019 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:32:30.897 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:30.897+0000 7f940b17b740 1 -- 172.21.15.47:0/2109583374 >> 172.21.15.47:0/2109583374 conn(0x557f67f42f80 msgr2=0x557f67f43360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:32:30.897 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:30.897+0000 7f940b17b740 1 -- 172.21.15.47:0/2109583374 shutdown_connections 2024-03-21T10:32:30.898 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:30.897+0000 7f940b17b740 1 -- 172.21.15.47:0/2109583374 wait complete. 2024-03-21T10:32:30.898 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:30.897+0000 7f940b17b740 1 -- start start 2024-03-21T10:32:30.898 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:30.898+0000 7f940b17b740 1 Processor -- start 2024-03-21T10:32:30.900 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:30.898+0000 7f940b17b740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x557f67f488b0 0x557f67f51700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:32:30.901 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:30.898+0000 7f940b17b740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557f67f49260 0x557f67f4ea90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:32:30.901 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:30.898+0000 7f940b17b740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557f67f51de0 0x557f67f4f030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:32:30.901 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:30.898+0000 7f940b17b740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x557f67f54c90 con 0x557f67f488b0 2024-03-21T10:32:30.901 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:30.898+0000 7f940b17b740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x557f67e680d0 con 0x557f67f51de0 2024-03-21T10:32:30.901 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:30.898+0000 7f940b17b740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x557f67f54e10 con 0x557f67f49260 2024-03-21T10:32:30.901 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:30.898+0000 7f940b17b740 10 client.0 _ll_register_callbacks cb 0x557f67e6ce80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T10:32:30.901 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:30.898+0000 7f9405000640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x557f67f488b0 0x557f67f51700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:32:30.901 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:30.898+0000 7f9405000640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x557f67f488b0 0x557f67f51700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:51320/0 (socket says 172.21.15.47:51320) 2024-03-21T10:32:30.901 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:30.898+0000 7f9405000640 1 -- 172.21.15.47:0/3215523126 learned_addr learned my addr 172.21.15.47:0/3215523126 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:32:30.901 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:30.899+0000 7f9405000640 1 --2- 172.21.15.47:0/3215523126 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557f67f51de0 0x557f67f4f030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:32:30.901 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:30.899+0000 7f9405a00640 1 --2- 172.21.15.47:0/3215523126 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557f67f49260 0x557f67f4ea90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:32:30.901 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:30.899+0000 7f9405000640 1 -- 172.21.15.47:0/3215523126 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557f67f49260 msgr2=0x557f67f4ea90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:32:30.901 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:30.899+0000 7f9405000640 1 --2- 172.21.15.47:0/3215523126 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557f67f49260 0x557f67f4ea90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:32:30.902 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:30.899+0000 7f9405000640 1 -- 172.21.15.47:0/3215523126 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557f67f51de0 msgr2=0x557f67f4f030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:32:30.902 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:30.899+0000 7f9405000640 1 --2- 172.21.15.47:0/3215523126 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557f67f51de0 0x557f67f4f030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:32:30.902 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:30.899+0000 7f9405000640 1 -- 172.21.15.47:0/3215523126 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x557f67bc03f0 con 0x557f67f488b0 2024-03-21T10:32:30.902 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:30.899+0000 7f9405000640 1 --2- 172.21.15.47:0/3215523126 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x557f67f488b0 0x557f67f51700 secure :-1 s=READY pgs=1020 cs=0 l=1 rev1=1 crypto rx=0x7f93ec005770 tx=0x7f93ec02c710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=557f64e6e776be12 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:32:30.902 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:30.899+0000 7f93e7400640 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-03-21T10:32:30.902 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:30.900+0000 7f93e7400640 1 -- 172.21.15.47:0/3215523126 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93ec056060 con 0x557f67f488b0 2024-03-21T10:32:30.902 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:30.900+0000 7f93e7400640 1 -- 172.21.15.47:0/3215523126 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(99..99 src has 1..99) v4 ==== 9562+0+0 (secure 0 0 0) 0x7f93ec003c30 con 0x557f67f488b0 2024-03-21T10:32:30.902 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:30.900+0000 7f940b17b740 1 -- 172.21.15.47:0/3215523126 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f93ec02c880 con 0x557f67f488b0 2024-03-21T10:32:30.902 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:30.900+0000 7f93e7400640 1 -- 172.21.15.47:0/3215523126 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f93ec05d050 con 0x557f67f488b0 2024-03-21T10:32:30.902 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:30.900+0000 7f940b17b740 10 client.17748 fetch_fsmap learned FSMap version 32 2024-03-21T10:32:30.902 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:30.900+0000 7f940b17b740 1 -- 172.21.15.47:0/3215523126 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x557f67c8a7d0 con 0x557f67f488b0 2024-03-21T10:32:30.902 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:30.900+0000 7f93e7400640 1 -- 172.21.15.47:0/3215523126 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f93ec02c880 con 0x557f67f488b0 2024-03-21T10:32:30.902 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:30.901+0000 7f93e7400640 1 -- 172.21.15.47:0/3215523126 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 32) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f93ec040030 con 0x557f67f488b0 2024-03-21T10:32:30.902 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:30.901+0000 7f940b17b740 10 client.17748 fetch_fsmap finished waiting for FSMap version 32 2024-03-21T10:32:30.902 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:30.901+0000 7f940b17b740 10 client.17748 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.a 2024-03-21T10:32:30.903 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:30.901+0000 7f940b17b740 20 client.17748 populate_metadata read hostname 'smithi047' 2024-03-21T10:32:30.903 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:30.901+0000 7f940b17b740 1 --2- 172.21.15.47:0/3215523126 >> [v2:172.21.15.47:6840/3401578978,v1:172.21.15.47:6841/3401578978] conn(0x557f67f61a60 0x557f67f63e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:32:30.903 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:30.901+0000 7f940b17b740 4 client.17748 mds_command: new command op to 14619 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T10:32:30.903 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:30.901+0000 7f940b17b740 1 -- 172.21.15.47:0/3215523126 --> [v2:172.21.15.47:6840/3401578978,v1:172.21.15.47:6841/3401578978] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x557f67eb9800 con 0x557f67f61a60 2024-03-21T10:32:30.903 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:30.901+0000 7f9405a00640 1 --2- 172.21.15.47:0/3215523126 >> [v2:172.21.15.47:6840/3401578978,v1:172.21.15.47:6841/3401578978] conn(0x557f67f61a60 0x557f67f63e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:32:30.904 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:30.902+0000 7f9405a00640 1 --2- 172.21.15.47:0/3215523126 >> [v2:172.21.15.47:6840/3401578978,v1:172.21.15.47:6841/3401578978] conn(0x557f67f61a60 0x557f67f63e50 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=326eab6135b0e411 server_cookie=86d7e5fe751c0ced in_seq=0 out_seq=0 2024-03-21T10:32:30.904 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:30.902+0000 7f93e7400640 10 client.17748 ms_handle_connect on v2:172.21.15.47:6840/3401578978 2024-03-21T10:32:30.904 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:30.902+0000 7f93e7400640 1 -- 172.21.15.47:0/3215523126 <== mds.4 v2:172.21.15.47:6840/3401578978 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x557f67eb9800 con 0x557f67f61a60 2024-03-21T10:32:30.904 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:30.902+0000 7f93e7400640 10 client.17748 handle_command_reply: tid=0 2024-03-21T10:32:30.904 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:30.903+0000 7f940b17b740 1 client.17748 shutdown 2024-03-21T10:32:30.904 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:30.903+0000 7f940b17b740 1 -- 172.21.15.47:0/3215523126 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x557f67f488b0 msgr2=0x557f67f51700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:32:30.904 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:30.903+0000 7f940b17b740 1 --2- 172.21.15.47:0/3215523126 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x557f67f488b0 0x557f67f51700 secure :-1 s=READY pgs=1020 cs=0 l=1 rev1=1 crypto rx=0x7f93ec005770 tx=0x7f93ec02c710 comp rx=0 tx=0).stop 2024-03-21T10:32:30.904 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:30.903+0000 7f940b17b740 1 -- 172.21.15.47:0/3215523126 shutdown_connections 2024-03-21T10:32:30.905 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:30.903+0000 7f940b17b740 1 --2- 172.21.15.47:0/3215523126 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557f67f51de0 0x557f67f4f030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:32:30.905 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:30.903+0000 7f940b17b740 1 --2- 172.21.15.47:0/3215523126 >> [v2:172.21.15.47:6840/3401578978,v1:172.21.15.47:6841/3401578978] conn(0x557f67f61a60 0x557f67f63e50 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:32:30.905 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:30.903+0000 7f940b17b740 1 --2- 172.21.15.47:0/3215523126 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557f67f49260 0x557f67f4ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:32:30.905 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:30.903+0000 7f940b17b740 1 --2- 172.21.15.47:0/3215523126 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x557f67f488b0 0x557f67f51700 unknown :-1 s=CLOSED pgs=1020 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:32:30.905 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:30.903+0000 7f940b17b740 1 -- 172.21.15.47:0/3215523126 >> 172.21.15.47:0/3215523126 conn(0x557f67f42f80 msgr2=0x557f67f44cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:32:30.905 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:30.903+0000 7f93e7400640 0 client.17748 ms_handle_reset on v2:172.21.15.47:6840/3401578978 2024-03-21T10:32:30.905 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:30.903+0000 7f940b17b740 1 -- 172.21.15.47:0/3215523126 shutdown_connections 2024-03-21T10:32:30.905 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:30.903+0000 7f940b17b740 1 -- 172.21.15.47:0/3215523126 wait complete. 2024-03-21T10:32:30.905 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:30.904+0000 7f940b17b740 20 client.17748 trim_cache size 0 max 16384 2024-03-21T10:32:30.918 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:30.918+0000 7f940b17b740 1 Processor -- start 2024-03-21T10:32:30.918 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:30.918+0000 7f940b17b740 1 -- start start 2024-03-21T10:32:30.918 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:30.918+0000 7f940b17b740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557f67f488b0 0x557f67f44ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:32:30.918 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:30.918+0000 7f940b17b740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x557f67f49260 0x557f67f511c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:32:30.918 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:30.918+0000 7f940b17b740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557f67f51de0 0x557f67f51700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:32:30.918 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:30.918+0000 7f940b17b740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94001b09f0 con 0x557f67f49260 2024-03-21T10:32:30.918 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:30.918+0000 7f940b17b740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94001b2370 con 0x557f67f488b0 2024-03-21T10:32:30.918 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:30.918+0000 7f940b17b740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94001b1050 con 0x557f67f51de0 2024-03-21T10:32:30.918 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:30.919+0000 7f9405a00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x557f67f49260 0x557f67f511c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:32:30.919 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:30.919+0000 7f9405000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557f67f488b0 0x557f67f44ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:32:30.919 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:30.919+0000 7f9405a00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x557f67f49260 0x557f67f511c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:51326/0 (socket says 172.21.15.47:51326) 2024-03-21T10:32:30.919 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:30.919+0000 7f9405a00640 1 -- 172.21.15.47:0/2584059033 learned_addr learned my addr 172.21.15.47:0/2584059033 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:32:30.920 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:30.919+0000 7f9405a00640 1 -- 172.21.15.47:0/2584059033 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557f67f51de0 msgr2=0x557f67f51700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:32:30.920 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:30.919+0000 7f9405a00640 1 --2- 172.21.15.47:0/2584059033 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557f67f51de0 0x557f67f51700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:32:30.920 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:30.919+0000 7f9405a00640 1 -- 172.21.15.47:0/2584059033 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557f67f488b0 msgr2=0x557f67f44ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:32:30.920 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:30.919+0000 7f9405a00640 1 --2- 172.21.15.47:0/2584059033 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557f67f488b0 0x557f67f44ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:32:30.920 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:30.919+0000 7f9405a00640 1 -- 172.21.15.47:0/2584059033 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557f67f4f3a0 con 0x557f67f49260 2024-03-21T10:32:30.921 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:30.919+0000 7f9405000640 1 --2- 172.21.15.47:0/2584059033 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557f67f488b0 0x557f67f44ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T10:32:30.921 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:30.919+0000 7f9405a00640 1 --2- 172.21.15.47:0/2584059033 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x557f67f49260 0x557f67f511c0 secure :-1 s=READY pgs=1021 cs=0 l=1 rev1=1 crypto rx=0x7f93f4005d00 tx=0x7f93f40045b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4abb164dd67fafcd server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:32:30.921 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:30.920+0000 7f93f2000640 1 -- 172.21.15.47:0/2584059033 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93f4015070 con 0x557f67f49260 2024-03-21T10:32:30.921 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:30.920+0000 7f93f2000640 1 -- 172.21.15.47:0/2584059033 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f93f401ddb0 con 0x557f67f49260 2024-03-21T10:32:30.921 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:30.920+0000 7f93f2000640 1 -- 172.21.15.47:0/2584059033 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93f4004da0 con 0x557f67f49260 2024-03-21T10:32:30.921 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:30.920+0000 7f940b17b740 1 -- 172.21.15.47:0/2584059033 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x557f67f49260 msgr2=0x557f67f511c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:32:30.921 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:30.920+0000 7f940b17b740 1 --2- 172.21.15.47:0/2584059033 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x557f67f49260 0x557f67f511c0 secure :-1 s=READY pgs=1021 cs=0 l=1 rev1=1 crypto rx=0x7f93f4005d00 tx=0x7f93f40045b0 comp rx=0 tx=0).stop 2024-03-21T10:32:30.921 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:30.920+0000 7f940b17b740 1 -- 172.21.15.47:0/2584059033 shutdown_connections 2024-03-21T10:32:30.921 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:30.920+0000 7f940b17b740 1 --2- 172.21.15.47:0/2584059033 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557f67f51de0 0x557f67f51700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:32:30.921 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:30.920+0000 7f940b17b740 1 --2- 172.21.15.47:0/2584059033 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x557f67f49260 0x557f67f511c0 unknown :-1 s=CLOSED pgs=1021 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:32:30.921 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:30.920+0000 7f940b17b740 1 --2- 172.21.15.47:0/2584059033 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557f67f488b0 0x557f67f44ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:32:30.921 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:30.920+0000 7f940b17b740 1 -- 172.21.15.47:0/2584059033 >> 172.21.15.47:0/2584059033 conn(0x557f67f42f80 msgr2=0x557f67f443d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:32:30.921 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:30.921+0000 7f940b17b740 1 -- 172.21.15.47:0/2584059033 shutdown_connections 2024-03-21T10:32:30.921 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:30.921+0000 7f940b17b740 1 -- 172.21.15.47:0/2584059033 wait complete. 2024-03-21T10:32:30.921 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:30.921+0000 7f940b17b740 1 -- start start 2024-03-21T10:32:30.923 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:30.921+0000 7f940b17b740 1 Processor -- start 2024-03-21T10:32:30.923 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:30.922+0000 7f940b17b740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557f67f488b0 0x557f67f64290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:32:30.923 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:30.922+0000 7f940b17b740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557f67f49260 0x557f67f647d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:32:30.924 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:30.922+0000 7f940b17b740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x557f67f51de0 0x557f67fbbf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:32:30.924 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:30.922+0000 7f940b17b740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94001b1d10 con 0x557f67f51de0 2024-03-21T10:32:30.924 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:30.922+0000 7f940b17b740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94001b16b0 con 0x557f67f488b0 2024-03-21T10:32:30.924 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:30.922+0000 7f940b17b740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94001aebd0 con 0x557f67f49260 2024-03-21T10:32:30.924 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:30.922+0000 7f940b17b740 10 client.0 _ll_register_callbacks cb 0x7f93f80093a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T10:32:30.924 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:30.922+0000 7f9405000640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x557f67f51de0 0x557f67fbbf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:32:30.924 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:30.922+0000 7f9405000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557f67f488b0 0x557f67f64290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:32:30.924 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:30.922+0000 7f9405000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557f67f488b0 0x557f67f64290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:33698/0 (socket says 172.21.15.47:33698) 2024-03-21T10:32:30.924 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:30.922+0000 7f9405000640 1 -- 172.21.15.47:0/4009075314 learned_addr learned my addr 172.21.15.47:0/4009075314 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:32:30.924 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:30.922+0000 7f9405a00640 1 --2- 172.21.15.47:0/4009075314 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557f67f49260 0x557f67f647d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:32:30.924 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:30.922+0000 7f9405000640 1 -- 172.21.15.47:0/4009075314 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557f67f49260 msgr2=0x557f67f647d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:32:30.924 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:30.923+0000 7f9405000640 1 --2- 172.21.15.47:0/4009075314 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557f67f49260 0x557f67f647d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:32:30.924 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:30.923+0000 7f9405000640 1 -- 172.21.15.47:0/4009075314 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x557f67f51de0 msgr2=0x557f67fbbf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:32:30.924 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:30.923+0000 7f9405000640 1 --2- 172.21.15.47:0/4009075314 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x557f67f51de0 0x557f67fbbf90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:32:30.924 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:30.923+0000 7f9405000640 1 -- 172.21.15.47:0/4009075314 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x557f67fbc4d0 con 0x557f67f488b0 2024-03-21T10:32:30.925 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:30.923+0000 7f9405000640 1 --2- 172.21.15.47:0/4009075314 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557f67f488b0 0x557f67f64290 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f93ec001eb0 tx=0x7f93ec02e040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5e37c918c8095378 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:32:30.925 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:30.923+0000 7f93e7400640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-21T10:32:30.925 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:30.923+0000 7f93e7400640 1 -- 172.21.15.47:0/4009075314 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93ec056040 con 0x557f67f488b0 2024-03-21T10:32:30.925 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:30.923+0000 7f940b17b740 1 -- 172.21.15.47:0/4009075314 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f93f401ddb0 con 0x557f67f488b0 2024-03-21T10:32:30.925 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:30.923+0000 7f93e7400640 1 -- 172.21.15.47:0/4009075314 <== mon.2 v2:172.21.15.100:3300/0 2 ==== osd_map(99..99 src has 1..99) v4 ==== 9562+0+0 (secure 0 0 0) 0x7f93ec04cce0 con 0x557f67f488b0 2024-03-21T10:32:30.925 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:30.924+0000 7f93e7400640 1 -- 172.21.15.47:0/4009075314 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f93ec06f050 con 0x557f67f488b0 2024-03-21T10:32:30.925 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:30.924+0000 7f940b17b740 10 client.37538 fetch_fsmap learned FSMap version 32 2024-03-21T10:32:30.925 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:30.924+0000 7f940b17b740 1 -- 172.21.15.47:0/4009075314 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x557f67fbd670 con 0x557f67f488b0 2024-03-21T10:32:30.927 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:30.924+0000 7f93e7400640 1 -- 172.21.15.47:0/4009075314 <== mon.2 v2:172.21.15.100:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f93f401ddb0 con 0x557f67f488b0 2024-03-21T10:32:30.927 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:30.925+0000 7f93e7400640 1 -- 172.21.15.47:0/4009075314 <== mon.2 v2:172.21.15.100:3300/0 5 ==== fsmap(e 32) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f93ec040040 con 0x557f67f488b0 2024-03-21T10:32:30.927 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:30.925+0000 7f940b17b740 10 client.37538 fetch_fsmap finished waiting for FSMap version 32 2024-03-21T10:32:30.927 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:30.925+0000 7f940b17b740 10 client.37538 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.a 2024-03-21T10:32:30.927 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:30.925+0000 7f940b17b740 20 client.37538 populate_metadata read hostname 'smithi047' 2024-03-21T10:32:30.927 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:30.925+0000 7f940b17b740 1 --2- 172.21.15.47:0/4009075314 >> [v2:172.21.15.47:6840/3401578978,v1:172.21.15.47:6841/3401578978] conn(0x557f67fbdc80 0x557f67fc00a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:32:30.927 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:30.925+0000 7f940b17b740 4 client.37538 mds_command: new command op to 14619 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-21T10:32:30.928 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:30.925+0000 7f940b17b740 1 -- 172.21.15.47:0/4009075314 --> [v2:172.21.15.47:6840/3401578978,v1:172.21.15.47:6841/3401578978] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x557f67eb3b50 con 0x557f67fbdc80 2024-03-21T10:32:30.928 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:30.925+0000 7f9405a00640 1 --2- 172.21.15.47:0/4009075314 >> [v2:172.21.15.47:6840/3401578978,v1:172.21.15.47:6841/3401578978] conn(0x557f67fbdc80 0x557f67fc00a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:32:30.928 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:30.926+0000 7f9405a00640 1 --2- 172.21.15.47:0/4009075314 >> [v2:172.21.15.47:6840/3401578978,v1:172.21.15.47:6841/3401578978] conn(0x557f67fbdc80 0x557f67fc00a0 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=f73b36e8c43aea16 server_cookie=6cdcbaf26b79d982 in_seq=0 out_seq=0 2024-03-21T10:32:30.928 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:30.926+0000 7f93e7400640 10 client.37538 ms_handle_connect on v2:172.21.15.47:6840/3401578978 2024-03-21T10:32:30.928 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:30.926+0000 7f93e7400640 1 -- 172.21.15.47:0/4009075314 <== mds.4 v2:172.21.15.47:6840/3401578978 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x557f67eb3b50 con 0x557f67fbdc80 2024-03-21T10:32:30.928 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:30.926+0000 7f93e7400640 10 client.37538 handle_command_reply: tid=0 2024-03-21T10:32:30.928 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:30.926+0000 7f940b17b740 1 client.37538 shutdown 2024-03-21T10:32:30.928 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:30.926+0000 7f940b17b740 1 -- 172.21.15.47:0/4009075314 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557f67f488b0 msgr2=0x557f67f64290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:32:30.928 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:30.926+0000 7f940b17b740 1 --2- 172.21.15.47:0/4009075314 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557f67f488b0 0x557f67f64290 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f93ec001eb0 tx=0x7f93ec02e040 comp rx=0 tx=0).stop 2024-03-21T10:32:30.928 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:30.926+0000 7f940b17b740 1 -- 172.21.15.47:0/4009075314 shutdown_connections 2024-03-21T10:32:30.928 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:30.926+0000 7f940b17b740 1 --2- 172.21.15.47:0/4009075314 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x557f67f51de0 0x557f67fbbf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:32:30.928 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:30.926+0000 7f940b17b740 1 --2- 172.21.15.47:0/4009075314 >> [v2:172.21.15.47:6840/3401578978,v1:172.21.15.47:6841/3401578978] conn(0x557f67fbdc80 0x557f67fc00a0 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:32:30.928 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:30.926+0000 7f940b17b740 1 --2- 172.21.15.47:0/4009075314 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x557f67f49260 0x557f67f647d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:32:30.928 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:30.926+0000 7f940b17b740 1 --2- 172.21.15.47:0/4009075314 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557f67f488b0 0x557f67f64290 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:32:30.929 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:30.926+0000 7f940b17b740 1 -- 172.21.15.47:0/4009075314 >> 172.21.15.47:0/4009075314 conn(0x557f67f42f80 msgr2=0x557f67f42160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:32:30.929 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:30.926+0000 7f93e7400640 0 client.37538 ms_handle_reset on v2:172.21.15.47:6840/3401578978 2024-03-21T10:32:30.929 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:30.927+0000 7f940b17b740 1 -- 172.21.15.47:0/4009075314 shutdown_connections 2024-03-21T10:32:30.929 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:30.927+0000 7f940b17b740 1 -- 172.21.15.47:0/4009075314 wait complete. 2024-03-21T10:32:30.929 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:30.927+0000 7f940b17b740 20 client.37538 trim_cache size 0 max 16384 2024-03-21T10:32:30.929 INFO:teuthology.orchestra.run.smithi047.stdout:[] 2024-03-21T10:32:30.930 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:30.927+0000 7f9406e00640 1 -- 172.21.15.47:0/2515679930 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f93e80a3110 msgr2=0x7f93e80a55d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:32:30.930 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:30.927+0000 7f9406e00640 1 --2- 172.21.15.47:0/2515679930 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f93e80a3110 0x7f93e80a55d0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f93f80cd5e0 tx=0x7f93f800c950 comp rx=0 tx=0).stop 2024-03-21T10:32:30.930 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:30.927+0000 7f9406e00640 1 -- 172.21.15.47:0/2515679930 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f940010baf0 msgr2=0x7f9400100da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:32:30.930 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:30.927+0000 7f9406e00640 1 --2- 172.21.15.47:0/2515679930 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f940010baf0 0x7f9400100da0 secure :-1 s=READY pgs=1018 cs=0 l=1 rev1=1 crypto rx=0x7f93f800eb10 tx=0x7f93f800c2d0 comp rx=0 tx=0).stop 2024-03-21T10:32:30.930 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:30.928+0000 7f9406e00640 1 -- 172.21.15.47:0/2515679930 shutdown_connections 2024-03-21T10:32:30.930 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:30.928+0000 7f9406e00640 1 --2- 172.21.15.47:0/2515679930 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f940010baf0 0x7f9400100da0 unknown :-1 s=CLOSED pgs=1018 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:32:30.930 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:30.928+0000 7f9406e00640 1 --2- 172.21.15.47:0/2515679930 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9400102ff0 0x7f9400100860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:32:30.930 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:30.928+0000 7f9406e00640 1 --2- 172.21.15.47:0/2515679930 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f93e80a3110 0x7f93e80a55d0 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:32:30.930 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:30.928+0000 7f9406e00640 1 --2- 172.21.15.47:0/2515679930 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f94001026d0 0x7f9400100320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:32:30.930 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:30.928+0000 7f9406e00640 1 -- 172.21.15.47:0/2515679930 >> 172.21.15.47:0/2515679930 conn(0x7f94000fbef0 msgr2=0x7f94000fcd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:32:30.931 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:30.928+0000 7f9406e00640 1 -- 172.21.15.47:0/2515679930 shutdown_connections 2024-03-21T10:32:30.931 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:30.928+0000 7f9406e00640 1 -- 172.21.15.47:0/2515679930 wait complete. 2024-03-21T10:32:30.946 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-21T10:32:30.946 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-03-21T10:32:31.157 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.156+0000 7fef81000640 1 -- 172.21.15.47:0/2342850176 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fef7c104200 msgr2=0x7fef7c110710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:32:31.157 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.156+0000 7fef81000640 1 --2- 172.21.15.47:0/2342850176 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fef7c104200 0x7fef7c110710 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fef74009ca0 tx=0x7fef7402c0f0 comp rx=0 tx=0).stop 2024-03-21T10:32:31.157 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.157+0000 7fef81000640 1 -- 172.21.15.47:0/2342850176 shutdown_connections 2024-03-21T10:32:31.157 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.157+0000 7fef81000640 1 --2- 172.21.15.47:0/2342850176 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fef7c104200 0x7fef7c110710 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:32:31.157 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.157+0000 7fef81000640 1 --2- 172.21.15.47:0/2342850176 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fef7c103840 0x7fef7c103cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:32:31.157 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.157+0000 7fef81000640 1 --2- 172.21.15.47:0/2342850176 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fef7c109850 0x7fef7c109c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:32:31.157 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.157+0000 7fef81000640 1 -- 172.21.15.47:0/2342850176 >> 172.21.15.47:0/2342850176 conn(0x7fef7c0ff310 msgr2=0x7fef7c101730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:32:31.157 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.157+0000 7fef81000640 1 -- 172.21.15.47:0/2342850176 shutdown_connections 2024-03-21T10:32:31.157 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.157+0000 7fef81000640 1 -- 172.21.15.47:0/2342850176 wait complete. 2024-03-21T10:32:31.158 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.158+0000 7fef81000640 1 Processor -- start 2024-03-21T10:32:31.158 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.158+0000 7fef81000640 1 -- start start 2024-03-21T10:32:31.159 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.159+0000 7fef81000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fef7c103840 0x7fef7c19a030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:32:31.159 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.159+0000 7fef81000640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fef7c104200 0x7fef7c19a570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:32:31.159 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.159+0000 7fef81000640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fef7c109850 0x7fef7c19e8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:32:31.159 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.159+0000 7fef81000640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef7c113b10 con 0x7fef7c104200 2024-03-21T10:32:31.159 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.159+0000 7fef81000640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef7c113990 con 0x7fef7c103840 2024-03-21T10:32:31.159 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.159+0000 7fef81000640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef7c113c90 con 0x7fef7c109850 2024-03-21T10:32:31.159 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.159+0000 7fef79e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fef7c103840 0x7fef7c19a030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:32:31.159 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.159+0000 7fef73e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fef7c104200 0x7fef7c19a570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:32:31.159 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.159+0000 7fef79e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fef7c103840 0x7fef7c19a030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:33706/0 (socket says 172.21.15.47:33706) 2024-03-21T10:32:31.159 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.159+0000 7fef79e00640 1 -- 172.21.15.47:0/1245453885 learned_addr learned my addr 172.21.15.47:0/1245453885 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:32:31.160 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.159+0000 7fef7a800640 1 --2- 172.21.15.47:0/1245453885 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fef7c109850 0x7fef7c19e8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:32:31.161 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.159+0000 7fef79e00640 1 -- 172.21.15.47:0/1245453885 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fef7c109850 msgr2=0x7fef7c19e8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:32:31.161 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.159+0000 7fef79e00640 1 --2- 172.21.15.47:0/1245453885 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fef7c109850 0x7fef7c19e8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:32:31.161 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.159+0000 7fef79e00640 1 -- 172.21.15.47:0/1245453885 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fef7c104200 msgr2=0x7fef7c19a570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:32:31.161 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.159+0000 7fef79e00640 1 --2- 172.21.15.47:0/1245453885 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fef7c104200 0x7fef7c19a570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:32:31.161 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.159+0000 7fef79e00640 1 -- 172.21.15.47:0/1245453885 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef7c19f020 con 0x7fef7c103840 2024-03-21T10:32:31.161 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.160+0000 7fef79e00640 1 --2- 172.21.15.47:0/1245453885 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fef7c103840 0x7fef7c19a030 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fef6c00ea60 tx=0x7fef6c00ef20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:32:31.161 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.160+0000 7fef72a00640 1 -- 172.21.15.47:0/1245453885 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef6c00cde0 con 0x7fef7c103840 2024-03-21T10:32:31.161 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.160+0000 7fef72a00640 1 -- 172.21.15.47:0/1245453885 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fef6c0035f0 con 0x7fef7c103840 2024-03-21T10:32:31.161 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.160+0000 7fef81000640 1 -- 172.21.15.47:0/1245453885 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fef7c19f310 con 0x7fef7c103840 2024-03-21T10:32:31.161 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.160+0000 7fef72a00640 1 -- 172.21.15.47:0/1245453885 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef6c010690 con 0x7fef7c103840 2024-03-21T10:32:31.161 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.160+0000 7fef81000640 1 -- 172.21.15.47:0/1245453885 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fef7c1a6ba0 con 0x7fef7c103840 2024-03-21T10:32:31.163 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.164+0000 7fef83e12740 1 Processor -- start 2024-03-21T10:32:31.163 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.164+0000 7fef83e12740 1 -- start start 2024-03-21T10:32:31.164 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.164+0000 7fef83e12740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564a0573e6d0 0x564a0573eab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:32:31.165 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.164+0000 7fef83e12740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564a0573f080 0x564a05747630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:32:31.165 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.164+0000 7fef83e12740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x564a05747c00 0x564a0574a050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:32:31.165 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.164+0000 7fef83e12740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x564a056a22c0 con 0x564a05747c00 2024-03-21T10:32:31.165 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.164+0000 7fef83e12740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x564a0564ac90 con 0x564a0573e6d0 2024-03-21T10:32:31.165 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.164+0000 7fef83e12740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x564a0569e360 con 0x564a0573f080 2024-03-21T10:32:31.165 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.165+0000 7fef73e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x564a05747c00 0x564a0574a050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:32:31.165 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.165+0000 7fef7a800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564a0573e6d0 0x564a0573eab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:32:31.165 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.165+0000 7fef7a800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564a0573e6d0 0x564a0573eab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:33712/0 (socket says 172.21.15.47:33712) 2024-03-21T10:32:31.165 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.165+0000 7fef7a800640 1 -- 172.21.15.47:0/3039288534 learned_addr learned my addr 172.21.15.47:0/3039288534 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:32:31.165 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.165+0000 7fef73e00640 1 -- 172.21.15.47:0/3039288534 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564a0573f080 msgr2=0x564a05747630 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-21T10:32:31.165 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.165+0000 7fef73e00640 1 --2- 172.21.15.47:0/3039288534 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564a0573f080 0x564a05747630 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:32:31.165 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.165+0000 7fef73e00640 1 -- 172.21.15.47:0/3039288534 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564a0573e6d0 msgr2=0x564a0573eab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:32:31.165 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.165+0000 7fef73e00640 1 --2- 172.21.15.47:0/3039288534 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564a0573e6d0 0x564a0573eab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:32:31.166 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.165+0000 7fef73e00640 1 -- 172.21.15.47:0/3039288534 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564a0566f080 con 0x564a05747c00 2024-03-21T10:32:31.168 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.165+0000 7fef73e00640 1 --2- 172.21.15.47:0/3039288534 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x564a05747c00 0x564a0574a050 secure :-1 s=READY pgs=1022 cs=0 l=1 rev1=1 crypto rx=0x7fef64009bd0 tx=0x7fef64011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a94ca7f5aa1de850 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:32:31.168 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.165+0000 7fef72a00640 1 -- 172.21.15.47:0/1245453885 <== mon.2 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 27) v1 ==== 137985+0+0 (secure 0 0 0) 0x7fef6c0037a0 con 0x7fef7c103840 2024-03-21T10:32:31.168 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.166+0000 7fef6b400640 1 -- 172.21.15.47:0/3039288534 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef64004230 con 0x564a05747c00 2024-03-21T10:32:31.168 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.166+0000 7fef6b400640 1 -- 172.21.15.47:0/3039288534 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fef64009e40 con 0x564a05747c00 2024-03-21T10:32:31.168 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.166+0000 7fef6b400640 1 -- 172.21.15.47:0/3039288534 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef64004d10 con 0x564a05747c00 2024-03-21T10:32:31.168 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.166+0000 7fef83e12740 1 -- 172.21.15.47:0/3039288534 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x564a05747c00 msgr2=0x564a0574a050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:32:31.169 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.166+0000 7fef83e12740 1 --2- 172.21.15.47:0/3039288534 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x564a05747c00 0x564a0574a050 secure :-1 s=READY pgs=1022 cs=0 l=1 rev1=1 crypto rx=0x7fef64009bd0 tx=0x7fef64011810 comp rx=0 tx=0).stop 2024-03-21T10:32:31.169 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.167+0000 7fef83e12740 1 -- 172.21.15.47:0/3039288534 shutdown_connections 2024-03-21T10:32:31.169 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.167+0000 7fef83e12740 1 --2- 172.21.15.47:0/3039288534 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x564a05747c00 0x564a0574a050 unknown :-1 s=CLOSED pgs=1022 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:32:31.169 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.167+0000 7fef83e12740 1 --2- 172.21.15.47:0/3039288534 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564a0573f080 0x564a05747630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:32:31.169 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.167+0000 7fef83e12740 1 --2- 172.21.15.47:0/3039288534 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564a0573e6d0 0x564a0573eab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:32:31.169 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.167+0000 7fef83e12740 1 -- 172.21.15.47:0/3039288534 >> 172.21.15.47:0/3039288534 conn(0x564a05738da0 msgr2=0x564a05739180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:32:31.169 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.167+0000 7fef83e12740 1 -- 172.21.15.47:0/3039288534 shutdown_connections 2024-03-21T10:32:31.169 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.167+0000 7fef83e12740 1 -- 172.21.15.47:0/3039288534 wait complete. 2024-03-21T10:32:31.169 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.167+0000 7fef72a00640 1 --2- 172.21.15.47:0/1245453885 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7fef600a2fb0 0x7fef600a5470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:32:31.169 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.167+0000 7fef83e12740 1 -- start start 2024-03-21T10:32:31.169 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.167+0000 7fef72a00640 1 -- 172.21.15.47:0/1245453885 <== mon.2 v2:172.21.15.100:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 9562+0+0 (secure 0 0 0) 0x7fef6c0cfee0 con 0x7fef7c103840 2024-03-21T10:32:31.169 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.167+0000 7fef7a800640 1 --2- 172.21.15.47:0/1245453885 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7fef600a2fb0 0x7fef600a5470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:32:31.169 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.168+0000 7fef83e12740 1 Processor -- start 2024-03-21T10:32:31.170 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.168+0000 7fef83e12740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x564a0573e6d0 0x564a05747540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:32:31.170 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.168+0000 7fef83e12740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564a0573f080 0x564a057448d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:32:31.170 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.168+0000 7fef83e12740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564a05747c00 0x564a05744ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:32:31.170 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.168+0000 7fef83e12740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x564a0574ab70 con 0x564a0573e6d0 2024-03-21T10:32:31.170 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.168+0000 7fef83e12740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x564a0565e0d0 con 0x564a05747c00 2024-03-21T10:32:31.170 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.168+0000 7fef83e12740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x564a0574acf0 con 0x564a0573f080 2024-03-21T10:32:31.170 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.168+0000 7fef7a800640 1 --2- 172.21.15.47:0/1245453885 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7fef600a2fb0 0x7fef600a5470 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fef74009bf0 tx=0x7fef74004a40 comp rx=0 tx=0).ready entity=mgr.15687 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:32:31.170 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.168+0000 7fef83e12740 10 client.0 _ll_register_callbacks cb 0x564a05662e80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T10:32:31.170 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.168+0000 7fef79e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x564a0573e6d0 0x564a05747540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:32:31.170 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.169+0000 7fef79e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x564a0573e6d0 0x564a05747540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:51374/0 (socket says 172.21.15.47:51374) 2024-03-21T10:32:31.170 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.169+0000 7fef79e00640 1 -- 172.21.15.47:0/3575735629 learned_addr learned my addr 172.21.15.47:0/3575735629 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:32:31.170 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.169+0000 7fef7a800640 1 --2- 172.21.15.47:0/3575735629 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564a05747c00 0x564a05744ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:32:31.170 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.169+0000 7fef73e00640 1 --2- 172.21.15.47:0/3575735629 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564a0573f080 0x564a057448d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:32:31.173 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.169+0000 7fef79e00640 1 -- 172.21.15.47:0/3575735629 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564a0573f080 msgr2=0x564a057448d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:32:31.173 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.169+0000 7fef79e00640 1 --2- 172.21.15.47:0/3575735629 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564a0573f080 0x564a057448d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:32:31.173 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.169+0000 7fef79e00640 1 -- 172.21.15.47:0/3575735629 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564a05747c00 msgr2=0x564a05744ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:32:31.173 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.169+0000 7fef79e00640 1 --2- 172.21.15.47:0/3575735629 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564a05747c00 0x564a05744ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:32:31.174 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.169+0000 7fef79e00640 1 -- 172.21.15.47:0/3575735629 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x564a053b63f0 con 0x564a0573e6d0 2024-03-21T10:32:31.174 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.169+0000 7fef79e00640 1 --2- 172.21.15.47:0/3575735629 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x564a0573e6d0 0x564a05747540 secure :-1 s=READY pgs=1023 cs=0 l=1 rev1=1 crypto rx=0x7fef6c00c760 tx=0x7fef6c09ba20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=69ecb3813e489b2f server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:32:31.174 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.169+0000 7fef5fe00640 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-03-21T10:32:31.174 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.170+0000 7fef5fe00640 1 -- 172.21.15.47:0/3575735629 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef6c0b4020 con 0x564a0573e6d0 2024-03-21T10:32:31.174 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.170+0000 7fef5fe00640 1 -- 172.21.15.47:0/3575735629 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(99..99 src has 1..99) v4 ==== 9562+0+0 (secure 0 0 0) 0x7fef6c0984d0 con 0x564a0573e6d0 2024-03-21T10:32:31.174 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.170+0000 7fef83e12740 1 -- 172.21.15.47:0/3575735629 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x564a056af800 con 0x564a0573e6d0 2024-03-21T10:32:31.174 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.170+0000 7fef5fe00640 1 -- 172.21.15.47:0/3575735629 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7fef6c09c560 con 0x564a0573e6d0 2024-03-21T10:32:31.174 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.170+0000 7fef83e12740 10 client.17766 fetch_fsmap learned FSMap version 32 2024-03-21T10:32:31.174 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.170+0000 7fef83e12740 1 -- 172.21.15.47:0/3575735629 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x564a054807d0 con 0x564a0573e6d0 2024-03-21T10:32:31.174 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.170+0000 7fef5fe00640 1 -- 172.21.15.47:0/3575735629 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x564a056af800 con 0x564a0573e6d0 2024-03-21T10:32:31.174 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.170+0000 7fef5fe00640 1 -- 172.21.15.47:0/3575735629 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 32) v1 ==== 7942+0+0 (secure 0 0 0) 0x7fef6c0babd0 con 0x564a0573e6d0 2024-03-21T10:32:31.174 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.171+0000 7fef83e12740 10 client.17766 fetch_fsmap finished waiting for FSMap version 32 2024-03-21T10:32:31.174 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.171+0000 7fef83e12740 10 client.17766 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.k 2024-03-21T10:32:31.175 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.171+0000 7fef83e12740 20 client.17766 populate_metadata read hostname 'smithi047' 2024-03-21T10:32:31.175 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.171+0000 7fef83e12740 1 --2- 172.21.15.47:0/3575735629 >> [v2:172.21.15.100:6832/2723302038,v1:172.21.15.100:6833/2723302038] conn(0x564a05757aa0 0x564a05759e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:32:31.175 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.171+0000 7fef83e12740 4 client.17766 mds_command: new command op to 34427 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T10:32:31.175 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.171+0000 7fef83e12740 1 -- 172.21.15.47:0/3575735629 --> [v2:172.21.15.100:6832/2723302038,v1:172.21.15.100:6833/2723302038] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x564a056a9b50 con 0x564a05757aa0 2024-03-21T10:32:31.175 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.171+0000 7fef73e00640 1 --2- 172.21.15.47:0/3575735629 >> [v2:172.21.15.100:6832/2723302038,v1:172.21.15.100:6833/2723302038] conn(0x564a05757aa0 0x564a05759e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:32:31.175 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.172+0000 7fef73e00640 1 --2- 172.21.15.47:0/3575735629 >> [v2:172.21.15.100:6832/2723302038,v1:172.21.15.100:6833/2723302038] conn(0x564a05757aa0 0x564a05759e90 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=661cf2acd48e1c59 server_cookie=99aac932d055aae9 in_seq=0 out_seq=0 2024-03-21T10:32:31.175 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.172+0000 7fef5fe00640 10 client.17766 ms_handle_connect on v2:172.21.15.100:6832/2723302038 2024-03-21T10:32:31.175 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.173+0000 7fef5fe00640 1 -- 172.21.15.47:0/3575735629 <== mds.3 v2:172.21.15.100:6832/2723302038 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x564a056a9b50 con 0x564a05757aa0 2024-03-21T10:32:31.175 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.173+0000 7fef5fe00640 10 client.17766 handle_command_reply: tid=0 2024-03-21T10:32:31.175 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.173+0000 7fef83e12740 1 client.17766 shutdown 2024-03-21T10:32:31.175 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.173+0000 7fef83e12740 1 -- 172.21.15.47:0/3575735629 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x564a0573e6d0 msgr2=0x564a05747540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:32:31.175 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.173+0000 7fef83e12740 1 --2- 172.21.15.47:0/3575735629 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x564a0573e6d0 0x564a05747540 secure :-1 s=READY pgs=1023 cs=0 l=1 rev1=1 crypto rx=0x7fef6c00c760 tx=0x7fef6c09ba20 comp rx=0 tx=0).stop 2024-03-21T10:32:31.175 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.173+0000 7fef83e12740 1 -- 172.21.15.47:0/3575735629 shutdown_connections 2024-03-21T10:32:31.175 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.173+0000 7fef83e12740 1 --2- 172.21.15.47:0/3575735629 >> [v2:172.21.15.100:6832/2723302038,v1:172.21.15.100:6833/2723302038] conn(0x564a05757aa0 0x564a05759e90 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:32:31.175 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.173+0000 7fef83e12740 1 --2- 172.21.15.47:0/3575735629 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564a05747c00 0x564a05744ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:32:31.176 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.173+0000 7fef83e12740 1 --2- 172.21.15.47:0/3575735629 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564a0573f080 0x564a057448d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:32:31.176 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.173+0000 7fef83e12740 1 --2- 172.21.15.47:0/3575735629 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x564a0573e6d0 0x564a05747540 unknown :-1 s=CLOSED pgs=1023 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:32:31.176 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.173+0000 7fef83e12740 1 -- 172.21.15.47:0/3575735629 >> 172.21.15.47:0/3575735629 conn(0x564a05738da0 msgr2=0x564a0573aaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:32:31.176 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.173+0000 7fef5fe00640 0 client.17766 ms_handle_reset on v2:172.21.15.100:6832/2723302038 2024-03-21T10:32:31.176 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.173+0000 7fef83e12740 1 -- 172.21.15.47:0/3575735629 shutdown_connections 2024-03-21T10:32:31.176 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.173+0000 7fef83e12740 1 -- 172.21.15.47:0/3575735629 wait complete. 2024-03-21T10:32:31.176 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.173+0000 7fef83e12740 20 client.17766 trim_cache size 0 max 16384 2024-03-21T10:32:31.187 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.187+0000 7fef83e12740 1 Processor -- start 2024-03-21T10:32:31.187 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.187+0000 7fef83e12740 1 -- start start 2024-03-21T10:32:31.187 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.187+0000 7fef83e12740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x564a0573e6d0 0x564a0573a8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:32:31.187 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.187+0000 7fef83e12740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564a0573f080 0x564a05747000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:32:31.187 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.187+0000 7fef83e12740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564a05747c00 0x564a05747540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:32:31.187 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.187+0000 7fef83e12740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef7c1b0c40 con 0x564a0573e6d0 2024-03-21T10:32:31.188 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.187+0000 7fef83e12740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef7c1b25c0 con 0x564a0573f080 2024-03-21T10:32:31.188 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.187+0000 7fef83e12740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef7c1b12a0 con 0x564a05747c00 2024-03-21T10:32:31.188 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.188+0000 7fef79e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x564a0573e6d0 0x564a0573a8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:32:31.188 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.188+0000 7fef73e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564a0573f080 0x564a05747000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:32:31.188 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.188+0000 7fef79e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x564a0573e6d0 0x564a0573a8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:51386/0 (socket says 172.21.15.47:51386) 2024-03-21T10:32:31.188 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.188+0000 7fef73e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564a0573f080 0x564a05747000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:33730/0 (socket says 172.21.15.47:33730) 2024-03-21T10:32:31.188 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.188+0000 7fef79e00640 1 -- 172.21.15.47:0/2496644484 learned_addr learned my addr 172.21.15.47:0/2496644484 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:32:31.190 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.188+0000 7fef7a800640 1 --2- 172.21.15.47:0/2496644484 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564a05747c00 0x564a05747540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:32:31.190 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.188+0000 7fef73e00640 1 -- 172.21.15.47:0/2496644484 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564a05747c00 msgr2=0x564a05747540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:32:31.190 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.188+0000 7fef73e00640 1 --2- 172.21.15.47:0/2496644484 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564a05747c00 0x564a05747540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:32:31.190 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.188+0000 7fef73e00640 1 -- 172.21.15.47:0/2496644484 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x564a0573e6d0 msgr2=0x564a0573a8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:32:31.190 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.188+0000 7fef73e00640 1 --2- 172.21.15.47:0/2496644484 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x564a0573e6d0 0x564a0573a8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:32:31.190 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.188+0000 7fef73e00640 1 -- 172.21.15.47:0/2496644484 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564a05742e40 con 0x564a0573f080 2024-03-21T10:32:31.190 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.188+0000 7fef73e00640 1 --2- 172.21.15.47:0/2496644484 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564a0573f080 0x564a05747000 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fef640049b0 tx=0x7fef64004270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d7bc6babc7939180 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:32:31.190 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.189+0000 7fef6aa00640 1 -- 172.21.15.47:0/2496644484 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef6402f070 con 0x564a0573f080 2024-03-21T10:32:31.190 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.189+0000 7fef6aa00640 1 -- 172.21.15.47:0/2496644484 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fef64029ce0 con 0x564a0573f080 2024-03-21T10:32:31.191 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.189+0000 7fef6aa00640 1 -- 172.21.15.47:0/2496644484 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef6401e040 con 0x564a0573f080 2024-03-21T10:32:31.191 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.189+0000 7fef83e12740 1 -- 172.21.15.47:0/2496644484 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564a0573f080 msgr2=0x564a05747000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:32:31.191 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.189+0000 7fef83e12740 1 --2- 172.21.15.47:0/2496644484 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564a0573f080 0x564a05747000 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fef640049b0 tx=0x7fef64004270 comp rx=0 tx=0).stop 2024-03-21T10:32:31.191 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.190+0000 7fef83e12740 1 -- 172.21.15.47:0/2496644484 shutdown_connections 2024-03-21T10:32:31.191 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.190+0000 7fef83e12740 1 --2- 172.21.15.47:0/2496644484 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564a05747c00 0x564a05747540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:32:31.191 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.190+0000 7fef83e12740 1 --2- 172.21.15.47:0/2496644484 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564a0573f080 0x564a05747000 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:32:31.191 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.190+0000 7fef83e12740 1 --2- 172.21.15.47:0/2496644484 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x564a0573e6d0 0x564a0573a8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:32:31.191 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.190+0000 7fef83e12740 1 -- 172.21.15.47:0/2496644484 >> 172.21.15.47:0/2496644484 conn(0x564a05738da0 msgr2=0x564a0573a1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:32:31.192 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.190+0000 7fef83e12740 1 -- 172.21.15.47:0/2496644484 shutdown_connections 2024-03-21T10:32:31.192 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.190+0000 7fef83e12740 1 -- 172.21.15.47:0/2496644484 wait complete. 2024-03-21T10:32:31.192 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.190+0000 7fef83e12740 1 -- start start 2024-03-21T10:32:31.192 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.190+0000 7fef83e12740 1 Processor -- start 2024-03-21T10:32:31.197 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.191+0000 7fef83e12740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564a0573e6d0 0x564a0574db00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:32:31.197 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.191+0000 7fef83e12740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x564a0573f080 0x564a0574e040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:32:31.197 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.191+0000 7fef83e12740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564a05747c00 0x564a057b2330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:32:31.197 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.191+0000 7fef83e12740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef7c1b1f60 con 0x564a0573f080 2024-03-21T10:32:31.197 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.191+0000 7fef83e12740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef7c1b1900 con 0x564a05747c00 2024-03-21T10:32:31.197 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.191+0000 7fef83e12740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef7c1aee20 con 0x564a0573e6d0 2024-03-21T10:32:31.197 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.191+0000 7fef83e12740 10 client.0 _ll_register_callbacks cb 0x7fef6c09bc30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T10:32:31.197 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.191+0000 7fef73e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x564a0573f080 0x564a0574e040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:32:31.197 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.191+0000 7fef7a800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564a05747c00 0x564a057b2330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:32:31.198 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.191+0000 7fef73e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x564a0573f080 0x564a0574e040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:51402/0 (socket says 172.21.15.47:51402) 2024-03-21T10:32:31.198 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.191+0000 7fef73e00640 1 -- 172.21.15.47:0/966443297 learned_addr learned my addr 172.21.15.47:0/966443297 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:32:31.198 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.192+0000 7fef79e00640 1 --2- 172.21.15.47:0/966443297 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564a0573e6d0 0x564a0574db00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:32:31.198 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.192+0000 7fef73e00640 1 -- 172.21.15.47:0/966443297 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564a0573e6d0 msgr2=0x564a0574db00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:32:31.198 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.192+0000 7fef73e00640 1 --2- 172.21.15.47:0/966443297 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564a0573e6d0 0x564a0574db00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:32:31.198 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.192+0000 7fef73e00640 1 -- 172.21.15.47:0/966443297 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564a05747c00 msgr2=0x564a057b2330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:32:31.198 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.192+0000 7fef73e00640 1 --2- 172.21.15.47:0/966443297 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564a05747c00 0x564a057b2330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:32:31.198 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.192+0000 7fef73e00640 1 -- 172.21.15.47:0/966443297 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x564a057b2920 con 0x564a0573f080 2024-03-21T10:32:31.198 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.192+0000 7fef73e00640 1 --2- 172.21.15.47:0/966443297 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x564a0573f080 0x564a0574e040 secure :-1 s=READY pgs=1024 cs=0 l=1 rev1=1 crypto rx=0x7fef64006010 tx=0x7fef640098d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4a85b2e775338075 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:32:31.198 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.192+0000 7fef5fe00640 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-03-21T10:32:31.198 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.192+0000 7fef5fe00640 1 -- 172.21.15.47:0/966443297 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef64038060 con 0x564a0573f080 2024-03-21T10:32:31.198 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.192+0000 7fef5fe00640 1 -- 172.21.15.47:0/966443297 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(99..99 src has 1..99) v4 ==== 9562+0+0 (secure 0 0 0) 0x7fef64005a70 con 0x564a0573f080 2024-03-21T10:32:31.198 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.192+0000 7fef83e12740 1 -- 172.21.15.47:0/966443297 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fef64029ce0 con 0x564a0573f080 2024-03-21T10:32:31.198 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.193+0000 7fef5fe00640 1 -- 172.21.15.47:0/966443297 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7fef6403f050 con 0x564a0573f080 2024-03-21T10:32:31.199 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.193+0000 7fef83e12740 10 client.17772 fetch_fsmap learned FSMap version 32 2024-03-21T10:32:31.199 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.193+0000 7fef83e12740 1 -- 172.21.15.47:0/966443297 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x564a057b3c00 con 0x564a0573f080 2024-03-21T10:32:31.199 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.193+0000 7fef5fe00640 1 -- 172.21.15.47:0/966443297 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fef64029ce0 con 0x564a0573f080 2024-03-21T10:32:31.199 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.193+0000 7fef5fe00640 1 -- 172.21.15.47:0/966443297 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 32) v1 ==== 7942+0+0 (secure 0 0 0) 0x7fef64018070 con 0x564a0573f080 2024-03-21T10:32:31.199 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.193+0000 7fef83e12740 10 client.17772 fetch_fsmap finished waiting for FSMap version 32 2024-03-21T10:32:31.199 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.193+0000 7fef83e12740 10 client.17772 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.k 2024-03-21T10:32:31.199 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.193+0000 7fef83e12740 20 client.17772 populate_metadata read hostname 'smithi047' 2024-03-21T10:32:31.199 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.193+0000 7fef83e12740 1 --2- 172.21.15.47:0/966443297 >> [v2:172.21.15.100:6832/2723302038,v1:172.21.15.100:6833/2723302038] conn(0x564a057b4090 0x564a057b64b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:32:31.199 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.193+0000 7fef83e12740 4 client.17772 mds_command: new command op to 34427 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-21T10:32:31.199 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.193+0000 7fef83e12740 1 -- 172.21.15.47:0/966443297 --> [v2:172.21.15.100:6832/2723302038,v1:172.21.15.100:6833/2723302038] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x564a05627500 con 0x564a057b4090 2024-03-21T10:32:31.199 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.194+0000 7fef79e00640 1 --2- 172.21.15.47:0/966443297 >> [v2:172.21.15.100:6832/2723302038,v1:172.21.15.100:6833/2723302038] conn(0x564a057b4090 0x564a057b64b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:32:31.199 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.194+0000 7fef79e00640 1 --2- 172.21.15.47:0/966443297 >> [v2:172.21.15.100:6832/2723302038,v1:172.21.15.100:6833/2723302038] conn(0x564a057b4090 0x564a057b64b0 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=10ad258d6c7f0f server_cookie=3b0745c848d0854e in_seq=0 out_seq=0 2024-03-21T10:32:31.199 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.194+0000 7fef5fe00640 10 client.17772 ms_handle_connect on v2:172.21.15.100:6832/2723302038 2024-03-21T10:32:31.199 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.195+0000 7fef5fe00640 1 -- 172.21.15.47:0/966443297 <== mds.3 v2:172.21.15.100:6832/2723302038 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x564a05627500 con 0x564a057b4090 2024-03-21T10:32:31.199 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.195+0000 7fef5fe00640 10 client.17772 handle_command_reply: tid=0 2024-03-21T10:32:31.200 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.195+0000 7fef83e12740 1 client.17772 shutdown 2024-03-21T10:32:31.200 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.195+0000 7fef83e12740 1 -- 172.21.15.47:0/966443297 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x564a0573f080 msgr2=0x564a0574e040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:32:31.200 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.195+0000 7fef83e12740 1 --2- 172.21.15.47:0/966443297 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x564a0573f080 0x564a0574e040 secure :-1 s=READY pgs=1024 cs=0 l=1 rev1=1 crypto rx=0x7fef64006010 tx=0x7fef640098d0 comp rx=0 tx=0).stop 2024-03-21T10:32:31.200 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.195+0000 7fef83e12740 1 -- 172.21.15.47:0/966443297 shutdown_connections 2024-03-21T10:32:31.200 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.195+0000 7fef83e12740 1 --2- 172.21.15.47:0/966443297 >> [v2:172.21.15.100:6832/2723302038,v1:172.21.15.100:6833/2723302038] conn(0x564a057b4090 0x564a057b64b0 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:32:31.200 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.195+0000 7fef83e12740 1 --2- 172.21.15.47:0/966443297 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564a05747c00 0x564a057b2330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:32:31.200 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.195+0000 7fef83e12740 1 --2- 172.21.15.47:0/966443297 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x564a0573f080 0x564a0574e040 unknown :-1 s=CLOSED pgs=1024 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:32:31.200 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.195+0000 7fef83e12740 1 --2- 172.21.15.47:0/966443297 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564a0573e6d0 0x564a0574db00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:32:31.200 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.195+0000 7fef83e12740 1 -- 172.21.15.47:0/966443297 >> 172.21.15.47:0/966443297 conn(0x564a05738da0 msgr2=0x564a0573aa40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:32:31.200 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.195+0000 7fef5fe00640 0 client.17772 ms_handle_reset on v2:172.21.15.100:6832/2723302038 2024-03-21T10:32:31.200 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.195+0000 7fef83e12740 1 -- 172.21.15.47:0/966443297 shutdown_connections 2024-03-21T10:32:31.200 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.195+0000 7fef83e12740 1 -- 172.21.15.47:0/966443297 wait complete. 2024-03-21T10:32:31.200 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.195+0000 7fef83e12740 20 client.17772 trim_cache size 0 max 16384 2024-03-21T10:32:31.201 INFO:teuthology.orchestra.run.smithi047.stdout:[] 2024-03-21T10:32:31.201 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.196+0000 7fef81000640 1 -- 172.21.15.47:0/1245453885 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7fef600a2fb0 msgr2=0x7fef600a5470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:32:31.202 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.196+0000 7fef81000640 1 --2- 172.21.15.47:0/1245453885 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7fef600a2fb0 0x7fef600a5470 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fef74009bf0 tx=0x7fef74004a40 comp rx=0 tx=0).stop 2024-03-21T10:32:31.202 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.196+0000 7fef81000640 1 -- 172.21.15.47:0/1245453885 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fef7c103840 msgr2=0x7fef7c19a030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:32:31.202 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.196+0000 7fef81000640 1 --2- 172.21.15.47:0/1245453885 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fef7c103840 0x7fef7c19a030 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fef6c00ea60 tx=0x7fef6c00ef20 comp rx=0 tx=0).stop 2024-03-21T10:32:31.202 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.196+0000 7fef81000640 1 -- 172.21.15.47:0/1245453885 shutdown_connections 2024-03-21T10:32:31.202 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.196+0000 7fef81000640 1 --2- 172.21.15.47:0/1245453885 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fef7c109850 0x7fef7c19e8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:32:31.202 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.196+0000 7fef81000640 1 --2- 172.21.15.47:0/1245453885 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fef7c104200 0x7fef7c19a570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:32:31.202 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.196+0000 7fef81000640 1 --2- 172.21.15.47:0/1245453885 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7fef600a2fb0 0x7fef600a5470 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:32:31.202 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.196+0000 7fef81000640 1 --2- 172.21.15.47:0/1245453885 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fef7c103840 0x7fef7c19a030 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:32:31.202 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.196+0000 7fef81000640 1 -- 172.21.15.47:0/1245453885 >> 172.21.15.47:0/1245453885 conn(0x7fef7c0ff310 msgr2=0x7fef7c0ffb00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:32:31.203 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.196+0000 7fef81000640 1 -- 172.21.15.47:0/1245453885 shutdown_connections 2024-03-21T10:32:31.203 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.196+0000 7fef81000640 1 -- 172.21.15.47:0/1245453885 wait complete. 2024-03-21T10:32:31.214 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-21T10:32:31.214 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-03-21T10:32:31.410 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.410+0000 7fabbe400640 1 -- 172.21.15.47:0/3251804275 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fabb8100e70 msgr2=0x7fabb810d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:32:31.410 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.410+0000 7fabbe400640 1 --2- 172.21.15.47:0/3251804275 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fabb8100e70 0x7fabb810d330 secure :-1 s=READY pgs=1025 cs=0 l=1 rev1=1 crypto rx=0x7fabb0009ca0 tx=0x7fabb002c0f0 comp rx=0 tx=0).stop 2024-03-21T10:32:31.410 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.410+0000 7fabbe400640 1 -- 172.21.15.47:0/3251804275 shutdown_connections 2024-03-21T10:32:31.410 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.410+0000 7fabbe400640 1 --2- 172.21.15.47:0/3251804275 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fabb8100e70 0x7fabb810d330 unknown :-1 s=CLOSED pgs=1025 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:32:31.410 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.410+0000 7fabbe400640 1 --2- 172.21.15.47:0/3251804275 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fabb81004b0 0x7fabb8100930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:32:31.410 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.410+0000 7fabbe400640 1 --2- 172.21.15.47:0/3251804275 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fabb8106570 0x7fabb8106950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:32:31.410 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.410+0000 7fabbe400640 1 -- 172.21.15.47:0/3251804275 >> 172.21.15.47:0/3251804275 conn(0x7fabb80fbfa0 msgr2=0x7fabb80fe3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:32:31.411 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.410+0000 7fabbe400640 1 -- 172.21.15.47:0/3251804275 shutdown_connections 2024-03-21T10:32:31.411 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.410+0000 7fabbe400640 1 -- 172.21.15.47:0/3251804275 wait complete. 2024-03-21T10:32:31.413 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.411+0000 7fabbe400640 1 Processor -- start 2024-03-21T10:32:31.413 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.411+0000 7fabbe400640 1 -- start start 2024-03-21T10:32:31.413 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.412+0000 7fabbe400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fabb81004b0 0x7fabb8199e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:32:31.413 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.412+0000 7fabbe400640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fabb8100e70 0x7fabb819a3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:32:31.413 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.412+0000 7fabbe400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fabb8106570 0x7fabb819e750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:32:31.413 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.412+0000 7fabbe400640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fabb806a860 con 0x7fabb81004b0 2024-03-21T10:32:31.413 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.412+0000 7fabbe400640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fabb806a6e0 con 0x7fabb8106570 2024-03-21T10:32:31.413 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.412+0000 7fabbe400640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fabb806a9e0 con 0x7fabb8100e70 2024-03-21T10:32:31.413 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.412+0000 7fabbd000640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fabb81004b0 0x7fabb8199e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:32:31.413 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.412+0000 7fabbd000640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fabb81004b0 0x7fabb8199e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:51420/0 (socket says 172.21.15.47:51420) 2024-03-21T10:32:31.414 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.412+0000 7fabbd000640 1 -- 172.21.15.47:0/843809675 learned_addr learned my addr 172.21.15.47:0/843809675 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:32:31.414 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.412+0000 7fabbda00640 1 --2- 172.21.15.47:0/843809675 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fabb8106570 0x7fabb819e750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:32:31.414 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.412+0000 7fabb7e00640 1 --2- 172.21.15.47:0/843809675 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fabb8100e70 0x7fabb819a3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:32:31.414 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.413+0000 7fabbd000640 1 -- 172.21.15.47:0/843809675 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fabb8100e70 msgr2=0x7fabb819a3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:32:31.414 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.413+0000 7fabbd000640 1 --2- 172.21.15.47:0/843809675 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fabb8100e70 0x7fabb819a3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:32:31.414 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.413+0000 7fabbd000640 1 -- 172.21.15.47:0/843809675 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fabb8106570 msgr2=0x7fabb819e750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:32:31.414 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.413+0000 7fabbd000640 1 --2- 172.21.15.47:0/843809675 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fabb8106570 0x7fabb819e750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:32:31.414 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.413+0000 7fabbd000640 1 -- 172.21.15.47:0/843809675 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fabb819ee80 con 0x7fabb81004b0 2024-03-21T10:32:31.414 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.413+0000 7fabbd000640 1 --2- 172.21.15.47:0/843809675 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fabb81004b0 0x7fabb8199e90 secure :-1 s=READY pgs=1026 cs=0 l=1 rev1=1 crypto rx=0x7fabac00d950 tx=0x7fabac00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:32:31.414 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.413+0000 7fabb5600640 1 -- 172.21.15.47:0/843809675 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabac017070 con 0x7fabb81004b0 2024-03-21T10:32:31.414 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.413+0000 7fabb5600640 1 -- 172.21.15.47:0/843809675 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fabac009c30 con 0x7fabb81004b0 2024-03-21T10:32:31.414 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.413+0000 7fabbe400640 1 -- 172.21.15.47:0/843809675 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fabb819f170 con 0x7fabb81004b0 2024-03-21T10:32:31.415 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.413+0000 7fabb5600640 1 -- 172.21.15.47:0/843809675 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabac00f620 con 0x7fabb81004b0 2024-03-21T10:32:31.415 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.413+0000 7fabbe400640 1 -- 172.21.15.47:0/843809675 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fabb81a6a00 con 0x7fabb81004b0 2024-03-21T10:32:31.416 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.417+0000 7fabb5600640 1 -- 172.21.15.47:0/843809675 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 27) v1 ==== 137985+0+0 (secure 0 0 0) 0x7fabac0109a0 con 0x7fabb81004b0 2024-03-21T10:32:31.417 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.418+0000 7fabc276d740 1 Processor -- start 2024-03-21T10:32:31.417 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.418+0000 7fabc276d740 1 -- start start 2024-03-21T10:32:31.418 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.419+0000 7fabc276d740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f2e88088b0 0x55f2e8808c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:32:31.418 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.419+0000 7fabb5600640 1 --2- 172.21.15.47:0/843809675 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7faba00a3110 0x7faba00a55d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:32:31.419 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.419+0000 7fabc276d740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f2e8809260 0x55f2e8811810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:32:31.419 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.419+0000 7fabc276d740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f2e8811de0 0x55f2e8814230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:32:31.419 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.419+0000 7fabc276d740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f2e876c2c0 con 0x55f2e8809260 2024-03-21T10:32:31.419 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.419+0000 7fabc276d740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f2e8714c90 con 0x55f2e88088b0 2024-03-21T10:32:31.419 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.419+0000 7fabc276d740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f2e8768360 con 0x55f2e8811de0 2024-03-21T10:32:31.419 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.419+0000 7fabb5600640 1 -- 172.21.15.47:0/843809675 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 9562+0+0 (secure 0 0 0) 0x7fabac0cf830 con 0x7fabb81004b0 2024-03-21T10:32:31.419 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.419+0000 7fabb7e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f2e8809260 0x55f2e8811810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:32:31.419 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.419+0000 7fabbda00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f2e88088b0 0x55f2e8808c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:32:31.419 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.419+0000 7fabbd000640 1 --2- 172.21.15.47:0/843809675 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7faba00a3110 0x7faba00a55d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:32:31.419 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.419+0000 7fabb7e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f2e8809260 0x55f2e8811810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:51424/0 (socket says 172.21.15.47:51424) 2024-03-21T10:32:31.419 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.419+0000 7fabbda00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f2e88088b0 0x55f2e8808c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:33770/0 (socket says 172.21.15.47:33770) 2024-03-21T10:32:31.419 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.419+0000 7fabb7e00640 1 -- 172.21.15.47:0/1386098946 learned_addr learned my addr 172.21.15.47:0/1386098946 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:32:31.421 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.419+0000 7fabbda00640 1 --2- 172.21.15.47:0/1386098946 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f2e8811de0 0x55f2e8814230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:32:31.421 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.420+0000 7fabb7e00640 1 -- 172.21.15.47:0/1386098946 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f2e8811de0 msgr2=0x55f2e8814230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:32:31.421 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.420+0000 7fabb7e00640 1 --2- 172.21.15.47:0/1386098946 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f2e8811de0 0x55f2e8814230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:32:31.421 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.420+0000 7fabb7e00640 1 -- 172.21.15.47:0/1386098946 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f2e88088b0 msgr2=0x55f2e8808c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:32:31.421 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.420+0000 7fabb7e00640 1 --2- 172.21.15.47:0/1386098946 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f2e88088b0 0x55f2e8808c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:32:31.421 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.420+0000 7fabb7e00640 1 -- 172.21.15.47:0/1386098946 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f2e8739080 con 0x55f2e8809260 2024-03-21T10:32:31.421 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.420+0000 7fabbda00640 1 --2- 172.21.15.47:0/1386098946 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f2e8811de0 0x55f2e8814230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T10:32:31.421 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.420+0000 7fabbd000640 1 --2- 172.21.15.47:0/843809675 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7faba00a3110 0x7faba00a55d0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fabac0a3c90 tx=0x7fabac010f70 comp rx=0 tx=0).ready entity=mgr.15687 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:32:31.421 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.420+0000 7fabb7e00640 1 --2- 172.21.15.47:0/1386098946 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f2e8809260 0x55f2e8811810 secure :-1 s=READY pgs=1027 cs=0 l=1 rev1=1 crypto rx=0x7faba400bc90 tx=0x7faba400f730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cc7b16d2184570f0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:32:31.421 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.420+0000 7fabaa000640 1 -- 172.21.15.47:0/1386098946 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faba4015070 con 0x55f2e8809260 2024-03-21T10:32:31.422 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.420+0000 7fabaa000640 1 -- 172.21.15.47:0/1386098946 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7faba401a910 con 0x55f2e8809260 2024-03-21T10:32:31.422 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.420+0000 7fabaa000640 1 -- 172.21.15.47:0/1386098946 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faba40118a0 con 0x55f2e8809260 2024-03-21T10:32:31.422 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.421+0000 7fabc276d740 1 -- 172.21.15.47:0/1386098946 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f2e8809260 msgr2=0x55f2e8811810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:32:31.422 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.421+0000 7fabc276d740 1 --2- 172.21.15.47:0/1386098946 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f2e8809260 0x55f2e8811810 secure :-1 s=READY pgs=1027 cs=0 l=1 rev1=1 crypto rx=0x7faba400bc90 tx=0x7faba400f730 comp rx=0 tx=0).stop 2024-03-21T10:32:31.422 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.421+0000 7fabc276d740 1 -- 172.21.15.47:0/1386098946 shutdown_connections 2024-03-21T10:32:31.422 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.421+0000 7fabc276d740 1 --2- 172.21.15.47:0/1386098946 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f2e8811de0 0x55f2e8814230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:32:31.422 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.421+0000 7fabc276d740 1 --2- 172.21.15.47:0/1386098946 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f2e8809260 0x55f2e8811810 unknown :-1 s=CLOSED pgs=1027 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:32:31.422 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.421+0000 7fabc276d740 1 --2- 172.21.15.47:0/1386098946 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f2e88088b0 0x55f2e8808c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:32:31.422 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.421+0000 7fabc276d740 1 -- 172.21.15.47:0/1386098946 >> 172.21.15.47:0/1386098946 conn(0x55f2e8802f80 msgr2=0x55f2e8803360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:32:31.422 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.421+0000 7fabc276d740 1 -- 172.21.15.47:0/1386098946 shutdown_connections 2024-03-21T10:32:31.422 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.422+0000 7fabc276d740 1 -- 172.21.15.47:0/1386098946 wait complete. 2024-03-21T10:32:31.425 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.422+0000 7fabc276d740 1 -- start start 2024-03-21T10:32:31.425 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.422+0000 7fabc276d740 1 Processor -- start 2024-03-21T10:32:31.425 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.423+0000 7fabc276d740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f2e88088b0 0x55f2e8811740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:32:31.425 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.423+0000 7fabc276d740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f2e8809260 0x55f2e880ead0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:32:31.425 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.423+0000 7fabc276d740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f2e8811de0 0x55f2e880f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:32:31.425 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.423+0000 7fabc276d740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f2e8814d50 con 0x55f2e8811de0 2024-03-21T10:32:31.425 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.423+0000 7fabc276d740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f2e87280d0 con 0x55f2e88088b0 2024-03-21T10:32:31.426 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.423+0000 7fabc276d740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f2e8814ed0 con 0x55f2e8809260 2024-03-21T10:32:31.426 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.423+0000 7fabc276d740 10 client.0 _ll_register_callbacks cb 0x55f2e872ce80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T10:32:31.426 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.423+0000 7fabbda00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f2e8811de0 0x55f2e880f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:32:31.426 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.423+0000 7fabbda00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f2e8811de0 0x55f2e880f0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:51426/0 (socket says 172.21.15.47:51426) 2024-03-21T10:32:31.426 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.423+0000 7fabbda00640 1 -- 172.21.15.47:0/2522874638 learned_addr learned my addr 172.21.15.47:0/2522874638 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:32:31.426 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.423+0000 7fabbda00640 1 --2- 172.21.15.47:0/2522874638 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f2e88088b0 0x55f2e8811740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:32:31.426 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.423+0000 7fabb7e00640 1 --2- 172.21.15.47:0/2522874638 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f2e8809260 0x55f2e880ead0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:32:31.426 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.423+0000 7fabbda00640 1 -- 172.21.15.47:0/2522874638 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f2e8809260 msgr2=0x55f2e880ead0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:32:31.426 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.423+0000 7fabbda00640 1 --2- 172.21.15.47:0/2522874638 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f2e8809260 0x55f2e880ead0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:32:31.426 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.423+0000 7fabbda00640 1 -- 172.21.15.47:0/2522874638 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f2e88088b0 msgr2=0x55f2e8811740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:32:31.426 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.424+0000 7fabbda00640 1 --2- 172.21.15.47:0/2522874638 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f2e88088b0 0x55f2e8811740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:32:31.426 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.424+0000 7fabbda00640 1 -- 172.21.15.47:0/2522874638 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55f2e84803f0 con 0x55f2e8811de0 2024-03-21T10:32:31.426 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.424+0000 7fabb7e00640 1 --2- 172.21.15.47:0/2522874638 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f2e8809260 0x55f2e880ead0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T10:32:31.426 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.424+0000 7fabbda00640 1 --2- 172.21.15.47:0/2522874638 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f2e8811de0 0x55f2e880f0e0 secure :-1 s=READY pgs=1028 cs=0 l=1 rev1=1 crypto rx=0x7fabb002c700 tx=0x7fabb002c0f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ea29d1a1965fff83 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:32:31.427 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.424+0000 7fab9ea00640 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-03-21T10:32:31.427 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.424+0000 7fab9ea00640 1 -- 172.21.15.47:0/2522874638 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabb0039040 con 0x55f2e8811de0 2024-03-21T10:32:31.427 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.424+0000 7fab9ea00640 1 -- 172.21.15.47:0/2522874638 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(99..99 src has 1..99) v4 ==== 9562+0+0 (secure 0 0 0) 0x7fabb0034640 con 0x55f2e8811de0 2024-03-21T10:32:31.427 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.424+0000 7fabc276d740 1 -- 172.21.15.47:0/2522874638 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7faba401a910 con 0x55f2e8811de0 2024-03-21T10:32:31.427 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.424+0000 7fab9ea00640 1 -- 172.21.15.47:0/2522874638 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7fabb0055050 con 0x55f2e8811de0 2024-03-21T10:32:31.427 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.424+0000 7fabc276d740 10 client.17784 fetch_fsmap learned FSMap version 32 2024-03-21T10:32:31.427 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.424+0000 7fabc276d740 1 -- 172.21.15.47:0/2522874638 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x55f2e854a7d0 con 0x55f2e8811de0 2024-03-21T10:32:31.427 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.425+0000 7fab9ea00640 1 -- 172.21.15.47:0/2522874638 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7faba401a910 con 0x55f2e8811de0 2024-03-21T10:32:31.427 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.425+0000 7fab9ea00640 1 -- 172.21.15.47:0/2522874638 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 32) v1 ==== 7942+0+0 (secure 0 0 0) 0x7fabb003f040 con 0x55f2e8811de0 2024-03-21T10:32:31.427 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.425+0000 7fabc276d740 10 client.17784 fetch_fsmap finished waiting for FSMap version 32 2024-03-21T10:32:31.427 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.425+0000 7fabc276d740 10 client.17784 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.e 2024-03-21T10:32:31.427 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.425+0000 7fabc276d740 20 client.17784 populate_metadata read hostname 'smithi047' 2024-03-21T10:32:31.427 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.425+0000 7fabc276d740 1 --2- 172.21.15.47:0/2522874638 >> [v2:172.21.15.100:6834/2597849957,v1:172.21.15.100:6835/2597849957] conn(0x55f2e8821c90 0x55f2e8824080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:32:31.427 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.425+0000 7fabc276d740 4 client.17784 mds_command: new command op to 34454 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T10:32:31.428 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.425+0000 7fabc276d740 1 -- 172.21.15.47:0/2522874638 --> [v2:172.21.15.100:6834/2597849957,v1:172.21.15.100:6835/2597849957] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55f2e8779800 con 0x55f2e8821c90 2024-03-21T10:32:31.428 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.426+0000 7fabb7e00640 1 --2- 172.21.15.47:0/2522874638 >> [v2:172.21.15.100:6834/2597849957,v1:172.21.15.100:6835/2597849957] conn(0x55f2e8821c90 0x55f2e8824080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:32:31.430 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.426+0000 7fabb7e00640 1 --2- 172.21.15.47:0/2522874638 >> [v2:172.21.15.100:6834/2597849957,v1:172.21.15.100:6835/2597849957] conn(0x55f2e8821c90 0x55f2e8824080 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=15e18d7b3e99704c server_cookie=61abb9451b738960 in_seq=0 out_seq=0 2024-03-21T10:32:31.430 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.426+0000 7fab9ea00640 10 client.17784 ms_handle_connect on v2:172.21.15.100:6834/2597849957 2024-03-21T10:32:31.430 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.427+0000 7fab9ea00640 1 -- 172.21.15.47:0/2522874638 <== mds.2 v2:172.21.15.100:6834/2597849957 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55f2e8779800 con 0x55f2e8821c90 2024-03-21T10:32:31.431 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.427+0000 7fab9ea00640 10 client.17784 handle_command_reply: tid=0 2024-03-21T10:32:31.431 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.427+0000 7fabc276d740 1 client.17784 shutdown 2024-03-21T10:32:31.431 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.428+0000 7fabc276d740 1 -- 172.21.15.47:0/2522874638 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f2e8811de0 msgr2=0x55f2e880f0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:32:31.431 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.428+0000 7fabc276d740 1 --2- 172.21.15.47:0/2522874638 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f2e8811de0 0x55f2e880f0e0 secure :-1 s=READY pgs=1028 cs=0 l=1 rev1=1 crypto rx=0x7fabb002c700 tx=0x7fabb002c0f0 comp rx=0 tx=0).stop 2024-03-21T10:32:31.431 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.428+0000 7fabc276d740 1 -- 172.21.15.47:0/2522874638 shutdown_connections 2024-03-21T10:32:31.431 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.428+0000 7fabc276d740 1 --2- 172.21.15.47:0/2522874638 >> [v2:172.21.15.100:6834/2597849957,v1:172.21.15.100:6835/2597849957] conn(0x55f2e8821c90 0x55f2e8824080 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:32:31.431 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.428+0000 7fabc276d740 1 --2- 172.21.15.47:0/2522874638 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f2e8811de0 0x55f2e880f0e0 unknown :-1 s=CLOSED pgs=1028 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:32:31.431 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.428+0000 7fabc276d740 1 --2- 172.21.15.47:0/2522874638 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f2e8809260 0x55f2e880ead0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:32:31.431 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.428+0000 7fabc276d740 1 --2- 172.21.15.47:0/2522874638 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f2e88088b0 0x55f2e8811740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:32:31.431 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.428+0000 7fab9ea00640 0 client.17784 ms_handle_reset on v2:172.21.15.100:6834/2597849957 2024-03-21T10:32:31.431 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.428+0000 7fabc276d740 1 -- 172.21.15.47:0/2522874638 >> 172.21.15.47:0/2522874638 conn(0x55f2e8802f80 msgr2=0x55f2e8804cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:32:31.431 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.428+0000 7fabc276d740 1 -- 172.21.15.47:0/2522874638 shutdown_connections 2024-03-21T10:32:31.431 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.428+0000 7fabc276d740 1 -- 172.21.15.47:0/2522874638 wait complete. 2024-03-21T10:32:31.431 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.428+0000 7fabc276d740 20 client.17784 trim_cache size 0 max 16384 2024-03-21T10:32:31.442 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.442+0000 7fabc276d740 1 Processor -- start 2024-03-21T10:32:31.442 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.442+0000 7fabc276d740 1 -- start start 2024-03-21T10:32:31.442 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.442+0000 7fabc276d740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f2e88088b0 0x55f2e880dc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:32:31.442 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.442+0000 7fabc276d740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f2e8809260 0x55f2e8801fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:32:31.442 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.442+0000 7fabc276d740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f2e8811de0 0x55f2e8811200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:32:31.442 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.442+0000 7fabc276d740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fabb81b08e0 con 0x55f2e88088b0 2024-03-21T10:32:31.442 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.442+0000 7fabc276d740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fabb81b2260 con 0x55f2e8811de0 2024-03-21T10:32:31.442 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.442+0000 7fabc276d740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fabb81b0f40 con 0x55f2e8809260 2024-03-21T10:32:31.442 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.442+0000 7fabbda00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f2e88088b0 0x55f2e880dc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:32:31.442 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.442+0000 7fabbda00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f2e88088b0 0x55f2e880dc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:51434/0 (socket says 172.21.15.47:51434) 2024-03-21T10:32:31.443 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.443+0000 7fabbda00640 1 -- 172.21.15.47:0/365149294 learned_addr learned my addr 172.21.15.47:0/365149294 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:32:31.444 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.443+0000 7fabbda00640 1 -- 172.21.15.47:0/365149294 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f2e8809260 msgr2=0x55f2e8801fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:32:31.444 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.443+0000 7fabb7e00640 1 --2- 172.21.15.47:0/365149294 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f2e8809260 0x55f2e8801fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:32:31.444 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.443+0000 7fabbda00640 1 --2- 172.21.15.47:0/365149294 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f2e8809260 0x55f2e8801fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:32:31.444 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.443+0000 7fabbda00640 1 -- 172.21.15.47:0/365149294 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f2e8811de0 msgr2=0x55f2e8811200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:32:31.444 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.443+0000 7fabbda00640 1 --2- 172.21.15.47:0/365149294 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f2e8811de0 0x55f2e8811200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:32:31.444 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.443+0000 7fabbda00640 1 -- 172.21.15.47:0/365149294 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f2e88246e0 con 0x55f2e88088b0 2024-03-21T10:32:31.444 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.443+0000 7fabbda00640 1 --2- 172.21.15.47:0/365149294 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f2e88088b0 0x55f2e880dc30 secure :-1 s=READY pgs=1029 cs=0 l=1 rev1=1 crypto rx=0x7fabb0009600 tx=0x7fabb002c790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5136fca59d2f123e server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:32:31.444 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.443+0000 7faba9600640 1 -- 172.21.15.47:0/365149294 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabb0039040 con 0x55f2e88088b0 2024-03-21T10:32:31.444 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.444+0000 7faba9600640 1 -- 172.21.15.47:0/365149294 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fabb0032e70 con 0x55f2e88088b0 2024-03-21T10:32:31.444 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.444+0000 7faba9600640 1 -- 172.21.15.47:0/365149294 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabb00346a0 con 0x55f2e88088b0 2024-03-21T10:32:31.445 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.444+0000 7fabc276d740 1 -- 172.21.15.47:0/365149294 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f2e88088b0 msgr2=0x55f2e880dc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:32:31.445 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.444+0000 7fabc276d740 1 --2- 172.21.15.47:0/365149294 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f2e88088b0 0x55f2e880dc30 secure :-1 s=READY pgs=1029 cs=0 l=1 rev1=1 crypto rx=0x7fabb0009600 tx=0x7fabb002c790 comp rx=0 tx=0).stop 2024-03-21T10:32:31.445 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.444+0000 7fabc276d740 1 -- 172.21.15.47:0/365149294 shutdown_connections 2024-03-21T10:32:31.445 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.444+0000 7fabc276d740 1 --2- 172.21.15.47:0/365149294 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f2e8811de0 0x55f2e8811200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:32:31.445 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.444+0000 7fabc276d740 1 --2- 172.21.15.47:0/365149294 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f2e8809260 0x55f2e8801fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:32:31.445 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.444+0000 7fabc276d740 1 --2- 172.21.15.47:0/365149294 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f2e88088b0 0x55f2e880dc30 unknown :-1 s=CLOSED pgs=1029 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:32:31.445 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.444+0000 7fabc276d740 1 -- 172.21.15.47:0/365149294 >> 172.21.15.47:0/365149294 conn(0x55f2e8802f80 msgr2=0x55f2e88018d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:32:31.445 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.444+0000 7fabc276d740 1 -- 172.21.15.47:0/365149294 shutdown_connections 2024-03-21T10:32:31.445 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.444+0000 7fabc276d740 1 -- 172.21.15.47:0/365149294 wait complete. 2024-03-21T10:32:31.448 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.445+0000 7fabc276d740 1 -- start start 2024-03-21T10:32:31.448 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.445+0000 7fabc276d740 1 Processor -- start 2024-03-21T10:32:31.448 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.445+0000 7fabc276d740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f2e88088b0 0x55f2e8819ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:32:31.448 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.445+0000 7fabc276d740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f2e8809260 0x55f2e8817270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:32:31.448 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.445+0000 7fabc276d740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f2e8811de0 0x55f2e88177b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:32:31.448 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.445+0000 7fabc276d740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fabb81b1c00 con 0x55f2e8811de0 2024-03-21T10:32:31.448 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.445+0000 7fabc276d740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fabb81b15a0 con 0x55f2e8809260 2024-03-21T10:32:31.448 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.445+0000 7fabc276d740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fabb81aeac0 con 0x55f2e88088b0 2024-03-21T10:32:31.448 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.445+0000 7fabc276d740 10 client.0 _ll_register_callbacks cb 0x7fabb00329d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T10:32:31.449 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.446+0000 7fabbda00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f2e8811de0 0x55f2e88177b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:32:31.449 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.446+0000 7fabbda00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f2e8811de0 0x55f2e88177b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:51442/0 (socket says 172.21.15.47:51442) 2024-03-21T10:32:31.449 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.446+0000 7fabbda00640 1 -- 172.21.15.47:0/1604050823 learned_addr learned my addr 172.21.15.47:0/1604050823 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:32:31.449 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.446+0000 7fabb7e00640 1 --2- 172.21.15.47:0/1604050823 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f2e8809260 0x55f2e8817270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:32:31.449 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.446+0000 7fabbda00640 1 --2- 172.21.15.47:0/1604050823 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f2e88088b0 0x55f2e8819ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:32:31.449 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.446+0000 7fabbda00640 1 -- 172.21.15.47:0/1604050823 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f2e88088b0 msgr2=0x55f2e8819ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:32:31.449 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.446+0000 7fabbda00640 1 --2- 172.21.15.47:0/1604050823 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f2e88088b0 0x55f2e8819ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:32:31.449 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.446+0000 7fabbda00640 1 -- 172.21.15.47:0/1604050823 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f2e8809260 msgr2=0x55f2e8817270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:32:31.449 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.446+0000 7fabbda00640 1 --2- 172.21.15.47:0/1604050823 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f2e8809260 0x55f2e8817270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:32:31.449 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.446+0000 7fabbda00640 1 -- 172.21.15.47:0/1604050823 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55f2e8817f90 con 0x55f2e8811de0 2024-03-21T10:32:31.449 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.447+0000 7fabbda00640 1 --2- 172.21.15.47:0/1604050823 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f2e8811de0 0x55f2e88177b0 secure :-1 s=READY pgs=1030 cs=0 l=1 rev1=1 crypto rx=0x7fabb002c700 tx=0x7fabb0005fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=55d1668275fdc7d9 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:32:31.449 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.447+0000 7fab9ea00640 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-03-21T10:32:31.449 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.447+0000 7fab9ea00640 1 -- 172.21.15.47:0/1604050823 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabb0050a80 con 0x55f2e8811de0 2024-03-21T10:32:31.450 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.447+0000 7fab9ea00640 1 -- 172.21.15.47:0/1604050823 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(99..99 src has 1..99) v4 ==== 9562+0+0 (secure 0 0 0) 0x7fabb005e070 con 0x55f2e8811de0 2024-03-21T10:32:31.450 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.447+0000 7fabc276d740 1 -- 172.21.15.47:0/1604050823 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55f2e8773b50 con 0x55f2e8811de0 2024-03-21T10:32:31.450 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.447+0000 7fab9ea00640 1 -- 172.21.15.47:0/1604050823 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7fabb0004b40 con 0x55f2e8811de0 2024-03-21T10:32:31.450 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.447+0000 7fabc276d740 10 client.17790 fetch_fsmap learned FSMap version 32 2024-03-21T10:32:31.450 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.447+0000 7fabc276d740 1 -- 172.21.15.47:0/1604050823 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x55f2e887d4f0 con 0x55f2e8811de0 2024-03-21T10:32:31.450 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.447+0000 7fab9ea00640 1 -- 172.21.15.47:0/1604050823 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x55f2e8773b50 con 0x55f2e8811de0 2024-03-21T10:32:31.450 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.448+0000 7fab9ea00640 1 -- 172.21.15.47:0/1604050823 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 32) v1 ==== 7942+0+0 (secure 0 0 0) 0x7fabb003f040 con 0x55f2e8811de0 2024-03-21T10:32:31.450 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.448+0000 7fabc276d740 10 client.17790 fetch_fsmap finished waiting for FSMap version 32 2024-03-21T10:32:31.450 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.448+0000 7fabc276d740 10 client.17790 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.e 2024-03-21T10:32:31.450 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.448+0000 7fabc276d740 20 client.17790 populate_metadata read hostname 'smithi047' 2024-03-21T10:32:31.450 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.448+0000 7fabc276d740 1 --2- 172.21.15.47:0/1604050823 >> [v2:172.21.15.100:6834/2597849957,v1:172.21.15.100:6835/2597849957] conn(0x55f2e887dad0 0x55f2e887fef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:32:31.450 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.448+0000 7fabc276d740 4 client.17790 mds_command: new command op to 34454 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-21T10:32:31.450 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.448+0000 7fabc276d740 1 -- 172.21.15.47:0/1604050823 --> [v2:172.21.15.100:6834/2597849957,v1:172.21.15.100:6835/2597849957] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55f2e86f1500 con 0x55f2e887dad0 2024-03-21T10:32:31.451 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.448+0000 7fabb7e00640 1 --2- 172.21.15.47:0/1604050823 >> [v2:172.21.15.100:6834/2597849957,v1:172.21.15.100:6835/2597849957] conn(0x55f2e887dad0 0x55f2e887fef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:32:31.451 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.449+0000 7fabb7e00640 1 --2- 172.21.15.47:0/1604050823 >> [v2:172.21.15.100:6834/2597849957,v1:172.21.15.100:6835/2597849957] conn(0x55f2e887dad0 0x55f2e887fef0 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=f6f13987522ec71b server_cookie=36f589f7889443b7 in_seq=0 out_seq=0 2024-03-21T10:32:31.452 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.449+0000 7fab9ea00640 10 client.17790 ms_handle_connect on v2:172.21.15.100:6834/2597849957 2024-03-21T10:32:31.452 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.449+0000 7fab9ea00640 1 -- 172.21.15.47:0/1604050823 <== mds.2 v2:172.21.15.100:6834/2597849957 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55f2e86f1500 con 0x55f2e887dad0 2024-03-21T10:32:31.452 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.449+0000 7fab9ea00640 10 client.17790 handle_command_reply: tid=0 2024-03-21T10:32:31.452 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.449+0000 7fabc276d740 1 client.17790 shutdown 2024-03-21T10:32:31.452 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.449+0000 7fabc276d740 1 -- 172.21.15.47:0/1604050823 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f2e8811de0 msgr2=0x55f2e88177b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:32:31.452 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.449+0000 7fabc276d740 1 --2- 172.21.15.47:0/1604050823 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f2e8811de0 0x55f2e88177b0 secure :-1 s=READY pgs=1030 cs=0 l=1 rev1=1 crypto rx=0x7fabb002c700 tx=0x7fabb0005fd0 comp rx=0 tx=0).stop 2024-03-21T10:32:31.452 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.449+0000 7fabc276d740 1 -- 172.21.15.47:0/1604050823 shutdown_connections 2024-03-21T10:32:31.452 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.449+0000 7fabc276d740 1 --2- 172.21.15.47:0/1604050823 >> [v2:172.21.15.100:6834/2597849957,v1:172.21.15.100:6835/2597849957] conn(0x55f2e887dad0 0x55f2e887fef0 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:32:31.452 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.449+0000 7fabc276d740 1 --2- 172.21.15.47:0/1604050823 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f2e8811de0 0x55f2e88177b0 unknown :-1 s=CLOSED pgs=1030 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:32:31.452 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.449+0000 7fabc276d740 1 --2- 172.21.15.47:0/1604050823 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f2e8809260 0x55f2e8817270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:32:31.452 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.449+0000 7fabc276d740 1 --2- 172.21.15.47:0/1604050823 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f2e88088b0 0x55f2e8819ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:32:31.452 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.449+0000 7fabc276d740 1 -- 172.21.15.47:0/1604050823 >> 172.21.15.47:0/1604050823 conn(0x55f2e8802f80 msgr2=0x55f2e880dab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:32:31.452 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.449+0000 7fab9ea00640 0 client.17790 ms_handle_reset on v2:172.21.15.100:6834/2597849957 2024-03-21T10:32:31.452 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.450+0000 7fabc276d740 1 -- 172.21.15.47:0/1604050823 shutdown_connections 2024-03-21T10:32:31.453 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.450+0000 7fabc276d740 1 -- 172.21.15.47:0/1604050823 wait complete. 2024-03-21T10:32:31.453 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.450+0000 7fabc276d740 20 client.17790 trim_cache size 0 max 16384 2024-03-21T10:32:31.453 INFO:teuthology.orchestra.run.smithi047.stdout:[] 2024-03-21T10:32:31.454 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.450+0000 7fabbe400640 1 -- 172.21.15.47:0/843809675 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7faba00a3110 msgr2=0x7faba00a55d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:32:31.454 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.450+0000 7fabbe400640 1 --2- 172.21.15.47:0/843809675 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7faba00a3110 0x7faba00a55d0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fabac0a3c90 tx=0x7fabac010f70 comp rx=0 tx=0).stop 2024-03-21T10:32:31.454 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.450+0000 7fabbe400640 1 -- 172.21.15.47:0/843809675 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fabb81004b0 msgr2=0x7fabb8199e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:32:31.454 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.450+0000 7fabbe400640 1 --2- 172.21.15.47:0/843809675 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fabb81004b0 0x7fabb8199e90 secure :-1 s=READY pgs=1026 cs=0 l=1 rev1=1 crypto rx=0x7fabac00d950 tx=0x7fabac00de10 comp rx=0 tx=0).stop 2024-03-21T10:32:31.454 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.450+0000 7fabbe400640 1 -- 172.21.15.47:0/843809675 shutdown_connections 2024-03-21T10:32:31.454 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.450+0000 7fabbe400640 1 --2- 172.21.15.47:0/843809675 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7faba00a3110 0x7faba00a55d0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:32:31.454 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.450+0000 7fabbe400640 1 --2- 172.21.15.47:0/843809675 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fabb8106570 0x7fabb819e750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:32:31.454 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.450+0000 7fabbe400640 1 --2- 172.21.15.47:0/843809675 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fabb8100e70 0x7fabb819a3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:32:31.454 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.450+0000 7fabbe400640 1 --2- 172.21.15.47:0/843809675 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fabb81004b0 0x7fabb8199e90 unknown :-1 s=CLOSED pgs=1026 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:32:31.454 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.450+0000 7fabbe400640 1 -- 172.21.15.47:0/843809675 >> 172.21.15.47:0/843809675 conn(0x7fabb80fbfa0 msgr2=0x7fabb80fd4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:32:31.454 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.451+0000 7fabbe400640 1 -- 172.21.15.47:0/843809675 shutdown_connections 2024-03-21T10:32:31.454 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.451+0000 7fabbe400640 1 -- 172.21.15.47:0/843809675 wait complete. 2024-03-21T10:32:31.461 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:32:31 smithi047 ceph-mon[31547]: pgmap v221: 129 pgs: 129 active+clean; 277 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-21T10:32:31.461 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:32:31 smithi047 ceph-mon[31547]: from='client.? 172.21.15.47:0/4175665670' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-21T10:32:31.468 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-21T10:32:31.468 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-03-21T10:32:31.698 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.699+0000 7ff0b6200640 1 -- 172.21.15.47:0/777067482 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff0b01065a0 msgr2=0x7ff0b0069c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:32:31.699 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.699+0000 7ff0b6200640 1 --2- 172.21.15.47:0/777067482 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff0b01065a0 0x7ff0b0069c10 secure :-1 s=READY pgs=1031 cs=0 l=1 rev1=1 crypto rx=0x7ff0a0009900 tx=0x7ff0a002bc50 comp rx=0 tx=0).stop 2024-03-21T10:32:31.699 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.699+0000 7ff0b6200640 1 -- 172.21.15.47:0/777067482 shutdown_connections 2024-03-21T10:32:31.699 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.699+0000 7ff0b6200640 1 --2- 172.21.15.47:0/777067482 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff0b006a830 0x7ff0b01040a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:32:31.699 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.699+0000 7ff0b6200640 1 --2- 172.21.15.47:0/777067482 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff0b0106a10 0x7ff0b006a150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:32:31.699 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.699+0000 7ff0b6200640 1 --2- 172.21.15.47:0/777067482 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff0b01065a0 0x7ff0b0069c10 unknown :-1 s=CLOSED pgs=1031 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:32:31.699 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.699+0000 7ff0b6200640 1 -- 172.21.15.47:0/777067482 >> 172.21.15.47:0/777067482 conn(0x7ff0b0077580 msgr2=0x7ff0b0077990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:32:31.699 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.699+0000 7ff0b6200640 1 -- 172.21.15.47:0/777067482 shutdown_connections 2024-03-21T10:32:31.699 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.699+0000 7ff0b6200640 1 -- 172.21.15.47:0/777067482 wait complete. 2024-03-21T10:32:31.701 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.700+0000 7ff0b6200640 1 Processor -- start 2024-03-21T10:32:31.701 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.700+0000 7ff0b6200640 1 -- start start 2024-03-21T10:32:31.701 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.700+0000 7ff0b6200640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff0b01065a0 0x7ff0b019e490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:32:31.701 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.700+0000 7ff0b6200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff0b0106a10 0x7ff0b019e9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:32:31.701 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.700+0000 7ff0b6200640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff0b01a2d50 0x7ff0b01a3200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:32:31.701 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.700+0000 7ff0b6200640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0b0117f50 con 0x7ff0b01065a0 2024-03-21T10:32:31.701 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.700+0000 7ff0b6200640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0b0117dd0 con 0x7ff0b0106a10 2024-03-21T10:32:31.701 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.700+0000 7ff0b6200640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0b01180d0 con 0x7ff0b01a2d50 2024-03-21T10:32:31.701 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.701+0000 7ff0af400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff0b01065a0 0x7ff0b019e490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:32:31.702 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.701+0000 7ff0af400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff0b01065a0 0x7ff0b019e490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:51460/0 (socket says 172.21.15.47:51460) 2024-03-21T10:32:31.702 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.701+0000 7ff0af400640 1 -- 172.21.15.47:0/1224626709 learned_addr learned my addr 172.21.15.47:0/1224626709 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:32:31.702 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.701+0000 7ff0a7e00640 1 --2- 172.21.15.47:0/1224626709 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff0b0106a10 0x7ff0b019e9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:32:31.702 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.701+0000 7ff0af400640 1 -- 172.21.15.47:0/1224626709 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff0b01a2d50 msgr2=0x7ff0b01a3200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:32:31.702 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.701+0000 7ff0afe00640 1 --2- 172.21.15.47:0/1224626709 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff0b01a2d50 0x7ff0b01a3200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:32:31.702 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.701+0000 7ff0af400640 1 --2- 172.21.15.47:0/1224626709 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff0b01a2d50 0x7ff0b01a3200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:32:31.702 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.701+0000 7ff0af400640 1 -- 172.21.15.47:0/1224626709 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff0b0106a10 msgr2=0x7ff0b019e9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:32:31.702 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.701+0000 7ff0af400640 1 --2- 172.21.15.47:0/1224626709 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff0b0106a10 0x7ff0b019e9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:32:31.702 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.701+0000 7ff0af400640 1 -- 172.21.15.47:0/1224626709 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0b01a3840 con 0x7ff0b01065a0 2024-03-21T10:32:31.702 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.701+0000 7ff0a7e00640 1 --2- 172.21.15.47:0/1224626709 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff0b0106a10 0x7ff0b019e9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T10:32:31.702 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.701+0000 7ff0afe00640 1 --2- 172.21.15.47:0/1224626709 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff0b01a2d50 0x7ff0b01a3200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T10:32:31.702 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.702+0000 7ff0af400640 1 --2- 172.21.15.47:0/1224626709 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff0b01065a0 0x7ff0b019e490 secure :-1 s=READY pgs=1032 cs=0 l=1 rev1=1 crypto rx=0x7ff0a0000c00 tx=0x7ff0a002c6e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:32:31.703 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.702+0000 7ff0acc00640 1 -- 172.21.15.47:0/1224626709 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0a0043020 con 0x7ff0b01065a0 2024-03-21T10:32:31.703 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.702+0000 7ff0acc00640 1 -- 172.21.15.47:0/1224626709 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7ff0a002cbd0 con 0x7ff0b01065a0 2024-03-21T10:32:31.703 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.702+0000 7ff0b6200640 1 -- 172.21.15.47:0/1224626709 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff0b01a3ad0 con 0x7ff0b01065a0 2024-03-21T10:32:31.703 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.702+0000 7ff0acc00640 1 -- 172.21.15.47:0/1224626709 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0a0003b10 con 0x7ff0b01065a0 2024-03-21T10:32:31.703 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.702+0000 7ff0b6200640 1 -- 172.21.15.47:0/1224626709 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff0b01ab320 con 0x7ff0b01065a0 2024-03-21T10:32:31.705 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.706+0000 7ff0b8df5740 1 Processor -- start 2024-03-21T10:32:31.706 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.706+0000 7ff0b8df5740 1 -- start start 2024-03-21T10:32:31.706 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.706+0000 7ff0b8df5740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55cd5ea696d0 0x55cd5ea69ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:32:31.706 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.706+0000 7ff0b8df5740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55cd5ea6a080 0x55cd5ea72630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:32:31.706 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.706+0000 7ff0b8df5740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55cd5ea72c00 0x55cd5ea75050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:32:31.706 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.706+0000 7ff0b8df5740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cd5e9cd2c0 con 0x55cd5ea6a080 2024-03-21T10:32:31.706 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.706+0000 7ff0b8df5740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cd5e975c90 con 0x55cd5ea696d0 2024-03-21T10:32:31.706 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.706+0000 7ff0b8df5740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cd5e9c9360 con 0x55cd5ea72c00 2024-03-21T10:32:31.707 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.707+0000 7ff0afe00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55cd5ea696d0 0x55cd5ea69ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:32:31.707 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.707+0000 7ff0afe00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55cd5ea696d0 0x55cd5ea69ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:33806/0 (socket says 172.21.15.47:33806) 2024-03-21T10:32:31.707 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.707+0000 7ff0afe00640 1 -- 172.21.15.47:0/3786249500 learned_addr learned my addr 172.21.15.47:0/3786249500 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:32:31.707 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.707+0000 7ff0a7e00640 1 --2- 172.21.15.47:0/3786249500 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55cd5ea72c00 0x55cd5ea75050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:32:31.707 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.707+0000 7ff0afe00640 1 -- 172.21.15.47:0/3786249500 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55cd5ea72c00 msgr2=0x55cd5ea75050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:32:31.707 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.707+0000 7ff0afe00640 1 --2- 172.21.15.47:0/3786249500 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55cd5ea72c00 0x55cd5ea75050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:32:31.707 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.707+0000 7ff0afe00640 1 -- 172.21.15.47:0/3786249500 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55cd5ea6a080 msgr2=0x55cd5ea72630 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-21T10:32:31.707 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.707+0000 7ff0afe00640 1 --2- 172.21.15.47:0/3786249500 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55cd5ea6a080 0x55cd5ea72630 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:32:31.708 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.707+0000 7ff0afe00640 1 -- 172.21.15.47:0/3786249500 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55cd5e99a080 con 0x55cd5ea696d0 2024-03-21T10:32:31.708 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.708+0000 7ff0acc00640 1 -- 172.21.15.47:0/1224626709 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 27) v1 ==== 137985+0+0 (secure 0 0 0) 0x7ff0a0046020 con 0x7ff0b01065a0 2024-03-21T10:32:31.708 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.708+0000 7ff0afe00640 1 --2- 172.21.15.47:0/3786249500 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55cd5ea696d0 0x55cd5ea69ab0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7ff0a8009f60 tx=0x7ff0a8009f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4e83f6f339851260 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:32:31.708 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.708+0000 7ff09fe00640 1 -- 172.21.15.47:0/3786249500 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0a8011640 con 0x55cd5ea696d0 2024-03-21T10:32:31.708 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.708+0000 7ff09fe00640 1 -- 172.21.15.47:0/3786249500 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7ff0a8011c90 con 0x55cd5ea696d0 2024-03-21T10:32:31.708 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.709+0000 7ff09fe00640 1 -- 172.21.15.47:0/3786249500 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0a801bd10 con 0x55cd5ea696d0 2024-03-21T10:32:31.710 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.709+0000 7ff0b8df5740 1 -- 172.21.15.47:0/3786249500 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55cd5ea696d0 msgr2=0x55cd5ea69ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:32:31.710 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.709+0000 7ff0b8df5740 1 --2- 172.21.15.47:0/3786249500 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55cd5ea696d0 0x55cd5ea69ab0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7ff0a8009f60 tx=0x7ff0a8009f90 comp rx=0 tx=0).stop 2024-03-21T10:32:31.710 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.709+0000 7ff0b8df5740 1 -- 172.21.15.47:0/3786249500 shutdown_connections 2024-03-21T10:32:31.710 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.709+0000 7ff0b8df5740 1 --2- 172.21.15.47:0/3786249500 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55cd5ea72c00 0x55cd5ea75050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:32:31.710 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.709+0000 7ff0b8df5740 1 --2- 172.21.15.47:0/3786249500 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55cd5ea6a080 0x55cd5ea72630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:32:31.710 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.709+0000 7ff0b8df5740 1 --2- 172.21.15.47:0/3786249500 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55cd5ea696d0 0x55cd5ea69ab0 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:32:31.710 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.709+0000 7ff0b8df5740 1 -- 172.21.15.47:0/3786249500 >> 172.21.15.47:0/3786249500 conn(0x55cd5ea63da0 msgr2=0x55cd5ea64180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:32:31.710 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.710+0000 7ff0b8df5740 1 -- 172.21.15.47:0/3786249500 shutdown_connections 2024-03-21T10:32:31.710 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.710+0000 7ff0b8df5740 1 -- 172.21.15.47:0/3786249500 wait complete. 2024-03-21T10:32:31.710 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.710+0000 7ff0acc00640 1 --2- 172.21.15.47:0/1224626709 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7ff0940a30a0 0x7ff0940a5560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:32:31.710 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.710+0000 7ff0acc00640 1 -- 172.21.15.47:0/1224626709 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 9562+0+0 (secure 0 0 0) 0x7ff0a00f1d00 con 0x7ff0b01065a0 2024-03-21T10:32:31.710 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.710+0000 7ff0a7e00640 1 --2- 172.21.15.47:0/1224626709 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7ff0940a30a0 0x7ff0940a5560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:32:31.710 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.710+0000 7ff0b8df5740 1 -- start start 2024-03-21T10:32:31.711 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.710+0000 7ff0b8df5740 1 Processor -- start 2024-03-21T10:32:31.713 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.711+0000 7ff0b8df5740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55cd5ea696d0 0x55cd5ea72560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:32:31.713 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.711+0000 7ff0b8df5740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55cd5ea6a080 0x55cd5ea6f8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:32:31.713 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.711+0000 7ff0b8df5740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55cd5ea72c00 0x55cd5ea6ff00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:32:31.713 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.711+0000 7ff0b8df5740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cd5ea75b70 con 0x55cd5ea6a080 2024-03-21T10:32:31.713 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.711+0000 7ff0b8df5740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cd5e9890d0 con 0x55cd5ea72c00 2024-03-21T10:32:31.714 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.711+0000 7ff0b8df5740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cd5ea75cf0 con 0x55cd5ea696d0 2024-03-21T10:32:31.714 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.711+0000 7ff0b8df5740 10 client.0 _ll_register_callbacks cb 0x55cd5e98de80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T10:32:31.714 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.711+0000 7ff0a7e00640 1 --2- 172.21.15.47:0/1224626709 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7ff0940a30a0 0x7ff0940a5560 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7ff098005fd0 tx=0x7ff098005850 comp rx=0 tx=0).ready entity=mgr.15687 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:32:31.714 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.711+0000 7ff0a7e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55cd5ea6a080 0x55cd5ea6f8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:32:31.714 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.711+0000 7ff0a7e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55cd5ea6a080 0x55cd5ea6f8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:51462/0 (socket says 172.21.15.47:51462) 2024-03-21T10:32:31.714 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.711+0000 7ff0a7e00640 1 -- 172.21.15.47:0/51976901 learned_addr learned my addr 172.21.15.47:0/51976901 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:32:31.714 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.711+0000 7ff0afe00640 1 --2- 172.21.15.47:0/51976901 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55cd5ea72c00 0x55cd5ea6ff00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:32:31.714 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.712+0000 7ff0af400640 1 --2- 172.21.15.47:0/51976901 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55cd5ea696d0 0x55cd5ea72560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:32:31.714 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.712+0000 7ff0a7e00640 1 -- 172.21.15.47:0/51976901 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55cd5ea696d0 msgr2=0x55cd5ea72560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:32:31.714 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.712+0000 7ff0a7e00640 1 --2- 172.21.15.47:0/51976901 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55cd5ea696d0 0x55cd5ea72560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:32:31.714 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.712+0000 7ff0a7e00640 1 -- 172.21.15.47:0/51976901 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55cd5ea72c00 msgr2=0x55cd5ea6ff00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:32:31.714 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.712+0000 7ff0a7e00640 1 --2- 172.21.15.47:0/51976901 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55cd5ea72c00 0x55cd5ea6ff00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:32:31.714 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.712+0000 7ff0a7e00640 1 -- 172.21.15.47:0/51976901 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55cd5e6e13f0 con 0x55cd5ea6a080 2024-03-21T10:32:31.714 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.712+0000 7ff0a7e00640 1 --2- 172.21.15.47:0/51976901 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55cd5ea6a080 0x55cd5ea6f8f0 secure :-1 s=READY pgs=1033 cs=0 l=1 rev1=1 crypto rx=0x7ff098010f70 tx=0x7ff098004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=df68eb2ac4ae7115 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:32:31.715 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.712+0000 7ff09cc00640 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-03-21T10:32:31.715 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.713+0000 7ff09cc00640 1 -- 172.21.15.47:0/51976901 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff098023070 con 0x55cd5ea6a080 2024-03-21T10:32:31.715 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.713+0000 7ff09cc00640 1 -- 172.21.15.47:0/51976901 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(99..99 src has 1..99) v4 ==== 9562+0+0 (secure 0 0 0) 0x7ff09801f860 con 0x55cd5ea6a080 2024-03-21T10:32:31.715 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.713+0000 7ff0b8df5740 1 -- 172.21.15.47:0/51976901 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ff0a8011c90 con 0x55cd5ea6a080 2024-03-21T10:32:31.715 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.713+0000 7ff09cc00640 1 -- 172.21.15.47:0/51976901 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff09802f050 con 0x55cd5ea6a080 2024-03-21T10:32:31.715 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.713+0000 7ff0b8df5740 10 client.17799 fetch_fsmap learned FSMap version 32 2024-03-21T10:32:31.715 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.713+0000 7ff0b8df5740 1 -- 172.21.15.47:0/51976901 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x55cd5e7ab7d0 con 0x55cd5ea6a080 2024-03-21T10:32:31.715 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.713+0000 7ff09cc00640 1 -- 172.21.15.47:0/51976901 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7ff0a8011c90 con 0x55cd5ea6a080 2024-03-21T10:32:31.715 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.714+0000 7ff09cc00640 1 -- 172.21.15.47:0/51976901 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 32) v1 ==== 7942+0+0 (secure 0 0 0) 0x7ff09800f040 con 0x55cd5ea6a080 2024-03-21T10:32:31.715 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.714+0000 7ff0b8df5740 10 client.17799 fetch_fsmap finished waiting for FSMap version 32 2024-03-21T10:32:31.715 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.714+0000 7ff0b8df5740 10 client.17799 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-03-21T10:32:31.715 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.714+0000 7ff0b8df5740 20 client.17799 populate_metadata read hostname 'smithi047' 2024-03-21T10:32:31.715 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.714+0000 7ff0b8df5740 1 --2- 172.21.15.47:0/51976901 >> [v2:172.21.15.100:6836/3497644162,v1:172.21.15.100:6837/3497644162] conn(0x55cd5ea82af0 0x55cd5ea84f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:32:31.715 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.714+0000 7ff0b8df5740 4 client.17799 mds_command: new command op to 34493 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T10:32:31.715 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.714+0000 7ff0b8df5740 1 -- 172.21.15.47:0/51976901 --> [v2:172.21.15.100:6836/3497644162,v1:172.21.15.100:6837/3497644162] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55cd5e9da800 con 0x55cd5ea82af0 2024-03-21T10:32:31.717 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.714+0000 7ff0af400640 1 --2- 172.21.15.47:0/51976901 >> [v2:172.21.15.100:6836/3497644162,v1:172.21.15.100:6837/3497644162] conn(0x55cd5ea82af0 0x55cd5ea84f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:32:31.717 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.715+0000 7ff0af400640 1 --2- 172.21.15.47:0/51976901 >> [v2:172.21.15.100:6836/3497644162,v1:172.21.15.100:6837/3497644162] conn(0x55cd5ea82af0 0x55cd5ea84f10 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=88a2af76bed2b373 server_cookie=abe8a56e3de9d952 in_seq=0 out_seq=0 2024-03-21T10:32:31.717 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.715+0000 7ff09cc00640 10 client.17799 ms_handle_connect on v2:172.21.15.100:6836/3497644162 2024-03-21T10:32:31.717 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.716+0000 7ff09cc00640 1 -- 172.21.15.47:0/51976901 <== mds.1 v2:172.21.15.100:6836/3497644162 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55cd5e9da800 con 0x55cd5ea82af0 2024-03-21T10:32:31.717 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.716+0000 7ff09cc00640 10 client.17799 handle_command_reply: tid=0 2024-03-21T10:32:31.717 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.716+0000 7ff0b8df5740 1 client.17799 shutdown 2024-03-21T10:32:31.717 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.716+0000 7ff0b8df5740 1 -- 172.21.15.47:0/51976901 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55cd5ea6a080 msgr2=0x55cd5ea6f8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:32:31.717 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.716+0000 7ff0b8df5740 1 --2- 172.21.15.47:0/51976901 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55cd5ea6a080 0x55cd5ea6f8f0 secure :-1 s=READY pgs=1033 cs=0 l=1 rev1=1 crypto rx=0x7ff098010f70 tx=0x7ff098004730 comp rx=0 tx=0).stop 2024-03-21T10:32:31.717 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.717+0000 7ff0b8df5740 1 -- 172.21.15.47:0/51976901 shutdown_connections 2024-03-21T10:32:31.717 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.717+0000 7ff0b8df5740 1 --2- 172.21.15.47:0/51976901 >> [v2:172.21.15.100:6836/3497644162,v1:172.21.15.100:6837/3497644162] conn(0x55cd5ea82af0 0x55cd5ea84f10 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:32:31.717 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.717+0000 7ff0b8df5740 1 --2- 172.21.15.47:0/51976901 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55cd5ea72c00 0x55cd5ea6ff00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:32:31.718 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.717+0000 7ff0b8df5740 1 --2- 172.21.15.47:0/51976901 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55cd5ea6a080 0x55cd5ea6f8f0 unknown :-1 s=CLOSED pgs=1033 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:32:31.718 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.717+0000 7ff09cc00640 0 client.17799 ms_handle_reset on v2:172.21.15.100:6836/3497644162 2024-03-21T10:32:31.718 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.717+0000 7ff0b8df5740 1 --2- 172.21.15.47:0/51976901 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55cd5ea696d0 0x55cd5ea72560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:32:31.718 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.717+0000 7ff0b8df5740 1 -- 172.21.15.47:0/51976901 >> 172.21.15.47:0/51976901 conn(0x55cd5ea63da0 msgr2=0x55cd5ea65af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:32:31.718 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.717+0000 7ff0b8df5740 1 -- 172.21.15.47:0/51976901 shutdown_connections 2024-03-21T10:32:31.718 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.717+0000 7ff0b8df5740 1 -- 172.21.15.47:0/51976901 wait complete. 2024-03-21T10:32:31.718 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.717+0000 7ff0b8df5740 20 client.17799 trim_cache size 0 max 16384 2024-03-21T10:32:31.726 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:32:31 smithi100 ceph-mon[35193]: pgmap v221: 129 pgs: 129 active+clean; 277 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-21T10:32:31.727 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:32:31 smithi100 ceph-mon[35193]: from='client.? 172.21.15.47:0/4175665670' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-21T10:32:31.730 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.731+0000 7ff0b8df5740 1 Processor -- start 2024-03-21T10:32:31.731 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.731+0000 7ff0b8df5740 1 -- start start 2024-03-21T10:32:31.731 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.731+0000 7ff0b8df5740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55cd5ea696d0 0x55cd5ea72020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:32:31.731 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.731+0000 7ff0b8df5740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55cd5ea6a080 0x55cd5ea72560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:32:31.731 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.731+0000 7ff0b8df5740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55cd5ea72c00 0x55cd5ea6cbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:32:31.731 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.731+0000 7ff0b8df5740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0b01b4fb0 con 0x55cd5ea72c00 2024-03-21T10:32:31.731 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.731+0000 7ff0b8df5740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0b01b6930 con 0x55cd5ea6a080 2024-03-21T10:32:31.731 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.731+0000 7ff0b8df5740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0b01b5610 con 0x55cd5ea696d0 2024-03-21T10:32:31.732 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.732+0000 7ff0afe00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55cd5ea72c00 0x55cd5ea6cbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:32:31.732 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.732+0000 7ff0afe00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55cd5ea72c00 0x55cd5ea6cbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:51472/0 (socket says 172.21.15.47:51472) 2024-03-21T10:32:31.732 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.732+0000 7ff0afe00640 1 -- 172.21.15.47:0/3158413993 learned_addr learned my addr 172.21.15.47:0/3158413993 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:32:31.732 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.732+0000 7ff0a7e00640 1 --2- 172.21.15.47:0/3158413993 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55cd5ea6a080 0x55cd5ea72560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:32:31.733 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.732+0000 7ff0af400640 1 --2- 172.21.15.47:0/3158413993 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55cd5ea696d0 0x55cd5ea72020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:32:31.733 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.732+0000 7ff0afe00640 1 -- 172.21.15.47:0/3158413993 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55cd5ea696d0 msgr2=0x55cd5ea72020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:32:31.733 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.732+0000 7ff0afe00640 1 --2- 172.21.15.47:0/3158413993 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55cd5ea696d0 0x55cd5ea72020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:32:31.733 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.732+0000 7ff0afe00640 1 -- 172.21.15.47:0/3158413993 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55cd5ea6a080 msgr2=0x55cd5ea72560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:32:31.733 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.732+0000 7ff0afe00640 1 --2- 172.21.15.47:0/3158413993 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55cd5ea6a080 0x55cd5ea72560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:32:31.733 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.732+0000 7ff0afe00640 1 -- 172.21.15.47:0/3158413993 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55cd5ea6dea0 con 0x55cd5ea72c00 2024-03-21T10:32:31.733 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.732+0000 7ff0afe00640 1 --2- 172.21.15.47:0/3158413993 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55cd5ea72c00 0x55cd5ea6cbd0 secure :-1 s=READY pgs=1034 cs=0 l=1 rev1=1 crypto rx=0x7ff0a800fc80 tx=0x7ff0a8009990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=75267a470db36def server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:32:31.733 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.733+0000 7ff09f400640 1 -- 172.21.15.47:0/3158413993 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0a8004c00 con 0x55cd5ea72c00 2024-03-21T10:32:31.733 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.733+0000 7ff09f400640 1 -- 172.21.15.47:0/3158413993 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7ff0a8009be0 con 0x55cd5ea72c00 2024-03-21T10:32:31.733 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.733+0000 7ff09f400640 1 -- 172.21.15.47:0/3158413993 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0a8004020 con 0x55cd5ea72c00 2024-03-21T10:32:31.736 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.733+0000 7ff0b8df5740 1 -- 172.21.15.47:0/3158413993 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55cd5ea72c00 msgr2=0x55cd5ea6cbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:32:31.736 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.733+0000 7ff0b8df5740 1 --2- 172.21.15.47:0/3158413993 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55cd5ea72c00 0x55cd5ea6cbd0 secure :-1 s=READY pgs=1034 cs=0 l=1 rev1=1 crypto rx=0x7ff0a800fc80 tx=0x7ff0a8009990 comp rx=0 tx=0).stop 2024-03-21T10:32:31.736 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.734+0000 7ff0b8df5740 1 -- 172.21.15.47:0/3158413993 shutdown_connections 2024-03-21T10:32:31.736 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.734+0000 7ff0b8df5740 1 --2- 172.21.15.47:0/3158413993 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55cd5ea72c00 0x55cd5ea6cbd0 unknown :-1 s=CLOSED pgs=1034 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:32:31.736 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.734+0000 7ff0b8df5740 1 --2- 172.21.15.47:0/3158413993 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55cd5ea6a080 0x55cd5ea72560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:32:31.736 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.734+0000 7ff0b8df5740 1 --2- 172.21.15.47:0/3158413993 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55cd5ea696d0 0x55cd5ea72020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:32:31.736 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.734+0000 7ff0b8df5740 1 -- 172.21.15.47:0/3158413993 >> 172.21.15.47:0/3158413993 conn(0x55cd5ea63da0 msgr2=0x55cd5ea62eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:32:31.736 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.734+0000 7ff0b8df5740 1 -- 172.21.15.47:0/3158413993 shutdown_connections 2024-03-21T10:32:31.736 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.734+0000 7ff0b8df5740 1 -- 172.21.15.47:0/3158413993 wait complete. 2024-03-21T10:32:31.736 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.734+0000 7ff0b8df5740 1 -- start start 2024-03-21T10:32:31.736 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.734+0000 7ff0b8df5740 1 Processor -- start 2024-03-21T10:32:31.736 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.735+0000 7ff0b8df5740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55cd5ea696d0 0x55cd5ea7a770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:32:31.737 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.735+0000 7ff0b8df5740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55cd5ea6a080 0x55cd5ea7acb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:32:31.737 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.735+0000 7ff0b8df5740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55cd5ea72c00 0x55cd5ea78bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:32:31.737 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.735+0000 7ff0b8df5740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0b01b62d0 con 0x55cd5ea6a080 2024-03-21T10:32:31.737 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.735+0000 7ff0b8df5740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0b01b5c70 con 0x55cd5ea696d0 2024-03-21T10:32:31.737 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.735+0000 7ff0b8df5740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0b01b3190 con 0x55cd5ea72c00 2024-03-21T10:32:31.737 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.735+0000 7ff0b8df5740 10 client.0 _ll_register_callbacks cb 0x7ff098016070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T10:32:31.737 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.735+0000 7ff0a7e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55cd5ea6a080 0x55cd5ea7acb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:32:31.737 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.735+0000 7ff0a7e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55cd5ea6a080 0x55cd5ea7acb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:51474/0 (socket says 172.21.15.47:51474) 2024-03-21T10:32:31.737 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.735+0000 7ff0a7e00640 1 -- 172.21.15.47:0/1634715764 learned_addr learned my addr 172.21.15.47:0/1634715764 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:32:31.737 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.735+0000 7ff0af400640 1 --2- 172.21.15.47:0/1634715764 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55cd5ea696d0 0x55cd5ea7a770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:32:31.737 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.735+0000 7ff0afe00640 1 --2- 172.21.15.47:0/1634715764 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55cd5ea72c00 0x55cd5ea78bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:32:31.737 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.735+0000 7ff0a7e00640 1 -- 172.21.15.47:0/1634715764 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55cd5ea72c00 msgr2=0x55cd5ea78bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:32:31.737 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.735+0000 7ff0a7e00640 1 --2- 172.21.15.47:0/1634715764 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55cd5ea72c00 0x55cd5ea78bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:32:31.737 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.735+0000 7ff0a7e00640 1 -- 172.21.15.47:0/1634715764 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55cd5ea696d0 msgr2=0x55cd5ea7a770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:32:31.737 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.735+0000 7ff0a7e00640 1 --2- 172.21.15.47:0/1634715764 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55cd5ea696d0 0x55cd5ea7a770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:32:31.738 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.735+0000 7ff0a7e00640 1 -- 172.21.15.47:0/1634715764 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55cd5ea793d0 con 0x55cd5ea6a080 2024-03-21T10:32:31.738 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.735+0000 7ff0af400640 1 --2- 172.21.15.47:0/1634715764 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55cd5ea696d0 0x55cd5ea7a770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T10:32:31.738 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.735+0000 7ff0afe00640 1 --2- 172.21.15.47:0/1634715764 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55cd5ea72c00 0x55cd5ea78bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T10:32:31.738 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.736+0000 7ff0a7e00640 1 --2- 172.21.15.47:0/1634715764 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55cd5ea6a080 0x55cd5ea7acb0 secure :-1 s=READY pgs=1035 cs=0 l=1 rev1=1 crypto rx=0x7ff09800c010 tx=0x7ff098010860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=adbdbee1d18992bc server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:32:31.738 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.736+0000 7ff09cc00640 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-03-21T10:32:31.738 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.736+0000 7ff09cc00640 1 -- 172.21.15.47:0/1634715764 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff098023040 con 0x55cd5ea6a080 2024-03-21T10:32:31.738 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.736+0000 7ff09cc00640 1 -- 172.21.15.47:0/1634715764 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(99..99 src has 1..99) v4 ==== 9562+0+0 (secure 0 0 0) 0x7ff098027660 con 0x55cd5ea6a080 2024-03-21T10:32:31.738 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.736+0000 7ff0b8df5740 1 -- 172.21.15.47:0/1634715764 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ff0a8009be0 con 0x55cd5ea6a080 2024-03-21T10:32:31.738 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.736+0000 7ff09cc00640 1 -- 172.21.15.47:0/1634715764 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff098044050 con 0x55cd5ea6a080 2024-03-21T10:32:31.738 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.736+0000 7ff0b8df5740 10 client.17805 fetch_fsmap learned FSMap version 32 2024-03-21T10:32:31.738 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.736+0000 7ff0b8df5740 1 -- 172.21.15.47:0/1634715764 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x55cd5eadef30 con 0x55cd5ea6a080 2024-03-21T10:32:31.739 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:32:31 smithi157 ceph-mon[35581]: pgmap v221: 129 pgs: 129 active+clean; 277 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-21T10:32:31.739 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:32:31 smithi157 ceph-mon[35581]: from='client.? 172.21.15.47:0/4175665670' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-21T10:32:31.740 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.737+0000 7ff09cc00640 1 -- 172.21.15.47:0/1634715764 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7ff0a8009be0 con 0x55cd5ea6a080 2024-03-21T10:32:31.740 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.737+0000 7ff09cc00640 1 -- 172.21.15.47:0/1634715764 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 32) v1 ==== 7942+0+0 (secure 0 0 0) 0x7ff09801e520 con 0x55cd5ea6a080 2024-03-21T10:32:31.740 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.737+0000 7ff0b8df5740 10 client.17805 fetch_fsmap finished waiting for FSMap version 32 2024-03-21T10:32:31.741 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.737+0000 7ff0b8df5740 10 client.17805 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-03-21T10:32:31.741 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.737+0000 7ff0b8df5740 20 client.17805 populate_metadata read hostname 'smithi047' 2024-03-21T10:32:31.741 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.737+0000 7ff0b8df5740 1 --2- 172.21.15.47:0/1634715764 >> [v2:172.21.15.100:6836/3497644162,v1:172.21.15.100:6837/3497644162] conn(0x55cd5eadf570 0x55cd5eae1990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:32:31.741 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.737+0000 7ff0b8df5740 4 client.17805 mds_command: new command op to 34493 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-21T10:32:31.741 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.737+0000 7ff0b8df5740 1 -- 172.21.15.47:0/1634715764 --> [v2:172.21.15.100:6836/3497644162,v1:172.21.15.100:6837/3497644162] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55cd5e9d4b50 con 0x55cd5eadf570 2024-03-21T10:32:31.741 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.737+0000 7ff0af400640 1 --2- 172.21.15.47:0/1634715764 >> [v2:172.21.15.100:6836/3497644162,v1:172.21.15.100:6837/3497644162] conn(0x55cd5eadf570 0x55cd5eae1990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:32:31.741 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.738+0000 7ff0af400640 1 --2- 172.21.15.47:0/1634715764 >> [v2:172.21.15.100:6836/3497644162,v1:172.21.15.100:6837/3497644162] conn(0x55cd5eadf570 0x55cd5eae1990 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=b740cb1a20abccad server_cookie=4dc00076b9be1c1a in_seq=0 out_seq=0 2024-03-21T10:32:31.741 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.738+0000 7ff09cc00640 10 client.17805 ms_handle_connect on v2:172.21.15.100:6836/3497644162 2024-03-21T10:32:31.741 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.738+0000 7ff09cc00640 1 -- 172.21.15.47:0/1634715764 <== mds.1 v2:172.21.15.100:6836/3497644162 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55cd5e9d4b50 con 0x55cd5eadf570 2024-03-21T10:32:31.741 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.738+0000 7ff09cc00640 10 client.17805 handle_command_reply: tid=0 2024-03-21T10:32:31.741 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.738+0000 7ff0b8df5740 1 client.17805 shutdown 2024-03-21T10:32:31.741 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.738+0000 7ff0b8df5740 1 -- 172.21.15.47:0/1634715764 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55cd5ea6a080 msgr2=0x55cd5ea7acb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:32:31.741 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.738+0000 7ff0b8df5740 1 --2- 172.21.15.47:0/1634715764 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55cd5ea6a080 0x55cd5ea7acb0 secure :-1 s=READY pgs=1035 cs=0 l=1 rev1=1 crypto rx=0x7ff09800c010 tx=0x7ff098010860 comp rx=0 tx=0).stop 2024-03-21T10:32:31.741 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.738+0000 7ff0b8df5740 1 -- 172.21.15.47:0/1634715764 shutdown_connections 2024-03-21T10:32:31.742 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.738+0000 7ff0b8df5740 1 --2- 172.21.15.47:0/1634715764 >> [v2:172.21.15.100:6836/3497644162,v1:172.21.15.100:6837/3497644162] conn(0x55cd5eadf570 0x55cd5eae1990 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:32:31.742 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.738+0000 7ff0b8df5740 1 --2- 172.21.15.47:0/1634715764 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55cd5ea72c00 0x55cd5ea78bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:32:31.742 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.738+0000 7ff0b8df5740 1 --2- 172.21.15.47:0/1634715764 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55cd5ea6a080 0x55cd5ea7acb0 unknown :-1 s=CLOSED pgs=1035 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:32:31.742 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.738+0000 7ff0b8df5740 1 --2- 172.21.15.47:0/1634715764 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55cd5ea696d0 0x55cd5ea7a770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:32:31.742 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.738+0000 7ff09cc00640 0 client.17805 ms_handle_reset on v2:172.21.15.100:6836/3497644162 2024-03-21T10:32:31.742 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.738+0000 7ff0b8df5740 1 -- 172.21.15.47:0/1634715764 >> 172.21.15.47:0/1634715764 conn(0x55cd5ea63da0 msgr2=0x55cd5ea65080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:32:31.742 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.739+0000 7ff0b8df5740 1 -- 172.21.15.47:0/1634715764 shutdown_connections 2024-03-21T10:32:31.742 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.739+0000 7ff0b8df5740 1 -- 172.21.15.47:0/1634715764 wait complete. 2024-03-21T10:32:31.742 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.739+0000 7ff0b8df5740 20 client.17805 trim_cache size 0 max 16384 2024-03-21T10:32:31.742 INFO:teuthology.orchestra.run.smithi047.stdout:[] 2024-03-21T10:32:31.743 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.739+0000 7ff0b6200640 1 -- 172.21.15.47:0/1224626709 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7ff0940a30a0 msgr2=0x7ff0940a5560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:32:31.743 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.739+0000 7ff0b6200640 1 --2- 172.21.15.47:0/1224626709 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7ff0940a30a0 0x7ff0940a5560 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7ff098005fd0 tx=0x7ff098005850 comp rx=0 tx=0).stop 2024-03-21T10:32:31.743 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.739+0000 7ff0b6200640 1 -- 172.21.15.47:0/1224626709 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff0b01065a0 msgr2=0x7ff0b019e490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:32:31.743 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.739+0000 7ff0b6200640 1 --2- 172.21.15.47:0/1224626709 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff0b01065a0 0x7ff0b019e490 secure :-1 s=READY pgs=1032 cs=0 l=1 rev1=1 crypto rx=0x7ff0a0000c00 tx=0x7ff0a002c6e0 comp rx=0 tx=0).stop 2024-03-21T10:32:31.743 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.740+0000 7ff0b6200640 1 -- 172.21.15.47:0/1224626709 shutdown_connections 2024-03-21T10:32:31.743 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.740+0000 7ff0b6200640 1 --2- 172.21.15.47:0/1224626709 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff0b01a2d50 0x7ff0b01a3200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:32:31.743 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.740+0000 7ff0b6200640 1 --2- 172.21.15.47:0/1224626709 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7ff0940a30a0 0x7ff0940a5560 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:32:31.743 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.740+0000 7ff0b6200640 1 --2- 172.21.15.47:0/1224626709 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff0b0106a10 0x7ff0b019e9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:32:31.744 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.740+0000 7ff0b6200640 1 --2- 172.21.15.47:0/1224626709 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff0b01065a0 0x7ff0b019e490 unknown :-1 s=CLOSED pgs=1032 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:32:31.744 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.740+0000 7ff0b6200640 1 -- 172.21.15.47:0/1224626709 >> 172.21.15.47:0/1224626709 conn(0x7ff0b0077580 msgr2=0x7ff0b0103080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:32:31.744 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.740+0000 7ff0b6200640 1 -- 172.21.15.47:0/1224626709 shutdown_connections 2024-03-21T10:32:31.744 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.740+0000 7ff0b6200640 1 -- 172.21.15.47:0/1224626709 wait complete. 2024-03-21T10:32:31.766 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-21T10:32:31.766 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-03-21T10:32:31.975 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.975+0000 7fe761600640 1 -- 172.21.15.47:0/80974722 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe75c1038b0 msgr2=0x7fe75c103d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:32:31.975 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.975+0000 7fe761600640 1 --2- 172.21.15.47:0/80974722 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe75c1038b0 0x7fe75c103d30 secure :-1 s=READY pgs=1036 cs=0 l=1 rev1=1 crypto rx=0x7fe748009930 tx=0x7fe74802bc20 comp rx=0 tx=0).stop 2024-03-21T10:32:31.975 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.975+0000 7fe761600640 1 -- 172.21.15.47:0/80974722 shutdown_connections 2024-03-21T10:32:31.976 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.975+0000 7fe761600640 1 --2- 172.21.15.47:0/80974722 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe75c104270 0x7fe75c110780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:32:31.976 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.975+0000 7fe761600640 1 --2- 172.21.15.47:0/80974722 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe75c1038b0 0x7fe75c103d30 unknown :-1 s=CLOSED pgs=1036 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:32:31.976 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.975+0000 7fe761600640 1 --2- 172.21.15.47:0/80974722 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe75c1098c0 0x7fe75c109ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:32:31.976 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.975+0000 7fe761600640 1 -- 172.21.15.47:0/80974722 >> 172.21.15.47:0/80974722 conn(0x7fe75c0ff360 msgr2=0x7fe75c101780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:32:31.976 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.975+0000 7fe761600640 1 -- 172.21.15.47:0/80974722 shutdown_connections 2024-03-21T10:32:31.976 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.975+0000 7fe761600640 1 -- 172.21.15.47:0/80974722 wait complete. 2024-03-21T10:32:31.978 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.976+0000 7fe761600640 1 Processor -- start 2024-03-21T10:32:31.978 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.976+0000 7fe761600640 1 -- start start 2024-03-21T10:32:31.978 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.977+0000 7fe761600640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe75c1038b0 0x7fe75c19a0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:32:31.978 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.977+0000 7fe761600640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe75c104270 0x7fe75c19a600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:32:31.978 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.977+0000 7fe761600640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe75c1098c0 0x7fe75c19e980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:32:31.978 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.977+0000 7fe761600640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe75c113b80 con 0x7fe75c1098c0 2024-03-21T10:32:31.978 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.977+0000 7fe761600640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe75c113a00 con 0x7fe75c104270 2024-03-21T10:32:31.978 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.977+0000 7fe761600640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe75c113d00 con 0x7fe75c1038b0 2024-03-21T10:32:31.978 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.977+0000 7fe75b400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe75c1098c0 0x7fe75c19e980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:32:31.978 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.977+0000 7fe75b400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe75c1098c0 0x7fe75c19e980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:50966/0 (socket says 172.21.15.47:50966) 2024-03-21T10:32:31.979 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.977+0000 7fe75b400640 1 -- 172.21.15.47:0/848798510 learned_addr learned my addr 172.21.15.47:0/848798510 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:32:31.979 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.977+0000 7fe75a000640 1 --2- 172.21.15.47:0/848798510 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe75c104270 0x7fe75c19a600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:32:31.979 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.977+0000 7fe75aa00640 1 --2- 172.21.15.47:0/848798510 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe75c1038b0 0x7fe75c19a0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:32:31.979 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.978+0000 7fe75b400640 1 -- 172.21.15.47:0/848798510 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe75c1038b0 msgr2=0x7fe75c19a0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:32:31.979 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.978+0000 7fe75b400640 1 --2- 172.21.15.47:0/848798510 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe75c1038b0 0x7fe75c19a0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:32:31.979 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.978+0000 7fe75b400640 1 -- 172.21.15.47:0/848798510 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe75c104270 msgr2=0x7fe75c19a600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:32:31.979 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.978+0000 7fe75b400640 1 --2- 172.21.15.47:0/848798510 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe75c104270 0x7fe75c19a600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:32:31.979 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.978+0000 7fe75b400640 1 -- 172.21.15.47:0/848798510 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe75c19f0b0 con 0x7fe75c1098c0 2024-03-21T10:32:31.979 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.978+0000 7fe75aa00640 1 --2- 172.21.15.47:0/848798510 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe75c1038b0 0x7fe75c19a0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T10:32:31.979 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.978+0000 7fe75b400640 1 --2- 172.21.15.47:0/848798510 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe75c1098c0 0x7fe75c19e980 secure :-1 s=READY pgs=1037 cs=0 l=1 rev1=1 crypto rx=0x7fe75400eaa0 tx=0x7fe75400ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:32:31.979 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.978+0000 7fe753400640 1 -- 172.21.15.47:0/848798510 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe75400cdb0 con 0x7fe75c1098c0 2024-03-21T10:32:31.979 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.978+0000 7fe753400640 1 -- 172.21.15.47:0/848798510 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fe754005020 con 0x7fe75c1098c0 2024-03-21T10:32:31.979 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.978+0000 7fe753400640 1 -- 172.21.15.47:0/848798510 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7540116d0 con 0x7fe75c1098c0 2024-03-21T10:32:31.980 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.978+0000 7fe761600640 1 -- 172.21.15.47:0/848798510 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe75c19f3a0 con 0x7fe75c1098c0 2024-03-21T10:32:31.980 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.979+0000 7fe761600640 1 -- 172.21.15.47:0/848798510 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe75c1a6c30 con 0x7fe75c1098c0 2024-03-21T10:32:31.982 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.982+0000 7fe7643a1740 1 Processor -- start 2024-03-21T10:32:31.982 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.982+0000 7fe753400640 1 -- 172.21.15.47:0/848798510 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 27) v1 ==== 137985+0+0 (secure 0 0 0) 0x7fe7540051d0 con 0x7fe75c1098c0 2024-03-21T10:32:31.982 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.982+0000 7fe7643a1740 1 -- start start 2024-03-21T10:32:31.982 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.983+0000 7fe7643a1740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5637f9f3b6d0 0x5637f9f3bab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:32:31.982 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.983+0000 7fe7643a1740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5637f9f3c080 0x5637f9f44630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:32:31.982 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.983+0000 7fe7643a1740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5637f9f44c00 0x5637f9f47050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:32:31.982 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.983+0000 7fe7643a1740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x5637f9e9f2c0 con 0x5637f9f44c00 2024-03-21T10:32:31.982 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.983+0000 7fe7643a1740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x5637f9e47c90 con 0x5637f9f3b6d0 2024-03-21T10:32:31.983 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.983+0000 7fe7643a1740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x5637f9e9b360 con 0x5637f9f3c080 2024-03-21T10:32:31.984 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.983+0000 7fe75a000640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5637f9f44c00 0x5637f9f47050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:32:31.984 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.983+0000 7fe75a000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5637f9f3b6d0 0x5637f9f3bab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:32:31.984 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.983+0000 7fe75a000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5637f9f3b6d0 0x5637f9f3bab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:57168/0 (socket says 172.21.15.47:57168) 2024-03-21T10:32:31.984 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.983+0000 7fe75a000640 1 -- 172.21.15.47:0/1955981482 learned_addr learned my addr 172.21.15.47:0/1955981482 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:32:31.984 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.983+0000 7fe75aa00640 1 --2- 172.21.15.47:0/1955981482 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5637f9f3c080 0x5637f9f44630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:32:31.984 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.984+0000 7fe75a000640 1 -- 172.21.15.47:0/1955981482 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5637f9f3c080 msgr2=0x5637f9f44630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:32:31.984 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.984+0000 7fe75a000640 1 --2- 172.21.15.47:0/1955981482 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5637f9f3c080 0x5637f9f44630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:32:31.985 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.984+0000 7fe75a000640 1 -- 172.21.15.47:0/1955981482 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5637f9f44c00 msgr2=0x5637f9f47050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:32:31.985 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.984+0000 7fe75a000640 1 --2- 172.21.15.47:0/1955981482 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5637f9f44c00 0x5637f9f47050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:32:31.985 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.984+0000 7fe75a000640 1 -- 172.21.15.47:0/1955981482 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5637f9e6c080 con 0x5637f9f3b6d0 2024-03-21T10:32:31.985 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.984+0000 7fe753400640 1 --2- 172.21.15.47:0/848798510 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7fe7400a3090 0x7fe7400a5550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:32:31.985 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.984+0000 7fe753400640 1 -- 172.21.15.47:0/848798510 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 9562+0+0 (secure 0 0 0) 0x7fe7540cfe40 con 0x7fe75c1098c0 2024-03-21T10:32:31.985 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.984+0000 7fe75a000640 1 --2- 172.21.15.47:0/1955981482 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5637f9f3b6d0 0x5637f9f3bab0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fe748009900 tx=0x7fe748005fd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4ba7fe9cd5db4496 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:32:31.985 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.984+0000 7fe75aa00640 1 --2- 172.21.15.47:0/848798510 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7fe7400a3090 0x7fe7400a5550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:32:31.985 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.984+0000 7fe747400640 1 -- 172.21.15.47:0/1955981482 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe74802c7d0 con 0x5637f9f3b6d0 2024-03-21T10:32:31.985 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.985+0000 7fe747400640 1 -- 172.21.15.47:0/1955981482 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fe748005040 con 0x5637f9f3b6d0 2024-03-21T10:32:31.985 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.985+0000 7fe747400640 1 -- 172.21.15.47:0/1955981482 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7480048c0 con 0x5637f9f3b6d0 2024-03-21T10:32:31.987 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.985+0000 7fe7643a1740 1 -- 172.21.15.47:0/1955981482 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5637f9f3b6d0 msgr2=0x5637f9f3bab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:32:31.988 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.985+0000 7fe75aa00640 1 --2- 172.21.15.47:0/848798510 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7fe7400a3090 0x7fe7400a5550 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x5637f9f36e10 tx=0x7fe74c005f90 comp rx=0 tx=0).ready entity=mgr.15687 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:32:31.988 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.985+0000 7fe7643a1740 1 --2- 172.21.15.47:0/1955981482 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5637f9f3b6d0 0x5637f9f3bab0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fe748009900 tx=0x7fe748005fd0 comp rx=0 tx=0).stop 2024-03-21T10:32:31.988 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.986+0000 7fe7643a1740 1 -- 172.21.15.47:0/1955981482 shutdown_connections 2024-03-21T10:32:31.988 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.986+0000 7fe7643a1740 1 --2- 172.21.15.47:0/1955981482 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5637f9f44c00 0x5637f9f47050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:32:31.988 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.986+0000 7fe7643a1740 1 --2- 172.21.15.47:0/1955981482 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5637f9f3c080 0x5637f9f44630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:32:31.988 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.986+0000 7fe7643a1740 1 --2- 172.21.15.47:0/1955981482 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5637f9f3b6d0 0x5637f9f3bab0 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:32:31.988 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.986+0000 7fe7643a1740 1 -- 172.21.15.47:0/1955981482 >> 172.21.15.47:0/1955981482 conn(0x5637f9f35da0 msgr2=0x5637f9f36180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:32:31.988 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.986+0000 7fe7643a1740 1 -- 172.21.15.47:0/1955981482 shutdown_connections 2024-03-21T10:32:31.988 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.986+0000 7fe7643a1740 1 -- 172.21.15.47:0/1955981482 wait complete. 2024-03-21T10:32:31.988 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.986+0000 7fe7643a1740 1 -- start start 2024-03-21T10:32:31.988 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.986+0000 7fe7643a1740 1 Processor -- start 2024-03-21T10:32:31.989 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.987+0000 7fe7643a1740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5637f9f3b6d0 0x5637f9f444e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:32:31.989 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.987+0000 7fe7643a1740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5637f9f3c080 0x5637f9f41870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:32:31.989 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.987+0000 7fe7643a1740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5637f9f44c00 0x5637f9f41dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:32:31.989 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.987+0000 7fe7643a1740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x5637f9f47b10 con 0x5637f9f44c00 2024-03-21T10:32:31.989 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.987+0000 7fe7643a1740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x5637f9e5b0d0 con 0x5637f9f3c080 2024-03-21T10:32:31.989 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.987+0000 7fe7643a1740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x5637f9f47c90 con 0x5637f9f3b6d0 2024-03-21T10:32:31.989 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.987+0000 7fe7643a1740 10 client.0 _ll_register_callbacks cb 0x5637f9e5fe80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T10:32:31.989 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.987+0000 7fe75b400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5637f9f44c00 0x5637f9f41dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:32:31.989 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.987+0000 7fe75a000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5637f9f3c080 0x5637f9f41870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:32:31.989 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.987+0000 7fe75b400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5637f9f44c00 0x5637f9f41dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:50986/0 (socket says 172.21.15.47:50986) 2024-03-21T10:32:31.989 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.987+0000 7fe75b400640 1 -- 172.21.15.47:0/2964593994 learned_addr learned my addr 172.21.15.47:0/2964593994 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:32:31.989 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.988+0000 7fe75aa00640 1 --2- 172.21.15.47:0/2964593994 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5637f9f3b6d0 0x5637f9f444e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:32:31.989 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.988+0000 7fe75b400640 1 -- 172.21.15.47:0/2964593994 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5637f9f3b6d0 msgr2=0x5637f9f444e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:32:31.990 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.988+0000 7fe75b400640 1 --2- 172.21.15.47:0/2964593994 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5637f9f3b6d0 0x5637f9f444e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:32:31.990 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.988+0000 7fe75b400640 1 -- 172.21.15.47:0/2964593994 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5637f9f3c080 msgr2=0x5637f9f41870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:32:31.990 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.988+0000 7fe75b400640 1 --2- 172.21.15.47:0/2964593994 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5637f9f3c080 0x5637f9f41870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:32:31.990 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.988+0000 7fe75b400640 1 -- 172.21.15.47:0/2964593994 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5637f9bb33f0 con 0x5637f9f44c00 2024-03-21T10:32:31.990 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.988+0000 7fe75b400640 1 --2- 172.21.15.47:0/2964593994 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5637f9f44c00 0x5637f9f41dd0 secure :-1 s=READY pgs=1038 cs=0 l=1 rev1=1 crypto rx=0x7fe754098790 tx=0x7fe75409c5f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f7124dc7491b61bd server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:32:31.990 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.988+0000 7fe73be00640 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-03-21T10:32:31.992 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.988+0000 7fe73be00640 1 -- 172.21.15.47:0/2964593994 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe75401f030 con 0x5637f9f44c00 2024-03-21T10:32:31.993 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.989+0000 7fe73be00640 1 -- 172.21.15.47:0/2964593994 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(99..99 src has 1..99) v4 ==== 9562+0+0 (secure 0 0 0) 0x7fe7540bab30 con 0x5637f9f44c00 2024-03-21T10:32:31.993 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.989+0000 7fe7643a1740 1 -- 172.21.15.47:0/2964593994 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fe748005040 con 0x5637f9f44c00 2024-03-21T10:32:31.993 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.989+0000 7fe73be00640 1 -- 172.21.15.47:0/2964593994 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe7540a47d0 con 0x5637f9f44c00 2024-03-21T10:32:31.993 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.989+0000 7fe7643a1740 10 client.17817 fetch_fsmap learned FSMap version 32 2024-03-21T10:32:31.993 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.989+0000 7fe7643a1740 1 -- 172.21.15.47:0/2964593994 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x5637f9c7d7d0 con 0x5637f9f44c00 2024-03-21T10:32:31.993 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.989+0000 7fe73be00640 1 -- 172.21.15.47:0/2964593994 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fe7540b7030 con 0x5637f9f44c00 2024-03-21T10:32:31.993 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.989+0000 7fe73be00640 1 -- 172.21.15.47:0/2964593994 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 32) v1 ==== 7942+0+0 (secure 0 0 0) 0x7fe75400f040 con 0x5637f9f44c00 2024-03-21T10:32:31.994 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.989+0000 7fe7643a1740 10 client.17817 fetch_fsmap finished waiting for FSMap version 32 2024-03-21T10:32:31.994 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.989+0000 7fe7643a1740 10 client.17817 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-21T10:32:31.994 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.990+0000 7fe7643a1740 20 client.17817 populate_metadata read hostname 'smithi047' 2024-03-21T10:32:31.994 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.990+0000 7fe7643a1740 1 --2- 172.21.15.47:0/2964593994 >> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] conn(0x5637f9f54a70 0x5637f9f56e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:32:31.994 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.990+0000 7fe7643a1740 4 client.17817 mds_command: new command op to 34535 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T10:32:31.994 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.990+0000 7fe7643a1740 1 -- 172.21.15.47:0/2964593994 --> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5637f9eac800 con 0x5637f9f54a70 2024-03-21T10:32:31.994 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.990+0000 7fe75aa00640 1 --2- 172.21.15.47:0/2964593994 >> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] conn(0x5637f9f54a70 0x5637f9f56e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:32:31.994 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.991+0000 7fe75aa00640 1 --2- 172.21.15.47:0/2964593994 >> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] conn(0x5637f9f54a70 0x5637f9f56e60 crc :-1 s=READY pgs=171 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=650d3de5fc824662 server_cookie=ba1e48994bea54f9 in_seq=0 out_seq=0 2024-03-21T10:32:31.994 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.991+0000 7fe73be00640 10 client.17817 ms_handle_connect on v2:172.21.15.100:6838/4173168440 2024-03-21T10:32:31.995 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.992+0000 7fe73be00640 1 -- 172.21.15.47:0/2964593994 <== mds.0 v2:172.21.15.100:6838/4173168440 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x5637f9eac800 con 0x5637f9f54a70 2024-03-21T10:32:31.995 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.992+0000 7fe73be00640 10 client.17817 handle_command_reply: tid=0 2024-03-21T10:32:31.995 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.992+0000 7fe7643a1740 1 client.17817 shutdown 2024-03-21T10:32:31.995 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.992+0000 7fe7643a1740 1 -- 172.21.15.47:0/2964593994 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5637f9f44c00 msgr2=0x5637f9f41dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:32:31.995 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.992+0000 7fe7643a1740 1 --2- 172.21.15.47:0/2964593994 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5637f9f44c00 0x5637f9f41dd0 secure :-1 s=READY pgs=1038 cs=0 l=1 rev1=1 crypto rx=0x7fe754098790 tx=0x7fe75409c5f0 comp rx=0 tx=0).stop 2024-03-21T10:32:31.995 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.992+0000 7fe7643a1740 1 -- 172.21.15.47:0/2964593994 shutdown_connections 2024-03-21T10:32:31.995 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.992+0000 7fe7643a1740 1 --2- 172.21.15.47:0/2964593994 >> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] conn(0x5637f9f54a70 0x5637f9f56e60 crc :-1 s=READY pgs=171 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:32:31.995 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.992+0000 7fe7643a1740 1 --2- 172.21.15.47:0/2964593994 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5637f9f44c00 0x5637f9f41dd0 unknown :-1 s=CLOSED pgs=1038 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:32:31.995 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.992+0000 7fe7643a1740 1 --2- 172.21.15.47:0/2964593994 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5637f9f3c080 0x5637f9f41870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:32:31.996 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.992+0000 7fe7643a1740 1 --2- 172.21.15.47:0/2964593994 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5637f9f3b6d0 0x5637f9f444e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:32:31.996 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.992+0000 7fe7643a1740 1 -- 172.21.15.47:0/2964593994 >> 172.21.15.47:0/2964593994 conn(0x5637f9f35da0 msgr2=0x5637f9f37af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:32:31.996 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.992+0000 7fe73be00640 0 client.17817 ms_handle_reset on v2:172.21.15.100:6838/4173168440 2024-03-21T10:32:31.996 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.992+0000 7fe7643a1740 1 -- 172.21.15.47:0/2964593994 shutdown_connections 2024-03-21T10:32:31.996 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.992+0000 7fe7643a1740 1 -- 172.21.15.47:0/2964593994 wait complete. 2024-03-21T10:32:31.996 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:31.993+0000 7fe7643a1740 20 client.17817 trim_cache size 0 max 16384 2024-03-21T10:32:32.006 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:32.006+0000 7fe7643a1740 1 Processor -- start 2024-03-21T10:32:32.006 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:32.006+0000 7fe7643a1740 1 -- start start 2024-03-21T10:32:32.007 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:32.007+0000 7fe7643a1740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5637f9f3b6d0 0x5637f9f3cbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:32:32.007 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:32.007+0000 7fe7643a1740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5637f9f3c080 0x5637f9f43fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:32:32.007 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:32.007+0000 7fe7643a1740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5637f9f44c00 0x5637f9f444e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:32:32.007 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:32.007+0000 7fe7643a1740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe75c1b0ba0 con 0x5637f9f3b6d0 2024-03-21T10:32:32.007 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:32.007+0000 7fe7643a1740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe75c1b2520 con 0x5637f9f44c00 2024-03-21T10:32:32.007 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:32.007+0000 7fe7643a1740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe75c1b1200 con 0x5637f9f3c080 2024-03-21T10:32:32.007 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:32.007+0000 7fe75aa00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5637f9f3b6d0 0x5637f9f3cbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:32:32.007 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:32.007+0000 7fe75aa00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5637f9f3b6d0 0x5637f9f3cbc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:50998/0 (socket says 172.21.15.47:50998) 2024-03-21T10:32:32.007 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:32.007+0000 7fe75aa00640 1 -- 172.21.15.47:0/983069015 learned_addr learned my addr 172.21.15.47:0/983069015 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:32:32.008 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:32.007+0000 7fe75b400640 1 --2- 172.21.15.47:0/983069015 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5637f9f44c00 0x5637f9f444e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:32:32.008 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:32.007+0000 7fe75a000640 1 --2- 172.21.15.47:0/983069015 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5637f9f3c080 0x5637f9f43fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:32:32.008 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:32.007+0000 7fe75aa00640 1 -- 172.21.15.47:0/983069015 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5637f9f3c080 msgr2=0x5637f9f43fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:32:32.008 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:32.007+0000 7fe75aa00640 1 --2- 172.21.15.47:0/983069015 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5637f9f3c080 0x5637f9f43fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:32:32.008 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:32.007+0000 7fe75aa00640 1 -- 172.21.15.47:0/983069015 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5637f9f44c00 msgr2=0x5637f9f444e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:32:32.008 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:32.007+0000 7fe75aa00640 1 --2- 172.21.15.47:0/983069015 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5637f9f44c00 0x5637f9f444e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:32:32.008 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:32.007+0000 7fe75aa00640 1 -- 172.21.15.47:0/983069015 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5637f9f36970 con 0x5637f9f3b6d0 2024-03-21T10:32:32.014 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:32.008+0000 7fe75aa00640 1 --2- 172.21.15.47:0/983069015 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5637f9f3b6d0 0x5637f9f3cbc0 secure :-1 s=READY pgs=1039 cs=0 l=1 rev1=1 crypto rx=0x5637f9f57680 tx=0x7fe74c01a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3ae4d05a1cc3495f server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:32:32.015 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:32.008+0000 7fe746a00640 1 -- 172.21.15.47:0/983069015 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe74c01ec60 con 0x5637f9f3b6d0 2024-03-21T10:32:32.015 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:32.008+0000 7fe746a00640 1 -- 172.21.15.47:0/983069015 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fe74c01ee00 con 0x5637f9f3b6d0 2024-03-21T10:32:32.015 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:32.008+0000 7fe746a00640 1 -- 172.21.15.47:0/983069015 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe74c003850 con 0x5637f9f3b6d0 2024-03-21T10:32:32.015 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:32.009+0000 7fe7643a1740 1 -- 172.21.15.47:0/983069015 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5637f9f3b6d0 msgr2=0x5637f9f3cbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:32:32.015 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:32.009+0000 7fe7643a1740 1 --2- 172.21.15.47:0/983069015 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5637f9f3b6d0 0x5637f9f3cbc0 secure :-1 s=READY pgs=1039 cs=0 l=1 rev1=1 crypto rx=0x5637f9f57680 tx=0x7fe74c01a000 comp rx=0 tx=0).stop 2024-03-21T10:32:32.015 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:32.009+0000 7fe7643a1740 1 -- 172.21.15.47:0/983069015 shutdown_connections 2024-03-21T10:32:32.015 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:32.009+0000 7fe7643a1740 1 --2- 172.21.15.47:0/983069015 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5637f9f44c00 0x5637f9f444e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:32:32.015 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:32.009+0000 7fe7643a1740 1 --2- 172.21.15.47:0/983069015 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5637f9f3c080 0x5637f9f43fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:32:32.015 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:32.009+0000 7fe7643a1740 1 --2- 172.21.15.47:0/983069015 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5637f9f3b6d0 0x5637f9f3cbc0 unknown :-1 s=CLOSED pgs=1039 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:32:32.015 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:32.009+0000 7fe7643a1740 1 -- 172.21.15.47:0/983069015 >> 172.21.15.47:0/983069015 conn(0x5637f9f35da0 msgr2=0x5637f9f3c4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:32:32.015 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:32.009+0000 7fe7643a1740 1 -- 172.21.15.47:0/983069015 shutdown_connections 2024-03-21T10:32:32.015 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:32.009+0000 7fe7643a1740 1 -- 172.21.15.47:0/983069015 wait complete. 2024-03-21T10:32:32.015 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:32.009+0000 7fe7643a1740 1 -- start start 2024-03-21T10:32:32.016 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:32.010+0000 7fe7643a1740 1 Processor -- start 2024-03-21T10:32:32.016 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:32.010+0000 7fe7643a1740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5637f9f3b6d0 0x5637f9f4cbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:32:32.016 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:32.010+0000 7fe7643a1740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5637f9f3c080 0x5637f9f4af60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:32:32.016 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:32.010+0000 7fe7643a1740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5637f9f44c00 0x5637f9f4b4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:32:32.016 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:32.010+0000 7fe7643a1740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe75c1b1ec0 con 0x5637f9f44c00 2024-03-21T10:32:32.016 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:32.010+0000 7fe7643a1740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe75c1b1860 con 0x5637f9f3b6d0 2024-03-21T10:32:32.016 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:32.010+0000 7fe7643a1740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe75c1aed80 con 0x5637f9f3c080 2024-03-21T10:32:32.016 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:32.010+0000 7fe7643a1740 10 client.0 _ll_register_callbacks cb 0x7fe754098060 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T10:32:32.016 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:32.010+0000 7fe75b400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5637f9f44c00 0x5637f9f4b4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:32:32.016 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:32.010+0000 7fe75aa00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5637f9f3b6d0 0x5637f9f4cbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:32:32.016 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:32.011+0000 7fe75aa00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5637f9f3b6d0 0x5637f9f4cbc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:57190/0 (socket says 172.21.15.47:57190) 2024-03-21T10:32:32.016 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:32.011+0000 7fe75aa00640 1 -- 172.21.15.47:0/1137719007 learned_addr learned my addr 172.21.15.47:0/1137719007 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:32:32.016 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:32.011+0000 7fe75a000640 1 --2- 172.21.15.47:0/1137719007 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5637f9f3c080 0x5637f9f4af60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:32:32.016 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:32.011+0000 7fe75b400640 1 -- 172.21.15.47:0/1137719007 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5637f9f3c080 msgr2=0x5637f9f4af60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:32:32.016 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:32.011+0000 7fe75b400640 1 --2- 172.21.15.47:0/1137719007 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5637f9f3c080 0x5637f9f4af60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:32:32.017 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:32.011+0000 7fe75b400640 1 -- 172.21.15.47:0/1137719007 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5637f9f3b6d0 msgr2=0x5637f9f4cbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:32:32.017 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:32.011+0000 7fe75b400640 1 --2- 172.21.15.47:0/1137719007 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5637f9f3b6d0 0x5637f9f4cbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:32:32.017 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:32.011+0000 7fe75b400640 1 -- 172.21.15.47:0/1137719007 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5637f9f4bb90 con 0x5637f9f44c00 2024-03-21T10:32:32.017 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:32.011+0000 7fe75aa00640 1 --2- 172.21.15.47:0/1137719007 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5637f9f3b6d0 0x5637f9f4cbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T10:32:32.017 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:32.011+0000 7fe75b400640 1 --2- 172.21.15.47:0/1137719007 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5637f9f44c00 0x5637f9f4b4a0 secure :-1 s=READY pgs=1040 cs=0 l=1 rev1=1 crypto rx=0x7fe75409bd50 tx=0x7fe75409b990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a2fc36bb2cec376d server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:32:32.017 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:32.011+0000 7fe73be00640 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-03-21T10:32:32.017 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:32.011+0000 7fe73be00640 1 -- 172.21.15.47:0/1137719007 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe75401f040 con 0x5637f9f44c00 2024-03-21T10:32:32.017 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:32.012+0000 7fe73be00640 1 -- 172.21.15.47:0/1137719007 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(99..99 src has 1..99) v4 ==== 9562+0+0 (secure 0 0 0) 0x7fe7540bad30 con 0x5637f9f44c00 2024-03-21T10:32:32.017 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:32.012+0000 7fe7643a1740 1 -- 172.21.15.47:0/1137719007 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fe74c01ee00 con 0x5637f9f44c00 2024-03-21T10:32:32.017 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:32.012+0000 7fe73be00640 1 -- 172.21.15.47:0/1137719007 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe75409b720 con 0x5637f9f44c00 2024-03-21T10:32:32.017 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:32.012+0000 7fe7643a1740 10 client.17823 fetch_fsmap learned FSMap version 32 2024-03-21T10:32:32.017 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:32.012+0000 7fe7643a1740 1 -- 172.21.15.47:0/1137719007 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x5637f9fb11e0 con 0x5637f9f44c00 2024-03-21T10:32:32.018 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:32.012+0000 7fe73be00640 1 -- 172.21.15.47:0/1137719007 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fe74c01ee00 con 0x5637f9f44c00 2024-03-21T10:32:32.018 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:32.012+0000 7fe73be00640 1 -- 172.21.15.47:0/1137719007 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 32) v1 ==== 7942+0+0 (secure 0 0 0) 0x7fe75409ae80 con 0x5637f9f44c00 2024-03-21T10:32:32.018 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:32.012+0000 7fe7643a1740 10 client.17823 fetch_fsmap finished waiting for FSMap version 32 2024-03-21T10:32:32.018 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:32.012+0000 7fe7643a1740 10 client.17823 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-21T10:32:32.018 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:32.012+0000 7fe7643a1740 20 client.17823 populate_metadata read hostname 'smithi047' 2024-03-21T10:32:32.018 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:32.012+0000 7fe7643a1740 1 --2- 172.21.15.47:0/1137719007 >> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] conn(0x5637f9fb1850 0x5637f9fb3c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:32:32.018 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:32.012+0000 7fe7643a1740 4 client.17823 mds_command: new command op to 34535 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-21T10:32:32.018 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:32.012+0000 7fe7643a1740 1 -- 172.21.15.47:0/1137719007 --> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x5637f9ea6b50 con 0x5637f9fb1850 2024-03-21T10:32:32.018 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:32.013+0000 7fe75aa00640 1 --2- 172.21.15.47:0/1137719007 >> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] conn(0x5637f9fb1850 0x5637f9fb3c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:32:32.018 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:32.013+0000 7fe75aa00640 1 --2- 172.21.15.47:0/1137719007 >> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] conn(0x5637f9fb1850 0x5637f9fb3c70 crc :-1 s=READY pgs=172 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=412e903a2b44a839 server_cookie=48975778ed838509 in_seq=0 out_seq=0 2024-03-21T10:32:32.018 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:32.014+0000 7fe73be00640 10 client.17823 ms_handle_connect on v2:172.21.15.100:6838/4173168440 2024-03-21T10:32:32.018 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:32.014+0000 7fe73be00640 1 -- 172.21.15.47:0/1137719007 <== mds.0 v2:172.21.15.100:6838/4173168440 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x5637f9ea6b50 con 0x5637f9fb1850 2024-03-21T10:32:32.018 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:32.014+0000 7fe73be00640 10 client.17823 handle_command_reply: tid=0 2024-03-21T10:32:32.019 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:32.014+0000 7fe7643a1740 1 client.17823 shutdown 2024-03-21T10:32:32.019 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:32.014+0000 7fe7643a1740 1 -- 172.21.15.47:0/1137719007 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5637f9f44c00 msgr2=0x5637f9f4b4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:32:32.019 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:32.014+0000 7fe7643a1740 1 --2- 172.21.15.47:0/1137719007 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5637f9f44c00 0x5637f9f4b4a0 secure :-1 s=READY pgs=1040 cs=0 l=1 rev1=1 crypto rx=0x7fe75409bd50 tx=0x7fe75409b990 comp rx=0 tx=0).stop 2024-03-21T10:32:32.019 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:32.014+0000 7fe7643a1740 1 -- 172.21.15.47:0/1137719007 shutdown_connections 2024-03-21T10:32:32.019 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:32.014+0000 7fe7643a1740 1 --2- 172.21.15.47:0/1137719007 >> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] conn(0x5637f9fb1850 0x5637f9fb3c70 crc :-1 s=READY pgs=172 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:32:32.019 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:32.014+0000 7fe7643a1740 1 --2- 172.21.15.47:0/1137719007 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5637f9f44c00 0x5637f9f4b4a0 unknown :-1 s=CLOSED pgs=1040 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:32:32.019 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:32.014+0000 7fe7643a1740 1 --2- 172.21.15.47:0/1137719007 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5637f9f3c080 0x5637f9f4af60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:32:32.019 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:32.014+0000 7fe7643a1740 1 --2- 172.21.15.47:0/1137719007 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5637f9f3b6d0 0x5637f9f4cbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:32:32.019 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:32.014+0000 7fe73be00640 0 client.17823 ms_handle_reset on v2:172.21.15.100:6838/4173168440 2024-03-21T10:32:32.019 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:32.014+0000 7fe7643a1740 1 -- 172.21.15.47:0/1137719007 >> 172.21.15.47:0/1137719007 conn(0x5637f9f35da0 msgr2=0x5637f9f3cd40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:32:32.019 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:32.014+0000 7fe7643a1740 1 -- 172.21.15.47:0/1137719007 shutdown_connections 2024-03-21T10:32:32.019 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:32.014+0000 7fe7643a1740 1 -- 172.21.15.47:0/1137719007 wait complete. 2024-03-21T10:32:32.019 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:32.015+0000 7fe7643a1740 20 client.17823 trim_cache size 0 max 16384 2024-03-21T10:32:32.020 INFO:teuthology.orchestra.run.smithi047.stdout:[] 2024-03-21T10:32:32.020 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:32.015+0000 7fe761600640 1 -- 172.21.15.47:0/848798510 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7fe7400a3090 msgr2=0x7fe7400a5550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:32:32.020 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:32.015+0000 7fe761600640 1 --2- 172.21.15.47:0/848798510 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7fe7400a3090 0x7fe7400a5550 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x5637f9f36e10 tx=0x7fe74c005f90 comp rx=0 tx=0).stop 2024-03-21T10:32:32.021 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:32.015+0000 7fe761600640 1 -- 172.21.15.47:0/848798510 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe75c1098c0 msgr2=0x7fe75c19e980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:32:32.021 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:32.015+0000 7fe761600640 1 --2- 172.21.15.47:0/848798510 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe75c1098c0 0x7fe75c19e980 secure :-1 s=READY pgs=1037 cs=0 l=1 rev1=1 crypto rx=0x7fe75400eaa0 tx=0x7fe75400ef60 comp rx=0 tx=0).stop 2024-03-21T10:32:32.021 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:32.015+0000 7fe761600640 1 -- 172.21.15.47:0/848798510 shutdown_connections 2024-03-21T10:32:32.021 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:32.015+0000 7fe761600640 1 --2- 172.21.15.47:0/848798510 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe75c1098c0 0x7fe75c19e980 unknown :-1 s=CLOSED pgs=1037 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:32:32.021 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:32.015+0000 7fe761600640 1 --2- 172.21.15.47:0/848798510 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7fe7400a3090 0x7fe7400a5550 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:32:32.021 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:32.015+0000 7fe761600640 1 --2- 172.21.15.47:0/848798510 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe75c104270 0x7fe75c19a600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:32:32.021 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:32.015+0000 7fe761600640 1 --2- 172.21.15.47:0/848798510 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe75c1038b0 0x7fe75c19a0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:32:32.021 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:32.015+0000 7fe761600640 1 -- 172.21.15.47:0/848798510 >> 172.21.15.47:0/848798510 conn(0x7fe75c0ff360 msgr2=0x7fe75c0ffc50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:32:32.021 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:32.015+0000 7fe761600640 1 -- 172.21.15.47:0/848798510 shutdown_connections 2024-03-21T10:32:32.021 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:32.016+0000 7fe761600640 1 -- 172.21.15.47:0/848798510 wait complete. 2024-03-21T10:32:33.033 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-03-21T10:32:33.033 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-21T10:32:33.033 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-03-21T10:32:33.283 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:33.283+0000 7fdea1800640 1 -- 172.21.15.47:0/3975932434 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fde9c068de0 msgr2=0x7fde9c069240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:32:33.284 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:33.283+0000 7fdea1800640 1 --2- 172.21.15.47:0/3975932434 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fde9c068de0 0x7fde9c069240 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fde88009930 tx=0x7fde8802bc30 comp rx=0 tx=0).stop 2024-03-21T10:32:33.284 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:33.283+0000 7fdea1800640 1 -- 172.21.15.47:0/3975932434 shutdown_connections 2024-03-21T10:32:33.284 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:33.283+0000 7fdea1800640 1 --2- 172.21.15.47:0/3975932434 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fde9c104160 0x7fde9c1045e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:32:33.284 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:33.283+0000 7fdea1800640 1 --2- 172.21.15.47:0/3975932434 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fde9c068de0 0x7fde9c069240 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:32:33.284 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:33.284+0000 7fdea1800640 1 --2- 172.21.15.47:0/3975932434 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fde9c1065f0 0x7fde9c0688a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:32:33.284 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:33.284+0000 7fdea1800640 1 -- 172.21.15.47:0/3975932434 >> 172.21.15.47:0/3975932434 conn(0x7fde9c0776a0 msgr2=0x7fde9c077ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:32:33.284 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:33.284+0000 7fdea1800640 1 -- 172.21.15.47:0/3975932434 shutdown_connections 2024-03-21T10:32:33.284 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:33.284+0000 7fdea1800640 1 -- 172.21.15.47:0/3975932434 wait complete. 2024-03-21T10:32:33.286 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:33.285+0000 7fdea1800640 1 Processor -- start 2024-03-21T10:32:33.286 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:33.285+0000 7fdea1800640 1 -- start start 2024-03-21T10:32:33.286 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:33.285+0000 7fdea1800640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fde9c068de0 0x7fde9c19e4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:32:33.286 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:33.286+0000 7fdea1800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fde9c104160 0x7fde9c19e9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:32:33.286 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:33.286+0000 7fdea1800640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fde9c1065f0 0x7fde9c1a2d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:32:33.286 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:33.286+0000 7fdea1800640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde9c117f00 con 0x7fde9c068de0 2024-03-21T10:32:33.286 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:33.286+0000 7fdea1800640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde9c117d80 con 0x7fde9c104160 2024-03-21T10:32:33.286 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:33.286+0000 7fdea1800640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde9c118080 con 0x7fde9c1065f0 2024-03-21T10:32:33.286 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:33.286+0000 7fde9aa00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fde9c068de0 0x7fde9c19e4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:32:33.286 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:33.286+0000 7fde9aa00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fde9c068de0 0x7fde9c19e4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:51034/0 (socket says 172.21.15.47:51034) 2024-03-21T10:32:33.287 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:33.286+0000 7fde9aa00640 1 -- 172.21.15.47:0/659367757 learned_addr learned my addr 172.21.15.47:0/659367757 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:32:33.287 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:33.286+0000 7fde9a000640 1 --2- 172.21.15.47:0/659367757 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fde9c104160 0x7fde9c19e9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:32:33.287 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:33.286+0000 7fde9b400640 1 --2- 172.21.15.47:0/659367757 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fde9c1065f0 0x7fde9c1a2d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:32:33.287 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:33.286+0000 7fde9aa00640 1 -- 172.21.15.47:0/659367757 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fde9c1065f0 msgr2=0x7fde9c1a2d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:32:33.287 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:33.286+0000 7fde9aa00640 1 --2- 172.21.15.47:0/659367757 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fde9c1065f0 0x7fde9c1a2d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:32:33.287 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:33.286+0000 7fde9aa00640 1 -- 172.21.15.47:0/659367757 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fde9c104160 msgr2=0x7fde9c19e9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:32:33.287 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:33.287+0000 7fde9aa00640 1 --2- 172.21.15.47:0/659367757 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fde9c104160 0x7fde9c19e9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:32:33.287 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:33.287+0000 7fde9aa00640 1 -- 172.21.15.47:0/659367757 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde9c1a3490 con 0x7fde9c068de0 2024-03-21T10:32:33.287 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:33.287+0000 7fde9a000640 1 --2- 172.21.15.47:0/659367757 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fde9c104160 0x7fde9c19e9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T10:32:33.288 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:33.287+0000 7fde9aa00640 1 --2- 172.21.15.47:0/659367757 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fde9c068de0 0x7fde9c19e4a0 secure :-1 s=READY pgs=1041 cs=0 l=1 rev1=1 crypto rx=0x7fde8c00e9f0 tx=0x7fde8c00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:32:33.288 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:33.287+0000 7fde93400640 1 -- 172.21.15.47:0/659367757 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde8c00cd80 con 0x7fde9c068de0 2024-03-21T10:32:33.288 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:33.287+0000 7fdea1800640 1 -- 172.21.15.47:0/659367757 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fde9c1a3780 con 0x7fde9c068de0 2024-03-21T10:32:33.288 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:33.287+0000 7fde93400640 1 -- 172.21.15.47:0/659367757 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fde8c0108e0 con 0x7fde9c068de0 2024-03-21T10:32:33.288 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:33.287+0000 7fde93400640 1 -- 172.21.15.47:0/659367757 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde8c018920 con 0x7fde9c068de0 2024-03-21T10:32:33.288 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:33.288+0000 7fdea1800640 1 -- 172.21.15.47:0/659367757 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fde9c1ab010 con 0x7fde9c068de0 2024-03-21T10:32:33.290 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:33.291+0000 7fde93400640 1 -- 172.21.15.47:0/659367757 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 27) v1 ==== 137985+0+0 (secure 0 0 0) 0x7fde8c010a90 con 0x7fde9c068de0 2024-03-21T10:32:33.290 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:33.291+0000 7fdea45c8740 1 Processor -- start 2024-03-21T10:32:33.291 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:33.291+0000 7fdea45c8740 1 -- start start 2024-03-21T10:32:33.291 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:33.291+0000 7fdea45c8740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561a76b33080 0x561a76b33460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:32:33.291 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:33.292+0000 7fdea45c8740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561a76b339a0 0x561a76b3c040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:32:33.291 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:33.292+0000 7fdea45c8740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561a76b3c580 0x561a76b3e970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:32:33.291 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:33.292+0000 7fdea45c8740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x561a76a97eb0 con 0x561a76b339a0 2024-03-21T10:32:33.291 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:33.292+0000 7fdea45c8740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x561a76a40a60 con 0x561a76b33080 2024-03-21T10:32:33.291 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:33.292+0000 7fdea45c8740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x561a76a93f50 con 0x561a76b3c580 2024-03-21T10:32:33.293 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:33.292+0000 7fde9aa00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561a76b339a0 0x561a76b3c040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:32:33.293 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:33.292+0000 7fde9aa00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561a76b339a0 0x561a76b3c040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:51044/0 (socket says 172.21.15.47:51044) 2024-03-21T10:32:33.293 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:33.292+0000 7fde9aa00640 1 -- 172.21.15.47:0/3839347744 learned_addr learned my addr 172.21.15.47:0/3839347744 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:32:33.293 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:33.292+0000 7fde9b400640 1 --2- 172.21.15.47:0/3839347744 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561a76b33080 0x561a76b33460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:32:33.293 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:33.292+0000 7fde9a000640 1 --2- 172.21.15.47:0/3839347744 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561a76b3c580 0x561a76b3e970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:32:33.293 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:33.292+0000 7fde9aa00640 1 -- 172.21.15.47:0/3839347744 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561a76b3c580 msgr2=0x561a76b3e970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:32:33.294 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:33.292+0000 7fde9aa00640 1 --2- 172.21.15.47:0/3839347744 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561a76b3c580 0x561a76b3e970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:32:33.294 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:33.293+0000 7fde9aa00640 1 -- 172.21.15.47:0/3839347744 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561a76b33080 msgr2=0x561a76b33460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:32:33.294 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:33.293+0000 7fde9aa00640 1 --2- 172.21.15.47:0/3839347744 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561a76b33080 0x561a76b33460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:32:33.294 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:33.293+0000 7fde9aa00640 1 -- 172.21.15.47:0/3839347744 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561a768747e0 con 0x561a76b339a0 2024-03-21T10:32:33.294 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:33.293+0000 7fde93400640 1 --2- 172.21.15.47:0/659367757 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7fde800a3090 0x7fde800a5550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:32:33.294 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:33.293+0000 7fde9b400640 1 --2- 172.21.15.47:0/3839347744 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561a76b33080 0x561a76b33460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T10:32:33.294 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:33.293+0000 7fde9a000640 1 --2- 172.21.15.47:0/659367757 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7fde800a3090 0x7fde800a5550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:32:33.294 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:33.293+0000 7fde93400640 1 -- 172.21.15.47:0/659367757 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 9562+0+0 (secure 0 0 0) 0x7fde8c0d0060 con 0x7fde9c068de0 2024-03-21T10:32:33.294 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:33.293+0000 7fde9aa00640 1 --2- 172.21.15.47:0/3839347744 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561a76b339a0 0x561a76b3c040 secure :-1 s=READY pgs=1042 cs=0 l=1 rev1=1 crypto rx=0x7fde8c0cd280 tx=0x7fde8c098280 comp rx=0 tx=0).ready entity=mon.0 client_cookie=933413923313c0bd server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:32:33.294 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:33.293+0000 7fde87e00640 1 -- 172.21.15.47:0/3839347744 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde8c026030 con 0x561a76b339a0 2024-03-21T10:32:33.294 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:33.294+0000 7fde87e00640 1 -- 172.21.15.47:0/3839347744 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fde8c010de0 con 0x561a76b339a0 2024-03-21T10:32:33.294 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:33.294+0000 7fde87e00640 1 -- 172.21.15.47:0/3839347744 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde8c099c50 con 0x561a76b339a0 2024-03-21T10:32:33.298 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:33.294+0000 7fdea45c8740 1 -- 172.21.15.47:0/3839347744 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561a76b339a0 msgr2=0x561a76b3c040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:32:33.298 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:33.294+0000 7fdea45c8740 1 --2- 172.21.15.47:0/3839347744 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561a76b339a0 0x561a76b3c040 secure :-1 s=READY pgs=1042 cs=0 l=1 rev1=1 crypto rx=0x7fde8c0cd280 tx=0x7fde8c098280 comp rx=0 tx=0).stop 2024-03-21T10:32:33.298 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:33.294+0000 7fde9a000640 1 --2- 172.21.15.47:0/659367757 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7fde800a3090 0x7fde800a5550 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fde88006010 tx=0x7fde88005a20 comp rx=0 tx=0).ready entity=mgr.15687 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:32:33.298 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:33.294+0000 7fdea45c8740 1 -- 172.21.15.47:0/3839347744 shutdown_connections 2024-03-21T10:32:33.298 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:33.294+0000 7fdea45c8740 1 --2- 172.21.15.47:0/3839347744 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561a76b3c580 0x561a76b3e970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:32:33.298 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:33.294+0000 7fdea45c8740 1 --2- 172.21.15.47:0/3839347744 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561a76b339a0 0x561a76b3c040 unknown :-1 s=CLOSED pgs=1042 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:32:33.298 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:33.294+0000 7fdea45c8740 1 --2- 172.21.15.47:0/3839347744 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561a76b33080 0x561a76b33460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:32:33.298 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:33.294+0000 7fdea45c8740 1 -- 172.21.15.47:0/3839347744 >> 172.21.15.47:0/3839347744 conn(0x561a76b2d7b0 msgr2=0x561a76b2db90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:32:33.298 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:33.295+0000 7fdea45c8740 1 -- 172.21.15.47:0/3839347744 shutdown_connections 2024-03-21T10:32:33.298 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:33.295+0000 7fdea45c8740 1 -- 172.21.15.47:0/3839347744 wait complete. 2024-03-21T10:32:33.298 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:33.295+0000 7fdea45c8740 1 -- start start 2024-03-21T10:32:33.299 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:33.295+0000 7fdea45c8740 1 Processor -- start 2024-03-21T10:32:33.299 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:33.296+0000 7fdea45c8740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561a76b33080 0x561a76b3bf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:32:33.299 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:33.296+0000 7fdea45c8740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561a76b339a0 0x561a76b39300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:32:33.299 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:33.296+0000 7fdea45c8740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561a76b3c580 0x561a76b39910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:32:33.299 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:33.296+0000 7fdea45c8740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x561a76b3f370 con 0x561a76b33080 2024-03-21T10:32:33.299 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:33.296+0000 7fdea45c8740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x561a76a53eb0 con 0x561a76b339a0 2024-03-21T10:32:33.299 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:33.296+0000 7fdea45c8740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x561a76b3f4f0 con 0x561a76b3c580 2024-03-21T10:32:33.299 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:33.296+0000 7fdea45c8740 10 client.0 _ll_register_callbacks cb 0x561a76a48150 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T10:32:33.299 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:33.296+0000 7fde9aa00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561a76b33080 0x561a76b3bf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:32:33.299 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:33.296+0000 7fde9aa00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561a76b33080 0x561a76b3bf70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:51056/0 (socket says 172.21.15.47:51056) 2024-03-21T10:32:33.299 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:33.296+0000 7fde9aa00640 1 -- 172.21.15.47:0/3939895255 learned_addr learned my addr 172.21.15.47:0/3939895255 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:32:33.299 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:33.296+0000 7fde9a000640 1 --2- 172.21.15.47:0/3939895255 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561a76b339a0 0x561a76b39300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:32:33.299 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:33.296+0000 7fde9b400640 1 --2- 172.21.15.47:0/3939895255 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561a76b3c580 0x561a76b39910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:32:33.299 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:33.296+0000 7fde9aa00640 1 -- 172.21.15.47:0/3939895255 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561a76b3c580 msgr2=0x561a76b39910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:32:33.299 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:33.297+0000 7fde9aa00640 1 --2- 172.21.15.47:0/3939895255 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561a76b3c580 0x561a76b39910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:32:33.300 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:33.297+0000 7fde9aa00640 1 -- 172.21.15.47:0/3939895255 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561a76b339a0 msgr2=0x561a76b39300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:32:33.300 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:33.297+0000 7fde9aa00640 1 --2- 172.21.15.47:0/3939895255 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561a76b339a0 0x561a76b39300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:32:33.300 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:33.297+0000 7fde9aa00640 1 -- 172.21.15.47:0/3939895255 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x561a768d4f70 con 0x561a76b33080 2024-03-21T10:32:33.300 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:33.297+0000 7fde9a000640 1 --2- 172.21.15.47:0/3939895255 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561a76b339a0 0x561a76b39300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T10:32:33.300 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:33.297+0000 7fde9b400640 1 --2- 172.21.15.47:0/3939895255 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561a76b3c580 0x561a76b39910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T10:32:33.300 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:33.297+0000 7fde9aa00640 1 --2- 172.21.15.47:0/3939895255 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561a76b33080 0x561a76b3bf70 secure :-1 s=READY pgs=1043 cs=0 l=1 rev1=1 crypto rx=0x7fde8c0a44c0 tx=0x7fde8c09ba40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=40593a6842b2768 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:32:33.300 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:33.297+0000 7fde84c00640 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-03-21T10:32:33.300 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:33.297+0000 7fde84c00640 1 -- 172.21.15.47:0/3939895255 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde8c09bdb0 con 0x561a76b33080 2024-03-21T10:32:33.300 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:33.297+0000 7fde84c00640 1 -- 172.21.15.47:0/3939895255 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(99..99 src has 1..99) v4 ==== 9562+0+0 (secure 0 0 0) 0x7fde8c097d30 con 0x561a76b33080 2024-03-21T10:32:33.300 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:33.297+0000 7fdea45c8740 1 -- 172.21.15.47:0/3939895255 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x561a76aa53f0 con 0x561a76b33080 2024-03-21T10:32:33.300 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:33.297+0000 7fde84c00640 1 -- 172.21.15.47:0/3939895255 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7fde8c09bb20 con 0x561a76b33080 2024-03-21T10:32:33.300 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:33.297+0000 7fdea45c8740 10 client.17835 fetch_fsmap learned FSMap version 32 2024-03-21T10:32:33.300 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:33.297+0000 7fdea45c8740 1 -- 172.21.15.47:0/3939895255 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x561a7676c170 con 0x561a76b33080 2024-03-21T10:32:33.300 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:33.298+0000 7fde84c00640 1 -- 172.21.15.47:0/3939895255 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x561a76aa53f0 con 0x561a76b33080 2024-03-21T10:32:33.301 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:33.298+0000 7fde84c00640 1 -- 172.21.15.47:0/3939895255 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 32) v1 ==== 7942+0+0 (secure 0 0 0) 0x7fde8c0cd5c0 con 0x561a76b33080 2024-03-21T10:32:33.301 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:33.298+0000 7fdea45c8740 10 client.17835 fetch_fsmap finished waiting for FSMap version 32 2024-03-21T10:32:33.301 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:33.298+0000 7fdea45c8740 10 client.17835 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-21T10:32:33.301 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:33.298+0000 7fdea45c8740 20 client.17835 populate_metadata read hostname 'smithi047' 2024-03-21T10:32:33.301 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:33.298+0000 7fdea45c8740 1 --2- 172.21.15.47:0/3939895255 >> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] conn(0x561a76b4c4e0 0x561a76b4e8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:32:33.301 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:33.298+0000 7fdea45c8740 4 client.17835 mds_command: new command op to 34535 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T10:32:33.301 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:33.298+0000 7fdea45c8740 1 -- 172.21.15.47:0/3939895255 --> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x561a76a9f740 con 0x561a76b4c4e0 2024-03-21T10:32:33.301 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:33.298+0000 7fde9a000640 1 --2- 172.21.15.47:0/3939895255 >> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] conn(0x561a76b4c4e0 0x561a76b4e8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:32:33.302 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:33.299+0000 7fde9a000640 1 --2- 172.21.15.47:0/3939895255 >> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] conn(0x561a76b4c4e0 0x561a76b4e8d0 crc :-1 s=READY pgs=173 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=47f2b2d97dae4fc5 server_cookie=9d9af1ec58986112 in_seq=0 out_seq=0 2024-03-21T10:32:33.302 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:33.299+0000 7fde84c00640 10 client.17835 ms_handle_connect on v2:172.21.15.100:6838/4173168440 2024-03-21T10:32:33.302 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:33.300+0000 7fde84c00640 1 -- 172.21.15.47:0/3939895255 <== mds.0 v2:172.21.15.100:6838/4173168440 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x561a76a9f740 con 0x561a76b4c4e0 2024-03-21T10:32:33.303 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:33.300+0000 7fde84c00640 10 client.17835 handle_command_reply: tid=0 2024-03-21T10:32:33.303 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:33.300+0000 7fdea45c8740 1 client.17835 shutdown 2024-03-21T10:32:33.303 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:33.301+0000 7fdea45c8740 1 -- 172.21.15.47:0/3939895255 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561a76b33080 msgr2=0x561a76b3bf70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:32:33.303 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:33.301+0000 7fdea45c8740 1 --2- 172.21.15.47:0/3939895255 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561a76b33080 0x561a76b3bf70 secure :-1 s=READY pgs=1043 cs=0 l=1 rev1=1 crypto rx=0x7fde8c0a44c0 tx=0x7fde8c09ba40 comp rx=0 tx=0).stop 2024-03-21T10:32:33.303 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:33.301+0000 7fdea45c8740 1 -- 172.21.15.47:0/3939895255 shutdown_connections 2024-03-21T10:32:33.303 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:33.301+0000 7fdea45c8740 1 --2- 172.21.15.47:0/3939895255 >> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] conn(0x561a76b4c4e0 0x561a76b4e8d0 crc :-1 s=READY pgs=173 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:32:33.303 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:33.301+0000 7fdea45c8740 1 --2- 172.21.15.47:0/3939895255 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561a76b3c580 0x561a76b39910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:32:33.303 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:33.301+0000 7fdea45c8740 1 --2- 172.21.15.47:0/3939895255 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561a76b339a0 0x561a76b39300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:32:33.303 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:33.301+0000 7fde84c00640 0 client.17835 ms_handle_reset on v2:172.21.15.100:6838/4173168440 2024-03-21T10:32:33.303 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:33.301+0000 7fdea45c8740 1 --2- 172.21.15.47:0/3939895255 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561a76b33080 0x561a76b3bf70 unknown :-1 s=CLOSED pgs=1043 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:32:33.303 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:33.301+0000 7fdea45c8740 1 -- 172.21.15.47:0/3939895255 >> 172.21.15.47:0/3939895255 conn(0x561a76b2d7b0 msgr2=0x561a76b2f500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:32:33.303 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:33.301+0000 7fdea45c8740 1 -- 172.21.15.47:0/3939895255 shutdown_connections 2024-03-21T10:32:33.303 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:33.301+0000 7fdea45c8740 1 -- 172.21.15.47:0/3939895255 wait complete. 2024-03-21T10:32:33.303 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:33.301+0000 7fdea45c8740 20 client.17835 trim_cache size 0 max 16384 2024-03-21T10:32:33.318 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:33.319+0000 7fdea45c8740 1 Processor -- start 2024-03-21T10:32:33.319 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:33.319+0000 7fdea45c8740 1 -- start start 2024-03-21T10:32:33.319 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:33.319+0000 7fdea45c8740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561a76b33080 0x561a76b47e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:32:33.319 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:33.319+0000 7fdea45c8740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561a76b339a0 0x561a76b3ba30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:32:33.319 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:33.319+0000 7fdea45c8740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561a76b3c580 0x561a76b3bf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:32:33.319 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:33.319+0000 7fdea45c8740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde9c1b50d0 con 0x561a76b339a0 2024-03-21T10:32:33.319 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:33.319+0000 7fdea45c8740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde9c1b6a50 con 0x561a76b3c580 2024-03-21T10:32:33.319 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:33.319+0000 7fdea45c8740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde9c1b5730 con 0x561a76b33080 2024-03-21T10:32:33.320 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:33.320+0000 7fde9b400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561a76b3c580 0x561a76b3bf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:32:33.320 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:33.320+0000 7fde9a000640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561a76b339a0 0x561a76b3ba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:32:33.320 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:33.320+0000 7fde9b400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561a76b3c580 0x561a76b3bf70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:57240/0 (socket says 172.21.15.47:57240) 2024-03-21T10:32:33.320 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:33.320+0000 7fde9b400640 1 -- 172.21.15.47:0/1125528439 learned_addr learned my addr 172.21.15.47:0/1125528439 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:32:33.320 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:33.320+0000 7fde9aa00640 1 --2- 172.21.15.47:0/1125528439 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561a76b33080 0x561a76b47e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:32:33.320 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:33.320+0000 7fde9b400640 1 -- 172.21.15.47:0/1125528439 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561a76b33080 msgr2=0x561a76b47e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:32:33.320 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:33.320+0000 7fde9b400640 1 --2- 172.21.15.47:0/1125528439 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561a76b33080 0x561a76b47e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:32:33.321 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:33.320+0000 7fde9b400640 1 -- 172.21.15.47:0/1125528439 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561a76b339a0 msgr2=0x561a76b3ba30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:32:33.321 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:33.320+0000 7fde9b400640 1 --2- 172.21.15.47:0/1125528439 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561a76b339a0 0x561a76b3ba30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:32:33.321 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:33.320+0000 7fde9b400640 1 -- 172.21.15.47:0/1125528439 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561a76b39c80 con 0x561a76b3c580 2024-03-21T10:32:33.321 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:33.320+0000 7fde9a000640 1 --2- 172.21.15.47:0/1125528439 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561a76b339a0 0x561a76b3ba30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T10:32:33.321 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:33.321+0000 7fde9b400640 1 --2- 172.21.15.47:0/1125528439 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561a76b3c580 0x561a76b3bf70 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fde94009ea0 tx=0x7fde94009f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3a72ab6892dcc4f5 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:32:33.322 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:33.321+0000 7fde87400640 1 -- 172.21.15.47:0/1125528439 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde9401b640 con 0x561a76b3c580 2024-03-21T10:32:33.322 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:33.321+0000 7fde87400640 1 -- 172.21.15.47:0/1125528439 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fde9401a890 con 0x561a76b3c580 2024-03-21T10:32:33.322 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:33.321+0000 7fde87400640 1 -- 172.21.15.47:0/1125528439 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde94010920 con 0x561a76b3c580 2024-03-21T10:32:33.322 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:33.322+0000 7fdea45c8740 1 -- 172.21.15.47:0/1125528439 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561a76b3c580 msgr2=0x561a76b3bf70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:32:33.322 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:33.322+0000 7fdea45c8740 1 --2- 172.21.15.47:0/1125528439 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561a76b3c580 0x561a76b3bf70 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fde94009ea0 tx=0x7fde94009f90 comp rx=0 tx=0).stop 2024-03-21T10:32:33.322 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:33.322+0000 7fdea45c8740 1 -- 172.21.15.47:0/1125528439 shutdown_connections 2024-03-21T10:32:33.322 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:33.322+0000 7fdea45c8740 1 --2- 172.21.15.47:0/1125528439 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561a76b3c580 0x561a76b3bf70 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:32:33.323 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:33.322+0000 7fdea45c8740 1 --2- 172.21.15.47:0/1125528439 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561a76b339a0 0x561a76b3ba30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:32:33.323 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:33.322+0000 7fdea45c8740 1 --2- 172.21.15.47:0/1125528439 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561a76b33080 0x561a76b47e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:32:33.323 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:33.322+0000 7fdea45c8740 1 -- 172.21.15.47:0/1125528439 >> 172.21.15.47:0/1125528439 conn(0x561a76b2d7b0 msgr2=0x561a76b47740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:32:33.323 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:33.322+0000 7fdea45c8740 1 -- 172.21.15.47:0/1125528439 shutdown_connections 2024-03-21T10:32:33.323 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:33.322+0000 7fdea45c8740 1 -- 172.21.15.47:0/1125528439 wait complete. 2024-03-21T10:32:33.323 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:33.323+0000 7fdea45c8740 1 -- start start 2024-03-21T10:32:33.323 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:33.323+0000 7fdea45c8740 1 Processor -- start 2024-03-21T10:32:33.325 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:33.323+0000 7fdea45c8740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561a76b33080 0x561a76b503a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:32:33.326 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:33.323+0000 7fdea45c8740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561a76b339a0 0x561a76b508e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:32:33.326 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:33.323+0000 7fdea45c8740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561a76b3c580 0x561a76ba7a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:32:33.326 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:33.323+0000 7fdea45c8740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde9c1b63f0 con 0x561a76b339a0 2024-03-21T10:32:33.326 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:33.323+0000 7fdea45c8740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde9c1b5d90 con 0x561a76b33080 2024-03-21T10:32:33.326 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:33.323+0000 7fdea45c8740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde9c1b32b0 con 0x561a76b3c580 2024-03-21T10:32:33.326 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:33.323+0000 7fdea45c8740 10 client.0 _ll_register_callbacks cb 0x7fde8c0d80e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T10:32:33.326 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:33.324+0000 7fde9a000640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561a76b339a0 0x561a76b508e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:32:33.326 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:33.324+0000 7fde9a000640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561a76b339a0 0x561a76b508e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:51082/0 (socket says 172.21.15.47:51082) 2024-03-21T10:32:33.326 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:33.324+0000 7fde9a000640 1 -- 172.21.15.47:0/741935338 learned_addr learned my addr 172.21.15.47:0/741935338 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:32:33.326 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:33.324+0000 7fde9aa00640 1 --2- 172.21.15.47:0/741935338 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561a76b33080 0x561a76b503a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:32:33.326 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:33.324+0000 7fde9a000640 1 -- 172.21.15.47:0/741935338 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561a76b3c580 msgr2=0x561a76ba7a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:32:33.326 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:33.324+0000 7fde9a000640 1 --2- 172.21.15.47:0/741935338 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561a76b3c580 0x561a76ba7a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:32:33.326 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:33.324+0000 7fde9a000640 1 -- 172.21.15.47:0/741935338 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561a76b33080 msgr2=0x561a76b503a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:32:33.327 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:33.324+0000 7fde9a000640 1 --2- 172.21.15.47:0/741935338 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561a76b33080 0x561a76b503a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:32:33.327 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:33.324+0000 7fde9a000640 1 -- 172.21.15.47:0/741935338 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x561a76ba7f60 con 0x561a76b339a0 2024-03-21T10:32:33.327 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:33.324+0000 7fde9a000640 1 --2- 172.21.15.47:0/741935338 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561a76b339a0 0x561a76b508e0 secure :-1 s=READY pgs=1044 cs=0 l=1 rev1=1 crypto rx=0x7fde88036040 tx=0x7fde8803f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=81d40be68964b4d8 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:32:33.327 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:33.324+0000 7fde84c00640 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-03-21T10:32:33.327 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:33.325+0000 7fde84c00640 1 -- 172.21.15.47:0/741935338 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde88057070 con 0x561a76b339a0 2024-03-21T10:32:33.327 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:33.325+0000 7fde84c00640 1 -- 172.21.15.47:0/741935338 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(99..99 src has 1..99) v4 ==== 9562+0+0 (secure 0 0 0) 0x7fde88005240 con 0x561a76b339a0 2024-03-21T10:32:33.327 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:33.325+0000 7fdea45c8740 1 -- 172.21.15.47:0/741935338 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fde9401a890 con 0x561a76b339a0 2024-03-21T10:32:33.327 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:33.325+0000 7fde84c00640 1 -- 172.21.15.47:0/741935338 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7fde88063050 con 0x561a76b339a0 2024-03-21T10:32:33.327 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:33.325+0000 7fdea45c8740 10 client.17841 fetch_fsmap learned FSMap version 32 2024-03-21T10:32:33.327 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:33.325+0000 7fdea45c8740 1 -- 172.21.15.47:0/741935338 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x561a76ba91d0 con 0x561a76b339a0 2024-03-21T10:32:33.327 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:33.325+0000 7fde84c00640 1 -- 172.21.15.47:0/741935338 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fde9401a890 con 0x561a76b339a0 2024-03-21T10:32:33.327 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:33.325+0000 7fde84c00640 1 -- 172.21.15.47:0/741935338 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 32) v1 ==== 7942+0+0 (secure 0 0 0) 0x7fde880038c0 con 0x561a76b339a0 2024-03-21T10:32:33.327 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:33.326+0000 7fdea45c8740 10 client.17841 fetch_fsmap finished waiting for FSMap version 32 2024-03-21T10:32:33.328 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:33.326+0000 7fdea45c8740 10 client.17841 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-21T10:32:33.328 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:33.326+0000 7fdea45c8740 20 client.17841 populate_metadata read hostname 'smithi047' 2024-03-21T10:32:33.328 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:33.326+0000 7fdea45c8740 1 --2- 172.21.15.47:0/741935338 >> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] conn(0x561a76ba98e0 0x561a76babd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:32:33.328 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:33.326+0000 7fdea45c8740 4 client.17841 mds_command: new command op to 34535 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-03-21T10:32:33.328 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:33.326+0000 7fdea45c8740 1 -- 172.21.15.47:0/741935338 --> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x561a76a1da40 con 0x561a76ba98e0 2024-03-21T10:32:33.328 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:33.326+0000 7fde9aa00640 1 --2- 172.21.15.47:0/741935338 >> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] conn(0x561a76ba98e0 0x561a76babd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:32:33.329 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:33.327+0000 7fde9aa00640 1 --2- 172.21.15.47:0/741935338 >> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] conn(0x561a76ba98e0 0x561a76babd00 crc :-1 s=READY pgs=174 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dee2ffe016914d83 server_cookie=5c8ab23b394d5d39 in_seq=0 out_seq=0 2024-03-21T10:32:33.329 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:33.327+0000 7fde84c00640 10 client.17841 ms_handle_connect on v2:172.21.15.100:6838/4173168440 2024-03-21T10:32:33.329 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:33.329+0000 7fde84c00640 1 -- 172.21.15.47:0/741935338 <== mds.0 v2:172.21.15.100:6838/4173168440 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x561a76a1da40 con 0x561a76ba98e0 2024-03-21T10:32:33.329 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:33.329+0000 7fde84c00640 10 client.17841 handle_command_reply: tid=0 2024-03-21T10:32:33.329 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:33.329+0000 7fdea45c8740 1 client.17841 shutdown 2024-03-21T10:32:33.329 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:33.329+0000 7fdea45c8740 1 -- 172.21.15.47:0/741935338 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561a76b339a0 msgr2=0x561a76b508e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:32:33.329 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:33.329+0000 7fdea45c8740 1 --2- 172.21.15.47:0/741935338 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561a76b339a0 0x561a76b508e0 secure :-1 s=READY pgs=1044 cs=0 l=1 rev1=1 crypto rx=0x7fde88036040 tx=0x7fde8803f000 comp rx=0 tx=0).stop 2024-03-21T10:32:33.329 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:33.329+0000 7fdea45c8740 1 -- 172.21.15.47:0/741935338 shutdown_connections 2024-03-21T10:32:33.330 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:33.329+0000 7fdea45c8740 1 --2- 172.21.15.47:0/741935338 >> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] conn(0x561a76ba98e0 0x561a76babd00 crc :-1 s=READY pgs=174 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:32:33.330 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:33.329+0000 7fdea45c8740 1 --2- 172.21.15.47:0/741935338 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561a76b3c580 0x561a76ba7a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:32:33.330 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:33.329+0000 7fdea45c8740 1 --2- 172.21.15.47:0/741935338 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561a76b339a0 0x561a76b508e0 unknown :-1 s=CLOSED pgs=1044 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:32:33.330 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:33.329+0000 7fdea45c8740 1 --2- 172.21.15.47:0/741935338 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561a76b33080 0x561a76b503a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:32:33.330 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:33.329+0000 7fde84c00640 0 client.17841 ms_handle_reset on v2:172.21.15.100:6838/4173168440 2024-03-21T10:32:33.330 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:33.329+0000 7fdea45c8740 1 -- 172.21.15.47:0/741935338 >> 172.21.15.47:0/741935338 conn(0x561a76b2d7b0 msgr2=0x561a76b47fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:32:33.330 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:33.330+0000 7fdea45c8740 1 -- 172.21.15.47:0/741935338 shutdown_connections 2024-03-21T10:32:33.330 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:33.330+0000 7fdea45c8740 1 -- 172.21.15.47:0/741935338 wait complete. 2024-03-21T10:32:33.330 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:33.330+0000 7fdea45c8740 20 client.17841 trim_cache size 0 max 16384 2024-03-21T10:32:33.330 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-03-21T10:32:33.331 INFO:teuthology.orchestra.run.smithi047.stdout: "return_code": 0, 2024-03-21T10:32:33.331 INFO:teuthology.orchestra.run.smithi047.stdout: "scrub_tag": "84f2aa71-549a-4b8d-bf9e-c53dee131bce", 2024-03-21T10:32:33.331 INFO:teuthology.orchestra.run.smithi047.stdout: "mode": "asynchronous" 2024-03-21T10:32:33.331 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-03-21T10:32:33.331 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:33.330+0000 7fdea1800640 1 -- 172.21.15.47:0/659367757 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7fde800a3090 msgr2=0x7fde800a5550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:32:33.331 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:33.330+0000 7fdea1800640 1 --2- 172.21.15.47:0/659367757 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7fde800a3090 0x7fde800a5550 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fde88006010 tx=0x7fde88005a20 comp rx=0 tx=0).stop 2024-03-21T10:32:33.331 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:33.330+0000 7fdea1800640 1 -- 172.21.15.47:0/659367757 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fde9c068de0 msgr2=0x7fde9c19e4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:32:33.331 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:33.330+0000 7fdea1800640 1 --2- 172.21.15.47:0/659367757 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fde9c068de0 0x7fde9c19e4a0 secure :-1 s=READY pgs=1041 cs=0 l=1 rev1=1 crypto rx=0x7fde8c00e9f0 tx=0x7fde8c00eeb0 comp rx=0 tx=0).stop 2024-03-21T10:32:33.332 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:33.330+0000 7fdea1800640 1 -- 172.21.15.47:0/659367757 shutdown_connections 2024-03-21T10:32:33.332 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:33.330+0000 7fdea1800640 1 --2- 172.21.15.47:0/659367757 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fde9c1065f0 0x7fde9c1a2d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:32:33.332 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:33.330+0000 7fdea1800640 1 --2- 172.21.15.47:0/659367757 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7fde800a3090 0x7fde800a5550 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:32:33.332 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:33.330+0000 7fdea1800640 1 --2- 172.21.15.47:0/659367757 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fde9c104160 0x7fde9c19e9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:32:33.332 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:33.330+0000 7fdea1800640 1 --2- 172.21.15.47:0/659367757 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fde9c068de0 0x7fde9c19e4a0 unknown :-1 s=CLOSED pgs=1041 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:32:33.332 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:33.331+0000 7fdea1800640 1 -- 172.21.15.47:0/659367757 >> 172.21.15.47:0/659367757 conn(0x7fde9c0776a0 msgr2=0x7fde9c101540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:32:33.332 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:33.331+0000 7fdea1800640 1 -- 172.21.15.47:0/659367757 shutdown_connections 2024-03-21T10:32:33.332 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:33.331+0000 7fdea1800640 1 -- 172.21.15.47:0/659367757 wait complete. 2024-03-21T10:32:33.347 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-21T10:32:33.347 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-21T10:32:33.428 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:32:33 smithi047 ceph-mon[31547]: pgmap v222: 129 pgs: 129 active+clean; 277 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-03-21T10:32:33.599 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:33.599+0000 7f4ce6400640 1 -- 172.21.15.47:0/386234419 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4ce00764f0 msgr2=0x7f4ce0111fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:32:33.599 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:33.599+0000 7f4ce6400640 1 --2- 172.21.15.47:0/386234419 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4ce00764f0 0x7f4ce0111fa0 secure :-1 s=READY pgs=1045 cs=0 l=1 rev1=1 crypto rx=0x7f4ccc009900 tx=0x7f4ccc02bc30 comp rx=0 tx=0).stop 2024-03-21T10:32:33.599 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:33.599+0000 7f4ce6400640 1 -- 172.21.15.47:0/386234419 shutdown_connections 2024-03-21T10:32:33.599 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:33.599+0000 7f4ce6400640 1 --2- 172.21.15.47:0/386234419 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4ce01124e0 0x7f4ce01148d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:32:33.599 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:33.599+0000 7f4ce6400640 1 --2- 172.21.15.47:0/386234419 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4ce00764f0 0x7f4ce0111fa0 unknown :-1 s=CLOSED pgs=1045 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:32:33.599 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:33.599+0000 7f4ce6400640 1 --2- 172.21.15.47:0/386234419 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4ce0075bd0 0x7f4ce0075fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:32:33.599 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:33.599+0000 7f4ce6400640 1 -- 172.21.15.47:0/386234419 >> 172.21.15.47:0/386234419 conn(0x7f4ce00ff2f0 msgr2=0x7f4ce0101710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:32:33.599 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:33.599+0000 7f4ce6400640 1 -- 172.21.15.47:0/386234419 shutdown_connections 2024-03-21T10:32:33.599 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:33.599+0000 7f4ce6400640 1 -- 172.21.15.47:0/386234419 wait complete. 2024-03-21T10:32:33.600 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:33.600+0000 7f4ce6400640 1 Processor -- start 2024-03-21T10:32:33.600 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:33.600+0000 7f4ce6400640 1 -- start start 2024-03-21T10:32:33.601 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:33.601+0000 7f4ce6400640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4ce0075bd0 0x7f4ce019e4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:32:33.601 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:33.601+0000 7f4ce6400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4ce00764f0 0x7f4ce019e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:32:33.601 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:33.601+0000 7f4ce6400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4ce01124e0 0x7f4ce01a2d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:32:33.601 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:33.601+0000 7f4ce6400640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ce0117ef0 con 0x7f4ce01124e0 2024-03-21T10:32:33.601 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:33.601+0000 7f4ce6400640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ce0117d70 con 0x7f4ce00764f0 2024-03-21T10:32:33.601 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:33.601+0000 7f4ce6400640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ce0118070 con 0x7f4ce0075bd0 2024-03-21T10:32:33.602 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:33.601+0000 7f4cdea00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4ce00764f0 0x7f4ce019e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:32:33.602 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:33.601+0000 7f4cdfe00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4ce01124e0 0x7f4ce01a2d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:32:33.602 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:33.601+0000 7f4cdea00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4ce00764f0 0x7f4ce019e9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:57268/0 (socket says 172.21.15.47:57268) 2024-03-21T10:32:33.602 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:33.601+0000 7f4cdea00640 1 -- 172.21.15.47:0/2314288436 learned_addr learned my addr 172.21.15.47:0/2314288436 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:32:33.602 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:33.602+0000 7f4cdf400640 1 --2- 172.21.15.47:0/2314288436 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4ce0075bd0 0x7f4ce019e4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:32:33.603 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:33.602+0000 7f4cdfe00640 1 -- 172.21.15.47:0/2314288436 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4ce0075bd0 msgr2=0x7f4ce019e4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:32:33.603 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:33.602+0000 7f4cdfe00640 1 --2- 172.21.15.47:0/2314288436 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4ce0075bd0 0x7f4ce019e4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:32:33.603 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:33.602+0000 7f4cdfe00640 1 -- 172.21.15.47:0/2314288436 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4ce00764f0 msgr2=0x7f4ce019e9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:32:33.603 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:33.602+0000 7f4cdfe00640 1 --2- 172.21.15.47:0/2314288436 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4ce00764f0 0x7f4ce019e9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:32:33.603 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:33.602+0000 7f4cdfe00640 1 -- 172.21.15.47:0/2314288436 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ce01a34a0 con 0x7f4ce01124e0 2024-03-21T10:32:33.603 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:33.602+0000 7f4cdea00640 1 --2- 172.21.15.47:0/2314288436 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4ce00764f0 0x7f4ce019e9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T10:32:33.603 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:33.602+0000 7f4cdf400640 1 --2- 172.21.15.47:0/2314288436 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4ce0075bd0 0x7f4ce019e4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T10:32:33.603 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:33.602+0000 7f4cdfe00640 1 --2- 172.21.15.47:0/2314288436 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4ce01124e0 0x7f4ce01a2d70 secure :-1 s=READY pgs=1046 cs=0 l=1 rev1=1 crypto rx=0x7f4cd8005fd0 tx=0x7f4cd800ef10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:32:33.603 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:33.603+0000 7f4cd7e00640 1 -- 172.21.15.47:0/2314288436 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4cd800cdf0 con 0x7f4ce01124e0 2024-03-21T10:32:33.603 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:33.603+0000 7f4cd7e00640 1 -- 172.21.15.47:0/2314288436 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f4cd8005020 con 0x7f4ce01124e0 2024-03-21T10:32:33.603 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:33.603+0000 7f4cd7e00640 1 -- 172.21.15.47:0/2314288436 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4cd8010690 con 0x7f4ce01124e0 2024-03-21T10:32:33.603 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:33.603+0000 7f4ce6400640 1 -- 172.21.15.47:0/2314288436 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4ce01a3790 con 0x7f4ce01124e0 2024-03-21T10:32:33.603 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:33.603+0000 7f4ce6400640 1 -- 172.21.15.47:0/2314288436 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4ce01ab020 con 0x7f4ce01124e0 2024-03-21T10:32:33.606 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:33.606+0000 7f4cd7e00640 1 -- 172.21.15.47:0/2314288436 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 27) v1 ==== 137985+0+0 (secure 0 0 0) 0x7f4cd8004a40 con 0x7f4ce01124e0 2024-03-21T10:32:33.606 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:33.606+0000 7f4ce91cb740 1 Processor -- start 2024-03-21T10:32:33.606 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:33.607+0000 7f4ce91cb740 1 -- start start 2024-03-21T10:32:33.607 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:33.607+0000 7f4ce91cb740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5584d5e8cc00 0x5584d5e8b950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:32:33.607 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:33.607+0000 7f4ce91cb740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5584d5e8bf20 0x5584d5e8c3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:32:33.608 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:33.607+0000 7f4ce91cb740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5584d5e95290 0x5584d5e97680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:32:33.608 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:33.607+0000 7f4ce91cb740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x5584d5df0250 con 0x5584d5e8bf20 2024-03-21T10:32:33.608 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:33.607+0000 7f4ce91cb740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x5584d5d98a30 con 0x5584d5e8cc00 2024-03-21T10:32:33.608 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:33.607+0000 7f4ce91cb740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x5584d5dec2f0 con 0x5584d5e95290 2024-03-21T10:32:33.608 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:33.608+0000 7f4cdf400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5584d5e8bf20 0x5584d5e8c3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:32:33.608 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:33.608+0000 7f4cdea00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5584d5e8cc00 0x5584d5e8b950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:32:33.608 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:33.608+0000 7f4cdea00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5584d5e8cc00 0x5584d5e8b950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:57272/0 (socket says 172.21.15.47:57272) 2024-03-21T10:32:33.608 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:33.608+0000 7f4cdea00640 1 -- 172.21.15.47:0/135201289 learned_addr learned my addr 172.21.15.47:0/135201289 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:32:33.608 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:33.608+0000 7f4cdea00640 1 --2- 172.21.15.47:0/135201289 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5584d5e95290 0x5584d5e97680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:32:33.608 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:33.608+0000 7f4cdf400640 1 -- 172.21.15.47:0/135201289 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5584d5e95290 msgr2=0x5584d5e97680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:32:33.608 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:33.608+0000 7f4cdf400640 1 --2- 172.21.15.47:0/135201289 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5584d5e95290 0x5584d5e97680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:32:33.608 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:33.608+0000 7f4cdf400640 1 -- 172.21.15.47:0/135201289 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5584d5e8cc00 msgr2=0x5584d5e8b950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:32:33.608 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:33.608+0000 7f4cdf400640 1 --2- 172.21.15.47:0/135201289 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5584d5e8cc00 0x5584d5e8b950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:32:33.608 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:33.608+0000 7f4cdf400640 1 -- 172.21.15.47:0/135201289 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5584d5d83690 con 0x5584d5e8bf20 2024-03-21T10:32:33.608 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:33.608+0000 7f4cdea00640 1 --2- 172.21.15.47:0/135201289 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5584d5e8cc00 0x5584d5e8b950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T10:32:33.610 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:33.608+0000 7f4cd7e00640 1 --2- 172.21.15.47:0/2314288436 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f4cc40a3110 0x7f4cc40a55d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:32:33.610 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:33.608+0000 7f4cdf400640 1 --2- 172.21.15.47:0/135201289 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5584d5e8bf20 0x5584d5e8c3a0 secure :-1 s=READY pgs=1047 cs=0 l=1 rev1=1 crypto rx=0x7f4cd000bc80 tx=0x7f4cd000f730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=45b8ba7e2ed5b089 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:32:33.610 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:33.609+0000 7f4cd7e00640 1 -- 172.21.15.47:0/2314288436 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 9562+0+0 (secure 0 0 0) 0x7f4cd80d02d0 con 0x7f4ce01124e0 2024-03-21T10:32:33.610 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:33.609+0000 7f4cdea00640 1 --2- 172.21.15.47:0/2314288436 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f4cc40a3110 0x7f4cc40a55d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:32:33.610 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:33.609+0000 7f4cd4c00640 1 -- 172.21.15.47:0/135201289 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4cd0015070 con 0x5584d5e8bf20 2024-03-21T10:32:33.610 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:33.609+0000 7f4cd4c00640 1 -- 172.21.15.47:0/135201289 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f4cd001a910 con 0x5584d5e8bf20 2024-03-21T10:32:33.610 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:33.609+0000 7f4cd4c00640 1 -- 172.21.15.47:0/135201289 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4cd00118a0 con 0x5584d5e8bf20 2024-03-21T10:32:33.610 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:33.610+0000 7f4ce91cb740 1 -- 172.21.15.47:0/135201289 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5584d5e8bf20 msgr2=0x5584d5e8c3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:32:33.611 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:33.610+0000 7f4cdea00640 1 --2- 172.21.15.47:0/2314288436 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f4cc40a3110 0x7f4cc40a55d0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f4ccc005c40 tx=0x7f4ccc005b50 comp rx=0 tx=0).ready entity=mgr.15687 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:32:33.611 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:33.610+0000 7f4ce91cb740 1 --2- 172.21.15.47:0/135201289 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5584d5e8bf20 0x5584d5e8c3a0 secure :-1 s=READY pgs=1047 cs=0 l=1 rev1=1 crypto rx=0x7f4cd000bc80 tx=0x7f4cd000f730 comp rx=0 tx=0).stop 2024-03-21T10:32:33.611 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:33.610+0000 7f4ce91cb740 1 -- 172.21.15.47:0/135201289 shutdown_connections 2024-03-21T10:32:33.611 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:33.610+0000 7f4ce91cb740 1 --2- 172.21.15.47:0/135201289 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5584d5e95290 0x5584d5e97680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:32:33.611 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:33.610+0000 7f4ce91cb740 1 --2- 172.21.15.47:0/135201289 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5584d5e8bf20 0x5584d5e8c3a0 unknown :-1 s=CLOSED pgs=1047 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:32:33.611 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:33.610+0000 7f4ce91cb740 1 --2- 172.21.15.47:0/135201289 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5584d5e8cc00 0x5584d5e8b950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:32:33.611 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:33.610+0000 7f4ce91cb740 1 -- 172.21.15.47:0/135201289 >> 172.21.15.47:0/135201289 conn(0x5584d5e53200 msgr2=0x5584d5e86970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:32:33.611 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:33.610+0000 7f4ce91cb740 1 -- 172.21.15.47:0/135201289 shutdown_connections 2024-03-21T10:32:33.611 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:33.610+0000 7f4ce91cb740 1 -- 172.21.15.47:0/135201289 wait complete. 2024-03-21T10:32:33.611 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:33.610+0000 7f4ce91cb740 1 -- start start 2024-03-21T10:32:33.611 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:33.611+0000 7f4ce91cb740 1 Processor -- start 2024-03-21T10:32:33.614 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:33.611+0000 7f4ce91cb740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5584d5e8bf20 0x5584d5e94890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:32:33.614 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:33.611+0000 7f4ce91cb740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5584d5e8cc00 0x5584d5e91c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:32:33.614 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:33.611+0000 7f4ce91cb740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5584d5e95290 0x5584d5e92160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:32:33.614 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:33.611+0000 7f4ce91cb740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x5584d5e98150 con 0x5584d5e95290 2024-03-21T10:32:33.615 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:33.611+0000 7f4ce91cb740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x5584d5dabe80 con 0x5584d5e8bf20 2024-03-21T10:32:33.615 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:33.611+0000 7f4ce91cb740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x5584d5e982d0 con 0x5584d5e8cc00 2024-03-21T10:32:33.615 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:33.611+0000 7f4ce91cb740 10 client.0 _ll_register_callbacks cb 0x5584d5dbe130 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T10:32:33.615 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:33.612+0000 7f4cdfe00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5584d5e95290 0x5584d5e92160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:32:33.615 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:33.612+0000 7f4cdf400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5584d5e8bf20 0x5584d5e94890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:32:33.615 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:33.612+0000 7f4cdfe00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5584d5e95290 0x5584d5e92160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:51110/0 (socket says 172.21.15.47:51110) 2024-03-21T10:32:33.615 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:33.612+0000 7f4cdfe00640 1 -- 172.21.15.47:0/3655719539 learned_addr learned my addr 172.21.15.47:0/3655719539 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:32:33.615 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:33.612+0000 7f4cdf400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5584d5e8bf20 0x5584d5e94890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:57276/0 (socket says 172.21.15.47:57276) 2024-03-21T10:32:33.615 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:33.612+0000 7f4cdea00640 1 --2- 172.21.15.47:0/3655719539 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5584d5e8cc00 0x5584d5e91c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:32:33.615 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:33.612+0000 7f4cdfe00640 1 -- 172.21.15.47:0/3655719539 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5584d5e8cc00 msgr2=0x5584d5e91c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:32:33.615 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:33.612+0000 7f4cdfe00640 1 --2- 172.21.15.47:0/3655719539 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5584d5e8cc00 0x5584d5e91c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:32:33.615 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:33.612+0000 7f4cdfe00640 1 -- 172.21.15.47:0/3655719539 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5584d5e8bf20 msgr2=0x5584d5e94890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:32:33.615 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:33.612+0000 7f4cdfe00640 1 --2- 172.21.15.47:0/3655719539 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5584d5e8bf20 0x5584d5e94890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:32:33.616 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:33.612+0000 7f4cdfe00640 1 -- 172.21.15.47:0/3655719539 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5584d5bce7d0 con 0x5584d5e95290 2024-03-21T10:32:33.616 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:33.612+0000 7f4cdfe00640 1 --2- 172.21.15.47:0/3655719539 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5584d5e95290 0x5584d5e92160 secure :-1 s=READY pgs=1048 cs=0 l=1 rev1=1 crypto rx=0x7f4cd8098c80 tx=0x7f4cd809c2b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5714eb94e09ab7fe server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:32:33.616 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:33.612+0000 7f4cc9600640 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-03-21T10:32:33.616 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:33.612+0000 7f4cc9600640 1 -- 172.21.15.47:0/3655719539 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4cd8004d30 con 0x5584d5e95290 2024-03-21T10:32:33.616 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:33.613+0000 7f4cc9600640 1 -- 172.21.15.47:0/3655719539 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(99..99 src has 1..99) v4 ==== 9562+0+0 (secure 0 0 0) 0x7f4cd80051d0 con 0x5584d5e95290 2024-03-21T10:32:33.616 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:33.613+0000 7f4ce91cb740 1 -- 172.21.15.47:0/3655719539 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f4cd001a910 con 0x5584d5e95290 2024-03-21T10:32:33.616 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:33.613+0000 7f4cc9600640 1 -- 172.21.15.47:0/3655719539 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4cd8098920 con 0x5584d5e95290 2024-03-21T10:32:33.616 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:33.613+0000 7f4ce91cb740 10 client.17853 fetch_fsmap learned FSMap version 32 2024-03-21T10:32:33.616 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:33.613+0000 7f4ce91cb740 1 -- 172.21.15.47:0/3655719539 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x5584d5b043f0 con 0x5584d5e95290 2024-03-21T10:32:33.616 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:33.613+0000 7f4cc9600640 1 -- 172.21.15.47:0/3655719539 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f4cd001a910 con 0x5584d5e95290 2024-03-21T10:32:33.616 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:33.613+0000 7f4cc9600640 1 -- 172.21.15.47:0/3655719539 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 32) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f4cd80a4830 con 0x5584d5e95290 2024-03-21T10:32:33.616 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:33.613+0000 7f4ce91cb740 10 client.17853 fetch_fsmap finished waiting for FSMap version 32 2024-03-21T10:32:33.616 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:33.613+0000 7f4ce91cb740 10 client.17853 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-21T10:32:33.617 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:33.614+0000 7f4ce91cb740 20 client.17853 populate_metadata read hostname 'smithi047' 2024-03-21T10:32:33.617 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:33.614+0000 7f4ce91cb740 1 --2- 172.21.15.47:0/3655719539 >> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] conn(0x5584d5ea50e0 0x5584d5ea74d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:32:33.617 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:33.614+0000 7f4ce91cb740 4 client.17853 mds_command: new command op to 34535 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T10:32:33.617 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:33.614+0000 7f4ce91cb740 1 -- 172.21.15.47:0/3655719539 --> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5584d5dfd790 con 0x5584d5ea50e0 2024-03-21T10:32:33.617 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:33.614+0000 7f4cdf400640 1 --2- 172.21.15.47:0/3655719539 >> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] conn(0x5584d5ea50e0 0x5584d5ea74d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:32:33.617 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:33.615+0000 7f4cdf400640 1 --2- 172.21.15.47:0/3655719539 >> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] conn(0x5584d5ea50e0 0x5584d5ea74d0 crc :-1 s=READY pgs=175 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3566d89f1094b509 server_cookie=e135b90b173bd753 in_seq=0 out_seq=0 2024-03-21T10:32:33.617 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:33.615+0000 7f4cc9600640 10 client.17853 ms_handle_connect on v2:172.21.15.100:6838/4173168440 2024-03-21T10:32:33.618 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:33.615+0000 7f4cc9600640 1 -- 172.21.15.47:0/3655719539 <== mds.0 v2:172.21.15.100:6838/4173168440 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x5584d5dfd790 con 0x5584d5ea50e0 2024-03-21T10:32:33.618 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:33.615+0000 7f4cc9600640 10 client.17853 handle_command_reply: tid=0 2024-03-21T10:32:33.618 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:33.615+0000 7f4ce91cb740 1 client.17853 shutdown 2024-03-21T10:32:33.618 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:33.616+0000 7f4ce91cb740 1 -- 172.21.15.47:0/3655719539 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5584d5e95290 msgr2=0x5584d5e92160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:32:33.619 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:33.616+0000 7f4ce91cb740 1 --2- 172.21.15.47:0/3655719539 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5584d5e95290 0x5584d5e92160 secure :-1 s=READY pgs=1048 cs=0 l=1 rev1=1 crypto rx=0x7f4cd8098c80 tx=0x7f4cd809c2b0 comp rx=0 tx=0).stop 2024-03-21T10:32:33.619 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:33.616+0000 7f4ce91cb740 1 -- 172.21.15.47:0/3655719539 shutdown_connections 2024-03-21T10:32:33.619 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:33.616+0000 7f4ce91cb740 1 --2- 172.21.15.47:0/3655719539 >> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] conn(0x5584d5ea50e0 0x5584d5ea74d0 crc :-1 s=READY pgs=175 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:32:33.619 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:33.616+0000 7f4ce91cb740 1 --2- 172.21.15.47:0/3655719539 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5584d5e95290 0x5584d5e92160 unknown :-1 s=CLOSED pgs=1048 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:32:33.619 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:33.616+0000 7f4ce91cb740 1 --2- 172.21.15.47:0/3655719539 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5584d5e8cc00 0x5584d5e91c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:32:33.619 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:33.616+0000 7f4ce91cb740 1 --2- 172.21.15.47:0/3655719539 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5584d5e8bf20 0x5584d5e94890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:32:33.619 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:33.616+0000 7f4ce91cb740 1 -- 172.21.15.47:0/3655719539 >> 172.21.15.47:0/3655719539 conn(0x5584d5e53200 msgr2=0x5584d5e886c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:32:33.619 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:33.616+0000 7f4cc9600640 0 client.17853 ms_handle_reset on v2:172.21.15.100:6838/4173168440 2024-03-21T10:32:33.619 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:33.616+0000 7f4ce91cb740 1 -- 172.21.15.47:0/3655719539 shutdown_connections 2024-03-21T10:32:33.619 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:33.616+0000 7f4ce91cb740 1 -- 172.21.15.47:0/3655719539 wait complete. 2024-03-21T10:32:33.619 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:33.616+0000 7f4ce91cb740 20 client.17853 trim_cache size 0 max 16384 2024-03-21T10:32:33.637 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:33.637+0000 7f4ce91cb740 1 Processor -- start 2024-03-21T10:32:33.637 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:33.637+0000 7f4ce91cb740 1 -- start start 2024-03-21T10:32:33.637 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:33.637+0000 7f4ce91cb740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5584d5e8bf20 0x5584d5e96680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:32:33.637 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:33.637+0000 7f4ce91cb740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5584d5e8cc00 0x5584d5e96bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:32:33.638 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:33.637+0000 7f4ce91cb740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5584d5e95290 0x5584d5e94350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:32:33.638 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:33.637+0000 7f4ce91cb740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ce01b4fd0 con 0x5584d5e8cc00 2024-03-21T10:32:33.638 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:33.637+0000 7f4ce91cb740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ce01b6950 con 0x5584d5e8bf20 2024-03-21T10:32:33.638 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:33.637+0000 7f4ce91cb740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ce01b5630 con 0x5584d5e95290 2024-03-21T10:32:33.638 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:33.638+0000 7f4cdea00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5584d5e8cc00 0x5584d5e96bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:32:33.638 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:33.638+0000 7f4cdea00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5584d5e8cc00 0x5584d5e96bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:51116/0 (socket says 172.21.15.47:51116) 2024-03-21T10:32:33.638 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:33.638+0000 7f4cdea00640 1 -- 172.21.15.47:0/3040084945 learned_addr learned my addr 172.21.15.47:0/3040084945 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:32:33.638 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:33.638+0000 7f4cdf400640 1 --2- 172.21.15.47:0/3040084945 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5584d5e8bf20 0x5584d5e96680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:32:33.640 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:33.638+0000 7f4cdea00640 1 -- 172.21.15.47:0/3040084945 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5584d5e95290 msgr2=0x5584d5e94350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:32:33.640 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:33.638+0000 7f4cdea00640 1 --2- 172.21.15.47:0/3040084945 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5584d5e95290 0x5584d5e94350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:32:33.640 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:33.638+0000 7f4cdea00640 1 -- 172.21.15.47:0/3040084945 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5584d5e8bf20 msgr2=0x5584d5e96680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:32:33.640 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:33.638+0000 7f4cdea00640 1 --2- 172.21.15.47:0/3040084945 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5584d5e8bf20 0x5584d5e96680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:32:33.640 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:33.638+0000 7f4cdea00640 1 -- 172.21.15.47:0/3040084945 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5584d5ac40f0 con 0x5584d5e8cc00 2024-03-21T10:32:33.640 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:33.639+0000 7f4cdea00640 1 --2- 172.21.15.47:0/3040084945 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5584d5e8cc00 0x5584d5e96bc0 secure :-1 s=READY pgs=1049 cs=0 l=1 rev1=1 crypto rx=0x7f4ccc036040 tx=0x7f4ccc042000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bc134fd1bf7056d0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:32:33.640 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:33.639+0000 7f4ccbe00640 1 -- 172.21.15.47:0/3040084945 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ccc0035f0 con 0x5584d5e8cc00 2024-03-21T10:32:33.640 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:33.639+0000 7f4ccbe00640 1 -- 172.21.15.47:0/3040084945 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f4ccc054070 con 0x5584d5e8cc00 2024-03-21T10:32:33.640 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:33.639+0000 7f4ccbe00640 1 -- 172.21.15.47:0/3040084945 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ccc042a70 con 0x5584d5e8cc00 2024-03-21T10:32:33.640 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:33.640+0000 7f4ce91cb740 1 -- 172.21.15.47:0/3040084945 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5584d5e8cc00 msgr2=0x5584d5e96bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:32:33.641 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:33.640+0000 7f4ce91cb740 1 --2- 172.21.15.47:0/3040084945 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5584d5e8cc00 0x5584d5e96bc0 secure :-1 s=READY pgs=1049 cs=0 l=1 rev1=1 crypto rx=0x7f4ccc036040 tx=0x7f4ccc042000 comp rx=0 tx=0).stop 2024-03-21T10:32:33.641 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:33.640+0000 7f4ce91cb740 1 -- 172.21.15.47:0/3040084945 shutdown_connections 2024-03-21T10:32:33.641 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:33.640+0000 7f4ce91cb740 1 --2- 172.21.15.47:0/3040084945 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5584d5e95290 0x5584d5e94350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:32:33.641 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:33.640+0000 7f4ce91cb740 1 --2- 172.21.15.47:0/3040084945 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5584d5e8cc00 0x5584d5e96bc0 unknown :-1 s=CLOSED pgs=1049 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:32:33.641 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:33.640+0000 7f4ce91cb740 1 --2- 172.21.15.47:0/3040084945 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5584d5e8bf20 0x5584d5e96680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:32:33.641 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:33.640+0000 7f4ce91cb740 1 -- 172.21.15.47:0/3040084945 >> 172.21.15.47:0/3040084945 conn(0x5584d5e53200 msgr2=0x5584d5e90530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:32:33.641 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:33.640+0000 7f4ce91cb740 1 -- 172.21.15.47:0/3040084945 shutdown_connections 2024-03-21T10:32:33.641 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:33.640+0000 7f4ce91cb740 1 -- 172.21.15.47:0/3040084945 wait complete. 2024-03-21T10:32:33.642 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:33.640+0000 7f4ce91cb740 1 -- start start 2024-03-21T10:32:33.642 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:33.640+0000 7f4ce91cb740 1 Processor -- start 2024-03-21T10:32:33.645 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:33.641+0000 7f4ce91cb740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5584d5e8bf20 0x5584d5e8e8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:32:33.645 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:33.641+0000 7f4ce91cb740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5584d5e8cc00 0x5584d5e8ee10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:32:33.646 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:33.641+0000 7f4ce91cb740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5584d5e95290 0x5584d5eff960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:32:33.646 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:33.641+0000 7f4ce91cb740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ce01b62f0 con 0x5584d5e95290 2024-03-21T10:32:33.646 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:33.641+0000 7f4ce91cb740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ce01b5c90 con 0x5584d5e8bf20 2024-03-21T10:32:33.646 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:33.641+0000 7f4ce91cb740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ce01b31b0 con 0x5584d5e8cc00 2024-03-21T10:32:33.646 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:33.641+0000 7f4ce91cb740 10 client.0 _ll_register_callbacks cb 0x7f4cd80057a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T10:32:33.646 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:33.641+0000 7f4cdfe00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5584d5e95290 0x5584d5eff960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:32:33.646 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:33.641+0000 7f4cdfe00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5584d5e95290 0x5584d5eff960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:51124/0 (socket says 172.21.15.47:51124) 2024-03-21T10:32:33.646 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:33.641+0000 7f4cdfe00640 1 -- 172.21.15.47:0/4115879508 learned_addr learned my addr 172.21.15.47:0/4115879508 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:32:33.646 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:33.641+0000 7f4cdf400640 1 --2- 172.21.15.47:0/4115879508 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5584d5e8bf20 0x5584d5e8e8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:32:33.646 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:33.642+0000 7f4cdfe00640 1 -- 172.21.15.47:0/4115879508 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5584d5e8cc00 msgr2=0x5584d5e8ee10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:32:33.646 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:33.642+0000 7f4cdfe00640 1 --2- 172.21.15.47:0/4115879508 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5584d5e8cc00 0x5584d5e8ee10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:32:33.647 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:33.642+0000 7f4cdfe00640 1 -- 172.21.15.47:0/4115879508 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5584d5e8bf20 msgr2=0x5584d5e8e8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:32:33.647 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:33.642+0000 7f4cdfe00640 1 --2- 172.21.15.47:0/4115879508 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5584d5e8bf20 0x5584d5e8e8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:32:33.647 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:33.642+0000 7f4cdfe00640 1 -- 172.21.15.47:0/4115879508 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5584d5effea0 con 0x5584d5e95290 2024-03-21T10:32:33.647 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:33.642+0000 7f4cdfe00640 1 --2- 172.21.15.47:0/4115879508 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5584d5e95290 0x5584d5eff960 secure :-1 s=READY pgs=1050 cs=0 l=1 rev1=1 crypto rx=0x7f4cd809bfb0 tx=0x7f4cd809c9b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7591e150a279139c server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:32:33.647 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:33.642+0000 7f4cc9600640 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-03-21T10:32:33.647 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:33.642+0000 7f4cc9600640 1 -- 172.21.15.47:0/4115879508 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4cd8098000 con 0x5584d5e95290 2024-03-21T10:32:33.647 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:33.642+0000 7f4cc9600640 1 -- 172.21.15.47:0/4115879508 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(99..99 src has 1..99) v4 ==== 9562+0+0 (secure 0 0 0) 0x7f4cd8004d30 con 0x5584d5e95290 2024-03-21T10:32:33.647 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:33.642+0000 7f4ce91cb740 1 -- 172.21.15.47:0/4115879508 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f4ccc054070 con 0x5584d5e95290 2024-03-21T10:32:33.647 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:33.643+0000 7f4cc9600640 1 -- 172.21.15.47:0/4115879508 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4cd80b5200 con 0x5584d5e95290 2024-03-21T10:32:33.647 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:33.643+0000 7f4ce91cb740 10 client.17859 fetch_fsmap learned FSMap version 32 2024-03-21T10:32:33.647 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:33.643+0000 7f4ce91cb740 1 -- 172.21.15.47:0/4115879508 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x5584d5f01050 con 0x5584d5e95290 2024-03-21T10:32:33.647 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:33.643+0000 7f4cc9600640 1 -- 172.21.15.47:0/4115879508 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f4ccc054070 con 0x5584d5e95290 2024-03-21T10:32:33.647 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:33.643+0000 7f4cc9600640 1 -- 172.21.15.47:0/4115879508 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 32) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f4cd809b170 con 0x5584d5e95290 2024-03-21T10:32:33.647 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:33.643+0000 7f4ce91cb740 10 client.17859 fetch_fsmap finished waiting for FSMap version 32 2024-03-21T10:32:33.647 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:33.643+0000 7f4ce91cb740 10 client.17859 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-21T10:32:33.648 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:33.643+0000 7f4ce91cb740 20 client.17859 populate_metadata read hostname 'smithi047' 2024-03-21T10:32:33.648 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:33.643+0000 7f4ce91cb740 1 --2- 172.21.15.47:0/4115879508 >> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] conn(0x5584d5f016c0 0x5584d5f03ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:32:33.648 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:33.643+0000 7f4ce91cb740 4 client.17859 mds_command: new command op to 34535 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-21T10:32:33.648 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:33.643+0000 7f4ce91cb740 1 -- 172.21.15.47:0/4115879508 --> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x5584d5df7ae0 con 0x5584d5f016c0 2024-03-21T10:32:33.648 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:33.644+0000 7f4cdf400640 1 --2- 172.21.15.47:0/4115879508 >> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] conn(0x5584d5f016c0 0x5584d5f03ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:32:33.648 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:33.644+0000 7f4cdf400640 1 --2- 172.21.15.47:0/4115879508 >> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] conn(0x5584d5f016c0 0x5584d5f03ae0 crc :-1 s=READY pgs=176 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e072e337ab7ecce8 server_cookie=23d7eab5d9856107 in_seq=0 out_seq=0 2024-03-21T10:32:33.648 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:33.644+0000 7f4cc9600640 10 client.17859 ms_handle_connect on v2:172.21.15.100:6838/4173168440 2024-03-21T10:32:33.648 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:33.644+0000 7f4cc9600640 1 -- 172.21.15.47:0/4115879508 <== mds.0 v2:172.21.15.100:6838/4173168440 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x5584d5df7ae0 con 0x5584d5f016c0 2024-03-21T10:32:33.648 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:33.644+0000 7f4cc9600640 10 client.17859 handle_command_reply: tid=0 2024-03-21T10:32:33.648 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:33.645+0000 7f4ce91cb740 1 client.17859 shutdown 2024-03-21T10:32:33.648 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:33.645+0000 7f4ce91cb740 1 -- 172.21.15.47:0/4115879508 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5584d5e95290 msgr2=0x5584d5eff960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:32:33.648 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:33.645+0000 7f4ce91cb740 1 --2- 172.21.15.47:0/4115879508 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5584d5e95290 0x5584d5eff960 secure :-1 s=READY pgs=1050 cs=0 l=1 rev1=1 crypto rx=0x7f4cd809bfb0 tx=0x7f4cd809c9b0 comp rx=0 tx=0).stop 2024-03-21T10:32:33.648 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:33.645+0000 7f4ce91cb740 1 -- 172.21.15.47:0/4115879508 shutdown_connections 2024-03-21T10:32:33.649 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:33.645+0000 7f4ce91cb740 1 --2- 172.21.15.47:0/4115879508 >> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] conn(0x5584d5f016c0 0x5584d5f03ae0 crc :-1 s=READY pgs=176 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:32:33.649 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:33.645+0000 7f4ce91cb740 1 --2- 172.21.15.47:0/4115879508 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5584d5e95290 0x5584d5eff960 unknown :-1 s=CLOSED pgs=1050 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:32:33.649 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:33.645+0000 7f4ce91cb740 1 --2- 172.21.15.47:0/4115879508 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5584d5e8cc00 0x5584d5e8ee10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:32:33.649 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:33.645+0000 7f4ce91cb740 1 --2- 172.21.15.47:0/4115879508 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5584d5e8bf20 0x5584d5e8e8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:32:33.649 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:33.645+0000 7f4cc9600640 0 client.17859 ms_handle_reset on v2:172.21.15.100:6838/4173168440 2024-03-21T10:32:33.649 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:33.645+0000 7f4ce91cb740 1 -- 172.21.15.47:0/4115879508 >> 172.21.15.47:0/4115879508 conn(0x5584d5e53200 msgr2=0x5584d5e85a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:32:33.649 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:33.645+0000 7f4ce91cb740 1 -- 172.21.15.47:0/4115879508 shutdown_connections 2024-03-21T10:32:33.649 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:33.645+0000 7f4ce91cb740 1 -- 172.21.15.47:0/4115879508 wait complete. 2024-03-21T10:32:33.649 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:33.645+0000 7f4ce91cb740 20 client.17859 trim_cache size 0 max 16384 2024-03-21T10:32:33.649 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-03-21T10:32:33.650 INFO:teuthology.orchestra.run.smithi047.stdout: "status": "no active scrubs running", 2024-03-21T10:32:33.650 INFO:teuthology.orchestra.run.smithi047.stdout: "scrubs": {} 2024-03-21T10:32:33.650 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-03-21T10:32:33.650 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:33.646+0000 7f4ce6400640 1 -- 172.21.15.47:0/2314288436 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f4cc40a3110 msgr2=0x7f4cc40a55d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:32:33.651 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:33.646+0000 7f4ce6400640 1 --2- 172.21.15.47:0/2314288436 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f4cc40a3110 0x7f4cc40a55d0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f4ccc005c40 tx=0x7f4ccc005b50 comp rx=0 tx=0).stop 2024-03-21T10:32:33.651 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:33.646+0000 7f4ce6400640 1 -- 172.21.15.47:0/2314288436 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4ce01124e0 msgr2=0x7f4ce01a2d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:32:33.651 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:33.646+0000 7f4ce6400640 1 --2- 172.21.15.47:0/2314288436 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4ce01124e0 0x7f4ce01a2d70 secure :-1 s=READY pgs=1046 cs=0 l=1 rev1=1 crypto rx=0x7f4cd8005fd0 tx=0x7f4cd800ef10 comp rx=0 tx=0).stop 2024-03-21T10:32:33.651 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:33.646+0000 7f4ce6400640 1 -- 172.21.15.47:0/2314288436 shutdown_connections 2024-03-21T10:32:33.651 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:33.646+0000 7f4ce6400640 1 --2- 172.21.15.47:0/2314288436 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4ce01124e0 0x7f4ce01a2d70 unknown :-1 s=CLOSED pgs=1046 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:32:33.651 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:33.646+0000 7f4ce6400640 1 --2- 172.21.15.47:0/2314288436 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f4cc40a3110 0x7f4cc40a55d0 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:32:33.651 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:33.646+0000 7f4ce6400640 1 --2- 172.21.15.47:0/2314288436 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4ce00764f0 0x7f4ce019e9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:32:33.651 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:33.646+0000 7f4ce6400640 1 --2- 172.21.15.47:0/2314288436 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4ce0075bd0 0x7f4ce019e4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:32:33.651 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:33.646+0000 7f4ce6400640 1 -- 172.21.15.47:0/2314288436 >> 172.21.15.47:0/2314288436 conn(0x7f4ce00ff2f0 msgr2=0x7f4ce01016e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:32:33.651 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:33.646+0000 7f4ce6400640 1 -- 172.21.15.47:0/2314288436 shutdown_connections 2024-03-21T10:32:33.652 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:33.646+0000 7f4ce6400640 1 -- 172.21.15.47:0/2314288436 wait complete. 2024-03-21T10:32:33.662 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-03-21T10:32:33.663 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-21T10:32:33.727 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:32:33 smithi100 ceph-mon[35193]: pgmap v222: 129 pgs: 129 active+clean; 277 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-03-21T10:32:33.737 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:32:33 smithi157 ceph-mon[35581]: pgmap v222: 129 pgs: 129 active+clean; 277 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-03-21T10:32:33.881 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:33.876+0000 7f6feda00640 1 -- 172.21.15.47:0/3111272950 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6fe8107640 msgr2=0x7f6fe8107a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:32:33.881 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:33.876+0000 7f6feda00640 1 --2- 172.21.15.47:0/3111272950 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6fe8107640 0x7f6fe8107a20 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f6fd8009ca0 tx=0x7f6fd802c100 comp rx=0 tx=0).stop 2024-03-21T10:32:33.882 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:33.876+0000 7f6feda00640 1 -- 172.21.15.47:0/3111272950 shutdown_connections 2024-03-21T10:32:33.882 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:33.876+0000 7f6feda00640 1 --2- 172.21.15.47:0/3111272950 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6fe8101ff0 0x7f6fe810e500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:32:33.882 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:33.876+0000 7f6feda00640 1 --2- 172.21.15.47:0/3111272950 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6fe8101630 0x7f6fe8101ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:32:33.882 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:33.876+0000 7f6feda00640 1 --2- 172.21.15.47:0/3111272950 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6fe8107640 0x7f6fe8107a20 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:32:33.882 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:33.876+0000 7f6feda00640 1 -- 172.21.15.47:0/3111272950 >> 172.21.15.47:0/3111272950 conn(0x7f6fe80fd140 msgr2=0x7f6fe80ff560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:32:33.882 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:33.877+0000 7f6feda00640 1 -- 172.21.15.47:0/3111272950 shutdown_connections 2024-03-21T10:32:33.882 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:33.877+0000 7f6feda00640 1 -- 172.21.15.47:0/3111272950 wait complete. 2024-03-21T10:32:33.882 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:33.878+0000 7f6feda00640 1 Processor -- start 2024-03-21T10:32:33.882 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:33.878+0000 7f6feda00640 1 -- start start 2024-03-21T10:32:33.882 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:33.878+0000 7f6feda00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6fe8101630 0x7f6fe8197e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:32:33.882 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:33.878+0000 7f6feda00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6fe8101ff0 0x7f6fe81983b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:32:33.882 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:33.878+0000 7f6feda00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6fe8107640 0x7f6fe819c730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:32:33.883 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:33.878+0000 7f6feda00640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6fe8111900 con 0x7f6fe8101ff0 2024-03-21T10:32:33.883 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:33.878+0000 7f6feda00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6fe8111780 con 0x7f6fe8107640 2024-03-21T10:32:33.883 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:33.878+0000 7f6feda00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6fe8111a80 con 0x7f6fe8101630 2024-03-21T10:32:33.883 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:33.878+0000 7f6fe6000640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6fe8101ff0 0x7f6fe81983b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:32:33.883 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:33.878+0000 7f6fe6000640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6fe8101ff0 0x7f6fe81983b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:51144/0 (socket says 172.21.15.47:51144) 2024-03-21T10:32:33.883 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:33.878+0000 7f6fe6000640 1 -- 172.21.15.47:0/2765549997 learned_addr learned my addr 172.21.15.47:0/2765549997 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:32:33.883 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:33.878+0000 7f6fe7400640 1 --2- 172.21.15.47:0/2765549997 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6fe8107640 0x7f6fe819c730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:32:33.883 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:33.879+0000 7f6fe6a00640 1 --2- 172.21.15.47:0/2765549997 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6fe8101630 0x7f6fe8197e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:32:33.883 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:33.879+0000 7f6fe7400640 1 -- 172.21.15.47:0/2765549997 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6fe8101630 msgr2=0x7f6fe8197e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:32:33.883 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:33.879+0000 7f6fe7400640 1 --2- 172.21.15.47:0/2765549997 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6fe8101630 0x7f6fe8197e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:32:33.883 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:33.879+0000 7f6fe7400640 1 -- 172.21.15.47:0/2765549997 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6fe8101ff0 msgr2=0x7f6fe81983b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:32:33.883 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:33.879+0000 7f6fe7400640 1 --2- 172.21.15.47:0/2765549997 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6fe8101ff0 0x7f6fe81983b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:32:33.883 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:33.879+0000 7f6fe7400640 1 -- 172.21.15.47:0/2765549997 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6fe819ce60 con 0x7f6fe8107640 2024-03-21T10:32:33.883 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:33.879+0000 7f6fe6000640 1 --2- 172.21.15.47:0/2765549997 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6fe8101ff0 0x7f6fe81983b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T10:32:33.884 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:33.879+0000 7f6fe7400640 1 --2- 172.21.15.47:0/2765549997 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6fe8107640 0x7f6fe819c730 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f6fe000e970 tx=0x7f6fe000ee30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:32:33.884 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:33.879+0000 7f6fdf400640 1 -- 172.21.15.47:0/2765549997 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fe000ccc0 con 0x7f6fe8107640 2024-03-21T10:32:33.884 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:33.880+0000 7f6fdf400640 1 -- 172.21.15.47:0/2765549997 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f6fe0005020 con 0x7f6fe8107640 2024-03-21T10:32:33.884 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:33.880+0000 7f6fdf400640 1 -- 172.21.15.47:0/2765549997 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fe0004a00 con 0x7f6fe8107640 2024-03-21T10:32:33.884 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:33.880+0000 7f6feda00640 1 -- 172.21.15.47:0/2765549997 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6fe819d150 con 0x7f6fe8107640 2024-03-21T10:32:33.884 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:33.880+0000 7f6feda00640 1 -- 172.21.15.47:0/2765549997 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6fe81a49e0 con 0x7f6fe8107640 2024-03-21T10:32:33.884 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:33.881+0000 7f6feda00640 1 -- 172.21.15.47:0/2765549997 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6fe80688e0 con 0x7f6fe8107640 2024-03-21T10:32:33.885 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:33.883+0000 7f6fdf400640 1 -- 172.21.15.47:0/2765549997 <== mon.2 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 27) v1 ==== 137985+0+0 (secure 0 0 0) 0x7f6fe00051d0 con 0x7f6fe8107640 2024-03-21T10:32:33.885 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:33.885+0000 7f6fdf400640 1 --2- 172.21.15.47:0/2765549997 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f6fcc0a3090 0x7f6fcc0a5550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:32:33.885 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:33.885+0000 7f6fdf400640 1 -- 172.21.15.47:0/2765549997 <== mon.2 v2:172.21.15.100:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 9562+0+0 (secure 0 0 0) 0x7f6fe00cfb90 con 0x7f6fe8107640 2024-03-21T10:32:33.885 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:33.885+0000 7f6fe6a00640 1 --2- 172.21.15.47:0/2765549997 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f6fcc0a3090 0x7f6fcc0a5550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:32:33.885 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:33.886+0000 7f6fe6a00640 1 --2- 172.21.15.47:0/2765549997 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f6fcc0a3090 0x7f6fcc0a5550 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f6fd8000f80 tx=0x7f6fd8005c30 comp rx=0 tx=0).ready entity=mgr.15687 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:32:33.890 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:33.889+0000 7f6fdf400640 1 -- 172.21.15.47:0/2765549997 <== mon.2 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f6fe009c210 con 0x7f6fe8107640 2024-03-21T10:32:34.168 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:34.168+0000 7f6feda00640 1 -- 172.21.15.47:0/2765549997 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f6fe802d070 con 0x7f6fe8107640 2024-03-21T10:32:34.170 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:34.170+0000 7f6fdf400640 1 -- 172.21.15.47:0/2765549997 <== mon.2 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 32 v32) v1 ==== 94+0+11471 (secure 0 0 0) 0x7f6fe009bb10 con 0x7f6fe8107640 2024-03-21T10:32:34.170 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-21T10:32:34.171 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":32,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14511,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6835/735724037","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6834","nonce":735724037},{"type":"v1","addr":"172.21.15.47:6835","nonce":735724037}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14529,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6837/2248191724","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6836","nonce":2248191724},{"type":"v1","addr":"172.21.15.47:6837","nonce":2248191724}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":32,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-21T10:21:58.329063+0000","modified":"2024-03-21T10:26:17.169892+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":34535,"mds_1":34493,"mds_2":34454,"mds_3":34427,"mds_4":14619},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14604":{"gid":14604,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.47:6839/2881335356","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6838","nonce":2881335356},{"type":"v1","addr":"172.21.15.47:6839","nonce":2881335356}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_14619":{"gid":14619,"name":"a","rank":4,"incarnation":27,"state":"up:active","state_seq":11,"addr":"172.21.15.47:6841/3401578978","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6840","nonce":3401578978},{"type":"v1","addr":"172.21.15.47:6841","nonce":3401578978}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24406":{"gid":24406,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.157:6833/4000010136","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":4000010136},{"type":"v1","addr":"172.21.15.157:6833","nonce":4000010136}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24457":{"gid":24457,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.157:6835/3902868651","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":3902868651},{"type":"v1","addr":"172.21.15.157:6835","nonce":3902868651}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24475":{"gid":24475,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.157:6837/1908087204","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6836","nonce":1908087204},{"type":"v1","addr":"172.21.15.157:6837","nonce":1908087204}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_25729":{"gid":25729,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.157:6839/1998408416","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6838","nonce":1998408416},{"type":"v1","addr":"172.21.15.157:6839","nonce":1998408416}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_34427":{"gid":34427,"name":"k","rank":3,"incarnation":25,"state":"up:active","state_seq":20,"addr":"172.21.15.100:6833/2723302038","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6832","nonce":2723302038},{"type":"v1","addr":"172.21.15.100:6833","nonce":2723302038}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_34454":{"gid":34454,"name":"e","rank":2,"incarnation":22,"state":"up:active","state_seq":17,"addr":"172.21.15.100:6835/2597849957","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":2597849957},{"type":"v1","addr":"172.21.15.100:6835","nonce":2597849957}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_34493":{"gid":34493,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":13,"addr":"172.21.15.100:6837/3497644162","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6836","nonce":3497644162},{"type":"v1","addr":"172.21.15.100:6837","nonce":3497644162}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_34535":{"gid":34535,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.100:6839/4173168440","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6838","nonce":4173168440},{"type":"v1","addr":"172.21.15.100:6839","nonce":4173168440}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":34535,"qdb_cluster":[34427,14619,34454,34535,34493]},"id":1}]} 2024-03-21T10:32:34.171 INFO:teuthology.orchestra.run.smithi047.stderr:dumped fsmap epoch 32 2024-03-21T10:32:34.173 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:34.173+0000 7f6feda00640 1 -- 172.21.15.47:0/2765549997 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f6fcc0a3090 msgr2=0x7f6fcc0a5550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:32:34.173 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:34.173+0000 7f6feda00640 1 --2- 172.21.15.47:0/2765549997 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f6fcc0a3090 0x7f6fcc0a5550 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f6fd8000f80 tx=0x7f6fd8005c30 comp rx=0 tx=0).stop 2024-03-21T10:32:34.173 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:34.173+0000 7f6feda00640 1 -- 172.21.15.47:0/2765549997 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6fe8107640 msgr2=0x7f6fe819c730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:32:34.173 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:34.173+0000 7f6feda00640 1 --2- 172.21.15.47:0/2765549997 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6fe8107640 0x7f6fe819c730 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f6fe000e970 tx=0x7f6fe000ee30 comp rx=0 tx=0).stop 2024-03-21T10:32:34.173 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:34.173+0000 7f6feda00640 1 -- 172.21.15.47:0/2765549997 shutdown_connections 2024-03-21T10:32:34.173 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:34.173+0000 7f6feda00640 1 --2- 172.21.15.47:0/2765549997 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f6fcc0a3090 0x7f6fcc0a5550 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:32:34.173 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:34.173+0000 7f6feda00640 1 --2- 172.21.15.47:0/2765549997 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6fe8107640 0x7f6fe819c730 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:32:34.173 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:34.173+0000 7f6feda00640 1 --2- 172.21.15.47:0/2765549997 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6fe8101ff0 0x7f6fe81983b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:32:34.173 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:34.173+0000 7f6feda00640 1 --2- 172.21.15.47:0/2765549997 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6fe8101630 0x7f6fe8197e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:32:34.173 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:34.173+0000 7f6feda00640 1 -- 172.21.15.47:0/2765549997 >> 172.21.15.47:0/2765549997 conn(0x7f6fe80fd140 msgr2=0x7f6fe80fda70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:32:34.174 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:34.173+0000 7f6feda00640 1 -- 172.21.15.47:0/2765549997 shutdown_connections 2024-03-21T10:32:34.174 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:34.173+0000 7f6feda00640 1 -- 172.21.15.47:0/2765549997 wait complete. 2024-03-21T10:32:34.192 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-21T10:32:34.192 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-03-21T10:32:34.420 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:34.421+0000 7fa2e7200640 1 -- 172.21.15.47:0/2995876286 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa2e810de90 msgr2=0x7fa2e8110280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:32:34.420 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:34.421+0000 7fa2e7200640 1 --2- 172.21.15.47:0/2995876286 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa2e810de90 0x7fa2e8110280 secure :-1 s=READY pgs=1051 cs=0 l=1 rev1=1 crypto rx=0x7fa2e0009900 tx=0x7fa2e002bc40 comp rx=0 tx=0).stop 2024-03-21T10:32:34.420 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:34.421+0000 7fa2e7200640 1 -- 172.21.15.47:0/2995876286 shutdown_connections 2024-03-21T10:32:34.420 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:34.421+0000 7fa2e7200640 1 --2- 172.21.15.47:0/2995876286 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa2e810de90 0x7fa2e8110280 unknown :-1 s=CLOSED pgs=1051 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:32:34.420 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:34.421+0000 7fa2e7200640 1 --2- 172.21.15.47:0/2995876286 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa2e8100c90 0x7fa2e810d8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:32:34.421 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:34.421+0000 7fa2e7200640 1 --2- 172.21.15.47:0/2995876286 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa2e81002e0 0x7fa2e81006c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:32:34.421 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:34.421+0000 7fa2e7200640 1 -- 172.21.15.47:0/2995876286 >> 172.21.15.47:0/2995876286 conn(0x7fa2e80fbef0 msgr2=0x7fa2e80fe310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:32:34.421 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:34.421+0000 7fa2e7200640 1 -- 172.21.15.47:0/2995876286 shutdown_connections 2024-03-21T10:32:34.421 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:34.421+0000 7fa2e7200640 1 -- 172.21.15.47:0/2995876286 wait complete. 2024-03-21T10:32:34.423 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:34.421+0000 7fa2e7200640 1 Processor -- start 2024-03-21T10:32:34.423 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:34.421+0000 7fa2e7200640 1 -- start start 2024-03-21T10:32:34.423 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:34.422+0000 7fa2e7200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa2e81002e0 0x7fa2e8199e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:32:34.423 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:34.422+0000 7fa2e7200640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa2e8100c90 0x7fa2e819a380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:32:34.423 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:34.422+0000 7fa2e7200640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa2e810de90 0x7fa2e819e700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:32:34.423 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:34.422+0000 7fa2e7200640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2e8113a00 con 0x7fa2e810de90 2024-03-21T10:32:34.423 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:34.422+0000 7fa2e7200640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2e8113880 con 0x7fa2e81002e0 2024-03-21T10:32:34.423 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:34.422+0000 7fa2e7200640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2e8113b80 con 0x7fa2e8100c90 2024-03-21T10:32:34.424 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:34.422+0000 7fa2e6800640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa2e810de90 0x7fa2e819e700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:32:34.424 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:34.422+0000 7fa2e6800640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa2e810de90 0x7fa2e819e700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:51156/0 (socket says 172.21.15.47:51156) 2024-03-21T10:32:34.424 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:34.422+0000 7fa2e5e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa2e81002e0 0x7fa2e8199e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:32:34.424 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:34.422+0000 7fa2e6800640 1 -- 172.21.15.47:0/156065256 learned_addr learned my addr 172.21.15.47:0/156065256 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:32:34.424 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:34.422+0000 7fa2dd400640 1 --2- 172.21.15.47:0/156065256 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa2e8100c90 0x7fa2e819a380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:32:34.424 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:34.422+0000 7fa2e6800640 1 -- 172.21.15.47:0/156065256 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa2e8100c90 msgr2=0x7fa2e819a380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:32:34.424 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:34.422+0000 7fa2e6800640 1 --2- 172.21.15.47:0/156065256 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa2e8100c90 0x7fa2e819a380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:32:34.424 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:34.423+0000 7fa2e6800640 1 -- 172.21.15.47:0/156065256 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa2e81002e0 msgr2=0x7fa2e8199e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:32:34.424 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:34.423+0000 7fa2e6800640 1 --2- 172.21.15.47:0/156065256 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa2e81002e0 0x7fa2e8199e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:32:34.424 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:34.423+0000 7fa2e6800640 1 -- 172.21.15.47:0/156065256 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2e819ee30 con 0x7fa2e810de90 2024-03-21T10:32:34.424 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:34.423+0000 7fa2e5e00640 1 --2- 172.21.15.47:0/156065256 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa2e81002e0 0x7fa2e8199e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T10:32:34.424 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:34.423+0000 7fa2e6800640 1 --2- 172.21.15.47:0/156065256 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa2e810de90 0x7fa2e819e700 secure :-1 s=READY pgs=1052 cs=0 l=1 rev1=1 crypto rx=0x7fa2e002c180 tx=0x7fa2e00045e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:32:34.424 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:34.423+0000 7fa2df400640 1 -- 172.21.15.47:0/156065256 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2e0043020 con 0x7fa2e810de90 2024-03-21T10:32:34.424 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:34.423+0000 7fa2df400640 1 -- 172.21.15.47:0/156065256 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fa2e002cbc0 con 0x7fa2e810de90 2024-03-21T10:32:34.424 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:34.423+0000 7fa2e7200640 1 -- 172.21.15.47:0/156065256 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa2e819f0c0 con 0x7fa2e810de90 2024-03-21T10:32:34.425 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:34.423+0000 7fa2df400640 1 -- 172.21.15.47:0/156065256 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2e0003a80 con 0x7fa2e810de90 2024-03-21T10:32:34.425 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:34.424+0000 7fa2e7200640 1 -- 172.21.15.47:0/156065256 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa2e81a69b0 con 0x7fa2e810de90 2024-03-21T10:32:34.428 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:34.428+0000 7fa2df400640 1 -- 172.21.15.47:0/156065256 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 27) v1 ==== 137985+0+0 (secure 0 0 0) 0x7fa2e0046020 con 0x7fa2e810de90 2024-03-21T10:32:34.428 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:34.428+0000 7fa2ef833740 1 Processor -- start 2024-03-21T10:32:34.428 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:34.428+0000 7fa2ef833740 1 -- start start 2024-03-21T10:32:34.428 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:34.428+0000 7fa2ef833740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5646dc87c8b0 0x5646dc87cc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:32:34.428 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:34.428+0000 7fa2ef833740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5646dc87d260 0x5646dc885810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:32:34.428 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:34.428+0000 7fa2ef833740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5646dc885de0 0x5646dc888230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:32:34.428 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:34.428+0000 7fa2ef833740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x5646dc7e02c0 con 0x5646dc87d260 2024-03-21T10:32:34.428 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:34.428+0000 7fa2ef833740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x5646dc788c90 con 0x5646dc885de0 2024-03-21T10:32:34.428 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:34.428+0000 7fa2ef833740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x5646dc7dc360 con 0x5646dc87c8b0 2024-03-21T10:32:34.429 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:34.428+0000 7fa2e5e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5646dc87d260 0x5646dc885810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:32:34.429 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:34.428+0000 7fa2e5e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5646dc87d260 0x5646dc885810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:51160/0 (socket says 172.21.15.47:51160) 2024-03-21T10:32:34.429 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:34.428+0000 7fa2e5e00640 1 -- 172.21.15.47:0/4127601940 learned_addr learned my addr 172.21.15.47:0/4127601940 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:32:34.429 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:34.428+0000 7fa2dd400640 1 --2- 172.21.15.47:0/4127601940 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5646dc885de0 0x5646dc888230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:32:34.429 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:34.428+0000 7fa2dd400640 1 --2- 172.21.15.47:0/4127601940 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5646dc87c8b0 0x5646dc87cc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:32:34.429 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:34.428+0000 7fa2e5e00640 1 -- 172.21.15.47:0/4127601940 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5646dc87c8b0 msgr2=0x5646dc87cc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:32:34.429 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:34.428+0000 7fa2e5e00640 1 --2- 172.21.15.47:0/4127601940 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5646dc87c8b0 0x5646dc87cc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:32:34.429 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:34.428+0000 7fa2e5e00640 1 -- 172.21.15.47:0/4127601940 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5646dc885de0 msgr2=0x5646dc888230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:32:34.429 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:34.428+0000 7fa2e5e00640 1 --2- 172.21.15.47:0/4127601940 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5646dc885de0 0x5646dc888230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:32:34.430 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:34.428+0000 7fa2e5e00640 1 -- 172.21.15.47:0/4127601940 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5646dc7ad080 con 0x5646dc87d260 2024-03-21T10:32:34.430 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:34.428+0000 7fa2dd400640 1 --2- 172.21.15.47:0/4127601940 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5646dc87c8b0 0x5646dc87cc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T10:32:34.430 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:34.429+0000 7fa2e5e00640 1 --2- 172.21.15.47:0/4127601940 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5646dc87d260 0x5646dc885810 secure :-1 s=READY pgs=1053 cs=0 l=1 rev1=1 crypto rx=0x7fa2d80115f0 tx=0x7fa2d8011ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ad5f292e4f9771f8 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:32:34.434 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:34.429+0000 7fa2d7000640 1 -- 172.21.15.47:0/4127601940 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2d801e070 con 0x5646dc87d260 2024-03-21T10:32:34.434 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:34.429+0000 7fa2d7000640 1 -- 172.21.15.47:0/4127601940 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fa2d800faf0 con 0x5646dc87d260 2024-03-21T10:32:34.434 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:34.429+0000 7fa2d7000640 1 -- 172.21.15.47:0/4127601940 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2d8019b60 con 0x5646dc87d260 2024-03-21T10:32:34.434 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:34.429+0000 7fa2df400640 1 --2- 172.21.15.47:0/156065256 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7fa2cc0a3110 0x7fa2cc0a55d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:32:34.434 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:34.430+0000 7fa2ef833740 1 -- 172.21.15.47:0/4127601940 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5646dc87d260 msgr2=0x5646dc885810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:32:34.434 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:34.430+0000 7fa2ef833740 1 --2- 172.21.15.47:0/4127601940 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5646dc87d260 0x5646dc885810 secure :-1 s=READY pgs=1053 cs=0 l=1 rev1=1 crypto rx=0x7fa2d80115f0 tx=0x7fa2d8011ab0 comp rx=0 tx=0).stop 2024-03-21T10:32:34.434 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:34.430+0000 7fa2df400640 1 -- 172.21.15.47:0/156065256 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 9562+0+0 (secure 0 0 0) 0x7fa2e00f1c90 con 0x7fa2e810de90 2024-03-21T10:32:34.434 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:34.430+0000 7fa2dd400640 1 --2- 172.21.15.47:0/156065256 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7fa2cc0a3110 0x7fa2cc0a55d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:32:34.434 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:34.430+0000 7fa2ef833740 1 -- 172.21.15.47:0/4127601940 shutdown_connections 2024-03-21T10:32:34.434 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:34.430+0000 7fa2ef833740 1 --2- 172.21.15.47:0/4127601940 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5646dc885de0 0x5646dc888230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:32:34.434 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:34.430+0000 7fa2ef833740 1 --2- 172.21.15.47:0/4127601940 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5646dc87d260 0x5646dc885810 unknown :-1 s=CLOSED pgs=1053 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:32:34.434 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:34.430+0000 7fa2ef833740 1 --2- 172.21.15.47:0/4127601940 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5646dc87c8b0 0x5646dc87cc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:32:34.435 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:34.430+0000 7fa2ef833740 1 -- 172.21.15.47:0/4127601940 >> 172.21.15.47:0/4127601940 conn(0x5646dc876f80 msgr2=0x5646dc877360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:32:34.435 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:34.430+0000 7fa2ef833740 1 -- 172.21.15.47:0/4127601940 shutdown_connections 2024-03-21T10:32:34.435 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:34.430+0000 7fa2ef833740 1 -- 172.21.15.47:0/4127601940 wait complete. 2024-03-21T10:32:34.435 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:34.431+0000 7fa2dd400640 1 --2- 172.21.15.47:0/156065256 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7fa2cc0a3110 0x7fa2cc0a55d0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fa2e80ff490 tx=0x7fa2d0004a40 comp rx=0 tx=0).ready entity=mgr.15687 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:32:34.435 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:34.431+0000 7fa2ef833740 1 -- start start 2024-03-21T10:32:34.435 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:34.431+0000 7fa2ef833740 1 Processor -- start 2024-03-21T10:32:34.435 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:34.431+0000 7fa2ef833740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5646dc87c8b0 0x5646dc885740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:32:34.435 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:34.431+0000 7fa2ef833740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5646dc87d260 0x5646dc882ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:32:34.435 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:34.431+0000 7fa2ef833740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5646dc885de0 0x5646dc8830e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:32:34.435 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:34.431+0000 7fa2ef833740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x5646dc888d50 con 0x5646dc885de0 2024-03-21T10:32:34.435 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:34.431+0000 7fa2ef833740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x5646dc79c0d0 con 0x5646dc87d260 2024-03-21T10:32:34.435 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:34.431+0000 7fa2ef833740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x5646dc888ed0 con 0x5646dc87c8b0 2024-03-21T10:32:34.435 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:34.432+0000 7fa2ef833740 10 client.0 _ll_register_callbacks cb 0x5646dc7a0e80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T10:32:34.435 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:34.432+0000 7fa2e6800640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5646dc885de0 0x5646dc8830e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:32:34.436 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:34.432+0000 7fa2dd400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5646dc87d260 0x5646dc882ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:32:34.436 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:34.432+0000 7fa2e6800640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5646dc885de0 0x5646dc8830e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:51176/0 (socket says 172.21.15.47:51176) 2024-03-21T10:32:34.436 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:34.432+0000 7fa2e6800640 1 -- 172.21.15.47:0/928710176 learned_addr learned my addr 172.21.15.47:0/928710176 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:32:34.436 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:34.432+0000 7fa2dd400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5646dc87d260 0x5646dc882ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:57350/0 (socket says 172.21.15.47:57350) 2024-03-21T10:32:34.436 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:34.432+0000 7fa2e5e00640 1 --2- 172.21.15.47:0/928710176 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5646dc87c8b0 0x5646dc885740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:32:34.436 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:34.432+0000 7fa2e6800640 1 -- 172.21.15.47:0/928710176 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5646dc87c8b0 msgr2=0x5646dc885740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:32:34.436 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:34.432+0000 7fa2e6800640 1 --2- 172.21.15.47:0/928710176 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5646dc87c8b0 0x5646dc885740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:32:34.436 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:34.432+0000 7fa2e6800640 1 -- 172.21.15.47:0/928710176 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5646dc87d260 msgr2=0x5646dc882ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:32:34.436 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:34.432+0000 7fa2e6800640 1 --2- 172.21.15.47:0/928710176 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5646dc87d260 0x5646dc882ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:32:34.436 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:34.432+0000 7fa2e6800640 1 -- 172.21.15.47:0/928710176 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5646dc4f43f0 con 0x5646dc885de0 2024-03-21T10:32:34.436 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:34.432+0000 7fa2e5e00640 1 --2- 172.21.15.47:0/928710176 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5646dc87c8b0 0x5646dc885740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T10:32:34.436 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:34.432+0000 7fa2e6800640 1 --2- 172.21.15.47:0/928710176 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5646dc885de0 0x5646dc8830e0 secure :-1 s=READY pgs=1054 cs=0 l=1 rev1=1 crypto rx=0x7fa2e00c6200 tx=0x7fa2e00bd740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ed306956d5e9bd95 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:32:34.436 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:34.432+0000 7fa2cb400640 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-03-21T10:32:34.436 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:34.433+0000 7fa2cb400640 1 -- 172.21.15.47:0/928710176 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2e00c1e70 con 0x5646dc885de0 2024-03-21T10:32:34.437 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:34.433+0000 7fa2cb400640 1 -- 172.21.15.47:0/928710176 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(99..99 src has 1..99) v4 ==== 9562+0+0 (secure 0 0 0) 0x7fa2e00bdf80 con 0x5646dc885de0 2024-03-21T10:32:34.437 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:34.433+0000 7fa2ef833740 1 -- 172.21.15.47:0/928710176 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fa2d800faf0 con 0x5646dc885de0 2024-03-21T10:32:34.437 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:34.433+0000 7fa2cb400640 1 -- 172.21.15.47:0/928710176 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa2e00be200 con 0x5646dc885de0 2024-03-21T10:32:34.437 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:34.433+0000 7fa2ef833740 10 client.17877 fetch_fsmap learned FSMap version 32 2024-03-21T10:32:34.437 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:34.433+0000 7fa2ef833740 1 -- 172.21.15.47:0/928710176 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x5646dc5be7d0 con 0x5646dc885de0 2024-03-21T10:32:34.437 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:34.433+0000 7fa2cb400640 1 -- 172.21.15.47:0/928710176 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fa2d800faf0 con 0x5646dc885de0 2024-03-21T10:32:34.437 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:34.433+0000 7fa2cb400640 1 -- 172.21.15.47:0/928710176 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 32) v1 ==== 7942+0+0 (secure 0 0 0) 0x7fa2e00b9a70 con 0x5646dc885de0 2024-03-21T10:32:34.437 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:34.434+0000 7fa2ef833740 10 client.17877 fetch_fsmap finished waiting for FSMap version 32 2024-03-21T10:32:34.437 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:34.434+0000 7fa2ef833740 10 client.17877 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.a 2024-03-21T10:32:34.437 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:34.434+0000 7fa2ef833740 20 client.17877 populate_metadata read hostname 'smithi047' 2024-03-21T10:32:34.437 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:34.434+0000 7fa2ef833740 1 --2- 172.21.15.47:0/928710176 >> [v2:172.21.15.47:6840/3401578978,v1:172.21.15.47:6841/3401578978] conn(0x5646dc895c30 0x5646dc8980b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:32:34.437 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:34.434+0000 7fa2ef833740 4 client.17877 mds_command: new command op to 14619 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T10:32:34.437 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:34.434+0000 7fa2ef833740 1 -- 172.21.15.47:0/928710176 --> [v2:172.21.15.47:6840/3401578978,v1:172.21.15.47:6841/3401578978] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5646dc7ed800 con 0x5646dc895c30 2024-03-21T10:32:34.437 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:34.434+0000 7fa2e5e00640 1 --2- 172.21.15.47:0/928710176 >> [v2:172.21.15.47:6840/3401578978,v1:172.21.15.47:6841/3401578978] conn(0x5646dc895c30 0x5646dc8980b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:32:34.437 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:34.434+0000 7fa2e5e00640 1 --2- 172.21.15.47:0/928710176 >> [v2:172.21.15.47:6840/3401578978,v1:172.21.15.47:6841/3401578978] conn(0x5646dc895c30 0x5646dc8980b0 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=39ce6a02d5eaec76 server_cookie=ac1bd8747fd2ece8 in_seq=0 out_seq=0 2024-03-21T10:32:34.438 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:34.434+0000 7fa2cb400640 10 client.17877 ms_handle_connect on v2:172.21.15.47:6840/3401578978 2024-03-21T10:32:34.439 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:34.435+0000 7fa2cb400640 1 -- 172.21.15.47:0/928710176 <== mds.4 v2:172.21.15.47:6840/3401578978 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x5646dc7ed800 con 0x5646dc895c30 2024-03-21T10:32:34.439 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:34.435+0000 7fa2cb400640 10 client.17877 handle_command_reply: tid=0 2024-03-21T10:32:34.439 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:34.435+0000 7fa2ef833740 1 client.17877 shutdown 2024-03-21T10:32:34.439 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:34.435+0000 7fa2ef833740 1 -- 172.21.15.47:0/928710176 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5646dc885de0 msgr2=0x5646dc8830e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:32:34.439 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:34.435+0000 7fa2ef833740 1 --2- 172.21.15.47:0/928710176 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5646dc885de0 0x5646dc8830e0 secure :-1 s=READY pgs=1054 cs=0 l=1 rev1=1 crypto rx=0x7fa2e00c6200 tx=0x7fa2e00bd740 comp rx=0 tx=0).stop 2024-03-21T10:32:34.439 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:34.436+0000 7fa2ef833740 1 -- 172.21.15.47:0/928710176 shutdown_connections 2024-03-21T10:32:34.439 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:34.436+0000 7fa2ef833740 1 --2- 172.21.15.47:0/928710176 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5646dc885de0 0x5646dc8830e0 unknown :-1 s=CLOSED pgs=1054 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:32:34.439 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:34.436+0000 7fa2ef833740 1 --2- 172.21.15.47:0/928710176 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5646dc87d260 0x5646dc882ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:32:34.439 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:34.436+0000 7fa2ef833740 1 --2- 172.21.15.47:0/928710176 >> [v2:172.21.15.47:6840/3401578978,v1:172.21.15.47:6841/3401578978] conn(0x5646dc895c30 0x5646dc8980b0 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:32:34.439 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:34.436+0000 7fa2ef833740 1 --2- 172.21.15.47:0/928710176 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5646dc87c8b0 0x5646dc885740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:32:34.440 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:34.436+0000 7fa2ef833740 1 -- 172.21.15.47:0/928710176 >> 172.21.15.47:0/928710176 conn(0x5646dc876f80 msgr2=0x5646dc878cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:32:34.440 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:34.436+0000 7fa2cb400640 0 client.17877 ms_handle_reset on v2:172.21.15.47:6840/3401578978 2024-03-21T10:32:34.440 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:34.436+0000 7fa2ef833740 1 -- 172.21.15.47:0/928710176 shutdown_connections 2024-03-21T10:32:34.440 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:34.436+0000 7fa2ef833740 1 -- 172.21.15.47:0/928710176 wait complete. 2024-03-21T10:32:34.440 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:34.436+0000 7fa2ef833740 20 client.17877 trim_cache size 0 max 16384 2024-03-21T10:32:34.454 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:34.453+0000 7fa2ef833740 1 Processor -- start 2024-03-21T10:32:34.454 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:34.453+0000 7fa2ef833740 1 -- start start 2024-03-21T10:32:34.454 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:34.453+0000 7fa2ef833740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5646dc87c8b0 0x5646dc885200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:32:34.454 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:34.454+0000 7fa2ef833740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5646dc87d260 0x5646dc885740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:32:34.454 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:34.454+0000 7fa2ef833740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5646dc885de0 0x5646dc87fd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:32:34.454 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:34.454+0000 7fa2ef833740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2e81b08d0 con 0x5646dc87d260 2024-03-21T10:32:34.454 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:34.454+0000 7fa2ef833740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2e81b2250 con 0x5646dc885de0 2024-03-21T10:32:34.454 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:34.454+0000 7fa2ef833740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2e81b0f30 con 0x5646dc87c8b0 2024-03-21T10:32:34.454 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:34.454+0000 7fa2dd400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5646dc87d260 0x5646dc885740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:32:34.454 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:34.454+0000 7fa2dd400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5646dc87d260 0x5646dc885740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:51178/0 (socket says 172.21.15.47:51178) 2024-03-21T10:32:34.454 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:34.454+0000 7fa2dd400640 1 -- 172.21.15.47:0/4261164628 learned_addr learned my addr 172.21.15.47:0/4261164628 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:32:34.455 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:34.454+0000 7fa2e6800640 1 --2- 172.21.15.47:0/4261164628 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5646dc885de0 0x5646dc87fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:32:34.455 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:34.454+0000 7fa2e5e00640 1 --2- 172.21.15.47:0/4261164628 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5646dc87c8b0 0x5646dc885200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:32:34.455 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:34.454+0000 7fa2dd400640 1 -- 172.21.15.47:0/4261164628 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5646dc87c8b0 msgr2=0x5646dc885200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:32:34.455 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:34.454+0000 7fa2dd400640 1 --2- 172.21.15.47:0/4261164628 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5646dc87c8b0 0x5646dc885200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:32:34.455 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:34.454+0000 7fa2dd400640 1 -- 172.21.15.47:0/4261164628 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5646dc885de0 msgr2=0x5646dc87fd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:32:34.455 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:34.454+0000 7fa2dd400640 1 --2- 172.21.15.47:0/4261164628 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5646dc885de0 0x5646dc87fd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:32:34.455 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:34.454+0000 7fa2dd400640 1 -- 172.21.15.47:0/4261164628 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5646dc871570 con 0x5646dc87d260 2024-03-21T10:32:34.457 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:34.455+0000 7fa2dd400640 1 --2- 172.21.15.47:0/4261164628 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5646dc87d260 0x5646dc885740 secure :-1 s=READY pgs=1055 cs=0 l=1 rev1=1 crypto rx=0x7fa2d0010750 tx=0x7fa2d000e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fee0528d771a1cfa server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:32:34.457 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:34.455+0000 7fa2d6600640 1 -- 172.21.15.47:0/4261164628 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2d002a070 con 0x5646dc87d260 2024-03-21T10:32:34.457 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:34.455+0000 7fa2d6600640 1 -- 172.21.15.47:0/4261164628 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fa2d001a460 con 0x5646dc87d260 2024-03-21T10:32:34.457 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:34.455+0000 7fa2d6600640 1 -- 172.21.15.47:0/4261164628 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2d0025680 con 0x5646dc87d260 2024-03-21T10:32:34.457 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:34.456+0000 7fa2ef833740 1 -- 172.21.15.47:0/4261164628 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5646dc87d260 msgr2=0x5646dc885740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:32:34.458 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:34.456+0000 7fa2ef833740 1 --2- 172.21.15.47:0/4261164628 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5646dc87d260 0x5646dc885740 secure :-1 s=READY pgs=1055 cs=0 l=1 rev1=1 crypto rx=0x7fa2d0010750 tx=0x7fa2d000e000 comp rx=0 tx=0).stop 2024-03-21T10:32:34.458 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:34.456+0000 7fa2ef833740 1 -- 172.21.15.47:0/4261164628 shutdown_connections 2024-03-21T10:32:34.458 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:34.456+0000 7fa2ef833740 1 --2- 172.21.15.47:0/4261164628 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5646dc885de0 0x5646dc87fd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:32:34.458 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:34.456+0000 7fa2ef833740 1 --2- 172.21.15.47:0/4261164628 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5646dc87d260 0x5646dc885740 unknown :-1 s=CLOSED pgs=1055 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:32:34.458 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:34.456+0000 7fa2ef833740 1 --2- 172.21.15.47:0/4261164628 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5646dc87c8b0 0x5646dc885200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:32:34.458 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:34.456+0000 7fa2ef833740 1 -- 172.21.15.47:0/4261164628 >> 172.21.15.47:0/4261164628 conn(0x5646dc876f80 msgr2=0x5646dc876160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:32:34.458 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:34.456+0000 7fa2ef833740 1 -- 172.21.15.47:0/4261164628 shutdown_connections 2024-03-21T10:32:34.458 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:34.456+0000 7fa2ef833740 1 -- 172.21.15.47:0/4261164628 wait complete. 2024-03-21T10:32:34.458 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:34.456+0000 7fa2ef833740 1 -- start start 2024-03-21T10:32:34.458 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:34.456+0000 7fa2ef833740 1 Processor -- start 2024-03-21T10:32:34.458 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:34.457+0000 7fa2ef833740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5646dc87c8b0 0x5646dc88b760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:32:34.458 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:34.457+0000 7fa2ef833740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5646dc87d260 0x5646dc88bca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:32:34.458 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:34.457+0000 7fa2ef833740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5646dc885de0 0x5646dc88c1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:32:34.458 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:34.457+0000 7fa2ef833740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2e81b1bf0 con 0x5646dc885de0 2024-03-21T10:32:34.459 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:34.457+0000 7fa2ef833740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2e81b1590 con 0x5646dc87d260 2024-03-21T10:32:34.459 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:34.457+0000 7fa2ef833740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2e81aeab0 con 0x5646dc87c8b0 2024-03-21T10:32:34.459 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:34.457+0000 7fa2ef833740 10 client.0 _ll_register_callbacks cb 0x7fa2e00bdb60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T10:32:34.459 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:34.457+0000 7fa2e6800640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5646dc885de0 0x5646dc88c1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:32:34.459 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:34.457+0000 7fa2e6800640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5646dc885de0 0x5646dc88c1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:51180/0 (socket says 172.21.15.47:51180) 2024-03-21T10:32:34.459 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:34.457+0000 7fa2e6800640 1 -- 172.21.15.47:0/619856299 learned_addr learned my addr 172.21.15.47:0/619856299 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:32:34.459 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:34.457+0000 7fa2dd400640 1 --2- 172.21.15.47:0/619856299 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5646dc87d260 0x5646dc88bca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:32:34.459 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:34.458+0000 7fa2e5e00640 1 --2- 172.21.15.47:0/619856299 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5646dc87c8b0 0x5646dc88b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:32:34.459 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:34.458+0000 7fa2e6800640 1 -- 172.21.15.47:0/619856299 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5646dc87c8b0 msgr2=0x5646dc88b760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:32:34.459 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:34.458+0000 7fa2e6800640 1 --2- 172.21.15.47:0/619856299 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5646dc87c8b0 0x5646dc88b760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:32:34.459 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:34.458+0000 7fa2e6800640 1 -- 172.21.15.47:0/619856299 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5646dc87d260 msgr2=0x5646dc88bca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:32:34.459 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:34.458+0000 7fa2e6800640 1 --2- 172.21.15.47:0/619856299 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5646dc87d260 0x5646dc88bca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:32:34.459 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:34.458+0000 7fa2e6800640 1 -- 172.21.15.47:0/619856299 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5646dc8f07e0 con 0x5646dc885de0 2024-03-21T10:32:34.459 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:34.458+0000 7fa2dd400640 1 --2- 172.21.15.47:0/619856299 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5646dc87d260 0x5646dc88bca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T10:32:34.460 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:34.458+0000 7fa2e6800640 1 --2- 172.21.15.47:0/619856299 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5646dc885de0 0x5646dc88c1e0 secure :-1 s=READY pgs=1056 cs=0 l=1 rev1=1 crypto rx=0x7fa2e00c2030 tx=0x7fa2e002cd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3d803c3165df7d31 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:32:34.460 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:34.458+0000 7fa2cb400640 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-03-21T10:32:34.462 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:34.458+0000 7fa2cb400640 1 -- 172.21.15.47:0/619856299 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2e00a0070 con 0x5646dc885de0 2024-03-21T10:32:34.462 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:34.458+0000 7fa2cb400640 1 -- 172.21.15.47:0/619856299 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(99..99 src has 1..99) v4 ==== 9562+0+0 (secure 0 0 0) 0x7fa2e00be180 con 0x5646dc885de0 2024-03-21T10:32:34.462 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:34.458+0000 7fa2ef833740 1 -- 172.21.15.47:0/619856299 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fa2d001a460 con 0x5646dc885de0 2024-03-21T10:32:34.462 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:34.458+0000 7fa2cb400640 1 -- 172.21.15.47:0/619856299 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa2e00c6730 con 0x5646dc885de0 2024-03-21T10:32:34.462 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:34.459+0000 7fa2ef833740 10 client.17883 fetch_fsmap learned FSMap version 32 2024-03-21T10:32:34.462 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:34.459+0000 7fa2ef833740 1 -- 172.21.15.47:0/619856299 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x5646dc8f07e0 con 0x5646dc885de0 2024-03-21T10:32:34.462 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:34.459+0000 7fa2cb400640 1 -- 172.21.15.47:0/619856299 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fa2d001a460 con 0x5646dc885de0 2024-03-21T10:32:34.462 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:34.459+0000 7fa2cb400640 1 -- 172.21.15.47:0/619856299 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 32) v1 ==== 7942+0+0 (secure 0 0 0) 0x7fa2e00a3030 con 0x5646dc885de0 2024-03-21T10:32:34.462 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:34.459+0000 7fa2ef833740 10 client.17883 fetch_fsmap finished waiting for FSMap version 32 2024-03-21T10:32:34.462 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:34.459+0000 7fa2ef833740 10 client.17883 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.a 2024-03-21T10:32:34.463 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:34.459+0000 7fa2ef833740 20 client.17883 populate_metadata read hostname 'smithi047' 2024-03-21T10:32:34.463 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:34.459+0000 7fa2ef833740 1 --2- 172.21.15.47:0/619856299 >> [v2:172.21.15.47:6840/3401578978,v1:172.21.15.47:6841/3401578978] conn(0x5646dc8f1d90 0x5646dc8f4180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:32:34.463 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:34.459+0000 7fa2ef833740 4 client.17883 mds_command: new command op to 14619 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-21T10:32:34.463 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:34.459+0000 7fa2ef833740 1 -- 172.21.15.47:0/619856299 --> [v2:172.21.15.47:6840/3401578978,v1:172.21.15.47:6841/3401578978] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x5646dc7e7b50 con 0x5646dc8f1d90 2024-03-21T10:32:34.463 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:34.459+0000 7fa2e5e00640 1 --2- 172.21.15.47:0/619856299 >> [v2:172.21.15.47:6840/3401578978,v1:172.21.15.47:6841/3401578978] conn(0x5646dc8f1d90 0x5646dc8f4180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:32:34.463 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:34.460+0000 7fa2e5e00640 1 --2- 172.21.15.47:0/619856299 >> [v2:172.21.15.47:6840/3401578978,v1:172.21.15.47:6841/3401578978] conn(0x5646dc8f1d90 0x5646dc8f4180 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=63bc500f31687fe2 server_cookie=c93aceb242c8345a in_seq=0 out_seq=0 2024-03-21T10:32:34.463 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:34.460+0000 7fa2cb400640 10 client.17883 ms_handle_connect on v2:172.21.15.47:6840/3401578978 2024-03-21T10:32:34.463 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:34.460+0000 7fa2cb400640 1 -- 172.21.15.47:0/619856299 <== mds.4 v2:172.21.15.47:6840/3401578978 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x5646dc7e7b50 con 0x5646dc8f1d90 2024-03-21T10:32:34.463 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:34.460+0000 7fa2cb400640 10 client.17883 handle_command_reply: tid=0 2024-03-21T10:32:34.463 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:34.460+0000 7fa2ef833740 1 client.17883 shutdown 2024-03-21T10:32:34.463 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:34.461+0000 7fa2ef833740 1 -- 172.21.15.47:0/619856299 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5646dc885de0 msgr2=0x5646dc88c1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:32:34.463 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:34.461+0000 7fa2ef833740 1 --2- 172.21.15.47:0/619856299 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5646dc885de0 0x5646dc88c1e0 secure :-1 s=READY pgs=1056 cs=0 l=1 rev1=1 crypto rx=0x7fa2e00c2030 tx=0x7fa2e002cd70 comp rx=0 tx=0).stop 2024-03-21T10:32:34.463 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:34.461+0000 7fa2ef833740 1 -- 172.21.15.47:0/619856299 shutdown_connections 2024-03-21T10:32:34.463 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:34.461+0000 7fa2ef833740 1 --2- 172.21.15.47:0/619856299 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5646dc885de0 0x5646dc88c1e0 unknown :-1 s=CLOSED pgs=1056 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:32:34.463 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:34.461+0000 7fa2ef833740 1 --2- 172.21.15.47:0/619856299 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5646dc87d260 0x5646dc88bca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:32:34.464 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:34.461+0000 7fa2ef833740 1 --2- 172.21.15.47:0/619856299 >> [v2:172.21.15.47:6840/3401578978,v1:172.21.15.47:6841/3401578978] conn(0x5646dc8f1d90 0x5646dc8f4180 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:32:34.464 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:34.461+0000 7fa2ef833740 1 --2- 172.21.15.47:0/619856299 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5646dc87c8b0 0x5646dc88b760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:32:34.464 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:34.461+0000 7fa2ef833740 1 -- 172.21.15.47:0/619856299 >> 172.21.15.47:0/619856299 conn(0x5646dc876f80 msgr2=0x5646dc876160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:32:34.464 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:34.461+0000 7fa2cb400640 0 client.17883 ms_handle_reset on v2:172.21.15.47:6840/3401578978 2024-03-21T10:32:34.464 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:34.461+0000 7fa2ef833740 1 -- 172.21.15.47:0/619856299 shutdown_connections 2024-03-21T10:32:34.464 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:34.461+0000 7fa2ef833740 1 -- 172.21.15.47:0/619856299 wait complete. 2024-03-21T10:32:34.464 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:34.461+0000 7fa2ef833740 20 client.17883 trim_cache size 0 max 16384 2024-03-21T10:32:34.464 INFO:teuthology.orchestra.run.smithi047.stdout:[] 2024-03-21T10:32:34.465 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:32:34 smithi047 ceph-mon[31547]: scrub queued for path: / 2024-03-21T10:32:34.465 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:32:34 smithi047 ceph-mon[31547]: scrub summary: idle+waiting paths [/] 2024-03-21T10:32:34.465 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:32:34 smithi047 ceph-mon[31547]: scrub summary: active paths [/] 2024-03-21T10:32:34.465 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:32:34 smithi047 ceph-mon[31547]: scrub summary: idle+waiting paths [/] 2024-03-21T10:32:34.465 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:32:34 smithi047 ceph-mon[31547]: scrub summary: idle 2024-03-21T10:32:34.465 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:32:34 smithi047 ceph-mon[31547]: from='client.? 172.21.15.47:0/2765549997' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-21T10:32:34.466 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:34.462+0000 7fa2e7200640 1 -- 172.21.15.47:0/156065256 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7fa2cc0a3110 msgr2=0x7fa2cc0a55d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:32:34.466 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:34.462+0000 7fa2e7200640 1 --2- 172.21.15.47:0/156065256 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7fa2cc0a3110 0x7fa2cc0a55d0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fa2e80ff490 tx=0x7fa2d0004a40 comp rx=0 tx=0).stop 2024-03-21T10:32:34.466 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:34.462+0000 7fa2e7200640 1 -- 172.21.15.47:0/156065256 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa2e810de90 msgr2=0x7fa2e819e700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:32:34.466 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:34.462+0000 7fa2e7200640 1 --2- 172.21.15.47:0/156065256 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa2e810de90 0x7fa2e819e700 secure :-1 s=READY pgs=1052 cs=0 l=1 rev1=1 crypto rx=0x7fa2e002c180 tx=0x7fa2e00045e0 comp rx=0 tx=0).stop 2024-03-21T10:32:34.466 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:34.462+0000 7fa2e7200640 1 -- 172.21.15.47:0/156065256 shutdown_connections 2024-03-21T10:32:34.466 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:34.462+0000 7fa2e7200640 1 --2- 172.21.15.47:0/156065256 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa2e810de90 0x7fa2e819e700 unknown :-1 s=CLOSED pgs=1052 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:32:34.466 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:34.462+0000 7fa2e7200640 1 --2- 172.21.15.47:0/156065256 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa2e8100c90 0x7fa2e819a380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:32:34.466 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:34.462+0000 7fa2e7200640 1 --2- 172.21.15.47:0/156065256 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7fa2cc0a3110 0x7fa2cc0a55d0 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:32:34.466 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:34.462+0000 7fa2e7200640 1 --2- 172.21.15.47:0/156065256 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa2e81002e0 0x7fa2e8199e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:32:34.466 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:34.462+0000 7fa2e7200640 1 -- 172.21.15.47:0/156065256 >> 172.21.15.47:0/156065256 conn(0x7fa2e80fbef0 msgr2=0x7fa2e810f160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:32:34.466 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:34.462+0000 7fa2e7200640 1 -- 172.21.15.47:0/156065256 shutdown_connections 2024-03-21T10:32:34.466 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:34.462+0000 7fa2e7200640 1 -- 172.21.15.47:0/156065256 wait complete. 2024-03-21T10:32:34.480 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-21T10:32:34.480 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-03-21T10:32:34.692 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:34.692+0000 7fb471400640 1 -- 172.21.15.47:0/1818304472 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb46c101ff0 msgr2=0x7fb46c10e500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:32:34.692 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:34.692+0000 7fb471400640 1 --2- 172.21.15.47:0/1818304472 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb46c101ff0 0x7fb46c10e500 secure :-1 s=READY pgs=1057 cs=0 l=1 rev1=1 crypto rx=0x7fb464009c10 tx=0x7fb46402bef0 comp rx=0 tx=0).stop 2024-03-21T10:32:34.692 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:34.692+0000 7fb471400640 1 -- 172.21.15.47:0/1818304472 shutdown_connections 2024-03-21T10:32:34.692 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:34.692+0000 7fb471400640 1 --2- 172.21.15.47:0/1818304472 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb46c101ff0 0x7fb46c10e500 unknown :-1 s=CLOSED pgs=1057 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:32:34.692 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:34.692+0000 7fb471400640 1 --2- 172.21.15.47:0/1818304472 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb46c101630 0x7fb46c101ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:32:34.692 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:34.692+0000 7fb471400640 1 --2- 172.21.15.47:0/1818304472 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb46c107640 0x7fb46c107a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:32:34.692 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:34.692+0000 7fb471400640 1 -- 172.21.15.47:0/1818304472 >> 172.21.15.47:0/1818304472 conn(0x7fb46c078aa0 msgr2=0x7fb46c078eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:32:34.692 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:34.692+0000 7fb471400640 1 -- 172.21.15.47:0/1818304472 shutdown_connections 2024-03-21T10:32:34.692 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:34.692+0000 7fb471400640 1 -- 172.21.15.47:0/1818304472 wait complete. 2024-03-21T10:32:34.694 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:34.693+0000 7fb471400640 1 Processor -- start 2024-03-21T10:32:34.694 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:34.693+0000 7fb471400640 1 -- start start 2024-03-21T10:32:34.694 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:34.694+0000 7fb471400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb46c101630 0x7fb46c0739a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:32:34.694 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:34.694+0000 7fb471400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb46c101ff0 0x7fb46c073ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:32:34.694 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:34.694+0000 7fb471400640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb46c107640 0x7fb46c074420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:32:34.694 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:34.694+0000 7fb471400640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb46c111900 con 0x7fb46c101ff0 2024-03-21T10:32:34.694 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:34.694+0000 7fb471400640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb46c111780 con 0x7fb46c101630 2024-03-21T10:32:34.694 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:34.694+0000 7fb471400640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb46c111a80 con 0x7fb46c107640 2024-03-21T10:32:34.694 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:34.694+0000 7fb463e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb46c101ff0 0x7fb46c073ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:32:34.695 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:34.694+0000 7fb46a800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb46c101630 0x7fb46c0739a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:32:34.695 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:34.694+0000 7fb463e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb46c101ff0 0x7fb46c073ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:51188/0 (socket says 172.21.15.47:51188) 2024-03-21T10:32:34.695 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:34.694+0000 7fb463e00640 1 -- 172.21.15.47:0/4086702470 learned_addr learned my addr 172.21.15.47:0/4086702470 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:32:34.695 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:34.694+0000 7fb46b200640 1 --2- 172.21.15.47:0/4086702470 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb46c107640 0x7fb46c074420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:32:34.695 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:34.694+0000 7fb463e00640 1 -- 172.21.15.47:0/4086702470 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb46c107640 msgr2=0x7fb46c074420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:32:34.695 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:34.694+0000 7fb463e00640 1 --2- 172.21.15.47:0/4086702470 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb46c107640 0x7fb46c074420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:32:34.695 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:34.694+0000 7fb463e00640 1 -- 172.21.15.47:0/4086702470 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb46c101630 msgr2=0x7fb46c0739a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:32:34.695 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:34.694+0000 7fb463e00640 1 --2- 172.21.15.47:0/4086702470 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb46c101630 0x7fb46c0739a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:32:34.695 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:34.694+0000 7fb463e00640 1 -- 172.21.15.47:0/4086702470 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb46c1a6300 con 0x7fb46c101ff0 2024-03-21T10:32:34.696 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:34.695+0000 7fb463e00640 1 --2- 172.21.15.47:0/4086702470 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb46c101ff0 0x7fb46c073ee0 secure :-1 s=READY pgs=1058 cs=0 l=1 rev1=1 crypto rx=0x7fb45400e9c0 tx=0x7fb45400ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:32:34.696 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:34.695+0000 7fb463400640 1 -- 172.21.15.47:0/4086702470 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb45400cde0 con 0x7fb46c101ff0 2024-03-21T10:32:34.696 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:34.695+0000 7fb471400640 1 -- 172.21.15.47:0/4086702470 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb46c1a65f0 con 0x7fb46c101ff0 2024-03-21T10:32:34.696 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:34.695+0000 7fb463400640 1 -- 172.21.15.47:0/4086702470 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fb4540035f0 con 0x7fb46c101ff0 2024-03-21T10:32:34.696 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:34.695+0000 7fb463400640 1 -- 172.21.15.47:0/4086702470 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb454010690 con 0x7fb46c101ff0 2024-03-21T10:32:34.696 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:34.695+0000 7fb471400640 1 -- 172.21.15.47:0/4086702470 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb46c1a6b90 con 0x7fb46c101ff0 2024-03-21T10:32:34.698 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:34.700+0000 7fb463400640 1 -- 172.21.15.47:0/4086702470 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 27) v1 ==== 137985+0+0 (secure 0 0 0) 0x7fb4540037a0 con 0x7fb46c101ff0 2024-03-21T10:32:34.700 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:34.701+0000 7fb463400640 1 --2- 172.21.15.47:0/4086702470 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7fb4500a3090 0x7fb4500a5550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:32:34.700 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:34.701+0000 7fb474188740 1 Processor -- start 2024-03-21T10:32:34.700 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:34.701+0000 7fb463400640 1 -- 172.21.15.47:0/4086702470 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 9562+0+0 (secure 0 0 0) 0x7fb4540cfff0 con 0x7fb46c101ff0 2024-03-21T10:32:34.700 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:34.701+0000 7fb46a800640 1 --2- 172.21.15.47:0/4086702470 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7fb4500a3090 0x7fb4500a5550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:32:34.700 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:34.701+0000 7fb474188740 1 -- start start 2024-03-21T10:32:34.703 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:34.701+0000 7fb474188740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560e4b17e6d0 0x560e4b17eab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:32:34.703 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:34.701+0000 7fb474188740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x560e4b17f080 0x560e4b187630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:32:34.703 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:34.701+0000 7fb474188740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560e4b187c00 0x560e4b18a050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:32:34.703 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:34.701+0000 7fb474188740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x560e4b0e22c0 con 0x560e4b17f080 2024-03-21T10:32:34.703 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:34.701+0000 7fb474188740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x560e4b08ac90 con 0x560e4b17e6d0 2024-03-21T10:32:34.703 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:34.701+0000 7fb474188740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x560e4b0de360 con 0x560e4b187c00 2024-03-21T10:32:34.703 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:34.701+0000 7fb463e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x560e4b17f080 0x560e4b187630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:32:34.703 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:34.701+0000 7fb46a800640 1 --2- 172.21.15.47:0/4086702470 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7fb4500a3090 0x7fb4500a5550 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fb45c00bb40 tx=0x7fb45c00b390 comp rx=0 tx=0).ready entity=mgr.15687 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:32:34.703 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:34.701+0000 7fb463e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x560e4b17f080 0x560e4b187630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:51202/0 (socket says 172.21.15.47:51202) 2024-03-21T10:32:34.703 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:34.701+0000 7fb463e00640 1 -- 172.21.15.47:0/912537958 learned_addr learned my addr 172.21.15.47:0/912537958 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:32:34.703 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:34.701+0000 7fb46a800640 1 --2- 172.21.15.47:0/912537958 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560e4b17e6d0 0x560e4b17eab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:32:34.703 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:34.702+0000 7fb46b200640 1 --2- 172.21.15.47:0/912537958 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560e4b187c00 0x560e4b18a050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:32:34.703 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:34.702+0000 7fb463e00640 1 -- 172.21.15.47:0/912537958 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560e4b187c00 msgr2=0x560e4b18a050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:32:34.704 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:34.702+0000 7fb463e00640 1 --2- 172.21.15.47:0/912537958 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560e4b187c00 0x560e4b18a050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:32:34.704 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:34.702+0000 7fb463e00640 1 -- 172.21.15.47:0/912537958 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560e4b17e6d0 msgr2=0x560e4b17eab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:32:34.704 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:34.702+0000 7fb463e00640 1 --2- 172.21.15.47:0/912537958 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560e4b17e6d0 0x560e4b17eab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:32:34.704 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:34.702+0000 7fb463e00640 1 -- 172.21.15.47:0/912537958 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560e4b0af080 con 0x560e4b17f080 2024-03-21T10:32:34.704 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:34.702+0000 7fb46a800640 1 --2- 172.21.15.47:0/912537958 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560e4b17e6d0 0x560e4b17eab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T10:32:34.704 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:34.702+0000 7fb46b200640 1 --2- 172.21.15.47:0/912537958 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560e4b187c00 0x560e4b18a050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T10:32:34.704 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:34.702+0000 7fb463e00640 1 --2- 172.21.15.47:0/912537958 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x560e4b17f080 0x560e4b187630 secure :-1 s=READY pgs=1059 cs=0 l=1 rev1=1 crypto rx=0x7fb4540a4470 tx=0x7fb454003b10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=332383f174c97010 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:32:34.704 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:34.702+0000 7fb45b400640 1 -- 172.21.15.47:0/912537958 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb45409c690 con 0x560e4b17f080 2024-03-21T10:32:34.704 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:34.702+0000 7fb45b400640 1 -- 172.21.15.47:0/912537958 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fb4540b5070 con 0x560e4b17f080 2024-03-21T10:32:34.704 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:34.702+0000 7fb45b400640 1 -- 172.21.15.47:0/912537958 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb454098600 con 0x560e4b17f080 2024-03-21T10:32:34.704 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:34.703+0000 7fb474188740 1 -- 172.21.15.47:0/912537958 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x560e4b17f080 msgr2=0x560e4b187630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:32:34.704 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:34.703+0000 7fb474188740 1 --2- 172.21.15.47:0/912537958 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x560e4b17f080 0x560e4b187630 secure :-1 s=READY pgs=1059 cs=0 l=1 rev1=1 crypto rx=0x7fb4540a4470 tx=0x7fb454003b10 comp rx=0 tx=0).stop 2024-03-21T10:32:34.704 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:34.703+0000 7fb474188740 1 -- 172.21.15.47:0/912537958 shutdown_connections 2024-03-21T10:32:34.704 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:34.703+0000 7fb474188740 1 --2- 172.21.15.47:0/912537958 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560e4b187c00 0x560e4b18a050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:32:34.704 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:34.703+0000 7fb474188740 1 --2- 172.21.15.47:0/912537958 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x560e4b17f080 0x560e4b187630 unknown :-1 s=CLOSED pgs=1059 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:32:34.705 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:34.703+0000 7fb474188740 1 --2- 172.21.15.47:0/912537958 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560e4b17e6d0 0x560e4b17eab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:32:34.705 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:34.703+0000 7fb474188740 1 -- 172.21.15.47:0/912537958 >> 172.21.15.47:0/912537958 conn(0x560e4b178da0 msgr2=0x560e4b179180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:32:34.705 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:34.703+0000 7fb474188740 1 -- 172.21.15.47:0/912537958 shutdown_connections 2024-03-21T10:32:34.705 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:34.703+0000 7fb474188740 1 -- 172.21.15.47:0/912537958 wait complete. 2024-03-21T10:32:34.707 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:34.704+0000 7fb474188740 1 -- start start 2024-03-21T10:32:34.708 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:34.704+0000 7fb474188740 1 Processor -- start 2024-03-21T10:32:34.708 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:34.704+0000 7fb474188740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x560e4b17e6d0 0x560e4b187520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:32:34.708 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:34.704+0000 7fb474188740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560e4b17f080 0x560e4b1848b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:32:34.708 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:34.704+0000 7fb474188740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560e4b187c00 0x560e4b184e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:32:34.708 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:34.704+0000 7fb474188740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x560e4b18ab10 con 0x560e4b17e6d0 2024-03-21T10:32:34.708 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:34.704+0000 7fb474188740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x560e4b09e0d0 con 0x560e4b17f080 2024-03-21T10:32:34.708 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:34.704+0000 7fb474188740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x560e4b18ac90 con 0x560e4b187c00 2024-03-21T10:32:34.708 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:34.704+0000 7fb474188740 10 client.0 _ll_register_callbacks cb 0x560e4b0a2e80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T10:32:34.708 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:34.705+0000 7fb46a800640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x560e4b17e6d0 0x560e4b187520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:32:34.708 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:34.705+0000 7fb46a800640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x560e4b17e6d0 0x560e4b187520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:51216/0 (socket says 172.21.15.47:51216) 2024-03-21T10:32:34.708 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:34.705+0000 7fb46a800640 1 -- 172.21.15.47:0/122326036 learned_addr learned my addr 172.21.15.47:0/122326036 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:32:34.708 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:34.705+0000 7fb463e00640 1 --2- 172.21.15.47:0/122326036 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560e4b17f080 0x560e4b1848b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:32:34.708 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:34.705+0000 7fb46b200640 1 --2- 172.21.15.47:0/122326036 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560e4b187c00 0x560e4b184e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:32:34.709 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:34.705+0000 7fb46a800640 1 -- 172.21.15.47:0/122326036 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560e4b187c00 msgr2=0x560e4b184e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:32:34.709 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:34.705+0000 7fb46a800640 1 --2- 172.21.15.47:0/122326036 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560e4b187c00 0x560e4b184e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:32:34.709 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:34.705+0000 7fb46a800640 1 -- 172.21.15.47:0/122326036 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560e4b17f080 msgr2=0x560e4b1848b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:32:34.709 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:34.705+0000 7fb46a800640 1 --2- 172.21.15.47:0/122326036 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560e4b17f080 0x560e4b1848b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:32:34.709 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:34.705+0000 7fb46a800640 1 -- 172.21.15.47:0/122326036 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x560e4adf63f0 con 0x560e4b17e6d0 2024-03-21T10:32:34.709 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:34.706+0000 7fb46a800640 1 --2- 172.21.15.47:0/122326036 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x560e4b17e6d0 0x560e4b187520 secure :-1 s=READY pgs=1060 cs=0 l=1 rev1=1 crypto rx=0x7fb45c011770 tx=0x7fb45c004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fe82cc7b0075e3c1 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:32:34.709 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:34.706+0000 7fb44fe00640 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-03-21T10:32:34.709 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:34.706+0000 7fb44fe00640 1 -- 172.21.15.47:0/122326036 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb45c01d900 con 0x560e4b17e6d0 2024-03-21T10:32:34.709 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:34.706+0000 7fb44fe00640 1 -- 172.21.15.47:0/122326036 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(99..99 src has 1..99) v4 ==== 9562+0+0 (secure 0 0 0) 0x7fb45c01daa0 con 0x560e4b17e6d0 2024-03-21T10:32:34.709 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:34.706+0000 7fb474188740 1 -- 172.21.15.47:0/122326036 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fb4540b5070 con 0x560e4b17e6d0 2024-03-21T10:32:34.709 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:34.706+0000 7fb44fe00640 1 -- 172.21.15.47:0/122326036 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb45c031050 con 0x560e4b17e6d0 2024-03-21T10:32:34.709 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:34.706+0000 7fb474188740 10 client.17895 fetch_fsmap learned FSMap version 32 2024-03-21T10:32:34.709 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:34.706+0000 7fb474188740 1 -- 172.21.15.47:0/122326036 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x560e4aec07d0 con 0x560e4b17e6d0 2024-03-21T10:32:34.709 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:34.707+0000 7fb44fe00640 1 -- 172.21.15.47:0/122326036 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fb4540b5070 con 0x560e4b17e6d0 2024-03-21T10:32:34.710 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:34.707+0000 7fb44fe00640 1 -- 172.21.15.47:0/122326036 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 32) v1 ==== 7942+0+0 (secure 0 0 0) 0x7fb45c0263f0 con 0x560e4b17e6d0 2024-03-21T10:32:34.710 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:34.707+0000 7fb474188740 10 client.17895 fetch_fsmap finished waiting for FSMap version 32 2024-03-21T10:32:34.710 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:34.707+0000 7fb474188740 10 client.17895 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.k 2024-03-21T10:32:34.710 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:34.707+0000 7fb474188740 20 client.17895 populate_metadata read hostname 'smithi047' 2024-03-21T10:32:34.710 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:34.707+0000 7fb474188740 1 --2- 172.21.15.47:0/122326036 >> [v2:172.21.15.100:6832/2723302038,v1:172.21.15.100:6833/2723302038] conn(0x560e4b197a20 0x560e4b199e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:32:34.710 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:34.707+0000 7fb474188740 4 client.17895 mds_command: new command op to 34427 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T10:32:34.710 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:34.707+0000 7fb474188740 1 -- 172.21.15.47:0/122326036 --> [v2:172.21.15.100:6832/2723302038,v1:172.21.15.100:6833/2723302038] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x560e4b0ef800 con 0x560e4b197a20 2024-03-21T10:32:34.710 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:34.707+0000 7fb463e00640 1 --2- 172.21.15.47:0/122326036 >> [v2:172.21.15.100:6832/2723302038,v1:172.21.15.100:6833/2723302038] conn(0x560e4b197a20 0x560e4b199e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:32:34.710 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:34.708+0000 7fb463e00640 1 --2- 172.21.15.47:0/122326036 >> [v2:172.21.15.100:6832/2723302038,v1:172.21.15.100:6833/2723302038] conn(0x560e4b197a20 0x560e4b199e30 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=9afd185ed0a412fc server_cookie=cbb2323a11399432 in_seq=0 out_seq=0 2024-03-21T10:32:34.710 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:34.708+0000 7fb44fe00640 10 client.17895 ms_handle_connect on v2:172.21.15.100:6832/2723302038 2024-03-21T10:32:34.711 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:34.711+0000 7fb44fe00640 1 -- 172.21.15.47:0/122326036 <== mds.3 v2:172.21.15.100:6832/2723302038 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x560e4b0ef800 con 0x560e4b197a20 2024-03-21T10:32:34.711 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:34.711+0000 7fb44fe00640 10 client.17895 handle_command_reply: tid=0 2024-03-21T10:32:34.711 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:34.711+0000 7fb474188740 1 client.17895 shutdown 2024-03-21T10:32:34.712 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:34.711+0000 7fb474188740 1 -- 172.21.15.47:0/122326036 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x560e4b17e6d0 msgr2=0x560e4b187520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:32:34.712 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:34.711+0000 7fb474188740 1 --2- 172.21.15.47:0/122326036 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x560e4b17e6d0 0x560e4b187520 secure :-1 s=READY pgs=1060 cs=0 l=1 rev1=1 crypto rx=0x7fb45c011770 tx=0x7fb45c004730 comp rx=0 tx=0).stop 2024-03-21T10:32:34.712 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:34.711+0000 7fb474188740 1 -- 172.21.15.47:0/122326036 shutdown_connections 2024-03-21T10:32:34.712 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:34.711+0000 7fb474188740 1 --2- 172.21.15.47:0/122326036 >> [v2:172.21.15.100:6832/2723302038,v1:172.21.15.100:6833/2723302038] conn(0x560e4b197a20 0x560e4b199e30 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:32:34.712 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:34.711+0000 7fb474188740 1 --2- 172.21.15.47:0/122326036 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560e4b187c00 0x560e4b184e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:32:34.712 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:34.711+0000 7fb474188740 1 --2- 172.21.15.47:0/122326036 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560e4b17f080 0x560e4b1848b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:32:34.712 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:34.711+0000 7fb474188740 1 --2- 172.21.15.47:0/122326036 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x560e4b17e6d0 0x560e4b187520 unknown :-1 s=CLOSED pgs=1060 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:32:34.712 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:34.711+0000 7fb44fe00640 0 client.17895 ms_handle_reset on v2:172.21.15.100:6832/2723302038 2024-03-21T10:32:34.712 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:34.711+0000 7fb474188740 1 -- 172.21.15.47:0/122326036 >> 172.21.15.47:0/122326036 conn(0x560e4b178da0 msgr2=0x560e4b17aaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:32:34.712 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:34.711+0000 7fb474188740 1 -- 172.21.15.47:0/122326036 shutdown_connections 2024-03-21T10:32:34.712 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:34.711+0000 7fb474188740 1 -- 172.21.15.47:0/122326036 wait complete. 2024-03-21T10:32:34.712 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:34.711+0000 7fb474188740 20 client.17895 trim_cache size 0 max 16384 2024-03-21T10:32:34.725 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:34.724+0000 7fb474188740 1 Processor -- start 2024-03-21T10:32:34.725 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:34.724+0000 7fb474188740 1 -- start start 2024-03-21T10:32:34.725 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:34.724+0000 7fb474188740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x560e4b17e6d0 0x560e4b17fe60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:32:34.725 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:34.725+0000 7fb474188740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560e4b17f080 0x560e4b17a8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:32:34.725 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:34.725+0000 7fb474188740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560e4b187c00 0x560e4b186fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:32:34.725 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:34.725+0000 7fb474188740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb46c1b0cb0 con 0x560e4b17e6d0 2024-03-21T10:32:34.725 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:34.725+0000 7fb474188740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb46c1b2630 con 0x560e4b17f080 2024-03-21T10:32:34.725 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:34.725+0000 7fb474188740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb46c1b1310 con 0x560e4b187c00 2024-03-21T10:32:34.725 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:34.725+0000 7fb46a800640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x560e4b17e6d0 0x560e4b17fe60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:32:34.725 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:34.725+0000 7fb46a800640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x560e4b17e6d0 0x560e4b17fe60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:51218/0 (socket says 172.21.15.47:51218) 2024-03-21T10:32:34.726 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:34.725+0000 7fb46a800640 1 -- 172.21.15.47:0/46712171 learned_addr learned my addr 172.21.15.47:0/46712171 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:32:34.726 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:34.725+0000 7fb463e00640 1 --2- 172.21.15.47:0/46712171 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560e4b17f080 0x560e4b17a8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:32:34.726 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:34.725+0000 7fb46b200640 1 --2- 172.21.15.47:0/46712171 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560e4b187c00 0x560e4b186fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:32:34.726 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:34.725+0000 7fb46a800640 1 -- 172.21.15.47:0/46712171 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560e4b187c00 msgr2=0x560e4b186fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:32:34.726 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:34.725+0000 7fb46a800640 1 --2- 172.21.15.47:0/46712171 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560e4b187c00 0x560e4b186fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:32:34.726 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:34.725+0000 7fb46a800640 1 -- 172.21.15.47:0/46712171 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560e4b17f080 msgr2=0x560e4b17a8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:32:34.726 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:34.725+0000 7fb46a800640 1 --2- 172.21.15.47:0/46712171 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560e4b17f080 0x560e4b17a8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:32:34.726 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:34.725+0000 7fb46a800640 1 -- 172.21.15.47:0/46712171 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560e4b182e00 con 0x560e4b17e6d0 2024-03-21T10:32:34.727 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:32:34 smithi100 ceph-mon[35193]: scrub queued for path: / 2024-03-21T10:32:34.727 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:32:34 smithi100 ceph-mon[35193]: scrub summary: idle+waiting paths [/] 2024-03-21T10:32:34.727 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:32:34 smithi100 ceph-mon[35193]: scrub summary: active paths [/] 2024-03-21T10:32:34.727 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:32:34 smithi100 ceph-mon[35193]: scrub summary: idle+waiting paths [/] 2024-03-21T10:32:34.727 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:32:34 smithi100 ceph-mon[35193]: scrub summary: idle 2024-03-21T10:32:34.727 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:32:34 smithi100 ceph-mon[35193]: from='client.? 172.21.15.47:0/2765549997' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-21T10:32:34.729 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:34.726+0000 7fb46a800640 1 --2- 172.21.15.47:0/46712171 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x560e4b17e6d0 0x560e4b17fe60 secure :-1 s=READY pgs=1061 cs=0 l=1 rev1=1 crypto rx=0x7fb45c000c00 tx=0x7fb45c005e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9126f460e454731 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:32:34.730 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:34.726+0000 7fb45aa00640 1 -- 172.21.15.47:0/46712171 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb45c026a00 con 0x560e4b17e6d0 2024-03-21T10:32:34.730 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:34.726+0000 7fb45aa00640 1 -- 172.21.15.47:0/46712171 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fb45c039030 con 0x560e4b17e6d0 2024-03-21T10:32:34.730 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:34.726+0000 7fb45aa00640 1 -- 172.21.15.47:0/46712171 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb45c01ddd0 con 0x560e4b17e6d0 2024-03-21T10:32:34.730 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:34.727+0000 7fb474188740 1 -- 172.21.15.47:0/46712171 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x560e4b17e6d0 msgr2=0x560e4b17fe60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:32:34.730 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:34.727+0000 7fb474188740 1 --2- 172.21.15.47:0/46712171 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x560e4b17e6d0 0x560e4b17fe60 secure :-1 s=READY pgs=1061 cs=0 l=1 rev1=1 crypto rx=0x7fb45c000c00 tx=0x7fb45c005e50 comp rx=0 tx=0).stop 2024-03-21T10:32:34.730 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:34.727+0000 7fb474188740 1 -- 172.21.15.47:0/46712171 shutdown_connections 2024-03-21T10:32:34.730 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:34.727+0000 7fb474188740 1 --2- 172.21.15.47:0/46712171 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560e4b187c00 0x560e4b186fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:32:34.730 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:34.727+0000 7fb474188740 1 --2- 172.21.15.47:0/46712171 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560e4b17f080 0x560e4b17a8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:32:34.730 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:34.727+0000 7fb474188740 1 --2- 172.21.15.47:0/46712171 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x560e4b17e6d0 0x560e4b17fe60 unknown :-1 s=CLOSED pgs=1061 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:32:34.730 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:34.727+0000 7fb474188740 1 -- 172.21.15.47:0/46712171 >> 172.21.15.47:0/46712171 conn(0x560e4b178da0 msgr2=0x560e4b17a1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:32:34.730 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:34.727+0000 7fb474188740 1 -- 172.21.15.47:0/46712171 shutdown_connections 2024-03-21T10:32:34.730 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:34.727+0000 7fb474188740 1 -- 172.21.15.47:0/46712171 wait complete. 2024-03-21T10:32:34.730 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:34.728+0000 7fb474188740 1 -- start start 2024-03-21T10:32:34.731 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:34.728+0000 7fb474188740 1 Processor -- start 2024-03-21T10:32:34.731 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:34.728+0000 7fb474188740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560e4b17e6d0 0x560e4b18fdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:32:34.731 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:34.728+0000 7fb474188740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x560e4b17f080 0x560e4b18e150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:32:34.731 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:34.728+0000 7fb474188740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560e4b187c00 0x560e4b18e7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:32:34.731 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:34.728+0000 7fb474188740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb46c1b1fd0 con 0x560e4b17f080 2024-03-21T10:32:34.731 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:34.728+0000 7fb474188740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb46c1b1970 con 0x560e4b17e6d0 2024-03-21T10:32:34.731 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:34.728+0000 7fb474188740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb46c1aee90 con 0x560e4b187c00 2024-03-21T10:32:34.731 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:34.728+0000 7fb474188740 10 client.0 _ll_register_callbacks cb 0x7fb45c010e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T10:32:34.731 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:34.729+0000 7fb463e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x560e4b17f080 0x560e4b18e150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:32:34.731 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:34.729+0000 7fb463e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x560e4b17f080 0x560e4b18e150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:51228/0 (socket says 172.21.15.47:51228) 2024-03-21T10:32:34.731 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:34.729+0000 7fb463e00640 1 -- 172.21.15.47:0/3141406224 learned_addr learned my addr 172.21.15.47:0/3141406224 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:32:34.731 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:34.729+0000 7fb46a800640 1 --2- 172.21.15.47:0/3141406224 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560e4b17e6d0 0x560e4b18fdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:32:34.731 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:34.729+0000 7fb463e00640 1 -- 172.21.15.47:0/3141406224 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560e4b187c00 msgr2=0x560e4b18e7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:32:34.731 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:34.729+0000 7fb463e00640 1 --2- 172.21.15.47:0/3141406224 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560e4b187c00 0x560e4b18e7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:32:34.732 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:34.729+0000 7fb463e00640 1 -- 172.21.15.47:0/3141406224 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560e4b17e6d0 msgr2=0x560e4b18fdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:32:34.732 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:34.729+0000 7fb463e00640 1 --2- 172.21.15.47:0/3141406224 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560e4b17e6d0 0x560e4b18fdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:32:34.732 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:34.729+0000 7fb463e00640 1 -- 172.21.15.47:0/3141406224 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x560e4b1f32e0 con 0x560e4b17f080 2024-03-21T10:32:34.732 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:34.729+0000 7fb463e00640 1 --2- 172.21.15.47:0/3141406224 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x560e4b17f080 0x560e4b18e150 secure :-1 s=READY pgs=1062 cs=0 l=1 rev1=1 crypto rx=0x7fb45409c150 tx=0x7fb454003e10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3b2836152d4d62b2 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:32:34.732 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:34.729+0000 7fb44fe00640 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-03-21T10:32:34.732 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:34.730+0000 7fb44fe00640 1 -- 172.21.15.47:0/3141406224 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4540cd570 con 0x560e4b17f080 2024-03-21T10:32:34.732 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:34.730+0000 7fb44fe00640 1 -- 172.21.15.47:0/3141406224 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(99..99 src has 1..99) v4 ==== 9562+0+0 (secure 0 0 0) 0x7fb4540cd710 con 0x560e4b17f080 2024-03-21T10:32:34.732 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:34.730+0000 7fb474188740 1 -- 172.21.15.47:0/3141406224 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fb45c039030 con 0x560e4b17f080 2024-03-21T10:32:34.732 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:34.730+0000 7fb44fe00640 1 -- 172.21.15.47:0/3141406224 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb45409b3e0 con 0x560e4b17f080 2024-03-21T10:32:34.732 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:34.730+0000 7fb474188740 10 client.17901 fetch_fsmap learned FSMap version 32 2024-03-21T10:32:34.732 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:34.730+0000 7fb474188740 1 -- 172.21.15.47:0/3141406224 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x560e4b1f44b0 con 0x560e4b17f080 2024-03-21T10:32:34.732 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:34.730+0000 7fb44fe00640 1 -- 172.21.15.47:0/3141406224 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fb45c039030 con 0x560e4b17f080 2024-03-21T10:32:34.734 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:34.730+0000 7fb44fe00640 1 -- 172.21.15.47:0/3141406224 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 32) v1 ==== 7942+0+0 (secure 0 0 0) 0x7fb45409b580 con 0x560e4b17f080 2024-03-21T10:32:34.734 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:34.731+0000 7fb474188740 10 client.17901 fetch_fsmap finished waiting for FSMap version 32 2024-03-21T10:32:34.734 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:34.731+0000 7fb474188740 10 client.17901 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.k 2024-03-21T10:32:34.734 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:34.731+0000 7fb474188740 20 client.17901 populate_metadata read hostname 'smithi047' 2024-03-21T10:32:34.734 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:34.731+0000 7fb474188740 1 --2- 172.21.15.47:0/3141406224 >> [v2:172.21.15.100:6832/2723302038,v1:172.21.15.100:6833/2723302038] conn(0x560e4b1f47a0 0x560e4b1f6b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:32:34.734 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:34.731+0000 7fb474188740 4 client.17901 mds_command: new command op to 34427 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-21T10:32:34.734 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:34.731+0000 7fb474188740 1 -- 172.21.15.47:0/3141406224 --> [v2:172.21.15.100:6832/2723302038,v1:172.21.15.100:6833/2723302038] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x560e4b0e9b50 con 0x560e4b1f47a0 2024-03-21T10:32:34.734 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:34.731+0000 7fb46a800640 1 --2- 172.21.15.47:0/3141406224 >> [v2:172.21.15.100:6832/2723302038,v1:172.21.15.100:6833/2723302038] conn(0x560e4b1f47a0 0x560e4b1f6b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:32:34.734 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:34.732+0000 7fb46a800640 1 --2- 172.21.15.47:0/3141406224 >> [v2:172.21.15.100:6832/2723302038,v1:172.21.15.100:6833/2723302038] conn(0x560e4b1f47a0 0x560e4b1f6b90 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=5ccc7cc776f78df4 server_cookie=c8c1dce5bc16a941 in_seq=0 out_seq=0 2024-03-21T10:32:34.734 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:34.732+0000 7fb44fe00640 10 client.17901 ms_handle_connect on v2:172.21.15.100:6832/2723302038 2024-03-21T10:32:34.735 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:34.732+0000 7fb44fe00640 1 -- 172.21.15.47:0/3141406224 <== mds.3 v2:172.21.15.100:6832/2723302038 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x560e4b0e9b50 con 0x560e4b1f47a0 2024-03-21T10:32:34.735 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:34.732+0000 7fb44fe00640 10 client.17901 handle_command_reply: tid=0 2024-03-21T10:32:34.735 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:34.732+0000 7fb474188740 1 client.17901 shutdown 2024-03-21T10:32:34.735 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:34.732+0000 7fb474188740 1 -- 172.21.15.47:0/3141406224 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x560e4b17f080 msgr2=0x560e4b18e150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:32:34.735 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:34.732+0000 7fb474188740 1 --2- 172.21.15.47:0/3141406224 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x560e4b17f080 0x560e4b18e150 secure :-1 s=READY pgs=1062 cs=0 l=1 rev1=1 crypto rx=0x7fb45409c150 tx=0x7fb454003e10 comp rx=0 tx=0).stop 2024-03-21T10:32:34.735 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:34.732+0000 7fb474188740 1 -- 172.21.15.47:0/3141406224 shutdown_connections 2024-03-21T10:32:34.735 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:34.732+0000 7fb474188740 1 --2- 172.21.15.47:0/3141406224 >> [v2:172.21.15.100:6832/2723302038,v1:172.21.15.100:6833/2723302038] conn(0x560e4b1f47a0 0x560e4b1f6b90 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:32:34.735 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:34.732+0000 7fb474188740 1 --2- 172.21.15.47:0/3141406224 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x560e4b187c00 0x560e4b18e7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:32:34.735 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:34.732+0000 7fb474188740 1 --2- 172.21.15.47:0/3141406224 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x560e4b17f080 0x560e4b18e150 unknown :-1 s=CLOSED pgs=1062 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:32:34.735 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:34.732+0000 7fb474188740 1 --2- 172.21.15.47:0/3141406224 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560e4b17e6d0 0x560e4b18fdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:32:34.735 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:34.732+0000 7fb44fe00640 0 client.17901 ms_handle_reset on v2:172.21.15.100:6832/2723302038 2024-03-21T10:32:34.735 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:34.732+0000 7fb474188740 1 -- 172.21.15.47:0/3141406224 >> 172.21.15.47:0/3141406224 conn(0x560e4b178da0 msgr2=0x560e4b177620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:32:34.735 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:34.732+0000 7fb474188740 1 -- 172.21.15.47:0/3141406224 shutdown_connections 2024-03-21T10:32:34.735 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:34.733+0000 7fb474188740 1 -- 172.21.15.47:0/3141406224 wait complete. 2024-03-21T10:32:34.735 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:34.733+0000 7fb474188740 20 client.17901 trim_cache size 0 max 16384 2024-03-21T10:32:34.736 INFO:teuthology.orchestra.run.smithi047.stdout:[] 2024-03-21T10:32:34.736 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:34.733+0000 7fb471400640 1 -- 172.21.15.47:0/4086702470 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7fb4500a3090 msgr2=0x7fb4500a5550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:32:34.736 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:34.733+0000 7fb471400640 1 --2- 172.21.15.47:0/4086702470 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7fb4500a3090 0x7fb4500a5550 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fb45c00bb40 tx=0x7fb45c00b390 comp rx=0 tx=0).stop 2024-03-21T10:32:34.737 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:34.733+0000 7fb471400640 1 -- 172.21.15.47:0/4086702470 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb46c101ff0 msgr2=0x7fb46c073ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:32:34.737 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:34.733+0000 7fb471400640 1 --2- 172.21.15.47:0/4086702470 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb46c101ff0 0x7fb46c073ee0 secure :-1 s=READY pgs=1058 cs=0 l=1 rev1=1 crypto rx=0x7fb45400e9c0 tx=0x7fb45400ee80 comp rx=0 tx=0).stop 2024-03-21T10:32:34.737 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:34.733+0000 7fb471400640 1 -- 172.21.15.47:0/4086702470 shutdown_connections 2024-03-21T10:32:34.737 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:34.733+0000 7fb471400640 1 --2- 172.21.15.47:0/4086702470 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb46c107640 0x7fb46c074420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:32:34.737 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:34.733+0000 7fb471400640 1 --2- 172.21.15.47:0/4086702470 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb46c101ff0 0x7fb46c073ee0 unknown :-1 s=CLOSED pgs=1058 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:32:34.737 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:34.733+0000 7fb471400640 1 --2- 172.21.15.47:0/4086702470 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7fb4500a3090 0x7fb4500a5550 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:32:34.737 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:34.733+0000 7fb471400640 1 --2- 172.21.15.47:0/4086702470 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb46c101630 0x7fb46c0739a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:32:34.737 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:34.733+0000 7fb471400640 1 -- 172.21.15.47:0/4086702470 >> 172.21.15.47:0/4086702470 conn(0x7fb46c078aa0 msgr2=0x7fb46c104050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:32:34.737 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:34.734+0000 7fb471400640 1 -- 172.21.15.47:0/4086702470 shutdown_connections 2024-03-21T10:32:34.737 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:34.734+0000 7fb471400640 1 -- 172.21.15.47:0/4086702470 wait complete. 2024-03-21T10:32:34.738 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:32:34 smithi157 ceph-mon[35581]: scrub queued for path: / 2024-03-21T10:32:34.738 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:32:34 smithi157 ceph-mon[35581]: scrub summary: idle+waiting paths [/] 2024-03-21T10:32:34.738 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:32:34 smithi157 ceph-mon[35581]: scrub summary: active paths [/] 2024-03-21T10:32:34.738 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:32:34 smithi157 ceph-mon[35581]: scrub summary: idle+waiting paths [/] 2024-03-21T10:32:34.738 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:32:34 smithi157 ceph-mon[35581]: scrub summary: idle 2024-03-21T10:32:34.738 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:32:34 smithi157 ceph-mon[35581]: from='client.? 172.21.15.47:0/2765549997' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-21T10:32:34.750 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-21T10:32:34.750 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-03-21T10:32:34.961 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:34.962+0000 7f326d200640 1 -- 172.21.15.47:0/806109901 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3268073ad0 msgr2=0x7f3268073eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:32:34.961 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:34.962+0000 7f326d200640 1 --2- 172.21.15.47:0/806109901 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3268073ad0 0x7f3268073eb0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f3258009be0 tx=0x7f325802bf80 comp rx=0 tx=0).stop 2024-03-21T10:32:34.961 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:34.962+0000 7f326d200640 1 -- 172.21.15.47:0/806109901 shutdown_connections 2024-03-21T10:32:34.962 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:34.962+0000 7f326d200640 1 --2- 172.21.15.47:0/806109901 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f326810dd40 0x7f3268110190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:32:34.962 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:34.962+0000 7f326d200640 1 --2- 172.21.15.47:0/806109901 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f32680743f0 0x7f326810d800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:32:34.962 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:34.962+0000 7f326d200640 1 --2- 172.21.15.47:0/806109901 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3268073ad0 0x7f3268073eb0 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:32:34.962 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:34.962+0000 7f326d200640 1 -- 172.21.15.47:0/806109901 >> 172.21.15.47:0/806109901 conn(0x7f32680fd010 msgr2=0x7f32680ff430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:32:34.962 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:34.963+0000 7f326d200640 1 -- 172.21.15.47:0/806109901 shutdown_connections 2024-03-21T10:32:34.962 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:34.963+0000 7f326d200640 1 -- 172.21.15.47:0/806109901 wait complete. 2024-03-21T10:32:34.962 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:34.964+0000 7f326d200640 1 Processor -- start 2024-03-21T10:32:34.963 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:34.964+0000 7f326d200640 1 -- start start 2024-03-21T10:32:34.964 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:34.965+0000 7f326d200640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3268073ad0 0x7f3268101460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:32:34.964 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:34.965+0000 7f326d200640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f32680743f0 0x7f32681019a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:32:34.964 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:34.965+0000 7f326d200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f326810dd40 0x7f3268101ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:32:34.964 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:34.965+0000 7f326d200640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32681139e0 con 0x7f32680743f0 2024-03-21T10:32:34.964 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:34.965+0000 7f326d200640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3268113860 con 0x7f326810dd40 2024-03-21T10:32:34.964 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:34.965+0000 7f326d200640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3268113b60 con 0x7f3268073ad0 2024-03-21T10:32:34.965 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:34.965+0000 7f3266a00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f32680743f0 0x7f32681019a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:32:34.965 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:34.965+0000 7f3266a00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f32680743f0 0x7f32681019a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:51242/0 (socket says 172.21.15.47:51242) 2024-03-21T10:32:34.965 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:34.965+0000 7f3266a00640 1 -- 172.21.15.47:0/435714492 learned_addr learned my addr 172.21.15.47:0/435714492 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:32:34.965 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:34.965+0000 7f3267e00640 1 --2- 172.21.15.47:0/435714492 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f326810dd40 0x7f3268101ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:32:34.965 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:34.965+0000 7f3267400640 1 --2- 172.21.15.47:0/435714492 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3268073ad0 0x7f3268101460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:32:34.965 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:34.966+0000 7f3266a00640 1 -- 172.21.15.47:0/435714492 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3268073ad0 msgr2=0x7f3268101460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:32:34.965 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:34.966+0000 7f3266a00640 1 --2- 172.21.15.47:0/435714492 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3268073ad0 0x7f3268101460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:32:34.965 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:34.966+0000 7f3266a00640 1 -- 172.21.15.47:0/435714492 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f326810dd40 msgr2=0x7f3268101ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:32:34.965 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:34.966+0000 7f3266a00640 1 --2- 172.21.15.47:0/435714492 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f326810dd40 0x7f3268101ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:32:34.965 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:34.966+0000 7f3266a00640 1 -- 172.21.15.47:0/435714492 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32681aa5e0 con 0x7f32680743f0 2024-03-21T10:32:34.965 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:34.966+0000 7f3266a00640 1 --2- 172.21.15.47:0/435714492 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f32680743f0 0x7f32681019a0 secure :-1 s=READY pgs=1063 cs=0 l=1 rev1=1 crypto rx=0x7f325400e9c0 tx=0x7f325400ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:32:34.966 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:34.966+0000 7f325fe00640 1 -- 172.21.15.47:0/435714492 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f325400ccc0 con 0x7f32680743f0 2024-03-21T10:32:34.966 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:34.966+0000 7f325fe00640 1 -- 172.21.15.47:0/435714492 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f32540054d0 con 0x7f32680743f0 2024-03-21T10:32:34.966 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:34.966+0000 7f325fe00640 1 -- 172.21.15.47:0/435714492 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3254010430 con 0x7f32680743f0 2024-03-21T10:32:34.966 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:34.966+0000 7f326d200640 1 -- 172.21.15.47:0/435714492 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f32681aa8d0 con 0x7f32680743f0 2024-03-21T10:32:34.966 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:34.966+0000 7f326d200640 1 -- 172.21.15.47:0/435714492 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f32681aae20 con 0x7f32680743f0 2024-03-21T10:32:34.970 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:34.971+0000 7f3271428740 1 Processor -- start 2024-03-21T10:32:34.970 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:34.971+0000 7f3271428740 1 -- start start 2024-03-21T10:32:34.971 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:34.972+0000 7f3271428740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55da92d2d8b0 0x55da92d2dc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:32:34.971 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:34.972+0000 7f3271428740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55da92d2e260 0x55da92d36810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:32:34.971 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:34.972+0000 7f3271428740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55da92d36de0 0x55da92d39230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:32:34.971 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:34.972+0000 7f3271428740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x55da92c912c0 con 0x55da92d36de0 2024-03-21T10:32:34.971 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:34.972+0000 7f3271428740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55da92c39c90 con 0x55da92d2d8b0 2024-03-21T10:32:34.971 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:34.972+0000 7f3271428740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55da92c8d360 con 0x55da92d2e260 2024-03-21T10:32:34.971 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:34.972+0000 7f325fe00640 1 -- 172.21.15.47:0/435714492 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 27) v1 ==== 137985+0+0 (secure 0 0 0) 0x7f3254004f50 con 0x7f32680743f0 2024-03-21T10:32:34.971 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:34.972+0000 7f3267e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55da92d2d8b0 0x55da92d2dc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:32:34.971 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:34.972+0000 7f3267e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55da92d2d8b0 0x55da92d2dc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:57412/0 (socket says 172.21.15.47:57412) 2024-03-21T10:32:34.971 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:34.972+0000 7f3267e00640 1 -- 172.21.15.47:0/126698855 learned_addr learned my addr 172.21.15.47:0/126698855 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:32:34.971 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:34.972+0000 7f3267400640 1 --2- 172.21.15.47:0/126698855 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55da92d2e260 0x55da92d36810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:32:34.973 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:34.973+0000 7f3266a00640 1 --2- 172.21.15.47:0/126698855 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55da92d36de0 0x55da92d39230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:32:34.974 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:34.973+0000 7f3267e00640 1 -- 172.21.15.47:0/126698855 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55da92d2e260 msgr2=0x55da92d36810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:32:34.974 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:34.973+0000 7f3267e00640 1 --2- 172.21.15.47:0/126698855 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55da92d2e260 0x55da92d36810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:32:34.974 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:34.973+0000 7f3267e00640 1 -- 172.21.15.47:0/126698855 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55da92d36de0 msgr2=0x55da92d39230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:32:34.974 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:34.973+0000 7f3267e00640 1 --2- 172.21.15.47:0/126698855 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55da92d36de0 0x55da92d39230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:32:34.974 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:34.973+0000 7f3267e00640 1 -- 172.21.15.47:0/126698855 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55da92c5e080 con 0x55da92d2d8b0 2024-03-21T10:32:34.974 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:34.974+0000 7f3267e00640 1 --2- 172.21.15.47:0/126698855 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55da92d2d8b0 0x55da92d2dc90 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f326000b6b0 tx=0x7f3260004610 comp rx=0 tx=0).ready entity=mon.2 client_cookie=803941765d0e9d2d server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:32:34.974 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:34.974+0000 7f325cc00640 1 -- 172.21.15.47:0/126698855 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3260003b20 con 0x55da92d2d8b0 2024-03-21T10:32:34.974 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:34.974+0000 7f325fe00640 1 --2- 172.21.15.47:0/435714492 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f324c0a3090 0x7f324c0a5550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:32:34.974 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:34.974+0000 7f325cc00640 1 -- 172.21.15.47:0/126698855 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f3260015070 con 0x55da92d2d8b0 2024-03-21T10:32:34.974 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:34.974+0000 7f325cc00640 1 -- 172.21.15.47:0/126698855 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32600116a0 con 0x55da92d2d8b0 2024-03-21T10:32:34.974 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:34.974+0000 7f325fe00640 1 -- 172.21.15.47:0/435714492 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 9562+0+0 (secure 0 0 0) 0x7f32540cff20 con 0x7f32680743f0 2024-03-21T10:32:34.974 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:34.974+0000 7f3267400640 1 --2- 172.21.15.47:0/435714492 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f324c0a3090 0x7f324c0a5550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:32:34.974 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:34.975+0000 7f3271428740 1 -- 172.21.15.47:0/126698855 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55da92d2d8b0 msgr2=0x55da92d2dc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:32:34.974 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:34.975+0000 7f3271428740 1 --2- 172.21.15.47:0/126698855 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55da92d2d8b0 0x55da92d2dc90 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f326000b6b0 tx=0x7f3260004610 comp rx=0 tx=0).stop 2024-03-21T10:32:34.975 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:34.975+0000 7f3271428740 1 -- 172.21.15.47:0/126698855 shutdown_connections 2024-03-21T10:32:34.975 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:34.975+0000 7f3271428740 1 --2- 172.21.15.47:0/126698855 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55da92d36de0 0x55da92d39230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:32:34.975 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:34.975+0000 7f3271428740 1 --2- 172.21.15.47:0/126698855 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55da92d2e260 0x55da92d36810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:32:34.975 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:34.975+0000 7f3271428740 1 --2- 172.21.15.47:0/126698855 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55da92d2d8b0 0x55da92d2dc90 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:32:34.975 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:34.975+0000 7f3271428740 1 -- 172.21.15.47:0/126698855 >> 172.21.15.47:0/126698855 conn(0x55da92d27f80 msgr2=0x55da92d28360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:32:34.975 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:34.975+0000 7f3271428740 1 -- 172.21.15.47:0/126698855 shutdown_connections 2024-03-21T10:32:34.975 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:34.975+0000 7f3271428740 1 -- 172.21.15.47:0/126698855 wait complete. 2024-03-21T10:32:34.975 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:34.975+0000 7f3267400640 1 --2- 172.21.15.47:0/435714492 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f324c0a3090 0x7f324c0a5550 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f3258006010 tx=0x7f3258004a40 comp rx=0 tx=0).ready entity=mgr.15687 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:32:34.978 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:34.976+0000 7f3271428740 1 -- start start 2024-03-21T10:32:34.978 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:34.976+0000 7f3271428740 1 Processor -- start 2024-03-21T10:32:34.978 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:34.976+0000 7f3271428740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55da92d2d8b0 0x55da92d366f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:32:34.978 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:34.976+0000 7f3271428740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55da92d2e260 0x55da92d33a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:32:34.978 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:34.976+0000 7f3271428740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55da92d36de0 0x55da92d34020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:32:34.978 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:34.976+0000 7f3271428740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x55da92d39d50 con 0x55da92d2e260 2024-03-21T10:32:34.979 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:34.976+0000 7f3271428740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55da92c4d0d0 con 0x55da92d2d8b0 2024-03-21T10:32:34.979 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:34.976+0000 7f3271428740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55da92d39ed0 con 0x55da92d36de0 2024-03-21T10:32:34.979 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:34.976+0000 7f3271428740 10 client.0 _ll_register_callbacks cb 0x55da92c51e80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T10:32:34.979 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:34.977+0000 7f3266a00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55da92d2e260 0x55da92d33a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:32:34.979 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:34.977+0000 7f3266a00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55da92d2e260 0x55da92d33a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:51256/0 (socket says 172.21.15.47:51256) 2024-03-21T10:32:34.979 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:34.977+0000 7f3266a00640 1 -- 172.21.15.47:0/3431600659 learned_addr learned my addr 172.21.15.47:0/3431600659 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:32:34.979 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:34.977+0000 7f3267400640 1 --2- 172.21.15.47:0/3431600659 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55da92d2d8b0 0x55da92d366f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:32:34.979 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:34.977+0000 7f3267e00640 1 --2- 172.21.15.47:0/3431600659 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55da92d36de0 0x55da92d34020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:32:34.979 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:34.977+0000 7f3266a00640 1 -- 172.21.15.47:0/3431600659 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55da92d36de0 msgr2=0x55da92d34020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:32:34.979 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:34.977+0000 7f3266a00640 1 --2- 172.21.15.47:0/3431600659 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55da92d36de0 0x55da92d34020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:32:34.979 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:34.977+0000 7f3266a00640 1 -- 172.21.15.47:0/3431600659 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55da92d2d8b0 msgr2=0x55da92d366f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:32:34.979 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:34.977+0000 7f3266a00640 1 --2- 172.21.15.47:0/3431600659 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55da92d2d8b0 0x55da92d366f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:32:34.979 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:34.977+0000 7f3266a00640 1 -- 172.21.15.47:0/3431600659 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55da929a53f0 con 0x55da92d2e260 2024-03-21T10:32:34.979 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:34.977+0000 7f3267400640 1 --2- 172.21.15.47:0/3431600659 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55da92d2d8b0 0x55da92d366f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T10:32:34.980 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:34.977+0000 7f3266a00640 1 --2- 172.21.15.47:0/3431600659 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55da92d2e260 0x55da92d33a80 secure :-1 s=READY pgs=1064 cs=0 l=1 rev1=1 crypto rx=0x7f32540a4380 tx=0x7f325409bd30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1fd46e5ce7e1e02f server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:32:34.980 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:34.977+0000 7f3251600640 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-03-21T10:32:34.980 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:34.978+0000 7f3251600640 1 -- 172.21.15.47:0/3431600659 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3254005680 con 0x55da92d2e260 2024-03-21T10:32:34.980 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:34.978+0000 7f3251600640 1 -- 172.21.15.47:0/3431600659 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(99..99 src has 1..99) v4 ==== 9562+0+0 (secure 0 0 0) 0x7f32540046c0 con 0x55da92d2e260 2024-03-21T10:32:34.980 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:34.978+0000 7f3271428740 1 -- 172.21.15.47:0/3431600659 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f3260015070 con 0x55da92d2e260 2024-03-21T10:32:34.980 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:34.978+0000 7f3251600640 1 -- 172.21.15.47:0/3431600659 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f325409c5a0 con 0x55da92d2e260 2024-03-21T10:32:34.980 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:34.978+0000 7f3271428740 10 client.17913 fetch_fsmap learned FSMap version 32 2024-03-21T10:32:34.980 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:34.978+0000 7f3271428740 1 -- 172.21.15.47:0/3431600659 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x55da92a6f7d0 con 0x55da92d2e260 2024-03-21T10:32:34.980 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:34.978+0000 7f3251600640 1 -- 172.21.15.47:0/3431600659 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f3260015070 con 0x55da92d2e260 2024-03-21T10:32:34.980 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:34.978+0000 7f3251600640 1 -- 172.21.15.47:0/3431600659 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 32) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f32540a4420 con 0x55da92d2e260 2024-03-21T10:32:34.980 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:34.979+0000 7f3271428740 10 client.17913 fetch_fsmap finished waiting for FSMap version 32 2024-03-21T10:32:34.980 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:34.979+0000 7f3271428740 10 client.17913 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.e 2024-03-21T10:32:34.980 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:34.979+0000 7f3271428740 20 client.17913 populate_metadata read hostname 'smithi047' 2024-03-21T10:32:34.980 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:34.979+0000 7f3271428740 1 --2- 172.21.15.47:0/3431600659 >> [v2:172.21.15.100:6834/2597849957,v1:172.21.15.100:6835/2597849957] conn(0x55da92d46c60 0x55da92d49070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:32:34.980 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:34.979+0000 7f3271428740 4 client.17913 mds_command: new command op to 34454 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T10:32:34.981 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:34.979+0000 7f3271428740 1 -- 172.21.15.47:0/3431600659 --> [v2:172.21.15.100:6834/2597849957,v1:172.21.15.100:6835/2597849957] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55da92c9e800 con 0x55da92d46c60 2024-03-21T10:32:34.981 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:34.979+0000 7f3267400640 1 --2- 172.21.15.47:0/3431600659 >> [v2:172.21.15.100:6834/2597849957,v1:172.21.15.100:6835/2597849957] conn(0x55da92d46c60 0x55da92d49070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:32:34.982 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:34.980+0000 7f3267400640 1 --2- 172.21.15.47:0/3431600659 >> [v2:172.21.15.100:6834/2597849957,v1:172.21.15.100:6835/2597849957] conn(0x55da92d46c60 0x55da92d49070 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=23b1c0f3471ec727 server_cookie=8ff6e614f51d5eb in_seq=0 out_seq=0 2024-03-21T10:32:34.982 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:34.980+0000 7f3251600640 10 client.17913 ms_handle_connect on v2:172.21.15.100:6834/2597849957 2024-03-21T10:32:34.982 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:34.982+0000 7f3251600640 1 -- 172.21.15.47:0/3431600659 <== mds.2 v2:172.21.15.100:6834/2597849957 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55da92c9e800 con 0x55da92d46c60 2024-03-21T10:32:34.982 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:34.982+0000 7f3251600640 10 client.17913 handle_command_reply: tid=0 2024-03-21T10:32:34.982 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:34.982+0000 7f3271428740 1 client.17913 shutdown 2024-03-21T10:32:34.982 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:34.982+0000 7f3271428740 1 -- 172.21.15.47:0/3431600659 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55da92d2e260 msgr2=0x55da92d33a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:32:34.982 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:34.982+0000 7f3271428740 1 --2- 172.21.15.47:0/3431600659 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55da92d2e260 0x55da92d33a80 secure :-1 s=READY pgs=1064 cs=0 l=1 rev1=1 crypto rx=0x7f32540a4380 tx=0x7f325409bd30 comp rx=0 tx=0).stop 2024-03-21T10:32:34.982 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:34.982+0000 7f3271428740 1 -- 172.21.15.47:0/3431600659 shutdown_connections 2024-03-21T10:32:34.982 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:34.982+0000 7f3271428740 1 --2- 172.21.15.47:0/3431600659 >> [v2:172.21.15.100:6834/2597849957,v1:172.21.15.100:6835/2597849957] conn(0x55da92d46c60 0x55da92d49070 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:32:34.982 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:34.982+0000 7f3271428740 1 --2- 172.21.15.47:0/3431600659 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55da92d36de0 0x55da92d34020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:32:34.982 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:34.982+0000 7f3271428740 1 --2- 172.21.15.47:0/3431600659 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55da92d2e260 0x55da92d33a80 unknown :-1 s=CLOSED pgs=1064 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:32:34.983 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:34.982+0000 7f3271428740 1 --2- 172.21.15.47:0/3431600659 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55da92d2d8b0 0x55da92d366f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:32:34.983 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:34.982+0000 7f3251600640 0 client.17913 ms_handle_reset on v2:172.21.15.100:6834/2597849957 2024-03-21T10:32:34.983 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:34.982+0000 7f3271428740 1 -- 172.21.15.47:0/3431600659 >> 172.21.15.47:0/3431600659 conn(0x55da92d27f80 msgr2=0x55da92d29cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:32:34.983 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:34.982+0000 7f3271428740 1 -- 172.21.15.47:0/3431600659 shutdown_connections 2024-03-21T10:32:34.983 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:34.982+0000 7f3271428740 1 -- 172.21.15.47:0/3431600659 wait complete. 2024-03-21T10:32:34.983 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:34.982+0000 7f3271428740 20 client.17913 trim_cache size 0 max 16384 2024-03-21T10:32:34.996 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:34.997+0000 7f3271428740 1 Processor -- start 2024-03-21T10:32:34.996 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:34.997+0000 7f3271428740 1 -- start start 2024-03-21T10:32:34.996 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:34.997+0000 7f3271428740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55da92d2d8b0 0x55da92d361b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:32:34.996 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:34.997+0000 7f3271428740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55da92d2e260 0x55da92d366f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:32:34.996 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:34.997+0000 7f3271428740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55da92d36de0 0x55da92d30ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:32:34.996 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:34.997+0000 7f3271428740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32681b4ed0 con 0x55da92d2d8b0 2024-03-21T10:32:34.996 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:34.997+0000 7f3271428740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32681b6850 con 0x55da92d36de0 2024-03-21T10:32:34.996 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:34.997+0000 7f3271428740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32681b5530 con 0x55da92d2e260 2024-03-21T10:32:34.997 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:34.997+0000 7f3267400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55da92d2d8b0 0x55da92d361b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:32:34.997 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:34.997+0000 7f3267400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55da92d2d8b0 0x55da92d361b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:51262/0 (socket says 172.21.15.47:51262) 2024-03-21T10:32:34.997 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:34.997+0000 7f3267400640 1 -- 172.21.15.47:0/21279081 learned_addr learned my addr 172.21.15.47:0/21279081 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:32:34.997 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:34.998+0000 7f3267e00640 1 --2- 172.21.15.47:0/21279081 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55da92d36de0 0x55da92d30ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:32:34.997 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:34.998+0000 7f3267400640 1 -- 172.21.15.47:0/21279081 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55da92d2e260 msgr2=0x55da92d366f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:32:34.997 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:34.998+0000 7f3267400640 1 --2- 172.21.15.47:0/21279081 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55da92d2e260 0x55da92d366f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:32:34.997 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:34.998+0000 7f3267400640 1 -- 172.21.15.47:0/21279081 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55da92d36de0 msgr2=0x55da92d30ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:32:34.997 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:34.998+0000 7f3267400640 1 --2- 172.21.15.47:0/21279081 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55da92d36de0 0x55da92d30ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:32:34.997 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:34.998+0000 7f3267400640 1 -- 172.21.15.47:0/21279081 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55da92d39c60 con 0x55da92d2d8b0 2024-03-21T10:32:35.000 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:34.998+0000 7f3267400640 1 --2- 172.21.15.47:0/21279081 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55da92d2d8b0 0x55da92d361b0 secure :-1 s=READY pgs=1065 cs=0 l=1 rev1=1 crypto rx=0x7f3258005f00 tx=0x7f3258044000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3eb7522b700ac6b4 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:32:35.000 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:34.998+0000 7f3253e00640 1 -- 172.21.15.47:0/21279081 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32580552d0 con 0x55da92d2d8b0 2024-03-21T10:32:35.000 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:34.998+0000 7f3253e00640 1 -- 172.21.15.47:0/21279081 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f3258055520 con 0x55da92d2d8b0 2024-03-21T10:32:35.001 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:34.998+0000 7f3253e00640 1 -- 172.21.15.47:0/21279081 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f325804e430 con 0x55da92d2d8b0 2024-03-21T10:32:35.001 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:34.999+0000 7f3271428740 1 -- 172.21.15.47:0/21279081 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55da92d2d8b0 msgr2=0x55da92d361b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:32:35.001 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:34.999+0000 7f3271428740 1 --2- 172.21.15.47:0/21279081 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55da92d2d8b0 0x55da92d361b0 secure :-1 s=READY pgs=1065 cs=0 l=1 rev1=1 crypto rx=0x7f3258005f00 tx=0x7f3258044000 comp rx=0 tx=0).stop 2024-03-21T10:32:35.001 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:34.999+0000 7f3271428740 1 -- 172.21.15.47:0/21279081 shutdown_connections 2024-03-21T10:32:35.001 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:34.999+0000 7f3271428740 1 --2- 172.21.15.47:0/21279081 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55da92d36de0 0x55da92d30ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:32:35.001 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:34.999+0000 7f3271428740 1 --2- 172.21.15.47:0/21279081 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55da92d2e260 0x55da92d366f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:32:35.001 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:34.999+0000 7f3271428740 1 --2- 172.21.15.47:0/21279081 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55da92d2d8b0 0x55da92d361b0 unknown :-1 s=CLOSED pgs=1065 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:32:35.001 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:34.999+0000 7f3271428740 1 -- 172.21.15.47:0/21279081 >> 172.21.15.47:0/21279081 conn(0x55da92d27f80 msgr2=0x55da92d27160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:32:35.001 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:34.999+0000 7f3271428740 1 -- 172.21.15.47:0/21279081 shutdown_connections 2024-03-21T10:32:35.001 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:34.999+0000 7f3271428740 1 -- 172.21.15.47:0/21279081 wait complete. 2024-03-21T10:32:35.001 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:35.000+0000 7f3271428740 1 -- start start 2024-03-21T10:32:35.001 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:35.000+0000 7f3271428740 1 Processor -- start 2024-03-21T10:32:35.001 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:35.000+0000 7f3271428740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55da92d2d8b0 0x55da92d3d640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:32:35.001 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:35.000+0000 7f3271428740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55da92d2e260 0x55da92d3db80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:32:35.002 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:35.000+0000 7f3271428740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55da92d36de0 0x55da92d3e0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:32:35.002 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:35.000+0000 7f3271428740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32681b61f0 con 0x55da92d2e260 2024-03-21T10:32:35.002 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:35.000+0000 7f3271428740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32681b5b90 con 0x55da92d2d8b0 2024-03-21T10:32:35.002 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:35.000+0000 7f3271428740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32681b30b0 con 0x55da92d36de0 2024-03-21T10:32:35.002 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:35.000+0000 7f3271428740 10 client.0 _ll_register_callbacks cb 0x7f325400cef0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T10:32:35.002 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:35.001+0000 7f3266a00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55da92d2e260 0x55da92d3db80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:32:35.002 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:35.001+0000 7f3266a00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55da92d2e260 0x55da92d3db80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:51270/0 (socket says 172.21.15.47:51270) 2024-03-21T10:32:35.002 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:35.001+0000 7f3266a00640 1 -- 172.21.15.47:0/586404120 learned_addr learned my addr 172.21.15.47:0/586404120 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:32:35.002 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:35.001+0000 7f3267400640 1 --2- 172.21.15.47:0/586404120 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55da92d2d8b0 0x55da92d3d640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:32:35.002 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:35.001+0000 7f3266a00640 1 -- 172.21.15.47:0/586404120 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55da92d36de0 msgr2=0x55da92d3e0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:32:35.002 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:35.001+0000 7f3267e00640 1 --2- 172.21.15.47:0/586404120 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55da92d36de0 0x55da92d3e0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:32:35.002 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:35.001+0000 7f3266a00640 1 --2- 172.21.15.47:0/586404120 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55da92d36de0 0x55da92d3e0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:32:35.002 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:35.001+0000 7f3266a00640 1 -- 172.21.15.47:0/586404120 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55da92d2d8b0 msgr2=0x55da92d3d640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:32:35.002 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:35.001+0000 7f3266a00640 1 --2- 172.21.15.47:0/586404120 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55da92d2d8b0 0x55da92d3d640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:32:35.003 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:35.001+0000 7f3266a00640 1 -- 172.21.15.47:0/586404120 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55da92da2680 con 0x55da92d2e260 2024-03-21T10:32:35.003 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:35.001+0000 7f3267400640 1 --2- 172.21.15.47:0/586404120 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55da92d2d8b0 0x55da92d3d640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T10:32:35.003 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:35.001+0000 7f3266a00640 1 --2- 172.21.15.47:0/586404120 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55da92d2e260 0x55da92d3db80 secure :-1 s=READY pgs=1066 cs=0 l=1 rev1=1 crypto rx=0x7f325409bba0 tx=0x7f325409bb10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6cc4eff6b7ba3891 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:32:35.003 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:35.001+0000 7f3251600640 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-03-21T10:32:35.003 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:35.002+0000 7f3251600640 1 -- 172.21.15.47:0/586404120 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f325409b800 con 0x55da92d2e260 2024-03-21T10:32:35.003 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:35.002+0000 7f3251600640 1 -- 172.21.15.47:0/586404120 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(99..99 src has 1..99) v4 ==== 9562+0+0 (secure 0 0 0) 0x7f32540b6020 con 0x55da92d2e260 2024-03-21T10:32:35.003 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:35.002+0000 7f3271428740 1 -- 172.21.15.47:0/586404120 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f3258055520 con 0x55da92d2e260 2024-03-21T10:32:35.003 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:35.002+0000 7f3251600640 1 -- 172.21.15.47:0/586404120 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3254082050 con 0x55da92d2e260 2024-03-21T10:32:35.003 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:35.002+0000 7f3271428740 10 client.17919 fetch_fsmap learned FSMap version 32 2024-03-21T10:32:35.003 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:35.002+0000 7f3271428740 1 -- 172.21.15.47:0/586404120 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x55da92da3980 con 0x55da92d2e260 2024-03-21T10:32:35.003 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:35.002+0000 7f3251600640 1 -- 172.21.15.47:0/586404120 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f3258055520 con 0x55da92d2e260 2024-03-21T10:32:35.005 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:35.002+0000 7f3251600640 1 -- 172.21.15.47:0/586404120 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 32) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f32540a4b00 con 0x55da92d2e260 2024-03-21T10:32:35.005 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:35.002+0000 7f3271428740 10 client.17919 fetch_fsmap finished waiting for FSMap version 32 2024-03-21T10:32:35.005 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:35.002+0000 7f3271428740 10 client.17919 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.e 2024-03-21T10:32:35.005 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:35.002+0000 7f3271428740 20 client.17919 populate_metadata read hostname 'smithi047' 2024-03-21T10:32:35.005 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:35.002+0000 7f3271428740 1 --2- 172.21.15.47:0/586404120 >> [v2:172.21.15.100:6834/2597849957,v1:172.21.15.100:6835/2597849957] conn(0x55da92da3ff0 0x55da92da6410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:32:35.005 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:35.003+0000 7f3271428740 4 client.17919 mds_command: new command op to 34454 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-21T10:32:35.005 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:35.003+0000 7f3271428740 1 -- 172.21.15.47:0/586404120 --> [v2:172.21.15.100:6834/2597849957,v1:172.21.15.100:6835/2597849957] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55da92c98b50 con 0x55da92da3ff0 2024-03-21T10:32:35.005 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:35.003+0000 7f3267400640 1 --2- 172.21.15.47:0/586404120 >> [v2:172.21.15.100:6834/2597849957,v1:172.21.15.100:6835/2597849957] conn(0x55da92da3ff0 0x55da92da6410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:32:35.005 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:35.004+0000 7f3267400640 1 --2- 172.21.15.47:0/586404120 >> [v2:172.21.15.100:6834/2597849957,v1:172.21.15.100:6835/2597849957] conn(0x55da92da3ff0 0x55da92da6410 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=71cd8a1f131986f0 server_cookie=75127151c7537809 in_seq=0 out_seq=0 2024-03-21T10:32:35.005 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:35.004+0000 7f3251600640 10 client.17919 ms_handle_connect on v2:172.21.15.100:6834/2597849957 2024-03-21T10:32:35.005 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:35.004+0000 7f3251600640 1 -- 172.21.15.47:0/586404120 <== mds.2 v2:172.21.15.100:6834/2597849957 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55da92c98b50 con 0x55da92da3ff0 2024-03-21T10:32:35.006 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:35.004+0000 7f3251600640 10 client.17919 handle_command_reply: tid=0 2024-03-21T10:32:35.006 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:35.004+0000 7f3271428740 1 client.17919 shutdown 2024-03-21T10:32:35.006 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:35.005+0000 7f3271428740 1 -- 172.21.15.47:0/586404120 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55da92d2e260 msgr2=0x55da92d3db80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:32:35.006 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:35.005+0000 7f3271428740 1 --2- 172.21.15.47:0/586404120 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55da92d2e260 0x55da92d3db80 secure :-1 s=READY pgs=1066 cs=0 l=1 rev1=1 crypto rx=0x7f325409bba0 tx=0x7f325409bb10 comp rx=0 tx=0).stop 2024-03-21T10:32:35.006 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:35.005+0000 7f3271428740 1 -- 172.21.15.47:0/586404120 shutdown_connections 2024-03-21T10:32:35.006 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:35.005+0000 7f3271428740 1 --2- 172.21.15.47:0/586404120 >> [v2:172.21.15.100:6834/2597849957,v1:172.21.15.100:6835/2597849957] conn(0x55da92da3ff0 0x55da92da6410 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:32:35.006 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:35.005+0000 7f3271428740 1 --2- 172.21.15.47:0/586404120 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55da92d36de0 0x55da92d3e0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:32:35.006 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:35.005+0000 7f3271428740 1 --2- 172.21.15.47:0/586404120 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55da92d2e260 0x55da92d3db80 unknown :-1 s=CLOSED pgs=1066 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:32:35.006 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:35.005+0000 7f3271428740 1 --2- 172.21.15.47:0/586404120 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55da92d2d8b0 0x55da92d3d640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:32:35.006 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:35.005+0000 7f3251600640 0 client.17919 ms_handle_reset on v2:172.21.15.100:6834/2597849957 2024-03-21T10:32:35.006 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:35.005+0000 7f3271428740 1 -- 172.21.15.47:0/586404120 >> 172.21.15.47:0/586404120 conn(0x55da92d27f80 msgr2=0x55da92d268d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:32:35.006 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:35.005+0000 7f3271428740 1 -- 172.21.15.47:0/586404120 shutdown_connections 2024-03-21T10:32:35.006 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:35.005+0000 7f3271428740 1 -- 172.21.15.47:0/586404120 wait complete. 2024-03-21T10:32:35.006 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:35.005+0000 7f3271428740 20 client.17919 trim_cache size 0 max 16384 2024-03-21T10:32:35.007 INFO:teuthology.orchestra.run.smithi047.stdout:[] 2024-03-21T10:32:35.007 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:35.005+0000 7f326d200640 1 -- 172.21.15.47:0/435714492 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f324c0a3090 msgr2=0x7f324c0a5550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:32:35.008 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:35.005+0000 7f326d200640 1 --2- 172.21.15.47:0/435714492 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f324c0a3090 0x7f324c0a5550 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f3258006010 tx=0x7f3258004a40 comp rx=0 tx=0).stop 2024-03-21T10:32:35.008 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:35.005+0000 7f326d200640 1 -- 172.21.15.47:0/435714492 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f32680743f0 msgr2=0x7f32681019a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:32:35.008 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:35.006+0000 7f326d200640 1 --2- 172.21.15.47:0/435714492 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f32680743f0 0x7f32681019a0 secure :-1 s=READY pgs=1063 cs=0 l=1 rev1=1 crypto rx=0x7f325400e9c0 tx=0x7f325400ee80 comp rx=0 tx=0).stop 2024-03-21T10:32:35.008 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:35.006+0000 7f326d200640 1 -- 172.21.15.47:0/435714492 shutdown_connections 2024-03-21T10:32:35.008 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:35.006+0000 7f326d200640 1 --2- 172.21.15.47:0/435714492 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f324c0a3090 0x7f324c0a5550 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:32:35.008 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:35.006+0000 7f326d200640 1 --2- 172.21.15.47:0/435714492 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f326810dd40 0x7f3268101ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:32:35.008 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:35.006+0000 7f326d200640 1 --2- 172.21.15.47:0/435714492 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f32680743f0 0x7f32681019a0 unknown :-1 s=CLOSED pgs=1063 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:32:35.008 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:35.006+0000 7f326d200640 1 --2- 172.21.15.47:0/435714492 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3268073ad0 0x7f3268101460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:32:35.008 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:35.006+0000 7f326d200640 1 -- 172.21.15.47:0/435714492 >> 172.21.15.47:0/435714492 conn(0x7f32680fd010 msgr2=0x7f32680fdd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:32:35.008 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:35.006+0000 7f326d200640 1 -- 172.21.15.47:0/435714492 shutdown_connections 2024-03-21T10:32:35.008 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:35.006+0000 7f326d200640 1 -- 172.21.15.47:0/435714492 wait complete. 2024-03-21T10:32:35.021 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-21T10:32:35.021 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-03-21T10:32:35.245 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:35.246+0000 7f1bc2000640 1 -- 172.21.15.47:0/4018147012 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1bbc103720 msgr2=0x7f1bbc103ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:32:35.245 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:35.246+0000 7f1bc2000640 1 --2- 172.21.15.47:0/4018147012 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1bbc103720 0x7f1bbc103ba0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f1ba8009900 tx=0x7f1ba802bc40 comp rx=0 tx=0).stop 2024-03-21T10:32:35.245 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:35.246+0000 7f1bc2000640 1 -- 172.21.15.47:0/4018147012 shutdown_connections 2024-03-21T10:32:35.245 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:35.246+0000 7f1bc2000640 1 --2- 172.21.15.47:0/4018147012 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1bbc1040e0 0x7f1bbc1105f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:32:35.245 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:35.246+0000 7f1bc2000640 1 --2- 172.21.15.47:0/4018147012 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1bbc103720 0x7f1bbc103ba0 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:32:35.245 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:35.246+0000 7f1bc2000640 1 --2- 172.21.15.47:0/4018147012 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1bbc109730 0x7f1bbc109b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:32:35.246 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:35.246+0000 7f1bc2000640 1 -- 172.21.15.47:0/4018147012 >> 172.21.15.47:0/4018147012 conn(0x7f1bbc0ff1b0 msgr2=0x7f1bbc1015d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:32:35.246 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:35.246+0000 7f1bc2000640 1 -- 172.21.15.47:0/4018147012 shutdown_connections 2024-03-21T10:32:35.246 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:35.246+0000 7f1bc2000640 1 -- 172.21.15.47:0/4018147012 wait complete. 2024-03-21T10:32:35.246 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:35.248+0000 7f1bc2000640 1 Processor -- start 2024-03-21T10:32:35.246 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:35.248+0000 7f1bc2000640 1 -- start start 2024-03-21T10:32:35.247 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:35.248+0000 7f1bc2000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1bbc1040e0 0x7f1bbc19a0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:32:35.247 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:35.249+0000 7f1bc2000640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1bbc109730 0x7f1bbc19a610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:32:35.247 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:35.249+0000 7f1bc2000640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1bbc19e990 0x7f1bbc19ee40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:32:35.247 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:35.249+0000 7f1bc2000640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1bbc113a00 con 0x7f1bbc109730 2024-03-21T10:32:35.248 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:35.249+0000 7f1bc2000640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1bbc113880 con 0x7f1bbc1040e0 2024-03-21T10:32:35.248 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:35.249+0000 7f1bc2000640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1bbc113b80 con 0x7f1bbc19e990 2024-03-21T10:32:35.248 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:35.249+0000 7f1bc0c00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1bbc1040e0 0x7f1bbc19a0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:32:35.248 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:35.249+0000 7f1bc0c00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1bbc1040e0 0x7f1bbc19a0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:57452/0 (socket says 172.21.15.47:57452) 2024-03-21T10:32:35.248 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:35.249+0000 7f1bc0c00640 1 -- 172.21.15.47:0/2829549149 learned_addr learned my addr 172.21.15.47:0/2829549149 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:32:35.248 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:35.249+0000 7f1bbbe00640 1 --2- 172.21.15.47:0/2829549149 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1bbc109730 0x7f1bbc19a610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:32:35.249 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:35.249+0000 7f1bc1600640 1 --2- 172.21.15.47:0/2829549149 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1bbc19e990 0x7f1bbc19ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:32:35.249 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:35.250+0000 7f1bc0c00640 1 -- 172.21.15.47:0/2829549149 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1bbc19e990 msgr2=0x7f1bbc19ee40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:32:35.249 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:35.250+0000 7f1bc0c00640 1 --2- 172.21.15.47:0/2829549149 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1bbc19e990 0x7f1bbc19ee40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:32:35.249 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:35.250+0000 7f1bc0c00640 1 -- 172.21.15.47:0/2829549149 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1bbc109730 msgr2=0x7f1bbc19a610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:32:35.249 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:35.250+0000 7f1bc0c00640 1 --2- 172.21.15.47:0/2829549149 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1bbc109730 0x7f1bbc19a610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:32:35.249 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:35.250+0000 7f1bc0c00640 1 -- 172.21.15.47:0/2829549149 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1bbc19f480 con 0x7f1bbc1040e0 2024-03-21T10:32:35.250 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:35.250+0000 7f1bc0c00640 1 --2- 172.21.15.47:0/2829549149 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1bbc1040e0 0x7f1bbc19a0d0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f1bb000e9f0 tx=0x7f1bb000eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:32:35.250 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:35.250+0000 7f1bb9600640 1 -- 172.21.15.47:0/2829549149 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1bb000cd80 con 0x7f1bbc1040e0 2024-03-21T10:32:35.250 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:35.251+0000 7f1bc2000640 1 -- 172.21.15.47:0/2829549149 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1bbc19f770 con 0x7f1bbc1040e0 2024-03-21T10:32:35.250 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:35.251+0000 7f1bb9600640 1 -- 172.21.15.47:0/2829549149 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f1bb00108e0 con 0x7f1bbc1040e0 2024-03-21T10:32:35.250 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:35.251+0000 7f1bb9600640 1 -- 172.21.15.47:0/2829549149 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1bb0018920 con 0x7f1bbc1040e0 2024-03-21T10:32:35.250 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:35.251+0000 7f1bc2000640 1 -- 172.21.15.47:0/2829549149 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1bbc1a6f30 con 0x7f1bbc1040e0 2024-03-21T10:32:35.255 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:35.254+0000 7f1bc6222740 1 Processor -- start 2024-03-21T10:32:35.255 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:35.254+0000 7f1bc6222740 1 -- start start 2024-03-21T10:32:35.255 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:35.255+0000 7f1bc6222740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56458aa558b0 0x56458aa55c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:32:35.255 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:35.255+0000 7f1bc6222740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56458aa56260 0x56458aa5e810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:32:35.255 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:35.255+0000 7f1bc6222740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56458aa5ede0 0x56458aa61230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:32:35.255 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:35.255+0000 7f1bc6222740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x56458a9b92c0 con 0x56458aa558b0 2024-03-21T10:32:35.255 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:35.255+0000 7f1bc6222740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x56458a961c90 con 0x56458aa5ede0 2024-03-21T10:32:35.255 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:35.255+0000 7f1bc6222740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x56458a9b5360 con 0x56458aa56260 2024-03-21T10:32:35.255 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:35.255+0000 7f1bc1600640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56458aa558b0 0x56458aa55c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:32:35.256 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:35.255+0000 7f1bc1600640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56458aa558b0 0x56458aa55c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:51292/0 (socket says 172.21.15.47:51292) 2024-03-21T10:32:35.256 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:35.255+0000 7f1bc1600640 1 -- 172.21.15.47:0/3278544944 learned_addr learned my addr 172.21.15.47:0/3278544944 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:32:35.256 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:35.255+0000 7f1bbbe00640 1 --2- 172.21.15.47:0/3278544944 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56458aa5ede0 0x56458aa61230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:32:35.256 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:35.255+0000 7f1bc1600640 1 -- 172.21.15.47:0/3278544944 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56458aa56260 msgr2=0x56458aa5e810 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-21T10:32:35.256 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:35.255+0000 7f1bc1600640 1 --2- 172.21.15.47:0/3278544944 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56458aa56260 0x56458aa5e810 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:32:35.256 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:35.255+0000 7f1bc1600640 1 -- 172.21.15.47:0/3278544944 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56458aa5ede0 msgr2=0x56458aa61230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:32:35.256 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:35.255+0000 7f1bc1600640 1 --2- 172.21.15.47:0/3278544944 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56458aa5ede0 0x56458aa61230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:32:35.256 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:35.255+0000 7f1bc1600640 1 -- 172.21.15.47:0/3278544944 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56458a986080 con 0x56458aa558b0 2024-03-21T10:32:35.256 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:35.256+0000 7f1bb9600640 1 -- 172.21.15.47:0/2829549149 <== mon.2 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 27) v1 ==== 137985+0+0 (secure 0 0 0) 0x7f1bb0010a90 con 0x7f1bbc1040e0 2024-03-21T10:32:35.256 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:35.256+0000 7f1bc1600640 1 --2- 172.21.15.47:0/3278544944 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56458aa558b0 0x56458aa55c90 secure :-1 s=READY pgs=1067 cs=0 l=1 rev1=1 crypto rx=0x7f1bb4009ee0 tx=0x7f1bb4010740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6743832d65a5e5d2 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:32:35.257 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:35.256+0000 7f1badc00640 1 -- 172.21.15.47:0/3278544944 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1bb4016070 con 0x56458aa558b0 2024-03-21T10:32:35.257 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:35.256+0000 7f1badc00640 1 -- 172.21.15.47:0/3278544944 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f1bb4004d20 con 0x56458aa558b0 2024-03-21T10:32:35.257 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:35.256+0000 7f1badc00640 1 -- 172.21.15.47:0/3278544944 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1bb40126f0 con 0x56458aa558b0 2024-03-21T10:32:35.261 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:35.257+0000 7f1bc6222740 1 -- 172.21.15.47:0/3278544944 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56458aa558b0 msgr2=0x56458aa55c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:32:35.261 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:35.257+0000 7f1bc6222740 1 --2- 172.21.15.47:0/3278544944 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56458aa558b0 0x56458aa55c90 secure :-1 s=READY pgs=1067 cs=0 l=1 rev1=1 crypto rx=0x7f1bb4009ee0 tx=0x7f1bb4010740 comp rx=0 tx=0).stop 2024-03-21T10:32:35.261 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:35.257+0000 7f1bc6222740 1 -- 172.21.15.47:0/3278544944 shutdown_connections 2024-03-21T10:32:35.261 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:35.257+0000 7f1bc6222740 1 --2- 172.21.15.47:0/3278544944 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56458aa5ede0 0x56458aa61230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:32:35.261 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:35.257+0000 7f1bc6222740 1 --2- 172.21.15.47:0/3278544944 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56458aa56260 0x56458aa5e810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:32:35.261 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:35.257+0000 7f1bc6222740 1 --2- 172.21.15.47:0/3278544944 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56458aa558b0 0x56458aa55c90 unknown :-1 s=CLOSED pgs=1067 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:32:35.261 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:35.257+0000 7f1bc6222740 1 -- 172.21.15.47:0/3278544944 >> 172.21.15.47:0/3278544944 conn(0x56458aa4ff80 msgr2=0x56458aa50360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:32:35.261 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:35.257+0000 7f1bc6222740 1 -- 172.21.15.47:0/3278544944 shutdown_connections 2024-03-21T10:32:35.261 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:35.257+0000 7f1bc6222740 1 -- 172.21.15.47:0/3278544944 wait complete. 2024-03-21T10:32:35.261 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:35.257+0000 7f1bb9600640 1 --2- 172.21.15.47:0/2829549149 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f1ba40a2fa0 0x7f1ba40a5460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:32:35.261 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:35.257+0000 7f1bb9600640 1 -- 172.21.15.47:0/2829549149 <== mon.2 v2:172.21.15.100:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 9562+0+0 (secure 0 0 0) 0x7f1bb00cff50 con 0x7f1bbc1040e0 2024-03-21T10:32:35.261 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:35.258+0000 7f1bc6222740 1 -- start start 2024-03-21T10:32:35.261 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:35.258+0000 7f1bbbe00640 1 --2- 172.21.15.47:0/2829549149 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f1ba40a2fa0 0x7f1ba40a5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:32:35.262 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:35.258+0000 7f1bc6222740 1 Processor -- start 2024-03-21T10:32:35.262 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:35.258+0000 7f1bc6222740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56458aa558b0 0x56458aa5e6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:32:35.262 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:35.258+0000 7f1bc6222740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56458aa56260 0x56458aa5ba80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:32:35.262 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:35.258+0000 7f1bc6222740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56458aa5ede0 0x56458aa5c020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:32:35.262 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:35.258+0000 7f1bc6222740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x56458aa61d50 con 0x56458aa558b0 2024-03-21T10:32:35.262 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:35.259+0000 7f1bc6222740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x56458a9750d0 con 0x56458aa5ede0 2024-03-21T10:32:35.262 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:35.259+0000 7f1bc6222740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x56458aa61ed0 con 0x56458aa56260 2024-03-21T10:32:35.262 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:35.259+0000 7f1bbbe00640 1 --2- 172.21.15.47:0/2829549149 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f1ba40a2fa0 0x7f1ba40a5460 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f1ba8036040 tx=0x7f1ba8005a20 comp rx=0 tx=0).ready entity=mgr.15687 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:32:35.262 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:35.259+0000 7f1bc6222740 10 client.0 _ll_register_callbacks cb 0x56458a979e80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T10:32:35.262 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:35.259+0000 7f1bc0c00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56458aa558b0 0x56458aa5e6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:32:35.262 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:35.259+0000 7f1bc0c00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56458aa558b0 0x56458aa5e6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:51306/0 (socket says 172.21.15.47:51306) 2024-03-21T10:32:35.262 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:35.259+0000 7f1bc0c00640 1 -- 172.21.15.47:0/188492627 learned_addr learned my addr 172.21.15.47:0/188492627 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:32:35.262 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:35.259+0000 7f1bc1600640 1 --2- 172.21.15.47:0/188492627 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56458aa5ede0 0x56458aa5c020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:32:35.262 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:35.259+0000 7f1bbbe00640 1 --2- 172.21.15.47:0/188492627 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56458aa56260 0x56458aa5ba80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:32:35.263 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:35.259+0000 7f1bc0c00640 1 -- 172.21.15.47:0/188492627 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56458aa56260 msgr2=0x56458aa5ba80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:32:35.263 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:35.259+0000 7f1bc0c00640 1 --2- 172.21.15.47:0/188492627 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56458aa56260 0x56458aa5ba80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:32:35.263 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:35.259+0000 7f1bc0c00640 1 -- 172.21.15.47:0/188492627 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56458aa5ede0 msgr2=0x56458aa5c020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:32:35.263 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:35.259+0000 7f1bc0c00640 1 --2- 172.21.15.47:0/188492627 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56458aa5ede0 0x56458aa5c020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:32:35.263 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:35.259+0000 7f1bc0c00640 1 -- 172.21.15.47:0/188492627 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56458a6cd3f0 con 0x56458aa558b0 2024-03-21T10:32:35.263 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:35.259+0000 7f1bbbe00640 1 --2- 172.21.15.47:0/188492627 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56458aa56260 0x56458aa5ba80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T10:32:35.263 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:35.260+0000 7f1bc0c00640 1 --2- 172.21.15.47:0/188492627 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56458aa558b0 0x56458aa5e6f0 secure :-1 s=READY pgs=1068 cs=0 l=1 rev1=1 crypto rx=0x7f1bb00cd890 tx=0x7f1bb0010ea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e71093ce4670e2b2 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:32:35.263 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:35.260+0000 7f1ba2000640 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-03-21T10:32:35.263 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:35.260+0000 7f1ba2000640 1 -- 172.21.15.47:0/188492627 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1bb00a0080 con 0x56458aa558b0 2024-03-21T10:32:35.263 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:35.260+0000 7f1ba2000640 1 -- 172.21.15.47:0/188492627 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(99..99 src has 1..99) v4 ==== 9562+0+0 (secure 0 0 0) 0x7f1bb0010430 con 0x56458aa558b0 2024-03-21T10:32:35.263 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:35.260+0000 7f1bc6222740 1 -- 172.21.15.47:0/188492627 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f1bb4004d20 con 0x56458aa558b0 2024-03-21T10:32:35.263 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:35.260+0000 7f1ba2000640 1 -- 172.21.15.47:0/188492627 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1bb00a4940 con 0x56458aa558b0 2024-03-21T10:32:35.263 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:35.260+0000 7f1bc6222740 10 client.17931 fetch_fsmap learned FSMap version 32 2024-03-21T10:32:35.263 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:35.260+0000 7f1bc6222740 1 -- 172.21.15.47:0/188492627 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x56458a7977d0 con 0x56458aa558b0 2024-03-21T10:32:35.264 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:35.260+0000 7f1ba2000640 1 -- 172.21.15.47:0/188492627 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f1bb4004d20 con 0x56458aa558b0 2024-03-21T10:32:35.264 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:35.261+0000 7f1ba2000640 1 -- 172.21.15.47:0/188492627 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 32) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f1bb0018ac0 con 0x56458aa558b0 2024-03-21T10:32:35.264 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:35.261+0000 7f1bc6222740 10 client.17931 fetch_fsmap finished waiting for FSMap version 32 2024-03-21T10:32:35.264 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:35.261+0000 7f1bc6222740 10 client.17931 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-03-21T10:32:35.264 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:35.261+0000 7f1bc6222740 20 client.17931 populate_metadata read hostname 'smithi047' 2024-03-21T10:32:35.264 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:35.261+0000 7f1bc6222740 1 --2- 172.21.15.47:0/188492627 >> [v2:172.21.15.100:6836/3497644162,v1:172.21.15.100:6837/3497644162] conn(0x56458aa6ec60 0x56458aa71050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:32:35.264 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:35.261+0000 7f1bc6222740 4 client.17931 mds_command: new command op to 34493 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T10:32:35.264 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:35.261+0000 7f1bc6222740 1 -- 172.21.15.47:0/188492627 --> [v2:172.21.15.100:6836/3497644162,v1:172.21.15.100:6837/3497644162] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56458a9c6800 con 0x56458aa6ec60 2024-03-21T10:32:35.264 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:35.261+0000 7f1bbbe00640 1 --2- 172.21.15.47:0/188492627 >> [v2:172.21.15.100:6836/3497644162,v1:172.21.15.100:6837/3497644162] conn(0x56458aa6ec60 0x56458aa71050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:32:35.264 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:35.262+0000 7f1bbbe00640 1 --2- 172.21.15.47:0/188492627 >> [v2:172.21.15.100:6836/3497644162,v1:172.21.15.100:6837/3497644162] conn(0x56458aa6ec60 0x56458aa71050 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=58579a8d5d6a769e server_cookie=e8f46df66af0add8 in_seq=0 out_seq=0 2024-03-21T10:32:35.264 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:35.262+0000 7f1ba2000640 10 client.17931 ms_handle_connect on v2:172.21.15.100:6836/3497644162 2024-03-21T10:32:35.265 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:35.264+0000 7f1ba2000640 1 -- 172.21.15.47:0/188492627 <== mds.1 v2:172.21.15.100:6836/3497644162 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x56458a9c6800 con 0x56458aa6ec60 2024-03-21T10:32:35.265 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:35.264+0000 7f1ba2000640 10 client.17931 handle_command_reply: tid=0 2024-03-21T10:32:35.265 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:35.264+0000 7f1bc6222740 1 client.17931 shutdown 2024-03-21T10:32:35.265 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:35.264+0000 7f1bc6222740 1 -- 172.21.15.47:0/188492627 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56458aa558b0 msgr2=0x56458aa5e6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:32:35.266 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:35.264+0000 7f1bc6222740 1 --2- 172.21.15.47:0/188492627 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56458aa558b0 0x56458aa5e6f0 secure :-1 s=READY pgs=1068 cs=0 l=1 rev1=1 crypto rx=0x7f1bb00cd890 tx=0x7f1bb0010ea0 comp rx=0 tx=0).stop 2024-03-21T10:32:35.266 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:35.264+0000 7f1bc6222740 1 -- 172.21.15.47:0/188492627 shutdown_connections 2024-03-21T10:32:35.266 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:35.264+0000 7f1bc6222740 1 --2- 172.21.15.47:0/188492627 >> [v2:172.21.15.100:6836/3497644162,v1:172.21.15.100:6837/3497644162] conn(0x56458aa6ec60 0x56458aa71050 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:32:35.266 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:35.264+0000 7f1bc6222740 1 --2- 172.21.15.47:0/188492627 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56458aa5ede0 0x56458aa5c020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:32:35.266 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:35.264+0000 7f1bc6222740 1 --2- 172.21.15.47:0/188492627 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56458aa56260 0x56458aa5ba80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:32:35.266 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:35.264+0000 7f1bc6222740 1 --2- 172.21.15.47:0/188492627 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56458aa558b0 0x56458aa5e6f0 unknown :-1 s=CLOSED pgs=1068 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:32:35.266 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:35.264+0000 7f1ba2000640 0 client.17931 ms_handle_reset on v2:172.21.15.100:6836/3497644162 2024-03-21T10:32:35.266 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:35.264+0000 7f1bc6222740 1 -- 172.21.15.47:0/188492627 >> 172.21.15.47:0/188492627 conn(0x56458aa4ff80 msgr2=0x56458aa51cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:32:35.266 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:35.264+0000 7f1bc6222740 1 -- 172.21.15.47:0/188492627 shutdown_connections 2024-03-21T10:32:35.266 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:35.264+0000 7f1bc6222740 1 -- 172.21.15.47:0/188492627 wait complete. 2024-03-21T10:32:35.266 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:35.265+0000 7f1bc6222740 20 client.17931 trim_cache size 0 max 16384 2024-03-21T10:32:35.277 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:35.279+0000 7f1bc6222740 1 Processor -- start 2024-03-21T10:32:35.277 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:35.279+0000 7f1bc6222740 1 -- start start 2024-03-21T10:32:35.279 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:35.279+0000 7f1bc6222740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56458aa558b0 0x56458aa4f3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:32:35.279 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:35.279+0000 7f1bc6222740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56458aa56260 0x56458aa5e1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:32:35.279 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:35.279+0000 7f1bc6222740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56458aa5ede0 0x56458aa5e6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:32:35.279 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:35.279+0000 7f1bc6222740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1bbc1b0ff0 con 0x56458aa56260 2024-03-21T10:32:35.279 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:35.279+0000 7f1bc6222740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1bbc1b2970 con 0x56458aa558b0 2024-03-21T10:32:35.279 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:35.279+0000 7f1bc6222740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1bbc1b1650 con 0x56458aa5ede0 2024-03-21T10:32:35.279 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:35.280+0000 7f1bbbe00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56458aa56260 0x56458aa5e1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:32:35.279 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:35.280+0000 7f1bbbe00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56458aa56260 0x56458aa5e1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:51312/0 (socket says 172.21.15.47:51312) 2024-03-21T10:32:35.279 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:35.280+0000 7f1bbbe00640 1 -- 172.21.15.47:0/3531563864 learned_addr learned my addr 172.21.15.47:0/3531563864 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:32:35.279 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:35.280+0000 7f1bc0c00640 1 --2- 172.21.15.47:0/3531563864 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56458aa558b0 0x56458aa4f3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:32:35.279 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:35.280+0000 7f1bc1600640 1 --2- 172.21.15.47:0/3531563864 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56458aa5ede0 0x56458aa5e6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:32:35.280 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:35.280+0000 7f1bbbe00640 1 -- 172.21.15.47:0/3531563864 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56458aa5ede0 msgr2=0x56458aa5e6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:32:35.280 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:35.280+0000 7f1bbbe00640 1 --2- 172.21.15.47:0/3531563864 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56458aa5ede0 0x56458aa5e6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:32:35.280 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:35.280+0000 7f1bbbe00640 1 -- 172.21.15.47:0/3531563864 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56458aa558b0 msgr2=0x56458aa4f3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:32:35.280 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:35.280+0000 7f1bbbe00640 1 --2- 172.21.15.47:0/3531563864 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56458aa558b0 0x56458aa4f3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:32:35.280 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:35.280+0000 7f1bbbe00640 1 -- 172.21.15.47:0/3531563864 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56458aa55490 con 0x56458aa56260 2024-03-21T10:32:35.280 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:35.280+0000 7f1bbbe00640 1 --2- 172.21.15.47:0/3531563864 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56458aa56260 0x56458aa5e1b0 secure :-1 s=READY pgs=1069 cs=0 l=1 rev1=1 crypto rx=0x7f1ba80096c0 tx=0x7f1ba8045000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2eb0d5587c4aa846 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:32:35.282 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:35.281+0000 7f1bad200640 1 -- 172.21.15.47:0/3531563864 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ba802c790 con 0x56458aa56260 2024-03-21T10:32:35.282 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:35.281+0000 7f1bad200640 1 -- 172.21.15.47:0/3531563864 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f1ba8005040 con 0x56458aa56260 2024-03-21T10:32:35.282 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:35.281+0000 7f1bad200640 1 -- 172.21.15.47:0/3531563864 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ba8004920 con 0x56458aa56260 2024-03-21T10:32:35.282 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:35.282+0000 7f1bc6222740 1 -- 172.21.15.47:0/3531563864 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56458aa56260 msgr2=0x56458aa5e1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:32:35.283 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:35.282+0000 7f1bc6222740 1 --2- 172.21.15.47:0/3531563864 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56458aa56260 0x56458aa5e1b0 secure :-1 s=READY pgs=1069 cs=0 l=1 rev1=1 crypto rx=0x7f1ba80096c0 tx=0x7f1ba8045000 comp rx=0 tx=0).stop 2024-03-21T10:32:35.283 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:35.282+0000 7f1bc6222740 1 -- 172.21.15.47:0/3531563864 shutdown_connections 2024-03-21T10:32:35.283 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:35.282+0000 7f1bc6222740 1 --2- 172.21.15.47:0/3531563864 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56458aa5ede0 0x56458aa5e6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:32:35.283 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:35.282+0000 7f1bc6222740 1 --2- 172.21.15.47:0/3531563864 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56458aa56260 0x56458aa5e1b0 unknown :-1 s=CLOSED pgs=1069 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:32:35.283 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:35.282+0000 7f1bc6222740 1 --2- 172.21.15.47:0/3531563864 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56458aa558b0 0x56458aa4f3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:32:35.283 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:35.282+0000 7f1bc6222740 1 -- 172.21.15.47:0/3531563864 >> 172.21.15.47:0/3531563864 conn(0x56458aa4ff80 msgr2=0x56458aa51b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:32:35.283 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:35.282+0000 7f1bc6222740 1 -- 172.21.15.47:0/3531563864 shutdown_connections 2024-03-21T10:32:35.283 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:35.282+0000 7f1bc6222740 1 -- 172.21.15.47:0/3531563864 wait complete. 2024-03-21T10:32:35.283 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:35.282+0000 7f1bc6222740 1 -- start start 2024-03-21T10:32:35.283 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:35.283+0000 7f1bc6222740 1 Processor -- start 2024-03-21T10:32:35.283 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:35.283+0000 7f1bc6222740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56458aa558b0 0x56458aa65b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:32:35.283 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:35.283+0000 7f1bc6222740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56458aa56260 0x56458aa660c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:32:35.283 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:35.283+0000 7f1bc6222740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56458aa5ede0 0x56458aaca460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:32:35.283 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:35.283+0000 7f1bc6222740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1bbc1b2310 con 0x56458aa5ede0 2024-03-21T10:32:35.284 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:35.283+0000 7f1bc6222740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1bbc1b1cb0 con 0x56458aa558b0 2024-03-21T10:32:35.284 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:35.283+0000 7f1bc6222740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1bbc1af1d0 con 0x56458aa56260 2024-03-21T10:32:35.284 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:35.283+0000 7f1bc6222740 10 client.0 _ll_register_callbacks cb 0x7f1bb0009700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T10:32:35.284 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:35.283+0000 7f1bc1600640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56458aa5ede0 0x56458aaca460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:32:35.284 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:35.283+0000 7f1bc1600640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56458aa5ede0 0x56458aaca460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:51318/0 (socket says 172.21.15.47:51318) 2024-03-21T10:32:35.284 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:35.283+0000 7f1bc1600640 1 -- 172.21.15.47:0/1078876940 learned_addr learned my addr 172.21.15.47:0/1078876940 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:32:35.284 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:35.284+0000 7f1bc0c00640 1 --2- 172.21.15.47:0/1078876940 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56458aa558b0 0x56458aa65b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:32:35.284 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:35.284+0000 7f1bbbe00640 1 --2- 172.21.15.47:0/1078876940 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56458aa56260 0x56458aa660c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:32:35.284 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:35.284+0000 7f1bc1600640 1 -- 172.21.15.47:0/1078876940 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56458aa56260 msgr2=0x56458aa660c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:32:35.284 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:35.284+0000 7f1bc1600640 1 --2- 172.21.15.47:0/1078876940 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56458aa56260 0x56458aa660c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:32:35.284 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:35.284+0000 7f1bc1600640 1 -- 172.21.15.47:0/1078876940 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56458aa558b0 msgr2=0x56458aa65b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:32:35.284 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:35.284+0000 7f1bc1600640 1 --2- 172.21.15.47:0/1078876940 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56458aa558b0 0x56458aa65b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:32:35.284 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:35.284+0000 7f1bc1600640 1 -- 172.21.15.47:0/1078876940 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56458aaca9a0 con 0x56458aa5ede0 2024-03-21T10:32:35.284 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:35.284+0000 7f1bc1600640 1 --2- 172.21.15.47:0/1078876940 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56458aa5ede0 0x56458aaca460 secure :-1 s=READY pgs=1070 cs=0 l=1 rev1=1 crypto rx=0x7f1bb4009ee0 tx=0x7f1bb4009c50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=43b4776beb47d39e server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:32:35.284 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:35.284+0000 7f1ba2000640 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-03-21T10:32:35.287 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:35.284+0000 7f1ba2000640 1 -- 172.21.15.47:0/1078876940 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1bb4016040 con 0x56458aa5ede0 2024-03-21T10:32:35.287 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:35.284+0000 7f1ba2000640 1 -- 172.21.15.47:0/1078876940 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(99..99 src has 1..99) v4 ==== 9562+0+0 (secure 0 0 0) 0x7f1bb40138f0 con 0x56458aa5ede0 2024-03-21T10:32:35.287 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:35.284+0000 7f1bc6222740 1 -- 172.21.15.47:0/1078876940 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f1ba8005040 con 0x56458aa5ede0 2024-03-21T10:32:35.287 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:35.285+0000 7f1ba2000640 1 -- 172.21.15.47:0/1078876940 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1bb4032050 con 0x56458aa5ede0 2024-03-21T10:32:35.287 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:35.285+0000 7f1bc6222740 10 client.17937 fetch_fsmap learned FSMap version 32 2024-03-21T10:32:35.287 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:35.285+0000 7f1bc6222740 1 -- 172.21.15.47:0/1078876940 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x56458aacbc10 con 0x56458aa5ede0 2024-03-21T10:32:35.287 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:35.285+0000 7f1ba2000640 1 -- 172.21.15.47:0/1078876940 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f1ba8005040 con 0x56458aa5ede0 2024-03-21T10:32:35.287 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:35.285+0000 7f1ba2000640 1 -- 172.21.15.47:0/1078876940 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 32) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f1bb402d030 con 0x56458aa5ede0 2024-03-21T10:32:35.287 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:35.285+0000 7f1bc6222740 10 client.17937 fetch_fsmap finished waiting for FSMap version 32 2024-03-21T10:32:35.287 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:35.285+0000 7f1bc6222740 10 client.17937 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-03-21T10:32:35.287 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:35.285+0000 7f1bc6222740 20 client.17937 populate_metadata read hostname 'smithi047' 2024-03-21T10:32:35.287 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:35.285+0000 7f1bc6222740 1 --2- 172.21.15.47:0/1078876940 >> [v2:172.21.15.100:6836/3497644162,v1:172.21.15.100:6837/3497644162] conn(0x56458aacc000 0x56458aace420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:32:35.288 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:35.285+0000 7f1bc6222740 4 client.17937 mds_command: new command op to 34493 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-21T10:32:35.288 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:35.285+0000 7f1bc6222740 1 -- 172.21.15.47:0/1078876940 --> [v2:172.21.15.100:6836/3497644162,v1:172.21.15.100:6837/3497644162] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x56458a9c0b50 con 0x56458aacc000 2024-03-21T10:32:35.288 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:35.286+0000 7f1bc0c00640 1 --2- 172.21.15.47:0/1078876940 >> [v2:172.21.15.100:6836/3497644162,v1:172.21.15.100:6837/3497644162] conn(0x56458aacc000 0x56458aace420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:32:35.288 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:35.287+0000 7f1bc0c00640 1 --2- 172.21.15.47:0/1078876940 >> [v2:172.21.15.100:6836/3497644162,v1:172.21.15.100:6837/3497644162] conn(0x56458aacc000 0x56458aace420 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=eb27e8d965b45eb8 server_cookie=849cdce29a62148b in_seq=0 out_seq=0 2024-03-21T10:32:35.288 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:35.287+0000 7f1ba2000640 10 client.17937 ms_handle_connect on v2:172.21.15.100:6836/3497644162 2024-03-21T10:32:35.288 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:35.287+0000 7f1ba2000640 1 -- 172.21.15.47:0/1078876940 <== mds.1 v2:172.21.15.100:6836/3497644162 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x56458a9c0b50 con 0x56458aacc000 2024-03-21T10:32:35.288 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:35.287+0000 7f1ba2000640 10 client.17937 handle_command_reply: tid=0 2024-03-21T10:32:35.288 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:35.287+0000 7f1bc6222740 1 client.17937 shutdown 2024-03-21T10:32:35.288 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:35.287+0000 7f1bc6222740 1 -- 172.21.15.47:0/1078876940 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56458aa5ede0 msgr2=0x56458aaca460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:32:35.288 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:35.287+0000 7f1bc6222740 1 --2- 172.21.15.47:0/1078876940 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56458aa5ede0 0x56458aaca460 secure :-1 s=READY pgs=1070 cs=0 l=1 rev1=1 crypto rx=0x7f1bb4009ee0 tx=0x7f1bb4009c50 comp rx=0 tx=0).stop 2024-03-21T10:32:35.288 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:35.287+0000 7f1bc6222740 1 -- 172.21.15.47:0/1078876940 shutdown_connections 2024-03-21T10:32:35.288 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:35.287+0000 7f1bc6222740 1 --2- 172.21.15.47:0/1078876940 >> [v2:172.21.15.100:6836/3497644162,v1:172.21.15.100:6837/3497644162] conn(0x56458aacc000 0x56458aace420 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:32:35.288 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:35.287+0000 7f1bc6222740 1 --2- 172.21.15.47:0/1078876940 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56458aa5ede0 0x56458aaca460 unknown :-1 s=CLOSED pgs=1070 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:32:35.288 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:35.287+0000 7f1bc6222740 1 --2- 172.21.15.47:0/1078876940 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56458aa56260 0x56458aa660c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:32:35.289 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:35.287+0000 7f1bc6222740 1 --2- 172.21.15.47:0/1078876940 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56458aa558b0 0x56458aa65b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:32:35.289 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:35.287+0000 7f1bc6222740 1 -- 172.21.15.47:0/1078876940 >> 172.21.15.47:0/1078876940 conn(0x56458aa4ff80 msgr2=0x56458aa51280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:32:35.289 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:35.287+0000 7f1ba2000640 0 client.17937 ms_handle_reset on v2:172.21.15.100:6836/3497644162 2024-03-21T10:32:35.289 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:35.287+0000 7f1bc6222740 1 -- 172.21.15.47:0/1078876940 shutdown_connections 2024-03-21T10:32:35.289 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:35.288+0000 7f1bc6222740 1 -- 172.21.15.47:0/1078876940 wait complete. 2024-03-21T10:32:35.289 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:35.288+0000 7f1bc6222740 20 client.17937 trim_cache size 0 max 16384 2024-03-21T10:32:35.289 INFO:teuthology.orchestra.run.smithi047.stdout:[] 2024-03-21T10:32:35.290 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:35.288+0000 7f1bc2000640 1 -- 172.21.15.47:0/2829549149 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f1ba40a2fa0 msgr2=0x7f1ba40a5460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:32:35.290 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:35.288+0000 7f1bc2000640 1 --2- 172.21.15.47:0/2829549149 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f1ba40a2fa0 0x7f1ba40a5460 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f1ba8036040 tx=0x7f1ba8005a20 comp rx=0 tx=0).stop 2024-03-21T10:32:35.290 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:35.288+0000 7f1bc2000640 1 -- 172.21.15.47:0/2829549149 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1bbc1040e0 msgr2=0x7f1bbc19a0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:32:35.290 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:35.288+0000 7f1bc2000640 1 --2- 172.21.15.47:0/2829549149 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1bbc1040e0 0x7f1bbc19a0d0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f1bb000e9f0 tx=0x7f1bb000eeb0 comp rx=0 tx=0).stop 2024-03-21T10:32:35.290 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:35.289+0000 7f1bc2000640 1 -- 172.21.15.47:0/2829549149 shutdown_connections 2024-03-21T10:32:35.290 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:35.289+0000 7f1bc2000640 1 --2- 172.21.15.47:0/2829549149 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1bbc19e990 0x7f1bbc19ee40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:32:35.290 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:35.289+0000 7f1bc2000640 1 --2- 172.21.15.47:0/2829549149 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1bbc109730 0x7f1bbc19a610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:32:35.290 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:35.289+0000 7f1bc2000640 1 --2- 172.21.15.47:0/2829549149 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f1ba40a2fa0 0x7f1ba40a5460 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:32:35.290 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:35.289+0000 7f1bc2000640 1 --2- 172.21.15.47:0/2829549149 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1bbc1040e0 0x7f1bbc19a0d0 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:32:35.291 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:35.289+0000 7f1bc2000640 1 -- 172.21.15.47:0/2829549149 >> 172.21.15.47:0/2829549149 conn(0x7f1bbc0ff1b0 msgr2=0x7f1bbc0ffa50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:32:35.291 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:35.289+0000 7f1bc2000640 1 -- 172.21.15.47:0/2829549149 shutdown_connections 2024-03-21T10:32:35.291 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:35.289+0000 7f1bc2000640 1 -- 172.21.15.47:0/2829549149 wait complete. 2024-03-21T10:32:35.302 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:32:35 smithi047 ceph-mon[31547]: pgmap v223: 129 pgs: 129 active+clean; 277 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-21T10:32:35.306 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-21T10:32:35.306 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-03-21T10:32:35.529 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:35.530+0000 7fa4ef200640 1 -- 172.21.15.47:0/865622751 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa4e81075e0 msgr2=0x7fa4e81079c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:32:35.529 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:35.530+0000 7fa4ef200640 1 --2- 172.21.15.47:0/865622751 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa4e81075e0 0x7fa4e81079c0 secure :-1 s=READY pgs=1071 cs=0 l=1 rev1=1 crypto rx=0x7fa4dc009900 tx=0x7fa4dc02bc40 comp rx=0 tx=0).stop 2024-03-21T10:32:35.529 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:35.530+0000 7fa4ef200640 1 -- 172.21.15.47:0/865622751 shutdown_connections 2024-03-21T10:32:35.529 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:35.530+0000 7fa4ef200640 1 --2- 172.21.15.47:0/865622751 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa4e8101ee0 0x7fa4e810e3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:32:35.529 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:35.530+0000 7fa4ef200640 1 --2- 172.21.15.47:0/865622751 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa4e8101520 0x7fa4e81019a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:32:35.529 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:35.530+0000 7fa4ef200640 1 --2- 172.21.15.47:0/865622751 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa4e81075e0 0x7fa4e81079c0 unknown :-1 s=CLOSED pgs=1071 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:32:35.529 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:35.530+0000 7fa4ef200640 1 -- 172.21.15.47:0/865622751 >> 172.21.15.47:0/865622751 conn(0x7fa4e80fd010 msgr2=0x7fa4e80ff430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:32:35.530 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:35.530+0000 7fa4ef200640 1 -- 172.21.15.47:0/865622751 shutdown_connections 2024-03-21T10:32:35.530 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:35.530+0000 7fa4ef200640 1 -- 172.21.15.47:0/865622751 wait complete. 2024-03-21T10:32:35.535 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:35.531+0000 7fa4ef200640 1 Processor -- start 2024-03-21T10:32:35.535 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:35.532+0000 7fa4ef200640 1 -- start start 2024-03-21T10:32:35.535 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:35.532+0000 7fa4ef200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa4e8101520 0x7fa4e8197d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:32:35.535 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:35.532+0000 7fa4ef200640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa4e8101ee0 0x7fa4e81982c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:32:35.535 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:35.532+0000 7fa4ef200640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa4e81075e0 0x7fa4e819c640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:32:35.535 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:35.532+0000 7fa4ef200640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4e8111820 con 0x7fa4e81075e0 2024-03-21T10:32:35.535 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:35.532+0000 7fa4ef200640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4e81116a0 con 0x7fa4e8101520 2024-03-21T10:32:35.535 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:35.533+0000 7fa4ef200640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4e81119a0 con 0x7fa4e8101ee0 2024-03-21T10:32:35.535 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:35.533+0000 7fa4ede00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa4e8101520 0x7fa4e8197d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:32:35.536 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:35.533+0000 7fa4ee800640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa4e81075e0 0x7fa4e819c640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:32:35.536 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:35.533+0000 7fa4ede00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa4e8101520 0x7fa4e8197d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:57516/0 (socket says 172.21.15.47:57516) 2024-03-21T10:32:35.536 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:35.533+0000 7fa4ede00640 1 -- 172.21.15.47:0/4060203130 learned_addr learned my addr 172.21.15.47:0/4060203130 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:32:35.536 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:35.533+0000 7fa4ed400640 1 --2- 172.21.15.47:0/4060203130 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa4e8101ee0 0x7fa4e81982c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:32:35.536 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:35.533+0000 7fa4ede00640 1 -- 172.21.15.47:0/4060203130 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa4e8101ee0 msgr2=0x7fa4e81982c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:32:35.536 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:35.533+0000 7fa4ede00640 1 --2- 172.21.15.47:0/4060203130 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa4e8101ee0 0x7fa4e81982c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:32:35.536 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:35.533+0000 7fa4ede00640 1 -- 172.21.15.47:0/4060203130 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa4e81075e0 msgr2=0x7fa4e819c640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:32:35.536 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:35.533+0000 7fa4ede00640 1 --2- 172.21.15.47:0/4060203130 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa4e81075e0 0x7fa4e819c640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:32:35.536 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:35.533+0000 7fa4ede00640 1 -- 172.21.15.47:0/4060203130 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa4e819cd70 con 0x7fa4e8101520 2024-03-21T10:32:35.536 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:35.534+0000 7fa4ede00640 1 --2- 172.21.15.47:0/4060203130 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa4e8101520 0x7fa4e8197d80 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fa4dc000c00 tx=0x7fa4dc0045e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:32:35.536 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:35.534+0000 7fa4e2a00640 1 -- 172.21.15.47:0/4060203130 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4dc043020 con 0x7fa4e8101520 2024-03-21T10:32:35.536 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:35.534+0000 7fa4ef200640 1 -- 172.21.15.47:0/4060203130 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa4e819d000 con 0x7fa4e8101520 2024-03-21T10:32:35.536 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:35.534+0000 7fa4e2a00640 1 -- 172.21.15.47:0/4060203130 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fa4dc02cbe0 con 0x7fa4e8101520 2024-03-21T10:32:35.536 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:35.534+0000 7fa4e2a00640 1 -- 172.21.15.47:0/4060203130 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4dc003bf0 con 0x7fa4e8101520 2024-03-21T10:32:35.537 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:35.534+0000 7fa4ef200640 1 -- 172.21.15.47:0/4060203130 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa4e81a48f0 con 0x7fa4e8101520 2024-03-21T10:32:35.542 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:35.537+0000 7fa4f359e740 1 Processor -- start 2024-03-21T10:32:35.542 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:35.538+0000 7fa4f359e740 1 -- start start 2024-03-21T10:32:35.542 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:35.538+0000 7fa4f359e740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561172a18630 0x561172a18a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:32:35.542 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:35.538+0000 7fa4f359e740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561172a18fe0 0x561172a25c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:32:35.542 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:35.538+0000 7fa4f359e740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561172a261f0 0x561172a28600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:32:35.543 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:35.538+0000 7fa4f359e740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x56117297c2c0 con 0x561172a261f0 2024-03-21T10:32:35.543 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:35.538+0000 7fa4f359e740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x561172924c90 con 0x561172a18630 2024-03-21T10:32:35.543 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:35.538+0000 7fa4f359e740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x561172978360 con 0x561172a18fe0 2024-03-21T10:32:35.543 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:35.538+0000 7fa4ed400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561172a261f0 0x561172a28600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:32:35.543 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:35.538+0000 7fa4ed400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561172a261f0 0x561172a28600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:51348/0 (socket says 172.21.15.47:51348) 2024-03-21T10:32:35.543 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:35.538+0000 7fa4ed400640 1 -- 172.21.15.47:0/3544356825 learned_addr learned my addr 172.21.15.47:0/3544356825 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:32:35.543 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:35.538+0000 7fa4ee800640 1 --2- 172.21.15.47:0/3544356825 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561172a18630 0x561172a18a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:32:35.543 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:35.539+0000 7fa4ed400640 1 -- 172.21.15.47:0/3544356825 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561172a18fe0 msgr2=0x561172a25c20 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-21T10:32:35.543 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:35.539+0000 7fa4ed400640 1 --2- 172.21.15.47:0/3544356825 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561172a18fe0 0x561172a25c20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:32:35.543 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:35.539+0000 7fa4ed400640 1 -- 172.21.15.47:0/3544356825 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561172a18630 msgr2=0x561172a18a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:32:35.543 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:35.539+0000 7fa4ed400640 1 --2- 172.21.15.47:0/3544356825 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561172a18630 0x561172a18a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:32:35.543 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:35.539+0000 7fa4ed400640 1 -- 172.21.15.47:0/3544356825 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561172949080 con 0x561172a261f0 2024-03-21T10:32:35.543 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:35.539+0000 7fa4ee800640 1 --2- 172.21.15.47:0/3544356825 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561172a18630 0x561172a18a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T10:32:35.543 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:35.539+0000 7fa4e2a00640 1 -- 172.21.15.47:0/4060203130 <== mon.2 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 27) v1 ==== 137985+0+0 (secure 0 0 0) 0x7fa4dc046020 con 0x7fa4e8101520 2024-03-21T10:32:35.544 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:35.539+0000 7fa4ed400640 1 --2- 172.21.15.47:0/3544356825 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561172a261f0 0x561172a28600 secure :-1 s=READY pgs=1072 cs=0 l=1 rev1=1 crypto rx=0x7fa4d8009c20 tx=0x7fa4d80117b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b98be65939532de3 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:32:35.544 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:35.539+0000 7fa4d7000640 1 -- 172.21.15.47:0/3544356825 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4d8012450 con 0x561172a261f0 2024-03-21T10:32:35.544 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:35.539+0000 7fa4d7000640 1 -- 172.21.15.47:0/3544356825 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fa4d8012aa0 con 0x561172a261f0 2024-03-21T10:32:35.544 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:35.539+0000 7fa4d7000640 1 -- 172.21.15.47:0/3544356825 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4d800fab0 con 0x561172a261f0 2024-03-21T10:32:35.544 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:35.540+0000 7fa4f359e740 1 -- 172.21.15.47:0/3544356825 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561172a261f0 msgr2=0x561172a28600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:32:35.544 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:35.540+0000 7fa4f359e740 1 --2- 172.21.15.47:0/3544356825 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561172a261f0 0x561172a28600 secure :-1 s=READY pgs=1072 cs=0 l=1 rev1=1 crypto rx=0x7fa4d8009c20 tx=0x7fa4d80117b0 comp rx=0 tx=0).stop 2024-03-21T10:32:35.544 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:35.540+0000 7fa4f359e740 1 -- 172.21.15.47:0/3544356825 shutdown_connections 2024-03-21T10:32:35.544 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:35.540+0000 7fa4f359e740 1 --2- 172.21.15.47:0/3544356825 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561172a261f0 0x561172a28600 unknown :-1 s=CLOSED pgs=1072 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:32:35.544 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:35.540+0000 7fa4f359e740 1 --2- 172.21.15.47:0/3544356825 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561172a18fe0 0x561172a25c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:32:35.544 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:35.540+0000 7fa4f359e740 1 --2- 172.21.15.47:0/3544356825 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561172a18630 0x561172a18a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:32:35.544 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:35.540+0000 7fa4f359e740 1 -- 172.21.15.47:0/3544356825 >> 172.21.15.47:0/3544356825 conn(0x561172a12f80 msgr2=0x561172a13360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:32:35.544 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:35.540+0000 7fa4f359e740 1 -- 172.21.15.47:0/3544356825 shutdown_connections 2024-03-21T10:32:35.544 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:35.540+0000 7fa4f359e740 1 -- 172.21.15.47:0/3544356825 wait complete. 2024-03-21T10:32:35.544 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:35.540+0000 7fa4e2a00640 1 --2- 172.21.15.47:0/4060203130 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7fa4d00a2fa0 0x7fa4d00a5460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:32:35.545 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:35.541+0000 7fa4e2a00640 1 -- 172.21.15.47:0/4060203130 <== mon.2 v2:172.21.15.100:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 9562+0+0 (secure 0 0 0) 0x7fa4dc0f1d70 con 0x7fa4e8101520 2024-03-21T10:32:35.545 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:35.541+0000 7fa4f359e740 1 -- start start 2024-03-21T10:32:35.545 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:35.541+0000 7fa4ed400640 1 --2- 172.21.15.47:0/4060203130 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7fa4d00a2fa0 0x7fa4d00a5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:32:35.545 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:35.541+0000 7fa4f359e740 1 Processor -- start 2024-03-21T10:32:35.545 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:35.542+0000 7fa4f359e740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561172a18630 0x561172a25ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:32:35.545 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:35.542+0000 7fa4ed400640 1 --2- 172.21.15.47:0/4060203130 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7fa4d00a2fa0 0x7fa4d00a5460 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fa4d800f7f0 tx=0x7fa4d8000bc0 comp rx=0 tx=0).ready entity=mgr.15687 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:32:35.545 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:35.542+0000 7fa4f359e740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561172a18fe0 0x561172a22e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:32:35.545 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:35.542+0000 7fa4f359e740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561172a261f0 0x561172a233a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:32:35.545 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:35.542+0000 7fa4f359e740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x561172a29170 con 0x561172a261f0 2024-03-21T10:32:35.545 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:35.542+0000 7fa4f359e740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x5611729380d0 con 0x561172a18fe0 2024-03-21T10:32:35.545 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:35.542+0000 7fa4f359e740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x561172a292f0 con 0x561172a18630 2024-03-21T10:32:35.545 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:35.542+0000 7fa4f359e740 10 client.0 _ll_register_callbacks cb 0x56117293ce80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T10:32:35.545 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:35.542+0000 7fa4ee800640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561172a261f0 0x561172a233a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:32:35.546 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:35.542+0000 7fa4ee800640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561172a261f0 0x561172a233a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:51358/0 (socket says 172.21.15.47:51358) 2024-03-21T10:32:35.546 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:35.542+0000 7fa4ee800640 1 -- 172.21.15.47:0/2994043537 learned_addr learned my addr 172.21.15.47:0/2994043537 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:32:35.546 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:35.542+0000 7fa4ed400640 1 --2- 172.21.15.47:0/2994043537 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561172a18fe0 0x561172a22e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:32:35.546 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:35.542+0000 7fa4ede00640 1 --2- 172.21.15.47:0/2994043537 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561172a18630 0x561172a25ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:32:35.546 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:35.543+0000 7fa4ee800640 1 -- 172.21.15.47:0/2994043537 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561172a18630 msgr2=0x561172a25ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:32:35.546 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:35.543+0000 7fa4ee800640 1 --2- 172.21.15.47:0/2994043537 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561172a18630 0x561172a25ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:32:35.546 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:35.543+0000 7fa4ee800640 1 -- 172.21.15.47:0/2994043537 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561172a18fe0 msgr2=0x561172a22e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:32:35.546 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:35.543+0000 7fa4ee800640 1 --2- 172.21.15.47:0/2994043537 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561172a18fe0 0x561172a22e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:32:35.546 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:35.543+0000 7fa4ee800640 1 -- 172.21.15.47:0/2994043537 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5611726903f0 con 0x561172a261f0 2024-03-21T10:32:35.546 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:35.543+0000 7fa4ede00640 1 --2- 172.21.15.47:0/2994043537 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561172a18630 0x561172a25ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T10:32:35.546 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:35.543+0000 7fa4ee800640 1 --2- 172.21.15.47:0/2994043537 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561172a261f0 0x561172a233a0 secure :-1 s=READY pgs=1073 cs=0 l=1 rev1=1 crypto rx=0x7fa4e400ae60 tx=0x7fa4e4007680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a6fd4670604464da server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:32:35.546 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:35.543+0000 7fa4cb400640 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-03-21T10:32:35.546 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:35.543+0000 7fa4cb400640 1 -- 172.21.15.47:0/2994043537 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4e4016610 con 0x561172a261f0 2024-03-21T10:32:35.546 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:35.543+0000 7fa4f359e740 1 -- 172.21.15.47:0/2994043537 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fa4d8012aa0 con 0x561172a261f0 2024-03-21T10:32:35.547 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:35.543+0000 7fa4cb400640 1 -- 172.21.15.47:0/2994043537 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(99..99 src has 1..99) v4 ==== 9562+0+0 (secure 0 0 0) 0x7fa4e40167b0 con 0x561172a261f0 2024-03-21T10:32:35.547 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:35.544+0000 7fa4cb400640 1 -- 172.21.15.47:0/2994043537 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa4e40169f0 con 0x561172a261f0 2024-03-21T10:32:35.547 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:35.544+0000 7fa4f359e740 10 client.17949 fetch_fsmap learned FSMap version 32 2024-03-21T10:32:35.547 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:35.544+0000 7fa4f359e740 1 -- 172.21.15.47:0/2994043537 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x56117275a7d0 con 0x561172a261f0 2024-03-21T10:32:35.549 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:35.544+0000 7fa4cb400640 1 -- 172.21.15.47:0/2994043537 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fa4d8012aa0 con 0x561172a261f0 2024-03-21T10:32:35.549 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:35.544+0000 7fa4cb400640 1 -- 172.21.15.47:0/2994043537 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 32) v1 ==== 7942+0+0 (secure 0 0 0) 0x7fa4e4020c70 con 0x561172a261f0 2024-03-21T10:32:35.549 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:35.544+0000 7fa4f359e740 10 client.17949 fetch_fsmap finished waiting for FSMap version 32 2024-03-21T10:32:35.549 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:35.544+0000 7fa4f359e740 10 client.17949 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-21T10:32:35.549 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:35.544+0000 7fa4f359e740 20 client.17949 populate_metadata read hostname 'smithi047' 2024-03-21T10:32:35.549 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:35.544+0000 7fa4f359e740 1 --2- 172.21.15.47:0/2994043537 >> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] conn(0x561172a360b0 0x561172a384d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:32:35.549 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:35.544+0000 7fa4f359e740 4 client.17949 mds_command: new command op to 34535 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T10:32:35.549 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:35.544+0000 7fa4f359e740 1 -- 172.21.15.47:0/2994043537 --> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x561172989800 con 0x561172a360b0 2024-03-21T10:32:35.550 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:35.545+0000 7fa4ede00640 1 --2- 172.21.15.47:0/2994043537 >> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] conn(0x561172a360b0 0x561172a384d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:32:35.550 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:35.546+0000 7fa4cb400640 10 client.17949 ms_handle_connect on v2:172.21.15.100:6838/4173168440 2024-03-21T10:32:35.550 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:35.545+0000 7fa4ede00640 1 --2- 172.21.15.47:0/2994043537 >> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] conn(0x561172a360b0 0x561172a384d0 crc :-1 s=READY pgs=177 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=46e2debb958fa402 server_cookie=74371d39cbe6abd in_seq=0 out_seq=0 2024-03-21T10:32:35.550 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:35.547+0000 7fa4cb400640 1 -- 172.21.15.47:0/2994043537 <== mds.0 v2:172.21.15.100:6838/4173168440 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x561172989800 con 0x561172a360b0 2024-03-21T10:32:35.550 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:35.547+0000 7fa4cb400640 10 client.17949 handle_command_reply: tid=0 2024-03-21T10:32:35.550 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:35.547+0000 7fa4f359e740 1 client.17949 shutdown 2024-03-21T10:32:35.550 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:35.547+0000 7fa4f359e740 1 -- 172.21.15.47:0/2994043537 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561172a261f0 msgr2=0x561172a233a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:32:35.550 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:35.547+0000 7fa4f359e740 1 --2- 172.21.15.47:0/2994043537 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561172a261f0 0x561172a233a0 secure :-1 s=READY pgs=1073 cs=0 l=1 rev1=1 crypto rx=0x7fa4e400ae60 tx=0x7fa4e4007680 comp rx=0 tx=0).stop 2024-03-21T10:32:35.550 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:35.547+0000 7fa4f359e740 1 -- 172.21.15.47:0/2994043537 shutdown_connections 2024-03-21T10:32:35.550 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:35.547+0000 7fa4f359e740 1 --2- 172.21.15.47:0/2994043537 >> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] conn(0x561172a360b0 0x561172a384d0 crc :-1 s=READY pgs=177 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:32:35.550 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:35.547+0000 7fa4f359e740 1 --2- 172.21.15.47:0/2994043537 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561172a261f0 0x561172a233a0 unknown :-1 s=CLOSED pgs=1073 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:32:35.550 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:35.547+0000 7fa4f359e740 1 --2- 172.21.15.47:0/2994043537 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561172a18fe0 0x561172a22e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:32:35.550 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:35.547+0000 7fa4f359e740 1 --2- 172.21.15.47:0/2994043537 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561172a18630 0x561172a25ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:32:35.550 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:35.547+0000 7fa4f359e740 1 -- 172.21.15.47:0/2994043537 >> 172.21.15.47:0/2994043537 conn(0x561172a12f80 msgr2=0x561172a14cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:32:35.551 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:35.548+0000 7fa4cb400640 0 client.17949 ms_handle_reset on v2:172.21.15.100:6838/4173168440 2024-03-21T10:32:35.551 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:35.548+0000 7fa4f359e740 1 -- 172.21.15.47:0/2994043537 shutdown_connections 2024-03-21T10:32:35.551 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:35.548+0000 7fa4f359e740 1 -- 172.21.15.47:0/2994043537 wait complete. 2024-03-21T10:32:35.551 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:35.548+0000 7fa4f359e740 20 client.17949 trim_cache size 0 max 16384 2024-03-21T10:32:35.561 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:35.562+0000 7fa4f359e740 1 Processor -- start 2024-03-21T10:32:35.561 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:35.562+0000 7fa4f359e740 1 -- start start 2024-03-21T10:32:35.561 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:35.562+0000 7fa4f359e740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561172a18630 0x561172a25590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:32:35.561 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:35.562+0000 7fa4f359e740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561172a18fe0 0x561172a25ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:32:35.561 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:35.562+0000 7fa4f359e740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561172a261f0 0x561172a200e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:32:35.561 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:35.562+0000 7fa4f359e740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4e81ac670 con 0x561172a18fe0 2024-03-21T10:32:35.561 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:35.562+0000 7fa4f359e740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4e81adff0 con 0x561172a261f0 2024-03-21T10:32:35.561 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:35.562+0000 7fa4f359e740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4e81accd0 con 0x561172a18630 2024-03-21T10:32:35.561 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:35.562+0000 7fa4ed400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561172a18fe0 0x561172a25ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:32:35.562 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:35.562+0000 7fa4ed400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561172a18fe0 0x561172a25ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:51362/0 (socket says 172.21.15.47:51362) 2024-03-21T10:32:35.562 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:35.563+0000 7fa4ed400640 1 -- 172.21.15.47:0/2521201468 learned_addr learned my addr 172.21.15.47:0/2521201468 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:32:35.562 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:35.563+0000 7fa4ee800640 1 --2- 172.21.15.47:0/2521201468 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561172a261f0 0x561172a200e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:32:35.563 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:35.563+0000 7fa4ede00640 1 --2- 172.21.15.47:0/2521201468 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561172a18630 0x561172a25590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:32:35.563 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:35.563+0000 7fa4ed400640 1 -- 172.21.15.47:0/2521201468 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561172a18630 msgr2=0x561172a25590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:32:35.563 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:35.563+0000 7fa4ed400640 1 --2- 172.21.15.47:0/2521201468 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561172a18630 0x561172a25590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:32:35.564 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:35.563+0000 7fa4ed400640 1 -- 172.21.15.47:0/2521201468 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561172a261f0 msgr2=0x561172a200e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:32:35.564 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:35.563+0000 7fa4ed400640 1 --2- 172.21.15.47:0/2521201468 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561172a261f0 0x561172a200e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:32:35.564 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:35.563+0000 7fa4ed400640 1 -- 172.21.15.47:0/2521201468 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561172a17940 con 0x561172a18fe0 2024-03-21T10:32:35.564 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:35.563+0000 7fa4ed400640 1 --2- 172.21.15.47:0/2521201468 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561172a18fe0 0x561172a25ad0 secure :-1 s=READY pgs=1074 cs=0 l=1 rev1=1 crypto rx=0x7fa4d8009800 tx=0x7fa4d8014070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cb096d7683c0a882 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:32:35.564 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:35.564+0000 7fa4d6600640 1 -- 172.21.15.47:0/2521201468 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4d8034070 con 0x561172a18fe0 2024-03-21T10:32:35.564 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:35.564+0000 7fa4d6600640 1 -- 172.21.15.47:0/2521201468 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fa4d8028070 con 0x561172a18fe0 2024-03-21T10:32:35.564 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:35.564+0000 7fa4d6600640 1 -- 172.21.15.47:0/2521201468 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4d8003830 con 0x561172a18fe0 2024-03-21T10:32:35.564 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:35.564+0000 7fa4f359e740 1 -- 172.21.15.47:0/2521201468 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561172a18fe0 msgr2=0x561172a25ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:32:35.564 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:35.564+0000 7fa4f359e740 1 --2- 172.21.15.47:0/2521201468 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561172a18fe0 0x561172a25ad0 secure :-1 s=READY pgs=1074 cs=0 l=1 rev1=1 crypto rx=0x7fa4d8009800 tx=0x7fa4d8014070 comp rx=0 tx=0).stop 2024-03-21T10:32:35.564 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:35.564+0000 7fa4f359e740 1 -- 172.21.15.47:0/2521201468 shutdown_connections 2024-03-21T10:32:35.564 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:35.564+0000 7fa4f359e740 1 --2- 172.21.15.47:0/2521201468 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561172a261f0 0x561172a200e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:32:35.564 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:35.564+0000 7fa4f359e740 1 --2- 172.21.15.47:0/2521201468 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561172a18fe0 0x561172a25ad0 unknown :-1 s=CLOSED pgs=1074 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:32:35.564 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:35.564+0000 7fa4f359e740 1 --2- 172.21.15.47:0/2521201468 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561172a18630 0x561172a25590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:32:35.564 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:35.564+0000 7fa4f359e740 1 -- 172.21.15.47:0/2521201468 >> 172.21.15.47:0/2521201468 conn(0x561172a12f80 msgr2=0x561172a1e450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:32:35.565 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:35.564+0000 7fa4f359e740 1 -- 172.21.15.47:0/2521201468 shutdown_connections 2024-03-21T10:32:35.565 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:35.565+0000 7fa4f359e740 1 -- 172.21.15.47:0/2521201468 wait complete. 2024-03-21T10:32:35.565 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:35.565+0000 7fa4f359e740 1 -- start start 2024-03-21T10:32:35.565 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:35.565+0000 7fa4f359e740 1 Processor -- start 2024-03-21T10:32:35.567 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:35.565+0000 7fa4f359e740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561172a18630 0x561172a2e200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:32:35.567 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:35.566+0000 7fa4f359e740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561172a18fe0 0x561172a2a5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:32:35.567 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:35.566+0000 7fa4f359e740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561172a261f0 0x561172a2ab10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:32:35.568 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:35.566+0000 7fa4f359e740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4e81ad990 con 0x561172a261f0 2024-03-21T10:32:35.568 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:35.566+0000 7fa4f359e740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4e81ad330 con 0x561172a18fe0 2024-03-21T10:32:35.568 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:35.566+0000 7fa4f359e740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4e81aa850 con 0x561172a18630 2024-03-21T10:32:35.568 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:35.566+0000 7fa4f359e740 10 client.0 _ll_register_callbacks cb 0x7fa4e4009b90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T10:32:35.568 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:35.566+0000 7fa4ee800640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561172a261f0 0x561172a2ab10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:32:35.568 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:35.566+0000 7fa4ee800640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561172a261f0 0x561172a2ab10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:51364/0 (socket says 172.21.15.47:51364) 2024-03-21T10:32:35.568 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:35.566+0000 7fa4ee800640 1 -- 172.21.15.47:0/711005 learned_addr learned my addr 172.21.15.47:0/711005 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:32:35.568 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:35.566+0000 7fa4ed400640 1 --2- 172.21.15.47:0/711005 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561172a18fe0 0x561172a2a5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:32:35.568 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:35.566+0000 7fa4ede00640 1 --2- 172.21.15.47:0/711005 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561172a18630 0x561172a2e200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:32:35.568 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:35.566+0000 7fa4ee800640 1 -- 172.21.15.47:0/711005 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561172a18630 msgr2=0x561172a2e200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:32:35.568 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:35.566+0000 7fa4ee800640 1 --2- 172.21.15.47:0/711005 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561172a18630 0x561172a2e200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:32:35.568 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:35.566+0000 7fa4ee800640 1 -- 172.21.15.47:0/711005 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561172a18fe0 msgr2=0x561172a2a5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:32:35.568 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:35.566+0000 7fa4ee800640 1 --2- 172.21.15.47:0/711005 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561172a18fe0 0x561172a2a5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:32:35.569 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:35.566+0000 7fa4ee800640 1 -- 172.21.15.47:0/711005 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x561172a23980 con 0x561172a261f0 2024-03-21T10:32:35.569 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:35.567+0000 7fa4ee800640 1 --2- 172.21.15.47:0/711005 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561172a261f0 0x561172a2ab10 secure :-1 s=READY pgs=1075 cs=0 l=1 rev1=1 crypto rx=0x7fa4e40070b0 tx=0x7fa4e4009650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8a39246fb22bf00e server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:32:35.569 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:35.567+0000 7fa4cb400640 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-03-21T10:32:35.569 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:35.567+0000 7fa4cb400640 1 -- 172.21.15.47:0/711005 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4e402e070 con 0x561172a261f0 2024-03-21T10:32:35.569 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:35.567+0000 7fa4f359e740 1 -- 172.21.15.47:0/711005 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fa4d8028070 con 0x561172a261f0 2024-03-21T10:32:35.569 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:35.567+0000 7fa4cb400640 1 -- 172.21.15.47:0/711005 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(99..99 src has 1..99) v4 ==== 9562+0+0 (secure 0 0 0) 0x7fa4e4020660 con 0x561172a261f0 2024-03-21T10:32:35.569 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:35.567+0000 7fa4cb400640 1 -- 172.21.15.47:0/711005 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa4e4007b80 con 0x561172a261f0 2024-03-21T10:32:35.569 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:35.568+0000 7fa4f359e740 10 client.17955 fetch_fsmap learned FSMap version 32 2024-03-21T10:32:35.569 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:35.568+0000 7fa4f359e740 1 -- 172.21.15.47:0/711005 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x561172a92680 con 0x561172a261f0 2024-03-21T10:32:35.569 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:35.568+0000 7fa4cb400640 1 -- 172.21.15.47:0/711005 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fa4d8028070 con 0x561172a261f0 2024-03-21T10:32:35.569 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:35.568+0000 7fa4cb400640 1 -- 172.21.15.47:0/711005 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 32) v1 ==== 7942+0+0 (secure 0 0 0) 0x7fa4e4015ca0 con 0x561172a261f0 2024-03-21T10:32:35.569 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:35.568+0000 7fa4f359e740 10 client.17955 fetch_fsmap finished waiting for FSMap version 32 2024-03-21T10:32:35.569 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:35.568+0000 7fa4f359e740 10 client.17955 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-21T10:32:35.569 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:35.568+0000 7fa4f359e740 20 client.17955 populate_metadata read hostname 'smithi047' 2024-03-21T10:32:35.570 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:35.568+0000 7fa4f359e740 1 --2- 172.21.15.47:0/711005 >> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] conn(0x561172a92da0 0x561172a951c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:32:35.570 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:35.568+0000 7fa4f359e740 4 client.17955 mds_command: new command op to 34535 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-21T10:32:35.570 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:35.568+0000 7fa4f359e740 1 -- 172.21.15.47:0/711005 --> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x561172983b50 con 0x561172a92da0 2024-03-21T10:32:35.570 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:35.569+0000 7fa4ede00640 1 --2- 172.21.15.47:0/711005 >> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] conn(0x561172a92da0 0x561172a951c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:32:35.570 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:35.569+0000 7fa4ede00640 1 --2- 172.21.15.47:0/711005 >> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] conn(0x561172a92da0 0x561172a951c0 crc :-1 s=READY pgs=178 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4a782186ffe31902 server_cookie=1caeee4dce832b9f in_seq=0 out_seq=0 2024-03-21T10:32:35.570 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:35.569+0000 7fa4cb400640 10 client.17955 ms_handle_connect on v2:172.21.15.100:6838/4173168440 2024-03-21T10:32:35.571 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:35.569+0000 7fa4cb400640 1 -- 172.21.15.47:0/711005 <== mds.0 v2:172.21.15.100:6838/4173168440 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x561172983b50 con 0x561172a92da0 2024-03-21T10:32:35.571 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:35.569+0000 7fa4cb400640 10 client.17955 handle_command_reply: tid=0 2024-03-21T10:32:35.571 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:35.570+0000 7fa4f359e740 1 client.17955 shutdown 2024-03-21T10:32:35.571 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:35.570+0000 7fa4f359e740 1 -- 172.21.15.47:0/711005 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561172a261f0 msgr2=0x561172a2ab10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:32:35.571 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:35.570+0000 7fa4f359e740 1 --2- 172.21.15.47:0/711005 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561172a261f0 0x561172a2ab10 secure :-1 s=READY pgs=1075 cs=0 l=1 rev1=1 crypto rx=0x7fa4e40070b0 tx=0x7fa4e4009650 comp rx=0 tx=0).stop 2024-03-21T10:32:35.571 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:35.570+0000 7fa4f359e740 1 -- 172.21.15.47:0/711005 shutdown_connections 2024-03-21T10:32:35.571 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:35.570+0000 7fa4f359e740 1 --2- 172.21.15.47:0/711005 >> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] conn(0x561172a92da0 0x561172a951c0 crc :-1 s=READY pgs=178 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:32:35.571 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:35.570+0000 7fa4f359e740 1 --2- 172.21.15.47:0/711005 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561172a261f0 0x561172a2ab10 unknown :-1 s=CLOSED pgs=1075 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:32:35.572 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:35.570+0000 7fa4f359e740 1 --2- 172.21.15.47:0/711005 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561172a18fe0 0x561172a2a5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:32:35.572 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:35.570+0000 7fa4f359e740 1 --2- 172.21.15.47:0/711005 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x561172a18630 0x561172a2e200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:32:35.572 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:35.570+0000 7fa4cb400640 0 client.17955 ms_handle_reset on v2:172.21.15.100:6838/4173168440 2024-03-21T10:32:35.572 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:35.570+0000 7fa4f359e740 1 -- 172.21.15.47:0/711005 >> 172.21.15.47:0/711005 conn(0x561172a12f80 msgr2=0x561172a118d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:32:35.572 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:35.570+0000 7fa4f359e740 1 -- 172.21.15.47:0/711005 shutdown_connections 2024-03-21T10:32:35.572 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:35.570+0000 7fa4f359e740 1 -- 172.21.15.47:0/711005 wait complete. 2024-03-21T10:32:35.572 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:35.570+0000 7fa4f359e740 20 client.17955 trim_cache size 0 max 16384 2024-03-21T10:32:35.572 INFO:teuthology.orchestra.run.smithi047.stdout:[] 2024-03-21T10:32:35.573 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:35.571+0000 7fa4ef200640 1 -- 172.21.15.47:0/4060203130 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7fa4d00a2fa0 msgr2=0x7fa4d00a5460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:32:35.573 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:35.571+0000 7fa4ef200640 1 --2- 172.21.15.47:0/4060203130 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7fa4d00a2fa0 0x7fa4d00a5460 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fa4d800f7f0 tx=0x7fa4d8000bc0 comp rx=0 tx=0).stop 2024-03-21T10:32:35.573 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:35.571+0000 7fa4ef200640 1 -- 172.21.15.47:0/4060203130 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa4e8101520 msgr2=0x7fa4e8197d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:32:35.573 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:35.571+0000 7fa4ef200640 1 --2- 172.21.15.47:0/4060203130 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa4e8101520 0x7fa4e8197d80 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fa4dc000c00 tx=0x7fa4dc0045e0 comp rx=0 tx=0).stop 2024-03-21T10:32:35.573 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:35.571+0000 7fa4ef200640 1 -- 172.21.15.47:0/4060203130 shutdown_connections 2024-03-21T10:32:35.573 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:35.571+0000 7fa4ef200640 1 --2- 172.21.15.47:0/4060203130 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa4e81075e0 0x7fa4e819c640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:32:35.573 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:35.571+0000 7fa4ef200640 1 --2- 172.21.15.47:0/4060203130 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa4e8101ee0 0x7fa4e81982c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:32:35.573 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:35.571+0000 7fa4ef200640 1 --2- 172.21.15.47:0/4060203130 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7fa4d00a2fa0 0x7fa4d00a5460 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:32:35.573 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:35.571+0000 7fa4ef200640 1 --2- 172.21.15.47:0/4060203130 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa4e8101520 0x7fa4e8197d80 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:32:35.574 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:35.571+0000 7fa4ef200640 1 -- 172.21.15.47:0/4060203130 >> 172.21.15.47:0/4060203130 conn(0x7fa4e80fd010 msgr2=0x7fa4e80fe810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:32:35.574 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:35.571+0000 7fa4ef200640 1 -- 172.21.15.47:0/4060203130 shutdown_connections 2024-03-21T10:32:35.574 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:35.571+0000 7fa4ef200640 1 -- 172.21.15.47:0/4060203130 wait complete. 2024-03-21T10:32:35.727 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:32:35 smithi100 ceph-mon[35193]: pgmap v223: 129 pgs: 129 active+clean; 277 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-21T10:32:35.737 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:32:35 smithi157 ceph-mon[35581]: pgmap v223: 129 pgs: 129 active+clean; 277 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-21T10:32:36.586 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-03-21T10:32:36.587 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-21T10:32:36.587 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-03-21T10:32:36.812 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:36.813+0000 7f6b17200640 1 -- 172.21.15.47:0/3975800573 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6b18101520 msgr2=0x7f6b181019a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:32:36.812 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:36.813+0000 7f6b17200640 1 --2- 172.21.15.47:0/3975800573 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6b18101520 0x7f6b181019a0 secure :-1 s=READY pgs=1076 cs=0 l=1 rev1=1 crypto rx=0x7f6b00009900 tx=0x7f6b0002bc30 comp rx=0 tx=0).stop 2024-03-21T10:32:36.812 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:36.813+0000 7f6b17200640 1 -- 172.21.15.47:0/3975800573 shutdown_connections 2024-03-21T10:32:36.812 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:36.813+0000 7f6b17200640 1 --2- 172.21.15.47:0/3975800573 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6b18101ee0 0x7f6b1810e3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:32:36.812 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:36.813+0000 7f6b17200640 1 --2- 172.21.15.47:0/3975800573 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6b18101520 0x7f6b181019a0 unknown :-1 s=CLOSED pgs=1076 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:32:36.812 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:36.813+0000 7f6b17200640 1 --2- 172.21.15.47:0/3975800573 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6b181075e0 0x7f6b181079c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:32:36.812 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:36.813+0000 7f6b17200640 1 -- 172.21.15.47:0/3975800573 >> 172.21.15.47:0/3975800573 conn(0x7f6b180fd010 msgr2=0x7f6b180ff430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:32:36.812 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:36.813+0000 7f6b17200640 1 -- 172.21.15.47:0/3975800573 shutdown_connections 2024-03-21T10:32:36.812 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:36.813+0000 7f6b17200640 1 -- 172.21.15.47:0/3975800573 wait complete. 2024-03-21T10:32:36.814 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:36.814+0000 7f6b17200640 1 Processor -- start 2024-03-21T10:32:36.815 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:36.814+0000 7f6b17200640 1 -- start start 2024-03-21T10:32:36.815 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:36.814+0000 7f6b17200640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6b18101520 0x7f6b180739a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:32:36.815 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:36.814+0000 7f6b17200640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6b18101ee0 0x7f6b18073ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:32:36.815 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:36.814+0000 7f6b17200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6b181075e0 0x7f6b18074420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:32:36.815 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:36.815+0000 7f6b17200640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b18111750 con 0x7f6b18101ee0 2024-03-21T10:32:36.815 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:36.815+0000 7f6b17200640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b181115d0 con 0x7f6b181075e0 2024-03-21T10:32:36.815 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:36.815+0000 7f6b17200640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b181118d0 con 0x7f6b18101520 2024-03-21T10:32:36.815 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:36.815+0000 7f6b0fe00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6b18101ee0 0x7f6b18073ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:32:36.815 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:36.815+0000 7f6b0fe00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6b18101ee0 0x7f6b18073ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:51376/0 (socket says 172.21.15.47:51376) 2024-03-21T10:32:36.815 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:36.815+0000 7f6b0fe00640 1 -- 172.21.15.47:0/687598721 learned_addr learned my addr 172.21.15.47:0/687598721 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:32:36.815 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:36.815+0000 7f6b16800640 1 --2- 172.21.15.47:0/687598721 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6b181075e0 0x7f6b18074420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:32:36.815 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:36.815+0000 7f6b15e00640 1 --2- 172.21.15.47:0/687598721 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6b18101520 0x7f6b180739a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:32:36.815 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:36.815+0000 7f6b0fe00640 1 -- 172.21.15.47:0/687598721 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6b18101520 msgr2=0x7f6b180739a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:32:36.816 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:36.815+0000 7f6b0fe00640 1 --2- 172.21.15.47:0/687598721 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6b18101520 0x7f6b180739a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:32:36.816 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:36.815+0000 7f6b0fe00640 1 -- 172.21.15.47:0/687598721 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6b181075e0 msgr2=0x7f6b18074420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:32:36.816 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:36.815+0000 7f6b0fe00640 1 --2- 172.21.15.47:0/687598721 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6b181075e0 0x7f6b18074420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:32:36.816 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:36.815+0000 7f6b0fe00640 1 -- 172.21.15.47:0/687598721 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b181a61d0 con 0x7f6b18101ee0 2024-03-21T10:32:36.816 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:36.815+0000 7f6b16800640 1 --2- 172.21.15.47:0/687598721 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6b181075e0 0x7f6b18074420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T10:32:36.816 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:36.816+0000 7f6b0fe00640 1 --2- 172.21.15.47:0/687598721 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6b18101ee0 0x7f6b18073ee0 secure :-1 s=READY pgs=1077 cs=0 l=1 rev1=1 crypto rx=0x7f6b00006010 tx=0x7f6b00005e20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:32:36.816 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:36.816+0000 7f6b0ea00640 1 -- 172.21.15.47:0/687598721 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b00043020 con 0x7f6b18101ee0 2024-03-21T10:32:36.816 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:36.816+0000 7f6b0ea00640 1 -- 172.21.15.47:0/687598721 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f6b000043d0 con 0x7f6b18101ee0 2024-03-21T10:32:36.816 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:36.816+0000 7f6b17200640 1 -- 172.21.15.47:0/687598721 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6b181a6460 con 0x7f6b18101ee0 2024-03-21T10:32:36.816 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:36.816+0000 7f6b0ea00640 1 -- 172.21.15.47:0/687598721 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b00003a50 con 0x7f6b18101ee0 2024-03-21T10:32:36.816 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:36.816+0000 7f6b17200640 1 -- 172.21.15.47:0/687598721 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6b181a69b0 con 0x7f6b18101ee0 2024-03-21T10:32:36.818 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:36.820+0000 7f6b0ea00640 1 -- 172.21.15.47:0/687598721 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 27) v1 ==== 137985+0+0 (secure 0 0 0) 0x7f6b00046020 con 0x7f6b18101ee0 2024-03-21T10:32:36.821 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:36.821+0000 7f6b1fc45740 1 Processor -- start 2024-03-21T10:32:36.821 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:36.821+0000 7f6b1fc45740 1 -- start start 2024-03-21T10:32:36.821 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:36.821+0000 7f6b0ea00640 1 --2- 172.21.15.47:0/687598721 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f6afc0a3090 0x7f6afc0a5550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:32:36.821 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:36.821+0000 7f6b1fc45740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555ecebc2290 0x555ecebc2670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:32:36.822 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:36.821+0000 7f6b1fc45740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x555ecebc2bb0 0x555ecebcb250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:32:36.822 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:36.821+0000 7f6b1fc45740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555ecebcb790 0x555ecebcdb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:32:36.822 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:36.821+0000 7f6b1fc45740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x555eceb26eb0 con 0x555ecebc2bb0 2024-03-21T10:32:36.822 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:36.821+0000 7f6b1fc45740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x555eceacfa60 con 0x555ecebcb790 2024-03-21T10:32:36.822 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:36.821+0000 7f6b1fc45740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x555eceb22f50 con 0x555ecebc2290 2024-03-21T10:32:36.822 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:36.822+0000 7f6b0ea00640 1 -- 172.21.15.47:0/687598721 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 9562+0+0 (secure 0 0 0) 0x7f6b000f1640 con 0x7f6b18101ee0 2024-03-21T10:32:36.822 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:36.822+0000 7f6b0fe00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x555ecebc2bb0 0x555ecebcb250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:32:36.822 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:36.822+0000 7f6b0fe00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x555ecebc2bb0 0x555ecebcb250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:51388/0 (socket says 172.21.15.47:51388) 2024-03-21T10:32:36.822 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:36.822+0000 7f6b0fe00640 1 -- 172.21.15.47:0/3920076892 learned_addr learned my addr 172.21.15.47:0/3920076892 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:32:36.822 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:36.822+0000 7f6b16800640 1 --2- 172.21.15.47:0/687598721 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f6afc0a3090 0x7f6afc0a5550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:32:36.822 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:36.822+0000 7f6b16800640 1 --2- 172.21.15.47:0/3920076892 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555ecebcb790 0x555ecebcdb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:32:36.822 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:36.822+0000 7f6b15e00640 1 --2- 172.21.15.47:0/3920076892 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555ecebc2290 0x555ecebc2670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:32:36.823 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:36.822+0000 7f6b0fe00640 1 -- 172.21.15.47:0/3920076892 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555ecebc2290 msgr2=0x555ecebc2670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:32:36.823 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:36.822+0000 7f6b0fe00640 1 --2- 172.21.15.47:0/3920076892 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555ecebc2290 0x555ecebc2670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:32:36.823 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:36.822+0000 7f6b0fe00640 1 -- 172.21.15.47:0/3920076892 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555ecebcb790 msgr2=0x555ecebcdb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:32:36.823 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:36.822+0000 7f6b0fe00640 1 --2- 172.21.15.47:0/3920076892 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555ecebcb790 0x555ecebcdb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:32:36.823 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:36.822+0000 7f6b0fe00640 1 -- 172.21.15.47:0/3920076892 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555ece9037e0 con 0x555ecebc2bb0 2024-03-21T10:32:36.823 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:36.822+0000 7f6b16800640 1 --2- 172.21.15.47:0/3920076892 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555ecebcb790 0x555ecebcdb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T10:32:36.823 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:36.823+0000 7f6b16800640 1 --2- 172.21.15.47:0/687598721 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f6afc0a3090 0x7f6afc0a5550 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x555ecebbdb50 tx=0x7f6b10009b10 comp rx=0 tx=0).ready entity=mgr.15687 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:32:36.823 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:36.823+0000 7f6b0fe00640 1 --2- 172.21.15.47:0/3920076892 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x555ecebc2bb0 0x555ecebcb250 secure :-1 s=READY pgs=1078 cs=0 l=1 rev1=1 crypto rx=0x7f6b000c5bb0 tx=0x7f6b000b9310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2afa6a1b10af2c30 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:32:36.823 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:36.823+0000 7f6b07400640 1 -- 172.21.15.47:0/3920076892 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b000bd030 con 0x555ecebc2bb0 2024-03-21T10:32:36.823 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:36.823+0000 7f6b07400640 1 -- 172.21.15.47:0/3920076892 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f6b000bddd0 con 0x555ecebc2bb0 2024-03-21T10:32:36.823 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:36.823+0000 7f6b07400640 1 -- 172.21.15.47:0/3920076892 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b0002e000 con 0x555ecebc2bb0 2024-03-21T10:32:36.827 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:36.824+0000 7f6b1fc45740 1 -- 172.21.15.47:0/3920076892 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x555ecebc2bb0 msgr2=0x555ecebcb250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:32:36.827 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:36.824+0000 7f6b1fc45740 1 --2- 172.21.15.47:0/3920076892 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x555ecebc2bb0 0x555ecebcb250 secure :-1 s=READY pgs=1078 cs=0 l=1 rev1=1 crypto rx=0x7f6b000c5bb0 tx=0x7f6b000b9310 comp rx=0 tx=0).stop 2024-03-21T10:32:36.827 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:36.824+0000 7f6b1fc45740 1 -- 172.21.15.47:0/3920076892 shutdown_connections 2024-03-21T10:32:36.827 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:36.824+0000 7f6b1fc45740 1 --2- 172.21.15.47:0/3920076892 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555ecebcb790 0x555ecebcdb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:32:36.827 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:36.824+0000 7f6b1fc45740 1 --2- 172.21.15.47:0/3920076892 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x555ecebc2bb0 0x555ecebcb250 unknown :-1 s=CLOSED pgs=1078 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:32:36.827 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:36.824+0000 7f6b1fc45740 1 --2- 172.21.15.47:0/3920076892 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555ecebc2290 0x555ecebc2670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:32:36.827 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:36.824+0000 7f6b1fc45740 1 -- 172.21.15.47:0/3920076892 >> 172.21.15.47:0/3920076892 conn(0x555ecebbc9c0 msgr2=0x555ecebbcda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:32:36.827 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:36.824+0000 7f6b1fc45740 1 -- 172.21.15.47:0/3920076892 shutdown_connections 2024-03-21T10:32:36.827 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:36.824+0000 7f6b1fc45740 1 -- 172.21.15.47:0/3920076892 wait complete. 2024-03-21T10:32:36.827 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:36.824+0000 7f6b1fc45740 1 -- start start 2024-03-21T10:32:36.828 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:36.825+0000 7f6b1fc45740 1 Processor -- start 2024-03-21T10:32:36.828 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:36.825+0000 7f6b1fc45740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555ecebc2290 0x555ecebcb0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:32:36.828 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:36.825+0000 7f6b1fc45740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x555ecebc2bb0 0x555ecebc8440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:32:36.828 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:36.825+0000 7f6b1fc45740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555ecebcb790 0x555ecebc8980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:32:36.828 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:36.825+0000 7f6b1fc45740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x555eceb26eb0 con 0x555ecebc2bb0 2024-03-21T10:32:36.828 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:36.825+0000 7f6b1fc45740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x555eceae2eb0 con 0x555ecebc2290 2024-03-21T10:32:36.828 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:36.825+0000 7f6b1fc45740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x555ecebce340 con 0x555ecebcb790 2024-03-21T10:32:36.828 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:36.825+0000 7f6b1fc45740 10 client.0 _ll_register_callbacks cb 0x555ecead7150 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T10:32:36.828 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:36.825+0000 7f6b0fe00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x555ecebc2bb0 0x555ecebc8440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:32:36.828 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:36.826+0000 7f6b0fe00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x555ecebc2bb0 0x555ecebc8440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:51404/0 (socket says 172.21.15.47:51404) 2024-03-21T10:32:36.828 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:36.826+0000 7f6b0fe00640 1 -- 172.21.15.47:0/3426998934 learned_addr learned my addr 172.21.15.47:0/3426998934 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:32:36.828 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:36.826+0000 7f6b15e00640 1 --2- 172.21.15.47:0/3426998934 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555ecebc2290 0x555ecebcb0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:32:36.828 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:36.826+0000 7f6b0fe00640 1 -- 172.21.15.47:0/3426998934 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555ecebcb790 msgr2=0x555ecebc8980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:32:36.829 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:36.826+0000 7f6b0fe00640 1 --2- 172.21.15.47:0/3426998934 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555ecebcb790 0x555ecebc8980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:32:36.829 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:36.826+0000 7f6b0fe00640 1 -- 172.21.15.47:0/3426998934 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555ecebc2290 msgr2=0x555ecebcb0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:32:36.829 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:36.826+0000 7f6b0fe00640 1 --2- 172.21.15.47:0/3426998934 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555ecebc2290 0x555ecebcb0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:32:36.829 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:36.826+0000 7f6b0fe00640 1 -- 172.21.15.47:0/3426998934 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x555ece963f70 con 0x555ecebc2bb0 2024-03-21T10:32:36.829 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:36.826+0000 7f6b0fe00640 1 --2- 172.21.15.47:0/3426998934 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x555ecebc2bb0 0x555ecebc8440 secure :-1 s=READY pgs=1079 cs=0 l=1 rev1=1 crypto rx=0x7f6b000bd660 tx=0x7f6b0002e2b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e80fd241ad2e0b9b server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:32:36.829 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:36.826+0000 7f6afbe00640 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-03-21T10:32:36.829 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:36.826+0000 7f6afbe00640 1 -- 172.21.15.47:0/3426998934 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b000bd030 con 0x555ecebc2bb0 2024-03-21T10:32:36.829 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:36.827+0000 7f6afbe00640 1 -- 172.21.15.47:0/3426998934 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(99..99 src has 1..99) v4 ==== 9562+0+0 (secure 0 0 0) 0x7f6b00003bf0 con 0x555ecebc2bb0 2024-03-21T10:32:36.829 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:36.827+0000 7f6b1fc45740 1 -- 172.21.15.47:0/3426998934 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f6b000bddd0 con 0x555ecebc2bb0 2024-03-21T10:32:36.829 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:36.827+0000 7f6afbe00640 1 -- 172.21.15.47:0/3426998934 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6b000eeda0 con 0x555ecebc2bb0 2024-03-21T10:32:36.829 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:36.827+0000 7f6b1fc45740 10 client.17967 fetch_fsmap learned FSMap version 32 2024-03-21T10:32:36.829 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:36.827+0000 7f6b1fc45740 1 -- 172.21.15.47:0/3426998934 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x555ece7fb170 con 0x555ecebc2bb0 2024-03-21T10:32:36.829 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:36.827+0000 7f6afbe00640 1 -- 172.21.15.47:0/3426998934 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f6b000bddd0 con 0x555ecebc2bb0 2024-03-21T10:32:36.829 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:36.827+0000 7f6afbe00640 1 -- 172.21.15.47:0/3426998934 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 32) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f6b000bd930 con 0x555ecebc2bb0 2024-03-21T10:32:36.829 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:36.827+0000 7f6b1fc45740 10 client.17967 fetch_fsmap finished waiting for FSMap version 32 2024-03-21T10:32:36.830 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:36.827+0000 7f6b1fc45740 10 client.17967 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-21T10:32:36.830 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:36.827+0000 7f6b1fc45740 20 client.17967 populate_metadata read hostname 'smithi047' 2024-03-21T10:32:36.830 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:36.827+0000 7f6b1fc45740 1 --2- 172.21.15.47:0/3426998934 >> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] conn(0x555ecebdb330 0x555ecebdd720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:32:36.830 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:36.827+0000 7f6b1fc45740 4 client.17967 mds_command: new command op to 34535 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T10:32:36.830 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:36.827+0000 7f6b1fc45740 1 -- 172.21.15.47:0/3426998934 --> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x555eceb343f0 con 0x555ecebdb330 2024-03-21T10:32:36.830 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:36.828+0000 7f6b15e00640 1 --2- 172.21.15.47:0/3426998934 >> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] conn(0x555ecebdb330 0x555ecebdd720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:32:36.830 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:36.829+0000 7f6b15e00640 1 --2- 172.21.15.47:0/3426998934 >> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] conn(0x555ecebdb330 0x555ecebdd720 crc :-1 s=READY pgs=179 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8f14d9790753f1e server_cookie=382e9a189653d182 in_seq=0 out_seq=0 2024-03-21T10:32:36.830 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:36.829+0000 7f6afbe00640 10 client.17967 ms_handle_connect on v2:172.21.15.100:6838/4173168440 2024-03-21T10:32:36.831 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:36.830+0000 7f6afbe00640 1 -- 172.21.15.47:0/3426998934 <== mds.0 v2:172.21.15.100:6838/4173168440 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x555eceb343f0 con 0x555ecebdb330 2024-03-21T10:32:36.831 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:36.830+0000 7f6afbe00640 10 client.17967 handle_command_reply: tid=0 2024-03-21T10:32:36.831 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:36.830+0000 7f6b1fc45740 1 client.17967 shutdown 2024-03-21T10:32:36.831 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:36.830+0000 7f6b1fc45740 1 -- 172.21.15.47:0/3426998934 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x555ecebc2bb0 msgr2=0x555ecebc8440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:32:36.831 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:36.830+0000 7f6b1fc45740 1 --2- 172.21.15.47:0/3426998934 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x555ecebc2bb0 0x555ecebc8440 secure :-1 s=READY pgs=1079 cs=0 l=1 rev1=1 crypto rx=0x7f6b000bd660 tx=0x7f6b0002e2b0 comp rx=0 tx=0).stop 2024-03-21T10:32:36.832 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:36.831+0000 7f6b1fc45740 1 -- 172.21.15.47:0/3426998934 shutdown_connections 2024-03-21T10:32:36.832 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:36.831+0000 7f6b1fc45740 1 --2- 172.21.15.47:0/3426998934 >> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] conn(0x555ecebdb330 0x555ecebdd720 crc :-1 s=READY pgs=179 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:32:36.832 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:36.831+0000 7f6b1fc45740 1 --2- 172.21.15.47:0/3426998934 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555ecebcb790 0x555ecebc8980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:32:36.832 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:36.831+0000 7f6b1fc45740 1 --2- 172.21.15.47:0/3426998934 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x555ecebc2bb0 0x555ecebc8440 unknown :-1 s=CLOSED pgs=1079 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:32:36.832 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:36.831+0000 7f6b1fc45740 1 --2- 172.21.15.47:0/3426998934 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555ecebc2290 0x555ecebcb0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:32:36.832 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:36.831+0000 7f6afbe00640 0 client.17967 ms_handle_reset on v2:172.21.15.100:6838/4173168440 2024-03-21T10:32:36.832 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:36.831+0000 7f6b1fc45740 1 -- 172.21.15.47:0/3426998934 >> 172.21.15.47:0/3426998934 conn(0x555ecebbc9c0 msgr2=0x555ecebbe710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:32:36.832 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:36.831+0000 7f6b1fc45740 1 -- 172.21.15.47:0/3426998934 shutdown_connections 2024-03-21T10:32:36.832 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:36.831+0000 7f6b1fc45740 1 -- 172.21.15.47:0/3426998934 wait complete. 2024-03-21T10:32:36.832 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:36.831+0000 7f6b1fc45740 20 client.17967 trim_cache size 0 max 16384 2024-03-21T10:32:36.847 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:36.849+0000 7f6b1fc45740 1 Processor -- start 2024-03-21T10:32:36.847 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:36.849+0000 7f6b1fc45740 1 -- start start 2024-03-21T10:32:36.847 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:36.849+0000 7f6b1fc45740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555ecebc2290 0x555ecebcab70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:32:36.848 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:36.849+0000 7f6b1fc45740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x555ecebcb790 0x555ecebcb0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:32:36.848 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:36.849+0000 7f6b1fc45740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555ecebd34d0 0x555ecebbe3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:32:36.848 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:36.849+0000 7f6b1fc45740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b181b09e0 con 0x555ecebcb790 2024-03-21T10:32:36.848 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:36.849+0000 7f6b1fc45740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b181b2360 con 0x555ecebc2290 2024-03-21T10:32:36.848 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:36.849+0000 7f6b1fc45740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b181b1040 con 0x555ecebd34d0 2024-03-21T10:32:36.850 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:36.849+0000 7f6b0fe00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x555ecebcb790 0x555ecebcb0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:32:36.850 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:36.849+0000 7f6b15e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555ecebc2290 0x555ecebcab70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:32:36.850 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:36.849+0000 7f6b15e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555ecebc2290 0x555ecebcab70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:57580/0 (socket says 172.21.15.47:57580) 2024-03-21T10:32:36.850 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:36.849+0000 7f6b15e00640 1 -- 172.21.15.47:0/758116658 learned_addr learned my addr 172.21.15.47:0/758116658 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:32:36.850 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:36.850+0000 7f6b16800640 1 --2- 172.21.15.47:0/758116658 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555ecebd34d0 0x555ecebbe3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:32:36.850 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:36.850+0000 7f6b0fe00640 1 -- 172.21.15.47:0/758116658 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555ecebd34d0 msgr2=0x555ecebbe3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:32:36.850 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:36.850+0000 7f6b0fe00640 1 --2- 172.21.15.47:0/758116658 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555ecebd34d0 0x555ecebbe3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:32:36.850 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:36.850+0000 7f6b0fe00640 1 -- 172.21.15.47:0/758116658 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555ecebc2290 msgr2=0x555ecebcab70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:32:36.850 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:36.850+0000 7f6b0fe00640 1 --2- 172.21.15.47:0/758116658 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555ecebc2290 0x555ecebcab70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:32:36.850 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:36.850+0000 7f6b0fe00640 1 -- 172.21.15.47:0/758116658 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555ecebb6c40 con 0x555ecebcb790 2024-03-21T10:32:36.851 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:36.850+0000 7f6b15e00640 1 --2- 172.21.15.47:0/758116658 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555ecebc2290 0x555ecebcab70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T10:32:36.851 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:36.850+0000 7f6b0fe00640 1 --2- 172.21.15.47:0/758116658 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x555ecebcb790 0x555ecebcb0b0 secure :-1 s=READY pgs=1080 cs=0 l=1 rev1=1 crypto rx=0x555ecebc9500 tx=0x7f6b000053b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9f240121ecff289c server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:32:36.851 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:36.850+0000 7f6b06a00640 1 -- 172.21.15.47:0/758116658 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b00086070 con 0x555ecebcb790 2024-03-21T10:32:36.851 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:36.850+0000 7f6b06a00640 1 -- 172.21.15.47:0/758116658 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f6b000a80b0 con 0x555ecebcb790 2024-03-21T10:32:36.851 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:36.851+0000 7f6b06a00640 1 -- 172.21.15.47:0/758116658 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b000c60e0 con 0x555ecebcb790 2024-03-21T10:32:36.851 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:36.851+0000 7f6b1fc45740 1 -- 172.21.15.47:0/758116658 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x555ecebcb790 msgr2=0x555ecebcb0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:32:36.851 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:36.851+0000 7f6b1fc45740 1 --2- 172.21.15.47:0/758116658 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x555ecebcb790 0x555ecebcb0b0 secure :-1 s=READY pgs=1080 cs=0 l=1 rev1=1 crypto rx=0x555ecebc9500 tx=0x7f6b000053b0 comp rx=0 tx=0).stop 2024-03-21T10:32:36.851 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:36.851+0000 7f6b1fc45740 1 -- 172.21.15.47:0/758116658 shutdown_connections 2024-03-21T10:32:36.851 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:36.851+0000 7f6b1fc45740 1 --2- 172.21.15.47:0/758116658 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555ecebd34d0 0x555ecebbe3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:32:36.851 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:36.851+0000 7f6b1fc45740 1 --2- 172.21.15.47:0/758116658 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x555ecebcb790 0x555ecebcb0b0 unknown :-1 s=CLOSED pgs=1080 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:32:36.851 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:36.851+0000 7f6b1fc45740 1 --2- 172.21.15.47:0/758116658 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555ecebc2290 0x555ecebcab70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:32:36.851 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:36.851+0000 7f6b1fc45740 1 -- 172.21.15.47:0/758116658 >> 172.21.15.47:0/758116658 conn(0x555ecebbc9c0 msgr2=0x555ecebbdcc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:32:36.851 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:36.851+0000 7f6b1fc45740 1 -- 172.21.15.47:0/758116658 shutdown_connections 2024-03-21T10:32:36.852 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:36.851+0000 7f6b1fc45740 1 -- 172.21.15.47:0/758116658 wait complete. 2024-03-21T10:32:36.855 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:36.852+0000 7f6b1fc45740 1 -- start start 2024-03-21T10:32:36.855 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:36.852+0000 7f6b1fc45740 1 Processor -- start 2024-03-21T10:32:36.855 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:36.853+0000 7f6b1fc45740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555ecebc2290 0x555ecebd03b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:32:36.855 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:36.853+0000 7f6b1fc45740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555ecebcb790 0x555ecebd08f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:32:36.855 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:36.853+0000 7f6b1fc45740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x555ecebd34d0 0x555ecebd0e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:32:36.855 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:36.853+0000 7f6b1fc45740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b181b1d00 con 0x555ecebd34d0 2024-03-21T10:32:36.855 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:36.853+0000 7f6b1fc45740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b181b16a0 con 0x555ecebcb790 2024-03-21T10:32:36.855 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:36.853+0000 7f6b1fc45740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b181aebc0 con 0x555ecebc2290 2024-03-21T10:32:36.855 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:36.853+0000 7f6b1fc45740 10 client.0 _ll_register_callbacks cb 0x7f6b000bd6d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T10:32:36.855 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:36.853+0000 7f6b16800640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x555ecebd34d0 0x555ecebd0e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:32:36.856 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:36.853+0000 7f6b16800640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x555ecebd34d0 0x555ecebd0e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:51414/0 (socket says 172.21.15.47:51414) 2024-03-21T10:32:36.856 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:36.853+0000 7f6b16800640 1 -- 172.21.15.47:0/4265322909 learned_addr learned my addr 172.21.15.47:0/4265322909 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:32:36.856 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:36.853+0000 7f6b0fe00640 1 --2- 172.21.15.47:0/4265322909 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555ecebcb790 0x555ecebd08f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:32:36.856 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:36.854+0000 7f6b15e00640 1 --2- 172.21.15.47:0/4265322909 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555ecebc2290 0x555ecebd03b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:32:36.856 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:36.854+0000 7f6b0fe00640 1 -- 172.21.15.47:0/4265322909 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555ecebc2290 msgr2=0x555ecebd03b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:32:36.856 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:36.854+0000 7f6b0fe00640 1 --2- 172.21.15.47:0/4265322909 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555ecebc2290 0x555ecebd03b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:32:36.856 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:36.854+0000 7f6b0fe00640 1 -- 172.21.15.47:0/4265322909 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x555ecebd34d0 msgr2=0x555ecebd0e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:32:36.856 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:36.854+0000 7f6b0fe00640 1 --2- 172.21.15.47:0/4265322909 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x555ecebd34d0 0x555ecebd0e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:32:36.856 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:36.854+0000 7f6b0fe00640 1 -- 172.21.15.47:0/4265322909 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x555ecec3f390 con 0x555ecebcb790 2024-03-21T10:32:36.856 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:36.854+0000 7f6b0fe00640 1 --2- 172.21.15.47:0/4265322909 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555ecebcb790 0x555ecebd08f0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f6b00005bc0 tx=0x7f6b00003e20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=40d43a3c5726d270 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:32:36.856 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:36.854+0000 7f6afbe00640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-21T10:32:36.856 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:36.854+0000 7f6afbe00640 1 -- 172.21.15.47:0/4265322909 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b00086050 con 0x555ecebcb790 2024-03-21T10:32:36.857 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:36.854+0000 7f6afbe00640 1 -- 172.21.15.47:0/4265322909 <== mon.2 v2:172.21.15.100:3300/0 2 ==== osd_map(99..99 src has 1..99) v4 ==== 9562+0+0 (secure 0 0 0) 0x7f6b000eeb10 con 0x555ecebcb790 2024-03-21T10:32:36.857 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:36.854+0000 7f6b1fc45740 1 -- 172.21.15.47:0/4265322909 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x555eceb2e740 con 0x555ecebcb790 2024-03-21T10:32:36.857 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:36.855+0000 7f6afbe00640 1 -- 172.21.15.47:0/4265322909 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6b000bddd0 con 0x555ecebcb790 2024-03-21T10:32:36.857 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:36.855+0000 7f6b1fc45740 10 client.37754 fetch_fsmap learned FSMap version 32 2024-03-21T10:32:36.857 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:36.855+0000 7f6b1fc45740 1 -- 172.21.15.47:0/4265322909 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x555ecec3f390 con 0x555ecebcb790 2024-03-21T10:32:36.857 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:36.855+0000 7f6afbe00640 1 -- 172.21.15.47:0/4265322909 <== mon.2 v2:172.21.15.100:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x555eceb2e740 con 0x555ecebcb790 2024-03-21T10:32:36.857 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:36.855+0000 7f6afbe00640 1 -- 172.21.15.47:0/4265322909 <== mon.2 v2:172.21.15.100:3300/0 5 ==== fsmap(e 32) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f6b000b9420 con 0x555ecebcb790 2024-03-21T10:32:36.857 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:36.855+0000 7f6b1fc45740 10 client.37754 fetch_fsmap finished waiting for FSMap version 32 2024-03-21T10:32:36.857 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:36.855+0000 7f6b1fc45740 10 client.37754 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-21T10:32:36.857 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:36.855+0000 7f6b1fc45740 20 client.37754 populate_metadata read hostname 'smithi047' 2024-03-21T10:32:36.857 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:36.855+0000 7f6b1fc45740 1 --2- 172.21.15.47:0/4265322909 >> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] conn(0x555ecec408e0 0x555ecec42cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:32:36.857 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:36.855+0000 7f6b1fc45740 4 client.37754 mds_command: new command op to 34535 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-03-21T10:32:36.857 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:36.855+0000 7f6b1fc45740 1 -- 172.21.15.47:0/4265322909 --> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x555eceaaca40 con 0x555ecec408e0 2024-03-21T10:32:36.857 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:36.856+0000 7f6b15e00640 1 --2- 172.21.15.47:0/4265322909 >> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] conn(0x555ecec408e0 0x555ecec42cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:32:36.858 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:36.856+0000 7f6b15e00640 1 --2- 172.21.15.47:0/4265322909 >> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] conn(0x555ecec408e0 0x555ecec42cd0 crc :-1 s=READY pgs=180 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9b86824ec99282e9 server_cookie=91b7d3aab075cfcc in_seq=0 out_seq=0 2024-03-21T10:32:36.858 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:36.856+0000 7f6afbe00640 10 client.37754 ms_handle_connect on v2:172.21.15.100:6838/4173168440 2024-03-21T10:32:36.859 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:36.858+0000 7f6afbe00640 1 -- 172.21.15.47:0/4265322909 <== mds.0 v2:172.21.15.100:6838/4173168440 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x555eceaaca40 con 0x555ecec408e0 2024-03-21T10:32:36.859 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:36.858+0000 7f6afbe00640 10 client.37754 handle_command_reply: tid=0 2024-03-21T10:32:36.859 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:36.858+0000 7f6b1fc45740 1 client.37754 shutdown 2024-03-21T10:32:36.859 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:36.858+0000 7f6b1fc45740 1 -- 172.21.15.47:0/4265322909 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555ecebcb790 msgr2=0x555ecebd08f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:32:36.859 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:36.858+0000 7f6b1fc45740 1 --2- 172.21.15.47:0/4265322909 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555ecebcb790 0x555ecebd08f0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f6b00005bc0 tx=0x7f6b00003e20 comp rx=0 tx=0).stop 2024-03-21T10:32:36.859 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:36.858+0000 7f6b1fc45740 1 -- 172.21.15.47:0/4265322909 shutdown_connections 2024-03-21T10:32:36.859 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:36.858+0000 7f6b1fc45740 1 --2- 172.21.15.47:0/4265322909 >> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] conn(0x555ecec408e0 0x555ecec42cd0 crc :-1 s=READY pgs=180 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:32:36.859 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:36.858+0000 7f6b1fc45740 1 --2- 172.21.15.47:0/4265322909 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x555ecebd34d0 0x555ecebd0e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:32:36.859 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:36.858+0000 7f6b1fc45740 1 --2- 172.21.15.47:0/4265322909 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555ecebcb790 0x555ecebd08f0 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:32:36.860 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:36.858+0000 7f6b1fc45740 1 --2- 172.21.15.47:0/4265322909 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555ecebc2290 0x555ecebd03b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:32:36.860 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:36.858+0000 7f6afbe00640 0 client.37754 ms_handle_reset on v2:172.21.15.100:6838/4173168440 2024-03-21T10:32:36.860 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:36.858+0000 7f6b1fc45740 1 -- 172.21.15.47:0/4265322909 >> 172.21.15.47:0/4265322909 conn(0x555ecebbc9c0 msgr2=0x555ecebbba80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:32:36.860 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:36.859+0000 7f6b1fc45740 1 -- 172.21.15.47:0/4265322909 shutdown_connections 2024-03-21T10:32:36.860 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:36.859+0000 7f6b1fc45740 1 -- 172.21.15.47:0/4265322909 wait complete. 2024-03-21T10:32:36.860 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:36.859+0000 7f6b1fc45740 20 client.37754 trim_cache size 0 max 16384 2024-03-21T10:32:36.860 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-03-21T10:32:36.860 INFO:teuthology.orchestra.run.smithi047.stdout: "return_code": 0, 2024-03-21T10:32:36.860 INFO:teuthology.orchestra.run.smithi047.stdout: "scrub_tag": "a93db2e1-1155-41d7-b775-a2dbb387576e", 2024-03-21T10:32:36.860 INFO:teuthology.orchestra.run.smithi047.stdout: "mode": "asynchronous" 2024-03-21T10:32:36.860 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-03-21T10:32:36.861 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:36.859+0000 7f6b17200640 1 -- 172.21.15.47:0/687598721 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f6afc0a3090 msgr2=0x7f6afc0a5550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:32:36.861 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:36.859+0000 7f6b17200640 1 --2- 172.21.15.47:0/687598721 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f6afc0a3090 0x7f6afc0a5550 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x555ecebbdb50 tx=0x7f6b10009b10 comp rx=0 tx=0).stop 2024-03-21T10:32:36.861 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:36.860+0000 7f6b17200640 1 -- 172.21.15.47:0/687598721 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6b18101ee0 msgr2=0x7f6b18073ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:32:36.861 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:36.860+0000 7f6b17200640 1 --2- 172.21.15.47:0/687598721 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6b18101ee0 0x7f6b18073ee0 secure :-1 s=READY pgs=1077 cs=0 l=1 rev1=1 crypto rx=0x7f6b00006010 tx=0x7f6b00005e20 comp rx=0 tx=0).stop 2024-03-21T10:32:36.861 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:36.860+0000 7f6b17200640 1 -- 172.21.15.47:0/687598721 shutdown_connections 2024-03-21T10:32:36.862 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:36.860+0000 7f6b17200640 1 --2- 172.21.15.47:0/687598721 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f6afc0a3090 0x7f6afc0a5550 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:32:36.862 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:36.860+0000 7f6b17200640 1 --2- 172.21.15.47:0/687598721 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6b181075e0 0x7f6b18074420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:32:36.862 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:36.860+0000 7f6b17200640 1 --2- 172.21.15.47:0/687598721 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6b18101ee0 0x7f6b18073ee0 unknown :-1 s=CLOSED pgs=1077 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:32:36.862 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:36.860+0000 7f6b17200640 1 --2- 172.21.15.47:0/687598721 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6b18101520 0x7f6b180739a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:32:36.862 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:36.860+0000 7f6b17200640 1 -- 172.21.15.47:0/687598721 >> 172.21.15.47:0/687598721 conn(0x7f6b180fd010 msgr2=0x7f6b180fea10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:32:36.862 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:36.860+0000 7f6b17200640 1 -- 172.21.15.47:0/687598721 shutdown_connections 2024-03-21T10:32:36.862 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:36.860+0000 7f6b17200640 1 -- 172.21.15.47:0/687598721 wait complete. 2024-03-21T10:32:36.885 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-21T10:32:36.885 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-21T10:32:37.106 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:37.107+0000 7fba7fe00640 1 -- 172.21.15.47:0/632279370 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fba80100400 msgr2=0x7fba80100880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:32:37.106 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:37.107+0000 7fba7fe00640 1 --2- 172.21.15.47:0/632279370 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fba80100400 0x7fba80100880 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fba6c009900 tx=0x7fba6c02bc50 comp rx=0 tx=0).stop 2024-03-21T10:32:37.106 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:37.107+0000 7fba7fe00640 1 -- 172.21.15.47:0/632279370 shutdown_connections 2024-03-21T10:32:37.106 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:37.107+0000 7fba7fe00640 1 --2- 172.21.15.47:0/632279370 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fba80100dc0 0x7fba8010d2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:32:37.106 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:37.107+0000 7fba7fe00640 1 --2- 172.21.15.47:0/632279370 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fba80100400 0x7fba80100880 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:32:37.106 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:37.107+0000 7fba7fe00640 1 --2- 172.21.15.47:0/632279370 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fba801064a0 0x7fba80106880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:32:37.106 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:37.107+0000 7fba7fe00640 1 -- 172.21.15.47:0/632279370 >> 172.21.15.47:0/632279370 conn(0x7fba800fbef0 msgr2=0x7fba800fe310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:32:37.106 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:37.107+0000 7fba7fe00640 1 -- 172.21.15.47:0/632279370 shutdown_connections 2024-03-21T10:32:37.107 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:37.107+0000 7fba7fe00640 1 -- 172.21.15.47:0/632279370 wait complete. 2024-03-21T10:32:37.108 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:37.108+0000 7fba7fe00640 1 Processor -- start 2024-03-21T10:32:37.108 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:37.108+0000 7fba7fe00640 1 -- start start 2024-03-21T10:32:37.108 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:37.109+0000 7fba7fe00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fba80100400 0x7fba8010dd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:32:37.109 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:37.109+0000 7fba7fe00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fba80100dc0 0x7fba8010e2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:32:37.109 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:37.109+0000 7fba7fe00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fba801064a0 0x7fba8010e7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:32:37.109 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:37.109+0000 7fba7fe00640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba801117e0 con 0x7fba801064a0 2024-03-21T10:32:37.109 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:37.109+0000 7fba7fe00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba80111660 con 0x7fba80100400 2024-03-21T10:32:37.109 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:37.109+0000 7fba7fe00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba80111960 con 0x7fba80100dc0 2024-03-21T10:32:37.109 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:37.109+0000 7fba7f400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fba801064a0 0x7fba8010e7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:32:37.109 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:37.109+0000 7fba7f400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fba801064a0 0x7fba8010e7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:51432/0 (socket says 172.21.15.47:51432) 2024-03-21T10:32:37.109 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:37.109+0000 7fba7f400640 1 -- 172.21.15.47:0/3382097738 learned_addr learned my addr 172.21.15.47:0/3382097738 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:32:37.109 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:37.109+0000 7fba7ea00640 1 --2- 172.21.15.47:0/3382097738 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fba80100400 0x7fba8010dd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:32:37.109 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:37.109+0000 7fba7e000640 1 --2- 172.21.15.47:0/3382097738 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fba80100dc0 0x7fba8010e2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:32:37.109 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:37.109+0000 7fba7f400640 1 -- 172.21.15.47:0/3382097738 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fba80100dc0 msgr2=0x7fba8010e2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:32:37.109 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:37.109+0000 7fba7f400640 1 --2- 172.21.15.47:0/3382097738 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fba80100dc0 0x7fba8010e2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:32:37.109 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:37.109+0000 7fba7f400640 1 -- 172.21.15.47:0/3382097738 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fba80100400 msgr2=0x7fba8010dd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:32:37.109 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:37.109+0000 7fba7f400640 1 --2- 172.21.15.47:0/3382097738 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fba80100400 0x7fba8010dd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:32:37.110 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:37.109+0000 7fba7f400640 1 -- 172.21.15.47:0/3382097738 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba801a6200 con 0x7fba801064a0 2024-03-21T10:32:37.110 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:37.110+0000 7fba7f400640 1 --2- 172.21.15.47:0/3382097738 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fba801064a0 0x7fba8010e7e0 secure :-1 s=READY pgs=1081 cs=0 l=1 rev1=1 crypto rx=0x7fba7800eb10 tx=0x7fba7800c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:32:37.110 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:37.110+0000 7fba77400640 1 -- 172.21.15.47:0/3382097738 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba78004b20 con 0x7fba801064a0 2024-03-21T10:32:37.110 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:37.110+0000 7fba77400640 1 -- 172.21.15.47:0/3382097738 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fba78004cc0 con 0x7fba801064a0 2024-03-21T10:32:37.110 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:37.110+0000 7fba77400640 1 -- 172.21.15.47:0/3382097738 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba78010690 con 0x7fba801064a0 2024-03-21T10:32:37.110 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:37.110+0000 7fba7fe00640 1 -- 172.21.15.47:0/3382097738 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fba801a64f0 con 0x7fba801064a0 2024-03-21T10:32:37.110 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:37.110+0000 7fba7fe00640 1 -- 172.21.15.47:0/3382097738 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fba801a69c0 con 0x7fba801064a0 2024-03-21T10:32:37.112 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:37.114+0000 7fba88775740 1 Processor -- start 2024-03-21T10:32:37.112 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:37.114+0000 7fba77400640 1 -- 172.21.15.47:0/3382097738 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 27) v1 ==== 137985+0+0 (secure 0 0 0) 0x7fba78005020 con 0x7fba801064a0 2024-03-21T10:32:37.112 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:37.114+0000 7fba88775740 1 -- start start 2024-03-21T10:32:37.113 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:37.115+0000 7fba88775740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b7e5faedd0 0x55b7e5fadb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:32:37.113 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:37.115+0000 7fba88775740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b7e5fae0f0 0x55b7e5fae570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:32:37.113 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:37.115+0000 7fba88775740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b7e5fb7460 0x55b7e5fb9850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:32:37.113 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:37.115+0000 7fba88775740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b7e5f12250 con 0x55b7e5faedd0 2024-03-21T10:32:37.114 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:37.115+0000 7fba88775740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b7e5ebaa30 con 0x55b7e5fb7460 2024-03-21T10:32:37.114 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:37.115+0000 7fba88775740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b7e5f0e2f0 con 0x55b7e5fae0f0 2024-03-21T10:32:37.114 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:37.115+0000 7fba7e000640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b7e5faedd0 0x55b7e5fadb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:32:37.114 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:37.115+0000 7fba7e000640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b7e5faedd0 0x55b7e5fadb20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:51436/0 (socket says 172.21.15.47:51436) 2024-03-21T10:32:37.114 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:37.115+0000 7fba7e000640 1 -- 172.21.15.47:0/480485323 learned_addr learned my addr 172.21.15.47:0/480485323 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:32:37.115 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:37.115+0000 7fba7e000640 1 --2- 172.21.15.47:0/480485323 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b7e5fb7460 0x55b7e5fb9850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:32:37.116 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:37.115+0000 7fba7ea00640 1 --2- 172.21.15.47:0/480485323 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b7e5fae0f0 0x55b7e5fae570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:32:37.116 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:37.115+0000 7fba7e000640 1 -- 172.21.15.47:0/480485323 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b7e5fae0f0 msgr2=0x55b7e5fae570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:32:37.116 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:37.115+0000 7fba7e000640 1 --2- 172.21.15.47:0/480485323 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b7e5fae0f0 0x55b7e5fae570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:32:37.116 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:37.115+0000 7fba7e000640 1 -- 172.21.15.47:0/480485323 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b7e5fb7460 msgr2=0x55b7e5fb9850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:32:37.116 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:37.115+0000 7fba7e000640 1 --2- 172.21.15.47:0/480485323 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b7e5fb7460 0x55b7e5fb9850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:32:37.116 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:37.115+0000 7fba7e000640 1 -- 172.21.15.47:0/480485323 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b7e5ea5690 con 0x55b7e5faedd0 2024-03-21T10:32:37.116 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:37.116+0000 7fba77400640 1 --2- 172.21.15.47:0/3382097738 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7fba640a30c0 0x7fba640a5580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:32:37.116 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:37.116+0000 7fba7e000640 1 --2- 172.21.15.47:0/480485323 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b7e5faedd0 0x55b7e5fadb20 secure :-1 s=READY pgs=1082 cs=0 l=1 rev1=1 crypto rx=0x7fba6c009900 tx=0x7fba6c005bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=54f31c4275bd9075 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:32:37.116 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:37.116+0000 7fba77400640 1 -- 172.21.15.47:0/3382097738 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 9562+0+0 (secure 0 0 0) 0x7fba780d0230 con 0x7fba801064a0 2024-03-21T10:32:37.116 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:37.116+0000 7fba6be00640 1 -- 172.21.15.47:0/480485323 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba6c02c7a0 con 0x55b7e5faedd0 2024-03-21T10:32:37.116 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:37.116+0000 7fba6be00640 1 -- 172.21.15.47:0/480485323 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fba6c048070 con 0x55b7e5faedd0 2024-03-21T10:32:37.116 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:37.116+0000 7fba6be00640 1 -- 172.21.15.47:0/480485323 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba6c004a40 con 0x55b7e5faedd0 2024-03-21T10:32:37.116 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:37.116+0000 7fba7ea00640 1 --2- 172.21.15.47:0/3382097738 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7fba640a30c0 0x7fba640a5580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:32:37.116 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:37.117+0000 7fba88775740 1 -- 172.21.15.47:0/480485323 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b7e5faedd0 msgr2=0x55b7e5fadb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:32:37.117 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:37.117+0000 7fba7ea00640 1 --2- 172.21.15.47:0/3382097738 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7fba640a30c0 0x7fba640a5580 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x55b7e5faf320 tx=0x7fba7000b390 comp rx=0 tx=0).ready entity=mgr.15687 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:32:37.117 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:37.117+0000 7fba88775740 1 --2- 172.21.15.47:0/480485323 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b7e5faedd0 0x55b7e5fadb20 secure :-1 s=READY pgs=1082 cs=0 l=1 rev1=1 crypto rx=0x7fba6c009900 tx=0x7fba6c005bc0 comp rx=0 tx=0).stop 2024-03-21T10:32:37.117 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:37.117+0000 7fba88775740 1 -- 172.21.15.47:0/480485323 shutdown_connections 2024-03-21T10:32:37.117 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:37.117+0000 7fba88775740 1 --2- 172.21.15.47:0/480485323 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b7e5fb7460 0x55b7e5fb9850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:32:37.117 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:37.117+0000 7fba88775740 1 --2- 172.21.15.47:0/480485323 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b7e5fae0f0 0x55b7e5fae570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:32:37.117 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:37.117+0000 7fba88775740 1 --2- 172.21.15.47:0/480485323 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b7e5faedd0 0x55b7e5fadb20 unknown :-1 s=CLOSED pgs=1082 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:32:37.117 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:37.117+0000 7fba88775740 1 -- 172.21.15.47:0/480485323 >> 172.21.15.47:0/480485323 conn(0x55b7e5f75200 msgr2=0x55b7e5fa8b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:32:37.117 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:37.117+0000 7fba88775740 1 -- 172.21.15.47:0/480485323 shutdown_connections 2024-03-21T10:32:37.117 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:37.117+0000 7fba88775740 1 -- 172.21.15.47:0/480485323 wait complete. 2024-03-21T10:32:37.120 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:37.118+0000 7fba88775740 1 -- start start 2024-03-21T10:32:37.120 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:37.118+0000 7fba88775740 1 Processor -- start 2024-03-21T10:32:37.120 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:37.118+0000 7fba88775740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b7e5fae0f0 0x55b7e5fb6b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:32:37.120 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:37.118+0000 7fba88775740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b7e5faedd0 0x55b7e5fb3f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:32:37.120 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:37.118+0000 7fba88775740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b7e5fb7460 0x55b7e5fb4450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:32:37.120 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:37.118+0000 7fba88775740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b7e5fba270 con 0x55b7e5faedd0 2024-03-21T10:32:37.120 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:37.118+0000 7fba88775740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b7e5ecde80 con 0x55b7e5fb7460 2024-03-21T10:32:37.120 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:37.118+0000 7fba88775740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b7e5fba3f0 con 0x55b7e5fae0f0 2024-03-21T10:32:37.120 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:37.119+0000 7fba88775740 10 client.0 _ll_register_callbacks cb 0x55b7e5ee0130 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T10:32:37.121 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:37.119+0000 7fba7e000640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b7e5faedd0 0x55b7e5fb3f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:32:37.121 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:37.119+0000 7fba7e000640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b7e5faedd0 0x55b7e5fb3f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:51444/0 (socket says 172.21.15.47:51444) 2024-03-21T10:32:37.121 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:37.119+0000 7fba7e000640 1 -- 172.21.15.47:0/1587734278 learned_addr learned my addr 172.21.15.47:0/1587734278 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:32:37.121 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:37.119+0000 7fba7f400640 1 --2- 172.21.15.47:0/1587734278 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b7e5fb7460 0x55b7e5fb4450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:32:37.121 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:37.119+0000 7fba7ea00640 1 --2- 172.21.15.47:0/1587734278 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b7e5fae0f0 0x55b7e5fb6b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:32:37.121 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:37.119+0000 7fba7e000640 1 -- 172.21.15.47:0/1587734278 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b7e5fae0f0 msgr2=0x55b7e5fb6b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:32:37.121 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:37.119+0000 7fba7e000640 1 --2- 172.21.15.47:0/1587734278 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b7e5fae0f0 0x55b7e5fb6b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:32:37.121 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:37.119+0000 7fba7e000640 1 -- 172.21.15.47:0/1587734278 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b7e5fb7460 msgr2=0x55b7e5fb4450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:32:37.121 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:37.119+0000 7fba7e000640 1 --2- 172.21.15.47:0/1587734278 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b7e5fb7460 0x55b7e5fb4450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:32:37.121 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:37.119+0000 7fba7e000640 1 -- 172.21.15.47:0/1587734278 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55b7e5cf07d0 con 0x55b7e5faedd0 2024-03-21T10:32:37.121 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:37.120+0000 7fba7e000640 1 --2- 172.21.15.47:0/1587734278 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b7e5faedd0 0x55b7e5fb3f10 secure :-1 s=READY pgs=1083 cs=0 l=1 rev1=1 crypto rx=0x7fba6c02ca50 tx=0x7fba6c02bc50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=db264d00f27d3692 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:32:37.121 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:37.120+0000 7fba68c00640 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-03-21T10:32:37.121 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:37.120+0000 7fba68c00640 1 -- 172.21.15.47:0/1587734278 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba6c003a00 con 0x55b7e5faedd0 2024-03-21T10:32:37.121 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:37.120+0000 7fba68c00640 1 -- 172.21.15.47:0/1587734278 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(99..99 src has 1..99) v4 ==== 9562+0+0 (secure 0 0 0) 0x7fba6c003ba0 con 0x55b7e5faedd0 2024-03-21T10:32:37.121 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:37.120+0000 7fba88775740 1 -- 172.21.15.47:0/1587734278 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fba6c048070 con 0x55b7e5faedd0 2024-03-21T10:32:37.122 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:37.120+0000 7fba68c00640 1 -- 172.21.15.47:0/1587734278 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7fba6c05e050 con 0x55b7e5faedd0 2024-03-21T10:32:37.122 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:37.121+0000 7fba88775740 10 client.17985 fetch_fsmap learned FSMap version 32 2024-03-21T10:32:37.122 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:37.121+0000 7fba88775740 1 -- 172.21.15.47:0/1587734278 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x55b7e5c263f0 con 0x55b7e5faedd0 2024-03-21T10:32:37.122 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:37.121+0000 7fba68c00640 1 -- 172.21.15.47:0/1587734278 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fba6c048070 con 0x55b7e5faedd0 2024-03-21T10:32:37.122 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:37.121+0000 7fba68c00640 1 -- 172.21.15.47:0/1587734278 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 32) v1 ==== 7942+0+0 (secure 0 0 0) 0x7fba6c043bf0 con 0x55b7e5faedd0 2024-03-21T10:32:37.122 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:37.121+0000 7fba88775740 10 client.17985 fetch_fsmap finished waiting for FSMap version 32 2024-03-21T10:32:37.122 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:37.121+0000 7fba88775740 10 client.17985 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-21T10:32:37.122 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:37.121+0000 7fba88775740 20 client.17985 populate_metadata read hostname 'smithi047' 2024-03-21T10:32:37.122 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:37.121+0000 7fba88775740 1 --2- 172.21.15.47:0/1587734278 >> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] conn(0x55b7e5fc7200 0x55b7e5fc95f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:32:37.122 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:37.121+0000 7fba88775740 4 client.17985 mds_command: new command op to 34535 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T10:32:37.122 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:37.121+0000 7fba88775740 1 -- 172.21.15.47:0/1587734278 --> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55b7e5f1f790 con 0x55b7e5fc7200 2024-03-21T10:32:37.124 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:37.122+0000 7fba7ea00640 1 --2- 172.21.15.47:0/1587734278 >> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] conn(0x55b7e5fc7200 0x55b7e5fc95f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:32:37.124 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:37.123+0000 7fba7ea00640 1 --2- 172.21.15.47:0/1587734278 >> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] conn(0x55b7e5fc7200 0x55b7e5fc95f0 crc :-1 s=READY pgs=181 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d62a5d3615f4ffc7 server_cookie=b19b2213b94a9781 in_seq=0 out_seq=0 2024-03-21T10:32:37.124 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:37.123+0000 7fba68c00640 10 client.17985 ms_handle_connect on v2:172.21.15.100:6838/4173168440 2024-03-21T10:32:37.124 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:37.124+0000 7fba68c00640 1 -- 172.21.15.47:0/1587734278 <== mds.0 v2:172.21.15.100:6838/4173168440 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55b7e5f1f790 con 0x55b7e5fc7200 2024-03-21T10:32:37.124 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:37.124+0000 7fba68c00640 10 client.17985 handle_command_reply: tid=0 2024-03-21T10:32:37.124 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:37.124+0000 7fba88775740 1 client.17985 shutdown 2024-03-21T10:32:37.124 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:37.124+0000 7fba88775740 1 -- 172.21.15.47:0/1587734278 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b7e5faedd0 msgr2=0x55b7e5fb3f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:32:37.124 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:37.124+0000 7fba88775740 1 --2- 172.21.15.47:0/1587734278 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b7e5faedd0 0x55b7e5fb3f10 secure :-1 s=READY pgs=1083 cs=0 l=1 rev1=1 crypto rx=0x7fba6c02ca50 tx=0x7fba6c02bc50 comp rx=0 tx=0).stop 2024-03-21T10:32:37.124 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:37.124+0000 7fba88775740 1 -- 172.21.15.47:0/1587734278 shutdown_connections 2024-03-21T10:32:37.124 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:37.124+0000 7fba88775740 1 --2- 172.21.15.47:0/1587734278 >> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] conn(0x55b7e5fc7200 0x55b7e5fc95f0 crc :-1 s=READY pgs=181 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:32:37.124 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:37.124+0000 7fba88775740 1 --2- 172.21.15.47:0/1587734278 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b7e5fb7460 0x55b7e5fb4450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:32:37.124 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:37.124+0000 7fba88775740 1 --2- 172.21.15.47:0/1587734278 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b7e5faedd0 0x55b7e5fb3f10 unknown :-1 s=CLOSED pgs=1083 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:32:37.125 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:37.124+0000 7fba88775740 1 --2- 172.21.15.47:0/1587734278 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b7e5fae0f0 0x55b7e5fb6b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:32:37.125 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:37.124+0000 7fba88775740 1 -- 172.21.15.47:0/1587734278 >> 172.21.15.47:0/1587734278 conn(0x55b7e5f75200 msgr2=0x55b7e5fa7330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:32:37.125 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:37.124+0000 7fba68c00640 0 client.17985 ms_handle_reset on v2:172.21.15.100:6838/4173168440 2024-03-21T10:32:37.125 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:37.125+0000 7fba88775740 1 -- 172.21.15.47:0/1587734278 shutdown_connections 2024-03-21T10:32:37.125 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:37.125+0000 7fba88775740 1 -- 172.21.15.47:0/1587734278 wait complete. 2024-03-21T10:32:37.125 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:37.125+0000 7fba88775740 20 client.17985 trim_cache size 0 max 16384 2024-03-21T10:32:37.141 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:37.142+0000 7fba88775740 1 Processor -- start 2024-03-21T10:32:37.141 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:37.142+0000 7fba88775740 1 -- start start 2024-03-21T10:32:37.141 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:37.143+0000 7fba88775740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b7e5fae0f0 0x55b7e5fb6640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:32:37.141 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:37.143+0000 7fba88775740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b7e5faedd0 0x55b7e5fb6b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:32:37.141 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:37.143+0000 7fba88775740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b7e5fb7460 0x55b7e5fb27e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:32:37.141 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:37.143+0000 7fba88775740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba801b08b0 con 0x55b7e5faedd0 2024-03-21T10:32:37.142 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:37.143+0000 7fba88775740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba801b2230 con 0x55b7e5fb7460 2024-03-21T10:32:37.142 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:37.143+0000 7fba88775740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba801b0f10 con 0x55b7e5fae0f0 2024-03-21T10:32:37.143 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:37.143+0000 7fba7e000640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b7e5faedd0 0x55b7e5fb6b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:32:37.143 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:37.143+0000 7fba7e000640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b7e5faedd0 0x55b7e5fb6b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:51450/0 (socket says 172.21.15.47:51450) 2024-03-21T10:32:37.144 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:37.143+0000 7fba7e000640 1 -- 172.21.15.47:0/2527755366 learned_addr learned my addr 172.21.15.47:0/2527755366 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:32:37.144 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:37.143+0000 7fba7f400640 1 --2- 172.21.15.47:0/2527755366 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b7e5fb7460 0x55b7e5fb27e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:32:37.144 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:37.143+0000 7fba7ea00640 1 --2- 172.21.15.47:0/2527755366 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b7e5fae0f0 0x55b7e5fb6640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:32:37.144 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:37.143+0000 7fba7e000640 1 -- 172.21.15.47:0/2527755366 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b7e5fae0f0 msgr2=0x55b7e5fb6640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:32:37.144 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:37.144+0000 7fba7e000640 1 --2- 172.21.15.47:0/2527755366 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b7e5fae0f0 0x55b7e5fb6640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:32:37.144 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:37.144+0000 7fba7e000640 1 -- 172.21.15.47:0/2527755366 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b7e5fb7460 msgr2=0x55b7e5fb27e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:32:37.144 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:37.144+0000 7fba7e000640 1 --2- 172.21.15.47:0/2527755366 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b7e5fb7460 0x55b7e5fb27e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:32:37.144 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:37.144+0000 7fba7e000640 1 -- 172.21.15.47:0/2527755366 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b7e5be60f0 con 0x55b7e5faedd0 2024-03-21T10:32:37.144 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:37.144+0000 7fba7ea00640 1 --2- 172.21.15.47:0/2527755366 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b7e5fae0f0 0x55b7e5fb6640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T10:32:37.144 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:37.144+0000 7fba7e000640 1 --2- 172.21.15.47:0/2527755366 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b7e5faedd0 0x55b7e5fb6b80 secure :-1 s=READY pgs=1084 cs=0 l=1 rev1=1 crypto rx=0x7fba6c02c190 tx=0x7fba6c005c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=47d98154120eae94 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:32:37.144 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:37.144+0000 7fba6b400640 1 -- 172.21.15.47:0/2527755366 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba6c005d50 con 0x55b7e5faedd0 2024-03-21T10:32:37.144 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:37.144+0000 7fba6b400640 1 -- 172.21.15.47:0/2527755366 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fba6c048040 con 0x55b7e5faedd0 2024-03-21T10:32:37.144 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:37.144+0000 7fba6b400640 1 -- 172.21.15.47:0/2527755366 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba6c003800 con 0x55b7e5faedd0 2024-03-21T10:32:37.145 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:37.145+0000 7fba88775740 1 -- 172.21.15.47:0/2527755366 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b7e5faedd0 msgr2=0x55b7e5fb6b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:32:37.145 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:37.145+0000 7fba88775740 1 --2- 172.21.15.47:0/2527755366 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b7e5faedd0 0x55b7e5fb6b80 secure :-1 s=READY pgs=1084 cs=0 l=1 rev1=1 crypto rx=0x7fba6c02c190 tx=0x7fba6c005c00 comp rx=0 tx=0).stop 2024-03-21T10:32:37.145 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:37.145+0000 7fba88775740 1 -- 172.21.15.47:0/2527755366 shutdown_connections 2024-03-21T10:32:37.145 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:37.145+0000 7fba88775740 1 --2- 172.21.15.47:0/2527755366 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b7e5fb7460 0x55b7e5fb27e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:32:37.145 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:37.145+0000 7fba88775740 1 --2- 172.21.15.47:0/2527755366 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b7e5faedd0 0x55b7e5fb6b80 unknown :-1 s=CLOSED pgs=1084 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:32:37.145 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:37.145+0000 7fba88775740 1 --2- 172.21.15.47:0/2527755366 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b7e5fae0f0 0x55b7e5fb6640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:32:37.145 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:37.145+0000 7fba88775740 1 -- 172.21.15.47:0/2527755366 >> 172.21.15.47:0/2527755366 conn(0x55b7e5f75200 msgr2=0x55b7e5fa8670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:32:37.145 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:37.145+0000 7fba88775740 1 -- 172.21.15.47:0/2527755366 shutdown_connections 2024-03-21T10:32:37.145 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:37.145+0000 7fba88775740 1 -- 172.21.15.47:0/2527755366 wait complete. 2024-03-21T10:32:37.147 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:37.146+0000 7fba88775740 1 -- start start 2024-03-21T10:32:37.148 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:37.146+0000 7fba88775740 1 Processor -- start 2024-03-21T10:32:37.148 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:37.146+0000 7fba88775740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b7e5fae0f0 0x55b7e5fc9870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:32:37.148 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:37.146+0000 7fba88775740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b7e5faedd0 0x55b7e5fc9db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:32:37.148 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:37.146+0000 7fba88775740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b7e5fb7460 0x55b7e5fca2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:32:37.148 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:37.146+0000 7fba88775740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba801b1bd0 con 0x55b7e5faedd0 2024-03-21T10:32:37.148 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:37.146+0000 7fba88775740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba801b1570 con 0x55b7e5fae0f0 2024-03-21T10:32:37.148 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:37.146+0000 7fba88775740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba801aea90 con 0x55b7e5fb7460 2024-03-21T10:32:37.148 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:37.146+0000 7fba88775740 10 client.0 _ll_register_callbacks cb 0x7fba6c009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T10:32:37.148 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:37.147+0000 7fba7e000640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b7e5faedd0 0x55b7e5fc9db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:32:37.148 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:37.147+0000 7fba7e000640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b7e5faedd0 0x55b7e5fc9db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:51454/0 (socket says 172.21.15.47:51454) 2024-03-21T10:32:37.148 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:37.147+0000 7fba7e000640 1 -- 172.21.15.47:0/382637846 learned_addr learned my addr 172.21.15.47:0/382637846 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:32:37.148 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:37.147+0000 7fba7ea00640 1 --2- 172.21.15.47:0/382637846 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b7e5fae0f0 0x55b7e5fc9870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:32:37.148 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:37.147+0000 7fba7e000640 1 -- 172.21.15.47:0/382637846 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b7e5fb7460 msgr2=0x55b7e5fca2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:32:37.148 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:37.147+0000 7fba7e000640 1 --2- 172.21.15.47:0/382637846 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b7e5fb7460 0x55b7e5fca2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:32:37.149 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:37.147+0000 7fba7e000640 1 -- 172.21.15.47:0/382637846 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b7e5fae0f0 msgr2=0x55b7e5fc9870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:32:37.149 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:37.147+0000 7fba7e000640 1 --2- 172.21.15.47:0/382637846 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b7e5fae0f0 0x55b7e5fc9870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:32:37.149 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:37.147+0000 7fba7e000640 1 -- 172.21.15.47:0/382637846 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55b7e6021980 con 0x55b7e5faedd0 2024-03-21T10:32:37.149 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:37.147+0000 7fba7e000640 1 --2- 172.21.15.47:0/382637846 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b7e5faedd0 0x55b7e5fc9db0 secure :-1 s=READY pgs=1085 cs=0 l=1 rev1=1 crypto rx=0x7fba6c02c740 tx=0x7fba6c0344e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=760f0cf3d3fc5be0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:32:37.149 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:37.147+0000 7fba68c00640 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-03-21T10:32:37.149 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:37.148+0000 7fba68c00640 1 -- 172.21.15.47:0/382637846 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba6c02ca50 con 0x55b7e5faedd0 2024-03-21T10:32:37.149 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:37.148+0000 7fba88775740 1 -- 172.21.15.47:0/382637846 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fba6c048040 con 0x55b7e5faedd0 2024-03-21T10:32:37.149 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:37.148+0000 7fba68c00640 1 -- 172.21.15.47:0/382637846 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(99..99 src has 1..99) v4 ==== 9562+0+0 (secure 0 0 0) 0x7fba6c034a20 con 0x55b7e5faedd0 2024-03-21T10:32:37.149 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:37.148+0000 7fba68c00640 1 -- 172.21.15.47:0/382637846 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7fba6c071050 con 0x55b7e5faedd0 2024-03-21T10:32:37.149 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:37.148+0000 7fba88775740 10 client.17991 fetch_fsmap learned FSMap version 32 2024-03-21T10:32:37.149 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:37.148+0000 7fba88775740 1 -- 172.21.15.47:0/382637846 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x55b7e6022bc0 con 0x55b7e5faedd0 2024-03-21T10:32:37.149 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:37.148+0000 7fba68c00640 1 -- 172.21.15.47:0/382637846 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fba6c048040 con 0x55b7e5faedd0 2024-03-21T10:32:37.149 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:37.148+0000 7fba68c00640 1 -- 172.21.15.47:0/382637846 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 32) v1 ==== 7942+0+0 (secure 0 0 0) 0x7fba6c03c3f0 con 0x55b7e5faedd0 2024-03-21T10:32:37.149 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:37.148+0000 7fba88775740 10 client.17991 fetch_fsmap finished waiting for FSMap version 32 2024-03-21T10:32:37.149 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:37.148+0000 7fba88775740 10 client.17991 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-21T10:32:37.150 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:37.149+0000 7fba88775740 20 client.17991 populate_metadata read hostname 'smithi047' 2024-03-21T10:32:37.150 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:37.149+0000 7fba88775740 1 --2- 172.21.15.47:0/382637846 >> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] conn(0x55b7e60231f0 0x55b7e6025610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:32:37.150 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:37.149+0000 7fba88775740 4 client.17991 mds_command: new command op to 34535 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-21T10:32:37.150 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:37.149+0000 7fba88775740 1 -- 172.21.15.47:0/382637846 --> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55b7e5f19ae0 con 0x55b7e60231f0 2024-03-21T10:32:37.150 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:37.149+0000 7fba7ea00640 1 --2- 172.21.15.47:0/382637846 >> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] conn(0x55b7e60231f0 0x55b7e6025610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:32:37.151 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:37.150+0000 7fba68c00640 10 client.17991 ms_handle_connect on v2:172.21.15.100:6838/4173168440 2024-03-21T10:32:37.151 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:37.150+0000 7fba7ea00640 1 --2- 172.21.15.47:0/382637846 >> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] conn(0x55b7e60231f0 0x55b7e6025610 crc :-1 s=READY pgs=182 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=36c3d0870850177f server_cookie=fc676516dea1d2b0 in_seq=0 out_seq=0 2024-03-21T10:32:37.151 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:37.150+0000 7fba68c00640 1 -- 172.21.15.47:0/382637846 <== mds.0 v2:172.21.15.100:6838/4173168440 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x55b7e5f19ae0 con 0x55b7e60231f0 2024-03-21T10:32:37.151 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:37.150+0000 7fba68c00640 10 client.17991 handle_command_reply: tid=0 2024-03-21T10:32:37.151 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:37.150+0000 7fba88775740 1 client.17991 shutdown 2024-03-21T10:32:37.151 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:37.150+0000 7fba88775740 1 -- 172.21.15.47:0/382637846 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b7e5faedd0 msgr2=0x55b7e5fc9db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:32:37.152 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:37.150+0000 7fba88775740 1 --2- 172.21.15.47:0/382637846 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b7e5faedd0 0x55b7e5fc9db0 secure :-1 s=READY pgs=1085 cs=0 l=1 rev1=1 crypto rx=0x7fba6c02c740 tx=0x7fba6c0344e0 comp rx=0 tx=0).stop 2024-03-21T10:32:37.152 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:37.150+0000 7fba88775740 1 -- 172.21.15.47:0/382637846 shutdown_connections 2024-03-21T10:32:37.152 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:37.150+0000 7fba88775740 1 --2- 172.21.15.47:0/382637846 >> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] conn(0x55b7e60231f0 0x55b7e6025610 crc :-1 s=READY pgs=182 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:32:37.152 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:37.150+0000 7fba88775740 1 --2- 172.21.15.47:0/382637846 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b7e5fb7460 0x55b7e5fca2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:32:37.152 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:37.150+0000 7fba88775740 1 --2- 172.21.15.47:0/382637846 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b7e5faedd0 0x55b7e5fc9db0 unknown :-1 s=CLOSED pgs=1085 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:32:37.152 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:37.150+0000 7fba88775740 1 --2- 172.21.15.47:0/382637846 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b7e5fae0f0 0x55b7e5fc9870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:32:37.152 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:37.150+0000 7fba88775740 1 -- 172.21.15.47:0/382637846 >> 172.21.15.47:0/382637846 conn(0x55b7e5f75200 msgr2=0x55b7e5fb27a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:32:37.152 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:37.150+0000 7fba68c00640 0 client.17991 ms_handle_reset on v2:172.21.15.100:6838/4173168440 2024-03-21T10:32:37.152 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:37.150+0000 7fba88775740 1 -- 172.21.15.47:0/382637846 shutdown_connections 2024-03-21T10:32:37.152 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:37.150+0000 7fba88775740 1 -- 172.21.15.47:0/382637846 wait complete. 2024-03-21T10:32:37.152 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:37.151+0000 7fba88775740 20 client.17991 trim_cache size 0 max 16384 2024-03-21T10:32:37.153 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-03-21T10:32:37.153 INFO:teuthology.orchestra.run.smithi047.stdout: "status": "IDLE (waiting for more scrubs)", 2024-03-21T10:32:37.153 INFO:teuthology.orchestra.run.smithi047.stdout: "scrubs": { 2024-03-21T10:32:37.153 INFO:teuthology.orchestra.run.smithi047.stdout: "a93db2e1-1155-41d7-b775-a2dbb387576e": { 2024-03-21T10:32:37.153 INFO:teuthology.orchestra.run.smithi047.stdout: "path": "/", 2024-03-21T10:32:37.153 INFO:teuthology.orchestra.run.smithi047.stdout: "tag": "a93db2e1-1155-41d7-b775-a2dbb387576e", 2024-03-21T10:32:37.153 INFO:teuthology.orchestra.run.smithi047.stdout: "options": "recursive,force" 2024-03-21T10:32:37.153 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-21T10:32:37.153 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-21T10:32:37.153 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-03-21T10:32:37.154 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:37.151+0000 7fba7fe00640 1 -- 172.21.15.47:0/3382097738 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7fba640a30c0 msgr2=0x7fba640a5580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:32:37.154 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:37.151+0000 7fba7fe00640 1 --2- 172.21.15.47:0/3382097738 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7fba640a30c0 0x7fba640a5580 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x55b7e5faf320 tx=0x7fba7000b390 comp rx=0 tx=0).stop 2024-03-21T10:32:37.154 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:37.151+0000 7fba7fe00640 1 -- 172.21.15.47:0/3382097738 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fba801064a0 msgr2=0x7fba8010e7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:32:37.154 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:37.151+0000 7fba7fe00640 1 --2- 172.21.15.47:0/3382097738 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fba801064a0 0x7fba8010e7e0 secure :-1 s=READY pgs=1081 cs=0 l=1 rev1=1 crypto rx=0x7fba7800eb10 tx=0x7fba7800c2d0 comp rx=0 tx=0).stop 2024-03-21T10:32:37.154 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:37.151+0000 7fba7fe00640 1 -- 172.21.15.47:0/3382097738 shutdown_connections 2024-03-21T10:32:37.154 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:37.151+0000 7fba7fe00640 1 --2- 172.21.15.47:0/3382097738 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fba801064a0 0x7fba8010e7e0 unknown :-1 s=CLOSED pgs=1081 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:32:37.154 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:37.152+0000 7fba7fe00640 1 --2- 172.21.15.47:0/3382097738 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fba80100dc0 0x7fba8010e2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:32:37.154 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:37.152+0000 7fba7fe00640 1 --2- 172.21.15.47:0/3382097738 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7fba640a30c0 0x7fba640a5580 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:32:37.154 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:37.152+0000 7fba7fe00640 1 --2- 172.21.15.47:0/3382097738 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fba80100400 0x7fba8010dd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:32:37.155 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:37.152+0000 7fba7fe00640 1 -- 172.21.15.47:0/3382097738 >> 172.21.15.47:0/3382097738 conn(0x7fba800fbef0 msgr2=0x7fba800fc710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:32:37.155 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:37.152+0000 7fba7fe00640 1 -- 172.21.15.47:0/3382097738 shutdown_connections 2024-03-21T10:32:37.155 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:32:37.152+0000 7fba7fe00640 1 -- 172.21.15.47:0/3382097738 wait complete. 2024-03-21T10:32:37.178 INFO:tasks.cephfs.filesystem:scrub status for tag:a93db2e1-1155-41d7-b775-a2dbb387576e - {'path': '/', 'tag': 'a93db2e1-1155-41d7-b775-a2dbb387576e', 'options': 'recursive,force'} 2024-03-21T10:32:37.711 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:32:37 smithi047 ceph-mon[31547]: pgmap v224: 129 pgs: 129 active+clean; 277 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 12 op/s 2024-03-21T10:32:37.727 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:32:37 smithi100 ceph-mon[35193]: pgmap v224: 129 pgs: 129 active+clean; 277 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 12 op/s 2024-03-21T10:32:37.737 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:32:37 smithi157 ceph-mon[35581]: pgmap v224: 129 pgs: 129 active+clean; 277 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 12 op/s 2024-03-21T10:32:38.711 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:32:38 smithi047 ceph-mon[31547]: scrub queued for path: / 2024-03-21T10:32:38.711 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:32:38 smithi047 ceph-mon[31547]: scrub summary: idle+waiting paths [/] 2024-03-21T10:32:38.712 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:32:38 smithi047 ceph-mon[31547]: scrub summary: active paths [/] 2024-03-21T10:32:38.712 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:32:38 smithi047 ceph-mon[31547]: scrub summary: idle+waiting paths [/] 2024-03-21T10:32:38.727 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:32:38 smithi100 ceph-mon[35193]: scrub queued for path: / 2024-03-21T10:32:38.727 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:32:38 smithi100 ceph-mon[35193]: scrub summary: idle+waiting paths [/] 2024-03-21T10:32:38.727 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:32:38 smithi100 ceph-mon[35193]: scrub summary: active paths [/] 2024-03-21T10:32:38.727 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:32:38 smithi100 ceph-mon[35193]: scrub summary: idle+waiting paths [/] 2024-03-21T10:32:38.737 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:32:38 smithi157 ceph-mon[35581]: scrub queued for path: / 2024-03-21T10:32:38.737 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:32:38 smithi157 ceph-mon[35581]: scrub summary: idle+waiting paths [/] 2024-03-21T10:32:38.737 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:32:38 smithi157 ceph-mon[35581]: scrub summary: active paths [/] 2024-03-21T10:32:38.737 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:32:38 smithi157 ceph-mon[35581]: scrub summary: idle+waiting paths [/] 2024-03-21T10:32:39.711 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:32:39 smithi047 ceph-mon[31547]: pgmap v225: 129 pgs: 129 active+clean; 277 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 9 op/s 2024-03-21T10:32:39.711 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:32:39 smithi047 ceph-mon[31547]: scrub summary: idle 2024-03-21T10:32:39.727 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:32:39 smithi100 ceph-mon[35193]: pgmap v225: 129 pgs: 129 active+clean; 277 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 9 op/s 2024-03-21T10:32:39.727 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:32:39 smithi100 ceph-mon[35193]: scrub summary: idle 2024-03-21T10:32:39.737 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:32:39 smithi157 ceph-mon[35581]: pgmap v225: 129 pgs: 129 active+clean; 277 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 9 op/s 2024-03-21T10:32:39.737 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:32:39 smithi157 ceph-mon[35581]: scrub summary: idle 2024-03-21T10:32:41.539 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:32:41 smithi157 ceph-mon[35581]: pgmap v226: 129 pgs: 129 active+clean; 277 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 9 op/s 2024-03-21T10:32:41.711 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:32:41 smithi047 ceph-mon[31547]: pgmap v226: 129 pgs: 129 active+clean; 277 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 9 op/s 2024-03-21T10:32:41.726 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:32:41 smithi100 ceph-mon[35193]: pgmap v226: 129 pgs: 129 active+clean; 277 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 9 op/s 2024-03-21T10:32:42.711 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:32:42 smithi047 ceph-mon[31547]: from='mgr.15687 172.21.15.47:0/8730661' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T10:32:42.727 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:32:42 smithi100 ceph-mon[35193]: from='mgr.15687 172.21.15.47:0/8730661' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T10:32:42.737 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:32:42 smithi157 ceph-mon[35581]: from='mgr.15687 172.21.15.47:0/8730661' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T10:32:43.711 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:32:43 smithi047 ceph-mon[31547]: pgmap v227: 129 pgs: 129 active+clean; 277 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.6 KiB/s rd, 14 op/s 2024-03-21T10:32:43.712 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:32:43 smithi047 ceph-mon[31547]: from='mgr.15687 172.21.15.47:0/8730661' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T10:32:43.712 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:32:43 smithi047 ceph-mon[31547]: from='mgr.15687 172.21.15.47:0/8730661' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T10:32:43.712 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:32:43 smithi047 ceph-mon[31547]: from='mgr.15687 172.21.15.47:0/8730661' entity='mgr.x' 2024-03-21T10:32:43.727 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:32:43 smithi100 ceph-mon[35193]: pgmap v227: 129 pgs: 129 active+clean; 277 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.6 KiB/s rd, 14 op/s 2024-03-21T10:32:43.727 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:32:43 smithi100 ceph-mon[35193]: from='mgr.15687 172.21.15.47:0/8730661' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T10:32:43.727 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:32:43 smithi100 ceph-mon[35193]: from='mgr.15687 172.21.15.47:0/8730661' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T10:32:43.727 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:32:43 smithi100 ceph-mon[35193]: from='mgr.15687 172.21.15.47:0/8730661' entity='mgr.x' 2024-03-21T10:32:43.737 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:32:43 smithi157 ceph-mon[35581]: pgmap v227: 129 pgs: 129 active+clean; 277 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.6 KiB/s rd, 14 op/s 2024-03-21T10:32:43.737 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:32:43 smithi157 ceph-mon[35581]: from='mgr.15687 172.21.15.47:0/8730661' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T10:32:43.737 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:32:43 smithi157 ceph-mon[35581]: from='mgr.15687 172.21.15.47:0/8730661' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T10:32:43.737 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:32:43 smithi157 ceph-mon[35581]: from='mgr.15687 172.21.15.47:0/8730661' entity='mgr.x' 2024-03-21T10:32:45.711 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:32:45 smithi047 ceph-mon[31547]: pgmap v228: 129 pgs: 129 active+clean; 277 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 10 op/s 2024-03-21T10:32:45.726 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:32:45 smithi100 ceph-mon[35193]: pgmap v228: 129 pgs: 129 active+clean; 277 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 10 op/s 2024-03-21T10:32:45.737 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:32:45 smithi157 ceph-mon[35581]: pgmap v228: 129 pgs: 129 active+clean; 277 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 10 op/s 2024-03-21T10:32:47.711 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:32:47 smithi047 ceph-mon[31547]: pgmap v229: 129 pgs: 129 active+clean; 277 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 13 op/s 2024-03-21T10:32:47.726 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:32:47 smithi100 ceph-mon[35193]: pgmap v229: 129 pgs: 129 active+clean; 277 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 13 op/s 2024-03-21T10:32:47.737 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:32:47 smithi157 ceph-mon[35581]: pgmap v229: 129 pgs: 129 active+clean; 277 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 13 op/s 2024-03-21T10:32:49.711 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:32:49 smithi047 ceph-mon[31547]: pgmap v230: 129 pgs: 129 active+clean; 277 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 9 op/s 2024-03-21T10:32:49.727 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:32:49 smithi100 ceph-mon[35193]: pgmap v230: 129 pgs: 129 active+clean; 277 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 9 op/s 2024-03-21T10:32:49.737 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:32:49 smithi157 ceph-mon[35581]: pgmap v230: 129 pgs: 129 active+clean; 277 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 9 op/s 2024-03-21T10:32:51.711 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:32:51 smithi047 ceph-mon[31547]: pgmap v231: 129 pgs: 129 active+clean; 277 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 9 op/s 2024-03-21T10:32:51.727 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:32:51 smithi100 ceph-mon[35193]: pgmap v231: 129 pgs: 129 active+clean; 277 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 9 op/s 2024-03-21T10:32:51.737 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:32:51 smithi157 ceph-mon[35581]: pgmap v231: 129 pgs: 129 active+clean; 277 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 9 op/s 2024-03-21T10:32:53.711 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:32:53 smithi047 ceph-mon[31547]: pgmap v232: 129 pgs: 129 active+clean; 277 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 13 op/s 2024-03-21T10:32:53.727 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:32:53 smithi100 ceph-mon[35193]: pgmap v232: 129 pgs: 129 active+clean; 277 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 13 op/s 2024-03-21T10:32:53.737 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:32:53 smithi157 ceph-mon[35581]: pgmap v232: 129 pgs: 129 active+clean; 277 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 13 op/s 2024-03-21T10:32:55.711 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:32:55 smithi047 ceph-mon[31547]: pgmap v233: 129 pgs: 129 active+clean; 277 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-21T10:32:55.726 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:32:55 smithi100 ceph-mon[35193]: pgmap v233: 129 pgs: 129 active+clean; 277 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-21T10:32:55.737 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:32:55 smithi157 ceph-mon[35581]: pgmap v233: 129 pgs: 129 active+clean; 277 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-21T10:32:57.711 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:32:57 smithi047 ceph-mon[31547]: pgmap v234: 129 pgs: 129 active+clean; 277 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 12 op/s 2024-03-21T10:32:57.726 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:32:57 smithi100 ceph-mon[35193]: pgmap v234: 129 pgs: 129 active+clean; 277 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 12 op/s 2024-03-21T10:32:57.737 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:32:57 smithi157 ceph-mon[35581]: pgmap v234: 129 pgs: 129 active+clean; 277 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 12 op/s 2024-03-21T10:32:59.711 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:32:59 smithi047 ceph-mon[31547]: pgmap v235: 129 pgs: 129 active+clean; 277 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-03-21T10:32:59.824 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:32:59 smithi157 ceph-mon[35581]: pgmap v235: 129 pgs: 129 active+clean; 277 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-03-21T10:32:59.824 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:32:59 smithi100 ceph-mon[35193]: pgmap v235: 129 pgs: 129 active+clean; 277 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-03-21T10:33:01.711 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:33:01 smithi047 ceph-mon[31547]: pgmap v236: 129 pgs: 129 active+clean; 277 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-21T10:33:01.726 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:33:01 smithi100 ceph-mon[35193]: pgmap v236: 129 pgs: 129 active+clean; 277 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-21T10:33:01.737 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:33:01 smithi157 ceph-mon[35581]: pgmap v236: 129 pgs: 129 active+clean; 277 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-21T10:33:03.711 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:33:03 smithi047 ceph-mon[31547]: pgmap v237: 129 pgs: 129 active+clean; 277 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 44 KiB/s wr, 29 op/s 2024-03-21T10:33:03.726 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:33:03 smithi100 ceph-mon[35193]: pgmap v237: 129 pgs: 129 active+clean; 277 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 44 KiB/s wr, 29 op/s 2024-03-21T10:33:03.737 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:33:03 smithi157 ceph-mon[35581]: pgmap v237: 129 pgs: 129 active+clean; 277 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 44 KiB/s wr, 29 op/s 2024-03-21T10:33:05.711 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:33:05 smithi047 ceph-mon[31547]: pgmap v238: 129 pgs: 129 active+clean; 277 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 44 KiB/s wr, 25 op/s 2024-03-21T10:33:05.727 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:33:05 smithi100 ceph-mon[35193]: pgmap v238: 129 pgs: 129 active+clean; 277 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 44 KiB/s wr, 25 op/s 2024-03-21T10:33:05.737 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:33:05 smithi157 ceph-mon[35581]: pgmap v238: 129 pgs: 129 active+clean; 277 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 44 KiB/s wr, 25 op/s 2024-03-21T10:33:07.180 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-21T10:33:07.180 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-21T10:33:07.424 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:07.425+0000 7fc88ba00640 1 -- 172.21.15.47:0/659247272 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc884109850 msgr2=0x7fc884109c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:33:07.425 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:07.425+0000 7fc88ba00640 1 --2- 172.21.15.47:0/659247272 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc884109850 0x7fc884109c30 secure :-1 s=READY pgs=1086 cs=0 l=1 rev1=1 crypto rx=0x7fc874009930 tx=0x7fc87402bc30 comp rx=0 tx=0).stop 2024-03-21T10:33:07.425 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:07.426+0000 7fc88ba00640 1 -- 172.21.15.47:0/659247272 shutdown_connections 2024-03-21T10:33:07.425 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:07.426+0000 7fc88ba00640 1 --2- 172.21.15.47:0/659247272 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc884104200 0x7fc884110710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:33:07.425 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:07.426+0000 7fc88ba00640 1 --2- 172.21.15.47:0/659247272 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc884103840 0x7fc884103cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:33:07.425 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:07.426+0000 7fc88ba00640 1 --2- 172.21.15.47:0/659247272 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc884109850 0x7fc884109c30 unknown :-1 s=CLOSED pgs=1086 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:33:07.425 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:07.426+0000 7fc88ba00640 1 -- 172.21.15.47:0/659247272 >> 172.21.15.47:0/659247272 conn(0x7fc8840ff310 msgr2=0x7fc884101730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:33:07.425 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:07.426+0000 7fc88ba00640 1 -- 172.21.15.47:0/659247272 shutdown_connections 2024-03-21T10:33:07.425 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:07.426+0000 7fc88ba00640 1 -- 172.21.15.47:0/659247272 wait complete. 2024-03-21T10:33:07.427 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:07.427+0000 7fc88ba00640 1 Processor -- start 2024-03-21T10:33:07.427 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:07.427+0000 7fc88ba00640 1 -- start start 2024-03-21T10:33:07.427 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:07.428+0000 7fc88ba00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc884103840 0x7fc88419a070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:33:07.427 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:07.428+0000 7fc88ba00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc884104200 0x7fc88419a5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:33:07.427 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:07.428+0000 7fc88ba00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc884109850 0x7fc88419e930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:33:07.427 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:07.428+0000 7fc88ba00640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc884113ad0 con 0x7fc884104200 2024-03-21T10:33:07.427 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:07.428+0000 7fc88ba00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc884113950 con 0x7fc884109850 2024-03-21T10:33:07.427 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:07.428+0000 7fc88ba00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc884113c50 con 0x7fc884103840 2024-03-21T10:33:07.427 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:07.428+0000 7fc87be00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc884104200 0x7fc88419a5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:33:07.427 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:07.428+0000 7fc87be00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc884104200 0x7fc88419a5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:46720/0 (socket says 172.21.15.47:46720) 2024-03-21T10:33:07.427 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:07.428+0000 7fc87be00640 1 -- 172.21.15.47:0/1835123701 learned_addr learned my addr 172.21.15.47:0/1835123701 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:33:07.427 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:07.428+0000 7fc889a00640 1 --2- 172.21.15.47:0/1835123701 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc884109850 0x7fc88419e930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:33:07.428 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:07.428+0000 7fc889000640 1 --2- 172.21.15.47:0/1835123701 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc884103840 0x7fc88419a070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:33:07.428 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:07.429+0000 7fc87be00640 1 -- 172.21.15.47:0/1835123701 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc884103840 msgr2=0x7fc88419a070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:33:07.428 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:07.429+0000 7fc87be00640 1 --2- 172.21.15.47:0/1835123701 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc884103840 0x7fc88419a070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:33:07.428 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:07.429+0000 7fc87be00640 1 -- 172.21.15.47:0/1835123701 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc884109850 msgr2=0x7fc88419e930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:33:07.428 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:07.429+0000 7fc87be00640 1 --2- 172.21.15.47:0/1835123701 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc884109850 0x7fc88419e930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:33:07.428 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:07.429+0000 7fc87be00640 1 -- 172.21.15.47:0/1835123701 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc88419f060 con 0x7fc884104200 2024-03-21T10:33:07.428 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:07.429+0000 7fc889000640 1 --2- 172.21.15.47:0/1835123701 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc884103840 0x7fc88419a070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T10:33:07.429 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:07.429+0000 7fc87be00640 1 --2- 172.21.15.47:0/1835123701 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc884104200 0x7fc88419a5b0 secure :-1 s=READY pgs=1087 cs=0 l=1 rev1=1 crypto rx=0x7fc86c00d950 tx=0x7fc86c00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:33:07.429 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:07.429+0000 7fc882000640 1 -- 172.21.15.47:0/1835123701 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc86c017070 con 0x7fc884104200 2024-03-21T10:33:07.429 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:07.429+0000 7fc88ba00640 1 -- 172.21.15.47:0/1835123701 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc88419f350 con 0x7fc884104200 2024-03-21T10:33:07.429 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:07.429+0000 7fc882000640 1 -- 172.21.15.47:0/1835123701 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fc86c009c30 con 0x7fc884104200 2024-03-21T10:33:07.429 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:07.429+0000 7fc882000640 1 -- 172.21.15.47:0/1835123701 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc86c00f620 con 0x7fc884104200 2024-03-21T10:33:07.429 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:07.430+0000 7fc88ba00640 1 -- 172.21.15.47:0/1835123701 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc8841a6be0 con 0x7fc884104200 2024-03-21T10:33:07.431 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:07.433+0000 7fc88e3fd740 1 Processor -- start 2024-03-21T10:33:07.431 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:07.433+0000 7fc88e3fd740 1 -- start start 2024-03-21T10:33:07.432 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:07.433+0000 7fc88e3fd740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556c43c8cc00 0x556c43c8b950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:33:07.432 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:07.433+0000 7fc88e3fd740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x556c43c8bf20 0x556c43c8c3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:33:07.432 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:07.433+0000 7fc88e3fd740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556c43c95290 0x556c43c97680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:33:07.433 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:07.433+0000 7fc88e3fd740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x556c43bf0250 con 0x556c43c8bf20 2024-03-21T10:33:07.433 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:07.433+0000 7fc88e3fd740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x556c43b98a30 con 0x556c43c8cc00 2024-03-21T10:33:07.433 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:07.433+0000 7fc88e3fd740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x556c43bec2f0 con 0x556c43c95290 2024-03-21T10:33:07.433 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:07.434+0000 7fc889000640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x556c43c8bf20 0x556c43c8c3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:33:07.433 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:07.434+0000 7fc889a00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556c43c8cc00 0x556c43c8b950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:33:07.433 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:07.434+0000 7fc889000640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x556c43c8bf20 0x556c43c8c3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:46724/0 (socket says 172.21.15.47:46724) 2024-03-21T10:33:07.433 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:07.434+0000 7fc889000640 1 -- 172.21.15.47:0/4017915253 learned_addr learned my addr 172.21.15.47:0/4017915253 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:33:07.433 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:07.434+0000 7fc889a00640 1 -- 172.21.15.47:0/4017915253 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556c43c95290 msgr2=0x556c43c97680 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-21T10:33:07.433 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:07.434+0000 7fc889a00640 1 --2- 172.21.15.47:0/4017915253 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556c43c95290 0x556c43c97680 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:33:07.433 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:07.434+0000 7fc889a00640 1 -- 172.21.15.47:0/4017915253 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x556c43c8bf20 msgr2=0x556c43c8c3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:33:07.433 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:07.434+0000 7fc889a00640 1 --2- 172.21.15.47:0/4017915253 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x556c43c8bf20 0x556c43c8c3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:33:07.433 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:07.434+0000 7fc889a00640 1 -- 172.21.15.47:0/4017915253 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556c43b83690 con 0x556c43c8cc00 2024-03-21T10:33:07.433 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:07.434+0000 7fc889000640 1 --2- 172.21.15.47:0/4017915253 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x556c43c8bf20 0x556c43c8c3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T10:33:07.437 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:07.434+0000 7fc882000640 1 -- 172.21.15.47:0/1835123701 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 27) v1 ==== 137985+0+0 (secure 0 0 0) 0x7fc86c0109a0 con 0x7fc884104200 2024-03-21T10:33:07.437 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:07.435+0000 7fc889a00640 1 --2- 172.21.15.47:0/4017915253 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556c43c8cc00 0x556c43c8b950 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fc87c009eb0 tx=0x7fc87c0107a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6079d92fbd5dc16f server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:33:07.437 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:07.435+0000 7fc87a000640 1 -- 172.21.15.47:0/4017915253 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc87c003b70 con 0x556c43c8cc00 2024-03-21T10:33:07.437 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:07.435+0000 7fc87a000640 1 -- 172.21.15.47:0/4017915253 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fc87c015070 con 0x556c43c8cc00 2024-03-21T10:33:07.437 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:07.435+0000 7fc87a000640 1 -- 172.21.15.47:0/4017915253 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc87c01a7a0 con 0x556c43c8cc00 2024-03-21T10:33:07.437 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:07.436+0000 7fc88e3fd740 1 -- 172.21.15.47:0/4017915253 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556c43c8cc00 msgr2=0x556c43c8b950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:33:07.437 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:07.436+0000 7fc88e3fd740 1 --2- 172.21.15.47:0/4017915253 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556c43c8cc00 0x556c43c8b950 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fc87c009eb0 tx=0x7fc87c0107a0 comp rx=0 tx=0).stop 2024-03-21T10:33:07.437 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:07.436+0000 7fc88e3fd740 1 -- 172.21.15.47:0/4017915253 shutdown_connections 2024-03-21T10:33:07.438 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:07.436+0000 7fc88e3fd740 1 --2- 172.21.15.47:0/4017915253 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556c43c95290 0x556c43c97680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:33:07.438 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:07.436+0000 7fc88e3fd740 1 --2- 172.21.15.47:0/4017915253 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x556c43c8bf20 0x556c43c8c3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:33:07.438 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:07.436+0000 7fc88e3fd740 1 --2- 172.21.15.47:0/4017915253 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556c43c8cc00 0x556c43c8b950 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:33:07.438 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:07.436+0000 7fc88e3fd740 1 -- 172.21.15.47:0/4017915253 >> 172.21.15.47:0/4017915253 conn(0x556c43c53200 msgr2=0x556c43c86970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:33:07.438 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:07.436+0000 7fc88e3fd740 1 -- 172.21.15.47:0/4017915253 shutdown_connections 2024-03-21T10:33:07.438 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:07.436+0000 7fc88e3fd740 1 -- 172.21.15.47:0/4017915253 wait complete. 2024-03-21T10:33:07.438 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:07.436+0000 7fc882000640 1 --2- 172.21.15.47:0/1835123701 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7fc8680a2fa0 0x7fc8680a5460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:33:07.438 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:07.436+0000 7fc88e3fd740 1 -- start start 2024-03-21T10:33:07.438 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:07.436+0000 7fc882000640 1 -- 172.21.15.47:0/1835123701 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 9562+0+0 (secure 0 0 0) 0x7fc86c0cf830 con 0x7fc884104200 2024-03-21T10:33:07.438 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:07.436+0000 7fc889000640 1 --2- 172.21.15.47:0/1835123701 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7fc8680a2fa0 0x7fc8680a5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:33:07.438 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:07.437+0000 7fc88e3fd740 1 Processor -- start 2024-03-21T10:33:07.438 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:07.437+0000 7fc88e3fd740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556c43c8bf20 0x556c43c94980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:33:07.439 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:07.437+0000 7fc88e3fd740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x556c43c8cc00 0x556c43c91d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:33:07.439 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:07.437+0000 7fc88e3fd740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556c43c95290 0x556c43c92250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:33:07.439 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:07.437+0000 7fc88e3fd740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x556c43c98100 con 0x556c43c8cc00 2024-03-21T10:33:07.439 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:07.437+0000 7fc88e3fd740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x556c43babe80 con 0x556c43c8bf20 2024-03-21T10:33:07.439 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:07.437+0000 7fc88e3fd740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x556c43c98280 con 0x556c43c95290 2024-03-21T10:33:07.439 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:07.437+0000 7fc88e3fd740 10 client.0 _ll_register_callbacks cb 0x556c43bbe130 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T10:33:07.439 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:07.438+0000 7fc889000640 1 --2- 172.21.15.47:0/1835123701 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7fc8680a2fa0 0x7fc8680a5460 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fc874006010 tx=0x7fc874005b50 comp rx=0 tx=0).ready entity=mgr.15687 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:33:07.439 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:07.438+0000 7fc87be00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x556c43c8cc00 0x556c43c91d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:33:07.439 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:07.438+0000 7fc87be00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x556c43c8cc00 0x556c43c91d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:46732/0 (socket says 172.21.15.47:46732) 2024-03-21T10:33:07.439 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:07.438+0000 7fc87be00640 1 -- 172.21.15.47:0/2733708368 learned_addr learned my addr 172.21.15.47:0/2733708368 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:33:07.439 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:07.438+0000 7fc889000640 1 --2- 172.21.15.47:0/2733708368 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556c43c8bf20 0x556c43c94980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:33:07.439 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:07.438+0000 7fc889a00640 1 --2- 172.21.15.47:0/2733708368 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556c43c95290 0x556c43c92250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:33:07.439 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:07.438+0000 7fc87be00640 1 -- 172.21.15.47:0/2733708368 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556c43c95290 msgr2=0x556c43c92250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:33:07.439 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:07.438+0000 7fc87be00640 1 --2- 172.21.15.47:0/2733708368 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556c43c95290 0x556c43c92250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:33:07.440 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:07.438+0000 7fc87be00640 1 -- 172.21.15.47:0/2733708368 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556c43c8bf20 msgr2=0x556c43c94980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:33:07.440 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:07.438+0000 7fc87be00640 1 --2- 172.21.15.47:0/2733708368 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556c43c8bf20 0x556c43c94980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:33:07.440 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:07.438+0000 7fc87be00640 1 -- 172.21.15.47:0/2733708368 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x556c439ce7d0 con 0x556c43c8cc00 2024-03-21T10:33:07.440 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:07.438+0000 7fc889000640 1 --2- 172.21.15.47:0/2733708368 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556c43c8bf20 0x556c43c94980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T10:33:07.440 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:07.438+0000 7fc87be00640 1 --2- 172.21.15.47:0/2733708368 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x556c43c8cc00 0x556c43c91d10 secure :-1 s=READY pgs=1088 cs=0 l=1 rev1=1 crypto rx=0x7fc86c0981e0 tx=0x7fc86c09b230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e059ce1b4e711d11 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:33:07.440 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:07.438+0000 7fc872a00640 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-03-21T10:33:07.440 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:07.439+0000 7fc872a00640 1 -- 172.21.15.47:0/2733708368 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc86c0afe60 con 0x556c43c8cc00 2024-03-21T10:33:07.440 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:07.439+0000 7fc872a00640 1 -- 172.21.15.47:0/2733708368 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(99..99 src has 1..99) v4 ==== 9562+0+0 (secure 0 0 0) 0x7fc86c097e80 con 0x556c43c8cc00 2024-03-21T10:33:07.440 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:07.439+0000 7fc88e3fd740 1 -- 172.21.15.47:0/2733708368 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fc87c015070 con 0x556c43c8cc00 2024-03-21T10:33:07.442 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:07.439+0000 7fc872a00640 1 -- 172.21.15.47:0/2733708368 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc86c020070 con 0x556c43c8cc00 2024-03-21T10:33:07.442 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:07.439+0000 7fc88e3fd740 10 client.18003 fetch_fsmap learned FSMap version 32 2024-03-21T10:33:07.443 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:07.439+0000 7fc88e3fd740 1 -- 172.21.15.47:0/2733708368 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x556c439043f0 con 0x556c43c8cc00 2024-03-21T10:33:07.443 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:07.439+0000 7fc872a00640 1 -- 172.21.15.47:0/2733708368 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fc87c015070 con 0x556c43c8cc00 2024-03-21T10:33:07.443 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:07.440+0000 7fc872a00640 1 -- 172.21.15.47:0/2733708368 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 32) v1 ==== 7942+0+0 (secure 0 0 0) 0x7fc86c0cf400 con 0x556c43c8cc00 2024-03-21T10:33:07.443 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:07.440+0000 7fc88e3fd740 10 client.18003 fetch_fsmap finished waiting for FSMap version 32 2024-03-21T10:33:07.443 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:07.440+0000 7fc88e3fd740 10 client.18003 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-21T10:33:07.443 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:07.440+0000 7fc88e3fd740 20 client.18003 populate_metadata read hostname 'smithi047' 2024-03-21T10:33:07.443 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:07.440+0000 7fc88e3fd740 1 --2- 172.21.15.47:0/2733708368 >> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] conn(0x556c43ca5220 0x556c43ca7610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:33:07.443 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:07.440+0000 7fc88e3fd740 4 client.18003 mds_command: new command op to 34535 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T10:33:07.443 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:07.440+0000 7fc88e3fd740 1 -- 172.21.15.47:0/2733708368 --> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x556c43bfd790 con 0x556c43ca5220 2024-03-21T10:33:07.443 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:07.441+0000 7fc889000640 1 --2- 172.21.15.47:0/2733708368 >> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] conn(0x556c43ca5220 0x556c43ca7610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:33:07.443 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:07.441+0000 7fc889000640 1 --2- 172.21.15.47:0/2733708368 >> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] conn(0x556c43ca5220 0x556c43ca7610 crc :-1 s=READY pgs=183 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=751a0a4eaccf2833 server_cookie=c8b62f1bbec80ed1 in_seq=0 out_seq=0 2024-03-21T10:33:07.443 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:07.441+0000 7fc872a00640 10 client.18003 ms_handle_connect on v2:172.21.15.100:6838/4173168440 2024-03-21T10:33:07.443 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:07.443+0000 7fc872a00640 1 -- 172.21.15.47:0/2733708368 <== mds.0 v2:172.21.15.100:6838/4173168440 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x556c43bfd790 con 0x556c43ca5220 2024-03-21T10:33:07.444 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:07.443+0000 7fc872a00640 10 client.18003 handle_command_reply: tid=0 2024-03-21T10:33:07.444 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:07.443+0000 7fc88e3fd740 1 client.18003 shutdown 2024-03-21T10:33:07.444 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:07.443+0000 7fc88e3fd740 1 -- 172.21.15.47:0/2733708368 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x556c43c8cc00 msgr2=0x556c43c91d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:33:07.444 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:07.443+0000 7fc88e3fd740 1 --2- 172.21.15.47:0/2733708368 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x556c43c8cc00 0x556c43c91d10 secure :-1 s=READY pgs=1088 cs=0 l=1 rev1=1 crypto rx=0x7fc86c0981e0 tx=0x7fc86c09b230 comp rx=0 tx=0).stop 2024-03-21T10:33:07.444 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:07.444+0000 7fc88e3fd740 1 -- 172.21.15.47:0/2733708368 shutdown_connections 2024-03-21T10:33:07.444 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:07.444+0000 7fc88e3fd740 1 --2- 172.21.15.47:0/2733708368 >> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] conn(0x556c43ca5220 0x556c43ca7610 crc :-1 s=READY pgs=183 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:33:07.444 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:07.444+0000 7fc88e3fd740 1 --2- 172.21.15.47:0/2733708368 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556c43c95290 0x556c43c92250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:33:07.444 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:07.444+0000 7fc88e3fd740 1 --2- 172.21.15.47:0/2733708368 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x556c43c8cc00 0x556c43c91d10 unknown :-1 s=CLOSED pgs=1088 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:33:07.444 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:07.444+0000 7fc88e3fd740 1 --2- 172.21.15.47:0/2733708368 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556c43c8bf20 0x556c43c94980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:33:07.444 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:07.444+0000 7fc88e3fd740 1 -- 172.21.15.47:0/2733708368 >> 172.21.15.47:0/2733708368 conn(0x556c43c53200 msgr2=0x556c43c886c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:33:07.444 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:07.444+0000 7fc872a00640 0 client.18003 ms_handle_reset on v2:172.21.15.100:6838/4173168440 2024-03-21T10:33:07.444 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:07.444+0000 7fc88e3fd740 1 -- 172.21.15.47:0/2733708368 shutdown_connections 2024-03-21T10:33:07.444 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:07.444+0000 7fc88e3fd740 1 -- 172.21.15.47:0/2733708368 wait complete. 2024-03-21T10:33:07.444 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:07.444+0000 7fc88e3fd740 20 client.18003 trim_cache size 0 max 16384 2024-03-21T10:33:07.460 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:07.462+0000 7fc88e3fd740 1 Processor -- start 2024-03-21T10:33:07.460 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:07.462+0000 7fc88e3fd740 1 -- start start 2024-03-21T10:33:07.460 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:07.462+0000 7fc88e3fd740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556c43c8bf20 0x556c43c94440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:33:07.461 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:07.462+0000 7fc88e3fd740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556c43c8cc00 0x556c43c94980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:33:07.461 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:07.462+0000 7fc88e3fd740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x556c43c95290 0x556c43c96680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:33:07.461 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:07.462+0000 7fc88e3fd740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8841b0c80 con 0x556c43c95290 2024-03-21T10:33:07.461 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:07.462+0000 7fc88e3fd740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8841b2600 con 0x556c43c8bf20 2024-03-21T10:33:07.461 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:07.462+0000 7fc88e3fd740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8841b12e0 con 0x556c43c8cc00 2024-03-21T10:33:07.463 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:07.462+0000 7fc889a00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x556c43c95290 0x556c43c96680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:33:07.463 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:07.462+0000 7fc889a00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x556c43c95290 0x556c43c96680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:46744/0 (socket says 172.21.15.47:46744) 2024-03-21T10:33:07.463 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:07.462+0000 7fc889a00640 1 -- 172.21.15.47:0/3720776359 learned_addr learned my addr 172.21.15.47:0/3720776359 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:33:07.463 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:07.462+0000 7fc889000640 1 --2- 172.21.15.47:0/3720776359 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556c43c8bf20 0x556c43c94440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:33:07.463 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:07.463+0000 7fc87be00640 1 --2- 172.21.15.47:0/3720776359 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556c43c8cc00 0x556c43c94980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:33:07.463 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:07.463+0000 7fc889a00640 1 -- 172.21.15.47:0/3720776359 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556c43c8cc00 msgr2=0x556c43c94980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:33:07.464 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:07.463+0000 7fc889a00640 1 --2- 172.21.15.47:0/3720776359 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556c43c8cc00 0x556c43c94980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:33:07.464 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:07.463+0000 7fc889a00640 1 -- 172.21.15.47:0/3720776359 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556c43c8bf20 msgr2=0x556c43c94440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:33:07.464 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:07.463+0000 7fc889a00640 1 --2- 172.21.15.47:0/3720776359 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556c43c8bf20 0x556c43c94440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:33:07.464 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:07.463+0000 7fc889a00640 1 -- 172.21.15.47:0/3720776359 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556c438c40f0 con 0x556c43c95290 2024-03-21T10:33:07.465 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:07.463+0000 7fc889a00640 1 --2- 172.21.15.47:0/3720776359 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x556c43c95290 0x556c43c96680 secure :-1 s=READY pgs=1089 cs=0 l=1 rev1=1 crypto rx=0x7fc87c009eb0 tx=0x7fc87c013040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8672152b3dd174e1 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:33:07.465 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:07.463+0000 7fc879600640 1 -- 172.21.15.47:0/3720776359 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc87c02a070 con 0x556c43c95290 2024-03-21T10:33:07.465 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:07.464+0000 7fc879600640 1 -- 172.21.15.47:0/3720776359 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fc87c015040 con 0x556c43c95290 2024-03-21T10:33:07.465 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:07.464+0000 7fc879600640 1 -- 172.21.15.47:0/3720776359 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc87c01a7f0 con 0x556c43c95290 2024-03-21T10:33:07.465 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:07.464+0000 7fc88e3fd740 1 -- 172.21.15.47:0/3720776359 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x556c43c95290 msgr2=0x556c43c96680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:33:07.465 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:07.464+0000 7fc88e3fd740 1 --2- 172.21.15.47:0/3720776359 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x556c43c95290 0x556c43c96680 secure :-1 s=READY pgs=1089 cs=0 l=1 rev1=1 crypto rx=0x7fc87c009eb0 tx=0x7fc87c013040 comp rx=0 tx=0).stop 2024-03-21T10:33:07.465 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:07.464+0000 7fc88e3fd740 1 -- 172.21.15.47:0/3720776359 shutdown_connections 2024-03-21T10:33:07.465 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:07.464+0000 7fc88e3fd740 1 --2- 172.21.15.47:0/3720776359 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x556c43c95290 0x556c43c96680 unknown :-1 s=CLOSED pgs=1089 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:33:07.465 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:07.464+0000 7fc88e3fd740 1 --2- 172.21.15.47:0/3720776359 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556c43c8cc00 0x556c43c94980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:33:07.465 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:07.464+0000 7fc88e3fd740 1 --2- 172.21.15.47:0/3720776359 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556c43c8bf20 0x556c43c94440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:33:07.465 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:07.464+0000 7fc88e3fd740 1 -- 172.21.15.47:0/3720776359 >> 172.21.15.47:0/3720776359 conn(0x556c43c53200 msgr2=0x556c43c864a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:33:07.465 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:07.464+0000 7fc88e3fd740 1 -- 172.21.15.47:0/3720776359 shutdown_connections 2024-03-21T10:33:07.466 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:07.464+0000 7fc88e3fd740 1 -- 172.21.15.47:0/3720776359 wait complete. 2024-03-21T10:33:07.466 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:07.465+0000 7fc88e3fd740 1 -- start start 2024-03-21T10:33:07.466 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:07.465+0000 7fc88e3fd740 1 Processor -- start 2024-03-21T10:33:07.469 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:07.465+0000 7fc88e3fd740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556c43c8bf20 0x556c43c8f770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:33:07.469 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:07.465+0000 7fc88e3fd740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556c43c8cc00 0x556c43c8df80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:33:07.469 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:07.465+0000 7fc88e3fd740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x556c43c95290 0x556c43c8e4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:33:07.470 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:07.465+0000 7fc88e3fd740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8841b1fa0 con 0x556c43c95290 2024-03-21T10:33:07.470 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:07.465+0000 7fc88e3fd740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8841b1940 con 0x556c43c8cc00 2024-03-21T10:33:07.470 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:07.465+0000 7fc88e3fd740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8841aee60 con 0x556c43c8bf20 2024-03-21T10:33:07.470 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:07.465+0000 7fc88e3fd740 10 client.0 _ll_register_callbacks cb 0x7fc86c00df40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T10:33:07.470 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:07.466+0000 7fc889a00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x556c43c95290 0x556c43c8e4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:33:07.470 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:07.466+0000 7fc889a00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x556c43c95290 0x556c43c8e4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:46756/0 (socket says 172.21.15.47:46756) 2024-03-21T10:33:07.470 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:07.466+0000 7fc889a00640 1 -- 172.21.15.47:0/1149840777 learned_addr learned my addr 172.21.15.47:0/1149840777 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:33:07.470 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:07.466+0000 7fc87be00640 1 --2- 172.21.15.47:0/1149840777 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556c43c8cc00 0x556c43c8df80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:33:07.470 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:07.466+0000 7fc889000640 1 --2- 172.21.15.47:0/1149840777 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556c43c8bf20 0x556c43c8f770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:33:07.470 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:07.466+0000 7fc889a00640 1 -- 172.21.15.47:0/1149840777 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556c43c8bf20 msgr2=0x556c43c8f770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:33:07.470 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:07.466+0000 7fc889a00640 1 --2- 172.21.15.47:0/1149840777 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556c43c8bf20 0x556c43c8f770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:33:07.470 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:07.466+0000 7fc889a00640 1 -- 172.21.15.47:0/1149840777 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556c43c8cc00 msgr2=0x556c43c8df80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:33:07.470 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:07.466+0000 7fc889a00640 1 --2- 172.21.15.47:0/1149840777 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556c43c8cc00 0x556c43c8df80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:33:07.470 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:07.466+0000 7fc889a00640 1 -- 172.21.15.47:0/1149840777 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x556c43c8ea00 con 0x556c43c95290 2024-03-21T10:33:07.471 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:07.466+0000 7fc87be00640 1 --2- 172.21.15.47:0/1149840777 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556c43c8cc00 0x556c43c8df80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T10:33:07.471 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:07.466+0000 7fc889a00640 1 --2- 172.21.15.47:0/1149840777 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x556c43c95290 0x556c43c8e4c0 secure :-1 s=READY pgs=1090 cs=0 l=1 rev1=1 crypto rx=0x7fc87c0107e0 tx=0x7fc87c003ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=21b14d1af7b57f04 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:33:07.471 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:07.467+0000 7fc872a00640 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-03-21T10:33:07.471 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:07.467+0000 7fc872a00640 1 -- 172.21.15.47:0/1149840777 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc87c02a030 con 0x556c43c95290 2024-03-21T10:33:07.471 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:07.467+0000 7fc872a00640 1 -- 172.21.15.47:0/1149840777 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(99..99 src has 1..99) v4 ==== 9562+0+0 (secure 0 0 0) 0x7fc87c005480 con 0x556c43c95290 2024-03-21T10:33:07.471 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:07.467+0000 7fc88e3fd740 1 -- 172.21.15.47:0/1149840777 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fc87c015040 con 0x556c43c95290 2024-03-21T10:33:07.471 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:07.467+0000 7fc872a00640 1 -- 172.21.15.47:0/1149840777 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc87c039050 con 0x556c43c95290 2024-03-21T10:33:07.471 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:07.467+0000 7fc88e3fd740 10 client.18009 fetch_fsmap learned FSMap version 32 2024-03-21T10:33:07.471 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:07.467+0000 7fc88e3fd740 1 -- 172.21.15.47:0/1149840777 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x556c43d00690 con 0x556c43c95290 2024-03-21T10:33:07.471 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:07.467+0000 7fc872a00640 1 -- 172.21.15.47:0/1149840777 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fc87c015040 con 0x556c43c95290 2024-03-21T10:33:07.471 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:07.468+0000 7fc872a00640 1 -- 172.21.15.47:0/1149840777 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 32) v1 ==== 7942+0+0 (secure 0 0 0) 0x7fc87c011440 con 0x556c43c95290 2024-03-21T10:33:07.471 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:07.468+0000 7fc88e3fd740 10 client.18009 fetch_fsmap finished waiting for FSMap version 32 2024-03-21T10:33:07.471 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:07.468+0000 7fc88e3fd740 10 client.18009 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-21T10:33:07.471 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:07.468+0000 7fc88e3fd740 20 client.18009 populate_metadata read hostname 'smithi047' 2024-03-21T10:33:07.471 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:07.468+0000 7fc88e3fd740 1 --2- 172.21.15.47:0/1149840777 >> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] conn(0x556c43d00a60 0x556c43d02e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:33:07.472 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:07.468+0000 7fc88e3fd740 4 client.18009 mds_command: new command op to 34535 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-21T10:33:07.472 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:07.468+0000 7fc88e3fd740 1 -- 172.21.15.47:0/1149840777 --> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x556c43bf7ae0 con 0x556c43d00a60 2024-03-21T10:33:07.472 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:07.468+0000 7fc889000640 1 --2- 172.21.15.47:0/1149840777 >> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] conn(0x556c43d00a60 0x556c43d02e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:33:07.472 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:07.469+0000 7fc889000640 1 --2- 172.21.15.47:0/1149840777 >> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] conn(0x556c43d00a60 0x556c43d02e50 crc :-1 s=READY pgs=184 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cabf9229232e9b4f server_cookie=1fbef65fe54a92c in_seq=0 out_seq=0 2024-03-21T10:33:07.472 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:07.469+0000 7fc872a00640 10 client.18009 ms_handle_connect on v2:172.21.15.100:6838/4173168440 2024-03-21T10:33:07.472 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:07.469+0000 7fc872a00640 1 -- 172.21.15.47:0/1149840777 <== mds.0 v2:172.21.15.100:6838/4173168440 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x556c43bf7ae0 con 0x556c43d00a60 2024-03-21T10:33:07.472 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:07.469+0000 7fc872a00640 10 client.18009 handle_command_reply: tid=0 2024-03-21T10:33:07.472 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:07.469+0000 7fc88e3fd740 1 client.18009 shutdown 2024-03-21T10:33:07.472 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:07.470+0000 7fc88e3fd740 1 -- 172.21.15.47:0/1149840777 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x556c43c95290 msgr2=0x556c43c8e4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:33:07.472 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:07.470+0000 7fc88e3fd740 1 --2- 172.21.15.47:0/1149840777 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x556c43c95290 0x556c43c8e4c0 secure :-1 s=READY pgs=1090 cs=0 l=1 rev1=1 crypto rx=0x7fc87c0107e0 tx=0x7fc87c003ba0 comp rx=0 tx=0).stop 2024-03-21T10:33:07.472 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:07.470+0000 7fc88e3fd740 1 -- 172.21.15.47:0/1149840777 shutdown_connections 2024-03-21T10:33:07.472 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:07.470+0000 7fc88e3fd740 1 --2- 172.21.15.47:0/1149840777 >> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] conn(0x556c43d00a60 0x556c43d02e50 crc :-1 s=READY pgs=184 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:33:07.472 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:07.470+0000 7fc88e3fd740 1 --2- 172.21.15.47:0/1149840777 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x556c43c95290 0x556c43c8e4c0 unknown :-1 s=CLOSED pgs=1090 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:33:07.472 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:07.470+0000 7fc88e3fd740 1 --2- 172.21.15.47:0/1149840777 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556c43c8cc00 0x556c43c8df80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:33:07.472 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:07.470+0000 7fc88e3fd740 1 --2- 172.21.15.47:0/1149840777 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556c43c8bf20 0x556c43c8f770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:33:07.473 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:07.470+0000 7fc88e3fd740 1 -- 172.21.15.47:0/1149840777 >> 172.21.15.47:0/1149840777 conn(0x556c43c53200 msgr2=0x556c43c90580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:33:07.473 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:07.470+0000 7fc872a00640 0 client.18009 ms_handle_reset on v2:172.21.15.100:6838/4173168440 2024-03-21T10:33:07.473 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:07.470+0000 7fc88e3fd740 1 -- 172.21.15.47:0/1149840777 shutdown_connections 2024-03-21T10:33:07.473 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:07.470+0000 7fc88e3fd740 1 -- 172.21.15.47:0/1149840777 wait complete. 2024-03-21T10:33:07.473 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:07.470+0000 7fc88e3fd740 20 client.18009 trim_cache size 0 max 16384 2024-03-21T10:33:07.473 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-03-21T10:33:07.473 INFO:teuthology.orchestra.run.smithi047.stdout: "status": "no active scrubs running", 2024-03-21T10:33:07.474 INFO:teuthology.orchestra.run.smithi047.stdout: "scrubs": {} 2024-03-21T10:33:07.474 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-03-21T10:33:07.475 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:07.471+0000 7fc88ba00640 1 -- 172.21.15.47:0/1835123701 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7fc8680a2fa0 msgr2=0x7fc8680a5460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:33:07.475 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:07.471+0000 7fc88ba00640 1 --2- 172.21.15.47:0/1835123701 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7fc8680a2fa0 0x7fc8680a5460 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fc874006010 tx=0x7fc874005b50 comp rx=0 tx=0).stop 2024-03-21T10:33:07.475 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:07.471+0000 7fc88ba00640 1 -- 172.21.15.47:0/1835123701 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc884104200 msgr2=0x7fc88419a5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:33:07.475 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:07.471+0000 7fc88ba00640 1 --2- 172.21.15.47:0/1835123701 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc884104200 0x7fc88419a5b0 secure :-1 s=READY pgs=1087 cs=0 l=1 rev1=1 crypto rx=0x7fc86c00d950 tx=0x7fc86c00de10 comp rx=0 tx=0).stop 2024-03-21T10:33:07.475 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:07.471+0000 7fc88ba00640 1 -- 172.21.15.47:0/1835123701 shutdown_connections 2024-03-21T10:33:07.475 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:07.471+0000 7fc88ba00640 1 --2- 172.21.15.47:0/1835123701 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7fc8680a2fa0 0x7fc8680a5460 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:33:07.475 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:07.471+0000 7fc88ba00640 1 --2- 172.21.15.47:0/1835123701 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc884109850 0x7fc88419e930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:33:07.475 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:07.471+0000 7fc88ba00640 1 --2- 172.21.15.47:0/1835123701 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc884104200 0x7fc88419a5b0 unknown :-1 s=CLOSED pgs=1087 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:33:07.475 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:07.471+0000 7fc88ba00640 1 --2- 172.21.15.47:0/1835123701 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc884103840 0x7fc88419a070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:33:07.475 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:07.471+0000 7fc88ba00640 1 -- 172.21.15.47:0/1835123701 >> 172.21.15.47:0/1835123701 conn(0x7fc8840ff310 msgr2=0x7fc8840ffc70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:33:07.475 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:07.471+0000 7fc88ba00640 1 -- 172.21.15.47:0/1835123701 shutdown_connections 2024-03-21T10:33:07.475 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:07.471+0000 7fc88ba00640 1 -- 172.21.15.47:0/1835123701 wait complete. 2024-03-21T10:33:07.487 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-03-21T10:33:07.487 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-21T10:33:07.565 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:33:07 smithi047 ceph-mon[31547]: pgmap v239: 129 pgs: 129 active+clean; 277 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 44 KiB/s wr, 29 op/s 2024-03-21T10:33:07.705 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:07.706+0000 7f3877e00640 1 -- 172.21.15.47:0/3600407698 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3870109860 msgr2=0x7f3870109c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:33:07.705 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:07.706+0000 7f3877e00640 1 --2- 172.21.15.47:0/3600407698 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3870109860 0x7f3870109c40 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f3864009900 tx=0x7f386402bc30 comp rx=0 tx=0).stop 2024-03-21T10:33:07.705 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:07.706+0000 7f3877e00640 1 -- 172.21.15.47:0/3600407698 shutdown_connections 2024-03-21T10:33:07.706 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:07.706+0000 7f3877e00640 1 --2- 172.21.15.47:0/3600407698 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3870104210 0x7f3870110720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:33:07.706 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:07.706+0000 7f3877e00640 1 --2- 172.21.15.47:0/3600407698 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3870103850 0x7f3870103cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:33:07.706 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:07.706+0000 7f3877e00640 1 --2- 172.21.15.47:0/3600407698 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3870109860 0x7f3870109c40 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:33:07.706 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:07.706+0000 7f3877e00640 1 -- 172.21.15.47:0/3600407698 >> 172.21.15.47:0/3600407698 conn(0x7f38700ff320 msgr2=0x7f3870101740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:33:07.706 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:07.706+0000 7f3877e00640 1 -- 172.21.15.47:0/3600407698 shutdown_connections 2024-03-21T10:33:07.706 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:07.706+0000 7f3877e00640 1 -- 172.21.15.47:0/3600407698 wait complete. 2024-03-21T10:33:07.707 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:07.708+0000 7f3877e00640 1 Processor -- start 2024-03-21T10:33:07.708 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:07.708+0000 7f3877e00640 1 -- start start 2024-03-21T10:33:07.708 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:07.708+0000 7f3877e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3870103850 0x7f387019a060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:33:07.708 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:07.708+0000 7f3877e00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3870104210 0x7f387019a5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:33:07.708 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:07.708+0000 7f3877e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3870109860 0x7f387019e920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:33:07.708 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:07.708+0000 7f3877e00640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3870113b20 con 0x7f3870103850 2024-03-21T10:33:07.708 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:07.708+0000 7f3877e00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38701139a0 con 0x7f3870109860 2024-03-21T10:33:07.708 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:07.708+0000 7f3877e00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3870113ca0 con 0x7f3870104210 2024-03-21T10:33:07.708 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:07.709+0000 7f3875400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3870103850 0x7f387019a060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:33:07.708 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:07.709+0000 7f3875e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3870109860 0x7f387019e920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:33:07.708 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:07.709+0000 7f3875400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3870103850 0x7f387019a060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:46778/0 (socket says 172.21.15.47:46778) 2024-03-21T10:33:07.708 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:07.709+0000 7f3875400640 1 -- 172.21.15.47:0/3464973505 learned_addr learned my addr 172.21.15.47:0/3464973505 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:33:07.708 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:07.709+0000 7f3874a00640 1 --2- 172.21.15.47:0/3464973505 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3870104210 0x7f387019a5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:33:07.709 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:07.709+0000 7f3875400640 1 -- 172.21.15.47:0/3464973505 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3870104210 msgr2=0x7f387019a5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:33:07.709 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:07.709+0000 7f3875400640 1 --2- 172.21.15.47:0/3464973505 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3870104210 0x7f387019a5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:33:07.709 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:07.709+0000 7f3875400640 1 -- 172.21.15.47:0/3464973505 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3870109860 msgr2=0x7f387019e920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:33:07.709 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:07.709+0000 7f3875400640 1 --2- 172.21.15.47:0/3464973505 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3870109860 0x7f387019e920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:33:07.709 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:07.709+0000 7f3875400640 1 -- 172.21.15.47:0/3464973505 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f387019f050 con 0x7f3870103850 2024-03-21T10:33:07.710 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:07.710+0000 7f3875400640 1 --2- 172.21.15.47:0/3464973505 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3870103850 0x7f387019a060 secure :-1 s=READY pgs=1091 cs=0 l=1 rev1=1 crypto rx=0x7f3864006010 tx=0x7f3864005d90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:33:07.710 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:07.710+0000 7f386e000640 1 -- 172.21.15.47:0/3464973505 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3864043020 con 0x7f3870103850 2024-03-21T10:33:07.710 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:07.710+0000 7f386e000640 1 -- 172.21.15.47:0/3464973505 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f3864004350 con 0x7f3870103850 2024-03-21T10:33:07.710 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:07.710+0000 7f3877e00640 1 -- 172.21.15.47:0/3464973505 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f387019f2e0 con 0x7f3870103850 2024-03-21T10:33:07.710 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:07.710+0000 7f386e000640 1 -- 172.21.15.47:0/3464973505 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3864003a10 con 0x7f3870103850 2024-03-21T10:33:07.710 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:07.710+0000 7f3877e00640 1 -- 172.21.15.47:0/3464973505 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f38701a6bd0 con 0x7f3870103850 2024-03-21T10:33:07.710 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:07.711+0000 7f3877e00640 1 -- 172.21.15.47:0/3464973505 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f38700688e0 con 0x7f3870103850 2024-03-21T10:33:07.713 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:07.715+0000 7f386e000640 1 -- 172.21.15.47:0/3464973505 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 27) v1 ==== 137985+0+0 (secure 0 0 0) 0x7f3864046020 con 0x7f3870103850 2024-03-21T10:33:07.714 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:07.716+0000 7f386e000640 1 --2- 172.21.15.47:0/3464973505 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f38580a3090 0x7f38580a5550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:33:07.715 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:07.716+0000 7f386e000640 1 -- 172.21.15.47:0/3464973505 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 9562+0+0 (secure 0 0 0) 0x7f38640f1be0 con 0x7f3870103850 2024-03-21T10:33:07.715 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:07.717+0000 7f3874a00640 1 --2- 172.21.15.47:0/3464973505 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f38580a3090 0x7f38580a5550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:33:07.715 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:07.717+0000 7f3874a00640 1 --2- 172.21.15.47:0/3464973505 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f38580a3090 0x7f38580a5550 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f385c0095f0 tx=0x7f385c005f90 comp rx=0 tx=0).ready entity=mgr.15687 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:33:07.718 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:07.720+0000 7f386e000640 1 -- 172.21.15.47:0/3464973505 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f386404d070 con 0x7f3870103850 2024-03-21T10:33:07.727 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:33:07 smithi100 ceph-mon[35193]: pgmap v239: 129 pgs: 129 active+clean; 277 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 44 KiB/s wr, 29 op/s 2024-03-21T10:33:07.737 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:33:07 smithi157 ceph-mon[35581]: pgmap v239: 129 pgs: 129 active+clean; 277 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 44 KiB/s wr, 29 op/s 2024-03-21T10:33:07.986 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:07.986+0000 7f3877e00640 1 -- 172.21.15.47:0/3464973505 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f387019b3b0 con 0x7f3870103850 2024-03-21T10:33:07.987 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:07.988+0000 7f386e000640 1 -- 172.21.15.47:0/3464973505 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 32 v32) v1 ==== 94+0+11471 (secure 0 0 0) 0x7f387019b3b0 con 0x7f3870103850 2024-03-21T10:33:07.988 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-21T10:33:07.988 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":32,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14511,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6835/735724037","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6834","nonce":735724037},{"type":"v1","addr":"172.21.15.47:6835","nonce":735724037}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14529,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6837/2248191724","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6836","nonce":2248191724},{"type":"v1","addr":"172.21.15.47:6837","nonce":2248191724}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":32,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-21T10:21:58.329063+0000","modified":"2024-03-21T10:26:17.169892+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":34535,"mds_1":34493,"mds_2":34454,"mds_3":34427,"mds_4":14619},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14604":{"gid":14604,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.47:6839/2881335356","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6838","nonce":2881335356},{"type":"v1","addr":"172.21.15.47:6839","nonce":2881335356}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_14619":{"gid":14619,"name":"a","rank":4,"incarnation":27,"state":"up:active","state_seq":11,"addr":"172.21.15.47:6841/3401578978","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6840","nonce":3401578978},{"type":"v1","addr":"172.21.15.47:6841","nonce":3401578978}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24406":{"gid":24406,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.157:6833/4000010136","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":4000010136},{"type":"v1","addr":"172.21.15.157:6833","nonce":4000010136}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24457":{"gid":24457,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.157:6835/3902868651","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":3902868651},{"type":"v1","addr":"172.21.15.157:6835","nonce":3902868651}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24475":{"gid":24475,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.157:6837/1908087204","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6836","nonce":1908087204},{"type":"v1","addr":"172.21.15.157:6837","nonce":1908087204}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_25729":{"gid":25729,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.157:6839/1998408416","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6838","nonce":1998408416},{"type":"v1","addr":"172.21.15.157:6839","nonce":1998408416}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_34427":{"gid":34427,"name":"k","rank":3,"incarnation":25,"state":"up:active","state_seq":20,"addr":"172.21.15.100:6833/2723302038","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6832","nonce":2723302038},{"type":"v1","addr":"172.21.15.100:6833","nonce":2723302038}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_34454":{"gid":34454,"name":"e","rank":2,"incarnation":22,"state":"up:active","state_seq":17,"addr":"172.21.15.100:6835/2597849957","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":2597849957},{"type":"v1","addr":"172.21.15.100:6835","nonce":2597849957}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_34493":{"gid":34493,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":13,"addr":"172.21.15.100:6837/3497644162","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6836","nonce":3497644162},{"type":"v1","addr":"172.21.15.100:6837","nonce":3497644162}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_34535":{"gid":34535,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.100:6839/4173168440","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6838","nonce":4173168440},{"type":"v1","addr":"172.21.15.100:6839","nonce":4173168440}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":34535,"qdb_cluster":[34427,14619,34454,34535,34493]},"id":1}]} 2024-03-21T10:33:07.988 INFO:teuthology.orchestra.run.smithi047.stderr:dumped fsmap epoch 32 2024-03-21T10:33:07.990 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:07.991+0000 7f3877e00640 1 -- 172.21.15.47:0/3464973505 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f38580a3090 msgr2=0x7f38580a5550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:33:07.990 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:07.991+0000 7f3877e00640 1 --2- 172.21.15.47:0/3464973505 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f38580a3090 0x7f38580a5550 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f385c0095f0 tx=0x7f385c005f90 comp rx=0 tx=0).stop 2024-03-21T10:33:07.991 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:07.991+0000 7f3877e00640 1 -- 172.21.15.47:0/3464973505 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3870103850 msgr2=0x7f387019a060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:33:07.991 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:07.991+0000 7f3877e00640 1 --2- 172.21.15.47:0/3464973505 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3870103850 0x7f387019a060 secure :-1 s=READY pgs=1091 cs=0 l=1 rev1=1 crypto rx=0x7f3864006010 tx=0x7f3864005d90 comp rx=0 tx=0).stop 2024-03-21T10:33:07.991 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:07.991+0000 7f3877e00640 1 -- 172.21.15.47:0/3464973505 shutdown_connections 2024-03-21T10:33:07.991 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:07.991+0000 7f3877e00640 1 --2- 172.21.15.47:0/3464973505 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f38580a3090 0x7f38580a5550 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:33:07.991 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:07.991+0000 7f3877e00640 1 --2- 172.21.15.47:0/3464973505 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3870109860 0x7f387019e920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:33:07.991 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:07.991+0000 7f3877e00640 1 --2- 172.21.15.47:0/3464973505 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3870104210 0x7f387019a5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:33:07.991 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:07.991+0000 7f3877e00640 1 --2- 172.21.15.47:0/3464973505 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3870103850 0x7f387019a060 unknown :-1 s=CLOSED pgs=1091 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:33:07.991 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:07.991+0000 7f3877e00640 1 -- 172.21.15.47:0/3464973505 >> 172.21.15.47:0/3464973505 conn(0x7f38700ff320 msgr2=0x7f38700ffb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:33:07.991 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:07.991+0000 7f3877e00640 1 -- 172.21.15.47:0/3464973505 shutdown_connections 2024-03-21T10:33:07.991 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:07.991+0000 7f3877e00640 1 -- 172.21.15.47:0/3464973505 wait complete. 2024-03-21T10:33:08.013 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-21T10:33:08.013 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-03-21T10:33:08.248 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:08.249+0000 7f75cbe00640 1 -- 172.21.15.47:0/2821251209 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f75cc101530 msgr2=0x7f75cc1019b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:33:08.249 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:08.249+0000 7f75cbe00640 1 --2- 172.21.15.47:0/2821251209 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f75cc101530 0x7f75cc1019b0 secure :-1 s=READY pgs=1092 cs=0 l=1 rev1=1 crypto rx=0x7f75b4009900 tx=0x7f75b402bc30 comp rx=0 tx=0).stop 2024-03-21T10:33:08.249 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:08.249+0000 7f75cbe00640 1 -- 172.21.15.47:0/2821251209 shutdown_connections 2024-03-21T10:33:08.249 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:08.249+0000 7f75cbe00640 1 --2- 172.21.15.47:0/2821251209 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f75cc101ef0 0x7f75cc10e3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:33:08.249 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:08.249+0000 7f75cbe00640 1 --2- 172.21.15.47:0/2821251209 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f75cc101530 0x7f75cc1019b0 unknown :-1 s=CLOSED pgs=1092 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:33:08.249 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:08.249+0000 7f75cbe00640 1 --2- 172.21.15.47:0/2821251209 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f75cc1075f0 0x7f75cc1079d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:33:08.249 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:08.249+0000 7f75cbe00640 1 -- 172.21.15.47:0/2821251209 >> 172.21.15.47:0/2821251209 conn(0x7f75cc0fd040 msgr2=0x7f75cc0ff460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:33:08.249 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:08.249+0000 7f75cbe00640 1 -- 172.21.15.47:0/2821251209 shutdown_connections 2024-03-21T10:33:08.249 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:08.249+0000 7f75cbe00640 1 -- 172.21.15.47:0/2821251209 wait complete. 2024-03-21T10:33:08.251 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:08.250+0000 7f75cbe00640 1 Processor -- start 2024-03-21T10:33:08.252 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:08.250+0000 7f75cbe00640 1 -- start start 2024-03-21T10:33:08.252 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:08.250+0000 7f75cbe00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f75cc101530 0x7f75cc199e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:33:08.252 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:08.250+0000 7f75cbe00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f75cc101ef0 0x7f75cc19a3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:33:08.252 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:08.250+0000 7f75cbe00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f75cc1075f0 0x7f75cc19e730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:33:08.252 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:08.250+0000 7f75cbe00640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75cc1117d0 con 0x7f75cc1075f0 2024-03-21T10:33:08.252 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:08.250+0000 7f75cbe00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75cc111650 con 0x7f75cc101530 2024-03-21T10:33:08.252 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:08.250+0000 7f75cbe00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75cc111950 con 0x7f75cc101ef0 2024-03-21T10:33:08.252 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:08.251+0000 7f75cb400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f75cc1075f0 0x7f75cc19e730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:33:08.252 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:08.251+0000 7f75cb400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f75cc1075f0 0x7f75cc19e730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:46786/0 (socket says 172.21.15.47:46786) 2024-03-21T10:33:08.252 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:08.251+0000 7f75cb400640 1 -- 172.21.15.47:0/3082696022 learned_addr learned my addr 172.21.15.47:0/3082696022 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:33:08.252 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:08.251+0000 7f75caa00640 1 --2- 172.21.15.47:0/3082696022 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f75cc101530 0x7f75cc199e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:33:08.253 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:08.251+0000 7f75ca000640 1 --2- 172.21.15.47:0/3082696022 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f75cc101ef0 0x7f75cc19a3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:33:08.253 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:08.251+0000 7f75cb400640 1 -- 172.21.15.47:0/3082696022 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f75cc101ef0 msgr2=0x7f75cc19a3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:33:08.253 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:08.251+0000 7f75cb400640 1 --2- 172.21.15.47:0/3082696022 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f75cc101ef0 0x7f75cc19a3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:33:08.253 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:08.251+0000 7f75cb400640 1 -- 172.21.15.47:0/3082696022 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f75cc101530 msgr2=0x7f75cc199e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:33:08.253 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:08.251+0000 7f75cb400640 1 --2- 172.21.15.47:0/3082696022 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f75cc101530 0x7f75cc199e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:33:08.253 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:08.251+0000 7f75cb400640 1 -- 172.21.15.47:0/3082696022 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f75cc19ee60 con 0x7f75cc1075f0 2024-03-21T10:33:08.253 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:08.251+0000 7f75cb400640 1 --2- 172.21.15.47:0/3082696022 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f75cc1075f0 0x7f75cc19e730 secure :-1 s=READY pgs=1093 cs=0 l=1 rev1=1 crypto rx=0x7f75c400d750 tx=0x7f75c400dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:33:08.253 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:08.252+0000 7f75c3400640 1 -- 172.21.15.47:0/3082696022 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75c4017070 con 0x7f75cc1075f0 2024-03-21T10:33:08.253 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:08.252+0000 7f75c3400640 1 -- 172.21.15.47:0/3082696022 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f75c40054d0 con 0x7f75cc1075f0 2024-03-21T10:33:08.253 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:08.252+0000 7f75cbe00640 1 -- 172.21.15.47:0/3082696022 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f75cc19f150 con 0x7f75cc1075f0 2024-03-21T10:33:08.253 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:08.252+0000 7f75c3400640 1 -- 172.21.15.47:0/3082696022 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75c4004c60 con 0x7f75cc1075f0 2024-03-21T10:33:08.253 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:08.252+0000 7f75cbe00640 1 -- 172.21.15.47:0/3082696022 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f75cc1a69e0 con 0x7f75cc1075f0 2024-03-21T10:33:08.254 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:08.255+0000 7f75d47d0740 1 Processor -- start 2024-03-21T10:33:08.255 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:08.256+0000 7f75c3400640 1 -- 172.21.15.47:0/3082696022 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 27) v1 ==== 137985+0+0 (secure 0 0 0) 0x7f75c4004e00 con 0x7f75cc1075f0 2024-03-21T10:33:08.255 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:08.256+0000 7f75d47d0740 1 -- start start 2024-03-21T10:33:08.259 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:08.256+0000 7f75d47d0740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55aeb76688b0 0x55aeb7668c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:33:08.259 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:08.256+0000 7f75d47d0740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55aeb7669260 0x55aeb7671810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:33:08.259 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:08.256+0000 7f75d47d0740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55aeb7671de0 0x55aeb7674230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:33:08.259 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:08.256+0000 7f75d47d0740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aeb75cc2c0 con 0x55aeb76688b0 2024-03-21T10:33:08.259 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:08.256+0000 7f75d47d0740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aeb7574c90 con 0x55aeb7671de0 2024-03-21T10:33:08.259 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:08.256+0000 7f75d47d0740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aeb75c8360 con 0x55aeb7669260 2024-03-21T10:33:08.259 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:08.257+0000 7f75ca000640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55aeb76688b0 0x55aeb7668c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:33:08.259 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:08.257+0000 7f75ca000640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55aeb76688b0 0x55aeb7668c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:46792/0 (socket says 172.21.15.47:46792) 2024-03-21T10:33:08.259 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:08.257+0000 7f75ca000640 1 -- 172.21.15.47:0/3538844802 learned_addr learned my addr 172.21.15.47:0/3538844802 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:33:08.259 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:08.257+0000 7f75ca000640 1 --2- 172.21.15.47:0/3538844802 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55aeb7671de0 0x55aeb7674230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:33:08.259 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:08.257+0000 7f75caa00640 1 --2- 172.21.15.47:0/3538844802 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55aeb7669260 0x55aeb7671810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:33:08.260 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:08.257+0000 7f75c3400640 1 --2- 172.21.15.47:0/3082696022 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f75b00a3090 0x7f75b00a5550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:33:08.260 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:08.257+0000 7f75ca000640 1 -- 172.21.15.47:0/3538844802 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55aeb7669260 msgr2=0x55aeb7671810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:33:08.260 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:08.257+0000 7f75ca000640 1 --2- 172.21.15.47:0/3538844802 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55aeb7669260 0x55aeb7671810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:33:08.260 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:08.257+0000 7f75ca000640 1 -- 172.21.15.47:0/3538844802 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55aeb7671de0 msgr2=0x55aeb7674230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:33:08.260 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:08.257+0000 7f75ca000640 1 --2- 172.21.15.47:0/3538844802 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55aeb7671de0 0x55aeb7674230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:33:08.260 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:08.257+0000 7f75ca000640 1 -- 172.21.15.47:0/3538844802 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55aeb7599080 con 0x55aeb76688b0 2024-03-21T10:33:08.260 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:08.257+0000 7f75c3400640 1 -- 172.21.15.47:0/3082696022 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 9562+0+0 (secure 0 0 0) 0x7f75c40cf510 con 0x7f75cc1075f0 2024-03-21T10:33:08.260 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:08.257+0000 7f75cb400640 1 --2- 172.21.15.47:0/3082696022 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f75b00a3090 0x7f75b00a5550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:33:08.260 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:08.258+0000 7f75ca000640 1 --2- 172.21.15.47:0/3538844802 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55aeb76688b0 0x55aeb7668c90 secure :-1 s=READY pgs=1094 cs=0 l=1 rev1=1 crypto rx=0x7f75b4005c40 tx=0x7f75b4005bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ccbba15531ce8410 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:33:08.260 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:08.258+0000 7f75bbe00640 1 -- 172.21.15.47:0/3538844802 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75b402c780 con 0x55aeb76688b0 2024-03-21T10:33:08.260 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:08.258+0000 7f75bbe00640 1 -- 172.21.15.47:0/3538844802 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f75b4048070 con 0x55aeb76688b0 2024-03-21T10:33:08.260 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:08.258+0000 7f75bbe00640 1 -- 172.21.15.47:0/3538844802 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75b4004920 con 0x55aeb76688b0 2024-03-21T10:33:08.261 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:08.259+0000 7f75d47d0740 1 -- 172.21.15.47:0/3538844802 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55aeb76688b0 msgr2=0x55aeb7668c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:33:08.261 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:08.259+0000 7f75d47d0740 1 --2- 172.21.15.47:0/3538844802 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55aeb76688b0 0x55aeb7668c90 secure :-1 s=READY pgs=1094 cs=0 l=1 rev1=1 crypto rx=0x7f75b4005c40 tx=0x7f75b4005bb0 comp rx=0 tx=0).stop 2024-03-21T10:33:08.261 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:08.259+0000 7f75cb400640 1 --2- 172.21.15.47:0/3082696022 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f75b00a3090 0x7f75b00a5550 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f75c40cc9a0 tx=0x7f75c40046c0 comp rx=0 tx=0).ready entity=mgr.15687 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:33:08.261 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:08.259+0000 7f75d47d0740 1 -- 172.21.15.47:0/3538844802 shutdown_connections 2024-03-21T10:33:08.261 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:08.259+0000 7f75d47d0740 1 --2- 172.21.15.47:0/3538844802 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55aeb7671de0 0x55aeb7674230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:33:08.261 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:08.259+0000 7f75d47d0740 1 --2- 172.21.15.47:0/3538844802 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55aeb7669260 0x55aeb7671810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:33:08.261 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:08.259+0000 7f75d47d0740 1 --2- 172.21.15.47:0/3538844802 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55aeb76688b0 0x55aeb7668c90 unknown :-1 s=CLOSED pgs=1094 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:33:08.261 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:08.259+0000 7f75d47d0740 1 -- 172.21.15.47:0/3538844802 >> 172.21.15.47:0/3538844802 conn(0x55aeb7662f80 msgr2=0x55aeb7663360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:33:08.261 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:08.259+0000 7f75d47d0740 1 -- 172.21.15.47:0/3538844802 shutdown_connections 2024-03-21T10:33:08.261 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:08.259+0000 7f75d47d0740 1 -- 172.21.15.47:0/3538844802 wait complete. 2024-03-21T10:33:08.261 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:08.260+0000 7f75d47d0740 1 -- start start 2024-03-21T10:33:08.265 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:08.260+0000 7f75d47d0740 1 Processor -- start 2024-03-21T10:33:08.265 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:08.260+0000 7f75d47d0740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55aeb76688b0 0x55aeb76716a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:33:08.265 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:08.260+0000 7f75d47d0740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55aeb7669260 0x55aeb766ea30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:33:08.265 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:08.260+0000 7f75d47d0740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55aeb7671de0 0x55aeb766ef90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:33:08.265 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:08.260+0000 7f75d47d0740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aeb7674cf0 con 0x55aeb7671de0 2024-03-21T10:33:08.265 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:08.260+0000 7f75d47d0740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aeb75880d0 con 0x55aeb7669260 2024-03-21T10:33:08.266 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:08.260+0000 7f75d47d0740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aeb7674e70 con 0x55aeb76688b0 2024-03-21T10:33:08.266 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:08.260+0000 7f75d47d0740 10 client.0 _ll_register_callbacks cb 0x55aeb758ce80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T10:33:08.266 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:08.261+0000 7f75ca000640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55aeb7671de0 0x55aeb766ef90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:33:08.266 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:08.261+0000 7f75ca000640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55aeb7671de0 0x55aeb766ef90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:46796/0 (socket says 172.21.15.47:46796) 2024-03-21T10:33:08.266 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:08.261+0000 7f75ca000640 1 -- 172.21.15.47:0/2796491698 learned_addr learned my addr 172.21.15.47:0/2796491698 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:33:08.266 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:08.261+0000 7f75ca000640 1 --2- 172.21.15.47:0/2796491698 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55aeb76688b0 0x55aeb76716a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:33:08.266 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:08.261+0000 7f75caa00640 1 --2- 172.21.15.47:0/2796491698 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55aeb7669260 0x55aeb766ea30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:33:08.266 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:08.261+0000 7f75ca000640 1 -- 172.21.15.47:0/2796491698 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55aeb76688b0 msgr2=0x55aeb76716a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:33:08.266 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:08.261+0000 7f75ca000640 1 --2- 172.21.15.47:0/2796491698 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55aeb76688b0 0x55aeb76716a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:33:08.266 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:08.261+0000 7f75ca000640 1 -- 172.21.15.47:0/2796491698 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55aeb7669260 msgr2=0x55aeb766ea30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:33:08.266 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:08.261+0000 7f75ca000640 1 --2- 172.21.15.47:0/2796491698 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55aeb7669260 0x55aeb766ea30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:33:08.266 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:08.261+0000 7f75ca000640 1 -- 172.21.15.47:0/2796491698 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55aeb72e03f0 con 0x55aeb7671de0 2024-03-21T10:33:08.266 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:08.261+0000 7f75caa00640 1 --2- 172.21.15.47:0/2796491698 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55aeb7669260 0x55aeb766ea30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T10:33:08.267 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:08.262+0000 7f75ca000640 1 --2- 172.21.15.47:0/2796491698 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55aeb7671de0 0x55aeb766ef90 secure :-1 s=READY pgs=1095 cs=0 l=1 rev1=1 crypto rx=0x7f75b4005610 tx=0x7f75b4000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=773bc8c08f044bb1 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:33:08.267 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:08.262+0000 7f75b8c00640 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-03-21T10:33:08.267 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:08.262+0000 7f75b8c00640 1 -- 172.21.15.47:0/2796491698 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75b4004d50 con 0x55aeb7671de0 2024-03-21T10:33:08.267 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:08.262+0000 7f75d47d0740 1 -- 172.21.15.47:0/2796491698 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f75b4048070 con 0x55aeb7671de0 2024-03-21T10:33:08.267 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:08.262+0000 7f75b8c00640 1 -- 172.21.15.47:0/2796491698 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(99..99 src has 1..99) v4 ==== 9562+0+0 (secure 0 0 0) 0x7f75b4005040 con 0x55aeb7671de0 2024-03-21T10:33:08.267 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:08.263+0000 7f75b8c00640 1 -- 172.21.15.47:0/2796491698 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f75b4060050 con 0x55aeb7671de0 2024-03-21T10:33:08.267 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:08.263+0000 7f75d47d0740 10 client.18027 fetch_fsmap learned FSMap version 32 2024-03-21T10:33:08.267 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:08.263+0000 7f75d47d0740 1 -- 172.21.15.47:0/2796491698 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x55aeb73aa7d0 con 0x55aeb7671de0 2024-03-21T10:33:08.268 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:08.263+0000 7f75b8c00640 1 -- 172.21.15.47:0/2796491698 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f75b4048070 con 0x55aeb7671de0 2024-03-21T10:33:08.268 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:08.263+0000 7f75b8c00640 1 -- 172.21.15.47:0/2796491698 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 32) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f75b4003710 con 0x55aeb7671de0 2024-03-21T10:33:08.268 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:08.263+0000 7f75d47d0740 10 client.18027 fetch_fsmap finished waiting for FSMap version 32 2024-03-21T10:33:08.268 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:08.263+0000 7f75d47d0740 10 client.18027 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.a 2024-03-21T10:33:08.268 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:08.263+0000 7f75d47d0740 20 client.18027 populate_metadata read hostname 'smithi047' 2024-03-21T10:33:08.268 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:08.263+0000 7f75d47d0740 1 --2- 172.21.15.47:0/2796491698 >> [v2:172.21.15.47:6840/3401578978,v1:172.21.15.47:6841/3401578978] conn(0x55aeb7681a80 0x55aeb7683e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:33:08.268 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:08.264+0000 7f75d47d0740 4 client.18027 mds_command: new command op to 14619 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T10:33:08.268 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:08.264+0000 7f75d47d0740 1 -- 172.21.15.47:0/2796491698 --> [v2:172.21.15.47:6840/3401578978,v1:172.21.15.47:6841/3401578978] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55aeb75d9800 con 0x55aeb7681a80 2024-03-21T10:33:08.268 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:08.264+0000 7f75caa00640 1 --2- 172.21.15.47:0/2796491698 >> [v2:172.21.15.47:6840/3401578978,v1:172.21.15.47:6841/3401578978] conn(0x55aeb7681a80 0x55aeb7683e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:33:08.268 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:08.264+0000 7f75caa00640 1 --2- 172.21.15.47:0/2796491698 >> [v2:172.21.15.47:6840/3401578978,v1:172.21.15.47:6841/3401578978] conn(0x55aeb7681a80 0x55aeb7683e70 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=d13b6c5863a20231 server_cookie=929a09f4b3a16018 in_seq=0 out_seq=0 2024-03-21T10:33:08.268 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:08.264+0000 7f75b8c00640 10 client.18027 ms_handle_connect on v2:172.21.15.47:6840/3401578978 2024-03-21T10:33:08.268 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:08.265+0000 7f75b8c00640 1 -- 172.21.15.47:0/2796491698 <== mds.4 v2:172.21.15.47:6840/3401578978 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55aeb75d9800 con 0x55aeb7681a80 2024-03-21T10:33:08.269 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:08.265+0000 7f75b8c00640 10 client.18027 handle_command_reply: tid=0 2024-03-21T10:33:08.269 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:08.265+0000 7f75d47d0740 1 client.18027 shutdown 2024-03-21T10:33:08.269 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:08.265+0000 7f75d47d0740 1 -- 172.21.15.47:0/2796491698 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55aeb7671de0 msgr2=0x55aeb766ef90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:33:08.269 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:08.265+0000 7f75d47d0740 1 --2- 172.21.15.47:0/2796491698 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55aeb7671de0 0x55aeb766ef90 secure :-1 s=READY pgs=1095 cs=0 l=1 rev1=1 crypto rx=0x7f75b4005610 tx=0x7f75b4000bc0 comp rx=0 tx=0).stop 2024-03-21T10:33:08.269 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:08.265+0000 7f75d47d0740 1 -- 172.21.15.47:0/2796491698 shutdown_connections 2024-03-21T10:33:08.269 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:08.265+0000 7f75d47d0740 1 --2- 172.21.15.47:0/2796491698 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55aeb7671de0 0x55aeb766ef90 unknown :-1 s=CLOSED pgs=1095 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:33:08.269 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:08.265+0000 7f75d47d0740 1 --2- 172.21.15.47:0/2796491698 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55aeb7669260 0x55aeb766ea30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:33:08.269 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:08.265+0000 7f75d47d0740 1 --2- 172.21.15.47:0/2796491698 >> [v2:172.21.15.47:6840/3401578978,v1:172.21.15.47:6841/3401578978] conn(0x55aeb7681a80 0x55aeb7683e70 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:33:08.269 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:08.265+0000 7f75d47d0740 1 --2- 172.21.15.47:0/2796491698 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55aeb76688b0 0x55aeb76716a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:33:08.269 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:08.265+0000 7f75b8c00640 0 client.18027 ms_handle_reset on v2:172.21.15.47:6840/3401578978 2024-03-21T10:33:08.269 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:08.265+0000 7f75d47d0740 1 -- 172.21.15.47:0/2796491698 >> 172.21.15.47:0/2796491698 conn(0x55aeb7662f80 msgr2=0x55aeb7664cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:33:08.270 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:08.266+0000 7f75d47d0740 1 -- 172.21.15.47:0/2796491698 shutdown_connections 2024-03-21T10:33:08.270 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:08.266+0000 7f75d47d0740 1 -- 172.21.15.47:0/2796491698 wait complete. 2024-03-21T10:33:08.270 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:08.266+0000 7f75d47d0740 20 client.18027 trim_cache size 0 max 16384 2024-03-21T10:33:08.279 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:08.280+0000 7f75d47d0740 1 Processor -- start 2024-03-21T10:33:08.279 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:08.280+0000 7f75d47d0740 1 -- start start 2024-03-21T10:33:08.279 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:08.280+0000 7f75d47d0740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55aeb76688b0 0x55aeb76643d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:33:08.279 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:08.280+0000 7f75d47d0740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55aeb7669260 0x55aeb7671160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:33:08.279 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:08.280+0000 7f75d47d0740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55aeb7671de0 0x55aeb76716a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:33:08.279 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:08.280+0000 7f75d47d0740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75cc1b08d0 con 0x55aeb76688b0 2024-03-21T10:33:08.280 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:08.280+0000 7f75d47d0740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75cc1b2250 con 0x55aeb7671de0 2024-03-21T10:33:08.280 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:08.280+0000 7f75d47d0740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75cc1b0f30 con 0x55aeb7669260 2024-03-21T10:33:08.281 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:08.280+0000 7f75ca000640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55aeb76688b0 0x55aeb76643d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:33:08.281 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:08.280+0000 7f75ca000640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55aeb76688b0 0x55aeb76643d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:46808/0 (socket says 172.21.15.47:46808) 2024-03-21T10:33:08.281 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:08.280+0000 7f75ca000640 1 -- 172.21.15.47:0/4186290943 learned_addr learned my addr 172.21.15.47:0/4186290943 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:33:08.281 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:08.281+0000 7f75caa00640 1 --2- 172.21.15.47:0/4186290943 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55aeb7669260 0x55aeb7671160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:33:08.281 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:08.281+0000 7f75ca000640 1 -- 172.21.15.47:0/4186290943 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55aeb7669260 msgr2=0x55aeb7671160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:33:08.281 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:08.281+0000 7f75ca000640 1 --2- 172.21.15.47:0/4186290943 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55aeb7669260 0x55aeb7671160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:33:08.281 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:08.281+0000 7f75ca000640 1 -- 172.21.15.47:0/4186290943 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55aeb7671de0 msgr2=0x55aeb76716a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:33:08.281 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:08.281+0000 7f75ca000640 1 --2- 172.21.15.47:0/4186290943 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55aeb7671de0 0x55aeb76716a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:33:08.281 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:08.281+0000 7f75ca000640 1 -- 172.21.15.47:0/4186290943 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55aeb7665280 con 0x55aeb76688b0 2024-03-21T10:33:08.282 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:08.281+0000 7f75ca000640 1 --2- 172.21.15.47:0/4186290943 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55aeb76688b0 0x55aeb76643d0 secure :-1 s=READY pgs=1096 cs=0 l=1 rev1=1 crypto rx=0x7f75b4009850 tx=0x7f75b402c2b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=32585cdac50f9f3f server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:33:08.282 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:08.281+0000 7f75bb400640 1 -- 172.21.15.47:0/4186290943 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75b4005d50 con 0x55aeb76688b0 2024-03-21T10:33:08.282 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:08.281+0000 7f75bb400640 1 -- 172.21.15.47:0/4186290943 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f75b4048040 con 0x55aeb76688b0 2024-03-21T10:33:08.282 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:08.281+0000 7f75bb400640 1 -- 172.21.15.47:0/4186290943 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75b4004280 con 0x55aeb76688b0 2024-03-21T10:33:08.287 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:08.282+0000 7f75d47d0740 1 -- 172.21.15.47:0/4186290943 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55aeb76688b0 msgr2=0x55aeb76643d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:33:08.287 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:08.282+0000 7f75d47d0740 1 --2- 172.21.15.47:0/4186290943 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55aeb76688b0 0x55aeb76643d0 secure :-1 s=READY pgs=1096 cs=0 l=1 rev1=1 crypto rx=0x7f75b4009850 tx=0x7f75b402c2b0 comp rx=0 tx=0).stop 2024-03-21T10:33:08.287 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:08.282+0000 7f75d47d0740 1 -- 172.21.15.47:0/4186290943 shutdown_connections 2024-03-21T10:33:08.287 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:08.282+0000 7f75d47d0740 1 --2- 172.21.15.47:0/4186290943 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55aeb7671de0 0x55aeb76716a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:33:08.287 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:08.282+0000 7f75d47d0740 1 --2- 172.21.15.47:0/4186290943 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55aeb7669260 0x55aeb7671160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:33:08.287 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:08.282+0000 7f75d47d0740 1 --2- 172.21.15.47:0/4186290943 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55aeb76688b0 0x55aeb76643d0 unknown :-1 s=CLOSED pgs=1096 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:33:08.287 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:08.282+0000 7f75d47d0740 1 -- 172.21.15.47:0/4186290943 >> 172.21.15.47:0/4186290943 conn(0x55aeb7662f80 msgr2=0x55aeb766da80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:33:08.287 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:08.282+0000 7f75d47d0740 1 -- 172.21.15.47:0/4186290943 shutdown_connections 2024-03-21T10:33:08.287 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:08.282+0000 7f75d47d0740 1 -- 172.21.15.47:0/4186290943 wait complete. 2024-03-21T10:33:08.287 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:08.283+0000 7f75d47d0740 1 -- start start 2024-03-21T10:33:08.288 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:08.283+0000 7f75d47d0740 1 Processor -- start 2024-03-21T10:33:08.288 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:08.283+0000 7f75d47d0740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55aeb76688b0 0x55aeb7684710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:33:08.288 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:08.283+0000 7f75d47d0740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55aeb7669260 0x55aeb7684c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:33:08.288 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:08.283+0000 7f75d47d0740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55aeb7671de0 0x55aeb76dbfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:33:08.288 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:08.283+0000 7f75d47d0740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75cc1b1bf0 con 0x55aeb7671de0 2024-03-21T10:33:08.288 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:08.283+0000 7f75d47d0740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75cc1b1590 con 0x55aeb7669260 2024-03-21T10:33:08.288 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:08.283+0000 7f75d47d0740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75cc1aeab0 con 0x55aeb76688b0 2024-03-21T10:33:08.288 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:08.283+0000 7f75d47d0740 10 client.0 _ll_register_callbacks cb 0x7f75b4009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T10:33:08.288 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:08.284+0000 7f75ca000640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55aeb7671de0 0x55aeb76dbfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:33:08.288 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:08.284+0000 7f75ca000640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55aeb7671de0 0x55aeb76dbfb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:46818/0 (socket says 172.21.15.47:46818) 2024-03-21T10:33:08.288 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:08.284+0000 7f75ca000640 1 -- 172.21.15.47:0/1533703211 learned_addr learned my addr 172.21.15.47:0/1533703211 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:33:08.288 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:08.284+0000 7f75caa00640 1 --2- 172.21.15.47:0/1533703211 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55aeb7669260 0x55aeb7684c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:33:08.288 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:08.284+0000 7f75ca000640 1 --2- 172.21.15.47:0/1533703211 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55aeb76688b0 0x55aeb7684710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:33:08.289 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:08.284+0000 7f75ca000640 1 -- 172.21.15.47:0/1533703211 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55aeb76688b0 msgr2=0x55aeb7684710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:33:08.289 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:08.284+0000 7f75ca000640 1 --2- 172.21.15.47:0/1533703211 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55aeb76688b0 0x55aeb7684710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:33:08.289 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:08.284+0000 7f75ca000640 1 -- 172.21.15.47:0/1533703211 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55aeb7669260 msgr2=0x55aeb7684c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:33:08.289 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:08.284+0000 7f75ca000640 1 --2- 172.21.15.47:0/1533703211 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55aeb7669260 0x55aeb7684c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:33:08.289 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:08.284+0000 7f75ca000640 1 -- 172.21.15.47:0/1533703211 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55aeb7669000 con 0x55aeb7671de0 2024-03-21T10:33:08.289 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:08.284+0000 7f75ca000640 1 --2- 172.21.15.47:0/1533703211 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55aeb7671de0 0x55aeb76dbfb0 secure :-1 s=READY pgs=1097 cs=0 l=1 rev1=1 crypto rx=0x7f75b4009430 tx=0x7f75b405d040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1c49ea61004dc1b5 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:33:08.289 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:08.284+0000 7f75b8c00640 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-03-21T10:33:08.289 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:08.285+0000 7f75b8c00640 1 -- 172.21.15.47:0/1533703211 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75b40037f0 con 0x55aeb7671de0 2024-03-21T10:33:08.289 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:08.285+0000 7f75b8c00640 1 -- 172.21.15.47:0/1533703211 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(99..99 src has 1..99) v4 ==== 9562+0+0 (secure 0 0 0) 0x7f75b402c780 con 0x55aeb7671de0 2024-03-21T10:33:08.289 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:08.285+0000 7f75d47d0740 1 -- 172.21.15.47:0/1533703211 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f75b4048040 con 0x55aeb7671de0 2024-03-21T10:33:08.289 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:08.285+0000 7f75b8c00640 1 -- 172.21.15.47:0/1533703211 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f75b402cb90 con 0x55aeb7671de0 2024-03-21T10:33:08.289 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:08.285+0000 7f75d47d0740 10 client.18033 fetch_fsmap learned FSMap version 32 2024-03-21T10:33:08.289 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:08.285+0000 7f75d47d0740 1 -- 172.21.15.47:0/1533703211 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x55aeb76dd4e0 con 0x55aeb7671de0 2024-03-21T10:33:08.289 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:08.285+0000 7f75b8c00640 1 -- 172.21.15.47:0/1533703211 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f75b4048040 con 0x55aeb7671de0 2024-03-21T10:33:08.290 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:08.285+0000 7f75b8c00640 1 -- 172.21.15.47:0/1533703211 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 32) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f75b4043480 con 0x55aeb7671de0 2024-03-21T10:33:08.290 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:08.285+0000 7f75d47d0740 10 client.18033 fetch_fsmap finished waiting for FSMap version 32 2024-03-21T10:33:08.290 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:08.285+0000 7f75d47d0740 10 client.18033 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.a 2024-03-21T10:33:08.290 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:08.285+0000 7f75d47d0740 20 client.18033 populate_metadata read hostname 'smithi047' 2024-03-21T10:33:08.290 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:08.286+0000 7f75d47d0740 1 --2- 172.21.15.47:0/1533703211 >> [v2:172.21.15.47:6840/3401578978,v1:172.21.15.47:6841/3401578978] conn(0x55aeb76dd850 0x55aeb76dfc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:33:08.290 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:08.286+0000 7f75d47d0740 4 client.18033 mds_command: new command op to 14619 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-21T10:33:08.290 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:08.286+0000 7f75d47d0740 1 -- 172.21.15.47:0/1533703211 --> [v2:172.21.15.47:6840/3401578978,v1:172.21.15.47:6841/3401578978] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55aeb75d3b50 con 0x55aeb76dd850 2024-03-21T10:33:08.290 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:08.286+0000 7f75caa00640 1 --2- 172.21.15.47:0/1533703211 >> [v2:172.21.15.47:6840/3401578978,v1:172.21.15.47:6841/3401578978] conn(0x55aeb76dd850 0x55aeb76dfc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:33:08.290 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:08.286+0000 7f75caa00640 1 --2- 172.21.15.47:0/1533703211 >> [v2:172.21.15.47:6840/3401578978,v1:172.21.15.47:6841/3401578978] conn(0x55aeb76dd850 0x55aeb76dfc70 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=56729fb7a4de70d0 server_cookie=5252f8e1c158f25f in_seq=0 out_seq=0 2024-03-21T10:33:08.290 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:08.286+0000 7f75b8c00640 10 client.18033 ms_handle_connect on v2:172.21.15.47:6840/3401578978 2024-03-21T10:33:08.290 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:08.287+0000 7f75b8c00640 1 -- 172.21.15.47:0/1533703211 <== mds.4 v2:172.21.15.47:6840/3401578978 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55aeb75d3b50 con 0x55aeb76dd850 2024-03-21T10:33:08.290 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:08.287+0000 7f75b8c00640 10 client.18033 handle_command_reply: tid=0 2024-03-21T10:33:08.290 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:08.287+0000 7f75d47d0740 1 client.18033 shutdown 2024-03-21T10:33:08.290 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:08.287+0000 7f75d47d0740 1 -- 172.21.15.47:0/1533703211 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55aeb7671de0 msgr2=0x55aeb76dbfb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:33:08.291 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:08.287+0000 7f75d47d0740 1 --2- 172.21.15.47:0/1533703211 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55aeb7671de0 0x55aeb76dbfb0 secure :-1 s=READY pgs=1097 cs=0 l=1 rev1=1 crypto rx=0x7f75b4009430 tx=0x7f75b405d040 comp rx=0 tx=0).stop 2024-03-21T10:33:08.291 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:08.287+0000 7f75d47d0740 1 -- 172.21.15.47:0/1533703211 shutdown_connections 2024-03-21T10:33:08.291 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:08.287+0000 7f75d47d0740 1 --2- 172.21.15.47:0/1533703211 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55aeb7671de0 0x55aeb76dbfb0 unknown :-1 s=CLOSED pgs=1097 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:33:08.291 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:08.287+0000 7f75d47d0740 1 --2- 172.21.15.47:0/1533703211 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55aeb7669260 0x55aeb7684c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:33:08.291 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:08.287+0000 7f75d47d0740 1 --2- 172.21.15.47:0/1533703211 >> [v2:172.21.15.47:6840/3401578978,v1:172.21.15.47:6841/3401578978] conn(0x55aeb76dd850 0x55aeb76dfc70 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:33:08.291 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:08.287+0000 7f75d47d0740 1 --2- 172.21.15.47:0/1533703211 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55aeb76688b0 0x55aeb7684710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:33:08.291 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:08.287+0000 7f75d47d0740 1 -- 172.21.15.47:0/1533703211 >> 172.21.15.47:0/1533703211 conn(0x55aeb7662f80 msgr2=0x55aeb76618d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:33:08.291 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:08.287+0000 7f75b8c00640 0 client.18033 ms_handle_reset on v2:172.21.15.47:6840/3401578978 2024-03-21T10:33:08.291 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:08.287+0000 7f75d47d0740 1 -- 172.21.15.47:0/1533703211 shutdown_connections 2024-03-21T10:33:08.291 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:08.287+0000 7f75d47d0740 1 -- 172.21.15.47:0/1533703211 wait complete. 2024-03-21T10:33:08.291 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:08.287+0000 7f75d47d0740 20 client.18033 trim_cache size 0 max 16384 2024-03-21T10:33:08.291 INFO:teuthology.orchestra.run.smithi047.stdout:[] 2024-03-21T10:33:08.292 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:08.288+0000 7f75cbe00640 1 -- 172.21.15.47:0/3082696022 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f75b00a3090 msgr2=0x7f75b00a5550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:33:08.292 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:08.288+0000 7f75cbe00640 1 --2- 172.21.15.47:0/3082696022 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f75b00a3090 0x7f75b00a5550 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f75c40cc9a0 tx=0x7f75c40046c0 comp rx=0 tx=0).stop 2024-03-21T10:33:08.292 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:08.288+0000 7f75cbe00640 1 -- 172.21.15.47:0/3082696022 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f75cc1075f0 msgr2=0x7f75cc19e730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:33:08.292 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:08.288+0000 7f75cbe00640 1 --2- 172.21.15.47:0/3082696022 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f75cc1075f0 0x7f75cc19e730 secure :-1 s=READY pgs=1093 cs=0 l=1 rev1=1 crypto rx=0x7f75c400d750 tx=0x7f75c400dc10 comp rx=0 tx=0).stop 2024-03-21T10:33:08.292 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:08.288+0000 7f75cbe00640 1 -- 172.21.15.47:0/3082696022 shutdown_connections 2024-03-21T10:33:08.293 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:08.288+0000 7f75cbe00640 1 --2- 172.21.15.47:0/3082696022 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f75cc1075f0 0x7f75cc19e730 unknown :-1 s=CLOSED pgs=1093 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:33:08.293 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:08.288+0000 7f75cbe00640 1 --2- 172.21.15.47:0/3082696022 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f75cc101ef0 0x7f75cc19a3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:33:08.293 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:08.288+0000 7f75cbe00640 1 --2- 172.21.15.47:0/3082696022 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f75b00a3090 0x7f75b00a5550 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:33:08.293 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:08.288+0000 7f75cbe00640 1 --2- 172.21.15.47:0/3082696022 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f75cc101530 0x7f75cc199e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:33:08.293 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:08.288+0000 7f75cbe00640 1 -- 172.21.15.47:0/3082696022 >> 172.21.15.47:0/3082696022 conn(0x7f75cc0fd040 msgr2=0x7f75cc0fe5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:33:08.293 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:08.288+0000 7f75cbe00640 1 -- 172.21.15.47:0/3082696022 shutdown_connections 2024-03-21T10:33:08.293 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:08.288+0000 7f75cbe00640 1 -- 172.21.15.47:0/3082696022 wait complete. 2024-03-21T10:33:08.305 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-21T10:33:08.306 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-03-21T10:33:08.515 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:08.515+0000 7f6f22a00640 1 -- 172.21.15.47:0/2521475394 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6f1c10baf0 msgr2=0x7f6f1c10df90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:33:08.515 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:08.515+0000 7f6f22a00640 1 --2- 172.21.15.47:0/2521475394 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6f1c10baf0 0x7f6f1c10df90 secure :-1 s=READY pgs=1098 cs=0 l=1 rev1=1 crypto rx=0x7f6f14009ca0 tx=0x7f6f1402c180 comp rx=0 tx=0).stop 2024-03-21T10:33:08.515 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:08.516+0000 7f6f22a00640 1 -- 172.21.15.47:0/2521475394 shutdown_connections 2024-03-21T10:33:08.516 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:08.516+0000 7f6f22a00640 1 --2- 172.21.15.47:0/2521475394 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6f1c10baf0 0x7f6f1c10df90 unknown :-1 s=CLOSED pgs=1098 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:33:08.516 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:08.516+0000 7f6f22a00640 1 --2- 172.21.15.47:0/2521475394 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6f1c102ff0 0x7f6f1c10b4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:33:08.516 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:08.516+0000 7f6f22a00640 1 --2- 172.21.15.47:0/2521475394 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6f1c1026d0 0x7f6f1c102ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:33:08.516 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:08.516+0000 7f6f22a00640 1 -- 172.21.15.47:0/2521475394 >> 172.21.15.47:0/2521475394 conn(0x7f6f1c0fbef0 msgr2=0x7f6f1c0fe310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:33:08.516 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:08.516+0000 7f6f22a00640 1 -- 172.21.15.47:0/2521475394 shutdown_connections 2024-03-21T10:33:08.516 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:08.516+0000 7f6f22a00640 1 -- 172.21.15.47:0/2521475394 wait complete. 2024-03-21T10:33:08.518 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:08.517+0000 7f6f22a00640 1 Processor -- start 2024-03-21T10:33:08.518 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:08.517+0000 7f6f22a00640 1 -- start start 2024-03-21T10:33:08.518 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:08.517+0000 7f6f22a00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6f1c1026d0 0x7f6f1c199ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:33:08.518 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:08.517+0000 7f6f22a00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6f1c102ff0 0x7f6f1c19a3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:33:08.518 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:08.517+0000 7f6f22a00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6f1c10baf0 0x7f6f1c19e760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:33:08.519 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:08.517+0000 7f6f22a00640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f1c111780 con 0x7f6f1c1026d0 2024-03-21T10:33:08.519 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:08.517+0000 7f6f22a00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f1c111600 con 0x7f6f1c10baf0 2024-03-21T10:33:08.519 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:08.517+0000 7f6f22a00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f1c111900 con 0x7f6f1c102ff0 2024-03-21T10:33:08.519 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:08.517+0000 7f6f21600640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6f1c1026d0 0x7f6f1c199ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:33:08.519 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:08.518+0000 7f6f21600640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6f1c1026d0 0x7f6f1c199ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:46824/0 (socket says 172.21.15.47:46824) 2024-03-21T10:33:08.519 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:08.518+0000 7f6f21600640 1 -- 172.21.15.47:0/2006104349 learned_addr learned my addr 172.21.15.47:0/2006104349 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:33:08.519 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:08.518+0000 7f6f21600640 1 -- 172.21.15.47:0/2006104349 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6f1c102ff0 msgr2=0x7f6f1c19a3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:33:08.520 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:08.518+0000 7f6f21600640 1 --2- 172.21.15.47:0/2006104349 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6f1c102ff0 0x7f6f1c19a3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:33:08.520 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:08.518+0000 7f6f21600640 1 -- 172.21.15.47:0/2006104349 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6f1c10baf0 msgr2=0x7f6f1c19e760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:33:08.520 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:08.518+0000 7f6f22000640 1 --2- 172.21.15.47:0/2006104349 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6f1c10baf0 0x7f6f1c19e760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:33:08.520 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:08.518+0000 7f6f21600640 1 --2- 172.21.15.47:0/2006104349 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6f1c10baf0 0x7f6f1c19e760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:33:08.520 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:08.518+0000 7f6f21600640 1 -- 172.21.15.47:0/2006104349 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f1c19ee90 con 0x7f6f1c1026d0 2024-03-21T10:33:08.520 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:08.518+0000 7f6f21600640 1 --2- 172.21.15.47:0/2006104349 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6f1c1026d0 0x7f6f1c199ea0 secure :-1 s=READY pgs=1099 cs=0 l=1 rev1=1 crypto rx=0x7f6f1000c970 tx=0x7f6f1000ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:33:08.520 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:08.519+0000 7f6f1a000640 1 -- 172.21.15.47:0/2006104349 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f10007bb0 con 0x7f6f1c1026d0 2024-03-21T10:33:08.520 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:08.519+0000 7f6f1a000640 1 -- 172.21.15.47:0/2006104349 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f6f10007d50 con 0x7f6f1c1026d0 2024-03-21T10:33:08.520 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:08.519+0000 7f6f22a00640 1 -- 172.21.15.47:0/2006104349 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6f1c19f180 con 0x7f6f1c1026d0 2024-03-21T10:33:08.520 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:08.519+0000 7f6f1a000640 1 -- 172.21.15.47:0/2006104349 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f1000f450 con 0x7f6f1c1026d0 2024-03-21T10:33:08.520 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:08.519+0000 7f6f22a00640 1 -- 172.21.15.47:0/2006104349 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6f1c1a6a10 con 0x7f6f1c1026d0 2024-03-21T10:33:08.523 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:08.523+0000 7f6f26c01740 1 Processor -- start 2024-03-21T10:33:08.523 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:08.524+0000 7f6f26c01740 1 -- start start 2024-03-21T10:33:08.523 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:08.524+0000 7f6f1a000640 1 -- 172.21.15.47:0/2006104349 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 27) v1 ==== 137985+0+0 (secure 0 0 0) 0x7f6f10004f20 con 0x7f6f1c1026d0 2024-03-21T10:33:08.524 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:08.524+0000 7f6f26c01740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a76db628b0 0x55a76db62c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:33:08.524 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:08.524+0000 7f6f26c01740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a76db63260 0x55a76db6b810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:33:08.524 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:08.524+0000 7f6f26c01740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a76db6bde0 0x55a76db6e230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:33:08.525 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:08.524+0000 7f6f26c01740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a76dac62c0 con 0x55a76db6bde0 2024-03-21T10:33:08.525 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:08.524+0000 7f6f26c01740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a76da6ec90 con 0x55a76db628b0 2024-03-21T10:33:08.525 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:08.524+0000 7f6f26c01740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a76dac2360 con 0x55a76db63260 2024-03-21T10:33:08.525 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:08.524+0000 7f6f20c00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a76db6bde0 0x55a76db6e230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:33:08.525 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:08.524+0000 7f6f20c00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a76db6bde0 0x55a76db6e230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:46840/0 (socket says 172.21.15.47:46840) 2024-03-21T10:33:08.525 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:08.524+0000 7f6f20c00640 1 -- 172.21.15.47:0/2038474284 learned_addr learned my addr 172.21.15.47:0/2038474284 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:33:08.525 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:08.525+0000 7f6f22000640 1 --2- 172.21.15.47:0/2038474284 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a76db628b0 0x55a76db62c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:33:08.525 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:08.525+0000 7f6f21600640 1 --2- 172.21.15.47:0/2038474284 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a76db63260 0x55a76db6b810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:33:08.525 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:08.525+0000 7f6f20c00640 1 -- 172.21.15.47:0/2038474284 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a76db63260 msgr2=0x55a76db6b810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:33:08.525 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:08.525+0000 7f6f20c00640 1 --2- 172.21.15.47:0/2038474284 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a76db63260 0x55a76db6b810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:33:08.525 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:08.525+0000 7f6f20c00640 1 -- 172.21.15.47:0/2038474284 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a76db628b0 msgr2=0x55a76db62c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:33:08.525 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:08.525+0000 7f6f20c00640 1 --2- 172.21.15.47:0/2038474284 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a76db628b0 0x55a76db62c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:33:08.526 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:08.525+0000 7f6f20c00640 1 -- 172.21.15.47:0/2038474284 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a76da93080 con 0x55a76db6bde0 2024-03-21T10:33:08.526 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:08.525+0000 7f6f20c00640 1 --2- 172.21.15.47:0/2038474284 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a76db6bde0 0x55a76db6e230 secure :-1 s=READY pgs=1100 cs=0 l=1 rev1=1 crypto rx=0x7f6f08009b50 tx=0x7f6f0800e720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bc94f8dcbe6bd583 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:33:08.530 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:08.525+0000 7f6f0e600640 1 -- 172.21.15.47:0/2038474284 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f08009e40 con 0x55a76db6bde0 2024-03-21T10:33:08.530 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:08.526+0000 7f6f0e600640 1 -- 172.21.15.47:0/2038474284 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f6f080054d0 con 0x55a76db6bde0 2024-03-21T10:33:08.531 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:08.526+0000 7f6f0e600640 1 -- 172.21.15.47:0/2038474284 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f08010690 con 0x55a76db6bde0 2024-03-21T10:33:08.531 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:08.526+0000 7f6f1a000640 1 --2- 172.21.15.47:0/2006104349 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f6f040a3090 0x7f6f040a5550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:33:08.531 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:08.526+0000 7f6f26c01740 1 -- 172.21.15.47:0/2038474284 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a76db6bde0 msgr2=0x55a76db6e230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:33:08.531 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:08.526+0000 7f6f26c01740 1 --2- 172.21.15.47:0/2038474284 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a76db6bde0 0x55a76db6e230 secure :-1 s=READY pgs=1100 cs=0 l=1 rev1=1 crypto rx=0x7f6f08009b50 tx=0x7f6f0800e720 comp rx=0 tx=0).stop 2024-03-21T10:33:08.531 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:08.526+0000 7f6f1a000640 1 -- 172.21.15.47:0/2006104349 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 9562+0+0 (secure 0 0 0) 0x7f6f100cfa20 con 0x7f6f1c1026d0 2024-03-21T10:33:08.531 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:08.526+0000 7f6f22000640 1 --2- 172.21.15.47:0/2006104349 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f6f040a3090 0x7f6f040a5550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:33:08.531 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:08.527+0000 7f6f26c01740 1 -- 172.21.15.47:0/2038474284 shutdown_connections 2024-03-21T10:33:08.531 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:08.527+0000 7f6f26c01740 1 --2- 172.21.15.47:0/2038474284 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a76db6bde0 0x55a76db6e230 unknown :-1 s=CLOSED pgs=1100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:33:08.531 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:08.527+0000 7f6f26c01740 1 --2- 172.21.15.47:0/2038474284 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a76db63260 0x55a76db6b810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:33:08.531 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:08.527+0000 7f6f26c01740 1 --2- 172.21.15.47:0/2038474284 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a76db628b0 0x55a76db62c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:33:08.531 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:08.527+0000 7f6f26c01740 1 -- 172.21.15.47:0/2038474284 >> 172.21.15.47:0/2038474284 conn(0x55a76db5cf80 msgr2=0x55a76db5d360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:33:08.532 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:08.527+0000 7f6f26c01740 1 -- 172.21.15.47:0/2038474284 shutdown_connections 2024-03-21T10:33:08.532 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:08.527+0000 7f6f26c01740 1 -- 172.21.15.47:0/2038474284 wait complete. 2024-03-21T10:33:08.532 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:08.527+0000 7f6f26c01740 1 -- start start 2024-03-21T10:33:08.532 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:08.528+0000 7f6f22000640 1 --2- 172.21.15.47:0/2006104349 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f6f040a3090 0x7f6f040a5550 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f6f14005cc0 tx=0x7f6f14005c50 comp rx=0 tx=0).ready entity=mgr.15687 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:33:08.532 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:08.528+0000 7f6f26c01740 1 Processor -- start 2024-03-21T10:33:08.532 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:08.528+0000 7f6f26c01740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a76db628b0 0x55a76db6b6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:33:08.532 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:08.528+0000 7f6f26c01740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a76db63260 0x55a76db68a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:33:08.532 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:08.528+0000 7f6f26c01740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a76db6bde0 0x55a76db68ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:33:08.532 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:08.528+0000 7f6f26c01740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a76db6ed50 con 0x55a76db628b0 2024-03-21T10:33:08.532 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:08.528+0000 7f6f26c01740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a76da820d0 con 0x55a76db6bde0 2024-03-21T10:33:08.532 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:08.528+0000 7f6f26c01740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a76db6eed0 con 0x55a76db63260 2024-03-21T10:33:08.532 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:08.528+0000 7f6f26c01740 10 client.0 _ll_register_callbacks cb 0x55a76da86e80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T10:33:08.533 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:08.528+0000 7f6f21600640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a76db628b0 0x55a76db6b6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:33:08.533 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:08.529+0000 7f6f21600640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a76db628b0 0x55a76db6b6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:46846/0 (socket says 172.21.15.47:46846) 2024-03-21T10:33:08.533 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:08.529+0000 7f6f21600640 1 -- 172.21.15.47:0/3943748761 learned_addr learned my addr 172.21.15.47:0/3943748761 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:33:08.533 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:08.529+0000 7f6f22000640 1 --2- 172.21.15.47:0/3943748761 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a76db6bde0 0x55a76db68ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:33:08.533 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:08.529+0000 7f6f20c00640 1 --2- 172.21.15.47:0/3943748761 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a76db63260 0x55a76db68a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:33:08.533 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:08.529+0000 7f6f21600640 1 -- 172.21.15.47:0/3943748761 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a76db63260 msgr2=0x55a76db68a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:33:08.533 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:08.529+0000 7f6f21600640 1 --2- 172.21.15.47:0/3943748761 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a76db63260 0x55a76db68a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:33:08.533 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:08.529+0000 7f6f21600640 1 -- 172.21.15.47:0/3943748761 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a76db6bde0 msgr2=0x55a76db68ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:33:08.533 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:08.529+0000 7f6f21600640 1 --2- 172.21.15.47:0/3943748761 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a76db6bde0 0x55a76db68ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:33:08.533 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:08.529+0000 7f6f21600640 1 -- 172.21.15.47:0/3943748761 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55a76d7da3f0 con 0x55a76db628b0 2024-03-21T10:33:08.533 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:08.529+0000 7f6f21600640 1 --2- 172.21.15.47:0/3943748761 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a76db628b0 0x55a76db6b6e0 secure :-1 s=READY pgs=1101 cs=0 l=1 rev1=1 crypto rx=0x7f6f100ccdc0 tx=0x7f6f1009b850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b531b554502b6efa server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:33:08.533 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:08.529+0000 7f6f02a00640 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-03-21T10:33:08.533 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:08.529+0000 7f6f02a00640 1 -- 172.21.15.47:0/3943748761 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f1001d040 con 0x55a76db628b0 2024-03-21T10:33:08.534 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:08.529+0000 7f6f02a00640 1 -- 172.21.15.47:0/3943748761 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(99..99 src has 1..99) v4 ==== 9562+0+0 (secure 0 0 0) 0x7f6f100a3f80 con 0x55a76db628b0 2024-03-21T10:33:08.534 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:08.530+0000 7f6f26c01740 1 -- 172.21.15.47:0/3943748761 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f6f080054d0 con 0x55a76db628b0 2024-03-21T10:33:08.534 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:08.530+0000 7f6f02a00640 1 -- 172.21.15.47:0/3943748761 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6f100cd1e0 con 0x55a76db628b0 2024-03-21T10:33:08.534 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:08.530+0000 7f6f26c01740 10 client.18045 fetch_fsmap learned FSMap version 32 2024-03-21T10:33:08.534 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:08.530+0000 7f6f26c01740 1 -- 172.21.15.47:0/3943748761 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x55a76d8a47d0 con 0x55a76db628b0 2024-03-21T10:33:08.534 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:08.530+0000 7f6f02a00640 1 -- 172.21.15.47:0/3943748761 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f6f080054d0 con 0x55a76db628b0 2024-03-21T10:33:08.534 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:08.530+0000 7f6f02a00640 1 -- 172.21.15.47:0/3943748761 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 32) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f6f100860c0 con 0x55a76db628b0 2024-03-21T10:33:08.534 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:08.530+0000 7f6f26c01740 10 client.18045 fetch_fsmap finished waiting for FSMap version 32 2024-03-21T10:33:08.534 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:08.530+0000 7f6f26c01740 10 client.18045 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.k 2024-03-21T10:33:08.534 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:08.530+0000 7f6f26c01740 20 client.18045 populate_metadata read hostname 'smithi047' 2024-03-21T10:33:08.534 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:08.530+0000 7f6f26c01740 1 --2- 172.21.15.47:0/3943748761 >> [v2:172.21.15.100:6832/2723302038,v1:172.21.15.100:6833/2723302038] conn(0x55a76db7bb90 0x55a76db7df80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:33:08.535 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:08.530+0000 7f6f26c01740 4 client.18045 mds_command: new command op to 34427 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T10:33:08.535 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:08.530+0000 7f6f26c01740 1 -- 172.21.15.47:0/3943748761 --> [v2:172.21.15.100:6832/2723302038,v1:172.21.15.100:6833/2723302038] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55a76dad3800 con 0x55a76db7bb90 2024-03-21T10:33:08.535 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:08.531+0000 7f6f20c00640 1 --2- 172.21.15.47:0/3943748761 >> [v2:172.21.15.100:6832/2723302038,v1:172.21.15.100:6833/2723302038] conn(0x55a76db7bb90 0x55a76db7df80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:33:08.536 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:08.532+0000 7f6f20c00640 1 --2- 172.21.15.47:0/3943748761 >> [v2:172.21.15.100:6832/2723302038,v1:172.21.15.100:6833/2723302038] conn(0x55a76db7bb90 0x55a76db7df80 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=2b02f7c673a6e24c server_cookie=6014cab769bfe8b2 in_seq=0 out_seq=0 2024-03-21T10:33:08.536 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:08.532+0000 7f6f02a00640 10 client.18045 ms_handle_connect on v2:172.21.15.100:6832/2723302038 2024-03-21T10:33:08.536 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:08.533+0000 7f6f02a00640 1 -- 172.21.15.47:0/3943748761 <== mds.3 v2:172.21.15.100:6832/2723302038 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55a76dad3800 con 0x55a76db7bb90 2024-03-21T10:33:08.536 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:08.533+0000 7f6f02a00640 10 client.18045 handle_command_reply: tid=0 2024-03-21T10:33:08.536 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:08.533+0000 7f6f26c01740 1 client.18045 shutdown 2024-03-21T10:33:08.536 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:08.533+0000 7f6f26c01740 1 -- 172.21.15.47:0/3943748761 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a76db628b0 msgr2=0x55a76db6b6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:33:08.536 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:08.533+0000 7f6f26c01740 1 --2- 172.21.15.47:0/3943748761 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a76db628b0 0x55a76db6b6e0 secure :-1 s=READY pgs=1101 cs=0 l=1 rev1=1 crypto rx=0x7f6f100ccdc0 tx=0x7f6f1009b850 comp rx=0 tx=0).stop 2024-03-21T10:33:08.537 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:08.533+0000 7f6f26c01740 1 -- 172.21.15.47:0/3943748761 shutdown_connections 2024-03-21T10:33:08.537 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:08.533+0000 7f6f26c01740 1 --2- 172.21.15.47:0/3943748761 >> [v2:172.21.15.100:6832/2723302038,v1:172.21.15.100:6833/2723302038] conn(0x55a76db7bb90 0x55a76db7df80 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:33:08.537 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:08.533+0000 7f6f26c01740 1 --2- 172.21.15.47:0/3943748761 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a76db6bde0 0x55a76db68ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:33:08.537 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:08.533+0000 7f6f26c01740 1 --2- 172.21.15.47:0/3943748761 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a76db63260 0x55a76db68a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:33:08.537 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:08.533+0000 7f6f26c01740 1 --2- 172.21.15.47:0/3943748761 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a76db628b0 0x55a76db6b6e0 unknown :-1 s=CLOSED pgs=1101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:33:08.537 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:08.533+0000 7f6f26c01740 1 -- 172.21.15.47:0/3943748761 >> 172.21.15.47:0/3943748761 conn(0x55a76db5cf80 msgr2=0x55a76db5ecd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:33:08.537 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:08.533+0000 7f6f02a00640 0 client.18045 ms_handle_reset on v2:172.21.15.100:6832/2723302038 2024-03-21T10:33:08.537 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:08.534+0000 7f6f26c01740 1 -- 172.21.15.47:0/3943748761 shutdown_connections 2024-03-21T10:33:08.537 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:08.534+0000 7f6f26c01740 1 -- 172.21.15.47:0/3943748761 wait complete. 2024-03-21T10:33:08.537 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:08.534+0000 7f6f26c01740 20 client.18045 trim_cache size 0 max 16384 2024-03-21T10:33:08.548 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:08.547+0000 7f6f26c01740 1 Processor -- start 2024-03-21T10:33:08.548 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:08.548+0000 7f6f26c01740 1 -- start start 2024-03-21T10:33:08.548 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:08.548+0000 7f6f26c01740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a76db628b0 0x55a76db5bfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:33:08.548 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:08.548+0000 7f6f26c01740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a76db63260 0x55a76db6b1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:33:08.548 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:08.548+0000 7f6f26c01740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a76db6bde0 0x55a76db6b6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:33:08.548 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:08.548+0000 7f6f26c01740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f1c1b0970 con 0x55a76db628b0 2024-03-21T10:33:08.549 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:08.548+0000 7f6f26c01740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f1c1b22f0 con 0x55a76db6bde0 2024-03-21T10:33:08.549 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:08.548+0000 7f6f26c01740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f1c1b0fd0 con 0x55a76db63260 2024-03-21T10:33:08.549 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:08.548+0000 7f6f21600640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a76db628b0 0x55a76db5bfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:33:08.549 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:08.548+0000 7f6f21600640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a76db628b0 0x55a76db5bfe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:46856/0 (socket says 172.21.15.47:46856) 2024-03-21T10:33:08.549 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:08.548+0000 7f6f21600640 1 -- 172.21.15.47:0/358255061 learned_addr learned my addr 172.21.15.47:0/358255061 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:33:08.549 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:08.548+0000 7f6f22000640 1 --2- 172.21.15.47:0/358255061 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a76db6bde0 0x55a76db6b6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:33:08.549 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:08.548+0000 7f6f20c00640 1 --2- 172.21.15.47:0/358255061 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a76db63260 0x55a76db6b1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:33:08.549 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:08.549+0000 7f6f21600640 1 -- 172.21.15.47:0/358255061 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a76db63260 msgr2=0x55a76db6b1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:33:08.549 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:08.549+0000 7f6f21600640 1 --2- 172.21.15.47:0/358255061 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a76db63260 0x55a76db6b1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:33:08.549 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:08.549+0000 7f6f21600640 1 -- 172.21.15.47:0/358255061 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a76db6bde0 msgr2=0x55a76db6b6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:33:08.549 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:08.549+0000 7f6f21600640 1 --2- 172.21.15.47:0/358255061 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a76db6bde0 0x55a76db6b6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:33:08.549 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:08.549+0000 7f6f21600640 1 -- 172.21.15.47:0/358255061 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a76db5cab0 con 0x55a76db628b0 2024-03-21T10:33:08.549 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:08.549+0000 7f6f21600640 1 --2- 172.21.15.47:0/358255061 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a76db628b0 0x55a76db5bfe0 secure :-1 s=READY pgs=1102 cs=0 l=1 rev1=1 crypto rx=0x7f6f10097e60 tx=0x7f6f10099f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b9029dd251872150 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:33:08.552 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:08.549+0000 7f6f0dc00640 1 -- 172.21.15.47:0/358255061 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f1009fb50 con 0x55a76db628b0 2024-03-21T10:33:08.552 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:08.549+0000 7f6f0dc00640 1 -- 172.21.15.47:0/358255061 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f6f100a4410 con 0x55a76db628b0 2024-03-21T10:33:08.552 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:08.549+0000 7f6f0dc00640 1 -- 172.21.15.47:0/358255061 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f10086240 con 0x55a76db628b0 2024-03-21T10:33:08.552 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:08.550+0000 7f6f26c01740 1 -- 172.21.15.47:0/358255061 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a76db628b0 msgr2=0x55a76db5bfe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:33:08.552 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:08.550+0000 7f6f26c01740 1 --2- 172.21.15.47:0/358255061 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a76db628b0 0x55a76db5bfe0 secure :-1 s=READY pgs=1102 cs=0 l=1 rev1=1 crypto rx=0x7f6f10097e60 tx=0x7f6f10099f10 comp rx=0 tx=0).stop 2024-03-21T10:33:08.553 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:08.550+0000 7f6f26c01740 1 -- 172.21.15.47:0/358255061 shutdown_connections 2024-03-21T10:33:08.553 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:08.550+0000 7f6f26c01740 1 --2- 172.21.15.47:0/358255061 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a76db6bde0 0x55a76db6b6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:33:08.553 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:08.550+0000 7f6f26c01740 1 --2- 172.21.15.47:0/358255061 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a76db63260 0x55a76db6b1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:33:08.553 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:08.550+0000 7f6f26c01740 1 --2- 172.21.15.47:0/358255061 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a76db628b0 0x55a76db5bfe0 unknown :-1 s=CLOSED pgs=1102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:33:08.553 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:08.550+0000 7f6f26c01740 1 -- 172.21.15.47:0/358255061 >> 172.21.15.47:0/358255061 conn(0x55a76db5cf80 msgr2=0x55a76db5b8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:33:08.553 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:08.550+0000 7f6f26c01740 1 -- 172.21.15.47:0/358255061 shutdown_connections 2024-03-21T10:33:08.553 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:08.550+0000 7f6f26c01740 1 -- 172.21.15.47:0/358255061 wait complete. 2024-03-21T10:33:08.553 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:08.551+0000 7f6f26c01740 1 -- start start 2024-03-21T10:33:08.553 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:08.551+0000 7f6f26c01740 1 Processor -- start 2024-03-21T10:33:08.553 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:08.551+0000 7f6f26c01740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a76db628b0 0x55a76db71bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:33:08.553 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:08.551+0000 7f6f26c01740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a76db63260 0x55a76db72110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:33:08.553 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:08.551+0000 7f6f26c01740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a76db6bde0 0x55a76dbd6420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:33:08.553 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:08.551+0000 7f6f26c01740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f1c1b1c90 con 0x55a76db6bde0 2024-03-21T10:33:08.554 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:08.551+0000 7f6f26c01740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f1c1b1630 con 0x55a76db63260 2024-03-21T10:33:08.554 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:08.551+0000 7f6f26c01740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f1c1aeb50 con 0x55a76db628b0 2024-03-21T10:33:08.554 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:08.551+0000 7f6f26c01740 10 client.0 _ll_register_callbacks cb 0x7f6f10007f00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T10:33:08.554 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:08.552+0000 7f6f22000640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a76db6bde0 0x55a76dbd6420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:33:08.554 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:08.552+0000 7f6f22000640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a76db6bde0 0x55a76dbd6420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:46870/0 (socket says 172.21.15.47:46870) 2024-03-21T10:33:08.554 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:08.552+0000 7f6f22000640 1 -- 172.21.15.47:0/84398176 learned_addr learned my addr 172.21.15.47:0/84398176 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:33:08.554 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:08.552+0000 7f6f21600640 1 --2- 172.21.15.47:0/84398176 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a76db628b0 0x55a76db71bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:33:08.554 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:08.552+0000 7f6f20c00640 1 --2- 172.21.15.47:0/84398176 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a76db63260 0x55a76db72110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:33:08.554 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:08.552+0000 7f6f22000640 1 -- 172.21.15.47:0/84398176 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a76db628b0 msgr2=0x55a76db71bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:33:08.554 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:08.552+0000 7f6f22000640 1 --2- 172.21.15.47:0/84398176 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a76db628b0 0x55a76db71bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:33:08.554 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:08.552+0000 7f6f22000640 1 -- 172.21.15.47:0/84398176 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a76db63260 msgr2=0x55a76db72110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:33:08.554 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:08.552+0000 7f6f22000640 1 --2- 172.21.15.47:0/84398176 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a76db63260 0x55a76db72110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:33:08.554 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:08.552+0000 7f6f22000640 1 -- 172.21.15.47:0/84398176 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55a76dbd6960 con 0x55a76db6bde0 2024-03-21T10:33:08.554 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:08.553+0000 7f6f22000640 1 --2- 172.21.15.47:0/84398176 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a76db6bde0 0x55a76dbd6420 secure :-1 s=READY pgs=1103 cs=0 l=1 rev1=1 crypto rx=0x7f6f14009ca0 tx=0x7f6f1402c180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=56e85cc286a3b777 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:33:08.555 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:08.553+0000 7f6f02a00640 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-03-21T10:33:08.555 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:08.553+0000 7f6f02a00640 1 -- 172.21.15.47:0/84398176 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f14050030 con 0x55a76db6bde0 2024-03-21T10:33:08.555 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:08.553+0000 7f6f26c01740 1 -- 172.21.15.47:0/84398176 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f6f100a4410 con 0x55a76db6bde0 2024-03-21T10:33:08.555 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:08.553+0000 7f6f02a00640 1 -- 172.21.15.47:0/84398176 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(99..99 src has 1..99) v4 ==== 9562+0+0 (secure 0 0 0) 0x7f6f140047a0 con 0x55a76db6bde0 2024-03-21T10:33:08.557 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:08.553+0000 7f6f02a00640 1 -- 172.21.15.47:0/84398176 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6f14039040 con 0x55a76db6bde0 2024-03-21T10:33:08.557 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:08.553+0000 7f6f26c01740 10 client.18051 fetch_fsmap learned FSMap version 32 2024-03-21T10:33:08.557 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:08.553+0000 7f6f26c01740 1 -- 172.21.15.47:0/84398176 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x55a76dbd7bd0 con 0x55a76db6bde0 2024-03-21T10:33:08.557 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:08.554+0000 7f6f02a00640 1 -- 172.21.15.47:0/84398176 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f6f100a4410 con 0x55a76db6bde0 2024-03-21T10:33:08.557 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:08.554+0000 7f6f02a00640 1 -- 172.21.15.47:0/84398176 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 32) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f6f140346e0 con 0x55a76db6bde0 2024-03-21T10:33:08.557 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:08.554+0000 7f6f26c01740 10 client.18051 fetch_fsmap finished waiting for FSMap version 32 2024-03-21T10:33:08.557 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:08.554+0000 7f6f26c01740 10 client.18051 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.k 2024-03-21T10:33:08.557 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:08.554+0000 7f6f26c01740 20 client.18051 populate_metadata read hostname 'smithi047' 2024-03-21T10:33:08.557 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:08.554+0000 7f6f26c01740 1 --2- 172.21.15.47:0/84398176 >> [v2:172.21.15.100:6832/2723302038,v1:172.21.15.100:6833/2723302038] conn(0x55a76dbd8380 0x55a76dbda7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:33:08.557 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:08.554+0000 7f6f26c01740 4 client.18051 mds_command: new command op to 34427 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-21T10:33:08.557 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:08.554+0000 7f6f26c01740 1 -- 172.21.15.47:0/84398176 --> [v2:172.21.15.100:6832/2723302038,v1:172.21.15.100:6833/2723302038] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55a76dacdb50 con 0x55a76dbd8380 2024-03-21T10:33:08.558 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:08.554+0000 7f6f21600640 1 --2- 172.21.15.47:0/84398176 >> [v2:172.21.15.100:6832/2723302038,v1:172.21.15.100:6833/2723302038] conn(0x55a76dbd8380 0x55a76dbda7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:33:08.558 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:08.555+0000 7f6f21600640 1 --2- 172.21.15.47:0/84398176 >> [v2:172.21.15.100:6832/2723302038,v1:172.21.15.100:6833/2723302038] conn(0x55a76dbd8380 0x55a76dbda7a0 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=24c83c8498d9f4d4 server_cookie=2d0870c05623caa3 in_seq=0 out_seq=0 2024-03-21T10:33:08.558 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:08.555+0000 7f6f02a00640 10 client.18051 ms_handle_connect on v2:172.21.15.100:6832/2723302038 2024-03-21T10:33:08.558 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:08.555+0000 7f6f02a00640 1 -- 172.21.15.47:0/84398176 <== mds.3 v2:172.21.15.100:6832/2723302038 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55a76dacdb50 con 0x55a76dbd8380 2024-03-21T10:33:08.558 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:08.555+0000 7f6f02a00640 10 client.18051 handle_command_reply: tid=0 2024-03-21T10:33:08.558 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:08.555+0000 7f6f26c01740 1 client.18051 shutdown 2024-03-21T10:33:08.558 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:08.556+0000 7f6f26c01740 1 -- 172.21.15.47:0/84398176 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a76db6bde0 msgr2=0x55a76dbd6420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:33:08.558 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:08.556+0000 7f6f26c01740 1 --2- 172.21.15.47:0/84398176 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a76db6bde0 0x55a76dbd6420 secure :-1 s=READY pgs=1103 cs=0 l=1 rev1=1 crypto rx=0x7f6f14009ca0 tx=0x7f6f1402c180 comp rx=0 tx=0).stop 2024-03-21T10:33:08.558 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:08.556+0000 7f6f26c01740 1 -- 172.21.15.47:0/84398176 shutdown_connections 2024-03-21T10:33:08.558 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:08.556+0000 7f6f26c01740 1 --2- 172.21.15.47:0/84398176 >> [v2:172.21.15.100:6832/2723302038,v1:172.21.15.100:6833/2723302038] conn(0x55a76dbd8380 0x55a76dbda7a0 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:33:08.558 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:08.556+0000 7f6f26c01740 1 --2- 172.21.15.47:0/84398176 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a76db6bde0 0x55a76dbd6420 unknown :-1 s=CLOSED pgs=1103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:33:08.558 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:08.556+0000 7f6f26c01740 1 --2- 172.21.15.47:0/84398176 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a76db63260 0x55a76db72110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:33:08.558 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:08.556+0000 7f6f26c01740 1 --2- 172.21.15.47:0/84398176 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a76db628b0 0x55a76db71bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:33:08.558 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:08.556+0000 7f6f02a00640 0 client.18051 ms_handle_reset on v2:172.21.15.100:6832/2723302038 2024-03-21T10:33:08.559 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:08.556+0000 7f6f26c01740 1 -- 172.21.15.47:0/84398176 >> 172.21.15.47:0/84398176 conn(0x55a76db5cf80 msgr2=0x55a76db5ec00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:33:08.559 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:08.556+0000 7f6f26c01740 1 -- 172.21.15.47:0/84398176 shutdown_connections 2024-03-21T10:33:08.559 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:08.556+0000 7f6f26c01740 1 -- 172.21.15.47:0/84398176 wait complete. 2024-03-21T10:33:08.559 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:08.556+0000 7f6f26c01740 20 client.18051 trim_cache size 0 max 16384 2024-03-21T10:33:08.559 INFO:teuthology.orchestra.run.smithi047.stdout:[] 2024-03-21T10:33:08.560 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:08.557+0000 7f6f22a00640 1 -- 172.21.15.47:0/2006104349 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f6f040a3090 msgr2=0x7f6f040a5550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:33:08.560 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:08.557+0000 7f6f22a00640 1 --2- 172.21.15.47:0/2006104349 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f6f040a3090 0x7f6f040a5550 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f6f14005cc0 tx=0x7f6f14005c50 comp rx=0 tx=0).stop 2024-03-21T10:33:08.560 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:08.557+0000 7f6f22a00640 1 -- 172.21.15.47:0/2006104349 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6f1c1026d0 msgr2=0x7f6f1c199ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:33:08.560 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:08.557+0000 7f6f22a00640 1 --2- 172.21.15.47:0/2006104349 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6f1c1026d0 0x7f6f1c199ea0 secure :-1 s=READY pgs=1099 cs=0 l=1 rev1=1 crypto rx=0x7f6f1000c970 tx=0x7f6f1000ce30 comp rx=0 tx=0).stop 2024-03-21T10:33:08.560 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:08.557+0000 7f6f22a00640 1 -- 172.21.15.47:0/2006104349 shutdown_connections 2024-03-21T10:33:08.560 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:08.557+0000 7f6f22a00640 1 --2- 172.21.15.47:0/2006104349 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f6f040a3090 0x7f6f040a5550 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:33:08.560 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:08.557+0000 7f6f22a00640 1 --2- 172.21.15.47:0/2006104349 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6f1c10baf0 0x7f6f1c19e760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:33:08.560 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:08.557+0000 7f6f22a00640 1 --2- 172.21.15.47:0/2006104349 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6f1c102ff0 0x7f6f1c19a3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:33:08.560 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:08.557+0000 7f6f22a00640 1 --2- 172.21.15.47:0/2006104349 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6f1c1026d0 0x7f6f1c199ea0 unknown :-1 s=CLOSED pgs=1099 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:33:08.560 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:08.557+0000 7f6f22a00640 1 -- 172.21.15.47:0/2006104349 >> 172.21.15.47:0/2006104349 conn(0x7f6f1c0fbef0 msgr2=0x7f6f1c10cd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:33:08.560 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:08.557+0000 7f6f22a00640 1 -- 172.21.15.47:0/2006104349 shutdown_connections 2024-03-21T10:33:08.561 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:08.557+0000 7f6f22a00640 1 -- 172.21.15.47:0/2006104349 wait complete. 2024-03-21T10:33:08.575 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:33:08 smithi047 ceph-mon[31547]: from='client.? 172.21.15.47:0/3464973505' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-21T10:33:08.580 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-21T10:33:08.580 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-03-21T10:33:08.727 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:33:08 smithi100 ceph-mon[35193]: from='client.? 172.21.15.47:0/3464973505' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-21T10:33:08.737 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:33:08 smithi157 ceph-mon[35581]: from='client.? 172.21.15.47:0/3464973505' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-21T10:33:08.805 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:08.805+0000 7f1f30a00640 1 -- 172.21.15.47:0/94828667 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1f2c102ff0 msgr2=0x7f1f2c10b4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:33:08.805 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:08.805+0000 7f1f30a00640 1 --2- 172.21.15.47:0/94828667 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1f2c102ff0 0x7f1f2c10b4d0 secure :-1 s=READY pgs=1104 cs=0 l=1 rev1=1 crypto rx=0x7f1f18009900 tx=0x7f1f1802bcd0 comp rx=0 tx=0).stop 2024-03-21T10:33:08.805 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:08.805+0000 7f1f30a00640 1 -- 172.21.15.47:0/94828667 shutdown_connections 2024-03-21T10:33:08.805 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:08.805+0000 7f1f30a00640 1 --2- 172.21.15.47:0/94828667 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1f2c10baf0 0x7f1f2c10df90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:33:08.805 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:08.805+0000 7f1f30a00640 1 --2- 172.21.15.47:0/94828667 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1f2c102ff0 0x7f1f2c10b4d0 unknown :-1 s=CLOSED pgs=1104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:33:08.805 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:08.805+0000 7f1f30a00640 1 --2- 172.21.15.47:0/94828667 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1f2c1026d0 0x7f1f2c102ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:33:08.805 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:08.805+0000 7f1f30a00640 1 -- 172.21.15.47:0/94828667 >> 172.21.15.47:0/94828667 conn(0x7f1f2c0fbef0 msgr2=0x7f1f2c0fe310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:33:08.805 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:08.805+0000 7f1f30a00640 1 -- 172.21.15.47:0/94828667 shutdown_connections 2024-03-21T10:33:08.805 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:08.805+0000 7f1f30a00640 1 -- 172.21.15.47:0/94828667 wait complete. 2024-03-21T10:33:08.808 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:08.806+0000 7f1f30a00640 1 Processor -- start 2024-03-21T10:33:08.808 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:08.806+0000 7f1f30a00640 1 -- start start 2024-03-21T10:33:08.808 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:08.806+0000 7f1f30a00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1f2c1026d0 0x7f1f2c199f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:33:08.808 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:08.807+0000 7f1f30a00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1f2c102ff0 0x7f1f2c19a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:33:08.809 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:08.807+0000 7f1f30a00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1f2c10baf0 0x7f1f2c19e800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:33:08.809 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:08.807+0000 7f1f30a00640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f2c111780 con 0x7f1f2c1026d0 2024-03-21T10:33:08.809 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:08.807+0000 7f1f30a00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f2c111600 con 0x7f1f2c102ff0 2024-03-21T10:33:08.809 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:08.807+0000 7f1f30a00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f2c111900 con 0x7f1f2c10baf0 2024-03-21T10:33:08.809 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:08.807+0000 7f1f2b400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1f2c1026d0 0x7f1f2c199f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:33:08.809 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:08.807+0000 7f1f2b400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1f2c1026d0 0x7f1f2c199f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:46888/0 (socket says 172.21.15.47:46888) 2024-03-21T10:33:08.809 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:08.807+0000 7f1f2b400640 1 -- 172.21.15.47:0/1463322451 learned_addr learned my addr 172.21.15.47:0/1463322451 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:33:08.809 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:08.807+0000 7f1f2aa00640 1 --2- 172.21.15.47:0/1463322451 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1f2c102ff0 0x7f1f2c19a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:33:08.809 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:08.807+0000 7f1f2be00640 1 --2- 172.21.15.47:0/1463322451 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1f2c10baf0 0x7f1f2c19e800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:33:08.809 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:08.807+0000 7f1f2b400640 1 -- 172.21.15.47:0/1463322451 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1f2c10baf0 msgr2=0x7f1f2c19e800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:33:08.809 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:08.807+0000 7f1f2b400640 1 --2- 172.21.15.47:0/1463322451 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1f2c10baf0 0x7f1f2c19e800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:33:08.810 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:08.807+0000 7f1f2b400640 1 -- 172.21.15.47:0/1463322451 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1f2c102ff0 msgr2=0x7f1f2c19a480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:33:08.810 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:08.807+0000 7f1f2b400640 1 --2- 172.21.15.47:0/1463322451 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1f2c102ff0 0x7f1f2c19a480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:33:08.810 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:08.807+0000 7f1f2b400640 1 -- 172.21.15.47:0/1463322451 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f2c19ef30 con 0x7f1f2c1026d0 2024-03-21T10:33:08.810 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:08.808+0000 7f1f2b400640 1 --2- 172.21.15.47:0/1463322451 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1f2c1026d0 0x7f1f2c199f40 secure :-1 s=READY pgs=1105 cs=0 l=1 rev1=1 crypto rx=0x7f1f1c00eb90 tx=0x7f1f1c00c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:33:08.810 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:08.808+0000 7f1f23e00640 1 -- 172.21.15.47:0/1463322451 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f1c004d20 con 0x7f1f2c1026d0 2024-03-21T10:33:08.810 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:08.808+0000 7f1f23e00640 1 -- 172.21.15.47:0/1463322451 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f1f1c00ee90 con 0x7f1f2c1026d0 2024-03-21T10:33:08.810 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:08.808+0000 7f1f30a00640 1 -- 172.21.15.47:0/1463322451 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1f2c19f220 con 0x7f1f2c1026d0 2024-03-21T10:33:08.810 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:08.808+0000 7f1f23e00640 1 -- 172.21.15.47:0/1463322451 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f1c010670 con 0x7f1f2c1026d0 2024-03-21T10:33:08.811 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:08.808+0000 7f1f30a00640 1 -- 172.21.15.47:0/1463322451 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1f2c1a6ab0 con 0x7f1f2c1026d0 2024-03-21T10:33:08.811 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:08.812+0000 7f1f34ddc740 1 Processor -- start 2024-03-21T10:33:08.811 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:08.812+0000 7f1f34ddc740 1 -- start start 2024-03-21T10:33:08.813 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:08.812+0000 7f1f34ddc740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b89bc238b0 0x55b89bc23c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:33:08.813 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:08.812+0000 7f1f34ddc740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b89bc24260 0x55b89bc2c810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:33:08.814 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:08.812+0000 7f1f34ddc740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b89bc2cde0 0x55b89bc2f230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:33:08.814 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:08.812+0000 7f1f34ddc740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b89bb872c0 con 0x55b89bc24260 2024-03-21T10:33:08.814 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:08.812+0000 7f1f34ddc740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b89bb2fc90 con 0x55b89bc238b0 2024-03-21T10:33:08.814 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:08.812+0000 7f1f34ddc740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b89bb83360 con 0x55b89bc2cde0 2024-03-21T10:33:08.814 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:08.813+0000 7f1f23e00640 1 -- 172.21.15.47:0/1463322451 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 27) v1 ==== 137985+0+0 (secure 0 0 0) 0x7f1f1c005020 con 0x7f1f2c1026d0 2024-03-21T10:33:08.814 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:08.813+0000 7f1f2be00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b89bc238b0 0x55b89bc23c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:33:08.814 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:08.813+0000 7f1f2be00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b89bc238b0 0x55b89bc23c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:55358/0 (socket says 172.21.15.47:55358) 2024-03-21T10:33:08.814 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:08.813+0000 7f1f2be00640 1 -- 172.21.15.47:0/986104810 learned_addr learned my addr 172.21.15.47:0/986104810 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:33:08.814 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:08.813+0000 7f1f2b400640 1 --2- 172.21.15.47:0/986104810 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b89bc24260 0x55b89bc2c810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:33:08.814 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:08.813+0000 7f1f2aa00640 1 --2- 172.21.15.47:0/986104810 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b89bc2cde0 0x55b89bc2f230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:33:08.815 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:08.813+0000 7f1f2be00640 1 -- 172.21.15.47:0/986104810 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b89bc2cde0 msgr2=0x55b89bc2f230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:33:08.815 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:08.813+0000 7f1f2be00640 1 --2- 172.21.15.47:0/986104810 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b89bc2cde0 0x55b89bc2f230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:33:08.815 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:08.813+0000 7f1f2be00640 1 -- 172.21.15.47:0/986104810 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b89bc24260 msgr2=0x55b89bc2c810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:33:08.815 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:08.813+0000 7f1f2be00640 1 --2- 172.21.15.47:0/986104810 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b89bc24260 0x55b89bc2c810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:33:08.815 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:08.813+0000 7f1f2be00640 1 -- 172.21.15.47:0/986104810 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b89bb54080 con 0x55b89bc238b0 2024-03-21T10:33:08.815 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:08.813+0000 7f1f2b400640 1 --2- 172.21.15.47:0/986104810 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b89bc24260 0x55b89bc2c810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T10:33:08.815 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:08.813+0000 7f1f2aa00640 1 --2- 172.21.15.47:0/986104810 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b89bc2cde0 0x55b89bc2f230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T10:33:08.815 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:08.814+0000 7f1f2be00640 1 --2- 172.21.15.47:0/986104810 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b89bc238b0 0x55b89bc23c90 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f1f24009c20 tx=0x7f1f24011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7582e15af1287616 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:33:08.816 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:08.814+0000 7f1f17e00640 1 -- 172.21.15.47:0/986104810 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f2401e070 con 0x55b89bc238b0 2024-03-21T10:33:08.816 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:08.814+0000 7f1f17e00640 1 -- 172.21.15.47:0/986104810 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f1f24019ae0 con 0x55b89bc238b0 2024-03-21T10:33:08.816 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:08.814+0000 7f1f17e00640 1 -- 172.21.15.47:0/986104810 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f2400e8b0 con 0x55b89bc238b0 2024-03-21T10:33:08.816 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:08.814+0000 7f1f23e00640 1 --2- 172.21.15.47:0/1463322451 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f1f100a3090 0x7f1f100a5550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:33:08.820 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:08.815+0000 7f1f23e00640 1 -- 172.21.15.47:0/1463322451 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 9562+0+0 (secure 0 0 0) 0x7f1f1c0d0250 con 0x7f1f2c1026d0 2024-03-21T10:33:08.820 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:08.815+0000 7f1f2b400640 1 --2- 172.21.15.47:0/1463322451 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f1f100a3090 0x7f1f100a5550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:33:08.820 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:08.815+0000 7f1f34ddc740 1 -- 172.21.15.47:0/986104810 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b89bc238b0 msgr2=0x55b89bc23c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:33:08.820 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:08.815+0000 7f1f34ddc740 1 --2- 172.21.15.47:0/986104810 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b89bc238b0 0x55b89bc23c90 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f1f24009c20 tx=0x7f1f24011810 comp rx=0 tx=0).stop 2024-03-21T10:33:08.820 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:08.815+0000 7f1f34ddc740 1 -- 172.21.15.47:0/986104810 shutdown_connections 2024-03-21T10:33:08.820 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:08.815+0000 7f1f34ddc740 1 --2- 172.21.15.47:0/986104810 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b89bc2cde0 0x55b89bc2f230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:33:08.820 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:08.815+0000 7f1f34ddc740 1 --2- 172.21.15.47:0/986104810 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b89bc24260 0x55b89bc2c810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:33:08.820 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:08.815+0000 7f1f34ddc740 1 --2- 172.21.15.47:0/986104810 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b89bc238b0 0x55b89bc23c90 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:33:08.820 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:08.815+0000 7f1f34ddc740 1 -- 172.21.15.47:0/986104810 >> 172.21.15.47:0/986104810 conn(0x55b89bc1df80 msgr2=0x55b89bc1e360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:33:08.820 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:08.815+0000 7f1f34ddc740 1 -- 172.21.15.47:0/986104810 shutdown_connections 2024-03-21T10:33:08.821 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:08.815+0000 7f1f2b400640 1 --2- 172.21.15.47:0/1463322451 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f1f100a3090 0x7f1f100a5550 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f1f1c0cd640 tx=0x7f1f1c009200 comp rx=0 tx=0).ready entity=mgr.15687 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:33:08.821 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:08.815+0000 7f1f34ddc740 1 -- 172.21.15.47:0/986104810 wait complete. 2024-03-21T10:33:08.821 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:08.816+0000 7f1f34ddc740 1 -- start start 2024-03-21T10:33:08.821 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:08.816+0000 7f1f34ddc740 1 Processor -- start 2024-03-21T10:33:08.821 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:08.817+0000 7f1f34ddc740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b89bc238b0 0x55b89bc2c710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:33:08.821 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:08.817+0000 7f1f34ddc740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b89bc24260 0x55b89bc29aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:33:08.821 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:08.817+0000 7f1f34ddc740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b89bc2cde0 0x55b89bc2a0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:33:08.821 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:08.817+0000 7f1f34ddc740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b89bc2fd50 con 0x55b89bc24260 2024-03-21T10:33:08.821 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:08.817+0000 7f1f34ddc740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b89bb430d0 con 0x55b89bc238b0 2024-03-21T10:33:08.821 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:08.817+0000 7f1f34ddc740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b89bc2fed0 con 0x55b89bc2cde0 2024-03-21T10:33:08.821 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:08.817+0000 7f1f34ddc740 10 client.0 _ll_register_callbacks cb 0x55b89bb47e80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T10:33:08.822 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:08.817+0000 7f1f2aa00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b89bc24260 0x55b89bc29aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:33:08.822 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:08.817+0000 7f1f2aa00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b89bc24260 0x55b89bc29aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:46904/0 (socket says 172.21.15.47:46904) 2024-03-21T10:33:08.822 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:08.817+0000 7f1f2aa00640 1 -- 172.21.15.47:0/3030756693 learned_addr learned my addr 172.21.15.47:0/3030756693 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:33:08.822 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:08.817+0000 7f1f2be00640 1 --2- 172.21.15.47:0/3030756693 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b89bc238b0 0x55b89bc2c710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:33:08.822 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:08.817+0000 7f1f2be00640 1 --2- 172.21.15.47:0/3030756693 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b89bc2cde0 0x55b89bc2a0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:33:08.822 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:08.817+0000 7f1f2aa00640 1 -- 172.21.15.47:0/3030756693 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b89bc2cde0 msgr2=0x55b89bc2a0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:33:08.822 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:08.818+0000 7f1f2aa00640 1 --2- 172.21.15.47:0/3030756693 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b89bc2cde0 0x55b89bc2a0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:33:08.822 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:08.818+0000 7f1f2aa00640 1 -- 172.21.15.47:0/3030756693 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b89bc238b0 msgr2=0x55b89bc2c710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:33:08.822 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:08.818+0000 7f1f2aa00640 1 --2- 172.21.15.47:0/3030756693 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b89bc238b0 0x55b89bc2c710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:33:08.822 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:08.818+0000 7f1f2aa00640 1 -- 172.21.15.47:0/3030756693 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55b89b89b3f0 con 0x55b89bc24260 2024-03-21T10:33:08.822 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:08.818+0000 7f1f2aa00640 1 --2- 172.21.15.47:0/3030756693 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b89bc24260 0x55b89bc29aa0 secure :-1 s=READY pgs=1106 cs=0 l=1 rev1=1 crypto rx=0x7f1f1802c210 tx=0x7f1f18005270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eea58766b0265e0e server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:33:08.822 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:08.818+0000 7f1f14c00640 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-03-21T10:33:08.823 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:08.818+0000 7f1f14c00640 1 -- 172.21.15.47:0/3030756693 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f18043030 con 0x55b89bc24260 2024-03-21T10:33:08.823 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:08.818+0000 7f1f34ddc740 1 -- 172.21.15.47:0/3030756693 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f1f24019ae0 con 0x55b89bc24260 2024-03-21T10:33:08.823 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:08.818+0000 7f1f14c00640 1 -- 172.21.15.47:0/3030756693 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(99..99 src has 1..99) v4 ==== 9562+0+0 (secure 0 0 0) 0x7f1f1803a040 con 0x55b89bc24260 2024-03-21T10:33:08.823 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:08.818+0000 7f1f14c00640 1 -- 172.21.15.47:0/3030756693 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1f18050050 con 0x55b89bc24260 2024-03-21T10:33:08.823 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:08.819+0000 7f1f34ddc740 10 client.18063 fetch_fsmap learned FSMap version 32 2024-03-21T10:33:08.823 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:08.819+0000 7f1f34ddc740 1 -- 172.21.15.47:0/3030756693 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x55b89b9657d0 con 0x55b89bc24260 2024-03-21T10:33:08.823 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:08.819+0000 7f1f14c00640 1 -- 172.21.15.47:0/3030756693 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f1f24019ae0 con 0x55b89bc24260 2024-03-21T10:33:08.823 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:08.819+0000 7f1f14c00640 1 -- 172.21.15.47:0/3030756693 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 32) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f1f18004850 con 0x55b89bc24260 2024-03-21T10:33:08.823 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:08.819+0000 7f1f34ddc740 10 client.18063 fetch_fsmap finished waiting for FSMap version 32 2024-03-21T10:33:08.823 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:08.819+0000 7f1f34ddc740 10 client.18063 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.e 2024-03-21T10:33:08.823 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:08.819+0000 7f1f34ddc740 20 client.18063 populate_metadata read hostname 'smithi047' 2024-03-21T10:33:08.824 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:08.819+0000 7f1f34ddc740 1 --2- 172.21.15.47:0/3030756693 >> [v2:172.21.15.100:6834/2597849957,v1:172.21.15.100:6835/2597849957] conn(0x55b89bc3cc60 0x55b89bc3f050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:33:08.824 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:08.819+0000 7f1f34ddc740 4 client.18063 mds_command: new command op to 34454 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T10:33:08.824 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:08.819+0000 7f1f34ddc740 1 -- 172.21.15.47:0/3030756693 --> [v2:172.21.15.100:6834/2597849957,v1:172.21.15.100:6835/2597849957] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55b89bb94800 con 0x55b89bc3cc60 2024-03-21T10:33:08.824 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:08.820+0000 7f1f2be00640 1 --2- 172.21.15.47:0/3030756693 >> [v2:172.21.15.100:6834/2597849957,v1:172.21.15.100:6835/2597849957] conn(0x55b89bc3cc60 0x55b89bc3f050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:33:08.824 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:08.820+0000 7f1f2be00640 1 --2- 172.21.15.47:0/3030756693 >> [v2:172.21.15.100:6834/2597849957,v1:172.21.15.100:6835/2597849957] conn(0x55b89bc3cc60 0x55b89bc3f050 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=4dfd231314203316 server_cookie=43ecb982c55cb37e in_seq=0 out_seq=0 2024-03-21T10:33:08.824 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:08.820+0000 7f1f14c00640 10 client.18063 ms_handle_connect on v2:172.21.15.100:6834/2597849957 2024-03-21T10:33:08.825 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:08.822+0000 7f1f14c00640 1 -- 172.21.15.47:0/3030756693 <== mds.2 v2:172.21.15.100:6834/2597849957 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55b89bb94800 con 0x55b89bc3cc60 2024-03-21T10:33:08.825 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:08.822+0000 7f1f14c00640 10 client.18063 handle_command_reply: tid=0 2024-03-21T10:33:08.826 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:08.822+0000 7f1f34ddc740 1 client.18063 shutdown 2024-03-21T10:33:08.826 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:08.822+0000 7f1f34ddc740 1 -- 172.21.15.47:0/3030756693 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b89bc24260 msgr2=0x55b89bc29aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:33:08.826 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:08.822+0000 7f1f34ddc740 1 --2- 172.21.15.47:0/3030756693 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b89bc24260 0x55b89bc29aa0 secure :-1 s=READY pgs=1106 cs=0 l=1 rev1=1 crypto rx=0x7f1f1802c210 tx=0x7f1f18005270 comp rx=0 tx=0).stop 2024-03-21T10:33:08.826 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:08.822+0000 7f1f34ddc740 1 -- 172.21.15.47:0/3030756693 shutdown_connections 2024-03-21T10:33:08.826 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:08.822+0000 7f1f34ddc740 1 --2- 172.21.15.47:0/3030756693 >> [v2:172.21.15.100:6834/2597849957,v1:172.21.15.100:6835/2597849957] conn(0x55b89bc3cc60 0x55b89bc3f050 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:33:08.826 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:08.822+0000 7f1f34ddc740 1 --2- 172.21.15.47:0/3030756693 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b89bc2cde0 0x55b89bc2a0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:33:08.826 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:08.822+0000 7f1f34ddc740 1 --2- 172.21.15.47:0/3030756693 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b89bc24260 0x55b89bc29aa0 unknown :-1 s=CLOSED pgs=1106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:33:08.826 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:08.822+0000 7f1f34ddc740 1 --2- 172.21.15.47:0/3030756693 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b89bc238b0 0x55b89bc2c710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:33:08.826 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:08.822+0000 7f1f14c00640 0 client.18063 ms_handle_reset on v2:172.21.15.100:6834/2597849957 2024-03-21T10:33:08.826 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:08.822+0000 7f1f34ddc740 1 -- 172.21.15.47:0/3030756693 >> 172.21.15.47:0/3030756693 conn(0x55b89bc1df80 msgr2=0x55b89bc1fcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:33:08.826 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:08.822+0000 7f1f34ddc740 1 -- 172.21.15.47:0/3030756693 shutdown_connections 2024-03-21T10:33:08.826 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:08.822+0000 7f1f34ddc740 1 -- 172.21.15.47:0/3030756693 wait complete. 2024-03-21T10:33:08.827 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:08.823+0000 7f1f34ddc740 20 client.18063 trim_cache size 0 max 16384 2024-03-21T10:33:08.836 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:08.836+0000 7f1f34ddc740 1 Processor -- start 2024-03-21T10:33:08.836 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:08.836+0000 7f1f34ddc740 1 -- start start 2024-03-21T10:33:08.836 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:08.837+0000 7f1f34ddc740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b89bc238b0 0x55b89bc2c1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:33:08.836 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:08.837+0000 7f1f34ddc740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b89bc24260 0x55b89bc2c710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:33:08.837 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:08.837+0000 7f1f34ddc740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b89bc2cde0 0x55b89bc26d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:33:08.837 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:08.837+0000 7f1f34ddc740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f2c1b0ab0 con 0x55b89bc24260 2024-03-21T10:33:08.838 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:08.837+0000 7f1f34ddc740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f2c1b2430 con 0x55b89bc238b0 2024-03-21T10:33:08.838 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:08.837+0000 7f1f34ddc740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f2c1b1110 con 0x55b89bc2cde0 2024-03-21T10:33:08.838 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:08.837+0000 7f1f2aa00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b89bc24260 0x55b89bc2c710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:33:08.838 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:08.837+0000 7f1f2aa00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b89bc24260 0x55b89bc2c710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:46918/0 (socket says 172.21.15.47:46918) 2024-03-21T10:33:08.838 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:08.837+0000 7f1f2aa00640 1 -- 172.21.15.47:0/375677735 learned_addr learned my addr 172.21.15.47:0/375677735 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:33:08.838 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:08.837+0000 7f1f2be00640 1 --2- 172.21.15.47:0/375677735 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b89bc238b0 0x55b89bc2c1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:33:08.838 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:08.837+0000 7f1f2be00640 1 --2- 172.21.15.47:0/375677735 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b89bc2cde0 0x55b89bc26d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:33:08.838 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:08.837+0000 7f1f2aa00640 1 -- 172.21.15.47:0/375677735 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b89bc2cde0 msgr2=0x55b89bc26d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:33:08.838 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:08.837+0000 7f1f2aa00640 1 --2- 172.21.15.47:0/375677735 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b89bc2cde0 0x55b89bc26d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:33:08.838 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:08.837+0000 7f1f2aa00640 1 -- 172.21.15.47:0/375677735 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b89bc238b0 msgr2=0x55b89bc2c1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:33:08.838 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:08.837+0000 7f1f2aa00640 1 --2- 172.21.15.47:0/375677735 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b89bc238b0 0x55b89bc2c1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:33:08.839 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:08.837+0000 7f1f2aa00640 1 -- 172.21.15.47:0/375677735 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b89bc1dab0 con 0x55b89bc24260 2024-03-21T10:33:08.839 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:08.838+0000 7f1f2aa00640 1 --2- 172.21.15.47:0/375677735 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b89bc24260 0x55b89bc2c710 secure :-1 s=READY pgs=1107 cs=0 l=1 rev1=1 crypto rx=0x7f1f18000c00 tx=0x7f1f180052e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a79d66e5ff2a299b server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:33:08.839 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:08.838+0000 7f1f17400640 1 -- 172.21.15.47:0/375677735 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f18043070 con 0x55b89bc24260 2024-03-21T10:33:08.839 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:08.838+0000 7f1f17400640 1 -- 172.21.15.47:0/375677735 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f1f180055f0 con 0x55b89bc24260 2024-03-21T10:33:08.839 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:08.838+0000 7f1f17400640 1 -- 172.21.15.47:0/375677735 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f1802cd70 con 0x55b89bc24260 2024-03-21T10:33:08.844 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:08.838+0000 7f1f34ddc740 1 -- 172.21.15.47:0/375677735 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b89bc24260 msgr2=0x55b89bc2c710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:33:08.844 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:08.838+0000 7f1f34ddc740 1 --2- 172.21.15.47:0/375677735 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b89bc24260 0x55b89bc2c710 secure :-1 s=READY pgs=1107 cs=0 l=1 rev1=1 crypto rx=0x7f1f18000c00 tx=0x7f1f180052e0 comp rx=0 tx=0).stop 2024-03-21T10:33:08.844 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:08.839+0000 7f1f34ddc740 1 -- 172.21.15.47:0/375677735 shutdown_connections 2024-03-21T10:33:08.845 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:08.839+0000 7f1f34ddc740 1 --2- 172.21.15.47:0/375677735 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b89bc2cde0 0x55b89bc26d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:33:08.845 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:08.839+0000 7f1f34ddc740 1 --2- 172.21.15.47:0/375677735 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b89bc24260 0x55b89bc2c710 unknown :-1 s=CLOSED pgs=1107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:33:08.845 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:08.839+0000 7f1f34ddc740 1 --2- 172.21.15.47:0/375677735 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b89bc238b0 0x55b89bc2c1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:33:08.845 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:08.839+0000 7f1f34ddc740 1 -- 172.21.15.47:0/375677735 >> 172.21.15.47:0/375677735 conn(0x55b89bc1df80 msgr2=0x55b89bc1d160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:33:08.845 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:08.839+0000 7f1f34ddc740 1 -- 172.21.15.47:0/375677735 shutdown_connections 2024-03-21T10:33:08.845 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:08.839+0000 7f1f34ddc740 1 -- 172.21.15.47:0/375677735 wait complete. 2024-03-21T10:33:08.845 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:08.839+0000 7f1f34ddc740 1 -- start start 2024-03-21T10:33:08.845 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:08.839+0000 7f1f34ddc740 1 Processor -- start 2024-03-21T10:33:08.845 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:08.840+0000 7f1f34ddc740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b89bc238b0 0x55b89bc34810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:33:08.845 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:08.840+0000 7f1f34ddc740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b89bc24260 0x55b89bc34d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:33:08.845 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:08.840+0000 7f1f34ddc740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b89bc2cde0 0x55b89bc32e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:33:08.845 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:08.840+0000 7f1f34ddc740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f2c1b1dd0 con 0x55b89bc2cde0 2024-03-21T10:33:08.846 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:08.840+0000 7f1f34ddc740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f2c1b1770 con 0x55b89bc24260 2024-03-21T10:33:08.846 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:08.840+0000 7f1f34ddc740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f2c1aec90 con 0x55b89bc238b0 2024-03-21T10:33:08.846 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:08.840+0000 7f1f34ddc740 10 client.0 _ll_register_callbacks cb 0x7f1f24005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T10:33:08.846 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:08.840+0000 7f1f2be00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b89bc2cde0 0x55b89bc32e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:33:08.846 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:08.840+0000 7f1f2be00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b89bc2cde0 0x55b89bc32e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:46920/0 (socket says 172.21.15.47:46920) 2024-03-21T10:33:08.846 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:08.840+0000 7f1f2be00640 1 -- 172.21.15.47:0/372172069 learned_addr learned my addr 172.21.15.47:0/372172069 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:33:08.846 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:08.840+0000 7f1f2aa00640 1 --2- 172.21.15.47:0/372172069 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b89bc24260 0x55b89bc34d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:33:08.846 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:08.840+0000 7f1f2be00640 1 --2- 172.21.15.47:0/372172069 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b89bc238b0 0x55b89bc34810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:33:08.846 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:08.841+0000 7f1f2be00640 1 -- 172.21.15.47:0/372172069 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b89bc238b0 msgr2=0x55b89bc34810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:33:08.846 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:08.841+0000 7f1f2be00640 1 --2- 172.21.15.47:0/372172069 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b89bc238b0 0x55b89bc34810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:33:08.846 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:08.841+0000 7f1f2be00640 1 -- 172.21.15.47:0/372172069 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b89bc24260 msgr2=0x55b89bc34d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:33:08.847 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:08.841+0000 7f1f2be00640 1 --2- 172.21.15.47:0/372172069 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b89bc24260 0x55b89bc34d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:33:08.847 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:08.841+0000 7f1f2be00640 1 -- 172.21.15.47:0/372172069 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55b89bc33540 con 0x55b89bc2cde0 2024-03-21T10:33:08.847 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:08.841+0000 7f1f2be00640 1 --2- 172.21.15.47:0/372172069 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b89bc2cde0 0x55b89bc32e50 secure :-1 s=READY pgs=1108 cs=0 l=1 rev1=1 crypto rx=0x7f1f24009bc0 tx=0x7f1f2402a040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2d34d73e1b6ba397 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:33:08.847 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:08.841+0000 7f1f14c00640 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-03-21T10:33:08.847 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:08.841+0000 7f1f14c00640 1 -- 172.21.15.47:0/372172069 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f2401e040 con 0x55b89bc2cde0 2024-03-21T10:33:08.847 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:08.841+0000 7f1f14c00640 1 -- 172.21.15.47:0/372172069 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(99..99 src has 1..99) v4 ==== 9562+0+0 (secure 0 0 0) 0x7f1f24005230 con 0x55b89bc2cde0 2024-03-21T10:33:08.847 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:08.841+0000 7f1f34ddc740 1 -- 172.21.15.47:0/372172069 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f1f180055f0 con 0x55b89bc2cde0 2024-03-21T10:33:08.847 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:08.841+0000 7f1f14c00640 1 -- 172.21.15.47:0/372172069 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1f24005450 con 0x55b89bc2cde0 2024-03-21T10:33:08.847 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:08.841+0000 7f1f34ddc740 10 client.18069 fetch_fsmap learned FSMap version 32 2024-03-21T10:33:08.847 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:08.841+0000 7f1f34ddc740 1 -- 172.21.15.47:0/372172069 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x55b89bc98fe0 con 0x55b89bc2cde0 2024-03-21T10:33:08.847 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:08.842+0000 7f1f14c00640 1 -- 172.21.15.47:0/372172069 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f1f180055f0 con 0x55b89bc2cde0 2024-03-21T10:33:08.848 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:08.842+0000 7f1f14c00640 1 -- 172.21.15.47:0/372172069 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 32) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f1f24019550 con 0x55b89bc2cde0 2024-03-21T10:33:08.848 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:08.842+0000 7f1f34ddc740 10 client.18069 fetch_fsmap finished waiting for FSMap version 32 2024-03-21T10:33:08.848 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:08.842+0000 7f1f34ddc740 10 client.18069 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.e 2024-03-21T10:33:08.848 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:08.842+0000 7f1f34ddc740 20 client.18069 populate_metadata read hostname 'smithi047' 2024-03-21T10:33:08.848 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:08.842+0000 7f1f34ddc740 1 --2- 172.21.15.47:0/372172069 >> [v2:172.21.15.100:6834/2597849957,v1:172.21.15.100:6835/2597849957] conn(0x55b89bc99490 0x55b89bc9b8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:33:08.848 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:08.842+0000 7f1f34ddc740 4 client.18069 mds_command: new command op to 34454 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-21T10:33:08.848 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:08.842+0000 7f1f34ddc740 1 -- 172.21.15.47:0/372172069 --> [v2:172.21.15.100:6834/2597849957,v1:172.21.15.100:6835/2597849957] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55b89bb8eb50 con 0x55b89bc99490 2024-03-21T10:33:08.848 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:08.842+0000 7f1f2aa00640 1 --2- 172.21.15.47:0/372172069 >> [v2:172.21.15.100:6834/2597849957,v1:172.21.15.100:6835/2597849957] conn(0x55b89bc99490 0x55b89bc9b8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:33:08.848 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:08.843+0000 7f1f2aa00640 1 --2- 172.21.15.47:0/372172069 >> [v2:172.21.15.100:6834/2597849957,v1:172.21.15.100:6835/2597849957] conn(0x55b89bc99490 0x55b89bc9b8b0 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=e57e06e453f86771 server_cookie=f6091575c4a1b38 in_seq=0 out_seq=0 2024-03-21T10:33:08.848 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:08.843+0000 7f1f14c00640 10 client.18069 ms_handle_connect on v2:172.21.15.100:6834/2597849957 2024-03-21T10:33:08.848 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:08.843+0000 7f1f14c00640 1 -- 172.21.15.47:0/372172069 <== mds.2 v2:172.21.15.100:6834/2597849957 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55b89bb8eb50 con 0x55b89bc99490 2024-03-21T10:33:08.849 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:08.843+0000 7f1f14c00640 10 client.18069 handle_command_reply: tid=0 2024-03-21T10:33:08.849 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:08.843+0000 7f1f34ddc740 1 client.18069 shutdown 2024-03-21T10:33:08.849 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:08.844+0000 7f1f34ddc740 1 -- 172.21.15.47:0/372172069 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b89bc2cde0 msgr2=0x55b89bc32e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:33:08.849 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:08.844+0000 7f1f34ddc740 1 --2- 172.21.15.47:0/372172069 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b89bc2cde0 0x55b89bc32e50 secure :-1 s=READY pgs=1108 cs=0 l=1 rev1=1 crypto rx=0x7f1f24009bc0 tx=0x7f1f2402a040 comp rx=0 tx=0).stop 2024-03-21T10:33:08.849 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:08.844+0000 7f1f34ddc740 1 -- 172.21.15.47:0/372172069 shutdown_connections 2024-03-21T10:33:08.849 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:08.844+0000 7f1f34ddc740 1 --2- 172.21.15.47:0/372172069 >> [v2:172.21.15.100:6834/2597849957,v1:172.21.15.100:6835/2597849957] conn(0x55b89bc99490 0x55b89bc9b8b0 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:33:08.849 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:08.844+0000 7f1f34ddc740 1 --2- 172.21.15.47:0/372172069 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b89bc2cde0 0x55b89bc32e50 unknown :-1 s=CLOSED pgs=1108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:33:08.849 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:08.844+0000 7f1f34ddc740 1 --2- 172.21.15.47:0/372172069 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b89bc24260 0x55b89bc34d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:33:08.849 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:08.844+0000 7f1f34ddc740 1 --2- 172.21.15.47:0/372172069 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55b89bc238b0 0x55b89bc34810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:33:08.849 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:08.844+0000 7f1f34ddc740 1 -- 172.21.15.47:0/372172069 >> 172.21.15.47:0/372172069 conn(0x55b89bc1df80 msgr2=0x55b89bc1f280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:33:08.849 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:08.844+0000 7f1f14c00640 0 client.18069 ms_handle_reset on v2:172.21.15.100:6834/2597849957 2024-03-21T10:33:08.849 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:08.844+0000 7f1f34ddc740 1 -- 172.21.15.47:0/372172069 shutdown_connections 2024-03-21T10:33:08.850 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:08.844+0000 7f1f34ddc740 1 -- 172.21.15.47:0/372172069 wait complete. 2024-03-21T10:33:08.850 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:08.844+0000 7f1f34ddc740 20 client.18069 trim_cache size 0 max 16384 2024-03-21T10:33:08.850 INFO:teuthology.orchestra.run.smithi047.stdout:[] 2024-03-21T10:33:08.851 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:08.845+0000 7f1f30a00640 1 -- 172.21.15.47:0/1463322451 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f1f100a3090 msgr2=0x7f1f100a5550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:33:08.851 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:08.845+0000 7f1f30a00640 1 --2- 172.21.15.47:0/1463322451 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f1f100a3090 0x7f1f100a5550 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f1f1c0cd640 tx=0x7f1f1c009200 comp rx=0 tx=0).stop 2024-03-21T10:33:08.851 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:08.845+0000 7f1f30a00640 1 -- 172.21.15.47:0/1463322451 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1f2c1026d0 msgr2=0x7f1f2c199f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:33:08.851 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:08.845+0000 7f1f30a00640 1 --2- 172.21.15.47:0/1463322451 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1f2c1026d0 0x7f1f2c199f40 secure :-1 s=READY pgs=1105 cs=0 l=1 rev1=1 crypto rx=0x7f1f1c00eb90 tx=0x7f1f1c00c6a0 comp rx=0 tx=0).stop 2024-03-21T10:33:08.851 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:08.845+0000 7f1f30a00640 1 -- 172.21.15.47:0/1463322451 shutdown_connections 2024-03-21T10:33:08.851 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:08.845+0000 7f1f30a00640 1 --2- 172.21.15.47:0/1463322451 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1f2c10baf0 0x7f1f2c19e800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:33:08.851 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:08.845+0000 7f1f30a00640 1 --2- 172.21.15.47:0/1463322451 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f1f100a3090 0x7f1f100a5550 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:33:08.851 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:08.845+0000 7f1f30a00640 1 --2- 172.21.15.47:0/1463322451 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1f2c102ff0 0x7f1f2c19a480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:33:08.851 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:08.845+0000 7f1f30a00640 1 --2- 172.21.15.47:0/1463322451 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1f2c1026d0 0x7f1f2c199f40 unknown :-1 s=CLOSED pgs=1105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:33:08.852 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:08.845+0000 7f1f30a00640 1 -- 172.21.15.47:0/1463322451 >> 172.21.15.47:0/1463322451 conn(0x7f1f2c0fbef0 msgr2=0x7f1f2c0fcb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:33:08.852 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:08.845+0000 7f1f30a00640 1 -- 172.21.15.47:0/1463322451 shutdown_connections 2024-03-21T10:33:08.852 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:08.845+0000 7f1f30a00640 1 -- 172.21.15.47:0/1463322451 wait complete. 2024-03-21T10:33:08.862 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-21T10:33:08.862 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-03-21T10:33:09.085 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:09.085+0000 7fe488a00640 1 -- 172.21.15.47:0/4000094191 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe4840764f0 msgr2=0x7fe484111fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:33:09.085 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:09.085+0000 7fe488a00640 1 --2- 172.21.15.47:0/4000094191 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe4840764f0 0x7fe484111fa0 secure :-1 s=READY pgs=1109 cs=0 l=1 rev1=1 crypto rx=0x7fe46c009900 tx=0x7fe46c02bc30 comp rx=0 tx=0).stop 2024-03-21T10:33:09.085 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:09.085+0000 7fe488a00640 1 -- 172.21.15.47:0/4000094191 shutdown_connections 2024-03-21T10:33:09.085 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:09.085+0000 7fe488a00640 1 --2- 172.21.15.47:0/4000094191 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe4841124e0 0x7fe4841148d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:33:09.085 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:09.085+0000 7fe488a00640 1 --2- 172.21.15.47:0/4000094191 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe4840764f0 0x7fe484111fa0 unknown :-1 s=CLOSED pgs=1109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:33:09.085 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:09.085+0000 7fe488a00640 1 --2- 172.21.15.47:0/4000094191 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe484075bd0 0x7fe484075fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:33:09.086 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:09.085+0000 7fe488a00640 1 -- 172.21.15.47:0/4000094191 >> 172.21.15.47:0/4000094191 conn(0x7fe4840ff310 msgr2=0x7fe484101730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:33:09.086 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:09.085+0000 7fe488a00640 1 -- 172.21.15.47:0/4000094191 shutdown_connections 2024-03-21T10:33:09.086 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:09.085+0000 7fe488a00640 1 -- 172.21.15.47:0/4000094191 wait complete. 2024-03-21T10:33:09.087 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:09.086+0000 7fe488a00640 1 Processor -- start 2024-03-21T10:33:09.087 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:09.087+0000 7fe488a00640 1 -- start start 2024-03-21T10:33:09.087 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:09.087+0000 7fe488a00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe484075bd0 0x7fe48419e4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:33:09.087 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:09.087+0000 7fe488a00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe4840764f0 0x7fe48419ea20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:33:09.087 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:09.087+0000 7fe488a00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe4841124e0 0x7fe4841a2da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:33:09.088 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:09.087+0000 7fe488a00640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe484117f40 con 0x7fe4840764f0 2024-03-21T10:33:09.088 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:09.087+0000 7fe488a00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe484117dc0 con 0x7fe4841124e0 2024-03-21T10:33:09.088 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:09.087+0000 7fe488a00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4841180c0 con 0x7fe484075bd0 2024-03-21T10:33:09.088 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:09.087+0000 7fe47be00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe4840764f0 0x7fe48419ea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:33:09.088 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:09.087+0000 7fe47be00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe4840764f0 0x7fe48419ea20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:46928/0 (socket says 172.21.15.47:46928) 2024-03-21T10:33:09.088 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:09.087+0000 7fe47be00640 1 -- 172.21.15.47:0/1576725051 learned_addr learned my addr 172.21.15.47:0/1576725051 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:33:09.088 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:09.087+0000 7fe481e00640 1 --2- 172.21.15.47:0/1576725051 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe484075bd0 0x7fe48419e4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:33:09.088 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:09.088+0000 7fe482800640 1 --2- 172.21.15.47:0/1576725051 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe4841124e0 0x7fe4841a2da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:33:09.088 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:09.088+0000 7fe47be00640 1 -- 172.21.15.47:0/1576725051 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe484075bd0 msgr2=0x7fe48419e4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:33:09.088 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:09.088+0000 7fe47be00640 1 --2- 172.21.15.47:0/1576725051 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe484075bd0 0x7fe48419e4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:33:09.088 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:09.088+0000 7fe47be00640 1 -- 172.21.15.47:0/1576725051 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe4841124e0 msgr2=0x7fe4841a2da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:33:09.088 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:09.088+0000 7fe47be00640 1 --2- 172.21.15.47:0/1576725051 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe4841124e0 0x7fe4841a2da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:33:09.088 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:09.088+0000 7fe47be00640 1 -- 172.21.15.47:0/1576725051 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe4841a34d0 con 0x7fe4840764f0 2024-03-21T10:33:09.088 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:09.088+0000 7fe482800640 1 --2- 172.21.15.47:0/1576725051 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe4841124e0 0x7fe4841a2da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T10:33:09.089 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:09.088+0000 7fe47be00640 1 --2- 172.21.15.47:0/1576725051 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe4840764f0 0x7fe48419ea20 secure :-1 s=READY pgs=1110 cs=0 l=1 rev1=1 crypto rx=0x7fe46c000c00 tx=0x7fe46c0045e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:33:09.089 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:09.088+0000 7fe47aa00640 1 -- 172.21.15.47:0/1576725051 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe46c043020 con 0x7fe4840764f0 2024-03-21T10:33:09.089 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:09.088+0000 7fe47aa00640 1 -- 172.21.15.47:0/1576725051 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fe46c02cbd0 con 0x7fe4840764f0 2024-03-21T10:33:09.089 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:09.088+0000 7fe47aa00640 1 -- 172.21.15.47:0/1576725051 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe46c003bf0 con 0x7fe4840764f0 2024-03-21T10:33:09.089 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:09.088+0000 7fe488a00640 1 -- 172.21.15.47:0/1576725051 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe4841a3760 con 0x7fe4840764f0 2024-03-21T10:33:09.090 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:09.089+0000 7fe488a00640 1 -- 172.21.15.47:0/1576725051 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe4841ab050 con 0x7fe4840764f0 2024-03-21T10:33:09.091 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:09.092+0000 7fe48b7da740 1 Processor -- start 2024-03-21T10:33:09.091 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:09.092+0000 7fe47aa00640 1 -- 172.21.15.47:0/1576725051 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 27) v1 ==== 137985+0+0 (secure 0 0 0) 0x7fe46c046020 con 0x7fe4840764f0 2024-03-21T10:33:09.091 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:09.092+0000 7fe48b7da740 1 -- start start 2024-03-21T10:33:09.095 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:09.092+0000 7fe48b7da740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55806fdae6d0 0x55806fdaeab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:33:09.095 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:09.092+0000 7fe48b7da740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55806fdaf080 0x55806fdb7630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:33:09.096 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:09.092+0000 7fe48b7da740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55806fdb7c00 0x55806fdba050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:33:09.096 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:09.092+0000 7fe48b7da740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x55806fd122c0 con 0x55806fdb7c00 2024-03-21T10:33:09.096 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:09.092+0000 7fe48b7da740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55806fcbac90 con 0x55806fdae6d0 2024-03-21T10:33:09.096 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:09.092+0000 7fe48b7da740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55806fd0e360 con 0x55806fdaf080 2024-03-21T10:33:09.096 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:09.093+0000 7fe47be00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55806fdb7c00 0x55806fdba050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:33:09.096 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:09.093+0000 7fe47be00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55806fdb7c00 0x55806fdba050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:46934/0 (socket says 172.21.15.47:46934) 2024-03-21T10:33:09.096 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:09.093+0000 7fe47be00640 1 -- 172.21.15.47:0/147511962 learned_addr learned my addr 172.21.15.47:0/147511962 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:33:09.096 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:09.093+0000 7fe482800640 1 --2- 172.21.15.47:0/147511962 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55806fdae6d0 0x55806fdaeab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:33:09.096 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:09.093+0000 7fe481e00640 1 --2- 172.21.15.47:0/147511962 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55806fdaf080 0x55806fdb7630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:33:09.096 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:09.093+0000 7fe47be00640 1 -- 172.21.15.47:0/147511962 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55806fdaf080 msgr2=0x55806fdb7630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:33:09.096 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:09.093+0000 7fe47be00640 1 --2- 172.21.15.47:0/147511962 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55806fdaf080 0x55806fdb7630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:33:09.096 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:09.093+0000 7fe47be00640 1 -- 172.21.15.47:0/147511962 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55806fdae6d0 msgr2=0x55806fdaeab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:33:09.096 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:09.093+0000 7fe47be00640 1 --2- 172.21.15.47:0/147511962 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55806fdae6d0 0x55806fdaeab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:33:09.097 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:09.093+0000 7fe47be00640 1 -- 172.21.15.47:0/147511962 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55806fcdf080 con 0x55806fdb7c00 2024-03-21T10:33:09.097 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:09.094+0000 7fe47be00640 1 --2- 172.21.15.47:0/147511962 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55806fdb7c00 0x55806fdba050 secure :-1 s=READY pgs=1111 cs=0 l=1 rev1=1 crypto rx=0x7fe46c0ef1d0 tx=0x7fe46c0f5eb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4893fb83347c0de9 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:33:09.097 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:09.094+0000 7fe47aa00640 1 --2- 172.21.15.47:0/1576725051 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7fe4680a3110 0x7fe4680a55d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:33:09.097 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:09.094+0000 7fe473000640 1 -- 172.21.15.47:0/147511962 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe46c0ba1d0 con 0x55806fdb7c00 2024-03-21T10:33:09.097 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:09.094+0000 7fe47aa00640 1 -- 172.21.15.47:0/1576725051 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 9562+0+0 (secure 0 0 0) 0x7fe46c0f1ea0 con 0x7fe4840764f0 2024-03-21T10:33:09.097 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:09.094+0000 7fe482800640 1 --2- 172.21.15.47:0/1576725051 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7fe4680a3110 0x7fe4680a55d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:33:09.097 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:09.094+0000 7fe473000640 1 -- 172.21.15.47:0/147511962 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fe46c0ff070 con 0x55806fdb7c00 2024-03-21T10:33:09.097 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:09.094+0000 7fe473000640 1 -- 172.21.15.47:0/147511962 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe46c0bd380 con 0x55806fdb7c00 2024-03-21T10:33:09.097 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:09.095+0000 7fe48b7da740 1 -- 172.21.15.47:0/147511962 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55806fdb7c00 msgr2=0x55806fdba050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:33:09.097 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:09.095+0000 7fe482800640 1 --2- 172.21.15.47:0/1576725051 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7fe4680a3110 0x7fe4680a55d0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fe48406ab50 tx=0x7fe47c009660 comp rx=0 tx=0).ready entity=mgr.15687 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:33:09.097 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:09.095+0000 7fe48b7da740 1 --2- 172.21.15.47:0/147511962 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55806fdb7c00 0x55806fdba050 secure :-1 s=READY pgs=1111 cs=0 l=1 rev1=1 crypto rx=0x7fe46c0ef1d0 tx=0x7fe46c0f5eb0 comp rx=0 tx=0).stop 2024-03-21T10:33:09.097 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:09.096+0000 7fe48b7da740 1 -- 172.21.15.47:0/147511962 shutdown_connections 2024-03-21T10:33:09.097 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:09.096+0000 7fe48b7da740 1 --2- 172.21.15.47:0/147511962 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55806fdb7c00 0x55806fdba050 unknown :-1 s=CLOSED pgs=1111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:33:09.097 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:09.096+0000 7fe48b7da740 1 --2- 172.21.15.47:0/147511962 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55806fdaf080 0x55806fdb7630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:33:09.098 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:09.096+0000 7fe48b7da740 1 --2- 172.21.15.47:0/147511962 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55806fdae6d0 0x55806fdaeab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:33:09.098 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:09.096+0000 7fe48b7da740 1 -- 172.21.15.47:0/147511962 >> 172.21.15.47:0/147511962 conn(0x55806fda8da0 msgr2=0x55806fda9180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:33:09.098 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:09.096+0000 7fe48b7da740 1 -- 172.21.15.47:0/147511962 shutdown_connections 2024-03-21T10:33:09.098 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:09.096+0000 7fe48b7da740 1 -- 172.21.15.47:0/147511962 wait complete. 2024-03-21T10:33:09.098 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:09.096+0000 7fe48b7da740 1 -- start start 2024-03-21T10:33:09.098 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:09.096+0000 7fe48b7da740 1 Processor -- start 2024-03-21T10:33:09.102 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:09.097+0000 7fe48b7da740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55806fdae6d0 0x55806fdb7510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:33:09.102 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:09.097+0000 7fe48b7da740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55806fdaf080 0x55806fdb4910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:33:09.102 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:09.097+0000 7fe48b7da740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55806fdb7c00 0x55806fdb4f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:33:09.102 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:09.097+0000 7fe48b7da740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x55806fdba930 con 0x55806fdae6d0 2024-03-21T10:33:09.102 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:09.097+0000 7fe48b7da740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55806fcce0d0 con 0x55806fdb7c00 2024-03-21T10:33:09.102 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:09.097+0000 7fe48b7da740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55806fdbaab0 con 0x55806fdaf080 2024-03-21T10:33:09.102 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:09.097+0000 7fe48b7da740 10 client.0 _ll_register_callbacks cb 0x55806fcd2e80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T10:33:09.102 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:09.097+0000 7fe481e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55806fdae6d0 0x55806fdb7510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:33:09.102 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:09.097+0000 7fe481e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55806fdae6d0 0x55806fdb7510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:46938/0 (socket says 172.21.15.47:46938) 2024-03-21T10:33:09.102 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:09.097+0000 7fe481e00640 1 -- 172.21.15.47:0/3858179631 learned_addr learned my addr 172.21.15.47:0/3858179631 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:33:09.102 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:09.097+0000 7fe47be00640 1 --2- 172.21.15.47:0/3858179631 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55806fdaf080 0x55806fdb4910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:33:09.103 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:09.097+0000 7fe482800640 1 --2- 172.21.15.47:0/3858179631 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55806fdb7c00 0x55806fdb4f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:33:09.103 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:09.098+0000 7fe481e00640 1 -- 172.21.15.47:0/3858179631 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55806fdaf080 msgr2=0x55806fdb4910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:33:09.103 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:09.098+0000 7fe481e00640 1 --2- 172.21.15.47:0/3858179631 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55806fdaf080 0x55806fdb4910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:33:09.103 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:09.098+0000 7fe481e00640 1 -- 172.21.15.47:0/3858179631 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55806fdb7c00 msgr2=0x55806fdb4f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:33:09.103 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:09.098+0000 7fe481e00640 1 --2- 172.21.15.47:0/3858179631 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55806fdb7c00 0x55806fdb4f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:33:09.103 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:09.098+0000 7fe481e00640 1 -- 172.21.15.47:0/3858179631 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55806fa263f0 con 0x55806fdae6d0 2024-03-21T10:33:09.103 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:09.098+0000 7fe482800640 1 --2- 172.21.15.47:0/3858179631 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55806fdb7c00 0x55806fdb4f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T10:33:09.103 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:09.098+0000 7fe481e00640 1 --2- 172.21.15.47:0/3858179631 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55806fdae6d0 0x55806fdb7510 secure :-1 s=READY pgs=1112 cs=0 l=1 rev1=1 crypto rx=0x7fe4740134b0 tx=0x7fe474013970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=abb77d0c880a4871 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:33:09.103 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:09.098+0000 7fe467400640 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-03-21T10:33:09.103 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:09.098+0000 7fe467400640 1 -- 172.21.15.47:0/3858179631 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe474020060 con 0x55806fdae6d0 2024-03-21T10:33:09.103 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:09.098+0000 7fe467400640 1 -- 172.21.15.47:0/3858179631 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(99..99 src has 1..99) v4 ==== 9562+0+0 (secure 0 0 0) 0x7fe474004460 con 0x55806fdae6d0 2024-03-21T10:33:09.103 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:09.098+0000 7fe48b7da740 1 -- 172.21.15.47:0/3858179631 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fe46c0ff070 con 0x55806fdae6d0 2024-03-21T10:33:09.103 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:09.099+0000 7fe467400640 1 -- 172.21.15.47:0/3858179631 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe474027050 con 0x55806fdae6d0 2024-03-21T10:33:09.103 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:09.099+0000 7fe48b7da740 10 client.18081 fetch_fsmap learned FSMap version 32 2024-03-21T10:33:09.103 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:09.099+0000 7fe48b7da740 1 -- 172.21.15.47:0/3858179631 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x55806faf07d0 con 0x55806fdae6d0 2024-03-21T10:33:09.104 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:09.099+0000 7fe467400640 1 -- 172.21.15.47:0/3858179631 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fe46c0ff070 con 0x55806fdae6d0 2024-03-21T10:33:09.104 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:09.099+0000 7fe467400640 1 -- 172.21.15.47:0/3858179631 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 32) v1 ==== 7942+0+0 (secure 0 0 0) 0x7fe47401a030 con 0x55806fdae6d0 2024-03-21T10:33:09.104 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:09.099+0000 7fe48b7da740 10 client.18081 fetch_fsmap finished waiting for FSMap version 32 2024-03-21T10:33:09.104 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:09.099+0000 7fe48b7da740 10 client.18081 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-03-21T10:33:09.104 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:09.099+0000 7fe48b7da740 20 client.18081 populate_metadata read hostname 'smithi047' 2024-03-21T10:33:09.104 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:09.099+0000 7fe48b7da740 1 --2- 172.21.15.47:0/3858179631 >> [v2:172.21.15.100:6836/3497644162,v1:172.21.15.100:6837/3497644162] conn(0x55806fdc7870 0x55806fdc9c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:33:09.104 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:09.099+0000 7fe48b7da740 4 client.18081 mds_command: new command op to 34493 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T10:33:09.104 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:09.099+0000 7fe48b7da740 1 -- 172.21.15.47:0/3858179631 --> [v2:172.21.15.100:6836/3497644162,v1:172.21.15.100:6837/3497644162] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55806fd1f800 con 0x55806fdc7870 2024-03-21T10:33:09.104 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:09.100+0000 7fe47be00640 1 --2- 172.21.15.47:0/3858179631 >> [v2:172.21.15.100:6836/3497644162,v1:172.21.15.100:6837/3497644162] conn(0x55806fdc7870 0x55806fdc9c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:33:09.104 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:09.100+0000 7fe47be00640 1 --2- 172.21.15.47:0/3858179631 >> [v2:172.21.15.100:6836/3497644162,v1:172.21.15.100:6837/3497644162] conn(0x55806fdc7870 0x55806fdc9c80 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=c3119ad665559c70 server_cookie=a03d6e4d4135fad0 in_seq=0 out_seq=0 2024-03-21T10:33:09.104 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:09.100+0000 7fe467400640 10 client.18081 ms_handle_connect on v2:172.21.15.100:6836/3497644162 2024-03-21T10:33:09.104 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:09.101+0000 7fe467400640 1 -- 172.21.15.47:0/3858179631 <== mds.1 v2:172.21.15.100:6836/3497644162 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55806fd1f800 con 0x55806fdc7870 2024-03-21T10:33:09.104 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:09.101+0000 7fe467400640 10 client.18081 handle_command_reply: tid=0 2024-03-21T10:33:09.104 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:09.101+0000 7fe48b7da740 1 client.18081 shutdown 2024-03-21T10:33:09.104 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:09.101+0000 7fe48b7da740 1 -- 172.21.15.47:0/3858179631 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55806fdae6d0 msgr2=0x55806fdb7510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:33:09.104 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:09.101+0000 7fe48b7da740 1 --2- 172.21.15.47:0/3858179631 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55806fdae6d0 0x55806fdb7510 secure :-1 s=READY pgs=1112 cs=0 l=1 rev1=1 crypto rx=0x7fe4740134b0 tx=0x7fe474013970 comp rx=0 tx=0).stop 2024-03-21T10:33:09.105 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:09.101+0000 7fe48b7da740 1 -- 172.21.15.47:0/3858179631 shutdown_connections 2024-03-21T10:33:09.105 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:09.101+0000 7fe48b7da740 1 --2- 172.21.15.47:0/3858179631 >> [v2:172.21.15.100:6836/3497644162,v1:172.21.15.100:6837/3497644162] conn(0x55806fdc7870 0x55806fdc9c80 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:33:09.105 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:09.101+0000 7fe48b7da740 1 --2- 172.21.15.47:0/3858179631 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55806fdb7c00 0x55806fdb4f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:33:09.105 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:09.101+0000 7fe48b7da740 1 --2- 172.21.15.47:0/3858179631 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55806fdaf080 0x55806fdb4910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:33:09.105 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:09.101+0000 7fe48b7da740 1 --2- 172.21.15.47:0/3858179631 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55806fdae6d0 0x55806fdb7510 unknown :-1 s=CLOSED pgs=1112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:33:09.105 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:09.101+0000 7fe48b7da740 1 -- 172.21.15.47:0/3858179631 >> 172.21.15.47:0/3858179631 conn(0x55806fda8da0 msgr2=0x55806fdaaaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:33:09.105 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:09.101+0000 7fe467400640 0 client.18081 ms_handle_reset on v2:172.21.15.100:6836/3497644162 2024-03-21T10:33:09.105 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:09.102+0000 7fe48b7da740 1 -- 172.21.15.47:0/3858179631 shutdown_connections 2024-03-21T10:33:09.105 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:09.102+0000 7fe48b7da740 1 -- 172.21.15.47:0/3858179631 wait complete. 2024-03-21T10:33:09.105 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:09.102+0000 7fe48b7da740 20 client.18081 trim_cache size 0 max 16384 2024-03-21T10:33:09.115 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:09.115+0000 7fe48b7da740 1 Processor -- start 2024-03-21T10:33:09.115 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:09.116+0000 7fe48b7da740 1 -- start start 2024-03-21T10:33:09.115 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:09.116+0000 7fe48b7da740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55806fdae6d0 0x55806fdb7040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:33:09.115 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:09.116+0000 7fe48b7da740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55806fdaf080 0x55806fdb1b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:33:09.115 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:09.116+0000 7fe48b7da740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55806fdb7c00 0x55806fdb20b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:33:09.115 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:09.116+0000 7fe48b7da740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4841b50a0 con 0x55806fdaf080 2024-03-21T10:33:09.116 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:09.116+0000 7fe48b7da740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4841b6a20 con 0x55806fdb7c00 2024-03-21T10:33:09.116 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:09.116+0000 7fe48b7da740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4841b5700 con 0x55806fdae6d0 2024-03-21T10:33:09.117 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:09.116+0000 7fe47be00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55806fdaf080 0x55806fdb1b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:33:09.117 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:09.116+0000 7fe47be00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55806fdaf080 0x55806fdb1b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:46946/0 (socket says 172.21.15.47:46946) 2024-03-21T10:33:09.117 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:09.116+0000 7fe47be00640 1 -- 172.21.15.47:0/1983193942 learned_addr learned my addr 172.21.15.47:0/1983193942 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:33:09.118 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:09.116+0000 7fe482800640 1 --2- 172.21.15.47:0/1983193942 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55806fdb7c00 0x55806fdb20b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:33:09.118 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:09.117+0000 7fe481e00640 1 --2- 172.21.15.47:0/1983193942 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55806fdae6d0 0x55806fdb7040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:33:09.118 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:09.117+0000 7fe47be00640 1 -- 172.21.15.47:0/1983193942 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55806fdae6d0 msgr2=0x55806fdb7040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:33:09.118 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:09.117+0000 7fe47be00640 1 --2- 172.21.15.47:0/1983193942 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55806fdae6d0 0x55806fdb7040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:33:09.118 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:09.117+0000 7fe47be00640 1 -- 172.21.15.47:0/1983193942 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55806fdb7c00 msgr2=0x55806fdb20b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:33:09.118 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:09.117+0000 7fe47be00640 1 --2- 172.21.15.47:0/1983193942 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55806fdb7c00 0x55806fdb20b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:33:09.118 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:09.117+0000 7fe47be00640 1 -- 172.21.15.47:0/1983193942 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55806fdaee20 con 0x55806fdaf080 2024-03-21T10:33:09.118 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:09.117+0000 7fe482800640 1 --2- 172.21.15.47:0/1983193942 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55806fdb7c00 0x55806fdb20b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T10:33:09.118 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:09.117+0000 7fe47be00640 1 --2- 172.21.15.47:0/1983193942 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55806fdaf080 0x55806fdb1b70 secure :-1 s=READY pgs=1113 cs=0 l=1 rev1=1 crypto rx=0x7fe46c0bdec0 tx=0x7fe46c0de040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a377c5310a89c808 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:33:09.118 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:09.117+0000 7fe472600640 1 -- 172.21.15.47:0/1983193942 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe46c0bd6b0 con 0x55806fdaf080 2024-03-21T10:33:09.118 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:09.117+0000 7fe472600640 1 -- 172.21.15.47:0/1983193942 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fe46c0ff040 con 0x55806fdaf080 2024-03-21T10:33:09.118 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:09.117+0000 7fe472600640 1 -- 172.21.15.47:0/1983193942 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe46c0bd890 con 0x55806fdaf080 2024-03-21T10:33:09.118 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:09.118+0000 7fe48b7da740 1 -- 172.21.15.47:0/1983193942 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55806fdaf080 msgr2=0x55806fdb1b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:33:09.118 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:09.118+0000 7fe48b7da740 1 --2- 172.21.15.47:0/1983193942 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55806fdaf080 0x55806fdb1b70 secure :-1 s=READY pgs=1113 cs=0 l=1 rev1=1 crypto rx=0x7fe46c0bdec0 tx=0x7fe46c0de040 comp rx=0 tx=0).stop 2024-03-21T10:33:09.119 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:09.118+0000 7fe48b7da740 1 -- 172.21.15.47:0/1983193942 shutdown_connections 2024-03-21T10:33:09.119 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:09.118+0000 7fe48b7da740 1 --2- 172.21.15.47:0/1983193942 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55806fdb7c00 0x55806fdb20b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:33:09.119 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:09.118+0000 7fe48b7da740 1 --2- 172.21.15.47:0/1983193942 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55806fdaf080 0x55806fdb1b70 unknown :-1 s=CLOSED pgs=1113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:33:09.119 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:09.118+0000 7fe48b7da740 1 --2- 172.21.15.47:0/1983193942 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55806fdae6d0 0x55806fdb7040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:33:09.119 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:09.118+0000 7fe48b7da740 1 -- 172.21.15.47:0/1983193942 >> 172.21.15.47:0/1983193942 conn(0x55806fda8da0 msgr2=0x55806fda7eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:33:09.119 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:09.118+0000 7fe48b7da740 1 -- 172.21.15.47:0/1983193942 shutdown_connections 2024-03-21T10:33:09.119 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:09.118+0000 7fe48b7da740 1 -- 172.21.15.47:0/1983193942 wait complete. 2024-03-21T10:33:09.122 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:09.118+0000 7fe48b7da740 1 -- start start 2024-03-21T10:33:09.122 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:09.119+0000 7fe48b7da740 1 Processor -- start 2024-03-21T10:33:09.122 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:09.119+0000 7fe48b7da740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55806fdae6d0 0x55806fdbf420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:33:09.122 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:09.119+0000 7fe48b7da740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55806fdaf080 0x55806fdbf960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:33:09.122 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:09.119+0000 7fe48b7da740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55806fdb7c00 0x55806fdbd850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:33:09.122 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:09.119+0000 7fe48b7da740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4841b63c0 con 0x55806fdaf080 2024-03-21T10:33:09.122 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:09.119+0000 7fe48b7da740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4841b5d60 con 0x55806fdb7c00 2024-03-21T10:33:09.122 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:09.119+0000 7fe48b7da740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4841b3280 con 0x55806fdae6d0 2024-03-21T10:33:09.122 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:09.119+0000 7fe48b7da740 10 client.0 _ll_register_callbacks cb 0x7fe474009570 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T10:33:09.122 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:09.119+0000 7fe47be00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55806fdaf080 0x55806fdbf960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:33:09.122 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:09.120+0000 7fe47be00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55806fdaf080 0x55806fdbf960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:46948/0 (socket says 172.21.15.47:46948) 2024-03-21T10:33:09.123 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:09.120+0000 7fe47be00640 1 -- 172.21.15.47:0/4217861503 learned_addr learned my addr 172.21.15.47:0/4217861503 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:33:09.123 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:09.120+0000 7fe481e00640 1 --2- 172.21.15.47:0/4217861503 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55806fdae6d0 0x55806fdbf420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:33:09.123 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:09.120+0000 7fe482800640 1 --2- 172.21.15.47:0/4217861503 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55806fdb7c00 0x55806fdbd850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:33:09.123 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:09.120+0000 7fe47be00640 1 -- 172.21.15.47:0/4217861503 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55806fdae6d0 msgr2=0x55806fdbf420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:33:09.123 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:09.120+0000 7fe47be00640 1 --2- 172.21.15.47:0/4217861503 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55806fdae6d0 0x55806fdbf420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:33:09.123 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:09.120+0000 7fe47be00640 1 -- 172.21.15.47:0/4217861503 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55806fdb7c00 msgr2=0x55806fdbd850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:33:09.123 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:09.120+0000 7fe47be00640 1 --2- 172.21.15.47:0/4217861503 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55806fdb7c00 0x55806fdbd850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:33:09.123 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:09.120+0000 7fe47be00640 1 -- 172.21.15.47:0/4217861503 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55806fdbe050 con 0x55806fdaf080 2024-03-21T10:33:09.123 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:09.120+0000 7fe47be00640 1 --2- 172.21.15.47:0/4217861503 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55806fdaf080 0x55806fdbf960 secure :-1 s=READY pgs=1114 cs=0 l=1 rev1=1 crypto rx=0x7fe46c02c720 tx=0x7fe46c0f56d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=28a27906e2e63e1 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:33:09.123 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:09.120+0000 7fe467400640 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-03-21T10:33:09.123 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:09.121+0000 7fe467400640 1 -- 172.21.15.47:0/4217861503 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe46c0d7020 con 0x55806fdaf080 2024-03-21T10:33:09.123 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:09.121+0000 7fe467400640 1 -- 172.21.15.47:0/4217861503 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(99..99 src has 1..99) v4 ==== 9562+0+0 (secure 0 0 0) 0x7fe46c003d90 con 0x55806fdaf080 2024-03-21T10:33:09.123 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:09.121+0000 7fe48b7da740 1 -- 172.21.15.47:0/4217861503 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fe46c0ff040 con 0x55806fdaf080 2024-03-21T10:33:09.124 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:09.121+0000 7fe467400640 1 -- 172.21.15.47:0/4217861503 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe46c0bda70 con 0x55806fdaf080 2024-03-21T10:33:09.124 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:09.121+0000 7fe48b7da740 10 client.18087 fetch_fsmap learned FSMap version 32 2024-03-21T10:33:09.124 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:09.121+0000 7fe48b7da740 1 -- 172.21.15.47:0/4217861503 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x55806fe23c40 con 0x55806fdaf080 2024-03-21T10:33:09.124 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:09.121+0000 7fe467400640 1 -- 172.21.15.47:0/4217861503 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fe46c0ff040 con 0x55806fdaf080 2024-03-21T10:33:09.124 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:09.121+0000 7fe467400640 1 -- 172.21.15.47:0/4217861503 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 32) v1 ==== 7942+0+0 (secure 0 0 0) 0x7fe46c0f4040 con 0x55806fdaf080 2024-03-21T10:33:09.124 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:09.121+0000 7fe48b7da740 10 client.18087 fetch_fsmap finished waiting for FSMap version 32 2024-03-21T10:33:09.124 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:09.121+0000 7fe48b7da740 10 client.18087 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-03-21T10:33:09.124 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:09.121+0000 7fe48b7da740 20 client.18087 populate_metadata read hostname 'smithi047' 2024-03-21T10:33:09.124 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:09.121+0000 7fe48b7da740 1 --2- 172.21.15.47:0/4217861503 >> [v2:172.21.15.100:6836/3497644162,v1:172.21.15.100:6837/3497644162] conn(0x55806fe23e40 0x55806fe26260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:33:09.124 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:09.121+0000 7fe48b7da740 4 client.18087 mds_command: new command op to 34493 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-21T10:33:09.124 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:09.121+0000 7fe48b7da740 1 -- 172.21.15.47:0/4217861503 --> [v2:172.21.15.100:6836/3497644162,v1:172.21.15.100:6837/3497644162] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55806fd19b50 con 0x55806fe23e40 2024-03-21T10:33:09.124 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:09.122+0000 7fe481e00640 1 --2- 172.21.15.47:0/4217861503 >> [v2:172.21.15.100:6836/3497644162,v1:172.21.15.100:6837/3497644162] conn(0x55806fe23e40 0x55806fe26260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:33:09.124 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:09.123+0000 7fe481e00640 1 --2- 172.21.15.47:0/4217861503 >> [v2:172.21.15.100:6836/3497644162,v1:172.21.15.100:6837/3497644162] conn(0x55806fe23e40 0x55806fe26260 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=af1b9efb4059265c server_cookie=8f2bf197df56fc57 in_seq=0 out_seq=0 2024-03-21T10:33:09.124 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:09.123+0000 7fe467400640 10 client.18087 ms_handle_connect on v2:172.21.15.100:6836/3497644162 2024-03-21T10:33:09.126 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:09.123+0000 7fe467400640 1 -- 172.21.15.47:0/4217861503 <== mds.1 v2:172.21.15.100:6836/3497644162 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55806fd19b50 con 0x55806fe23e40 2024-03-21T10:33:09.126 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:09.123+0000 7fe467400640 10 client.18087 handle_command_reply: tid=0 2024-03-21T10:33:09.126 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:09.123+0000 7fe48b7da740 1 client.18087 shutdown 2024-03-21T10:33:09.126 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:09.123+0000 7fe48b7da740 1 -- 172.21.15.47:0/4217861503 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55806fdaf080 msgr2=0x55806fdbf960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:33:09.126 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:09.123+0000 7fe48b7da740 1 --2- 172.21.15.47:0/4217861503 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55806fdaf080 0x55806fdbf960 secure :-1 s=READY pgs=1114 cs=0 l=1 rev1=1 crypto rx=0x7fe46c02c720 tx=0x7fe46c0f56d0 comp rx=0 tx=0).stop 2024-03-21T10:33:09.126 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:09.123+0000 7fe48b7da740 1 -- 172.21.15.47:0/4217861503 shutdown_connections 2024-03-21T10:33:09.126 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:09.123+0000 7fe48b7da740 1 --2- 172.21.15.47:0/4217861503 >> [v2:172.21.15.100:6836/3497644162,v1:172.21.15.100:6837/3497644162] conn(0x55806fe23e40 0x55806fe26260 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:33:09.126 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:09.123+0000 7fe48b7da740 1 --2- 172.21.15.47:0/4217861503 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55806fdb7c00 0x55806fdbd850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:33:09.126 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:09.123+0000 7fe48b7da740 1 --2- 172.21.15.47:0/4217861503 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55806fdaf080 0x55806fdbf960 unknown :-1 s=CLOSED pgs=1114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:33:09.126 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:09.123+0000 7fe48b7da740 1 --2- 172.21.15.47:0/4217861503 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55806fdae6d0 0x55806fdbf420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:33:09.126 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:09.123+0000 7fe467400640 0 client.18087 ms_handle_reset on v2:172.21.15.100:6836/3497644162 2024-03-21T10:33:09.126 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:09.123+0000 7fe48b7da740 1 -- 172.21.15.47:0/4217861503 >> 172.21.15.47:0/4217861503 conn(0x55806fda8da0 msgr2=0x55806fdaf460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:33:09.127 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:09.123+0000 7fe48b7da740 1 -- 172.21.15.47:0/4217861503 shutdown_connections 2024-03-21T10:33:09.127 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:09.124+0000 7fe48b7da740 1 -- 172.21.15.47:0/4217861503 wait complete. 2024-03-21T10:33:09.127 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:09.124+0000 7fe48b7da740 20 client.18087 trim_cache size 0 max 16384 2024-03-21T10:33:09.127 INFO:teuthology.orchestra.run.smithi047.stdout:[] 2024-03-21T10:33:09.128 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:09.124+0000 7fe488a00640 1 -- 172.21.15.47:0/1576725051 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7fe4680a3110 msgr2=0x7fe4680a55d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:33:09.128 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:09.124+0000 7fe488a00640 1 --2- 172.21.15.47:0/1576725051 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7fe4680a3110 0x7fe4680a55d0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fe48406ab50 tx=0x7fe47c009660 comp rx=0 tx=0).stop 2024-03-21T10:33:09.128 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:09.124+0000 7fe488a00640 1 -- 172.21.15.47:0/1576725051 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe4840764f0 msgr2=0x7fe48419ea20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:33:09.128 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:09.124+0000 7fe488a00640 1 --2- 172.21.15.47:0/1576725051 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe4840764f0 0x7fe48419ea20 secure :-1 s=READY pgs=1110 cs=0 l=1 rev1=1 crypto rx=0x7fe46c000c00 tx=0x7fe46c0045e0 comp rx=0 tx=0).stop 2024-03-21T10:33:09.128 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:09.124+0000 7fe488a00640 1 -- 172.21.15.47:0/1576725051 shutdown_connections 2024-03-21T10:33:09.128 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:09.124+0000 7fe488a00640 1 --2- 172.21.15.47:0/1576725051 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7fe4680a3110 0x7fe4680a55d0 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:33:09.128 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:09.124+0000 7fe488a00640 1 --2- 172.21.15.47:0/1576725051 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe4841124e0 0x7fe4841a2da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:33:09.128 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:09.124+0000 7fe488a00640 1 --2- 172.21.15.47:0/1576725051 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe4840764f0 0x7fe48419ea20 unknown :-1 s=CLOSED pgs=1110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:33:09.128 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:09.124+0000 7fe488a00640 1 --2- 172.21.15.47:0/1576725051 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe484075bd0 0x7fe48419e4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:33:09.128 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:09.124+0000 7fe488a00640 1 -- 172.21.15.47:0/1576725051 >> 172.21.15.47:0/1576725051 conn(0x7fe4840ff310 msgr2=0x7fe484113c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:33:09.128 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:09.124+0000 7fe488a00640 1 -- 172.21.15.47:0/1576725051 shutdown_connections 2024-03-21T10:33:09.129 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:09.125+0000 7fe488a00640 1 -- 172.21.15.47:0/1576725051 wait complete. 2024-03-21T10:33:09.141 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-21T10:33:09.141 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-03-21T10:33:09.347 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:09.347+0000 7fb59bc00640 1 -- 172.21.15.47:0/273221406 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb59406a5b0 msgr2=0x7fb59410e4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:33:09.347 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:09.347+0000 7fb59bc00640 1 --2- 172.21.15.47:0/273221406 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb59406a5b0 0x7fb59410e4a0 secure :-1 s=READY pgs=1115 cs=0 l=1 rev1=1 crypto rx=0x7fb58c009ca0 tx=0x7fb58c02c100 comp rx=0 tx=0).stop 2024-03-21T10:33:09.347 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:09.347+0000 7fb59bc00640 1 -- 172.21.15.47:0/273221406 shutdown_connections 2024-03-21T10:33:09.347 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:09.347+0000 7fb59bc00640 1 --2- 172.21.15.47:0/273221406 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb59406a5b0 0x7fb59410e4a0 unknown :-1 s=CLOSED pgs=1115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:33:09.347 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:09.347+0000 7fb59bc00640 1 --2- 172.21.15.47:0/273221406 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb594069c10 0x7fb59406a070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:33:09.347 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:09.347+0000 7fb59bc00640 1 --2- 172.21.15.47:0/273221406 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb594107650 0x7fb594107a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:33:09.347 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:09.347+0000 7fb59bc00640 1 -- 172.21.15.47:0/273221406 >> 172.21.15.47:0/273221406 conn(0x7fb594077490 msgr2=0x7fb5940778a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:33:09.347 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:09.348+0000 7fb59bc00640 1 -- 172.21.15.47:0/273221406 shutdown_connections 2024-03-21T10:33:09.347 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:09.348+0000 7fb59bc00640 1 -- 172.21.15.47:0/273221406 wait complete. 2024-03-21T10:33:09.349 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:09.349+0000 7fb59bc00640 1 Processor -- start 2024-03-21T10:33:09.349 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:09.349+0000 7fb59bc00640 1 -- start start 2024-03-21T10:33:09.349 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:09.349+0000 7fb59bc00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb594069c10 0x7fb594197e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:33:09.349 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:09.349+0000 7fb59bc00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb59406a5b0 0x7fb594198390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:33:09.349 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:09.349+0000 7fb59bc00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb594107650 0x7fb59419c710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:33:09.349 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:09.349+0000 7fb59bc00640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5941118f0 con 0x7fb59406a5b0 2024-03-21T10:33:09.349 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:09.349+0000 7fb59bc00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb594111770 con 0x7fb594069c10 2024-03-21T10:33:09.349 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:09.349+0000 7fb59bc00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb594111a70 con 0x7fb594107650 2024-03-21T10:33:09.349 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:09.349+0000 7fb593e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb59406a5b0 0x7fb594198390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:33:09.349 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:09.350+0000 7fb593e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb59406a5b0 0x7fb594198390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:46960/0 (socket says 172.21.15.47:46960) 2024-03-21T10:33:09.349 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:09.350+0000 7fb593e00640 1 -- 172.21.15.47:0/2051183530 learned_addr learned my addr 172.21.15.47:0/2051183530 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:33:09.350 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:09.350+0000 7fb599200640 1 --2- 172.21.15.47:0/2051183530 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb594069c10 0x7fb594197e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:33:09.351 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:09.350+0000 7fb599c00640 1 --2- 172.21.15.47:0/2051183530 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb594107650 0x7fb59419c710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:33:09.351 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:09.350+0000 7fb593e00640 1 -- 172.21.15.47:0/2051183530 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb594107650 msgr2=0x7fb59419c710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:33:09.351 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:09.350+0000 7fb593e00640 1 --2- 172.21.15.47:0/2051183530 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb594107650 0x7fb59419c710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:33:09.351 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:09.350+0000 7fb593e00640 1 -- 172.21.15.47:0/2051183530 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb594069c10 msgr2=0x7fb594197e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:33:09.351 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:09.350+0000 7fb593e00640 1 --2- 172.21.15.47:0/2051183530 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb594069c10 0x7fb594197e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:33:09.351 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:09.350+0000 7fb593e00640 1 -- 172.21.15.47:0/2051183530 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb59419ce40 con 0x7fb59406a5b0 2024-03-21T10:33:09.351 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:09.350+0000 7fb593e00640 1 --2- 172.21.15.47:0/2051183530 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb59406a5b0 0x7fb594198390 secure :-1 s=READY pgs=1116 cs=0 l=1 rev1=1 crypto rx=0x7fb58000e9f0 tx=0x7fb58000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:33:09.351 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:09.351+0000 7fb591600640 1 -- 172.21.15.47:0/2051183530 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb58000cd80 con 0x7fb59406a5b0 2024-03-21T10:33:09.352 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:09.351+0000 7fb591600640 1 -- 172.21.15.47:0/2051183530 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fb5800108e0 con 0x7fb59406a5b0 2024-03-21T10:33:09.352 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:09.351+0000 7fb591600640 1 -- 172.21.15.47:0/2051183530 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb580018920 con 0x7fb59406a5b0 2024-03-21T10:33:09.352 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:09.351+0000 7fb59bc00640 1 -- 172.21.15.47:0/2051183530 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb59419d130 con 0x7fb59406a5b0 2024-03-21T10:33:09.352 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:09.351+0000 7fb59bc00640 1 -- 172.21.15.47:0/2051183530 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb5941a49c0 con 0x7fb59406a5b0 2024-03-21T10:33:09.353 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:09.354+0000 7fb591600640 1 -- 172.21.15.47:0/2051183530 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 27) v1 ==== 137985+0+0 (secure 0 0 0) 0x7fb580010a90 con 0x7fb59406a5b0 2024-03-21T10:33:09.353 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:09.354+0000 7fb59e822740 1 Processor -- start 2024-03-21T10:33:09.354 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:09.354+0000 7fb59e822740 1 -- start start 2024-03-21T10:33:09.355 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:09.355+0000 7fb59e822740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a2bf311450 0x55a2bf311830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:33:09.355 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:09.355+0000 7fb59e822740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a2bf311e00 0x55a2bf31ea40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:33:09.355 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:09.355+0000 7fb59e822740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a2bf31f010 0x55a2bf321420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:33:09.355 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:09.355+0000 7fb59e822740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a2bf2752c0 con 0x55a2bf31f010 2024-03-21T10:33:09.355 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:09.355+0000 7fb59e822740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a2bf21dc90 con 0x55a2bf311e00 2024-03-21T10:33:09.356 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:09.355+0000 7fb59e822740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a2bf271360 con 0x55a2bf311450 2024-03-21T10:33:09.356 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:09.355+0000 7fb593e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a2bf31f010 0x55a2bf321420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:33:09.356 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:09.355+0000 7fb593e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a2bf31f010 0x55a2bf321420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:46970/0 (socket says 172.21.15.47:46970) 2024-03-21T10:33:09.356 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:09.355+0000 7fb593e00640 1 -- 172.21.15.47:0/1547197125 learned_addr learned my addr 172.21.15.47:0/1547197125 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:33:09.356 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:09.355+0000 7fb599c00640 1 --2- 172.21.15.47:0/1547197125 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a2bf311450 0x55a2bf311830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:33:09.356 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:09.355+0000 7fb599200640 1 --2- 172.21.15.47:0/1547197125 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a2bf311e00 0x55a2bf31ea40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:33:09.356 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:09.356+0000 7fb593e00640 1 -- 172.21.15.47:0/1547197125 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a2bf311450 msgr2=0x55a2bf311830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:33:09.356 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:09.356+0000 7fb593e00640 1 --2- 172.21.15.47:0/1547197125 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a2bf311450 0x55a2bf311830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:33:09.356 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:09.356+0000 7fb593e00640 1 -- 172.21.15.47:0/1547197125 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a2bf311e00 msgr2=0x55a2bf31ea40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:33:09.356 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:09.356+0000 7fb593e00640 1 --2- 172.21.15.47:0/1547197125 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a2bf311e00 0x55a2bf31ea40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:33:09.356 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:09.356+0000 7fb593e00640 1 -- 172.21.15.47:0/1547197125 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a2bf242080 con 0x55a2bf31f010 2024-03-21T10:33:09.356 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:09.356+0000 7fb591600640 1 --2- 172.21.15.47:0/2051183530 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7fb57c0a3110 0x7fb57c0a55d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:33:09.356 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:09.356+0000 7fb593e00640 1 --2- 172.21.15.47:0/1547197125 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a2bf31f010 0x55a2bf321420 secure :-1 s=READY pgs=1117 cs=0 l=1 rev1=1 crypto rx=0x7fb5800cd1c0 tx=0x7fb5800d8640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6f00c3077a06f0c6 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:33:09.358 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:09.356+0000 7fb591600640 1 -- 172.21.15.47:0/2051183530 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 9562+0+0 (secure 0 0 0) 0x7fb5800cff10 con 0x7fb59406a5b0 2024-03-21T10:33:09.358 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:09.356+0000 7fb599c00640 1 --2- 172.21.15.47:0/2051183530 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7fb57c0a3110 0x7fb57c0a55d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:33:09.358 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:09.357+0000 7fb586000640 1 -- 172.21.15.47:0/1547197125 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb580026030 con 0x55a2bf31f010 2024-03-21T10:33:09.358 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:09.357+0000 7fb586000640 1 -- 172.21.15.47:0/1547197125 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fb5800d8e40 con 0x55a2bf31f010 2024-03-21T10:33:09.358 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:09.357+0000 7fb586000640 1 -- 172.21.15.47:0/1547197125 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb58009ba30 con 0x55a2bf31f010 2024-03-21T10:33:09.358 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:09.358+0000 7fb59e822740 1 -- 172.21.15.47:0/1547197125 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a2bf31f010 msgr2=0x55a2bf321420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:33:09.358 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:09.358+0000 7fb59e822740 1 --2- 172.21.15.47:0/1547197125 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a2bf31f010 0x55a2bf321420 secure :-1 s=READY pgs=1117 cs=0 l=1 rev1=1 crypto rx=0x7fb5800cd1c0 tx=0x7fb5800d8640 comp rx=0 tx=0).stop 2024-03-21T10:33:09.358 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:09.358+0000 7fb599c00640 1 --2- 172.21.15.47:0/2051183530 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7fb57c0a3110 0x7fb57c0a55d0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fb58c005e30 tx=0x7fb58c005d80 comp rx=0 tx=0).ready entity=mgr.15687 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:33:09.358 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:09.358+0000 7fb59e822740 1 -- 172.21.15.47:0/1547197125 shutdown_connections 2024-03-21T10:33:09.358 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:09.358+0000 7fb59e822740 1 --2- 172.21.15.47:0/1547197125 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a2bf31f010 0x55a2bf321420 unknown :-1 s=CLOSED pgs=1117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:33:09.359 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:09.358+0000 7fb59e822740 1 --2- 172.21.15.47:0/1547197125 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a2bf311e00 0x55a2bf31ea40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:33:09.359 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:09.358+0000 7fb59e822740 1 --2- 172.21.15.47:0/1547197125 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a2bf311450 0x55a2bf311830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:33:09.359 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:09.358+0000 7fb59e822740 1 -- 172.21.15.47:0/1547197125 >> 172.21.15.47:0/1547197125 conn(0x55a2bf30bda0 msgr2=0x55a2bf30c180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:33:09.359 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:09.358+0000 7fb59e822740 1 -- 172.21.15.47:0/1547197125 shutdown_connections 2024-03-21T10:33:09.359 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:09.358+0000 7fb59e822740 1 -- 172.21.15.47:0/1547197125 wait complete. 2024-03-21T10:33:09.359 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:09.359+0000 7fb59e822740 1 -- start start 2024-03-21T10:33:09.359 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:09.359+0000 7fb59e822740 1 Processor -- start 2024-03-21T10:33:09.362 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:09.359+0000 7fb59e822740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a2bf311450 0x55a2bf31e8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:33:09.362 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:09.360+0000 7fb59e822740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a2bf311e00 0x55a2bf31bc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:33:09.363 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:09.360+0000 7fb59e822740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a2bf31f010 0x55a2bf31c1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:33:09.363 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:09.360+0000 7fb59e822740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a2bf321f40 con 0x55a2bf311450 2024-03-21T10:33:09.363 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:09.360+0000 7fb59e822740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a2bf2310d0 con 0x55a2bf31f010 2024-03-21T10:33:09.363 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:09.360+0000 7fb59e822740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a2bf3220c0 con 0x55a2bf311e00 2024-03-21T10:33:09.363 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:09.360+0000 7fb59e822740 10 client.0 _ll_register_callbacks cb 0x55a2bf235e80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T10:33:09.363 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:09.360+0000 7fb599200640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a2bf311450 0x55a2bf31e8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:33:09.363 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:09.360+0000 7fb599200640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a2bf311450 0x55a2bf31e8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:46972/0 (socket says 172.21.15.47:46972) 2024-03-21T10:33:09.363 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:09.360+0000 7fb599200640 1 -- 172.21.15.47:0/1194816080 learned_addr learned my addr 172.21.15.47:0/1194816080 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:33:09.363 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:09.360+0000 7fb599c00640 1 --2- 172.21.15.47:0/1194816080 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a2bf31f010 0x55a2bf31c1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:33:09.363 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:09.360+0000 7fb593e00640 1 --2- 172.21.15.47:0/1194816080 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a2bf311e00 0x55a2bf31bc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:33:09.363 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:09.360+0000 7fb599200640 1 -- 172.21.15.47:0/1194816080 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a2bf311e00 msgr2=0x55a2bf31bc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:33:09.363 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:09.360+0000 7fb599200640 1 --2- 172.21.15.47:0/1194816080 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a2bf311e00 0x55a2bf31bc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:33:09.363 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:09.360+0000 7fb599200640 1 -- 172.21.15.47:0/1194816080 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a2bf31f010 msgr2=0x55a2bf31c1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:33:09.364 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:09.360+0000 7fb599200640 1 --2- 172.21.15.47:0/1194816080 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a2bf31f010 0x55a2bf31c1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:33:09.364 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:09.360+0000 7fb599200640 1 -- 172.21.15.47:0/1194816080 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55a2bef893f0 con 0x55a2bf311450 2024-03-21T10:33:09.364 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:09.361+0000 7fb599200640 1 --2- 172.21.15.47:0/1194816080 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a2bf311450 0x55a2bf31e8f0 secure :-1 s=READY pgs=1118 cs=0 l=1 rev1=1 crypto rx=0x7fb58800b9f0 tx=0x7fb58800beb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f7f9e9d410e8e3c7 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:33:09.364 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:09.361+0000 7fb57aa00640 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-03-21T10:33:09.364 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:09.361+0000 7fb57aa00640 1 -- 172.21.15.47:0/1194816080 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb588003d60 con 0x55a2bf311450 2024-03-21T10:33:09.364 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:09.361+0000 7fb57aa00640 1 -- 172.21.15.47:0/1194816080 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(99..99 src has 1..99) v4 ==== 9562+0+0 (secure 0 0 0) 0x7fb588011440 con 0x55a2bf311450 2024-03-21T10:33:09.364 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:09.361+0000 7fb59e822740 1 -- 172.21.15.47:0/1194816080 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55a2bf282800 con 0x55a2bf311450 2024-03-21T10:33:09.364 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:09.361+0000 7fb57aa00640 1 -- 172.21.15.47:0/1194816080 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb588011640 con 0x55a2bf311450 2024-03-21T10:33:09.364 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:09.361+0000 7fb59e822740 10 client.18099 fetch_fsmap learned FSMap version 32 2024-03-21T10:33:09.364 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:09.361+0000 7fb59e822740 1 -- 172.21.15.47:0/1194816080 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x55a2bf0537d0 con 0x55a2bf311450 2024-03-21T10:33:09.364 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:09.362+0000 7fb57aa00640 1 -- 172.21.15.47:0/1194816080 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x55a2bf282800 con 0x55a2bf311450 2024-03-21T10:33:09.364 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:09.362+0000 7fb57aa00640 1 -- 172.21.15.47:0/1194816080 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 32) v1 ==== 7942+0+0 (secure 0 0 0) 0x7fb58801b870 con 0x55a2bf311450 2024-03-21T10:33:09.364 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:09.362+0000 7fb59e822740 10 client.18099 fetch_fsmap finished waiting for FSMap version 32 2024-03-21T10:33:09.364 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:09.362+0000 7fb59e822740 10 client.18099 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-21T10:33:09.364 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:09.362+0000 7fb59e822740 20 client.18099 populate_metadata read hostname 'smithi047' 2024-03-21T10:33:09.364 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:09.362+0000 7fb59e822740 1 --2- 172.21.15.47:0/1194816080 >> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] conn(0x55a2bf32eee0 0x55a2bf3312f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:33:09.365 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:09.362+0000 7fb59e822740 4 client.18099 mds_command: new command op to 34535 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T10:33:09.365 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:09.362+0000 7fb59e822740 1 -- 172.21.15.47:0/1194816080 --> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55a2bf27cb50 con 0x55a2bf32eee0 2024-03-21T10:33:09.365 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:09.363+0000 7fb593e00640 1 --2- 172.21.15.47:0/1194816080 >> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] conn(0x55a2bf32eee0 0x55a2bf3312f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:33:09.365 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:09.363+0000 7fb593e00640 1 --2- 172.21.15.47:0/1194816080 >> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] conn(0x55a2bf32eee0 0x55a2bf3312f0 crc :-1 s=READY pgs=185 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9ab5e6b94da8085c server_cookie=a7bef40034114141 in_seq=0 out_seq=0 2024-03-21T10:33:09.365 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:09.363+0000 7fb57aa00640 10 client.18099 ms_handle_connect on v2:172.21.15.100:6838/4173168440 2024-03-21T10:33:09.366 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:09.365+0000 7fb57aa00640 1 -- 172.21.15.47:0/1194816080 <== mds.0 v2:172.21.15.100:6838/4173168440 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55a2bf27cb50 con 0x55a2bf32eee0 2024-03-21T10:33:09.366 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:09.365+0000 7fb57aa00640 10 client.18099 handle_command_reply: tid=0 2024-03-21T10:33:09.366 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:09.365+0000 7fb59e822740 1 client.18099 shutdown 2024-03-21T10:33:09.366 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:09.365+0000 7fb59e822740 1 -- 172.21.15.47:0/1194816080 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a2bf311450 msgr2=0x55a2bf31e8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:33:09.366 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:09.365+0000 7fb59e822740 1 --2- 172.21.15.47:0/1194816080 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a2bf311450 0x55a2bf31e8f0 secure :-1 s=READY pgs=1118 cs=0 l=1 rev1=1 crypto rx=0x7fb58800b9f0 tx=0x7fb58800beb0 comp rx=0 tx=0).stop 2024-03-21T10:33:09.366 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:09.365+0000 7fb59e822740 1 -- 172.21.15.47:0/1194816080 shutdown_connections 2024-03-21T10:33:09.366 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:09.365+0000 7fb59e822740 1 --2- 172.21.15.47:0/1194816080 >> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] conn(0x55a2bf32eee0 0x55a2bf3312f0 crc :-1 s=READY pgs=185 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:33:09.366 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:09.365+0000 7fb59e822740 1 --2- 172.21.15.47:0/1194816080 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a2bf31f010 0x55a2bf31c1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:33:09.366 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:09.365+0000 7fb59e822740 1 --2- 172.21.15.47:0/1194816080 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a2bf311e00 0x55a2bf31bc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:33:09.367 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:09.365+0000 7fb59e822740 1 --2- 172.21.15.47:0/1194816080 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a2bf311450 0x55a2bf31e8f0 unknown :-1 s=CLOSED pgs=1118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:33:09.367 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:09.365+0000 7fb57aa00640 0 client.18099 ms_handle_reset on v2:172.21.15.100:6838/4173168440 2024-03-21T10:33:09.367 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:09.365+0000 7fb59e822740 1 -- 172.21.15.47:0/1194816080 >> 172.21.15.47:0/1194816080 conn(0x55a2bf30bda0 msgr2=0x55a2bf30daf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:33:09.367 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:09.365+0000 7fb59e822740 1 -- 172.21.15.47:0/1194816080 shutdown_connections 2024-03-21T10:33:09.367 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:09.365+0000 7fb59e822740 1 -- 172.21.15.47:0/1194816080 wait complete. 2024-03-21T10:33:09.367 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:09.365+0000 7fb59e822740 20 client.18099 trim_cache size 0 max 16384 2024-03-21T10:33:09.379 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:09.379+0000 7fb59e822740 1 Processor -- start 2024-03-21T10:33:09.379 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:09.379+0000 7fb59e822740 1 -- start start 2024-03-21T10:33:09.379 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:09.379+0000 7fb59e822740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a2bf311450 0x55a2bf30c180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:33:09.379 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:09.379+0000 7fb59e822740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a2bf311e00 0x55a2bf31e3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:33:09.379 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:09.379+0000 7fb59e822740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a2bf31f010 0x55a2bf31e8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:33:09.380 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:09.379+0000 7fb59e822740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5941ac850 con 0x55a2bf31f010 2024-03-21T10:33:09.380 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:09.379+0000 7fb59e822740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5941ae1d0 con 0x55a2bf311450 2024-03-21T10:33:09.380 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:09.379+0000 7fb59e822740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5941aceb0 con 0x55a2bf311e00 2024-03-21T10:33:09.380 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:09.380+0000 7fb599c00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a2bf31f010 0x55a2bf31e8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:33:09.380 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:09.380+0000 7fb599c00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a2bf31f010 0x55a2bf31e8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:46978/0 (socket says 172.21.15.47:46978) 2024-03-21T10:33:09.380 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:09.380+0000 7fb599c00640 1 -- 172.21.15.47:0/1103231254 learned_addr learned my addr 172.21.15.47:0/1103231254 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:33:09.380 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:09.380+0000 7fb599200640 1 --2- 172.21.15.47:0/1103231254 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a2bf311450 0x55a2bf30c180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:33:09.380 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:09.380+0000 7fb593e00640 1 --2- 172.21.15.47:0/1103231254 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a2bf311e00 0x55a2bf31e3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:33:09.383 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:09.380+0000 7fb599c00640 1 -- 172.21.15.47:0/1103231254 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a2bf311e00 msgr2=0x55a2bf31e3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:33:09.383 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:09.380+0000 7fb599c00640 1 --2- 172.21.15.47:0/1103231254 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a2bf311e00 0x55a2bf31e3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:33:09.383 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:09.380+0000 7fb599c00640 1 -- 172.21.15.47:0/1103231254 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a2bf311450 msgr2=0x55a2bf30c180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:33:09.383 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:09.380+0000 7fb599c00640 1 --2- 172.21.15.47:0/1103231254 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a2bf311450 0x55a2bf30c180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:33:09.383 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:09.380+0000 7fb599c00640 1 -- 172.21.15.47:0/1103231254 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a2bf311ba0 con 0x55a2bf31f010 2024-03-21T10:33:09.384 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:09.381+0000 7fb599c00640 1 --2- 172.21.15.47:0/1103231254 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a2bf31f010 0x55a2bf31e8f0 secure :-1 s=READY pgs=1119 cs=0 l=1 rev1=1 crypto rx=0x7fb58c037040 tx=0x7fb58c040000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=deabb58dbb8d99f1 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:33:09.384 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:09.381+0000 7fb585600640 1 -- 172.21.15.47:0/1103231254 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb58c039040 con 0x55a2bf31f010 2024-03-21T10:33:09.384 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:09.381+0000 7fb585600640 1 -- 172.21.15.47:0/1103231254 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fb58c053070 con 0x55a2bf31f010 2024-03-21T10:33:09.384 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:09.381+0000 7fb585600640 1 -- 172.21.15.47:0/1103231254 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb58c0406c0 con 0x55a2bf31f010 2024-03-21T10:33:09.384 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:09.382+0000 7fb59e822740 1 -- 172.21.15.47:0/1103231254 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a2bf31f010 msgr2=0x55a2bf31e8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:33:09.384 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:09.382+0000 7fb59e822740 1 --2- 172.21.15.47:0/1103231254 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a2bf31f010 0x55a2bf31e8f0 secure :-1 s=READY pgs=1119 cs=0 l=1 rev1=1 crypto rx=0x7fb58c037040 tx=0x7fb58c040000 comp rx=0 tx=0).stop 2024-03-21T10:33:09.384 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:09.382+0000 7fb59e822740 1 -- 172.21.15.47:0/1103231254 shutdown_connections 2024-03-21T10:33:09.384 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:09.382+0000 7fb59e822740 1 --2- 172.21.15.47:0/1103231254 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a2bf31f010 0x55a2bf31e8f0 unknown :-1 s=CLOSED pgs=1119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:33:09.384 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:09.382+0000 7fb59e822740 1 --2- 172.21.15.47:0/1103231254 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a2bf311e00 0x55a2bf31e3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:33:09.384 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:09.382+0000 7fb59e822740 1 --2- 172.21.15.47:0/1103231254 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a2bf311450 0x55a2bf30c180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:33:09.384 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:09.382+0000 7fb59e822740 1 -- 172.21.15.47:0/1103231254 >> 172.21.15.47:0/1103231254 conn(0x55a2bf30bda0 msgr2=0x55a2bf30aeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:33:09.384 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:09.382+0000 7fb59e822740 1 -- 172.21.15.47:0/1103231254 shutdown_connections 2024-03-21T10:33:09.384 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:09.382+0000 7fb59e822740 1 -- 172.21.15.47:0/1103231254 wait complete. 2024-03-21T10:33:09.384 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:09.382+0000 7fb59e822740 1 -- start start 2024-03-21T10:33:09.385 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:09.383+0000 7fb59e822740 1 Processor -- start 2024-03-21T10:33:09.385 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:09.383+0000 7fb59e822740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a2bf311450 0x55a2bf322e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:33:09.385 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:09.383+0000 7fb59e822740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a2bf311e00 0x55a2bf3233c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:33:09.385 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:09.383+0000 7fb59e822740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a2bf31f010 0x55a2bf389780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:33:09.385 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:09.383+0000 7fb59e822740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5941adb70 con 0x55a2bf311e00 2024-03-21T10:33:09.385 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:09.383+0000 7fb59e822740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5941ad510 con 0x55a2bf311450 2024-03-21T10:33:09.385 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:09.383+0000 7fb59e822740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5941aaa30 con 0x55a2bf31f010 2024-03-21T10:33:09.385 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:09.383+0000 7fb59e822740 10 client.0 _ll_register_callbacks cb 0x7fb588004040 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T10:33:09.385 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:09.383+0000 7fb593e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a2bf311e00 0x55a2bf3233c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:33:09.385 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:09.383+0000 7fb599200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a2bf311450 0x55a2bf322e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:33:09.385 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:09.383+0000 7fb593e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a2bf311e00 0x55a2bf3233c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:46984/0 (socket says 172.21.15.47:46984) 2024-03-21T10:33:09.385 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:09.384+0000 7fb593e00640 1 -- 172.21.15.47:0/3485559015 learned_addr learned my addr 172.21.15.47:0/3485559015 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:33:09.385 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:09.384+0000 7fb599c00640 1 --2- 172.21.15.47:0/3485559015 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a2bf31f010 0x55a2bf389780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:33:09.385 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:09.384+0000 7fb593e00640 1 -- 172.21.15.47:0/3485559015 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a2bf31f010 msgr2=0x55a2bf389780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:33:09.385 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:09.384+0000 7fb593e00640 1 --2- 172.21.15.47:0/3485559015 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a2bf31f010 0x55a2bf389780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:33:09.386 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:09.384+0000 7fb593e00640 1 -- 172.21.15.47:0/3485559015 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a2bf311450 msgr2=0x55a2bf322e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:33:09.386 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:09.384+0000 7fb593e00640 1 --2- 172.21.15.47:0/3485559015 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a2bf311450 0x55a2bf322e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:33:09.386 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:09.384+0000 7fb593e00640 1 -- 172.21.15.47:0/3485559015 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55a2bf389cc0 con 0x55a2bf311e00 2024-03-21T10:33:09.386 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:09.384+0000 7fb599200640 1 --2- 172.21.15.47:0/3485559015 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a2bf311450 0x55a2bf322e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T10:33:09.388 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:09.384+0000 7fb593e00640 1 --2- 172.21.15.47:0/3485559015 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a2bf311e00 0x55a2bf3233c0 secure :-1 s=READY pgs=1120 cs=0 l=1 rev1=1 crypto rx=0x7fb58009bbf0 tx=0x7fb580000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3ae74916614263b9 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:33:09.388 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:09.384+0000 7fb57aa00640 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-03-21T10:33:09.388 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:09.384+0000 7fb57aa00640 1 -- 172.21.15.47:0/3485559015 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb58009b850 con 0x55a2bf311e00 2024-03-21T10:33:09.388 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:09.385+0000 7fb57aa00640 1 -- 172.21.15.47:0/3485559015 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(99..99 src has 1..99) v4 ==== 9562+0+0 (secure 0 0 0) 0x7fb5800d3b90 con 0x55a2bf311e00 2024-03-21T10:33:09.388 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:09.385+0000 7fb59e822740 1 -- 172.21.15.47:0/3485559015 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fb58c053070 con 0x55a2bf311e00 2024-03-21T10:33:09.388 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:09.385+0000 7fb57aa00640 1 -- 172.21.15.47:0/3485559015 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb58009ba30 con 0x55a2bf311e00 2024-03-21T10:33:09.388 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:09.385+0000 7fb59e822740 10 client.18105 fetch_fsmap learned FSMap version 32 2024-03-21T10:33:09.389 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:09.385+0000 7fb59e822740 1 -- 172.21.15.47:0/3485559015 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x55a2bf38af90 con 0x55a2bf311e00 2024-03-21T10:33:09.389 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:09.385+0000 7fb57aa00640 1 -- 172.21.15.47:0/3485559015 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fb58c053070 con 0x55a2bf311e00 2024-03-21T10:33:09.389 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:09.385+0000 7fb57aa00640 1 -- 172.21.15.47:0/3485559015 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 32) v1 ==== 7942+0+0 (secure 0 0 0) 0x7fb5800cd4d0 con 0x55a2bf311e00 2024-03-21T10:33:09.389 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:09.385+0000 7fb59e822740 10 client.18105 fetch_fsmap finished waiting for FSMap version 32 2024-03-21T10:33:09.389 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:09.385+0000 7fb59e822740 10 client.18105 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-21T10:33:09.389 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:09.385+0000 7fb59e822740 20 client.18105 populate_metadata read hostname 'smithi047' 2024-03-21T10:33:09.389 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:09.385+0000 7fb59e822740 1 --2- 172.21.15.47:0/3485559015 >> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] conn(0x55a2bf38b320 0x55a2bf38d710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:33:09.389 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:09.385+0000 7fb59e822740 4 client.18105 mds_command: new command op to 34535 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-21T10:33:09.389 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:09.385+0000 7fb59e822740 1 -- 172.21.15.47:0/3485559015 --> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55a2bf1fa500 con 0x55a2bf38b320 2024-03-21T10:33:09.389 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:09.386+0000 7fb599200640 1 --2- 172.21.15.47:0/3485559015 >> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] conn(0x55a2bf38b320 0x55a2bf38d710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:33:09.389 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:09.387+0000 7fb599200640 1 --2- 172.21.15.47:0/3485559015 >> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] conn(0x55a2bf38b320 0x55a2bf38d710 crc :-1 s=READY pgs=186 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=78866f39a1c898a4 server_cookie=26dd54fe7e79b9f in_seq=0 out_seq=0 2024-03-21T10:33:09.389 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:09.387+0000 7fb57aa00640 10 client.18105 ms_handle_connect on v2:172.21.15.100:6838/4173168440 2024-03-21T10:33:09.389 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:09.387+0000 7fb57aa00640 1 -- 172.21.15.47:0/3485559015 <== mds.0 v2:172.21.15.100:6838/4173168440 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55a2bf1fa500 con 0x55a2bf38b320 2024-03-21T10:33:09.389 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:09.387+0000 7fb57aa00640 10 client.18105 handle_command_reply: tid=0 2024-03-21T10:33:09.389 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:09.387+0000 7fb59e822740 1 client.18105 shutdown 2024-03-21T10:33:09.390 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:09.387+0000 7fb59e822740 1 -- 172.21.15.47:0/3485559015 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a2bf311e00 msgr2=0x55a2bf3233c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:33:09.390 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:09.387+0000 7fb59e822740 1 --2- 172.21.15.47:0/3485559015 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a2bf311e00 0x55a2bf3233c0 secure :-1 s=READY pgs=1120 cs=0 l=1 rev1=1 crypto rx=0x7fb58009bbf0 tx=0x7fb580000bc0 comp rx=0 tx=0).stop 2024-03-21T10:33:09.390 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:09.387+0000 7fb59e822740 1 -- 172.21.15.47:0/3485559015 shutdown_connections 2024-03-21T10:33:09.390 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:09.387+0000 7fb59e822740 1 --2- 172.21.15.47:0/3485559015 >> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] conn(0x55a2bf38b320 0x55a2bf38d710 crc :-1 s=READY pgs=186 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:33:09.390 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:09.387+0000 7fb59e822740 1 --2- 172.21.15.47:0/3485559015 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a2bf31f010 0x55a2bf389780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:33:09.390 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:09.387+0000 7fb59e822740 1 --2- 172.21.15.47:0/3485559015 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a2bf311e00 0x55a2bf3233c0 unknown :-1 s=CLOSED pgs=1120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:33:09.390 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:09.387+0000 7fb59e822740 1 --2- 172.21.15.47:0/3485559015 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a2bf311450 0x55a2bf322e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:33:09.390 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:09.387+0000 7fb59e822740 1 -- 172.21.15.47:0/3485559015 >> 172.21.15.47:0/3485559015 conn(0x55a2bf30bda0 msgr2=0x55a2bf30aeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:33:09.390 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:09.387+0000 7fb57aa00640 0 client.18105 ms_handle_reset on v2:172.21.15.100:6838/4173168440 2024-03-21T10:33:09.390 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:09.387+0000 7fb59e822740 1 -- 172.21.15.47:0/3485559015 shutdown_connections 2024-03-21T10:33:09.390 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:09.388+0000 7fb59e822740 1 -- 172.21.15.47:0/3485559015 wait complete. 2024-03-21T10:33:09.391 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:09.388+0000 7fb59e822740 20 client.18105 trim_cache size 0 max 16384 2024-03-21T10:33:09.391 INFO:teuthology.orchestra.run.smithi047.stdout:[] 2024-03-21T10:33:09.392 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:09.388+0000 7fb59bc00640 1 -- 172.21.15.47:0/2051183530 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7fb57c0a3110 msgr2=0x7fb57c0a55d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:33:09.392 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:09.388+0000 7fb59bc00640 1 --2- 172.21.15.47:0/2051183530 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7fb57c0a3110 0x7fb57c0a55d0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fb58c005e30 tx=0x7fb58c005d80 comp rx=0 tx=0).stop 2024-03-21T10:33:09.392 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:09.388+0000 7fb59bc00640 1 -- 172.21.15.47:0/2051183530 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb59406a5b0 msgr2=0x7fb594198390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:33:09.392 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:09.388+0000 7fb59bc00640 1 --2- 172.21.15.47:0/2051183530 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb59406a5b0 0x7fb594198390 secure :-1 s=READY pgs=1116 cs=0 l=1 rev1=1 crypto rx=0x7fb58000e9f0 tx=0x7fb58000eeb0 comp rx=0 tx=0).stop 2024-03-21T10:33:09.392 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:09.389+0000 7fb59bc00640 1 -- 172.21.15.47:0/2051183530 shutdown_connections 2024-03-21T10:33:09.392 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:09.389+0000 7fb59bc00640 1 --2- 172.21.15.47:0/2051183530 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb594107650 0x7fb59419c710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:33:09.392 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:09.389+0000 7fb59bc00640 1 --2- 172.21.15.47:0/2051183530 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb59406a5b0 0x7fb594198390 unknown :-1 s=CLOSED pgs=1116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:33:09.393 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:09.389+0000 7fb59bc00640 1 --2- 172.21.15.47:0/2051183530 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7fb57c0a3110 0x7fb57c0a55d0 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:33:09.393 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:09.389+0000 7fb59bc00640 1 --2- 172.21.15.47:0/2051183530 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb594069c10 0x7fb594197e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:33:09.393 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:09.389+0000 7fb59bc00640 1 -- 172.21.15.47:0/2051183530 >> 172.21.15.47:0/2051183530 conn(0x7fb594077490 msgr2=0x7fb5940ff5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:33:09.393 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:09.389+0000 7fb59bc00640 1 -- 172.21.15.47:0/2051183530 shutdown_connections 2024-03-21T10:33:09.393 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:09.389+0000 7fb59bc00640 1 -- 172.21.15.47:0/2051183530 wait complete. 2024-03-21T10:33:09.408 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:33:09 smithi047 ceph-mon[31547]: pgmap v240: 129 pgs: 129 active+clean; 277 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 44 KiB/s wr, 26 op/s 2024-03-21T10:33:09.727 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:33:09 smithi100 ceph-mon[35193]: pgmap v240: 129 pgs: 129 active+clean; 277 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 44 KiB/s wr, 26 op/s 2024-03-21T10:33:09.737 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:33:09 smithi157 ceph-mon[35581]: pgmap v240: 129 pgs: 129 active+clean; 277 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 44 KiB/s wr, 26 op/s 2024-03-21T10:33:10.413 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-03-21T10:33:10.413 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-21T10:33:10.413 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-03-21T10:33:10.666 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:10.662+0000 7f2127200640 1 -- 172.21.15.47:0/869500126 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f21281027c0 msgr2=0x7f2128102ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:33:10.666 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:10.662+0000 7f2127200640 1 --2- 172.21.15.47:0/869500126 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f21281027c0 0x7f2128102ba0 secure :-1 s=READY pgs=1121 cs=0 l=1 rev1=1 crypto rx=0x7f2118009900 tx=0x7f211802bcd0 comp rx=0 tx=0).stop 2024-03-21T10:33:10.666 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:10.662+0000 7f2127200640 1 -- 172.21.15.47:0/869500126 shutdown_connections 2024-03-21T10:33:10.666 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:10.662+0000 7f2127200640 1 --2- 172.21.15.47:0/869500126 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f212810bb90 0x7f212810e010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:33:10.667 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:10.662+0000 7f2127200640 1 --2- 172.21.15.47:0/869500126 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f21281030e0 0x7f212810b5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:33:10.667 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:10.662+0000 7f2127200640 1 --2- 172.21.15.47:0/869500126 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f21281027c0 0x7f2128102ba0 unknown :-1 s=CLOSED pgs=1121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:33:10.667 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:10.662+0000 7f2127200640 1 -- 172.21.15.47:0/869500126 >> 172.21.15.47:0/869500126 conn(0x7f21280fbfa0 msgr2=0x7f21280fe3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:33:10.667 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:10.662+0000 7f2127200640 1 -- 172.21.15.47:0/869500126 shutdown_connections 2024-03-21T10:33:10.667 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:10.662+0000 7f2127200640 1 -- 172.21.15.47:0/869500126 wait complete. 2024-03-21T10:33:10.667 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:10.663+0000 7f2127200640 1 Processor -- start 2024-03-21T10:33:10.668 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:10.663+0000 7f2127200640 1 -- start start 2024-03-21T10:33:10.668 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:10.664+0000 7f2127200640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f21281027c0 0x7f2128199fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:33:10.668 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:10.664+0000 7f2127200640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f21281030e0 0x7f212819a4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:33:10.668 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:10.664+0000 7f2127200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f212810bb90 0x7f212819e870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:33:10.668 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:10.664+0000 7f2127200640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2128111830 con 0x7f21281027c0 2024-03-21T10:33:10.669 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:10.664+0000 7f2127200640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21281116b0 con 0x7f212810bb90 2024-03-21T10:33:10.669 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:10.664+0000 7f2127200640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21281119b0 con 0x7f21281030e0 2024-03-21T10:33:10.669 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:10.664+0000 7f2125e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f21281027c0 0x7f2128199fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:33:10.669 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:10.664+0000 7f2125e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f21281027c0 0x7f2128199fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:46994/0 (socket says 172.21.15.47:46994) 2024-03-21T10:33:10.669 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:10.664+0000 7f2125e00640 1 -- 172.21.15.47:0/4279974808 learned_addr learned my addr 172.21.15.47:0/4279974808 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:33:10.669 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:10.664+0000 7f2126800640 1 --2- 172.21.15.47:0/4279974808 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f212810bb90 0x7f212819e870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:33:10.670 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:10.664+0000 7f2125400640 1 --2- 172.21.15.47:0/4279974808 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f21281030e0 0x7f212819a4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:33:10.670 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:10.664+0000 7f2125e00640 1 -- 172.21.15.47:0/4279974808 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f21281030e0 msgr2=0x7f212819a4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:33:10.670 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:10.664+0000 7f2125e00640 1 --2- 172.21.15.47:0/4279974808 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f21281030e0 0x7f212819a4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:33:10.670 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:10.664+0000 7f2125e00640 1 -- 172.21.15.47:0/4279974808 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f212810bb90 msgr2=0x7f212819e870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:33:10.670 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:10.664+0000 7f2125e00640 1 --2- 172.21.15.47:0/4279974808 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f212810bb90 0x7f212819e870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:33:10.671 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:10.664+0000 7f2125e00640 1 -- 172.21.15.47:0/4279974808 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f212819efa0 con 0x7f21281027c0 2024-03-21T10:33:10.671 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:10.664+0000 7f2126800640 1 --2- 172.21.15.47:0/4279974808 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f212810bb90 0x7f212819e870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T10:33:10.671 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:10.665+0000 7f2125e00640 1 --2- 172.21.15.47:0/4279974808 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f21281027c0 0x7f2128199fb0 secure :-1 s=READY pgs=1122 cs=0 l=1 rev1=1 crypto rx=0x7f2118000c00 tx=0x7f211802d890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:33:10.671 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:10.665+0000 7f211ea00640 1 -- 172.21.15.47:0/4279974808 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2118043020 con 0x7f21281027c0 2024-03-21T10:33:10.671 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:10.665+0000 7f211ea00640 1 -- 172.21.15.47:0/4279974808 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f211802dd50 con 0x7f21281027c0 2024-03-21T10:33:10.671 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:10.665+0000 7f211ea00640 1 -- 172.21.15.47:0/4279974808 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2118003ab0 con 0x7f21281027c0 2024-03-21T10:33:10.672 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:10.665+0000 7f2127200640 1 -- 172.21.15.47:0/4279974808 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f212819f230 con 0x7f21281027c0 2024-03-21T10:33:10.672 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:10.665+0000 7f2127200640 1 -- 172.21.15.47:0/4279974808 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f21281a6b20 con 0x7f21281027c0 2024-03-21T10:33:10.685 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:10.669+0000 7f212f770740 1 Processor -- start 2024-03-21T10:33:10.685 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:10.669+0000 7f212f770740 1 -- start start 2024-03-21T10:33:10.685 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:10.669+0000 7f212f770740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a586d87290 0x55a586d87670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:33:10.685 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:10.669+0000 7f212f770740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a586d87bb0 0x55a586d90250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:33:10.686 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:10.669+0000 7f212f770740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a586d90790 0x55a586d92b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:33:10.686 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:10.669+0000 7f212f770740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a586cebeb0 con 0x55a586d90790 2024-03-21T10:33:10.686 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:10.669+0000 7f212f770740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a586c94a60 con 0x55a586d87bb0 2024-03-21T10:33:10.686 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:10.669+0000 7f212f770740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a586ce7f50 con 0x55a586d87290 2024-03-21T10:33:10.686 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:10.670+0000 7f2125400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a586d90790 0x55a586d92b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:33:10.686 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:10.670+0000 7f2125400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a586d90790 0x55a586d92b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:46996/0 (socket says 172.21.15.47:46996) 2024-03-21T10:33:10.687 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:10.670+0000 7f2125400640 1 -- 172.21.15.47:0/1785503851 learned_addr learned my addr 172.21.15.47:0/1785503851 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:33:10.687 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:10.670+0000 7f2126800640 1 --2- 172.21.15.47:0/1785503851 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a586d87290 0x55a586d87670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:33:10.687 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:10.670+0000 7f2125400640 1 -- 172.21.15.47:0/1785503851 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a586d87290 msgr2=0x55a586d87670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:33:10.687 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:10.670+0000 7f2125400640 1 --2- 172.21.15.47:0/1785503851 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a586d87290 0x55a586d87670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:33:10.687 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:10.670+0000 7f2125400640 1 -- 172.21.15.47:0/1785503851 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a586d87bb0 msgr2=0x55a586d90250 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-21T10:33:10.687 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:10.670+0000 7f2125400640 1 --2- 172.21.15.47:0/1785503851 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a586d87bb0 0x55a586d90250 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:33:10.687 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:10.670+0000 7f2125400640 1 -- 172.21.15.47:0/1785503851 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a586ac87e0 con 0x55a586d90790 2024-03-21T10:33:10.687 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:10.670+0000 7f2126800640 1 --2- 172.21.15.47:0/1785503851 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a586d87290 0x55a586d87670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T10:33:10.687 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:10.670+0000 7f2125400640 1 --2- 172.21.15.47:0/1785503851 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a586d90790 0x55a586d92b80 secure :-1 s=READY pgs=1123 cs=0 l=1 rev1=1 crypto rx=0x7f2110009f50 tx=0x7f2110010800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cb5dafb039ff39f7 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:33:10.688 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:10.671+0000 7f211ea00640 1 -- 172.21.15.47:0/4279974808 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 27) v1 ==== 137985+0+0 (secure 0 0 0) 0x7f2118046020 con 0x7f21281027c0 2024-03-21T10:33:10.688 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:10.671+0000 7f2117000640 1 -- 172.21.15.47:0/1785503851 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2110016070 con 0x55a586d90790 2024-03-21T10:33:10.688 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:10.671+0000 7f2117000640 1 -- 172.21.15.47:0/1785503851 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f2110012900 con 0x55a586d90790 2024-03-21T10:33:10.688 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:10.671+0000 7f2117000640 1 -- 172.21.15.47:0/1785503851 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21100119e0 con 0x55a586d90790 2024-03-21T10:33:10.688 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:10.671+0000 7f212f770740 1 -- 172.21.15.47:0/1785503851 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a586d90790 msgr2=0x55a586d92b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:33:10.688 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:10.672+0000 7f212f770740 1 --2- 172.21.15.47:0/1785503851 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a586d90790 0x55a586d92b80 secure :-1 s=READY pgs=1123 cs=0 l=1 rev1=1 crypto rx=0x7f2110009f50 tx=0x7f2110010800 comp rx=0 tx=0).stop 2024-03-21T10:33:10.688 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:10.672+0000 7f212f770740 1 -- 172.21.15.47:0/1785503851 shutdown_connections 2024-03-21T10:33:10.688 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:10.672+0000 7f212f770740 1 --2- 172.21.15.47:0/1785503851 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a586d90790 0x55a586d92b80 unknown :-1 s=CLOSED pgs=1123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:33:10.688 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:10.672+0000 7f212f770740 1 --2- 172.21.15.47:0/1785503851 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a586d87bb0 0x55a586d90250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:33:10.688 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:10.672+0000 7f212f770740 1 --2- 172.21.15.47:0/1785503851 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a586d87290 0x55a586d87670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:33:10.688 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:10.672+0000 7f212f770740 1 -- 172.21.15.47:0/1785503851 >> 172.21.15.47:0/1785503851 conn(0x55a586d819c0 msgr2=0x55a586d81da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:33:10.688 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:10.672+0000 7f212f770740 1 -- 172.21.15.47:0/1785503851 shutdown_connections 2024-03-21T10:33:10.688 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:10.672+0000 7f212f770740 1 -- 172.21.15.47:0/1785503851 wait complete. 2024-03-21T10:33:10.688 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:10.672+0000 7f212f770740 1 -- start start 2024-03-21T10:33:10.689 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:10.672+0000 7f211ea00640 1 --2- 172.21.15.47:0/4279974808 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f210c0a3030 0x7f210c0a54f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:33:10.689 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:10.673+0000 7f212f770740 1 Processor -- start 2024-03-21T10:33:10.689 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:10.673+0000 7f211ea00640 1 -- 172.21.15.47:0/4279974808 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(100..100 src has 1..100) v4 ==== 9650+0+0 (secure 0 0 0) 0x7f21180f1cf0 con 0x7f21281027c0 2024-03-21T10:33:10.689 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:10.673+0000 7f2126800640 1 --2- 172.21.15.47:0/4279974808 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f210c0a3030 0x7f210c0a54f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:33:10.689 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:10.673+0000 7f212f770740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a586d87290 0x55a586d90100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:33:10.689 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:10.673+0000 7f212f770740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a586d87bb0 0x55a586d8d490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:33:10.689 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:10.673+0000 7f212f770740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a586d90790 0x55a586d8d9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:33:10.689 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:10.673+0000 7f212f770740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a586d93580 con 0x55a586d90790 2024-03-21T10:33:10.689 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:10.673+0000 7f212f770740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a586ca7eb0 con 0x55a586d87bb0 2024-03-21T10:33:10.689 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:10.673+0000 7f212f770740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a586d93700 con 0x55a586d87290 2024-03-21T10:33:10.689 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:10.673+0000 7f212f770740 10 client.0 _ll_register_callbacks cb 0x55a586c9c150 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T10:33:10.689 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:10.674+0000 7f2125400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a586d87bb0 0x55a586d8d490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:33:10.689 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:10.674+0000 7f2126800640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a586d90790 0x55a586d8d9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:33:10.689 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:10.674+0000 7f2125400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a586d87bb0 0x55a586d8d490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:55514/0 (socket says 172.21.15.47:55514) 2024-03-21T10:33:10.689 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:10.674+0000 7f2125400640 1 -- 172.21.15.47:0/603974995 learned_addr learned my addr 172.21.15.47:0/603974995 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:33:10.690 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:10.674+0000 7f2125e00640 1 --2- 172.21.15.47:0/603974995 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a586d87290 0x55a586d90100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:33:10.690 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:10.674+0000 7f2126800640 1 --2- 172.21.15.47:0/4279974808 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f210c0a3030 0x7f210c0a54f0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f2120005fd0 tx=0x7f2120010370 comp rx=0 tx=0).ready entity=mgr.15687 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:33:10.690 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:10.674+0000 7f2126800640 1 -- 172.21.15.47:0/603974995 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a586d87290 msgr2=0x55a586d90100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:33:10.690 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:10.674+0000 7f2126800640 1 --2- 172.21.15.47:0/603974995 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a586d87290 0x55a586d90100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:33:10.690 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:10.674+0000 7f2126800640 1 -- 172.21.15.47:0/603974995 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a586d87bb0 msgr2=0x55a586d8d490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:33:10.690 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:10.674+0000 7f2126800640 1 --2- 172.21.15.47:0/603974995 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a586d87bb0 0x55a586d8d490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:33:10.690 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:10.674+0000 7f2126800640 1 -- 172.21.15.47:0/603974995 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55a586b28f70 con 0x55a586d90790 2024-03-21T10:33:10.690 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:10.674+0000 7f2125e00640 1 --2- 172.21.15.47:0/603974995 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a586d87290 0x55a586d90100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T10:33:10.690 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:10.674+0000 7f2125400640 1 --2- 172.21.15.47:0/603974995 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a586d87bb0 0x55a586d8d490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T10:33:10.690 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:10.675+0000 7f2126800640 1 --2- 172.21.15.47:0/603974995 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a586d90790 0x55a586d8d9f0 secure :-1 s=READY pgs=1124 cs=0 l=1 rev1=1 crypto rx=0x7f2120013630 tx=0x7f2120015070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=35d7ebf10e274131 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:33:10.690 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:10.675+0000 7f210b400640 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-03-21T10:33:10.690 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:10.675+0000 7f210b400640 1 -- 172.21.15.47:0/603974995 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2120004220 con 0x55a586d90790 2024-03-21T10:33:10.690 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:10.675+0000 7f210b400640 1 -- 172.21.15.47:0/603974995 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(100..100 src has 1..100) v4 ==== 9650+0+0 (secure 0 0 0) 0x7f21200043c0 con 0x55a586d90790 2024-03-21T10:33:10.691 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:10.675+0000 7f212f770740 1 -- 172.21.15.47:0/603974995 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f2110012900 con 0x55a586d90790 2024-03-21T10:33:10.691 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:10.675+0000 7f210b400640 1 -- 172.21.15.47:0/603974995 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f212002f050 con 0x55a586d90790 2024-03-21T10:33:10.691 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:10.675+0000 7f212f770740 10 client.18117 fetch_fsmap learned FSMap version 32 2024-03-21T10:33:10.691 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:10.675+0000 7f212f770740 1 -- 172.21.15.47:0/603974995 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x55a5869c0170 con 0x55a586d90790 2024-03-21T10:33:10.691 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:10.676+0000 7f210b400640 1 -- 172.21.15.47:0/603974995 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f2110012900 con 0x55a586d90790 2024-03-21T10:33:10.691 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:10.676+0000 7f210b400640 1 -- 172.21.15.47:0/603974995 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 32) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f212000f040 con 0x55a586d90790 2024-03-21T10:33:10.691 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:10.676+0000 7f212f770740 10 client.18117 fetch_fsmap finished waiting for FSMap version 32 2024-03-21T10:33:10.691 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:10.676+0000 7f212f770740 10 client.18117 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-21T10:33:10.691 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:10.676+0000 7f212f770740 20 client.18117 populate_metadata read hostname 'smithi047' 2024-03-21T10:33:10.691 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:10.676+0000 7f212f770740 1 --2- 172.21.15.47:0/603974995 >> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] conn(0x55a586da0810 0x55a586da2c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:33:10.691 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:10.676+0000 7f212f770740 4 client.18117 mds_command: new command op to 34535 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T10:33:10.691 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:10.676+0000 7f212f770740 1 -- 172.21.15.47:0/603974995 --> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55a586cf93f0 con 0x55a586da0810 2024-03-21T10:33:10.691 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:10.677+0000 7f2125e00640 1 --2- 172.21.15.47:0/603974995 >> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] conn(0x55a586da0810 0x55a586da2c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:33:10.691 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:10.678+0000 7f2125e00640 1 --2- 172.21.15.47:0/603974995 >> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] conn(0x55a586da0810 0x55a586da2c00 crc :-1 s=READY pgs=187 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fa8424ad08faf41c server_cookie=c3ab9c79e0b0c0d1 in_seq=0 out_seq=0 2024-03-21T10:33:10.691 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:10.678+0000 7f210b400640 10 client.18117 ms_handle_connect on v2:172.21.15.100:6838/4173168440 2024-03-21T10:33:10.692 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:10.679+0000 7f210b400640 1 -- 172.21.15.47:0/603974995 <== mds.0 v2:172.21.15.100:6838/4173168440 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55a586cf93f0 con 0x55a586da0810 2024-03-21T10:33:10.692 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:10.679+0000 7f210b400640 10 client.18117 handle_command_reply: tid=0 2024-03-21T10:33:10.692 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:10.679+0000 7f212f770740 1 client.18117 shutdown 2024-03-21T10:33:10.692 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:10.679+0000 7f212f770740 1 -- 172.21.15.47:0/603974995 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a586d90790 msgr2=0x55a586d8d9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:33:10.692 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:10.679+0000 7f212f770740 1 --2- 172.21.15.47:0/603974995 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a586d90790 0x55a586d8d9f0 secure :-1 s=READY pgs=1124 cs=0 l=1 rev1=1 crypto rx=0x7f2120013630 tx=0x7f2120015070 comp rx=0 tx=0).stop 2024-03-21T10:33:10.692 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:10.679+0000 7f212f770740 1 -- 172.21.15.47:0/603974995 shutdown_connections 2024-03-21T10:33:10.692 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:10.679+0000 7f212f770740 1 --2- 172.21.15.47:0/603974995 >> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] conn(0x55a586da0810 0x55a586da2c00 crc :-1 s=READY pgs=187 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:33:10.692 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:10.679+0000 7f212f770740 1 --2- 172.21.15.47:0/603974995 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a586d90790 0x55a586d8d9f0 unknown :-1 s=CLOSED pgs=1124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:33:10.692 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:10.679+0000 7f212f770740 1 --2- 172.21.15.47:0/603974995 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a586d87bb0 0x55a586d8d490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:33:10.692 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:10.679+0000 7f212f770740 1 --2- 172.21.15.47:0/603974995 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a586d87290 0x55a586d90100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:33:10.692 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:10.679+0000 7f212f770740 1 -- 172.21.15.47:0/603974995 >> 172.21.15.47:0/603974995 conn(0x55a586d819c0 msgr2=0x55a586d83710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:33:10.692 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:10.679+0000 7f210b400640 0 client.18117 ms_handle_reset on v2:172.21.15.100:6838/4173168440 2024-03-21T10:33:10.693 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:10.679+0000 7f212f770740 1 -- 172.21.15.47:0/603974995 shutdown_connections 2024-03-21T10:33:10.693 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:10.679+0000 7f212f770740 1 -- 172.21.15.47:0/603974995 wait complete. 2024-03-21T10:33:10.693 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:10.680+0000 7f212f770740 20 client.18117 trim_cache size 0 max 16384 2024-03-21T10:33:10.701 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:10.702+0000 7f212f770740 1 Processor -- start 2024-03-21T10:33:10.701 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:10.702+0000 7f212f770740 1 -- start start 2024-03-21T10:33:10.702 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:10.702+0000 7f212f770740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a586d87290 0x55a586d88960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:33:10.702 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:10.702+0000 7f212f770740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a586d87bb0 0x55a586d8fbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:33:10.702 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:10.702+0000 7f212f770740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a586d90790 0x55a586d90100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:33:10.702 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:10.702+0000 7f212f770740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21281b0b40 con 0x55a586d87290 2024-03-21T10:33:10.702 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:10.702+0000 7f212f770740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21281b24c0 con 0x55a586d90790 2024-03-21T10:33:10.702 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:10.702+0000 7f212f770740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21281b11a0 con 0x55a586d87bb0 2024-03-21T10:33:10.705 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:10.702+0000 7f2125e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a586d87290 0x55a586d88960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:33:10.705 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:10.703+0000 7f2125e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a586d87290 0x55a586d88960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:47004/0 (socket says 172.21.15.47:47004) 2024-03-21T10:33:10.705 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:10.703+0000 7f2125e00640 1 -- 172.21.15.47:0/249632252 learned_addr learned my addr 172.21.15.47:0/249632252 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:33:10.705 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:10.703+0000 7f2126800640 1 --2- 172.21.15.47:0/249632252 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a586d90790 0x55a586d90100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:33:10.705 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:10.703+0000 7f2125400640 1 --2- 172.21.15.47:0/249632252 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a586d87bb0 0x55a586d8fbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:33:10.705 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:10.703+0000 7f2125e00640 1 -- 172.21.15.47:0/249632252 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a586d87bb0 msgr2=0x55a586d8fbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:33:10.705 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:10.703+0000 7f2125e00640 1 --2- 172.21.15.47:0/249632252 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a586d87bb0 0x55a586d8fbc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:33:10.706 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:10.703+0000 7f2125e00640 1 -- 172.21.15.47:0/249632252 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a586d90790 msgr2=0x55a586d90100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:33:10.706 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:10.703+0000 7f2125e00640 1 --2- 172.21.15.47:0/249632252 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a586d90790 0x55a586d90100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:33:10.706 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:10.703+0000 7f2125e00640 1 -- 172.21.15.47:0/249632252 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a586d879d0 con 0x55a586d87290 2024-03-21T10:33:10.706 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:10.703+0000 7f2126800640 1 --2- 172.21.15.47:0/249632252 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a586d90790 0x55a586d90100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T10:33:10.706 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:10.703+0000 7f2125e00640 1 --2- 172.21.15.47:0/249632252 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a586d87290 0x55a586d88960 secure :-1 s=READY pgs=1125 cs=0 l=1 rev1=1 crypto rx=0x7f21180bda80 tx=0x7f21180f1040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a9243c36ef4c4796 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:33:10.706 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:10.704+0000 7f2116600640 1 -- 172.21.15.47:0/249632252 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21180be480 con 0x55a586d87290 2024-03-21T10:33:10.706 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:10.704+0000 7f2116600640 1 -- 172.21.15.47:0/249632252 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f21180bdd80 con 0x55a586d87290 2024-03-21T10:33:10.706 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:10.704+0000 7f2116600640 1 -- 172.21.15.47:0/249632252 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2118003c50 con 0x55a586d87290 2024-03-21T10:33:10.706 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:10.704+0000 7f212f770740 1 -- 172.21.15.47:0/249632252 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a586d87290 msgr2=0x55a586d88960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:33:10.706 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:10.704+0000 7f212f770740 1 --2- 172.21.15.47:0/249632252 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a586d87290 0x55a586d88960 secure :-1 s=READY pgs=1125 cs=0 l=1 rev1=1 crypto rx=0x7f21180bda80 tx=0x7f21180f1040 comp rx=0 tx=0).stop 2024-03-21T10:33:10.706 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:10.705+0000 7f212f770740 1 -- 172.21.15.47:0/249632252 shutdown_connections 2024-03-21T10:33:10.707 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:10.705+0000 7f212f770740 1 --2- 172.21.15.47:0/249632252 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a586d90790 0x55a586d90100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:33:10.707 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:10.705+0000 7f212f770740 1 --2- 172.21.15.47:0/249632252 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a586d87bb0 0x55a586d8fbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:33:10.707 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:10.705+0000 7f212f770740 1 --2- 172.21.15.47:0/249632252 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a586d87290 0x55a586d88960 unknown :-1 s=CLOSED pgs=1125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:33:10.707 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:10.705+0000 7f212f770740 1 -- 172.21.15.47:0/249632252 >> 172.21.15.47:0/249632252 conn(0x55a586d819c0 msgr2=0x55a586d801f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:33:10.707 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:10.705+0000 7f212f770740 1 -- 172.21.15.47:0/249632252 shutdown_connections 2024-03-21T10:33:10.707 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:10.705+0000 7f212f770740 1 -- 172.21.15.47:0/249632252 wait complete. 2024-03-21T10:33:10.707 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:10.705+0000 7f212f770740 1 -- start start 2024-03-21T10:33:10.707 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:10.706+0000 7f212f770740 1 Processor -- start 2024-03-21T10:33:10.710 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:10.706+0000 7f212f770740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a586d87290 0x55a586d98740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:33:10.710 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:10.706+0000 7f212f770740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a586d87bb0 0x55a586d94ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:33:10.710 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:10.706+0000 7f212f770740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a586d90790 0x55a586d95000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:33:10.711 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:10.706+0000 7f212f770740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21281b1e60 con 0x55a586d87290 2024-03-21T10:33:10.711 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:10.706+0000 7f212f770740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21281b1800 con 0x55a586d87bb0 2024-03-21T10:33:10.711 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:10.706+0000 7f212f770740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21281aed20 con 0x55a586d90790 2024-03-21T10:33:10.711 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:10.706+0000 7f212f770740 10 client.0 _ll_register_callbacks cb 0x7f212000b8b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T10:33:10.711 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:10.706+0000 7f2125e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a586d87290 0x55a586d98740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:33:10.711 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:10.707+0000 7f2125e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a586d87290 0x55a586d98740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:47014/0 (socket says 172.21.15.47:47014) 2024-03-21T10:33:10.712 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:10.707+0000 7f2125e00640 1 -- 172.21.15.47:0/3011381591 learned_addr learned my addr 172.21.15.47:0/3011381591 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:33:10.712 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:10.707+0000 7f2125400640 1 --2- 172.21.15.47:0/3011381591 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a586d87bb0 0x55a586d94ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:33:10.712 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:10.707+0000 7f2125e00640 1 -- 172.21.15.47:0/3011381591 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a586d90790 msgr2=0x55a586d95000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:33:10.712 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:10.707+0000 7f2125e00640 1 --2- 172.21.15.47:0/3011381591 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a586d90790 0x55a586d95000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:33:10.712 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:10.707+0000 7f2125e00640 1 -- 172.21.15.47:0/3011381591 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a586d87bb0 msgr2=0x55a586d94ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:33:10.712 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:10.707+0000 7f2125e00640 1 --2- 172.21.15.47:0/3011381591 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a586d87bb0 0x55a586d94ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:33:10.713 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:10.707+0000 7f2125e00640 1 -- 172.21.15.47:0/3011381591 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55a586d957e0 con 0x55a586d87290 2024-03-21T10:33:10.713 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:10.707+0000 7f2125e00640 1 --2- 172.21.15.47:0/3011381591 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a586d87290 0x55a586d98740 secure :-1 s=READY pgs=1126 cs=0 l=1 rev1=1 crypto rx=0x7f21180ef5d0 tx=0x7f21180ef2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4204480fc61e0d4b server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:33:10.713 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:10.707+0000 7f210b400640 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-03-21T10:33:10.713 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:10.708+0000 7f210b400640 1 -- 172.21.15.47:0/3011381591 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21180c6790 con 0x55a586d87290 2024-03-21T10:33:10.713 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:10.708+0000 7f210b400640 1 -- 172.21.15.47:0/3011381591 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(100..100 src has 1..100) v4 ==== 9650+0+0 (secure 0 0 0) 0x7f2118003c50 con 0x55a586d87290 2024-03-21T10:33:10.713 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:10.708+0000 7f212f770740 1 -- 172.21.15.47:0/3011381591 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f21180bdd80 con 0x55a586d87290 2024-03-21T10:33:10.713 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:10.708+0000 7f210b400640 1 -- 172.21.15.47:0/3011381591 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2118003e50 con 0x55a586d87290 2024-03-21T10:33:10.713 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:10.708+0000 7f212f770740 10 client.18123 fetch_fsmap learned FSMap version 32 2024-03-21T10:33:10.714 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:10.708+0000 7f212f770740 1 -- 172.21.15.47:0/3011381591 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x55a586dfd0b0 con 0x55a586d87290 2024-03-21T10:33:10.714 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:10.708+0000 7f210b400640 1 -- 172.21.15.47:0/3011381591 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f21180bdd80 con 0x55a586d87290 2024-03-21T10:33:10.714 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:10.708+0000 7f210b400640 1 -- 172.21.15.47:0/3011381591 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 32) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f211809cba0 con 0x55a586d87290 2024-03-21T10:33:10.714 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:10.708+0000 7f212f770740 10 client.18123 fetch_fsmap finished waiting for FSMap version 32 2024-03-21T10:33:10.714 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:10.708+0000 7f212f770740 10 client.18123 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-21T10:33:10.714 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:10.708+0000 7f212f770740 20 client.18123 populate_metadata read hostname 'smithi047' 2024-03-21T10:33:10.714 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:10.708+0000 7f212f770740 1 --2- 172.21.15.47:0/3011381591 >> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] conn(0x55a586dfd720 0x55a586dffb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:33:10.714 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:10.708+0000 7f212f770740 4 client.18123 mds_command: new command op to 34535 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-03-21T10:33:10.714 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:10.708+0000 7f212f770740 1 -- 172.21.15.47:0/3011381591 --> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x55a586cf3740 con 0x55a586dfd720 2024-03-21T10:33:10.714 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:10.709+0000 7f2125400640 1 --2- 172.21.15.47:0/3011381591 >> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] conn(0x55a586dfd720 0x55a586dffb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:33:10.714 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:10.710+0000 7f2125400640 1 --2- 172.21.15.47:0/3011381591 >> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] conn(0x55a586dfd720 0x55a586dffb40 crc :-1 s=READY pgs=188 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6c7f8f43a330ab8c server_cookie=cbe3a227ebcab931 in_seq=0 out_seq=0 2024-03-21T10:33:10.714 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:10.710+0000 7f210b400640 10 client.18123 ms_handle_connect on v2:172.21.15.100:6838/4173168440 2024-03-21T10:33:10.714 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:10.711+0000 7f210b400640 1 -- 172.21.15.47:0/3011381591 <== mds.0 v2:172.21.15.100:6838/4173168440 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x55a586cf3740 con 0x55a586dfd720 2024-03-21T10:33:10.715 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:10.711+0000 7f210b400640 10 client.18123 handle_command_reply: tid=0 2024-03-21T10:33:10.715 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:10.711+0000 7f212f770740 1 client.18123 shutdown 2024-03-21T10:33:10.715 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:33:10 smithi047 ceph-mon[31547]: osdmap e100: 12 total, 12 up, 12 in 2024-03-21T10:33:10.716 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:10.711+0000 7f212f770740 1 -- 172.21.15.47:0/3011381591 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a586d87290 msgr2=0x55a586d98740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:33:10.716 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:10.711+0000 7f212f770740 1 --2- 172.21.15.47:0/3011381591 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a586d87290 0x55a586d98740 secure :-1 s=READY pgs=1126 cs=0 l=1 rev1=1 crypto rx=0x7f21180ef5d0 tx=0x7f21180ef2d0 comp rx=0 tx=0).stop 2024-03-21T10:33:10.716 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:10.712+0000 7f212f770740 1 -- 172.21.15.47:0/3011381591 shutdown_connections 2024-03-21T10:33:10.716 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:10.712+0000 7f212f770740 1 --2- 172.21.15.47:0/3011381591 >> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] conn(0x55a586dfd720 0x55a586dffb40 crc :-1 s=READY pgs=188 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:33:10.716 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:10.712+0000 7f212f770740 1 --2- 172.21.15.47:0/3011381591 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a586d90790 0x55a586d95000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:33:10.716 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:10.712+0000 7f212f770740 1 --2- 172.21.15.47:0/3011381591 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a586d87bb0 0x55a586d94ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:33:10.716 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:10.712+0000 7f212f770740 1 --2- 172.21.15.47:0/3011381591 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a586d87290 0x55a586d98740 unknown :-1 s=CLOSED pgs=1126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:33:10.716 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:10.712+0000 7f212f770740 1 -- 172.21.15.47:0/3011381591 >> 172.21.15.47:0/3011381591 conn(0x55a586d819c0 msgr2=0x55a586d887e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:33:10.716 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:10.712+0000 7f210b400640 0 client.18123 ms_handle_reset on v2:172.21.15.100:6838/4173168440 2024-03-21T10:33:10.716 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:10.712+0000 7f212f770740 1 -- 172.21.15.47:0/3011381591 shutdown_connections 2024-03-21T10:33:10.717 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:10.712+0000 7f212f770740 1 -- 172.21.15.47:0/3011381591 wait complete. 2024-03-21T10:33:10.717 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:10.712+0000 7f212f770740 20 client.18123 trim_cache size 0 max 16384 2024-03-21T10:33:10.717 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-03-21T10:33:10.717 INFO:teuthology.orchestra.run.smithi047.stdout: "return_code": 0, 2024-03-21T10:33:10.717 INFO:teuthology.orchestra.run.smithi047.stdout: "scrub_tag": "65135dcd-a63f-4d1b-9f22-6eecde26fa02", 2024-03-21T10:33:10.717 INFO:teuthology.orchestra.run.smithi047.stdout: "mode": "asynchronous" 2024-03-21T10:33:10.717 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-03-21T10:33:10.718 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:10.713+0000 7f2127200640 1 -- 172.21.15.47:0/4279974808 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f210c0a3030 msgr2=0x7f210c0a54f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:33:10.718 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:10.713+0000 7f2127200640 1 --2- 172.21.15.47:0/4279974808 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f210c0a3030 0x7f210c0a54f0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f2120005fd0 tx=0x7f2120010370 comp rx=0 tx=0).stop 2024-03-21T10:33:10.718 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:10.713+0000 7f2127200640 1 -- 172.21.15.47:0/4279974808 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f21281027c0 msgr2=0x7f2128199fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:33:10.718 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:10.713+0000 7f2127200640 1 --2- 172.21.15.47:0/4279974808 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f21281027c0 0x7f2128199fb0 secure :-1 s=READY pgs=1122 cs=0 l=1 rev1=1 crypto rx=0x7f2118000c00 tx=0x7f211802d890 comp rx=0 tx=0).stop 2024-03-21T10:33:10.718 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:10.713+0000 7f2127200640 1 -- 172.21.15.47:0/4279974808 shutdown_connections 2024-03-21T10:33:10.718 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:10.713+0000 7f2127200640 1 --2- 172.21.15.47:0/4279974808 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f210c0a3030 0x7f210c0a54f0 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:33:10.718 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:10.713+0000 7f2127200640 1 --2- 172.21.15.47:0/4279974808 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f212810bb90 0x7f212819e870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:33:10.718 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:10.713+0000 7f2127200640 1 --2- 172.21.15.47:0/4279974808 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f21281030e0 0x7f212819a4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:33:10.718 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:10.713+0000 7f2127200640 1 --2- 172.21.15.47:0/4279974808 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f21281027c0 0x7f2128199fb0 unknown :-1 s=CLOSED pgs=1122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:33:10.719 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:10.713+0000 7f2127200640 1 -- 172.21.15.47:0/4279974808 >> 172.21.15.47:0/4279974808 conn(0x7f21280fbfa0 msgr2=0x7f212810ce90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:33:10.719 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:10.713+0000 7f2127200640 1 -- 172.21.15.47:0/4279974808 shutdown_connections 2024-03-21T10:33:10.719 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:10.713+0000 7f2127200640 1 -- 172.21.15.47:0/4279974808 wait complete. 2024-03-21T10:33:10.726 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:33:10 smithi100 ceph-mon[35193]: osdmap e100: 12 total, 12 up, 12 in 2024-03-21T10:33:10.730 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-21T10:33:10.731 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-21T10:33:10.737 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:33:10 smithi157 ceph-mon[35581]: osdmap e100: 12 total, 12 up, 12 in 2024-03-21T10:33:10.959 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:10.959+0000 7fca32c00640 1 -- 172.21.15.47:0/1186383671 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fca2c103200 msgr2=0x7fca2c10b6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:33:10.959 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:10.959+0000 7fca32c00640 1 --2- 172.21.15.47:0/1186383671 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fca2c103200 0x7fca2c10b6e0 secure :-1 s=READY pgs=1127 cs=0 l=1 rev1=1 crypto rx=0x7fca18009900 tx=0x7fca1802bcd0 comp rx=0 tx=0).stop 2024-03-21T10:33:10.959 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:10.959+0000 7fca32c00640 1 -- 172.21.15.47:0/1186383671 shutdown_connections 2024-03-21T10:33:10.959 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:10.959+0000 7fca32c00640 1 --2- 172.21.15.47:0/1186383671 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fca2c10bcb0 0x7fca2c10e0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:33:10.959 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:10.959+0000 7fca32c00640 1 --2- 172.21.15.47:0/1186383671 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fca2c103200 0x7fca2c10b6e0 unknown :-1 s=CLOSED pgs=1127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:33:10.959 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:10.959+0000 7fca32c00640 1 --2- 172.21.15.47:0/1186383671 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fca2c1028e0 0x7fca2c102cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:33:10.960 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:10.959+0000 7fca32c00640 1 -- 172.21.15.47:0/1186383671 >> 172.21.15.47:0/1186383671 conn(0x7fca2c076920 msgr2=0x7fca2c076d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:33:10.960 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:10.959+0000 7fca32c00640 1 -- 172.21.15.47:0/1186383671 shutdown_connections 2024-03-21T10:33:10.960 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:10.959+0000 7fca32c00640 1 -- 172.21.15.47:0/1186383671 wait complete. 2024-03-21T10:33:10.962 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:10.961+0000 7fca32c00640 1 Processor -- start 2024-03-21T10:33:10.962 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:10.961+0000 7fca32c00640 1 -- start start 2024-03-21T10:33:10.962 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:10.961+0000 7fca32c00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fca2c1028e0 0x7fca2c19a070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:33:10.963 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:10.961+0000 7fca32c00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fca2c103200 0x7fca2c19a5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:33:10.963 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:10.961+0000 7fca32c00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fca2c10bcb0 0x7fca2c19e930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:33:10.963 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:10.961+0000 7fca32c00640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca2c1118f0 con 0x7fca2c1028e0 2024-03-21T10:33:10.963 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:10.961+0000 7fca32c00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca2c111770 con 0x7fca2c103200 2024-03-21T10:33:10.963 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:10.961+0000 7fca32c00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca2c111a70 con 0x7fca2c10bcb0 2024-03-21T10:33:10.963 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:10.962+0000 7fca2be00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fca2c1028e0 0x7fca2c19a070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:33:10.963 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:10.962+0000 7fca2be00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fca2c1028e0 0x7fca2c19a070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:47026/0 (socket says 172.21.15.47:47026) 2024-03-21T10:33:10.963 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:10.962+0000 7fca2be00640 1 -- 172.21.15.47:0/3042615136 learned_addr learned my addr 172.21.15.47:0/3042615136 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:33:10.963 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:10.962+0000 7fca2b400640 1 --2- 172.21.15.47:0/3042615136 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fca2c103200 0x7fca2c19a5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:33:10.963 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:10.962+0000 7fca30c00640 1 --2- 172.21.15.47:0/3042615136 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fca2c10bcb0 0x7fca2c19e930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:33:10.963 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:10.962+0000 7fca2be00640 1 -- 172.21.15.47:0/3042615136 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fca2c10bcb0 msgr2=0x7fca2c19e930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:33:10.963 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:10.962+0000 7fca2be00640 1 --2- 172.21.15.47:0/3042615136 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fca2c10bcb0 0x7fca2c19e930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:33:10.963 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:10.962+0000 7fca2be00640 1 -- 172.21.15.47:0/3042615136 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fca2c103200 msgr2=0x7fca2c19a5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:33:10.963 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:10.962+0000 7fca2be00640 1 --2- 172.21.15.47:0/3042615136 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fca2c103200 0x7fca2c19a5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:33:10.963 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:10.962+0000 7fca2be00640 1 -- 172.21.15.47:0/3042615136 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca2c19f060 con 0x7fca2c1028e0 2024-03-21T10:33:10.964 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:10.963+0000 7fca2be00640 1 --2- 172.21.15.47:0/3042615136 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fca2c1028e0 0x7fca2c19a070 secure :-1 s=READY pgs=1128 cs=0 l=1 rev1=1 crypto rx=0x7fca1c00d770 tx=0x7fca1c00dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:33:10.964 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:10.963+0000 7fca28c00640 1 -- 172.21.15.47:0/3042615136 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca1c017070 con 0x7fca2c1028e0 2024-03-21T10:33:10.964 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:10.963+0000 7fca32c00640 1 -- 172.21.15.47:0/3042615136 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fca2c19f350 con 0x7fca2c1028e0 2024-03-21T10:33:10.964 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:10.963+0000 7fca28c00640 1 -- 172.21.15.47:0/3042615136 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fca1c0054d0 con 0x7fca2c1028e0 2024-03-21T10:33:10.964 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:10.963+0000 7fca28c00640 1 -- 172.21.15.47:0/3042615136 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca1c004cf0 con 0x7fca2c1028e0 2024-03-21T10:33:10.964 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:10.963+0000 7fca32c00640 1 -- 172.21.15.47:0/3042615136 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fca2c1a6be0 con 0x7fca2c1028e0 2024-03-21T10:33:10.966 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:10.966+0000 7fca35998740 1 Processor -- start 2024-03-21T10:33:10.967 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:10.966+0000 7fca35998740 1 -- start start 2024-03-21T10:33:10.967 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:10.967+0000 7fca28c00640 1 -- 172.21.15.47:0/3042615136 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 27) v1 ==== 137985+0+0 (secure 0 0 0) 0x7fca1c004e90 con 0x7fca2c1028e0 2024-03-21T10:33:10.967 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:10.967+0000 7fca35998740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55eeb9a71c00 0x55eeb9a70950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:33:10.967 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:10.967+0000 7fca35998740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55eeb9a70f20 0x55eeb9a713a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:33:10.967 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:10.967+0000 7fca35998740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55eeb9a7a290 0x55eeb9a7c680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:33:10.967 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:10.967+0000 7fca35998740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eeb99d5250 con 0x55eeb9a7a290 2024-03-21T10:33:10.967 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:10.967+0000 7fca35998740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eeb997da30 con 0x55eeb9a71c00 2024-03-21T10:33:10.967 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:10.967+0000 7fca35998740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eeb99d12f0 con 0x55eeb9a70f20 2024-03-21T10:33:10.967 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:10.967+0000 7fca2b400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55eeb9a7a290 0x55eeb9a7c680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:33:10.967 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:10.967+0000 7fca2b400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55eeb9a7a290 0x55eeb9a7c680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:47036/0 (socket says 172.21.15.47:47036) 2024-03-21T10:33:10.967 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:10.967+0000 7fca2b400640 1 -- 172.21.15.47:0/3127076931 learned_addr learned my addr 172.21.15.47:0/3127076931 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:33:10.967 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:10.967+0000 7fca30c00640 1 --2- 172.21.15.47:0/3127076931 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55eeb9a71c00 0x55eeb9a70950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:33:10.967 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:10.967+0000 7fca2be00640 1 --2- 172.21.15.47:0/3127076931 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55eeb9a70f20 0x55eeb9a713a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:33:10.969 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:10.968+0000 7fca2b400640 1 -- 172.21.15.47:0/3127076931 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55eeb9a70f20 msgr2=0x55eeb9a713a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:33:10.969 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:10.968+0000 7fca2b400640 1 --2- 172.21.15.47:0/3127076931 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55eeb9a70f20 0x55eeb9a713a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:33:10.969 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:10.968+0000 7fca2b400640 1 -- 172.21.15.47:0/3127076931 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55eeb9a71c00 msgr2=0x55eeb9a70950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:33:10.969 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:10.968+0000 7fca2b400640 1 --2- 172.21.15.47:0/3127076931 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55eeb9a71c00 0x55eeb9a70950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:33:10.970 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:10.968+0000 7fca2b400640 1 -- 172.21.15.47:0/3127076931 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55eeb9968690 con 0x55eeb9a7a290 2024-03-21T10:33:10.970 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:10.968+0000 7fca2b400640 1 --2- 172.21.15.47:0/3127076931 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55eeb9a7a290 0x55eeb9a7c680 secure :-1 s=READY pgs=1129 cs=0 l=1 rev1=1 crypto rx=0x7fca18009900 tx=0x7fca1802c760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e2d26f8abfebd855 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:33:10.970 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:10.968+0000 7fca28c00640 1 --2- 172.21.15.47:0/3042615136 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7fca100a3090 0x7fca100a5550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:33:10.970 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:10.968+0000 7fca28c00640 1 -- 172.21.15.47:0/3042615136 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(100..100 src has 1..100) v4 ==== 9650+0+0 (secure 0 0 0) 0x7fca1c0cf6c0 con 0x7fca2c1028e0 2024-03-21T10:33:10.970 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:10.969+0000 7fca21200640 1 -- 172.21.15.47:0/3127076931 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca18004180 con 0x55eeb9a7a290 2024-03-21T10:33:10.970 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:10.969+0000 7fca30c00640 1 --2- 172.21.15.47:0/3042615136 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7fca100a3090 0x7fca100a5550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:33:10.970 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:10.969+0000 7fca21200640 1 -- 172.21.15.47:0/3127076931 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fca180054f0 con 0x55eeb9a7a290 2024-03-21T10:33:10.970 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:10.969+0000 7fca21200640 1 -- 172.21.15.47:0/3127076931 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca180046c0 con 0x55eeb9a7a290 2024-03-21T10:33:10.970 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:10.970+0000 7fca35998740 1 -- 172.21.15.47:0/3127076931 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55eeb9a7a290 msgr2=0x55eeb9a7c680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:33:10.970 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:10.970+0000 7fca35998740 1 --2- 172.21.15.47:0/3127076931 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55eeb9a7a290 0x55eeb9a7c680 secure :-1 s=READY pgs=1129 cs=0 l=1 rev1=1 crypto rx=0x7fca18009900 tx=0x7fca1802c760 comp rx=0 tx=0).stop 2024-03-21T10:33:10.970 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:10.970+0000 7fca35998740 1 -- 172.21.15.47:0/3127076931 shutdown_connections 2024-03-21T10:33:10.970 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:10.970+0000 7fca35998740 1 --2- 172.21.15.47:0/3127076931 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55eeb9a7a290 0x55eeb9a7c680 unknown :-1 s=CLOSED pgs=1129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:33:10.970 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:10.970+0000 7fca35998740 1 --2- 172.21.15.47:0/3127076931 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55eeb9a70f20 0x55eeb9a713a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:33:10.970 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:10.970+0000 7fca35998740 1 --2- 172.21.15.47:0/3127076931 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55eeb9a71c00 0x55eeb9a70950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:33:10.971 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:10.970+0000 7fca35998740 1 -- 172.21.15.47:0/3127076931 >> 172.21.15.47:0/3127076931 conn(0x55eeb9a38200 msgr2=0x55eeb9a6b970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:33:10.971 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:10.970+0000 7fca30c00640 1 --2- 172.21.15.47:0/3042615136 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7fca100a3090 0x7fca100a5550 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fca2c06ad50 tx=0x7fca240096e0 comp rx=0 tx=0).ready entity=mgr.15687 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:33:10.971 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:10.970+0000 7fca35998740 1 -- 172.21.15.47:0/3127076931 shutdown_connections 2024-03-21T10:33:10.971 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:10.970+0000 7fca35998740 1 -- 172.21.15.47:0/3127076931 wait complete. 2024-03-21T10:33:10.974 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:10.970+0000 7fca35998740 1 -- start start 2024-03-21T10:33:10.974 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:10.971+0000 7fca35998740 1 Processor -- start 2024-03-21T10:33:10.974 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:10.971+0000 7fca35998740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55eeb9a70f20 0x55eeb9a798c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:33:10.974 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:10.971+0000 7fca35998740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55eeb9a71c00 0x55eeb9a76cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:33:10.974 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:10.971+0000 7fca35998740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55eeb9a7a290 0x55eeb9a77200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:33:10.974 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:10.971+0000 7fca35998740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eeb9a7d0a0 con 0x55eeb9a70f20 2024-03-21T10:33:10.974 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:10.971+0000 7fca35998740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eeb9990e80 con 0x55eeb9a7a290 2024-03-21T10:33:10.975 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:10.971+0000 7fca35998740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eeb9a7d220 con 0x55eeb9a71c00 2024-03-21T10:33:10.975 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:10.971+0000 7fca35998740 10 client.0 _ll_register_callbacks cb 0x55eeb99a3130 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T10:33:10.975 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:10.971+0000 7fca2be00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55eeb9a70f20 0x55eeb9a798c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:33:10.975 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:10.972+0000 7fca2be00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55eeb9a70f20 0x55eeb9a798c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:47048/0 (socket says 172.21.15.47:47048) 2024-03-21T10:33:10.975 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:10.972+0000 7fca30c00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55eeb9a7a290 0x55eeb9a77200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:33:10.975 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:10.972+0000 7fca2be00640 1 -- 172.21.15.47:0/1380468749 learned_addr learned my addr 172.21.15.47:0/1380468749 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:33:10.975 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:10.972+0000 7fca2b400640 1 --2- 172.21.15.47:0/1380468749 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55eeb9a71c00 0x55eeb9a76cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:33:10.975 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:10.972+0000 7fca2be00640 1 -- 172.21.15.47:0/1380468749 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55eeb9a71c00 msgr2=0x55eeb9a76cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:33:10.975 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:10.972+0000 7fca2be00640 1 --2- 172.21.15.47:0/1380468749 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55eeb9a71c00 0x55eeb9a76cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:33:10.975 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:10.972+0000 7fca2be00640 1 -- 172.21.15.47:0/1380468749 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55eeb9a7a290 msgr2=0x55eeb9a77200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:33:10.975 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:10.972+0000 7fca2be00640 1 --2- 172.21.15.47:0/1380468749 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55eeb9a7a290 0x55eeb9a77200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:33:10.975 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:10.972+0000 7fca2be00640 1 -- 172.21.15.47:0/1380468749 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55eeb97b37d0 con 0x55eeb9a70f20 2024-03-21T10:33:10.975 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:10.972+0000 7fca2be00640 1 --2- 172.21.15.47:0/1380468749 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55eeb9a70f20 0x55eeb9a798c0 secure :-1 s=READY pgs=1130 cs=0 l=1 rev1=1 crypto rx=0x7fca1c0cca00 tx=0x7fca1c09b410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=507d1a68c8aeb683 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:33:10.975 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:10.972+0000 7fca15600640 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-03-21T10:33:10.976 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:10.972+0000 7fca15600640 1 -- 172.21.15.47:0/1380468749 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca1c020070 con 0x55eeb9a70f20 2024-03-21T10:33:10.976 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:10.972+0000 7fca15600640 1 -- 172.21.15.47:0/1380468749 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(100..100 src has 1..100) v4 ==== 9650+0+0 (secure 0 0 0) 0x7fca1c084b80 con 0x55eeb9a70f20 2024-03-21T10:33:10.976 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:10.973+0000 7fca35998740 1 -- 172.21.15.47:0/1380468749 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fca180054f0 con 0x55eeb9a70f20 2024-03-21T10:33:10.976 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:10.973+0000 7fca15600640 1 -- 172.21.15.47:0/1380468749 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7fca1c005180 con 0x55eeb9a70f20 2024-03-21T10:33:10.976 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:10.973+0000 7fca35998740 10 client.18135 fetch_fsmap learned FSMap version 32 2024-03-21T10:33:10.976 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:10.973+0000 7fca35998740 1 -- 172.21.15.47:0/1380468749 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x55eeb96e93f0 con 0x55eeb9a70f20 2024-03-21T10:33:10.976 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:10.973+0000 7fca15600640 1 -- 172.21.15.47:0/1380468749 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fca180054f0 con 0x55eeb9a70f20 2024-03-21T10:33:10.976 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:10.973+0000 7fca15600640 1 -- 172.21.15.47:0/1380468749 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 32) v1 ==== 7942+0+0 (secure 0 0 0) 0x7fca1c09aa90 con 0x55eeb9a70f20 2024-03-21T10:33:10.976 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:10.973+0000 7fca35998740 10 client.18135 fetch_fsmap finished waiting for FSMap version 32 2024-03-21T10:33:10.976 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:10.973+0000 7fca35998740 10 client.18135 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-21T10:33:10.976 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:10.973+0000 7fca35998740 20 client.18135 populate_metadata read hostname 'smithi047' 2024-03-21T10:33:10.976 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:10.973+0000 7fca35998740 1 --2- 172.21.15.47:0/1380468749 >> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] conn(0x55eeb9a8a040 0x55eeb9a8c450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:33:10.976 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:10.973+0000 7fca35998740 4 client.18135 mds_command: new command op to 34535 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T10:33:10.976 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:10.973+0000 7fca35998740 1 -- 172.21.15.47:0/1380468749 --> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55eeb99e2790 con 0x55eeb9a8a040 2024-03-21T10:33:10.976 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:10.974+0000 7fca2b400640 1 --2- 172.21.15.47:0/1380468749 >> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] conn(0x55eeb9a8a040 0x55eeb9a8c450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:33:10.977 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:10.974+0000 7fca2b400640 1 --2- 172.21.15.47:0/1380468749 >> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] conn(0x55eeb9a8a040 0x55eeb9a8c450 crc :-1 s=READY pgs=189 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=52a89fe1ae8ff711 server_cookie=2d25008bfbfa7e13 in_seq=0 out_seq=0 2024-03-21T10:33:10.977 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:10.974+0000 7fca15600640 10 client.18135 ms_handle_connect on v2:172.21.15.100:6838/4173168440 2024-03-21T10:33:10.978 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:10.975+0000 7fca15600640 1 -- 172.21.15.47:0/1380468749 <== mds.0 v2:172.21.15.100:6838/4173168440 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55eeb99e2790 con 0x55eeb9a8a040 2024-03-21T10:33:10.978 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:10.975+0000 7fca15600640 10 client.18135 handle_command_reply: tid=0 2024-03-21T10:33:10.978 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:10.975+0000 7fca35998740 1 client.18135 shutdown 2024-03-21T10:33:10.978 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:10.975+0000 7fca35998740 1 -- 172.21.15.47:0/1380468749 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55eeb9a70f20 msgr2=0x55eeb9a798c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:33:10.978 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:10.975+0000 7fca35998740 1 --2- 172.21.15.47:0/1380468749 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55eeb9a70f20 0x55eeb9a798c0 secure :-1 s=READY pgs=1130 cs=0 l=1 rev1=1 crypto rx=0x7fca1c0cca00 tx=0x7fca1c09b410 comp rx=0 tx=0).stop 2024-03-21T10:33:10.978 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:10.976+0000 7fca35998740 1 -- 172.21.15.47:0/1380468749 shutdown_connections 2024-03-21T10:33:10.978 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:10.976+0000 7fca35998740 1 --2- 172.21.15.47:0/1380468749 >> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] conn(0x55eeb9a8a040 0x55eeb9a8c450 crc :-1 s=READY pgs=189 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:33:10.978 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:10.976+0000 7fca35998740 1 --2- 172.21.15.47:0/1380468749 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55eeb9a7a290 0x55eeb9a77200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:33:10.978 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:10.976+0000 7fca35998740 1 --2- 172.21.15.47:0/1380468749 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55eeb9a71c00 0x55eeb9a76cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:33:10.978 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:10.976+0000 7fca35998740 1 --2- 172.21.15.47:0/1380468749 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55eeb9a70f20 0x55eeb9a798c0 unknown :-1 s=CLOSED pgs=1130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:33:10.978 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:10.976+0000 7fca15600640 0 client.18135 ms_handle_reset on v2:172.21.15.100:6838/4173168440 2024-03-21T10:33:10.978 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:10.976+0000 7fca35998740 1 -- 172.21.15.47:0/1380468749 >> 172.21.15.47:0/1380468749 conn(0x55eeb9a38200 msgr2=0x55eeb9a6d6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:33:10.978 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:10.976+0000 7fca35998740 1 -- 172.21.15.47:0/1380468749 shutdown_connections 2024-03-21T10:33:10.978 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:10.976+0000 7fca35998740 1 -- 172.21.15.47:0/1380468749 wait complete. 2024-03-21T10:33:10.979 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:10.976+0000 7fca35998740 20 client.18135 trim_cache size 0 max 16384 2024-03-21T10:33:10.993 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:10.994+0000 7fca35998740 1 Processor -- start 2024-03-21T10:33:10.993 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:10.994+0000 7fca35998740 1 -- start start 2024-03-21T10:33:10.993 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:10.994+0000 7fca35998740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55eeb9a70f20 0x55eeb9a793f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:33:10.994 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:10.994+0000 7fca35998740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55eeb9a71c00 0x55eeb9a7b680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:33:10.994 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:10.994+0000 7fca35998740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55eeb9a7a290 0x55eeb9a7bbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:33:10.994 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:10.994+0000 7fca35998740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca2c1b0cd0 con 0x55eeb9a70f20 2024-03-21T10:33:10.994 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:10.994+0000 7fca35998740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca2c1b2650 con 0x55eeb9a71c00 2024-03-21T10:33:10.994 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:10.994+0000 7fca35998740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca2c1b1330 con 0x55eeb9a7a290 2024-03-21T10:33:10.996 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:10.994+0000 7fca2be00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55eeb9a70f20 0x55eeb9a793f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:33:10.996 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:10.994+0000 7fca2be00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55eeb9a70f20 0x55eeb9a793f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:47064/0 (socket says 172.21.15.47:47064) 2024-03-21T10:33:10.996 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:10.994+0000 7fca2be00640 1 -- 172.21.15.47:0/242051766 learned_addr learned my addr 172.21.15.47:0/242051766 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:33:10.996 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:10.995+0000 7fca2b400640 1 --2- 172.21.15.47:0/242051766 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55eeb9a71c00 0x55eeb9a7b680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:33:10.996 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:10.995+0000 7fca30c00640 1 --2- 172.21.15.47:0/242051766 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55eeb9a7a290 0x55eeb9a7bbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:33:10.996 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:10.995+0000 7fca2be00640 1 -- 172.21.15.47:0/242051766 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55eeb9a7a290 msgr2=0x55eeb9a7bbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:33:10.996 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:10.995+0000 7fca2be00640 1 --2- 172.21.15.47:0/242051766 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55eeb9a7a290 0x55eeb9a7bbc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:33:10.996 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:10.995+0000 7fca2be00640 1 -- 172.21.15.47:0/242051766 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55eeb9a71c00 msgr2=0x55eeb9a7b680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:33:10.996 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:10.995+0000 7fca2be00640 1 --2- 172.21.15.47:0/242051766 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55eeb9a71c00 0x55eeb9a7b680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:33:10.997 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:10.995+0000 7fca2be00640 1 -- 172.21.15.47:0/242051766 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55eeb96a90f0 con 0x55eeb9a70f20 2024-03-21T10:33:10.997 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:10.995+0000 7fca2be00640 1 --2- 172.21.15.47:0/242051766 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55eeb9a70f20 0x55eeb9a793f0 secure :-1 s=READY pgs=1131 cs=0 l=1 rev1=1 crypto rx=0x7fca1c09b5d0 tx=0x7fca1c005bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ce5d6ec8f6b64545 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:33:10.997 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:10.995+0000 7fca17e00640 1 -- 172.21.15.47:0/242051766 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca1c020070 con 0x55eeb9a70f20 2024-03-21T10:33:10.997 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:10.996+0000 7fca17e00640 1 -- 172.21.15.47:0/242051766 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fca1c09bd40 con 0x55eeb9a70f20 2024-03-21T10:33:10.997 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:10.996+0000 7fca17e00640 1 -- 172.21.15.47:0/242051766 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca1c0ccc20 con 0x55eeb9a70f20 2024-03-21T10:33:10.997 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:10.996+0000 7fca35998740 1 -- 172.21.15.47:0/242051766 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55eeb9a70f20 msgr2=0x55eeb9a793f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:33:10.997 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:10.996+0000 7fca35998740 1 --2- 172.21.15.47:0/242051766 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55eeb9a70f20 0x55eeb9a793f0 secure :-1 s=READY pgs=1131 cs=0 l=1 rev1=1 crypto rx=0x7fca1c09b5d0 tx=0x7fca1c005bb0 comp rx=0 tx=0).stop 2024-03-21T10:33:10.997 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:10.996+0000 7fca35998740 1 -- 172.21.15.47:0/242051766 shutdown_connections 2024-03-21T10:33:10.997 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:10.996+0000 7fca35998740 1 --2- 172.21.15.47:0/242051766 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55eeb9a7a290 0x55eeb9a7bbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:33:10.997 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:10.996+0000 7fca35998740 1 --2- 172.21.15.47:0/242051766 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55eeb9a71c00 0x55eeb9a7b680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:33:10.997 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:10.996+0000 7fca35998740 1 --2- 172.21.15.47:0/242051766 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55eeb9a70f20 0x55eeb9a793f0 unknown :-1 s=CLOSED pgs=1131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:33:10.997 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:10.996+0000 7fca35998740 1 -- 172.21.15.47:0/242051766 >> 172.21.15.47:0/242051766 conn(0x55eeb9a38200 msgr2=0x55eeb9a755b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:33:10.997 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:10.996+0000 7fca35998740 1 -- 172.21.15.47:0/242051766 shutdown_connections 2024-03-21T10:33:10.997 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:10.996+0000 7fca35998740 1 -- 172.21.15.47:0/242051766 wait complete. 2024-03-21T10:33:10.997 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:10.997+0000 7fca35998740 1 -- start start 2024-03-21T10:33:11.000 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:10.997+0000 7fca35998740 1 Processor -- start 2024-03-21T10:33:11.001 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:10.997+0000 7fca35998740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55eeb9a70f20 0x55eeb9a81aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:33:11.001 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:10.997+0000 7fca35998740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55eeb9a71c00 0x55eeb9a81fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:33:11.001 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:10.997+0000 7fca35998740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55eeb9a7a290 0x55eeb9a73a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:33:11.001 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:10.997+0000 7fca35998740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca2c1b1ff0 con 0x55eeb9a7a290 2024-03-21T10:33:11.001 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:10.998+0000 7fca35998740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca2c1b1990 con 0x55eeb9a70f20 2024-03-21T10:33:11.001 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:10.998+0000 7fca35998740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca2c1aeeb0 con 0x55eeb9a71c00 2024-03-21T10:33:11.001 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:10.998+0000 7fca35998740 10 client.0 _ll_register_callbacks cb 0x7fca1c009f30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T10:33:11.001 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:10.998+0000 7fca30c00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55eeb9a7a290 0x55eeb9a73a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:33:11.001 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:10.998+0000 7fca30c00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55eeb9a7a290 0x55eeb9a73a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:47066/0 (socket says 172.21.15.47:47066) 2024-03-21T10:33:11.001 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:10.998+0000 7fca30c00640 1 -- 172.21.15.47:0/3894327608 learned_addr learned my addr 172.21.15.47:0/3894327608 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:33:11.001 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:10.998+0000 7fca2be00640 1 --2- 172.21.15.47:0/3894327608 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55eeb9a70f20 0x55eeb9a81aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:33:11.001 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:10.998+0000 7fca2b400640 1 --2- 172.21.15.47:0/3894327608 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55eeb9a71c00 0x55eeb9a81fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:33:11.001 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:10.998+0000 7fca30c00640 1 -- 172.21.15.47:0/3894327608 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55eeb9a71c00 msgr2=0x55eeb9a81fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:33:11.002 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:10.998+0000 7fca30c00640 1 --2- 172.21.15.47:0/3894327608 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55eeb9a71c00 0x55eeb9a81fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:33:11.002 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:10.998+0000 7fca30c00640 1 -- 172.21.15.47:0/3894327608 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55eeb9a70f20 msgr2=0x55eeb9a81aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:33:11.002 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:10.998+0000 7fca30c00640 1 --2- 172.21.15.47:0/3894327608 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55eeb9a70f20 0x55eeb9a81aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:33:11.002 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:10.998+0000 7fca30c00640 1 -- 172.21.15.47:0/3894327608 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55eeb9a742b0 con 0x55eeb9a7a290 2024-03-21T10:33:11.002 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:10.998+0000 7fca2be00640 1 --2- 172.21.15.47:0/3894327608 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55eeb9a70f20 0x55eeb9a81aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T10:33:11.002 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:10.998+0000 7fca2b400640 1 --2- 172.21.15.47:0/3894327608 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55eeb9a71c00 0x55eeb9a81fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T10:33:11.002 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:10.999+0000 7fca30c00640 1 --2- 172.21.15.47:0/3894327608 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55eeb9a7a290 0x55eeb9a73a80 secure :-1 s=READY pgs=1132 cs=0 l=1 rev1=1 crypto rx=0x7fca24009910 tx=0x7fca24004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=546766594f470e7 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:33:11.002 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:10.999+0000 7fca15600640 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-03-21T10:33:11.002 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:10.999+0000 7fca15600640 1 -- 172.21.15.47:0/3894327608 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca24010ea0 con 0x55eeb9a7a290 2024-03-21T10:33:11.002 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:10.999+0000 7fca35998740 1 -- 172.21.15.47:0/3894327608 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fca1c09bd40 con 0x55eeb9a7a290 2024-03-21T10:33:11.002 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:10.999+0000 7fca15600640 1 -- 172.21.15.47:0/3894327608 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(100..100 src has 1..100) v4 ==== 9650+0+0 (secure 0 0 0) 0x7fca24021da0 con 0x55eeb9a7a290 2024-03-21T10:33:11.002 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:10.999+0000 7fca15600640 1 -- 172.21.15.47:0/3894327608 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7fca2402a440 con 0x55eeb9a7a290 2024-03-21T10:33:11.003 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:10.999+0000 7fca35998740 10 client.18141 fetch_fsmap learned FSMap version 32 2024-03-21T10:33:11.003 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:10.999+0000 7fca35998740 1 -- 172.21.15.47:0/3894327608 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x55eeb9ae6390 con 0x55eeb9a7a290 2024-03-21T10:33:11.003 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:11.000+0000 7fca15600640 1 -- 172.21.15.47:0/3894327608 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fca1c09bd40 con 0x55eeb9a7a290 2024-03-21T10:33:11.003 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:11.000+0000 7fca15600640 1 -- 172.21.15.47:0/3894327608 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 32) v1 ==== 7942+0+0 (secure 0 0 0) 0x7fca24028070 con 0x55eeb9a7a290 2024-03-21T10:33:11.003 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:11.000+0000 7fca35998740 10 client.18141 fetch_fsmap finished waiting for FSMap version 32 2024-03-21T10:33:11.003 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:11.000+0000 7fca35998740 10 client.18141 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-21T10:33:11.003 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:11.000+0000 7fca35998740 20 client.18141 populate_metadata read hostname 'smithi047' 2024-03-21T10:33:11.003 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:11.000+0000 7fca35998740 1 --2- 172.21.15.47:0/3894327608 >> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] conn(0x55eeb9ae6af0 0x55eeb9ae8f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:33:11.003 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:11.000+0000 7fca35998740 4 client.18141 mds_command: new command op to 34535 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-21T10:33:11.003 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:11.000+0000 7fca35998740 1 -- 172.21.15.47:0/3894327608 --> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55eeb99dcae0 con 0x55eeb9ae6af0 2024-03-21T10:33:11.003 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:11.000+0000 7fca2be00640 1 --2- 172.21.15.47:0/3894327608 >> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] conn(0x55eeb9ae6af0 0x55eeb9ae8f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:33:11.003 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:11.001+0000 7fca15600640 10 client.18141 ms_handle_connect on v2:172.21.15.100:6838/4173168440 2024-03-21T10:33:11.003 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:11.001+0000 7fca2be00640 1 --2- 172.21.15.47:0/3894327608 >> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] conn(0x55eeb9ae6af0 0x55eeb9ae8f10 crc :-1 s=READY pgs=190 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4d587f92a4dff96 server_cookie=c7a3bbc57e6db4dc in_seq=0 out_seq=0 2024-03-21T10:33:11.005 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:11.002+0000 7fca15600640 1 -- 172.21.15.47:0/3894327608 <== mds.0 v2:172.21.15.100:6838/4173168440 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x55eeb99dcae0 con 0x55eeb9ae6af0 2024-03-21T10:33:11.005 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:11.002+0000 7fca15600640 10 client.18141 handle_command_reply: tid=0 2024-03-21T10:33:11.005 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:11.002+0000 7fca35998740 1 client.18141 shutdown 2024-03-21T10:33:11.005 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:11.002+0000 7fca35998740 1 -- 172.21.15.47:0/3894327608 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55eeb9a7a290 msgr2=0x55eeb9a73a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:33:11.005 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:11.002+0000 7fca35998740 1 --2- 172.21.15.47:0/3894327608 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55eeb9a7a290 0x55eeb9a73a80 secure :-1 s=READY pgs=1132 cs=0 l=1 rev1=1 crypto rx=0x7fca24009910 tx=0x7fca24004a40 comp rx=0 tx=0).stop 2024-03-21T10:33:11.005 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:11.002+0000 7fca35998740 1 -- 172.21.15.47:0/3894327608 shutdown_connections 2024-03-21T10:33:11.005 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:11.002+0000 7fca35998740 1 --2- 172.21.15.47:0/3894327608 >> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] conn(0x55eeb9ae6af0 0x55eeb9ae8f10 crc :-1 s=READY pgs=190 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:33:11.005 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:11.002+0000 7fca35998740 1 --2- 172.21.15.47:0/3894327608 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55eeb9a7a290 0x55eeb9a73a80 unknown :-1 s=CLOSED pgs=1132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:33:11.005 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:11.002+0000 7fca35998740 1 --2- 172.21.15.47:0/3894327608 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55eeb9a71c00 0x55eeb9a81fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:33:11.005 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:11.002+0000 7fca35998740 1 --2- 172.21.15.47:0/3894327608 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55eeb9a70f20 0x55eeb9a81aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:33:11.005 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:11.002+0000 7fca35998740 1 -- 172.21.15.47:0/3894327608 >> 172.21.15.47:0/3894327608 conn(0x55eeb9a38200 msgr2=0x55eeb9a6cd20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:33:11.005 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:11.002+0000 7fca15600640 0 client.18141 ms_handle_reset on v2:172.21.15.100:6838/4173168440 2024-03-21T10:33:11.005 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:11.002+0000 7fca35998740 1 -- 172.21.15.47:0/3894327608 shutdown_connections 2024-03-21T10:33:11.005 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:11.002+0000 7fca35998740 1 -- 172.21.15.47:0/3894327608 wait complete. 2024-03-21T10:33:11.006 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:11.002+0000 7fca35998740 20 client.18141 trim_cache size 0 max 16384 2024-03-21T10:33:11.006 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-03-21T10:33:11.006 INFO:teuthology.orchestra.run.smithi047.stdout: "status": "IDLE (waiting for more scrubs)", 2024-03-21T10:33:11.006 INFO:teuthology.orchestra.run.smithi047.stdout: "scrubs": { 2024-03-21T10:33:11.006 INFO:teuthology.orchestra.run.smithi047.stdout: "65135dcd-a63f-4d1b-9f22-6eecde26fa02": { 2024-03-21T10:33:11.006 INFO:teuthology.orchestra.run.smithi047.stdout: "path": "/", 2024-03-21T10:33:11.006 INFO:teuthology.orchestra.run.smithi047.stdout: "tag": "65135dcd-a63f-4d1b-9f22-6eecde26fa02", 2024-03-21T10:33:11.006 INFO:teuthology.orchestra.run.smithi047.stdout: "options": "recursive,force" 2024-03-21T10:33:11.007 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-21T10:33:11.007 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-21T10:33:11.007 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-03-21T10:33:11.007 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:11.003+0000 7fca32c00640 1 -- 172.21.15.47:0/3042615136 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7fca100a3090 msgr2=0x7fca100a5550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:33:11.007 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:11.003+0000 7fca32c00640 1 --2- 172.21.15.47:0/3042615136 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7fca100a3090 0x7fca100a5550 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fca2c06ad50 tx=0x7fca240096e0 comp rx=0 tx=0).stop 2024-03-21T10:33:11.008 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:11.003+0000 7fca32c00640 1 -- 172.21.15.47:0/3042615136 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fca2c1028e0 msgr2=0x7fca2c19a070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:33:11.008 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:11.003+0000 7fca32c00640 1 --2- 172.21.15.47:0/3042615136 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fca2c1028e0 0x7fca2c19a070 secure :-1 s=READY pgs=1128 cs=0 l=1 rev1=1 crypto rx=0x7fca1c00d770 tx=0x7fca1c00dc30 comp rx=0 tx=0).stop 2024-03-21T10:33:11.008 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:11.003+0000 7fca32c00640 1 -- 172.21.15.47:0/3042615136 shutdown_connections 2024-03-21T10:33:11.008 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:11.003+0000 7fca32c00640 1 --2- 172.21.15.47:0/3042615136 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fca2c10bcb0 0x7fca2c19e930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:33:11.008 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:11.003+0000 7fca32c00640 1 --2- 172.21.15.47:0/3042615136 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7fca100a3090 0x7fca100a5550 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:33:11.008 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:11.003+0000 7fca32c00640 1 --2- 172.21.15.47:0/3042615136 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fca2c103200 0x7fca2c19a5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:33:11.008 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:11.003+0000 7fca32c00640 1 --2- 172.21.15.47:0/3042615136 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fca2c1028e0 0x7fca2c19a070 unknown :-1 s=CLOSED pgs=1128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:33:11.008 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:11.003+0000 7fca32c00640 1 -- 172.21.15.47:0/3042615136 >> 172.21.15.47:0/3042615136 conn(0x7fca2c076920 msgr2=0x7fca2c1004c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:33:11.008 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:11.003+0000 7fca32c00640 1 -- 172.21.15.47:0/3042615136 shutdown_connections 2024-03-21T10:33:11.008 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:11.003+0000 7fca32c00640 1 -- 172.21.15.47:0/3042615136 wait complete. 2024-03-21T10:33:11.021 INFO:tasks.cephfs.filesystem:scrub status for tag:65135dcd-a63f-4d1b-9f22-6eecde26fa02 - {'path': '/', 'tag': '65135dcd-a63f-4d1b-9f22-6eecde26fa02', 'options': 'recursive,force'} 2024-03-21T10:33:11.711 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:33:11 smithi047 ceph-mon[31547]: pgmap v242: 129 pgs: 129 active+clean; 277 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 53 KiB/s wr, 30 op/s 2024-03-21T10:33:11.726 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:33:11 smithi100 ceph-mon[35193]: pgmap v242: 129 pgs: 129 active+clean; 277 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 53 KiB/s wr, 30 op/s 2024-03-21T10:33:11.737 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:33:11 smithi157 ceph-mon[35581]: pgmap v242: 129 pgs: 129 active+clean; 277 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 53 KiB/s wr, 30 op/s 2024-03-21T10:33:12.711 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:33:12 smithi047 ceph-mon[31547]: scrub queued for path: / 2024-03-21T10:33:12.711 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:33:12 smithi047 ceph-mon[31547]: scrub summary: idle+waiting paths [/] 2024-03-21T10:33:12.711 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:33:12 smithi047 ceph-mon[31547]: scrub summary: active paths [/] 2024-03-21T10:33:12.711 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:33:12 smithi047 ceph-mon[31547]: scrub summary: idle+waiting paths [/] 2024-03-21T10:33:12.727 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:33:12 smithi100 ceph-mon[35193]: scrub queued for path: / 2024-03-21T10:33:12.727 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:33:12 smithi100 ceph-mon[35193]: scrub summary: idle+waiting paths [/] 2024-03-21T10:33:12.727 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:33:12 smithi100 ceph-mon[35193]: scrub summary: active paths [/] 2024-03-21T10:33:12.727 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:33:12 smithi100 ceph-mon[35193]: scrub summary: idle+waiting paths [/] 2024-03-21T10:33:12.737 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:33:12 smithi157 ceph-mon[35581]: scrub queued for path: / 2024-03-21T10:33:12.737 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:33:12 smithi157 ceph-mon[35581]: scrub summary: idle+waiting paths [/] 2024-03-21T10:33:12.737 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:33:12 smithi157 ceph-mon[35581]: scrub summary: active paths [/] 2024-03-21T10:33:12.737 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:33:12 smithi157 ceph-mon[35581]: scrub summary: idle+waiting paths [/] 2024-03-21T10:33:13.711 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:33:13 smithi047 ceph-mon[31547]: pgmap v243: 129 pgs: 129 active+clean; 260 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 204 B/s wr, 11 op/s 2024-03-21T10:33:13.727 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:33:13 smithi100 ceph-mon[35193]: pgmap v243: 129 pgs: 129 active+clean; 260 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 204 B/s wr, 11 op/s 2024-03-21T10:33:13.737 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:33:13 smithi157 ceph-mon[35581]: pgmap v243: 129 pgs: 129 active+clean; 260 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 204 B/s wr, 11 op/s 2024-03-21T10:33:14.711 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:33:14 smithi047 ceph-mon[31547]: scrub summary: idle 2024-03-21T10:33:14.711 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:33:14 smithi047 ceph-mon[31547]: osdmap e101: 12 total, 12 up, 12 in 2024-03-21T10:33:14.726 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:33:14 smithi100 ceph-mon[35193]: scrub summary: idle 2024-03-21T10:33:14.726 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:33:14 smithi100 ceph-mon[35193]: osdmap e101: 12 total, 12 up, 12 in 2024-03-21T10:33:14.737 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:33:14 smithi157 ceph-mon[35581]: scrub summary: idle 2024-03-21T10:33:14.737 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:33:14 smithi157 ceph-mon[35581]: osdmap e101: 12 total, 12 up, 12 in 2024-03-21T10:33:15.711 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:33:15 smithi047 ceph-mon[31547]: pgmap v245: 129 pgs: 129 active+clean; 260 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 127 B/s wr, 8 op/s 2024-03-21T10:33:15.726 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:33:15 smithi100 ceph-mon[35193]: pgmap v245: 129 pgs: 129 active+clean; 260 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 127 B/s wr, 8 op/s 2024-03-21T10:33:15.737 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:33:15 smithi157 ceph-mon[35581]: pgmap v245: 129 pgs: 129 active+clean; 260 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 127 B/s wr, 8 op/s 2024-03-21T10:33:17.711 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:33:17 smithi047 ceph-mon[31547]: pgmap v246: 129 pgs: 129 active+clean; 260 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 KiB/s rd, 255 B/s wr, 13 op/s 2024-03-21T10:33:17.726 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:33:17 smithi100 ceph-mon[35193]: pgmap v246: 129 pgs: 129 active+clean; 260 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 KiB/s rd, 255 B/s wr, 13 op/s 2024-03-21T10:33:17.737 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:33:17 smithi157 ceph-mon[35581]: pgmap v246: 129 pgs: 129 active+clean; 260 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 KiB/s rd, 255 B/s wr, 13 op/s 2024-03-21T10:33:19.711 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:33:19 smithi047 ceph-mon[31547]: pgmap v247: 129 pgs: 129 active+clean; 260 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 KiB/s rd, 233 B/s wr, 13 op/s 2024-03-21T10:33:19.726 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:33:19 smithi100 ceph-mon[35193]: pgmap v247: 129 pgs: 129 active+clean; 260 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 KiB/s rd, 233 B/s wr, 13 op/s 2024-03-21T10:33:19.737 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:33:19 smithi157 ceph-mon[35581]: pgmap v247: 129 pgs: 129 active+clean; 260 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 KiB/s rd, 233 B/s wr, 13 op/s 2024-03-21T10:33:21.711 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:33:21 smithi047 ceph-mon[31547]: pgmap v248: 129 pgs: 129 active+clean; 260 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 204 B/s wr, 11 op/s 2024-03-21T10:33:21.726 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:33:21 smithi100 ceph-mon[35193]: pgmap v248: 129 pgs: 129 active+clean; 260 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 204 B/s wr, 11 op/s 2024-03-21T10:33:21.737 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:33:21 smithi157 ceph-mon[35581]: pgmap v248: 129 pgs: 129 active+clean; 260 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 204 B/s wr, 11 op/s 2024-03-21T10:33:23.711 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:33:23 smithi047 ceph-mon[31547]: pgmap v249: 129 pgs: 129 active+clean; 260 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 10 op/s 2024-03-21T10:33:23.726 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:33:23 smithi100 ceph-mon[35193]: pgmap v249: 129 pgs: 129 active+clean; 260 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 10 op/s 2024-03-21T10:33:23.737 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:33:23 smithi157 ceph-mon[35581]: pgmap v249: 129 pgs: 129 active+clean; 260 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 10 op/s 2024-03-21T10:33:25.727 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:33:25 smithi100 ceph-mon[35193]: pgmap v250: 129 pgs: 129 active+clean; 260 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 98 B/s wr, 9 op/s 2024-03-21T10:33:25.737 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:33:25 smithi157 ceph-mon[35581]: pgmap v250: 129 pgs: 129 active+clean; 260 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 98 B/s wr, 9 op/s 2024-03-21T10:33:25.961 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:33:25 smithi047 ceph-mon[31547]: pgmap v250: 129 pgs: 129 active+clean; 260 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 98 B/s wr, 9 op/s 2024-03-21T10:33:27.727 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:33:27 smithi100 ceph-mon[35193]: pgmap v251: 129 pgs: 129 active+clean; 260 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 85 B/s wr, 12 op/s 2024-03-21T10:33:27.737 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:33:27 smithi157 ceph-mon[35581]: pgmap v251: 129 pgs: 129 active+clean; 260 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 85 B/s wr, 12 op/s 2024-03-21T10:33:27.961 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:33:27 smithi047 ceph-mon[31547]: pgmap v251: 129 pgs: 129 active+clean; 260 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 85 B/s wr, 12 op/s 2024-03-21T10:33:29.737 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:33:29 smithi157 ceph-mon[35581]: pgmap v252: 129 pgs: 129 active+clean; 260 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-03-21T10:33:29.961 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:33:29 smithi047 ceph-mon[31547]: pgmap v252: 129 pgs: 129 active+clean; 260 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-03-21T10:33:29.976 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:33:29 smithi100 ceph-mon[35193]: pgmap v252: 129 pgs: 129 active+clean; 260 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-03-21T10:33:31.737 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:33:31 smithi157 ceph-mon[35581]: pgmap v253: 129 pgs: 129 active+clean; 260 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-21T10:33:31.961 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:33:31 smithi047 ceph-mon[31547]: pgmap v253: 129 pgs: 129 active+clean; 260 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-21T10:33:31.976 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:33:31 smithi100 ceph-mon[35193]: pgmap v253: 129 pgs: 129 active+clean; 260 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-21T10:33:33.961 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:33:33 smithi047 ceph-mon[31547]: pgmap v254: 129 pgs: 129 active+clean; 260 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-03-21T10:33:33.976 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:33:33 smithi100 ceph-mon[35193]: pgmap v254: 129 pgs: 129 active+clean; 260 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-03-21T10:33:33.987 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:33:33 smithi157 ceph-mon[35581]: pgmap v254: 129 pgs: 129 active+clean; 260 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-03-21T10:33:35.961 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:33:35 smithi047 ceph-mon[31547]: pgmap v255: 129 pgs: 129 active+clean; 260 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-21T10:33:35.976 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:33:35 smithi100 ceph-mon[35193]: pgmap v255: 129 pgs: 129 active+clean; 260 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-21T10:33:35.987 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:33:35 smithi157 ceph-mon[35581]: pgmap v255: 129 pgs: 129 active+clean; 260 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-21T10:33:37.961 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:33:37 smithi047 ceph-mon[31547]: pgmap v256: 129 pgs: 129 active+clean; 260 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 12 op/s 2024-03-21T10:33:37.976 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:33:37 smithi100 ceph-mon[35193]: pgmap v256: 129 pgs: 129 active+clean; 260 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 12 op/s 2024-03-21T10:33:37.987 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:33:37 smithi157 ceph-mon[35581]: pgmap v256: 129 pgs: 129 active+clean; 260 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 12 op/s 2024-03-21T10:33:39.961 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:33:39 smithi047 ceph-mon[31547]: pgmap v257: 129 pgs: 129 active+clean; 260 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-03-21T10:33:39.977 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:33:39 smithi100 ceph-mon[35193]: pgmap v257: 129 pgs: 129 active+clean; 260 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-03-21T10:33:39.987 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:33:39 smithi157 ceph-mon[35581]: pgmap v257: 129 pgs: 129 active+clean; 260 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-03-21T10:33:41.021 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-21T10:33:41.022 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-21T10:33:41.248 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:41.249+0000 7fa888c00640 1 -- 172.21.15.47:0/1604348271 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa884100dc0 msgr2=0x7fa88410d2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:33:41.248 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:41.249+0000 7fa888c00640 1 --2- 172.21.15.47:0/1604348271 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa884100dc0 0x7fa88410d2d0 secure :-1 s=READY pgs=1133 cs=0 l=1 rev1=1 crypto rx=0x7fa87c009cd0 tx=0x7fa87c02c0e0 comp rx=0 tx=0).stop 2024-03-21T10:33:41.248 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:41.250+0000 7fa888c00640 1 -- 172.21.15.47:0/1604348271 shutdown_connections 2024-03-21T10:33:41.248 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:41.250+0000 7fa888c00640 1 --2- 172.21.15.47:0/1604348271 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa884100dc0 0x7fa88410d2d0 unknown :-1 s=CLOSED pgs=1133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:33:41.249 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:41.250+0000 7fa888c00640 1 --2- 172.21.15.47:0/1604348271 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa884100400 0x7fa884100880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:33:41.249 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:41.250+0000 7fa888c00640 1 --2- 172.21.15.47:0/1604348271 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa8841064a0 0x7fa884106880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:33:41.249 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:41.250+0000 7fa888c00640 1 -- 172.21.15.47:0/1604348271 >> 172.21.15.47:0/1604348271 conn(0x7fa8840fbef0 msgr2=0x7fa8840fe310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:33:41.249 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:41.250+0000 7fa888c00640 1 -- 172.21.15.47:0/1604348271 shutdown_connections 2024-03-21T10:33:41.249 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:41.250+0000 7fa888c00640 1 -- 172.21.15.47:0/1604348271 wait complete. 2024-03-21T10:33:41.249 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:41.251+0000 7fa888c00640 1 Processor -- start 2024-03-21T10:33:41.249 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:41.251+0000 7fa888c00640 1 -- start start 2024-03-21T10:33:41.251 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:41.252+0000 7fa888c00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa884100400 0x7fa884199fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:33:41.251 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:41.252+0000 7fa888c00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa884100dc0 0x7fa88419a4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:33:41.252 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:41.252+0000 7fa888c00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa8841064a0 0x7fa88419e860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:33:41.252 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:41.252+0000 7fa888c00640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa884110680 con 0x7fa884100dc0 2024-03-21T10:33:41.252 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:41.252+0000 7fa888c00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa884110500 con 0x7fa884100400 2024-03-21T10:33:41.252 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:41.252+0000 7fa888c00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa884110800 con 0x7fa8841064a0 2024-03-21T10:33:41.252 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:41.252+0000 7fa882a00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa884100dc0 0x7fa88419a4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:33:41.252 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:41.252+0000 7fa882a00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa884100dc0 0x7fa88419a4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:57838/0 (socket says 172.21.15.47:57838) 2024-03-21T10:33:41.252 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:41.252+0000 7fa882a00640 1 -- 172.21.15.47:0/3673891066 learned_addr learned my addr 172.21.15.47:0/3673891066 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:33:41.252 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:41.252+0000 7fa883400640 1 --2- 172.21.15.47:0/3673891066 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa884100400 0x7fa884199fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:33:41.252 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:41.252+0000 7fa883e00640 1 --2- 172.21.15.47:0/3673891066 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa8841064a0 0x7fa88419e860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:33:41.252 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:41.253+0000 7fa882a00640 1 -- 172.21.15.47:0/3673891066 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa8841064a0 msgr2=0x7fa88419e860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:33:41.252 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:41.253+0000 7fa882a00640 1 --2- 172.21.15.47:0/3673891066 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa8841064a0 0x7fa88419e860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:33:41.252 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:41.253+0000 7fa882a00640 1 -- 172.21.15.47:0/3673891066 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa884100400 msgr2=0x7fa884199fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:33:41.253 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:41.253+0000 7fa882a00640 1 --2- 172.21.15.47:0/3673891066 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa884100400 0x7fa884199fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:33:41.253 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:41.253+0000 7fa882a00640 1 -- 172.21.15.47:0/3673891066 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa88419ef90 con 0x7fa884100dc0 2024-03-21T10:33:41.253 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:41.253+0000 7fa883e00640 1 --2- 172.21.15.47:0/3673891066 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa8841064a0 0x7fa88419e860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T10:33:41.253 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:41.253+0000 7fa882a00640 1 --2- 172.21.15.47:0/3673891066 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa884100dc0 0x7fa88419a4e0 secure :-1 s=READY pgs=1134 cs=0 l=1 rev1=1 crypto rx=0x7fa87000d770 tx=0x7fa87000dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:33:41.253 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:41.253+0000 7fa87be00640 1 -- 172.21.15.47:0/3673891066 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa870017070 con 0x7fa884100dc0 2024-03-21T10:33:41.253 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:41.253+0000 7fa888c00640 1 -- 172.21.15.47:0/3673891066 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa88419f280 con 0x7fa884100dc0 2024-03-21T10:33:41.253 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:41.253+0000 7fa87be00640 1 -- 172.21.15.47:0/3673891066 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fa8700054d0 con 0x7fa884100dc0 2024-03-21T10:33:41.253 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:41.253+0000 7fa87be00640 1 -- 172.21.15.47:0/3673891066 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa870004cf0 con 0x7fa884100dc0 2024-03-21T10:33:41.253 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:41.253+0000 7fa888c00640 1 -- 172.21.15.47:0/3673891066 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa8841a6b10 con 0x7fa884100dc0 2024-03-21T10:33:41.255 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:41.257+0000 7fa88cf88740 1 Processor -- start 2024-03-21T10:33:41.255 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:41.257+0000 7fa88cf88740 1 -- start start 2024-03-21T10:33:41.256 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:41.257+0000 7fa88cf88740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f438e47dd0 0x55f438e46b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:33:41.256 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:41.257+0000 7fa88cf88740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f438e470f0 0x55f438e47570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:33:41.256 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:41.257+0000 7fa88cf88740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f438e50460 0x55f438e52850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:33:41.257 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:41.257+0000 7fa88cf88740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f438dab250 con 0x55f438e470f0 2024-03-21T10:33:41.257 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:41.257+0000 7fa88cf88740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f438d53a30 con 0x55f438e47dd0 2024-03-21T10:33:41.257 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:41.257+0000 7fa88cf88740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f438da72f0 con 0x55f438e50460 2024-03-21T10:33:41.257 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:41.258+0000 7fa883400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f438e470f0 0x55f438e47570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:33:41.257 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:41.258+0000 7fa883400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f438e470f0 0x55f438e47570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:57848/0 (socket says 172.21.15.47:57848) 2024-03-21T10:33:41.257 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:41.258+0000 7fa883400640 1 -- 172.21.15.47:0/2510533630 learned_addr learned my addr 172.21.15.47:0/2510533630 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:33:41.257 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:41.258+0000 7fa883e00640 1 --2- 172.21.15.47:0/2510533630 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f438e47dd0 0x55f438e46b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:33:41.257 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:41.258+0000 7fa883400640 1 -- 172.21.15.47:0/2510533630 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f438e50460 msgr2=0x55f438e52850 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-21T10:33:41.257 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:41.258+0000 7fa883400640 1 --2- 172.21.15.47:0/2510533630 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f438e50460 0x55f438e52850 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:33:41.257 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:41.258+0000 7fa883400640 1 -- 172.21.15.47:0/2510533630 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f438e47dd0 msgr2=0x55f438e46b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:33:41.257 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:41.258+0000 7fa883400640 1 --2- 172.21.15.47:0/2510533630 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f438e47dd0 0x55f438e46b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:33:41.257 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:41.258+0000 7fa883400640 1 -- 172.21.15.47:0/2510533630 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f438d3e690 con 0x55f438e470f0 2024-03-21T10:33:41.259 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:41.259+0000 7fa883400640 1 --2- 172.21.15.47:0/2510533630 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f438e470f0 0x55f438e47570 secure :-1 s=READY pgs=1135 cs=0 l=1 rev1=1 crypto rx=0x7fa87400bc90 tx=0x7fa8740107a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1242ffc7104118e8 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:33:41.259 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:41.259+0000 7fa87be00640 1 -- 172.21.15.47:0/3673891066 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 27) v1 ==== 137985+0+0 (secure 0 0 0) 0x7fa870004e90 con 0x7fa884100dc0 2024-03-21T10:33:41.259 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:41.259+0000 7fa86fe00640 1 -- 172.21.15.47:0/2510533630 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa87401c480 con 0x55f438e470f0 2024-03-21T10:33:41.259 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:41.259+0000 7fa86fe00640 1 -- 172.21.15.47:0/2510533630 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fa87401cad0 con 0x55f438e470f0 2024-03-21T10:33:41.259 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:41.259+0000 7fa86fe00640 1 -- 172.21.15.47:0/2510533630 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa874011bb0 con 0x55f438e470f0 2024-03-21T10:33:41.259 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:41.260+0000 7fa88cf88740 1 -- 172.21.15.47:0/2510533630 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f438e470f0 msgr2=0x55f438e47570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:33:41.259 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:41.260+0000 7fa88cf88740 1 --2- 172.21.15.47:0/2510533630 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f438e470f0 0x55f438e47570 secure :-1 s=READY pgs=1135 cs=0 l=1 rev1=1 crypto rx=0x7fa87400bc90 tx=0x7fa8740107a0 comp rx=0 tx=0).stop 2024-03-21T10:33:41.259 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:41.260+0000 7fa88cf88740 1 -- 172.21.15.47:0/2510533630 shutdown_connections 2024-03-21T10:33:41.260 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:41.260+0000 7fa88cf88740 1 --2- 172.21.15.47:0/2510533630 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f438e50460 0x55f438e52850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:33:41.260 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:41.260+0000 7fa88cf88740 1 --2- 172.21.15.47:0/2510533630 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f438e470f0 0x55f438e47570 unknown :-1 s=CLOSED pgs=1135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:33:41.260 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:41.260+0000 7fa88cf88740 1 --2- 172.21.15.47:0/2510533630 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f438e47dd0 0x55f438e46b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:33:41.260 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:41.260+0000 7fa88cf88740 1 -- 172.21.15.47:0/2510533630 >> 172.21.15.47:0/2510533630 conn(0x55f438e0e200 msgr2=0x55f438e41b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:33:41.260 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:41.260+0000 7fa88cf88740 1 -- 172.21.15.47:0/2510533630 shutdown_connections 2024-03-21T10:33:41.260 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:41.260+0000 7fa88cf88740 1 -- 172.21.15.47:0/2510533630 wait complete. 2024-03-21T10:33:41.260 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:41.260+0000 7fa87be00640 1 --2- 172.21.15.47:0/3673891066 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7fa8680a2fb0 0x7fa8680a5470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:33:41.260 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:41.261+0000 7fa87be00640 1 -- 172.21.15.47:0/3673891066 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(101..101 src has 1..101) v4 ==== 9562+0+0 (secure 0 0 0) 0x7fa8700cf6c0 con 0x7fa884100dc0 2024-03-21T10:33:41.260 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:41.261+0000 7fa88cf88740 1 -- start start 2024-03-21T10:33:41.260 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:41.261+0000 7fa883e00640 1 --2- 172.21.15.47:0/3673891066 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7fa8680a2fb0 0x7fa8680a5470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:33:41.260 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:41.261+0000 7fa88cf88740 1 Processor -- start 2024-03-21T10:33:41.265 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:41.261+0000 7fa883e00640 1 --2- 172.21.15.47:0/3673891066 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7fa8680a2fb0 0x7fa8680a5470 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fa87c006010 tx=0x7fa87c004a40 comp rx=0 tx=0).ready entity=mgr.15687 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:33:41.266 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:41.262+0000 7fa88cf88740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f438e470f0 0x55f438e4fb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:33:41.266 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:41.262+0000 7fa88cf88740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f438e47dd0 0x55f438e4cec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:33:41.266 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:41.262+0000 7fa88cf88740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f438e50460 0x55f438e4d400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:33:41.266 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:41.262+0000 7fa88cf88740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f438e532d0 con 0x55f438e470f0 2024-03-21T10:33:41.266 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:41.262+0000 7fa88cf88740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f438d66e80 con 0x55f438e50460 2024-03-21T10:33:41.266 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:41.262+0000 7fa88cf88740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f438e53450 con 0x55f438e47dd0 2024-03-21T10:33:41.266 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:41.262+0000 7fa88cf88740 10 client.0 _ll_register_callbacks cb 0x55f438d79130 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T10:33:41.266 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:41.262+0000 7fa883400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f438e470f0 0x55f438e4fb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:33:41.266 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:41.262+0000 7fa883400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f438e470f0 0x55f438e4fb30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:57858/0 (socket says 172.21.15.47:57858) 2024-03-21T10:33:41.267 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:41.262+0000 7fa883400640 1 -- 172.21.15.47:0/3863627655 learned_addr learned my addr 172.21.15.47:0/3863627655 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:33:41.267 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:41.262+0000 7fa883e00640 1 --2- 172.21.15.47:0/3863627655 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f438e50460 0x55f438e4d400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:33:41.267 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:41.262+0000 7fa882a00640 1 --2- 172.21.15.47:0/3863627655 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f438e47dd0 0x55f438e4cec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:33:41.267 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:41.262+0000 7fa883400640 1 -- 172.21.15.47:0/3863627655 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f438e47dd0 msgr2=0x55f438e4cec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:33:41.267 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:41.262+0000 7fa883400640 1 --2- 172.21.15.47:0/3863627655 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f438e47dd0 0x55f438e4cec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:33:41.267 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:41.262+0000 7fa883400640 1 -- 172.21.15.47:0/3863627655 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f438e50460 msgr2=0x55f438e4d400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:33:41.267 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:41.262+0000 7fa883400640 1 --2- 172.21.15.47:0/3863627655 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f438e50460 0x55f438e4d400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:33:41.267 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:41.262+0000 7fa883400640 1 -- 172.21.15.47:0/3863627655 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55f438b897d0 con 0x55f438e470f0 2024-03-21T10:33:41.267 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:41.262+0000 7fa883e00640 1 --2- 172.21.15.47:0/3863627655 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f438e50460 0x55f438e4d400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T10:33:41.267 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:41.263+0000 7fa883400640 1 --2- 172.21.15.47:0/3863627655 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f438e470f0 0x55f438e4fb30 secure :-1 s=READY pgs=1136 cs=0 l=1 rev1=1 crypto rx=0x7fa874011010 tx=0x7fa87401ceb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e6d4592363026c06 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:33:41.267 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:41.263+0000 7fa86cc00640 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-03-21T10:33:41.267 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:41.263+0000 7fa86cc00640 1 -- 172.21.15.47:0/3863627655 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa87400bd80 con 0x55f438e470f0 2024-03-21T10:33:41.267 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:41.263+0000 7fa86cc00640 1 -- 172.21.15.47:0/3863627655 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(101..101 src has 1..101) v4 ==== 9562+0+0 (secure 0 0 0) 0x7fa87401c690 con 0x55f438e470f0 2024-03-21T10:33:41.267 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:41.263+0000 7fa88cf88740 1 -- 172.21.15.47:0/3863627655 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fa87401cad0 con 0x55f438e470f0 2024-03-21T10:33:41.268 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:41.264+0000 7fa86cc00640 1 -- 172.21.15.47:0/3863627655 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa87402e050 con 0x55f438e470f0 2024-03-21T10:33:41.268 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:41.264+0000 7fa88cf88740 10 client.18153 fetch_fsmap learned FSMap version 32 2024-03-21T10:33:41.268 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:41.264+0000 7fa88cf88740 1 -- 172.21.15.47:0/3863627655 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x55f438abf3f0 con 0x55f438e470f0 2024-03-21T10:33:41.268 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:41.264+0000 7fa86cc00640 1 -- 172.21.15.47:0/3863627655 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fa87401cad0 con 0x55f438e470f0 2024-03-21T10:33:41.268 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:41.264+0000 7fa86cc00640 1 -- 172.21.15.47:0/3863627655 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 32) v1 ==== 7942+0+0 (secure 0 0 0) 0x7fa874005450 con 0x55f438e470f0 2024-03-21T10:33:41.268 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:41.264+0000 7fa88cf88740 10 client.18153 fetch_fsmap finished waiting for FSMap version 32 2024-03-21T10:33:41.268 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:41.264+0000 7fa88cf88740 10 client.18153 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-21T10:33:41.268 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:41.264+0000 7fa88cf88740 20 client.18153 populate_metadata read hostname 'smithi047' 2024-03-21T10:33:41.268 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:41.264+0000 7fa88cf88740 1 --2- 172.21.15.47:0/3863627655 >> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] conn(0x55f438e60300 0x55f438e626f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:33:41.268 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:41.264+0000 7fa88cf88740 4 client.18153 mds_command: new command op to 34535 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T10:33:41.268 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:41.264+0000 7fa88cf88740 1 -- 172.21.15.47:0/3863627655 --> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55f438db8790 con 0x55f438e60300 2024-03-21T10:33:41.268 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:41.265+0000 7fa882a00640 1 --2- 172.21.15.47:0/3863627655 >> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] conn(0x55f438e60300 0x55f438e626f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:33:41.268 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:41.265+0000 7fa882a00640 1 --2- 172.21.15.47:0/3863627655 >> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] conn(0x55f438e60300 0x55f438e626f0 crc :-1 s=READY pgs=191 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f8470fd821aa2ee4 server_cookie=91032c5e8ab0f3 in_seq=0 out_seq=0 2024-03-21T10:33:41.268 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:41.265+0000 7fa86cc00640 10 client.18153 ms_handle_connect on v2:172.21.15.100:6838/4173168440 2024-03-21T10:33:41.268 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:41.266+0000 7fa86cc00640 1 -- 172.21.15.47:0/3863627655 <== mds.0 v2:172.21.15.100:6838/4173168440 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55f438db8790 con 0x55f438e60300 2024-03-21T10:33:41.268 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:41.266+0000 7fa86cc00640 10 client.18153 handle_command_reply: tid=0 2024-03-21T10:33:41.269 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:41.266+0000 7fa88cf88740 1 client.18153 shutdown 2024-03-21T10:33:41.269 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:41.267+0000 7fa88cf88740 1 -- 172.21.15.47:0/3863627655 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f438e470f0 msgr2=0x55f438e4fb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:33:41.269 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:41.267+0000 7fa88cf88740 1 --2- 172.21.15.47:0/3863627655 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f438e470f0 0x55f438e4fb30 secure :-1 s=READY pgs=1136 cs=0 l=1 rev1=1 crypto rx=0x7fa874011010 tx=0x7fa87401ceb0 comp rx=0 tx=0).stop 2024-03-21T10:33:41.269 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:41.267+0000 7fa88cf88740 1 -- 172.21.15.47:0/3863627655 shutdown_connections 2024-03-21T10:33:41.269 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:41.267+0000 7fa88cf88740 1 --2- 172.21.15.47:0/3863627655 >> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] conn(0x55f438e60300 0x55f438e626f0 crc :-1 s=READY pgs=191 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:33:41.269 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:41.267+0000 7fa88cf88740 1 --2- 172.21.15.47:0/3863627655 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f438e50460 0x55f438e4d400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:33:41.269 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:41.267+0000 7fa88cf88740 1 --2- 172.21.15.47:0/3863627655 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f438e47dd0 0x55f438e4cec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:33:41.269 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:41.267+0000 7fa86cc00640 0 client.18153 ms_handle_reset on v2:172.21.15.100:6838/4173168440 2024-03-21T10:33:41.269 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:41.267+0000 7fa88cf88740 1 --2- 172.21.15.47:0/3863627655 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f438e470f0 0x55f438e4fb30 unknown :-1 s=CLOSED pgs=1136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:33:41.269 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:41.267+0000 7fa88cf88740 1 -- 172.21.15.47:0/3863627655 >> 172.21.15.47:0/3863627655 conn(0x55f438e0e200 msgr2=0x55f438e43890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:33:41.269 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:41.267+0000 7fa88cf88740 1 -- 172.21.15.47:0/3863627655 shutdown_connections 2024-03-21T10:33:41.269 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:41.267+0000 7fa88cf88740 1 -- 172.21.15.47:0/3863627655 wait complete. 2024-03-21T10:33:41.269 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:41.267+0000 7fa88cf88740 20 client.18153 trim_cache size 0 max 16384 2024-03-21T10:33:41.284 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:41.285+0000 7fa88cf88740 1 Processor -- start 2024-03-21T10:33:41.284 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:41.285+0000 7fa88cf88740 1 -- start start 2024-03-21T10:33:41.284 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:41.285+0000 7fa88cf88740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f438e470f0 0x55f438e4c7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:33:41.284 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:41.285+0000 7fa88cf88740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f438e47dd0 0x55f438e40a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:33:41.285 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:41.285+0000 7fa88cf88740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f438e50460 0x55f438e4f5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:33:41.285 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:41.285+0000 7fa88cf88740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8841b0a40 con 0x55f438e470f0 2024-03-21T10:33:41.285 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:41.285+0000 7fa88cf88740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8841b23c0 con 0x55f438e47dd0 2024-03-21T10:33:41.285 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:41.285+0000 7fa88cf88740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8841b10a0 con 0x55f438e50460 2024-03-21T10:33:41.285 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:41.285+0000 7fa883400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f438e470f0 0x55f438e4c7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:33:41.285 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:41.285+0000 7fa883400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f438e470f0 0x55f438e4c7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:57866/0 (socket says 172.21.15.47:57866) 2024-03-21T10:33:41.285 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:41.285+0000 7fa883400640 1 -- 172.21.15.47:0/3884107343 learned_addr learned my addr 172.21.15.47:0/3884107343 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:33:41.285 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:41.286+0000 7fa882a00640 1 --2- 172.21.15.47:0/3884107343 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f438e47dd0 0x55f438e40a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:33:41.285 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:41.286+0000 7fa883e00640 1 --2- 172.21.15.47:0/3884107343 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f438e50460 0x55f438e4f5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:33:41.285 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:41.286+0000 7fa883400640 1 -- 172.21.15.47:0/3884107343 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f438e50460 msgr2=0x55f438e4f5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:33:41.285 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:41.286+0000 7fa883400640 1 --2- 172.21.15.47:0/3884107343 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f438e50460 0x55f438e4f5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:33:41.285 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:41.286+0000 7fa883400640 1 -- 172.21.15.47:0/3884107343 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f438e47dd0 msgr2=0x55f438e40a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:33:41.285 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:41.286+0000 7fa883400640 1 --2- 172.21.15.47:0/3884107343 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f438e47dd0 0x55f438e40a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:33:41.286 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:41.286+0000 7fa883400640 1 -- 172.21.15.47:0/3884107343 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f438a7f0f0 con 0x55f438e470f0 2024-03-21T10:33:41.286 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:41.286+0000 7fa882a00640 1 --2- 172.21.15.47:0/3884107343 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f438e47dd0 0x55f438e40a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T10:33:41.286 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:41.286+0000 7fa883400640 1 --2- 172.21.15.47:0/3884107343 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f438e470f0 0x55f438e4c7a0 secure :-1 s=READY pgs=1137 cs=0 l=1 rev1=1 crypto rx=0x7fa8740115c0 tx=0x7fa874004830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=614ff4a08b4c83f5 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:33:41.289 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:41.287+0000 7fa86f400640 1 -- 172.21.15.47:0/3884107343 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa874000e60 con 0x55f438e470f0 2024-03-21T10:33:41.290 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:41.287+0000 7fa86f400640 1 -- 172.21.15.47:0/3884107343 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fa87400bd80 con 0x55f438e470f0 2024-03-21T10:33:41.290 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:41.287+0000 7fa86f400640 1 -- 172.21.15.47:0/3884107343 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa874004320 con 0x55f438e470f0 2024-03-21T10:33:41.290 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:41.287+0000 7fa88cf88740 1 -- 172.21.15.47:0/3884107343 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f438e470f0 msgr2=0x55f438e4c7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:33:41.290 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:41.287+0000 7fa88cf88740 1 --2- 172.21.15.47:0/3884107343 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f438e470f0 0x55f438e4c7a0 secure :-1 s=READY pgs=1137 cs=0 l=1 rev1=1 crypto rx=0x7fa8740115c0 tx=0x7fa874004830 comp rx=0 tx=0).stop 2024-03-21T10:33:41.290 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:41.287+0000 7fa88cf88740 1 -- 172.21.15.47:0/3884107343 shutdown_connections 2024-03-21T10:33:41.290 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:41.287+0000 7fa88cf88740 1 --2- 172.21.15.47:0/3884107343 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f438e50460 0x55f438e4f5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:33:41.290 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:41.287+0000 7fa88cf88740 1 --2- 172.21.15.47:0/3884107343 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f438e47dd0 0x55f438e40a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:33:41.290 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:41.287+0000 7fa88cf88740 1 --2- 172.21.15.47:0/3884107343 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f438e470f0 0x55f438e4c7a0 unknown :-1 s=CLOSED pgs=1137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:33:41.290 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:41.287+0000 7fa88cf88740 1 -- 172.21.15.47:0/3884107343 >> 172.21.15.47:0/3884107343 conn(0x55f438e0e200 msgr2=0x55f438e40330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:33:41.290 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:41.287+0000 7fa88cf88740 1 -- 172.21.15.47:0/3884107343 shutdown_connections 2024-03-21T10:33:41.291 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:41.287+0000 7fa88cf88740 1 -- 172.21.15.47:0/3884107343 wait complete. 2024-03-21T10:33:41.291 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:41.288+0000 7fa88cf88740 1 -- start start 2024-03-21T10:33:41.291 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:41.288+0000 7fa88cf88740 1 Processor -- start 2024-03-21T10:33:41.291 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:41.289+0000 7fa88cf88740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f438e470f0 0x55f438e4a9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:33:41.291 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:41.289+0000 7fa88cf88740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f438e47dd0 0x55f438e49130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:33:41.291 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:41.289+0000 7fa88cf88740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f438e50460 0x55f438e49670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:33:41.291 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:41.289+0000 7fa88cf88740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8841b1d60 con 0x55f438e50460 2024-03-21T10:33:41.291 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:41.289+0000 7fa88cf88740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8841b1700 con 0x55f438e470f0 2024-03-21T10:33:41.292 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:41.289+0000 7fa88cf88740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8841aec20 con 0x55f438e47dd0 2024-03-21T10:33:41.292 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:41.289+0000 7fa88cf88740 10 client.0 _ll_register_callbacks cb 0x7fa87c037070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T10:33:41.292 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:41.289+0000 7fa883e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f438e50460 0x55f438e49670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:33:41.292 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:41.289+0000 7fa883e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f438e50460 0x55f438e49670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:57868/0 (socket says 172.21.15.47:57868) 2024-03-21T10:33:41.292 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:41.289+0000 7fa883e00640 1 -- 172.21.15.47:0/173502656 learned_addr learned my addr 172.21.15.47:0/173502656 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:33:41.292 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:41.289+0000 7fa883400640 1 --2- 172.21.15.47:0/173502656 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f438e470f0 0x55f438e4a9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:33:41.292 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:41.289+0000 7fa882a00640 1 --2- 172.21.15.47:0/173502656 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f438e47dd0 0x55f438e49130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:33:41.292 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:41.290+0000 7fa883e00640 1 -- 172.21.15.47:0/173502656 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f438e47dd0 msgr2=0x55f438e49130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:33:41.292 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:41.290+0000 7fa883e00640 1 --2- 172.21.15.47:0/173502656 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f438e47dd0 0x55f438e49130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:33:41.292 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:41.290+0000 7fa883e00640 1 -- 172.21.15.47:0/173502656 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f438e470f0 msgr2=0x55f438e4a9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:33:41.292 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:41.290+0000 7fa883e00640 1 --2- 172.21.15.47:0/173502656 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f438e470f0 0x55f438e4a9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:33:41.292 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:41.290+0000 7fa883e00640 1 -- 172.21.15.47:0/173502656 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55f438e49c00 con 0x55f438e50460 2024-03-21T10:33:41.292 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:41.290+0000 7fa883400640 1 --2- 172.21.15.47:0/173502656 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f438e470f0 0x55f438e4a9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T10:33:41.292 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:41.290+0000 7fa883e00640 1 --2- 172.21.15.47:0/173502656 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f438e50460 0x55f438e49670 secure :-1 s=READY pgs=1138 cs=0 l=1 rev1=1 crypto rx=0x7fa87c009cd0 tx=0x7fa87c0095c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e4aebe5e85f11f86 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:33:41.293 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:41.290+0000 7fa86cc00640 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-03-21T10:33:41.293 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:41.290+0000 7fa86cc00640 1 -- 172.21.15.47:0/173502656 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa87c051070 con 0x55f438e50460 2024-03-21T10:33:41.293 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:41.290+0000 7fa88cf88740 1 -- 172.21.15.47:0/173502656 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fa87400bd80 con 0x55f438e50460 2024-03-21T10:33:41.293 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:41.290+0000 7fa86cc00640 1 -- 172.21.15.47:0/173502656 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(101..101 src has 1..101) v4 ==== 9562+0+0 (secure 0 0 0) 0x7fa87c040660 con 0x55f438e50460 2024-03-21T10:33:41.293 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:41.291+0000 7fa86cc00640 1 -- 172.21.15.47:0/173502656 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa87c05d050 con 0x55f438e50460 2024-03-21T10:33:41.293 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:41.291+0000 7fa88cf88740 10 client.18159 fetch_fsmap learned FSMap version 32 2024-03-21T10:33:41.293 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:41.291+0000 7fa88cf88740 1 -- 172.21.15.47:0/173502656 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x55f438ebb640 con 0x55f438e50460 2024-03-21T10:33:41.293 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:41.291+0000 7fa86cc00640 1 -- 172.21.15.47:0/173502656 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fa87400bd80 con 0x55f438e50460 2024-03-21T10:33:41.293 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:41.291+0000 7fa86cc00640 1 -- 172.21.15.47:0/173502656 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 32) v1 ==== 7942+0+0 (secure 0 0 0) 0x7fa87c0349d0 con 0x55f438e50460 2024-03-21T10:33:41.293 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:41.291+0000 7fa88cf88740 10 client.18159 fetch_fsmap finished waiting for FSMap version 32 2024-03-21T10:33:41.293 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:41.291+0000 7fa88cf88740 10 client.18159 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-21T10:33:41.293 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:41.291+0000 7fa88cf88740 20 client.18159 populate_metadata read hostname 'smithi047' 2024-03-21T10:33:41.293 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:41.291+0000 7fa88cf88740 1 --2- 172.21.15.47:0/173502656 >> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] conn(0x55f438e60300 0x55f438ebdd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:33:41.293 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:41.291+0000 7fa88cf88740 4 client.18159 mds_command: new command op to 34535 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-21T10:33:41.294 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:41.291+0000 7fa88cf88740 1 -- 172.21.15.47:0/173502656 --> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55f438db2ae0 con 0x55f438e60300 2024-03-21T10:33:41.295 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:41.292+0000 7fa883400640 1 --2- 172.21.15.47:0/173502656 >> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] conn(0x55f438e60300 0x55f438ebdd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:33:41.295 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:41.292+0000 7fa883400640 1 --2- 172.21.15.47:0/173502656 >> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] conn(0x55f438e60300 0x55f438ebdd60 crc :-1 s=READY pgs=192 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=76cd4db8e61fe495 server_cookie=5e4aa92d762896f8 in_seq=0 out_seq=0 2024-03-21T10:33:41.295 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:41.292+0000 7fa86cc00640 10 client.18159 ms_handle_connect on v2:172.21.15.100:6838/4173168440 2024-03-21T10:33:41.295 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:41.293+0000 7fa86cc00640 1 -- 172.21.15.47:0/173502656 <== mds.0 v2:172.21.15.100:6838/4173168440 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x55f438db2ae0 con 0x55f438e60300 2024-03-21T10:33:41.295 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:41.293+0000 7fa86cc00640 10 client.18159 handle_command_reply: tid=0 2024-03-21T10:33:41.295 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:41.293+0000 7fa88cf88740 1 client.18159 shutdown 2024-03-21T10:33:41.296 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:41.293+0000 7fa88cf88740 1 -- 172.21.15.47:0/173502656 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f438e50460 msgr2=0x55f438e49670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:33:41.296 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:41.293+0000 7fa88cf88740 1 --2- 172.21.15.47:0/173502656 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f438e50460 0x55f438e49670 secure :-1 s=READY pgs=1138 cs=0 l=1 rev1=1 crypto rx=0x7fa87c009cd0 tx=0x7fa87c0095c0 comp rx=0 tx=0).stop 2024-03-21T10:33:41.296 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:41.293+0000 7fa88cf88740 1 -- 172.21.15.47:0/173502656 shutdown_connections 2024-03-21T10:33:41.296 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:41.293+0000 7fa88cf88740 1 --2- 172.21.15.47:0/173502656 >> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] conn(0x55f438e60300 0x55f438ebdd60 crc :-1 s=READY pgs=192 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:33:41.296 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:41.293+0000 7fa88cf88740 1 --2- 172.21.15.47:0/173502656 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f438e50460 0x55f438e49670 unknown :-1 s=CLOSED pgs=1138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:33:41.296 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:41.293+0000 7fa88cf88740 1 --2- 172.21.15.47:0/173502656 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f438e47dd0 0x55f438e49130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:33:41.296 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:41.293+0000 7fa88cf88740 1 --2- 172.21.15.47:0/173502656 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f438e470f0 0x55f438e4a9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:33:41.296 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:41.293+0000 7fa86cc00640 0 client.18159 ms_handle_reset on v2:172.21.15.100:6838/4173168440 2024-03-21T10:33:41.296 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:41.293+0000 7fa88cf88740 1 -- 172.21.15.47:0/173502656 >> 172.21.15.47:0/173502656 conn(0x55f438e0e200 msgr2=0x55f438e40bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:33:41.296 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:41.293+0000 7fa88cf88740 1 -- 172.21.15.47:0/173502656 shutdown_connections 2024-03-21T10:33:41.296 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:41.293+0000 7fa88cf88740 1 -- 172.21.15.47:0/173502656 wait complete. 2024-03-21T10:33:41.297 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:41.293+0000 7fa88cf88740 20 client.18159 trim_cache size 0 max 16384 2024-03-21T10:33:41.297 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-03-21T10:33:41.297 INFO:teuthology.orchestra.run.smithi047.stdout: "status": "no active scrubs running", 2024-03-21T10:33:41.297 INFO:teuthology.orchestra.run.smithi047.stdout: "scrubs": {} 2024-03-21T10:33:41.297 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-03-21T10:33:41.298 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:41.294+0000 7fa888c00640 1 -- 172.21.15.47:0/3673891066 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7fa8680a2fb0 msgr2=0x7fa8680a5470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:33:41.298 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:41.294+0000 7fa888c00640 1 --2- 172.21.15.47:0/3673891066 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7fa8680a2fb0 0x7fa8680a5470 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fa87c006010 tx=0x7fa87c004a40 comp rx=0 tx=0).stop 2024-03-21T10:33:41.298 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:41.294+0000 7fa888c00640 1 -- 172.21.15.47:0/3673891066 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa884100dc0 msgr2=0x7fa88419a4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:33:41.298 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:41.294+0000 7fa888c00640 1 --2- 172.21.15.47:0/3673891066 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa884100dc0 0x7fa88419a4e0 secure :-1 s=READY pgs=1134 cs=0 l=1 rev1=1 crypto rx=0x7fa87000d770 tx=0x7fa87000dc30 comp rx=0 tx=0).stop 2024-03-21T10:33:41.298 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:41.294+0000 7fa888c00640 1 -- 172.21.15.47:0/3673891066 shutdown_connections 2024-03-21T10:33:41.298 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:41.294+0000 7fa888c00640 1 --2- 172.21.15.47:0/3673891066 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa8841064a0 0x7fa88419e860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:33:41.298 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:41.294+0000 7fa888c00640 1 --2- 172.21.15.47:0/3673891066 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa884100dc0 0x7fa88419a4e0 unknown :-1 s=CLOSED pgs=1134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:33:41.298 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:41.294+0000 7fa888c00640 1 --2- 172.21.15.47:0/3673891066 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7fa8680a2fb0 0x7fa8680a5470 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:33:41.299 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:41.294+0000 7fa888c00640 1 --2- 172.21.15.47:0/3673891066 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa884100400 0x7fa884199fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:33:41.299 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:41.294+0000 7fa888c00640 1 -- 172.21.15.47:0/3673891066 >> 172.21.15.47:0/3673891066 conn(0x7fa8840fbef0 msgr2=0x7fa88410c200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:33:41.299 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:41.294+0000 7fa888c00640 1 -- 172.21.15.47:0/3673891066 shutdown_connections 2024-03-21T10:33:41.299 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:41.294+0000 7fa888c00640 1 -- 172.21.15.47:0/3673891066 wait complete. 2024-03-21T10:33:41.311 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-03-21T10:33:41.311 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-21T10:33:41.535 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:41.536+0000 7f5e47800640 1 -- 172.21.15.47:0/504885166 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5e40073a70 msgr2=0x7f5e40073e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:33:41.535 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:41.536+0000 7f5e47800640 1 --2- 172.21.15.47:0/504885166 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5e40073a70 0x7f5e40073e50 secure :-1 s=READY pgs=1139 cs=0 l=1 rev1=1 crypto rx=0x7f5e30009900 tx=0x7f5e3002bcd0 comp rx=0 tx=0).stop 2024-03-21T10:33:41.535 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:41.536+0000 7f5e47800640 1 -- 172.21.15.47:0/504885166 shutdown_connections 2024-03-21T10:33:41.535 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:41.536+0000 7f5e47800640 1 --2- 172.21.15.47:0/504885166 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5e4010ded0 0x7f5e401102e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:33:41.535 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:41.536+0000 7f5e47800640 1 --2- 172.21.15.47:0/504885166 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5e40074390 0x7f5e4010d990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:33:41.535 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:41.536+0000 7f5e47800640 1 --2- 172.21.15.47:0/504885166 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5e40073a70 0x7f5e40073e50 unknown :-1 s=CLOSED pgs=1139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:33:41.535 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:41.536+0000 7f5e47800640 1 -- 172.21.15.47:0/504885166 >> 172.21.15.47:0/504885166 conn(0x7f5e400fd140 msgr2=0x7f5e400ff560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:33:41.535 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:41.536+0000 7f5e47800640 1 -- 172.21.15.47:0/504885166 shutdown_connections 2024-03-21T10:33:41.536 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:41.536+0000 7f5e47800640 1 -- 172.21.15.47:0/504885166 wait complete. 2024-03-21T10:33:41.537 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:41.537+0000 7f5e47800640 1 Processor -- start 2024-03-21T10:33:41.537 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:41.537+0000 7f5e47800640 1 -- start start 2024-03-21T10:33:41.537 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:41.537+0000 7f5e47800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5e40073a70 0x7f5e4019a0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:33:41.538 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:41.537+0000 7f5e47800640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5e40074390 0x7f5e4019a630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:33:41.538 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:41.537+0000 7f5e47800640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5e4010ded0 0x7f5e4019e9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:33:41.538 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:41.537+0000 7f5e47800640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e40113b30 con 0x7f5e4010ded0 2024-03-21T10:33:41.538 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:41.537+0000 7f5e47800640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e401139b0 con 0x7f5e40073a70 2024-03-21T10:33:41.538 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:41.537+0000 7f5e47800640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e40113cb0 con 0x7f5e40074390 2024-03-21T10:33:41.538 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:41.538+0000 7f5e45800640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5e4010ded0 0x7f5e4019e9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:33:41.538 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:41.538+0000 7f5e45800640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5e4010ded0 0x7f5e4019e9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:57892/0 (socket says 172.21.15.47:57892) 2024-03-21T10:33:41.538 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:41.538+0000 7f5e45800640 1 -- 172.21.15.47:0/2181225323 learned_addr learned my addr 172.21.15.47:0/2181225323 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:33:41.538 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:41.538+0000 7f5e44e00640 1 --2- 172.21.15.47:0/2181225323 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5e40073a70 0x7f5e4019a0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:33:41.538 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:41.538+0000 7f5e37e00640 1 --2- 172.21.15.47:0/2181225323 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5e40074390 0x7f5e4019a630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:33:41.538 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:41.538+0000 7f5e45800640 1 -- 172.21.15.47:0/2181225323 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5e40074390 msgr2=0x7f5e4019a630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:33:41.538 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:41.538+0000 7f5e45800640 1 --2- 172.21.15.47:0/2181225323 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5e40074390 0x7f5e4019a630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:33:41.538 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:41.538+0000 7f5e45800640 1 -- 172.21.15.47:0/2181225323 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5e40073a70 msgr2=0x7f5e4019a0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:33:41.539 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:41.538+0000 7f5e45800640 1 --2- 172.21.15.47:0/2181225323 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5e40073a70 0x7f5e4019a0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:33:41.539 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:41.538+0000 7f5e45800640 1 -- 172.21.15.47:0/2181225323 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e4019f0e0 con 0x7f5e4010ded0 2024-03-21T10:33:41.539 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:41.538+0000 7f5e37e00640 1 --2- 172.21.15.47:0/2181225323 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5e40074390 0x7f5e4019a630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T10:33:41.539 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:41.539+0000 7f5e45800640 1 --2- 172.21.15.47:0/2181225323 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5e4010ded0 0x7f5e4019e9b0 secure :-1 s=READY pgs=1140 cs=0 l=1 rev1=1 crypto rx=0x7f5e3800eb10 tx=0x7f5e3800c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:33:41.539 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:41.539+0000 7f5e3e000640 1 -- 172.21.15.47:0/2181225323 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e38004b20 con 0x7f5e4010ded0 2024-03-21T10:33:41.539 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:41.539+0000 7f5e3e000640 1 -- 172.21.15.47:0/2181225323 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f5e38004cc0 con 0x7f5e4010ded0 2024-03-21T10:33:41.539 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:41.539+0000 7f5e47800640 1 -- 172.21.15.47:0/2181225323 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5e4019f3d0 con 0x7f5e4010ded0 2024-03-21T10:33:41.539 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:41.539+0000 7f5e3e000640 1 -- 172.21.15.47:0/2181225323 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e38010720 con 0x7f5e4010ded0 2024-03-21T10:33:41.539 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:41.539+0000 7f5e47800640 1 -- 172.21.15.47:0/2181225323 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5e401a6c60 con 0x7f5e4010ded0 2024-03-21T10:33:41.539 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:41.540+0000 7f5e47800640 1 -- 172.21.15.47:0/2181225323 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5e400688e0 con 0x7f5e4010ded0 2024-03-21T10:33:41.543 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:41.545+0000 7f5e3e000640 1 -- 172.21.15.47:0/2181225323 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 27) v1 ==== 137985+0+0 (secure 0 0 0) 0x7f5e38005020 con 0x7f5e4010ded0 2024-03-21T10:33:41.545 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:41.547+0000 7f5e3e000640 1 --2- 172.21.15.47:0/2181225323 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f5e240a3090 0x7f5e240a5550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:33:41.545 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:41.547+0000 7f5e3e000640 1 -- 172.21.15.47:0/2181225323 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(101..101 src has 1..101) v4 ==== 9562+0+0 (secure 0 0 0) 0x7f5e380d0370 con 0x7f5e4010ded0 2024-03-21T10:33:41.545 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:41.547+0000 7f5e44e00640 1 --2- 172.21.15.47:0/2181225323 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f5e240a3090 0x7f5e240a5550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:33:41.549 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:41.548+0000 7f5e44e00640 1 --2- 172.21.15.47:0/2181225323 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f5e240a3090 0x7f5e240a5550 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f5e30005ab0 tx=0x7f5e30005a20 comp rx=0 tx=0).ready entity=mgr.15687 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:33:41.549 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:41.549+0000 7f5e3e000640 1 -- 172.21.15.47:0/2181225323 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f5e3809ca50 con 0x7f5e4010ded0 2024-03-21T10:33:41.711 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:33:41 smithi047 ceph-mon[31547]: pgmap v258: 129 pgs: 129 active+clean; 260 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-21T10:33:41.789 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:33:41 smithi157 ceph-mon[35581]: pgmap v258: 129 pgs: 129 active+clean; 260 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-21T10:33:41.853 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:41.855+0000 7f5e47800640 1 -- 172.21.15.47:0/2181225323 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f5e4002d070 con 0x7f5e4010ded0 2024-03-21T10:33:41.854 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:41.856+0000 7f5e3e000640 1 -- 172.21.15.47:0/2181225323 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 32 v32) v1 ==== 94+0+11471 (secure 0 0 0) 0x7f5e3809c350 con 0x7f5e4010ded0 2024-03-21T10:33:41.855 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-21T10:33:41.855 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":32,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14511,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6835/735724037","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6834","nonce":735724037},{"type":"v1","addr":"172.21.15.47:6835","nonce":735724037}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14529,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6837/2248191724","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6836","nonce":2248191724},{"type":"v1","addr":"172.21.15.47:6837","nonce":2248191724}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":32,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-21T10:21:58.329063+0000","modified":"2024-03-21T10:26:17.169892+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":34535,"mds_1":34493,"mds_2":34454,"mds_3":34427,"mds_4":14619},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14604":{"gid":14604,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.47:6839/2881335356","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6838","nonce":2881335356},{"type":"v1","addr":"172.21.15.47:6839","nonce":2881335356}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_14619":{"gid":14619,"name":"a","rank":4,"incarnation":27,"state":"up:active","state_seq":11,"addr":"172.21.15.47:6841/3401578978","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6840","nonce":3401578978},{"type":"v1","addr":"172.21.15.47:6841","nonce":3401578978}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24406":{"gid":24406,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.157:6833/4000010136","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":4000010136},{"type":"v1","addr":"172.21.15.157:6833","nonce":4000010136}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24457":{"gid":24457,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.157:6835/3902868651","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":3902868651},{"type":"v1","addr":"172.21.15.157:6835","nonce":3902868651}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24475":{"gid":24475,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.157:6837/1908087204","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6836","nonce":1908087204},{"type":"v1","addr":"172.21.15.157:6837","nonce":1908087204}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_25729":{"gid":25729,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.157:6839/1998408416","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6838","nonce":1998408416},{"type":"v1","addr":"172.21.15.157:6839","nonce":1998408416}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_34427":{"gid":34427,"name":"k","rank":3,"incarnation":25,"state":"up:active","state_seq":20,"addr":"172.21.15.100:6833/2723302038","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6832","nonce":2723302038},{"type":"v1","addr":"172.21.15.100:6833","nonce":2723302038}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_34454":{"gid":34454,"name":"e","rank":2,"incarnation":22,"state":"up:active","state_seq":17,"addr":"172.21.15.100:6835/2597849957","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":2597849957},{"type":"v1","addr":"172.21.15.100:6835","nonce":2597849957}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_34493":{"gid":34493,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":13,"addr":"172.21.15.100:6837/3497644162","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6836","nonce":3497644162},{"type":"v1","addr":"172.21.15.100:6837","nonce":3497644162}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_34535":{"gid":34535,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.100:6839/4173168440","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6838","nonce":4173168440},{"type":"v1","addr":"172.21.15.100:6839","nonce":4173168440}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":34535,"qdb_cluster":[34427,14619,34454,34535,34493]},"id":1}]} 2024-03-21T10:33:41.855 INFO:teuthology.orchestra.run.smithi047.stderr:dumped fsmap epoch 32 2024-03-21T10:33:41.857 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:41.859+0000 7f5e47800640 1 -- 172.21.15.47:0/2181225323 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f5e240a3090 msgr2=0x7f5e240a5550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:33:41.857 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:41.859+0000 7f5e47800640 1 --2- 172.21.15.47:0/2181225323 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f5e240a3090 0x7f5e240a5550 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f5e30005ab0 tx=0x7f5e30005a20 comp rx=0 tx=0).stop 2024-03-21T10:33:41.857 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:41.859+0000 7f5e47800640 1 -- 172.21.15.47:0/2181225323 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5e4010ded0 msgr2=0x7f5e4019e9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:33:41.857 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:41.859+0000 7f5e47800640 1 --2- 172.21.15.47:0/2181225323 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5e4010ded0 0x7f5e4019e9b0 secure :-1 s=READY pgs=1140 cs=0 l=1 rev1=1 crypto rx=0x7f5e3800eb10 tx=0x7f5e3800c2d0 comp rx=0 tx=0).stop 2024-03-21T10:33:41.858 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:41.859+0000 7f5e47800640 1 -- 172.21.15.47:0/2181225323 shutdown_connections 2024-03-21T10:33:41.858 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:41.859+0000 7f5e47800640 1 --2- 172.21.15.47:0/2181225323 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5e4010ded0 0x7f5e4019e9b0 unknown :-1 s=CLOSED pgs=1140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:33:41.858 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:41.859+0000 7f5e47800640 1 --2- 172.21.15.47:0/2181225323 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5e40074390 0x7f5e4019a630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:33:41.858 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:41.859+0000 7f5e47800640 1 --2- 172.21.15.47:0/2181225323 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f5e240a3090 0x7f5e240a5550 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:33:41.858 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:41.859+0000 7f5e47800640 1 --2- 172.21.15.47:0/2181225323 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5e40073a70 0x7f5e4019a0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:33:41.858 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:41.859+0000 7f5e47800640 1 -- 172.21.15.47:0/2181225323 >> 172.21.15.47:0/2181225323 conn(0x7f5e400fd140 msgr2=0x7f5e400fea00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:33:41.858 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:41.859+0000 7f5e47800640 1 -- 172.21.15.47:0/2181225323 shutdown_connections 2024-03-21T10:33:41.858 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:41.859+0000 7f5e47800640 1 -- 172.21.15.47:0/2181225323 wait complete. 2024-03-21T10:33:41.876 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-21T10:33:41.876 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-03-21T10:33:41.976 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:33:41 smithi100 ceph-mon[35193]: pgmap v258: 129 pgs: 129 active+clean; 260 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-21T10:33:42.090 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.092+0000 7f277fc00640 1 -- 172.21.15.47:0/2225658568 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2778103200 msgr2=0x7f277810b6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:33:42.090 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.092+0000 7f277fc00640 1 --2- 172.21.15.47:0/2225658568 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2778103200 0x7f277810b6e0 secure :-1 s=READY pgs=1141 cs=0 l=1 rev1=1 crypto rx=0x7f2764009860 tx=0x7f276402bc10 comp rx=0 tx=0).stop 2024-03-21T10:33:42.091 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.092+0000 7f277fc00640 1 -- 172.21.15.47:0/2225658568 shutdown_connections 2024-03-21T10:33:42.091 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.092+0000 7f277fc00640 1 --2- 172.21.15.47:0/2225658568 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f277810bcb0 0x7f277810e0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:33:42.091 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.092+0000 7f277fc00640 1 --2- 172.21.15.47:0/2225658568 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2778103200 0x7f277810b6e0 unknown :-1 s=CLOSED pgs=1141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:33:42.091 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.092+0000 7f277fc00640 1 --2- 172.21.15.47:0/2225658568 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f27781028e0 0x7f2778102cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:33:42.091 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.092+0000 7f277fc00640 1 -- 172.21.15.47:0/2225658568 >> 172.21.15.47:0/2225658568 conn(0x7f2778076920 msgr2=0x7f2778076d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:33:42.091 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.092+0000 7f277fc00640 1 -- 172.21.15.47:0/2225658568 shutdown_connections 2024-03-21T10:33:42.091 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.092+0000 7f277fc00640 1 -- 172.21.15.47:0/2225658568 wait complete. 2024-03-21T10:33:42.091 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.093+0000 7f277fc00640 1 Processor -- start 2024-03-21T10:33:42.092 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.094+0000 7f277fc00640 1 -- start start 2024-03-21T10:33:42.093 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.094+0000 7f277fc00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f27781028e0 0x7f2778100530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:33:42.093 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.094+0000 7f277fc00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2778103200 0x7f2778100a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:33:42.093 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.094+0000 7f277fc00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f277810bcb0 0x7f2778100fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:33:42.093 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.094+0000 7f277fc00640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27781118f0 con 0x7f2778103200 2024-03-21T10:33:42.093 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.094+0000 7f277fc00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2778111770 con 0x7f277810bcb0 2024-03-21T10:33:42.093 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.094+0000 7f277fc00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2778111a70 con 0x7f27781028e0 2024-03-21T10:33:42.093 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.094+0000 7f2777e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2778103200 0x7f2778100a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:33:42.093 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.095+0000 7f2777e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2778103200 0x7f2778100a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:58144/0 (socket says 172.21.15.47:58144) 2024-03-21T10:33:42.093 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.095+0000 7f2777e00640 1 -- 172.21.15.47:0/76657522 learned_addr learned my addr 172.21.15.47:0/76657522 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:33:42.094 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.095+0000 7f277dc00640 1 --2- 172.21.15.47:0/76657522 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f277810bcb0 0x7f2778100fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:33:42.094 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.095+0000 7f277d200640 1 --2- 172.21.15.47:0/76657522 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f27781028e0 0x7f2778100530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:33:42.094 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.095+0000 7f2777e00640 1 -- 172.21.15.47:0/76657522 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f27781028e0 msgr2=0x7f2778100530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:33:42.094 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.095+0000 7f2777e00640 1 --2- 172.21.15.47:0/76657522 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f27781028e0 0x7f2778100530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:33:42.094 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.095+0000 7f2777e00640 1 -- 172.21.15.47:0/76657522 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f277810bcb0 msgr2=0x7f2778100fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:33:42.094 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.095+0000 7f2777e00640 1 --2- 172.21.15.47:0/76657522 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f277810bcb0 0x7f2778100fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:33:42.094 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.095+0000 7f2777e00640 1 -- 172.21.15.47:0/76657522 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f27781a63c0 con 0x7f2778103200 2024-03-21T10:33:42.095 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.095+0000 7f2777e00640 1 --2- 172.21.15.47:0/76657522 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2778103200 0x7f2778100a70 secure :-1 s=READY pgs=1142 cs=0 l=1 rev1=1 crypto rx=0x7f2764009980 tx=0x7f2764005cd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:33:42.095 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.096+0000 7f2775600640 1 -- 172.21.15.47:0/76657522 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2764041020 con 0x7f2778103200 2024-03-21T10:33:42.095 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.096+0000 7f2775600640 1 -- 172.21.15.47:0/76657522 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f276402ca60 con 0x7f2778103200 2024-03-21T10:33:42.095 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.096+0000 7f277fc00640 1 -- 172.21.15.47:0/76657522 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f27781a6650 con 0x7f2778103200 2024-03-21T10:33:42.095 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.096+0000 7f2775600640 1 -- 172.21.15.47:0/76657522 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2764039040 con 0x7f2778103200 2024-03-21T10:33:42.095 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.096+0000 7f277fc00640 1 -- 172.21.15.47:0/76657522 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f27781a6b90 con 0x7f2778103200 2024-03-21T10:33:42.097 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.099+0000 7f2775600640 1 -- 172.21.15.47:0/76657522 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 27) v1 ==== 137985+0+0 (secure 0 0 0) 0x7f2764044020 con 0x7f2778103200 2024-03-21T10:33:42.097 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.099+0000 7f27829c1740 1 Processor -- start 2024-03-21T10:33:42.098 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.099+0000 7f27829c1740 1 -- start start 2024-03-21T10:33:42.101 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.100+0000 7f27829c1740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a1c5f8f6d0 0x55a1c5f8fab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:33:42.101 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.100+0000 7f27829c1740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a1c5f90080 0x55a1c5f98630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:33:42.101 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.100+0000 7f27829c1740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a1c5f98c00 0x55a1c5f9b050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:33:42.101 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.100+0000 7f27829c1740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a1c5ef32c0 con 0x55a1c5f8f6d0 2024-03-21T10:33:42.101 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.100+0000 7f27829c1740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a1c5e9bc90 con 0x55a1c5f98c00 2024-03-21T10:33:42.101 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.100+0000 7f27829c1740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a1c5eef360 con 0x55a1c5f90080 2024-03-21T10:33:42.101 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.100+0000 7f277dc00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a1c5f8f6d0 0x55a1c5f8fab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:33:42.101 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.100+0000 7f277dc00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a1c5f8f6d0 0x55a1c5f8fab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:58158/0 (socket says 172.21.15.47:58158) 2024-03-21T10:33:42.101 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.100+0000 7f277dc00640 1 -- 172.21.15.47:0/4210975096 learned_addr learned my addr 172.21.15.47:0/4210975096 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:33:42.101 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.100+0000 7f2777e00640 1 --2- 172.21.15.47:0/4210975096 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a1c5f98c00 0x55a1c5f9b050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:33:42.101 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.100+0000 7f277d200640 1 --2- 172.21.15.47:0/4210975096 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a1c5f90080 0x55a1c5f98630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:33:42.102 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.101+0000 7f277dc00640 1 -- 172.21.15.47:0/4210975096 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a1c5f90080 msgr2=0x55a1c5f98630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:33:42.102 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.101+0000 7f277dc00640 1 --2- 172.21.15.47:0/4210975096 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a1c5f90080 0x55a1c5f98630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:33:42.102 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.101+0000 7f277dc00640 1 -- 172.21.15.47:0/4210975096 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a1c5f98c00 msgr2=0x55a1c5f9b050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:33:42.102 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.101+0000 7f277dc00640 1 --2- 172.21.15.47:0/4210975096 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a1c5f98c00 0x55a1c5f9b050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:33:42.102 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.101+0000 7f277dc00640 1 -- 172.21.15.47:0/4210975096 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a1c5ec0080 con 0x55a1c5f8f6d0 2024-03-21T10:33:42.102 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.101+0000 7f277dc00640 1 --2- 172.21.15.47:0/4210975096 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a1c5f8f6d0 0x55a1c5f8fab0 secure :-1 s=READY pgs=1143 cs=0 l=1 rev1=1 crypto rx=0x7f27700099e0 tx=0x7f27700095c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a92260ba7b722017 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:33:42.102 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.101+0000 7f2775600640 1 --2- 172.21.15.47:0/76657522 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f27600a3110 0x7f27600a55d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:33:42.102 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.101+0000 7f276a000640 1 -- 172.21.15.47:0/4210975096 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2770003970 con 0x55a1c5f8f6d0 2024-03-21T10:33:42.102 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.101+0000 7f2775600640 1 -- 172.21.15.47:0/76657522 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(101..101 src has 1..101) v4 ==== 9562+0+0 (secure 0 0 0) 0x7f27640f0ff0 con 0x7f2778103200 2024-03-21T10:33:42.102 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.101+0000 7f276a000640 1 -- 172.21.15.47:0/4210975096 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f277000d6b0 con 0x55a1c5f8f6d0 2024-03-21T10:33:42.102 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.101+0000 7f277d200640 1 --2- 172.21.15.47:0/76657522 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f27600a3110 0x7f27600a55d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:33:42.102 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.102+0000 7f276a000640 1 -- 172.21.15.47:0/4210975096 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2770004280 con 0x55a1c5f8f6d0 2024-03-21T10:33:42.102 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.102+0000 7f27829c1740 1 -- 172.21.15.47:0/4210975096 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a1c5f8f6d0 msgr2=0x55a1c5f8fab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:33:42.102 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.102+0000 7f27829c1740 1 --2- 172.21.15.47:0/4210975096 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a1c5f8f6d0 0x55a1c5f8fab0 secure :-1 s=READY pgs=1143 cs=0 l=1 rev1=1 crypto rx=0x7f27700099e0 tx=0x7f27700095c0 comp rx=0 tx=0).stop 2024-03-21T10:33:42.103 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.102+0000 7f277d200640 1 --2- 172.21.15.47:0/76657522 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f27600a3110 0x7f27600a55d0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x55a1c5f8ae10 tx=0x7f276c005f90 comp rx=0 tx=0).ready entity=mgr.15687 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:33:42.103 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.102+0000 7f27829c1740 1 -- 172.21.15.47:0/4210975096 shutdown_connections 2024-03-21T10:33:42.103 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.102+0000 7f27829c1740 1 --2- 172.21.15.47:0/4210975096 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a1c5f98c00 0x55a1c5f9b050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:33:42.103 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.102+0000 7f27829c1740 1 --2- 172.21.15.47:0/4210975096 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a1c5f90080 0x55a1c5f98630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:33:42.103 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.102+0000 7f27829c1740 1 --2- 172.21.15.47:0/4210975096 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a1c5f8f6d0 0x55a1c5f8fab0 unknown :-1 s=CLOSED pgs=1143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:33:42.103 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.102+0000 7f27829c1740 1 -- 172.21.15.47:0/4210975096 >> 172.21.15.47:0/4210975096 conn(0x55a1c5f89da0 msgr2=0x55a1c5f8a180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:33:42.103 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.102+0000 7f27829c1740 1 -- 172.21.15.47:0/4210975096 shutdown_connections 2024-03-21T10:33:42.103 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.103+0000 7f27829c1740 1 -- 172.21.15.47:0/4210975096 wait complete. 2024-03-21T10:33:42.107 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.103+0000 7f27829c1740 1 -- start start 2024-03-21T10:33:42.107 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.103+0000 7f27829c1740 1 Processor -- start 2024-03-21T10:33:42.107 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.104+0000 7f27829c1740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a1c5f8f6d0 0x55a1c5f98570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:33:42.107 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.104+0000 7f27829c1740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a1c5f90080 0x55a1c5f95900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:33:42.107 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.104+0000 7f27829c1740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a1c5f98c00 0x55a1c5f95f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:33:42.107 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.104+0000 7f27829c1740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a1c5f9bb70 con 0x55a1c5f98c00 2024-03-21T10:33:42.107 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.104+0000 7f27829c1740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a1c5eaf0d0 con 0x55a1c5f90080 2024-03-21T10:33:42.108 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.104+0000 7f27829c1740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a1c5f9bcf0 con 0x55a1c5f8f6d0 2024-03-21T10:33:42.108 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.104+0000 7f27829c1740 10 client.0 _ll_register_callbacks cb 0x55a1c5eb3e80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T10:33:42.108 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.104+0000 7f277dc00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a1c5f98c00 0x55a1c5f95f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:33:42.108 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.104+0000 7f2777e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a1c5f90080 0x55a1c5f95900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:33:42.108 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.104+0000 7f277dc00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a1c5f98c00 0x55a1c5f95f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:58164/0 (socket says 172.21.15.47:58164) 2024-03-21T10:33:42.108 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.104+0000 7f277dc00640 1 -- 172.21.15.47:0/234209651 learned_addr learned my addr 172.21.15.47:0/234209651 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:33:42.108 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.104+0000 7f2777e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a1c5f90080 0x55a1c5f95900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:36978/0 (socket says 172.21.15.47:36978) 2024-03-21T10:33:42.108 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.104+0000 7f277d200640 1 --2- 172.21.15.47:0/234209651 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a1c5f8f6d0 0x55a1c5f98570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:33:42.108 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.104+0000 7f277dc00640 1 -- 172.21.15.47:0/234209651 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a1c5f8f6d0 msgr2=0x55a1c5f98570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:33:42.108 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.104+0000 7f277dc00640 1 --2- 172.21.15.47:0/234209651 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a1c5f8f6d0 0x55a1c5f98570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:33:42.108 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.104+0000 7f277dc00640 1 -- 172.21.15.47:0/234209651 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a1c5f90080 msgr2=0x55a1c5f95900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:33:42.108 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.104+0000 7f277dc00640 1 --2- 172.21.15.47:0/234209651 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a1c5f90080 0x55a1c5f95900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:33:42.108 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.104+0000 7f277dc00640 1 -- 172.21.15.47:0/234209651 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55a1c5c073f0 con 0x55a1c5f98c00 2024-03-21T10:33:42.109 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.104+0000 7f2777e00640 1 --2- 172.21.15.47:0/234209651 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a1c5f90080 0x55a1c5f95900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T10:33:42.109 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.105+0000 7f277dc00640 1 --2- 172.21.15.47:0/234209651 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a1c5f98c00 0x55a1c5f95f10 secure :-1 s=READY pgs=1144 cs=0 l=1 rev1=1 crypto rx=0x7f27700099b0 tx=0x7f277000db10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e8bd61aa2fbbfa02 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:33:42.109 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.105+0000 7f275ea00640 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-03-21T10:33:42.109 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.105+0000 7f275ea00640 1 -- 172.21.15.47:0/234209651 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2770025070 con 0x55a1c5f98c00 2024-03-21T10:33:42.109 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.105+0000 7f275ea00640 1 -- 172.21.15.47:0/234209651 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(101..101 src has 1..101) v4 ==== 9562+0+0 (secure 0 0 0) 0x7f2770021630 con 0x55a1c5f98c00 2024-03-21T10:33:42.109 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.105+0000 7f27829c1740 1 -- 172.21.15.47:0/234209651 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f277000d6b0 con 0x55a1c5f98c00 2024-03-21T10:33:42.109 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.105+0000 7f275ea00640 1 -- 172.21.15.47:0/234209651 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2770033050 con 0x55a1c5f98c00 2024-03-21T10:33:42.109 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.105+0000 7f27829c1740 10 client.18177 fetch_fsmap learned FSMap version 32 2024-03-21T10:33:42.109 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.105+0000 7f27829c1740 1 -- 172.21.15.47:0/234209651 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x55a1c5cd17d0 con 0x55a1c5f98c00 2024-03-21T10:33:42.109 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.106+0000 7f275ea00640 1 -- 172.21.15.47:0/234209651 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f277000d6b0 con 0x55a1c5f98c00 2024-03-21T10:33:42.109 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.106+0000 7f275ea00640 1 -- 172.21.15.47:0/234209651 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 32) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f277000b040 con 0x55a1c5f98c00 2024-03-21T10:33:42.109 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.106+0000 7f27829c1740 10 client.18177 fetch_fsmap finished waiting for FSMap version 32 2024-03-21T10:33:42.109 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.106+0000 7f27829c1740 10 client.18177 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.a 2024-03-21T10:33:42.110 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.106+0000 7f27829c1740 20 client.18177 populate_metadata read hostname 'smithi047' 2024-03-21T10:33:42.110 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.106+0000 7f27829c1740 1 --2- 172.21.15.47:0/234209651 >> [v2:172.21.15.47:6840/3401578978,v1:172.21.15.47:6841/3401578978] conn(0x55a1c5fa8a60 0x55a1c5faae50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:33:42.110 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.106+0000 7f27829c1740 4 client.18177 mds_command: new command op to 14619 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T10:33:42.110 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.106+0000 7f27829c1740 1 -- 172.21.15.47:0/234209651 --> [v2:172.21.15.47:6840/3401578978,v1:172.21.15.47:6841/3401578978] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55a1c5f00800 con 0x55a1c5fa8a60 2024-03-21T10:33:42.110 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.106+0000 7f277d200640 1 --2- 172.21.15.47:0/234209651 >> [v2:172.21.15.47:6840/3401578978,v1:172.21.15.47:6841/3401578978] conn(0x55a1c5fa8a60 0x55a1c5faae50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:33:42.110 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.107+0000 7f277d200640 1 --2- 172.21.15.47:0/234209651 >> [v2:172.21.15.47:6840/3401578978,v1:172.21.15.47:6841/3401578978] conn(0x55a1c5fa8a60 0x55a1c5faae50 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=f724e2e5d0c91821 server_cookie=b0be85b5158107ce in_seq=0 out_seq=0 2024-03-21T10:33:42.110 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.107+0000 7f275ea00640 10 client.18177 ms_handle_connect on v2:172.21.15.47:6840/3401578978 2024-03-21T10:33:42.110 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.108+0000 7f275ea00640 1 -- 172.21.15.47:0/234209651 <== mds.4 v2:172.21.15.47:6840/3401578978 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55a1c5f00800 con 0x55a1c5fa8a60 2024-03-21T10:33:42.110 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.108+0000 7f275ea00640 10 client.18177 handle_command_reply: tid=0 2024-03-21T10:33:42.110 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.108+0000 7f27829c1740 1 client.18177 shutdown 2024-03-21T10:33:42.110 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.108+0000 7f27829c1740 1 -- 172.21.15.47:0/234209651 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a1c5f98c00 msgr2=0x55a1c5f95f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:33:42.110 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.108+0000 7f27829c1740 1 --2- 172.21.15.47:0/234209651 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a1c5f98c00 0x55a1c5f95f10 secure :-1 s=READY pgs=1144 cs=0 l=1 rev1=1 crypto rx=0x7f27700099b0 tx=0x7f277000db10 comp rx=0 tx=0).stop 2024-03-21T10:33:42.110 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.108+0000 7f27829c1740 1 -- 172.21.15.47:0/234209651 shutdown_connections 2024-03-21T10:33:42.111 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.108+0000 7f27829c1740 1 --2- 172.21.15.47:0/234209651 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a1c5f98c00 0x55a1c5f95f10 unknown :-1 s=CLOSED pgs=1144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:33:42.111 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.108+0000 7f27829c1740 1 --2- 172.21.15.47:0/234209651 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a1c5f90080 0x55a1c5f95900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:33:42.111 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.108+0000 7f27829c1740 1 --2- 172.21.15.47:0/234209651 >> [v2:172.21.15.47:6840/3401578978,v1:172.21.15.47:6841/3401578978] conn(0x55a1c5fa8a60 0x55a1c5faae50 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:33:42.111 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.108+0000 7f27829c1740 1 --2- 172.21.15.47:0/234209651 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a1c5f8f6d0 0x55a1c5f98570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:33:42.111 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.108+0000 7f27829c1740 1 -- 172.21.15.47:0/234209651 >> 172.21.15.47:0/234209651 conn(0x55a1c5f89da0 msgr2=0x55a1c5f8baf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:33:42.111 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.108+0000 7f275ea00640 0 client.18177 ms_handle_reset on v2:172.21.15.47:6840/3401578978 2024-03-21T10:33:42.111 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.108+0000 7f27829c1740 1 -- 172.21.15.47:0/234209651 shutdown_connections 2024-03-21T10:33:42.111 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.108+0000 7f27829c1740 1 -- 172.21.15.47:0/234209651 wait complete. 2024-03-21T10:33:42.111 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.109+0000 7f27829c1740 20 client.18177 trim_cache size 0 max 16384 2024-03-21T10:33:42.120 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.122+0000 7f27829c1740 1 Processor -- start 2024-03-21T10:33:42.121 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.122+0000 7f27829c1740 1 -- start start 2024-03-21T10:33:42.122 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.123+0000 7f27829c1740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a1c5f8f6d0 0x55a1c5fa4f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:33:42.122 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.123+0000 7f27829c1740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a1c5f90080 0x55a1c5f98030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:33:42.122 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.123+0000 7f27829c1740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a1c5f98c00 0x55a1c5f98570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:33:42.122 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.123+0000 7f27829c1740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27781b0bd0 con 0x55a1c5f98c00 2024-03-21T10:33:42.122 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.123+0000 7f27829c1740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27781b2550 con 0x55a1c5f8f6d0 2024-03-21T10:33:42.122 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.123+0000 7f27829c1740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27781b1230 con 0x55a1c5f90080 2024-03-21T10:33:42.122 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.123+0000 7f277dc00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a1c5f98c00 0x55a1c5f98570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:33:42.122 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.123+0000 7f277d200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a1c5f8f6d0 0x55a1c5fa4f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:33:42.122 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.123+0000 7f277dc00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a1c5f98c00 0x55a1c5f98570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:58172/0 (socket says 172.21.15.47:58172) 2024-03-21T10:33:42.123 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.123+0000 7f277dc00640 1 -- 172.21.15.47:0/2682181450 learned_addr learned my addr 172.21.15.47:0/2682181450 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:33:42.123 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.123+0000 7f277d200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a1c5f8f6d0 0x55a1c5fa4f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:36988/0 (socket says 172.21.15.47:36988) 2024-03-21T10:33:42.123 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.123+0000 7f2777e00640 1 --2- 172.21.15.47:0/2682181450 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a1c5f90080 0x55a1c5f98030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:33:42.123 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.124+0000 7f277dc00640 1 -- 172.21.15.47:0/2682181450 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a1c5f90080 msgr2=0x55a1c5f98030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:33:42.123 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.124+0000 7f277dc00640 1 --2- 172.21.15.47:0/2682181450 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a1c5f90080 0x55a1c5f98030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:33:42.123 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.124+0000 7f277dc00640 1 -- 172.21.15.47:0/2682181450 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a1c5f8f6d0 msgr2=0x55a1c5fa4f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:33:42.123 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.124+0000 7f277dc00640 1 --2- 172.21.15.47:0/2682181450 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a1c5f8f6d0 0x55a1c5fa4f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:33:42.123 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.124+0000 7f277dc00640 1 -- 172.21.15.47:0/2682181450 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a1c5f8fe20 con 0x55a1c5f98c00 2024-03-21T10:33:42.123 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.124+0000 7f277dc00640 1 --2- 172.21.15.47:0/2682181450 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a1c5f98c00 0x55a1c5f98570 secure :-1 s=READY pgs=1145 cs=0 l=1 rev1=1 crypto rx=0x7f2770006010 tx=0x7f2770009ee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8836a82557d9e0f5 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:33:42.126 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.124+0000 7f2769600640 1 -- 172.21.15.47:0/2682181450 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2770025070 con 0x55a1c5f98c00 2024-03-21T10:33:42.126 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.124+0000 7f2769600640 1 -- 172.21.15.47:0/2682181450 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f277000d6b0 con 0x55a1c5f98c00 2024-03-21T10:33:42.126 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.124+0000 7f2769600640 1 -- 172.21.15.47:0/2682181450 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f277000db80 con 0x55a1c5f98c00 2024-03-21T10:33:42.126 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.125+0000 7f27829c1740 1 -- 172.21.15.47:0/2682181450 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a1c5f98c00 msgr2=0x55a1c5f98570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:33:42.126 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.125+0000 7f27829c1740 1 --2- 172.21.15.47:0/2682181450 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a1c5f98c00 0x55a1c5f98570 secure :-1 s=READY pgs=1145 cs=0 l=1 rev1=1 crypto rx=0x7f2770006010 tx=0x7f2770009ee0 comp rx=0 tx=0).stop 2024-03-21T10:33:42.126 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.125+0000 7f27829c1740 1 -- 172.21.15.47:0/2682181450 shutdown_connections 2024-03-21T10:33:42.126 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.125+0000 7f27829c1740 1 --2- 172.21.15.47:0/2682181450 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a1c5f98c00 0x55a1c5f98570 unknown :-1 s=CLOSED pgs=1145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:33:42.126 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.125+0000 7f27829c1740 1 --2- 172.21.15.47:0/2682181450 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a1c5f90080 0x55a1c5f98030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:33:42.126 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.125+0000 7f27829c1740 1 --2- 172.21.15.47:0/2682181450 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a1c5f8f6d0 0x55a1c5fa4f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:33:42.126 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.125+0000 7f27829c1740 1 -- 172.21.15.47:0/2682181450 >> 172.21.15.47:0/2682181450 conn(0x55a1c5f89da0 msgr2=0x55a1c5f8ba40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:33:42.126 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.125+0000 7f27829c1740 1 -- 172.21.15.47:0/2682181450 shutdown_connections 2024-03-21T10:33:42.126 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.125+0000 7f27829c1740 1 -- 172.21.15.47:0/2682181450 wait complete. 2024-03-21T10:33:42.127 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.125+0000 7f27829c1740 1 -- start start 2024-03-21T10:33:42.127 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.126+0000 7f27829c1740 1 Processor -- start 2024-03-21T10:33:42.127 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.126+0000 7f27829c1740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a1c5f8f6d0 0x55a1c5fab5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:33:42.127 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.126+0000 7f27829c1740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a1c5f90080 0x55a1c5fabb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:33:42.127 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.126+0000 7f27829c1740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a1c5f98c00 0x55a1c6002f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:33:42.127 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.126+0000 7f27829c1740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27781b1ef0 con 0x55a1c5f98c00 2024-03-21T10:33:42.127 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.126+0000 7f27829c1740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27781b1890 con 0x55a1c5f8f6d0 2024-03-21T10:33:42.127 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.126+0000 7f27829c1740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27781aedb0 con 0x55a1c5f90080 2024-03-21T10:33:42.127 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.126+0000 7f27829c1740 10 client.0 _ll_register_callbacks cb 0x7f27640f4990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T10:33:42.127 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.127+0000 7f277d200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a1c5f8f6d0 0x55a1c5fab5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:33:42.127 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.127+0000 7f277dc00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a1c5f98c00 0x55a1c6002f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:33:42.127 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.127+0000 7f277d200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a1c5f8f6d0 0x55a1c5fab5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:36994/0 (socket says 172.21.15.47:36994) 2024-03-21T10:33:42.127 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.127+0000 7f277d200640 1 -- 172.21.15.47:0/3057059265 learned_addr learned my addr 172.21.15.47:0/3057059265 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:33:42.127 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.127+0000 7f2777e00640 1 --2- 172.21.15.47:0/3057059265 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a1c5f90080 0x55a1c5fabb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:33:42.127 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.127+0000 7f277d200640 1 -- 172.21.15.47:0/3057059265 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a1c5f90080 msgr2=0x55a1c5fabb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:33:42.128 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.127+0000 7f277d200640 1 --2- 172.21.15.47:0/3057059265 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a1c5f90080 0x55a1c5fabb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:33:42.128 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.127+0000 7f277d200640 1 -- 172.21.15.47:0/3057059265 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a1c5f98c00 msgr2=0x55a1c6002f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:33:42.128 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.127+0000 7f277d200640 1 --2- 172.21.15.47:0/3057059265 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a1c5f98c00 0x55a1c6002f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:33:42.128 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.127+0000 7f277d200640 1 -- 172.21.15.47:0/3057059265 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55a1c5fac3a0 con 0x55a1c5f8f6d0 2024-03-21T10:33:42.128 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.127+0000 7f2777e00640 1 --2- 172.21.15.47:0/3057059265 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a1c5f90080 0x55a1c5fabb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T10:33:42.128 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.127+0000 7f277d200640 1 --2- 172.21.15.47:0/3057059265 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a1c5f8f6d0 0x55a1c5fab5c0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f276c009fd0 tx=0x7f276c0046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1ebd73e0d5100339 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:33:42.128 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.127+0000 7f275ea00640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-21T10:33:42.130 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.128+0000 7f275ea00640 1 -- 172.21.15.47:0/3057059265 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f276c005220 con 0x55a1c5f8f6d0 2024-03-21T10:33:42.130 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.128+0000 7f27829c1740 1 -- 172.21.15.47:0/3057059265 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f277000d6b0 con 0x55a1c5f8f6d0 2024-03-21T10:33:42.130 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.128+0000 7f275ea00640 1 -- 172.21.15.47:0/3057059265 <== mon.2 v2:172.21.15.100:3300/0 2 ==== osd_map(101..101 src has 1..101) v4 ==== 9562+0+0 (secure 0 0 0) 0x7f276c0053c0 con 0x55a1c5f8f6d0 2024-03-21T10:33:42.131 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.128+0000 7f275ea00640 1 -- 172.21.15.47:0/3057059265 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f276c0055c0 con 0x55a1c5f8f6d0 2024-03-21T10:33:42.131 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.128+0000 7f27829c1740 10 client.37946 fetch_fsmap learned FSMap version 32 2024-03-21T10:33:42.131 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.128+0000 7f27829c1740 1 -- 172.21.15.47:0/3057059265 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x55a1c6004600 con 0x55a1c5f8f6d0 2024-03-21T10:33:42.131 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.129+0000 7f275ea00640 1 -- 172.21.15.47:0/3057059265 <== mon.2 v2:172.21.15.100:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f277000d6b0 con 0x55a1c5f8f6d0 2024-03-21T10:33:42.131 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.129+0000 7f275ea00640 1 -- 172.21.15.47:0/3057059265 <== mon.2 v2:172.21.15.100:3300/0 5 ==== fsmap(e 32) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f276c031070 con 0x55a1c5f8f6d0 2024-03-21T10:33:42.131 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.129+0000 7f27829c1740 10 client.37946 fetch_fsmap finished waiting for FSMap version 32 2024-03-21T10:33:42.131 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.129+0000 7f27829c1740 10 client.37946 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.a 2024-03-21T10:33:42.131 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.129+0000 7f27829c1740 20 client.37946 populate_metadata read hostname 'smithi047' 2024-03-21T10:33:42.131 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.129+0000 7f27829c1740 1 --2- 172.21.15.47:0/3057059265 >> [v2:172.21.15.47:6840/3401578978,v1:172.21.15.47:6841/3401578978] conn(0x55a1c6004d30 0x55a1c6007150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:33:42.131 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.129+0000 7f27829c1740 4 client.37946 mds_command: new command op to 14619 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-21T10:33:42.131 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.129+0000 7f27829c1740 1 -- 172.21.15.47:0/3057059265 --> [v2:172.21.15.47:6840/3401578978,v1:172.21.15.47:6841/3401578978] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55a1c5efab50 con 0x55a1c6004d30 2024-03-21T10:33:42.131 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.130+0000 7f2777e00640 1 --2- 172.21.15.47:0/3057059265 >> [v2:172.21.15.47:6840/3401578978,v1:172.21.15.47:6841/3401578978] conn(0x55a1c6004d30 0x55a1c6007150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:33:42.131 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.130+0000 7f2777e00640 1 --2- 172.21.15.47:0/3057059265 >> [v2:172.21.15.47:6840/3401578978,v1:172.21.15.47:6841/3401578978] conn(0x55a1c6004d30 0x55a1c6007150 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=9479b16530f6187 server_cookie=49db0d08f0feb1f5 in_seq=0 out_seq=0 2024-03-21T10:33:42.132 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.130+0000 7f275ea00640 10 client.37946 ms_handle_connect on v2:172.21.15.47:6840/3401578978 2024-03-21T10:33:42.132 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.130+0000 7f275ea00640 1 -- 172.21.15.47:0/3057059265 <== mds.4 v2:172.21.15.47:6840/3401578978 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55a1c5efab50 con 0x55a1c6004d30 2024-03-21T10:33:42.132 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.130+0000 7f275ea00640 10 client.37946 handle_command_reply: tid=0 2024-03-21T10:33:42.132 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.130+0000 7f27829c1740 1 client.37946 shutdown 2024-03-21T10:33:42.132 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.131+0000 7f27829c1740 1 -- 172.21.15.47:0/3057059265 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a1c5f8f6d0 msgr2=0x55a1c5fab5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:33:42.132 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.131+0000 7f27829c1740 1 --2- 172.21.15.47:0/3057059265 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a1c5f8f6d0 0x55a1c5fab5c0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f276c009fd0 tx=0x7f276c0046c0 comp rx=0 tx=0).stop 2024-03-21T10:33:42.132 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.131+0000 7f27829c1740 1 -- 172.21.15.47:0/3057059265 shutdown_connections 2024-03-21T10:33:42.132 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.131+0000 7f27829c1740 1 --2- 172.21.15.47:0/3057059265 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a1c5f98c00 0x55a1c6002f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:33:42.132 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.131+0000 7f27829c1740 1 --2- 172.21.15.47:0/3057059265 >> [v2:172.21.15.47:6840/3401578978,v1:172.21.15.47:6841/3401578978] conn(0x55a1c6004d30 0x55a1c6007150 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:33:42.132 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.131+0000 7f27829c1740 1 --2- 172.21.15.47:0/3057059265 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a1c5f90080 0x55a1c5fabb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:33:42.132 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.131+0000 7f27829c1740 1 --2- 172.21.15.47:0/3057059265 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a1c5f8f6d0 0x55a1c5fab5c0 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:33:42.132 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.131+0000 7f27829c1740 1 -- 172.21.15.47:0/3057059265 >> 172.21.15.47:0/3057059265 conn(0x55a1c5f89da0 msgr2=0x55a1c5f88eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:33:42.132 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.131+0000 7f275ea00640 0 client.37946 ms_handle_reset on v2:172.21.15.47:6840/3401578978 2024-03-21T10:33:42.133 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.131+0000 7f27829c1740 1 -- 172.21.15.47:0/3057059265 shutdown_connections 2024-03-21T10:33:42.133 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.131+0000 7f27829c1740 1 -- 172.21.15.47:0/3057059265 wait complete. 2024-03-21T10:33:42.133 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.131+0000 7f27829c1740 20 client.37946 trim_cache size 0 max 16384 2024-03-21T10:33:42.133 INFO:teuthology.orchestra.run.smithi047.stdout:[] 2024-03-21T10:33:42.133 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.132+0000 7f277fc00640 1 -- 172.21.15.47:0/76657522 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f27600a3110 msgr2=0x7f27600a55d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:33:42.134 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.132+0000 7f277fc00640 1 --2- 172.21.15.47:0/76657522 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f27600a3110 0x7f27600a55d0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x55a1c5f8ae10 tx=0x7f276c005f90 comp rx=0 tx=0).stop 2024-03-21T10:33:42.134 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.132+0000 7f277fc00640 1 -- 172.21.15.47:0/76657522 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2778103200 msgr2=0x7f2778100a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:33:42.134 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.132+0000 7f277fc00640 1 --2- 172.21.15.47:0/76657522 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2778103200 0x7f2778100a70 secure :-1 s=READY pgs=1142 cs=0 l=1 rev1=1 crypto rx=0x7f2764009980 tx=0x7f2764005cd0 comp rx=0 tx=0).stop 2024-03-21T10:33:42.134 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.132+0000 7f277fc00640 1 -- 172.21.15.47:0/76657522 shutdown_connections 2024-03-21T10:33:42.134 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.132+0000 7f277fc00640 1 --2- 172.21.15.47:0/76657522 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f27600a3110 0x7f27600a55d0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:33:42.134 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.132+0000 7f277fc00640 1 --2- 172.21.15.47:0/76657522 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f277810bcb0 0x7f2778100fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:33:42.134 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.132+0000 7f277fc00640 1 --2- 172.21.15.47:0/76657522 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2778103200 0x7f2778100a70 unknown :-1 s=CLOSED pgs=1142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:33:42.134 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.132+0000 7f277fc00640 1 --2- 172.21.15.47:0/76657522 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f27781028e0 0x7f2778100530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:33:42.134 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.132+0000 7f277fc00640 1 -- 172.21.15.47:0/76657522 >> 172.21.15.47:0/76657522 conn(0x7f2778076920 msgr2=0x7f27780fd5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:33:42.134 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.132+0000 7f277fc00640 1 -- 172.21.15.47:0/76657522 shutdown_connections 2024-03-21T10:33:42.134 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.132+0000 7f277fc00640 1 -- 172.21.15.47:0/76657522 wait complete. 2024-03-21T10:33:42.148 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-21T10:33:42.148 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-03-21T10:33:42.373 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.374+0000 7f3567e00640 1 -- 172.21.15.47:0/1105198849 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3568103050 msgr2=0x7f356810b530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:33:42.373 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.374+0000 7f3567e00640 1 --2- 172.21.15.47:0/1105198849 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3568103050 0x7f356810b530 secure :-1 s=READY pgs=1146 cs=0 l=1 rev1=1 crypto rx=0x7f3554009900 tx=0x7f355402bcc0 comp rx=0 tx=0).stop 2024-03-21T10:33:42.373 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.374+0000 7f3567e00640 1 -- 172.21.15.47:0/1105198849 shutdown_connections 2024-03-21T10:33:42.373 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.374+0000 7f3567e00640 1 --2- 172.21.15.47:0/1105198849 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f356810bb00 0x7f356810df80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:33:42.373 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.374+0000 7f3567e00640 1 --2- 172.21.15.47:0/1105198849 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3568103050 0x7f356810b530 unknown :-1 s=CLOSED pgs=1146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:33:42.373 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.374+0000 7f3567e00640 1 --2- 172.21.15.47:0/1105198849 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3568102730 0x7f3568102b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:33:42.373 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.374+0000 7f3567e00640 1 -- 172.21.15.47:0/1105198849 >> 172.21.15.47:0/1105198849 conn(0x7f35680fbef0 msgr2=0x7f35680fe310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:33:42.373 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.374+0000 7f3567e00640 1 -- 172.21.15.47:0/1105198849 shutdown_connections 2024-03-21T10:33:42.374 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.374+0000 7f3567e00640 1 -- 172.21.15.47:0/1105198849 wait complete. 2024-03-21T10:33:42.375 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.375+0000 7f3567e00640 1 Processor -- start 2024-03-21T10:33:42.375 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.376+0000 7f3567e00640 1 -- start start 2024-03-21T10:33:42.375 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.376+0000 7f3567e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3568102730 0x7f3568199f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:33:42.375 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.376+0000 7f3567e00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3568103050 0x7f356819a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:33:42.376 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.376+0000 7f3567e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f356810bb00 0x7f356819e800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:33:42.376 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.376+0000 7f3567e00640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3568111780 con 0x7f356810bb00 2024-03-21T10:33:42.376 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.376+0000 7f3567e00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3568111600 con 0x7f3568102730 2024-03-21T10:33:42.376 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.376+0000 7f3567e00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3568111900 con 0x7f3568103050 2024-03-21T10:33:42.376 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.376+0000 7f3566a00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3568102730 0x7f3568199f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:33:42.376 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.376+0000 7f3567400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f356810bb00 0x7f356819e800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:33:42.376 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.376+0000 7f3566a00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3568102730 0x7f3568199f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:37022/0 (socket says 172.21.15.47:37022) 2024-03-21T10:33:42.376 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.376+0000 7f3566a00640 1 -- 172.21.15.47:0/2360057951 learned_addr learned my addr 172.21.15.47:0/2360057951 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:33:42.376 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.377+0000 7f3566000640 1 --2- 172.21.15.47:0/2360057951 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3568103050 0x7f356819a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:33:42.376 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.377+0000 7f3567400640 1 -- 172.21.15.47:0/2360057951 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3568103050 msgr2=0x7f356819a480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:33:42.376 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.377+0000 7f3567400640 1 --2- 172.21.15.47:0/2360057951 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3568103050 0x7f356819a480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:33:42.376 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.377+0000 7f3567400640 1 -- 172.21.15.47:0/2360057951 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3568102730 msgr2=0x7f3568199f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:33:42.376 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.377+0000 7f3567400640 1 --2- 172.21.15.47:0/2360057951 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3568102730 0x7f3568199f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:33:42.376 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.377+0000 7f3567400640 1 -- 172.21.15.47:0/2360057951 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f356819ef30 con 0x7f356810bb00 2024-03-21T10:33:42.377 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.377+0000 7f3567400640 1 --2- 172.21.15.47:0/2360057951 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f356810bb00 0x7f356819e800 secure :-1 s=READY pgs=1147 cs=0 l=1 rev1=1 crypto rx=0x7f356000e940 tx=0x7f356000ee00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:33:42.377 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.378+0000 7f355f400640 1 -- 172.21.15.47:0/2360057951 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3560004d20 con 0x7f356810bb00 2024-03-21T10:33:42.377 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.378+0000 7f355f400640 1 -- 172.21.15.47:0/2360057951 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f3560005020 con 0x7f356810bb00 2024-03-21T10:33:42.377 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.378+0000 7f355f400640 1 -- 172.21.15.47:0/2360057951 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35600116d0 con 0x7f356810bb00 2024-03-21T10:33:42.377 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.378+0000 7f3567e00640 1 -- 172.21.15.47:0/2360057951 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f356819f220 con 0x7f356810bb00 2024-03-21T10:33:42.378 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.378+0000 7f3567e00640 1 -- 172.21.15.47:0/2360057951 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f35681a6ab0 con 0x7f356810bb00 2024-03-21T10:33:42.379 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.381+0000 7f355f400640 1 -- 172.21.15.47:0/2360057951 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 27) v1 ==== 137985+0+0 (secure 0 0 0) 0x7f356000cb20 con 0x7f356810bb00 2024-03-21T10:33:42.380 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.382+0000 7f357060d740 1 Processor -- start 2024-03-21T10:33:42.380 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.382+0000 7f357060d740 1 -- start start 2024-03-21T10:33:42.382 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.383+0000 7f357060d740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5638472698b0 0x563847269c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:33:42.382 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.383+0000 7f357060d740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56384726a260 0x563847272810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:33:42.382 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.383+0000 7f357060d740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563847272de0 0x563847275230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:33:42.383 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.383+0000 7f357060d740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x5638471cd2c0 con 0x56384726a260 2024-03-21T10:33:42.383 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.383+0000 7f357060d740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x563847175c90 con 0x563847272de0 2024-03-21T10:33:42.383 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.383+0000 7f357060d740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x5638471c9360 con 0x5638472698b0 2024-03-21T10:33:42.383 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.383+0000 7f3566a00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56384726a260 0x563847272810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:33:42.383 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.383+0000 7f3566a00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56384726a260 0x563847272810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:58230/0 (socket says 172.21.15.47:58230) 2024-03-21T10:33:42.383 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.383+0000 7f3566a00640 1 -- 172.21.15.47:0/1003986857 learned_addr learned my addr 172.21.15.47:0/1003986857 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:33:42.383 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.383+0000 7f355f400640 1 --2- 172.21.15.47:0/2360057951 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f354c0a3110 0x7f354c0a55d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:33:42.383 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.383+0000 7f3566000640 1 --2- 172.21.15.47:0/1003986857 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5638472698b0 0x563847269c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:33:42.383 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.383+0000 7f3566000640 1 --2- 172.21.15.47:0/1003986857 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563847272de0 0x563847275230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:33:42.383 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.384+0000 7f3566a00640 1 -- 172.21.15.47:0/1003986857 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5638472698b0 msgr2=0x563847269c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:33:42.383 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.384+0000 7f3566a00640 1 --2- 172.21.15.47:0/1003986857 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5638472698b0 0x563847269c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:33:42.383 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.384+0000 7f3566a00640 1 -- 172.21.15.47:0/1003986857 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563847272de0 msgr2=0x563847275230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:33:42.383 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.384+0000 7f3566a00640 1 --2- 172.21.15.47:0/1003986857 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563847272de0 0x563847275230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:33:42.383 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.384+0000 7f3566a00640 1 -- 172.21.15.47:0/1003986857 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56384719a080 con 0x56384726a260 2024-03-21T10:33:42.384 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.384+0000 7f355f400640 1 -- 172.21.15.47:0/2360057951 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(101..101 src has 1..101) v4 ==== 9562+0+0 (secure 0 0 0) 0x7f35600cff00 con 0x7f356810bb00 2024-03-21T10:33:42.384 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.384+0000 7f3566000640 1 --2- 172.21.15.47:0/1003986857 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563847272de0 0x563847275230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T10:33:42.384 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.384+0000 7f3567400640 1 --2- 172.21.15.47:0/2360057951 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f354c0a3110 0x7f354c0a55d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:33:42.384 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.384+0000 7f3566a00640 1 --2- 172.21.15.47:0/1003986857 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56384726a260 0x563847272810 secure :-1 s=READY pgs=1148 cs=0 l=1 rev1=1 crypto rx=0x7f355800bda0 tx=0x7f35580107a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f8e1209324c2b690 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:33:42.384 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.384+0000 7f3553e00640 1 -- 172.21.15.47:0/1003986857 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f355801c680 con 0x56384726a260 2024-03-21T10:33:42.384 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.384+0000 7f3553e00640 1 -- 172.21.15.47:0/1003986857 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f35580118f0 con 0x56384726a260 2024-03-21T10:33:42.384 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.384+0000 7f3553e00640 1 -- 172.21.15.47:0/1003986857 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f355801a780 con 0x56384726a260 2024-03-21T10:33:42.387 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.385+0000 7f357060d740 1 -- 172.21.15.47:0/1003986857 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56384726a260 msgr2=0x563847272810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:33:42.387 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.385+0000 7f3567400640 1 --2- 172.21.15.47:0/2360057951 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f354c0a3110 0x7f354c0a55d0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f3560098900 tx=0x7f3560004a40 comp rx=0 tx=0).ready entity=mgr.15687 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:33:42.387 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.385+0000 7f357060d740 1 --2- 172.21.15.47:0/1003986857 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56384726a260 0x563847272810 secure :-1 s=READY pgs=1148 cs=0 l=1 rev1=1 crypto rx=0x7f355800bda0 tx=0x7f35580107a0 comp rx=0 tx=0).stop 2024-03-21T10:33:42.387 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.385+0000 7f357060d740 1 -- 172.21.15.47:0/1003986857 shutdown_connections 2024-03-21T10:33:42.387 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.385+0000 7f357060d740 1 --2- 172.21.15.47:0/1003986857 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563847272de0 0x563847275230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:33:42.387 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.385+0000 7f357060d740 1 --2- 172.21.15.47:0/1003986857 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56384726a260 0x563847272810 unknown :-1 s=CLOSED pgs=1148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:33:42.387 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.385+0000 7f357060d740 1 --2- 172.21.15.47:0/1003986857 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5638472698b0 0x563847269c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:33:42.387 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.385+0000 7f357060d740 1 -- 172.21.15.47:0/1003986857 >> 172.21.15.47:0/1003986857 conn(0x563847263f80 msgr2=0x563847264360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:33:42.388 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.385+0000 7f357060d740 1 -- 172.21.15.47:0/1003986857 shutdown_connections 2024-03-21T10:33:42.388 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.385+0000 7f357060d740 1 -- 172.21.15.47:0/1003986857 wait complete. 2024-03-21T10:33:42.388 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.386+0000 7f357060d740 1 -- start start 2024-03-21T10:33:42.388 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.386+0000 7f357060d740 1 Processor -- start 2024-03-21T10:33:42.388 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.386+0000 7f357060d740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5638472698b0 0x5638472726e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:33:42.388 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.386+0000 7f357060d740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56384726a260 0x56384726fa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:33:42.388 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.386+0000 7f357060d740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x563847272de0 0x56384726fff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:33:42.388 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.387+0000 7f357060d740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x563847275d50 con 0x563847272de0 2024-03-21T10:33:42.388 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.387+0000 7f357060d740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x5638471890d0 con 0x56384726a260 2024-03-21T10:33:42.388 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.387+0000 7f357060d740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x563847275ed0 con 0x5638472698b0 2024-03-21T10:33:42.388 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.387+0000 7f357060d740 10 client.0 _ll_register_callbacks cb 0x56384718de80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T10:33:42.388 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.387+0000 7f3566000640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x563847272de0 0x56384726fff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:33:42.388 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.387+0000 7f3566a00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56384726a260 0x56384726fa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:33:42.389 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.387+0000 7f3566000640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x563847272de0 0x56384726fff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:58246/0 (socket says 172.21.15.47:58246) 2024-03-21T10:33:42.389 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.387+0000 7f3566000640 1 -- 172.21.15.47:0/3833338079 learned_addr learned my addr 172.21.15.47:0/3833338079 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:33:42.389 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.387+0000 7f3566000640 1 --2- 172.21.15.47:0/3833338079 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5638472698b0 0x5638472726e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:33:42.389 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.388+0000 7f3566a00640 1 -- 172.21.15.47:0/3833338079 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5638472698b0 msgr2=0x5638472726e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:33:42.389 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.388+0000 7f3566a00640 1 --2- 172.21.15.47:0/3833338079 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5638472698b0 0x5638472726e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:33:42.389 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.388+0000 7f3566a00640 1 -- 172.21.15.47:0/3833338079 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x563847272de0 msgr2=0x56384726fff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:33:42.389 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.388+0000 7f3566a00640 1 --2- 172.21.15.47:0/3833338079 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x563847272de0 0x56384726fff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:33:42.389 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.388+0000 7f3566a00640 1 -- 172.21.15.47:0/3833338079 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x563846ee13f0 con 0x56384726a260 2024-03-21T10:33:42.389 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.388+0000 7f3566000640 1 --2- 172.21.15.47:0/3833338079 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x563847272de0 0x56384726fff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T10:33:42.389 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.388+0000 7f3566a00640 1 --2- 172.21.15.47:0/3833338079 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56384726a260 0x56384726fa70 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x563847264ff0 tx=0x7f3558011cd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=915550e3ab102ce1 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:33:42.389 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.388+0000 7f3550c00640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-21T10:33:42.389 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.388+0000 7f3550c00640 1 -- 172.21.15.47:0/3833338079 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3558004020 con 0x56384726a260 2024-03-21T10:33:42.389 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.389+0000 7f357060d740 1 -- 172.21.15.47:0/3833338079 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f35580118f0 con 0x56384726a260 2024-03-21T10:33:42.389 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.389+0000 7f3550c00640 1 -- 172.21.15.47:0/3833338079 <== mon.2 v2:172.21.15.100:3300/0 2 ==== osd_map(101..101 src has 1..101) v4 ==== 9562+0+0 (secure 0 0 0) 0x7f35580041c0 con 0x56384726a260 2024-03-21T10:33:42.392 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.389+0000 7f3550c00640 1 -- 172.21.15.47:0/3833338079 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f35580043e0 con 0x56384726a260 2024-03-21T10:33:42.392 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.389+0000 7f357060d740 10 client.37955 fetch_fsmap learned FSMap version 32 2024-03-21T10:33:42.392 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.389+0000 7f357060d740 1 -- 172.21.15.47:0/3833338079 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x563846fab7d0 con 0x56384726a260 2024-03-21T10:33:42.392 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.390+0000 7f3550c00640 1 -- 172.21.15.47:0/3833338079 <== mon.2 v2:172.21.15.100:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f35580118f0 con 0x56384726a260 2024-03-21T10:33:42.392 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.390+0000 7f3550c00640 1 -- 172.21.15.47:0/3833338079 <== mon.2 v2:172.21.15.100:3300/0 5 ==== fsmap(e 32) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f3558024070 con 0x56384726a260 2024-03-21T10:33:42.392 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.390+0000 7f357060d740 10 client.37955 fetch_fsmap finished waiting for FSMap version 32 2024-03-21T10:33:42.392 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.390+0000 7f357060d740 10 client.37955 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.k 2024-03-21T10:33:42.392 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.390+0000 7f357060d740 20 client.37955 populate_metadata read hostname 'smithi047' 2024-03-21T10:33:42.393 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.390+0000 7f357060d740 1 --2- 172.21.15.47:0/3833338079 >> [v2:172.21.15.100:6832/2723302038,v1:172.21.15.100:6833/2723302038] conn(0x563847282b00 0x563847284ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:33:42.393 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.390+0000 7f357060d740 4 client.37955 mds_command: new command op to 34427 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T10:33:42.393 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.390+0000 7f357060d740 1 -- 172.21.15.47:0/3833338079 --> [v2:172.21.15.100:6832/2723302038,v1:172.21.15.100:6833/2723302038] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5638471da800 con 0x563847282b00 2024-03-21T10:33:42.393 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.391+0000 7f3566000640 1 --2- 172.21.15.47:0/3833338079 >> [v2:172.21.15.100:6832/2723302038,v1:172.21.15.100:6833/2723302038] conn(0x563847282b00 0x563847284ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:33:42.393 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.391+0000 7f3566000640 1 --2- 172.21.15.47:0/3833338079 >> [v2:172.21.15.100:6832/2723302038,v1:172.21.15.100:6833/2723302038] conn(0x563847282b00 0x563847284ef0 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=ea8c0bc97687cad server_cookie=e4430970c939084b in_seq=0 out_seq=0 2024-03-21T10:33:42.393 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.391+0000 7f3550c00640 10 client.37955 ms_handle_connect on v2:172.21.15.100:6832/2723302038 2024-03-21T10:33:42.393 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.392+0000 7f3550c00640 1 -- 172.21.15.47:0/3833338079 <== mds.3 v2:172.21.15.100:6832/2723302038 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x5638471da800 con 0x563847282b00 2024-03-21T10:33:42.393 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.392+0000 7f3550c00640 10 client.37955 handle_command_reply: tid=0 2024-03-21T10:33:42.393 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.392+0000 7f357060d740 1 client.37955 shutdown 2024-03-21T10:33:42.393 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.392+0000 7f357060d740 1 -- 172.21.15.47:0/3833338079 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56384726a260 msgr2=0x56384726fa70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:33:42.393 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.392+0000 7f357060d740 1 --2- 172.21.15.47:0/3833338079 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56384726a260 0x56384726fa70 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x563847264ff0 tx=0x7f3558011cd0 comp rx=0 tx=0).stop 2024-03-21T10:33:42.393 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.392+0000 7f357060d740 1 -- 172.21.15.47:0/3833338079 shutdown_connections 2024-03-21T10:33:42.393 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.392+0000 7f357060d740 1 --2- 172.21.15.47:0/3833338079 >> [v2:172.21.15.100:6832/2723302038,v1:172.21.15.100:6833/2723302038] conn(0x563847282b00 0x563847284ef0 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:33:42.393 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.392+0000 7f357060d740 1 --2- 172.21.15.47:0/3833338079 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x563847272de0 0x56384726fff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:33:42.393 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.392+0000 7f357060d740 1 --2- 172.21.15.47:0/3833338079 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56384726a260 0x56384726fa70 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:33:42.394 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.393+0000 7f357060d740 1 --2- 172.21.15.47:0/3833338079 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5638472698b0 0x5638472726e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:33:42.394 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.393+0000 7f3550c00640 0 client.37955 ms_handle_reset on v2:172.21.15.100:6832/2723302038 2024-03-21T10:33:42.394 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.393+0000 7f357060d740 1 -- 172.21.15.47:0/3833338079 >> 172.21.15.47:0/3833338079 conn(0x563847263f80 msgr2=0x563847265cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:33:42.394 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.393+0000 7f357060d740 1 -- 172.21.15.47:0/3833338079 shutdown_connections 2024-03-21T10:33:42.394 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.393+0000 7f357060d740 1 -- 172.21.15.47:0/3833338079 wait complete. 2024-03-21T10:33:42.394 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.393+0000 7f357060d740 20 client.37955 trim_cache size 0 max 16384 2024-03-21T10:33:42.407 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.409+0000 7f357060d740 1 Processor -- start 2024-03-21T10:33:42.407 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.409+0000 7f357060d740 1 -- start start 2024-03-21T10:33:42.409 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.409+0000 7f357060d740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5638472698b0 0x563847262fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:33:42.409 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.409+0000 7f357060d740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56384726a260 0x5638472721a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:33:42.409 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.410+0000 7f357060d740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x563847272de0 0x5638472726e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:33:42.409 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.410+0000 7f357060d740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35681b0a70 con 0x563847272de0 2024-03-21T10:33:42.409 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.410+0000 7f357060d740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35681b23f0 con 0x56384726a260 2024-03-21T10:33:42.409 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.410+0000 7f357060d740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35681b10d0 con 0x5638472698b0 2024-03-21T10:33:42.409 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.410+0000 7f3566000640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x563847272de0 0x5638472726e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:33:42.409 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.410+0000 7f3566a00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56384726a260 0x5638472721a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:33:42.409 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.410+0000 7f3566000640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x563847272de0 0x5638472726e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:58248/0 (socket says 172.21.15.47:58248) 2024-03-21T10:33:42.409 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.410+0000 7f3566a00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56384726a260 0x5638472721a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:37052/0 (socket says 172.21.15.47:37052) 2024-03-21T10:33:42.410 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.410+0000 7f3566000640 1 -- 172.21.15.47:0/2572730226 learned_addr learned my addr 172.21.15.47:0/2572730226 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:33:42.410 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.410+0000 7f3566000640 1 --2- 172.21.15.47:0/2572730226 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5638472698b0 0x563847262fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:33:42.410 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.411+0000 7f3566a00640 1 -- 172.21.15.47:0/2572730226 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5638472698b0 msgr2=0x563847262fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:33:42.410 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.411+0000 7f3566a00640 1 --2- 172.21.15.47:0/2572730226 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5638472698b0 0x563847262fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:33:42.410 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.411+0000 7f3566a00640 1 -- 172.21.15.47:0/2572730226 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x563847272de0 msgr2=0x5638472726e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:33:42.410 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.411+0000 7f3566a00640 1 --2- 172.21.15.47:0/2572730226 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x563847272de0 0x5638472726e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:33:42.410 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.411+0000 7f3566a00640 1 -- 172.21.15.47:0/2572730226 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563847275c30 con 0x56384726a260 2024-03-21T10:33:42.410 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.411+0000 7f3566000640 1 --2- 172.21.15.47:0/2572730226 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x563847272de0 0x5638472726e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T10:33:42.412 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.411+0000 7f3566a00640 1 --2- 172.21.15.47:0/2572730226 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56384726a260 0x5638472721a0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f3558011010 tx=0x7f355800b2b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fac85c8b8549f633 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:33:42.412 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.411+0000 7f3553400640 1 -- 172.21.15.47:0/2572730226 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f355800be90 con 0x56384726a260 2024-03-21T10:33:42.413 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.412+0000 7f3553400640 1 -- 172.21.15.47:0/2572730226 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f3558011d40 con 0x56384726a260 2024-03-21T10:33:42.413 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.412+0000 7f3553400640 1 -- 172.21.15.47:0/2572730226 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3558011650 con 0x56384726a260 2024-03-21T10:33:42.413 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.412+0000 7f357060d740 1 -- 172.21.15.47:0/2572730226 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56384726a260 msgr2=0x5638472721a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:33:42.413 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.412+0000 7f357060d740 1 --2- 172.21.15.47:0/2572730226 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56384726a260 0x5638472721a0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f3558011010 tx=0x7f355800b2b0 comp rx=0 tx=0).stop 2024-03-21T10:33:42.413 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.412+0000 7f357060d740 1 -- 172.21.15.47:0/2572730226 shutdown_connections 2024-03-21T10:33:42.413 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.412+0000 7f357060d740 1 --2- 172.21.15.47:0/2572730226 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x563847272de0 0x5638472726e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:33:42.413 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.412+0000 7f357060d740 1 --2- 172.21.15.47:0/2572730226 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56384726a260 0x5638472721a0 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:33:42.413 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.412+0000 7f357060d740 1 --2- 172.21.15.47:0/2572730226 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5638472698b0 0x563847262fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:33:42.413 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.412+0000 7f357060d740 1 -- 172.21.15.47:0/2572730226 >> 172.21.15.47:0/2572730226 conn(0x563847263f80 msgr2=0x5638472628d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:33:42.413 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.412+0000 7f357060d740 1 -- 172.21.15.47:0/2572730226 shutdown_connections 2024-03-21T10:33:42.413 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.413+0000 7f357060d740 1 -- 172.21.15.47:0/2572730226 wait complete. 2024-03-21T10:33:42.414 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.413+0000 7f357060d740 1 -- start start 2024-03-21T10:33:42.414 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.413+0000 7f357060d740 1 Processor -- start 2024-03-21T10:33:42.414 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.413+0000 7f357060d740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5638472698b0 0x56384727b0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:33:42.414 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.414+0000 7f357060d740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563847272de0 0x5638472783a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:33:42.414 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.414+0000 7f357060d740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5638472789f0 0x563847278e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:33:42.414 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.414+0000 7f357060d740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35681b1d90 con 0x5638472789f0 2024-03-21T10:33:42.414 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.414+0000 7f357060d740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35681b1730 con 0x563847272de0 2024-03-21T10:33:42.414 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.414+0000 7f357060d740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35681aec50 con 0x5638472698b0 2024-03-21T10:33:42.414 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.414+0000 7f357060d740 10 client.0 _ll_register_callbacks cb 0x7f355800bbf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T10:33:42.414 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.414+0000 7f3566000640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5638472789f0 0x563847278e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:33:42.414 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.414+0000 7f3566000640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5638472789f0 0x563847278e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:58252/0 (socket says 172.21.15.47:58252) 2024-03-21T10:33:42.414 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.414+0000 7f3566000640 1 -- 172.21.15.47:0/152073385 learned_addr learned my addr 172.21.15.47:0/152073385 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:33:42.414 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.414+0000 7f3566a00640 1 --2- 172.21.15.47:0/152073385 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563847272de0 0x5638472783a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:33:42.414 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.414+0000 7f3566000640 1 --2- 172.21.15.47:0/152073385 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5638472698b0 0x56384727b0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:33:42.415 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.414+0000 7f3566000640 1 -- 172.21.15.47:0/152073385 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5638472698b0 msgr2=0x56384727b0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:33:42.415 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.414+0000 7f3566000640 1 --2- 172.21.15.47:0/152073385 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5638472698b0 0x56384727b0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:33:42.415 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.414+0000 7f3566000640 1 -- 172.21.15.47:0/152073385 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563847272de0 msgr2=0x5638472783a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:33:42.415 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.414+0000 7f3566000640 1 --2- 172.21.15.47:0/152073385 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563847272de0 0x5638472783a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:33:42.415 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.414+0000 7f3566000640 1 -- 172.21.15.47:0/152073385 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5638472dd4a0 con 0x5638472789f0 2024-03-21T10:33:42.419 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.415+0000 7f3566000640 1 --2- 172.21.15.47:0/152073385 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5638472789f0 0x563847278e70 secure :-1 s=READY pgs=1149 cs=0 l=1 rev1=1 crypto rx=0x7f355402c200 tx=0x7f35540056d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6852e4e413188929 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:33:42.419 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.415+0000 7f3550c00640 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-03-21T10:33:42.419 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.415+0000 7f3550c00640 1 -- 172.21.15.47:0/152073385 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3554003b00 con 0x5638472789f0 2024-03-21T10:33:42.419 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.415+0000 7f357060d740 1 -- 172.21.15.47:0/152073385 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f3558011d40 con 0x5638472789f0 2024-03-21T10:33:42.419 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.415+0000 7f3550c00640 1 -- 172.21.15.47:0/152073385 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(101..101 src has 1..101) v4 ==== 9562+0+0 (secure 0 0 0) 0x7f355403a040 con 0x5638472789f0 2024-03-21T10:33:42.419 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.415+0000 7f3550c00640 1 -- 172.21.15.47:0/152073385 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3554003ca0 con 0x5638472789f0 2024-03-21T10:33:42.419 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.415+0000 7f357060d740 10 client.18201 fetch_fsmap learned FSMap version 32 2024-03-21T10:33:42.419 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.415+0000 7f357060d740 1 -- 172.21.15.47:0/152073385 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x5638472de7e0 con 0x5638472789f0 2024-03-21T10:33:42.420 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.416+0000 7f3550c00640 1 -- 172.21.15.47:0/152073385 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f3558011d40 con 0x5638472789f0 2024-03-21T10:33:42.420 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.416+0000 7f3550c00640 1 -- 172.21.15.47:0/152073385 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 32) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f3554043570 con 0x5638472789f0 2024-03-21T10:33:42.420 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.416+0000 7f357060d740 10 client.18201 fetch_fsmap finished waiting for FSMap version 32 2024-03-21T10:33:42.420 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.416+0000 7f357060d740 10 client.18201 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.k 2024-03-21T10:33:42.420 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.416+0000 7f357060d740 20 client.18201 populate_metadata read hostname 'smithi047' 2024-03-21T10:33:42.420 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.416+0000 7f357060d740 1 --2- 172.21.15.47:0/152073385 >> [v2:172.21.15.100:6832/2723302038,v1:172.21.15.100:6833/2723302038] conn(0x5638472dee90 0x5638472e12b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:33:42.420 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.416+0000 7f357060d740 4 client.18201 mds_command: new command op to 34427 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-21T10:33:42.421 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.416+0000 7f357060d740 1 -- 172.21.15.47:0/152073385 --> [v2:172.21.15.100:6832/2723302038,v1:172.21.15.100:6833/2723302038] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x5638471d4b50 con 0x5638472dee90 2024-03-21T10:33:42.421 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.416+0000 7f3566a00640 1 --2- 172.21.15.47:0/152073385 >> [v2:172.21.15.100:6832/2723302038,v1:172.21.15.100:6833/2723302038] conn(0x5638472dee90 0x5638472e12b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:33:42.421 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.417+0000 7f3566a00640 1 --2- 172.21.15.47:0/152073385 >> [v2:172.21.15.100:6832/2723302038,v1:172.21.15.100:6833/2723302038] conn(0x5638472dee90 0x5638472e12b0 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=49ab0d8f53636a7f server_cookie=f7006dd59e57cfb6 in_seq=0 out_seq=0 2024-03-21T10:33:42.421 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.417+0000 7f3550c00640 10 client.18201 ms_handle_connect on v2:172.21.15.100:6832/2723302038 2024-03-21T10:33:42.421 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.418+0000 7f3550c00640 1 -- 172.21.15.47:0/152073385 <== mds.3 v2:172.21.15.100:6832/2723302038 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x5638471d4b50 con 0x5638472dee90 2024-03-21T10:33:42.421 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.418+0000 7f3550c00640 10 client.18201 handle_command_reply: tid=0 2024-03-21T10:33:42.421 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.418+0000 7f357060d740 1 client.18201 shutdown 2024-03-21T10:33:42.421 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.418+0000 7f357060d740 1 -- 172.21.15.47:0/152073385 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5638472789f0 msgr2=0x563847278e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:33:42.422 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.418+0000 7f357060d740 1 --2- 172.21.15.47:0/152073385 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5638472789f0 0x563847278e70 secure :-1 s=READY pgs=1149 cs=0 l=1 rev1=1 crypto rx=0x7f355402c200 tx=0x7f35540056d0 comp rx=0 tx=0).stop 2024-03-21T10:33:42.422 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.418+0000 7f357060d740 1 -- 172.21.15.47:0/152073385 shutdown_connections 2024-03-21T10:33:42.422 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.418+0000 7f357060d740 1 --2- 172.21.15.47:0/152073385 >> [v2:172.21.15.100:6832/2723302038,v1:172.21.15.100:6833/2723302038] conn(0x5638472dee90 0x5638472e12b0 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:33:42.422 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.418+0000 7f357060d740 1 --2- 172.21.15.47:0/152073385 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5638472789f0 0x563847278e70 unknown :-1 s=CLOSED pgs=1149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:33:42.422 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.418+0000 7f357060d740 1 --2- 172.21.15.47:0/152073385 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563847272de0 0x5638472783a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:33:42.422 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.418+0000 7f357060d740 1 --2- 172.21.15.47:0/152073385 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5638472698b0 0x56384727b0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:33:42.422 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.418+0000 7f3550c00640 0 client.18201 ms_handle_reset on v2:172.21.15.100:6832/2723302038 2024-03-21T10:33:42.422 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.418+0000 7f357060d740 1 -- 172.21.15.47:0/152073385 >> 172.21.15.47:0/152073385 conn(0x563847263f80 msgr2=0x563847263160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:33:42.423 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.418+0000 7f357060d740 1 -- 172.21.15.47:0/152073385 shutdown_connections 2024-03-21T10:33:42.423 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.418+0000 7f357060d740 1 -- 172.21.15.47:0/152073385 wait complete. 2024-03-21T10:33:42.423 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.418+0000 7f357060d740 20 client.18201 trim_cache size 0 max 16384 2024-03-21T10:33:42.423 INFO:teuthology.orchestra.run.smithi047.stdout:[] 2024-03-21T10:33:42.424 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.419+0000 7f3567e00640 1 -- 172.21.15.47:0/2360057951 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f354c0a3110 msgr2=0x7f354c0a55d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:33:42.424 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.419+0000 7f3567e00640 1 --2- 172.21.15.47:0/2360057951 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f354c0a3110 0x7f354c0a55d0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f3560098900 tx=0x7f3560004a40 comp rx=0 tx=0).stop 2024-03-21T10:33:42.425 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.419+0000 7f3567e00640 1 -- 172.21.15.47:0/2360057951 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f356810bb00 msgr2=0x7f356819e800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:33:42.425 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.419+0000 7f3567e00640 1 --2- 172.21.15.47:0/2360057951 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f356810bb00 0x7f356819e800 secure :-1 s=READY pgs=1147 cs=0 l=1 rev1=1 crypto rx=0x7f356000e940 tx=0x7f356000ee00 comp rx=0 tx=0).stop 2024-03-21T10:33:42.425 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.419+0000 7f3567e00640 1 -- 172.21.15.47:0/2360057951 shutdown_connections 2024-03-21T10:33:42.425 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.419+0000 7f3567e00640 1 --2- 172.21.15.47:0/2360057951 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f356810bb00 0x7f356819e800 unknown :-1 s=CLOSED pgs=1147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:33:42.425 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.419+0000 7f3567e00640 1 --2- 172.21.15.47:0/2360057951 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3568103050 0x7f356819a480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:33:42.425 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.419+0000 7f3567e00640 1 --2- 172.21.15.47:0/2360057951 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f354c0a3110 0x7f354c0a55d0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:33:42.425 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.419+0000 7f3567e00640 1 --2- 172.21.15.47:0/2360057951 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3568102730 0x7f3568199f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:33:42.425 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.419+0000 7f3567e00640 1 -- 172.21.15.47:0/2360057951 >> 172.21.15.47:0/2360057951 conn(0x7f35680fbef0 msgr2=0x7f35680fcb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:33:42.425 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.419+0000 7f3567e00640 1 -- 172.21.15.47:0/2360057951 shutdown_connections 2024-03-21T10:33:42.426 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.419+0000 7f3567e00640 1 -- 172.21.15.47:0/2360057951 wait complete. 2024-03-21T10:33:42.436 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-21T10:33:42.436 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-03-21T10:33:42.646 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.647+0000 7fcc32000640 1 -- 172.21.15.47:0/1134974168 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fcc2c102ff0 msgr2=0x7fcc2c10b4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:33:42.646 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.647+0000 7fcc32000640 1 --2- 172.21.15.47:0/1134974168 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fcc2c102ff0 0x7fcc2c10b4d0 secure :-1 s=READY pgs=1150 cs=0 l=1 rev1=1 crypto rx=0x7fcc14009900 tx=0x7fcc1402bcc0 comp rx=0 tx=0).stop 2024-03-21T10:33:42.646 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.647+0000 7fcc32000640 1 -- 172.21.15.47:0/1134974168 shutdown_connections 2024-03-21T10:33:42.646 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.648+0000 7fcc32000640 1 --2- 172.21.15.47:0/1134974168 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcc2c10baf0 0x7fcc2c10df90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:33:42.647 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.648+0000 7fcc32000640 1 --2- 172.21.15.47:0/1134974168 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fcc2c102ff0 0x7fcc2c10b4d0 unknown :-1 s=CLOSED pgs=1150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:33:42.647 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.648+0000 7fcc32000640 1 --2- 172.21.15.47:0/1134974168 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcc2c1026d0 0x7fcc2c102ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:33:42.647 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.648+0000 7fcc32000640 1 -- 172.21.15.47:0/1134974168 >> 172.21.15.47:0/1134974168 conn(0x7fcc2c0fbef0 msgr2=0x7fcc2c0fe310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:33:42.647 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.648+0000 7fcc32000640 1 -- 172.21.15.47:0/1134974168 shutdown_connections 2024-03-21T10:33:42.647 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.648+0000 7fcc32000640 1 -- 172.21.15.47:0/1134974168 wait complete. 2024-03-21T10:33:42.649 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.649+0000 7fcc32000640 1 Processor -- start 2024-03-21T10:33:42.649 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.649+0000 7fcc32000640 1 -- start start 2024-03-21T10:33:42.649 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.649+0000 7fcc32000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcc2c1026d0 0x7fcc2c100320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:33:42.650 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.649+0000 7fcc32000640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fcc2c102ff0 0x7fcc2c100860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:33:42.650 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.649+0000 7fcc32000640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcc2c10baf0 0x7fcc2c100da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:33:42.650 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.649+0000 7fcc32000640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc2c111780 con 0x7fcc2c102ff0 2024-03-21T10:33:42.650 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.649+0000 7fcc32000640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc2c111600 con 0x7fcc2c1026d0 2024-03-21T10:33:42.650 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.649+0000 7fcc32000640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc2c111900 con 0x7fcc2c10baf0 2024-03-21T10:33:42.650 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.650+0000 7fcc23e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fcc2c102ff0 0x7fcc2c100860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:33:42.650 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.650+0000 7fcc23e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fcc2c102ff0 0x7fcc2c100860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:58280/0 (socket says 172.21.15.47:58280) 2024-03-21T10:33:42.650 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.650+0000 7fcc23e00640 1 -- 172.21.15.47:0/1300177907 learned_addr learned my addr 172.21.15.47:0/1300177907 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:33:42.650 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.650+0000 7fcc30c00640 1 --2- 172.21.15.47:0/1300177907 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcc2c1026d0 0x7fcc2c100320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:33:42.650 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.650+0000 7fcc31600640 1 --2- 172.21.15.47:0/1300177907 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcc2c10baf0 0x7fcc2c100da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:33:42.650 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.650+0000 7fcc23e00640 1 -- 172.21.15.47:0/1300177907 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcc2c10baf0 msgr2=0x7fcc2c100da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:33:42.650 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.650+0000 7fcc23e00640 1 --2- 172.21.15.47:0/1300177907 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcc2c10baf0 0x7fcc2c100da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:33:42.650 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.650+0000 7fcc23e00640 1 -- 172.21.15.47:0/1300177907 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcc2c1026d0 msgr2=0x7fcc2c100320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:33:42.650 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.650+0000 7fcc23e00640 1 --2- 172.21.15.47:0/1300177907 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcc2c1026d0 0x7fcc2c100320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:33:42.651 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.650+0000 7fcc23e00640 1 -- 172.21.15.47:0/1300177907 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc2c1a61f0 con 0x7fcc2c102ff0 2024-03-21T10:33:42.651 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.650+0000 7fcc31600640 1 --2- 172.21.15.47:0/1300177907 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcc2c10baf0 0x7fcc2c100da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T10:33:42.651 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.651+0000 7fcc23e00640 1 --2- 172.21.15.47:0/1300177907 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fcc2c102ff0 0x7fcc2c100860 secure :-1 s=READY pgs=1151 cs=0 l=1 rev1=1 crypto rx=0x7fcc14005940 tx=0x7fcc140045e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:33:42.651 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.651+0000 7fcc2a000640 1 -- 172.21.15.47:0/1300177907 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc14043020 con 0x7fcc2c102ff0 2024-03-21T10:33:42.651 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.651+0000 7fcc2a000640 1 -- 172.21.15.47:0/1300177907 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fcc1402db90 con 0x7fcc2c102ff0 2024-03-21T10:33:42.651 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.651+0000 7fcc2a000640 1 -- 172.21.15.47:0/1300177907 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc14003a10 con 0x7fcc2c102ff0 2024-03-21T10:33:42.651 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.651+0000 7fcc32000640 1 -- 172.21.15.47:0/1300177907 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcc2c1a6480 con 0x7fcc2c102ff0 2024-03-21T10:33:42.651 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.651+0000 7fcc32000640 1 -- 172.21.15.47:0/1300177907 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcc2c1a6a20 con 0x7fcc2c102ff0 2024-03-21T10:33:42.653 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.654+0000 7fcc36390740 1 Processor -- start 2024-03-21T10:33:42.653 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.655+0000 7fcc2a000640 1 -- 172.21.15.47:0/1300177907 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 27) v1 ==== 137985+0+0 (secure 0 0 0) 0x7fcc14046020 con 0x7fcc2c102ff0 2024-03-21T10:33:42.653 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.655+0000 7fcc36390740 1 -- start start 2024-03-21T10:33:42.655 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.655+0000 7fcc36390740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5572b9bf88b0 0x5572b9bf8c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:33:42.655 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.655+0000 7fcc36390740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5572b9bf9260 0x5572b9c01810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:33:42.655 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.655+0000 7fcc36390740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5572b9c01de0 0x5572b9c04230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:33:42.655 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.655+0000 7fcc36390740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x5572b9b5c2c0 con 0x5572b9bf9260 2024-03-21T10:33:42.655 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.655+0000 7fcc36390740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x5572b9b04c90 con 0x5572b9bf88b0 2024-03-21T10:33:42.655 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.655+0000 7fcc36390740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x5572b9b58360 con 0x5572b9c01de0 2024-03-21T10:33:42.655 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.655+0000 7fcc30c00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5572b9bf9260 0x5572b9c01810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:33:42.655 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.655+0000 7fcc30c00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5572b9bf9260 0x5572b9c01810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:58292/0 (socket says 172.21.15.47:58292) 2024-03-21T10:33:42.655 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.655+0000 7fcc30c00640 1 -- 172.21.15.47:0/1883701964 learned_addr learned my addr 172.21.15.47:0/1883701964 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:33:42.655 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.656+0000 7fcc31600640 1 --2- 172.21.15.47:0/1883701964 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5572b9bf88b0 0x5572b9bf8c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:33:42.655 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.656+0000 7fcc23e00640 1 --2- 172.21.15.47:0/1883701964 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5572b9c01de0 0x5572b9c04230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:33:42.656 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.656+0000 7fcc30c00640 1 -- 172.21.15.47:0/1883701964 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5572b9c01de0 msgr2=0x5572b9c04230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:33:42.656 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.656+0000 7fcc30c00640 1 --2- 172.21.15.47:0/1883701964 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5572b9c01de0 0x5572b9c04230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:33:42.656 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.656+0000 7fcc30c00640 1 -- 172.21.15.47:0/1883701964 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5572b9bf88b0 msgr2=0x5572b9bf8c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:33:42.656 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.656+0000 7fcc30c00640 1 --2- 172.21.15.47:0/1883701964 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5572b9bf88b0 0x5572b9bf8c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:33:42.656 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.656+0000 7fcc30c00640 1 -- 172.21.15.47:0/1883701964 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5572b9b29080 con 0x5572b9bf9260 2024-03-21T10:33:42.656 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.656+0000 7fcc23e00640 1 --2- 172.21.15.47:0/1883701964 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5572b9c01de0 0x5572b9c04230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T10:33:42.656 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.656+0000 7fcc31600640 1 --2- 172.21.15.47:0/1883701964 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5572b9bf88b0 0x5572b9bf8c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T10:33:42.656 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.656+0000 7fcc30c00640 1 --2- 172.21.15.47:0/1883701964 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5572b9bf9260 0x5572b9c01810 secure :-1 s=READY pgs=1152 cs=0 l=1 rev1=1 crypto rx=0x7fcc1c00b760 tx=0x7fcc1c00bc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5bd6e50b83547ca0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:33:42.656 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.657+0000 7fcc21c00640 1 -- 172.21.15.47:0/1883701964 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc1c003dd0 con 0x5572b9bf9260 2024-03-21T10:33:42.656 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.657+0000 7fcc21c00640 1 -- 172.21.15.47:0/1883701964 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fcc1c015070 con 0x5572b9bf9260 2024-03-21T10:33:42.656 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.657+0000 7fcc21c00640 1 -- 172.21.15.47:0/1883701964 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc1c004dc0 con 0x5572b9bf9260 2024-03-21T10:33:42.660 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.657+0000 7fcc2a000640 1 --2- 172.21.15.47:0/1300177907 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7fcc0c0a3140 0x7fcc0c0a5600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:33:42.660 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.657+0000 7fcc36390740 1 -- 172.21.15.47:0/1883701964 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5572b9bf9260 msgr2=0x5572b9c01810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:33:42.660 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.657+0000 7fcc36390740 1 --2- 172.21.15.47:0/1883701964 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5572b9bf9260 0x5572b9c01810 secure :-1 s=READY pgs=1152 cs=0 l=1 rev1=1 crypto rx=0x7fcc1c00b760 tx=0x7fcc1c00bc20 comp rx=0 tx=0).stop 2024-03-21T10:33:42.660 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.657+0000 7fcc2a000640 1 -- 172.21.15.47:0/1300177907 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(101..101 src has 1..101) v4 ==== 9562+0+0 (secure 0 0 0) 0x7fcc140f1b70 con 0x7fcc2c102ff0 2024-03-21T10:33:42.660 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.657+0000 7fcc31600640 1 --2- 172.21.15.47:0/1300177907 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7fcc0c0a3140 0x7fcc0c0a5600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:33:42.661 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.658+0000 7fcc36390740 1 -- 172.21.15.47:0/1883701964 shutdown_connections 2024-03-21T10:33:42.661 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.658+0000 7fcc36390740 1 --2- 172.21.15.47:0/1883701964 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5572b9c01de0 0x5572b9c04230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:33:42.661 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.658+0000 7fcc36390740 1 --2- 172.21.15.47:0/1883701964 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5572b9bf9260 0x5572b9c01810 unknown :-1 s=CLOSED pgs=1152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:33:42.661 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.658+0000 7fcc36390740 1 --2- 172.21.15.47:0/1883701964 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5572b9bf88b0 0x5572b9bf8c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:33:42.661 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.658+0000 7fcc36390740 1 -- 172.21.15.47:0/1883701964 >> 172.21.15.47:0/1883701964 conn(0x5572b9bf2f80 msgr2=0x5572b9bf3360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:33:42.661 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.658+0000 7fcc36390740 1 -- 172.21.15.47:0/1883701964 shutdown_connections 2024-03-21T10:33:42.661 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.658+0000 7fcc36390740 1 -- 172.21.15.47:0/1883701964 wait complete. 2024-03-21T10:33:42.661 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.658+0000 7fcc31600640 1 --2- 172.21.15.47:0/1300177907 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7fcc0c0a3140 0x7fcc0c0a5600 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fcc24005fd0 tx=0x7fcc24009530 comp rx=0 tx=0).ready entity=mgr.15687 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:33:42.661 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.658+0000 7fcc36390740 1 -- start start 2024-03-21T10:33:42.661 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.659+0000 7fcc36390740 1 Processor -- start 2024-03-21T10:33:42.661 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.659+0000 7fcc36390740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5572b9bf88b0 0x5572b9c01740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:33:42.661 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.659+0000 7fcc36390740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5572b9bf9260 0x5572b9bfead0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:33:42.661 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.659+0000 7fcc36390740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5572b9c01de0 0x5572b9bff0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:33:42.662 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.659+0000 7fcc36390740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x5572b9c04d50 con 0x5572b9bf88b0 2024-03-21T10:33:42.662 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.659+0000 7fcc36390740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x5572b9b180d0 con 0x5572b9c01de0 2024-03-21T10:33:42.662 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.659+0000 7fcc36390740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x5572b9c04ed0 con 0x5572b9bf9260 2024-03-21T10:33:42.662 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.659+0000 7fcc36390740 10 client.0 _ll_register_callbacks cb 0x5572b9b1ce80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T10:33:42.662 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.660+0000 7fcc30c00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5572b9bf88b0 0x5572b9c01740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:33:42.662 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.660+0000 7fcc30c00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5572b9bf88b0 0x5572b9c01740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:58300/0 (socket says 172.21.15.47:58300) 2024-03-21T10:33:42.662 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.660+0000 7fcc30c00640 1 -- 172.21.15.47:0/3171079319 learned_addr learned my addr 172.21.15.47:0/3171079319 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:33:42.662 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.660+0000 7fcc31600640 1 --2- 172.21.15.47:0/3171079319 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5572b9c01de0 0x5572b9bff0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:33:42.662 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.660+0000 7fcc23e00640 1 --2- 172.21.15.47:0/3171079319 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5572b9bf9260 0x5572b9bfead0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:33:42.662 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.660+0000 7fcc30c00640 1 -- 172.21.15.47:0/3171079319 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5572b9bf9260 msgr2=0x5572b9bfead0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:33:42.662 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.660+0000 7fcc30c00640 1 --2- 172.21.15.47:0/3171079319 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5572b9bf9260 0x5572b9bfead0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:33:42.662 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.660+0000 7fcc30c00640 1 -- 172.21.15.47:0/3171079319 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5572b9c01de0 msgr2=0x5572b9bff0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:33:42.662 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.660+0000 7fcc30c00640 1 --2- 172.21.15.47:0/3171079319 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5572b9c01de0 0x5572b9bff0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:33:42.663 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.660+0000 7fcc30c00640 1 -- 172.21.15.47:0/3171079319 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5572b98703f0 con 0x5572b9bf88b0 2024-03-21T10:33:42.663 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.660+0000 7fcc23e00640 1 --2- 172.21.15.47:0/3171079319 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5572b9bf9260 0x5572b9bfead0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T10:33:42.663 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.661+0000 7fcc30c00640 1 --2- 172.21.15.47:0/3171079319 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5572b9bf88b0 0x5572b9c01740 secure :-1 s=READY pgs=1153 cs=0 l=1 rev1=1 crypto rx=0x7fcc1c018000 tx=0x7fcc1c003e00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ab5d5eae50732543 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:33:42.663 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.661+0000 7fcc1a000640 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-03-21T10:33:42.663 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.661+0000 7fcc1a000640 1 -- 172.21.15.47:0/3171079319 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc1c028060 con 0x5572b9bf88b0 2024-03-21T10:33:42.663 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.661+0000 7fcc1a000640 1 -- 172.21.15.47:0/3171079319 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(101..101 src has 1..101) v4 ==== 9562+0+0 (secure 0 0 0) 0x7fcc1c011850 con 0x5572b9bf88b0 2024-03-21T10:33:42.663 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.661+0000 7fcc36390740 1 -- 172.21.15.47:0/3171079319 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fcc1c015070 con 0x5572b9bf88b0 2024-03-21T10:33:42.663 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.661+0000 7fcc1a000640 1 -- 172.21.15.47:0/3171079319 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7fcc1c02f050 con 0x5572b9bf88b0 2024-03-21T10:33:42.663 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.661+0000 7fcc36390740 10 client.18213 fetch_fsmap learned FSMap version 32 2024-03-21T10:33:42.663 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.661+0000 7fcc36390740 1 -- 172.21.15.47:0/3171079319 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x5572b993a7d0 con 0x5572b9bf88b0 2024-03-21T10:33:42.663 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.662+0000 7fcc1a000640 1 -- 172.21.15.47:0/3171079319 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fcc1c015070 con 0x5572b9bf88b0 2024-03-21T10:33:42.663 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.662+0000 7fcc1a000640 1 -- 172.21.15.47:0/3171079319 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 32) v1 ==== 7942+0+0 (secure 0 0 0) 0x7fcc1c022070 con 0x5572b9bf88b0 2024-03-21T10:33:42.663 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.662+0000 7fcc36390740 10 client.18213 fetch_fsmap finished waiting for FSMap version 32 2024-03-21T10:33:42.663 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.662+0000 7fcc36390740 10 client.18213 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.e 2024-03-21T10:33:42.663 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.662+0000 7fcc36390740 20 client.18213 populate_metadata read hostname 'smithi047' 2024-03-21T10:33:42.664 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.662+0000 7fcc36390740 1 --2- 172.21.15.47:0/3171079319 >> [v2:172.21.15.100:6834/2597849957,v1:172.21.15.100:6835/2597849957] conn(0x5572b9c11b80 0x5572b9c13f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:33:42.664 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.662+0000 7fcc36390740 4 client.18213 mds_command: new command op to 34454 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T10:33:42.664 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.662+0000 7fcc36390740 1 -- 172.21.15.47:0/3171079319 --> [v2:172.21.15.100:6834/2597849957,v1:172.21.15.100:6835/2597849957] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5572b9b69800 con 0x5572b9c11b80 2024-03-21T10:33:42.665 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.662+0000 7fcc23e00640 1 --2- 172.21.15.47:0/3171079319 >> [v2:172.21.15.100:6834/2597849957,v1:172.21.15.100:6835/2597849957] conn(0x5572b9c11b80 0x5572b9c13f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:33:42.665 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.663+0000 7fcc23e00640 1 --2- 172.21.15.47:0/3171079319 >> [v2:172.21.15.100:6834/2597849957,v1:172.21.15.100:6835/2597849957] conn(0x5572b9c11b80 0x5572b9c13f70 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=a42c0b75fe9b67b3 server_cookie=9c08d1689db7aef3 in_seq=0 out_seq=0 2024-03-21T10:33:42.665 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.663+0000 7fcc1a000640 10 client.18213 ms_handle_connect on v2:172.21.15.100:6834/2597849957 2024-03-21T10:33:42.665 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.665+0000 7fcc1a000640 1 -- 172.21.15.47:0/3171079319 <== mds.2 v2:172.21.15.100:6834/2597849957 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x5572b9b69800 con 0x5572b9c11b80 2024-03-21T10:33:42.665 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.665+0000 7fcc1a000640 10 client.18213 handle_command_reply: tid=0 2024-03-21T10:33:42.665 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.665+0000 7fcc36390740 1 client.18213 shutdown 2024-03-21T10:33:42.666 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.665+0000 7fcc36390740 1 -- 172.21.15.47:0/3171079319 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5572b9bf88b0 msgr2=0x5572b9c01740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:33:42.666 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.665+0000 7fcc36390740 1 --2- 172.21.15.47:0/3171079319 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5572b9bf88b0 0x5572b9c01740 secure :-1 s=READY pgs=1153 cs=0 l=1 rev1=1 crypto rx=0x7fcc1c018000 tx=0x7fcc1c003e00 comp rx=0 tx=0).stop 2024-03-21T10:33:42.666 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.665+0000 7fcc36390740 1 -- 172.21.15.47:0/3171079319 shutdown_connections 2024-03-21T10:33:42.666 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.665+0000 7fcc36390740 1 --2- 172.21.15.47:0/3171079319 >> [v2:172.21.15.100:6834/2597849957,v1:172.21.15.100:6835/2597849957] conn(0x5572b9c11b80 0x5572b9c13f70 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:33:42.666 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.665+0000 7fcc36390740 1 --2- 172.21.15.47:0/3171079319 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5572b9c01de0 0x5572b9bff0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:33:42.666 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.665+0000 7fcc36390740 1 --2- 172.21.15.47:0/3171079319 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5572b9bf9260 0x5572b9bfead0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:33:42.666 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.665+0000 7fcc36390740 1 --2- 172.21.15.47:0/3171079319 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5572b9bf88b0 0x5572b9c01740 unknown :-1 s=CLOSED pgs=1153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:33:42.666 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.665+0000 7fcc1a000640 0 client.18213 ms_handle_reset on v2:172.21.15.100:6834/2597849957 2024-03-21T10:33:42.666 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.665+0000 7fcc36390740 1 -- 172.21.15.47:0/3171079319 >> 172.21.15.47:0/3171079319 conn(0x5572b9bf2f80 msgr2=0x5572b9bf4cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:33:42.666 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.665+0000 7fcc36390740 1 -- 172.21.15.47:0/3171079319 shutdown_connections 2024-03-21T10:33:42.666 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.665+0000 7fcc36390740 1 -- 172.21.15.47:0/3171079319 wait complete. 2024-03-21T10:33:42.666 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.666+0000 7fcc36390740 20 client.18213 trim_cache size 0 max 16384 2024-03-21T10:33:42.685 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.686+0000 7fcc36390740 1 Processor -- start 2024-03-21T10:33:42.685 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.686+0000 7fcc36390740 1 -- start start 2024-03-21T10:33:42.685 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.687+0000 7fcc36390740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5572b9bf9260 0x5572b9c0e160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:33:42.685 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.687+0000 7fcc36390740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5572b9c01de0 0x5572b9bf1fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:33:42.685 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.687+0000 7fcc36390740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5572b9bf2520 0x5572b9c01200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:33:42.685 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.687+0000 7fcc36390740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc2c1b0a30 con 0x5572b9bf2520 2024-03-21T10:33:42.685 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.687+0000 7fcc36390740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc2c1b23b0 con 0x5572b9bf9260 2024-03-21T10:33:42.685 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.687+0000 7fcc36390740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc2c1b1090 con 0x5572b9c01de0 2024-03-21T10:33:42.686 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.687+0000 7fcc31600640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5572b9bf2520 0x5572b9c01200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:33:42.687 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.687+0000 7fcc30c00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5572b9bf9260 0x5572b9c0e160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:33:42.687 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.687+0000 7fcc31600640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5572b9bf2520 0x5572b9c01200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:58316/0 (socket says 172.21.15.47:58316) 2024-03-21T10:33:42.687 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.687+0000 7fcc31600640 1 -- 172.21.15.47:0/584514102 learned_addr learned my addr 172.21.15.47:0/584514102 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:33:42.687 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.687+0000 7fcc23e00640 1 --2- 172.21.15.47:0/584514102 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5572b9c01de0 0x5572b9bf1fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:33:42.687 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.687+0000 7fcc31600640 1 -- 172.21.15.47:0/584514102 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5572b9c01de0 msgr2=0x5572b9bf1fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:33:42.687 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.687+0000 7fcc31600640 1 --2- 172.21.15.47:0/584514102 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5572b9c01de0 0x5572b9bf1fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:33:42.687 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.687+0000 7fcc31600640 1 -- 172.21.15.47:0/584514102 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5572b9bf9260 msgr2=0x5572b9c0e160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:33:42.687 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.687+0000 7fcc31600640 1 --2- 172.21.15.47:0/584514102 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5572b9bf9260 0x5572b9c0e160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:33:42.687 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.688+0000 7fcc31600640 1 -- 172.21.15.47:0/584514102 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5572b9bed570 con 0x5572b9bf2520 2024-03-21T10:33:42.687 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.688+0000 7fcc31600640 1 --2- 172.21.15.47:0/584514102 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5572b9bf2520 0x5572b9c01200 secure :-1 s=READY pgs=1154 cs=0 l=1 rev1=1 crypto rx=0x7fcc24011740 tx=0x7fcc24013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3c0f5e39b9a9abac server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:33:42.687 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.688+0000 7fcc21200640 1 -- 172.21.15.47:0/584514102 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc2401e800 con 0x5572b9bf2520 2024-03-21T10:33:42.687 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.688+0000 7fcc21200640 1 -- 172.21.15.47:0/584514102 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fcc2401ee50 con 0x5572b9bf2520 2024-03-21T10:33:42.687 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.688+0000 7fcc21200640 1 -- 172.21.15.47:0/584514102 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc240273d0 con 0x5572b9bf2520 2024-03-21T10:33:42.691 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.689+0000 7fcc36390740 1 -- 172.21.15.47:0/584514102 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5572b9bf2520 msgr2=0x5572b9c01200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:33:42.691 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.689+0000 7fcc36390740 1 --2- 172.21.15.47:0/584514102 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5572b9bf2520 0x5572b9c01200 secure :-1 s=READY pgs=1154 cs=0 l=1 rev1=1 crypto rx=0x7fcc24011740 tx=0x7fcc24013000 comp rx=0 tx=0).stop 2024-03-21T10:33:42.691 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.689+0000 7fcc36390740 1 -- 172.21.15.47:0/584514102 shutdown_connections 2024-03-21T10:33:42.692 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.689+0000 7fcc36390740 1 --2- 172.21.15.47:0/584514102 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5572b9bf2520 0x5572b9c01200 unknown :-1 s=CLOSED pgs=1154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:33:42.692 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.689+0000 7fcc36390740 1 --2- 172.21.15.47:0/584514102 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5572b9c01de0 0x5572b9bf1fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:33:42.692 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.689+0000 7fcc36390740 1 --2- 172.21.15.47:0/584514102 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5572b9bf9260 0x5572b9c0e160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:33:42.692 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.689+0000 7fcc36390740 1 -- 172.21.15.47:0/584514102 >> 172.21.15.47:0/584514102 conn(0x5572b9bf2f80 msgr2=0x5572b9bf18d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:33:42.692 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.689+0000 7fcc36390740 1 -- 172.21.15.47:0/584514102 shutdown_connections 2024-03-21T10:33:42.692 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.689+0000 7fcc36390740 1 -- 172.21.15.47:0/584514102 wait complete. 2024-03-21T10:33:42.692 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.690+0000 7fcc36390740 1 -- start start 2024-03-21T10:33:42.692 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.690+0000 7fcc36390740 1 Processor -- start 2024-03-21T10:33:42.692 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.690+0000 7fcc36390740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5572b9bf2520 0x5572b9c14bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:33:42.692 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.690+0000 7fcc36390740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5572b9bf9260 0x5572b9c15100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:33:42.692 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.690+0000 7fcc36390740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5572b9c01de0 0x5572b9c66080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:33:42.693 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.690+0000 7fcc36390740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc2c1b1d50 con 0x5572b9bf9260 2024-03-21T10:33:42.693 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.690+0000 7fcc36390740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc2c1b16f0 con 0x5572b9bf2520 2024-03-21T10:33:42.693 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.690+0000 7fcc36390740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc2c1aec10 con 0x5572b9c01de0 2024-03-21T10:33:42.693 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.690+0000 7fcc36390740 10 client.0 _ll_register_callbacks cb 0x7fcc1c004590 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T10:33:42.693 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.691+0000 7fcc23e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5572b9bf9260 0x5572b9c15100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:33:42.693 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.691+0000 7fcc23e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5572b9bf9260 0x5572b9c15100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:58318/0 (socket says 172.21.15.47:58318) 2024-03-21T10:33:42.693 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.691+0000 7fcc23e00640 1 -- 172.21.15.47:0/50084748 learned_addr learned my addr 172.21.15.47:0/50084748 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:33:42.693 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.691+0000 7fcc30c00640 1 --2- 172.21.15.47:0/50084748 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5572b9bf2520 0x5572b9c14bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:33:42.693 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.691+0000 7fcc31600640 1 --2- 172.21.15.47:0/50084748 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5572b9c01de0 0x5572b9c66080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:33:42.693 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.691+0000 7fcc23e00640 1 -- 172.21.15.47:0/50084748 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5572b9c01de0 msgr2=0x5572b9c66080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:33:42.693 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.691+0000 7fcc23e00640 1 --2- 172.21.15.47:0/50084748 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5572b9c01de0 0x5572b9c66080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:33:42.693 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.691+0000 7fcc23e00640 1 -- 172.21.15.47:0/50084748 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5572b9bf2520 msgr2=0x5572b9c14bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:33:42.693 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.691+0000 7fcc23e00640 1 --2- 172.21.15.47:0/50084748 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5572b9bf2520 0x5572b9c14bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:33:42.693 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.691+0000 7fcc23e00640 1 -- 172.21.15.47:0/50084748 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5572b9c66640 con 0x5572b9bf9260 2024-03-21T10:33:42.694 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.691+0000 7fcc31600640 1 --2- 172.21.15.47:0/50084748 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5572b9c01de0 0x5572b9c66080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T10:33:42.694 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.692+0000 7fcc23e00640 1 --2- 172.21.15.47:0/50084748 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5572b9bf9260 0x5572b9c15100 secure :-1 s=READY pgs=1155 cs=0 l=1 rev1=1 crypto rx=0x7fcc14006010 tx=0x7fcc140f3fc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4fd487e8d4ff6d6 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:33:42.694 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.692+0000 7fcc1a000640 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-03-21T10:33:42.694 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.692+0000 7fcc1a000640 1 -- 172.21.15.47:0/50084748 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc140c6610 con 0x5572b9bf9260 2024-03-21T10:33:42.694 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.692+0000 7fcc1a000640 1 -- 172.21.15.47:0/50084748 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(101..101 src has 1..101) v4 ==== 9562+0+0 (secure 0 0 0) 0x7fcc140ba270 con 0x5572b9bf9260 2024-03-21T10:33:42.694 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.692+0000 7fcc36390740 1 -- 172.21.15.47:0/50084748 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fcc2401ee50 con 0x5572b9bf9260 2024-03-21T10:33:42.694 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.692+0000 7fcc1a000640 1 -- 172.21.15.47:0/50084748 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7fcc14005340 con 0x5572b9bf9260 2024-03-21T10:33:42.694 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.692+0000 7fcc36390740 10 client.18219 fetch_fsmap learned FSMap version 32 2024-03-21T10:33:42.694 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.692+0000 7fcc36390740 1 -- 172.21.15.47:0/50084748 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x5572b9c6da30 con 0x5572b9bf9260 2024-03-21T10:33:42.694 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.692+0000 7fcc1a000640 1 -- 172.21.15.47:0/50084748 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fcc2401ee50 con 0x5572b9bf9260 2024-03-21T10:33:42.694 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.693+0000 7fcc1a000640 1 -- 172.21.15.47:0/50084748 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 32) v1 ==== 7942+0+0 (secure 0 0 0) 0x7fcc140d4070 con 0x5572b9bf9260 2024-03-21T10:33:42.694 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.693+0000 7fcc36390740 10 client.18219 fetch_fsmap finished waiting for FSMap version 32 2024-03-21T10:33:42.694 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.693+0000 7fcc36390740 10 client.18219 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.e 2024-03-21T10:33:42.695 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.693+0000 7fcc36390740 20 client.18219 populate_metadata read hostname 'smithi047' 2024-03-21T10:33:42.695 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.693+0000 7fcc36390740 1 --2- 172.21.15.47:0/50084748 >> [v2:172.21.15.100:6834/2597849957,v1:172.21.15.100:6835/2597849957] conn(0x5572b9c6e160 0x5572b9c70580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:33:42.695 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.693+0000 7fcc36390740 4 client.18219 mds_command: new command op to 34454 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-21T10:33:42.695 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.693+0000 7fcc36390740 1 -- 172.21.15.47:0/50084748 --> [v2:172.21.15.100:6834/2597849957,v1:172.21.15.100:6835/2597849957] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x5572b9b63b50 con 0x5572b9c6e160 2024-03-21T10:33:42.695 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.693+0000 7fcc30c00640 1 --2- 172.21.15.47:0/50084748 >> [v2:172.21.15.100:6834/2597849957,v1:172.21.15.100:6835/2597849957] conn(0x5572b9c6e160 0x5572b9c70580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:33:42.696 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.694+0000 7fcc30c00640 1 --2- 172.21.15.47:0/50084748 >> [v2:172.21.15.100:6834/2597849957,v1:172.21.15.100:6835/2597849957] conn(0x5572b9c6e160 0x5572b9c70580 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=2130c4bd0cc63afa server_cookie=4aa46f5838838a99 in_seq=0 out_seq=0 2024-03-21T10:33:42.696 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.694+0000 7fcc1a000640 10 client.18219 ms_handle_connect on v2:172.21.15.100:6834/2597849957 2024-03-21T10:33:42.696 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.694+0000 7fcc1a000640 1 -- 172.21.15.47:0/50084748 <== mds.2 v2:172.21.15.100:6834/2597849957 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x5572b9b63b50 con 0x5572b9c6e160 2024-03-21T10:33:42.696 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.694+0000 7fcc1a000640 10 client.18219 handle_command_reply: tid=0 2024-03-21T10:33:42.696 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.694+0000 7fcc36390740 1 client.18219 shutdown 2024-03-21T10:33:42.697 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.695+0000 7fcc36390740 1 -- 172.21.15.47:0/50084748 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5572b9bf9260 msgr2=0x5572b9c15100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:33:42.697 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.695+0000 7fcc36390740 1 --2- 172.21.15.47:0/50084748 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5572b9bf9260 0x5572b9c15100 secure :-1 s=READY pgs=1155 cs=0 l=1 rev1=1 crypto rx=0x7fcc14006010 tx=0x7fcc140f3fc0 comp rx=0 tx=0).stop 2024-03-21T10:33:42.697 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.695+0000 7fcc36390740 1 -- 172.21.15.47:0/50084748 shutdown_connections 2024-03-21T10:33:42.697 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.695+0000 7fcc36390740 1 --2- 172.21.15.47:0/50084748 >> [v2:172.21.15.100:6834/2597849957,v1:172.21.15.100:6835/2597849957] conn(0x5572b9c6e160 0x5572b9c70580 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:33:42.697 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.695+0000 7fcc36390740 1 --2- 172.21.15.47:0/50084748 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5572b9c01de0 0x5572b9c66080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:33:42.697 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.695+0000 7fcc36390740 1 --2- 172.21.15.47:0/50084748 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5572b9bf9260 0x5572b9c15100 unknown :-1 s=CLOSED pgs=1155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:33:42.697 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.695+0000 7fcc36390740 1 --2- 172.21.15.47:0/50084748 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5572b9bf2520 0x5572b9c14bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:33:42.697 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.695+0000 7fcc36390740 1 -- 172.21.15.47:0/50084748 >> 172.21.15.47:0/50084748 conn(0x5572b9bf2f80 msgr2=0x5572b9bf9640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:33:42.697 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.695+0000 7fcc1a000640 0 client.18219 ms_handle_reset on v2:172.21.15.100:6834/2597849957 2024-03-21T10:33:42.697 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.695+0000 7fcc36390740 1 -- 172.21.15.47:0/50084748 shutdown_connections 2024-03-21T10:33:42.697 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.695+0000 7fcc36390740 1 -- 172.21.15.47:0/50084748 wait complete. 2024-03-21T10:33:42.697 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.695+0000 7fcc36390740 20 client.18219 trim_cache size 0 max 16384 2024-03-21T10:33:42.698 INFO:teuthology.orchestra.run.smithi047.stdout:[] 2024-03-21T10:33:42.698 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.696+0000 7fcc32000640 1 -- 172.21.15.47:0/1300177907 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7fcc0c0a3140 msgr2=0x7fcc0c0a5600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:33:42.698 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.696+0000 7fcc32000640 1 --2- 172.21.15.47:0/1300177907 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7fcc0c0a3140 0x7fcc0c0a5600 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fcc24005fd0 tx=0x7fcc24009530 comp rx=0 tx=0).stop 2024-03-21T10:33:42.698 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.696+0000 7fcc32000640 1 -- 172.21.15.47:0/1300177907 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fcc2c102ff0 msgr2=0x7fcc2c100860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:33:42.698 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.696+0000 7fcc32000640 1 --2- 172.21.15.47:0/1300177907 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fcc2c102ff0 0x7fcc2c100860 secure :-1 s=READY pgs=1151 cs=0 l=1 rev1=1 crypto rx=0x7fcc14005940 tx=0x7fcc140045e0 comp rx=0 tx=0).stop 2024-03-21T10:33:42.699 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.696+0000 7fcc32000640 1 -- 172.21.15.47:0/1300177907 shutdown_connections 2024-03-21T10:33:42.699 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.696+0000 7fcc32000640 1 --2- 172.21.15.47:0/1300177907 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcc2c10baf0 0x7fcc2c100da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:33:42.699 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.696+0000 7fcc32000640 1 --2- 172.21.15.47:0/1300177907 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fcc2c102ff0 0x7fcc2c100860 unknown :-1 s=CLOSED pgs=1151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:33:42.699 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.696+0000 7fcc32000640 1 --2- 172.21.15.47:0/1300177907 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7fcc0c0a3140 0x7fcc0c0a5600 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:33:42.699 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.696+0000 7fcc32000640 1 --2- 172.21.15.47:0/1300177907 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcc2c1026d0 0x7fcc2c100320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:33:42.699 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.696+0000 7fcc32000640 1 -- 172.21.15.47:0/1300177907 >> 172.21.15.47:0/1300177907 conn(0x7fcc2c0fbef0 msgr2=0x7fcc2c0fcd40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:33:42.699 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.696+0000 7fcc32000640 1 -- 172.21.15.47:0/1300177907 shutdown_connections 2024-03-21T10:33:42.699 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.696+0000 7fcc32000640 1 -- 172.21.15.47:0/1300177907 wait complete. 2024-03-21T10:33:42.709 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:33:42 smithi047 ceph-mon[31547]: from='client.? 172.21.15.47:0/2181225323' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-21T10:33:42.712 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-21T10:33:42.712 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-03-21T10:33:42.946 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.947+0000 7fcdcb800640 1 -- 172.21.15.47:0/357700078 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fcdc4103840 msgr2=0x7fcdc4103cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:33:42.946 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.947+0000 7fcdcb800640 1 --2- 172.21.15.47:0/357700078 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fcdc4103840 0x7fcdc4103cc0 secure :-1 s=READY pgs=1156 cs=0 l=1 rev1=1 crypto rx=0x7fcdac009930 tx=0x7fcdac02bc30 comp rx=0 tx=0).stop 2024-03-21T10:33:42.946 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.947+0000 7fcdcb800640 1 -- 172.21.15.47:0/357700078 shutdown_connections 2024-03-21T10:33:42.946 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.947+0000 7fcdcb800640 1 --2- 172.21.15.47:0/357700078 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcdc4104200 0x7fcdc4110710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:33:42.946 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.947+0000 7fcdcb800640 1 --2- 172.21.15.47:0/357700078 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fcdc4103840 0x7fcdc4103cc0 unknown :-1 s=CLOSED pgs=1156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:33:42.946 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.947+0000 7fcdcb800640 1 --2- 172.21.15.47:0/357700078 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcdc4109850 0x7fcdc4109c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:33:42.946 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.947+0000 7fcdcb800640 1 -- 172.21.15.47:0/357700078 >> 172.21.15.47:0/357700078 conn(0x7fcdc40ff2f0 msgr2=0x7fcdc4101710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:33:42.947 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.947+0000 7fcdcb800640 1 -- 172.21.15.47:0/357700078 shutdown_connections 2024-03-21T10:33:42.947 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.947+0000 7fcdcb800640 1 -- 172.21.15.47:0/357700078 wait complete. 2024-03-21T10:33:42.948 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.948+0000 7fcdcb800640 1 Processor -- start 2024-03-21T10:33:42.948 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.948+0000 7fcdcb800640 1 -- start start 2024-03-21T10:33:42.948 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.949+0000 7fcdcb800640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcdc4103840 0x7fcdc419a0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:33:42.948 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.949+0000 7fcdcb800640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fcdc4104200 0x7fcdc419a600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:33:42.948 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.949+0000 7fcdcb800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcdc4109850 0x7fcdc419e980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:33:42.948 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.949+0000 7fcdcb800640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcdc4113b60 con 0x7fcdc4104200 2024-03-21T10:33:42.948 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.949+0000 7fcdcb800640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcdc41139e0 con 0x7fcdc4109850 2024-03-21T10:33:42.949 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.949+0000 7fcdcb800640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcdc4113ce0 con 0x7fcdc4103840 2024-03-21T10:33:42.949 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.949+0000 7fcdbbe00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fcdc4104200 0x7fcdc419a600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:33:42.949 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.949+0000 7fcdbbe00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fcdc4104200 0x7fcdc419a600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:58338/0 (socket says 172.21.15.47:58338) 2024-03-21T10:33:42.949 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.949+0000 7fcdbbe00640 1 -- 172.21.15.47:0/4050005162 learned_addr learned my addr 172.21.15.47:0/4050005162 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:33:42.949 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.949+0000 7fcdc9800640 1 --2- 172.21.15.47:0/4050005162 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcdc4109850 0x7fcdc419e980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:33:42.949 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.949+0000 7fcdc8e00640 1 --2- 172.21.15.47:0/4050005162 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcdc4103840 0x7fcdc419a0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:33:42.949 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.950+0000 7fcdbbe00640 1 -- 172.21.15.47:0/4050005162 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcdc4103840 msgr2=0x7fcdc419a0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:33:42.949 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.950+0000 7fcdbbe00640 1 --2- 172.21.15.47:0/4050005162 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcdc4103840 0x7fcdc419a0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:33:42.949 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.950+0000 7fcdbbe00640 1 -- 172.21.15.47:0/4050005162 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcdc4109850 msgr2=0x7fcdc419e980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:33:42.949 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.950+0000 7fcdbbe00640 1 --2- 172.21.15.47:0/4050005162 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcdc4109850 0x7fcdc419e980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:33:42.949 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.950+0000 7fcdbbe00640 1 -- 172.21.15.47:0/4050005162 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcdc419f0b0 con 0x7fcdc4104200 2024-03-21T10:33:42.949 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.950+0000 7fcdbbe00640 1 --2- 172.21.15.47:0/4050005162 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fcdc4104200 0x7fcdc419a600 secure :-1 s=READY pgs=1157 cs=0 l=1 rev1=1 crypto rx=0x7fcdac000c00 tx=0x7fcdac0045e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:33:42.950 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.951+0000 7fcdc2000640 1 -- 172.21.15.47:0/4050005162 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcdac043020 con 0x7fcdc4104200 2024-03-21T10:33:42.950 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.951+0000 7fcdcb800640 1 -- 172.21.15.47:0/4050005162 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcdc419f340 con 0x7fcdc4104200 2024-03-21T10:33:42.950 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.951+0000 7fcdc2000640 1 -- 172.21.15.47:0/4050005162 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fcdac02cb70 con 0x7fcdc4104200 2024-03-21T10:33:42.950 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.951+0000 7fcdc2000640 1 -- 172.21.15.47:0/4050005162 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcdac003b50 con 0x7fcdc4104200 2024-03-21T10:33:42.950 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.951+0000 7fcdcb800640 1 -- 172.21.15.47:0/4050005162 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcdc41a6c30 con 0x7fcdc4104200 2024-03-21T10:33:42.953 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.954+0000 7fcdce57c740 1 Processor -- start 2024-03-21T10:33:42.953 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.954+0000 7fcdce57c740 1 -- start start 2024-03-21T10:33:42.953 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.955+0000 7fcdce57c740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555fa28ba6d0 0x555fa28baab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:33:42.953 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.955+0000 7fcdce57c740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555fa28bb080 0x555fa28c3630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:33:42.953 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.955+0000 7fcdce57c740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x555fa28c3c00 0x555fa28c6050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:33:42.953 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.955+0000 7fcdce57c740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x555fa281e2c0 con 0x555fa28c3c00 2024-03-21T10:33:42.953 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.955+0000 7fcdce57c740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x555fa27c6c90 con 0x555fa28ba6d0 2024-03-21T10:33:42.953 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.955+0000 7fcdce57c740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x555fa281a360 con 0x555fa28bb080 2024-03-21T10:33:42.955 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.955+0000 7fcdc9800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555fa28ba6d0 0x555fa28baab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:33:42.955 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.955+0000 7fcdc9800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555fa28ba6d0 0x555fa28baab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:37118/0 (socket says 172.21.15.47:37118) 2024-03-21T10:33:42.955 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.955+0000 7fcdc9800640 1 -- 172.21.15.47:0/3796320997 learned_addr learned my addr 172.21.15.47:0/3796320997 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:33:42.955 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.955+0000 7fcdc8e00640 1 --2- 172.21.15.47:0/3796320997 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555fa28bb080 0x555fa28c3630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:33:42.955 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.955+0000 7fcdc2000640 1 -- 172.21.15.47:0/4050005162 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 27) v1 ==== 137985+0+0 (secure 0 0 0) 0x7fcdac046020 con 0x7fcdc4104200 2024-03-21T10:33:42.955 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.956+0000 7fcdc9800640 1 -- 172.21.15.47:0/3796320997 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555fa28bb080 msgr2=0x555fa28c3630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:33:42.955 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.956+0000 7fcdc9800640 1 --2- 172.21.15.47:0/3796320997 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555fa28bb080 0x555fa28c3630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:33:42.955 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.956+0000 7fcdc9800640 1 -- 172.21.15.47:0/3796320997 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x555fa28c3c00 msgr2=0x555fa28c6050 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-21T10:33:42.955 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.956+0000 7fcdc9800640 1 --2- 172.21.15.47:0/3796320997 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x555fa28c3c00 0x555fa28c6050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:33:42.955 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.956+0000 7fcdc9800640 1 -- 172.21.15.47:0/3796320997 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555fa27eb080 con 0x555fa28ba6d0 2024-03-21T10:33:42.955 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.956+0000 7fcdc9800640 1 --2- 172.21.15.47:0/3796320997 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555fa28ba6d0 0x555fa28baab0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fcdbc010370 tx=0x7fcdbc010830 comp rx=0 tx=0).ready entity=mon.2 client_cookie=67eedac044fd2e5a server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:33:42.956 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.957+0000 7fcdba000640 1 -- 172.21.15.47:0/3796320997 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcdbc016070 con 0x555fa28ba6d0 2024-03-21T10:33:42.956 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.957+0000 7fcdba000640 1 -- 172.21.15.47:0/3796320997 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fcdbc004d20 con 0x555fa28ba6d0 2024-03-21T10:33:42.956 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.957+0000 7fcdba000640 1 -- 172.21.15.47:0/3796320997 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcdbc005350 con 0x555fa28ba6d0 2024-03-21T10:33:42.958 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.957+0000 7fcdc2000640 1 --2- 172.21.15.47:0/4050005162 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7fcda80a3090 0x7fcda80a5550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:33:42.958 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.957+0000 7fcdc2000640 1 -- 172.21.15.47:0/4050005162 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(101..101 src has 1..101) v4 ==== 9562+0+0 (secure 0 0 0) 0x7fcdac0f1c00 con 0x7fcdc4104200 2024-03-21T10:33:42.958 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.957+0000 7fcdce57c740 1 -- 172.21.15.47:0/3796320997 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555fa28ba6d0 msgr2=0x555fa28baab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:33:42.958 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.957+0000 7fcdce57c740 1 --2- 172.21.15.47:0/3796320997 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555fa28ba6d0 0x555fa28baab0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fcdbc010370 tx=0x7fcdbc010830 comp rx=0 tx=0).stop 2024-03-21T10:33:42.958 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.958+0000 7fcdce57c740 1 -- 172.21.15.47:0/3796320997 shutdown_connections 2024-03-21T10:33:42.958 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.958+0000 7fcdc8e00640 1 --2- 172.21.15.47:0/4050005162 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7fcda80a3090 0x7fcda80a5550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:33:42.958 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.958+0000 7fcdce57c740 1 --2- 172.21.15.47:0/3796320997 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x555fa28c3c00 0x555fa28c6050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:33:42.958 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.958+0000 7fcdce57c740 1 --2- 172.21.15.47:0/3796320997 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555fa28bb080 0x555fa28c3630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:33:42.959 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.958+0000 7fcdce57c740 1 --2- 172.21.15.47:0/3796320997 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555fa28ba6d0 0x555fa28baab0 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:33:42.959 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.958+0000 7fcdce57c740 1 -- 172.21.15.47:0/3796320997 >> 172.21.15.47:0/3796320997 conn(0x555fa28b4da0 msgr2=0x555fa28b5180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:33:42.959 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.958+0000 7fcdce57c740 1 -- 172.21.15.47:0/3796320997 shutdown_connections 2024-03-21T10:33:42.959 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.958+0000 7fcdce57c740 1 -- 172.21.15.47:0/3796320997 wait complete. 2024-03-21T10:33:42.959 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.958+0000 7fcdc8e00640 1 --2- 172.21.15.47:0/4050005162 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7fcda80a3090 0x7fcda80a5550 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x555fa28b5e10 tx=0x7fcdb4005f90 comp rx=0 tx=0).ready entity=mgr.15687 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:33:42.959 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.958+0000 7fcdce57c740 1 -- start start 2024-03-21T10:33:42.959 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.959+0000 7fcdce57c740 1 Processor -- start 2024-03-21T10:33:42.959 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.959+0000 7fcdce57c740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x555fa28ba6d0 0x555fa28c09d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:33:42.959 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.959+0000 7fcdce57c740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555fa28bb080 0x555fa28c0f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:33:42.959 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.959+0000 7fcdce57c740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555fa28c3c00 0x555fa28c1450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:33:42.959 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.959+0000 7fcdce57c740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x555fa28c6b70 con 0x555fa28ba6d0 2024-03-21T10:33:42.959 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.959+0000 7fcdce57c740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x555fa27da0d0 con 0x555fa28c3c00 2024-03-21T10:33:42.959 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.959+0000 7fcdce57c740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x555fa28c6cf0 con 0x555fa28bb080 2024-03-21T10:33:42.959 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.959+0000 7fcdce57c740 10 client.0 _ll_register_callbacks cb 0x555fa27dee80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T10:33:42.960 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.959+0000 7fcdc8e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x555fa28ba6d0 0x555fa28c09d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:33:42.960 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.959+0000 7fcdc8e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x555fa28ba6d0 0x555fa28c09d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:58358/0 (socket says 172.21.15.47:58358) 2024-03-21T10:33:42.960 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.959+0000 7fcdc8e00640 1 -- 172.21.15.47:0/1728117708 learned_addr learned my addr 172.21.15.47:0/1728117708 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:33:42.960 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.959+0000 7fcdc9800640 1 --2- 172.21.15.47:0/1728117708 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555fa28c3c00 0x555fa28c1450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:33:42.960 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.960+0000 7fcdbbe00640 1 --2- 172.21.15.47:0/1728117708 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555fa28bb080 0x555fa28c0f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:33:42.960 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.960+0000 7fcdc8e00640 1 -- 172.21.15.47:0/1728117708 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555fa28bb080 msgr2=0x555fa28c0f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:33:42.960 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.960+0000 7fcdc8e00640 1 --2- 172.21.15.47:0/1728117708 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555fa28bb080 0x555fa28c0f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:33:42.960 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.960+0000 7fcdc8e00640 1 -- 172.21.15.47:0/1728117708 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555fa28c3c00 msgr2=0x555fa28c1450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:33:42.960 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.960+0000 7fcdc8e00640 1 --2- 172.21.15.47:0/1728117708 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555fa28c3c00 0x555fa28c1450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:33:42.960 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.960+0000 7fcdc8e00640 1 -- 172.21.15.47:0/1728117708 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x555fa25323f0 con 0x555fa28ba6d0 2024-03-21T10:33:42.962 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.960+0000 7fcdc8e00640 1 --2- 172.21.15.47:0/1728117708 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x555fa28ba6d0 0x555fa28c09d0 secure :-1 s=READY pgs=1158 cs=0 l=1 rev1=1 crypto rx=0x7fcdb40042f0 tx=0x7fcdb4004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a2454047d7ebdece server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:33:42.962 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.960+0000 7fcdb2a00640 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-03-21T10:33:42.962 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.961+0000 7fcdb2a00640 1 -- 172.21.15.47:0/1728117708 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcdb4029440 con 0x555fa28ba6d0 2024-03-21T10:33:42.962 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.961+0000 7fcdb2a00640 1 -- 172.21.15.47:0/1728117708 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(101..101 src has 1..101) v4 ==== 9562+0+0 (secure 0 0 0) 0x7fcdb401f420 con 0x555fa28ba6d0 2024-03-21T10:33:42.962 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.961+0000 7fcdce57c740 1 -- 172.21.15.47:0/1728117708 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fcdbc004d20 con 0x555fa28ba6d0 2024-03-21T10:33:42.962 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.961+0000 7fcdb2a00640 1 -- 172.21.15.47:0/1728117708 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7fcdb401f620 con 0x555fa28ba6d0 2024-03-21T10:33:42.962 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.961+0000 7fcdce57c740 10 client.18228 fetch_fsmap learned FSMap version 32 2024-03-21T10:33:42.962 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.961+0000 7fcdce57c740 1 -- 172.21.15.47:0/1728117708 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x555fa25fc7d0 con 0x555fa28ba6d0 2024-03-21T10:33:42.962 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.961+0000 7fcdb2a00640 1 -- 172.21.15.47:0/1728117708 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fcdbc004d20 con 0x555fa28ba6d0 2024-03-21T10:33:42.963 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.962+0000 7fcdb2a00640 1 -- 172.21.15.47:0/1728117708 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 32) v1 ==== 7942+0+0 (secure 0 0 0) 0x7fcdb4027070 con 0x555fa28ba6d0 2024-03-21T10:33:42.963 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.962+0000 7fcdce57c740 10 client.18228 fetch_fsmap finished waiting for FSMap version 32 2024-03-21T10:33:42.963 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.962+0000 7fcdce57c740 10 client.18228 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-03-21T10:33:42.963 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.962+0000 7fcdce57c740 20 client.18228 populate_metadata read hostname 'smithi047' 2024-03-21T10:33:42.963 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.962+0000 7fcdce57c740 1 --2- 172.21.15.47:0/1728117708 >> [v2:172.21.15.100:6836/3497644162,v1:172.21.15.100:6837/3497644162] conn(0x555fa28d3c60 0x555fa28d6090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:33:42.963 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.962+0000 7fcdce57c740 4 client.18228 mds_command: new command op to 34493 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T10:33:42.963 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.962+0000 7fcdce57c740 1 -- 172.21.15.47:0/1728117708 --> [v2:172.21.15.100:6836/3497644162,v1:172.21.15.100:6837/3497644162] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x555fa282b800 con 0x555fa28d3c60 2024-03-21T10:33:42.963 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.962+0000 7fcdbbe00640 1 --2- 172.21.15.47:0/1728117708 >> [v2:172.21.15.100:6836/3497644162,v1:172.21.15.100:6837/3497644162] conn(0x555fa28d3c60 0x555fa28d6090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:33:42.963 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.963+0000 7fcdbbe00640 1 --2- 172.21.15.47:0/1728117708 >> [v2:172.21.15.100:6836/3497644162,v1:172.21.15.100:6837/3497644162] conn(0x555fa28d3c60 0x555fa28d6090 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=a08f26632a087380 server_cookie=b7114de145ddb0a8 in_seq=0 out_seq=0 2024-03-21T10:33:42.963 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.963+0000 7fcdb2a00640 10 client.18228 ms_handle_connect on v2:172.21.15.100:6836/3497644162 2024-03-21T10:33:42.964 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.965+0000 7fcdb2a00640 1 -- 172.21.15.47:0/1728117708 <== mds.1 v2:172.21.15.100:6836/3497644162 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x555fa282b800 con 0x555fa28d3c60 2024-03-21T10:33:42.964 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.965+0000 7fcdb2a00640 10 client.18228 handle_command_reply: tid=0 2024-03-21T10:33:42.964 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.965+0000 7fcdce57c740 1 client.18228 shutdown 2024-03-21T10:33:42.965 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.965+0000 7fcdce57c740 1 -- 172.21.15.47:0/1728117708 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x555fa28ba6d0 msgr2=0x555fa28c09d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:33:42.965 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.965+0000 7fcdce57c740 1 --2- 172.21.15.47:0/1728117708 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x555fa28ba6d0 0x555fa28c09d0 secure :-1 s=READY pgs=1158 cs=0 l=1 rev1=1 crypto rx=0x7fcdb40042f0 tx=0x7fcdb4004730 comp rx=0 tx=0).stop 2024-03-21T10:33:42.965 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.965+0000 7fcdce57c740 1 -- 172.21.15.47:0/1728117708 shutdown_connections 2024-03-21T10:33:42.965 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.965+0000 7fcdce57c740 1 --2- 172.21.15.47:0/1728117708 >> [v2:172.21.15.100:6836/3497644162,v1:172.21.15.100:6837/3497644162] conn(0x555fa28d3c60 0x555fa28d6090 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:33:42.965 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.965+0000 7fcdce57c740 1 --2- 172.21.15.47:0/1728117708 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555fa28c3c00 0x555fa28c1450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:33:42.965 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.965+0000 7fcdb2a00640 0 client.18228 ms_handle_reset on v2:172.21.15.100:6836/3497644162 2024-03-21T10:33:42.965 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.965+0000 7fcdce57c740 1 --2- 172.21.15.47:0/1728117708 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555fa28bb080 0x555fa28c0f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:33:42.965 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.965+0000 7fcdce57c740 1 --2- 172.21.15.47:0/1728117708 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x555fa28ba6d0 0x555fa28c09d0 unknown :-1 s=CLOSED pgs=1158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:33:42.965 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.965+0000 7fcdce57c740 1 -- 172.21.15.47:0/1728117708 >> 172.21.15.47:0/1728117708 conn(0x555fa28b4da0 msgr2=0x555fa28b6260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:33:42.965 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.965+0000 7fcdce57c740 1 -- 172.21.15.47:0/1728117708 shutdown_connections 2024-03-21T10:33:42.965 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.965+0000 7fcdce57c740 1 -- 172.21.15.47:0/1728117708 wait complete. 2024-03-21T10:33:42.965 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.965+0000 7fcdce57c740 20 client.18228 trim_cache size 0 max 16384 2024-03-21T10:33:42.977 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:33:42 smithi100 ceph-mon[35193]: from='client.? 172.21.15.47:0/2181225323' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-21T10:33:42.978 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.979+0000 7fcdce57c740 1 Processor -- start 2024-03-21T10:33:42.978 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.979+0000 7fcdce57c740 1 -- start start 2024-03-21T10:33:42.978 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.980+0000 7fcdce57c740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555fa28ba6d0 0x555fa28c3100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:33:42.978 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.980+0000 7fcdce57c740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x555fa28bb080 0x555fa28bdc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:33:42.978 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.980+0000 7fcdce57c740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555fa28c3c00 0x555fa28be170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:33:42.978 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.980+0000 7fcdce57c740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcdc41b0c90 con 0x555fa28bb080 2024-03-21T10:33:42.978 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.980+0000 7fcdce57c740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcdc41b2610 con 0x555fa28c3c00 2024-03-21T10:33:42.978 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.980+0000 7fcdce57c740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcdc41b12f0 con 0x555fa28ba6d0 2024-03-21T10:33:42.980 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.980+0000 7fcdbbe00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x555fa28bb080 0x555fa28bdc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:33:42.980 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.980+0000 7fcdbbe00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x555fa28bb080 0x555fa28bdc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:58372/0 (socket says 172.21.15.47:58372) 2024-03-21T10:33:42.980 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.980+0000 7fcdbbe00640 1 -- 172.21.15.47:0/22871608 learned_addr learned my addr 172.21.15.47:0/22871608 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:33:42.980 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.980+0000 7fcdc9800640 1 --2- 172.21.15.47:0/22871608 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555fa28c3c00 0x555fa28be170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:33:42.980 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.980+0000 7fcdc8e00640 1 --2- 172.21.15.47:0/22871608 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555fa28ba6d0 0x555fa28c3100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:33:42.980 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.980+0000 7fcdbbe00640 1 -- 172.21.15.47:0/22871608 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555fa28ba6d0 msgr2=0x555fa28c3100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:33:42.980 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.980+0000 7fcdbbe00640 1 --2- 172.21.15.47:0/22871608 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555fa28ba6d0 0x555fa28c3100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:33:42.981 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.980+0000 7fcdbbe00640 1 -- 172.21.15.47:0/22871608 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555fa28c3c00 msgr2=0x555fa28be170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:33:42.981 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.980+0000 7fcdbbe00640 1 --2- 172.21.15.47:0/22871608 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555fa28c3c00 0x555fa28be170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:33:42.981 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.980+0000 7fcdbbe00640 1 -- 172.21.15.47:0/22871608 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555fa28ba180 con 0x555fa28bb080 2024-03-21T10:33:42.981 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.980+0000 7fcdc9800640 1 --2- 172.21.15.47:0/22871608 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555fa28c3c00 0x555fa28be170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T10:33:42.981 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.981+0000 7fcdbbe00640 1 --2- 172.21.15.47:0/22871608 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x555fa28bb080 0x555fa28bdc30 secure :-1 s=READY pgs=1159 cs=0 l=1 rev1=1 crypto rx=0x7fcdac0bdc20 tx=0x7fcdac02bc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7784339fb2a38f17 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:33:42.981 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.981+0000 7fcdb9600640 1 -- 172.21.15.47:0/22871608 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcdac0bdc90 con 0x555fa28bb080 2024-03-21T10:33:42.981 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.981+0000 7fcdb9600640 1 -- 172.21.15.47:0/22871608 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fcdac0c1de0 con 0x555fa28bb080 2024-03-21T10:33:42.981 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.981+0000 7fcdb9600640 1 -- 172.21.15.47:0/22871608 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcdac003cf0 con 0x555fa28bb080 2024-03-21T10:33:42.981 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.982+0000 7fcdce57c740 1 -- 172.21.15.47:0/22871608 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x555fa28bb080 msgr2=0x555fa28bdc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:33:42.981 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.982+0000 7fcdce57c740 1 --2- 172.21.15.47:0/22871608 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x555fa28bb080 0x555fa28bdc30 secure :-1 s=READY pgs=1159 cs=0 l=1 rev1=1 crypto rx=0x7fcdac0bdc20 tx=0x7fcdac02bc30 comp rx=0 tx=0).stop 2024-03-21T10:33:42.981 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.982+0000 7fcdce57c740 1 -- 172.21.15.47:0/22871608 shutdown_connections 2024-03-21T10:33:42.981 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.982+0000 7fcdce57c740 1 --2- 172.21.15.47:0/22871608 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555fa28c3c00 0x555fa28be170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:33:42.981 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.982+0000 7fcdce57c740 1 --2- 172.21.15.47:0/22871608 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x555fa28bb080 0x555fa28bdc30 unknown :-1 s=CLOSED pgs=1159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:33:42.981 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.982+0000 7fcdce57c740 1 --2- 172.21.15.47:0/22871608 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555fa28ba6d0 0x555fa28c3100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:33:42.981 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.982+0000 7fcdce57c740 1 -- 172.21.15.47:0/22871608 >> 172.21.15.47:0/22871608 conn(0x555fa28b4da0 msgr2=0x555fa28b6930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:33:42.982 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.982+0000 7fcdce57c740 1 -- 172.21.15.47:0/22871608 shutdown_connections 2024-03-21T10:33:42.982 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.982+0000 7fcdce57c740 1 -- 172.21.15.47:0/22871608 wait complete. 2024-03-21T10:33:42.984 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.982+0000 7fcdce57c740 1 -- start start 2024-03-21T10:33:42.984 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.982+0000 7fcdce57c740 1 Processor -- start 2024-03-21T10:33:42.984 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.983+0000 7fcdce57c740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x555fa28ba6d0 0x555fa28c9b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:33:42.985 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.983+0000 7fcdce57c740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555fa28bb080 0x555fa28ca060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:33:42.985 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.983+0000 7fcdce57c740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555fa28c3c00 0x555fa292e530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:33:42.985 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.983+0000 7fcdce57c740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcdc41b1fb0 con 0x555fa28ba6d0 2024-03-21T10:33:42.985 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.983+0000 7fcdce57c740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcdc41b1950 con 0x555fa28bb080 2024-03-21T10:33:42.985 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.983+0000 7fcdce57c740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcdc41aee70 con 0x555fa28c3c00 2024-03-21T10:33:42.985 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.983+0000 7fcdce57c740 10 client.0 _ll_register_callbacks cb 0x7fcdb4012b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T10:33:42.985 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.983+0000 7fcdc8e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x555fa28ba6d0 0x555fa28c9b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:33:42.985 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.983+0000 7fcdc8e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x555fa28ba6d0 0x555fa28c9b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:58376/0 (socket says 172.21.15.47:58376) 2024-03-21T10:33:42.985 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.983+0000 7fcdc8e00640 1 -- 172.21.15.47:0/4099738087 learned_addr learned my addr 172.21.15.47:0/4099738087 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:33:42.985 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.983+0000 7fcdbbe00640 1 --2- 172.21.15.47:0/4099738087 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555fa28bb080 0x555fa28ca060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:33:42.985 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.984+0000 7fcdc9800640 1 --2- 172.21.15.47:0/4099738087 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555fa28c3c00 0x555fa292e530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:33:42.985 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.984+0000 7fcdc8e00640 1 -- 172.21.15.47:0/4099738087 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555fa28c3c00 msgr2=0x555fa292e530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:33:42.985 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.984+0000 7fcdc8e00640 1 --2- 172.21.15.47:0/4099738087 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555fa28c3c00 0x555fa292e530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:33:42.986 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.984+0000 7fcdc8e00640 1 -- 172.21.15.47:0/4099738087 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555fa28bb080 msgr2=0x555fa28ca060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:33:42.986 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.984+0000 7fcdc8e00640 1 --2- 172.21.15.47:0/4099738087 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555fa28bb080 0x555fa28ca060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:33:42.986 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.984+0000 7fcdc8e00640 1 -- 172.21.15.47:0/4099738087 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x555fa292eac0 con 0x555fa28ba6d0 2024-03-21T10:33:42.986 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.984+0000 7fcdc9800640 1 --2- 172.21.15.47:0/4099738087 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555fa28c3c00 0x555fa292e530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T10:33:42.986 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.984+0000 7fcdc8e00640 1 --2- 172.21.15.47:0/4099738087 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x555fa28ba6d0 0x555fa28c9b20 secure :-1 s=READY pgs=1160 cs=0 l=1 rev1=1 crypto rx=0x7fcdb40093f0 tx=0x7fcdb4009420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a6275ca22bca6473 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:33:42.986 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.984+0000 7fcdb2a00640 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-03-21T10:33:42.986 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.984+0000 7fcdb2a00640 1 -- 172.21.15.47:0/4099738087 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcdb403b020 con 0x555fa28ba6d0 2024-03-21T10:33:42.986 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.984+0000 7fcdb2a00640 1 -- 172.21.15.47:0/4099738087 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(101..101 src has 1..101) v4 ==== 9562+0+0 (secure 0 0 0) 0x7fcdb401f9d0 con 0x555fa28ba6d0 2024-03-21T10:33:42.986 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.984+0000 7fcdce57c740 1 -- 172.21.15.47:0/4099738087 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fcdac0c1de0 con 0x555fa28ba6d0 2024-03-21T10:33:42.986 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.985+0000 7fcdb2a00640 1 -- 172.21.15.47:0/4099738087 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7fcdb4043050 con 0x555fa28ba6d0 2024-03-21T10:33:42.986 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.985+0000 7fcdce57c740 10 client.18234 fetch_fsmap learned FSMap version 32 2024-03-21T10:33:42.986 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.985+0000 7fcdce57c740 1 -- 172.21.15.47:0/4099738087 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x555fa292fdc0 con 0x555fa28ba6d0 2024-03-21T10:33:42.986 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.985+0000 7fcdb2a00640 1 -- 172.21.15.47:0/4099738087 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fcdac0c1de0 con 0x555fa28ba6d0 2024-03-21T10:33:42.986 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.985+0000 7fcdb2a00640 1 -- 172.21.15.47:0/4099738087 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 32) v1 ==== 7942+0+0 (secure 0 0 0) 0x7fcdb4027040 con 0x555fa28ba6d0 2024-03-21T10:33:42.987 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.985+0000 7fcdce57c740 10 client.18234 fetch_fsmap finished waiting for FSMap version 32 2024-03-21T10:33:42.987 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.985+0000 7fcdce57c740 10 client.18234 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-03-21T10:33:42.987 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.985+0000 7fcdce57c740 20 client.18234 populate_metadata read hostname 'smithi047' 2024-03-21T10:33:42.987 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.985+0000 7fcdce57c740 1 --2- 172.21.15.47:0/4099738087 >> [v2:172.21.15.100:6836/3497644162,v1:172.21.15.100:6837/3497644162] conn(0x555fa28d3c60 0x555fa2932580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:33:42.987 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.985+0000 7fcdce57c740 4 client.18234 mds_command: new command op to 34493 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-21T10:33:42.987 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.985+0000 7fcdce57c740 1 -- 172.21.15.47:0/4099738087 --> [v2:172.21.15.100:6836/3497644162,v1:172.21.15.100:6837/3497644162] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x555fa2825b50 con 0x555fa28d3c60 2024-03-21T10:33:42.987 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.986+0000 7fcdbbe00640 1 --2- 172.21.15.47:0/4099738087 >> [v2:172.21.15.100:6836/3497644162,v1:172.21.15.100:6837/3497644162] conn(0x555fa28d3c60 0x555fa2932580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:33:42.987 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.986+0000 7fcdbbe00640 1 --2- 172.21.15.47:0/4099738087 >> [v2:172.21.15.100:6836/3497644162,v1:172.21.15.100:6837/3497644162] conn(0x555fa28d3c60 0x555fa2932580 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=553919dc5abead41 server_cookie=576365e71fe20a02 in_seq=0 out_seq=0 2024-03-21T10:33:42.987 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.986+0000 7fcdb2a00640 10 client.18234 ms_handle_connect on v2:172.21.15.100:6836/3497644162 2024-03-21T10:33:42.988 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:33:42 smithi157 ceph-mon[35581]: from='client.? 172.21.15.47:0/2181225323' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-21T10:33:42.989 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.987+0000 7fcdb2a00640 1 -- 172.21.15.47:0/4099738087 <== mds.1 v2:172.21.15.100:6836/3497644162 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x555fa2825b50 con 0x555fa28d3c60 2024-03-21T10:33:42.989 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.987+0000 7fcdb2a00640 10 client.18234 handle_command_reply: tid=0 2024-03-21T10:33:42.989 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.987+0000 7fcdce57c740 1 client.18234 shutdown 2024-03-21T10:33:42.989 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.987+0000 7fcdce57c740 1 -- 172.21.15.47:0/4099738087 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x555fa28ba6d0 msgr2=0x555fa28c9b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:33:42.989 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.987+0000 7fcdce57c740 1 --2- 172.21.15.47:0/4099738087 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x555fa28ba6d0 0x555fa28c9b20 secure :-1 s=READY pgs=1160 cs=0 l=1 rev1=1 crypto rx=0x7fcdb40093f0 tx=0x7fcdb4009420 comp rx=0 tx=0).stop 2024-03-21T10:33:42.989 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.987+0000 7fcdce57c740 1 -- 172.21.15.47:0/4099738087 shutdown_connections 2024-03-21T10:33:42.989 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.987+0000 7fcdce57c740 1 --2- 172.21.15.47:0/4099738087 >> [v2:172.21.15.100:6836/3497644162,v1:172.21.15.100:6837/3497644162] conn(0x555fa28d3c60 0x555fa2932580 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:33:42.989 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.987+0000 7fcdce57c740 1 --2- 172.21.15.47:0/4099738087 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555fa28c3c00 0x555fa292e530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:33:42.989 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.987+0000 7fcdce57c740 1 --2- 172.21.15.47:0/4099738087 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555fa28bb080 0x555fa28ca060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:33:42.989 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.987+0000 7fcdb2a00640 0 client.18234 ms_handle_reset on v2:172.21.15.100:6836/3497644162 2024-03-21T10:33:42.989 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.987+0000 7fcdce57c740 1 --2- 172.21.15.47:0/4099738087 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x555fa28ba6d0 0x555fa28c9b20 unknown :-1 s=CLOSED pgs=1160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:33:42.990 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.987+0000 7fcdce57c740 1 -- 172.21.15.47:0/4099738087 >> 172.21.15.47:0/4099738087 conn(0x555fa28b4da0 msgr2=0x555fa28c07c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:33:42.990 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.987+0000 7fcdce57c740 1 -- 172.21.15.47:0/4099738087 shutdown_connections 2024-03-21T10:33:42.990 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.987+0000 7fcdce57c740 1 -- 172.21.15.47:0/4099738087 wait complete. 2024-03-21T10:33:42.990 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.987+0000 7fcdce57c740 20 client.18234 trim_cache size 0 max 16384 2024-03-21T10:33:42.990 INFO:teuthology.orchestra.run.smithi047.stdout:[] 2024-03-21T10:33:42.991 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.988+0000 7fcdcb800640 1 -- 172.21.15.47:0/4050005162 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7fcda80a3090 msgr2=0x7fcda80a5550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:33:42.991 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.988+0000 7fcdcb800640 1 --2- 172.21.15.47:0/4050005162 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7fcda80a3090 0x7fcda80a5550 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x555fa28b5e10 tx=0x7fcdb4005f90 comp rx=0 tx=0).stop 2024-03-21T10:33:42.991 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.988+0000 7fcdcb800640 1 -- 172.21.15.47:0/4050005162 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fcdc4104200 msgr2=0x7fcdc419a600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:33:42.991 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.988+0000 7fcdcb800640 1 --2- 172.21.15.47:0/4050005162 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fcdc4104200 0x7fcdc419a600 secure :-1 s=READY pgs=1157 cs=0 l=1 rev1=1 crypto rx=0x7fcdac000c00 tx=0x7fcdac0045e0 comp rx=0 tx=0).stop 2024-03-21T10:33:42.991 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.988+0000 7fcdcb800640 1 -- 172.21.15.47:0/4050005162 shutdown_connections 2024-03-21T10:33:42.991 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.988+0000 7fcdcb800640 1 --2- 172.21.15.47:0/4050005162 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7fcda80a3090 0x7fcda80a5550 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:33:42.991 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.988+0000 7fcdcb800640 1 --2- 172.21.15.47:0/4050005162 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcdc4109850 0x7fcdc419e980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:33:42.991 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.988+0000 7fcdcb800640 1 --2- 172.21.15.47:0/4050005162 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fcdc4104200 0x7fcdc419a600 unknown :-1 s=CLOSED pgs=1157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:33:42.991 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.988+0000 7fcdcb800640 1 --2- 172.21.15.47:0/4050005162 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcdc4103840 0x7fcdc419a0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:33:42.991 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.988+0000 7fcdcb800640 1 -- 172.21.15.47:0/4050005162 >> 172.21.15.47:0/4050005162 conn(0x7fcdc40ff2f0 msgr2=0x7fcdc40ffae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:33:42.991 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.988+0000 7fcdcb800640 1 -- 172.21.15.47:0/4050005162 shutdown_connections 2024-03-21T10:33:42.992 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:42.988+0000 7fcdcb800640 1 -- 172.21.15.47:0/4050005162 wait complete. 2024-03-21T10:33:43.007 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-21T10:33:43.007 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-03-21T10:33:43.231 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:43.232+0000 7ff0f6000640 1 -- 172.21.15.47:0/2064714637 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff0f0106630 msgr2=0x7ff0f0106a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:33:43.231 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:43.232+0000 7ff0f6000640 1 --2- 172.21.15.47:0/2064714637 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff0f0106630 0x7ff0f0106a10 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7ff0e0009930 tx=0x7ff0e002bc20 comp rx=0 tx=0).stop 2024-03-21T10:33:43.231 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:43.232+0000 7ff0f6000640 1 -- 172.21.15.47:0/2064714637 shutdown_connections 2024-03-21T10:33:43.231 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:43.232+0000 7ff0f6000640 1 --2- 172.21.15.47:0/2064714637 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff0f0100f30 0x7ff0f010d3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:33:43.231 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:43.232+0000 7ff0f6000640 1 --2- 172.21.15.47:0/2064714637 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff0f0100570 0x7ff0f01009f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:33:43.231 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:43.232+0000 7ff0f6000640 1 --2- 172.21.15.47:0/2064714637 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff0f0106630 0x7ff0f0106a10 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:33:43.232 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:43.232+0000 7ff0f6000640 1 -- 172.21.15.47:0/2064714637 >> 172.21.15.47:0/2064714637 conn(0x7ff0f0076920 msgr2=0x7ff0f0076d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:33:43.232 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:43.232+0000 7ff0f6000640 1 -- 172.21.15.47:0/2064714637 shutdown_connections 2024-03-21T10:33:43.232 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:43.232+0000 7ff0f6000640 1 -- 172.21.15.47:0/2064714637 wait complete. 2024-03-21T10:33:43.232 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:43.234+0000 7ff0f6000640 1 Processor -- start 2024-03-21T10:33:43.232 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:43.234+0000 7ff0f6000640 1 -- start start 2024-03-21T10:33:43.234 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:43.234+0000 7ff0f6000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff0f0100570 0x7ff0f019a050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:33:43.234 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:43.234+0000 7ff0f6000640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff0f0100f30 0x7ff0f019a590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:33:43.234 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:43.234+0000 7ff0f6000640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff0f0106630 0x7ff0f019e910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:33:43.234 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:43.235+0000 7ff0f6000640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0f0110810 con 0x7ff0f0106630 2024-03-21T10:33:43.234 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:43.235+0000 7ff0f6000640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0f0110690 con 0x7ff0f0100570 2024-03-21T10:33:43.234 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:43.235+0000 7ff0f6000640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0f0110990 con 0x7ff0f0100f30 2024-03-21T10:33:43.234 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:43.235+0000 7ff0ef400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff0f0100570 0x7ff0f019a050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:33:43.234 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:43.235+0000 7ff0ef400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff0f0100570 0x7ff0f019a050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:37154/0 (socket says 172.21.15.47:37154) 2024-03-21T10:33:43.235 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:43.235+0000 7ff0ef400640 1 -- 172.21.15.47:0/2190380560 learned_addr learned my addr 172.21.15.47:0/2190380560 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:33:43.235 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:43.235+0000 7ff0efe00640 1 --2- 172.21.15.47:0/2190380560 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff0f0106630 0x7ff0f019e910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:33:43.235 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:43.235+0000 7ff0eea00640 1 --2- 172.21.15.47:0/2190380560 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff0f0100f30 0x7ff0f019a590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:33:43.235 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:43.235+0000 7ff0ef400640 1 -- 172.21.15.47:0/2190380560 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff0f0100f30 msgr2=0x7ff0f019a590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:33:43.235 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:43.235+0000 7ff0ef400640 1 --2- 172.21.15.47:0/2190380560 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff0f0100f30 0x7ff0f019a590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:33:43.235 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:43.235+0000 7ff0ef400640 1 -- 172.21.15.47:0/2190380560 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff0f0106630 msgr2=0x7ff0f019e910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:33:43.235 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:43.235+0000 7ff0ef400640 1 --2- 172.21.15.47:0/2190380560 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff0f0106630 0x7ff0f019e910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:33:43.235 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:43.235+0000 7ff0ef400640 1 -- 172.21.15.47:0/2190380560 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0f019f040 con 0x7ff0f0100570 2024-03-21T10:33:43.235 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:43.235+0000 7ff0eea00640 1 --2- 172.21.15.47:0/2190380560 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff0f0100f30 0x7ff0f019a590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T10:33:43.235 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:43.236+0000 7ff0ef400640 1 --2- 172.21.15.47:0/2190380560 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff0f0100570 0x7ff0f019a050 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7ff0e0000c00 tx=0x7ff0e002c650 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:33:43.236 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:43.236+0000 7ff0e7e00640 1 -- 172.21.15.47:0/2190380560 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0e0043020 con 0x7ff0f0100570 2024-03-21T10:33:43.236 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:43.236+0000 7ff0f6000640 1 -- 172.21.15.47:0/2190380560 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff0f019f2d0 con 0x7ff0f0100570 2024-03-21T10:33:43.236 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:43.237+0000 7ff0e7e00640 1 -- 172.21.15.47:0/2190380560 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7ff0e002cae0 con 0x7ff0f0100570 2024-03-21T10:33:43.236 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:43.237+0000 7ff0e7e00640 1 -- 172.21.15.47:0/2190380560 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0e0003a80 con 0x7ff0f0100570 2024-03-21T10:33:43.236 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:43.237+0000 7ff0f6000640 1 -- 172.21.15.47:0/2190380560 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff0f01a6bc0 con 0x7ff0f0100570 2024-03-21T10:33:43.238 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:43.240+0000 7ff0f8c19740 1 Processor -- start 2024-03-21T10:33:43.239 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:43.240+0000 7ff0f8c19740 1 -- start start 2024-03-21T10:33:43.239 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:43.240+0000 7ff0f8c19740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558a351e96d0 0x558a351e9ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:33:43.239 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:43.240+0000 7ff0f8c19740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x558a351ea080 0x558a351f2630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:33:43.239 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:43.240+0000 7ff0f8c19740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558a351f2c00 0x558a351f5050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:33:43.239 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:43.240+0000 7ff0f8c19740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x558a3514d2c0 con 0x558a351ea080 2024-03-21T10:33:43.239 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:43.240+0000 7ff0f8c19740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x558a350f5c90 con 0x558a351f2c00 2024-03-21T10:33:43.239 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:43.240+0000 7ff0f8c19740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x558a35149360 con 0x558a351e96d0 2024-03-21T10:33:43.240 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:43.241+0000 7ff0eea00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558a351f2c00 0x558a351f5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:33:43.240 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:43.241+0000 7ff0eea00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558a351f2c00 0x558a351f5050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:37164/0 (socket says 172.21.15.47:37164) 2024-03-21T10:33:43.240 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:43.241+0000 7ff0efe00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558a351e96d0 0x558a351e9ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:33:43.240 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:43.241+0000 7ff0eea00640 1 -- 172.21.15.47:0/1728343590 learned_addr learned my addr 172.21.15.47:0/1728343590 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:33:43.240 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:43.241+0000 7ff0eea00640 1 -- 172.21.15.47:0/1728343590 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558a351e96d0 msgr2=0x558a351e9ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:33:43.240 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:43.241+0000 7ff0eea00640 1 --2- 172.21.15.47:0/1728343590 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558a351e96d0 0x558a351e9ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:33:43.240 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:43.241+0000 7ff0eea00640 1 -- 172.21.15.47:0/1728343590 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x558a351ea080 msgr2=0x558a351f2630 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-21T10:33:43.241 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:43.241+0000 7ff0eea00640 1 --2- 172.21.15.47:0/1728343590 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x558a351ea080 0x558a351f2630 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:33:43.241 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:43.241+0000 7ff0eea00640 1 -- 172.21.15.47:0/1728343590 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558a3511a080 con 0x558a351f2c00 2024-03-21T10:33:43.241 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:43.242+0000 7ff0eea00640 1 --2- 172.21.15.47:0/1728343590 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558a351f2c00 0x558a351f5050 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7ff0dc009cf0 tx=0x7ff0dc0117b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=433f50ba569dcf57 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:33:43.243 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:43.242+0000 7ff0dbe00640 1 -- 172.21.15.47:0/1728343590 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0dc012450 con 0x558a351f2c00 2024-03-21T10:33:43.243 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:43.242+0000 7ff0dbe00640 1 -- 172.21.15.47:0/1728343590 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7ff0dc012aa0 con 0x558a351f2c00 2024-03-21T10:33:43.243 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:43.242+0000 7ff0dbe00640 1 -- 172.21.15.47:0/1728343590 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0dc00fc00 con 0x558a351f2c00 2024-03-21T10:33:43.243 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:43.242+0000 7ff0e7e00640 1 -- 172.21.15.47:0/2190380560 <== mon.2 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 27) v1 ==== 137985+0+0 (secure 0 0 0) 0x7ff0e0046020 con 0x7ff0f0100570 2024-03-21T10:33:43.243 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:43.243+0000 7ff0f8c19740 1 -- 172.21.15.47:0/1728343590 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558a351f2c00 msgr2=0x558a351f5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:33:43.243 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:43.243+0000 7ff0f8c19740 1 --2- 172.21.15.47:0/1728343590 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558a351f2c00 0x558a351f5050 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7ff0dc009cf0 tx=0x7ff0dc0117b0 comp rx=0 tx=0).stop 2024-03-21T10:33:43.243 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:43.243+0000 7ff0f8c19740 1 -- 172.21.15.47:0/1728343590 shutdown_connections 2024-03-21T10:33:43.243 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:43.243+0000 7ff0f8c19740 1 --2- 172.21.15.47:0/1728343590 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558a351f2c00 0x558a351f5050 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:33:43.244 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:43.243+0000 7ff0f8c19740 1 --2- 172.21.15.47:0/1728343590 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x558a351ea080 0x558a351f2630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:33:43.244 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:43.243+0000 7ff0f8c19740 1 --2- 172.21.15.47:0/1728343590 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558a351e96d0 0x558a351e9ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:33:43.244 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:43.243+0000 7ff0f8c19740 1 -- 172.21.15.47:0/1728343590 >> 172.21.15.47:0/1728343590 conn(0x558a351e3da0 msgr2=0x558a351e4180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:33:43.244 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:43.243+0000 7ff0f8c19740 1 -- 172.21.15.47:0/1728343590 shutdown_connections 2024-03-21T10:33:43.244 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:43.243+0000 7ff0f8c19740 1 -- 172.21.15.47:0/1728343590 wait complete. 2024-03-21T10:33:43.244 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:43.244+0000 7ff0f8c19740 1 -- start start 2024-03-21T10:33:43.244 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:43.244+0000 7ff0f8c19740 1 Processor -- start 2024-03-21T10:33:43.244 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:43.244+0000 7ff0e7e00640 1 --2- 172.21.15.47:0/2190380560 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7ff0d40a3020 0x7ff0d40a54e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:33:43.244 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:43.244+0000 7ff0e7e00640 1 -- 172.21.15.47:0/2190380560 <== mon.2 v2:172.21.15.100:3300/0 5 ==== osd_map(101..101 src has 1..101) v4 ==== 9562+0+0 (secure 0 0 0) 0x7ff0e00f1a20 con 0x7ff0f0100570 2024-03-21T10:33:43.244 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:43.245+0000 7ff0efe00640 1 --2- 172.21.15.47:0/2190380560 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7ff0d40a3020 0x7ff0d40a54e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:33:43.244 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:43.245+0000 7ff0f8c19740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558a351e96d0 0x558a351f25a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:33:43.247 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:43.245+0000 7ff0f8c19740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x558a351ea080 0x558a351ef9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:33:43.250 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:43.245+0000 7ff0f8c19740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558a351f2c00 0x558a351f00e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:33:43.250 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:43.245+0000 7ff0f8c19740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x558a351f5b70 con 0x558a351ea080 2024-03-21T10:33:43.250 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:43.245+0000 7ff0f8c19740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x558a351090d0 con 0x558a351f2c00 2024-03-21T10:33:43.251 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:43.245+0000 7ff0f8c19740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x558a351f5cf0 con 0x558a351e96d0 2024-03-21T10:33:43.251 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:43.245+0000 7ff0f8c19740 10 client.0 _ll_register_callbacks cb 0x558a3510de80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T10:33:43.251 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:43.245+0000 7ff0eea00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x558a351ea080 0x558a351ef9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:33:43.251 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:43.245+0000 7ff0eea00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x558a351ea080 0x558a351ef9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:58402/0 (socket says 172.21.15.47:58402) 2024-03-21T10:33:43.251 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:43.245+0000 7ff0eea00640 1 -- 172.21.15.47:0/4120708549 learned_addr learned my addr 172.21.15.47:0/4120708549 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:33:43.251 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:43.245+0000 7ff0efe00640 1 --2- 172.21.15.47:0/4120708549 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558a351f2c00 0x558a351f00e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:33:43.252 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:43.245+0000 7ff0ef400640 1 --2- 172.21.15.47:0/4120708549 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558a351e96d0 0x558a351f25a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:33:43.252 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:43.246+0000 7ff0efe00640 1 --2- 172.21.15.47:0/2190380560 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7ff0d40a3020 0x7ff0d40a54e0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7ff0e8005fd0 tx=0x7ff0e8009f90 comp rx=0 tx=0).ready entity=mgr.15687 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:33:43.252 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:43.246+0000 7ff0eea00640 1 -- 172.21.15.47:0/4120708549 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558a351e96d0 msgr2=0x558a351f25a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:33:43.252 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:43.246+0000 7ff0eea00640 1 --2- 172.21.15.47:0/4120708549 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558a351e96d0 0x558a351f25a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:33:43.252 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:43.246+0000 7ff0eea00640 1 -- 172.21.15.47:0/4120708549 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558a351f2c00 msgr2=0x558a351f00e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:33:43.252 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:43.246+0000 7ff0eea00640 1 --2- 172.21.15.47:0/4120708549 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558a351f2c00 0x558a351f00e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:33:43.252 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:43.246+0000 7ff0eea00640 1 -- 172.21.15.47:0/4120708549 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x558a34e613f0 con 0x558a351ea080 2024-03-21T10:33:43.252 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:43.246+0000 7ff0eea00640 1 --2- 172.21.15.47:0/4120708549 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x558a351ea080 0x558a351ef9a0 secure :-1 s=READY pgs=1161 cs=0 l=1 rev1=1 crypto rx=0x7ff0dc01c040 tx=0x7ff0dc0099c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f7589e6a58b21a23 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:33:43.252 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:43.246+0000 7ff0d8c00640 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-03-21T10:33:43.252 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:43.247+0000 7ff0d8c00640 1 -- 172.21.15.47:0/4120708549 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0dc029030 con 0x558a351ea080 2024-03-21T10:33:43.252 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:43.247+0000 7ff0d8c00640 1 -- 172.21.15.47:0/4120708549 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(101..101 src has 1..101) v4 ==== 9562+0+0 (secure 0 0 0) 0x7ff0dc020040 con 0x558a351ea080 2024-03-21T10:33:43.252 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:43.247+0000 7ff0f8c19740 1 -- 172.21.15.47:0/4120708549 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ff0dc012aa0 con 0x558a351ea080 2024-03-21T10:33:43.253 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:43.247+0000 7ff0d8c00640 1 -- 172.21.15.47:0/4120708549 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff0dc033050 con 0x558a351ea080 2024-03-21T10:33:43.253 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:43.247+0000 7ff0f8c19740 10 client.18243 fetch_fsmap learned FSMap version 32 2024-03-21T10:33:43.253 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:43.247+0000 7ff0f8c19740 1 -- 172.21.15.47:0/4120708549 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x558a34f2b7d0 con 0x558a351ea080 2024-03-21T10:33:43.253 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:43.247+0000 7ff0d8c00640 1 -- 172.21.15.47:0/4120708549 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7ff0dc012aa0 con 0x558a351ea080 2024-03-21T10:33:43.253 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:43.248+0000 7ff0d8c00640 1 -- 172.21.15.47:0/4120708549 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 32) v1 ==== 7942+0+0 (secure 0 0 0) 0x7ff0dc005040 con 0x558a351ea080 2024-03-21T10:33:43.253 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:43.248+0000 7ff0f8c19740 10 client.18243 fetch_fsmap finished waiting for FSMap version 32 2024-03-21T10:33:43.253 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:43.248+0000 7ff0f8c19740 10 client.18243 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-21T10:33:43.253 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:43.248+0000 7ff0f8c19740 20 client.18243 populate_metadata read hostname 'smithi047' 2024-03-21T10:33:43.253 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:43.248+0000 7ff0f8c19740 1 --2- 172.21.15.47:0/4120708549 >> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] conn(0x558a35202ae0 0x558a35204ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:33:43.253 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:43.248+0000 7ff0f8c19740 4 client.18243 mds_command: new command op to 34535 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T10:33:43.253 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:43.248+0000 7ff0f8c19740 1 -- 172.21.15.47:0/4120708549 --> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x558a3515a800 con 0x558a35202ae0 2024-03-21T10:33:43.253 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:43.248+0000 7ff0ef400640 1 --2- 172.21.15.47:0/4120708549 >> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] conn(0x558a35202ae0 0x558a35204ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:33:43.253 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:43.249+0000 7ff0ef400640 1 --2- 172.21.15.47:0/4120708549 >> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] conn(0x558a35202ae0 0x558a35204ed0 crc :-1 s=READY pgs=193 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9b97a27311f87b08 server_cookie=9f2920348d60dd20 in_seq=0 out_seq=0 2024-03-21T10:33:43.253 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:43.249+0000 7ff0d8c00640 10 client.18243 ms_handle_connect on v2:172.21.15.100:6838/4173168440 2024-03-21T10:33:43.256 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:43.250+0000 7ff0d8c00640 1 -- 172.21.15.47:0/4120708549 <== mds.0 v2:172.21.15.100:6838/4173168440 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x558a3515a800 con 0x558a35202ae0 2024-03-21T10:33:43.256 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:43.250+0000 7ff0d8c00640 10 client.18243 handle_command_reply: tid=0 2024-03-21T10:33:43.256 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:43.250+0000 7ff0f8c19740 1 client.18243 shutdown 2024-03-21T10:33:43.256 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:43.250+0000 7ff0f8c19740 1 -- 172.21.15.47:0/4120708549 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x558a351ea080 msgr2=0x558a351ef9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:33:43.256 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:43.250+0000 7ff0f8c19740 1 --2- 172.21.15.47:0/4120708549 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x558a351ea080 0x558a351ef9a0 secure :-1 s=READY pgs=1161 cs=0 l=1 rev1=1 crypto rx=0x7ff0dc01c040 tx=0x7ff0dc0099c0 comp rx=0 tx=0).stop 2024-03-21T10:33:43.256 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:43.250+0000 7ff0f8c19740 1 -- 172.21.15.47:0/4120708549 shutdown_connections 2024-03-21T10:33:43.256 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:43.250+0000 7ff0f8c19740 1 --2- 172.21.15.47:0/4120708549 >> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] conn(0x558a35202ae0 0x558a35204ed0 crc :-1 s=READY pgs=193 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:33:43.256 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:43.250+0000 7ff0f8c19740 1 --2- 172.21.15.47:0/4120708549 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558a351f2c00 0x558a351f00e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:33:43.256 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:43.250+0000 7ff0f8c19740 1 --2- 172.21.15.47:0/4120708549 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x558a351ea080 0x558a351ef9a0 unknown :-1 s=CLOSED pgs=1161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:33:43.257 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:43.250+0000 7ff0f8c19740 1 --2- 172.21.15.47:0/4120708549 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558a351e96d0 0x558a351f25a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:33:43.257 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:43.250+0000 7ff0f8c19740 1 -- 172.21.15.47:0/4120708549 >> 172.21.15.47:0/4120708549 conn(0x558a351e3da0 msgr2=0x558a351e5af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:33:43.257 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:43.250+0000 7ff0d8c00640 0 client.18243 ms_handle_reset on v2:172.21.15.100:6838/4173168440 2024-03-21T10:33:43.257 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:43.250+0000 7ff0f8c19740 1 -- 172.21.15.47:0/4120708549 shutdown_connections 2024-03-21T10:33:43.257 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:43.250+0000 7ff0f8c19740 1 -- 172.21.15.47:0/4120708549 wait complete. 2024-03-21T10:33:43.257 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:43.251+0000 7ff0f8c19740 20 client.18243 trim_cache size 0 max 16384 2024-03-21T10:33:43.266 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:43.267+0000 7ff0f8c19740 1 Processor -- start 2024-03-21T10:33:43.266 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:43.267+0000 7ff0f8c19740 1 -- start start 2024-03-21T10:33:43.266 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:43.267+0000 7ff0f8c19740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x558a351e96d0 0x558a351f20d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:33:43.266 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:43.267+0000 7ff0f8c19740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558a351ea080 0x558a351f2610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:33:43.266 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:43.267+0000 7ff0f8c19740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558a351f2c00 0x558a35209cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:33:43.267 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:43.267+0000 7ff0f8c19740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0f01b0be0 con 0x558a351e96d0 2024-03-21T10:33:43.267 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:43.267+0000 7ff0f8c19740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0f01b2560 con 0x558a351f2c00 2024-03-21T10:33:43.267 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:43.267+0000 7ff0f8c19740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0f01b1240 con 0x558a351ea080 2024-03-21T10:33:43.267 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:43.267+0000 7ff0ef400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x558a351e96d0 0x558a351f20d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:33:43.267 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:43.267+0000 7ff0ef400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x558a351e96d0 0x558a351f20d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:58404/0 (socket says 172.21.15.47:58404) 2024-03-21T10:33:43.267 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:43.267+0000 7ff0ef400640 1 -- 172.21.15.47:0/3231733191 learned_addr learned my addr 172.21.15.47:0/3231733191 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:33:43.267 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:43.267+0000 7ff0efe00640 1 --2- 172.21.15.47:0/3231733191 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558a351f2c00 0x558a35209cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:33:43.267 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:43.268+0000 7ff0eea00640 1 --2- 172.21.15.47:0/3231733191 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558a351ea080 0x558a351f2610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:33:43.267 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:43.268+0000 7ff0ef400640 1 -- 172.21.15.47:0/3231733191 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558a351ea080 msgr2=0x558a351f2610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:33:43.267 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:43.268+0000 7ff0ef400640 1 --2- 172.21.15.47:0/3231733191 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558a351ea080 0x558a351f2610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:33:43.267 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:43.268+0000 7ff0ef400640 1 -- 172.21.15.47:0/3231733191 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558a351f2c00 msgr2=0x558a35209cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:33:43.267 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:43.268+0000 7ff0ef400640 1 --2- 172.21.15.47:0/3231733191 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558a351f2c00 0x558a35209cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:33:43.267 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:43.268+0000 7ff0ef400640 1 -- 172.21.15.47:0/3231733191 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558a351e4970 con 0x558a351e96d0 2024-03-21T10:33:43.269 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:43.268+0000 7ff0ef400640 1 --2- 172.21.15.47:0/3231733191 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x558a351e96d0 0x558a351f20d0 secure :-1 s=READY pgs=1162 cs=0 l=1 rev1=1 crypto rx=0x7ff0e00ef360 tx=0x7ff0e004a040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e3abc38834b8d8fb server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:33:43.269 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:43.269+0000 7ff0db400640 1 -- 172.21.15.47:0/3231733191 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0e00d5050 con 0x558a351e96d0 2024-03-21T10:33:43.269 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:43.269+0000 7ff0db400640 1 -- 172.21.15.47:0/3231733191 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7ff0e00db070 con 0x558a351e96d0 2024-03-21T10:33:43.269 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:43.269+0000 7ff0db400640 1 -- 172.21.15.47:0/3231733191 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0e00ba180 con 0x558a351e96d0 2024-03-21T10:33:43.269 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:43.270+0000 7ff0f8c19740 1 -- 172.21.15.47:0/3231733191 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x558a351e96d0 msgr2=0x558a351f20d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:33:43.269 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:43.270+0000 7ff0f8c19740 1 --2- 172.21.15.47:0/3231733191 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x558a351e96d0 0x558a351f20d0 secure :-1 s=READY pgs=1162 cs=0 l=1 rev1=1 crypto rx=0x7ff0e00ef360 tx=0x7ff0e004a040 comp rx=0 tx=0).stop 2024-03-21T10:33:43.269 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:43.270+0000 7ff0f8c19740 1 -- 172.21.15.47:0/3231733191 shutdown_connections 2024-03-21T10:33:43.269 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:43.270+0000 7ff0f8c19740 1 --2- 172.21.15.47:0/3231733191 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558a351f2c00 0x558a35209cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:33:43.269 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:43.270+0000 7ff0f8c19740 1 --2- 172.21.15.47:0/3231733191 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558a351ea080 0x558a351f2610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:33:43.269 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:43.270+0000 7ff0f8c19740 1 --2- 172.21.15.47:0/3231733191 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x558a351e96d0 0x558a351f20d0 unknown :-1 s=CLOSED pgs=1162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:33:43.269 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:43.270+0000 7ff0f8c19740 1 -- 172.21.15.47:0/3231733191 >> 172.21.15.47:0/3231733191 conn(0x558a351e3da0 msgr2=0x558a351eaf30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:33:43.269 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:43.270+0000 7ff0f8c19740 1 -- 172.21.15.47:0/3231733191 shutdown_connections 2024-03-21T10:33:43.270 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:43.270+0000 7ff0f8c19740 1 -- 172.21.15.47:0/3231733191 wait complete. 2024-03-21T10:33:43.270 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:43.270+0000 7ff0f8c19740 1 -- start start 2024-03-21T10:33:43.270 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:43.271+0000 7ff0f8c19740 1 Processor -- start 2024-03-21T10:33:43.272 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:43.271+0000 7ff0f8c19740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x558a351e96d0 0x558a351f8680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:33:43.273 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:43.271+0000 7ff0f8c19740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558a351ea080 0x558a351f8bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:33:43.273 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:43.271+0000 7ff0f8c19740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558a351f2c00 0x558a351f9140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:33:43.273 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:43.271+0000 7ff0f8c19740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0f01b1f00 con 0x558a351e96d0 2024-03-21T10:33:43.273 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:43.271+0000 7ff0f8c19740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0f01b18a0 con 0x558a351ea080 2024-03-21T10:33:43.273 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:43.271+0000 7ff0f8c19740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0f01aedc0 con 0x558a351f2c00 2024-03-21T10:33:43.273 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:43.271+0000 7ff0f8c19740 10 client.0 _ll_register_callbacks cb 0x7ff0e8009b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T10:33:43.273 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:43.271+0000 7ff0ef400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x558a351e96d0 0x558a351f8680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:33:43.273 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:43.272+0000 7ff0ef400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x558a351e96d0 0x558a351f8680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:58410/0 (socket says 172.21.15.47:58410) 2024-03-21T10:33:43.273 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:43.272+0000 7ff0ef400640 1 -- 172.21.15.47:0/3650417321 learned_addr learned my addr 172.21.15.47:0/3650417321 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:33:43.273 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:43.272+0000 7ff0eea00640 1 --2- 172.21.15.47:0/3650417321 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558a351ea080 0x558a351f8bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:33:43.273 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:43.272+0000 7ff0efe00640 1 --2- 172.21.15.47:0/3650417321 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558a351f2c00 0x558a351f9140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:33:43.274 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:43.272+0000 7ff0ef400640 1 -- 172.21.15.47:0/3650417321 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558a351f2c00 msgr2=0x558a351f9140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:33:43.274 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:43.272+0000 7ff0ef400640 1 --2- 172.21.15.47:0/3650417321 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558a351f2c00 0x558a351f9140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:33:43.274 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:43.272+0000 7ff0ef400640 1 -- 172.21.15.47:0/3650417321 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558a351ea080 msgr2=0x558a351f8bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:33:43.274 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:43.272+0000 7ff0ef400640 1 --2- 172.21.15.47:0/3650417321 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558a351ea080 0x558a351f8bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:33:43.274 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:43.272+0000 7ff0ef400640 1 -- 172.21.15.47:0/3650417321 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x558a3525d330 con 0x558a351e96d0 2024-03-21T10:33:43.274 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:43.272+0000 7ff0eea00640 1 --2- 172.21.15.47:0/3650417321 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558a351ea080 0x558a351f8bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T10:33:43.274 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:43.272+0000 7ff0ef400640 1 --2- 172.21.15.47:0/3650417321 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x558a351e96d0 0x558a351f8680 secure :-1 s=READY pgs=1163 cs=0 l=1 rev1=1 crypto rx=0x7ff0e00bd720 tx=0x7ff0e00092e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a3e2097f8a35b2a6 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:33:43.274 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:43.272+0000 7ff0d8c00640 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-03-21T10:33:43.274 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:43.273+0000 7ff0d8c00640 1 -- 172.21.15.47:0/3650417321 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0e00a1070 con 0x558a351e96d0 2024-03-21T10:33:43.274 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:43.273+0000 7ff0d8c00640 1 -- 172.21.15.47:0/3650417321 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(101..101 src has 1..101) v4 ==== 9562+0+0 (secure 0 0 0) 0x7ff0e00ba180 con 0x558a351e96d0 2024-03-21T10:33:43.274 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:43.273+0000 7ff0f8c19740 1 -- 172.21.15.47:0/3650417321 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ff0e00db070 con 0x558a351e96d0 2024-03-21T10:33:43.274 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:43.273+0000 7ff0d8c00640 1 -- 172.21.15.47:0/3650417321 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff0e00c6520 con 0x558a351e96d0 2024-03-21T10:33:43.274 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:43.273+0000 7ff0f8c19740 10 client.18249 fetch_fsmap learned FSMap version 32 2024-03-21T10:33:43.275 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:43.273+0000 7ff0f8c19740 1 -- 172.21.15.47:0/3650417321 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x558a3525e630 con 0x558a351e96d0 2024-03-21T10:33:43.275 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:43.274+0000 7ff0d8c00640 1 -- 172.21.15.47:0/3650417321 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7ff0e00db070 con 0x558a351e96d0 2024-03-21T10:33:43.275 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:43.274+0000 7ff0d8c00640 1 -- 172.21.15.47:0/3650417321 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 32) v1 ==== 7942+0+0 (secure 0 0 0) 0x7ff0e00c8ce0 con 0x558a351e96d0 2024-03-21T10:33:43.275 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:43.274+0000 7ff0f8c19740 10 client.18249 fetch_fsmap finished waiting for FSMap version 32 2024-03-21T10:33:43.275 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:43.274+0000 7ff0f8c19740 10 client.18249 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-21T10:33:43.275 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:43.274+0000 7ff0f8c19740 20 client.18249 populate_metadata read hostname 'smithi047' 2024-03-21T10:33:43.275 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:43.274+0000 7ff0f8c19740 1 --2- 172.21.15.47:0/3650417321 >> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] conn(0x558a3525eae0 0x558a35260f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:33:43.275 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:43.274+0000 7ff0f8c19740 4 client.18249 mds_command: new command op to 34535 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-21T10:33:43.275 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:43.274+0000 7ff0f8c19740 1 -- 172.21.15.47:0/3650417321 --> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x558a35154b50 con 0x558a3525eae0 2024-03-21T10:33:43.275 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:43.274+0000 7ff0eea00640 1 --2- 172.21.15.47:0/3650417321 >> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] conn(0x558a3525eae0 0x558a35260f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:33:43.277 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:43.275+0000 7ff0eea00640 1 --2- 172.21.15.47:0/3650417321 >> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] conn(0x558a3525eae0 0x558a35260f00 crc :-1 s=READY pgs=194 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2d9d7f5447af7c5e server_cookie=fc2901c15fc45b39 in_seq=0 out_seq=0 2024-03-21T10:33:43.277 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:43.275+0000 7ff0d8c00640 10 client.18249 ms_handle_connect on v2:172.21.15.100:6838/4173168440 2024-03-21T10:33:43.277 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:43.275+0000 7ff0d8c00640 1 -- 172.21.15.47:0/3650417321 <== mds.0 v2:172.21.15.100:6838/4173168440 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x558a35154b50 con 0x558a3525eae0 2024-03-21T10:33:43.277 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:43.275+0000 7ff0d8c00640 10 client.18249 handle_command_reply: tid=0 2024-03-21T10:33:43.277 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:43.275+0000 7ff0f8c19740 1 client.18249 shutdown 2024-03-21T10:33:43.277 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:43.275+0000 7ff0f8c19740 1 -- 172.21.15.47:0/3650417321 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x558a351e96d0 msgr2=0x558a351f8680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:33:43.277 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:43.275+0000 7ff0f8c19740 1 --2- 172.21.15.47:0/3650417321 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x558a351e96d0 0x558a351f8680 secure :-1 s=READY pgs=1163 cs=0 l=1 rev1=1 crypto rx=0x7ff0e00bd720 tx=0x7ff0e00092e0 comp rx=0 tx=0).stop 2024-03-21T10:33:43.277 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:43.276+0000 7ff0f8c19740 1 -- 172.21.15.47:0/3650417321 shutdown_connections 2024-03-21T10:33:43.277 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:43.276+0000 7ff0f8c19740 1 --2- 172.21.15.47:0/3650417321 >> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] conn(0x558a3525eae0 0x558a35260f00 crc :-1 s=READY pgs=194 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:33:43.277 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:43.276+0000 7ff0f8c19740 1 --2- 172.21.15.47:0/3650417321 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558a351f2c00 0x558a351f9140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:33:43.277 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:43.276+0000 7ff0f8c19740 1 --2- 172.21.15.47:0/3650417321 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558a351ea080 0x558a351f8bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:33:43.278 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:43.276+0000 7ff0f8c19740 1 --2- 172.21.15.47:0/3650417321 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x558a351e96d0 0x558a351f8680 unknown :-1 s=CLOSED pgs=1163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:33:43.278 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:43.276+0000 7ff0f8c19740 1 -- 172.21.15.47:0/3650417321 >> 172.21.15.47:0/3650417321 conn(0x558a351e3da0 msgr2=0x558a35209cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:33:43.278 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:43.276+0000 7ff0d8c00640 0 client.18249 ms_handle_reset on v2:172.21.15.100:6838/4173168440 2024-03-21T10:33:43.278 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:43.276+0000 7ff0f8c19740 1 -- 172.21.15.47:0/3650417321 shutdown_connections 2024-03-21T10:33:43.278 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:43.276+0000 7ff0f8c19740 1 -- 172.21.15.47:0/3650417321 wait complete. 2024-03-21T10:33:43.278 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:43.276+0000 7ff0f8c19740 20 client.18249 trim_cache size 0 max 16384 2024-03-21T10:33:43.278 INFO:teuthology.orchestra.run.smithi047.stdout:[] 2024-03-21T10:33:43.279 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:43.277+0000 7ff0f6000640 1 -- 172.21.15.47:0/2190380560 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7ff0d40a3020 msgr2=0x7ff0d40a54e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:33:43.279 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:43.277+0000 7ff0f6000640 1 --2- 172.21.15.47:0/2190380560 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7ff0d40a3020 0x7ff0d40a54e0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7ff0e8005fd0 tx=0x7ff0e8009f90 comp rx=0 tx=0).stop 2024-03-21T10:33:43.279 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:43.277+0000 7ff0f6000640 1 -- 172.21.15.47:0/2190380560 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff0f0100570 msgr2=0x7ff0f019a050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:33:43.279 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:43.277+0000 7ff0f6000640 1 --2- 172.21.15.47:0/2190380560 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff0f0100570 0x7ff0f019a050 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7ff0e0000c00 tx=0x7ff0e002c650 comp rx=0 tx=0).stop 2024-03-21T10:33:43.279 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:43.277+0000 7ff0f6000640 1 -- 172.21.15.47:0/2190380560 shutdown_connections 2024-03-21T10:33:43.279 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:43.277+0000 7ff0f6000640 1 --2- 172.21.15.47:0/2190380560 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff0f0106630 0x7ff0f019e910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:33:43.280 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:43.277+0000 7ff0f6000640 1 --2- 172.21.15.47:0/2190380560 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff0f0100f30 0x7ff0f019a590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:33:43.280 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:43.277+0000 7ff0f6000640 1 --2- 172.21.15.47:0/2190380560 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7ff0d40a3020 0x7ff0d40a54e0 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:33:43.280 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:43.277+0000 7ff0f6000640 1 --2- 172.21.15.47:0/2190380560 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff0f0100570 0x7ff0f019a050 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:33:43.280 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:43.277+0000 7ff0f6000640 1 -- 172.21.15.47:0/2190380560 >> 172.21.15.47:0/2190380560 conn(0x7ff0f0076920 msgr2=0x7ff0f0069cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:33:43.280 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:43.277+0000 7ff0f6000640 1 -- 172.21.15.47:0/2190380560 shutdown_connections 2024-03-21T10:33:43.280 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:43.277+0000 7ff0f6000640 1 -- 172.21.15.47:0/2190380560 wait complete. 2024-03-21T10:33:43.713 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:33:43 smithi047 ceph-mon[31547]: pgmap v259: 129 pgs: 129 active+clean; 260 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-03-21T10:33:43.714 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:33:43 smithi047 ceph-mon[31547]: from='mgr.15687 172.21.15.47:0/8730661' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T10:33:43.977 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:33:43 smithi100 ceph-mon[35193]: pgmap v259: 129 pgs: 129 active+clean; 260 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-03-21T10:33:43.977 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:33:43 smithi100 ceph-mon[35193]: from='mgr.15687 172.21.15.47:0/8730661' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T10:33:43.986 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:33:43 smithi157 ceph-mon[35581]: pgmap v259: 129 pgs: 129 active+clean; 260 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-03-21T10:33:43.986 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:33:43 smithi157 ceph-mon[35581]: from='mgr.15687 172.21.15.47:0/8730661' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T10:33:44.294 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-03-21T10:33:44.294 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-21T10:33:44.294 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-03-21T10:33:44.502 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:44.503+0000 7f741f600640 1 -- 172.21.15.47:0/1985360825 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7418107630 msgr2=0x7f7418107a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:33:44.502 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:44.503+0000 7f741f600640 1 --2- 172.21.15.47:0/1985360825 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7418107630 0x7f7418107a10 secure :-1 s=READY pgs=1164 cs=0 l=1 rev1=1 crypto rx=0x7f7408009930 tx=0x7f740802bc30 comp rx=0 tx=0).stop 2024-03-21T10:33:44.502 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:44.503+0000 7f741f600640 1 -- 172.21.15.47:0/1985360825 shutdown_connections 2024-03-21T10:33:44.502 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:44.503+0000 7f741f600640 1 --2- 172.21.15.47:0/1985360825 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7418101fe0 0x7f741810e4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:33:44.502 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:44.503+0000 7f741f600640 1 --2- 172.21.15.47:0/1985360825 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7418101620 0x7f7418101aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:33:44.502 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:44.503+0000 7f741f600640 1 --2- 172.21.15.47:0/1985360825 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7418107630 0x7f7418107a10 unknown :-1 s=CLOSED pgs=1164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:33:44.502 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:44.503+0000 7f741f600640 1 -- 172.21.15.47:0/1985360825 >> 172.21.15.47:0/1985360825 conn(0x7f74180fd130 msgr2=0x7f74180ff550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:33:44.502 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:44.503+0000 7f741f600640 1 -- 172.21.15.47:0/1985360825 shutdown_connections 2024-03-21T10:33:44.502 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:44.503+0000 7f741f600640 1 -- 172.21.15.47:0/1985360825 wait complete. 2024-03-21T10:33:44.504 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:44.504+0000 7f741f600640 1 Processor -- start 2024-03-21T10:33:44.504 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:44.505+0000 7f741f600640 1 -- start start 2024-03-21T10:33:44.504 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:44.505+0000 7f741f600640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7418101620 0x7f741819a0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:33:44.504 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:44.505+0000 7f741f600640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7418101fe0 0x7f741819a5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:33:44.504 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:44.505+0000 7f741f600640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7418107630 0x7f741819e960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:33:44.505 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:44.505+0000 7f741f600640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7418111940 con 0x7f7418101620 2024-03-21T10:33:44.505 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:44.505+0000 7f741f600640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74181117c0 con 0x7f7418107630 2024-03-21T10:33:44.505 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:44.505+0000 7f741f600640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7418111ac0 con 0x7f7418101fe0 2024-03-21T10:33:44.505 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:44.505+0000 7f741cc00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7418101620 0x7f741819a0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:33:44.505 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:44.505+0000 7f741cc00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7418101620 0x7f741819a0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:58426/0 (socket says 172.21.15.47:58426) 2024-03-21T10:33:44.505 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:44.505+0000 7f741cc00640 1 -- 172.21.15.47:0/674073750 learned_addr learned my addr 172.21.15.47:0/674073750 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:33:44.505 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:44.505+0000 7f741d600640 1 --2- 172.21.15.47:0/674073750 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7418107630 0x7f741819e960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:33:44.505 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:44.506+0000 7f740fe00640 1 --2- 172.21.15.47:0/674073750 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7418101fe0 0x7f741819a5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:33:44.505 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:44.506+0000 7f741cc00640 1 -- 172.21.15.47:0/674073750 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7418101fe0 msgr2=0x7f741819a5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:33:44.505 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:44.506+0000 7f741cc00640 1 --2- 172.21.15.47:0/674073750 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7418101fe0 0x7f741819a5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:33:44.505 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:44.506+0000 7f741cc00640 1 -- 172.21.15.47:0/674073750 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7418107630 msgr2=0x7f741819e960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:33:44.505 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:44.506+0000 7f741cc00640 1 --2- 172.21.15.47:0/674073750 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7418107630 0x7f741819e960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:33:44.505 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:44.506+0000 7f741cc00640 1 -- 172.21.15.47:0/674073750 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f741819f090 con 0x7f7418101620 2024-03-21T10:33:44.505 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:44.506+0000 7f741cc00640 1 --2- 172.21.15.47:0/674073750 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7418101620 0x7f741819a0a0 secure :-1 s=READY pgs=1165 cs=0 l=1 rev1=1 crypto rx=0x7f7408000c00 tx=0x7f74080045e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:33:44.506 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:44.507+0000 7f7416000640 1 -- 172.21.15.47:0/674073750 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7408043020 con 0x7f7418101620 2024-03-21T10:33:44.506 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:44.507+0000 7f7416000640 1 -- 172.21.15.47:0/674073750 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f740802cb70 con 0x7f7418101620 2024-03-21T10:33:44.506 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:44.507+0000 7f7416000640 1 -- 172.21.15.47:0/674073750 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7408003b50 con 0x7f7418101620 2024-03-21T10:33:44.507 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:44.507+0000 7f741f600640 1 -- 172.21.15.47:0/674073750 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f741819f320 con 0x7f7418101620 2024-03-21T10:33:44.507 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:44.507+0000 7f741f600640 1 -- 172.21.15.47:0/674073750 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f74181a6c10 con 0x7f7418101620 2024-03-21T10:33:44.509 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:44.510+0000 7f74223db740 1 Processor -- start 2024-03-21T10:33:44.509 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:44.510+0000 7f7416000640 1 -- 172.21.15.47:0/674073750 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 27) v1 ==== 137985+0+0 (secure 0 0 0) 0x7f7408046020 con 0x7f7418101620 2024-03-21T10:33:44.509 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:44.511+0000 7f74223db740 1 -- start start 2024-03-21T10:33:44.510 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:44.511+0000 7f74223db740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5592bdeb7080 0x5592bdeb7460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:33:44.510 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:44.511+0000 7f74223db740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5592bdeb79a0 0x5592bdec0040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:33:44.510 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:44.511+0000 7f74223db740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5592bdec0580 0x5592bdec2970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:33:44.510 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:44.511+0000 7f74223db740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x5592bde1beb0 con 0x5592bdeb79a0 2024-03-21T10:33:44.510 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:44.511+0000 7f74223db740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x5592bddc4a60 con 0x5592bdeb7080 2024-03-21T10:33:44.510 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:44.511+0000 7f74223db740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x5592bde17f50 con 0x5592bdec0580 2024-03-21T10:33:44.510 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:44.512+0000 7f741d600640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5592bdeb7080 0x5592bdeb7460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:33:44.510 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:44.512+0000 7f741d600640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5592bdeb7080 0x5592bdeb7460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:37206/0 (socket says 172.21.15.47:37206) 2024-03-21T10:33:44.510 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:44.512+0000 7f741cc00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5592bdeb79a0 0x5592bdec0040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:33:44.511 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:44.512+0000 7f741d600640 1 -- 172.21.15.47:0/1120495495 learned_addr learned my addr 172.21.15.47:0/1120495495 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:33:44.511 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:44.512+0000 7f740fe00640 1 --2- 172.21.15.47:0/1120495495 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5592bdec0580 0x5592bdec2970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:33:44.512 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:44.512+0000 7f741d600640 1 -- 172.21.15.47:0/1120495495 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5592bdec0580 msgr2=0x5592bdec2970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:33:44.512 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:44.512+0000 7f741d600640 1 --2- 172.21.15.47:0/1120495495 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5592bdec0580 0x5592bdec2970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:33:44.512 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:44.512+0000 7f741d600640 1 -- 172.21.15.47:0/1120495495 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5592bdeb79a0 msgr2=0x5592bdec0040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:33:44.512 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:44.512+0000 7f741d600640 1 --2- 172.21.15.47:0/1120495495 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5592bdeb79a0 0x5592bdec0040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:33:44.512 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:44.512+0000 7f741d600640 1 -- 172.21.15.47:0/1120495495 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5592bdbf87e0 con 0x5592bdeb7080 2024-03-21T10:33:44.512 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:44.512+0000 7f7416000640 1 --2- 172.21.15.47:0/674073750 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f73fc0a3190 0x7f73fc0a5650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:33:44.512 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:44.513+0000 7f741d600640 1 --2- 172.21.15.47:0/1120495495 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5592bdeb7080 0x5592bdeb7460 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f7410009f50 tx=0x7f74100118d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c1acda30efa13aab server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:33:44.512 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:44.513+0000 7f7416000640 1 -- 172.21.15.47:0/674073750 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(101..101 src has 1..101) v4 ==== 9562+0+0 (secure 0 0 0) 0x7f74080f1c40 con 0x7f7418101620 2024-03-21T10:33:44.512 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:44.513+0000 7f741cc00640 1 --2- 172.21.15.47:0/674073750 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f73fc0a3190 0x7f73fc0a5650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:33:44.512 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:44.513+0000 7f740dc00640 1 -- 172.21.15.47:0/1120495495 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7410004d20 con 0x5592bdeb7080 2024-03-21T10:33:44.512 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:44.513+0000 7f740dc00640 1 -- 172.21.15.47:0/1120495495 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f7410004050 con 0x5592bdeb7080 2024-03-21T10:33:44.512 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:44.513+0000 7f740dc00640 1 -- 172.21.15.47:0/1120495495 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7410005280 con 0x5592bdeb7080 2024-03-21T10:33:44.515 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:44.514+0000 7f74223db740 1 -- 172.21.15.47:0/1120495495 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5592bdeb7080 msgr2=0x5592bdeb7460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:33:44.515 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:44.514+0000 7f74223db740 1 --2- 172.21.15.47:0/1120495495 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5592bdeb7080 0x5592bdeb7460 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f7410009f50 tx=0x7f74100118d0 comp rx=0 tx=0).stop 2024-03-21T10:33:44.515 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:44.514+0000 7f741cc00640 1 --2- 172.21.15.47:0/674073750 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f73fc0a3190 0x7f73fc0a5650 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x5592bdeb6b50 tx=0x7f74080f5670 comp rx=0 tx=0).ready entity=mgr.15687 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:33:44.515 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:44.514+0000 7f74223db740 1 -- 172.21.15.47:0/1120495495 shutdown_connections 2024-03-21T10:33:44.515 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:44.514+0000 7f74223db740 1 --2- 172.21.15.47:0/1120495495 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5592bdec0580 0x5592bdec2970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:33:44.516 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:44.514+0000 7f74223db740 1 --2- 172.21.15.47:0/1120495495 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5592bdeb79a0 0x5592bdec0040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:33:44.516 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:44.514+0000 7f74223db740 1 --2- 172.21.15.47:0/1120495495 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5592bdeb7080 0x5592bdeb7460 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:33:44.516 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:44.514+0000 7f74223db740 1 -- 172.21.15.47:0/1120495495 >> 172.21.15.47:0/1120495495 conn(0x5592bdeb17b0 msgr2=0x5592bdeb1b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:33:44.516 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:44.514+0000 7f74223db740 1 -- 172.21.15.47:0/1120495495 shutdown_connections 2024-03-21T10:33:44.516 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:44.514+0000 7f74223db740 1 -- 172.21.15.47:0/1120495495 wait complete. 2024-03-21T10:33:44.516 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:44.515+0000 7f74223db740 1 -- start start 2024-03-21T10:33:44.516 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:44.515+0000 7f74223db740 1 Processor -- start 2024-03-21T10:33:44.516 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:44.516+0000 7f74223db740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5592bdeb7080 0x5592bdebfe90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:33:44.516 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:44.516+0000 7f74223db740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5592bdeb79a0 0x5592bdebd220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:33:44.516 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:44.516+0000 7f74223db740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5592bdec0580 0x5592bdebd760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:33:44.516 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:44.516+0000 7f74223db740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x5592bdec3370 con 0x5592bdec0580 2024-03-21T10:33:44.516 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:44.516+0000 7f74223db740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x5592bddd7eb0 con 0x5592bdeb79a0 2024-03-21T10:33:44.516 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:44.516+0000 7f74223db740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x5592bdec34f0 con 0x5592bdeb7080 2024-03-21T10:33:44.516 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:44.516+0000 7f74223db740 10 client.0 _ll_register_callbacks cb 0x5592bddcc150 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T10:33:44.517 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:44.516+0000 7f740fe00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5592bdeb79a0 0x5592bdebd220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:33:44.517 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:44.516+0000 7f741d600640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5592bdec0580 0x5592bdebd760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:33:44.517 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:44.516+0000 7f740fe00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5592bdeb79a0 0x5592bdebd220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:37214/0 (socket says 172.21.15.47:37214) 2024-03-21T10:33:44.517 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:44.516+0000 7f740fe00640 1 -- 172.21.15.47:0/2161115013 learned_addr learned my addr 172.21.15.47:0/2161115013 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:33:44.517 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:44.516+0000 7f741d600640 1 --2- 172.21.15.47:0/2161115013 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5592bdeb7080 0x5592bdebfe90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:33:44.517 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:44.517+0000 7f740fe00640 1 -- 172.21.15.47:0/2161115013 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5592bdeb7080 msgr2=0x5592bdebfe90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:33:44.517 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:44.517+0000 7f740fe00640 1 --2- 172.21.15.47:0/2161115013 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5592bdeb7080 0x5592bdebfe90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:33:44.517 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:44.517+0000 7f740fe00640 1 -- 172.21.15.47:0/2161115013 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5592bdec0580 msgr2=0x5592bdebd760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:33:44.517 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:44.517+0000 7f740fe00640 1 --2- 172.21.15.47:0/2161115013 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5592bdec0580 0x5592bdebd760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:33:44.517 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:44.517+0000 7f740fe00640 1 -- 172.21.15.47:0/2161115013 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5592bdc58f70 con 0x5592bdeb79a0 2024-03-21T10:33:44.517 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:44.517+0000 7f741d600640 1 --2- 172.21.15.47:0/2161115013 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5592bdec0580 0x5592bdebd760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T10:33:44.517 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:44.517+0000 7f740fe00640 1 --2- 172.21.15.47:0/2161115013 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5592bdeb79a0 0x5592bdebd220 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f740000be10 tx=0x7f74000117b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c0ddb97ab604a33b server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:33:44.517 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:44.517+0000 7f7406000640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-21T10:33:44.519 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:44.517+0000 7f7406000640 1 -- 172.21.15.47:0/2161115013 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7400012650 con 0x5592bdeb79a0 2024-03-21T10:33:44.519 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:44.517+0000 7f74223db740 1 -- 172.21.15.47:0/2161115013 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f7410004050 con 0x5592bdeb79a0 2024-03-21T10:33:44.519 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:44.518+0000 7f7406000640 1 -- 172.21.15.47:0/2161115013 <== mon.2 v2:172.21.15.100:3300/0 2 ==== osd_map(101..101 src has 1..101) v4 ==== 9562+0+0 (secure 0 0 0) 0x7f74000127f0 con 0x5592bdeb79a0 2024-03-21T10:33:44.519 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:44.518+0000 7f7406000640 1 -- 172.21.15.47:0/2161115013 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f74000129f0 con 0x5592bdeb79a0 2024-03-21T10:33:44.519 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:44.518+0000 7f74223db740 10 client.38027 fetch_fsmap learned FSMap version 32 2024-03-21T10:33:44.519 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:44.518+0000 7f74223db740 1 -- 172.21.15.47:0/2161115013 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x5592bdaf0170 con 0x5592bdeb79a0 2024-03-21T10:33:44.519 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:44.519+0000 7f7406000640 1 -- 172.21.15.47:0/2161115013 <== mon.2 v2:172.21.15.100:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f7410004050 con 0x5592bdeb79a0 2024-03-21T10:33:44.520 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:44.519+0000 7f7406000640 1 -- 172.21.15.47:0/2161115013 <== mon.2 v2:172.21.15.100:3300/0 5 ==== fsmap(e 32) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f740001a3d0 con 0x5592bdeb79a0 2024-03-21T10:33:44.520 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:44.519+0000 7f74223db740 10 client.38027 fetch_fsmap finished waiting for FSMap version 32 2024-03-21T10:33:44.520 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:44.519+0000 7f74223db740 10 client.38027 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-21T10:33:44.520 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:44.519+0000 7f74223db740 20 client.38027 populate_metadata read hostname 'smithi047' 2024-03-21T10:33:44.520 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:44.519+0000 7f74223db740 1 --2- 172.21.15.47:0/2161115013 >> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] conn(0x5592bded05b0 0x5592bded29a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:33:44.520 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:44.519+0000 7f74223db740 4 client.38027 mds_command: new command op to 34535 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T10:33:44.520 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:44.519+0000 7f74223db740 1 -- 172.21.15.47:0/2161115013 --> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5592bde293f0 con 0x5592bded05b0 2024-03-21T10:33:44.520 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:44.520+0000 7f741d600640 1 --2- 172.21.15.47:0/2161115013 >> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] conn(0x5592bded05b0 0x5592bded29a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:33:44.520 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:44.520+0000 7f741d600640 1 --2- 172.21.15.47:0/2161115013 >> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] conn(0x5592bded05b0 0x5592bded29a0 crc :-1 s=READY pgs=195 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=228537aa25febbd9 server_cookie=5686e23388bf1cbd in_seq=0 out_seq=0 2024-03-21T10:33:44.520 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:44.520+0000 7f7406000640 10 client.38027 ms_handle_connect on v2:172.21.15.100:6838/4173168440 2024-03-21T10:33:44.522 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:44.522+0000 7f7406000640 1 -- 172.21.15.47:0/2161115013 <== mds.0 v2:172.21.15.100:6838/4173168440 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x5592bde293f0 con 0x5592bded05b0 2024-03-21T10:33:44.522 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:44.522+0000 7f7406000640 10 client.38027 handle_command_reply: tid=0 2024-03-21T10:33:44.522 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:44.522+0000 7f74223db740 1 client.38027 shutdown 2024-03-21T10:33:44.522 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:44.522+0000 7f74223db740 1 -- 172.21.15.47:0/2161115013 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5592bdeb79a0 msgr2=0x5592bdebd220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:33:44.522 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:44.522+0000 7f74223db740 1 --2- 172.21.15.47:0/2161115013 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5592bdeb79a0 0x5592bdebd220 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f740000be10 tx=0x7f74000117b0 comp rx=0 tx=0).stop 2024-03-21T10:33:44.522 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:44.522+0000 7f74223db740 1 -- 172.21.15.47:0/2161115013 shutdown_connections 2024-03-21T10:33:44.522 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:44.522+0000 7f74223db740 1 --2- 172.21.15.47:0/2161115013 >> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] conn(0x5592bded05b0 0x5592bded29a0 crc :-1 s=READY pgs=195 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:33:44.522 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:44.522+0000 7f74223db740 1 --2- 172.21.15.47:0/2161115013 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5592bdec0580 0x5592bdebd760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:33:44.522 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:44.522+0000 7f74223db740 1 --2- 172.21.15.47:0/2161115013 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5592bdeb79a0 0x5592bdebd220 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:33:44.522 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:44.522+0000 7f74223db740 1 --2- 172.21.15.47:0/2161115013 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5592bdeb7080 0x5592bdebfe90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:33:44.522 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:44.522+0000 7f7406000640 0 client.38027 ms_handle_reset on v2:172.21.15.100:6838/4173168440 2024-03-21T10:33:44.522 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:44.522+0000 7f74223db740 1 -- 172.21.15.47:0/2161115013 >> 172.21.15.47:0/2161115013 conn(0x5592bdeb17b0 msgr2=0x5592bdeb3500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:33:44.522 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:44.522+0000 7f74223db740 1 -- 172.21.15.47:0/2161115013 shutdown_connections 2024-03-21T10:33:44.523 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:44.522+0000 7f74223db740 1 -- 172.21.15.47:0/2161115013 wait complete. 2024-03-21T10:33:44.523 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:44.522+0000 7f74223db740 20 client.38027 trim_cache size 0 max 16384 2024-03-21T10:33:44.539 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:44.541+0000 7f74223db740 1 Processor -- start 2024-03-21T10:33:44.539 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:44.541+0000 7f74223db740 1 -- start start 2024-03-21T10:33:44.540 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:44.541+0000 7f74223db740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5592bdeb7080 0x5592bdebf950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:33:44.541 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:44.541+0000 7f74223db740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5592bdeb79a0 0x5592bdebfe90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:33:44.541 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:44.541+0000 7f74223db740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5592bdec0580 0x5592bdeb06b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:33:44.541 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:44.541+0000 7f74223db740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74181b0c30 con 0x5592bdeb79a0 2024-03-21T10:33:44.541 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:44.541+0000 7f74223db740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74181b25b0 con 0x5592bdec0580 2024-03-21T10:33:44.541 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:44.541+0000 7f74223db740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74181b1290 con 0x5592bdeb7080 2024-03-21T10:33:44.541 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:44.541+0000 7f740fe00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5592bdeb79a0 0x5592bdebfe90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:33:44.541 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:44.542+0000 7f740fe00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5592bdeb79a0 0x5592bdebfe90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:58470/0 (socket says 172.21.15.47:58470) 2024-03-21T10:33:44.541 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:44.542+0000 7f740fe00640 1 -- 172.21.15.47:0/3074278444 learned_addr learned my addr 172.21.15.47:0/3074278444 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:33:44.541 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:44.542+0000 7f741d600640 1 --2- 172.21.15.47:0/3074278444 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5592bdec0580 0x5592bdeb06b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:33:44.541 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:44.542+0000 7f741d600640 1 --2- 172.21.15.47:0/3074278444 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5592bdeb7080 0x5592bdebf950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:33:44.541 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:44.542+0000 7f740fe00640 1 -- 172.21.15.47:0/3074278444 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5592bdeb7080 msgr2=0x5592bdebf950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:33:44.541 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:44.542+0000 7f740fe00640 1 --2- 172.21.15.47:0/3074278444 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5592bdeb7080 0x5592bdebf950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:33:44.541 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:44.542+0000 7f740fe00640 1 -- 172.21.15.47:0/3074278444 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5592bdec0580 msgr2=0x5592bdeb06b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:33:44.541 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:44.542+0000 7f740fe00640 1 --2- 172.21.15.47:0/3074278444 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5592bdec0580 0x5592bdeb06b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:33:44.542 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:44.542+0000 7f740fe00640 1 -- 172.21.15.47:0/3074278444 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5592bdeb6430 con 0x5592bdeb79a0 2024-03-21T10:33:44.542 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:44.543+0000 7f740fe00640 1 --2- 172.21.15.47:0/3074278444 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5592bdeb79a0 0x5592bdebfe90 secure :-1 s=READY pgs=1166 cs=0 l=1 rev1=1 crypto rx=0x7f74000058c0 tx=0x7f740000b5e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ae7f1c5a7c4341d7 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:33:44.543 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:44.543+0000 7f740d200640 1 -- 172.21.15.47:0/3074278444 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f740002d070 con 0x5592bdeb79a0 2024-03-21T10:33:44.543 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:44.543+0000 7f740d200640 1 -- 172.21.15.47:0/3074278444 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f7400012b90 con 0x5592bdeb79a0 2024-03-21T10:33:44.543 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:44.543+0000 7f740d200640 1 -- 172.21.15.47:0/3074278444 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7400005040 con 0x5592bdeb79a0 2024-03-21T10:33:44.543 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:44.544+0000 7f74223db740 1 -- 172.21.15.47:0/3074278444 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5592bdeb79a0 msgr2=0x5592bdebfe90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:33:44.543 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:44.544+0000 7f74223db740 1 --2- 172.21.15.47:0/3074278444 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5592bdeb79a0 0x5592bdebfe90 secure :-1 s=READY pgs=1166 cs=0 l=1 rev1=1 crypto rx=0x7f74000058c0 tx=0x7f740000b5e0 comp rx=0 tx=0).stop 2024-03-21T10:33:44.543 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:44.544+0000 7f74223db740 1 -- 172.21.15.47:0/3074278444 shutdown_connections 2024-03-21T10:33:44.543 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:44.544+0000 7f74223db740 1 --2- 172.21.15.47:0/3074278444 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5592bdec0580 0x5592bdeb06b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:33:44.543 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:44.544+0000 7f74223db740 1 --2- 172.21.15.47:0/3074278444 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5592bdeb79a0 0x5592bdebfe90 unknown :-1 s=CLOSED pgs=1166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:33:44.543 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:44.544+0000 7f74223db740 1 --2- 172.21.15.47:0/3074278444 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5592bdeb7080 0x5592bdebf950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:33:44.543 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:44.544+0000 7f74223db740 1 -- 172.21.15.47:0/3074278444 >> 172.21.15.47:0/3074278444 conn(0x5592bdeb17b0 msgr2=0x5592bdeaffa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:33:44.543 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:44.544+0000 7f74223db740 1 -- 172.21.15.47:0/3074278444 shutdown_connections 2024-03-21T10:33:44.543 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:44.544+0000 7f74223db740 1 -- 172.21.15.47:0/3074278444 wait complete. 2024-03-21T10:33:44.543 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:44.544+0000 7f74223db740 1 -- start start 2024-03-21T10:33:44.544 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:44.545+0000 7f74223db740 1 Processor -- start 2024-03-21T10:33:44.546 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:44.545+0000 7f74223db740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5592bdeb7080 0x5592bdec5740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:33:44.546 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:44.545+0000 7f74223db740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5592bdeb79a0 0x5592bdec5c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:33:44.546 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:44.545+0000 7f74223db740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5592bdec0580 0x5592bdec61c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:33:44.546 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:44.545+0000 7f74223db740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74181b1f50 con 0x5592bdeb79a0 2024-03-21T10:33:44.547 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:44.545+0000 7f74223db740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74181b18f0 con 0x5592bdec0580 2024-03-21T10:33:44.547 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:44.545+0000 7f74223db740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74181aee10 con 0x5592bdeb7080 2024-03-21T10:33:44.547 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:44.545+0000 7f74223db740 10 client.0 _ll_register_callbacks cb 0x7f740000b3a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T10:33:44.547 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:44.545+0000 7f740fe00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5592bdeb79a0 0x5592bdec5c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:33:44.547 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:44.546+0000 7f740fe00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5592bdeb79a0 0x5592bdec5c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:58484/0 (socket says 172.21.15.47:58484) 2024-03-21T10:33:44.547 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:44.546+0000 7f740fe00640 1 -- 172.21.15.47:0/823612119 learned_addr learned my addr 172.21.15.47:0/823612119 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:33:44.547 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:44.546+0000 7f741d600640 1 --2- 172.21.15.47:0/823612119 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5592bdec0580 0x5592bdec61c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:33:44.547 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:44.546+0000 7f741d600640 1 --2- 172.21.15.47:0/823612119 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5592bdeb7080 0x5592bdec5740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:33:44.547 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:44.546+0000 7f740fe00640 1 -- 172.21.15.47:0/823612119 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5592bdeb7080 msgr2=0x5592bdec5740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:33:44.547 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:44.546+0000 7f740fe00640 1 --2- 172.21.15.47:0/823612119 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5592bdeb7080 0x5592bdec5740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:33:44.547 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:44.546+0000 7f740fe00640 1 -- 172.21.15.47:0/823612119 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5592bdec0580 msgr2=0x5592bdec61c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:33:44.547 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:44.546+0000 7f740fe00640 1 --2- 172.21.15.47:0/823612119 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5592bdec0580 0x5592bdec61c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:33:44.547 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:44.546+0000 7f740fe00640 1 -- 172.21.15.47:0/823612119 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5592bdf2be50 con 0x5592bdeb79a0 2024-03-21T10:33:44.547 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:44.546+0000 7f741d600640 1 --2- 172.21.15.47:0/823612119 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5592bdeb7080 0x5592bdec5740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T10:33:44.548 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:44.546+0000 7f740fe00640 1 --2- 172.21.15.47:0/823612119 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5592bdeb79a0 0x5592bdec5c80 secure :-1 s=READY pgs=1167 cs=0 l=1 rev1=1 crypto rx=0x7f74000117f0 tx=0x7f740000b4e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b7e2c939b489a294 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:33:44.548 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:44.546+0000 7f7406000640 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-03-21T10:33:44.548 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:44.547+0000 7f7406000640 1 -- 172.21.15.47:0/823612119 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f740002d040 con 0x5592bdeb79a0 2024-03-21T10:33:44.548 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:44.547+0000 7f7406000640 1 -- 172.21.15.47:0/823612119 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(101..101 src has 1..101) v4 ==== 9562+0+0 (secure 0 0 0) 0x7f740001adf0 con 0x5592bdeb79a0 2024-03-21T10:33:44.548 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:44.547+0000 7f74223db740 1 -- 172.21.15.47:0/823612119 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f7400012b90 con 0x5592bdeb79a0 2024-03-21T10:33:44.548 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:44.547+0000 7f7406000640 1 -- 172.21.15.47:0/823612119 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7400012e40 con 0x5592bdeb79a0 2024-03-21T10:33:44.548 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:44.547+0000 7f74223db740 10 client.18267 fetch_fsmap learned FSMap version 32 2024-03-21T10:33:44.548 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:44.547+0000 7f74223db740 1 -- 172.21.15.47:0/823612119 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x5592bdf2d190 con 0x5592bdeb79a0 2024-03-21T10:33:44.548 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:44.547+0000 7f7406000640 1 -- 172.21.15.47:0/823612119 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f7400012b90 con 0x5592bdeb79a0 2024-03-21T10:33:44.548 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:44.547+0000 7f7406000640 1 -- 172.21.15.47:0/823612119 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 32) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f7400039020 con 0x5592bdeb79a0 2024-03-21T10:33:44.548 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:44.547+0000 7f74223db740 10 client.18267 fetch_fsmap finished waiting for FSMap version 32 2024-03-21T10:33:44.548 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:44.548+0000 7f74223db740 10 client.18267 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-21T10:33:44.548 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:44.548+0000 7f74223db740 20 client.18267 populate_metadata read hostname 'smithi047' 2024-03-21T10:33:44.548 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:44.548+0000 7f74223db740 1 --2- 172.21.15.47:0/823612119 >> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] conn(0x5592bdf2d7f0 0x5592bdf2fc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:33:44.548 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:44.548+0000 7f74223db740 4 client.18267 mds_command: new command op to 34535 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-03-21T10:33:44.549 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:44.548+0000 7f74223db740 1 -- 172.21.15.47:0/823612119 --> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x5592bde23740 con 0x5592bdf2d7f0 2024-03-21T10:33:44.549 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:44.548+0000 7f741d600640 1 --2- 172.21.15.47:0/823612119 >> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] conn(0x5592bdf2d7f0 0x5592bdf2fc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:33:44.549 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:44.549+0000 7f741d600640 1 --2- 172.21.15.47:0/823612119 >> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] conn(0x5592bdf2d7f0 0x5592bdf2fc10 crc :-1 s=READY pgs=196 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1eda0dce0018c8d8 server_cookie=3df184affeb72db6 in_seq=0 out_seq=0 2024-03-21T10:33:44.550 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:44.549+0000 7f7406000640 10 client.18267 ms_handle_connect on v2:172.21.15.100:6838/4173168440 2024-03-21T10:33:44.550 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:44.550+0000 7f7406000640 1 -- 172.21.15.47:0/823612119 <== mds.0 v2:172.21.15.100:6838/4173168440 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x5592bde23740 con 0x5592bdf2d7f0 2024-03-21T10:33:44.550 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:44.550+0000 7f7406000640 10 client.18267 handle_command_reply: tid=0 2024-03-21T10:33:44.550 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:44.550+0000 7f74223db740 1 client.18267 shutdown 2024-03-21T10:33:44.550 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:44.550+0000 7f74223db740 1 -- 172.21.15.47:0/823612119 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5592bdeb79a0 msgr2=0x5592bdec5c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:33:44.550 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:44.550+0000 7f74223db740 1 --2- 172.21.15.47:0/823612119 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5592bdeb79a0 0x5592bdec5c80 secure :-1 s=READY pgs=1167 cs=0 l=1 rev1=1 crypto rx=0x7f74000117f0 tx=0x7f740000b4e0 comp rx=0 tx=0).stop 2024-03-21T10:33:44.550 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:44.551+0000 7f74223db740 1 -- 172.21.15.47:0/823612119 shutdown_connections 2024-03-21T10:33:44.550 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:44.551+0000 7f74223db740 1 --2- 172.21.15.47:0/823612119 >> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] conn(0x5592bdf2d7f0 0x5592bdf2fc10 crc :-1 s=READY pgs=196 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:33:44.550 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:44.551+0000 7f74223db740 1 --2- 172.21.15.47:0/823612119 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5592bdec0580 0x5592bdec61c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:33:44.550 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:44.551+0000 7f74223db740 1 --2- 172.21.15.47:0/823612119 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5592bdeb79a0 0x5592bdec5c80 unknown :-1 s=CLOSED pgs=1167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:33:44.550 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:44.551+0000 7f74223db740 1 --2- 172.21.15.47:0/823612119 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5592bdeb7080 0x5592bdec5740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:33:44.550 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:44.551+0000 7f7406000640 0 client.18267 ms_handle_reset on v2:172.21.15.100:6838/4173168440 2024-03-21T10:33:44.550 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:44.551+0000 7f74223db740 1 -- 172.21.15.47:0/823612119 >> 172.21.15.47:0/823612119 conn(0x5592bdeb17b0 msgr2=0x5592bdeb0830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:33:44.550 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:44.551+0000 7f74223db740 1 -- 172.21.15.47:0/823612119 shutdown_connections 2024-03-21T10:33:44.551 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:44.551+0000 7f74223db740 1 -- 172.21.15.47:0/823612119 wait complete. 2024-03-21T10:33:44.551 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:44.551+0000 7f74223db740 20 client.18267 trim_cache size 0 max 16384 2024-03-21T10:33:44.551 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-03-21T10:33:44.551 INFO:teuthology.orchestra.run.smithi047.stdout: "return_code": 0, 2024-03-21T10:33:44.551 INFO:teuthology.orchestra.run.smithi047.stdout: "scrub_tag": "0f268c6f-174b-452d-941f-5bf4502614bf", 2024-03-21T10:33:44.551 INFO:teuthology.orchestra.run.smithi047.stdout: "mode": "asynchronous" 2024-03-21T10:33:44.551 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-03-21T10:33:44.552 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:44.552+0000 7f741f600640 1 -- 172.21.15.47:0/674073750 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f73fc0a3190 msgr2=0x7f73fc0a5650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:33:44.552 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:44.552+0000 7f741f600640 1 --2- 172.21.15.47:0/674073750 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f73fc0a3190 0x7f73fc0a5650 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x5592bdeb6b50 tx=0x7f74080f5670 comp rx=0 tx=0).stop 2024-03-21T10:33:44.552 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:44.552+0000 7f741f600640 1 -- 172.21.15.47:0/674073750 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7418101620 msgr2=0x7f741819a0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:33:44.552 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:44.552+0000 7f741f600640 1 --2- 172.21.15.47:0/674073750 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7418101620 0x7f741819a0a0 secure :-1 s=READY pgs=1165 cs=0 l=1 rev1=1 crypto rx=0x7f7408000c00 tx=0x7f74080045e0 comp rx=0 tx=0).stop 2024-03-21T10:33:44.552 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:44.552+0000 7f741f600640 1 -- 172.21.15.47:0/674073750 shutdown_connections 2024-03-21T10:33:44.552 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:44.552+0000 7f741f600640 1 --2- 172.21.15.47:0/674073750 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f73fc0a3190 0x7f73fc0a5650 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:33:44.553 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:44.552+0000 7f741f600640 1 --2- 172.21.15.47:0/674073750 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7418107630 0x7f741819e960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:33:44.553 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:44.552+0000 7f741f600640 1 --2- 172.21.15.47:0/674073750 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7418101fe0 0x7f741819a5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:33:44.553 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:44.552+0000 7f741f600640 1 --2- 172.21.15.47:0/674073750 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7418101620 0x7f741819a0a0 unknown :-1 s=CLOSED pgs=1165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:33:44.553 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:44.552+0000 7f741f600640 1 -- 172.21.15.47:0/674073750 >> 172.21.15.47:0/674073750 conn(0x7f74180fd130 msgr2=0x7f74180fd740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:33:44.553 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:44.552+0000 7f741f600640 1 -- 172.21.15.47:0/674073750 shutdown_connections 2024-03-21T10:33:44.553 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:44.552+0000 7f741f600640 1 -- 172.21.15.47:0/674073750 wait complete. 2024-03-21T10:33:44.568 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-21T10:33:44.568 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-21T10:33:44.786 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:44.787+0000 7febc8e00640 1 -- 172.21.15.47:0/1302554595 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7febc410de90 msgr2=0x7febc4110280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:33:44.786 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:44.787+0000 7febc8e00640 1 --2- 172.21.15.47:0/1302554595 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7febc410de90 0x7febc4110280 secure :-1 s=READY pgs=1168 cs=0 l=1 rev1=1 crypto rx=0x7febbc009ca0 tx=0x7febbc02c100 comp rx=0 tx=0).stop 2024-03-21T10:33:44.787 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:44.788+0000 7febc8e00640 1 -- 172.21.15.47:0/1302554595 shutdown_connections 2024-03-21T10:33:44.787 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:44.788+0000 7febc8e00640 1 --2- 172.21.15.47:0/1302554595 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7febc410de90 0x7febc4110280 unknown :-1 s=CLOSED pgs=1168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:33:44.787 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:44.788+0000 7febc8e00640 1 --2- 172.21.15.47:0/1302554595 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7febc4100c90 0x7febc410d8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:33:44.787 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:44.788+0000 7febc8e00640 1 --2- 172.21.15.47:0/1302554595 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7febc41002e0 0x7febc41006c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:33:44.787 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:44.788+0000 7febc8e00640 1 -- 172.21.15.47:0/1302554595 >> 172.21.15.47:0/1302554595 conn(0x7febc40fbef0 msgr2=0x7febc40fe310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:33:44.787 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:44.788+0000 7febc8e00640 1 -- 172.21.15.47:0/1302554595 shutdown_connections 2024-03-21T10:33:44.787 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:44.788+0000 7febc8e00640 1 -- 172.21.15.47:0/1302554595 wait complete. 2024-03-21T10:33:44.788 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:44.789+0000 7febc8e00640 1 Processor -- start 2024-03-21T10:33:44.788 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:44.790+0000 7febc8e00640 1 -- start start 2024-03-21T10:33:44.789 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:44.790+0000 7febc8e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7febc41002e0 0x7febc4199f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:33:44.789 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:44.790+0000 7febc8e00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7febc4100c90 0x7febc419a4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:33:44.789 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:44.790+0000 7febc8e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7febc410de90 0x7febc419e830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:33:44.789 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:44.790+0000 7febc8e00640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7febc41139b0 con 0x7febc41002e0 2024-03-21T10:33:44.789 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:44.790+0000 7febc8e00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7febc4113830 con 0x7febc410de90 2024-03-21T10:33:44.789 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:44.790+0000 7febc8e00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7febc4113b30 con 0x7febc4100c90 2024-03-21T10:33:44.790 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:44.790+0000 7febc3400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7febc41002e0 0x7febc4199f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:33:44.790 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:44.791+0000 7febc3400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7febc41002e0 0x7febc4199f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:58504/0 (socket says 172.21.15.47:58504) 2024-03-21T10:33:44.790 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:44.791+0000 7febc3400640 1 -- 172.21.15.47:0/354696378 learned_addr learned my addr 172.21.15.47:0/354696378 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:33:44.790 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:44.791+0000 7febc3e00640 1 --2- 172.21.15.47:0/354696378 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7febc410de90 0x7febc419e830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:33:44.790 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:44.791+0000 7febc2a00640 1 --2- 172.21.15.47:0/354696378 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7febc4100c90 0x7febc419a4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:33:44.790 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:44.791+0000 7febc3400640 1 -- 172.21.15.47:0/354696378 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7febc4100c90 msgr2=0x7febc419a4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:33:44.790 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:44.791+0000 7febc3400640 1 --2- 172.21.15.47:0/354696378 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7febc4100c90 0x7febc419a4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:33:44.790 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:44.791+0000 7febc3400640 1 -- 172.21.15.47:0/354696378 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7febc410de90 msgr2=0x7febc419e830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:33:44.790 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:44.791+0000 7febc3400640 1 --2- 172.21.15.47:0/354696378 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7febc410de90 0x7febc419e830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:33:44.790 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:44.791+0000 7febc3400640 1 -- 172.21.15.47:0/354696378 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7febc419ef60 con 0x7febc41002e0 2024-03-21T10:33:44.791 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:44.792+0000 7febc3400640 1 --2- 172.21.15.47:0/354696378 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7febc41002e0 0x7febc4199f70 secure :-1 s=READY pgs=1169 cs=0 l=1 rev1=1 crypto rx=0x7febb400e9f0 tx=0x7febb400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:33:44.791 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:44.792+0000 7febbbe00640 1 -- 172.21.15.47:0/354696378 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febb400cd80 con 0x7febc41002e0 2024-03-21T10:33:44.791 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:44.792+0000 7febbbe00640 1 -- 172.21.15.47:0/354696378 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7febb40108e0 con 0x7febc41002e0 2024-03-21T10:33:44.791 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:44.792+0000 7febbbe00640 1 -- 172.21.15.47:0/354696378 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febb4018920 con 0x7febc41002e0 2024-03-21T10:33:44.791 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:44.792+0000 7febc8e00640 1 -- 172.21.15.47:0/354696378 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7febc419f250 con 0x7febc41002e0 2024-03-21T10:33:44.791 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:44.792+0000 7febc8e00640 1 -- 172.21.15.47:0/354696378 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7febc41a6ae0 con 0x7febc41002e0 2024-03-21T10:33:44.793 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:44.795+0000 7febcd1b8740 1 Processor -- start 2024-03-21T10:33:44.793 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:44.795+0000 7febcd1b8740 1 -- start start 2024-03-21T10:33:44.794 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:44.796+0000 7febcd1b8740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55bbdeb16dd0 0x55bbdeb15b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:33:44.794 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:44.796+0000 7febcd1b8740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55bbdeb160f0 0x55bbdeb16570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:33:44.794 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:44.796+0000 7febcd1b8740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55bbdeb1f460 0x55bbdeb21850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:33:44.794 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:44.796+0000 7febcd1b8740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bbdea7a250 con 0x55bbdeb160f0 2024-03-21T10:33:44.795 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:44.796+0000 7febcd1b8740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bbdea22a30 con 0x55bbdeb16dd0 2024-03-21T10:33:44.795 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:44.796+0000 7febcd1b8740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bbdea762f0 con 0x55bbdeb1f460 2024-03-21T10:33:44.795 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:44.797+0000 7febc3e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55bbdeb16dd0 0x55bbdeb15b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:33:44.796 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:44.797+0000 7febc2a00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55bbdeb1f460 0x55bbdeb21850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:33:44.796 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:44.797+0000 7febc3e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55bbdeb16dd0 0x55bbdeb15b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:37240/0 (socket says 172.21.15.47:37240) 2024-03-21T10:33:44.796 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:44.797+0000 7febc3e00640 1 -- 172.21.15.47:0/570251198 learned_addr learned my addr 172.21.15.47:0/570251198 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:33:44.796 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:44.797+0000 7febc3e00640 1 -- 172.21.15.47:0/570251198 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55bbdeb1f460 msgr2=0x55bbdeb21850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:33:44.796 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:44.797+0000 7febc3e00640 1 --2- 172.21.15.47:0/570251198 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55bbdeb1f460 0x55bbdeb21850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:33:44.796 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:44.797+0000 7febc3e00640 1 -- 172.21.15.47:0/570251198 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55bbdeb160f0 msgr2=0x55bbdeb16570 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-21T10:33:44.796 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:44.797+0000 7febc3e00640 1 --2- 172.21.15.47:0/570251198 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55bbdeb160f0 0x55bbdeb16570 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:33:44.796 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:44.797+0000 7febc3e00640 1 -- 172.21.15.47:0/570251198 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55bbdea0d690 con 0x55bbdeb16dd0 2024-03-21T10:33:44.796 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:44.797+0000 7febbbe00640 1 -- 172.21.15.47:0/354696378 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 27) v1 ==== 137985+0+0 (secure 0 0 0) 0x7febb4010a90 con 0x7febc41002e0 2024-03-21T10:33:44.797 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:44.798+0000 7febc3e00640 1 --2- 172.21.15.47:0/570251198 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55bbdeb16dd0 0x55bbdeb15b20 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7febbc005cc0 tx=0x7febbc005ef0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=15ba058378983a73 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:33:44.797 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:44.798+0000 7febb8c00640 1 -- 172.21.15.47:0/570251198 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febbc043070 con 0x55bbdeb16dd0 2024-03-21T10:33:44.797 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:44.798+0000 7febb8c00640 1 -- 172.21.15.47:0/570251198 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7febbc032970 con 0x55bbdeb16dd0 2024-03-21T10:33:44.797 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:44.799+0000 7febb8c00640 1 -- 172.21.15.47:0/570251198 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febbc039040 con 0x55bbdeb16dd0 2024-03-21T10:33:44.797 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:44.799+0000 7febbbe00640 1 --2- 172.21.15.47:0/354696378 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7feba80a3110 0x7feba80a55d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:33:44.797 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:44.799+0000 7febbbe00640 1 -- 172.21.15.47:0/354696378 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(101..101 src has 1..101) v4 ==== 9562+0+0 (secure 0 0 0) 0x7febb40cff10 con 0x7febc41002e0 2024-03-21T10:33:44.798 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:44.799+0000 7febc3400640 1 --2- 172.21.15.47:0/354696378 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7feba80a3110 0x7feba80a55d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:33:44.798 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:44.799+0000 7febcd1b8740 1 -- 172.21.15.47:0/570251198 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55bbdeb16dd0 msgr2=0x55bbdeb15b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:33:44.799 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:44.799+0000 7febcd1b8740 1 --2- 172.21.15.47:0/570251198 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55bbdeb16dd0 0x55bbdeb15b20 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7febbc005cc0 tx=0x7febbc005ef0 comp rx=0 tx=0).stop 2024-03-21T10:33:44.799 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:44.800+0000 7febcd1b8740 1 -- 172.21.15.47:0/570251198 shutdown_connections 2024-03-21T10:33:44.799 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:44.800+0000 7febcd1b8740 1 --2- 172.21.15.47:0/570251198 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55bbdeb1f460 0x55bbdeb21850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:33:44.799 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:44.800+0000 7febcd1b8740 1 --2- 172.21.15.47:0/570251198 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55bbdeb160f0 0x55bbdeb16570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:33:44.799 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:44.800+0000 7febcd1b8740 1 --2- 172.21.15.47:0/570251198 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55bbdeb16dd0 0x55bbdeb15b20 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:33:44.799 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:44.800+0000 7febcd1b8740 1 -- 172.21.15.47:0/570251198 >> 172.21.15.47:0/570251198 conn(0x55bbdeadd200 msgr2=0x55bbdeb10b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:33:44.799 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:44.800+0000 7febcd1b8740 1 -- 172.21.15.47:0/570251198 shutdown_connections 2024-03-21T10:33:44.799 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:44.800+0000 7febcd1b8740 1 -- 172.21.15.47:0/570251198 wait complete. 2024-03-21T10:33:44.799 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:44.800+0000 7febc3400640 1 --2- 172.21.15.47:0/354696378 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7feba80a3110 0x7feba80a55d0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x55bbdeb17320 tx=0x7febb4009310 comp rx=0 tx=0).ready entity=mgr.15687 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:33:44.801 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:44.801+0000 7febcd1b8740 1 -- start start 2024-03-21T10:33:44.801 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:44.801+0000 7febcd1b8740 1 Processor -- start 2024-03-21T10:33:44.801 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:44.801+0000 7febcd1b8740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55bbdeb160f0 0x55bbdeb1ebd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:33:44.801 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:44.801+0000 7febcd1b8740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55bbdeb16dd0 0x55bbdeb1bf60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:33:44.801 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:44.801+0000 7febcd1b8740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55bbdeb1f460 0x55bbdeb1c4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:33:44.801 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:44.801+0000 7febcd1b8740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bbdeb22270 con 0x55bbdeb1f460 2024-03-21T10:33:44.801 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:44.801+0000 7febcd1b8740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bbdea35e80 con 0x55bbdeb16dd0 2024-03-21T10:33:44.801 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:44.801+0000 7febcd1b8740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bbdeb223f0 con 0x55bbdeb160f0 2024-03-21T10:33:44.801 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:44.802+0000 7febcd1b8740 10 client.0 _ll_register_callbacks cb 0x55bbdea48130 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T10:33:44.801 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:44.802+0000 7febc3e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55bbdeb1f460 0x55bbdeb1c4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:33:44.801 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:44.802+0000 7febc3e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55bbdeb1f460 0x55bbdeb1c4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:58520/0 (socket says 172.21.15.47:58520) 2024-03-21T10:33:44.801 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:44.802+0000 7febc3e00640 1 -- 172.21.15.47:0/3523678079 learned_addr learned my addr 172.21.15.47:0/3523678079 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:33:44.801 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:44.802+0000 7febc3e00640 1 --2- 172.21.15.47:0/3523678079 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55bbdeb160f0 0x55bbdeb1ebd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:33:44.802 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:44.802+0000 7febc2a00640 1 --2- 172.21.15.47:0/3523678079 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55bbdeb16dd0 0x55bbdeb1bf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:33:44.802 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:44.802+0000 7febc3e00640 1 -- 172.21.15.47:0/3523678079 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55bbdeb160f0 msgr2=0x55bbdeb1ebd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:33:44.802 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:44.802+0000 7febc3e00640 1 --2- 172.21.15.47:0/3523678079 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55bbdeb160f0 0x55bbdeb1ebd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:33:44.802 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:44.802+0000 7febc3e00640 1 -- 172.21.15.47:0/3523678079 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55bbdeb16dd0 msgr2=0x55bbdeb1bf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:33:44.802 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:44.802+0000 7febc3e00640 1 --2- 172.21.15.47:0/3523678079 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55bbdeb16dd0 0x55bbdeb1bf60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:33:44.802 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:44.803+0000 7febc3e00640 1 -- 172.21.15.47:0/3523678079 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55bbde8587d0 con 0x55bbdeb1f460 2024-03-21T10:33:44.804 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:44.803+0000 7febc3e00640 1 --2- 172.21.15.47:0/3523678079 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55bbdeb1f460 0x55bbdeb1c4a0 secure :-1 s=READY pgs=1170 cs=0 l=1 rev1=1 crypto rx=0x7febbc009620 tx=0x7febbc02e040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8dc763480372e7c9 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:33:44.804 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:44.803+0000 7febad600640 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-03-21T10:33:44.804 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:44.803+0000 7febad600640 1 -- 172.21.15.47:0/3523678079 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febbc039030 con 0x55bbdeb1f460 2024-03-21T10:33:44.804 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:44.803+0000 7febad600640 1 -- 172.21.15.47:0/3523678079 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(101..101 src has 1..101) v4 ==== 9562+0+0 (secure 0 0 0) 0x7febbc004040 con 0x55bbdeb1f460 2024-03-21T10:33:44.804 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:44.803+0000 7febcd1b8740 1 -- 172.21.15.47:0/3523678079 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7febbc032970 con 0x55bbdeb1f460 2024-03-21T10:33:44.804 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:44.804+0000 7febad600640 1 -- 172.21.15.47:0/3523678079 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7febbc05c050 con 0x55bbdeb1f460 2024-03-21T10:33:44.804 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:44.804+0000 7febcd1b8740 10 client.18276 fetch_fsmap learned FSMap version 32 2024-03-21T10:33:44.804 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:44.804+0000 7febcd1b8740 1 -- 172.21.15.47:0/3523678079 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x55bbde78e3f0 con 0x55bbdeb1f460 2024-03-21T10:33:44.804 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:44.804+0000 7febad600640 1 -- 172.21.15.47:0/3523678079 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7febbc032970 con 0x55bbdeb1f460 2024-03-21T10:33:44.804 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:44.804+0000 7febad600640 1 -- 172.21.15.47:0/3523678079 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 32) v1 ==== 7942+0+0 (secure 0 0 0) 0x7febbc0487b0 con 0x55bbdeb1f460 2024-03-21T10:33:44.804 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:44.804+0000 7febcd1b8740 10 client.18276 fetch_fsmap finished waiting for FSMap version 32 2024-03-21T10:33:44.804 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:44.804+0000 7febcd1b8740 10 client.18276 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-21T10:33:44.805 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:44.804+0000 7febcd1b8740 20 client.18276 populate_metadata read hostname 'smithi047' 2024-03-21T10:33:44.805 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:44.804+0000 7febcd1b8740 1 --2- 172.21.15.47:0/3523678079 >> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] conn(0x55bbdeb2f250 0x55bbdeb31640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:33:44.805 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:44.804+0000 7febcd1b8740 4 client.18276 mds_command: new command op to 34535 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T10:33:44.805 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:44.804+0000 7febcd1b8740 1 -- 172.21.15.47:0/3523678079 --> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55bbdea87790 con 0x55bbdeb2f250 2024-03-21T10:33:44.805 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:44.805+0000 7febc2a00640 1 --2- 172.21.15.47:0/3523678079 >> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] conn(0x55bbdeb2f250 0x55bbdeb31640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:33:44.805 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:44.806+0000 7febc2a00640 1 --2- 172.21.15.47:0/3523678079 >> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] conn(0x55bbdeb2f250 0x55bbdeb31640 crc :-1 s=READY pgs=197 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b91f73f29b6f66ad server_cookie=30356c7dc5f68b2c in_seq=0 out_seq=0 2024-03-21T10:33:44.805 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:44.806+0000 7febad600640 10 client.18276 ms_handle_connect on v2:172.21.15.100:6838/4173168440 2024-03-21T10:33:44.806 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:44.807+0000 7febad600640 1 -- 172.21.15.47:0/3523678079 <== mds.0 v2:172.21.15.100:6838/4173168440 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55bbdea87790 con 0x55bbdeb2f250 2024-03-21T10:33:44.806 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:44.807+0000 7febad600640 10 client.18276 handle_command_reply: tid=0 2024-03-21T10:33:44.806 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:44.807+0000 7febcd1b8740 1 client.18276 shutdown 2024-03-21T10:33:44.806 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:44.807+0000 7febcd1b8740 1 -- 172.21.15.47:0/3523678079 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55bbdeb1f460 msgr2=0x55bbdeb1c4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:33:44.806 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:44.807+0000 7febcd1b8740 1 --2- 172.21.15.47:0/3523678079 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55bbdeb1f460 0x55bbdeb1c4a0 secure :-1 s=READY pgs=1170 cs=0 l=1 rev1=1 crypto rx=0x7febbc009620 tx=0x7febbc02e040 comp rx=0 tx=0).stop 2024-03-21T10:33:44.807 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:44.807+0000 7febcd1b8740 1 -- 172.21.15.47:0/3523678079 shutdown_connections 2024-03-21T10:33:44.807 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:44.807+0000 7febcd1b8740 1 --2- 172.21.15.47:0/3523678079 >> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] conn(0x55bbdeb2f250 0x55bbdeb31640 crc :-1 s=READY pgs=197 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:33:44.807 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:44.807+0000 7febcd1b8740 1 --2- 172.21.15.47:0/3523678079 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55bbdeb1f460 0x55bbdeb1c4a0 unknown :-1 s=CLOSED pgs=1170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:33:44.807 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:44.807+0000 7febcd1b8740 1 --2- 172.21.15.47:0/3523678079 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55bbdeb16dd0 0x55bbdeb1bf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:33:44.807 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:44.807+0000 7febcd1b8740 1 --2- 172.21.15.47:0/3523678079 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55bbdeb160f0 0x55bbdeb1ebd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:33:44.807 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:44.807+0000 7febcd1b8740 1 -- 172.21.15.47:0/3523678079 >> 172.21.15.47:0/3523678079 conn(0x55bbdeadd200 msgr2=0x55bbdeb12890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:33:44.807 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:44.807+0000 7febad600640 0 client.18276 ms_handle_reset on v2:172.21.15.100:6838/4173168440 2024-03-21T10:33:44.807 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:44.808+0000 7febcd1b8740 1 -- 172.21.15.47:0/3523678079 shutdown_connections 2024-03-21T10:33:44.807 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:44.808+0000 7febcd1b8740 1 -- 172.21.15.47:0/3523678079 wait complete. 2024-03-21T10:33:44.807 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:44.808+0000 7febcd1b8740 20 client.18276 trim_cache size 0 max 16384 2024-03-21T10:33:44.824 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:44.825+0000 7febcd1b8740 1 Processor -- start 2024-03-21T10:33:44.824 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:44.825+0000 7febcd1b8740 1 -- start start 2024-03-21T10:33:44.824 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:44.826+0000 7febcd1b8740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55bbdeb160f0 0x55bbdeb1e690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:33:44.824 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:44.826+0000 7febcd1b8740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55bbdeb16dd0 0x55bbdeb1ebd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:33:44.824 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:44.826+0000 7febcd1b8740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55bbdeb1f460 0x55bbdeb20850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:33:44.824 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:44.826+0000 7febcd1b8740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7febc41b0a00 con 0x55bbdeb1f460 2024-03-21T10:33:44.824 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:44.826+0000 7febcd1b8740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7febc41b2380 con 0x55bbdeb160f0 2024-03-21T10:33:44.824 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:44.826+0000 7febcd1b8740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7febc41b1060 con 0x55bbdeb16dd0 2024-03-21T10:33:44.826 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:44.826+0000 7febc3e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55bbdeb1f460 0x55bbdeb20850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:33:44.826 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:44.826+0000 7febc3e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55bbdeb160f0 0x55bbdeb1e690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:33:44.826 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:44.826+0000 7febc3e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55bbdeb160f0 0x55bbdeb1e690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:37258/0 (socket says 172.21.15.47:37258) 2024-03-21T10:33:44.826 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:44.826+0000 7febc3e00640 1 -- 172.21.15.47:0/612983156 learned_addr learned my addr 172.21.15.47:0/612983156 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:33:44.826 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:44.826+0000 7febc2a00640 1 --2- 172.21.15.47:0/612983156 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55bbdeb16dd0 0x55bbdeb1ebd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:33:44.826 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:44.827+0000 7febc3e00640 1 -- 172.21.15.47:0/612983156 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55bbdeb16dd0 msgr2=0x55bbdeb1ebd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:33:44.826 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:44.827+0000 7febc3e00640 1 --2- 172.21.15.47:0/612983156 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55bbdeb16dd0 0x55bbdeb1ebd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:33:44.826 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:44.827+0000 7febc3e00640 1 -- 172.21.15.47:0/612983156 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55bbdeb1f460 msgr2=0x55bbdeb20850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:33:44.826 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:44.827+0000 7febc3e00640 1 --2- 172.21.15.47:0/612983156 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55bbdeb1f460 0x55bbdeb20850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:33:44.827 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:44.827+0000 7febc3e00640 1 -- 172.21.15.47:0/612983156 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55bbde74e0f0 con 0x55bbdeb160f0 2024-03-21T10:33:44.827 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:44.827+0000 7febc3e00640 1 --2- 172.21.15.47:0/612983156 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55bbdeb160f0 0x55bbdeb1e690 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7febbc02c710 tx=0x7febbc000f40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d94351e7c062d469 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:33:44.827 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:44.828+0000 7febafe00640 1 -- 172.21.15.47:0/612983156 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febbc039040 con 0x55bbdeb160f0 2024-03-21T10:33:44.827 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:44.828+0000 7febafe00640 1 -- 172.21.15.47:0/612983156 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7febbc004810 con 0x55bbdeb160f0 2024-03-21T10:33:44.827 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:44.828+0000 7febafe00640 1 -- 172.21.15.47:0/612983156 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febbc043040 con 0x55bbdeb160f0 2024-03-21T10:33:44.829 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:44.828+0000 7febcd1b8740 1 -- 172.21.15.47:0/612983156 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55bbdeb160f0 msgr2=0x55bbdeb1e690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:33:44.829 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:44.828+0000 7febcd1b8740 1 --2- 172.21.15.47:0/612983156 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55bbdeb160f0 0x55bbdeb1e690 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7febbc02c710 tx=0x7febbc000f40 comp rx=0 tx=0).stop 2024-03-21T10:33:44.829 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:44.829+0000 7febcd1b8740 1 -- 172.21.15.47:0/612983156 shutdown_connections 2024-03-21T10:33:44.829 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:44.829+0000 7febcd1b8740 1 --2- 172.21.15.47:0/612983156 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55bbdeb1f460 0x55bbdeb20850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:33:44.829 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:44.829+0000 7febcd1b8740 1 --2- 172.21.15.47:0/612983156 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55bbdeb16dd0 0x55bbdeb1ebd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:33:44.829 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:44.829+0000 7febcd1b8740 1 --2- 172.21.15.47:0/612983156 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55bbdeb160f0 0x55bbdeb1e690 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:33:44.829 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:44.829+0000 7febcd1b8740 1 -- 172.21.15.47:0/612983156 >> 172.21.15.47:0/612983156 conn(0x55bbdeadd200 msgr2=0x55bbdeb10670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:33:44.829 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:44.829+0000 7febcd1b8740 1 -- 172.21.15.47:0/612983156 shutdown_connections 2024-03-21T10:33:44.829 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:44.829+0000 7febcd1b8740 1 -- 172.21.15.47:0/612983156 wait complete. 2024-03-21T10:33:44.829 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:44.829+0000 7febcd1b8740 1 -- start start 2024-03-21T10:33:44.830 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:44.829+0000 7febcd1b8740 1 Processor -- start 2024-03-21T10:33:44.830 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:44.830+0000 7febcd1b8740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55bbdeb160f0 0x55bbdeb2fc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:33:44.830 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:44.830+0000 7febcd1b8740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55bbdeb16dd0 0x55bbdeb30170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:33:44.830 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:44.830+0000 7febcd1b8740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55bbdeb1f460 0x55bbdeb306b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:33:44.830 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:44.830+0000 7febcd1b8740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7febc41b1d20 con 0x55bbdeb160f0 2024-03-21T10:33:44.830 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:44.830+0000 7febcd1b8740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7febc41b16c0 con 0x55bbdeb1f460 2024-03-21T10:33:44.830 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:44.830+0000 7febcd1b8740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7febc41aebe0 con 0x55bbdeb16dd0 2024-03-21T10:33:44.830 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:44.830+0000 7febcd1b8740 10 client.0 _ll_register_callbacks cb 0x7febb000b3a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T10:33:44.830 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:44.830+0000 7febc3e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55bbdeb160f0 0x55bbdeb2fc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:33:44.830 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:44.830+0000 7febc3e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55bbdeb160f0 0x55bbdeb2fc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:58534/0 (socket says 172.21.15.47:58534) 2024-03-21T10:33:44.831 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:44.830+0000 7febc3e00640 1 -- 172.21.15.47:0/620752520 learned_addr learned my addr 172.21.15.47:0/620752520 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:33:44.831 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:44.831+0000 7febc3e00640 1 --2- 172.21.15.47:0/620752520 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55bbdeb1f460 0x55bbdeb306b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:33:44.831 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:44.831+0000 7febc2a00640 1 --2- 172.21.15.47:0/620752520 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55bbdeb16dd0 0x55bbdeb30170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:33:44.831 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:44.831+0000 7febc3e00640 1 -- 172.21.15.47:0/620752520 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55bbdeb16dd0 msgr2=0x55bbdeb30170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:33:44.831 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:44.831+0000 7febc3e00640 1 --2- 172.21.15.47:0/620752520 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55bbdeb16dd0 0x55bbdeb30170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:33:44.831 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:44.831+0000 7febc3e00640 1 -- 172.21.15.47:0/620752520 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55bbdeb1f460 msgr2=0x55bbdeb306b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:33:44.831 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:44.831+0000 7febc3e00640 1 --2- 172.21.15.47:0/620752520 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55bbdeb1f460 0x55bbdeb306b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:33:44.831 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:44.831+0000 7febc3e00640 1 -- 172.21.15.47:0/620752520 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55bbdeb89870 con 0x55bbdeb160f0 2024-03-21T10:33:44.834 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:44.831+0000 7febc3e00640 1 --2- 172.21.15.47:0/620752520 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55bbdeb160f0 0x55bbdeb2fc30 secure :-1 s=READY pgs=1171 cs=0 l=1 rev1=1 crypto rx=0x7febbc02c6e0 tx=0x7febbc0021f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=177c2f95563f9b75 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:33:44.834 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:44.831+0000 7febad600640 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-03-21T10:33:44.834 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:44.831+0000 7febad600640 1 -- 172.21.15.47:0/620752520 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febbc039070 con 0x55bbdeb160f0 2024-03-21T10:33:44.834 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:44.831+0000 7febad600640 1 -- 172.21.15.47:0/620752520 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(101..101 src has 1..101) v4 ==== 9562+0+0 (secure 0 0 0) 0x7febbc004b40 con 0x55bbdeb160f0 2024-03-21T10:33:44.834 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:44.831+0000 7febcd1b8740 1 -- 172.21.15.47:0/620752520 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7febbc004810 con 0x55bbdeb160f0 2024-03-21T10:33:44.834 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:44.832+0000 7febad600640 1 -- 172.21.15.47:0/620752520 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7febbc043040 con 0x55bbdeb160f0 2024-03-21T10:33:44.834 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:44.832+0000 7febcd1b8740 10 client.18282 fetch_fsmap learned FSMap version 32 2024-03-21T10:33:44.834 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:44.832+0000 7febcd1b8740 1 -- 172.21.15.47:0/620752520 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x55bbdeb8ab80 con 0x55bbdeb160f0 2024-03-21T10:33:44.834 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:44.832+0000 7febad600640 1 -- 172.21.15.47:0/620752520 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7febbc004810 con 0x55bbdeb160f0 2024-03-21T10:33:44.834 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:44.832+0000 7febad600640 1 -- 172.21.15.47:0/620752520 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 32) v1 ==== 7942+0+0 (secure 0 0 0) 0x7febbc003e70 con 0x55bbdeb160f0 2024-03-21T10:33:44.834 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:44.832+0000 7febcd1b8740 10 client.18282 fetch_fsmap finished waiting for FSMap version 32 2024-03-21T10:33:44.835 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:44.832+0000 7febcd1b8740 10 client.18282 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-21T10:33:44.835 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:44.832+0000 7febcd1b8740 20 client.18282 populate_metadata read hostname 'smithi047' 2024-03-21T10:33:44.835 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:44.832+0000 7febcd1b8740 1 --2- 172.21.15.47:0/620752520 >> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] conn(0x55bbdeb8b140 0x55bbdeb8d530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:33:44.835 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:44.832+0000 7febcd1b8740 4 client.18282 mds_command: new command op to 34535 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-21T10:33:44.835 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:44.832+0000 7febcd1b8740 1 -- 172.21.15.47:0/620752520 --> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55bbdea81ae0 con 0x55bbdeb8b140 2024-03-21T10:33:44.835 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:44.833+0000 7febc2a00640 1 --2- 172.21.15.47:0/620752520 >> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] conn(0x55bbdeb8b140 0x55bbdeb8d530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:33:44.835 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:44.834+0000 7febc2a00640 1 --2- 172.21.15.47:0/620752520 >> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] conn(0x55bbdeb8b140 0x55bbdeb8d530 crc :-1 s=READY pgs=198 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3fa24cd4f23bf924 server_cookie=95648a7ca1f02245 in_seq=0 out_seq=0 2024-03-21T10:33:44.835 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:44.834+0000 7febad600640 10 client.18282 ms_handle_connect on v2:172.21.15.100:6838/4173168440 2024-03-21T10:33:44.835 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:44.834+0000 7febad600640 1 -- 172.21.15.47:0/620752520 <== mds.0 v2:172.21.15.100:6838/4173168440 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x55bbdea81ae0 con 0x55bbdeb8b140 2024-03-21T10:33:44.835 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:44.834+0000 7febad600640 10 client.18282 handle_command_reply: tid=0 2024-03-21T10:33:44.835 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:44.834+0000 7febcd1b8740 1 client.18282 shutdown 2024-03-21T10:33:44.835 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:44.835+0000 7febcd1b8740 1 -- 172.21.15.47:0/620752520 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55bbdeb160f0 msgr2=0x55bbdeb2fc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:33:44.835 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:44.835+0000 7febcd1b8740 1 --2- 172.21.15.47:0/620752520 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55bbdeb160f0 0x55bbdeb2fc30 secure :-1 s=READY pgs=1171 cs=0 l=1 rev1=1 crypto rx=0x7febbc02c6e0 tx=0x7febbc0021f0 comp rx=0 tx=0).stop 2024-03-21T10:33:44.835 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:44.835+0000 7febcd1b8740 1 -- 172.21.15.47:0/620752520 shutdown_connections 2024-03-21T10:33:44.836 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:44.835+0000 7febcd1b8740 1 --2- 172.21.15.47:0/620752520 >> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] conn(0x55bbdeb8b140 0x55bbdeb8d530 crc :-1 s=READY pgs=198 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:33:44.836 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:44.835+0000 7febcd1b8740 1 --2- 172.21.15.47:0/620752520 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55bbdeb1f460 0x55bbdeb306b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:33:44.836 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:44.835+0000 7febcd1b8740 1 --2- 172.21.15.47:0/620752520 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55bbdeb16dd0 0x55bbdeb30170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:33:44.836 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:44.835+0000 7febcd1b8740 1 --2- 172.21.15.47:0/620752520 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55bbdeb160f0 0x55bbdeb2fc30 unknown :-1 s=CLOSED pgs=1171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:33:44.836 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:44.835+0000 7febad600640 0 client.18282 ms_handle_reset on v2:172.21.15.100:6838/4173168440 2024-03-21T10:33:44.836 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:44.835+0000 7febcd1b8740 1 -- 172.21.15.47:0/620752520 >> 172.21.15.47:0/620752520 conn(0x55bbdeadd200 msgr2=0x55bbdeb1a830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:33:44.836 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:44.835+0000 7febcd1b8740 1 -- 172.21.15.47:0/620752520 shutdown_connections 2024-03-21T10:33:44.836 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:44.835+0000 7febcd1b8740 1 -- 172.21.15.47:0/620752520 wait complete. 2024-03-21T10:33:44.836 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:44.835+0000 7febcd1b8740 20 client.18282 trim_cache size 0 max 16384 2024-03-21T10:33:44.836 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-03-21T10:33:44.837 INFO:teuthology.orchestra.run.smithi047.stdout: "status": "IDLE (waiting for more scrubs)", 2024-03-21T10:33:44.837 INFO:teuthology.orchestra.run.smithi047.stdout: "scrubs": { 2024-03-21T10:33:44.837 INFO:teuthology.orchestra.run.smithi047.stdout: "0f268c6f-174b-452d-941f-5bf4502614bf": { 2024-03-21T10:33:44.837 INFO:teuthology.orchestra.run.smithi047.stdout: "path": "/", 2024-03-21T10:33:44.837 INFO:teuthology.orchestra.run.smithi047.stdout: "tag": "0f268c6f-174b-452d-941f-5bf4502614bf", 2024-03-21T10:33:44.837 INFO:teuthology.orchestra.run.smithi047.stdout: "options": "recursive,force" 2024-03-21T10:33:44.837 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-21T10:33:44.837 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-21T10:33:44.837 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-03-21T10:33:44.838 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:44.836+0000 7febc8e00640 1 -- 172.21.15.47:0/354696378 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7feba80a3110 msgr2=0x7feba80a55d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:33:44.838 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:44.836+0000 7febc8e00640 1 --2- 172.21.15.47:0/354696378 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7feba80a3110 0x7feba80a55d0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x55bbdeb17320 tx=0x7febb4009310 comp rx=0 tx=0).stop 2024-03-21T10:33:44.838 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:44.836+0000 7febc8e00640 1 -- 172.21.15.47:0/354696378 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7febc41002e0 msgr2=0x7febc4199f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:33:44.838 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:44.836+0000 7febc8e00640 1 --2- 172.21.15.47:0/354696378 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7febc41002e0 0x7febc4199f70 secure :-1 s=READY pgs=1169 cs=0 l=1 rev1=1 crypto rx=0x7febb400e9f0 tx=0x7febb400eeb0 comp rx=0 tx=0).stop 2024-03-21T10:33:44.838 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:44.836+0000 7febc8e00640 1 -- 172.21.15.47:0/354696378 shutdown_connections 2024-03-21T10:33:44.838 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:44.836+0000 7febc8e00640 1 --2- 172.21.15.47:0/354696378 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7feba80a3110 0x7feba80a55d0 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:33:44.838 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:44.836+0000 7febc8e00640 1 --2- 172.21.15.47:0/354696378 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7febc410de90 0x7febc419e830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:33:44.838 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:44.836+0000 7febc8e00640 1 --2- 172.21.15.47:0/354696378 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7febc4100c90 0x7febc419a4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:33:44.839 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:44.836+0000 7febc8e00640 1 --2- 172.21.15.47:0/354696378 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7febc41002e0 0x7febc4199f70 unknown :-1 s=CLOSED pgs=1169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:33:44.839 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:44.836+0000 7febc8e00640 1 -- 172.21.15.47:0/354696378 >> 172.21.15.47:0/354696378 conn(0x7febc40fbef0 msgr2=0x7febc40fe2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:33:44.839 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:44.836+0000 7febc8e00640 1 -- 172.21.15.47:0/354696378 shutdown_connections 2024-03-21T10:33:44.839 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:33:44.836+0000 7febc8e00640 1 -- 172.21.15.47:0/354696378 wait complete. 2024-03-21T10:33:44.853 INFO:tasks.cephfs.filesystem:scrub status for tag:0f268c6f-174b-452d-941f-5bf4502614bf - {'path': '/', 'tag': '0f268c6f-174b-452d-941f-5bf4502614bf', 'options': 'recursive,force'} 2024-03-21T10:33:45.461 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:33:45 smithi047 ceph-mon[31547]: from='mgr.15687 172.21.15.47:0/8730661' entity='mgr.x' 2024-03-21T10:33:45.461 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:33:45 smithi047 ceph-mon[31547]: from='mgr.15687 172.21.15.47:0/8730661' entity='mgr.x' 2024-03-21T10:33:45.461 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:33:45 smithi047 ceph-mon[31547]: from='mgr.15687 172.21.15.47:0/8730661' entity='mgr.x' 2024-03-21T10:33:45.462 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:33:45 smithi047 ceph-mon[31547]: from='mgr.15687 172.21.15.47:0/8730661' entity='mgr.x' 2024-03-21T10:33:45.462 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:33:45 smithi047 ceph-mon[31547]: pgmap v260: 129 pgs: 129 active+clean; 260 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-03-21T10:33:45.462 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:33:45 smithi047 ceph-mon[31547]: scrub queued for path: / 2024-03-21T10:33:45.462 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:33:45 smithi047 ceph-mon[31547]: scrub summary: idle+waiting paths [/] 2024-03-21T10:33:45.462 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:33:45 smithi047 ceph-mon[31547]: scrub summary: active paths [/] 2024-03-21T10:33:45.462 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:33:45 smithi047 ceph-mon[31547]: scrub summary: idle+waiting paths [/] 2024-03-21T10:33:45.476 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:33:45 smithi100 ceph-mon[35193]: from='mgr.15687 172.21.15.47:0/8730661' entity='mgr.x' 2024-03-21T10:33:45.477 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:33:45 smithi100 ceph-mon[35193]: from='mgr.15687 172.21.15.47:0/8730661' entity='mgr.x' 2024-03-21T10:33:45.477 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:33:45 smithi100 ceph-mon[35193]: from='mgr.15687 172.21.15.47:0/8730661' entity='mgr.x' 2024-03-21T10:33:45.477 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:33:45 smithi100 ceph-mon[35193]: from='mgr.15687 172.21.15.47:0/8730661' entity='mgr.x' 2024-03-21T10:33:45.477 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:33:45 smithi100 ceph-mon[35193]: pgmap v260: 129 pgs: 129 active+clean; 260 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-03-21T10:33:45.477 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:33:45 smithi100 ceph-mon[35193]: scrub queued for path: / 2024-03-21T10:33:45.477 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:33:45 smithi100 ceph-mon[35193]: scrub summary: idle+waiting paths [/] 2024-03-21T10:33:45.477 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:33:45 smithi100 ceph-mon[35193]: scrub summary: active paths [/] 2024-03-21T10:33:45.477 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:33:45 smithi100 ceph-mon[35193]: scrub summary: idle+waiting paths [/] 2024-03-21T10:33:45.487 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:33:45 smithi157 ceph-mon[35581]: from='mgr.15687 172.21.15.47:0/8730661' entity='mgr.x' 2024-03-21T10:33:45.487 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:33:45 smithi157 ceph-mon[35581]: from='mgr.15687 172.21.15.47:0/8730661' entity='mgr.x' 2024-03-21T10:33:45.487 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:33:45 smithi157 ceph-mon[35581]: from='mgr.15687 172.21.15.47:0/8730661' entity='mgr.x' 2024-03-21T10:33:45.487 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:33:45 smithi157 ceph-mon[35581]: from='mgr.15687 172.21.15.47:0/8730661' entity='mgr.x' 2024-03-21T10:33:45.487 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:33:45 smithi157 ceph-mon[35581]: pgmap v260: 129 pgs: 129 active+clean; 260 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-03-21T10:33:45.488 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:33:45 smithi157 ceph-mon[35581]: scrub queued for path: / 2024-03-21T10:33:45.488 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:33:45 smithi157 ceph-mon[35581]: scrub summary: idle+waiting paths [/] 2024-03-21T10:33:45.488 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:33:45 smithi157 ceph-mon[35581]: scrub summary: active paths [/] 2024-03-21T10:33:45.488 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:33:45 smithi157 ceph-mon[35581]: scrub summary: idle+waiting paths [/] 2024-03-21T10:33:46.461 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:33:46 smithi047 ceph-mon[31547]: from='mgr.15687 172.21.15.47:0/8730661' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T10:33:46.461 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:33:46 smithi047 ceph-mon[31547]: from='mgr.15687 172.21.15.47:0/8730661' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T10:33:46.461 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:33:46 smithi047 ceph-mon[31547]: from='mgr.15687 172.21.15.47:0/8730661' entity='mgr.x' 2024-03-21T10:33:46.476 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:33:46 smithi100 ceph-mon[35193]: from='mgr.15687 172.21.15.47:0/8730661' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T10:33:46.476 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:33:46 smithi100 ceph-mon[35193]: from='mgr.15687 172.21.15.47:0/8730661' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T10:33:46.477 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:33:46 smithi100 ceph-mon[35193]: from='mgr.15687 172.21.15.47:0/8730661' entity='mgr.x' 2024-03-21T10:33:46.487 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:33:46 smithi157 ceph-mon[35581]: from='mgr.15687 172.21.15.47:0/8730661' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T10:33:46.487 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:33:46 smithi157 ceph-mon[35581]: from='mgr.15687 172.21.15.47:0/8730661' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T10:33:46.487 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:33:46 smithi157 ceph-mon[35581]: from='mgr.15687 172.21.15.47:0/8730661' entity='mgr.x' 2024-03-21T10:33:47.461 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:33:47 smithi047 ceph-mon[31547]: pgmap v261: 129 pgs: 129 active+clean; 260 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 12 op/s 2024-03-21T10:33:47.476 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:33:47 smithi100 ceph-mon[35193]: pgmap v261: 129 pgs: 129 active+clean; 260 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 12 op/s 2024-03-21T10:33:47.487 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:33:47 smithi157 ceph-mon[35581]: pgmap v261: 129 pgs: 129 active+clean; 260 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 12 op/s 2024-03-21T10:33:49.461 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:33:49 smithi047 ceph-mon[31547]: pgmap v262: 129 pgs: 129 active+clean; 260 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 9 op/s 2024-03-21T10:33:49.461 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:33:49 smithi047 ceph-mon[31547]: scrub summary: idle 2024-03-21T10:33:49.476 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:33:49 smithi100 ceph-mon[35193]: pgmap v262: 129 pgs: 129 active+clean; 260 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 9 op/s 2024-03-21T10:33:49.477 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:33:49 smithi100 ceph-mon[35193]: scrub summary: idle 2024-03-21T10:33:49.487 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:33:49 smithi157 ceph-mon[35581]: pgmap v262: 129 pgs: 129 active+clean; 260 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 9 op/s 2024-03-21T10:33:49.487 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:33:49 smithi157 ceph-mon[35581]: scrub summary: idle 2024-03-21T10:33:51.476 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:33:51 smithi100 ceph-mon[35193]: pgmap v263: 129 pgs: 129 active+clean; 260 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 9 op/s 2024-03-21T10:33:51.487 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:33:51 smithi157 ceph-mon[35581]: pgmap v263: 129 pgs: 129 active+clean; 260 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 9 op/s 2024-03-21T10:33:51.711 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:33:51 smithi047 ceph-mon[31547]: pgmap v263: 129 pgs: 129 active+clean; 260 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 9 op/s 2024-03-21T10:33:53.477 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:33:53 smithi100 ceph-mon[35193]: pgmap v264: 129 pgs: 129 active+clean; 260 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 13 op/s 2024-03-21T10:33:53.487 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:33:53 smithi157 ceph-mon[35581]: pgmap v264: 129 pgs: 129 active+clean; 260 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 13 op/s 2024-03-21T10:33:53.513 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:33:53 smithi047 ceph-mon[31547]: pgmap v264: 129 pgs: 129 active+clean; 260 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 13 op/s 2024-03-21T10:33:55.487 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:33:55 smithi157 ceph-mon[35581]: pgmap v265: 129 pgs: 129 active+clean; 260 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 9 op/s 2024-03-21T10:33:55.711 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:33:55 smithi047 ceph-mon[31547]: pgmap v265: 129 pgs: 129 active+clean; 260 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 9 op/s 2024-03-21T10:33:55.726 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:33:55 smithi100 ceph-mon[35193]: pgmap v265: 129 pgs: 129 active+clean; 260 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 9 op/s 2024-03-21T10:33:57.487 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:33:57 smithi157 ceph-mon[35581]: pgmap v266: 129 pgs: 129 active+clean; 260 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 13 op/s 2024-03-21T10:33:57.711 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:33:57 smithi047 ceph-mon[31547]: pgmap v266: 129 pgs: 129 active+clean; 260 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 13 op/s 2024-03-21T10:33:57.726 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:33:57 smithi100 ceph-mon[35193]: pgmap v266: 129 pgs: 129 active+clean; 260 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 13 op/s 2024-03-21T10:33:59.711 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:33:59 smithi047 ceph-mon[31547]: pgmap v267: 129 pgs: 129 active+clean; 260 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 8 op/s 2024-03-21T10:33:59.726 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:33:59 smithi100 ceph-mon[35193]: pgmap v267: 129 pgs: 129 active+clean; 260 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 8 op/s 2024-03-21T10:33:59.737 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:33:59 smithi157 ceph-mon[35581]: pgmap v267: 129 pgs: 129 active+clean; 260 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 8 op/s 2024-03-21T10:34:01.711 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:34:01 smithi047 ceph-mon[31547]: pgmap v268: 129 pgs: 129 active+clean; 260 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-21T10:34:01.726 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:34:01 smithi100 ceph-mon[35193]: pgmap v268: 129 pgs: 129 active+clean; 260 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-21T10:34:01.737 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:34:01 smithi157 ceph-mon[35581]: pgmap v268: 129 pgs: 129 active+clean; 260 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-21T10:34:03.711 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:34:03 smithi047 ceph-mon[31547]: pgmap v269: 129 pgs: 129 active+clean; 260 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 44 KiB/s wr, 29 op/s 2024-03-21T10:34:03.726 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:34:03 smithi100 ceph-mon[35193]: pgmap v269: 129 pgs: 129 active+clean; 260 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 44 KiB/s wr, 29 op/s 2024-03-21T10:34:03.737 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:34:03 smithi157 ceph-mon[35581]: pgmap v269: 129 pgs: 129 active+clean; 260 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 44 KiB/s wr, 29 op/s 2024-03-21T10:34:05.711 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:34:05 smithi047 ceph-mon[31547]: pgmap v270: 129 pgs: 129 active+clean; 260 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 25 op/s 2024-03-21T10:34:05.726 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:34:05 smithi100 ceph-mon[35193]: pgmap v270: 129 pgs: 129 active+clean; 260 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 25 op/s 2024-03-21T10:34:05.736 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:34:05 smithi157 ceph-mon[35581]: pgmap v270: 129 pgs: 129 active+clean; 260 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 25 op/s 2024-03-21T10:34:07.711 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:34:07 smithi047 ceph-mon[31547]: pgmap v271: 129 pgs: 129 active+clean; 260 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 44 KiB/s wr, 29 op/s 2024-03-21T10:34:07.726 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:34:07 smithi100 ceph-mon[35193]: pgmap v271: 129 pgs: 129 active+clean; 260 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 44 KiB/s wr, 29 op/s 2024-03-21T10:34:07.737 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:34:07 smithi157 ceph-mon[35581]: pgmap v271: 129 pgs: 129 active+clean; 260 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 44 KiB/s wr, 29 op/s 2024-03-21T10:34:09.711 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:34:09 smithi047 ceph-mon[31547]: pgmap v272: 129 pgs: 129 active+clean; 260 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 44 KiB/s wr, 26 op/s 2024-03-21T10:34:09.726 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:34:09 smithi100 ceph-mon[35193]: pgmap v272: 129 pgs: 129 active+clean; 260 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 44 KiB/s wr, 26 op/s 2024-03-21T10:34:09.736 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:34:09 smithi157 ceph-mon[35581]: pgmap v272: 129 pgs: 129 active+clean; 260 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 44 KiB/s wr, 26 op/s 2024-03-21T10:34:10.711 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:34:10 smithi047 ceph-mon[31547]: osdmap e102: 12 total, 12 up, 12 in 2024-03-21T10:34:10.726 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:34:10 smithi100 ceph-mon[35193]: osdmap e102: 12 total, 12 up, 12 in 2024-03-21T10:34:10.736 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:34:10 smithi157 ceph-mon[35581]: osdmap e102: 12 total, 12 up, 12 in 2024-03-21T10:34:11.638 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:34:11 smithi157 ceph-mon[35581]: pgmap v274: 129 pgs: 129 active+clean; 260 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 53 KiB/s wr, 30 op/s 2024-03-21T10:34:11.711 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:34:11 smithi047 ceph-mon[31547]: pgmap v274: 129 pgs: 129 active+clean; 260 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 53 KiB/s wr, 30 op/s 2024-03-21T10:34:11.726 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:34:11 smithi100 ceph-mon[35193]: pgmap v274: 129 pgs: 129 active+clean; 260 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 53 KiB/s wr, 30 op/s 2024-03-21T10:34:13.711 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:34:13 smithi047 ceph-mon[31547]: pgmap v275: 129 pgs: 129 active+clean; 2.5 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 204 B/s wr, 10 op/s 2024-03-21T10:34:13.726 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:34:13 smithi100 ceph-mon[35193]: pgmap v275: 129 pgs: 129 active+clean; 2.5 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 204 B/s wr, 10 op/s 2024-03-21T10:34:13.737 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:34:13 smithi157 ceph-mon[35581]: pgmap v275: 129 pgs: 129 active+clean; 2.5 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 204 B/s wr, 10 op/s 2024-03-21T10:34:14.854 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-21T10:34:14.855 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-21T10:34:15.075 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:15.077+0000 7f2d3d800640 1 -- 172.21.15.47:0/886738345 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2d38104200 msgr2=0x7f2d38110710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:34:15.076 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:15.077+0000 7f2d3d800640 1 --2- 172.21.15.47:0/886738345 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2d38104200 0x7f2d38110710 secure :-1 s=READY pgs=1172 cs=0 l=1 rev1=1 crypto rx=0x7f2d30009cd0 tx=0x7f2d3002c0e0 comp rx=0 tx=0).stop 2024-03-21T10:34:15.076 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:15.077+0000 7f2d3d800640 1 -- 172.21.15.47:0/886738345 shutdown_connections 2024-03-21T10:34:15.076 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:15.077+0000 7f2d3d800640 1 --2- 172.21.15.47:0/886738345 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2d38104200 0x7f2d38110710 unknown :-1 s=CLOSED pgs=1172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:15.076 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:15.077+0000 7f2d3d800640 1 --2- 172.21.15.47:0/886738345 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2d38103840 0x7f2d38103cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:15.076 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:15.077+0000 7f2d3d800640 1 --2- 172.21.15.47:0/886738345 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2d38109850 0x7f2d38109c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:15.076 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:15.077+0000 7f2d3d800640 1 -- 172.21.15.47:0/886738345 >> 172.21.15.47:0/886738345 conn(0x7f2d380ff310 msgr2=0x7f2d38101730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:34:15.076 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:15.077+0000 7f2d3d800640 1 -- 172.21.15.47:0/886738345 shutdown_connections 2024-03-21T10:34:15.076 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:15.077+0000 7f2d3d800640 1 -- 172.21.15.47:0/886738345 wait complete. 2024-03-21T10:34:15.078 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:15.078+0000 7f2d3d800640 1 Processor -- start 2024-03-21T10:34:15.078 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:15.078+0000 7f2d3d800640 1 -- start start 2024-03-21T10:34:15.078 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:15.078+0000 7f2d3d800640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2d38103840 0x7f2d3819a070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:34:15.078 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:15.078+0000 7f2d3d800640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2d38104200 0x7f2d3819a5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:34:15.079 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:15.078+0000 7f2d3d800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2d38109850 0x7f2d3819e930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:34:15.079 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:15.078+0000 7f2d3d800640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d38113ad0 con 0x7f2d38103840 2024-03-21T10:34:15.079 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:15.078+0000 7f2d3d800640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d38113950 con 0x7f2d38109850 2024-03-21T10:34:15.079 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:15.078+0000 7f2d3d800640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d38113c50 con 0x7f2d38104200 2024-03-21T10:34:15.079 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:15.079+0000 7f2d36a00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2d38103840 0x7f2d3819a070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:34:15.079 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:15.079+0000 7f2d36a00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2d38103840 0x7f2d3819a070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:35052/0 (socket says 172.21.15.47:35052) 2024-03-21T10:34:15.079 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:15.079+0000 7f2d36a00640 1 -- 172.21.15.47:0/1260955758 learned_addr learned my addr 172.21.15.47:0/1260955758 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:34:15.079 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:15.079+0000 7f2d37400640 1 --2- 172.21.15.47:0/1260955758 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2d38109850 0x7f2d3819e930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:34:15.079 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:15.079+0000 7f2d36000640 1 --2- 172.21.15.47:0/1260955758 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2d38104200 0x7f2d3819a5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:34:15.079 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:15.079+0000 7f2d36a00640 1 -- 172.21.15.47:0/1260955758 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2d38104200 msgr2=0x7f2d3819a5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:34:15.079 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:15.079+0000 7f2d36a00640 1 --2- 172.21.15.47:0/1260955758 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2d38104200 0x7f2d3819a5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:15.079 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:15.079+0000 7f2d36a00640 1 -- 172.21.15.47:0/1260955758 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2d38109850 msgr2=0x7f2d3819e930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:34:15.079 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:15.079+0000 7f2d36a00640 1 --2- 172.21.15.47:0/1260955758 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2d38109850 0x7f2d3819e930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:15.079 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:15.079+0000 7f2d36a00640 1 -- 172.21.15.47:0/1260955758 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d3819f060 con 0x7f2d38103840 2024-03-21T10:34:15.080 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:15.079+0000 7f2d37400640 1 --2- 172.21.15.47:0/1260955758 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2d38109850 0x7f2d3819e930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T10:34:15.080 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:15.080+0000 7f2d36a00640 1 --2- 172.21.15.47:0/1260955758 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2d38103840 0x7f2d3819a070 secure :-1 s=READY pgs=1173 cs=0 l=1 rev1=1 crypto rx=0x7f2d2800d950 tx=0x7f2d2800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:34:15.080 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:15.080+0000 7f2d2f400640 1 -- 172.21.15.47:0/1260955758 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d28017070 con 0x7f2d38103840 2024-03-21T10:34:15.080 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:15.080+0000 7f2d3d800640 1 -- 172.21.15.47:0/1260955758 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2d3819f350 con 0x7f2d38103840 2024-03-21T10:34:15.080 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:15.080+0000 7f2d2f400640 1 -- 172.21.15.47:0/1260955758 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f2d28009c30 con 0x7f2d38103840 2024-03-21T10:34:15.080 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:15.080+0000 7f2d2f400640 1 -- 172.21.15.47:0/1260955758 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d2800f620 con 0x7f2d38103840 2024-03-21T10:34:15.080 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:15.080+0000 7f2d3d800640 1 -- 172.21.15.47:0/1260955758 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2d381a6be0 con 0x7f2d38103840 2024-03-21T10:34:15.082 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:15.084+0000 7f2d40405740 1 Processor -- start 2024-03-21T10:34:15.082 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:15.085+0000 7f2d40405740 1 -- start start 2024-03-21T10:34:15.084 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:15.085+0000 7f2d2f400640 1 -- 172.21.15.47:0/1260955758 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 27) v1 ==== 137985+0+0 (secure 0 0 0) 0x7f2d280109a0 con 0x7f2d38103840 2024-03-21T10:34:15.084 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:15.085+0000 7f2d40405740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x565432434c00 0x565432433950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:34:15.084 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:15.085+0000 7f2d40405740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x565432433f20 0x5654324343a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:34:15.084 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:15.085+0000 7f2d40405740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56543243d290 0x56543243f680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:34:15.084 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:15.085+0000 7f2d40405740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x565432398250 con 0x565432433f20 2024-03-21T10:34:15.084 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:15.085+0000 7f2d40405740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x565432340a30 con 0x565432434c00 2024-03-21T10:34:15.084 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:15.085+0000 7f2d40405740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x5654323942f0 con 0x56543243d290 2024-03-21T10:34:15.085 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:15.085+0000 7f2d36a00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x565432433f20 0x5654324343a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:34:15.085 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:15.085+0000 7f2d37400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x565432434c00 0x565432433950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:34:15.085 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:15.085+0000 7f2d36a00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x565432433f20 0x5654324343a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:35066/0 (socket says 172.21.15.47:35066) 2024-03-21T10:34:15.085 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:15.085+0000 7f2d37400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x565432434c00 0x565432433950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:49750/0 (socket says 172.21.15.47:49750) 2024-03-21T10:34:15.085 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:15.086+0000 7f2d36a00640 1 -- 172.21.15.47:0/818267549 learned_addr learned my addr 172.21.15.47:0/818267549 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:34:15.085 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:15.086+0000 7f2d36000640 1 --2- 172.21.15.47:0/818267549 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56543243d290 0x56543243f680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:34:15.085 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:15.086+0000 7f2d37400640 1 -- 172.21.15.47:0/818267549 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56543243d290 msgr2=0x56543243f680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:34:15.085 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:15.086+0000 7f2d37400640 1 --2- 172.21.15.47:0/818267549 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56543243d290 0x56543243f680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:15.085 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:15.086+0000 7f2d37400640 1 -- 172.21.15.47:0/818267549 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x565432433f20 msgr2=0x5654324343a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:34:15.085 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:15.086+0000 7f2d37400640 1 --2- 172.21.15.47:0/818267549 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x565432433f20 0x5654324343a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:15.085 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:15.086+0000 7f2d37400640 1 -- 172.21.15.47:0/818267549 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56543232b690 con 0x565432434c00 2024-03-21T10:34:15.085 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:15.086+0000 7f2d36a00640 1 --2- 172.21.15.47:0/818267549 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x565432433f20 0x5654324343a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T10:34:15.089 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:15.086+0000 7f2d37400640 1 --2- 172.21.15.47:0/818267549 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x565432434c00 0x565432433950 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f2d30037040 tx=0x7f2d300323d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4d609a884d01daed server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:34:15.089 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:15.087+0000 7f2d2f400640 1 --2- 172.21.15.47:0/1260955758 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f2d1c0a3150 0x7f2d1c0a5610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:34:15.089 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:15.087+0000 7f2d2f400640 1 -- 172.21.15.47:0/1260955758 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 9562+0+0 (secure 0 0 0) 0x7f2d280cf830 con 0x7f2d38103840 2024-03-21T10:34:15.089 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:15.087+0000 7f2d23e00640 1 -- 172.21.15.47:0/818267549 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d30043020 con 0x565432434c00 2024-03-21T10:34:15.089 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:15.087+0000 7f2d36a00640 1 --2- 172.21.15.47:0/1260955758 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f2d1c0a3150 0x7f2d1c0a5610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:34:15.089 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:15.087+0000 7f2d23e00640 1 -- 172.21.15.47:0/818267549 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f2d300329a0 con 0x565432434c00 2024-03-21T10:34:15.089 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:15.087+0000 7f2d23e00640 1 -- 172.21.15.47:0/818267549 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d30039040 con 0x565432434c00 2024-03-21T10:34:15.089 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:15.088+0000 7f2d40405740 1 -- 172.21.15.47:0/818267549 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x565432434c00 msgr2=0x565432433950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:34:15.089 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:15.088+0000 7f2d40405740 1 --2- 172.21.15.47:0/818267549 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x565432434c00 0x565432433950 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f2d30037040 tx=0x7f2d300323d0 comp rx=0 tx=0).stop 2024-03-21T10:34:15.089 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:15.088+0000 7f2d36a00640 1 --2- 172.21.15.47:0/1260955758 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f2d1c0a3150 0x7f2d1c0a5610 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f2d280cfad0 tx=0x7f2d280d5530 comp rx=0 tx=0).ready entity=mgr.15687 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:34:15.089 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:15.088+0000 7f2d40405740 1 -- 172.21.15.47:0/818267549 shutdown_connections 2024-03-21T10:34:15.090 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:15.088+0000 7f2d40405740 1 --2- 172.21.15.47:0/818267549 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56543243d290 0x56543243f680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:15.090 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:15.088+0000 7f2d40405740 1 --2- 172.21.15.47:0/818267549 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x565432433f20 0x5654324343a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:15.090 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:15.088+0000 7f2d40405740 1 --2- 172.21.15.47:0/818267549 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x565432434c00 0x565432433950 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:15.090 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:15.088+0000 7f2d40405740 1 -- 172.21.15.47:0/818267549 >> 172.21.15.47:0/818267549 conn(0x5654323fb200 msgr2=0x56543242e970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:34:15.090 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:15.088+0000 7f2d40405740 1 -- 172.21.15.47:0/818267549 shutdown_connections 2024-03-21T10:34:15.090 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:15.088+0000 7f2d40405740 1 -- 172.21.15.47:0/818267549 wait complete. 2024-03-21T10:34:15.090 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:15.088+0000 7f2d40405740 1 -- start start 2024-03-21T10:34:15.090 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:15.088+0000 7f2d40405740 1 Processor -- start 2024-03-21T10:34:15.090 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:15.089+0000 7f2d40405740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x565432433f20 0x56543243c990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:34:15.090 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:15.089+0000 7f2d40405740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x565432434c00 0x565432439d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:34:15.090 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:15.089+0000 7f2d40405740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56543243d290 0x56543243a260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:34:15.090 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:15.089+0000 7f2d40405740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x5654324400a0 con 0x565432433f20 2024-03-21T10:34:15.090 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:15.089+0000 7f2d40405740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x565432353e80 con 0x565432434c00 2024-03-21T10:34:15.091 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:15.089+0000 7f2d40405740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x565432440220 con 0x56543243d290 2024-03-21T10:34:15.091 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:15.089+0000 7f2d40405740 10 client.0 _ll_register_callbacks cb 0x565432366130 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T10:34:15.091 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:15.089+0000 7f2d37400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x565432433f20 0x56543243c990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:34:15.091 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:15.089+0000 7f2d36000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x565432434c00 0x565432439d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:34:15.091 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:15.089+0000 7f2d37400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x565432433f20 0x56543243c990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:35076/0 (socket says 172.21.15.47:35076) 2024-03-21T10:34:15.091 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:15.089+0000 7f2d37400640 1 -- 172.21.15.47:0/2536212442 learned_addr learned my addr 172.21.15.47:0/2536212442 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:34:15.091 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:15.090+0000 7f2d37400640 1 -- 172.21.15.47:0/2536212442 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56543243d290 msgr2=0x56543243a260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:34:15.091 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:15.090+0000 7f2d37400640 1 --2- 172.21.15.47:0/2536212442 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56543243d290 0x56543243a260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:15.091 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:15.090+0000 7f2d37400640 1 -- 172.21.15.47:0/2536212442 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x565432434c00 msgr2=0x565432439d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:34:15.091 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:15.090+0000 7f2d37400640 1 --2- 172.21.15.47:0/2536212442 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x565432434c00 0x565432439d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:15.091 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:15.090+0000 7f2d37400640 1 -- 172.21.15.47:0/2536212442 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5654321767d0 con 0x565432433f20 2024-03-21T10:34:15.091 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:15.090+0000 7f2d37400640 1 --2- 172.21.15.47:0/2536212442 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x565432433f20 0x56543243c990 secure :-1 s=READY pgs=1174 cs=0 l=1 rev1=1 crypto rx=0x7f2d300099a0 tx=0x7f2d300099d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8d42f3c6981b6752 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:34:15.091 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:15.090+0000 7f2d20c00640 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-03-21T10:34:15.091 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:15.090+0000 7f2d20c00640 1 -- 172.21.15.47:0/2536212442 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d3004c070 con 0x565432433f20 2024-03-21T10:34:15.092 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:15.090+0000 7f2d20c00640 1 -- 172.21.15.47:0/2536212442 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(103..103 src has 1..103) v4 ==== 9562+0+0 (secure 0 0 0) 0x7f2d3003f470 con 0x565432433f20 2024-03-21T10:34:15.092 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:15.090+0000 7f2d40405740 1 -- 172.21.15.47:0/2536212442 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f2d300329a0 con 0x565432433f20 2024-03-21T10:34:15.092 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:15.091+0000 7f2d20c00640 1 -- 172.21.15.47:0/2536212442 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2d30058050 con 0x565432433f20 2024-03-21T10:34:15.092 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:15.091+0000 7f2d40405740 10 client.18294 fetch_fsmap learned FSMap version 32 2024-03-21T10:34:15.092 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:15.091+0000 7f2d40405740 1 -- 172.21.15.47:0/2536212442 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x5654320ac3f0 con 0x565432433f20 2024-03-21T10:34:15.094 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:15.091+0000 7f2d20c00640 1 -- 172.21.15.47:0/2536212442 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f2d300329a0 con 0x565432433f20 2024-03-21T10:34:15.095 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:15.091+0000 7f2d20c00640 1 -- 172.21.15.47:0/2536212442 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 32) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f2d30034600 con 0x565432433f20 2024-03-21T10:34:15.095 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:15.091+0000 7f2d40405740 10 client.18294 fetch_fsmap finished waiting for FSMap version 32 2024-03-21T10:34:15.095 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:15.091+0000 7f2d40405740 10 client.18294 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-21T10:34:15.095 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:15.091+0000 7f2d40405740 20 client.18294 populate_metadata read hostname 'smithi047' 2024-03-21T10:34:15.095 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:15.091+0000 7f2d40405740 1 --2- 172.21.15.47:0/2536212442 >> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] conn(0x56543244cfe0 0x56543244f3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:34:15.095 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:15.092+0000 7f2d40405740 4 client.18294 mds_command: new command op to 34535 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T10:34:15.095 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:15.092+0000 7f2d40405740 1 -- 172.21.15.47:0/2536212442 --> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5654323a5790 con 0x56543244cfe0 2024-03-21T10:34:15.095 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:15.092+0000 7f2d36000640 1 --2- 172.21.15.47:0/2536212442 >> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] conn(0x56543244cfe0 0x56543244f3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:34:15.095 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:15.092+0000 7f2d36000640 1 --2- 172.21.15.47:0/2536212442 >> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] conn(0x56543244cfe0 0x56543244f3d0 crc :-1 s=READY pgs=199 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=334acd581ab41b59 server_cookie=21a92dda4d5e74d7 in_seq=0 out_seq=0 2024-03-21T10:34:15.095 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:15.092+0000 7f2d20c00640 10 client.18294 ms_handle_connect on v2:172.21.15.100:6838/4173168440 2024-03-21T10:34:15.095 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:15.093+0000 7f2d20c00640 1 -- 172.21.15.47:0/2536212442 <== mds.0 v2:172.21.15.100:6838/4173168440 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x5654323a5790 con 0x56543244cfe0 2024-03-21T10:34:15.095 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:15.093+0000 7f2d20c00640 10 client.18294 handle_command_reply: tid=0 2024-03-21T10:34:15.096 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:15.093+0000 7f2d40405740 1 client.18294 shutdown 2024-03-21T10:34:15.096 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:15.094+0000 7f2d40405740 1 -- 172.21.15.47:0/2536212442 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x565432433f20 msgr2=0x56543243c990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:34:15.096 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:15.094+0000 7f2d40405740 1 --2- 172.21.15.47:0/2536212442 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x565432433f20 0x56543243c990 secure :-1 s=READY pgs=1174 cs=0 l=1 rev1=1 crypto rx=0x7f2d300099a0 tx=0x7f2d300099d0 comp rx=0 tx=0).stop 2024-03-21T10:34:15.096 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:15.094+0000 7f2d40405740 1 -- 172.21.15.47:0/2536212442 shutdown_connections 2024-03-21T10:34:15.096 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:15.094+0000 7f2d40405740 1 --2- 172.21.15.47:0/2536212442 >> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] conn(0x56543244cfe0 0x56543244f3d0 crc :-1 s=READY pgs=199 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:15.096 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:15.094+0000 7f2d40405740 1 --2- 172.21.15.47:0/2536212442 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56543243d290 0x56543243a260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:15.096 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:15.094+0000 7f2d40405740 1 --2- 172.21.15.47:0/2536212442 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x565432434c00 0x565432439d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:15.096 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:15.094+0000 7f2d40405740 1 --2- 172.21.15.47:0/2536212442 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x565432433f20 0x56543243c990 unknown :-1 s=CLOSED pgs=1174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:15.096 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:15.094+0000 7f2d20c00640 0 client.18294 ms_handle_reset on v2:172.21.15.100:6838/4173168440 2024-03-21T10:34:15.096 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:15.094+0000 7f2d40405740 1 -- 172.21.15.47:0/2536212442 >> 172.21.15.47:0/2536212442 conn(0x5654323fb200 msgr2=0x5654324306c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:34:15.096 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:15.094+0000 7f2d40405740 1 -- 172.21.15.47:0/2536212442 shutdown_connections 2024-03-21T10:34:15.096 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:15.094+0000 7f2d40405740 1 -- 172.21.15.47:0/2536212442 wait complete. 2024-03-21T10:34:15.096 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:15.094+0000 7f2d40405740 20 client.18294 trim_cache size 0 max 16384 2024-03-21T10:34:15.111 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:15.112+0000 7f2d40405740 1 Processor -- start 2024-03-21T10:34:15.111 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:15.113+0000 7f2d40405740 1 -- start start 2024-03-21T10:34:15.112 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:15.113+0000 7f2d40405740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x565432433f20 0x56543243c450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:34:15.112 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:15.113+0000 7f2d40405740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x565432434c00 0x56543243c990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:34:15.112 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:15.113+0000 7f2d40405740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56543243d290 0x56543243e680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:34:15.112 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:15.113+0000 7f2d40405740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d381b0be0 con 0x565432433f20 2024-03-21T10:34:15.112 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:15.113+0000 7f2d40405740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d381b2560 con 0x565432434c00 2024-03-21T10:34:15.112 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:15.113+0000 7f2d40405740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d381b1240 con 0x56543243d290 2024-03-21T10:34:15.112 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:15.113+0000 7f2d37400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x565432433f20 0x56543243c450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:34:15.112 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:15.113+0000 7f2d36000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x565432434c00 0x56543243c990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:34:15.112 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:15.113+0000 7f2d37400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x565432433f20 0x56543243c450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:35086/0 (socket says 172.21.15.47:35086) 2024-03-21T10:34:15.112 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:15.113+0000 7f2d37400640 1 -- 172.21.15.47:0/4039956575 learned_addr learned my addr 172.21.15.47:0/4039956575 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:34:15.112 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:15.114+0000 7f2d36000640 1 -- 172.21.15.47:0/4039956575 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56543243d290 msgr2=0x56543243e680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:34:15.113 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:15.114+0000 7f2d36000640 1 --2- 172.21.15.47:0/4039956575 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56543243d290 0x56543243e680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:15.113 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:15.114+0000 7f2d36000640 1 -- 172.21.15.47:0/4039956575 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x565432433f20 msgr2=0x56543243c450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:34:15.113 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:15.114+0000 7f2d36000640 1 --2- 172.21.15.47:0/4039956575 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x565432433f20 0x56543243c450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:15.113 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:15.114+0000 7f2d36000640 1 -- 172.21.15.47:0/4039956575 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56543206c0f0 con 0x565432434c00 2024-03-21T10:34:15.113 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:15.114+0000 7f2d37400640 1 --2- 172.21.15.47:0/4039956575 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x565432433f20 0x56543243c450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T10:34:15.115 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:15.114+0000 7f2d36000640 1 --2- 172.21.15.47:0/4039956575 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x565432434c00 0x56543243c990 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x565432440250 tx=0x7f2d2401ea50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bedee914bdb7f00c server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:34:15.115 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:15.114+0000 7f2d23400640 1 -- 172.21.15.47:0/4039956575 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d24004230 con 0x565432434c00 2024-03-21T10:34:15.115 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:15.115+0000 7f2d23400640 1 -- 172.21.15.47:0/4039956575 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f2d240043d0 con 0x565432434c00 2024-03-21T10:34:15.115 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:15.115+0000 7f2d23400640 1 -- 172.21.15.47:0/4039956575 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d24012800 con 0x565432434c00 2024-03-21T10:34:15.115 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:15.115+0000 7f2d40405740 1 -- 172.21.15.47:0/4039956575 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x565432434c00 msgr2=0x56543243c990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:34:15.115 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:15.115+0000 7f2d40405740 1 --2- 172.21.15.47:0/4039956575 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x565432434c00 0x56543243c990 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x565432440250 tx=0x7f2d2401ea50 comp rx=0 tx=0).stop 2024-03-21T10:34:15.115 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:15.115+0000 7f2d40405740 1 -- 172.21.15.47:0/4039956575 shutdown_connections 2024-03-21T10:34:15.115 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:15.115+0000 7f2d40405740 1 --2- 172.21.15.47:0/4039956575 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56543243d290 0x56543243e680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:15.115 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:15.115+0000 7f2d40405740 1 --2- 172.21.15.47:0/4039956575 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x565432434c00 0x56543243c990 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:15.115 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:15.115+0000 7f2d40405740 1 --2- 172.21.15.47:0/4039956575 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x565432433f20 0x56543243c450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:15.115 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:15.115+0000 7f2d40405740 1 -- 172.21.15.47:0/4039956575 >> 172.21.15.47:0/4039956575 conn(0x5654323fb200 msgr2=0x56543242e4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:34:15.116 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:15.115+0000 7f2d40405740 1 -- 172.21.15.47:0/4039956575 shutdown_connections 2024-03-21T10:34:15.116 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:15.116+0000 7f2d40405740 1 -- 172.21.15.47:0/4039956575 wait complete. 2024-03-21T10:34:15.116 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:15.116+0000 7f2d40405740 1 -- start start 2024-03-21T10:34:15.116 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:15.116+0000 7f2d40405740 1 Processor -- start 2024-03-21T10:34:15.116 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:15.116+0000 7f2d40405740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x565432433f20 0x5654324365b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:34:15.116 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:15.116+0000 7f2d40405740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x565432434c00 0x565432436af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:34:15.116 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:15.116+0000 7f2d40405740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56543243d290 0x565432437030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:34:15.116 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:15.116+0000 7f2d40405740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d381b1f00 con 0x565432433f20 2024-03-21T10:34:15.116 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:15.117+0000 7f2d40405740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d381b18a0 con 0x565432434c00 2024-03-21T10:34:15.116 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:15.117+0000 7f2d40405740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d381aedc0 con 0x56543243d290 2024-03-21T10:34:15.116 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:15.117+0000 7f2d40405740 10 client.0 _ll_register_callbacks cb 0x7f2d280d5400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T10:34:15.116 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:15.117+0000 7f2d37400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x565432433f20 0x5654324365b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:34:15.116 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:15.117+0000 7f2d37400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x565432433f20 0x5654324365b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:35088/0 (socket says 172.21.15.47:35088) 2024-03-21T10:34:15.116 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:15.117+0000 7f2d37400640 1 -- 172.21.15.47:0/2614734573 learned_addr learned my addr 172.21.15.47:0/2614734573 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:34:15.116 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:15.117+0000 7f2d36000640 1 --2- 172.21.15.47:0/2614734573 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x565432434c00 0x565432436af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:34:15.119 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:15.117+0000 7f2d37400640 1 -- 172.21.15.47:0/2614734573 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56543243d290 msgr2=0x565432437030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:34:15.119 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:15.117+0000 7f2d37400640 1 --2- 172.21.15.47:0/2614734573 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56543243d290 0x565432437030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:15.119 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:15.117+0000 7f2d37400640 1 -- 172.21.15.47:0/2614734573 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x565432434c00 msgr2=0x565432436af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:34:15.119 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:15.117+0000 7f2d37400640 1 --2- 172.21.15.47:0/2614734573 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x565432434c00 0x565432436af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:15.119 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:15.117+0000 7f2d37400640 1 -- 172.21.15.47:0/2614734573 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5654324a7580 con 0x565432433f20 2024-03-21T10:34:15.119 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:15.118+0000 7f2d37400640 1 --2- 172.21.15.47:0/2614734573 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x565432433f20 0x5654324365b0 secure :-1 s=READY pgs=1175 cs=0 l=1 rev1=1 crypto rx=0x7f2d30009a60 tx=0x7f2d30034bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b40941274706d1c8 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:34:15.119 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:15.118+0000 7f2d20c00640 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-03-21T10:34:15.119 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:15.118+0000 7f2d20c00640 1 -- 172.21.15.47:0/2614734573 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d3004c040 con 0x565432433f20 2024-03-21T10:34:15.120 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:15.118+0000 7f2d20c00640 1 -- 172.21.15.47:0/2614734573 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(103..103 src has 1..103) v4 ==== 9562+0+0 (secure 0 0 0) 0x7f2d30034650 con 0x565432433f20 2024-03-21T10:34:15.120 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:15.118+0000 7f2d40405740 1 -- 172.21.15.47:0/2614734573 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f2d240043d0 con 0x565432433f20 2024-03-21T10:34:15.120 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:15.118+0000 7f2d20c00640 1 -- 172.21.15.47:0/2614734573 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2d30064050 con 0x565432433f20 2024-03-21T10:34:15.120 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:15.118+0000 7f2d40405740 10 client.18300 fetch_fsmap learned FSMap version 32 2024-03-21T10:34:15.120 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:15.118+0000 7f2d40405740 1 -- 172.21.15.47:0/2614734573 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x5654324a8890 con 0x565432433f20 2024-03-21T10:34:15.120 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:15.118+0000 7f2d20c00640 1 -- 172.21.15.47:0/2614734573 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f2d240043d0 con 0x565432433f20 2024-03-21T10:34:15.120 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:15.119+0000 7f2d20c00640 1 -- 172.21.15.47:0/2614734573 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 32) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f2d3003f7a0 con 0x565432433f20 2024-03-21T10:34:15.120 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:15.119+0000 7f2d40405740 10 client.18300 fetch_fsmap finished waiting for FSMap version 32 2024-03-21T10:34:15.120 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:15.119+0000 7f2d40405740 10 client.18300 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-21T10:34:15.120 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:15.119+0000 7f2d40405740 20 client.18300 populate_metadata read hostname 'smithi047' 2024-03-21T10:34:15.120 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:15.119+0000 7f2d40405740 1 --2- 172.21.15.47:0/2614734573 >> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] conn(0x5654324a8e80 0x5654324ab2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:34:15.120 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:15.119+0000 7f2d40405740 4 client.18300 mds_command: new command op to 34535 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-21T10:34:15.120 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:15.119+0000 7f2d40405740 1 -- 172.21.15.47:0/2614734573 --> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x56543239fae0 con 0x5654324a8e80 2024-03-21T10:34:15.120 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:15.119+0000 7f2d36000640 1 --2- 172.21.15.47:0/2614734573 >> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] conn(0x5654324a8e80 0x5654324ab2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:34:15.121 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:15.120+0000 7f2d36000640 1 --2- 172.21.15.47:0/2614734573 >> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] conn(0x5654324a8e80 0x5654324ab2a0 crc :-1 s=READY pgs=200 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a24d761a0dd62530 server_cookie=74333257c99154e0 in_seq=0 out_seq=0 2024-03-21T10:34:15.121 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:15.120+0000 7f2d20c00640 10 client.18300 ms_handle_connect on v2:172.21.15.100:6838/4173168440 2024-03-21T10:34:15.121 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:15.120+0000 7f2d20c00640 1 -- 172.21.15.47:0/2614734573 <== mds.0 v2:172.21.15.100:6838/4173168440 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x56543239fae0 con 0x5654324a8e80 2024-03-21T10:34:15.121 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:15.120+0000 7f2d20c00640 10 client.18300 handle_command_reply: tid=0 2024-03-21T10:34:15.121 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:15.120+0000 7f2d40405740 1 client.18300 shutdown 2024-03-21T10:34:15.121 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:15.121+0000 7f2d40405740 1 -- 172.21.15.47:0/2614734573 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x565432433f20 msgr2=0x5654324365b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:34:15.121 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:15.121+0000 7f2d40405740 1 --2- 172.21.15.47:0/2614734573 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x565432433f20 0x5654324365b0 secure :-1 s=READY pgs=1175 cs=0 l=1 rev1=1 crypto rx=0x7f2d30009a60 tx=0x7f2d30034bb0 comp rx=0 tx=0).stop 2024-03-21T10:34:15.121 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:15.121+0000 7f2d40405740 1 -- 172.21.15.47:0/2614734573 shutdown_connections 2024-03-21T10:34:15.121 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:15.121+0000 7f2d40405740 1 --2- 172.21.15.47:0/2614734573 >> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] conn(0x5654324a8e80 0x5654324ab2a0 crc :-1 s=READY pgs=200 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:15.121 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:15.121+0000 7f2d40405740 1 --2- 172.21.15.47:0/2614734573 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56543243d290 0x565432437030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:15.121 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:15.121+0000 7f2d40405740 1 --2- 172.21.15.47:0/2614734573 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x565432434c00 0x565432436af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:15.121 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:15.121+0000 7f2d40405740 1 --2- 172.21.15.47:0/2614734573 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x565432433f20 0x5654324365b0 unknown :-1 s=CLOSED pgs=1175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:15.121 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:15.121+0000 7f2d20c00640 0 client.18300 ms_handle_reset on v2:172.21.15.100:6838/4173168440 2024-03-21T10:34:15.121 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:15.121+0000 7f2d40405740 1 -- 172.21.15.47:0/2614734573 >> 172.21.15.47:0/2614734573 conn(0x5654323fb200 msgr2=0x565432438610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:34:15.121 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:15.121+0000 7f2d40405740 1 -- 172.21.15.47:0/2614734573 shutdown_connections 2024-03-21T10:34:15.122 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:15.121+0000 7f2d40405740 1 -- 172.21.15.47:0/2614734573 wait complete. 2024-03-21T10:34:15.122 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:15.121+0000 7f2d40405740 20 client.18300 trim_cache size 0 max 16384 2024-03-21T10:34:15.122 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-03-21T10:34:15.122 INFO:teuthology.orchestra.run.smithi047.stdout: "status": "no active scrubs running", 2024-03-21T10:34:15.122 INFO:teuthology.orchestra.run.smithi047.stdout: "scrubs": {} 2024-03-21T10:34:15.122 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-03-21T10:34:15.123 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:15.122+0000 7f2d3d800640 1 -- 172.21.15.47:0/1260955758 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f2d1c0a3150 msgr2=0x7f2d1c0a5610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:34:15.123 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:15.122+0000 7f2d3d800640 1 --2- 172.21.15.47:0/1260955758 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f2d1c0a3150 0x7f2d1c0a5610 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f2d280cfad0 tx=0x7f2d280d5530 comp rx=0 tx=0).stop 2024-03-21T10:34:15.123 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:15.122+0000 7f2d3d800640 1 -- 172.21.15.47:0/1260955758 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2d38103840 msgr2=0x7f2d3819a070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:34:15.123 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:15.122+0000 7f2d3d800640 1 --2- 172.21.15.47:0/1260955758 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2d38103840 0x7f2d3819a070 secure :-1 s=READY pgs=1173 cs=0 l=1 rev1=1 crypto rx=0x7f2d2800d950 tx=0x7f2d2800de10 comp rx=0 tx=0).stop 2024-03-21T10:34:15.123 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:15.122+0000 7f2d3d800640 1 -- 172.21.15.47:0/1260955758 shutdown_connections 2024-03-21T10:34:15.123 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:15.122+0000 7f2d3d800640 1 --2- 172.21.15.47:0/1260955758 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f2d1c0a3150 0x7f2d1c0a5610 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:15.123 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:15.122+0000 7f2d3d800640 1 --2- 172.21.15.47:0/1260955758 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2d38109850 0x7f2d3819e930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:15.123 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:15.122+0000 7f2d3d800640 1 --2- 172.21.15.47:0/1260955758 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2d38104200 0x7f2d3819a5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:15.123 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:15.122+0000 7f2d3d800640 1 --2- 172.21.15.47:0/1260955758 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2d38103840 0x7f2d3819a070 unknown :-1 s=CLOSED pgs=1173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:15.123 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:15.122+0000 7f2d3d800640 1 -- 172.21.15.47:0/1260955758 >> 172.21.15.47:0/1260955758 conn(0x7f2d380ff310 msgr2=0x7f2d380ffb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:34:15.124 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:15.122+0000 7f2d3d800640 1 -- 172.21.15.47:0/1260955758 shutdown_connections 2024-03-21T10:34:15.124 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:15.122+0000 7f2d3d800640 1 -- 172.21.15.47:0/1260955758 wait complete. 2024-03-21T10:34:15.135 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:34:14 smithi047 ceph-mon[31547]: osdmap e103: 12 total, 12 up, 12 in 2024-03-21T10:34:15.135 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:34:14 smithi047 ceph-mon[31547]: pgmap v277: 129 pgs: 129 active+clean; 2.5 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 127 B/s wr, 7 op/s 2024-03-21T10:34:15.140 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-03-21T10:34:15.140 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-21T10:34:15.226 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:34:14 smithi100 ceph-mon[35193]: osdmap e103: 12 total, 12 up, 12 in 2024-03-21T10:34:15.226 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:34:14 smithi100 ceph-mon[35193]: pgmap v277: 129 pgs: 129 active+clean; 2.5 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 127 B/s wr, 7 op/s 2024-03-21T10:34:15.237 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:34:14 smithi157 ceph-mon[35581]: osdmap e103: 12 total, 12 up, 12 in 2024-03-21T10:34:15.237 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:34:14 smithi157 ceph-mon[35581]: pgmap v277: 129 pgs: 129 active+clean; 2.5 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 127 B/s wr, 7 op/s 2024-03-21T10:34:15.350 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:15.351+0000 7fcbf3200640 1 -- 172.21.15.47:0/95484901 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fcbf41014b0 msgr2=0x7fcbf4101930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:34:15.350 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:15.351+0000 7fcbf3200640 1 --2- 172.21.15.47:0/95484901 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fcbf41014b0 0x7fcbf4101930 secure :-1 s=READY pgs=1176 cs=0 l=1 rev1=1 crypto rx=0x7fcbe0009900 tx=0x7fcbe002bc30 comp rx=0 tx=0).stop 2024-03-21T10:34:15.350 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:15.351+0000 7fcbf3200640 1 -- 172.21.15.47:0/95484901 shutdown_connections 2024-03-21T10:34:15.350 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:15.351+0000 7fcbf3200640 1 --2- 172.21.15.47:0/95484901 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcbf4101e70 0x7fcbf410e330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:15.350 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:15.351+0000 7fcbf3200640 1 --2- 172.21.15.47:0/95484901 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fcbf41014b0 0x7fcbf4101930 unknown :-1 s=CLOSED pgs=1176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:15.350 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:15.351+0000 7fcbf3200640 1 --2- 172.21.15.47:0/95484901 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcbf4107570 0x7fcbf4107950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:15.350 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:15.351+0000 7fcbf3200640 1 -- 172.21.15.47:0/95484901 >> 172.21.15.47:0/95484901 conn(0x7fcbf40fcf80 msgr2=0x7fcbf40ff3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:34:15.350 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:15.351+0000 7fcbf3200640 1 -- 172.21.15.47:0/95484901 shutdown_connections 2024-03-21T10:34:15.351 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:15.352+0000 7fcbf3200640 1 -- 172.21.15.47:0/95484901 wait complete. 2024-03-21T10:34:15.353 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:15.352+0000 7fcbf3200640 1 Processor -- start 2024-03-21T10:34:15.353 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:15.353+0000 7fcbf3200640 1 -- start start 2024-03-21T10:34:15.353 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:15.353+0000 7fcbf3200640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fcbf41014b0 0x7fcbf40739a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:34:15.353 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:15.353+0000 7fcbf3200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcbf4101e70 0x7fcbf4073ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:34:15.353 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:15.353+0000 7fcbf3200640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcbf4107570 0x7fcbf4074420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:34:15.353 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:15.353+0000 7fcbf3200640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcbf4111750 con 0x7fcbf41014b0 2024-03-21T10:34:15.353 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:15.353+0000 7fcbf3200640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcbf41115d0 con 0x7fcbf4101e70 2024-03-21T10:34:15.353 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:15.353+0000 7fcbf3200640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcbf41118d0 con 0x7fcbf4107570 2024-03-21T10:34:15.353 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:15.353+0000 7fcbf1e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fcbf41014b0 0x7fcbf40739a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:34:15.353 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:15.353+0000 7fcbf1e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fcbf41014b0 0x7fcbf40739a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:35106/0 (socket says 172.21.15.47:35106) 2024-03-21T10:34:15.353 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:15.353+0000 7fcbf1e00640 1 -- 172.21.15.47:0/3450661005 learned_addr learned my addr 172.21.15.47:0/3450661005 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:34:15.353 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:15.353+0000 7fcbf1400640 1 --2- 172.21.15.47:0/3450661005 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcbf4101e70 0x7fcbf4073ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:34:15.354 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:15.354+0000 7fcbf2800640 1 --2- 172.21.15.47:0/3450661005 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcbf4107570 0x7fcbf4074420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:34:15.354 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:15.354+0000 7fcbf1e00640 1 -- 172.21.15.47:0/3450661005 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcbf4107570 msgr2=0x7fcbf4074420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:34:15.354 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:15.354+0000 7fcbf1e00640 1 --2- 172.21.15.47:0/3450661005 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcbf4107570 0x7fcbf4074420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:15.354 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:15.354+0000 7fcbf1e00640 1 -- 172.21.15.47:0/3450661005 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcbf4101e70 msgr2=0x7fcbf4073ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:34:15.354 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:15.354+0000 7fcbf1e00640 1 --2- 172.21.15.47:0/3450661005 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcbf4101e70 0x7fcbf4073ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:15.354 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:15.354+0000 7fcbf1e00640 1 -- 172.21.15.47:0/3450661005 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcbf41a6150 con 0x7fcbf41014b0 2024-03-21T10:34:15.354 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:15.354+0000 7fcbf1e00640 1 --2- 172.21.15.47:0/3450661005 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fcbf41014b0 0x7fcbf40739a0 secure :-1 s=READY pgs=1177 cs=0 l=1 rev1=1 crypto rx=0x7fcbe400d770 tx=0x7fcbe400dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:34:15.354 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:15.355+0000 7fcbeaa00640 1 -- 172.21.15.47:0/3450661005 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbe4017070 con 0x7fcbf41014b0 2024-03-21T10:34:15.354 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:15.355+0000 7fcbf3200640 1 -- 172.21.15.47:0/3450661005 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcbf41a6440 con 0x7fcbf41014b0 2024-03-21T10:34:15.354 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:15.355+0000 7fcbeaa00640 1 -- 172.21.15.47:0/3450661005 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fcbe40054d0 con 0x7fcbf41014b0 2024-03-21T10:34:15.355 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:15.355+0000 7fcbf3200640 1 -- 172.21.15.47:0/3450661005 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcbf41a6960 con 0x7fcbf41014b0 2024-03-21T10:34:15.355 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:15.355+0000 7fcbeaa00640 1 -- 172.21.15.47:0/3450661005 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbe4020060 con 0x7fcbf41014b0 2024-03-21T10:34:15.355 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:15.356+0000 7fcbf3200640 1 -- 172.21.15.47:0/3450661005 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcbf4102bd0 con 0x7fcbf41014b0 2024-03-21T10:34:15.356 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:15.358+0000 7fcbeaa00640 1 -- 172.21.15.47:0/3450661005 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 27) v1 ==== 137985+0+0 (secure 0 0 0) 0x7fcbe4004c90 con 0x7fcbf41014b0 2024-03-21T10:34:15.358 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:15.361+0000 7fcbeaa00640 1 --2- 172.21.15.47:0/3450661005 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7fcbd80a3090 0x7fcbd80a5550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:34:15.359 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:15.361+0000 7fcbeaa00640 1 -- 172.21.15.47:0/3450661005 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 9562+0+0 (secure 0 0 0) 0x7fcbe40cf3a0 con 0x7fcbf41014b0 2024-03-21T10:34:15.359 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:15.361+0000 7fcbf1400640 1 --2- 172.21.15.47:0/3450661005 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7fcbd80a3090 0x7fcbd80a5550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:34:15.360 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:15.362+0000 7fcbf1400640 1 --2- 172.21.15.47:0/3450661005 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7fcbd80a3090 0x7fcbd80a5550 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fcbe0005ab0 tx=0x7fcbe0005a20 comp rx=0 tx=0).ready entity=mgr.15687 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:34:15.363 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:15.365+0000 7fcbeaa00640 1 -- 172.21.15.47:0/3450661005 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fcbe409ba20 con 0x7fcbf41014b0 2024-03-21T10:34:15.656 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:15.658+0000 7fcbf3200640 1 -- 172.21.15.47:0/3450661005 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fcbf40683b0 con 0x7fcbf41014b0 2024-03-21T10:34:15.658 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:15.660+0000 7fcbeaa00640 1 -- 172.21.15.47:0/3450661005 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 32 v32) v1 ==== 94+0+11471 (secure 0 0 0) 0x7fcbe409b320 con 0x7fcbf41014b0 2024-03-21T10:34:15.659 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-21T10:34:15.659 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":32,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14511,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6835/735724037","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6834","nonce":735724037},{"type":"v1","addr":"172.21.15.47:6835","nonce":735724037}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14529,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6837/2248191724","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6836","nonce":2248191724},{"type":"v1","addr":"172.21.15.47:6837","nonce":2248191724}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":32,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-21T10:21:58.329063+0000","modified":"2024-03-21T10:26:17.169892+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":34535,"mds_1":34493,"mds_2":34454,"mds_3":34427,"mds_4":14619},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14604":{"gid":14604,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.47:6839/2881335356","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6838","nonce":2881335356},{"type":"v1","addr":"172.21.15.47:6839","nonce":2881335356}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_14619":{"gid":14619,"name":"a","rank":4,"incarnation":27,"state":"up:active","state_seq":11,"addr":"172.21.15.47:6841/3401578978","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6840","nonce":3401578978},{"type":"v1","addr":"172.21.15.47:6841","nonce":3401578978}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24406":{"gid":24406,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.157:6833/4000010136","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":4000010136},{"type":"v1","addr":"172.21.15.157:6833","nonce":4000010136}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24457":{"gid":24457,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.157:6835/3902868651","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":3902868651},{"type":"v1","addr":"172.21.15.157:6835","nonce":3902868651}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24475":{"gid":24475,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.157:6837/1908087204","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6836","nonce":1908087204},{"type":"v1","addr":"172.21.15.157:6837","nonce":1908087204}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_25729":{"gid":25729,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.157:6839/1998408416","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6838","nonce":1998408416},{"type":"v1","addr":"172.21.15.157:6839","nonce":1998408416}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_34427":{"gid":34427,"name":"k","rank":3,"incarnation":25,"state":"up:active","state_seq":20,"addr":"172.21.15.100:6833/2723302038","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6832","nonce":2723302038},{"type":"v1","addr":"172.21.15.100:6833","nonce":2723302038}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_34454":{"gid":34454,"name":"e","rank":2,"incarnation":22,"state":"up:active","state_seq":17,"addr":"172.21.15.100:6835/2597849957","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":2597849957},{"type":"v1","addr":"172.21.15.100:6835","nonce":2597849957}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_34493":{"gid":34493,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":13,"addr":"172.21.15.100:6837/3497644162","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6836","nonce":3497644162},{"type":"v1","addr":"172.21.15.100:6837","nonce":3497644162}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_34535":{"gid":34535,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.100:6839/4173168440","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6838","nonce":4173168440},{"type":"v1","addr":"172.21.15.100:6839","nonce":4173168440}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":34535,"qdb_cluster":[34427,14619,34454,34535,34493]},"id":1}]} 2024-03-21T10:34:15.659 INFO:teuthology.orchestra.run.smithi047.stderr:dumped fsmap epoch 32 2024-03-21T10:34:15.661 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:15.663+0000 7fcbf3200640 1 -- 172.21.15.47:0/3450661005 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7fcbd80a3090 msgr2=0x7fcbd80a5550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:34:15.661 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:15.663+0000 7fcbf3200640 1 --2- 172.21.15.47:0/3450661005 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7fcbd80a3090 0x7fcbd80a5550 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fcbe0005ab0 tx=0x7fcbe0005a20 comp rx=0 tx=0).stop 2024-03-21T10:34:15.661 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:15.663+0000 7fcbf3200640 1 -- 172.21.15.47:0/3450661005 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fcbf41014b0 msgr2=0x7fcbf40739a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:34:15.661 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:15.663+0000 7fcbf3200640 1 --2- 172.21.15.47:0/3450661005 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fcbf41014b0 0x7fcbf40739a0 secure :-1 s=READY pgs=1177 cs=0 l=1 rev1=1 crypto rx=0x7fcbe400d770 tx=0x7fcbe400dc30 comp rx=0 tx=0).stop 2024-03-21T10:34:15.661 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:15.663+0000 7fcbf3200640 1 -- 172.21.15.47:0/3450661005 shutdown_connections 2024-03-21T10:34:15.662 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:15.663+0000 7fcbf3200640 1 --2- 172.21.15.47:0/3450661005 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcbf4107570 0x7fcbf4074420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:15.662 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:15.663+0000 7fcbf3200640 1 --2- 172.21.15.47:0/3450661005 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7fcbd80a3090 0x7fcbd80a5550 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:15.662 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:15.663+0000 7fcbf3200640 1 --2- 172.21.15.47:0/3450661005 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcbf4101e70 0x7fcbf4073ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:15.662 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:15.663+0000 7fcbf3200640 1 --2- 172.21.15.47:0/3450661005 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fcbf41014b0 0x7fcbf40739a0 unknown :-1 s=CLOSED pgs=1177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:15.662 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:15.663+0000 7fcbf3200640 1 -- 172.21.15.47:0/3450661005 >> 172.21.15.47:0/3450661005 conn(0x7fcbf40fcf80 msgr2=0x7fcbf40fe670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:34:15.662 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:15.663+0000 7fcbf3200640 1 -- 172.21.15.47:0/3450661005 shutdown_connections 2024-03-21T10:34:15.662 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:15.663+0000 7fcbf3200640 1 -- 172.21.15.47:0/3450661005 wait complete. 2024-03-21T10:34:15.681 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-21T10:34:15.681 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-03-21T10:34:15.917 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:15.919+0000 7fa169200640 1 -- 172.21.15.47:0/1480491824 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa1641028e0 msgr2=0x7fa164102cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:34:15.918 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:15.919+0000 7fa169200640 1 --2- 172.21.15.47:0/1480491824 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa1641028e0 0x7fa164102cc0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7fa154009ca0 tx=0x7fa15402c180 comp rx=0 tx=0).stop 2024-03-21T10:34:15.918 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:15.919+0000 7fa169200640 1 -- 172.21.15.47:0/1480491824 shutdown_connections 2024-03-21T10:34:15.918 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:15.919+0000 7fa169200640 1 --2- 172.21.15.47:0/1480491824 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa16410bcb0 0x7fa16410e0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:15.918 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:15.919+0000 7fa169200640 1 --2- 172.21.15.47:0/1480491824 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa164103200 0x7fa16410b6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:15.918 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:15.919+0000 7fa169200640 1 --2- 172.21.15.47:0/1480491824 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa1641028e0 0x7fa164102cc0 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:15.918 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:15.919+0000 7fa169200640 1 -- 172.21.15.47:0/1480491824 >> 172.21.15.47:0/1480491824 conn(0x7fa164076920 msgr2=0x7fa164076d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:34:15.918 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:15.919+0000 7fa169200640 1 -- 172.21.15.47:0/1480491824 shutdown_connections 2024-03-21T10:34:15.918 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:15.919+0000 7fa169200640 1 -- 172.21.15.47:0/1480491824 wait complete. 2024-03-21T10:34:15.920 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:15.920+0000 7fa169200640 1 Processor -- start 2024-03-21T10:34:15.920 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:15.920+0000 7fa169200640 1 -- start start 2024-03-21T10:34:15.920 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:15.921+0000 7fa169200640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa1641028e0 0x7fa16419a090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:34:15.920 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:15.921+0000 7fa169200640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa164103200 0x7fa16419a5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:34:15.920 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:15.921+0000 7fa169200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa16410bcb0 0x7fa16419e950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:34:15.920 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:15.921+0000 7fa169200640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1641118f0 con 0x7fa164103200 2024-03-21T10:34:15.920 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:15.921+0000 7fa169200640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa164111770 con 0x7fa16410bcb0 2024-03-21T10:34:15.920 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:15.921+0000 7fa169200640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa164111a70 con 0x7fa1641028e0 2024-03-21T10:34:15.920 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:15.921+0000 7fa161400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa164103200 0x7fa16419a5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:34:15.920 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:15.921+0000 7fa161400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa164103200 0x7fa16419a5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:35120/0 (socket says 172.21.15.47:35120) 2024-03-21T10:34:15.920 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:15.921+0000 7fa161400640 1 -- 172.21.15.47:0/267531806 learned_addr learned my addr 172.21.15.47:0/267531806 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:34:15.920 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:15.921+0000 7fa162800640 1 --2- 172.21.15.47:0/267531806 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa16410bcb0 0x7fa16419e950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:34:15.920 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:15.921+0000 7fa161e00640 1 --2- 172.21.15.47:0/267531806 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa1641028e0 0x7fa16419a090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:34:15.921 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:15.921+0000 7fa161400640 1 -- 172.21.15.47:0/267531806 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa1641028e0 msgr2=0x7fa16419a090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:34:15.921 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:15.921+0000 7fa161400640 1 --2- 172.21.15.47:0/267531806 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa1641028e0 0x7fa16419a090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:15.921 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:15.921+0000 7fa161400640 1 -- 172.21.15.47:0/267531806 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa16410bcb0 msgr2=0x7fa16419e950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:34:15.921 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:15.921+0000 7fa161400640 1 --2- 172.21.15.47:0/267531806 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa16410bcb0 0x7fa16419e950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:15.921 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:15.922+0000 7fa161400640 1 -- 172.21.15.47:0/267531806 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa16419f080 con 0x7fa164103200 2024-03-21T10:34:15.921 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:15.922+0000 7fa161400640 1 --2- 172.21.15.47:0/267531806 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa164103200 0x7fa16419a5d0 secure :-1 s=READY pgs=1178 cs=0 l=1 rev1=1 crypto rx=0x7fa14c00e9f0 tx=0x7fa14c00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:34:15.922 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:15.922+0000 7fa15aa00640 1 -- 172.21.15.47:0/267531806 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa14c00cd80 con 0x7fa164103200 2024-03-21T10:34:15.922 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:15.922+0000 7fa15aa00640 1 -- 172.21.15.47:0/267531806 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fa14c0108e0 con 0x7fa164103200 2024-03-21T10:34:15.922 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:15.922+0000 7fa169200640 1 -- 172.21.15.47:0/267531806 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa16419f370 con 0x7fa164103200 2024-03-21T10:34:15.922 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:15.922+0000 7fa15aa00640 1 -- 172.21.15.47:0/267531806 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa14c018920 con 0x7fa164103200 2024-03-21T10:34:15.922 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:15.922+0000 7fa169200640 1 -- 172.21.15.47:0/267531806 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa1641a6c00 con 0x7fa164103200 2024-03-21T10:34:15.924 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:15.926+0000 7fa16bfe6740 1 Processor -- start 2024-03-21T10:34:15.924 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:15.926+0000 7fa16bfe6740 1 -- start start 2024-03-21T10:34:15.925 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:15.926+0000 7fa16bfe6740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f7fce9d6d0 0x55f7fce9dab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:34:15.925 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:15.926+0000 7fa16bfe6740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f7fce9e080 0x55f7fcea6630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:34:15.925 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:15.926+0000 7fa16bfe6740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f7fcea6c00 0x55f7fcea9050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:34:15.925 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:15.926+0000 7fa16bfe6740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f7fce012c0 con 0x55f7fce9d6d0 2024-03-21T10:34:15.925 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:15.926+0000 7fa16bfe6740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f7fcda9c90 con 0x55f7fcea6c00 2024-03-21T10:34:15.925 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:15.926+0000 7fa16bfe6740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f7fcdfd360 con 0x55f7fce9e080 2024-03-21T10:34:15.925 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:15.927+0000 7fa162800640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f7fce9d6d0 0x55f7fce9dab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:34:15.926 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:15.927+0000 7fa162800640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f7fce9d6d0 0x55f7fce9dab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:35134/0 (socket says 172.21.15.47:35134) 2024-03-21T10:34:15.926 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:15.927+0000 7fa162800640 1 -- 172.21.15.47:0/747754902 learned_addr learned my addr 172.21.15.47:0/747754902 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:34:15.926 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:15.927+0000 7fa15aa00640 1 -- 172.21.15.47:0/267531806 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 27) v1 ==== 137985+0+0 (secure 0 0 0) 0x7fa14c010a90 con 0x7fa164103200 2024-03-21T10:34:15.926 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:15.927+0000 7fa161e00640 1 --2- 172.21.15.47:0/747754902 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f7fce9e080 0x55f7fcea6630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:34:15.926 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:15.927+0000 7fa162800640 1 -- 172.21.15.47:0/747754902 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f7fce9e080 msgr2=0x55f7fcea6630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:34:15.926 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:15.927+0000 7fa162800640 1 --2- 172.21.15.47:0/747754902 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f7fce9e080 0x55f7fcea6630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:15.926 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:15.927+0000 7fa162800640 1 -- 172.21.15.47:0/747754902 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f7fcea6c00 msgr2=0x55f7fcea9050 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-03-21T10:34:15.926 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:15.927+0000 7fa162800640 1 --2- 172.21.15.47:0/747754902 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f7fcea6c00 0x55f7fcea9050 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:15.926 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:15.927+0000 7fa162800640 1 -- 172.21.15.47:0/747754902 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f7fcdce080 con 0x55f7fce9d6d0 2024-03-21T10:34:15.926 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:15.927+0000 7fa161e00640 1 --2- 172.21.15.47:0/747754902 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f7fce9e080 0x55f7fcea6630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T10:34:15.928 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:15.928+0000 7fa162800640 1 --2- 172.21.15.47:0/747754902 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f7fce9d6d0 0x55f7fce9dab0 secure :-1 s=READY pgs=1179 cs=0 l=1 rev1=1 crypto rx=0x7fa15c00b600 tx=0x7fa15c00bac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fa0cc8edd287b16c server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:34:15.928 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:15.928+0000 7fa153400640 1 -- 172.21.15.47:0/747754902 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa15c003b20 con 0x55f7fce9d6d0 2024-03-21T10:34:15.928 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:15.928+0000 7fa153400640 1 -- 172.21.15.47:0/747754902 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fa15c015070 con 0x55f7fce9d6d0 2024-03-21T10:34:15.928 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:15.928+0000 7fa153400640 1 -- 172.21.15.47:0/747754902 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa15c0116a0 con 0x55f7fce9d6d0 2024-03-21T10:34:15.928 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:15.928+0000 7fa15aa00640 1 --2- 172.21.15.47:0/267531806 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7fa1480a3110 0x7fa1480a55d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:34:15.928 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:15.929+0000 7fa16bfe6740 1 -- 172.21.15.47:0/747754902 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f7fce9d6d0 msgr2=0x55f7fce9dab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:34:15.928 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:15.929+0000 7fa16bfe6740 1 --2- 172.21.15.47:0/747754902 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f7fce9d6d0 0x55f7fce9dab0 secure :-1 s=READY pgs=1179 cs=0 l=1 rev1=1 crypto rx=0x7fa15c00b600 tx=0x7fa15c00bac0 comp rx=0 tx=0).stop 2024-03-21T10:34:15.928 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:15.929+0000 7fa15aa00640 1 -- 172.21.15.47:0/267531806 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 9562+0+0 (secure 0 0 0) 0x7fa14c0cff70 con 0x7fa164103200 2024-03-21T10:34:15.928 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:15.929+0000 7fa161e00640 1 --2- 172.21.15.47:0/267531806 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7fa1480a3110 0x7fa1480a55d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:34:15.928 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:15.929+0000 7fa16bfe6740 1 -- 172.21.15.47:0/747754902 shutdown_connections 2024-03-21T10:34:15.928 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:15.929+0000 7fa16bfe6740 1 --2- 172.21.15.47:0/747754902 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f7fcea6c00 0x55f7fcea9050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:15.929 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:15.929+0000 7fa16bfe6740 1 --2- 172.21.15.47:0/747754902 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f7fce9e080 0x55f7fcea6630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:15.929 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:15.929+0000 7fa16bfe6740 1 --2- 172.21.15.47:0/747754902 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f7fce9d6d0 0x55f7fce9dab0 unknown :-1 s=CLOSED pgs=1179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:15.929 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:15.929+0000 7fa16bfe6740 1 -- 172.21.15.47:0/747754902 >> 172.21.15.47:0/747754902 conn(0x55f7fce97da0 msgr2=0x55f7fce98180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:34:15.929 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:15.929+0000 7fa16bfe6740 1 -- 172.21.15.47:0/747754902 shutdown_connections 2024-03-21T10:34:15.929 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:15.929+0000 7fa16bfe6740 1 -- 172.21.15.47:0/747754902 wait complete. 2024-03-21T10:34:15.929 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:15.929+0000 7fa16bfe6740 1 -- start start 2024-03-21T10:34:15.929 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:15.930+0000 7fa161e00640 1 --2- 172.21.15.47:0/267531806 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7fa1480a3110 0x7fa1480a55d0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fa154000f80 tx=0x7fa15402e040 comp rx=0 tx=0).ready entity=mgr.15687 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:34:15.929 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:15.930+0000 7fa16bfe6740 1 Processor -- start 2024-03-21T10:34:15.932 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:15.930+0000 7fa16bfe6740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f7fce9d6d0 0x55f7fcea6540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:34:15.932 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:15.930+0000 7fa16bfe6740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f7fce9e080 0x55f7fcea38d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:34:15.932 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:15.930+0000 7fa16bfe6740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f7fcea6c00 0x55f7fcea3ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:34:15.932 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:15.930+0000 7fa16bfe6740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f7fcea9b70 con 0x55f7fce9e080 2024-03-21T10:34:15.932 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:15.930+0000 7fa16bfe6740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f7fcdbd0d0 con 0x55f7fcea6c00 2024-03-21T10:34:15.932 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:15.930+0000 7fa16bfe6740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f7fcea9cf0 con 0x55f7fce9d6d0 2024-03-21T10:34:15.932 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:15.930+0000 7fa16bfe6740 10 client.0 _ll_register_callbacks cb 0x55f7fcdc1e80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T10:34:15.932 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:15.931+0000 7fa161400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f7fce9e080 0x55f7fcea38d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:34:15.932 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:15.931+0000 7fa161400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f7fce9e080 0x55f7fcea38d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:35150/0 (socket says 172.21.15.47:35150) 2024-03-21T10:34:15.932 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:15.931+0000 7fa161400640 1 -- 172.21.15.47:0/198797449 learned_addr learned my addr 172.21.15.47:0/198797449 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:34:15.933 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:15.931+0000 7fa161e00640 1 --2- 172.21.15.47:0/198797449 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f7fce9d6d0 0x55f7fcea6540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:34:15.933 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:15.931+0000 7fa162800640 1 --2- 172.21.15.47:0/198797449 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f7fcea6c00 0x55f7fcea3ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:34:15.933 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:15.931+0000 7fa161400640 1 -- 172.21.15.47:0/198797449 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f7fce9d6d0 msgr2=0x55f7fcea6540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:34:15.933 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:15.931+0000 7fa161400640 1 --2- 172.21.15.47:0/198797449 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f7fce9d6d0 0x55f7fcea6540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:15.933 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:15.931+0000 7fa161400640 1 -- 172.21.15.47:0/198797449 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f7fcea6c00 msgr2=0x55f7fcea3ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:34:15.933 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:15.931+0000 7fa161400640 1 --2- 172.21.15.47:0/198797449 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f7fcea6c00 0x55f7fcea3ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:15.933 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:15.931+0000 7fa161400640 1 -- 172.21.15.47:0/198797449 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55f7fcb153f0 con 0x55f7fce9e080 2024-03-21T10:34:15.933 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:15.931+0000 7fa162800640 1 --2- 172.21.15.47:0/198797449 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f7fcea6c00 0x55f7fcea3ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T10:34:15.933 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:15.931+0000 7fa161400640 1 --2- 172.21.15.47:0/198797449 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f7fce9e080 0x55f7fcea38d0 secure :-1 s=READY pgs=1180 cs=0 l=1 rev1=1 crypto rx=0x7fa14c0988c0 tx=0x7fa14c09bb40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=59aadba3d9c9cbfb server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:34:15.933 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:15.931+0000 7fa147e00640 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-03-21T10:34:15.933 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:15.932+0000 7fa147e00640 1 -- 172.21.15.47:0/198797449 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa14c0cd6d0 con 0x55f7fce9e080 2024-03-21T10:34:15.933 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:15.932+0000 7fa147e00640 1 -- 172.21.15.47:0/198797449 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(103..103 src has 1..103) v4 ==== 9562+0+0 (secure 0 0 0) 0x7fa14c010430 con 0x55f7fce9e080 2024-03-21T10:34:15.933 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:15.932+0000 7fa16bfe6740 1 -- 172.21.15.47:0/198797449 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fa15c015070 con 0x55f7fce9e080 2024-03-21T10:34:15.933 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:15.932+0000 7fa147e00640 1 -- 172.21.15.47:0/198797449 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa14c0a4900 con 0x55f7fce9e080 2024-03-21T10:34:15.933 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:15.932+0000 7fa16bfe6740 10 client.18318 fetch_fsmap learned FSMap version 32 2024-03-21T10:34:15.934 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:15.932+0000 7fa16bfe6740 1 -- 172.21.15.47:0/198797449 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x55f7fcbdf7d0 con 0x55f7fce9e080 2024-03-21T10:34:15.934 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:15.932+0000 7fa147e00640 1 -- 172.21.15.47:0/198797449 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fa15c015070 con 0x55f7fce9e080 2024-03-21T10:34:15.934 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:15.932+0000 7fa147e00640 1 -- 172.21.15.47:0/198797449 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 32) v1 ==== 7942+0+0 (secure 0 0 0) 0x7fa14c0b8020 con 0x55f7fce9e080 2024-03-21T10:34:15.934 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:15.932+0000 7fa16bfe6740 10 client.18318 fetch_fsmap finished waiting for FSMap version 32 2024-03-21T10:34:15.934 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:15.932+0000 7fa16bfe6740 10 client.18318 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.a 2024-03-21T10:34:15.934 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:15.933+0000 7fa16bfe6740 20 client.18318 populate_metadata read hostname 'smithi047' 2024-03-21T10:34:15.934 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:15.933+0000 7fa16bfe6740 1 --2- 172.21.15.47:0/198797449 >> [v2:172.21.15.47:6840/3401578978,v1:172.21.15.47:6841/3401578978] conn(0x55f7fceb6a60 0x55f7fceb8e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:34:15.934 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:15.933+0000 7fa16bfe6740 4 client.18318 mds_command: new command op to 14619 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T10:34:15.934 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:15.933+0000 7fa16bfe6740 1 -- 172.21.15.47:0/198797449 --> [v2:172.21.15.47:6840/3401578978,v1:172.21.15.47:6841/3401578978] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55f7fce0e800 con 0x55f7fceb6a60 2024-03-21T10:34:15.934 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:15.933+0000 7fa161e00640 1 --2- 172.21.15.47:0/198797449 >> [v2:172.21.15.47:6840/3401578978,v1:172.21.15.47:6841/3401578978] conn(0x55f7fceb6a60 0x55f7fceb8e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:34:15.934 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:15.933+0000 7fa147e00640 10 client.18318 ms_handle_connect on v2:172.21.15.47:6840/3401578978 2024-03-21T10:34:15.934 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:15.933+0000 7fa161e00640 1 --2- 172.21.15.47:0/198797449 >> [v2:172.21.15.47:6840/3401578978,v1:172.21.15.47:6841/3401578978] conn(0x55f7fceb6a60 0x55f7fceb8e50 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=a82cfebfa362aa7 server_cookie=321cb702adf3e4ec in_seq=0 out_seq=0 2024-03-21T10:34:15.936 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:15.934+0000 7fa147e00640 1 -- 172.21.15.47:0/198797449 <== mds.4 v2:172.21.15.47:6840/3401578978 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55f7fce0e800 con 0x55f7fceb6a60 2024-03-21T10:34:15.936 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:15.934+0000 7fa147e00640 10 client.18318 handle_command_reply: tid=0 2024-03-21T10:34:15.936 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:15.934+0000 7fa16bfe6740 1 client.18318 shutdown 2024-03-21T10:34:15.936 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:15.934+0000 7fa16bfe6740 1 -- 172.21.15.47:0/198797449 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f7fce9e080 msgr2=0x55f7fcea38d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:34:15.936 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:15.934+0000 7fa16bfe6740 1 --2- 172.21.15.47:0/198797449 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f7fce9e080 0x55f7fcea38d0 secure :-1 s=READY pgs=1180 cs=0 l=1 rev1=1 crypto rx=0x7fa14c0988c0 tx=0x7fa14c09bb40 comp rx=0 tx=0).stop 2024-03-21T10:34:15.936 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:15.935+0000 7fa16bfe6740 1 -- 172.21.15.47:0/198797449 shutdown_connections 2024-03-21T10:34:15.936 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:15.935+0000 7fa16bfe6740 1 --2- 172.21.15.47:0/198797449 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f7fcea6c00 0x55f7fcea3ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:15.936 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:15.935+0000 7fa16bfe6740 1 --2- 172.21.15.47:0/198797449 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f7fce9e080 0x55f7fcea38d0 unknown :-1 s=CLOSED pgs=1180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:15.936 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:15.935+0000 7fa16bfe6740 1 --2- 172.21.15.47:0/198797449 >> [v2:172.21.15.47:6840/3401578978,v1:172.21.15.47:6841/3401578978] conn(0x55f7fceb6a60 0x55f7fceb8e50 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:15.936 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:15.935+0000 7fa16bfe6740 1 --2- 172.21.15.47:0/198797449 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f7fce9d6d0 0x55f7fcea6540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:15.936 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:15.935+0000 7fa16bfe6740 1 -- 172.21.15.47:0/198797449 >> 172.21.15.47:0/198797449 conn(0x55f7fce97da0 msgr2=0x55f7fce99af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:34:15.936 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:15.935+0000 7fa147e00640 0 client.18318 ms_handle_reset on v2:172.21.15.47:6840/3401578978 2024-03-21T10:34:15.936 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:15.935+0000 7fa16bfe6740 1 -- 172.21.15.47:0/198797449 shutdown_connections 2024-03-21T10:34:15.936 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:15.935+0000 7fa16bfe6740 1 -- 172.21.15.47:0/198797449 wait complete. 2024-03-21T10:34:15.937 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:15.935+0000 7fa16bfe6740 20 client.18318 trim_cache size 0 max 16384 2024-03-21T10:34:15.947 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:15.949+0000 7fa16bfe6740 1 Processor -- start 2024-03-21T10:34:15.947 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:15.949+0000 7fa16bfe6740 1 -- start start 2024-03-21T10:34:15.949 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:15.949+0000 7fa16bfe6740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f7fce9d6d0 0x55f7fce99a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:34:15.949 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:15.949+0000 7fa16bfe6740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f7fce9e080 0x55f7fcea6000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:34:15.950 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:15.949+0000 7fa16bfe6740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f7fcea6c00 0x55f7fcea6540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:34:15.950 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:15.949+0000 7fa16bfe6740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1641b0ca0 con 0x55f7fce9d6d0 2024-03-21T10:34:15.950 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:15.949+0000 7fa16bfe6740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1641b2620 con 0x55f7fce9e080 2024-03-21T10:34:15.950 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:15.949+0000 7fa16bfe6740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1641b1300 con 0x55f7fcea6c00 2024-03-21T10:34:15.950 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:15.949+0000 7fa161e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f7fce9d6d0 0x55f7fce99a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:34:15.950 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:15.949+0000 7fa161e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f7fce9d6d0 0x55f7fce99a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:35158/0 (socket says 172.21.15.47:35158) 2024-03-21T10:34:15.950 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:15.950+0000 7fa161e00640 1 -- 172.21.15.47:0/3704772392 learned_addr learned my addr 172.21.15.47:0/3704772392 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:34:15.950 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:15.950+0000 7fa161400640 1 --2- 172.21.15.47:0/3704772392 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f7fce9e080 0x55f7fcea6000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:34:15.950 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:15.950+0000 7fa162800640 1 --2- 172.21.15.47:0/3704772392 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f7fcea6c00 0x55f7fcea6540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:34:15.950 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:15.950+0000 7fa161e00640 1 -- 172.21.15.47:0/3704772392 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f7fcea6c00 msgr2=0x55f7fcea6540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:34:15.950 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:15.950+0000 7fa161e00640 1 --2- 172.21.15.47:0/3704772392 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f7fcea6c00 0x55f7fcea6540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:15.950 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:15.950+0000 7fa161e00640 1 -- 172.21.15.47:0/3704772392 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f7fce9e080 msgr2=0x55f7fcea6000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:34:15.951 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:15.950+0000 7fa161e00640 1 --2- 172.21.15.47:0/3704772392 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f7fce9e080 0x55f7fcea6000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:15.951 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:15.950+0000 7fa161e00640 1 -- 172.21.15.47:0/3704772392 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f7fcea2880 con 0x55f7fce9d6d0 2024-03-21T10:34:15.951 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:15.950+0000 7fa162800640 1 --2- 172.21.15.47:0/3704772392 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f7fcea6c00 0x55f7fcea6540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T10:34:15.951 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:15.950+0000 7fa161e00640 1 --2- 172.21.15.47:0/3704772392 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f7fce9d6d0 0x55f7fce99a80 secure :-1 s=READY pgs=1181 cs=0 l=1 rev1=1 crypto rx=0x7fa15402e010 tx=0x7fa15403e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3fcacfb238e925d5 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:34:15.951 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:15.951+0000 7fa152a00640 1 -- 172.21.15.47:0/3704772392 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa154039050 con 0x55f7fce9d6d0 2024-03-21T10:34:15.951 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:15.951+0000 7fa152a00640 1 -- 172.21.15.47:0/3704772392 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fa154048070 con 0x55f7fce9d6d0 2024-03-21T10:34:15.951 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:15.951+0000 7fa152a00640 1 -- 172.21.15.47:0/3704772392 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa154040480 con 0x55f7fce9d6d0 2024-03-21T10:34:15.951 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:15.951+0000 7fa16bfe6740 1 -- 172.21.15.47:0/3704772392 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f7fce9d6d0 msgr2=0x55f7fce99a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:34:15.951 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:15.951+0000 7fa16bfe6740 1 --2- 172.21.15.47:0/3704772392 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f7fce9d6d0 0x55f7fce99a80 secure :-1 s=READY pgs=1181 cs=0 l=1 rev1=1 crypto rx=0x7fa15402e010 tx=0x7fa15403e000 comp rx=0 tx=0).stop 2024-03-21T10:34:15.951 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:15.951+0000 7fa16bfe6740 1 -- 172.21.15.47:0/3704772392 shutdown_connections 2024-03-21T10:34:15.951 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:15.951+0000 7fa16bfe6740 1 --2- 172.21.15.47:0/3704772392 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f7fcea6c00 0x55f7fcea6540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:15.951 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:15.951+0000 7fa16bfe6740 1 --2- 172.21.15.47:0/3704772392 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f7fce9e080 0x55f7fcea6000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:15.951 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:15.951+0000 7fa16bfe6740 1 --2- 172.21.15.47:0/3704772392 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f7fce9d6d0 0x55f7fce99a80 unknown :-1 s=CLOSED pgs=1181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:15.951 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:15.951+0000 7fa16bfe6740 1 -- 172.21.15.47:0/3704772392 >> 172.21.15.47:0/3704772392 conn(0x55f7fce97da0 msgr2=0x55f7fce98180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:34:15.952 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:15.951+0000 7fa16bfe6740 1 -- 172.21.15.47:0/3704772392 shutdown_connections 2024-03-21T10:34:15.952 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:15.951+0000 7fa16bfe6740 1 -- 172.21.15.47:0/3704772392 wait complete. 2024-03-21T10:34:15.955 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:15.952+0000 7fa16bfe6740 1 -- start start 2024-03-21T10:34:15.955 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:15.952+0000 7fa16bfe6740 1 Processor -- start 2024-03-21T10:34:15.955 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:15.953+0000 7fa16bfe6740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f7fce9d6d0 0x55f7fceacf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:34:15.956 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:15.953+0000 7fa16bfe6740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f7fce9e080 0x55f7fcead480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:34:15.956 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:15.953+0000 7fa16bfe6740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f7fcea6c00 0x55f7fceae390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:34:15.956 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:15.953+0000 7fa16bfe6740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1641b1fc0 con 0x55f7fce9d6d0 2024-03-21T10:34:15.956 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:15.953+0000 7fa16bfe6740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1641b1960 con 0x55f7fcea6c00 2024-03-21T10:34:15.956 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:15.953+0000 7fa16bfe6740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1641aee80 con 0x55f7fce9e080 2024-03-21T10:34:15.956 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:15.953+0000 7fa16bfe6740 10 client.0 _ll_register_callbacks cb 0x7fa14c009700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T10:34:15.956 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:15.953+0000 7fa161e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f7fce9d6d0 0x55f7fceacf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:34:15.956 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:15.953+0000 7fa161e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f7fce9d6d0 0x55f7fceacf40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:35166/0 (socket says 172.21.15.47:35166) 2024-03-21T10:34:15.956 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:15.953+0000 7fa161e00640 1 -- 172.21.15.47:0/3398058103 learned_addr learned my addr 172.21.15.47:0/3398058103 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:34:15.956 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:15.953+0000 7fa161400640 1 --2- 172.21.15.47:0/3398058103 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f7fce9e080 0x55f7fcead480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:34:15.956 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:15.953+0000 7fa162800640 1 --2- 172.21.15.47:0/3398058103 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f7fcea6c00 0x55f7fceae390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:34:15.956 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:15.953+0000 7fa161e00640 1 -- 172.21.15.47:0/3398058103 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f7fce9e080 msgr2=0x55f7fcead480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:34:15.956 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:15.953+0000 7fa161e00640 1 --2- 172.21.15.47:0/3398058103 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f7fce9e080 0x55f7fcead480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:15.957 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:15.953+0000 7fa161e00640 1 -- 172.21.15.47:0/3398058103 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f7fcea6c00 msgr2=0x55f7fceae390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:34:15.957 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:15.953+0000 7fa161e00640 1 --2- 172.21.15.47:0/3398058103 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f7fcea6c00 0x55f7fceae390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:15.957 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:15.953+0000 7fa161e00640 1 -- 172.21.15.47:0/3398058103 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55f7fcea4250 con 0x55f7fce9d6d0 2024-03-21T10:34:15.957 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:15.954+0000 7fa161e00640 1 --2- 172.21.15.47:0/3398058103 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f7fce9d6d0 0x55f7fceacf40 secure :-1 s=READY pgs=1182 cs=0 l=1 rev1=1 crypto rx=0x7fa154037040 tx=0x7fa154005ef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d97de72f4bb591d0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:34:15.957 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:15.954+0000 7fa147e00640 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-03-21T10:34:15.957 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:15.954+0000 7fa147e00640 1 -- 172.21.15.47:0/3398058103 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa154039040 con 0x55f7fce9d6d0 2024-03-21T10:34:15.957 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:15.954+0000 7fa147e00640 1 -- 172.21.15.47:0/3398058103 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(103..103 src has 1..103) v4 ==== 9562+0+0 (secure 0 0 0) 0x7fa1540325f0 con 0x55f7fce9d6d0 2024-03-21T10:34:15.957 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:15.954+0000 7fa16bfe6740 1 -- 172.21.15.47:0/3398058103 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fa154048070 con 0x55f7fce9d6d0 2024-03-21T10:34:15.957 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:15.954+0000 7fa147e00640 1 -- 172.21.15.47:0/3398058103 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa15406c050 con 0x55f7fce9d6d0 2024-03-21T10:34:15.957 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:15.954+0000 7fa16bfe6740 10 client.18324 fetch_fsmap learned FSMap version 32 2024-03-21T10:34:15.957 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:15.954+0000 7fa16bfe6740 1 -- 172.21.15.47:0/3398058103 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x55f7fcf12c70 con 0x55f7fce9d6d0 2024-03-21T10:34:15.957 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:15.955+0000 7fa147e00640 1 -- 172.21.15.47:0/3398058103 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fa154048070 con 0x55f7fce9d6d0 2024-03-21T10:34:15.957 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:15.955+0000 7fa147e00640 1 -- 172.21.15.47:0/3398058103 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 32) v1 ==== 7942+0+0 (secure 0 0 0) 0x7fa154040c90 con 0x55f7fce9d6d0 2024-03-21T10:34:15.957 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:15.955+0000 7fa16bfe6740 10 client.18324 fetch_fsmap finished waiting for FSMap version 32 2024-03-21T10:34:15.958 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:15.955+0000 7fa16bfe6740 10 client.18324 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.a 2024-03-21T10:34:15.958 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:15.955+0000 7fa16bfe6740 20 client.18324 populate_metadata read hostname 'smithi047' 2024-03-21T10:34:15.958 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:15.955+0000 7fa16bfe6740 1 --2- 172.21.15.47:0/3398058103 >> [v2:172.21.15.47:6840/3401578978,v1:172.21.15.47:6841/3401578978] conn(0x55f7fcf13340 0x55f7fcf15760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:34:15.958 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:15.955+0000 7fa16bfe6740 4 client.18324 mds_command: new command op to 14619 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-21T10:34:15.958 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:15.955+0000 7fa16bfe6740 1 -- 172.21.15.47:0/3398058103 --> [v2:172.21.15.47:6840/3401578978,v1:172.21.15.47:6841/3401578978] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55f7fce08b50 con 0x55f7fcf13340 2024-03-21T10:34:15.958 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:15.955+0000 7fa161400640 1 --2- 172.21.15.47:0/3398058103 >> [v2:172.21.15.47:6840/3401578978,v1:172.21.15.47:6841/3401578978] conn(0x55f7fcf13340 0x55f7fcf15760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:34:15.958 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:15.956+0000 7fa161400640 1 --2- 172.21.15.47:0/3398058103 >> [v2:172.21.15.47:6840/3401578978,v1:172.21.15.47:6841/3401578978] conn(0x55f7fcf13340 0x55f7fcf15760 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=d4d287c7c05a029b server_cookie=ceab6decf6d171fd in_seq=0 out_seq=0 2024-03-21T10:34:15.958 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:15.956+0000 7fa147e00640 10 client.18324 ms_handle_connect on v2:172.21.15.47:6840/3401578978 2024-03-21T10:34:15.958 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:15.956+0000 7fa147e00640 1 -- 172.21.15.47:0/3398058103 <== mds.4 v2:172.21.15.47:6840/3401578978 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55f7fce08b50 con 0x55f7fcf13340 2024-03-21T10:34:15.958 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:15.956+0000 7fa147e00640 10 client.18324 handle_command_reply: tid=0 2024-03-21T10:34:15.958 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:15.956+0000 7fa16bfe6740 1 client.18324 shutdown 2024-03-21T10:34:15.958 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:15.956+0000 7fa16bfe6740 1 -- 172.21.15.47:0/3398058103 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f7fce9d6d0 msgr2=0x55f7fceacf40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:34:15.958 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:15.956+0000 7fa16bfe6740 1 --2- 172.21.15.47:0/3398058103 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f7fce9d6d0 0x55f7fceacf40 secure :-1 s=READY pgs=1182 cs=0 l=1 rev1=1 crypto rx=0x7fa154037040 tx=0x7fa154005ef0 comp rx=0 tx=0).stop 2024-03-21T10:34:15.958 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:15.956+0000 7fa16bfe6740 1 -- 172.21.15.47:0/3398058103 shutdown_connections 2024-03-21T10:34:15.959 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:15.956+0000 7fa16bfe6740 1 --2- 172.21.15.47:0/3398058103 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f7fcea6c00 0x55f7fceae390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:15.959 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:15.956+0000 7fa16bfe6740 1 --2- 172.21.15.47:0/3398058103 >> [v2:172.21.15.47:6840/3401578978,v1:172.21.15.47:6841/3401578978] conn(0x55f7fcf13340 0x55f7fcf15760 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:15.959 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:15.956+0000 7fa16bfe6740 1 --2- 172.21.15.47:0/3398058103 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f7fce9e080 0x55f7fcead480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:15.959 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:15.956+0000 7fa16bfe6740 1 --2- 172.21.15.47:0/3398058103 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f7fce9d6d0 0x55f7fceacf40 unknown :-1 s=CLOSED pgs=1182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:15.959 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:15.956+0000 7fa16bfe6740 1 -- 172.21.15.47:0/3398058103 >> 172.21.15.47:0/3398058103 conn(0x55f7fce97da0 msgr2=0x55f7fce96620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:34:15.959 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:15.956+0000 7fa147e00640 0 client.18324 ms_handle_reset on v2:172.21.15.47:6840/3401578978 2024-03-21T10:34:15.959 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:15.957+0000 7fa16bfe6740 1 -- 172.21.15.47:0/3398058103 shutdown_connections 2024-03-21T10:34:15.959 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:15.957+0000 7fa16bfe6740 1 -- 172.21.15.47:0/3398058103 wait complete. 2024-03-21T10:34:15.959 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:15.957+0000 7fa16bfe6740 20 client.18324 trim_cache size 0 max 16384 2024-03-21T10:34:15.959 INFO:teuthology.orchestra.run.smithi047.stdout:[] 2024-03-21T10:34:15.960 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:15.957+0000 7fa169200640 1 -- 172.21.15.47:0/267531806 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7fa1480a3110 msgr2=0x7fa1480a55d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:34:15.960 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:15.957+0000 7fa169200640 1 --2- 172.21.15.47:0/267531806 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7fa1480a3110 0x7fa1480a55d0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fa154000f80 tx=0x7fa15402e040 comp rx=0 tx=0).stop 2024-03-21T10:34:15.960 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:15.957+0000 7fa169200640 1 -- 172.21.15.47:0/267531806 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa164103200 msgr2=0x7fa16419a5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:34:15.960 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:15.958+0000 7fa169200640 1 --2- 172.21.15.47:0/267531806 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa164103200 0x7fa16419a5d0 secure :-1 s=READY pgs=1178 cs=0 l=1 rev1=1 crypto rx=0x7fa14c00e9f0 tx=0x7fa14c00eeb0 comp rx=0 tx=0).stop 2024-03-21T10:34:15.960 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:15.958+0000 7fa169200640 1 -- 172.21.15.47:0/267531806 shutdown_connections 2024-03-21T10:34:15.960 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:15.958+0000 7fa169200640 1 --2- 172.21.15.47:0/267531806 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7fa1480a3110 0x7fa1480a55d0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:15.961 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:15.958+0000 7fa169200640 1 --2- 172.21.15.47:0/267531806 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa16410bcb0 0x7fa16419e950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:15.961 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:15.958+0000 7fa169200640 1 --2- 172.21.15.47:0/267531806 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa164103200 0x7fa16419a5d0 unknown :-1 s=CLOSED pgs=1178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:15.961 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:15.958+0000 7fa169200640 1 --2- 172.21.15.47:0/267531806 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa1641028e0 0x7fa16419a090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:15.961 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:15.958+0000 7fa169200640 1 -- 172.21.15.47:0/267531806 >> 172.21.15.47:0/267531806 conn(0x7fa164076920 msgr2=0x7fa164100500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:34:15.961 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:15.958+0000 7fa169200640 1 -- 172.21.15.47:0/267531806 shutdown_connections 2024-03-21T10:34:15.961 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:15.958+0000 7fa169200640 1 -- 172.21.15.47:0/267531806 wait complete. 2024-03-21T10:34:15.961 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:34:15 smithi047 ceph-mon[31547]: from='client.? 172.21.15.47:0/3450661005' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-21T10:34:15.974 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-21T10:34:15.974 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-03-21T10:34:16.181 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:16.183+0000 7f9e63e00640 1 -- 172.21.15.47:0/902049819 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9e64102750 msgr2=0x7f9e64102b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:34:16.182 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:16.183+0000 7f9e63e00640 1 --2- 172.21.15.47:0/902049819 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9e64102750 0x7f9e64102b30 secure :-1 s=READY pgs=1183 cs=0 l=1 rev1=1 crypto rx=0x7f9e54009900 tx=0x7f9e5402bcc0 comp rx=0 tx=0).stop 2024-03-21T10:34:16.182 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:16.183+0000 7f9e63e00640 1 -- 172.21.15.47:0/902049819 shutdown_connections 2024-03-21T10:34:16.182 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:16.183+0000 7f9e63e00640 1 --2- 172.21.15.47:0/902049819 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9e6410bb70 0x7f9e6410e010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:16.182 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:16.183+0000 7f9e63e00640 1 --2- 172.21.15.47:0/902049819 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9e64103070 0x7f9e6410b550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:16.182 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:16.183+0000 7f9e63e00640 1 --2- 172.21.15.47:0/902049819 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9e64102750 0x7f9e64102b30 unknown :-1 s=CLOSED pgs=1183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:16.182 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:16.183+0000 7f9e63e00640 1 -- 172.21.15.47:0/902049819 >> 172.21.15.47:0/902049819 conn(0x7f9e640fbf50 msgr2=0x7f9e640fe370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:34:16.182 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:16.183+0000 7f9e63e00640 1 -- 172.21.15.47:0/902049819 shutdown_connections 2024-03-21T10:34:16.182 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:16.183+0000 7f9e63e00640 1 -- 172.21.15.47:0/902049819 wait complete. 2024-03-21T10:34:16.183 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:16.185+0000 7f9e63e00640 1 Processor -- start 2024-03-21T10:34:16.183 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:16.185+0000 7f9e63e00640 1 -- start start 2024-03-21T10:34:16.184 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:16.185+0000 7f9e63e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9e64102750 0x7f9e64199fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:34:16.184 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:16.185+0000 7f9e63e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9e64103070 0x7f9e6419a500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:34:16.184 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:16.185+0000 7f9e63e00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9e6410bb70 0x7f9e6419e830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:34:16.184 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:16.185+0000 7f9e63e00640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e64111800 con 0x7f9e64102750 2024-03-21T10:34:16.184 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:16.185+0000 7f9e63e00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e64111680 con 0x7f9e64103070 2024-03-21T10:34:16.184 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:16.185+0000 7f9e63e00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e64111980 con 0x7f9e6410bb70 2024-03-21T10:34:16.184 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:16.186+0000 7f9e62a00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9e64102750 0x7f9e64199fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:34:16.184 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:16.186+0000 7f9e62a00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9e64102750 0x7f9e64199fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:35186/0 (socket says 172.21.15.47:35186) 2024-03-21T10:34:16.184 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:16.186+0000 7f9e62a00640 1 -- 172.21.15.47:0/1835347847 learned_addr learned my addr 172.21.15.47:0/1835347847 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:34:16.185 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:16.186+0000 7f9e62000640 1 --2- 172.21.15.47:0/1835347847 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9e64103070 0x7f9e6419a500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:34:16.185 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:16.186+0000 7f9e63400640 1 --2- 172.21.15.47:0/1835347847 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9e6410bb70 0x7f9e6419e830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:34:16.186 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:16.186+0000 7f9e62a00640 1 -- 172.21.15.47:0/1835347847 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9e6410bb70 msgr2=0x7f9e6419e830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:34:16.186 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:16.186+0000 7f9e62a00640 1 --2- 172.21.15.47:0/1835347847 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9e6410bb70 0x7f9e6419e830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:16.186 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:16.186+0000 7f9e62a00640 1 -- 172.21.15.47:0/1835347847 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9e64103070 msgr2=0x7f9e6419a500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:34:16.186 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:16.186+0000 7f9e62a00640 1 --2- 172.21.15.47:0/1835347847 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9e64103070 0x7f9e6419a500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:16.186 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:16.186+0000 7f9e62a00640 1 -- 172.21.15.47:0/1835347847 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e6419ef60 con 0x7f9e64102750 2024-03-21T10:34:16.186 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:16.186+0000 7f9e63400640 1 --2- 172.21.15.47:0/1835347847 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9e6410bb70 0x7f9e6419e830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T10:34:16.186 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:16.187+0000 7f9e62a00640 1 --2- 172.21.15.47:0/1835347847 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9e64102750 0x7f9e64199fc0 secure :-1 s=READY pgs=1184 cs=0 l=1 rev1=1 crypto rx=0x7f9e54005940 tx=0x7f9e540045e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:34:16.186 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:16.187+0000 7f9e68a00640 1 -- 172.21.15.47:0/1835347847 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e54043020 con 0x7f9e64102750 2024-03-21T10:34:16.186 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:16.187+0000 7f9e63e00640 1 -- 172.21.15.47:0/1835347847 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9e6419f1f0 con 0x7f9e64102750 2024-03-21T10:34:16.186 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:16.187+0000 7f9e68a00640 1 -- 172.21.15.47:0/1835347847 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f9e5402db90 con 0x7f9e64102750 2024-03-21T10:34:16.187 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:16.187+0000 7f9e68a00640 1 -- 172.21.15.47:0/1835347847 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e54003a10 con 0x7f9e64102750 2024-03-21T10:34:16.187 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:16.187+0000 7f9e63e00640 1 -- 172.21.15.47:0/1835347847 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9e641a6ae0 con 0x7f9e64102750 2024-03-21T10:34:16.189 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:16.191+0000 7f9e6cc16740 1 Processor -- start 2024-03-21T10:34:16.189 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:16.191+0000 7f9e6cc16740 1 -- start start 2024-03-21T10:34:16.190 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:16.191+0000 7f9e6cc16740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55aa909938b0 0x55aa90993c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:34:16.190 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:16.191+0000 7f9e6cc16740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55aa90994260 0x55aa9099c810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:34:16.190 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:16.191+0000 7f9e6cc16740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55aa9099cde0 0x55aa9099f230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:34:16.190 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:16.191+0000 7f9e6cc16740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aa908f72c0 con 0x55aa9099cde0 2024-03-21T10:34:16.190 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:16.191+0000 7f9e6cc16740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aa9089fc90 con 0x55aa90994260 2024-03-21T10:34:16.190 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:16.191+0000 7f9e6cc16740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aa908f3360 con 0x55aa909938b0 2024-03-21T10:34:16.191 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:16.191+0000 7f9e68a00640 1 -- 172.21.15.47:0/1835347847 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 27) v1 ==== 137985+0+0 (secure 0 0 0) 0x7f9e54046020 con 0x7f9e64102750 2024-03-21T10:34:16.191 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:16.192+0000 7f9e62000640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55aa9099cde0 0x55aa9099f230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:34:16.191 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:16.192+0000 7f9e62000640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55aa9099cde0 0x55aa9099f230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:35202/0 (socket says 172.21.15.47:35202) 2024-03-21T10:34:16.191 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:16.192+0000 7f9e62000640 1 -- 172.21.15.47:0/3865908568 learned_addr learned my addr 172.21.15.47:0/3865908568 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:34:16.191 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:16.192+0000 7f9e62a00640 1 --2- 172.21.15.47:0/3865908568 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55aa90994260 0x55aa9099c810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:34:16.191 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:16.192+0000 7f9e63400640 1 --2- 172.21.15.47:0/3865908568 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55aa909938b0 0x55aa90993c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:34:16.191 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:16.192+0000 7f9e62000640 1 -- 172.21.15.47:0/3865908568 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55aa909938b0 msgr2=0x55aa90993c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:34:16.191 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:16.192+0000 7f9e62000640 1 --2- 172.21.15.47:0/3865908568 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55aa909938b0 0x55aa90993c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:16.191 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:16.192+0000 7f9e62000640 1 -- 172.21.15.47:0/3865908568 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55aa90994260 msgr2=0x55aa9099c810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:34:16.191 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:16.192+0000 7f9e62000640 1 --2- 172.21.15.47:0/3865908568 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55aa90994260 0x55aa9099c810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:16.191 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:16.192+0000 7f9e62000640 1 -- 172.21.15.47:0/3865908568 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55aa908c4080 con 0x55aa9099cde0 2024-03-21T10:34:16.191 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:16.192+0000 7f9e62a00640 1 --2- 172.21.15.47:0/3865908568 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55aa90994260 0x55aa9099c810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T10:34:16.195 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:16.193+0000 7f9e62000640 1 --2- 172.21.15.47:0/3865908568 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55aa9099cde0 0x55aa9099f230 secure :-1 s=READY pgs=1185 cs=0 l=1 rev1=1 crypto rx=0x7f9e50009c80 tx=0x7f9e50011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=455f32b4447811c9 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:34:16.195 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:16.193+0000 7f9e68a00640 1 --2- 172.21.15.47:0/1835347847 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f9e480a3090 0x7f9e480a5550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:34:16.195 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:16.193+0000 7f9e4fe00640 1 -- 172.21.15.47:0/3865908568 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e5001e070 con 0x55aa9099cde0 2024-03-21T10:34:16.195 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:16.193+0000 7f9e4fe00640 1 -- 172.21.15.47:0/3865908568 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f9e50019ae0 con 0x55aa9099cde0 2024-03-21T10:34:16.195 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:16.193+0000 7f9e4fe00640 1 -- 172.21.15.47:0/3865908568 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e5000e9b0 con 0x55aa9099cde0 2024-03-21T10:34:16.195 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:16.193+0000 7f9e68a00640 1 -- 172.21.15.47:0/1835347847 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 9562+0+0 (secure 0 0 0) 0x7f9e540f1b90 con 0x7f9e64102750 2024-03-21T10:34:16.195 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:16.193+0000 7f9e63400640 1 --2- 172.21.15.47:0/1835347847 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f9e480a3090 0x7f9e480a5550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:34:16.195 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:16.194+0000 7f9e6cc16740 1 -- 172.21.15.47:0/3865908568 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55aa9099cde0 msgr2=0x55aa9099f230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:34:16.195 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:16.194+0000 7f9e6cc16740 1 --2- 172.21.15.47:0/3865908568 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55aa9099cde0 0x55aa9099f230 secure :-1 s=READY pgs=1185 cs=0 l=1 rev1=1 crypto rx=0x7f9e50009c80 tx=0x7f9e50011810 comp rx=0 tx=0).stop 2024-03-21T10:34:16.195 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:16.194+0000 7f9e63400640 1 --2- 172.21.15.47:0/1835347847 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f9e480a3090 0x7f9e480a5550 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f9e5c005fd0 tx=0x7f9e5c009530 comp rx=0 tx=0).ready entity=mgr.15687 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:34:16.195 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:16.194+0000 7f9e6cc16740 1 -- 172.21.15.47:0/3865908568 shutdown_connections 2024-03-21T10:34:16.195 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:16.194+0000 7f9e6cc16740 1 --2- 172.21.15.47:0/3865908568 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55aa9099cde0 0x55aa9099f230 unknown :-1 s=CLOSED pgs=1185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:16.196 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:16.194+0000 7f9e6cc16740 1 --2- 172.21.15.47:0/3865908568 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55aa90994260 0x55aa9099c810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:16.196 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:16.194+0000 7f9e6cc16740 1 --2- 172.21.15.47:0/3865908568 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55aa909938b0 0x55aa90993c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:16.196 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:16.194+0000 7f9e6cc16740 1 -- 172.21.15.47:0/3865908568 >> 172.21.15.47:0/3865908568 conn(0x55aa9098df80 msgr2=0x55aa9098e360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:34:16.196 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:16.194+0000 7f9e6cc16740 1 -- 172.21.15.47:0/3865908568 shutdown_connections 2024-03-21T10:34:16.196 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:16.194+0000 7f9e6cc16740 1 -- 172.21.15.47:0/3865908568 wait complete. 2024-03-21T10:34:16.196 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:16.195+0000 7f9e6cc16740 1 -- start start 2024-03-21T10:34:16.196 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:16.195+0000 7f9e6cc16740 1 Processor -- start 2024-03-21T10:34:16.196 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:16.195+0000 7f9e6cc16740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55aa909938b0 0x55aa9099c6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:34:16.196 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:16.195+0000 7f9e6cc16740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55aa90994260 0x55aa90999a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:34:16.196 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:16.195+0000 7f9e6cc16740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55aa9099cde0 0x55aa90999fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:34:16.196 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:16.195+0000 7f9e6cc16740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aa9099fd50 con 0x55aa909938b0 2024-03-21T10:34:16.196 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:16.195+0000 7f9e6cc16740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aa908b30d0 con 0x55aa9099cde0 2024-03-21T10:34:16.196 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:16.195+0000 7f9e6cc16740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aa9099fed0 con 0x55aa90994260 2024-03-21T10:34:16.196 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:16.196+0000 7f9e6cc16740 10 client.0 _ll_register_callbacks cb 0x55aa908b7e80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T10:34:16.197 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:16.196+0000 7f9e62a00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55aa909938b0 0x55aa9099c6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:34:16.197 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:16.196+0000 7f9e62a00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55aa909938b0 0x55aa9099c6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:35210/0 (socket says 172.21.15.47:35210) 2024-03-21T10:34:16.197 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:16.196+0000 7f9e62a00640 1 -- 172.21.15.47:0/1693335907 learned_addr learned my addr 172.21.15.47:0/1693335907 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:34:16.197 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:16.196+0000 7f9e63400640 1 --2- 172.21.15.47:0/1693335907 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55aa9099cde0 0x55aa90999fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:34:16.197 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:16.196+0000 7f9e62000640 1 --2- 172.21.15.47:0/1693335907 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55aa90994260 0x55aa90999a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:34:16.197 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:16.196+0000 7f9e62a00640 1 -- 172.21.15.47:0/1693335907 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55aa90994260 msgr2=0x55aa90999a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:34:16.197 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:16.196+0000 7f9e62a00640 1 --2- 172.21.15.47:0/1693335907 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55aa90994260 0x55aa90999a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:16.197 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:16.196+0000 7f9e62a00640 1 -- 172.21.15.47:0/1693335907 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55aa9099cde0 msgr2=0x55aa90999fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:34:16.197 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:16.196+0000 7f9e62a00640 1 --2- 172.21.15.47:0/1693335907 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55aa9099cde0 0x55aa90999fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:16.197 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:16.196+0000 7f9e62a00640 1 -- 172.21.15.47:0/1693335907 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55aa9060b3f0 con 0x55aa909938b0 2024-03-21T10:34:16.197 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:16.197+0000 7f9e62a00640 1 --2- 172.21.15.47:0/1693335907 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55aa909938b0 0x55aa9099c6c0 secure :-1 s=READY pgs=1186 cs=0 l=1 rev1=1 crypto rx=0x7f9e540bd980 tx=0x7f9e540bdb10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=68f17a63676fff27 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:34:16.197 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:16.197+0000 7f9e4cc00640 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-03-21T10:34:16.197 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:16.197+0000 7f9e4cc00640 1 -- 172.21.15.47:0/1693335907 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e540bdc80 con 0x55aa909938b0 2024-03-21T10:34:16.197 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:16.197+0000 7f9e4cc00640 1 -- 172.21.15.47:0/1693335907 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(103..103 src has 1..103) v4 ==== 9562+0+0 (secure 0 0 0) 0x7f9e54005340 con 0x55aa909938b0 2024-03-21T10:34:16.198 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:16.197+0000 7f9e6cc16740 1 -- 172.21.15.47:0/1693335907 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f9e50019ae0 con 0x55aa909938b0 2024-03-21T10:34:16.199 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:16.197+0000 7f9e4cc00640 1 -- 172.21.15.47:0/1693335907 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9e540c1dd0 con 0x55aa909938b0 2024-03-21T10:34:16.199 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:16.197+0000 7f9e6cc16740 10 client.18336 fetch_fsmap learned FSMap version 32 2024-03-21T10:34:16.200 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:16.197+0000 7f9e6cc16740 1 -- 172.21.15.47:0/1693335907 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x55aa906d57d0 con 0x55aa909938b0 2024-03-21T10:34:16.200 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:16.197+0000 7f9e4cc00640 1 -- 172.21.15.47:0/1693335907 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f9e50019ae0 con 0x55aa909938b0 2024-03-21T10:34:16.200 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:16.198+0000 7f9e4cc00640 1 -- 172.21.15.47:0/1693335907 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 32) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f9e54003bb0 con 0x55aa909938b0 2024-03-21T10:34:16.200 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:16.198+0000 7f9e6cc16740 10 client.18336 fetch_fsmap finished waiting for FSMap version 32 2024-03-21T10:34:16.200 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:16.198+0000 7f9e6cc16740 10 client.18336 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.k 2024-03-21T10:34:16.200 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:16.198+0000 7f9e6cc16740 20 client.18336 populate_metadata read hostname 'smithi047' 2024-03-21T10:34:16.200 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:16.198+0000 7f9e6cc16740 1 --2- 172.21.15.47:0/1693335907 >> [v2:172.21.15.100:6832/2723302038,v1:172.21.15.100:6833/2723302038] conn(0x55aa909acb00 0x55aa909aef10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:34:16.200 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:16.198+0000 7f9e6cc16740 4 client.18336 mds_command: new command op to 34427 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T10:34:16.200 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:16.198+0000 7f9e6cc16740 1 -- 172.21.15.47:0/1693335907 --> [v2:172.21.15.100:6832/2723302038,v1:172.21.15.100:6833/2723302038] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55aa90904800 con 0x55aa909acb00 2024-03-21T10:34:16.200 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:16.198+0000 7f9e62000640 1 --2- 172.21.15.47:0/1693335907 >> [v2:172.21.15.100:6832/2723302038,v1:172.21.15.100:6833/2723302038] conn(0x55aa909acb00 0x55aa909aef10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:34:16.200 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:16.199+0000 7f9e62000640 1 --2- 172.21.15.47:0/1693335907 >> [v2:172.21.15.100:6832/2723302038,v1:172.21.15.100:6833/2723302038] conn(0x55aa909acb00 0x55aa909aef10 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=f849360f3bd75fcc server_cookie=b5c05305f419ed3 in_seq=0 out_seq=0 2024-03-21T10:34:16.201 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:16.199+0000 7f9e4cc00640 10 client.18336 ms_handle_connect on v2:172.21.15.100:6832/2723302038 2024-03-21T10:34:16.201 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:16.201+0000 7f9e4cc00640 1 -- 172.21.15.47:0/1693335907 <== mds.3 v2:172.21.15.100:6832/2723302038 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55aa90904800 con 0x55aa909acb00 2024-03-21T10:34:16.201 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:16.201+0000 7f9e4cc00640 10 client.18336 handle_command_reply: tid=0 2024-03-21T10:34:16.201 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:16.201+0000 7f9e6cc16740 1 client.18336 shutdown 2024-03-21T10:34:16.201 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:16.201+0000 7f9e6cc16740 1 -- 172.21.15.47:0/1693335907 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55aa909938b0 msgr2=0x55aa9099c6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:34:16.201 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:16.201+0000 7f9e6cc16740 1 --2- 172.21.15.47:0/1693335907 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55aa909938b0 0x55aa9099c6c0 secure :-1 s=READY pgs=1186 cs=0 l=1 rev1=1 crypto rx=0x7f9e540bd980 tx=0x7f9e540bdb10 comp rx=0 tx=0).stop 2024-03-21T10:34:16.201 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:16.201+0000 7f9e6cc16740 1 -- 172.21.15.47:0/1693335907 shutdown_connections 2024-03-21T10:34:16.201 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:16.201+0000 7f9e6cc16740 1 --2- 172.21.15.47:0/1693335907 >> [v2:172.21.15.100:6832/2723302038,v1:172.21.15.100:6833/2723302038] conn(0x55aa909acb00 0x55aa909aef10 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:16.201 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:16.201+0000 7f9e6cc16740 1 --2- 172.21.15.47:0/1693335907 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55aa9099cde0 0x55aa90999fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:16.201 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:16.201+0000 7f9e6cc16740 1 --2- 172.21.15.47:0/1693335907 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55aa90994260 0x55aa90999a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:16.201 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:16.201+0000 7f9e4cc00640 0 client.18336 ms_handle_reset on v2:172.21.15.100:6832/2723302038 2024-03-21T10:34:16.201 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:16.201+0000 7f9e6cc16740 1 --2- 172.21.15.47:0/1693335907 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55aa909938b0 0x55aa9099c6c0 unknown :-1 s=CLOSED pgs=1186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:16.201 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:16.201+0000 7f9e6cc16740 1 -- 172.21.15.47:0/1693335907 >> 172.21.15.47:0/1693335907 conn(0x55aa9098df80 msgr2=0x55aa9098fcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:34:16.201 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:16.201+0000 7f9e6cc16740 1 -- 172.21.15.47:0/1693335907 shutdown_connections 2024-03-21T10:34:16.201 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:16.201+0000 7f9e6cc16740 1 -- 172.21.15.47:0/1693335907 wait complete. 2024-03-21T10:34:16.202 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:16.201+0000 7f9e6cc16740 20 client.18336 trim_cache size 0 max 16384 2024-03-21T10:34:16.215 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:16.215+0000 7f9e6cc16740 1 Processor -- start 2024-03-21T10:34:16.215 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:16.215+0000 7f9e6cc16740 1 -- start start 2024-03-21T10:34:16.215 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:16.216+0000 7f9e6cc16740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55aa909938b0 0x55aa9099c180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:34:16.215 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:16.216+0000 7f9e6cc16740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55aa90994260 0x55aa9099c6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:34:16.215 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:16.216+0000 7f9e6cc16740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55aa9099cde0 0x55aa90996cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:34:16.216 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:16.216+0000 7f9e6cc16740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e641b09e0 con 0x55aa909938b0 2024-03-21T10:34:16.216 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:16.216+0000 7f9e6cc16740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e641b2360 con 0x55aa9099cde0 2024-03-21T10:34:16.216 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:16.216+0000 7f9e6cc16740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e641b1040 con 0x55aa90994260 2024-03-21T10:34:16.216 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:16.216+0000 7f9e62a00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55aa909938b0 0x55aa9099c180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:34:16.216 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:16.216+0000 7f9e62a00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55aa909938b0 0x55aa9099c180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:35216/0 (socket says 172.21.15.47:35216) 2024-03-21T10:34:16.216 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:16.216+0000 7f9e62a00640 1 -- 172.21.15.47:0/127822293 learned_addr learned my addr 172.21.15.47:0/127822293 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:34:16.216 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:16.216+0000 7f9e63400640 1 --2- 172.21.15.47:0/127822293 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55aa9099cde0 0x55aa90996cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:34:16.216 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:16.217+0000 7f9e62000640 1 --2- 172.21.15.47:0/127822293 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55aa90994260 0x55aa9099c6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:34:16.216 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:16.217+0000 7f9e62a00640 1 -- 172.21.15.47:0/127822293 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55aa90994260 msgr2=0x55aa9099c6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:34:16.216 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:16.217+0000 7f9e62a00640 1 --2- 172.21.15.47:0/127822293 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55aa90994260 0x55aa9099c6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:16.216 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:16.217+0000 7f9e62a00640 1 -- 172.21.15.47:0/127822293 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55aa9099cde0 msgr2=0x55aa90996cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:34:16.216 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:16.217+0000 7f9e62a00640 1 --2- 172.21.15.47:0/127822293 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55aa9099cde0 0x55aa90996cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:16.216 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:16.217+0000 7f9e62a00640 1 -- 172.21.15.47:0/127822293 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55aa90997fa0 con 0x55aa909938b0 2024-03-21T10:34:16.216 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:16.217+0000 7f9e62000640 1 --2- 172.21.15.47:0/127822293 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55aa90994260 0x55aa9099c6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T10:34:16.217 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:16.217+0000 7f9e63400640 1 --2- 172.21.15.47:0/127822293 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55aa9099cde0 0x55aa90996cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T10:34:16.217 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:16.217+0000 7f9e62a00640 1 --2- 172.21.15.47:0/127822293 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55aa909938b0 0x55aa9099c180 secure :-1 s=READY pgs=1187 cs=0 l=1 rev1=1 crypto rx=0x7f9e54006010 tx=0x7f9e5402bcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fc40ed1fa41fc023 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:34:16.217 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:16.217+0000 7f9e4f400640 1 -- 172.21.15.47:0/127822293 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e540bd5e0 con 0x55aa909938b0 2024-03-21T10:34:16.217 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:16.217+0000 7f9e4f400640 1 -- 172.21.15.47:0/127822293 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f9e540bdc80 con 0x55aa909938b0 2024-03-21T10:34:16.217 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:16.217+0000 7f9e4f400640 1 -- 172.21.15.47:0/127822293 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e54005340 con 0x55aa909938b0 2024-03-21T10:34:16.220 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:16.218+0000 7f9e6cc16740 1 -- 172.21.15.47:0/127822293 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55aa909938b0 msgr2=0x55aa9099c180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:34:16.221 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:16.218+0000 7f9e6cc16740 1 --2- 172.21.15.47:0/127822293 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55aa909938b0 0x55aa9099c180 secure :-1 s=READY pgs=1187 cs=0 l=1 rev1=1 crypto rx=0x7f9e54006010 tx=0x7f9e5402bcc0 comp rx=0 tx=0).stop 2024-03-21T10:34:16.221 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:16.218+0000 7f9e6cc16740 1 -- 172.21.15.47:0/127822293 shutdown_connections 2024-03-21T10:34:16.221 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:16.218+0000 7f9e6cc16740 1 --2- 172.21.15.47:0/127822293 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55aa9099cde0 0x55aa90996cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:16.221 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:16.218+0000 7f9e6cc16740 1 --2- 172.21.15.47:0/127822293 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55aa90994260 0x55aa9099c6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:16.221 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:16.218+0000 7f9e6cc16740 1 --2- 172.21.15.47:0/127822293 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55aa909938b0 0x55aa9099c180 unknown :-1 s=CLOSED pgs=1187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:16.221 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:16.218+0000 7f9e6cc16740 1 -- 172.21.15.47:0/127822293 >> 172.21.15.47:0/127822293 conn(0x55aa9098df80 msgr2=0x55aa9098d160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:34:16.221 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:16.218+0000 7f9e6cc16740 1 -- 172.21.15.47:0/127822293 shutdown_connections 2024-03-21T10:34:16.221 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:16.218+0000 7f9e6cc16740 1 -- 172.21.15.47:0/127822293 wait complete. 2024-03-21T10:34:16.221 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:16.219+0000 7f9e6cc16740 1 -- start start 2024-03-21T10:34:16.221 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:16.219+0000 7f9e6cc16740 1 Processor -- start 2024-03-21T10:34:16.222 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:16.219+0000 7f9e6cc16740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55aa909938b0 0x55aa909a4c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:34:16.222 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:16.219+0000 7f9e6cc16740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55aa90994260 0x55aa909a1010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:34:16.222 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:16.219+0000 7f9e6cc16740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55aa9099cde0 0x55aa909a1550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:34:16.222 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:16.219+0000 7f9e6cc16740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e641b1d00 con 0x55aa909938b0 2024-03-21T10:34:16.222 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:16.219+0000 7f9e6cc16740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e641b16a0 con 0x55aa90994260 2024-03-21T10:34:16.222 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:16.219+0000 7f9e6cc16740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e641aebc0 con 0x55aa9099cde0 2024-03-21T10:34:16.222 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:16.219+0000 7f9e6cc16740 10 client.0 _ll_register_callbacks cb 0x7f9e540b9f20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T10:34:16.222 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:16.220+0000 7f9e62a00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55aa909938b0 0x55aa909a4c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:34:16.222 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:16.220+0000 7f9e62a00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55aa909938b0 0x55aa909a4c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:35218/0 (socket says 172.21.15.47:35218) 2024-03-21T10:34:16.222 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:16.220+0000 7f9e62a00640 1 -- 172.21.15.47:0/2137443672 learned_addr learned my addr 172.21.15.47:0/2137443672 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:34:16.222 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:16.220+0000 7f9e62000640 1 --2- 172.21.15.47:0/2137443672 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55aa90994260 0x55aa909a1010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:34:16.222 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:16.220+0000 7f9e63400640 1 --2- 172.21.15.47:0/2137443672 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55aa9099cde0 0x55aa909a1550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:34:16.222 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:16.220+0000 7f9e62a00640 1 -- 172.21.15.47:0/2137443672 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55aa9099cde0 msgr2=0x55aa909a1550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:34:16.223 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:16.220+0000 7f9e62a00640 1 --2- 172.21.15.47:0/2137443672 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55aa9099cde0 0x55aa909a1550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:16.223 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:16.220+0000 7f9e62a00640 1 -- 172.21.15.47:0/2137443672 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55aa90994260 msgr2=0x55aa909a1010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:34:16.223 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:16.220+0000 7f9e62a00640 1 --2- 172.21.15.47:0/2137443672 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55aa90994260 0x55aa909a1010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:16.223 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:16.220+0000 7f9e62a00640 1 -- 172.21.15.47:0/2137443672 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55aa909a1ac0 con 0x55aa909938b0 2024-03-21T10:34:16.223 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:16.220+0000 7f9e62a00640 1 --2- 172.21.15.47:0/2137443672 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55aa909938b0 0x55aa909a4c40 secure :-1 s=READY pgs=1188 cs=0 l=1 rev1=1 crypto rx=0x7f9e54005ec0 tx=0x7f9e540bd5e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bcc7ea5adeb91e23 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:34:16.223 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:16.220+0000 7f9e4cc00640 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-03-21T10:34:16.223 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:16.221+0000 7f9e4cc00640 1 -- 172.21.15.47:0/2137443672 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e54087070 con 0x55aa909938b0 2024-03-21T10:34:16.223 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:16.221+0000 7f9e4cc00640 1 -- 172.21.15.47:0/2137443672 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(103..103 src has 1..103) v4 ==== 9562+0+0 (secure 0 0 0) 0x7f9e54005340 con 0x55aa909938b0 2024-03-21T10:34:16.223 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:16.221+0000 7f9e6cc16740 1 -- 172.21.15.47:0/2137443672 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f9e540bdc80 con 0x55aa909938b0 2024-03-21T10:34:16.223 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:16.221+0000 7f9e4cc00640 1 -- 172.21.15.47:0/2137443672 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9e540c6690 con 0x55aa909938b0 2024-03-21T10:34:16.223 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:16.221+0000 7f9e6cc16740 10 client.18342 fetch_fsmap learned FSMap version 32 2024-03-21T10:34:16.223 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:16.221+0000 7f9e6cc16740 1 -- 172.21.15.47:0/2137443672 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x55aa909a1ac0 con 0x55aa909938b0 2024-03-21T10:34:16.224 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:16.221+0000 7f9e4cc00640 1 -- 172.21.15.47:0/2137443672 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f9e540bdc80 con 0x55aa909938b0 2024-03-21T10:34:16.224 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:16.222+0000 7f9e4cc00640 1 -- 172.21.15.47:0/2137443672 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 32) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f9e540bcd70 con 0x55aa909938b0 2024-03-21T10:34:16.224 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:16.222+0000 7f9e6cc16740 10 client.18342 fetch_fsmap finished waiting for FSMap version 32 2024-03-21T10:34:16.224 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:16.222+0000 7f9e6cc16740 10 client.18342 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.k 2024-03-21T10:34:16.224 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:16.222+0000 7f9e6cc16740 20 client.18342 populate_metadata read hostname 'smithi047' 2024-03-21T10:34:16.224 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:16.222+0000 7f9e6cc16740 1 --2- 172.21.15.47:0/2137443672 >> [v2:172.21.15.100:6832/2723302038,v1:172.21.15.100:6833/2723302038] conn(0x55aa90a09360 0x55aa90a0b750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:34:16.224 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:16.222+0000 7f9e6cc16740 4 client.18342 mds_command: new command op to 34427 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-21T10:34:16.224 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:16.222+0000 7f9e6cc16740 1 -- 172.21.15.47:0/2137443672 --> [v2:172.21.15.100:6832/2723302038,v1:172.21.15.100:6833/2723302038] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55aa908feb50 con 0x55aa90a09360 2024-03-21T10:34:16.224 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:16.222+0000 7f9e62000640 1 --2- 172.21.15.47:0/2137443672 >> [v2:172.21.15.100:6832/2723302038,v1:172.21.15.100:6833/2723302038] conn(0x55aa90a09360 0x55aa90a0b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:34:16.224 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:16.223+0000 7f9e62000640 1 --2- 172.21.15.47:0/2137443672 >> [v2:172.21.15.100:6832/2723302038,v1:172.21.15.100:6833/2723302038] conn(0x55aa90a09360 0x55aa90a0b750 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=c2e0dc83e5da60e5 server_cookie=8943c32f5a79d0ed in_seq=0 out_seq=0 2024-03-21T10:34:16.224 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:16.223+0000 7f9e4cc00640 10 client.18342 ms_handle_connect on v2:172.21.15.100:6832/2723302038 2024-03-21T10:34:16.225 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:16.223+0000 7f9e4cc00640 1 -- 172.21.15.47:0/2137443672 <== mds.3 v2:172.21.15.100:6832/2723302038 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55aa908feb50 con 0x55aa90a09360 2024-03-21T10:34:16.225 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:16.223+0000 7f9e4cc00640 10 client.18342 handle_command_reply: tid=0 2024-03-21T10:34:16.226 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:16.223+0000 7f9e6cc16740 1 client.18342 shutdown 2024-03-21T10:34:16.226 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:16.223+0000 7f9e6cc16740 1 -- 172.21.15.47:0/2137443672 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55aa909938b0 msgr2=0x55aa909a4c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:34:16.226 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:16.223+0000 7f9e6cc16740 1 --2- 172.21.15.47:0/2137443672 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55aa909938b0 0x55aa909a4c40 secure :-1 s=READY pgs=1188 cs=0 l=1 rev1=1 crypto rx=0x7f9e54005ec0 tx=0x7f9e540bd5e0 comp rx=0 tx=0).stop 2024-03-21T10:34:16.226 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:16.223+0000 7f9e6cc16740 1 -- 172.21.15.47:0/2137443672 shutdown_connections 2024-03-21T10:34:16.226 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:16.223+0000 7f9e6cc16740 1 --2- 172.21.15.47:0/2137443672 >> [v2:172.21.15.100:6832/2723302038,v1:172.21.15.100:6833/2723302038] conn(0x55aa90a09360 0x55aa90a0b750 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:16.226 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:16.223+0000 7f9e6cc16740 1 --2- 172.21.15.47:0/2137443672 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55aa9099cde0 0x55aa909a1550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:16.226 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:16.223+0000 7f9e6cc16740 1 --2- 172.21.15.47:0/2137443672 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55aa90994260 0x55aa909a1010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:16.226 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:16.223+0000 7f9e6cc16740 1 --2- 172.21.15.47:0/2137443672 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55aa909938b0 0x55aa909a4c40 unknown :-1 s=CLOSED pgs=1188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:16.226 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:16.223+0000 7f9e4cc00640 0 client.18342 ms_handle_reset on v2:172.21.15.100:6832/2723302038 2024-03-21T10:34:16.226 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:16.223+0000 7f9e6cc16740 1 -- 172.21.15.47:0/2137443672 >> 172.21.15.47:0/2137443672 conn(0x55aa9098df80 msgr2=0x55aa90994640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:34:16.226 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:16.224+0000 7f9e6cc16740 1 -- 172.21.15.47:0/2137443672 shutdown_connections 2024-03-21T10:34:16.227 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:16.224+0000 7f9e6cc16740 1 -- 172.21.15.47:0/2137443672 wait complete. 2024-03-21T10:34:16.227 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:16.224+0000 7f9e6cc16740 20 client.18342 trim_cache size 0 max 16384 2024-03-21T10:34:16.227 INFO:teuthology.orchestra.run.smithi047.stdout:[] 2024-03-21T10:34:16.228 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:16.224+0000 7f9e63e00640 1 -- 172.21.15.47:0/1835347847 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f9e480a3090 msgr2=0x7f9e480a5550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:34:16.228 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:16.224+0000 7f9e63e00640 1 --2- 172.21.15.47:0/1835347847 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f9e480a3090 0x7f9e480a5550 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f9e5c005fd0 tx=0x7f9e5c009530 comp rx=0 tx=0).stop 2024-03-21T10:34:16.228 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:16.224+0000 7f9e63e00640 1 -- 172.21.15.47:0/1835347847 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9e64102750 msgr2=0x7f9e64199fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:34:16.228 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:16.224+0000 7f9e63e00640 1 --2- 172.21.15.47:0/1835347847 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9e64102750 0x7f9e64199fc0 secure :-1 s=READY pgs=1184 cs=0 l=1 rev1=1 crypto rx=0x7f9e54005940 tx=0x7f9e540045e0 comp rx=0 tx=0).stop 2024-03-21T10:34:16.228 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:16.224+0000 7f9e63e00640 1 -- 172.21.15.47:0/1835347847 shutdown_connections 2024-03-21T10:34:16.228 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:16.224+0000 7f9e63e00640 1 --2- 172.21.15.47:0/1835347847 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9e6410bb70 0x7f9e6419e830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:16.228 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:16.224+0000 7f9e63e00640 1 --2- 172.21.15.47:0/1835347847 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f9e480a3090 0x7f9e480a5550 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:16.228 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:16.224+0000 7f9e63e00640 1 --2- 172.21.15.47:0/1835347847 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9e64103070 0x7f9e6419a500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:16.228 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:16.224+0000 7f9e63e00640 1 --2- 172.21.15.47:0/1835347847 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9e64102750 0x7f9e64199fc0 unknown :-1 s=CLOSED pgs=1184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:16.228 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:16.225+0000 7f9e63e00640 1 -- 172.21.15.47:0/1835347847 >> 172.21.15.47:0/1835347847 conn(0x7f9e640fbf50 msgr2=0x7f9e641013d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:34:16.229 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:16.225+0000 7f9e63e00640 1 -- 172.21.15.47:0/1835347847 shutdown_connections 2024-03-21T10:34:16.229 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:16.225+0000 7f9e63e00640 1 -- 172.21.15.47:0/1835347847 wait complete. 2024-03-21T10:34:16.229 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:34:15 smithi100 ceph-mon[35193]: from='client.? 172.21.15.47:0/3450661005' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-21T10:34:16.237 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:34:15 smithi157 ceph-mon[35581]: from='client.? 172.21.15.47:0/3450661005' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-21T10:34:16.240 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-21T10:34:16.241 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-03-21T10:34:16.453 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:16.454+0000 7f2d2a400640 1 -- 172.21.15.47:0/3182552830 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2d241002e0 msgr2=0x7f2d241006c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:34:16.453 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:16.454+0000 7f2d2a400640 1 --2- 172.21.15.47:0/3182552830 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2d241002e0 0x7f2d241006c0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f2d18009900 tx=0x7f2d1802bc30 comp rx=0 tx=0).stop 2024-03-21T10:34:16.453 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:16.455+0000 7f2d2a400640 1 -- 172.21.15.47:0/3182552830 shutdown_connections 2024-03-21T10:34:16.453 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:16.455+0000 7f2d2a400640 1 --2- 172.21.15.47:0/3182552830 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2d2410de90 0x7f2d24110280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:16.453 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:16.455+0000 7f2d2a400640 1 --2- 172.21.15.47:0/3182552830 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2d24100c90 0x7f2d2410d8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:16.453 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:16.455+0000 7f2d2a400640 1 --2- 172.21.15.47:0/3182552830 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2d241002e0 0x7f2d241006c0 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:16.454 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:16.455+0000 7f2d2a400640 1 -- 172.21.15.47:0/3182552830 >> 172.21.15.47:0/3182552830 conn(0x7f2d240fbef0 msgr2=0x7f2d240fe310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:34:16.454 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:16.455+0000 7f2d2a400640 1 -- 172.21.15.47:0/3182552830 shutdown_connections 2024-03-21T10:34:16.454 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:16.455+0000 7f2d2a400640 1 -- 172.21.15.47:0/3182552830 wait complete. 2024-03-21T10:34:16.455 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:16.456+0000 7f2d2a400640 1 Processor -- start 2024-03-21T10:34:16.455 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:16.456+0000 7f2d2a400640 1 -- start start 2024-03-21T10:34:16.456 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:16.456+0000 7f2d2a400640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2d241002e0 0x7f2d24199f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:34:16.456 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:16.456+0000 7f2d2a400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2d24100c90 0x7f2d2419a470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:34:16.456 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:16.456+0000 7f2d2a400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2d2410de90 0x7f2d2419e7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:34:16.456 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:16.456+0000 7f2d2a400640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d241139b0 con 0x7f2d24100c90 2024-03-21T10:34:16.456 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:16.456+0000 7f2d2a400640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d24113830 con 0x7f2d2410de90 2024-03-21T10:34:16.456 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:16.456+0000 7f2d2a400640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d24113b30 con 0x7f2d241002e0 2024-03-21T10:34:16.456 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:16.457+0000 7f2d23e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2d24100c90 0x7f2d2419a470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:34:16.456 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:16.457+0000 7f2d23e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2d24100c90 0x7f2d2419a470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:35238/0 (socket says 172.21.15.47:35238) 2024-03-21T10:34:16.456 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:16.457+0000 7f2d23e00640 1 -- 172.21.15.47:0/2010882855 learned_addr learned my addr 172.21.15.47:0/2010882855 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:34:16.456 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:16.457+0000 7f2d29a00640 1 --2- 172.21.15.47:0/2010882855 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2d2410de90 0x7f2d2419e7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:34:16.456 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:16.457+0000 7f2d29000640 1 --2- 172.21.15.47:0/2010882855 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2d241002e0 0x7f2d24199f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:34:16.456 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:16.457+0000 7f2d23e00640 1 -- 172.21.15.47:0/2010882855 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2d241002e0 msgr2=0x7f2d24199f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:34:16.456 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:16.457+0000 7f2d23e00640 1 --2- 172.21.15.47:0/2010882855 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2d241002e0 0x7f2d24199f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:16.456 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:16.457+0000 7f2d23e00640 1 -- 172.21.15.47:0/2010882855 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2d2410de90 msgr2=0x7f2d2419e7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:34:16.456 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:16.457+0000 7f2d23e00640 1 --2- 172.21.15.47:0/2010882855 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2d2410de90 0x7f2d2419e7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:16.457 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:16.457+0000 7f2d23e00640 1 -- 172.21.15.47:0/2010882855 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d2419ef20 con 0x7f2d24100c90 2024-03-21T10:34:16.457 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:16.457+0000 7f2d29000640 1 --2- 172.21.15.47:0/2010882855 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2d241002e0 0x7f2d24199f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T10:34:16.457 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:16.457+0000 7f2d23e00640 1 --2- 172.21.15.47:0/2010882855 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2d24100c90 0x7f2d2419a470 secure :-1 s=READY pgs=1189 cs=0 l=1 rev1=1 crypto rx=0x7f2d1000d950 tx=0x7f2d1000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:34:16.457 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:16.458+0000 7f2d21600640 1 -- 172.21.15.47:0/2010882855 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d10017030 con 0x7f2d24100c90 2024-03-21T10:34:16.457 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:16.458+0000 7f2d2a400640 1 -- 172.21.15.47:0/2010882855 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2d2419f210 con 0x7f2d24100c90 2024-03-21T10:34:16.458 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:16.458+0000 7f2d21600640 1 -- 172.21.15.47:0/2010882855 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f2d10004020 con 0x7f2d24100c90 2024-03-21T10:34:16.458 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:16.458+0000 7f2d21600640 1 -- 172.21.15.47:0/2010882855 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d100047a0 con 0x7f2d24100c90 2024-03-21T10:34:16.458 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:16.458+0000 7f2d2a400640 1 -- 172.21.15.47:0/2010882855 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2d241a6aa0 con 0x7f2d24100c90 2024-03-21T10:34:16.459 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:16.461+0000 7f2d2e7dc740 1 Processor -- start 2024-03-21T10:34:16.460 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:16.462+0000 7f2d2e7dc740 1 -- start start 2024-03-21T10:34:16.460 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:16.462+0000 7f2d2e7dc740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564256c898b0 0x564256c89c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:34:16.460 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:16.462+0000 7f2d2e7dc740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564256c8a260 0x564256c92810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:34:16.460 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:16.462+0000 7f2d2e7dc740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x564256c92de0 0x564256c95230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:34:16.461 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:16.462+0000 7f2d2e7dc740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x564256bed2c0 con 0x564256c92de0 2024-03-21T10:34:16.461 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:16.462+0000 7f2d2e7dc740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x564256b95c90 con 0x564256c898b0 2024-03-21T10:34:16.461 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:16.462+0000 7f2d2e7dc740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x564256be9360 con 0x564256c8a260 2024-03-21T10:34:16.461 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:16.462+0000 7f2d29a00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564256c898b0 0x564256c89c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:34:16.461 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:16.462+0000 7f2d29a00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564256c898b0 0x564256c89c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:49936/0 (socket says 172.21.15.47:49936) 2024-03-21T10:34:16.461 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:16.462+0000 7f2d29a00640 1 -- 172.21.15.47:0/1815690482 learned_addr learned my addr 172.21.15.47:0/1815690482 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:34:16.461 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:16.463+0000 7f2d29000640 1 --2- 172.21.15.47:0/1815690482 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564256c8a260 0x564256c92810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:34:16.462 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:16.463+0000 7f2d29a00640 1 -- 172.21.15.47:0/1815690482 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564256c8a260 msgr2=0x564256c92810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:34:16.462 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:16.463+0000 7f2d29a00640 1 --2- 172.21.15.47:0/1815690482 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564256c8a260 0x564256c92810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:16.463 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:16.463+0000 7f2d29a00640 1 -- 172.21.15.47:0/1815690482 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x564256c92de0 msgr2=0x564256c95230 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-21T10:34:16.463 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:16.463+0000 7f2d29a00640 1 --2- 172.21.15.47:0/1815690482 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x564256c92de0 0x564256c95230 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:16.463 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:16.463+0000 7f2d29a00640 1 -- 172.21.15.47:0/1815690482 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564256bba080 con 0x564256c898b0 2024-03-21T10:34:16.463 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:16.463+0000 7f2d29000640 1 --2- 172.21.15.47:0/1815690482 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564256c8a260 0x564256c92810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T10:34:16.463 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:16.463+0000 7f2d29a00640 1 --2- 172.21.15.47:0/1815690482 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564256c898b0 0x564256c89c90 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f2d1c009f30 tx=0x7f2d1c0118d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fbd1db94ae0a8c83 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:34:16.463 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:16.463+0000 7f2d16000640 1 -- 172.21.15.47:0/1815690482 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d1c004d20 con 0x564256c898b0 2024-03-21T10:34:16.463 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:16.464+0000 7f2d16000640 1 -- 172.21.15.47:0/1815690482 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f2d1c004050 con 0x564256c898b0 2024-03-21T10:34:16.463 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:16.464+0000 7f2d16000640 1 -- 172.21.15.47:0/1815690482 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d1c005280 con 0x564256c898b0 2024-03-21T10:34:16.463 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:16.464+0000 7f2d21600640 1 -- 172.21.15.47:0/2010882855 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 27) v1 ==== 137985+0+0 (secure 0 0 0) 0x7f2d100041d0 con 0x7f2d24100c90 2024-03-21T10:34:16.463 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:16.464+0000 7f2d2e7dc740 1 -- 172.21.15.47:0/1815690482 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564256c898b0 msgr2=0x564256c89c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:34:16.463 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:16.464+0000 7f2d2e7dc740 1 --2- 172.21.15.47:0/1815690482 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564256c898b0 0x564256c89c90 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f2d1c009f30 tx=0x7f2d1c0118d0 comp rx=0 tx=0).stop 2024-03-21T10:34:16.463 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:16.464+0000 7f2d2e7dc740 1 -- 172.21.15.47:0/1815690482 shutdown_connections 2024-03-21T10:34:16.463 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:16.464+0000 7f2d2e7dc740 1 --2- 172.21.15.47:0/1815690482 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x564256c92de0 0x564256c95230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:16.463 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:16.464+0000 7f2d2e7dc740 1 --2- 172.21.15.47:0/1815690482 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564256c8a260 0x564256c92810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:16.464 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:16.464+0000 7f2d2e7dc740 1 --2- 172.21.15.47:0/1815690482 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564256c898b0 0x564256c89c90 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:16.464 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:16.464+0000 7f2d2e7dc740 1 -- 172.21.15.47:0/1815690482 >> 172.21.15.47:0/1815690482 conn(0x564256c83f80 msgr2=0x564256c84360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:34:16.464 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:16.464+0000 7f2d2e7dc740 1 -- 172.21.15.47:0/1815690482 shutdown_connections 2024-03-21T10:34:16.464 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:16.464+0000 7f2d2e7dc740 1 -- 172.21.15.47:0/1815690482 wait complete. 2024-03-21T10:34:16.467 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:16.465+0000 7f2d2e7dc740 1 -- start start 2024-03-21T10:34:16.467 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:16.465+0000 7f2d2e7dc740 1 Processor -- start 2024-03-21T10:34:16.467 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:16.466+0000 7f2d21600640 1 --2- 172.21.15.47:0/2010882855 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f2d0c0a2fa0 0x7f2d0c0a5460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:34:16.467 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:16.466+0000 7f2d2e7dc740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564256c898b0 0x564256c92740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:34:16.467 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:16.466+0000 7f2d2e7dc740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x564256c8a260 0x564256c8fad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:34:16.467 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:16.466+0000 7f2d2e7dc740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564256c92de0 0x564256c900e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:34:16.467 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:16.466+0000 7f2d2e7dc740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x564256c95d50 con 0x564256c8a260 2024-03-21T10:34:16.467 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:16.466+0000 7f2d21600640 1 -- 172.21.15.47:0/2010882855 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 9562+0+0 (secure 0 0 0) 0x7f2d100cf6a0 con 0x7f2d24100c90 2024-03-21T10:34:16.468 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:16.466+0000 7f2d2e7dc740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x564256ba90d0 con 0x564256c898b0 2024-03-21T10:34:16.468 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:16.466+0000 7f2d2e7dc740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x564256c95ed0 con 0x564256c92de0 2024-03-21T10:34:16.468 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:16.466+0000 7f2d29a00640 1 --2- 172.21.15.47:0/2010882855 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f2d0c0a2fa0 0x7f2d0c0a5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:34:16.468 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:16.466+0000 7f2d2e7dc740 10 client.0 _ll_register_callbacks cb 0x564256bade80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T10:34:16.468 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:16.466+0000 7f2d29000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564256c898b0 0x564256c92740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:34:16.468 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:16.466+0000 7f2d29000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564256c898b0 0x564256c92740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:49944/0 (socket says 172.21.15.47:49944) 2024-03-21T10:34:16.468 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:16.466+0000 7f2d29000640 1 -- 172.21.15.47:0/2987684945 learned_addr learned my addr 172.21.15.47:0/2987684945 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:34:16.468 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:16.466+0000 7f2d29a00640 1 --2- 172.21.15.47:0/2987684945 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564256c92de0 0x564256c900e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:34:16.468 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:16.466+0000 7f2d23e00640 1 --2- 172.21.15.47:0/2987684945 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x564256c8a260 0x564256c8fad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:34:16.468 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:16.467+0000 7f2d29000640 1 -- 172.21.15.47:0/2987684945 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564256c92de0 msgr2=0x564256c900e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:34:16.468 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:16.467+0000 7f2d29000640 1 --2- 172.21.15.47:0/2987684945 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564256c92de0 0x564256c900e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:16.468 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:16.467+0000 7f2d29000640 1 -- 172.21.15.47:0/2987684945 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x564256c8a260 msgr2=0x564256c8fad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:34:16.468 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:16.467+0000 7f2d29000640 1 --2- 172.21.15.47:0/2987684945 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x564256c8a260 0x564256c8fad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:16.468 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:16.467+0000 7f2d29000640 1 -- 172.21.15.47:0/2987684945 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5642569013f0 con 0x564256c898b0 2024-03-21T10:34:16.469 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:16.467+0000 7f2d29a00640 1 --2- 172.21.15.47:0/2010882855 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f2d0c0a2fa0 0x7f2d0c0a5460 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f2d1c011dd0 tx=0x7f2d1c004e40 comp rx=0 tx=0).ready entity=mgr.15687 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:34:16.469 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:16.467+0000 7f2d29000640 1 --2- 172.21.15.47:0/2987684945 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564256c898b0 0x564256c92740 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f2d1802c170 tx=0x7f2d1802ca90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2bb9e5d7c45a66f1 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:34:16.469 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:16.467+0000 7f2d0aa00640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-21T10:34:16.469 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:16.468+0000 7f2d0aa00640 1 -- 172.21.15.47:0/2987684945 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d18005230 con 0x564256c898b0 2024-03-21T10:34:16.469 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:16.468+0000 7f2d2e7dc740 1 -- 172.21.15.47:0/2987684945 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f2d1c004050 con 0x564256c898b0 2024-03-21T10:34:16.469 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:16.468+0000 7f2d0aa00640 1 -- 172.21.15.47:0/2987684945 <== mon.2 v2:172.21.15.100:3300/0 2 ==== osd_map(103..103 src has 1..103) v4 ==== 9562+0+0 (secure 0 0 0) 0x7f2d180053d0 con 0x564256c898b0 2024-03-21T10:34:16.469 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:16.468+0000 7f2d0aa00640 1 -- 172.21.15.47:0/2987684945 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2d18050050 con 0x564256c898b0 2024-03-21T10:34:16.469 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:16.468+0000 7f2d2e7dc740 10 client.38117 fetch_fsmap learned FSMap version 32 2024-03-21T10:34:16.469 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:16.468+0000 7f2d2e7dc740 1 -- 172.21.15.47:0/2987684945 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x5642569cb7d0 con 0x564256c898b0 2024-03-21T10:34:16.469 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:16.468+0000 7f2d0aa00640 1 -- 172.21.15.47:0/2987684945 <== mon.2 v2:172.21.15.100:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f2d1c004050 con 0x564256c898b0 2024-03-21T10:34:16.469 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:16.469+0000 7f2d0aa00640 1 -- 172.21.15.47:0/2987684945 <== mon.2 v2:172.21.15.100:3300/0 5 ==== fsmap(e 32) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f2d180035f0 con 0x564256c898b0 2024-03-21T10:34:16.469 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:16.469+0000 7f2d2e7dc740 10 client.38117 fetch_fsmap finished waiting for FSMap version 32 2024-03-21T10:34:16.469 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:16.469+0000 7f2d2e7dc740 10 client.38117 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.e 2024-03-21T10:34:16.469 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:16.469+0000 7f2d2e7dc740 20 client.38117 populate_metadata read hostname 'smithi047' 2024-03-21T10:34:16.469 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:16.469+0000 7f2d2e7dc740 1 --2- 172.21.15.47:0/2987684945 >> [v2:172.21.15.100:6834/2597849957,v1:172.21.15.100:6835/2597849957] conn(0x564256ca2c40 0x564256ca5030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:34:16.470 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:16.469+0000 7f2d2e7dc740 4 client.38117 mds_command: new command op to 34454 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T10:34:16.470 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:16.469+0000 7f2d2e7dc740 1 -- 172.21.15.47:0/2987684945 --> [v2:172.21.15.100:6834/2597849957,v1:172.21.15.100:6835/2597849957] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x564256bfa800 con 0x564256ca2c40 2024-03-21T10:34:16.470 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:16.469+0000 7f2d23e00640 1 --2- 172.21.15.47:0/2987684945 >> [v2:172.21.15.100:6834/2597849957,v1:172.21.15.100:6835/2597849957] conn(0x564256ca2c40 0x564256ca5030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:34:16.471 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:16.470+0000 7f2d23e00640 1 --2- 172.21.15.47:0/2987684945 >> [v2:172.21.15.100:6834/2597849957,v1:172.21.15.100:6835/2597849957] conn(0x564256ca2c40 0x564256ca5030 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=36f60214462dd31a server_cookie=11ef4ab02263faec in_seq=0 out_seq=0 2024-03-21T10:34:16.471 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:16.470+0000 7f2d0aa00640 10 client.38117 ms_handle_connect on v2:172.21.15.100:6834/2597849957 2024-03-21T10:34:16.471 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:16.471+0000 7f2d0aa00640 1 -- 172.21.15.47:0/2987684945 <== mds.2 v2:172.21.15.100:6834/2597849957 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x564256bfa800 con 0x564256ca2c40 2024-03-21T10:34:16.471 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:16.471+0000 7f2d0aa00640 10 client.38117 handle_command_reply: tid=0 2024-03-21T10:34:16.471 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:16.471+0000 7f2d2e7dc740 1 client.38117 shutdown 2024-03-21T10:34:16.471 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:16.471+0000 7f2d2e7dc740 1 -- 172.21.15.47:0/2987684945 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564256c898b0 msgr2=0x564256c92740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:34:16.471 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:16.471+0000 7f2d2e7dc740 1 --2- 172.21.15.47:0/2987684945 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564256c898b0 0x564256c92740 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f2d1802c170 tx=0x7f2d1802ca90 comp rx=0 tx=0).stop 2024-03-21T10:34:16.471 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:16.471+0000 7f2d2e7dc740 1 -- 172.21.15.47:0/2987684945 shutdown_connections 2024-03-21T10:34:16.471 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:16.471+0000 7f2d2e7dc740 1 --2- 172.21.15.47:0/2987684945 >> [v2:172.21.15.100:6834/2597849957,v1:172.21.15.100:6835/2597849957] conn(0x564256ca2c40 0x564256ca5030 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:16.472 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:16.471+0000 7f2d2e7dc740 1 --2- 172.21.15.47:0/2987684945 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564256c92de0 0x564256c900e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:16.472 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:16.471+0000 7f2d2e7dc740 1 --2- 172.21.15.47:0/2987684945 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x564256c8a260 0x564256c8fad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:16.472 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:16.471+0000 7f2d2e7dc740 1 --2- 172.21.15.47:0/2987684945 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564256c898b0 0x564256c92740 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:16.472 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:16.471+0000 7f2d2e7dc740 1 -- 172.21.15.47:0/2987684945 >> 172.21.15.47:0/2987684945 conn(0x564256c83f80 msgr2=0x564256c85cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:34:16.472 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:16.471+0000 7f2d0aa00640 0 client.38117 ms_handle_reset on v2:172.21.15.100:6834/2597849957 2024-03-21T10:34:16.472 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:16.471+0000 7f2d2e7dc740 1 -- 172.21.15.47:0/2987684945 shutdown_connections 2024-03-21T10:34:16.472 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:16.471+0000 7f2d2e7dc740 1 -- 172.21.15.47:0/2987684945 wait complete. 2024-03-21T10:34:16.472 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:16.472+0000 7f2d2e7dc740 20 client.38117 trim_cache size 0 max 16384 2024-03-21T10:34:16.484 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:16.486+0000 7f2d2e7dc740 1 Processor -- start 2024-03-21T10:34:16.484 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:16.486+0000 7f2d2e7dc740 1 -- start start 2024-03-21T10:34:16.484 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:16.486+0000 7f2d2e7dc740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564256c898b0 0x564256c85c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:34:16.484 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:16.486+0000 7f2d2e7dc740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564256c8a260 0x564256c82fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:34:16.484 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:16.486+0000 7f2d2e7dc740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x564256c92de0 0x564256c92200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:34:16.484 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:16.486+0000 7f2d2e7dc740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d241b09c0 con 0x564256c92de0 2024-03-21T10:34:16.484 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:16.486+0000 7f2d2e7dc740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d241b2340 con 0x564256c898b0 2024-03-21T10:34:16.484 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:16.486+0000 7f2d2e7dc740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d241b1020 con 0x564256c8a260 2024-03-21T10:34:16.486 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:16.486+0000 7f2d29000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564256c898b0 0x564256c85c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:34:16.486 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:16.486+0000 7f2d29a00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x564256c92de0 0x564256c92200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:34:16.486 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:16.486+0000 7f2d29000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564256c898b0 0x564256c85c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:49954/0 (socket says 172.21.15.47:49954) 2024-03-21T10:34:16.486 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:16.486+0000 7f2d29000640 1 -- 172.21.15.47:0/2033317870 learned_addr learned my addr 172.21.15.47:0/2033317870 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:34:16.486 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:16.487+0000 7f2d23e00640 1 --2- 172.21.15.47:0/2033317870 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564256c8a260 0x564256c82fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:34:16.486 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:16.487+0000 7f2d29a00640 1 -- 172.21.15.47:0/2033317870 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564256c8a260 msgr2=0x564256c82fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:34:16.486 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:16.487+0000 7f2d29a00640 1 --2- 172.21.15.47:0/2033317870 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564256c8a260 0x564256c82fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:16.486 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:16.487+0000 7f2d29a00640 1 -- 172.21.15.47:0/2033317870 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564256c898b0 msgr2=0x564256c85c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:34:16.486 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:16.487+0000 7f2d29a00640 1 --2- 172.21.15.47:0/2033317870 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564256c898b0 0x564256c85c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:16.486 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:16.487+0000 7f2d29a00640 1 -- 172.21.15.47:0/2033317870 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564256ca5610 con 0x564256c92de0 2024-03-21T10:34:16.486 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:16.487+0000 7f2d29000640 1 --2- 172.21.15.47:0/2033317870 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564256c898b0 0x564256c85c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T10:34:16.486 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:16.487+0000 7f2d29a00640 1 --2- 172.21.15.47:0/2033317870 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x564256c92de0 0x564256c92200 secure :-1 s=READY pgs=1190 cs=0 l=1 rev1=1 crypto rx=0x7f2d1c009d40 tx=0x7f2d1c02c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7e186a6ff187ddf6 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:34:16.487 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:16.488+0000 7f2d15600640 1 -- 172.21.15.47:0/2033317870 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d1c004520 con 0x564256c92de0 2024-03-21T10:34:16.487 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:16.488+0000 7f2d15600640 1 -- 172.21.15.47:0/2033317870 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f2d1c0046c0 con 0x564256c92de0 2024-03-21T10:34:16.487 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:16.488+0000 7f2d15600640 1 -- 172.21.15.47:0/2033317870 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d1c028450 con 0x564256c92de0 2024-03-21T10:34:16.487 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:16.488+0000 7f2d2e7dc740 1 -- 172.21.15.47:0/2033317870 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x564256c92de0 msgr2=0x564256c92200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:34:16.488 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:16.488+0000 7f2d2e7dc740 1 --2- 172.21.15.47:0/2033317870 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x564256c92de0 0x564256c92200 secure :-1 s=READY pgs=1190 cs=0 l=1 rev1=1 crypto rx=0x7f2d1c009d40 tx=0x7f2d1c02c000 comp rx=0 tx=0).stop 2024-03-21T10:34:16.488 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:16.488+0000 7f2d2e7dc740 1 -- 172.21.15.47:0/2033317870 shutdown_connections 2024-03-21T10:34:16.488 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:16.489+0000 7f2d2e7dc740 1 --2- 172.21.15.47:0/2033317870 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x564256c92de0 0x564256c92200 unknown :-1 s=CLOSED pgs=1190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:16.488 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:16.489+0000 7f2d2e7dc740 1 --2- 172.21.15.47:0/2033317870 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564256c8a260 0x564256c82fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:16.488 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:16.489+0000 7f2d2e7dc740 1 --2- 172.21.15.47:0/2033317870 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564256c898b0 0x564256c85c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:16.488 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:16.489+0000 7f2d2e7dc740 1 -- 172.21.15.47:0/2033317870 >> 172.21.15.47:0/2033317870 conn(0x564256c83f80 msgr2=0x564256c828d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:34:16.488 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:16.489+0000 7f2d2e7dc740 1 -- 172.21.15.47:0/2033317870 shutdown_connections 2024-03-21T10:34:16.488 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:16.489+0000 7f2d2e7dc740 1 -- 172.21.15.47:0/2033317870 wait complete. 2024-03-21T10:34:16.488 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:16.489+0000 7f2d2e7dc740 1 -- start start 2024-03-21T10:34:16.488 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:16.489+0000 7f2d2e7dc740 1 Processor -- start 2024-03-21T10:34:16.491 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:16.490+0000 7f2d2e7dc740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564256c898b0 0x564256c9ae60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:34:16.491 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:16.490+0000 7f2d2e7dc740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x564256c8a260 0x564256c99200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:34:16.491 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:16.490+0000 7f2d2e7dc740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564256c92de0 0x564256c99740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:34:16.491 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:16.490+0000 7f2d2e7dc740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d241b1ce0 con 0x564256c8a260 2024-03-21T10:34:16.491 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:16.490+0000 7f2d2e7dc740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d241b1680 con 0x564256c898b0 2024-03-21T10:34:16.491 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:16.490+0000 7f2d2e7dc740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d241aeba0 con 0x564256c92de0 2024-03-21T10:34:16.491 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:16.490+0000 7f2d2e7dc740 10 client.0 _ll_register_callbacks cb 0x7f2d1c0093a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T10:34:16.492 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:16.490+0000 7f2d23e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x564256c8a260 0x564256c99200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:34:16.492 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:16.491+0000 7f2d29000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564256c898b0 0x564256c9ae60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:34:16.492 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:16.491+0000 7f2d23e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x564256c8a260 0x564256c99200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:35270/0 (socket says 172.21.15.47:35270) 2024-03-21T10:34:16.492 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:16.491+0000 7f2d23e00640 1 -- 172.21.15.47:0/1664424484 learned_addr learned my addr 172.21.15.47:0/1664424484 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:34:16.492 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:16.491+0000 7f2d29000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564256c898b0 0x564256c9ae60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:49958/0 (socket says 172.21.15.47:49958) 2024-03-21T10:34:16.492 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:16.491+0000 7f2d29a00640 1 --2- 172.21.15.47:0/1664424484 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564256c92de0 0x564256c99740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:34:16.492 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:16.491+0000 7f2d23e00640 1 -- 172.21.15.47:0/1664424484 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564256c92de0 msgr2=0x564256c99740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:34:16.492 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:16.491+0000 7f2d23e00640 1 --2- 172.21.15.47:0/1664424484 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564256c92de0 0x564256c99740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:16.492 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:16.491+0000 7f2d23e00640 1 -- 172.21.15.47:0/1664424484 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564256c898b0 msgr2=0x564256c9ae60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:34:16.492 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:16.491+0000 7f2d23e00640 1 --2- 172.21.15.47:0/1664424484 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564256c898b0 0x564256c9ae60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:16.492 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:16.491+0000 7f2d23e00640 1 -- 172.21.15.47:0/1664424484 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x564256c99f70 con 0x564256c8a260 2024-03-21T10:34:16.492 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:16.491+0000 7f2d29a00640 1 --2- 172.21.15.47:0/1664424484 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564256c92de0 0x564256c99740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T10:34:16.492 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:16.491+0000 7f2d29000640 1 --2- 172.21.15.47:0/1664424484 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564256c898b0 0x564256c9ae60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T10:34:16.492 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:16.491+0000 7f2d23e00640 1 --2- 172.21.15.47:0/1664424484 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x564256c8a260 0x564256c99200 secure :-1 s=READY pgs=1191 cs=0 l=1 rev1=1 crypto rx=0x7f2d10098050 tx=0x7f2d100860b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a9b3f4a0a4359328 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:34:16.492 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:16.491+0000 7f2d0aa00640 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-03-21T10:34:16.493 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:16.492+0000 7f2d0aa00640 1 -- 172.21.15.47:0/1664424484 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d1009b680 con 0x564256c8a260 2024-03-21T10:34:16.493 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:16.492+0000 7f2d0aa00640 1 -- 172.21.15.47:0/1664424484 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(103..103 src has 1..103) v4 ==== 9562+0+0 (secure 0 0 0) 0x7f2d10097cf0 con 0x564256c8a260 2024-03-21T10:34:16.493 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:16.492+0000 7f2d2e7dc740 1 -- 172.21.15.47:0/1664424484 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f2d1c0046c0 con 0x564256c8a260 2024-03-21T10:34:16.493 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:16.492+0000 7f2d0aa00640 1 -- 172.21.15.47:0/1664424484 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2d1009f7d0 con 0x564256c8a260 2024-03-21T10:34:16.493 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:16.492+0000 7f2d2e7dc740 10 client.18357 fetch_fsmap learned FSMap version 32 2024-03-21T10:34:16.493 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:16.492+0000 7f2d2e7dc740 1 -- 172.21.15.47:0/1664424484 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x564256cff430 con 0x564256c8a260 2024-03-21T10:34:16.493 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:16.492+0000 7f2d0aa00640 1 -- 172.21.15.47:0/1664424484 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f2d1c0046c0 con 0x564256c8a260 2024-03-21T10:34:16.493 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:16.492+0000 7f2d0aa00640 1 -- 172.21.15.47:0/1664424484 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 32) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f2d10084bc0 con 0x564256c8a260 2024-03-21T10:34:16.493 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:16.492+0000 7f2d2e7dc740 10 client.18357 fetch_fsmap finished waiting for FSMap version 32 2024-03-21T10:34:16.493 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:16.492+0000 7f2d2e7dc740 10 client.18357 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.e 2024-03-21T10:34:16.493 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:16.492+0000 7f2d2e7dc740 20 client.18357 populate_metadata read hostname 'smithi047' 2024-03-21T10:34:16.493 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:16.492+0000 7f2d2e7dc740 1 --2- 172.21.15.47:0/1664424484 >> [v2:172.21.15.100:6834/2597849957,v1:172.21.15.100:6835/2597849957] conn(0x564256cffb10 0x564256d01f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:34:16.493 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:16.492+0000 7f2d2e7dc740 4 client.18357 mds_command: new command op to 34454 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-21T10:34:16.493 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:16.492+0000 7f2d2e7dc740 1 -- 172.21.15.47:0/1664424484 --> [v2:172.21.15.100:6834/2597849957,v1:172.21.15.100:6835/2597849957] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x564256bf4b50 con 0x564256cffb10 2024-03-21T10:34:16.493 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:16.493+0000 7f2d29000640 1 --2- 172.21.15.47:0/1664424484 >> [v2:172.21.15.100:6834/2597849957,v1:172.21.15.100:6835/2597849957] conn(0x564256cffb10 0x564256d01f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:34:16.495 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:16.493+0000 7f2d29000640 1 --2- 172.21.15.47:0/1664424484 >> [v2:172.21.15.100:6834/2597849957,v1:172.21.15.100:6835/2597849957] conn(0x564256cffb10 0x564256d01f30 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=58b8e69a437519c4 server_cookie=5bbd99f30a973df1 in_seq=0 out_seq=0 2024-03-21T10:34:16.495 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:16.493+0000 7f2d0aa00640 10 client.18357 ms_handle_connect on v2:172.21.15.100:6834/2597849957 2024-03-21T10:34:16.495 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:16.494+0000 7f2d0aa00640 1 -- 172.21.15.47:0/1664424484 <== mds.2 v2:172.21.15.100:6834/2597849957 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x564256bf4b50 con 0x564256cffb10 2024-03-21T10:34:16.495 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:16.494+0000 7f2d0aa00640 10 client.18357 handle_command_reply: tid=0 2024-03-21T10:34:16.495 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:16.494+0000 7f2d2e7dc740 1 client.18357 shutdown 2024-03-21T10:34:16.495 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:16.494+0000 7f2d2e7dc740 1 -- 172.21.15.47:0/1664424484 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x564256c8a260 msgr2=0x564256c99200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:34:16.495 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:16.494+0000 7f2d2e7dc740 1 --2- 172.21.15.47:0/1664424484 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x564256c8a260 0x564256c99200 secure :-1 s=READY pgs=1191 cs=0 l=1 rev1=1 crypto rx=0x7f2d10098050 tx=0x7f2d100860b0 comp rx=0 tx=0).stop 2024-03-21T10:34:16.495 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:16.494+0000 7f2d2e7dc740 1 -- 172.21.15.47:0/1664424484 shutdown_connections 2024-03-21T10:34:16.495 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:16.494+0000 7f2d2e7dc740 1 --2- 172.21.15.47:0/1664424484 >> [v2:172.21.15.100:6834/2597849957,v1:172.21.15.100:6835/2597849957] conn(0x564256cffb10 0x564256d01f30 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:16.495 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:16.494+0000 7f2d2e7dc740 1 --2- 172.21.15.47:0/1664424484 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564256c92de0 0x564256c99740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:16.495 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:16.494+0000 7f2d2e7dc740 1 --2- 172.21.15.47:0/1664424484 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x564256c8a260 0x564256c99200 unknown :-1 s=CLOSED pgs=1191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:16.496 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:16.494+0000 7f2d2e7dc740 1 --2- 172.21.15.47:0/1664424484 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564256c898b0 0x564256c9ae60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:16.496 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:16.494+0000 7f2d0aa00640 0 client.18357 ms_handle_reset on v2:172.21.15.100:6834/2597849957 2024-03-21T10:34:16.496 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:16.494+0000 7f2d2e7dc740 1 -- 172.21.15.47:0/1664424484 >> 172.21.15.47:0/1664424484 conn(0x564256c83f80 msgr2=0x564256c83160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:34:16.496 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:16.494+0000 7f2d2e7dc740 1 -- 172.21.15.47:0/1664424484 shutdown_connections 2024-03-21T10:34:16.496 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:16.494+0000 7f2d2e7dc740 1 -- 172.21.15.47:0/1664424484 wait complete. 2024-03-21T10:34:16.496 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:16.494+0000 7f2d2e7dc740 20 client.18357 trim_cache size 0 max 16384 2024-03-21T10:34:16.496 INFO:teuthology.orchestra.run.smithi047.stdout:[] 2024-03-21T10:34:16.497 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:16.495+0000 7f2d2a400640 1 -- 172.21.15.47:0/2010882855 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f2d0c0a2fa0 msgr2=0x7f2d0c0a5460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:34:16.497 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:16.495+0000 7f2d2a400640 1 --2- 172.21.15.47:0/2010882855 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f2d0c0a2fa0 0x7f2d0c0a5460 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f2d1c011dd0 tx=0x7f2d1c004e40 comp rx=0 tx=0).stop 2024-03-21T10:34:16.497 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:16.495+0000 7f2d2a400640 1 -- 172.21.15.47:0/2010882855 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2d24100c90 msgr2=0x7f2d2419a470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:34:16.497 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:16.495+0000 7f2d2a400640 1 --2- 172.21.15.47:0/2010882855 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2d24100c90 0x7f2d2419a470 secure :-1 s=READY pgs=1189 cs=0 l=1 rev1=1 crypto rx=0x7f2d1000d950 tx=0x7f2d1000de10 comp rx=0 tx=0).stop 2024-03-21T10:34:16.497 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:16.495+0000 7f2d2a400640 1 -- 172.21.15.47:0/2010882855 shutdown_connections 2024-03-21T10:34:16.497 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:16.495+0000 7f2d2a400640 1 --2- 172.21.15.47:0/2010882855 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f2d0c0a2fa0 0x7f2d0c0a5460 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:16.497 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:16.495+0000 7f2d2a400640 1 --2- 172.21.15.47:0/2010882855 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2d2410de90 0x7f2d2419e7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:16.497 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:16.495+0000 7f2d2a400640 1 --2- 172.21.15.47:0/2010882855 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2d24100c90 0x7f2d2419a470 unknown :-1 s=CLOSED pgs=1189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:16.497 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:16.495+0000 7f2d2a400640 1 --2- 172.21.15.47:0/2010882855 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2d241002e0 0x7f2d24199f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:16.497 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:16.495+0000 7f2d2a400640 1 -- 172.21.15.47:0/2010882855 >> 172.21.15.47:0/2010882855 conn(0x7f2d240fbef0 msgr2=0x7f2d240fe2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:34:16.497 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:16.495+0000 7f2d2a400640 1 -- 172.21.15.47:0/2010882855 shutdown_connections 2024-03-21T10:34:16.498 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:16.495+0000 7f2d2a400640 1 -- 172.21.15.47:0/2010882855 wait complete. 2024-03-21T10:34:16.511 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-21T10:34:16.511 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-03-21T10:34:16.729 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:16.730+0000 7f9ceda00640 1 -- 172.21.15.47:0/4082980595 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9ce8068880 msgr2=0x7f9ce8068c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:34:16.729 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:16.730+0000 7f9ceda00640 1 --2- 172.21.15.47:0/4082980595 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9ce8068880 0x7f9ce8068c60 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f9cd8009860 tx=0x7f9cd802bb70 comp rx=0 tx=0).stop 2024-03-21T10:34:16.729 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:16.731+0000 7f9ceda00640 1 -- 172.21.15.47:0/4082980595 shutdown_connections 2024-03-21T10:34:16.729 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:16.731+0000 7f9ceda00640 1 --2- 172.21.15.47:0/4082980595 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9ce810e040 0x7f9ce8110430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:16.730 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:16.731+0000 7f9ceda00640 1 --2- 172.21.15.47:0/4082980595 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9ce80691a0 0x7f9ce810db00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:16.730 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:16.731+0000 7f9ceda00640 1 --2- 172.21.15.47:0/4082980595 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9ce8068880 0x7f9ce8068c60 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:16.730 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:16.731+0000 7f9ceda00640 1 -- 172.21.15.47:0/4082980595 >> 172.21.15.47:0/4082980595 conn(0x7f9ce80776a0 msgr2=0x7f9ce8077ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:34:16.730 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:16.731+0000 7f9ceda00640 1 -- 172.21.15.47:0/4082980595 shutdown_connections 2024-03-21T10:34:16.730 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:16.731+0000 7f9ceda00640 1 -- 172.21.15.47:0/4082980595 wait complete. 2024-03-21T10:34:16.731 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:16.732+0000 7f9ceda00640 1 Processor -- start 2024-03-21T10:34:16.732 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:16.732+0000 7f9ceda00640 1 -- start start 2024-03-21T10:34:16.732 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:16.732+0000 7f9ceda00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9ce8068880 0x7f9ce819a070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:34:16.732 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:16.732+0000 7f9ceda00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9ce80691a0 0x7f9ce819a5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:34:16.732 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:16.732+0000 7f9ceda00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9ce810e040 0x7f9ce819e930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:34:16.732 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:16.732+0000 7f9ceda00640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ce8113ab0 con 0x7f9ce8068880 2024-03-21T10:34:16.732 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:16.732+0000 7f9ceda00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ce8113930 con 0x7f9ce80691a0 2024-03-21T10:34:16.732 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:16.732+0000 7f9ceda00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ce8113c30 con 0x7f9ce810e040 2024-03-21T10:34:16.732 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:16.733+0000 7f9ce6a00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9ce8068880 0x7f9ce819a070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:34:16.732 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:16.733+0000 7f9ce6a00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9ce8068880 0x7f9ce819a070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:35296/0 (socket says 172.21.15.47:35296) 2024-03-21T10:34:16.732 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:16.733+0000 7f9ce6a00640 1 -- 172.21.15.47:0/1432293519 learned_addr learned my addr 172.21.15.47:0/1432293519 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:34:16.732 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:16.733+0000 7f9ce6000640 1 --2- 172.21.15.47:0/1432293519 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9ce80691a0 0x7f9ce819a5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:34:16.732 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:16.733+0000 7f9ce7400640 1 --2- 172.21.15.47:0/1432293519 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9ce810e040 0x7f9ce819e930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:34:16.732 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:16.733+0000 7f9ce6a00640 1 -- 172.21.15.47:0/1432293519 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9ce810e040 msgr2=0x7f9ce819e930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:34:16.733 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:16.733+0000 7f9ce6a00640 1 --2- 172.21.15.47:0/1432293519 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9ce810e040 0x7f9ce819e930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:16.733 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:16.733+0000 7f9ce6a00640 1 -- 172.21.15.47:0/1432293519 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9ce80691a0 msgr2=0x7f9ce819a5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:34:16.733 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:16.733+0000 7f9ce6a00640 1 --2- 172.21.15.47:0/1432293519 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9ce80691a0 0x7f9ce819a5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:16.733 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:16.733+0000 7f9ce6a00640 1 -- 172.21.15.47:0/1432293519 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ce819f060 con 0x7f9ce8068880 2024-03-21T10:34:16.733 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:16.734+0000 7f9ce6a00640 1 --2- 172.21.15.47:0/1432293519 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9ce8068880 0x7f9ce819a070 secure :-1 s=READY pgs=1192 cs=0 l=1 rev1=1 crypto rx=0x7f9cd8037040 tx=0x7f9cd8035a10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:34:16.734 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:16.734+0000 7f9cdf400640 1 -- 172.21.15.47:0/1432293519 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9cd8041020 con 0x7f9ce8068880 2024-03-21T10:34:16.734 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:16.734+0000 7f9ceda00640 1 -- 172.21.15.47:0/1432293519 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9ce819f2f0 con 0x7f9ce8068880 2024-03-21T10:34:16.734 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:16.734+0000 7f9cdf400640 1 -- 172.21.15.47:0/1432293519 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f9cd8035e30 con 0x7f9ce8068880 2024-03-21T10:34:16.734 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:16.734+0000 7f9cdf400640 1 -- 172.21.15.47:0/1432293519 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9cd8039040 con 0x7f9ce8068880 2024-03-21T10:34:16.734 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:16.734+0000 7f9ceda00640 1 -- 172.21.15.47:0/1432293519 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9ce81a6be0 con 0x7f9ce8068880 2024-03-21T10:34:16.736 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:16.738+0000 7f9cf0637740 1 Processor -- start 2024-03-21T10:34:16.736 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:16.739+0000 7f9cf0637740 1 -- start start 2024-03-21T10:34:16.738 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:16.739+0000 7f9cf0637740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56210dcbd6d0 0x56210dcbdab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:34:16.738 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:16.739+0000 7f9cf0637740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56210dcbe080 0x56210dcc6630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:34:16.738 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:16.739+0000 7f9cf0637740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56210dcc6c00 0x56210dcc9050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:34:16.738 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:16.739+0000 7f9cf0637740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x56210dc212c0 con 0x56210dcbd6d0 2024-03-21T10:34:16.738 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:16.739+0000 7f9cf0637740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x56210dbc9c90 con 0x56210dcc6c00 2024-03-21T10:34:16.738 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:16.739+0000 7f9cf0637740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x56210dc1d360 con 0x56210dcbe080 2024-03-21T10:34:16.738 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:16.739+0000 7f9cdf400640 1 -- 172.21.15.47:0/1432293519 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 27) v1 ==== 137985+0+0 (secure 0 0 0) 0x7f9cd8044020 con 0x7f9ce8068880 2024-03-21T10:34:16.738 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:16.739+0000 7f9ce7400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56210dcbd6d0 0x56210dcbdab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:34:16.739 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:16.739+0000 7f9ce7400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56210dcbd6d0 0x56210dcbdab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:35306/0 (socket says 172.21.15.47:35306) 2024-03-21T10:34:16.739 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:16.739+0000 7f9ce7400640 1 -- 172.21.15.47:0/1407392434 learned_addr learned my addr 172.21.15.47:0/1407392434 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:34:16.739 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:16.740+0000 7f9ce6000640 1 --2- 172.21.15.47:0/1407392434 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56210dcc6c00 0x56210dcc9050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:34:16.739 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:16.740+0000 7f9ce7400640 1 -- 172.21.15.47:0/1407392434 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56210dcbe080 msgr2=0x56210dcc6630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:34:16.739 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:16.740+0000 7f9ce7400640 1 --2- 172.21.15.47:0/1407392434 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56210dcbe080 0x56210dcc6630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:16.739 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:16.740+0000 7f9ce7400640 1 -- 172.21.15.47:0/1407392434 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56210dcc6c00 msgr2=0x56210dcc9050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:34:16.739 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:16.740+0000 7f9ce7400640 1 --2- 172.21.15.47:0/1407392434 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56210dcc6c00 0x56210dcc9050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:16.739 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:16.740+0000 7f9ce7400640 1 -- 172.21.15.47:0/1407392434 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56210dbee080 con 0x56210dcbd6d0 2024-03-21T10:34:16.741 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:16.740+0000 7f9ce7400640 1 --2- 172.21.15.47:0/1407392434 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56210dcbd6d0 0x56210dcbdab0 secure :-1 s=READY pgs=1193 cs=0 l=1 rev1=1 crypto rx=0x7f9ce0009fa0 tx=0x7f9ce000f890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5129e5f7e6cecf6 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:34:16.741 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:16.740+0000 7f9cd3e00640 1 -- 172.21.15.47:0/1407392434 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ce0004b20 con 0x56210dcbd6d0 2024-03-21T10:34:16.741 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:16.741+0000 7f9cd3e00640 1 -- 172.21.15.47:0/1407392434 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f9ce0004cc0 con 0x56210dcbd6d0 2024-03-21T10:34:16.741 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:16.741+0000 7f9cd3e00640 1 -- 172.21.15.47:0/1407392434 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ce00054a0 con 0x56210dcbd6d0 2024-03-21T10:34:16.741 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:16.741+0000 7f9cdf400640 1 --2- 172.21.15.47:0/1432293519 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f9ccc0a3090 0x7f9ccc0a5550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:34:16.741 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:16.741+0000 7f9cdf400640 1 -- 172.21.15.47:0/1432293519 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 9562+0+0 (secure 0 0 0) 0x7f9cd80f1510 con 0x7f9ce8068880 2024-03-21T10:34:16.741 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:16.741+0000 7f9ce6a00640 1 --2- 172.21.15.47:0/1432293519 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f9ccc0a3090 0x7f9ccc0a5550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:34:16.741 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:16.741+0000 7f9cf0637740 1 -- 172.21.15.47:0/1407392434 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56210dcbd6d0 msgr2=0x56210dcbdab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:34:16.741 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:16.741+0000 7f9cf0637740 1 --2- 172.21.15.47:0/1407392434 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56210dcbd6d0 0x56210dcbdab0 secure :-1 s=READY pgs=1193 cs=0 l=1 rev1=1 crypto rx=0x7f9ce0009fa0 tx=0x7f9ce000f890 comp rx=0 tx=0).stop 2024-03-21T10:34:16.742 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:16.742+0000 7f9cf0637740 1 -- 172.21.15.47:0/1407392434 shutdown_connections 2024-03-21T10:34:16.742 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:16.742+0000 7f9cf0637740 1 --2- 172.21.15.47:0/1407392434 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56210dcc6c00 0x56210dcc9050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:16.742 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:16.742+0000 7f9cf0637740 1 --2- 172.21.15.47:0/1407392434 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56210dcbe080 0x56210dcc6630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:16.742 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:16.742+0000 7f9cf0637740 1 --2- 172.21.15.47:0/1407392434 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56210dcbd6d0 0x56210dcbdab0 unknown :-1 s=CLOSED pgs=1193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:16.742 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:16.742+0000 7f9cf0637740 1 -- 172.21.15.47:0/1407392434 >> 172.21.15.47:0/1407392434 conn(0x56210dcb7da0 msgr2=0x56210dcb8180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:34:16.742 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:16.742+0000 7f9cf0637740 1 -- 172.21.15.47:0/1407392434 shutdown_connections 2024-03-21T10:34:16.742 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:16.742+0000 7f9cf0637740 1 -- 172.21.15.47:0/1407392434 wait complete. 2024-03-21T10:34:16.742 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:16.742+0000 7f9ce6a00640 1 --2- 172.21.15.47:0/1432293519 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f9ccc0a3090 0x7f9ccc0a5550 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x56210dcbd1a0 tx=0x7f9cd80057c0 comp rx=0 tx=0).ready entity=mgr.15687 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:34:16.742 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:16.742+0000 7f9cf0637740 1 -- start start 2024-03-21T10:34:16.742 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:16.743+0000 7f9cf0637740 1 Processor -- start 2024-03-21T10:34:16.745 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:16.743+0000 7f9cf0637740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56210dcbd6d0 0x56210dcc6480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:34:16.745 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:16.743+0000 7f9cf0637740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56210dcbe080 0x56210dcc3810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:34:16.745 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:16.743+0000 7f9cf0637740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56210dcc6c00 0x56210dcc3d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:34:16.745 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:16.743+0000 7f9cf0637740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x56210dcc9b70 con 0x56210dcbd6d0 2024-03-21T10:34:16.745 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:16.744+0000 7f9cf0637740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x56210dbdd0d0 con 0x56210dcc6c00 2024-03-21T10:34:16.745 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:16.744+0000 7f9cf0637740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x56210dcc9cf0 con 0x56210dcbe080 2024-03-21T10:34:16.745 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:16.744+0000 7f9cf0637740 10 client.0 _ll_register_callbacks cb 0x56210dbe1e80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T10:34:16.746 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:16.744+0000 7f9ce7400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56210dcbd6d0 0x56210dcc6480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:34:16.746 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:16.744+0000 7f9ce7400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56210dcbd6d0 0x56210dcc6480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:35312/0 (socket says 172.21.15.47:35312) 2024-03-21T10:34:16.746 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:16.744+0000 7f9ce7400640 1 -- 172.21.15.47:0/2225247025 learned_addr learned my addr 172.21.15.47:0/2225247025 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:34:16.746 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:16.744+0000 7f9ce7400640 1 --2- 172.21.15.47:0/2225247025 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56210dcc6c00 0x56210dcc3d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:34:16.746 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:16.744+0000 7f9ce6000640 1 --2- 172.21.15.47:0/2225247025 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56210dcbe080 0x56210dcc3810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:34:16.746 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:16.744+0000 7f9ce7400640 1 -- 172.21.15.47:0/2225247025 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56210dcbe080 msgr2=0x56210dcc3810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:34:16.746 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:16.744+0000 7f9ce7400640 1 --2- 172.21.15.47:0/2225247025 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56210dcbe080 0x56210dcc3810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:16.746 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:16.744+0000 7f9ce7400640 1 -- 172.21.15.47:0/2225247025 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56210dcc6c00 msgr2=0x56210dcc3d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:34:16.746 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:16.744+0000 7f9ce7400640 1 --2- 172.21.15.47:0/2225247025 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56210dcc6c00 0x56210dcc3d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:16.746 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:16.744+0000 7f9ce7400640 1 -- 172.21.15.47:0/2225247025 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56210d9353f0 con 0x56210dcbd6d0 2024-03-21T10:34:16.746 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:16.745+0000 7f9ce7400640 1 --2- 172.21.15.47:0/2225247025 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56210dcbd6d0 0x56210dcc6480 secure :-1 s=READY pgs=1194 cs=0 l=1 rev1=1 crypto rx=0x7f9ce0009db0 tx=0x7f9ce0005dc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6b50f220714e5538 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:34:16.746 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:16.745+0000 7f9cd0c00640 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-03-21T10:34:16.746 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:16.745+0000 7f9cd0c00640 1 -- 172.21.15.47:0/2225247025 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ce002c060 con 0x56210dcbd6d0 2024-03-21T10:34:16.746 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:16.745+0000 7f9cd0c00640 1 -- 172.21.15.47:0/2225247025 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(103..103 src has 1..103) v4 ==== 9562+0+0 (secure 0 0 0) 0x7f9ce000cb50 con 0x56210dcbd6d0 2024-03-21T10:34:16.747 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:16.745+0000 7f9cf0637740 1 -- 172.21.15.47:0/2225247025 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f9ce0004cc0 con 0x56210dcbd6d0 2024-03-21T10:34:16.747 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:16.745+0000 7f9cd0c00640 1 -- 172.21.15.47:0/2225247025 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9ce0033050 con 0x56210dcbd6d0 2024-03-21T10:34:16.747 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:16.745+0000 7f9cf0637740 10 client.18369 fetch_fsmap learned FSMap version 32 2024-03-21T10:34:16.747 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:16.745+0000 7f9cf0637740 1 -- 172.21.15.47:0/2225247025 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x56210d9ff7d0 con 0x56210dcbd6d0 2024-03-21T10:34:16.747 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:16.746+0000 7f9cd0c00640 1 -- 172.21.15.47:0/2225247025 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f9ce0004cc0 con 0x56210dcbd6d0 2024-03-21T10:34:16.747 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:16.746+0000 7f9cd0c00640 1 -- 172.21.15.47:0/2225247025 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 32) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f9ce000b040 con 0x56210dcbd6d0 2024-03-21T10:34:16.747 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:16.746+0000 7f9cf0637740 10 client.18369 fetch_fsmap finished waiting for FSMap version 32 2024-03-21T10:34:16.747 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:16.746+0000 7f9cf0637740 10 client.18369 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-03-21T10:34:16.747 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:16.746+0000 7f9cf0637740 20 client.18369 populate_metadata read hostname 'smithi047' 2024-03-21T10:34:16.747 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:16.746+0000 7f9cf0637740 1 --2- 172.21.15.47:0/2225247025 >> [v2:172.21.15.100:6836/3497644162,v1:172.21.15.100:6837/3497644162] conn(0x56210dcd6990 0x56210dcd8d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:34:16.747 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:16.746+0000 7f9cf0637740 4 client.18369 mds_command: new command op to 34493 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T10:34:16.747 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:16.746+0000 7f9cf0637740 1 -- 172.21.15.47:0/2225247025 --> [v2:172.21.15.100:6836/3497644162,v1:172.21.15.100:6837/3497644162] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56210dc2e800 con 0x56210dcd6990 2024-03-21T10:34:16.747 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:16.747+0000 7f9ce6000640 1 --2- 172.21.15.47:0/2225247025 >> [v2:172.21.15.100:6836/3497644162,v1:172.21.15.100:6837/3497644162] conn(0x56210dcd6990 0x56210dcd8d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:34:16.747 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:16.747+0000 7f9ce6000640 1 --2- 172.21.15.47:0/2225247025 >> [v2:172.21.15.100:6836/3497644162,v1:172.21.15.100:6837/3497644162] conn(0x56210dcd6990 0x56210dcd8d80 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=87e37808ae31702 server_cookie=5750e10b208b0e57 in_seq=0 out_seq=0 2024-03-21T10:34:16.748 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:16.747+0000 7f9cd0c00640 10 client.18369 ms_handle_connect on v2:172.21.15.100:6836/3497644162 2024-03-21T10:34:16.748 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:16.748+0000 7f9cd0c00640 1 -- 172.21.15.47:0/2225247025 <== mds.1 v2:172.21.15.100:6836/3497644162 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x56210dc2e800 con 0x56210dcd6990 2024-03-21T10:34:16.749 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:16.748+0000 7f9cd0c00640 10 client.18369 handle_command_reply: tid=0 2024-03-21T10:34:16.749 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:16.749+0000 7f9cf0637740 1 client.18369 shutdown 2024-03-21T10:34:16.749 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:16.749+0000 7f9cf0637740 1 -- 172.21.15.47:0/2225247025 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56210dcbd6d0 msgr2=0x56210dcc6480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:34:16.749 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:16.749+0000 7f9cf0637740 1 --2- 172.21.15.47:0/2225247025 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56210dcbd6d0 0x56210dcc6480 secure :-1 s=READY pgs=1194 cs=0 l=1 rev1=1 crypto rx=0x7f9ce0009db0 tx=0x7f9ce0005dc0 comp rx=0 tx=0).stop 2024-03-21T10:34:16.749 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:16.749+0000 7f9cf0637740 1 -- 172.21.15.47:0/2225247025 shutdown_connections 2024-03-21T10:34:16.749 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:16.749+0000 7f9cf0637740 1 --2- 172.21.15.47:0/2225247025 >> [v2:172.21.15.100:6836/3497644162,v1:172.21.15.100:6837/3497644162] conn(0x56210dcd6990 0x56210dcd8d80 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:16.749 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:16.749+0000 7f9cf0637740 1 --2- 172.21.15.47:0/2225247025 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56210dcc6c00 0x56210dcc3d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:16.749 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:16.749+0000 7f9cf0637740 1 --2- 172.21.15.47:0/2225247025 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56210dcbe080 0x56210dcc3810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:16.749 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:16.749+0000 7f9cf0637740 1 --2- 172.21.15.47:0/2225247025 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56210dcbd6d0 0x56210dcc6480 unknown :-1 s=CLOSED pgs=1194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:16.749 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:16.749+0000 7f9cd0c00640 0 client.18369 ms_handle_reset on v2:172.21.15.100:6836/3497644162 2024-03-21T10:34:16.749 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:16.749+0000 7f9cf0637740 1 -- 172.21.15.47:0/2225247025 >> 172.21.15.47:0/2225247025 conn(0x56210dcb7da0 msgr2=0x56210dcb9af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:34:16.749 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:16.749+0000 7f9cf0637740 1 -- 172.21.15.47:0/2225247025 shutdown_connections 2024-03-21T10:34:16.749 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:16.749+0000 7f9cf0637740 1 -- 172.21.15.47:0/2225247025 wait complete. 2024-03-21T10:34:16.749 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:16.749+0000 7f9cf0637740 20 client.18369 trim_cache size 0 max 16384 2024-03-21T10:34:16.761 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:16.763+0000 7f9cf0637740 1 Processor -- start 2024-03-21T10:34:16.761 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:16.763+0000 7f9cf0637740 1 -- start start 2024-03-21T10:34:16.761 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:16.763+0000 7f9cf0637740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56210dcbd6d0 0x56210dcc5f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:34:16.762 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:16.763+0000 7f9cf0637740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56210dcbe080 0x56210dcc6480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:34:16.762 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:16.763+0000 7f9cf0637740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56210dcc6c00 0x56210dcc0af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:34:16.762 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:16.763+0000 7f9cf0637740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ce81b0c30 con 0x56210dcbd6d0 2024-03-21T10:34:16.762 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:16.763+0000 7f9cf0637740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ce81b25b0 con 0x56210dcc6c00 2024-03-21T10:34:16.762 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:16.763+0000 7f9cf0637740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ce81b1290 con 0x56210dcbe080 2024-03-21T10:34:16.764 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:16.764+0000 7f9ce7400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56210dcbd6d0 0x56210dcc5f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:34:16.764 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:16.764+0000 7f9ce7400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56210dcbd6d0 0x56210dcc5f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:35314/0 (socket says 172.21.15.47:35314) 2024-03-21T10:34:16.764 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:16.764+0000 7f9ce7400640 1 -- 172.21.15.47:0/1621419359 learned_addr learned my addr 172.21.15.47:0/1621419359 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:34:16.764 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:16.764+0000 7f9ce7400640 1 --2- 172.21.15.47:0/1621419359 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56210dcc6c00 0x56210dcc0af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:34:16.764 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:16.764+0000 7f9ce6000640 1 --2- 172.21.15.47:0/1621419359 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56210dcbe080 0x56210dcc6480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:34:16.764 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:16.764+0000 7f9ce7400640 1 -- 172.21.15.47:0/1621419359 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56210dcbe080 msgr2=0x56210dcc6480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:34:16.764 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:16.764+0000 7f9ce7400640 1 --2- 172.21.15.47:0/1621419359 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56210dcbe080 0x56210dcc6480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:16.764 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:16.764+0000 7f9ce7400640 1 -- 172.21.15.47:0/1621419359 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56210dcc6c00 msgr2=0x56210dcc0af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:34:16.764 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:16.764+0000 7f9ce7400640 1 --2- 172.21.15.47:0/1621419359 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56210dcc6c00 0x56210dcc0af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:16.764 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:16.764+0000 7f9ce7400640 1 -- 172.21.15.47:0/1621419359 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56210dcc1dc0 con 0x56210dcbd6d0 2024-03-21T10:34:16.764 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:16.764+0000 7f9ce6000640 1 --2- 172.21.15.47:0/1621419359 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56210dcbe080 0x56210dcc6480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T10:34:16.765 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:16.764+0000 7f9ce7400640 1 --2- 172.21.15.47:0/1621419359 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56210dcbd6d0 0x56210dcc5f40 secure :-1 s=READY pgs=1195 cs=0 l=1 rev1=1 crypto rx=0x7f9ce0006010 tx=0x7f9ce00021f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=36fce39a4d9295dd server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:34:16.765 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:16.765+0000 7f9cd3400640 1 -- 172.21.15.47:0/1621419359 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ce002c070 con 0x56210dcbd6d0 2024-03-21T10:34:16.765 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:16.765+0000 7f9cd3400640 1 -- 172.21.15.47:0/1621419359 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f9ce0009e70 con 0x56210dcbd6d0 2024-03-21T10:34:16.765 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:16.765+0000 7f9cd3400640 1 -- 172.21.15.47:0/1621419359 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ce0025d70 con 0x56210dcbd6d0 2024-03-21T10:34:16.765 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:16.765+0000 7f9cf0637740 1 -- 172.21.15.47:0/1621419359 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56210dcbd6d0 msgr2=0x56210dcc5f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:34:16.765 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:16.765+0000 7f9cf0637740 1 --2- 172.21.15.47:0/1621419359 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56210dcbd6d0 0x56210dcc5f40 secure :-1 s=READY pgs=1195 cs=0 l=1 rev1=1 crypto rx=0x7f9ce0006010 tx=0x7f9ce00021f0 comp rx=0 tx=0).stop 2024-03-21T10:34:16.765 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:16.765+0000 7f9cf0637740 1 -- 172.21.15.47:0/1621419359 shutdown_connections 2024-03-21T10:34:16.765 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:16.765+0000 7f9cf0637740 1 --2- 172.21.15.47:0/1621419359 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56210dcc6c00 0x56210dcc0af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:16.765 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:16.765+0000 7f9cf0637740 1 --2- 172.21.15.47:0/1621419359 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56210dcbe080 0x56210dcc6480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:16.765 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:16.765+0000 7f9cf0637740 1 --2- 172.21.15.47:0/1621419359 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56210dcbd6d0 0x56210dcc5f40 unknown :-1 s=CLOSED pgs=1195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:16.765 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:16.765+0000 7f9cf0637740 1 -- 172.21.15.47:0/1621419359 >> 172.21.15.47:0/1621419359 conn(0x56210dcb7da0 msgr2=0x56210dcb6eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:34:16.765 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:16.766+0000 7f9cf0637740 1 -- 172.21.15.47:0/1621419359 shutdown_connections 2024-03-21T10:34:16.765 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:16.766+0000 7f9cf0637740 1 -- 172.21.15.47:0/1621419359 wait complete. 2024-03-21T10:34:16.765 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:16.766+0000 7f9cf0637740 1 -- start start 2024-03-21T10:34:16.768 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:16.766+0000 7f9cf0637740 1 Processor -- start 2024-03-21T10:34:16.768 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:16.767+0000 7f9cf0637740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56210dcbd6d0 0x56210dcdb740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:34:16.768 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:16.767+0000 7f9cf0637740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56210dcbe080 0x56210dcdbc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:34:16.768 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:16.767+0000 7f9cf0637740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56210dcc6c00 0x56210dcd9ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:34:16.768 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:16.767+0000 7f9cf0637740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ce81b1f50 con 0x56210dcc6c00 2024-03-21T10:34:16.769 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:16.767+0000 7f9cf0637740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ce81b18f0 con 0x56210dcbe080 2024-03-21T10:34:16.769 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:16.767+0000 7f9cf0637740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ce81aee10 con 0x56210dcbd6d0 2024-03-21T10:34:16.769 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:16.767+0000 7f9cf0637740 10 client.0 _ll_register_callbacks cb 0x7f9ce000ff50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T10:34:16.769 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:16.767+0000 7f9ce7400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56210dcc6c00 0x56210dcd9ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:34:16.769 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:16.767+0000 7f9ce6000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56210dcbe080 0x56210dcdbc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:34:16.769 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:16.767+0000 7f9ce7400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56210dcc6c00 0x56210dcd9ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:35316/0 (socket says 172.21.15.47:35316) 2024-03-21T10:34:16.769 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:16.767+0000 7f9ce7400640 1 -- 172.21.15.47:0/3563209250 learned_addr learned my addr 172.21.15.47:0/3563209250 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:34:16.769 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:16.767+0000 7f9ce6000640 1 --2- 172.21.15.47:0/3563209250 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56210dcbe080 0x56210dcdbc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:50004/0 (socket says 172.21.15.47:50004) 2024-03-21T10:34:16.769 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:16.767+0000 7f9ce7400640 1 --2- 172.21.15.47:0/3563209250 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56210dcbd6d0 0x56210dcdb740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:34:16.769 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:16.767+0000 7f9ce7400640 1 -- 172.21.15.47:0/3563209250 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56210dcbd6d0 msgr2=0x56210dcdb740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:34:16.769 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:16.767+0000 7f9ce7400640 1 --2- 172.21.15.47:0/3563209250 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56210dcbd6d0 0x56210dcdb740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:16.769 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:16.767+0000 7f9ce7400640 1 -- 172.21.15.47:0/3563209250 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56210dcbe080 msgr2=0x56210dcdbc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:34:16.769 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:16.767+0000 7f9ce7400640 1 --2- 172.21.15.47:0/3563209250 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56210dcbe080 0x56210dcdbc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:16.769 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:16.767+0000 7f9ce7400640 1 -- 172.21.15.47:0/3563209250 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56210dcda4e0 con 0x56210dcc6c00 2024-03-21T10:34:16.770 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:16.767+0000 7f9ce6000640 1 --2- 172.21.15.47:0/3563209250 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56210dcbe080 0x56210dcdbc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T10:34:16.770 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:16.768+0000 7f9ce7400640 1 --2- 172.21.15.47:0/3563209250 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56210dcc6c00 0x56210dcd9ce0 secure :-1 s=READY pgs=1196 cs=0 l=1 rev1=1 crypto rx=0x7f9ce00095c0 tx=0x7f9ce0004cc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f3138015be3213d5 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:34:16.770 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:16.768+0000 7f9cd0c00640 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-03-21T10:34:16.770 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:16.768+0000 7f9cd0c00640 1 -- 172.21.15.47:0/3563209250 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ce002c040 con 0x56210dcc6c00 2024-03-21T10:34:16.770 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:16.768+0000 7f9cd0c00640 1 -- 172.21.15.47:0/3563209250 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(103..103 src has 1..103) v4 ==== 9562+0+0 (secure 0 0 0) 0x7f9ce003d040 con 0x56210dcc6c00 2024-03-21T10:34:16.770 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:16.768+0000 7f9cf0637740 1 -- 172.21.15.47:0/3563209250 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f9ce0009e70 con 0x56210dcc6c00 2024-03-21T10:34:16.770 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:16.768+0000 7f9cd0c00640 1 -- 172.21.15.47:0/3563209250 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9ce0025c30 con 0x56210dcc6c00 2024-03-21T10:34:16.770 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:16.768+0000 7f9cf0637740 10 client.18375 fetch_fsmap learned FSMap version 32 2024-03-21T10:34:16.770 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:16.768+0000 7f9cf0637740 1 -- 172.21.15.47:0/3563209250 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x56210dd3b2b0 con 0x56210dcc6c00 2024-03-21T10:34:16.770 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:16.769+0000 7f9cd0c00640 1 -- 172.21.15.47:0/3563209250 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f9ce0009e70 con 0x56210dcc6c00 2024-03-21T10:34:16.770 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:16.769+0000 7f9cd0c00640 1 -- 172.21.15.47:0/3563209250 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 32) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f9ce000b040 con 0x56210dcc6c00 2024-03-21T10:34:16.770 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:16.769+0000 7f9cf0637740 10 client.18375 fetch_fsmap finished waiting for FSMap version 32 2024-03-21T10:34:16.770 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:16.769+0000 7f9cf0637740 10 client.18375 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-03-21T10:34:16.770 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:16.769+0000 7f9cf0637740 20 client.18375 populate_metadata read hostname 'smithi047' 2024-03-21T10:34:16.771 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:16.769+0000 7f9cf0637740 1 --2- 172.21.15.47:0/3563209250 >> [v2:172.21.15.100:6836/3497644162,v1:172.21.15.100:6837/3497644162] conn(0x56210dd3b810 0x56210dd3dc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:34:16.771 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:16.769+0000 7f9cf0637740 4 client.18375 mds_command: new command op to 34493 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-21T10:34:16.771 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:16.769+0000 7f9cf0637740 1 -- 172.21.15.47:0/3563209250 --> [v2:172.21.15.100:6836/3497644162,v1:172.21.15.100:6837/3497644162] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x56210dcc40e0 con 0x56210dd3b810 2024-03-21T10:34:16.771 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:16.770+0000 7f9ce6000640 1 --2- 172.21.15.47:0/3563209250 >> [v2:172.21.15.100:6836/3497644162,v1:172.21.15.100:6837/3497644162] conn(0x56210dd3b810 0x56210dd3dc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:34:16.771 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:16.770+0000 7f9ce6000640 1 --2- 172.21.15.47:0/3563209250 >> [v2:172.21.15.100:6836/3497644162,v1:172.21.15.100:6837/3497644162] conn(0x56210dd3b810 0x56210dd3dc30 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=c8c147546f623a57 server_cookie=1d087458b7fb8c29 in_seq=0 out_seq=0 2024-03-21T10:34:16.771 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:16.770+0000 7f9cd0c00640 10 client.18375 ms_handle_connect on v2:172.21.15.100:6836/3497644162 2024-03-21T10:34:16.772 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:16.771+0000 7f9cd0c00640 1 -- 172.21.15.47:0/3563209250 <== mds.1 v2:172.21.15.100:6836/3497644162 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x56210dcc40e0 con 0x56210dd3b810 2024-03-21T10:34:16.772 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:16.771+0000 7f9cd0c00640 10 client.18375 handle_command_reply: tid=0 2024-03-21T10:34:16.772 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:16.771+0000 7f9cf0637740 1 client.18375 shutdown 2024-03-21T10:34:16.772 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:16.771+0000 7f9cf0637740 1 -- 172.21.15.47:0/3563209250 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56210dcc6c00 msgr2=0x56210dcd9ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:34:16.772 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:16.771+0000 7f9cf0637740 1 --2- 172.21.15.47:0/3563209250 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56210dcc6c00 0x56210dcd9ce0 secure :-1 s=READY pgs=1196 cs=0 l=1 rev1=1 crypto rx=0x7f9ce00095c0 tx=0x7f9ce0004cc0 comp rx=0 tx=0).stop 2024-03-21T10:34:16.772 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:16.771+0000 7f9cf0637740 1 -- 172.21.15.47:0/3563209250 shutdown_connections 2024-03-21T10:34:16.772 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:16.771+0000 7f9cf0637740 1 --2- 172.21.15.47:0/3563209250 >> [v2:172.21.15.100:6836/3497644162,v1:172.21.15.100:6837/3497644162] conn(0x56210dd3b810 0x56210dd3dc30 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:16.772 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:16.771+0000 7f9cf0637740 1 --2- 172.21.15.47:0/3563209250 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56210dcc6c00 0x56210dcd9ce0 unknown :-1 s=CLOSED pgs=1196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:16.772 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:16.771+0000 7f9cf0637740 1 --2- 172.21.15.47:0/3563209250 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56210dcbe080 0x56210dcdbc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:16.773 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:16.771+0000 7f9cf0637740 1 --2- 172.21.15.47:0/3563209250 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56210dcbd6d0 0x56210dcdb740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:16.773 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:16.771+0000 7f9cf0637740 1 -- 172.21.15.47:0/3563209250 >> 172.21.15.47:0/3563209250 conn(0x56210dcb7da0 msgr2=0x56210dcbe480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:34:16.773 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:16.771+0000 7f9cd0c00640 0 client.18375 ms_handle_reset on v2:172.21.15.100:6836/3497644162 2024-03-21T10:34:16.773 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:16.771+0000 7f9cf0637740 1 -- 172.21.15.47:0/3563209250 shutdown_connections 2024-03-21T10:34:16.773 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:16.771+0000 7f9cf0637740 1 -- 172.21.15.47:0/3563209250 wait complete. 2024-03-21T10:34:16.773 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:16.771+0000 7f9cf0637740 20 client.18375 trim_cache size 0 max 16384 2024-03-21T10:34:16.773 INFO:teuthology.orchestra.run.smithi047.stdout:[] 2024-03-21T10:34:16.774 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:16.772+0000 7f9ceda00640 1 -- 172.21.15.47:0/1432293519 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f9ccc0a3090 msgr2=0x7f9ccc0a5550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:34:16.774 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:16.772+0000 7f9ceda00640 1 --2- 172.21.15.47:0/1432293519 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f9ccc0a3090 0x7f9ccc0a5550 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x56210dcbd1a0 tx=0x7f9cd80057c0 comp rx=0 tx=0).stop 2024-03-21T10:34:16.774 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:16.772+0000 7f9ceda00640 1 -- 172.21.15.47:0/1432293519 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9ce8068880 msgr2=0x7f9ce819a070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:34:16.774 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:16.772+0000 7f9ceda00640 1 --2- 172.21.15.47:0/1432293519 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9ce8068880 0x7f9ce819a070 secure :-1 s=READY pgs=1192 cs=0 l=1 rev1=1 crypto rx=0x7f9cd8037040 tx=0x7f9cd8035a10 comp rx=0 tx=0).stop 2024-03-21T10:34:16.774 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:16.772+0000 7f9ceda00640 1 -- 172.21.15.47:0/1432293519 shutdown_connections 2024-03-21T10:34:16.774 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:16.772+0000 7f9ceda00640 1 --2- 172.21.15.47:0/1432293519 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9ce810e040 0x7f9ce819e930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:16.774 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:16.772+0000 7f9ceda00640 1 --2- 172.21.15.47:0/1432293519 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f9ccc0a3090 0x7f9ccc0a5550 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:16.774 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:16.772+0000 7f9ceda00640 1 --2- 172.21.15.47:0/1432293519 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9ce80691a0 0x7f9ce819a5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:16.774 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:16.772+0000 7f9ceda00640 1 --2- 172.21.15.47:0/1432293519 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9ce8068880 0x7f9ce819a070 unknown :-1 s=CLOSED pgs=1192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:16.774 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:16.772+0000 7f9ceda00640 1 -- 172.21.15.47:0/1432293519 >> 172.21.15.47:0/1432293519 conn(0x7f9ce80776a0 msgr2=0x7f9ce810f720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:34:16.775 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:16.772+0000 7f9ceda00640 1 -- 172.21.15.47:0/1432293519 shutdown_connections 2024-03-21T10:34:16.775 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:16.772+0000 7f9ceda00640 1 -- 172.21.15.47:0/1432293519 wait complete. 2024-03-21T10:34:16.788 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-21T10:34:16.788 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-03-21T10:34:16.992 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:16.994+0000 7fa33b000640 1 -- 172.21.15.47:0/2869299436 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa334104200 msgr2=0x7fa334110710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:34:16.993 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:16.994+0000 7fa33b000640 1 --2- 172.21.15.47:0/2869299436 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa334104200 0x7fa334110710 secure :-1 s=READY pgs=1197 cs=0 l=1 rev1=1 crypto rx=0x7fa32c009cd0 tx=0x7fa32c02c0f0 comp rx=0 tx=0).stop 2024-03-21T10:34:16.993 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:16.994+0000 7fa33b000640 1 -- 172.21.15.47:0/2869299436 shutdown_connections 2024-03-21T10:34:16.993 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:16.994+0000 7fa33b000640 1 --2- 172.21.15.47:0/2869299436 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa334104200 0x7fa334110710 unknown :-1 s=CLOSED pgs=1197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:16.993 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:16.994+0000 7fa33b000640 1 --2- 172.21.15.47:0/2869299436 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa334103840 0x7fa334103cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:16.993 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:16.994+0000 7fa33b000640 1 --2- 172.21.15.47:0/2869299436 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa334109850 0x7fa334109c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:16.993 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:16.994+0000 7fa33b000640 1 -- 172.21.15.47:0/2869299436 >> 172.21.15.47:0/2869299436 conn(0x7fa3340ff310 msgr2=0x7fa334101730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:34:16.993 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:16.994+0000 7fa33b000640 1 -- 172.21.15.47:0/2869299436 shutdown_connections 2024-03-21T10:34:16.993 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:16.994+0000 7fa33b000640 1 -- 172.21.15.47:0/2869299436 wait complete. 2024-03-21T10:34:16.994 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:16.996+0000 7fa33b000640 1 Processor -- start 2024-03-21T10:34:16.994 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:16.996+0000 7fa33b000640 1 -- start start 2024-03-21T10:34:16.995 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:16.996+0000 7fa33b000640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa334103840 0x7fa33419a050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:34:16.996 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:16.996+0000 7fa33b000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa334104200 0x7fa33419a590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:34:16.996 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:16.997+0000 7fa33b000640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa334109850 0x7fa33419e910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:34:16.996 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:16.997+0000 7fa33b000640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa334113b10 con 0x7fa334103840 2024-03-21T10:34:16.996 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:16.997+0000 7fa33b000640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa334113990 con 0x7fa334104200 2024-03-21T10:34:16.996 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:16.997+0000 7fa33b000640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa334113c90 con 0x7fa334109850 2024-03-21T10:34:16.996 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:16.997+0000 7fa333e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa334103840 0x7fa33419a050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:34:16.996 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:16.997+0000 7fa333e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa334103840 0x7fa33419a050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:35334/0 (socket says 172.21.15.47:35334) 2024-03-21T10:34:16.996 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:16.997+0000 7fa333e00640 1 -- 172.21.15.47:0/3780603671 learned_addr learned my addr 172.21.15.47:0/3780603671 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:34:16.996 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:16.997+0000 7fa333400640 1 --2- 172.21.15.47:0/3780603671 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa334104200 0x7fa33419a590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:34:16.996 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:16.997+0000 7fa339000640 1 --2- 172.21.15.47:0/3780603671 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa334109850 0x7fa33419e910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:34:16.996 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:16.997+0000 7fa333e00640 1 -- 172.21.15.47:0/3780603671 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa334109850 msgr2=0x7fa33419e910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:34:16.996 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:16.997+0000 7fa333e00640 1 --2- 172.21.15.47:0/3780603671 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa334109850 0x7fa33419e910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:16.996 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:16.997+0000 7fa333e00640 1 -- 172.21.15.47:0/3780603671 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa334104200 msgr2=0x7fa33419a590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:34:16.996 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:16.997+0000 7fa333e00640 1 --2- 172.21.15.47:0/3780603671 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa334104200 0x7fa33419a590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:16.997 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:16.997+0000 7fa333e00640 1 -- 172.21.15.47:0/3780603671 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa33419f040 con 0x7fa334103840 2024-03-21T10:34:16.997 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:16.997+0000 7fa339000640 1 --2- 172.21.15.47:0/3780603671 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa334109850 0x7fa33419e910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T10:34:16.997 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:16.998+0000 7fa333e00640 1 --2- 172.21.15.47:0/3780603671 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa334103840 0x7fa33419a050 secure :-1 s=READY pgs=1198 cs=0 l=1 rev1=1 crypto rx=0x7fa32400e9f0 tx=0x7fa32400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:34:16.997 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:16.998+0000 7fa330c00640 1 -- 172.21.15.47:0/3780603671 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa32400cd80 con 0x7fa334103840 2024-03-21T10:34:16.997 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:16.998+0000 7fa330c00640 1 -- 172.21.15.47:0/3780603671 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fa3240108e0 con 0x7fa334103840 2024-03-21T10:34:16.998 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:16.998+0000 7fa330c00640 1 -- 172.21.15.47:0/3780603671 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa324018920 con 0x7fa334103840 2024-03-21T10:34:16.998 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:16.998+0000 7fa33b000640 1 -- 172.21.15.47:0/3780603671 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa33419f330 con 0x7fa334103840 2024-03-21T10:34:16.998 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:16.998+0000 7fa33b000640 1 -- 172.21.15.47:0/3780603671 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa3341a6bc0 con 0x7fa334103840 2024-03-21T10:34:17.000 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:17.002+0000 7fa330c00640 1 -- 172.21.15.47:0/3780603671 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 27) v1 ==== 137985+0+0 (secure 0 0 0) 0x7fa324010a90 con 0x7fa334103840 2024-03-21T10:34:17.000 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:17.002+0000 7fa33ddb8740 1 Processor -- start 2024-03-21T10:34:17.000 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:17.002+0000 7fa33ddb8740 1 -- start start 2024-03-21T10:34:17.002 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:17.002+0000 7fa33ddb8740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55fab039a6d0 0x55fab039aab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:34:17.002 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:17.002+0000 7fa33ddb8740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fab039b080 0x55fab03a3630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:34:17.002 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:17.003+0000 7fa33ddb8740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fab03a3c00 0x55fab03a6050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:34:17.002 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:17.003+0000 7fa33ddb8740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fab02fe2c0 con 0x55fab039a6d0 2024-03-21T10:34:17.002 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:17.003+0000 7fa33ddb8740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fab02a6c90 con 0x55fab039b080 2024-03-21T10:34:17.003 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:17.003+0000 7fa33ddb8740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fab02fa360 con 0x55fab03a3c00 2024-03-21T10:34:17.003 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:17.003+0000 7fa339000640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55fab039a6d0 0x55fab039aab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:34:17.003 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:17.003+0000 7fa339000640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55fab039a6d0 0x55fab039aab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:35344/0 (socket says 172.21.15.47:35344) 2024-03-21T10:34:17.003 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:17.003+0000 7fa333e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fab039b080 0x55fab03a3630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:34:17.003 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:17.003+0000 7fa339000640 1 -- 172.21.15.47:0/1312651523 learned_addr learned my addr 172.21.15.47:0/1312651523 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:34:17.003 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:17.003+0000 7fa333400640 1 --2- 172.21.15.47:0/1312651523 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fab03a3c00 0x55fab03a6050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:34:17.003 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:17.003+0000 7fa333e00640 1 -- 172.21.15.47:0/1312651523 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fab03a3c00 msgr2=0x55fab03a6050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:34:17.003 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:17.003+0000 7fa333e00640 1 --2- 172.21.15.47:0/1312651523 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fab03a3c00 0x55fab03a6050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:17.003 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:17.003+0000 7fa333e00640 1 -- 172.21.15.47:0/1312651523 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55fab039a6d0 msgr2=0x55fab039aab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:34:17.003 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:17.003+0000 7fa333e00640 1 --2- 172.21.15.47:0/1312651523 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55fab039a6d0 0x55fab039aab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:17.003 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:17.003+0000 7fa333e00640 1 -- 172.21.15.47:0/1312651523 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55fab02cb080 con 0x55fab039b080 2024-03-21T10:34:17.003 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:17.003+0000 7fa339000640 1 --2- 172.21.15.47:0/1312651523 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55fab039a6d0 0x55fab039aab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T10:34:17.003 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:17.004+0000 7fa330c00640 1 --2- 172.21.15.47:0/3780603671 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7fa3180a3110 0x7fa3180a55d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:34:17.003 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:17.004+0000 7fa333e00640 1 --2- 172.21.15.47:0/1312651523 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fab039b080 0x55fab03a3630 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fa3240cd200 tx=0x7fa3240d85f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e7cf646a2259464 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:34:17.003 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:17.004+0000 7fa339000640 1 --2- 172.21.15.47:0/3780603671 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7fa3180a3110 0x7fa3180a55d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:34:17.004 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:17.004+0000 7fa330c00640 1 -- 172.21.15.47:0/3780603671 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 9562+0+0 (secure 0 0 0) 0x7fa3240cfec0 con 0x7fa334103840 2024-03-21T10:34:17.004 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:17.004+0000 7fa329600640 1 -- 172.21.15.47:0/1312651523 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa324098140 con 0x55fab039b080 2024-03-21T10:34:17.004 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:17.004+0000 7fa329600640 1 -- 172.21.15.47:0/1312651523 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fa324010de0 con 0x55fab039b080 2024-03-21T10:34:17.004 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:17.004+0000 7fa329600640 1 -- 172.21.15.47:0/1312651523 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa32409b800 con 0x55fab039b080 2024-03-21T10:34:17.006 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:17.005+0000 7fa33ddb8740 1 -- 172.21.15.47:0/1312651523 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fab039b080 msgr2=0x55fab03a3630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:34:17.006 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:17.005+0000 7fa33ddb8740 1 --2- 172.21.15.47:0/1312651523 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fab039b080 0x55fab03a3630 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fa3240cd200 tx=0x7fa3240d85f0 comp rx=0 tx=0).stop 2024-03-21T10:34:17.007 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:17.005+0000 7fa339000640 1 --2- 172.21.15.47:0/3780603671 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7fa3180a3110 0x7fa3180a55d0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fa32c006010 tx=0x7fa32c004a80 comp rx=0 tx=0).ready entity=mgr.15687 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:34:17.007 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:17.005+0000 7fa33ddb8740 1 -- 172.21.15.47:0/1312651523 shutdown_connections 2024-03-21T10:34:17.007 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:17.005+0000 7fa33ddb8740 1 --2- 172.21.15.47:0/1312651523 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fab03a3c00 0x55fab03a6050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:17.007 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:17.005+0000 7fa33ddb8740 1 --2- 172.21.15.47:0/1312651523 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fab039b080 0x55fab03a3630 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:17.007 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:17.005+0000 7fa33ddb8740 1 --2- 172.21.15.47:0/1312651523 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55fab039a6d0 0x55fab039aab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:17.007 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:17.005+0000 7fa33ddb8740 1 -- 172.21.15.47:0/1312651523 >> 172.21.15.47:0/1312651523 conn(0x55fab0394da0 msgr2=0x55fab0395180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:34:17.007 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:17.005+0000 7fa33ddb8740 1 -- 172.21.15.47:0/1312651523 shutdown_connections 2024-03-21T10:34:17.007 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:17.005+0000 7fa33ddb8740 1 -- 172.21.15.47:0/1312651523 wait complete. 2024-03-21T10:34:17.007 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:17.006+0000 7fa33ddb8740 1 -- start start 2024-03-21T10:34:17.007 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:17.006+0000 7fa33ddb8740 1 Processor -- start 2024-03-21T10:34:17.007 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:17.007+0000 7fa33ddb8740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55fab039a6d0 0x55fab03a3490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:34:17.007 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:17.007+0000 7fa33ddb8740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fab039b080 0x55fab03a0890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:34:17.007 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:17.007+0000 7fa33ddb8740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fab03a3c00 0x55fab03a0e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:34:17.008 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:17.007+0000 7fa33ddb8740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fab03a6b70 con 0x55fab039a6d0 2024-03-21T10:34:17.008 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:17.007+0000 7fa33ddb8740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fab02ba0d0 con 0x55fab03a3c00 2024-03-21T10:34:17.008 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:17.007+0000 7fa33ddb8740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fab03a6cf0 con 0x55fab039b080 2024-03-21T10:34:17.008 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:17.007+0000 7fa33ddb8740 10 client.0 _ll_register_callbacks cb 0x55fab02bee80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T10:34:17.008 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:17.007+0000 7fa333e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55fab039a6d0 0x55fab03a3490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:34:17.008 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:17.007+0000 7fa333e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55fab039a6d0 0x55fab03a3490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:35356/0 (socket says 172.21.15.47:35356) 2024-03-21T10:34:17.008 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:17.007+0000 7fa333e00640 1 -- 172.21.15.47:0/977116719 learned_addr learned my addr 172.21.15.47:0/977116719 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:34:17.008 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:17.007+0000 7fa339000640 1 --2- 172.21.15.47:0/977116719 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fab03a3c00 0x55fab03a0e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:34:17.008 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:17.007+0000 7fa333400640 1 --2- 172.21.15.47:0/977116719 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fab039b080 0x55fab03a0890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:34:17.008 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:17.008+0000 7fa333e00640 1 -- 172.21.15.47:0/977116719 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fab039b080 msgr2=0x55fab03a0890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:34:17.008 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:17.008+0000 7fa333e00640 1 --2- 172.21.15.47:0/977116719 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fab039b080 0x55fab03a0890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:17.008 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:17.008+0000 7fa333e00640 1 -- 172.21.15.47:0/977116719 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fab03a3c00 msgr2=0x55fab03a0e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:34:17.008 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:17.008+0000 7fa333e00640 1 --2- 172.21.15.47:0/977116719 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fab03a3c00 0x55fab03a0e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:17.008 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:17.008+0000 7fa333e00640 1 -- 172.21.15.47:0/977116719 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55fab00123f0 con 0x55fab039a6d0 2024-03-21T10:34:17.008 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:17.008+0000 7fa333e00640 1 --2- 172.21.15.47:0/977116719 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55fab039a6d0 0x55fab03a3490 secure :-1 s=READY pgs=1199 cs=0 l=1 rev1=1 crypto rx=0x7fa32400e9c0 tx=0x7fa324009700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=35fdbb9a725904d7 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:34:17.009 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:17.008+0000 7fa31e000640 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-03-21T10:34:17.009 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:17.008+0000 7fa31e000640 1 -- 172.21.15.47:0/977116719 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3240b4020 con 0x55fab039a6d0 2024-03-21T10:34:17.009 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:17.008+0000 7fa31e000640 1 -- 172.21.15.47:0/977116719 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(103..103 src has 1..103) v4 ==== 9562+0+0 (secure 0 0 0) 0x7fa32409e040 con 0x55fab039a6d0 2024-03-21T10:34:17.009 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:17.008+0000 7fa33ddb8740 1 -- 172.21.15.47:0/977116719 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55fab030b800 con 0x55fab039a6d0 2024-03-21T10:34:17.009 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:17.008+0000 7fa31e000640 1 -- 172.21.15.47:0/977116719 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa32409b9e0 con 0x55fab039a6d0 2024-03-21T10:34:17.009 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:17.009+0000 7fa33ddb8740 10 client.18387 fetch_fsmap learned FSMap version 32 2024-03-21T10:34:17.009 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:17.009+0000 7fa33ddb8740 1 -- 172.21.15.47:0/977116719 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x55fab00dc7d0 con 0x55fab039a6d0 2024-03-21T10:34:17.011 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:17.009+0000 7fa31e000640 1 -- 172.21.15.47:0/977116719 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x55fab030b800 con 0x55fab039a6d0 2024-03-21T10:34:17.011 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:17.009+0000 7fa31e000640 1 -- 172.21.15.47:0/977116719 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 32) v1 ==== 7942+0+0 (secure 0 0 0) 0x7fa324098510 con 0x55fab039a6d0 2024-03-21T10:34:17.011 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:17.009+0000 7fa33ddb8740 10 client.18387 fetch_fsmap finished waiting for FSMap version 32 2024-03-21T10:34:17.011 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:17.009+0000 7fa33ddb8740 10 client.18387 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-21T10:34:17.011 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:17.009+0000 7fa33ddb8740 20 client.18387 populate_metadata read hostname 'smithi047' 2024-03-21T10:34:17.011 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:17.009+0000 7fa33ddb8740 1 --2- 172.21.15.47:0/977116719 >> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] conn(0x55fab03b38f0 0x55fab03b5ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:34:17.011 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:17.009+0000 7fa33ddb8740 4 client.18387 mds_command: new command op to 34535 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T10:34:17.012 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:17.009+0000 7fa33ddb8740 1 -- 172.21.15.47:0/977116719 --> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55fab0305b50 con 0x55fab03b38f0 2024-03-21T10:34:17.012 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:17.010+0000 7fa333400640 1 --2- 172.21.15.47:0/977116719 >> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] conn(0x55fab03b38f0 0x55fab03b5ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:34:17.012 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:17.010+0000 7fa333400640 1 --2- 172.21.15.47:0/977116719 >> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] conn(0x55fab03b38f0 0x55fab03b5ce0 crc :-1 s=READY pgs=201 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bbda94f5a76bbefe server_cookie=2bb920955f309bdd in_seq=0 out_seq=0 2024-03-21T10:34:17.012 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:17.010+0000 7fa31e000640 10 client.18387 ms_handle_connect on v2:172.21.15.100:6838/4173168440 2024-03-21T10:34:17.012 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:17.012+0000 7fa31e000640 1 -- 172.21.15.47:0/977116719 <== mds.0 v2:172.21.15.100:6838/4173168440 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55fab0305b50 con 0x55fab03b38f0 2024-03-21T10:34:17.012 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:17.012+0000 7fa31e000640 10 client.18387 handle_command_reply: tid=0 2024-03-21T10:34:17.012 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:17.012+0000 7fa33ddb8740 1 client.18387 shutdown 2024-03-21T10:34:17.012 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:17.012+0000 7fa33ddb8740 1 -- 172.21.15.47:0/977116719 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55fab039a6d0 msgr2=0x55fab03a3490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:34:17.012 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:17.012+0000 7fa33ddb8740 1 --2- 172.21.15.47:0/977116719 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55fab039a6d0 0x55fab03a3490 secure :-1 s=READY pgs=1199 cs=0 l=1 rev1=1 crypto rx=0x7fa32400e9c0 tx=0x7fa324009700 comp rx=0 tx=0).stop 2024-03-21T10:34:17.012 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:17.012+0000 7fa33ddb8740 1 -- 172.21.15.47:0/977116719 shutdown_connections 2024-03-21T10:34:17.012 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:17.012+0000 7fa33ddb8740 1 --2- 172.21.15.47:0/977116719 >> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] conn(0x55fab03b38f0 0x55fab03b5ce0 crc :-1 s=READY pgs=201 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:17.012 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:17.012+0000 7fa33ddb8740 1 --2- 172.21.15.47:0/977116719 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fab03a3c00 0x55fab03a0e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:17.012 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:17.012+0000 7fa33ddb8740 1 --2- 172.21.15.47:0/977116719 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fab039b080 0x55fab03a0890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:17.012 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:17.012+0000 7fa33ddb8740 1 --2- 172.21.15.47:0/977116719 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55fab039a6d0 0x55fab03a3490 unknown :-1 s=CLOSED pgs=1199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:17.012 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:17.012+0000 7fa33ddb8740 1 -- 172.21.15.47:0/977116719 >> 172.21.15.47:0/977116719 conn(0x55fab0394da0 msgr2=0x55fab0396af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:34:17.013 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:17.012+0000 7fa31e000640 0 client.18387 ms_handle_reset on v2:172.21.15.100:6838/4173168440 2024-03-21T10:34:17.013 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:17.013+0000 7fa33ddb8740 1 -- 172.21.15.47:0/977116719 shutdown_connections 2024-03-21T10:34:17.013 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:17.013+0000 7fa33ddb8740 1 -- 172.21.15.47:0/977116719 wait complete. 2024-03-21T10:34:17.013 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:17.013+0000 7fa33ddb8740 20 client.18387 trim_cache size 0 max 16384 2024-03-21T10:34:17.024 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:17.026+0000 7fa33ddb8740 1 Processor -- start 2024-03-21T10:34:17.024 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:17.026+0000 7fa33ddb8740 1 -- start start 2024-03-21T10:34:17.025 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:17.027+0000 7fa33ddb8740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fab039a6d0 0x55fab0393620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:34:17.025 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:17.027+0000 7fa33ddb8740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55fab039b080 0x55fab03a0040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:34:17.025 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:17.027+0000 7fa33ddb8740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fab03a3c00 0x55fab03a2fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:34:17.026 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:17.027+0000 7fa33ddb8740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3341b0d40 con 0x55fab039b080 2024-03-21T10:34:17.026 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:17.027+0000 7fa33ddb8740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3341b26c0 con 0x55fab03a3c00 2024-03-21T10:34:17.026 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:17.027+0000 7fa33ddb8740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3341b13a0 con 0x55fab039a6d0 2024-03-21T10:34:17.026 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:17.027+0000 7fa333400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55fab039b080 0x55fab03a0040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:34:17.026 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:17.027+0000 7fa333400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55fab039b080 0x55fab03a0040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:35372/0 (socket says 172.21.15.47:35372) 2024-03-21T10:34:17.026 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:17.027+0000 7fa333400640 1 -- 172.21.15.47:0/4231595494 learned_addr learned my addr 172.21.15.47:0/4231595494 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:34:17.026 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:17.027+0000 7fa339000640 1 --2- 172.21.15.47:0/4231595494 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fab03a3c00 0x55fab03a2fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:34:17.026 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:17.027+0000 7fa333400640 1 -- 172.21.15.47:0/4231595494 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fab039a6d0 msgr2=0x55fab0393620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:34:17.026 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:17.027+0000 7fa333400640 1 --2- 172.21.15.47:0/4231595494 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fab039a6d0 0x55fab0393620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:17.026 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:17.027+0000 7fa333400640 1 -- 172.21.15.47:0/4231595494 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fab03a3c00 msgr2=0x55fab03a2fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:34:17.026 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:17.027+0000 7fa333400640 1 --2- 172.21.15.47:0/4231595494 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fab03a3c00 0x55fab03a2fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:17.026 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:17.027+0000 7fa333400640 1 -- 172.21.15.47:0/4231595494 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55fab039c420 con 0x55fab039b080 2024-03-21T10:34:17.028 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:17.028+0000 7fa333400640 1 --2- 172.21.15.47:0/4231595494 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55fab039b080 0x55fab03a0040 secure :-1 s=READY pgs=1200 cs=0 l=1 rev1=1 crypto rx=0x7fa32000b780 tx=0x7fa32000bc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=114599dd1bdae440 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:34:17.028 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:17.028+0000 7fa328c00640 1 -- 172.21.15.47:0/4231595494 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa32001dde0 con 0x55fab039b080 2024-03-21T10:34:17.028 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:17.028+0000 7fa328c00640 1 -- 172.21.15.47:0/4231595494 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fa3200054d0 con 0x55fab039b080 2024-03-21T10:34:17.028 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:17.028+0000 7fa328c00640 1 -- 172.21.15.47:0/4231595494 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa320004bc0 con 0x55fab039b080 2024-03-21T10:34:17.029 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:17.029+0000 7fa33ddb8740 1 -- 172.21.15.47:0/4231595494 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55fab039b080 msgr2=0x55fab03a0040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:34:17.029 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:17.029+0000 7fa33ddb8740 1 --2- 172.21.15.47:0/4231595494 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55fab039b080 0x55fab03a0040 secure :-1 s=READY pgs=1200 cs=0 l=1 rev1=1 crypto rx=0x7fa32000b780 tx=0x7fa32000bc60 comp rx=0 tx=0).stop 2024-03-21T10:34:17.029 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:17.029+0000 7fa33ddb8740 1 -- 172.21.15.47:0/4231595494 shutdown_connections 2024-03-21T10:34:17.029 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:17.029+0000 7fa33ddb8740 1 --2- 172.21.15.47:0/4231595494 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fab03a3c00 0x55fab03a2fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:17.029 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:17.029+0000 7fa33ddb8740 1 --2- 172.21.15.47:0/4231595494 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55fab039b080 0x55fab03a0040 unknown :-1 s=CLOSED pgs=1200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:17.029 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:17.029+0000 7fa33ddb8740 1 --2- 172.21.15.47:0/4231595494 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fab039a6d0 0x55fab0393620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:17.029 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:17.029+0000 7fa33ddb8740 1 -- 172.21.15.47:0/4231595494 >> 172.21.15.47:0/4231595494 conn(0x55fab0394da0 msgr2=0x55fab039bd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:34:17.029 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:17.029+0000 7fa33ddb8740 1 -- 172.21.15.47:0/4231595494 shutdown_connections 2024-03-21T10:34:17.029 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:17.029+0000 7fa33ddb8740 1 -- 172.21.15.47:0/4231595494 wait complete. 2024-03-21T10:34:17.029 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:17.029+0000 7fa33ddb8740 1 -- start start 2024-03-21T10:34:17.029 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:17.029+0000 7fa33ddb8740 1 Processor -- start 2024-03-21T10:34:17.029 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:17.030+0000 7fa33ddb8740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fab039a6d0 0x55fab03b5df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:34:17.030 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:17.030+0000 7fa33ddb8740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fab039b080 0x55fab03b6330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:34:17.030 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:17.030+0000 7fa33ddb8740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55fab03a3c00 0x55fab03b6870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:34:17.030 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:17.030+0000 7fa33ddb8740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3341b2060 con 0x55fab03a3c00 2024-03-21T10:34:17.030 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:17.030+0000 7fa33ddb8740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3341b1a00 con 0x55fab039a6d0 2024-03-21T10:34:17.030 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:17.030+0000 7fa33ddb8740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3341aef20 con 0x55fab039b080 2024-03-21T10:34:17.030 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:17.030+0000 7fa33ddb8740 10 client.0 _ll_register_callbacks cb 0x7fa32c037070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T10:34:17.030 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:17.030+0000 7fa333e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fab039a6d0 0x55fab03b5df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:34:17.030 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:17.030+0000 7fa339000640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55fab03a3c00 0x55fab03b6870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:34:17.030 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:17.030+0000 7fa333e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fab039a6d0 0x55fab03b5df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:50054/0 (socket says 172.21.15.47:50054) 2024-03-21T10:34:17.030 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:17.030+0000 7fa333e00640 1 -- 172.21.15.47:0/695191750 learned_addr learned my addr 172.21.15.47:0/695191750 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:34:17.030 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:17.030+0000 7fa333400640 1 --2- 172.21.15.47:0/695191750 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fab039b080 0x55fab03b6330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:34:17.033 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:17.031+0000 7fa339000640 1 -- 172.21.15.47:0/695191750 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fab039b080 msgr2=0x55fab03b6330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:34:17.033 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:17.031+0000 7fa339000640 1 --2- 172.21.15.47:0/695191750 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fab039b080 0x55fab03b6330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:17.033 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:17.031+0000 7fa339000640 1 -- 172.21.15.47:0/695191750 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fab039a6d0 msgr2=0x55fab03b5df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:34:17.033 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:17.031+0000 7fa339000640 1 --2- 172.21.15.47:0/695191750 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fab039a6d0 0x55fab03b5df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:17.033 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:17.031+0000 7fa339000640 1 -- 172.21.15.47:0/695191750 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55fab040ded0 con 0x55fab03a3c00 2024-03-21T10:34:17.033 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:17.031+0000 7fa333e00640 1 --2- 172.21.15.47:0/695191750 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fab039a6d0 0x55fab03b5df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T10:34:17.033 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:17.031+0000 7fa339000640 1 --2- 172.21.15.47:0/695191750 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55fab03a3c00 0x55fab03b6870 secure :-1 s=READY pgs=1201 cs=0 l=1 rev1=1 crypto rx=0x7fa32c02c700 tx=0x7fa32c0095c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=94fed70779b5630b server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:34:17.033 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:17.031+0000 7fa31e000640 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-03-21T10:34:17.033 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:17.031+0000 7fa31e000640 1 -- 172.21.15.47:0/695191750 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa32c052070 con 0x55fab03a3c00 2024-03-21T10:34:17.033 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:17.031+0000 7fa33ddb8740 1 -- 172.21.15.47:0/695191750 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fa3200054d0 con 0x55fab03a3c00 2024-03-21T10:34:17.034 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:17.031+0000 7fa31e000640 1 -- 172.21.15.47:0/695191750 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(103..103 src has 1..103) v4 ==== 9562+0+0 (secure 0 0 0) 0x7fa32c004450 con 0x55fab03a3c00 2024-03-21T10:34:17.034 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:17.032+0000 7fa31e000640 1 -- 172.21.15.47:0/695191750 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa32c039040 con 0x55fab03a3c00 2024-03-21T10:34:17.034 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:17.032+0000 7fa33ddb8740 10 client.18393 fetch_fsmap learned FSMap version 32 2024-03-21T10:34:17.034 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:17.032+0000 7fa33ddb8740 1 -- 172.21.15.47:0/695191750 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x55fab040f1b0 con 0x55fab03a3c00 2024-03-21T10:34:17.034 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:17.032+0000 7fa31e000640 1 -- 172.21.15.47:0/695191750 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fa3200054d0 con 0x55fab03a3c00 2024-03-21T10:34:17.034 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:17.032+0000 7fa31e000640 1 -- 172.21.15.47:0/695191750 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 32) v1 ==== 7942+0+0 (secure 0 0 0) 0x7fa32c034c90 con 0x55fab03a3c00 2024-03-21T10:34:17.034 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:17.032+0000 7fa33ddb8740 10 client.18393 fetch_fsmap finished waiting for FSMap version 32 2024-03-21T10:34:17.034 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:17.032+0000 7fa33ddb8740 10 client.18393 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-21T10:34:17.034 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:17.032+0000 7fa33ddb8740 20 client.18393 populate_metadata read hostname 'smithi047' 2024-03-21T10:34:17.034 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:17.032+0000 7fa33ddb8740 1 --2- 172.21.15.47:0/695191750 >> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] conn(0x55fab040f8e0 0x55fab0411d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:34:17.034 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:17.032+0000 7fa33ddb8740 4 client.18393 mds_command: new command op to 34535 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-21T10:34:17.034 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:17.032+0000 7fa33ddb8740 1 -- 172.21.15.47:0/695191750 --> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55fab0283500 con 0x55fab040f8e0 2024-03-21T10:34:17.034 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:17.033+0000 7fa333e00640 1 --2- 172.21.15.47:0/695191750 >> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] conn(0x55fab040f8e0 0x55fab0411d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:34:17.034 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:17.033+0000 7fa333e00640 1 --2- 172.21.15.47:0/695191750 >> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] conn(0x55fab040f8e0 0x55fab0411d00 crc :-1 s=READY pgs=202 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b37461adeda30b90 server_cookie=e70d3ba68f5f3e0 in_seq=0 out_seq=0 2024-03-21T10:34:17.034 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:17.033+0000 7fa31e000640 10 client.18393 ms_handle_connect on v2:172.21.15.100:6838/4173168440 2024-03-21T10:34:17.035 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:17.034+0000 7fa31e000640 1 -- 172.21.15.47:0/695191750 <== mds.0 v2:172.21.15.100:6838/4173168440 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55fab0283500 con 0x55fab040f8e0 2024-03-21T10:34:17.035 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:17.034+0000 7fa31e000640 10 client.18393 handle_command_reply: tid=0 2024-03-21T10:34:17.035 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:17.034+0000 7fa33ddb8740 1 client.18393 shutdown 2024-03-21T10:34:17.035 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:17.034+0000 7fa33ddb8740 1 -- 172.21.15.47:0/695191750 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55fab03a3c00 msgr2=0x55fab03b6870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:34:17.035 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:17.034+0000 7fa33ddb8740 1 --2- 172.21.15.47:0/695191750 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55fab03a3c00 0x55fab03b6870 secure :-1 s=READY pgs=1201 cs=0 l=1 rev1=1 crypto rx=0x7fa32c02c700 tx=0x7fa32c0095c0 comp rx=0 tx=0).stop 2024-03-21T10:34:17.035 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:17.034+0000 7fa33ddb8740 1 -- 172.21.15.47:0/695191750 shutdown_connections 2024-03-21T10:34:17.035 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:17.034+0000 7fa33ddb8740 1 --2- 172.21.15.47:0/695191750 >> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] conn(0x55fab040f8e0 0x55fab0411d00 crc :-1 s=READY pgs=202 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:17.035 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:17.034+0000 7fa33ddb8740 1 --2- 172.21.15.47:0/695191750 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55fab03a3c00 0x55fab03b6870 unknown :-1 s=CLOSED pgs=1201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:17.035 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:17.034+0000 7fa33ddb8740 1 --2- 172.21.15.47:0/695191750 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fab039b080 0x55fab03b6330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:17.035 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:17.034+0000 7fa33ddb8740 1 --2- 172.21.15.47:0/695191750 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fab039a6d0 0x55fab03b5df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:17.035 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:17.034+0000 7fa31e000640 0 client.18393 ms_handle_reset on v2:172.21.15.100:6838/4173168440 2024-03-21T10:34:17.035 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:17.034+0000 7fa33ddb8740 1 -- 172.21.15.47:0/695191750 >> 172.21.15.47:0/695191750 conn(0x55fab0394da0 msgr2=0x55fab0395da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:34:17.035 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:17.034+0000 7fa33ddb8740 1 -- 172.21.15.47:0/695191750 shutdown_connections 2024-03-21T10:34:17.035 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:17.034+0000 7fa33ddb8740 1 -- 172.21.15.47:0/695191750 wait complete. 2024-03-21T10:34:17.036 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:17.034+0000 7fa33ddb8740 20 client.18393 trim_cache size 0 max 16384 2024-03-21T10:34:17.036 INFO:teuthology.orchestra.run.smithi047.stdout:[] 2024-03-21T10:34:17.037 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:17.035+0000 7fa33b000640 1 -- 172.21.15.47:0/3780603671 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7fa3180a3110 msgr2=0x7fa3180a55d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:34:17.037 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:17.035+0000 7fa33b000640 1 --2- 172.21.15.47:0/3780603671 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7fa3180a3110 0x7fa3180a55d0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fa32c006010 tx=0x7fa32c004a80 comp rx=0 tx=0).stop 2024-03-21T10:34:17.037 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:17.035+0000 7fa33b000640 1 -- 172.21.15.47:0/3780603671 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa334103840 msgr2=0x7fa33419a050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:34:17.037 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:17.035+0000 7fa33b000640 1 --2- 172.21.15.47:0/3780603671 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa334103840 0x7fa33419a050 secure :-1 s=READY pgs=1198 cs=0 l=1 rev1=1 crypto rx=0x7fa32400e9f0 tx=0x7fa32400eeb0 comp rx=0 tx=0).stop 2024-03-21T10:34:17.037 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:17.035+0000 7fa33b000640 1 -- 172.21.15.47:0/3780603671 shutdown_connections 2024-03-21T10:34:17.037 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:17.035+0000 7fa33b000640 1 --2- 172.21.15.47:0/3780603671 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa334109850 0x7fa33419e910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:17.037 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:17.035+0000 7fa33b000640 1 --2- 172.21.15.47:0/3780603671 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7fa3180a3110 0x7fa3180a55d0 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:17.037 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:17.035+0000 7fa33b000640 1 --2- 172.21.15.47:0/3780603671 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa334104200 0x7fa33419a590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:17.037 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:17.035+0000 7fa33b000640 1 --2- 172.21.15.47:0/3780603671 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa334103840 0x7fa33419a050 unknown :-1 s=CLOSED pgs=1198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:17.037 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:17.035+0000 7fa33b000640 1 -- 172.21.15.47:0/3780603671 >> 172.21.15.47:0/3780603671 conn(0x7fa3340ff310 msgr2=0x7fa3340ffaa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:34:17.037 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:17.035+0000 7fa33b000640 1 -- 172.21.15.47:0/3780603671 shutdown_connections 2024-03-21T10:34:17.037 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:17.035+0000 7fa33b000640 1 -- 172.21.15.47:0/3780603671 wait complete. 2024-03-21T10:34:17.047 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:34:16 smithi047 ceph-mon[31547]: pgmap v278: 129 pgs: 129 active+clean; 2.5 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 255 B/s wr, 12 op/s 2024-03-21T10:34:17.226 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:34:16 smithi100 ceph-mon[35193]: pgmap v278: 129 pgs: 129 active+clean; 2.5 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 255 B/s wr, 12 op/s 2024-03-21T10:34:17.236 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:34:16 smithi157 ceph-mon[35581]: pgmap v278: 129 pgs: 129 active+clean; 2.5 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 255 B/s wr, 12 op/s 2024-03-21T10:34:18.051 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-03-21T10:34:18.052 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-21T10:34:18.052 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-03-21T10:34:18.285 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:18.287+0000 7ff9cde00640 1 -- 172.21.15.47:0/1383351248 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff9c81026d0 msgr2=0x7ff9c8102ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:34:18.286 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:18.287+0000 7ff9cde00640 1 --2- 172.21.15.47:0/1383351248 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff9c81026d0 0x7ff9c8102ab0 secure :-1 s=READY pgs=1202 cs=0 l=1 rev1=1 crypto rx=0x7ff9bc009ca0 tx=0x7ff9bc02c180 comp rx=0 tx=0).stop 2024-03-21T10:34:18.286 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:18.287+0000 7ff9cde00640 1 -- 172.21.15.47:0/1383351248 shutdown_connections 2024-03-21T10:34:18.286 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:18.287+0000 7ff9cde00640 1 --2- 172.21.15.47:0/1383351248 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff9c810baf0 0x7ff9c810df90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:18.286 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:18.287+0000 7ff9cde00640 1 --2- 172.21.15.47:0/1383351248 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff9c8102ff0 0x7ff9c810b4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:18.286 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:18.287+0000 7ff9cde00640 1 --2- 172.21.15.47:0/1383351248 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff9c81026d0 0x7ff9c8102ab0 unknown :-1 s=CLOSED pgs=1202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:18.286 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:18.287+0000 7ff9cde00640 1 -- 172.21.15.47:0/1383351248 >> 172.21.15.47:0/1383351248 conn(0x7ff9c80fbef0 msgr2=0x7ff9c80fe310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:34:18.286 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:18.287+0000 7ff9cde00640 1 -- 172.21.15.47:0/1383351248 shutdown_connections 2024-03-21T10:34:18.286 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:18.287+0000 7ff9cde00640 1 -- 172.21.15.47:0/1383351248 wait complete. 2024-03-21T10:34:18.288 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:18.288+0000 7ff9cde00640 1 Processor -- start 2024-03-21T10:34:18.288 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:18.288+0000 7ff9cde00640 1 -- start start 2024-03-21T10:34:18.288 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:18.289+0000 7ff9cde00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff9c81026d0 0x7ff9c8199ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:34:18.288 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:18.289+0000 7ff9cde00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff9c8102ff0 0x7ff9c819a420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:34:18.288 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:18.289+0000 7ff9cde00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff9c810baf0 0x7ff9c819e7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:34:18.288 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:18.289+0000 7ff9cde00640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9c8111760 con 0x7ff9c8102ff0 2024-03-21T10:34:18.288 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:18.289+0000 7ff9cde00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9c81115e0 con 0x7ff9c81026d0 2024-03-21T10:34:18.288 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:18.289+0000 7ff9cde00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9c81118e0 con 0x7ff9c810baf0 2024-03-21T10:34:18.288 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:18.289+0000 7ff9c7e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff9c8102ff0 0x7ff9c819a420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:34:18.288 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:18.289+0000 7ff9c7e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff9c8102ff0 0x7ff9c819a420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:35406/0 (socket says 172.21.15.47:35406) 2024-03-21T10:34:18.288 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:18.289+0000 7ff9c7e00640 1 -- 172.21.15.47:0/1451515775 learned_addr learned my addr 172.21.15.47:0/1451515775 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:34:18.289 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:18.289+0000 7ff9cca00640 1 --2- 172.21.15.47:0/1451515775 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff9c81026d0 0x7ff9c8199ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:34:18.289 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:18.289+0000 7ff9cd400640 1 --2- 172.21.15.47:0/1451515775 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff9c810baf0 0x7ff9c819e7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:34:18.289 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:18.289+0000 7ff9c7e00640 1 -- 172.21.15.47:0/1451515775 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff9c810baf0 msgr2=0x7ff9c819e7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:34:18.289 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:18.289+0000 7ff9c7e00640 1 --2- 172.21.15.47:0/1451515775 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff9c810baf0 0x7ff9c819e7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:18.289 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:18.290+0000 7ff9c7e00640 1 -- 172.21.15.47:0/1451515775 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff9c81026d0 msgr2=0x7ff9c8199ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:34:18.289 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:18.290+0000 7ff9c7e00640 1 --2- 172.21.15.47:0/1451515775 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff9c81026d0 0x7ff9c8199ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:18.289 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:18.290+0000 7ff9c7e00640 1 -- 172.21.15.47:0/1451515775 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9c819eed0 con 0x7ff9c8102ff0 2024-03-21T10:34:18.289 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:18.290+0000 7ff9cd400640 1 --2- 172.21.15.47:0/1451515775 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff9c810baf0 0x7ff9c819e7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T10:34:18.289 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:18.290+0000 7ff9c7e00640 1 --2- 172.21.15.47:0/1451515775 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff9c8102ff0 0x7ff9c819a420 secure :-1 s=READY pgs=1203 cs=0 l=1 rev1=1 crypto rx=0x7ff9b400e9f0 tx=0x7ff9b400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:34:18.290 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:18.290+0000 7ff9c5600640 1 -- 172.21.15.47:0/1451515775 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9b400ccc0 con 0x7ff9c8102ff0 2024-03-21T10:34:18.290 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:18.290+0000 7ff9c5600640 1 -- 172.21.15.47:0/1451515775 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7ff9b40108e0 con 0x7ff9c8102ff0 2024-03-21T10:34:18.290 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:18.290+0000 7ff9cde00640 1 -- 172.21.15.47:0/1451515775 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff9c819f1c0 con 0x7ff9c8102ff0 2024-03-21T10:34:18.290 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:18.290+0000 7ff9c5600640 1 -- 172.21.15.47:0/1451515775 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9b40187c0 con 0x7ff9c8102ff0 2024-03-21T10:34:18.290 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:18.290+0000 7ff9cde00640 1 -- 172.21.15.47:0/1451515775 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff9c81a6a50 con 0x7ff9c8102ff0 2024-03-21T10:34:18.292 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:18.294+0000 7ff9c5600640 1 -- 172.21.15.47:0/1451515775 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 27) v1 ==== 137985+0+0 (secure 0 0 0) 0x7ff9b4010430 con 0x7ff9c8102ff0 2024-03-21T10:34:18.292 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:18.294+0000 7ff9d217f740 1 Processor -- start 2024-03-21T10:34:18.292 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:18.294+0000 7ff9d217f740 1 -- start start 2024-03-21T10:34:18.293 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:18.294+0000 7ff9d217f740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c259877290 0x55c259877670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:34:18.293 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:18.294+0000 7ff9d217f740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c259877bb0 0x55c259880250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:34:18.293 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:18.294+0000 7ff9d217f740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c259880790 0x55c259882b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:34:18.293 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:18.294+0000 7ff9d217f740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c2597dbeb0 con 0x55c259877290 2024-03-21T10:34:18.293 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:18.294+0000 7ff9d217f740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c259784a60 con 0x55c259877bb0 2024-03-21T10:34:18.293 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:18.294+0000 7ff9d217f740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c2597d7f50 con 0x55c259880790 2024-03-21T10:34:18.294 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:18.294+0000 7ff9cd400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c259877290 0x55c259877670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:34:18.294 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:18.295+0000 7ff9cd400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c259877290 0x55c259877670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:35414/0 (socket says 172.21.15.47:35414) 2024-03-21T10:34:18.294 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:18.295+0000 7ff9cd400640 1 -- 172.21.15.47:0/1645577791 learned_addr learned my addr 172.21.15.47:0/1645577791 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:34:18.294 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:18.295+0000 7ff9cca00640 1 --2- 172.21.15.47:0/1645577791 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c259877bb0 0x55c259880250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:34:18.294 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:18.295+0000 7ff9c7e00640 1 --2- 172.21.15.47:0/1645577791 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c259880790 0x55c259882b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:34:18.294 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:18.295+0000 7ff9cd400640 1 -- 172.21.15.47:0/1645577791 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c259880790 msgr2=0x55c259882b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:34:18.294 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:18.295+0000 7ff9cd400640 1 --2- 172.21.15.47:0/1645577791 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c259880790 0x55c259882b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:18.294 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:18.295+0000 7ff9cd400640 1 -- 172.21.15.47:0/1645577791 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c259877bb0 msgr2=0x55c259880250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:34:18.294 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:18.295+0000 7ff9cd400640 1 --2- 172.21.15.47:0/1645577791 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c259877bb0 0x55c259880250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:18.294 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:18.295+0000 7ff9cd400640 1 -- 172.21.15.47:0/1645577791 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c2595b87e0 con 0x55c259877290 2024-03-21T10:34:18.294 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:18.295+0000 7ff9cca00640 1 --2- 172.21.15.47:0/1645577791 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c259877bb0 0x55c259880250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T10:34:18.294 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:18.295+0000 7ff9c7e00640 1 --2- 172.21.15.47:0/1645577791 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c259880790 0x55c259882b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T10:34:18.297 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:18.295+0000 7ff9cd400640 1 --2- 172.21.15.47:0/1645577791 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c259877290 0x55c259877670 secure :-1 s=READY pgs=1204 cs=0 l=1 rev1=1 crypto rx=0x7ff9c000bb00 tx=0x7ff9c000d340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8f5e01470bd61220 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:34:18.297 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:18.296+0000 7ff9b9c00640 1 -- 172.21.15.47:0/1645577791 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9c0017420 con 0x55c259877290 2024-03-21T10:34:18.297 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:18.296+0000 7ff9b9c00640 1 -- 172.21.15.47:0/1645577791 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7ff9c0017a70 con 0x55c259877290 2024-03-21T10:34:18.297 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:18.296+0000 7ff9b9c00640 1 -- 172.21.15.47:0/1645577791 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9c000fa20 con 0x55c259877290 2024-03-21T10:34:18.297 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:18.296+0000 7ff9c5600640 1 --2- 172.21.15.47:0/1451515775 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7ff9b00a3110 0x7ff9b00a55d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:34:18.297 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:18.297+0000 7ff9d217f740 1 -- 172.21.15.47:0/1645577791 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c259877290 msgr2=0x55c259877670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:34:18.297 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:18.297+0000 7ff9d217f740 1 --2- 172.21.15.47:0/1645577791 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c259877290 0x55c259877670 secure :-1 s=READY pgs=1204 cs=0 l=1 rev1=1 crypto rx=0x7ff9c000bb00 tx=0x7ff9c000d340 comp rx=0 tx=0).stop 2024-03-21T10:34:18.297 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:18.297+0000 7ff9d217f740 1 -- 172.21.15.47:0/1645577791 shutdown_connections 2024-03-21T10:34:18.297 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:18.297+0000 7ff9d217f740 1 --2- 172.21.15.47:0/1645577791 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c259880790 0x55c259882b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:18.298 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:18.297+0000 7ff9d217f740 1 --2- 172.21.15.47:0/1645577791 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c259877bb0 0x55c259880250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:18.298 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:18.297+0000 7ff9d217f740 1 --2- 172.21.15.47:0/1645577791 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c259877290 0x55c259877670 unknown :-1 s=CLOSED pgs=1204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:18.298 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:18.297+0000 7ff9d217f740 1 -- 172.21.15.47:0/1645577791 >> 172.21.15.47:0/1645577791 conn(0x55c2598719c0 msgr2=0x55c259871da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:34:18.298 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:18.297+0000 7ff9c5600640 1 -- 172.21.15.47:0/1451515775 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 9562+0+0 (secure 0 0 0) 0x7ff9b40cff90 con 0x7ff9c8102ff0 2024-03-21T10:34:18.298 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:18.297+0000 7ff9d217f740 1 -- 172.21.15.47:0/1645577791 shutdown_connections 2024-03-21T10:34:18.298 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:18.297+0000 7ff9cca00640 1 --2- 172.21.15.47:0/1451515775 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7ff9b00a3110 0x7ff9b00a55d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:34:18.298 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:18.297+0000 7ff9d217f740 1 -- 172.21.15.47:0/1645577791 wait complete. 2024-03-21T10:34:18.298 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:18.297+0000 7ff9d217f740 1 -- start start 2024-03-21T10:34:18.298 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:18.298+0000 7ff9d217f740 1 Processor -- start 2024-03-21T10:34:18.298 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:18.298+0000 7ff9cca00640 1 --2- 172.21.15.47:0/1451515775 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7ff9b00a3110 0x7ff9b00a55d0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7ff9bc004af0 tx=0x7ff9bc02e040 comp rx=0 tx=0).ready entity=mgr.15687 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:34:18.298 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:18.298+0000 7ff9d217f740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c259877290 0x55c259880000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:34:18.298 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:18.298+0000 7ff9d217f740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c259877bb0 0x55c25987d390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:34:18.298 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:18.298+0000 7ff9d217f740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c259880790 0x55c25987d8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:34:18.298 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:18.298+0000 7ff9d217f740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c2598835d0 con 0x55c259877bb0 2024-03-21T10:34:18.299 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:18.298+0000 7ff9d217f740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c259797eb0 con 0x55c259877290 2024-03-21T10:34:18.299 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:18.298+0000 7ff9d217f740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c259883750 con 0x55c259880790 2024-03-21T10:34:18.299 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:18.298+0000 7ff9d217f740 10 client.0 _ll_register_callbacks cb 0x55c25978c150 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T10:34:18.299 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:18.298+0000 7ff9c7e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c259877bb0 0x55c25987d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:34:18.299 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:18.299+0000 7ff9cca00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c259877290 0x55c259880000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:34:18.299 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:18.299+0000 7ff9cca00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c259877290 0x55c259880000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:50084/0 (socket says 172.21.15.47:50084) 2024-03-21T10:34:18.299 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:18.299+0000 7ff9cca00640 1 -- 172.21.15.47:0/2856620569 learned_addr learned my addr 172.21.15.47:0/2856620569 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:34:18.299 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:18.299+0000 7ff9cd400640 1 --2- 172.21.15.47:0/2856620569 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c259880790 0x55c25987d8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:34:18.299 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:18.299+0000 7ff9cca00640 1 -- 172.21.15.47:0/2856620569 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c259880790 msgr2=0x55c25987d8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:34:18.299 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:18.299+0000 7ff9cca00640 1 --2- 172.21.15.47:0/2856620569 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c259880790 0x55c25987d8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:18.299 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:18.299+0000 7ff9cca00640 1 -- 172.21.15.47:0/2856620569 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c259877bb0 msgr2=0x55c25987d390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:34:18.299 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:18.299+0000 7ff9cca00640 1 --2- 172.21.15.47:0/2856620569 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c259877bb0 0x55c25987d390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:18.299 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:18.299+0000 7ff9cca00640 1 -- 172.21.15.47:0/2856620569 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55c259618f70 con 0x55c259877290 2024-03-21T10:34:18.302 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:18.299+0000 7ff9cca00640 1 --2- 172.21.15.47:0/2856620569 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c259877290 0x55c259880000 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7ff9bc009ca0 tx=0x7ff9bc02c180 comp rx=0 tx=0).ready entity=mon.2 client_cookie=48cac66e4e28f70f server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:34:18.302 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:18.299+0000 7ff9ae000640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-21T10:34:18.302 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:18.300+0000 7ff9ae000640 1 -- 172.21.15.47:0/2856620569 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9bc046070 con 0x55c259877290 2024-03-21T10:34:18.302 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:18.300+0000 7ff9ae000640 1 -- 172.21.15.47:0/2856620569 <== mon.2 v2:172.21.15.100:3300/0 2 ==== osd_map(103..103 src has 1..103) v4 ==== 9562+0+0 (secure 0 0 0) 0x7ff9bc0325f0 con 0x55c259877290 2024-03-21T10:34:18.302 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:18.300+0000 7ff9d217f740 1 -- 172.21.15.47:0/2856620569 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ff9c0017a70 con 0x55c259877290 2024-03-21T10:34:18.302 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:18.300+0000 7ff9ae000640 1 -- 172.21.15.47:0/2856620569 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff9bc05b050 con 0x55c259877290 2024-03-21T10:34:18.302 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:18.300+0000 7ff9d217f740 10 client.38171 fetch_fsmap learned FSMap version 32 2024-03-21T10:34:18.302 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:18.300+0000 7ff9d217f740 1 -- 172.21.15.47:0/2856620569 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x55c2594b0170 con 0x55c259877290 2024-03-21T10:34:18.302 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:18.301+0000 7ff9ae000640 1 -- 172.21.15.47:0/2856620569 <== mon.2 v2:172.21.15.100:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7ff9c0017a70 con 0x55c259877290 2024-03-21T10:34:18.302 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:18.301+0000 7ff9ae000640 1 -- 172.21.15.47:0/2856620569 <== mon.2 v2:172.21.15.100:3300/0 5 ==== fsmap(e 32) v1 ==== 7942+0+0 (secure 0 0 0) 0x7ff9bc04d7a0 con 0x55c259877290 2024-03-21T10:34:18.303 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:18.301+0000 7ff9d217f740 10 client.38171 fetch_fsmap finished waiting for FSMap version 32 2024-03-21T10:34:18.303 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:18.301+0000 7ff9d217f740 10 client.38171 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-21T10:34:18.303 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:18.301+0000 7ff9d217f740 20 client.38171 populate_metadata read hostname 'smithi047' 2024-03-21T10:34:18.303 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:18.301+0000 7ff9d217f740 1 --2- 172.21.15.47:0/2856620569 >> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] conn(0x55c2598906d0 0x55c259892b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:34:18.303 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:18.301+0000 7ff9d217f740 4 client.38171 mds_command: new command op to 34535 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T10:34:18.303 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:18.301+0000 7ff9d217f740 1 -- 172.21.15.47:0/2856620569 --> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55c2597e93f0 con 0x55c2598906d0 2024-03-21T10:34:18.303 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:18.301+0000 7ff9c7e00640 1 --2- 172.21.15.47:0/2856620569 >> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] conn(0x55c2598906d0 0x55c259892b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:34:18.303 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:18.302+0000 7ff9c7e00640 1 --2- 172.21.15.47:0/2856620569 >> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] conn(0x55c2598906d0 0x55c259892b00 crc :-1 s=READY pgs=203 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fb15e12dc69174fc server_cookie=517b751b3a9e46a1 in_seq=0 out_seq=0 2024-03-21T10:34:18.303 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:18.302+0000 7ff9ae000640 10 client.38171 ms_handle_connect on v2:172.21.15.100:6838/4173168440 2024-03-21T10:34:18.303 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:18.303+0000 7ff9ae000640 1 -- 172.21.15.47:0/2856620569 <== mds.0 v2:172.21.15.100:6838/4173168440 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55c2597e93f0 con 0x55c2598906d0 2024-03-21T10:34:18.303 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:18.303+0000 7ff9ae000640 10 client.38171 handle_command_reply: tid=0 2024-03-21T10:34:18.303 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:18.303+0000 7ff9d217f740 1 client.38171 shutdown 2024-03-21T10:34:18.303 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:18.303+0000 7ff9d217f740 1 -- 172.21.15.47:0/2856620569 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c259877290 msgr2=0x55c259880000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:34:18.303 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:18.303+0000 7ff9d217f740 1 --2- 172.21.15.47:0/2856620569 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c259877290 0x55c259880000 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7ff9bc009ca0 tx=0x7ff9bc02c180 comp rx=0 tx=0).stop 2024-03-21T10:34:18.303 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:18.304+0000 7ff9d217f740 1 -- 172.21.15.47:0/2856620569 shutdown_connections 2024-03-21T10:34:18.304 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:18.304+0000 7ff9d217f740 1 --2- 172.21.15.47:0/2856620569 >> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] conn(0x55c2598906d0 0x55c259892b00 crc :-1 s=READY pgs=203 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:18.304 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:18.304+0000 7ff9d217f740 1 --2- 172.21.15.47:0/2856620569 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c259880790 0x55c25987d8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:18.304 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:18.304+0000 7ff9ae000640 0 client.38171 ms_handle_reset on v2:172.21.15.100:6838/4173168440 2024-03-21T10:34:18.304 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:18.304+0000 7ff9d217f740 1 --2- 172.21.15.47:0/2856620569 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c259877bb0 0x55c25987d390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:18.304 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:18.304+0000 7ff9d217f740 1 --2- 172.21.15.47:0/2856620569 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c259877290 0x55c259880000 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:18.304 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:18.304+0000 7ff9d217f740 1 -- 172.21.15.47:0/2856620569 >> 172.21.15.47:0/2856620569 conn(0x55c2598719c0 msgr2=0x55c259870a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:34:18.304 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:18.304+0000 7ff9d217f740 1 -- 172.21.15.47:0/2856620569 shutdown_connections 2024-03-21T10:34:18.304 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:18.304+0000 7ff9d217f740 1 -- 172.21.15.47:0/2856620569 wait complete. 2024-03-21T10:34:18.304 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:18.304+0000 7ff9d217f740 20 client.38171 trim_cache size 0 max 16384 2024-03-21T10:34:18.320 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:18.322+0000 7ff9d217f740 1 Processor -- start 2024-03-21T10:34:18.320 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:18.322+0000 7ff9d217f740 1 -- start start 2024-03-21T10:34:18.321 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:18.322+0000 7ff9d217f740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c259877290 0x55c2598787e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:34:18.321 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:18.322+0000 7ff9d217f740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c259877bb0 0x55c25987fac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:34:18.321 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:18.322+0000 7ff9d217f740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c259880790 0x55c259880000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:34:18.321 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:18.322+0000 7ff9d217f740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9c81b09f0 con 0x55c259880790 2024-03-21T10:34:18.321 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:18.322+0000 7ff9d217f740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9c81b2370 con 0x55c259877290 2024-03-21T10:34:18.321 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:18.322+0000 7ff9d217f740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9c81b1050 con 0x55c259877bb0 2024-03-21T10:34:18.321 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:18.322+0000 7ff9cd400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c259880790 0x55c259880000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:34:18.321 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:18.322+0000 7ff9cca00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c259877290 0x55c2598787e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:34:18.321 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:18.322+0000 7ff9cca00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c259877290 0x55c2598787e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:50086/0 (socket says 172.21.15.47:50086) 2024-03-21T10:34:18.321 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:18.322+0000 7ff9cca00640 1 -- 172.21.15.47:0/2970746357 learned_addr learned my addr 172.21.15.47:0/2970746357 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:34:18.321 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:18.323+0000 7ff9c7e00640 1 --2- 172.21.15.47:0/2970746357 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c259877bb0 0x55c25987fac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:34:18.326 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:18.323+0000 7ff9cca00640 1 -- 172.21.15.47:0/2970746357 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c259877bb0 msgr2=0x55c25987fac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:34:18.326 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:18.323+0000 7ff9cca00640 1 --2- 172.21.15.47:0/2970746357 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c259877bb0 0x55c25987fac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:18.326 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:18.323+0000 7ff9cca00640 1 -- 172.21.15.47:0/2970746357 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c259880790 msgr2=0x55c259880000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:34:18.326 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:18.323+0000 7ff9cca00640 1 --2- 172.21.15.47:0/2970746357 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c259880790 0x55c259880000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:18.326 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:18.323+0000 7ff9cca00640 1 -- 172.21.15.47:0/2970746357 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c25987b900 con 0x55c259877290 2024-03-21T10:34:18.326 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:18.323+0000 7ff9cd400640 1 --2- 172.21.15.47:0/2970746357 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c259880790 0x55c259880000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T10:34:18.326 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:18.323+0000 7ff9cca00640 1 --2- 172.21.15.47:0/2970746357 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c259877290 0x55c2598787e0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7ff9bc0404d0 tx=0x7ff9bc032550 comp rx=0 tx=0).ready entity=mon.2 client_cookie=41f6a8d2c9b1ce9b server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:34:18.326 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:18.323+0000 7ff9b9200640 1 -- 172.21.15.47:0/2970746357 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9bc046040 con 0x55c259877290 2024-03-21T10:34:18.326 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:18.324+0000 7ff9b9200640 1 -- 172.21.15.47:0/2970746357 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7ff9bc064070 con 0x55c259877290 2024-03-21T10:34:18.326 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:18.324+0000 7ff9b9200640 1 -- 172.21.15.47:0/2970746357 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9bc039040 con 0x55c259877290 2024-03-21T10:34:18.327 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:18.324+0000 7ff9d217f740 1 -- 172.21.15.47:0/2970746357 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c259877290 msgr2=0x55c2598787e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:34:18.327 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:18.324+0000 7ff9d217f740 1 --2- 172.21.15.47:0/2970746357 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c259877290 0x55c2598787e0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7ff9bc0404d0 tx=0x7ff9bc032550 comp rx=0 tx=0).stop 2024-03-21T10:34:18.327 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:18.324+0000 7ff9d217f740 1 -- 172.21.15.47:0/2970746357 shutdown_connections 2024-03-21T10:34:18.327 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:18.324+0000 7ff9d217f740 1 --2- 172.21.15.47:0/2970746357 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c259880790 0x55c259880000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:18.327 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:18.324+0000 7ff9d217f740 1 --2- 172.21.15.47:0/2970746357 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c259877bb0 0x55c25987fac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:18.327 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:18.324+0000 7ff9d217f740 1 --2- 172.21.15.47:0/2970746357 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c259877290 0x55c2598787e0 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:18.327 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:18.324+0000 7ff9d217f740 1 -- 172.21.15.47:0/2970746357 >> 172.21.15.47:0/2970746357 conn(0x55c2598719c0 msgr2=0x55c2598780d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:34:18.327 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:18.324+0000 7ff9d217f740 1 -- 172.21.15.47:0/2970746357 shutdown_connections 2024-03-21T10:34:18.327 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:18.324+0000 7ff9d217f740 1 -- 172.21.15.47:0/2970746357 wait complete. 2024-03-21T10:34:18.327 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:18.325+0000 7ff9d217f740 1 -- start start 2024-03-21T10:34:18.327 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:18.325+0000 7ff9d217f740 1 Processor -- start 2024-03-21T10:34:18.327 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:18.325+0000 7ff9d217f740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c259877290 0x55c2598859a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:34:18.327 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:18.325+0000 7ff9d217f740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c259877bb0 0x55c259885ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:34:18.327 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:18.325+0000 7ff9d217f740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c259880790 0x55c259886420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:34:18.328 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:18.325+0000 7ff9d217f740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9c81b1d10 con 0x55c259877290 2024-03-21T10:34:18.328 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:18.325+0000 7ff9d217f740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9c81b16b0 con 0x55c259880790 2024-03-21T10:34:18.328 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:18.325+0000 7ff9d217f740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9c81aebd0 con 0x55c259877bb0 2024-03-21T10:34:18.328 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:18.326+0000 7ff9d217f740 10 client.0 _ll_register_callbacks cb 0x7ff9bc009b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T10:34:18.328 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:18.326+0000 7ff9cca00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c259877290 0x55c2598859a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:34:18.328 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:18.326+0000 7ff9cca00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c259877290 0x55c2598859a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:35440/0 (socket says 172.21.15.47:35440) 2024-03-21T10:34:18.328 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:18.326+0000 7ff9cca00640 1 -- 172.21.15.47:0/3790741638 learned_addr learned my addr 172.21.15.47:0/3790741638 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:34:18.328 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:18.326+0000 7ff9cd400640 1 --2- 172.21.15.47:0/3790741638 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c259880790 0x55c259886420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:34:18.328 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:18.326+0000 7ff9c7e00640 1 --2- 172.21.15.47:0/3790741638 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c259877bb0 0x55c259885ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:34:18.328 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:18.326+0000 7ff9cca00640 1 -- 172.21.15.47:0/3790741638 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c259877bb0 msgr2=0x55c259885ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:34:18.328 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:18.326+0000 7ff9cca00640 1 --2- 172.21.15.47:0/3790741638 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c259877bb0 0x55c259885ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:18.328 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:18.326+0000 7ff9cca00640 1 -- 172.21.15.47:0/3790741638 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c259880790 msgr2=0x55c259886420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:34:18.328 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:18.326+0000 7ff9cca00640 1 --2- 172.21.15.47:0/3790741638 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c259880790 0x55c259886420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:18.329 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:18.326+0000 7ff9cca00640 1 -- 172.21.15.47:0/3790741638 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55c2598ec0d0 con 0x55c259877290 2024-03-21T10:34:18.329 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:18.326+0000 7ff9c7e00640 1 --2- 172.21.15.47:0/3790741638 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c259877bb0 0x55c259885ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T10:34:18.329 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:18.327+0000 7ff9cca00640 1 --2- 172.21.15.47:0/3790741638 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c259877290 0x55c2598859a0 secure :-1 s=READY pgs=1205 cs=0 l=1 rev1=1 crypto rx=0x7ff9bc02e010 tx=0x7ff9bc003e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=34893d34e9fe8982 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:34:18.329 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:18.327+0000 7ff9ae000640 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-03-21T10:34:18.329 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:18.327+0000 7ff9ae000640 1 -- 172.21.15.47:0/3790741638 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9bc039070 con 0x55c259877290 2024-03-21T10:34:18.329 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:18.327+0000 7ff9ae000640 1 -- 172.21.15.47:0/3790741638 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(103..103 src has 1..103) v4 ==== 9562+0+0 (secure 0 0 0) 0x7ff9bc04d470 con 0x55c259877290 2024-03-21T10:34:18.329 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:18.327+0000 7ff9d217f740 1 -- 172.21.15.47:0/3790741638 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ff9bc064070 con 0x55c259877290 2024-03-21T10:34:18.329 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:18.327+0000 7ff9ae000640 1 -- 172.21.15.47:0/3790741638 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff9bc075050 con 0x55c259877290 2024-03-21T10:34:18.330 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:18.327+0000 7ff9d217f740 10 client.18411 fetch_fsmap learned FSMap version 32 2024-03-21T10:34:18.330 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:18.327+0000 7ff9d217f740 1 -- 172.21.15.47:0/3790741638 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x55c2598ed3f0 con 0x55c259877290 2024-03-21T10:34:18.330 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:18.327+0000 7ff9ae000640 1 -- 172.21.15.47:0/3790741638 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7ff9bc064070 con 0x55c259877290 2024-03-21T10:34:18.330 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:18.328+0000 7ff9ae000640 1 -- 172.21.15.47:0/3790741638 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 32) v1 ==== 7942+0+0 (secure 0 0 0) 0x7ff9bc040500 con 0x55c259877290 2024-03-21T10:34:18.330 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:18.328+0000 7ff9d217f740 10 client.18411 fetch_fsmap finished waiting for FSMap version 32 2024-03-21T10:34:18.330 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:18.328+0000 7ff9d217f740 10 client.18411 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-21T10:34:18.330 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:18.328+0000 7ff9d217f740 20 client.18411 populate_metadata read hostname 'smithi047' 2024-03-21T10:34:18.330 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:18.328+0000 7ff9d217f740 1 --2- 172.21.15.47:0/3790741638 >> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] conn(0x55c2598eda00 0x55c2598efdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:34:18.330 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:18.328+0000 7ff9d217f740 4 client.18411 mds_command: new command op to 34535 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-03-21T10:34:18.330 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:18.328+0000 7ff9d217f740 1 -- 172.21.15.47:0/3790741638 --> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x55c2597e3740 con 0x55c2598eda00 2024-03-21T10:34:18.332 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:18.328+0000 7ff9c7e00640 1 --2- 172.21.15.47:0/3790741638 >> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] conn(0x55c2598eda00 0x55c2598efdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:34:18.332 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:18.329+0000 7ff9c7e00640 1 --2- 172.21.15.47:0/3790741638 >> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] conn(0x55c2598eda00 0x55c2598efdf0 crc :-1 s=READY pgs=204 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d263b3dfd5e1070c server_cookie=d784bc6738cca59 in_seq=0 out_seq=0 2024-03-21T10:34:18.332 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:18.329+0000 7ff9ae000640 10 client.18411 ms_handle_connect on v2:172.21.15.100:6838/4173168440 2024-03-21T10:34:18.332 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:18.331+0000 7ff9ae000640 1 -- 172.21.15.47:0/3790741638 <== mds.0 v2:172.21.15.100:6838/4173168440 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x55c2597e3740 con 0x55c2598eda00 2024-03-21T10:34:18.332 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:18.331+0000 7ff9ae000640 10 client.18411 handle_command_reply: tid=0 2024-03-21T10:34:18.332 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:18.331+0000 7ff9d217f740 1 client.18411 shutdown 2024-03-21T10:34:18.332 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:18.331+0000 7ff9d217f740 1 -- 172.21.15.47:0/3790741638 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c259877290 msgr2=0x55c2598859a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:34:18.332 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:18.331+0000 7ff9d217f740 1 --2- 172.21.15.47:0/3790741638 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c259877290 0x55c2598859a0 secure :-1 s=READY pgs=1205 cs=0 l=1 rev1=1 crypto rx=0x7ff9bc02e010 tx=0x7ff9bc003e50 comp rx=0 tx=0).stop 2024-03-21T10:34:18.332 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:18.331+0000 7ff9d217f740 1 -- 172.21.15.47:0/3790741638 shutdown_connections 2024-03-21T10:34:18.332 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:18.331+0000 7ff9d217f740 1 --2- 172.21.15.47:0/3790741638 >> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] conn(0x55c2598eda00 0x55c2598efdf0 crc :-1 s=READY pgs=204 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:18.332 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:18.331+0000 7ff9d217f740 1 --2- 172.21.15.47:0/3790741638 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c259880790 0x55c259886420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:18.332 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:18.331+0000 7ff9d217f740 1 --2- 172.21.15.47:0/3790741638 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c259877bb0 0x55c259885ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:18.333 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:18.331+0000 7ff9d217f740 1 --2- 172.21.15.47:0/3790741638 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c259877290 0x55c2598859a0 unknown :-1 s=CLOSED pgs=1205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:18.333 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:18.331+0000 7ff9ae000640 0 client.18411 ms_handle_reset on v2:172.21.15.100:6838/4173168440 2024-03-21T10:34:18.333 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:18.331+0000 7ff9d217f740 1 -- 172.21.15.47:0/3790741638 >> 172.21.15.47:0/3790741638 conn(0x55c2598719c0 msgr2=0x55c259878960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:34:18.333 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:18.331+0000 7ff9d217f740 1 -- 172.21.15.47:0/3790741638 shutdown_connections 2024-03-21T10:34:18.333 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:18.331+0000 7ff9d217f740 1 -- 172.21.15.47:0/3790741638 wait complete. 2024-03-21T10:34:18.333 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:18.332+0000 7ff9d217f740 20 client.18411 trim_cache size 0 max 16384 2024-03-21T10:34:18.333 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-03-21T10:34:18.333 INFO:teuthology.orchestra.run.smithi047.stdout: "return_code": 0, 2024-03-21T10:34:18.333 INFO:teuthology.orchestra.run.smithi047.stdout: "scrub_tag": "592d6102-8bc5-4fb4-846f-34cdf23924e4", 2024-03-21T10:34:18.333 INFO:teuthology.orchestra.run.smithi047.stdout: "mode": "asynchronous" 2024-03-21T10:34:18.333 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-03-21T10:34:18.334 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:18.332+0000 7ff9cde00640 1 -- 172.21.15.47:0/1451515775 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7ff9b00a3110 msgr2=0x7ff9b00a55d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:34:18.334 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:18.332+0000 7ff9cde00640 1 --2- 172.21.15.47:0/1451515775 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7ff9b00a3110 0x7ff9b00a55d0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7ff9bc004af0 tx=0x7ff9bc02e040 comp rx=0 tx=0).stop 2024-03-21T10:34:18.334 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:18.332+0000 7ff9cde00640 1 -- 172.21.15.47:0/1451515775 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff9c8102ff0 msgr2=0x7ff9c819a420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:34:18.334 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:18.332+0000 7ff9cde00640 1 --2- 172.21.15.47:0/1451515775 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff9c8102ff0 0x7ff9c819a420 secure :-1 s=READY pgs=1203 cs=0 l=1 rev1=1 crypto rx=0x7ff9b400e9f0 tx=0x7ff9b400eeb0 comp rx=0 tx=0).stop 2024-03-21T10:34:18.335 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:18.332+0000 7ff9cde00640 1 -- 172.21.15.47:0/1451515775 shutdown_connections 2024-03-21T10:34:18.335 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:18.332+0000 7ff9cde00640 1 --2- 172.21.15.47:0/1451515775 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff9c810baf0 0x7ff9c819e7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:18.335 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:18.332+0000 7ff9cde00640 1 --2- 172.21.15.47:0/1451515775 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff9c8102ff0 0x7ff9c819a420 unknown :-1 s=CLOSED pgs=1203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:18.335 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:18.332+0000 7ff9cde00640 1 --2- 172.21.15.47:0/1451515775 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7ff9b00a3110 0x7ff9b00a55d0 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:18.335 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:18.332+0000 7ff9cde00640 1 --2- 172.21.15.47:0/1451515775 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff9c81026d0 0x7ff9c8199ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:18.335 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:18.332+0000 7ff9cde00640 1 -- 172.21.15.47:0/1451515775 >> 172.21.15.47:0/1451515775 conn(0x7ff9c80fbef0 msgr2=0x7ff9c810cd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:34:18.335 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:18.333+0000 7ff9cde00640 1 -- 172.21.15.47:0/1451515775 shutdown_connections 2024-03-21T10:34:18.335 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:18.333+0000 7ff9cde00640 1 -- 172.21.15.47:0/1451515775 wait complete. 2024-03-21T10:34:18.348 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-21T10:34:18.348 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-21T10:34:18.570 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:18.570+0000 7f6b3d000640 1 -- 172.21.15.47:0/1518639111 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6b3810cf80 msgr2=0x7f6b38114b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:34:18.570 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:18.570+0000 7f6b3d000640 1 --2- 172.21.15.47:0/1518639111 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6b3810cf80 0x7f6b38114b50 secure :-1 s=READY pgs=1206 cs=0 l=1 rev1=1 crypto rx=0x7f6b30009900 tx=0x7f6b3002bc50 comp rx=0 tx=0).stop 2024-03-21T10:34:18.570 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:18.571+0000 7f6b3d000640 1 -- 172.21.15.47:0/1518639111 shutdown_connections 2024-03-21T10:34:18.570 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:18.571+0000 7f6b3d000640 1 --2- 172.21.15.47:0/1518639111 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6b3810cf80 0x7f6b38114b50 unknown :-1 s=CLOSED pgs=1206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:18.570 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:18.571+0000 7f6b3d000640 1 --2- 172.21.15.47:0/1518639111 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6b38108860 0x7f6b3810c8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:18.570 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:18.571+0000 7f6b3d000640 1 --2- 172.21.15.47:0/1518639111 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6b38107eb0 0x7f6b38108290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:18.570 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:18.571+0000 7f6b3d000640 1 -- 172.21.15.47:0/1518639111 >> 172.21.15.47:0/1518639111 conn(0x7f6b38075be0 msgr2=0x7f6b38078000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:34:18.570 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:18.571+0000 7f6b3d000640 1 -- 172.21.15.47:0/1518639111 shutdown_connections 2024-03-21T10:34:18.571 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:18.571+0000 7f6b3d000640 1 -- 172.21.15.47:0/1518639111 wait complete. 2024-03-21T10:34:18.571 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:18.572+0000 7f6b3d000640 1 Processor -- start 2024-03-21T10:34:18.571 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:18.572+0000 7f6b3d000640 1 -- start start 2024-03-21T10:34:18.573 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:18.572+0000 7f6b3d000640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6b38107eb0 0x7f6b3819e400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:34:18.573 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:18.572+0000 7f6b3d000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6b3810cf80 0x7f6b3819e940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:34:18.573 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:18.572+0000 7f6b3d000640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6b381a2cc0 0x7f6b381a3170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:34:18.573 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:18.572+0000 7f6b3d000640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b38117ee0 con 0x7f6b38107eb0 2024-03-21T10:34:18.573 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:18.572+0000 7f6b3d000640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b38117d60 con 0x7f6b3810cf80 2024-03-21T10:34:18.573 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:18.572+0000 7f6b3d000640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b38118060 con 0x7f6b381a2cc0 2024-03-21T10:34:18.573 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:18.572+0000 7f6b35e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6b38107eb0 0x7f6b3819e400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:34:18.573 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:18.572+0000 7f6b35e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6b38107eb0 0x7f6b3819e400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:35470/0 (socket says 172.21.15.47:35470) 2024-03-21T10:34:18.573 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:18.573+0000 7f6b35e00640 1 -- 172.21.15.47:0/400847838 learned_addr learned my addr 172.21.15.47:0/400847838 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:34:18.574 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:18.573+0000 7f6b36800640 1 --2- 172.21.15.47:0/400847838 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6b381a2cc0 0x7f6b381a3170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:34:18.574 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:18.573+0000 7f6b35400640 1 --2- 172.21.15.47:0/400847838 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6b3810cf80 0x7f6b3819e940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:34:18.574 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:18.573+0000 7f6b35e00640 1 -- 172.21.15.47:0/400847838 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6b381a2cc0 msgr2=0x7f6b381a3170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:34:18.574 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:18.573+0000 7f6b35e00640 1 --2- 172.21.15.47:0/400847838 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6b381a2cc0 0x7f6b381a3170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:18.574 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:18.573+0000 7f6b35e00640 1 -- 172.21.15.47:0/400847838 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6b3810cf80 msgr2=0x7f6b3819e940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:34:18.574 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:18.573+0000 7f6b35e00640 1 --2- 172.21.15.47:0/400847838 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6b3810cf80 0x7f6b3819e940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:18.574 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:18.573+0000 7f6b35e00640 1 -- 172.21.15.47:0/400847838 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b381a37b0 con 0x7f6b38107eb0 2024-03-21T10:34:18.574 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:18.573+0000 7f6b35400640 1 --2- 172.21.15.47:0/400847838 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6b3810cf80 0x7f6b3819e940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T10:34:18.574 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:18.573+0000 7f6b35e00640 1 --2- 172.21.15.47:0/400847838 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6b38107eb0 0x7f6b3819e400 secure :-1 s=READY pgs=1207 cs=0 l=1 rev1=1 crypto rx=0x7f6b2800db80 tx=0x7f6b2800b630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:34:18.574 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:18.574+0000 7f6b2ea00640 1 -- 172.21.15.47:0/400847838 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b28017070 con 0x7f6b38107eb0 2024-03-21T10:34:18.574 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:18.574+0000 7f6b2ea00640 1 -- 172.21.15.47:0/400847838 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f6b28003b70 con 0x7f6b38107eb0 2024-03-21T10:34:18.574 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:18.574+0000 7f6b3d000640 1 -- 172.21.15.47:0/400847838 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6b381a3aa0 con 0x7f6b38107eb0 2024-03-21T10:34:18.574 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:18.574+0000 7f6b2ea00640 1 -- 172.21.15.47:0/400847838 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b280104c0 con 0x7f6b38107eb0 2024-03-21T10:34:18.575 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:18.574+0000 7f6b3d000640 1 -- 172.21.15.47:0/400847838 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6b381ab230 con 0x7f6b38107eb0 2024-03-21T10:34:18.576 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:18.578+0000 7f6b3fe32740 1 Processor -- start 2024-03-21T10:34:18.576 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:18.578+0000 7f6b2ea00640 1 -- 172.21.15.47:0/400847838 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 27) v1 ==== 137985+0+0 (secure 0 0 0) 0x7f6b28004020 con 0x7f6b38107eb0 2024-03-21T10:34:18.576 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:18.578+0000 7f6b3fe32740 1 -- start start 2024-03-21T10:34:18.579 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:18.578+0000 7f6b3fe32740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5622c67dac00 0x5622c67d9950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:34:18.579 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:18.578+0000 7f6b3fe32740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5622c67d9f20 0x5622c67da3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:34:18.579 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:18.578+0000 7f6b3fe32740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5622c67e3290 0x5622c67e5680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:34:18.579 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:18.578+0000 7f6b3fe32740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x5622c673e250 con 0x5622c67dac00 2024-03-21T10:34:18.579 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:18.578+0000 7f6b3fe32740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x5622c66e6a30 con 0x5622c67d9f20 2024-03-21T10:34:18.579 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:18.578+0000 7f6b3fe32740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x5622c673a2f0 con 0x5622c67e3290 2024-03-21T10:34:18.579 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:18.578+0000 7f6b36800640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5622c67dac00 0x5622c67d9950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:34:18.579 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:18.578+0000 7f6b36800640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5622c67dac00 0x5622c67d9950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:35478/0 (socket says 172.21.15.47:35478) 2024-03-21T10:34:18.579 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:18.579+0000 7f6b36800640 1 -- 172.21.15.47:0/36164096 learned_addr learned my addr 172.21.15.47:0/36164096 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:34:18.579 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:18.579+0000 7f6b35e00640 1 --2- 172.21.15.47:0/36164096 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5622c67d9f20 0x5622c67da3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:34:18.579 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:18.579+0000 7f6b35400640 1 --2- 172.21.15.47:0/36164096 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5622c67e3290 0x5622c67e5680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:34:18.579 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:18.579+0000 7f6b36800640 1 -- 172.21.15.47:0/36164096 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5622c67e3290 msgr2=0x5622c67e5680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:34:18.579 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:18.579+0000 7f6b36800640 1 --2- 172.21.15.47:0/36164096 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5622c67e3290 0x5622c67e5680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:18.580 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:18.579+0000 7f6b36800640 1 -- 172.21.15.47:0/36164096 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5622c67d9f20 msgr2=0x5622c67da3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:34:18.580 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:18.579+0000 7f6b36800640 1 --2- 172.21.15.47:0/36164096 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5622c67d9f20 0x5622c67da3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:18.580 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:18.579+0000 7f6b36800640 1 -- 172.21.15.47:0/36164096 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5622c66d1690 con 0x5622c67dac00 2024-03-21T10:34:18.580 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:18.579+0000 7f6b35e00640 1 --2- 172.21.15.47:0/36164096 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5622c67d9f20 0x5622c67da3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T10:34:18.580 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:18.579+0000 7f6b35400640 1 --2- 172.21.15.47:0/36164096 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5622c67e3290 0x5622c67e5680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T10:34:18.580 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:18.580+0000 7f6b36800640 1 --2- 172.21.15.47:0/36164096 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5622c67dac00 0x5622c67d9950 secure :-1 s=READY pgs=1208 cs=0 l=1 rev1=1 crypto rx=0x7f6b30009900 tx=0x7f6b30005d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8b0f0c2fe598700f server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:34:18.580 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:18.580+0000 7f6b2ea00640 1 --2- 172.21.15.47:0/400847838 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f6b1c0a3090 0x7f6b1c0a5550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:34:18.580 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:18.580+0000 7f6b2ea00640 1 -- 172.21.15.47:0/400847838 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 9562+0+0 (secure 0 0 0) 0x7f6b280cfd80 con 0x7f6b38107eb0 2024-03-21T10:34:18.580 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:18.580+0000 7f6b27400640 1 -- 172.21.15.47:0/36164096 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b30044020 con 0x5622c67dac00 2024-03-21T10:34:18.580 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:18.580+0000 7f6b35e00640 1 --2- 172.21.15.47:0/400847838 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f6b1c0a3090 0x7f6b1c0a5550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:34:18.580 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:18.580+0000 7f6b27400640 1 -- 172.21.15.47:0/36164096 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f6b3002ccb0 con 0x5622c67dac00 2024-03-21T10:34:18.580 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:18.580+0000 7f6b27400640 1 -- 172.21.15.47:0/36164096 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b300046c0 con 0x5622c67dac00 2024-03-21T10:34:18.584 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:18.581+0000 7f6b3fe32740 1 -- 172.21.15.47:0/36164096 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5622c67dac00 msgr2=0x5622c67d9950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:34:18.584 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:18.581+0000 7f6b3fe32740 1 --2- 172.21.15.47:0/36164096 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5622c67dac00 0x5622c67d9950 secure :-1 s=READY pgs=1208 cs=0 l=1 rev1=1 crypto rx=0x7f6b30009900 tx=0x7f6b30005d50 comp rx=0 tx=0).stop 2024-03-21T10:34:18.584 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:18.581+0000 7f6b35e00640 1 --2- 172.21.15.47:0/400847838 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f6b1c0a3090 0x7f6b1c0a5550 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f6b280cd170 tx=0x7f6b2800df10 comp rx=0 tx=0).ready entity=mgr.15687 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:34:18.584 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:18.581+0000 7f6b3fe32740 1 -- 172.21.15.47:0/36164096 shutdown_connections 2024-03-21T10:34:18.584 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:18.581+0000 7f6b3fe32740 1 --2- 172.21.15.47:0/36164096 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5622c67e3290 0x5622c67e5680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:18.584 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:18.581+0000 7f6b3fe32740 1 --2- 172.21.15.47:0/36164096 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5622c67d9f20 0x5622c67da3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:18.584 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:18.581+0000 7f6b3fe32740 1 --2- 172.21.15.47:0/36164096 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5622c67dac00 0x5622c67d9950 unknown :-1 s=CLOSED pgs=1208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:18.584 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:18.581+0000 7f6b3fe32740 1 -- 172.21.15.47:0/36164096 >> 172.21.15.47:0/36164096 conn(0x5622c67a1200 msgr2=0x5622c67d4970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:34:18.584 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:18.581+0000 7f6b3fe32740 1 -- 172.21.15.47:0/36164096 shutdown_connections 2024-03-21T10:34:18.585 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:18.581+0000 7f6b3fe32740 1 -- 172.21.15.47:0/36164096 wait complete. 2024-03-21T10:34:18.585 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:18.582+0000 7f6b3fe32740 1 -- start start 2024-03-21T10:34:18.585 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:18.582+0000 7f6b3fe32740 1 Processor -- start 2024-03-21T10:34:18.585 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:18.582+0000 7f6b3fe32740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5622c67d9f20 0x5622c67e2960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:34:18.585 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:18.583+0000 7f6b3fe32740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5622c67dac00 0x5622c67dfcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:34:18.585 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:18.583+0000 7f6b3fe32740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5622c67e3290 0x5622c67e0230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:34:18.585 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:18.583+0000 7f6b3fe32740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x5622c67e60a0 con 0x5622c67dac00 2024-03-21T10:34:18.585 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:18.583+0000 7f6b3fe32740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x5622c66f9e80 con 0x5622c67d9f20 2024-03-21T10:34:18.585 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:18.583+0000 7f6b3fe32740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x5622c67e6220 con 0x5622c67e3290 2024-03-21T10:34:18.585 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:18.583+0000 7f6b3fe32740 10 client.0 _ll_register_callbacks cb 0x5622c670c130 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T10:34:18.585 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:18.583+0000 7f6b35400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5622c67dac00 0x5622c67dfcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:34:18.585 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:18.583+0000 7f6b35400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5622c67dac00 0x5622c67dfcf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:35480/0 (socket says 172.21.15.47:35480) 2024-03-21T10:34:18.585 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:18.583+0000 7f6b35400640 1 -- 172.21.15.47:0/3552725640 learned_addr learned my addr 172.21.15.47:0/3552725640 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:34:18.585 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:18.583+0000 7f6b36800640 1 --2- 172.21.15.47:0/3552725640 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5622c67d9f20 0x5622c67e2960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:34:18.586 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:18.583+0000 7f6b36800640 1 --2- 172.21.15.47:0/3552725640 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5622c67e3290 0x5622c67e0230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:34:18.586 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:18.583+0000 7f6b35400640 1 -- 172.21.15.47:0/3552725640 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5622c67e3290 msgr2=0x5622c67e0230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:34:18.586 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:18.583+0000 7f6b35400640 1 --2- 172.21.15.47:0/3552725640 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5622c67e3290 0x5622c67e0230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:18.586 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:18.584+0000 7f6b35400640 1 -- 172.21.15.47:0/3552725640 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5622c67d9f20 msgr2=0x5622c67e2960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:34:18.586 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:18.584+0000 7f6b35400640 1 --2- 172.21.15.47:0/3552725640 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5622c67d9f20 0x5622c67e2960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:18.586 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:18.584+0000 7f6b35400640 1 -- 172.21.15.47:0/3552725640 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5622c651c7d0 con 0x5622c67dac00 2024-03-21T10:34:18.586 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:18.584+0000 7f6b36800640 1 --2- 172.21.15.47:0/3552725640 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5622c67d9f20 0x5622c67e2960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T10:34:18.586 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:18.584+0000 7f6b35400640 1 --2- 172.21.15.47:0/3552725640 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5622c67dac00 0x5622c67dfcf0 secure :-1 s=READY pgs=1209 cs=0 l=1 rev1=1 crypto rx=0x7f6b20013790 tx=0x7f6b20013c50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ed728bc540a5fba4 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:34:18.586 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:18.584+0000 7f6b1be00640 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-03-21T10:34:18.586 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:18.584+0000 7f6b1be00640 1 -- 172.21.15.47:0/3552725640 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b20020060 con 0x5622c67dac00 2024-03-21T10:34:18.586 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:18.584+0000 7f6b1be00640 1 -- 172.21.15.47:0/3552725640 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(103..103 src has 1..103) v4 ==== 9562+0+0 (secure 0 0 0) 0x7f6b20011dd0 con 0x5622c67dac00 2024-03-21T10:34:18.586 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:18.584+0000 7f6b3fe32740 1 -- 172.21.15.47:0/3552725640 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f6b3002ccb0 con 0x5622c67dac00 2024-03-21T10:34:18.586 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:18.585+0000 7f6b1be00640 1 -- 172.21.15.47:0/3552725640 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6b20027050 con 0x5622c67dac00 2024-03-21T10:34:18.586 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:18.585+0000 7f6b3fe32740 10 client.18423 fetch_fsmap learned FSMap version 32 2024-03-21T10:34:18.587 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:18.585+0000 7f6b3fe32740 1 -- 172.21.15.47:0/3552725640 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x5622c64523f0 con 0x5622c67dac00 2024-03-21T10:34:18.587 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:18.585+0000 7f6b1be00640 1 -- 172.21.15.47:0/3552725640 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f6b3002ccb0 con 0x5622c67dac00 2024-03-21T10:34:18.587 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:18.585+0000 7f6b1be00640 1 -- 172.21.15.47:0/3552725640 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 32) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f6b2001a030 con 0x5622c67dac00 2024-03-21T10:34:18.587 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:18.585+0000 7f6b3fe32740 10 client.18423 fetch_fsmap finished waiting for FSMap version 32 2024-03-21T10:34:18.587 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:18.585+0000 7f6b3fe32740 10 client.18423 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-21T10:34:18.587 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:18.585+0000 7f6b3fe32740 20 client.18423 populate_metadata read hostname 'smithi047' 2024-03-21T10:34:18.587 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:18.585+0000 7f6b3fe32740 1 --2- 172.21.15.47:0/3552725640 >> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] conn(0x5622c67f3190 0x5622c67f5580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:34:18.587 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:18.585+0000 7f6b3fe32740 4 client.18423 mds_command: new command op to 34535 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T10:34:18.587 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:18.585+0000 7f6b3fe32740 1 -- 172.21.15.47:0/3552725640 --> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5622c674b790 con 0x5622c67f3190 2024-03-21T10:34:18.587 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:18.586+0000 7f6b36800640 1 --2- 172.21.15.47:0/3552725640 >> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] conn(0x5622c67f3190 0x5622c67f5580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:34:18.588 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:18.587+0000 7f6b1be00640 10 client.18423 ms_handle_connect on v2:172.21.15.100:6838/4173168440 2024-03-21T10:34:18.588 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:18.587+0000 7f6b36800640 1 --2- 172.21.15.47:0/3552725640 >> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] conn(0x5622c67f3190 0x5622c67f5580 crc :-1 s=READY pgs=205 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6dd74616e919c090 server_cookie=3985dfaaa227e6bc in_seq=0 out_seq=0 2024-03-21T10:34:18.588 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:18.588+0000 7f6b1be00640 1 -- 172.21.15.47:0/3552725640 <== mds.0 v2:172.21.15.100:6838/4173168440 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x5622c674b790 con 0x5622c67f3190 2024-03-21T10:34:18.588 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:18.588+0000 7f6b1be00640 10 client.18423 handle_command_reply: tid=0 2024-03-21T10:34:18.589 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:18.588+0000 7f6b3fe32740 1 client.18423 shutdown 2024-03-21T10:34:18.589 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:18.588+0000 7f6b3fe32740 1 -- 172.21.15.47:0/3552725640 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5622c67dac00 msgr2=0x5622c67dfcf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:34:18.589 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:18.588+0000 7f6b3fe32740 1 --2- 172.21.15.47:0/3552725640 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5622c67dac00 0x5622c67dfcf0 secure :-1 s=READY pgs=1209 cs=0 l=1 rev1=1 crypto rx=0x7f6b20013790 tx=0x7f6b20013c50 comp rx=0 tx=0).stop 2024-03-21T10:34:18.589 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:18.588+0000 7f6b3fe32740 1 -- 172.21.15.47:0/3552725640 shutdown_connections 2024-03-21T10:34:18.589 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:18.588+0000 7f6b3fe32740 1 --2- 172.21.15.47:0/3552725640 >> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] conn(0x5622c67f3190 0x5622c67f5580 crc :-1 s=READY pgs=205 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:18.589 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:18.588+0000 7f6b3fe32740 1 --2- 172.21.15.47:0/3552725640 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5622c67e3290 0x5622c67e0230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:18.589 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:18.588+0000 7f6b3fe32740 1 --2- 172.21.15.47:0/3552725640 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5622c67dac00 0x5622c67dfcf0 unknown :-1 s=CLOSED pgs=1209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:18.589 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:18.588+0000 7f6b1be00640 0 client.18423 ms_handle_reset on v2:172.21.15.100:6838/4173168440 2024-03-21T10:34:18.589 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:18.588+0000 7f6b3fe32740 1 --2- 172.21.15.47:0/3552725640 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5622c67d9f20 0x5622c67e2960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:18.589 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:18.588+0000 7f6b3fe32740 1 -- 172.21.15.47:0/3552725640 >> 172.21.15.47:0/3552725640 conn(0x5622c67a1200 msgr2=0x5622c67d66c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:34:18.589 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:18.588+0000 7f6b3fe32740 1 -- 172.21.15.47:0/3552725640 shutdown_connections 2024-03-21T10:34:18.589 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:18.588+0000 7f6b3fe32740 1 -- 172.21.15.47:0/3552725640 wait complete. 2024-03-21T10:34:18.589 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:18.588+0000 7f6b3fe32740 20 client.18423 trim_cache size 0 max 16384 2024-03-21T10:34:18.606 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:18.606+0000 7f6b3fe32740 1 Processor -- start 2024-03-21T10:34:18.606 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:18.606+0000 7f6b3fe32740 1 -- start start 2024-03-21T10:34:18.606 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:18.606+0000 7f6b3fe32740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5622c67d9f20 0x5622c67e2420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:34:18.606 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:18.606+0000 7f6b3fe32740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5622c67dac00 0x5622c67e2960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:34:18.606 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:18.606+0000 7f6b3fe32740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5622c67e3290 0x5622c67e4680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:34:18.606 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:18.606+0000 7f6b3fe32740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b381b5060 con 0x5622c67e3290 2024-03-21T10:34:18.606 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:18.606+0000 7f6b3fe32740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b381b69e0 con 0x5622c67d9f20 2024-03-21T10:34:18.606 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:18.606+0000 7f6b3fe32740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b381b56c0 con 0x5622c67dac00 2024-03-21T10:34:18.606 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:18.607+0000 7f6b36800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5622c67d9f20 0x5622c67e2420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:34:18.606 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:18.607+0000 7f6b36800640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5622c67e3290 0x5622c67e4680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:34:18.606 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:18.607+0000 7f6b35400640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5622c67dac00 0x5622c67e2960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:34:18.606 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:18.607+0000 7f6b36800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5622c67d9f20 0x5622c67e2420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:50156/0 (socket says 172.21.15.47:50156) 2024-03-21T10:34:18.607 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:18.607+0000 7f6b35400640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5622c67dac00 0x5622c67e2960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.47:47024/0 (socket says 172.21.15.47:47024) 2024-03-21T10:34:18.607 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:18.607+0000 7f6b36800640 1 -- 172.21.15.47:0/3898738870 learned_addr learned my addr 172.21.15.47:0/3898738870 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:34:18.607 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:18.607+0000 7f6b36800640 1 -- 172.21.15.47:0/3898738870 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5622c67dac00 msgr2=0x5622c67e2960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:34:18.607 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:18.607+0000 7f6b36800640 1 --2- 172.21.15.47:0/3898738870 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5622c67dac00 0x5622c67e2960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:18.607 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:18.607+0000 7f6b36800640 1 -- 172.21.15.47:0/3898738870 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5622c67e3290 msgr2=0x5622c67e4680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:34:18.607 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:18.607+0000 7f6b36800640 1 --2- 172.21.15.47:0/3898738870 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5622c67e3290 0x5622c67e4680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:18.607 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:18.607+0000 7f6b36800640 1 -- 172.21.15.47:0/3898738870 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5622c64120f0 con 0x5622c67d9f20 2024-03-21T10:34:18.607 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:18.607+0000 7f6b36800640 1 --2- 172.21.15.47:0/3898738870 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5622c67d9f20 0x5622c67e2420 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f6b30000c00 tx=0x7f6b300052a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ab07d97e749356b1 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:34:18.610 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:18.608+0000 7f6b26a00640 1 -- 172.21.15.47:0/3898738870 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b30044040 con 0x5622c67d9f20 2024-03-21T10:34:18.610 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:18.608+0000 7f6b26a00640 1 -- 172.21.15.47:0/3898738870 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f6b30004040 con 0x5622c67d9f20 2024-03-21T10:34:18.611 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:18.608+0000 7f6b26a00640 1 -- 172.21.15.47:0/3898738870 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b30003970 con 0x5622c67d9f20 2024-03-21T10:34:18.611 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:18.609+0000 7f6b3fe32740 1 -- 172.21.15.47:0/3898738870 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5622c67d9f20 msgr2=0x5622c67e2420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:34:18.611 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:18.609+0000 7f6b3fe32740 1 --2- 172.21.15.47:0/3898738870 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5622c67d9f20 0x5622c67e2420 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f6b30000c00 tx=0x7f6b300052a0 comp rx=0 tx=0).stop 2024-03-21T10:34:18.611 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:18.609+0000 7f6b3fe32740 1 -- 172.21.15.47:0/3898738870 shutdown_connections 2024-03-21T10:34:18.611 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:18.609+0000 7f6b3fe32740 1 --2- 172.21.15.47:0/3898738870 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5622c67e3290 0x5622c67e4680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:18.611 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:18.609+0000 7f6b3fe32740 1 --2- 172.21.15.47:0/3898738870 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5622c67dac00 0x5622c67e2960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:18.611 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:18.609+0000 7f6b3fe32740 1 --2- 172.21.15.47:0/3898738870 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5622c67d9f20 0x5622c67e2420 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:18.611 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:18.609+0000 7f6b3fe32740 1 -- 172.21.15.47:0/3898738870 >> 172.21.15.47:0/3898738870 conn(0x5622c67a1200 msgr2=0x5622c67d44a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:34:18.611 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:18.609+0000 7f6b3fe32740 1 -- 172.21.15.47:0/3898738870 shutdown_connections 2024-03-21T10:34:18.611 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:18.609+0000 7f6b3fe32740 1 -- 172.21.15.47:0/3898738870 wait complete. 2024-03-21T10:34:18.611 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:18.609+0000 7f6b3fe32740 1 -- start start 2024-03-21T10:34:18.611 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:18.609+0000 7f6b3fe32740 1 Processor -- start 2024-03-21T10:34:18.611 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:18.610+0000 7f6b3fe32740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5622c67d9f20 0x5622c67dc560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:34:18.612 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:18.610+0000 7f6b3fe32740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5622c67dac00 0x5622c67dcaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:34:18.612 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:18.610+0000 7f6b3fe32740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5622c67e3290 0x5622c67dcfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:34:18.612 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:18.610+0000 7f6b3fe32740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b381b6380 con 0x5622c67dac00 2024-03-21T10:34:18.612 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:18.610+0000 7f6b3fe32740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b381b5d20 con 0x5622c67d9f20 2024-03-21T10:34:18.612 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:18.610+0000 7f6b3fe32740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b381b3240 con 0x5622c67e3290 2024-03-21T10:34:18.612 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:18.610+0000 7f6b3fe32740 10 client.0 _ll_register_callbacks cb 0x7f6b20009630 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T10:34:18.612 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:18.610+0000 7f6b35400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5622c67dac00 0x5622c67dcaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:34:18.612 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:18.610+0000 7f6b35400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5622c67dac00 0x5622c67dcaa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:35500/0 (socket says 172.21.15.47:35500) 2024-03-21T10:34:18.612 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:18.610+0000 7f6b35400640 1 -- 172.21.15.47:0/1618845754 learned_addr learned my addr 172.21.15.47:0/1618845754 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:34:18.612 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:18.610+0000 7f6b36800640 1 --2- 172.21.15.47:0/1618845754 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5622c67d9f20 0x5622c67dc560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:34:18.612 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:18.611+0000 7f6b36800640 1 --2- 172.21.15.47:0/1618845754 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5622c67e3290 0x5622c67dcfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:34:18.612 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:18.611+0000 7f6b35400640 1 -- 172.21.15.47:0/1618845754 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5622c67e3290 msgr2=0x5622c67dcfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:34:18.612 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:18.611+0000 7f6b35400640 1 --2- 172.21.15.47:0/1618845754 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5622c67e3290 0x5622c67dcfe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:18.612 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:18.611+0000 7f6b35400640 1 -- 172.21.15.47:0/1618845754 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5622c67d9f20 msgr2=0x5622c67dc560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:34:18.612 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:18.611+0000 7f6b35400640 1 --2- 172.21.15.47:0/1618845754 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5622c67d9f20 0x5622c67dc560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:18.613 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:18.611+0000 7f6b35400640 1 -- 172.21.15.47:0/1618845754 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5622c684dbe0 con 0x5622c67dac00 2024-03-21T10:34:18.613 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:18.611+0000 7f6b36800640 1 --2- 172.21.15.47:0/1618845754 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5622c67d9f20 0x5622c67dc560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T10:34:18.613 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:18.611+0000 7f6b35400640 1 --2- 172.21.15.47:0/1618845754 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5622c67dac00 0x5622c67dcaa0 secure :-1 s=READY pgs=1210 cs=0 l=1 rev1=1 crypto rx=0x7f6b200059f0 tx=0x7f6b2000e9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4628588becfa8b08 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:34:18.613 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:18.611+0000 7f6b1be00640 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-03-21T10:34:18.613 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:18.611+0000 7f6b1be00640 1 -- 172.21.15.47:0/1618845754 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b20000e60 con 0x5622c67dac00 2024-03-21T10:34:18.613 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:18.611+0000 7f6b1be00640 1 -- 172.21.15.47:0/1618845754 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(103..103 src has 1..103) v4 ==== 9562+0+0 (secure 0 0 0) 0x7f6b20011de0 con 0x5622c67dac00 2024-03-21T10:34:18.613 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:18.611+0000 7f6b3fe32740 1 -- 172.21.15.47:0/1618845754 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f6b30004040 con 0x5622c67dac00 2024-03-21T10:34:18.613 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:18.611+0000 7f6b1be00640 1 -- 172.21.15.47:0/1618845754 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6b2000ebc0 con 0x5622c67dac00 2024-03-21T10:34:18.613 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:18.612+0000 7f6b3fe32740 10 client.18429 fetch_fsmap learned FSMap version 32 2024-03-21T10:34:18.613 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:18.612+0000 7f6b3fe32740 1 -- 172.21.15.47:0/1618845754 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x5622c684eee0 con 0x5622c67dac00 2024-03-21T10:34:18.613 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:18.612+0000 7f6b1be00640 1 -- 172.21.15.47:0/1618845754 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f6b30004040 con 0x5622c67dac00 2024-03-21T10:34:18.613 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:18.612+0000 7f6b1be00640 1 -- 172.21.15.47:0/1618845754 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 32) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f6b2001a070 con 0x5622c67dac00 2024-03-21T10:34:18.613 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:18.612+0000 7f6b3fe32740 10 client.18429 fetch_fsmap finished waiting for FSMap version 32 2024-03-21T10:34:18.614 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:18.612+0000 7f6b3fe32740 10 client.18429 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-21T10:34:18.614 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:18.612+0000 7f6b3fe32740 20 client.18429 populate_metadata read hostname 'smithi047' 2024-03-21T10:34:18.614 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:18.612+0000 7f6b3fe32740 1 --2- 172.21.15.47:0/1618845754 >> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] conn(0x5622c684f600 0x5622c6851a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:34:18.614 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:18.612+0000 7f6b3fe32740 4 client.18429 mds_command: new command op to 34535 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-21T10:34:18.614 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:18.612+0000 7f6b3fe32740 1 -- 172.21.15.47:0/1618845754 --> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x5622c6745ae0 con 0x5622c684f600 2024-03-21T10:34:18.614 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:18.612+0000 7f6b36800640 1 --2- 172.21.15.47:0/1618845754 >> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] conn(0x5622c684f600 0x5622c6851a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:34:18.615 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:18.613+0000 7f6b36800640 1 --2- 172.21.15.47:0/1618845754 >> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] conn(0x5622c684f600 0x5622c6851a20 crc :-1 s=READY pgs=206 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=91dcef373848ac0f server_cookie=c770ffdc799b284e in_seq=0 out_seq=0 2024-03-21T10:34:18.616 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:18.613+0000 7f6b1be00640 10 client.18429 ms_handle_connect on v2:172.21.15.100:6838/4173168440 2024-03-21T10:34:18.616 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:18.613+0000 7f6b1be00640 1 -- 172.21.15.47:0/1618845754 <== mds.0 v2:172.21.15.100:6838/4173168440 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x5622c6745ae0 con 0x5622c684f600 2024-03-21T10:34:18.616 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:18.613+0000 7f6b1be00640 10 client.18429 handle_command_reply: tid=0 2024-03-21T10:34:18.616 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:18.613+0000 7f6b3fe32740 1 client.18429 shutdown 2024-03-21T10:34:18.616 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:18.614+0000 7f6b3fe32740 1 -- 172.21.15.47:0/1618845754 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5622c67dac00 msgr2=0x5622c67dcaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:34:18.616 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:18.614+0000 7f6b3fe32740 1 --2- 172.21.15.47:0/1618845754 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5622c67dac00 0x5622c67dcaa0 secure :-1 s=READY pgs=1210 cs=0 l=1 rev1=1 crypto rx=0x7f6b200059f0 tx=0x7f6b2000e9c0 comp rx=0 tx=0).stop 2024-03-21T10:34:18.616 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:18.614+0000 7f6b3fe32740 1 -- 172.21.15.47:0/1618845754 shutdown_connections 2024-03-21T10:34:18.616 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:18.614+0000 7f6b3fe32740 1 --2- 172.21.15.47:0/1618845754 >> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] conn(0x5622c684f600 0x5622c6851a20 crc :-1 s=READY pgs=206 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:18.616 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:18.614+0000 7f6b3fe32740 1 --2- 172.21.15.47:0/1618845754 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5622c67e3290 0x5622c67dcfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:18.616 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:18.614+0000 7f6b3fe32740 1 --2- 172.21.15.47:0/1618845754 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5622c67dac00 0x5622c67dcaa0 unknown :-1 s=CLOSED pgs=1210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:18.616 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:18.614+0000 7f6b3fe32740 1 --2- 172.21.15.47:0/1618845754 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5622c67d9f20 0x5622c67dc560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:18.616 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:18.614+0000 7f6b1be00640 0 client.18429 ms_handle_reset on v2:172.21.15.100:6838/4173168440 2024-03-21T10:34:18.616 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:18.614+0000 7f6b3fe32740 1 -- 172.21.15.47:0/1618845754 >> 172.21.15.47:0/1618845754 conn(0x5622c67a1200 msgr2=0x5622c67de500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:34:18.616 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:18.614+0000 7f6b3fe32740 1 -- 172.21.15.47:0/1618845754 shutdown_connections 2024-03-21T10:34:18.617 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:18.614+0000 7f6b3fe32740 1 -- 172.21.15.47:0/1618845754 wait complete. 2024-03-21T10:34:18.617 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:18.614+0000 7f6b3fe32740 20 client.18429 trim_cache size 0 max 16384 2024-03-21T10:34:18.617 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-03-21T10:34:18.617 INFO:teuthology.orchestra.run.smithi047.stdout: "status": "no active scrubs running", 2024-03-21T10:34:18.617 INFO:teuthology.orchestra.run.smithi047.stdout: "scrubs": {} 2024-03-21T10:34:18.617 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-03-21T10:34:18.618 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:18.615+0000 7f6b3d000640 1 -- 172.21.15.47:0/400847838 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f6b1c0a3090 msgr2=0x7f6b1c0a5550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:34:18.618 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:18.615+0000 7f6b3d000640 1 --2- 172.21.15.47:0/400847838 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f6b1c0a3090 0x7f6b1c0a5550 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f6b280cd170 tx=0x7f6b2800df10 comp rx=0 tx=0).stop 2024-03-21T10:34:18.618 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:18.615+0000 7f6b3d000640 1 -- 172.21.15.47:0/400847838 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6b38107eb0 msgr2=0x7f6b3819e400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:34:18.618 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:18.615+0000 7f6b3d000640 1 --2- 172.21.15.47:0/400847838 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6b38107eb0 0x7f6b3819e400 secure :-1 s=READY pgs=1207 cs=0 l=1 rev1=1 crypto rx=0x7f6b2800db80 tx=0x7f6b2800b630 comp rx=0 tx=0).stop 2024-03-21T10:34:18.618 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:18.615+0000 7f6b3d000640 1 -- 172.21.15.47:0/400847838 shutdown_connections 2024-03-21T10:34:18.618 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:18.615+0000 7f6b3d000640 1 --2- 172.21.15.47:0/400847838 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6b381a2cc0 0x7f6b381a3170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:18.618 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:18.615+0000 7f6b3d000640 1 --2- 172.21.15.47:0/400847838 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f6b1c0a3090 0x7f6b1c0a5550 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:18.618 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:18.615+0000 7f6b3d000640 1 --2- 172.21.15.47:0/400847838 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6b3810cf80 0x7f6b3819e940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:18.618 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:18.615+0000 7f6b3d000640 1 --2- 172.21.15.47:0/400847838 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6b38107eb0 0x7f6b3819e400 unknown :-1 s=CLOSED pgs=1207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:18.618 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:18.615+0000 7f6b3d000640 1 -- 172.21.15.47:0/400847838 >> 172.21.15.47:0/400847838 conn(0x7f6b38075be0 msgr2=0x7f6b38077f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:34:18.619 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:18.615+0000 7f6b3d000640 1 -- 172.21.15.47:0/400847838 shutdown_connections 2024-03-21T10:34:18.619 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:18.615+0000 7f6b3d000640 1 -- 172.21.15.47:0/400847838 wait complete. 2024-03-21T10:34:18.631 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-03-21T10:34:18.631 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-21T10:34:18.849 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:18.850+0000 7f61b7200640 1 -- 172.21.15.47:0/2362144673 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f61b810dd50 msgr2=0x7f61b8110140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:34:18.849 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:18.850+0000 7f61b7200640 1 --2- 172.21.15.47:0/2362144673 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f61b810dd50 0x7f61b8110140 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f61b0009ca0 tx=0x7f61b002c100 comp rx=0 tx=0).stop 2024-03-21T10:34:18.849 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:18.850+0000 7f61b7200640 1 -- 172.21.15.47:0/2362144673 shutdown_connections 2024-03-21T10:34:18.849 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:18.850+0000 7f61b7200640 1 --2- 172.21.15.47:0/2362144673 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f61b810dd50 0x7f61b8110140 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:18.849 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:18.850+0000 7f61b7200640 1 --2- 172.21.15.47:0/2362144673 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f61b8068d40 0x7f61b80691a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:18.849 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:18.850+0000 7f61b7200640 1 --2- 172.21.15.47:0/2362144673 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f61b81023b0 0x7f61b8068800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:18.849 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:18.850+0000 7f61b7200640 1 -- 172.21.15.47:0/2362144673 >> 172.21.15.47:0/2362144673 conn(0x7f61b8075fb0 msgr2=0x7f61b80763c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:34:18.849 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:18.850+0000 7f61b7200640 1 -- 172.21.15.47:0/2362144673 shutdown_connections 2024-03-21T10:34:18.849 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:18.850+0000 7f61b7200640 1 -- 172.21.15.47:0/2362144673 wait complete. 2024-03-21T10:34:18.851 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:18.852+0000 7f61b7200640 1 Processor -- start 2024-03-21T10:34:18.851 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:18.852+0000 7f61b7200640 1 -- start start 2024-03-21T10:34:18.851 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:18.852+0000 7f61b7200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f61b8068d40 0x7f61b8199d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:34:18.851 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:18.852+0000 7f61b7200640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f61b81023b0 0x7f61b819a2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:34:18.851 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:18.852+0000 7f61b7200640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f61b810dd50 0x7f61b819e650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:34:18.851 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:18.852+0000 7f61b7200640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61b8113950 con 0x7f61b81023b0 2024-03-21T10:34:18.851 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:18.852+0000 7f61b7200640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61b81137d0 con 0x7f61b8068d40 2024-03-21T10:34:18.851 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:18.852+0000 7f61b7200640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61b8113ad0 con 0x7f61b810dd50 2024-03-21T10:34:18.851 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:18.853+0000 7f61b5400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f61b81023b0 0x7f61b819a2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:34:18.852 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:18.853+0000 7f61b5e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f61b8068d40 0x7f61b8199d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:34:18.852 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:18.853+0000 7f61b5e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f61b8068d40 0x7f61b8199d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:50186/0 (socket says 172.21.15.47:50186) 2024-03-21T10:34:18.852 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:18.853+0000 7f61b5e00640 1 -- 172.21.15.47:0/959131576 learned_addr learned my addr 172.21.15.47:0/959131576 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:34:18.852 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:18.853+0000 7f61b6800640 1 --2- 172.21.15.47:0/959131576 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f61b810dd50 0x7f61b819e650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:34:18.853 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:18.853+0000 7f61b5e00640 1 -- 172.21.15.47:0/959131576 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f61b810dd50 msgr2=0x7f61b819e650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:34:18.853 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:18.853+0000 7f61b5e00640 1 --2- 172.21.15.47:0/959131576 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f61b810dd50 0x7f61b819e650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:18.853 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:18.853+0000 7f61b5e00640 1 -- 172.21.15.47:0/959131576 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f61b81023b0 msgr2=0x7f61b819a2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:34:18.853 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:18.853+0000 7f61b5e00640 1 --2- 172.21.15.47:0/959131576 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f61b81023b0 0x7f61b819a2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:18.853 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:18.853+0000 7f61b5e00640 1 -- 172.21.15.47:0/959131576 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61b819ed80 con 0x7f61b8068d40 2024-03-21T10:34:18.853 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:18.854+0000 7f61b5e00640 1 --2- 172.21.15.47:0/959131576 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f61b8068d40 0x7f61b8199d90 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f61a800c920 tx=0x7f61a800cde0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:34:18.853 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:18.854+0000 7f61aea00640 1 -- 172.21.15.47:0/959131576 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61a8007c70 con 0x7f61b8068d40 2024-03-21T10:34:18.853 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:18.854+0000 7f61b7200640 1 -- 172.21.15.47:0/959131576 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f61b819f070 con 0x7f61b8068d40 2024-03-21T10:34:18.853 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:18.854+0000 7f61b7200640 1 -- 172.21.15.47:0/959131576 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f61b81a6900 con 0x7f61b8068d40 2024-03-21T10:34:18.853 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:18.855+0000 7f61aea00640 1 -- 172.21.15.47:0/959131576 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f61a8007e10 con 0x7f61b8068d40 2024-03-21T10:34:18.854 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:18.855+0000 7f61aea00640 1 -- 172.21.15.47:0/959131576 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61a8004de0 con 0x7f61b8068d40 2024-03-21T10:34:18.854 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:18.855+0000 7f61b7200640 1 -- 172.21.15.47:0/959131576 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6198005290 con 0x7f61b8068d40 2024-03-21T10:34:18.856 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:18.858+0000 7f61aea00640 1 -- 172.21.15.47:0/959131576 <== mon.2 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 27) v1 ==== 137985+0+0 (secure 0 0 0) 0x7f61a8004f80 con 0x7f61b8068d40 2024-03-21T10:34:18.858 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:18.859+0000 7f61aea00640 1 --2- 172.21.15.47:0/959131576 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f619c0a3110 0x7f619c0a55d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:34:18.858 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:18.860+0000 7f61aea00640 1 -- 172.21.15.47:0/959131576 <== mon.2 v2:172.21.15.100:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 9562+0+0 (secure 0 0 0) 0x7f61a80cfbc0 con 0x7f61b8068d40 2024-03-21T10:34:18.858 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:18.860+0000 7f61b5400640 1 --2- 172.21.15.47:0/959131576 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f619c0a3110 0x7f619c0a55d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:34:18.858 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:18.861+0000 7f61b5400640 1 --2- 172.21.15.47:0/959131576 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f619c0a3110 0x7f619c0a55d0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f61b8076840 tx=0x7f61a4005f90 comp rx=0 tx=0).ready entity=mgr.15687 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:34:18.869 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:18.871+0000 7f61aea00640 1 -- 172.21.15.47:0/959131576 <== mon.2 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f61a8014030 con 0x7f61b8068d40 2024-03-21T10:34:19.153 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:19.155+0000 7f61b7200640 1 -- 172.21.15.47:0/959131576 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f6198005580 con 0x7f61b8068d40 2024-03-21T10:34:19.154 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:19.156+0000 7f61aea00640 1 -- 172.21.15.47:0/959131576 <== mon.2 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 32 v32) v1 ==== 94+0+11471 (secure 0 0 0) 0x7f61a809c2d0 con 0x7f61b8068d40 2024-03-21T10:34:19.155 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-21T10:34:19.155 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":32,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14511,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6835/735724037","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6834","nonce":735724037},{"type":"v1","addr":"172.21.15.47:6835","nonce":735724037}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14529,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6837/2248191724","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6836","nonce":2248191724},{"type":"v1","addr":"172.21.15.47:6837","nonce":2248191724}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":32,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-21T10:21:58.329063+0000","modified":"2024-03-21T10:26:17.169892+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":34535,"mds_1":34493,"mds_2":34454,"mds_3":34427,"mds_4":14619},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14604":{"gid":14604,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.47:6839/2881335356","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6838","nonce":2881335356},{"type":"v1","addr":"172.21.15.47:6839","nonce":2881335356}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_14619":{"gid":14619,"name":"a","rank":4,"incarnation":27,"state":"up:active","state_seq":11,"addr":"172.21.15.47:6841/3401578978","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6840","nonce":3401578978},{"type":"v1","addr":"172.21.15.47:6841","nonce":3401578978}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24406":{"gid":24406,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.157:6833/4000010136","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":4000010136},{"type":"v1","addr":"172.21.15.157:6833","nonce":4000010136}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24457":{"gid":24457,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.157:6835/3902868651","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":3902868651},{"type":"v1","addr":"172.21.15.157:6835","nonce":3902868651}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24475":{"gid":24475,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.157:6837/1908087204","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6836","nonce":1908087204},{"type":"v1","addr":"172.21.15.157:6837","nonce":1908087204}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_25729":{"gid":25729,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.157:6839/1998408416","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6838","nonce":1998408416},{"type":"v1","addr":"172.21.15.157:6839","nonce":1998408416}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_34427":{"gid":34427,"name":"k","rank":3,"incarnation":25,"state":"up:active","state_seq":20,"addr":"172.21.15.100:6833/2723302038","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6832","nonce":2723302038},{"type":"v1","addr":"172.21.15.100:6833","nonce":2723302038}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_34454":{"gid":34454,"name":"e","rank":2,"incarnation":22,"state":"up:active","state_seq":17,"addr":"172.21.15.100:6835/2597849957","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":2597849957},{"type":"v1","addr":"172.21.15.100:6835","nonce":2597849957}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_34493":{"gid":34493,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":13,"addr":"172.21.15.100:6837/3497644162","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6836","nonce":3497644162},{"type":"v1","addr":"172.21.15.100:6837","nonce":3497644162}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_34535":{"gid":34535,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.100:6839/4173168440","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6838","nonce":4173168440},{"type":"v1","addr":"172.21.15.100:6839","nonce":4173168440}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":34535,"qdb_cluster":[34427,14619,34454,34535,34493]},"id":1}]} 2024-03-21T10:34:19.155 INFO:teuthology.orchestra.run.smithi047.stderr:dumped fsmap epoch 32 2024-03-21T10:34:19.157 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:19.159+0000 7f61b7200640 1 -- 172.21.15.47:0/959131576 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f619c0a3110 msgr2=0x7f619c0a55d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:34:19.157 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:19.159+0000 7f61b7200640 1 --2- 172.21.15.47:0/959131576 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f619c0a3110 0x7f619c0a55d0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f61b8076840 tx=0x7f61a4005f90 comp rx=0 tx=0).stop 2024-03-21T10:34:19.157 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:19.159+0000 7f61b7200640 1 -- 172.21.15.47:0/959131576 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f61b8068d40 msgr2=0x7f61b8199d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:34:19.158 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:19.159+0000 7f61b7200640 1 --2- 172.21.15.47:0/959131576 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f61b8068d40 0x7f61b8199d90 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f61a800c920 tx=0x7f61a800cde0 comp rx=0 tx=0).stop 2024-03-21T10:34:19.158 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:19.159+0000 7f61b7200640 1 -- 172.21.15.47:0/959131576 shutdown_connections 2024-03-21T10:34:19.158 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:19.159+0000 7f61b7200640 1 --2- 172.21.15.47:0/959131576 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f61b810dd50 0x7f61b819e650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:19.158 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:19.159+0000 7f61b7200640 1 --2- 172.21.15.47:0/959131576 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f61b81023b0 0x7f61b819a2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:19.158 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:19.159+0000 7f61b7200640 1 --2- 172.21.15.47:0/959131576 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f619c0a3110 0x7f619c0a55d0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:19.158 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:19.159+0000 7f61b7200640 1 --2- 172.21.15.47:0/959131576 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f61b8068d40 0x7f61b8199d90 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:19.158 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:19.159+0000 7f61b7200640 1 -- 172.21.15.47:0/959131576 >> 172.21.15.47:0/959131576 conn(0x7f61b8075fb0 msgr2=0x7f61b80ff640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:34:19.158 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:19.159+0000 7f61b7200640 1 -- 172.21.15.47:0/959131576 shutdown_connections 2024-03-21T10:34:19.158 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:19.159+0000 7f61b7200640 1 -- 172.21.15.47:0/959131576 wait complete. 2024-03-21T10:34:19.185 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-21T10:34:19.186 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-03-21T10:34:19.423 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:19.424+0000 7fa63b200640 1 -- 172.21.15.47:0/1242714565 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa63c109730 msgr2=0x7fa63c109b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:34:19.423 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:19.424+0000 7fa63b200640 1 --2- 172.21.15.47:0/1242714565 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa63c109730 0x7fa63c109b10 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7fa62c009860 tx=0x7fa62c02bb70 comp rx=0 tx=0).stop 2024-03-21T10:34:19.423 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:19.424+0000 7fa63b200640 1 -- 172.21.15.47:0/1242714565 shutdown_connections 2024-03-21T10:34:19.423 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:19.424+0000 7fa63b200640 1 --2- 172.21.15.47:0/1242714565 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa63c1040e0 0x7fa63c1105f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:19.423 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:19.424+0000 7fa63b200640 1 --2- 172.21.15.47:0/1242714565 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa63c103720 0x7fa63c103ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:19.423 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:19.424+0000 7fa63b200640 1 --2- 172.21.15.47:0/1242714565 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa63c109730 0x7fa63c109b10 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:19.423 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:19.424+0000 7fa63b200640 1 -- 172.21.15.47:0/1242714565 >> 172.21.15.47:0/1242714565 conn(0x7fa63c0ff1b0 msgr2=0x7fa63c1015d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:34:19.423 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:19.424+0000 7fa63b200640 1 -- 172.21.15.47:0/1242714565 shutdown_connections 2024-03-21T10:34:19.423 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:19.424+0000 7fa63b200640 1 -- 172.21.15.47:0/1242714565 wait complete. 2024-03-21T10:34:19.426 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:19.425+0000 7fa63b200640 1 Processor -- start 2024-03-21T10:34:19.426 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:19.426+0000 7fa63b200640 1 -- start start 2024-03-21T10:34:19.426 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:19.426+0000 7fa63b200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa63c103720 0x7fa63c199f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:34:19.426 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:19.426+0000 7fa63b200640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa63c1040e0 0x7fa63c19a450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:34:19.427 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:19.426+0000 7fa63b200640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa63c109730 0x7fa63c19e7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:34:19.427 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:19.426+0000 7fa63b200640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa63c1139b0 con 0x7fa63c1040e0 2024-03-21T10:34:19.427 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:19.426+0000 7fa63b200640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa63c113830 con 0x7fa63c103720 2024-03-21T10:34:19.427 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:19.426+0000 7fa63b200640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa63c113b30 con 0x7fa63c109730 2024-03-21T10:34:19.427 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:19.426+0000 7fa631400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa63c1040e0 0x7fa63c19a450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:34:19.427 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:19.426+0000 7fa631400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa63c1040e0 0x7fa63c19a450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:35526/0 (socket says 172.21.15.47:35526) 2024-03-21T10:34:19.427 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:19.426+0000 7fa631400640 1 -- 172.21.15.47:0/4282186141 learned_addr learned my addr 172.21.15.47:0/4282186141 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:34:19.427 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:19.427+0000 7fa639e00640 1 --2- 172.21.15.47:0/4282186141 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa63c103720 0x7fa63c199f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:34:19.427 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:19.427+0000 7fa63a800640 1 --2- 172.21.15.47:0/4282186141 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa63c109730 0x7fa63c19e7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:34:19.427 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:19.427+0000 7fa631400640 1 -- 172.21.15.47:0/4282186141 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa63c109730 msgr2=0x7fa63c19e7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:34:19.427 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:19.427+0000 7fa631400640 1 --2- 172.21.15.47:0/4282186141 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa63c109730 0x7fa63c19e7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:19.427 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:19.427+0000 7fa631400640 1 -- 172.21.15.47:0/4282186141 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa63c103720 msgr2=0x7fa63c199f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:34:19.427 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:19.427+0000 7fa631400640 1 --2- 172.21.15.47:0/4282186141 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa63c103720 0x7fa63c199f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:19.427 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:19.427+0000 7fa631400640 1 -- 172.21.15.47:0/4282186141 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa63c19ef00 con 0x7fa63c1040e0 2024-03-21T10:34:19.428 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:19.427+0000 7fa631400640 1 --2- 172.21.15.47:0/4282186141 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa63c1040e0 0x7fa63c19a450 secure :-1 s=READY pgs=1211 cs=0 l=1 rev1=1 crypto rx=0x7fa62400e9f0 tx=0x7fa62400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:34:19.428 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:19.427+0000 7fa633400640 1 -- 172.21.15.47:0/4282186141 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa62400cd80 con 0x7fa63c1040e0 2024-03-21T10:34:19.428 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:19.427+0000 7fa633400640 1 -- 172.21.15.47:0/4282186141 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fa6240108e0 con 0x7fa63c1040e0 2024-03-21T10:34:19.428 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:19.427+0000 7fa63b200640 1 -- 172.21.15.47:0/4282186141 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa63c19f1f0 con 0x7fa63c1040e0 2024-03-21T10:34:19.428 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:19.427+0000 7fa633400640 1 -- 172.21.15.47:0/4282186141 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa624018920 con 0x7fa63c1040e0 2024-03-21T10:34:19.428 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:19.428+0000 7fa63b200640 1 -- 172.21.15.47:0/4282186141 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa63c1a6a80 con 0x7fa63c1040e0 2024-03-21T10:34:19.429 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:19.431+0000 7fa633400640 1 -- 172.21.15.47:0/4282186141 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 27) v1 ==== 137985+0+0 (secure 0 0 0) 0x7fa624010a90 con 0x7fa63c1040e0 2024-03-21T10:34:19.431 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:19.432+0000 7fa643f91740 1 Processor -- start 2024-03-21T10:34:19.431 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:19.432+0000 7fa633400640 1 --2- 172.21.15.47:0/4282186141 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7fa6200a3110 0x7fa6200a55d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:34:19.431 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:19.432+0000 7fa643f91740 1 -- start start 2024-03-21T10:34:19.431 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:19.433+0000 7fa633400640 1 -- 172.21.15.47:0/4282186141 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 9562+0+0 (secure 0 0 0) 0x7fa6240cffe0 con 0x7fa63c1040e0 2024-03-21T10:34:19.431 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:19.433+0000 7fa63a800640 1 --2- 172.21.15.47:0/4282186141 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7fa6200a3110 0x7fa6200a55d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:34:19.431 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:19.433+0000 7fa643f91740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562e423618b0 0x562e42361c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:34:19.431 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:19.433+0000 7fa643f91740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562e42362260 0x562e4236a810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:34:19.431 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:19.433+0000 7fa643f91740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562e4236ade0 0x562e4236d230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:34:19.431 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:19.433+0000 7fa643f91740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x562e422c52c0 con 0x562e4236ade0 2024-03-21T10:34:19.432 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:19.433+0000 7fa643f91740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x562e4226dc90 con 0x562e423618b0 2024-03-21T10:34:19.432 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:19.433+0000 7fa643f91740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x562e422c1360 con 0x562e42362260 2024-03-21T10:34:19.433 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:19.433+0000 7fa63a800640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562e4236ade0 0x562e4236d230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:34:19.433 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:19.433+0000 7fa63a800640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562e4236ade0 0x562e4236d230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:35530/0 (socket says 172.21.15.47:35530) 2024-03-21T10:34:19.433 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:19.433+0000 7fa63a800640 1 -- 172.21.15.47:0/2236905863 learned_addr learned my addr 172.21.15.47:0/2236905863 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:34:19.433 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:19.434+0000 7fa639e00640 1 --2- 172.21.15.47:0/2236905863 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562e423618b0 0x562e42361c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:34:19.433 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:19.434+0000 7fa631400640 1 --2- 172.21.15.47:0/2236905863 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562e42362260 0x562e4236a810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:34:19.433 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:19.434+0000 7fa63a800640 1 -- 172.21.15.47:0/2236905863 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562e42362260 msgr2=0x562e4236a810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:34:19.434 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:19.434+0000 7fa63a800640 1 --2- 172.21.15.47:0/2236905863 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562e42362260 0x562e4236a810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:19.434 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:19.434+0000 7fa63a800640 1 -- 172.21.15.47:0/2236905863 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562e423618b0 msgr2=0x562e42361c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:34:19.434 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:19.434+0000 7fa63a800640 1 --2- 172.21.15.47:0/2236905863 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562e423618b0 0x562e42361c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:19.434 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:19.434+0000 7fa63a800640 1 -- 172.21.15.47:0/2236905863 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562e42292080 con 0x562e4236ade0 2024-03-21T10:34:19.434 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:19.434+0000 7fa63a800640 1 --2- 172.21.15.47:0/4282186141 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7fa6200a3110 0x7fa6200a55d0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fa63400f770 tx=0x7fa63400fb70 comp rx=0 tx=0).ready entity=mgr.15687 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:34:19.434 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:19.434+0000 7fa63a800640 1 --2- 172.21.15.47:0/2236905863 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562e4236ade0 0x562e4236d230 secure :-1 s=READY pgs=1212 cs=0 l=1 rev1=1 crypto rx=0x7fa63400c740 tx=0x7fa634020000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dd2f04e57f1c4e35 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:34:19.434 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:19.435+0000 7fa62b400640 1 -- 172.21.15.47:0/2236905863 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa63401fc40 con 0x562e4236ade0 2024-03-21T10:34:19.434 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:19.435+0000 7fa62b400640 1 -- 172.21.15.47:0/2236905863 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fa634023070 con 0x562e4236ade0 2024-03-21T10:34:19.434 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:19.435+0000 7fa62b400640 1 -- 172.21.15.47:0/2236905863 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa634026610 con 0x562e4236ade0 2024-03-21T10:34:19.437 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:19.436+0000 7fa643f91740 1 -- 172.21.15.47:0/2236905863 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562e4236ade0 msgr2=0x562e4236d230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:34:19.437 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:19.436+0000 7fa643f91740 1 --2- 172.21.15.47:0/2236905863 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562e4236ade0 0x562e4236d230 secure :-1 s=READY pgs=1212 cs=0 l=1 rev1=1 crypto rx=0x7fa63400c740 tx=0x7fa634020000 comp rx=0 tx=0).stop 2024-03-21T10:34:19.437 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:19.436+0000 7fa643f91740 1 -- 172.21.15.47:0/2236905863 shutdown_connections 2024-03-21T10:34:19.437 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:19.436+0000 7fa643f91740 1 --2- 172.21.15.47:0/2236905863 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562e4236ade0 0x562e4236d230 unknown :-1 s=CLOSED pgs=1212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:19.437 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:19.436+0000 7fa643f91740 1 --2- 172.21.15.47:0/2236905863 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562e42362260 0x562e4236a810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:19.437 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:19.436+0000 7fa643f91740 1 --2- 172.21.15.47:0/2236905863 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562e423618b0 0x562e42361c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:19.437 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:19.436+0000 7fa643f91740 1 -- 172.21.15.47:0/2236905863 >> 172.21.15.47:0/2236905863 conn(0x562e4235bf80 msgr2=0x562e4235c360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:34:19.437 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:19.436+0000 7fa643f91740 1 -- 172.21.15.47:0/2236905863 shutdown_connections 2024-03-21T10:34:19.437 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:19.436+0000 7fa643f91740 1 -- 172.21.15.47:0/2236905863 wait complete. 2024-03-21T10:34:19.438 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:19.436+0000 7fa643f91740 1 -- start start 2024-03-21T10:34:19.438 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:19.437+0000 7fa643f91740 1 Processor -- start 2024-03-21T10:34:19.438 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:19.437+0000 7fa643f91740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562e423618b0 0x562e4236a6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:34:19.438 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:19.437+0000 7fa643f91740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562e42362260 0x562e42367a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:34:19.438 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:19.437+0000 7fa643f91740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562e4236ade0 0x562e42367fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:34:19.438 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:19.437+0000 7fa643f91740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x562e4236dcf0 con 0x562e42362260 2024-03-21T10:34:19.438 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:19.437+0000 7fa643f91740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x562e422810d0 con 0x562e4236ade0 2024-03-21T10:34:19.438 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:19.437+0000 7fa643f91740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x562e4236de70 con 0x562e423618b0 2024-03-21T10:34:19.438 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:19.437+0000 7fa643f91740 10 client.0 _ll_register_callbacks cb 0x562e42285e80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T10:34:19.438 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:19.437+0000 7fa631400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562e42362260 0x562e42367a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:34:19.438 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:19.437+0000 7fa631400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562e42362260 0x562e42367a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:35540/0 (socket says 172.21.15.47:35540) 2024-03-21T10:34:19.438 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:19.437+0000 7fa631400640 1 -- 172.21.15.47:0/3551770244 learned_addr learned my addr 172.21.15.47:0/3551770244 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:34:19.438 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:19.438+0000 7fa63a800640 1 --2- 172.21.15.47:0/3551770244 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562e4236ade0 0x562e42367fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:34:19.439 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:19.438+0000 7fa639e00640 1 --2- 172.21.15.47:0/3551770244 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562e423618b0 0x562e4236a6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:34:19.439 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:19.438+0000 7fa631400640 1 -- 172.21.15.47:0/3551770244 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562e423618b0 msgr2=0x562e4236a6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:34:19.439 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:19.438+0000 7fa631400640 1 --2- 172.21.15.47:0/3551770244 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562e423618b0 0x562e4236a6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:19.439 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:19.438+0000 7fa631400640 1 -- 172.21.15.47:0/3551770244 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562e4236ade0 msgr2=0x562e42367fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:34:19.439 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:19.438+0000 7fa631400640 1 --2- 172.21.15.47:0/3551770244 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562e4236ade0 0x562e42367fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:19.439 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:19.438+0000 7fa631400640 1 -- 172.21.15.47:0/3551770244 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x562e41fd93f0 con 0x562e42362260 2024-03-21T10:34:19.439 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:19.438+0000 7fa631400640 1 --2- 172.21.15.47:0/3551770244 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562e42362260 0x562e42367a40 secure :-1 s=READY pgs=1213 cs=0 l=1 rev1=1 crypto rx=0x7fa62400c6a0 tx=0x7fa624010f20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1a65086d5484d4e1 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:34:19.439 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:19.438+0000 7fa61fe00640 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-03-21T10:34:19.439 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:19.438+0000 7fa61fe00640 1 -- 172.21.15.47:0/3551770244 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6240a49b0 con 0x562e42362260 2024-03-21T10:34:19.439 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:19.438+0000 7fa61fe00640 1 -- 172.21.15.47:0/3551770244 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(103..103 src has 1..103) v4 ==== 9562+0+0 (secure 0 0 0) 0x7fa624010430 con 0x562e42362260 2024-03-21T10:34:19.439 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:19.439+0000 7fa643f91740 1 -- 172.21.15.47:0/3551770244 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fa634023070 con 0x562e42362260 2024-03-21T10:34:19.439 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:19.439+0000 7fa61fe00640 1 -- 172.21.15.47:0/3551770244 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa624010630 con 0x562e42362260 2024-03-21T10:34:19.439 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:19.439+0000 7fa643f91740 10 client.18447 fetch_fsmap learned FSMap version 32 2024-03-21T10:34:19.439 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:19.439+0000 7fa643f91740 1 -- 172.21.15.47:0/3551770244 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x562e420a37d0 con 0x562e42362260 2024-03-21T10:34:19.439 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:19.439+0000 7fa61fe00640 1 -- 172.21.15.47:0/3551770244 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fa634023070 con 0x562e42362260 2024-03-21T10:34:19.442 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:19.439+0000 7fa61fe00640 1 -- 172.21.15.47:0/3551770244 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 32) v1 ==== 7942+0+0 (secure 0 0 0) 0x7fa624018ac0 con 0x562e42362260 2024-03-21T10:34:19.442 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:19.439+0000 7fa643f91740 10 client.18447 fetch_fsmap finished waiting for FSMap version 32 2024-03-21T10:34:19.442 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:19.439+0000 7fa643f91740 10 client.18447 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.a 2024-03-21T10:34:19.442 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:19.439+0000 7fa643f91740 20 client.18447 populate_metadata read hostname 'smithi047' 2024-03-21T10:34:19.442 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:19.439+0000 7fa643f91740 1 --2- 172.21.15.47:0/3551770244 >> [v2:172.21.15.47:6840/3401578978,v1:172.21.15.47:6841/3401578978] conn(0x562e4237ae40 0x562e4237d230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:34:19.443 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:19.440+0000 7fa643f91740 4 client.18447 mds_command: new command op to 14619 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T10:34:19.443 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:19.440+0000 7fa643f91740 1 -- 172.21.15.47:0/3551770244 --> [v2:172.21.15.47:6840/3401578978,v1:172.21.15.47:6841/3401578978] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x562e422d2800 con 0x562e4237ae40 2024-03-21T10:34:19.443 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:19.440+0000 7fa639e00640 1 --2- 172.21.15.47:0/3551770244 >> [v2:172.21.15.47:6840/3401578978,v1:172.21.15.47:6841/3401578978] conn(0x562e4237ae40 0x562e4237d230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:34:19.443 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:19.440+0000 7fa639e00640 1 --2- 172.21.15.47:0/3551770244 >> [v2:172.21.15.47:6840/3401578978,v1:172.21.15.47:6841/3401578978] conn(0x562e4237ae40 0x562e4237d230 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=2ccac694363bd406 server_cookie=1155ecc3faff1017 in_seq=0 out_seq=0 2024-03-21T10:34:19.443 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:19.440+0000 7fa61fe00640 10 client.18447 ms_handle_connect on v2:172.21.15.47:6840/3401578978 2024-03-21T10:34:19.444 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:19.441+0000 7fa61fe00640 1 -- 172.21.15.47:0/3551770244 <== mds.4 v2:172.21.15.47:6840/3401578978 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x562e422d2800 con 0x562e4237ae40 2024-03-21T10:34:19.444 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:19.441+0000 7fa61fe00640 10 client.18447 handle_command_reply: tid=0 2024-03-21T10:34:19.444 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:19.441+0000 7fa643f91740 1 client.18447 shutdown 2024-03-21T10:34:19.444 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:19.442+0000 7fa643f91740 1 -- 172.21.15.47:0/3551770244 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562e42362260 msgr2=0x562e42367a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:34:19.444 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:19.442+0000 7fa643f91740 1 --2- 172.21.15.47:0/3551770244 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562e42362260 0x562e42367a40 secure :-1 s=READY pgs=1213 cs=0 l=1 rev1=1 crypto rx=0x7fa62400c6a0 tx=0x7fa624010f20 comp rx=0 tx=0).stop 2024-03-21T10:34:19.444 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:19.442+0000 7fa643f91740 1 -- 172.21.15.47:0/3551770244 shutdown_connections 2024-03-21T10:34:19.444 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:19.442+0000 7fa643f91740 1 --2- 172.21.15.47:0/3551770244 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562e4236ade0 0x562e42367fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:19.444 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:19.442+0000 7fa643f91740 1 --2- 172.21.15.47:0/3551770244 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562e42362260 0x562e42367a40 unknown :-1 s=CLOSED pgs=1213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:19.444 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:19.442+0000 7fa643f91740 1 --2- 172.21.15.47:0/3551770244 >> [v2:172.21.15.47:6840/3401578978,v1:172.21.15.47:6841/3401578978] conn(0x562e4237ae40 0x562e4237d230 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:19.445 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:19.442+0000 7fa643f91740 1 --2- 172.21.15.47:0/3551770244 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562e423618b0 0x562e4236a6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:19.445 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:19.442+0000 7fa643f91740 1 -- 172.21.15.47:0/3551770244 >> 172.21.15.47:0/3551770244 conn(0x562e4235bf80 msgr2=0x562e4235dcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:34:19.445 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:19.442+0000 7fa61fe00640 0 client.18447 ms_handle_reset on v2:172.21.15.47:6840/3401578978 2024-03-21T10:34:19.445 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:19.442+0000 7fa643f91740 1 -- 172.21.15.47:0/3551770244 shutdown_connections 2024-03-21T10:34:19.445 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:19.442+0000 7fa643f91740 1 -- 172.21.15.47:0/3551770244 wait complete. 2024-03-21T10:34:19.445 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:19.442+0000 7fa643f91740 20 client.18447 trim_cache size 0 max 16384 2024-03-21T10:34:19.462 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:19.456+0000 7fa643f91740 1 Processor -- start 2024-03-21T10:34:19.462 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:19.456+0000 7fa643f91740 1 -- start start 2024-03-21T10:34:19.462 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:19.457+0000 7fa643f91740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562e423618b0 0x562e4235b3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:34:19.462 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:19.457+0000 7fa643f91740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562e42362260 0x562e4236a170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:34:19.463 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:19.457+0000 7fa643f91740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562e4236ade0 0x562e4236a6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:34:19.463 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:19.457+0000 7fa643f91740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa63c1b0a00 con 0x562e42362260 2024-03-21T10:34:19.463 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:19.457+0000 7fa643f91740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa63c1b2380 con 0x562e4236ade0 2024-03-21T10:34:19.463 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:19.457+0000 7fa643f91740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa63c1b1060 con 0x562e423618b0 2024-03-21T10:34:19.463 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:19.457+0000 7fa631400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562e42362260 0x562e4236a170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:34:19.463 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:19.457+0000 7fa631400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562e42362260 0x562e4236a170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:35554/0 (socket says 172.21.15.47:35554) 2024-03-21T10:34:19.463 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:19.457+0000 7fa631400640 1 -- 172.21.15.47:0/1198146618 learned_addr learned my addr 172.21.15.47:0/1198146618 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:34:19.464 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:19.457+0000 7fa63a800640 1 --2- 172.21.15.47:0/1198146618 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562e4236ade0 0x562e4236a6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:34:19.464 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:19.457+0000 7fa639e00640 1 --2- 172.21.15.47:0/1198146618 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562e423618b0 0x562e4235b3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:34:19.464 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:19.457+0000 7fa631400640 1 -- 172.21.15.47:0/1198146618 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562e423618b0 msgr2=0x562e4235b3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:34:19.464 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:19.457+0000 7fa631400640 1 --2- 172.21.15.47:0/1198146618 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562e423618b0 0x562e4235b3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:19.464 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:19.457+0000 7fa631400640 1 -- 172.21.15.47:0/1198146618 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562e4236ade0 msgr2=0x562e4236a6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:34:19.464 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:19.457+0000 7fa631400640 1 --2- 172.21.15.47:0/1198146618 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562e4236ade0 0x562e4236a6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:19.465 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:19.457+0000 7fa631400640 1 -- 172.21.15.47:0/1198146618 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562e42365f90 con 0x562e42362260 2024-03-21T10:34:19.465 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:19.458+0000 7fa631400640 1 --2- 172.21.15.47:0/1198146618 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562e42362260 0x562e4236a170 secure :-1 s=READY pgs=1214 cs=0 l=1 rev1=1 crypto rx=0x7fa624098970 tx=0x7fa62409c120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2877c5a07cc0eeb4 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:34:19.465 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:19.458+0000 7fa62aa00640 1 -- 172.21.15.47:0/1198146618 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6240a49b0 con 0x562e42362260 2024-03-21T10:34:19.465 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:19.458+0000 7fa62aa00640 1 -- 172.21.15.47:0/1198146618 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fa624098610 con 0x562e42362260 2024-03-21T10:34:19.465 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:19.458+0000 7fa62aa00640 1 -- 172.21.15.47:0/1198146618 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa624096b10 con 0x562e42362260 2024-03-21T10:34:19.465 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:19.459+0000 7fa643f91740 1 -- 172.21.15.47:0/1198146618 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562e42362260 msgr2=0x562e4236a170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:34:19.466 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:19.459+0000 7fa643f91740 1 --2- 172.21.15.47:0/1198146618 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562e42362260 0x562e4236a170 secure :-1 s=READY pgs=1214 cs=0 l=1 rev1=1 crypto rx=0x7fa624098970 tx=0x7fa62409c120 comp rx=0 tx=0).stop 2024-03-21T10:34:19.466 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:19.459+0000 7fa643f91740 1 -- 172.21.15.47:0/1198146618 shutdown_connections 2024-03-21T10:34:19.466 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:19.459+0000 7fa643f91740 1 --2- 172.21.15.47:0/1198146618 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562e4236ade0 0x562e4236a6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:19.466 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:19.459+0000 7fa643f91740 1 --2- 172.21.15.47:0/1198146618 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562e42362260 0x562e4236a170 unknown :-1 s=CLOSED pgs=1214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:19.466 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:19.459+0000 7fa643f91740 1 --2- 172.21.15.47:0/1198146618 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562e423618b0 0x562e4235b3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:19.466 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:19.459+0000 7fa643f91740 1 -- 172.21.15.47:0/1198146618 >> 172.21.15.47:0/1198146618 conn(0x562e4235bf80 msgr2=0x562e42362ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:34:19.467 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:19.459+0000 7fa643f91740 1 -- 172.21.15.47:0/1198146618 shutdown_connections 2024-03-21T10:34:19.467 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:19.459+0000 7fa643f91740 1 -- 172.21.15.47:0/1198146618 wait complete. 2024-03-21T10:34:19.467 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:19.459+0000 7fa643f91740 1 -- start start 2024-03-21T10:34:19.467 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:19.460+0000 7fa643f91740 1 Processor -- start 2024-03-21T10:34:19.467 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:19.460+0000 7fa643f91740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562e423618b0 0x562e4236f530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:34:19.467 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:19.460+0000 7fa643f91740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562e42362260 0x562e4236fa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:34:19.468 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:19.460+0000 7fa643f91740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562e4236ade0 0x562e4236ffb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:34:19.468 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:19.460+0000 7fa643f91740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa63c1b1d20 con 0x562e4236ade0 2024-03-21T10:34:19.468 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:19.460+0000 7fa643f91740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa63c1b16c0 con 0x562e423618b0 2024-03-21T10:34:19.468 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:19.460+0000 7fa643f91740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa63c1aebe0 con 0x562e42362260 2024-03-21T10:34:19.468 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:19.461+0000 7fa643f91740 10 client.0 _ll_register_callbacks cb 0x7fa62409bae0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T10:34:19.468 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:19.461+0000 7fa63a800640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562e4236ade0 0x562e4236ffb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:34:19.469 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:19.461+0000 7fa639e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562e423618b0 0x562e4236f530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:34:19.469 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:19.461+0000 7fa639e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562e423618b0 0x562e4236f530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:50212/0 (socket says 172.21.15.47:50212) 2024-03-21T10:34:19.469 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:19.461+0000 7fa639e00640 1 -- 172.21.15.47:0/1663711807 learned_addr learned my addr 172.21.15.47:0/1663711807 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:34:19.469 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:19.461+0000 7fa63a800640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562e4236ade0 0x562e4236ffb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:35560/0 (socket says 172.21.15.47:35560) 2024-03-21T10:34:19.469 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:19.461+0000 7fa631400640 1 --2- 172.21.15.47:0/1663711807 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562e42362260 0x562e4236fa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:34:19.469 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:19.461+0000 7fa639e00640 1 -- 172.21.15.47:0/1663711807 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562e42362260 msgr2=0x562e4236fa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:34:19.470 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:19.461+0000 7fa639e00640 1 --2- 172.21.15.47:0/1663711807 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562e42362260 0x562e4236fa70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:19.470 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:19.461+0000 7fa639e00640 1 -- 172.21.15.47:0/1663711807 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562e4236ade0 msgr2=0x562e4236ffb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:34:19.470 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:19.461+0000 7fa639e00640 1 --2- 172.21.15.47:0/1663711807 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562e4236ade0 0x562e4236ffb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:19.470 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:19.461+0000 7fa639e00640 1 -- 172.21.15.47:0/1663711807 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x562e423d6490 con 0x562e423618b0 2024-03-21T10:34:19.470 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:19.462+0000 7fa639e00640 1 --2- 172.21.15.47:0/1663711807 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562e423618b0 0x562e4236f530 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7fa62c02e010 tx=0x7fa62c0346a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a8d2c7e4293a2e44 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:34:19.470 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:19.462+0000 7fa61fe00640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-21T10:34:19.471 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:19.462+0000 7fa61fe00640 1 -- 172.21.15.47:0/1663711807 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa62c039040 con 0x562e423618b0 2024-03-21T10:34:19.471 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:19.462+0000 7fa61fe00640 1 -- 172.21.15.47:0/1663711807 <== mon.2 v2:172.21.15.100:3300/0 2 ==== osd_map(103..103 src has 1..103) v4 ==== 9562+0+0 (secure 0 0 0) 0x7fa62c004140 con 0x562e423618b0 2024-03-21T10:34:19.471 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:19.462+0000 7fa643f91740 1 -- 172.21.15.47:0/1663711807 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fa624098610 con 0x562e423618b0 2024-03-21T10:34:19.471 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:19.462+0000 7fa61fe00640 1 -- 172.21.15.47:0/1663711807 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa62c052050 con 0x562e423618b0 2024-03-21T10:34:19.471 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:19.462+0000 7fa643f91740 10 client.38219 fetch_fsmap learned FSMap version 32 2024-03-21T10:34:19.471 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:19.462+0000 7fa643f91740 1 -- 172.21.15.47:0/1663711807 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x562e423d76f0 con 0x562e423618b0 2024-03-21T10:34:19.472 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:34:19 smithi047 ceph-mon[31547]: pgmap v279: 129 pgs: 129 active+clean; 2.5 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 229 B/s wr, 11 op/s 2024-03-21T10:34:19.472 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:34:19 smithi047 ceph-mon[31547]: scrub queued for path: / 2024-03-21T10:34:19.472 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:34:19 smithi047 ceph-mon[31547]: scrub summary: idle+waiting paths [/] 2024-03-21T10:34:19.472 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:34:19 smithi047 ceph-mon[31547]: scrub summary: active paths [/] 2024-03-21T10:34:19.473 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:34:19 smithi047 ceph-mon[31547]: scrub summary: idle+waiting paths [/] 2024-03-21T10:34:19.473 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:34:19 smithi047 ceph-mon[31547]: scrub summary: idle 2024-03-21T10:34:19.473 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:34:19 smithi047 ceph-mon[31547]: from='client.? 172.21.15.47:0/959131576' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-21T10:34:19.476 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:19.463+0000 7fa61fe00640 1 -- 172.21.15.47:0/1663711807 <== mon.2 v2:172.21.15.100:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fa624098610 con 0x562e423618b0 2024-03-21T10:34:19.476 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:19.463+0000 7fa61fe00640 1 -- 172.21.15.47:0/1663711807 <== mon.2 v2:172.21.15.100:3300/0 5 ==== fsmap(e 32) v1 ==== 7942+0+0 (secure 0 0 0) 0x7fa62c005490 con 0x562e423618b0 2024-03-21T10:34:19.476 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:19.463+0000 7fa643f91740 10 client.38219 fetch_fsmap finished waiting for FSMap version 32 2024-03-21T10:34:19.477 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:19.463+0000 7fa643f91740 10 client.38219 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.a 2024-03-21T10:34:19.477 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:19.463+0000 7fa643f91740 20 client.38219 populate_metadata read hostname 'smithi047' 2024-03-21T10:34:19.477 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:19.463+0000 7fa643f91740 1 --2- 172.21.15.47:0/1663711807 >> [v2:172.21.15.47:6840/3401578978,v1:172.21.15.47:6841/3401578978] conn(0x562e423d7df0 0x562e423da210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:34:19.477 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:19.463+0000 7fa643f91740 4 client.38219 mds_command: new command op to 14619 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-21T10:34:19.477 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:19.463+0000 7fa643f91740 1 -- 172.21.15.47:0/1663711807 --> [v2:172.21.15.47:6840/3401578978,v1:172.21.15.47:6841/3401578978] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x562e422ccb50 con 0x562e423d7df0 2024-03-21T10:34:19.477 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:19.464+0000 7fa631400640 1 --2- 172.21.15.47:0/1663711807 >> [v2:172.21.15.47:6840/3401578978,v1:172.21.15.47:6841/3401578978] conn(0x562e423d7df0 0x562e423da210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:34:19.478 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:19.464+0000 7fa631400640 1 --2- 172.21.15.47:0/1663711807 >> [v2:172.21.15.47:6840/3401578978,v1:172.21.15.47:6841/3401578978] conn(0x562e423d7df0 0x562e423da210 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=7b556dfd5dfc4a88 server_cookie=2a7a8a63fc3ddbc7 in_seq=0 out_seq=0 2024-03-21T10:34:19.478 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:19.464+0000 7fa61fe00640 10 client.38219 ms_handle_connect on v2:172.21.15.47:6840/3401578978 2024-03-21T10:34:19.478 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:19.464+0000 7fa61fe00640 1 -- 172.21.15.47:0/1663711807 <== mds.4 v2:172.21.15.47:6840/3401578978 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x562e422ccb50 con 0x562e423d7df0 2024-03-21T10:34:19.478 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:19.464+0000 7fa61fe00640 10 client.38219 handle_command_reply: tid=0 2024-03-21T10:34:19.478 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:19.464+0000 7fa643f91740 1 client.38219 shutdown 2024-03-21T10:34:19.478 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:19.464+0000 7fa643f91740 1 -- 172.21.15.47:0/1663711807 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562e423618b0 msgr2=0x562e4236f530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:34:19.479 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:19.465+0000 7fa643f91740 1 --2- 172.21.15.47:0/1663711807 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562e423618b0 0x562e4236f530 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7fa62c02e010 tx=0x7fa62c0346a0 comp rx=0 tx=0).stop 2024-03-21T10:34:19.479 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:19.465+0000 7fa643f91740 1 -- 172.21.15.47:0/1663711807 shutdown_connections 2024-03-21T10:34:19.479 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:19.465+0000 7fa643f91740 1 --2- 172.21.15.47:0/1663711807 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562e4236ade0 0x562e4236ffb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:19.479 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:19.465+0000 7fa643f91740 1 --2- 172.21.15.47:0/1663711807 >> [v2:172.21.15.47:6840/3401578978,v1:172.21.15.47:6841/3401578978] conn(0x562e423d7df0 0x562e423da210 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:19.479 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:19.465+0000 7fa643f91740 1 --2- 172.21.15.47:0/1663711807 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562e42362260 0x562e4236fa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:19.479 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:19.465+0000 7fa643f91740 1 --2- 172.21.15.47:0/1663711807 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562e423618b0 0x562e4236f530 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:19.480 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:19.465+0000 7fa61fe00640 0 client.38219 ms_handle_reset on v2:172.21.15.47:6840/3401578978 2024-03-21T10:34:19.480 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:19.465+0000 7fa643f91740 1 -- 172.21.15.47:0/1663711807 >> 172.21.15.47:0/1663711807 conn(0x562e4235bf80 msgr2=0x562e4235d120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:34:19.480 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:19.465+0000 7fa643f91740 1 -- 172.21.15.47:0/1663711807 shutdown_connections 2024-03-21T10:34:19.480 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:19.465+0000 7fa643f91740 1 -- 172.21.15.47:0/1663711807 wait complete. 2024-03-21T10:34:19.480 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:19.465+0000 7fa643f91740 20 client.38219 trim_cache size 0 max 16384 2024-03-21T10:34:19.481 INFO:teuthology.orchestra.run.smithi047.stdout:[] 2024-03-21T10:34:19.483 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:19.466+0000 7fa63b200640 1 -- 172.21.15.47:0/4282186141 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7fa6200a3110 msgr2=0x7fa6200a55d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:34:19.483 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:19.466+0000 7fa63b200640 1 --2- 172.21.15.47:0/4282186141 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7fa6200a3110 0x7fa6200a55d0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fa63400f770 tx=0x7fa63400fb70 comp rx=0 tx=0).stop 2024-03-21T10:34:19.483 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:19.466+0000 7fa63b200640 1 -- 172.21.15.47:0/4282186141 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa63c1040e0 msgr2=0x7fa63c19a450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:34:19.483 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:19.466+0000 7fa63b200640 1 --2- 172.21.15.47:0/4282186141 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa63c1040e0 0x7fa63c19a450 secure :-1 s=READY pgs=1211 cs=0 l=1 rev1=1 crypto rx=0x7fa62400e9f0 tx=0x7fa62400eeb0 comp rx=0 tx=0).stop 2024-03-21T10:34:19.483 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:19.466+0000 7fa63b200640 1 -- 172.21.15.47:0/4282186141 shutdown_connections 2024-03-21T10:34:19.483 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:19.466+0000 7fa63b200640 1 --2- 172.21.15.47:0/4282186141 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa63c109730 0x7fa63c19e7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:19.484 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:19.466+0000 7fa63b200640 1 --2- 172.21.15.47:0/4282186141 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa63c1040e0 0x7fa63c19a450 unknown :-1 s=CLOSED pgs=1211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:19.484 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:19.466+0000 7fa63b200640 1 --2- 172.21.15.47:0/4282186141 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7fa6200a3110 0x7fa6200a55d0 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:19.484 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:19.466+0000 7fa63b200640 1 --2- 172.21.15.47:0/4282186141 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa63c103720 0x7fa63c199f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:19.484 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:19.466+0000 7fa63b200640 1 -- 172.21.15.47:0/4282186141 >> 172.21.15.47:0/4282186141 conn(0x7fa63c0ff1b0 msgr2=0x7fa63c0ffa90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:34:19.484 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:19.466+0000 7fa63b200640 1 -- 172.21.15.47:0/4282186141 shutdown_connections 2024-03-21T10:34:19.484 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:19.466+0000 7fa63b200640 1 -- 172.21.15.47:0/4282186141 wait complete. 2024-03-21T10:34:19.484 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-21T10:34:19.485 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-03-21T10:34:19.702 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:19.704+0000 7f92ae600640 1 -- 172.21.15.47:0/1454013533 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f92a81058c0 msgr2=0x7f92a8105d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:34:19.702 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:19.704+0000 7f92ae600640 1 --2- 172.21.15.47:0/1454013533 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f92a81058c0 0x7f92a8105d50 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f92a0009ca0 tx=0x7f92a002c190 comp rx=0 tx=0).stop 2024-03-21T10:34:19.702 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:19.704+0000 7f92ae600640 1 -- 172.21.15.47:0/1454013533 shutdown_connections 2024-03-21T10:34:19.703 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:19.704+0000 7f92ae600640 1 --2- 172.21.15.47:0/1454013533 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f92a81058c0 0x7f92a8105d50 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:19.703 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:19.704+0000 7f92ae600640 1 --2- 172.21.15.47:0/1454013533 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f92a8100de0 0x7f92a81052a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:19.703 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:19.704+0000 7f92ae600640 1 --2- 172.21.15.47:0/1454013533 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f92a81004c0 0x7f92a81008a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:19.703 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:19.704+0000 7f92ae600640 1 -- 172.21.15.47:0/1454013533 >> 172.21.15.47:0/1454013533 conn(0x7f92a80fbef0 msgr2=0x7f92a80fe310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:34:19.703 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:19.704+0000 7f92ae600640 1 -- 172.21.15.47:0/1454013533 shutdown_connections 2024-03-21T10:34:19.703 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:19.704+0000 7f92ae600640 1 -- 172.21.15.47:0/1454013533 wait complete. 2024-03-21T10:34:19.704 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:19.705+0000 7f92ae600640 1 Processor -- start 2024-03-21T10:34:19.704 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:19.705+0000 7f92ae600640 1 -- start start 2024-03-21T10:34:19.705 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:19.705+0000 7f92ae600640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f92a81004c0 0x7f92a819e290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:34:19.705 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:19.705+0000 7f92ae600640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f92a8100de0 0x7f92a819e7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:34:19.705 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:19.705+0000 7f92ae600640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f92a81058c0 0x7f92a81a2b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:34:19.705 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:19.705+0000 7f92ae600640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92a8117d50 con 0x7f92a8100de0 2024-03-21T10:34:19.705 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:19.705+0000 7f92ae600640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92a8117bd0 con 0x7f92a81004c0 2024-03-21T10:34:19.705 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:19.705+0000 7f92ae600640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92a8117ed0 con 0x7f92a81058c0 2024-03-21T10:34:19.705 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:19.706+0000 7f92a7e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f92a8100de0 0x7f92a819e7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:34:19.705 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:19.706+0000 7f92ad200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f92a81004c0 0x7f92a819e290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:34:19.705 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:19.706+0000 7f92a7e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f92a8100de0 0x7f92a819e7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:35576/0 (socket says 172.21.15.47:35576) 2024-03-21T10:34:19.705 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:19.706+0000 7f92a7e00640 1 -- 172.21.15.47:0/1121738860 learned_addr learned my addr 172.21.15.47:0/1121738860 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:34:19.705 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:19.706+0000 7f92adc00640 1 --2- 172.21.15.47:0/1121738860 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f92a81058c0 0x7f92a81a2b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:34:19.705 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:19.706+0000 7f92a7e00640 1 -- 172.21.15.47:0/1121738860 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f92a81058c0 msgr2=0x7f92a81a2b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:34:19.705 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:19.706+0000 7f92a7e00640 1 --2- 172.21.15.47:0/1121738860 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f92a81058c0 0x7f92a81a2b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:19.706 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:19.706+0000 7f92a7e00640 1 -- 172.21.15.47:0/1121738860 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f92a81004c0 msgr2=0x7f92a819e290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:34:19.706 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:19.706+0000 7f92a7e00640 1 --2- 172.21.15.47:0/1121738860 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f92a81004c0 0x7f92a819e290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:19.706 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:19.706+0000 7f92a7e00640 1 -- 172.21.15.47:0/1121738860 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f92a81a3280 con 0x7f92a8100de0 2024-03-21T10:34:19.706 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:19.707+0000 7f92a7e00640 1 --2- 172.21.15.47:0/1121738860 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f92a8100de0 0x7f92a819e7d0 secure :-1 s=READY pgs=1215 cs=0 l=1 rev1=1 crypto rx=0x7f929400e9f0 tx=0x7f929400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:34:19.706 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:19.707+0000 7f92a5600640 1 -- 172.21.15.47:0/1121738860 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f929400ccc0 con 0x7f92a8100de0 2024-03-21T10:34:19.706 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:19.707+0000 7f92ae600640 1 -- 172.21.15.47:0/1121738860 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f92a81a3570 con 0x7f92a8100de0 2024-03-21T10:34:19.707 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:19.707+0000 7f92a5600640 1 -- 172.21.15.47:0/1121738860 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f92940108e0 con 0x7f92a8100de0 2024-03-21T10:34:19.707 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:19.707+0000 7f92a5600640 1 -- 172.21.15.47:0/1121738860 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92940187c0 con 0x7f92a8100de0 2024-03-21T10:34:19.707 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:19.707+0000 7f92ae600640 1 -- 172.21.15.47:0/1121738860 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f92a81aae00 con 0x7f92a8100de0 2024-03-21T10:34:19.709 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:19.711+0000 7f92b29d4740 1 Processor -- start 2024-03-21T10:34:19.709 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:19.711+0000 7f92b29d4740 1 -- start start 2024-03-21T10:34:19.709 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:19.711+0000 7f92a5600640 1 -- 172.21.15.47:0/1121738860 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 27) v1 ==== 137985+0+0 (secure 0 0 0) 0x7f9294010430 con 0x7f92a8100de0 2024-03-21T10:34:19.710 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:19.711+0000 7f92b29d4740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5635246298b0 0x563524629c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:34:19.710 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:19.711+0000 7f92b29d4740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56352462a260 0x563524632810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:34:19.710 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:19.711+0000 7f92b29d4740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x563524632de0 0x563524635230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:34:19.711 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:19.711+0000 7f92b29d4740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x56352458d2c0 con 0x563524632de0 2024-03-21T10:34:19.711 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:19.711+0000 7f92b29d4740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x563524535c90 con 0x56352462a260 2024-03-21T10:34:19.711 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:19.711+0000 7f92b29d4740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x563524589360 con 0x5635246298b0 2024-03-21T10:34:19.711 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:19.712+0000 7f92a7e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x563524632de0 0x563524635230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:34:19.711 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:19.712+0000 7f92ad200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56352462a260 0x563524632810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:34:19.711 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:19.712+0000 7f92ad200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56352462a260 0x563524632810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:50232/0 (socket says 172.21.15.47:50232) 2024-03-21T10:34:19.711 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:19.712+0000 7f92ad200640 1 -- 172.21.15.47:0/4036374858 learned_addr learned my addr 172.21.15.47:0/4036374858 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:34:19.711 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:19.712+0000 7f92adc00640 1 --2- 172.21.15.47:0/4036374858 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5635246298b0 0x563524629c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:34:19.711 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:19.712+0000 7f92ad200640 1 -- 172.21.15.47:0/4036374858 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5635246298b0 msgr2=0x563524629c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:34:19.711 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:19.712+0000 7f92ad200640 1 --2- 172.21.15.47:0/4036374858 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5635246298b0 0x563524629c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:19.711 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:19.712+0000 7f92ad200640 1 -- 172.21.15.47:0/4036374858 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x563524632de0 msgr2=0x563524635230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:34:19.711 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:19.712+0000 7f92ad200640 1 --2- 172.21.15.47:0/4036374858 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x563524632de0 0x563524635230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:19.711 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:19.712+0000 7f92ad200640 1 -- 172.21.15.47:0/4036374858 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56352455a080 con 0x56352462a260 2024-03-21T10:34:19.711 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:19.712+0000 7f92a7e00640 1 --2- 172.21.15.47:0/4036374858 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x563524632de0 0x563524635230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T10:34:19.712 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:19.713+0000 7f92a5600640 1 --2- 172.21.15.47:0/1121738860 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f92900a3110 0x7f92900a55d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:34:19.712 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:19.713+0000 7f92ad200640 1 --2- 172.21.15.47:0/4036374858 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56352462a260 0x563524632810 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f929c00bd10 tx=0x7f929c0107a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c3bf6d6a8a567990 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:34:19.714 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:19.713+0000 7f92a5600640 1 -- 172.21.15.47:0/1121738860 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 9562+0+0 (secure 0 0 0) 0x7f92940cff30 con 0x7f92a8100de0 2024-03-21T10:34:19.715 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:19.713+0000 7f929a000640 1 -- 172.21.15.47:0/4036374858 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f929c004030 con 0x56352462a260 2024-03-21T10:34:19.715 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:19.713+0000 7f92adc00640 1 --2- 172.21.15.47:0/1121738860 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f92900a3110 0x7f92900a55d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:34:19.715 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:19.713+0000 7f929a000640 1 -- 172.21.15.47:0/4036374858 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f929c0035f0 con 0x56352462a260 2024-03-21T10:34:19.715 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:19.713+0000 7f929a000640 1 -- 172.21.15.47:0/4036374858 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f929c0117b0 con 0x56352462a260 2024-03-21T10:34:19.715 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:19.714+0000 7f92b29d4740 1 -- 172.21.15.47:0/4036374858 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56352462a260 msgr2=0x563524632810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:34:19.715 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:19.714+0000 7f92b29d4740 1 --2- 172.21.15.47:0/4036374858 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56352462a260 0x563524632810 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f929c00bd10 tx=0x7f929c0107a0 comp rx=0 tx=0).stop 2024-03-21T10:34:19.715 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:19.714+0000 7f92adc00640 1 --2- 172.21.15.47:0/1121738860 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f92900a3110 0x7f92900a55d0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f92a0006010 tx=0x7f92a0009800 comp rx=0 tx=0).ready entity=mgr.15687 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:34:19.715 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:19.714+0000 7f92b29d4740 1 -- 172.21.15.47:0/4036374858 shutdown_connections 2024-03-21T10:34:19.715 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:19.714+0000 7f92b29d4740 1 --2- 172.21.15.47:0/4036374858 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x563524632de0 0x563524635230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:19.715 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:19.714+0000 7f92b29d4740 1 --2- 172.21.15.47:0/4036374858 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56352462a260 0x563524632810 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:19.715 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:19.714+0000 7f92b29d4740 1 --2- 172.21.15.47:0/4036374858 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5635246298b0 0x563524629c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:19.715 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:19.714+0000 7f92b29d4740 1 -- 172.21.15.47:0/4036374858 >> 172.21.15.47:0/4036374858 conn(0x563524623f80 msgr2=0x563524624360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:34:19.715 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:19.714+0000 7f92b29d4740 1 -- 172.21.15.47:0/4036374858 shutdown_connections 2024-03-21T10:34:19.716 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:19.714+0000 7f92b29d4740 1 -- 172.21.15.47:0/4036374858 wait complete. 2024-03-21T10:34:19.716 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:19.715+0000 7f92b29d4740 1 -- start start 2024-03-21T10:34:19.716 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:19.715+0000 7f92b29d4740 1 Processor -- start 2024-03-21T10:34:19.716 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:19.715+0000 7f92b29d4740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5635246298b0 0x5635246326b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:34:19.716 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:19.715+0000 7f92b29d4740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56352462a260 0x56352462fa40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:34:19.716 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:19.715+0000 7f92b29d4740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x563524632de0 0x56352462ffa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:34:19.716 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:19.715+0000 7f92b29d4740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x563524635d50 con 0x563524632de0 2024-03-21T10:34:19.716 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:19.715+0000 7f92b29d4740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x5635245490d0 con 0x5635246298b0 2024-03-21T10:34:19.716 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:19.715+0000 7f92b29d4740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x563524635ed0 con 0x56352462a260 2024-03-21T10:34:19.716 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:19.716+0000 7f92b29d4740 10 client.0 _ll_register_callbacks cb 0x56352454de80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T10:34:19.716 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:19.716+0000 7f92adc00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x563524632de0 0x56352462ffa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:34:19.716 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:19.716+0000 7f92ad200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5635246298b0 0x5635246326b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:34:19.716 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:19.716+0000 7f92adc00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x563524632de0 0x56352462ffa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:35594/0 (socket says 172.21.15.47:35594) 2024-03-21T10:34:19.716 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:19.716+0000 7f92adc00640 1 -- 172.21.15.47:0/1696048737 learned_addr learned my addr 172.21.15.47:0/1696048737 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:34:19.717 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:19.716+0000 7f92a7e00640 1 --2- 172.21.15.47:0/1696048737 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56352462a260 0x56352462fa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:34:19.717 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:19.716+0000 7f92adc00640 1 -- 172.21.15.47:0/1696048737 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56352462a260 msgr2=0x56352462fa40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:34:19.717 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:19.716+0000 7f92adc00640 1 --2- 172.21.15.47:0/1696048737 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56352462a260 0x56352462fa40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:19.717 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:19.716+0000 7f92adc00640 1 -- 172.21.15.47:0/1696048737 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5635246298b0 msgr2=0x5635246326b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:34:19.717 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:19.716+0000 7f92adc00640 1 --2- 172.21.15.47:0/1696048737 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5635246298b0 0x5635246326b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:19.717 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:19.716+0000 7f92adc00640 1 -- 172.21.15.47:0/1696048737 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5635242a13f0 con 0x563524632de0 2024-03-21T10:34:19.717 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:19.717+0000 7f92adc00640 1 --2- 172.21.15.47:0/1696048737 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x563524632de0 0x56352462ffa0 secure :-1 s=READY pgs=1216 cs=0 l=1 rev1=1 crypto rx=0x7f92a0036000 tx=0x7f92a002c190 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b6b0361e7aa408e6 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:34:19.717 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:19.717+0000 7f928ea00640 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-03-21T10:34:19.719 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:19.717+0000 7f928ea00640 1 -- 172.21.15.47:0/1696048737 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92a004f070 con 0x563524632de0 2024-03-21T10:34:19.719 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:19.717+0000 7f928ea00640 1 -- 172.21.15.47:0/1696048737 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(103..103 src has 1..103) v4 ==== 9562+0+0 (secure 0 0 0) 0x7f92a00047d0 con 0x563524632de0 2024-03-21T10:34:19.719 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:19.717+0000 7f92b29d4740 1 -- 172.21.15.47:0/1696048737 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f929c0035f0 con 0x563524632de0 2024-03-21T10:34:19.719 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:19.717+0000 7f928ea00640 1 -- 172.21.15.47:0/1696048737 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f92a005b050 con 0x563524632de0 2024-03-21T10:34:19.719 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:19.717+0000 7f92b29d4740 10 client.18465 fetch_fsmap learned FSMap version 32 2024-03-21T10:34:19.719 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:19.717+0000 7f92b29d4740 1 -- 172.21.15.47:0/1696048737 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x56352436b7d0 con 0x563524632de0 2024-03-21T10:34:19.719 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:19.718+0000 7f928ea00640 1 -- 172.21.15.47:0/1696048737 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f929c0035f0 con 0x563524632de0 2024-03-21T10:34:19.719 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:19.718+0000 7f928ea00640 1 -- 172.21.15.47:0/1696048737 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 32) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f92a004a070 con 0x563524632de0 2024-03-21T10:34:19.719 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:19.718+0000 7f92b29d4740 10 client.18465 fetch_fsmap finished waiting for FSMap version 32 2024-03-21T10:34:19.719 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:19.718+0000 7f92b29d4740 10 client.18465 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.k 2024-03-21T10:34:19.719 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:19.718+0000 7f92b29d4740 20 client.18465 populate_metadata read hostname 'smithi047' 2024-03-21T10:34:19.719 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:19.718+0000 7f92b29d4740 1 --2- 172.21.15.47:0/1696048737 >> [v2:172.21.15.100:6832/2723302038,v1:172.21.15.100:6833/2723302038] conn(0x563524642bf0 0x563524645020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:34:19.719 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:19.718+0000 7f92b29d4740 4 client.18465 mds_command: new command op to 34427 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T10:34:19.719 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:19.718+0000 7f92b29d4740 1 -- 172.21.15.47:0/1696048737 --> [v2:172.21.15.100:6832/2723302038,v1:172.21.15.100:6833/2723302038] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56352459a800 con 0x563524642bf0 2024-03-21T10:34:19.720 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:19.719+0000 7f92ad200640 1 --2- 172.21.15.47:0/1696048737 >> [v2:172.21.15.100:6832/2723302038,v1:172.21.15.100:6833/2723302038] conn(0x563524642bf0 0x563524645020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:34:19.720 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:19.720+0000 7f92ad200640 1 --2- 172.21.15.47:0/1696048737 >> [v2:172.21.15.100:6832/2723302038,v1:172.21.15.100:6833/2723302038] conn(0x563524642bf0 0x563524645020 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=9167e1663060ba2d server_cookie=b4be098dfdd533a2 in_seq=0 out_seq=0 2024-03-21T10:34:19.720 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:19.720+0000 7f928ea00640 10 client.18465 ms_handle_connect on v2:172.21.15.100:6832/2723302038 2024-03-21T10:34:19.721 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:19.721+0000 7f928ea00640 1 -- 172.21.15.47:0/1696048737 <== mds.3 v2:172.21.15.100:6832/2723302038 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x56352459a800 con 0x563524642bf0 2024-03-21T10:34:19.721 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:19.721+0000 7f928ea00640 10 client.18465 handle_command_reply: tid=0 2024-03-21T10:34:19.721 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:19.721+0000 7f92b29d4740 1 client.18465 shutdown 2024-03-21T10:34:19.721 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:19.721+0000 7f92b29d4740 1 -- 172.21.15.47:0/1696048737 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x563524632de0 msgr2=0x56352462ffa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:34:19.721 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:19.721+0000 7f92b29d4740 1 --2- 172.21.15.47:0/1696048737 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x563524632de0 0x56352462ffa0 secure :-1 s=READY pgs=1216 cs=0 l=1 rev1=1 crypto rx=0x7f92a0036000 tx=0x7f92a002c190 comp rx=0 tx=0).stop 2024-03-21T10:34:19.722 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:19.721+0000 7f92b29d4740 1 -- 172.21.15.47:0/1696048737 shutdown_connections 2024-03-21T10:34:19.722 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:19.721+0000 7f92b29d4740 1 --2- 172.21.15.47:0/1696048737 >> [v2:172.21.15.100:6832/2723302038,v1:172.21.15.100:6833/2723302038] conn(0x563524642bf0 0x563524645020 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:19.722 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:19.721+0000 7f92b29d4740 1 --2- 172.21.15.47:0/1696048737 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x563524632de0 0x56352462ffa0 unknown :-1 s=CLOSED pgs=1216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:19.722 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:19.721+0000 7f92b29d4740 1 --2- 172.21.15.47:0/1696048737 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56352462a260 0x56352462fa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:19.722 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:19.721+0000 7f92b29d4740 1 --2- 172.21.15.47:0/1696048737 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5635246298b0 0x5635246326b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:19.722 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:19.721+0000 7f928ea00640 0 client.18465 ms_handle_reset on v2:172.21.15.100:6832/2723302038 2024-03-21T10:34:19.722 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:19.721+0000 7f92b29d4740 1 -- 172.21.15.47:0/1696048737 >> 172.21.15.47:0/1696048737 conn(0x563524623f80 msgr2=0x563524625cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:34:19.722 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:19.722+0000 7f92b29d4740 1 -- 172.21.15.47:0/1696048737 shutdown_connections 2024-03-21T10:34:19.722 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:19.722+0000 7f92b29d4740 1 -- 172.21.15.47:0/1696048737 wait complete. 2024-03-21T10:34:19.722 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:19.722+0000 7f92b29d4740 20 client.18465 trim_cache size 0 max 16384 2024-03-21T10:34:19.726 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:34:19 smithi100 ceph-mon[35193]: pgmap v279: 129 pgs: 129 active+clean; 2.5 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 229 B/s wr, 11 op/s 2024-03-21T10:34:19.727 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:34:19 smithi100 ceph-mon[35193]: scrub queued for path: / 2024-03-21T10:34:19.727 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:34:19 smithi100 ceph-mon[35193]: scrub summary: idle+waiting paths [/] 2024-03-21T10:34:19.727 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:34:19 smithi100 ceph-mon[35193]: scrub summary: active paths [/] 2024-03-21T10:34:19.727 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:34:19 smithi100 ceph-mon[35193]: scrub summary: idle+waiting paths [/] 2024-03-21T10:34:19.727 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:34:19 smithi100 ceph-mon[35193]: scrub summary: idle 2024-03-21T10:34:19.727 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:34:19 smithi100 ceph-mon[35193]: from='client.? 172.21.15.47:0/959131576' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-21T10:34:19.734 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:19.736+0000 7f92b29d4740 1 Processor -- start 2024-03-21T10:34:19.734 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:19.736+0000 7f92b29d4740 1 -- start start 2024-03-21T10:34:19.734 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:19.736+0000 7f92b29d4740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5635246298b0 0x563524622fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:34:19.734 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:19.736+0000 7f92b29d4740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56352462a260 0x563524632170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:34:19.734 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:19.736+0000 7f92b29d4740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x563524632de0 0x5635246326b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:34:19.735 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:19.736+0000 7f92b29d4740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92a81b4d80 con 0x563524632de0 2024-03-21T10:34:19.735 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:19.736+0000 7f92b29d4740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92a81b6700 con 0x56352462a260 2024-03-21T10:34:19.735 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:19.736+0000 7f92b29d4740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92a81b53e0 con 0x5635246298b0 2024-03-21T10:34:19.737 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:19.737+0000 7f92adc00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x563524632de0 0x5635246326b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:34:19.737 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:19.737+0000 7f92a7e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56352462a260 0x563524632170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:34:19.737 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:19.737+0000 7f92a7e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56352462a260 0x563524632170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:50254/0 (socket says 172.21.15.47:50254) 2024-03-21T10:34:19.737 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:19.737+0000 7f92a7e00640 1 -- 172.21.15.47:0/2341029739 learned_addr learned my addr 172.21.15.47:0/2341029739 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:34:19.737 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:19.737+0000 7f92ad200640 1 --2- 172.21.15.47:0/2341029739 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5635246298b0 0x563524622fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:34:19.737 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:19.737+0000 7f92adc00640 1 -- 172.21.15.47:0/2341029739 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5635246298b0 msgr2=0x563524622fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:34:19.737 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:19.737+0000 7f92adc00640 1 --2- 172.21.15.47:0/2341029739 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5635246298b0 0x563524622fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:19.737 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:19.737+0000 7f92adc00640 1 -- 172.21.15.47:0/2341029739 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56352462a260 msgr2=0x563524632170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:34:19.737 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:19.737+0000 7f92adc00640 1 --2- 172.21.15.47:0/2341029739 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56352462a260 0x563524632170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:19.738 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:19.737+0000 7f92adc00640 1 -- 172.21.15.47:0/2341029739 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56352462dfb0 con 0x563524632de0 2024-03-21T10:34:19.738 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:19.737+0000 7f92a7e00640 1 --2- 172.21.15.47:0/2341029739 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56352462a260 0x563524632170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T10:34:19.738 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:19.738+0000 7f92adc00640 1 --2- 172.21.15.47:0/2341029739 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x563524632de0 0x5635246326b0 secure :-1 s=READY pgs=1217 cs=0 l=1 rev1=1 crypto rx=0x7f92a0009ca0 tx=0x7f92a00099c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ba93004a27a7019b server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:34:19.738 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:19.738+0000 7f9299600640 1 -- 172.21.15.47:0/2341029739 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92a004f050 con 0x563524632de0 2024-03-21T10:34:19.738 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:19.738+0000 7f9299600640 1 -- 172.21.15.47:0/2341029739 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f92a0064070 con 0x563524632de0 2024-03-21T10:34:19.738 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:19.738+0000 7f9299600640 1 -- 172.21.15.47:0/2341029739 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92a0039040 con 0x563524632de0 2024-03-21T10:34:19.738 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:19.739+0000 7f92b29d4740 1 -- 172.21.15.47:0/2341029739 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x563524632de0 msgr2=0x5635246326b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:34:19.738 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:19.739+0000 7f92b29d4740 1 --2- 172.21.15.47:0/2341029739 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x563524632de0 0x5635246326b0 secure :-1 s=READY pgs=1217 cs=0 l=1 rev1=1 crypto rx=0x7f92a0009ca0 tx=0x7f92a00099c0 comp rx=0 tx=0).stop 2024-03-21T10:34:19.738 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:19.739+0000 7f92b29d4740 1 -- 172.21.15.47:0/2341029739 shutdown_connections 2024-03-21T10:34:19.738 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:19.739+0000 7f92b29d4740 1 --2- 172.21.15.47:0/2341029739 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x563524632de0 0x5635246326b0 unknown :-1 s=CLOSED pgs=1217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:19.738 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:19.739+0000 7f92b29d4740 1 --2- 172.21.15.47:0/2341029739 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56352462a260 0x563524632170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:19.738 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:19.739+0000 7f92b29d4740 1 --2- 172.21.15.47:0/2341029739 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5635246298b0 0x563524622fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:19.738 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:19.739+0000 7f92b29d4740 1 -- 172.21.15.47:0/2341029739 >> 172.21.15.47:0/2341029739 conn(0x563524623f80 msgr2=0x5635246228d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:34:19.739 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:19.739+0000 7f92b29d4740 1 -- 172.21.15.47:0/2341029739 shutdown_connections 2024-03-21T10:34:19.739 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:19.739+0000 7f92b29d4740 1 -- 172.21.15.47:0/2341029739 wait complete. 2024-03-21T10:34:19.739 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:19.739+0000 7f92b29d4740 1 -- start start 2024-03-21T10:34:19.739 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:34:19 smithi157 ceph-mon[35581]: pgmap v279: 129 pgs: 129 active+clean; 2.5 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 229 B/s wr, 11 op/s 2024-03-21T10:34:19.739 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:34:19 smithi157 ceph-mon[35581]: scrub queued for path: / 2024-03-21T10:34:19.740 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:34:19 smithi157 ceph-mon[35581]: scrub summary: idle+waiting paths [/] 2024-03-21T10:34:19.740 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:34:19 smithi157 ceph-mon[35581]: scrub summary: active paths [/] 2024-03-21T10:34:19.740 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:34:19 smithi157 ceph-mon[35581]: scrub summary: idle+waiting paths [/] 2024-03-21T10:34:19.740 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:34:19 smithi157 ceph-mon[35581]: scrub summary: idle 2024-03-21T10:34:19.740 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:34:19 smithi157 ceph-mon[35581]: from='client.? 172.21.15.47:0/959131576' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-21T10:34:19.744 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:19.739+0000 7f92b29d4740 1 Processor -- start 2024-03-21T10:34:19.745 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:19.740+0000 7f92b29d4740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5635246298b0 0x563524636d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:34:19.745 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:19.740+0000 7f92b29d4740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56352462a260 0x563524637240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:34:19.745 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:19.740+0000 7f92b29d4740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563524632de0 0x56352469d4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:34:19.745 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:19.740+0000 7f92b29d4740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92a81b60a0 con 0x56352462a260 2024-03-21T10:34:19.745 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:19.740+0000 7f92b29d4740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92a81b5a40 con 0x563524632de0 2024-03-21T10:34:19.745 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:19.740+0000 7f92b29d4740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92a81b2f60 con 0x5635246298b0 2024-03-21T10:34:19.745 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:19.740+0000 7f92b29d4740 10 client.0 _ll_register_callbacks cb 0x7f92a0037070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T10:34:19.745 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:19.740+0000 7f92a7e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56352462a260 0x563524637240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:34:19.745 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:19.740+0000 7f92a7e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56352462a260 0x563524637240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:35614/0 (socket says 172.21.15.47:35614) 2024-03-21T10:34:19.745 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:19.740+0000 7f92a7e00640 1 -- 172.21.15.47:0/4066455708 learned_addr learned my addr 172.21.15.47:0/4066455708 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:34:19.745 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:19.740+0000 7f92adc00640 1 --2- 172.21.15.47:0/4066455708 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563524632de0 0x56352469d4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:34:19.745 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:19.740+0000 7f92ad200640 1 --2- 172.21.15.47:0/4066455708 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5635246298b0 0x563524636d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:34:19.745 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:19.741+0000 7f92a7e00640 1 -- 172.21.15.47:0/4066455708 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5635246298b0 msgr2=0x563524636d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:34:19.745 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:19.741+0000 7f92a7e00640 1 --2- 172.21.15.47:0/4066455708 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5635246298b0 0x563524636d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:19.746 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:19.741+0000 7f92a7e00640 1 -- 172.21.15.47:0/4066455708 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563524632de0 msgr2=0x56352469d4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:34:19.746 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:19.741+0000 7f92a7e00640 1 --2- 172.21.15.47:0/4066455708 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563524632de0 0x56352469d4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:19.746 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:19.741+0000 7f92a7e00640 1 -- 172.21.15.47:0/4066455708 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56352469daa0 con 0x56352462a260 2024-03-21T10:34:19.746 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:19.741+0000 7f92a7e00640 1 --2- 172.21.15.47:0/4066455708 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56352462a260 0x563524637240 secure :-1 s=READY pgs=1218 cs=0 l=1 rev1=1 crypto rx=0x7f929409bbb0 tx=0x7f929400ce60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=55b1d466451937a1 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:34:19.746 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:19.741+0000 7f928ea00640 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-03-21T10:34:19.746 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:19.741+0000 7f928ea00640 1 -- 172.21.15.47:0/4066455708 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92940cd690 con 0x56352462a260 2024-03-21T10:34:19.746 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:19.741+0000 7f928ea00640 1 -- 172.21.15.47:0/4066455708 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(103..103 src has 1..103) v4 ==== 9562+0+0 (secure 0 0 0) 0x7f92940d3b50 con 0x56352462a260 2024-03-21T10:34:19.746 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:19.741+0000 7f92b29d4740 1 -- 172.21.15.47:0/4066455708 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f92a0064070 con 0x56352462a260 2024-03-21T10:34:19.746 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:19.741+0000 7f928ea00640 1 -- 172.21.15.47:0/4066455708 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9294010720 con 0x56352462a260 2024-03-21T10:34:19.746 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:19.741+0000 7f92b29d4740 10 client.18471 fetch_fsmap learned FSMap version 32 2024-03-21T10:34:19.746 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:19.742+0000 7f92b29d4740 1 -- 172.21.15.47:0/4066455708 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x56352469edb0 con 0x56352462a260 2024-03-21T10:34:19.746 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:19.742+0000 7f928ea00640 1 -- 172.21.15.47:0/4066455708 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f92a0064070 con 0x56352462a260 2024-03-21T10:34:19.746 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:19.742+0000 7f928ea00640 1 -- 172.21.15.47:0/4066455708 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 32) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f9294097c00 con 0x56352462a260 2024-03-21T10:34:19.747 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:19.742+0000 7f92b29d4740 10 client.18471 fetch_fsmap finished waiting for FSMap version 32 2024-03-21T10:34:19.747 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:19.742+0000 7f92b29d4740 10 client.18471 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.k 2024-03-21T10:34:19.747 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:19.742+0000 7f92b29d4740 20 client.18471 populate_metadata read hostname 'smithi047' 2024-03-21T10:34:19.747 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:19.742+0000 7f92b29d4740 1 --2- 172.21.15.47:0/4066455708 >> [v2:172.21.15.100:6832/2723302038,v1:172.21.15.100:6833/2723302038] conn(0x56352469f510 0x5635246a1930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:34:19.747 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:19.742+0000 7f92b29d4740 4 client.18471 mds_command: new command op to 34427 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-21T10:34:19.747 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:19.742+0000 7f92b29d4740 1 -- 172.21.15.47:0/4066455708 --> [v2:172.21.15.100:6832/2723302038,v1:172.21.15.100:6833/2723302038] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x563524594b50 con 0x56352469f510 2024-03-21T10:34:19.747 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:19.743+0000 7f92ad200640 1 --2- 172.21.15.47:0/4066455708 >> [v2:172.21.15.100:6832/2723302038,v1:172.21.15.100:6833/2723302038] conn(0x56352469f510 0x5635246a1930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:34:19.747 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:19.743+0000 7f92ad200640 1 --2- 172.21.15.47:0/4066455708 >> [v2:172.21.15.100:6832/2723302038,v1:172.21.15.100:6833/2723302038] conn(0x56352469f510 0x5635246a1930 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=2cca045a52a042e7 server_cookie=c37417ade7aff9a0 in_seq=0 out_seq=0 2024-03-21T10:34:19.747 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:19.743+0000 7f928ea00640 10 client.18471 ms_handle_connect on v2:172.21.15.100:6832/2723302038 2024-03-21T10:34:19.747 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:19.744+0000 7f928ea00640 1 -- 172.21.15.47:0/4066455708 <== mds.3 v2:172.21.15.100:6832/2723302038 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x563524594b50 con 0x56352469f510 2024-03-21T10:34:19.747 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:19.744+0000 7f928ea00640 10 client.18471 handle_command_reply: tid=0 2024-03-21T10:34:19.747 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:19.744+0000 7f92b29d4740 1 client.18471 shutdown 2024-03-21T10:34:19.747 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:19.744+0000 7f92b29d4740 1 -- 172.21.15.47:0/4066455708 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56352462a260 msgr2=0x563524637240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:34:19.747 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:19.744+0000 7f92b29d4740 1 --2- 172.21.15.47:0/4066455708 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56352462a260 0x563524637240 secure :-1 s=READY pgs=1218 cs=0 l=1 rev1=1 crypto rx=0x7f929409bbb0 tx=0x7f929400ce60 comp rx=0 tx=0).stop 2024-03-21T10:34:19.747 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:19.744+0000 7f92b29d4740 1 -- 172.21.15.47:0/4066455708 shutdown_connections 2024-03-21T10:34:19.747 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:19.744+0000 7f92b29d4740 1 --2- 172.21.15.47:0/4066455708 >> [v2:172.21.15.100:6832/2723302038,v1:172.21.15.100:6833/2723302038] conn(0x56352469f510 0x5635246a1930 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:19.748 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:19.744+0000 7f92b29d4740 1 --2- 172.21.15.47:0/4066455708 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563524632de0 0x56352469d4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:19.748 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:19.744+0000 7f92b29d4740 1 --2- 172.21.15.47:0/4066455708 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56352462a260 0x563524637240 unknown :-1 s=CLOSED pgs=1218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:19.748 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:19.744+0000 7f928ea00640 0 client.18471 ms_handle_reset on v2:172.21.15.100:6832/2723302038 2024-03-21T10:34:19.748 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:19.744+0000 7f92b29d4740 1 --2- 172.21.15.47:0/4066455708 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5635246298b0 0x563524636d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:19.748 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:19.744+0000 7f92b29d4740 1 -- 172.21.15.47:0/4066455708 >> 172.21.15.47:0/4066455708 conn(0x563524623f80 msgr2=0x563524623160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:34:19.748 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:19.744+0000 7f92b29d4740 1 -- 172.21.15.47:0/4066455708 shutdown_connections 2024-03-21T10:34:19.748 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:19.744+0000 7f92b29d4740 1 -- 172.21.15.47:0/4066455708 wait complete. 2024-03-21T10:34:19.748 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:19.744+0000 7f92b29d4740 20 client.18471 trim_cache size 0 max 16384 2024-03-21T10:34:19.748 INFO:teuthology.orchestra.run.smithi047.stdout:[] 2024-03-21T10:34:19.749 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:19.745+0000 7f92ae600640 1 -- 172.21.15.47:0/1121738860 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f92900a3110 msgr2=0x7f92900a55d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:34:19.749 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:19.745+0000 7f92ae600640 1 --2- 172.21.15.47:0/1121738860 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f92900a3110 0x7f92900a55d0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f92a0006010 tx=0x7f92a0009800 comp rx=0 tx=0).stop 2024-03-21T10:34:19.749 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:19.745+0000 7f92ae600640 1 -- 172.21.15.47:0/1121738860 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f92a8100de0 msgr2=0x7f92a819e7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:34:19.749 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:19.745+0000 7f92ae600640 1 --2- 172.21.15.47:0/1121738860 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f92a8100de0 0x7f92a819e7d0 secure :-1 s=READY pgs=1215 cs=0 l=1 rev1=1 crypto rx=0x7f929400e9f0 tx=0x7f929400eeb0 comp rx=0 tx=0).stop 2024-03-21T10:34:19.749 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:19.745+0000 7f92ae600640 1 -- 172.21.15.47:0/1121738860 shutdown_connections 2024-03-21T10:34:19.749 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:19.745+0000 7f92ae600640 1 --2- 172.21.15.47:0/1121738860 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f92a81058c0 0x7f92a81a2b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:19.749 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:19.745+0000 7f92ae600640 1 --2- 172.21.15.47:0/1121738860 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f92a8100de0 0x7f92a819e7d0 unknown :-1 s=CLOSED pgs=1215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:19.749 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:19.745+0000 7f92ae600640 1 --2- 172.21.15.47:0/1121738860 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f92900a3110 0x7f92900a55d0 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:19.749 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:19.745+0000 7f92ae600640 1 --2- 172.21.15.47:0/1121738860 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f92a81004c0 0x7f92a819e290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:19.749 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:19.745+0000 7f92ae600640 1 -- 172.21.15.47:0/1121738860 >> 172.21.15.47:0/1121738860 conn(0x7f92a80fbef0 msgr2=0x7f92a8103270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:34:19.750 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:19.745+0000 7f92ae600640 1 -- 172.21.15.47:0/1121738860 shutdown_connections 2024-03-21T10:34:19.750 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:19.745+0000 7f92ae600640 1 -- 172.21.15.47:0/1121738860 wait complete. 2024-03-21T10:34:19.760 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-21T10:34:19.760 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-03-21T10:34:19.966 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:19.968+0000 7f1f32600640 1 -- 172.21.15.47:0/365969082 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1f2c104200 msgr2=0x7f1f2c110710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:34:19.967 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:19.968+0000 7f1f32600640 1 --2- 172.21.15.47:0/365969082 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1f2c104200 0x7f1f2c110710 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f1f24009900 tx=0x7f1f2402bc30 comp rx=0 tx=0).stop 2024-03-21T10:34:19.967 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:19.968+0000 7f1f32600640 1 -- 172.21.15.47:0/365969082 shutdown_connections 2024-03-21T10:34:19.967 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:19.968+0000 7f1f32600640 1 --2- 172.21.15.47:0/365969082 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1f2c104200 0x7f1f2c110710 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:19.967 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:19.968+0000 7f1f32600640 1 --2- 172.21.15.47:0/365969082 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1f2c103840 0x7f1f2c103cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:19.967 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:19.968+0000 7f1f32600640 1 --2- 172.21.15.47:0/365969082 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1f2c109850 0x7f1f2c109c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:19.967 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:19.968+0000 7f1f32600640 1 -- 172.21.15.47:0/365969082 >> 172.21.15.47:0/365969082 conn(0x7f1f2c0ff2f0 msgr2=0x7f1f2c101710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:34:19.967 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:19.968+0000 7f1f32600640 1 -- 172.21.15.47:0/365969082 shutdown_connections 2024-03-21T10:34:19.967 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:19.968+0000 7f1f32600640 1 -- 172.21.15.47:0/365969082 wait complete. 2024-03-21T10:34:19.968 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:19.970+0000 7f1f32600640 1 Processor -- start 2024-03-21T10:34:19.968 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:19.970+0000 7f1f32600640 1 -- start start 2024-03-21T10:34:19.969 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:19.970+0000 7f1f32600640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1f2c103840 0x7f1f2c19a050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:34:19.969 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:19.970+0000 7f1f32600640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1f2c104200 0x7f1f2c19a590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:34:19.969 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:19.970+0000 7f1f32600640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1f2c109850 0x7f1f2c19e910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:34:19.969 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:19.970+0000 7f1f32600640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f2c113b10 con 0x7f1f2c104200 2024-03-21T10:34:19.969 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:19.970+0000 7f1f32600640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f2c113990 con 0x7f1f2c103840 2024-03-21T10:34:19.969 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:19.970+0000 7f1f32600640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f2c113c90 con 0x7f1f2c109850 2024-03-21T10:34:19.969 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:19.971+0000 7f1f22a00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1f2c104200 0x7f1f2c19a590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:34:19.969 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:19.971+0000 7f1f2b400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1f2c103840 0x7f1f2c19a050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:34:19.969 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:19.971+0000 7f1f2b400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1f2c103840 0x7f1f2c19a050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:50282/0 (socket says 172.21.15.47:50282) 2024-03-21T10:34:19.969 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:19.971+0000 7f1f2b400640 1 -- 172.21.15.47:0/3975453024 learned_addr learned my addr 172.21.15.47:0/3975453024 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:34:19.970 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:19.971+0000 7f1f2be00640 1 --2- 172.21.15.47:0/3975453024 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1f2c109850 0x7f1f2c19e910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:34:19.971 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:19.971+0000 7f1f22a00640 1 -- 172.21.15.47:0/3975453024 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1f2c109850 msgr2=0x7f1f2c19e910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:34:19.971 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:19.971+0000 7f1f22a00640 1 --2- 172.21.15.47:0/3975453024 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1f2c109850 0x7f1f2c19e910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:19.971 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:19.971+0000 7f1f22a00640 1 -- 172.21.15.47:0/3975453024 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1f2c103840 msgr2=0x7f1f2c19a050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:34:19.971 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:19.971+0000 7f1f22a00640 1 --2- 172.21.15.47:0/3975453024 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1f2c103840 0x7f1f2c19a050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:19.971 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:19.971+0000 7f1f22a00640 1 -- 172.21.15.47:0/3975453024 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f2c19f040 con 0x7f1f2c104200 2024-03-21T10:34:19.971 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:19.971+0000 7f1f2b400640 1 --2- 172.21.15.47:0/3975453024 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1f2c103840 0x7f1f2c19a050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T10:34:19.971 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:19.972+0000 7f1f22a00640 1 --2- 172.21.15.47:0/3975453024 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1f2c104200 0x7f1f2c19a590 secure :-1 s=READY pgs=1219 cs=0 l=1 rev1=1 crypto rx=0x7f1f1400eb00 tx=0x7f1f1400c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:34:19.971 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:19.972+0000 7f1f28c00640 1 -- 172.21.15.47:0/3975453024 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f14010430 con 0x7f1f2c104200 2024-03-21T10:34:19.971 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:19.972+0000 7f1f28c00640 1 -- 172.21.15.47:0/3975453024 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f1f14010a80 con 0x7f1f2c104200 2024-03-21T10:34:19.971 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:19.972+0000 7f1f32600640 1 -- 172.21.15.47:0/3975453024 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1f2c19f330 con 0x7f1f2c104200 2024-03-21T10:34:19.971 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:19.972+0000 7f1f28c00640 1 -- 172.21.15.47:0/3975453024 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f14017ad0 con 0x7f1f2c104200 2024-03-21T10:34:19.971 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:19.972+0000 7f1f32600640 1 -- 172.21.15.47:0/3975453024 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1f2c1a6bc0 con 0x7f1f2c104200 2024-03-21T10:34:19.974 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:19.976+0000 7f1f353a1740 1 Processor -- start 2024-03-21T10:34:19.974 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:19.976+0000 7f1f28c00640 1 -- 172.21.15.47:0/3975453024 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 27) v1 ==== 137985+0+0 (secure 0 0 0) 0x7f1f14010c30 con 0x7f1f2c104200 2024-03-21T10:34:19.974 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:19.976+0000 7f1f353a1740 1 -- start start 2024-03-21T10:34:19.975 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:19.976+0000 7f1f353a1740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564c2d46f6d0 0x564c2d46fab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:34:19.975 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:19.976+0000 7f1f353a1740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x564c2d470080 0x564c2d478630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:34:19.975 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:19.976+0000 7f1f353a1740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564c2d478c00 0x564c2d47b050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:34:19.975 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:19.976+0000 7f1f353a1740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x564c2d3d32c0 con 0x564c2d470080 2024-03-21T10:34:19.975 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:19.976+0000 7f1f353a1740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x564c2d37bc90 con 0x564c2d46f6d0 2024-03-21T10:34:19.975 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:19.976+0000 7f1f353a1740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x564c2d3cf360 con 0x564c2d478c00 2024-03-21T10:34:19.975 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:19.976+0000 7f1f2b400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x564c2d470080 0x564c2d478630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:34:19.975 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:19.977+0000 7f1f2b400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x564c2d470080 0x564c2d478630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:35652/0 (socket says 172.21.15.47:35652) 2024-03-21T10:34:19.976 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:19.977+0000 7f1f2b400640 1 -- 172.21.15.47:0/1041945161 learned_addr learned my addr 172.21.15.47:0/1041945161 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:34:19.976 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:19.977+0000 7f1f2be00640 1 --2- 172.21.15.47:0/1041945161 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564c2d46f6d0 0x564c2d46fab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:34:19.976 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:19.977+0000 7f1f2b400640 1 -- 172.21.15.47:0/1041945161 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564c2d478c00 msgr2=0x564c2d47b050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:34:19.976 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:19.977+0000 7f1f2b400640 1 --2- 172.21.15.47:0/1041945161 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564c2d478c00 0x564c2d47b050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:19.976 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:19.977+0000 7f1f2b400640 1 -- 172.21.15.47:0/1041945161 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564c2d46f6d0 msgr2=0x564c2d46fab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:34:19.976 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:19.977+0000 7f1f2b400640 1 --2- 172.21.15.47:0/1041945161 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564c2d46f6d0 0x564c2d46fab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:19.976 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:19.977+0000 7f1f2b400640 1 -- 172.21.15.47:0/1041945161 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564c2d3a0080 con 0x564c2d470080 2024-03-21T10:34:19.977 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:19.977+0000 7f1f2b400640 1 --2- 172.21.15.47:0/1041945161 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x564c2d470080 0x564c2d478630 secure :-1 s=READY pgs=1220 cs=0 l=1 rev1=1 crypto rx=0x7f1f1c0115f0 tx=0x7f1f1c011ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b3ee5812f9f9f43f server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:34:19.978 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:19.978+0000 7f1f28c00640 1 --2- 172.21.15.47:0/3975453024 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f1f100a3090 0x7f1f100a5550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:34:19.978 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:19.978+0000 7f1f20c00640 1 -- 172.21.15.47:0/1041945161 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f1c01e070 con 0x564c2d470080 2024-03-21T10:34:19.978 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:19.978+0000 7f1f20c00640 1 -- 172.21.15.47:0/1041945161 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f1f1c00fda0 con 0x564c2d470080 2024-03-21T10:34:19.978 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:19.978+0000 7f1f20c00640 1 -- 172.21.15.47:0/1041945161 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f1c00e3b0 con 0x564c2d470080 2024-03-21T10:34:19.978 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:19.978+0000 7f1f28c00640 1 -- 172.21.15.47:0/3975453024 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 9562+0+0 (secure 0 0 0) 0x7f1f140d0630 con 0x7f1f2c104200 2024-03-21T10:34:19.978 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:19.978+0000 7f1f2be00640 1 --2- 172.21.15.47:0/3975453024 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f1f100a3090 0x7f1f100a5550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:34:19.978 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:19.979+0000 7f1f353a1740 1 -- 172.21.15.47:0/1041945161 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x564c2d470080 msgr2=0x564c2d478630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:34:19.978 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:19.979+0000 7f1f353a1740 1 --2- 172.21.15.47:0/1041945161 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x564c2d470080 0x564c2d478630 secure :-1 s=READY pgs=1220 cs=0 l=1 rev1=1 crypto rx=0x7f1f1c0115f0 tx=0x7f1f1c011ab0 comp rx=0 tx=0).stop 2024-03-21T10:34:19.978 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:19.979+0000 7f1f353a1740 1 -- 172.21.15.47:0/1041945161 shutdown_connections 2024-03-21T10:34:19.978 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:19.979+0000 7f1f353a1740 1 --2- 172.21.15.47:0/1041945161 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564c2d478c00 0x564c2d47b050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:19.978 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:19.979+0000 7f1f353a1740 1 --2- 172.21.15.47:0/1041945161 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x564c2d470080 0x564c2d478630 unknown :-1 s=CLOSED pgs=1220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:19.978 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:19.979+0000 7f1f353a1740 1 --2- 172.21.15.47:0/1041945161 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564c2d46f6d0 0x564c2d46fab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:19.978 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:19.979+0000 7f1f353a1740 1 -- 172.21.15.47:0/1041945161 >> 172.21.15.47:0/1041945161 conn(0x564c2d469da0 msgr2=0x564c2d46a180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:34:19.978 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:19.979+0000 7f1f353a1740 1 -- 172.21.15.47:0/1041945161 shutdown_connections 2024-03-21T10:34:19.978 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:19.979+0000 7f1f2be00640 1 --2- 172.21.15.47:0/3975453024 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f1f100a3090 0x7f1f100a5550 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f1f24009230 tx=0x7f1f240058d0 comp rx=0 tx=0).ready entity=mgr.15687 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:34:19.979 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:19.979+0000 7f1f353a1740 1 -- 172.21.15.47:0/1041945161 wait complete. 2024-03-21T10:34:19.979 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:19.979+0000 7f1f353a1740 1 -- start start 2024-03-21T10:34:19.979 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:19.980+0000 7f1f353a1740 1 Processor -- start 2024-03-21T10:34:19.982 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:19.980+0000 7f1f353a1740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564c2d46f6d0 0x564c2d4785a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:34:19.982 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:19.980+0000 7f1f353a1740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x564c2d470080 0x564c2d475930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:34:19.982 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:19.980+0000 7f1f353a1740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564c2d478c00 0x564c2d476070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:34:19.982 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:19.980+0000 7f1f353a1740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x564c2d47bb70 con 0x564c2d470080 2024-03-21T10:34:19.982 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:19.980+0000 7f1f353a1740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x564c2d38f0d0 con 0x564c2d46f6d0 2024-03-21T10:34:19.982 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:19.980+0000 7f1f353a1740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x564c2d47bcf0 con 0x564c2d478c00 2024-03-21T10:34:19.982 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:19.981+0000 7f1f353a1740 10 client.0 _ll_register_callbacks cb 0x564c2d393e80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T10:34:19.982 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:19.981+0000 7f1f22a00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x564c2d470080 0x564c2d475930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:34:19.982 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:19.981+0000 7f1f2b400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564c2d46f6d0 0x564c2d4785a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:34:19.982 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:19.981+0000 7f1f2b400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564c2d46f6d0 0x564c2d4785a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:50296/0 (socket says 172.21.15.47:50296) 2024-03-21T10:34:19.982 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:19.981+0000 7f1f2b400640 1 -- 172.21.15.47:0/1140115286 learned_addr learned my addr 172.21.15.47:0/1140115286 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:34:19.982 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:19.981+0000 7f1f22a00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x564c2d470080 0x564c2d475930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:35654/0 (socket says 172.21.15.47:35654) 2024-03-21T10:34:19.983 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:19.981+0000 7f1f2be00640 1 --2- 172.21.15.47:0/1140115286 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564c2d478c00 0x564c2d476070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:34:19.983 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:19.981+0000 7f1f2b400640 1 -- 172.21.15.47:0/1140115286 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564c2d478c00 msgr2=0x564c2d476070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:34:19.983 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:19.981+0000 7f1f2b400640 1 --2- 172.21.15.47:0/1140115286 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564c2d478c00 0x564c2d476070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:19.983 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:19.981+0000 7f1f2b400640 1 -- 172.21.15.47:0/1140115286 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x564c2d470080 msgr2=0x564c2d475930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:34:19.983 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:19.981+0000 7f1f2b400640 1 --2- 172.21.15.47:0/1140115286 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x564c2d470080 0x564c2d475930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:19.983 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:19.981+0000 7f1f2b400640 1 -- 172.21.15.47:0/1140115286 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x564c2d0e73f0 con 0x564c2d46f6d0 2024-03-21T10:34:19.983 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:19.981+0000 7f1f2be00640 1 --2- 172.21.15.47:0/1140115286 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564c2d478c00 0x564c2d476070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T10:34:19.983 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:19.981+0000 7f1f22a00640 1 --2- 172.21.15.47:0/1140115286 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x564c2d470080 0x564c2d475930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T10:34:19.983 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:19.982+0000 7f1f2b400640 1 --2- 172.21.15.47:0/1140115286 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564c2d46f6d0 0x564c2d4785a0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f1f1c0115c0 tx=0x7f1f1c0095c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=45282df20d216b7d server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:34:19.983 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:19.982+0000 7f1f19600640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-21T10:34:19.983 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:19.982+0000 7f1f19600640 1 -- 172.21.15.47:0/1140115286 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f1c026070 con 0x564c2d46f6d0 2024-03-21T10:34:19.983 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:19.982+0000 7f1f353a1740 1 -- 172.21.15.47:0/1140115286 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f1f1c00fda0 con 0x564c2d46f6d0 2024-03-21T10:34:19.983 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:19.982+0000 7f1f19600640 1 -- 172.21.15.47:0/1140115286 <== mon.2 v2:172.21.15.100:3300/0 2 ==== osd_map(103..103 src has 1..103) v4 ==== 9562+0+0 (secure 0 0 0) 0x7f1f1c00fb00 con 0x564c2d46f6d0 2024-03-21T10:34:19.983 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:19.982+0000 7f1f19600640 1 -- 172.21.15.47:0/1140115286 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1f1c032050 con 0x564c2d46f6d0 2024-03-21T10:34:19.983 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:19.983+0000 7f1f353a1740 10 client.38249 fetch_fsmap learned FSMap version 32 2024-03-21T10:34:19.984 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:19.983+0000 7f1f353a1740 1 -- 172.21.15.47:0/1140115286 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x564c2d1b17d0 con 0x564c2d46f6d0 2024-03-21T10:34:19.984 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:19.983+0000 7f1f19600640 1 -- 172.21.15.47:0/1140115286 <== mon.2 v2:172.21.15.100:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f1f1c00fda0 con 0x564c2d46f6d0 2024-03-21T10:34:19.984 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:19.983+0000 7f1f19600640 1 -- 172.21.15.47:0/1140115286 <== mon.2 v2:172.21.15.100:3300/0 5 ==== fsmap(e 32) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f1f1c0048f0 con 0x564c2d46f6d0 2024-03-21T10:34:19.984 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:19.983+0000 7f1f353a1740 10 client.38249 fetch_fsmap finished waiting for FSMap version 32 2024-03-21T10:34:19.984 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:19.983+0000 7f1f353a1740 10 client.38249 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.e 2024-03-21T10:34:19.984 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:19.983+0000 7f1f353a1740 20 client.38249 populate_metadata read hostname 'smithi047' 2024-03-21T10:34:19.984 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:19.983+0000 7f1f353a1740 1 --2- 172.21.15.47:0/1140115286 >> [v2:172.21.15.100:6834/2597849957,v1:172.21.15.100:6835/2597849957] conn(0x564c2d4889e0 0x564c2d48add0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:34:19.984 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:19.983+0000 7f1f353a1740 4 client.38249 mds_command: new command op to 34454 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T10:34:19.984 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:19.983+0000 7f1f353a1740 1 -- 172.21.15.47:0/1140115286 --> [v2:172.21.15.100:6834/2597849957,v1:172.21.15.100:6835/2597849957] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x564c2d3e0800 con 0x564c2d4889e0 2024-03-21T10:34:19.984 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:19.984+0000 7f1f22a00640 1 --2- 172.21.15.47:0/1140115286 >> [v2:172.21.15.100:6834/2597849957,v1:172.21.15.100:6835/2597849957] conn(0x564c2d4889e0 0x564c2d48add0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:34:19.985 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:19.984+0000 7f1f22a00640 1 --2- 172.21.15.47:0/1140115286 >> [v2:172.21.15.100:6834/2597849957,v1:172.21.15.100:6835/2597849957] conn(0x564c2d4889e0 0x564c2d48add0 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=ca4d4daa0919d124 server_cookie=9a9cf51502cb0135 in_seq=0 out_seq=0 2024-03-21T10:34:19.985 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:19.984+0000 7f1f19600640 10 client.38249 ms_handle_connect on v2:172.21.15.100:6834/2597849957 2024-03-21T10:34:19.985 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:19.986+0000 7f1f19600640 1 -- 172.21.15.47:0/1140115286 <== mds.2 v2:172.21.15.100:6834/2597849957 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x564c2d3e0800 con 0x564c2d4889e0 2024-03-21T10:34:19.986 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:19.986+0000 7f1f19600640 10 client.38249 handle_command_reply: tid=0 2024-03-21T10:34:19.986 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:19.986+0000 7f1f353a1740 1 client.38249 shutdown 2024-03-21T10:34:19.986 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:19.986+0000 7f1f353a1740 1 -- 172.21.15.47:0/1140115286 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564c2d46f6d0 msgr2=0x564c2d4785a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:34:19.986 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:19.986+0000 7f1f353a1740 1 --2- 172.21.15.47:0/1140115286 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564c2d46f6d0 0x564c2d4785a0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f1f1c0115c0 tx=0x7f1f1c0095c0 comp rx=0 tx=0).stop 2024-03-21T10:34:19.986 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:19.986+0000 7f1f353a1740 1 -- 172.21.15.47:0/1140115286 shutdown_connections 2024-03-21T10:34:19.986 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:19.986+0000 7f1f353a1740 1 --2- 172.21.15.47:0/1140115286 >> [v2:172.21.15.100:6834/2597849957,v1:172.21.15.100:6835/2597849957] conn(0x564c2d4889e0 0x564c2d48add0 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:19.986 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:19.986+0000 7f1f353a1740 1 --2- 172.21.15.47:0/1140115286 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564c2d478c00 0x564c2d476070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:19.986 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:19.986+0000 7f1f353a1740 1 --2- 172.21.15.47:0/1140115286 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x564c2d470080 0x564c2d475930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:19.986 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:19.986+0000 7f1f353a1740 1 --2- 172.21.15.47:0/1140115286 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564c2d46f6d0 0x564c2d4785a0 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:19.986 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:19.986+0000 7f1f353a1740 1 -- 172.21.15.47:0/1140115286 >> 172.21.15.47:0/1140115286 conn(0x564c2d469da0 msgr2=0x564c2d46baf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:34:19.986 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:19.986+0000 7f1f19600640 0 client.38249 ms_handle_reset on v2:172.21.15.100:6834/2597849957 2024-03-21T10:34:19.986 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:19.986+0000 7f1f353a1740 1 -- 172.21.15.47:0/1140115286 shutdown_connections 2024-03-21T10:34:19.986 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:19.986+0000 7f1f353a1740 1 -- 172.21.15.47:0/1140115286 wait complete. 2024-03-21T10:34:19.986 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:19.986+0000 7f1f353a1740 20 client.38249 trim_cache size 0 max 16384 2024-03-21T10:34:20.003 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:20.005+0000 7f1f353a1740 1 Processor -- start 2024-03-21T10:34:20.003 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:20.005+0000 7f1f353a1740 1 -- start start 2024-03-21T10:34:20.005 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:20.005+0000 7f1f353a1740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564c2d46f6d0 0x564c2d478060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:34:20.005 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:20.005+0000 7f1f353a1740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x564c2d470080 0x564c2d4785a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:34:20.005 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:20.005+0000 7f1f353a1740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564c2d478c00 0x564c2d472b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:34:20.005 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:20.005+0000 7f1f353a1740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f2c1b0c90 con 0x564c2d470080 2024-03-21T10:34:20.005 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:20.005+0000 7f1f353a1740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f2c1b2610 con 0x564c2d46f6d0 2024-03-21T10:34:20.005 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:20.005+0000 7f1f353a1740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f2c1b12f0 con 0x564c2d478c00 2024-03-21T10:34:20.005 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:20.006+0000 7f1f22a00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x564c2d470080 0x564c2d4785a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:34:20.006 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:20.006+0000 7f1f22a00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x564c2d470080 0x564c2d4785a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:35660/0 (socket says 172.21.15.47:35660) 2024-03-21T10:34:20.006 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:20.006+0000 7f1f22a00640 1 -- 172.21.15.47:0/990976164 learned_addr learned my addr 172.21.15.47:0/990976164 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:34:20.006 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:20.006+0000 7f1f2b400640 1 --2- 172.21.15.47:0/990976164 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564c2d46f6d0 0x564c2d478060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:34:20.006 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:20.006+0000 7f1f22a00640 1 -- 172.21.15.47:0/990976164 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564c2d478c00 msgr2=0x564c2d472b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:34:20.006 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:20.006+0000 7f1f22a00640 1 --2- 172.21.15.47:0/990976164 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564c2d478c00 0x564c2d472b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:20.006 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:20.006+0000 7f1f22a00640 1 -- 172.21.15.47:0/990976164 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564c2d46f6d0 msgr2=0x564c2d478060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:34:20.006 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:20.006+0000 7f1f22a00640 1 --2- 172.21.15.47:0/990976164 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564c2d46f6d0 0x564c2d478060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:20.006 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:20.006+0000 7f1f22a00640 1 -- 172.21.15.47:0/990976164 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564c2d471490 con 0x564c2d470080 2024-03-21T10:34:20.006 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:20.006+0000 7f1f2b400640 1 --2- 172.21.15.47:0/990976164 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564c2d46f6d0 0x564c2d478060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T10:34:20.006 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:20.006+0000 7f1f22a00640 1 --2- 172.21.15.47:0/990976164 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x564c2d470080 0x564c2d4785a0 secure :-1 s=READY pgs=1221 cs=0 l=1 rev1=1 crypto rx=0x7f1f14098a70 tx=0x7f1f14098220 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2f1751de8ba37b2 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:34:20.006 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:20.007+0000 7f1f1be00640 1 -- 172.21.15.47:0/990976164 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f1409c610 con 0x564c2d470080 2024-03-21T10:34:20.006 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:20.007+0000 7f1f1be00640 1 -- 172.21.15.47:0/990976164 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f1f140a0760 con 0x564c2d470080 2024-03-21T10:34:20.006 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:20.007+0000 7f1f1be00640 1 -- 172.21.15.47:0/990976164 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f14017c70 con 0x564c2d470080 2024-03-21T10:34:20.006 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:20.007+0000 7f1f353a1740 1 -- 172.21.15.47:0/990976164 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x564c2d470080 msgr2=0x564c2d4785a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:34:20.007 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:20.007+0000 7f1f353a1740 1 --2- 172.21.15.47:0/990976164 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x564c2d470080 0x564c2d4785a0 secure :-1 s=READY pgs=1221 cs=0 l=1 rev1=1 crypto rx=0x7f1f14098a70 tx=0x7f1f14098220 comp rx=0 tx=0).stop 2024-03-21T10:34:20.009 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:20.007+0000 7f1f353a1740 1 -- 172.21.15.47:0/990976164 shutdown_connections 2024-03-21T10:34:20.009 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:20.007+0000 7f1f353a1740 1 --2- 172.21.15.47:0/990976164 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564c2d478c00 0x564c2d472b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:20.010 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:20.007+0000 7f1f353a1740 1 --2- 172.21.15.47:0/990976164 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x564c2d470080 0x564c2d4785a0 unknown :-1 s=CLOSED pgs=1221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:20.010 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:20.007+0000 7f1f353a1740 1 --2- 172.21.15.47:0/990976164 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564c2d46f6d0 0x564c2d478060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:20.010 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:20.007+0000 7f1f353a1740 1 -- 172.21.15.47:0/990976164 >> 172.21.15.47:0/990976164 conn(0x564c2d469da0 msgr2=0x564c2d468eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:34:20.010 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:20.008+0000 7f1f353a1740 1 -- 172.21.15.47:0/990976164 shutdown_connections 2024-03-21T10:34:20.010 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:20.008+0000 7f1f353a1740 1 -- 172.21.15.47:0/990976164 wait complete. 2024-03-21T10:34:20.010 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:20.008+0000 7f1f353a1740 1 -- start start 2024-03-21T10:34:20.010 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:20.008+0000 7f1f353a1740 1 Processor -- start 2024-03-21T10:34:20.010 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:20.009+0000 7f1f353a1740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564c2d46f6d0 0x564c2d47e810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:34:20.010 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:20.009+0000 7f1f353a1740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x564c2d470080 0x564c2d47ed50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:34:20.010 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:20.009+0000 7f1f353a1740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564c2d478c00 0x564c2d4e2fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:34:20.010 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:20.009+0000 7f1f353a1740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f2c1b1fb0 con 0x564c2d470080 2024-03-21T10:34:20.010 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:20.009+0000 7f1f353a1740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f2c1b1950 con 0x564c2d478c00 2024-03-21T10:34:20.010 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:20.009+0000 7f1f353a1740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f2c1aee70 con 0x564c2d46f6d0 2024-03-21T10:34:20.011 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:20.009+0000 7f1f353a1740 10 client.0 _ll_register_callbacks cb 0x7f1f1c01c070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T10:34:20.011 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:20.009+0000 7f1f22a00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x564c2d470080 0x564c2d47ed50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:34:20.011 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:20.009+0000 7f1f22a00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x564c2d470080 0x564c2d47ed50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:35670/0 (socket says 172.21.15.47:35670) 2024-03-21T10:34:20.011 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:20.009+0000 7f1f22a00640 1 -- 172.21.15.47:0/83720343 learned_addr learned my addr 172.21.15.47:0/83720343 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:34:20.011 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:20.009+0000 7f1f2be00640 1 --2- 172.21.15.47:0/83720343 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564c2d478c00 0x564c2d4e2fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:34:20.011 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:20.009+0000 7f1f2b400640 1 --2- 172.21.15.47:0/83720343 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564c2d46f6d0 0x564c2d47e810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:34:20.011 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:20.010+0000 7f1f22a00640 1 -- 172.21.15.47:0/83720343 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564c2d46f6d0 msgr2=0x564c2d47e810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:34:20.011 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:20.010+0000 7f1f22a00640 1 --2- 172.21.15.47:0/83720343 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564c2d46f6d0 0x564c2d47e810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:20.011 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:20.010+0000 7f1f22a00640 1 -- 172.21.15.47:0/83720343 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564c2d478c00 msgr2=0x564c2d4e2fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:34:20.011 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:20.010+0000 7f1f22a00640 1 --2- 172.21.15.47:0/83720343 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564c2d478c00 0x564c2d4e2fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:20.011 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:20.010+0000 7f1f22a00640 1 -- 172.21.15.47:0/83720343 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x564c2d4e3510 con 0x564c2d470080 2024-03-21T10:34:20.011 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:20.010+0000 7f1f2b400640 1 --2- 172.21.15.47:0/83720343 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564c2d46f6d0 0x564c2d47e810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T10:34:20.011 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:20.010+0000 7f1f22a00640 1 --2- 172.21.15.47:0/83720343 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x564c2d470080 0x564c2d47ed50 secure :-1 s=READY pgs=1222 cs=0 l=1 rev1=1 crypto rx=0x7f1f1409c5a0 tx=0x7f1f1409c750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c285c0a42c40f36d server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:34:20.011 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:20.010+0000 7f1f19600640 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-03-21T10:34:20.011 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:20.010+0000 7f1f19600640 1 -- 172.21.15.47:0/83720343 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f140b4070 con 0x564c2d470080 2024-03-21T10:34:20.012 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:20.010+0000 7f1f353a1740 1 -- 172.21.15.47:0/83720343 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f1f140a0760 con 0x564c2d470080 2024-03-21T10:34:20.012 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:20.010+0000 7f1f19600640 1 -- 172.21.15.47:0/83720343 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(103..103 src has 1..103) v4 ==== 9562+0+0 (secure 0 0 0) 0x7f1f140cdbf0 con 0x564c2d470080 2024-03-21T10:34:20.012 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:20.011+0000 7f1f19600640 1 -- 172.21.15.47:0/83720343 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1f14017c70 con 0x564c2d470080 2024-03-21T10:34:20.012 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:20.011+0000 7f1f353a1740 10 client.18489 fetch_fsmap learned FSMap version 32 2024-03-21T10:34:20.012 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:20.011+0000 7f1f353a1740 1 -- 172.21.15.47:0/83720343 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x564c2d4e46d0 con 0x564c2d470080 2024-03-21T10:34:20.012 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:20.011+0000 7f1f19600640 1 -- 172.21.15.47:0/83720343 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f1f140a0760 con 0x564c2d470080 2024-03-21T10:34:20.012 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:20.011+0000 7f1f19600640 1 -- 172.21.15.47:0/83720343 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 32) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f1f140105d0 con 0x564c2d470080 2024-03-21T10:34:20.012 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:20.011+0000 7f1f353a1740 10 client.18489 fetch_fsmap finished waiting for FSMap version 32 2024-03-21T10:34:20.012 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:20.011+0000 7f1f353a1740 10 client.18489 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.e 2024-03-21T10:34:20.012 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:20.011+0000 7f1f353a1740 20 client.18489 populate_metadata read hostname 'smithi047' 2024-03-21T10:34:20.012 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:20.011+0000 7f1f353a1740 1 --2- 172.21.15.47:0/83720343 >> [v2:172.21.15.100:6834/2597849957,v1:172.21.15.100:6835/2597849957] conn(0x564c2d4e4d20 0x564c2d4e7140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:34:20.012 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:20.011+0000 7f1f353a1740 4 client.18489 mds_command: new command op to 34454 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-21T10:34:20.012 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:20.011+0000 7f1f353a1740 1 -- 172.21.15.47:0/83720343 --> [v2:172.21.15.100:6834/2597849957,v1:172.21.15.100:6835/2597849957] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x564c2d3dab50 con 0x564c2d4e4d20 2024-03-21T10:34:20.013 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:20.012+0000 7f1f2b400640 1 --2- 172.21.15.47:0/83720343 >> [v2:172.21.15.100:6834/2597849957,v1:172.21.15.100:6835/2597849957] conn(0x564c2d4e4d20 0x564c2d4e7140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:34:20.014 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:20.012+0000 7f1f2b400640 1 --2- 172.21.15.47:0/83720343 >> [v2:172.21.15.100:6834/2597849957,v1:172.21.15.100:6835/2597849957] conn(0x564c2d4e4d20 0x564c2d4e7140 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=eb9c43191184f40a server_cookie=9d0f4cc23ab6245f in_seq=0 out_seq=0 2024-03-21T10:34:20.014 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:20.012+0000 7f1f19600640 10 client.18489 ms_handle_connect on v2:172.21.15.100:6834/2597849957 2024-03-21T10:34:20.014 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:20.013+0000 7f1f19600640 1 -- 172.21.15.47:0/83720343 <== mds.2 v2:172.21.15.100:6834/2597849957 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x564c2d3dab50 con 0x564c2d4e4d20 2024-03-21T10:34:20.014 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:20.013+0000 7f1f19600640 10 client.18489 handle_command_reply: tid=0 2024-03-21T10:34:20.014 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:20.013+0000 7f1f353a1740 1 client.18489 shutdown 2024-03-21T10:34:20.014 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:20.013+0000 7f1f353a1740 1 -- 172.21.15.47:0/83720343 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x564c2d470080 msgr2=0x564c2d47ed50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:34:20.014 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:20.013+0000 7f1f353a1740 1 --2- 172.21.15.47:0/83720343 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x564c2d470080 0x564c2d47ed50 secure :-1 s=READY pgs=1222 cs=0 l=1 rev1=1 crypto rx=0x7f1f1409c5a0 tx=0x7f1f1409c750 comp rx=0 tx=0).stop 2024-03-21T10:34:20.014 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:20.013+0000 7f1f353a1740 1 -- 172.21.15.47:0/83720343 shutdown_connections 2024-03-21T10:34:20.014 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:20.013+0000 7f1f353a1740 1 --2- 172.21.15.47:0/83720343 >> [v2:172.21.15.100:6834/2597849957,v1:172.21.15.100:6835/2597849957] conn(0x564c2d4e4d20 0x564c2d4e7140 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:20.014 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:20.013+0000 7f1f353a1740 1 --2- 172.21.15.47:0/83720343 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564c2d478c00 0x564c2d4e2fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:20.014 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:20.013+0000 7f1f19600640 0 client.18489 ms_handle_reset on v2:172.21.15.100:6834/2597849957 2024-03-21T10:34:20.014 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:20.013+0000 7f1f353a1740 1 --2- 172.21.15.47:0/83720343 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x564c2d470080 0x564c2d47ed50 unknown :-1 s=CLOSED pgs=1222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:20.014 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:20.013+0000 7f1f353a1740 1 --2- 172.21.15.47:0/83720343 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x564c2d46f6d0 0x564c2d47e810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:20.015 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:20.013+0000 7f1f353a1740 1 -- 172.21.15.47:0/83720343 >> 172.21.15.47:0/83720343 conn(0x564c2d469da0 msgr2=0x564c2d468620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:34:20.015 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:20.013+0000 7f1f353a1740 1 -- 172.21.15.47:0/83720343 shutdown_connections 2024-03-21T10:34:20.015 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:20.014+0000 7f1f353a1740 1 -- 172.21.15.47:0/83720343 wait complete. 2024-03-21T10:34:20.015 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:20.014+0000 7f1f353a1740 20 client.18489 trim_cache size 0 max 16384 2024-03-21T10:34:20.015 INFO:teuthology.orchestra.run.smithi047.stdout:[] 2024-03-21T10:34:20.016 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:20.014+0000 7f1f32600640 1 -- 172.21.15.47:0/3975453024 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f1f100a3090 msgr2=0x7f1f100a5550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:34:20.016 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:20.014+0000 7f1f32600640 1 --2- 172.21.15.47:0/3975453024 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f1f100a3090 0x7f1f100a5550 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f1f24009230 tx=0x7f1f240058d0 comp rx=0 tx=0).stop 2024-03-21T10:34:20.016 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:20.014+0000 7f1f32600640 1 -- 172.21.15.47:0/3975453024 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1f2c104200 msgr2=0x7f1f2c19a590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:34:20.016 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:20.014+0000 7f1f32600640 1 --2- 172.21.15.47:0/3975453024 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1f2c104200 0x7f1f2c19a590 secure :-1 s=READY pgs=1219 cs=0 l=1 rev1=1 crypto rx=0x7f1f1400eb00 tx=0x7f1f1400c2d0 comp rx=0 tx=0).stop 2024-03-21T10:34:20.016 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:20.014+0000 7f1f32600640 1 -- 172.21.15.47:0/3975453024 shutdown_connections 2024-03-21T10:34:20.016 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:20.014+0000 7f1f32600640 1 --2- 172.21.15.47:0/3975453024 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1f2c109850 0x7f1f2c19e910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:20.016 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:20.014+0000 7f1f32600640 1 --2- 172.21.15.47:0/3975453024 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1f2c104200 0x7f1f2c19a590 unknown :-1 s=CLOSED pgs=1219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:20.016 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:20.014+0000 7f1f32600640 1 --2- 172.21.15.47:0/3975453024 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f1f100a3090 0x7f1f100a5550 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:20.016 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:20.014+0000 7f1f32600640 1 --2- 172.21.15.47:0/3975453024 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1f2c103840 0x7f1f2c19a050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:20.016 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:20.014+0000 7f1f32600640 1 -- 172.21.15.47:0/3975453024 >> 172.21.15.47:0/3975453024 conn(0x7f1f2c0ff2f0 msgr2=0x7f1f2c0ffb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:34:20.016 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:20.015+0000 7f1f32600640 1 -- 172.21.15.47:0/3975453024 shutdown_connections 2024-03-21T10:34:20.016 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:20.015+0000 7f1f32600640 1 -- 172.21.15.47:0/3975453024 wait complete. 2024-03-21T10:34:20.030 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-21T10:34:20.030 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-03-21T10:34:20.258 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:20.260+0000 7ff45d600640 1 -- 172.21.15.47:0/1161379603 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff45810bb00 msgr2=0x7ff45810df80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:34:20.258 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:20.260+0000 7ff45d600640 1 --2- 172.21.15.47:0/1161379603 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff45810bb00 0x7ff45810df80 secure :-1 s=READY pgs=1223 cs=0 l=1 rev1=1 crypto rx=0x7ff450009ca0 tx=0x7ff45002c180 comp rx=0 tx=0).stop 2024-03-21T10:34:20.258 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:20.260+0000 7ff45d600640 1 -- 172.21.15.47:0/1161379603 shutdown_connections 2024-03-21T10:34:20.258 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:20.260+0000 7ff45d600640 1 --2- 172.21.15.47:0/1161379603 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff45810bb00 0x7ff45810df80 unknown :-1 s=CLOSED pgs=1223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:20.259 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:20.260+0000 7ff45d600640 1 --2- 172.21.15.47:0/1161379603 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff458103050 0x7ff45810b530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:20.259 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:20.260+0000 7ff45d600640 1 --2- 172.21.15.47:0/1161379603 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff458102730 0x7ff458102b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:20.259 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:20.260+0000 7ff45d600640 1 -- 172.21.15.47:0/1161379603 >> 172.21.15.47:0/1161379603 conn(0x7ff4580fbef0 msgr2=0x7ff4580fe310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:34:20.259 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:20.260+0000 7ff45d600640 1 -- 172.21.15.47:0/1161379603 shutdown_connections 2024-03-21T10:34:20.259 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:20.260+0000 7ff45d600640 1 -- 172.21.15.47:0/1161379603 wait complete. 2024-03-21T10:34:20.260 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:20.261+0000 7ff45d600640 1 Processor -- start 2024-03-21T10:34:20.260 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:20.261+0000 7ff45d600640 1 -- start start 2024-03-21T10:34:20.260 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:20.261+0000 7ff45d600640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff458102730 0x7ff458199e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:34:20.261 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:20.261+0000 7ff45d600640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff458103050 0x7ff45819a3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:34:20.261 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:20.261+0000 7ff45d600640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff45810bb00 0x7ff45819e750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:34:20.261 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:20.261+0000 7ff45d600640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff458111750 con 0x7ff45810bb00 2024-03-21T10:34:20.261 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:20.262+0000 7ff45d600640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4581115d0 con 0x7ff458102730 2024-03-21T10:34:20.261 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:20.262+0000 7ff45d600640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4581118d0 con 0x7ff458103050 2024-03-21T10:34:20.261 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:20.262+0000 7ff45cc00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff45810bb00 0x7ff45819e750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:34:20.261 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:20.262+0000 7ff45cc00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff45810bb00 0x7ff45819e750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:35696/0 (socket says 172.21.15.47:35696) 2024-03-21T10:34:20.261 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:20.262+0000 7ff45cc00640 1 -- 172.21.15.47:0/1678204680 learned_addr learned my addr 172.21.15.47:0/1678204680 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:34:20.261 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:20.262+0000 7ff457e00640 1 --2- 172.21.15.47:0/1678204680 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff458102730 0x7ff458199e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:34:20.261 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:20.262+0000 7ff44f400640 1 --2- 172.21.15.47:0/1678204680 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff458103050 0x7ff45819a3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:34:20.261 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:20.262+0000 7ff45cc00640 1 -- 172.21.15.47:0/1678204680 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff458103050 msgr2=0x7ff45819a3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:34:20.261 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:20.262+0000 7ff45cc00640 1 --2- 172.21.15.47:0/1678204680 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff458103050 0x7ff45819a3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:20.261 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:20.262+0000 7ff45cc00640 1 -- 172.21.15.47:0/1678204680 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff458102730 msgr2=0x7ff458199e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:34:20.261 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:20.262+0000 7ff45cc00640 1 --2- 172.21.15.47:0/1678204680 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff458102730 0x7ff458199e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:20.262 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:20.262+0000 7ff45cc00640 1 -- 172.21.15.47:0/1678204680 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff45819ee80 con 0x7ff45810bb00 2024-03-21T10:34:20.262 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:20.263+0000 7ff45cc00640 1 --2- 172.21.15.47:0/1678204680 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff45810bb00 0x7ff45819e750 secure :-1 s=READY pgs=1224 cs=0 l=1 rev1=1 crypto rx=0x7ff450037040 tx=0x7ff450003ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:34:20.262 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:20.263+0000 7ff455600640 1 -- 172.21.15.47:0/1678204680 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff450039040 con 0x7ff45810bb00 2024-03-21T10:34:20.263 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:20.263+0000 7ff455600640 1 -- 172.21.15.47:0/1678204680 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7ff450044070 con 0x7ff45810bb00 2024-03-21T10:34:20.263 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:20.263+0000 7ff45d600640 1 -- 172.21.15.47:0/1678204680 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff45819f110 con 0x7ff45810bb00 2024-03-21T10:34:20.263 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:20.263+0000 7ff455600640 1 -- 172.21.15.47:0/1678204680 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff450032850 con 0x7ff45810bb00 2024-03-21T10:34:20.263 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:20.263+0000 7ff45d600640 1 -- 172.21.15.47:0/1678204680 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff4581a6a00 con 0x7ff45810bb00 2024-03-21T10:34:20.265 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:20.267+0000 7ff4619f9740 1 Processor -- start 2024-03-21T10:34:20.266 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:20.268+0000 7ff455600640 1 -- 172.21.15.47:0/1678204680 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 27) v1 ==== 137985+0+0 (secure 0 0 0) 0x7ff4500329f0 con 0x7ff45810bb00 2024-03-21T10:34:20.266 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:20.268+0000 7ff4619f9740 1 -- start start 2024-03-21T10:34:20.267 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:20.269+0000 7ff4619f9740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55df26e488b0 0x55df26e48c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:34:20.267 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:20.269+0000 7ff4619f9740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55df26e49260 0x55df26e51810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:34:20.267 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:20.269+0000 7ff4619f9740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55df26e51de0 0x55df26e54230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:34:20.267 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:20.269+0000 7ff4619f9740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x55df26dac2c0 con 0x55df26e488b0 2024-03-21T10:34:20.267 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:20.269+0000 7ff4619f9740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55df26d54c90 con 0x55df26e49260 2024-03-21T10:34:20.268 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:20.269+0000 7ff4619f9740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55df26da8360 con 0x55df26e51de0 2024-03-21T10:34:20.268 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:20.269+0000 7ff44f400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55df26e488b0 0x55df26e48c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:34:20.268 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:20.269+0000 7ff44f400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55df26e488b0 0x55df26e48c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:35702/0 (socket says 172.21.15.47:35702) 2024-03-21T10:34:20.268 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:20.269+0000 7ff44f400640 1 -- 172.21.15.47:0/1539398267 learned_addr learned my addr 172.21.15.47:0/1539398267 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:34:20.268 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:20.269+0000 7ff457e00640 1 --2- 172.21.15.47:0/1539398267 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55df26e49260 0x55df26e51810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:34:20.269 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:20.270+0000 7ff44f400640 1 -- 172.21.15.47:0/1539398267 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55df26e51de0 msgr2=0x55df26e54230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:34:20.269 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:20.270+0000 7ff44f400640 1 --2- 172.21.15.47:0/1539398267 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55df26e51de0 0x55df26e54230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:20.269 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:20.270+0000 7ff44f400640 1 -- 172.21.15.47:0/1539398267 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55df26e49260 msgr2=0x55df26e51810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:34:20.269 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:20.270+0000 7ff44f400640 1 --2- 172.21.15.47:0/1539398267 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55df26e49260 0x55df26e51810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:20.269 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:20.270+0000 7ff44f400640 1 -- 172.21.15.47:0/1539398267 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55df26d79080 con 0x55df26e488b0 2024-03-21T10:34:20.269 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:20.270+0000 7ff44f400640 1 --2- 172.21.15.47:0/1539398267 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55df26e488b0 0x55df26e48c90 secure :-1 s=READY pgs=1225 cs=0 l=1 rev1=1 crypto rx=0x7ff440005e00 tx=0x7ff440011750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=77273d7305cb5eb5 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:34:20.269 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:20.270+0000 7ff455600640 1 --2- 172.21.15.47:0/1678204680 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7ff43c0a3090 0x7ff43c0a5550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:34:20.269 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:20.270+0000 7ff455600640 1 -- 172.21.15.47:0/1678204680 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 9562+0+0 (secure 0 0 0) 0x7ff4500f2090 con 0x7ff45810bb00 2024-03-21T10:34:20.269 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:20.271+0000 7ff44d200640 1 -- 172.21.15.47:0/1539398267 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff44001a400 con 0x55df26e488b0 2024-03-21T10:34:20.269 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:20.271+0000 7ff44d200640 1 -- 172.21.15.47:0/1539398267 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7ff44001aa50 con 0x55df26e488b0 2024-03-21T10:34:20.270 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:20.271+0000 7ff457e00640 1 --2- 172.21.15.47:0/1678204680 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7ff43c0a3090 0x7ff43c0a5550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:34:20.270 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:20.271+0000 7ff44d200640 1 -- 172.21.15.47:0/1539398267 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff440012ae0 con 0x55df26e488b0 2024-03-21T10:34:20.272 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:20.272+0000 7ff4619f9740 1 -- 172.21.15.47:0/1539398267 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55df26e488b0 msgr2=0x55df26e48c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:34:20.272 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:20.272+0000 7ff4619f9740 1 --2- 172.21.15.47:0/1539398267 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55df26e488b0 0x55df26e48c90 secure :-1 s=READY pgs=1225 cs=0 l=1 rev1=1 crypto rx=0x7ff440005e00 tx=0x7ff440011750 comp rx=0 tx=0).stop 2024-03-21T10:34:20.272 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:20.272+0000 7ff457e00640 1 --2- 172.21.15.47:0/1678204680 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7ff43c0a3090 0x7ff43c0a5550 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x55df26e43ff0 tx=0x7ff44800b390 comp rx=0 tx=0).ready entity=mgr.15687 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:34:20.272 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:20.272+0000 7ff4619f9740 1 -- 172.21.15.47:0/1539398267 shutdown_connections 2024-03-21T10:34:20.272 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:20.272+0000 7ff4619f9740 1 --2- 172.21.15.47:0/1539398267 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55df26e51de0 0x55df26e54230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:20.272 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:20.272+0000 7ff4619f9740 1 --2- 172.21.15.47:0/1539398267 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55df26e49260 0x55df26e51810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:20.272 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:20.272+0000 7ff4619f9740 1 --2- 172.21.15.47:0/1539398267 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55df26e488b0 0x55df26e48c90 unknown :-1 s=CLOSED pgs=1225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:20.272 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:20.272+0000 7ff4619f9740 1 -- 172.21.15.47:0/1539398267 >> 172.21.15.47:0/1539398267 conn(0x55df26e42f80 msgr2=0x55df26e43360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:34:20.272 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:20.272+0000 7ff4619f9740 1 -- 172.21.15.47:0/1539398267 shutdown_connections 2024-03-21T10:34:20.272 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:20.272+0000 7ff4619f9740 1 -- 172.21.15.47:0/1539398267 wait complete. 2024-03-21T10:34:20.272 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:20.272+0000 7ff4619f9740 1 -- start start 2024-03-21T10:34:20.273 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:20.273+0000 7ff4619f9740 1 Processor -- start 2024-03-21T10:34:20.273 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:20.273+0000 7ff4619f9740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55df26e488b0 0x55df26e516c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:34:20.273 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:20.273+0000 7ff4619f9740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55df26e49260 0x55df26e4ea50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:34:20.273 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:20.273+0000 7ff4619f9740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55df26e51de0 0x55df26e4efb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:34:20.273 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:20.273+0000 7ff4619f9740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x55df26e54d50 con 0x55df26e49260 2024-03-21T10:34:20.273 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:20.273+0000 7ff4619f9740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55df26d680d0 con 0x55df26e488b0 2024-03-21T10:34:20.273 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:20.273+0000 7ff4619f9740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55df26e54ed0 con 0x55df26e51de0 2024-03-21T10:34:20.273 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:20.273+0000 7ff4619f9740 10 client.0 _ll_register_callbacks cb 0x55df26d6ce80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T10:34:20.273 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:20.273+0000 7ff44f400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55df26e49260 0x55df26e4ea50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:34:20.273 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:20.273+0000 7ff44f400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55df26e49260 0x55df26e4ea50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:35718/0 (socket says 172.21.15.47:35718) 2024-03-21T10:34:20.273 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:20.273+0000 7ff44f400640 1 -- 172.21.15.47:0/3795474341 learned_addr learned my addr 172.21.15.47:0/3795474341 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:34:20.273 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:20.273+0000 7ff457e00640 1 --2- 172.21.15.47:0/3795474341 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55df26e488b0 0x55df26e516c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:34:20.273 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:20.274+0000 7ff45cc00640 1 --2- 172.21.15.47:0/3795474341 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55df26e51de0 0x55df26e4efb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:34:20.273 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:20.274+0000 7ff44f400640 1 -- 172.21.15.47:0/3795474341 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55df26e51de0 msgr2=0x55df26e4efb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:34:20.273 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:20.274+0000 7ff44f400640 1 --2- 172.21.15.47:0/3795474341 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55df26e51de0 0x55df26e4efb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:20.273 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:20.274+0000 7ff44f400640 1 -- 172.21.15.47:0/3795474341 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55df26e488b0 msgr2=0x55df26e516c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:34:20.274 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:20.274+0000 7ff44f400640 1 --2- 172.21.15.47:0/3795474341 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55df26e488b0 0x55df26e516c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:20.274 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:20.274+0000 7ff44f400640 1 -- 172.21.15.47:0/3795474341 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55df26ac03f0 con 0x55df26e49260 2024-03-21T10:34:20.275 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:20.274+0000 7ff44f400640 1 --2- 172.21.15.47:0/3795474341 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55df26e49260 0x55df26e4ea50 secure :-1 s=READY pgs=1226 cs=0 l=1 rev1=1 crypto rx=0x7ff4400126c0 tx=0x7ff44000da30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=25d609bc76daf019 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:34:20.276 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:20.274+0000 7ff445600640 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-03-21T10:34:20.276 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:20.275+0000 7ff445600640 1 -- 172.21.15.47:0/3795474341 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff440029060 con 0x55df26e49260 2024-03-21T10:34:20.276 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:20.275+0000 7ff445600640 1 -- 172.21.15.47:0/3795474341 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(103..103 src has 1..103) v4 ==== 9562+0+0 (secure 0 0 0) 0x7ff44001a600 con 0x55df26e49260 2024-03-21T10:34:20.276 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:20.275+0000 7ff4619f9740 1 -- 172.21.15.47:0/3795474341 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ff44001aa50 con 0x55df26e49260 2024-03-21T10:34:20.276 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:20.275+0000 7ff445600640 1 -- 172.21.15.47:0/3795474341 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff440030050 con 0x55df26e49260 2024-03-21T10:34:20.276 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:20.275+0000 7ff4619f9740 10 client.18501 fetch_fsmap learned FSMap version 32 2024-03-21T10:34:20.276 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:20.275+0000 7ff4619f9740 1 -- 172.21.15.47:0/3795474341 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x55df26b8a7d0 con 0x55df26e49260 2024-03-21T10:34:20.276 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:20.275+0000 7ff445600640 1 -- 172.21.15.47:0/3795474341 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7ff44001aa50 con 0x55df26e49260 2024-03-21T10:34:20.276 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:20.276+0000 7ff445600640 1 -- 172.21.15.47:0/3795474341 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 32) v1 ==== 7942+0+0 (secure 0 0 0) 0x7ff440012b90 con 0x55df26e49260 2024-03-21T10:34:20.276 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:20.276+0000 7ff4619f9740 10 client.18501 fetch_fsmap finished waiting for FSMap version 32 2024-03-21T10:34:20.276 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:20.276+0000 7ff4619f9740 10 client.18501 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-03-21T10:34:20.276 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:20.276+0000 7ff4619f9740 20 client.18501 populate_metadata read hostname 'smithi047' 2024-03-21T10:34:20.276 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:20.276+0000 7ff4619f9740 1 --2- 172.21.15.47:0/3795474341 >> [v2:172.21.15.100:6836/3497644162,v1:172.21.15.100:6837/3497644162] conn(0x55df26e61b10 0x55df26e63f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:34:20.276 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:20.276+0000 7ff4619f9740 4 client.18501 mds_command: new command op to 34493 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T10:34:20.277 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:20.276+0000 7ff4619f9740 1 -- 172.21.15.47:0/3795474341 --> [v2:172.21.15.100:6836/3497644162,v1:172.21.15.100:6837/3497644162] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55df26db9800 con 0x55df26e61b10 2024-03-21T10:34:20.277 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:20.276+0000 7ff457e00640 1 --2- 172.21.15.47:0/3795474341 >> [v2:172.21.15.100:6836/3497644162,v1:172.21.15.100:6837/3497644162] conn(0x55df26e61b10 0x55df26e63f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:34:20.277 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:20.277+0000 7ff457e00640 1 --2- 172.21.15.47:0/3795474341 >> [v2:172.21.15.100:6836/3497644162,v1:172.21.15.100:6837/3497644162] conn(0x55df26e61b10 0x55df26e63f00 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=cfe7e61143c6db4f server_cookie=9c4142668fda5299 in_seq=0 out_seq=0 2024-03-21T10:34:20.277 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:20.277+0000 7ff445600640 10 client.18501 ms_handle_connect on v2:172.21.15.100:6836/3497644162 2024-03-21T10:34:20.278 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:20.278+0000 7ff445600640 1 -- 172.21.15.47:0/3795474341 <== mds.1 v2:172.21.15.100:6836/3497644162 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55df26db9800 con 0x55df26e61b10 2024-03-21T10:34:20.278 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:20.278+0000 7ff445600640 10 client.18501 handle_command_reply: tid=0 2024-03-21T10:34:20.278 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:20.278+0000 7ff4619f9740 1 client.18501 shutdown 2024-03-21T10:34:20.278 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:20.279+0000 7ff4619f9740 1 -- 172.21.15.47:0/3795474341 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55df26e49260 msgr2=0x55df26e4ea50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:34:20.278 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:20.279+0000 7ff4619f9740 1 --2- 172.21.15.47:0/3795474341 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55df26e49260 0x55df26e4ea50 secure :-1 s=READY pgs=1226 cs=0 l=1 rev1=1 crypto rx=0x7ff4400126c0 tx=0x7ff44000da30 comp rx=0 tx=0).stop 2024-03-21T10:34:20.278 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:20.279+0000 7ff4619f9740 1 -- 172.21.15.47:0/3795474341 shutdown_connections 2024-03-21T10:34:20.278 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:20.279+0000 7ff4619f9740 1 --2- 172.21.15.47:0/3795474341 >> [v2:172.21.15.100:6836/3497644162,v1:172.21.15.100:6837/3497644162] conn(0x55df26e61b10 0x55df26e63f00 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:20.278 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:20.279+0000 7ff4619f9740 1 --2- 172.21.15.47:0/3795474341 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55df26e51de0 0x55df26e4efb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:20.278 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:20.279+0000 7ff4619f9740 1 --2- 172.21.15.47:0/3795474341 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55df26e49260 0x55df26e4ea50 unknown :-1 s=CLOSED pgs=1226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:20.278 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:20.279+0000 7ff4619f9740 1 --2- 172.21.15.47:0/3795474341 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55df26e488b0 0x55df26e516c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:20.279 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:20.279+0000 7ff4619f9740 1 -- 172.21.15.47:0/3795474341 >> 172.21.15.47:0/3795474341 conn(0x55df26e42f80 msgr2=0x55df26e44cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:34:20.279 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:20.279+0000 7ff445600640 0 client.18501 ms_handle_reset on v2:172.21.15.100:6836/3497644162 2024-03-21T10:34:20.279 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:20.279+0000 7ff4619f9740 1 -- 172.21.15.47:0/3795474341 shutdown_connections 2024-03-21T10:34:20.279 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:20.279+0000 7ff4619f9740 1 -- 172.21.15.47:0/3795474341 wait complete. 2024-03-21T10:34:20.279 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:20.279+0000 7ff4619f9740 20 client.18501 trim_cache size 0 max 16384 2024-03-21T10:34:20.291 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:20.293+0000 7ff4619f9740 1 Processor -- start 2024-03-21T10:34:20.291 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:20.293+0000 7ff4619f9740 1 -- start start 2024-03-21T10:34:20.291 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:20.293+0000 7ff4619f9740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55df26e488b0 0x55df26e41fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:34:20.292 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:20.293+0000 7ff4619f9740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55df26e49260 0x55df26e51110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:34:20.292 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:20.293+0000 7ff4619f9740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55df26e51de0 0x55df26e51650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:34:20.292 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:20.293+0000 7ff4619f9740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4581b09b0 con 0x55df26e49260 2024-03-21T10:34:20.292 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:20.293+0000 7ff4619f9740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4581b2330 con 0x55df26e488b0 2024-03-21T10:34:20.292 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:20.293+0000 7ff4619f9740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4581b1010 con 0x55df26e51de0 2024-03-21T10:34:20.293 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:20.294+0000 7ff44f400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55df26e49260 0x55df26e51110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:34:20.293 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:20.294+0000 7ff457e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55df26e488b0 0x55df26e41fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:34:20.293 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:20.294+0000 7ff44f400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55df26e49260 0x55df26e51110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:35728/0 (socket says 172.21.15.47:35728) 2024-03-21T10:34:20.293 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:20.294+0000 7ff457e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55df26e488b0 0x55df26e41fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:50348/0 (socket says 172.21.15.47:50348) 2024-03-21T10:34:20.293 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:20.294+0000 7ff44f400640 1 -- 172.21.15.47:0/3775777038 learned_addr learned my addr 172.21.15.47:0/3775777038 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:34:20.293 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:20.294+0000 7ff45cc00640 1 --2- 172.21.15.47:0/3775777038 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55df26e51de0 0x55df26e51650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:34:20.293 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:20.294+0000 7ff44f400640 1 -- 172.21.15.47:0/3775777038 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55df26e51de0 msgr2=0x55df26e51650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:34:20.293 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:20.294+0000 7ff44f400640 1 --2- 172.21.15.47:0/3775777038 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55df26e51de0 0x55df26e51650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:20.293 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:20.294+0000 7ff44f400640 1 -- 172.21.15.47:0/3775777038 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55df26e488b0 msgr2=0x55df26e41fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:34:20.293 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:20.294+0000 7ff44f400640 1 --2- 172.21.15.47:0/3775777038 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55df26e488b0 0x55df26e41fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:20.293 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:20.294+0000 7ff44f400640 1 -- 172.21.15.47:0/3775777038 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55df26e42520 con 0x55df26e49260 2024-03-21T10:34:20.294 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:20.295+0000 7ff44f400640 1 --2- 172.21.15.47:0/3775777038 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55df26e49260 0x55df26e51110 secure :-1 s=READY pgs=1227 cs=0 l=1 rev1=1 crypto rx=0x7ff44001d040 tx=0x7ff440004a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9e855f1c123b81f server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:34:20.294 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:20.295+0000 7ff447e00640 1 -- 172.21.15.47:0/3775777038 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff440029040 con 0x55df26e49260 2024-03-21T10:34:20.294 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:20.295+0000 7ff447e00640 1 -- 172.21.15.47:0/3775777038 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7ff440004de0 con 0x55df26e49260 2024-03-21T10:34:20.294 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:20.295+0000 7ff447e00640 1 -- 172.21.15.47:0/3775777038 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff44001a660 con 0x55df26e49260 2024-03-21T10:34:20.296 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:20.295+0000 7ff4619f9740 1 -- 172.21.15.47:0/3775777038 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55df26e49260 msgr2=0x55df26e51110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:34:20.296 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:20.295+0000 7ff4619f9740 1 --2- 172.21.15.47:0/3775777038 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55df26e49260 0x55df26e51110 secure :-1 s=READY pgs=1227 cs=0 l=1 rev1=1 crypto rx=0x7ff44001d040 tx=0x7ff440004a20 comp rx=0 tx=0).stop 2024-03-21T10:34:20.296 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:20.296+0000 7ff4619f9740 1 -- 172.21.15.47:0/3775777038 shutdown_connections 2024-03-21T10:34:20.296 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:20.296+0000 7ff4619f9740 1 --2- 172.21.15.47:0/3775777038 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55df26e51de0 0x55df26e51650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:20.296 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:20.296+0000 7ff4619f9740 1 --2- 172.21.15.47:0/3775777038 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55df26e49260 0x55df26e51110 unknown :-1 s=CLOSED pgs=1227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:20.296 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:20.296+0000 7ff4619f9740 1 --2- 172.21.15.47:0/3775777038 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55df26e488b0 0x55df26e41fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:20.296 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:20.296+0000 7ff4619f9740 1 -- 172.21.15.47:0/3775777038 >> 172.21.15.47:0/3775777038 conn(0x55df26e42f80 msgr2=0x55df26e418d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:34:20.296 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:20.296+0000 7ff4619f9740 1 -- 172.21.15.47:0/3775777038 shutdown_connections 2024-03-21T10:34:20.296 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:20.296+0000 7ff4619f9740 1 -- 172.21.15.47:0/3775777038 wait complete. 2024-03-21T10:34:20.296 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:20.296+0000 7ff4619f9740 1 -- start start 2024-03-21T10:34:20.296 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:20.296+0000 7ff4619f9740 1 Processor -- start 2024-03-21T10:34:20.297 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:20.297+0000 7ff4619f9740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55df26e488b0 0x55df26e55c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:34:20.297 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:20.297+0000 7ff4619f9740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55df26e49260 0x55df26e56190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:34:20.297 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:20.297+0000 7ff4619f9740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55df26e51de0 0x55df26ebc180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:34:20.297 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:20.297+0000 7ff4619f9740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4581b1cd0 con 0x55df26e49260 2024-03-21T10:34:20.297 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:20.297+0000 7ff4619f9740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4581b1670 con 0x55df26e51de0 2024-03-21T10:34:20.297 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:20.297+0000 7ff4619f9740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4581aeb90 con 0x55df26e488b0 2024-03-21T10:34:20.297 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:20.297+0000 7ff4619f9740 10 client.0 _ll_register_callbacks cb 0x7ff44800b770 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T10:34:20.297 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:20.297+0000 7ff44f400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55df26e49260 0x55df26e56190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:34:20.297 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:20.297+0000 7ff44f400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55df26e49260 0x55df26e56190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:35744/0 (socket says 172.21.15.47:35744) 2024-03-21T10:34:20.297 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:20.297+0000 7ff44f400640 1 -- 172.21.15.47:0/525946808 learned_addr learned my addr 172.21.15.47:0/525946808 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:34:20.297 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:20.297+0000 7ff45cc00640 1 --2- 172.21.15.47:0/525946808 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55df26e51de0 0x55df26ebc180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:34:20.297 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:20.297+0000 7ff457e00640 1 --2- 172.21.15.47:0/525946808 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55df26e488b0 0x55df26e55c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:34:20.297 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:20.298+0000 7ff44f400640 1 -- 172.21.15.47:0/525946808 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55df26e488b0 msgr2=0x55df26e55c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:34:20.297 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:20.298+0000 7ff44f400640 1 --2- 172.21.15.47:0/525946808 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55df26e488b0 0x55df26e55c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:20.297 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:20.298+0000 7ff44f400640 1 -- 172.21.15.47:0/525946808 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55df26e51de0 msgr2=0x55df26ebc180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:34:20.298 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:20.298+0000 7ff44f400640 1 --2- 172.21.15.47:0/525946808 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55df26e51de0 0x55df26ebc180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:20.298 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:20.298+0000 7ff44f400640 1 -- 172.21.15.47:0/525946808 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55df26ebc6c0 con 0x55df26e49260 2024-03-21T10:34:20.298 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:20.298+0000 7ff44f400640 1 --2- 172.21.15.47:0/525946808 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55df26e49260 0x55df26e56190 secure :-1 s=READY pgs=1228 cs=0 l=1 rev1=1 crypto rx=0x7ff440012570 tx=0x7ff440004a90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=948a93ab59a1d5fc server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:34:20.298 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:20.298+0000 7ff445600640 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-03-21T10:34:20.300 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:20.298+0000 7ff445600640 1 -- 172.21.15.47:0/525946808 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff440029040 con 0x55df26e49260 2024-03-21T10:34:20.300 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:20.298+0000 7ff4619f9740 1 -- 172.21.15.47:0/525946808 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ff440004de0 con 0x55df26e49260 2024-03-21T10:34:20.300 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:20.298+0000 7ff445600640 1 -- 172.21.15.47:0/525946808 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(103..103 src has 1..103) v4 ==== 9562+0+0 (secure 0 0 0) 0x7ff440004be0 con 0x55df26e49260 2024-03-21T10:34:20.300 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:20.299+0000 7ff445600640 1 -- 172.21.15.47:0/525946808 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff440005b90 con 0x55df26e49260 2024-03-21T10:34:20.300 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:20.299+0000 7ff4619f9740 10 client.18507 fetch_fsmap learned FSMap version 32 2024-03-21T10:34:20.300 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:20.299+0000 7ff4619f9740 1 -- 172.21.15.47:0/525946808 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x55df26ebd970 con 0x55df26e49260 2024-03-21T10:34:20.300 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:20.299+0000 7ff445600640 1 -- 172.21.15.47:0/525946808 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7ff440004de0 con 0x55df26e49260 2024-03-21T10:34:20.300 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:20.299+0000 7ff445600640 1 -- 172.21.15.47:0/525946808 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 32) v1 ==== 7942+0+0 (secure 0 0 0) 0x7ff44001a890 con 0x55df26e49260 2024-03-21T10:34:20.300 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:20.299+0000 7ff4619f9740 10 client.18507 fetch_fsmap finished waiting for FSMap version 32 2024-03-21T10:34:20.300 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:20.299+0000 7ff4619f9740 10 client.18507 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-03-21T10:34:20.300 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:20.299+0000 7ff4619f9740 20 client.18507 populate_metadata read hostname 'smithi047' 2024-03-21T10:34:20.300 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:20.299+0000 7ff4619f9740 1 --2- 172.21.15.47:0/525946808 >> [v2:172.21.15.100:6836/3497644162,v1:172.21.15.100:6837/3497644162] conn(0x55df26ebe030 0x55df26ec0420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:34:20.301 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:20.299+0000 7ff4619f9740 4 client.18507 mds_command: new command op to 34493 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-21T10:34:20.301 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:20.299+0000 7ff4619f9740 1 -- 172.21.15.47:0/525946808 --> [v2:172.21.15.100:6836/3497644162,v1:172.21.15.100:6837/3497644162] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55df26db3b50 con 0x55df26ebe030 2024-03-21T10:34:20.301 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:20.300+0000 7ff457e00640 1 --2- 172.21.15.47:0/525946808 >> [v2:172.21.15.100:6836/3497644162,v1:172.21.15.100:6837/3497644162] conn(0x55df26ebe030 0x55df26ec0420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:34:20.301 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:20.300+0000 7ff457e00640 1 --2- 172.21.15.47:0/525946808 >> [v2:172.21.15.100:6836/3497644162,v1:172.21.15.100:6837/3497644162] conn(0x55df26ebe030 0x55df26ec0420 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=2f27c7227c66eb24 server_cookie=ac23845f6f03dfde in_seq=0 out_seq=0 2024-03-21T10:34:20.301 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:20.300+0000 7ff445600640 10 client.18507 ms_handle_connect on v2:172.21.15.100:6836/3497644162 2024-03-21T10:34:20.301 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:20.301+0000 7ff445600640 1 -- 172.21.15.47:0/525946808 <== mds.1 v2:172.21.15.100:6836/3497644162 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55df26db3b50 con 0x55df26ebe030 2024-03-21T10:34:20.301 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:20.301+0000 7ff445600640 10 client.18507 handle_command_reply: tid=0 2024-03-21T10:34:20.301 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:20.301+0000 7ff4619f9740 1 client.18507 shutdown 2024-03-21T10:34:20.301 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:20.301+0000 7ff4619f9740 1 -- 172.21.15.47:0/525946808 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55df26e49260 msgr2=0x55df26e56190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:34:20.301 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:20.301+0000 7ff4619f9740 1 --2- 172.21.15.47:0/525946808 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55df26e49260 0x55df26e56190 secure :-1 s=READY pgs=1228 cs=0 l=1 rev1=1 crypto rx=0x7ff440012570 tx=0x7ff440004a90 comp rx=0 tx=0).stop 2024-03-21T10:34:20.301 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:20.301+0000 7ff4619f9740 1 -- 172.21.15.47:0/525946808 shutdown_connections 2024-03-21T10:34:20.301 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:20.301+0000 7ff4619f9740 1 --2- 172.21.15.47:0/525946808 >> [v2:172.21.15.100:6836/3497644162,v1:172.21.15.100:6837/3497644162] conn(0x55df26ebe030 0x55df26ec0420 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:20.301 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:20.301+0000 7ff4619f9740 1 --2- 172.21.15.47:0/525946808 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55df26e51de0 0x55df26ebc180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:20.301 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:20.301+0000 7ff4619f9740 1 --2- 172.21.15.47:0/525946808 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55df26e49260 0x55df26e56190 unknown :-1 s=CLOSED pgs=1228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:20.301 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:20.301+0000 7ff4619f9740 1 --2- 172.21.15.47:0/525946808 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55df26e488b0 0x55df26e55c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:20.302 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:20.301+0000 7ff445600640 0 client.18507 ms_handle_reset on v2:172.21.15.100:6836/3497644162 2024-03-21T10:34:20.302 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:20.301+0000 7ff4619f9740 1 -- 172.21.15.47:0/525946808 >> 172.21.15.47:0/525946808 conn(0x55df26e42f80 msgr2=0x55df26e44280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:34:20.302 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:20.301+0000 7ff4619f9740 1 -- 172.21.15.47:0/525946808 shutdown_connections 2024-03-21T10:34:20.302 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:20.301+0000 7ff4619f9740 1 -- 172.21.15.47:0/525946808 wait complete. 2024-03-21T10:34:20.302 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:20.301+0000 7ff4619f9740 20 client.18507 trim_cache size 0 max 16384 2024-03-21T10:34:20.302 INFO:teuthology.orchestra.run.smithi047.stdout:[] 2024-03-21T10:34:20.303 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:20.302+0000 7ff45d600640 1 -- 172.21.15.47:0/1678204680 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7ff43c0a3090 msgr2=0x7ff43c0a5550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:34:20.303 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:20.302+0000 7ff45d600640 1 --2- 172.21.15.47:0/1678204680 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7ff43c0a3090 0x7ff43c0a5550 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x55df26e43ff0 tx=0x7ff44800b390 comp rx=0 tx=0).stop 2024-03-21T10:34:20.303 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:20.302+0000 7ff45d600640 1 -- 172.21.15.47:0/1678204680 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff45810bb00 msgr2=0x7ff45819e750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:34:20.303 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:20.302+0000 7ff45d600640 1 --2- 172.21.15.47:0/1678204680 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff45810bb00 0x7ff45819e750 secure :-1 s=READY pgs=1224 cs=0 l=1 rev1=1 crypto rx=0x7ff450037040 tx=0x7ff450003ba0 comp rx=0 tx=0).stop 2024-03-21T10:34:20.303 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:20.302+0000 7ff45d600640 1 -- 172.21.15.47:0/1678204680 shutdown_connections 2024-03-21T10:34:20.303 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:20.302+0000 7ff45d600640 1 --2- 172.21.15.47:0/1678204680 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff45810bb00 0x7ff45819e750 unknown :-1 s=CLOSED pgs=1224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:20.303 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:20.302+0000 7ff45d600640 1 --2- 172.21.15.47:0/1678204680 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff458103050 0x7ff45819a3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:20.303 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:20.302+0000 7ff45d600640 1 --2- 172.21.15.47:0/1678204680 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7ff43c0a3090 0x7ff43c0a5550 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:20.303 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:20.302+0000 7ff45d600640 1 --2- 172.21.15.47:0/1678204680 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff458102730 0x7ff458199e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:20.303 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:20.302+0000 7ff45d600640 1 -- 172.21.15.47:0/1678204680 >> 172.21.15.47:0/1678204680 conn(0x7ff4580fbef0 msgr2=0x7ff45810cd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:34:20.303 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:20.302+0000 7ff45d600640 1 -- 172.21.15.47:0/1678204680 shutdown_connections 2024-03-21T10:34:20.304 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:20.302+0000 7ff45d600640 1 -- 172.21.15.47:0/1678204680 wait complete. 2024-03-21T10:34:20.319 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-21T10:34:20.319 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-03-21T10:34:20.534 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:20.535+0000 7f309b400640 1 -- 172.21.15.47:0/626553064 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3094068880 msgr2=0x7f3094068d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:34:20.534 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:20.535+0000 7f309b400640 1 --2- 172.21.15.47:0/626553064 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3094068880 0x7f3094068d00 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f3080009900 tx=0x7f308002bc30 comp rx=0 tx=0).stop 2024-03-21T10:34:20.535 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:20.536+0000 7f309b400640 1 -- 172.21.15.47:0/626553064 shutdown_connections 2024-03-21T10:34:20.535 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:20.536+0000 7f309b400640 1 --2- 172.21.15.47:0/626553064 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3094069240 0x7f309410e4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:20.535 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:20.536+0000 7f309b400640 1 --2- 172.21.15.47:0/626553064 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3094068880 0x7f3094068d00 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:20.535 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:20.536+0000 7f309b400640 1 --2- 172.21.15.47:0/626553064 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f30941076b0 0x7f3094107a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:20.535 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:20.536+0000 7f309b400640 1 -- 172.21.15.47:0/626553064 >> 172.21.15.47:0/626553064 conn(0x7f30940776a0 msgr2=0x7f3094077ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:34:20.535 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:20.536+0000 7f309b400640 1 -- 172.21.15.47:0/626553064 shutdown_connections 2024-03-21T10:34:20.535 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:20.536+0000 7f309b400640 1 -- 172.21.15.47:0/626553064 wait complete. 2024-03-21T10:34:20.537 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:20.537+0000 7f309b400640 1 Processor -- start 2024-03-21T10:34:20.537 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:20.537+0000 7f309b400640 1 -- start start 2024-03-21T10:34:20.537 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:20.537+0000 7f309b400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3094068880 0x7f3094197e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:34:20.537 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:20.537+0000 7f309b400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3094069240 0x7f3094198340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:34:20.537 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:20.537+0000 7f309b400640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f30941076b0 0x7f309419c670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:34:20.537 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:20.537+0000 7f309b400640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30941118a0 con 0x7f3094069240 2024-03-21T10:34:20.537 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:20.537+0000 7f309b400640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3094111720 con 0x7f3094068880 2024-03-21T10:34:20.538 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:20.537+0000 7f309b400640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3094111a20 con 0x7f30941076b0 2024-03-21T10:34:20.538 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:20.538+0000 7f3098a00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3094068880 0x7f3094197e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:34:20.538 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:20.538+0000 7f3093e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3094069240 0x7f3094198340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:34:20.538 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:20.538+0000 7f3098a00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3094068880 0x7f3094197e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:50378/0 (socket says 172.21.15.47:50378) 2024-03-21T10:34:20.538 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:20.538+0000 7f3098a00640 1 -- 172.21.15.47:0/515293122 learned_addr learned my addr 172.21.15.47:0/515293122 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:34:20.538 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:20.538+0000 7f3099400640 1 --2- 172.21.15.47:0/515293122 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f30941076b0 0x7f309419c670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:34:20.538 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:20.538+0000 7f3098a00640 1 -- 172.21.15.47:0/515293122 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f30941076b0 msgr2=0x7f309419c670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:34:20.538 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:20.538+0000 7f3098a00640 1 --2- 172.21.15.47:0/515293122 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f30941076b0 0x7f309419c670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:20.538 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:20.538+0000 7f3098a00640 1 -- 172.21.15.47:0/515293122 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3094069240 msgr2=0x7f3094198340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:34:20.538 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:20.538+0000 7f3098a00640 1 --2- 172.21.15.47:0/515293122 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3094069240 0x7f3094198340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:20.538 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:20.538+0000 7f3098a00640 1 -- 172.21.15.47:0/515293122 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f309419cda0 con 0x7f3094068880 2024-03-21T10:34:20.538 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:20.538+0000 7f3099400640 1 --2- 172.21.15.47:0/515293122 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f30941076b0 0x7f309419c670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T10:34:20.538 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:20.538+0000 7f3093e00640 1 --2- 172.21.15.47:0/515293122 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3094069240 0x7f3094198340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T10:34:20.539 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:20.539+0000 7f3098a00640 1 --2- 172.21.15.47:0/515293122 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3094068880 0x7f3094197e00 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f308800e9c0 tx=0x7f308800ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:34:20.539 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:20.539+0000 7f3091600640 1 -- 172.21.15.47:0/515293122 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f308800cd80 con 0x7f3094068880 2024-03-21T10:34:20.539 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:20.539+0000 7f309b400640 1 -- 172.21.15.47:0/515293122 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f309419d090 con 0x7f3094068880 2024-03-21T10:34:20.539 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:20.539+0000 7f3091600640 1 -- 172.21.15.47:0/515293122 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f30880035f0 con 0x7f3094068880 2024-03-21T10:34:20.539 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:20.539+0000 7f3091600640 1 -- 172.21.15.47:0/515293122 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3088010690 con 0x7f3094068880 2024-03-21T10:34:20.539 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:20.539+0000 7f309b400640 1 -- 172.21.15.47:0/515293122 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f30941a4920 con 0x7f3094068880 2024-03-21T10:34:20.540 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:20.542+0000 7f309e17d740 1 Processor -- start 2024-03-21T10:34:20.540 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:20.543+0000 7f309e17d740 1 -- start start 2024-03-21T10:34:20.541 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:20.543+0000 7f309e17d740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ee066826d0 0x55ee06682ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:34:20.542 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:20.543+0000 7f309e17d740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ee06683080 0x55ee0668b630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:34:20.542 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:20.543+0000 7f309e17d740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ee0668bc00 0x55ee0668e050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:34:20.542 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:20.543+0000 7f309e17d740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ee065e62c0 con 0x55ee0668bc00 2024-03-21T10:34:20.542 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:20.543+0000 7f309e17d740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ee0658ec90 con 0x55ee066826d0 2024-03-21T10:34:20.542 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:20.543+0000 7f309e17d740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ee065e2360 con 0x55ee06683080 2024-03-21T10:34:20.542 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:20.543+0000 7f3093e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ee0668bc00 0x55ee0668e050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:34:20.542 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:20.543+0000 7f3091600640 1 -- 172.21.15.47:0/515293122 <== mon.2 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 27) v1 ==== 137985+0+0 (secure 0 0 0) 0x7f30880037a0 con 0x7f3094068880 2024-03-21T10:34:20.542 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:20.543+0000 7f3093e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ee0668bc00 0x55ee0668e050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:35772/0 (socket says 172.21.15.47:35772) 2024-03-21T10:34:20.542 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:20.543+0000 7f3093e00640 1 -- 172.21.15.47:0/2415134980 learned_addr learned my addr 172.21.15.47:0/2415134980 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:34:20.542 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:20.544+0000 7f3099400640 1 --2- 172.21.15.47:0/2415134980 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ee066826d0 0x55ee06682ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:34:20.543 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:20.544+0000 7f3093e00640 1 -- 172.21.15.47:0/2415134980 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ee06683080 msgr2=0x55ee0668b630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:34:20.543 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:20.544+0000 7f3093e00640 1 --2- 172.21.15.47:0/2415134980 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ee06683080 0x55ee0668b630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:20.543 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:20.544+0000 7f3093e00640 1 -- 172.21.15.47:0/2415134980 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ee066826d0 msgr2=0x55ee06682ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:34:20.543 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:20.544+0000 7f3093e00640 1 --2- 172.21.15.47:0/2415134980 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ee066826d0 0x55ee06682ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:20.544 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:20.544+0000 7f3093e00640 1 -- 172.21.15.47:0/2415134980 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ee065b3080 con 0x55ee0668bc00 2024-03-21T10:34:20.544 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:20.545+0000 7f3093e00640 1 --2- 172.21.15.47:0/2415134980 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ee0668bc00 0x55ee0668e050 secure :-1 s=READY pgs=1229 cs=0 l=1 rev1=1 crypto rx=0x7f3080009900 tx=0x7f3080009560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3578718916eaa6bf server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:34:20.544 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:20.545+0000 7f3085c00640 1 -- 172.21.15.47:0/2415134980 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f308002c800 con 0x55ee0668bc00 2024-03-21T10:34:20.544 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:20.545+0000 7f3085c00640 1 -- 172.21.15.47:0/2415134980 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f308002c9a0 con 0x55ee0668bc00 2024-03-21T10:34:20.544 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:20.545+0000 7f3085c00640 1 -- 172.21.15.47:0/2415134980 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3080003e00 con 0x55ee0668bc00 2024-03-21T10:34:20.544 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:20.545+0000 7f3091600640 1 --2- 172.21.15.47:0/515293122 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f307c0a3090 0x7f307c0a5550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:34:20.545 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:20.546+0000 7f3091600640 1 -- 172.21.15.47:0/515293122 <== mon.2 v2:172.21.15.100:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 9562+0+0 (secure 0 0 0) 0x7f30880d0090 con 0x7f3094068880 2024-03-21T10:34:20.545 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:20.546+0000 7f3099400640 1 --2- 172.21.15.47:0/515293122 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f307c0a3090 0x7f307c0a5550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:34:20.545 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:20.546+0000 7f309e17d740 1 -- 172.21.15.47:0/2415134980 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ee0668bc00 msgr2=0x55ee0668e050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:34:20.545 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:20.546+0000 7f309e17d740 1 --2- 172.21.15.47:0/2415134980 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ee0668bc00 0x55ee0668e050 secure :-1 s=READY pgs=1229 cs=0 l=1 rev1=1 crypto rx=0x7f3080009900 tx=0x7f3080009560 comp rx=0 tx=0).stop 2024-03-21T10:34:20.545 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:20.546+0000 7f309e17d740 1 -- 172.21.15.47:0/2415134980 shutdown_connections 2024-03-21T10:34:20.545 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:20.546+0000 7f309e17d740 1 --2- 172.21.15.47:0/2415134980 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ee0668bc00 0x55ee0668e050 unknown :-1 s=CLOSED pgs=1229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:20.545 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:20.546+0000 7f309e17d740 1 --2- 172.21.15.47:0/2415134980 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ee06683080 0x55ee0668b630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:20.546 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:20.546+0000 7f309e17d740 1 --2- 172.21.15.47:0/2415134980 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ee066826d0 0x55ee06682ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:20.546 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:20.546+0000 7f309e17d740 1 -- 172.21.15.47:0/2415134980 >> 172.21.15.47:0/2415134980 conn(0x55ee0667cda0 msgr2=0x55ee0667d180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:34:20.546 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:20.547+0000 7f309e17d740 1 -- 172.21.15.47:0/2415134980 shutdown_connections 2024-03-21T10:34:20.546 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:20.547+0000 7f3099400640 1 --2- 172.21.15.47:0/515293122 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f307c0a3090 0x7f307c0a5550 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f308c005fd0 tx=0x7f308c009700 comp rx=0 tx=0).ready entity=mgr.15687 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:34:20.546 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:20.547+0000 7f309e17d740 1 -- 172.21.15.47:0/2415134980 wait complete. 2024-03-21T10:34:20.546 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:20.547+0000 7f309e17d740 1 -- start start 2024-03-21T10:34:20.549 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:20.547+0000 7f309e17d740 1 Processor -- start 2024-03-21T10:34:20.549 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:20.548+0000 7f309e17d740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ee066826d0 0x55ee0668b5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:34:20.549 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:20.548+0000 7f309e17d740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ee06683080 0x55ee06688930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:34:20.549 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:20.548+0000 7f309e17d740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ee0668bc00 0x55ee06689070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:34:20.549 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:20.548+0000 7f309e17d740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ee0668eb10 con 0x55ee066826d0 2024-03-21T10:34:20.549 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:20.548+0000 7f309e17d740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ee065a20d0 con 0x55ee0668bc00 2024-03-21T10:34:20.549 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:20.548+0000 7f309e17d740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ee0668ec90 con 0x55ee06683080 2024-03-21T10:34:20.549 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:20.548+0000 7f309e17d740 10 client.0 _ll_register_callbacks cb 0x55ee065a6e80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T10:34:20.549 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:20.548+0000 7f3098a00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ee066826d0 0x55ee0668b5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:34:20.549 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:20.548+0000 7f3098a00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ee066826d0 0x55ee0668b5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:35788/0 (socket says 172.21.15.47:35788) 2024-03-21T10:34:20.550 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:20.549+0000 7f3098a00640 1 -- 172.21.15.47:0/849837856 learned_addr learned my addr 172.21.15.47:0/849837856 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:34:20.550 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:20.549+0000 7f3099400640 1 --2- 172.21.15.47:0/849837856 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ee0668bc00 0x55ee06689070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:34:20.550 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:20.549+0000 7f3093e00640 1 --2- 172.21.15.47:0/849837856 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ee06683080 0x55ee06688930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:34:20.550 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:20.549+0000 7f3098a00640 1 -- 172.21.15.47:0/849837856 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ee06683080 msgr2=0x55ee06688930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:34:20.550 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:20.549+0000 7f3098a00640 1 --2- 172.21.15.47:0/849837856 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ee06683080 0x55ee06688930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:20.550 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:20.549+0000 7f3098a00640 1 -- 172.21.15.47:0/849837856 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ee0668bc00 msgr2=0x55ee06689070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:34:20.550 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:20.549+0000 7f3098a00640 1 --2- 172.21.15.47:0/849837856 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ee0668bc00 0x55ee06689070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:20.550 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:20.549+0000 7f3098a00640 1 -- 172.21.15.47:0/849837856 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55ee062fa3f0 con 0x55ee066826d0 2024-03-21T10:34:20.550 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:20.549+0000 7f3098a00640 1 --2- 172.21.15.47:0/849837856 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ee066826d0 0x55ee0668b5a0 secure :-1 s=READY pgs=1230 cs=0 l=1 rev1=1 crypto rx=0x7f3088018000 tx=0x7f308809ba30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b74caf89ba01251a server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:34:20.550 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:20.550+0000 7f307a000640 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-03-21T10:34:20.550 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:20.550+0000 7f307a000640 1 -- 172.21.15.47:0/849837856 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30880cd7f0 con 0x55ee066826d0 2024-03-21T10:34:20.550 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:20.550+0000 7f307a000640 1 -- 172.21.15.47:0/849837856 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(103..103 src has 1..103) v4 ==== 9562+0+0 (secure 0 0 0) 0x7f30880d20e0 con 0x55ee066826d0 2024-03-21T10:34:20.550 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:20.550+0000 7f309e17d740 1 -- 172.21.15.47:0/849837856 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f308002c9a0 con 0x55ee066826d0 2024-03-21T10:34:20.550 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:20.550+0000 7f307a000640 1 -- 172.21.15.47:0/849837856 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f30880d2300 con 0x55ee066826d0 2024-03-21T10:34:20.551 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:20.550+0000 7f309e17d740 10 client.18519 fetch_fsmap learned FSMap version 32 2024-03-21T10:34:20.551 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:20.550+0000 7f309e17d740 1 -- 172.21.15.47:0/849837856 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x55ee063c47d0 con 0x55ee066826d0 2024-03-21T10:34:20.551 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:20.550+0000 7f307a000640 1 -- 172.21.15.47:0/849837856 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f308002c9a0 con 0x55ee066826d0 2024-03-21T10:34:20.551 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:20.551+0000 7f307a000640 1 -- 172.21.15.47:0/849837856 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 32) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f30880a4590 con 0x55ee066826d0 2024-03-21T10:34:20.551 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:20.551+0000 7f309e17d740 10 client.18519 fetch_fsmap finished waiting for FSMap version 32 2024-03-21T10:34:20.551 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:20.551+0000 7f309e17d740 10 client.18519 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-21T10:34:20.551 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:20.551+0000 7f309e17d740 20 client.18519 populate_metadata read hostname 'smithi047' 2024-03-21T10:34:20.551 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:20.551+0000 7f309e17d740 1 --2- 172.21.15.47:0/849837856 >> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] conn(0x55ee0669ff00 0x55ee066a2310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:34:20.551 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:20.551+0000 7f309e17d740 4 client.18519 mds_command: new command op to 34535 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T10:34:20.551 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:20.551+0000 7f309e17d740 1 -- 172.21.15.47:0/849837856 --> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55ee065f3800 con 0x55ee0669ff00 2024-03-21T10:34:20.552 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:20.551+0000 7f3093e00640 1 --2- 172.21.15.47:0/849837856 >> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] conn(0x55ee0669ff00 0x55ee066a2310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:34:20.552 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:20.552+0000 7f3093e00640 1 --2- 172.21.15.47:0/849837856 >> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] conn(0x55ee0669ff00 0x55ee066a2310 crc :-1 s=READY pgs=207 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d16e0581d6a284e4 server_cookie=98b7221edb682227 in_seq=0 out_seq=0 2024-03-21T10:34:20.553 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:20.552+0000 7f307a000640 10 client.18519 ms_handle_connect on v2:172.21.15.100:6838/4173168440 2024-03-21T10:34:20.553 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:20.553+0000 7f307a000640 1 -- 172.21.15.47:0/849837856 <== mds.0 v2:172.21.15.100:6838/4173168440 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55ee065f3800 con 0x55ee0669ff00 2024-03-21T10:34:20.553 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:20.553+0000 7f307a000640 10 client.18519 handle_command_reply: tid=0 2024-03-21T10:34:20.553 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:20.553+0000 7f309e17d740 1 client.18519 shutdown 2024-03-21T10:34:20.553 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:20.554+0000 7f309e17d740 1 -- 172.21.15.47:0/849837856 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ee066826d0 msgr2=0x55ee0668b5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:34:20.553 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:20.554+0000 7f309e17d740 1 --2- 172.21.15.47:0/849837856 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ee066826d0 0x55ee0668b5a0 secure :-1 s=READY pgs=1230 cs=0 l=1 rev1=1 crypto rx=0x7f3088018000 tx=0x7f308809ba30 comp rx=0 tx=0).stop 2024-03-21T10:34:20.553 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:20.554+0000 7f309e17d740 1 -- 172.21.15.47:0/849837856 shutdown_connections 2024-03-21T10:34:20.553 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:20.554+0000 7f309e17d740 1 --2- 172.21.15.47:0/849837856 >> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] conn(0x55ee0669ff00 0x55ee066a2310 crc :-1 s=READY pgs=207 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:20.553 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:20.554+0000 7f309e17d740 1 --2- 172.21.15.47:0/849837856 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ee0668bc00 0x55ee06689070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:20.553 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:20.554+0000 7f309e17d740 1 --2- 172.21.15.47:0/849837856 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ee06683080 0x55ee06688930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:20.553 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:20.554+0000 7f309e17d740 1 --2- 172.21.15.47:0/849837856 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ee066826d0 0x55ee0668b5a0 unknown :-1 s=CLOSED pgs=1230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:20.553 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:20.554+0000 7f307a000640 0 client.18519 ms_handle_reset on v2:172.21.15.100:6838/4173168440 2024-03-21T10:34:20.553 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:20.554+0000 7f309e17d740 1 -- 172.21.15.47:0/849837856 >> 172.21.15.47:0/849837856 conn(0x55ee0667cda0 msgr2=0x55ee0667eaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:34:20.553 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:20.554+0000 7f309e17d740 1 -- 172.21.15.47:0/849837856 shutdown_connections 2024-03-21T10:34:20.553 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:20.554+0000 7f309e17d740 1 -- 172.21.15.47:0/849837856 wait complete. 2024-03-21T10:34:20.554 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:20.554+0000 7f309e17d740 20 client.18519 trim_cache size 0 max 16384 2024-03-21T10:34:20.570 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:20.571+0000 7f309e17d740 1 Processor -- start 2024-03-21T10:34:20.571 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:20.571+0000 7f309e17d740 1 -- start start 2024-03-21T10:34:20.571 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:20.571+0000 7f309e17d740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ee066826d0 0x55ee0668b060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:34:20.571 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:20.571+0000 7f309e17d740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ee06683080 0x55ee0668b5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:34:20.571 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:20.571+0000 7f309e17d740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ee0668bc00 0x55ee06685b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:34:20.571 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:20.571+0000 7f309e17d740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30941ac740 con 0x55ee066826d0 2024-03-21T10:34:20.571 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:20.571+0000 7f309e17d740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30941ae0c0 con 0x55ee0668bc00 2024-03-21T10:34:20.571 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:20.571+0000 7f309e17d740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30941acda0 con 0x55ee06683080 2024-03-21T10:34:20.571 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:20.572+0000 7f3098a00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ee066826d0 0x55ee0668b060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:34:20.571 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:20.572+0000 7f3099400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ee0668bc00 0x55ee06685b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:34:20.571 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:20.572+0000 7f3098a00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ee066826d0 0x55ee0668b060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:35794/0 (socket says 172.21.15.47:35794) 2024-03-21T10:34:20.571 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:20.572+0000 7f3098a00640 1 -- 172.21.15.47:0/1470505989 learned_addr learned my addr 172.21.15.47:0/1470505989 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:34:20.571 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:20.572+0000 7f3093e00640 1 --2- 172.21.15.47:0/1470505989 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ee06683080 0x55ee0668b5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:34:20.572 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:20.572+0000 7f3098a00640 1 -- 172.21.15.47:0/1470505989 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ee06683080 msgr2=0x55ee0668b5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:34:20.572 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:20.572+0000 7f3098a00640 1 --2- 172.21.15.47:0/1470505989 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ee06683080 0x55ee0668b5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:20.572 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:20.572+0000 7f3098a00640 1 -- 172.21.15.47:0/1470505989 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ee0668bc00 msgr2=0x55ee06685b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:34:20.572 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:20.572+0000 7f3098a00640 1 --2- 172.21.15.47:0/1470505989 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ee0668bc00 0x55ee06685b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:20.572 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:20.572+0000 7f3098a00640 1 -- 172.21.15.47:0/1470505989 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ee066893e0 con 0x55ee066826d0 2024-03-21T10:34:20.572 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:20.572+0000 7f3093e00640 1 --2- 172.21.15.47:0/1470505989 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ee06683080 0x55ee0668b5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T10:34:20.576 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:20.573+0000 7f3098a00640 1 --2- 172.21.15.47:0/1470505989 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ee066826d0 0x55ee0668b060 secure :-1 s=READY pgs=1231 cs=0 l=1 rev1=1 crypto rx=0x7f308809bfa0 tx=0x7f308809baa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8edeaa383d7a3a47 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:34:20.577 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:20.573+0000 7f3085200640 1 -- 172.21.15.47:0/1470505989 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3088066070 con 0x55ee066826d0 2024-03-21T10:34:20.577 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:20.573+0000 7f3085200640 1 -- 172.21.15.47:0/1470505989 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f308809c010 con 0x55ee066826d0 2024-03-21T10:34:20.577 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:20.573+0000 7f3085200640 1 -- 172.21.15.47:0/1470505989 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30880a4590 con 0x55ee066826d0 2024-03-21T10:34:20.577 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:20.574+0000 7f309e17d740 1 -- 172.21.15.47:0/1470505989 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ee066826d0 msgr2=0x55ee0668b060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:34:20.577 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:20.574+0000 7f309e17d740 1 --2- 172.21.15.47:0/1470505989 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ee066826d0 0x55ee0668b060 secure :-1 s=READY pgs=1231 cs=0 l=1 rev1=1 crypto rx=0x7f308809bfa0 tx=0x7f308809baa0 comp rx=0 tx=0).stop 2024-03-21T10:34:20.577 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:20.574+0000 7f309e17d740 1 -- 172.21.15.47:0/1470505989 shutdown_connections 2024-03-21T10:34:20.577 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:20.574+0000 7f309e17d740 1 --2- 172.21.15.47:0/1470505989 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ee0668bc00 0x55ee06685b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:20.578 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:20.574+0000 7f309e17d740 1 --2- 172.21.15.47:0/1470505989 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ee06683080 0x55ee0668b5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:20.578 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:20.574+0000 7f309e17d740 1 --2- 172.21.15.47:0/1470505989 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ee066826d0 0x55ee0668b060 unknown :-1 s=CLOSED pgs=1231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:20.578 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:20.574+0000 7f309e17d740 1 -- 172.21.15.47:0/1470505989 >> 172.21.15.47:0/1470505989 conn(0x55ee0667cda0 msgr2=0x55ee0667ea60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:34:20.578 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:20.574+0000 7f309e17d740 1 -- 172.21.15.47:0/1470505989 shutdown_connections 2024-03-21T10:34:20.578 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:20.574+0000 7f309e17d740 1 -- 172.21.15.47:0/1470505989 wait complete. 2024-03-21T10:34:20.578 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:20.574+0000 7f309e17d740 1 -- start start 2024-03-21T10:34:20.578 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:20.575+0000 7f309e17d740 1 Processor -- start 2024-03-21T10:34:20.579 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:20.575+0000 7f309e17d740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ee066826d0 0x55ee066961f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:34:20.579 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:20.575+0000 7f309e17d740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ee06683080 0x55ee06696730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:34:20.579 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:20.575+0000 7f309e17d740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ee0668bc00 0x55ee06690f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:34:20.579 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:20.575+0000 7f309e17d740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30941ada60 con 0x55ee066826d0 2024-03-21T10:34:20.579 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:20.575+0000 7f309e17d740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30941ad400 con 0x55ee0668bc00 2024-03-21T10:34:20.579 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:20.575+0000 7f309e17d740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30941aa920 con 0x55ee06683080 2024-03-21T10:34:20.579 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:20.575+0000 7f309e17d740 10 client.0 _ll_register_callbacks cb 0x7f3088003d20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T10:34:20.579 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:20.575+0000 7f3098a00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ee066826d0 0x55ee066961f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:34:20.579 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:20.576+0000 7f3098a00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ee066826d0 0x55ee066961f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:35806/0 (socket says 172.21.15.47:35806) 2024-03-21T10:34:20.579 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:20.576+0000 7f3098a00640 1 -- 172.21.15.47:0/111707300 learned_addr learned my addr 172.21.15.47:0/111707300 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:34:20.579 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:20.576+0000 7f3099400640 1 --2- 172.21.15.47:0/111707300 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ee0668bc00 0x55ee06690f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:34:20.579 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:20.576+0000 7f3093e00640 1 --2- 172.21.15.47:0/111707300 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ee06683080 0x55ee06696730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:34:20.579 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:20.576+0000 7f3098a00640 1 -- 172.21.15.47:0/111707300 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ee06683080 msgr2=0x55ee06696730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:34:20.580 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:20.576+0000 7f3098a00640 1 --2- 172.21.15.47:0/111707300 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ee06683080 0x55ee06696730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:20.580 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:20.576+0000 7f3098a00640 1 -- 172.21.15.47:0/111707300 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ee0668bc00 msgr2=0x55ee06690f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:34:20.580 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:20.576+0000 7f3098a00640 1 --2- 172.21.15.47:0/111707300 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ee0668bc00 0x55ee06690f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:20.580 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:20.576+0000 7f3098a00640 1 -- 172.21.15.47:0/111707300 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55ee06691480 con 0x55ee066826d0 2024-03-21T10:34:20.580 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:20.576+0000 7f3099400640 1 --2- 172.21.15.47:0/111707300 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ee0668bc00 0x55ee06690f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T10:34:20.580 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:20.576+0000 7f3098a00640 1 --2- 172.21.15.47:0/111707300 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ee066826d0 0x55ee066961f0 secure :-1 s=READY pgs=1232 cs=0 l=1 rev1=1 crypto rx=0x7f308801b040 tx=0x7f30880cd820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d9990554b3f97943 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:34:20.580 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:20.576+0000 7f307a000640 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-03-21T10:34:20.580 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:20.577+0000 7f307a000640 1 -- 172.21.15.47:0/111707300 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3088066060 con 0x55ee066826d0 2024-03-21T10:34:20.580 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:20.577+0000 7f307a000640 1 -- 172.21.15.47:0/111707300 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(103..103 src has 1..103) v4 ==== 9562+0+0 (secure 0 0 0) 0x7f308809b460 con 0x55ee066826d0 2024-03-21T10:34:20.580 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:20.577+0000 7f309e17d740 1 -- 172.21.15.47:0/111707300 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f308809c010 con 0x55ee066826d0 2024-03-21T10:34:20.580 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:20.577+0000 7f307a000640 1 -- 172.21.15.47:0/111707300 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f30880a0160 con 0x55ee066826d0 2024-03-21T10:34:20.580 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:20.577+0000 7f309e17d740 10 client.18525 fetch_fsmap learned FSMap version 32 2024-03-21T10:34:20.580 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:20.577+0000 7f309e17d740 1 -- 172.21.15.47:0/111707300 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x55ee06700750 con 0x55ee066826d0 2024-03-21T10:34:20.580 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:20.577+0000 7f307a000640 1 -- 172.21.15.47:0/111707300 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f308809c010 con 0x55ee066826d0 2024-03-21T10:34:20.581 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:20.577+0000 7f307a000640 1 -- 172.21.15.47:0/111707300 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 32) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f308809aed0 con 0x55ee066826d0 2024-03-21T10:34:20.581 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:20.577+0000 7f309e17d740 10 client.18525 fetch_fsmap finished waiting for FSMap version 32 2024-03-21T10:34:20.581 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:20.577+0000 7f309e17d740 10 client.18525 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-21T10:34:20.581 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:20.577+0000 7f309e17d740 20 client.18525 populate_metadata read hostname 'smithi047' 2024-03-21T10:34:20.581 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:20.577+0000 7f309e17d740 1 --2- 172.21.15.47:0/111707300 >> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] conn(0x55ee06700ae0 0x55ee06702f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:34:20.581 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:20.578+0000 7f309e17d740 4 client.18525 mds_command: new command op to 34535 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-21T10:34:20.581 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:20.578+0000 7f309e17d740 1 -- 172.21.15.47:0/111707300 --> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55ee065edb50 con 0x55ee06700ae0 2024-03-21T10:34:20.581 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:20.578+0000 7f3093e00640 1 --2- 172.21.15.47:0/111707300 >> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] conn(0x55ee06700ae0 0x55ee06702f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:34:20.582 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:20.579+0000 7f3093e00640 1 --2- 172.21.15.47:0/111707300 >> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] conn(0x55ee06700ae0 0x55ee06702f00 crc :-1 s=READY pgs=208 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=23b96daf36bae987 server_cookie=8391ae49944fda3 in_seq=0 out_seq=0 2024-03-21T10:34:20.583 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:20.579+0000 7f307a000640 10 client.18525 ms_handle_connect on v2:172.21.15.100:6838/4173168440 2024-03-21T10:34:20.583 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:20.579+0000 7f307a000640 1 -- 172.21.15.47:0/111707300 <== mds.0 v2:172.21.15.100:6838/4173168440 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55ee065edb50 con 0x55ee06700ae0 2024-03-21T10:34:20.583 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:20.579+0000 7f307a000640 10 client.18525 handle_command_reply: tid=0 2024-03-21T10:34:20.583 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:20.579+0000 7f309e17d740 1 client.18525 shutdown 2024-03-21T10:34:20.583 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:20.580+0000 7f309e17d740 1 -- 172.21.15.47:0/111707300 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ee066826d0 msgr2=0x55ee066961f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:34:20.583 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:20.580+0000 7f309e17d740 1 --2- 172.21.15.47:0/111707300 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ee066826d0 0x55ee066961f0 secure :-1 s=READY pgs=1232 cs=0 l=1 rev1=1 crypto rx=0x7f308801b040 tx=0x7f30880cd820 comp rx=0 tx=0).stop 2024-03-21T10:34:20.583 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:20.580+0000 7f309e17d740 1 -- 172.21.15.47:0/111707300 shutdown_connections 2024-03-21T10:34:20.583 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:20.580+0000 7f309e17d740 1 --2- 172.21.15.47:0/111707300 >> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] conn(0x55ee06700ae0 0x55ee06702f00 crc :-1 s=READY pgs=208 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:20.583 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:20.580+0000 7f309e17d740 1 --2- 172.21.15.47:0/111707300 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ee0668bc00 0x55ee06690f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:20.583 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:20.580+0000 7f309e17d740 1 --2- 172.21.15.47:0/111707300 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ee06683080 0x55ee06696730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:20.583 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:20.580+0000 7f309e17d740 1 --2- 172.21.15.47:0/111707300 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ee066826d0 0x55ee066961f0 unknown :-1 s=CLOSED pgs=1232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:20.583 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:20.580+0000 7f309e17d740 1 -- 172.21.15.47:0/111707300 >> 172.21.15.47:0/111707300 conn(0x55ee0667cda0 msgr2=0x55ee0667b620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:34:20.583 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:20.580+0000 7f307a000640 0 client.18525 ms_handle_reset on v2:172.21.15.100:6838/4173168440 2024-03-21T10:34:20.584 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:20.580+0000 7f309e17d740 1 -- 172.21.15.47:0/111707300 shutdown_connections 2024-03-21T10:34:20.584 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:20.580+0000 7f309e17d740 1 -- 172.21.15.47:0/111707300 wait complete. 2024-03-21T10:34:20.584 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:20.580+0000 7f309e17d740 20 client.18525 trim_cache size 0 max 16384 2024-03-21T10:34:20.584 INFO:teuthology.orchestra.run.smithi047.stdout:[] 2024-03-21T10:34:20.585 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:20.581+0000 7f309b400640 1 -- 172.21.15.47:0/515293122 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f307c0a3090 msgr2=0x7f307c0a5550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:34:20.585 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:20.581+0000 7f309b400640 1 --2- 172.21.15.47:0/515293122 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f307c0a3090 0x7f307c0a5550 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f308c005fd0 tx=0x7f308c009700 comp rx=0 tx=0).stop 2024-03-21T10:34:20.585 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:20.581+0000 7f309b400640 1 -- 172.21.15.47:0/515293122 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3094068880 msgr2=0x7f3094197e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:34:20.585 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:20.581+0000 7f309b400640 1 --2- 172.21.15.47:0/515293122 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3094068880 0x7f3094197e00 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f308800e9c0 tx=0x7f308800ee80 comp rx=0 tx=0).stop 2024-03-21T10:34:20.585 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:20.581+0000 7f309b400640 1 -- 172.21.15.47:0/515293122 shutdown_connections 2024-03-21T10:34:20.585 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:20.581+0000 7f309b400640 1 --2- 172.21.15.47:0/515293122 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f30941076b0 0x7f309419c670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:20.585 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:20.581+0000 7f309b400640 1 --2- 172.21.15.47:0/515293122 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3094069240 0x7f3094198340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:20.585 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:20.581+0000 7f309b400640 1 --2- 172.21.15.47:0/515293122 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f307c0a3090 0x7f307c0a5550 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:20.585 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:20.581+0000 7f309b400640 1 --2- 172.21.15.47:0/515293122 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3094068880 0x7f3094197e00 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:20.585 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:20.581+0000 7f309b400640 1 -- 172.21.15.47:0/515293122 >> 172.21.15.47:0/515293122 conn(0x7f30940776a0 msgr2=0x7f30940ff0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:34:20.585 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:20.581+0000 7f309b400640 1 -- 172.21.15.47:0/515293122 shutdown_connections 2024-03-21T10:34:20.586 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:20.581+0000 7f309b400640 1 -- 172.21.15.47:0/515293122 wait complete. 2024-03-21T10:34:21.598 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-03-21T10:34:21.598 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-21T10:34:21.598 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-03-21T10:34:21.687 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:34:21 smithi047 ceph-mon[31547]: pgmap v280: 129 pgs: 129 active+clean; 2.5 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-03-21T10:34:21.727 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:34:21 smithi100 ceph-mon[35193]: pgmap v280: 129 pgs: 129 active+clean; 2.5 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-03-21T10:34:21.737 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:34:21 smithi157 ceph-mon[35581]: pgmap v280: 129 pgs: 129 active+clean; 2.5 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-03-21T10:34:21.836 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:21.837+0000 7fd04be00640 1 -- 172.21.15.47:0/4257510831 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd04c108800 msgr2=0x7fd04c10c7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:34:21.836 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:21.837+0000 7fd04be00640 1 --2- 172.21.15.47:0/4257510831 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd04c108800 0x7fd04c10c7c0 secure :-1 s=READY pgs=1233 cs=0 l=1 rev1=1 crypto rx=0x7fd038009900 tx=0x7fd03802bc40 comp rx=0 tx=0).stop 2024-03-21T10:34:21.836 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:21.837+0000 7fd04be00640 1 -- 172.21.15.47:0/4257510831 shutdown_connections 2024-03-21T10:34:21.836 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:21.837+0000 7fd04be00640 1 --2- 172.21.15.47:0/4257510831 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd04c10cea0 0x7fd04c114a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:21.836 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:21.837+0000 7fd04be00640 1 --2- 172.21.15.47:0/4257510831 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd04c108800 0x7fd04c10c7c0 unknown :-1 s=CLOSED pgs=1233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:21.836 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:21.837+0000 7fd04be00640 1 --2- 172.21.15.47:0/4257510831 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd04c107e50 0x7fd04c108230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:21.836 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:21.837+0000 7fd04be00640 1 -- 172.21.15.47:0/4257510831 >> 172.21.15.47:0/4257510831 conn(0x7fd04c0774c0 msgr2=0x7fd04c1064c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:34:21.836 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:21.837+0000 7fd04be00640 1 -- 172.21.15.47:0/4257510831 shutdown_connections 2024-03-21T10:34:21.836 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:21.838+0000 7fd04be00640 1 -- 172.21.15.47:0/4257510831 wait complete. 2024-03-21T10:34:21.837 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:21.839+0000 7fd04be00640 1 Processor -- start 2024-03-21T10:34:21.837 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:21.839+0000 7fd04be00640 1 -- start start 2024-03-21T10:34:21.839 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:21.840+0000 7fd04be00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd04c107e50 0x7fd04c19e360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:34:21.839 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:21.840+0000 7fd04be00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd04c10cea0 0x7fd04c19e8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:34:21.839 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:21.840+0000 7fd04be00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd04c1a2c20 0x7fd04c1a30d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:34:21.839 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:21.840+0000 7fd04be00640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd04c117e00 con 0x7fd04c107e50 2024-03-21T10:34:21.839 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:21.840+0000 7fd04be00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd04c117c80 con 0x7fd04c10cea0 2024-03-21T10:34:21.839 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:21.840+0000 7fd04be00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd04c117f80 con 0x7fd04c1a2c20 2024-03-21T10:34:21.839 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:21.840+0000 7fd04aa00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd04c107e50 0x7fd04c19e360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:34:21.839 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:21.840+0000 7fd04aa00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd04c107e50 0x7fd04c19e360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:35820/0 (socket says 172.21.15.47:35820) 2024-03-21T10:34:21.839 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:21.840+0000 7fd04aa00640 1 -- 172.21.15.47:0/4165651614 learned_addr learned my addr 172.21.15.47:0/4165651614 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:34:21.839 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:21.840+0000 7fd04a000640 1 --2- 172.21.15.47:0/4165651614 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd04c10cea0 0x7fd04c19e8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:34:21.839 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:21.840+0000 7fd04b400640 1 --2- 172.21.15.47:0/4165651614 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd04c1a2c20 0x7fd04c1a30d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:34:21.839 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:21.841+0000 7fd04aa00640 1 -- 172.21.15.47:0/4165651614 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd04c1a2c20 msgr2=0x7fd04c1a30d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:34:21.840 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:21.841+0000 7fd04aa00640 1 --2- 172.21.15.47:0/4165651614 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd04c1a2c20 0x7fd04c1a30d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:21.840 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:21.841+0000 7fd04aa00640 1 -- 172.21.15.47:0/4165651614 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd04c10cea0 msgr2=0x7fd04c19e8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:34:21.840 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:21.841+0000 7fd04aa00640 1 --2- 172.21.15.47:0/4165651614 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd04c10cea0 0x7fd04c19e8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:21.840 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:21.841+0000 7fd04aa00640 1 -- 172.21.15.47:0/4165651614 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd04c1a3710 con 0x7fd04c107e50 2024-03-21T10:34:21.840 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:21.841+0000 7fd04b400640 1 --2- 172.21.15.47:0/4165651614 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd04c1a2c20 0x7fd04c1a30d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T10:34:21.840 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:21.841+0000 7fd04aa00640 1 --2- 172.21.15.47:0/4165651614 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd04c107e50 0x7fd04c19e360 secure :-1 s=READY pgs=1234 cs=0 l=1 rev1=1 crypto rx=0x7fd03c00d950 tx=0x7fd03c00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:34:21.841 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:21.841+0000 7fd043400640 1 -- 172.21.15.47:0/4165651614 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd03c017070 con 0x7fd04c107e50 2024-03-21T10:34:21.841 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:21.841+0000 7fd04be00640 1 -- 172.21.15.47:0/4165651614 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd04c1a3a00 con 0x7fd04c107e50 2024-03-21T10:34:21.841 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:21.841+0000 7fd043400640 1 -- 172.21.15.47:0/4165651614 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fd03c009c30 con 0x7fd04c107e50 2024-03-21T10:34:21.841 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:21.841+0000 7fd043400640 1 -- 172.21.15.47:0/4165651614 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd03c00f620 con 0x7fd04c107e50 2024-03-21T10:34:21.841 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:21.842+0000 7fd04be00640 1 -- 172.21.15.47:0/4165651614 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd04c1ab1c0 con 0x7fd04c107e50 2024-03-21T10:34:21.843 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:21.845+0000 7fd043400640 1 -- 172.21.15.47:0/4165651614 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 27) v1 ==== 137985+0+0 (secure 0 0 0) 0x7fd03c0109a0 con 0x7fd04c107e50 2024-03-21T10:34:21.843 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:21.845+0000 7fd05463a740 1 Processor -- start 2024-03-21T10:34:21.843 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:21.845+0000 7fd05463a740 1 -- start start 2024-03-21T10:34:21.845 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:21.845+0000 7fd05463a740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559cb7d3a290 0x559cb7d3a670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:34:21.845 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:21.846+0000 7fd05463a740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x559cb7d3abb0 0x559cb7d43250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:34:21.846 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:21.846+0000 7fd05463a740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559cb7d43790 0x559cb7d45b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:34:21.846 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:21.846+0000 7fd05463a740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x559cb7c9eeb0 con 0x559cb7d3abb0 2024-03-21T10:34:21.846 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:21.846+0000 7fd05463a740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x559cb7c47a60 con 0x559cb7d3a290 2024-03-21T10:34:21.846 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:21.846+0000 7fd05463a740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x559cb7c9af50 con 0x559cb7d43790 2024-03-21T10:34:21.846 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:21.846+0000 7fd04aa00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x559cb7d3abb0 0x559cb7d43250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:34:21.846 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:21.846+0000 7fd04b400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559cb7d3a290 0x559cb7d3a670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:34:21.846 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:21.846+0000 7fd04b400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559cb7d3a290 0x559cb7d3a670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:50426/0 (socket says 172.21.15.47:50426) 2024-03-21T10:34:21.846 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:21.846+0000 7fd04b400640 1 -- 172.21.15.47:0/103496817 learned_addr learned my addr 172.21.15.47:0/103496817 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:34:21.846 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:21.846+0000 7fd04a000640 1 --2- 172.21.15.47:0/103496817 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559cb7d43790 0x559cb7d45b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:34:21.846 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:21.846+0000 7fd04b400640 1 -- 172.21.15.47:0/103496817 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559cb7d43790 msgr2=0x559cb7d45b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:34:21.846 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:21.846+0000 7fd04b400640 1 --2- 172.21.15.47:0/103496817 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559cb7d43790 0x559cb7d45b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:21.846 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:21.846+0000 7fd04b400640 1 -- 172.21.15.47:0/103496817 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x559cb7d3abb0 msgr2=0x559cb7d43250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:34:21.846 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:21.846+0000 7fd04b400640 1 --2- 172.21.15.47:0/103496817 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x559cb7d3abb0 0x559cb7d43250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:21.847 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:21.846+0000 7fd043400640 1 --2- 172.21.15.47:0/4165651614 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7fd0300a3090 0x7fd0300a5550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:34:21.847 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:21.847+0000 7fd04b400640 1 -- 172.21.15.47:0/103496817 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559cb7a7b7e0 con 0x559cb7d3a290 2024-03-21T10:34:21.847 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:21.847+0000 7fd04aa00640 1 --2- 172.21.15.47:0/103496817 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x559cb7d3abb0 0x559cb7d43250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T10:34:21.847 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:21.847+0000 7fd043400640 1 -- 172.21.15.47:0/4165651614 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 9562+0+0 (secure 0 0 0) 0x7fd03c0cf890 con 0x7fd04c107e50 2024-03-21T10:34:21.847 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:21.847+0000 7fd04a000640 1 --2- 172.21.15.47:0/4165651614 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7fd0300a3090 0x7fd0300a5550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:34:21.847 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:21.847+0000 7fd04b400640 1 --2- 172.21.15.47:0/103496817 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559cb7d3a290 0x559cb7d3a670 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7fd044009f20 tx=0x7fd044010800 comp rx=0 tx=0).ready entity=mon.2 client_cookie=471dcf094f90c325 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:34:21.847 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:21.847+0000 7fd037e00640 1 -- 172.21.15.47:0/103496817 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd044016070 con 0x559cb7d3a290 2024-03-21T10:34:21.847 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:21.847+0000 7fd037e00640 1 -- 172.21.15.47:0/103496817 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fd04401bb10 con 0x559cb7d3a290 2024-03-21T10:34:21.847 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:21.847+0000 7fd037e00640 1 -- 172.21.15.47:0/103496817 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd044012c30 con 0x559cb7d3a290 2024-03-21T10:34:21.849 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:21.848+0000 7fd05463a740 1 -- 172.21.15.47:0/103496817 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559cb7d3a290 msgr2=0x559cb7d3a670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:34:21.850 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:21.848+0000 7fd04a000640 1 --2- 172.21.15.47:0/4165651614 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7fd0300a3090 0x7fd0300a5550 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fd038000c00 tx=0x7fd038005fd0 comp rx=0 tx=0).ready entity=mgr.15687 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:34:21.850 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:21.848+0000 7fd05463a740 1 --2- 172.21.15.47:0/103496817 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559cb7d3a290 0x559cb7d3a670 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7fd044009f20 tx=0x7fd044010800 comp rx=0 tx=0).stop 2024-03-21T10:34:21.850 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:21.848+0000 7fd05463a740 1 -- 172.21.15.47:0/103496817 shutdown_connections 2024-03-21T10:34:21.850 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:21.848+0000 7fd05463a740 1 --2- 172.21.15.47:0/103496817 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559cb7d43790 0x559cb7d45b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:21.850 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:21.848+0000 7fd05463a740 1 --2- 172.21.15.47:0/103496817 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x559cb7d3abb0 0x559cb7d43250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:21.850 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:21.848+0000 7fd05463a740 1 --2- 172.21.15.47:0/103496817 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559cb7d3a290 0x559cb7d3a670 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:21.850 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:21.848+0000 7fd05463a740 1 -- 172.21.15.47:0/103496817 >> 172.21.15.47:0/103496817 conn(0x559cb7d349c0 msgr2=0x559cb7d34da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:34:21.850 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:21.848+0000 7fd05463a740 1 -- 172.21.15.47:0/103496817 shutdown_connections 2024-03-21T10:34:21.850 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:21.848+0000 7fd05463a740 1 -- 172.21.15.47:0/103496817 wait complete. 2024-03-21T10:34:21.850 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:21.849+0000 7fd05463a740 1 -- start start 2024-03-21T10:34:21.850 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:21.849+0000 7fd05463a740 1 Processor -- start 2024-03-21T10:34:21.850 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:21.850+0000 7fd05463a740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559cb7d3a290 0x559cb7d431b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:34:21.850 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:21.850+0000 7fd05463a740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x559cb7d3abb0 0x559cb7d40540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:34:21.851 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:21.850+0000 7fd05463a740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559cb7d43790 0x559cb7d40b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:34:21.851 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:21.850+0000 7fd05463a740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x559cb7d46580 con 0x559cb7d3abb0 2024-03-21T10:34:21.851 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:21.850+0000 7fd05463a740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x559cb7c5aeb0 con 0x559cb7d3a290 2024-03-21T10:34:21.851 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:21.850+0000 7fd05463a740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x559cb7d46700 con 0x559cb7d43790 2024-03-21T10:34:21.851 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:21.850+0000 7fd05463a740 10 client.0 _ll_register_callbacks cb 0x559cb7c4f150 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T10:34:21.851 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:21.850+0000 7fd04a000640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x559cb7d3abb0 0x559cb7d40540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:34:21.851 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:21.850+0000 7fd04a000640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x559cb7d3abb0 0x559cb7d40540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:35844/0 (socket says 172.21.15.47:35844) 2024-03-21T10:34:21.851 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:21.850+0000 7fd04aa00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559cb7d3a290 0x559cb7d431b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:34:21.851 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:21.850+0000 7fd04a000640 1 -- 172.21.15.47:0/4179798736 learned_addr learned my addr 172.21.15.47:0/4179798736 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:34:21.851 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:21.850+0000 7fd04b400640 1 --2- 172.21.15.47:0/4179798736 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559cb7d43790 0x559cb7d40b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:34:21.851 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:21.850+0000 7fd04a000640 1 -- 172.21.15.47:0/4179798736 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559cb7d43790 msgr2=0x559cb7d40b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:34:21.851 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:21.850+0000 7fd04a000640 1 --2- 172.21.15.47:0/4179798736 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559cb7d43790 0x559cb7d40b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:21.851 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:21.850+0000 7fd04a000640 1 -- 172.21.15.47:0/4179798736 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559cb7d3a290 msgr2=0x559cb7d431b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:34:21.851 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:21.850+0000 7fd04a000640 1 --2- 172.21.15.47:0/4179798736 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559cb7d3a290 0x559cb7d431b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:21.852 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:21.850+0000 7fd04a000640 1 -- 172.21.15.47:0/4179798736 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x559cb7adbf70 con 0x559cb7d3abb0 2024-03-21T10:34:21.852 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:21.851+0000 7fd04a000640 1 --2- 172.21.15.47:0/4179798736 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x559cb7d3abb0 0x559cb7d40540 secure :-1 s=READY pgs=1235 cs=0 l=1 rev1=1 crypto rx=0x7fd038009900 tx=0x7fd03802bc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b3e34b17cf23c2a6 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:34:21.852 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:21.851+0000 7fd034c00640 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-03-21T10:34:21.852 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:21.851+0000 7fd034c00640 1 -- 172.21.15.47:0/4179798736 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd03804f030 con 0x559cb7d3abb0 2024-03-21T10:34:21.852 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:21.851+0000 7fd05463a740 1 -- 172.21.15.47:0/4179798736 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fd04401bb10 con 0x559cb7d3abb0 2024-03-21T10:34:21.852 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:21.851+0000 7fd034c00640 1 -- 172.21.15.47:0/4179798736 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(103..103 src has 1..103) v4 ==== 9562+0+0 (secure 0 0 0) 0x7fd03803a040 con 0x559cb7d3abb0 2024-03-21T10:34:21.852 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:21.851+0000 7fd034c00640 1 -- 172.21.15.47:0/4179798736 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd03805c050 con 0x559cb7d3abb0 2024-03-21T10:34:21.852 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:21.851+0000 7fd05463a740 10 client.18537 fetch_fsmap learned FSMap version 32 2024-03-21T10:34:21.852 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:21.851+0000 7fd05463a740 1 -- 172.21.15.47:0/4179798736 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x559cb7973170 con 0x559cb7d3abb0 2024-03-21T10:34:21.852 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:21.852+0000 7fd034c00640 1 -- 172.21.15.47:0/4179798736 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fd04401bb10 con 0x559cb7d3abb0 2024-03-21T10:34:21.854 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:21.852+0000 7fd034c00640 1 -- 172.21.15.47:0/4179798736 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 32) v1 ==== 7942+0+0 (secure 0 0 0) 0x7fd0380035f0 con 0x559cb7d3abb0 2024-03-21T10:34:21.854 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:21.852+0000 7fd05463a740 10 client.18537 fetch_fsmap finished waiting for FSMap version 32 2024-03-21T10:34:21.854 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:21.852+0000 7fd05463a740 10 client.18537 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-21T10:34:21.854 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:21.852+0000 7fd05463a740 20 client.18537 populate_metadata read hostname 'smithi047' 2024-03-21T10:34:21.854 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:21.852+0000 7fd05463a740 1 --2- 172.21.15.47:0/4179798736 >> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] conn(0x559cb7d538f0 0x559cb7d55ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:34:21.854 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:21.852+0000 7fd05463a740 4 client.18537 mds_command: new command op to 34535 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T10:34:21.854 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:21.852+0000 7fd05463a740 1 -- 172.21.15.47:0/4179798736 --> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x559cb7cac3f0 con 0x559cb7d538f0 2024-03-21T10:34:21.855 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:21.853+0000 7fd04aa00640 1 --2- 172.21.15.47:0/4179798736 >> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] conn(0x559cb7d538f0 0x559cb7d55ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:34:21.855 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:21.853+0000 7fd04aa00640 1 --2- 172.21.15.47:0/4179798736 >> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] conn(0x559cb7d538f0 0x559cb7d55ce0 crc :-1 s=READY pgs=209 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bcb83088ce16f15f server_cookie=fd6fe332839e5a32 in_seq=0 out_seq=0 2024-03-21T10:34:21.855 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:21.853+0000 7fd034c00640 10 client.18537 ms_handle_connect on v2:172.21.15.100:6838/4173168440 2024-03-21T10:34:21.855 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:21.854+0000 7fd034c00640 1 -- 172.21.15.47:0/4179798736 <== mds.0 v2:172.21.15.100:6838/4173168440 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x559cb7cac3f0 con 0x559cb7d538f0 2024-03-21T10:34:21.855 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:21.854+0000 7fd034c00640 10 client.18537 handle_command_reply: tid=0 2024-03-21T10:34:21.855 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:21.854+0000 7fd05463a740 1 client.18537 shutdown 2024-03-21T10:34:21.855 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:21.855+0000 7fd05463a740 1 -- 172.21.15.47:0/4179798736 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x559cb7d3abb0 msgr2=0x559cb7d40540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:34:21.855 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:21.855+0000 7fd05463a740 1 --2- 172.21.15.47:0/4179798736 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x559cb7d3abb0 0x559cb7d40540 secure :-1 s=READY pgs=1235 cs=0 l=1 rev1=1 crypto rx=0x7fd038009900 tx=0x7fd03802bc40 comp rx=0 tx=0).stop 2024-03-21T10:34:21.855 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:21.855+0000 7fd05463a740 1 -- 172.21.15.47:0/4179798736 shutdown_connections 2024-03-21T10:34:21.855 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:21.855+0000 7fd05463a740 1 --2- 172.21.15.47:0/4179798736 >> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] conn(0x559cb7d538f0 0x559cb7d55ce0 crc :-1 s=READY pgs=209 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:21.855 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:21.855+0000 7fd05463a740 1 --2- 172.21.15.47:0/4179798736 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559cb7d43790 0x559cb7d40b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:21.855 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:21.855+0000 7fd05463a740 1 --2- 172.21.15.47:0/4179798736 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x559cb7d3abb0 0x559cb7d40540 unknown :-1 s=CLOSED pgs=1235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:21.855 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:21.855+0000 7fd05463a740 1 --2- 172.21.15.47:0/4179798736 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559cb7d3a290 0x559cb7d431b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:21.855 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:21.855+0000 7fd034c00640 0 client.18537 ms_handle_reset on v2:172.21.15.100:6838/4173168440 2024-03-21T10:34:21.855 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:21.855+0000 7fd05463a740 1 -- 172.21.15.47:0/4179798736 >> 172.21.15.47:0/4179798736 conn(0x559cb7d349c0 msgr2=0x559cb7d36710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:34:21.856 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:21.855+0000 7fd05463a740 1 -- 172.21.15.47:0/4179798736 shutdown_connections 2024-03-21T10:34:21.856 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:21.855+0000 7fd05463a740 1 -- 172.21.15.47:0/4179798736 wait complete. 2024-03-21T10:34:21.856 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:21.855+0000 7fd05463a740 20 client.18537 trim_cache size 0 max 16384 2024-03-21T10:34:21.872 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:21.873+0000 7fd05463a740 1 Processor -- start 2024-03-21T10:34:21.872 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:21.873+0000 7fd05463a740 1 -- start start 2024-03-21T10:34:21.872 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:21.874+0000 7fd05463a740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559cb7d3a290 0x559cb7d42c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:34:21.873 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:21.874+0000 7fd05463a740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559cb7d43790 0x559cb7d431b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:34:21.873 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:21.874+0000 7fd05463a740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x559cb7d4b940 0x559cb7d4ee10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:34:21.873 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:21.874+0000 7fd05463a740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd04c1b4db0 con 0x559cb7d4b940 2024-03-21T10:34:21.873 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:21.874+0000 7fd05463a740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd04c1b6730 con 0x559cb7d43790 2024-03-21T10:34:21.873 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:21.874+0000 7fd05463a740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd04c1b5410 con 0x559cb7d3a290 2024-03-21T10:34:21.873 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:21.874+0000 7fd04b400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x559cb7d4b940 0x559cb7d4ee10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:34:21.873 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:21.874+0000 7fd04b400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x559cb7d4b940 0x559cb7d4ee10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:33002/0 (socket says 172.21.15.47:33002) 2024-03-21T10:34:21.873 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:21.874+0000 7fd04b400640 1 -- 172.21.15.47:0/3394806689 learned_addr learned my addr 172.21.15.47:0/3394806689 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:34:21.873 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:21.874+0000 7fd04a000640 1 --2- 172.21.15.47:0/3394806689 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559cb7d43790 0x559cb7d431b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:34:21.873 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:21.874+0000 7fd04aa00640 1 --2- 172.21.15.47:0/3394806689 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559cb7d3a290 0x559cb7d42c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:34:21.874 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:21.875+0000 7fd04b400640 1 -- 172.21.15.47:0/3394806689 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559cb7d3a290 msgr2=0x559cb7d42c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:34:21.875 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:21.875+0000 7fd04b400640 1 --2- 172.21.15.47:0/3394806689 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559cb7d3a290 0x559cb7d42c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:21.875 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:21.875+0000 7fd04b400640 1 -- 172.21.15.47:0/3394806689 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559cb7d43790 msgr2=0x559cb7d431b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:34:21.875 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:21.875+0000 7fd04b400640 1 --2- 172.21.15.47:0/3394806689 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559cb7d43790 0x559cb7d431b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:21.875 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:21.875+0000 7fd04b400640 1 -- 172.21.15.47:0/3394806689 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559cb7d40ec0 con 0x559cb7d4b940 2024-03-21T10:34:21.875 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:21.875+0000 7fd04aa00640 1 --2- 172.21.15.47:0/3394806689 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559cb7d3a290 0x559cb7d42c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T10:34:21.875 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:21.875+0000 7fd04b400640 1 --2- 172.21.15.47:0/3394806689 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x559cb7d4b940 0x559cb7d4ee10 secure :-1 s=READY pgs=1236 cs=0 l=1 rev1=1 crypto rx=0x7fd044010d00 tx=0x7fd044009c30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8f6d72d17766c3b9 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:34:21.875 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:21.876+0000 7fd037400640 1 -- 172.21.15.47:0/3394806689 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd044029070 con 0x559cb7d4b940 2024-03-21T10:34:21.875 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:21.876+0000 7fd037400640 1 -- 172.21.15.47:0/3394806689 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fd044012590 con 0x559cb7d4b940 2024-03-21T10:34:21.875 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:21.876+0000 7fd037400640 1 -- 172.21.15.47:0/3394806689 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd044016040 con 0x559cb7d4b940 2024-03-21T10:34:21.875 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:21.876+0000 7fd05463a740 1 -- 172.21.15.47:0/3394806689 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x559cb7d4b940 msgr2=0x559cb7d4ee10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:34:21.875 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:21.876+0000 7fd05463a740 1 --2- 172.21.15.47:0/3394806689 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x559cb7d4b940 0x559cb7d4ee10 secure :-1 s=READY pgs=1236 cs=0 l=1 rev1=1 crypto rx=0x7fd044010d00 tx=0x7fd044009c30 comp rx=0 tx=0).stop 2024-03-21T10:34:21.877 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:21.876+0000 7fd05463a740 1 -- 172.21.15.47:0/3394806689 shutdown_connections 2024-03-21T10:34:21.877 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:21.876+0000 7fd05463a740 1 --2- 172.21.15.47:0/3394806689 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x559cb7d4b940 0x559cb7d4ee10 unknown :-1 s=CLOSED pgs=1236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:21.877 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:21.876+0000 7fd05463a740 1 --2- 172.21.15.47:0/3394806689 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559cb7d43790 0x559cb7d431b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:21.877 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:21.876+0000 7fd05463a740 1 --2- 172.21.15.47:0/3394806689 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559cb7d3a290 0x559cb7d42c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:21.877 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:21.876+0000 7fd05463a740 1 -- 172.21.15.47:0/3394806689 >> 172.21.15.47:0/3394806689 conn(0x559cb7d349c0 msgr2=0x559cb7d33a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:34:21.877 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:21.876+0000 7fd05463a740 1 -- 172.21.15.47:0/3394806689 shutdown_connections 2024-03-21T10:34:21.877 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:21.877+0000 7fd05463a740 1 -- 172.21.15.47:0/3394806689 wait complete. 2024-03-21T10:34:21.880 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:21.877+0000 7fd05463a740 1 -- start start 2024-03-21T10:34:21.880 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:21.877+0000 7fd05463a740 1 Processor -- start 2024-03-21T10:34:21.880 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:21.878+0000 7fd05463a740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559cb7d3a290 0x559cb7d4af20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:34:21.880 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:21.878+0000 7fd05463a740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559cb7d43790 0x559cb7d48920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:34:21.881 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:21.878+0000 7fd05463a740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x559cb7d4b940 0x559cb7d48e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:34:21.881 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:21.878+0000 7fd05463a740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd04c1b60d0 con 0x559cb7d4b940 2024-03-21T10:34:21.881 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:21.878+0000 7fd05463a740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd04c1b5a70 con 0x559cb7d3a290 2024-03-21T10:34:21.881 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:21.878+0000 7fd05463a740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd04c1b2f90 con 0x559cb7d43790 2024-03-21T10:34:21.881 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:21.878+0000 7fd05463a740 10 client.0 _ll_register_callbacks cb 0x7fd038043050 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T10:34:21.881 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:21.878+0000 7fd04b400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x559cb7d4b940 0x559cb7d48e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:34:21.881 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:21.878+0000 7fd04b400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x559cb7d4b940 0x559cb7d48e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:33018/0 (socket says 172.21.15.47:33018) 2024-03-21T10:34:21.881 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:21.878+0000 7fd04b400640 1 -- 172.21.15.47:0/1069590810 learned_addr learned my addr 172.21.15.47:0/1069590810 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:34:21.881 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:21.879+0000 7fd04aa00640 1 --2- 172.21.15.47:0/1069590810 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559cb7d3a290 0x559cb7d4af20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:34:21.881 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:21.879+0000 7fd04a000640 1 --2- 172.21.15.47:0/1069590810 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559cb7d43790 0x559cb7d48920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:34:21.881 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:21.879+0000 7fd04b400640 1 -- 172.21.15.47:0/1069590810 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559cb7d43790 msgr2=0x559cb7d48920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:34:21.881 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:21.879+0000 7fd04b400640 1 --2- 172.21.15.47:0/1069590810 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559cb7d43790 0x559cb7d48920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:21.881 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:21.879+0000 7fd04b400640 1 -- 172.21.15.47:0/1069590810 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559cb7d3a290 msgr2=0x559cb7d4af20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:34:21.881 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:21.879+0000 7fd04b400640 1 --2- 172.21.15.47:0/1069590810 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559cb7d3a290 0x559cb7d4af20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:21.882 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:21.879+0000 7fd04b400640 1 -- 172.21.15.47:0/1069590810 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x559cb7d493a0 con 0x559cb7d4b940 2024-03-21T10:34:21.882 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:21.879+0000 7fd04b400640 1 --2- 172.21.15.47:0/1069590810 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x559cb7d4b940 0x559cb7d48e60 secure :-1 s=READY pgs=1237 cs=0 l=1 rev1=1 crypto rx=0x7fd044005c10 tx=0x7fd044004cb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ddb0b7c80323ebb3 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:34:21.882 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:21.879+0000 7fd034c00640 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-03-21T10:34:21.882 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:21.880+0000 7fd034c00640 1 -- 172.21.15.47:0/1069590810 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd044016050 con 0x559cb7d4b940 2024-03-21T10:34:21.882 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:21.880+0000 7fd05463a740 1 -- 172.21.15.47:0/1069590810 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fd044012590 con 0x559cb7d4b940 2024-03-21T10:34:21.882 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:21.880+0000 7fd034c00640 1 -- 172.21.15.47:0/1069590810 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(103..103 src has 1..103) v4 ==== 9562+0+0 (secure 0 0 0) 0x7fd044021040 con 0x559cb7d4b940 2024-03-21T10:34:21.882 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:21.880+0000 7fd034c00640 1 -- 172.21.15.47:0/1069590810 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd04403a050 con 0x559cb7d4b940 2024-03-21T10:34:21.882 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:21.880+0000 7fd05463a740 10 client.18543 fetch_fsmap learned FSMap version 32 2024-03-21T10:34:21.882 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:21.880+0000 7fd05463a740 1 -- 172.21.15.47:0/1069590810 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x559cb7db0ee0 con 0x559cb7d4b940 2024-03-21T10:34:21.882 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:21.880+0000 7fd034c00640 1 -- 172.21.15.47:0/1069590810 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fd044012590 con 0x559cb7d4b940 2024-03-21T10:34:21.882 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:21.881+0000 7fd034c00640 1 -- 172.21.15.47:0/1069590810 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 32) v1 ==== 7942+0+0 (secure 0 0 0) 0x7fd04400f040 con 0x559cb7d4b940 2024-03-21T10:34:21.882 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:21.881+0000 7fd05463a740 10 client.18543 fetch_fsmap finished waiting for FSMap version 32 2024-03-21T10:34:21.882 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:21.881+0000 7fd05463a740 10 client.18543 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-21T10:34:21.882 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:21.881+0000 7fd05463a740 20 client.18543 populate_metadata read hostname 'smithi047' 2024-03-21T10:34:21.883 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:21.881+0000 7fd05463a740 1 --2- 172.21.15.47:0/1069590810 >> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] conn(0x559cb7db1390 0x559cb7db37b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:34:21.883 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:21.881+0000 7fd05463a740 4 client.18543 mds_command: new command op to 34535 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-03-21T10:34:21.883 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:21.881+0000 7fd05463a740 1 -- 172.21.15.47:0/1069590810 --> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x559cb7ca6740 con 0x559cb7db1390 2024-03-21T10:34:21.883 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:21.881+0000 7fd04aa00640 1 --2- 172.21.15.47:0/1069590810 >> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] conn(0x559cb7db1390 0x559cb7db37b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:34:21.883 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:21.882+0000 7fd04aa00640 1 --2- 172.21.15.47:0/1069590810 >> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] conn(0x559cb7db1390 0x559cb7db37b0 crc :-1 s=READY pgs=210 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=81de65f5d8786a38 server_cookie=2d08d115078ae090 in_seq=0 out_seq=0 2024-03-21T10:34:21.883 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:21.882+0000 7fd034c00640 10 client.18543 ms_handle_connect on v2:172.21.15.100:6838/4173168440 2024-03-21T10:34:21.884 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:21.884+0000 7fd034c00640 1 -- 172.21.15.47:0/1069590810 <== mds.0 v2:172.21.15.100:6838/4173168440 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x559cb7ca6740 con 0x559cb7db1390 2024-03-21T10:34:21.884 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:21.884+0000 7fd034c00640 10 client.18543 handle_command_reply: tid=0 2024-03-21T10:34:21.884 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:21.884+0000 7fd05463a740 1 client.18543 shutdown 2024-03-21T10:34:21.884 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:21.884+0000 7fd05463a740 1 -- 172.21.15.47:0/1069590810 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x559cb7d4b940 msgr2=0x559cb7d48e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:34:21.884 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:21.884+0000 7fd05463a740 1 --2- 172.21.15.47:0/1069590810 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x559cb7d4b940 0x559cb7d48e60 secure :-1 s=READY pgs=1237 cs=0 l=1 rev1=1 crypto rx=0x7fd044005c10 tx=0x7fd044004cb0 comp rx=0 tx=0).stop 2024-03-21T10:34:21.884 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:21.884+0000 7fd05463a740 1 -- 172.21.15.47:0/1069590810 shutdown_connections 2024-03-21T10:34:21.884 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:21.884+0000 7fd05463a740 1 --2- 172.21.15.47:0/1069590810 >> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] conn(0x559cb7db1390 0x559cb7db37b0 crc :-1 s=READY pgs=210 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:21.884 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:21.884+0000 7fd05463a740 1 --2- 172.21.15.47:0/1069590810 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x559cb7d4b940 0x559cb7d48e60 unknown :-1 s=CLOSED pgs=1237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:21.885 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:21.884+0000 7fd05463a740 1 --2- 172.21.15.47:0/1069590810 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559cb7d43790 0x559cb7d48920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:21.885 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:21.884+0000 7fd05463a740 1 --2- 172.21.15.47:0/1069590810 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559cb7d3a290 0x559cb7d4af20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:21.885 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:21.884+0000 7fd05463a740 1 -- 172.21.15.47:0/1069590810 >> 172.21.15.47:0/1069590810 conn(0x559cb7d349c0 msgr2=0x559cb7d3b0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:34:21.885 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:21.884+0000 7fd034c00640 0 client.18543 ms_handle_reset on v2:172.21.15.100:6838/4173168440 2024-03-21T10:34:21.885 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:21.885+0000 7fd05463a740 1 -- 172.21.15.47:0/1069590810 shutdown_connections 2024-03-21T10:34:21.885 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:21.885+0000 7fd05463a740 1 -- 172.21.15.47:0/1069590810 wait complete. 2024-03-21T10:34:21.885 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:21.885+0000 7fd05463a740 20 client.18543 trim_cache size 0 max 16384 2024-03-21T10:34:21.885 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-03-21T10:34:21.885 INFO:teuthology.orchestra.run.smithi047.stdout: "return_code": 0, 2024-03-21T10:34:21.885 INFO:teuthology.orchestra.run.smithi047.stdout: "scrub_tag": "e5a79add-446e-4805-8b0f-34329e0854e1", 2024-03-21T10:34:21.885 INFO:teuthology.orchestra.run.smithi047.stdout: "mode": "asynchronous" 2024-03-21T10:34:21.886 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-03-21T10:34:21.886 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:21.885+0000 7fd04be00640 1 -- 172.21.15.47:0/4165651614 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7fd0300a3090 msgr2=0x7fd0300a5550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:34:21.886 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:21.885+0000 7fd04be00640 1 --2- 172.21.15.47:0/4165651614 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7fd0300a3090 0x7fd0300a5550 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fd038000c00 tx=0x7fd038005fd0 comp rx=0 tx=0).stop 2024-03-21T10:34:21.886 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:21.885+0000 7fd04be00640 1 -- 172.21.15.47:0/4165651614 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd04c107e50 msgr2=0x7fd04c19e360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:34:21.886 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:21.885+0000 7fd04be00640 1 --2- 172.21.15.47:0/4165651614 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd04c107e50 0x7fd04c19e360 secure :-1 s=READY pgs=1234 cs=0 l=1 rev1=1 crypto rx=0x7fd03c00d950 tx=0x7fd03c00de10 comp rx=0 tx=0).stop 2024-03-21T10:34:21.886 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:21.886+0000 7fd04be00640 1 -- 172.21.15.47:0/4165651614 shutdown_connections 2024-03-21T10:34:21.887 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:21.886+0000 7fd04be00640 1 --2- 172.21.15.47:0/4165651614 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd04c1a2c20 0x7fd04c1a30d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:21.887 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:21.886+0000 7fd04be00640 1 --2- 172.21.15.47:0/4165651614 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7fd0300a3090 0x7fd0300a5550 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:21.887 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:21.886+0000 7fd04be00640 1 --2- 172.21.15.47:0/4165651614 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd04c10cea0 0x7fd04c19e8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:21.887 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:21.886+0000 7fd04be00640 1 --2- 172.21.15.47:0/4165651614 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd04c107e50 0x7fd04c19e360 unknown :-1 s=CLOSED pgs=1234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:21.887 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:21.886+0000 7fd04be00640 1 -- 172.21.15.47:0/4165651614 >> 172.21.15.47:0/4165651614 conn(0x7fd04c0774c0 msgr2=0x7fd04c079110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:34:21.887 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:21.886+0000 7fd04be00640 1 -- 172.21.15.47:0/4165651614 shutdown_connections 2024-03-21T10:34:21.887 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:21.886+0000 7fd04be00640 1 -- 172.21.15.47:0/4165651614 wait complete. 2024-03-21T10:34:21.908 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-21T10:34:21.908 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-21T10:34:22.134 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:22.135+0000 7fcfda800640 1 -- 172.21.15.47:0/3982329023 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fcfd4101630 msgr2=0x7fcfd4101ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:34:22.134 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:22.135+0000 7fcfda800640 1 --2- 172.21.15.47:0/3982329023 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fcfd4101630 0x7fcfd4101ab0 secure :-1 s=READY pgs=1238 cs=0 l=1 rev1=1 crypto rx=0x7fcfbc009930 tx=0x7fcfbc02bc20 comp rx=0 tx=0).stop 2024-03-21T10:34:22.134 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:22.135+0000 7fcfda800640 1 -- 172.21.15.47:0/3982329023 shutdown_connections 2024-03-21T10:34:22.134 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:22.135+0000 7fcfda800640 1 --2- 172.21.15.47:0/3982329023 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcfd4101ff0 0x7fcfd410e500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:22.134 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:22.135+0000 7fcfda800640 1 --2- 172.21.15.47:0/3982329023 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fcfd4101630 0x7fcfd4101ab0 unknown :-1 s=CLOSED pgs=1238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:22.134 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:22.135+0000 7fcfda800640 1 --2- 172.21.15.47:0/3982329023 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcfd4107640 0x7fcfd4107a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:22.135 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:22.135+0000 7fcfda800640 1 -- 172.21.15.47:0/3982329023 >> 172.21.15.47:0/3982329023 conn(0x7fcfd4078aa0 msgr2=0x7fcfd4078eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:34:22.135 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:22.136+0000 7fcfda800640 1 -- 172.21.15.47:0/3982329023 shutdown_connections 2024-03-21T10:34:22.135 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:22.136+0000 7fcfda800640 1 -- 172.21.15.47:0/3982329023 wait complete. 2024-03-21T10:34:22.135 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:22.137+0000 7fcfda800640 1 Processor -- start 2024-03-21T10:34:22.135 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:22.137+0000 7fcfda800640 1 -- start start 2024-03-21T10:34:22.137 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:22.138+0000 7fcfda800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcfd4101630 0x7fcfd40739a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:34:22.137 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:22.138+0000 7fcfda800640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fcfd4101ff0 0x7fcfd4073ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:34:22.137 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:22.138+0000 7fcfda800640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcfd4107640 0x7fcfd4074420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:34:22.137 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:22.138+0000 7fcfda800640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcfd4111900 con 0x7fcfd4101ff0 2024-03-21T10:34:22.137 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:22.138+0000 7fcfda800640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcfd4111780 con 0x7fcfd4101630 2024-03-21T10:34:22.137 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:22.138+0000 7fcfda800640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcfd4111a80 con 0x7fcfd4107640 2024-03-21T10:34:22.138 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:22.138+0000 7fcfcaa00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fcfd4101ff0 0x7fcfd4073ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:34:22.138 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:22.138+0000 7fcfcaa00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fcfd4101ff0 0x7fcfd4073ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:33024/0 (socket says 172.21.15.47:33024) 2024-03-21T10:34:22.138 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:22.138+0000 7fcfcaa00640 1 -- 172.21.15.47:0/951943043 learned_addr learned my addr 172.21.15.47:0/951943043 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:34:22.138 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:22.138+0000 7fcfd3400640 1 --2- 172.21.15.47:0/951943043 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcfd4101630 0x7fcfd40739a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:34:22.138 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:22.139+0000 7fcfd3e00640 1 --2- 172.21.15.47:0/951943043 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcfd4107640 0x7fcfd4074420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:34:22.138 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:22.139+0000 7fcfcaa00640 1 -- 172.21.15.47:0/951943043 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcfd4107640 msgr2=0x7fcfd4074420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:34:22.138 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:22.139+0000 7fcfcaa00640 1 --2- 172.21.15.47:0/951943043 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcfd4107640 0x7fcfd4074420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:22.138 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:22.139+0000 7fcfcaa00640 1 -- 172.21.15.47:0/951943043 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcfd4101630 msgr2=0x7fcfd40739a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:34:22.138 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:22.139+0000 7fcfcaa00640 1 --2- 172.21.15.47:0/951943043 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcfd4101630 0x7fcfd40739a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:22.138 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:22.139+0000 7fcfcaa00640 1 -- 172.21.15.47:0/951943043 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcfd41a6360 con 0x7fcfd4101ff0 2024-03-21T10:34:22.138 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:22.139+0000 7fcfcaa00640 1 --2- 172.21.15.47:0/951943043 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fcfd4101ff0 0x7fcfd4073ee0 secure :-1 s=READY pgs=1239 cs=0 l=1 rev1=1 crypto rx=0x7fcfbc009230 tx=0x7fcfbc0045e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:34:22.139 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:22.139+0000 7fcfd0c00640 1 -- 172.21.15.47:0/951943043 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcfbc043020 con 0x7fcfd4101ff0 2024-03-21T10:34:22.139 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:22.140+0000 7fcfda800640 1 -- 172.21.15.47:0/951943043 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcfd41a65f0 con 0x7fcfd4101ff0 2024-03-21T10:34:22.139 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:22.140+0000 7fcfd0c00640 1 -- 172.21.15.47:0/951943043 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fcfbc02cb60 con 0x7fcfd4101ff0 2024-03-21T10:34:22.140 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:22.140+0000 7fcfd0c00640 1 -- 172.21.15.47:0/951943043 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcfbc003b50 con 0x7fcfd4101ff0 2024-03-21T10:34:22.140 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:22.140+0000 7fcfda800640 1 -- 172.21.15.47:0/951943043 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcfd41a6b90 con 0x7fcfd4101ff0 2024-03-21T10:34:22.142 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:22.144+0000 7fcfdd423740 1 Processor -- start 2024-03-21T10:34:22.142 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:22.144+0000 7fcfdd423740 1 -- start start 2024-03-21T10:34:22.144 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:22.144+0000 7fcfdd423740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55cc0c9fb410 0x55cc0c9fa160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:34:22.144 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:22.145+0000 7fcfdd423740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55cc0c9fa730 0x55cc0c9fabb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:34:22.145 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:22.145+0000 7fcfdd423740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55cc0ca03aa0 0x55cc0ca05e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:34:22.145 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:22.145+0000 7fcfdd423740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cc0c956250 con 0x55cc0c9fa730 2024-03-21T10:34:22.145 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:22.145+0000 7fcfdd423740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cc0c8fea30 con 0x55cc0ca03aa0 2024-03-21T10:34:22.145 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:22.145+0000 7fcfdd423740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cc0c9522f0 con 0x55cc0c9fb410 2024-03-21T10:34:22.145 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:22.145+0000 7fcfd0c00640 1 -- 172.21.15.47:0/951943043 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 27) v1 ==== 137985+0+0 (secure 0 0 0) 0x7fcfbc046020 con 0x7fcfd4101ff0 2024-03-21T10:34:22.145 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:22.145+0000 7fcfd3400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55cc0c9fa730 0x55cc0c9fabb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:34:22.145 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:22.145+0000 7fcfd3400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55cc0c9fa730 0x55cc0c9fabb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:33032/0 (socket says 172.21.15.47:33032) 2024-03-21T10:34:22.145 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:22.145+0000 7fcfd3400640 1 -- 172.21.15.47:0/2135623235 learned_addr learned my addr 172.21.15.47:0/2135623235 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:34:22.145 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:22.145+0000 7fcfd3e00640 1 --2- 172.21.15.47:0/2135623235 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55cc0c9fb410 0x55cc0c9fa160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:34:22.145 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:22.145+0000 7fcfcaa00640 1 --2- 172.21.15.47:0/2135623235 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55cc0ca03aa0 0x55cc0ca05e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:34:22.145 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:22.145+0000 7fcfd3400640 1 -- 172.21.15.47:0/2135623235 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55cc0c9fb410 msgr2=0x55cc0c9fa160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:34:22.145 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:22.145+0000 7fcfd3400640 1 --2- 172.21.15.47:0/2135623235 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55cc0c9fb410 0x55cc0c9fa160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:22.146 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:22.145+0000 7fcfd3400640 1 -- 172.21.15.47:0/2135623235 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55cc0ca03aa0 msgr2=0x55cc0ca05e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:34:22.146 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:22.145+0000 7fcfd3400640 1 --2- 172.21.15.47:0/2135623235 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55cc0ca03aa0 0x55cc0ca05e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:22.146 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:22.146+0000 7fcfd3400640 1 -- 172.21.15.47:0/2135623235 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55cc0c8e9690 con 0x55cc0c9fa730 2024-03-21T10:34:22.146 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:22.146+0000 7fcfcaa00640 1 --2- 172.21.15.47:0/2135623235 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55cc0ca03aa0 0x55cc0ca05e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T10:34:22.146 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:22.146+0000 7fcfd3400640 1 --2- 172.21.15.47:0/2135623235 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55cc0c9fa730 0x55cc0c9fabb0 secure :-1 s=READY pgs=1240 cs=0 l=1 rev1=1 crypto rx=0x7fcfc4010370 tx=0x7fcfc4010830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=223b9082e02b33a2 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:34:22.146 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:22.146+0000 7fcfc8c00640 1 -- 172.21.15.47:0/2135623235 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcfc4016070 con 0x55cc0c9fa730 2024-03-21T10:34:22.146 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:22.146+0000 7fcfc8c00640 1 -- 172.21.15.47:0/2135623235 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fcfc4004d20 con 0x55cc0c9fa730 2024-03-21T10:34:22.146 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:22.146+0000 7fcfc8c00640 1 -- 172.21.15.47:0/2135623235 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcfc4005360 con 0x55cc0c9fa730 2024-03-21T10:34:22.150 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:22.147+0000 7fcfdd423740 1 -- 172.21.15.47:0/2135623235 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55cc0c9fa730 msgr2=0x55cc0c9fabb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:34:22.151 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:22.147+0000 7fcfdd423740 1 --2- 172.21.15.47:0/2135623235 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55cc0c9fa730 0x55cc0c9fabb0 secure :-1 s=READY pgs=1240 cs=0 l=1 rev1=1 crypto rx=0x7fcfc4010370 tx=0x7fcfc4010830 comp rx=0 tx=0).stop 2024-03-21T10:34:22.151 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:22.147+0000 7fcfdd423740 1 -- 172.21.15.47:0/2135623235 shutdown_connections 2024-03-21T10:34:22.151 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:22.147+0000 7fcfdd423740 1 --2- 172.21.15.47:0/2135623235 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55cc0ca03aa0 0x55cc0ca05e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:22.151 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:22.147+0000 7fcfdd423740 1 --2- 172.21.15.47:0/2135623235 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55cc0c9fa730 0x55cc0c9fabb0 unknown :-1 s=CLOSED pgs=1240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:22.151 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:22.147+0000 7fcfdd423740 1 --2- 172.21.15.47:0/2135623235 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55cc0c9fb410 0x55cc0c9fa160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:22.151 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:22.147+0000 7fcfdd423740 1 -- 172.21.15.47:0/2135623235 >> 172.21.15.47:0/2135623235 conn(0x55cc0c9b9200 msgr2=0x55cc0c9f5180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:34:22.151 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:22.147+0000 7fcfdd423740 1 -- 172.21.15.47:0/2135623235 shutdown_connections 2024-03-21T10:34:22.151 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:22.147+0000 7fcfd0c00640 1 --2- 172.21.15.47:0/951943043 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7fcfb80a3020 0x7fcfb80a54e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:34:22.151 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:22.147+0000 7fcfdd423740 1 -- 172.21.15.47:0/2135623235 wait complete. 2024-03-21T10:34:22.151 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:22.148+0000 7fcfd0c00640 1 -- 172.21.15.47:0/951943043 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 9562+0+0 (secure 0 0 0) 0x7fcfbc0f1ba0 con 0x7fcfd4101ff0 2024-03-21T10:34:22.151 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:22.148+0000 7fcfd3400640 1 --2- 172.21.15.47:0/951943043 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7fcfb80a3020 0x7fcfb80a54e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:34:22.152 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:22.148+0000 7fcfdd423740 1 -- start start 2024-03-21T10:34:22.152 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:22.148+0000 7fcfdd423740 1 Processor -- start 2024-03-21T10:34:22.152 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:22.148+0000 7fcfd3400640 1 --2- 172.21.15.47:0/951943043 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7fcfb80a3020 0x7fcfb80a54e0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fcfc4005060 tx=0x7fcfc4012f90 comp rx=0 tx=0).ready entity=mgr.15687 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:34:22.152 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:22.149+0000 7fcfdd423740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55cc0c9fa730 0x55cc0ca03090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:34:22.152 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:22.149+0000 7fcfdd423740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55cc0c9fb410 0x55cc0ca00420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:34:22.152 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:22.149+0000 7fcfdd423740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55cc0ca03aa0 0x55cc0ca00960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:34:22.152 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:22.149+0000 7fcfdd423740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cc0ca06910 con 0x55cc0ca03aa0 2024-03-21T10:34:22.152 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:22.149+0000 7fcfdd423740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cc0c911e80 con 0x55cc0c9fa730 2024-03-21T10:34:22.152 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:22.149+0000 7fcfdd423740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cc0ca06a90 con 0x55cc0c9fb410 2024-03-21T10:34:22.152 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:22.149+0000 7fcfdd423740 10 client.0 _ll_register_callbacks cb 0x55cc0c924130 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T10:34:22.152 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:22.149+0000 7fcfd3e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55cc0ca03aa0 0x55cc0ca00960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:34:22.153 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:22.149+0000 7fcfd3400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55cc0c9fa730 0x55cc0ca03090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:34:22.153 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:22.149+0000 7fcfd3e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55cc0ca03aa0 0x55cc0ca00960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:33048/0 (socket says 172.21.15.47:33048) 2024-03-21T10:34:22.153 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:22.149+0000 7fcfd3e00640 1 -- 172.21.15.47:0/437550332 learned_addr learned my addr 172.21.15.47:0/437550332 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:34:22.153 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:22.149+0000 7fcfcaa00640 1 --2- 172.21.15.47:0/437550332 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55cc0c9fb410 0x55cc0ca00420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:34:22.153 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:22.149+0000 7fcfd3e00640 1 -- 172.21.15.47:0/437550332 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55cc0c9fb410 msgr2=0x55cc0ca00420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:34:22.153 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:22.149+0000 7fcfd3e00640 1 --2- 172.21.15.47:0/437550332 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55cc0c9fb410 0x55cc0ca00420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:22.153 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:22.149+0000 7fcfd3e00640 1 -- 172.21.15.47:0/437550332 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55cc0c9fa730 msgr2=0x55cc0ca03090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:34:22.153 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:22.149+0000 7fcfd3e00640 1 --2- 172.21.15.47:0/437550332 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55cc0c9fa730 0x55cc0ca03090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:22.153 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:22.149+0000 7fcfd3e00640 1 -- 172.21.15.47:0/437550332 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55cc0c7347d0 con 0x55cc0ca03aa0 2024-03-21T10:34:22.153 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:22.150+0000 7fcfd3e00640 1 --2- 172.21.15.47:0/437550332 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55cc0ca03aa0 0x55cc0ca00960 secure :-1 s=READY pgs=1241 cs=0 l=1 rev1=1 crypto rx=0x7fcfcc00bf60 tx=0x7fcfcc00bf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b14c8ce1ec3b2262 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:34:22.153 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:22.150+0000 7fcfc1600640 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-03-21T10:34:22.153 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:22.150+0000 7fcfc1600640 1 -- 172.21.15.47:0/437550332 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcfcc012710 con 0x55cc0ca03aa0 2024-03-21T10:34:22.154 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:22.150+0000 7fcfc1600640 1 -- 172.21.15.47:0/437550332 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(103..103 src has 1..103) v4 ==== 9562+0+0 (secure 0 0 0) 0x7fcfcc0128b0 con 0x55cc0ca03aa0 2024-03-21T10:34:22.154 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:22.150+0000 7fcfdd423740 1 -- 172.21.15.47:0/437550332 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fcfc4004d20 con 0x55cc0ca03aa0 2024-03-21T10:34:22.154 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:22.150+0000 7fcfc1600640 1 -- 172.21.15.47:0/437550332 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7fcfcc012ab0 con 0x55cc0ca03aa0 2024-03-21T10:34:22.154 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:22.150+0000 7fcfdd423740 10 client.18555 fetch_fsmap learned FSMap version 32 2024-03-21T10:34:22.154 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:22.150+0000 7fcfdd423740 1 -- 172.21.15.47:0/437550332 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x55cc0c66a3f0 con 0x55cc0ca03aa0 2024-03-21T10:34:22.154 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:22.151+0000 7fcfc1600640 1 -- 172.21.15.47:0/437550332 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fcfc4004d20 con 0x55cc0ca03aa0 2024-03-21T10:34:22.154 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:22.151+0000 7fcfc1600640 1 -- 172.21.15.47:0/437550332 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 32) v1 ==== 7942+0+0 (secure 0 0 0) 0x7fcfcc01a3d0 con 0x55cc0ca03aa0 2024-03-21T10:34:22.154 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:22.151+0000 7fcfdd423740 10 client.18555 fetch_fsmap finished waiting for FSMap version 32 2024-03-21T10:34:22.154 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:22.151+0000 7fcfdd423740 10 client.18555 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-21T10:34:22.154 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:22.151+0000 7fcfdd423740 20 client.18555 populate_metadata read hostname 'smithi047' 2024-03-21T10:34:22.154 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:22.151+0000 7fcfdd423740 1 --2- 172.21.15.47:0/437550332 >> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] conn(0x55cc0ca139b0 0x55cc0ca15dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:34:22.154 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:22.151+0000 7fcfdd423740 4 client.18555 mds_command: new command op to 34535 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T10:34:22.155 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:22.151+0000 7fcfdd423740 1 -- 172.21.15.47:0/437550332 --> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55cc0c963790 con 0x55cc0ca139b0 2024-03-21T10:34:22.155 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:22.152+0000 7fcfd3400640 1 --2- 172.21.15.47:0/437550332 >> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] conn(0x55cc0ca139b0 0x55cc0ca15dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:34:22.155 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:22.152+0000 7fcfd3400640 1 --2- 172.21.15.47:0/437550332 >> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] conn(0x55cc0ca139b0 0x55cc0ca15dc0 crc :-1 s=READY pgs=211 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=842cbea21a7e53f3 server_cookie=71b53fde9e150eb2 in_seq=0 out_seq=0 2024-03-21T10:34:22.155 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:22.152+0000 7fcfc1600640 10 client.18555 ms_handle_connect on v2:172.21.15.100:6838/4173168440 2024-03-21T10:34:22.156 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:22.153+0000 7fcfc1600640 1 -- 172.21.15.47:0/437550332 <== mds.0 v2:172.21.15.100:6838/4173168440 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55cc0c963790 con 0x55cc0ca139b0 2024-03-21T10:34:22.156 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:22.153+0000 7fcfc1600640 10 client.18555 handle_command_reply: tid=0 2024-03-21T10:34:22.156 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:22.153+0000 7fcfdd423740 1 client.18555 shutdown 2024-03-21T10:34:22.156 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:22.154+0000 7fcfdd423740 1 -- 172.21.15.47:0/437550332 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55cc0ca03aa0 msgr2=0x55cc0ca00960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:34:22.156 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:22.154+0000 7fcfdd423740 1 --2- 172.21.15.47:0/437550332 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55cc0ca03aa0 0x55cc0ca00960 secure :-1 s=READY pgs=1241 cs=0 l=1 rev1=1 crypto rx=0x7fcfcc00bf60 tx=0x7fcfcc00bf90 comp rx=0 tx=0).stop 2024-03-21T10:34:22.157 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:22.154+0000 7fcfdd423740 1 -- 172.21.15.47:0/437550332 shutdown_connections 2024-03-21T10:34:22.157 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:22.154+0000 7fcfdd423740 1 --2- 172.21.15.47:0/437550332 >> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] conn(0x55cc0ca139b0 0x55cc0ca15dc0 crc :-1 s=READY pgs=211 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:22.157 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:22.154+0000 7fcfdd423740 1 --2- 172.21.15.47:0/437550332 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55cc0ca03aa0 0x55cc0ca00960 unknown :-1 s=CLOSED pgs=1241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:22.157 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:22.154+0000 7fcfdd423740 1 --2- 172.21.15.47:0/437550332 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55cc0c9fb410 0x55cc0ca00420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:22.157 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:22.154+0000 7fcfc1600640 0 client.18555 ms_handle_reset on v2:172.21.15.100:6838/4173168440 2024-03-21T10:34:22.157 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:22.154+0000 7fcfdd423740 1 --2- 172.21.15.47:0/437550332 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55cc0c9fa730 0x55cc0ca03090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:22.157 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:22.154+0000 7fcfdd423740 1 -- 172.21.15.47:0/437550332 >> 172.21.15.47:0/437550332 conn(0x55cc0c9b9200 msgr2=0x55cc0c9f6ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:34:22.157 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:22.154+0000 7fcfdd423740 1 -- 172.21.15.47:0/437550332 shutdown_connections 2024-03-21T10:34:22.157 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:22.154+0000 7fcfdd423740 1 -- 172.21.15.47:0/437550332 wait complete. 2024-03-21T10:34:22.158 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:22.154+0000 7fcfdd423740 20 client.18555 trim_cache size 0 max 16384 2024-03-21T10:34:22.171 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:22.172+0000 7fcfdd423740 1 Processor -- start 2024-03-21T10:34:22.172 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:22.172+0000 7fcfdd423740 1 -- start start 2024-03-21T10:34:22.172 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:22.172+0000 7fcfdd423740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55cc0c9fa730 0x55cc0ca02b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:34:22.172 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:22.172+0000 7fcfdd423740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55cc0c9fb410 0x55cc0ca03090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:34:22.172 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:22.172+0000 7fcfdd423740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55cc0ca03aa0 0x55cc0ca04e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:34:22.172 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:22.172+0000 7fcfdd423740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcfd41ac840 con 0x55cc0c9fa730 2024-03-21T10:34:22.172 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:22.172+0000 7fcfdd423740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcfd41ae1c0 con 0x55cc0c9fb410 2024-03-21T10:34:22.172 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:22.172+0000 7fcfdd423740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcfd41acea0 con 0x55cc0ca03aa0 2024-03-21T10:34:22.172 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:22.172+0000 7fcfd3400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55cc0c9fa730 0x55cc0ca02b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:34:22.172 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:22.172+0000 7fcfd3400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55cc0c9fa730 0x55cc0ca02b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:33062/0 (socket says 172.21.15.47:33062) 2024-03-21T10:34:22.172 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:22.172+0000 7fcfd3400640 1 -- 172.21.15.47:0/4035222289 learned_addr learned my addr 172.21.15.47:0/4035222289 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:34:22.172 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:22.172+0000 7fcfcaa00640 1 --2- 172.21.15.47:0/4035222289 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55cc0c9fb410 0x55cc0ca03090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:34:22.173 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:22.173+0000 7fcfd3e00640 1 --2- 172.21.15.47:0/4035222289 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55cc0ca03aa0 0x55cc0ca04e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:34:22.173 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:22.173+0000 7fcfd3400640 1 -- 172.21.15.47:0/4035222289 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55cc0ca03aa0 msgr2=0x55cc0ca04e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:34:22.173 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:22.173+0000 7fcfd3400640 1 --2- 172.21.15.47:0/4035222289 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55cc0ca03aa0 0x55cc0ca04e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:22.173 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:22.173+0000 7fcfd3400640 1 -- 172.21.15.47:0/4035222289 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55cc0c9fb410 msgr2=0x55cc0ca03090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:34:22.173 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:22.173+0000 7fcfd3400640 1 --2- 172.21.15.47:0/4035222289 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55cc0c9fb410 0x55cc0ca03090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:22.173 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:22.173+0000 7fcfd3400640 1 -- 172.21.15.47:0/4035222289 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55cc0c62a0f0 con 0x55cc0c9fa730 2024-03-21T10:34:22.173 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:22.173+0000 7fcfd3e00640 1 --2- 172.21.15.47:0/4035222289 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55cc0ca03aa0 0x55cc0ca04e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T10:34:22.173 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:22.173+0000 7fcfd3400640 1 --2- 172.21.15.47:0/4035222289 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55cc0c9fa730 0x55cc0ca02b50 secure :-1 s=READY pgs=1242 cs=0 l=1 rev1=1 crypto rx=0x7fcfc4009950 tx=0x7fcfc4023000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4c1fbe01bf69059d server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:34:22.173 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:22.173+0000 7fcfc3e00640 1 -- 172.21.15.47:0/4035222289 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcfc4004b70 con 0x55cc0c9fa730 2024-03-21T10:34:22.173 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:22.173+0000 7fcfc3e00640 1 -- 172.21.15.47:0/4035222289 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fcfc4004d10 con 0x55cc0c9fa730 2024-03-21T10:34:22.173 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:22.173+0000 7fcfc3e00640 1 -- 172.21.15.47:0/4035222289 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcfc4004520 con 0x55cc0c9fa730 2024-03-21T10:34:22.177 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:22.174+0000 7fcfdd423740 1 -- 172.21.15.47:0/4035222289 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55cc0c9fa730 msgr2=0x55cc0ca02b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:34:22.177 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:22.174+0000 7fcfdd423740 1 --2- 172.21.15.47:0/4035222289 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55cc0c9fa730 0x55cc0ca02b50 secure :-1 s=READY pgs=1242 cs=0 l=1 rev1=1 crypto rx=0x7fcfc4009950 tx=0x7fcfc4023000 comp rx=0 tx=0).stop 2024-03-21T10:34:22.177 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:22.174+0000 7fcfdd423740 1 -- 172.21.15.47:0/4035222289 shutdown_connections 2024-03-21T10:34:22.178 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:22.174+0000 7fcfdd423740 1 --2- 172.21.15.47:0/4035222289 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55cc0ca03aa0 0x55cc0ca04e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:22.178 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:22.174+0000 7fcfdd423740 1 --2- 172.21.15.47:0/4035222289 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55cc0c9fb410 0x55cc0ca03090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:22.178 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:22.174+0000 7fcfdd423740 1 --2- 172.21.15.47:0/4035222289 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55cc0c9fa730 0x55cc0ca02b50 unknown :-1 s=CLOSED pgs=1242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:22.178 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:22.174+0000 7fcfdd423740 1 -- 172.21.15.47:0/4035222289 >> 172.21.15.47:0/4035222289 conn(0x55cc0c9b9200 msgr2=0x55cc0c9f4cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:34:22.178 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:22.174+0000 7fcfdd423740 1 -- 172.21.15.47:0/4035222289 shutdown_connections 2024-03-21T10:34:22.178 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:22.174+0000 7fcfdd423740 1 -- 172.21.15.47:0/4035222289 wait complete. 2024-03-21T10:34:22.178 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:22.175+0000 7fcfdd423740 1 -- start start 2024-03-21T10:34:22.178 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:22.175+0000 7fcfdd423740 1 Processor -- start 2024-03-21T10:34:22.178 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:22.175+0000 7fcfdd423740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55cc0c9fa730 0x55cc0c9fd750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:34:22.178 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:22.176+0000 7fcfdd423740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55cc0c9fb410 0x55cc0c9fdc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:34:22.178 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:22.176+0000 7fcfdd423740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55cc0ca03aa0 0x55cc0ca67210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:34:22.178 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:22.176+0000 7fcfdd423740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcfd41adb60 con 0x55cc0c9fa730 2024-03-21T10:34:22.179 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:22.176+0000 7fcfdd423740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcfd41ad500 con 0x55cc0ca03aa0 2024-03-21T10:34:22.179 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:22.176+0000 7fcfdd423740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcfd41aaa20 con 0x55cc0c9fb410 2024-03-21T10:34:22.179 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:22.176+0000 7fcfdd423740 10 client.0 _ll_register_callbacks cb 0x7fcfcc00b480 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T10:34:22.179 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:22.176+0000 7fcfd3400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55cc0c9fa730 0x55cc0c9fd750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:34:22.179 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:22.176+0000 7fcfd3e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55cc0ca03aa0 0x55cc0ca67210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:34:22.179 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:22.176+0000 7fcfd3400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55cc0c9fa730 0x55cc0c9fd750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:33074/0 (socket says 172.21.15.47:33074) 2024-03-21T10:34:22.179 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:22.176+0000 7fcfd3400640 1 -- 172.21.15.47:0/3214280388 learned_addr learned my addr 172.21.15.47:0/3214280388 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:34:22.179 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:22.176+0000 7fcfcaa00640 1 --2- 172.21.15.47:0/3214280388 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55cc0c9fb410 0x55cc0c9fdc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:34:22.179 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:22.176+0000 7fcfd3e00640 1 -- 172.21.15.47:0/3214280388 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55cc0c9fb410 msgr2=0x55cc0c9fdc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:34:22.179 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:22.176+0000 7fcfd3e00640 1 --2- 172.21.15.47:0/3214280388 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55cc0c9fb410 0x55cc0c9fdc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:22.180 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:22.176+0000 7fcfd3e00640 1 -- 172.21.15.47:0/3214280388 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55cc0c9fa730 msgr2=0x55cc0c9fd750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:34:22.180 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:22.176+0000 7fcfd3e00640 1 --2- 172.21.15.47:0/3214280388 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55cc0c9fa730 0x55cc0c9fd750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:22.180 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:22.176+0000 7fcfd3e00640 1 -- 172.21.15.47:0/3214280388 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55cc0ca67a10 con 0x55cc0ca03aa0 2024-03-21T10:34:22.180 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:22.176+0000 7fcfd3400640 1 --2- 172.21.15.47:0/3214280388 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55cc0c9fa730 0x55cc0c9fd750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T10:34:22.180 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:22.177+0000 7fcfd3e00640 1 --2- 172.21.15.47:0/3214280388 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55cc0ca03aa0 0x55cc0ca67210 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7fcfcc01a7c0 tx=0x7fcfcc011e40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d0002d1e3726ca84 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:34:22.180 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:22.177+0000 7fcfc1600640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-21T10:34:22.180 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:22.177+0000 7fcfc1600640 1 -- 172.21.15.47:0/3214280388 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcfcc01a5d0 con 0x55cc0ca03aa0 2024-03-21T10:34:22.180 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:22.177+0000 7fcfdd423740 1 -- 172.21.15.47:0/3214280388 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55cc0c95dae0 con 0x55cc0ca03aa0 2024-03-21T10:34:22.180 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:22.177+0000 7fcfc1600640 1 -- 172.21.15.47:0/3214280388 <== mon.2 v2:172.21.15.100:3300/0 2 ==== osd_map(103..103 src has 1..103) v4 ==== 9562+0+0 (secure 0 0 0) 0x7fcfcc012510 con 0x55cc0ca03aa0 2024-03-21T10:34:22.180 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:22.178+0000 7fcfc1600640 1 -- 172.21.15.47:0/3214280388 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7fcfcc012710 con 0x55cc0ca03aa0 2024-03-21T10:34:22.180 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:22.178+0000 7fcfdd423740 10 client.38327 fetch_fsmap learned FSMap version 32 2024-03-21T10:34:22.180 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:22.178+0000 7fcfdd423740 1 -- 172.21.15.47:0/3214280388 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x55cc0ca6fcf0 con 0x55cc0ca03aa0 2024-03-21T10:34:22.180 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:22.178+0000 7fcfc1600640 1 -- 172.21.15.47:0/3214280388 <== mon.2 v2:172.21.15.100:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x55cc0c95dae0 con 0x55cc0ca03aa0 2024-03-21T10:34:22.180 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:22.178+0000 7fcfc1600640 1 -- 172.21.15.47:0/3214280388 <== mon.2 v2:172.21.15.100:3300/0 5 ==== fsmap(e 32) v1 ==== 7942+0+0 (secure 0 0 0) 0x7fcfcc005350 con 0x55cc0ca03aa0 2024-03-21T10:34:22.181 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:22.178+0000 7fcfdd423740 10 client.38327 fetch_fsmap finished waiting for FSMap version 32 2024-03-21T10:34:22.181 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:22.178+0000 7fcfdd423740 10 client.38327 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-21T10:34:22.181 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:22.178+0000 7fcfdd423740 20 client.38327 populate_metadata read hostname 'smithi047' 2024-03-21T10:34:22.181 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:22.178+0000 7fcfdd423740 1 --2- 172.21.15.47:0/3214280388 >> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] conn(0x55cc0ca703c0 0x55cc0ca727e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:34:22.181 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:22.178+0000 7fcfdd423740 4 client.38327 mds_command: new command op to 34535 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-21T10:34:22.181 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:22.178+0000 7fcfdd423740 1 -- 172.21.15.47:0/3214280388 --> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55cc0c8db3a0 con 0x55cc0ca703c0 2024-03-21T10:34:22.181 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:22.179+0000 7fcfd3400640 1 --2- 172.21.15.47:0/3214280388 >> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] conn(0x55cc0ca703c0 0x55cc0ca727e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:34:22.181 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:22.179+0000 7fcfd3400640 1 --2- 172.21.15.47:0/3214280388 >> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] conn(0x55cc0ca703c0 0x55cc0ca727e0 crc :-1 s=READY pgs=212 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=befbb1598084e176 server_cookie=89506c4bd8ac48b7 in_seq=0 out_seq=0 2024-03-21T10:34:22.181 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:22.179+0000 7fcfc1600640 10 client.38327 ms_handle_connect on v2:172.21.15.100:6838/4173168440 2024-03-21T10:34:22.182 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:22.180+0000 7fcfc1600640 1 -- 172.21.15.47:0/3214280388 <== mds.0 v2:172.21.15.100:6838/4173168440 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x55cc0c8db3a0 con 0x55cc0ca703c0 2024-03-21T10:34:22.182 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:22.180+0000 7fcfc1600640 10 client.38327 handle_command_reply: tid=0 2024-03-21T10:34:22.182 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:22.180+0000 7fcfdd423740 1 client.38327 shutdown 2024-03-21T10:34:22.182 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:22.180+0000 7fcfdd423740 1 -- 172.21.15.47:0/3214280388 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55cc0ca03aa0 msgr2=0x55cc0ca67210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:34:22.183 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:22.180+0000 7fcfdd423740 1 --2- 172.21.15.47:0/3214280388 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55cc0ca03aa0 0x55cc0ca67210 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7fcfcc01a7c0 tx=0x7fcfcc011e40 comp rx=0 tx=0).stop 2024-03-21T10:34:22.183 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:22.180+0000 7fcfdd423740 1 -- 172.21.15.47:0/3214280388 shutdown_connections 2024-03-21T10:34:22.183 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:22.180+0000 7fcfdd423740 1 --2- 172.21.15.47:0/3214280388 >> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] conn(0x55cc0ca703c0 0x55cc0ca727e0 crc :-1 s=READY pgs=212 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:22.183 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:22.180+0000 7fcfdd423740 1 --2- 172.21.15.47:0/3214280388 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55cc0ca03aa0 0x55cc0ca67210 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:22.183 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:22.180+0000 7fcfdd423740 1 --2- 172.21.15.47:0/3214280388 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55cc0c9fb410 0x55cc0c9fdc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:22.183 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:22.180+0000 7fcfdd423740 1 --2- 172.21.15.47:0/3214280388 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55cc0c9fa730 0x55cc0c9fd750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:22.183 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:22.180+0000 7fcfdd423740 1 -- 172.21.15.47:0/3214280388 >> 172.21.15.47:0/3214280388 conn(0x55cc0c9b9200 msgr2=0x55cc0c9eb200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:34:22.183 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:22.180+0000 7fcfc1600640 0 client.38327 ms_handle_reset on v2:172.21.15.100:6838/4173168440 2024-03-21T10:34:22.183 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:22.180+0000 7fcfdd423740 1 -- 172.21.15.47:0/3214280388 shutdown_connections 2024-03-21T10:34:22.183 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:22.180+0000 7fcfdd423740 1 -- 172.21.15.47:0/3214280388 wait complete. 2024-03-21T10:34:22.183 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:22.180+0000 7fcfdd423740 20 client.38327 trim_cache size 0 max 16384 2024-03-21T10:34:22.184 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-03-21T10:34:22.184 INFO:teuthology.orchestra.run.smithi047.stdout: "status": "IDLE (waiting for more scrubs)", 2024-03-21T10:34:22.184 INFO:teuthology.orchestra.run.smithi047.stdout: "scrubs": { 2024-03-21T10:34:22.184 INFO:teuthology.orchestra.run.smithi047.stdout: "e5a79add-446e-4805-8b0f-34329e0854e1": { 2024-03-21T10:34:22.184 INFO:teuthology.orchestra.run.smithi047.stdout: "path": "/", 2024-03-21T10:34:22.184 INFO:teuthology.orchestra.run.smithi047.stdout: "tag": "e5a79add-446e-4805-8b0f-34329e0854e1", 2024-03-21T10:34:22.184 INFO:teuthology.orchestra.run.smithi047.stdout: "options": "recursive,force" 2024-03-21T10:34:22.184 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-21T10:34:22.184 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-21T10:34:22.184 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-03-21T10:34:22.185 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:22.181+0000 7fcfda800640 1 -- 172.21.15.47:0/951943043 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7fcfb80a3020 msgr2=0x7fcfb80a54e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:34:22.185 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:22.181+0000 7fcfda800640 1 --2- 172.21.15.47:0/951943043 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7fcfb80a3020 0x7fcfb80a54e0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fcfc4005060 tx=0x7fcfc4012f90 comp rx=0 tx=0).stop 2024-03-21T10:34:22.185 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:22.181+0000 7fcfda800640 1 -- 172.21.15.47:0/951943043 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fcfd4101ff0 msgr2=0x7fcfd4073ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:34:22.185 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:22.181+0000 7fcfda800640 1 --2- 172.21.15.47:0/951943043 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fcfd4101ff0 0x7fcfd4073ee0 secure :-1 s=READY pgs=1239 cs=0 l=1 rev1=1 crypto rx=0x7fcfbc009230 tx=0x7fcfbc0045e0 comp rx=0 tx=0).stop 2024-03-21T10:34:22.185 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:22.181+0000 7fcfda800640 1 -- 172.21.15.47:0/951943043 shutdown_connections 2024-03-21T10:34:22.185 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:22.181+0000 7fcfda800640 1 --2- 172.21.15.47:0/951943043 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcfd4107640 0x7fcfd4074420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:22.185 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:22.181+0000 7fcfda800640 1 --2- 172.21.15.47:0/951943043 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fcfd4101ff0 0x7fcfd4073ee0 unknown :-1 s=CLOSED pgs=1239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:22.185 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:22.181+0000 7fcfda800640 1 --2- 172.21.15.47:0/951943043 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7fcfb80a3020 0x7fcfb80a54e0 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:22.185 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:22.181+0000 7fcfda800640 1 --2- 172.21.15.47:0/951943043 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcfd4101630 0x7fcfd40739a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:22.186 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:22.181+0000 7fcfda800640 1 -- 172.21.15.47:0/951943043 >> 172.21.15.47:0/951943043 conn(0x7fcfd4078aa0 msgr2=0x7fcfd4104110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:34:22.186 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:22.181+0000 7fcfda800640 1 -- 172.21.15.47:0/951943043 shutdown_connections 2024-03-21T10:34:22.186 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:22.181+0000 7fcfda800640 1 -- 172.21.15.47:0/951943043 wait complete. 2024-03-21T10:34:22.196 INFO:tasks.cephfs.filesystem:scrub status for tag:e5a79add-446e-4805-8b0f-34329e0854e1 - {'path': '/', 'tag': 'e5a79add-446e-4805-8b0f-34329e0854e1', 'options': 'recursive,force'} 2024-03-21T10:34:23.711 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:34:23 smithi047 ceph-mon[31547]: scrub queued for path: / 2024-03-21T10:34:23.711 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:34:23 smithi047 ceph-mon[31547]: scrub summary: idle+waiting paths [/] 2024-03-21T10:34:23.711 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:34:23 smithi047 ceph-mon[31547]: scrub summary: active paths [/] 2024-03-21T10:34:23.711 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:34:23 smithi047 ceph-mon[31547]: scrub summary: idle+waiting paths [/] 2024-03-21T10:34:23.711 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:34:23 smithi047 ceph-mon[31547]: pgmap v281: 129 pgs: 129 active+clean; 2.5 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 102 B/s wr, 11 op/s 2024-03-21T10:34:23.726 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:34:23 smithi100 ceph-mon[35193]: scrub queued for path: / 2024-03-21T10:34:23.727 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:34:23 smithi100 ceph-mon[35193]: scrub summary: idle+waiting paths [/] 2024-03-21T10:34:23.727 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:34:23 smithi100 ceph-mon[35193]: scrub summary: active paths [/] 2024-03-21T10:34:23.727 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:34:23 smithi100 ceph-mon[35193]: scrub summary: idle+waiting paths [/] 2024-03-21T10:34:23.727 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:34:23 smithi100 ceph-mon[35193]: pgmap v281: 129 pgs: 129 active+clean; 2.5 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 102 B/s wr, 11 op/s 2024-03-21T10:34:23.737 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:34:23 smithi157 ceph-mon[35581]: scrub queued for path: / 2024-03-21T10:34:23.737 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:34:23 smithi157 ceph-mon[35581]: scrub summary: idle+waiting paths [/] 2024-03-21T10:34:23.737 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:34:23 smithi157 ceph-mon[35581]: scrub summary: active paths [/] 2024-03-21T10:34:23.737 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:34:23 smithi157 ceph-mon[35581]: scrub summary: idle+waiting paths [/] 2024-03-21T10:34:23.737 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:34:23 smithi157 ceph-mon[35581]: pgmap v281: 129 pgs: 129 active+clean; 2.5 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 102 B/s wr, 11 op/s 2024-03-21T10:34:24.711 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:34:24 smithi047 ceph-mon[31547]: scrub summary: idle 2024-03-21T10:34:24.726 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:34:24 smithi100 ceph-mon[35193]: scrub summary: idle 2024-03-21T10:34:24.737 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:34:24 smithi157 ceph-mon[35581]: scrub summary: idle 2024-03-21T10:34:25.711 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:34:25 smithi047 ceph-mon[31547]: pgmap v282: 129 pgs: 129 active+clean; 2.5 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 98 B/s wr, 10 op/s 2024-03-21T10:34:25.726 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:34:25 smithi100 ceph-mon[35193]: pgmap v282: 129 pgs: 129 active+clean; 2.5 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 98 B/s wr, 10 op/s 2024-03-21T10:34:25.737 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:34:25 smithi157 ceph-mon[35581]: pgmap v282: 129 pgs: 129 active+clean; 2.5 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 98 B/s wr, 10 op/s 2024-03-21T10:34:27.711 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:34:27 smithi047 ceph-mon[31547]: pgmap v283: 129 pgs: 129 active+clean; 2.5 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 8.0 KiB/s rd, 85 B/s wr, 13 op/s 2024-03-21T10:34:27.726 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:34:27 smithi100 ceph-mon[35193]: pgmap v283: 129 pgs: 129 active+clean; 2.5 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 8.0 KiB/s rd, 85 B/s wr, 13 op/s 2024-03-21T10:34:27.736 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:34:27 smithi157 ceph-mon[35581]: pgmap v283: 129 pgs: 129 active+clean; 2.5 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 8.0 KiB/s rd, 85 B/s wr, 13 op/s 2024-03-21T10:34:29.711 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:34:29 smithi047 ceph-mon[31547]: pgmap v284: 129 pgs: 129 active+clean; 2.5 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 10 op/s 2024-03-21T10:34:29.726 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:34:29 smithi100 ceph-mon[35193]: pgmap v284: 129 pgs: 129 active+clean; 2.5 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 10 op/s 2024-03-21T10:34:29.736 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:34:29 smithi157 ceph-mon[35581]: pgmap v284: 129 pgs: 129 active+clean; 2.5 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 10 op/s 2024-03-21T10:34:31.711 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:34:31 smithi047 ceph-mon[31547]: pgmap v285: 129 pgs: 129 active+clean; 2.5 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 10 op/s 2024-03-21T10:34:31.726 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:34:31 smithi100 ceph-mon[35193]: pgmap v285: 129 pgs: 129 active+clean; 2.5 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 10 op/s 2024-03-21T10:34:31.736 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:34:31 smithi157 ceph-mon[35581]: pgmap v285: 129 pgs: 129 active+clean; 2.5 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 10 op/s 2024-03-21T10:34:33.711 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:34:33 smithi047 ceph-mon[31547]: pgmap v286: 129 pgs: 129 active+clean; 2.5 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 8.6 KiB/s rd, 14 op/s 2024-03-21T10:34:33.726 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:34:33 smithi100 ceph-mon[35193]: pgmap v286: 129 pgs: 129 active+clean; 2.5 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 8.6 KiB/s rd, 14 op/s 2024-03-21T10:34:33.736 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:34:33 smithi157 ceph-mon[35581]: pgmap v286: 129 pgs: 129 active+clean; 2.5 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 8.6 KiB/s rd, 14 op/s 2024-03-21T10:34:35.711 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:34:35 smithi047 ceph-mon[31547]: pgmap v287: 129 pgs: 129 active+clean; 2.5 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 9 op/s 2024-03-21T10:34:35.726 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:34:35 smithi100 ceph-mon[35193]: pgmap v287: 129 pgs: 129 active+clean; 2.5 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 9 op/s 2024-03-21T10:34:35.736 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:34:35 smithi157 ceph-mon[35581]: pgmap v287: 129 pgs: 129 active+clean; 2.5 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 9 op/s 2024-03-21T10:34:37.711 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:34:37 smithi047 ceph-mon[31547]: pgmap v288: 129 pgs: 129 active+clean; 2.5 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 13 op/s 2024-03-21T10:34:37.727 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:34:37 smithi100 ceph-mon[35193]: pgmap v288: 129 pgs: 129 active+clean; 2.5 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 13 op/s 2024-03-21T10:34:37.737 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:34:37 smithi157 ceph-mon[35581]: pgmap v288: 129 pgs: 129 active+clean; 2.5 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 13 op/s 2024-03-21T10:34:39.711 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:34:39 smithi047 ceph-mon[31547]: pgmap v289: 129 pgs: 129 active+clean; 2.5 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 8 op/s 2024-03-21T10:34:39.726 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:34:39 smithi100 ceph-mon[35193]: pgmap v289: 129 pgs: 129 active+clean; 2.5 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 8 op/s 2024-03-21T10:34:39.736 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:34:39 smithi157 ceph-mon[35581]: pgmap v289: 129 pgs: 129 active+clean; 2.5 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 8 op/s 2024-03-21T10:34:41.711 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:34:41 smithi047 ceph-mon[31547]: pgmap v290: 129 pgs: 129 active+clean; 2.5 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-21T10:34:41.727 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:34:41 smithi100 ceph-mon[35193]: pgmap v290: 129 pgs: 129 active+clean; 2.5 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-21T10:34:41.737 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:34:41 smithi157 ceph-mon[35581]: pgmap v290: 129 pgs: 129 active+clean; 2.5 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-21T10:34:43.711 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:34:43 smithi047 ceph-mon[31547]: pgmap v291: 129 pgs: 129 active+clean; 2.5 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-03-21T10:34:43.726 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:34:43 smithi100 ceph-mon[35193]: pgmap v291: 129 pgs: 129 active+clean; 2.5 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-03-21T10:34:43.736 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:34:43 smithi157 ceph-mon[35581]: pgmap v291: 129 pgs: 129 active+clean; 2.5 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-03-21T10:34:45.711 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:34:45 smithi047 ceph-mon[31547]: pgmap v292: 129 pgs: 129 active+clean; 2.5 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-21T10:34:45.726 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:34:45 smithi100 ceph-mon[35193]: pgmap v292: 129 pgs: 129 active+clean; 2.5 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-21T10:34:45.736 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:34:45 smithi157 ceph-mon[35581]: pgmap v292: 129 pgs: 129 active+clean; 2.5 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-21T10:34:46.710 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:34:46 smithi047 ceph-mon[31547]: from='mgr.15687 172.21.15.47:0/8730661' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T10:34:46.726 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:34:46 smithi100 ceph-mon[35193]: from='mgr.15687 172.21.15.47:0/8730661' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T10:34:46.736 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:34:46 smithi157 ceph-mon[35581]: from='mgr.15687 172.21.15.47:0/8730661' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T10:34:47.711 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:34:47 smithi047 ceph-mon[31547]: pgmap v293: 129 pgs: 129 active+clean; 2.5 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 12 op/s 2024-03-21T10:34:47.711 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:34:47 smithi047 ceph-mon[31547]: from='mgr.15687 172.21.15.47:0/8730661' entity='mgr.x' 2024-03-21T10:34:47.711 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:34:47 smithi047 ceph-mon[31547]: from='mgr.15687 172.21.15.47:0/8730661' entity='mgr.x' 2024-03-21T10:34:47.726 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:34:47 smithi100 ceph-mon[35193]: pgmap v293: 129 pgs: 129 active+clean; 2.5 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 12 op/s 2024-03-21T10:34:47.726 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:34:47 smithi100 ceph-mon[35193]: from='mgr.15687 172.21.15.47:0/8730661' entity='mgr.x' 2024-03-21T10:34:47.727 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:34:47 smithi100 ceph-mon[35193]: from='mgr.15687 172.21.15.47:0/8730661' entity='mgr.x' 2024-03-21T10:34:47.736 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:34:47 smithi157 ceph-mon[35581]: pgmap v293: 129 pgs: 129 active+clean; 2.5 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 12 op/s 2024-03-21T10:34:47.736 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:34:47 smithi157 ceph-mon[35581]: from='mgr.15687 172.21.15.47:0/8730661' entity='mgr.x' 2024-03-21T10:34:47.736 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:34:47 smithi157 ceph-mon[35581]: from='mgr.15687 172.21.15.47:0/8730661' entity='mgr.x' 2024-03-21T10:34:48.711 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:34:48 smithi047 ceph-mon[31547]: from='mgr.15687 172.21.15.47:0/8730661' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T10:34:48.711 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:34:48 smithi047 ceph-mon[31547]: from='mgr.15687 172.21.15.47:0/8730661' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T10:34:48.711 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:34:48 smithi047 ceph-mon[31547]: from='mgr.15687 172.21.15.47:0/8730661' entity='mgr.x' 2024-03-21T10:34:48.726 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:34:48 smithi100 ceph-mon[35193]: from='mgr.15687 172.21.15.47:0/8730661' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T10:34:48.726 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:34:48 smithi100 ceph-mon[35193]: from='mgr.15687 172.21.15.47:0/8730661' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T10:34:48.727 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:34:48 smithi100 ceph-mon[35193]: from='mgr.15687 172.21.15.47:0/8730661' entity='mgr.x' 2024-03-21T10:34:48.737 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:34:48 smithi157 ceph-mon[35581]: from='mgr.15687 172.21.15.47:0/8730661' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T10:34:48.737 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:34:48 smithi157 ceph-mon[35581]: from='mgr.15687 172.21.15.47:0/8730661' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T10:34:48.737 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:34:48 smithi157 ceph-mon[35581]: from='mgr.15687 172.21.15.47:0/8730661' entity='mgr.x' 2024-03-21T10:34:49.628 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:34:49 smithi100 ceph-mon[35193]: pgmap v294: 129 pgs: 129 active+clean; 2.5 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-03-21T10:34:49.711 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:34:49 smithi047 ceph-mon[31547]: pgmap v294: 129 pgs: 129 active+clean; 2.5 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-03-21T10:34:49.737 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:34:49 smithi157 ceph-mon[35581]: pgmap v294: 129 pgs: 129 active+clean; 2.5 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-03-21T10:34:51.711 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:34:51 smithi047 ceph-mon[31547]: pgmap v295: 129 pgs: 129 active+clean; 2.5 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-21T10:34:51.726 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:34:51 smithi100 ceph-mon[35193]: pgmap v295: 129 pgs: 129 active+clean; 2.5 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-21T10:34:51.737 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:34:51 smithi157 ceph-mon[35581]: pgmap v295: 129 pgs: 129 active+clean; 2.5 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-21T10:34:52.197 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-21T10:34:52.198 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-21T10:34:52.438 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:52.440+0000 7fb343600640 1 -- 172.21.15.47:0/655178566 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb33c103840 msgr2=0x7fb33c103cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:34:52.439 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:52.440+0000 7fb343600640 1 --2- 172.21.15.47:0/655178566 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb33c103840 0x7fb33c103cc0 secure :-1 s=READY pgs=1243 cs=0 l=1 rev1=1 crypto rx=0x7fb328009930 tx=0x7fb32802bc30 comp rx=0 tx=0).stop 2024-03-21T10:34:52.439 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:52.440+0000 7fb343600640 1 -- 172.21.15.47:0/655178566 shutdown_connections 2024-03-21T10:34:52.439 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:52.440+0000 7fb343600640 1 --2- 172.21.15.47:0/655178566 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb33c104200 0x7fb33c110710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:52.439 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:52.440+0000 7fb343600640 1 --2- 172.21.15.47:0/655178566 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb33c103840 0x7fb33c103cc0 unknown :-1 s=CLOSED pgs=1243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:52.439 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:52.440+0000 7fb343600640 1 --2- 172.21.15.47:0/655178566 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb33c109850 0x7fb33c109c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:52.439 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:52.440+0000 7fb343600640 1 -- 172.21.15.47:0/655178566 >> 172.21.15.47:0/655178566 conn(0x7fb33c0ff310 msgr2=0x7fb33c101730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:34:52.439 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:52.440+0000 7fb343600640 1 -- 172.21.15.47:0/655178566 shutdown_connections 2024-03-21T10:34:52.439 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:52.440+0000 7fb343600640 1 -- 172.21.15.47:0/655178566 wait complete. 2024-03-21T10:34:52.441 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:52.441+0000 7fb343600640 1 Processor -- start 2024-03-21T10:34:52.441 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:52.442+0000 7fb343600640 1 -- start start 2024-03-21T10:34:52.441 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:52.442+0000 7fb343600640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb33c103840 0x7fb33c19a080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:34:52.441 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:52.442+0000 7fb343600640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb33c104200 0x7fb33c19a5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:34:52.441 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:52.442+0000 7fb343600640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb33c109850 0x7fb33c19e940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:34:52.441 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:52.442+0000 7fb343600640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb33c113b60 con 0x7fb33c104200 2024-03-21T10:34:52.441 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:52.442+0000 7fb343600640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb33c1139e0 con 0x7fb33c103840 2024-03-21T10:34:52.441 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:52.442+0000 7fb343600640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb33c113ce0 con 0x7fb33c109850 2024-03-21T10:34:52.442 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:52.443+0000 7fb33be00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb33c104200 0x7fb33c19a5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:34:52.442 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:52.443+0000 7fb340c00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb33c103840 0x7fb33c19a080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:34:52.442 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:52.443+0000 7fb340c00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb33c103840 0x7fb33c19a080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:45594/0 (socket says 172.21.15.47:45594) 2024-03-21T10:34:52.442 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:52.443+0000 7fb340c00640 1 -- 172.21.15.47:0/1348572539 learned_addr learned my addr 172.21.15.47:0/1348572539 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:34:52.442 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:52.443+0000 7fb341600640 1 --2- 172.21.15.47:0/1348572539 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb33c109850 0x7fb33c19e940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:34:52.442 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:52.443+0000 7fb340c00640 1 -- 172.21.15.47:0/1348572539 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb33c109850 msgr2=0x7fb33c19e940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:34:52.442 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:52.443+0000 7fb340c00640 1 --2- 172.21.15.47:0/1348572539 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb33c109850 0x7fb33c19e940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:52.442 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:52.443+0000 7fb340c00640 1 -- 172.21.15.47:0/1348572539 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb33c104200 msgr2=0x7fb33c19a5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:34:52.442 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:52.443+0000 7fb340c00640 1 --2- 172.21.15.47:0/1348572539 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb33c104200 0x7fb33c19a5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:52.442 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:52.443+0000 7fb340c00640 1 -- 172.21.15.47:0/1348572539 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb33c19f070 con 0x7fb33c103840 2024-03-21T10:34:52.443 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:52.443+0000 7fb340c00640 1 --2- 172.21.15.47:0/1348572539 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb33c103840 0x7fb33c19a080 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7fb33000d950 tx=0x7fb33000de10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:34:52.443 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:52.444+0000 7fb339600640 1 -- 172.21.15.47:0/1348572539 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb330017070 con 0x7fb33c103840 2024-03-21T10:34:52.444 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:52.444+0000 7fb343600640 1 -- 172.21.15.47:0/1348572539 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb33c19f360 con 0x7fb33c103840 2024-03-21T10:34:52.444 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:52.444+0000 7fb339600640 1 -- 172.21.15.47:0/1348572539 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fb330009c30 con 0x7fb33c103840 2024-03-21T10:34:52.444 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:52.444+0000 7fb339600640 1 -- 172.21.15.47:0/1348572539 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb33000f620 con 0x7fb33c103840 2024-03-21T10:34:52.444 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:52.444+0000 7fb343600640 1 -- 172.21.15.47:0/1348572539 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb33c1a6bf0 con 0x7fb33c103840 2024-03-21T10:34:52.445 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:52.447+0000 7fb3463e1740 1 Processor -- start 2024-03-21T10:34:52.445 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:52.447+0000 7fb3463e1740 1 -- start start 2024-03-21T10:34:52.446 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:52.448+0000 7fb3463e1740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555ea9002c00 0x555ea9001950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:34:52.446 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:52.448+0000 7fb3463e1740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x555ea9001f20 0x555ea90023a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:34:52.446 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:52.448+0000 7fb3463e1740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555ea900b290 0x555ea900d680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:34:52.446 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:52.448+0000 7fb3463e1740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x555ea8f66250 con 0x555ea9001f20 2024-03-21T10:34:52.446 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:52.448+0000 7fb3463e1740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x555ea8f0ea30 con 0x555ea9002c00 2024-03-21T10:34:52.446 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:52.448+0000 7fb3463e1740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x555ea8f622f0 con 0x555ea900b290 2024-03-21T10:34:52.447 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:52.449+0000 7fb33be00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555ea900b290 0x555ea900d680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:34:52.448 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:52.449+0000 7fb341600640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555ea9002c00 0x555ea9001950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:34:52.448 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:52.449+0000 7fb33be00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555ea900b290 0x555ea900d680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.47:44390/0 (socket says 172.21.15.47:44390) 2024-03-21T10:34:52.448 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:52.449+0000 7fb33be00640 1 -- 172.21.15.47:0/4281447420 learned_addr learned my addr 172.21.15.47:0/4281447420 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:34:52.448 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:52.449+0000 7fb341600640 1 -- 172.21.15.47:0/4281447420 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555ea900b290 msgr2=0x555ea900d680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:34:52.448 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:52.449+0000 7fb341600640 1 --2- 172.21.15.47:0/4281447420 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555ea900b290 0x555ea900d680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:52.448 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:52.449+0000 7fb341600640 1 -- 172.21.15.47:0/4281447420 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x555ea9001f20 msgr2=0x555ea90023a0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-21T10:34:52.448 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:52.449+0000 7fb341600640 1 --2- 172.21.15.47:0/4281447420 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x555ea9001f20 0x555ea90023a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:52.448 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:52.449+0000 7fb341600640 1 -- 172.21.15.47:0/4281447420 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555ea8ef9690 con 0x555ea9002c00 2024-03-21T10:34:52.448 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:52.449+0000 7fb339600640 1 -- 172.21.15.47:0/1348572539 <== mon.2 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 27) v1 ==== 137985+0+0 (secure 0 0 0) 0x7fb3300109a0 con 0x7fb33c103840 2024-03-21T10:34:52.448 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:52.449+0000 7fb341600640 1 --2- 172.21.15.47:0/4281447420 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555ea9002c00 0x555ea9001950 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7fb334009e20 tx=0x7fb334010740 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d409acfebe55e8b3 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:34:52.450 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:52.450+0000 7fb32dc00640 1 -- 172.21.15.47:0/4281447420 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb334016070 con 0x555ea9002c00 2024-03-21T10:34:52.450 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:52.450+0000 7fb32dc00640 1 -- 172.21.15.47:0/4281447420 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fb334004d20 con 0x555ea9002c00 2024-03-21T10:34:52.450 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:52.450+0000 7fb32dc00640 1 -- 172.21.15.47:0/4281447420 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3340127d0 con 0x555ea9002c00 2024-03-21T10:34:52.450 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:52.451+0000 7fb339600640 1 --2- 172.21.15.47:0/1348572539 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7fb3240a3090 0x7fb3240a5550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:34:52.450 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:52.451+0000 7fb339600640 1 -- 172.21.15.47:0/1348572539 <== mon.2 v2:172.21.15.100:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 9562+0+0 (secure 0 0 0) 0x7fb3300cf7a0 con 0x7fb33c103840 2024-03-21T10:34:52.450 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:52.451+0000 7fb3463e1740 1 -- 172.21.15.47:0/4281447420 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555ea9002c00 msgr2=0x555ea9001950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:34:52.450 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:52.451+0000 7fb3463e1740 1 --2- 172.21.15.47:0/4281447420 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555ea9002c00 0x555ea9001950 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7fb334009e20 tx=0x7fb334010740 comp rx=0 tx=0).stop 2024-03-21T10:34:52.451 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:52.451+0000 7fb340c00640 1 --2- 172.21.15.47:0/1348572539 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7fb3240a3090 0x7fb3240a5550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:34:52.451 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:52.451+0000 7fb3463e1740 1 -- 172.21.15.47:0/4281447420 shutdown_connections 2024-03-21T10:34:52.451 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:52.451+0000 7fb3463e1740 1 --2- 172.21.15.47:0/4281447420 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555ea900b290 0x555ea900d680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:52.451 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:52.451+0000 7fb3463e1740 1 --2- 172.21.15.47:0/4281447420 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x555ea9001f20 0x555ea90023a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:52.451 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:52.451+0000 7fb3463e1740 1 --2- 172.21.15.47:0/4281447420 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555ea9002c00 0x555ea9001950 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:52.451 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:52.451+0000 7fb3463e1740 1 -- 172.21.15.47:0/4281447420 >> 172.21.15.47:0/4281447420 conn(0x555ea8fc9200 msgr2=0x555ea8ffc970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:34:52.451 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:52.451+0000 7fb3463e1740 1 -- 172.21.15.47:0/4281447420 shutdown_connections 2024-03-21T10:34:52.451 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:52.451+0000 7fb3463e1740 1 -- 172.21.15.47:0/4281447420 wait complete. 2024-03-21T10:34:52.451 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:52.452+0000 7fb340c00640 1 --2- 172.21.15.47:0/1348572539 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7fb3240a3090 0x7fb3240a5550 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x555ea9003150 tx=0x7fb33009b680 comp rx=0 tx=0).ready entity=mgr.15687 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:34:52.451 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:52.452+0000 7fb3463e1740 1 -- start start 2024-03-21T10:34:52.451 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:52.452+0000 7fb3463e1740 1 Processor -- start 2024-03-21T10:34:52.454 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:52.453+0000 7fb3463e1740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x555ea9001f20 0x555ea900a950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:34:52.455 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:52.453+0000 7fb3463e1740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555ea9002c00 0x555ea9007ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:34:52.455 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:52.453+0000 7fb3463e1740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555ea900b290 0x555ea9008220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:34:52.455 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:52.453+0000 7fb3463e1740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x555ea900e100 con 0x555ea9001f20 2024-03-21T10:34:52.455 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:52.453+0000 7fb3463e1740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x555ea8f21e80 con 0x555ea900b290 2024-03-21T10:34:52.455 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:52.453+0000 7fb3463e1740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x555ea900e280 con 0x555ea9002c00 2024-03-21T10:34:52.455 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:52.453+0000 7fb3463e1740 10 client.0 _ll_register_callbacks cb 0x555ea8f34130 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T10:34:52.455 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:52.453+0000 7fb341600640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x555ea9001f20 0x555ea900a950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:34:52.455 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:52.453+0000 7fb341600640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x555ea9001f20 0x555ea900a950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:55560/0 (socket says 172.21.15.47:55560) 2024-03-21T10:34:52.455 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:52.453+0000 7fb341600640 1 -- 172.21.15.47:0/2063638354 learned_addr learned my addr 172.21.15.47:0/2063638354 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:34:52.455 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:52.453+0000 7fb33be00640 1 --2- 172.21.15.47:0/2063638354 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555ea9002c00 0x555ea9007ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:34:52.455 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:52.453+0000 7fb341600640 1 -- 172.21.15.47:0/2063638354 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555ea9002c00 msgr2=0x555ea9007ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:34:52.456 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:52.453+0000 7fb341600640 1 --2- 172.21.15.47:0/2063638354 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555ea9002c00 0x555ea9007ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:52.456 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:52.453+0000 7fb341600640 1 -- 172.21.15.47:0/2063638354 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555ea900b290 msgr2=0x555ea9008220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:34:52.456 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:52.453+0000 7fb341600640 1 --2- 172.21.15.47:0/2063638354 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555ea900b290 0x555ea9008220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:52.456 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:52.454+0000 7fb341600640 1 -- 172.21.15.47:0/2063638354 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x555ea8d447d0 con 0x555ea9001f20 2024-03-21T10:34:52.456 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:52.454+0000 7fb33be00640 1 --2- 172.21.15.47:0/2063638354 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555ea9002c00 0x555ea9007ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T10:34:52.456 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:52.454+0000 7fb341600640 1 --2- 172.21.15.47:0/2063638354 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x555ea9001f20 0x555ea900a950 secure :-1 s=READY pgs=1244 cs=0 l=1 rev1=1 crypto rx=0x7fb334004cf0 tx=0x7fb334005960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2534b813bb6fd999 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:34:52.456 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:52.454+0000 7fb322000640 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-03-21T10:34:52.456 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:52.454+0000 7fb322000640 1 -- 172.21.15.47:0/2063638354 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb334016050 con 0x555ea9001f20 2024-03-21T10:34:52.456 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:52.454+0000 7fb322000640 1 -- 172.21.15.47:0/2063638354 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(103..103 src has 1..103) v4 ==== 9562+0+0 (secure 0 0 0) 0x7fb3340052e0 con 0x555ea9001f20 2024-03-21T10:34:52.456 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:52.454+0000 7fb3463e1740 1 -- 172.21.15.47:0/2063638354 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fb334004d20 con 0x555ea9001f20 2024-03-21T10:34:52.456 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:52.455+0000 7fb322000640 1 -- 172.21.15.47:0/2063638354 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb334030050 con 0x555ea9001f20 2024-03-21T10:34:52.456 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:52.455+0000 7fb3463e1740 10 client.18570 fetch_fsmap learned FSMap version 32 2024-03-21T10:34:52.456 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:52.455+0000 7fb3463e1740 1 -- 172.21.15.47:0/2063638354 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x555ea8c7a3f0 con 0x555ea9001f20 2024-03-21T10:34:52.457 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:52.455+0000 7fb322000640 1 -- 172.21.15.47:0/2063638354 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fb334004d20 con 0x555ea9001f20 2024-03-21T10:34:52.457 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:52.455+0000 7fb322000640 1 -- 172.21.15.47:0/2063638354 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 32) v1 ==== 7942+0+0 (secure 0 0 0) 0x7fb33400f040 con 0x555ea9001f20 2024-03-21T10:34:52.457 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:52.455+0000 7fb3463e1740 10 client.18570 fetch_fsmap finished waiting for FSMap version 32 2024-03-21T10:34:52.457 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:52.455+0000 7fb3463e1740 10 client.18570 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-21T10:34:52.457 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:52.456+0000 7fb3463e1740 20 client.18570 populate_metadata read hostname 'smithi047' 2024-03-21T10:34:52.457 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:52.456+0000 7fb3463e1740 1 --2- 172.21.15.47:0/2063638354 >> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] conn(0x555ea901b130 0x555ea901d520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:34:52.457 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:52.456+0000 7fb3463e1740 4 client.18570 mds_command: new command op to 34535 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T10:34:52.457 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:52.456+0000 7fb3463e1740 1 -- 172.21.15.47:0/2063638354 --> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x555ea8f73790 con 0x555ea901b130 2024-03-21T10:34:52.457 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:52.456+0000 7fb33be00640 1 --2- 172.21.15.47:0/2063638354 >> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] conn(0x555ea901b130 0x555ea901d520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:34:52.458 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:52.457+0000 7fb33be00640 1 --2- 172.21.15.47:0/2063638354 >> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] conn(0x555ea901b130 0x555ea901d520 crc :-1 s=READY pgs=213 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e3f8e4dc43127792 server_cookie=222b3e9981894b38 in_seq=0 out_seq=0 2024-03-21T10:34:52.459 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:52.457+0000 7fb322000640 10 client.18570 ms_handle_connect on v2:172.21.15.100:6838/4173168440 2024-03-21T10:34:52.459 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:52.459+0000 7fb322000640 1 -- 172.21.15.47:0/2063638354 <== mds.0 v2:172.21.15.100:6838/4173168440 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x555ea8f73790 con 0x555ea901b130 2024-03-21T10:34:52.459 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:52.459+0000 7fb322000640 10 client.18570 handle_command_reply: tid=0 2024-03-21T10:34:52.459 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:52.459+0000 7fb3463e1740 1 client.18570 shutdown 2024-03-21T10:34:52.459 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:52.459+0000 7fb3463e1740 1 -- 172.21.15.47:0/2063638354 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x555ea9001f20 msgr2=0x555ea900a950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:34:52.459 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:52.459+0000 7fb3463e1740 1 --2- 172.21.15.47:0/2063638354 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x555ea9001f20 0x555ea900a950 secure :-1 s=READY pgs=1244 cs=0 l=1 rev1=1 crypto rx=0x7fb334004cf0 tx=0x7fb334005960 comp rx=0 tx=0).stop 2024-03-21T10:34:52.459 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:52.459+0000 7fb3463e1740 1 -- 172.21.15.47:0/2063638354 shutdown_connections 2024-03-21T10:34:52.459 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:52.459+0000 7fb3463e1740 1 --2- 172.21.15.47:0/2063638354 >> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] conn(0x555ea901b130 0x555ea901d520 crc :-1 s=READY pgs=213 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:52.459 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:52.459+0000 7fb3463e1740 1 --2- 172.21.15.47:0/2063638354 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555ea900b290 0x555ea9008220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:52.459 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:52.459+0000 7fb3463e1740 1 --2- 172.21.15.47:0/2063638354 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555ea9002c00 0x555ea9007ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:52.459 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:52.459+0000 7fb3463e1740 1 --2- 172.21.15.47:0/2063638354 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x555ea9001f20 0x555ea900a950 unknown :-1 s=CLOSED pgs=1244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:52.460 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:52.459+0000 7fb3463e1740 1 -- 172.21.15.47:0/2063638354 >> 172.21.15.47:0/2063638354 conn(0x555ea8fc9200 msgr2=0x555ea8ffe6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:34:52.460 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:52.459+0000 7fb322000640 0 client.18570 ms_handle_reset on v2:172.21.15.100:6838/4173168440 2024-03-21T10:34:52.460 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:52.459+0000 7fb3463e1740 1 -- 172.21.15.47:0/2063638354 shutdown_connections 2024-03-21T10:34:52.460 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:52.459+0000 7fb3463e1740 1 -- 172.21.15.47:0/2063638354 wait complete. 2024-03-21T10:34:52.460 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:52.460+0000 7fb3463e1740 20 client.18570 trim_cache size 0 max 16384 2024-03-21T10:34:52.476 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:52.477+0000 7fb3463e1740 1 Processor -- start 2024-03-21T10:34:52.476 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:52.477+0000 7fb3463e1740 1 -- start start 2024-03-21T10:34:52.476 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:52.478+0000 7fb3463e1740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x555ea9001f20 0x555ea9006cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:34:52.476 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:52.478+0000 7fb3463e1740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555ea9002c00 0x555ea900a410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:34:52.476 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:52.478+0000 7fb3463e1740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555ea900b290 0x555ea900a950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:34:52.476 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:52.478+0000 7fb3463e1740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb33c1b0d10 con 0x555ea9001f20 2024-03-21T10:34:52.476 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:52.478+0000 7fb3463e1740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb33c1b2690 con 0x555ea9002c00 2024-03-21T10:34:52.477 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:52.478+0000 7fb3463e1740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb33c1b1370 con 0x555ea900b290 2024-03-21T10:34:52.477 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:52.478+0000 7fb341600640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x555ea9001f20 0x555ea9006cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:34:52.477 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:52.478+0000 7fb341600640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x555ea9001f20 0x555ea9006cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:55562/0 (socket says 172.21.15.47:55562) 2024-03-21T10:34:52.477 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:52.478+0000 7fb341600640 1 -- 172.21.15.47:0/2738760268 learned_addr learned my addr 172.21.15.47:0/2738760268 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:34:52.478 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:52.478+0000 7fb33be00640 1 --2- 172.21.15.47:0/2738760268 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555ea9002c00 0x555ea900a410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:34:52.479 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:52.478+0000 7fb341600640 1 -- 172.21.15.47:0/2738760268 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555ea900b290 msgr2=0x555ea900a950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:34:52.479 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:52.478+0000 7fb341600640 1 --2- 172.21.15.47:0/2738760268 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555ea900b290 0x555ea900a950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:52.479 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:52.478+0000 7fb341600640 1 -- 172.21.15.47:0/2738760268 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555ea9002c00 msgr2=0x555ea900a410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:34:52.479 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:52.479+0000 7fb341600640 1 --2- 172.21.15.47:0/2738760268 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555ea9002c00 0x555ea900a410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:52.479 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:52.479+0000 7fb341600640 1 -- 172.21.15.47:0/2738760268 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555ea8c3a0f0 con 0x555ea9001f20 2024-03-21T10:34:52.479 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:52.479+0000 7fb33be00640 1 --2- 172.21.15.47:0/2738760268 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555ea9002c00 0x555ea900a410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T10:34:52.479 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:52.479+0000 7fb341600640 1 --2- 172.21.15.47:0/2738760268 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x555ea9001f20 0x555ea9006cc0 secure :-1 s=READY pgs=1245 cs=0 l=1 rev1=1 crypto rx=0x7fb334005c40 tx=0x7fb3340256f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4c91cb56bd335fa4 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:34:52.479 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:52.479+0000 7fb32d200640 1 -- 172.21.15.47:0/2738760268 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb334039070 con 0x555ea9001f20 2024-03-21T10:34:52.479 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:52.479+0000 7fb32d200640 1 -- 172.21.15.47:0/2738760268 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fb334021070 con 0x555ea9001f20 2024-03-21T10:34:52.479 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:52.479+0000 7fb32d200640 1 -- 172.21.15.47:0/2738760268 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb334016070 con 0x555ea9001f20 2024-03-21T10:34:52.479 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:52.480+0000 7fb3463e1740 1 -- 172.21.15.47:0/2738760268 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x555ea9001f20 msgr2=0x555ea9006cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:34:52.479 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:52.480+0000 7fb3463e1740 1 --2- 172.21.15.47:0/2738760268 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x555ea9001f20 0x555ea9006cc0 secure :-1 s=READY pgs=1245 cs=0 l=1 rev1=1 crypto rx=0x7fb334005c40 tx=0x7fb3340256f0 comp rx=0 tx=0).stop 2024-03-21T10:34:52.479 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:52.480+0000 7fb3463e1740 1 -- 172.21.15.47:0/2738760268 shutdown_connections 2024-03-21T10:34:52.479 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:52.480+0000 7fb3463e1740 1 --2- 172.21.15.47:0/2738760268 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555ea900b290 0x555ea900a950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:52.480 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:52.480+0000 7fb3463e1740 1 --2- 172.21.15.47:0/2738760268 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555ea9002c00 0x555ea900a410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:52.480 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:52.480+0000 7fb3463e1740 1 --2- 172.21.15.47:0/2738760268 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x555ea9001f20 0x555ea9006cc0 unknown :-1 s=CLOSED pgs=1245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:52.480 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:52.480+0000 7fb3463e1740 1 -- 172.21.15.47:0/2738760268 >> 172.21.15.47:0/2738760268 conn(0x555ea8fc9200 msgr2=0x555ea90065b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:34:52.481 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:52.480+0000 7fb3463e1740 1 -- 172.21.15.47:0/2738760268 shutdown_connections 2024-03-21T10:34:52.481 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:52.480+0000 7fb3463e1740 1 -- 172.21.15.47:0/2738760268 wait complete. 2024-03-21T10:34:52.481 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:52.480+0000 7fb3463e1740 1 -- start start 2024-03-21T10:34:52.481 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:52.481+0000 7fb3463e1740 1 Processor -- start 2024-03-21T10:34:52.484 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:52.481+0000 7fb3463e1740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x555ea9001f20 0x555ea9004360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:34:52.484 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:52.481+0000 7fb3463e1740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555ea9002c00 0x555ea90048a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:34:52.484 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:52.481+0000 7fb3463e1740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555ea900b290 0x555ea9004de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:34:52.484 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:52.481+0000 7fb3463e1740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb33c1b2030 con 0x555ea9001f20 2024-03-21T10:34:52.484 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:52.481+0000 7fb3463e1740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb33c1b19d0 con 0x555ea900b290 2024-03-21T10:34:52.484 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:52.481+0000 7fb3463e1740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb33c1aeef0 con 0x555ea9002c00 2024-03-21T10:34:52.484 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:52.481+0000 7fb3463e1740 10 client.0 _ll_register_callbacks cb 0x7fb328005b50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T10:34:52.484 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:52.481+0000 7fb341600640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x555ea9001f20 0x555ea9004360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:34:52.485 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:52.482+0000 7fb341600640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x555ea9001f20 0x555ea9004360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:55570/0 (socket says 172.21.15.47:55570) 2024-03-21T10:34:52.485 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:52.482+0000 7fb341600640 1 -- 172.21.15.47:0/3018152992 learned_addr learned my addr 172.21.15.47:0/3018152992 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:34:52.485 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:52.482+0000 7fb33be00640 1 --2- 172.21.15.47:0/3018152992 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555ea9002c00 0x555ea90048a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:34:52.485 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:52.482+0000 7fb341600640 1 --2- 172.21.15.47:0/3018152992 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555ea900b290 0x555ea9004de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:34:52.485 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:52.482+0000 7fb341600640 1 -- 172.21.15.47:0/3018152992 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555ea9002c00 msgr2=0x555ea90048a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:34:52.485 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:52.482+0000 7fb341600640 1 --2- 172.21.15.47:0/3018152992 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555ea9002c00 0x555ea90048a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:52.485 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:52.482+0000 7fb341600640 1 -- 172.21.15.47:0/3018152992 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555ea900b290 msgr2=0x555ea9004de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:34:52.485 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:52.482+0000 7fb341600640 1 --2- 172.21.15.47:0/3018152992 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555ea900b290 0x555ea9004de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:52.485 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:52.482+0000 7fb341600640 1 -- 172.21.15.47:0/3018152992 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x555ea9075700 con 0x555ea9001f20 2024-03-21T10:34:52.485 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:52.482+0000 7fb341600640 1 --2- 172.21.15.47:0/3018152992 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x555ea9001f20 0x555ea9004360 secure :-1 s=READY pgs=1246 cs=0 l=1 rev1=1 crypto rx=0x7fb334006010 tx=0x7fb334009580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fbe7384d22a9c36e server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:34:52.485 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:52.482+0000 7fb322000640 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-03-21T10:34:52.485 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:52.483+0000 7fb322000640 1 -- 172.21.15.47:0/3018152992 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb334016070 con 0x555ea9001f20 2024-03-21T10:34:52.486 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:52.483+0000 7fb322000640 1 -- 172.21.15.47:0/3018152992 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(103..103 src has 1..103) v4 ==== 9562+0+0 (secure 0 0 0) 0x7fb3340052e0 con 0x555ea9001f20 2024-03-21T10:34:52.486 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:52.483+0000 7fb3463e1740 1 -- 172.21.15.47:0/3018152992 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fb334021070 con 0x555ea9001f20 2024-03-21T10:34:52.486 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:52.483+0000 7fb322000640 1 -- 172.21.15.47:0/3018152992 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb334039040 con 0x555ea9001f20 2024-03-21T10:34:52.486 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:52.483+0000 7fb3463e1740 10 client.18576 fetch_fsmap learned FSMap version 32 2024-03-21T10:34:52.486 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:52.483+0000 7fb3463e1740 1 -- 172.21.15.47:0/3018152992 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x555ea90767c0 con 0x555ea9001f20 2024-03-21T10:34:52.486 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:52.483+0000 7fb322000640 1 -- 172.21.15.47:0/3018152992 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fb334021070 con 0x555ea9001f20 2024-03-21T10:34:52.486 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:52.483+0000 7fb322000640 1 -- 172.21.15.47:0/3018152992 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 32) v1 ==== 7942+0+0 (secure 0 0 0) 0x7fb33403c070 con 0x555ea9001f20 2024-03-21T10:34:52.486 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:52.484+0000 7fb3463e1740 10 client.18576 fetch_fsmap finished waiting for FSMap version 32 2024-03-21T10:34:52.486 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:52.484+0000 7fb3463e1740 10 client.18576 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-21T10:34:52.486 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:52.484+0000 7fb3463e1740 20 client.18576 populate_metadata read hostname 'smithi047' 2024-03-21T10:34:52.487 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:52.484+0000 7fb3463e1740 1 --2- 172.21.15.47:0/3018152992 >> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] conn(0x555ea9076b50 0x555ea9078f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:34:52.487 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:52.484+0000 7fb3463e1740 4 client.18576 mds_command: new command op to 34535 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-21T10:34:52.487 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:52.484+0000 7fb3463e1740 1 -- 172.21.15.47:0/3018152992 --> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x555ea8f6dae0 con 0x555ea9076b50 2024-03-21T10:34:52.487 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:52.484+0000 7fb33be00640 1 --2- 172.21.15.47:0/3018152992 >> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] conn(0x555ea9076b50 0x555ea9078f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:34:52.487 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:52.485+0000 7fb33be00640 1 --2- 172.21.15.47:0/3018152992 >> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] conn(0x555ea9076b50 0x555ea9078f70 crc :-1 s=READY pgs=214 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1135d27f84466aa9 server_cookie=ead6881629aa5b61 in_seq=0 out_seq=0 2024-03-21T10:34:52.487 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:52.485+0000 7fb322000640 10 client.18576 ms_handle_connect on v2:172.21.15.100:6838/4173168440 2024-03-21T10:34:52.487 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:52.485+0000 7fb322000640 1 -- 172.21.15.47:0/3018152992 <== mds.0 v2:172.21.15.100:6838/4173168440 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x555ea8f6dae0 con 0x555ea9076b50 2024-03-21T10:34:52.487 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:52.485+0000 7fb322000640 10 client.18576 handle_command_reply: tid=0 2024-03-21T10:34:52.487 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:52.485+0000 7fb3463e1740 1 client.18576 shutdown 2024-03-21T10:34:52.487 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:52.486+0000 7fb3463e1740 1 -- 172.21.15.47:0/3018152992 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x555ea9001f20 msgr2=0x555ea9004360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:34:52.487 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:52.486+0000 7fb3463e1740 1 --2- 172.21.15.47:0/3018152992 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x555ea9001f20 0x555ea9004360 secure :-1 s=READY pgs=1246 cs=0 l=1 rev1=1 crypto rx=0x7fb334006010 tx=0x7fb334009580 comp rx=0 tx=0).stop 2024-03-21T10:34:52.487 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:52.486+0000 7fb3463e1740 1 -- 172.21.15.47:0/3018152992 shutdown_connections 2024-03-21T10:34:52.487 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:52.486+0000 7fb3463e1740 1 --2- 172.21.15.47:0/3018152992 >> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] conn(0x555ea9076b50 0x555ea9078f70 crc :-1 s=READY pgs=214 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:52.487 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:52.486+0000 7fb3463e1740 1 --2- 172.21.15.47:0/3018152992 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555ea900b290 0x555ea9004de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:52.488 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:52.486+0000 7fb3463e1740 1 --2- 172.21.15.47:0/3018152992 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x555ea9002c00 0x555ea90048a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:52.488 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:52.486+0000 7fb3463e1740 1 --2- 172.21.15.47:0/3018152992 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x555ea9001f20 0x555ea9004360 unknown :-1 s=CLOSED pgs=1246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:52.488 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:52.486+0000 7fb322000640 0 client.18576 ms_handle_reset on v2:172.21.15.100:6838/4173168440 2024-03-21T10:34:52.488 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:52.486+0000 7fb3463e1740 1 -- 172.21.15.47:0/3018152992 >> 172.21.15.47:0/3018152992 conn(0x555ea8fc9200 msgr2=0x555ea9006e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:34:52.488 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:52.486+0000 7fb3463e1740 1 -- 172.21.15.47:0/3018152992 shutdown_connections 2024-03-21T10:34:52.488 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:52.486+0000 7fb3463e1740 1 -- 172.21.15.47:0/3018152992 wait complete. 2024-03-21T10:34:52.488 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:52.486+0000 7fb3463e1740 20 client.18576 trim_cache size 0 max 16384 2024-03-21T10:34:52.488 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-03-21T10:34:52.488 INFO:teuthology.orchestra.run.smithi047.stdout: "status": "no active scrubs running", 2024-03-21T10:34:52.488 INFO:teuthology.orchestra.run.smithi047.stdout: "scrubs": {} 2024-03-21T10:34:52.489 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-03-21T10:34:52.489 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:52.487+0000 7fb343600640 1 -- 172.21.15.47:0/1348572539 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7fb3240a3090 msgr2=0x7fb3240a5550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:34:52.489 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:52.487+0000 7fb343600640 1 --2- 172.21.15.47:0/1348572539 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7fb3240a3090 0x7fb3240a5550 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x555ea9003150 tx=0x7fb33009b680 comp rx=0 tx=0).stop 2024-03-21T10:34:52.489 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:52.487+0000 7fb343600640 1 -- 172.21.15.47:0/1348572539 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb33c103840 msgr2=0x7fb33c19a080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:34:52.489 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:52.487+0000 7fb343600640 1 --2- 172.21.15.47:0/1348572539 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb33c103840 0x7fb33c19a080 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7fb33000d950 tx=0x7fb33000de10 comp rx=0 tx=0).stop 2024-03-21T10:34:52.489 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:52.487+0000 7fb343600640 1 -- 172.21.15.47:0/1348572539 shutdown_connections 2024-03-21T10:34:52.490 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:52.487+0000 7fb343600640 1 --2- 172.21.15.47:0/1348572539 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb33c109850 0x7fb33c19e940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:52.490 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:52.487+0000 7fb343600640 1 --2- 172.21.15.47:0/1348572539 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb33c104200 0x7fb33c19a5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:52.490 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:52.487+0000 7fb343600640 1 --2- 172.21.15.47:0/1348572539 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7fb3240a3090 0x7fb3240a5550 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:52.490 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:52.487+0000 7fb343600640 1 --2- 172.21.15.47:0/1348572539 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb33c103840 0x7fb33c19a080 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:52.490 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:52.487+0000 7fb343600640 1 -- 172.21.15.47:0/1348572539 >> 172.21.15.47:0/1348572539 conn(0x7fb33c0ff310 msgr2=0x7fb33c0ffbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:34:52.490 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:52.487+0000 7fb343600640 1 -- 172.21.15.47:0/1348572539 shutdown_connections 2024-03-21T10:34:52.490 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:52.487+0000 7fb343600640 1 -- 172.21.15.47:0/1348572539 wait complete. 2024-03-21T10:34:52.503 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-03-21T10:34:52.503 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-21T10:34:52.724 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:52.726+0000 7fa306c00640 1 -- 172.21.15.47:0/1091463654 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa3001041f0 msgr2=0x7fa300110700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:34:52.724 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:52.726+0000 7fa306c00640 1 --2- 172.21.15.47:0/1091463654 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa3001041f0 0x7fa300110700 secure :-1 s=READY pgs=1247 cs=0 l=1 rev1=1 crypto rx=0x7fa2f8009cd0 tx=0x7fa2f802c0e0 comp rx=0 tx=0).stop 2024-03-21T10:34:52.725 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:52.726+0000 7fa306c00640 1 -- 172.21.15.47:0/1091463654 shutdown_connections 2024-03-21T10:34:52.725 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:52.726+0000 7fa306c00640 1 --2- 172.21.15.47:0/1091463654 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa3001041f0 0x7fa300110700 unknown :-1 s=CLOSED pgs=1247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:52.725 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:52.726+0000 7fa306c00640 1 --2- 172.21.15.47:0/1091463654 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa300103830 0x7fa300103cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:52.725 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:52.726+0000 7fa306c00640 1 --2- 172.21.15.47:0/1091463654 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa300109840 0x7fa300109c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:52.725 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:52.726+0000 7fa306c00640 1 -- 172.21.15.47:0/1091463654 >> 172.21.15.47:0/1091463654 conn(0x7fa3000ff300 msgr2=0x7fa300101720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:34:52.725 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:52.726+0000 7fa306c00640 1 -- 172.21.15.47:0/1091463654 shutdown_connections 2024-03-21T10:34:52.725 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:52.726+0000 7fa306c00640 1 -- 172.21.15.47:0/1091463654 wait complete. 2024-03-21T10:34:52.726 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:52.728+0000 7fa306c00640 1 Processor -- start 2024-03-21T10:34:52.726 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:52.728+0000 7fa306c00640 1 -- start start 2024-03-21T10:34:52.729 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:52.728+0000 7fa306c00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa300103830 0x7fa30019a000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:34:52.729 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:52.728+0000 7fa306c00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa3001041f0 0x7fa30019a540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:34:52.729 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:52.728+0000 7fa306c00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa300109840 0x7fa30019e8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:34:52.729 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:52.728+0000 7fa306c00640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa300113b00 con 0x7fa300103830 2024-03-21T10:34:52.729 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:52.728+0000 7fa306c00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa300113980 con 0x7fa300109840 2024-03-21T10:34:52.730 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:52.728+0000 7fa306c00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa300113c80 con 0x7fa3001041f0 2024-03-21T10:34:52.730 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:52.729+0000 7fa2ffe00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa300103830 0x7fa30019a000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:34:52.730 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:52.729+0000 7fa2ffe00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa300103830 0x7fa30019a000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:55590/0 (socket says 172.21.15.47:55590) 2024-03-21T10:34:52.730 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:52.729+0000 7fa2ffe00640 1 -- 172.21.15.47:0/3030542532 learned_addr learned my addr 172.21.15.47:0/3030542532 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:34:52.730 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:52.729+0000 7fa304c00640 1 --2- 172.21.15.47:0/3030542532 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa300109840 0x7fa30019e8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:34:52.730 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:52.729+0000 7fa2ff400640 1 --2- 172.21.15.47:0/3030542532 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa3001041f0 0x7fa30019a540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:34:52.730 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:52.729+0000 7fa2ffe00640 1 -- 172.21.15.47:0/3030542532 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa3001041f0 msgr2=0x7fa30019a540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:34:52.730 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:52.729+0000 7fa2ffe00640 1 --2- 172.21.15.47:0/3030542532 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa3001041f0 0x7fa30019a540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:52.730 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:52.729+0000 7fa2ffe00640 1 -- 172.21.15.47:0/3030542532 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa300109840 msgr2=0x7fa30019e8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:34:52.730 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:52.729+0000 7fa2ffe00640 1 --2- 172.21.15.47:0/3030542532 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa300109840 0x7fa30019e8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:52.730 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:52.729+0000 7fa2ffe00640 1 -- 172.21.15.47:0/3030542532 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa30019eff0 con 0x7fa300103830 2024-03-21T10:34:52.730 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:52.729+0000 7fa2ff400640 1 --2- 172.21.15.47:0/3030542532 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa3001041f0 0x7fa30019a540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T10:34:52.730 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:52.730+0000 7fa2ffe00640 1 --2- 172.21.15.47:0/3030542532 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa300103830 0x7fa30019a000 secure :-1 s=READY pgs=1248 cs=0 l=1 rev1=1 crypto rx=0x7fa2f000e9f0 tx=0x7fa2f000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:34:52.730 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:52.730+0000 7fa2fcc00640 1 -- 172.21.15.47:0/3030542532 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2f000cd80 con 0x7fa300103830 2024-03-21T10:34:52.731 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:52.730+0000 7fa2fcc00640 1 -- 172.21.15.47:0/3030542532 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fa2f00108e0 con 0x7fa300103830 2024-03-21T10:34:52.731 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:52.730+0000 7fa2fcc00640 1 -- 172.21.15.47:0/3030542532 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2f0018920 con 0x7fa300103830 2024-03-21T10:34:52.731 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:52.730+0000 7fa306c00640 1 -- 172.21.15.47:0/3030542532 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa30019f2e0 con 0x7fa300103830 2024-03-21T10:34:52.731 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:52.730+0000 7fa306c00640 1 -- 172.21.15.47:0/3030542532 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa3001a6b70 con 0x7fa300103830 2024-03-21T10:34:52.731 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:52.732+0000 7fa306c00640 1 -- 172.21.15.47:0/3030542532 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa3000688e0 con 0x7fa300103830 2024-03-21T10:34:52.731 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:52.733+0000 7fa2fcc00640 1 -- 172.21.15.47:0/3030542532 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 27) v1 ==== 137985+0+0 (secure 0 0 0) 0x7fa2f0010a90 con 0x7fa300103830 2024-03-21T10:34:52.734 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:52.735+0000 7fa2fcc00640 1 --2- 172.21.15.47:0/3030542532 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7fa2e40a3110 0x7fa2e40a55d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:34:52.734 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:52.736+0000 7fa2fcc00640 1 -- 172.21.15.47:0/3030542532 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 9562+0+0 (secure 0 0 0) 0x7fa2f00cff20 con 0x7fa300103830 2024-03-21T10:34:52.734 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:52.736+0000 7fa2ff400640 1 --2- 172.21.15.47:0/3030542532 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7fa2e40a3110 0x7fa2e40a55d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:34:52.735 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:52.737+0000 7fa2ff400640 1 --2- 172.21.15.47:0/3030542532 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7fa2e40a3110 0x7fa2e40a55d0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fa300101980 tx=0x7fa2ec005e50 comp rx=0 tx=0).ready entity=mgr.15687 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:34:52.737 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:52.739+0000 7fa2fcc00640 1 -- 172.21.15.47:0/3030542532 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fa2f009c5a0 con 0x7fa300103830 2024-03-21T10:34:53.189 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:53.191+0000 7fa306c00640 1 -- 172.21.15.47:0/3030542532 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa30010bdc0 con 0x7fa300103830 2024-03-21T10:34:53.191 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:53.193+0000 7fa2fcc00640 1 -- 172.21.15.47:0/3030542532 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 32 v32) v1 ==== 94+0+11471 (secure 0 0 0) 0x7fa2f009bea0 con 0x7fa300103830 2024-03-21T10:34:53.191 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-21T10:34:53.191 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":32,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14511,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6835/735724037","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6834","nonce":735724037},{"type":"v1","addr":"172.21.15.47:6835","nonce":735724037}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14529,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6837/2248191724","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6836","nonce":2248191724},{"type":"v1","addr":"172.21.15.47:6837","nonce":2248191724}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":32,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-21T10:21:58.329063+0000","modified":"2024-03-21T10:26:17.169892+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":34535,"mds_1":34493,"mds_2":34454,"mds_3":34427,"mds_4":14619},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14604":{"gid":14604,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.47:6839/2881335356","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6838","nonce":2881335356},{"type":"v1","addr":"172.21.15.47:6839","nonce":2881335356}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_14619":{"gid":14619,"name":"a","rank":4,"incarnation":27,"state":"up:active","state_seq":11,"addr":"172.21.15.47:6841/3401578978","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6840","nonce":3401578978},{"type":"v1","addr":"172.21.15.47:6841","nonce":3401578978}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24406":{"gid":24406,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.157:6833/4000010136","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":4000010136},{"type":"v1","addr":"172.21.15.157:6833","nonce":4000010136}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24457":{"gid":24457,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.157:6835/3902868651","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":3902868651},{"type":"v1","addr":"172.21.15.157:6835","nonce":3902868651}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24475":{"gid":24475,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.157:6837/1908087204","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6836","nonce":1908087204},{"type":"v1","addr":"172.21.15.157:6837","nonce":1908087204}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_25729":{"gid":25729,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.157:6839/1998408416","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6838","nonce":1998408416},{"type":"v1","addr":"172.21.15.157:6839","nonce":1998408416}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_34427":{"gid":34427,"name":"k","rank":3,"incarnation":25,"state":"up:active","state_seq":20,"addr":"172.21.15.100:6833/2723302038","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6832","nonce":2723302038},{"type":"v1","addr":"172.21.15.100:6833","nonce":2723302038}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_34454":{"gid":34454,"name":"e","rank":2,"incarnation":22,"state":"up:active","state_seq":17,"addr":"172.21.15.100:6835/2597849957","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":2597849957},{"type":"v1","addr":"172.21.15.100:6835","nonce":2597849957}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_34493":{"gid":34493,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":13,"addr":"172.21.15.100:6837/3497644162","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6836","nonce":3497644162},{"type":"v1","addr":"172.21.15.100:6837","nonce":3497644162}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_34535":{"gid":34535,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.100:6839/4173168440","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6838","nonce":4173168440},{"type":"v1","addr":"172.21.15.100:6839","nonce":4173168440}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":34535,"qdb_cluster":[34427,14619,34454,34535,34493]},"id":1}]} 2024-03-21T10:34:53.191 INFO:teuthology.orchestra.run.smithi047.stderr:dumped fsmap epoch 32 2024-03-21T10:34:53.194 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:53.196+0000 7fa306c00640 1 -- 172.21.15.47:0/3030542532 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7fa2e40a3110 msgr2=0x7fa2e40a55d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:34:53.195 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:53.196+0000 7fa306c00640 1 --2- 172.21.15.47:0/3030542532 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7fa2e40a3110 0x7fa2e40a55d0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fa300101980 tx=0x7fa2ec005e50 comp rx=0 tx=0).stop 2024-03-21T10:34:53.195 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:53.196+0000 7fa306c00640 1 -- 172.21.15.47:0/3030542532 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa300103830 msgr2=0x7fa30019a000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:34:53.195 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:53.196+0000 7fa306c00640 1 --2- 172.21.15.47:0/3030542532 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa300103830 0x7fa30019a000 secure :-1 s=READY pgs=1248 cs=0 l=1 rev1=1 crypto rx=0x7fa2f000e9f0 tx=0x7fa2f000eeb0 comp rx=0 tx=0).stop 2024-03-21T10:34:53.195 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:53.196+0000 7fa306c00640 1 -- 172.21.15.47:0/3030542532 shutdown_connections 2024-03-21T10:34:53.195 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:53.196+0000 7fa306c00640 1 --2- 172.21.15.47:0/3030542532 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7fa2e40a3110 0x7fa2e40a55d0 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:53.195 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:53.196+0000 7fa306c00640 1 --2- 172.21.15.47:0/3030542532 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa300109840 0x7fa30019e8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:53.195 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:53.196+0000 7fa306c00640 1 --2- 172.21.15.47:0/3030542532 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa3001041f0 0x7fa30019a540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:53.195 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:53.196+0000 7fa306c00640 1 --2- 172.21.15.47:0/3030542532 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa300103830 0x7fa30019a000 unknown :-1 s=CLOSED pgs=1248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:53.195 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:53.196+0000 7fa306c00640 1 -- 172.21.15.47:0/3030542532 >> 172.21.15.47:0/3030542532 conn(0x7fa3000ff300 msgr2=0x7fa3000ffac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:34:53.195 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:53.196+0000 7fa306c00640 1 -- 172.21.15.47:0/3030542532 shutdown_connections 2024-03-21T10:34:53.195 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:53.196+0000 7fa306c00640 1 -- 172.21.15.47:0/3030542532 wait complete. 2024-03-21T10:34:53.214 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-21T10:34:53.215 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-03-21T10:34:53.445 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:53.447+0000 7ff8b0e00640 1 -- 172.21.15.47:0/899209987 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff8ac107660 msgr2=0x7ff8ac107a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:34:53.446 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:53.447+0000 7ff8b0e00640 1 --2- 172.21.15.47:0/899209987 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff8ac107660 0x7ff8ac107a40 secure :-1 s=READY pgs=1249 cs=0 l=1 rev1=1 crypto rx=0x7ff89c009930 tx=0x7ff89c02bc30 comp rx=0 tx=0).stop 2024-03-21T10:34:53.446 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:53.447+0000 7ff8b0e00640 1 -- 172.21.15.47:0/899209987 shutdown_connections 2024-03-21T10:34:53.446 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:53.447+0000 7ff8b0e00640 1 --2- 172.21.15.47:0/899209987 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff8ac06a590 0x7ff8ac10e520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:53.446 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:53.447+0000 7ff8b0e00640 1 --2- 172.21.15.47:0/899209987 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff8ac069c10 0x7ff8ac06a050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:53.446 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:53.447+0000 7ff8b0e00640 1 --2- 172.21.15.47:0/899209987 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff8ac107660 0x7ff8ac107a40 unknown :-1 s=CLOSED pgs=1249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:53.446 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:53.447+0000 7ff8b0e00640 1 -- 172.21.15.47:0/899209987 >> 172.21.15.47:0/899209987 conn(0x7ff8ac077580 msgr2=0x7ff8ac077990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:34:53.446 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:53.447+0000 7ff8b0e00640 1 -- 172.21.15.47:0/899209987 shutdown_connections 2024-03-21T10:34:53.446 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:53.447+0000 7ff8b0e00640 1 -- 172.21.15.47:0/899209987 wait complete. 2024-03-21T10:34:53.449 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:53.448+0000 7ff8b0e00640 1 Processor -- start 2024-03-21T10:34:53.449 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:53.448+0000 7ff8b0e00640 1 -- start start 2024-03-21T10:34:53.449 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:53.448+0000 7ff8b0e00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff8ac069c10 0x7ff8ac19a0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:34:53.449 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:53.449+0000 7ff8b0e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff8ac06a590 0x7ff8ac19a630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:34:53.449 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:53.449+0000 7ff8b0e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff8ac107660 0x7ff8ac19e9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:34:53.449 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:53.449+0000 7ff8b0e00640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8ac111910 con 0x7ff8ac107660 2024-03-21T10:34:53.449 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:53.449+0000 7ff8b0e00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8ac111790 con 0x7ff8ac06a590 2024-03-21T10:34:53.449 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:53.449+0000 7ff8b0e00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8ac111a90 con 0x7ff8ac069c10 2024-03-21T10:34:53.449 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:53.449+0000 7ff8aa800640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff8ac107660 0x7ff8ac19e9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:34:53.449 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:53.449+0000 7ff8a9400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff8ac06a590 0x7ff8ac19a630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:34:53.450 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:53.449+0000 7ff8a9400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff8ac06a590 0x7ff8ac19a630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:45672/0 (socket says 172.21.15.47:45672) 2024-03-21T10:34:53.450 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:53.449+0000 7ff8a9400640 1 -- 172.21.15.47:0/317137689 learned_addr learned my addr 172.21.15.47:0/317137689 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:34:53.450 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:53.449+0000 7ff8aa800640 1 --2- 172.21.15.47:0/317137689 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff8ac107660 0x7ff8ac19e9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:55606/0 (socket says 172.21.15.47:55606) 2024-03-21T10:34:53.450 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:53.449+0000 7ff8a9e00640 1 --2- 172.21.15.47:0/317137689 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff8ac069c10 0x7ff8ac19a0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:34:53.450 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:53.450+0000 7ff8aa800640 1 -- 172.21.15.47:0/317137689 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff8ac069c10 msgr2=0x7ff8ac19a0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:34:53.450 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:53.450+0000 7ff8aa800640 1 --2- 172.21.15.47:0/317137689 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff8ac069c10 0x7ff8ac19a0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:53.450 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:53.450+0000 7ff8aa800640 1 -- 172.21.15.47:0/317137689 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff8ac06a590 msgr2=0x7ff8ac19a630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:34:53.450 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:53.450+0000 7ff8aa800640 1 --2- 172.21.15.47:0/317137689 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff8ac06a590 0x7ff8ac19a630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:53.450 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:53.450+0000 7ff8aa800640 1 -- 172.21.15.47:0/317137689 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff8ac19f0e0 con 0x7ff8ac107660 2024-03-21T10:34:53.450 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:53.450+0000 7ff8a9400640 1 --2- 172.21.15.47:0/317137689 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff8ac06a590 0x7ff8ac19a630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T10:34:53.450 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:53.450+0000 7ff8aa800640 1 --2- 172.21.15.47:0/317137689 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff8ac107660 0x7ff8ac19e9b0 secure :-1 s=READY pgs=1250 cs=0 l=1 rev1=1 crypto rx=0x7ff8a400be20 tx=0x7ff8a400bf10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:34:53.450 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:53.450+0000 7ff8a2a00640 1 -- 172.21.15.47:0/317137689 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8a4019070 con 0x7ff8ac107660 2024-03-21T10:34:53.451 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:53.450+0000 7ff8a2a00640 1 -- 172.21.15.47:0/317137689 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7ff8a40148f0 con 0x7ff8ac107660 2024-03-21T10:34:53.451 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:53.450+0000 7ff8b0e00640 1 -- 172.21.15.47:0/317137689 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff8ac19f3d0 con 0x7ff8ac107660 2024-03-21T10:34:53.451 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:53.450+0000 7ff8a2a00640 1 -- 172.21.15.47:0/317137689 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8a40127f0 con 0x7ff8ac107660 2024-03-21T10:34:53.451 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:53.450+0000 7ff8b0e00640 1 -- 172.21.15.47:0/317137689 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff8ac1a6c60 con 0x7ff8ac107660 2024-03-21T10:34:53.451 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:53.454+0000 7ff8b3a50740 1 Processor -- start 2024-03-21T10:34:53.453 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:53.454+0000 7ff8a2a00640 1 -- 172.21.15.47:0/317137689 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 27) v1 ==== 137985+0+0 (secure 0 0 0) 0x7ff8a401c070 con 0x7ff8ac107660 2024-03-21T10:34:53.453 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:53.454+0000 7ff8b3a50740 1 -- start start 2024-03-21T10:34:53.453 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:53.454+0000 7ff8b3a50740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fceeab56d0 0x55fceeab5ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:34:53.453 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:53.454+0000 7ff8b3a50740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55fceeab6080 0x55fceeabe630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:34:53.453 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:53.454+0000 7ff8b3a50740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fceeabec00 0x55fceeac1050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:34:53.453 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:53.454+0000 7ff8b3a50740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fceea192c0 con 0x55fceeab6080 2024-03-21T10:34:53.453 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:53.454+0000 7ff8b3a50740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fcee9c1c90 con 0x55fceeab56d0 2024-03-21T10:34:53.453 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:53.454+0000 7ff8b3a50740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fceea15360 con 0x55fceeabec00 2024-03-21T10:34:53.453 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:53.455+0000 7ff8a9e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55fceeab6080 0x55fceeabe630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:34:53.453 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:53.455+0000 7ff8a9e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55fceeab6080 0x55fceeabe630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:55614/0 (socket says 172.21.15.47:55614) 2024-03-21T10:34:53.453 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:53.455+0000 7ff8a9e00640 1 -- 172.21.15.47:0/4226747578 learned_addr learned my addr 172.21.15.47:0/4226747578 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:34:53.453 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:53.455+0000 7ff8a9400640 1 --2- 172.21.15.47:0/4226747578 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fceeab56d0 0x55fceeab5ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:34:53.454 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:53.455+0000 7ff8a9e00640 1 -- 172.21.15.47:0/4226747578 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fceeabec00 msgr2=0x55fceeac1050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:34:53.454 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:53.455+0000 7ff8a9e00640 1 --2- 172.21.15.47:0/4226747578 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fceeabec00 0x55fceeac1050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:53.454 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:53.455+0000 7ff8a9e00640 1 -- 172.21.15.47:0/4226747578 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fceeab56d0 msgr2=0x55fceeab5ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:34:53.455 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:53.455+0000 7ff8a9e00640 1 --2- 172.21.15.47:0/4226747578 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fceeab56d0 0x55fceeab5ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:53.455 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:53.455+0000 7ff8a9e00640 1 -- 172.21.15.47:0/4226747578 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55fcee9e6080 con 0x55fceeab6080 2024-03-21T10:34:53.455 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:53.456+0000 7ff8a9e00640 1 --2- 172.21.15.47:0/4226747578 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55fceeab6080 0x55fceeabe630 secure :-1 s=READY pgs=1251 cs=0 l=1 rev1=1 crypto rx=0x7ff89c036040 tx=0x7ff89c02c5d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e81b488c0e065cb0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:34:53.455 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:53.456+0000 7ff8a2a00640 1 --2- 172.21.15.47:0/317137689 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7ff8900a3110 0x7ff8900a55d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:34:53.455 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:53.456+0000 7ff8a2a00640 1 -- 172.21.15.47:0/317137689 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 9562+0+0 (secure 0 0 0) 0x7ff8a40cf300 con 0x7ff8ac107660 2024-03-21T10:34:53.455 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:53.456+0000 7ff8a9400640 1 --2- 172.21.15.47:0/317137689 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7ff8900a3110 0x7ff8900a55d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:34:53.455 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:53.456+0000 7ff897400640 1 -- 172.21.15.47:0/4226747578 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff89c045020 con 0x55fceeab6080 2024-03-21T10:34:53.455 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:53.456+0000 7ff897400640 1 -- 172.21.15.47:0/4226747578 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7ff89c02c7e0 con 0x55fceeab6080 2024-03-21T10:34:53.455 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:53.456+0000 7ff897400640 1 -- 172.21.15.47:0/4226747578 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff89c005590 con 0x55fceeab6080 2024-03-21T10:34:53.458 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:53.457+0000 7ff8b3a50740 1 -- 172.21.15.47:0/4226747578 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55fceeab6080 msgr2=0x55fceeabe630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:34:53.458 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:53.457+0000 7ff8a9400640 1 --2- 172.21.15.47:0/317137689 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7ff8900a3110 0x7ff8900a55d0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7ff898009fd0 tx=0x7ff898009c60 comp rx=0 tx=0).ready entity=mgr.15687 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:34:53.458 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:53.457+0000 7ff8b3a50740 1 --2- 172.21.15.47:0/4226747578 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55fceeab6080 0x55fceeabe630 secure :-1 s=READY pgs=1251 cs=0 l=1 rev1=1 crypto rx=0x7ff89c036040 tx=0x7ff89c02c5d0 comp rx=0 tx=0).stop 2024-03-21T10:34:53.458 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:53.457+0000 7ff8b3a50740 1 -- 172.21.15.47:0/4226747578 shutdown_connections 2024-03-21T10:34:53.458 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:53.457+0000 7ff8b3a50740 1 --2- 172.21.15.47:0/4226747578 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fceeabec00 0x55fceeac1050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:53.458 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:53.457+0000 7ff8b3a50740 1 --2- 172.21.15.47:0/4226747578 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55fceeab6080 0x55fceeabe630 unknown :-1 s=CLOSED pgs=1251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:53.458 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:53.457+0000 7ff8b3a50740 1 --2- 172.21.15.47:0/4226747578 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fceeab56d0 0x55fceeab5ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:53.458 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:53.457+0000 7ff8b3a50740 1 -- 172.21.15.47:0/4226747578 >> 172.21.15.47:0/4226747578 conn(0x55fceeaafda0 msgr2=0x55fceeab0180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:34:53.459 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:53.457+0000 7ff8b3a50740 1 -- 172.21.15.47:0/4226747578 shutdown_connections 2024-03-21T10:34:53.459 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:53.457+0000 7ff8b3a50740 1 -- 172.21.15.47:0/4226747578 wait complete. 2024-03-21T10:34:53.459 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:53.458+0000 7ff8b3a50740 1 -- start start 2024-03-21T10:34:53.459 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:53.458+0000 7ff8b3a50740 1 Processor -- start 2024-03-21T10:34:53.459 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:53.458+0000 7ff8b3a50740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55fceeab56d0 0x55fceeabe4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:34:53.459 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:53.458+0000 7ff8b3a50740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fceeab6080 0x55fceeabb870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:34:53.459 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:53.458+0000 7ff8b3a50740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fceeabec00 0x55fceeabbdd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:34:53.459 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:53.458+0000 7ff8b3a50740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fceeac1b10 con 0x55fceeab56d0 2024-03-21T10:34:53.459 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:53.458+0000 7ff8b3a50740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fcee9d50d0 con 0x55fceeabec00 2024-03-21T10:34:53.459 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:53.458+0000 7ff8b3a50740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fceeac1c90 con 0x55fceeab6080 2024-03-21T10:34:53.459 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:53.458+0000 7ff8b3a50740 10 client.0 _ll_register_callbacks cb 0x55fcee9d9e80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T10:34:53.459 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:53.459+0000 7ff8a9e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55fceeab56d0 0x55fceeabe4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:34:53.459 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:53.459+0000 7ff8a9e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55fceeab56d0 0x55fceeabe4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:55620/0 (socket says 172.21.15.47:55620) 2024-03-21T10:34:53.459 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:53.459+0000 7ff8a9e00640 1 -- 172.21.15.47:0/1002711142 learned_addr learned my addr 172.21.15.47:0/1002711142 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:34:53.460 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:53.459+0000 7ff8aa800640 1 --2- 172.21.15.47:0/1002711142 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fceeabec00 0x55fceeabbdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:34:53.460 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:53.459+0000 7ff8a9400640 1 --2- 172.21.15.47:0/1002711142 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fceeab6080 0x55fceeabb870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:34:53.460 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:53.459+0000 7ff8a9e00640 1 -- 172.21.15.47:0/1002711142 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fceeab6080 msgr2=0x55fceeabb870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:34:53.460 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:53.459+0000 7ff8a9e00640 1 --2- 172.21.15.47:0/1002711142 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fceeab6080 0x55fceeabb870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:53.460 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:53.459+0000 7ff8a9e00640 1 -- 172.21.15.47:0/1002711142 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fceeabec00 msgr2=0x55fceeabbdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:34:53.460 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:53.459+0000 7ff8a9e00640 1 --2- 172.21.15.47:0/1002711142 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fceeabec00 0x55fceeabbdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:53.460 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:53.459+0000 7ff8a9e00640 1 -- 172.21.15.47:0/1002711142 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55fcee72d3f0 con 0x55fceeab56d0 2024-03-21T10:34:53.460 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:53.459+0000 7ff8a9e00640 1 --2- 172.21.15.47:0/1002711142 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55fceeab56d0 0x55fceeabe4e0 secure :-1 s=READY pgs=1252 cs=0 l=1 rev1=1 crypto rx=0x7ff89c009900 tx=0x7ff89c0055c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a157fe87aab19c5b server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:34:53.460 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:53.459+0000 7ff88be00640 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-03-21T10:34:53.460 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:53.460+0000 7ff88be00640 1 -- 172.21.15.47:0/1002711142 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff89c0035f0 con 0x55fceeab56d0 2024-03-21T10:34:53.460 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:53.460+0000 7ff8b3a50740 1 -- 172.21.15.47:0/1002711142 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ff89c02c7e0 con 0x55fceeab56d0 2024-03-21T10:34:53.460 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:53.460+0000 7ff88be00640 1 -- 172.21.15.47:0/1002711142 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(103..103 src has 1..103) v4 ==== 9562+0+0 (secure 0 0 0) 0x7ff89c003790 con 0x55fceeab56d0 2024-03-21T10:34:53.460 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:53.460+0000 7ff88be00640 1 -- 172.21.15.47:0/1002711142 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff89c05c050 con 0x55fceeab56d0 2024-03-21T10:34:53.460 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:53.460+0000 7ff8b3a50740 10 client.18594 fetch_fsmap learned FSMap version 32 2024-03-21T10:34:53.460 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:53.460+0000 7ff8b3a50740 1 -- 172.21.15.47:0/1002711142 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x55fcee7f77d0 con 0x55fceeab56d0 2024-03-21T10:34:53.462 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:53.460+0000 7ff88be00640 1 -- 172.21.15.47:0/1002711142 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7ff89c02c7e0 con 0x55fceeab56d0 2024-03-21T10:34:53.462 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:53.460+0000 7ff88be00640 1 -- 172.21.15.47:0/1002711142 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 32) v1 ==== 7942+0+0 (secure 0 0 0) 0x7ff89c004bd0 con 0x55fceeab56d0 2024-03-21T10:34:53.462 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:53.461+0000 7ff8b3a50740 10 client.18594 fetch_fsmap finished waiting for FSMap version 32 2024-03-21T10:34:53.462 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:53.461+0000 7ff8b3a50740 10 client.18594 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.a 2024-03-21T10:34:53.462 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:53.461+0000 7ff8b3a50740 20 client.18594 populate_metadata read hostname 'smithi047' 2024-03-21T10:34:53.462 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:53.461+0000 7ff8b3a50740 1 --2- 172.21.15.47:0/1002711142 >> [v2:172.21.15.47:6840/3401578978,v1:172.21.15.47:6841/3401578978] conn(0x55fceeace8d0 0x55fceead0cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:34:53.462 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:53.461+0000 7ff8b3a50740 4 client.18594 mds_command: new command op to 14619 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T10:34:53.462 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:53.461+0000 7ff8b3a50740 1 -- 172.21.15.47:0/1002711142 --> [v2:172.21.15.47:6840/3401578978,v1:172.21.15.47:6841/3401578978] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55fceea26800 con 0x55fceeace8d0 2024-03-21T10:34:53.462 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:53.461+0000 7ff8a9400640 1 --2- 172.21.15.47:0/1002711142 >> [v2:172.21.15.47:6840/3401578978,v1:172.21.15.47:6841/3401578978] conn(0x55fceeace8d0 0x55fceead0cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:34:53.462 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:53.462+0000 7ff88be00640 10 client.18594 ms_handle_connect on v2:172.21.15.47:6840/3401578978 2024-03-21T10:34:53.462 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:53.462+0000 7ff8a9400640 1 --2- 172.21.15.47:0/1002711142 >> [v2:172.21.15.47:6840/3401578978,v1:172.21.15.47:6841/3401578978] conn(0x55fceeace8d0 0x55fceead0cc0 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=5030139047e9f21f server_cookie=52707f6f65bf956 in_seq=0 out_seq=0 2024-03-21T10:34:53.463 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:34:53 smithi047 ceph-mon[31547]: pgmap v296: 129 pgs: 129 active+clean; 2.5 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-03-21T10:34:53.463 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:34:53 smithi047 ceph-mon[31547]: from='client.? 172.21.15.47:0/3030542532' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-21T10:34:53.464 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:53.462+0000 7ff88be00640 1 -- 172.21.15.47:0/1002711142 <== mds.4 v2:172.21.15.47:6840/3401578978 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55fceea26800 con 0x55fceeace8d0 2024-03-21T10:34:53.464 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:53.462+0000 7ff88be00640 10 client.18594 handle_command_reply: tid=0 2024-03-21T10:34:53.464 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:53.462+0000 7ff8b3a50740 1 client.18594 shutdown 2024-03-21T10:34:53.464 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:53.463+0000 7ff8b3a50740 1 -- 172.21.15.47:0/1002711142 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55fceeab56d0 msgr2=0x55fceeabe4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:34:53.464 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:53.463+0000 7ff8b3a50740 1 --2- 172.21.15.47:0/1002711142 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55fceeab56d0 0x55fceeabe4e0 secure :-1 s=READY pgs=1252 cs=0 l=1 rev1=1 crypto rx=0x7ff89c009900 tx=0x7ff89c0055c0 comp rx=0 tx=0).stop 2024-03-21T10:34:53.464 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:53.463+0000 7ff8b3a50740 1 -- 172.21.15.47:0/1002711142 shutdown_connections 2024-03-21T10:34:53.464 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:53.463+0000 7ff8b3a50740 1 --2- 172.21.15.47:0/1002711142 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fceeabec00 0x55fceeabbdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:53.464 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:53.463+0000 7ff8b3a50740 1 --2- 172.21.15.47:0/1002711142 >> [v2:172.21.15.47:6840/3401578978,v1:172.21.15.47:6841/3401578978] conn(0x55fceeace8d0 0x55fceead0cc0 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:53.464 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:53.463+0000 7ff8b3a50740 1 --2- 172.21.15.47:0/1002711142 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fceeab6080 0x55fceeabb870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:53.464 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:53.463+0000 7ff8b3a50740 1 --2- 172.21.15.47:0/1002711142 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55fceeab56d0 0x55fceeabe4e0 unknown :-1 s=CLOSED pgs=1252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:53.464 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:53.463+0000 7ff8b3a50740 1 -- 172.21.15.47:0/1002711142 >> 172.21.15.47:0/1002711142 conn(0x55fceeaafda0 msgr2=0x55fceeab1af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:34:53.465 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:53.463+0000 7ff88be00640 0 client.18594 ms_handle_reset on v2:172.21.15.47:6840/3401578978 2024-03-21T10:34:53.465 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:53.463+0000 7ff8b3a50740 1 -- 172.21.15.47:0/1002711142 shutdown_connections 2024-03-21T10:34:53.465 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:53.463+0000 7ff8b3a50740 1 -- 172.21.15.47:0/1002711142 wait complete. 2024-03-21T10:34:53.465 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:53.463+0000 7ff8b3a50740 20 client.18594 trim_cache size 0 max 16384 2024-03-21T10:34:53.475 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:53.477+0000 7ff8b3a50740 1 Processor -- start 2024-03-21T10:34:53.476 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:53.477+0000 7ff8b3a50740 1 -- start start 2024-03-21T10:34:53.476 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:53.478+0000 7ff8b3a50740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fceeab56d0 0x55fceeabdfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:34:53.476 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:53.478+0000 7ff8b3a50740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fceeab6080 0x55fceeabe4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:34:53.476 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:53.478+0000 7ff8b3a50740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55fceeabec00 0x55fceeab6b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:34:53.476 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:53.478+0000 7ff8b3a50740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8ac1b0ad0 con 0x55fceeabec00 2024-03-21T10:34:53.476 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:53.478+0000 7ff8b3a50740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8ac1b2450 con 0x55fceeab56d0 2024-03-21T10:34:53.476 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:53.478+0000 7ff8b3a50740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8ac1b1130 con 0x55fceeab6080 2024-03-21T10:34:53.478 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:53.478+0000 7ff8aa800640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55fceeabec00 0x55fceeab6b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:34:53.478 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:53.478+0000 7ff8a9e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fceeab56d0 0x55fceeabdfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:34:53.478 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:53.478+0000 7ff8aa800640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55fceeabec00 0x55fceeab6b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:55628/0 (socket says 172.21.15.47:55628) 2024-03-21T10:34:53.478 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:53.478+0000 7ff8aa800640 1 -- 172.21.15.47:0/1592215479 learned_addr learned my addr 172.21.15.47:0/1592215479 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:34:53.478 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:53.478+0000 7ff8a9400640 1 --2- 172.21.15.47:0/1592215479 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fceeab6080 0x55fceeabe4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:34:53.478 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:53.478+0000 7ff8aa800640 1 -- 172.21.15.47:0/1592215479 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fceeab6080 msgr2=0x55fceeabe4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:34:53.478 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:53.478+0000 7ff8aa800640 1 --2- 172.21.15.47:0/1592215479 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fceeab6080 0x55fceeabe4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:53.478 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:53.478+0000 7ff8aa800640 1 -- 172.21.15.47:0/1592215479 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fceeab56d0 msgr2=0x55fceeabdfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:34:53.478 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:53.478+0000 7ff8aa800640 1 --2- 172.21.15.47:0/1592215479 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fceeab56d0 0x55fceeabdfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:53.478 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:53.478+0000 7ff8aa800640 1 -- 172.21.15.47:0/1592215479 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55fceeab9dc0 con 0x55fceeabec00 2024-03-21T10:34:53.479 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:53.479+0000 7ff8aa800640 1 --2- 172.21.15.47:0/1592215479 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55fceeabec00 0x55fceeab6b90 secure :-1 s=READY pgs=1253 cs=0 l=1 rev1=1 crypto rx=0x7ff8a400bdf0 tx=0x7ff8a409b090 comp rx=0 tx=0).ready entity=mon.0 client_cookie=13d41c1cc5b34d60 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:34:53.479 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:53.479+0000 7ff896a00640 1 -- 172.21.15.47:0/1592215479 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8a409f4e0 con 0x55fceeabec00 2024-03-21T10:34:53.479 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:53.479+0000 7ff896a00640 1 -- 172.21.15.47:0/1592215479 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7ff8a40a3da0 con 0x55fceeabec00 2024-03-21T10:34:53.479 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:53.479+0000 7ff896a00640 1 -- 172.21.15.47:0/1592215479 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8a409f0d0 con 0x55fceeabec00 2024-03-21T10:34:53.479 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:53.480+0000 7ff8b3a50740 1 -- 172.21.15.47:0/1592215479 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55fceeabec00 msgr2=0x55fceeab6b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:34:53.479 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:53.480+0000 7ff8b3a50740 1 --2- 172.21.15.47:0/1592215479 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55fceeabec00 0x55fceeab6b90 secure :-1 s=READY pgs=1253 cs=0 l=1 rev1=1 crypto rx=0x7ff8a400bdf0 tx=0x7ff8a409b090 comp rx=0 tx=0).stop 2024-03-21T10:34:53.479 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:53.480+0000 7ff8b3a50740 1 -- 172.21.15.47:0/1592215479 shutdown_connections 2024-03-21T10:34:53.479 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:53.480+0000 7ff8b3a50740 1 --2- 172.21.15.47:0/1592215479 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55fceeabec00 0x55fceeab6b90 unknown :-1 s=CLOSED pgs=1253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:53.479 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:53.480+0000 7ff8b3a50740 1 --2- 172.21.15.47:0/1592215479 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fceeab6080 0x55fceeabe4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:53.479 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:53.480+0000 7ff8b3a50740 1 --2- 172.21.15.47:0/1592215479 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fceeab56d0 0x55fceeabdfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:53.479 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:53.480+0000 7ff8b3a50740 1 -- 172.21.15.47:0/1592215479 >> 172.21.15.47:0/1592215479 conn(0x55fceeaafda0 msgr2=0x55fceeab6480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:34:53.479 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:53.480+0000 7ff8b3a50740 1 -- 172.21.15.47:0/1592215479 shutdown_connections 2024-03-21T10:34:53.479 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:53.480+0000 7ff8b3a50740 1 -- 172.21.15.47:0/1592215479 wait complete. 2024-03-21T10:34:53.483 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:53.481+0000 7ff8b3a50740 1 -- start start 2024-03-21T10:34:53.483 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:53.481+0000 7ff8b3a50740 1 Processor -- start 2024-03-21T10:34:53.483 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:53.481+0000 7ff8b3a50740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55fceeab56d0 0x55fceead3770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:34:53.484 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:53.481+0000 7ff8b3a50740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fceeab6080 0x55fceead1b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:34:53.484 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:53.481+0000 7ff8b3a50740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fceeabec00 0x55fceead2050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:34:53.484 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:53.482+0000 7ff8b3a50740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8ac1b1df0 con 0x55fceeab56d0 2024-03-21T10:34:53.484 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:53.482+0000 7ff8b3a50740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8ac1b1790 con 0x55fceeabec00 2024-03-21T10:34:53.484 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:53.482+0000 7ff8b3a50740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8ac1aecb0 con 0x55fceeab6080 2024-03-21T10:34:53.484 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:53.482+0000 7ff8b3a50740 10 client.0 _ll_register_callbacks cb 0x7ff898009550 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T10:34:53.484 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:53.482+0000 7ff8a9e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55fceeab56d0 0x55fceead3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:34:53.484 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:53.482+0000 7ff8a9e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55fceeab56d0 0x55fceead3770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:55642/0 (socket says 172.21.15.47:55642) 2024-03-21T10:34:53.484 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:53.482+0000 7ff8a9e00640 1 -- 172.21.15.47:0/1459503611 learned_addr learned my addr 172.21.15.47:0/1459503611 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:34:53.484 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:53.482+0000 7ff8aa800640 1 --2- 172.21.15.47:0/1459503611 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fceeabec00 0x55fceead2050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:34:53.484 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:53.482+0000 7ff8a9e00640 1 -- 172.21.15.47:0/1459503611 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fceeab6080 msgr2=0x55fceead1b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:34:53.484 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:53.482+0000 7ff8a9e00640 1 --2- 172.21.15.47:0/1459503611 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fceeab6080 0x55fceead1b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:53.484 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:53.482+0000 7ff8a9e00640 1 -- 172.21.15.47:0/1459503611 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fceeabec00 msgr2=0x55fceead2050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:34:53.485 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:53.482+0000 7ff8a9e00640 1 --2- 172.21.15.47:0/1459503611 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fceeabec00 0x55fceead2050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:53.485 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:53.482+0000 7ff8a9e00640 1 -- 172.21.15.47:0/1459503611 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55fceead2590 con 0x55fceeab56d0 2024-03-21T10:34:53.485 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:53.482+0000 7ff8a9e00640 1 --2- 172.21.15.47:0/1459503611 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55fceeab56d0 0x55fceead3770 secure :-1 s=READY pgs=1254 cs=0 l=1 rev1=1 crypto rx=0x7ff89c005070 tx=0x7ff89c0045c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cac762a2a51f0af6 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:34:53.485 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:53.482+0000 7ff88be00640 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-03-21T10:34:53.485 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:53.483+0000 7ff88be00640 1 -- 172.21.15.47:0/1459503611 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff89c045020 con 0x55fceeab56d0 2024-03-21T10:34:53.485 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:53.483+0000 7ff88be00640 1 -- 172.21.15.47:0/1459503611 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(103..103 src has 1..103) v4 ==== 9562+0+0 (secure 0 0 0) 0x7ff89c041bf0 con 0x55fceeab56d0 2024-03-21T10:34:53.485 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:53.483+0000 7ff8b3a50740 1 -- 172.21.15.47:0/1459503611 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ff8a40a3da0 con 0x55fceeab56d0 2024-03-21T10:34:53.485 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:53.483+0000 7ff88be00640 1 -- 172.21.15.47:0/1459503611 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff89c02cca0 con 0x55fceeab56d0 2024-03-21T10:34:53.485 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:53.483+0000 7ff8b3a50740 10 client.18600 fetch_fsmap learned FSMap version 32 2024-03-21T10:34:53.485 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:53.483+0000 7ff8b3a50740 1 -- 172.21.15.47:0/1459503611 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x55fceeb2aae0 con 0x55fceeab56d0 2024-03-21T10:34:53.485 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:53.483+0000 7ff88be00640 1 -- 172.21.15.47:0/1459503611 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7ff8a40a3da0 con 0x55fceeab56d0 2024-03-21T10:34:53.485 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:53.484+0000 7ff88be00640 1 -- 172.21.15.47:0/1459503611 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 32) v1 ==== 7942+0+0 (secure 0 0 0) 0x7ff89c05f070 con 0x55fceeab56d0 2024-03-21T10:34:53.485 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:53.484+0000 7ff8b3a50740 10 client.18600 fetch_fsmap finished waiting for FSMap version 32 2024-03-21T10:34:53.485 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:53.484+0000 7ff8b3a50740 10 client.18600 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.a 2024-03-21T10:34:53.486 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:53.484+0000 7ff8b3a50740 20 client.18600 populate_metadata read hostname 'smithi047' 2024-03-21T10:34:53.486 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:53.484+0000 7ff8b3a50740 1 --2- 172.21.15.47:0/1459503611 >> [v2:172.21.15.47:6840/3401578978,v1:172.21.15.47:6841/3401578978] conn(0x55fceeb2b170 0x55fceeb2d590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:34:53.486 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:53.484+0000 7ff8b3a50740 4 client.18600 mds_command: new command op to 14619 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-21T10:34:53.486 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:53.484+0000 7ff8b3a50740 1 -- 172.21.15.47:0/1459503611 --> [v2:172.21.15.47:6840/3401578978,v1:172.21.15.47:6841/3401578978] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55fceea20b50 con 0x55fceeb2b170 2024-03-21T10:34:53.486 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:53.484+0000 7ff8a9400640 1 --2- 172.21.15.47:0/1459503611 >> [v2:172.21.15.47:6840/3401578978,v1:172.21.15.47:6841/3401578978] conn(0x55fceeb2b170 0x55fceeb2d590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:34:53.486 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:53.484+0000 7ff8a9400640 1 --2- 172.21.15.47:0/1459503611 >> [v2:172.21.15.47:6840/3401578978,v1:172.21.15.47:6841/3401578978] conn(0x55fceeb2b170 0x55fceeb2d590 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=f037a4d9e974fd8e server_cookie=5d413584d33dedb3 in_seq=0 out_seq=0 2024-03-21T10:34:53.486 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:53.484+0000 7ff88be00640 10 client.18600 ms_handle_connect on v2:172.21.15.47:6840/3401578978 2024-03-21T10:34:53.488 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:53.485+0000 7ff88be00640 1 -- 172.21.15.47:0/1459503611 <== mds.4 v2:172.21.15.47:6840/3401578978 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55fceea20b50 con 0x55fceeb2b170 2024-03-21T10:34:53.488 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:53.485+0000 7ff88be00640 10 client.18600 handle_command_reply: tid=0 2024-03-21T10:34:53.488 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:53.485+0000 7ff8b3a50740 1 client.18600 shutdown 2024-03-21T10:34:53.488 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:53.485+0000 7ff8b3a50740 1 -- 172.21.15.47:0/1459503611 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55fceeab56d0 msgr2=0x55fceead3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:34:53.488 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:53.485+0000 7ff8b3a50740 1 --2- 172.21.15.47:0/1459503611 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55fceeab56d0 0x55fceead3770 secure :-1 s=READY pgs=1254 cs=0 l=1 rev1=1 crypto rx=0x7ff89c005070 tx=0x7ff89c0045c0 comp rx=0 tx=0).stop 2024-03-21T10:34:53.488 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:53.485+0000 7ff8b3a50740 1 -- 172.21.15.47:0/1459503611 shutdown_connections 2024-03-21T10:34:53.488 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:53.485+0000 7ff8b3a50740 1 --2- 172.21.15.47:0/1459503611 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fceeabec00 0x55fceead2050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:53.488 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:53.485+0000 7ff8b3a50740 1 --2- 172.21.15.47:0/1459503611 >> [v2:172.21.15.47:6840/3401578978,v1:172.21.15.47:6841/3401578978] conn(0x55fceeb2b170 0x55fceeb2d590 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:53.488 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:53.485+0000 7ff8b3a50740 1 --2- 172.21.15.47:0/1459503611 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55fceeab6080 0x55fceead1b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:53.489 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:53.485+0000 7ff8b3a50740 1 --2- 172.21.15.47:0/1459503611 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55fceeab56d0 0x55fceead3770 unknown :-1 s=CLOSED pgs=1254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:53.489 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:53.485+0000 7ff88be00640 0 client.18600 ms_handle_reset on v2:172.21.15.47:6840/3401578978 2024-03-21T10:34:53.489 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:53.485+0000 7ff8b3a50740 1 -- 172.21.15.47:0/1459503611 >> 172.21.15.47:0/1459503611 conn(0x55fceeaafda0 msgr2=0x55fceeab6cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:34:53.489 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:53.485+0000 7ff8b3a50740 1 -- 172.21.15.47:0/1459503611 shutdown_connections 2024-03-21T10:34:53.489 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:53.485+0000 7ff8b3a50740 1 -- 172.21.15.47:0/1459503611 wait complete. 2024-03-21T10:34:53.489 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:53.485+0000 7ff8b3a50740 20 client.18600 trim_cache size 0 max 16384 2024-03-21T10:34:53.489 INFO:teuthology.orchestra.run.smithi047.stdout:[] 2024-03-21T10:34:53.490 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:53.486+0000 7ff8b0e00640 1 -- 172.21.15.47:0/317137689 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7ff8900a3110 msgr2=0x7ff8900a55d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:34:53.490 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:53.486+0000 7ff8b0e00640 1 --2- 172.21.15.47:0/317137689 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7ff8900a3110 0x7ff8900a55d0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7ff898009fd0 tx=0x7ff898009c60 comp rx=0 tx=0).stop 2024-03-21T10:34:53.490 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:53.486+0000 7ff8b0e00640 1 -- 172.21.15.47:0/317137689 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff8ac107660 msgr2=0x7ff8ac19e9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:34:53.490 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:53.486+0000 7ff8b0e00640 1 --2- 172.21.15.47:0/317137689 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff8ac107660 0x7ff8ac19e9b0 secure :-1 s=READY pgs=1250 cs=0 l=1 rev1=1 crypto rx=0x7ff8a400be20 tx=0x7ff8a400bf10 comp rx=0 tx=0).stop 2024-03-21T10:34:53.490 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:53.486+0000 7ff8b0e00640 1 -- 172.21.15.47:0/317137689 shutdown_connections 2024-03-21T10:34:53.490 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:53.486+0000 7ff8b0e00640 1 --2- 172.21.15.47:0/317137689 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff8ac107660 0x7ff8ac19e9b0 unknown :-1 s=CLOSED pgs=1250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:53.490 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:53.486+0000 7ff8b0e00640 1 --2- 172.21.15.47:0/317137689 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7ff8900a3110 0x7ff8900a55d0 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:53.490 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:53.486+0000 7ff8b0e00640 1 --2- 172.21.15.47:0/317137689 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff8ac06a590 0x7ff8ac19a630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:53.491 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:53.486+0000 7ff8b0e00640 1 --2- 172.21.15.47:0/317137689 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff8ac069c10 0x7ff8ac19a0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:53.491 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:53.486+0000 7ff8b0e00640 1 -- 172.21.15.47:0/317137689 >> 172.21.15.47:0/317137689 conn(0x7ff8ac077580 msgr2=0x7ff8ac10cdb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:34:53.491 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:53.486+0000 7ff8b0e00640 1 -- 172.21.15.47:0/317137689 shutdown_connections 2024-03-21T10:34:53.491 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:53.486+0000 7ff8b0e00640 1 -- 172.21.15.47:0/317137689 wait complete. 2024-03-21T10:34:53.500 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-21T10:34:53.500 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-03-21T10:34:53.711 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:53.712+0000 7fa73da00640 1 -- 172.21.15.47:0/3917294629 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa738102ff0 msgr2=0x7fa73810b4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:34:53.711 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:53.712+0000 7fa73da00640 1 --2- 172.21.15.47:0/3917294629 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa738102ff0 0x7fa73810b4d0 secure :-1 s=READY pgs=1255 cs=0 l=1 rev1=1 crypto rx=0x7fa720009900 tx=0x7fa72002bcd0 comp rx=0 tx=0).stop 2024-03-21T10:34:53.711 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:53.712+0000 7fa73da00640 1 -- 172.21.15.47:0/3917294629 shutdown_connections 2024-03-21T10:34:53.711 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:53.712+0000 7fa73da00640 1 --2- 172.21.15.47:0/3917294629 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa73810baf0 0x7fa73810df90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:53.711 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:53.712+0000 7fa73da00640 1 --2- 172.21.15.47:0/3917294629 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa738102ff0 0x7fa73810b4d0 unknown :-1 s=CLOSED pgs=1255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:53.711 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:53.712+0000 7fa73da00640 1 --2- 172.21.15.47:0/3917294629 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa7381026d0 0x7fa738102ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:53.711 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:53.712+0000 7fa73da00640 1 -- 172.21.15.47:0/3917294629 >> 172.21.15.47:0/3917294629 conn(0x7fa7380fbef0 msgr2=0x7fa7380fe310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:34:53.711 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:53.712+0000 7fa73da00640 1 -- 172.21.15.47:0/3917294629 shutdown_connections 2024-03-21T10:34:53.711 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:53.712+0000 7fa73da00640 1 -- 172.21.15.47:0/3917294629 wait complete. 2024-03-21T10:34:53.713 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:53.713+0000 7fa73da00640 1 Processor -- start 2024-03-21T10:34:53.713 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:53.714+0000 7fa73da00640 1 -- start start 2024-03-21T10:34:53.713 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:53.714+0000 7fa73da00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa7381026d0 0x7fa738100320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:34:53.713 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:53.714+0000 7fa73da00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa738102ff0 0x7fa738100860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:34:53.713 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:53.714+0000 7fa73da00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa73810baf0 0x7fa738100da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:34:53.713 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:53.714+0000 7fa73da00640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7381117d0 con 0x7fa738102ff0 2024-03-21T10:34:53.713 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:53.714+0000 7fa73da00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa738111650 con 0x7fa73810baf0 2024-03-21T10:34:53.713 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:53.714+0000 7fa73da00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa738111950 con 0x7fa7381026d0 2024-03-21T10:34:53.713 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:53.714+0000 7fa737400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa738102ff0 0x7fa738100860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:34:53.713 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:53.714+0000 7fa737400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa738102ff0 0x7fa738100860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:55654/0 (socket says 172.21.15.47:55654) 2024-03-21T10:34:53.713 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:53.714+0000 7fa737400640 1 -- 172.21.15.47:0/405301029 learned_addr learned my addr 172.21.15.47:0/405301029 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:34:53.713 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:53.715+0000 7fa73d000640 1 --2- 172.21.15.47:0/405301029 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa73810baf0 0x7fa738100da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:34:53.714 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:53.715+0000 7fa737e00640 1 --2- 172.21.15.47:0/405301029 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa7381026d0 0x7fa738100320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:34:53.714 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:53.715+0000 7fa737400640 1 -- 172.21.15.47:0/405301029 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa7381026d0 msgr2=0x7fa738100320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:34:53.714 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:53.715+0000 7fa737400640 1 --2- 172.21.15.47:0/405301029 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa7381026d0 0x7fa738100320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:53.714 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:53.715+0000 7fa737400640 1 -- 172.21.15.47:0/405301029 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa73810baf0 msgr2=0x7fa738100da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:34:53.714 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:53.715+0000 7fa737400640 1 --2- 172.21.15.47:0/405301029 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa73810baf0 0x7fa738100da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:53.714 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:53.715+0000 7fa737400640 1 -- 172.21.15.47:0/405301029 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7381a62c0 con 0x7fa738102ff0 2024-03-21T10:34:53.714 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:53.715+0000 7fa73d000640 1 --2- 172.21.15.47:0/405301029 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa73810baf0 0x7fa738100da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T10:34:53.715 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:53.715+0000 7fa737400640 1 --2- 172.21.15.47:0/405301029 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa738102ff0 0x7fa738100860 secure :-1 s=READY pgs=1256 cs=0 l=1 rev1=1 crypto rx=0x7fa720000c00 tx=0x7fa7200045b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:34:53.715 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:53.715+0000 7fa734c00640 1 -- 172.21.15.47:0/405301029 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa720042020 con 0x7fa738102ff0 2024-03-21T10:34:53.715 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:53.715+0000 7fa73da00640 1 -- 172.21.15.47:0/405301029 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa7381a6550 con 0x7fa738102ff0 2024-03-21T10:34:53.715 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:53.715+0000 7fa734c00640 1 -- 172.21.15.47:0/405301029 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fa720004e00 con 0x7fa738102ff0 2024-03-21T10:34:53.715 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:53.715+0000 7fa734c00640 1 -- 172.21.15.47:0/405301029 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7200038e0 con 0x7fa738102ff0 2024-03-21T10:34:53.715 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:53.716+0000 7fa73da00640 1 -- 172.21.15.47:0/405301029 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa7381a6aa0 con 0x7fa738102ff0 2024-03-21T10:34:53.717 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:53.718+0000 7fa741e03740 1 Processor -- start 2024-03-21T10:34:53.717 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:53.719+0000 7fa741e03740 1 -- start start 2024-03-21T10:34:53.717 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:53.719+0000 7fa734c00640 1 -- 172.21.15.47:0/405301029 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 27) v1 ==== 137985+0+0 (secure 0 0 0) 0x7fa720045020 con 0x7fa738102ff0 2024-03-21T10:34:53.717 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:53.719+0000 7fa741e03740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d4683338b0 0x55d468333c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:34:53.717 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:53.719+0000 7fa741e03740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d468334260 0x55d46833c810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:34:53.717 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:53.719+0000 7fa741e03740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d46833cde0 0x55d46833f230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:34:53.717 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:53.719+0000 7fa741e03740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d4682972c0 con 0x55d46833cde0 2024-03-21T10:34:53.717 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:53.719+0000 7fa741e03740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d46823fc90 con 0x55d468334260 2024-03-21T10:34:53.718 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:53.719+0000 7fa741e03740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d468293360 con 0x55d4683338b0 2024-03-21T10:34:53.720 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:53.719+0000 7fa737400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d46833cde0 0x55d46833f230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:34:53.720 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:53.720+0000 7fa737400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d46833cde0 0x55d46833f230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:55668/0 (socket says 172.21.15.47:55668) 2024-03-21T10:34:53.720 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:53.720+0000 7fa737400640 1 -- 172.21.15.47:0/1181566388 learned_addr learned my addr 172.21.15.47:0/1181566388 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:34:53.720 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:53.720+0000 7fa737e00640 1 --2- 172.21.15.47:0/1181566388 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d468334260 0x55d46833c810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:34:53.720 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:53.720+0000 7fa73d000640 1 --2- 172.21.15.47:0/1181566388 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d4683338b0 0x55d468333c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:34:53.720 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:53.720+0000 7fa737400640 1 -- 172.21.15.47:0/1181566388 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d4683338b0 msgr2=0x55d468333c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:34:53.720 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:53.720+0000 7fa737400640 1 --2- 172.21.15.47:0/1181566388 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d4683338b0 0x55d468333c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:53.720 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:53.720+0000 7fa737400640 1 -- 172.21.15.47:0/1181566388 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d468334260 msgr2=0x55d46833c810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:34:53.721 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:53.720+0000 7fa737400640 1 --2- 172.21.15.47:0/1181566388 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d468334260 0x55d46833c810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:53.721 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:53.720+0000 7fa737400640 1 -- 172.21.15.47:0/1181566388 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d468264080 con 0x55d46833cde0 2024-03-21T10:34:53.721 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:53.720+0000 7fa734c00640 1 --2- 172.21.15.47:0/405301029 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7fa71c0a3090 0x7fa71c0a5550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:34:53.721 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:53.721+0000 7fa734c00640 1 -- 172.21.15.47:0/405301029 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 9562+0+0 (secure 0 0 0) 0x7fa7200f1770 con 0x7fa738102ff0 2024-03-21T10:34:53.721 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:53.721+0000 7fa737400640 1 --2- 172.21.15.47:0/1181566388 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d46833cde0 0x55d46833f230 secure :-1 s=READY pgs=1257 cs=0 l=1 rev1=1 crypto rx=0x7fa7200eeaa0 tx=0x7fa7200f7b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=88b6ce50906d2c8c server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:34:53.721 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:53.721+0000 7fa73d000640 1 --2- 172.21.15.47:0/405301029 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7fa71c0a3090 0x7fa71c0a5550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:34:53.721 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:53.721+0000 7fa72d600640 1 -- 172.21.15.47:0/1181566388 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7200b9aa0 con 0x55d46833cde0 2024-03-21T10:34:53.721 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:53.721+0000 7fa72d600640 1 -- 172.21.15.47:0/1181566388 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fa7200bdf00 con 0x55d46833cde0 2024-03-21T10:34:53.721 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:53.721+0000 7fa72d600640 1 -- 172.21.15.47:0/1181566388 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7200bd160 con 0x55d46833cde0 2024-03-21T10:34:53.721 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:53.722+0000 7fa741e03740 1 -- 172.21.15.47:0/1181566388 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d46833cde0 msgr2=0x55d46833f230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:34:53.721 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:53.722+0000 7fa741e03740 1 --2- 172.21.15.47:0/1181566388 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d46833cde0 0x55d46833f230 secure :-1 s=READY pgs=1257 cs=0 l=1 rev1=1 crypto rx=0x7fa7200eeaa0 tx=0x7fa7200f7b20 comp rx=0 tx=0).stop 2024-03-21T10:34:53.721 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:53.722+0000 7fa73d000640 1 --2- 172.21.15.47:0/405301029 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7fa71c0a3090 0x7fa71c0a5550 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fa7300095f0 tx=0x7fa730005f90 comp rx=0 tx=0).ready entity=mgr.15687 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:34:53.721 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:53.722+0000 7fa741e03740 1 -- 172.21.15.47:0/1181566388 shutdown_connections 2024-03-21T10:34:53.721 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:53.722+0000 7fa741e03740 1 --2- 172.21.15.47:0/1181566388 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d46833cde0 0x55d46833f230 unknown :-1 s=CLOSED pgs=1257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:53.721 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:53.722+0000 7fa741e03740 1 --2- 172.21.15.47:0/1181566388 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d468334260 0x55d46833c810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:53.722 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:53.722+0000 7fa741e03740 1 --2- 172.21.15.47:0/1181566388 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d4683338b0 0x55d468333c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:53.722 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:53.722+0000 7fa741e03740 1 -- 172.21.15.47:0/1181566388 >> 172.21.15.47:0/1181566388 conn(0x55d46832df80 msgr2=0x55d46832e360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:34:53.722 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:53.722+0000 7fa741e03740 1 -- 172.21.15.47:0/1181566388 shutdown_connections 2024-03-21T10:34:53.722 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:53.722+0000 7fa741e03740 1 -- 172.21.15.47:0/1181566388 wait complete. 2024-03-21T10:34:53.725 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:53.722+0000 7fa741e03740 1 -- start start 2024-03-21T10:34:53.725 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:53.723+0000 7fa741e03740 1 Processor -- start 2024-03-21T10:34:53.726 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:53.723+0000 7fa741e03740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d4683338b0 0x55d46833c740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:34:53.726 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:53.723+0000 7fa741e03740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d468334260 0x55d468339ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:34:53.726 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:53.723+0000 7fa741e03740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d46833cde0 0x55d46833a0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:34:53.726 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:53.723+0000 7fa741e03740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d46833fb10 con 0x55d46833cde0 2024-03-21T10:34:53.726 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:53.723+0000 7fa741e03740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d4682530d0 con 0x55d4683338b0 2024-03-21T10:34:53.726 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:53.723+0000 7fa741e03740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d46833fc90 con 0x55d468334260 2024-03-21T10:34:53.726 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:53.723+0000 7fa741e03740 10 client.0 _ll_register_callbacks cb 0x55d468257e80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T10:34:53.726 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:53.723+0000 7fa73d000640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d46833cde0 0x55d46833a0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:34:53.726 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:53.723+0000 7fa73d000640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d46833cde0 0x55d46833a0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:55672/0 (socket says 172.21.15.47:55672) 2024-03-21T10:34:53.726 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:53.723+0000 7fa73d000640 1 -- 172.21.15.47:0/1119987963 learned_addr learned my addr 172.21.15.47:0/1119987963 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:34:53.726 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:53.724+0000 7fa737e00640 1 --2- 172.21.15.47:0/1119987963 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d4683338b0 0x55d46833c740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:34:53.726 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:53.724+0000 7fa737400640 1 --2- 172.21.15.47:0/1119987963 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d468334260 0x55d468339ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:34:53.727 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:53.724+0000 7fa73d000640 1 -- 172.21.15.47:0/1119987963 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d468334260 msgr2=0x55d468339ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:34:53.727 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:53.724+0000 7fa73d000640 1 --2- 172.21.15.47:0/1119987963 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d468334260 0x55d468339ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:53.727 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:53.724+0000 7fa73d000640 1 -- 172.21.15.47:0/1119987963 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d4683338b0 msgr2=0x55d46833c740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:34:53.727 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:53.724+0000 7fa73d000640 1 --2- 172.21.15.47:0/1119987963 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d4683338b0 0x55d46833c740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:53.727 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:53.724+0000 7fa73d000640 1 -- 172.21.15.47:0/1119987963 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55d467fab3f0 con 0x55d46833cde0 2024-03-21T10:34:53.727 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:53.724+0000 7fa73d000640 1 --2- 172.21.15.47:0/1119987963 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d46833cde0 0x55d46833a0e0 secure :-1 s=READY pgs=1258 cs=0 l=1 rev1=1 crypto rx=0x7fa73001e680 tx=0x7fa7300046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3d98cf7080534a80 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:34:53.727 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:53.724+0000 7fa726000640 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-03-21T10:34:53.727 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:53.725+0000 7fa726000640 1 -- 172.21.15.47:0/1119987963 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa730005020 con 0x55d46833cde0 2024-03-21T10:34:53.727 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:53.725+0000 7fa726000640 1 -- 172.21.15.47:0/1119987963 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(103..103 src has 1..103) v4 ==== 9562+0+0 (secure 0 0 0) 0x7fa7300051c0 con 0x55d46833cde0 2024-03-21T10:34:53.727 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:53.725+0000 7fa741e03740 1 -- 172.21.15.47:0/1119987963 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fa7200bdf00 con 0x55d46833cde0 2024-03-21T10:34:53.727 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:53.725+0000 7fa726000640 1 -- 172.21.15.47:0/1119987963 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa730031050 con 0x55d46833cde0 2024-03-21T10:34:53.727 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:53.725+0000 7fa741e03740 10 client.18612 fetch_fsmap learned FSMap version 32 2024-03-21T10:34:53.727 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:53.725+0000 7fa741e03740 1 -- 172.21.15.47:0/1119987963 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x55d4680757d0 con 0x55d46833cde0 2024-03-21T10:34:53.728 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:53.725+0000 7fa726000640 1 -- 172.21.15.47:0/1119987963 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fa7200bdf00 con 0x55d46833cde0 2024-03-21T10:34:53.728 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:53.726+0000 7fa726000640 1 -- 172.21.15.47:0/1119987963 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 32) v1 ==== 7942+0+0 (secure 0 0 0) 0x7fa73001a040 con 0x55d46833cde0 2024-03-21T10:34:53.728 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:53.726+0000 7fa741e03740 10 client.18612 fetch_fsmap finished waiting for FSMap version 32 2024-03-21T10:34:53.728 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:53.726+0000 7fa741e03740 10 client.18612 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.k 2024-03-21T10:34:53.728 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:53.726+0000 7fa741e03740 20 client.18612 populate_metadata read hostname 'smithi047' 2024-03-21T10:34:53.728 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:53.726+0000 7fa741e03740 1 --2- 172.21.15.47:0/1119987963 >> [v2:172.21.15.100:6832/2723302038,v1:172.21.15.100:6833/2723302038] conn(0x55d46834ca30 0x55d46834ee20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:34:53.728 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:53.726+0000 7fa741e03740 4 client.18612 mds_command: new command op to 34427 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T10:34:53.728 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:53.726+0000 7fa741e03740 1 -- 172.21.15.47:0/1119987963 --> [v2:172.21.15.100:6832/2723302038,v1:172.21.15.100:6833/2723302038] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55d4682a4800 con 0x55d46834ca30 2024-03-21T10:34:53.728 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:53.726+0000 7fa737e00640 1 --2- 172.21.15.47:0/1119987963 >> [v2:172.21.15.100:6832/2723302038,v1:172.21.15.100:6833/2723302038] conn(0x55d46834ca30 0x55d46834ee20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:34:53.728 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:53.727+0000 7fa737e00640 1 --2- 172.21.15.47:0/1119987963 >> [v2:172.21.15.100:6832/2723302038,v1:172.21.15.100:6833/2723302038] conn(0x55d46834ca30 0x55d46834ee20 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=7219e8483d5bfbbe server_cookie=a19d0df9958f6996 in_seq=0 out_seq=0 2024-03-21T10:34:53.728 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:53.727+0000 7fa726000640 10 client.18612 ms_handle_connect on v2:172.21.15.100:6832/2723302038 2024-03-21T10:34:53.729 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:34:53 smithi100 ceph-mon[35193]: pgmap v296: 129 pgs: 129 active+clean; 2.5 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-03-21T10:34:53.729 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:34:53 smithi100 ceph-mon[35193]: from='client.? 172.21.15.47:0/3030542532' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-21T10:34:53.730 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:53.729+0000 7fa726000640 1 -- 172.21.15.47:0/1119987963 <== mds.3 v2:172.21.15.100:6832/2723302038 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55d4682a4800 con 0x55d46834ca30 2024-03-21T10:34:53.730 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:53.729+0000 7fa726000640 10 client.18612 handle_command_reply: tid=0 2024-03-21T10:34:53.730 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:53.729+0000 7fa741e03740 1 client.18612 shutdown 2024-03-21T10:34:53.730 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:53.729+0000 7fa741e03740 1 -- 172.21.15.47:0/1119987963 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d46833cde0 msgr2=0x55d46833a0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:34:53.730 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:53.729+0000 7fa741e03740 1 --2- 172.21.15.47:0/1119987963 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d46833cde0 0x55d46833a0e0 secure :-1 s=READY pgs=1258 cs=0 l=1 rev1=1 crypto rx=0x7fa73001e680 tx=0x7fa7300046c0 comp rx=0 tx=0).stop 2024-03-21T10:34:53.730 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:53.729+0000 7fa741e03740 1 -- 172.21.15.47:0/1119987963 shutdown_connections 2024-03-21T10:34:53.730 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:53.729+0000 7fa741e03740 1 --2- 172.21.15.47:0/1119987963 >> [v2:172.21.15.100:6832/2723302038,v1:172.21.15.100:6833/2723302038] conn(0x55d46834ca30 0x55d46834ee20 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:53.730 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:53.729+0000 7fa741e03740 1 --2- 172.21.15.47:0/1119987963 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d46833cde0 0x55d46833a0e0 unknown :-1 s=CLOSED pgs=1258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:53.731 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:53.729+0000 7fa741e03740 1 --2- 172.21.15.47:0/1119987963 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d468334260 0x55d468339ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:53.731 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:53.729+0000 7fa741e03740 1 --2- 172.21.15.47:0/1119987963 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d4683338b0 0x55d46833c740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:53.731 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:53.729+0000 7fa741e03740 1 -- 172.21.15.47:0/1119987963 >> 172.21.15.47:0/1119987963 conn(0x55d46832df80 msgr2=0x55d46832fcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:34:53.731 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:53.729+0000 7fa726000640 0 client.18612 ms_handle_reset on v2:172.21.15.100:6832/2723302038 2024-03-21T10:34:53.731 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:53.729+0000 7fa741e03740 1 -- 172.21.15.47:0/1119987963 shutdown_connections 2024-03-21T10:34:53.731 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:53.729+0000 7fa741e03740 1 -- 172.21.15.47:0/1119987963 wait complete. 2024-03-21T10:34:53.731 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:53.729+0000 7fa741e03740 20 client.18612 trim_cache size 0 max 16384 2024-03-21T10:34:53.736 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:34:53 smithi157 ceph-mon[35581]: pgmap v296: 129 pgs: 129 active+clean; 2.5 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-03-21T10:34:53.737 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:34:53 smithi157 ceph-mon[35581]: from='client.? 172.21.15.47:0/3030542532' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-21T10:34:53.741 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:53.743+0000 7fa741e03740 1 Processor -- start 2024-03-21T10:34:53.741 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:53.743+0000 7fa741e03740 1 -- start start 2024-03-21T10:34:53.742 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:53.743+0000 7fa741e03740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d4683338b0 0x55d46832d3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:34:53.742 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:53.743+0000 7fa741e03740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d468334260 0x55d46833c200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:34:53.742 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:53.743+0000 7fa741e03740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d46833cde0 0x55d46833c740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:34:53.742 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:53.743+0000 7fa741e03740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7381b09f0 con 0x55d468334260 2024-03-21T10:34:53.742 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:53.743+0000 7fa741e03740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7381b2370 con 0x55d46833cde0 2024-03-21T10:34:53.742 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:53.743+0000 7fa741e03740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7381b1050 con 0x55d4683338b0 2024-03-21T10:34:53.742 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:53.743+0000 7fa737400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d468334260 0x55d46833c200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:34:53.742 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:53.743+0000 7fa737400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d468334260 0x55d46833c200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:55678/0 (socket says 172.21.15.47:55678) 2024-03-21T10:34:53.742 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:53.743+0000 7fa737400640 1 -- 172.21.15.47:0/1748706178 learned_addr learned my addr 172.21.15.47:0/1748706178 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:34:53.742 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:53.743+0000 7fa73d000640 1 --2- 172.21.15.47:0/1748706178 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d46833cde0 0x55d46833c740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:34:53.742 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:53.744+0000 7fa737e00640 1 --2- 172.21.15.47:0/1748706178 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d4683338b0 0x55d46832d3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:34:53.742 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:53.744+0000 7fa737400640 1 -- 172.21.15.47:0/1748706178 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d4683338b0 msgr2=0x55d46832d3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:34:53.743 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:53.744+0000 7fa737400640 1 --2- 172.21.15.47:0/1748706178 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d4683338b0 0x55d46832d3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:53.743 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:53.744+0000 7fa737400640 1 -- 172.21.15.47:0/1748706178 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d46833cde0 msgr2=0x55d46833c740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:34:53.743 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:53.744+0000 7fa737400640 1 --2- 172.21.15.47:0/1748706178 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d46833cde0 0x55d46833c740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:53.743 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:53.744+0000 7fa737400640 1 -- 172.21.15.47:0/1748706178 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d46833f540 con 0x55d468334260 2024-03-21T10:34:53.746 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:53.744+0000 7fa737400640 1 --2- 172.21.15.47:0/1748706178 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d468334260 0x55d46833c200 secure :-1 s=READY pgs=1259 cs=0 l=1 rev1=1 crypto rx=0x7fa7200bbab0 tx=0x7fa72002bcd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=de6c08fa13d63c8d server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:34:53.747 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:53.744+0000 7fa72cc00640 1 -- 172.21.15.47:0/1748706178 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7200bd340 con 0x55d468334260 2024-03-21T10:34:53.747 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:53.744+0000 7fa72cc00640 1 -- 172.21.15.47:0/1748706178 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fa7200d1e50 con 0x55d468334260 2024-03-21T10:34:53.747 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:53.744+0000 7fa72cc00640 1 -- 172.21.15.47:0/1748706178 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7200bd5d0 con 0x55d468334260 2024-03-21T10:34:53.747 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:53.745+0000 7fa741e03740 1 -- 172.21.15.47:0/1748706178 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d468334260 msgr2=0x55d46833c200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:34:53.747 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:53.745+0000 7fa741e03740 1 --2- 172.21.15.47:0/1748706178 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d468334260 0x55d46833c200 secure :-1 s=READY pgs=1259 cs=0 l=1 rev1=1 crypto rx=0x7fa7200bbab0 tx=0x7fa72002bcd0 comp rx=0 tx=0).stop 2024-03-21T10:34:53.747 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:53.745+0000 7fa741e03740 1 -- 172.21.15.47:0/1748706178 shutdown_connections 2024-03-21T10:34:53.747 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:53.745+0000 7fa741e03740 1 --2- 172.21.15.47:0/1748706178 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d46833cde0 0x55d46833c740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:53.747 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:53.745+0000 7fa741e03740 1 --2- 172.21.15.47:0/1748706178 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d468334260 0x55d46833c200 unknown :-1 s=CLOSED pgs=1259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:53.748 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:53.745+0000 7fa741e03740 1 --2- 172.21.15.47:0/1748706178 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d4683338b0 0x55d46832d3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:53.748 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:53.745+0000 7fa741e03740 1 -- 172.21.15.47:0/1748706178 >> 172.21.15.47:0/1748706178 conn(0x55d46832df80 msgr2=0x55d46832fbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:34:53.748 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:53.745+0000 7fa741e03740 1 -- 172.21.15.47:0/1748706178 shutdown_connections 2024-03-21T10:34:53.748 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:53.745+0000 7fa741e03740 1 -- 172.21.15.47:0/1748706178 wait complete. 2024-03-21T10:34:53.748 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:53.745+0000 7fa741e03740 1 -- start start 2024-03-21T10:34:53.748 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:53.745+0000 7fa741e03740 1 Processor -- start 2024-03-21T10:34:53.748 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:53.746+0000 7fa741e03740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d4683338b0 0x55d468343220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:34:53.748 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:53.746+0000 7fa741e03740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d468334260 0x55d468343760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:34:53.748 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:53.746+0000 7fa741e03740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d46833cde0 0x55d468343ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:34:53.748 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:53.746+0000 7fa741e03740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7381b1d10 con 0x55d4683338b0 2024-03-21T10:34:53.748 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:53.746+0000 7fa741e03740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7381b16b0 con 0x55d46833cde0 2024-03-21T10:34:53.748 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:53.746+0000 7fa741e03740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7381aebd0 con 0x55d468334260 2024-03-21T10:34:53.748 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:53.746+0000 7fa741e03740 10 client.0 _ll_register_callbacks cb 0x7fa730013f50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T10:34:53.748 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:53.746+0000 7fa737e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d4683338b0 0x55d468343220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:34:53.749 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:53.746+0000 7fa737e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d4683338b0 0x55d468343220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:55694/0 (socket says 172.21.15.47:55694) 2024-03-21T10:34:53.749 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:53.746+0000 7fa737e00640 1 -- 172.21.15.47:0/1608090944 learned_addr learned my addr 172.21.15.47:0/1608090944 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:34:53.749 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:53.746+0000 7fa73d000640 1 --2- 172.21.15.47:0/1608090944 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d46833cde0 0x55d468343ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:34:53.749 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:53.746+0000 7fa737400640 1 --2- 172.21.15.47:0/1608090944 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d468334260 0x55d468343760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:34:53.749 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:53.747+0000 7fa737e00640 1 -- 172.21.15.47:0/1608090944 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d468334260 msgr2=0x55d468343760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:34:53.749 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:53.747+0000 7fa737e00640 1 --2- 172.21.15.47:0/1608090944 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d468334260 0x55d468343760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:53.749 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:53.747+0000 7fa737e00640 1 -- 172.21.15.47:0/1608090944 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d46833cde0 msgr2=0x55d468343ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:34:53.749 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:53.747+0000 7fa737e00640 1 --2- 172.21.15.47:0/1608090944 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d46833cde0 0x55d468343ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:53.749 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:53.747+0000 7fa737e00640 1 -- 172.21.15.47:0/1608090944 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55d4683a82b0 con 0x55d4683338b0 2024-03-21T10:34:53.749 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:53.747+0000 7fa73d000640 1 --2- 172.21.15.47:0/1608090944 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d46833cde0 0x55d468343ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T10:34:53.749 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:53.747+0000 7fa737e00640 1 --2- 172.21.15.47:0/1608090944 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d4683338b0 0x55d468343220 secure :-1 s=READY pgs=1260 cs=0 l=1 rev1=1 crypto rx=0x55d468334660 tx=0x7fa7280118d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=911e5587f88bb1b6 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:34:53.749 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:53.747+0000 7fa726000640 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-03-21T10:34:53.749 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:53.747+0000 7fa726000640 1 -- 172.21.15.47:0/1608090944 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa728024020 con 0x55d4683338b0 2024-03-21T10:34:53.749 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:53.747+0000 7fa726000640 1 -- 172.21.15.47:0/1608090944 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(103..103 src has 1..103) v4 ==== 9562+0+0 (secure 0 0 0) 0x7fa7280209e0 con 0x55d4683338b0 2024-03-21T10:34:53.749 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:53.747+0000 7fa741e03740 1 -- 172.21.15.47:0/1608090944 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fa7200d1e50 con 0x55d4683338b0 2024-03-21T10:34:53.750 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:53.748+0000 7fa726000640 1 -- 172.21.15.47:0/1608090944 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa72802f050 con 0x55d4683338b0 2024-03-21T10:34:53.750 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:53.748+0000 7fa741e03740 10 client.18618 fetch_fsmap learned FSMap version 32 2024-03-21T10:34:53.750 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:53.748+0000 7fa741e03740 1 -- 172.21.15.47:0/1608090944 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x55d4683a95c0 con 0x55d4683338b0 2024-03-21T10:34:53.750 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:53.748+0000 7fa726000640 1 -- 172.21.15.47:0/1608090944 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fa7200d1e50 con 0x55d4683338b0 2024-03-21T10:34:53.750 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:53.748+0000 7fa726000640 1 -- 172.21.15.47:0/1608090944 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 32) v1 ==== 7942+0+0 (secure 0 0 0) 0x7fa728010430 con 0x55d4683338b0 2024-03-21T10:34:53.750 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:53.748+0000 7fa741e03740 10 client.18618 fetch_fsmap finished waiting for FSMap version 32 2024-03-21T10:34:53.750 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:53.748+0000 7fa741e03740 10 client.18618 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.k 2024-03-21T10:34:53.750 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:53.748+0000 7fa741e03740 20 client.18618 populate_metadata read hostname 'smithi047' 2024-03-21T10:34:53.750 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:53.748+0000 7fa741e03740 1 --2- 172.21.15.47:0/1608090944 >> [v2:172.21.15.100:6832/2723302038,v1:172.21.15.100:6833/2723302038] conn(0x55d4683a9d20 0x55d4683ac140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:34:53.750 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:53.748+0000 7fa741e03740 4 client.18618 mds_command: new command op to 34427 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-21T10:34:53.750 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:53.748+0000 7fa741e03740 1 -- 172.21.15.47:0/1608090944 --> [v2:172.21.15.100:6832/2723302038,v1:172.21.15.100:6833/2723302038] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55d46829eb50 con 0x55d4683a9d20 2024-03-21T10:34:53.750 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:53.749+0000 7fa737400640 1 --2- 172.21.15.47:0/1608090944 >> [v2:172.21.15.100:6832/2723302038,v1:172.21.15.100:6833/2723302038] conn(0x55d4683a9d20 0x55d4683ac140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:34:53.752 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:53.749+0000 7fa737400640 1 --2- 172.21.15.47:0/1608090944 >> [v2:172.21.15.100:6832/2723302038,v1:172.21.15.100:6833/2723302038] conn(0x55d4683a9d20 0x55d4683ac140 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=8c99896b35b5d5b1 server_cookie=4ddb612544215a27 in_seq=0 out_seq=0 2024-03-21T10:34:53.752 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:53.749+0000 7fa726000640 10 client.18618 ms_handle_connect on v2:172.21.15.100:6832/2723302038 2024-03-21T10:34:53.752 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:53.749+0000 7fa726000640 1 -- 172.21.15.47:0/1608090944 <== mds.3 v2:172.21.15.100:6832/2723302038 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55d46829eb50 con 0x55d4683a9d20 2024-03-21T10:34:53.752 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:53.749+0000 7fa726000640 10 client.18618 handle_command_reply: tid=0 2024-03-21T10:34:53.752 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:53.749+0000 7fa741e03740 1 client.18618 shutdown 2024-03-21T10:34:53.752 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:53.749+0000 7fa741e03740 1 -- 172.21.15.47:0/1608090944 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d4683338b0 msgr2=0x55d468343220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:34:53.752 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:53.749+0000 7fa741e03740 1 --2- 172.21.15.47:0/1608090944 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d4683338b0 0x55d468343220 secure :-1 s=READY pgs=1260 cs=0 l=1 rev1=1 crypto rx=0x55d468334660 tx=0x7fa7280118d0 comp rx=0 tx=0).stop 2024-03-21T10:34:53.752 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:53.750+0000 7fa741e03740 1 -- 172.21.15.47:0/1608090944 shutdown_connections 2024-03-21T10:34:53.752 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:53.750+0000 7fa741e03740 1 --2- 172.21.15.47:0/1608090944 >> [v2:172.21.15.100:6832/2723302038,v1:172.21.15.100:6833/2723302038] conn(0x55d4683a9d20 0x55d4683ac140 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:53.752 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:53.750+0000 7fa741e03740 1 --2- 172.21.15.47:0/1608090944 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d46833cde0 0x55d468343ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:53.752 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:53.750+0000 7fa741e03740 1 --2- 172.21.15.47:0/1608090944 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d468334260 0x55d468343760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:53.753 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:53.750+0000 7fa741e03740 1 --2- 172.21.15.47:0/1608090944 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d4683338b0 0x55d468343220 unknown :-1 s=CLOSED pgs=1260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:53.753 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:53.750+0000 7fa741e03740 1 -- 172.21.15.47:0/1608090944 >> 172.21.15.47:0/1608090944 conn(0x55d46832df80 msgr2=0x55d468334690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:34:53.753 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:53.750+0000 7fa726000640 0 client.18618 ms_handle_reset on v2:172.21.15.100:6832/2723302038 2024-03-21T10:34:53.753 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:53.750+0000 7fa741e03740 1 -- 172.21.15.47:0/1608090944 shutdown_connections 2024-03-21T10:34:53.753 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:53.750+0000 7fa741e03740 1 -- 172.21.15.47:0/1608090944 wait complete. 2024-03-21T10:34:53.753 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:53.750+0000 7fa741e03740 20 client.18618 trim_cache size 0 max 16384 2024-03-21T10:34:53.753 INFO:teuthology.orchestra.run.smithi047.stdout:[] 2024-03-21T10:34:53.754 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:53.750+0000 7fa73da00640 1 -- 172.21.15.47:0/405301029 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7fa71c0a3090 msgr2=0x7fa71c0a5550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:34:53.754 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:53.750+0000 7fa73da00640 1 --2- 172.21.15.47:0/405301029 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7fa71c0a3090 0x7fa71c0a5550 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fa7300095f0 tx=0x7fa730005f90 comp rx=0 tx=0).stop 2024-03-21T10:34:53.754 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:53.750+0000 7fa73da00640 1 -- 172.21.15.47:0/405301029 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa738102ff0 msgr2=0x7fa738100860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:34:53.754 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:53.750+0000 7fa73da00640 1 --2- 172.21.15.47:0/405301029 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa738102ff0 0x7fa738100860 secure :-1 s=READY pgs=1256 cs=0 l=1 rev1=1 crypto rx=0x7fa720000c00 tx=0x7fa7200045b0 comp rx=0 tx=0).stop 2024-03-21T10:34:53.754 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:53.751+0000 7fa73da00640 1 -- 172.21.15.47:0/405301029 shutdown_connections 2024-03-21T10:34:53.754 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:53.751+0000 7fa73da00640 1 --2- 172.21.15.47:0/405301029 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7fa71c0a3090 0x7fa71c0a5550 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:53.754 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:53.751+0000 7fa73da00640 1 --2- 172.21.15.47:0/405301029 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa73810baf0 0x7fa738100da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:53.754 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:53.751+0000 7fa73da00640 1 --2- 172.21.15.47:0/405301029 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa738102ff0 0x7fa738100860 unknown :-1 s=CLOSED pgs=1256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:53.754 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:53.751+0000 7fa73da00640 1 --2- 172.21.15.47:0/405301029 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa7381026d0 0x7fa738100320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:53.755 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:53.751+0000 7fa73da00640 1 -- 172.21.15.47:0/405301029 >> 172.21.15.47:0/405301029 conn(0x7fa7380fbef0 msgr2=0x7fa7380fcca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:34:53.755 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:53.751+0000 7fa73da00640 1 -- 172.21.15.47:0/405301029 shutdown_connections 2024-03-21T10:34:53.755 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:53.751+0000 7fa73da00640 1 -- 172.21.15.47:0/405301029 wait complete. 2024-03-21T10:34:53.765 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-21T10:34:53.765 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-03-21T10:34:53.989 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:53.991+0000 7fbdeea00640 1 -- 172.21.15.47:0/339590220 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbde8103720 msgr2=0x7fbde8103ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:34:53.989 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:53.991+0000 7fbdeea00640 1 --2- 172.21.15.47:0/339590220 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbde8103720 0x7fbde8103ba0 secure :-1 s=READY pgs=1261 cs=0 l=1 rev1=1 crypto rx=0x7fbdd4009930 tx=0x7fbdd402bc30 comp rx=0 tx=0).stop 2024-03-21T10:34:53.990 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:53.991+0000 7fbdeea00640 1 -- 172.21.15.47:0/339590220 shutdown_connections 2024-03-21T10:34:53.990 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:53.991+0000 7fbdeea00640 1 --2- 172.21.15.47:0/339590220 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbde81040e0 0x7fbde81105f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:53.990 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:53.991+0000 7fbdeea00640 1 --2- 172.21.15.47:0/339590220 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbde8103720 0x7fbde8103ba0 unknown :-1 s=CLOSED pgs=1261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:53.990 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:53.991+0000 7fbdeea00640 1 --2- 172.21.15.47:0/339590220 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbde8109730 0x7fbde8109b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:53.990 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:53.991+0000 7fbdeea00640 1 -- 172.21.15.47:0/339590220 >> 172.21.15.47:0/339590220 conn(0x7fbde80ff1d0 msgr2=0x7fbde81015f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:34:53.990 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:53.991+0000 7fbdeea00640 1 -- 172.21.15.47:0/339590220 shutdown_connections 2024-03-21T10:34:53.990 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:53.991+0000 7fbdeea00640 1 -- 172.21.15.47:0/339590220 wait complete. 2024-03-21T10:34:53.992 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:53.992+0000 7fbdeea00640 1 Processor -- start 2024-03-21T10:34:53.992 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:53.992+0000 7fbdeea00640 1 -- start start 2024-03-21T10:34:53.992 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:53.992+0000 7fbdeea00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbde8103720 0x7fbde8199fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:34:53.993 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:53.992+0000 7fbdeea00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbde81040e0 0x7fbde819a500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:34:53.993 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:53.993+0000 7fbdeea00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbde8109730 0x7fbde819e880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:34:53.993 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:53.993+0000 7fbdeea00640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbde8113a40 con 0x7fbde8109730 2024-03-21T10:34:53.993 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:53.993+0000 7fbdeea00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbde81138c0 con 0x7fbde81040e0 2024-03-21T10:34:53.993 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:53.993+0000 7fbdeea00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbde8113bc0 con 0x7fbde8103720 2024-03-21T10:34:53.993 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:53.993+0000 7fbdee000640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbde8109730 0x7fbde819e880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:34:53.993 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:53.993+0000 7fbdee000640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbde8109730 0x7fbde819e880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:55716/0 (socket says 172.21.15.47:55716) 2024-03-21T10:34:53.993 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:53.993+0000 7fbdee000640 1 -- 172.21.15.47:0/2332535704 learned_addr learned my addr 172.21.15.47:0/2332535704 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:34:53.993 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:53.993+0000 7fbdecc00640 1 --2- 172.21.15.47:0/2332535704 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbde81040e0 0x7fbde819a500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:34:53.993 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:53.993+0000 7fbded600640 1 --2- 172.21.15.47:0/2332535704 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbde8103720 0x7fbde8199fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:34:53.993 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:53.993+0000 7fbdee000640 1 -- 172.21.15.47:0/2332535704 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbde8103720 msgr2=0x7fbde8199fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:34:53.993 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:53.993+0000 7fbdee000640 1 --2- 172.21.15.47:0/2332535704 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbde8103720 0x7fbde8199fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:53.993 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:53.993+0000 7fbdee000640 1 -- 172.21.15.47:0/2332535704 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbde81040e0 msgr2=0x7fbde819a500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:34:53.993 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:53.993+0000 7fbdee000640 1 --2- 172.21.15.47:0/2332535704 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbde81040e0 0x7fbde819a500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:53.994 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:53.993+0000 7fbdee000640 1 -- 172.21.15.47:0/2332535704 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbde819efb0 con 0x7fbde8109730 2024-03-21T10:34:53.994 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:53.994+0000 7fbdee000640 1 --2- 172.21.15.47:0/2332535704 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbde8109730 0x7fbde819e880 secure :-1 s=READY pgs=1262 cs=0 l=1 rev1=1 crypto rx=0x7fbde000eaa0 tx=0x7fbde000ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:34:53.994 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:53.994+0000 7fbde6000640 1 -- 172.21.15.47:0/2332535704 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbde000cdb0 con 0x7fbde8109730 2024-03-21T10:34:53.994 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:53.994+0000 7fbde6000640 1 -- 172.21.15.47:0/2332535704 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fbde0005020 con 0x7fbde8109730 2024-03-21T10:34:53.994 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:53.994+0000 7fbdeea00640 1 -- 172.21.15.47:0/2332535704 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbde819f2a0 con 0x7fbde8109730 2024-03-21T10:34:53.994 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:53.994+0000 7fbde6000640 1 -- 172.21.15.47:0/2332535704 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbde00116d0 con 0x7fbde8109730 2024-03-21T10:34:53.994 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:53.994+0000 7fbdeea00640 1 -- 172.21.15.47:0/2332535704 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbde81a6b30 con 0x7fbde8109730 2024-03-21T10:34:53.995 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:53.998+0000 7fbde6000640 1 -- 172.21.15.47:0/2332535704 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 27) v1 ==== 137985+0+0 (secure 0 0 0) 0x7fbde00051d0 con 0x7fbde8109730 2024-03-21T10:34:53.996 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:53.998+0000 7fbdf2dc0740 1 Processor -- start 2024-03-21T10:34:53.996 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:53.998+0000 7fbdf2dc0740 1 -- start start 2024-03-21T10:34:53.998 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:53.999+0000 7fbdf2dc0740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f214be48b0 0x55f214be4c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:34:53.999 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:53.999+0000 7fbdf2dc0740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f214be5260 0x55f214bed810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:34:53.999 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:53.999+0000 7fbdf2dc0740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f214bedde0 0x55f214bf0230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:34:53.999 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:53.999+0000 7fbdf2dc0740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f214b482c0 con 0x55f214be48b0 2024-03-21T10:34:53.999 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:53.999+0000 7fbdf2dc0740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f214af0c90 con 0x55f214bedde0 2024-03-21T10:34:53.999 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:53.999+0000 7fbdf2dc0740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f214b44360 con 0x55f214be5260 2024-03-21T10:34:53.999 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:53.999+0000 7fbdecc00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f214be48b0 0x55f214be4c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:34:53.999 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:53.999+0000 7fbdecc00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f214be48b0 0x55f214be4c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:55722/0 (socket says 172.21.15.47:55722) 2024-03-21T10:34:53.999 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:53.999+0000 7fbdecc00640 1 -- 172.21.15.47:0/3479605514 learned_addr learned my addr 172.21.15.47:0/3479605514 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:34:53.999 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:53.999+0000 7fbde6000640 1 --2- 172.21.15.47:0/2332535704 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7fbdd00a3090 0x7fbdd00a5550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:34:53.999 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:53.999+0000 7fbded600640 1 --2- 172.21.15.47:0/3479605514 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f214be5260 0x55f214bed810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:34:53.999 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:53.999+0000 7fbdecc00640 1 -- 172.21.15.47:0/3479605514 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f214be5260 msgr2=0x55f214bed810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:34:53.999 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:54.000+0000 7fbdecc00640 1 --2- 172.21.15.47:0/3479605514 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f214be5260 0x55f214bed810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:54.000 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:54.000+0000 7fbdecc00640 1 -- 172.21.15.47:0/3479605514 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f214bedde0 msgr2=0x55f214bf0230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:34:54.000 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:54.000+0000 7fbdecc00640 1 --2- 172.21.15.47:0/3479605514 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f214bedde0 0x55f214bf0230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:54.000 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:54.000+0000 7fbdecc00640 1 -- 172.21.15.47:0/3479605514 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f214b15080 con 0x55f214be48b0 2024-03-21T10:34:54.000 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:54.000+0000 7fbdee000640 1 --2- 172.21.15.47:0/2332535704 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7fbdd00a3090 0x7fbdd00a5550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:34:54.000 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:54.000+0000 7fbded600640 1 --2- 172.21.15.47:0/3479605514 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f214be5260 0x55f214bed810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T10:34:54.000 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:54.000+0000 7fbde6000640 1 -- 172.21.15.47:0/2332535704 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 9562+0+0 (secure 0 0 0) 0x7fbde00cfe10 con 0x7fbde8109730 2024-03-21T10:34:54.000 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:54.000+0000 7fbdecc00640 1 --2- 172.21.15.47:0/3479605514 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f214be48b0 0x55f214be4c90 secure :-1 s=READY pgs=1263 cs=0 l=1 rev1=1 crypto rx=0x7fbdd4005ab0 tx=0x7fbdd402c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=13eb86dfc819f2d6 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:34:54.000 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:54.000+0000 7fbdda600640 1 -- 172.21.15.47:0/3479605514 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbdd4045020 con 0x55f214be48b0 2024-03-21T10:34:54.000 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:54.000+0000 7fbdee000640 1 --2- 172.21.15.47:0/2332535704 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7fbdd00a3090 0x7fbdd00a5550 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fbde0009270 tx=0x7fbde00513d0 comp rx=0 tx=0).ready entity=mgr.15687 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:34:54.000 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:54.000+0000 7fbdda600640 1 -- 172.21.15.47:0/3479605514 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fbdd402c7e0 con 0x55f214be48b0 2024-03-21T10:34:54.000 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:54.001+0000 7fbdda600640 1 -- 172.21.15.47:0/3479605514 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbdd4005580 con 0x55f214be48b0 2024-03-21T10:34:54.004 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:54.001+0000 7fbdf2dc0740 1 -- 172.21.15.47:0/3479605514 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f214be48b0 msgr2=0x55f214be4c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:34:54.004 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:54.001+0000 7fbdf2dc0740 1 --2- 172.21.15.47:0/3479605514 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f214be48b0 0x55f214be4c90 secure :-1 s=READY pgs=1263 cs=0 l=1 rev1=1 crypto rx=0x7fbdd4005ab0 tx=0x7fbdd402c6a0 comp rx=0 tx=0).stop 2024-03-21T10:34:54.004 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:54.001+0000 7fbdf2dc0740 1 -- 172.21.15.47:0/3479605514 shutdown_connections 2024-03-21T10:34:54.004 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:54.001+0000 7fbdf2dc0740 1 --2- 172.21.15.47:0/3479605514 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f214bedde0 0x55f214bf0230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:54.004 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:54.001+0000 7fbdf2dc0740 1 --2- 172.21.15.47:0/3479605514 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f214be5260 0x55f214bed810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:54.004 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:54.001+0000 7fbdf2dc0740 1 --2- 172.21.15.47:0/3479605514 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f214be48b0 0x55f214be4c90 unknown :-1 s=CLOSED pgs=1263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:54.005 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:54.001+0000 7fbdf2dc0740 1 -- 172.21.15.47:0/3479605514 >> 172.21.15.47:0/3479605514 conn(0x55f214bdef80 msgr2=0x55f214bdf360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:34:54.005 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:54.001+0000 7fbdf2dc0740 1 -- 172.21.15.47:0/3479605514 shutdown_connections 2024-03-21T10:34:54.005 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:54.002+0000 7fbdf2dc0740 1 -- 172.21.15.47:0/3479605514 wait complete. 2024-03-21T10:34:54.005 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:54.002+0000 7fbdf2dc0740 1 -- start start 2024-03-21T10:34:54.005 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:54.002+0000 7fbdf2dc0740 1 Processor -- start 2024-03-21T10:34:54.005 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:54.003+0000 7fbdf2dc0740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f214be48b0 0x55f214bed6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:34:54.005 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:54.003+0000 7fbdf2dc0740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f214be5260 0x55f214beaa80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:34:54.005 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:54.003+0000 7fbdf2dc0740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f214bedde0 0x55f214beb020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:34:54.005 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:54.003+0000 7fbdf2dc0740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f214bf0cf0 con 0x55f214be5260 2024-03-21T10:34:54.005 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:54.003+0000 7fbdf2dc0740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f214b040d0 con 0x55f214bedde0 2024-03-21T10:34:54.005 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:54.003+0000 7fbdf2dc0740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f214bf0e70 con 0x55f214be48b0 2024-03-21T10:34:54.005 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:54.003+0000 7fbdf2dc0740 10 client.0 _ll_register_callbacks cb 0x55f214b08e80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T10:34:54.006 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:54.003+0000 7fbded600640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f214be5260 0x55f214beaa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:34:54.006 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:54.004+0000 7fbded600640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f214be5260 0x55f214beaa80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:55738/0 (socket says 172.21.15.47:55738) 2024-03-21T10:34:54.006 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:54.004+0000 7fbded600640 1 -- 172.21.15.47:0/2113021406 learned_addr learned my addr 172.21.15.47:0/2113021406 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:34:54.006 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:54.004+0000 7fbdecc00640 1 --2- 172.21.15.47:0/2113021406 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f214bedde0 0x55f214beb020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:34:54.006 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:54.004+0000 7fbdecc00640 1 --2- 172.21.15.47:0/2113021406 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f214be48b0 0x55f214bed6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:34:54.006 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:54.004+0000 7fbded600640 1 -- 172.21.15.47:0/2113021406 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f214be48b0 msgr2=0x55f214bed6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:34:54.006 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:54.004+0000 7fbded600640 1 --2- 172.21.15.47:0/2113021406 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f214be48b0 0x55f214bed6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:54.006 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:54.004+0000 7fbded600640 1 -- 172.21.15.47:0/2113021406 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f214bedde0 msgr2=0x55f214beb020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:34:54.006 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:54.004+0000 7fbded600640 1 --2- 172.21.15.47:0/2113021406 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f214bedde0 0x55f214beb020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:54.006 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:54.004+0000 7fbded600640 1 -- 172.21.15.47:0/2113021406 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55f21485c3f0 con 0x55f214be5260 2024-03-21T10:34:54.006 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:54.004+0000 7fbded600640 1 --2- 172.21.15.47:0/2113021406 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f214be5260 0x55f214beaa80 secure :-1 s=READY pgs=1264 cs=0 l=1 rev1=1 crypto rx=0x7fbddc0124f0 tx=0x7fbddc0129b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8645eb29178e4a20 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:34:54.006 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:54.004+0000 7fbdcea00640 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-03-21T10:34:54.006 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:54.005+0000 7fbdcea00640 1 -- 172.21.15.47:0/2113021406 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbddc01f060 con 0x55f214be5260 2024-03-21T10:34:54.007 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:54.005+0000 7fbdcea00640 1 -- 172.21.15.47:0/2113021406 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(103..103 src has 1..103) v4 ==== 9562+0+0 (secure 0 0 0) 0x7fbddc004460 con 0x55f214be5260 2024-03-21T10:34:54.007 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:54.005+0000 7fbdf2dc0740 1 -- 172.21.15.47:0/2113021406 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fbdd402c7e0 con 0x55f214be5260 2024-03-21T10:34:54.007 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:54.005+0000 7fbdcea00640 1 -- 172.21.15.47:0/2113021406 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7fbddc026050 con 0x55f214be5260 2024-03-21T10:34:54.007 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:54.005+0000 7fbdf2dc0740 10 client.18630 fetch_fsmap learned FSMap version 32 2024-03-21T10:34:54.007 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:54.005+0000 7fbdf2dc0740 1 -- 172.21.15.47:0/2113021406 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x55f2149267d0 con 0x55f214be5260 2024-03-21T10:34:54.007 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:54.005+0000 7fbdcea00640 1 -- 172.21.15.47:0/2113021406 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fbdd402c7e0 con 0x55f214be5260 2024-03-21T10:34:54.007 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:54.005+0000 7fbdcea00640 1 -- 172.21.15.47:0/2113021406 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 32) v1 ==== 7942+0+0 (secure 0 0 0) 0x7fbddc010a50 con 0x55f214be5260 2024-03-21T10:34:54.007 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:54.006+0000 7fbdf2dc0740 10 client.18630 fetch_fsmap finished waiting for FSMap version 32 2024-03-21T10:34:54.007 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:54.006+0000 7fbdf2dc0740 10 client.18630 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.e 2024-03-21T10:34:54.007 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:54.006+0000 7fbdf2dc0740 20 client.18630 populate_metadata read hostname 'smithi047' 2024-03-21T10:34:54.007 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:54.006+0000 7fbdf2dc0740 1 --2- 172.21.15.47:0/2113021406 >> [v2:172.21.15.100:6834/2597849957,v1:172.21.15.100:6835/2597849957] conn(0x55f214bfdbd0 0x55f214bfffc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:34:54.007 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:54.006+0000 7fbdf2dc0740 4 client.18630 mds_command: new command op to 34454 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T10:34:54.007 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:54.006+0000 7fbdf2dc0740 1 -- 172.21.15.47:0/2113021406 --> [v2:172.21.15.100:6834/2597849957,v1:172.21.15.100:6835/2597849957] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55f214b55800 con 0x55f214bfdbd0 2024-03-21T10:34:54.007 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:54.006+0000 7fbdecc00640 1 --2- 172.21.15.47:0/2113021406 >> [v2:172.21.15.100:6834/2597849957,v1:172.21.15.100:6835/2597849957] conn(0x55f214bfdbd0 0x55f214bfffc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:34:54.009 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:54.007+0000 7fbdecc00640 1 --2- 172.21.15.47:0/2113021406 >> [v2:172.21.15.100:6834/2597849957,v1:172.21.15.100:6835/2597849957] conn(0x55f214bfdbd0 0x55f214bfffc0 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=aac61ac63cf9cb98 server_cookie=cf39a855e7d20fe9 in_seq=0 out_seq=0 2024-03-21T10:34:54.009 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:54.007+0000 7fbdcea00640 10 client.18630 ms_handle_connect on v2:172.21.15.100:6834/2597849957 2024-03-21T10:34:54.009 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:54.008+0000 7fbdcea00640 1 -- 172.21.15.47:0/2113021406 <== mds.2 v2:172.21.15.100:6834/2597849957 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55f214b55800 con 0x55f214bfdbd0 2024-03-21T10:34:54.009 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:54.008+0000 7fbdcea00640 10 client.18630 handle_command_reply: tid=0 2024-03-21T10:34:54.009 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:54.009+0000 7fbdf2dc0740 1 client.18630 shutdown 2024-03-21T10:34:54.009 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:54.009+0000 7fbdf2dc0740 1 -- 172.21.15.47:0/2113021406 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f214be5260 msgr2=0x55f214beaa80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:34:54.009 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:54.009+0000 7fbdf2dc0740 1 --2- 172.21.15.47:0/2113021406 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f214be5260 0x55f214beaa80 secure :-1 s=READY pgs=1264 cs=0 l=1 rev1=1 crypto rx=0x7fbddc0124f0 tx=0x7fbddc0129b0 comp rx=0 tx=0).stop 2024-03-21T10:34:54.009 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:54.009+0000 7fbdf2dc0740 1 -- 172.21.15.47:0/2113021406 shutdown_connections 2024-03-21T10:34:54.009 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:54.009+0000 7fbdf2dc0740 1 --2- 172.21.15.47:0/2113021406 >> [v2:172.21.15.100:6834/2597849957,v1:172.21.15.100:6835/2597849957] conn(0x55f214bfdbd0 0x55f214bfffc0 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:54.009 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:54.009+0000 7fbdf2dc0740 1 --2- 172.21.15.47:0/2113021406 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f214bedde0 0x55f214beb020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:54.009 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:54.009+0000 7fbdf2dc0740 1 --2- 172.21.15.47:0/2113021406 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f214be5260 0x55f214beaa80 unknown :-1 s=CLOSED pgs=1264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:54.009 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:54.009+0000 7fbdf2dc0740 1 --2- 172.21.15.47:0/2113021406 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f214be48b0 0x55f214bed6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:54.009 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:54.009+0000 7fbdcea00640 0 client.18630 ms_handle_reset on v2:172.21.15.100:6834/2597849957 2024-03-21T10:34:54.010 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:54.009+0000 7fbdf2dc0740 1 -- 172.21.15.47:0/2113021406 >> 172.21.15.47:0/2113021406 conn(0x55f214bdef80 msgr2=0x55f214be0cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:34:54.010 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:54.009+0000 7fbdf2dc0740 1 -- 172.21.15.47:0/2113021406 shutdown_connections 2024-03-21T10:34:54.010 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:54.009+0000 7fbdf2dc0740 1 -- 172.21.15.47:0/2113021406 wait complete. 2024-03-21T10:34:54.010 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:54.009+0000 7fbdf2dc0740 20 client.18630 trim_cache size 0 max 16384 2024-03-21T10:34:54.023 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:54.024+0000 7fbdf2dc0740 1 Processor -- start 2024-03-21T10:34:54.023 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:54.024+0000 7fbdf2dc0740 1 -- start start 2024-03-21T10:34:54.023 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:54.024+0000 7fbdf2dc0740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f214be48b0 0x55f214bda060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:34:54.023 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:54.024+0000 7fbdf2dc0740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f214be5260 0x55f214bed1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:34:54.023 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:54.024+0000 7fbdf2dc0740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f214bedde0 0x55f214bed6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:34:54.023 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:54.024+0000 7fbdf2dc0740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbde81b0a70 con 0x55f214be5260 2024-03-21T10:34:54.023 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:54.024+0000 7fbdf2dc0740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbde81b23f0 con 0x55f214bedde0 2024-03-21T10:34:54.024 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:54.024+0000 7fbdf2dc0740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbde81b10d0 con 0x55f214be48b0 2024-03-21T10:34:54.024 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:54.024+0000 7fbded600640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f214be5260 0x55f214bed1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:34:54.024 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:54.025+0000 7fbded600640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f214be5260 0x55f214bed1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:55746/0 (socket says 172.21.15.47:55746) 2024-03-21T10:34:54.024 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:54.025+0000 7fbded600640 1 -- 172.21.15.47:0/3810541108 learned_addr learned my addr 172.21.15.47:0/3810541108 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:34:54.024 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:54.025+0000 7fbdecc00640 1 --2- 172.21.15.47:0/3810541108 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f214bedde0 0x55f214bed6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:34:54.024 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:54.025+0000 7fbded600640 1 -- 172.21.15.47:0/3810541108 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f214be48b0 msgr2=0x55f214bda060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:34:54.024 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:54.025+0000 7fbded600640 1 --2- 172.21.15.47:0/3810541108 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f214be48b0 0x55f214bda060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:54.024 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:54.025+0000 7fbded600640 1 -- 172.21.15.47:0/3810541108 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f214bedde0 msgr2=0x55f214bed6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:34:54.024 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:54.025+0000 7fbded600640 1 --2- 172.21.15.47:0/3810541108 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f214bedde0 0x55f214bed6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:54.024 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:54.025+0000 7fbded600640 1 -- 172.21.15.47:0/3810541108 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f214c00620 con 0x55f214be5260 2024-03-21T10:34:54.024 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:54.025+0000 7fbded600640 1 --2- 172.21.15.47:0/3810541108 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f214be5260 0x55f214bed1b0 secure :-1 s=READY pgs=1265 cs=0 l=1 rev1=1 crypto rx=0x7fbddc010420 tx=0x7fbddc00e5c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2f551bfcbbed0056 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:34:54.027 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:54.025+0000 7fbdd9c00640 1 -- 172.21.15.47:0/3810541108 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbddc0052d0 con 0x55f214be5260 2024-03-21T10:34:54.027 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:54.025+0000 7fbdd9c00640 1 -- 172.21.15.47:0/3810541108 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fbddc010a80 con 0x55f214be5260 2024-03-21T10:34:54.027 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:54.025+0000 7fbdd9c00640 1 -- 172.21.15.47:0/3810541108 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbddc00ed60 con 0x55f214be5260 2024-03-21T10:34:54.028 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:54.026+0000 7fbdf2dc0740 1 -- 172.21.15.47:0/3810541108 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f214be5260 msgr2=0x55f214bed1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:34:54.028 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:54.026+0000 7fbdf2dc0740 1 --2- 172.21.15.47:0/3810541108 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f214be5260 0x55f214bed1b0 secure :-1 s=READY pgs=1265 cs=0 l=1 rev1=1 crypto rx=0x7fbddc010420 tx=0x7fbddc00e5c0 comp rx=0 tx=0).stop 2024-03-21T10:34:54.028 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:54.026+0000 7fbdf2dc0740 1 -- 172.21.15.47:0/3810541108 shutdown_connections 2024-03-21T10:34:54.028 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:54.026+0000 7fbdf2dc0740 1 --2- 172.21.15.47:0/3810541108 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f214bedde0 0x55f214bed6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:54.028 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:54.026+0000 7fbdf2dc0740 1 --2- 172.21.15.47:0/3810541108 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f214be5260 0x55f214bed1b0 unknown :-1 s=CLOSED pgs=1265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:54.028 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:54.026+0000 7fbdf2dc0740 1 --2- 172.21.15.47:0/3810541108 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f214be48b0 0x55f214bda060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:54.028 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:54.026+0000 7fbdf2dc0740 1 -- 172.21.15.47:0/3810541108 >> 172.21.15.47:0/3810541108 conn(0x55f214bdef80 msgr2=0x55f214be0c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:34:54.028 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:54.026+0000 7fbdf2dc0740 1 -- 172.21.15.47:0/3810541108 shutdown_connections 2024-03-21T10:34:54.028 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:54.026+0000 7fbdf2dc0740 1 -- 172.21.15.47:0/3810541108 wait complete. 2024-03-21T10:34:54.028 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:54.027+0000 7fbdf2dc0740 1 -- start start 2024-03-21T10:34:54.028 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:54.027+0000 7fbdf2dc0740 1 Processor -- start 2024-03-21T10:34:54.028 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:54.027+0000 7fbdf2dc0740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f214be48b0 0x55f214bf5c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:34:54.028 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:54.027+0000 7fbdf2dc0740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f214be5260 0x55f214bf4020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:34:54.029 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:54.027+0000 7fbdf2dc0740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f214bedde0 0x55f214bf4560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:34:54.029 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:54.027+0000 7fbdf2dc0740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbde81b1d90 con 0x55f214be48b0 2024-03-21T10:34:54.029 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:54.027+0000 7fbdf2dc0740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbde81b1730 con 0x55f214be5260 2024-03-21T10:34:54.029 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:54.027+0000 7fbdf2dc0740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbde81aec50 con 0x55f214bedde0 2024-03-21T10:34:54.029 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:54.027+0000 7fbdf2dc0740 10 client.0 _ll_register_callbacks cb 0x7fbddc009530 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T10:34:54.029 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:54.028+0000 7fbdecc00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f214be48b0 0x55f214bf5c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:34:54.029 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:54.028+0000 7fbdecc00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f214be48b0 0x55f214bf5c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:55752/0 (socket says 172.21.15.47:55752) 2024-03-21T10:34:54.029 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:54.028+0000 7fbdecc00640 1 -- 172.21.15.47:0/622154985 learned_addr learned my addr 172.21.15.47:0/622154985 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:34:54.029 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:54.028+0000 7fbded600640 1 --2- 172.21.15.47:0/622154985 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f214be5260 0x55f214bf4020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:34:54.029 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:54.028+0000 7fbdecc00640 1 -- 172.21.15.47:0/622154985 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f214bedde0 msgr2=0x55f214bf4560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:34:54.029 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:54.028+0000 7fbdecc00640 1 --2- 172.21.15.47:0/622154985 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f214bedde0 0x55f214bf4560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:54.029 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:54.028+0000 7fbdecc00640 1 -- 172.21.15.47:0/622154985 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f214be5260 msgr2=0x55f214bf4020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:34:54.029 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:54.028+0000 7fbdecc00640 1 --2- 172.21.15.47:0/622154985 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f214be5260 0x55f214bf4020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:54.029 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:54.028+0000 7fbdecc00640 1 -- 172.21.15.47:0/622154985 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55f214bf4bd0 con 0x55f214be48b0 2024-03-21T10:34:54.029 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:54.029+0000 7fbdecc00640 1 --2- 172.21.15.47:0/622154985 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f214be48b0 0x55f214bf5c80 secure :-1 s=READY pgs=1266 cs=0 l=1 rev1=1 crypto rx=0x7fbdd4009930 tx=0x7fbdd4005a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2e205c2be485db55 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:34:54.030 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:54.029+0000 7fbdcea00640 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-03-21T10:34:54.030 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:54.029+0000 7fbdcea00640 1 -- 172.21.15.47:0/622154985 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbdd4056070 con 0x55f214be48b0 2024-03-21T10:34:54.030 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:54.029+0000 7fbdcea00640 1 -- 172.21.15.47:0/622154985 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(103..103 src has 1..103) v4 ==== 9562+0+0 (secure 0 0 0) 0x7fbdd4004140 con 0x55f214be48b0 2024-03-21T10:34:54.030 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:54.029+0000 7fbdf2dc0740 1 -- 172.21.15.47:0/622154985 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fbddc010a80 con 0x55f214be48b0 2024-03-21T10:34:54.030 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:54.029+0000 7fbdcea00640 1 -- 172.21.15.47:0/622154985 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7fbdd405f050 con 0x55f214be48b0 2024-03-21T10:34:54.030 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:54.029+0000 7fbdf2dc0740 10 client.18636 fetch_fsmap learned FSMap version 32 2024-03-21T10:34:54.030 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:54.029+0000 7fbdf2dc0740 1 -- 172.21.15.47:0/622154985 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x55f214c5a2e0 con 0x55f214be48b0 2024-03-21T10:34:54.030 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:54.030+0000 7fbdcea00640 1 -- 172.21.15.47:0/622154985 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fbddc010a80 con 0x55f214be48b0 2024-03-21T10:34:54.032 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:54.030+0000 7fbdcea00640 1 -- 172.21.15.47:0/622154985 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 32) v1 ==== 7942+0+0 (secure 0 0 0) 0x7fbdd4059020 con 0x55f214be48b0 2024-03-21T10:34:54.032 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:54.030+0000 7fbdf2dc0740 10 client.18636 fetch_fsmap finished waiting for FSMap version 32 2024-03-21T10:34:54.032 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:54.030+0000 7fbdf2dc0740 10 client.18636 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.e 2024-03-21T10:34:54.032 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:54.030+0000 7fbdf2dc0740 20 client.18636 populate_metadata read hostname 'smithi047' 2024-03-21T10:34:54.032 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:54.030+0000 7fbdf2dc0740 1 --2- 172.21.15.47:0/622154985 >> [v2:172.21.15.100:6834/2597849957,v1:172.21.15.100:6835/2597849957] conn(0x55f214c5a620 0x55f214c5ca10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:34:54.032 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:54.030+0000 7fbdf2dc0740 4 client.18636 mds_command: new command op to 34454 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-21T10:34:54.032 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:54.030+0000 7fbdf2dc0740 1 -- 172.21.15.47:0/622154985 --> [v2:172.21.15.100:6834/2597849957,v1:172.21.15.100:6835/2597849957] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55f214b4fb50 con 0x55f214c5a620 2024-03-21T10:34:54.032 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:54.030+0000 7fbded600640 1 --2- 172.21.15.47:0/622154985 >> [v2:172.21.15.100:6834/2597849957,v1:172.21.15.100:6835/2597849957] conn(0x55f214c5a620 0x55f214c5ca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:34:54.032 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:54.031+0000 7fbded600640 1 --2- 172.21.15.47:0/622154985 >> [v2:172.21.15.100:6834/2597849957,v1:172.21.15.100:6835/2597849957] conn(0x55f214c5a620 0x55f214c5ca10 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=e0a1e408c67ffc82 server_cookie=fa9e73a486fe6e42 in_seq=0 out_seq=0 2024-03-21T10:34:54.033 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:54.031+0000 7fbdcea00640 10 client.18636 ms_handle_connect on v2:172.21.15.100:6834/2597849957 2024-03-21T10:34:54.033 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:54.032+0000 7fbdcea00640 1 -- 172.21.15.47:0/622154985 <== mds.2 v2:172.21.15.100:6834/2597849957 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55f214b4fb50 con 0x55f214c5a620 2024-03-21T10:34:54.033 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:54.032+0000 7fbdcea00640 10 client.18636 handle_command_reply: tid=0 2024-03-21T10:34:54.033 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:54.032+0000 7fbdf2dc0740 1 client.18636 shutdown 2024-03-21T10:34:54.033 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:54.032+0000 7fbdf2dc0740 1 -- 172.21.15.47:0/622154985 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f214be48b0 msgr2=0x55f214bf5c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:34:54.033 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:54.032+0000 7fbdf2dc0740 1 --2- 172.21.15.47:0/622154985 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f214be48b0 0x55f214bf5c80 secure :-1 s=READY pgs=1266 cs=0 l=1 rev1=1 crypto rx=0x7fbdd4009930 tx=0x7fbdd4005a20 comp rx=0 tx=0).stop 2024-03-21T10:34:54.033 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:54.032+0000 7fbdf2dc0740 1 -- 172.21.15.47:0/622154985 shutdown_connections 2024-03-21T10:34:54.033 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:54.032+0000 7fbdf2dc0740 1 --2- 172.21.15.47:0/622154985 >> [v2:172.21.15.100:6834/2597849957,v1:172.21.15.100:6835/2597849957] conn(0x55f214c5a620 0x55f214c5ca10 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:54.033 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:54.032+0000 7fbdf2dc0740 1 --2- 172.21.15.47:0/622154985 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f214bedde0 0x55f214bf4560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:54.033 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:54.032+0000 7fbdf2dc0740 1 --2- 172.21.15.47:0/622154985 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f214be5260 0x55f214bf4020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:54.033 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:54.032+0000 7fbdcea00640 0 client.18636 ms_handle_reset on v2:172.21.15.100:6834/2597849957 2024-03-21T10:34:54.033 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:54.032+0000 7fbdf2dc0740 1 --2- 172.21.15.47:0/622154985 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f214be48b0 0x55f214bf5c80 unknown :-1 s=CLOSED pgs=1266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:54.033 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:54.032+0000 7fbdf2dc0740 1 -- 172.21.15.47:0/622154985 >> 172.21.15.47:0/622154985 conn(0x55f214bdef80 msgr2=0x55f214be0340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:34:54.033 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:54.032+0000 7fbdf2dc0740 1 -- 172.21.15.47:0/622154985 shutdown_connections 2024-03-21T10:34:54.033 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:54.032+0000 7fbdf2dc0740 1 -- 172.21.15.47:0/622154985 wait complete. 2024-03-21T10:34:54.034 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:54.032+0000 7fbdf2dc0740 20 client.18636 trim_cache size 0 max 16384 2024-03-21T10:34:54.034 INFO:teuthology.orchestra.run.smithi047.stdout:[] 2024-03-21T10:34:54.035 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:54.033+0000 7fbdeea00640 1 -- 172.21.15.47:0/2332535704 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7fbdd00a3090 msgr2=0x7fbdd00a5550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:34:54.035 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:54.033+0000 7fbdeea00640 1 --2- 172.21.15.47:0/2332535704 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7fbdd00a3090 0x7fbdd00a5550 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fbde0009270 tx=0x7fbde00513d0 comp rx=0 tx=0).stop 2024-03-21T10:34:54.035 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:54.033+0000 7fbdeea00640 1 -- 172.21.15.47:0/2332535704 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbde8109730 msgr2=0x7fbde819e880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:34:54.035 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:54.033+0000 7fbdeea00640 1 --2- 172.21.15.47:0/2332535704 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbde8109730 0x7fbde819e880 secure :-1 s=READY pgs=1262 cs=0 l=1 rev1=1 crypto rx=0x7fbde000eaa0 tx=0x7fbde000ef60 comp rx=0 tx=0).stop 2024-03-21T10:34:54.035 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:54.033+0000 7fbdeea00640 1 -- 172.21.15.47:0/2332535704 shutdown_connections 2024-03-21T10:34:54.035 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:54.033+0000 7fbdeea00640 1 --2- 172.21.15.47:0/2332535704 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbde8109730 0x7fbde819e880 unknown :-1 s=CLOSED pgs=1262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:54.035 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:54.033+0000 7fbdeea00640 1 --2- 172.21.15.47:0/2332535704 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7fbdd00a3090 0x7fbdd00a5550 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:54.035 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:54.033+0000 7fbdeea00640 1 --2- 172.21.15.47:0/2332535704 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbde81040e0 0x7fbde819a500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:54.035 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:54.033+0000 7fbdeea00640 1 --2- 172.21.15.47:0/2332535704 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbde8103720 0x7fbde8199fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:54.035 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:54.033+0000 7fbdeea00640 1 -- 172.21.15.47:0/2332535704 >> 172.21.15.47:0/2332535704 conn(0x7fbde80ff1d0 msgr2=0x7fbde80ffa20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:34:54.035 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:54.033+0000 7fbdeea00640 1 -- 172.21.15.47:0/2332535704 shutdown_connections 2024-03-21T10:34:54.035 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:54.033+0000 7fbdeea00640 1 -- 172.21.15.47:0/2332535704 wait complete. 2024-03-21T10:34:54.049 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-21T10:34:54.049 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-03-21T10:34:54.270 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:54.272+0000 7f19f1600640 1 -- 172.21.15.47:0/2290208621 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f19ec101530 msgr2=0x7f19ec1019b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:34:54.271 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:54.272+0000 7f19f1600640 1 --2- 172.21.15.47:0/2290208621 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f19ec101530 0x7f19ec1019b0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f19d4009930 tx=0x7f19d402bc30 comp rx=0 tx=0).stop 2024-03-21T10:34:54.271 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:54.272+0000 7f19f1600640 1 -- 172.21.15.47:0/2290208621 shutdown_connections 2024-03-21T10:34:54.271 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:54.272+0000 7f19f1600640 1 --2- 172.21.15.47:0/2290208621 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f19ec101ef0 0x7f19ec10e3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:54.271 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:54.272+0000 7f19f1600640 1 --2- 172.21.15.47:0/2290208621 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f19ec101530 0x7f19ec1019b0 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:54.271 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:54.272+0000 7f19f1600640 1 --2- 172.21.15.47:0/2290208621 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f19ec1075f0 0x7f19ec1079d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:54.271 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:54.272+0000 7f19f1600640 1 -- 172.21.15.47:0/2290208621 >> 172.21.15.47:0/2290208621 conn(0x7f19ec0fd000 msgr2=0x7f19ec0ff420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:34:54.271 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:54.272+0000 7f19f1600640 1 -- 172.21.15.47:0/2290208621 shutdown_connections 2024-03-21T10:34:54.271 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:54.272+0000 7f19f1600640 1 -- 172.21.15.47:0/2290208621 wait complete. 2024-03-21T10:34:54.273 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:54.273+0000 7f19f1600640 1 Processor -- start 2024-03-21T10:34:54.273 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:54.273+0000 7f19f1600640 1 -- start start 2024-03-21T10:34:54.273 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:54.274+0000 7f19f1600640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f19ec101530 0x7f19ec199fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:34:54.273 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:54.274+0000 7f19f1600640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f19ec101ef0 0x7f19ec19a4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:34:54.273 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:54.274+0000 7f19f1600640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f19ec1075f0 0x7f19ec19e870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:34:54.273 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:54.274+0000 7f19f1600640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19ec1117f0 con 0x7f19ec101530 2024-03-21T10:34:54.273 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:54.274+0000 7f19f1600640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19ec111670 con 0x7f19ec1075f0 2024-03-21T10:34:54.273 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:54.274+0000 7f19f1600640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19ec111970 con 0x7f19ec101ef0 2024-03-21T10:34:54.273 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:54.274+0000 7f19ebe00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f19ec101530 0x7f19ec199fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:34:54.273 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:54.274+0000 7f19ebe00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f19ec101530 0x7f19ec199fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:55762/0 (socket says 172.21.15.47:55762) 2024-03-21T10:34:54.273 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:54.274+0000 7f19ebe00640 1 -- 172.21.15.47:0/1353919523 learned_addr learned my addr 172.21.15.47:0/1353919523 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:34:54.273 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:54.274+0000 7f19f0c00640 1 --2- 172.21.15.47:0/1353919523 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f19ec1075f0 0x7f19ec19e870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:34:54.273 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:54.274+0000 7f19eb400640 1 --2- 172.21.15.47:0/1353919523 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f19ec101ef0 0x7f19ec19a4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:34:54.273 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:54.275+0000 7f19ebe00640 1 -- 172.21.15.47:0/1353919523 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f19ec101ef0 msgr2=0x7f19ec19a4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:34:54.274 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:54.275+0000 7f19ebe00640 1 --2- 172.21.15.47:0/1353919523 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f19ec101ef0 0x7f19ec19a4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:54.274 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:54.275+0000 7f19ebe00640 1 -- 172.21.15.47:0/1353919523 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f19ec1075f0 msgr2=0x7f19ec19e870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:34:54.274 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:54.275+0000 7f19ebe00640 1 --2- 172.21.15.47:0/1353919523 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f19ec1075f0 0x7f19ec19e870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:54.274 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:54.275+0000 7f19ebe00640 1 -- 172.21.15.47:0/1353919523 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f19ec19efa0 con 0x7f19ec101530 2024-03-21T10:34:54.274 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:54.275+0000 7f19ebe00640 1 --2- 172.21.15.47:0/1353919523 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f19ec101530 0x7f19ec199fb0 secure :-1 s=READY pgs=1267 cs=0 l=1 rev1=1 crypto rx=0x7f19dc00d770 tx=0x7f19dc00dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:34:54.275 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:54.275+0000 7f19e8c00640 1 -- 172.21.15.47:0/1353919523 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19dc017070 con 0x7f19ec101530 2024-03-21T10:34:54.275 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:54.275+0000 7f19e8c00640 1 -- 172.21.15.47:0/1353919523 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f19dc0054d0 con 0x7f19ec101530 2024-03-21T10:34:54.275 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:54.275+0000 7f19f1600640 1 -- 172.21.15.47:0/1353919523 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f19ec19f290 con 0x7f19ec101530 2024-03-21T10:34:54.275 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:54.275+0000 7f19e8c00640 1 -- 172.21.15.47:0/1353919523 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19dc004cf0 con 0x7f19ec101530 2024-03-21T10:34:54.275 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:54.276+0000 7f19f1600640 1 -- 172.21.15.47:0/1353919523 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f19ec1a6b20 con 0x7f19ec101530 2024-03-21T10:34:54.277 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:54.279+0000 7f19e8c00640 1 -- 172.21.15.47:0/1353919523 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 27) v1 ==== 137985+0+0 (secure 0 0 0) 0x7f19dc004e90 con 0x7f19ec101530 2024-03-21T10:34:54.277 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:54.279+0000 7f19f597b740 1 Processor -- start 2024-03-21T10:34:54.277 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:54.279+0000 7f19f597b740 1 -- start start 2024-03-21T10:34:54.279 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:54.279+0000 7f19f597b740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55cd5374a8b0 0x55cd5374ac90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:34:54.279 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:54.279+0000 7f19f597b740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55cd5374b260 0x55cd53753810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:34:54.279 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:54.279+0000 7f19f597b740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55cd53753de0 0x55cd53756230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:34:54.279 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:54.280+0000 7f19f597b740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cd536ae2c0 con 0x55cd5374a8b0 2024-03-21T10:34:54.279 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:54.280+0000 7f19f597b740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cd53656c90 con 0x55cd5374b260 2024-03-21T10:34:54.279 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:54.280+0000 7f19f597b740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cd536aa360 con 0x55cd53753de0 2024-03-21T10:34:54.279 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:54.280+0000 7f19f0c00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55cd5374a8b0 0x55cd5374ac90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:34:54.280 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:54.280+0000 7f19f0c00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55cd5374a8b0 0x55cd5374ac90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:55778/0 (socket says 172.21.15.47:55778) 2024-03-21T10:34:54.280 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:54.280+0000 7f19f0c00640 1 -- 172.21.15.47:0/518347556 learned_addr learned my addr 172.21.15.47:0/518347556 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:34:54.280 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:54.280+0000 7f19ebe00640 1 --2- 172.21.15.47:0/518347556 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55cd5374b260 0x55cd53753810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:34:54.280 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:54.280+0000 7f19eb400640 1 --2- 172.21.15.47:0/518347556 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55cd53753de0 0x55cd53756230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:34:54.280 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:54.280+0000 7f19f0c00640 1 -- 172.21.15.47:0/518347556 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55cd53753de0 msgr2=0x55cd53756230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:34:54.280 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:54.280+0000 7f19f0c00640 1 --2- 172.21.15.47:0/518347556 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55cd53753de0 0x55cd53756230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:54.280 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:54.280+0000 7f19f0c00640 1 -- 172.21.15.47:0/518347556 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55cd5374b260 msgr2=0x55cd53753810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:34:54.280 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:54.280+0000 7f19f0c00640 1 --2- 172.21.15.47:0/518347556 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55cd5374b260 0x55cd53753810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:54.280 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:54.280+0000 7f19f0c00640 1 -- 172.21.15.47:0/518347556 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55cd5367b080 con 0x55cd5374a8b0 2024-03-21T10:34:54.280 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:54.280+0000 7f19ebe00640 1 --2- 172.21.15.47:0/518347556 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55cd5374b260 0x55cd53753810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T10:34:54.280 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:54.281+0000 7f19e8c00640 1 --2- 172.21.15.47:0/1353919523 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f19d00a3110 0x7f19d00a55d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:34:54.280 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:54.281+0000 7f19f0c00640 1 --2- 172.21.15.47:0/518347556 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55cd5374a8b0 0x55cd5374ac90 secure :-1 s=READY pgs=1268 cs=0 l=1 rev1=1 crypto rx=0x7f19e4009fd0 tx=0x7f19e40118d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6bbd1631e705b736 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:34:54.280 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:54.281+0000 7f19e8c00640 1 -- 172.21.15.47:0/1353919523 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 9562+0+0 (secure 0 0 0) 0x7f19dc0cf550 con 0x7f19ec101530 2024-03-21T10:34:54.280 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:54.281+0000 7f19ebe00640 1 --2- 172.21.15.47:0/1353919523 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f19d00a3110 0x7f19d00a55d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:34:54.281 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:54.281+0000 7f19e1200640 1 -- 172.21.15.47:0/518347556 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19e4004d20 con 0x55cd5374a8b0 2024-03-21T10:34:54.283 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:54.281+0000 7f19e1200640 1 -- 172.21.15.47:0/518347556 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f19e4004050 con 0x55cd5374a8b0 2024-03-21T10:34:54.283 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:54.281+0000 7f19e1200640 1 -- 172.21.15.47:0/518347556 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19e4005280 con 0x55cd5374a8b0 2024-03-21T10:34:54.283 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:54.282+0000 7f19f597b740 1 -- 172.21.15.47:0/518347556 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55cd5374a8b0 msgr2=0x55cd5374ac90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:34:54.283 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:54.282+0000 7f19ebe00640 1 --2- 172.21.15.47:0/1353919523 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f19d00a3110 0x7f19d00a55d0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f19dc0cc940 tx=0x7f19dc0d5110 comp rx=0 tx=0).ready entity=mgr.15687 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:34:54.284 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:54.282+0000 7f19f597b740 1 --2- 172.21.15.47:0/518347556 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55cd5374a8b0 0x55cd5374ac90 secure :-1 s=READY pgs=1268 cs=0 l=1 rev1=1 crypto rx=0x7f19e4009fd0 tx=0x7f19e40118d0 comp rx=0 tx=0).stop 2024-03-21T10:34:54.284 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:54.282+0000 7f19f597b740 1 -- 172.21.15.47:0/518347556 shutdown_connections 2024-03-21T10:34:54.284 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:54.282+0000 7f19f597b740 1 --2- 172.21.15.47:0/518347556 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55cd53753de0 0x55cd53756230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:54.284 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:54.282+0000 7f19f597b740 1 --2- 172.21.15.47:0/518347556 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55cd5374b260 0x55cd53753810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:54.284 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:54.282+0000 7f19f597b740 1 --2- 172.21.15.47:0/518347556 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55cd5374a8b0 0x55cd5374ac90 unknown :-1 s=CLOSED pgs=1268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:54.284 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:54.282+0000 7f19f597b740 1 -- 172.21.15.47:0/518347556 >> 172.21.15.47:0/518347556 conn(0x55cd53744f80 msgr2=0x55cd53745360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:34:54.284 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:54.282+0000 7f19f597b740 1 -- 172.21.15.47:0/518347556 shutdown_connections 2024-03-21T10:34:54.284 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:54.282+0000 7f19f597b740 1 -- 172.21.15.47:0/518347556 wait complete. 2024-03-21T10:34:54.284 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:54.283+0000 7f19f597b740 1 -- start start 2024-03-21T10:34:54.284 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:54.283+0000 7f19f597b740 1 Processor -- start 2024-03-21T10:34:54.284 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:54.284+0000 7f19f597b740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55cd5374a8b0 0x55cd537536c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:34:54.284 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:54.284+0000 7f19f597b740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55cd5374b260 0x55cd53750a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:34:54.284 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:54.284+0000 7f19f597b740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55cd53753de0 0x55cd53750fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:34:54.284 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:54.284+0000 7f19f597b740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cd53756d50 con 0x55cd53753de0 2024-03-21T10:34:54.285 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:54.284+0000 7f19f597b740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cd5366a0d0 con 0x55cd5374b260 2024-03-21T10:34:54.285 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:54.284+0000 7f19f597b740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cd53756ed0 con 0x55cd5374a8b0 2024-03-21T10:34:54.285 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:54.284+0000 7f19f597b740 10 client.0 _ll_register_callbacks cb 0x55cd5366ee80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T10:34:54.285 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:54.284+0000 7f19f0c00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55cd53753de0 0x55cd53750fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:34:54.285 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:54.284+0000 7f19eb400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55cd5374b260 0x55cd53750a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:34:54.285 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:54.284+0000 7f19f0c00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55cd53753de0 0x55cd53750fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:55780/0 (socket says 172.21.15.47:55780) 2024-03-21T10:34:54.285 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:54.284+0000 7f19f0c00640 1 -- 172.21.15.47:0/2801999527 learned_addr learned my addr 172.21.15.47:0/2801999527 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:34:54.285 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:54.284+0000 7f19f0c00640 1 --2- 172.21.15.47:0/2801999527 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55cd5374a8b0 0x55cd537536c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:34:54.285 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:54.285+0000 7f19eb400640 1 -- 172.21.15.47:0/2801999527 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55cd5374a8b0 msgr2=0x55cd537536c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:34:54.285 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:54.285+0000 7f19eb400640 1 --2- 172.21.15.47:0/2801999527 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55cd5374a8b0 0x55cd537536c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:54.285 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:54.285+0000 7f19eb400640 1 -- 172.21.15.47:0/2801999527 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55cd53753de0 msgr2=0x55cd53750fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:34:54.285 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:54.285+0000 7f19eb400640 1 --2- 172.21.15.47:0/2801999527 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55cd53753de0 0x55cd53750fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:54.285 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:54.285+0000 7f19eb400640 1 -- 172.21.15.47:0/2801999527 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55cd533c23f0 con 0x55cd5374b260 2024-03-21T10:34:54.285 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:54.285+0000 7f19f0c00640 1 --2- 172.21.15.47:0/2801999527 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55cd53753de0 0x55cd53750fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T10:34:54.286 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:54.285+0000 7f19eb400640 1 --2- 172.21.15.47:0/2801999527 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55cd5374b260 0x55cd53750a50 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f19d4036040 tx=0x7f19d402ca10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d700e4251b044eb9 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:34:54.286 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:54.285+0000 7f19d9600640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-21T10:34:54.286 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:54.286+0000 7f19d9600640 1 -- 172.21.15.47:0/2801999527 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19d4044070 con 0x55cd5374b260 2024-03-21T10:34:54.286 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:54.286+0000 7f19f597b740 1 -- 172.21.15.47:0/2801999527 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f19e4004050 con 0x55cd5374b260 2024-03-21T10:34:54.286 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:54.286+0000 7f19d9600640 1 -- 172.21.15.47:0/2801999527 <== mon.2 v2:172.21.15.100:3300/0 2 ==== osd_map(103..103 src has 1..103) v4 ==== 9562+0+0 (secure 0 0 0) 0x7f19d4005220 con 0x55cd5374b260 2024-03-21T10:34:54.288 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:54.286+0000 7f19d9600640 1 -- 172.21.15.47:0/2801999527 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f19d404d050 con 0x55cd5374b260 2024-03-21T10:34:54.288 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:54.286+0000 7f19f597b740 10 client.38405 fetch_fsmap learned FSMap version 32 2024-03-21T10:34:54.288 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:54.286+0000 7f19f597b740 1 -- 172.21.15.47:0/2801999527 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x55cd5348c7d0 con 0x55cd5374b260 2024-03-21T10:34:54.288 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:54.286+0000 7f19d9600640 1 -- 172.21.15.47:0/2801999527 <== mon.2 v2:172.21.15.100:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f19e4004050 con 0x55cd5374b260 2024-03-21T10:34:54.288 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:54.287+0000 7f19d9600640 1 -- 172.21.15.47:0/2801999527 <== mon.2 v2:172.21.15.100:3300/0 5 ==== fsmap(e 32) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f19d4047020 con 0x55cd5374b260 2024-03-21T10:34:54.288 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:54.287+0000 7f19f597b740 10 client.38405 fetch_fsmap finished waiting for FSMap version 32 2024-03-21T10:34:54.288 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:54.287+0000 7f19f597b740 10 client.38405 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-03-21T10:34:54.288 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:54.287+0000 7f19f597b740 20 client.38405 populate_metadata read hostname 'smithi047' 2024-03-21T10:34:54.289 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:54.287+0000 7f19f597b740 1 --2- 172.21.15.47:0/2801999527 >> [v2:172.21.15.100:6836/3497644162,v1:172.21.15.100:6837/3497644162] conn(0x55cd53763c70 0x55cd53766090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:34:54.289 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:54.287+0000 7f19f597b740 4 client.38405 mds_command: new command op to 34493 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T10:34:54.289 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:54.287+0000 7f19f597b740 1 -- 172.21.15.47:0/2801999527 --> [v2:172.21.15.100:6836/3497644162,v1:172.21.15.100:6837/3497644162] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55cd536bb800 con 0x55cd53763c70 2024-03-21T10:34:54.289 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:54.287+0000 7f19f0c00640 1 --2- 172.21.15.47:0/2801999527 >> [v2:172.21.15.100:6836/3497644162,v1:172.21.15.100:6837/3497644162] conn(0x55cd53763c70 0x55cd53766090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:34:54.289 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:54.288+0000 7f19f0c00640 1 --2- 172.21.15.47:0/2801999527 >> [v2:172.21.15.100:6836/3497644162,v1:172.21.15.100:6837/3497644162] conn(0x55cd53763c70 0x55cd53766090 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=6995aee61dc5f508 server_cookie=7c4038da47aa065b in_seq=0 out_seq=0 2024-03-21T10:34:54.289 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:54.288+0000 7f19d9600640 10 client.38405 ms_handle_connect on v2:172.21.15.100:6836/3497644162 2024-03-21T10:34:54.289 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:54.289+0000 7f19d9600640 1 -- 172.21.15.47:0/2801999527 <== mds.1 v2:172.21.15.100:6836/3497644162 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55cd536bb800 con 0x55cd53763c70 2024-03-21T10:34:54.289 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:54.289+0000 7f19d9600640 10 client.38405 handle_command_reply: tid=0 2024-03-21T10:34:54.289 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:54.289+0000 7f19f597b740 1 client.38405 shutdown 2024-03-21T10:34:54.289 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:54.290+0000 7f19f597b740 1 -- 172.21.15.47:0/2801999527 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55cd5374b260 msgr2=0x55cd53750a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:34:54.289 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:54.290+0000 7f19f597b740 1 --2- 172.21.15.47:0/2801999527 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55cd5374b260 0x55cd53750a50 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f19d4036040 tx=0x7f19d402ca10 comp rx=0 tx=0).stop 2024-03-21T10:34:54.289 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:54.290+0000 7f19f597b740 1 -- 172.21.15.47:0/2801999527 shutdown_connections 2024-03-21T10:34:54.289 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:54.290+0000 7f19f597b740 1 --2- 172.21.15.47:0/2801999527 >> [v2:172.21.15.100:6836/3497644162,v1:172.21.15.100:6837/3497644162] conn(0x55cd53763c70 0x55cd53766090 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:54.289 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:54.290+0000 7f19f597b740 1 --2- 172.21.15.47:0/2801999527 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55cd53753de0 0x55cd53750fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:54.290 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:54.290+0000 7f19f597b740 1 --2- 172.21.15.47:0/2801999527 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55cd5374b260 0x55cd53750a50 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:54.290 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:54.290+0000 7f19f597b740 1 --2- 172.21.15.47:0/2801999527 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55cd5374a8b0 0x55cd537536c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:54.290 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:54.290+0000 7f19d9600640 0 client.38405 ms_handle_reset on v2:172.21.15.100:6836/3497644162 2024-03-21T10:34:54.290 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:54.290+0000 7f19f597b740 1 -- 172.21.15.47:0/2801999527 >> 172.21.15.47:0/2801999527 conn(0x55cd53744f80 msgr2=0x55cd53746cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:34:54.290 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:54.290+0000 7f19f597b740 1 -- 172.21.15.47:0/2801999527 shutdown_connections 2024-03-21T10:34:54.290 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:54.290+0000 7f19f597b740 1 -- 172.21.15.47:0/2801999527 wait complete. 2024-03-21T10:34:54.290 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:54.290+0000 7f19f597b740 20 client.38405 trim_cache size 0 max 16384 2024-03-21T10:34:54.302 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:54.304+0000 7f19f597b740 1 Processor -- start 2024-03-21T10:34:54.302 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:54.304+0000 7f19f597b740 1 -- start start 2024-03-21T10:34:54.302 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:54.304+0000 7f19f597b740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55cd5374a8b0 0x55cd53744160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:34:54.302 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:54.304+0000 7f19f597b740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55cd5374b260 0x55cd53753180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:34:54.302 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:54.304+0000 7f19f597b740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55cd53753de0 0x55cd537536c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:34:54.303 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:54.304+0000 7f19f597b740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19ec1b0980 con 0x55cd5374a8b0 2024-03-21T10:34:54.303 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:54.304+0000 7f19f597b740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19ec1b2300 con 0x55cd5374b260 2024-03-21T10:34:54.303 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:54.304+0000 7f19f597b740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19ec1b0fe0 con 0x55cd53753de0 2024-03-21T10:34:54.303 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:54.304+0000 7f19f0c00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55cd5374a8b0 0x55cd53744160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:34:54.303 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:54.304+0000 7f19f0c00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55cd5374a8b0 0x55cd53744160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:55788/0 (socket says 172.21.15.47:55788) 2024-03-21T10:34:54.303 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:54.304+0000 7f19eb400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55cd5374b260 0x55cd53753180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:34:54.303 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:54.304+0000 7f19f0c00640 1 -- 172.21.15.47:0/353175994 learned_addr learned my addr 172.21.15.47:0/353175994 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:34:54.304 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:54.305+0000 7f19f0c00640 1 -- 172.21.15.47:0/353175994 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55cd53753de0 msgr2=0x55cd537536c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:34:54.304 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:54.305+0000 7f19f0c00640 1 --2- 172.21.15.47:0/353175994 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55cd53753de0 0x55cd537536c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:54.305 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:54.305+0000 7f19f0c00640 1 -- 172.21.15.47:0/353175994 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55cd5374b260 msgr2=0x55cd53753180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:34:54.305 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:54.305+0000 7f19f0c00640 1 --2- 172.21.15.47:0/353175994 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55cd5374b260 0x55cd53753180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:54.305 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:54.305+0000 7f19f0c00640 1 -- 172.21.15.47:0/353175994 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55cd53751320 con 0x55cd5374a8b0 2024-03-21T10:34:54.305 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:54.305+0000 7f19f0c00640 1 --2- 172.21.15.47:0/353175994 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55cd5374a8b0 0x55cd53744160 secure :-1 s=READY pgs=1269 cs=0 l=1 rev1=1 crypto rx=0x7f19e4013010 tx=0x7f19e4004870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=52740860b5d52891 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:34:54.305 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:54.305+0000 7f19dbe00640 1 -- 172.21.15.47:0/353175994 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19e40049a0 con 0x55cd5374a8b0 2024-03-21T10:34:54.305 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:54.305+0000 7f19dbe00640 1 -- 172.21.15.47:0/353175994 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f19e402e030 con 0x55cd5374a8b0 2024-03-21T10:34:54.305 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:54.305+0000 7f19dbe00640 1 -- 172.21.15.47:0/353175994 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19e4005040 con 0x55cd5374a8b0 2024-03-21T10:34:54.305 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:54.306+0000 7f19f597b740 1 -- 172.21.15.47:0/353175994 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55cd5374a8b0 msgr2=0x55cd53744160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:34:54.305 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:54.306+0000 7f19f597b740 1 --2- 172.21.15.47:0/353175994 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55cd5374a8b0 0x55cd53744160 secure :-1 s=READY pgs=1269 cs=0 l=1 rev1=1 crypto rx=0x7f19e4013010 tx=0x7f19e4004870 comp rx=0 tx=0).stop 2024-03-21T10:34:54.305 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:54.306+0000 7f19f597b740 1 -- 172.21.15.47:0/353175994 shutdown_connections 2024-03-21T10:34:54.305 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:54.306+0000 7f19f597b740 1 --2- 172.21.15.47:0/353175994 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55cd53753de0 0x55cd537536c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:54.305 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:54.306+0000 7f19f597b740 1 --2- 172.21.15.47:0/353175994 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55cd5374b260 0x55cd53753180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:54.305 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:54.306+0000 7f19f597b740 1 --2- 172.21.15.47:0/353175994 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55cd5374a8b0 0x55cd53744160 unknown :-1 s=CLOSED pgs=1269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:54.305 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:54.306+0000 7f19f597b740 1 -- 172.21.15.47:0/353175994 >> 172.21.15.47:0/353175994 conn(0x55cd53744f80 msgr2=0x55cd53746280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:34:54.306 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:54.306+0000 7f19f597b740 1 -- 172.21.15.47:0/353175994 shutdown_connections 2024-03-21T10:34:54.306 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:54.306+0000 7f19f597b740 1 -- 172.21.15.47:0/353175994 wait complete. 2024-03-21T10:34:54.306 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:54.307+0000 7f19f597b740 1 -- start start 2024-03-21T10:34:54.306 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:54.307+0000 7f19f597b740 1 Processor -- start 2024-03-21T10:34:54.308 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:54.307+0000 7f19f597b740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55cd5374a8b0 0x55cd5375b950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:34:54.308 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:54.307+0000 7f19f597b740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55cd5374b260 0x55cd5375be90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:34:54.308 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:54.307+0000 7f19f597b740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55cd53753de0 0x55cd53759e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:34:54.308 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:54.307+0000 7f19f597b740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19ec1b1ca0 con 0x55cd5374b260 2024-03-21T10:34:54.309 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:54.307+0000 7f19f597b740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19ec1b1640 con 0x55cd5374a8b0 2024-03-21T10:34:54.309 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:54.307+0000 7f19f597b740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19ec1aeb60 con 0x55cd53753de0 2024-03-21T10:34:54.309 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:54.307+0000 7f19f597b740 10 client.0 _ll_register_callbacks cb 0x7f19e40093a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T10:34:54.309 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:54.307+0000 7f19eb400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55cd5374b260 0x55cd5375be90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:34:54.309 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:54.308+0000 7f19f0c00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55cd5374a8b0 0x55cd5375b950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:34:54.309 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:54.308+0000 7f19eb400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55cd5374b260 0x55cd5375be90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:55802/0 (socket says 172.21.15.47:55802) 2024-03-21T10:34:54.309 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:54.308+0000 7f19eb400640 1 -- 172.21.15.47:0/438707701 learned_addr learned my addr 172.21.15.47:0/438707701 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:34:54.309 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:54.308+0000 7f19f0c00640 1 -- 172.21.15.47:0/438707701 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55cd53753de0 msgr2=0x55cd53759e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:34:54.309 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:54.308+0000 7f19f0c00640 1 --2- 172.21.15.47:0/438707701 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55cd53753de0 0x55cd53759e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:54.309 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:54.308+0000 7f19f0c00640 1 -- 172.21.15.47:0/438707701 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55cd5374b260 msgr2=0x55cd5375be90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:34:54.309 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:54.308+0000 7f19f0c00640 1 --2- 172.21.15.47:0/438707701 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55cd5374b260 0x55cd5375be90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:54.309 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:54.308+0000 7f19f0c00640 1 -- 172.21.15.47:0/438707701 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55cd5375a650 con 0x55cd5374a8b0 2024-03-21T10:34:54.309 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:54.308+0000 7f19eb400640 1 --2- 172.21.15.47:0/438707701 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55cd5374b260 0x55cd5375be90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T10:34:54.310 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:54.308+0000 7f19f0c00640 1 --2- 172.21.15.47:0/438707701 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55cd5374a8b0 0x55cd5375b950 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f19e402e000 tx=0x7f19e40118d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4273a2846198d79 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:34:54.310 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:54.308+0000 7f19d9600640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-21T10:34:54.310 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:54.309+0000 7f19d9600640 1 -- 172.21.15.47:0/438707701 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19e4004050 con 0x55cd5374a8b0 2024-03-21T10:34:54.310 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:54.309+0000 7f19f597b740 1 -- 172.21.15.47:0/438707701 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f19e402e030 con 0x55cd5374a8b0 2024-03-21T10:34:54.310 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:54.309+0000 7f19d9600640 1 -- 172.21.15.47:0/438707701 <== mon.2 v2:172.21.15.100:3300/0 2 ==== osd_map(103..103 src has 1..103) v4 ==== 9562+0+0 (secure 0 0 0) 0x7f19e4004450 con 0x55cd5374a8b0 2024-03-21T10:34:54.310 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:54.309+0000 7f19d9600640 1 -- 172.21.15.47:0/438707701 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f19e40049a0 con 0x55cd5374a8b0 2024-03-21T10:34:54.310 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:54.309+0000 7f19f597b740 10 client.38411 fetch_fsmap learned FSMap version 32 2024-03-21T10:34:54.310 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:54.309+0000 7f19f597b740 1 -- 172.21.15.47:0/438707701 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x55cd537c00b0 con 0x55cd5374a8b0 2024-03-21T10:34:54.310 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:54.310+0000 7f19d9600640 1 -- 172.21.15.47:0/438707701 <== mon.2 v2:172.21.15.100:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f19e402e030 con 0x55cd5374a8b0 2024-03-21T10:34:54.310 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:54.310+0000 7f19d9600640 1 -- 172.21.15.47:0/438707701 <== mon.2 v2:172.21.15.100:3300/0 5 ==== fsmap(e 32) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f19e40056b0 con 0x55cd5374a8b0 2024-03-21T10:34:54.310 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:54.310+0000 7f19f597b740 10 client.38411 fetch_fsmap finished waiting for FSMap version 32 2024-03-21T10:34:54.310 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:54.310+0000 7f19f597b740 10 client.38411 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-03-21T10:34:54.310 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:54.310+0000 7f19f597b740 20 client.38411 populate_metadata read hostname 'smithi047' 2024-03-21T10:34:54.310 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:54.310+0000 7f19f597b740 1 --2- 172.21.15.47:0/438707701 >> [v2:172.21.15.100:6836/3497644162,v1:172.21.15.100:6837/3497644162] conn(0x55cd537c04e0 0x55cd537c2900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:34:54.311 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:54.310+0000 7f19f597b740 4 client.38411 mds_command: new command op to 34493 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-21T10:34:54.311 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:54.310+0000 7f19f597b740 1 -- 172.21.15.47:0/438707701 --> [v2:172.21.15.100:6836/3497644162,v1:172.21.15.100:6837/3497644162] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55cd536b5b50 con 0x55cd537c04e0 2024-03-21T10:34:54.311 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:54.310+0000 7f19eb400640 1 --2- 172.21.15.47:0/438707701 >> [v2:172.21.15.100:6836/3497644162,v1:172.21.15.100:6837/3497644162] conn(0x55cd537c04e0 0x55cd537c2900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:34:54.312 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:54.311+0000 7f19eb400640 1 --2- 172.21.15.47:0/438707701 >> [v2:172.21.15.100:6836/3497644162,v1:172.21.15.100:6837/3497644162] conn(0x55cd537c04e0 0x55cd537c2900 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=b10a815a0d3d34ce server_cookie=dc82258ed6da7b03 in_seq=0 out_seq=0 2024-03-21T10:34:54.312 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:54.311+0000 7f19d9600640 10 client.38411 ms_handle_connect on v2:172.21.15.100:6836/3497644162 2024-03-21T10:34:54.312 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:54.311+0000 7f19d9600640 1 -- 172.21.15.47:0/438707701 <== mds.1 v2:172.21.15.100:6836/3497644162 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55cd536b5b50 con 0x55cd537c04e0 2024-03-21T10:34:54.312 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:54.311+0000 7f19d9600640 10 client.38411 handle_command_reply: tid=0 2024-03-21T10:34:54.312 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:54.311+0000 7f19f597b740 1 client.38411 shutdown 2024-03-21T10:34:54.312 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:54.311+0000 7f19f597b740 1 -- 172.21.15.47:0/438707701 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55cd5374a8b0 msgr2=0x55cd5375b950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:34:54.312 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:54.311+0000 7f19f597b740 1 --2- 172.21.15.47:0/438707701 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55cd5374a8b0 0x55cd5375b950 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f19e402e000 tx=0x7f19e40118d0 comp rx=0 tx=0).stop 2024-03-21T10:34:54.313 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:54.311+0000 7f19f597b740 1 -- 172.21.15.47:0/438707701 shutdown_connections 2024-03-21T10:34:54.313 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:54.311+0000 7f19f597b740 1 --2- 172.21.15.47:0/438707701 >> [v2:172.21.15.100:6836/3497644162,v1:172.21.15.100:6837/3497644162] conn(0x55cd537c04e0 0x55cd537c2900 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:54.313 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:54.311+0000 7f19f597b740 1 --2- 172.21.15.47:0/438707701 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55cd53753de0 0x55cd53759e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:54.313 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:54.311+0000 7f19f597b740 1 --2- 172.21.15.47:0/438707701 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55cd5374b260 0x55cd5375be90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:54.313 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:54.311+0000 7f19f597b740 1 --2- 172.21.15.47:0/438707701 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55cd5374a8b0 0x55cd5375b950 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:54.313 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:54.311+0000 7f19f597b740 1 -- 172.21.15.47:0/438707701 >> 172.21.15.47:0/438707701 conn(0x55cd53744f80 msgr2=0x55cd5374bed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:34:54.313 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:54.311+0000 7f19d9600640 0 client.38411 ms_handle_reset on v2:172.21.15.100:6836/3497644162 2024-03-21T10:34:54.313 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:54.312+0000 7f19f597b740 1 -- 172.21.15.47:0/438707701 shutdown_connections 2024-03-21T10:34:54.313 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:54.312+0000 7f19f597b740 1 -- 172.21.15.47:0/438707701 wait complete. 2024-03-21T10:34:54.313 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:54.312+0000 7f19f597b740 20 client.38411 trim_cache size 0 max 16384 2024-03-21T10:34:54.313 INFO:teuthology.orchestra.run.smithi047.stdout:[] 2024-03-21T10:34:54.314 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:54.312+0000 7f19f1600640 1 -- 172.21.15.47:0/1353919523 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f19d00a3110 msgr2=0x7f19d00a55d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:34:54.314 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:54.312+0000 7f19f1600640 1 --2- 172.21.15.47:0/1353919523 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f19d00a3110 0x7f19d00a55d0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f19dc0cc940 tx=0x7f19dc0d5110 comp rx=0 tx=0).stop 2024-03-21T10:34:54.314 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:54.312+0000 7f19f1600640 1 -- 172.21.15.47:0/1353919523 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f19ec101530 msgr2=0x7f19ec199fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:34:54.315 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:54.312+0000 7f19f1600640 1 --2- 172.21.15.47:0/1353919523 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f19ec101530 0x7f19ec199fb0 secure :-1 s=READY pgs=1267 cs=0 l=1 rev1=1 crypto rx=0x7f19dc00d770 tx=0x7f19dc00dc30 comp rx=0 tx=0).stop 2024-03-21T10:34:54.315 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:54.312+0000 7f19f1600640 1 -- 172.21.15.47:0/1353919523 shutdown_connections 2024-03-21T10:34:54.315 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:54.312+0000 7f19f1600640 1 --2- 172.21.15.47:0/1353919523 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f19d00a3110 0x7f19d00a55d0 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:54.315 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:54.312+0000 7f19f1600640 1 --2- 172.21.15.47:0/1353919523 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f19ec1075f0 0x7f19ec19e870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:54.315 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:54.312+0000 7f19f1600640 1 --2- 172.21.15.47:0/1353919523 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f19ec101ef0 0x7f19ec19a4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:54.315 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:54.312+0000 7f19f1600640 1 --2- 172.21.15.47:0/1353919523 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f19ec101530 0x7f19ec199fb0 unknown :-1 s=CLOSED pgs=1267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:54.315 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:54.312+0000 7f19f1600640 1 -- 172.21.15.47:0/1353919523 >> 172.21.15.47:0/1353919523 conn(0x7f19ec0fd000 msgr2=0x7f19ec0fe4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:34:54.315 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:54.313+0000 7f19f1600640 1 -- 172.21.15.47:0/1353919523 shutdown_connections 2024-03-21T10:34:54.315 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:54.313+0000 7f19f1600640 1 -- 172.21.15.47:0/1353919523 wait complete. 2024-03-21T10:34:54.330 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-21T10:34:54.330 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-03-21T10:34:54.534 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:54.536+0000 7fc0f5e00640 1 -- 172.21.15.47:0/2852084067 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc0f01040e0 msgr2=0x7fc0f01105f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:34:54.535 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:54.536+0000 7fc0f5e00640 1 --2- 172.21.15.47:0/2852084067 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc0f01040e0 0x7fc0f01105f0 secure :-1 s=READY pgs=1270 cs=0 l=1 rev1=1 crypto rx=0x7fc0e8009cd0 tx=0x7fc0e802c0f0 comp rx=0 tx=0).stop 2024-03-21T10:34:54.535 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:54.536+0000 7fc0f5e00640 1 -- 172.21.15.47:0/2852084067 shutdown_connections 2024-03-21T10:34:54.535 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:54.536+0000 7fc0f5e00640 1 --2- 172.21.15.47:0/2852084067 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc0f01040e0 0x7fc0f01105f0 unknown :-1 s=CLOSED pgs=1270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:54.535 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:54.536+0000 7fc0f5e00640 1 --2- 172.21.15.47:0/2852084067 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc0f0103720 0x7fc0f0103ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:54.535 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:54.536+0000 7fc0f5e00640 1 --2- 172.21.15.47:0/2852084067 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc0f0109730 0x7fc0f0109b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:54.535 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:54.536+0000 7fc0f5e00640 1 -- 172.21.15.47:0/2852084067 >> 172.21.15.47:0/2852084067 conn(0x7fc0f00ff1d0 msgr2=0x7fc0f01015f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:34:54.535 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:54.536+0000 7fc0f5e00640 1 -- 172.21.15.47:0/2852084067 shutdown_connections 2024-03-21T10:34:54.535 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:54.536+0000 7fc0f5e00640 1 -- 172.21.15.47:0/2852084067 wait complete. 2024-03-21T10:34:54.537 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:54.537+0000 7fc0f5e00640 1 Processor -- start 2024-03-21T10:34:54.537 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:54.537+0000 7fc0f5e00640 1 -- start start 2024-03-21T10:34:54.537 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:54.538+0000 7fc0f5e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc0f0103720 0x7fc0f0199f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:34:54.537 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:54.538+0000 7fc0f5e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc0f01040e0 0x7fc0f019a4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:34:54.537 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:54.538+0000 7fc0f5e00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc0f0109730 0x7fc0f019e7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:34:54.537 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:54.538+0000 7fc0f5e00640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0f01139f0 con 0x7fc0f01040e0 2024-03-21T10:34:54.537 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:54.538+0000 7fc0f5e00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0f0113870 con 0x7fc0f0103720 2024-03-21T10:34:54.537 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:54.538+0000 7fc0f5e00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0f0113b70 con 0x7fc0f0109730 2024-03-21T10:34:54.537 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:54.538+0000 7fc0efe00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc0f01040e0 0x7fc0f019a4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:34:54.537 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:54.538+0000 7fc0efe00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc0f01040e0 0x7fc0f019a4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:55816/0 (socket says 172.21.15.47:55816) 2024-03-21T10:34:54.538 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:54.538+0000 7fc0efe00640 1 -- 172.21.15.47:0/2387540531 learned_addr learned my addr 172.21.15.47:0/2387540531 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:34:54.538 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:54.538+0000 7fc0f4a00640 1 --2- 172.21.15.47:0/2387540531 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc0f0103720 0x7fc0f0199f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:34:54.538 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:54.538+0000 7fc0f5400640 1 --2- 172.21.15.47:0/2387540531 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc0f0109730 0x7fc0f019e7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:34:54.538 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:54.539+0000 7fc0efe00640 1 -- 172.21.15.47:0/2387540531 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc0f0109730 msgr2=0x7fc0f019e7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:34:54.538 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:54.539+0000 7fc0efe00640 1 --2- 172.21.15.47:0/2387540531 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc0f0109730 0x7fc0f019e7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:54.538 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:54.539+0000 7fc0efe00640 1 -- 172.21.15.47:0/2387540531 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc0f0103720 msgr2=0x7fc0f0199f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:34:54.539 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:54.539+0000 7fc0efe00640 1 --2- 172.21.15.47:0/2387540531 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc0f0103720 0x7fc0f0199f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:54.539 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:54.539+0000 7fc0efe00640 1 -- 172.21.15.47:0/2387540531 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc0f019ef10 con 0x7fc0f01040e0 2024-03-21T10:34:54.539 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:54.539+0000 7fc0efe00640 1 --2- 172.21.15.47:0/2387540531 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc0f01040e0 0x7fc0f019a4b0 secure :-1 s=READY pgs=1271 cs=0 l=1 rev1=1 crypto rx=0x7fc0dc00e9f0 tx=0x7fc0dc00edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:34:54.540 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:54.539+0000 7fc0ed600640 1 -- 172.21.15.47:0/2387540531 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0dc00cd80 con 0x7fc0f01040e0 2024-03-21T10:34:54.540 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:54.539+0000 7fc0ed600640 1 -- 172.21.15.47:0/2387540531 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fc0dc0054d0 con 0x7fc0f01040e0 2024-03-21T10:34:54.540 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:54.539+0000 7fc0f5e00640 1 -- 172.21.15.47:0/2387540531 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc0f019f200 con 0x7fc0f01040e0 2024-03-21T10:34:54.540 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:54.539+0000 7fc0ed600640 1 -- 172.21.15.47:0/2387540531 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0dc010490 con 0x7fc0f01040e0 2024-03-21T10:34:54.540 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:54.539+0000 7fc0f5e00640 1 -- 172.21.15.47:0/2387540531 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc0f01a6a90 con 0x7fc0f01040e0 2024-03-21T10:34:54.542 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:54.544+0000 7fc0fa1d4740 1 Processor -- start 2024-03-21T10:34:54.542 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:54.544+0000 7fc0fa1d4740 1 -- start start 2024-03-21T10:34:54.543 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:54.544+0000 7fc0fa1d4740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5623e0ad88b0 0x5623e0ad8c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:34:54.544 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:54.544+0000 7fc0fa1d4740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5623e0ad9260 0x5623e0ae1810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:34:54.544 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:54.544+0000 7fc0fa1d4740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5623e0ae1de0 0x5623e0ae4230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:34:54.544 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:54.544+0000 7fc0fa1d4740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x5623e0a3c2c0 con 0x5623e0ad88b0 2024-03-21T10:34:54.544 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:54.544+0000 7fc0fa1d4740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x5623e09e4c90 con 0x5623e0ae1de0 2024-03-21T10:34:54.544 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:54.544+0000 7fc0fa1d4740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x5623e0a38360 con 0x5623e0ad9260 2024-03-21T10:34:54.544 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:54.545+0000 7fc0ed600640 1 -- 172.21.15.47:0/2387540531 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 27) v1 ==== 137985+0+0 (secure 0 0 0) 0x7fc0dc005020 con 0x7fc0f01040e0 2024-03-21T10:34:54.544 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:54.545+0000 7fc0f5400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5623e0ad88b0 0x5623e0ad8c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:34:54.544 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:54.545+0000 7fc0f5400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5623e0ad88b0 0x5623e0ad8c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:55820/0 (socket says 172.21.15.47:55820) 2024-03-21T10:34:54.544 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:54.545+0000 7fc0f5400640 1 -- 172.21.15.47:0/260639621 learned_addr learned my addr 172.21.15.47:0/260639621 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:34:54.544 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:54.545+0000 7fc0efe00640 1 --2- 172.21.15.47:0/260639621 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5623e0ae1de0 0x5623e0ae4230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:34:54.544 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:54.545+0000 7fc0f4a00640 1 --2- 172.21.15.47:0/260639621 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5623e0ad9260 0x5623e0ae1810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:34:54.544 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:54.545+0000 7fc0f5400640 1 -- 172.21.15.47:0/260639621 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5623e0ad9260 msgr2=0x5623e0ae1810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:34:54.545 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:54.545+0000 7fc0f5400640 1 --2- 172.21.15.47:0/260639621 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5623e0ad9260 0x5623e0ae1810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:54.545 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:54.545+0000 7fc0f5400640 1 -- 172.21.15.47:0/260639621 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5623e0ae1de0 msgr2=0x5623e0ae4230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:34:54.545 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:54.545+0000 7fc0f5400640 1 --2- 172.21.15.47:0/260639621 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5623e0ae1de0 0x5623e0ae4230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:54.545 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:54.545+0000 7fc0f5400640 1 -- 172.21.15.47:0/260639621 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5623e0a09080 con 0x5623e0ad88b0 2024-03-21T10:34:54.545 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:54.545+0000 7fc0f4a00640 1 --2- 172.21.15.47:0/260639621 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5623e0ad9260 0x5623e0ae1810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T10:34:54.545 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:54.545+0000 7fc0efe00640 1 --2- 172.21.15.47:0/260639621 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5623e0ae1de0 0x5623e0ae4230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T10:34:54.545 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:54.546+0000 7fc0f5400640 1 --2- 172.21.15.47:0/260639621 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5623e0ad88b0 0x5623e0ad8c90 secure :-1 s=READY pgs=1272 cs=0 l=1 rev1=1 crypto rx=0x7fc0e8009ca0 tx=0x7fc0e8032340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fc1d371d53a9dbc5 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:34:54.548 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:54.546+0000 7fc0e1c00640 1 -- 172.21.15.47:0/260639621 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0e8043020 con 0x5623e0ad88b0 2024-03-21T10:34:54.549 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:54.546+0000 7fc0e1c00640 1 -- 172.21.15.47:0/260639621 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fc0e8032bb0 con 0x5623e0ad88b0 2024-03-21T10:34:54.549 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:54.546+0000 7fc0e1c00640 1 -- 172.21.15.47:0/260639621 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0e8039040 con 0x5623e0ad88b0 2024-03-21T10:34:54.549 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:54.546+0000 7fc0ed600640 1 --2- 172.21.15.47:0/2387540531 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7fc0d80a3110 0x7fc0d80a55d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:34:54.549 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:54.547+0000 7fc0ed600640 1 -- 172.21.15.47:0/2387540531 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 9562+0+0 (secure 0 0 0) 0x7fc0dc0cfff0 con 0x7fc0f01040e0 2024-03-21T10:34:54.549 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:54.547+0000 7fc0f4a00640 1 --2- 172.21.15.47:0/2387540531 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7fc0d80a3110 0x7fc0d80a55d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:34:54.549 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:54.547+0000 7fc0fa1d4740 1 -- 172.21.15.47:0/260639621 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5623e0ad88b0 msgr2=0x5623e0ad8c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:34:54.549 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:54.547+0000 7fc0fa1d4740 1 --2- 172.21.15.47:0/260639621 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5623e0ad88b0 0x5623e0ad8c90 secure :-1 s=READY pgs=1272 cs=0 l=1 rev1=1 crypto rx=0x7fc0e8009ca0 tx=0x7fc0e8032340 comp rx=0 tx=0).stop 2024-03-21T10:34:54.549 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:54.547+0000 7fc0fa1d4740 1 -- 172.21.15.47:0/260639621 shutdown_connections 2024-03-21T10:34:54.549 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:54.547+0000 7fc0fa1d4740 1 --2- 172.21.15.47:0/260639621 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5623e0ae1de0 0x5623e0ae4230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:54.549 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:54.547+0000 7fc0fa1d4740 1 --2- 172.21.15.47:0/260639621 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5623e0ad9260 0x5623e0ae1810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:54.549 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:54.547+0000 7fc0fa1d4740 1 --2- 172.21.15.47:0/260639621 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5623e0ad88b0 0x5623e0ad8c90 unknown :-1 s=CLOSED pgs=1272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:54.549 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:54.547+0000 7fc0fa1d4740 1 -- 172.21.15.47:0/260639621 >> 172.21.15.47:0/260639621 conn(0x5623e0ad2f80 msgr2=0x5623e0ad3360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:34:54.549 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:54.548+0000 7fc0fa1d4740 1 -- 172.21.15.47:0/260639621 shutdown_connections 2024-03-21T10:34:54.550 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:54.548+0000 7fc0fa1d4740 1 -- 172.21.15.47:0/260639621 wait complete. 2024-03-21T10:34:54.550 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:54.548+0000 7fc0f4a00640 1 --2- 172.21.15.47:0/2387540531 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7fc0d80a3110 0x7fc0d80a55d0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x5623e0ad3ff0 tx=0x7fc0e4005f90 comp rx=0 tx=0).ready entity=mgr.15687 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:34:54.550 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:54.548+0000 7fc0fa1d4740 1 -- start start 2024-03-21T10:34:54.550 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:54.548+0000 7fc0fa1d4740 1 Processor -- start 2024-03-21T10:34:54.550 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:54.549+0000 7fc0fa1d4740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5623e0ad88b0 0x5623e0ae1670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:34:54.550 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:54.549+0000 7fc0fa1d4740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5623e0ad9260 0x5623e0adea00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:34:54.550 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:54.549+0000 7fc0fa1d4740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5623e0ae1de0 0x5623e0adef40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:34:54.550 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:54.549+0000 7fc0fa1d4740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x5623e0ae4cf0 con 0x5623e0ad9260 2024-03-21T10:34:54.550 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:54.549+0000 7fc0fa1d4740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x5623e09f80d0 con 0x5623e0ae1de0 2024-03-21T10:34:54.550 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:54.549+0000 7fc0fa1d4740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x5623e0ae4e70 con 0x5623e0ad88b0 2024-03-21T10:34:54.550 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:54.549+0000 7fc0efe00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5623e0ad9260 0x5623e0adea00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:34:54.550 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:54.549+0000 7fc0fa1d4740 10 client.0 _ll_register_callbacks cb 0x5623e09fce80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T10:34:54.550 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:54.549+0000 7fc0efe00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5623e0ad9260 0x5623e0adea00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:55824/0 (socket says 172.21.15.47:55824) 2024-03-21T10:34:54.550 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:54.549+0000 7fc0efe00640 1 -- 172.21.15.47:0/3529864800 learned_addr learned my addr 172.21.15.47:0/3529864800 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:34:54.551 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:54.549+0000 7fc0f5400640 1 --2- 172.21.15.47:0/3529864800 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5623e0ae1de0 0x5623e0adef40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:34:54.551 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:54.549+0000 7fc0f4a00640 1 --2- 172.21.15.47:0/3529864800 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5623e0ad88b0 0x5623e0ae1670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:34:54.551 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:54.550+0000 7fc0efe00640 1 -- 172.21.15.47:0/3529864800 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5623e0ad88b0 msgr2=0x5623e0ae1670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:34:54.551 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:54.550+0000 7fc0efe00640 1 --2- 172.21.15.47:0/3529864800 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5623e0ad88b0 0x5623e0ae1670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:54.551 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:54.550+0000 7fc0efe00640 1 -- 172.21.15.47:0/3529864800 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5623e0ae1de0 msgr2=0x5623e0adef40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:34:54.551 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:54.550+0000 7fc0efe00640 1 --2- 172.21.15.47:0/3529864800 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5623e0ae1de0 0x5623e0adef40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:54.551 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:54.550+0000 7fc0efe00640 1 -- 172.21.15.47:0/3529864800 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5623e07503f0 con 0x5623e0ad9260 2024-03-21T10:34:54.551 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:54.550+0000 7fc0efe00640 1 --2- 172.21.15.47:0/3529864800 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5623e0ad9260 0x5623e0adea00 secure :-1 s=READY pgs=1273 cs=0 l=1 rev1=1 crypto rx=0x7fc0dc0a4450 tx=0x7fc0dc09b990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dfb142b35e629854 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:34:54.551 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:54.550+0000 7fc0d6000640 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-03-21T10:34:54.551 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:54.550+0000 7fc0d6000640 1 -- 172.21.15.47:0/3529864800 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0dc0b5060 con 0x5623e0ad9260 2024-03-21T10:34:54.551 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:54.550+0000 7fc0d6000640 1 -- 172.21.15.47:0/3529864800 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(103..103 src has 1..103) v4 ==== 9562+0+0 (secure 0 0 0) 0x7fc0dc0cd550 con 0x5623e0ad9260 2024-03-21T10:34:54.551 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:54.550+0000 7fc0fa1d4740 1 -- 172.21.15.47:0/3529864800 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fc0e8032bb0 con 0x5623e0ad9260 2024-03-21T10:34:54.551 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:54.550+0000 7fc0d6000640 1 -- 172.21.15.47:0/3529864800 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc0dc0cd750 con 0x5623e0ad9260 2024-03-21T10:34:54.551 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:54.550+0000 7fc0fa1d4740 10 client.18666 fetch_fsmap learned FSMap version 32 2024-03-21T10:34:54.552 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:54.550+0000 7fc0fa1d4740 1 -- 172.21.15.47:0/3529864800 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x5623e081a7d0 con 0x5623e0ad9260 2024-03-21T10:34:54.552 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:54.551+0000 7fc0d6000640 1 -- 172.21.15.47:0/3529864800 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fc0e8032bb0 con 0x5623e0ad9260 2024-03-21T10:34:54.553 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:54.551+0000 7fc0d6000640 1 -- 172.21.15.47:0/3529864800 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 32) v1 ==== 7942+0+0 (secure 0 0 0) 0x7fc0dc0b7030 con 0x5623e0ad9260 2024-03-21T10:34:54.553 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:54.551+0000 7fc0fa1d4740 10 client.18666 fetch_fsmap finished waiting for FSMap version 32 2024-03-21T10:34:54.553 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:54.551+0000 7fc0fa1d4740 10 client.18666 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-21T10:34:54.553 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:54.551+0000 7fc0fa1d4740 20 client.18666 populate_metadata read hostname 'smithi047' 2024-03-21T10:34:54.553 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:54.551+0000 7fc0fa1d4740 1 --2- 172.21.15.47:0/3529864800 >> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] conn(0x5623e0af1b70 0x5623e0af3f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:34:54.553 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:54.551+0000 7fc0fa1d4740 4 client.18666 mds_command: new command op to 34535 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T10:34:54.553 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:54.551+0000 7fc0fa1d4740 1 -- 172.21.15.47:0/3529864800 --> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5623e0a49800 con 0x5623e0af1b70 2024-03-21T10:34:54.554 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:54.552+0000 7fc0f4a00640 1 --2- 172.21.15.47:0/3529864800 >> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] conn(0x5623e0af1b70 0x5623e0af3f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:34:54.554 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:54.552+0000 7fc0f4a00640 1 --2- 172.21.15.47:0/3529864800 >> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] conn(0x5623e0af1b70 0x5623e0af3f60 crc :-1 s=READY pgs=215 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6225a767a38e0756 server_cookie=663b3a6ddc0ed3 in_seq=0 out_seq=0 2024-03-21T10:34:54.554 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:54.552+0000 7fc0d6000640 10 client.18666 ms_handle_connect on v2:172.21.15.100:6838/4173168440 2024-03-21T10:34:54.554 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:54.554+0000 7fc0d6000640 1 -- 172.21.15.47:0/3529864800 <== mds.0 v2:172.21.15.100:6838/4173168440 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x5623e0a49800 con 0x5623e0af1b70 2024-03-21T10:34:54.554 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:54.554+0000 7fc0d6000640 10 client.18666 handle_command_reply: tid=0 2024-03-21T10:34:54.554 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:54.554+0000 7fc0fa1d4740 1 client.18666 shutdown 2024-03-21T10:34:54.554 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:54.554+0000 7fc0fa1d4740 1 -- 172.21.15.47:0/3529864800 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5623e0ad9260 msgr2=0x5623e0adea00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:34:54.554 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:54.554+0000 7fc0fa1d4740 1 --2- 172.21.15.47:0/3529864800 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5623e0ad9260 0x5623e0adea00 secure :-1 s=READY pgs=1273 cs=0 l=1 rev1=1 crypto rx=0x7fc0dc0a4450 tx=0x7fc0dc09b990 comp rx=0 tx=0).stop 2024-03-21T10:34:54.554 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:54.554+0000 7fc0fa1d4740 1 -- 172.21.15.47:0/3529864800 shutdown_connections 2024-03-21T10:34:54.554 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:54.554+0000 7fc0fa1d4740 1 --2- 172.21.15.47:0/3529864800 >> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] conn(0x5623e0af1b70 0x5623e0af3f60 crc :-1 s=READY pgs=215 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:54.554 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:54.554+0000 7fc0fa1d4740 1 --2- 172.21.15.47:0/3529864800 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5623e0ae1de0 0x5623e0adef40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:54.554 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:54.554+0000 7fc0fa1d4740 1 --2- 172.21.15.47:0/3529864800 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5623e0ad9260 0x5623e0adea00 unknown :-1 s=CLOSED pgs=1273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:54.555 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:54.554+0000 7fc0fa1d4740 1 --2- 172.21.15.47:0/3529864800 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5623e0ad88b0 0x5623e0ae1670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:54.555 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:54.554+0000 7fc0d6000640 0 client.18666 ms_handle_reset on v2:172.21.15.100:6838/4173168440 2024-03-21T10:34:54.555 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:54.554+0000 7fc0fa1d4740 1 -- 172.21.15.47:0/3529864800 >> 172.21.15.47:0/3529864800 conn(0x5623e0ad2f80 msgr2=0x5623e0ad4cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:34:54.555 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:54.554+0000 7fc0fa1d4740 1 -- 172.21.15.47:0/3529864800 shutdown_connections 2024-03-21T10:34:54.555 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:54.554+0000 7fc0fa1d4740 1 -- 172.21.15.47:0/3529864800 wait complete. 2024-03-21T10:34:54.555 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:54.554+0000 7fc0fa1d4740 20 client.18666 trim_cache size 0 max 16384 2024-03-21T10:34:54.567 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:54.568+0000 7fc0fa1d4740 1 Processor -- start 2024-03-21T10:34:54.567 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:54.568+0000 7fc0fa1d4740 1 -- start start 2024-03-21T10:34:54.567 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:54.569+0000 7fc0fa1d4740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5623e0ad88b0 0x5623e0ae1130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:34:54.567 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:54.569+0000 7fc0fa1d4740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5623e0ad9260 0x5623e0ae1670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:34:54.567 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:54.569+0000 7fc0fa1d4740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5623e0ae1de0 0x5623e0adbc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:34:54.567 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:54.569+0000 7fc0fa1d4740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0f01b0950 con 0x5623e0ad88b0 2024-03-21T10:34:54.567 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:54.569+0000 7fc0fa1d4740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0f01b22d0 con 0x5623e0ae1de0 2024-03-21T10:34:54.567 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:54.569+0000 7fc0fa1d4740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0f01b0fb0 con 0x5623e0ad9260 2024-03-21T10:34:54.567 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:54.569+0000 7fc0f4a00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5623e0ad88b0 0x5623e0ae1130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:34:54.568 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:54.569+0000 7fc0f4a00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5623e0ad88b0 0x5623e0ae1130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:55840/0 (socket says 172.21.15.47:55840) 2024-03-21T10:34:54.568 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:54.569+0000 7fc0f4a00640 1 -- 172.21.15.47:0/948420389 learned_addr learned my addr 172.21.15.47:0/948420389 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:34:54.568 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:54.569+0000 7fc0f5400640 1 --2- 172.21.15.47:0/948420389 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5623e0ae1de0 0x5623e0adbc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:34:54.569 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:54.569+0000 7fc0efe00640 1 --2- 172.21.15.47:0/948420389 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5623e0ad9260 0x5623e0ae1670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:34:54.569 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:54.569+0000 7fc0f4a00640 1 -- 172.21.15.47:0/948420389 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5623e0ad9260 msgr2=0x5623e0ae1670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:34:54.570 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:54.569+0000 7fc0f4a00640 1 --2- 172.21.15.47:0/948420389 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5623e0ad9260 0x5623e0ae1670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:54.570 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:54.569+0000 7fc0f4a00640 1 -- 172.21.15.47:0/948420389 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5623e0ae1de0 msgr2=0x5623e0adbc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:34:54.570 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:54.569+0000 7fc0f4a00640 1 --2- 172.21.15.47:0/948420389 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5623e0ae1de0 0x5623e0adbc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:54.570 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:54.569+0000 7fc0f4a00640 1 -- 172.21.15.47:0/948420389 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5623e0ace6c0 con 0x5623e0ad88b0 2024-03-21T10:34:54.570 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:54.569+0000 7fc0f5400640 1 --2- 172.21.15.47:0/948420389 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5623e0ae1de0 0x5623e0adbc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T10:34:54.570 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:54.569+0000 7fc0efe00640 1 --2- 172.21.15.47:0/948420389 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5623e0ad9260 0x5623e0ae1670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T10:34:54.570 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:54.570+0000 7fc0f4a00640 1 --2- 172.21.15.47:0/948420389 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5623e0ad88b0 0x5623e0ae1130 secure :-1 s=READY pgs=1274 cs=0 l=1 rev1=1 crypto rx=0x5623e0af4780 tx=0x7fc0e4016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d7f48cee10d1f4cd server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:34:54.570 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:54.570+0000 7fc0e1200640 1 -- 172.21.15.47:0/948420389 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0e4005020 con 0x5623e0ad88b0 2024-03-21T10:34:54.570 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:54.570+0000 7fc0e1200640 1 -- 172.21.15.47:0/948420389 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fc0e4005670 con 0x5623e0ad88b0 2024-03-21T10:34:54.570 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:54.570+0000 7fc0e1200640 1 -- 172.21.15.47:0/948420389 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0e4004e60 con 0x5623e0ad88b0 2024-03-21T10:34:54.570 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:54.571+0000 7fc0fa1d4740 1 -- 172.21.15.47:0/948420389 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5623e0ad88b0 msgr2=0x5623e0ae1130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:34:54.570 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:54.571+0000 7fc0fa1d4740 1 --2- 172.21.15.47:0/948420389 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5623e0ad88b0 0x5623e0ae1130 secure :-1 s=READY pgs=1274 cs=0 l=1 rev1=1 crypto rx=0x5623e0af4780 tx=0x7fc0e4016000 comp rx=0 tx=0).stop 2024-03-21T10:34:54.570 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:54.571+0000 7fc0fa1d4740 1 -- 172.21.15.47:0/948420389 shutdown_connections 2024-03-21T10:34:54.571 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:54.571+0000 7fc0fa1d4740 1 --2- 172.21.15.47:0/948420389 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5623e0ae1de0 0x5623e0adbc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:54.571 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:54.571+0000 7fc0fa1d4740 1 --2- 172.21.15.47:0/948420389 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5623e0ad9260 0x5623e0ae1670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:54.571 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:54.571+0000 7fc0fa1d4740 1 --2- 172.21.15.47:0/948420389 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5623e0ad88b0 0x5623e0ae1130 unknown :-1 s=CLOSED pgs=1274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:54.571 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:54.571+0000 7fc0fa1d4740 1 -- 172.21.15.47:0/948420389 >> 172.21.15.47:0/948420389 conn(0x5623e0ad2f80 msgr2=0x5623e0ad2160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:34:54.571 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:54.571+0000 7fc0fa1d4740 1 -- 172.21.15.47:0/948420389 shutdown_connections 2024-03-21T10:34:54.571 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:54.571+0000 7fc0fa1d4740 1 -- 172.21.15.47:0/948420389 wait complete. 2024-03-21T10:34:54.571 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:54.571+0000 7fc0fa1d4740 1 -- start start 2024-03-21T10:34:54.571 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:54.572+0000 7fc0fa1d4740 1 Processor -- start 2024-03-21T10:34:54.575 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:54.572+0000 7fc0fa1d4740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5623e0ad88b0 0x5623e0ae5c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:34:54.575 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:54.572+0000 7fc0fa1d4740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5623e0ad9260 0x5623e0ae6160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:34:54.575 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:54.572+0000 7fc0fa1d4740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5623e0ae1de0 0x5623e0b4c3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:34:54.575 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:54.572+0000 7fc0fa1d4740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0f01b1c70 con 0x5623e0ae1de0 2024-03-21T10:34:54.575 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:54.572+0000 7fc0fa1d4740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0f01b1610 con 0x5623e0ad88b0 2024-03-21T10:34:54.575 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:54.572+0000 7fc0fa1d4740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0f01aeb30 con 0x5623e0ad9260 2024-03-21T10:34:54.576 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:54.572+0000 7fc0fa1d4740 10 client.0 _ll_register_callbacks cb 0x7fc0dc00cf20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T10:34:54.576 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:54.572+0000 7fc0f5400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5623e0ae1de0 0x5623e0b4c3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:34:54.576 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:54.572+0000 7fc0f5400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5623e0ae1de0 0x5623e0b4c3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:55850/0 (socket says 172.21.15.47:55850) 2024-03-21T10:34:54.576 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:54.572+0000 7fc0f5400640 1 -- 172.21.15.47:0/1779527656 learned_addr learned my addr 172.21.15.47:0/1779527656 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:34:54.576 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:54.573+0000 7fc0f4a00640 1 --2- 172.21.15.47:0/1779527656 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5623e0ad88b0 0x5623e0ae5c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:34:54.576 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:54.573+0000 7fc0efe00640 1 --2- 172.21.15.47:0/1779527656 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5623e0ad9260 0x5623e0ae6160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:34:54.576 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:54.573+0000 7fc0f5400640 1 -- 172.21.15.47:0/1779527656 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5623e0ad9260 msgr2=0x5623e0ae6160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:34:54.576 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:54.573+0000 7fc0f5400640 1 --2- 172.21.15.47:0/1779527656 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5623e0ad9260 0x5623e0ae6160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:54.576 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:54.573+0000 7fc0f5400640 1 -- 172.21.15.47:0/1779527656 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5623e0ad88b0 msgr2=0x5623e0ae5c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:34:54.576 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:54.573+0000 7fc0f5400640 1 --2- 172.21.15.47:0/1779527656 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5623e0ad88b0 0x5623e0ae5c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:54.576 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:54.573+0000 7fc0f5400640 1 -- 172.21.15.47:0/1779527656 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5623e0b4c8e0 con 0x5623e0ae1de0 2024-03-21T10:34:54.576 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:54.573+0000 7fc0f4a00640 1 --2- 172.21.15.47:0/1779527656 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5623e0ad88b0 0x5623e0ae5c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T10:34:54.576 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:54.573+0000 7fc0f5400640 1 --2- 172.21.15.47:0/1779527656 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5623e0ae1de0 0x5623e0b4c3a0 secure :-1 s=READY pgs=1275 cs=0 l=1 rev1=1 crypto rx=0x7fc0e8003d20 tx=0x7fc0e80098c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dc4078bc286ece67 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:34:54.577 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:54.573+0000 7fc0d6000640 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-03-21T10:34:54.577 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:54.574+0000 7fc0d6000640 1 -- 172.21.15.47:0/1779527656 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0e8043040 con 0x5623e0ae1de0 2024-03-21T10:34:54.577 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:54.574+0000 7fc0d6000640 1 -- 172.21.15.47:0/1779527656 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(103..103 src has 1..103) v4 ==== 9562+0+0 (secure 0 0 0) 0x7fc0e80326f0 con 0x5623e0ae1de0 2024-03-21T10:34:54.577 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:54.574+0000 7fc0fa1d4740 1 -- 172.21.15.47:0/1779527656 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fc0e4005670 con 0x5623e0ae1de0 2024-03-21T10:34:54.577 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:54.574+0000 7fc0d6000640 1 -- 172.21.15.47:0/1779527656 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc0e8059050 con 0x5623e0ae1de0 2024-03-21T10:34:54.577 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:54.574+0000 7fc0fa1d4740 10 client.18672 fetch_fsmap learned FSMap version 32 2024-03-21T10:34:54.577 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:54.574+0000 7fc0fa1d4740 1 -- 172.21.15.47:0/1779527656 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x5623e0b4db50 con 0x5623e0ae1de0 2024-03-21T10:34:54.577 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:54.574+0000 7fc0d6000640 1 -- 172.21.15.47:0/1779527656 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fc0e4005670 con 0x5623e0ae1de0 2024-03-21T10:34:54.577 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:54.574+0000 7fc0d6000640 1 -- 172.21.15.47:0/1779527656 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 32) v1 ==== 7942+0+0 (secure 0 0 0) 0x7fc0e8034440 con 0x5623e0ae1de0 2024-03-21T10:34:54.577 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:54.575+0000 7fc0fa1d4740 10 client.18672 fetch_fsmap finished waiting for FSMap version 32 2024-03-21T10:34:54.577 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:54.575+0000 7fc0fa1d4740 10 client.18672 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-21T10:34:54.577 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:54.575+0000 7fc0fa1d4740 20 client.18672 populate_metadata read hostname 'smithi047' 2024-03-21T10:34:54.577 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:54.575+0000 7fc0fa1d4740 1 --2- 172.21.15.47:0/1779527656 >> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] conn(0x5623e0b4de30 0x5623e0b50220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:34:54.577 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:54.575+0000 7fc0fa1d4740 4 client.18672 mds_command: new command op to 34535 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-21T10:34:54.578 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:54.575+0000 7fc0fa1d4740 1 -- 172.21.15.47:0/1779527656 --> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x5623e0a43b50 con 0x5623e0b4de30 2024-03-21T10:34:54.578 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:54.575+0000 7fc0f4a00640 1 --2- 172.21.15.47:0/1779527656 >> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] conn(0x5623e0b4de30 0x5623e0b50220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:34:54.578 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:54.576+0000 7fc0f4a00640 1 --2- 172.21.15.47:0/1779527656 >> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] conn(0x5623e0b4de30 0x5623e0b50220 crc :-1 s=READY pgs=216 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=61e10aeff3a7645f server_cookie=d245bb8c7724844a in_seq=0 out_seq=0 2024-03-21T10:34:54.578 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:54.576+0000 7fc0d6000640 10 client.18672 ms_handle_connect on v2:172.21.15.100:6838/4173168440 2024-03-21T10:34:54.578 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:54.576+0000 7fc0d6000640 1 -- 172.21.15.47:0/1779527656 <== mds.0 v2:172.21.15.100:6838/4173168440 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x5623e0a43b50 con 0x5623e0b4de30 2024-03-21T10:34:54.578 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:54.576+0000 7fc0d6000640 10 client.18672 handle_command_reply: tid=0 2024-03-21T10:34:54.578 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:54.576+0000 7fc0fa1d4740 1 client.18672 shutdown 2024-03-21T10:34:54.578 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:54.576+0000 7fc0fa1d4740 1 -- 172.21.15.47:0/1779527656 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5623e0ae1de0 msgr2=0x5623e0b4c3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:34:54.578 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:54.576+0000 7fc0fa1d4740 1 --2- 172.21.15.47:0/1779527656 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5623e0ae1de0 0x5623e0b4c3a0 secure :-1 s=READY pgs=1275 cs=0 l=1 rev1=1 crypto rx=0x7fc0e8003d20 tx=0x7fc0e80098c0 comp rx=0 tx=0).stop 2024-03-21T10:34:54.578 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:54.577+0000 7fc0fa1d4740 1 -- 172.21.15.47:0/1779527656 shutdown_connections 2024-03-21T10:34:54.578 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:54.577+0000 7fc0fa1d4740 1 --2- 172.21.15.47:0/1779527656 >> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] conn(0x5623e0b4de30 0x5623e0b50220 crc :-1 s=READY pgs=216 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:54.578 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:54.577+0000 7fc0fa1d4740 1 --2- 172.21.15.47:0/1779527656 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5623e0ae1de0 0x5623e0b4c3a0 unknown :-1 s=CLOSED pgs=1275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:54.578 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:54.577+0000 7fc0fa1d4740 1 --2- 172.21.15.47:0/1779527656 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5623e0ad9260 0x5623e0ae6160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:54.579 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:54.577+0000 7fc0fa1d4740 1 --2- 172.21.15.47:0/1779527656 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5623e0ad88b0 0x5623e0ae5c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:54.579 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:54.577+0000 7fc0d6000640 0 client.18672 ms_handle_reset on v2:172.21.15.100:6838/4173168440 2024-03-21T10:34:54.579 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:54.577+0000 7fc0fa1d4740 1 -- 172.21.15.47:0/1779527656 >> 172.21.15.47:0/1779527656 conn(0x5623e0ad2f80 msgr2=0x5623e0ad4370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:34:54.579 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:54.577+0000 7fc0fa1d4740 1 -- 172.21.15.47:0/1779527656 shutdown_connections 2024-03-21T10:34:54.579 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:54.577+0000 7fc0fa1d4740 1 -- 172.21.15.47:0/1779527656 wait complete. 2024-03-21T10:34:54.579 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:54.577+0000 7fc0fa1d4740 20 client.18672 trim_cache size 0 max 16384 2024-03-21T10:34:54.579 INFO:teuthology.orchestra.run.smithi047.stdout:[] 2024-03-21T10:34:54.580 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:54.578+0000 7fc0f5e00640 1 -- 172.21.15.47:0/2387540531 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7fc0d80a3110 msgr2=0x7fc0d80a55d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:34:54.580 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:54.578+0000 7fc0f5e00640 1 --2- 172.21.15.47:0/2387540531 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7fc0d80a3110 0x7fc0d80a55d0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x5623e0ad3ff0 tx=0x7fc0e4005f90 comp rx=0 tx=0).stop 2024-03-21T10:34:54.580 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:54.578+0000 7fc0f5e00640 1 -- 172.21.15.47:0/2387540531 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc0f01040e0 msgr2=0x7fc0f019a4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:34:54.580 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:54.578+0000 7fc0f5e00640 1 --2- 172.21.15.47:0/2387540531 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc0f01040e0 0x7fc0f019a4b0 secure :-1 s=READY pgs=1271 cs=0 l=1 rev1=1 crypto rx=0x7fc0dc00e9f0 tx=0x7fc0dc00edf0 comp rx=0 tx=0).stop 2024-03-21T10:34:54.580 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:54.578+0000 7fc0f5e00640 1 -- 172.21.15.47:0/2387540531 shutdown_connections 2024-03-21T10:34:54.580 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:54.578+0000 7fc0f5e00640 1 --2- 172.21.15.47:0/2387540531 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc0f0109730 0x7fc0f019e7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:54.580 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:54.578+0000 7fc0f5e00640 1 --2- 172.21.15.47:0/2387540531 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc0f01040e0 0x7fc0f019a4b0 unknown :-1 s=CLOSED pgs=1271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:54.580 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:54.578+0000 7fc0f5e00640 1 --2- 172.21.15.47:0/2387540531 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7fc0d80a3110 0x7fc0d80a55d0 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:54.581 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:54.578+0000 7fc0f5e00640 1 --2- 172.21.15.47:0/2387540531 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc0f0103720 0x7fc0f0199f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:54.581 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:54.578+0000 7fc0f5e00640 1 -- 172.21.15.47:0/2387540531 >> 172.21.15.47:0/2387540531 conn(0x7fc0f00ff1d0 msgr2=0x7fc0f00ff9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:34:54.581 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:54.578+0000 7fc0f5e00640 1 -- 172.21.15.47:0/2387540531 shutdown_connections 2024-03-21T10:34:54.581 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:54.578+0000 7fc0f5e00640 1 -- 172.21.15.47:0/2387540531 wait complete. 2024-03-21T10:34:55.596 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-03-21T10:34:55.596 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-21T10:34:55.596 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-03-21T10:34:55.686 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:34:55 smithi047 ceph-mon[31547]: pgmap v297: 129 pgs: 129 active+clean; 2.5 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-21T10:34:55.726 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:34:55 smithi100 ceph-mon[35193]: pgmap v297: 129 pgs: 129 active+clean; 2.5 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-21T10:34:55.737 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:34:55 smithi157 ceph-mon[35581]: pgmap v297: 129 pgs: 129 active+clean; 2.5 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-21T10:34:55.828 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:55.830+0000 7f0a78a00640 1 -- 172.21.15.47:0/104128379 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0a74101530 msgr2=0x7f0a741019b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:34:55.828 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:55.830+0000 7f0a78a00640 1 --2- 172.21.15.47:0/104128379 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0a74101530 0x7f0a741019b0 secure :-1 s=READY pgs=1276 cs=0 l=1 rev1=1 crypto rx=0x7f0a5c009900 tx=0x7f0a5c02bc30 comp rx=0 tx=0).stop 2024-03-21T10:34:55.828 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:55.830+0000 7f0a78a00640 1 -- 172.21.15.47:0/104128379 shutdown_connections 2024-03-21T10:34:55.828 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:55.830+0000 7f0a78a00640 1 --2- 172.21.15.47:0/104128379 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0a74101ef0 0x7f0a7410e3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:55.829 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:55.830+0000 7f0a78a00640 1 --2- 172.21.15.47:0/104128379 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0a74101530 0x7f0a741019b0 unknown :-1 s=CLOSED pgs=1276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:55.829 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:55.830+0000 7f0a78a00640 1 --2- 172.21.15.47:0/104128379 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0a741075f0 0x7f0a741079d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:55.829 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:55.830+0000 7f0a78a00640 1 -- 172.21.15.47:0/104128379 >> 172.21.15.47:0/104128379 conn(0x7f0a740fd000 msgr2=0x7f0a740ff420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:34:55.829 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:55.830+0000 7f0a78a00640 1 -- 172.21.15.47:0/104128379 shutdown_connections 2024-03-21T10:34:55.829 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:55.830+0000 7f0a78a00640 1 -- 172.21.15.47:0/104128379 wait complete. 2024-03-21T10:34:55.831 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:55.831+0000 7f0a78a00640 1 Processor -- start 2024-03-21T10:34:55.831 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:55.831+0000 7f0a78a00640 1 -- start start 2024-03-21T10:34:55.831 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:55.831+0000 7f0a78a00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0a74101530 0x7f0a740739a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:34:55.831 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:55.831+0000 7f0a78a00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0a74101ef0 0x7f0a74073ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:34:55.831 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:55.831+0000 7f0a78a00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0a741075f0 0x7f0a74074420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:34:55.831 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:55.831+0000 7f0a78a00640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a74111760 con 0x7f0a74101530 2024-03-21T10:34:55.832 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:55.831+0000 7f0a78a00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a741115e0 con 0x7f0a741075f0 2024-03-21T10:34:55.832 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:55.831+0000 7f0a78a00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a741118e0 con 0x7f0a74101ef0 2024-03-21T10:34:55.832 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:55.832+0000 7f0a72800640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0a74101530 0x7f0a740739a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:34:55.832 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:55.832+0000 7f0a72800640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0a74101530 0x7f0a740739a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:55862/0 (socket says 172.21.15.47:55862) 2024-03-21T10:34:55.832 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:55.832+0000 7f0a72800640 1 -- 172.21.15.47:0/1783076286 learned_addr learned my addr 172.21.15.47:0/1783076286 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:34:55.832 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:55.832+0000 7f0a73200640 1 --2- 172.21.15.47:0/1783076286 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0a741075f0 0x7f0a74074420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:34:55.832 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:55.832+0000 7f0a6be00640 1 --2- 172.21.15.47:0/1783076286 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0a74101ef0 0x7f0a74073ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:34:55.832 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:55.832+0000 7f0a72800640 1 -- 172.21.15.47:0/1783076286 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0a74101ef0 msgr2=0x7f0a74073ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:34:55.832 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:55.832+0000 7f0a72800640 1 --2- 172.21.15.47:0/1783076286 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0a74101ef0 0x7f0a74073ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:55.832 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:55.832+0000 7f0a72800640 1 -- 172.21.15.47:0/1783076286 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0a741075f0 msgr2=0x7f0a74074420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:34:55.832 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:55.832+0000 7f0a72800640 1 --2- 172.21.15.47:0/1783076286 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0a741075f0 0x7f0a74074420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:55.833 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:55.832+0000 7f0a72800640 1 -- 172.21.15.47:0/1783076286 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a741a6120 con 0x7f0a74101530 2024-03-21T10:34:55.833 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:55.832+0000 7f0a72800640 1 --2- 172.21.15.47:0/1783076286 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0a74101530 0x7f0a740739a0 secure :-1 s=READY pgs=1277 cs=0 l=1 rev1=1 crypto rx=0x7f0a6400c920 tx=0x7f0a6400cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:34:55.833 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:55.833+0000 7f0a6b400640 1 -- 172.21.15.47:0/1783076286 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a64007c70 con 0x7f0a74101530 2024-03-21T10:34:55.833 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:55.833+0000 7f0a78a00640 1 -- 172.21.15.47:0/1783076286 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0a741a6410 con 0x7f0a74101530 2024-03-21T10:34:55.833 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:55.833+0000 7f0a6b400640 1 -- 172.21.15.47:0/1783076286 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f0a64007e10 con 0x7f0a74101530 2024-03-21T10:34:55.833 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:55.833+0000 7f0a6b400640 1 -- 172.21.15.47:0/1783076286 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a64005220 con 0x7f0a74101530 2024-03-21T10:34:55.833 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:55.833+0000 7f0a78a00640 1 -- 172.21.15.47:0/1783076286 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0a741a69b0 con 0x7f0a74101530 2024-03-21T10:34:55.834 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:55.836+0000 7f0a6b400640 1 -- 172.21.15.47:0/1783076286 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 27) v1 ==== 137985+0+0 (secure 0 0 0) 0x7f0a640053c0 con 0x7f0a74101530 2024-03-21T10:34:55.836 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:55.838+0000 7f0a7c1c8740 1 Processor -- start 2024-03-21T10:34:55.836 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:55.838+0000 7f0a7c1c8740 1 -- start start 2024-03-21T10:34:55.837 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:55.838+0000 7f0a7c1c8740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563ebd991290 0x563ebd991670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:34:55.837 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:55.838+0000 7f0a7c1c8740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563ebd991bb0 0x563ebd99a250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:34:55.837 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:55.838+0000 7f0a7c1c8740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x563ebd99a790 0x563ebd99cb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:34:55.837 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:55.838+0000 7f0a7c1c8740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x563ebd8f5eb0 con 0x563ebd99a790 2024-03-21T10:34:55.837 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:55.838+0000 7f0a7c1c8740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x563ebd89ea60 con 0x563ebd991bb0 2024-03-21T10:34:55.837 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:55.838+0000 7f0a7c1c8740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x563ebd8f1f50 con 0x563ebd991290 2024-03-21T10:34:55.838 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:55.839+0000 7f0a6b400640 1 --2- 172.21.15.47:0/1783076286 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f0a580a3090 0x7f0a580a5550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:34:55.838 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:55.839+0000 7f0a6be00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x563ebd99a790 0x563ebd99cb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:34:55.838 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:55.839+0000 7f0a6be00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x563ebd99a790 0x563ebd99cb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:55870/0 (socket says 172.21.15.47:55870) 2024-03-21T10:34:55.838 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:55.839+0000 7f0a6be00640 1 -- 172.21.15.47:0/1174783475 learned_addr learned my addr 172.21.15.47:0/1174783475 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:34:55.838 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:55.839+0000 7f0a6b400640 1 -- 172.21.15.47:0/1783076286 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 9562+0+0 (secure 0 0 0) 0x7f0a640cfdb0 con 0x7f0a74101530 2024-03-21T10:34:55.838 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:55.839+0000 7f0a73200640 1 --2- 172.21.15.47:0/1174783475 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563ebd991290 0x563ebd991670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:34:55.838 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:55.839+0000 7f0a72800640 1 --2- 172.21.15.47:0/1174783475 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563ebd991bb0 0x563ebd99a250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:34:55.838 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:55.839+0000 7f0a73200640 1 --2- 172.21.15.47:0/1783076286 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f0a580a3090 0x7f0a580a5550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:34:55.840 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:55.839+0000 7f0a6be00640 1 -- 172.21.15.47:0/1174783475 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563ebd991290 msgr2=0x563ebd991670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:34:55.840 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:55.839+0000 7f0a6be00640 1 --2- 172.21.15.47:0/1174783475 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563ebd991290 0x563ebd991670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:55.840 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:55.840+0000 7f0a6be00640 1 -- 172.21.15.47:0/1174783475 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563ebd991bb0 msgr2=0x563ebd99a250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:34:55.840 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:55.840+0000 7f0a6be00640 1 --2- 172.21.15.47:0/1174783475 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563ebd991bb0 0x563ebd99a250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:55.840 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:55.840+0000 7f0a6be00640 1 -- 172.21.15.47:0/1174783475 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563ebd6d27e0 con 0x563ebd99a790 2024-03-21T10:34:55.840 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:55.840+0000 7f0a6be00640 1 --2- 172.21.15.47:0/1174783475 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x563ebd99a790 0x563ebd99cb80 secure :-1 s=READY pgs=1278 cs=0 l=1 rev1=1 crypto rx=0x7f0a5c036040 tx=0x7f0a5c009630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1786cac746bb0df4 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:34:55.840 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:55.840+0000 7f0a73200640 1 --2- 172.21.15.47:0/1783076286 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f0a580a3090 0x7f0a580a5550 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f0a6c005c40 tx=0x7f0a6c005bd0 comp rx=0 tx=0).ready entity=mgr.15687 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:34:55.840 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:55.840+0000 7f0a63400640 1 -- 172.21.15.47:0/1174783475 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a5c02c840 con 0x563ebd99a790 2024-03-21T10:34:55.840 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:55.840+0000 7f0a63400640 1 -- 172.21.15.47:0/1174783475 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f0a5c02c9e0 con 0x563ebd99a790 2024-03-21T10:34:55.840 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:55.840+0000 7f0a63400640 1 -- 172.21.15.47:0/1174783475 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a5c0046c0 con 0x563ebd99a790 2024-03-21T10:34:55.840 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:55.841+0000 7f0a7c1c8740 1 -- 172.21.15.47:0/1174783475 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x563ebd99a790 msgr2=0x563ebd99cb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:34:55.841 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:55.841+0000 7f0a7c1c8740 1 --2- 172.21.15.47:0/1174783475 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x563ebd99a790 0x563ebd99cb80 secure :-1 s=READY pgs=1278 cs=0 l=1 rev1=1 crypto rx=0x7f0a5c036040 tx=0x7f0a5c009630 comp rx=0 tx=0).stop 2024-03-21T10:34:55.841 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:55.841+0000 7f0a7c1c8740 1 -- 172.21.15.47:0/1174783475 shutdown_connections 2024-03-21T10:34:55.841 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:55.841+0000 7f0a7c1c8740 1 --2- 172.21.15.47:0/1174783475 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x563ebd99a790 0x563ebd99cb80 unknown :-1 s=CLOSED pgs=1278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:55.841 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:55.841+0000 7f0a7c1c8740 1 --2- 172.21.15.47:0/1174783475 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563ebd991bb0 0x563ebd99a250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:55.841 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:55.841+0000 7f0a7c1c8740 1 --2- 172.21.15.47:0/1174783475 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563ebd991290 0x563ebd991670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:55.841 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:55.841+0000 7f0a7c1c8740 1 -- 172.21.15.47:0/1174783475 >> 172.21.15.47:0/1174783475 conn(0x563ebd98b9c0 msgr2=0x563ebd98bda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:34:55.841 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:55.841+0000 7f0a7c1c8740 1 -- 172.21.15.47:0/1174783475 shutdown_connections 2024-03-21T10:34:55.841 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:55.841+0000 7f0a7c1c8740 1 -- 172.21.15.47:0/1174783475 wait complete. 2024-03-21T10:34:55.841 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:55.842+0000 7f0a7c1c8740 1 -- start start 2024-03-21T10:34:55.841 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:55.842+0000 7f0a7c1c8740 1 Processor -- start 2024-03-21T10:34:55.844 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:55.842+0000 7f0a7c1c8740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563ebd991290 0x563ebd99a190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:34:55.844 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:55.842+0000 7f0a7c1c8740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563ebd991bb0 0x563ebd997520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:34:55.844 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:55.843+0000 7f0a7c1c8740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x563ebd99a790 0x563ebd997b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:34:55.845 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:55.843+0000 7f0a7c1c8740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x563ebd99d520 con 0x563ebd99a790 2024-03-21T10:34:55.845 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:55.843+0000 7f0a7c1c8740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x563ebd8b1eb0 con 0x563ebd991bb0 2024-03-21T10:34:55.845 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:55.843+0000 7f0a7c1c8740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x563ebd99d6a0 con 0x563ebd991290 2024-03-21T10:34:55.845 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:55.843+0000 7f0a7c1c8740 10 client.0 _ll_register_callbacks cb 0x563ebd8a6150 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T10:34:55.845 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:55.843+0000 7f0a73200640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x563ebd99a790 0x563ebd997b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:34:55.845 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:55.843+0000 7f0a73200640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x563ebd99a790 0x563ebd997b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:55886/0 (socket says 172.21.15.47:55886) 2024-03-21T10:34:55.845 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:55.843+0000 7f0a73200640 1 -- 172.21.15.47:0/3442725169 learned_addr learned my addr 172.21.15.47:0/3442725169 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:34:55.845 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:55.843+0000 7f0a6be00640 1 --2- 172.21.15.47:0/3442725169 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563ebd991bb0 0x563ebd997520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:34:55.845 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:55.843+0000 7f0a72800640 1 --2- 172.21.15.47:0/3442725169 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563ebd991290 0x563ebd99a190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:34:55.845 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:55.843+0000 7f0a73200640 1 -- 172.21.15.47:0/3442725169 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563ebd991290 msgr2=0x563ebd99a190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:34:55.845 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:55.843+0000 7f0a73200640 1 --2- 172.21.15.47:0/3442725169 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563ebd991290 0x563ebd99a190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:55.845 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:55.843+0000 7f0a73200640 1 -- 172.21.15.47:0/3442725169 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563ebd991bb0 msgr2=0x563ebd997520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:34:55.845 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:55.843+0000 7f0a73200640 1 --2- 172.21.15.47:0/3442725169 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563ebd991bb0 0x563ebd997520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:55.845 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:55.843+0000 7f0a73200640 1 -- 172.21.15.47:0/3442725169 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x563ebd732f70 con 0x563ebd99a790 2024-03-21T10:34:55.846 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:55.844+0000 7f0a73200640 1 --2- 172.21.15.47:0/3442725169 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x563ebd99a790 0x563ebd997b30 secure :-1 s=READY pgs=1279 cs=0 l=1 rev1=1 crypto rx=0x7f0a6c0121a0 tx=0x7f0a6c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3ea87eb1558a3761 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:34:55.846 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:55.844+0000 7f0a57e00640 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-03-21T10:34:55.846 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:55.844+0000 7f0a57e00640 1 -- 172.21.15.47:0/3442725169 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a6c025070 con 0x563ebd99a790 2024-03-21T10:34:55.846 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:55.844+0000 7f0a57e00640 1 -- 172.21.15.47:0/3442725169 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(103..103 src has 1..103) v4 ==== 9562+0+0 (secure 0 0 0) 0x7f0a6c010980 con 0x563ebd99a790 2024-03-21T10:34:55.846 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:55.844+0000 7f0a7c1c8740 1 -- 172.21.15.47:0/3442725169 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f0a5c02c9e0 con 0x563ebd99a790 2024-03-21T10:34:55.846 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:55.844+0000 7f0a57e00640 1 -- 172.21.15.47:0/3442725169 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0a6c031050 con 0x563ebd99a790 2024-03-21T10:34:55.846 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:55.844+0000 7f0a7c1c8740 10 client.18684 fetch_fsmap learned FSMap version 32 2024-03-21T10:34:55.846 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:55.844+0000 7f0a7c1c8740 1 -- 172.21.15.47:0/3442725169 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x563ebd5ca170 con 0x563ebd99a790 2024-03-21T10:34:55.846 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:55.845+0000 7f0a57e00640 1 -- 172.21.15.47:0/3442725169 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f0a5c02c9e0 con 0x563ebd99a790 2024-03-21T10:34:55.846 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:55.845+0000 7f0a57e00640 1 -- 172.21.15.47:0/3442725169 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 32) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f0a6c021980 con 0x563ebd99a790 2024-03-21T10:34:55.846 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:55.845+0000 7f0a7c1c8740 10 client.18684 fetch_fsmap finished waiting for FSMap version 32 2024-03-21T10:34:55.846 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:55.845+0000 7f0a7c1c8740 10 client.18684 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-21T10:34:55.846 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:55.845+0000 7f0a7c1c8740 20 client.18684 populate_metadata read hostname 'smithi047' 2024-03-21T10:34:55.846 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:55.845+0000 7f0a7c1c8740 1 --2- 172.21.15.47:0/3442725169 >> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] conn(0x563ebd9aa7b0 0x563ebd9acc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:34:55.847 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:55.845+0000 7f0a7c1c8740 4 client.18684 mds_command: new command op to 34535 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T10:34:55.847 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:55.845+0000 7f0a7c1c8740 1 -- 172.21.15.47:0/3442725169 --> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x563ebd9033f0 con 0x563ebd9aa7b0 2024-03-21T10:34:55.847 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:55.846+0000 7f0a72800640 1 --2- 172.21.15.47:0/3442725169 >> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] conn(0x563ebd9aa7b0 0x563ebd9acc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:34:55.847 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:55.846+0000 7f0a72800640 1 --2- 172.21.15.47:0/3442725169 >> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] conn(0x563ebd9aa7b0 0x563ebd9acc10 crc :-1 s=READY pgs=217 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f3c2730c77187349 server_cookie=b59cddeac4ea2e69 in_seq=0 out_seq=0 2024-03-21T10:34:55.847 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:55.846+0000 7f0a57e00640 10 client.18684 ms_handle_connect on v2:172.21.15.100:6838/4173168440 2024-03-21T10:34:55.848 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:55.847+0000 7f0a57e00640 1 -- 172.21.15.47:0/3442725169 <== mds.0 v2:172.21.15.100:6838/4173168440 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x563ebd9033f0 con 0x563ebd9aa7b0 2024-03-21T10:34:55.848 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:55.847+0000 7f0a57e00640 10 client.18684 handle_command_reply: tid=0 2024-03-21T10:34:55.848 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:55.847+0000 7f0a7c1c8740 1 client.18684 shutdown 2024-03-21T10:34:55.848 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:55.847+0000 7f0a7c1c8740 1 -- 172.21.15.47:0/3442725169 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x563ebd99a790 msgr2=0x563ebd997b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:34:55.848 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:55.847+0000 7f0a7c1c8740 1 --2- 172.21.15.47:0/3442725169 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x563ebd99a790 0x563ebd997b30 secure :-1 s=READY pgs=1279 cs=0 l=1 rev1=1 crypto rx=0x7f0a6c0121a0 tx=0x7f0a6c004a40 comp rx=0 tx=0).stop 2024-03-21T10:34:55.848 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:55.847+0000 7f0a7c1c8740 1 -- 172.21.15.47:0/3442725169 shutdown_connections 2024-03-21T10:34:55.848 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:55.847+0000 7f0a7c1c8740 1 --2- 172.21.15.47:0/3442725169 >> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] conn(0x563ebd9aa7b0 0x563ebd9acc10 crc :-1 s=READY pgs=217 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:55.848 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:55.847+0000 7f0a7c1c8740 1 --2- 172.21.15.47:0/3442725169 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x563ebd99a790 0x563ebd997b30 unknown :-1 s=CLOSED pgs=1279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:55.848 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:55.847+0000 7f0a7c1c8740 1 --2- 172.21.15.47:0/3442725169 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563ebd991bb0 0x563ebd997520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:55.848 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:55.847+0000 7f0a7c1c8740 1 --2- 172.21.15.47:0/3442725169 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563ebd991290 0x563ebd99a190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:55.848 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:55.847+0000 7f0a57e00640 0 client.18684 ms_handle_reset on v2:172.21.15.100:6838/4173168440 2024-03-21T10:34:55.849 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:55.847+0000 7f0a7c1c8740 1 -- 172.21.15.47:0/3442725169 >> 172.21.15.47:0/3442725169 conn(0x563ebd98b9c0 msgr2=0x563ebd98d710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:34:55.849 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:55.848+0000 7f0a7c1c8740 1 -- 172.21.15.47:0/3442725169 shutdown_connections 2024-03-21T10:34:55.849 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:55.848+0000 7f0a7c1c8740 1 -- 172.21.15.47:0/3442725169 wait complete. 2024-03-21T10:34:55.849 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:55.848+0000 7f0a7c1c8740 20 client.18684 trim_cache size 0 max 16384 2024-03-21T10:34:55.872 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:55.874+0000 7f0a7c1c8740 1 Processor -- start 2024-03-21T10:34:55.873 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:55.874+0000 7f0a7c1c8740 1 -- start start 2024-03-21T10:34:55.873 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:55.874+0000 7f0a7c1c8740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x563ebd991290 0x563ebd999c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:34:55.873 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:55.874+0000 7f0a7c1c8740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563ebd991bb0 0x563ebd99a190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:34:55.873 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:55.874+0000 7f0a7c1c8740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563ebd99a790 0x563ebd9a5db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:34:55.873 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:55.874+0000 7f0a7c1c8740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a741b09d0 con 0x563ebd991290 2024-03-21T10:34:55.873 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:55.874+0000 7f0a7c1c8740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a741b2350 con 0x563ebd991bb0 2024-03-21T10:34:55.873 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:55.874+0000 7f0a7c1c8740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a741b1030 con 0x563ebd99a790 2024-03-21T10:34:55.874 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:55.875+0000 7f0a72800640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x563ebd991290 0x563ebd999c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:34:55.874 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:55.875+0000 7f0a6be00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563ebd991bb0 0x563ebd99a190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:34:55.875 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:55.875+0000 7f0a72800640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x563ebd991290 0x563ebd999c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:55896/0 (socket says 172.21.15.47:55896) 2024-03-21T10:34:55.875 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:55.875+0000 7f0a6be00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563ebd991bb0 0x563ebd99a190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:45974/0 (socket says 172.21.15.47:45974) 2024-03-21T10:34:55.875 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:55.875+0000 7f0a72800640 1 -- 172.21.15.47:0/3186718327 learned_addr learned my addr 172.21.15.47:0/3186718327 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:34:55.875 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:55.875+0000 7f0a73200640 1 --2- 172.21.15.47:0/3186718327 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563ebd99a790 0x563ebd9a5db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:34:55.875 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:55.875+0000 7f0a6be00640 1 -- 172.21.15.47:0/3186718327 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563ebd99a790 msgr2=0x563ebd9a5db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:34:55.875 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:55.875+0000 7f0a6be00640 1 --2- 172.21.15.47:0/3186718327 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563ebd99a790 0x563ebd9a5db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:55.875 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:55.875+0000 7f0a6be00640 1 -- 172.21.15.47:0/3186718327 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x563ebd991290 msgr2=0x563ebd999c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:34:55.875 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:55.876+0000 7f0a6be00640 1 --2- 172.21.15.47:0/3186718327 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x563ebd991290 0x563ebd999c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:55.875 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:55.876+0000 7f0a6be00640 1 -- 172.21.15.47:0/3186718327 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563ebd98b0a0 con 0x563ebd991bb0 2024-03-21T10:34:55.875 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:55.876+0000 7f0a72800640 1 --2- 172.21.15.47:0/3186718327 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x563ebd991290 0x563ebd999c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T10:34:55.876 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:55.876+0000 7f0a6be00640 1 --2- 172.21.15.47:0/3186718327 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563ebd991bb0 0x563ebd99a190 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f0a5c009900 tx=0x7f0a5c02bc30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=39bee3a9dfae8bac server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:34:55.876 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:55.876+0000 7f0a62a00640 1 -- 172.21.15.47:0/3186718327 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a5c04b030 con 0x563ebd991bb0 2024-03-21T10:34:55.876 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:55.876+0000 7f0a62a00640 1 -- 172.21.15.47:0/3186718327 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f0a5c004040 con 0x563ebd991bb0 2024-03-21T10:34:55.876 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:55.876+0000 7f0a62a00640 1 -- 172.21.15.47:0/3186718327 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a5c005440 con 0x563ebd991bb0 2024-03-21T10:34:55.879 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:55.877+0000 7f0a7c1c8740 1 -- 172.21.15.47:0/3186718327 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563ebd991bb0 msgr2=0x563ebd99a190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:34:55.879 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:55.877+0000 7f0a7c1c8740 1 --2- 172.21.15.47:0/3186718327 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563ebd991bb0 0x563ebd99a190 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f0a5c009900 tx=0x7f0a5c02bc30 comp rx=0 tx=0).stop 2024-03-21T10:34:55.879 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:55.877+0000 7f0a7c1c8740 1 -- 172.21.15.47:0/3186718327 shutdown_connections 2024-03-21T10:34:55.879 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:55.877+0000 7f0a7c1c8740 1 --2- 172.21.15.47:0/3186718327 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563ebd99a790 0x563ebd9a5db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:55.880 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:55.877+0000 7f0a7c1c8740 1 --2- 172.21.15.47:0/3186718327 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563ebd991bb0 0x563ebd99a190 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:55.880 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:55.877+0000 7f0a7c1c8740 1 --2- 172.21.15.47:0/3186718327 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x563ebd991290 0x563ebd999c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:55.880 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:55.877+0000 7f0a7c1c8740 1 -- 172.21.15.47:0/3186718327 >> 172.21.15.47:0/3186718327 conn(0x563ebd98b9c0 msgr2=0x563ebd992960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:34:55.880 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:55.878+0000 7f0a7c1c8740 1 -- 172.21.15.47:0/3186718327 shutdown_connections 2024-03-21T10:34:55.880 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:55.878+0000 7f0a7c1c8740 1 -- 172.21.15.47:0/3186718327 wait complete. 2024-03-21T10:34:55.880 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:55.878+0000 7f0a7c1c8740 1 -- start start 2024-03-21T10:34:55.880 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:55.878+0000 7f0a7c1c8740 1 Processor -- start 2024-03-21T10:34:55.880 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:55.879+0000 7f0a7c1c8740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x563ebd991290 0x563ebd99eed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:34:55.880 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:55.879+0000 7f0a7c1c8740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563ebd991bb0 0x563ebd99f410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:34:55.880 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:55.879+0000 7f0a7c1c8740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563ebd99a790 0x563ebd99f950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:34:55.880 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:55.879+0000 7f0a7c1c8740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a741b1cf0 con 0x563ebd991290 2024-03-21T10:34:55.880 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:55.879+0000 7f0a7c1c8740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a741b1690 con 0x563ebd99a790 2024-03-21T10:34:55.880 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:55.879+0000 7f0a7c1c8740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a741aebb0 con 0x563ebd991bb0 2024-03-21T10:34:55.880 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:55.879+0000 7f0a7c1c8740 10 client.0 _ll_register_callbacks cb 0x7f0a6c012030 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T10:34:55.881 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:55.879+0000 7f0a72800640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x563ebd991290 0x563ebd99eed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:34:55.881 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:55.880+0000 7f0a72800640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x563ebd991290 0x563ebd99eed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:55902/0 (socket says 172.21.15.47:55902) 2024-03-21T10:34:55.881 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:55.880+0000 7f0a72800640 1 -- 172.21.15.47:0/2422290912 learned_addr learned my addr 172.21.15.47:0/2422290912 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:34:55.881 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:55.880+0000 7f0a73200640 1 --2- 172.21.15.47:0/2422290912 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563ebd99a790 0x563ebd99f950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:34:55.881 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:55.880+0000 7f0a6be00640 1 --2- 172.21.15.47:0/2422290912 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563ebd991bb0 0x563ebd99f410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:34:55.881 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:55.880+0000 7f0a72800640 1 -- 172.21.15.47:0/2422290912 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563ebd991bb0 msgr2=0x563ebd99f410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:34:55.881 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:55.880+0000 7f0a72800640 1 --2- 172.21.15.47:0/2422290912 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563ebd991bb0 0x563ebd99f410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:55.881 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:55.880+0000 7f0a72800640 1 -- 172.21.15.47:0/2422290912 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563ebd99a790 msgr2=0x563ebd99f950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:34:55.881 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:55.880+0000 7f0a72800640 1 --2- 172.21.15.47:0/2422290912 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563ebd99a790 0x563ebd99f950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:55.881 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:55.880+0000 7f0a72800640 1 -- 172.21.15.47:0/2422290912 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x563ebd9919e0 con 0x563ebd991290 2024-03-21T10:34:55.881 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:55.880+0000 7f0a72800640 1 --2- 172.21.15.47:0/2422290912 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x563ebd991290 0x563ebd99eed0 secure :-1 s=READY pgs=1280 cs=0 l=1 rev1=1 crypto rx=0x7f0a640cd030 tx=0x7f0a6400c870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=868677b9b224dda7 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:34:55.881 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:55.880+0000 7f0a57e00640 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-03-21T10:34:55.881 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:55.880+0000 7f0a57e00640 1 -- 172.21.15.47:0/2422290912 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a6409bd30 con 0x563ebd991290 2024-03-21T10:34:55.881 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:55.880+0000 7f0a57e00640 1 -- 172.21.15.47:0/2422290912 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(103..103 src has 1..103) v4 ==== 9562+0+0 (secure 0 0 0) 0x7f0a6409a030 con 0x563ebd991290 2024-03-21T10:34:55.882 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:55.880+0000 7f0a7c1c8740 1 -- 172.21.15.47:0/2422290912 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f0a5c004040 con 0x563ebd991290 2024-03-21T10:34:55.882 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:55.881+0000 7f0a57e00640 1 -- 172.21.15.47:0/2422290912 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0a6409fe80 con 0x563ebd991290 2024-03-21T10:34:55.882 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:55.881+0000 7f0a7c1c8740 10 client.18690 fetch_fsmap learned FSMap version 32 2024-03-21T10:34:55.882 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:55.881+0000 7f0a7c1c8740 1 -- 172.21.15.47:0/2422290912 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x563ebda07460 con 0x563ebd991290 2024-03-21T10:34:55.882 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:55.881+0000 7f0a57e00640 1 -- 172.21.15.47:0/2422290912 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f0a5c004040 con 0x563ebd991290 2024-03-21T10:34:55.882 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:55.881+0000 7f0a57e00640 1 -- 172.21.15.47:0/2422290912 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 32) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f0a640a42b0 con 0x563ebd991290 2024-03-21T10:34:55.882 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:55.881+0000 7f0a7c1c8740 10 client.18690 fetch_fsmap finished waiting for FSMap version 32 2024-03-21T10:34:55.882 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:55.881+0000 7f0a7c1c8740 10 client.18690 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-21T10:34:55.882 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:55.881+0000 7f0a7c1c8740 20 client.18690 populate_metadata read hostname 'smithi047' 2024-03-21T10:34:55.882 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:55.881+0000 7f0a7c1c8740 1 --2- 172.21.15.47:0/2422290912 >> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] conn(0x563ebda07b40 0x563ebda09f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:34:55.882 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:55.881+0000 7f0a7c1c8740 4 client.18690 mds_command: new command op to 34535 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-03-21T10:34:55.882 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:55.881+0000 7f0a7c1c8740 1 -- 172.21.15.47:0/2422290912 --> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x563ebd8fd740 con 0x563ebda07b40 2024-03-21T10:34:55.884 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:55.882+0000 7f0a6be00640 1 --2- 172.21.15.47:0/2422290912 >> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] conn(0x563ebda07b40 0x563ebda09f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:34:55.884 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:55.882+0000 7f0a6be00640 1 --2- 172.21.15.47:0/2422290912 >> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] conn(0x563ebda07b40 0x563ebda09f60 crc :-1 s=READY pgs=218 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bea4e791fe2d76f2 server_cookie=6627f58bfa354f80 in_seq=0 out_seq=0 2024-03-21T10:34:55.884 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:55.882+0000 7f0a57e00640 10 client.18690 ms_handle_connect on v2:172.21.15.100:6838/4173168440 2024-03-21T10:34:55.884 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:55.885+0000 7f0a57e00640 1 -- 172.21.15.47:0/2422290912 <== mds.0 v2:172.21.15.100:6838/4173168440 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x563ebd8fd740 con 0x563ebda07b40 2024-03-21T10:34:55.884 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:55.885+0000 7f0a57e00640 10 client.18690 handle_command_reply: tid=0 2024-03-21T10:34:55.884 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:55.885+0000 7f0a7c1c8740 1 client.18690 shutdown 2024-03-21T10:34:55.884 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:55.885+0000 7f0a7c1c8740 1 -- 172.21.15.47:0/2422290912 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x563ebd991290 msgr2=0x563ebd99eed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:34:55.884 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:55.885+0000 7f0a7c1c8740 1 --2- 172.21.15.47:0/2422290912 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x563ebd991290 0x563ebd99eed0 secure :-1 s=READY pgs=1280 cs=0 l=1 rev1=1 crypto rx=0x7f0a640cd030 tx=0x7f0a6400c870 comp rx=0 tx=0).stop 2024-03-21T10:34:55.884 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:55.885+0000 7f0a7c1c8740 1 -- 172.21.15.47:0/2422290912 shutdown_connections 2024-03-21T10:34:55.884 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:55.885+0000 7f0a7c1c8740 1 --2- 172.21.15.47:0/2422290912 >> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] conn(0x563ebda07b40 0x563ebda09f60 crc :-1 s=READY pgs=218 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:55.884 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:55.885+0000 7f0a7c1c8740 1 --2- 172.21.15.47:0/2422290912 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563ebd99a790 0x563ebd99f950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:55.884 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:55.885+0000 7f0a7c1c8740 1 --2- 172.21.15.47:0/2422290912 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x563ebd991bb0 0x563ebd99f410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:55.884 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:55.885+0000 7f0a7c1c8740 1 --2- 172.21.15.47:0/2422290912 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x563ebd991290 0x563ebd99eed0 unknown :-1 s=CLOSED pgs=1280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:55.885 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:55.885+0000 7f0a7c1c8740 1 -- 172.21.15.47:0/2422290912 >> 172.21.15.47:0/2422290912 conn(0x563ebd98b9c0 msgr2=0x563ebd98cad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:34:55.885 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:55.885+0000 7f0a57e00640 0 client.18690 ms_handle_reset on v2:172.21.15.100:6838/4173168440 2024-03-21T10:34:55.885 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:55.885+0000 7f0a7c1c8740 1 -- 172.21.15.47:0/2422290912 shutdown_connections 2024-03-21T10:34:55.885 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:55.885+0000 7f0a7c1c8740 1 -- 172.21.15.47:0/2422290912 wait complete. 2024-03-21T10:34:55.885 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:55.886+0000 7f0a7c1c8740 20 client.18690 trim_cache size 0 max 16384 2024-03-21T10:34:55.885 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-03-21T10:34:55.885 INFO:teuthology.orchestra.run.smithi047.stdout: "return_code": 0, 2024-03-21T10:34:55.885 INFO:teuthology.orchestra.run.smithi047.stdout: "scrub_tag": "a013be24-92aa-4d12-9eed-493ef8447c26", 2024-03-21T10:34:55.885 INFO:teuthology.orchestra.run.smithi047.stdout: "mode": "asynchronous" 2024-03-21T10:34:55.885 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-03-21T10:34:55.886 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:55.886+0000 7f0a78a00640 1 -- 172.21.15.47:0/1783076286 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f0a580a3090 msgr2=0x7f0a580a5550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:34:55.886 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:55.886+0000 7f0a78a00640 1 --2- 172.21.15.47:0/1783076286 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f0a580a3090 0x7f0a580a5550 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f0a6c005c40 tx=0x7f0a6c005bd0 comp rx=0 tx=0).stop 2024-03-21T10:34:55.886 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:55.886+0000 7f0a78a00640 1 -- 172.21.15.47:0/1783076286 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0a74101530 msgr2=0x7f0a740739a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:34:55.886 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:55.886+0000 7f0a78a00640 1 --2- 172.21.15.47:0/1783076286 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0a74101530 0x7f0a740739a0 secure :-1 s=READY pgs=1277 cs=0 l=1 rev1=1 crypto rx=0x7f0a6400c920 tx=0x7f0a6400cde0 comp rx=0 tx=0).stop 2024-03-21T10:34:55.886 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:55.886+0000 7f0a78a00640 1 -- 172.21.15.47:0/1783076286 shutdown_connections 2024-03-21T10:34:55.886 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:55.886+0000 7f0a78a00640 1 --2- 172.21.15.47:0/1783076286 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f0a580a3090 0x7f0a580a5550 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:55.886 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:55.886+0000 7f0a78a00640 1 --2- 172.21.15.47:0/1783076286 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0a741075f0 0x7f0a74074420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:55.887 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:55.886+0000 7f0a78a00640 1 --2- 172.21.15.47:0/1783076286 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0a74101ef0 0x7f0a74073ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:55.887 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:55.886+0000 7f0a78a00640 1 --2- 172.21.15.47:0/1783076286 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0a74101530 0x7f0a740739a0 unknown :-1 s=CLOSED pgs=1277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:55.887 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:55.886+0000 7f0a78a00640 1 -- 172.21.15.47:0/1783076286 >> 172.21.15.47:0/1783076286 conn(0x7f0a740fd000 msgr2=0x7f0a740fe8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:34:55.887 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:55.887+0000 7f0a78a00640 1 -- 172.21.15.47:0/1783076286 shutdown_connections 2024-03-21T10:34:55.887 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:55.887+0000 7f0a78a00640 1 -- 172.21.15.47:0/1783076286 wait complete. 2024-03-21T10:34:55.907 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-21T10:34:55.908 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-21T10:34:56.130 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:56.131+0000 7fb10ac00640 1 -- 172.21.15.47:0/3116254294 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb10410dd40 msgr2=0x7fb104110190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:34:56.130 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:56.131+0000 7fb10ac00640 1 --2- 172.21.15.47:0/3116254294 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb10410dd40 0x7fb104110190 secure :-1 s=READY pgs=1281 cs=0 l=1 rev1=1 crypto rx=0x7fb0fc009ca0 tx=0x7fb0fc02c190 comp rx=0 tx=0).stop 2024-03-21T10:34:56.130 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:56.131+0000 7fb10ac00640 1 -- 172.21.15.47:0/3116254294 shutdown_connections 2024-03-21T10:34:56.130 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:56.131+0000 7fb10ac00640 1 --2- 172.21.15.47:0/3116254294 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb10410dd40 0x7fb104110190 unknown :-1 s=CLOSED pgs=1281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:56.130 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:56.131+0000 7fb10ac00640 1 --2- 172.21.15.47:0/3116254294 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb1040743f0 0x7fb10410d800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:56.131 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:56.131+0000 7fb10ac00640 1 --2- 172.21.15.47:0/3116254294 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb104073ad0 0x7fb104073eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:56.131 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:56.131+0000 7fb10ac00640 1 -- 172.21.15.47:0/3116254294 >> 172.21.15.47:0/3116254294 conn(0x7fb1040fd010 msgr2=0x7fb1040ff430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:34:56.131 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:56.132+0000 7fb10ac00640 1 -- 172.21.15.47:0/3116254294 shutdown_connections 2024-03-21T10:34:56.131 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:56.132+0000 7fb10ac00640 1 -- 172.21.15.47:0/3116254294 wait complete. 2024-03-21T10:34:56.133 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:56.133+0000 7fb10ac00640 1 Processor -- start 2024-03-21T10:34:56.133 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:56.133+0000 7fb10ac00640 1 -- start start 2024-03-21T10:34:56.133 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:56.133+0000 7fb10ac00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb104073ad0 0x7fb104199f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:34:56.133 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:56.133+0000 7fb10ac00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb1040743f0 0x7fb10419a4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:34:56.133 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:56.133+0000 7fb10ac00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb10410dd40 0x7fb10419e7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:34:56.133 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:56.133+0000 7fb10ac00640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb104113990 con 0x7fb104073ad0 2024-03-21T10:34:56.133 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:56.133+0000 7fb10ac00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb104113810 con 0x7fb10410dd40 2024-03-21T10:34:56.133 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:56.133+0000 7fb10ac00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb104113b10 con 0x7fb1040743f0 2024-03-21T10:34:56.133 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:56.133+0000 7fb109800640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb104073ad0 0x7fb104199f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:34:56.133 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:56.133+0000 7fb109800640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb104073ad0 0x7fb104199f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:55924/0 (socket says 172.21.15.47:55924) 2024-03-21T10:34:56.134 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:56.133+0000 7fb109800640 1 -- 172.21.15.47:0/603553061 learned_addr learned my addr 172.21.15.47:0/603553061 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:34:56.134 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:56.134+0000 7fb10a200640 1 --2- 172.21.15.47:0/603553061 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb10410dd40 0x7fb10419e7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:34:56.134 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:56.134+0000 7fb108e00640 1 --2- 172.21.15.47:0/603553061 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb1040743f0 0x7fb10419a4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:34:56.134 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:56.134+0000 7fb109800640 1 -- 172.21.15.47:0/603553061 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb1040743f0 msgr2=0x7fb10419a4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:34:56.134 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:56.134+0000 7fb109800640 1 --2- 172.21.15.47:0/603553061 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb1040743f0 0x7fb10419a4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:56.134 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:56.134+0000 7fb109800640 1 -- 172.21.15.47:0/603553061 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb10410dd40 msgr2=0x7fb10419e7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:34:56.134 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:56.134+0000 7fb109800640 1 --2- 172.21.15.47:0/603553061 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb10410dd40 0x7fb10419e7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:56.134 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:56.134+0000 7fb109800640 1 -- 172.21.15.47:0/603553061 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb10419ef20 con 0x7fb104073ad0 2024-03-21T10:34:56.134 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:56.134+0000 7fb10a200640 1 --2- 172.21.15.47:0/603553061 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb10410dd40 0x7fb10419e7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T10:34:56.134 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:56.134+0000 7fb109800640 1 --2- 172.21.15.47:0/603553061 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb104073ad0 0x7fb104199f80 secure :-1 s=READY pgs=1282 cs=0 l=1 rev1=1 crypto rx=0x7fb0f800d950 tx=0x7fb0f800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:34:56.134 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:56.135+0000 7fb102000640 1 -- 172.21.15.47:0/603553061 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0f8017070 con 0x7fb104073ad0 2024-03-21T10:34:56.134 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:56.135+0000 7fb10ac00640 1 -- 172.21.15.47:0/603553061 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb10419f210 con 0x7fb104073ad0 2024-03-21T10:34:56.134 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:56.135+0000 7fb102000640 1 -- 172.21.15.47:0/603553061 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fb0f8009c30 con 0x7fb104073ad0 2024-03-21T10:34:56.134 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:56.135+0000 7fb102000640 1 -- 172.21.15.47:0/603553061 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0f800f620 con 0x7fb104073ad0 2024-03-21T10:34:56.135 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:56.135+0000 7fb10ac00640 1 -- 172.21.15.47:0/603553061 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb1041a6aa0 con 0x7fb104073ad0 2024-03-21T10:34:56.137 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:56.138+0000 7fb10ef76740 1 Processor -- start 2024-03-21T10:34:56.138 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:56.138+0000 7fb10ef76740 1 -- start start 2024-03-21T10:34:56.138 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:56.138+0000 7fb102000640 1 -- 172.21.15.47:0/603553061 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 27) v1 ==== 137985+0+0 (secure 0 0 0) 0x7fb0f80109a0 con 0x7fb104073ad0 2024-03-21T10:34:56.138 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:56.139+0000 7fb10ef76740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559e75f57dd0 0x559e75f56b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:34:56.138 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:56.139+0000 7fb10ef76740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x559e75f570f0 0x559e75f57570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:34:56.138 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:56.139+0000 7fb10ef76740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559e75f60460 0x559e75f62850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:34:56.138 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:56.139+0000 7fb10ef76740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x559e75ebb250 con 0x559e75f570f0 2024-03-21T10:34:56.138 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:56.139+0000 7fb10ef76740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x559e75e63a30 con 0x559e75f60460 2024-03-21T10:34:56.138 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:56.139+0000 7fb10ef76740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x559e75eb72f0 con 0x559e75f57dd0 2024-03-21T10:34:56.138 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:56.139+0000 7fb109800640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x559e75f570f0 0x559e75f57570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:34:56.138 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:56.139+0000 7fb109800640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x559e75f570f0 0x559e75f57570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:55930/0 (socket says 172.21.15.47:55930) 2024-03-21T10:34:56.138 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:56.139+0000 7fb109800640 1 -- 172.21.15.47:0/3231343692 learned_addr learned my addr 172.21.15.47:0/3231343692 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:34:56.138 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:56.139+0000 7fb108e00640 1 --2- 172.21.15.47:0/3231343692 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559e75f60460 0x559e75f62850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:34:56.139 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:56.139+0000 7fb10a200640 1 --2- 172.21.15.47:0/3231343692 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559e75f57dd0 0x559e75f56b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:34:56.139 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:56.140+0000 7fb109800640 1 -- 172.21.15.47:0/3231343692 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559e75f57dd0 msgr2=0x559e75f56b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:34:56.139 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:56.140+0000 7fb109800640 1 --2- 172.21.15.47:0/3231343692 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559e75f57dd0 0x559e75f56b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:56.139 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:56.140+0000 7fb109800640 1 -- 172.21.15.47:0/3231343692 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559e75f60460 msgr2=0x559e75f62850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:34:56.139 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:56.140+0000 7fb109800640 1 --2- 172.21.15.47:0/3231343692 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559e75f60460 0x559e75f62850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:56.139 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:56.140+0000 7fb109800640 1 -- 172.21.15.47:0/3231343692 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559e75e4e690 con 0x559e75f570f0 2024-03-21T10:34:56.139 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:56.140+0000 7fb10a200640 1 --2- 172.21.15.47:0/3231343692 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559e75f57dd0 0x559e75f56b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T10:34:56.139 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:56.140+0000 7fb108e00640 1 --2- 172.21.15.47:0/3231343692 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559e75f60460 0x559e75f62850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T10:34:56.141 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:56.140+0000 7fb109800640 1 --2- 172.21.15.47:0/3231343692 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x559e75f570f0 0x559e75f57570 secure :-1 s=READY pgs=1283 cs=0 l=1 rev1=1 crypto rx=0x7fb0f80ccab0 tx=0x7fb0f80d5ef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c69a951082f9e47a server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:34:56.141 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:56.140+0000 7fb102000640 1 --2- 172.21.15.47:0/603553061 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7fb0ec0a3090 0x7fb0ec0a5550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:34:56.141 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:56.140+0000 7fb0f6a00640 1 -- 172.21.15.47:0/3231343692 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0f80de070 con 0x559e75f570f0 2024-03-21T10:34:56.141 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:56.141+0000 7fb10a200640 1 --2- 172.21.15.47:0/603553061 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7fb0ec0a3090 0x7fb0ec0a5550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:34:56.141 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:56.141+0000 7fb102000640 1 -- 172.21.15.47:0/603553061 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 9562+0+0 (secure 0 0 0) 0x7fb0f80cf890 con 0x7fb104073ad0 2024-03-21T10:34:56.141 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:56.141+0000 7fb0f6a00640 1 -- 172.21.15.47:0/3231343692 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fb0f8009de0 con 0x559e75f570f0 2024-03-21T10:34:56.141 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:56.141+0000 7fb0f6a00640 1 -- 172.21.15.47:0/3231343692 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0f809b170 con 0x559e75f570f0 2024-03-21T10:34:56.141 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:56.142+0000 7fb10ef76740 1 -- 172.21.15.47:0/3231343692 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x559e75f570f0 msgr2=0x559e75f57570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:34:56.141 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:56.142+0000 7fb10a200640 1 --2- 172.21.15.47:0/603553061 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7fb0ec0a3090 0x7fb0ec0a5550 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fb0fc004af0 tx=0x7fb0fc02e040 comp rx=0 tx=0).ready entity=mgr.15687 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:34:56.141 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:56.142+0000 7fb10ef76740 1 --2- 172.21.15.47:0/3231343692 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x559e75f570f0 0x559e75f57570 secure :-1 s=READY pgs=1283 cs=0 l=1 rev1=1 crypto rx=0x7fb0f80ccab0 tx=0x7fb0f80d5ef0 comp rx=0 tx=0).stop 2024-03-21T10:34:56.141 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:56.142+0000 7fb10ef76740 1 -- 172.21.15.47:0/3231343692 shutdown_connections 2024-03-21T10:34:56.141 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:56.142+0000 7fb10ef76740 1 --2- 172.21.15.47:0/3231343692 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559e75f60460 0x559e75f62850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:56.142 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:56.142+0000 7fb10ef76740 1 --2- 172.21.15.47:0/3231343692 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x559e75f570f0 0x559e75f57570 unknown :-1 s=CLOSED pgs=1283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:56.142 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:56.142+0000 7fb10ef76740 1 --2- 172.21.15.47:0/3231343692 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559e75f57dd0 0x559e75f56b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:56.142 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:56.142+0000 7fb10ef76740 1 -- 172.21.15.47:0/3231343692 >> 172.21.15.47:0/3231343692 conn(0x559e75f1e200 msgr2=0x559e75f51b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:34:56.142 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:56.142+0000 7fb10ef76740 1 -- 172.21.15.47:0/3231343692 shutdown_connections 2024-03-21T10:34:56.142 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:56.142+0000 7fb10ef76740 1 -- 172.21.15.47:0/3231343692 wait complete. 2024-03-21T10:34:56.142 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:56.143+0000 7fb10ef76740 1 -- start start 2024-03-21T10:34:56.142 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:56.143+0000 7fb10ef76740 1 Processor -- start 2024-03-21T10:34:56.145 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:56.144+0000 7fb10ef76740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559e75f570f0 0x559e75f5fb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:34:56.145 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:56.144+0000 7fb10ef76740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559e75f57dd0 0x559e75f5cf00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:34:56.145 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:56.144+0000 7fb10ef76740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x559e75f60460 0x559e75f5d440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:34:56.145 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:56.144+0000 7fb10ef76740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x559e75f632d0 con 0x559e75f60460 2024-03-21T10:34:56.145 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:56.144+0000 7fb10ef76740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x559e75e76e80 con 0x559e75f570f0 2024-03-21T10:34:56.145 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:56.144+0000 7fb10ef76740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x559e75f63450 con 0x559e75f57dd0 2024-03-21T10:34:56.146 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:56.144+0000 7fb10ef76740 10 client.0 _ll_register_callbacks cb 0x559e75e89130 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T10:34:56.146 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:56.144+0000 7fb109800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559e75f570f0 0x559e75f5fb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:34:56.146 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:56.144+0000 7fb109800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559e75f570f0 0x559e75f5fb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:46032/0 (socket says 172.21.15.47:46032) 2024-03-21T10:34:56.146 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:56.144+0000 7fb10a200640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x559e75f60460 0x559e75f5d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:34:56.146 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:56.144+0000 7fb109800640 1 -- 172.21.15.47:0/2829906070 learned_addr learned my addr 172.21.15.47:0/2829906070 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:34:56.146 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:56.145+0000 7fb108e00640 1 --2- 172.21.15.47:0/2829906070 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559e75f57dd0 0x559e75f5cf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:34:56.146 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:56.145+0000 7fb109800640 1 -- 172.21.15.47:0/2829906070 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559e75f57dd0 msgr2=0x559e75f5cf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:34:56.146 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:56.145+0000 7fb109800640 1 --2- 172.21.15.47:0/2829906070 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559e75f57dd0 0x559e75f5cf00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:56.146 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:56.145+0000 7fb109800640 1 -- 172.21.15.47:0/2829906070 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x559e75f60460 msgr2=0x559e75f5d440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:34:56.146 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:56.145+0000 7fb109800640 1 --2- 172.21.15.47:0/2829906070 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x559e75f60460 0x559e75f5d440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:56.146 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:56.145+0000 7fb109800640 1 -- 172.21.15.47:0/2829906070 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x559e75c997d0 con 0x559e75f570f0 2024-03-21T10:34:56.146 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:56.145+0000 7fb109800640 1 --2- 172.21.15.47:0/2829906070 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559e75f570f0 0x559e75f5fb70 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7fb0f809b7a0 tx=0x7fb0f80ccae0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fb63b1ccaed3f27 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:34:56.146 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:56.145+0000 7fb0eb400640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-03-21T10:34:56.147 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:56.145+0000 7fb0eb400640 1 -- 172.21.15.47:0/2829906070 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0f8020030 con 0x559e75f570f0 2024-03-21T10:34:56.147 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:56.146+0000 7fb0eb400640 1 -- 172.21.15.47:0/2829906070 <== mon.2 v2:172.21.15.100:3300/0 2 ==== osd_map(103..103 src has 1..103) v4 ==== 9562+0+0 (secure 0 0 0) 0x7fb0f8086070 con 0x559e75f570f0 2024-03-21T10:34:56.147 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:56.146+0000 7fb10ef76740 1 -- 172.21.15.47:0/2829906070 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fb0f8009de0 con 0x559e75f570f0 2024-03-21T10:34:56.147 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:56.146+0000 7fb0eb400640 1 -- 172.21.15.47:0/2829906070 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb0f80afe60 con 0x559e75f570f0 2024-03-21T10:34:56.147 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:56.146+0000 7fb10ef76740 10 client.38456 fetch_fsmap learned FSMap version 32 2024-03-21T10:34:56.147 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:56.146+0000 7fb10ef76740 1 -- 172.21.15.47:0/2829906070 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x559e75bcf3f0 con 0x559e75f570f0 2024-03-21T10:34:56.147 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:56.146+0000 7fb0eb400640 1 -- 172.21.15.47:0/2829906070 <== mon.2 v2:172.21.15.100:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fb0f8009de0 con 0x559e75f570f0 2024-03-21T10:34:56.147 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:56.146+0000 7fb0eb400640 1 -- 172.21.15.47:0/2829906070 <== mon.2 v2:172.21.15.100:3300/0 5 ==== fsmap(e 32) v1 ==== 7942+0+0 (secure 0 0 0) 0x7fb0f80ccdf0 con 0x559e75f570f0 2024-03-21T10:34:56.147 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:56.146+0000 7fb10ef76740 10 client.38456 fetch_fsmap finished waiting for FSMap version 32 2024-03-21T10:34:56.147 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:56.146+0000 7fb10ef76740 10 client.38456 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-21T10:34:56.147 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:56.147+0000 7fb10ef76740 20 client.38456 populate_metadata read hostname 'smithi047' 2024-03-21T10:34:56.147 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:56.147+0000 7fb10ef76740 1 --2- 172.21.15.47:0/2829906070 >> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] conn(0x559e75f78a40 0x559e75f7ae30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:34:56.147 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:56.147+0000 7fb10ef76740 4 client.38456 mds_command: new command op to 34535 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T10:34:56.147 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:56.147+0000 7fb10ef76740 1 -- 172.21.15.47:0/2829906070 --> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x559e75ec8790 con 0x559e75f78a40 2024-03-21T10:34:56.148 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:56.147+0000 7fb108e00640 1 --2- 172.21.15.47:0/2829906070 >> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] conn(0x559e75f78a40 0x559e75f7ae30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:34:56.149 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:56.148+0000 7fb108e00640 1 --2- 172.21.15.47:0/2829906070 >> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] conn(0x559e75f78a40 0x559e75f7ae30 crc :-1 s=READY pgs=219 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=659e2533ccc1cbdd server_cookie=3830a4e92276b72 in_seq=0 out_seq=0 2024-03-21T10:34:56.149 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:56.148+0000 7fb0eb400640 10 client.38456 ms_handle_connect on v2:172.21.15.100:6838/4173168440 2024-03-21T10:34:56.149 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:56.149+0000 7fb0eb400640 1 -- 172.21.15.47:0/2829906070 <== mds.0 v2:172.21.15.100:6838/4173168440 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x559e75ec8790 con 0x559e75f78a40 2024-03-21T10:34:56.149 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:56.149+0000 7fb0eb400640 10 client.38456 handle_command_reply: tid=0 2024-03-21T10:34:56.149 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:56.149+0000 7fb10ef76740 1 client.38456 shutdown 2024-03-21T10:34:56.149 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:56.149+0000 7fb10ef76740 1 -- 172.21.15.47:0/2829906070 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559e75f570f0 msgr2=0x559e75f5fb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:34:56.149 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:56.149+0000 7fb10ef76740 1 --2- 172.21.15.47:0/2829906070 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559e75f570f0 0x559e75f5fb70 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7fb0f809b7a0 tx=0x7fb0f80ccae0 comp rx=0 tx=0).stop 2024-03-21T10:34:56.149 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:56.149+0000 7fb10ef76740 1 -- 172.21.15.47:0/2829906070 shutdown_connections 2024-03-21T10:34:56.150 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:56.149+0000 7fb10ef76740 1 --2- 172.21.15.47:0/2829906070 >> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] conn(0x559e75f78a40 0x559e75f7ae30 crc :-1 s=READY pgs=219 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:56.150 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:56.149+0000 7fb10ef76740 1 --2- 172.21.15.47:0/2829906070 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x559e75f60460 0x559e75f5d440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:56.150 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:56.149+0000 7fb0eb400640 0 client.38456 ms_handle_reset on v2:172.21.15.100:6838/4173168440 2024-03-21T10:34:56.150 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:56.149+0000 7fb10ef76740 1 --2- 172.21.15.47:0/2829906070 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559e75f57dd0 0x559e75f5cf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:56.150 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:56.149+0000 7fb10ef76740 1 --2- 172.21.15.47:0/2829906070 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559e75f570f0 0x559e75f5fb70 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:56.150 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:56.149+0000 7fb10ef76740 1 -- 172.21.15.47:0/2829906070 >> 172.21.15.47:0/2829906070 conn(0x559e75f1e200 msgr2=0x559e75f53890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:34:56.150 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:56.150+0000 7fb10ef76740 1 -- 172.21.15.47:0/2829906070 shutdown_connections 2024-03-21T10:34:56.150 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:56.150+0000 7fb10ef76740 1 -- 172.21.15.47:0/2829906070 wait complete. 2024-03-21T10:34:56.150 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:56.150+0000 7fb10ef76740 20 client.38456 trim_cache size 0 max 16384 2024-03-21T10:34:56.177 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:56.178+0000 7fb10ef76740 1 Processor -- start 2024-03-21T10:34:56.177 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:56.178+0000 7fb10ef76740 1 -- start start 2024-03-21T10:34:56.177 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:56.178+0000 7fb10ef76740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x559e75f570f0 0x559e75f5f630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:34:56.177 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:56.178+0000 7fb10ef76740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559e75f57dd0 0x559e75f5fb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:34:56.177 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:56.178+0000 7fb10ef76740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559e75f60460 0x559e75f61850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:34:56.178 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:56.178+0000 7fb10ef76740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1041ac650 con 0x559e75f570f0 2024-03-21T10:34:56.178 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:56.178+0000 7fb10ef76740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1041adfd0 con 0x559e75f57dd0 2024-03-21T10:34:56.178 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:56.178+0000 7fb10ef76740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1041accb0 con 0x559e75f60460 2024-03-21T10:34:56.178 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:56.179+0000 7fb109800640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x559e75f570f0 0x559e75f5f630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:34:56.178 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:56.179+0000 7fb109800640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x559e75f570f0 0x559e75f5f630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:55944/0 (socket says 172.21.15.47:55944) 2024-03-21T10:34:56.178 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:56.179+0000 7fb109800640 1 -- 172.21.15.47:0/969646985 learned_addr learned my addr 172.21.15.47:0/969646985 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:34:56.178 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:56.179+0000 7fb108e00640 1 --2- 172.21.15.47:0/969646985 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559e75f57dd0 0x559e75f5fb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:34:56.178 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:56.179+0000 7fb10a200640 1 --2- 172.21.15.47:0/969646985 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559e75f60460 0x559e75f61850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:34:56.178 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:56.179+0000 7fb109800640 1 -- 172.21.15.47:0/969646985 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559e75f60460 msgr2=0x559e75f61850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:34:56.178 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:56.179+0000 7fb109800640 1 --2- 172.21.15.47:0/969646985 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559e75f60460 0x559e75f61850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:56.178 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:56.179+0000 7fb109800640 1 -- 172.21.15.47:0/969646985 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559e75f57dd0 msgr2=0x559e75f5fb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:34:56.178 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:56.179+0000 7fb109800640 1 --2- 172.21.15.47:0/969646985 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559e75f57dd0 0x559e75f5fb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:56.178 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:56.179+0000 7fb109800640 1 -- 172.21.15.47:0/969646985 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559e75b8f0f0 con 0x559e75f570f0 2024-03-21T10:34:56.178 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:56.179+0000 7fb109800640 1 --2- 172.21.15.47:0/969646985 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x559e75f570f0 0x559e75f5f630 secure :-1 s=READY pgs=1284 cs=0 l=1 rev1=1 crypto rx=0x7fb0f8010cf0 tx=0x7fb0f8099d20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d4d702598c5de52e server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:34:56.182 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:56.180+0000 7fb0f6000640 1 -- 172.21.15.47:0/969646985 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0f8020070 con 0x559e75f570f0 2024-03-21T10:34:56.182 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:56.180+0000 7fb0f6000640 1 -- 172.21.15.47:0/969646985 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fb0f80d7730 con 0x559e75f570f0 2024-03-21T10:34:56.182 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:56.180+0000 7fb0f6000640 1 -- 172.21.15.47:0/969646985 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0f80de070 con 0x559e75f570f0 2024-03-21T10:34:56.182 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:56.180+0000 7fb10ef76740 1 -- 172.21.15.47:0/969646985 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x559e75f570f0 msgr2=0x559e75f5f630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:34:56.182 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:56.180+0000 7fb10ef76740 1 --2- 172.21.15.47:0/969646985 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x559e75f570f0 0x559e75f5f630 secure :-1 s=READY pgs=1284 cs=0 l=1 rev1=1 crypto rx=0x7fb0f8010cf0 tx=0x7fb0f8099d20 comp rx=0 tx=0).stop 2024-03-21T10:34:56.183 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:56.181+0000 7fb10ef76740 1 -- 172.21.15.47:0/969646985 shutdown_connections 2024-03-21T10:34:56.183 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:56.181+0000 7fb10ef76740 1 --2- 172.21.15.47:0/969646985 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559e75f60460 0x559e75f61850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:56.183 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:56.181+0000 7fb10ef76740 1 --2- 172.21.15.47:0/969646985 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559e75f57dd0 0x559e75f5fb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:56.183 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:56.181+0000 7fb10ef76740 1 --2- 172.21.15.47:0/969646985 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x559e75f570f0 0x559e75f5f630 unknown :-1 s=CLOSED pgs=1284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:56.183 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:56.181+0000 7fb10ef76740 1 -- 172.21.15.47:0/969646985 >> 172.21.15.47:0/969646985 conn(0x559e75f1e200 msgr2=0x559e75f51670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:34:56.183 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:56.181+0000 7fb10ef76740 1 -- 172.21.15.47:0/969646985 shutdown_connections 2024-03-21T10:34:56.183 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:56.181+0000 7fb10ef76740 1 -- 172.21.15.47:0/969646985 wait complete. 2024-03-21T10:34:56.183 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:56.181+0000 7fb10ef76740 1 -- start start 2024-03-21T10:34:56.183 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:56.181+0000 7fb10ef76740 1 Processor -- start 2024-03-21T10:34:56.183 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:56.182+0000 7fb10ef76740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559e75f570f0 0x559e75f59310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:34:56.183 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:56.182+0000 7fb10ef76740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559e75f57dd0 0x559e75f59850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:34:56.183 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:56.182+0000 7fb10ef76740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x559e75f60460 0x559e75f59d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:34:56.183 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:56.182+0000 7fb10ef76740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1041ad970 con 0x559e75f60460 2024-03-21T10:34:56.183 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:56.182+0000 7fb10ef76740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1041ad310 con 0x559e75f570f0 2024-03-21T10:34:56.183 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:56.182+0000 7fb10ef76740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1041aa830 con 0x559e75f57dd0 2024-03-21T10:34:56.184 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:56.182+0000 7fb10ef76740 10 client.0 _ll_register_callbacks cb 0x7fb0f0009470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T10:34:56.184 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:56.182+0000 7fb10a200640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x559e75f60460 0x559e75f59d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:34:56.184 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:56.182+0000 7fb10a200640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x559e75f60460 0x559e75f59d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:55952/0 (socket says 172.21.15.47:55952) 2024-03-21T10:34:56.184 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:56.182+0000 7fb10a200640 1 -- 172.21.15.47:0/10195745 learned_addr learned my addr 172.21.15.47:0/10195745 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:34:56.184 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:56.182+0000 7fb109800640 1 --2- 172.21.15.47:0/10195745 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559e75f570f0 0x559e75f59310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:34:56.184 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:56.182+0000 7fb108e00640 1 --2- 172.21.15.47:0/10195745 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559e75f57dd0 0x559e75f59850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:34:56.184 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:56.183+0000 7fb10a200640 1 -- 172.21.15.47:0/10195745 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559e75f57dd0 msgr2=0x559e75f59850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:34:56.184 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:56.183+0000 7fb10a200640 1 --2- 172.21.15.47:0/10195745 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559e75f57dd0 0x559e75f59850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:56.184 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:56.183+0000 7fb10a200640 1 -- 172.21.15.47:0/10195745 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559e75f570f0 msgr2=0x559e75f59310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:34:56.184 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:56.183+0000 7fb10a200640 1 --2- 172.21.15.47:0/10195745 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559e75f570f0 0x559e75f59310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:56.184 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:56.183+0000 7fb10a200640 1 -- 172.21.15.47:0/10195745 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x559e75f5aee0 con 0x559e75f60460 2024-03-21T10:34:56.184 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:56.183+0000 7fb10a200640 1 --2- 172.21.15.47:0/10195745 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x559e75f60460 0x559e75f59d90 secure :-1 s=READY pgs=1285 cs=0 l=1 rev1=1 crypto rx=0x7fb0fc009ca0 tx=0x7fb0fc02c190 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a60155a7ab3a3063 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:34:56.184 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:56.183+0000 7fb0eb400640 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-03-21T10:34:56.184 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:56.183+0000 7fb0eb400640 1 -- 172.21.15.47:0/10195745 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0fc039040 con 0x559e75f60460 2024-03-21T10:34:56.185 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:56.183+0000 7fb10ef76740 1 -- 172.21.15.47:0/10195745 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fb0f80d7730 con 0x559e75f60460 2024-03-21T10:34:56.185 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:56.183+0000 7fb0eb400640 1 -- 172.21.15.47:0/10195745 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(103..103 src has 1..103) v4 ==== 9562+0+0 (secure 0 0 0) 0x7fb0fc032ac0 con 0x559e75f60460 2024-03-21T10:34:56.185 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:56.183+0000 7fb0eb400640 1 -- 172.21.15.47:0/10195745 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb0fc05d050 con 0x559e75f60460 2024-03-21T10:34:56.185 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:56.184+0000 7fb10ef76740 10 client.18708 fetch_fsmap learned FSMap version 32 2024-03-21T10:34:56.185 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:56.184+0000 7fb10ef76740 1 -- 172.21.15.47:0/10195745 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x559e75fd3fa0 con 0x559e75f60460 2024-03-21T10:34:56.185 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:56.184+0000 7fb0eb400640 1 -- 172.21.15.47:0/10195745 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fb0f80d7730 con 0x559e75f60460 2024-03-21T10:34:56.185 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:56.184+0000 7fb0eb400640 1 -- 172.21.15.47:0/10195745 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 32) v1 ==== 7942+0+0 (secure 0 0 0) 0x7fb0fc04e420 con 0x559e75f60460 2024-03-21T10:34:56.185 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:56.184+0000 7fb10ef76740 10 client.18708 fetch_fsmap finished waiting for FSMap version 32 2024-03-21T10:34:56.185 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:56.184+0000 7fb10ef76740 10 client.18708 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-21T10:34:56.185 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:56.184+0000 7fb10ef76740 20 client.18708 populate_metadata read hostname 'smithi047' 2024-03-21T10:34:56.185 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:56.184+0000 7fb10ef76740 1 --2- 172.21.15.47:0/10195745 >> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] conn(0x559e75fd46c0 0x559e75fd6ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:34:56.185 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:56.184+0000 7fb10ef76740 4 client.18708 mds_command: new command op to 34535 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-21T10:34:56.185 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:56.184+0000 7fb10ef76740 1 -- 172.21.15.47:0/10195745 --> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x559e75ec2ae0 con 0x559e75fd46c0 2024-03-21T10:34:56.187 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:56.185+0000 7fb109800640 1 --2- 172.21.15.47:0/10195745 >> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] conn(0x559e75fd46c0 0x559e75fd6ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:34:56.187 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:56.185+0000 7fb109800640 1 --2- 172.21.15.47:0/10195745 >> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] conn(0x559e75fd46c0 0x559e75fd6ae0 crc :-1 s=READY pgs=220 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=32cfc1367fed74ac server_cookie=462122c6ba60ef69 in_seq=0 out_seq=0 2024-03-21T10:34:56.187 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:56.185+0000 7fb0eb400640 10 client.18708 ms_handle_connect on v2:172.21.15.100:6838/4173168440 2024-03-21T10:34:56.187 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:56.186+0000 7fb0eb400640 1 -- 172.21.15.47:0/10195745 <== mds.0 v2:172.21.15.100:6838/4173168440 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x559e75ec2ae0 con 0x559e75fd46c0 2024-03-21T10:34:56.187 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:56.186+0000 7fb0eb400640 10 client.18708 handle_command_reply: tid=0 2024-03-21T10:34:56.187 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:56.186+0000 7fb10ef76740 1 client.18708 shutdown 2024-03-21T10:34:56.187 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:56.186+0000 7fb10ef76740 1 -- 172.21.15.47:0/10195745 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x559e75f60460 msgr2=0x559e75f59d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:34:56.187 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:56.186+0000 7fb10ef76740 1 --2- 172.21.15.47:0/10195745 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x559e75f60460 0x559e75f59d90 secure :-1 s=READY pgs=1285 cs=0 l=1 rev1=1 crypto rx=0x7fb0fc009ca0 tx=0x7fb0fc02c190 comp rx=0 tx=0).stop 2024-03-21T10:34:56.188 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:56.186+0000 7fb10ef76740 1 -- 172.21.15.47:0/10195745 shutdown_connections 2024-03-21T10:34:56.188 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:56.186+0000 7fb10ef76740 1 --2- 172.21.15.47:0/10195745 >> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] conn(0x559e75fd46c0 0x559e75fd6ae0 crc :-1 s=READY pgs=220 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:56.188 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:56.186+0000 7fb10ef76740 1 --2- 172.21.15.47:0/10195745 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x559e75f60460 0x559e75f59d90 unknown :-1 s=CLOSED pgs=1285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:56.188 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:56.186+0000 7fb10ef76740 1 --2- 172.21.15.47:0/10195745 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x559e75f57dd0 0x559e75f59850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:56.188 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:56.186+0000 7fb10ef76740 1 --2- 172.21.15.47:0/10195745 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559e75f570f0 0x559e75f59310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:56.188 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:56.186+0000 7fb0eb400640 0 client.18708 ms_handle_reset on v2:172.21.15.100:6838/4173168440 2024-03-21T10:34:56.188 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:56.186+0000 7fb10ef76740 1 -- 172.21.15.47:0/10195745 >> 172.21.15.47:0/10195745 conn(0x559e75f1e200 msgr2=0x559e75f5b810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:34:56.188 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:56.186+0000 7fb10ef76740 1 -- 172.21.15.47:0/10195745 shutdown_connections 2024-03-21T10:34:56.188 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:56.186+0000 7fb10ef76740 1 -- 172.21.15.47:0/10195745 wait complete. 2024-03-21T10:34:56.188 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:56.187+0000 7fb10ef76740 20 client.18708 trim_cache size 0 max 16384 2024-03-21T10:34:56.189 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-03-21T10:34:56.189 INFO:teuthology.orchestra.run.smithi047.stdout: "status": "IDLE (waiting for more scrubs)", 2024-03-21T10:34:56.189 INFO:teuthology.orchestra.run.smithi047.stdout: "scrubs": { 2024-03-21T10:34:56.189 INFO:teuthology.orchestra.run.smithi047.stdout: "a013be24-92aa-4d12-9eed-493ef8447c26": { 2024-03-21T10:34:56.189 INFO:teuthology.orchestra.run.smithi047.stdout: "path": "/", 2024-03-21T10:34:56.189 INFO:teuthology.orchestra.run.smithi047.stdout: "tag": "a013be24-92aa-4d12-9eed-493ef8447c26", 2024-03-21T10:34:56.189 INFO:teuthology.orchestra.run.smithi047.stdout: "options": "recursive,force" 2024-03-21T10:34:56.189 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-21T10:34:56.189 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-21T10:34:56.189 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-03-21T10:34:56.190 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:56.187+0000 7fb10ac00640 1 -- 172.21.15.47:0/603553061 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7fb0ec0a3090 msgr2=0x7fb0ec0a5550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:34:56.190 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:56.187+0000 7fb10ac00640 1 --2- 172.21.15.47:0/603553061 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7fb0ec0a3090 0x7fb0ec0a5550 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fb0fc004af0 tx=0x7fb0fc02e040 comp rx=0 tx=0).stop 2024-03-21T10:34:56.190 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:56.187+0000 7fb10ac00640 1 -- 172.21.15.47:0/603553061 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb104073ad0 msgr2=0x7fb104199f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:34:56.191 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:56.187+0000 7fb10ac00640 1 --2- 172.21.15.47:0/603553061 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb104073ad0 0x7fb104199f80 secure :-1 s=READY pgs=1282 cs=0 l=1 rev1=1 crypto rx=0x7fb0f800d950 tx=0x7fb0f800de10 comp rx=0 tx=0).stop 2024-03-21T10:34:56.191 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:56.187+0000 7fb10ac00640 1 -- 172.21.15.47:0/603553061 shutdown_connections 2024-03-21T10:34:56.191 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:56.187+0000 7fb10ac00640 1 --2- 172.21.15.47:0/603553061 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7fb0ec0a3090 0x7fb0ec0a5550 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:56.191 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:56.187+0000 7fb10ac00640 1 --2- 172.21.15.47:0/603553061 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb10410dd40 0x7fb10419e7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:56.191 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:56.187+0000 7fb10ac00640 1 --2- 172.21.15.47:0/603553061 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb1040743f0 0x7fb10419a4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:56.191 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:56.187+0000 7fb10ac00640 1 --2- 172.21.15.47:0/603553061 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb104073ad0 0x7fb104199f80 unknown :-1 s=CLOSED pgs=1282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:56.192 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:56.187+0000 7fb10ac00640 1 -- 172.21.15.47:0/603553061 >> 172.21.15.47:0/603553061 conn(0x7fb1040fd010 msgr2=0x7fb1040fde60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:34:56.192 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:56.187+0000 7fb10ac00640 1 -- 172.21.15.47:0/603553061 shutdown_connections 2024-03-21T10:34:56.192 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:34:56.187+0000 7fb10ac00640 1 -- 172.21.15.47:0/603553061 wait complete. 2024-03-21T10:34:56.205 INFO:tasks.cephfs.filesystem:scrub status for tag:a013be24-92aa-4d12-9eed-493ef8447c26 - {'path': '/', 'tag': 'a013be24-92aa-4d12-9eed-493ef8447c26', 'options': 'recursive,force'} 2024-03-21T10:34:57.711 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:34:57 smithi047 ceph-mon[31547]: scrub queued for path: / 2024-03-21T10:34:57.711 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:34:57 smithi047 ceph-mon[31547]: scrub summary: idle+waiting paths [/] 2024-03-21T10:34:57.712 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:34:57 smithi047 ceph-mon[31547]: scrub summary: active paths [/] 2024-03-21T10:34:57.712 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:34:57 smithi047 ceph-mon[31547]: scrub summary: idle+waiting paths [/] 2024-03-21T10:34:57.712 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:34:57 smithi047 ceph-mon[31547]: pgmap v298: 129 pgs: 129 active+clean; 2.5 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 12 op/s 2024-03-21T10:34:57.727 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:34:57 smithi100 ceph-mon[35193]: scrub queued for path: / 2024-03-21T10:34:57.727 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:34:57 smithi100 ceph-mon[35193]: scrub summary: idle+waiting paths [/] 2024-03-21T10:34:57.727 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:34:57 smithi100 ceph-mon[35193]: scrub summary: active paths [/] 2024-03-21T10:34:57.727 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:34:57 smithi100 ceph-mon[35193]: scrub summary: idle+waiting paths [/] 2024-03-21T10:34:57.727 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:34:57 smithi100 ceph-mon[35193]: pgmap v298: 129 pgs: 129 active+clean; 2.5 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 12 op/s 2024-03-21T10:34:57.737 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:34:57 smithi157 ceph-mon[35581]: scrub queued for path: / 2024-03-21T10:34:57.737 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:34:57 smithi157 ceph-mon[35581]: scrub summary: idle+waiting paths [/] 2024-03-21T10:34:57.737 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:34:57 smithi157 ceph-mon[35581]: scrub summary: active paths [/] 2024-03-21T10:34:57.737 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:34:57 smithi157 ceph-mon[35581]: scrub summary: idle+waiting paths [/] 2024-03-21T10:34:57.737 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:34:57 smithi157 ceph-mon[35581]: pgmap v298: 129 pgs: 129 active+clean; 2.5 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 12 op/s 2024-03-21T10:34:59.711 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:34:59 smithi047 ceph-mon[31547]: pgmap v299: 129 pgs: 129 active+clean; 2.5 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 9 op/s 2024-03-21T10:34:59.711 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:34:59 smithi047 ceph-mon[31547]: scrub summary: idle 2024-03-21T10:34:59.726 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:34:59 smithi100 ceph-mon[35193]: pgmap v299: 129 pgs: 129 active+clean; 2.5 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 9 op/s 2024-03-21T10:34:59.726 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:34:59 smithi100 ceph-mon[35193]: scrub summary: idle 2024-03-21T10:34:59.736 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:34:59 smithi157 ceph-mon[35581]: pgmap v299: 129 pgs: 129 active+clean; 2.5 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 9 op/s 2024-03-21T10:34:59.736 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:34:59 smithi157 ceph-mon[35581]: scrub summary: idle 2024-03-21T10:35:01.711 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:35:01 smithi047 ceph-mon[31547]: pgmap v300: 129 pgs: 129 active+clean; 2.5 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 9 op/s 2024-03-21T10:35:01.726 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:35:01 smithi100 ceph-mon[35193]: pgmap v300: 129 pgs: 129 active+clean; 2.5 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 9 op/s 2024-03-21T10:35:01.736 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:35:01 smithi157 ceph-mon[35581]: pgmap v300: 129 pgs: 129 active+clean; 2.5 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 9 op/s 2024-03-21T10:35:03.711 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:35:03 smithi047 ceph-mon[31547]: pgmap v301: 129 pgs: 129 active+clean; 2.5 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 44 KiB/s wr, 30 op/s 2024-03-21T10:35:03.726 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:35:03 smithi100 ceph-mon[35193]: pgmap v301: 129 pgs: 129 active+clean; 2.5 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 44 KiB/s wr, 30 op/s 2024-03-21T10:35:03.736 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:35:03 smithi157 ceph-mon[35581]: pgmap v301: 129 pgs: 129 active+clean; 2.5 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 44 KiB/s wr, 30 op/s 2024-03-21T10:35:05.711 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:35:05 smithi047 ceph-mon[31547]: pgmap v302: 129 pgs: 129 active+clean; 2.5 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 44 KiB/s wr, 26 op/s 2024-03-21T10:35:05.726 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:35:05 smithi100 ceph-mon[35193]: pgmap v302: 129 pgs: 129 active+clean; 2.5 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 44 KiB/s wr, 26 op/s 2024-03-21T10:35:05.736 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:35:05 smithi157 ceph-mon[35581]: pgmap v302: 129 pgs: 129 active+clean; 2.5 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 44 KiB/s wr, 26 op/s 2024-03-21T10:35:07.711 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:35:07 smithi047 ceph-mon[31547]: pgmap v303: 129 pgs: 129 active+clean; 2.5 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 44 KiB/s wr, 30 op/s 2024-03-21T10:35:07.726 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:35:07 smithi100 ceph-mon[35193]: pgmap v303: 129 pgs: 129 active+clean; 2.5 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 44 KiB/s wr, 30 op/s 2024-03-21T10:35:07.737 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:35:07 smithi157 ceph-mon[35581]: pgmap v303: 129 pgs: 129 active+clean; 2.5 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 44 KiB/s wr, 30 op/s 2024-03-21T10:35:09.711 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:35:09 smithi047 ceph-mon[31547]: pgmap v304: 129 pgs: 129 active+clean; 2.5 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 44 KiB/s wr, 26 op/s 2024-03-21T10:35:09.726 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:35:09 smithi100 ceph-mon[35193]: pgmap v304: 129 pgs: 129 active+clean; 2.5 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 44 KiB/s wr, 26 op/s 2024-03-21T10:35:09.736 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:35:09 smithi157 ceph-mon[35581]: pgmap v304: 129 pgs: 129 active+clean; 2.5 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 44 KiB/s wr, 26 op/s 2024-03-21T10:35:10.711 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:35:10 smithi047 ceph-mon[31547]: osdmap e104: 12 total, 12 up, 12 in 2024-03-21T10:35:10.726 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:35:10 smithi100 ceph-mon[35193]: osdmap e104: 12 total, 12 up, 12 in 2024-03-21T10:35:10.736 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:35:10 smithi157 ceph-mon[35581]: osdmap e104: 12 total, 12 up, 12 in 2024-03-21T10:35:11.711 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:35:11 smithi047 ceph-mon[31547]: pgmap v306: 129 pgs: 129 active+clean; 2.5 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 53 KiB/s wr, 30 op/s 2024-03-21T10:35:11.726 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:35:11 smithi100 ceph-mon[35193]: pgmap v306: 129 pgs: 129 active+clean; 2.5 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 53 KiB/s wr, 30 op/s 2024-03-21T10:35:11.737 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:35:11 smithi157 ceph-mon[35581]: pgmap v306: 129 pgs: 129 active+clean; 2.5 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 53 KiB/s wr, 30 op/s 2024-03-21T10:35:13.710 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:35:13 smithi047 ceph-mon[31547]: pgmap v307: 129 pgs: 129 active+clean; 2.5 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 204 B/s wr, 10 op/s 2024-03-21T10:35:13.726 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:35:13 smithi100 ceph-mon[35193]: pgmap v307: 129 pgs: 129 active+clean; 2.5 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 204 B/s wr, 10 op/s 2024-03-21T10:35:13.736 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:35:13 smithi157 ceph-mon[35581]: pgmap v307: 129 pgs: 129 active+clean; 2.5 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 204 B/s wr, 10 op/s 2024-03-21T10:35:15.211 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:35:14 smithi047 ceph-mon[31547]: osdmap e105: 12 total, 12 up, 12 in 2024-03-21T10:35:15.211 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:35:14 smithi047 ceph-mon[31547]: pgmap v309: 129 pgs: 129 active+clean; 2.5 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 127 B/s wr, 7 op/s 2024-03-21T10:35:15.226 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:35:14 smithi100 ceph-mon[35193]: osdmap e105: 12 total, 12 up, 12 in 2024-03-21T10:35:15.226 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:35:14 smithi100 ceph-mon[35193]: pgmap v309: 129 pgs: 129 active+clean; 2.5 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 127 B/s wr, 7 op/s 2024-03-21T10:35:15.236 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:35:14 smithi157 ceph-mon[35581]: osdmap e105: 12 total, 12 up, 12 in 2024-03-21T10:35:15.237 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:35:14 smithi157 ceph-mon[35581]: pgmap v309: 129 pgs: 129 active+clean; 2.5 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 127 B/s wr, 7 op/s 2024-03-21T10:35:17.711 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:35:17 smithi047 ceph-mon[31547]: pgmap v310: 129 pgs: 129 active+clean; 2.5 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 255 B/s wr, 12 op/s 2024-03-21T10:35:17.726 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:35:17 smithi100 ceph-mon[35193]: pgmap v310: 129 pgs: 129 active+clean; 2.5 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 255 B/s wr, 12 op/s 2024-03-21T10:35:17.736 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:35:17 smithi157 ceph-mon[35581]: pgmap v310: 129 pgs: 129 active+clean; 2.5 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 255 B/s wr, 12 op/s 2024-03-21T10:35:19.711 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:35:19 smithi047 ceph-mon[31547]: pgmap v311: 129 pgs: 129 active+clean; 2.5 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 232 B/s wr, 11 op/s 2024-03-21T10:35:19.726 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:35:19 smithi100 ceph-mon[35193]: pgmap v311: 129 pgs: 129 active+clean; 2.5 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 232 B/s wr, 11 op/s 2024-03-21T10:35:19.736 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:35:19 smithi157 ceph-mon[35581]: pgmap v311: 129 pgs: 129 active+clean; 2.5 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 232 B/s wr, 11 op/s 2024-03-21T10:35:21.711 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:35:21 smithi047 ceph-mon[31547]: pgmap v312: 129 pgs: 129 active+clean; 2.5 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-03-21T10:35:21.726 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:35:21 smithi100 ceph-mon[35193]: pgmap v312: 129 pgs: 129 active+clean; 2.5 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-03-21T10:35:21.736 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:35:21 smithi157 ceph-mon[35581]: pgmap v312: 129 pgs: 129 active+clean; 2.5 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-03-21T10:35:22.240 DEBUG:teuthology.orchestra.run.smithi047:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule status --fs=cephfs --path=/' 2024-03-21T10:35:22.457 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:22.459+0000 7f3e7ca00640 1 -- 172.21.15.47:0/1995969215 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3e78100fa0 msgr2=0x7f3e78104fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:35:22.458 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:22.459+0000 7f3e7ca00640 1 --2- 172.21.15.47:0/1995969215 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3e78100fa0 0x7f3e78104fe0 secure :-1 s=READY pgs=1286 cs=0 l=1 rev1=1 crypto rx=0x7f3e60009900 tx=0x7f3e6002bc30 comp rx=0 tx=0).stop 2024-03-21T10:35:22.458 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:22.459+0000 7f3e7ca00640 1 -- 172.21.15.47:0/1995969215 shutdown_connections 2024-03-21T10:35:22.458 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:22.459+0000 7f3e7ca00640 1 --2- 172.21.15.47:0/1995969215 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3e781056c0 0x7f3e7810d290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:35:22.458 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:22.459+0000 7f3e7ca00640 1 --2- 172.21.15.47:0/1995969215 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3e78100fa0 0x7f3e78104fe0 unknown :-1 s=CLOSED pgs=1286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:35:22.458 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:22.459+0000 7f3e7ca00640 1 --2- 172.21.15.47:0/1995969215 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3e781005f0 0x7f3e781009d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:35:22.458 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:22.459+0000 7f3e7ca00640 1 -- 172.21.15.47:0/1995969215 >> 172.21.15.47:0/1995969215 conn(0x7f3e780fbeb0 msgr2=0x7f3e780fe2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:35:22.458 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:22.459+0000 7f3e7ca00640 1 -- 172.21.15.47:0/1995969215 shutdown_connections 2024-03-21T10:35:22.458 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:22.459+0000 7f3e7ca00640 1 -- 172.21.15.47:0/1995969215 wait complete. 2024-03-21T10:35:22.460 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:22.460+0000 7f3e7ca00640 1 Processor -- start 2024-03-21T10:35:22.460 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:22.461+0000 7f3e7ca00640 1 -- start start 2024-03-21T10:35:22.460 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:22.461+0000 7f3e7ca00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3e781005f0 0x7f3e7819da50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:35:22.460 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:22.461+0000 7f3e7ca00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3e781056c0 0x7f3e7819df90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:35:22.460 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:22.461+0000 7f3e7ca00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3e78197b10 0x7f3e78197f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:35:22.460 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:22.461+0000 7f3e7ca00640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e78110650 con 0x7f3e781056c0 2024-03-21T10:35:22.460 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:22.461+0000 7f3e7ca00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e781104d0 con 0x7f3e781005f0 2024-03-21T10:35:22.461 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:22.461+0000 7f3e7ca00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e781107d0 con 0x7f3e78197b10 2024-03-21T10:35:22.461 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:22.461+0000 7f3e75e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3e781056c0 0x7f3e7819df90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:35:22.461 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:22.461+0000 7f3e75e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3e781056c0 0x7f3e7819df90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:58598/0 (socket says 172.21.15.47:58598) 2024-03-21T10:35:22.461 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:22.461+0000 7f3e75e00640 1 -- 172.21.15.47:0/2604829027 learned_addr learned my addr 172.21.15.47:0/2604829027 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:35:22.461 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:22.462+0000 7f3e76800640 1 --2- 172.21.15.47:0/2604829027 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3e781005f0 0x7f3e7819da50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:35:22.461 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:22.462+0000 7f3e77200640 1 --2- 172.21.15.47:0/2604829027 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3e78197b10 0x7f3e78197f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:35:22.461 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:22.462+0000 7f3e75e00640 1 -- 172.21.15.47:0/2604829027 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3e78197b10 msgr2=0x7f3e78197f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:35:22.461 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:22.462+0000 7f3e75e00640 1 --2- 172.21.15.47:0/2604829027 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3e78197b10 0x7f3e78197f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:35:22.461 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:22.462+0000 7f3e75e00640 1 -- 172.21.15.47:0/2604829027 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3e781005f0 msgr2=0x7f3e7819da50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:35:22.461 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:22.462+0000 7f3e75e00640 1 --2- 172.21.15.47:0/2604829027 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3e781005f0 0x7f3e7819da50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:35:22.461 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:22.462+0000 7f3e75e00640 1 -- 172.21.15.47:0/2604829027 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e78198710 con 0x7f3e781056c0 2024-03-21T10:35:22.461 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:22.462+0000 7f3e76800640 1 --2- 172.21.15.47:0/2604829027 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3e781005f0 0x7f3e7819da50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T10:35:22.461 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:22.462+0000 7f3e75e00640 1 --2- 172.21.15.47:0/2604829027 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3e781056c0 0x7f3e7819df90 secure :-1 s=READY pgs=1287 cs=0 l=1 rev1=1 crypto rx=0x7f3e60000c00 tx=0x7f3e6002c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:35:22.462 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:22.463+0000 7f3e6f400640 1 -- 172.21.15.47:0/2604829027 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e60043020 con 0x7f3e781056c0 2024-03-21T10:35:22.463 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:22.463+0000 7f3e6f400640 1 -- 172.21.15.47:0/2604829027 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f3e6002cc60 con 0x7f3e781056c0 2024-03-21T10:35:22.463 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:22.463+0000 7f3e6f400640 1 -- 172.21.15.47:0/2604829027 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e60003c30 con 0x7f3e781056c0 2024-03-21T10:35:22.463 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:22.463+0000 7f3e7ca00640 1 -- 172.21.15.47:0/2604829027 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3e781989a0 con 0x7f3e781056c0 2024-03-21T10:35:22.463 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:22.463+0000 7f3e7ca00640 1 -- 172.21.15.47:0/2604829027 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3e781a4af0 con 0x7f3e781056c0 2024-03-21T10:35:22.463 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:22.465+0000 7f3e7ca00640 1 -- 172.21.15.47:0/2604829027 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3e78101ac0 con 0x7f3e781056c0 2024-03-21T10:35:22.464 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:22.466+0000 7f3e6f400640 1 -- 172.21.15.47:0/2604829027 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 27) v1 ==== 137985+0+0 (secure 0 0 0) 0x7f3e60046020 con 0x7f3e781056c0 2024-03-21T10:35:22.466 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:22.467+0000 7f3e6f400640 1 --2- 172.21.15.47:0/2604829027 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f3e5c0a3080 0x7f3e5c0a5540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:35:22.466 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:22.468+0000 7f3e6f400640 1 -- 172.21.15.47:0/2604829027 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(105..105 src has 1..105) v4 ==== 9562+0+0 (secure 0 0 0) 0x7f3e600f1eb0 con 0x7f3e781056c0 2024-03-21T10:35:22.466 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:22.468+0000 7f3e76800640 1 --2- 172.21.15.47:0/2604829027 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f3e5c0a3080 0x7f3e5c0a5540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:35:22.466 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:22.469+0000 7f3e76800640 1 --2- 172.21.15.47:0/2604829027 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f3e5c0a3080 0x7f3e5c0a5540 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f3e68005fd0 tx=0x7f3e68009520 comp rx=0 tx=0).ready entity=mgr.15687 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:35:22.470 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:22.473+0000 7f3e6f400640 1 -- 172.21.15.47:0/2604829027 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f3e6004d020 con 0x7f3e781056c0 2024-03-21T10:35:22.814 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:22.816+0000 7f3e7ca00640 1 -- 172.21.15.47:0/2604829027 --> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}) v1 -- 0x7f3e78100fa0 con 0x7f3e5c0a3080 2024-03-21T10:35:22.830 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:22.832+0000 7f3e6f400640 1 -- 172.21.15.47:0/2604829027 <== mgr.15687 v2:172.21.15.47:6800/196172753 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+350 (secure 0 0 0) 0x7f3e78100fa0 con 0x7f3e5c0a3080 2024-03-21T10:35:22.830 INFO:teuthology.orchestra.run.smithi047.stdout:{"fs": "cephfs", "subvol": null, "group": null, "path": "/", "rel_path": "/", "schedule": "1m", "retention": {"m": 6, "h": 3}, "start": "2024-03-21T00:00:00", "created": "2024-03-21T10:25:20", "first": "2024-03-21T10:26:00", "last": "2024-03-21T10:35:00", "last_pruned": "2024-03-21T10:35:00", "created_count": 10, "pruned_count": 4, "active": true} 2024-03-21T10:35:22.833 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:22.834+0000 7f3e7ca00640 1 -- 172.21.15.47:0/2604829027 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f3e5c0a3080 msgr2=0x7f3e5c0a5540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:35:22.833 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:22.834+0000 7f3e7ca00640 1 --2- 172.21.15.47:0/2604829027 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f3e5c0a3080 0x7f3e5c0a5540 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f3e68005fd0 tx=0x7f3e68009520 comp rx=0 tx=0).stop 2024-03-21T10:35:22.833 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:22.834+0000 7f3e7ca00640 1 -- 172.21.15.47:0/2604829027 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3e781056c0 msgr2=0x7f3e7819df90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:35:22.833 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:22.834+0000 7f3e7ca00640 1 --2- 172.21.15.47:0/2604829027 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3e781056c0 0x7f3e7819df90 secure :-1 s=READY pgs=1287 cs=0 l=1 rev1=1 crypto rx=0x7f3e60000c00 tx=0x7f3e6002c860 comp rx=0 tx=0).stop 2024-03-21T10:35:22.833 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:22.834+0000 7f3e7ca00640 1 -- 172.21.15.47:0/2604829027 shutdown_connections 2024-03-21T10:35:22.833 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:22.834+0000 7f3e7ca00640 1 --2- 172.21.15.47:0/2604829027 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3e78197b10 0x7f3e78197f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:35:22.833 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:22.834+0000 7f3e7ca00640 1 --2- 172.21.15.47:0/2604829027 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3e781056c0 0x7f3e7819df90 unknown :-1 s=CLOSED pgs=1287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:35:22.833 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:22.834+0000 7f3e7ca00640 1 --2- 172.21.15.47:0/2604829027 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f3e5c0a3080 0x7f3e5c0a5540 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:35:22.833 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:22.834+0000 7f3e7ca00640 1 --2- 172.21.15.47:0/2604829027 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3e781005f0 0x7f3e7819da50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:35:22.833 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:22.834+0000 7f3e7ca00640 1 -- 172.21.15.47:0/2604829027 >> 172.21.15.47:0/2604829027 conn(0x7f3e780fbeb0 msgr2=0x7f3e78069120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:35:22.833 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:22.835+0000 7f3e7ca00640 1 -- 172.21.15.47:0/2604829027 shutdown_connections 2024-03-21T10:35:22.833 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:22.835+0000 7f3e7ca00640 1 -- 172.21.15.47:0/2604829027 wait complete. 2024-03-21T10:35:22.849 DEBUG:teuthology.orchestra.run.smithi047:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true' 2024-03-21T10:35:23.067 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:23.069+0000 7ff641200640 1 -- 172.21.15.47:0/3362422543 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff63c10dcc0 msgr2=0x7ff63c110110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:35:23.067 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:23.069+0000 7ff641200640 1 --2- 172.21.15.47:0/3362422543 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff63c10dcc0 0x7ff63c110110 secure :-1 s=READY pgs=1288 cs=0 l=1 rev1=1 crypto rx=0x7ff634009ca0 tx=0x7ff63402c180 comp rx=0 tx=0).stop 2024-03-21T10:35:23.067 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:23.069+0000 7ff641200640 1 -- 172.21.15.47:0/3362422543 shutdown_connections 2024-03-21T10:35:23.068 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:23.069+0000 7ff641200640 1 --2- 172.21.15.47:0/3362422543 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff63c10dcc0 0x7ff63c110110 unknown :-1 s=CLOSED pgs=1288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:35:23.068 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:23.069+0000 7ff641200640 1 --2- 172.21.15.47:0/3362422543 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff63c0743f0 0x7ff63c10d780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:35:23.068 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:23.069+0000 7ff641200640 1 --2- 172.21.15.47:0/3362422543 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff63c073ad0 0x7ff63c073eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:35:23.068 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:23.069+0000 7ff641200640 1 -- 172.21.15.47:0/3362422543 >> 172.21.15.47:0/3362422543 conn(0x7ff63c0fcff0 msgr2=0x7ff63c0ff410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:35:23.068 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:23.069+0000 7ff641200640 1 -- 172.21.15.47:0/3362422543 shutdown_connections 2024-03-21T10:35:23.068 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:23.069+0000 7ff641200640 1 -- 172.21.15.47:0/3362422543 wait complete. 2024-03-21T10:35:23.069 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:23.070+0000 7ff641200640 1 Processor -- start 2024-03-21T10:35:23.070 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:23.070+0000 7ff641200640 1 -- start start 2024-03-21T10:35:23.070 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:23.070+0000 7ff641200640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff63c073ad0 0x7ff63c199e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:35:23.070 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:23.070+0000 7ff641200640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff63c0743f0 0x7ff63c19a370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:35:23.070 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:23.071+0000 7ff641200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff63c10dcc0 0x7ff63c19e6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:35:23.070 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:23.071+0000 7ff641200640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff63c113910 con 0x7ff63c0743f0 2024-03-21T10:35:23.070 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:23.071+0000 7ff641200640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff63c113790 con 0x7ff63c10dcc0 2024-03-21T10:35:23.070 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:23.071+0000 7ff641200640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff63c113a90 con 0x7ff63c073ad0 2024-03-21T10:35:23.070 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:23.071+0000 7ff63aa00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff63c0743f0 0x7ff63c19a370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:35:23.070 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:23.071+0000 7ff63aa00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff63c0743f0 0x7ff63c19a370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:58606/0 (socket says 172.21.15.47:58606) 2024-03-21T10:35:23.070 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:23.071+0000 7ff63aa00640 1 -- 172.21.15.47:0/1390923931 learned_addr learned my addr 172.21.15.47:0/1390923931 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:35:23.070 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:23.071+0000 7ff63b400640 1 --2- 172.21.15.47:0/1390923931 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff63c073ad0 0x7ff63c199e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:35:23.070 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:23.071+0000 7ff63be00640 1 --2- 172.21.15.47:0/1390923931 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff63c10dcc0 0x7ff63c19e6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:35:23.071 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:23.072+0000 7ff63aa00640 1 -- 172.21.15.47:0/1390923931 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff63c073ad0 msgr2=0x7ff63c199e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:35:23.071 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:23.072+0000 7ff63aa00640 1 --2- 172.21.15.47:0/1390923931 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff63c073ad0 0x7ff63c199e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:35:23.071 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:23.072+0000 7ff63aa00640 1 -- 172.21.15.47:0/1390923931 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff63c10dcc0 msgr2=0x7ff63c19e6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:35:23.071 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:23.072+0000 7ff63aa00640 1 --2- 172.21.15.47:0/1390923931 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff63c10dcc0 0x7ff63c19e6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:35:23.071 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:23.072+0000 7ff63aa00640 1 -- 172.21.15.47:0/1390923931 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff63c19ee20 con 0x7ff63c0743f0 2024-03-21T10:35:23.071 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:23.072+0000 7ff63aa00640 1 --2- 172.21.15.47:0/1390923931 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff63c0743f0 0x7ff63c19a370 secure :-1 s=READY pgs=1289 cs=0 l=1 rev1=1 crypto rx=0x7ff62800d950 tx=0x7ff62800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:35:23.071 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:23.072+0000 7ff633e00640 1 -- 172.21.15.47:0/1390923931 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff628017030 con 0x7ff63c0743f0 2024-03-21T10:35:23.072 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:23.072+0000 7ff633e00640 1 -- 172.21.15.47:0/1390923931 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7ff628004020 con 0x7ff63c0743f0 2024-03-21T10:35:23.072 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:23.072+0000 7ff633e00640 1 -- 172.21.15.47:0/1390923931 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6280047a0 con 0x7ff63c0743f0 2024-03-21T10:35:23.072 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:23.072+0000 7ff641200640 1 -- 172.21.15.47:0/1390923931 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff63c19f110 con 0x7ff63c0743f0 2024-03-21T10:35:23.072 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:23.073+0000 7ff641200640 1 -- 172.21.15.47:0/1390923931 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff63c1a69a0 con 0x7ff63c0743f0 2024-03-21T10:35:23.072 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:23.074+0000 7ff641200640 1 -- 172.21.15.47:0/1390923931 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff63c075180 con 0x7ff63c0743f0 2024-03-21T10:35:23.074 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:23.076+0000 7ff633e00640 1 -- 172.21.15.47:0/1390923931 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 27) v1 ==== 137985+0+0 (secure 0 0 0) 0x7ff628009a30 con 0x7ff63c0743f0 2024-03-21T10:35:23.076 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:23.078+0000 7ff633e00640 1 --2- 172.21.15.47:0/1390923931 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7ff6200a3100 0x7ff6200a55c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:35:23.076 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:23.078+0000 7ff633e00640 1 -- 172.21.15.47:0/1390923931 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(105..105 src has 1..105) v4 ==== 9562+0+0 (secure 0 0 0) 0x7ff6280cf330 con 0x7ff63c0743f0 2024-03-21T10:35:23.076 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:23.078+0000 7ff63b400640 1 --2- 172.21.15.47:0/1390923931 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7ff6200a3100 0x7ff6200a55c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:35:23.077 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:23.079+0000 7ff63b400640 1 --2- 172.21.15.47:0/1390923931 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7ff6200a3100 0x7ff6200a55c0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7ff62c009bd0 tx=0x7ff62c005f90 comp rx=0 tx=0).ready entity=mgr.15687 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:35:23.080 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:23.083+0000 7ff633e00640 1 -- 172.21.15.47:0/1390923931 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7ff62809b9b0 con 0x7ff63c0743f0 2024-03-21T10:35:23.324 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:23.326+0000 7ff641200640 1 -- 172.21.15.47:0/1390923931 --> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] -- mgr_command(tid 0: {"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}) v1 -- 0x7ff63c108e70 con 0x7ff6200a3100 2024-03-21T10:35:23.349 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:23.343+0000 7ff633e00640 1 -- 172.21.15.47:0/1390923931 <== mgr.15687 v2:172.21.15.47:6800/196172753 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+10 (secure 0 0 0) 0x7ff63c108e70 con 0x7ff6200a3100 2024-03-21T10:35:23.350 INFO:teuthology.orchestra.run.smithi047.stdout:/ 1m 6m3h 2024-03-21T10:35:23.350 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:23.345+0000 7ff641200640 1 -- 172.21.15.47:0/1390923931 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7ff6200a3100 msgr2=0x7ff6200a55c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:35:23.351 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:23.345+0000 7ff641200640 1 --2- 172.21.15.47:0/1390923931 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7ff6200a3100 0x7ff6200a55c0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7ff62c009bd0 tx=0x7ff62c005f90 comp rx=0 tx=0).stop 2024-03-21T10:35:23.351 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:23.345+0000 7ff641200640 1 -- 172.21.15.47:0/1390923931 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff63c0743f0 msgr2=0x7ff63c19a370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:35:23.351 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:23.345+0000 7ff641200640 1 --2- 172.21.15.47:0/1390923931 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff63c0743f0 0x7ff63c19a370 secure :-1 s=READY pgs=1289 cs=0 l=1 rev1=1 crypto rx=0x7ff62800d950 tx=0x7ff62800de10 comp rx=0 tx=0).stop 2024-03-21T10:35:23.351 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:23.345+0000 7ff641200640 1 -- 172.21.15.47:0/1390923931 shutdown_connections 2024-03-21T10:35:23.351 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:23.345+0000 7ff641200640 1 --2- 172.21.15.47:0/1390923931 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7ff6200a3100 0x7ff6200a55c0 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:35:23.351 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:23.345+0000 7ff641200640 1 --2- 172.21.15.47:0/1390923931 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff63c10dcc0 0x7ff63c19e6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:35:23.351 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:23.345+0000 7ff641200640 1 --2- 172.21.15.47:0/1390923931 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff63c0743f0 0x7ff63c19a370 unknown :-1 s=CLOSED pgs=1289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:35:23.351 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:23.345+0000 7ff641200640 1 --2- 172.21.15.47:0/1390923931 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff63c073ad0 0x7ff63c199e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:35:23.351 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:23.345+0000 7ff641200640 1 -- 172.21.15.47:0/1390923931 >> 172.21.15.47:0/1390923931 conn(0x7ff63c0fcff0 msgr2=0x7ff63c0fdd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:35:23.351 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:23.346+0000 7ff641200640 1 -- 172.21.15.47:0/1390923931 shutdown_connections 2024-03-21T10:35:23.351 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:23.346+0000 7ff641200640 1 -- 172.21.15.47:0/1390923931 wait complete. 2024-03-21T10:35:23.365 DEBUG:teuthology.run_tasks:Unwinding manager fwd_scrub 2024-03-21T10:35:23.498 INFO:tasks.fwd_scrub:joining ForwardScrubbers 2024-03-21T10:35:23.711 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:35:23 smithi047 ceph-mon[31547]: pgmap v313: 129 pgs: 129 active+clean; 2.5 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 10 op/s 2024-03-21T10:35:23.726 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:35:23 smithi100 ceph-mon[35193]: pgmap v313: 129 pgs: 129 active+clean; 2.5 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 10 op/s 2024-03-21T10:35:23.736 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:35:23 smithi157 ceph-mon[35581]: pgmap v313: 129 pgs: 129 active+clean; 2.5 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 10 op/s 2024-03-21T10:35:24.711 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:35:24 smithi047 ceph-mon[31547]: from='client.18714 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:35:24.711 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:35:24 smithi047 ceph-mon[31547]: from='client.18720 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:35:24.726 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:35:24 smithi100 ceph-mon[35193]: from='client.18714 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:35:24.726 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:35:24 smithi100 ceph-mon[35193]: from='client.18720 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:35:24.736 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:35:24 smithi157 ceph-mon[35581]: from='client.18714 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:35:24.737 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:35:24 smithi157 ceph-mon[35581]: from='client.18720 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:35:25.711 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:35:25 smithi047 ceph-mon[31547]: pgmap v314: 129 pgs: 129 active+clean; 2.5 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 98 B/s wr, 9 op/s 2024-03-21T10:35:25.726 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:35:25 smithi100 ceph-mon[35193]: pgmap v314: 129 pgs: 129 active+clean; 2.5 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 98 B/s wr, 9 op/s 2024-03-21T10:35:25.736 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:35:25 smithi157 ceph-mon[35581]: pgmap v314: 129 pgs: 129 active+clean; 2.5 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 98 B/s wr, 9 op/s 2024-03-21T10:35:26.205 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-21T10:35:26.206 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-21T10:35:26.428 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:26.429+0000 7f4336c00640 1 -- 172.21.15.47:0/1645334734 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4330108860 msgr2=0x7f433010c8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:35:26.428 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:26.429+0000 7f4336c00640 1 --2- 172.21.15.47:0/1645334734 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4330108860 0x7f433010c8a0 secure :-1 s=READY pgs=1290 cs=0 l=1 rev1=1 crypto rx=0x7f431c009900 tx=0x7f431c02bc40 comp rx=0 tx=0).stop 2024-03-21T10:35:26.428 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:26.429+0000 7f4336c00640 1 -- 172.21.15.47:0/1645334734 shutdown_connections 2024-03-21T10:35:26.428 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:26.429+0000 7f4336c00640 1 --2- 172.21.15.47:0/1645334734 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f433010cf80 0x7f4330114b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:35:26.428 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:26.429+0000 7f4336c00640 1 --2- 172.21.15.47:0/1645334734 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4330108860 0x7f433010c8a0 unknown :-1 s=CLOSED pgs=1290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:35:26.429 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:26.429+0000 7f4336c00640 1 --2- 172.21.15.47:0/1645334734 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4330107eb0 0x7f4330108290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:35:26.429 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:26.429+0000 7f4336c00640 1 -- 172.21.15.47:0/1645334734 >> 172.21.15.47:0/1645334734 conn(0x7f4330075be0 msgr2=0x7f4330078000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:35:26.429 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:26.429+0000 7f4336c00640 1 -- 172.21.15.47:0/1645334734 shutdown_connections 2024-03-21T10:35:26.429 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:26.429+0000 7f4336c00640 1 -- 172.21.15.47:0/1645334734 wait complete. 2024-03-21T10:35:26.430 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:26.430+0000 7f4336c00640 1 Processor -- start 2024-03-21T10:35:26.430 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:26.430+0000 7f4336c00640 1 -- start start 2024-03-21T10:35:26.431 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:26.430+0000 7f4336c00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4330107eb0 0x7f433019e460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:35:26.431 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:26.430+0000 7f4336c00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f433010cf80 0x7f433019e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:35:26.431 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:26.430+0000 7f4336c00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f43301a2d20 0x7f43301a31d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:35:26.431 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:26.430+0000 7f4336c00640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4330117ee0 con 0x7f43301a2d20 2024-03-21T10:35:26.431 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:26.430+0000 7f4336c00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4330117d60 con 0x7f433010cf80 2024-03-21T10:35:26.431 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:26.430+0000 7f4336c00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4330118060 con 0x7f4330107eb0 2024-03-21T10:35:26.431 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:26.431+0000 7f4334c00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f43301a2d20 0x7f43301a31d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:35:26.431 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:26.431+0000 7f432f400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f433010cf80 0x7f433019e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:35:26.431 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:26.431+0000 7f432f400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f433010cf80 0x7f433019e9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:59766/0 (socket says 172.21.15.47:59766) 2024-03-21T10:35:26.431 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:26.431+0000 7f432f400640 1 -- 172.21.15.47:0/262198381 learned_addr learned my addr 172.21.15.47:0/262198381 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:35:26.431 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:26.431+0000 7f4334c00640 1 --2- 172.21.15.47:0/262198381 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f43301a2d20 0x7f43301a31d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:58614/0 (socket says 172.21.15.47:58614) 2024-03-21T10:35:26.431 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:26.431+0000 7f432fe00640 1 --2- 172.21.15.47:0/262198381 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4330107eb0 0x7f433019e460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:35:26.431 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:26.431+0000 7f4334c00640 1 -- 172.21.15.47:0/262198381 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4330107eb0 msgr2=0x7f433019e460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:35:26.431 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:26.431+0000 7f4334c00640 1 --2- 172.21.15.47:0/262198381 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4330107eb0 0x7f433019e460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:35:26.432 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:26.431+0000 7f4334c00640 1 -- 172.21.15.47:0/262198381 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f433010cf80 msgr2=0x7f433019e9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:35:26.432 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:26.431+0000 7f4334c00640 1 --2- 172.21.15.47:0/262198381 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f433010cf80 0x7f433019e9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:35:26.432 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:26.431+0000 7f4334c00640 1 -- 172.21.15.47:0/262198381 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43301a3810 con 0x7f43301a2d20 2024-03-21T10:35:26.432 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:26.431+0000 7f432f400640 1 --2- 172.21.15.47:0/262198381 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f433010cf80 0x7f433019e9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T10:35:26.432 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:26.432+0000 7f4334c00640 1 --2- 172.21.15.47:0/262198381 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f43301a2d20 0x7f43301a31d0 secure :-1 s=READY pgs=1291 cs=0 l=1 rev1=1 crypto rx=0x7f432800cab0 tx=0x7f432800cf70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:35:26.433 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:26.432+0000 7f432cc00640 1 -- 172.21.15.47:0/262198381 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f432800ae20 con 0x7f43301a2d20 2024-03-21T10:35:26.433 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:26.432+0000 7f432cc00640 1 -- 172.21.15.47:0/262198381 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f432800f900 con 0x7f43301a2d20 2024-03-21T10:35:26.433 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:26.432+0000 7f4336c00640 1 -- 172.21.15.47:0/262198381 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f43301a3b00 con 0x7f43301a2d20 2024-03-21T10:35:26.433 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:26.432+0000 7f432cc00640 1 -- 172.21.15.47:0/262198381 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4328018a00 con 0x7f43301a2d20 2024-03-21T10:35:26.433 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:26.432+0000 7f4336c00640 1 -- 172.21.15.47:0/262198381 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f43301ab2c0 con 0x7f43301a2d20 2024-03-21T10:35:26.434 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:26.436+0000 7f43399f6740 1 Processor -- start 2024-03-21T10:35:26.438 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:26.440+0000 7f432cc00640 1 -- 172.21.15.47:0/262198381 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 27) v1 ==== 137985+0+0 (secure 0 0 0) 0x7f432800fab0 con 0x7f43301a2d20 2024-03-21T10:35:26.439 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:26.440+0000 7f43399f6740 1 -- start start 2024-03-21T10:35:26.440 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:26.441+0000 7f43399f6740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c1f781dc00 0x55c1f781c950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:35:26.440 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:26.441+0000 7f43399f6740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c1f781cf20 0x55c1f781d3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:35:26.440 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:26.441+0000 7f43399f6740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c1f7826290 0x55c1f7828680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:35:26.441 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:26.441+0000 7f43399f6740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c1f7781250 con 0x55c1f781cf20 2024-03-21T10:35:26.441 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:26.441+0000 7f43399f6740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c1f7729a30 con 0x55c1f7826290 2024-03-21T10:35:26.441 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:26.441+0000 7f43399f6740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c1f777d2f0 con 0x55c1f781dc00 2024-03-21T10:35:26.441 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:26.441+0000 7f432fe00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c1f781cf20 0x55c1f781d3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:35:26.441 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:26.441+0000 7f432fe00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c1f781cf20 0x55c1f781d3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:58616/0 (socket says 172.21.15.47:58616) 2024-03-21T10:35:26.441 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:26.441+0000 7f432fe00640 1 -- 172.21.15.47:0/2656553453 learned_addr learned my addr 172.21.15.47:0/2656553453 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:35:26.441 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:26.441+0000 7f432f400640 1 --2- 172.21.15.47:0/2656553453 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c1f7826290 0x55c1f7828680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:35:26.441 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:26.441+0000 7f432f400640 1 --2- 172.21.15.47:0/2656553453 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c1f781dc00 0x55c1f781c950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:35:26.441 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:26.441+0000 7f432fe00640 1 -- 172.21.15.47:0/2656553453 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c1f781dc00 msgr2=0x55c1f781c950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:35:26.441 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:26.441+0000 7f432fe00640 1 --2- 172.21.15.47:0/2656553453 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c1f781dc00 0x55c1f781c950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:35:26.441 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:26.441+0000 7f432fe00640 1 -- 172.21.15.47:0/2656553453 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c1f7826290 msgr2=0x55c1f7828680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:35:26.441 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:26.441+0000 7f432fe00640 1 --2- 172.21.15.47:0/2656553453 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c1f7826290 0x55c1f7828680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:35:26.441 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:26.441+0000 7f432fe00640 1 -- 172.21.15.47:0/2656553453 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c1f7714690 con 0x55c1f781cf20 2024-03-21T10:35:26.441 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:26.442+0000 7f432f400640 1 --2- 172.21.15.47:0/2656553453 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c1f7826290 0x55c1f7828680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T10:35:26.443 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:26.442+0000 7f432fe00640 1 --2- 172.21.15.47:0/2656553453 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c1f781cf20 0x55c1f781d3a0 secure :-1 s=READY pgs=1292 cs=0 l=1 rev1=1 crypto rx=0x7f4320009cf0 tx=0x7f4320011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=328c048ec2cf357c server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:35:26.443 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:26.442+0000 7f432cc00640 1 --2- 172.21.15.47:0/262198381 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f43140a3110 0x7f43140a55d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:35:26.443 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:26.442+0000 7f432cc00640 1 -- 172.21.15.47:0/262198381 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(105..105 src has 1..105) v4 ==== 9562+0+0 (secure 0 0 0) 0x7f43280d0270 con 0x7f43301a2d20 2024-03-21T10:35:26.443 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:26.442+0000 7f4325200640 1 -- 172.21.15.47:0/2656553453 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4320004230 con 0x55c1f781cf20 2024-03-21T10:35:26.443 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:26.442+0000 7f432f400640 1 --2- 172.21.15.47:0/262198381 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f43140a3110 0x7f43140a55d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:35:26.443 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:26.442+0000 7f4325200640 1 -- 172.21.15.47:0/2656553453 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f43200043d0 con 0x55c1f781cf20 2024-03-21T10:35:26.443 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:26.442+0000 7f4325200640 1 -- 172.21.15.47:0/2656553453 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43200197e0 con 0x55c1f781cf20 2024-03-21T10:35:26.443 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:26.443+0000 7f432f400640 1 --2- 172.21.15.47:0/262198381 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f43140a3110 0x7f43140a55d0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f4330107050 tx=0x7f431c009260 comp rx=0 tx=0).ready entity=mgr.15687 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:35:26.443 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:26.443+0000 7f43399f6740 1 -- 172.21.15.47:0/2656553453 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c1f781cf20 msgr2=0x55c1f781d3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:35:26.444 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:26.443+0000 7f43399f6740 1 --2- 172.21.15.47:0/2656553453 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c1f781cf20 0x55c1f781d3a0 secure :-1 s=READY pgs=1292 cs=0 l=1 rev1=1 crypto rx=0x7f4320009cf0 tx=0x7f4320011810 comp rx=0 tx=0).stop 2024-03-21T10:35:26.444 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:26.444+0000 7f43399f6740 1 -- 172.21.15.47:0/2656553453 shutdown_connections 2024-03-21T10:35:26.444 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:26.444+0000 7f43399f6740 1 --2- 172.21.15.47:0/2656553453 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c1f7826290 0x55c1f7828680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:35:26.444 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:26.444+0000 7f43399f6740 1 --2- 172.21.15.47:0/2656553453 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c1f781cf20 0x55c1f781d3a0 unknown :-1 s=CLOSED pgs=1292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:35:26.444 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:26.444+0000 7f43399f6740 1 --2- 172.21.15.47:0/2656553453 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c1f781dc00 0x55c1f781c950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:35:26.444 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:26.444+0000 7f43399f6740 1 -- 172.21.15.47:0/2656553453 >> 172.21.15.47:0/2656553453 conn(0x55c1f77e4200 msgr2=0x55c1f7817970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:35:26.444 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:26.444+0000 7f43399f6740 1 -- 172.21.15.47:0/2656553453 shutdown_connections 2024-03-21T10:35:26.444 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:26.444+0000 7f43399f6740 1 -- 172.21.15.47:0/2656553453 wait complete. 2024-03-21T10:35:26.444 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:26.444+0000 7f43399f6740 1 -- start start 2024-03-21T10:35:26.447 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:26.445+0000 7f43399f6740 1 Processor -- start 2024-03-21T10:35:26.448 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:26.445+0000 7f43399f6740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c1f781cf20 0x55c1f7825980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:35:26.448 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:26.445+0000 7f43399f6740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c1f781dc00 0x55c1f7822d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:35:26.448 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:26.445+0000 7f43399f6740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c1f7826290 0x55c1f7823250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:35:26.448 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:26.445+0000 7f43399f6740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c1f7829100 con 0x55c1f781dc00 2024-03-21T10:35:26.448 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:26.445+0000 7f43399f6740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c1f773ce80 con 0x55c1f781cf20 2024-03-21T10:35:26.448 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:26.445+0000 7f43399f6740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c1f7829280 con 0x55c1f7826290 2024-03-21T10:35:26.448 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:26.446+0000 7f43399f6740 10 client.0 _ll_register_callbacks cb 0x55c1f774f130 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T10:35:26.448 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:26.446+0000 7f432fe00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c1f781cf20 0x55c1f7825980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:35:26.448 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:26.446+0000 7f432f400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c1f781dc00 0x55c1f7822d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:35:26.449 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:26.446+0000 7f432fe00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c1f781cf20 0x55c1f7825980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:59794/0 (socket says 172.21.15.47:59794) 2024-03-21T10:35:26.449 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:26.446+0000 7f432fe00640 1 -- 172.21.15.47:0/3158378322 learned_addr learned my addr 172.21.15.47:0/3158378322 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:35:26.449 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:26.446+0000 7f4334c00640 1 --2- 172.21.15.47:0/3158378322 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c1f7826290 0x55c1f7823250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:35:26.449 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:26.446+0000 7f432f400640 1 -- 172.21.15.47:0/3158378322 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c1f7826290 msgr2=0x55c1f7823250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:35:26.449 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:26.446+0000 7f432f400640 1 --2- 172.21.15.47:0/3158378322 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c1f7826290 0x55c1f7823250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:35:26.449 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:26.446+0000 7f432f400640 1 -- 172.21.15.47:0/3158378322 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c1f781cf20 msgr2=0x55c1f7825980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:35:26.449 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:26.446+0000 7f432f400640 1 --2- 172.21.15.47:0/3158378322 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c1f781cf20 0x55c1f7825980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:35:26.449 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:26.446+0000 7f432f400640 1 -- 172.21.15.47:0/3158378322 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55c1f755f7d0 con 0x55c1f781dc00 2024-03-21T10:35:26.449 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:26.446+0000 7f432fe00640 1 --2- 172.21.15.47:0/3158378322 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c1f781cf20 0x55c1f7825980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T10:35:26.449 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:26.447+0000 7f432f400640 1 --2- 172.21.15.47:0/3158378322 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c1f781dc00 0x55c1f7822d10 secure :-1 s=READY pgs=1293 cs=0 l=1 rev1=1 crypto rx=0x7f431c006010 tx=0x7f431c02bc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b1d19e9eaac5b8e7 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:35:26.449 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:26.447+0000 7f4319600640 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-03-21T10:35:26.449 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:26.447+0000 7f4319600640 1 -- 172.21.15.47:0/3158378322 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f431c053070 con 0x55c1f781dc00 2024-03-21T10:35:26.449 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:26.447+0000 7f4319600640 1 -- 172.21.15.47:0/3158378322 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(105..105 src has 1..105) v4 ==== 9562+0+0 (secure 0 0 0) 0x7f431c003d50 con 0x55c1f781dc00 2024-03-21T10:35:26.449 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:26.447+0000 7f43399f6740 1 -- 172.21.15.47:0/3158378322 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f43200043d0 con 0x55c1f781dc00 2024-03-21T10:35:26.450 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:26.447+0000 7f4319600640 1 -- 172.21.15.47:0/3158378322 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f431c05f050 con 0x55c1f781dc00 2024-03-21T10:35:26.450 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:26.447+0000 7f43399f6740 10 client.18732 fetch_fsmap learned FSMap version 32 2024-03-21T10:35:26.450 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:26.447+0000 7f43399f6740 1 -- 172.21.15.47:0/3158378322 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x55c1f74953f0 con 0x55c1f781dc00 2024-03-21T10:35:26.450 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:26.448+0000 7f4319600640 1 -- 172.21.15.47:0/3158378322 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f43200043d0 con 0x55c1f781dc00 2024-03-21T10:35:26.450 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:26.448+0000 7f4319600640 1 -- 172.21.15.47:0/3158378322 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 32) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f431c04e070 con 0x55c1f781dc00 2024-03-21T10:35:26.450 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:26.448+0000 7f43399f6740 10 client.18732 fetch_fsmap finished waiting for FSMap version 32 2024-03-21T10:35:26.450 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:26.448+0000 7f43399f6740 10 client.18732 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-21T10:35:26.450 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:26.448+0000 7f43399f6740 20 client.18732 populate_metadata read hostname 'smithi047' 2024-03-21T10:35:26.450 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:26.448+0000 7f43399f6740 1 --2- 172.21.15.47:0/3158378322 >> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] conn(0x55c1f78360f0 0x55c1f7838500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:35:26.450 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:26.448+0000 7f43399f6740 4 client.18732 mds_command: new command op to 34535 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T10:35:26.450 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:26.448+0000 7f43399f6740 1 -- 172.21.15.47:0/3158378322 --> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55c1f778e790 con 0x55c1f78360f0 2024-03-21T10:35:26.450 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:26.449+0000 7f432fe00640 1 --2- 172.21.15.47:0/3158378322 >> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] conn(0x55c1f78360f0 0x55c1f7838500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:35:26.452 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:26.449+0000 7f432fe00640 1 --2- 172.21.15.47:0/3158378322 >> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] conn(0x55c1f78360f0 0x55c1f7838500 crc :-1 s=READY pgs=221 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2a361c61e42b08ed server_cookie=f7c62e1708e2e7f0 in_seq=0 out_seq=0 2024-03-21T10:35:26.452 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:26.449+0000 7f4319600640 10 client.18732 ms_handle_connect on v2:172.21.15.100:6838/4173168440 2024-03-21T10:35:26.452 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:26.450+0000 7f4319600640 1 -- 172.21.15.47:0/3158378322 <== mds.0 v2:172.21.15.100:6838/4173168440 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55c1f778e790 con 0x55c1f78360f0 2024-03-21T10:35:26.452 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:26.450+0000 7f4319600640 10 client.18732 handle_command_reply: tid=0 2024-03-21T10:35:26.452 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:26.450+0000 7f43399f6740 1 client.18732 shutdown 2024-03-21T10:35:26.452 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:26.450+0000 7f43399f6740 1 -- 172.21.15.47:0/3158378322 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c1f781dc00 msgr2=0x55c1f7822d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:35:26.452 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:26.450+0000 7f43399f6740 1 --2- 172.21.15.47:0/3158378322 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c1f781dc00 0x55c1f7822d10 secure :-1 s=READY pgs=1293 cs=0 l=1 rev1=1 crypto rx=0x7f431c006010 tx=0x7f431c02bc40 comp rx=0 tx=0).stop 2024-03-21T10:35:26.452 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:26.450+0000 7f43399f6740 1 -- 172.21.15.47:0/3158378322 shutdown_connections 2024-03-21T10:35:26.452 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:26.450+0000 7f43399f6740 1 --2- 172.21.15.47:0/3158378322 >> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] conn(0x55c1f78360f0 0x55c1f7838500 crc :-1 s=READY pgs=221 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:35:26.452 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:26.450+0000 7f43399f6740 1 --2- 172.21.15.47:0/3158378322 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c1f7826290 0x55c1f7823250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:35:26.452 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:26.450+0000 7f43399f6740 1 --2- 172.21.15.47:0/3158378322 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c1f781dc00 0x55c1f7822d10 unknown :-1 s=CLOSED pgs=1293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:35:26.452 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:26.450+0000 7f43399f6740 1 --2- 172.21.15.47:0/3158378322 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c1f781cf20 0x55c1f7825980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:35:26.452 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:26.450+0000 7f4319600640 0 client.18732 ms_handle_reset on v2:172.21.15.100:6838/4173168440 2024-03-21T10:35:26.453 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:26.450+0000 7f43399f6740 1 -- 172.21.15.47:0/3158378322 >> 172.21.15.47:0/3158378322 conn(0x55c1f77e4200 msgr2=0x55c1f78196c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:35:26.453 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:26.450+0000 7f43399f6740 1 -- 172.21.15.47:0/3158378322 shutdown_connections 2024-03-21T10:35:26.453 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:26.451+0000 7f43399f6740 1 -- 172.21.15.47:0/3158378322 wait complete. 2024-03-21T10:35:26.453 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:26.451+0000 7f43399f6740 20 client.18732 trim_cache size 0 max 16384 2024-03-21T10:35:26.468 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:26.468+0000 7f43399f6740 1 Processor -- start 2024-03-21T10:35:26.468 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:26.469+0000 7f43399f6740 1 -- start start 2024-03-21T10:35:26.468 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:26.469+0000 7f43399f6740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c1f781cf20 0x55c1f7825440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:35:26.468 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:26.469+0000 7f43399f6740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c1f781dc00 0x55c1f7825980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:35:26.468 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:26.469+0000 7f43399f6740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c1f7826290 0x55c1f7827680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:35:26.468 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:26.469+0000 7f43399f6740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43301b4f70 con 0x55c1f7826290 2024-03-21T10:35:26.468 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:26.469+0000 7f43399f6740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43301b68f0 con 0x55c1f781dc00 2024-03-21T10:35:26.468 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:26.469+0000 7f43399f6740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43301b55d0 con 0x55c1f781cf20 2024-03-21T10:35:26.470 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:26.469+0000 7f432f400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c1f781dc00 0x55c1f7825980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:35:26.470 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:26.469+0000 7f4334c00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c1f7826290 0x55c1f7827680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:35:26.470 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:26.469+0000 7f432f400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c1f781dc00 0x55c1f7825980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:59804/0 (socket says 172.21.15.47:59804) 2024-03-21T10:35:26.470 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:26.469+0000 7f432f400640 1 -- 172.21.15.47:0/209181566 learned_addr learned my addr 172.21.15.47:0/209181566 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:35:26.470 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:26.470+0000 7f432fe00640 1 --2- 172.21.15.47:0/209181566 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c1f781cf20 0x55c1f7825440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:35:26.470 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:26.470+0000 7f432f400640 1 -- 172.21.15.47:0/209181566 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c1f781cf20 msgr2=0x55c1f7825440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:35:26.471 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:26.470+0000 7f432f400640 1 --2- 172.21.15.47:0/209181566 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c1f781cf20 0x55c1f7825440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:35:26.471 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:26.470+0000 7f432f400640 1 -- 172.21.15.47:0/209181566 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c1f7826290 msgr2=0x55c1f7827680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:35:26.471 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:26.470+0000 7f432f400640 1 --2- 172.21.15.47:0/209181566 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c1f7826290 0x55c1f7827680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:35:26.471 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:26.470+0000 7f432f400640 1 -- 172.21.15.47:0/209181566 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c1f74550f0 con 0x55c1f781dc00 2024-03-21T10:35:26.471 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:26.470+0000 7f432f400640 1 --2- 172.21.15.47:0/209181566 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c1f781dc00 0x55c1f7825980 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f431c036040 tx=0x7f431c009890 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d255fce2cd46aadc server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:35:26.471 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:26.470+0000 7f431be00640 1 -- 172.21.15.47:0/209181566 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f431c068070 con 0x55c1f781dc00 2024-03-21T10:35:26.471 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:26.470+0000 7f431be00640 1 -- 172.21.15.47:0/209181566 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f431c049070 con 0x55c1f781dc00 2024-03-21T10:35:26.471 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:26.470+0000 7f431be00640 1 -- 172.21.15.47:0/209181566 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f431c053040 con 0x55c1f781dc00 2024-03-21T10:35:26.471 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:26.471+0000 7f43399f6740 1 -- 172.21.15.47:0/209181566 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c1f781dc00 msgr2=0x55c1f7825980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:35:26.471 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:26.471+0000 7f43399f6740 1 --2- 172.21.15.47:0/209181566 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c1f781dc00 0x55c1f7825980 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f431c036040 tx=0x7f431c009890 comp rx=0 tx=0).stop 2024-03-21T10:35:26.471 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:26.471+0000 7f43399f6740 1 -- 172.21.15.47:0/209181566 shutdown_connections 2024-03-21T10:35:26.471 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:26.471+0000 7f43399f6740 1 --2- 172.21.15.47:0/209181566 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c1f7826290 0x55c1f7827680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:35:26.472 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:26.471+0000 7f43399f6740 1 --2- 172.21.15.47:0/209181566 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c1f781dc00 0x55c1f7825980 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:35:26.472 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:26.471+0000 7f43399f6740 1 --2- 172.21.15.47:0/209181566 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c1f781cf20 0x55c1f7825440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:35:26.472 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:26.471+0000 7f43399f6740 1 -- 172.21.15.47:0/209181566 >> 172.21.15.47:0/209181566 conn(0x55c1f77e4200 msgr2=0x55c1f78174a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:35:26.472 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:26.471+0000 7f43399f6740 1 -- 172.21.15.47:0/209181566 shutdown_connections 2024-03-21T10:35:26.472 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:26.471+0000 7f43399f6740 1 -- 172.21.15.47:0/209181566 wait complete. 2024-03-21T10:35:26.476 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:26.472+0000 7f43399f6740 1 -- start start 2024-03-21T10:35:26.476 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:26.472+0000 7f43399f6740 1 Processor -- start 2024-03-21T10:35:26.476 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:26.472+0000 7f43399f6740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c1f781cf20 0x55c1f782dbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:35:26.476 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:26.472+0000 7f43399f6740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c1f781dc00 0x55c1f782e130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:35:26.476 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:26.472+0000 7f43399f6740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c1f7826290 0x55c1f782a090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:35:26.476 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:26.472+0000 7f43399f6740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43301b6290 con 0x55c1f7826290 2024-03-21T10:35:26.476 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:26.472+0000 7f43399f6740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43301b5c30 con 0x55c1f781cf20 2024-03-21T10:35:26.476 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:26.472+0000 7f43399f6740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43301b3150 con 0x55c1f781dc00 2024-03-21T10:35:26.477 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:26.472+0000 7f43399f6740 10 client.0 _ll_register_callbacks cb 0x7f431c009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T10:35:26.477 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:26.473+0000 7f432fe00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c1f781cf20 0x55c1f782dbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:35:26.477 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:26.473+0000 7f4334c00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c1f7826290 0x55c1f782a090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:35:26.477 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:26.473+0000 7f432fe00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c1f781cf20 0x55c1f782dbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:59812/0 (socket says 172.21.15.47:59812) 2024-03-21T10:35:26.477 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:26.473+0000 7f432fe00640 1 -- 172.21.15.47:0/4230192481 learned_addr learned my addr 172.21.15.47:0/4230192481 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:35:26.477 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:26.473+0000 7f432f400640 1 --2- 172.21.15.47:0/4230192481 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c1f781dc00 0x55c1f782e130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:35:26.477 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:26.473+0000 7f4334c00640 1 -- 172.21.15.47:0/4230192481 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c1f781dc00 msgr2=0x55c1f782e130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:35:26.477 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:26.473+0000 7f4334c00640 1 --2- 172.21.15.47:0/4230192481 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c1f781dc00 0x55c1f782e130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:35:26.477 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:26.473+0000 7f4334c00640 1 -- 172.21.15.47:0/4230192481 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c1f781cf20 msgr2=0x55c1f782dbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:35:26.477 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:26.473+0000 7f4334c00640 1 --2- 172.21.15.47:0/4230192481 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c1f781cf20 0x55c1f782dbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:35:26.477 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:26.473+0000 7f4334c00640 1 -- 172.21.15.47:0/4230192481 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55c1f782a800 con 0x55c1f7826290 2024-03-21T10:35:26.477 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:26.473+0000 7f432f400640 1 --2- 172.21.15.47:0/4230192481 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c1f781dc00 0x55c1f782e130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T10:35:26.477 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:26.473+0000 7f432fe00640 1 --2- 172.21.15.47:0/4230192481 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c1f781cf20 0x55c1f782dbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T10:35:26.478 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:26.473+0000 7f4334c00640 1 --2- 172.21.15.47:0/4230192481 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c1f7826290 0x55c1f782a090 secure :-1 s=READY pgs=1294 cs=0 l=1 rev1=1 crypto rx=0x7f43280cd550 tx=0x7f432809c9b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d8c6d8b4ea62b22b server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:35:26.478 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:26.473+0000 7f4319600640 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-03-21T10:35:26.478 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:26.474+0000 7f4319600640 1 -- 172.21.15.47:0/4230192481 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43280a0340 con 0x55c1f7826290 2024-03-21T10:35:26.478 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:26.474+0000 7f4319600640 1 -- 172.21.15.47:0/4230192481 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(105..105 src has 1..105) v4 ==== 9562+0+0 (secure 0 0 0) 0x7f43280a4770 con 0x55c1f7826290 2024-03-21T10:35:26.478 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:26.474+0000 7f43399f6740 1 -- 172.21.15.47:0/4230192481 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f431c049070 con 0x55c1f7826290 2024-03-21T10:35:26.478 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:26.474+0000 7f4319600640 1 -- 172.21.15.47:0/4230192481 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f43280a4c00 con 0x55c1f7826290 2024-03-21T10:35:26.478 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:26.474+0000 7f43399f6740 10 client.18738 fetch_fsmap learned FSMap version 32 2024-03-21T10:35:26.478 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:26.474+0000 7f43399f6740 1 -- 172.21.15.47:0/4230192481 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x55c1f7892470 con 0x55c1f7826290 2024-03-21T10:35:26.478 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:26.474+0000 7f4319600640 1 -- 172.21.15.47:0/4230192481 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f432807f080 con 0x55c1f7826290 2024-03-21T10:35:26.478 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:26.474+0000 7f4319600640 1 -- 172.21.15.47:0/4230192481 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 32) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f4328004c60 con 0x55c1f7826290 2024-03-21T10:35:26.478 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:26.474+0000 7f43399f6740 10 client.18738 fetch_fsmap finished waiting for FSMap version 32 2024-03-21T10:35:26.478 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:26.474+0000 7f43399f6740 10 client.18738 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-21T10:35:26.478 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:26.474+0000 7f43399f6740 20 client.18738 populate_metadata read hostname 'smithi047' 2024-03-21T10:35:26.478 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:26.474+0000 7f43399f6740 1 --2- 172.21.15.47:0/4230192481 >> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] conn(0x55c1f7892bc0 0x55c1f7894fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:35:26.479 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:26.474+0000 7f43399f6740 4 client.18738 mds_command: new command op to 34535 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-21T10:35:26.479 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:26.474+0000 7f43399f6740 1 -- 172.21.15.47:0/4230192481 --> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55c1f7788ae0 con 0x55c1f7892bc0 2024-03-21T10:35:26.479 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:26.475+0000 7f432fe00640 1 --2- 172.21.15.47:0/4230192481 >> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] conn(0x55c1f7892bc0 0x55c1f7894fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:35:26.479 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:26.475+0000 7f432fe00640 1 --2- 172.21.15.47:0/4230192481 >> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] conn(0x55c1f7892bc0 0x55c1f7894fe0 crc :-1 s=READY pgs=222 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d60294362e9a2744 server_cookie=8f8feaa724afa813 in_seq=0 out_seq=0 2024-03-21T10:35:26.479 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:26.475+0000 7f4319600640 10 client.18738 ms_handle_connect on v2:172.21.15.100:6838/4173168440 2024-03-21T10:35:26.479 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:26.476+0000 7f4319600640 1 -- 172.21.15.47:0/4230192481 <== mds.0 v2:172.21.15.100:6838/4173168440 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x55c1f7788ae0 con 0x55c1f7892bc0 2024-03-21T10:35:26.479 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:26.476+0000 7f4319600640 10 client.18738 handle_command_reply: tid=0 2024-03-21T10:35:26.479 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:26.476+0000 7f43399f6740 1 client.18738 shutdown 2024-03-21T10:35:26.479 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:26.476+0000 7f43399f6740 1 -- 172.21.15.47:0/4230192481 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c1f7826290 msgr2=0x55c1f782a090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:35:26.479 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:26.476+0000 7f43399f6740 1 --2- 172.21.15.47:0/4230192481 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c1f7826290 0x55c1f782a090 secure :-1 s=READY pgs=1294 cs=0 l=1 rev1=1 crypto rx=0x7f43280cd550 tx=0x7f432809c9b0 comp rx=0 tx=0).stop 2024-03-21T10:35:26.479 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:26.476+0000 7f43399f6740 1 -- 172.21.15.47:0/4230192481 shutdown_connections 2024-03-21T10:35:26.479 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:26.476+0000 7f43399f6740 1 --2- 172.21.15.47:0/4230192481 >> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] conn(0x55c1f7892bc0 0x55c1f7894fe0 crc :-1 s=READY pgs=222 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:35:26.479 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:26.476+0000 7f43399f6740 1 --2- 172.21.15.47:0/4230192481 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c1f7826290 0x55c1f782a090 unknown :-1 s=CLOSED pgs=1294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:35:26.479 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:26.476+0000 7f43399f6740 1 --2- 172.21.15.47:0/4230192481 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55c1f781dc00 0x55c1f782e130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:35:26.480 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:26.476+0000 7f43399f6740 1 --2- 172.21.15.47:0/4230192481 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c1f781cf20 0x55c1f782dbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:35:26.480 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:26.476+0000 7f43399f6740 1 -- 172.21.15.47:0/4230192481 >> 172.21.15.47:0/4230192481 conn(0x55c1f77e4200 msgr2=0x55c1f7816200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:35:26.480 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:26.476+0000 7f4319600640 0 client.18738 ms_handle_reset on v2:172.21.15.100:6838/4173168440 2024-03-21T10:35:26.480 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:26.476+0000 7f43399f6740 1 -- 172.21.15.47:0/4230192481 shutdown_connections 2024-03-21T10:35:26.480 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:26.476+0000 7f43399f6740 1 -- 172.21.15.47:0/4230192481 wait complete. 2024-03-21T10:35:26.480 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:26.476+0000 7f43399f6740 20 client.18738 trim_cache size 0 max 16384 2024-03-21T10:35:26.480 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-03-21T10:35:26.480 INFO:teuthology.orchestra.run.smithi047.stdout: "status": "no active scrubs running", 2024-03-21T10:35:26.480 INFO:teuthology.orchestra.run.smithi047.stdout: "scrubs": {} 2024-03-21T10:35:26.480 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-03-21T10:35:26.481 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:26.477+0000 7f4336c00640 1 -- 172.21.15.47:0/262198381 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f43140a3110 msgr2=0x7f43140a55d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:35:26.481 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:26.477+0000 7f4336c00640 1 --2- 172.21.15.47:0/262198381 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f43140a3110 0x7f43140a55d0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f4330107050 tx=0x7f431c009260 comp rx=0 tx=0).stop 2024-03-21T10:35:26.481 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:26.477+0000 7f4336c00640 1 -- 172.21.15.47:0/262198381 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f43301a2d20 msgr2=0x7f43301a31d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:35:26.481 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:26.477+0000 7f4336c00640 1 --2- 172.21.15.47:0/262198381 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f43301a2d20 0x7f43301a31d0 secure :-1 s=READY pgs=1291 cs=0 l=1 rev1=1 crypto rx=0x7f432800cab0 tx=0x7f432800cf70 comp rx=0 tx=0).stop 2024-03-21T10:35:26.482 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:26.477+0000 7f4336c00640 1 -- 172.21.15.47:0/262198381 shutdown_connections 2024-03-21T10:35:26.482 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:26.477+0000 7f4336c00640 1 --2- 172.21.15.47:0/262198381 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f43301a2d20 0x7f43301a31d0 unknown :-1 s=CLOSED pgs=1291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:35:26.482 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:26.477+0000 7f4336c00640 1 --2- 172.21.15.47:0/262198381 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f43140a3110 0x7f43140a55d0 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:35:26.482 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:26.477+0000 7f4336c00640 1 --2- 172.21.15.47:0/262198381 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f433010cf80 0x7f433019e9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:35:26.482 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:26.477+0000 7f4336c00640 1 --2- 172.21.15.47:0/262198381 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4330107eb0 0x7f433019e460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:35:26.482 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:26.477+0000 7f4336c00640 1 -- 172.21.15.47:0/262198381 >> 172.21.15.47:0/262198381 conn(0x7f4330075be0 msgr2=0x7f4330077f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:35:26.482 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:26.477+0000 7f4336c00640 1 -- 172.21.15.47:0/262198381 shutdown_connections 2024-03-21T10:35:26.482 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:26.478+0000 7f4336c00640 1 -- 172.21.15.47:0/262198381 wait complete. 2024-03-21T10:35:26.501 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-03-21T10:35:26.501 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-21T10:35:26.717 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:26.718+0000 7f170d200640 1 -- 172.21.15.47:0/262483183 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f170810e0b0 msgr2=0x7f17081104a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:35:26.717 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:26.718+0000 7f170d200640 1 --2- 172.21.15.47:0/262483183 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f170810e0b0 0x7f17081104a0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f1700009ca0 tx=0x7f170002c180 comp rx=0 tx=0).stop 2024-03-21T10:35:26.718 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:26.718+0000 7f170d200640 1 -- 172.21.15.47:0/262483183 shutdown_connections 2024-03-21T10:35:26.718 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:26.718+0000 7f170d200640 1 --2- 172.21.15.47:0/262483183 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f170810e0b0 0x7f17081104a0 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:35:26.718 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:26.718+0000 7f170d200640 1 --2- 172.21.15.47:0/262483183 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f170806a3c0 0x7f170810db70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:35:26.718 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:26.718+0000 7f170d200640 1 --2- 172.21.15.47:0/262483183 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1708069aa0 0x7f1708069e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:35:26.718 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:26.718+0000 7f170d200640 1 -- 172.21.15.47:0/262483183 >> 172.21.15.47:0/262483183 conn(0x7f17080776d0 msgr2=0x7f1708077ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:35:26.718 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:26.718+0000 7f170d200640 1 -- 172.21.15.47:0/262483183 shutdown_connections 2024-03-21T10:35:26.718 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:26.718+0000 7f170d200640 1 -- 172.21.15.47:0/262483183 wait complete. 2024-03-21T10:35:26.721 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:26.719+0000 7f170d200640 1 Processor -- start 2024-03-21T10:35:26.721 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:26.719+0000 7f170d200640 1 -- start start 2024-03-21T10:35:26.721 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:26.720+0000 7f170d200640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1708069aa0 0x7f170819a0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:35:26.721 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:26.720+0000 7f170d200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f170806a3c0 0x7f170819a600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:35:26.721 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:26.720+0000 7f170d200640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f170810e0b0 0x7f170819e930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:35:26.721 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:26.720+0000 7f170d200640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1708113b20 con 0x7f1708069aa0 2024-03-21T10:35:26.721 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:26.720+0000 7f170d200640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17081139a0 con 0x7f170806a3c0 2024-03-21T10:35:26.721 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:26.720+0000 7f170d200640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1708113ca0 con 0x7f170810e0b0 2024-03-21T10:35:26.721 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:26.720+0000 7f1705e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1708069aa0 0x7f170819a0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:35:26.722 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:26.720+0000 7f16ffe00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f170806a3c0 0x7f170819a600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:35:26.722 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:26.720+0000 7f1705e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1708069aa0 0x7f170819a0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:58680/0 (socket says 172.21.15.47:58680) 2024-03-21T10:35:26.722 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:26.720+0000 7f1705e00640 1 -- 172.21.15.47:0/2595294947 learned_addr learned my addr 172.21.15.47:0/2595294947 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:35:26.722 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:26.720+0000 7f1706800640 1 --2- 172.21.15.47:0/2595294947 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f170810e0b0 0x7f170819e930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:35:26.722 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:26.720+0000 7f1705e00640 1 -- 172.21.15.47:0/2595294947 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f170810e0b0 msgr2=0x7f170819e930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:35:26.722 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:26.720+0000 7f1705e00640 1 --2- 172.21.15.47:0/2595294947 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f170810e0b0 0x7f170819e930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:35:26.722 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:26.720+0000 7f1705e00640 1 -- 172.21.15.47:0/2595294947 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f170806a3c0 msgr2=0x7f170819a600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:35:26.722 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:26.720+0000 7f1705e00640 1 --2- 172.21.15.47:0/2595294947 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f170806a3c0 0x7f170819a600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:35:26.722 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:26.720+0000 7f1705e00640 1 -- 172.21.15.47:0/2595294947 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f170819f060 con 0x7f1708069aa0 2024-03-21T10:35:26.722 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:26.720+0000 7f16ffe00640 1 --2- 172.21.15.47:0/2595294947 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f170806a3c0 0x7f170819a600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T10:35:26.723 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:26.721+0000 7f1705e00640 1 --2- 172.21.15.47:0/2595294947 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1708069aa0 0x7f170819a0c0 secure :-1 s=READY pgs=1295 cs=0 l=1 rev1=1 crypto rx=0x7f16f800e9f0 tx=0x7f16f800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:35:26.723 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:26.721+0000 7f16fea00640 1 -- 172.21.15.47:0/2595294947 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16f800cd80 con 0x7f1708069aa0 2024-03-21T10:35:26.723 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:26.721+0000 7f16fea00640 1 -- 172.21.15.47:0/2595294947 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f16f80108e0 con 0x7f1708069aa0 2024-03-21T10:35:26.723 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:26.721+0000 7f170d200640 1 -- 172.21.15.47:0/2595294947 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f170819f350 con 0x7f1708069aa0 2024-03-21T10:35:26.723 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:26.721+0000 7f16fea00640 1 -- 172.21.15.47:0/2595294947 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16f8018920 con 0x7f1708069aa0 2024-03-21T10:35:26.723 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:26.721+0000 7f170d200640 1 -- 172.21.15.47:0/2595294947 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f17081a6be0 con 0x7f1708069aa0 2024-03-21T10:35:26.724 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:26.722+0000 7f170d200640 1 -- 172.21.15.47:0/2595294947 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1708067c00 con 0x7f1708069aa0 2024-03-21T10:35:26.724 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:26.725+0000 7f16fea00640 1 -- 172.21.15.47:0/2595294947 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 27) v1 ==== 137985+0+0 (secure 0 0 0) 0x7f16f8010a90 con 0x7f1708069aa0 2024-03-21T10:35:26.725 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:26.726+0000 7f16fea00640 1 --2- 172.21.15.47:0/2595294947 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f16ec0a3090 0x7f16ec0a5550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:35:26.726 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:26.727+0000 7f16fea00640 1 -- 172.21.15.47:0/2595294947 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(105..105 src has 1..105) v4 ==== 9562+0+0 (secure 0 0 0) 0x7f16f80d0410 con 0x7f1708069aa0 2024-03-21T10:35:26.726 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:26.727+0000 7f16ffe00640 1 --2- 172.21.15.47:0/2595294947 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f16ec0a3090 0x7f16ec0a5550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:35:26.726 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:26.728+0000 7f16ffe00640 1 --2- 172.21.15.47:0/2595294947 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f16ec0a3090 0x7f16ec0a5550 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f1708100060 tx=0x7f16f0005f90 comp rx=0 tx=0).ready entity=mgr.15687 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:35:26.728 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:26.729+0000 7f16fea00640 1 -- 172.21.15.47:0/2595294947 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f16f809caf0 con 0x7f1708069aa0 2024-03-21T10:35:26.998 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:26.999+0000 7f170d200640 1 -- 172.21.15.47:0/2595294947 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f17081092a0 con 0x7f1708069aa0 2024-03-21T10:35:26.999 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:27.000+0000 7f16fea00640 1 -- 172.21.15.47:0/2595294947 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 32 v32) v1 ==== 94+0+11471 (secure 0 0 0) 0x7f16f809c3f0 con 0x7f1708069aa0 2024-03-21T10:35:27.000 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-21T10:35:27.000 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":32,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14511,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6835/735724037","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6834","nonce":735724037},{"type":"v1","addr":"172.21.15.47:6835","nonce":735724037}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14529,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6837/2248191724","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6836","nonce":2248191724},{"type":"v1","addr":"172.21.15.47:6837","nonce":2248191724}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":32,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-21T10:21:58.329063+0000","modified":"2024-03-21T10:26:17.169892+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":34535,"mds_1":34493,"mds_2":34454,"mds_3":34427,"mds_4":14619},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14604":{"gid":14604,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.47:6839/2881335356","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6838","nonce":2881335356},{"type":"v1","addr":"172.21.15.47:6839","nonce":2881335356}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_14619":{"gid":14619,"name":"a","rank":4,"incarnation":27,"state":"up:active","state_seq":11,"addr":"172.21.15.47:6841/3401578978","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6840","nonce":3401578978},{"type":"v1","addr":"172.21.15.47:6841","nonce":3401578978}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24406":{"gid":24406,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.157:6833/4000010136","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":4000010136},{"type":"v1","addr":"172.21.15.157:6833","nonce":4000010136}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24457":{"gid":24457,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.157:6835/3902868651","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":3902868651},{"type":"v1","addr":"172.21.15.157:6835","nonce":3902868651}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24475":{"gid":24475,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.157:6837/1908087204","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6836","nonce":1908087204},{"type":"v1","addr":"172.21.15.157:6837","nonce":1908087204}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_25729":{"gid":25729,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.157:6839/1998408416","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6838","nonce":1998408416},{"type":"v1","addr":"172.21.15.157:6839","nonce":1998408416}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_34427":{"gid":34427,"name":"k","rank":3,"incarnation":25,"state":"up:active","state_seq":20,"addr":"172.21.15.100:6833/2723302038","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6832","nonce":2723302038},{"type":"v1","addr":"172.21.15.100:6833","nonce":2723302038}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_34454":{"gid":34454,"name":"e","rank":2,"incarnation":22,"state":"up:active","state_seq":17,"addr":"172.21.15.100:6835/2597849957","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":2597849957},{"type":"v1","addr":"172.21.15.100:6835","nonce":2597849957}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_34493":{"gid":34493,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":13,"addr":"172.21.15.100:6837/3497644162","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6836","nonce":3497644162},{"type":"v1","addr":"172.21.15.100:6837","nonce":3497644162}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_34535":{"gid":34535,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.100:6839/4173168440","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6838","nonce":4173168440},{"type":"v1","addr":"172.21.15.100:6839","nonce":4173168440}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":34535,"qdb_cluster":[34427,14619,34454,34535,34493]},"id":1}]} 2024-03-21T10:35:27.000 INFO:teuthology.orchestra.run.smithi047.stderr:dumped fsmap epoch 32 2024-03-21T10:35:27.002 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:27.003+0000 7f170d200640 1 -- 172.21.15.47:0/2595294947 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f16ec0a3090 msgr2=0x7f16ec0a5550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:35:27.002 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:27.003+0000 7f170d200640 1 --2- 172.21.15.47:0/2595294947 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f16ec0a3090 0x7f16ec0a5550 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f1708100060 tx=0x7f16f0005f90 comp rx=0 tx=0).stop 2024-03-21T10:35:27.002 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:27.003+0000 7f170d200640 1 -- 172.21.15.47:0/2595294947 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1708069aa0 msgr2=0x7f170819a0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:35:27.002 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:27.003+0000 7f170d200640 1 --2- 172.21.15.47:0/2595294947 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1708069aa0 0x7f170819a0c0 secure :-1 s=READY pgs=1295 cs=0 l=1 rev1=1 crypto rx=0x7f16f800e9f0 tx=0x7f16f800eeb0 comp rx=0 tx=0).stop 2024-03-21T10:35:27.002 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:27.003+0000 7f170d200640 1 -- 172.21.15.47:0/2595294947 shutdown_connections 2024-03-21T10:35:27.002 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:27.003+0000 7f170d200640 1 --2- 172.21.15.47:0/2595294947 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f170810e0b0 0x7f170819e930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:35:27.003 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:27.003+0000 7f170d200640 1 --2- 172.21.15.47:0/2595294947 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f16ec0a3090 0x7f16ec0a5550 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:35:27.003 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:27.003+0000 7f170d200640 1 --2- 172.21.15.47:0/2595294947 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f170806a3c0 0x7f170819a600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:35:27.003 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:27.003+0000 7f170d200640 1 --2- 172.21.15.47:0/2595294947 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1708069aa0 0x7f170819a0c0 unknown :-1 s=CLOSED pgs=1295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:35:27.003 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:27.003+0000 7f170d200640 1 -- 172.21.15.47:0/2595294947 >> 172.21.15.47:0/2595294947 conn(0x7f17080776d0 msgr2=0x7f170810f610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:35:27.003 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:27.003+0000 7f170d200640 1 -- 172.21.15.47:0/2595294947 shutdown_connections 2024-03-21T10:35:27.003 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:27.003+0000 7f170d200640 1 -- 172.21.15.47:0/2595294947 wait complete. 2024-03-21T10:35:27.022 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-21T10:35:27.022 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-03-21T10:35:27.250 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:27.251+0000 7f8ea9200640 1 -- 172.21.15.47:0/2355605584 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8ea40764f0 msgr2=0x7f8ea4111fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:35:27.251 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:27.251+0000 7f8ea9200640 1 --2- 172.21.15.47:0/2355605584 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8ea40764f0 0x7f8ea4111fa0 secure :-1 s=READY pgs=1296 cs=0 l=1 rev1=1 crypto rx=0x7f8e90009900 tx=0x7f8e9002bc30 comp rx=0 tx=0).stop 2024-03-21T10:35:27.251 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:27.251+0000 7f8ea9200640 1 -- 172.21.15.47:0/2355605584 shutdown_connections 2024-03-21T10:35:27.251 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:27.251+0000 7f8ea9200640 1 --2- 172.21.15.47:0/2355605584 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8ea41124e0 0x7f8ea41148d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:35:27.251 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:27.251+0000 7f8ea9200640 1 --2- 172.21.15.47:0/2355605584 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8ea40764f0 0x7f8ea4111fa0 unknown :-1 s=CLOSED pgs=1296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:35:27.251 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:27.251+0000 7f8ea9200640 1 --2- 172.21.15.47:0/2355605584 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8ea4075bd0 0x7f8ea4075fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:35:27.251 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:27.251+0000 7f8ea9200640 1 -- 172.21.15.47:0/2355605584 >> 172.21.15.47:0/2355605584 conn(0x7f8ea40ff310 msgr2=0x7f8ea4101730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:35:27.251 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:27.251+0000 7f8ea9200640 1 -- 172.21.15.47:0/2355605584 shutdown_connections 2024-03-21T10:35:27.251 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:27.251+0000 7f8ea9200640 1 -- 172.21.15.47:0/2355605584 wait complete. 2024-03-21T10:35:27.253 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:27.252+0000 7f8ea9200640 1 Processor -- start 2024-03-21T10:35:27.253 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:27.253+0000 7f8ea9200640 1 -- start start 2024-03-21T10:35:27.253 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:27.253+0000 7f8ea9200640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8ea4075bd0 0x7f8ea419e4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:35:27.253 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:27.253+0000 7f8ea9200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8ea40764f0 0x7f8ea419ea20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:35:27.253 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:27.253+0000 7f8ea9200640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8ea41124e0 0x7f8ea41a2da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:35:27.253 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:27.253+0000 7f8ea9200640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ea4117f40 con 0x7f8ea41124e0 2024-03-21T10:35:27.253 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:27.253+0000 7f8ea9200640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ea4117dc0 con 0x7f8ea40764f0 2024-03-21T10:35:27.253 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:27.253+0000 7f8ea9200640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ea41180c0 con 0x7f8ea4075bd0 2024-03-21T10:35:27.253 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:27.253+0000 7f8ea2800640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8ea41124e0 0x7f8ea41a2da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:35:27.253 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:27.253+0000 7f8ea2800640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8ea41124e0 0x7f8ea41a2da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:58708/0 (socket says 172.21.15.47:58708) 2024-03-21T10:35:27.253 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:27.253+0000 7f8ea2800640 1 -- 172.21.15.47:0/4189914259 learned_addr learned my addr 172.21.15.47:0/4189914259 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:35:27.253 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:27.254+0000 7f8ea1e00640 1 --2- 172.21.15.47:0/4189914259 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8ea4075bd0 0x7f8ea419e4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:35:27.254 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:27.254+0000 7f8ea1400640 1 --2- 172.21.15.47:0/4189914259 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8ea40764f0 0x7f8ea419ea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:35:27.254 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:27.254+0000 7f8ea2800640 1 -- 172.21.15.47:0/4189914259 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8ea4075bd0 msgr2=0x7f8ea419e4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:35:27.254 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:27.254+0000 7f8ea2800640 1 --2- 172.21.15.47:0/4189914259 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8ea4075bd0 0x7f8ea419e4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:35:27.254 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:27.254+0000 7f8ea2800640 1 -- 172.21.15.47:0/4189914259 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8ea40764f0 msgr2=0x7f8ea419ea20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:35:27.254 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:27.254+0000 7f8ea2800640 1 --2- 172.21.15.47:0/4189914259 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8ea40764f0 0x7f8ea419ea20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:35:27.254 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:27.254+0000 7f8ea2800640 1 -- 172.21.15.47:0/4189914259 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ea41a34d0 con 0x7f8ea41124e0 2024-03-21T10:35:27.254 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:27.254+0000 7f8ea2800640 1 --2- 172.21.15.47:0/4189914259 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8ea41124e0 0x7f8ea41a2da0 secure :-1 s=READY pgs=1297 cs=0 l=1 rev1=1 crypto rx=0x7f8e9c00eb40 tx=0x7f8e9c00c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:35:27.255 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:27.254+0000 7f8e9aa00640 1 -- 172.21.15.47:0/4189914259 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e9c004b20 con 0x7f8ea41124e0 2024-03-21T10:35:27.255 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:27.254+0000 7f8e9aa00640 1 -- 172.21.15.47:0/4189914259 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f8e9c004cc0 con 0x7f8ea41124e0 2024-03-21T10:35:27.255 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:27.254+0000 7f8e9aa00640 1 -- 172.21.15.47:0/4189914259 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e9c0107b0 con 0x7f8ea41124e0 2024-03-21T10:35:27.255 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:27.254+0000 7f8ea9200640 1 -- 172.21.15.47:0/4189914259 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8ea41a37c0 con 0x7f8ea41124e0 2024-03-21T10:35:27.255 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:27.255+0000 7f8ea9200640 1 -- 172.21.15.47:0/4189914259 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8ea41ab050 con 0x7f8ea41124e0 2024-03-21T10:35:27.257 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:27.259+0000 7f8eabfd5740 1 Processor -- start 2024-03-21T10:35:27.258 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:27.259+0000 7f8e9aa00640 1 -- 172.21.15.47:0/4189914259 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 27) v1 ==== 137985+0+0 (secure 0 0 0) 0x7f8e9c005020 con 0x7f8ea41124e0 2024-03-21T10:35:27.258 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:27.259+0000 7f8eabfd5740 1 -- start start 2024-03-21T10:35:27.259 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:27.260+0000 7f8eabfd5740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55cfd5c536d0 0x55cfd5c53ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:35:27.259 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:27.260+0000 7f8eabfd5740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55cfd5c54080 0x55cfd5c5c630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:35:27.259 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:27.260+0000 7f8eabfd5740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55cfd5c5cc00 0x55cfd5c5f050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:35:27.259 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:27.260+0000 7f8eabfd5740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cfd5bb72c0 con 0x55cfd5c5cc00 2024-03-21T10:35:27.259 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:27.260+0000 7f8eabfd5740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cfd5b5fc90 con 0x55cfd5c54080 2024-03-21T10:35:27.259 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:27.260+0000 7f8eabfd5740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cfd5bb3360 con 0x55cfd5c536d0 2024-03-21T10:35:27.260 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:27.260+0000 7f8ea1e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55cfd5c54080 0x55cfd5c5c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:35:27.260 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:27.260+0000 7f8ea1400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55cfd5c5cc00 0x55cfd5c5f050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:35:27.261 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:27.260+0000 7f8ea1e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55cfd5c54080 0x55cfd5c5c630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:59878/0 (socket says 172.21.15.47:59878) 2024-03-21T10:35:27.261 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:27.260+0000 7f8ea1e00640 1 -- 172.21.15.47:0/2383365134 learned_addr learned my addr 172.21.15.47:0/2383365134 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:35:27.261 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:27.260+0000 7f8ea1400640 1 --2- 172.21.15.47:0/2383365134 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55cfd5c536d0 0x55cfd5c53ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:35:27.261 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:27.261+0000 7f8ea1e00640 1 -- 172.21.15.47:0/2383365134 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55cfd5c536d0 msgr2=0x55cfd5c53ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:35:27.261 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:27.261+0000 7f8ea1e00640 1 --2- 172.21.15.47:0/2383365134 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55cfd5c536d0 0x55cfd5c53ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:35:27.261 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:27.261+0000 7f8ea1e00640 1 -- 172.21.15.47:0/2383365134 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55cfd5c5cc00 msgr2=0x55cfd5c5f050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:35:27.261 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:27.261+0000 7f8ea1e00640 1 --2- 172.21.15.47:0/2383365134 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55cfd5c5cc00 0x55cfd5c5f050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:35:27.261 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:27.261+0000 7f8ea1e00640 1 -- 172.21.15.47:0/2383365134 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55cfd5b84080 con 0x55cfd5c54080 2024-03-21T10:35:27.261 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:27.261+0000 7f8e9aa00640 1 --2- 172.21.15.47:0/4189914259 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f8e880a3090 0x7f8e880a5550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:35:27.261 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:27.261+0000 7f8ea1e00640 1 --2- 172.21.15.47:0/2383365134 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55cfd5c54080 0x55cfd5c5c630 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f8e94009c90 tx=0x7f8e940117b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8f2459e5fa850cdb server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:35:27.261 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:27.261+0000 7f8e9aa00640 1 -- 172.21.15.47:0/4189914259 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(105..105 src has 1..105) v4 ==== 9562+0+0 (secure 0 0 0) 0x7f8e9c0d04c0 con 0x7f8ea41124e0 2024-03-21T10:35:27.261 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:27.261+0000 7f8ea1400640 1 --2- 172.21.15.47:0/4189914259 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f8e880a3090 0x7f8e880a5550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:35:27.261 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:27.261+0000 7f8e8f400640 1 -- 172.21.15.47:0/2383365134 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e94012450 con 0x55cfd5c54080 2024-03-21T10:35:27.261 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:27.262+0000 7f8e8f400640 1 -- 172.21.15.47:0/2383365134 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f8e94012aa0 con 0x55cfd5c54080 2024-03-21T10:35:27.261 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:27.262+0000 7f8e8f400640 1 -- 172.21.15.47:0/2383365134 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e9400fb50 con 0x55cfd5c54080 2024-03-21T10:35:27.264 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:27.262+0000 7f8eabfd5740 1 -- 172.21.15.47:0/2383365134 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55cfd5c54080 msgr2=0x55cfd5c5c630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:35:27.264 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:27.262+0000 7f8eabfd5740 1 --2- 172.21.15.47:0/2383365134 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55cfd5c54080 0x55cfd5c5c630 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f8e94009c90 tx=0x7f8e940117b0 comp rx=0 tx=0).stop 2024-03-21T10:35:27.264 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:27.262+0000 7f8ea1400640 1 --2- 172.21.15.47:0/4189914259 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f8e880a3090 0x7f8e880a5550 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f8e90006010 tx=0x7f8e90005a20 comp rx=0 tx=0).ready entity=mgr.15687 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:35:27.264 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:27.262+0000 7f8eabfd5740 1 -- 172.21.15.47:0/2383365134 shutdown_connections 2024-03-21T10:35:27.264 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:27.262+0000 7f8eabfd5740 1 --2- 172.21.15.47:0/2383365134 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55cfd5c5cc00 0x55cfd5c5f050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:35:27.264 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:27.262+0000 7f8eabfd5740 1 --2- 172.21.15.47:0/2383365134 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55cfd5c54080 0x55cfd5c5c630 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:35:27.264 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:27.262+0000 7f8eabfd5740 1 --2- 172.21.15.47:0/2383365134 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55cfd5c536d0 0x55cfd5c53ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:35:27.264 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:27.262+0000 7f8eabfd5740 1 -- 172.21.15.47:0/2383365134 >> 172.21.15.47:0/2383365134 conn(0x55cfd5c4dda0 msgr2=0x55cfd5c4e180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:35:27.264 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:27.263+0000 7f8eabfd5740 1 -- 172.21.15.47:0/2383365134 shutdown_connections 2024-03-21T10:35:27.264 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:27.263+0000 7f8eabfd5740 1 -- 172.21.15.47:0/2383365134 wait complete. 2024-03-21T10:35:27.264 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:27.263+0000 7f8eabfd5740 1 -- start start 2024-03-21T10:35:27.264 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:27.263+0000 7f8eabfd5740 1 Processor -- start 2024-03-21T10:35:27.264 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:27.264+0000 7f8eabfd5740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55cfd5c536d0 0x55cfd5c5c4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:35:27.264 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:27.264+0000 7f8eabfd5740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55cfd5c54080 0x55cfd5c59840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:35:27.265 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:27.264+0000 7f8eabfd5740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55cfd5c5cc00 0x55cfd5c59d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:35:27.265 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:27.264+0000 7f8eabfd5740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cfd5c5fb70 con 0x55cfd5c536d0 2024-03-21T10:35:27.265 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:27.264+0000 7f8eabfd5740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cfd5b730d0 con 0x55cfd5c54080 2024-03-21T10:35:27.265 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:27.264+0000 7f8eabfd5740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cfd5c5fcf0 con 0x55cfd5c5cc00 2024-03-21T10:35:27.265 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:27.264+0000 7f8eabfd5740 10 client.0 _ll_register_callbacks cb 0x55cfd5b77e80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T10:35:27.265 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:27.264+0000 7f8ea1e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55cfd5c536d0 0x55cfd5c5c4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:35:27.265 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:27.264+0000 7f8ea1e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55cfd5c536d0 0x55cfd5c5c4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:58716/0 (socket says 172.21.15.47:58716) 2024-03-21T10:35:27.265 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:27.264+0000 7f8ea1e00640 1 -- 172.21.15.47:0/3019370679 learned_addr learned my addr 172.21.15.47:0/3019370679 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:35:27.265 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:27.264+0000 7f8ea1400640 1 --2- 172.21.15.47:0/3019370679 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55cfd5c54080 0x55cfd5c59840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:35:27.265 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:27.264+0000 7f8ea2800640 1 --2- 172.21.15.47:0/3019370679 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55cfd5c5cc00 0x55cfd5c59d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:35:27.265 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:27.265+0000 7f8ea1e00640 1 -- 172.21.15.47:0/3019370679 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55cfd5c5cc00 msgr2=0x55cfd5c59d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:35:27.265 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:27.265+0000 7f8ea1e00640 1 --2- 172.21.15.47:0/3019370679 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55cfd5c5cc00 0x55cfd5c59d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:35:27.265 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:27.265+0000 7f8ea1e00640 1 -- 172.21.15.47:0/3019370679 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55cfd5c54080 msgr2=0x55cfd5c59840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:35:27.265 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:27.265+0000 7f8ea1e00640 1 --2- 172.21.15.47:0/3019370679 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55cfd5c54080 0x55cfd5c59840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:35:27.266 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:27.265+0000 7f8ea1e00640 1 -- 172.21.15.47:0/3019370679 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55cfd58cb3f0 con 0x55cfd5c536d0 2024-03-21T10:35:27.267 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:27.265+0000 7f8ea1e00640 1 --2- 172.21.15.47:0/3019370679 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55cfd5c536d0 0x55cfd5c5c4b0 secure :-1 s=READY pgs=1298 cs=0 l=1 rev1=1 crypto rx=0x7f8e9401c040 tx=0x7f8e9400fac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cd4679c566b2a6a4 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:35:27.267 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:27.265+0000 7f8e83e00640 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-03-21T10:35:27.267 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:27.265+0000 7f8e83e00640 1 -- 172.21.15.47:0/3019370679 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e94029020 con 0x55cfd5c536d0 2024-03-21T10:35:27.268 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:27.266+0000 7f8e83e00640 1 -- 172.21.15.47:0/3019370679 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(105..105 src has 1..105) v4 ==== 9562+0+0 (secure 0 0 0) 0x7f8e94009d80 con 0x55cfd5c536d0 2024-03-21T10:35:27.268 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:27.266+0000 7f8eabfd5740 1 -- 172.21.15.47:0/3019370679 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f8e94012aa0 con 0x55cfd5c536d0 2024-03-21T10:35:27.268 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:27.266+0000 7f8e83e00640 1 -- 172.21.15.47:0/3019370679 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8e94034050 con 0x55cfd5c536d0 2024-03-21T10:35:27.268 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:27.266+0000 7f8eabfd5740 10 client.18756 fetch_fsmap learned FSMap version 32 2024-03-21T10:35:27.268 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:27.266+0000 7f8eabfd5740 1 -- 172.21.15.47:0/3019370679 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x55cfd59957d0 con 0x55cfd5c536d0 2024-03-21T10:35:27.268 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:27.266+0000 7f8e83e00640 1 -- 172.21.15.47:0/3019370679 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f8e94012aa0 con 0x55cfd5c536d0 2024-03-21T10:35:27.268 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:27.266+0000 7f8e83e00640 1 -- 172.21.15.47:0/3019370679 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 32) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f8e94004680 con 0x55cfd5c536d0 2024-03-21T10:35:27.268 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:27.267+0000 7f8eabfd5740 10 client.18756 fetch_fsmap finished waiting for FSMap version 32 2024-03-21T10:35:27.268 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:27.267+0000 7f8eabfd5740 10 client.18756 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.a 2024-03-21T10:35:27.268 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:27.267+0000 7f8eabfd5740 20 client.18756 populate_metadata read hostname 'smithi047' 2024-03-21T10:35:27.268 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:27.267+0000 7f8eabfd5740 1 --2- 172.21.15.47:0/3019370679 >> [v2:172.21.15.47:6840/3401578978,v1:172.21.15.47:6841/3401578978] conn(0x55cfd5c6c9f0 0x55cfd5c6ede0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:35:27.268 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:27.267+0000 7f8eabfd5740 4 client.18756 mds_command: new command op to 14619 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T10:35:27.268 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:27.267+0000 7f8eabfd5740 1 -- 172.21.15.47:0/3019370679 --> [v2:172.21.15.47:6840/3401578978,v1:172.21.15.47:6841/3401578978] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55cfd5bc4800 con 0x55cfd5c6c9f0 2024-03-21T10:35:27.268 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:27.267+0000 7f8ea1400640 1 --2- 172.21.15.47:0/3019370679 >> [v2:172.21.15.47:6840/3401578978,v1:172.21.15.47:6841/3401578978] conn(0x55cfd5c6c9f0 0x55cfd5c6ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:35:27.268 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:27.268+0000 7f8ea1400640 1 --2- 172.21.15.47:0/3019370679 >> [v2:172.21.15.47:6840/3401578978,v1:172.21.15.47:6841/3401578978] conn(0x55cfd5c6c9f0 0x55cfd5c6ede0 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=92d1940baf59935 server_cookie=634ccbcd007ec01 in_seq=0 out_seq=0 2024-03-21T10:35:27.269 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:27.268+0000 7f8e83e00640 10 client.18756 ms_handle_connect on v2:172.21.15.47:6840/3401578978 2024-03-21T10:35:27.269 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:27.268+0000 7f8e83e00640 1 -- 172.21.15.47:0/3019370679 <== mds.4 v2:172.21.15.47:6840/3401578978 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55cfd5bc4800 con 0x55cfd5c6c9f0 2024-03-21T10:35:27.269 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:27.268+0000 7f8e83e00640 10 client.18756 handle_command_reply: tid=0 2024-03-21T10:35:27.269 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:27.268+0000 7f8eabfd5740 1 client.18756 shutdown 2024-03-21T10:35:27.269 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:27.269+0000 7f8eabfd5740 1 -- 172.21.15.47:0/3019370679 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55cfd5c536d0 msgr2=0x55cfd5c5c4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:35:27.269 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:27.269+0000 7f8eabfd5740 1 --2- 172.21.15.47:0/3019370679 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55cfd5c536d0 0x55cfd5c5c4b0 secure :-1 s=READY pgs=1298 cs=0 l=1 rev1=1 crypto rx=0x7f8e9401c040 tx=0x7f8e9400fac0 comp rx=0 tx=0).stop 2024-03-21T10:35:27.270 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:27.269+0000 7f8eabfd5740 1 -- 172.21.15.47:0/3019370679 shutdown_connections 2024-03-21T10:35:27.270 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:27.269+0000 7f8eabfd5740 1 --2- 172.21.15.47:0/3019370679 >> [v2:172.21.15.47:6840/3401578978,v1:172.21.15.47:6841/3401578978] conn(0x55cfd5c6c9f0 0x55cfd5c6ede0 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:35:27.270 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:27.269+0000 7f8eabfd5740 1 --2- 172.21.15.47:0/3019370679 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55cfd5c5cc00 0x55cfd5c59d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:35:27.270 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:27.269+0000 7f8eabfd5740 1 --2- 172.21.15.47:0/3019370679 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55cfd5c54080 0x55cfd5c59840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:35:27.270 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:27.269+0000 7f8eabfd5740 1 --2- 172.21.15.47:0/3019370679 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55cfd5c536d0 0x55cfd5c5c4b0 unknown :-1 s=CLOSED pgs=1298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:35:27.270 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:27.269+0000 7f8eabfd5740 1 -- 172.21.15.47:0/3019370679 >> 172.21.15.47:0/3019370679 conn(0x55cfd5c4dda0 msgr2=0x55cfd5c4faf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:35:27.270 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:27.269+0000 7f8e83e00640 0 client.18756 ms_handle_reset on v2:172.21.15.47:6840/3401578978 2024-03-21T10:35:27.270 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:27.269+0000 7f8eabfd5740 1 -- 172.21.15.47:0/3019370679 shutdown_connections 2024-03-21T10:35:27.270 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:27.269+0000 7f8eabfd5740 1 -- 172.21.15.47:0/3019370679 wait complete. 2024-03-21T10:35:27.270 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:27.269+0000 7f8eabfd5740 20 client.18756 trim_cache size 0 max 16384 2024-03-21T10:35:27.283 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:27.283+0000 7f8eabfd5740 1 Processor -- start 2024-03-21T10:35:27.283 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:27.283+0000 7f8eabfd5740 1 -- start start 2024-03-21T10:35:27.283 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:27.284+0000 7f8eabfd5740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55cfd5c536d0 0x55cfd5c550b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:35:27.283 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:27.284+0000 7f8eabfd5740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55cfd5c54080 0x55cfd5c59120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:35:27.283 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:27.284+0000 7f8eabfd5740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55cfd5c5cc00 0x55cfd5c5bf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:35:27.283 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:27.284+0000 7f8eabfd5740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ea41b5050 con 0x55cfd5c5cc00 2024-03-21T10:35:27.283 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:27.284+0000 7f8eabfd5740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ea41b69d0 con 0x55cfd5c536d0 2024-03-21T10:35:27.284 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:27.284+0000 7f8eabfd5740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ea41b56b0 con 0x55cfd5c54080 2024-03-21T10:35:27.284 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:27.284+0000 7f8ea2800640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55cfd5c5cc00 0x55cfd5c5bf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:35:27.284 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:27.284+0000 7f8ea2800640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55cfd5c5cc00 0x55cfd5c5bf70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:58730/0 (socket says 172.21.15.47:58730) 2024-03-21T10:35:27.284 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:27.284+0000 7f8ea2800640 1 -- 172.21.15.47:0/854594165 learned_addr learned my addr 172.21.15.47:0/854594165 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:35:27.284 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:27.284+0000 7f8ea1e00640 1 --2- 172.21.15.47:0/854594165 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55cfd5c536d0 0x55cfd5c550b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:35:27.284 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:27.284+0000 7f8ea1400640 1 --2- 172.21.15.47:0/854594165 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55cfd5c54080 0x55cfd5c59120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:35:27.284 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:27.284+0000 7f8ea2800640 1 -- 172.21.15.47:0/854594165 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55cfd5c54080 msgr2=0x55cfd5c59120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:35:27.284 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:27.284+0000 7f8ea2800640 1 --2- 172.21.15.47:0/854594165 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55cfd5c54080 0x55cfd5c59120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:35:27.284 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:27.284+0000 7f8ea2800640 1 -- 172.21.15.47:0/854594165 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55cfd5c536d0 msgr2=0x55cfd5c550b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:35:27.284 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:27.284+0000 7f8ea2800640 1 --2- 172.21.15.47:0/854594165 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55cfd5c536d0 0x55cfd5c550b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:35:27.284 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:27.284+0000 7f8ea2800640 1 -- 172.21.15.47:0/854594165 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55cfd5c745f0 con 0x55cfd5c5cc00 2024-03-21T10:35:27.287 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:27.285+0000 7f8ea2800640 1 --2- 172.21.15.47:0/854594165 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55cfd5c5cc00 0x55cfd5c5bf70 secure :-1 s=READY pgs=1299 cs=0 l=1 rev1=1 crypto rx=0x7f8e9c0cd800 tx=0x7f8e9c09c040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=df8c5b3ff147964b server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:35:27.287 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:27.285+0000 7f8e8ea00640 1 -- 172.21.15.47:0/854594165 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e9c0b4020 con 0x55cfd5c5cc00 2024-03-21T10:35:27.287 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:27.285+0000 7f8e8ea00640 1 -- 172.21.15.47:0/854594165 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f8e9c09cb40 con 0x55cfd5c5cc00 2024-03-21T10:35:27.287 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:27.285+0000 7f8e8ea00640 1 -- 172.21.15.47:0/854594165 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e9c0d0150 con 0x55cfd5c5cc00 2024-03-21T10:35:27.287 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:27.286+0000 7f8eabfd5740 1 -- 172.21.15.47:0/854594165 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55cfd5c5cc00 msgr2=0x55cfd5c5bf70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:35:27.287 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:27.286+0000 7f8eabfd5740 1 --2- 172.21.15.47:0/854594165 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55cfd5c5cc00 0x55cfd5c5bf70 secure :-1 s=READY pgs=1299 cs=0 l=1 rev1=1 crypto rx=0x7f8e9c0cd800 tx=0x7f8e9c09c040 comp rx=0 tx=0).stop 2024-03-21T10:35:27.287 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:27.286+0000 7f8eabfd5740 1 -- 172.21.15.47:0/854594165 shutdown_connections 2024-03-21T10:35:27.287 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:27.286+0000 7f8eabfd5740 1 --2- 172.21.15.47:0/854594165 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55cfd5c5cc00 0x55cfd5c5bf70 unknown :-1 s=CLOSED pgs=1299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:35:27.287 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:27.286+0000 7f8eabfd5740 1 --2- 172.21.15.47:0/854594165 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55cfd5c54080 0x55cfd5c59120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:35:27.287 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:27.286+0000 7f8eabfd5740 1 --2- 172.21.15.47:0/854594165 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55cfd5c536d0 0x55cfd5c550b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:35:27.287 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:27.286+0000 7f8eabfd5740 1 -- 172.21.15.47:0/854594165 >> 172.21.15.47:0/854594165 conn(0x55cfd5c4dda0 msgr2=0x55cfd5c4ceb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:35:27.288 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:27.286+0000 7f8eabfd5740 1 -- 172.21.15.47:0/854594165 shutdown_connections 2024-03-21T10:35:27.288 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:27.286+0000 7f8eabfd5740 1 -- 172.21.15.47:0/854594165 wait complete. 2024-03-21T10:35:27.288 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:27.286+0000 7f8eabfd5740 1 -- start start 2024-03-21T10:35:27.288 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:27.286+0000 7f8eabfd5740 1 Processor -- start 2024-03-21T10:35:27.288 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:27.287+0000 7f8eabfd5740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55cfd5c536d0 0x55cfd5c6f930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:35:27.288 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:27.287+0000 7f8eabfd5740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55cfd5c54080 0x55cfd5c6fe70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:35:27.288 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:27.287+0000 7f8eabfd5740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55cfd5c5cc00 0x55cfd5cc6f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:35:27.288 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:27.287+0000 7f8eabfd5740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ea41b6370 con 0x55cfd5c536d0 2024-03-21T10:35:27.288 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:27.287+0000 7f8eabfd5740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ea41b5d10 con 0x55cfd5c54080 2024-03-21T10:35:27.288 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:27.287+0000 7f8eabfd5740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ea41b3230 con 0x55cfd5c5cc00 2024-03-21T10:35:27.288 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:27.287+0000 7f8eabfd5740 10 client.0 _ll_register_callbacks cb 0x7f8e94009ae0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T10:35:27.288 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:27.287+0000 7f8ea1e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55cfd5c536d0 0x55cfd5c6f930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:35:27.288 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:27.287+0000 7f8ea1e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55cfd5c536d0 0x55cfd5c6f930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:58742/0 (socket says 172.21.15.47:58742) 2024-03-21T10:35:27.288 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:27.287+0000 7f8ea1e00640 1 -- 172.21.15.47:0/1744499493 learned_addr learned my addr 172.21.15.47:0/1744499493 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:35:27.289 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:27.287+0000 7f8ea1400640 1 --2- 172.21.15.47:0/1744499493 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55cfd5c54080 0x55cfd5c6fe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:35:27.289 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:27.287+0000 7f8ea2800640 1 --2- 172.21.15.47:0/1744499493 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55cfd5c5cc00 0x55cfd5cc6f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:35:27.289 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:27.288+0000 7f8ea1e00640 1 -- 172.21.15.47:0/1744499493 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55cfd5c5cc00 msgr2=0x55cfd5cc6f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:35:27.289 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:27.288+0000 7f8ea1e00640 1 --2- 172.21.15.47:0/1744499493 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55cfd5c5cc00 0x55cfd5cc6f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:35:27.289 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:27.288+0000 7f8ea1e00640 1 -- 172.21.15.47:0/1744499493 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55cfd5c54080 msgr2=0x55cfd5c6fe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:35:27.289 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:27.288+0000 7f8ea1e00640 1 --2- 172.21.15.47:0/1744499493 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55cfd5c54080 0x55cfd5c6fe70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:35:27.289 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:27.288+0000 7f8ea1e00640 1 -- 172.21.15.47:0/1744499493 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55cfd5cc7460 con 0x55cfd5c536d0 2024-03-21T10:35:27.289 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:27.288+0000 7f8ea1e00640 1 --2- 172.21.15.47:0/1744499493 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55cfd5c536d0 0x55cfd5c6f930 secure :-1 s=READY pgs=1300 cs=0 l=1 rev1=1 crypto rx=0x7f8e94009bb0 tx=0x7f8e94011de0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d776e6e4a089190d server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:35:27.289 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:27.288+0000 7f8e83e00640 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-03-21T10:35:27.291 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:27.288+0000 7f8e83e00640 1 -- 172.21.15.47:0/1744499493 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e9403c020 con 0x55cfd5c536d0 2024-03-21T10:35:27.292 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:27.288+0000 7f8eabfd5740 1 -- 172.21.15.47:0/1744499493 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f8e9c09cb40 con 0x55cfd5c536d0 2024-03-21T10:35:27.292 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:27.288+0000 7f8e83e00640 1 -- 172.21.15.47:0/1744499493 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(105..105 src has 1..105) v4 ==== 9562+0+0 (secure 0 0 0) 0x7f8e94004c40 con 0x55cfd5c536d0 2024-03-21T10:35:27.292 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:27.289+0000 7f8e83e00640 1 -- 172.21.15.47:0/1744499493 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8e94047050 con 0x55cfd5c536d0 2024-03-21T10:35:27.292 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:27.289+0000 7f8eabfd5740 10 client.18762 fetch_fsmap learned FSMap version 32 2024-03-21T10:35:27.292 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:27.289+0000 7f8eabfd5740 1 -- 172.21.15.47:0/1744499493 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x55cfd5cc8700 con 0x55cfd5c536d0 2024-03-21T10:35:27.292 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:27.289+0000 7f8e83e00640 1 -- 172.21.15.47:0/1744499493 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f8e9c09cb40 con 0x55cfd5c536d0 2024-03-21T10:35:27.292 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:27.289+0000 7f8e83e00640 1 -- 172.21.15.47:0/1744499493 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 32) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f8e94004140 con 0x55cfd5c536d0 2024-03-21T10:35:27.292 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:27.289+0000 7f8eabfd5740 10 client.18762 fetch_fsmap finished waiting for FSMap version 32 2024-03-21T10:35:27.292 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:27.289+0000 7f8eabfd5740 10 client.18762 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.a 2024-03-21T10:35:27.292 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:27.289+0000 7f8eabfd5740 20 client.18762 populate_metadata read hostname 'smithi047' 2024-03-21T10:35:27.292 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:27.289+0000 7f8eabfd5740 1 --2- 172.21.15.47:0/1744499493 >> [v2:172.21.15.47:6840/3401578978,v1:172.21.15.47:6841/3401578978] conn(0x55cfd5cc8dc0 0x55cfd5ccb1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:35:27.292 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:27.289+0000 7f8eabfd5740 4 client.18762 mds_command: new command op to 14619 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-21T10:35:27.292 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:27.289+0000 7f8eabfd5740 1 -- 172.21.15.47:0/1744499493 --> [v2:172.21.15.47:6840/3401578978,v1:172.21.15.47:6841/3401578978] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55cfd5bbeb50 con 0x55cfd5cc8dc0 2024-03-21T10:35:27.293 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:27.290+0000 7f8ea1400640 1 --2- 172.21.15.47:0/1744499493 >> [v2:172.21.15.47:6840/3401578978,v1:172.21.15.47:6841/3401578978] conn(0x55cfd5cc8dc0 0x55cfd5ccb1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:35:27.293 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:27.290+0000 7f8ea1400640 1 --2- 172.21.15.47:0/1744499493 >> [v2:172.21.15.47:6840/3401578978,v1:172.21.15.47:6841/3401578978] conn(0x55cfd5cc8dc0 0x55cfd5ccb1e0 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=8457019ec6fffa7 server_cookie=3c1c099f39082790 in_seq=0 out_seq=0 2024-03-21T10:35:27.293 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:27.290+0000 7f8e83e00640 10 client.18762 ms_handle_connect on v2:172.21.15.47:6840/3401578978 2024-03-21T10:35:27.293 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:27.290+0000 7f8e83e00640 1 -- 172.21.15.47:0/1744499493 <== mds.4 v2:172.21.15.47:6840/3401578978 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55cfd5bbeb50 con 0x55cfd5cc8dc0 2024-03-21T10:35:27.293 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:27.290+0000 7f8e83e00640 10 client.18762 handle_command_reply: tid=0 2024-03-21T10:35:27.293 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:27.291+0000 7f8eabfd5740 1 client.18762 shutdown 2024-03-21T10:35:27.293 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:27.291+0000 7f8eabfd5740 1 -- 172.21.15.47:0/1744499493 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55cfd5c536d0 msgr2=0x55cfd5c6f930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:35:27.293 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:27.291+0000 7f8eabfd5740 1 --2- 172.21.15.47:0/1744499493 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55cfd5c536d0 0x55cfd5c6f930 secure :-1 s=READY pgs=1300 cs=0 l=1 rev1=1 crypto rx=0x7f8e94009bb0 tx=0x7f8e94011de0 comp rx=0 tx=0).stop 2024-03-21T10:35:27.293 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:27.291+0000 7f8eabfd5740 1 -- 172.21.15.47:0/1744499493 shutdown_connections 2024-03-21T10:35:27.293 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:27.291+0000 7f8eabfd5740 1 --2- 172.21.15.47:0/1744499493 >> [v2:172.21.15.47:6840/3401578978,v1:172.21.15.47:6841/3401578978] conn(0x55cfd5cc8dc0 0x55cfd5ccb1e0 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:35:27.293 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:27.291+0000 7f8eabfd5740 1 --2- 172.21.15.47:0/1744499493 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55cfd5c5cc00 0x55cfd5cc6f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:35:27.293 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:27.291+0000 7f8eabfd5740 1 --2- 172.21.15.47:0/1744499493 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55cfd5c54080 0x55cfd5c6fe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:35:27.293 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:27.291+0000 7f8eabfd5740 1 --2- 172.21.15.47:0/1744499493 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55cfd5c536d0 0x55cfd5c6f930 unknown :-1 s=CLOSED pgs=1300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:35:27.293 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:27.291+0000 7f8e83e00640 0 client.18762 ms_handle_reset on v2:172.21.15.47:6840/3401578978 2024-03-21T10:35:27.294 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:27.291+0000 7f8eabfd5740 1 -- 172.21.15.47:0/1744499493 >> 172.21.15.47:0/1744499493 conn(0x55cfd5c4dda0 msgr2=0x55cfd5c4ef40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:35:27.294 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:27.291+0000 7f8eabfd5740 1 -- 172.21.15.47:0/1744499493 shutdown_connections 2024-03-21T10:35:27.294 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:27.291+0000 7f8eabfd5740 1 -- 172.21.15.47:0/1744499493 wait complete. 2024-03-21T10:35:27.294 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:27.291+0000 7f8eabfd5740 20 client.18762 trim_cache size 0 max 16384 2024-03-21T10:35:27.294 INFO:teuthology.orchestra.run.smithi047.stdout:[] 2024-03-21T10:35:27.295 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:27.292+0000 7f8ea9200640 1 -- 172.21.15.47:0/4189914259 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f8e880a3090 msgr2=0x7f8e880a5550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:35:27.295 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:27.292+0000 7f8ea9200640 1 --2- 172.21.15.47:0/4189914259 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f8e880a3090 0x7f8e880a5550 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f8e90006010 tx=0x7f8e90005a20 comp rx=0 tx=0).stop 2024-03-21T10:35:27.295 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:27.292+0000 7f8ea9200640 1 -- 172.21.15.47:0/4189914259 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8ea41124e0 msgr2=0x7f8ea41a2da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:35:27.295 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:27.292+0000 7f8ea9200640 1 --2- 172.21.15.47:0/4189914259 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8ea41124e0 0x7f8ea41a2da0 secure :-1 s=READY pgs=1297 cs=0 l=1 rev1=1 crypto rx=0x7f8e9c00eb40 tx=0x7f8e9c00c2d0 comp rx=0 tx=0).stop 2024-03-21T10:35:27.295 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:27.292+0000 7f8ea9200640 1 -- 172.21.15.47:0/4189914259 shutdown_connections 2024-03-21T10:35:27.295 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:27.292+0000 7f8ea9200640 1 --2- 172.21.15.47:0/4189914259 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8ea41124e0 0x7f8ea41a2da0 unknown :-1 s=CLOSED pgs=1297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:35:27.295 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:27.292+0000 7f8ea9200640 1 --2- 172.21.15.47:0/4189914259 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f8e880a3090 0x7f8e880a5550 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:35:27.295 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:27.292+0000 7f8ea9200640 1 --2- 172.21.15.47:0/4189914259 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8ea40764f0 0x7f8ea419ea20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:35:27.295 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:27.292+0000 7f8ea9200640 1 --2- 172.21.15.47:0/4189914259 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8ea4075bd0 0x7f8ea419e4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:35:27.296 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:27.292+0000 7f8ea9200640 1 -- 172.21.15.47:0/4189914259 >> 172.21.15.47:0/4189914259 conn(0x7f8ea40ff310 msgr2=0x7f8ea4113c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:35:27.296 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:27.292+0000 7f8ea9200640 1 -- 172.21.15.47:0/4189914259 shutdown_connections 2024-03-21T10:35:27.296 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:27.292+0000 7f8ea9200640 1 -- 172.21.15.47:0/4189914259 wait complete. 2024-03-21T10:35:27.308 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-21T10:35:27.308 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-03-21T10:35:27.509 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:27.510+0000 7f4157200640 1 -- 172.21.15.47:0/3827330698 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4150103250 msgr2=0x7f415010b730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:35:27.509 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:27.510+0000 7f4157200640 1 --2- 172.21.15.47:0/3827330698 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4150103250 0x7f415010b730 secure :-1 s=READY pgs=1301 cs=0 l=1 rev1=1 crypto rx=0x7f413c009860 tx=0x7f413c02bc00 comp rx=0 tx=0).stop 2024-03-21T10:35:27.510 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:27.510+0000 7f4157200640 1 -- 172.21.15.47:0/3827330698 shutdown_connections 2024-03-21T10:35:27.510 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:27.510+0000 7f4157200640 1 --2- 172.21.15.47:0/3827330698 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f415010bd00 0x7f415010e130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:35:27.510 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:27.510+0000 7f4157200640 1 --2- 172.21.15.47:0/3827330698 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4150103250 0x7f415010b730 unknown :-1 s=CLOSED pgs=1301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:35:27.510 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:27.510+0000 7f4157200640 1 --2- 172.21.15.47:0/3827330698 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4150102930 0x7f4150102d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:35:27.510 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:27.510+0000 7f4157200640 1 -- 172.21.15.47:0/3827330698 >> 172.21.15.47:0/3827330698 conn(0x7f4150077700 msgr2=0x7f4150077b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:35:27.510 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:27.510+0000 7f4157200640 1 -- 172.21.15.47:0/3827330698 shutdown_connections 2024-03-21T10:35:27.510 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:27.510+0000 7f4157200640 1 -- 172.21.15.47:0/3827330698 wait complete. 2024-03-21T10:35:27.513 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:27.511+0000 7f4157200640 1 Processor -- start 2024-03-21T10:35:27.513 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:27.511+0000 7f4157200640 1 -- start start 2024-03-21T10:35:27.513 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:27.512+0000 7f4157200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4150102930 0x7f4150100580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:35:27.513 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:27.512+0000 7f4157200640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4150103250 0x7f4150100ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:35:27.513 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:27.512+0000 7f4157200640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f415010bd00 0x7f4150101000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:35:27.513 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:27.512+0000 7f4157200640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4150111940 con 0x7f415010bd00 2024-03-21T10:35:27.513 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:27.512+0000 7f4157200640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41501117c0 con 0x7f4150102930 2024-03-21T10:35:27.513 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:27.512+0000 7f4157200640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4150111ac0 con 0x7f4150103250 2024-03-21T10:35:27.513 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:27.512+0000 7f4155200640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f415010bd00 0x7f4150101000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:35:27.513 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:27.512+0000 7f4155200640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f415010bd00 0x7f4150101000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:58772/0 (socket says 172.21.15.47:58772) 2024-03-21T10:35:27.513 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:27.512+0000 7f4155200640 1 -- 172.21.15.47:0/2023821772 learned_addr learned my addr 172.21.15.47:0/2023821772 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:35:27.513 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:27.512+0000 7f414f400640 1 --2- 172.21.15.47:0/2023821772 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4150103250 0x7f4150100ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:35:27.514 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:27.512+0000 7f414fe00640 1 --2- 172.21.15.47:0/2023821772 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4150102930 0x7f4150100580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:35:27.514 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:27.513+0000 7f4155200640 1 -- 172.21.15.47:0/2023821772 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4150103250 msgr2=0x7f4150100ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:35:27.514 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:27.513+0000 7f4155200640 1 --2- 172.21.15.47:0/2023821772 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4150103250 0x7f4150100ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:35:27.514 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:27.513+0000 7f4155200640 1 -- 172.21.15.47:0/2023821772 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4150102930 msgr2=0x7f4150100580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:35:27.514 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:27.513+0000 7f4155200640 1 --2- 172.21.15.47:0/2023821772 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4150102930 0x7f4150100580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:35:27.514 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:27.513+0000 7f4155200640 1 -- 172.21.15.47:0/2023821772 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f41501a6470 con 0x7f415010bd00 2024-03-21T10:35:27.514 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:27.513+0000 7f414f400640 1 --2- 172.21.15.47:0/2023821772 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4150103250 0x7f4150100ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T10:35:27.514 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:27.513+0000 7f4155200640 1 --2- 172.21.15.47:0/2023821772 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f415010bd00 0x7f4150101000 secure :-1 s=READY pgs=1302 cs=0 l=1 rev1=1 crypto rx=0x7f414800efc0 tx=0x7f414800c800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:35:27.514 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:27.513+0000 7f414cc00640 1 -- 172.21.15.47:0/2023821772 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4148003970 con 0x7f415010bd00 2024-03-21T10:35:27.514 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:27.513+0000 7f4157200640 1 -- 172.21.15.47:0/2023821772 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f41501a6760 con 0x7f415010bd00 2024-03-21T10:35:27.515 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:27.513+0000 7f414cc00640 1 -- 172.21.15.47:0/2023821772 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f4148004020 con 0x7f415010bd00 2024-03-21T10:35:27.515 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:27.513+0000 7f414cc00640 1 -- 172.21.15.47:0/2023821772 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4148004d40 con 0x7f415010bd00 2024-03-21T10:35:27.515 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:27.513+0000 7f4157200640 1 -- 172.21.15.47:0/2023821772 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f41501a6c80 con 0x7f415010bd00 2024-03-21T10:35:27.516 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:27.517+0000 7f414cc00640 1 -- 172.21.15.47:0/2023821772 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 27) v1 ==== 137985+0+0 (secure 0 0 0) 0x7f414801d020 con 0x7f415010bd00 2024-03-21T10:35:27.516 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:27.517+0000 7f4159df4740 1 Processor -- start 2024-03-21T10:35:27.516 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:27.517+0000 7f4159df4740 1 -- start start 2024-03-21T10:35:27.516 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:27.517+0000 7f4159df4740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5588f954f6d0 0x5588f954fab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:35:27.516 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:27.517+0000 7f4159df4740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5588f9550080 0x5588f9558630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:35:27.516 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:27.517+0000 7f4159df4740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5588f9558c00 0x5588f955b050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:35:27.517 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:27.517+0000 7f4159df4740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x5588f94b32c0 con 0x5588f9550080 2024-03-21T10:35:27.517 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:27.517+0000 7f4159df4740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x5588f945bc90 con 0x5588f954f6d0 2024-03-21T10:35:27.517 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:27.517+0000 7f4159df4740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x5588f94af360 con 0x5588f9558c00 2024-03-21T10:35:27.519 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:27.518+0000 7f414fe00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5588f9550080 0x5588f9558630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:35:27.519 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:27.518+0000 7f414fe00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5588f9550080 0x5588f9558630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:58774/0 (socket says 172.21.15.47:58774) 2024-03-21T10:35:27.519 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:27.518+0000 7f414fe00640 1 -- 172.21.15.47:0/1763530012 learned_addr learned my addr 172.21.15.47:0/1763530012 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:35:27.519 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:27.518+0000 7f414f400640 1 --2- 172.21.15.47:0/1763530012 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5588f954f6d0 0x5588f954fab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:35:27.519 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:27.518+0000 7f414f400640 1 --2- 172.21.15.47:0/1763530012 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5588f9558c00 0x5588f955b050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:35:27.519 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:27.518+0000 7f414fe00640 1 -- 172.21.15.47:0/1763530012 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5588f9558c00 msgr2=0x5588f955b050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:35:27.520 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:27.518+0000 7f414fe00640 1 --2- 172.21.15.47:0/1763530012 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5588f9558c00 0x5588f955b050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:35:27.520 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:27.518+0000 7f414fe00640 1 -- 172.21.15.47:0/1763530012 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5588f954f6d0 msgr2=0x5588f954fab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:35:27.520 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:27.518+0000 7f414fe00640 1 --2- 172.21.15.47:0/1763530012 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5588f954f6d0 0x5588f954fab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:35:27.520 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:27.518+0000 7f414fe00640 1 -- 172.21.15.47:0/1763530012 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5588f9480080 con 0x5588f9550080 2024-03-21T10:35:27.520 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:27.518+0000 7f414f400640 1 --2- 172.21.15.47:0/1763530012 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5588f9558c00 0x5588f955b050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T10:35:27.520 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:27.518+0000 7f414cc00640 1 --2- 172.21.15.47:0/2023821772 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f41340a3090 0x7f41340a5550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:35:27.520 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:27.518+0000 7f414fe00640 1 --2- 172.21.15.47:0/1763530012 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5588f9550080 0x5588f9558630 secure :-1 s=READY pgs=1303 cs=0 l=1 rev1=1 crypto rx=0x7f4140009ca0 tx=0x7f41400117b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6247171ac960ea36 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:35:27.520 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:27.518+0000 7f414cc00640 1 -- 172.21.15.47:0/2023821772 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(105..105 src has 1..105) v4 ==== 9562+0+0 (secure 0 0 0) 0x7f41480cfcd0 con 0x7f415010bd00 2024-03-21T10:35:27.520 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:27.519+0000 7f414f400640 1 --2- 172.21.15.47:0/2023821772 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f41340a3090 0x7f41340a5550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:35:27.520 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:27.519+0000 7f4145600640 1 -- 172.21.15.47:0/1763530012 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4140012450 con 0x5588f9550080 2024-03-21T10:35:27.520 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:27.519+0000 7f4145600640 1 -- 172.21.15.47:0/1763530012 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f4140012aa0 con 0x5588f9550080 2024-03-21T10:35:27.520 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:27.519+0000 7f4145600640 1 -- 172.21.15.47:0/1763530012 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f414000fb00 con 0x5588f9550080 2024-03-21T10:35:27.520 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:27.520+0000 7f4159df4740 1 -- 172.21.15.47:0/1763530012 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5588f9550080 msgr2=0x5588f9558630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:35:27.521 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:27.520+0000 7f4159df4740 1 --2- 172.21.15.47:0/1763530012 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5588f9550080 0x5588f9558630 secure :-1 s=READY pgs=1303 cs=0 l=1 rev1=1 crypto rx=0x7f4140009ca0 tx=0x7f41400117b0 comp rx=0 tx=0).stop 2024-03-21T10:35:27.521 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:27.520+0000 7f414f400640 1 --2- 172.21.15.47:0/2023821772 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f41340a3090 0x7f41340a5550 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f413c02c210 tx=0x7f413c02e040 comp rx=0 tx=0).ready entity=mgr.15687 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:35:27.521 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:27.520+0000 7f4159df4740 1 -- 172.21.15.47:0/1763530012 shutdown_connections 2024-03-21T10:35:27.521 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:27.520+0000 7f4159df4740 1 --2- 172.21.15.47:0/1763530012 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5588f9558c00 0x5588f955b050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:35:27.521 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:27.520+0000 7f4159df4740 1 --2- 172.21.15.47:0/1763530012 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5588f9550080 0x5588f9558630 unknown :-1 s=CLOSED pgs=1303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:35:27.521 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:27.520+0000 7f4159df4740 1 --2- 172.21.15.47:0/1763530012 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5588f954f6d0 0x5588f954fab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:35:27.521 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:27.520+0000 7f4159df4740 1 -- 172.21.15.47:0/1763530012 >> 172.21.15.47:0/1763530012 conn(0x5588f9549da0 msgr2=0x5588f954a180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:35:27.521 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:27.520+0000 7f4159df4740 1 -- 172.21.15.47:0/1763530012 shutdown_connections 2024-03-21T10:35:27.521 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:27.520+0000 7f4159df4740 1 -- 172.21.15.47:0/1763530012 wait complete. 2024-03-21T10:35:27.521 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:27.520+0000 7f4159df4740 1 -- start start 2024-03-21T10:35:27.521 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:27.520+0000 7f4159df4740 1 Processor -- start 2024-03-21T10:35:27.524 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:27.521+0000 7f4159df4740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5588f954f6d0 0x5588f95559e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:35:27.524 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:27.521+0000 7f4159df4740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5588f9550080 0x5588f9555f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:35:27.524 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:27.521+0000 7f4159df4740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5588f9558c00 0x5588f9556460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:35:27.524 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:27.521+0000 7f4159df4740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x5588f955bb70 con 0x5588f9558c00 2024-03-21T10:35:27.524 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:27.521+0000 7f4159df4740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x5588f946f0d0 con 0x5588f9550080 2024-03-21T10:35:27.524 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:27.521+0000 7f4159df4740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x5588f955bcf0 con 0x5588f954f6d0 2024-03-21T10:35:27.525 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:27.521+0000 7f4159df4740 10 client.0 _ll_register_callbacks cb 0x5588f9473e80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T10:35:27.525 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:27.521+0000 7f4155200640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5588f9558c00 0x5588f9556460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:35:27.525 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:27.521+0000 7f4155200640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5588f9558c00 0x5588f9556460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:58784/0 (socket says 172.21.15.47:58784) 2024-03-21T10:35:27.525 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:27.521+0000 7f4155200640 1 -- 172.21.15.47:0/3545613286 learned_addr learned my addr 172.21.15.47:0/3545613286 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:35:27.525 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:27.521+0000 7f414fe00640 1 --2- 172.21.15.47:0/3545613286 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5588f954f6d0 0x5588f95559e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:35:27.525 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:27.522+0000 7f414f400640 1 --2- 172.21.15.47:0/3545613286 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5588f9550080 0x5588f9555f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:35:27.525 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:27.522+0000 7f4155200640 1 -- 172.21.15.47:0/3545613286 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5588f954f6d0 msgr2=0x5588f95559e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:35:27.525 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:27.522+0000 7f4155200640 1 --2- 172.21.15.47:0/3545613286 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5588f954f6d0 0x5588f95559e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:35:27.525 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:27.522+0000 7f4155200640 1 -- 172.21.15.47:0/3545613286 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5588f9550080 msgr2=0x5588f9555f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:35:27.525 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:27.522+0000 7f4155200640 1 --2- 172.21.15.47:0/3545613286 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5588f9550080 0x5588f9555f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:35:27.525 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:27.522+0000 7f4155200640 1 -- 172.21.15.47:0/3545613286 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5588f91c73f0 con 0x5588f9558c00 2024-03-21T10:35:27.525 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:27.522+0000 7f4155200640 1 --2- 172.21.15.47:0/3545613286 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5588f9558c00 0x5588f9556460 secure :-1 s=READY pgs=1304 cs=0 l=1 rev1=1 crypto rx=0x7f4148098620 tx=0x7f4148004540 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aa728c8663082e68 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:35:27.525 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:27.522+0000 7f413a000640 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-03-21T10:35:27.525 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:27.522+0000 7f413a000640 1 -- 172.21.15.47:0/3545613286 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f414809fda0 con 0x5588f9558c00 2024-03-21T10:35:27.525 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:27.522+0000 7f413a000640 1 -- 172.21.15.47:0/3545613286 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(105..105 src has 1..105) v4 ==== 9562+0+0 (secure 0 0 0) 0x7f41480a41d0 con 0x5588f9558c00 2024-03-21T10:35:27.526 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:27.523+0000 7f4159df4740 1 -- 172.21.15.47:0/3545613286 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f4140012aa0 con 0x5588f9558c00 2024-03-21T10:35:27.526 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:27.523+0000 7f413a000640 1 -- 172.21.15.47:0/3545613286 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f41480a4660 con 0x5588f9558c00 2024-03-21T10:35:27.526 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:27.523+0000 7f4159df4740 10 client.18774 fetch_fsmap learned FSMap version 32 2024-03-21T10:35:27.526 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:27.523+0000 7f4159df4740 1 -- 172.21.15.47:0/3545613286 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x5588f92917d0 con 0x5588f9558c00 2024-03-21T10:35:27.526 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:27.523+0000 7f413a000640 1 -- 172.21.15.47:0/3545613286 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f4140012aa0 con 0x5588f9558c00 2024-03-21T10:35:27.526 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:27.523+0000 7f413a000640 1 -- 172.21.15.47:0/3545613286 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 32) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f414800b040 con 0x5588f9558c00 2024-03-21T10:35:27.526 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:27.523+0000 7f4159df4740 10 client.18774 fetch_fsmap finished waiting for FSMap version 32 2024-03-21T10:35:27.526 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:27.523+0000 7f4159df4740 10 client.18774 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.k 2024-03-21T10:35:27.526 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:27.523+0000 7f4159df4740 20 client.18774 populate_metadata read hostname 'smithi047' 2024-03-21T10:35:27.526 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:27.523+0000 7f4159df4740 1 --2- 172.21.15.47:0/3545613286 >> [v2:172.21.15.100:6832/2723302038,v1:172.21.15.100:6833/2723302038] conn(0x5588f9568a80 0x5588f956ae90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:35:27.526 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:27.523+0000 7f4159df4740 4 client.18774 mds_command: new command op to 34427 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T10:35:27.526 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:27.523+0000 7f4159df4740 1 -- 172.21.15.47:0/3545613286 --> [v2:172.21.15.100:6832/2723302038,v1:172.21.15.100:6833/2723302038] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5588f94c0800 con 0x5588f9568a80 2024-03-21T10:35:27.527 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:27.524+0000 7f414fe00640 1 --2- 172.21.15.47:0/3545613286 >> [v2:172.21.15.100:6832/2723302038,v1:172.21.15.100:6833/2723302038] conn(0x5588f9568a80 0x5588f956ae90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:35:27.527 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:27.525+0000 7f414fe00640 1 --2- 172.21.15.47:0/3545613286 >> [v2:172.21.15.100:6832/2723302038,v1:172.21.15.100:6833/2723302038] conn(0x5588f9568a80 0x5588f956ae90 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=5d6b91030c4e1bf0 server_cookie=f1f0cfad29e55950 in_seq=0 out_seq=0 2024-03-21T10:35:27.527 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:27.525+0000 7f413a000640 10 client.18774 ms_handle_connect on v2:172.21.15.100:6832/2723302038 2024-03-21T10:35:27.528 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:27.527+0000 7f413a000640 1 -- 172.21.15.47:0/3545613286 <== mds.3 v2:172.21.15.100:6832/2723302038 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x5588f94c0800 con 0x5588f9568a80 2024-03-21T10:35:27.528 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:27.527+0000 7f413a000640 10 client.18774 handle_command_reply: tid=0 2024-03-21T10:35:27.528 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:27.527+0000 7f4159df4740 1 client.18774 shutdown 2024-03-21T10:35:27.528 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:27.527+0000 7f4159df4740 1 -- 172.21.15.47:0/3545613286 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5588f9558c00 msgr2=0x5588f9556460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:35:27.528 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:27.527+0000 7f4159df4740 1 --2- 172.21.15.47:0/3545613286 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5588f9558c00 0x5588f9556460 secure :-1 s=READY pgs=1304 cs=0 l=1 rev1=1 crypto rx=0x7f4148098620 tx=0x7f4148004540 comp rx=0 tx=0).stop 2024-03-21T10:35:27.528 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:27.527+0000 7f4159df4740 1 -- 172.21.15.47:0/3545613286 shutdown_connections 2024-03-21T10:35:27.528 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:27.527+0000 7f4159df4740 1 --2- 172.21.15.47:0/3545613286 >> [v2:172.21.15.100:6832/2723302038,v1:172.21.15.100:6833/2723302038] conn(0x5588f9568a80 0x5588f956ae90 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:35:27.528 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:27.527+0000 7f4159df4740 1 --2- 172.21.15.47:0/3545613286 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5588f9558c00 0x5588f9556460 unknown :-1 s=CLOSED pgs=1304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:35:27.528 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:27.527+0000 7f4159df4740 1 --2- 172.21.15.47:0/3545613286 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5588f9550080 0x5588f9555f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:35:27.528 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:27.527+0000 7f4159df4740 1 --2- 172.21.15.47:0/3545613286 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5588f954f6d0 0x5588f95559e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:35:27.528 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:27.527+0000 7f4159df4740 1 -- 172.21.15.47:0/3545613286 >> 172.21.15.47:0/3545613286 conn(0x5588f9549da0 msgr2=0x5588f954baf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:35:27.528 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:27.527+0000 7f413a000640 0 client.18774 ms_handle_reset on v2:172.21.15.100:6832/2723302038 2024-03-21T10:35:27.529 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:27.527+0000 7f4159df4740 1 -- 172.21.15.47:0/3545613286 shutdown_connections 2024-03-21T10:35:27.529 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:27.527+0000 7f4159df4740 1 -- 172.21.15.47:0/3545613286 wait complete. 2024-03-21T10:35:27.529 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:27.528+0000 7f4159df4740 20 client.18774 trim_cache size 0 max 16384 2024-03-21T10:35:27.541 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:27.541+0000 7f4159df4740 1 Processor -- start 2024-03-21T10:35:27.541 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:27.541+0000 7f4159df4740 1 -- start start 2024-03-21T10:35:27.541 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:27.542+0000 7f4159df4740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5588f954f6d0 0x5588f9558110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:35:27.541 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:27.542+0000 7f4159df4740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5588f9550080 0x5588f954b1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:35:27.541 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:27.542+0000 7f4159df4740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5588f9558c00 0x5588f9552c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:35:27.541 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:27.542+0000 7f4159df4740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41501b0b90 con 0x5588f9558c00 2024-03-21T10:35:27.541 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:27.542+0000 7f4159df4740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41501b2510 con 0x5588f954f6d0 2024-03-21T10:35:27.541 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:27.542+0000 7f4159df4740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41501b11f0 con 0x5588f9550080 2024-03-21T10:35:27.541 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:27.542+0000 7f4155200640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5588f9558c00 0x5588f9552c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:35:27.542 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:27.542+0000 7f4155200640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5588f9558c00 0x5588f9552c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:58794/0 (socket says 172.21.15.47:58794) 2024-03-21T10:35:27.542 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:27.542+0000 7f4155200640 1 -- 172.21.15.47:0/1794887997 learned_addr learned my addr 172.21.15.47:0/1794887997 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:35:27.542 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:27.542+0000 7f414fe00640 1 --2- 172.21.15.47:0/1794887997 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5588f954f6d0 0x5588f9558110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:35:27.545 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:27.542+0000 7f4155200640 1 -- 172.21.15.47:0/1794887997 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5588f9550080 msgr2=0x5588f954b1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:35:27.545 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:27.542+0000 7f414f400640 1 --2- 172.21.15.47:0/1794887997 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5588f9550080 0x5588f954b1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:35:27.545 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:27.542+0000 7f4155200640 1 --2- 172.21.15.47:0/1794887997 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5588f9550080 0x5588f954b1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:35:27.546 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:27.542+0000 7f4155200640 1 -- 172.21.15.47:0/1794887997 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5588f954f6d0 msgr2=0x5588f9558110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:35:27.546 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:27.542+0000 7f4155200640 1 --2- 172.21.15.47:0/1794887997 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5588f954f6d0 0x5588f9558110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:35:27.546 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:27.542+0000 7f4155200640 1 -- 172.21.15.47:0/1794887997 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5588f954c0a0 con 0x5588f9558c00 2024-03-21T10:35:27.546 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:27.542+0000 7f414fe00640 1 --2- 172.21.15.47:0/1794887997 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5588f954f6d0 0x5588f9558110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T10:35:27.546 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:27.543+0000 7f4155200640 1 --2- 172.21.15.47:0/1794887997 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5588f9558c00 0x5588f9552c40 secure :-1 s=READY pgs=1305 cs=0 l=1 rev1=1 crypto rx=0x7f4148099f00 tx=0x7f41480979a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9732c491af45eed7 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:35:27.546 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:27.543+0000 7f4144c00640 1 -- 172.21.15.47:0/1794887997 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f414809fda0 con 0x5588f9558c00 2024-03-21T10:35:27.546 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:27.543+0000 7f4144c00640 1 -- 172.21.15.47:0/1794887997 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f414809b0a0 con 0x5588f9558c00 2024-03-21T10:35:27.546 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:27.543+0000 7f4144c00640 1 -- 172.21.15.47:0/1794887997 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f414800b040 con 0x5588f9558c00 2024-03-21T10:35:27.546 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:27.544+0000 7f4159df4740 1 -- 172.21.15.47:0/1794887997 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5588f9558c00 msgr2=0x5588f9552c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:35:27.546 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:27.544+0000 7f4159df4740 1 --2- 172.21.15.47:0/1794887997 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5588f9558c00 0x5588f9552c40 secure :-1 s=READY pgs=1305 cs=0 l=1 rev1=1 crypto rx=0x7f4148099f00 tx=0x7f41480979a0 comp rx=0 tx=0).stop 2024-03-21T10:35:27.546 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:27.544+0000 7f4159df4740 1 -- 172.21.15.47:0/1794887997 shutdown_connections 2024-03-21T10:35:27.546 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:27.544+0000 7f4159df4740 1 --2- 172.21.15.47:0/1794887997 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5588f9558c00 0x5588f9552c40 unknown :-1 s=CLOSED pgs=1305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:35:27.547 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:27.544+0000 7f4159df4740 1 --2- 172.21.15.47:0/1794887997 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5588f9550080 0x5588f954b1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:35:27.547 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:27.544+0000 7f4159df4740 1 --2- 172.21.15.47:0/1794887997 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5588f954f6d0 0x5588f9558110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:35:27.547 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:27.544+0000 7f4159df4740 1 -- 172.21.15.47:0/1794887997 >> 172.21.15.47:0/1794887997 conn(0x5588f9549da0 msgr2=0x5588f9556230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:35:27.547 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:27.544+0000 7f4159df4740 1 -- 172.21.15.47:0/1794887997 shutdown_connections 2024-03-21T10:35:27.547 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:27.544+0000 7f4159df4740 1 -- 172.21.15.47:0/1794887997 wait complete. 2024-03-21T10:35:27.547 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:27.544+0000 7f4159df4740 1 -- start start 2024-03-21T10:35:27.547 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:27.545+0000 7f4159df4740 1 Processor -- start 2024-03-21T10:35:27.547 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:27.545+0000 7f4159df4740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5588f954f6d0 0x5588f955ebe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:35:27.547 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:27.545+0000 7f4159df4740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5588f9550080 0x5588f955f120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:35:27.547 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:27.545+0000 7f4159df4740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5588f9558c00 0x5588f95bd370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:35:27.547 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:27.545+0000 7f4159df4740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41501b1eb0 con 0x5588f9550080 2024-03-21T10:35:27.547 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:27.545+0000 7f4159df4740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41501b1850 con 0x5588f9558c00 2024-03-21T10:35:27.547 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:27.545+0000 7f4159df4740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41501aed70 con 0x5588f954f6d0 2024-03-21T10:35:27.547 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:27.545+0000 7f4159df4740 10 client.0 _ll_register_callbacks cb 0x5588f9558ac0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T10:35:27.548 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:27.545+0000 7f414f400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5588f9550080 0x5588f955f120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:35:27.548 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:27.545+0000 7f414f400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5588f9550080 0x5588f955f120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:58808/0 (socket says 172.21.15.47:58808) 2024-03-21T10:35:27.548 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:27.545+0000 7f414f400640 1 -- 172.21.15.47:0/3647948353 learned_addr learned my addr 172.21.15.47:0/3647948353 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:35:27.548 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:27.546+0000 7f4155200640 1 --2- 172.21.15.47:0/3647948353 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5588f9558c00 0x5588f95bd370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:35:27.548 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:27.546+0000 7f414fe00640 1 --2- 172.21.15.47:0/3647948353 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5588f954f6d0 0x5588f955ebe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:35:27.548 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:27.546+0000 7f414f400640 1 -- 172.21.15.47:0/3647948353 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5588f954f6d0 msgr2=0x5588f955ebe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:35:27.548 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:27.546+0000 7f414f400640 1 --2- 172.21.15.47:0/3647948353 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5588f954f6d0 0x5588f955ebe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:35:27.548 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:27.546+0000 7f414f400640 1 -- 172.21.15.47:0/3647948353 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5588f9558c00 msgr2=0x5588f95bd370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:35:27.548 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:27.546+0000 7f414f400640 1 --2- 172.21.15.47:0/3647948353 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5588f9558c00 0x5588f95bd370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:35:27.548 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:27.546+0000 7f414f400640 1 -- 172.21.15.47:0/3647948353 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5588f95bd930 con 0x5588f9550080 2024-03-21T10:35:27.548 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:27.546+0000 7f414fe00640 1 --2- 172.21.15.47:0/3647948353 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5588f954f6d0 0x5588f955ebe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T10:35:27.548 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:27.546+0000 7f414f400640 1 --2- 172.21.15.47:0/3647948353 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5588f9550080 0x5588f955f120 secure :-1 s=READY pgs=1306 cs=0 l=1 rev1=1 crypto rx=0x7f413c009860 tx=0x7f413c005710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f7dd8369004a87bc server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:35:27.548 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:27.546+0000 7f413a000640 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-03-21T10:35:27.551 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:27.547+0000 7f413a000640 1 -- 172.21.15.47:0/3647948353 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f413c050030 con 0x5588f9550080 2024-03-21T10:35:27.551 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:27.547+0000 7f413a000640 1 -- 172.21.15.47:0/3647948353 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(105..105 src has 1..105) v4 ==== 9562+0+0 (secure 0 0 0) 0x7f413c02ce00 con 0x5588f9550080 2024-03-21T10:35:27.551 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:27.547+0000 7f4159df4740 1 -- 172.21.15.47:0/3647948353 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f414809b0a0 con 0x5588f9550080 2024-03-21T10:35:27.551 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:27.547+0000 7f413a000640 1 -- 172.21.15.47:0/3647948353 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f413c05d050 con 0x5588f9550080 2024-03-21T10:35:27.551 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:27.547+0000 7f4159df4740 10 client.18780 fetch_fsmap learned FSMap version 32 2024-03-21T10:35:27.551 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:27.547+0000 7f4159df4740 1 -- 172.21.15.47:0/3647948353 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x5588f95c4d10 con 0x5588f9550080 2024-03-21T10:35:27.551 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:27.548+0000 7f413a000640 1 -- 172.21.15.47:0/3647948353 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f414809b0a0 con 0x5588f9550080 2024-03-21T10:35:27.551 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:27.548+0000 7f413a000640 1 -- 172.21.15.47:0/3647948353 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 32) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f413c003ae0 con 0x5588f9550080 2024-03-21T10:35:27.552 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:27.548+0000 7f4159df4740 10 client.18780 fetch_fsmap finished waiting for FSMap version 32 2024-03-21T10:35:27.552 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:27.548+0000 7f4159df4740 10 client.18780 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.k 2024-03-21T10:35:27.552 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:27.548+0000 7f4159df4740 20 client.18780 populate_metadata read hostname 'smithi047' 2024-03-21T10:35:27.552 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:27.548+0000 7f4159df4740 1 --2- 172.21.15.47:0/3647948353 >> [v2:172.21.15.100:6832/2723302038,v1:172.21.15.100:6833/2723302038] conn(0x5588f95c54f0 0x5588f95c7910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:35:27.552 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:27.548+0000 7f4159df4740 4 client.18780 mds_command: new command op to 34427 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-21T10:35:27.552 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:27.548+0000 7f4159df4740 1 -- 172.21.15.47:0/3647948353 --> [v2:172.21.15.100:6832/2723302038,v1:172.21.15.100:6833/2723302038] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x5588f94bab50 con 0x5588f95c54f0 2024-03-21T10:35:27.552 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:27.549+0000 7f414fe00640 1 --2- 172.21.15.47:0/3647948353 >> [v2:172.21.15.100:6832/2723302038,v1:172.21.15.100:6833/2723302038] conn(0x5588f95c54f0 0x5588f95c7910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:35:27.552 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:27.549+0000 7f414fe00640 1 --2- 172.21.15.47:0/3647948353 >> [v2:172.21.15.100:6832/2723302038,v1:172.21.15.100:6833/2723302038] conn(0x5588f95c54f0 0x5588f95c7910 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=f722da56c5d171c9 server_cookie=ff428cb85742eecd in_seq=0 out_seq=0 2024-03-21T10:35:27.552 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:27.549+0000 7f413a000640 10 client.18780 ms_handle_connect on v2:172.21.15.100:6832/2723302038 2024-03-21T10:35:27.552 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:27.550+0000 7f413a000640 1 -- 172.21.15.47:0/3647948353 <== mds.3 v2:172.21.15.100:6832/2723302038 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x5588f94bab50 con 0x5588f95c54f0 2024-03-21T10:35:27.552 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:27.550+0000 7f413a000640 10 client.18780 handle_command_reply: tid=0 2024-03-21T10:35:27.552 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:27.550+0000 7f4159df4740 1 client.18780 shutdown 2024-03-21T10:35:27.553 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:27.550+0000 7f4159df4740 1 -- 172.21.15.47:0/3647948353 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5588f9550080 msgr2=0x5588f955f120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:35:27.553 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:27.550+0000 7f4159df4740 1 --2- 172.21.15.47:0/3647948353 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5588f9550080 0x5588f955f120 secure :-1 s=READY pgs=1306 cs=0 l=1 rev1=1 crypto rx=0x7f413c009860 tx=0x7f413c005710 comp rx=0 tx=0).stop 2024-03-21T10:35:27.553 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:27.550+0000 7f4159df4740 1 -- 172.21.15.47:0/3647948353 shutdown_connections 2024-03-21T10:35:27.553 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:27.550+0000 7f4159df4740 1 --2- 172.21.15.47:0/3647948353 >> [v2:172.21.15.100:6832/2723302038,v1:172.21.15.100:6833/2723302038] conn(0x5588f95c54f0 0x5588f95c7910 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:35:27.553 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:27.550+0000 7f4159df4740 1 --2- 172.21.15.47:0/3647948353 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5588f9558c00 0x5588f95bd370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:35:27.553 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:27.550+0000 7f4159df4740 1 --2- 172.21.15.47:0/3647948353 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5588f9550080 0x5588f955f120 unknown :-1 s=CLOSED pgs=1306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:35:27.553 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:27.550+0000 7f413a000640 0 client.18780 ms_handle_reset on v2:172.21.15.100:6832/2723302038 2024-03-21T10:35:27.553 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:27.550+0000 7f4159df4740 1 --2- 172.21.15.47:0/3647948353 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5588f954f6d0 0x5588f955ebe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:35:27.553 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:27.550+0000 7f4159df4740 1 -- 172.21.15.47:0/3647948353 >> 172.21.15.47:0/3647948353 conn(0x5588f9549da0 msgr2=0x5588f9548eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:35:27.553 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:27.550+0000 7f4159df4740 1 -- 172.21.15.47:0/3647948353 shutdown_connections 2024-03-21T10:35:27.553 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:27.550+0000 7f4159df4740 1 -- 172.21.15.47:0/3647948353 wait complete. 2024-03-21T10:35:27.553 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:27.550+0000 7f4159df4740 20 client.18780 trim_cache size 0 max 16384 2024-03-21T10:35:27.554 INFO:teuthology.orchestra.run.smithi047.stdout:[] 2024-03-21T10:35:27.555 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:27.551+0000 7f4157200640 1 -- 172.21.15.47:0/2023821772 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f41340a3090 msgr2=0x7f41340a5550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:35:27.555 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:27.551+0000 7f4157200640 1 --2- 172.21.15.47:0/2023821772 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f41340a3090 0x7f41340a5550 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f413c02c210 tx=0x7f413c02e040 comp rx=0 tx=0).stop 2024-03-21T10:35:27.555 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:27.551+0000 7f4157200640 1 -- 172.21.15.47:0/2023821772 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f415010bd00 msgr2=0x7f4150101000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:35:27.555 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:27.551+0000 7f4157200640 1 --2- 172.21.15.47:0/2023821772 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f415010bd00 0x7f4150101000 secure :-1 s=READY pgs=1302 cs=0 l=1 rev1=1 crypto rx=0x7f414800efc0 tx=0x7f414800c800 comp rx=0 tx=0).stop 2024-03-21T10:35:27.555 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:27.551+0000 7f4157200640 1 -- 172.21.15.47:0/2023821772 shutdown_connections 2024-03-21T10:35:27.555 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:27.551+0000 7f4157200640 1 --2- 172.21.15.47:0/2023821772 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f415010bd00 0x7f4150101000 unknown :-1 s=CLOSED pgs=1302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:35:27.555 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:27.551+0000 7f4157200640 1 --2- 172.21.15.47:0/2023821772 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4150103250 0x7f4150100ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:35:27.555 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:27.551+0000 7f4157200640 1 --2- 172.21.15.47:0/2023821772 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f41340a3090 0x7f41340a5550 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:35:27.555 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:27.551+0000 7f4157200640 1 --2- 172.21.15.47:0/2023821772 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4150102930 0x7f4150100580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:35:27.555 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:27.551+0000 7f4157200640 1 -- 172.21.15.47:0/2023821772 >> 172.21.15.47:0/2023821772 conn(0x7f4150077700 msgr2=0x7f41500feb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:35:27.556 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:27.551+0000 7f4157200640 1 -- 172.21.15.47:0/2023821772 shutdown_connections 2024-03-21T10:35:27.556 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:27.551+0000 7f4157200640 1 -- 172.21.15.47:0/2023821772 wait complete. 2024-03-21T10:35:27.565 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:35:27 smithi047 ceph-mon[31547]: pgmap v315: 129 pgs: 129 active+clean; 2.5 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 85 B/s wr, 12 op/s 2024-03-21T10:35:27.565 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:35:27 smithi047 ceph-mon[31547]: from='client.? 172.21.15.47:0/2595294947' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-21T10:35:27.569 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-21T10:35:27.569 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-03-21T10:35:27.726 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:35:27 smithi100 ceph-mon[35193]: pgmap v315: 129 pgs: 129 active+clean; 2.5 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 85 B/s wr, 12 op/s 2024-03-21T10:35:27.726 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:35:27 smithi100 ceph-mon[35193]: from='client.? 172.21.15.47:0/2595294947' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-21T10:35:27.736 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:35:27 smithi157 ceph-mon[35581]: pgmap v315: 129 pgs: 129 active+clean; 2.5 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 85 B/s wr, 12 op/s 2024-03-21T10:35:27.736 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:35:27 smithi157 ceph-mon[35581]: from='client.? 172.21.15.47:0/2595294947' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-21T10:35:27.791 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:27.791+0000 7f117b000640 1 -- 172.21.15.47:0/1338427970 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1174103200 msgr2=0x7f117410b6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:35:27.791 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:27.791+0000 7f117b000640 1 --2- 172.21.15.47:0/1338427970 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1174103200 0x7f117410b6e0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f1160009900 tx=0x7f116002bcc0 comp rx=0 tx=0).stop 2024-03-21T10:35:27.791 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:27.792+0000 7f117b000640 1 -- 172.21.15.47:0/1338427970 shutdown_connections 2024-03-21T10:35:27.791 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:27.792+0000 7f117b000640 1 --2- 172.21.15.47:0/1338427970 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f117410bcb0 0x7f117410e0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:35:27.791 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:27.792+0000 7f117b000640 1 --2- 172.21.15.47:0/1338427970 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1174103200 0x7f117410b6e0 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:35:27.791 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:27.792+0000 7f117b000640 1 --2- 172.21.15.47:0/1338427970 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f11741028e0 0x7f1174102cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:35:27.791 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:27.792+0000 7f117b000640 1 -- 172.21.15.47:0/1338427970 >> 172.21.15.47:0/1338427970 conn(0x7f1174076920 msgr2=0x7f1174076d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:35:27.791 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:27.792+0000 7f117b000640 1 -- 172.21.15.47:0/1338427970 shutdown_connections 2024-03-21T10:35:27.791 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:27.792+0000 7f117b000640 1 -- 172.21.15.47:0/1338427970 wait complete. 2024-03-21T10:35:27.792 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:27.793+0000 7f117b000640 1 Processor -- start 2024-03-21T10:35:27.792 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:27.793+0000 7f117b000640 1 -- start start 2024-03-21T10:35:27.794 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:27.794+0000 7f117b000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f11741028e0 0x7f117419a0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:35:27.794 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:27.794+0000 7f117b000640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1174103200 0x7f117419a5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:35:27.794 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:27.794+0000 7f117b000640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f117410bcb0 0x7f117419e920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:35:27.794 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:27.794+0000 7f117b000640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11741118f0 con 0x7f117410bcb0 2024-03-21T10:35:27.794 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:27.794+0000 7f117b000640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1174111770 con 0x7f11741028e0 2024-03-21T10:35:27.794 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:27.794+0000 7f117b000640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1174111a70 con 0x7f1174103200 2024-03-21T10:35:27.794 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:27.794+0000 7f1173e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f11741028e0 0x7f117419a0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:35:27.794 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:27.794+0000 7f1173e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f11741028e0 0x7f117419a0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:59986/0 (socket says 172.21.15.47:59986) 2024-03-21T10:35:27.794 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:27.794+0000 7f1173e00640 1 -- 172.21.15.47:0/2041486107 learned_addr learned my addr 172.21.15.47:0/2041486107 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:35:27.794 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:27.794+0000 7f1179000640 1 --2- 172.21.15.47:0/2041486107 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f117410bcb0 0x7f117419e920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:35:27.794 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:27.794+0000 7f1173400640 1 --2- 172.21.15.47:0/2041486107 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1174103200 0x7f117419a5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:35:27.795 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:27.795+0000 7f1173e00640 1 -- 172.21.15.47:0/2041486107 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1174103200 msgr2=0x7f117419a5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:35:27.795 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:27.795+0000 7f1173e00640 1 --2- 172.21.15.47:0/2041486107 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1174103200 0x7f117419a5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:35:27.795 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:27.795+0000 7f1173e00640 1 -- 172.21.15.47:0/2041486107 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f117410bcb0 msgr2=0x7f117419e920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:35:27.795 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:27.795+0000 7f1173e00640 1 --2- 172.21.15.47:0/2041486107 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f117410bcb0 0x7f117419e920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:35:27.795 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:27.795+0000 7f1173e00640 1 -- 172.21.15.47:0/2041486107 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f117419f050 con 0x7f11741028e0 2024-03-21T10:35:27.795 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:27.795+0000 7f1173e00640 1 --2- 172.21.15.47:0/2041486107 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f11741028e0 0x7f117419a0b0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f116400d950 tx=0x7f116400de10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:35:27.796 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:27.795+0000 7f1170c00640 1 -- 172.21.15.47:0/2041486107 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1164017070 con 0x7f11741028e0 2024-03-21T10:35:27.796 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:27.795+0000 7f117b000640 1 -- 172.21.15.47:0/2041486107 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f117419f340 con 0x7f11741028e0 2024-03-21T10:35:27.796 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:27.796+0000 7f1170c00640 1 -- 172.21.15.47:0/2041486107 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f1164009c30 con 0x7f11741028e0 2024-03-21T10:35:27.796 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:27.796+0000 7f1170c00640 1 -- 172.21.15.47:0/2041486107 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1164020060 con 0x7f11741028e0 2024-03-21T10:35:27.796 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:27.796+0000 7f117b000640 1 -- 172.21.15.47:0/2041486107 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f11741a6bd0 con 0x7f11741028e0 2024-03-21T10:35:27.798 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:27.799+0000 7f117dd72740 1 Processor -- start 2024-03-21T10:35:27.798 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:27.799+0000 7f117dd72740 1 -- start start 2024-03-21T10:35:27.799 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:27.799+0000 7f117dd72740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56505d6ef6d0 0x56505d6efab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:35:27.799 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:27.799+0000 7f117dd72740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56505d6f0080 0x56505d6f8630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:35:27.799 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:27.799+0000 7f1170c00640 1 -- 172.21.15.47:0/2041486107 <== mon.2 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 27) v1 ==== 137985+0+0 (secure 0 0 0) 0x7f1164010460 con 0x7f11741028e0 2024-03-21T10:35:27.799 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:27.800+0000 7f117dd72740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56505d6f8c00 0x56505d6fb050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:35:27.800 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:27.800+0000 7f117dd72740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x56505d6532c0 con 0x56505d6f0080 2024-03-21T10:35:27.800 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:27.800+0000 7f117dd72740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x56505d5fbc90 con 0x56505d6ef6d0 2024-03-21T10:35:27.800 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:27.800+0000 7f117dd72740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x56505d64f360 con 0x56505d6f8c00 2024-03-21T10:35:27.800 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:27.800+0000 7f1173e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56505d6f0080 0x56505d6f8630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:35:27.800 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:27.800+0000 7f1179000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56505d6ef6d0 0x56505d6efab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:35:27.800 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:27.800+0000 7f1179000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56505d6ef6d0 0x56505d6efab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:59998/0 (socket says 172.21.15.47:59998) 2024-03-21T10:35:27.800 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:27.800+0000 7f1179000640 1 -- 172.21.15.47:0/3489312141 learned_addr learned my addr 172.21.15.47:0/3489312141 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:35:27.800 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:27.800+0000 7f1173400640 1 --2- 172.21.15.47:0/3489312141 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56505d6f8c00 0x56505d6fb050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:35:27.800 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:27.800+0000 7f1179000640 1 -- 172.21.15.47:0/3489312141 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56505d6f8c00 msgr2=0x56505d6fb050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:35:27.800 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:27.800+0000 7f1179000640 1 --2- 172.21.15.47:0/3489312141 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56505d6f8c00 0x56505d6fb050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:35:27.801 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:27.800+0000 7f1179000640 1 -- 172.21.15.47:0/3489312141 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56505d6f0080 msgr2=0x56505d6f8630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:35:27.801 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:27.800+0000 7f1179000640 1 --2- 172.21.15.47:0/3489312141 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56505d6f0080 0x56505d6f8630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:35:27.801 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:27.800+0000 7f1179000640 1 -- 172.21.15.47:0/3489312141 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56505d620080 con 0x56505d6ef6d0 2024-03-21T10:35:27.801 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:27.801+0000 7f1173e00640 1 --2- 172.21.15.47:0/3489312141 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56505d6f0080 0x56505d6f8630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T10:35:27.804 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:27.801+0000 7f1179000640 1 --2- 172.21.15.47:0/3489312141 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56505d6ef6d0 0x56505d6efab0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f116c009f30 tx=0x7f116c0118d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8d1b141e252186f4 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:35:27.804 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:27.801+0000 7f1170c00640 1 --2- 172.21.15.47:0/2041486107 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f11580a3110 0x7f11580a55d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:35:27.804 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:27.801+0000 7f1169600640 1 -- 172.21.15.47:0/3489312141 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f116c004d20 con 0x56505d6ef6d0 2024-03-21T10:35:27.804 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:27.801+0000 7f1169600640 1 -- 172.21.15.47:0/3489312141 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f116c004050 con 0x56505d6ef6d0 2024-03-21T10:35:27.804 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:27.801+0000 7f1170c00640 1 -- 172.21.15.47:0/2041486107 <== mon.2 v2:172.21.15.100:3300/0 5 ==== osd_map(105..105 src has 1..105) v4 ==== 9562+0+0 (secure 0 0 0) 0x7f11640cf4d0 con 0x7f11741028e0 2024-03-21T10:35:27.804 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:27.801+0000 7f1169600640 1 -- 172.21.15.47:0/3489312141 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f116c005280 con 0x56505d6ef6d0 2024-03-21T10:35:27.804 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:27.802+0000 7f1173e00640 1 --2- 172.21.15.47:0/2041486107 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f11580a3110 0x7f11580a55d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:35:27.804 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:27.802+0000 7f117dd72740 1 -- 172.21.15.47:0/3489312141 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56505d6ef6d0 msgr2=0x56505d6efab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:35:27.804 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:27.802+0000 7f117dd72740 1 --2- 172.21.15.47:0/3489312141 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56505d6ef6d0 0x56505d6efab0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f116c009f30 tx=0x7f116c0118d0 comp rx=0 tx=0).stop 2024-03-21T10:35:27.804 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:27.802+0000 7f117dd72740 1 -- 172.21.15.47:0/3489312141 shutdown_connections 2024-03-21T10:35:27.804 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:27.802+0000 7f1173e00640 1 --2- 172.21.15.47:0/2041486107 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f11580a3110 0x7f11580a55d0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x56505d6ef1a0 tx=0x7f1164097040 comp rx=0 tx=0).ready entity=mgr.15687 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:35:27.805 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:27.802+0000 7f117dd72740 1 --2- 172.21.15.47:0/3489312141 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56505d6f8c00 0x56505d6fb050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:35:27.805 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:27.802+0000 7f117dd72740 1 --2- 172.21.15.47:0/3489312141 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56505d6f0080 0x56505d6f8630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:35:27.805 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:27.802+0000 7f117dd72740 1 --2- 172.21.15.47:0/3489312141 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56505d6ef6d0 0x56505d6efab0 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:35:27.805 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:27.802+0000 7f117dd72740 1 -- 172.21.15.47:0/3489312141 >> 172.21.15.47:0/3489312141 conn(0x56505d6e9da0 msgr2=0x56505d6ea180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:35:27.805 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:27.802+0000 7f117dd72740 1 -- 172.21.15.47:0/3489312141 shutdown_connections 2024-03-21T10:35:27.805 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:27.802+0000 7f117dd72740 1 -- 172.21.15.47:0/3489312141 wait complete. 2024-03-21T10:35:27.805 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:27.803+0000 7f117dd72740 1 -- start start 2024-03-21T10:35:27.805 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:27.803+0000 7f117dd72740 1 Processor -- start 2024-03-21T10:35:27.805 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:27.804+0000 7f117dd72740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56505d6ef6d0 0x56505d6f8460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:35:27.805 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:27.804+0000 7f117dd72740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56505d6f0080 0x56505d6f57f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:35:27.805 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:27.804+0000 7f117dd72740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56505d6f8c00 0x56505d6f5d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:35:27.805 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:27.804+0000 7f117dd72740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x56505d6fbb70 con 0x56505d6ef6d0 2024-03-21T10:35:27.805 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:27.804+0000 7f117dd72740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x56505d60f0d0 con 0x56505d6f0080 2024-03-21T10:35:27.805 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:27.804+0000 7f117dd72740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x56505d6fbcf0 con 0x56505d6f8c00 2024-03-21T10:35:27.806 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:27.804+0000 7f117dd72740 10 client.0 _ll_register_callbacks cb 0x56505d613e80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T10:35:27.806 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:27.804+0000 7f1179000640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56505d6ef6d0 0x56505d6f8460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:35:27.806 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:27.804+0000 7f1179000640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56505d6ef6d0 0x56505d6f8460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:58848/0 (socket says 172.21.15.47:58848) 2024-03-21T10:35:27.806 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:27.804+0000 7f1179000640 1 -- 172.21.15.47:0/1384479827 learned_addr learned my addr 172.21.15.47:0/1384479827 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:35:27.806 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:27.804+0000 7f1173400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56505d6f0080 0x56505d6f57f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:35:27.806 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:27.804+0000 7f1179000640 1 -- 172.21.15.47:0/1384479827 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56505d6f8c00 msgr2=0x56505d6f5d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:35:27.806 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:27.804+0000 7f1179000640 1 --2- 172.21.15.47:0/1384479827 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56505d6f8c00 0x56505d6f5d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:35:27.806 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:27.804+0000 7f1179000640 1 -- 172.21.15.47:0/1384479827 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56505d6f0080 msgr2=0x56505d6f57f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:35:27.806 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:27.804+0000 7f1179000640 1 --2- 172.21.15.47:0/1384479827 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56505d6f0080 0x56505d6f57f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:35:27.806 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:27.804+0000 7f1179000640 1 -- 172.21.15.47:0/1384479827 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56505d3673f0 con 0x56505d6ef6d0 2024-03-21T10:35:27.806 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:27.805+0000 7f1179000640 1 --2- 172.21.15.47:0/1384479827 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56505d6ef6d0 0x56505d6f8460 secure :-1 s=READY pgs=1307 cs=0 l=1 rev1=1 crypto rx=0x7f116c009b00 tx=0x7f116c009c70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=994f903687cdfc0e server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:35:27.806 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:27.805+0000 7f115e000640 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-03-21T10:35:27.809 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:27.805+0000 7f115e000640 1 -- 172.21.15.47:0/1384479827 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f116c028070 con 0x56505d6ef6d0 2024-03-21T10:35:27.809 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:27.805+0000 7f115e000640 1 -- 172.21.15.47:0/1384479827 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(105..105 src has 1..105) v4 ==== 9562+0+0 (secure 0 0 0) 0x7f116c003c30 con 0x56505d6ef6d0 2024-03-21T10:35:27.809 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:27.805+0000 7f117dd72740 1 -- 172.21.15.47:0/1384479827 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f116c004050 con 0x56505d6ef6d0 2024-03-21T10:35:27.809 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:27.806+0000 7f115e000640 1 -- 172.21.15.47:0/1384479827 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f116c034050 con 0x56505d6ef6d0 2024-03-21T10:35:27.809 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:27.806+0000 7f117dd72740 10 client.18792 fetch_fsmap learned FSMap version 32 2024-03-21T10:35:27.809 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:27.806+0000 7f117dd72740 1 -- 172.21.15.47:0/1384479827 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x56505d4317d0 con 0x56505d6ef6d0 2024-03-21T10:35:27.809 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:27.806+0000 7f115e000640 1 -- 172.21.15.47:0/1384479827 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f116c004050 con 0x56505d6ef6d0 2024-03-21T10:35:27.810 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:27.806+0000 7f115e000640 1 -- 172.21.15.47:0/1384479827 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 32) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f116c00f480 con 0x56505d6ef6d0 2024-03-21T10:35:27.810 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:27.806+0000 7f117dd72740 10 client.18792 fetch_fsmap finished waiting for FSMap version 32 2024-03-21T10:35:27.810 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:27.806+0000 7f117dd72740 10 client.18792 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.e 2024-03-21T10:35:27.810 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:27.806+0000 7f117dd72740 20 client.18792 populate_metadata read hostname 'smithi047' 2024-03-21T10:35:27.810 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:27.806+0000 7f117dd72740 1 --2- 172.21.15.47:0/1384479827 >> [v2:172.21.15.100:6834/2597849957,v1:172.21.15.100:6835/2597849957] conn(0x56505d708930 0x56505d70ad20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:35:27.810 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:27.806+0000 7f117dd72740 4 client.18792 mds_command: new command op to 34454 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T10:35:27.810 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:27.806+0000 7f117dd72740 1 -- 172.21.15.47:0/1384479827 --> [v2:172.21.15.100:6834/2597849957,v1:172.21.15.100:6835/2597849957] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56505d660800 con 0x56505d708930 2024-03-21T10:35:27.810 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:27.807+0000 7f1173400640 1 --2- 172.21.15.47:0/1384479827 >> [v2:172.21.15.100:6834/2597849957,v1:172.21.15.100:6835/2597849957] conn(0x56505d708930 0x56505d70ad20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:35:27.810 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:27.807+0000 7f1173400640 1 --2- 172.21.15.47:0/1384479827 >> [v2:172.21.15.100:6834/2597849957,v1:172.21.15.100:6835/2597849957] conn(0x56505d708930 0x56505d70ad20 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=f14cf61822313496 server_cookie=11eabad14b738917 in_seq=0 out_seq=0 2024-03-21T10:35:27.810 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:27.807+0000 7f115e000640 10 client.18792 ms_handle_connect on v2:172.21.15.100:6834/2597849957 2024-03-21T10:35:27.811 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:27.809+0000 7f115e000640 1 -- 172.21.15.47:0/1384479827 <== mds.2 v2:172.21.15.100:6834/2597849957 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x56505d660800 con 0x56505d708930 2024-03-21T10:35:27.811 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:27.809+0000 7f115e000640 10 client.18792 handle_command_reply: tid=0 2024-03-21T10:35:27.811 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:27.809+0000 7f117dd72740 1 client.18792 shutdown 2024-03-21T10:35:27.811 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:27.809+0000 7f117dd72740 1 -- 172.21.15.47:0/1384479827 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56505d6ef6d0 msgr2=0x56505d6f8460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:35:27.811 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:27.809+0000 7f117dd72740 1 --2- 172.21.15.47:0/1384479827 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56505d6ef6d0 0x56505d6f8460 secure :-1 s=READY pgs=1307 cs=0 l=1 rev1=1 crypto rx=0x7f116c009b00 tx=0x7f116c009c70 comp rx=0 tx=0).stop 2024-03-21T10:35:27.811 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:27.810+0000 7f117dd72740 1 -- 172.21.15.47:0/1384479827 shutdown_connections 2024-03-21T10:35:27.811 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:27.810+0000 7f117dd72740 1 --2- 172.21.15.47:0/1384479827 >> [v2:172.21.15.100:6834/2597849957,v1:172.21.15.100:6835/2597849957] conn(0x56505d708930 0x56505d70ad20 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:35:27.811 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:27.810+0000 7f117dd72740 1 --2- 172.21.15.47:0/1384479827 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56505d6f8c00 0x56505d6f5d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:35:27.811 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:27.810+0000 7f117dd72740 1 --2- 172.21.15.47:0/1384479827 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56505d6f0080 0x56505d6f57f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:35:27.811 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:27.810+0000 7f117dd72740 1 --2- 172.21.15.47:0/1384479827 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56505d6ef6d0 0x56505d6f8460 unknown :-1 s=CLOSED pgs=1307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:35:27.811 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:27.810+0000 7f117dd72740 1 -- 172.21.15.47:0/1384479827 >> 172.21.15.47:0/1384479827 conn(0x56505d6e9da0 msgr2=0x56505d6ebaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:35:27.811 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:27.810+0000 7f115e000640 0 client.18792 ms_handle_reset on v2:172.21.15.100:6834/2597849957 2024-03-21T10:35:27.811 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:27.810+0000 7f117dd72740 1 -- 172.21.15.47:0/1384479827 shutdown_connections 2024-03-21T10:35:27.811 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:27.810+0000 7f117dd72740 1 -- 172.21.15.47:0/1384479827 wait complete. 2024-03-21T10:35:27.812 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:27.810+0000 7f117dd72740 20 client.18792 trim_cache size 0 max 16384 2024-03-21T10:35:27.825 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:27.825+0000 7f117dd72740 1 Processor -- start 2024-03-21T10:35:27.825 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:27.826+0000 7f117dd72740 1 -- start start 2024-03-21T10:35:27.825 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:27.826+0000 7f117dd72740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56505d6ef6d0 0x56505d6f4d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:35:27.825 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:27.826+0000 7f117dd72740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56505d6f0080 0x56505d6f7f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:35:27.825 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:27.826+0000 7f117dd72740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56505d6f8c00 0x56505d6f8460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:35:27.825 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:27.826+0000 7f117dd72740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11741b0ba0 con 0x56505d6f0080 2024-03-21T10:35:27.825 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:27.826+0000 7f117dd72740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11741b2520 con 0x56505d6f8c00 2024-03-21T10:35:27.825 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:27.826+0000 7f117dd72740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11741b1200 con 0x56505d6ef6d0 2024-03-21T10:35:27.826 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:27.826+0000 7f1173400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56505d6f0080 0x56505d6f7f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:35:27.826 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:27.826+0000 7f1173400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56505d6f0080 0x56505d6f7f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:58858/0 (socket says 172.21.15.47:58858) 2024-03-21T10:35:27.826 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:27.826+0000 7f1173400640 1 -- 172.21.15.47:0/470786251 learned_addr learned my addr 172.21.15.47:0/470786251 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:35:27.827 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:27.826+0000 7f1179000640 1 --2- 172.21.15.47:0/470786251 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56505d6f8c00 0x56505d6f8460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:35:27.828 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:27.827+0000 7f1179000640 1 --2- 172.21.15.47:0/470786251 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56505d6ef6d0 0x56505d6f4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:35:27.828 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:27.827+0000 7f1173400640 1 -- 172.21.15.47:0/470786251 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56505d6ef6d0 msgr2=0x56505d6f4d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:35:27.828 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:27.827+0000 7f1173400640 1 --2- 172.21.15.47:0/470786251 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56505d6ef6d0 0x56505d6f4d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:35:27.828 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:27.827+0000 7f1173400640 1 -- 172.21.15.47:0/470786251 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56505d6f8c00 msgr2=0x56505d6f8460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:35:27.828 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:27.827+0000 7f1173400640 1 --2- 172.21.15.47:0/470786251 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56505d6f8c00 0x56505d6f8460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:35:27.828 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:27.827+0000 7f1173400640 1 -- 172.21.15.47:0/470786251 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56505d6efe20 con 0x56505d6f0080 2024-03-21T10:35:27.828 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:27.827+0000 7f1179000640 1 --2- 172.21.15.47:0/470786251 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56505d6ef6d0 0x56505d6f4d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T10:35:27.828 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:27.827+0000 7f1173400640 1 --2- 172.21.15.47:0/470786251 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56505d6f0080 0x56505d6f7f20 secure :-1 s=READY pgs=1308 cs=0 l=1 rev1=1 crypto rx=0x7f11600096c0 tx=0x7f1160004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=543636a615a41af6 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:35:27.828 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:27.827+0000 7f1168c00640 1 -- 172.21.15.47:0/470786251 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f116004e070 con 0x56505d6f0080 2024-03-21T10:35:27.828 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:27.828+0000 7f1168c00640 1 -- 172.21.15.47:0/470786251 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f116002cd20 con 0x56505d6f0080 2024-03-21T10:35:27.828 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:27.828+0000 7f1168c00640 1 -- 172.21.15.47:0/470786251 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1160005640 con 0x56505d6f0080 2024-03-21T10:35:27.828 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:27.828+0000 7f117dd72740 1 -- 172.21.15.47:0/470786251 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56505d6f0080 msgr2=0x56505d6f7f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:35:27.829 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:27.828+0000 7f117dd72740 1 --2- 172.21.15.47:0/470786251 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56505d6f0080 0x56505d6f7f20 secure :-1 s=READY pgs=1308 cs=0 l=1 rev1=1 crypto rx=0x7f11600096c0 tx=0x7f1160004040 comp rx=0 tx=0).stop 2024-03-21T10:35:27.829 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:27.828+0000 7f117dd72740 1 -- 172.21.15.47:0/470786251 shutdown_connections 2024-03-21T10:35:27.829 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:27.828+0000 7f117dd72740 1 --2- 172.21.15.47:0/470786251 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56505d6f8c00 0x56505d6f8460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:35:27.829 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:27.828+0000 7f117dd72740 1 --2- 172.21.15.47:0/470786251 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56505d6f0080 0x56505d6f7f20 unknown :-1 s=CLOSED pgs=1308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:35:27.829 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:27.828+0000 7f117dd72740 1 --2- 172.21.15.47:0/470786251 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56505d6ef6d0 0x56505d6f4d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:35:27.829 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:27.828+0000 7f117dd72740 1 -- 172.21.15.47:0/470786251 >> 172.21.15.47:0/470786251 conn(0x56505d6e9da0 msgr2=0x56505d6f0cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:35:27.829 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:27.829+0000 7f117dd72740 1 -- 172.21.15.47:0/470786251 shutdown_connections 2024-03-21T10:35:27.829 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:27.829+0000 7f117dd72740 1 -- 172.21.15.47:0/470786251 wait complete. 2024-03-21T10:35:27.832 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:27.829+0000 7f117dd72740 1 -- start start 2024-03-21T10:35:27.832 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:27.829+0000 7f117dd72740 1 Processor -- start 2024-03-21T10:35:27.832 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:27.830+0000 7f117dd72740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56505d6ef6d0 0x56505d700b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:35:27.832 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:27.830+0000 7f117dd72740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56505d6f0080 0x56505d6fbf00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:35:27.833 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:27.830+0000 7f117dd72740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56505d6f8c00 0x56505d6fc440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:35:27.833 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:27.830+0000 7f117dd72740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11741b1ec0 con 0x56505d6f0080 2024-03-21T10:35:27.833 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:27.830+0000 7f117dd72740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11741b1860 con 0x56505d6f8c00 2024-03-21T10:35:27.833 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:27.830+0000 7f117dd72740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11741aed80 con 0x56505d6ef6d0 2024-03-21T10:35:27.833 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:27.830+0000 7f117dd72740 10 client.0 _ll_register_callbacks cb 0x7f11640d2e50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T10:35:27.833 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:27.830+0000 7f1173400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56505d6f0080 0x56505d6fbf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:35:27.833 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:27.830+0000 7f1173400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56505d6f0080 0x56505d6fbf00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:58866/0 (socket says 172.21.15.47:58866) 2024-03-21T10:35:27.833 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:27.830+0000 7f1173400640 1 -- 172.21.15.47:0/201477297 learned_addr learned my addr 172.21.15.47:0/201477297 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:35:27.833 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:27.831+0000 7f1179000640 1 --2- 172.21.15.47:0/201477297 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56505d6f8c00 0x56505d6fc440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:35:27.833 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:27.831+0000 7f1179000640 1 --2- 172.21.15.47:0/201477297 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56505d6ef6d0 0x56505d700b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:35:27.833 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:27.831+0000 7f1173400640 1 -- 172.21.15.47:0/201477297 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56505d6ef6d0 msgr2=0x56505d700b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:35:27.833 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:27.831+0000 7f1173400640 1 --2- 172.21.15.47:0/201477297 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56505d6ef6d0 0x56505d700b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:35:27.834 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:27.831+0000 7f1173400640 1 -- 172.21.15.47:0/201477297 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56505d6f8c00 msgr2=0x56505d6fc440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:35:27.834 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:27.831+0000 7f1173400640 1 --2- 172.21.15.47:0/201477297 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56505d6f8c00 0x56505d6fc440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:35:27.834 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:27.831+0000 7f1173400640 1 -- 172.21.15.47:0/201477297 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56505d6fcae0 con 0x56505d6f0080 2024-03-21T10:35:27.834 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:27.831+0000 7f1173400640 1 --2- 172.21.15.47:0/201477297 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56505d6f0080 0x56505d6fbf00 secure :-1 s=READY pgs=1309 cs=0 l=1 rev1=1 crypto rx=0x7f116002bd00 tx=0x7f1160005040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2f4621ae0d827a16 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:35:27.834 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:27.831+0000 7f115e000640 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-03-21T10:35:27.834 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:27.831+0000 7f115e000640 1 -- 172.21.15.47:0/201477297 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f116004e070 con 0x56505d6f0080 2024-03-21T10:35:27.834 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:27.831+0000 7f115e000640 1 -- 172.21.15.47:0/201477297 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(105..105 src has 1..105) v4 ==== 9562+0+0 (secure 0 0 0) 0x7f11600055d0 con 0x56505d6f0080 2024-03-21T10:35:27.834 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:27.831+0000 7f117dd72740 1 -- 172.21.15.47:0/201477297 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f116002cd20 con 0x56505d6f0080 2024-03-21T10:35:27.834 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:27.832+0000 7f115e000640 1 -- 172.21.15.47:0/201477297 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f11600041f0 con 0x56505d6f0080 2024-03-21T10:35:27.835 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:27.832+0000 7f117dd72740 10 client.18798 fetch_fsmap learned FSMap version 32 2024-03-21T10:35:27.835 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:27.832+0000 7f117dd72740 1 -- 172.21.15.47:0/201477297 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x56505d763e00 con 0x56505d6f0080 2024-03-21T10:35:27.835 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:27.832+0000 7f115e000640 1 -- 172.21.15.47:0/201477297 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f116002cd20 con 0x56505d6f0080 2024-03-21T10:35:27.835 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:27.832+0000 7f115e000640 1 -- 172.21.15.47:0/201477297 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 32) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f1160004c60 con 0x56505d6f0080 2024-03-21T10:35:27.835 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:27.832+0000 7f117dd72740 10 client.18798 fetch_fsmap finished waiting for FSMap version 32 2024-03-21T10:35:27.835 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:27.832+0000 7f117dd72740 10 client.18798 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.e 2024-03-21T10:35:27.835 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:27.832+0000 7f117dd72740 20 client.18798 populate_metadata read hostname 'smithi047' 2024-03-21T10:35:27.835 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:27.832+0000 7f117dd72740 1 --2- 172.21.15.47:0/201477297 >> [v2:172.21.15.100:6834/2597849957,v1:172.21.15.100:6835/2597849957] conn(0x56505d7644b0 0x56505d7668d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:35:27.835 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:27.832+0000 7f117dd72740 4 client.18798 mds_command: new command op to 34454 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-21T10:35:27.835 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:27.832+0000 7f117dd72740 1 -- 172.21.15.47:0/201477297 --> [v2:172.21.15.100:6834/2597849957,v1:172.21.15.100:6835/2597849957] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x56505d65ab50 con 0x56505d7644b0 2024-03-21T10:35:27.836 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:27.833+0000 7f1179000640 1 --2- 172.21.15.47:0/201477297 >> [v2:172.21.15.100:6834/2597849957,v1:172.21.15.100:6835/2597849957] conn(0x56505d7644b0 0x56505d7668d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:35:27.837 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:27.834+0000 7f1179000640 1 --2- 172.21.15.47:0/201477297 >> [v2:172.21.15.100:6834/2597849957,v1:172.21.15.100:6835/2597849957] conn(0x56505d7644b0 0x56505d7668d0 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=b6a5e417afaeff7b server_cookie=bf2535e2cb578de0 in_seq=0 out_seq=0 2024-03-21T10:35:27.837 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:27.834+0000 7f115e000640 10 client.18798 ms_handle_connect on v2:172.21.15.100:6834/2597849957 2024-03-21T10:35:27.837 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:27.834+0000 7f115e000640 1 -- 172.21.15.47:0/201477297 <== mds.2 v2:172.21.15.100:6834/2597849957 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x56505d65ab50 con 0x56505d7644b0 2024-03-21T10:35:27.837 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:27.834+0000 7f115e000640 10 client.18798 handle_command_reply: tid=0 2024-03-21T10:35:27.837 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:27.834+0000 7f117dd72740 1 client.18798 shutdown 2024-03-21T10:35:27.837 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:27.834+0000 7f117dd72740 1 -- 172.21.15.47:0/201477297 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56505d6f0080 msgr2=0x56505d6fbf00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:35:27.837 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:27.834+0000 7f117dd72740 1 --2- 172.21.15.47:0/201477297 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56505d6f0080 0x56505d6fbf00 secure :-1 s=READY pgs=1309 cs=0 l=1 rev1=1 crypto rx=0x7f116002bd00 tx=0x7f1160005040 comp rx=0 tx=0).stop 2024-03-21T10:35:27.837 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:27.834+0000 7f117dd72740 1 -- 172.21.15.47:0/201477297 shutdown_connections 2024-03-21T10:35:27.837 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:27.834+0000 7f117dd72740 1 --2- 172.21.15.47:0/201477297 >> [v2:172.21.15.100:6834/2597849957,v1:172.21.15.100:6835/2597849957] conn(0x56505d7644b0 0x56505d7668d0 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:35:27.837 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:27.834+0000 7f117dd72740 1 --2- 172.21.15.47:0/201477297 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56505d6f8c00 0x56505d6fc440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:35:27.837 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:27.834+0000 7f117dd72740 1 --2- 172.21.15.47:0/201477297 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56505d6f0080 0x56505d6fbf00 unknown :-1 s=CLOSED pgs=1309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:35:27.837 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:27.834+0000 7f117dd72740 1 --2- 172.21.15.47:0/201477297 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56505d6ef6d0 0x56505d700b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:35:27.838 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:27.834+0000 7f115e000640 0 client.18798 ms_handle_reset on v2:172.21.15.100:6834/2597849957 2024-03-21T10:35:27.838 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:27.834+0000 7f117dd72740 1 -- 172.21.15.47:0/201477297 >> 172.21.15.47:0/201477297 conn(0x56505d6e9da0 msgr2=0x56505d6f0460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:35:27.838 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:27.834+0000 7f117dd72740 1 -- 172.21.15.47:0/201477297 shutdown_connections 2024-03-21T10:35:27.838 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:27.835+0000 7f117dd72740 1 -- 172.21.15.47:0/201477297 wait complete. 2024-03-21T10:35:27.838 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:27.835+0000 7f117dd72740 20 client.18798 trim_cache size 0 max 16384 2024-03-21T10:35:27.838 INFO:teuthology.orchestra.run.smithi047.stdout:[] 2024-03-21T10:35:27.839 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:27.835+0000 7f117b000640 1 -- 172.21.15.47:0/2041486107 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f11580a3110 msgr2=0x7f11580a55d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:35:27.839 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:27.835+0000 7f117b000640 1 --2- 172.21.15.47:0/2041486107 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f11580a3110 0x7f11580a55d0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x56505d6ef1a0 tx=0x7f1164097040 comp rx=0 tx=0).stop 2024-03-21T10:35:27.839 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:27.835+0000 7f117b000640 1 -- 172.21.15.47:0/2041486107 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f11741028e0 msgr2=0x7f117419a0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:35:27.839 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:27.835+0000 7f117b000640 1 --2- 172.21.15.47:0/2041486107 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f11741028e0 0x7f117419a0b0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f116400d950 tx=0x7f116400de10 comp rx=0 tx=0).stop 2024-03-21T10:35:27.839 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:27.835+0000 7f117b000640 1 -- 172.21.15.47:0/2041486107 shutdown_connections 2024-03-21T10:35:27.839 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:27.835+0000 7f117b000640 1 --2- 172.21.15.47:0/2041486107 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f117410bcb0 0x7f117419e920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:35:27.840 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:27.835+0000 7f117b000640 1 --2- 172.21.15.47:0/2041486107 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1174103200 0x7f117419a5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:35:27.840 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:27.835+0000 7f117b000640 1 --2- 172.21.15.47:0/2041486107 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f11580a3110 0x7f11580a55d0 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:35:27.840 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:27.835+0000 7f117b000640 1 --2- 172.21.15.47:0/2041486107 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f11741028e0 0x7f117419a0b0 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:35:27.840 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:27.835+0000 7f117b000640 1 -- 172.21.15.47:0/2041486107 >> 172.21.15.47:0/2041486107 conn(0x7f1174076920 msgr2=0x7f1174100500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:35:27.840 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:27.836+0000 7f117b000640 1 -- 172.21.15.47:0/2041486107 shutdown_connections 2024-03-21T10:35:27.840 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:27.836+0000 7f117b000640 1 -- 172.21.15.47:0/2041486107 wait complete. 2024-03-21T10:35:27.852 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-21T10:35:27.852 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-03-21T10:35:28.059 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:28.060+0000 7f5b49800640 1 -- 172.21.15.47:0/2895522061 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5b44102910 msgr2=0x7f5b44102cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:35:28.060 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:28.060+0000 7f5b49800640 1 --2- 172.21.15.47:0/2895522061 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5b44102910 0x7f5b44102cf0 secure :-1 s=READY pgs=1310 cs=0 l=1 rev1=1 crypto rx=0x7f5b34009ca0 tx=0x7f5b3402c180 comp rx=0 tx=0).stop 2024-03-21T10:35:28.060 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:28.060+0000 7f5b49800640 1 -- 172.21.15.47:0/2895522061 shutdown_connections 2024-03-21T10:35:28.060 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:28.060+0000 7f5b49800640 1 --2- 172.21.15.47:0/2895522061 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5b4410bce0 0x7f5b4410e150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:35:28.060 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:28.060+0000 7f5b49800640 1 --2- 172.21.15.47:0/2895522061 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5b44103230 0x7f5b4410b710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:35:28.060 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:28.060+0000 7f5b49800640 1 --2- 172.21.15.47:0/2895522061 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5b44102910 0x7f5b44102cf0 unknown :-1 s=CLOSED pgs=1310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:35:28.060 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:28.060+0000 7f5b49800640 1 -- 172.21.15.47:0/2895522061 >> 172.21.15.47:0/2895522061 conn(0x7f5b44077700 msgr2=0x7f5b44077b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:35:28.060 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:28.060+0000 7f5b49800640 1 -- 172.21.15.47:0/2895522061 shutdown_connections 2024-03-21T10:35:28.060 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:28.060+0000 7f5b49800640 1 -- 172.21.15.47:0/2895522061 wait complete. 2024-03-21T10:35:28.060 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:28.062+0000 7f5b49800640 1 Processor -- start 2024-03-21T10:35:28.061 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:28.062+0000 7f5b49800640 1 -- start start 2024-03-21T10:35:28.062 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:28.063+0000 7f5b49800640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5b44102910 0x7f5b44197ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:35:28.062 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:28.063+0000 7f5b49800640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5b44103230 0x7f5b44198400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:35:28.063 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:28.063+0000 7f5b49800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5b4410bce0 0x7f5b4419c780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:35:28.063 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:28.063+0000 7f5b49800640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b44111960 con 0x7f5b44102910 2024-03-21T10:35:28.063 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:28.063+0000 7f5b49800640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b441117e0 con 0x7f5b4410bce0 2024-03-21T10:35:28.063 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:28.063+0000 7f5b49800640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b44111ae0 con 0x7f5b44103230 2024-03-21T10:35:28.063 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:28.063+0000 7f5b42a00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5b44102910 0x7f5b44197ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:35:28.063 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:28.063+0000 7f5b42a00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5b44102910 0x7f5b44197ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:58894/0 (socket says 172.21.15.47:58894) 2024-03-21T10:35:28.063 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:28.063+0000 7f5b42a00640 1 -- 172.21.15.47:0/1101816119 learned_addr learned my addr 172.21.15.47:0/1101816119 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:35:28.063 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:28.063+0000 7f5b43400640 1 --2- 172.21.15.47:0/1101816119 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5b4410bce0 0x7f5b4419c780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:35:28.063 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:28.063+0000 7f5b3a000640 1 --2- 172.21.15.47:0/1101816119 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5b44103230 0x7f5b44198400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:35:28.063 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:28.064+0000 7f5b42a00640 1 -- 172.21.15.47:0/1101816119 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5b44103230 msgr2=0x7f5b44198400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:35:28.063 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:28.064+0000 7f5b42a00640 1 --2- 172.21.15.47:0/1101816119 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5b44103230 0x7f5b44198400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:35:28.063 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:28.064+0000 7f5b42a00640 1 -- 172.21.15.47:0/1101816119 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5b4410bce0 msgr2=0x7f5b4419c780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:35:28.063 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:28.064+0000 7f5b42a00640 1 --2- 172.21.15.47:0/1101816119 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5b4410bce0 0x7f5b4419c780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:35:28.063 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:28.064+0000 7f5b42a00640 1 -- 172.21.15.47:0/1101816119 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b4419ceb0 con 0x7f5b44102910 2024-03-21T10:35:28.064 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:28.064+0000 7f5b42a00640 1 --2- 172.21.15.47:0/1101816119 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5b44102910 0x7f5b44197ec0 secure :-1 s=READY pgs=1311 cs=0 l=1 rev1=1 crypto rx=0x7f5b34005e30 tx=0x7f5b34003c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:35:28.064 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:28.064+0000 7f5b3be00640 1 -- 172.21.15.47:0/1101816119 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b34039040 con 0x7f5b44102910 2024-03-21T10:35:28.064 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:28.064+0000 7f5b3be00640 1 -- 172.21.15.47:0/1101816119 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f5b34044070 con 0x7f5b44102910 2024-03-21T10:35:28.064 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:28.064+0000 7f5b3be00640 1 -- 172.21.15.47:0/1101816119 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b340329b0 con 0x7f5b44102910 2024-03-21T10:35:28.065 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:28.064+0000 7f5b49800640 1 -- 172.21.15.47:0/1101816119 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5b4419d140 con 0x7f5b44102910 2024-03-21T10:35:28.065 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:28.065+0000 7f5b49800640 1 -- 172.21.15.47:0/1101816119 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5b441a4a30 con 0x7f5b44102910 2024-03-21T10:35:28.066 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:28.068+0000 7f5b3be00640 1 -- 172.21.15.47:0/1101816119 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 27) v1 ==== 137985+0+0 (secure 0 0 0) 0x7f5b34032b50 con 0x7f5b44102910 2024-03-21T10:35:28.067 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:28.069+0000 7f5b4c5a6740 1 Processor -- start 2024-03-21T10:35:28.068 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:28.069+0000 7f5b4c5a6740 1 -- start start 2024-03-21T10:35:28.069 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:28.069+0000 7f5b4c5a6740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5650256d3450 0x5650256d3830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:35:28.069 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:28.069+0000 7f5b3be00640 1 --2- 172.21.15.47:0/1101816119 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f5b280a3110 0x7f5b280a55d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:35:28.069 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:28.069+0000 7f5b4c5a6740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5650256d3e00 0x5650256e0a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:35:28.069 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:28.069+0000 7f5b4c5a6740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5650256e1010 0x5650256e3420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:35:28.069 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:28.069+0000 7f5b4c5a6740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x5650256372c0 con 0x5650256d3e00 2024-03-21T10:35:28.069 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:28.069+0000 7f5b4c5a6740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x5650255dfc90 con 0x5650256e1010 2024-03-21T10:35:28.069 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:28.069+0000 7f5b4c5a6740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x565025633360 con 0x5650256d3450 2024-03-21T10:35:28.069 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:28.070+0000 7f5b3be00640 1 -- 172.21.15.47:0/1101816119 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(105..105 src has 1..105) v4 ==== 9562+0+0 (secure 0 0 0) 0x7f5b340f2100 con 0x7f5b44102910 2024-03-21T10:35:28.069 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:28.070+0000 7f5b3a000640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5650256d3e00 0x5650256e0a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:35:28.070 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:28.070+0000 7f5b42a00640 1 --2- 172.21.15.47:0/1101816119 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f5b280a3110 0x7f5b280a55d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:35:28.070 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:28.070+0000 7f5b3a000640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5650256d3e00 0x5650256e0a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:58898/0 (socket says 172.21.15.47:58898) 2024-03-21T10:35:28.070 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:28.070+0000 7f5b3a000640 1 -- 172.21.15.47:0/1096102361 learned_addr learned my addr 172.21.15.47:0/1096102361 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:35:28.070 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:28.070+0000 7f5b43400640 1 --2- 172.21.15.47:0/1096102361 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5650256e1010 0x5650256e3420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:35:28.070 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:28.070+0000 7f5b43400640 1 --2- 172.21.15.47:0/1096102361 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5650256d3450 0x5650256d3830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:35:28.071 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:28.070+0000 7f5b3a000640 1 -- 172.21.15.47:0/1096102361 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5650256d3450 msgr2=0x5650256d3830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:35:28.071 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:28.070+0000 7f5b3a000640 1 --2- 172.21.15.47:0/1096102361 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5650256d3450 0x5650256d3830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:35:28.071 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:28.070+0000 7f5b3a000640 1 -- 172.21.15.47:0/1096102361 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5650256e1010 msgr2=0x5650256e3420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:35:28.071 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:28.070+0000 7f5b3a000640 1 --2- 172.21.15.47:0/1096102361 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5650256e1010 0x5650256e3420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:35:28.071 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:28.070+0000 7f5b3a000640 1 -- 172.21.15.47:0/1096102361 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x565025604080 con 0x5650256d3e00 2024-03-21T10:35:28.071 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:28.070+0000 7f5b43400640 1 --2- 172.21.15.47:0/1096102361 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5650256d3450 0x5650256d3830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T10:35:28.072 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:28.071+0000 7f5b42a00640 1 --2- 172.21.15.47:0/1101816119 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f5b280a3110 0x7f5b280a55d0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f5b340ef550 tx=0x7f5b34005c30 comp rx=0 tx=0).ready entity=mgr.15687 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:35:28.072 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:28.071+0000 7f5b3a000640 1 --2- 172.21.15.47:0/1096102361 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5650256d3e00 0x5650256e0a40 secure :-1 s=READY pgs=1312 cs=0 l=1 rev1=1 crypto rx=0x7f5b2c00bc50 tx=0x7f5b2c00f730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b639f3a3d4702184 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:35:28.072 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:28.071+0000 7f5b33e00640 1 -- 172.21.15.47:0/1096102361 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b2c015070 con 0x5650256d3e00 2024-03-21T10:35:28.072 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:28.071+0000 7f5b33e00640 1 -- 172.21.15.47:0/1096102361 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f5b2c01a910 con 0x5650256d3e00 2024-03-21T10:35:28.072 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:28.071+0000 7f5b33e00640 1 -- 172.21.15.47:0/1096102361 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b2c0118a0 con 0x5650256d3e00 2024-03-21T10:35:28.072 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:28.072+0000 7f5b4c5a6740 1 -- 172.21.15.47:0/1096102361 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5650256d3e00 msgr2=0x5650256e0a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:35:28.072 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:28.072+0000 7f5b4c5a6740 1 --2- 172.21.15.47:0/1096102361 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5650256d3e00 0x5650256e0a40 secure :-1 s=READY pgs=1312 cs=0 l=1 rev1=1 crypto rx=0x7f5b2c00bc50 tx=0x7f5b2c00f730 comp rx=0 tx=0).stop 2024-03-21T10:35:28.072 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:28.072+0000 7f5b4c5a6740 1 -- 172.21.15.47:0/1096102361 shutdown_connections 2024-03-21T10:35:28.072 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:28.072+0000 7f5b4c5a6740 1 --2- 172.21.15.47:0/1096102361 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5650256e1010 0x5650256e3420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:35:28.072 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:28.072+0000 7f5b4c5a6740 1 --2- 172.21.15.47:0/1096102361 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5650256d3e00 0x5650256e0a40 unknown :-1 s=CLOSED pgs=1312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:35:28.072 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:28.072+0000 7f5b4c5a6740 1 --2- 172.21.15.47:0/1096102361 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5650256d3450 0x5650256d3830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:35:28.072 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:28.072+0000 7f5b4c5a6740 1 -- 172.21.15.47:0/1096102361 >> 172.21.15.47:0/1096102361 conn(0x5650256cdda0 msgr2=0x5650256ce180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:35:28.075 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:28.073+0000 7f5b4c5a6740 1 -- 172.21.15.47:0/1096102361 shutdown_connections 2024-03-21T10:35:28.075 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:28.073+0000 7f5b4c5a6740 1 -- 172.21.15.47:0/1096102361 wait complete. 2024-03-21T10:35:28.075 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:28.073+0000 7f5b4c5a6740 1 -- start start 2024-03-21T10:35:28.076 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:28.073+0000 7f5b4c5a6740 1 Processor -- start 2024-03-21T10:35:28.076 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:28.074+0000 7f5b4c5a6740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5650256d3450 0x5650256e0930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:35:28.076 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:28.074+0000 7f5b4c5a6740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5650256d3e00 0x5650256ddcc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:35:28.076 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:28.074+0000 7f5b4c5a6740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5650256e1010 0x5650256de260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:35:28.076 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:28.074+0000 7f5b4c5a6740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x5650256e3f40 con 0x5650256d3450 2024-03-21T10:35:28.076 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:28.074+0000 7f5b4c5a6740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x5650255f30d0 con 0x5650256d3e00 2024-03-21T10:35:28.076 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:28.074+0000 7f5b4c5a6740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x5650256e40c0 con 0x5650256e1010 2024-03-21T10:35:28.076 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:28.074+0000 7f5b4c5a6740 10 client.0 _ll_register_callbacks cb 0x5650255f7e80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T10:35:28.076 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:28.074+0000 7f5b43400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5650256d3450 0x5650256e0930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:35:28.076 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:28.074+0000 7f5b43400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5650256d3450 0x5650256e0930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:58900/0 (socket says 172.21.15.47:58900) 2024-03-21T10:35:28.076 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:28.074+0000 7f5b43400640 1 -- 172.21.15.47:0/281172136 learned_addr learned my addr 172.21.15.47:0/281172136 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:35:28.076 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:28.074+0000 7f5b3a000640 1 --2- 172.21.15.47:0/281172136 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5650256d3e00 0x5650256ddcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:35:28.077 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:28.075+0000 7f5b43400640 1 -- 172.21.15.47:0/281172136 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5650256e1010 msgr2=0x5650256de260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:35:28.077 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:28.075+0000 7f5b43400640 1 --2- 172.21.15.47:0/281172136 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5650256e1010 0x5650256de260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:35:28.077 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:28.075+0000 7f5b43400640 1 -- 172.21.15.47:0/281172136 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5650256d3e00 msgr2=0x5650256ddcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:35:28.077 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:28.075+0000 7f5b43400640 1 --2- 172.21.15.47:0/281172136 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5650256d3e00 0x5650256ddcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:35:28.077 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:28.075+0000 7f5b43400640 1 -- 172.21.15.47:0/281172136 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56502534b3f0 con 0x5650256d3450 2024-03-21T10:35:28.077 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:28.075+0000 7f5b3a000640 1 --2- 172.21.15.47:0/281172136 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5650256d3e00 0x5650256ddcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T10:35:28.077 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:28.075+0000 7f5b43400640 1 --2- 172.21.15.47:0/281172136 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5650256d3450 0x5650256e0930 secure :-1 s=READY pgs=1313 cs=0 l=1 rev1=1 crypto rx=0x7f5b3c010d90 tx=0x7f5b3c00d710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=59b6298ea5ed92f5 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:35:28.077 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:28.075+0000 7f5b30c00640 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-03-21T10:35:28.077 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:28.075+0000 7f5b30c00640 1 -- 172.21.15.47:0/281172136 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b3c01d440 con 0x5650256d3450 2024-03-21T10:35:28.077 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:28.075+0000 7f5b30c00640 1 -- 172.21.15.47:0/281172136 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(105..105 src has 1..105) v4 ==== 9562+0+0 (secure 0 0 0) 0x7f5b3c01c3e0 con 0x5650256d3450 2024-03-21T10:35:28.077 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:28.075+0000 7f5b4c5a6740 1 -- 172.21.15.47:0/281172136 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f5b2c01a910 con 0x5650256d3450 2024-03-21T10:35:28.077 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:28.076+0000 7f5b30c00640 1 -- 172.21.15.47:0/281172136 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5b3c01d5e0 con 0x5650256d3450 2024-03-21T10:35:28.077 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:28.076+0000 7f5b4c5a6740 10 client.18810 fetch_fsmap learned FSMap version 32 2024-03-21T10:35:28.077 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:28.076+0000 7f5b4c5a6740 1 -- 172.21.15.47:0/281172136 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x5650254157d0 con 0x5650256d3450 2024-03-21T10:35:28.078 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:28.076+0000 7f5b30c00640 1 -- 172.21.15.47:0/281172136 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f5b2c01a910 con 0x5650256d3450 2024-03-21T10:35:28.078 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:28.076+0000 7f5b30c00640 1 -- 172.21.15.47:0/281172136 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 32) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f5b3c01b070 con 0x5650256d3450 2024-03-21T10:35:28.078 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:28.076+0000 7f5b4c5a6740 10 client.18810 fetch_fsmap finished waiting for FSMap version 32 2024-03-21T10:35:28.078 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:28.076+0000 7f5b4c5a6740 10 client.18810 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-03-21T10:35:28.078 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:28.076+0000 7f5b4c5a6740 20 client.18810 populate_metadata read hostname 'smithi047' 2024-03-21T10:35:28.078 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:28.076+0000 7f5b4c5a6740 1 --2- 172.21.15.47:0/281172136 >> [v2:172.21.15.100:6836/3497644162,v1:172.21.15.100:6837/3497644162] conn(0x5650256f1090 0x5650256f34f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:35:28.078 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:28.076+0000 7f5b4c5a6740 4 client.18810 mds_command: new command op to 34493 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T10:35:28.078 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:28.076+0000 7f5b4c5a6740 1 -- 172.21.15.47:0/281172136 --> [v2:172.21.15.100:6836/3497644162,v1:172.21.15.100:6837/3497644162] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x565025644800 con 0x5650256f1090 2024-03-21T10:35:28.079 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:28.077+0000 7f5b3a000640 1 --2- 172.21.15.47:0/281172136 >> [v2:172.21.15.100:6836/3497644162,v1:172.21.15.100:6837/3497644162] conn(0x5650256f1090 0x5650256f34f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:35:28.080 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:28.078+0000 7f5b3a000640 1 --2- 172.21.15.47:0/281172136 >> [v2:172.21.15.100:6836/3497644162,v1:172.21.15.100:6837/3497644162] conn(0x5650256f1090 0x5650256f34f0 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=1ad864307dd224b5 server_cookie=9efb52758fdaa6bb in_seq=0 out_seq=0 2024-03-21T10:35:28.080 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:28.078+0000 7f5b30c00640 10 client.18810 ms_handle_connect on v2:172.21.15.100:6836/3497644162 2024-03-21T10:35:28.080 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:28.079+0000 7f5b30c00640 1 -- 172.21.15.47:0/281172136 <== mds.1 v2:172.21.15.100:6836/3497644162 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x565025644800 con 0x5650256f1090 2024-03-21T10:35:28.080 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:28.079+0000 7f5b30c00640 10 client.18810 handle_command_reply: tid=0 2024-03-21T10:35:28.080 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:28.079+0000 7f5b4c5a6740 1 client.18810 shutdown 2024-03-21T10:35:28.080 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:28.079+0000 7f5b4c5a6740 1 -- 172.21.15.47:0/281172136 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5650256d3450 msgr2=0x5650256e0930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:35:28.080 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:28.079+0000 7f5b4c5a6740 1 --2- 172.21.15.47:0/281172136 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5650256d3450 0x5650256e0930 secure :-1 s=READY pgs=1313 cs=0 l=1 rev1=1 crypto rx=0x7f5b3c010d90 tx=0x7f5b3c00d710 comp rx=0 tx=0).stop 2024-03-21T10:35:28.080 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:28.079+0000 7f5b4c5a6740 1 -- 172.21.15.47:0/281172136 shutdown_connections 2024-03-21T10:35:28.080 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:28.079+0000 7f5b4c5a6740 1 --2- 172.21.15.47:0/281172136 >> [v2:172.21.15.100:6836/3497644162,v1:172.21.15.100:6837/3497644162] conn(0x5650256f1090 0x5650256f34f0 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:35:28.080 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:28.079+0000 7f5b4c5a6740 1 --2- 172.21.15.47:0/281172136 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5650256e1010 0x5650256de260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:35:28.080 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:28.079+0000 7f5b4c5a6740 1 --2- 172.21.15.47:0/281172136 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5650256d3e00 0x5650256ddcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:35:28.080 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:28.079+0000 7f5b4c5a6740 1 --2- 172.21.15.47:0/281172136 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5650256d3450 0x5650256e0930 unknown :-1 s=CLOSED pgs=1313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:35:28.080 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:28.079+0000 7f5b4c5a6740 1 -- 172.21.15.47:0/281172136 >> 172.21.15.47:0/281172136 conn(0x5650256cdda0 msgr2=0x5650256cfaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:35:28.081 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:28.079+0000 7f5b30c00640 0 client.18810 ms_handle_reset on v2:172.21.15.100:6836/3497644162 2024-03-21T10:35:28.081 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:28.080+0000 7f5b4c5a6740 1 -- 172.21.15.47:0/281172136 shutdown_connections 2024-03-21T10:35:28.081 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:28.080+0000 7f5b4c5a6740 1 -- 172.21.15.47:0/281172136 wait complete. 2024-03-21T10:35:28.081 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:28.080+0000 7f5b4c5a6740 20 client.18810 trim_cache size 0 max 16384 2024-03-21T10:35:28.093 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:28.093+0000 7f5b4c5a6740 1 Processor -- start 2024-03-21T10:35:28.093 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:28.093+0000 7f5b4c5a6740 1 -- start start 2024-03-21T10:35:28.094 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:28.094+0000 7f5b4c5a6740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5650256d3450 0x5650256ccd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:35:28.094 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:28.094+0000 7f5b4c5a6740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5650256d3e00 0x5650256e03f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:35:28.094 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:28.094+0000 7f5b4c5a6740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5650256e1010 0x5650256e0930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:35:28.094 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:28.094+0000 7f5b4c5a6740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b441ac760 con 0x5650256d3e00 2024-03-21T10:35:28.094 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:28.094+0000 7f5b4c5a6740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b441ae0e0 con 0x5650256d3450 2024-03-21T10:35:28.094 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:28.094+0000 7f5b4c5a6740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b441acdc0 con 0x5650256e1010 2024-03-21T10:35:28.094 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:28.094+0000 7f5b3a000640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5650256d3e00 0x5650256e03f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:35:28.094 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:28.094+0000 7f5b3a000640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5650256d3e00 0x5650256e03f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:58910/0 (socket says 172.21.15.47:58910) 2024-03-21T10:35:28.094 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:28.094+0000 7f5b3a000640 1 -- 172.21.15.47:0/3562547351 learned_addr learned my addr 172.21.15.47:0/3562547351 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:35:28.094 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:28.094+0000 7f5b43400640 1 --2- 172.21.15.47:0/3562547351 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5650256d3450 0x5650256ccd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:35:28.094 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:28.094+0000 7f5b43400640 1 --2- 172.21.15.47:0/3562547351 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5650256e1010 0x5650256e0930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:35:28.094 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:28.094+0000 7f5b3a000640 1 -- 172.21.15.47:0/3562547351 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5650256e1010 msgr2=0x5650256e0930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:35:28.095 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:28.094+0000 7f5b3a000640 1 --2- 172.21.15.47:0/3562547351 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5650256e1010 0x5650256e0930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:35:28.095 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:28.094+0000 7f5b3a000640 1 -- 172.21.15.47:0/3562547351 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5650256d3450 msgr2=0x5650256ccd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:35:28.095 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:28.094+0000 7f5b3a000640 1 --2- 172.21.15.47:0/3562547351 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5650256d3450 0x5650256ccd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:35:28.095 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:28.094+0000 7f5b3a000640 1 -- 172.21.15.47:0/3562547351 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5650256d2760 con 0x5650256d3e00 2024-03-21T10:35:28.095 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:28.094+0000 7f5b43400640 1 --2- 172.21.15.47:0/3562547351 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5650256e1010 0x5650256e0930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T10:35:28.095 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:28.095+0000 7f5b3a000640 1 --2- 172.21.15.47:0/3562547351 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5650256d3e00 0x5650256e03f0 secure :-1 s=READY pgs=1314 cs=0 l=1 rev1=1 crypto rx=0x5650256d4240 tx=0x7f5b2c011440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7966c6362f24f407 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:35:28.099 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:28.095+0000 7f5b33400640 1 -- 172.21.15.47:0/3562547351 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b2c015040 con 0x5650256d3e00 2024-03-21T10:35:28.099 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:28.095+0000 7f5b33400640 1 -- 172.21.15.47:0/3562547351 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f5b2c026080 con 0x5650256d3e00 2024-03-21T10:35:28.100 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:28.095+0000 7f5b33400640 1 -- 172.21.15.47:0/3562547351 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b2c005540 con 0x5650256d3e00 2024-03-21T10:35:28.100 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:28.096+0000 7f5b4c5a6740 1 -- 172.21.15.47:0/3562547351 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5650256d3e00 msgr2=0x5650256e03f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:35:28.100 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:28.096+0000 7f5b4c5a6740 1 --2- 172.21.15.47:0/3562547351 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5650256d3e00 0x5650256e03f0 secure :-1 s=READY pgs=1314 cs=0 l=1 rev1=1 crypto rx=0x5650256d4240 tx=0x7f5b2c011440 comp rx=0 tx=0).stop 2024-03-21T10:35:28.100 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:28.096+0000 7f5b4c5a6740 1 -- 172.21.15.47:0/3562547351 shutdown_connections 2024-03-21T10:35:28.100 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:28.096+0000 7f5b4c5a6740 1 --2- 172.21.15.47:0/3562547351 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5650256e1010 0x5650256e0930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:35:28.100 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:28.096+0000 7f5b4c5a6740 1 --2- 172.21.15.47:0/3562547351 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5650256d3e00 0x5650256e03f0 unknown :-1 s=CLOSED pgs=1314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:35:28.100 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:28.096+0000 7f5b4c5a6740 1 --2- 172.21.15.47:0/3562547351 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5650256d3450 0x5650256ccd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:35:28.100 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:28.096+0000 7f5b4c5a6740 1 -- 172.21.15.47:0/3562547351 >> 172.21.15.47:0/3562547351 conn(0x5650256cdda0 msgr2=0x5650256cc620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:35:28.100 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:28.096+0000 7f5b4c5a6740 1 -- 172.21.15.47:0/3562547351 shutdown_connections 2024-03-21T10:35:28.100 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:28.096+0000 7f5b4c5a6740 1 -- 172.21.15.47:0/3562547351 wait complete. 2024-03-21T10:35:28.100 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:28.096+0000 7f5b4c5a6740 1 -- start start 2024-03-21T10:35:28.100 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:28.097+0000 7f5b4c5a6740 1 Processor -- start 2024-03-21T10:35:28.100 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:28.097+0000 7f5b4c5a6740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5650256d3450 0x5650256e69d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:35:28.100 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:28.097+0000 7f5b4c5a6740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5650256d3e00 0x5650256e6f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:35:28.101 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:28.097+0000 7f5b4c5a6740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5650256e1010 0x5650256e7450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:35:28.101 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:28.097+0000 7f5b4c5a6740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b441ada80 con 0x5650256e1010 2024-03-21T10:35:28.101 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:28.097+0000 7f5b4c5a6740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b441ad420 con 0x5650256d3e00 2024-03-21T10:35:28.101 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:28.097+0000 7f5b4c5a6740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b441aa940 con 0x5650256d3450 2024-03-21T10:35:28.101 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:28.097+0000 7f5b4c5a6740 10 client.0 _ll_register_callbacks cb 0x7f5b3c00aba0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T10:35:28.101 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:28.097+0000 7f5b43400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5650256e1010 0x5650256e7450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:35:28.101 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:28.097+0000 7f5b43400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5650256e1010 0x5650256e7450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:58926/0 (socket says 172.21.15.47:58926) 2024-03-21T10:35:28.101 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:28.097+0000 7f5b43400640 1 -- 172.21.15.47:0/1927864736 learned_addr learned my addr 172.21.15.47:0/1927864736 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:35:28.101 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:28.098+0000 7f5b3a000640 1 --2- 172.21.15.47:0/1927864736 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5650256d3e00 0x5650256e6f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:35:28.101 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:28.098+0000 7f5b43400640 1 --2- 172.21.15.47:0/1927864736 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5650256d3450 0x5650256e69d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:35:28.101 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:28.098+0000 7f5b43400640 1 -- 172.21.15.47:0/1927864736 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5650256d3450 msgr2=0x5650256e69d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:35:28.101 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:28.098+0000 7f5b43400640 1 --2- 172.21.15.47:0/1927864736 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5650256d3450 0x5650256e69d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:35:28.101 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:28.098+0000 7f5b43400640 1 -- 172.21.15.47:0/1927864736 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5650256d3e00 msgr2=0x5650256e6f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:35:28.101 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:28.098+0000 7f5b43400640 1 --2- 172.21.15.47:0/1927864736 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5650256d3e00 0x5650256e6f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:35:28.101 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:28.098+0000 7f5b43400640 1 -- 172.21.15.47:0/1927864736 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56502574bbe0 con 0x5650256e1010 2024-03-21T10:35:28.102 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:28.098+0000 7f5b43400640 1 --2- 172.21.15.47:0/1927864736 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5650256e1010 0x5650256e7450 secure :-1 s=READY pgs=1315 cs=0 l=1 rev1=1 crypto rx=0x7f5b3c010ae0 tx=0x7f5b3c013040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dec09a39729b7619 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:35:28.102 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:28.098+0000 7f5b30c00640 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-03-21T10:35:28.102 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:28.098+0000 7f5b30c00640 1 -- 172.21.15.47:0/1927864736 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b3c032020 con 0x5650256e1010 2024-03-21T10:35:28.102 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:28.098+0000 7f5b30c00640 1 -- 172.21.15.47:0/1927864736 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(105..105 src has 1..105) v4 ==== 9562+0+0 (secure 0 0 0) 0x7f5b3c01d780 con 0x5650256e1010 2024-03-21T10:35:28.102 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:28.098+0000 7f5b4c5a6740 1 -- 172.21.15.47:0/1927864736 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f5b2c026080 con 0x5650256e1010 2024-03-21T10:35:28.102 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:28.099+0000 7f5b30c00640 1 -- 172.21.15.47:0/1927864736 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5b3c03a050 con 0x5650256e1010 2024-03-21T10:35:28.102 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:28.099+0000 7f5b4c5a6740 10 client.18816 fetch_fsmap learned FSMap version 32 2024-03-21T10:35:28.102 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:28.099+0000 7f5b4c5a6740 1 -- 172.21.15.47:0/1927864736 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x56502574ce10 con 0x5650256e1010 2024-03-21T10:35:28.102 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:28.099+0000 7f5b30c00640 1 -- 172.21.15.47:0/1927864736 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f5b2c026080 con 0x5650256e1010 2024-03-21T10:35:28.102 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:28.099+0000 7f5b30c00640 1 -- 172.21.15.47:0/1927864736 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 32) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f5b3c01b040 con 0x5650256e1010 2024-03-21T10:35:28.102 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:28.099+0000 7f5b4c5a6740 10 client.18816 fetch_fsmap finished waiting for FSMap version 32 2024-03-21T10:35:28.102 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:28.099+0000 7f5b4c5a6740 10 client.18816 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-03-21T10:35:28.102 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:28.099+0000 7f5b4c5a6740 20 client.18816 populate_metadata read hostname 'smithi047' 2024-03-21T10:35:28.103 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:28.099+0000 7f5b4c5a6740 1 --2- 172.21.15.47:0/1927864736 >> [v2:172.21.15.100:6836/3497644162,v1:172.21.15.100:6837/3497644162] conn(0x56502574d4b0 0x56502574f8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:35:28.103 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:28.099+0000 7f5b4c5a6740 4 client.18816 mds_command: new command op to 34493 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-21T10:35:28.103 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:28.099+0000 7f5b4c5a6740 1 -- 172.21.15.47:0/1927864736 --> [v2:172.21.15.100:6836/3497644162,v1:172.21.15.100:6837/3497644162] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x56502563eb50 con 0x56502574d4b0 2024-03-21T10:35:28.103 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:28.100+0000 7f5b3a000640 1 --2- 172.21.15.47:0/1927864736 >> [v2:172.21.15.100:6836/3497644162,v1:172.21.15.100:6837/3497644162] conn(0x56502574d4b0 0x56502574f8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:35:28.103 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:28.101+0000 7f5b3a000640 1 --2- 172.21.15.47:0/1927864736 >> [v2:172.21.15.100:6836/3497644162,v1:172.21.15.100:6837/3497644162] conn(0x56502574d4b0 0x56502574f8d0 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=a3c69bd6792a84b0 server_cookie=48a72a9b8b213492 in_seq=0 out_seq=0 2024-03-21T10:35:28.103 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:28.101+0000 7f5b30c00640 10 client.18816 ms_handle_connect on v2:172.21.15.100:6836/3497644162 2024-03-21T10:35:28.104 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:28.101+0000 7f5b30c00640 1 -- 172.21.15.47:0/1927864736 <== mds.1 v2:172.21.15.100:6836/3497644162 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x56502563eb50 con 0x56502574d4b0 2024-03-21T10:35:28.104 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:28.101+0000 7f5b30c00640 10 client.18816 handle_command_reply: tid=0 2024-03-21T10:35:28.104 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:28.101+0000 7f5b4c5a6740 1 client.18816 shutdown 2024-03-21T10:35:28.104 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:28.101+0000 7f5b4c5a6740 1 -- 172.21.15.47:0/1927864736 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5650256e1010 msgr2=0x5650256e7450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:35:28.104 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:28.101+0000 7f5b4c5a6740 1 --2- 172.21.15.47:0/1927864736 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5650256e1010 0x5650256e7450 secure :-1 s=READY pgs=1315 cs=0 l=1 rev1=1 crypto rx=0x7f5b3c010ae0 tx=0x7f5b3c013040 comp rx=0 tx=0).stop 2024-03-21T10:35:28.104 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:28.101+0000 7f5b4c5a6740 1 -- 172.21.15.47:0/1927864736 shutdown_connections 2024-03-21T10:35:28.104 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:28.101+0000 7f5b4c5a6740 1 --2- 172.21.15.47:0/1927864736 >> [v2:172.21.15.100:6836/3497644162,v1:172.21.15.100:6837/3497644162] conn(0x56502574d4b0 0x56502574f8d0 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:35:28.104 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:28.101+0000 7f5b4c5a6740 1 --2- 172.21.15.47:0/1927864736 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5650256e1010 0x5650256e7450 unknown :-1 s=CLOSED pgs=1315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:35:28.105 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:28.102+0000 7f5b4c5a6740 1 --2- 172.21.15.47:0/1927864736 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5650256d3e00 0x5650256e6f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:35:28.105 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:28.102+0000 7f5b4c5a6740 1 --2- 172.21.15.47:0/1927864736 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5650256d3450 0x5650256e69d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:35:28.105 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:28.102+0000 7f5b4c5a6740 1 -- 172.21.15.47:0/1927864736 >> 172.21.15.47:0/1927864736 conn(0x5650256cdda0 msgr2=0x5650256cceb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:35:28.105 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:28.102+0000 7f5b30c00640 0 client.18816 ms_handle_reset on v2:172.21.15.100:6836/3497644162 2024-03-21T10:35:28.105 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:28.102+0000 7f5b4c5a6740 1 -- 172.21.15.47:0/1927864736 shutdown_connections 2024-03-21T10:35:28.105 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:28.102+0000 7f5b4c5a6740 1 -- 172.21.15.47:0/1927864736 wait complete. 2024-03-21T10:35:28.105 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:28.102+0000 7f5b4c5a6740 20 client.18816 trim_cache size 0 max 16384 2024-03-21T10:35:28.105 INFO:teuthology.orchestra.run.smithi047.stdout:[] 2024-03-21T10:35:28.106 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:28.102+0000 7f5b49800640 1 -- 172.21.15.47:0/1101816119 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f5b280a3110 msgr2=0x7f5b280a55d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:35:28.106 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:28.102+0000 7f5b49800640 1 --2- 172.21.15.47:0/1101816119 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f5b280a3110 0x7f5b280a55d0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f5b340ef550 tx=0x7f5b34005c30 comp rx=0 tx=0).stop 2024-03-21T10:35:28.106 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:28.102+0000 7f5b49800640 1 -- 172.21.15.47:0/1101816119 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5b44102910 msgr2=0x7f5b44197ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:35:28.106 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:28.102+0000 7f5b49800640 1 --2- 172.21.15.47:0/1101816119 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5b44102910 0x7f5b44197ec0 secure :-1 s=READY pgs=1311 cs=0 l=1 rev1=1 crypto rx=0x7f5b34005e30 tx=0x7f5b34003c20 comp rx=0 tx=0).stop 2024-03-21T10:35:28.106 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:28.103+0000 7f5b49800640 1 -- 172.21.15.47:0/1101816119 shutdown_connections 2024-03-21T10:35:28.106 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:28.103+0000 7f5b49800640 1 --2- 172.21.15.47:0/1101816119 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f5b280a3110 0x7f5b280a55d0 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:35:28.106 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:28.103+0000 7f5b49800640 1 --2- 172.21.15.47:0/1101816119 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5b4410bce0 0x7f5b4419c780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:35:28.107 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:28.103+0000 7f5b49800640 1 --2- 172.21.15.47:0/1101816119 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5b44103230 0x7f5b44198400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:35:28.107 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:28.103+0000 7f5b49800640 1 --2- 172.21.15.47:0/1101816119 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5b44102910 0x7f5b44197ec0 unknown :-1 s=CLOSED pgs=1311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:35:28.107 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:28.103+0000 7f5b49800640 1 -- 172.21.15.47:0/1101816119 >> 172.21.15.47:0/1101816119 conn(0x7f5b44077700 msgr2=0x7f5b440fd670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:35:28.107 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:28.103+0000 7f5b49800640 1 -- 172.21.15.47:0/1101816119 shutdown_connections 2024-03-21T10:35:28.107 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:28.103+0000 7f5b49800640 1 -- 172.21.15.47:0/1101816119 wait complete. 2024-03-21T10:35:28.118 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-21T10:35:28.118 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-03-21T10:35:28.314 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:28.314+0000 7f34bf000640 1 -- 172.21.15.47:0/3647003643 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f34b8075bd0 msgr2=0x7f34b8075fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:35:28.314 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:28.314+0000 7f34bf000640 1 --2- 172.21.15.47:0/3647003643 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f34b8075bd0 0x7f34b8075fb0 secure :-1 s=READY pgs=1316 cs=0 l=1 rev1=1 crypto rx=0x7f34a8009900 tx=0x7f34a802bc30 comp rx=0 tx=0).stop 2024-03-21T10:35:28.314 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:28.314+0000 7f34bf000640 1 -- 172.21.15.47:0/3647003643 shutdown_connections 2024-03-21T10:35:28.314 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:28.314+0000 7f34bf000640 1 --2- 172.21.15.47:0/3647003643 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f34b81124e0 0x7f34b81148d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:35:28.314 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:28.314+0000 7f34bf000640 1 --2- 172.21.15.47:0/3647003643 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f34b80764f0 0x7f34b8111fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:35:28.314 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:28.314+0000 7f34bf000640 1 --2- 172.21.15.47:0/3647003643 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f34b8075bd0 0x7f34b8075fb0 unknown :-1 s=CLOSED pgs=1316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:35:28.314 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:28.314+0000 7f34bf000640 1 -- 172.21.15.47:0/3647003643 >> 172.21.15.47:0/3647003643 conn(0x7f34b80ff310 msgr2=0x7f34b8101730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:35:28.314 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:28.314+0000 7f34bf000640 1 -- 172.21.15.47:0/3647003643 shutdown_connections 2024-03-21T10:35:28.314 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:28.314+0000 7f34bf000640 1 -- 172.21.15.47:0/3647003643 wait complete. 2024-03-21T10:35:28.317 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:28.315+0000 7f34bf000640 1 Processor -- start 2024-03-21T10:35:28.317 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:28.316+0000 7f34bf000640 1 -- start start 2024-03-21T10:35:28.317 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:28.316+0000 7f34bf000640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f34b8075bd0 0x7f34b819e4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:35:28.317 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:28.316+0000 7f34bf000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f34b80764f0 0x7f34b819e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:35:28.317 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:28.316+0000 7f34bf000640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f34b81124e0 0x7f34b81a2d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:35:28.317 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:28.316+0000 7f34bf000640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34b8117ef0 con 0x7f34b81124e0 2024-03-21T10:35:28.317 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:28.316+0000 7f34bf000640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34b8117d70 con 0x7f34b80764f0 2024-03-21T10:35:28.317 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:28.316+0000 7f34bf000640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34b8118070 con 0x7f34b8075bd0 2024-03-21T10:35:28.317 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:28.316+0000 7f34bd000640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f34b81124e0 0x7f34b81a2d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:35:28.317 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:28.316+0000 7f34bd000640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f34b81124e0 0x7f34b81a2d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:58944/0 (socket says 172.21.15.47:58944) 2024-03-21T10:35:28.317 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:28.316+0000 7f34bd000640 1 -- 172.21.15.47:0/3587655678 learned_addr learned my addr 172.21.15.47:0/3587655678 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:35:28.317 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:28.317+0000 7f34b7e00640 1 --2- 172.21.15.47:0/3587655678 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f34b8075bd0 0x7f34b819e4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:35:28.318 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:28.317+0000 7f34b7400640 1 --2- 172.21.15.47:0/3587655678 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f34b80764f0 0x7f34b819e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:35:28.318 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:28.317+0000 7f34bd000640 1 -- 172.21.15.47:0/3587655678 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f34b8075bd0 msgr2=0x7f34b819e4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:35:28.318 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:28.317+0000 7f34bd000640 1 --2- 172.21.15.47:0/3587655678 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f34b8075bd0 0x7f34b819e4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:35:28.318 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:28.317+0000 7f34bd000640 1 -- 172.21.15.47:0/3587655678 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f34b80764f0 msgr2=0x7f34b819e9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:35:28.318 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:28.317+0000 7f34bd000640 1 --2- 172.21.15.47:0/3587655678 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f34b80764f0 0x7f34b819e9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:35:28.318 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:28.317+0000 7f34bd000640 1 -- 172.21.15.47:0/3587655678 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f34b81a3450 con 0x7f34b81124e0 2024-03-21T10:35:28.318 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:28.317+0000 7f34b7e00640 1 --2- 172.21.15.47:0/3587655678 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f34b8075bd0 0x7f34b819e4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T10:35:28.318 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:28.317+0000 7f34b7400640 1 --2- 172.21.15.47:0/3587655678 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f34b80764f0 0x7f34b819e9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T10:35:28.318 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:28.317+0000 7f34bd000640 1 --2- 172.21.15.47:0/3587655678 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f34b81124e0 0x7f34b81a2d20 secure :-1 s=READY pgs=1317 cs=0 l=1 rev1=1 crypto rx=0x7f34b000eb40 tx=0x7f34b000c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:35:28.318 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:28.318+0000 7f34b4c00640 1 -- 172.21.15.47:0/3587655678 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34b0004b20 con 0x7f34b81124e0 2024-03-21T10:35:28.318 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:28.318+0000 7f34b4c00640 1 -- 172.21.15.47:0/3587655678 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f34b0004cc0 con 0x7f34b81124e0 2024-03-21T10:35:28.318 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:28.318+0000 7f34bf000640 1 -- 172.21.15.47:0/3587655678 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f34b81a3740 con 0x7f34b81124e0 2024-03-21T10:35:28.318 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:28.318+0000 7f34b4c00640 1 -- 172.21.15.47:0/3587655678 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34b00107b0 con 0x7f34b81124e0 2024-03-21T10:35:28.319 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:28.318+0000 7f34bf000640 1 -- 172.21.15.47:0/3587655678 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f34b81aafd0 con 0x7f34b81124e0 2024-03-21T10:35:28.320 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:28.321+0000 7f34b4c00640 1 -- 172.21.15.47:0/3587655678 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 27) v1 ==== 137985+0+0 (secure 0 0 0) 0x7f34b0005020 con 0x7f34b81124e0 2024-03-21T10:35:28.320 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:28.321+0000 7f34c1e2c740 1 Processor -- start 2024-03-21T10:35:28.320 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:28.321+0000 7f34c1e2c740 1 -- start start 2024-03-21T10:35:28.322 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:28.322+0000 7f34c1e2c740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ff4759d6d0 0x55ff4759dab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:35:28.322 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:28.322+0000 7f34c1e2c740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ff4759e080 0x55ff475a6630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:35:28.322 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:28.322+0000 7f34c1e2c740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ff475a6c00 0x55ff475a9050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:35:28.322 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:28.322+0000 7f34c1e2c740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ff475012c0 con 0x55ff4759d6d0 2024-03-21T10:35:28.322 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:28.322+0000 7f34c1e2c740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ff474a9c90 con 0x55ff475a6c00 2024-03-21T10:35:28.322 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:28.322+0000 7f34c1e2c740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ff474fd360 con 0x55ff4759e080 2024-03-21T10:35:28.322 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:28.322+0000 7f34b7400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ff4759d6d0 0x55ff4759dab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:35:28.322 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:28.322+0000 7f34b7400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ff4759d6d0 0x55ff4759dab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:58948/0 (socket says 172.21.15.47:58948) 2024-03-21T10:35:28.322 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:28.322+0000 7f34b7400640 1 -- 172.21.15.47:0/2563472106 learned_addr learned my addr 172.21.15.47:0/2563472106 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:35:28.322 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:28.322+0000 7f34b7e00640 1 --2- 172.21.15.47:0/2563472106 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ff4759e080 0x55ff475a6630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:35:28.322 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:28.322+0000 7f34b7400640 1 --2- 172.21.15.47:0/2563472106 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ff475a6c00 0x55ff475a9050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:35:28.322 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:28.323+0000 7f34b4c00640 1 --2- 172.21.15.47:0/3587655678 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f349c0a3090 0x7f349c0a5550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:35:28.323 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:28.323+0000 7f34b7400640 1 -- 172.21.15.47:0/2563472106 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ff4759e080 msgr2=0x55ff475a6630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:35:28.323 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:28.323+0000 7f34b7400640 1 --2- 172.21.15.47:0/2563472106 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ff4759e080 0x55ff475a6630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:35:28.323 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:28.323+0000 7f34b7400640 1 -- 172.21.15.47:0/2563472106 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ff475a6c00 msgr2=0x55ff475a9050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:35:28.323 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:28.323+0000 7f34b7400640 1 --2- 172.21.15.47:0/2563472106 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ff475a6c00 0x55ff475a9050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:35:28.323 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:28.323+0000 7f34b7400640 1 -- 172.21.15.47:0/2563472106 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ff474ce080 con 0x55ff4759d6d0 2024-03-21T10:35:28.323 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:28.323+0000 7f34b4c00640 1 -- 172.21.15.47:0/3587655678 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(105..105 src has 1..105) v4 ==== 9562+0+0 (secure 0 0 0) 0x7f34b00d04b0 con 0x7f34b81124e0 2024-03-21T10:35:28.323 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:28.323+0000 7f34bd000640 1 --2- 172.21.15.47:0/3587655678 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f349c0a3090 0x7f349c0a5550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:35:28.325 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:28.323+0000 7f34b7400640 1 --2- 172.21.15.47:0/2563472106 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ff4759d6d0 0x55ff4759dab0 secure :-1 s=READY pgs=1318 cs=0 l=1 rev1=1 crypto rx=0x7f34a4013610 tx=0x7f34a40097d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c94106e7f35815fd server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:35:28.325 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:28.324+0000 7f34ad600640 1 -- 172.21.15.47:0/2563472106 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34a400f520 con 0x55ff4759d6d0 2024-03-21T10:35:28.325 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:28.324+0000 7f34ad600640 1 -- 172.21.15.47:0/2563472106 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f34a400fb70 con 0x55ff4759d6d0 2024-03-21T10:35:28.325 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:28.324+0000 7f34ad600640 1 -- 172.21.15.47:0/2563472106 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34a40119c0 con 0x55ff4759d6d0 2024-03-21T10:35:28.326 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:28.324+0000 7f34c1e2c740 1 -- 172.21.15.47:0/2563472106 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ff4759d6d0 msgr2=0x55ff4759dab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:35:28.326 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:28.324+0000 7f34bd000640 1 --2- 172.21.15.47:0/3587655678 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f349c0a3090 0x7f349c0a5550 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f34b00a4910 tx=0x7f34b0098040 comp rx=0 tx=0).ready entity=mgr.15687 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:35:28.326 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:28.324+0000 7f34c1e2c740 1 --2- 172.21.15.47:0/2563472106 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ff4759d6d0 0x55ff4759dab0 secure :-1 s=READY pgs=1318 cs=0 l=1 rev1=1 crypto rx=0x7f34a4013610 tx=0x7f34a40097d0 comp rx=0 tx=0).stop 2024-03-21T10:35:28.326 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:28.324+0000 7f34c1e2c740 1 -- 172.21.15.47:0/2563472106 shutdown_connections 2024-03-21T10:35:28.326 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:28.324+0000 7f34c1e2c740 1 --2- 172.21.15.47:0/2563472106 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ff475a6c00 0x55ff475a9050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:35:28.326 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:28.324+0000 7f34c1e2c740 1 --2- 172.21.15.47:0/2563472106 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ff4759e080 0x55ff475a6630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:35:28.326 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:28.324+0000 7f34c1e2c740 1 --2- 172.21.15.47:0/2563472106 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ff4759d6d0 0x55ff4759dab0 unknown :-1 s=CLOSED pgs=1318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:35:28.326 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:28.324+0000 7f34c1e2c740 1 -- 172.21.15.47:0/2563472106 >> 172.21.15.47:0/2563472106 conn(0x55ff47597da0 msgr2=0x55ff47598180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:35:28.326 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:28.325+0000 7f34c1e2c740 1 -- 172.21.15.47:0/2563472106 shutdown_connections 2024-03-21T10:35:28.326 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:28.325+0000 7f34c1e2c740 1 -- 172.21.15.47:0/2563472106 wait complete. 2024-03-21T10:35:28.326 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:28.325+0000 7f34c1e2c740 1 -- start start 2024-03-21T10:35:28.326 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:28.325+0000 7f34c1e2c740 1 Processor -- start 2024-03-21T10:35:28.326 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:28.326+0000 7f34c1e2c740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ff4759d6d0 0x55ff475a6580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:35:28.327 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:28.326+0000 7f34c1e2c740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ff4759e080 0x55ff475a3910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:35:28.327 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:28.326+0000 7f34c1e2c740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ff475a6c00 0x55ff475a3f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:35:28.327 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:28.326+0000 7f34c1e2c740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ff475a9b70 con 0x55ff4759e080 2024-03-21T10:35:28.327 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:28.326+0000 7f34c1e2c740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ff474bd0d0 con 0x55ff4759d6d0 2024-03-21T10:35:28.327 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:28.326+0000 7f34c1e2c740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ff475a9cf0 con 0x55ff475a6c00 2024-03-21T10:35:28.327 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:28.326+0000 7f34c1e2c740 10 client.0 _ll_register_callbacks cb 0x55ff474c1e80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T10:35:28.327 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:28.326+0000 7f34b7e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ff4759e080 0x55ff475a3910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:35:28.327 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:28.326+0000 7f34b7400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ff4759d6d0 0x55ff475a6580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:35:28.327 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:28.326+0000 7f34b7e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ff4759e080 0x55ff475a3910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:58962/0 (socket says 172.21.15.47:58962) 2024-03-21T10:35:28.327 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:28.326+0000 7f34b7400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ff4759d6d0 0x55ff475a6580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:60114/0 (socket says 172.21.15.47:60114) 2024-03-21T10:35:28.327 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:28.326+0000 7f34b7e00640 1 -- 172.21.15.47:0/125581492 learned_addr learned my addr 172.21.15.47:0/125581492 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:35:28.327 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:28.326+0000 7f34b7400640 1 --2- 172.21.15.47:0/125581492 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ff475a6c00 0x55ff475a3f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:35:28.328 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:28.327+0000 7f34b7e00640 1 -- 172.21.15.47:0/125581492 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ff475a6c00 msgr2=0x55ff475a3f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:35:28.328 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:28.327+0000 7f34b7e00640 1 --2- 172.21.15.47:0/125581492 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ff475a6c00 0x55ff475a3f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:35:28.328 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:28.327+0000 7f34b7e00640 1 -- 172.21.15.47:0/125581492 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ff4759d6d0 msgr2=0x55ff475a6580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:35:28.328 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:28.327+0000 7f34b7e00640 1 --2- 172.21.15.47:0/125581492 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ff4759d6d0 0x55ff475a6580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:35:28.328 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:28.327+0000 7f34b7e00640 1 -- 172.21.15.47:0/125581492 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55ff472153f0 con 0x55ff4759e080 2024-03-21T10:35:28.328 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:28.328+0000 7f34b7e00640 1 --2- 172.21.15.47:0/125581492 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ff4759e080 0x55ff475a3910 secure :-1 s=READY pgs=1319 cs=0 l=1 rev1=1 crypto rx=0x7f34a802c170 tx=0x7f34a8004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=283b6378851216e6 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:35:28.328 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:28.328+0000 7f34a2000640 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-03-21T10:35:28.329 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:28.328+0000 7f34a2000640 1 -- 172.21.15.47:0/125581492 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34a8005d50 con 0x55ff4759e080 2024-03-21T10:35:28.329 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:28.328+0000 7f34c1e2c740 1 -- 172.21.15.47:0/125581492 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f34a400fb70 con 0x55ff4759e080 2024-03-21T10:35:28.329 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:28.328+0000 7f34a2000640 1 -- 172.21.15.47:0/125581492 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(105..105 src has 1..105) v4 ==== 9562+0+0 (secure 0 0 0) 0x7f34a8005230 con 0x55ff4759e080 2024-03-21T10:35:28.331 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:28.332+0000 7f34a2000640 1 -- 172.21.15.47:0/125581492 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f34a8005430 con 0x55ff4759e080 2024-03-21T10:35:28.331 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:28.332+0000 7f34c1e2c740 10 client.18828 fetch_fsmap learned FSMap version 32 2024-03-21T10:35:28.331 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:28.332+0000 7f34c1e2c740 1 -- 172.21.15.47:0/125581492 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x55ff472df7d0 con 0x55ff4759e080 2024-03-21T10:35:28.332 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:28.333+0000 7f34a2000640 1 -- 172.21.15.47:0/125581492 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f34a400fb70 con 0x55ff4759e080 2024-03-21T10:35:28.332 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:28.333+0000 7f34a2000640 1 -- 172.21.15.47:0/125581492 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 32) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f34a8003680 con 0x55ff4759e080 2024-03-21T10:35:28.332 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:28.333+0000 7f34c1e2c740 10 client.18828 fetch_fsmap finished waiting for FSMap version 32 2024-03-21T10:35:28.332 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:28.333+0000 7f34c1e2c740 10 client.18828 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-21T10:35:28.332 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:28.333+0000 7f34c1e2c740 20 client.18828 populate_metadata read hostname 'smithi047' 2024-03-21T10:35:28.332 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:28.333+0000 7f34c1e2c740 1 --2- 172.21.15.47:0/125581492 >> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] conn(0x55ff475b6990 0x55ff475b8da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:35:28.333 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:28.333+0000 7f34c1e2c740 4 client.18828 mds_command: new command op to 34535 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T10:35:28.333 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:28.333+0000 7f34c1e2c740 1 -- 172.21.15.47:0/125581492 --> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55ff4750e800 con 0x55ff475b6990 2024-03-21T10:35:28.333 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:28.333+0000 7f34b7400640 1 --2- 172.21.15.47:0/125581492 >> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] conn(0x55ff475b6990 0x55ff475b8da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:35:28.333 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:28.334+0000 7f34b7400640 1 --2- 172.21.15.47:0/125581492 >> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] conn(0x55ff475b6990 0x55ff475b8da0 crc :-1 s=READY pgs=223 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=55f75f6eef58c1dd server_cookie=ff03ea1db7f2de64 in_seq=0 out_seq=0 2024-03-21T10:35:28.333 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:28.334+0000 7f34a2000640 10 client.18828 ms_handle_connect on v2:172.21.15.100:6838/4173168440 2024-03-21T10:35:28.335 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:28.336+0000 7f34a2000640 1 -- 172.21.15.47:0/125581492 <== mds.0 v2:172.21.15.100:6838/4173168440 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55ff4750e800 con 0x55ff475b6990 2024-03-21T10:35:28.335 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:28.336+0000 7f34a2000640 10 client.18828 handle_command_reply: tid=0 2024-03-21T10:35:28.335 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:28.336+0000 7f34c1e2c740 1 client.18828 shutdown 2024-03-21T10:35:28.335 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:28.336+0000 7f34c1e2c740 1 -- 172.21.15.47:0/125581492 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ff4759e080 msgr2=0x55ff475a3910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:35:28.335 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:28.336+0000 7f34c1e2c740 1 --2- 172.21.15.47:0/125581492 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ff4759e080 0x55ff475a3910 secure :-1 s=READY pgs=1319 cs=0 l=1 rev1=1 crypto rx=0x7f34a802c170 tx=0x7f34a8004270 comp rx=0 tx=0).stop 2024-03-21T10:35:28.335 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:28.336+0000 7f34c1e2c740 1 -- 172.21.15.47:0/125581492 shutdown_connections 2024-03-21T10:35:28.335 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:28.336+0000 7f34c1e2c740 1 --2- 172.21.15.47:0/125581492 >> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] conn(0x55ff475b6990 0x55ff475b8da0 crc :-1 s=READY pgs=223 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:35:28.335 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:28.336+0000 7f34c1e2c740 1 --2- 172.21.15.47:0/125581492 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ff475a6c00 0x55ff475a3f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:35:28.336 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:28.336+0000 7f34c1e2c740 1 --2- 172.21.15.47:0/125581492 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ff4759e080 0x55ff475a3910 unknown :-1 s=CLOSED pgs=1319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:35:28.336 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:28.336+0000 7f34c1e2c740 1 --2- 172.21.15.47:0/125581492 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ff4759d6d0 0x55ff475a6580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:35:28.336 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:28.336+0000 7f34a2000640 0 client.18828 ms_handle_reset on v2:172.21.15.100:6838/4173168440 2024-03-21T10:35:28.336 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:28.336+0000 7f34c1e2c740 1 -- 172.21.15.47:0/125581492 >> 172.21.15.47:0/125581492 conn(0x55ff47597da0 msgr2=0x55ff47599af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:35:28.336 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:28.336+0000 7f34c1e2c740 1 -- 172.21.15.47:0/125581492 shutdown_connections 2024-03-21T10:35:28.336 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:28.336+0000 7f34c1e2c740 1 -- 172.21.15.47:0/125581492 wait complete. 2024-03-21T10:35:28.336 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:28.336+0000 7f34c1e2c740 20 client.18828 trim_cache size 0 max 16384 2024-03-21T10:35:28.349 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:28.350+0000 7f34c1e2c740 1 Processor -- start 2024-03-21T10:35:28.349 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:28.350+0000 7f34c1e2c740 1 -- start start 2024-03-21T10:35:28.351 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:28.350+0000 7f34c1e2c740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ff4759d6d0 0x55ff47599cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:35:28.351 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:28.351+0000 7f34c1e2c740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ff4759e080 0x55ff475a6040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:35:28.351 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:28.351+0000 7f34c1e2c740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ff475a6c00 0x55ff475a6580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:35:28.351 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:28.351+0000 7f34c1e2c740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34b81b4fe0 con 0x55ff4759d6d0 2024-03-21T10:35:28.351 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:28.351+0000 7f34c1e2c740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34b81b6960 con 0x55ff475a6c00 2024-03-21T10:35:28.351 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:28.351+0000 7f34c1e2c740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34b81b5640 con 0x55ff4759e080 2024-03-21T10:35:28.351 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:28.351+0000 7f34b7400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ff4759d6d0 0x55ff47599cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:35:28.351 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:28.351+0000 7f34b7400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ff4759d6d0 0x55ff47599cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:58978/0 (socket says 172.21.15.47:58978) 2024-03-21T10:35:28.351 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:28.351+0000 7f34b7400640 1 -- 172.21.15.47:0/409656250 learned_addr learned my addr 172.21.15.47:0/409656250 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:35:28.351 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:28.351+0000 7f34b7400640 1 --2- 172.21.15.47:0/409656250 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ff475a6c00 0x55ff475a6580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:35:28.351 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:28.351+0000 7f34b7e00640 1 --2- 172.21.15.47:0/409656250 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ff4759e080 0x55ff475a6040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:35:28.352 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:28.351+0000 7f34b7400640 1 -- 172.21.15.47:0/409656250 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ff4759e080 msgr2=0x55ff475a6040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:35:28.352 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:28.351+0000 7f34b7400640 1 --2- 172.21.15.47:0/409656250 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ff4759e080 0x55ff475a6040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:35:28.352 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:28.351+0000 7f34b7400640 1 -- 172.21.15.47:0/409656250 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ff475a6c00 msgr2=0x55ff475a6580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:35:28.352 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:28.351+0000 7f34b7400640 1 --2- 172.21.15.47:0/409656250 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ff475a6c00 0x55ff475a6580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:35:28.352 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:28.351+0000 7f34b7400640 1 -- 172.21.15.47:0/409656250 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ff4759d180 con 0x55ff4759d6d0 2024-03-21T10:35:28.352 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:28.352+0000 7f34b7400640 1 --2- 172.21.15.47:0/409656250 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ff4759d6d0 0x55ff47599cc0 secure :-1 s=READY pgs=1320 cs=0 l=1 rev1=1 crypto rx=0x7f34a4014fd0 tx=0x7f34a4013400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e3cb284ace5b80e server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:35:28.352 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:28.352+0000 7f34acc00640 1 -- 172.21.15.47:0/409656250 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34a4013070 con 0x55ff4759d6d0 2024-03-21T10:35:28.352 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:28.352+0000 7f34acc00640 1 -- 172.21.15.47:0/409656250 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f34a4013210 con 0x55ff4759d6d0 2024-03-21T10:35:28.352 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:28.352+0000 7f34acc00640 1 -- 172.21.15.47:0/409656250 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34a400f780 con 0x55ff4759d6d0 2024-03-21T10:35:28.355 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:28.353+0000 7f34c1e2c740 1 -- 172.21.15.47:0/409656250 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ff4759d6d0 msgr2=0x55ff47599cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:35:28.355 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:28.353+0000 7f34c1e2c740 1 --2- 172.21.15.47:0/409656250 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ff4759d6d0 0x55ff47599cc0 secure :-1 s=READY pgs=1320 cs=0 l=1 rev1=1 crypto rx=0x7f34a4014fd0 tx=0x7f34a4013400 comp rx=0 tx=0).stop 2024-03-21T10:35:28.355 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:28.353+0000 7f34c1e2c740 1 -- 172.21.15.47:0/409656250 shutdown_connections 2024-03-21T10:35:28.355 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:28.353+0000 7f34c1e2c740 1 --2- 172.21.15.47:0/409656250 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ff475a6c00 0x55ff475a6580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:35:28.356 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:28.353+0000 7f34c1e2c740 1 --2- 172.21.15.47:0/409656250 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ff4759e080 0x55ff475a6040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:35:28.356 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:28.353+0000 7f34c1e2c740 1 --2- 172.21.15.47:0/409656250 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ff4759d6d0 0x55ff47599cc0 unknown :-1 s=CLOSED pgs=1320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:35:28.356 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:28.353+0000 7f34c1e2c740 1 -- 172.21.15.47:0/409656250 >> 172.21.15.47:0/409656250 conn(0x55ff47597da0 msgr2=0x55ff47596eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:35:28.356 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:28.353+0000 7f34c1e2c740 1 -- 172.21.15.47:0/409656250 shutdown_connections 2024-03-21T10:35:28.356 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:28.353+0000 7f34c1e2c740 1 -- 172.21.15.47:0/409656250 wait complete. 2024-03-21T10:35:28.356 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:28.353+0000 7f34c1e2c740 1 -- start start 2024-03-21T10:35:28.356 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:28.353+0000 7f34c1e2c740 1 Processor -- start 2024-03-21T10:35:28.356 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:28.354+0000 7f34c1e2c740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ff4759d6d0 0x55ff475aeb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:35:28.356 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:28.354+0000 7f34c1e2c740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ff4759e080 0x55ff475acef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:35:28.356 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:28.354+0000 7f34c1e2c740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ff475a6c00 0x55ff475ad430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:35:28.356 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:28.354+0000 7f34c1e2c740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34b81b6300 con 0x55ff4759d6d0 2024-03-21T10:35:28.356 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:28.354+0000 7f34c1e2c740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34b81b5ca0 con 0x55ff475a6c00 2024-03-21T10:35:28.356 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:28.354+0000 7f34c1e2c740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34b81b31c0 con 0x55ff4759e080 2024-03-21T10:35:28.356 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:28.354+0000 7f34c1e2c740 10 client.0 _ll_register_callbacks cb 0x7f34a4005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T10:35:28.356 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:28.354+0000 7f34b7400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ff4759d6d0 0x55ff475aeb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:35:28.357 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:28.354+0000 7f34b7400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ff4759d6d0 0x55ff475aeb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:58986/0 (socket says 172.21.15.47:58986) 2024-03-21T10:35:28.357 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:28.354+0000 7f34b7400640 1 -- 172.21.15.47:0/4160304526 learned_addr learned my addr 172.21.15.47:0/4160304526 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:35:28.357 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:28.354+0000 7f34b7400640 1 --2- 172.21.15.47:0/4160304526 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ff475a6c00 0x55ff475ad430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:35:28.357 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:28.355+0000 7f34b7e00640 1 --2- 172.21.15.47:0/4160304526 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ff4759e080 0x55ff475acef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:35:28.357 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:28.355+0000 7f34b7400640 1 -- 172.21.15.47:0/4160304526 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ff4759e080 msgr2=0x55ff475acef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:35:28.357 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:28.355+0000 7f34b7400640 1 --2- 172.21.15.47:0/4160304526 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ff4759e080 0x55ff475acef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:35:28.357 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:28.355+0000 7f34b7400640 1 -- 172.21.15.47:0/4160304526 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ff475a6c00 msgr2=0x55ff475ad430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T10:35:28.357 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:28.355+0000 7f34b7400640 1 --2- 172.21.15.47:0/4160304526 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ff475a6c00 0x55ff475ad430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:35:28.357 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:28.355+0000 7f34b7400640 1 -- 172.21.15.47:0/4160304526 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55ff475adb20 con 0x55ff4759d6d0 2024-03-21T10:35:28.357 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:28.355+0000 7f34b7e00640 1 --2- 172.21.15.47:0/4160304526 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ff4759e080 0x55ff475acef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T10:35:28.357 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:28.355+0000 7f34b7400640 1 --2- 172.21.15.47:0/4160304526 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ff4759d6d0 0x55ff475aeb50 secure :-1 s=READY pgs=1321 cs=0 l=1 rev1=1 crypto rx=0x7f34a40303b0 tx=0x7f34a40302b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2b230e99e89bb610 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:35:28.357 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:28.355+0000 7f34a2000640 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-03-21T10:35:28.357 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:28.355+0000 7f34a2000640 1 -- 172.21.15.47:0/4160304526 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34a403b060 con 0x55ff4759d6d0 2024-03-21T10:35:28.357 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:28.355+0000 7f34a2000640 1 -- 172.21.15.47:0/4160304526 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(105..105 src has 1..105) v4 ==== 9562+0+0 (secure 0 0 0) 0x7f34a4005240 con 0x55ff4759d6d0 2024-03-21T10:35:28.357 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:28.355+0000 7f34c1e2c740 1 -- 172.21.15.47:0/4160304526 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f34a4013210 con 0x55ff4759d6d0 2024-03-21T10:35:28.357 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:28.356+0000 7f34a2000640 1 -- 172.21.15.47:0/4160304526 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f34a4042050 con 0x55ff4759d6d0 2024-03-21T10:35:28.358 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:28.356+0000 7f34c1e2c740 10 client.18834 fetch_fsmap learned FSMap version 32 2024-03-21T10:35:28.358 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:28.356+0000 7f34c1e2c740 1 -- 172.21.15.47:0/4160304526 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x55ff47613130 con 0x55ff4759d6d0 2024-03-21T10:35:28.358 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:28.356+0000 7f34a2000640 1 -- 172.21.15.47:0/4160304526 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f34a4013210 con 0x55ff4759d6d0 2024-03-21T10:35:28.358 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:28.356+0000 7f34a2000640 1 -- 172.21.15.47:0/4160304526 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 32) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f34a4030990 con 0x55ff4759d6d0 2024-03-21T10:35:28.358 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:28.356+0000 7f34c1e2c740 10 client.18834 fetch_fsmap finished waiting for FSMap version 32 2024-03-21T10:35:28.358 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:28.356+0000 7f34c1e2c740 10 client.18834 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-21T10:35:28.358 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:28.356+0000 7f34c1e2c740 20 client.18834 populate_metadata read hostname 'smithi047' 2024-03-21T10:35:28.358 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:28.356+0000 7f34c1e2c740 1 --2- 172.21.15.47:0/4160304526 >> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] conn(0x55ff47613500 0x55ff476158f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:35:28.358 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:28.356+0000 7f34c1e2c740 4 client.18834 mds_command: new command op to 34535 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-21T10:35:28.358 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:28.356+0000 7f34c1e2c740 1 -- 172.21.15.47:0/4160304526 --> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55ff47508b50 con 0x55ff47613500 2024-03-21T10:35:28.359 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:28.357+0000 7f34b7e00640 1 --2- 172.21.15.47:0/4160304526 >> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] conn(0x55ff47613500 0x55ff476158f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:35:28.359 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:28.357+0000 7f34b7e00640 1 --2- 172.21.15.47:0/4160304526 >> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] conn(0x55ff47613500 0x55ff476158f0 crc :-1 s=READY pgs=224 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4b6db9d273cfc251 server_cookie=3bb2b1a8329154be in_seq=0 out_seq=0 2024-03-21T10:35:28.360 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:28.357+0000 7f34a2000640 10 client.18834 ms_handle_connect on v2:172.21.15.100:6838/4173168440 2024-03-21T10:35:28.360 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:28.358+0000 7f34a2000640 1 -- 172.21.15.47:0/4160304526 <== mds.0 v2:172.21.15.100:6838/4173168440 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55ff47508b50 con 0x55ff47613500 2024-03-21T10:35:28.360 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:28.358+0000 7f34a2000640 10 client.18834 handle_command_reply: tid=0 2024-03-21T10:35:28.360 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:28.358+0000 7f34c1e2c740 1 client.18834 shutdown 2024-03-21T10:35:28.360 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:28.358+0000 7f34c1e2c740 1 -- 172.21.15.47:0/4160304526 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ff4759d6d0 msgr2=0x55ff475aeb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:35:28.360 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:28.358+0000 7f34c1e2c740 1 --2- 172.21.15.47:0/4160304526 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ff4759d6d0 0x55ff475aeb50 secure :-1 s=READY pgs=1321 cs=0 l=1 rev1=1 crypto rx=0x7f34a40303b0 tx=0x7f34a40302b0 comp rx=0 tx=0).stop 2024-03-21T10:35:28.360 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:28.358+0000 7f34c1e2c740 1 -- 172.21.15.47:0/4160304526 shutdown_connections 2024-03-21T10:35:28.360 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:28.358+0000 7f34c1e2c740 1 --2- 172.21.15.47:0/4160304526 >> [v2:172.21.15.100:6838/4173168440,v1:172.21.15.100:6839/4173168440] conn(0x55ff47613500 0x55ff476158f0 crc :-1 s=READY pgs=224 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:35:28.360 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:28.358+0000 7f34c1e2c740 1 --2- 172.21.15.47:0/4160304526 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ff475a6c00 0x55ff475ad430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:35:28.361 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:28.358+0000 7f34c1e2c740 1 --2- 172.21.15.47:0/4160304526 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55ff4759e080 0x55ff475acef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:35:28.361 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:28.358+0000 7f34a2000640 0 client.18834 ms_handle_reset on v2:172.21.15.100:6838/4173168440 2024-03-21T10:35:28.361 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:28.358+0000 7f34c1e2c740 1 --2- 172.21.15.47:0/4160304526 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ff4759d6d0 0x55ff475aeb50 unknown :-1 s=CLOSED pgs=1321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:35:28.361 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:28.358+0000 7f34c1e2c740 1 -- 172.21.15.47:0/4160304526 >> 172.21.15.47:0/4160304526 conn(0x55ff47597da0 msgr2=0x55ff4759e460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:35:28.361 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:28.358+0000 7f34c1e2c740 1 -- 172.21.15.47:0/4160304526 shutdown_connections 2024-03-21T10:35:28.361 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:28.358+0000 7f34c1e2c740 1 -- 172.21.15.47:0/4160304526 wait complete. 2024-03-21T10:35:28.361 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:28.359+0000 7f34c1e2c740 20 client.18834 trim_cache size 0 max 16384 2024-03-21T10:35:28.361 INFO:teuthology.orchestra.run.smithi047.stdout:[] 2024-03-21T10:35:28.362 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:28.359+0000 7f34bf000640 1 -- 172.21.15.47:0/3587655678 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f349c0a3090 msgr2=0x7f349c0a5550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:35:28.362 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:28.359+0000 7f34bf000640 1 --2- 172.21.15.47:0/3587655678 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f349c0a3090 0x7f349c0a5550 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f34b00a4910 tx=0x7f34b0098040 comp rx=0 tx=0).stop 2024-03-21T10:35:28.363 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:28.359+0000 7f34bf000640 1 -- 172.21.15.47:0/3587655678 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f34b81124e0 msgr2=0x7f34b81a2d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:35:28.363 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:28.359+0000 7f34bf000640 1 --2- 172.21.15.47:0/3587655678 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f34b81124e0 0x7f34b81a2d20 secure :-1 s=READY pgs=1317 cs=0 l=1 rev1=1 crypto rx=0x7f34b000eb40 tx=0x7f34b000c2d0 comp rx=0 tx=0).stop 2024-03-21T10:35:28.363 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:28.359+0000 7f34bf000640 1 -- 172.21.15.47:0/3587655678 shutdown_connections 2024-03-21T10:35:28.363 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:28.359+0000 7f34bf000640 1 --2- 172.21.15.47:0/3587655678 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f34b81124e0 0x7f34b81a2d20 unknown :-1 s=CLOSED pgs=1317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:35:28.363 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:28.359+0000 7f34bf000640 1 --2- 172.21.15.47:0/3587655678 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f349c0a3090 0x7f349c0a5550 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:35:28.363 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:28.359+0000 7f34bf000640 1 --2- 172.21.15.47:0/3587655678 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f34b80764f0 0x7f34b819e9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:35:28.363 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:28.359+0000 7f34bf000640 1 --2- 172.21.15.47:0/3587655678 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f34b8075bd0 0x7f34b819e4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:35:28.363 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:28.359+0000 7f34bf000640 1 -- 172.21.15.47:0/3587655678 >> 172.21.15.47:0/3587655678 conn(0x7f34b80ff310 msgr2=0x7f34b8113ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:35:28.363 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:28.360+0000 7f34bf000640 1 -- 172.21.15.47:0/3587655678 shutdown_connections 2024-03-21T10:35:28.363 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:28.360+0000 7f34bf000640 1 -- 172.21.15.47:0/3587655678 wait complete. 2024-03-21T10:35:29.380 INFO:tasks.fwd_scrub.fs.[cephfs]:end scrubbing fs: cephfs 2024-03-21T10:35:29.380 INFO:tasks.fwd_scrub:done joining 2024-03-21T10:35:29.380 DEBUG:teuthology.run_tasks:Unwinding manager check-counter 2024-03-21T10:35:29.392 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-21T10:35:29.647 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:29.648+0000 7efe31a00640 1 -- 172.21.15.47:0/1335977367 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7efe2c104030 msgr2=0x7efe2c110540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:35:29.648 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:29.648+0000 7efe31a00640 1 --2- 172.21.15.47:0/1335977367 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7efe2c104030 0x7efe2c110540 secure :-1 s=READY pgs=1322 cs=0 l=1 rev1=1 crypto rx=0x7efe24009cd0 tx=0x7efe2402c0f0 comp rx=0 tx=0).stop 2024-03-21T10:35:29.648 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:29.648+0000 7efe31a00640 1 -- 172.21.15.47:0/1335977367 shutdown_connections 2024-03-21T10:35:29.648 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:29.648+0000 7efe31a00640 1 --2- 172.21.15.47:0/1335977367 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7efe2c104030 0x7efe2c110540 unknown :-1 s=CLOSED pgs=1322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:35:29.648 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:29.648+0000 7efe31a00640 1 --2- 172.21.15.47:0/1335977367 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7efe2c103670 0x7efe2c103af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:35:29.648 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:29.648+0000 7efe31a00640 1 --2- 172.21.15.47:0/1335977367 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7efe2c109680 0x7efe2c109a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:35:29.648 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:29.648+0000 7efe31a00640 1 -- 172.21.15.47:0/1335977367 >> 172.21.15.47:0/1335977367 conn(0x7efe2c0ff140 msgr2=0x7efe2c101560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:35:29.648 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:29.648+0000 7efe31a00640 1 -- 172.21.15.47:0/1335977367 shutdown_connections 2024-03-21T10:35:29.648 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:29.648+0000 7efe31a00640 1 -- 172.21.15.47:0/1335977367 wait complete. 2024-03-21T10:35:29.650 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:29.649+0000 7efe31a00640 1 Processor -- start 2024-03-21T10:35:29.650 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:29.649+0000 7efe31a00640 1 -- start start 2024-03-21T10:35:29.650 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:29.650+0000 7efe31a00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7efe2c103670 0x7efe2c199e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:35:29.650 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:29.650+0000 7efe31a00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7efe2c104030 0x7efe2c19a360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:35:29.650 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:29.650+0000 7efe31a00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7efe2c109680 0x7efe2c19e6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:35:29.650 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:29.650+0000 7efe31a00640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe2c113900 con 0x7efe2c109680 2024-03-21T10:35:29.650 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:29.650+0000 7efe31a00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe2c113780 con 0x7efe2c103670 2024-03-21T10:35:29.650 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:29.650+0000 7efe31a00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe2c113a80 con 0x7efe2c104030 2024-03-21T10:35:29.651 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:29.650+0000 7efe31000640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7efe2c109680 0x7efe2c19e6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:35:29.651 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:29.650+0000 7efe2be00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7efe2c103670 0x7efe2c199e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:35:29.651 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:29.650+0000 7efe31000640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7efe2c109680 0x7efe2c19e6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:58996/0 (socket says 172.21.15.47:58996) 2024-03-21T10:35:29.651 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:29.650+0000 7efe31000640 1 -- 172.21.15.47:0/555838864 learned_addr learned my addr 172.21.15.47:0/555838864 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:35:29.651 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:29.650+0000 7efe2b400640 1 --2- 172.21.15.47:0/555838864 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7efe2c104030 0x7efe2c19a360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:35:29.651 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:29.651+0000 7efe31000640 1 -- 172.21.15.47:0/555838864 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7efe2c104030 msgr2=0x7efe2c19a360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:35:29.651 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:29.651+0000 7efe31000640 1 --2- 172.21.15.47:0/555838864 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7efe2c104030 0x7efe2c19a360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:35:29.651 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:29.651+0000 7efe31000640 1 -- 172.21.15.47:0/555838864 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7efe2c103670 msgr2=0x7efe2c199e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:35:29.651 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:29.651+0000 7efe31000640 1 --2- 172.21.15.47:0/555838864 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7efe2c103670 0x7efe2c199e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:35:29.651 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:29.651+0000 7efe31000640 1 -- 172.21.15.47:0/555838864 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe2c19ee10 con 0x7efe2c109680 2024-03-21T10:35:29.651 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:29.651+0000 7efe2b400640 1 --2- 172.21.15.47:0/555838864 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7efe2c104030 0x7efe2c19a360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T10:35:29.651 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:29.651+0000 7efe31000640 1 --2- 172.21.15.47:0/555838864 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7efe2c109680 0x7efe2c19e6e0 secure :-1 s=READY pgs=1323 cs=0 l=1 rev1=1 crypto rx=0x7efe24005e30 tx=0x7efe24035ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:35:29.652 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:29.652+0000 7efe28c00640 1 -- 172.21.15.47:0/555838864 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe24039040 con 0x7efe2c109680 2024-03-21T10:35:29.652 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:29.652+0000 7efe31a00640 1 -- 172.21.15.47:0/555838864 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efe2c19f0a0 con 0x7efe2c109680 2024-03-21T10:35:29.652 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:29.652+0000 7efe28c00640 1 -- 172.21.15.47:0/555838864 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7efe24044070 con 0x7efe2c109680 2024-03-21T10:35:29.652 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:29.652+0000 7efe28c00640 1 -- 172.21.15.47:0/555838864 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe240042a0 con 0x7efe2c109680 2024-03-21T10:35:29.652 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:29.652+0000 7efe31a00640 1 -- 172.21.15.47:0/555838864 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efe2c1a6990 con 0x7efe2c109680 2024-03-21T10:35:29.652 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:29.653+0000 7efe31a00640 1 -- 172.21.15.47:0/555838864 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efe2c068800 con 0x7efe2c109680 2024-03-21T10:35:29.654 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:29.655+0000 7efe28c00640 1 -- 172.21.15.47:0/555838864 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 27) v1 ==== 137985+0+0 (secure 0 0 0) 0x7efe24004440 con 0x7efe2c109680 2024-03-21T10:35:29.655 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:29.656+0000 7efe28c00640 1 --2- 172.21.15.47:0/555838864 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7efe100a3090 0x7efe100a5550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:35:29.655 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:29.656+0000 7efe28c00640 1 -- 172.21.15.47:0/555838864 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(105..105 src has 1..105) v4 ==== 9562+0+0 (secure 0 0 0) 0x7efe240f2090 con 0x7efe2c109680 2024-03-21T10:35:29.655 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:29.657+0000 7efe2be00640 1 --2- 172.21.15.47:0/555838864 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7efe100a3090 0x7efe100a5550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:35:29.656 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:29.657+0000 7efe2be00640 1 --2- 172.21.15.47:0/555838864 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7efe100a3090 0x7efe100a5550 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7efe1c005d00 tx=0x7efe1c00b410 comp rx=0 tx=0).ready entity=mgr.15687 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:35:29.664 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:29.665+0000 7efe28c00640 1 -- 172.21.15.47:0/555838864 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7efe240be820 con 0x7efe2c109680 2024-03-21T10:35:29.710 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:35:29 smithi047 ceph-mon[31547]: pgmap v316: 129 pgs: 129 active+clean; 2.5 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 0 B/s wr, 11 op/s 2024-03-21T10:35:29.726 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:35:29 smithi100 ceph-mon[35193]: pgmap v316: 129 pgs: 129 active+clean; 2.5 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 0 B/s wr, 11 op/s 2024-03-21T10:35:29.736 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:35:29 smithi157 ceph-mon[35581]: pgmap v316: 129 pgs: 129 active+clean; 2.5 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 0 B/s wr, 11 op/s 2024-03-21T10:35:29.868 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:29.869+0000 7efe31a00640 1 -- 172.21.15.47:0/555838864 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7efe2c10bc00 con 0x7efe2c109680 2024-03-21T10:35:29.873 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:29.875+0000 7efe28c00640 1 -- 172.21.15.47:0/555838864 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v105) v1 ==== 74+0+25412 (secure 0 0 0) 0x7efe240be120 con 0x7efe2c109680 2024-03-21T10:35:29.874 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-21T10:35:29.874 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":105,"fsid":"3056d51a-e76b-11ee-95cd-87774f69a715","created":"2024-03-21T10:11:17.019965+0000","modified":"2024-03-21T10:35:13.861435+0000","last_up_change":"2024-03-21T10:20:52.675328+0000","last_in_change":"2024-03-21T10:20:27.648212+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-21T10:15:40.553114+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-21T10:21:56.399518+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-21T10:21:57.812940+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"104","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":15,"snap_epoch":104,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-21T10:22:01.822568+0000","flags":8197,"flags_names":"hashpspool,ec_overwrites,selfmanaged_snaps","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"104","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":15,"snap_epoch":104,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"b0c7a102-8d89-428b-b793-cbbe67151f50","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6802","nonce":775837313},{"type":"v1","addr":"172.21.15.47:6803","nonce":775837313}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6804","nonce":775837313},{"type":"v1","addr":"172.21.15.47:6805","nonce":775837313}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6808","nonce":775837313},{"type":"v1","addr":"172.21.15.47:6809","nonce":775837313}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6806","nonce":775837313},{"type":"v1","addr":"172.21.15.47:6807","nonce":775837313}]},"public_addr":"172.21.15.47:6803/775837313","cluster_addr":"172.21.15.47:6805/775837313","heartbeat_back_addr":"172.21.15.47:6809/775837313","heartbeat_front_addr":"172.21.15.47:6807/775837313","state":["exists","up"]},{"osd":1,"uuid":"07bb81bb-7cf3-4bbc-ac9e-802f95185d11","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6800","nonce":2923304710},{"type":"v1","addr":"172.21.15.100:6801","nonce":2923304710}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":2923304710},{"type":"v1","addr":"172.21.15.100:6803","nonce":2923304710}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":2923304710},{"type":"v1","addr":"172.21.15.100:6807","nonce":2923304710}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6804","nonce":2923304710},{"type":"v1","addr":"172.21.15.100:6805","nonce":2923304710}]},"public_addr":"172.21.15.100:6801/2923304710","cluster_addr":"172.21.15.100:6803/2923304710","heartbeat_back_addr":"172.21.15.100:6807/2923304710","heartbeat_front_addr":"172.21.15.100:6805/2923304710","state":["exists","up"]},{"osd":2,"uuid":"88ce76b9-6081-4836-aad0-8c249c56e8a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":171377005},{"type":"v1","addr":"172.21.15.157:6801","nonce":171377005}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":171377005},{"type":"v1","addr":"172.21.15.157:6803","nonce":171377005}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":171377005},{"type":"v1","addr":"172.21.15.157:6807","nonce":171377005}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":171377005},{"type":"v1","addr":"172.21.15.157:6805","nonce":171377005}]},"public_addr":"172.21.15.157:6801/171377005","cluster_addr":"172.21.15.157:6803/171377005","heartbeat_back_addr":"172.21.15.157:6807/171377005","heartbeat_front_addr":"172.21.15.157:6805/171377005","state":["exists","up"]},{"osd":3,"uuid":"518aa33d-263f-45e6-aab5-2201ceb40e74","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":27,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6810","nonce":640760101},{"type":"v1","addr":"172.21.15.47:6811","nonce":640760101}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6812","nonce":640760101},{"type":"v1","addr":"172.21.15.47:6813","nonce":640760101}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6816","nonce":640760101},{"type":"v1","addr":"172.21.15.47:6817","nonce":640760101}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6814","nonce":640760101},{"type":"v1","addr":"172.21.15.47:6815","nonce":640760101}]},"public_addr":"172.21.15.47:6811/640760101","cluster_addr":"172.21.15.47:6813/640760101","heartbeat_back_addr":"172.21.15.47:6817/640760101","heartbeat_front_addr":"172.21.15.47:6815/640760101","state":["exists","up"]},{"osd":4,"uuid":"987bde83-ecc6-42e5-bd56-c143b3de4226","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":35,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6808","nonce":2785478639},{"type":"v1","addr":"172.21.15.100:6809","nonce":2785478639}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6810","nonce":2785478639},{"type":"v1","addr":"172.21.15.100:6811","nonce":2785478639}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6814","nonce":2785478639},{"type":"v1","addr":"172.21.15.100:6815","nonce":2785478639}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":2785478639},{"type":"v1","addr":"172.21.15.100:6813","nonce":2785478639}]},"public_addr":"172.21.15.100:6809/2785478639","cluster_addr":"172.21.15.100:6811/2785478639","heartbeat_back_addr":"172.21.15.100:6815/2785478639","heartbeat_front_addr":"172.21.15.100:6813/2785478639","state":["exists","up"]},{"osd":5,"uuid":"012d2188-3a61-4a5f-8d01-2001c15ffbe2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":40,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":1525645410},{"type":"v1","addr":"172.21.15.157:6809","nonce":1525645410}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":1525645410},{"type":"v1","addr":"172.21.15.157:6811","nonce":1525645410}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":1525645410},{"type":"v1","addr":"172.21.15.157:6815","nonce":1525645410}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":1525645410},{"type":"v1","addr":"172.21.15.157:6813","nonce":1525645410}]},"public_addr":"172.21.15.157:6809/1525645410","cluster_addr":"172.21.15.157:6811/1525645410","heartbeat_back_addr":"172.21.15.157:6815/1525645410","heartbeat_front_addr":"172.21.15.157:6813/1525645410","state":["exists","up"]},{"osd":6,"uuid":"c12c14b4-85f0-4d88-937c-de4f556437de","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6818","nonce":2321642790},{"type":"v1","addr":"172.21.15.47:6819","nonce":2321642790}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6820","nonce":2321642790},{"type":"v1","addr":"172.21.15.47:6821","nonce":2321642790}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6824","nonce":2321642790},{"type":"v1","addr":"172.21.15.47:6825","nonce":2321642790}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6822","nonce":2321642790},{"type":"v1","addr":"172.21.15.47:6823","nonce":2321642790}]},"public_addr":"172.21.15.47:6819/2321642790","cluster_addr":"172.21.15.47:6821/2321642790","heartbeat_back_addr":"172.21.15.47:6825/2321642790","heartbeat_front_addr":"172.21.15.47:6823/2321642790","state":["exists","up"]},{"osd":7,"uuid":"3c87c6d0-1d1f-4294-9917-aaaad7dda824","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6816","nonce":4104727330},{"type":"v1","addr":"172.21.15.100:6817","nonce":4104727330}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":4104727330},{"type":"v1","addr":"172.21.15.100:6819","nonce":4104727330}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":4104727330},{"type":"v1","addr":"172.21.15.100:6823","nonce":4104727330}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6820","nonce":4104727330},{"type":"v1","addr":"172.21.15.100:6821","nonce":4104727330}]},"public_addr":"172.21.15.100:6817/4104727330","cluster_addr":"172.21.15.100:6819/4104727330","heartbeat_back_addr":"172.21.15.100:6823/4104727330","heartbeat_front_addr":"172.21.15.100:6821/4104727330","state":["exists","up"]},{"osd":8,"uuid":"502d65cf-53c8-4936-8de4-e11b2a6e1e18","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":61,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":4067715347},{"type":"v1","addr":"172.21.15.157:6817","nonce":4067715347}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":4067715347},{"type":"v1","addr":"172.21.15.157:6819","nonce":4067715347}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":4067715347},{"type":"v1","addr":"172.21.15.157:6823","nonce":4067715347}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":4067715347},{"type":"v1","addr":"172.21.15.157:6821","nonce":4067715347}]},"public_addr":"172.21.15.157:6817/4067715347","cluster_addr":"172.21.15.157:6819/4067715347","heartbeat_back_addr":"172.21.15.157:6823/4067715347","heartbeat_front_addr":"172.21.15.157:6821/4067715347","state":["exists","up"]},{"osd":9,"uuid":"5ea1f5be-8458-4c93-9e61-c2e867a514f3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6826","nonce":1982108738},{"type":"v1","addr":"172.21.15.47:6827","nonce":1982108738}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6828","nonce":1982108738},{"type":"v1","addr":"172.21.15.47:6829","nonce":1982108738}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6832","nonce":1982108738},{"type":"v1","addr":"172.21.15.47:6833","nonce":1982108738}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6830","nonce":1982108738},{"type":"v1","addr":"172.21.15.47:6831","nonce":1982108738}]},"public_addr":"172.21.15.47:6827/1982108738","cluster_addr":"172.21.15.47:6829/1982108738","heartbeat_back_addr":"172.21.15.47:6833/1982108738","heartbeat_front_addr":"172.21.15.47:6831/1982108738","state":["exists","up"]},{"osd":10,"uuid":"2da3da90-a0be-4a54-a9ff-b2d6c9b36870","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":74,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":396391028},{"type":"v1","addr":"172.21.15.100:6825","nonce":396391028}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":396391028},{"type":"v1","addr":"172.21.15.100:6827","nonce":396391028}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6830","nonce":396391028},{"type":"v1","addr":"172.21.15.100:6831","nonce":396391028}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":396391028},{"type":"v1","addr":"172.21.15.100:6829","nonce":396391028}]},"public_addr":"172.21.15.100:6825/396391028","cluster_addr":"172.21.15.100:6827/396391028","heartbeat_back_addr":"172.21.15.100:6831/396391028","heartbeat_front_addr":"172.21.15.100:6829/396391028","state":["exists","up"]},{"osd":11,"uuid":"11798d41-7f7e-4319-95b8-8d3d0946bde9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":82,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":2671029483},{"type":"v1","addr":"172.21.15.157:6825","nonce":2671029483}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":2671029483},{"type":"v1","addr":"172.21.15.157:6827","nonce":2671029483}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":2671029483},{"type":"v1","addr":"172.21.15.157:6831","nonce":2671029483}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":2671029483},{"type":"v1","addr":"172.21.15.157:6829","nonce":2671029483}]},"public_addr":"172.21.15.157:6825/2671029483","cluster_addr":"172.21.15.157:6827/2671029483","heartbeat_back_addr":"172.21.15.157:6831/2671029483","heartbeat_front_addr":"172.21.15.157:6829/2671029483","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T10:14:31.822399+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T10:15:04.175721+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T10:15:35.902595+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T10:16:12.088891+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T10:16:46.428704+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T10:17:20.158270+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T10:17:55.915253+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T10:18:30.064641+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T10:19:04.410004+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T10:19:40.605181+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T10:20:15.636605+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T10:20:50.151365+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.11","mappings":[{"from":7,"to":11}]},{"pgid":"2.14","mappings":[{"from":2,"to":4}]},{"pgid":"2.1a","mappings":[{"from":8,"to":1}]},{"pgid":"2.1b","mappings":[{"from":2,"to":4}]},{"pgid":"2.1d","mappings":[{"from":8,"to":11}]},{"pgid":"3.0","mappings":[{"from":4,"to":8},{"from":5,"to":9}]},{"pgid":"3.6","mappings":[{"from":3,"to":7}]},{"pgid":"3.1b","mappings":[{"from":4,"to":6}]},{"pgid":"3.33","mappings":[{"from":5,"to":9}]},{"pgid":"3.3c","mappings":[{"from":4,"to":9}]},{"pgid":"4.c","mappings":[{"from":9,"to":10}]},{"pgid":"4.f","mappings":[{"from":9,"to":8}]},{"pgid":"4.19","mappings":[{"from":9,"to":8}]},{"pgid":"4.1d","mappings":[{"from":5,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.47:0/936039355":"2024-03-22T10:25:19.912777+0000","172.21.15.47:0/2983192238":"2024-03-22T10:25:19.912777+0000","172.21.15.47:6801/2823885994":"2024-03-22T10:25:19.912777+0000","172.21.15.47:0/356822431":"2024-03-22T10:12:25.844530+0000","172.21.15.47:0/1167490879":"2024-03-22T10:12:25.844530+0000","172.21.15.47:0/1540718138":"2024-03-22T10:12:25.844530+0000","172.21.15.47:6801/3836903972":"2024-03-22T10:12:25.844530+0000","172.21.15.47:0/235436602":"2024-03-22T10:11:50.301838+0000","172.21.15.47:0/2052815572":"2024-03-22T10:11:50.301838+0000","172.21.15.47:0/1607664848":"2024-03-22T10:25:19.912777+0000","172.21.15.47:6800/3942658432":"2024-03-22T10:11:50.301838+0000","172.21.15.47:0/3101796817":"2024-03-22T10:25:19.912777+0000","172.21.15.47:6801/3942658432":"2024-03-22T10:11:50.301838+0000","172.21.15.47:6800/2823885994":"2024-03-22T10:25:19.912777+0000","172.21.15.47:0/1419516257":"2024-03-22T10:11:50.301838+0000","172.21.15.47:6800/3836903972":"2024-03-22T10:12:25.844530+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[{"pool":3,"snaps":[{"begin":5,"length":1},{"begin":15,"length":1}]},{"pool":4,"snaps":[{"begin":5,"length":1},{"begin":15,"length":1}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-21T10:35:29.876 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:29.877+0000 7efe31a00640 1 -- 172.21.15.47:0/555838864 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7efe100a3090 msgr2=0x7efe100a5550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:35:29.876 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:29.877+0000 7efe31a00640 1 --2- 172.21.15.47:0/555838864 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7efe100a3090 0x7efe100a5550 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7efe1c005d00 tx=0x7efe1c00b410 comp rx=0 tx=0).stop 2024-03-21T10:35:29.876 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:29.877+0000 7efe31a00640 1 -- 172.21.15.47:0/555838864 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7efe2c109680 msgr2=0x7efe2c19e6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:35:29.876 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:29.877+0000 7efe31a00640 1 --2- 172.21.15.47:0/555838864 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7efe2c109680 0x7efe2c19e6e0 secure :-1 s=READY pgs=1323 cs=0 l=1 rev1=1 crypto rx=0x7efe24005e30 tx=0x7efe24035ce0 comp rx=0 tx=0).stop 2024-03-21T10:35:29.877 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:29.877+0000 7efe31a00640 1 -- 172.21.15.47:0/555838864 shutdown_connections 2024-03-21T10:35:29.877 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:29.878+0000 7efe31a00640 1 --2- 172.21.15.47:0/555838864 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7efe2c109680 0x7efe2c19e6e0 unknown :-1 s=CLOSED pgs=1323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:35:29.877 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:29.878+0000 7efe31a00640 1 --2- 172.21.15.47:0/555838864 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7efe2c104030 0x7efe2c19a360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:35:29.877 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:29.878+0000 7efe31a00640 1 --2- 172.21.15.47:0/555838864 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7efe100a3090 0x7efe100a5550 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:35:29.877 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:29.878+0000 7efe31a00640 1 --2- 172.21.15.47:0/555838864 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7efe2c103670 0x7efe2c199e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:35:29.877 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:29.878+0000 7efe31a00640 1 -- 172.21.15.47:0/555838864 >> 172.21.15.47:0/555838864 conn(0x7efe2c0ff140 msgr2=0x7efe2c0ff890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:35:29.877 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:29.878+0000 7efe31a00640 1 -- 172.21.15.47:0/555838864 shutdown_connections 2024-03-21T10:35:29.877 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:29.878+0000 7efe31a00640 1 -- 172.21.15.47:0/555838864 wait complete. 2024-03-21T10:35:29.898 INFO:tasks.check_counter.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-21T10:15:40.553114+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-21T10:21:56.399518+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '86', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-21T10:21:57.812940+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '104', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 15, 'snap_epoch': 104, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-03-21T10:22:01.822568+0000', 'flags': 8197, 'flags_names': 'hashpspool,ec_overwrites,selfmanaged_snaps', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '104', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 15, 'snap_epoch': 104, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-03-21T10:35:29.899 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-21T10:35:30.123 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:30.124+0000 7f874e600640 1 -- 172.21.15.47:0/2897685832 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8748108860 msgr2=0x7f874810c8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:35:30.123 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:30.124+0000 7f874e600640 1 --2- 172.21.15.47:0/2897685832 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8748108860 0x7f874810c8a0 secure :-1 s=READY pgs=1324 cs=0 l=1 rev1=1 crypto rx=0x7f8730009900 tx=0x7f873002bc30 comp rx=0 tx=0).stop 2024-03-21T10:35:30.124 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:30.124+0000 7f874e600640 1 -- 172.21.15.47:0/2897685832 shutdown_connections 2024-03-21T10:35:30.124 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:30.124+0000 7f874e600640 1 --2- 172.21.15.47:0/2897685832 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f874810cf80 0x7f8748114b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:35:30.124 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:30.124+0000 7f874e600640 1 --2- 172.21.15.47:0/2897685832 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8748108860 0x7f874810c8a0 unknown :-1 s=CLOSED pgs=1324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:35:30.124 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:30.124+0000 7f874e600640 1 --2- 172.21.15.47:0/2897685832 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8748107eb0 0x7f8748108290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:35:30.124 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:30.124+0000 7f874e600640 1 -- 172.21.15.47:0/2897685832 >> 172.21.15.47:0/2897685832 conn(0x7f8748075be0 msgr2=0x7f8748078000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:35:30.124 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:30.124+0000 7f874e600640 1 -- 172.21.15.47:0/2897685832 shutdown_connections 2024-03-21T10:35:30.124 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:30.124+0000 7f874e600640 1 -- 172.21.15.47:0/2897685832 wait complete. 2024-03-21T10:35:30.126 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:30.125+0000 7f874e600640 1 Processor -- start 2024-03-21T10:35:30.126 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:30.125+0000 7f874e600640 1 -- start start 2024-03-21T10:35:30.126 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:30.126+0000 7f874e600640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8748107eb0 0x7f874819e500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:35:30.126 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:30.126+0000 7f874e600640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f874810cf80 0x7f874819ea40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:35:30.126 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:30.126+0000 7f874e600640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f87481a2d70 0x7f87481a3220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:35:30.126 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:30.126+0000 7f874e600640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8748117ee0 con 0x7f8748107eb0 2024-03-21T10:35:30.126 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:30.126+0000 7f874e600640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8748117d60 con 0x7f87481a2d70 2024-03-21T10:35:30.126 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:30.126+0000 7f874e600640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8748118060 con 0x7f874810cf80 2024-03-21T10:35:30.126 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:30.126+0000 7f8747400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8748107eb0 0x7f874819e500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:35:30.126 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:30.126+0000 7f8747400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8748107eb0 0x7f874819e500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:59012/0 (socket says 172.21.15.47:59012) 2024-03-21T10:35:30.126 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:30.126+0000 7f8747400640 1 -- 172.21.15.47:0/1376706801 learned_addr learned my addr 172.21.15.47:0/1376706801 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:35:30.127 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:30.126+0000 7f8747e00640 1 --2- 172.21.15.47:0/1376706801 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f87481a2d70 0x7f87481a3220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:35:30.127 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:30.126+0000 7f873ea00640 1 --2- 172.21.15.47:0/1376706801 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f874810cf80 0x7f874819ea40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:35:30.127 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:30.127+0000 7f8747e00640 1 -- 172.21.15.47:0/1376706801 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f874810cf80 msgr2=0x7f874819ea40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:35:30.127 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:30.127+0000 7f8747e00640 1 --2- 172.21.15.47:0/1376706801 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f874810cf80 0x7f874819ea40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:35:30.127 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:30.127+0000 7f8747e00640 1 -- 172.21.15.47:0/1376706801 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8748107eb0 msgr2=0x7f874819e500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:35:30.127 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:30.127+0000 7f8747e00640 1 --2- 172.21.15.47:0/1376706801 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8748107eb0 0x7f874819e500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:35:30.127 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:30.127+0000 7f8747e00640 1 -- 172.21.15.47:0/1376706801 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87481a3860 con 0x7f87481a2d70 2024-03-21T10:35:30.127 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:30.127+0000 7f8747e00640 1 --2- 172.21.15.47:0/1376706801 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f87481a2d70 0x7f87481a3220 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f874000e970 tx=0x7f874000ee30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:35:30.128 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:30.127+0000 7f8744c00640 1 -- 172.21.15.47:0/1376706801 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f874000ccc0 con 0x7f87481a2d70 2024-03-21T10:35:30.128 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:30.127+0000 7f874e600640 1 -- 172.21.15.47:0/1376706801 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f87481a3b50 con 0x7f87481a2d70 2024-03-21T10:35:30.128 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:30.128+0000 7f8744c00640 1 -- 172.21.15.47:0/1376706801 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f87400044d0 con 0x7f87481a2d70 2024-03-21T10:35:30.128 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:30.128+0000 7f874e600640 1 -- 172.21.15.47:0/1376706801 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f87481ab310 con 0x7f87481a2d70 2024-03-21T10:35:30.128 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:30.128+0000 7f8744c00640 1 -- 172.21.15.47:0/1376706801 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f874001c070 con 0x7f87481a2d70 2024-03-21T10:35:30.128 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:30.129+0000 7f874e600640 1 -- 172.21.15.47:0/1376706801 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f87480688e0 con 0x7f87481a2d70 2024-03-21T10:35:30.129 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:30.131+0000 7f8744c00640 1 -- 172.21.15.47:0/1376706801 <== mon.2 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 27) v1 ==== 137985+0+0 (secure 0 0 0) 0x7f8740005020 con 0x7f87481a2d70 2024-03-21T10:35:30.131 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:30.133+0000 7f8744c00640 1 --2- 172.21.15.47:0/1376706801 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f872c0a3090 0x7f872c0a5550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:35:30.132 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:30.133+0000 7f8744c00640 1 -- 172.21.15.47:0/1376706801 <== mon.2 v2:172.21.15.100:3300/0 5 ==== osd_map(105..105 src has 1..105) v4 ==== 9562+0+0 (secure 0 0 0) 0x7f87400cfc70 con 0x7f87481a2d70 2024-03-21T10:35:30.132 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:30.133+0000 7f8747400640 1 --2- 172.21.15.47:0/1376706801 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f872c0a3090 0x7f872c0a5550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:35:30.132 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:30.134+0000 7f8747400640 1 --2- 172.21.15.47:0/1376706801 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f872c0a3090 0x7f872c0a5550 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f8738009270 tx=0x7f8738009700 comp rx=0 tx=0).ready entity=mgr.15687 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:35:30.143 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:30.145+0000 7f8744c00640 1 -- 172.21.15.47:0/1376706801 <== mon.2 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f874009c380 con 0x7f87481a2d70 2024-03-21T10:35:30.357 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:30.358+0000 7f874e600640 1 -- 172.21.15.47:0/1376706801 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f8748110270 con 0x7f87481a2d70 2024-03-21T10:35:30.358 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:30.359+0000 7f8744c00640 1 -- 172.21.15.47:0/1376706801 <== mon.2 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v105) v1 ==== 93+0+10 (secure 0 0 0) 0x7f874009bc80 con 0x7f87481a2d70 2024-03-21T10:35:30.358 INFO:teuthology.orchestra.run.smithi047.stdout:pg_num: 1 2024-03-21T10:35:30.361 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:30.362+0000 7f874e600640 1 -- 172.21.15.47:0/1376706801 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f872c0a3090 msgr2=0x7f872c0a5550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:35:30.361 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:30.362+0000 7f874e600640 1 --2- 172.21.15.47:0/1376706801 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f872c0a3090 0x7f872c0a5550 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f8738009270 tx=0x7f8738009700 comp rx=0 tx=0).stop 2024-03-21T10:35:30.361 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:30.362+0000 7f874e600640 1 -- 172.21.15.47:0/1376706801 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f87481a2d70 msgr2=0x7f87481a3220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:35:30.362 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:30.362+0000 7f874e600640 1 --2- 172.21.15.47:0/1376706801 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f87481a2d70 0x7f87481a3220 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f874000e970 tx=0x7f874000ee30 comp rx=0 tx=0).stop 2024-03-21T10:35:30.362 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:30.362+0000 7f874e600640 1 -- 172.21.15.47:0/1376706801 shutdown_connections 2024-03-21T10:35:30.362 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:30.362+0000 7f874e600640 1 --2- 172.21.15.47:0/1376706801 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f872c0a3090 0x7f872c0a5550 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:35:30.362 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:30.362+0000 7f874e600640 1 --2- 172.21.15.47:0/1376706801 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f87481a2d70 0x7f87481a3220 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:35:30.362 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:30.362+0000 7f874e600640 1 --2- 172.21.15.47:0/1376706801 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f874810cf80 0x7f874819ea40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:35:30.362 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:30.362+0000 7f874e600640 1 --2- 172.21.15.47:0/1376706801 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8748107eb0 0x7f874819e500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:35:30.362 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:30.362+0000 7f874e600640 1 -- 172.21.15.47:0/1376706801 >> 172.21.15.47:0/1376706801 conn(0x7f8748075be0 msgr2=0x7f8748076f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:35:30.362 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:30.362+0000 7f874e600640 1 -- 172.21.15.47:0/1376706801 shutdown_connections 2024-03-21T10:35:30.362 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:30.362+0000 7f874e600640 1 -- 172.21.15.47:0/1376706801 wait complete. 2024-03-21T10:35:30.383 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-21T10:35:30.616 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:30.617+0000 7f3f41e00640 1 -- 172.21.15.47:0/2142803517 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3f3c100d80 msgr2=0x7f3f3c10dad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:35:30.616 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:30.617+0000 7f3f41e00640 1 --2- 172.21.15.47:0/2142803517 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3f3c100d80 0x7f3f3c10dad0 secure :-1 s=READY pgs=1325 cs=0 l=1 rev1=1 crypto rx=0x7f3f28009900 tx=0x7f3f2802bc30 comp rx=0 tx=0).stop 2024-03-21T10:35:30.616 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:30.617+0000 7f3f41e00640 1 -- 172.21.15.47:0/2142803517 shutdown_connections 2024-03-21T10:35:30.617 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:30.617+0000 7f3f41e00640 1 --2- 172.21.15.47:0/2142803517 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3f3c10e010 0x7f3f3c110400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:35:30.617 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:30.617+0000 7f3f41e00640 1 --2- 172.21.15.47:0/2142803517 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3f3c100d80 0x7f3f3c10dad0 unknown :-1 s=CLOSED pgs=1325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:35:30.617 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:30.617+0000 7f3f41e00640 1 --2- 172.21.15.47:0/2142803517 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3f3c100460 0x7f3f3c100840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:35:30.617 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:30.617+0000 7f3f41e00640 1 -- 172.21.15.47:0/2142803517 >> 172.21.15.47:0/2142803517 conn(0x7f3f3c076920 msgr2=0x7f3f3c076d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:35:30.617 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:30.617+0000 7f3f41e00640 1 -- 172.21.15.47:0/2142803517 shutdown_connections 2024-03-21T10:35:30.617 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:30.617+0000 7f3f41e00640 1 -- 172.21.15.47:0/2142803517 wait complete. 2024-03-21T10:35:30.620 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:30.618+0000 7f3f41e00640 1 Processor -- start 2024-03-21T10:35:30.620 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:30.618+0000 7f3f41e00640 1 -- start start 2024-03-21T10:35:30.620 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:30.618+0000 7f3f41e00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3f3c100460 0x7f3f3c19a0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:35:30.620 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:30.618+0000 7f3f41e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3f3c100d80 0x7f3f3c19a600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:35:30.620 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:30.618+0000 7f3f41e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3f3c10e010 0x7f3f3c19e980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:35:30.620 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:30.618+0000 7f3f41e00640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f3c113b20 con 0x7f3f3c10e010 2024-03-21T10:35:30.620 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:30.618+0000 7f3f41e00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f3c1139a0 con 0x7f3f3c100d80 2024-03-21T10:35:30.620 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:30.618+0000 7f3f41e00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f3c113ca0 con 0x7f3f3c100460 2024-03-21T10:35:30.620 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:30.619+0000 7f3f3a000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3f3c100d80 0x7f3f3c19a600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:35:30.621 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:30.619+0000 7f3f3b400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3f3c10e010 0x7f3f3c19e980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:35:30.621 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:30.619+0000 7f3f3a000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3f3c100d80 0x7f3f3c19a600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:60174/0 (socket says 172.21.15.47:60174) 2024-03-21T10:35:30.621 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:30.619+0000 7f3f3a000640 1 -- 172.21.15.47:0/1011182353 learned_addr learned my addr 172.21.15.47:0/1011182353 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:35:30.622 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:30.619+0000 7f3f3aa00640 1 --2- 172.21.15.47:0/1011182353 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3f3c100460 0x7f3f3c19a0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:35:30.622 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:30.619+0000 7f3f3a000640 1 -- 172.21.15.47:0/1011182353 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3f3c100460 msgr2=0x7f3f3c19a0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:35:30.622 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:30.619+0000 7f3f3a000640 1 --2- 172.21.15.47:0/1011182353 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3f3c100460 0x7f3f3c19a0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:35:30.622 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:30.619+0000 7f3f3a000640 1 -- 172.21.15.47:0/1011182353 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3f3c10e010 msgr2=0x7f3f3c19e980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:35:30.622 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:30.619+0000 7f3f3a000640 1 --2- 172.21.15.47:0/1011182353 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3f3c10e010 0x7f3f3c19e980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:35:30.622 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:30.619+0000 7f3f3a000640 1 -- 172.21.15.47:0/1011182353 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f3c19f0b0 con 0x7f3f3c100d80 2024-03-21T10:35:30.622 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:30.619+0000 7f3f3b400640 1 --2- 172.21.15.47:0/1011182353 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3f3c10e010 0x7f3f3c19e980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T10:35:30.622 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:30.620+0000 7f3f3a000640 1 --2- 172.21.15.47:0/1011182353 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3f3c100d80 0x7f3f3c19a600 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f3f28005940 tx=0x7f3f280045e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:35:30.622 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:30.620+0000 7f3f40a00640 1 -- 172.21.15.47:0/1011182353 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f28043020 con 0x7f3f3c100d80 2024-03-21T10:35:30.622 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:30.620+0000 7f3f41e00640 1 -- 172.21.15.47:0/1011182353 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3f3c19f340 con 0x7f3f3c100d80 2024-03-21T10:35:30.622 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:30.620+0000 7f3f40a00640 1 -- 172.21.15.47:0/1011182353 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f3f2802caf0 con 0x7f3f3c100d80 2024-03-21T10:35:30.622 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:30.620+0000 7f3f40a00640 1 -- 172.21.15.47:0/1011182353 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f28003a50 con 0x7f3f3c100d80 2024-03-21T10:35:30.622 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:30.620+0000 7f3f41e00640 1 -- 172.21.15.47:0/1011182353 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3f3c1a6c30 con 0x7f3f3c100d80 2024-03-21T10:35:30.623 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:30.621+0000 7f3f41e00640 1 -- 172.21.15.47:0/1011182353 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3f3c06a490 con 0x7f3f3c100d80 2024-03-21T10:35:30.624 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:30.625+0000 7f3f40a00640 1 -- 172.21.15.47:0/1011182353 <== mon.2 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 27) v1 ==== 137985+0+0 (secure 0 0 0) 0x7f3f28046020 con 0x7f3f3c100d80 2024-03-21T10:35:30.626 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:30.627+0000 7f3f40a00640 1 --2- 172.21.15.47:0/1011182353 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f3f200a3090 0x7f3f200a5550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:35:30.626 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:30.627+0000 7f3f40a00640 1 -- 172.21.15.47:0/1011182353 <== mon.2 v2:172.21.15.100:3300/0 5 ==== osd_map(105..105 src has 1..105) v4 ==== 9562+0+0 (secure 0 0 0) 0x7f3f280f2320 con 0x7f3f3c100d80 2024-03-21T10:35:30.626 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:30.627+0000 7f3f3aa00640 1 --2- 172.21.15.47:0/1011182353 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f3f200a3090 0x7f3f200a5550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:35:30.627 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:30.628+0000 7f3f3aa00640 1 --2- 172.21.15.47:0/1011182353 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f3f200a3090 0x7f3f200a5550 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f3f2c0058c0 tx=0x7f3f2c00a4c0 comp rx=0 tx=0).ready entity=mgr.15687 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:35:30.630 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:30.632+0000 7f3f40a00640 1 -- 172.21.15.47:0/1011182353 <== mon.2 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f3f2804d020 con 0x7f3f3c100d80 2024-03-21T10:35:30.710 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:35:30 smithi047 ceph-mon[31547]: from='client.? 172.21.15.47:0/555838864' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-21T10:35:30.726 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:35:30 smithi100 ceph-mon[35193]: from='client.? 172.21.15.47:0/555838864' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-21T10:35:30.736 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:35:30 smithi157 ceph-mon[35581]: from='client.? 172.21.15.47:0/555838864' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-21T10:35:30.886 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:30.887+0000 7f3f41e00640 1 -- 172.21.15.47:0/1011182353 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f3f3c1091d0 con 0x7f3f3c100d80 2024-03-21T10:35:30.886 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:30.887+0000 7f3f40a00640 1 -- 172.21.15.47:0/1011182353 <== mon.2 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v105) v1 ==== 104+0+11 (secure 0 0 0) 0x7f3f280beab0 con 0x7f3f3c100d80 2024-03-21T10:35:30.886 INFO:teuthology.orchestra.run.smithi047.stdout:pg_num: 32 2024-03-21T10:35:30.890 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:30.891+0000 7f3f41e00640 1 -- 172.21.15.47:0/1011182353 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f3f200a3090 msgr2=0x7f3f200a5550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:35:30.890 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:30.891+0000 7f3f41e00640 1 --2- 172.21.15.47:0/1011182353 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f3f200a3090 0x7f3f200a5550 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f3f2c0058c0 tx=0x7f3f2c00a4c0 comp rx=0 tx=0).stop 2024-03-21T10:35:30.890 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:30.891+0000 7f3f41e00640 1 -- 172.21.15.47:0/1011182353 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3f3c100d80 msgr2=0x7f3f3c19a600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:35:30.890 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:30.891+0000 7f3f41e00640 1 --2- 172.21.15.47:0/1011182353 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3f3c100d80 0x7f3f3c19a600 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f3f28005940 tx=0x7f3f280045e0 comp rx=0 tx=0).stop 2024-03-21T10:35:30.890 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:30.891+0000 7f3f41e00640 1 -- 172.21.15.47:0/1011182353 shutdown_connections 2024-03-21T10:35:30.890 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:30.891+0000 7f3f41e00640 1 --2- 172.21.15.47:0/1011182353 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3f3c10e010 0x7f3f3c19e980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:35:30.890 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:30.891+0000 7f3f41e00640 1 --2- 172.21.15.47:0/1011182353 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f3f200a3090 0x7f3f200a5550 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:35:30.890 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:30.891+0000 7f3f41e00640 1 --2- 172.21.15.47:0/1011182353 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3f3c100d80 0x7f3f3c19a600 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:35:30.891 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:30.891+0000 7f3f41e00640 1 --2- 172.21.15.47:0/1011182353 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3f3c100460 0x7f3f3c19a0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:35:30.891 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:30.891+0000 7f3f41e00640 1 -- 172.21.15.47:0/1011182353 >> 172.21.15.47:0/1011182353 conn(0x7f3f3c076920 msgr2=0x7f3f3c10f750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:35:30.891 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:30.891+0000 7f3f41e00640 1 -- 172.21.15.47:0/1011182353 shutdown_connections 2024-03-21T10:35:30.891 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:30.891+0000 7f3f41e00640 1 -- 172.21.15.47:0/1011182353 wait complete. 2024-03-21T10:35:30.909 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-21T10:35:31.131 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:31.131+0000 7f02e2200640 1 -- 172.21.15.47:0/1649982473 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f02dc10baf0 msgr2=0x7f02dc10df90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:35:31.131 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:31.131+0000 7f02e2200640 1 --2- 172.21.15.47:0/1649982473 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f02dc10baf0 0x7f02dc10df90 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f02d4009ca0 tx=0x7f02d402c190 comp rx=0 tx=0).stop 2024-03-21T10:35:31.131 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:31.132+0000 7f02e2200640 1 -- 172.21.15.47:0/1649982473 shutdown_connections 2024-03-21T10:35:31.131 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:31.132+0000 7f02e2200640 1 --2- 172.21.15.47:0/1649982473 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f02dc10baf0 0x7f02dc10df90 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:35:31.131 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:31.132+0000 7f02e2200640 1 --2- 172.21.15.47:0/1649982473 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f02dc102ff0 0x7f02dc10b4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:35:31.131 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:31.132+0000 7f02e2200640 1 --2- 172.21.15.47:0/1649982473 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f02dc1026d0 0x7f02dc102ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:35:31.131 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:31.132+0000 7f02e2200640 1 -- 172.21.15.47:0/1649982473 >> 172.21.15.47:0/1649982473 conn(0x7f02dc0fbef0 msgr2=0x7f02dc0fe310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:35:31.132 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:31.132+0000 7f02e2200640 1 -- 172.21.15.47:0/1649982473 shutdown_connections 2024-03-21T10:35:31.132 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:31.132+0000 7f02e2200640 1 -- 172.21.15.47:0/1649982473 wait complete. 2024-03-21T10:35:31.134 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:31.133+0000 7f02e2200640 1 Processor -- start 2024-03-21T10:35:31.134 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:31.133+0000 7f02e2200640 1 -- start start 2024-03-21T10:35:31.134 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:31.133+0000 7f02e2200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f02dc1026d0 0x7f02dc199f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:35:31.134 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:31.133+0000 7f02e2200640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f02dc102ff0 0x7f02dc19a440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:35:31.134 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:31.133+0000 7f02e2200640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f02dc10baf0 0x7f02dc19e7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:35:31.134 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:31.133+0000 7f02e2200640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02dc111780 con 0x7f02dc10baf0 2024-03-21T10:35:31.134 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:31.133+0000 7f02e2200640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02dc111600 con 0x7f02dc1026d0 2024-03-21T10:35:31.134 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:31.133+0000 7f02e2200640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02dc111900 con 0x7f02dc102ff0 2024-03-21T10:35:31.135 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:31.134+0000 7f02e0e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f02dc1026d0 0x7f02dc199f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:35:31.135 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:31.134+0000 7f02e0e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f02dc1026d0 0x7f02dc199f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:60198/0 (socket says 172.21.15.47:60198) 2024-03-21T10:35:31.135 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:31.134+0000 7f02e0e00640 1 -- 172.21.15.47:0/3326126196 learned_addr learned my addr 172.21.15.47:0/3326126196 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:35:31.135 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:31.134+0000 7f02e1800640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f02dc10baf0 0x7f02dc19e7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:35:31.135 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:31.134+0000 7f02dbe00640 1 --2- 172.21.15.47:0/3326126196 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f02dc102ff0 0x7f02dc19a440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:35:31.135 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:31.134+0000 7f02e0e00640 1 -- 172.21.15.47:0/3326126196 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f02dc102ff0 msgr2=0x7f02dc19a440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:35:31.135 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:31.134+0000 7f02e0e00640 1 --2- 172.21.15.47:0/3326126196 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f02dc102ff0 0x7f02dc19a440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:35:31.135 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:31.134+0000 7f02e0e00640 1 -- 172.21.15.47:0/3326126196 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f02dc10baf0 msgr2=0x7f02dc19e7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:35:31.136 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:31.134+0000 7f02e0e00640 1 --2- 172.21.15.47:0/3326126196 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f02dc10baf0 0x7f02dc19e7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:35:31.136 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:31.134+0000 7f02e0e00640 1 -- 172.21.15.47:0/3326126196 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f02dc19eef0 con 0x7f02dc1026d0 2024-03-21T10:35:31.136 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:31.134+0000 7f02dbe00640 1 --2- 172.21.15.47:0/3326126196 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f02dc102ff0 0x7f02dc19a440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T10:35:31.136 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:31.135+0000 7f02e0e00640 1 --2- 172.21.15.47:0/3326126196 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f02dc1026d0 0x7f02dc199f00 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f02d000e9f0 tx=0x7f02d000edf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:35:31.136 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:31.135+0000 7f02d9600640 1 -- 172.21.15.47:0/3326126196 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02d000cd80 con 0x7f02dc1026d0 2024-03-21T10:35:31.136 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:31.135+0000 7f02d9600640 1 -- 172.21.15.47:0/3326126196 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f02d00054d0 con 0x7f02dc1026d0 2024-03-21T10:35:31.136 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:31.135+0000 7f02e2200640 1 -- 172.21.15.47:0/3326126196 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f02dc19f1e0 con 0x7f02dc1026d0 2024-03-21T10:35:31.136 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:31.135+0000 7f02d9600640 1 -- 172.21.15.47:0/3326126196 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02d0010490 con 0x7f02dc1026d0 2024-03-21T10:35:31.136 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:31.135+0000 7f02e2200640 1 -- 172.21.15.47:0/3326126196 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f02dc1a6a70 con 0x7f02dc1026d0 2024-03-21T10:35:31.137 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:31.136+0000 7f02e2200640 1 -- 172.21.15.47:0/3326126196 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f02dc0683b0 con 0x7f02dc1026d0 2024-03-21T10:35:31.137 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:31.139+0000 7f02d9600640 1 -- 172.21.15.47:0/3326126196 <== mon.2 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 27) v1 ==== 137985+0+0 (secure 0 0 0) 0x7f02d0005000 con 0x7f02dc1026d0 2024-03-21T10:35:31.139 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:31.140+0000 7f02d9600640 1 --2- 172.21.15.47:0/3326126196 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f02c40a3090 0x7f02c40a5550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:35:31.139 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:31.140+0000 7f02d9600640 1 -- 172.21.15.47:0/3326126196 <== mon.2 v2:172.21.15.100:3300/0 5 ==== osd_map(105..105 src has 1..105) v4 ==== 9562+0+0 (secure 0 0 0) 0x7f02d00d0260 con 0x7f02dc1026d0 2024-03-21T10:35:31.139 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:31.140+0000 7f02dbe00640 1 --2- 172.21.15.47:0/3326126196 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f02c40a3090 0x7f02c40a5550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:35:31.140 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:31.141+0000 7f02dbe00640 1 --2- 172.21.15.47:0/3326126196 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f02c40a3090 0x7f02c40a5550 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f02dc10e3d0 tx=0x7f02c8005f90 comp rx=0 tx=0).ready entity=mgr.15687 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:35:31.149 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:31.150+0000 7f02d9600640 1 -- 172.21.15.47:0/3326126196 <== mon.2 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f02d0053740 con 0x7f02dc1026d0 2024-03-21T10:35:31.369 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:31.371+0000 7f02e2200640 1 -- 172.21.15.47:0/3326126196 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f02dc19b250 con 0x7f02dc1026d0 2024-03-21T10:35:31.370 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:31.371+0000 7f02d9600640 1 -- 172.21.15.47:0/3326126196 <== mon.2 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v105) v1 ==== 100+0+11 (secure 0 0 0) 0x7f02d00052f0 con 0x7f02dc1026d0 2024-03-21T10:35:31.370 INFO:teuthology.orchestra.run.smithi047.stdout:pg_num: 64 2024-03-21T10:35:31.374 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:31.374+0000 7f02e2200640 1 -- 172.21.15.47:0/3326126196 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f02c40a3090 msgr2=0x7f02c40a5550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:35:31.374 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:31.374+0000 7f02e2200640 1 --2- 172.21.15.47:0/3326126196 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f02c40a3090 0x7f02c40a5550 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f02dc10e3d0 tx=0x7f02c8005f90 comp rx=0 tx=0).stop 2024-03-21T10:35:31.374 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:31.374+0000 7f02e2200640 1 -- 172.21.15.47:0/3326126196 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f02dc1026d0 msgr2=0x7f02dc199f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:35:31.374 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:31.374+0000 7f02e2200640 1 --2- 172.21.15.47:0/3326126196 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f02dc1026d0 0x7f02dc199f00 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f02d000e9f0 tx=0x7f02d000edf0 comp rx=0 tx=0).stop 2024-03-21T10:35:31.374 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:31.374+0000 7f02e2200640 1 -- 172.21.15.47:0/3326126196 shutdown_connections 2024-03-21T10:35:31.374 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:31.374+0000 7f02e2200640 1 --2- 172.21.15.47:0/3326126196 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f02dc10baf0 0x7f02dc19e7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:35:31.374 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:31.375+0000 7f02e2200640 1 --2- 172.21.15.47:0/3326126196 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f02dc102ff0 0x7f02dc19a440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:35:31.374 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:31.375+0000 7f02e2200640 1 --2- 172.21.15.47:0/3326126196 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f02c40a3090 0x7f02c40a5550 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:35:31.374 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:31.375+0000 7f02e2200640 1 --2- 172.21.15.47:0/3326126196 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f02dc1026d0 0x7f02dc199f00 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:35:31.374 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:31.375+0000 7f02e2200640 1 -- 172.21.15.47:0/3326126196 >> 172.21.15.47:0/3326126196 conn(0x7f02dc0fbef0 msgr2=0x7f02dc0fe2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:35:31.374 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:31.375+0000 7f02e2200640 1 -- 172.21.15.47:0/3326126196 shutdown_connections 2024-03-21T10:35:31.375 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:31.375+0000 7f02e2200640 1 -- 172.21.15.47:0/3326126196 wait complete. 2024-03-21T10:35:31.393 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-03-21T10:35:31.632 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:31.632+0000 7fab7e400640 1 -- 172.21.15.47:0/3349539523 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fab781064a0 msgr2=0x7fab78106880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:35:31.632 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:31.632+0000 7fab7e400640 1 --2- 172.21.15.47:0/3349539523 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fab781064a0 0x7fab78106880 secure :-1 s=READY pgs=1326 cs=0 l=1 rev1=1 crypto rx=0x7fab68009930 tx=0x7fab6802bc20 comp rx=0 tx=0).stop 2024-03-21T10:35:31.632 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:31.633+0000 7fab7e400640 1 -- 172.21.15.47:0/3349539523 shutdown_connections 2024-03-21T10:35:31.632 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:31.633+0000 7fab7e400640 1 --2- 172.21.15.47:0/3349539523 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fab78100dc0 0x7fab7810d2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:35:31.632 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:31.633+0000 7fab7e400640 1 --2- 172.21.15.47:0/3349539523 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fab78100400 0x7fab78100880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:35:31.632 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:31.633+0000 7fab7e400640 1 --2- 172.21.15.47:0/3349539523 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fab781064a0 0x7fab78106880 unknown :-1 s=CLOSED pgs=1326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:35:31.632 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:31.633+0000 7fab7e400640 1 -- 172.21.15.47:0/3349539523 >> 172.21.15.47:0/3349539523 conn(0x7fab780fbef0 msgr2=0x7fab780fe310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:35:31.633 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:31.633+0000 7fab7e400640 1 -- 172.21.15.47:0/3349539523 shutdown_connections 2024-03-21T10:35:31.633 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:31.633+0000 7fab7e400640 1 -- 172.21.15.47:0/3349539523 wait complete. 2024-03-21T10:35:31.635 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:31.634+0000 7fab7e400640 1 Processor -- start 2024-03-21T10:35:31.635 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:31.634+0000 7fab7e400640 1 -- start start 2024-03-21T10:35:31.635 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:31.634+0000 7fab7e400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fab78100400 0x7fab78199f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:35:31.635 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:31.634+0000 7fab7e400640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fab78100dc0 0x7fab7819a4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:35:31.635 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:31.634+0000 7fab7e400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fab781064a0 0x7fab7819e7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:35:31.635 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:31.634+0000 7fab7e400640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab78110670 con 0x7fab781064a0 2024-03-21T10:35:31.635 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:31.634+0000 7fab7e400640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab781104f0 con 0x7fab78100400 2024-03-21T10:35:31.635 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:31.634+0000 7fab7e400640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab781107f0 con 0x7fab78100dc0 2024-03-21T10:35:31.635 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:31.635+0000 7fab7d000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fab78100400 0x7fab78199f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:35:31.636 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:31.635+0000 7fab7da00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fab781064a0 0x7fab7819e7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:35:31.636 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:31.635+0000 7fab7d000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fab78100400 0x7fab78199f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:60218/0 (socket says 172.21.15.47:60218) 2024-03-21T10:35:31.636 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:31.635+0000 7fab7d000640 1 -- 172.21.15.47:0/301756020 learned_addr learned my addr 172.21.15.47:0/301756020 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:35:31.636 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:31.635+0000 7fab77e00640 1 --2- 172.21.15.47:0/301756020 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fab78100dc0 0x7fab7819a4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:35:31.636 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:31.635+0000 7fab7d000640 1 -- 172.21.15.47:0/301756020 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fab78100dc0 msgr2=0x7fab7819a4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:35:31.636 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:31.635+0000 7fab7d000640 1 --2- 172.21.15.47:0/301756020 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fab78100dc0 0x7fab7819a4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:35:31.636 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:31.635+0000 7fab7d000640 1 -- 172.21.15.47:0/301756020 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fab781064a0 msgr2=0x7fab7819e7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:35:31.636 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:31.635+0000 7fab7d000640 1 --2- 172.21.15.47:0/301756020 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fab781064a0 0x7fab7819e7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:35:31.636 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:31.635+0000 7fab7d000640 1 -- 172.21.15.47:0/301756020 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab7819ef20 con 0x7fab78100400 2024-03-21T10:35:31.636 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:31.635+0000 7fab77e00640 1 --2- 172.21.15.47:0/301756020 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fab78100dc0 0x7fab7819a4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T10:35:31.636 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:31.635+0000 7fab7da00640 1 --2- 172.21.15.47:0/301756020 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fab781064a0 0x7fab7819e7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T10:35:31.636 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:31.636+0000 7fab7d000640 1 --2- 172.21.15.47:0/301756020 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fab78100400 0x7fab78199f80 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7fab68009230 tx=0x7fab6802c700 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:35:31.636 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:31.636+0000 7fab75600640 1 -- 172.21.15.47:0/301756020 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab68043020 con 0x7fab78100400 2024-03-21T10:35:31.636 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:31.636+0000 7fab7e400640 1 -- 172.21.15.47:0/301756020 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fab7819f1b0 con 0x7fab78100400 2024-03-21T10:35:31.637 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:31.636+0000 7fab75600640 1 -- 172.21.15.47:0/301756020 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fab6802cae0 con 0x7fab78100400 2024-03-21T10:35:31.637 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:31.636+0000 7fab75600640 1 -- 172.21.15.47:0/301756020 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab68003b50 con 0x7fab78100400 2024-03-21T10:35:31.637 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:31.636+0000 7fab7e400640 1 -- 172.21.15.47:0/301756020 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fab781a6aa0 con 0x7fab78100400 2024-03-21T10:35:31.637 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:31.638+0000 7fab7e400640 1 -- 172.21.15.47:0/301756020 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fab780683b0 con 0x7fab78100400 2024-03-21T10:35:31.639 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:31.640+0000 7fab75600640 1 -- 172.21.15.47:0/301756020 <== mon.2 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 27) v1 ==== 137985+0+0 (secure 0 0 0) 0x7fab68046020 con 0x7fab78100400 2024-03-21T10:35:31.640 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:31.641+0000 7fab75600640 1 --2- 172.21.15.47:0/301756020 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7fab5c0a3090 0x7fab5c0a5550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:35:31.640 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:31.641+0000 7fab75600640 1 -- 172.21.15.47:0/301756020 <== mon.2 v2:172.21.15.100:3300/0 5 ==== osd_map(105..105 src has 1..105) v4 ==== 9562+0+0 (secure 0 0 0) 0x7fab680f1dc0 con 0x7fab78100400 2024-03-21T10:35:31.640 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:31.642+0000 7fab77e00640 1 --2- 172.21.15.47:0/301756020 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7fab5c0a3090 0x7fab5c0a5550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:35:31.641 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:31.642+0000 7fab77e00640 1 --2- 172.21.15.47:0/301756020 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7fab5c0a3090 0x7fab5c0a5550 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fab78104470 tx=0x7fab64005e50 comp rx=0 tx=0).ready entity=mgr.15687 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:35:31.651 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:31.652+0000 7fab75600640 1 -- 172.21.15.47:0/301756020 <== mon.2 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fab6804d020 con 0x7fab78100400 2024-03-21T10:35:31.711 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:35:31 smithi047 ceph-mon[31547]: pgmap v317: 129 pgs: 129 active+clean; 2.5 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 0 B/s wr, 10 op/s 2024-03-21T10:35:31.711 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:35:31 smithi047 ceph-mon[31547]: from='client.? 172.21.15.47:0/1376706801' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-21T10:35:31.711 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:35:31 smithi047 ceph-mon[31547]: from='client.? 172.21.15.47:0/1011182353' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-21T10:35:31.726 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:35:31 smithi100 ceph-mon[35193]: pgmap v317: 129 pgs: 129 active+clean; 2.5 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 0 B/s wr, 10 op/s 2024-03-21T10:35:31.726 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:35:31 smithi100 ceph-mon[35193]: from='client.? 172.21.15.47:0/1376706801' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-21T10:35:31.726 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:35:31 smithi100 ceph-mon[35193]: from='client.? 172.21.15.47:0/1011182353' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-21T10:35:31.736 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:35:31 smithi157 ceph-mon[35581]: pgmap v317: 129 pgs: 129 active+clean; 2.5 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 0 B/s wr, 10 op/s 2024-03-21T10:35:31.737 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:35:31 smithi157 ceph-mon[35581]: from='client.? 172.21.15.47:0/1376706801' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-21T10:35:31.737 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:35:31 smithi157 ceph-mon[35581]: from='client.? 172.21.15.47:0/1011182353' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-21T10:35:31.954 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:31.955+0000 7fab7e400640 1 -- 172.21.15.47:0/301756020 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7fab781a6c70 con 0x7fab78100400 2024-03-21T10:35:31.955 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:31.956+0000 7fab75600640 1 -- 172.21.15.47:0/301756020 <== mon.2 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v105) v1 ==== 103+0+11 (secure 0 0 0) 0x7fab680be550 con 0x7fab78100400 2024-03-21T10:35:31.955 INFO:teuthology.orchestra.run.smithi047.stdout:pg_num: 32 2024-03-21T10:35:31.958 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:31.959+0000 7fab7e400640 1 -- 172.21.15.47:0/301756020 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7fab5c0a3090 msgr2=0x7fab5c0a5550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:35:31.958 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:31.959+0000 7fab7e400640 1 --2- 172.21.15.47:0/301756020 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7fab5c0a3090 0x7fab5c0a5550 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fab78104470 tx=0x7fab64005e50 comp rx=0 tx=0).stop 2024-03-21T10:35:31.958 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:31.959+0000 7fab7e400640 1 -- 172.21.15.47:0/301756020 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fab78100400 msgr2=0x7fab78199f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:35:31.958 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:31.959+0000 7fab7e400640 1 --2- 172.21.15.47:0/301756020 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fab78100400 0x7fab78199f80 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7fab68009230 tx=0x7fab6802c700 comp rx=0 tx=0).stop 2024-03-21T10:35:31.959 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:31.959+0000 7fab7e400640 1 -- 172.21.15.47:0/301756020 shutdown_connections 2024-03-21T10:35:31.959 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:31.959+0000 7fab7e400640 1 --2- 172.21.15.47:0/301756020 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fab781064a0 0x7fab7819e7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:35:31.959 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:31.959+0000 7fab7e400640 1 --2- 172.21.15.47:0/301756020 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fab78100dc0 0x7fab7819a4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:35:31.959 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:31.959+0000 7fab7e400640 1 --2- 172.21.15.47:0/301756020 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7fab5c0a3090 0x7fab5c0a5550 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:35:31.959 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:31.959+0000 7fab7e400640 1 --2- 172.21.15.47:0/301756020 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fab78100400 0x7fab78199f80 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:35:31.959 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:31.959+0000 7fab7e400640 1 -- 172.21.15.47:0/301756020 >> 172.21.15.47:0/301756020 conn(0x7fab780fbef0 msgr2=0x7fab78068a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:35:31.959 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:31.959+0000 7fab7e400640 1 -- 172.21.15.47:0/301756020 shutdown_connections 2024-03-21T10:35:31.959 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:31.959+0000 7fab7e400640 1 -- 172.21.15.47:0/301756020 wait complete. 2024-03-21T10:35:31.978 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json-pretty 2024-03-21T10:35:32.223 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:32.223+0000 7f64f9a00640 1 -- 172.21.15.47:0/1039193384 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f64f40743f0 msgr2=0x7f64f410d790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:35:32.223 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:32.223+0000 7f64f9a00640 1 --2- 172.21.15.47:0/1039193384 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f64f40743f0 0x7f64f410d790 secure :-1 s=READY pgs=1327 cs=0 l=1 rev1=1 crypto rx=0x7f64dc009900 tx=0x7f64dc02bcd0 comp rx=0 tx=0).stop 2024-03-21T10:35:32.223 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:32.223+0000 7f64f9a00640 1 -- 172.21.15.47:0/1039193384 shutdown_connections 2024-03-21T10:35:32.223 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:32.223+0000 7f64f9a00640 1 --2- 172.21.15.47:0/1039193384 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f64f410dcd0 0x7f64f4110120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:35:32.223 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:32.223+0000 7f64f9a00640 1 --2- 172.21.15.47:0/1039193384 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f64f40743f0 0x7f64f410d790 unknown :-1 s=CLOSED pgs=1327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:35:32.223 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:32.223+0000 7f64f9a00640 1 --2- 172.21.15.47:0/1039193384 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f64f4073ad0 0x7f64f4073eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:35:32.223 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:32.223+0000 7f64f9a00640 1 -- 172.21.15.47:0/1039193384 >> 172.21.15.47:0/1039193384 conn(0x7f64f40fcfa0 msgr2=0x7f64f40ff3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:35:32.223 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:32.224+0000 7f64f9a00640 1 -- 172.21.15.47:0/1039193384 shutdown_connections 2024-03-21T10:35:32.224 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:32.224+0000 7f64f9a00640 1 -- 172.21.15.47:0/1039193384 wait complete. 2024-03-21T10:35:32.225 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:32.225+0000 7f64f9a00640 1 Processor -- start 2024-03-21T10:35:32.225 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:32.225+0000 7f64f9a00640 1 -- start start 2024-03-21T10:35:32.225 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:32.225+0000 7f64f9a00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f64f4073ad0 0x7f64f4199ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:35:32.225 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:32.225+0000 7f64f9a00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f64f40743f0 0x7f64f419a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:35:32.225 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:32.225+0000 7f64f9a00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f64f410dcd0 0x7f64f419e7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:35:32.225 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:32.225+0000 7f64f9a00640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64f4113970 con 0x7f64f410dcd0 2024-03-21T10:35:32.225 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:32.225+0000 7f64f9a00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64f41137f0 con 0x7f64f40743f0 2024-03-21T10:35:32.225 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:32.225+0000 7f64f9a00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64f4113af0 con 0x7f64f4073ad0 2024-03-21T10:35:32.225 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:32.225+0000 7f64f9000640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f64f410dcd0 0x7f64f419e7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:35:32.226 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:32.225+0000 7f64f9000640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f64f410dcd0 0x7f64f419e7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:49364/0 (socket says 172.21.15.47:49364) 2024-03-21T10:35:32.226 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:32.225+0000 7f64f9000640 1 -- 172.21.15.47:0/4165729570 learned_addr learned my addr 172.21.15.47:0/4165729570 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:35:32.226 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:32.225+0000 7f64ebe00640 1 --2- 172.21.15.47:0/4165729570 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f64f40743f0 0x7f64f419a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:35:32.226 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:32.226+0000 7f64f3e00640 1 --2- 172.21.15.47:0/4165729570 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f64f4073ad0 0x7f64f4199ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:35:32.226 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:32.226+0000 7f64f9000640 1 -- 172.21.15.47:0/4165729570 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f64f4073ad0 msgr2=0x7f64f4199ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:35:32.226 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:32.226+0000 7f64f9000640 1 --2- 172.21.15.47:0/4165729570 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f64f4073ad0 0x7f64f4199ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:35:32.226 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:32.226+0000 7f64f9000640 1 -- 172.21.15.47:0/4165729570 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f64f40743f0 msgr2=0x7f64f419a430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:35:32.226 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:32.226+0000 7f64f9000640 1 --2- 172.21.15.47:0/4165729570 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f64f40743f0 0x7f64f419a430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:35:32.226 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:32.226+0000 7f64f9000640 1 -- 172.21.15.47:0/4165729570 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64f419eee0 con 0x7f64f410dcd0 2024-03-21T10:35:32.226 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:32.226+0000 7f64f9000640 1 --2- 172.21.15.47:0/4165729570 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f64f410dcd0 0x7f64f419e7b0 secure :-1 s=READY pgs=1328 cs=0 l=1 rev1=1 crypto rx=0x7f64ec00efd0 tx=0x7f64ec00c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:35:32.227 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:32.226+0000 7f64f1600640 1 -- 172.21.15.47:0/4165729570 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64ec003b70 con 0x7f64f410dcd0 2024-03-21T10:35:32.227 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:32.227+0000 7f64f9a00640 1 -- 172.21.15.47:0/4165729570 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f64f419f1d0 con 0x7f64f410dcd0 2024-03-21T10:35:32.227 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:32.227+0000 7f64f1600640 1 -- 172.21.15.47:0/4165729570 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f64ec003d10 con 0x7f64f410dcd0 2024-03-21T10:35:32.227 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:32.227+0000 7f64f1600640 1 -- 172.21.15.47:0/4165729570 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64ec010980 con 0x7f64f410dcd0 2024-03-21T10:35:32.227 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:32.227+0000 7f64f9a00640 1 -- 172.21.15.47:0/4165729570 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f64f41a6a60 con 0x7f64f410dcd0 2024-03-21T10:35:32.227 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:32.228+0000 7f64f9a00640 1 -- 172.21.15.47:0/4165729570 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f64f40683b0 con 0x7f64f410dcd0 2024-03-21T10:35:32.228 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:32.230+0000 7f64f1600640 1 -- 172.21.15.47:0/4165729570 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 27) v1 ==== 137985+0+0 (secure 0 0 0) 0x7f64ec004020 con 0x7f64f410dcd0 2024-03-21T10:35:32.230 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:32.231+0000 7f64f1600640 1 --2- 172.21.15.47:0/4165729570 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f64d80a3090 0x7f64d80a5550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:35:32.230 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:32.231+0000 7f64f1600640 1 -- 172.21.15.47:0/4165729570 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(105..105 src has 1..105) v4 ==== 9562+0+0 (secure 0 0 0) 0x7f64ec0d00d0 con 0x7f64f410dcd0 2024-03-21T10:35:32.230 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:32.231+0000 7f64f3e00640 1 --2- 172.21.15.47:0/4165729570 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f64d80a3090 0x7f64d80a5550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:35:32.231 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:32.232+0000 7f64f3e00640 1 --2- 172.21.15.47:0/4165729570 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f64d80a3090 0x7f64d80a5550 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f64e4009b60 tx=0x7f64e4005f90 comp rx=0 tx=0).ready entity=mgr.15687 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:35:32.234 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:32.236+0000 7f64f1600640 1 -- 172.21.15.47:0/4165729570 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f64ec09c750 con 0x7f64f410dcd0 2024-03-21T10:35:32.491 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:32.492+0000 7f64f9a00640 1 -- 172.21.15.47:0/4165729570 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json-pretty"} v 0) v1 -- 0x7f64f4108eb0 con 0x7f64f410dcd0 2024-03-21T10:35:32.525 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:32.526+0000 7f64f1600640 1 -- 172.21.15.47:0/4165729570 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json-pretty"}]=0 v27) v1 ==== 81+0+742370 (secure 0 0 0) 0x7f64ec09c050 con 0x7f64f410dcd0 2024-03-21T10:35:32.533 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-21T10:35:32.534 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-03-21T10:35:32.534 INFO:teuthology.orchestra.run.smithi047.stdout: "epoch": 27, 2024-03-21T10:35:32.534 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T10:35:32.534 INFO:teuthology.orchestra.run.smithi047.stdout: "active_gid": 15687, 2024-03-21T10:35:32.534 INFO:teuthology.orchestra.run.smithi047.stdout: "active_name": "x", 2024-03-21T10:35:32.534 INFO:teuthology.orchestra.run.smithi047.stdout: "active_addrs": { 2024-03-21T10:35:32.534 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-03-21T10:35:32.534 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-03-21T10:35:32.534 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-03-21T10:35:32.534 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6800", 2024-03-21T10:35:32.534 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 196172753 2024-03-21T10:35:32.534 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:32.534 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-03-21T10:35:32.535 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-03-21T10:35:32.535 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6801", 2024-03-21T10:35:32.535 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 196172753 2024-03-21T10:35:32.535 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-21T10:35:32.535 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-03-21T10:35:32.535 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:32.535 INFO:teuthology.orchestra.run.smithi047.stdout: "active_addr": "172.21.15.47:6801/196172753", 2024-03-21T10:35:32.535 INFO:teuthology.orchestra.run.smithi047.stdout: "active_change": "2024-03-21T10:25:19.913132+0000", 2024-03-21T10:35:32.535 INFO:teuthology.orchestra.run.smithi047.stdout: "active_mgr_features": 4540701547738038271, 2024-03-21T10:35:32.535 INFO:teuthology.orchestra.run.smithi047.stdout: "available": true, 2024-03-21T10:35:32.535 INFO:teuthology.orchestra.run.smithi047.stdout: "standbys": [ 2024-03-21T10:35:32.535 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-03-21T10:35:32.535 INFO:teuthology.orchestra.run.smithi047.stdout: "gid": 15681, 2024-03-21T10:35:32.535 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "y", 2024-03-21T10:35:32.536 INFO:teuthology.orchestra.run.smithi047.stdout: "mgr_features": 4540701547738038271, 2024-03-21T10:35:32.536 INFO:teuthology.orchestra.run.smithi047.stdout: "available_modules": [ 2024-03-21T10:35:32.536 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-03-21T10:35:32.536 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "alerts", 2024-03-21T10:35:32.536 INFO:teuthology.orchestra.run.smithi047.stdout: "can_run": true, 2024-03-21T10:35:32.536 INFO:teuthology.orchestra.run.smithi047.stdout: "error_string": "", 2024-03-21T10:35:32.536 INFO:teuthology.orchestra.run.smithi047.stdout: "module_options": { 2024-03-21T10:35:32.536 INFO:teuthology.orchestra.run.smithi047.stdout: "interval": { 2024-03-21T10:35:32.536 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "interval", 2024-03-21T10:35:32.536 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "secs", 2024-03-21T10:35:32.536 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:32.536 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T10:35:32.536 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "60", 2024-03-21T10:35:32.536 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:32.536 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:32.537 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:32.537 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "How frequently to reexamine health status", 2024-03-21T10:35:32.537 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:32.537 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:32.537 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:32.537 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:32.537 INFO:teuthology.orchestra.run.smithi047.stdout: "log_level": { 2024-03-21T10:35:32.537 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_level", 2024-03-21T10:35:32.537 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T10:35:32.537 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:32.537 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T10:35:32.537 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-03-21T10:35:32.537 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:32.537 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:32.538 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-03-21T10:35:32.538 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-03-21T10:35:32.538 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-03-21T10:35:32.538 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-03-21T10:35:32.538 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-03-21T10:35:32.538 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-03-21T10:35:32.538 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-03-21T10:35:32.538 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-03-21T10:35:32.538 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T10:35:32.538 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:32.538 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:32.538 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:32.538 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:32.538 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster": { 2024-03-21T10:35:32.539 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster", 2024-03-21T10:35:32.539 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T10:35:32.539 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:32.539 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T10:35:32.539 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-03-21T10:35:32.539 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:32.539 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:32.539 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:32.539 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T10:35:32.539 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:32.539 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:32.539 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:32.539 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:32.539 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster_level": { 2024-03-21T10:35:32.539 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster_level", 2024-03-21T10:35:32.540 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T10:35:32.540 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:32.540 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T10:35:32.540 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "info", 2024-03-21T10:35:32.540 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:32.540 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:32.540 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-03-21T10:35:32.540 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-03-21T10:35:32.540 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-03-21T10:35:32.540 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-03-21T10:35:32.540 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-03-21T10:35:32.540 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-03-21T10:35:32.540 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-03-21T10:35:32.540 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-03-21T10:35:32.540 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T10:35:32.541 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:32.541 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:32.541 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:32.541 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:32.541 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_file": { 2024-03-21T10:35:32.541 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_file", 2024-03-21T10:35:32.541 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T10:35:32.541 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:32.541 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T10:35:32.541 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-03-21T10:35:32.541 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:32.541 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:32.541 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:32.541 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T10:35:32.541 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:32.542 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:32.542 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:32.542 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:32.542 INFO:teuthology.orchestra.run.smithi047.stdout: "smtp_destination": { 2024-03-21T10:35:32.542 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "smtp_destination", 2024-03-21T10:35:32.542 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T10:35:32.542 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:32.542 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T10:35:32.542 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-03-21T10:35:32.542 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:32.542 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:32.542 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:32.542 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Email address to send alerts to", 2024-03-21T10:35:32.542 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:32.542 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:32.543 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:32.543 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:32.543 INFO:teuthology.orchestra.run.smithi047.stdout: "smtp_from_name": { 2024-03-21T10:35:32.543 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "smtp_from_name", 2024-03-21T10:35:32.543 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T10:35:32.543 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:32.543 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T10:35:32.543 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "Ceph", 2024-03-21T10:35:32.543 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:32.543 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:32.543 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:32.543 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Email From: name", 2024-03-21T10:35:32.543 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:32.543 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:32.543 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:32.544 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:32.544 INFO:teuthology.orchestra.run.smithi047.stdout: "smtp_host": { 2024-03-21T10:35:32.544 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "smtp_host", 2024-03-21T10:35:32.544 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T10:35:32.544 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:32.544 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T10:35:32.544 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-03-21T10:35:32.544 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:32.544 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:32.544 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:32.544 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "SMTP server", 2024-03-21T10:35:32.544 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:32.544 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:32.544 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:32.544 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:32.545 INFO:teuthology.orchestra.run.smithi047.stdout: "smtp_password": { 2024-03-21T10:35:32.545 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "smtp_password", 2024-03-21T10:35:32.545 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T10:35:32.545 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:32.545 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T10:35:32.545 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-03-21T10:35:32.545 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:32.545 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:32.545 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:32.545 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Password to authenticate with", 2024-03-21T10:35:32.545 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:32.545 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:32.545 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:32.545 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:32.546 INFO:teuthology.orchestra.run.smithi047.stdout: "smtp_port": { 2024-03-21T10:35:32.546 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "smtp_port", 2024-03-21T10:35:32.546 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "int", 2024-03-21T10:35:32.546 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:32.546 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T10:35:32.546 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "465", 2024-03-21T10:35:32.546 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:32.546 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:32.546 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:32.546 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "SMTP port", 2024-03-21T10:35:32.546 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:32.546 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:32.546 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:32.546 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:32.547 INFO:teuthology.orchestra.run.smithi047.stdout: "smtp_sender": { 2024-03-21T10:35:32.547 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "smtp_sender", 2024-03-21T10:35:32.547 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T10:35:32.547 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:32.547 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T10:35:32.547 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-03-21T10:35:32.547 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:32.547 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:32.547 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:32.547 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "SMTP envelope sender", 2024-03-21T10:35:32.547 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:32.547 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:32.547 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:32.548 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:32.548 INFO:teuthology.orchestra.run.smithi047.stdout: "smtp_ssl": { 2024-03-21T10:35:32.548 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "smtp_ssl", 2024-03-21T10:35:32.548 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T10:35:32.548 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:32.548 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T10:35:32.548 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "True", 2024-03-21T10:35:32.548 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:32.548 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:32.548 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:32.548 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Use SSL to connect to SMTP server", 2024-03-21T10:35:32.548 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:32.548 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:32.548 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:32.548 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:32.549 INFO:teuthology.orchestra.run.smithi047.stdout: "smtp_user": { 2024-03-21T10:35:32.549 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "smtp_user", 2024-03-21T10:35:32.549 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T10:35:32.549 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:32.549 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T10:35:32.549 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-03-21T10:35:32.549 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:32.549 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:32.549 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:32.549 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "User to authenticate as", 2024-03-21T10:35:32.549 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:32.549 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:32.549 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:32.549 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-21T10:35:32.549 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-21T10:35:32.549 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:32.550 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-03-21T10:35:32.550 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "balancer", 2024-03-21T10:35:32.550 INFO:teuthology.orchestra.run.smithi047.stdout: "can_run": true, 2024-03-21T10:35:32.550 INFO:teuthology.orchestra.run.smithi047.stdout: "error_string": "", 2024-03-21T10:35:32.550 INFO:teuthology.orchestra.run.smithi047.stdout: "module_options": { 2024-03-21T10:35:32.550 INFO:teuthology.orchestra.run.smithi047.stdout: "active": { 2024-03-21T10:35:32.550 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "active", 2024-03-21T10:35:32.550 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T10:35:32.550 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:32.550 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T10:35:32.550 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "True", 2024-03-21T10:35:32.551 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:32.551 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:32.551 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:32.551 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "automatically balance PGs across cluster", 2024-03-21T10:35:32.551 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:32.551 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:32.551 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:32.551 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:32.551 INFO:teuthology.orchestra.run.smithi047.stdout: "begin_time": { 2024-03-21T10:35:32.551 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "begin_time", 2024-03-21T10:35:32.551 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T10:35:32.551 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:32.551 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T10:35:32.552 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "0000", 2024-03-21T10:35:32.552 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:32.552 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:32.552 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:32.552 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "beginning time of day to automatically balance", 2024-03-21T10:35:32.552 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-03-21T10:35:32.552 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:32.552 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:32.552 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:32.552 INFO:teuthology.orchestra.run.smithi047.stdout: "begin_weekday": { 2024-03-21T10:35:32.552 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "begin_weekday", 2024-03-21T10:35:32.552 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "uint", 2024-03-21T10:35:32.552 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:32.552 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T10:35:32.553 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "0", 2024-03-21T10:35:32.553 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "0", 2024-03-21T10:35:32.553 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "6", 2024-03-21T10:35:32.553 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:32.553 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-03-21T10:35:32.553 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-03-21T10:35:32.553 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:32.553 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:32.553 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:32.553 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_compat_max_iterations": { 2024-03-21T10:35:32.554 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "crush_compat_max_iterations", 2024-03-21T10:35:32.554 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "uint", 2024-03-21T10:35:32.554 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:32.554 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T10:35:32.554 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "25", 2024-03-21T10:35:32.554 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "1", 2024-03-21T10:35:32.554 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "250", 2024-03-21T10:35:32.554 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:32.554 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-03-21T10:35:32.554 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:32.554 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:32.554 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:32.554 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:32.555 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_compat_metrics": { 2024-03-21T10:35:32.555 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "crush_compat_metrics", 2024-03-21T10:35:32.555 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T10:35:32.555 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:32.555 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T10:35:32.555 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "pgs,objects,bytes", 2024-03-21T10:35:32.555 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:32.555 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:32.555 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:32.555 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-03-21T10:35:32.555 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-03-21T10:35:32.555 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:32.555 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:32.555 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:32.556 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_compat_step": { 2024-03-21T10:35:32.556 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "crush_compat_step", 2024-03-21T10:35:32.556 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "float", 2024-03-21T10:35:32.556 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:32.556 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T10:35:32.556 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "0.5", 2024-03-21T10:35:32.556 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "0.001", 2024-03-21T10:35:32.556 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "0.999", 2024-03-21T10:35:32.556 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:32.556 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "aggressiveness of optimization", 2024-03-21T10:35:32.556 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-03-21T10:35:32.556 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:32.556 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:32.556 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:32.557 INFO:teuthology.orchestra.run.smithi047.stdout: "end_time": { 2024-03-21T10:35:32.557 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "end_time", 2024-03-21T10:35:32.557 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T10:35:32.557 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:32.557 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T10:35:32.557 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "2359", 2024-03-21T10:35:32.557 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:32.557 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:32.557 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:32.557 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "ending time of day to automatically balance", 2024-03-21T10:35:32.557 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-03-21T10:35:32.557 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:32.557 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:32.558 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:32.558 INFO:teuthology.orchestra.run.smithi047.stdout: "end_weekday": { 2024-03-21T10:35:32.558 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "end_weekday", 2024-03-21T10:35:32.558 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "uint", 2024-03-21T10:35:32.558 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:32.558 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T10:35:32.558 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "0", 2024-03-21T10:35:32.558 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "0", 2024-03-21T10:35:32.558 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "6", 2024-03-21T10:35:32.558 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:32.558 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-03-21T10:35:32.558 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-03-21T10:35:32.558 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:32.558 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:32.559 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:32.559 INFO:teuthology.orchestra.run.smithi047.stdout: "log_level": { 2024-03-21T10:35:32.559 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_level", 2024-03-21T10:35:32.559 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T10:35:32.559 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:32.559 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T10:35:32.559 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-03-21T10:35:32.559 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:32.559 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:32.559 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-03-21T10:35:32.559 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-03-21T10:35:32.559 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-03-21T10:35:32.559 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-03-21T10:35:32.559 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-03-21T10:35:32.559 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-03-21T10:35:32.560 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-03-21T10:35:32.560 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-03-21T10:35:32.560 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T10:35:32.560 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:32.560 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:32.560 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:32.560 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:32.560 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster": { 2024-03-21T10:35:32.560 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster", 2024-03-21T10:35:32.560 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T10:35:32.560 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:32.560 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T10:35:32.560 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-03-21T10:35:32.561 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:32.561 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:32.561 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:32.561 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T10:35:32.561 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:32.561 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:32.561 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:32.561 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:32.561 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster_level": { 2024-03-21T10:35:32.561 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster_level", 2024-03-21T10:35:32.561 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T10:35:32.561 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:32.562 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T10:35:32.562 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "info", 2024-03-21T10:35:32.562 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:32.562 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:32.562 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-03-21T10:35:32.562 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-03-21T10:35:32.562 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-03-21T10:35:32.562 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-03-21T10:35:32.562 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-03-21T10:35:32.562 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-03-21T10:35:32.562 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-03-21T10:35:32.562 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-03-21T10:35:32.562 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T10:35:32.562 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:32.563 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:32.563 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:32.563 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:32.563 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_file": { 2024-03-21T10:35:32.563 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_file", 2024-03-21T10:35:32.563 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T10:35:32.563 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:32.563 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T10:35:32.563 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-03-21T10:35:32.563 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:32.563 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:32.563 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:32.564 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T10:35:32.564 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:32.564 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:32.564 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:32.564 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:32.564 INFO:teuthology.orchestra.run.smithi047.stdout: "min_score": { 2024-03-21T10:35:32.564 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "min_score", 2024-03-21T10:35:32.564 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "float", 2024-03-21T10:35:32.564 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:32.564 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T10:35:32.564 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "0", 2024-03-21T10:35:32.564 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:32.564 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:32.564 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:32.564 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-03-21T10:35:32.565 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:32.565 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:32.565 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:32.565 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:32.565 INFO:teuthology.orchestra.run.smithi047.stdout: "mode": { 2024-03-21T10:35:32.565 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "mode", 2024-03-21T10:35:32.565 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T10:35:32.565 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:32.565 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T10:35:32.565 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "upmap", 2024-03-21T10:35:32.565 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:32.565 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:32.565 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-03-21T10:35:32.566 INFO:teuthology.orchestra.run.smithi047.stdout: "crush-compat", 2024-03-21T10:35:32.566 INFO:teuthology.orchestra.run.smithi047.stdout: "none", 2024-03-21T10:35:32.566 INFO:teuthology.orchestra.run.smithi047.stdout: "read", 2024-03-21T10:35:32.566 INFO:teuthology.orchestra.run.smithi047.stdout: "upmap", 2024-03-21T10:35:32.566 INFO:teuthology.orchestra.run.smithi047.stdout: "upmap-read" 2024-03-21T10:35:32.566 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-03-21T10:35:32.566 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Balancer mode", 2024-03-21T10:35:32.566 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:32.566 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:32.566 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:32.566 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:32.566 INFO:teuthology.orchestra.run.smithi047.stdout: "pool_ids": { 2024-03-21T10:35:32.567 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "pool_ids", 2024-03-21T10:35:32.567 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T10:35:32.567 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:32.567 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T10:35:32.567 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-03-21T10:35:32.567 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:32.567 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:32.567 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:32.567 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-03-21T10:35:32.567 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:32.567 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:32.567 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:32.567 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:32.567 INFO:teuthology.orchestra.run.smithi047.stdout: "sleep_interval": { 2024-03-21T10:35:32.568 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "sleep_interval", 2024-03-21T10:35:32.568 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "secs", 2024-03-21T10:35:32.568 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:32.568 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T10:35:32.568 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "60", 2024-03-21T10:35:32.568 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:32.568 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:32.568 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:32.568 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-03-21T10:35:32.568 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:32.568 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:32.568 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:32.568 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:32.569 INFO:teuthology.orchestra.run.smithi047.stdout: "upmap_max_deviation": { 2024-03-21T10:35:32.569 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "upmap_max_deviation", 2024-03-21T10:35:32.569 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "int", 2024-03-21T10:35:32.569 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:32.569 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T10:35:32.569 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "5", 2024-03-21T10:35:32.569 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "1", 2024-03-21T10:35:32.569 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:32.569 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:32.569 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "deviation below which no optimization is attempted", 2024-03-21T10:35:32.569 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-03-21T10:35:32.569 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:32.569 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:32.569 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:32.569 INFO:teuthology.orchestra.run.smithi047.stdout: "upmap_max_optimizations": { 2024-03-21T10:35:32.569 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "upmap_max_optimizations", 2024-03-21T10:35:32.570 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "uint", 2024-03-21T10:35:32.570 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:32.570 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T10:35:32.570 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "10", 2024-03-21T10:35:32.570 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:32.570 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:32.570 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:32.570 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-03-21T10:35:32.570 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:32.570 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:32.570 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:32.570 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-21T10:35:32.570 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-21T10:35:32.570 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:32.571 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-03-21T10:35:32.571 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "cephadm", 2024-03-21T10:35:32.571 INFO:teuthology.orchestra.run.smithi047.stdout: "can_run": true, 2024-03-21T10:35:32.571 INFO:teuthology.orchestra.run.smithi047.stdout: "error_string": "", 2024-03-21T10:35:32.571 INFO:teuthology.orchestra.run.smithi047.stdout: "module_options": { 2024-03-21T10:35:32.571 INFO:teuthology.orchestra.run.smithi047.stdout: "agent_down_multiplier": { 2024-03-21T10:35:32.571 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "agent_down_multiplier", 2024-03-21T10:35:32.571 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "float", 2024-03-21T10:35:32.571 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:32.571 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T10:35:32.571 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "3.0", 2024-03-21T10:35:32.571 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:32.572 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:32.572 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:32.572 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-03-21T10:35:32.572 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:32.572 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:32.572 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:32.572 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:32.572 INFO:teuthology.orchestra.run.smithi047.stdout: "agent_refresh_rate": { 2024-03-21T10:35:32.572 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "agent_refresh_rate", 2024-03-21T10:35:32.572 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "secs", 2024-03-21T10:35:32.572 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:32.572 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T10:35:32.572 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "20", 2024-03-21T10:35:32.572 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:32.572 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:32.572 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:32.573 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-03-21T10:35:32.573 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:32.573 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:32.573 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:32.573 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:32.573 INFO:teuthology.orchestra.run.smithi047.stdout: "agent_starting_port": { 2024-03-21T10:35:32.573 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "agent_starting_port", 2024-03-21T10:35:32.573 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "int", 2024-03-21T10:35:32.573 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:32.573 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T10:35:32.573 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "4721", 2024-03-21T10:35:32.573 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:32.573 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:32.573 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:32.573 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-03-21T10:35:32.574 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:32.574 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:32.574 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:32.574 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:32.574 INFO:teuthology.orchestra.run.smithi047.stdout: "allow_ptrace": { 2024-03-21T10:35:32.574 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "allow_ptrace", 2024-03-21T10:35:32.574 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T10:35:32.574 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:32.574 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T10:35:32.574 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-03-21T10:35:32.574 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:32.574 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:32.574 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:32.575 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-03-21T10:35:32.575 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-03-21T10:35:32.575 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:32.575 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:32.575 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:32.575 INFO:teuthology.orchestra.run.smithi047.stdout: "autotune_interval": { 2024-03-21T10:35:32.575 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "autotune_interval", 2024-03-21T10:35:32.575 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "secs", 2024-03-21T10:35:32.575 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:32.575 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T10:35:32.575 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "600", 2024-03-21T10:35:32.575 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:32.575 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:32.575 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:32.576 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "how frequently to autotune daemon memory", 2024-03-21T10:35:32.576 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:32.576 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:32.576 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:32.576 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:32.576 INFO:teuthology.orchestra.run.smithi047.stdout: "autotune_memory_target_ratio": { 2024-03-21T10:35:32.576 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "autotune_memory_target_ratio", 2024-03-21T10:35:32.576 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "float", 2024-03-21T10:35:32.576 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:32.576 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T10:35:32.576 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "0.7", 2024-03-21T10:35:32.576 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:32.576 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:32.576 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:32.576 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-03-21T10:35:32.576 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:32.576 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:32.577 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:32.577 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:32.577 INFO:teuthology.orchestra.run.smithi047.stdout: "cephadm_log_destination": { 2024-03-21T10:35:32.577 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "cephadm_log_destination", 2024-03-21T10:35:32.577 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T10:35:32.577 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:32.577 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T10:35:32.577 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-03-21T10:35:32.577 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:32.577 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:32.577 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-03-21T10:35:32.577 INFO:teuthology.orchestra.run.smithi047.stdout: "file", 2024-03-21T10:35:32.577 INFO:teuthology.orchestra.run.smithi047.stdout: "file,syslog", 2024-03-21T10:35:32.577 INFO:teuthology.orchestra.run.smithi047.stdout: "syslog" 2024-03-21T10:35:32.577 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-03-21T10:35:32.577 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-03-21T10:35:32.578 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:32.578 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:32.578 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:32.578 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:32.578 INFO:teuthology.orchestra.run.smithi047.stdout: "cgroups_split": { 2024-03-21T10:35:32.578 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "cgroups_split", 2024-03-21T10:35:32.578 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T10:35:32.578 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:32.578 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T10:35:32.578 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "True", 2024-03-21T10:35:32.578 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:32.578 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:32.578 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:32.578 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-03-21T10:35:32.579 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:32.579 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:32.579 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:32.579 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:32.579 INFO:teuthology.orchestra.run.smithi047.stdout: "config_checks_enabled": { 2024-03-21T10:35:32.579 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "config_checks_enabled", 2024-03-21T10:35:32.579 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T10:35:32.579 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:32.579 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T10:35:32.579 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-03-21T10:35:32.579 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:32.579 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:32.579 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:32.579 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-03-21T10:35:32.579 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:32.579 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:32.579 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:32.580 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:32.580 INFO:teuthology.orchestra.run.smithi047.stdout: "config_dashboard": { 2024-03-21T10:35:32.580 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "config_dashboard", 2024-03-21T10:35:32.580 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T10:35:32.580 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:32.580 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T10:35:32.580 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "True", 2024-03-21T10:35:32.580 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:32.580 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:32.580 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:32.580 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-03-21T10:35:32.580 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:32.580 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:32.580 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:32.580 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:32.581 INFO:teuthology.orchestra.run.smithi047.stdout: "container_image_alertmanager": { 2024-03-21T10:35:32.581 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "container_image_alertmanager", 2024-03-21T10:35:32.581 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T10:35:32.581 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:32.581 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T10:35:32.581 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-03-21T10:35:32.581 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:32.581 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:32.581 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:32.581 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Prometheus container image", 2024-03-21T10:35:32.581 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:32.581 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:32.581 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:32.581 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:32.582 INFO:teuthology.orchestra.run.smithi047.stdout: "container_image_base": { 2024-03-21T10:35:32.582 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "container_image_base", 2024-03-21T10:35:32.582 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T10:35:32.582 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:32.582 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T10:35:32.582 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "quay.io/ceph/ceph", 2024-03-21T10:35:32.582 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:32.582 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:32.582 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:32.582 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Container image name, without the tag", 2024-03-21T10:35:32.582 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:32.582 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:32.582 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:32.582 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:32.582 INFO:teuthology.orchestra.run.smithi047.stdout: "container_image_elasticsearch": { 2024-03-21T10:35:32.583 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "container_image_elasticsearch", 2024-03-21T10:35:32.583 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T10:35:32.583 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:32.583 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T10:35:32.583 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-03-21T10:35:32.583 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:32.583 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:32.583 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:32.583 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "elasticsearch container image", 2024-03-21T10:35:32.583 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:32.583 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:32.583 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:32.583 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:32.583 INFO:teuthology.orchestra.run.smithi047.stdout: "container_image_grafana": { 2024-03-21T10:35:32.583 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "container_image_grafana", 2024-03-21T10:35:32.583 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T10:35:32.584 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:32.584 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T10:35:32.584 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "quay.io/ceph/grafana:9.4.12", 2024-03-21T10:35:32.584 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:32.584 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:32.584 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:32.584 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Prometheus container image", 2024-03-21T10:35:32.584 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:32.584 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:32.584 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:32.584 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:32.584 INFO:teuthology.orchestra.run.smithi047.stdout: "container_image_haproxy": { 2024-03-21T10:35:32.584 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "container_image_haproxy", 2024-03-21T10:35:32.584 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T10:35:32.584 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:32.585 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T10:35:32.585 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-03-21T10:35:32.585 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:32.585 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:32.585 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:32.585 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "HAproxy container image", 2024-03-21T10:35:32.585 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:32.585 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:32.585 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:32.585 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:32.585 INFO:teuthology.orchestra.run.smithi047.stdout: "container_image_jaeger_agent": { 2024-03-21T10:35:32.585 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "container_image_jaeger_agent", 2024-03-21T10:35:32.585 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T10:35:32.586 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:32.586 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T10:35:32.586 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-03-21T10:35:32.586 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:32.586 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:32.586 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:32.586 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Jaeger agent container image", 2024-03-21T10:35:32.586 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:32.586 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:32.586 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:32.586 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:32.586 INFO:teuthology.orchestra.run.smithi047.stdout: "container_image_jaeger_collector": { 2024-03-21T10:35:32.586 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "container_image_jaeger_collector", 2024-03-21T10:35:32.586 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T10:35:32.586 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:32.587 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T10:35:32.587 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-03-21T10:35:32.587 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:32.587 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:32.587 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:32.587 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Jaeger collector container image", 2024-03-21T10:35:32.587 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:32.587 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:32.587 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:32.587 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:32.587 INFO:teuthology.orchestra.run.smithi047.stdout: "container_image_jaeger_query": { 2024-03-21T10:35:32.587 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "container_image_jaeger_query", 2024-03-21T10:35:32.587 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T10:35:32.587 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:32.587 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T10:35:32.587 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-03-21T10:35:32.587 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:32.588 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:32.588 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:32.588 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Jaeger query container image", 2024-03-21T10:35:32.588 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:32.588 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:32.588 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:32.588 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:32.588 INFO:teuthology.orchestra.run.smithi047.stdout: "container_image_keepalived": { 2024-03-21T10:35:32.588 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "container_image_keepalived", 2024-03-21T10:35:32.588 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T10:35:32.588 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:32.588 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T10:35:32.588 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-03-21T10:35:32.588 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:32.589 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:32.589 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:32.589 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Keepalived container image", 2024-03-21T10:35:32.589 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:32.589 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:32.589 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:32.589 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:32.589 INFO:teuthology.orchestra.run.smithi047.stdout: "container_image_loki": { 2024-03-21T10:35:32.589 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "container_image_loki", 2024-03-21T10:35:32.589 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T10:35:32.589 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:32.589 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T10:35:32.589 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2024-03-21T10:35:32.590 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:32.590 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:32.590 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:32.590 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Loki container image", 2024-03-21T10:35:32.590 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:32.590 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:32.590 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:32.590 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:32.590 INFO:teuthology.orchestra.run.smithi047.stdout: "container_image_node_exporter": { 2024-03-21T10:35:32.590 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "container_image_node_exporter", 2024-03-21T10:35:32.590 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T10:35:32.590 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:32.591 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T10:35:32.591 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-03-21T10:35:32.591 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:32.591 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:32.591 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:32.591 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Prometheus container image", 2024-03-21T10:35:32.591 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:32.591 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:32.591 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:32.591 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:32.591 INFO:teuthology.orchestra.run.smithi047.stdout: "container_image_nvmeof": { 2024-03-21T10:35:32.591 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "container_image_nvmeof", 2024-03-21T10:35:32.591 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T10:35:32.591 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:32.591 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T10:35:32.591 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "quay.io/ceph/nvmeof:1.0.0", 2024-03-21T10:35:32.592 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:32.592 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:32.592 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:32.592 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Nvme-of container image", 2024-03-21T10:35:32.592 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:32.592 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:32.592 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:32.592 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:32.592 INFO:teuthology.orchestra.run.smithi047.stdout: "container_image_prometheus": { 2024-03-21T10:35:32.592 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "container_image_prometheus", 2024-03-21T10:35:32.592 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T10:35:32.592 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:32.592 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T10:35:32.592 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-03-21T10:35:32.592 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:32.593 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:32.593 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:32.593 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Prometheus container image", 2024-03-21T10:35:32.593 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:32.593 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:32.593 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:32.593 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:32.593 INFO:teuthology.orchestra.run.smithi047.stdout: "container_image_promtail": { 2024-03-21T10:35:32.593 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "container_image_promtail", 2024-03-21T10:35:32.593 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T10:35:32.593 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:32.593 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T10:35:32.593 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2024-03-21T10:35:32.593 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:32.593 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:32.593 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:32.594 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Promtail container image", 2024-03-21T10:35:32.594 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:32.594 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:32.594 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:32.594 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:32.594 INFO:teuthology.orchestra.run.smithi047.stdout: "container_image_snmp_gateway": { 2024-03-21T10:35:32.594 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "container_image_snmp_gateway", 2024-03-21T10:35:32.594 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T10:35:32.594 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:32.594 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T10:35:32.594 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-03-21T10:35:32.594 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:32.594 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:32.594 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:32.594 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "SNMP Gateway container image", 2024-03-21T10:35:32.595 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:32.595 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:32.595 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:32.595 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:32.595 INFO:teuthology.orchestra.run.smithi047.stdout: "container_init": { 2024-03-21T10:35:32.595 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "container_init", 2024-03-21T10:35:32.595 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T10:35:32.595 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:32.595 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T10:35:32.595 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "True", 2024-03-21T10:35:32.595 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:32.595 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:32.595 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:32.595 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Run podman/docker with `--init`", 2024-03-21T10:35:32.596 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:32.596 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:32.596 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:32.596 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:32.596 INFO:teuthology.orchestra.run.smithi047.stdout: "daemon_cache_timeout": { 2024-03-21T10:35:32.596 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "daemon_cache_timeout", 2024-03-21T10:35:32.596 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "secs", 2024-03-21T10:35:32.596 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:32.596 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T10:35:32.596 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "600", 2024-03-21T10:35:32.596 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:32.596 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:32.596 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:32.596 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-03-21T10:35:32.596 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:32.597 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:32.597 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:32.597 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:32.597 INFO:teuthology.orchestra.run.smithi047.stdout: "default_cephadm_command_timeout": { 2024-03-21T10:35:32.597 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "default_cephadm_command_timeout", 2024-03-21T10:35:32.597 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "int", 2024-03-21T10:35:32.597 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:32.597 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T10:35:32.597 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "900", 2024-03-21T10:35:32.597 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:32.597 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:32.597 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:32.597 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-03-21T10:35:32.597 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:32.597 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:32.598 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:32.598 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:32.598 INFO:teuthology.orchestra.run.smithi047.stdout: "default_registry": { 2024-03-21T10:35:32.598 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "default_registry", 2024-03-21T10:35:32.598 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T10:35:32.598 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:32.598 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T10:35:32.598 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "docker.io", 2024-03-21T10:35:32.598 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:32.598 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:32.598 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:32.598 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-03-21T10:35:32.598 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:32.598 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:32.598 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:32.599 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:32.599 INFO:teuthology.orchestra.run.smithi047.stdout: "device_cache_timeout": { 2024-03-21T10:35:32.599 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "device_cache_timeout", 2024-03-21T10:35:32.599 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "secs", 2024-03-21T10:35:32.599 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:32.599 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T10:35:32.599 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "1800", 2024-03-21T10:35:32.599 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:32.599 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:32.599 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:32.599 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "seconds to cache device inventory", 2024-03-21T10:35:32.599 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:32.599 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:32.599 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:32.600 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:32.600 INFO:teuthology.orchestra.run.smithi047.stdout: "device_enhanced_scan": { 2024-03-21T10:35:32.600 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "device_enhanced_scan", 2024-03-21T10:35:32.600 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T10:35:32.600 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:32.600 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T10:35:32.600 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-03-21T10:35:32.600 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:32.600 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:32.600 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:32.600 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Use libstoragemgmt during device scans", 2024-03-21T10:35:32.600 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:32.600 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:32.600 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:32.601 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:32.601 INFO:teuthology.orchestra.run.smithi047.stdout: "facts_cache_timeout": { 2024-03-21T10:35:32.601 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "facts_cache_timeout", 2024-03-21T10:35:32.601 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "secs", 2024-03-21T10:35:32.601 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:32.601 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T10:35:32.601 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "60", 2024-03-21T10:35:32.601 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:32.601 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:32.601 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:32.601 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "seconds to cache host facts data", 2024-03-21T10:35:32.601 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:32.601 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:32.601 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:32.601 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:32.602 INFO:teuthology.orchestra.run.smithi047.stdout: "grafana_dashboards_path": { 2024-03-21T10:35:32.602 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "grafana_dashboards_path", 2024-03-21T10:35:32.602 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T10:35:32.602 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:32.602 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T10:35:32.602 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-03-21T10:35:32.602 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:32.602 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:32.602 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:32.602 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-03-21T10:35:32.602 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:32.602 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:32.602 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:32.603 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:32.603 INFO:teuthology.orchestra.run.smithi047.stdout: "host_check_interval": { 2024-03-21T10:35:32.603 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "host_check_interval", 2024-03-21T10:35:32.603 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "secs", 2024-03-21T10:35:32.603 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:32.603 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T10:35:32.603 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "600", 2024-03-21T10:35:32.603 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:32.603 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:32.603 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:32.603 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "how frequently to perform a host check", 2024-03-21T10:35:32.603 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:32.603 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:32.603 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:32.603 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:32.603 INFO:teuthology.orchestra.run.smithi047.stdout: "hw_monitoring": { 2024-03-21T10:35:32.604 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "hw_monitoring", 2024-03-21T10:35:32.604 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T10:35:32.604 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:32.604 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T10:35:32.604 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-03-21T10:35:32.604 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:32.604 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:32.604 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:32.604 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-03-21T10:35:32.604 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:32.604 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:32.604 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:32.604 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:32.604 INFO:teuthology.orchestra.run.smithi047.stdout: "inventory_list_all": { 2024-03-21T10:35:32.604 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "inventory_list_all", 2024-03-21T10:35:32.604 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T10:35:32.604 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:32.605 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T10:35:32.605 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-03-21T10:35:32.605 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:32.605 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:32.605 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:32.605 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-03-21T10:35:32.605 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:32.605 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:32.605 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:32.605 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:32.605 INFO:teuthology.orchestra.run.smithi047.stdout: "log_level": { 2024-03-21T10:35:32.605 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_level", 2024-03-21T10:35:32.605 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T10:35:32.605 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:32.605 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T10:35:32.605 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-03-21T10:35:32.605 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:32.606 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:32.606 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-03-21T10:35:32.606 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-03-21T10:35:32.606 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-03-21T10:35:32.606 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-03-21T10:35:32.606 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-03-21T10:35:32.606 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-03-21T10:35:32.606 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-03-21T10:35:32.606 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-03-21T10:35:32.606 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T10:35:32.606 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:32.606 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:32.606 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:32.606 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:32.607 INFO:teuthology.orchestra.run.smithi047.stdout: "log_refresh_metadata": { 2024-03-21T10:35:32.607 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_refresh_metadata", 2024-03-21T10:35:32.607 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T10:35:32.607 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:32.607 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T10:35:32.607 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-03-21T10:35:32.607 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:32.607 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:32.607 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:32.607 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-03-21T10:35:32.607 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:32.607 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:32.607 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:32.607 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:32.607 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster": { 2024-03-21T10:35:32.608 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster", 2024-03-21T10:35:32.608 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T10:35:32.608 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:32.608 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T10:35:32.608 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "True", 2024-03-21T10:35:32.608 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:32.608 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:32.608 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:32.608 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-03-21T10:35:32.608 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:32.608 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:32.608 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:32.608 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:32.608 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster_level": { 2024-03-21T10:35:32.609 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster_level", 2024-03-21T10:35:32.609 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T10:35:32.609 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:32.609 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T10:35:32.609 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "info", 2024-03-21T10:35:32.609 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:32.609 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:32.609 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-03-21T10:35:32.609 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-03-21T10:35:32.609 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-03-21T10:35:32.609 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-03-21T10:35:32.609 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-03-21T10:35:32.609 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-03-21T10:35:32.609 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-03-21T10:35:32.609 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-03-21T10:35:32.609 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T10:35:32.610 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:32.610 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:32.610 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:32.610 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:32.610 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_file": { 2024-03-21T10:35:32.610 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_file", 2024-03-21T10:35:32.610 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T10:35:32.610 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:32.610 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T10:35:32.610 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-03-21T10:35:32.610 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:32.610 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:32.611 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:32.611 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T10:35:32.611 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:32.611 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:32.611 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:32.611 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:32.611 INFO:teuthology.orchestra.run.smithi047.stdout: "manage_etc_ceph_ceph_conf": { 2024-03-21T10:35:32.611 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-03-21T10:35:32.611 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T10:35:32.611 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:32.611 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T10:35:32.611 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-03-21T10:35:32.611 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:32.612 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:32.612 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:32.612 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-03-21T10:35:32.612 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:32.612 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:32.612 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:32.612 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:32.612 INFO:teuthology.orchestra.run.smithi047.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-03-21T10:35:32.612 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-03-21T10:35:32.612 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T10:35:32.612 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:32.612 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T10:35:32.612 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "*", 2024-03-21T10:35:32.613 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:32.613 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:32.613 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:32.613 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-03-21T10:35:32.613 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:32.613 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:32.613 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:32.613 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:32.613 INFO:teuthology.orchestra.run.smithi047.stdout: "max_count_per_host": { 2024-03-21T10:35:32.613 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "max_count_per_host", 2024-03-21T10:35:32.613 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "int", 2024-03-21T10:35:32.613 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:32.613 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T10:35:32.614 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "10", 2024-03-21T10:35:32.614 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:32.614 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:32.614 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:32.614 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "max number of daemons per service per host", 2024-03-21T10:35:32.614 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:32.614 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:32.614 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:32.614 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:32.614 INFO:teuthology.orchestra.run.smithi047.stdout: "max_osd_draining_count": { 2024-03-21T10:35:32.614 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "max_osd_draining_count", 2024-03-21T10:35:32.614 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "int", 2024-03-21T10:35:32.614 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:32.614 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T10:35:32.615 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "10", 2024-03-21T10:35:32.615 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:32.615 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:32.615 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:32.615 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-03-21T10:35:32.615 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:32.615 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:32.615 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:32.615 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:32.615 INFO:teuthology.orchestra.run.smithi047.stdout: "migration_current": { 2024-03-21T10:35:32.615 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "migration_current", 2024-03-21T10:35:32.615 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "int", 2024-03-21T10:35:32.615 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:32.615 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T10:35:32.616 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-03-21T10:35:32.616 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:32.616 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:32.616 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:32.616 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "internal - do not modify", 2024-03-21T10:35:32.616 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:32.616 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:32.616 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:32.616 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:32.616 INFO:teuthology.orchestra.run.smithi047.stdout: "mode": { 2024-03-21T10:35:32.616 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "mode", 2024-03-21T10:35:32.616 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T10:35:32.616 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:32.616 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T10:35:32.617 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "root", 2024-03-21T10:35:32.617 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:32.617 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:32.617 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-03-21T10:35:32.617 INFO:teuthology.orchestra.run.smithi047.stdout: "cephadm-package", 2024-03-21T10:35:32.617 INFO:teuthology.orchestra.run.smithi047.stdout: "root" 2024-03-21T10:35:32.617 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-03-21T10:35:32.617 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "mode for remote execution of cephadm", 2024-03-21T10:35:32.617 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:32.617 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:32.617 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:32.617 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:32.617 INFO:teuthology.orchestra.run.smithi047.stdout: "oob_default_addr": { 2024-03-21T10:35:32.617 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "oob_default_addr", 2024-03-21T10:35:32.617 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T10:35:32.617 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:32.618 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T10:35:32.618 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "169.254.1.1", 2024-03-21T10:35:32.618 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:32.618 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:32.618 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:32.618 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Default address for RedFish API (oob management).", 2024-03-21T10:35:32.618 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:32.618 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:32.618 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:32.618 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:32.618 INFO:teuthology.orchestra.run.smithi047.stdout: "prometheus_alerts_path": { 2024-03-21T10:35:32.618 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "prometheus_alerts_path", 2024-03-21T10:35:32.618 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T10:35:32.618 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:32.619 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T10:35:32.619 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-03-21T10:35:32.619 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:32.619 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:32.619 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:32.619 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-03-21T10:35:32.619 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:32.619 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:32.619 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:32.619 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:32.619 INFO:teuthology.orchestra.run.smithi047.stdout: "registry_insecure": { 2024-03-21T10:35:32.619 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "registry_insecure", 2024-03-21T10:35:32.619 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T10:35:32.619 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:32.620 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T10:35:32.620 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-03-21T10:35:32.620 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:32.620 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:32.620 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:32.620 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-03-21T10:35:32.620 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:32.620 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:32.620 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:32.620 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:32.620 INFO:teuthology.orchestra.run.smithi047.stdout: "registry_password": { 2024-03-21T10:35:32.620 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "registry_password", 2024-03-21T10:35:32.620 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T10:35:32.620 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:32.621 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T10:35:32.621 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-03-21T10:35:32.621 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:32.621 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:32.621 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:32.621 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-03-21T10:35:32.621 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:32.621 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:32.621 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:32.621 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:32.621 INFO:teuthology.orchestra.run.smithi047.stdout: "registry_url": { 2024-03-21T10:35:32.621 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "registry_url", 2024-03-21T10:35:32.621 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T10:35:32.621 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:32.622 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T10:35:32.622 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-03-21T10:35:32.622 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:32.622 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:32.622 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:32.622 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-03-21T10:35:32.622 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:32.622 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:32.622 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:32.622 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:32.622 INFO:teuthology.orchestra.run.smithi047.stdout: "registry_username": { 2024-03-21T10:35:32.622 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "registry_username", 2024-03-21T10:35:32.622 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T10:35:32.623 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:32.623 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T10:35:32.623 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-03-21T10:35:32.623 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:32.623 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:32.623 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:32.623 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-03-21T10:35:32.623 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:32.623 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:32.623 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:32.623 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:32.623 INFO:teuthology.orchestra.run.smithi047.stdout: "secure_monitoring_stack": { 2024-03-21T10:35:32.623 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "secure_monitoring_stack", 2024-03-21T10:35:32.623 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T10:35:32.624 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:32.624 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T10:35:32.624 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-03-21T10:35:32.624 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:32.624 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:32.624 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:32.624 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-03-21T10:35:32.624 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:32.624 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:32.624 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:32.624 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:32.624 INFO:teuthology.orchestra.run.smithi047.stdout: "service_discovery_port": { 2024-03-21T10:35:32.624 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "service_discovery_port", 2024-03-21T10:35:32.624 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "int", 2024-03-21T10:35:32.624 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:32.625 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T10:35:32.625 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "8765", 2024-03-21T10:35:32.625 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:32.625 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:32.625 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:32.625 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "cephadm service discovery port", 2024-03-21T10:35:32.625 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:32.625 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:32.625 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:32.625 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:32.625 INFO:teuthology.orchestra.run.smithi047.stdout: "ssh_config_file": { 2024-03-21T10:35:32.625 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "ssh_config_file", 2024-03-21T10:35:32.625 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T10:35:32.625 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:32.625 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T10:35:32.626 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-03-21T10:35:32.626 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:32.626 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:32.626 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:32.626 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-03-21T10:35:32.626 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:32.626 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:32.626 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:32.626 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:32.626 INFO:teuthology.orchestra.run.smithi047.stdout: "use_agent": { 2024-03-21T10:35:32.626 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "use_agent", 2024-03-21T10:35:32.626 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T10:35:32.626 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:32.626 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T10:35:32.626 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-03-21T10:35:32.627 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:32.627 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:32.627 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:32.627 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-03-21T10:35:32.627 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:32.627 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:32.627 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:32.627 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:32.627 INFO:teuthology.orchestra.run.smithi047.stdout: "use_repo_digest": { 2024-03-21T10:35:32.627 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "use_repo_digest", 2024-03-21T10:35:32.627 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T10:35:32.627 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:32.627 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T10:35:32.627 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "True", 2024-03-21T10:35:32.627 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:32.627 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:32.628 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:32.628 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-03-21T10:35:32.628 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:32.628 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:32.628 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:32.628 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:32.628 INFO:teuthology.orchestra.run.smithi047.stdout: "warn_on_failed_host_check": { 2024-03-21T10:35:32.628 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "warn_on_failed_host_check", 2024-03-21T10:35:32.628 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T10:35:32.628 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:32.628 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T10:35:32.628 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "True", 2024-03-21T10:35:32.628 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:32.628 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:32.628 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:32.628 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "raise a health warning if the host check fails", 2024-03-21T10:35:32.628 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:32.629 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:32.629 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:32.629 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:32.629 INFO:teuthology.orchestra.run.smithi047.stdout: "warn_on_stray_daemons": { 2024-03-21T10:35:32.629 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "warn_on_stray_daemons", 2024-03-21T10:35:32.629 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T10:35:32.629 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:32.629 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T10:35:32.629 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "True", 2024-03-21T10:35:32.629 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:32.629 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:32.629 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:32.629 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-03-21T10:35:32.629 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:32.629 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:32.630 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:32.630 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:32.630 INFO:teuthology.orchestra.run.smithi047.stdout: "warn_on_stray_hosts": { 2024-03-21T10:35:32.630 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "warn_on_stray_hosts", 2024-03-21T10:35:32.630 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T10:35:32.630 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:32.630 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T10:35:32.630 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "True", 2024-03-21T10:35:32.630 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:32.630 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:32.630 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:32.630 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-03-21T10:35:32.630 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:32.630 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:32.631 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:32.631 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-21T10:35:32.631 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-21T10:35:32.631 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:32.631 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-03-21T10:35:32.631 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "crash", 2024-03-21T10:35:32.631 INFO:teuthology.orchestra.run.smithi047.stdout: "can_run": true, 2024-03-21T10:35:32.631 INFO:teuthology.orchestra.run.smithi047.stdout: "error_string": "", 2024-03-21T10:35:32.631 INFO:teuthology.orchestra.run.smithi047.stdout: "module_options": { 2024-03-21T10:35:32.631 INFO:teuthology.orchestra.run.smithi047.stdout: "log_level": { 2024-03-21T10:35:32.631 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_level", 2024-03-21T10:35:32.631 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T10:35:32.631 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:32.631 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T10:35:32.631 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-03-21T10:35:32.632 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:32.632 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:32.632 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-03-21T10:35:32.632 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-03-21T10:35:32.632 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-03-21T10:35:32.632 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-03-21T10:35:32.632 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-03-21T10:35:32.632 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-03-21T10:35:32.632 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-03-21T10:35:32.632 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-03-21T10:35:32.632 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T10:35:32.632 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:32.632 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:32.633 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:32.633 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:32.633 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster": { 2024-03-21T10:35:32.633 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster", 2024-03-21T10:35:32.633 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T10:35:32.633 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:32.633 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T10:35:32.633 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-03-21T10:35:32.633 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:32.633 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:32.633 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:32.633 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T10:35:32.634 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:32.634 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:32.634 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:32.634 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:32.634 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster_level": { 2024-03-21T10:35:32.634 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster_level", 2024-03-21T10:35:32.634 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T10:35:32.634 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:32.634 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T10:35:32.634 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "info", 2024-03-21T10:35:32.635 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:32.635 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:32.635 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-03-21T10:35:32.635 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-03-21T10:35:32.635 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-03-21T10:35:32.635 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-03-21T10:35:32.635 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-03-21T10:35:32.635 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-03-21T10:35:32.635 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-03-21T10:35:32.635 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-03-21T10:35:32.635 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T10:35:32.635 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:32.635 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:32.635 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:32.635 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:32.636 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_file": { 2024-03-21T10:35:32.636 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_file", 2024-03-21T10:35:32.636 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T10:35:32.636 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:32.636 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T10:35:32.636 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-03-21T10:35:32.636 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:32.636 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:32.636 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:32.636 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T10:35:32.636 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:32.636 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:32.636 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:32.637 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:32.637 INFO:teuthology.orchestra.run.smithi047.stdout: "retain_interval": { 2024-03-21T10:35:32.637 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "retain_interval", 2024-03-21T10:35:32.637 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "secs", 2024-03-21T10:35:32.637 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:32.637 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T10:35:32.637 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "31536000", 2024-03-21T10:35:32.637 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:32.637 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:32.637 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:32.637 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "how long to retain crashes before pruning them", 2024-03-21T10:35:32.638 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:32.638 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:32.638 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:32.638 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:32.638 INFO:teuthology.orchestra.run.smithi047.stdout: "warn_recent_interval": { 2024-03-21T10:35:32.638 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "warn_recent_interval", 2024-03-21T10:35:32.638 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "secs", 2024-03-21T10:35:32.638 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:32.638 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T10:35:32.638 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "1209600", 2024-03-21T10:35:32.638 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:32.638 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:32.638 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:32.638 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "time interval in which to warn about recent crashes", 2024-03-21T10:35:32.639 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:32.639 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:32.639 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:32.639 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-21T10:35:32.639 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-21T10:35:32.639 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:32.639 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-03-21T10:35:32.639 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "dashboard", 2024-03-21T10:35:32.639 INFO:teuthology.orchestra.run.smithi047.stdout: "can_run": true, 2024-03-21T10:35:32.639 INFO:teuthology.orchestra.run.smithi047.stdout: "error_string": "", 2024-03-21T10:35:32.639 INFO:teuthology.orchestra.run.smithi047.stdout: "module_options": { 2024-03-21T10:35:32.639 INFO:teuthology.orchestra.run.smithi047.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-03-21T10:35:32.639 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-03-21T10:35:32.639 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "int", 2024-03-21T10:35:32.639 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:32.640 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T10:35:32.640 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "10", 2024-03-21T10:35:32.640 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:32.640 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:32.640 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:32.640 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T10:35:32.640 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:32.640 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:32.640 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:32.640 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:32.640 INFO:teuthology.orchestra.run.smithi047.stdout: "ALERTMANAGER_API_HOST": { 2024-03-21T10:35:32.640 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "ALERTMANAGER_API_HOST", 2024-03-21T10:35:32.640 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T10:35:32.640 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:32.640 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T10:35:32.640 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-03-21T10:35:32.640 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:32.641 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:32.641 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:32.641 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T10:35:32.641 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:32.641 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:32.641 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:32.641 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:32.641 INFO:teuthology.orchestra.run.smithi047.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-03-21T10:35:32.641 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-03-21T10:35:32.641 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T10:35:32.641 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:32.641 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T10:35:32.641 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "True", 2024-03-21T10:35:32.641 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:32.641 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:32.641 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:32.641 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T10:35:32.642 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:32.642 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:32.642 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:32.642 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:32.642 INFO:teuthology.orchestra.run.smithi047.stdout: "AUDIT_API_ENABLED": { 2024-03-21T10:35:32.642 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "AUDIT_API_ENABLED", 2024-03-21T10:35:32.642 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T10:35:32.642 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:32.642 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T10:35:32.642 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-03-21T10:35:32.642 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:32.642 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:32.642 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:32.643 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T10:35:32.643 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:32.643 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:32.643 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:32.643 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:32.643 INFO:teuthology.orchestra.run.smithi047.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-03-21T10:35:32.643 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-03-21T10:35:32.643 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T10:35:32.643 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:32.643 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T10:35:32.643 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "True", 2024-03-21T10:35:32.643 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:32.643 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:32.643 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:32.643 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T10:35:32.644 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:32.644 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:32.644 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:32.644 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:32.644 INFO:teuthology.orchestra.run.smithi047.stdout: "ENABLE_BROWSABLE_API": { 2024-03-21T10:35:32.644 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "ENABLE_BROWSABLE_API", 2024-03-21T10:35:32.644 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T10:35:32.644 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:32.644 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T10:35:32.644 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "True", 2024-03-21T10:35:32.644 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:32.644 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:32.644 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:32.644 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T10:35:32.644 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:32.645 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:32.645 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:32.645 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:32.645 INFO:teuthology.orchestra.run.smithi047.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-03-21T10:35:32.645 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-03-21T10:35:32.645 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T10:35:32.645 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:32.645 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T10:35:32.645 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "True", 2024-03-21T10:35:32.645 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:32.645 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:32.645 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:32.645 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T10:35:32.645 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:32.645 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:32.646 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:32.646 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:32.646 INFO:teuthology.orchestra.run.smithi047.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-03-21T10:35:32.646 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-03-21T10:35:32.646 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T10:35:32.646 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:32.646 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T10:35:32.646 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "True", 2024-03-21T10:35:32.646 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:32.646 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:32.646 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:32.646 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T10:35:32.646 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:32.647 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:32.647 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:32.647 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:32.647 INFO:teuthology.orchestra.run.smithi047.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-03-21T10:35:32.647 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-03-21T10:35:32.647 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T10:35:32.647 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:32.647 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T10:35:32.647 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "True", 2024-03-21T10:35:32.647 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:32.647 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:32.647 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:32.647 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T10:35:32.647 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:32.648 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:32.648 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:32.648 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:32.648 INFO:teuthology.orchestra.run.smithi047.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-03-21T10:35:32.648 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-03-21T10:35:32.648 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T10:35:32.648 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:32.648 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T10:35:32.648 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "True", 2024-03-21T10:35:32.648 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:32.648 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:32.648 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:32.648 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T10:35:32.648 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:32.649 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:32.649 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:32.649 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:32.649 INFO:teuthology.orchestra.run.smithi047.stdout: "FEATURE_TOGGLE_NFS": { 2024-03-21T10:35:32.649 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-03-21T10:35:32.649 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T10:35:32.649 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:32.649 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T10:35:32.649 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "True", 2024-03-21T10:35:32.649 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:32.649 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:32.649 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:32.649 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T10:35:32.650 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:32.650 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:32.650 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:32.650 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:32.650 INFO:teuthology.orchestra.run.smithi047.stdout: "FEATURE_TOGGLE_RBD": { 2024-03-21T10:35:32.650 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-03-21T10:35:32.650 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T10:35:32.650 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:32.650 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T10:35:32.650 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "True", 2024-03-21T10:35:32.650 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:32.650 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:32.650 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:32.651 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T10:35:32.651 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:32.651 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:32.651 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:32.651 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:32.651 INFO:teuthology.orchestra.run.smithi047.stdout: "FEATURE_TOGGLE_RGW": { 2024-03-21T10:35:32.651 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-03-21T10:35:32.651 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T10:35:32.651 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:32.651 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T10:35:32.651 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "True", 2024-03-21T10:35:32.651 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:32.651 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:32.652 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:32.652 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T10:35:32.652 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:32.652 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:32.652 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:32.652 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:32.652 INFO:teuthology.orchestra.run.smithi047.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-03-21T10:35:32.652 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-03-21T10:35:32.652 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T10:35:32.652 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:32.652 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T10:35:32.652 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-03-21T10:35:32.652 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:32.653 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:32.653 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:32.653 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T10:35:32.653 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:32.653 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:32.653 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:32.653 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:32.653 INFO:teuthology.orchestra.run.smithi047.stdout: "GRAFANA_API_PASSWORD": { 2024-03-21T10:35:32.653 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "GRAFANA_API_PASSWORD", 2024-03-21T10:35:32.654 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T10:35:32.654 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:32.654 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T10:35:32.654 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "admin", 2024-03-21T10:35:32.654 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:32.654 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:32.654 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:32.654 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T10:35:32.654 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:32.654 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:32.654 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:32.654 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:32.655 INFO:teuthology.orchestra.run.smithi047.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-03-21T10:35:32.655 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-03-21T10:35:32.655 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T10:35:32.655 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:32.655 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T10:35:32.655 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "True", 2024-03-21T10:35:32.655 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:32.655 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:32.655 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:32.655 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T10:35:32.655 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:32.655 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:32.655 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:32.655 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:32.656 INFO:teuthology.orchestra.run.smithi047.stdout: "GRAFANA_API_URL": { 2024-03-21T10:35:32.656 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "GRAFANA_API_URL", 2024-03-21T10:35:32.656 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T10:35:32.656 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:32.656 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T10:35:32.656 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-03-21T10:35:32.656 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:32.656 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:32.656 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:32.656 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T10:35:32.656 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:32.656 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:32.657 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:32.657 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:32.657 INFO:teuthology.orchestra.run.smithi047.stdout: "GRAFANA_API_USERNAME": { 2024-03-21T10:35:32.657 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "GRAFANA_API_USERNAME", 2024-03-21T10:35:32.657 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T10:35:32.657 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:32.657 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T10:35:32.657 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "admin", 2024-03-21T10:35:32.657 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:32.657 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:32.657 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:32.657 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T10:35:32.658 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:32.658 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:32.658 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:32.658 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:32.658 INFO:teuthology.orchestra.run.smithi047.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-03-21T10:35:32.658 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-03-21T10:35:32.658 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T10:35:32.658 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:32.658 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T10:35:32.658 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-03-21T10:35:32.658 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:32.658 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:32.658 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:32.659 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T10:35:32.659 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:32.659 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:32.659 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:32.659 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:32.659 INFO:teuthology.orchestra.run.smithi047.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-03-21T10:35:32.659 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-03-21T10:35:32.659 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T10:35:32.659 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:32.659 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T10:35:32.659 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-03-21T10:35:32.659 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:32.659 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:32.659 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:32.660 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T10:35:32.660 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:32.660 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:32.660 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:32.660 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:32.660 INFO:teuthology.orchestra.run.smithi047.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-03-21T10:35:32.660 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-03-21T10:35:32.660 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T10:35:32.660 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:32.660 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T10:35:32.660 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "True", 2024-03-21T10:35:32.660 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:32.660 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:32.660 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:32.661 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T10:35:32.661 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:32.661 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:32.661 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:32.661 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:32.661 INFO:teuthology.orchestra.run.smithi047.stdout: "ISSUE_TRACKER_API_KEY": { 2024-03-21T10:35:32.661 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-03-21T10:35:32.661 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T10:35:32.661 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:32.661 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T10:35:32.661 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-03-21T10:35:32.661 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:32.661 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:32.661 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:32.662 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T10:35:32.662 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:32.662 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:32.662 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:32.662 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:32.662 INFO:teuthology.orchestra.run.smithi047.stdout: "MULTICLUSTER_CONFIG": { 2024-03-21T10:35:32.662 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "MULTICLUSTER_CONFIG", 2024-03-21T10:35:32.662 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T10:35:32.662 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:32.662 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T10:35:32.662 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "{}", 2024-03-21T10:35:32.662 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:32.662 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:32.663 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:32.663 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T10:35:32.663 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:32.663 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:32.663 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:32.663 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:32.663 INFO:teuthology.orchestra.run.smithi047.stdout: "PROMETHEUS_API_HOST": { 2024-03-21T10:35:32.663 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "PROMETHEUS_API_HOST", 2024-03-21T10:35:32.663 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T10:35:32.663 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:32.663 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T10:35:32.663 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-03-21T10:35:32.663 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:32.664 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:32.664 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:32.664 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T10:35:32.664 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:32.664 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:32.664 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:32.664 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:32.664 INFO:teuthology.orchestra.run.smithi047.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-03-21T10:35:32.664 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-03-21T10:35:32.664 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T10:35:32.664 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:32.664 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T10:35:32.664 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "True", 2024-03-21T10:35:32.664 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:32.664 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:32.665 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:32.665 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T10:35:32.665 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:32.665 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:32.665 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:32.665 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:32.665 INFO:teuthology.orchestra.run.smithi047.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-03-21T10:35:32.665 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-03-21T10:35:32.665 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T10:35:32.665 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:32.665 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T10:35:32.665 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-03-21T10:35:32.665 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:32.665 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:32.665 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:32.666 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T10:35:32.666 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:32.666 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:32.666 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:32.666 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:32.666 INFO:teuthology.orchestra.run.smithi047.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-03-21T10:35:32.666 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-03-21T10:35:32.666 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T10:35:32.666 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:32.666 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T10:35:32.666 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-03-21T10:35:32.666 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:32.666 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:32.667 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:32.667 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T10:35:32.667 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:32.667 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:32.667 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:32.667 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:32.667 INFO:teuthology.orchestra.run.smithi047.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-03-21T10:35:32.667 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-03-21T10:35:32.667 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T10:35:32.667 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:32.667 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T10:35:32.667 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "True", 2024-03-21T10:35:32.667 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:32.667 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:32.667 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:32.667 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T10:35:32.668 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:32.668 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:32.668 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:32.668 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:32.668 INFO:teuthology.orchestra.run.smithi047.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-03-21T10:35:32.668 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-03-21T10:35:32.668 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T10:35:32.668 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:32.668 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T10:35:32.668 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "True", 2024-03-21T10:35:32.668 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:32.668 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:32.668 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:32.668 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T10:35:32.669 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:32.669 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:32.669 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:32.669 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:32.669 INFO:teuthology.orchestra.run.smithi047.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-03-21T10:35:32.669 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-03-21T10:35:32.669 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T10:35:32.669 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:32.669 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T10:35:32.669 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-03-21T10:35:32.669 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:32.669 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:32.669 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:32.669 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T10:35:32.669 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:32.670 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:32.670 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:32.670 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:32.670 INFO:teuthology.orchestra.run.smithi047.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-03-21T10:35:32.670 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-03-21T10:35:32.670 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T10:35:32.670 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:32.670 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T10:35:32.670 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-03-21T10:35:32.670 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:32.670 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:32.670 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:32.670 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T10:35:32.670 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:32.670 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:32.671 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:32.671 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:32.671 INFO:teuthology.orchestra.run.smithi047.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-03-21T10:35:32.671 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-03-21T10:35:32.671 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T10:35:32.671 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:32.671 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T10:35:32.671 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-03-21T10:35:32.671 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:32.671 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:32.671 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:32.671 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T10:35:32.671 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:32.671 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:32.671 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:32.672 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:32.672 INFO:teuthology.orchestra.run.smithi047.stdout: "PWD_POLICY_ENABLED": { 2024-03-21T10:35:32.672 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "PWD_POLICY_ENABLED", 2024-03-21T10:35:32.672 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T10:35:32.672 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:32.672 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T10:35:32.672 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "True", 2024-03-21T10:35:32.672 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:32.672 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:32.672 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:32.672 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T10:35:32.672 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:32.672 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:32.672 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:32.672 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:32.672 INFO:teuthology.orchestra.run.smithi047.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-03-21T10:35:32.673 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-03-21T10:35:32.673 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T10:35:32.673 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:32.673 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T10:35:32.673 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-03-21T10:35:32.673 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:32.673 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:32.673 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:32.673 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T10:35:32.674 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:32.674 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:32.674 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:32.674 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:32.674 INFO:teuthology.orchestra.run.smithi047.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-03-21T10:35:32.674 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-03-21T10:35:32.674 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "int", 2024-03-21T10:35:32.674 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:32.674 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T10:35:32.674 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "10", 2024-03-21T10:35:32.674 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:32.674 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:32.674 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:32.674 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T10:35:32.675 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:32.675 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:32.675 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:32.675 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:32.675 INFO:teuthology.orchestra.run.smithi047.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-03-21T10:35:32.675 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-03-21T10:35:32.675 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "int", 2024-03-21T10:35:32.675 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:32.675 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T10:35:32.675 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "8", 2024-03-21T10:35:32.675 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:32.675 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:32.675 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:32.675 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T10:35:32.676 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:32.676 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:32.676 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:32.676 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:32.676 INFO:teuthology.orchestra.run.smithi047.stdout: "REST_REQUESTS_TIMEOUT": { 2024-03-21T10:35:32.676 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-03-21T10:35:32.676 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "int", 2024-03-21T10:35:32.676 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:32.676 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T10:35:32.676 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "45", 2024-03-21T10:35:32.676 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:32.676 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:32.676 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:32.676 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T10:35:32.676 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:32.677 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:32.677 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:32.677 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:32.677 INFO:teuthology.orchestra.run.smithi047.stdout: "RGW_API_ACCESS_KEY": { 2024-03-21T10:35:32.677 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "RGW_API_ACCESS_KEY", 2024-03-21T10:35:32.677 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T10:35:32.677 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:32.677 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T10:35:32.677 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-03-21T10:35:32.677 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:32.677 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:32.677 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:32.677 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T10:35:32.677 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:32.677 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:32.678 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:32.678 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:32.678 INFO:teuthology.orchestra.run.smithi047.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-03-21T10:35:32.678 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-03-21T10:35:32.678 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T10:35:32.678 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:32.678 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T10:35:32.678 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "admin", 2024-03-21T10:35:32.678 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:32.678 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:32.678 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:32.678 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T10:35:32.678 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:32.679 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:32.679 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:32.679 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:32.679 INFO:teuthology.orchestra.run.smithi047.stdout: "RGW_API_SECRET_KEY": { 2024-03-21T10:35:32.679 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "RGW_API_SECRET_KEY", 2024-03-21T10:35:32.679 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T10:35:32.679 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:32.679 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T10:35:32.679 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-03-21T10:35:32.679 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:32.679 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:32.679 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:32.679 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T10:35:32.679 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:32.680 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:32.680 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:32.680 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:32.680 INFO:teuthology.orchestra.run.smithi047.stdout: "RGW_API_SSL_VERIFY": { 2024-03-21T10:35:32.680 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "RGW_API_SSL_VERIFY", 2024-03-21T10:35:32.680 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T10:35:32.680 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:32.680 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T10:35:32.680 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "True", 2024-03-21T10:35:32.680 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:32.680 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:32.680 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:32.680 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T10:35:32.680 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:32.681 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:32.681 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:32.681 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:32.681 INFO:teuthology.orchestra.run.smithi047.stdout: "UNSAFE_TLS_v1_2": { 2024-03-21T10:35:32.681 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "UNSAFE_TLS_v1_2", 2024-03-21T10:35:32.681 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T10:35:32.681 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:32.681 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T10:35:32.681 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-03-21T10:35:32.681 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:32.681 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:32.681 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:32.681 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T10:35:32.681 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:32.681 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:32.682 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:32.682 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:32.682 INFO:teuthology.orchestra.run.smithi047.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-03-21T10:35:32.682 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-03-21T10:35:32.682 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "int", 2024-03-21T10:35:32.682 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:32.682 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T10:35:32.682 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "0", 2024-03-21T10:35:32.682 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:32.682 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:32.682 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:32.682 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T10:35:32.682 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:32.682 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:32.682 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:32.683 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:32.683 INFO:teuthology.orchestra.run.smithi047.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-03-21T10:35:32.683 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-03-21T10:35:32.683 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "int", 2024-03-21T10:35:32.683 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:32.683 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T10:35:32.683 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "10", 2024-03-21T10:35:32.683 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:32.683 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:32.683 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:32.683 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T10:35:32.683 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:32.683 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:32.684 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:32.684 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:32.684 INFO:teuthology.orchestra.run.smithi047.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-03-21T10:35:32.684 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-03-21T10:35:32.684 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "int", 2024-03-21T10:35:32.684 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:32.684 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T10:35:32.684 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "5", 2024-03-21T10:35:32.684 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:32.684 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:32.684 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:32.684 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T10:35:32.684 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:32.685 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:32.685 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:32.685 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:32.685 INFO:teuthology.orchestra.run.smithi047.stdout: "cross_origin_url": { 2024-03-21T10:35:32.685 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "cross_origin_url", 2024-03-21T10:35:32.685 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T10:35:32.685 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:32.685 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T10:35:32.685 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-03-21T10:35:32.685 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:32.685 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:32.685 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:32.685 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T10:35:32.685 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:32.685 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:32.686 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:32.686 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:32.686 INFO:teuthology.orchestra.run.smithi047.stdout: "crt_file": { 2024-03-21T10:35:32.686 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "crt_file", 2024-03-21T10:35:32.686 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T10:35:32.686 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:32.686 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T10:35:32.686 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-03-21T10:35:32.686 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:32.686 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:32.686 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:32.686 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T10:35:32.686 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:32.687 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:32.687 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:32.687 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:32.687 INFO:teuthology.orchestra.run.smithi047.stdout: "debug": { 2024-03-21T10:35:32.687 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "debug", 2024-03-21T10:35:32.687 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T10:35:32.687 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:32.687 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T10:35:32.687 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-03-21T10:35:32.687 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:32.687 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:32.687 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:32.687 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Enable/disable debug options", 2024-03-21T10:35:32.687 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:32.687 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:32.688 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:32.688 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:32.688 INFO:teuthology.orchestra.run.smithi047.stdout: "jwt_token_ttl": { 2024-03-21T10:35:32.688 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "jwt_token_ttl", 2024-03-21T10:35:32.688 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "int", 2024-03-21T10:35:32.688 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:32.688 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T10:35:32.688 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "28800", 2024-03-21T10:35:32.688 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:32.688 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:32.688 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:32.688 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T10:35:32.688 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:32.688 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:32.689 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:32.689 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:32.689 INFO:teuthology.orchestra.run.smithi047.stdout: "key_file": { 2024-03-21T10:35:32.689 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "key_file", 2024-03-21T10:35:32.689 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T10:35:32.689 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:32.689 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T10:35:32.689 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-03-21T10:35:32.689 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:32.689 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:32.689 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:32.689 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T10:35:32.689 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:32.689 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:32.690 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:32.690 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:32.690 INFO:teuthology.orchestra.run.smithi047.stdout: "log_level": { 2024-03-21T10:35:32.690 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_level", 2024-03-21T10:35:32.690 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T10:35:32.690 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:32.690 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T10:35:32.690 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-03-21T10:35:32.690 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:32.690 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:32.690 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-03-21T10:35:32.690 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-03-21T10:35:32.690 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-03-21T10:35:32.691 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-03-21T10:35:32.691 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-03-21T10:35:32.691 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-03-21T10:35:32.691 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-03-21T10:35:32.691 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-03-21T10:35:32.691 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T10:35:32.691 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:32.691 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:32.691 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:32.691 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:32.691 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster": { 2024-03-21T10:35:32.691 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster", 2024-03-21T10:35:32.691 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T10:35:32.691 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:32.691 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T10:35:32.692 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-03-21T10:35:32.692 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:32.692 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:32.692 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:32.692 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T10:35:32.692 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:32.692 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:32.692 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:32.692 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:32.692 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster_level": { 2024-03-21T10:35:32.692 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster_level", 2024-03-21T10:35:32.692 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T10:35:32.692 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:32.692 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T10:35:32.692 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "info", 2024-03-21T10:35:32.693 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:32.693 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:32.693 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-03-21T10:35:32.693 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-03-21T10:35:32.693 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-03-21T10:35:32.693 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-03-21T10:35:32.693 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-03-21T10:35:32.693 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-03-21T10:35:32.693 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-03-21T10:35:32.693 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-03-21T10:35:32.693 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T10:35:32.693 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:32.693 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:32.693 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:32.693 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:32.694 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_file": { 2024-03-21T10:35:32.694 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_file", 2024-03-21T10:35:32.694 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T10:35:32.694 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:32.694 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T10:35:32.694 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-03-21T10:35:32.694 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:32.694 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:32.694 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:32.694 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T10:35:32.694 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:32.694 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:32.694 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:32.694 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:32.695 INFO:teuthology.orchestra.run.smithi047.stdout: "motd": { 2024-03-21T10:35:32.695 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "motd", 2024-03-21T10:35:32.695 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T10:35:32.695 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:32.695 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T10:35:32.695 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-03-21T10:35:32.695 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:32.695 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:32.695 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:32.695 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "The message of the day", 2024-03-21T10:35:32.695 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:32.695 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:32.695 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:32.695 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:32.695 INFO:teuthology.orchestra.run.smithi047.stdout: "redirect_resolve_ip_addr": { 2024-03-21T10:35:32.696 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "redirect_resolve_ip_addr", 2024-03-21T10:35:32.696 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T10:35:32.696 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:32.696 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T10:35:32.696 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-03-21T10:35:32.696 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:32.696 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:32.696 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:32.696 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T10:35:32.696 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:32.696 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:32.696 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:32.696 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:32.696 INFO:teuthology.orchestra.run.smithi047.stdout: "server_addr": { 2024-03-21T10:35:32.697 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "server_addr", 2024-03-21T10:35:32.697 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T10:35:32.697 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:32.697 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T10:35:32.697 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "::", 2024-03-21T10:35:32.697 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:32.697 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:32.697 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:32.697 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T10:35:32.697 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:32.697 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:32.697 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:32.697 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:32.697 INFO:teuthology.orchestra.run.smithi047.stdout: "server_port": { 2024-03-21T10:35:32.698 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "server_port", 2024-03-21T10:35:32.698 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "int", 2024-03-21T10:35:32.698 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:32.698 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T10:35:32.698 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "8080", 2024-03-21T10:35:32.698 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:32.698 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:32.698 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:32.698 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T10:35:32.698 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:32.698 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:32.698 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:32.698 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:32.698 INFO:teuthology.orchestra.run.smithi047.stdout: "ssl": { 2024-03-21T10:35:32.699 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "ssl", 2024-03-21T10:35:32.699 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T10:35:32.699 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:32.699 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T10:35:32.699 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "True", 2024-03-21T10:35:32.699 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:32.699 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:32.699 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:32.699 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T10:35:32.699 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:32.699 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:32.699 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:32.699 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:32.700 INFO:teuthology.orchestra.run.smithi047.stdout: "ssl_server_port": { 2024-03-21T10:35:32.700 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "ssl_server_port", 2024-03-21T10:35:32.700 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "int", 2024-03-21T10:35:32.700 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:32.700 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T10:35:32.700 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "8443", 2024-03-21T10:35:32.700 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:32.700 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:32.700 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:32.700 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T10:35:32.700 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:32.700 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:32.701 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:32.701 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:32.701 INFO:teuthology.orchestra.run.smithi047.stdout: "standby_behaviour": { 2024-03-21T10:35:32.701 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "standby_behaviour", 2024-03-21T10:35:32.701 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T10:35:32.701 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:32.701 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T10:35:32.701 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "redirect", 2024-03-21T10:35:32.701 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:32.701 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:32.701 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-03-21T10:35:32.701 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-03-21T10:35:32.701 INFO:teuthology.orchestra.run.smithi047.stdout: "redirect" 2024-03-21T10:35:32.702 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-03-21T10:35:32.702 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T10:35:32.702 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:32.702 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:32.702 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:32.702 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:32.702 INFO:teuthology.orchestra.run.smithi047.stdout: "standby_error_status_code": { 2024-03-21T10:35:32.702 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "standby_error_status_code", 2024-03-21T10:35:32.702 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "int", 2024-03-21T10:35:32.702 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:32.702 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T10:35:32.702 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "500", 2024-03-21T10:35:32.702 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "400", 2024-03-21T10:35:32.703 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "599", 2024-03-21T10:35:32.703 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:32.703 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T10:35:32.703 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:32.703 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:32.703 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:32.703 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:32.703 INFO:teuthology.orchestra.run.smithi047.stdout: "url_prefix": { 2024-03-21T10:35:32.703 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "url_prefix", 2024-03-21T10:35:32.703 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T10:35:32.703 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:32.703 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T10:35:32.703 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-03-21T10:35:32.703 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:32.703 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:32.704 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:32.704 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T10:35:32.704 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:32.704 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:32.704 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:32.704 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-21T10:35:32.704 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-21T10:35:32.704 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:32.704 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-03-21T10:35:32.704 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "devicehealth", 2024-03-21T10:35:32.704 INFO:teuthology.orchestra.run.smithi047.stdout: "can_run": true, 2024-03-21T10:35:32.704 INFO:teuthology.orchestra.run.smithi047.stdout: "error_string": "", 2024-03-21T10:35:32.704 INFO:teuthology.orchestra.run.smithi047.stdout: "module_options": { 2024-03-21T10:35:32.704 INFO:teuthology.orchestra.run.smithi047.stdout: "enable_monitoring": { 2024-03-21T10:35:32.705 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "enable_monitoring", 2024-03-21T10:35:32.705 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T10:35:32.705 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:32.705 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T10:35:32.705 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "True", 2024-03-21T10:35:32.705 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:32.705 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:32.705 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:32.705 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "monitor device health metrics", 2024-03-21T10:35:32.705 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:32.705 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:32.705 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:32.705 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:32.705 INFO:teuthology.orchestra.run.smithi047.stdout: "log_level": { 2024-03-21T10:35:32.705 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_level", 2024-03-21T10:35:32.706 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T10:35:32.706 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:32.706 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T10:35:32.706 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-03-21T10:35:32.706 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:32.706 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:32.706 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-03-21T10:35:32.706 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-03-21T10:35:32.706 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-03-21T10:35:32.706 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-03-21T10:35:32.706 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-03-21T10:35:32.706 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-03-21T10:35:32.706 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-03-21T10:35:32.707 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-03-21T10:35:32.707 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T10:35:32.707 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:32.707 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:32.707 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:32.707 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:32.707 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster": { 2024-03-21T10:35:32.707 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster", 2024-03-21T10:35:32.707 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T10:35:32.707 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:32.707 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T10:35:32.707 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-03-21T10:35:32.708 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:32.708 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:32.708 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:32.708 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T10:35:32.708 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:32.708 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:32.708 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:32.708 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:32.708 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster_level": { 2024-03-21T10:35:32.708 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster_level", 2024-03-21T10:35:32.708 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T10:35:32.708 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:32.708 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T10:35:32.709 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "info", 2024-03-21T10:35:32.709 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:32.709 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:32.709 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-03-21T10:35:32.709 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-03-21T10:35:32.709 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-03-21T10:35:32.709 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-03-21T10:35:32.709 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-03-21T10:35:32.709 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-03-21T10:35:32.709 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-03-21T10:35:32.709 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-03-21T10:35:32.709 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T10:35:32.709 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:32.709 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:32.710 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:32.710 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:32.710 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_file": { 2024-03-21T10:35:32.710 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_file", 2024-03-21T10:35:32.710 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T10:35:32.710 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:32.710 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T10:35:32.710 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-03-21T10:35:32.710 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:32.710 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:32.710 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:32.710 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T10:35:32.710 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:32.710 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:32.711 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:32.711 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:32.711 INFO:teuthology.orchestra.run.smithi047.stdout: "mark_out_threshold": { 2024-03-21T10:35:32.711 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "mark_out_threshold", 2024-03-21T10:35:32.711 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "secs", 2024-03-21T10:35:32.711 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:32.711 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T10:35:32.711 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "2419200", 2024-03-21T10:35:32.711 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:32.711 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:32.711 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:32.711 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-03-21T10:35:32.711 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:32.711 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:32.712 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:32.712 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:32.712 INFO:teuthology.orchestra.run.smithi047.stdout: "pool_name": { 2024-03-21T10:35:32.712 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "pool_name", 2024-03-21T10:35:32.712 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T10:35:32.712 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:32.712 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T10:35:32.712 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "device_health_metrics", 2024-03-21T10:35:32.712 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:32.712 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:32.712 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:32.712 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "name of pool in which to store device health metrics", 2024-03-21T10:35:32.712 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:32.712 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:32.713 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:32.713 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:32.713 INFO:teuthology.orchestra.run.smithi047.stdout: "retention_period": { 2024-03-21T10:35:32.713 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "retention_period", 2024-03-21T10:35:32.713 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "secs", 2024-03-21T10:35:32.713 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:32.713 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T10:35:32.713 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "15552000", 2024-03-21T10:35:32.713 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:32.713 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:32.713 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:32.713 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "how long to retain device health metrics", 2024-03-21T10:35:32.713 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:32.713 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:32.714 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:32.714 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:32.714 INFO:teuthology.orchestra.run.smithi047.stdout: "scrape_frequency": { 2024-03-21T10:35:32.714 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "scrape_frequency", 2024-03-21T10:35:32.714 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "secs", 2024-03-21T10:35:32.714 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:32.714 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T10:35:32.714 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "86400", 2024-03-21T10:35:32.714 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:32.714 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:32.714 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:32.714 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "how frequently to scrape device health metrics", 2024-03-21T10:35:32.714 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:32.714 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:32.715 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:32.715 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:32.715 INFO:teuthology.orchestra.run.smithi047.stdout: "self_heal": { 2024-03-21T10:35:32.715 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "self_heal", 2024-03-21T10:35:32.715 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T10:35:32.715 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:32.715 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T10:35:32.715 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "True", 2024-03-21T10:35:32.715 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:32.715 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:32.715 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:32.715 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-03-21T10:35:32.715 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:32.715 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:32.716 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:32.716 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:32.716 INFO:teuthology.orchestra.run.smithi047.stdout: "sleep_interval": { 2024-03-21T10:35:32.716 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "sleep_interval", 2024-03-21T10:35:32.716 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "secs", 2024-03-21T10:35:32.716 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:32.716 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T10:35:32.716 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "600", 2024-03-21T10:35:32.716 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:32.716 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:32.716 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:32.716 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "how frequently to wake up and check device health", 2024-03-21T10:35:32.716 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:32.716 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:32.717 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:32.717 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:32.717 INFO:teuthology.orchestra.run.smithi047.stdout: "warn_threshold": { 2024-03-21T10:35:32.717 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "warn_threshold", 2024-03-21T10:35:32.717 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "secs", 2024-03-21T10:35:32.717 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:32.717 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T10:35:32.717 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "7257600", 2024-03-21T10:35:32.717 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:32.717 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:32.717 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:32.717 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-03-21T10:35:32.717 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:32.717 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:32.717 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:32.718 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-21T10:35:32.718 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-21T10:35:32.718 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:32.718 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-03-21T10:35:32.718 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "diskprediction_local", 2024-03-21T10:35:32.718 INFO:teuthology.orchestra.run.smithi047.stdout: "can_run": true, 2024-03-21T10:35:32.718 INFO:teuthology.orchestra.run.smithi047.stdout: "error_string": "", 2024-03-21T10:35:32.718 INFO:teuthology.orchestra.run.smithi047.stdout: "module_options": { 2024-03-21T10:35:32.718 INFO:teuthology.orchestra.run.smithi047.stdout: "log_level": { 2024-03-21T10:35:32.718 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_level", 2024-03-21T10:35:32.718 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T10:35:32.718 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:32.718 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T10:35:32.718 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-03-21T10:35:32.719 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:32.719 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:32.719 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-03-21T10:35:32.719 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-03-21T10:35:32.719 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-03-21T10:35:32.719 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-03-21T10:35:32.719 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-03-21T10:35:32.719 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-03-21T10:35:32.719 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-03-21T10:35:32.719 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-03-21T10:35:32.719 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T10:35:32.719 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:32.719 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:32.719 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:32.719 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:32.720 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster": { 2024-03-21T10:35:32.720 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster", 2024-03-21T10:35:32.720 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T10:35:32.720 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:32.720 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T10:35:32.720 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-03-21T10:35:32.720 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:32.720 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:32.720 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:32.720 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T10:35:32.720 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:32.720 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:32.720 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:32.720 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:32.721 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster_level": { 2024-03-21T10:35:32.721 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster_level", 2024-03-21T10:35:32.721 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T10:35:32.721 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:32.721 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T10:35:32.721 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "info", 2024-03-21T10:35:32.721 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:32.721 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:32.721 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-03-21T10:35:32.721 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-03-21T10:35:32.721 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-03-21T10:35:32.721 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-03-21T10:35:32.721 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-03-21T10:35:32.721 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-03-21T10:35:32.722 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-03-21T10:35:32.722 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-03-21T10:35:32.722 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T10:35:32.722 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:32.722 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:32.722 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:32.722 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:32.722 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_file": { 2024-03-21T10:35:32.722 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_file", 2024-03-21T10:35:32.722 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T10:35:32.722 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:32.722 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T10:35:32.722 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-03-21T10:35:32.723 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:32.723 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:32.723 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:32.723 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T10:35:32.723 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:32.723 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:32.723 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:32.723 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:32.723 INFO:teuthology.orchestra.run.smithi047.stdout: "predict_interval": { 2024-03-21T10:35:32.723 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "predict_interval", 2024-03-21T10:35:32.723 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T10:35:32.723 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:32.723 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T10:35:32.723 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "86400", 2024-03-21T10:35:32.724 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:32.724 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:32.724 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:32.724 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T10:35:32.724 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:32.724 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:32.724 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:32.724 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:32.724 INFO:teuthology.orchestra.run.smithi047.stdout: "predictor_model": { 2024-03-21T10:35:32.724 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "predictor_model", 2024-03-21T10:35:32.724 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T10:35:32.724 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:32.724 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T10:35:32.724 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "prophetstor", 2024-03-21T10:35:32.724 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:32.725 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:32.725 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:32.725 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T10:35:32.725 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:32.725 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:32.725 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:32.725 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:32.725 INFO:teuthology.orchestra.run.smithi047.stdout: "sleep_interval": { 2024-03-21T10:35:32.725 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "sleep_interval", 2024-03-21T10:35:32.725 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T10:35:32.725 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:32.725 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T10:35:32.725 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "600", 2024-03-21T10:35:32.725 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:32.725 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:32.726 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:32.726 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T10:35:32.726 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:32.726 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:32.726 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:32.726 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-21T10:35:32.726 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-21T10:35:32.726 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:32.726 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-03-21T10:35:32.726 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "influx", 2024-03-21T10:35:32.726 INFO:teuthology.orchestra.run.smithi047.stdout: "can_run": false, 2024-03-21T10:35:32.726 INFO:teuthology.orchestra.run.smithi047.stdout: "error_string": "influxdb python module not found", 2024-03-21T10:35:32.726 INFO:teuthology.orchestra.run.smithi047.stdout: "module_options": { 2024-03-21T10:35:32.726 INFO:teuthology.orchestra.run.smithi047.stdout: "batch_size": { 2024-03-21T10:35:32.726 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "batch_size", 2024-03-21T10:35:32.727 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "int", 2024-03-21T10:35:32.727 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:32.727 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T10:35:32.727 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "5000", 2024-03-21T10:35:32.727 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:32.727 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:32.727 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:32.727 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-03-21T10:35:32.727 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:32.727 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:32.727 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:32.727 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:32.727 INFO:teuthology.orchestra.run.smithi047.stdout: "database": { 2024-03-21T10:35:32.727 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "database", 2024-03-21T10:35:32.728 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T10:35:32.728 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:32.728 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T10:35:32.728 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "ceph", 2024-03-21T10:35:32.728 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:32.728 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:32.728 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:32.728 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-03-21T10:35:32.728 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:32.728 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:32.728 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:32.728 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:32.728 INFO:teuthology.orchestra.run.smithi047.stdout: "hostname": { 2024-03-21T10:35:32.729 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "hostname", 2024-03-21T10:35:32.729 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T10:35:32.729 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:32.729 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T10:35:32.729 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-03-21T10:35:32.729 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:32.729 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:32.729 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:32.729 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "InfluxDB server hostname", 2024-03-21T10:35:32.729 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:32.729 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:32.729 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:32.729 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:32.729 INFO:teuthology.orchestra.run.smithi047.stdout: "interval": { 2024-03-21T10:35:32.729 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "interval", 2024-03-21T10:35:32.730 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "secs", 2024-03-21T10:35:32.730 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:32.730 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T10:35:32.730 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "30", 2024-03-21T10:35:32.730 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "5", 2024-03-21T10:35:32.730 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:32.730 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:32.730 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-03-21T10:35:32.730 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:32.730 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:32.730 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:32.730 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:32.730 INFO:teuthology.orchestra.run.smithi047.stdout: "log_level": { 2024-03-21T10:35:32.731 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_level", 2024-03-21T10:35:32.731 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T10:35:32.731 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:32.731 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T10:35:32.731 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-03-21T10:35:32.731 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:32.731 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:32.731 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-03-21T10:35:32.731 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-03-21T10:35:32.731 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-03-21T10:35:32.731 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-03-21T10:35:32.731 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-03-21T10:35:32.731 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-03-21T10:35:32.731 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-03-21T10:35:32.731 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-03-21T10:35:32.732 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T10:35:32.732 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:32.732 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:32.732 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:32.732 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:32.732 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster": { 2024-03-21T10:35:32.732 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster", 2024-03-21T10:35:32.732 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T10:35:32.732 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:32.732 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T10:35:32.732 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-03-21T10:35:32.732 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:32.732 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:32.732 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:32.732 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T10:35:32.733 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:32.733 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:32.733 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:32.733 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:32.733 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster_level": { 2024-03-21T10:35:32.733 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster_level", 2024-03-21T10:35:32.733 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T10:35:32.733 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:32.733 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T10:35:32.733 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "info", 2024-03-21T10:35:32.733 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:32.733 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:32.733 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-03-21T10:35:32.733 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-03-21T10:35:32.733 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-03-21T10:35:32.734 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-03-21T10:35:32.734 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-03-21T10:35:32.734 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-03-21T10:35:32.734 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-03-21T10:35:32.734 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-03-21T10:35:32.734 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T10:35:32.734 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:32.734 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:32.734 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:32.734 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:32.734 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_file": { 2024-03-21T10:35:32.734 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_file", 2024-03-21T10:35:32.734 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T10:35:32.734 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:32.734 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T10:35:32.735 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-03-21T10:35:32.735 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:32.735 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:32.735 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:32.735 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T10:35:32.735 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:32.735 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:32.735 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:32.735 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:32.735 INFO:teuthology.orchestra.run.smithi047.stdout: "password": { 2024-03-21T10:35:32.735 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "password", 2024-03-21T10:35:32.735 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T10:35:32.735 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:32.735 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T10:35:32.735 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-03-21T10:35:32.735 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:32.736 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:32.736 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:32.736 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "password of InfluxDB server user", 2024-03-21T10:35:32.736 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:32.736 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:32.736 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:32.736 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:32.736 INFO:teuthology.orchestra.run.smithi047.stdout: "port": { 2024-03-21T10:35:32.736 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "port", 2024-03-21T10:35:32.736 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "int", 2024-03-21T10:35:32.736 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:32.736 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T10:35:32.736 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "8086", 2024-03-21T10:35:32.736 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:32.736 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:32.736 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:32.737 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "InfluxDB server port", 2024-03-21T10:35:32.737 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:32.737 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:32.737 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:32.737 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:32.737 INFO:teuthology.orchestra.run.smithi047.stdout: "ssl": { 2024-03-21T10:35:32.737 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "ssl", 2024-03-21T10:35:32.737 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T10:35:32.737 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:32.737 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T10:35:32.737 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "false", 2024-03-21T10:35:32.737 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:32.737 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:32.737 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:32.738 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-03-21T10:35:32.738 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:32.738 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:32.738 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:32.738 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:32.738 INFO:teuthology.orchestra.run.smithi047.stdout: "threads": { 2024-03-21T10:35:32.738 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "threads", 2024-03-21T10:35:32.738 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "int", 2024-03-21T10:35:32.738 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:32.738 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T10:35:32.738 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "5", 2024-03-21T10:35:32.738 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "1", 2024-03-21T10:35:32.738 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "32", 2024-03-21T10:35:32.738 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:32.739 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-03-21T10:35:32.739 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:32.739 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:32.739 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:32.739 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:32.739 INFO:teuthology.orchestra.run.smithi047.stdout: "username": { 2024-03-21T10:35:32.739 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "username", 2024-03-21T10:35:32.739 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T10:35:32.739 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:32.739 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T10:35:32.739 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-03-21T10:35:32.739 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:32.739 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:32.739 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:32.739 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "username of InfluxDB server user", 2024-03-21T10:35:32.740 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:32.740 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:32.740 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:32.740 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:32.740 INFO:teuthology.orchestra.run.smithi047.stdout: "verify_ssl": { 2024-03-21T10:35:32.740 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "verify_ssl", 2024-03-21T10:35:32.740 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T10:35:32.740 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:32.740 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T10:35:32.740 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "true", 2024-03-21T10:35:32.740 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:32.740 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:32.740 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:32.740 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-03-21T10:35:32.740 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:32.741 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:32.741 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:32.741 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-21T10:35:32.741 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-21T10:35:32.741 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:32.741 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-03-21T10:35:32.741 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "insights", 2024-03-21T10:35:32.741 INFO:teuthology.orchestra.run.smithi047.stdout: "can_run": true, 2024-03-21T10:35:32.741 INFO:teuthology.orchestra.run.smithi047.stdout: "error_string": "", 2024-03-21T10:35:32.741 INFO:teuthology.orchestra.run.smithi047.stdout: "module_options": { 2024-03-21T10:35:32.741 INFO:teuthology.orchestra.run.smithi047.stdout: "log_level": { 2024-03-21T10:35:32.741 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_level", 2024-03-21T10:35:32.741 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T10:35:32.741 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:32.741 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T10:35:32.741 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-03-21T10:35:32.742 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:32.742 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:32.742 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-03-21T10:35:32.742 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-03-21T10:35:32.742 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-03-21T10:35:32.742 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-03-21T10:35:32.742 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-03-21T10:35:32.742 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-03-21T10:35:32.742 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-03-21T10:35:32.742 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-03-21T10:35:32.742 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T10:35:32.742 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:32.742 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:32.743 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:32.743 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:32.743 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster": { 2024-03-21T10:35:32.743 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster", 2024-03-21T10:35:32.743 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T10:35:32.743 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:32.743 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T10:35:32.743 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-03-21T10:35:32.743 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:32.743 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:32.743 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:32.743 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T10:35:32.743 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:32.743 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:32.743 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:32.744 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:32.744 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster_level": { 2024-03-21T10:35:32.744 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster_level", 2024-03-21T10:35:32.744 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T10:35:32.744 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:32.744 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T10:35:32.744 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "info", 2024-03-21T10:35:32.744 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:32.744 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:32.744 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-03-21T10:35:32.744 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-03-21T10:35:32.744 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-03-21T10:35:32.744 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-03-21T10:35:32.744 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-03-21T10:35:32.744 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-03-21T10:35:32.745 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-03-21T10:35:32.745 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-03-21T10:35:32.745 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T10:35:32.745 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:32.745 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:32.745 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:32.745 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:32.745 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_file": { 2024-03-21T10:35:32.745 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_file", 2024-03-21T10:35:32.745 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T10:35:32.745 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:32.745 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T10:35:32.745 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-03-21T10:35:32.745 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:32.745 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:32.746 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:32.746 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T10:35:32.746 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:32.746 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:32.746 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:32.746 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-21T10:35:32.746 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-21T10:35:32.746 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:32.746 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-03-21T10:35:32.746 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "iostat", 2024-03-21T10:35:32.746 INFO:teuthology.orchestra.run.smithi047.stdout: "can_run": true, 2024-03-21T10:35:32.746 INFO:teuthology.orchestra.run.smithi047.stdout: "error_string": "", 2024-03-21T10:35:32.746 INFO:teuthology.orchestra.run.smithi047.stdout: "module_options": { 2024-03-21T10:35:32.746 INFO:teuthology.orchestra.run.smithi047.stdout: "log_level": { 2024-03-21T10:35:32.746 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_level", 2024-03-21T10:35:32.746 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T10:35:32.747 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:32.747 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T10:35:32.747 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-03-21T10:35:32.747 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:32.747 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:32.747 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-03-21T10:35:32.747 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-03-21T10:35:32.747 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-03-21T10:35:32.747 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-03-21T10:35:32.747 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-03-21T10:35:32.747 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-03-21T10:35:32.747 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-03-21T10:35:32.747 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-03-21T10:35:32.747 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T10:35:32.747 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:32.748 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:32.748 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:32.748 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:32.748 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster": { 2024-03-21T10:35:32.748 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster", 2024-03-21T10:35:32.748 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T10:35:32.748 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:32.748 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T10:35:32.748 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-03-21T10:35:32.748 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:32.748 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:32.748 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:32.748 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T10:35:32.748 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:32.748 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:32.748 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:32.749 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:32.749 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster_level": { 2024-03-21T10:35:32.749 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster_level", 2024-03-21T10:35:32.749 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T10:35:32.749 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:32.749 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T10:35:32.749 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "info", 2024-03-21T10:35:32.749 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:32.749 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:32.749 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-03-21T10:35:32.749 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-03-21T10:35:32.749 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-03-21T10:35:32.749 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-03-21T10:35:32.749 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-03-21T10:35:32.750 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-03-21T10:35:32.750 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-03-21T10:35:32.750 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-03-21T10:35:32.750 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T10:35:32.750 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:32.750 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:32.750 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:32.750 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:32.750 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_file": { 2024-03-21T10:35:32.750 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_file", 2024-03-21T10:35:32.750 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T10:35:32.750 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:32.750 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T10:35:32.751 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-03-21T10:35:32.751 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:32.751 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:32.751 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:32.751 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T10:35:32.751 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:32.751 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:32.751 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:32.751 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-21T10:35:32.751 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-21T10:35:32.751 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:32.751 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-03-21T10:35:32.751 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "k8sevents", 2024-03-21T10:35:32.751 INFO:teuthology.orchestra.run.smithi047.stdout: "can_run": true, 2024-03-21T10:35:32.751 INFO:teuthology.orchestra.run.smithi047.stdout: "error_string": "", 2024-03-21T10:35:32.751 INFO:teuthology.orchestra.run.smithi047.stdout: "module_options": { 2024-03-21T10:35:32.752 INFO:teuthology.orchestra.run.smithi047.stdout: "ceph_event_retention_days": { 2024-03-21T10:35:32.752 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "ceph_event_retention_days", 2024-03-21T10:35:32.752 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "int", 2024-03-21T10:35:32.752 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:32.752 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T10:35:32.752 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "7", 2024-03-21T10:35:32.752 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:32.752 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:32.752 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:32.752 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Days to hold ceph event information within local cache", 2024-03-21T10:35:32.752 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:32.752 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:32.752 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:32.753 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:32.753 INFO:teuthology.orchestra.run.smithi047.stdout: "config_check_secs": { 2024-03-21T10:35:32.753 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "config_check_secs", 2024-03-21T10:35:32.753 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "int", 2024-03-21T10:35:32.753 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:32.753 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T10:35:32.753 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "10", 2024-03-21T10:35:32.753 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "10", 2024-03-21T10:35:32.753 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:32.753 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:32.753 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-03-21T10:35:32.753 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:32.753 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:32.753 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:32.753 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:32.754 INFO:teuthology.orchestra.run.smithi047.stdout: "log_level": { 2024-03-21T10:35:32.754 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_level", 2024-03-21T10:35:32.754 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T10:35:32.754 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:32.754 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T10:35:32.754 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-03-21T10:35:32.754 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:32.754 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:32.754 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-03-21T10:35:32.754 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-03-21T10:35:32.754 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-03-21T10:35:32.754 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-03-21T10:35:32.754 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-03-21T10:35:32.755 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-03-21T10:35:32.755 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-03-21T10:35:32.755 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-03-21T10:35:32.755 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T10:35:32.755 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:32.755 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:32.755 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:32.755 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:32.755 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster": { 2024-03-21T10:35:32.755 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster", 2024-03-21T10:35:32.755 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T10:35:32.755 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:32.755 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T10:35:32.755 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-03-21T10:35:32.756 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:32.756 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:32.756 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:32.756 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T10:35:32.756 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:32.756 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:32.756 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:32.756 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:32.756 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster_level": { 2024-03-21T10:35:32.756 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster_level", 2024-03-21T10:35:32.756 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T10:35:32.756 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:32.756 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T10:35:32.756 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "info", 2024-03-21T10:35:32.756 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:32.756 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:32.757 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-03-21T10:35:32.757 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-03-21T10:35:32.757 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-03-21T10:35:32.757 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-03-21T10:35:32.757 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-03-21T10:35:32.757 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-03-21T10:35:32.757 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-03-21T10:35:32.757 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-03-21T10:35:32.757 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T10:35:32.757 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:32.757 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:32.757 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:32.757 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:32.757 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_file": { 2024-03-21T10:35:32.758 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_file", 2024-03-21T10:35:32.758 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T10:35:32.758 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:32.758 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T10:35:32.758 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-03-21T10:35:32.758 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:32.758 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:32.758 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:32.758 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T10:35:32.758 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:32.758 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:32.758 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:32.758 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-21T10:35:32.758 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-21T10:35:32.758 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:32.759 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-03-21T10:35:32.759 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "localpool", 2024-03-21T10:35:32.759 INFO:teuthology.orchestra.run.smithi047.stdout: "can_run": true, 2024-03-21T10:35:32.759 INFO:teuthology.orchestra.run.smithi047.stdout: "error_string": "", 2024-03-21T10:35:32.759 INFO:teuthology.orchestra.run.smithi047.stdout: "module_options": { 2024-03-21T10:35:32.759 INFO:teuthology.orchestra.run.smithi047.stdout: "failure_domain": { 2024-03-21T10:35:32.759 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "failure_domain", 2024-03-21T10:35:32.759 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T10:35:32.759 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:32.759 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T10:35:32.759 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "host", 2024-03-21T10:35:32.759 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:32.759 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:32.759 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:32.759 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "failure domain for any created local pool", 2024-03-21T10:35:32.759 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-03-21T10:35:32.760 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:32.760 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:32.760 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:32.760 INFO:teuthology.orchestra.run.smithi047.stdout: "log_level": { 2024-03-21T10:35:32.760 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_level", 2024-03-21T10:35:32.760 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T10:35:32.760 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:32.760 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T10:35:32.760 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-03-21T10:35:32.760 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:32.760 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:32.760 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-03-21T10:35:32.760 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-03-21T10:35:32.760 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-03-21T10:35:32.760 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-03-21T10:35:32.760 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-03-21T10:35:32.761 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-03-21T10:35:32.761 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-03-21T10:35:32.761 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-03-21T10:35:32.761 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T10:35:32.761 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:32.761 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:32.761 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:32.761 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:32.761 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster": { 2024-03-21T10:35:32.761 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster", 2024-03-21T10:35:32.761 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T10:35:32.761 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:32.761 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T10:35:32.761 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-03-21T10:35:32.761 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:32.762 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:32.762 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:32.762 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T10:35:32.762 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:32.762 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:32.762 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:32.762 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:32.762 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster_level": { 2024-03-21T10:35:32.762 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster_level", 2024-03-21T10:35:32.762 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T10:35:32.762 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:32.762 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T10:35:32.762 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "info", 2024-03-21T10:35:32.762 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:32.763 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:32.763 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-03-21T10:35:32.763 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-03-21T10:35:32.763 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-03-21T10:35:32.763 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-03-21T10:35:32.763 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-03-21T10:35:32.763 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-03-21T10:35:32.763 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-03-21T10:35:32.763 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-03-21T10:35:32.763 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T10:35:32.763 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:32.763 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:32.763 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:32.763 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:32.763 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_file": { 2024-03-21T10:35:32.764 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_file", 2024-03-21T10:35:32.764 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T10:35:32.764 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:32.764 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T10:35:32.764 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-03-21T10:35:32.764 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:32.764 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:32.764 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:32.764 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T10:35:32.764 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:32.764 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:32.764 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:32.764 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:32.764 INFO:teuthology.orchestra.run.smithi047.stdout: "min_size": { 2024-03-21T10:35:32.764 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "min_size", 2024-03-21T10:35:32.764 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "int", 2024-03-21T10:35:32.765 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:32.765 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T10:35:32.765 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-03-21T10:35:32.765 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:32.765 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:32.765 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:32.765 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "default min_size for any created local pool", 2024-03-21T10:35:32.765 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-03-21T10:35:32.765 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:32.765 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:32.765 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:32.765 INFO:teuthology.orchestra.run.smithi047.stdout: "num_rep": { 2024-03-21T10:35:32.766 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "num_rep", 2024-03-21T10:35:32.766 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "int", 2024-03-21T10:35:32.766 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:32.766 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T10:35:32.766 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "3", 2024-03-21T10:35:32.766 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:32.766 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:32.766 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:32.766 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "default replica count for any created local pool", 2024-03-21T10:35:32.766 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:32.766 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:32.766 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:32.766 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:32.766 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_num": { 2024-03-21T10:35:32.767 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "pg_num", 2024-03-21T10:35:32.767 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "int", 2024-03-21T10:35:32.767 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:32.767 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T10:35:32.767 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "128", 2024-03-21T10:35:32.767 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:32.767 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:32.767 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:32.767 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "default pg_num for any created local pool", 2024-03-21T10:35:32.767 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:32.767 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:32.767 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:32.767 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:32.767 INFO:teuthology.orchestra.run.smithi047.stdout: "prefix": { 2024-03-21T10:35:32.767 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "prefix", 2024-03-21T10:35:32.768 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T10:35:32.768 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:32.768 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T10:35:32.768 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-03-21T10:35:32.768 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:32.768 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:32.768 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:32.768 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "name prefix for any created local pool", 2024-03-21T10:35:32.768 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:32.768 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:32.768 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:32.768 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:32.768 INFO:teuthology.orchestra.run.smithi047.stdout: "subtree": { 2024-03-21T10:35:32.768 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "subtree", 2024-03-21T10:35:32.768 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T10:35:32.769 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:32.769 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T10:35:32.769 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "rack", 2024-03-21T10:35:32.769 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:32.769 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:32.769 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:32.769 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "CRUSH level for which to create a local pool", 2024-03-21T10:35:32.769 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-03-21T10:35:32.769 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:32.769 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:32.769 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-21T10:35:32.769 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-21T10:35:32.769 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:32.769 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-03-21T10:35:32.769 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "mds_autoscaler", 2024-03-21T10:35:32.770 INFO:teuthology.orchestra.run.smithi047.stdout: "can_run": true, 2024-03-21T10:35:32.770 INFO:teuthology.orchestra.run.smithi047.stdout: "error_string": "", 2024-03-21T10:35:32.770 INFO:teuthology.orchestra.run.smithi047.stdout: "module_options": { 2024-03-21T10:35:32.770 INFO:teuthology.orchestra.run.smithi047.stdout: "log_level": { 2024-03-21T10:35:32.770 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_level", 2024-03-21T10:35:32.770 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T10:35:32.770 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:32.770 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T10:35:32.770 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-03-21T10:35:32.770 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:32.770 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:32.770 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-03-21T10:35:32.770 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-03-21T10:35:32.770 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-03-21T10:35:32.771 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-03-21T10:35:32.771 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-03-21T10:35:32.771 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-03-21T10:35:32.771 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-03-21T10:35:32.771 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-03-21T10:35:32.771 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T10:35:32.771 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:32.771 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:32.771 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:32.771 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:32.771 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster": { 2024-03-21T10:35:32.771 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster", 2024-03-21T10:35:32.771 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T10:35:32.771 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:32.771 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T10:35:32.772 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-03-21T10:35:32.772 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:32.772 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:32.772 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:32.772 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T10:35:32.772 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:32.772 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:32.772 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:32.772 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:32.772 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster_level": { 2024-03-21T10:35:32.772 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster_level", 2024-03-21T10:35:32.772 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T10:35:32.772 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:32.772 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T10:35:32.772 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "info", 2024-03-21T10:35:32.773 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:32.773 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:32.773 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-03-21T10:35:32.773 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-03-21T10:35:32.773 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-03-21T10:35:32.773 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-03-21T10:35:32.773 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-03-21T10:35:32.773 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-03-21T10:35:32.773 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-03-21T10:35:32.773 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-03-21T10:35:32.773 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T10:35:32.773 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:32.773 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:32.773 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:32.774 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:32.774 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_file": { 2024-03-21T10:35:32.774 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_file", 2024-03-21T10:35:32.774 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T10:35:32.774 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:32.774 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T10:35:32.774 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-03-21T10:35:32.774 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:32.774 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:32.774 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:32.774 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T10:35:32.774 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:32.774 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:32.775 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:32.775 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-21T10:35:32.775 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-21T10:35:32.775 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:32.775 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-03-21T10:35:32.775 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "mirroring", 2024-03-21T10:35:32.775 INFO:teuthology.orchestra.run.smithi047.stdout: "can_run": true, 2024-03-21T10:35:32.775 INFO:teuthology.orchestra.run.smithi047.stdout: "error_string": "", 2024-03-21T10:35:32.775 INFO:teuthology.orchestra.run.smithi047.stdout: "module_options": { 2024-03-21T10:35:32.775 INFO:teuthology.orchestra.run.smithi047.stdout: "log_level": { 2024-03-21T10:35:32.775 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_level", 2024-03-21T10:35:32.775 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T10:35:32.775 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:32.775 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T10:35:32.775 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-03-21T10:35:32.776 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:32.776 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:32.776 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-03-21T10:35:32.776 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-03-21T10:35:32.776 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-03-21T10:35:32.776 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-03-21T10:35:32.776 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-03-21T10:35:32.776 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-03-21T10:35:32.776 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-03-21T10:35:32.776 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-03-21T10:35:32.776 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T10:35:32.776 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:32.776 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:32.776 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:32.776 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:32.776 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster": { 2024-03-21T10:35:32.777 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster", 2024-03-21T10:35:32.777 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T10:35:32.777 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:32.777 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T10:35:32.777 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-03-21T10:35:32.777 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:32.777 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:32.777 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:32.777 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T10:35:32.777 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:32.777 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:32.777 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:32.777 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:32.777 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster_level": { 2024-03-21T10:35:32.777 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster_level", 2024-03-21T10:35:32.778 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T10:35:32.778 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:32.778 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T10:35:32.778 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "info", 2024-03-21T10:35:32.778 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:32.778 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:32.778 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-03-21T10:35:32.778 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-03-21T10:35:32.778 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-03-21T10:35:32.778 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-03-21T10:35:32.778 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-03-21T10:35:32.778 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-03-21T10:35:32.778 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-03-21T10:35:32.778 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-03-21T10:35:32.779 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T10:35:32.779 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:32.779 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:32.779 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:32.779 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:32.779 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_file": { 2024-03-21T10:35:32.779 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_file", 2024-03-21T10:35:32.779 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T10:35:32.779 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:32.779 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T10:35:32.779 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-03-21T10:35:32.779 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:32.779 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:32.779 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:32.779 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T10:35:32.780 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:32.780 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:32.780 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:32.780 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-21T10:35:32.780 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-21T10:35:32.780 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:32.780 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-03-21T10:35:32.780 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "nfs", 2024-03-21T10:35:32.780 INFO:teuthology.orchestra.run.smithi047.stdout: "can_run": true, 2024-03-21T10:35:32.780 INFO:teuthology.orchestra.run.smithi047.stdout: "error_string": "", 2024-03-21T10:35:32.780 INFO:teuthology.orchestra.run.smithi047.stdout: "module_options": { 2024-03-21T10:35:32.780 INFO:teuthology.orchestra.run.smithi047.stdout: "log_level": { 2024-03-21T10:35:32.780 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_level", 2024-03-21T10:35:32.780 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T10:35:32.780 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:32.781 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T10:35:32.781 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-03-21T10:35:32.781 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:32.781 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:32.781 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-03-21T10:35:32.781 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-03-21T10:35:32.781 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-03-21T10:35:32.781 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-03-21T10:35:32.781 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-03-21T10:35:32.781 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-03-21T10:35:32.781 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-03-21T10:35:32.781 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-03-21T10:35:32.781 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T10:35:32.781 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:32.781 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:32.782 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:32.782 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:32.782 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster": { 2024-03-21T10:35:32.782 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster", 2024-03-21T10:35:32.782 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T10:35:32.782 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:32.782 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T10:35:32.782 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-03-21T10:35:32.782 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:32.782 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:32.782 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:32.782 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T10:35:32.782 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:32.783 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:32.783 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:32.783 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:32.783 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster_level": { 2024-03-21T10:35:32.783 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster_level", 2024-03-21T10:35:32.783 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T10:35:32.783 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:32.783 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T10:35:32.783 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "info", 2024-03-21T10:35:32.783 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:32.783 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:32.783 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-03-21T10:35:32.783 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-03-21T10:35:32.783 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-03-21T10:35:32.784 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-03-21T10:35:32.784 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-03-21T10:35:32.784 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-03-21T10:35:32.784 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-03-21T10:35:32.784 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-03-21T10:35:32.784 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T10:35:32.784 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:32.784 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:32.784 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:32.784 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:32.784 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_file": { 2024-03-21T10:35:32.784 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_file", 2024-03-21T10:35:32.784 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T10:35:32.784 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:32.785 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T10:35:32.785 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-03-21T10:35:32.785 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:32.785 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:32.785 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:32.785 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T10:35:32.785 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:32.785 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:32.785 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:32.785 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-21T10:35:32.785 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-21T10:35:32.785 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:32.785 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-03-21T10:35:32.785 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "orchestrator", 2024-03-21T10:35:32.785 INFO:teuthology.orchestra.run.smithi047.stdout: "can_run": true, 2024-03-21T10:35:32.786 INFO:teuthology.orchestra.run.smithi047.stdout: "error_string": "", 2024-03-21T10:35:32.786 INFO:teuthology.orchestra.run.smithi047.stdout: "module_options": { 2024-03-21T10:35:32.786 INFO:teuthology.orchestra.run.smithi047.stdout: "fail_fs": { 2024-03-21T10:35:32.786 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "fail_fs", 2024-03-21T10:35:32.786 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T10:35:32.786 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:32.786 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T10:35:32.786 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-03-21T10:35:32.786 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:32.786 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:32.786 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:32.786 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-03-21T10:35:32.786 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:32.787 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:32.787 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:32.787 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:32.787 INFO:teuthology.orchestra.run.smithi047.stdout: "log_level": { 2024-03-21T10:35:32.787 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_level", 2024-03-21T10:35:32.787 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T10:35:32.787 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:32.787 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T10:35:32.787 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-03-21T10:35:32.787 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:32.787 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:32.787 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-03-21T10:35:32.787 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-03-21T10:35:32.787 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-03-21T10:35:32.787 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-03-21T10:35:32.788 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-03-21T10:35:32.788 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-03-21T10:35:32.788 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-03-21T10:35:32.788 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-03-21T10:35:32.788 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T10:35:32.788 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:32.788 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:32.788 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:32.788 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:32.788 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster": { 2024-03-21T10:35:32.788 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster", 2024-03-21T10:35:32.788 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T10:35:32.788 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:32.788 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T10:35:32.789 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-03-21T10:35:32.789 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:32.789 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:32.789 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:32.789 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T10:35:32.789 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:32.789 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:32.789 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:32.789 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:32.789 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster_level": { 2024-03-21T10:35:32.789 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster_level", 2024-03-21T10:35:32.789 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T10:35:32.789 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:32.789 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T10:35:32.790 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "info", 2024-03-21T10:35:32.790 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:32.790 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:32.790 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-03-21T10:35:32.790 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-03-21T10:35:32.790 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-03-21T10:35:32.790 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-03-21T10:35:32.790 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-03-21T10:35:32.790 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-03-21T10:35:32.790 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-03-21T10:35:32.790 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-03-21T10:35:32.790 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T10:35:32.791 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:32.791 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:32.791 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:32.791 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:32.791 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_file": { 2024-03-21T10:35:32.791 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_file", 2024-03-21T10:35:32.791 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T10:35:32.791 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:32.791 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T10:35:32.791 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-03-21T10:35:32.791 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:32.791 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:32.791 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:32.792 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T10:35:32.792 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:32.792 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:32.792 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:32.792 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:32.792 INFO:teuthology.orchestra.run.smithi047.stdout: "orchestrator": { 2024-03-21T10:35:32.792 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "orchestrator", 2024-03-21T10:35:32.792 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T10:35:32.792 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:32.792 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T10:35:32.792 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-03-21T10:35:32.792 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:32.792 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:32.792 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-03-21T10:35:32.792 INFO:teuthology.orchestra.run.smithi047.stdout: "cephadm", 2024-03-21T10:35:32.793 INFO:teuthology.orchestra.run.smithi047.stdout: "rook", 2024-03-21T10:35:32.793 INFO:teuthology.orchestra.run.smithi047.stdout: "test_orchestrator" 2024-03-21T10:35:32.793 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-03-21T10:35:32.793 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Orchestrator backend", 2024-03-21T10:35:32.793 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:32.793 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:32.793 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:32.793 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-21T10:35:32.793 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-21T10:35:32.793 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:32.793 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-03-21T10:35:32.793 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "osd_perf_query", 2024-03-21T10:35:32.793 INFO:teuthology.orchestra.run.smithi047.stdout: "can_run": true, 2024-03-21T10:35:32.793 INFO:teuthology.orchestra.run.smithi047.stdout: "error_string": "", 2024-03-21T10:35:32.794 INFO:teuthology.orchestra.run.smithi047.stdout: "module_options": { 2024-03-21T10:35:32.794 INFO:teuthology.orchestra.run.smithi047.stdout: "log_level": { 2024-03-21T10:35:32.794 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_level", 2024-03-21T10:35:32.794 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T10:35:32.794 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:32.794 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T10:35:32.794 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-03-21T10:35:32.794 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:32.794 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:32.794 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-03-21T10:35:32.794 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-03-21T10:35:32.794 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-03-21T10:35:32.794 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-03-21T10:35:32.794 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-03-21T10:35:32.794 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-03-21T10:35:32.795 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-03-21T10:35:32.795 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-03-21T10:35:32.795 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T10:35:32.795 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:32.795 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:32.795 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:32.795 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:32.795 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster": { 2024-03-21T10:35:32.795 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster", 2024-03-21T10:35:32.795 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T10:35:32.795 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:32.795 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T10:35:32.795 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-03-21T10:35:32.795 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:32.796 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:32.796 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:32.796 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T10:35:32.796 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:32.796 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:32.796 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:32.796 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:32.796 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster_level": { 2024-03-21T10:35:32.796 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster_level", 2024-03-21T10:35:32.796 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T10:35:32.796 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:32.796 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T10:35:32.796 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "info", 2024-03-21T10:35:32.796 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:32.796 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:32.797 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-03-21T10:35:32.797 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-03-21T10:35:32.797 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-03-21T10:35:32.797 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-03-21T10:35:32.797 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-03-21T10:35:32.797 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-03-21T10:35:32.797 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-03-21T10:35:32.797 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-03-21T10:35:32.797 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T10:35:32.797 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:32.797 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:32.797 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:32.797 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:32.797 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_file": { 2024-03-21T10:35:32.797 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_file", 2024-03-21T10:35:32.797 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T10:35:32.798 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:32.798 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T10:35:32.798 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-03-21T10:35:32.798 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:32.798 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:32.798 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:32.798 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T10:35:32.798 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:32.798 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:32.798 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:32.798 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-21T10:35:32.798 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-21T10:35:32.798 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:32.799 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-03-21T10:35:32.799 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "osd_support", 2024-03-21T10:35:32.799 INFO:teuthology.orchestra.run.smithi047.stdout: "can_run": true, 2024-03-21T10:35:32.799 INFO:teuthology.orchestra.run.smithi047.stdout: "error_string": "", 2024-03-21T10:35:32.799 INFO:teuthology.orchestra.run.smithi047.stdout: "module_options": { 2024-03-21T10:35:32.799 INFO:teuthology.orchestra.run.smithi047.stdout: "log_level": { 2024-03-21T10:35:32.799 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_level", 2024-03-21T10:35:32.799 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T10:35:32.799 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:32.799 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T10:35:32.799 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-03-21T10:35:32.799 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:32.799 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:32.799 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-03-21T10:35:32.800 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-03-21T10:35:32.800 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-03-21T10:35:32.800 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-03-21T10:35:32.800 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-03-21T10:35:32.800 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-03-21T10:35:32.800 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-03-21T10:35:32.800 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-03-21T10:35:32.800 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T10:35:32.800 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:32.800 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:32.800 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:32.800 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:32.800 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster": { 2024-03-21T10:35:32.800 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster", 2024-03-21T10:35:32.801 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T10:35:32.801 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:32.801 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T10:35:32.801 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-03-21T10:35:32.801 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:32.801 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:32.801 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:32.801 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T10:35:32.801 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:32.801 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:32.801 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:32.801 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:32.801 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster_level": { 2024-03-21T10:35:32.801 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster_level", 2024-03-21T10:35:32.802 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T10:35:32.802 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:32.802 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T10:35:32.802 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "info", 2024-03-21T10:35:32.802 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:32.802 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:32.802 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-03-21T10:35:32.802 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-03-21T10:35:32.802 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-03-21T10:35:32.802 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-03-21T10:35:32.802 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-03-21T10:35:32.802 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-03-21T10:35:32.802 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-03-21T10:35:32.802 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-03-21T10:35:32.803 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T10:35:32.803 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:32.803 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:32.803 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:32.803 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:32.803 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_file": { 2024-03-21T10:35:32.803 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_file", 2024-03-21T10:35:32.803 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T10:35:32.803 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:32.803 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T10:35:32.803 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-03-21T10:35:32.803 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:32.803 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:32.804 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:32.804 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T10:35:32.804 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:32.804 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:32.804 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:32.804 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-21T10:35:32.804 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-21T10:35:32.804 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:32.804 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-03-21T10:35:32.804 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "pg_autoscaler", 2024-03-21T10:35:32.804 INFO:teuthology.orchestra.run.smithi047.stdout: "can_run": true, 2024-03-21T10:35:32.804 INFO:teuthology.orchestra.run.smithi047.stdout: "error_string": "", 2024-03-21T10:35:32.804 INFO:teuthology.orchestra.run.smithi047.stdout: "module_options": { 2024-03-21T10:35:32.805 INFO:teuthology.orchestra.run.smithi047.stdout: "log_level": { 2024-03-21T10:35:32.805 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_level", 2024-03-21T10:35:32.805 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T10:35:32.805 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:32.805 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T10:35:32.805 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-03-21T10:35:32.805 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:32.805 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:32.805 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-03-21T10:35:32.805 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-03-21T10:35:32.805 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-03-21T10:35:32.805 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-03-21T10:35:32.805 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-03-21T10:35:32.805 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-03-21T10:35:32.806 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-03-21T10:35:32.806 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-03-21T10:35:32.806 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T10:35:32.806 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:32.806 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:32.806 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:32.806 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:32.806 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster": { 2024-03-21T10:35:32.806 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster", 2024-03-21T10:35:32.806 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T10:35:32.806 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:32.806 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T10:35:32.806 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-03-21T10:35:32.806 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:32.807 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:32.807 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:32.807 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T10:35:32.807 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:32.807 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:32.807 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:32.807 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:32.807 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster_level": { 2024-03-21T10:35:32.807 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster_level", 2024-03-21T10:35:32.807 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T10:35:32.807 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:32.807 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T10:35:32.807 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "info", 2024-03-21T10:35:32.808 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:32.808 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:32.808 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-03-21T10:35:32.808 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-03-21T10:35:32.808 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-03-21T10:35:32.808 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-03-21T10:35:32.808 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-03-21T10:35:32.808 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-03-21T10:35:32.808 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-03-21T10:35:32.808 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-03-21T10:35:32.808 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T10:35:32.808 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:32.808 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:32.808 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:32.809 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:32.809 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_file": { 2024-03-21T10:35:32.809 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_file", 2024-03-21T10:35:32.809 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T10:35:32.809 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:32.809 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T10:35:32.809 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-03-21T10:35:32.809 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:32.809 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:32.809 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:32.809 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T10:35:32.809 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:32.809 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:32.809 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:32.809 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:32.810 INFO:teuthology.orchestra.run.smithi047.stdout: "sleep_interval": { 2024-03-21T10:35:32.810 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "sleep_interval", 2024-03-21T10:35:32.810 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "secs", 2024-03-21T10:35:32.810 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:32.810 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T10:35:32.810 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "60", 2024-03-21T10:35:32.810 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:32.810 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:32.810 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:32.810 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T10:35:32.810 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:32.810 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:32.810 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:32.810 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:32.811 INFO:teuthology.orchestra.run.smithi047.stdout: "threshold": { 2024-03-21T10:35:32.811 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "threshold", 2024-03-21T10:35:32.811 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "float", 2024-03-21T10:35:32.811 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:32.811 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T10:35:32.811 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "3.0", 2024-03-21T10:35:32.811 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "1.0", 2024-03-21T10:35:32.811 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:32.811 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:32.811 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "scaling threshold", 2024-03-21T10:35:32.811 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-03-21T10:35:32.811 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:32.811 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:32.811 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-21T10:35:32.811 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-21T10:35:32.812 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:32.812 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-03-21T10:35:32.812 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "progress", 2024-03-21T10:35:32.812 INFO:teuthology.orchestra.run.smithi047.stdout: "can_run": true, 2024-03-21T10:35:32.812 INFO:teuthology.orchestra.run.smithi047.stdout: "error_string": "", 2024-03-21T10:35:32.812 INFO:teuthology.orchestra.run.smithi047.stdout: "module_options": { 2024-03-21T10:35:32.812 INFO:teuthology.orchestra.run.smithi047.stdout: "allow_pg_recovery_event": { 2024-03-21T10:35:32.812 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "allow_pg_recovery_event", 2024-03-21T10:35:32.812 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T10:35:32.812 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:32.812 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T10:35:32.812 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-03-21T10:35:32.812 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:32.812 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:32.812 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:32.813 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "allow the module to show pg recovery progress", 2024-03-21T10:35:32.813 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:32.813 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:32.813 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:32.813 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:32.813 INFO:teuthology.orchestra.run.smithi047.stdout: "enabled": { 2024-03-21T10:35:32.813 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "enabled", 2024-03-21T10:35:32.813 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T10:35:32.813 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:32.813 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T10:35:32.813 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "True", 2024-03-21T10:35:32.813 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:32.813 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:32.813 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:32.813 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T10:35:32.813 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:32.814 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:32.814 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:32.814 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:32.814 INFO:teuthology.orchestra.run.smithi047.stdout: "log_level": { 2024-03-21T10:35:32.814 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_level", 2024-03-21T10:35:32.814 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T10:35:32.814 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:32.814 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T10:35:32.814 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-03-21T10:35:32.814 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:32.814 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:32.814 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-03-21T10:35:32.814 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-03-21T10:35:32.814 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-03-21T10:35:32.815 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-03-21T10:35:32.815 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-03-21T10:35:32.815 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-03-21T10:35:32.815 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-03-21T10:35:32.815 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-03-21T10:35:32.815 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T10:35:32.815 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:32.815 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:32.815 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:32.815 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:32.815 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster": { 2024-03-21T10:35:32.815 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster", 2024-03-21T10:35:32.815 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T10:35:32.815 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:32.815 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T10:35:32.816 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-03-21T10:35:32.816 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:32.816 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:32.816 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:32.816 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T10:35:32.816 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:32.816 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:32.816 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:32.816 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:32.816 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster_level": { 2024-03-21T10:35:32.816 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster_level", 2024-03-21T10:35:32.816 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T10:35:32.816 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:32.816 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T10:35:32.816 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "info", 2024-03-21T10:35:32.816 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:32.817 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:32.817 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-03-21T10:35:32.817 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-03-21T10:35:32.817 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-03-21T10:35:32.817 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-03-21T10:35:32.817 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-03-21T10:35:32.817 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-03-21T10:35:32.817 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-03-21T10:35:32.817 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-03-21T10:35:32.817 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T10:35:32.817 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:32.817 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:32.817 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:32.817 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:32.817 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_file": { 2024-03-21T10:35:32.818 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_file", 2024-03-21T10:35:32.818 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T10:35:32.818 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:32.818 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T10:35:32.818 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-03-21T10:35:32.818 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:32.818 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:32.818 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:32.818 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T10:35:32.818 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:32.818 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:32.818 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:32.818 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:32.818 INFO:teuthology.orchestra.run.smithi047.stdout: "max_completed_events": { 2024-03-21T10:35:32.818 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "max_completed_events", 2024-03-21T10:35:32.819 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "int", 2024-03-21T10:35:32.819 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:32.819 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T10:35:32.819 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "50", 2024-03-21T10:35:32.819 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:32.819 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:32.819 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:32.819 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "number of past completed events to remember", 2024-03-21T10:35:32.819 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:32.819 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:32.819 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:32.819 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:32.819 INFO:teuthology.orchestra.run.smithi047.stdout: "sleep_interval": { 2024-03-21T10:35:32.819 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "sleep_interval", 2024-03-21T10:35:32.820 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "secs", 2024-03-21T10:35:32.820 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:32.820 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T10:35:32.820 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "5", 2024-03-21T10:35:32.820 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:32.820 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:32.820 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:32.820 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "how long the module is going to sleep", 2024-03-21T10:35:32.820 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:32.820 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:32.820 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:32.820 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-21T10:35:32.820 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-21T10:35:32.820 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:32.820 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-03-21T10:35:32.821 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "prometheus", 2024-03-21T10:35:32.821 INFO:teuthology.orchestra.run.smithi047.stdout: "can_run": true, 2024-03-21T10:35:32.821 INFO:teuthology.orchestra.run.smithi047.stdout: "error_string": "", 2024-03-21T10:35:32.821 INFO:teuthology.orchestra.run.smithi047.stdout: "module_options": { 2024-03-21T10:35:32.821 INFO:teuthology.orchestra.run.smithi047.stdout: "cache": { 2024-03-21T10:35:32.821 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "cache", 2024-03-21T10:35:32.821 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T10:35:32.821 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:32.821 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T10:35:32.821 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "True", 2024-03-21T10:35:32.821 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:32.821 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:32.821 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:32.821 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T10:35:32.821 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:32.821 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:32.822 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:32.822 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:32.822 INFO:teuthology.orchestra.run.smithi047.stdout: "exclude_perf_counters": { 2024-03-21T10:35:32.822 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "exclude_perf_counters", 2024-03-21T10:35:32.822 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T10:35:32.822 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:32.822 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T10:35:32.822 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "True", 2024-03-21T10:35:32.822 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:32.822 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:32.822 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:32.822 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-03-21T10:35:32.822 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-03-21T10:35:32.823 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:32.823 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:32.823 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:32.823 INFO:teuthology.orchestra.run.smithi047.stdout: "log_level": { 2024-03-21T10:35:32.823 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_level", 2024-03-21T10:35:32.823 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T10:35:32.823 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:32.823 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T10:35:32.823 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-03-21T10:35:32.823 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:32.823 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:32.823 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-03-21T10:35:32.823 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-03-21T10:35:32.823 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-03-21T10:35:32.823 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-03-21T10:35:32.824 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-03-21T10:35:32.824 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-03-21T10:35:32.824 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-03-21T10:35:32.824 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-03-21T10:35:32.824 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T10:35:32.824 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:32.824 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:32.824 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:32.824 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:32.824 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster": { 2024-03-21T10:35:32.824 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster", 2024-03-21T10:35:32.824 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T10:35:32.824 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:32.824 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T10:35:32.824 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-03-21T10:35:32.825 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:32.825 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:32.825 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:32.825 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T10:35:32.825 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:32.825 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:32.825 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:32.825 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:32.825 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster_level": { 2024-03-21T10:35:32.825 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster_level", 2024-03-21T10:35:32.825 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T10:35:32.825 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:32.825 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T10:35:32.825 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "info", 2024-03-21T10:35:32.826 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:32.826 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:32.826 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-03-21T10:35:32.826 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-03-21T10:35:32.826 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-03-21T10:35:32.826 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-03-21T10:35:32.826 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-03-21T10:35:32.826 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-03-21T10:35:32.826 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-03-21T10:35:32.826 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-03-21T10:35:32.826 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T10:35:32.826 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:32.826 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:32.826 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:32.826 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:32.827 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_file": { 2024-03-21T10:35:32.827 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_file", 2024-03-21T10:35:32.827 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T10:35:32.827 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:32.827 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T10:35:32.827 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-03-21T10:35:32.827 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:32.827 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:32.827 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:32.827 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T10:35:32.827 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:32.827 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:32.827 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:32.827 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:32.827 INFO:teuthology.orchestra.run.smithi047.stdout: "rbd_stats_pools": { 2024-03-21T10:35:32.828 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "rbd_stats_pools", 2024-03-21T10:35:32.828 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T10:35:32.828 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:32.828 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T10:35:32.828 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-03-21T10:35:32.828 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:32.828 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:32.828 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:32.828 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T10:35:32.828 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:32.828 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:32.828 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:32.828 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:32.828 INFO:teuthology.orchestra.run.smithi047.stdout: "rbd_stats_pools_refresh_interval": { 2024-03-21T10:35:32.828 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-03-21T10:35:32.828 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "int", 2024-03-21T10:35:32.829 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:32.829 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T10:35:32.829 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "300", 2024-03-21T10:35:32.829 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:32.829 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:32.829 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:32.829 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T10:35:32.829 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:32.829 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:32.829 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:32.829 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:32.829 INFO:teuthology.orchestra.run.smithi047.stdout: "scrape_interval": { 2024-03-21T10:35:32.829 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "scrape_interval", 2024-03-21T10:35:32.829 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "float", 2024-03-21T10:35:32.829 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:32.830 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T10:35:32.830 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "15.0", 2024-03-21T10:35:32.830 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:32.830 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:32.830 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:32.830 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T10:35:32.830 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:32.830 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:32.830 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:32.830 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:32.830 INFO:teuthology.orchestra.run.smithi047.stdout: "server_addr": { 2024-03-21T10:35:32.830 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "server_addr", 2024-03-21T10:35:32.830 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T10:35:32.830 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:32.830 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T10:35:32.830 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "::", 2024-03-21T10:35:32.831 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:32.831 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:32.831 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:32.831 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-03-21T10:35:32.831 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:32.831 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:32.831 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:32.831 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:32.831 INFO:teuthology.orchestra.run.smithi047.stdout: "server_port": { 2024-03-21T10:35:32.831 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "server_port", 2024-03-21T10:35:32.831 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "int", 2024-03-21T10:35:32.831 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:32.831 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T10:35:32.831 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "9283", 2024-03-21T10:35:32.832 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:32.832 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:32.832 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:32.832 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-03-21T10:35:32.832 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:32.832 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:32.832 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:32.832 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:32.832 INFO:teuthology.orchestra.run.smithi047.stdout: "stale_cache_strategy": { 2024-03-21T10:35:32.832 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "stale_cache_strategy", 2024-03-21T10:35:32.832 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T10:35:32.832 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:32.832 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T10:35:32.832 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "log", 2024-03-21T10:35:32.832 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:32.833 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:32.833 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:32.833 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T10:35:32.833 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:32.833 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:32.833 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:32.833 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:32.833 INFO:teuthology.orchestra.run.smithi047.stdout: "standby_behaviour": { 2024-03-21T10:35:32.833 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "standby_behaviour", 2024-03-21T10:35:32.833 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T10:35:32.833 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:32.833 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T10:35:32.833 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "default", 2024-03-21T10:35:32.833 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:32.834 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:32.834 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-03-21T10:35:32.834 INFO:teuthology.orchestra.run.smithi047.stdout: "default", 2024-03-21T10:35:32.834 INFO:teuthology.orchestra.run.smithi047.stdout: "error" 2024-03-21T10:35:32.834 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-03-21T10:35:32.834 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T10:35:32.834 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:32.834 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:32.834 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:32.834 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:32.834 INFO:teuthology.orchestra.run.smithi047.stdout: "standby_error_status_code": { 2024-03-21T10:35:32.834 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "standby_error_status_code", 2024-03-21T10:35:32.834 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "int", 2024-03-21T10:35:32.834 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:32.835 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T10:35:32.835 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "500", 2024-03-21T10:35:32.835 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "400", 2024-03-21T10:35:32.835 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "599", 2024-03-21T10:35:32.835 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:32.835 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T10:35:32.835 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:32.835 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:32.835 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:32.835 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-21T10:35:32.835 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-21T10:35:32.835 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:32.835 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-03-21T10:35:32.835 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "rbd_support", 2024-03-21T10:35:32.835 INFO:teuthology.orchestra.run.smithi047.stdout: "can_run": true, 2024-03-21T10:35:32.836 INFO:teuthology.orchestra.run.smithi047.stdout: "error_string": "", 2024-03-21T10:35:32.836 INFO:teuthology.orchestra.run.smithi047.stdout: "module_options": { 2024-03-21T10:35:32.836 INFO:teuthology.orchestra.run.smithi047.stdout: "log_level": { 2024-03-21T10:35:32.836 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_level", 2024-03-21T10:35:32.836 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T10:35:32.836 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:32.836 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T10:35:32.836 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-03-21T10:35:32.836 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:32.836 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:32.836 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-03-21T10:35:32.836 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-03-21T10:35:32.836 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-03-21T10:35:32.836 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-03-21T10:35:32.836 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-03-21T10:35:32.836 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-03-21T10:35:32.837 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-03-21T10:35:32.837 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-03-21T10:35:32.837 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T10:35:32.837 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:32.837 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:32.837 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:32.837 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:32.837 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster": { 2024-03-21T10:35:32.837 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster", 2024-03-21T10:35:32.837 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T10:35:32.837 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:32.837 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T10:35:32.837 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-03-21T10:35:32.837 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:32.837 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:32.838 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:32.838 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T10:35:32.838 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:32.838 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:32.838 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:32.838 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:32.838 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster_level": { 2024-03-21T10:35:32.838 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster_level", 2024-03-21T10:35:32.838 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T10:35:32.838 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:32.838 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T10:35:32.838 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "info", 2024-03-21T10:35:32.838 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:32.838 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:32.839 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-03-21T10:35:32.839 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-03-21T10:35:32.839 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-03-21T10:35:32.839 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-03-21T10:35:32.839 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-03-21T10:35:32.839 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-03-21T10:35:32.839 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-03-21T10:35:32.839 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-03-21T10:35:32.839 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T10:35:32.839 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:32.839 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:32.839 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:32.839 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:32.839 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_file": { 2024-03-21T10:35:32.839 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_file", 2024-03-21T10:35:32.840 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T10:35:32.840 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:32.840 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T10:35:32.840 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-03-21T10:35:32.840 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:32.840 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:32.840 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:32.840 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T10:35:32.840 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:32.840 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:32.840 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:32.840 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:32.840 INFO:teuthology.orchestra.run.smithi047.stdout: "max_concurrent_snap_create": { 2024-03-21T10:35:32.840 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "max_concurrent_snap_create", 2024-03-21T10:35:32.840 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "int", 2024-03-21T10:35:32.841 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:32.841 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T10:35:32.841 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "10", 2024-03-21T10:35:32.841 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:32.841 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:32.841 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:32.841 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T10:35:32.841 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:32.841 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:32.841 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:32.841 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:32.841 INFO:teuthology.orchestra.run.smithi047.stdout: "mirror_snapshot_schedule": { 2024-03-21T10:35:32.841 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "mirror_snapshot_schedule", 2024-03-21T10:35:32.841 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T10:35:32.841 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:32.842 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T10:35:32.842 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-03-21T10:35:32.842 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:32.842 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:32.842 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:32.842 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T10:35:32.842 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:32.842 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:32.842 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:32.842 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:32.842 INFO:teuthology.orchestra.run.smithi047.stdout: "trash_purge_schedule": { 2024-03-21T10:35:32.842 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "trash_purge_schedule", 2024-03-21T10:35:32.842 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T10:35:32.842 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:32.842 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T10:35:32.843 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-03-21T10:35:32.843 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:32.843 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:32.843 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:32.843 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T10:35:32.843 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:32.843 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:32.843 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:32.843 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-21T10:35:32.843 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-21T10:35:32.843 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:32.843 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-03-21T10:35:32.843 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "restful", 2024-03-21T10:35:32.843 INFO:teuthology.orchestra.run.smithi047.stdout: "can_run": true, 2024-03-21T10:35:32.843 INFO:teuthology.orchestra.run.smithi047.stdout: "error_string": "", 2024-03-21T10:35:32.844 INFO:teuthology.orchestra.run.smithi047.stdout: "module_options": { 2024-03-21T10:35:32.844 INFO:teuthology.orchestra.run.smithi047.stdout: "enable_auth": { 2024-03-21T10:35:32.844 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "enable_auth", 2024-03-21T10:35:32.844 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T10:35:32.844 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:32.844 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T10:35:32.844 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "True", 2024-03-21T10:35:32.844 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:32.844 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:32.844 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:32.844 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T10:35:32.844 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:32.844 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:32.844 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:32.845 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:32.845 INFO:teuthology.orchestra.run.smithi047.stdout: "key_file": { 2024-03-21T10:35:32.845 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "key_file", 2024-03-21T10:35:32.845 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T10:35:32.845 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:32.845 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T10:35:32.845 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-03-21T10:35:32.845 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:32.845 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:32.845 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:32.845 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T10:35:32.845 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:32.846 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:32.846 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:32.846 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:32.846 INFO:teuthology.orchestra.run.smithi047.stdout: "log_level": { 2024-03-21T10:35:32.846 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_level", 2024-03-21T10:35:32.846 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T10:35:32.846 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:32.846 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T10:35:32.846 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-03-21T10:35:32.846 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:32.846 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:32.846 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-03-21T10:35:32.846 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-03-21T10:35:32.847 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-03-21T10:35:32.847 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-03-21T10:35:32.847 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-03-21T10:35:32.847 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-03-21T10:35:32.847 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-03-21T10:35:32.847 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-03-21T10:35:32.847 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T10:35:32.847 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:32.847 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:32.847 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:32.847 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:32.847 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster": { 2024-03-21T10:35:32.847 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster", 2024-03-21T10:35:32.847 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T10:35:32.847 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:32.848 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T10:35:32.848 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-03-21T10:35:32.848 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:32.848 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:32.848 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:32.848 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T10:35:32.848 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:32.848 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:32.848 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:32.848 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:32.848 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster_level": { 2024-03-21T10:35:32.848 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster_level", 2024-03-21T10:35:32.848 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T10:35:32.849 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:32.849 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T10:35:32.849 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "info", 2024-03-21T10:35:32.849 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:32.849 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:32.849 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-03-21T10:35:32.849 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-03-21T10:35:32.849 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-03-21T10:35:32.849 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-03-21T10:35:32.849 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-03-21T10:35:32.849 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-03-21T10:35:32.849 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-03-21T10:35:32.849 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-03-21T10:35:32.849 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T10:35:32.850 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:32.850 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:32.850 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:32.850 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:32.850 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_file": { 2024-03-21T10:35:32.850 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_file", 2024-03-21T10:35:32.850 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T10:35:32.850 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:32.850 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T10:35:32.850 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-03-21T10:35:32.850 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:32.850 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:32.850 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:32.850 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T10:35:32.851 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:32.851 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:32.851 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:32.851 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:32.851 INFO:teuthology.orchestra.run.smithi047.stdout: "server_addr": { 2024-03-21T10:35:32.851 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "server_addr", 2024-03-21T10:35:32.851 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T10:35:32.851 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:32.851 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T10:35:32.851 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-03-21T10:35:32.851 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:32.851 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:32.851 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:32.851 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T10:35:32.852 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:32.852 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:32.852 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:32.852 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:32.852 INFO:teuthology.orchestra.run.smithi047.stdout: "server_port": { 2024-03-21T10:35:32.852 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "server_port", 2024-03-21T10:35:32.852 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T10:35:32.852 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:32.852 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T10:35:32.852 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-03-21T10:35:32.852 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:32.852 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:32.852 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:32.852 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T10:35:32.852 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:32.853 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:32.853 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:32.853 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-21T10:35:32.853 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-21T10:35:32.853 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:32.853 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-03-21T10:35:32.853 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "rgw", 2024-03-21T10:35:32.853 INFO:teuthology.orchestra.run.smithi047.stdout: "can_run": true, 2024-03-21T10:35:32.853 INFO:teuthology.orchestra.run.smithi047.stdout: "error_string": "", 2024-03-21T10:35:32.853 INFO:teuthology.orchestra.run.smithi047.stdout: "module_options": { 2024-03-21T10:35:32.853 INFO:teuthology.orchestra.run.smithi047.stdout: "log_level": { 2024-03-21T10:35:32.853 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_level", 2024-03-21T10:35:32.853 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T10:35:32.853 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:32.854 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T10:35:32.854 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-03-21T10:35:32.854 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:32.854 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:32.854 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-03-21T10:35:32.854 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-03-21T10:35:32.854 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-03-21T10:35:32.854 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-03-21T10:35:32.854 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-03-21T10:35:32.854 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-03-21T10:35:32.854 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-03-21T10:35:32.854 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-03-21T10:35:32.854 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T10:35:32.854 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:32.854 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:32.855 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:32.855 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:32.855 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster": { 2024-03-21T10:35:32.855 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster", 2024-03-21T10:35:32.855 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T10:35:32.855 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:32.855 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T10:35:32.855 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-03-21T10:35:32.855 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:32.855 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:32.855 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:32.855 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T10:35:32.855 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:32.855 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:32.855 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:32.856 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:32.856 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster_level": { 2024-03-21T10:35:32.856 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster_level", 2024-03-21T10:35:32.856 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T10:35:32.856 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:32.856 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T10:35:32.856 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "info", 2024-03-21T10:35:32.856 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:32.856 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:32.856 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-03-21T10:35:32.856 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-03-21T10:35:32.856 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-03-21T10:35:32.856 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-03-21T10:35:32.856 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-03-21T10:35:32.856 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-03-21T10:35:32.857 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-03-21T10:35:32.857 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-03-21T10:35:32.857 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T10:35:32.857 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:32.857 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:32.857 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:32.857 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:32.857 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_file": { 2024-03-21T10:35:32.857 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_file", 2024-03-21T10:35:32.857 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T10:35:32.857 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:32.857 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T10:35:32.857 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-03-21T10:35:32.857 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:32.857 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:32.858 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:32.858 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T10:35:32.858 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:32.858 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:32.858 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:32.858 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-21T10:35:32.858 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-21T10:35:32.858 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:32.858 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-03-21T10:35:32.858 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "rook", 2024-03-21T10:35:32.858 INFO:teuthology.orchestra.run.smithi047.stdout: "can_run": true, 2024-03-21T10:35:32.858 INFO:teuthology.orchestra.run.smithi047.stdout: "error_string": "", 2024-03-21T10:35:32.858 INFO:teuthology.orchestra.run.smithi047.stdout: "module_options": { 2024-03-21T10:35:32.859 INFO:teuthology.orchestra.run.smithi047.stdout: "log_level": { 2024-03-21T10:35:32.859 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_level", 2024-03-21T10:35:32.859 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T10:35:32.859 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:32.859 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T10:35:32.859 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-03-21T10:35:32.859 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:32.859 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:32.859 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-03-21T10:35:32.859 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-03-21T10:35:32.859 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-03-21T10:35:32.859 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-03-21T10:35:32.859 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-03-21T10:35:32.859 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-03-21T10:35:32.859 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-03-21T10:35:32.860 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-03-21T10:35:32.860 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T10:35:32.860 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:32.860 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:32.860 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:32.860 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:32.860 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster": { 2024-03-21T10:35:32.860 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster", 2024-03-21T10:35:32.860 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T10:35:32.860 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:32.860 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T10:35:32.860 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-03-21T10:35:32.860 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:32.860 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:32.860 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:32.860 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T10:35:32.861 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:32.861 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:32.861 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:32.861 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:32.861 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster_level": { 2024-03-21T10:35:32.861 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster_level", 2024-03-21T10:35:32.861 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T10:35:32.861 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:32.861 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T10:35:32.861 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "info", 2024-03-21T10:35:32.861 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:32.861 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:32.861 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-03-21T10:35:32.861 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-03-21T10:35:32.861 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-03-21T10:35:32.862 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-03-21T10:35:32.862 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-03-21T10:35:32.862 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-03-21T10:35:32.862 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-03-21T10:35:32.862 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-03-21T10:35:32.862 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T10:35:32.862 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:32.862 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:32.862 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:32.862 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:32.862 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_file": { 2024-03-21T10:35:32.862 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_file", 2024-03-21T10:35:32.862 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T10:35:32.863 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:32.863 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T10:35:32.863 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-03-21T10:35:32.863 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:32.863 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:32.863 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:32.863 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T10:35:32.863 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:32.863 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:32.863 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:32.863 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:32.863 INFO:teuthology.orchestra.run.smithi047.stdout: "storage_class": { 2024-03-21T10:35:32.863 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "storage_class", 2024-03-21T10:35:32.863 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T10:35:32.863 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:32.864 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T10:35:32.864 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "local", 2024-03-21T10:35:32.864 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:32.864 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:32.864 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:32.864 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-03-21T10:35:32.864 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:32.864 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:32.864 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:32.864 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-21T10:35:32.864 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-21T10:35:32.864 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:32.864 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-03-21T10:35:32.864 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "selftest", 2024-03-21T10:35:32.864 INFO:teuthology.orchestra.run.smithi047.stdout: "can_run": true, 2024-03-21T10:35:32.865 INFO:teuthology.orchestra.run.smithi047.stdout: "error_string": "", 2024-03-21T10:35:32.865 INFO:teuthology.orchestra.run.smithi047.stdout: "module_options": { 2024-03-21T10:35:32.865 INFO:teuthology.orchestra.run.smithi047.stdout: "log_level": { 2024-03-21T10:35:32.865 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_level", 2024-03-21T10:35:32.865 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T10:35:32.865 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:32.865 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T10:35:32.865 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-03-21T10:35:32.865 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:32.865 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:32.865 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-03-21T10:35:32.865 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-03-21T10:35:32.865 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-03-21T10:35:32.865 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-03-21T10:35:32.865 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-03-21T10:35:32.865 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-03-21T10:35:32.865 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-03-21T10:35:32.866 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-03-21T10:35:32.866 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T10:35:32.866 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:32.866 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:32.866 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:32.866 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:32.866 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster": { 2024-03-21T10:35:32.866 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster", 2024-03-21T10:35:32.866 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T10:35:32.866 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:32.866 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T10:35:32.866 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-03-21T10:35:32.866 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:32.866 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:32.866 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:32.867 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T10:35:32.867 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:32.867 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:32.867 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:32.867 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:32.867 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster_level": { 2024-03-21T10:35:32.867 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster_level", 2024-03-21T10:35:32.867 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T10:35:32.867 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:32.867 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T10:35:32.867 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "info", 2024-03-21T10:35:32.867 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:32.867 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:32.867 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-03-21T10:35:32.867 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-03-21T10:35:32.868 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-03-21T10:35:32.868 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-03-21T10:35:32.868 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-03-21T10:35:32.868 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-03-21T10:35:32.868 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-03-21T10:35:32.868 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-03-21T10:35:32.868 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T10:35:32.868 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:32.868 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:32.868 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:32.868 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:32.868 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_file": { 2024-03-21T10:35:32.868 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_file", 2024-03-21T10:35:32.868 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T10:35:32.868 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:32.868 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T10:35:32.869 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-03-21T10:35:32.869 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:32.869 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:32.869 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:32.869 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T10:35:32.869 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:32.869 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:32.869 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:32.869 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:32.869 INFO:teuthology.orchestra.run.smithi047.stdout: "roption1": { 2024-03-21T10:35:32.869 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "roption1", 2024-03-21T10:35:32.869 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T10:35:32.869 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:32.869 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T10:35:32.869 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-03-21T10:35:32.870 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:32.870 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:32.870 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:32.870 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T10:35:32.870 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:32.870 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:32.870 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:32.870 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:32.870 INFO:teuthology.orchestra.run.smithi047.stdout: "roption2": { 2024-03-21T10:35:32.870 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "roption2", 2024-03-21T10:35:32.870 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T10:35:32.870 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:32.870 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T10:35:32.870 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "xyz", 2024-03-21T10:35:32.871 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:32.871 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:32.871 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:32.871 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T10:35:32.871 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:32.871 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:32.871 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:32.871 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:32.871 INFO:teuthology.orchestra.run.smithi047.stdout: "rwoption1": { 2024-03-21T10:35:32.871 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "rwoption1", 2024-03-21T10:35:32.871 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T10:35:32.871 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:32.871 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T10:35:32.871 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-03-21T10:35:32.872 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:32.872 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:32.872 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:32.872 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T10:35:32.872 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:32.872 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:32.872 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:32.872 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:32.872 INFO:teuthology.orchestra.run.smithi047.stdout: "rwoption2": { 2024-03-21T10:35:32.872 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "rwoption2", 2024-03-21T10:35:32.872 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "int", 2024-03-21T10:35:32.872 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:32.872 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T10:35:32.872 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-03-21T10:35:32.872 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:32.873 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:32.873 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:32.873 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T10:35:32.873 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:32.873 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:32.873 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:32.873 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:32.873 INFO:teuthology.orchestra.run.smithi047.stdout: "rwoption3": { 2024-03-21T10:35:32.873 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "rwoption3", 2024-03-21T10:35:32.873 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "float", 2024-03-21T10:35:32.873 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:32.873 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T10:35:32.873 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-03-21T10:35:32.873 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:32.874 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:32.874 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:32.874 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T10:35:32.874 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:32.874 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:32.874 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:32.874 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:32.874 INFO:teuthology.orchestra.run.smithi047.stdout: "rwoption4": { 2024-03-21T10:35:32.874 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "rwoption4", 2024-03-21T10:35:32.874 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T10:35:32.874 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:32.874 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T10:35:32.874 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-03-21T10:35:32.874 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:32.875 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:32.875 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:32.875 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T10:35:32.875 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:32.875 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:32.875 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:32.875 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:32.875 INFO:teuthology.orchestra.run.smithi047.stdout: "rwoption5": { 2024-03-21T10:35:32.875 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "rwoption5", 2024-03-21T10:35:32.875 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T10:35:32.875 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:32.875 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T10:35:32.875 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-03-21T10:35:32.875 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:32.875 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:32.875 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:32.876 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T10:35:32.876 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:32.876 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:32.876 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:32.876 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:32.876 INFO:teuthology.orchestra.run.smithi047.stdout: "rwoption6": { 2024-03-21T10:35:32.876 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "rwoption6", 2024-03-21T10:35:32.876 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T10:35:32.876 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:32.876 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T10:35:32.876 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "True", 2024-03-21T10:35:32.876 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:32.876 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:32.876 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:32.876 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T10:35:32.877 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:32.877 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:32.877 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:32.877 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:32.877 INFO:teuthology.orchestra.run.smithi047.stdout: "rwoption7": { 2024-03-21T10:35:32.877 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "rwoption7", 2024-03-21T10:35:32.877 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "int", 2024-03-21T10:35:32.877 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:32.877 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T10:35:32.877 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-03-21T10:35:32.877 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "1", 2024-03-21T10:35:32.877 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "42", 2024-03-21T10:35:32.877 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:32.877 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T10:35:32.877 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:32.877 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:32.878 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:32.878 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:32.878 INFO:teuthology.orchestra.run.smithi047.stdout: "testkey": { 2024-03-21T10:35:32.878 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "testkey", 2024-03-21T10:35:32.878 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T10:35:32.878 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:32.878 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T10:35:32.878 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-03-21T10:35:32.878 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:32.878 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:32.878 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:32.878 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T10:35:32.878 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:32.878 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:32.879 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:32.879 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:32.879 INFO:teuthology.orchestra.run.smithi047.stdout: "testlkey": { 2024-03-21T10:35:32.879 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "testlkey", 2024-03-21T10:35:32.879 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T10:35:32.879 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:32.879 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T10:35:32.879 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-03-21T10:35:32.879 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:32.879 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:32.879 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:32.879 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T10:35:32.879 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:32.879 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:32.879 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:32.880 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:32.880 INFO:teuthology.orchestra.run.smithi047.stdout: "testnewline": { 2024-03-21T10:35:32.880 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "testnewline", 2024-03-21T10:35:32.880 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T10:35:32.880 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:32.880 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T10:35:32.880 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-03-21T10:35:32.880 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:32.880 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:32.880 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:32.880 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T10:35:32.880 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:32.880 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:32.880 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:32.880 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-21T10:35:32.881 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-21T10:35:32.881 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:32.881 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-03-21T10:35:32.881 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "snap_schedule", 2024-03-21T10:35:32.881 INFO:teuthology.orchestra.run.smithi047.stdout: "can_run": true, 2024-03-21T10:35:32.881 INFO:teuthology.orchestra.run.smithi047.stdout: "error_string": "", 2024-03-21T10:35:32.881 INFO:teuthology.orchestra.run.smithi047.stdout: "module_options": { 2024-03-21T10:35:32.881 INFO:teuthology.orchestra.run.smithi047.stdout: "allow_m_granularity": { 2024-03-21T10:35:32.881 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "allow_m_granularity", 2024-03-21T10:35:32.881 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T10:35:32.881 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:32.881 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T10:35:32.881 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-03-21T10:35:32.881 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:32.881 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:32.882 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:32.882 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "allow minute scheduled snapshots", 2024-03-21T10:35:32.882 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:32.882 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:32.882 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:32.882 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:32.882 INFO:teuthology.orchestra.run.smithi047.stdout: "dump_on_update": { 2024-03-21T10:35:32.882 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "dump_on_update", 2024-03-21T10:35:32.882 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T10:35:32.882 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:32.882 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T10:35:32.882 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-03-21T10:35:32.882 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:32.883 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:32.883 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:32.883 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "dump database to debug log on update", 2024-03-21T10:35:32.883 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:32.883 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:32.883 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:32.883 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:32.883 INFO:teuthology.orchestra.run.smithi047.stdout: "log_level": { 2024-03-21T10:35:32.883 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_level", 2024-03-21T10:35:32.883 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T10:35:32.883 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:32.883 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T10:35:32.883 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-03-21T10:35:32.883 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:32.884 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:32.884 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-03-21T10:35:32.884 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-03-21T10:35:32.884 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-03-21T10:35:32.884 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-03-21T10:35:32.884 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-03-21T10:35:32.884 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-03-21T10:35:32.884 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-03-21T10:35:32.884 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-03-21T10:35:32.884 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T10:35:32.884 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:32.884 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:32.884 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:32.884 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:32.884 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster": { 2024-03-21T10:35:32.885 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster", 2024-03-21T10:35:32.885 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T10:35:32.885 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:32.885 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T10:35:32.885 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-03-21T10:35:32.885 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:32.885 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:32.885 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:32.885 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T10:35:32.885 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:32.885 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:32.885 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:32.885 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:32.885 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster_level": { 2024-03-21T10:35:32.885 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster_level", 2024-03-21T10:35:32.886 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T10:35:32.886 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:32.886 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T10:35:32.886 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "info", 2024-03-21T10:35:32.886 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:32.886 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:32.886 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-03-21T10:35:32.886 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-03-21T10:35:32.886 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-03-21T10:35:32.886 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-03-21T10:35:32.886 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-03-21T10:35:32.886 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-03-21T10:35:32.886 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-03-21T10:35:32.887 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-03-21T10:35:32.887 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T10:35:32.887 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:32.887 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:32.887 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:32.887 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:32.887 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_file": { 2024-03-21T10:35:32.887 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_file", 2024-03-21T10:35:32.887 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T10:35:32.887 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:32.887 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T10:35:32.887 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-03-21T10:35:32.887 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:32.887 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:32.888 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:32.888 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T10:35:32.888 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:32.888 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:32.888 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:32.888 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-21T10:35:32.888 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-21T10:35:32.888 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:32.888 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-03-21T10:35:32.888 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "stats", 2024-03-21T10:35:32.888 INFO:teuthology.orchestra.run.smithi047.stdout: "can_run": true, 2024-03-21T10:35:32.888 INFO:teuthology.orchestra.run.smithi047.stdout: "error_string": "", 2024-03-21T10:35:32.888 INFO:teuthology.orchestra.run.smithi047.stdout: "module_options": { 2024-03-21T10:35:32.888 INFO:teuthology.orchestra.run.smithi047.stdout: "log_level": { 2024-03-21T10:35:32.888 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_level", 2024-03-21T10:35:32.889 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T10:35:32.889 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:32.889 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T10:35:32.889 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-03-21T10:35:32.889 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:32.889 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:32.889 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-03-21T10:35:32.889 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-03-21T10:35:32.889 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-03-21T10:35:32.889 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-03-21T10:35:32.889 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-03-21T10:35:32.889 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-03-21T10:35:32.889 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-03-21T10:35:32.889 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-03-21T10:35:32.889 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T10:35:32.890 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:32.890 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:32.890 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:32.890 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:32.890 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster": { 2024-03-21T10:35:32.890 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster", 2024-03-21T10:35:32.890 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T10:35:32.890 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:32.890 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T10:35:32.890 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-03-21T10:35:32.890 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:32.890 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:32.890 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:32.891 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T10:35:32.891 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:32.891 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:32.891 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:32.891 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:32.891 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster_level": { 2024-03-21T10:35:32.891 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster_level", 2024-03-21T10:35:32.891 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T10:35:32.891 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:32.891 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T10:35:32.891 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "info", 2024-03-21T10:35:32.891 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:32.891 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:32.891 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-03-21T10:35:32.892 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-03-21T10:35:32.892 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-03-21T10:35:32.892 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-03-21T10:35:32.892 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-03-21T10:35:32.892 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-03-21T10:35:32.892 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-03-21T10:35:32.892 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-03-21T10:35:32.892 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T10:35:32.892 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:32.892 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:32.892 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:32.892 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:32.892 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_file": { 2024-03-21T10:35:32.892 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_file", 2024-03-21T10:35:32.892 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T10:35:32.893 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:32.893 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T10:35:32.893 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-03-21T10:35:32.893 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:32.893 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:32.893 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:32.893 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T10:35:32.893 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:32.893 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:32.893 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:32.893 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-21T10:35:32.893 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-21T10:35:32.893 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:32.893 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-03-21T10:35:32.894 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "status", 2024-03-21T10:35:32.894 INFO:teuthology.orchestra.run.smithi047.stdout: "can_run": true, 2024-03-21T10:35:32.894 INFO:teuthology.orchestra.run.smithi047.stdout: "error_string": "", 2024-03-21T10:35:32.894 INFO:teuthology.orchestra.run.smithi047.stdout: "module_options": { 2024-03-21T10:35:32.894 INFO:teuthology.orchestra.run.smithi047.stdout: "log_level": { 2024-03-21T10:35:32.894 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_level", 2024-03-21T10:35:32.894 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T10:35:32.894 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:32.894 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T10:35:32.894 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-03-21T10:35:32.894 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:32.894 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:32.894 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-03-21T10:35:32.895 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-03-21T10:35:32.895 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-03-21T10:35:32.895 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-03-21T10:35:32.895 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-03-21T10:35:32.895 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-03-21T10:35:32.895 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-03-21T10:35:32.895 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-03-21T10:35:32.895 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T10:35:32.895 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:32.895 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:32.895 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:32.895 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:32.895 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster": { 2024-03-21T10:35:32.895 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster", 2024-03-21T10:35:32.895 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T10:35:32.896 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:32.896 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T10:35:32.896 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-03-21T10:35:32.896 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:32.898 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:32.898 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:32.898 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T10:35:32.898 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:32.898 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:32.898 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:32.898 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:32.898 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster_level": { 2024-03-21T10:35:32.898 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster_level", 2024-03-21T10:35:32.898 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T10:35:32.898 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:32.898 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T10:35:32.899 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "info", 2024-03-21T10:35:32.899 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:32.899 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:32.899 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-03-21T10:35:32.899 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-03-21T10:35:32.899 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-03-21T10:35:32.899 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-03-21T10:35:32.899 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-03-21T10:35:32.899 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-03-21T10:35:32.899 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-03-21T10:35:32.899 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-03-21T10:35:32.899 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T10:35:32.899 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:32.900 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:32.900 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:32.900 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:32.900 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_file": { 2024-03-21T10:35:32.900 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_file", 2024-03-21T10:35:32.900 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T10:35:32.900 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:32.900 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T10:35:32.900 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-03-21T10:35:32.900 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:32.900 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:32.900 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:32.900 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T10:35:32.900 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:32.901 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:32.901 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:32.901 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-21T10:35:32.901 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-21T10:35:32.901 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:32.901 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-03-21T10:35:32.901 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "telegraf", 2024-03-21T10:35:32.901 INFO:teuthology.orchestra.run.smithi047.stdout: "can_run": true, 2024-03-21T10:35:32.901 INFO:teuthology.orchestra.run.smithi047.stdout: "error_string": "", 2024-03-21T10:35:32.901 INFO:teuthology.orchestra.run.smithi047.stdout: "module_options": { 2024-03-21T10:35:32.901 INFO:teuthology.orchestra.run.smithi047.stdout: "address": { 2024-03-21T10:35:32.901 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "address", 2024-03-21T10:35:32.902 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T10:35:32.902 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:32.902 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T10:35:32.902 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-03-21T10:35:32.902 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:32.902 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:32.902 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:32.902 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T10:35:32.902 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:32.902 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:32.902 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:32.902 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:32.902 INFO:teuthology.orchestra.run.smithi047.stdout: "interval": { 2024-03-21T10:35:32.902 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "interval", 2024-03-21T10:35:32.903 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "secs", 2024-03-21T10:35:32.903 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:32.903 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T10:35:32.903 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "15", 2024-03-21T10:35:32.903 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:32.903 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:32.903 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:32.903 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T10:35:32.903 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:32.903 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:32.903 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:32.903 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:32.903 INFO:teuthology.orchestra.run.smithi047.stdout: "log_level": { 2024-03-21T10:35:32.903 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_level", 2024-03-21T10:35:32.904 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T10:35:32.904 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:32.904 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T10:35:32.904 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-03-21T10:35:32.904 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:32.904 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:32.904 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-03-21T10:35:32.904 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-03-21T10:35:32.904 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-03-21T10:35:32.904 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-03-21T10:35:32.904 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-03-21T10:35:32.904 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-03-21T10:35:32.904 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-03-21T10:35:32.904 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-03-21T10:35:32.904 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T10:35:32.905 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:32.905 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:32.905 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:32.905 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:32.905 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster": { 2024-03-21T10:35:32.905 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster", 2024-03-21T10:35:32.905 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T10:35:32.905 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:32.905 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T10:35:32.905 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-03-21T10:35:32.905 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:32.905 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:32.905 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:32.905 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T10:35:32.905 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:32.906 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:32.906 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:32.906 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:32.906 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster_level": { 2024-03-21T10:35:32.906 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster_level", 2024-03-21T10:35:32.906 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T10:35:32.906 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:32.906 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T10:35:32.906 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "info", 2024-03-21T10:35:32.906 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:32.906 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:32.906 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-03-21T10:35:32.906 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-03-21T10:35:32.906 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-03-21T10:35:32.907 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-03-21T10:35:32.907 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-03-21T10:35:32.907 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-03-21T10:35:32.907 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-03-21T10:35:32.907 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-03-21T10:35:32.907 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T10:35:32.907 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:32.907 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:32.907 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:32.907 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:32.907 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_file": { 2024-03-21T10:35:32.907 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_file", 2024-03-21T10:35:32.907 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T10:35:32.907 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:32.907 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T10:35:32.908 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-03-21T10:35:32.908 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:32.908 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:32.908 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:32.908 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T10:35:32.908 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:32.908 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:32.908 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:32.908 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-21T10:35:32.908 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-21T10:35:32.908 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:32.908 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-03-21T10:35:32.908 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "telemetry", 2024-03-21T10:35:32.908 INFO:teuthology.orchestra.run.smithi047.stdout: "can_run": true, 2024-03-21T10:35:32.908 INFO:teuthology.orchestra.run.smithi047.stdout: "error_string": "", 2024-03-21T10:35:32.909 INFO:teuthology.orchestra.run.smithi047.stdout: "module_options": { 2024-03-21T10:35:32.909 INFO:teuthology.orchestra.run.smithi047.stdout: "channel_basic": { 2024-03-21T10:35:32.909 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "channel_basic", 2024-03-21T10:35:32.909 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T10:35:32.909 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:32.909 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T10:35:32.909 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "True", 2024-03-21T10:35:32.909 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:32.909 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:32.909 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:32.909 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Share basic cluster information (size, version)", 2024-03-21T10:35:32.909 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:32.909 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:32.909 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:32.909 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:32.909 INFO:teuthology.orchestra.run.smithi047.stdout: "channel_crash": { 2024-03-21T10:35:32.910 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "channel_crash", 2024-03-21T10:35:32.910 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T10:35:32.910 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:32.910 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T10:35:32.910 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "True", 2024-03-21T10:35:32.910 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:32.910 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:32.910 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:32.910 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-03-21T10:35:32.910 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:32.910 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:32.910 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:32.910 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:32.911 INFO:teuthology.orchestra.run.smithi047.stdout: "channel_device": { 2024-03-21T10:35:32.911 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "channel_device", 2024-03-21T10:35:32.911 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T10:35:32.911 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:32.911 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T10:35:32.911 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "True", 2024-03-21T10:35:32.911 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:32.911 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:32.911 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:32.911 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-03-21T10:35:32.911 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:32.911 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:32.911 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:32.911 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:32.912 INFO:teuthology.orchestra.run.smithi047.stdout: "channel_ident": { 2024-03-21T10:35:32.912 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "channel_ident", 2024-03-21T10:35:32.912 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T10:35:32.912 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:32.912 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T10:35:32.912 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-03-21T10:35:32.912 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:32.912 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:32.912 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:32.912 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-03-21T10:35:32.912 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:32.912 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:32.912 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:32.912 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:32.913 INFO:teuthology.orchestra.run.smithi047.stdout: "channel_perf": { 2024-03-21T10:35:32.913 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "channel_perf", 2024-03-21T10:35:32.913 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T10:35:32.913 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:32.913 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T10:35:32.913 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-03-21T10:35:32.913 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:32.913 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:32.913 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:32.913 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Share various performance metrics of a cluster", 2024-03-21T10:35:32.913 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:32.913 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:32.913 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:32.913 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:32.913 INFO:teuthology.orchestra.run.smithi047.stdout: "contact": { 2024-03-21T10:35:32.914 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "contact", 2024-03-21T10:35:32.914 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T10:35:32.914 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:32.914 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T10:35:32.914 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-03-21T10:35:32.914 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:32.914 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:32.914 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:32.914 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T10:35:32.914 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:32.914 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:32.914 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:32.914 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:32.914 INFO:teuthology.orchestra.run.smithi047.stdout: "description": { 2024-03-21T10:35:32.915 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "description", 2024-03-21T10:35:32.915 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T10:35:32.915 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:32.915 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T10:35:32.915 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-03-21T10:35:32.915 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:32.915 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:32.915 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:32.915 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T10:35:32.915 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:32.915 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:32.915 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:32.915 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:32.915 INFO:teuthology.orchestra.run.smithi047.stdout: "device_url": { 2024-03-21T10:35:32.915 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "device_url", 2024-03-21T10:35:32.916 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T10:35:32.916 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:32.916 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T10:35:32.916 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-03-21T10:35:32.916 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:32.916 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:32.916 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:32.916 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T10:35:32.916 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:32.916 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:32.916 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:32.916 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:32.916 INFO:teuthology.orchestra.run.smithi047.stdout: "enabled": { 2024-03-21T10:35:32.916 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "enabled", 2024-03-21T10:35:32.916 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T10:35:32.917 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:32.917 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T10:35:32.917 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-03-21T10:35:32.917 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:32.917 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:32.917 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:32.917 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T10:35:32.917 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:32.917 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:32.917 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:32.917 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:32.917 INFO:teuthology.orchestra.run.smithi047.stdout: "interval": { 2024-03-21T10:35:32.917 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "interval", 2024-03-21T10:35:32.917 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "int", 2024-03-21T10:35:32.917 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:32.918 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T10:35:32.918 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "24", 2024-03-21T10:35:32.918 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "8", 2024-03-21T10:35:32.918 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:32.918 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:32.918 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T10:35:32.918 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:32.918 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:32.918 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:32.918 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:32.918 INFO:teuthology.orchestra.run.smithi047.stdout: "last_opt_revision": { 2024-03-21T10:35:32.918 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "last_opt_revision", 2024-03-21T10:35:32.918 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "int", 2024-03-21T10:35:32.918 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:32.919 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T10:35:32.919 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "1", 2024-03-21T10:35:32.919 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:32.919 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:32.919 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:32.919 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T10:35:32.919 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:32.919 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:32.919 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:32.919 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:32.919 INFO:teuthology.orchestra.run.smithi047.stdout: "leaderboard": { 2024-03-21T10:35:32.919 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "leaderboard", 2024-03-21T10:35:32.919 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T10:35:32.919 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:32.919 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T10:35:32.920 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-03-21T10:35:32.920 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:32.920 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:32.920 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:32.920 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T10:35:32.920 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:32.920 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:32.920 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:32.920 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:32.920 INFO:teuthology.orchestra.run.smithi047.stdout: "leaderboard_description": { 2024-03-21T10:35:32.920 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "leaderboard_description", 2024-03-21T10:35:32.920 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T10:35:32.920 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:32.920 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T10:35:32.920 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-03-21T10:35:32.921 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:32.921 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:32.921 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:32.921 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T10:35:32.921 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:32.921 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:32.921 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:32.921 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:32.921 INFO:teuthology.orchestra.run.smithi047.stdout: "log_level": { 2024-03-21T10:35:32.921 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_level", 2024-03-21T10:35:32.921 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T10:35:32.921 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:32.921 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T10:35:32.921 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-03-21T10:35:32.921 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:32.922 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:32.922 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-03-21T10:35:32.922 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-03-21T10:35:32.922 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-03-21T10:35:32.922 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-03-21T10:35:32.922 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-03-21T10:35:32.922 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-03-21T10:35:32.922 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-03-21T10:35:32.922 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-03-21T10:35:32.922 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T10:35:32.922 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:32.922 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:32.922 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:32.922 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:32.922 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster": { 2024-03-21T10:35:32.923 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster", 2024-03-21T10:35:32.923 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T10:35:32.923 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:32.923 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T10:35:32.923 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-03-21T10:35:32.923 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:32.923 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:32.923 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:32.923 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T10:35:32.923 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:32.923 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:32.923 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:32.923 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:32.923 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster_level": { 2024-03-21T10:35:32.923 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster_level", 2024-03-21T10:35:32.924 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T10:35:32.924 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:32.924 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T10:35:32.924 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "info", 2024-03-21T10:35:32.924 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:32.924 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:32.924 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-03-21T10:35:32.924 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-03-21T10:35:32.924 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-03-21T10:35:32.924 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-03-21T10:35:32.924 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-03-21T10:35:32.924 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-03-21T10:35:32.924 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-03-21T10:35:32.924 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-03-21T10:35:32.925 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T10:35:32.925 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:32.925 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:32.925 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:32.925 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:32.925 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_file": { 2024-03-21T10:35:32.925 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_file", 2024-03-21T10:35:32.925 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T10:35:32.925 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:32.925 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T10:35:32.925 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-03-21T10:35:32.925 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:32.925 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:32.925 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:32.925 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T10:35:32.926 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:32.926 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:32.926 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:32.926 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:32.926 INFO:teuthology.orchestra.run.smithi047.stdout: "organization": { 2024-03-21T10:35:32.926 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "organization", 2024-03-21T10:35:32.926 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T10:35:32.926 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:32.926 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T10:35:32.926 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-03-21T10:35:32.926 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:32.926 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:32.926 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:32.926 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T10:35:32.927 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:32.927 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:32.927 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:32.927 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:32.927 INFO:teuthology.orchestra.run.smithi047.stdout: "proxy": { 2024-03-21T10:35:32.927 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "proxy", 2024-03-21T10:35:32.927 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T10:35:32.927 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:32.927 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T10:35:32.927 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-03-21T10:35:32.927 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:32.927 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:32.927 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:32.927 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T10:35:32.927 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:32.928 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:32.928 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:32.928 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:32.928 INFO:teuthology.orchestra.run.smithi047.stdout: "url": { 2024-03-21T10:35:32.928 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "url", 2024-03-21T10:35:32.928 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T10:35:32.928 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:32.928 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T10:35:32.928 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-03-21T10:35:32.928 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:32.928 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:32.928 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:32.928 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T10:35:32.928 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:32.928 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:32.929 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:32.929 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-21T10:35:32.929 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-21T10:35:32.929 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:32.929 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-03-21T10:35:32.929 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "test_orchestrator", 2024-03-21T10:35:32.929 INFO:teuthology.orchestra.run.smithi047.stdout: "can_run": true, 2024-03-21T10:35:32.929 INFO:teuthology.orchestra.run.smithi047.stdout: "error_string": "", 2024-03-21T10:35:32.929 INFO:teuthology.orchestra.run.smithi047.stdout: "module_options": { 2024-03-21T10:35:32.929 INFO:teuthology.orchestra.run.smithi047.stdout: "log_level": { 2024-03-21T10:35:32.929 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_level", 2024-03-21T10:35:32.929 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T10:35:32.929 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:32.929 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T10:35:32.930 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-03-21T10:35:32.930 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:32.930 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:32.930 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-03-21T10:35:32.930 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-03-21T10:35:32.930 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-03-21T10:35:32.930 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-03-21T10:35:32.930 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-03-21T10:35:32.930 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-03-21T10:35:32.930 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-03-21T10:35:32.930 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-03-21T10:35:32.930 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T10:35:32.930 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:32.930 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:32.931 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:32.931 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:32.931 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster": { 2024-03-21T10:35:32.931 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster", 2024-03-21T10:35:32.931 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T10:35:32.931 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:32.931 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T10:35:32.931 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-03-21T10:35:32.931 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:32.931 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:32.931 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:32.931 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T10:35:32.931 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:32.932 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:32.932 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:32.932 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:32.932 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster_level": { 2024-03-21T10:35:32.932 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster_level", 2024-03-21T10:35:32.932 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T10:35:32.932 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:32.932 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T10:35:32.932 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "info", 2024-03-21T10:35:32.932 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:32.932 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:32.932 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-03-21T10:35:32.932 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-03-21T10:35:32.933 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-03-21T10:35:32.933 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-03-21T10:35:32.933 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-03-21T10:35:32.933 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-03-21T10:35:32.933 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-03-21T10:35:32.933 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-03-21T10:35:32.933 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T10:35:32.933 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:32.933 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:32.933 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:32.933 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:32.933 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_file": { 2024-03-21T10:35:32.933 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_file", 2024-03-21T10:35:32.933 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T10:35:32.934 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:32.934 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T10:35:32.934 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-03-21T10:35:32.934 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:32.934 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:32.934 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:32.934 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T10:35:32.934 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:32.934 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:32.934 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:32.934 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-21T10:35:32.934 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-21T10:35:32.934 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:32.934 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-03-21T10:35:32.935 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "volumes", 2024-03-21T10:35:32.935 INFO:teuthology.orchestra.run.smithi047.stdout: "can_run": true, 2024-03-21T10:35:32.935 INFO:teuthology.orchestra.run.smithi047.stdout: "error_string": "", 2024-03-21T10:35:32.935 INFO:teuthology.orchestra.run.smithi047.stdout: "module_options": { 2024-03-21T10:35:32.935 INFO:teuthology.orchestra.run.smithi047.stdout: "log_level": { 2024-03-21T10:35:32.935 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_level", 2024-03-21T10:35:32.935 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T10:35:32.935 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:32.935 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T10:35:32.935 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-03-21T10:35:32.935 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:32.935 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:32.935 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-03-21T10:35:32.935 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-03-21T10:35:32.935 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-03-21T10:35:32.936 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-03-21T10:35:32.936 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-03-21T10:35:32.936 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-03-21T10:35:32.936 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-03-21T10:35:32.936 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-03-21T10:35:32.936 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T10:35:32.936 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:32.936 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:32.936 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:32.936 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:32.936 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster": { 2024-03-21T10:35:32.936 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster", 2024-03-21T10:35:32.936 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T10:35:32.936 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:32.936 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T10:35:32.937 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-03-21T10:35:32.937 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:32.937 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:32.937 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:32.937 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T10:35:32.937 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:32.937 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:32.937 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:32.937 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:32.937 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster_level": { 2024-03-21T10:35:32.937 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster_level", 2024-03-21T10:35:32.937 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T10:35:32.937 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:32.937 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T10:35:32.937 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "info", 2024-03-21T10:35:32.938 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:32.938 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:32.938 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-03-21T10:35:32.938 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-03-21T10:35:32.938 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-03-21T10:35:32.938 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-03-21T10:35:32.938 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-03-21T10:35:32.938 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-03-21T10:35:32.938 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-03-21T10:35:32.938 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-03-21T10:35:32.938 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T10:35:32.938 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:32.938 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:32.938 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:32.939 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:32.939 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_file": { 2024-03-21T10:35:32.939 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_file", 2024-03-21T10:35:32.939 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T10:35:32.939 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:32.939 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T10:35:32.939 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-03-21T10:35:32.939 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:32.939 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:32.939 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:32.939 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T10:35:32.939 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:32.939 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:32.939 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:32.939 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:32.939 INFO:teuthology.orchestra.run.smithi047.stdout: "max_concurrent_clones": { 2024-03-21T10:35:32.940 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "max_concurrent_clones", 2024-03-21T10:35:32.940 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "int", 2024-03-21T10:35:32.940 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:32.940 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T10:35:32.940 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "4", 2024-03-21T10:35:32.940 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:32.940 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:32.940 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:32.940 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Number of asynchronous cloner threads", 2024-03-21T10:35:32.940 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:32.940 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:32.940 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:32.940 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:32.940 INFO:teuthology.orchestra.run.smithi047.stdout: "periodic_async_work": { 2024-03-21T10:35:32.941 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "periodic_async_work", 2024-03-21T10:35:32.941 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T10:35:32.941 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:32.941 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T10:35:32.941 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-03-21T10:35:32.941 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:32.941 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:32.941 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:32.941 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Periodically check for async work", 2024-03-21T10:35:32.941 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:32.941 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:32.941 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:32.941 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:32.941 INFO:teuthology.orchestra.run.smithi047.stdout: "snapshot_clone_delay": { 2024-03-21T10:35:32.941 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "snapshot_clone_delay", 2024-03-21T10:35:32.941 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "int", 2024-03-21T10:35:32.942 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:32.942 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T10:35:32.942 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "0", 2024-03-21T10:35:32.942 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:32.942 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:32.942 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:32.942 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-03-21T10:35:32.942 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:32.942 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:32.942 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:32.942 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:32.942 INFO:teuthology.orchestra.run.smithi047.stdout: "snapshot_clone_no_wait": { 2024-03-21T10:35:32.942 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "snapshot_clone_no_wait", 2024-03-21T10:35:32.942 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T10:35:32.943 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:32.943 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T10:35:32.943 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "True", 2024-03-21T10:35:32.943 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:32.943 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:32.943 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:32.943 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-03-21T10:35:32.943 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:32.943 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:32.943 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:32.943 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-21T10:35:32.943 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-21T10:35:32.943 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:32.943 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-03-21T10:35:32.944 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "zabbix", 2024-03-21T10:35:32.944 INFO:teuthology.orchestra.run.smithi047.stdout: "can_run": true, 2024-03-21T10:35:32.944 INFO:teuthology.orchestra.run.smithi047.stdout: "error_string": "", 2024-03-21T10:35:32.944 INFO:teuthology.orchestra.run.smithi047.stdout: "module_options": { 2024-03-21T10:35:32.944 INFO:teuthology.orchestra.run.smithi047.stdout: "discovery_interval": { 2024-03-21T10:35:32.944 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "discovery_interval", 2024-03-21T10:35:32.944 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "uint", 2024-03-21T10:35:32.944 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:32.944 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T10:35:32.944 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "100", 2024-03-21T10:35:32.944 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:32.944 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:32.944 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:32.944 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T10:35:32.945 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:32.945 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:32.945 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:32.945 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:32.945 INFO:teuthology.orchestra.run.smithi047.stdout: "identifier": { 2024-03-21T10:35:32.945 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "identifier", 2024-03-21T10:35:32.945 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T10:35:32.945 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:32.945 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T10:35:32.945 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-03-21T10:35:32.945 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:32.945 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:32.945 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:32.945 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T10:35:32.946 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:32.946 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:32.946 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:32.946 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:32.946 INFO:teuthology.orchestra.run.smithi047.stdout: "interval": { 2024-03-21T10:35:32.946 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "interval", 2024-03-21T10:35:32.946 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "secs", 2024-03-21T10:35:32.946 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:32.946 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T10:35:32.946 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "60", 2024-03-21T10:35:32.946 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:32.946 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:32.946 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:32.947 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T10:35:32.947 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:32.947 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:32.947 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:32.947 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:32.947 INFO:teuthology.orchestra.run.smithi047.stdout: "log_level": { 2024-03-21T10:35:32.947 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_level", 2024-03-21T10:35:32.947 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T10:35:32.947 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:32.947 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T10:35:32.947 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-03-21T10:35:32.947 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:32.947 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:32.947 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-03-21T10:35:32.947 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-03-21T10:35:32.948 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-03-21T10:35:32.948 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-03-21T10:35:32.948 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-03-21T10:35:32.948 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-03-21T10:35:32.948 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-03-21T10:35:32.948 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-03-21T10:35:32.948 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T10:35:32.948 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:32.948 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:32.948 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:32.948 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:32.948 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster": { 2024-03-21T10:35:32.948 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster", 2024-03-21T10:35:32.948 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T10:35:32.949 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:32.949 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T10:35:32.949 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-03-21T10:35:32.949 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:32.949 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:32.949 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:32.949 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T10:35:32.949 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:32.949 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:32.949 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:32.949 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:32.949 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster_level": { 2024-03-21T10:35:32.949 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster_level", 2024-03-21T10:35:32.949 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T10:35:32.950 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:32.950 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T10:35:32.950 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "info", 2024-03-21T10:35:32.950 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:32.950 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:32.950 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-03-21T10:35:32.950 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-03-21T10:35:32.950 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-03-21T10:35:32.950 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-03-21T10:35:32.950 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-03-21T10:35:32.950 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-03-21T10:35:32.950 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-03-21T10:35:32.950 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-03-21T10:35:32.951 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T10:35:32.951 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:32.951 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:32.951 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:32.951 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:32.951 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_file": { 2024-03-21T10:35:32.951 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_file", 2024-03-21T10:35:32.951 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T10:35:32.951 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:32.951 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T10:35:32.951 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-03-21T10:35:32.951 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:32.951 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:32.951 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:32.951 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T10:35:32.952 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:32.952 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:32.952 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:32.952 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:32.952 INFO:teuthology.orchestra.run.smithi047.stdout: "zabbix_host": { 2024-03-21T10:35:32.952 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "zabbix_host", 2024-03-21T10:35:32.952 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T10:35:32.952 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:32.952 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T10:35:32.952 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-03-21T10:35:32.952 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:32.952 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:32.952 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:32.952 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T10:35:32.953 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:32.953 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:32.953 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:32.953 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:32.953 INFO:teuthology.orchestra.run.smithi047.stdout: "zabbix_port": { 2024-03-21T10:35:32.953 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "zabbix_port", 2024-03-21T10:35:32.953 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "int", 2024-03-21T10:35:32.953 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:32.953 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T10:35:32.953 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "10051", 2024-03-21T10:35:32.953 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:32.953 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:32.953 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:32.953 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T10:35:32.953 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:32.954 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:32.954 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:32.954 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:32.954 INFO:teuthology.orchestra.run.smithi047.stdout: "zabbix_sender": { 2024-03-21T10:35:32.954 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "zabbix_sender", 2024-03-21T10:35:32.954 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T10:35:32.954 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:32.954 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T10:35:32.954 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-03-21T10:35:32.954 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:32.954 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:32.954 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:32.954 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T10:35:32.955 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:32.955 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:32.955 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:32.955 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-21T10:35:32.955 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-21T10:35:32.955 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-21T10:35:32.955 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-03-21T10:35:32.955 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:32.955 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-03-21T10:35:32.955 INFO:teuthology.orchestra.run.smithi047.stdout: "gid": 25384, 2024-03-21T10:35:32.955 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "z", 2024-03-21T10:35:32.955 INFO:teuthology.orchestra.run.smithi047.stdout: "mgr_features": 4540701547738038271, 2024-03-21T10:35:32.955 INFO:teuthology.orchestra.run.smithi047.stdout: "available_modules": [ 2024-03-21T10:35:32.955 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-03-21T10:35:32.956 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "alerts", 2024-03-21T10:35:32.956 INFO:teuthology.orchestra.run.smithi047.stdout: "can_run": true, 2024-03-21T10:35:32.956 INFO:teuthology.orchestra.run.smithi047.stdout: "error_string": "", 2024-03-21T10:35:32.956 INFO:teuthology.orchestra.run.smithi047.stdout: "module_options": { 2024-03-21T10:35:32.956 INFO:teuthology.orchestra.run.smithi047.stdout: "interval": { 2024-03-21T10:35:32.956 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "interval", 2024-03-21T10:35:32.956 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "secs", 2024-03-21T10:35:32.956 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:32.956 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T10:35:32.956 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "60", 2024-03-21T10:35:32.956 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:32.956 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:32.956 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:32.956 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "How frequently to reexamine health status", 2024-03-21T10:35:32.957 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:32.957 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:32.957 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:32.957 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:32.957 INFO:teuthology.orchestra.run.smithi047.stdout: "log_level": { 2024-03-21T10:35:32.957 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_level", 2024-03-21T10:35:32.957 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T10:35:32.957 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:32.957 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T10:35:32.957 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-03-21T10:35:32.957 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:32.957 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:32.957 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-03-21T10:35:32.958 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-03-21T10:35:32.958 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-03-21T10:35:32.958 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-03-21T10:35:32.958 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-03-21T10:35:32.958 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-03-21T10:35:32.958 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-03-21T10:35:32.958 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-03-21T10:35:32.958 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T10:35:32.958 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:32.958 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:32.958 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:32.958 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:32.958 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster": { 2024-03-21T10:35:32.959 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster", 2024-03-21T10:35:32.959 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T10:35:32.959 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:32.959 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T10:35:32.959 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-03-21T10:35:32.959 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:32.959 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:32.959 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:32.959 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T10:35:32.959 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:32.959 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:32.959 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:32.959 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:32.959 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster_level": { 2024-03-21T10:35:32.960 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster_level", 2024-03-21T10:35:32.960 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T10:35:32.960 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:32.960 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T10:35:32.960 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "info", 2024-03-21T10:35:32.960 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:32.960 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:32.960 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-03-21T10:35:32.960 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-03-21T10:35:32.960 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-03-21T10:35:32.960 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-03-21T10:35:32.960 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-03-21T10:35:32.960 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-03-21T10:35:32.960 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-03-21T10:35:32.961 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-03-21T10:35:32.961 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T10:35:32.961 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:32.961 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:32.961 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:32.961 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:32.961 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_file": { 2024-03-21T10:35:32.961 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_file", 2024-03-21T10:35:32.961 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T10:35:32.961 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:32.961 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T10:35:32.961 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-03-21T10:35:32.961 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:32.961 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:32.962 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:32.962 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T10:35:32.962 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:32.962 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:32.962 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:32.962 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:32.962 INFO:teuthology.orchestra.run.smithi047.stdout: "smtp_destination": { 2024-03-21T10:35:32.962 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "smtp_destination", 2024-03-21T10:35:32.962 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T10:35:32.962 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:32.962 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T10:35:32.962 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-03-21T10:35:32.962 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:32.962 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:32.962 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:32.963 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Email address to send alerts to", 2024-03-21T10:35:32.963 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:32.963 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:32.963 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:32.963 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:32.963 INFO:teuthology.orchestra.run.smithi047.stdout: "smtp_from_name": { 2024-03-21T10:35:32.963 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "smtp_from_name", 2024-03-21T10:35:32.963 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T10:35:32.963 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:32.963 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T10:35:32.963 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "Ceph", 2024-03-21T10:35:32.963 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:32.963 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:32.963 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:32.963 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Email From: name", 2024-03-21T10:35:32.964 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:32.964 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:32.964 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:32.964 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:32.964 INFO:teuthology.orchestra.run.smithi047.stdout: "smtp_host": { 2024-03-21T10:35:32.964 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "smtp_host", 2024-03-21T10:35:32.964 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T10:35:32.964 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:32.964 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T10:35:32.964 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-03-21T10:35:32.964 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:32.964 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:32.964 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:32.964 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "SMTP server", 2024-03-21T10:35:32.964 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:32.964 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:32.965 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:32.965 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:32.965 INFO:teuthology.orchestra.run.smithi047.stdout: "smtp_password": { 2024-03-21T10:35:32.965 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "smtp_password", 2024-03-21T10:35:32.965 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T10:35:32.965 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:32.965 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T10:35:32.965 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-03-21T10:35:32.965 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:32.965 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:32.965 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:32.965 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Password to authenticate with", 2024-03-21T10:35:32.965 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:32.965 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:32.965 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:32.966 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:32.966 INFO:teuthology.orchestra.run.smithi047.stdout: "smtp_port": { 2024-03-21T10:35:32.966 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "smtp_port", 2024-03-21T10:35:32.966 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "int", 2024-03-21T10:35:32.966 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:32.966 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T10:35:32.966 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "465", 2024-03-21T10:35:32.966 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:32.966 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:32.966 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:32.966 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "SMTP port", 2024-03-21T10:35:32.966 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:32.966 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:32.966 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:32.967 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:32.967 INFO:teuthology.orchestra.run.smithi047.stdout: "smtp_sender": { 2024-03-21T10:35:32.967 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "smtp_sender", 2024-03-21T10:35:32.967 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T10:35:32.967 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:32.967 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T10:35:32.967 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-03-21T10:35:32.967 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:32.967 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:32.967 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:32.967 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "SMTP envelope sender", 2024-03-21T10:35:32.967 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:32.967 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:32.967 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:32.967 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:32.968 INFO:teuthology.orchestra.run.smithi047.stdout: "smtp_ssl": { 2024-03-21T10:35:32.968 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "smtp_ssl", 2024-03-21T10:35:32.968 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T10:35:32.968 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:32.968 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T10:35:32.968 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "True", 2024-03-21T10:35:32.968 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:32.968 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:32.968 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:32.968 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Use SSL to connect to SMTP server", 2024-03-21T10:35:32.968 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:32.968 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:32.968 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:32.968 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:32.968 INFO:teuthology.orchestra.run.smithi047.stdout: "smtp_user": { 2024-03-21T10:35:32.969 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "smtp_user", 2024-03-21T10:35:32.969 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T10:35:32.969 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:32.969 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T10:35:32.969 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-03-21T10:35:32.969 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:32.969 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:32.969 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:32.969 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "User to authenticate as", 2024-03-21T10:35:32.969 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:32.969 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:32.969 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:32.969 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-21T10:35:32.969 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-21T10:35:32.969 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:32.970 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-03-21T10:35:32.970 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "balancer", 2024-03-21T10:35:32.970 INFO:teuthology.orchestra.run.smithi047.stdout: "can_run": true, 2024-03-21T10:35:32.970 INFO:teuthology.orchestra.run.smithi047.stdout: "error_string": "", 2024-03-21T10:35:32.970 INFO:teuthology.orchestra.run.smithi047.stdout: "module_options": { 2024-03-21T10:35:32.970 INFO:teuthology.orchestra.run.smithi047.stdout: "active": { 2024-03-21T10:35:32.970 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "active", 2024-03-21T10:35:32.970 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T10:35:32.970 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:32.970 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T10:35:32.970 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "True", 2024-03-21T10:35:32.970 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:32.970 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:32.970 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:32.971 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "automatically balance PGs across cluster", 2024-03-21T10:35:32.971 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:32.971 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:32.971 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:32.971 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:32.971 INFO:teuthology.orchestra.run.smithi047.stdout: "begin_time": { 2024-03-21T10:35:32.971 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "begin_time", 2024-03-21T10:35:32.971 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T10:35:32.971 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:32.971 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T10:35:32.971 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "0000", 2024-03-21T10:35:32.971 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:32.971 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:32.971 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:32.972 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "beginning time of day to automatically balance", 2024-03-21T10:35:32.972 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-03-21T10:35:32.972 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:32.972 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:32.972 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:32.972 INFO:teuthology.orchestra.run.smithi047.stdout: "begin_weekday": { 2024-03-21T10:35:32.972 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "begin_weekday", 2024-03-21T10:35:32.972 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "uint", 2024-03-21T10:35:32.972 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:32.972 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T10:35:32.972 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "0", 2024-03-21T10:35:32.972 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "0", 2024-03-21T10:35:32.972 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "6", 2024-03-21T10:35:32.972 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:32.973 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-03-21T10:35:32.973 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-03-21T10:35:32.973 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:32.973 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:32.973 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:32.973 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_compat_max_iterations": { 2024-03-21T10:35:32.973 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "crush_compat_max_iterations", 2024-03-21T10:35:32.973 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "uint", 2024-03-21T10:35:32.973 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:32.973 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T10:35:32.973 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "25", 2024-03-21T10:35:32.973 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "1", 2024-03-21T10:35:32.973 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "250", 2024-03-21T10:35:32.973 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:32.973 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-03-21T10:35:32.974 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:32.974 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:32.974 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:32.974 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:32.974 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_compat_metrics": { 2024-03-21T10:35:32.974 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "crush_compat_metrics", 2024-03-21T10:35:32.974 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T10:35:32.974 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:32.974 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T10:35:32.974 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "pgs,objects,bytes", 2024-03-21T10:35:32.974 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:32.974 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:32.974 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:32.974 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-03-21T10:35:32.975 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-03-21T10:35:32.975 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:32.975 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:32.975 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:32.975 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_compat_step": { 2024-03-21T10:35:32.975 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "crush_compat_step", 2024-03-21T10:35:32.975 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "float", 2024-03-21T10:35:32.975 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:32.975 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T10:35:32.975 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "0.5", 2024-03-21T10:35:32.975 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "0.001", 2024-03-21T10:35:32.975 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "0.999", 2024-03-21T10:35:32.975 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:32.975 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "aggressiveness of optimization", 2024-03-21T10:35:32.975 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-03-21T10:35:32.976 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:32.976 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:32.976 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:32.976 INFO:teuthology.orchestra.run.smithi047.stdout: "end_time": { 2024-03-21T10:35:32.976 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "end_time", 2024-03-21T10:35:32.976 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T10:35:32.976 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:32.976 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T10:35:32.976 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "2359", 2024-03-21T10:35:32.976 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:32.976 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:32.976 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:32.976 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "ending time of day to automatically balance", 2024-03-21T10:35:32.976 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-03-21T10:35:32.977 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:32.977 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:32.977 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:32.977 INFO:teuthology.orchestra.run.smithi047.stdout: "end_weekday": { 2024-03-21T10:35:32.977 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "end_weekday", 2024-03-21T10:35:32.977 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "uint", 2024-03-21T10:35:32.977 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:32.977 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T10:35:32.977 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "0", 2024-03-21T10:35:32.977 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "0", 2024-03-21T10:35:32.977 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "6", 2024-03-21T10:35:32.977 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:32.977 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-03-21T10:35:32.978 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-03-21T10:35:32.978 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:32.978 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:32.978 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:32.978 INFO:teuthology.orchestra.run.smithi047.stdout: "log_level": { 2024-03-21T10:35:32.978 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_level", 2024-03-21T10:35:32.978 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T10:35:32.978 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:32.978 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T10:35:32.978 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-03-21T10:35:32.978 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:32.978 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:32.978 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-03-21T10:35:32.979 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-03-21T10:35:32.979 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-03-21T10:35:32.979 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-03-21T10:35:32.979 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-03-21T10:35:32.979 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-03-21T10:35:32.979 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-03-21T10:35:32.979 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-03-21T10:35:32.979 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T10:35:32.979 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:32.979 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:32.979 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:32.979 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:32.979 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster": { 2024-03-21T10:35:32.979 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster", 2024-03-21T10:35:32.980 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T10:35:32.980 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:32.980 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T10:35:32.980 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-03-21T10:35:32.980 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:32.980 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:32.980 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:32.980 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T10:35:32.980 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:32.980 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:32.980 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:32.980 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:32.980 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster_level": { 2024-03-21T10:35:32.980 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster_level", 2024-03-21T10:35:32.981 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T10:35:32.981 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:32.981 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T10:35:32.981 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "info", 2024-03-21T10:35:32.981 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:32.981 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:32.981 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-03-21T10:35:32.981 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-03-21T10:35:32.981 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-03-21T10:35:32.981 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-03-21T10:35:32.981 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-03-21T10:35:32.981 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-03-21T10:35:32.981 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-03-21T10:35:32.981 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-03-21T10:35:32.981 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T10:35:32.982 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:32.982 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:32.982 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:32.982 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:32.982 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_file": { 2024-03-21T10:35:32.982 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_file", 2024-03-21T10:35:32.982 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T10:35:32.982 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:32.982 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T10:35:32.982 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-03-21T10:35:32.982 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:32.982 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:32.982 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:32.983 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T10:35:32.983 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:32.983 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:32.983 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:32.983 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:32.983 INFO:teuthology.orchestra.run.smithi047.stdout: "min_score": { 2024-03-21T10:35:32.983 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "min_score", 2024-03-21T10:35:32.983 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "float", 2024-03-21T10:35:32.983 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:32.983 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T10:35:32.983 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "0", 2024-03-21T10:35:32.983 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:32.983 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:32.983 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:32.983 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-03-21T10:35:32.984 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:32.984 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:32.984 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:32.984 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:32.984 INFO:teuthology.orchestra.run.smithi047.stdout: "mode": { 2024-03-21T10:35:32.984 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "mode", 2024-03-21T10:35:32.984 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T10:35:32.984 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:32.984 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T10:35:32.984 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "upmap", 2024-03-21T10:35:32.984 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:32.984 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:32.984 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-03-21T10:35:32.984 INFO:teuthology.orchestra.run.smithi047.stdout: "crush-compat", 2024-03-21T10:35:32.984 INFO:teuthology.orchestra.run.smithi047.stdout: "none", 2024-03-21T10:35:32.985 INFO:teuthology.orchestra.run.smithi047.stdout: "read", 2024-03-21T10:35:32.985 INFO:teuthology.orchestra.run.smithi047.stdout: "upmap", 2024-03-21T10:35:32.985 INFO:teuthology.orchestra.run.smithi047.stdout: "upmap-read" 2024-03-21T10:35:32.985 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-03-21T10:35:32.985 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Balancer mode", 2024-03-21T10:35:32.985 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:32.985 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:32.985 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:32.985 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:32.985 INFO:teuthology.orchestra.run.smithi047.stdout: "pool_ids": { 2024-03-21T10:35:32.985 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "pool_ids", 2024-03-21T10:35:32.985 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T10:35:32.985 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:32.985 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T10:35:32.985 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-03-21T10:35:32.985 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:32.986 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:32.986 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:32.986 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-03-21T10:35:32.986 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:32.986 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:32.986 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:32.986 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:32.986 INFO:teuthology.orchestra.run.smithi047.stdout: "sleep_interval": { 2024-03-21T10:35:32.986 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "sleep_interval", 2024-03-21T10:35:32.986 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "secs", 2024-03-21T10:35:32.986 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:32.986 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T10:35:32.986 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "60", 2024-03-21T10:35:32.986 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:32.987 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:32.987 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:32.987 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-03-21T10:35:32.987 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:32.987 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:32.987 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:32.987 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:32.987 INFO:teuthology.orchestra.run.smithi047.stdout: "upmap_max_deviation": { 2024-03-21T10:35:32.987 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "upmap_max_deviation", 2024-03-21T10:35:32.987 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "int", 2024-03-21T10:35:32.987 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:32.987 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T10:35:32.987 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "5", 2024-03-21T10:35:32.987 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "1", 2024-03-21T10:35:32.987 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:32.988 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:32.988 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "deviation below which no optimization is attempted", 2024-03-21T10:35:32.988 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-03-21T10:35:32.988 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:32.988 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:32.988 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:32.988 INFO:teuthology.orchestra.run.smithi047.stdout: "upmap_max_optimizations": { 2024-03-21T10:35:32.988 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "upmap_max_optimizations", 2024-03-21T10:35:32.988 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "uint", 2024-03-21T10:35:32.988 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:32.988 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T10:35:32.988 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "10", 2024-03-21T10:35:32.988 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:32.988 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:32.988 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:32.989 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-03-21T10:35:32.989 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:32.989 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:32.989 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:32.989 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-21T10:35:32.989 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-21T10:35:32.989 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:32.989 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-03-21T10:35:32.989 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "cephadm", 2024-03-21T10:35:32.989 INFO:teuthology.orchestra.run.smithi047.stdout: "can_run": true, 2024-03-21T10:35:32.989 INFO:teuthology.orchestra.run.smithi047.stdout: "error_string": "", 2024-03-21T10:35:32.989 INFO:teuthology.orchestra.run.smithi047.stdout: "module_options": { 2024-03-21T10:35:32.989 INFO:teuthology.orchestra.run.smithi047.stdout: "agent_down_multiplier": { 2024-03-21T10:35:32.989 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "agent_down_multiplier", 2024-03-21T10:35:32.989 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "float", 2024-03-21T10:35:32.990 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:32.990 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T10:35:32.990 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "3.0", 2024-03-21T10:35:32.990 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:32.990 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:32.990 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:32.990 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-03-21T10:35:32.990 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:32.990 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:32.990 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:32.990 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:32.990 INFO:teuthology.orchestra.run.smithi047.stdout: "agent_refresh_rate": { 2024-03-21T10:35:32.990 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "agent_refresh_rate", 2024-03-21T10:35:32.990 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "secs", 2024-03-21T10:35:32.990 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:32.991 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T10:35:32.991 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "20", 2024-03-21T10:35:32.991 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:32.991 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:32.991 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:32.991 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-03-21T10:35:32.991 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:32.991 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:32.991 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:32.991 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:32.991 INFO:teuthology.orchestra.run.smithi047.stdout: "agent_starting_port": { 2024-03-21T10:35:32.991 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "agent_starting_port", 2024-03-21T10:35:32.991 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "int", 2024-03-21T10:35:32.991 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:32.991 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T10:35:32.992 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "4721", 2024-03-21T10:35:32.992 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:32.992 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:32.992 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:32.992 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-03-21T10:35:32.992 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:32.992 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:32.992 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:32.992 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:32.992 INFO:teuthology.orchestra.run.smithi047.stdout: "allow_ptrace": { 2024-03-21T10:35:32.992 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "allow_ptrace", 2024-03-21T10:35:32.992 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T10:35:32.992 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:32.992 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T10:35:32.993 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-03-21T10:35:32.993 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:32.993 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:32.993 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:32.993 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-03-21T10:35:32.993 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-03-21T10:35:32.993 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:32.993 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:32.993 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:32.993 INFO:teuthology.orchestra.run.smithi047.stdout: "autotune_interval": { 2024-03-21T10:35:32.993 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "autotune_interval", 2024-03-21T10:35:32.993 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "secs", 2024-03-21T10:35:32.993 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:32.993 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T10:35:32.993 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "600", 2024-03-21T10:35:32.994 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:32.994 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:32.994 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:32.994 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "how frequently to autotune daemon memory", 2024-03-21T10:35:32.994 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:32.994 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:32.994 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:32.994 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:32.994 INFO:teuthology.orchestra.run.smithi047.stdout: "autotune_memory_target_ratio": { 2024-03-21T10:35:32.994 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "autotune_memory_target_ratio", 2024-03-21T10:35:32.994 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "float", 2024-03-21T10:35:32.994 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:32.994 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T10:35:32.994 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "0.7", 2024-03-21T10:35:32.995 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:32.995 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:32.995 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:32.995 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-03-21T10:35:32.995 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:32.995 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:32.995 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:32.995 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:32.995 INFO:teuthology.orchestra.run.smithi047.stdout: "cephadm_log_destination": { 2024-03-21T10:35:32.995 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "cephadm_log_destination", 2024-03-21T10:35:32.995 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T10:35:32.995 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:32.995 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T10:35:32.995 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-03-21T10:35:32.996 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:32.996 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:32.996 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-03-21T10:35:32.996 INFO:teuthology.orchestra.run.smithi047.stdout: "file", 2024-03-21T10:35:32.996 INFO:teuthology.orchestra.run.smithi047.stdout: "file,syslog", 2024-03-21T10:35:32.996 INFO:teuthology.orchestra.run.smithi047.stdout: "syslog" 2024-03-21T10:35:32.996 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-03-21T10:35:32.996 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-03-21T10:35:32.996 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:32.996 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:32.996 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:32.996 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:32.996 INFO:teuthology.orchestra.run.smithi047.stdout: "cgroups_split": { 2024-03-21T10:35:32.996 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "cgroups_split", 2024-03-21T10:35:32.996 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T10:35:32.996 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:32.997 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T10:35:32.997 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "True", 2024-03-21T10:35:32.997 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:32.997 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:32.997 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:32.997 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-03-21T10:35:32.997 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:32.997 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:32.997 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:32.997 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:32.997 INFO:teuthology.orchestra.run.smithi047.stdout: "config_checks_enabled": { 2024-03-21T10:35:32.997 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "config_checks_enabled", 2024-03-21T10:35:32.997 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T10:35:32.997 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:32.997 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T10:35:32.997 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-03-21T10:35:32.998 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:32.998 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:32.998 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:32.998 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-03-21T10:35:32.998 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:32.998 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:32.998 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:32.998 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:32.998 INFO:teuthology.orchestra.run.smithi047.stdout: "config_dashboard": { 2024-03-21T10:35:32.998 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "config_dashboard", 2024-03-21T10:35:32.998 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T10:35:32.998 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:32.998 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T10:35:32.998 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "True", 2024-03-21T10:35:32.999 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:32.999 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:32.999 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:32.999 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-03-21T10:35:32.999 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:32.999 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:32.999 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:32.999 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:32.999 INFO:teuthology.orchestra.run.smithi047.stdout: "container_image_alertmanager": { 2024-03-21T10:35:32.999 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "container_image_alertmanager", 2024-03-21T10:35:32.999 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T10:35:32.999 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:32.999 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T10:35:32.999 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-03-21T10:35:32.999 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:32.999 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:32.999 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:33.000 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Prometheus container image", 2024-03-21T10:35:33.000 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:33.000 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:33.000 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:33.000 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:33.000 INFO:teuthology.orchestra.run.smithi047.stdout: "container_image_base": { 2024-03-21T10:35:33.000 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "container_image_base", 2024-03-21T10:35:33.000 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T10:35:33.000 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:33.000 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T10:35:33.000 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "quay.io/ceph/ceph", 2024-03-21T10:35:33.000 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:33.000 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:33.000 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:33.000 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Container image name, without the tag", 2024-03-21T10:35:33.001 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:33.001 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:33.001 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:33.001 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:33.001 INFO:teuthology.orchestra.run.smithi047.stdout: "container_image_elasticsearch": { 2024-03-21T10:35:33.001 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "container_image_elasticsearch", 2024-03-21T10:35:33.001 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T10:35:33.001 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:33.001 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T10:35:33.001 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-03-21T10:35:33.001 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:33.001 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:33.001 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:33.001 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "elasticsearch container image", 2024-03-21T10:35:33.001 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:33.002 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:33.002 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:33.002 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:33.002 INFO:teuthology.orchestra.run.smithi047.stdout: "container_image_grafana": { 2024-03-21T10:35:33.002 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "container_image_grafana", 2024-03-21T10:35:33.002 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T10:35:33.002 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:33.002 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T10:35:33.002 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "quay.io/ceph/grafana:9.4.12", 2024-03-21T10:35:33.002 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:33.002 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:33.002 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:33.002 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Prometheus container image", 2024-03-21T10:35:33.002 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:33.003 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:33.003 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:33.003 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:33.003 INFO:teuthology.orchestra.run.smithi047.stdout: "container_image_haproxy": { 2024-03-21T10:35:33.003 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "container_image_haproxy", 2024-03-21T10:35:33.003 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T10:35:33.003 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:33.003 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T10:35:33.003 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-03-21T10:35:33.003 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:33.003 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:33.003 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:33.003 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "HAproxy container image", 2024-03-21T10:35:33.003 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:33.003 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:33.003 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:33.004 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:33.004 INFO:teuthology.orchestra.run.smithi047.stdout: "container_image_jaeger_agent": { 2024-03-21T10:35:33.004 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "container_image_jaeger_agent", 2024-03-21T10:35:33.004 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T10:35:33.004 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:33.004 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T10:35:33.004 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-03-21T10:35:33.004 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:33.004 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:33.004 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:33.004 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Jaeger agent container image", 2024-03-21T10:35:33.004 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:33.004 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:33.004 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:33.005 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:33.005 INFO:teuthology.orchestra.run.smithi047.stdout: "container_image_jaeger_collector": { 2024-03-21T10:35:33.005 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "container_image_jaeger_collector", 2024-03-21T10:35:33.005 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T10:35:33.005 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:33.005 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T10:35:33.005 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-03-21T10:35:33.005 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:33.005 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:33.005 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:33.005 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Jaeger collector container image", 2024-03-21T10:35:33.005 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:33.005 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:33.005 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:33.005 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:33.006 INFO:teuthology.orchestra.run.smithi047.stdout: "container_image_jaeger_query": { 2024-03-21T10:35:33.006 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "container_image_jaeger_query", 2024-03-21T10:35:33.006 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T10:35:33.006 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:33.006 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T10:35:33.006 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-03-21T10:35:33.006 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:33.006 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:33.006 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:33.006 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Jaeger query container image", 2024-03-21T10:35:33.006 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:33.006 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:33.006 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:33.006 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:33.007 INFO:teuthology.orchestra.run.smithi047.stdout: "container_image_keepalived": { 2024-03-21T10:35:33.007 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "container_image_keepalived", 2024-03-21T10:35:33.007 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T10:35:33.007 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:33.007 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T10:35:33.007 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-03-21T10:35:33.007 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:33.007 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:33.007 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:33.007 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Keepalived container image", 2024-03-21T10:35:33.007 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:33.007 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:33.007 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:33.007 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:33.007 INFO:teuthology.orchestra.run.smithi047.stdout: "container_image_loki": { 2024-03-21T10:35:33.007 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "container_image_loki", 2024-03-21T10:35:33.008 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T10:35:33.008 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:33.008 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T10:35:33.008 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2024-03-21T10:35:33.008 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:33.008 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:33.008 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:33.008 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Loki container image", 2024-03-21T10:35:33.008 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:33.008 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:33.008 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:33.008 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:33.008 INFO:teuthology.orchestra.run.smithi047.stdout: "container_image_node_exporter": { 2024-03-21T10:35:33.008 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "container_image_node_exporter", 2024-03-21T10:35:33.008 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T10:35:33.009 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:33.009 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T10:35:33.009 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-03-21T10:35:33.009 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:33.009 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:33.009 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:33.009 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Prometheus container image", 2024-03-21T10:35:33.009 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:33.009 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:33.009 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:33.009 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:33.009 INFO:teuthology.orchestra.run.smithi047.stdout: "container_image_nvmeof": { 2024-03-21T10:35:33.009 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "container_image_nvmeof", 2024-03-21T10:35:33.009 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T10:35:33.009 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:33.010 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T10:35:33.010 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "quay.io/ceph/nvmeof:1.0.0", 2024-03-21T10:35:33.010 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:33.010 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:33.010 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:33.010 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Nvme-of container image", 2024-03-21T10:35:33.010 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:33.010 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:33.010 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:33.010 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:33.010 INFO:teuthology.orchestra.run.smithi047.stdout: "container_image_prometheus": { 2024-03-21T10:35:33.010 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "container_image_prometheus", 2024-03-21T10:35:33.010 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T10:35:33.010 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:33.011 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T10:35:33.011 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-03-21T10:35:33.011 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:33.011 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:33.011 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:33.011 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Prometheus container image", 2024-03-21T10:35:33.011 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:33.011 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:33.011 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:33.011 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:33.011 INFO:teuthology.orchestra.run.smithi047.stdout: "container_image_promtail": { 2024-03-21T10:35:33.011 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "container_image_promtail", 2024-03-21T10:35:33.011 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T10:35:33.011 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:33.012 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T10:35:33.012 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2024-03-21T10:35:33.012 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:33.012 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:33.012 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:33.012 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Promtail container image", 2024-03-21T10:35:33.012 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:33.012 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:33.012 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:33.012 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:33.012 INFO:teuthology.orchestra.run.smithi047.stdout: "container_image_snmp_gateway": { 2024-03-21T10:35:33.012 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "container_image_snmp_gateway", 2024-03-21T10:35:33.012 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T10:35:33.013 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:33.013 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T10:35:33.013 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-03-21T10:35:33.013 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:33.013 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:33.013 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:33.013 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "SNMP Gateway container image", 2024-03-21T10:35:33.013 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:33.013 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:33.014 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:33.014 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:33.014 INFO:teuthology.orchestra.run.smithi047.stdout: "container_init": { 2024-03-21T10:35:33.014 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "container_init", 2024-03-21T10:35:33.014 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T10:35:33.014 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:33.014 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T10:35:33.014 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "True", 2024-03-21T10:35:33.015 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:33.015 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:33.015 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:33.015 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Run podman/docker with `--init`", 2024-03-21T10:35:33.015 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:33.015 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:33.015 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:33.015 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:33.015 INFO:teuthology.orchestra.run.smithi047.stdout: "daemon_cache_timeout": { 2024-03-21T10:35:33.016 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "daemon_cache_timeout", 2024-03-21T10:35:33.016 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "secs", 2024-03-21T10:35:33.016 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:33.016 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T10:35:33.016 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "600", 2024-03-21T10:35:33.016 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:33.016 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:33.016 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:33.016 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-03-21T10:35:33.016 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:33.016 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:33.016 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:33.017 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:33.017 INFO:teuthology.orchestra.run.smithi047.stdout: "default_cephadm_command_timeout": { 2024-03-21T10:35:33.017 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "default_cephadm_command_timeout", 2024-03-21T10:35:33.017 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "int", 2024-03-21T10:35:33.017 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:33.017 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T10:35:33.017 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "900", 2024-03-21T10:35:33.017 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:33.017 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:33.017 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:33.017 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-03-21T10:35:33.017 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:33.017 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:33.017 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:33.017 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:33.018 INFO:teuthology.orchestra.run.smithi047.stdout: "default_registry": { 2024-03-21T10:35:33.018 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "default_registry", 2024-03-21T10:35:33.018 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T10:35:33.018 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:33.018 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T10:35:33.018 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "docker.io", 2024-03-21T10:35:33.018 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:33.018 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:33.018 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:33.018 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-03-21T10:35:33.018 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:33.018 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:33.019 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:33.019 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:33.019 INFO:teuthology.orchestra.run.smithi047.stdout: "device_cache_timeout": { 2024-03-21T10:35:33.019 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "device_cache_timeout", 2024-03-21T10:35:33.019 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "secs", 2024-03-21T10:35:33.019 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:33.019 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T10:35:33.019 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "1800", 2024-03-21T10:35:33.019 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:33.019 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:33.019 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:33.019 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "seconds to cache device inventory", 2024-03-21T10:35:33.019 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:33.020 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:33.020 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:33.020 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:33.020 INFO:teuthology.orchestra.run.smithi047.stdout: "device_enhanced_scan": { 2024-03-21T10:35:33.020 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "device_enhanced_scan", 2024-03-21T10:35:33.020 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T10:35:33.020 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:33.020 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T10:35:33.020 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-03-21T10:35:33.020 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:33.020 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:33.020 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:33.020 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Use libstoragemgmt during device scans", 2024-03-21T10:35:33.021 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:33.021 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:33.021 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:33.021 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:33.021 INFO:teuthology.orchestra.run.smithi047.stdout: "facts_cache_timeout": { 2024-03-21T10:35:33.021 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "facts_cache_timeout", 2024-03-21T10:35:33.021 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "secs", 2024-03-21T10:35:33.021 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:33.021 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T10:35:33.021 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "60", 2024-03-21T10:35:33.021 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:33.021 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:33.021 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:33.021 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "seconds to cache host facts data", 2024-03-21T10:35:33.022 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:33.022 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:33.022 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:33.022 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:33.022 INFO:teuthology.orchestra.run.smithi047.stdout: "grafana_dashboards_path": { 2024-03-21T10:35:33.022 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "grafana_dashboards_path", 2024-03-21T10:35:33.022 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T10:35:33.022 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:33.022 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T10:35:33.022 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-03-21T10:35:33.022 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:33.022 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:33.022 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:33.023 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-03-21T10:35:33.023 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:33.023 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:33.023 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:33.023 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:33.023 INFO:teuthology.orchestra.run.smithi047.stdout: "host_check_interval": { 2024-03-21T10:35:33.023 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "host_check_interval", 2024-03-21T10:35:33.023 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "secs", 2024-03-21T10:35:33.023 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:33.023 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T10:35:33.023 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "600", 2024-03-21T10:35:33.023 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:33.023 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:33.023 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:33.023 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "how frequently to perform a host check", 2024-03-21T10:35:33.024 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:33.024 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:33.024 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:33.024 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:33.024 INFO:teuthology.orchestra.run.smithi047.stdout: "hw_monitoring": { 2024-03-21T10:35:33.024 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "hw_monitoring", 2024-03-21T10:35:33.024 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T10:35:33.024 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:33.024 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T10:35:33.024 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-03-21T10:35:33.024 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:33.024 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:33.024 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:33.024 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-03-21T10:35:33.025 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:33.025 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:33.025 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:33.025 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:33.025 INFO:teuthology.orchestra.run.smithi047.stdout: "inventory_list_all": { 2024-03-21T10:35:33.025 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "inventory_list_all", 2024-03-21T10:35:33.025 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T10:35:33.025 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:33.025 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T10:35:33.025 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-03-21T10:35:33.025 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:33.025 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:33.025 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:33.025 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-03-21T10:35:33.025 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:33.026 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:33.026 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:33.026 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:33.026 INFO:teuthology.orchestra.run.smithi047.stdout: "log_level": { 2024-03-21T10:35:33.026 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_level", 2024-03-21T10:35:33.026 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T10:35:33.026 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:33.026 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T10:35:33.026 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-03-21T10:35:33.026 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:33.026 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:33.026 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-03-21T10:35:33.026 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-03-21T10:35:33.026 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-03-21T10:35:33.027 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-03-21T10:35:33.027 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-03-21T10:35:33.027 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-03-21T10:35:33.027 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-03-21T10:35:33.027 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-03-21T10:35:33.027 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T10:35:33.027 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:33.027 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:33.027 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:33.027 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:33.027 INFO:teuthology.orchestra.run.smithi047.stdout: "log_refresh_metadata": { 2024-03-21T10:35:33.027 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_refresh_metadata", 2024-03-21T10:35:33.027 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T10:35:33.027 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:33.028 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T10:35:33.028 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-03-21T10:35:33.028 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:33.028 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:33.028 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:33.028 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-03-21T10:35:33.028 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:33.028 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:33.028 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:33.028 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:33.028 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster": { 2024-03-21T10:35:33.028 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster", 2024-03-21T10:35:33.028 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T10:35:33.028 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:33.029 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T10:35:33.029 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "True", 2024-03-21T10:35:33.029 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:33.029 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:33.029 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:33.029 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-03-21T10:35:33.029 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:33.029 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:33.029 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:33.029 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:33.029 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster_level": { 2024-03-21T10:35:33.029 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster_level", 2024-03-21T10:35:33.029 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T10:35:33.029 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:33.030 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T10:35:33.030 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "info", 2024-03-21T10:35:33.030 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:33.030 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:33.030 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-03-21T10:35:33.030 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-03-21T10:35:33.030 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-03-21T10:35:33.030 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-03-21T10:35:33.030 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-03-21T10:35:33.030 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-03-21T10:35:33.030 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-03-21T10:35:33.030 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-03-21T10:35:33.030 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T10:35:33.030 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:33.031 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:33.031 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:33.031 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:33.031 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_file": { 2024-03-21T10:35:33.031 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_file", 2024-03-21T10:35:33.031 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T10:35:33.031 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:33.031 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T10:35:33.031 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-03-21T10:35:33.031 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:33.031 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:33.031 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:33.031 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T10:35:33.032 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:33.032 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:33.032 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:33.032 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:33.032 INFO:teuthology.orchestra.run.smithi047.stdout: "manage_etc_ceph_ceph_conf": { 2024-03-21T10:35:33.032 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-03-21T10:35:33.032 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T10:35:33.032 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:33.032 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T10:35:33.032 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-03-21T10:35:33.032 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:33.032 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:33.032 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:33.032 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-03-21T10:35:33.032 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:33.032 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:33.033 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:33.033 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:33.033 INFO:teuthology.orchestra.run.smithi047.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-03-21T10:35:33.033 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-03-21T10:35:33.033 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T10:35:33.033 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:33.033 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T10:35:33.033 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "*", 2024-03-21T10:35:33.033 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:33.033 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:33.033 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:33.033 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-03-21T10:35:33.033 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:33.033 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:33.033 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:33.033 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:33.033 INFO:teuthology.orchestra.run.smithi047.stdout: "max_count_per_host": { 2024-03-21T10:35:33.034 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "max_count_per_host", 2024-03-21T10:35:33.034 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "int", 2024-03-21T10:35:33.034 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:33.034 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T10:35:33.034 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "10", 2024-03-21T10:35:33.034 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:33.034 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:33.034 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:33.034 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "max number of daemons per service per host", 2024-03-21T10:35:33.034 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:33.034 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:33.034 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:33.034 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:33.035 INFO:teuthology.orchestra.run.smithi047.stdout: "max_osd_draining_count": { 2024-03-21T10:35:33.035 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "max_osd_draining_count", 2024-03-21T10:35:33.035 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "int", 2024-03-21T10:35:33.035 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:33.035 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T10:35:33.035 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "10", 2024-03-21T10:35:33.035 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:33.035 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:33.035 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:33.035 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-03-21T10:35:33.035 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:33.035 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:33.035 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:33.035 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:33.035 INFO:teuthology.orchestra.run.smithi047.stdout: "migration_current": { 2024-03-21T10:35:33.035 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "migration_current", 2024-03-21T10:35:33.036 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "int", 2024-03-21T10:35:33.036 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:33.036 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T10:35:33.036 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-03-21T10:35:33.036 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:33.036 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:33.036 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:33.036 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "internal - do not modify", 2024-03-21T10:35:33.036 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:33.036 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:33.036 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:33.036 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:33.036 INFO:teuthology.orchestra.run.smithi047.stdout: "mode": { 2024-03-21T10:35:33.036 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "mode", 2024-03-21T10:35:33.036 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T10:35:33.037 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:33.037 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T10:35:33.037 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "root", 2024-03-21T10:35:33.037 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:33.037 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:33.037 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-03-21T10:35:33.037 INFO:teuthology.orchestra.run.smithi047.stdout: "cephadm-package", 2024-03-21T10:35:33.037 INFO:teuthology.orchestra.run.smithi047.stdout: "root" 2024-03-21T10:35:33.037 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-03-21T10:35:33.037 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "mode for remote execution of cephadm", 2024-03-21T10:35:33.037 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:33.037 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:33.037 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:33.037 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:33.037 INFO:teuthology.orchestra.run.smithi047.stdout: "oob_default_addr": { 2024-03-21T10:35:33.037 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "oob_default_addr", 2024-03-21T10:35:33.038 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T10:35:33.038 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:33.038 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T10:35:33.038 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "169.254.1.1", 2024-03-21T10:35:33.038 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:33.038 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:33.038 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:33.038 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Default address for RedFish API (oob management).", 2024-03-21T10:35:33.038 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:33.038 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:33.038 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:33.038 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:33.038 INFO:teuthology.orchestra.run.smithi047.stdout: "prometheus_alerts_path": { 2024-03-21T10:35:33.038 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "prometheus_alerts_path", 2024-03-21T10:35:33.038 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T10:35:33.039 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:33.039 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T10:35:33.039 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-03-21T10:35:33.039 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:33.039 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:33.039 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:33.039 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-03-21T10:35:33.039 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:33.039 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:33.039 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:33.039 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:33.039 INFO:teuthology.orchestra.run.smithi047.stdout: "registry_insecure": { 2024-03-21T10:35:33.039 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "registry_insecure", 2024-03-21T10:35:33.039 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T10:35:33.040 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:33.040 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T10:35:33.040 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-03-21T10:35:33.040 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:33.040 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:33.040 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:33.040 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-03-21T10:35:33.040 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:33.040 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:33.040 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:33.040 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:33.040 INFO:teuthology.orchestra.run.smithi047.stdout: "registry_password": { 2024-03-21T10:35:33.040 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "registry_password", 2024-03-21T10:35:33.040 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T10:35:33.040 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:33.041 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T10:35:33.041 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-03-21T10:35:33.041 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:33.041 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:33.041 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:33.041 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-03-21T10:35:33.041 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:33.041 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:33.041 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:33.041 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:33.041 INFO:teuthology.orchestra.run.smithi047.stdout: "registry_url": { 2024-03-21T10:35:33.041 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "registry_url", 2024-03-21T10:35:33.041 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T10:35:33.042 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:33.042 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T10:35:33.042 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-03-21T10:35:33.042 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:33.042 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:33.042 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:33.042 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-03-21T10:35:33.042 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:33.042 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:33.042 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:33.042 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:33.042 INFO:teuthology.orchestra.run.smithi047.stdout: "registry_username": { 2024-03-21T10:35:33.043 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "registry_username", 2024-03-21T10:35:33.043 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T10:35:33.043 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:33.043 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T10:35:33.043 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-03-21T10:35:33.043 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:33.043 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:33.043 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:33.043 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-03-21T10:35:33.043 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:33.043 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:33.043 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:33.043 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:33.043 INFO:teuthology.orchestra.run.smithi047.stdout: "secure_monitoring_stack": { 2024-03-21T10:35:33.044 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "secure_monitoring_stack", 2024-03-21T10:35:33.044 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T10:35:33.044 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:33.044 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T10:35:33.044 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-03-21T10:35:33.044 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:33.044 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:33.044 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:33.044 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-03-21T10:35:33.044 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:33.044 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:33.044 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:33.044 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:33.044 INFO:teuthology.orchestra.run.smithi047.stdout: "service_discovery_port": { 2024-03-21T10:35:33.044 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "service_discovery_port", 2024-03-21T10:35:33.045 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "int", 2024-03-21T10:35:33.045 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:33.045 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T10:35:33.045 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "8765", 2024-03-21T10:35:33.045 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:33.045 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:33.045 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:33.045 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "cephadm service discovery port", 2024-03-21T10:35:33.045 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:33.045 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:33.045 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:33.045 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:33.045 INFO:teuthology.orchestra.run.smithi047.stdout: "ssh_config_file": { 2024-03-21T10:35:33.045 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "ssh_config_file", 2024-03-21T10:35:33.045 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T10:35:33.046 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:33.046 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T10:35:33.046 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-03-21T10:35:33.046 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:33.046 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:33.046 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:33.046 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-03-21T10:35:33.046 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:33.046 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:33.046 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:33.046 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:33.046 INFO:teuthology.orchestra.run.smithi047.stdout: "use_agent": { 2024-03-21T10:35:33.046 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "use_agent", 2024-03-21T10:35:33.047 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T10:35:33.047 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:33.047 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T10:35:33.047 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-03-21T10:35:33.047 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:33.047 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:33.047 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:33.047 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-03-21T10:35:33.047 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:33.047 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:33.047 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:33.047 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:33.047 INFO:teuthology.orchestra.run.smithi047.stdout: "use_repo_digest": { 2024-03-21T10:35:33.047 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "use_repo_digest", 2024-03-21T10:35:33.047 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T10:35:33.047 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:33.048 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T10:35:33.048 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "True", 2024-03-21T10:35:33.048 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:33.048 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:33.048 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:33.048 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-03-21T10:35:33.048 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:33.048 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:33.048 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:33.048 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:33.048 INFO:teuthology.orchestra.run.smithi047.stdout: "warn_on_failed_host_check": { 2024-03-21T10:35:33.048 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "warn_on_failed_host_check", 2024-03-21T10:35:33.048 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T10:35:33.048 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:33.049 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T10:35:33.049 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "True", 2024-03-21T10:35:33.049 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:33.049 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:33.049 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:33.049 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "raise a health warning if the host check fails", 2024-03-21T10:35:33.049 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:33.049 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:33.049 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:33.049 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:33.049 INFO:teuthology.orchestra.run.smithi047.stdout: "warn_on_stray_daemons": { 2024-03-21T10:35:33.049 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "warn_on_stray_daemons", 2024-03-21T10:35:33.049 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T10:35:33.049 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:33.049 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T10:35:33.050 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "True", 2024-03-21T10:35:33.050 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:33.050 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:33.050 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:33.050 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-03-21T10:35:33.050 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:33.050 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:33.050 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:33.050 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:33.050 INFO:teuthology.orchestra.run.smithi047.stdout: "warn_on_stray_hosts": { 2024-03-21T10:35:33.050 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "warn_on_stray_hosts", 2024-03-21T10:35:33.050 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T10:35:33.050 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:33.050 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T10:35:33.051 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "True", 2024-03-21T10:35:33.051 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:33.051 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:33.051 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:33.051 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-03-21T10:35:33.051 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:33.051 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:33.051 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:33.051 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-21T10:35:33.051 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-21T10:35:33.051 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:33.051 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-03-21T10:35:33.051 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "crash", 2024-03-21T10:35:33.051 INFO:teuthology.orchestra.run.smithi047.stdout: "can_run": true, 2024-03-21T10:35:33.051 INFO:teuthology.orchestra.run.smithi047.stdout: "error_string": "", 2024-03-21T10:35:33.051 INFO:teuthology.orchestra.run.smithi047.stdout: "module_options": { 2024-03-21T10:35:33.052 INFO:teuthology.orchestra.run.smithi047.stdout: "log_level": { 2024-03-21T10:35:33.052 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_level", 2024-03-21T10:35:33.052 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T10:35:33.052 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:33.052 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T10:35:33.052 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-03-21T10:35:33.052 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:33.052 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:33.052 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-03-21T10:35:33.052 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-03-21T10:35:33.052 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-03-21T10:35:33.052 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-03-21T10:35:33.052 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-03-21T10:35:33.052 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-03-21T10:35:33.053 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-03-21T10:35:33.053 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-03-21T10:35:33.053 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T10:35:33.053 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:33.053 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:33.053 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:33.053 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:33.053 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster": { 2024-03-21T10:35:33.053 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster", 2024-03-21T10:35:33.053 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T10:35:33.053 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:33.053 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T10:35:33.053 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-03-21T10:35:33.053 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:33.053 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:33.054 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:33.054 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T10:35:33.054 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:33.054 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:33.054 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:33.054 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:33.054 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster_level": { 2024-03-21T10:35:33.054 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster_level", 2024-03-21T10:35:33.054 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T10:35:33.054 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:33.054 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T10:35:33.054 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "info", 2024-03-21T10:35:33.054 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:33.054 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:33.055 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-03-21T10:35:33.055 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-03-21T10:35:33.055 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-03-21T10:35:33.055 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-03-21T10:35:33.055 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-03-21T10:35:33.055 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-03-21T10:35:33.055 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-03-21T10:35:33.055 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-03-21T10:35:33.055 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T10:35:33.055 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:33.055 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:33.055 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:33.055 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:33.055 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_file": { 2024-03-21T10:35:33.056 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_file", 2024-03-21T10:35:33.056 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T10:35:33.056 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:33.056 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T10:35:33.056 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-03-21T10:35:33.056 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:33.056 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:33.056 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:33.056 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T10:35:33.056 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:33.056 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:33.056 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:33.056 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:33.056 INFO:teuthology.orchestra.run.smithi047.stdout: "retain_interval": { 2024-03-21T10:35:33.056 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "retain_interval", 2024-03-21T10:35:33.057 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "secs", 2024-03-21T10:35:33.057 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:33.057 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T10:35:33.057 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "31536000", 2024-03-21T10:35:33.057 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:33.057 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:33.057 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:33.057 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "how long to retain crashes before pruning them", 2024-03-21T10:35:33.057 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:33.057 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:33.057 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:33.057 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:33.057 INFO:teuthology.orchestra.run.smithi047.stdout: "warn_recent_interval": { 2024-03-21T10:35:33.057 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "warn_recent_interval", 2024-03-21T10:35:33.057 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "secs", 2024-03-21T10:35:33.058 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:33.058 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T10:35:33.058 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "1209600", 2024-03-21T10:35:33.058 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:33.058 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:33.058 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:33.058 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "time interval in which to warn about recent crashes", 2024-03-21T10:35:33.058 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:33.058 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:33.058 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:33.058 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-21T10:35:33.058 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-21T10:35:33.058 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:33.058 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-03-21T10:35:33.059 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "dashboard", 2024-03-21T10:35:33.059 INFO:teuthology.orchestra.run.smithi047.stdout: "can_run": true, 2024-03-21T10:35:33.059 INFO:teuthology.orchestra.run.smithi047.stdout: "error_string": "", 2024-03-21T10:35:33.059 INFO:teuthology.orchestra.run.smithi047.stdout: "module_options": { 2024-03-21T10:35:33.059 INFO:teuthology.orchestra.run.smithi047.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-03-21T10:35:33.059 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-03-21T10:35:33.059 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "int", 2024-03-21T10:35:33.059 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:33.059 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T10:35:33.059 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "10", 2024-03-21T10:35:33.059 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:33.059 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:33.059 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:33.059 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T10:35:33.059 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:33.060 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:33.060 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:33.060 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:33.060 INFO:teuthology.orchestra.run.smithi047.stdout: "ALERTMANAGER_API_HOST": { 2024-03-21T10:35:33.060 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "ALERTMANAGER_API_HOST", 2024-03-21T10:35:33.060 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T10:35:33.060 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:33.060 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T10:35:33.060 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-03-21T10:35:33.060 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:33.060 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:33.060 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:33.060 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T10:35:33.060 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:33.061 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:33.061 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:33.061 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:33.061 INFO:teuthology.orchestra.run.smithi047.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-03-21T10:35:33.061 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-03-21T10:35:33.061 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T10:35:33.061 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:33.061 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T10:35:33.061 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "True", 2024-03-21T10:35:33.061 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:33.061 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:33.061 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:33.061 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T10:35:33.061 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:33.061 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:33.062 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:33.062 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:33.062 INFO:teuthology.orchestra.run.smithi047.stdout: "AUDIT_API_ENABLED": { 2024-03-21T10:35:33.062 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "AUDIT_API_ENABLED", 2024-03-21T10:35:33.062 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T10:35:33.062 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:33.062 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T10:35:33.062 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-03-21T10:35:33.062 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:33.062 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:33.062 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:33.062 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T10:35:33.062 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:33.062 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:33.063 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:33.063 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:33.063 INFO:teuthology.orchestra.run.smithi047.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-03-21T10:35:33.063 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-03-21T10:35:33.063 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T10:35:33.063 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:33.063 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T10:35:33.063 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "True", 2024-03-21T10:35:33.063 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:33.063 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:33.063 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:33.063 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T10:35:33.063 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:33.063 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:33.064 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:33.064 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:33.064 INFO:teuthology.orchestra.run.smithi047.stdout: "ENABLE_BROWSABLE_API": { 2024-03-21T10:35:33.064 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "ENABLE_BROWSABLE_API", 2024-03-21T10:35:33.064 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T10:35:33.064 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:33.064 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T10:35:33.064 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "True", 2024-03-21T10:35:33.064 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:33.064 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:33.064 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:33.064 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T10:35:33.064 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:33.064 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:33.064 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:33.064 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:33.065 INFO:teuthology.orchestra.run.smithi047.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-03-21T10:35:33.065 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-03-21T10:35:33.065 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T10:35:33.065 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:33.065 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T10:35:33.065 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "True", 2024-03-21T10:35:33.065 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:33.065 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:33.065 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:33.065 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T10:35:33.065 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:33.065 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:33.065 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:33.066 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:33.066 INFO:teuthology.orchestra.run.smithi047.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-03-21T10:35:33.066 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-03-21T10:35:33.066 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T10:35:33.066 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:33.066 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T10:35:33.066 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "True", 2024-03-21T10:35:33.066 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:33.066 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:33.066 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:33.066 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T10:35:33.066 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:33.066 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:33.066 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:33.067 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:33.067 INFO:teuthology.orchestra.run.smithi047.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-03-21T10:35:33.067 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-03-21T10:35:33.067 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T10:35:33.067 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:33.067 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T10:35:33.067 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "True", 2024-03-21T10:35:33.067 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:33.067 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:33.067 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:33.067 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T10:35:33.067 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:33.067 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:33.067 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:33.068 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:33.068 INFO:teuthology.orchestra.run.smithi047.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-03-21T10:35:33.068 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-03-21T10:35:33.068 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T10:35:33.068 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:33.068 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T10:35:33.068 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "True", 2024-03-21T10:35:33.068 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:33.068 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:33.068 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:33.068 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T10:35:33.068 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:33.068 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:33.068 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:33.068 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:33.069 INFO:teuthology.orchestra.run.smithi047.stdout: "FEATURE_TOGGLE_NFS": { 2024-03-21T10:35:33.069 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-03-21T10:35:33.069 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T10:35:33.069 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:33.069 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T10:35:33.069 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "True", 2024-03-21T10:35:33.069 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:33.069 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:33.069 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:33.069 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T10:35:33.069 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:33.069 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:33.069 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:33.069 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:33.070 INFO:teuthology.orchestra.run.smithi047.stdout: "FEATURE_TOGGLE_RBD": { 2024-03-21T10:35:33.070 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-03-21T10:35:33.070 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T10:35:33.070 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:33.070 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T10:35:33.070 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "True", 2024-03-21T10:35:33.070 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:33.070 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:33.070 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:33.070 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T10:35:33.070 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:33.070 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:33.070 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:33.071 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:33.071 INFO:teuthology.orchestra.run.smithi047.stdout: "FEATURE_TOGGLE_RGW": { 2024-03-21T10:35:33.071 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-03-21T10:35:33.071 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T10:35:33.071 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:33.071 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T10:35:33.071 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "True", 2024-03-21T10:35:33.071 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:33.071 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:33.071 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:33.071 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T10:35:33.071 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:33.071 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:33.071 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:33.072 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:33.072 INFO:teuthology.orchestra.run.smithi047.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-03-21T10:35:33.072 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-03-21T10:35:33.072 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T10:35:33.072 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:33.072 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T10:35:33.072 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-03-21T10:35:33.072 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:33.072 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:33.072 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:33.072 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T10:35:33.072 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:33.072 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:33.072 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:33.073 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:33.073 INFO:teuthology.orchestra.run.smithi047.stdout: "GRAFANA_API_PASSWORD": { 2024-03-21T10:35:33.073 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "GRAFANA_API_PASSWORD", 2024-03-21T10:35:33.073 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T10:35:33.073 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:33.073 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T10:35:33.073 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "admin", 2024-03-21T10:35:33.073 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:33.073 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:33.073 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:33.073 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T10:35:33.073 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:33.073 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:33.073 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:33.073 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:33.074 INFO:teuthology.orchestra.run.smithi047.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-03-21T10:35:33.074 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-03-21T10:35:33.074 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T10:35:33.074 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:33.074 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T10:35:33.074 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "True", 2024-03-21T10:35:33.074 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:33.074 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:33.074 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:33.074 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T10:35:33.074 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:33.074 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:33.074 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:33.074 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:33.075 INFO:teuthology.orchestra.run.smithi047.stdout: "GRAFANA_API_URL": { 2024-03-21T10:35:33.075 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "GRAFANA_API_URL", 2024-03-21T10:35:33.075 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T10:35:33.075 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:33.075 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T10:35:33.075 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-03-21T10:35:33.075 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:33.075 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:33.075 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:33.075 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T10:35:33.075 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:33.075 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:33.075 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:33.075 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:33.075 INFO:teuthology.orchestra.run.smithi047.stdout: "GRAFANA_API_USERNAME": { 2024-03-21T10:35:33.075 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "GRAFANA_API_USERNAME", 2024-03-21T10:35:33.076 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T10:35:33.076 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:33.076 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T10:35:33.076 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "admin", 2024-03-21T10:35:33.076 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:33.076 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:33.076 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:33.076 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T10:35:33.076 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:33.076 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:33.076 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:33.076 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:33.076 INFO:teuthology.orchestra.run.smithi047.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-03-21T10:35:33.076 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-03-21T10:35:33.077 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T10:35:33.077 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:33.077 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T10:35:33.077 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-03-21T10:35:33.077 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:33.077 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:33.077 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:33.077 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T10:35:33.077 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:33.077 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:33.077 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:33.077 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:33.077 INFO:teuthology.orchestra.run.smithi047.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-03-21T10:35:33.078 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-03-21T10:35:33.078 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T10:35:33.078 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:33.078 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T10:35:33.078 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-03-21T10:35:33.078 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:33.078 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:33.078 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:33.078 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T10:35:33.078 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:33.078 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:33.078 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:33.078 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:33.078 INFO:teuthology.orchestra.run.smithi047.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-03-21T10:35:33.079 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-03-21T10:35:33.079 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T10:35:33.079 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:33.079 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T10:35:33.079 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "True", 2024-03-21T10:35:33.079 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:33.079 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:33.079 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:33.079 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T10:35:33.079 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:33.079 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:33.079 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:33.080 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:33.080 INFO:teuthology.orchestra.run.smithi047.stdout: "ISSUE_TRACKER_API_KEY": { 2024-03-21T10:35:33.080 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-03-21T10:35:33.080 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T10:35:33.080 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:33.080 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T10:35:33.080 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-03-21T10:35:33.080 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:33.080 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:33.080 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:33.080 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T10:35:33.080 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:33.080 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:33.080 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:33.081 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:33.081 INFO:teuthology.orchestra.run.smithi047.stdout: "MULTICLUSTER_CONFIG": { 2024-03-21T10:35:33.081 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "MULTICLUSTER_CONFIG", 2024-03-21T10:35:33.081 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T10:35:33.081 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:33.081 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T10:35:33.081 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "{}", 2024-03-21T10:35:33.081 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:33.081 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:33.081 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:33.081 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T10:35:33.081 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:33.081 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:33.081 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:33.082 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:33.082 INFO:teuthology.orchestra.run.smithi047.stdout: "PROMETHEUS_API_HOST": { 2024-03-21T10:35:33.082 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "PROMETHEUS_API_HOST", 2024-03-21T10:35:33.082 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T10:35:33.082 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:33.082 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T10:35:33.082 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-03-21T10:35:33.082 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:33.082 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:33.082 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:33.082 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T10:35:33.082 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:33.082 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:33.083 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:33.083 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:33.083 INFO:teuthology.orchestra.run.smithi047.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-03-21T10:35:33.083 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-03-21T10:35:33.083 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T10:35:33.083 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:33.083 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T10:35:33.083 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "True", 2024-03-21T10:35:33.083 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:33.083 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:33.083 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:33.083 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T10:35:33.083 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:33.083 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:33.083 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:33.084 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:33.084 INFO:teuthology.orchestra.run.smithi047.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-03-21T10:35:33.084 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-03-21T10:35:33.084 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T10:35:33.084 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:33.084 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T10:35:33.084 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-03-21T10:35:33.084 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:33.084 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:33.084 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:33.084 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T10:35:33.084 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:33.084 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:33.084 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:33.084 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:33.085 INFO:teuthology.orchestra.run.smithi047.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-03-21T10:35:33.085 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-03-21T10:35:33.085 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T10:35:33.085 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:33.085 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T10:35:33.085 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-03-21T10:35:33.085 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:33.085 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:33.085 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:33.085 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T10:35:33.085 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:33.085 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:33.085 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:33.085 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:33.085 INFO:teuthology.orchestra.run.smithi047.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-03-21T10:35:33.086 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-03-21T10:35:33.086 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T10:35:33.086 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:33.086 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T10:35:33.086 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "True", 2024-03-21T10:35:33.086 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:33.086 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:33.086 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:33.086 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T10:35:33.086 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:33.086 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:33.086 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:33.086 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:33.086 INFO:teuthology.orchestra.run.smithi047.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-03-21T10:35:33.087 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-03-21T10:35:33.087 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T10:35:33.087 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:33.087 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T10:35:33.087 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "True", 2024-03-21T10:35:33.087 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:33.087 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:33.087 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:33.087 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T10:35:33.087 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:33.087 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:33.087 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:33.087 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:33.087 INFO:teuthology.orchestra.run.smithi047.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-03-21T10:35:33.088 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-03-21T10:35:33.088 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T10:35:33.088 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:33.088 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T10:35:33.088 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-03-21T10:35:33.088 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:33.088 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:33.088 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:33.088 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T10:35:33.088 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:33.088 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:33.088 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:33.088 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:33.088 INFO:teuthology.orchestra.run.smithi047.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-03-21T10:35:33.088 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-03-21T10:35:33.089 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T10:35:33.089 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:33.089 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T10:35:33.089 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-03-21T10:35:33.089 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:33.089 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:33.089 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:33.089 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T10:35:33.089 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:33.089 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:33.089 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:33.089 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:33.089 INFO:teuthology.orchestra.run.smithi047.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-03-21T10:35:33.090 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-03-21T10:35:33.090 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T10:35:33.090 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:33.090 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T10:35:33.090 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-03-21T10:35:33.090 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:33.090 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:33.090 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:33.090 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T10:35:33.090 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:33.090 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:33.090 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:33.090 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:33.091 INFO:teuthology.orchestra.run.smithi047.stdout: "PWD_POLICY_ENABLED": { 2024-03-21T10:35:33.091 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "PWD_POLICY_ENABLED", 2024-03-21T10:35:33.091 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T10:35:33.091 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:33.091 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T10:35:33.091 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "True", 2024-03-21T10:35:33.091 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:33.091 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:33.091 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:33.091 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T10:35:33.091 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:33.091 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:33.091 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:33.091 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:33.091 INFO:teuthology.orchestra.run.smithi047.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-03-21T10:35:33.092 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-03-21T10:35:33.092 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T10:35:33.092 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:33.092 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T10:35:33.092 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-03-21T10:35:33.092 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:33.092 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:33.092 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:33.092 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T10:35:33.092 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:33.092 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:33.092 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:33.092 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:33.093 INFO:teuthology.orchestra.run.smithi047.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-03-21T10:35:33.093 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-03-21T10:35:33.093 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "int", 2024-03-21T10:35:33.093 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:33.093 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T10:35:33.093 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "10", 2024-03-21T10:35:33.093 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:33.093 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:33.093 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:33.093 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T10:35:33.093 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:33.093 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:33.093 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:33.094 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:33.094 INFO:teuthology.orchestra.run.smithi047.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-03-21T10:35:33.094 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-03-21T10:35:33.094 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "int", 2024-03-21T10:35:33.094 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:33.094 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T10:35:33.094 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "8", 2024-03-21T10:35:33.094 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:33.094 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:33.094 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:33.094 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T10:35:33.094 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:33.094 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:33.094 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:33.095 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:33.095 INFO:teuthology.orchestra.run.smithi047.stdout: "REST_REQUESTS_TIMEOUT": { 2024-03-21T10:35:33.095 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-03-21T10:35:33.095 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "int", 2024-03-21T10:35:33.095 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:33.095 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T10:35:33.095 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "45", 2024-03-21T10:35:33.095 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:33.095 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:33.095 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:33.095 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T10:35:33.095 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:33.095 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:33.095 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:33.096 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:33.096 INFO:teuthology.orchestra.run.smithi047.stdout: "RGW_API_ACCESS_KEY": { 2024-03-21T10:35:33.096 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "RGW_API_ACCESS_KEY", 2024-03-21T10:35:33.096 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T10:35:33.096 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:33.096 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T10:35:33.096 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-03-21T10:35:33.096 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:33.096 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:33.096 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:33.096 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T10:35:33.096 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:33.096 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:33.096 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:33.096 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:33.097 INFO:teuthology.orchestra.run.smithi047.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-03-21T10:35:33.097 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-03-21T10:35:33.097 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T10:35:33.097 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:33.097 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T10:35:33.097 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "admin", 2024-03-21T10:35:33.097 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:33.097 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:33.097 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:33.097 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T10:35:33.097 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:33.097 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:33.097 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:33.097 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:33.098 INFO:teuthology.orchestra.run.smithi047.stdout: "RGW_API_SECRET_KEY": { 2024-03-21T10:35:33.098 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "RGW_API_SECRET_KEY", 2024-03-21T10:35:33.098 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T10:35:33.098 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:33.098 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T10:35:33.098 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-03-21T10:35:33.098 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:33.098 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:33.098 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:33.098 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T10:35:33.098 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:33.098 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:33.098 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:33.098 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:33.098 INFO:teuthology.orchestra.run.smithi047.stdout: "RGW_API_SSL_VERIFY": { 2024-03-21T10:35:33.099 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "RGW_API_SSL_VERIFY", 2024-03-21T10:35:33.099 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T10:35:33.099 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:33.099 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T10:35:33.099 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "True", 2024-03-21T10:35:33.099 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:33.102 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:33.102 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:33.102 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T10:35:33.102 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:33.102 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:33.102 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:33.102 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:33.102 INFO:teuthology.orchestra.run.smithi047.stdout: "UNSAFE_TLS_v1_2": { 2024-03-21T10:35:33.102 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "UNSAFE_TLS_v1_2", 2024-03-21T10:35:33.102 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T10:35:33.102 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:33.102 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T10:35:33.102 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-03-21T10:35:33.103 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:33.103 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:33.103 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:33.103 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T10:35:33.103 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:33.103 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:33.103 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:33.103 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:33.103 INFO:teuthology.orchestra.run.smithi047.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-03-21T10:35:33.103 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-03-21T10:35:33.103 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "int", 2024-03-21T10:35:33.103 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:33.103 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T10:35:33.103 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "0", 2024-03-21T10:35:33.103 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:33.104 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:33.104 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:33.104 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T10:35:33.104 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:33.104 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:33.104 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:33.104 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:33.104 INFO:teuthology.orchestra.run.smithi047.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-03-21T10:35:33.104 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-03-21T10:35:33.104 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "int", 2024-03-21T10:35:33.104 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:33.104 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T10:35:33.104 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "10", 2024-03-21T10:35:33.105 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:33.105 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:33.105 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:33.105 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T10:35:33.105 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:33.105 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:33.105 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:33.105 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:33.105 INFO:teuthology.orchestra.run.smithi047.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-03-21T10:35:33.105 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-03-21T10:35:33.105 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "int", 2024-03-21T10:35:33.105 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:33.105 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T10:35:33.105 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "5", 2024-03-21T10:35:33.105 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:33.105 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:33.106 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:33.106 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T10:35:33.106 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:33.106 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:33.106 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:33.106 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:33.106 INFO:teuthology.orchestra.run.smithi047.stdout: "cross_origin_url": { 2024-03-21T10:35:33.106 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "cross_origin_url", 2024-03-21T10:35:33.106 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T10:35:33.106 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:33.106 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T10:35:33.106 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-03-21T10:35:33.106 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:33.106 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:33.107 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:33.107 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T10:35:33.107 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:33.107 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:33.107 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:33.107 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:33.107 INFO:teuthology.orchestra.run.smithi047.stdout: "crt_file": { 2024-03-21T10:35:33.107 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "crt_file", 2024-03-21T10:35:33.107 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T10:35:33.107 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:33.107 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T10:35:33.107 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-03-21T10:35:33.107 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:33.107 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:33.108 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:33.108 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T10:35:33.108 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:33.108 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:33.108 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:33.108 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:33.108 INFO:teuthology.orchestra.run.smithi047.stdout: "debug": { 2024-03-21T10:35:33.108 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "debug", 2024-03-21T10:35:33.108 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T10:35:33.108 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:33.108 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T10:35:33.108 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-03-21T10:35:33.108 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:33.108 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:33.109 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:33.109 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Enable/disable debug options", 2024-03-21T10:35:33.109 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:33.109 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:33.109 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:33.109 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:33.109 INFO:teuthology.orchestra.run.smithi047.stdout: "jwt_token_ttl": { 2024-03-21T10:35:33.109 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "jwt_token_ttl", 2024-03-21T10:35:33.109 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "int", 2024-03-21T10:35:33.109 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:33.109 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T10:35:33.109 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "28800", 2024-03-21T10:35:33.109 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:33.109 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:33.110 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:33.110 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T10:35:33.110 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:33.110 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:33.110 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:33.110 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:33.110 INFO:teuthology.orchestra.run.smithi047.stdout: "key_file": { 2024-03-21T10:35:33.110 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "key_file", 2024-03-21T10:35:33.110 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T10:35:33.110 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:33.110 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T10:35:33.110 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-03-21T10:35:33.110 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:33.111 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:33.111 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:33.111 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T10:35:33.111 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:33.111 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:33.111 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:33.111 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:33.111 INFO:teuthology.orchestra.run.smithi047.stdout: "log_level": { 2024-03-21T10:35:33.111 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_level", 2024-03-21T10:35:33.111 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T10:35:33.111 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:33.111 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T10:35:33.111 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-03-21T10:35:33.111 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:33.112 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:33.112 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-03-21T10:35:33.112 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-03-21T10:35:33.112 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-03-21T10:35:33.112 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-03-21T10:35:33.112 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-03-21T10:35:33.112 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-03-21T10:35:33.112 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-03-21T10:35:33.112 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-03-21T10:35:33.112 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T10:35:33.112 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:33.112 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:33.112 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:33.112 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:33.112 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster": { 2024-03-21T10:35:33.113 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster", 2024-03-21T10:35:33.113 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T10:35:33.113 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:33.113 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T10:35:33.113 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-03-21T10:35:33.113 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:33.113 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:33.113 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:33.113 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T10:35:33.113 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:33.113 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:33.113 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:33.113 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:33.113 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster_level": { 2024-03-21T10:35:33.113 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster_level", 2024-03-21T10:35:33.114 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T10:35:33.114 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:33.114 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T10:35:33.114 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "info", 2024-03-21T10:35:33.114 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:33.114 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:33.114 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-03-21T10:35:33.114 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-03-21T10:35:33.114 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-03-21T10:35:33.114 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-03-21T10:35:33.114 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-03-21T10:35:33.114 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-03-21T10:35:33.114 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-03-21T10:35:33.115 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-03-21T10:35:33.115 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T10:35:33.115 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:33.115 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:33.115 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:33.115 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:33.115 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_file": { 2024-03-21T10:35:33.115 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_file", 2024-03-21T10:35:33.115 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T10:35:33.115 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:33.115 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T10:35:33.115 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-03-21T10:35:33.115 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:33.115 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:33.115 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:33.115 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T10:35:33.116 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:33.116 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:33.116 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:33.116 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:33.116 INFO:teuthology.orchestra.run.smithi047.stdout: "motd": { 2024-03-21T10:35:33.116 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "motd", 2024-03-21T10:35:33.116 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T10:35:33.116 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:33.116 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T10:35:33.116 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-03-21T10:35:33.116 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:33.116 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:33.116 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:33.116 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "The message of the day", 2024-03-21T10:35:33.117 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:33.117 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:33.117 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:33.117 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:33.117 INFO:teuthology.orchestra.run.smithi047.stdout: "redirect_resolve_ip_addr": { 2024-03-21T10:35:33.117 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "redirect_resolve_ip_addr", 2024-03-21T10:35:33.117 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T10:35:33.117 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:33.117 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T10:35:33.117 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-03-21T10:35:33.117 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:33.117 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:33.117 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:33.117 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T10:35:33.117 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:33.118 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:33.118 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:33.118 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:33.118 INFO:teuthology.orchestra.run.smithi047.stdout: "server_addr": { 2024-03-21T10:35:33.118 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "server_addr", 2024-03-21T10:35:33.118 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T10:35:33.118 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:33.118 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T10:35:33.118 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "::", 2024-03-21T10:35:33.118 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:33.118 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:33.118 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:33.118 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T10:35:33.118 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:33.118 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:33.119 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:33.119 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:33.119 INFO:teuthology.orchestra.run.smithi047.stdout: "server_port": { 2024-03-21T10:35:33.119 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "server_port", 2024-03-21T10:35:33.119 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "int", 2024-03-21T10:35:33.119 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:33.119 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T10:35:33.119 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "8080", 2024-03-21T10:35:33.119 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:33.119 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:33.119 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:33.119 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T10:35:33.119 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:33.119 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:33.119 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:33.120 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:33.120 INFO:teuthology.orchestra.run.smithi047.stdout: "ssl": { 2024-03-21T10:35:33.120 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "ssl", 2024-03-21T10:35:33.120 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T10:35:33.120 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:33.120 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T10:35:33.120 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "True", 2024-03-21T10:35:33.120 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:33.120 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:33.120 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:33.120 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T10:35:33.120 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:33.120 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:33.120 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:33.120 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:33.120 INFO:teuthology.orchestra.run.smithi047.stdout: "ssl_server_port": { 2024-03-21T10:35:33.121 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "ssl_server_port", 2024-03-21T10:35:33.121 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "int", 2024-03-21T10:35:33.121 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:33.121 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T10:35:33.121 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "8443", 2024-03-21T10:35:33.121 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:33.121 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:33.121 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:33.121 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T10:35:33.121 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:33.121 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:33.121 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:33.122 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:33.122 INFO:teuthology.orchestra.run.smithi047.stdout: "standby_behaviour": { 2024-03-21T10:35:33.122 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "standby_behaviour", 2024-03-21T10:35:33.122 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T10:35:33.122 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:33.122 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T10:35:33.122 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "redirect", 2024-03-21T10:35:33.122 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:33.122 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:33.122 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-03-21T10:35:33.122 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-03-21T10:35:33.122 INFO:teuthology.orchestra.run.smithi047.stdout: "redirect" 2024-03-21T10:35:33.122 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-03-21T10:35:33.122 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T10:35:33.123 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:33.123 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:33.123 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:33.123 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:33.123 INFO:teuthology.orchestra.run.smithi047.stdout: "standby_error_status_code": { 2024-03-21T10:35:33.123 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "standby_error_status_code", 2024-03-21T10:35:33.123 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "int", 2024-03-21T10:35:33.123 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:33.123 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T10:35:33.123 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "500", 2024-03-21T10:35:33.123 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "400", 2024-03-21T10:35:33.123 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "599", 2024-03-21T10:35:33.123 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:33.124 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T10:35:33.124 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:33.124 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:33.124 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:33.124 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:33.124 INFO:teuthology.orchestra.run.smithi047.stdout: "url_prefix": { 2024-03-21T10:35:33.124 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "url_prefix", 2024-03-21T10:35:33.124 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T10:35:33.124 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:33.124 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T10:35:33.124 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-03-21T10:35:33.124 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:33.125 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:33.125 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:33.125 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T10:35:33.125 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:33.125 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:33.125 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:33.125 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-21T10:35:33.125 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-21T10:35:33.125 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:33.125 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-03-21T10:35:33.125 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "devicehealth", 2024-03-21T10:35:33.125 INFO:teuthology.orchestra.run.smithi047.stdout: "can_run": true, 2024-03-21T10:35:33.125 INFO:teuthology.orchestra.run.smithi047.stdout: "error_string": "", 2024-03-21T10:35:33.126 INFO:teuthology.orchestra.run.smithi047.stdout: "module_options": { 2024-03-21T10:35:33.126 INFO:teuthology.orchestra.run.smithi047.stdout: "enable_monitoring": { 2024-03-21T10:35:33.126 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "enable_monitoring", 2024-03-21T10:35:33.126 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T10:35:33.126 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:33.126 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T10:35:33.126 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "True", 2024-03-21T10:35:33.126 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:33.126 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:33.126 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:33.126 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "monitor device health metrics", 2024-03-21T10:35:33.126 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:33.126 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:33.127 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:33.127 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:33.127 INFO:teuthology.orchestra.run.smithi047.stdout: "log_level": { 2024-03-21T10:35:33.127 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_level", 2024-03-21T10:35:33.127 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T10:35:33.127 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:33.127 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T10:35:33.127 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-03-21T10:35:33.127 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:33.127 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:33.127 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-03-21T10:35:33.127 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-03-21T10:35:33.127 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-03-21T10:35:33.127 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-03-21T10:35:33.128 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-03-21T10:35:33.128 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-03-21T10:35:33.128 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-03-21T10:35:33.128 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-03-21T10:35:33.128 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T10:35:33.128 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:33.128 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:33.128 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:33.128 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:33.128 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster": { 2024-03-21T10:35:33.128 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster", 2024-03-21T10:35:33.128 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T10:35:33.128 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:33.128 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T10:35:33.129 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-03-21T10:35:33.129 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:33.129 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:33.129 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:33.129 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T10:35:33.129 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:33.129 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:33.129 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:33.129 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:33.129 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster_level": { 2024-03-21T10:35:33.129 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster_level", 2024-03-21T10:35:33.129 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T10:35:33.129 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:33.129 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T10:35:33.129 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "info", 2024-03-21T10:35:33.130 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:33.130 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:33.130 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-03-21T10:35:33.130 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-03-21T10:35:33.130 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-03-21T10:35:33.130 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-03-21T10:35:33.130 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-03-21T10:35:33.130 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-03-21T10:35:33.130 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-03-21T10:35:33.130 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-03-21T10:35:33.130 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T10:35:33.130 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:33.131 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:33.131 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:33.131 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:33.131 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_file": { 2024-03-21T10:35:33.131 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_file", 2024-03-21T10:35:33.131 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T10:35:33.131 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:33.131 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T10:35:33.131 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-03-21T10:35:33.131 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:33.131 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:33.131 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:33.131 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T10:35:33.131 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:33.132 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:33.132 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:33.132 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:33.132 INFO:teuthology.orchestra.run.smithi047.stdout: "mark_out_threshold": { 2024-03-21T10:35:33.132 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "mark_out_threshold", 2024-03-21T10:35:33.132 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "secs", 2024-03-21T10:35:33.132 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:33.132 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T10:35:33.132 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "2419200", 2024-03-21T10:35:33.132 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:33.132 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:33.132 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:33.132 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-03-21T10:35:33.133 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:33.133 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:33.133 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:33.133 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:33.133 INFO:teuthology.orchestra.run.smithi047.stdout: "pool_name": { 2024-03-21T10:35:33.133 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "pool_name", 2024-03-21T10:35:33.133 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T10:35:33.133 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:33.133 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T10:35:33.133 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "device_health_metrics", 2024-03-21T10:35:33.133 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:33.133 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:33.133 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:33.133 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "name of pool in which to store device health metrics", 2024-03-21T10:35:33.133 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:33.134 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:33.134 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:33.134 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:33.134 INFO:teuthology.orchestra.run.smithi047.stdout: "retention_period": { 2024-03-21T10:35:33.134 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "retention_period", 2024-03-21T10:35:33.134 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "secs", 2024-03-21T10:35:33.134 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:33.134 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T10:35:33.134 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "15552000", 2024-03-21T10:35:33.134 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:33.134 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:33.134 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:33.134 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "how long to retain device health metrics", 2024-03-21T10:35:33.135 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:33.135 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:33.135 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:33.135 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:33.135 INFO:teuthology.orchestra.run.smithi047.stdout: "scrape_frequency": { 2024-03-21T10:35:33.135 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "scrape_frequency", 2024-03-21T10:35:33.135 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "secs", 2024-03-21T10:35:33.135 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:33.135 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T10:35:33.135 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "86400", 2024-03-21T10:35:33.135 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:33.135 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:33.135 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:33.135 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "how frequently to scrape device health metrics", 2024-03-21T10:35:33.135 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:33.136 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:33.136 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:33.136 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:33.136 INFO:teuthology.orchestra.run.smithi047.stdout: "self_heal": { 2024-03-21T10:35:33.136 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "self_heal", 2024-03-21T10:35:33.136 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T10:35:33.136 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:33.136 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T10:35:33.136 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "True", 2024-03-21T10:35:33.136 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:33.136 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:33.136 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:33.136 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-03-21T10:35:33.136 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:33.137 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:33.137 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:33.137 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:33.137 INFO:teuthology.orchestra.run.smithi047.stdout: "sleep_interval": { 2024-03-21T10:35:33.137 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "sleep_interval", 2024-03-21T10:35:33.137 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "secs", 2024-03-21T10:35:33.137 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:33.137 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T10:35:33.137 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "600", 2024-03-21T10:35:33.137 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:33.137 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:33.137 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:33.137 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "how frequently to wake up and check device health", 2024-03-21T10:35:33.137 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:33.138 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:33.138 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:33.138 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:33.138 INFO:teuthology.orchestra.run.smithi047.stdout: "warn_threshold": { 2024-03-21T10:35:33.138 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "warn_threshold", 2024-03-21T10:35:33.138 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "secs", 2024-03-21T10:35:33.138 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:33.138 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T10:35:33.138 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "7257600", 2024-03-21T10:35:33.138 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:33.138 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:33.138 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:33.138 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-03-21T10:35:33.138 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:33.139 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:33.139 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:33.139 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-21T10:35:33.139 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-21T10:35:33.139 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:33.139 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-03-21T10:35:33.139 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "diskprediction_local", 2024-03-21T10:35:33.139 INFO:teuthology.orchestra.run.smithi047.stdout: "can_run": true, 2024-03-21T10:35:33.139 INFO:teuthology.orchestra.run.smithi047.stdout: "error_string": "", 2024-03-21T10:35:33.139 INFO:teuthology.orchestra.run.smithi047.stdout: "module_options": { 2024-03-21T10:35:33.139 INFO:teuthology.orchestra.run.smithi047.stdout: "log_level": { 2024-03-21T10:35:33.139 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_level", 2024-03-21T10:35:33.139 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T10:35:33.139 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:33.140 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T10:35:33.140 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-03-21T10:35:33.140 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:33.140 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:33.140 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-03-21T10:35:33.140 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-03-21T10:35:33.140 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-03-21T10:35:33.140 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-03-21T10:35:33.140 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-03-21T10:35:33.140 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-03-21T10:35:33.140 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-03-21T10:35:33.140 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-03-21T10:35:33.140 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T10:35:33.140 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:33.141 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:33.141 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:33.141 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:33.141 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster": { 2024-03-21T10:35:33.141 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster", 2024-03-21T10:35:33.141 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T10:35:33.141 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:33.141 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T10:35:33.141 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-03-21T10:35:33.141 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:33.141 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:33.141 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:33.141 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T10:35:33.141 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:33.142 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:33.142 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:33.142 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:33.142 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster_level": { 2024-03-21T10:35:33.142 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster_level", 2024-03-21T10:35:33.142 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T10:35:33.142 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:33.142 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T10:35:33.142 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "info", 2024-03-21T10:35:33.142 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:33.142 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:33.142 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-03-21T10:35:33.142 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-03-21T10:35:33.143 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-03-21T10:35:33.143 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-03-21T10:35:33.143 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-03-21T10:35:33.143 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-03-21T10:35:33.143 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-03-21T10:35:33.143 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-03-21T10:35:33.143 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T10:35:33.143 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:33.143 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:33.143 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:33.143 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:33.143 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_file": { 2024-03-21T10:35:33.143 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_file", 2024-03-21T10:35:33.143 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T10:35:33.143 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:33.144 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T10:35:33.144 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-03-21T10:35:33.144 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:33.144 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:33.144 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:33.144 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T10:35:33.144 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:33.144 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:33.144 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:33.144 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:33.144 INFO:teuthology.orchestra.run.smithi047.stdout: "predict_interval": { 2024-03-21T10:35:33.144 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "predict_interval", 2024-03-21T10:35:33.144 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T10:35:33.144 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:33.145 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T10:35:33.145 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "86400", 2024-03-21T10:35:33.145 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:33.145 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:33.145 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:33.145 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T10:35:33.145 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:33.145 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:33.145 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:33.145 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:33.145 INFO:teuthology.orchestra.run.smithi047.stdout: "predictor_model": { 2024-03-21T10:35:33.145 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "predictor_model", 2024-03-21T10:35:33.145 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T10:35:33.146 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:33.146 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T10:35:33.146 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "prophetstor", 2024-03-21T10:35:33.146 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:33.146 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:33.146 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:33.146 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T10:35:33.146 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:33.146 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:33.146 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:33.146 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:33.147 INFO:teuthology.orchestra.run.smithi047.stdout: "sleep_interval": { 2024-03-21T10:35:33.147 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "sleep_interval", 2024-03-21T10:35:33.147 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T10:35:33.147 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:33.147 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T10:35:33.147 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "600", 2024-03-21T10:35:33.147 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:33.147 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:33.147 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:33.147 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T10:35:33.147 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:33.147 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:33.147 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:33.147 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-21T10:35:33.148 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-21T10:35:33.148 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:33.148 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-03-21T10:35:33.148 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "influx", 2024-03-21T10:35:33.148 INFO:teuthology.orchestra.run.smithi047.stdout: "can_run": false, 2024-03-21T10:35:33.148 INFO:teuthology.orchestra.run.smithi047.stdout: "error_string": "influxdb python module not found", 2024-03-21T10:35:33.148 INFO:teuthology.orchestra.run.smithi047.stdout: "module_options": { 2024-03-21T10:35:33.148 INFO:teuthology.orchestra.run.smithi047.stdout: "batch_size": { 2024-03-21T10:35:33.148 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "batch_size", 2024-03-21T10:35:33.148 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "int", 2024-03-21T10:35:33.148 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:33.148 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T10:35:33.148 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "5000", 2024-03-21T10:35:33.148 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:33.149 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:33.149 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:33.149 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-03-21T10:35:33.149 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:33.149 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:33.149 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:33.149 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:33.149 INFO:teuthology.orchestra.run.smithi047.stdout: "database": { 2024-03-21T10:35:33.149 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "database", 2024-03-21T10:35:33.149 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T10:35:33.149 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:33.149 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T10:35:33.149 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "ceph", 2024-03-21T10:35:33.149 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:33.149 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:33.150 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:33.150 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-03-21T10:35:33.150 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:33.150 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:33.150 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:33.150 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:33.150 INFO:teuthology.orchestra.run.smithi047.stdout: "hostname": { 2024-03-21T10:35:33.150 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "hostname", 2024-03-21T10:35:33.150 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T10:35:33.150 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:33.150 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T10:35:33.150 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-03-21T10:35:33.150 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:33.151 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:33.151 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:33.151 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "InfluxDB server hostname", 2024-03-21T10:35:33.151 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:33.151 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:33.151 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:33.151 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:33.151 INFO:teuthology.orchestra.run.smithi047.stdout: "interval": { 2024-03-21T10:35:33.151 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "interval", 2024-03-21T10:35:33.151 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "secs", 2024-03-21T10:35:33.151 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:33.151 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T10:35:33.151 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "30", 2024-03-21T10:35:33.151 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "5", 2024-03-21T10:35:33.152 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:33.152 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:33.152 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-03-21T10:35:33.152 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:33.152 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:33.152 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:33.152 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:33.152 INFO:teuthology.orchestra.run.smithi047.stdout: "log_level": { 2024-03-21T10:35:33.152 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_level", 2024-03-21T10:35:33.152 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T10:35:33.152 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:33.152 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T10:35:33.152 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-03-21T10:35:33.152 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:33.153 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:33.153 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-03-21T10:35:33.153 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-03-21T10:35:33.153 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-03-21T10:35:33.153 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-03-21T10:35:33.153 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-03-21T10:35:33.153 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-03-21T10:35:33.153 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-03-21T10:35:33.153 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-03-21T10:35:33.153 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T10:35:33.153 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:33.153 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:33.153 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:33.153 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:33.153 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster": { 2024-03-21T10:35:33.154 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster", 2024-03-21T10:35:33.154 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T10:35:33.154 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:33.154 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T10:35:33.154 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-03-21T10:35:33.154 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:33.154 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:33.154 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:33.154 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T10:35:33.154 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:33.154 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:33.154 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:33.154 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:33.154 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster_level": { 2024-03-21T10:35:33.155 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster_level", 2024-03-21T10:35:33.155 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T10:35:33.155 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:33.155 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T10:35:33.155 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "info", 2024-03-21T10:35:33.155 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:33.155 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:33.155 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-03-21T10:35:33.155 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-03-21T10:35:33.155 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-03-21T10:35:33.155 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-03-21T10:35:33.155 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-03-21T10:35:33.155 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-03-21T10:35:33.155 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-03-21T10:35:33.155 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-03-21T10:35:33.156 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T10:35:33.156 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:33.156 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:33.156 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:33.156 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:33.156 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_file": { 2024-03-21T10:35:33.156 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_file", 2024-03-21T10:35:33.156 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T10:35:33.156 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:33.156 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T10:35:33.156 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-03-21T10:35:33.156 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:33.156 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:33.156 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:33.156 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T10:35:33.157 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:33.157 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:33.157 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:33.157 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:33.157 INFO:teuthology.orchestra.run.smithi047.stdout: "password": { 2024-03-21T10:35:33.157 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "password", 2024-03-21T10:35:33.157 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T10:35:33.157 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:33.157 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T10:35:33.157 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-03-21T10:35:33.157 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:33.157 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:33.157 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:33.157 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "password of InfluxDB server user", 2024-03-21T10:35:33.158 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:33.158 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:33.158 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:33.158 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:33.158 INFO:teuthology.orchestra.run.smithi047.stdout: "port": { 2024-03-21T10:35:33.158 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "port", 2024-03-21T10:35:33.158 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "int", 2024-03-21T10:35:33.158 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:33.158 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T10:35:33.158 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "8086", 2024-03-21T10:35:33.158 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:33.158 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:33.158 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:33.158 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "InfluxDB server port", 2024-03-21T10:35:33.159 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:33.159 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:33.159 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:33.159 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:33.159 INFO:teuthology.orchestra.run.smithi047.stdout: "ssl": { 2024-03-21T10:35:33.159 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "ssl", 2024-03-21T10:35:33.159 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T10:35:33.159 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:33.159 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T10:35:33.159 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "false", 2024-03-21T10:35:33.159 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:33.159 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:33.159 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:33.160 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-03-21T10:35:33.160 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:33.160 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:33.160 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:33.160 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:33.160 INFO:teuthology.orchestra.run.smithi047.stdout: "threads": { 2024-03-21T10:35:33.160 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "threads", 2024-03-21T10:35:33.160 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "int", 2024-03-21T10:35:33.160 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:33.160 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T10:35:33.160 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "5", 2024-03-21T10:35:33.160 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "1", 2024-03-21T10:35:33.160 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "32", 2024-03-21T10:35:33.160 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:33.160 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-03-21T10:35:33.161 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:33.161 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:33.161 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:33.161 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:33.161 INFO:teuthology.orchestra.run.smithi047.stdout: "username": { 2024-03-21T10:35:33.161 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "username", 2024-03-21T10:35:33.161 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T10:35:33.161 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:33.161 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T10:35:33.161 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-03-21T10:35:33.161 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:33.161 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:33.161 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:33.162 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "username of InfluxDB server user", 2024-03-21T10:35:33.162 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:33.162 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:33.162 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:33.162 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:33.162 INFO:teuthology.orchestra.run.smithi047.stdout: "verify_ssl": { 2024-03-21T10:35:33.162 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "verify_ssl", 2024-03-21T10:35:33.162 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T10:35:33.162 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:33.162 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T10:35:33.162 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "true", 2024-03-21T10:35:33.162 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:33.162 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:33.163 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:33.163 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-03-21T10:35:33.163 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:33.163 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:33.163 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:33.163 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-21T10:35:33.163 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-21T10:35:33.163 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:33.163 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-03-21T10:35:33.163 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "insights", 2024-03-21T10:35:33.163 INFO:teuthology.orchestra.run.smithi047.stdout: "can_run": true, 2024-03-21T10:35:33.163 INFO:teuthology.orchestra.run.smithi047.stdout: "error_string": "", 2024-03-21T10:35:33.163 INFO:teuthology.orchestra.run.smithi047.stdout: "module_options": { 2024-03-21T10:35:33.163 INFO:teuthology.orchestra.run.smithi047.stdout: "log_level": { 2024-03-21T10:35:33.163 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_level", 2024-03-21T10:35:33.164 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T10:35:33.164 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:33.164 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T10:35:33.164 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-03-21T10:35:33.164 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:33.164 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:33.164 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-03-21T10:35:33.164 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-03-21T10:35:33.164 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-03-21T10:35:33.164 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-03-21T10:35:33.164 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-03-21T10:35:33.164 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-03-21T10:35:33.164 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-03-21T10:35:33.164 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-03-21T10:35:33.165 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T10:35:33.165 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:33.165 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:33.165 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:33.165 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:33.165 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster": { 2024-03-21T10:35:33.165 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster", 2024-03-21T10:35:33.165 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T10:35:33.165 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:33.165 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T10:35:33.165 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-03-21T10:35:33.165 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:33.165 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:33.166 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:33.166 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T10:35:33.166 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:33.166 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:33.166 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:33.166 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:33.166 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster_level": { 2024-03-21T10:35:33.166 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster_level", 2024-03-21T10:35:33.166 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T10:35:33.166 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:33.166 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T10:35:33.166 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "info", 2024-03-21T10:35:33.166 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:33.166 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:33.166 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-03-21T10:35:33.167 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-03-21T10:35:33.167 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-03-21T10:35:33.167 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-03-21T10:35:33.167 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-03-21T10:35:33.167 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-03-21T10:35:33.167 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-03-21T10:35:33.167 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-03-21T10:35:33.167 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T10:35:33.167 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:33.167 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:33.167 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:33.167 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:33.167 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_file": { 2024-03-21T10:35:33.167 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_file", 2024-03-21T10:35:33.167 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T10:35:33.168 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:33.168 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T10:35:33.168 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-03-21T10:35:33.168 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:33.168 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:33.168 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:33.168 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T10:35:33.168 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:33.168 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:33.168 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:33.168 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-21T10:35:33.168 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-21T10:35:33.168 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:33.168 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-03-21T10:35:33.169 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "iostat", 2024-03-21T10:35:33.169 INFO:teuthology.orchestra.run.smithi047.stdout: "can_run": true, 2024-03-21T10:35:33.169 INFO:teuthology.orchestra.run.smithi047.stdout: "error_string": "", 2024-03-21T10:35:33.169 INFO:teuthology.orchestra.run.smithi047.stdout: "module_options": { 2024-03-21T10:35:33.169 INFO:teuthology.orchestra.run.smithi047.stdout: "log_level": { 2024-03-21T10:35:33.169 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_level", 2024-03-21T10:35:33.169 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T10:35:33.169 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:33.169 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T10:35:33.169 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-03-21T10:35:33.169 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:33.169 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:33.169 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-03-21T10:35:33.169 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-03-21T10:35:33.169 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-03-21T10:35:33.169 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-03-21T10:35:33.169 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-03-21T10:35:33.170 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-03-21T10:35:33.170 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-03-21T10:35:33.170 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-03-21T10:35:33.170 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T10:35:33.170 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:33.170 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:33.170 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:33.170 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:33.170 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster": { 2024-03-21T10:35:33.170 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster", 2024-03-21T10:35:33.170 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T10:35:33.170 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:33.170 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T10:35:33.171 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-03-21T10:35:33.171 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:33.171 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:33.171 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:33.171 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T10:35:33.171 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:33.171 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:33.171 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:33.171 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:33.171 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster_level": { 2024-03-21T10:35:33.171 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster_level", 2024-03-21T10:35:33.171 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T10:35:33.171 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:33.171 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T10:35:33.171 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "info", 2024-03-21T10:35:33.171 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:33.172 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:33.172 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-03-21T10:35:33.172 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-03-21T10:35:33.172 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-03-21T10:35:33.172 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-03-21T10:35:33.172 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-03-21T10:35:33.172 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-03-21T10:35:33.172 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-03-21T10:35:33.172 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-03-21T10:35:33.172 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T10:35:33.172 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:33.172 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:33.172 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:33.172 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:33.173 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_file": { 2024-03-21T10:35:33.173 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_file", 2024-03-21T10:35:33.173 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T10:35:33.173 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:33.173 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T10:35:33.173 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-03-21T10:35:33.173 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:33.173 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:33.173 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:33.173 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T10:35:33.173 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:33.173 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:33.173 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:33.173 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-21T10:35:33.173 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-21T10:35:33.174 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:33.174 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-03-21T10:35:33.174 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "k8sevents", 2024-03-21T10:35:33.174 INFO:teuthology.orchestra.run.smithi047.stdout: "can_run": true, 2024-03-21T10:35:33.174 INFO:teuthology.orchestra.run.smithi047.stdout: "error_string": "", 2024-03-21T10:35:33.174 INFO:teuthology.orchestra.run.smithi047.stdout: "module_options": { 2024-03-21T10:35:33.174 INFO:teuthology.orchestra.run.smithi047.stdout: "ceph_event_retention_days": { 2024-03-21T10:35:33.174 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "ceph_event_retention_days", 2024-03-21T10:35:33.174 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "int", 2024-03-21T10:35:33.174 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:33.174 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T10:35:33.174 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "7", 2024-03-21T10:35:33.174 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:33.175 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:33.175 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:33.175 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Days to hold ceph event information within local cache", 2024-03-21T10:35:33.175 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:33.175 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:33.175 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:33.175 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:33.175 INFO:teuthology.orchestra.run.smithi047.stdout: "config_check_secs": { 2024-03-21T10:35:33.175 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "config_check_secs", 2024-03-21T10:35:33.175 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "int", 2024-03-21T10:35:33.175 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:33.175 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T10:35:33.175 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "10", 2024-03-21T10:35:33.175 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "10", 2024-03-21T10:35:33.175 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:33.175 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:33.176 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-03-21T10:35:33.176 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:33.176 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:33.176 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:33.176 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:33.176 INFO:teuthology.orchestra.run.smithi047.stdout: "log_level": { 2024-03-21T10:35:33.176 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_level", 2024-03-21T10:35:33.176 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T10:35:33.176 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:33.176 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T10:35:33.176 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-03-21T10:35:33.176 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:33.176 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:33.176 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-03-21T10:35:33.176 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-03-21T10:35:33.176 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-03-21T10:35:33.176 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-03-21T10:35:33.177 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-03-21T10:35:33.177 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-03-21T10:35:33.177 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-03-21T10:35:33.177 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-03-21T10:35:33.177 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T10:35:33.177 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:33.177 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:33.177 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:33.177 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:33.177 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster": { 2024-03-21T10:35:33.177 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster", 2024-03-21T10:35:33.177 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T10:35:33.177 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:33.177 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T10:35:33.178 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-03-21T10:35:33.178 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:33.178 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:33.178 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:33.178 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T10:35:33.178 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:33.178 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:33.178 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:33.178 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:33.178 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster_level": { 2024-03-21T10:35:33.178 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster_level", 2024-03-21T10:35:33.178 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T10:35:33.178 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:33.178 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T10:35:33.179 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "info", 2024-03-21T10:35:33.179 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:33.179 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:33.179 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-03-21T10:35:33.179 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-03-21T10:35:33.179 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-03-21T10:35:33.179 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-03-21T10:35:33.179 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-03-21T10:35:33.179 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-03-21T10:35:33.179 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-03-21T10:35:33.179 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-03-21T10:35:33.179 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T10:35:33.179 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:33.179 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:33.180 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:33.180 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:33.180 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_file": { 2024-03-21T10:35:33.180 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_file", 2024-03-21T10:35:33.180 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T10:35:33.180 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:33.180 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T10:35:33.180 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-03-21T10:35:33.180 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:33.180 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:33.180 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:33.180 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T10:35:33.180 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:33.180 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:33.180 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:33.180 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-21T10:35:33.181 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-21T10:35:33.181 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:33.181 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-03-21T10:35:33.181 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "localpool", 2024-03-21T10:35:33.181 INFO:teuthology.orchestra.run.smithi047.stdout: "can_run": true, 2024-03-21T10:35:33.181 INFO:teuthology.orchestra.run.smithi047.stdout: "error_string": "", 2024-03-21T10:35:33.181 INFO:teuthology.orchestra.run.smithi047.stdout: "module_options": { 2024-03-21T10:35:33.181 INFO:teuthology.orchestra.run.smithi047.stdout: "failure_domain": { 2024-03-21T10:35:33.181 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "failure_domain", 2024-03-21T10:35:33.181 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T10:35:33.181 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:33.181 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T10:35:33.181 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "host", 2024-03-21T10:35:33.181 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:33.181 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:33.182 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:33.182 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "failure domain for any created local pool", 2024-03-21T10:35:33.182 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-03-21T10:35:33.182 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:33.182 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:33.182 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:33.182 INFO:teuthology.orchestra.run.smithi047.stdout: "log_level": { 2024-03-21T10:35:33.182 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_level", 2024-03-21T10:35:33.182 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T10:35:33.182 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:33.182 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T10:35:33.182 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-03-21T10:35:33.182 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:33.182 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:33.183 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-03-21T10:35:33.183 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-03-21T10:35:33.183 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-03-21T10:35:33.183 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-03-21T10:35:33.183 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-03-21T10:35:33.183 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-03-21T10:35:33.183 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-03-21T10:35:33.183 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-03-21T10:35:33.183 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T10:35:33.183 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:33.183 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:33.183 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:33.183 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:33.183 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster": { 2024-03-21T10:35:33.183 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster", 2024-03-21T10:35:33.184 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T10:35:33.184 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:33.184 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T10:35:33.184 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-03-21T10:35:33.184 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:33.184 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:33.184 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:33.184 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T10:35:33.184 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:33.184 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:33.184 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:33.184 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:33.184 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster_level": { 2024-03-21T10:35:33.184 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster_level", 2024-03-21T10:35:33.185 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T10:35:33.185 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:33.185 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T10:35:33.185 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "info", 2024-03-21T10:35:33.185 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:33.185 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:33.185 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-03-21T10:35:33.185 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-03-21T10:35:33.185 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-03-21T10:35:33.185 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-03-21T10:35:33.185 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-03-21T10:35:33.185 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-03-21T10:35:33.185 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-03-21T10:35:33.185 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-03-21T10:35:33.185 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T10:35:33.186 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:33.186 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:33.186 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:33.186 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:33.186 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_file": { 2024-03-21T10:35:33.186 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_file", 2024-03-21T10:35:33.186 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T10:35:33.186 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:33.186 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T10:35:33.186 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-03-21T10:35:33.186 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:33.186 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:33.186 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:33.186 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T10:35:33.186 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:33.187 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:33.187 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:33.187 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:33.187 INFO:teuthology.orchestra.run.smithi047.stdout: "min_size": { 2024-03-21T10:35:33.187 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "min_size", 2024-03-21T10:35:33.187 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "int", 2024-03-21T10:35:33.187 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:33.187 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T10:35:33.187 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-03-21T10:35:33.187 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:33.187 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:33.187 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:33.187 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "default min_size for any created local pool", 2024-03-21T10:35:33.188 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-03-21T10:35:33.188 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:33.188 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:33.188 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:33.188 INFO:teuthology.orchestra.run.smithi047.stdout: "num_rep": { 2024-03-21T10:35:33.188 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "num_rep", 2024-03-21T10:35:33.188 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "int", 2024-03-21T10:35:33.188 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:33.188 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T10:35:33.188 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "3", 2024-03-21T10:35:33.188 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:33.188 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:33.188 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:33.189 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "default replica count for any created local pool", 2024-03-21T10:35:33.189 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:33.189 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:33.189 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:33.189 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:33.189 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_num": { 2024-03-21T10:35:33.189 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "pg_num", 2024-03-21T10:35:33.189 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "int", 2024-03-21T10:35:33.189 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:33.189 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T10:35:33.189 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "128", 2024-03-21T10:35:33.189 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:33.190 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:33.190 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:33.190 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "default pg_num for any created local pool", 2024-03-21T10:35:33.190 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:33.190 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:33.190 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:33.190 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:33.190 INFO:teuthology.orchestra.run.smithi047.stdout: "prefix": { 2024-03-21T10:35:33.190 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "prefix", 2024-03-21T10:35:33.190 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T10:35:33.190 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:33.190 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T10:35:33.191 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-03-21T10:35:33.191 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:33.191 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:33.191 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:33.191 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "name prefix for any created local pool", 2024-03-21T10:35:33.191 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:33.191 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:33.191 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:33.192 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:33.192 INFO:teuthology.orchestra.run.smithi047.stdout: "subtree": { 2024-03-21T10:35:33.192 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "subtree", 2024-03-21T10:35:33.192 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T10:35:33.192 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:33.192 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T10:35:33.192 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "rack", 2024-03-21T10:35:33.193 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:33.193 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:33.193 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:33.193 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "CRUSH level for which to create a local pool", 2024-03-21T10:35:33.193 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-03-21T10:35:33.193 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:33.193 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:33.193 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-21T10:35:33.193 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-21T10:35:33.193 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:33.194 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-03-21T10:35:33.194 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "mds_autoscaler", 2024-03-21T10:35:33.194 INFO:teuthology.orchestra.run.smithi047.stdout: "can_run": true, 2024-03-21T10:35:33.194 INFO:teuthology.orchestra.run.smithi047.stdout: "error_string": "", 2024-03-21T10:35:33.194 INFO:teuthology.orchestra.run.smithi047.stdout: "module_options": { 2024-03-21T10:35:33.194 INFO:teuthology.orchestra.run.smithi047.stdout: "log_level": { 2024-03-21T10:35:33.194 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_level", 2024-03-21T10:35:33.194 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T10:35:33.194 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:33.194 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T10:35:33.194 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-03-21T10:35:33.194 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:33.195 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:33.195 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-03-21T10:35:33.195 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-03-21T10:35:33.195 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-03-21T10:35:33.195 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-03-21T10:35:33.195 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-03-21T10:35:33.195 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-03-21T10:35:33.195 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-03-21T10:35:33.195 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-03-21T10:35:33.195 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T10:35:33.195 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:33.195 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:33.195 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:33.195 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:33.196 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster": { 2024-03-21T10:35:33.196 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster", 2024-03-21T10:35:33.196 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T10:35:33.196 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:33.196 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T10:35:33.196 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-03-21T10:35:33.196 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:33.196 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:33.196 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:33.196 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T10:35:33.196 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:33.196 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:33.196 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:33.196 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:33.197 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster_level": { 2024-03-21T10:35:33.197 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster_level", 2024-03-21T10:35:33.197 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T10:35:33.197 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:33.197 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T10:35:33.197 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "info", 2024-03-21T10:35:33.197 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:33.197 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:33.197 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-03-21T10:35:33.197 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-03-21T10:35:33.197 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-03-21T10:35:33.197 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-03-21T10:35:33.197 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-03-21T10:35:33.197 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-03-21T10:35:33.197 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-03-21T10:35:33.198 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-03-21T10:35:33.198 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T10:35:33.198 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:33.198 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:33.198 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:33.198 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:33.198 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_file": { 2024-03-21T10:35:33.198 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_file", 2024-03-21T10:35:33.198 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T10:35:33.198 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:33.198 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T10:35:33.198 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-03-21T10:35:33.198 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:33.199 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:33.199 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:33.199 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T10:35:33.199 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:33.199 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:33.199 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:33.199 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-21T10:35:33.199 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-21T10:35:33.199 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:33.199 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-03-21T10:35:33.199 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "mirroring", 2024-03-21T10:35:33.199 INFO:teuthology.orchestra.run.smithi047.stdout: "can_run": true, 2024-03-21T10:35:33.199 INFO:teuthology.orchestra.run.smithi047.stdout: "error_string": "", 2024-03-21T10:35:33.200 INFO:teuthology.orchestra.run.smithi047.stdout: "module_options": { 2024-03-21T10:35:33.200 INFO:teuthology.orchestra.run.smithi047.stdout: "log_level": { 2024-03-21T10:35:33.200 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_level", 2024-03-21T10:35:33.200 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T10:35:33.200 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:33.200 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T10:35:33.200 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-03-21T10:35:33.200 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:33.200 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:33.200 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-03-21T10:35:33.200 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-03-21T10:35:33.200 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-03-21T10:35:33.200 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-03-21T10:35:33.200 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-03-21T10:35:33.201 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-03-21T10:35:33.201 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-03-21T10:35:33.201 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-03-21T10:35:33.201 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T10:35:33.201 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:33.201 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:33.201 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:33.201 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:33.201 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster": { 2024-03-21T10:35:33.201 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster", 2024-03-21T10:35:33.201 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T10:35:33.201 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:33.201 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T10:35:33.202 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-03-21T10:35:33.202 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:33.202 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:33.202 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:33.202 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T10:35:33.202 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:33.202 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:33.202 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:33.202 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:33.202 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster_level": { 2024-03-21T10:35:33.202 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster_level", 2024-03-21T10:35:33.202 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T10:35:33.202 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:33.202 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T10:35:33.203 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "info", 2024-03-21T10:35:33.203 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:33.203 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:33.203 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-03-21T10:35:33.203 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-03-21T10:35:33.203 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-03-21T10:35:33.203 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-03-21T10:35:33.203 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-03-21T10:35:33.203 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-03-21T10:35:33.203 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-03-21T10:35:33.203 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-03-21T10:35:33.203 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T10:35:33.203 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:33.203 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:33.204 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:33.204 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:33.204 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_file": { 2024-03-21T10:35:33.204 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_file", 2024-03-21T10:35:33.204 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T10:35:33.204 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:33.204 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T10:35:33.204 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-03-21T10:35:33.204 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:33.204 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:33.204 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:33.204 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T10:35:33.204 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:33.204 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:33.205 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:33.205 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-21T10:35:33.205 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-21T10:35:33.205 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:33.205 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-03-21T10:35:33.205 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "nfs", 2024-03-21T10:35:33.205 INFO:teuthology.orchestra.run.smithi047.stdout: "can_run": true, 2024-03-21T10:35:33.205 INFO:teuthology.orchestra.run.smithi047.stdout: "error_string": "", 2024-03-21T10:35:33.205 INFO:teuthology.orchestra.run.smithi047.stdout: "module_options": { 2024-03-21T10:35:33.205 INFO:teuthology.orchestra.run.smithi047.stdout: "log_level": { 2024-03-21T10:35:33.205 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_level", 2024-03-21T10:35:33.205 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T10:35:33.205 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:33.205 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T10:35:33.205 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-03-21T10:35:33.206 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:33.206 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:33.206 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-03-21T10:35:33.206 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-03-21T10:35:33.206 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-03-21T10:35:33.206 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-03-21T10:35:33.206 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-03-21T10:35:33.206 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-03-21T10:35:33.206 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-03-21T10:35:33.206 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-03-21T10:35:33.206 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T10:35:33.206 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:33.207 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:33.207 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:33.207 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:33.207 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster": { 2024-03-21T10:35:33.207 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster", 2024-03-21T10:35:33.207 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T10:35:33.207 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:33.207 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T10:35:33.207 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-03-21T10:35:33.207 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:33.207 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:33.207 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:33.207 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T10:35:33.207 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:33.208 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:33.208 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:33.208 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:33.208 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster_level": { 2024-03-21T10:35:33.208 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster_level", 2024-03-21T10:35:33.208 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T10:35:33.208 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:33.208 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T10:35:33.208 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "info", 2024-03-21T10:35:33.208 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:33.208 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:33.208 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-03-21T10:35:33.208 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-03-21T10:35:33.209 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-03-21T10:35:33.209 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-03-21T10:35:33.209 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-03-21T10:35:33.209 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-03-21T10:35:33.209 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-03-21T10:35:33.209 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-03-21T10:35:33.209 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T10:35:33.209 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:33.209 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:33.209 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:33.209 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:33.209 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_file": { 2024-03-21T10:35:33.209 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_file", 2024-03-21T10:35:33.209 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T10:35:33.209 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:33.210 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T10:35:33.210 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-03-21T10:35:33.210 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:33.210 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:33.210 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:33.210 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T10:35:33.210 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:33.210 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:33.210 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:33.210 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-21T10:35:33.210 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-21T10:35:33.210 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:33.210 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-03-21T10:35:33.210 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "orchestrator", 2024-03-21T10:35:33.211 INFO:teuthology.orchestra.run.smithi047.stdout: "can_run": true, 2024-03-21T10:35:33.211 INFO:teuthology.orchestra.run.smithi047.stdout: "error_string": "", 2024-03-21T10:35:33.211 INFO:teuthology.orchestra.run.smithi047.stdout: "module_options": { 2024-03-21T10:35:33.211 INFO:teuthology.orchestra.run.smithi047.stdout: "fail_fs": { 2024-03-21T10:35:33.211 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "fail_fs", 2024-03-21T10:35:33.211 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T10:35:33.211 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:33.211 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T10:35:33.211 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-03-21T10:35:33.211 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:33.211 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:33.211 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:33.211 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-03-21T10:35:33.211 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:33.212 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:33.212 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:33.212 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:33.212 INFO:teuthology.orchestra.run.smithi047.stdout: "log_level": { 2024-03-21T10:35:33.212 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_level", 2024-03-21T10:35:33.212 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T10:35:33.212 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:33.212 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T10:35:33.212 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-03-21T10:35:33.212 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:33.212 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:33.212 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-03-21T10:35:33.212 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-03-21T10:35:33.212 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-03-21T10:35:33.213 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-03-21T10:35:33.213 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-03-21T10:35:33.213 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-03-21T10:35:33.213 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-03-21T10:35:33.213 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-03-21T10:35:33.213 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T10:35:33.213 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:33.213 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:33.213 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:33.213 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:33.213 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster": { 2024-03-21T10:35:33.213 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster", 2024-03-21T10:35:33.213 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T10:35:33.214 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:33.214 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T10:35:33.214 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-03-21T10:35:33.214 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:33.214 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:33.214 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:33.214 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T10:35:33.214 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:33.214 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:33.214 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:33.214 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:33.214 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster_level": { 2024-03-21T10:35:33.215 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster_level", 2024-03-21T10:35:33.215 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T10:35:33.215 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:33.215 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T10:35:33.215 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "info", 2024-03-21T10:35:33.215 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:33.215 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:33.215 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-03-21T10:35:33.215 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-03-21T10:35:33.215 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-03-21T10:35:33.215 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-03-21T10:35:33.215 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-03-21T10:35:33.215 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-03-21T10:35:33.215 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-03-21T10:35:33.216 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-03-21T10:35:33.216 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T10:35:33.216 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:33.216 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:33.216 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:33.216 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:33.216 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_file": { 2024-03-21T10:35:33.216 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_file", 2024-03-21T10:35:33.216 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T10:35:33.216 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:33.216 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T10:35:33.216 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-03-21T10:35:33.216 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:33.216 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:33.217 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:33.217 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T10:35:33.217 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:33.217 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:33.217 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:33.217 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:33.217 INFO:teuthology.orchestra.run.smithi047.stdout: "orchestrator": { 2024-03-21T10:35:33.217 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "orchestrator", 2024-03-21T10:35:33.217 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T10:35:33.217 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:33.217 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T10:35:33.217 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-03-21T10:35:33.217 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:33.217 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:33.218 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-03-21T10:35:33.218 INFO:teuthology.orchestra.run.smithi047.stdout: "cephadm", 2024-03-21T10:35:33.218 INFO:teuthology.orchestra.run.smithi047.stdout: "rook", 2024-03-21T10:35:33.218 INFO:teuthology.orchestra.run.smithi047.stdout: "test_orchestrator" 2024-03-21T10:35:33.218 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-03-21T10:35:33.218 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Orchestrator backend", 2024-03-21T10:35:33.218 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:33.218 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:33.218 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:33.218 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-21T10:35:33.218 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-21T10:35:33.218 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:33.218 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-03-21T10:35:33.219 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "osd_perf_query", 2024-03-21T10:35:33.219 INFO:teuthology.orchestra.run.smithi047.stdout: "can_run": true, 2024-03-21T10:35:33.219 INFO:teuthology.orchestra.run.smithi047.stdout: "error_string": "", 2024-03-21T10:35:33.219 INFO:teuthology.orchestra.run.smithi047.stdout: "module_options": { 2024-03-21T10:35:33.219 INFO:teuthology.orchestra.run.smithi047.stdout: "log_level": { 2024-03-21T10:35:33.219 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_level", 2024-03-21T10:35:33.219 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T10:35:33.219 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:33.219 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T10:35:33.219 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-03-21T10:35:33.219 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:33.219 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:33.219 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-03-21T10:35:33.219 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-03-21T10:35:33.220 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-03-21T10:35:33.220 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-03-21T10:35:33.220 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-03-21T10:35:33.220 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-03-21T10:35:33.220 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-03-21T10:35:33.220 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-03-21T10:35:33.220 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T10:35:33.220 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:33.220 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:33.220 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:33.220 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:33.220 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster": { 2024-03-21T10:35:33.220 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster", 2024-03-21T10:35:33.220 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T10:35:33.221 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:33.221 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T10:35:33.221 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-03-21T10:35:33.221 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:33.221 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:33.221 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:33.221 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T10:35:33.221 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:33.221 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:33.221 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:33.221 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:33.221 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster_level": { 2024-03-21T10:35:33.221 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster_level", 2024-03-21T10:35:33.221 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T10:35:33.222 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:33.222 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T10:35:33.222 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "info", 2024-03-21T10:35:33.222 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:33.222 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:33.222 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-03-21T10:35:33.222 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-03-21T10:35:33.222 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-03-21T10:35:33.222 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-03-21T10:35:33.222 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-03-21T10:35:33.222 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-03-21T10:35:33.222 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-03-21T10:35:33.222 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-03-21T10:35:33.223 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T10:35:33.223 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:33.223 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:33.223 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:33.223 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:33.223 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_file": { 2024-03-21T10:35:33.223 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_file", 2024-03-21T10:35:33.223 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T10:35:33.223 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:33.223 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T10:35:33.223 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-03-21T10:35:33.223 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:33.223 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:33.223 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:33.224 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T10:35:33.224 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:33.224 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:33.224 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:33.224 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-21T10:35:33.224 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-21T10:35:33.224 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:33.224 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-03-21T10:35:33.224 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "osd_support", 2024-03-21T10:35:33.224 INFO:teuthology.orchestra.run.smithi047.stdout: "can_run": true, 2024-03-21T10:35:33.224 INFO:teuthology.orchestra.run.smithi047.stdout: "error_string": "", 2024-03-21T10:35:33.224 INFO:teuthology.orchestra.run.smithi047.stdout: "module_options": { 2024-03-21T10:35:33.224 INFO:teuthology.orchestra.run.smithi047.stdout: "log_level": { 2024-03-21T10:35:33.225 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_level", 2024-03-21T10:35:33.225 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T10:35:33.225 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:33.225 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T10:35:33.225 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-03-21T10:35:33.225 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:33.225 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:33.225 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-03-21T10:35:33.225 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-03-21T10:35:33.225 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-03-21T10:35:33.225 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-03-21T10:35:33.225 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-03-21T10:35:33.225 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-03-21T10:35:33.226 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-03-21T10:35:33.226 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-03-21T10:35:33.226 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T10:35:33.226 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:33.226 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:33.226 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:33.226 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:33.226 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster": { 2024-03-21T10:35:33.226 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster", 2024-03-21T10:35:33.226 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T10:35:33.226 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:33.226 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T10:35:33.226 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-03-21T10:35:33.227 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:33.227 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:33.227 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:33.227 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T10:35:33.227 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:33.227 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:33.227 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:33.227 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:33.227 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster_level": { 2024-03-21T10:35:33.227 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster_level", 2024-03-21T10:35:33.227 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T10:35:33.227 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:33.227 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T10:35:33.228 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "info", 2024-03-21T10:35:33.228 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:33.228 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:33.228 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-03-21T10:35:33.228 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-03-21T10:35:33.228 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-03-21T10:35:33.228 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-03-21T10:35:33.228 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-03-21T10:35:33.228 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-03-21T10:35:33.228 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-03-21T10:35:33.228 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-03-21T10:35:33.228 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T10:35:33.228 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:33.228 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:33.229 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:33.229 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:33.229 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_file": { 2024-03-21T10:35:33.229 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_file", 2024-03-21T10:35:33.229 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T10:35:33.229 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:33.229 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T10:35:33.229 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-03-21T10:35:33.229 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:33.229 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:33.229 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:33.229 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T10:35:33.229 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:33.229 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:33.229 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:33.230 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-21T10:35:33.230 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-21T10:35:33.230 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:33.230 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-03-21T10:35:33.230 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "pg_autoscaler", 2024-03-21T10:35:33.230 INFO:teuthology.orchestra.run.smithi047.stdout: "can_run": true, 2024-03-21T10:35:33.230 INFO:teuthology.orchestra.run.smithi047.stdout: "error_string": "", 2024-03-21T10:35:33.230 INFO:teuthology.orchestra.run.smithi047.stdout: "module_options": { 2024-03-21T10:35:33.230 INFO:teuthology.orchestra.run.smithi047.stdout: "log_level": { 2024-03-21T10:35:33.230 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_level", 2024-03-21T10:35:33.230 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T10:35:33.230 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:33.230 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T10:35:33.230 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-03-21T10:35:33.231 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:33.231 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:33.231 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-03-21T10:35:33.231 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-03-21T10:35:33.231 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-03-21T10:35:33.231 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-03-21T10:35:33.231 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-03-21T10:35:33.231 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-03-21T10:35:33.231 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-03-21T10:35:33.231 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-03-21T10:35:33.231 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T10:35:33.231 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:33.231 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:33.231 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:33.231 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:33.231 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster": { 2024-03-21T10:35:33.232 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster", 2024-03-21T10:35:33.232 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T10:35:33.232 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:33.232 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T10:35:33.232 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-03-21T10:35:33.232 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:33.232 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:33.232 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:33.232 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T10:35:33.232 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:33.232 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:33.232 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:33.232 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:33.233 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster_level": { 2024-03-21T10:35:33.233 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster_level", 2024-03-21T10:35:33.233 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T10:35:33.233 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:33.233 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T10:35:33.233 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "info", 2024-03-21T10:35:33.233 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:33.233 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:33.233 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-03-21T10:35:33.233 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-03-21T10:35:33.233 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-03-21T10:35:33.233 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-03-21T10:35:33.233 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-03-21T10:35:33.233 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-03-21T10:35:33.233 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-03-21T10:35:33.234 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-03-21T10:35:33.234 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T10:35:33.234 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:33.234 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:33.234 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:33.234 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:33.234 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_file": { 2024-03-21T10:35:33.234 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_file", 2024-03-21T10:35:33.234 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T10:35:33.234 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:33.234 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T10:35:33.234 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-03-21T10:35:33.234 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:33.235 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:33.235 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:33.235 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T10:35:33.235 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:33.235 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:33.235 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:33.235 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:33.235 INFO:teuthology.orchestra.run.smithi047.stdout: "sleep_interval": { 2024-03-21T10:35:33.235 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "sleep_interval", 2024-03-21T10:35:33.235 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "secs", 2024-03-21T10:35:33.235 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:33.235 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T10:35:33.235 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "60", 2024-03-21T10:35:33.235 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:33.236 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:33.236 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:33.236 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T10:35:33.236 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:33.236 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:33.236 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:33.236 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:33.236 INFO:teuthology.orchestra.run.smithi047.stdout: "threshold": { 2024-03-21T10:35:33.236 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "threshold", 2024-03-21T10:35:33.236 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "float", 2024-03-21T10:35:33.236 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:33.236 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T10:35:33.236 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "3.0", 2024-03-21T10:35:33.236 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "1.0", 2024-03-21T10:35:33.236 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:33.237 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:33.237 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "scaling threshold", 2024-03-21T10:35:33.237 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-03-21T10:35:33.237 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:33.237 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:33.237 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-21T10:35:33.237 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-21T10:35:33.237 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:33.237 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-03-21T10:35:33.237 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "progress", 2024-03-21T10:35:33.237 INFO:teuthology.orchestra.run.smithi047.stdout: "can_run": true, 2024-03-21T10:35:33.237 INFO:teuthology.orchestra.run.smithi047.stdout: "error_string": "", 2024-03-21T10:35:33.237 INFO:teuthology.orchestra.run.smithi047.stdout: "module_options": { 2024-03-21T10:35:33.238 INFO:teuthology.orchestra.run.smithi047.stdout: "allow_pg_recovery_event": { 2024-03-21T10:35:33.238 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "allow_pg_recovery_event", 2024-03-21T10:35:33.238 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T10:35:33.238 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:33.238 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T10:35:33.238 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-03-21T10:35:33.238 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:33.238 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:33.238 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:33.238 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "allow the module to show pg recovery progress", 2024-03-21T10:35:33.238 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:33.238 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:33.239 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:33.239 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:33.239 INFO:teuthology.orchestra.run.smithi047.stdout: "enabled": { 2024-03-21T10:35:33.239 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "enabled", 2024-03-21T10:35:33.239 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T10:35:33.239 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:33.239 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T10:35:33.239 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "True", 2024-03-21T10:35:33.239 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:33.239 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:33.239 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:33.239 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T10:35:33.239 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:33.239 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:33.240 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:33.240 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:33.240 INFO:teuthology.orchestra.run.smithi047.stdout: "log_level": { 2024-03-21T10:35:33.240 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_level", 2024-03-21T10:35:33.240 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T10:35:33.240 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:33.240 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T10:35:33.240 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-03-21T10:35:33.240 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:33.240 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:33.240 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-03-21T10:35:33.240 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-03-21T10:35:33.240 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-03-21T10:35:33.241 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-03-21T10:35:33.241 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-03-21T10:35:33.241 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-03-21T10:35:33.241 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-03-21T10:35:33.241 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-03-21T10:35:33.241 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T10:35:33.241 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:33.241 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:33.241 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:33.241 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:33.241 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster": { 2024-03-21T10:35:33.241 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster", 2024-03-21T10:35:33.241 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T10:35:33.241 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:33.241 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T10:35:33.242 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-03-21T10:35:33.242 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:33.242 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:33.242 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:33.242 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T10:35:33.242 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:33.242 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:33.242 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:33.242 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:33.242 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster_level": { 2024-03-21T10:35:33.242 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster_level", 2024-03-21T10:35:33.242 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T10:35:33.242 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:33.242 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T10:35:33.242 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "info", 2024-03-21T10:35:33.243 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:33.243 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:33.243 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-03-21T10:35:33.243 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-03-21T10:35:33.243 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-03-21T10:35:33.243 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-03-21T10:35:33.243 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-03-21T10:35:33.243 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-03-21T10:35:33.243 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-03-21T10:35:33.243 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-03-21T10:35:33.243 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T10:35:33.243 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:33.243 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:33.243 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:33.243 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:33.244 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_file": { 2024-03-21T10:35:33.244 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_file", 2024-03-21T10:35:33.244 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T10:35:33.244 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:33.244 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T10:35:33.244 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-03-21T10:35:33.244 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:33.244 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:33.244 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:33.244 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T10:35:33.244 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:33.244 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:33.244 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:33.244 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:33.244 INFO:teuthology.orchestra.run.smithi047.stdout: "max_completed_events": { 2024-03-21T10:35:33.245 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "max_completed_events", 2024-03-21T10:35:33.245 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "int", 2024-03-21T10:35:33.245 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:33.245 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T10:35:33.245 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "50", 2024-03-21T10:35:33.245 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:33.245 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:33.245 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:33.245 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "number of past completed events to remember", 2024-03-21T10:35:33.245 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:33.245 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:33.245 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:33.245 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:33.245 INFO:teuthology.orchestra.run.smithi047.stdout: "sleep_interval": { 2024-03-21T10:35:33.246 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "sleep_interval", 2024-03-21T10:35:33.246 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "secs", 2024-03-21T10:35:33.246 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:33.246 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T10:35:33.246 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "5", 2024-03-21T10:35:33.246 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:33.246 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:33.246 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:33.246 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "how long the module is going to sleep", 2024-03-21T10:35:33.246 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:33.246 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:33.246 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:33.246 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-21T10:35:33.246 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-21T10:35:33.247 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:33.247 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-03-21T10:35:33.247 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "prometheus", 2024-03-21T10:35:33.247 INFO:teuthology.orchestra.run.smithi047.stdout: "can_run": true, 2024-03-21T10:35:33.247 INFO:teuthology.orchestra.run.smithi047.stdout: "error_string": "", 2024-03-21T10:35:33.247 INFO:teuthology.orchestra.run.smithi047.stdout: "module_options": { 2024-03-21T10:35:33.247 INFO:teuthology.orchestra.run.smithi047.stdout: "cache": { 2024-03-21T10:35:33.247 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "cache", 2024-03-21T10:35:33.247 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T10:35:33.247 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:33.247 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T10:35:33.247 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "True", 2024-03-21T10:35:33.247 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:33.247 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:33.247 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:33.248 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T10:35:33.248 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:33.248 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:33.248 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:33.248 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:33.248 INFO:teuthology.orchestra.run.smithi047.stdout: "exclude_perf_counters": { 2024-03-21T10:35:33.248 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "exclude_perf_counters", 2024-03-21T10:35:33.248 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T10:35:33.248 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:33.248 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T10:35:33.248 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "True", 2024-03-21T10:35:33.248 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:33.248 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:33.248 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:33.249 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-03-21T10:35:33.249 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-03-21T10:35:33.249 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:33.249 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:33.249 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:33.249 INFO:teuthology.orchestra.run.smithi047.stdout: "log_level": { 2024-03-21T10:35:33.249 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_level", 2024-03-21T10:35:33.249 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T10:35:33.249 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:33.249 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T10:35:33.249 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-03-21T10:35:33.249 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:33.249 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:33.250 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-03-21T10:35:33.250 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-03-21T10:35:33.250 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-03-21T10:35:33.250 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-03-21T10:35:33.250 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-03-21T10:35:33.250 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-03-21T10:35:33.250 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-03-21T10:35:33.250 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-03-21T10:35:33.250 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T10:35:33.250 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:33.250 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:33.250 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:33.250 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:33.251 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster": { 2024-03-21T10:35:33.251 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster", 2024-03-21T10:35:33.251 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T10:35:33.251 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:33.251 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T10:35:33.251 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-03-21T10:35:33.251 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:33.251 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:33.251 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:33.251 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T10:35:33.251 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:33.251 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:33.251 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:33.251 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:33.251 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster_level": { 2024-03-21T10:35:33.252 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster_level", 2024-03-21T10:35:33.252 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T10:35:33.252 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:33.252 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T10:35:33.252 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "info", 2024-03-21T10:35:33.252 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:33.252 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:33.252 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-03-21T10:35:33.252 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-03-21T10:35:33.252 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-03-21T10:35:33.252 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-03-21T10:35:33.252 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-03-21T10:35:33.252 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-03-21T10:35:33.252 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-03-21T10:35:33.252 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-03-21T10:35:33.253 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T10:35:33.253 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:33.253 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:33.253 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:33.253 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:33.253 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_file": { 2024-03-21T10:35:33.253 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_file", 2024-03-21T10:35:33.253 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T10:35:33.253 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:33.253 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T10:35:33.253 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-03-21T10:35:33.253 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:33.253 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:33.253 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:33.254 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T10:35:33.254 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:33.254 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:33.254 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:33.254 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:33.254 INFO:teuthology.orchestra.run.smithi047.stdout: "rbd_stats_pools": { 2024-03-21T10:35:33.254 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "rbd_stats_pools", 2024-03-21T10:35:33.254 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T10:35:33.254 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:33.254 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T10:35:33.254 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-03-21T10:35:33.254 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:33.255 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:33.255 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:33.255 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T10:35:33.255 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:33.255 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:33.255 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:33.255 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:33.255 INFO:teuthology.orchestra.run.smithi047.stdout: "rbd_stats_pools_refresh_interval": { 2024-03-21T10:35:33.255 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-03-21T10:35:33.255 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "int", 2024-03-21T10:35:33.255 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:33.255 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T10:35:33.255 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "300", 2024-03-21T10:35:33.255 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:33.255 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:33.256 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:33.256 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T10:35:33.256 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:33.256 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:33.256 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:33.256 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:33.256 INFO:teuthology.orchestra.run.smithi047.stdout: "scrape_interval": { 2024-03-21T10:35:33.256 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "scrape_interval", 2024-03-21T10:35:33.256 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "float", 2024-03-21T10:35:33.256 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:33.256 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T10:35:33.256 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "15.0", 2024-03-21T10:35:33.256 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:33.256 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:33.257 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:33.257 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T10:35:33.257 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:33.257 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:33.257 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:33.257 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:33.257 INFO:teuthology.orchestra.run.smithi047.stdout: "server_addr": { 2024-03-21T10:35:33.257 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "server_addr", 2024-03-21T10:35:33.257 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T10:35:33.257 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:33.257 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T10:35:33.257 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "::", 2024-03-21T10:35:33.257 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:33.257 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:33.257 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:33.258 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-03-21T10:35:33.258 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:33.258 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:33.258 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:33.258 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:33.258 INFO:teuthology.orchestra.run.smithi047.stdout: "server_port": { 2024-03-21T10:35:33.258 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "server_port", 2024-03-21T10:35:33.258 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "int", 2024-03-21T10:35:33.258 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:33.258 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T10:35:33.258 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "9283", 2024-03-21T10:35:33.258 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:33.258 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:33.258 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:33.259 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-03-21T10:35:33.259 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:33.259 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:33.259 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:33.259 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:33.259 INFO:teuthology.orchestra.run.smithi047.stdout: "stale_cache_strategy": { 2024-03-21T10:35:33.259 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "stale_cache_strategy", 2024-03-21T10:35:33.259 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T10:35:33.259 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:33.259 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T10:35:33.259 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "log", 2024-03-21T10:35:33.259 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:33.259 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:33.259 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:33.260 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T10:35:33.260 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:33.260 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:33.260 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:33.260 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:33.260 INFO:teuthology.orchestra.run.smithi047.stdout: "standby_behaviour": { 2024-03-21T10:35:33.260 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "standby_behaviour", 2024-03-21T10:35:33.260 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T10:35:33.260 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:33.260 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T10:35:33.260 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "default", 2024-03-21T10:35:33.260 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:33.260 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:33.260 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-03-21T10:35:33.261 INFO:teuthology.orchestra.run.smithi047.stdout: "default", 2024-03-21T10:35:33.261 INFO:teuthology.orchestra.run.smithi047.stdout: "error" 2024-03-21T10:35:33.261 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-03-21T10:35:33.261 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T10:35:33.261 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:33.261 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:33.261 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:33.261 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:33.261 INFO:teuthology.orchestra.run.smithi047.stdout: "standby_error_status_code": { 2024-03-21T10:35:33.261 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "standby_error_status_code", 2024-03-21T10:35:33.261 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "int", 2024-03-21T10:35:33.261 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:33.261 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T10:35:33.262 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "500", 2024-03-21T10:35:33.262 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "400", 2024-03-21T10:35:33.262 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "599", 2024-03-21T10:35:33.262 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:33.262 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T10:35:33.262 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:33.262 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:33.262 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:33.262 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-21T10:35:33.262 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-21T10:35:33.262 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:33.262 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-03-21T10:35:33.262 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "rbd_support", 2024-03-21T10:35:33.263 INFO:teuthology.orchestra.run.smithi047.stdout: "can_run": true, 2024-03-21T10:35:33.263 INFO:teuthology.orchestra.run.smithi047.stdout: "error_string": "", 2024-03-21T10:35:33.263 INFO:teuthology.orchestra.run.smithi047.stdout: "module_options": { 2024-03-21T10:35:33.263 INFO:teuthology.orchestra.run.smithi047.stdout: "log_level": { 2024-03-21T10:35:33.263 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_level", 2024-03-21T10:35:33.263 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T10:35:33.263 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:33.263 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T10:35:33.263 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-03-21T10:35:33.263 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:33.263 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:33.263 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-03-21T10:35:33.263 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-03-21T10:35:33.263 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-03-21T10:35:33.264 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-03-21T10:35:33.264 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-03-21T10:35:33.264 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-03-21T10:35:33.264 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-03-21T10:35:33.264 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-03-21T10:35:33.264 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T10:35:33.264 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:33.264 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:33.264 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:33.264 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:33.264 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster": { 2024-03-21T10:35:33.264 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster", 2024-03-21T10:35:33.264 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T10:35:33.264 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:33.264 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T10:35:33.264 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-03-21T10:35:33.265 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:33.265 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:33.265 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:33.265 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T10:35:33.265 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:33.265 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:33.265 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:33.265 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:33.265 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster_level": { 2024-03-21T10:35:33.265 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster_level", 2024-03-21T10:35:33.265 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T10:35:33.265 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:33.265 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T10:35:33.265 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "info", 2024-03-21T10:35:33.265 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:33.265 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:33.266 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-03-21T10:35:33.266 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-03-21T10:35:33.266 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-03-21T10:35:33.266 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-03-21T10:35:33.266 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-03-21T10:35:33.266 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-03-21T10:35:33.266 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-03-21T10:35:33.266 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-03-21T10:35:33.266 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T10:35:33.266 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:33.266 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:33.266 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:33.266 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:33.266 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_file": { 2024-03-21T10:35:33.267 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_file", 2024-03-21T10:35:33.267 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T10:35:33.267 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:33.267 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T10:35:33.267 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-03-21T10:35:33.267 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:33.267 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:33.267 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:33.267 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T10:35:33.267 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:33.267 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:33.267 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:33.267 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:33.267 INFO:teuthology.orchestra.run.smithi047.stdout: "max_concurrent_snap_create": { 2024-03-21T10:35:33.267 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "max_concurrent_snap_create", 2024-03-21T10:35:33.268 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "int", 2024-03-21T10:35:33.268 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:33.268 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T10:35:33.268 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "10", 2024-03-21T10:35:33.268 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:33.268 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:33.268 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:33.268 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T10:35:33.268 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:33.268 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:33.268 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:33.268 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:33.268 INFO:teuthology.orchestra.run.smithi047.stdout: "mirror_snapshot_schedule": { 2024-03-21T10:35:33.268 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "mirror_snapshot_schedule", 2024-03-21T10:35:33.268 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T10:35:33.268 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:33.269 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T10:35:33.269 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-03-21T10:35:33.269 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:33.269 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:33.269 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:33.269 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T10:35:33.269 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:33.269 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:33.269 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:33.269 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:33.269 INFO:teuthology.orchestra.run.smithi047.stdout: "trash_purge_schedule": { 2024-03-21T10:35:33.269 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "trash_purge_schedule", 2024-03-21T10:35:33.269 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T10:35:33.269 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:33.270 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T10:35:33.270 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-03-21T10:35:33.270 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:33.270 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:33.270 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:33.270 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T10:35:33.270 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:33.270 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:33.270 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:33.270 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-21T10:35:33.270 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-21T10:35:33.270 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:33.270 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-03-21T10:35:33.270 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "restful", 2024-03-21T10:35:33.271 INFO:teuthology.orchestra.run.smithi047.stdout: "can_run": true, 2024-03-21T10:35:33.271 INFO:teuthology.orchestra.run.smithi047.stdout: "error_string": "", 2024-03-21T10:35:33.271 INFO:teuthology.orchestra.run.smithi047.stdout: "module_options": { 2024-03-21T10:35:33.271 INFO:teuthology.orchestra.run.smithi047.stdout: "enable_auth": { 2024-03-21T10:35:33.271 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "enable_auth", 2024-03-21T10:35:33.271 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T10:35:33.271 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:33.271 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T10:35:33.271 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "True", 2024-03-21T10:35:33.271 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:33.271 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:33.271 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:33.271 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T10:35:33.271 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:33.271 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:33.271 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:33.272 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:35:32 smithi157 ceph-mon[35581]: from='client.? 172.21.15.47:0/3326126196' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-21T10:35:33.272 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:35:32 smithi157 ceph-mon[35581]: from='client.? 172.21.15.47:0/301756020' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-03-21T10:35:33.273 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:35:32 smithi100 ceph-mon[35193]: from='client.? 172.21.15.47:0/3326126196' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-21T10:35:33.273 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:35:32 smithi100 ceph-mon[35193]: from='client.? 172.21.15.47:0/301756020' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-03-21T10:35:33.276 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:33.276 INFO:teuthology.orchestra.run.smithi047.stdout: "key_file": { 2024-03-21T10:35:33.276 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "key_file", 2024-03-21T10:35:33.276 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T10:35:33.276 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:33.276 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T10:35:33.276 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-03-21T10:35:33.276 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:33.276 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:33.276 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:33.277 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T10:35:33.277 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:33.277 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:33.277 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:33.277 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:33.277 INFO:teuthology.orchestra.run.smithi047.stdout: "log_level": { 2024-03-21T10:35:33.277 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_level", 2024-03-21T10:35:33.277 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T10:35:33.277 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:33.277 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T10:35:33.277 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-03-21T10:35:33.277 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:33.277 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:33.277 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-03-21T10:35:33.277 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-03-21T10:35:33.277 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-03-21T10:35:33.278 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-03-21T10:35:33.278 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-03-21T10:35:33.278 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-03-21T10:35:33.278 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-03-21T10:35:33.278 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-03-21T10:35:33.278 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T10:35:33.278 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:33.278 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:33.278 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:33.278 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:33.278 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster": { 2024-03-21T10:35:33.278 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster", 2024-03-21T10:35:33.278 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T10:35:33.278 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:33.279 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T10:35:33.279 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-03-21T10:35:33.279 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:33.279 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:33.279 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:33.279 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T10:35:33.279 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:33.279 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:33.279 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:33.279 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:33.279 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster_level": { 2024-03-21T10:35:33.279 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster_level", 2024-03-21T10:35:33.279 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T10:35:33.279 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:33.279 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T10:35:33.279 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "info", 2024-03-21T10:35:33.280 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:33.280 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:33.280 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-03-21T10:35:33.280 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-03-21T10:35:33.280 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-03-21T10:35:33.280 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-03-21T10:35:33.280 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-03-21T10:35:33.280 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-03-21T10:35:33.280 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-03-21T10:35:33.280 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-03-21T10:35:33.280 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T10:35:33.280 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:33.280 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:33.280 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:33.280 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:33.280 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_file": { 2024-03-21T10:35:33.281 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_file", 2024-03-21T10:35:33.281 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T10:35:33.281 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:33.281 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T10:35:33.281 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-03-21T10:35:33.281 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:33.281 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:33.281 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:33.281 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T10:35:33.281 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:33.281 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:33.281 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:33.281 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:33.281 INFO:teuthology.orchestra.run.smithi047.stdout: "server_addr": { 2024-03-21T10:35:33.281 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "server_addr", 2024-03-21T10:35:33.281 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T10:35:33.282 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:33.282 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T10:35:33.282 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-03-21T10:35:33.282 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:33.282 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:33.282 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:33.282 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T10:35:33.282 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:33.282 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:33.282 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:33.282 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:33.282 INFO:teuthology.orchestra.run.smithi047.stdout: "server_port": { 2024-03-21T10:35:33.282 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "server_port", 2024-03-21T10:35:33.282 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T10:35:33.282 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:33.282 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T10:35:33.283 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-03-21T10:35:33.283 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:33.283 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:33.283 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:33.283 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T10:35:33.283 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:33.283 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:33.283 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:33.283 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-21T10:35:33.283 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-21T10:35:33.283 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:33.283 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-03-21T10:35:33.283 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "rgw", 2024-03-21T10:35:33.283 INFO:teuthology.orchestra.run.smithi047.stdout: "can_run": true, 2024-03-21T10:35:33.283 INFO:teuthology.orchestra.run.smithi047.stdout: "error_string": "", 2024-03-21T10:35:33.283 INFO:teuthology.orchestra.run.smithi047.stdout: "module_options": { 2024-03-21T10:35:33.283 INFO:teuthology.orchestra.run.smithi047.stdout: "log_level": { 2024-03-21T10:35:33.284 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_level", 2024-03-21T10:35:33.284 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T10:35:33.284 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:33.284 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T10:35:33.284 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-03-21T10:35:33.284 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:33.284 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:33.284 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-03-21T10:35:33.284 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-03-21T10:35:33.284 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-03-21T10:35:33.284 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-03-21T10:35:33.284 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-03-21T10:35:33.284 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-03-21T10:35:33.284 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-03-21T10:35:33.285 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-03-21T10:35:33.285 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T10:35:33.285 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:33.285 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:33.285 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:33.285 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:33.285 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster": { 2024-03-21T10:35:33.285 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster", 2024-03-21T10:35:33.285 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T10:35:33.285 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:33.285 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T10:35:33.285 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-03-21T10:35:33.285 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:33.285 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:33.285 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:33.285 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T10:35:33.285 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:33.286 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:33.286 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:33.286 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:33.286 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster_level": { 2024-03-21T10:35:33.286 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster_level", 2024-03-21T10:35:33.286 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T10:35:33.286 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:33.286 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T10:35:33.286 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "info", 2024-03-21T10:35:33.286 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:33.286 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:33.286 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-03-21T10:35:33.286 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-03-21T10:35:33.286 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-03-21T10:35:33.287 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-03-21T10:35:33.287 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-03-21T10:35:33.287 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-03-21T10:35:33.287 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-03-21T10:35:33.287 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-03-21T10:35:33.287 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T10:35:33.287 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:33.287 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:33.287 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:33.287 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:33.287 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_file": { 2024-03-21T10:35:33.287 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_file", 2024-03-21T10:35:33.287 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T10:35:33.287 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:33.287 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T10:35:33.288 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-03-21T10:35:33.288 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:33.288 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:33.288 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:33.288 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T10:35:33.288 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:33.288 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:33.288 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:33.288 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-21T10:35:33.288 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-21T10:35:33.288 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:33.288 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-03-21T10:35:33.288 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "rook", 2024-03-21T10:35:33.288 INFO:teuthology.orchestra.run.smithi047.stdout: "can_run": true, 2024-03-21T10:35:33.289 INFO:teuthology.orchestra.run.smithi047.stdout: "error_string": "", 2024-03-21T10:35:33.289 INFO:teuthology.orchestra.run.smithi047.stdout: "module_options": { 2024-03-21T10:35:33.289 INFO:teuthology.orchestra.run.smithi047.stdout: "log_level": { 2024-03-21T10:35:33.289 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_level", 2024-03-21T10:35:33.289 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T10:35:33.289 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:33.289 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T10:35:33.289 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-03-21T10:35:33.289 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:33.289 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:33.289 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-03-21T10:35:33.289 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-03-21T10:35:33.289 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-03-21T10:35:33.289 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-03-21T10:35:33.290 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-03-21T10:35:33.290 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-03-21T10:35:33.290 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-03-21T10:35:33.290 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-03-21T10:35:33.290 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T10:35:33.290 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:33.290 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:33.290 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:33.290 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:33.290 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster": { 2024-03-21T10:35:33.290 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster", 2024-03-21T10:35:33.290 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T10:35:33.290 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:33.291 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T10:35:33.291 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-03-21T10:35:33.291 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:33.291 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:33.291 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:33.291 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T10:35:33.291 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:33.291 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:33.291 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:33.291 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:33.291 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster_level": { 2024-03-21T10:35:33.291 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster_level", 2024-03-21T10:35:33.292 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T10:35:33.292 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:33.292 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T10:35:33.292 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "info", 2024-03-21T10:35:33.292 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:33.292 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:33.292 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-03-21T10:35:33.292 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-03-21T10:35:33.292 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-03-21T10:35:33.292 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-03-21T10:35:33.292 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-03-21T10:35:33.292 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-03-21T10:35:33.292 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-03-21T10:35:33.292 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-03-21T10:35:33.293 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T10:35:33.293 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:33.293 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:33.293 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:33.293 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:33.293 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_file": { 2024-03-21T10:35:33.293 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_file", 2024-03-21T10:35:33.293 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T10:35:33.293 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:33.293 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T10:35:33.293 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-03-21T10:35:33.293 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:33.293 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:33.294 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:33.294 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T10:35:33.294 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:33.294 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:33.294 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:33.294 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:33.294 INFO:teuthology.orchestra.run.smithi047.stdout: "storage_class": { 2024-03-21T10:35:33.294 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "storage_class", 2024-03-21T10:35:33.294 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T10:35:33.294 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:33.294 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T10:35:33.294 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "local", 2024-03-21T10:35:33.294 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:33.294 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:33.295 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:33.295 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-03-21T10:35:33.295 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:33.295 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:33.295 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:33.295 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-21T10:35:33.295 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-21T10:35:33.295 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:33.295 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-03-21T10:35:33.295 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "selftest", 2024-03-21T10:35:33.295 INFO:teuthology.orchestra.run.smithi047.stdout: "can_run": true, 2024-03-21T10:35:33.295 INFO:teuthology.orchestra.run.smithi047.stdout: "error_string": "", 2024-03-21T10:35:33.295 INFO:teuthology.orchestra.run.smithi047.stdout: "module_options": { 2024-03-21T10:35:33.295 INFO:teuthology.orchestra.run.smithi047.stdout: "log_level": { 2024-03-21T10:35:33.296 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_level", 2024-03-21T10:35:33.296 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T10:35:33.296 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:33.296 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T10:35:33.296 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-03-21T10:35:33.296 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:33.296 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:33.296 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-03-21T10:35:33.296 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-03-21T10:35:33.296 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-03-21T10:35:33.296 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-03-21T10:35:33.296 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-03-21T10:35:33.296 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-03-21T10:35:33.296 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-03-21T10:35:33.296 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-03-21T10:35:33.297 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T10:35:33.297 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:33.297 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:33.297 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:33.297 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:33.297 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster": { 2024-03-21T10:35:33.297 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster", 2024-03-21T10:35:33.297 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T10:35:33.297 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:33.297 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T10:35:33.297 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-03-21T10:35:33.297 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:33.297 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:33.297 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:33.297 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T10:35:33.298 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:33.298 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:33.298 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:33.298 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:33.298 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster_level": { 2024-03-21T10:35:33.298 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster_level", 2024-03-21T10:35:33.298 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T10:35:33.298 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:33.298 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T10:35:33.298 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "info", 2024-03-21T10:35:33.298 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:33.298 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:33.298 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-03-21T10:35:33.299 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-03-21T10:35:33.299 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-03-21T10:35:33.299 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-03-21T10:35:33.299 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-03-21T10:35:33.299 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-03-21T10:35:33.299 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-03-21T10:35:33.299 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-03-21T10:35:33.299 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T10:35:33.299 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:33.299 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:33.299 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:33.299 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:33.299 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_file": { 2024-03-21T10:35:33.299 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_file", 2024-03-21T10:35:33.300 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T10:35:33.300 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:33.300 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T10:35:33.300 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-03-21T10:35:33.300 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:33.300 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:33.300 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:33.300 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T10:35:33.300 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:33.300 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:33.300 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:33.300 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:33.300 INFO:teuthology.orchestra.run.smithi047.stdout: "roption1": { 2024-03-21T10:35:33.300 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "roption1", 2024-03-21T10:35:33.300 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T10:35:33.301 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:33.301 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T10:35:33.301 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-03-21T10:35:33.301 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:33.301 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:33.301 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:33.301 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T10:35:33.301 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:33.301 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:33.301 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:33.301 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:33.301 INFO:teuthology.orchestra.run.smithi047.stdout: "roption2": { 2024-03-21T10:35:33.301 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "roption2", 2024-03-21T10:35:33.302 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T10:35:33.302 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:33.302 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T10:35:33.302 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "xyz", 2024-03-21T10:35:33.302 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:33.302 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:33.302 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:33.302 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T10:35:33.302 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:33.302 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:33.302 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:33.302 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:33.302 INFO:teuthology.orchestra.run.smithi047.stdout: "rwoption1": { 2024-03-21T10:35:33.303 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "rwoption1", 2024-03-21T10:35:33.303 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T10:35:33.303 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:33.303 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T10:35:33.303 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-03-21T10:35:33.303 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:33.303 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:33.303 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:33.303 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T10:35:33.303 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:33.303 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:33.303 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:33.304 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:33.304 INFO:teuthology.orchestra.run.smithi047.stdout: "rwoption2": { 2024-03-21T10:35:33.304 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "rwoption2", 2024-03-21T10:35:33.304 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "int", 2024-03-21T10:35:33.304 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:33.304 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T10:35:33.304 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-03-21T10:35:33.304 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:33.304 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:33.304 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:33.304 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T10:35:33.304 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:33.304 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:33.304 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:33.305 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:33.305 INFO:teuthology.orchestra.run.smithi047.stdout: "rwoption3": { 2024-03-21T10:35:33.305 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "rwoption3", 2024-03-21T10:35:33.305 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "float", 2024-03-21T10:35:33.305 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:33.305 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T10:35:33.305 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-03-21T10:35:33.305 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:33.305 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:33.305 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:33.305 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T10:35:33.305 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:33.305 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:33.305 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:33.306 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:33.306 INFO:teuthology.orchestra.run.smithi047.stdout: "rwoption4": { 2024-03-21T10:35:33.306 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "rwoption4", 2024-03-21T10:35:33.306 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T10:35:33.306 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:33.306 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T10:35:33.306 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-03-21T10:35:33.306 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:33.306 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:33.306 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:33.306 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T10:35:33.306 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:33.306 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:33.306 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:33.307 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:33.307 INFO:teuthology.orchestra.run.smithi047.stdout: "rwoption5": { 2024-03-21T10:35:33.307 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "rwoption5", 2024-03-21T10:35:33.307 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T10:35:33.307 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:33.307 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T10:35:33.307 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-03-21T10:35:33.307 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:33.307 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:33.307 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:33.307 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T10:35:33.307 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:33.307 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:33.307 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:33.308 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:33.308 INFO:teuthology.orchestra.run.smithi047.stdout: "rwoption6": { 2024-03-21T10:35:33.308 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "rwoption6", 2024-03-21T10:35:33.308 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T10:35:33.308 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:33.308 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T10:35:33.308 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "True", 2024-03-21T10:35:33.308 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:33.308 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:33.308 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:33.308 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T10:35:33.308 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:33.308 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:33.308 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:33.308 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:33.308 INFO:teuthology.orchestra.run.smithi047.stdout: "rwoption7": { 2024-03-21T10:35:33.309 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "rwoption7", 2024-03-21T10:35:33.309 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "int", 2024-03-21T10:35:33.309 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:33.309 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T10:35:33.309 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-03-21T10:35:33.309 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "1", 2024-03-21T10:35:33.309 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "42", 2024-03-21T10:35:33.309 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:33.309 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T10:35:33.309 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:33.309 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:33.309 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:33.309 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:33.309 INFO:teuthology.orchestra.run.smithi047.stdout: "testkey": { 2024-03-21T10:35:33.309 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "testkey", 2024-03-21T10:35:33.309 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T10:35:33.310 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:33.310 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T10:35:33.310 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-03-21T10:35:33.310 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:33.310 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:33.310 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:33.310 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T10:35:33.310 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:33.310 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:33.310 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:33.310 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:33.310 INFO:teuthology.orchestra.run.smithi047.stdout: "testlkey": { 2024-03-21T10:35:33.310 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "testlkey", 2024-03-21T10:35:33.310 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T10:35:33.311 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:33.311 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T10:35:33.311 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-03-21T10:35:33.311 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:33.311 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:33.311 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:33.311 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T10:35:33.311 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:33.311 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:33.311 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:33.311 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:33.311 INFO:teuthology.orchestra.run.smithi047.stdout: "testnewline": { 2024-03-21T10:35:33.311 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "testnewline", 2024-03-21T10:35:33.311 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T10:35:33.311 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:33.311 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T10:35:33.312 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-03-21T10:35:33.312 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:33.312 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:33.312 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:33.312 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T10:35:33.312 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:33.312 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:33.312 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:33.312 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-21T10:35:33.312 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-21T10:35:33.312 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:33.312 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-03-21T10:35:33.312 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "snap_schedule", 2024-03-21T10:35:33.312 INFO:teuthology.orchestra.run.smithi047.stdout: "can_run": true, 2024-03-21T10:35:33.312 INFO:teuthology.orchestra.run.smithi047.stdout: "error_string": "", 2024-03-21T10:35:33.313 INFO:teuthology.orchestra.run.smithi047.stdout: "module_options": { 2024-03-21T10:35:33.313 INFO:teuthology.orchestra.run.smithi047.stdout: "allow_m_granularity": { 2024-03-21T10:35:33.313 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "allow_m_granularity", 2024-03-21T10:35:33.313 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T10:35:33.313 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:33.313 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T10:35:33.313 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-03-21T10:35:33.313 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:33.313 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:33.313 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:33.313 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "allow minute scheduled snapshots", 2024-03-21T10:35:33.313 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:33.313 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:33.313 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:33.313 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:33.314 INFO:teuthology.orchestra.run.smithi047.stdout: "dump_on_update": { 2024-03-21T10:35:33.314 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "dump_on_update", 2024-03-21T10:35:33.314 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T10:35:33.314 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:33.314 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T10:35:33.314 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-03-21T10:35:33.314 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:33.314 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:33.314 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:33.314 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "dump database to debug log on update", 2024-03-21T10:35:33.314 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:33.314 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:33.314 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:33.314 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:33.314 INFO:teuthology.orchestra.run.smithi047.stdout: "log_level": { 2024-03-21T10:35:33.315 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_level", 2024-03-21T10:35:33.315 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T10:35:33.315 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:33.315 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T10:35:33.315 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-03-21T10:35:33.315 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:33.315 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:33.315 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-03-21T10:35:33.315 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-03-21T10:35:33.315 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-03-21T10:35:33.315 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-03-21T10:35:33.315 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-03-21T10:35:33.315 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-03-21T10:35:33.315 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-03-21T10:35:33.315 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-03-21T10:35:33.315 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T10:35:33.316 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:33.316 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:33.316 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:33.316 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:33.316 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster": { 2024-03-21T10:35:33.316 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster", 2024-03-21T10:35:33.316 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T10:35:33.316 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:33.316 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T10:35:33.316 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-03-21T10:35:33.316 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:33.316 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:33.316 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:33.317 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T10:35:33.317 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:33.317 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:33.317 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:33.317 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:33.317 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster_level": { 2024-03-21T10:35:33.317 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster_level", 2024-03-21T10:35:33.317 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T10:35:33.317 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:33.317 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T10:35:33.317 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "info", 2024-03-21T10:35:33.317 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:33.317 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:33.317 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-03-21T10:35:33.317 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-03-21T10:35:33.317 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-03-21T10:35:33.318 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-03-21T10:35:33.318 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-03-21T10:35:33.318 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-03-21T10:35:33.318 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-03-21T10:35:33.318 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-03-21T10:35:33.318 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T10:35:33.318 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:33.318 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:33.318 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:33.318 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:33.318 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_file": { 2024-03-21T10:35:33.318 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_file", 2024-03-21T10:35:33.318 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T10:35:33.318 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:33.319 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T10:35:33.319 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-03-21T10:35:33.319 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:33.319 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:33.319 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:33.319 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T10:35:33.319 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:33.319 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:33.319 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:33.319 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-21T10:35:33.319 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-21T10:35:33.319 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:33.319 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-03-21T10:35:33.320 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "stats", 2024-03-21T10:35:33.320 INFO:teuthology.orchestra.run.smithi047.stdout: "can_run": true, 2024-03-21T10:35:33.320 INFO:teuthology.orchestra.run.smithi047.stdout: "error_string": "", 2024-03-21T10:35:33.320 INFO:teuthology.orchestra.run.smithi047.stdout: "module_options": { 2024-03-21T10:35:33.320 INFO:teuthology.orchestra.run.smithi047.stdout: "log_level": { 2024-03-21T10:35:33.320 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_level", 2024-03-21T10:35:33.320 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T10:35:33.320 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:33.320 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T10:35:33.320 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-03-21T10:35:33.320 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:33.320 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:33.320 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-03-21T10:35:33.320 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-03-21T10:35:33.320 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-03-21T10:35:33.321 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-03-21T10:35:33.321 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-03-21T10:35:33.321 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-03-21T10:35:33.321 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-03-21T10:35:33.321 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-03-21T10:35:33.321 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T10:35:33.321 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:33.321 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:33.321 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:33.321 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:33.321 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster": { 2024-03-21T10:35:33.321 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster", 2024-03-21T10:35:33.321 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T10:35:33.321 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:33.321 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T10:35:33.322 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-03-21T10:35:33.322 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:33.322 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:33.322 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:33.322 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T10:35:33.322 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:33.322 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:33.322 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:33.322 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:33.322 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster_level": { 2024-03-21T10:35:33.322 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster_level", 2024-03-21T10:35:33.322 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T10:35:33.322 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:33.322 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T10:35:33.323 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "info", 2024-03-21T10:35:33.323 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:33.323 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:33.323 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-03-21T10:35:33.323 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-03-21T10:35:33.323 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-03-21T10:35:33.323 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-03-21T10:35:33.323 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-03-21T10:35:33.323 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-03-21T10:35:33.323 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-03-21T10:35:33.323 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-03-21T10:35:33.323 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T10:35:33.323 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:33.323 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:33.324 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:33.324 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:33.324 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_file": { 2024-03-21T10:35:33.324 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_file", 2024-03-21T10:35:33.324 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T10:35:33.324 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:33.324 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T10:35:33.324 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-03-21T10:35:33.324 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:33.324 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:33.324 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:33.324 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T10:35:33.324 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:33.324 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:33.325 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:33.325 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-21T10:35:33.325 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-21T10:35:33.325 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:33.325 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-03-21T10:35:33.325 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "status", 2024-03-21T10:35:33.325 INFO:teuthology.orchestra.run.smithi047.stdout: "can_run": true, 2024-03-21T10:35:33.325 INFO:teuthology.orchestra.run.smithi047.stdout: "error_string": "", 2024-03-21T10:35:33.325 INFO:teuthology.orchestra.run.smithi047.stdout: "module_options": { 2024-03-21T10:35:33.325 INFO:teuthology.orchestra.run.smithi047.stdout: "log_level": { 2024-03-21T10:35:33.325 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_level", 2024-03-21T10:35:33.325 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T10:35:33.325 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:33.325 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T10:35:33.325 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-03-21T10:35:33.326 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:33.326 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:33.326 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-03-21T10:35:33.326 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-03-21T10:35:33.326 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-03-21T10:35:33.326 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-03-21T10:35:33.326 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-03-21T10:35:33.326 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-03-21T10:35:33.326 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-03-21T10:35:33.326 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-03-21T10:35:33.326 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T10:35:33.326 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:33.326 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:33.326 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:33.327 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:33.327 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster": { 2024-03-21T10:35:33.327 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster", 2024-03-21T10:35:33.327 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T10:35:33.327 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:33.327 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T10:35:33.327 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-03-21T10:35:33.327 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:33.327 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:33.327 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:33.327 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T10:35:33.327 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:33.327 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:33.328 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:33.328 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:33.328 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster_level": { 2024-03-21T10:35:33.328 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster_level", 2024-03-21T10:35:33.328 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T10:35:33.328 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:33.328 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T10:35:33.328 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "info", 2024-03-21T10:35:33.328 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:33.328 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:33.328 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-03-21T10:35:33.328 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-03-21T10:35:33.328 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-03-21T10:35:33.328 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-03-21T10:35:33.329 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-03-21T10:35:33.329 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-03-21T10:35:33.329 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-03-21T10:35:33.329 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-03-21T10:35:33.329 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T10:35:33.329 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:33.329 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:33.329 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:33.329 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:33.329 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_file": { 2024-03-21T10:35:33.329 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_file", 2024-03-21T10:35:33.329 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T10:35:33.329 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:33.329 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T10:35:33.329 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-03-21T10:35:33.330 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:33.330 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:33.330 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:33.330 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T10:35:33.330 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:33.330 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:33.330 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:33.330 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-21T10:35:33.330 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-21T10:35:33.330 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:33.330 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-03-21T10:35:33.330 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "telegraf", 2024-03-21T10:35:33.330 INFO:teuthology.orchestra.run.smithi047.stdout: "can_run": true, 2024-03-21T10:35:33.330 INFO:teuthology.orchestra.run.smithi047.stdout: "error_string": "", 2024-03-21T10:35:33.330 INFO:teuthology.orchestra.run.smithi047.stdout: "module_options": { 2024-03-21T10:35:33.331 INFO:teuthology.orchestra.run.smithi047.stdout: "address": { 2024-03-21T10:35:33.331 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "address", 2024-03-21T10:35:33.331 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T10:35:33.331 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:33.331 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T10:35:33.331 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-03-21T10:35:33.331 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:33.331 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:33.331 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:33.331 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T10:35:33.331 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:33.331 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:33.331 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:33.331 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:33.331 INFO:teuthology.orchestra.run.smithi047.stdout: "interval": { 2024-03-21T10:35:33.332 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "interval", 2024-03-21T10:35:33.332 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "secs", 2024-03-21T10:35:33.332 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:33.332 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T10:35:33.332 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "15", 2024-03-21T10:35:33.332 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:33.332 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:33.332 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:33.332 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T10:35:33.332 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:33.332 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:33.332 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:33.332 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:33.332 INFO:teuthology.orchestra.run.smithi047.stdout: "log_level": { 2024-03-21T10:35:33.333 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_level", 2024-03-21T10:35:33.333 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T10:35:33.333 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:33.333 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T10:35:33.333 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-03-21T10:35:33.333 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:33.333 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:33.333 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-03-21T10:35:33.333 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-03-21T10:35:33.333 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-03-21T10:35:33.333 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-03-21T10:35:33.333 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-03-21T10:35:33.333 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-03-21T10:35:33.333 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-03-21T10:35:33.333 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-03-21T10:35:33.334 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T10:35:33.334 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:33.334 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:33.334 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:33.334 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:33.334 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster": { 2024-03-21T10:35:33.334 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster", 2024-03-21T10:35:33.334 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T10:35:33.334 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:33.334 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T10:35:33.334 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-03-21T10:35:33.334 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:33.334 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:33.334 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:33.334 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T10:35:33.335 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:33.335 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:33.335 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:33.335 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:33.335 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster_level": { 2024-03-21T10:35:33.335 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster_level", 2024-03-21T10:35:33.335 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T10:35:33.335 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:33.335 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T10:35:33.335 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "info", 2024-03-21T10:35:33.335 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:33.335 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:33.335 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-03-21T10:35:33.335 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-03-21T10:35:33.335 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-03-21T10:35:33.335 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-03-21T10:35:33.335 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-03-21T10:35:33.336 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-03-21T10:35:33.336 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-03-21T10:35:33.336 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-03-21T10:35:33.336 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T10:35:33.336 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:33.336 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:33.336 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:33.336 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:33.336 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_file": { 2024-03-21T10:35:33.336 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_file", 2024-03-21T10:35:33.336 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T10:35:33.336 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:33.336 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T10:35:33.336 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-03-21T10:35:33.336 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:33.337 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:33.337 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:33.337 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T10:35:33.337 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:33.337 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:33.337 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:33.337 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-21T10:35:33.337 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-21T10:35:33.337 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:33.337 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-03-21T10:35:33.337 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "telemetry", 2024-03-21T10:35:33.337 INFO:teuthology.orchestra.run.smithi047.stdout: "can_run": true, 2024-03-21T10:35:33.338 INFO:teuthology.orchestra.run.smithi047.stdout: "error_string": "", 2024-03-21T10:35:33.338 INFO:teuthology.orchestra.run.smithi047.stdout: "module_options": { 2024-03-21T10:35:33.338 INFO:teuthology.orchestra.run.smithi047.stdout: "channel_basic": { 2024-03-21T10:35:33.338 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "channel_basic", 2024-03-21T10:35:33.338 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T10:35:33.338 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:33.338 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T10:35:33.338 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "True", 2024-03-21T10:35:33.338 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:33.338 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:33.338 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:33.338 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Share basic cluster information (size, version)", 2024-03-21T10:35:33.338 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:33.339 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:33.339 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:33.339 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:33.339 INFO:teuthology.orchestra.run.smithi047.stdout: "channel_crash": { 2024-03-21T10:35:33.339 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "channel_crash", 2024-03-21T10:35:33.339 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T10:35:33.339 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:33.339 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T10:35:33.339 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "True", 2024-03-21T10:35:33.339 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:33.339 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:33.339 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:33.339 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-03-21T10:35:33.339 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:33.339 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:33.340 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:33.340 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:33.340 INFO:teuthology.orchestra.run.smithi047.stdout: "channel_device": { 2024-03-21T10:35:33.340 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "channel_device", 2024-03-21T10:35:33.340 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T10:35:33.340 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:33.340 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T10:35:33.340 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "True", 2024-03-21T10:35:33.340 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:33.340 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:33.340 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:33.340 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-03-21T10:35:33.340 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:33.340 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:33.341 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:33.341 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:33.341 INFO:teuthology.orchestra.run.smithi047.stdout: "channel_ident": { 2024-03-21T10:35:33.341 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "channel_ident", 2024-03-21T10:35:33.341 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T10:35:33.341 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:33.341 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T10:35:33.341 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-03-21T10:35:33.341 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:33.341 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:33.341 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:33.341 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-03-21T10:35:33.341 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:33.341 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:33.341 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:33.341 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:33.342 INFO:teuthology.orchestra.run.smithi047.stdout: "channel_perf": { 2024-03-21T10:35:33.342 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "channel_perf", 2024-03-21T10:35:33.342 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T10:35:33.342 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:33.342 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T10:35:33.342 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-03-21T10:35:33.342 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:33.342 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:33.342 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:33.342 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Share various performance metrics of a cluster", 2024-03-21T10:35:33.342 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:33.342 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:33.342 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:33.342 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:33.343 INFO:teuthology.orchestra.run.smithi047.stdout: "contact": { 2024-03-21T10:35:33.343 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "contact", 2024-03-21T10:35:33.343 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T10:35:33.343 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:33.343 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T10:35:33.343 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-03-21T10:35:33.343 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:33.343 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:33.343 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:33.343 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T10:35:33.343 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:33.343 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:33.343 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:33.344 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:33.344 INFO:teuthology.orchestra.run.smithi047.stdout: "description": { 2024-03-21T10:35:33.344 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "description", 2024-03-21T10:35:33.344 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T10:35:33.344 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:33.344 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T10:35:33.344 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-03-21T10:35:33.344 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:33.344 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:33.344 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:33.344 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T10:35:33.344 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:33.344 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:33.344 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:33.344 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:33.345 INFO:teuthology.orchestra.run.smithi047.stdout: "device_url": { 2024-03-21T10:35:33.345 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "device_url", 2024-03-21T10:35:33.345 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T10:35:33.345 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:33.345 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T10:35:33.345 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-03-21T10:35:33.345 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:33.345 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:33.345 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:33.345 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T10:35:33.345 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:33.345 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:33.345 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:33.346 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:33.346 INFO:teuthology.orchestra.run.smithi047.stdout: "enabled": { 2024-03-21T10:35:33.346 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "enabled", 2024-03-21T10:35:33.346 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T10:35:33.346 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:33.346 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T10:35:33.346 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-03-21T10:35:33.346 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:33.346 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:33.346 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:33.346 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T10:35:33.346 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:33.347 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:33.347 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:33.347 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:33.347 INFO:teuthology.orchestra.run.smithi047.stdout: "interval": { 2024-03-21T10:35:33.347 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "interval", 2024-03-21T10:35:33.347 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "int", 2024-03-21T10:35:33.347 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:33.347 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T10:35:33.347 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "24", 2024-03-21T10:35:33.347 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "8", 2024-03-21T10:35:33.347 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:33.347 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:33.347 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T10:35:33.348 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:33.348 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:33.348 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:33.348 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:33.348 INFO:teuthology.orchestra.run.smithi047.stdout: "last_opt_revision": { 2024-03-21T10:35:33.348 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "last_opt_revision", 2024-03-21T10:35:33.348 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "int", 2024-03-21T10:35:33.348 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:33.348 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T10:35:33.348 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "1", 2024-03-21T10:35:33.348 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:33.348 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:33.348 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:33.349 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T10:35:33.349 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:33.349 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:33.349 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:33.349 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:33.349 INFO:teuthology.orchestra.run.smithi047.stdout: "leaderboard": { 2024-03-21T10:35:33.349 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "leaderboard", 2024-03-21T10:35:33.349 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T10:35:33.349 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:33.349 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T10:35:33.349 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-03-21T10:35:33.349 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:33.349 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:33.349 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:33.350 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T10:35:33.350 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:33.350 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:33.350 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:33.350 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:33.350 INFO:teuthology.orchestra.run.smithi047.stdout: "leaderboard_description": { 2024-03-21T10:35:33.350 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "leaderboard_description", 2024-03-21T10:35:33.350 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T10:35:33.350 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:33.350 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T10:35:33.350 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-03-21T10:35:33.350 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:33.350 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:33.350 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:33.351 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T10:35:33.351 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:33.351 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:33.351 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:33.351 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:33.351 INFO:teuthology.orchestra.run.smithi047.stdout: "log_level": { 2024-03-21T10:35:33.351 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_level", 2024-03-21T10:35:33.351 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T10:35:33.351 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:33.351 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T10:35:33.351 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-03-21T10:35:33.351 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:33.351 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:33.351 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-03-21T10:35:33.352 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-03-21T10:35:33.352 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-03-21T10:35:33.352 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-03-21T10:35:33.352 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-03-21T10:35:33.352 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-03-21T10:35:33.352 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-03-21T10:35:33.352 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-03-21T10:35:33.352 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T10:35:33.352 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:33.352 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:33.352 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:33.352 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:33.352 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster": { 2024-03-21T10:35:33.352 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster", 2024-03-21T10:35:33.352 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T10:35:33.353 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:33.353 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T10:35:33.353 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-03-21T10:35:33.353 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:33.353 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:33.353 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:33.353 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T10:35:33.353 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:33.353 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:33.353 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:33.353 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:33.353 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster_level": { 2024-03-21T10:35:33.353 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster_level", 2024-03-21T10:35:33.353 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T10:35:33.354 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:33.354 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T10:35:33.354 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "info", 2024-03-21T10:35:33.354 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:33.354 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:33.354 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-03-21T10:35:33.354 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-03-21T10:35:33.354 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-03-21T10:35:33.354 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-03-21T10:35:33.354 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-03-21T10:35:33.354 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-03-21T10:35:33.354 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-03-21T10:35:33.354 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-03-21T10:35:33.355 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T10:35:33.355 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:33.355 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:33.355 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:33.355 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:33.355 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_file": { 2024-03-21T10:35:33.355 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_file", 2024-03-21T10:35:33.355 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T10:35:33.355 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:33.355 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T10:35:33.355 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-03-21T10:35:33.355 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:33.355 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:33.355 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:33.355 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T10:35:33.355 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:33.356 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:33.356 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:33.356 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:33.356 INFO:teuthology.orchestra.run.smithi047.stdout: "organization": { 2024-03-21T10:35:33.356 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "organization", 2024-03-21T10:35:33.356 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T10:35:33.356 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:33.356 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T10:35:33.356 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-03-21T10:35:33.356 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:33.356 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:33.356 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:33.356 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T10:35:33.356 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:33.357 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:33.357 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:33.357 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:33.357 INFO:teuthology.orchestra.run.smithi047.stdout: "proxy": { 2024-03-21T10:35:33.357 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "proxy", 2024-03-21T10:35:33.357 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T10:35:33.357 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:33.357 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T10:35:33.357 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-03-21T10:35:33.357 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:33.357 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:33.357 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:33.357 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T10:35:33.358 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:33.358 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:33.358 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:33.358 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:33.358 INFO:teuthology.orchestra.run.smithi047.stdout: "url": { 2024-03-21T10:35:33.358 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "url", 2024-03-21T10:35:33.358 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T10:35:33.358 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:33.358 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T10:35:33.358 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-03-21T10:35:33.358 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:33.358 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:33.358 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:33.358 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T10:35:33.358 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:33.359 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:33.359 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:33.359 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-21T10:35:33.359 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-21T10:35:33.359 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:33.359 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-03-21T10:35:33.359 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "test_orchestrator", 2024-03-21T10:35:33.359 INFO:teuthology.orchestra.run.smithi047.stdout: "can_run": true, 2024-03-21T10:35:33.359 INFO:teuthology.orchestra.run.smithi047.stdout: "error_string": "", 2024-03-21T10:35:33.359 INFO:teuthology.orchestra.run.smithi047.stdout: "module_options": { 2024-03-21T10:35:33.359 INFO:teuthology.orchestra.run.smithi047.stdout: "log_level": { 2024-03-21T10:35:33.359 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_level", 2024-03-21T10:35:33.359 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T10:35:33.359 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:33.359 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T10:35:33.360 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-03-21T10:35:33.360 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:33.360 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:33.360 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-03-21T10:35:33.360 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-03-21T10:35:33.360 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-03-21T10:35:33.360 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-03-21T10:35:33.360 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-03-21T10:35:33.360 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-03-21T10:35:33.360 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-03-21T10:35:33.360 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-03-21T10:35:33.360 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T10:35:33.360 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:33.360 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:33.361 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:33.361 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:33.361 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster": { 2024-03-21T10:35:33.361 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster", 2024-03-21T10:35:33.361 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T10:35:33.361 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:33.361 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T10:35:33.361 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-03-21T10:35:33.361 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:33.361 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:33.361 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:33.361 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T10:35:33.361 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:33.361 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:33.362 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:33.362 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:33.362 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster_level": { 2024-03-21T10:35:33.362 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster_level", 2024-03-21T10:35:33.362 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T10:35:33.362 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:33.362 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T10:35:33.362 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "info", 2024-03-21T10:35:33.362 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:33.362 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:33.362 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-03-21T10:35:33.362 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-03-21T10:35:33.362 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-03-21T10:35:33.362 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-03-21T10:35:33.362 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-03-21T10:35:33.363 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-03-21T10:35:33.363 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-03-21T10:35:33.363 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-03-21T10:35:33.363 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T10:35:33.363 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:33.363 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:33.363 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:33.363 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:33.363 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_file": { 2024-03-21T10:35:33.363 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_file", 2024-03-21T10:35:33.363 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T10:35:33.363 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:33.363 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T10:35:33.363 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-03-21T10:35:33.363 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:33.364 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:33.364 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:33.364 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T10:35:33.364 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:33.364 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:33.364 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:33.364 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-21T10:35:33.364 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-21T10:35:33.364 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:33.364 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-03-21T10:35:33.364 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "volumes", 2024-03-21T10:35:33.364 INFO:teuthology.orchestra.run.smithi047.stdout: "can_run": true, 2024-03-21T10:35:33.364 INFO:teuthology.orchestra.run.smithi047.stdout: "error_string": "", 2024-03-21T10:35:33.364 INFO:teuthology.orchestra.run.smithi047.stdout: "module_options": { 2024-03-21T10:35:33.364 INFO:teuthology.orchestra.run.smithi047.stdout: "log_level": { 2024-03-21T10:35:33.364 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_level", 2024-03-21T10:35:33.365 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T10:35:33.365 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:33.365 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T10:35:33.365 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-03-21T10:35:33.365 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:33.365 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:33.365 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-03-21T10:35:33.365 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-03-21T10:35:33.365 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-03-21T10:35:33.365 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-03-21T10:35:33.365 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-03-21T10:35:33.365 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-03-21T10:35:33.365 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-03-21T10:35:33.365 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-03-21T10:35:33.365 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T10:35:33.366 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:33.366 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:33.366 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:33.366 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:33.366 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster": { 2024-03-21T10:35:33.366 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster", 2024-03-21T10:35:33.366 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T10:35:33.366 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:33.366 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T10:35:33.366 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-03-21T10:35:33.366 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:33.366 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:33.366 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:33.366 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T10:35:33.367 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:33.367 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:33.367 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:33.367 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:33.367 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster_level": { 2024-03-21T10:35:33.367 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster_level", 2024-03-21T10:35:33.367 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T10:35:33.367 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:33.367 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T10:35:33.367 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "info", 2024-03-21T10:35:33.367 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:33.367 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:33.367 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-03-21T10:35:33.367 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-03-21T10:35:33.367 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-03-21T10:35:33.367 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-03-21T10:35:33.367 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-03-21T10:35:33.368 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-03-21T10:35:33.368 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-03-21T10:35:33.368 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-03-21T10:35:33.368 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T10:35:33.368 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:33.368 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:33.368 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:33.368 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:33.368 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_file": { 2024-03-21T10:35:33.368 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_file", 2024-03-21T10:35:33.368 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T10:35:33.368 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:33.368 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T10:35:33.368 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-03-21T10:35:33.369 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:33.369 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:33.369 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:33.369 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T10:35:33.369 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:33.369 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:33.369 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:33.369 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:33.369 INFO:teuthology.orchestra.run.smithi047.stdout: "max_concurrent_clones": { 2024-03-21T10:35:33.369 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "max_concurrent_clones", 2024-03-21T10:35:33.369 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "int", 2024-03-21T10:35:33.369 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:33.369 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T10:35:33.369 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "4", 2024-03-21T10:35:33.369 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:33.370 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:33.370 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:33.370 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Number of asynchronous cloner threads", 2024-03-21T10:35:33.370 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:33.370 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:33.370 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:33.370 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:33.370 INFO:teuthology.orchestra.run.smithi047.stdout: "periodic_async_work": { 2024-03-21T10:35:33.370 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "periodic_async_work", 2024-03-21T10:35:33.370 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T10:35:33.370 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:33.370 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T10:35:33.370 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-03-21T10:35:33.371 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:33.371 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:33.371 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:33.371 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Periodically check for async work", 2024-03-21T10:35:33.371 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:33.371 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:33.371 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:33.371 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:33.371 INFO:teuthology.orchestra.run.smithi047.stdout: "snapshot_clone_delay": { 2024-03-21T10:35:33.371 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "snapshot_clone_delay", 2024-03-21T10:35:33.371 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "int", 2024-03-21T10:35:33.371 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:33.371 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T10:35:33.371 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "0", 2024-03-21T10:35:33.371 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:33.372 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:33.372 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:33.372 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-03-21T10:35:33.372 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:33.372 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:33.372 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:33.372 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:33.372 INFO:teuthology.orchestra.run.smithi047.stdout: "snapshot_clone_no_wait": { 2024-03-21T10:35:33.372 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "snapshot_clone_no_wait", 2024-03-21T10:35:33.372 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T10:35:33.372 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:33.372 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T10:35:33.372 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "True", 2024-03-21T10:35:33.372 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:33.372 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:33.372 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:33.372 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-03-21T10:35:33.373 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:33.373 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:33.373 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:33.373 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-21T10:35:33.373 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-21T10:35:33.373 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:33.373 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-03-21T10:35:33.373 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "zabbix", 2024-03-21T10:35:33.373 INFO:teuthology.orchestra.run.smithi047.stdout: "can_run": true, 2024-03-21T10:35:33.373 INFO:teuthology.orchestra.run.smithi047.stdout: "error_string": "", 2024-03-21T10:35:33.373 INFO:teuthology.orchestra.run.smithi047.stdout: "module_options": { 2024-03-21T10:35:33.373 INFO:teuthology.orchestra.run.smithi047.stdout: "discovery_interval": { 2024-03-21T10:35:33.374 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "discovery_interval", 2024-03-21T10:35:33.374 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "uint", 2024-03-21T10:35:33.374 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:33.374 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T10:35:33.374 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "100", 2024-03-21T10:35:33.374 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:33.374 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:33.374 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:33.374 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T10:35:33.374 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:33.374 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:33.374 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:33.374 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:33.375 INFO:teuthology.orchestra.run.smithi047.stdout: "identifier": { 2024-03-21T10:35:33.375 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "identifier", 2024-03-21T10:35:33.375 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T10:35:33.375 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:33.375 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T10:35:33.375 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-03-21T10:35:33.375 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:33.375 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:33.375 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:33.375 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T10:35:33.375 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:33.375 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:33.375 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:33.375 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:33.375 INFO:teuthology.orchestra.run.smithi047.stdout: "interval": { 2024-03-21T10:35:33.376 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "interval", 2024-03-21T10:35:33.376 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "secs", 2024-03-21T10:35:33.376 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:33.376 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T10:35:33.376 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "60", 2024-03-21T10:35:33.376 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:33.376 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:33.376 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:33.376 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T10:35:33.376 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:33.376 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:33.376 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:33.376 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:33.376 INFO:teuthology.orchestra.run.smithi047.stdout: "log_level": { 2024-03-21T10:35:33.376 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_level", 2024-03-21T10:35:33.377 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T10:35:33.377 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:33.377 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T10:35:33.377 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-03-21T10:35:33.377 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:33.377 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:33.377 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-03-21T10:35:33.377 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-03-21T10:35:33.377 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-03-21T10:35:33.377 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-03-21T10:35:33.377 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-03-21T10:35:33.377 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-03-21T10:35:33.377 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-03-21T10:35:33.377 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-03-21T10:35:33.378 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T10:35:33.378 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:33.378 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:33.378 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:33.378 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:33.378 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster": { 2024-03-21T10:35:33.378 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster", 2024-03-21T10:35:33.378 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T10:35:33.378 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:33.378 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T10:35:33.378 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-03-21T10:35:33.378 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:33.379 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:33.379 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:33.379 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T10:35:33.379 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:33.379 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:33.379 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:33.379 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:33.379 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster_level": { 2024-03-21T10:35:33.379 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster_level", 2024-03-21T10:35:33.379 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T10:35:33.379 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:33.379 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T10:35:33.379 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "info", 2024-03-21T10:35:33.379 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:33.380 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:33.380 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-03-21T10:35:33.380 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-03-21T10:35:33.380 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-03-21T10:35:33.380 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-03-21T10:35:33.380 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-03-21T10:35:33.380 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-03-21T10:35:33.380 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-03-21T10:35:33.380 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-03-21T10:35:33.380 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T10:35:33.380 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:33.380 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:33.380 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:33.380 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:33.380 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_file": { 2024-03-21T10:35:33.381 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_file", 2024-03-21T10:35:33.381 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T10:35:33.381 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:33.381 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T10:35:33.381 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-03-21T10:35:33.381 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:33.381 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:33.381 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:33.381 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T10:35:33.381 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:33.381 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:33.381 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:33.381 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:33.381 INFO:teuthology.orchestra.run.smithi047.stdout: "zabbix_host": { 2024-03-21T10:35:33.381 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "zabbix_host", 2024-03-21T10:35:33.382 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T10:35:33.382 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:33.382 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T10:35:33.382 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-03-21T10:35:33.382 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:33.382 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:33.382 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:33.382 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T10:35:33.382 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:33.382 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:33.382 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:33.382 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:33.382 INFO:teuthology.orchestra.run.smithi047.stdout: "zabbix_port": { 2024-03-21T10:35:33.382 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "zabbix_port", 2024-03-21T10:35:33.383 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "int", 2024-03-21T10:35:33.383 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:33.383 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T10:35:33.383 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "10051", 2024-03-21T10:35:33.383 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:33.383 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:33.383 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:33.383 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T10:35:33.383 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:33.383 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:33.383 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:33.383 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:33.383 INFO:teuthology.orchestra.run.smithi047.stdout: "zabbix_sender": { 2024-03-21T10:35:33.383 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "zabbix_sender", 2024-03-21T10:35:33.384 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T10:35:33.384 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:33.384 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T10:35:33.384 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-03-21T10:35:33.384 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:33.384 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:33.384 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:33.384 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T10:35:33.384 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:33.384 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:33.384 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:33.384 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-21T10:35:33.384 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-21T10:35:33.385 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-21T10:35:33.385 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-03-21T10:35:33.385 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-21T10:35:33.385 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-03-21T10:35:33.385 INFO:teuthology.orchestra.run.smithi047.stdout: "modules": [ 2024-03-21T10:35:33.385 INFO:teuthology.orchestra.run.smithi047.stdout: "cephadm", 2024-03-21T10:35:33.385 INFO:teuthology.orchestra.run.smithi047.stdout: "dashboard", 2024-03-21T10:35:33.385 INFO:teuthology.orchestra.run.smithi047.stdout: "iostat", 2024-03-21T10:35:33.385 INFO:teuthology.orchestra.run.smithi047.stdout: "nfs", 2024-03-21T10:35:33.385 INFO:teuthology.orchestra.run.smithi047.stdout: "restful", 2024-03-21T10:35:33.385 INFO:teuthology.orchestra.run.smithi047.stdout: "snap_schedule" 2024-03-21T10:35:33.385 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-03-21T10:35:33.385 INFO:teuthology.orchestra.run.smithi047.stdout: "available_modules": [ 2024-03-21T10:35:33.386 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-03-21T10:35:33.386 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "alerts", 2024-03-21T10:35:33.386 INFO:teuthology.orchestra.run.smithi047.stdout: "can_run": true, 2024-03-21T10:35:33.386 INFO:teuthology.orchestra.run.smithi047.stdout: "error_string": "", 2024-03-21T10:35:33.386 INFO:teuthology.orchestra.run.smithi047.stdout: "module_options": { 2024-03-21T10:35:33.386 INFO:teuthology.orchestra.run.smithi047.stdout: "interval": { 2024-03-21T10:35:33.386 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "interval", 2024-03-21T10:35:33.386 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "secs", 2024-03-21T10:35:33.386 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:33.386 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T10:35:33.386 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "60", 2024-03-21T10:35:33.386 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:33.386 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:33.386 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:33.387 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "How frequently to reexamine health status", 2024-03-21T10:35:33.387 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:33.387 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:33.387 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:33.387 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:33.387 INFO:teuthology.orchestra.run.smithi047.stdout: "log_level": { 2024-03-21T10:35:33.387 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_level", 2024-03-21T10:35:33.387 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T10:35:33.387 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:33.387 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T10:35:33.387 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-03-21T10:35:33.387 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:33.387 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:33.387 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-03-21T10:35:33.387 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-03-21T10:35:33.387 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-03-21T10:35:33.388 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-03-21T10:35:33.388 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-03-21T10:35:33.388 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-03-21T10:35:33.388 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-03-21T10:35:33.388 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-03-21T10:35:33.388 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T10:35:33.388 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:33.388 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:33.388 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:33.388 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:33.388 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster": { 2024-03-21T10:35:33.388 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster", 2024-03-21T10:35:33.388 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T10:35:33.388 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:33.389 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T10:35:33.389 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-03-21T10:35:33.389 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:33.389 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:33.389 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:33.389 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T10:35:33.389 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:33.389 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:33.389 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:33.389 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:33.389 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster_level": { 2024-03-21T10:35:33.389 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster_level", 2024-03-21T10:35:33.389 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T10:35:33.389 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:33.390 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T10:35:33.390 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "info", 2024-03-21T10:35:33.390 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:33.390 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:33.390 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-03-21T10:35:33.390 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-03-21T10:35:33.390 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-03-21T10:35:33.390 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-03-21T10:35:33.390 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-03-21T10:35:33.390 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-03-21T10:35:33.390 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-03-21T10:35:33.390 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-03-21T10:35:33.390 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T10:35:33.391 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:33.391 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:33.391 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:33.391 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:33.391 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_file": { 2024-03-21T10:35:33.391 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_file", 2024-03-21T10:35:33.391 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T10:35:33.391 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:33.391 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T10:35:33.391 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-03-21T10:35:33.391 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:33.391 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:33.391 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:33.391 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T10:35:33.391 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:33.391 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:33.392 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:33.392 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:33.392 INFO:teuthology.orchestra.run.smithi047.stdout: "smtp_destination": { 2024-03-21T10:35:33.392 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "smtp_destination", 2024-03-21T10:35:33.392 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T10:35:33.392 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:33.392 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T10:35:33.392 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-03-21T10:35:33.392 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:33.392 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:33.392 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:33.392 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Email address to send alerts to", 2024-03-21T10:35:33.392 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:33.392 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:33.393 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:33.393 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:33.393 INFO:teuthology.orchestra.run.smithi047.stdout: "smtp_from_name": { 2024-03-21T10:35:33.393 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "smtp_from_name", 2024-03-21T10:35:33.393 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T10:35:33.393 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:33.393 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T10:35:33.393 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "Ceph", 2024-03-21T10:35:33.393 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:33.393 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:33.393 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:33.393 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Email From: name", 2024-03-21T10:35:33.393 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:33.393 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:33.394 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:33.394 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:33.394 INFO:teuthology.orchestra.run.smithi047.stdout: "smtp_host": { 2024-03-21T10:35:33.394 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "smtp_host", 2024-03-21T10:35:33.394 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T10:35:33.394 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:33.394 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T10:35:33.394 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-03-21T10:35:33.394 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:33.394 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:33.394 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:33.394 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "SMTP server", 2024-03-21T10:35:33.394 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:33.394 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:33.394 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:33.395 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:33.395 INFO:teuthology.orchestra.run.smithi047.stdout: "smtp_password": { 2024-03-21T10:35:33.395 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "smtp_password", 2024-03-21T10:35:33.395 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T10:35:33.395 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:33.395 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T10:35:33.395 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-03-21T10:35:33.395 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:33.395 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:33.395 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:33.395 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Password to authenticate with", 2024-03-21T10:35:33.395 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:33.395 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:33.395 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:33.395 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:33.396 INFO:teuthology.orchestra.run.smithi047.stdout: "smtp_port": { 2024-03-21T10:35:33.396 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "smtp_port", 2024-03-21T10:35:33.396 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "int", 2024-03-21T10:35:33.396 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:33.396 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T10:35:33.396 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "465", 2024-03-21T10:35:33.396 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:33.396 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:33.396 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:33.396 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "SMTP port", 2024-03-21T10:35:33.396 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:33.396 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:33.396 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:33.396 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:33.396 INFO:teuthology.orchestra.run.smithi047.stdout: "smtp_sender": { 2024-03-21T10:35:33.397 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "smtp_sender", 2024-03-21T10:35:33.397 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T10:35:33.397 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:33.397 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T10:35:33.397 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-03-21T10:35:33.397 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:33.397 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:33.397 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:33.397 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "SMTP envelope sender", 2024-03-21T10:35:33.397 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:33.397 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:33.397 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:33.397 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:33.397 INFO:teuthology.orchestra.run.smithi047.stdout: "smtp_ssl": { 2024-03-21T10:35:33.397 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "smtp_ssl", 2024-03-21T10:35:33.398 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T10:35:33.398 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:33.398 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T10:35:33.398 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "True", 2024-03-21T10:35:33.398 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:33.398 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:33.398 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:33.398 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Use SSL to connect to SMTP server", 2024-03-21T10:35:33.398 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:33.398 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:33.398 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:33.398 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:33.398 INFO:teuthology.orchestra.run.smithi047.stdout: "smtp_user": { 2024-03-21T10:35:33.398 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "smtp_user", 2024-03-21T10:35:33.399 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T10:35:33.399 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:33.399 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T10:35:33.399 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-03-21T10:35:33.399 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:33.399 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:33.399 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:33.399 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "User to authenticate as", 2024-03-21T10:35:33.399 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:33.399 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:33.399 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:33.399 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-21T10:35:33.399 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-21T10:35:33.399 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:33.399 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-03-21T10:35:33.400 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "balancer", 2024-03-21T10:35:33.400 INFO:teuthology.orchestra.run.smithi047.stdout: "can_run": true, 2024-03-21T10:35:33.400 INFO:teuthology.orchestra.run.smithi047.stdout: "error_string": "", 2024-03-21T10:35:33.400 INFO:teuthology.orchestra.run.smithi047.stdout: "module_options": { 2024-03-21T10:35:33.400 INFO:teuthology.orchestra.run.smithi047.stdout: "active": { 2024-03-21T10:35:33.400 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "active", 2024-03-21T10:35:33.400 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T10:35:33.400 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:33.400 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T10:35:33.400 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "True", 2024-03-21T10:35:33.400 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:33.400 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:33.400 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:33.400 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "automatically balance PGs across cluster", 2024-03-21T10:35:33.400 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:33.401 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:33.401 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:33.401 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:33.401 INFO:teuthology.orchestra.run.smithi047.stdout: "begin_time": { 2024-03-21T10:35:33.401 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "begin_time", 2024-03-21T10:35:33.401 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T10:35:33.401 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:33.401 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T10:35:33.401 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "0000", 2024-03-21T10:35:33.401 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:33.401 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:33.401 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:33.401 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "beginning time of day to automatically balance", 2024-03-21T10:35:33.401 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-03-21T10:35:33.401 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:33.402 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:33.402 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:33.402 INFO:teuthology.orchestra.run.smithi047.stdout: "begin_weekday": { 2024-03-21T10:35:33.402 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "begin_weekday", 2024-03-21T10:35:33.402 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "uint", 2024-03-21T10:35:33.402 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:33.402 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T10:35:33.402 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "0", 2024-03-21T10:35:33.402 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "0", 2024-03-21T10:35:33.402 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "6", 2024-03-21T10:35:33.402 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:33.402 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-03-21T10:35:33.402 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-03-21T10:35:33.402 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:33.403 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:33.403 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:33.403 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_compat_max_iterations": { 2024-03-21T10:35:33.403 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "crush_compat_max_iterations", 2024-03-21T10:35:33.403 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "uint", 2024-03-21T10:35:33.403 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:33.403 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T10:35:33.403 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "25", 2024-03-21T10:35:33.403 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "1", 2024-03-21T10:35:33.403 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "250", 2024-03-21T10:35:33.403 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:33.403 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-03-21T10:35:33.403 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:33.403 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:33.403 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:33.403 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:33.404 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_compat_metrics": { 2024-03-21T10:35:33.404 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "crush_compat_metrics", 2024-03-21T10:35:33.404 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T10:35:33.404 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:33.404 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T10:35:33.404 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "pgs,objects,bytes", 2024-03-21T10:35:33.404 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:33.404 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:33.404 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:33.404 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-03-21T10:35:33.404 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-03-21T10:35:33.404 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:33.404 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:33.405 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:33.405 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_compat_step": { 2024-03-21T10:35:33.405 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "crush_compat_step", 2024-03-21T10:35:33.405 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "float", 2024-03-21T10:35:33.405 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:33.405 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T10:35:33.405 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "0.5", 2024-03-21T10:35:33.405 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "0.001", 2024-03-21T10:35:33.405 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "0.999", 2024-03-21T10:35:33.405 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:33.405 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "aggressiveness of optimization", 2024-03-21T10:35:33.405 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-03-21T10:35:33.405 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:33.405 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:33.406 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:33.406 INFO:teuthology.orchestra.run.smithi047.stdout: "end_time": { 2024-03-21T10:35:33.406 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "end_time", 2024-03-21T10:35:33.406 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T10:35:33.406 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:33.406 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T10:35:33.406 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "2359", 2024-03-21T10:35:33.406 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:33.406 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:33.406 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:33.406 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "ending time of day to automatically balance", 2024-03-21T10:35:33.406 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-03-21T10:35:33.406 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:33.406 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:33.407 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:33.407 INFO:teuthology.orchestra.run.smithi047.stdout: "end_weekday": { 2024-03-21T10:35:33.407 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "end_weekday", 2024-03-21T10:35:33.407 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "uint", 2024-03-21T10:35:33.407 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:33.407 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T10:35:33.407 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "0", 2024-03-21T10:35:33.407 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "0", 2024-03-21T10:35:33.407 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "6", 2024-03-21T10:35:33.407 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:33.407 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-03-21T10:35:33.407 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-03-21T10:35:33.407 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:33.407 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:33.407 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:33.408 INFO:teuthology.orchestra.run.smithi047.stdout: "log_level": { 2024-03-21T10:35:33.408 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_level", 2024-03-21T10:35:33.408 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T10:35:33.408 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:33.408 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T10:35:33.408 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-03-21T10:35:33.408 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:33.408 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:33.408 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-03-21T10:35:33.408 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-03-21T10:35:33.408 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-03-21T10:35:33.408 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-03-21T10:35:33.408 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-03-21T10:35:33.408 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-03-21T10:35:33.409 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-03-21T10:35:33.409 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-03-21T10:35:33.409 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T10:35:33.409 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:33.409 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:33.409 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:33.409 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:33.409 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster": { 2024-03-21T10:35:33.409 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster", 2024-03-21T10:35:33.409 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T10:35:33.409 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:33.409 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T10:35:33.409 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-03-21T10:35:33.409 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:33.409 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:33.409 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:33.410 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T10:35:33.410 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:33.410 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:33.410 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:33.410 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:33.410 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster_level": { 2024-03-21T10:35:33.410 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster_level", 2024-03-21T10:35:33.410 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T10:35:33.410 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:33.410 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T10:35:33.410 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "info", 2024-03-21T10:35:33.410 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:33.410 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:33.410 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-03-21T10:35:33.410 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-03-21T10:35:33.411 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-03-21T10:35:33.411 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-03-21T10:35:33.411 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-03-21T10:35:33.411 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-03-21T10:35:33.411 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-03-21T10:35:33.411 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-03-21T10:35:33.411 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T10:35:33.411 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:33.411 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:33.411 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:33.411 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:33.411 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_file": { 2024-03-21T10:35:33.411 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_file", 2024-03-21T10:35:33.411 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T10:35:33.411 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:33.412 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T10:35:33.412 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-03-21T10:35:33.412 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:33.412 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:33.412 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:33.412 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T10:35:33.412 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:33.412 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:33.412 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:33.412 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:33.412 INFO:teuthology.orchestra.run.smithi047.stdout: "min_score": { 2024-03-21T10:35:33.412 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "min_score", 2024-03-21T10:35:33.412 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "float", 2024-03-21T10:35:33.412 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:33.413 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T10:35:33.413 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "0", 2024-03-21T10:35:33.413 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:33.413 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:33.413 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:33.413 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-03-21T10:35:33.413 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:33.413 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:33.413 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:33.413 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:33.413 INFO:teuthology.orchestra.run.smithi047.stdout: "mode": { 2024-03-21T10:35:33.413 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "mode", 2024-03-21T10:35:33.413 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T10:35:33.413 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:33.413 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T10:35:33.413 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "upmap", 2024-03-21T10:35:33.414 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:33.414 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:33.414 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-03-21T10:35:33.414 INFO:teuthology.orchestra.run.smithi047.stdout: "crush-compat", 2024-03-21T10:35:33.414 INFO:teuthology.orchestra.run.smithi047.stdout: "none", 2024-03-21T10:35:33.414 INFO:teuthology.orchestra.run.smithi047.stdout: "read", 2024-03-21T10:35:33.414 INFO:teuthology.orchestra.run.smithi047.stdout: "upmap", 2024-03-21T10:35:33.414 INFO:teuthology.orchestra.run.smithi047.stdout: "upmap-read" 2024-03-21T10:35:33.414 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-03-21T10:35:33.414 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Balancer mode", 2024-03-21T10:35:33.414 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:33.414 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:33.414 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:33.414 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:33.414 INFO:teuthology.orchestra.run.smithi047.stdout: "pool_ids": { 2024-03-21T10:35:33.415 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "pool_ids", 2024-03-21T10:35:33.415 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T10:35:33.415 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:33.415 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T10:35:33.415 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-03-21T10:35:33.415 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:33.415 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:33.415 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:33.415 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-03-21T10:35:33.415 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:33.415 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:33.415 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:33.415 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:33.415 INFO:teuthology.orchestra.run.smithi047.stdout: "sleep_interval": { 2024-03-21T10:35:33.415 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "sleep_interval", 2024-03-21T10:35:33.415 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "secs", 2024-03-21T10:35:33.416 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:33.416 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T10:35:33.416 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "60", 2024-03-21T10:35:33.416 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:33.416 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:33.416 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:33.416 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-03-21T10:35:33.416 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:33.416 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:33.416 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:33.416 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:33.416 INFO:teuthology.orchestra.run.smithi047.stdout: "upmap_max_deviation": { 2024-03-21T10:35:33.416 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "upmap_max_deviation", 2024-03-21T10:35:33.416 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "int", 2024-03-21T10:35:33.416 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:33.416 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T10:35:33.416 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "5", 2024-03-21T10:35:33.417 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "1", 2024-03-21T10:35:33.417 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:33.417 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:33.417 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "deviation below which no optimization is attempted", 2024-03-21T10:35:33.417 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-03-21T10:35:33.417 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:33.417 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:33.417 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:33.417 INFO:teuthology.orchestra.run.smithi047.stdout: "upmap_max_optimizations": { 2024-03-21T10:35:33.417 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "upmap_max_optimizations", 2024-03-21T10:35:33.417 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "uint", 2024-03-21T10:35:33.417 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:33.417 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T10:35:33.417 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "10", 2024-03-21T10:35:33.417 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:33.417 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:33.418 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:33.418 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-03-21T10:35:33.418 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:33.418 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:33.418 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:33.418 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-21T10:35:33.418 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-21T10:35:33.418 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:33.418 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-03-21T10:35:33.418 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "cephadm", 2024-03-21T10:35:33.418 INFO:teuthology.orchestra.run.smithi047.stdout: "can_run": true, 2024-03-21T10:35:33.418 INFO:teuthology.orchestra.run.smithi047.stdout: "error_string": "", 2024-03-21T10:35:33.418 INFO:teuthology.orchestra.run.smithi047.stdout: "module_options": { 2024-03-21T10:35:33.418 INFO:teuthology.orchestra.run.smithi047.stdout: "agent_down_multiplier": { 2024-03-21T10:35:33.419 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "agent_down_multiplier", 2024-03-21T10:35:33.419 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "float", 2024-03-21T10:35:33.419 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:33.419 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T10:35:33.419 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "3.0", 2024-03-21T10:35:33.419 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:33.419 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:33.419 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:33.419 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-03-21T10:35:33.419 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:33.419 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:33.419 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:33.419 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:33.419 INFO:teuthology.orchestra.run.smithi047.stdout: "agent_refresh_rate": { 2024-03-21T10:35:33.419 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "agent_refresh_rate", 2024-03-21T10:35:33.419 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "secs", 2024-03-21T10:35:33.420 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:33.420 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T10:35:33.420 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "20", 2024-03-21T10:35:33.420 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:33.420 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:33.420 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:33.420 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-03-21T10:35:33.420 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:33.420 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:33.420 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:33.420 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:33.420 INFO:teuthology.orchestra.run.smithi047.stdout: "agent_starting_port": { 2024-03-21T10:35:33.420 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "agent_starting_port", 2024-03-21T10:35:33.420 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "int", 2024-03-21T10:35:33.420 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:33.420 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T10:35:33.421 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "4721", 2024-03-21T10:35:33.421 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:33.421 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:33.421 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:33.421 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-03-21T10:35:33.421 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:33.421 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:33.421 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:33.421 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:33.421 INFO:teuthology.orchestra.run.smithi047.stdout: "allow_ptrace": { 2024-03-21T10:35:33.421 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "allow_ptrace", 2024-03-21T10:35:33.421 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T10:35:33.421 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:33.421 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T10:35:33.421 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-03-21T10:35:33.422 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:33.422 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:33.422 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:33.422 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-03-21T10:35:33.422 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-03-21T10:35:33.422 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:33.422 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:33.422 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:33.422 INFO:teuthology.orchestra.run.smithi047.stdout: "autotune_interval": { 2024-03-21T10:35:33.422 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "autotune_interval", 2024-03-21T10:35:33.422 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "secs", 2024-03-21T10:35:33.422 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:33.423 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T10:35:33.423 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "600", 2024-03-21T10:35:33.423 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:33.423 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:33.423 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:33.423 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "how frequently to autotune daemon memory", 2024-03-21T10:35:33.423 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:33.423 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:33.423 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:33.423 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:33.423 INFO:teuthology.orchestra.run.smithi047.stdout: "autotune_memory_target_ratio": { 2024-03-21T10:35:33.423 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "autotune_memory_target_ratio", 2024-03-21T10:35:33.423 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "float", 2024-03-21T10:35:33.423 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:33.424 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T10:35:33.424 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "0.7", 2024-03-21T10:35:33.424 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:33.424 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:33.424 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:33.424 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-03-21T10:35:33.424 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:33.424 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:33.424 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:33.424 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:33.424 INFO:teuthology.orchestra.run.smithi047.stdout: "cephadm_log_destination": { 2024-03-21T10:35:33.424 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "cephadm_log_destination", 2024-03-21T10:35:33.424 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T10:35:33.424 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:33.424 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T10:35:33.424 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-03-21T10:35:33.425 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:33.425 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:33.425 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-03-21T10:35:33.425 INFO:teuthology.orchestra.run.smithi047.stdout: "file", 2024-03-21T10:35:33.425 INFO:teuthology.orchestra.run.smithi047.stdout: "file,syslog", 2024-03-21T10:35:33.425 INFO:teuthology.orchestra.run.smithi047.stdout: "syslog" 2024-03-21T10:35:33.425 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-03-21T10:35:33.425 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-03-21T10:35:33.425 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:33.425 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:33.425 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:33.425 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:33.425 INFO:teuthology.orchestra.run.smithi047.stdout: "cgroups_split": { 2024-03-21T10:35:33.425 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "cgroups_split", 2024-03-21T10:35:33.425 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T10:35:33.425 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:33.425 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T10:35:33.426 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "True", 2024-03-21T10:35:33.426 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:33.426 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:33.426 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:33.426 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-03-21T10:35:33.426 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:33.426 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:33.426 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:33.426 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:33.426 INFO:teuthology.orchestra.run.smithi047.stdout: "config_checks_enabled": { 2024-03-21T10:35:33.426 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "config_checks_enabled", 2024-03-21T10:35:33.426 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T10:35:33.426 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:33.427 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T10:35:33.427 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-03-21T10:35:33.427 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:33.427 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:33.427 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:33.427 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-03-21T10:35:33.427 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:33.427 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:33.427 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:33.427 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:33.427 INFO:teuthology.orchestra.run.smithi047.stdout: "config_dashboard": { 2024-03-21T10:35:33.427 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "config_dashboard", 2024-03-21T10:35:33.427 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T10:35:33.427 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:33.427 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T10:35:33.428 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "True", 2024-03-21T10:35:33.428 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:33.428 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:33.428 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:33.428 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-03-21T10:35:33.428 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:33.428 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:33.428 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:33.428 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:33.428 INFO:teuthology.orchestra.run.smithi047.stdout: "container_image_alertmanager": { 2024-03-21T10:35:33.428 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "container_image_alertmanager", 2024-03-21T10:35:33.428 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T10:35:33.428 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:33.428 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T10:35:33.428 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-03-21T10:35:33.428 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:33.429 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:33.429 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:33.429 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Prometheus container image", 2024-03-21T10:35:33.429 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:33.429 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:33.429 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:33.429 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:33.429 INFO:teuthology.orchestra.run.smithi047.stdout: "container_image_base": { 2024-03-21T10:35:33.429 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "container_image_base", 2024-03-21T10:35:33.429 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T10:35:33.429 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:33.429 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T10:35:33.429 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "quay.io/ceph/ceph", 2024-03-21T10:35:33.429 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:33.429 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:33.429 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:33.430 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Container image name, without the tag", 2024-03-21T10:35:33.430 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:33.430 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:33.430 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:33.430 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:33.430 INFO:teuthology.orchestra.run.smithi047.stdout: "container_image_elasticsearch": { 2024-03-21T10:35:33.430 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "container_image_elasticsearch", 2024-03-21T10:35:33.430 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T10:35:33.430 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:33.430 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T10:35:33.430 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-03-21T10:35:33.430 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:33.430 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:33.430 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:33.431 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "elasticsearch container image", 2024-03-21T10:35:33.431 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:33.431 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:33.431 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:33.431 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:33.431 INFO:teuthology.orchestra.run.smithi047.stdout: "container_image_grafana": { 2024-03-21T10:35:33.431 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "container_image_grafana", 2024-03-21T10:35:33.431 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T10:35:33.431 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:33.431 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T10:35:33.431 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "quay.io/ceph/grafana:9.4.12", 2024-03-21T10:35:33.431 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:33.431 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:33.431 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:33.431 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Prometheus container image", 2024-03-21T10:35:33.432 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:33.432 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:33.432 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:33.432 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:33.432 INFO:teuthology.orchestra.run.smithi047.stdout: "container_image_haproxy": { 2024-03-21T10:35:33.432 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "container_image_haproxy", 2024-03-21T10:35:33.432 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T10:35:33.432 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:33.432 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T10:35:33.432 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-03-21T10:35:33.432 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:33.432 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:33.432 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:33.432 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "HAproxy container image", 2024-03-21T10:35:33.432 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:33.432 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:33.433 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:33.433 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:33.433 INFO:teuthology.orchestra.run.smithi047.stdout: "container_image_jaeger_agent": { 2024-03-21T10:35:33.433 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "container_image_jaeger_agent", 2024-03-21T10:35:33.433 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T10:35:33.433 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:33.433 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T10:35:33.433 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-03-21T10:35:33.433 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:33.433 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:33.433 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:33.433 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Jaeger agent container image", 2024-03-21T10:35:33.433 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:33.434 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:33.434 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:33.434 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:33.434 INFO:teuthology.orchestra.run.smithi047.stdout: "container_image_jaeger_collector": { 2024-03-21T10:35:33.434 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "container_image_jaeger_collector", 2024-03-21T10:35:33.434 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T10:35:33.434 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:33.434 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T10:35:33.434 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-03-21T10:35:33.434 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:33.434 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:33.435 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:33.435 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Jaeger collector container image", 2024-03-21T10:35:33.435 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:33.435 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:33.435 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:33.435 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:33.435 INFO:teuthology.orchestra.run.smithi047.stdout: "container_image_jaeger_query": { 2024-03-21T10:35:33.435 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "container_image_jaeger_query", 2024-03-21T10:35:33.435 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T10:35:33.435 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:33.435 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T10:35:33.435 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-03-21T10:35:33.435 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:33.435 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:33.435 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:33.436 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Jaeger query container image", 2024-03-21T10:35:33.436 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:33.436 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:33.436 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:33.436 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:33.436 INFO:teuthology.orchestra.run.smithi047.stdout: "container_image_keepalived": { 2024-03-21T10:35:33.436 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "container_image_keepalived", 2024-03-21T10:35:33.436 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T10:35:33.436 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:33.436 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T10:35:33.436 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-03-21T10:35:33.436 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:33.436 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:33.436 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:33.436 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Keepalived container image", 2024-03-21T10:35:33.437 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:33.437 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:33.437 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:33.437 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:33.437 INFO:teuthology.orchestra.run.smithi047.stdout: "container_image_loki": { 2024-03-21T10:35:33.437 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "container_image_loki", 2024-03-21T10:35:33.437 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T10:35:33.437 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:33.437 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T10:35:33.437 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2024-03-21T10:35:33.437 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:33.437 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:33.437 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:33.437 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Loki container image", 2024-03-21T10:35:33.438 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:33.438 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:33.438 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:33.438 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:33.438 INFO:teuthology.orchestra.run.smithi047.stdout: "container_image_node_exporter": { 2024-03-21T10:35:33.438 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "container_image_node_exporter", 2024-03-21T10:35:33.438 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T10:35:33.438 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:33.438 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T10:35:33.438 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-03-21T10:35:33.438 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:33.438 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:33.438 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:33.438 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Prometheus container image", 2024-03-21T10:35:33.439 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:33.439 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:33.439 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:33.439 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:33.439 INFO:teuthology.orchestra.run.smithi047.stdout: "container_image_nvmeof": { 2024-03-21T10:35:33.439 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "container_image_nvmeof", 2024-03-21T10:35:33.439 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T10:35:33.439 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:33.439 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T10:35:33.439 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "quay.io/ceph/nvmeof:1.0.0", 2024-03-21T10:35:33.439 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:33.439 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:33.439 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:33.439 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Nvme-of container image", 2024-03-21T10:35:33.439 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:33.440 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:33.440 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:33.440 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:33.440 INFO:teuthology.orchestra.run.smithi047.stdout: "container_image_prometheus": { 2024-03-21T10:35:33.440 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "container_image_prometheus", 2024-03-21T10:35:33.440 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T10:35:33.440 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:33.440 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T10:35:33.440 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-03-21T10:35:33.440 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:33.440 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:33.440 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:33.440 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Prometheus container image", 2024-03-21T10:35:33.440 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:33.440 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:33.441 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:33.441 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:33.441 INFO:teuthology.orchestra.run.smithi047.stdout: "container_image_promtail": { 2024-03-21T10:35:33.441 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "container_image_promtail", 2024-03-21T10:35:33.441 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T10:35:33.441 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:33.441 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T10:35:33.441 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2024-03-21T10:35:33.441 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:33.441 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:33.441 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:33.441 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Promtail container image", 2024-03-21T10:35:33.441 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:33.441 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:33.441 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:33.441 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:33.442 INFO:teuthology.orchestra.run.smithi047.stdout: "container_image_snmp_gateway": { 2024-03-21T10:35:33.442 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "container_image_snmp_gateway", 2024-03-21T10:35:33.442 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T10:35:33.442 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:33.442 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T10:35:33.442 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-03-21T10:35:33.442 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:33.442 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:33.442 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:33.442 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "SNMP Gateway container image", 2024-03-21T10:35:33.442 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:33.442 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:33.443 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:33.443 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:33.443 INFO:teuthology.orchestra.run.smithi047.stdout: "container_init": { 2024-03-21T10:35:33.443 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "container_init", 2024-03-21T10:35:33.443 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T10:35:33.443 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:33.443 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T10:35:33.443 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "True", 2024-03-21T10:35:33.443 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:33.443 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:33.443 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:33.443 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Run podman/docker with `--init`", 2024-03-21T10:35:33.443 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:33.443 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:33.443 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:33.443 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:33.444 INFO:teuthology.orchestra.run.smithi047.stdout: "daemon_cache_timeout": { 2024-03-21T10:35:33.444 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "daemon_cache_timeout", 2024-03-21T10:35:33.444 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "secs", 2024-03-21T10:35:33.444 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:33.444 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T10:35:33.444 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "600", 2024-03-21T10:35:33.444 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:33.444 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:33.444 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:33.444 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-03-21T10:35:33.444 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:33.444 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:33.444 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:33.444 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:33.445 INFO:teuthology.orchestra.run.smithi047.stdout: "default_cephadm_command_timeout": { 2024-03-21T10:35:33.445 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "default_cephadm_command_timeout", 2024-03-21T10:35:33.445 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "int", 2024-03-21T10:35:33.445 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:33.445 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T10:35:33.445 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "900", 2024-03-21T10:35:33.445 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:33.445 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:33.445 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:33.445 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-03-21T10:35:33.445 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:33.445 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:33.445 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:33.445 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:33.445 INFO:teuthology.orchestra.run.smithi047.stdout: "default_registry": { 2024-03-21T10:35:33.446 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "default_registry", 2024-03-21T10:35:33.446 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T10:35:33.446 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:33.446 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T10:35:33.446 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "docker.io", 2024-03-21T10:35:33.446 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:33.446 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:33.446 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:33.446 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-03-21T10:35:33.446 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:33.446 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:33.446 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:33.446 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:33.447 INFO:teuthology.orchestra.run.smithi047.stdout: "device_cache_timeout": { 2024-03-21T10:35:33.447 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "device_cache_timeout", 2024-03-21T10:35:33.447 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "secs", 2024-03-21T10:35:33.447 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:33.447 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T10:35:33.447 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "1800", 2024-03-21T10:35:33.447 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:33.447 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:33.447 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:33.447 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "seconds to cache device inventory", 2024-03-21T10:35:33.447 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:33.447 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:33.447 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:33.448 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:33.448 INFO:teuthology.orchestra.run.smithi047.stdout: "device_enhanced_scan": { 2024-03-21T10:35:33.448 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "device_enhanced_scan", 2024-03-21T10:35:33.448 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T10:35:33.448 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:33.448 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T10:35:33.448 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-03-21T10:35:33.448 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:33.448 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:33.448 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:33.448 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Use libstoragemgmt during device scans", 2024-03-21T10:35:33.448 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:33.448 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:33.448 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:33.448 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:33.449 INFO:teuthology.orchestra.run.smithi047.stdout: "facts_cache_timeout": { 2024-03-21T10:35:33.449 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "facts_cache_timeout", 2024-03-21T10:35:33.449 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "secs", 2024-03-21T10:35:33.449 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:33.449 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T10:35:33.449 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "60", 2024-03-21T10:35:33.449 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:33.449 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:33.449 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:33.449 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "seconds to cache host facts data", 2024-03-21T10:35:33.449 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:33.449 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:33.449 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:33.449 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:33.450 INFO:teuthology.orchestra.run.smithi047.stdout: "grafana_dashboards_path": { 2024-03-21T10:35:33.450 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "grafana_dashboards_path", 2024-03-21T10:35:33.450 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T10:35:33.450 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:33.450 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T10:35:33.450 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-03-21T10:35:33.450 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:33.450 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:33.450 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:33.450 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-03-21T10:35:33.450 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:33.450 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:33.450 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:33.451 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:33.451 INFO:teuthology.orchestra.run.smithi047.stdout: "host_check_interval": { 2024-03-21T10:35:33.451 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "host_check_interval", 2024-03-21T10:35:33.451 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "secs", 2024-03-21T10:35:33.451 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:33.451 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T10:35:33.451 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "600", 2024-03-21T10:35:33.451 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:33.451 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:33.451 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:33.451 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "how frequently to perform a host check", 2024-03-21T10:35:33.451 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:33.451 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:33.451 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:33.452 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:33.452 INFO:teuthology.orchestra.run.smithi047.stdout: "hw_monitoring": { 2024-03-21T10:35:33.452 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "hw_monitoring", 2024-03-21T10:35:33.452 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T10:35:33.452 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:33.452 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T10:35:33.452 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-03-21T10:35:33.452 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:33.452 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:33.452 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:33.452 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-03-21T10:35:33.452 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:33.452 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:33.452 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:33.452 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:33.453 INFO:teuthology.orchestra.run.smithi047.stdout: "inventory_list_all": { 2024-03-21T10:35:33.453 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "inventory_list_all", 2024-03-21T10:35:33.453 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T10:35:33.453 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:33.453 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T10:35:33.453 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-03-21T10:35:33.453 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:33.453 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:33.453 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:33.453 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-03-21T10:35:33.453 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:33.453 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:33.453 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:33.454 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:33.454 INFO:teuthology.orchestra.run.smithi047.stdout: "log_level": { 2024-03-21T10:35:33.454 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_level", 2024-03-21T10:35:33.454 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T10:35:33.454 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:33.454 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T10:35:33.454 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-03-21T10:35:33.454 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:33.454 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:33.454 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-03-21T10:35:33.454 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-03-21T10:35:33.454 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-03-21T10:35:33.454 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-03-21T10:35:33.455 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-03-21T10:35:33.455 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-03-21T10:35:33.455 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-03-21T10:35:33.455 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-03-21T10:35:33.455 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T10:35:33.455 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:33.455 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:33.455 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:33.455 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:33.455 INFO:teuthology.orchestra.run.smithi047.stdout: "log_refresh_metadata": { 2024-03-21T10:35:33.455 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_refresh_metadata", 2024-03-21T10:35:33.455 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T10:35:33.455 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:33.455 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T10:35:33.456 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-03-21T10:35:33.456 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:33.456 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:33.456 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:33.456 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-03-21T10:35:33.456 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:33.456 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:33.456 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:33.456 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:33.456 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster": { 2024-03-21T10:35:33.456 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster", 2024-03-21T10:35:33.456 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T10:35:33.456 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:33.456 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T10:35:33.457 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "True", 2024-03-21T10:35:33.457 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:33.457 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:33.457 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:33.457 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-03-21T10:35:33.457 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:33.457 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:33.457 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:33.457 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:33.457 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster_level": { 2024-03-21T10:35:33.457 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster_level", 2024-03-21T10:35:33.457 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T10:35:33.458 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:33.458 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T10:35:33.458 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "info", 2024-03-21T10:35:33.458 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:33.458 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:33.458 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-03-21T10:35:33.458 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-03-21T10:35:33.458 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-03-21T10:35:33.458 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-03-21T10:35:33.458 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-03-21T10:35:33.458 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-03-21T10:35:33.458 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-03-21T10:35:33.459 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-03-21T10:35:33.459 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T10:35:33.459 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:33.459 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:33.459 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:33.459 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:33.459 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_file": { 2024-03-21T10:35:33.459 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_file", 2024-03-21T10:35:33.459 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T10:35:33.459 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:33.459 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T10:35:33.459 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-03-21T10:35:33.459 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:33.460 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:33.460 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:33.460 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T10:35:33.460 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:33.460 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:33.460 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:33.460 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:33.460 INFO:teuthology.orchestra.run.smithi047.stdout: "manage_etc_ceph_ceph_conf": { 2024-03-21T10:35:33.460 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-03-21T10:35:33.460 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T10:35:33.460 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:33.460 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T10:35:33.460 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-03-21T10:35:33.460 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:33.461 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:33.461 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:33.461 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-03-21T10:35:33.461 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:33.461 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:33.461 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:33.461 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:33.461 INFO:teuthology.orchestra.run.smithi047.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-03-21T10:35:33.461 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-03-21T10:35:33.461 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T10:35:33.461 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:33.461 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T10:35:33.461 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "*", 2024-03-21T10:35:33.462 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:33.462 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:33.462 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:33.462 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-03-21T10:35:33.462 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:33.462 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:33.462 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:33.462 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:33.462 INFO:teuthology.orchestra.run.smithi047.stdout: "max_count_per_host": { 2024-03-21T10:35:33.462 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "max_count_per_host", 2024-03-21T10:35:33.462 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "int", 2024-03-21T10:35:33.462 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:33.462 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T10:35:33.463 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "10", 2024-03-21T10:35:33.463 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:33.463 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:33.463 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:33.463 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "max number of daemons per service per host", 2024-03-21T10:35:33.463 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:33.463 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:33.463 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:33.463 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:33.463 INFO:teuthology.orchestra.run.smithi047.stdout: "max_osd_draining_count": { 2024-03-21T10:35:33.463 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "max_osd_draining_count", 2024-03-21T10:35:33.463 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "int", 2024-03-21T10:35:33.463 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:33.463 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T10:35:33.464 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "10", 2024-03-21T10:35:33.464 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:33.464 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:33.464 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:33.464 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-03-21T10:35:33.464 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:33.464 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:33.464 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:33.464 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:33.464 INFO:teuthology.orchestra.run.smithi047.stdout: "migration_current": { 2024-03-21T10:35:33.464 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "migration_current", 2024-03-21T10:35:33.464 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "int", 2024-03-21T10:35:33.464 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:33.464 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T10:35:33.465 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-03-21T10:35:33.465 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:33.465 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:33.465 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:33.465 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "internal - do not modify", 2024-03-21T10:35:33.465 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:33.465 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:33.465 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:33.465 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:33.465 INFO:teuthology.orchestra.run.smithi047.stdout: "mode": { 2024-03-21T10:35:33.465 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "mode", 2024-03-21T10:35:33.465 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T10:35:33.465 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:33.466 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T10:35:33.466 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "root", 2024-03-21T10:35:33.466 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:33.466 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:33.466 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-03-21T10:35:33.466 INFO:teuthology.orchestra.run.smithi047.stdout: "cephadm-package", 2024-03-21T10:35:33.466 INFO:teuthology.orchestra.run.smithi047.stdout: "root" 2024-03-21T10:35:33.466 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-03-21T10:35:33.466 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "mode for remote execution of cephadm", 2024-03-21T10:35:33.466 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:33.466 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:33.466 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:33.466 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:33.466 INFO:teuthology.orchestra.run.smithi047.stdout: "oob_default_addr": { 2024-03-21T10:35:33.467 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "oob_default_addr", 2024-03-21T10:35:33.467 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T10:35:33.467 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:33.467 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T10:35:33.467 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "169.254.1.1", 2024-03-21T10:35:33.467 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:33.467 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:33.467 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:33.467 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Default address for RedFish API (oob management).", 2024-03-21T10:35:33.467 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:33.467 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:33.467 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:33.467 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:33.467 INFO:teuthology.orchestra.run.smithi047.stdout: "prometheus_alerts_path": { 2024-03-21T10:35:33.467 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "prometheus_alerts_path", 2024-03-21T10:35:33.467 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T10:35:33.468 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:33.468 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T10:35:33.468 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-03-21T10:35:33.468 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:33.468 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:33.468 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:33.468 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-03-21T10:35:33.468 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:33.468 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:33.468 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:33.468 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:33.468 INFO:teuthology.orchestra.run.smithi047.stdout: "registry_insecure": { 2024-03-21T10:35:33.468 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "registry_insecure", 2024-03-21T10:35:33.468 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T10:35:33.468 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:33.468 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T10:35:33.469 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-03-21T10:35:33.469 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:33.469 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:33.469 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:33.469 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-03-21T10:35:33.469 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:33.469 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:33.469 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:33.469 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:33.469 INFO:teuthology.orchestra.run.smithi047.stdout: "registry_password": { 2024-03-21T10:35:33.469 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "registry_password", 2024-03-21T10:35:33.469 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T10:35:33.469 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:33.469 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T10:35:33.469 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-03-21T10:35:33.470 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:33.470 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:33.470 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:33.470 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-03-21T10:35:33.470 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:33.470 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:33.470 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:33.470 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:33.470 INFO:teuthology.orchestra.run.smithi047.stdout: "registry_url": { 2024-03-21T10:35:33.470 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "registry_url", 2024-03-21T10:35:33.470 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T10:35:33.470 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:33.470 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T10:35:33.470 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-03-21T10:35:33.470 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:33.471 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:33.471 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:33.471 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-03-21T10:35:33.471 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:33.471 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:33.471 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:33.471 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:33.471 INFO:teuthology.orchestra.run.smithi047.stdout: "registry_username": { 2024-03-21T10:35:33.471 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "registry_username", 2024-03-21T10:35:33.471 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T10:35:33.471 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:33.471 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T10:35:33.471 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-03-21T10:35:33.471 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:33.471 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:33.472 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:33.472 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-03-21T10:35:33.472 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:33.472 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:33.472 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:33.472 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:33.472 INFO:teuthology.orchestra.run.smithi047.stdout: "secure_monitoring_stack": { 2024-03-21T10:35:33.472 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "secure_monitoring_stack", 2024-03-21T10:35:33.472 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T10:35:33.472 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:33.472 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T10:35:33.472 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-03-21T10:35:33.472 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:33.472 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:33.473 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:33.473 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-03-21T10:35:33.473 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:33.473 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:33.473 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:33.473 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:33.473 INFO:teuthology.orchestra.run.smithi047.stdout: "service_discovery_port": { 2024-03-21T10:35:33.473 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "service_discovery_port", 2024-03-21T10:35:33.473 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "int", 2024-03-21T10:35:33.473 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:33.473 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T10:35:33.473 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "8765", 2024-03-21T10:35:33.473 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:33.473 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:33.473 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:33.474 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "cephadm service discovery port", 2024-03-21T10:35:33.474 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:33.474 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:33.474 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:33.474 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:33.474 INFO:teuthology.orchestra.run.smithi047.stdout: "ssh_config_file": { 2024-03-21T10:35:33.474 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "ssh_config_file", 2024-03-21T10:35:33.474 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T10:35:33.474 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:33.474 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T10:35:33.474 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-03-21T10:35:33.474 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:33.474 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:33.474 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:33.475 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-03-21T10:35:33.475 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:33.475 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:33.475 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:33.475 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:33.475 INFO:teuthology.orchestra.run.smithi047.stdout: "use_agent": { 2024-03-21T10:35:33.475 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "use_agent", 2024-03-21T10:35:33.475 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T10:35:33.475 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:33.475 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T10:35:33.475 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-03-21T10:35:33.475 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:33.475 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:33.475 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:33.476 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-03-21T10:35:33.476 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:33.476 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:33.476 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:33.476 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:33.476 INFO:teuthology.orchestra.run.smithi047.stdout: "use_repo_digest": { 2024-03-21T10:35:33.476 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "use_repo_digest", 2024-03-21T10:35:33.476 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T10:35:33.476 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:33.476 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T10:35:33.476 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "True", 2024-03-21T10:35:33.476 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:33.476 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:33.477 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:33.477 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-03-21T10:35:33.477 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:33.477 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:33.477 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:33.477 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:33.477 INFO:teuthology.orchestra.run.smithi047.stdout: "warn_on_failed_host_check": { 2024-03-21T10:35:33.477 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "warn_on_failed_host_check", 2024-03-21T10:35:33.477 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T10:35:33.477 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:33.477 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T10:35:33.477 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "True", 2024-03-21T10:35:33.477 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:33.477 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:33.477 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:33.478 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "raise a health warning if the host check fails", 2024-03-21T10:35:33.478 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:33.478 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:33.478 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:33.478 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:33.478 INFO:teuthology.orchestra.run.smithi047.stdout: "warn_on_stray_daemons": { 2024-03-21T10:35:33.478 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "warn_on_stray_daemons", 2024-03-21T10:35:33.478 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T10:35:33.478 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:33.478 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T10:35:33.478 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "True", 2024-03-21T10:35:33.478 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:33.478 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:33.478 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:33.479 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-03-21T10:35:33.479 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:33.479 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:33.479 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:33.479 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:33.479 INFO:teuthology.orchestra.run.smithi047.stdout: "warn_on_stray_hosts": { 2024-03-21T10:35:33.479 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "warn_on_stray_hosts", 2024-03-21T10:35:33.479 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T10:35:33.479 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:33.479 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T10:35:33.479 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "True", 2024-03-21T10:35:33.479 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:33.479 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:33.479 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:33.480 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-03-21T10:35:33.480 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:33.480 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:33.480 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:33.480 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-21T10:35:33.480 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-21T10:35:33.480 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:33.480 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-03-21T10:35:33.480 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "crash", 2024-03-21T10:35:33.480 INFO:teuthology.orchestra.run.smithi047.stdout: "can_run": true, 2024-03-21T10:35:33.480 INFO:teuthology.orchestra.run.smithi047.stdout: "error_string": "", 2024-03-21T10:35:33.480 INFO:teuthology.orchestra.run.smithi047.stdout: "module_options": { 2024-03-21T10:35:33.480 INFO:teuthology.orchestra.run.smithi047.stdout: "log_level": { 2024-03-21T10:35:33.480 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_level", 2024-03-21T10:35:33.481 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T10:35:33.481 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:33.481 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T10:35:33.481 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-03-21T10:35:33.481 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:33.481 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:33.481 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-03-21T10:35:33.481 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-03-21T10:35:33.481 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-03-21T10:35:33.481 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-03-21T10:35:33.481 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-03-21T10:35:33.481 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-03-21T10:35:33.481 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-03-21T10:35:33.482 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-03-21T10:35:33.482 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T10:35:33.482 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:33.482 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:33.482 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:33.482 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:33.482 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster": { 2024-03-21T10:35:33.482 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster", 2024-03-21T10:35:33.482 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T10:35:33.482 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:33.482 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T10:35:33.482 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-03-21T10:35:33.482 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:33.482 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:33.483 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:33.483 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T10:35:33.483 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:33.483 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:33.483 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:33.483 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:33.483 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster_level": { 2024-03-21T10:35:33.483 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster_level", 2024-03-21T10:35:33.483 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T10:35:33.483 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:33.483 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T10:35:33.483 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "info", 2024-03-21T10:35:33.483 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:33.483 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:33.483 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-03-21T10:35:33.484 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-03-21T10:35:33.484 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-03-21T10:35:33.484 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-03-21T10:35:33.484 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-03-21T10:35:33.484 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-03-21T10:35:33.484 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-03-21T10:35:33.484 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-03-21T10:35:33.484 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T10:35:33.484 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:33.484 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:33.484 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:33.484 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:33.484 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_file": { 2024-03-21T10:35:33.484 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_file", 2024-03-21T10:35:33.485 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T10:35:33.485 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:33.485 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T10:35:33.485 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-03-21T10:35:33.485 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:33.485 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:33.485 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:33.485 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T10:35:33.485 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:33.485 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:33.485 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:33.485 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:33.485 INFO:teuthology.orchestra.run.smithi047.stdout: "retain_interval": { 2024-03-21T10:35:33.485 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "retain_interval", 2024-03-21T10:35:33.485 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "secs", 2024-03-21T10:35:33.486 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:33.486 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T10:35:33.486 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "31536000", 2024-03-21T10:35:33.486 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:33.486 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:33.486 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:33.486 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "how long to retain crashes before pruning them", 2024-03-21T10:35:33.486 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:33.486 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:33.486 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:33.486 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:33.486 INFO:teuthology.orchestra.run.smithi047.stdout: "warn_recent_interval": { 2024-03-21T10:35:33.486 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "warn_recent_interval", 2024-03-21T10:35:33.486 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "secs", 2024-03-21T10:35:33.487 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:33.487 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T10:35:33.487 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "1209600", 2024-03-21T10:35:33.487 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:33.487 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:33.487 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:33.487 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "time interval in which to warn about recent crashes", 2024-03-21T10:35:33.487 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:33.487 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:33.487 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:33.487 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-21T10:35:33.487 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-21T10:35:33.487 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:33.487 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-03-21T10:35:33.488 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "dashboard", 2024-03-21T10:35:33.488 INFO:teuthology.orchestra.run.smithi047.stdout: "can_run": true, 2024-03-21T10:35:33.488 INFO:teuthology.orchestra.run.smithi047.stdout: "error_string": "", 2024-03-21T10:35:33.488 INFO:teuthology.orchestra.run.smithi047.stdout: "module_options": { 2024-03-21T10:35:33.488 INFO:teuthology.orchestra.run.smithi047.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-03-21T10:35:33.488 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-03-21T10:35:33.488 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "int", 2024-03-21T10:35:33.488 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:33.488 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T10:35:33.488 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "10", 2024-03-21T10:35:33.488 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:33.488 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:33.488 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:33.488 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T10:35:33.488 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:33.489 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:33.489 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:33.489 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:33.489 INFO:teuthology.orchestra.run.smithi047.stdout: "ALERTMANAGER_API_HOST": { 2024-03-21T10:35:33.489 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "ALERTMANAGER_API_HOST", 2024-03-21T10:35:33.489 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T10:35:33.489 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:33.489 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T10:35:33.489 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-03-21T10:35:33.489 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:33.489 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:33.489 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:33.489 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T10:35:33.489 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:33.490 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:33.490 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:33.490 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:33.490 INFO:teuthology.orchestra.run.smithi047.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-03-21T10:35:33.490 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-03-21T10:35:33.490 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T10:35:33.490 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:33.490 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T10:35:33.490 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "True", 2024-03-21T10:35:33.490 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:33.490 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:33.490 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:33.490 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T10:35:33.490 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:33.491 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:33.491 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:33.491 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:33.491 INFO:teuthology.orchestra.run.smithi047.stdout: "AUDIT_API_ENABLED": { 2024-03-21T10:35:33.491 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "AUDIT_API_ENABLED", 2024-03-21T10:35:33.491 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T10:35:33.491 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:33.491 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T10:35:33.491 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-03-21T10:35:33.491 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:33.491 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:33.491 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:33.491 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T10:35:33.492 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:33.492 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:33.492 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:33.492 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:33.492 INFO:teuthology.orchestra.run.smithi047.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-03-21T10:35:33.492 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-03-21T10:35:33.492 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T10:35:33.492 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:33.492 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T10:35:33.492 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "True", 2024-03-21T10:35:33.492 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:33.492 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:33.492 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:33.493 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T10:35:33.493 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:33.493 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:33.493 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:33.493 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:33.493 INFO:teuthology.orchestra.run.smithi047.stdout: "ENABLE_BROWSABLE_API": { 2024-03-21T10:35:33.493 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "ENABLE_BROWSABLE_API", 2024-03-21T10:35:33.493 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T10:35:33.493 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:33.493 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T10:35:33.493 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "True", 2024-03-21T10:35:33.493 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:33.493 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:33.493 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:33.493 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T10:35:33.494 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:33.494 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:33.494 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:33.494 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:33.494 INFO:teuthology.orchestra.run.smithi047.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-03-21T10:35:33.494 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-03-21T10:35:33.494 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T10:35:33.494 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:33.494 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T10:35:33.494 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "True", 2024-03-21T10:35:33.494 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:33.494 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:33.494 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:33.494 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T10:35:33.495 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:33.495 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:33.495 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:33.495 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:33.495 INFO:teuthology.orchestra.run.smithi047.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-03-21T10:35:33.495 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-03-21T10:35:33.495 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T10:35:33.495 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:33.495 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T10:35:33.495 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "True", 2024-03-21T10:35:33.495 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:33.495 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:33.495 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:33.495 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T10:35:33.496 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:33.496 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:33.496 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:33.496 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:33.496 INFO:teuthology.orchestra.run.smithi047.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-03-21T10:35:33.496 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-03-21T10:35:33.496 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T10:35:33.496 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:33.496 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T10:35:33.496 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "True", 2024-03-21T10:35:33.496 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:33.496 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:33.496 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:33.496 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T10:35:33.497 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:33.497 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:33.497 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:33.497 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:33.497 INFO:teuthology.orchestra.run.smithi047.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-03-21T10:35:33.497 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-03-21T10:35:33.497 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T10:35:33.497 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:33.497 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T10:35:33.497 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "True", 2024-03-21T10:35:33.497 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:33.497 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:33.497 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:33.497 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T10:35:33.497 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:33.497 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:33.497 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:33.498 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:33.498 INFO:teuthology.orchestra.run.smithi047.stdout: "FEATURE_TOGGLE_NFS": { 2024-03-21T10:35:33.498 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-03-21T10:35:33.498 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T10:35:33.498 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:33.498 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T10:35:33.498 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "True", 2024-03-21T10:35:33.498 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:33.498 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:33.498 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:33.498 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T10:35:33.498 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:33.498 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:33.498 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:33.499 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:33.499 INFO:teuthology.orchestra.run.smithi047.stdout: "FEATURE_TOGGLE_RBD": { 2024-03-21T10:35:33.499 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-03-21T10:35:33.499 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T10:35:33.499 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:33.499 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T10:35:33.499 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "True", 2024-03-21T10:35:33.499 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:33.499 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:33.499 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:33.499 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T10:35:33.499 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:33.499 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:33.499 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:33.499 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:33.500 INFO:teuthology.orchestra.run.smithi047.stdout: "FEATURE_TOGGLE_RGW": { 2024-03-21T10:35:33.500 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-03-21T10:35:33.500 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T10:35:33.500 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:33.500 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T10:35:33.500 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "True", 2024-03-21T10:35:33.500 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:33.500 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:33.500 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:33.500 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T10:35:33.500 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:33.500 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:33.500 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:33.500 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:33.500 INFO:teuthology.orchestra.run.smithi047.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-03-21T10:35:33.500 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-03-21T10:35:33.501 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T10:35:33.501 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:33.501 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T10:35:33.501 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-03-21T10:35:33.501 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:33.501 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:33.501 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:33.501 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T10:35:33.501 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:33.501 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:33.501 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:33.501 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:33.501 INFO:teuthology.orchestra.run.smithi047.stdout: "GRAFANA_API_PASSWORD": { 2024-03-21T10:35:33.501 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "GRAFANA_API_PASSWORD", 2024-03-21T10:35:33.501 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T10:35:33.501 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:33.502 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T10:35:33.502 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "admin", 2024-03-21T10:35:33.502 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:33.502 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:33.502 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:33.502 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T10:35:33.502 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:33.502 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:33.502 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:33.502 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:33.502 INFO:teuthology.orchestra.run.smithi047.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-03-21T10:35:33.502 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-03-21T10:35:33.502 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T10:35:33.502 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:33.502 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T10:35:33.503 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "True", 2024-03-21T10:35:33.503 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:33.503 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:33.503 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:33.503 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T10:35:33.503 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:33.503 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:33.503 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:33.503 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:33.503 INFO:teuthology.orchestra.run.smithi047.stdout: "GRAFANA_API_URL": { 2024-03-21T10:35:33.503 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "GRAFANA_API_URL", 2024-03-21T10:35:33.503 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T10:35:33.503 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:33.503 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T10:35:33.504 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-03-21T10:35:33.504 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:33.504 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:33.504 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:33.504 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T10:35:33.504 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:33.504 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:33.504 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:33.504 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:33.504 INFO:teuthology.orchestra.run.smithi047.stdout: "GRAFANA_API_USERNAME": { 2024-03-21T10:35:33.504 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "GRAFANA_API_USERNAME", 2024-03-21T10:35:33.504 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T10:35:33.504 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:33.504 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T10:35:33.505 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "admin", 2024-03-21T10:35:33.505 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:33.505 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:33.505 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:33.505 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T10:35:33.505 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:33.505 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:33.505 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:33.505 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:33.505 INFO:teuthology.orchestra.run.smithi047.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-03-21T10:35:33.505 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-03-21T10:35:33.505 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T10:35:33.505 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:33.505 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T10:35:33.505 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-03-21T10:35:33.506 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:33.506 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:33.506 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:33.506 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T10:35:33.506 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:33.506 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:33.506 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:33.506 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:33.506 INFO:teuthology.orchestra.run.smithi047.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-03-21T10:35:33.506 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-03-21T10:35:33.506 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T10:35:33.506 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:33.506 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T10:35:33.506 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-03-21T10:35:33.506 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:33.507 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:33.507 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:33.507 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T10:35:33.507 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:33.507 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:33.507 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:33.507 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:33.507 INFO:teuthology.orchestra.run.smithi047.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-03-21T10:35:33.507 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-03-21T10:35:33.507 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T10:35:33.507 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:33.507 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T10:35:33.507 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "True", 2024-03-21T10:35:33.507 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:33.510 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:33.510 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:33.510 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T10:35:33.510 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:33.510 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:33.510 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:33.510 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:33.510 INFO:teuthology.orchestra.run.smithi047.stdout: "ISSUE_TRACKER_API_KEY": { 2024-03-21T10:35:33.510 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-03-21T10:35:33.510 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T10:35:33.510 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:33.510 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T10:35:33.510 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-03-21T10:35:33.511 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:33.511 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:33.511 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:33.511 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T10:35:33.511 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:33.511 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:33.511 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:33.511 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:33.511 INFO:teuthology.orchestra.run.smithi047.stdout: "MULTICLUSTER_CONFIG": { 2024-03-21T10:35:33.511 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "MULTICLUSTER_CONFIG", 2024-03-21T10:35:33.511 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T10:35:33.511 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:33.511 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T10:35:33.511 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "{}", 2024-03-21T10:35:33.512 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:33.512 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:33.512 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:33.512 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T10:35:33.512 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:33.512 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:33.512 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:33.512 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:33.512 INFO:teuthology.orchestra.run.smithi047.stdout: "PROMETHEUS_API_HOST": { 2024-03-21T10:35:33.512 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "PROMETHEUS_API_HOST", 2024-03-21T10:35:33.512 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T10:35:33.512 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:33.512 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T10:35:33.512 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-03-21T10:35:33.513 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:33.513 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:33.513 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:33.513 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T10:35:33.513 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:33.513 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:33.513 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:33.513 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:33.513 INFO:teuthology.orchestra.run.smithi047.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-03-21T10:35:33.513 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-03-21T10:35:33.513 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T10:35:33.513 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:33.513 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T10:35:33.513 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "True", 2024-03-21T10:35:33.513 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:33.514 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:33.514 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:33.514 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T10:35:33.514 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:33.514 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:33.514 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:33.514 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:33.514 INFO:teuthology.orchestra.run.smithi047.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-03-21T10:35:33.514 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-03-21T10:35:33.514 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T10:35:33.514 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:33.514 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T10:35:33.514 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-03-21T10:35:33.514 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:33.515 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:33.515 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:33.515 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T10:35:33.515 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:33.515 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:33.515 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:33.515 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:33.515 INFO:teuthology.orchestra.run.smithi047.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-03-21T10:35:33.515 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-03-21T10:35:33.515 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T10:35:33.515 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:33.515 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T10:35:33.515 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-03-21T10:35:33.515 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:33.515 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:33.516 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:33.516 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T10:35:33.516 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:33.516 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:33.516 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:33.516 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:33.516 INFO:teuthology.orchestra.run.smithi047.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-03-21T10:35:33.516 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-03-21T10:35:33.516 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T10:35:33.516 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:33.516 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T10:35:33.516 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "True", 2024-03-21T10:35:33.516 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:33.516 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:33.516 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:33.517 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T10:35:33.517 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:33.517 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:33.517 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:33.517 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:33.517 INFO:teuthology.orchestra.run.smithi047.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-03-21T10:35:33.517 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-03-21T10:35:33.517 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T10:35:33.517 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:33.517 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T10:35:33.517 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "True", 2024-03-21T10:35:33.517 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:33.517 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:33.517 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:33.518 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T10:35:33.518 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:33.518 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:33.518 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:33.518 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:33.518 INFO:teuthology.orchestra.run.smithi047.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-03-21T10:35:33.518 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-03-21T10:35:33.518 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T10:35:33.518 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:33.518 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T10:35:33.518 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-03-21T10:35:33.518 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:33.518 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:33.518 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:33.519 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T10:35:33.519 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:33.519 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:33.519 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:33.519 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:33.519 INFO:teuthology.orchestra.run.smithi047.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-03-21T10:35:33.519 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-03-21T10:35:33.519 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T10:35:33.519 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:33.519 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T10:35:33.519 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-03-21T10:35:33.519 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:33.519 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:33.520 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:33.520 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T10:35:33.520 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:33.520 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:33.520 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:33.520 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:33.520 INFO:teuthology.orchestra.run.smithi047.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-03-21T10:35:33.520 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-03-21T10:35:33.520 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T10:35:33.520 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:33.520 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T10:35:33.520 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-03-21T10:35:33.520 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:33.520 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:33.521 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:33.521 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T10:35:33.521 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:33.521 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:33.521 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:33.521 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:33.521 INFO:teuthology.orchestra.run.smithi047.stdout: "PWD_POLICY_ENABLED": { 2024-03-21T10:35:33.521 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "PWD_POLICY_ENABLED", 2024-03-21T10:35:33.521 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T10:35:33.521 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:33.521 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T10:35:33.521 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "True", 2024-03-21T10:35:33.521 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:33.521 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:33.521 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:33.522 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T10:35:33.522 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:33.522 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:33.522 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:33.522 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:33.522 INFO:teuthology.orchestra.run.smithi047.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-03-21T10:35:33.522 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-03-21T10:35:33.522 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T10:35:33.522 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:33.522 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T10:35:33.522 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-03-21T10:35:33.522 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:33.522 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:33.522 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:33.523 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T10:35:33.523 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:33.523 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:33.523 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:33.523 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:33.523 INFO:teuthology.orchestra.run.smithi047.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-03-21T10:35:33.523 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-03-21T10:35:33.523 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "int", 2024-03-21T10:35:33.523 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:33.523 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T10:35:33.523 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "10", 2024-03-21T10:35:33.523 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:33.523 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:33.523 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:33.523 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T10:35:33.523 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:33.524 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:33.524 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:33.524 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:33.524 INFO:teuthology.orchestra.run.smithi047.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-03-21T10:35:33.524 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-03-21T10:35:33.524 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "int", 2024-03-21T10:35:33.524 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:33.524 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T10:35:33.524 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "8", 2024-03-21T10:35:33.524 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:33.524 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:33.524 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:33.524 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T10:35:33.524 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:33.524 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:33.524 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:33.525 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:33.525 INFO:teuthology.orchestra.run.smithi047.stdout: "REST_REQUESTS_TIMEOUT": { 2024-03-21T10:35:33.525 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-03-21T10:35:33.525 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "int", 2024-03-21T10:35:33.525 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:33.525 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T10:35:33.525 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "45", 2024-03-21T10:35:33.525 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:33.525 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:33.525 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:33.525 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T10:35:33.525 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:33.525 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:33.525 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:33.525 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:33.525 INFO:teuthology.orchestra.run.smithi047.stdout: "RGW_API_ACCESS_KEY": { 2024-03-21T10:35:33.525 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "RGW_API_ACCESS_KEY", 2024-03-21T10:35:33.526 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T10:35:33.526 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:33.526 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T10:35:33.526 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-03-21T10:35:33.526 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:33.526 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:33.526 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:33.526 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T10:35:33.526 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:33.526 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:33.526 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:33.526 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:33.526 INFO:teuthology.orchestra.run.smithi047.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-03-21T10:35:33.526 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-03-21T10:35:33.527 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T10:35:33.527 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:33.527 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T10:35:33.527 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "admin", 2024-03-21T10:35:33.527 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:33.527 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:33.527 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:33.527 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T10:35:33.527 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:33.527 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:33.527 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:33.527 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:33.527 INFO:teuthology.orchestra.run.smithi047.stdout: "RGW_API_SECRET_KEY": { 2024-03-21T10:35:33.527 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "RGW_API_SECRET_KEY", 2024-03-21T10:35:33.528 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T10:35:33.528 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:33.528 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T10:35:33.528 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-03-21T10:35:33.528 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:33.528 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:33.528 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:33.528 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T10:35:33.528 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:33.528 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:33.528 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:33.528 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:33.528 INFO:teuthology.orchestra.run.smithi047.stdout: "RGW_API_SSL_VERIFY": { 2024-03-21T10:35:33.528 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "RGW_API_SSL_VERIFY", 2024-03-21T10:35:33.529 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T10:35:33.529 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:33.529 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T10:35:33.529 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "True", 2024-03-21T10:35:33.529 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:33.529 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:33.529 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:33.529 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T10:35:33.529 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:33.529 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:33.529 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:33.529 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:33.529 INFO:teuthology.orchestra.run.smithi047.stdout: "UNSAFE_TLS_v1_2": { 2024-03-21T10:35:33.529 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "UNSAFE_TLS_v1_2", 2024-03-21T10:35:33.529 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T10:35:33.529 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:33.530 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T10:35:33.530 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-03-21T10:35:33.530 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:33.530 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:33.530 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:33.530 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T10:35:33.530 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:33.530 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:33.530 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:33.530 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:33.530 INFO:teuthology.orchestra.run.smithi047.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-03-21T10:35:33.530 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-03-21T10:35:33.530 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "int", 2024-03-21T10:35:33.530 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:33.530 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T10:35:33.531 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "0", 2024-03-21T10:35:33.531 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:33.531 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:33.531 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:33.531 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T10:35:33.531 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:33.531 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:33.531 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:33.531 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:33.531 INFO:teuthology.orchestra.run.smithi047.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-03-21T10:35:33.531 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-03-21T10:35:33.531 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "int", 2024-03-21T10:35:33.531 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:33.531 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T10:35:33.531 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "10", 2024-03-21T10:35:33.531 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:33.532 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:33.532 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:33.532 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T10:35:33.532 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:33.532 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:33.532 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:33.532 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:33.532 INFO:teuthology.orchestra.run.smithi047.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-03-21T10:35:33.532 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-03-21T10:35:33.532 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "int", 2024-03-21T10:35:33.532 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:33.532 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T10:35:33.532 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "5", 2024-03-21T10:35:33.532 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:33.533 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:33.533 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:33.533 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T10:35:33.533 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:33.533 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:33.533 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:33.533 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:33.533 INFO:teuthology.orchestra.run.smithi047.stdout: "cross_origin_url": { 2024-03-21T10:35:33.533 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "cross_origin_url", 2024-03-21T10:35:33.533 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T10:35:33.533 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:33.533 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T10:35:33.533 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-03-21T10:35:33.533 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:33.533 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:33.534 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:33.534 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T10:35:33.534 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:33.534 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:33.534 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:33.534 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:33.534 INFO:teuthology.orchestra.run.smithi047.stdout: "crt_file": { 2024-03-21T10:35:33.534 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "crt_file", 2024-03-21T10:35:33.534 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T10:35:33.534 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:33.534 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T10:35:33.534 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-03-21T10:35:33.534 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:33.534 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:33.535 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:33.535 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T10:35:33.535 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:33.535 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:33.535 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:33.535 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:33.535 INFO:teuthology.orchestra.run.smithi047.stdout: "debug": { 2024-03-21T10:35:33.535 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "debug", 2024-03-21T10:35:33.535 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T10:35:33.535 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:33.535 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T10:35:33.535 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-03-21T10:35:33.535 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:33.535 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:33.536 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:33.536 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Enable/disable debug options", 2024-03-21T10:35:33.536 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:33.536 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:33.536 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:33.536 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:33.536 INFO:teuthology.orchestra.run.smithi047.stdout: "jwt_token_ttl": { 2024-03-21T10:35:33.536 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "jwt_token_ttl", 2024-03-21T10:35:33.536 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "int", 2024-03-21T10:35:33.536 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:33.536 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T10:35:33.536 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "28800", 2024-03-21T10:35:33.536 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:33.536 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:33.536 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:33.537 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T10:35:33.537 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:33.537 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:33.537 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:33.537 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:33.537 INFO:teuthology.orchestra.run.smithi047.stdout: "key_file": { 2024-03-21T10:35:33.537 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "key_file", 2024-03-21T10:35:33.537 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T10:35:33.537 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:33.537 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T10:35:33.537 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-03-21T10:35:33.537 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:33.537 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:33.537 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:33.537 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T10:35:33.538 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:33.538 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:33.538 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:33.538 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:33.538 INFO:teuthology.orchestra.run.smithi047.stdout: "log_level": { 2024-03-21T10:35:33.538 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_level", 2024-03-21T10:35:33.538 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T10:35:33.538 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:33.538 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T10:35:33.538 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-03-21T10:35:33.538 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:33.538 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:33.538 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-03-21T10:35:33.539 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-03-21T10:35:33.539 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-03-21T10:35:33.539 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-03-21T10:35:33.539 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-03-21T10:35:33.539 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-03-21T10:35:33.539 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-03-21T10:35:33.539 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-03-21T10:35:33.539 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T10:35:33.539 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:33.539 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:33.539 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:33.539 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:33.539 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster": { 2024-03-21T10:35:33.540 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster", 2024-03-21T10:35:33.540 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T10:35:33.540 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:33.540 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T10:35:33.540 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-03-21T10:35:33.540 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:33.540 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:33.540 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:33.540 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T10:35:33.540 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:33.540 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:33.540 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:33.540 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:33.540 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster_level": { 2024-03-21T10:35:33.541 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster_level", 2024-03-21T10:35:33.541 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T10:35:33.541 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:33.541 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T10:35:33.541 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "info", 2024-03-21T10:35:33.541 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:33.541 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:33.541 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-03-21T10:35:33.541 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-03-21T10:35:33.541 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-03-21T10:35:33.541 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-03-21T10:35:33.541 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-03-21T10:35:33.541 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-03-21T10:35:33.542 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-03-21T10:35:33.542 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-03-21T10:35:33.542 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T10:35:33.542 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:33.542 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:33.542 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:33.542 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:33.542 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_file": { 2024-03-21T10:35:33.542 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_file", 2024-03-21T10:35:33.542 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T10:35:33.542 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:33.542 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T10:35:33.542 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-03-21T10:35:33.542 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:33.543 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:33.543 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:33.543 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T10:35:33.543 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:33.543 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:33.543 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:33.543 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:33.543 INFO:teuthology.orchestra.run.smithi047.stdout: "motd": { 2024-03-21T10:35:33.543 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "motd", 2024-03-21T10:35:33.543 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T10:35:33.543 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:33.543 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T10:35:33.543 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-03-21T10:35:33.543 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:33.543 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:33.543 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:33.543 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "The message of the day", 2024-03-21T10:35:33.544 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:33.544 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:33.544 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:33.544 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:33.544 INFO:teuthology.orchestra.run.smithi047.stdout: "redirect_resolve_ip_addr": { 2024-03-21T10:35:33.544 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "redirect_resolve_ip_addr", 2024-03-21T10:35:33.544 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T10:35:33.544 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:33.544 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T10:35:33.544 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-03-21T10:35:33.544 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:33.544 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:33.544 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:33.544 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T10:35:33.544 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:33.544 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:33.544 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:33.545 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:33.545 INFO:teuthology.orchestra.run.smithi047.stdout: "server_addr": { 2024-03-21T10:35:33.545 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "server_addr", 2024-03-21T10:35:33.545 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T10:35:33.545 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:33.545 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T10:35:33.545 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "::", 2024-03-21T10:35:33.545 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:33.545 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:33.545 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:33.545 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T10:35:33.545 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:33.545 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:33.545 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:33.545 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:33.545 INFO:teuthology.orchestra.run.smithi047.stdout: "server_port": { 2024-03-21T10:35:33.545 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "server_port", 2024-03-21T10:35:33.546 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "int", 2024-03-21T10:35:33.546 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:33.546 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T10:35:33.546 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "8080", 2024-03-21T10:35:33.546 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:33.546 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:33.546 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:33.546 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T10:35:33.546 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:33.546 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:33.546 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:33.546 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:33.546 INFO:teuthology.orchestra.run.smithi047.stdout: "ssl": { 2024-03-21T10:35:33.547 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "ssl", 2024-03-21T10:35:33.547 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T10:35:33.547 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:33.547 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T10:35:33.547 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "True", 2024-03-21T10:35:33.547 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:33.547 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:33.547 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:33.547 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T10:35:33.547 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:33.547 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:33.547 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:33.547 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:33.548 INFO:teuthology.orchestra.run.smithi047.stdout: "ssl_server_port": { 2024-03-21T10:35:33.548 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "ssl_server_port", 2024-03-21T10:35:33.548 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "int", 2024-03-21T10:35:33.548 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:33.548 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T10:35:33.548 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "8443", 2024-03-21T10:35:33.548 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:33.548 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:33.548 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:33.548 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T10:35:33.548 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:33.548 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:33.548 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:33.548 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:33.549 INFO:teuthology.orchestra.run.smithi047.stdout: "standby_behaviour": { 2024-03-21T10:35:33.549 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "standby_behaviour", 2024-03-21T10:35:33.549 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T10:35:33.549 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:33.549 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T10:35:33.549 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "redirect", 2024-03-21T10:35:33.549 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:33.549 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:33.549 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-03-21T10:35:33.549 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-03-21T10:35:33.549 INFO:teuthology.orchestra.run.smithi047.stdout: "redirect" 2024-03-21T10:35:33.549 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-03-21T10:35:33.549 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T10:35:33.549 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:33.549 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:33.549 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:33.550 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:33.550 INFO:teuthology.orchestra.run.smithi047.stdout: "standby_error_status_code": { 2024-03-21T10:35:33.550 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "standby_error_status_code", 2024-03-21T10:35:33.550 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "int", 2024-03-21T10:35:33.550 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:33.550 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T10:35:33.550 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "500", 2024-03-21T10:35:33.550 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "400", 2024-03-21T10:35:33.550 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "599", 2024-03-21T10:35:33.550 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:33.550 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T10:35:33.550 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:33.550 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:33.550 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:33.551 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:33.551 INFO:teuthology.orchestra.run.smithi047.stdout: "url_prefix": { 2024-03-21T10:35:33.551 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "url_prefix", 2024-03-21T10:35:33.551 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T10:35:33.551 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:33.551 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T10:35:33.551 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-03-21T10:35:33.551 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:33.551 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:33.551 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:33.551 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T10:35:33.551 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:33.551 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:33.551 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:33.551 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-21T10:35:33.552 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-21T10:35:33.552 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:33.552 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-03-21T10:35:33.552 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "devicehealth", 2024-03-21T10:35:33.552 INFO:teuthology.orchestra.run.smithi047.stdout: "can_run": true, 2024-03-21T10:35:33.552 INFO:teuthology.orchestra.run.smithi047.stdout: "error_string": "", 2024-03-21T10:35:33.552 INFO:teuthology.orchestra.run.smithi047.stdout: "module_options": { 2024-03-21T10:35:33.552 INFO:teuthology.orchestra.run.smithi047.stdout: "enable_monitoring": { 2024-03-21T10:35:33.552 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "enable_monitoring", 2024-03-21T10:35:33.552 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T10:35:33.552 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:33.552 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T10:35:33.552 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "True", 2024-03-21T10:35:33.553 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:33.553 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:33.553 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:33.553 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "monitor device health metrics", 2024-03-21T10:35:33.553 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:33.553 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:33.553 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:33.553 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:33.553 INFO:teuthology.orchestra.run.smithi047.stdout: "log_level": { 2024-03-21T10:35:33.553 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_level", 2024-03-21T10:35:33.553 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T10:35:33.553 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:33.553 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T10:35:33.553 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-03-21T10:35:33.553 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:33.554 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:33.554 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-03-21T10:35:33.554 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-03-21T10:35:33.554 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-03-21T10:35:33.554 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-03-21T10:35:33.554 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-03-21T10:35:33.554 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-03-21T10:35:33.554 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-03-21T10:35:33.554 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-03-21T10:35:33.554 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T10:35:33.554 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:33.554 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:33.554 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:33.554 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:33.555 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster": { 2024-03-21T10:35:33.555 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster", 2024-03-21T10:35:33.555 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T10:35:33.555 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:33.555 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T10:35:33.555 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-03-21T10:35:33.555 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:33.555 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:33.555 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:33.555 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T10:35:33.555 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:33.555 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:33.555 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:33.555 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:33.556 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster_level": { 2024-03-21T10:35:33.556 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster_level", 2024-03-21T10:35:33.556 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T10:35:33.556 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:33.556 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T10:35:33.556 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "info", 2024-03-21T10:35:33.556 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:33.556 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:33.556 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-03-21T10:35:33.556 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-03-21T10:35:33.556 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-03-21T10:35:33.556 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-03-21T10:35:33.556 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-03-21T10:35:33.556 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-03-21T10:35:33.556 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-03-21T10:35:33.556 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-03-21T10:35:33.557 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T10:35:33.557 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:33.557 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:33.557 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:33.557 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:33.557 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_file": { 2024-03-21T10:35:33.557 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_file", 2024-03-21T10:35:33.557 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T10:35:33.557 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:33.557 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T10:35:33.557 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-03-21T10:35:33.557 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:33.557 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:33.557 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:33.557 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T10:35:33.558 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:33.558 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:33.558 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:33.558 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:33.558 INFO:teuthology.orchestra.run.smithi047.stdout: "mark_out_threshold": { 2024-03-21T10:35:33.558 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "mark_out_threshold", 2024-03-21T10:35:33.558 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "secs", 2024-03-21T10:35:33.558 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:33.558 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T10:35:33.558 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "2419200", 2024-03-21T10:35:33.558 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:33.558 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:33.558 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:33.558 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-03-21T10:35:33.559 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:33.559 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:33.559 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:33.559 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:33.559 INFO:teuthology.orchestra.run.smithi047.stdout: "pool_name": { 2024-03-21T10:35:33.559 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "pool_name", 2024-03-21T10:35:33.559 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T10:35:33.559 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:33.559 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T10:35:33.559 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "device_health_metrics", 2024-03-21T10:35:33.559 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:33.559 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:33.559 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:33.559 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "name of pool in which to store device health metrics", 2024-03-21T10:35:33.559 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:33.560 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:33.560 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:33.560 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:33.560 INFO:teuthology.orchestra.run.smithi047.stdout: "retention_period": { 2024-03-21T10:35:33.560 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "retention_period", 2024-03-21T10:35:33.560 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "secs", 2024-03-21T10:35:33.560 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:33.560 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T10:35:33.560 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "15552000", 2024-03-21T10:35:33.560 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:33.560 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:33.560 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:33.560 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "how long to retain device health metrics", 2024-03-21T10:35:33.560 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:33.560 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:33.561 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:33.561 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:33.561 INFO:teuthology.orchestra.run.smithi047.stdout: "scrape_frequency": { 2024-03-21T10:35:33.561 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "scrape_frequency", 2024-03-21T10:35:33.561 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "secs", 2024-03-21T10:35:33.561 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:33.561 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T10:35:33.561 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "86400", 2024-03-21T10:35:33.561 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:33.561 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:33.561 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:33.561 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "how frequently to scrape device health metrics", 2024-03-21T10:35:33.561 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:33.561 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:33.561 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:33.562 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:33.562 INFO:teuthology.orchestra.run.smithi047.stdout: "self_heal": { 2024-03-21T10:35:33.562 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "self_heal", 2024-03-21T10:35:33.562 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T10:35:33.562 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:33.562 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T10:35:33.562 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "True", 2024-03-21T10:35:33.562 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:33.562 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:33.562 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:33.562 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-03-21T10:35:33.562 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:33.562 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:33.563 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:33.563 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:33.563 INFO:teuthology.orchestra.run.smithi047.stdout: "sleep_interval": { 2024-03-21T10:35:33.563 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "sleep_interval", 2024-03-21T10:35:33.563 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "secs", 2024-03-21T10:35:33.563 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:33.563 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T10:35:33.563 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "600", 2024-03-21T10:35:33.563 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:33.563 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:33.563 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:33.563 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "how frequently to wake up and check device health", 2024-03-21T10:35:33.563 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:33.563 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:33.563 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:33.563 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:33.564 INFO:teuthology.orchestra.run.smithi047.stdout: "warn_threshold": { 2024-03-21T10:35:33.564 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "warn_threshold", 2024-03-21T10:35:33.564 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "secs", 2024-03-21T10:35:33.564 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:33.564 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T10:35:33.564 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "7257600", 2024-03-21T10:35:33.564 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:33.564 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:33.564 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:33.564 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-03-21T10:35:33.564 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:33.564 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:33.564 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:33.564 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-21T10:35:33.564 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-21T10:35:33.565 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:33.565 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-03-21T10:35:33.565 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "diskprediction_local", 2024-03-21T10:35:33.565 INFO:teuthology.orchestra.run.smithi047.stdout: "can_run": true, 2024-03-21T10:35:33.565 INFO:teuthology.orchestra.run.smithi047.stdout: "error_string": "", 2024-03-21T10:35:33.565 INFO:teuthology.orchestra.run.smithi047.stdout: "module_options": { 2024-03-21T10:35:33.565 INFO:teuthology.orchestra.run.smithi047.stdout: "log_level": { 2024-03-21T10:35:33.565 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_level", 2024-03-21T10:35:33.565 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T10:35:33.565 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:33.565 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T10:35:33.565 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-03-21T10:35:33.565 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:33.566 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:33.566 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-03-21T10:35:33.566 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-03-21T10:35:33.566 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-03-21T10:35:33.566 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-03-21T10:35:33.566 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-03-21T10:35:33.566 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-03-21T10:35:33.566 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-03-21T10:35:33.566 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-03-21T10:35:33.566 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T10:35:33.566 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:33.566 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:33.566 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:33.567 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:33.567 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster": { 2024-03-21T10:35:33.567 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster", 2024-03-21T10:35:33.567 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T10:35:33.567 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:33.567 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T10:35:33.567 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-03-21T10:35:33.567 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:33.567 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:33.567 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:33.567 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T10:35:33.567 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:33.568 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:33.568 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:33.568 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:33.568 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster_level": { 2024-03-21T10:35:33.568 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster_level", 2024-03-21T10:35:33.568 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T10:35:33.568 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:33.568 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T10:35:33.568 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "info", 2024-03-21T10:35:33.568 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:33.568 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:33.569 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-03-21T10:35:33.569 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-03-21T10:35:33.569 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-03-21T10:35:33.569 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-03-21T10:35:33.569 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-03-21T10:35:33.569 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-03-21T10:35:33.569 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-03-21T10:35:33.569 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-03-21T10:35:33.569 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T10:35:33.569 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:33.569 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:33.569 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:33.569 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:33.569 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_file": { 2024-03-21T10:35:33.570 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_file", 2024-03-21T10:35:33.570 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T10:35:33.570 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:33.570 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T10:35:33.570 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-03-21T10:35:33.570 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:33.570 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:33.570 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:33.570 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T10:35:33.570 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:33.570 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:33.570 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:33.570 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:33.570 INFO:teuthology.orchestra.run.smithi047.stdout: "predict_interval": { 2024-03-21T10:35:33.571 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "predict_interval", 2024-03-21T10:35:33.571 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T10:35:33.571 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:33.571 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T10:35:33.571 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "86400", 2024-03-21T10:35:33.571 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:33.571 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:33.571 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:33.571 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T10:35:33.571 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:33.571 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:33.571 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:33.571 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:33.571 INFO:teuthology.orchestra.run.smithi047.stdout: "predictor_model": { 2024-03-21T10:35:33.571 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "predictor_model", 2024-03-21T10:35:33.572 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T10:35:33.572 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:33.572 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T10:35:33.572 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "prophetstor", 2024-03-21T10:35:33.572 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:33.572 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:33.572 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:33.572 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T10:35:33.572 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:33.572 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:33.572 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:33.572 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:33.572 INFO:teuthology.orchestra.run.smithi047.stdout: "sleep_interval": { 2024-03-21T10:35:33.572 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "sleep_interval", 2024-03-21T10:35:33.573 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T10:35:33.573 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:33.573 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T10:35:33.573 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "600", 2024-03-21T10:35:33.573 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:33.573 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:33.573 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:33.573 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T10:35:33.573 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:33.573 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:33.573 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:33.573 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-21T10:35:33.573 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-21T10:35:33.573 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:33.573 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-03-21T10:35:33.574 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "influx", 2024-03-21T10:35:33.574 INFO:teuthology.orchestra.run.smithi047.stdout: "can_run": false, 2024-03-21T10:35:33.574 INFO:teuthology.orchestra.run.smithi047.stdout: "error_string": "influxdb python module not found", 2024-03-21T10:35:33.574 INFO:teuthology.orchestra.run.smithi047.stdout: "module_options": { 2024-03-21T10:35:33.574 INFO:teuthology.orchestra.run.smithi047.stdout: "batch_size": { 2024-03-21T10:35:33.574 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "batch_size", 2024-03-21T10:35:33.574 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "int", 2024-03-21T10:35:33.574 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:33.574 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T10:35:33.574 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "5000", 2024-03-21T10:35:33.574 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:33.574 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:33.574 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:33.575 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-03-21T10:35:33.575 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:33.575 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:33.575 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:33.575 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:33.575 INFO:teuthology.orchestra.run.smithi047.stdout: "database": { 2024-03-21T10:35:33.575 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "database", 2024-03-21T10:35:33.575 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T10:35:33.575 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:33.575 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T10:35:33.575 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "ceph", 2024-03-21T10:35:33.575 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:33.575 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:33.575 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:33.575 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-03-21T10:35:33.575 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:33.576 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:33.576 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:33.576 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:33.576 INFO:teuthology.orchestra.run.smithi047.stdout: "hostname": { 2024-03-21T10:35:33.576 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "hostname", 2024-03-21T10:35:33.576 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T10:35:33.576 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:33.576 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T10:35:33.576 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-03-21T10:35:33.576 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:33.576 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:33.576 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:33.576 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "InfluxDB server hostname", 2024-03-21T10:35:33.576 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:33.577 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:33.577 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:33.577 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:33.577 INFO:teuthology.orchestra.run.smithi047.stdout: "interval": { 2024-03-21T10:35:33.577 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "interval", 2024-03-21T10:35:33.577 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "secs", 2024-03-21T10:35:33.577 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:33.577 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T10:35:33.577 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "30", 2024-03-21T10:35:33.577 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "5", 2024-03-21T10:35:33.577 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:33.577 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:33.577 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-03-21T10:35:33.577 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:33.577 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:33.578 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:33.578 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:33.578 INFO:teuthology.orchestra.run.smithi047.stdout: "log_level": { 2024-03-21T10:35:33.578 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_level", 2024-03-21T10:35:33.578 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T10:35:33.578 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:33.578 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T10:35:33.578 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-03-21T10:35:33.578 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:33.578 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:33.578 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-03-21T10:35:33.578 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-03-21T10:35:33.578 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-03-21T10:35:33.578 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-03-21T10:35:33.579 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-03-21T10:35:33.579 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-03-21T10:35:33.579 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-03-21T10:35:33.579 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-03-21T10:35:33.579 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T10:35:33.579 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:33.579 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:33.579 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:33.579 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:33.579 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster": { 2024-03-21T10:35:33.579 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster", 2024-03-21T10:35:33.579 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T10:35:33.579 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:33.579 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T10:35:33.580 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-03-21T10:35:33.580 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:33.580 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:33.580 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:33.580 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T10:35:33.580 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:33.580 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:33.580 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:33.580 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:33.580 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster_level": { 2024-03-21T10:35:33.580 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster_level", 2024-03-21T10:35:33.580 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T10:35:33.580 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:33.580 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T10:35:33.581 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "info", 2024-03-21T10:35:33.581 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:33.581 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:33.581 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-03-21T10:35:33.581 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-03-21T10:35:33.581 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-03-21T10:35:33.581 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-03-21T10:35:33.581 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-03-21T10:35:33.581 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-03-21T10:35:33.581 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-03-21T10:35:33.581 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-03-21T10:35:33.581 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T10:35:33.581 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:33.581 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:33.582 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:33.582 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:33.582 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_file": { 2024-03-21T10:35:33.582 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_file", 2024-03-21T10:35:33.582 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T10:35:33.582 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:33.582 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T10:35:33.582 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-03-21T10:35:33.582 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:33.582 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:33.582 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:33.582 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T10:35:33.583 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:33.583 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:33.583 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:33.583 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:33.583 INFO:teuthology.orchestra.run.smithi047.stdout: "password": { 2024-03-21T10:35:33.583 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "password", 2024-03-21T10:35:33.583 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T10:35:33.583 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:33.583 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T10:35:33.583 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-03-21T10:35:33.583 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:33.583 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:33.583 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:33.583 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "password of InfluxDB server user", 2024-03-21T10:35:33.583 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:33.584 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:33.584 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:33.584 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:33.584 INFO:teuthology.orchestra.run.smithi047.stdout: "port": { 2024-03-21T10:35:33.584 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "port", 2024-03-21T10:35:33.584 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "int", 2024-03-21T10:35:33.584 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:33.584 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T10:35:33.584 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "8086", 2024-03-21T10:35:33.584 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:33.584 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:33.584 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:33.584 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "InfluxDB server port", 2024-03-21T10:35:33.584 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:33.584 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:33.585 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:33.585 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:33.585 INFO:teuthology.orchestra.run.smithi047.stdout: "ssl": { 2024-03-21T10:35:33.585 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "ssl", 2024-03-21T10:35:33.585 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T10:35:33.585 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:33.585 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T10:35:33.585 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "false", 2024-03-21T10:35:33.585 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:33.585 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:33.585 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:33.585 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-03-21T10:35:33.585 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:33.585 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:33.585 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:33.586 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:33.586 INFO:teuthology.orchestra.run.smithi047.stdout: "threads": { 2024-03-21T10:35:33.586 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "threads", 2024-03-21T10:35:33.586 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "int", 2024-03-21T10:35:33.586 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:33.586 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T10:35:33.586 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "5", 2024-03-21T10:35:33.586 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "1", 2024-03-21T10:35:33.586 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "32", 2024-03-21T10:35:33.586 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:33.586 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-03-21T10:35:33.586 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:33.586 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:33.586 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:33.587 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:33.587 INFO:teuthology.orchestra.run.smithi047.stdout: "username": { 2024-03-21T10:35:33.587 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "username", 2024-03-21T10:35:33.587 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T10:35:33.587 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:33.587 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T10:35:33.587 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-03-21T10:35:33.587 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:33.587 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:33.587 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:33.587 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "username of InfluxDB server user", 2024-03-21T10:35:33.587 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:33.587 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:33.587 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:33.587 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:33.588 INFO:teuthology.orchestra.run.smithi047.stdout: "verify_ssl": { 2024-03-21T10:35:33.588 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "verify_ssl", 2024-03-21T10:35:33.588 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T10:35:33.588 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:33.588 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T10:35:33.588 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "true", 2024-03-21T10:35:33.588 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:33.588 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:33.588 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:33.588 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-03-21T10:35:33.588 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:33.588 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:33.588 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:33.589 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-21T10:35:33.589 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-21T10:35:33.589 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:33.589 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-03-21T10:35:33.589 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "insights", 2024-03-21T10:35:33.589 INFO:teuthology.orchestra.run.smithi047.stdout: "can_run": true, 2024-03-21T10:35:33.589 INFO:teuthology.orchestra.run.smithi047.stdout: "error_string": "", 2024-03-21T10:35:33.589 INFO:teuthology.orchestra.run.smithi047.stdout: "module_options": { 2024-03-21T10:35:33.589 INFO:teuthology.orchestra.run.smithi047.stdout: "log_level": { 2024-03-21T10:35:33.589 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_level", 2024-03-21T10:35:33.589 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T10:35:33.589 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:33.589 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T10:35:33.590 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-03-21T10:35:33.590 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:33.590 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:33.590 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-03-21T10:35:33.590 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-03-21T10:35:33.590 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-03-21T10:35:33.590 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-03-21T10:35:33.590 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-03-21T10:35:33.590 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-03-21T10:35:33.590 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-03-21T10:35:33.590 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-03-21T10:35:33.590 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T10:35:33.591 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:33.591 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:33.591 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:33.591 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:33.591 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster": { 2024-03-21T10:35:33.591 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster", 2024-03-21T10:35:33.591 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T10:35:33.591 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:33.591 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T10:35:33.591 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-03-21T10:35:33.591 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:33.591 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:33.591 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:33.591 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T10:35:33.592 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:33.592 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:33.592 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:33.592 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:33.592 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster_level": { 2024-03-21T10:35:33.592 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster_level", 2024-03-21T10:35:33.592 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T10:35:33.592 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:33.592 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T10:35:33.592 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "info", 2024-03-21T10:35:33.592 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:33.592 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:33.592 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-03-21T10:35:33.592 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-03-21T10:35:33.592 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-03-21T10:35:33.593 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-03-21T10:35:33.593 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-03-21T10:35:33.593 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-03-21T10:35:33.593 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-03-21T10:35:33.593 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-03-21T10:35:33.593 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T10:35:33.593 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:33.593 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:33.593 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:33.593 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:33.593 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_file": { 2024-03-21T10:35:33.593 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_file", 2024-03-21T10:35:33.593 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T10:35:33.593 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:33.593 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T10:35:33.594 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-03-21T10:35:33.594 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:33.594 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:33.594 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:33.594 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T10:35:33.594 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:33.594 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:33.594 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:33.594 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-21T10:35:33.594 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-21T10:35:33.594 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:33.594 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-03-21T10:35:33.594 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "iostat", 2024-03-21T10:35:33.594 INFO:teuthology.orchestra.run.smithi047.stdout: "can_run": true, 2024-03-21T10:35:33.595 INFO:teuthology.orchestra.run.smithi047.stdout: "error_string": "", 2024-03-21T10:35:33.595 INFO:teuthology.orchestra.run.smithi047.stdout: "module_options": { 2024-03-21T10:35:33.595 INFO:teuthology.orchestra.run.smithi047.stdout: "log_level": { 2024-03-21T10:35:33.595 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_level", 2024-03-21T10:35:33.595 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T10:35:33.595 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:33.595 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T10:35:33.595 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-03-21T10:35:33.595 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:33.595 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:33.595 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-03-21T10:35:33.595 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-03-21T10:35:33.595 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-03-21T10:35:33.595 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-03-21T10:35:33.595 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-03-21T10:35:33.596 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-03-21T10:35:33.596 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-03-21T10:35:33.596 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-03-21T10:35:33.596 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T10:35:33.596 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:33.596 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:33.596 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:33.596 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:33.596 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster": { 2024-03-21T10:35:33.596 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster", 2024-03-21T10:35:33.596 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T10:35:33.596 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:33.596 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T10:35:33.596 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-03-21T10:35:33.596 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:33.597 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:33.597 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:33.597 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T10:35:33.597 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:33.597 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:33.597 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:33.597 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:33.597 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster_level": { 2024-03-21T10:35:33.597 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster_level", 2024-03-21T10:35:33.597 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T10:35:33.597 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:33.597 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T10:35:33.597 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "info", 2024-03-21T10:35:33.597 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:33.598 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:33.598 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-03-21T10:35:33.598 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-03-21T10:35:33.598 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-03-21T10:35:33.598 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-03-21T10:35:33.598 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-03-21T10:35:33.598 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-03-21T10:35:33.598 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-03-21T10:35:33.598 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-03-21T10:35:33.598 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T10:35:33.598 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:33.598 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:33.598 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:33.598 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:33.598 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_file": { 2024-03-21T10:35:33.599 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_file", 2024-03-21T10:35:33.599 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T10:35:33.599 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:33.599 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T10:35:33.599 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-03-21T10:35:33.599 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:33.599 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:33.599 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:33.599 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T10:35:33.599 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:33.599 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:33.599 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:33.599 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-21T10:35:33.599 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-21T10:35:33.599 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:33.600 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-03-21T10:35:33.600 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "k8sevents", 2024-03-21T10:35:33.600 INFO:teuthology.orchestra.run.smithi047.stdout: "can_run": true, 2024-03-21T10:35:33.600 INFO:teuthology.orchestra.run.smithi047.stdout: "error_string": "", 2024-03-21T10:35:33.600 INFO:teuthology.orchestra.run.smithi047.stdout: "module_options": { 2024-03-21T10:35:33.600 INFO:teuthology.orchestra.run.smithi047.stdout: "ceph_event_retention_days": { 2024-03-21T10:35:33.600 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "ceph_event_retention_days", 2024-03-21T10:35:33.600 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "int", 2024-03-21T10:35:33.600 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:33.600 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T10:35:33.600 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "7", 2024-03-21T10:35:33.600 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:33.600 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:33.600 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:33.600 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Days to hold ceph event information within local cache", 2024-03-21T10:35:33.600 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:33.601 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:33.601 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:33.601 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:33.601 INFO:teuthology.orchestra.run.smithi047.stdout: "config_check_secs": { 2024-03-21T10:35:33.601 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "config_check_secs", 2024-03-21T10:35:33.601 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "int", 2024-03-21T10:35:33.601 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:33.601 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T10:35:33.601 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "10", 2024-03-21T10:35:33.601 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "10", 2024-03-21T10:35:33.601 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:33.601 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:33.601 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-03-21T10:35:33.601 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:33.601 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:33.601 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:33.602 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:33.602 INFO:teuthology.orchestra.run.smithi047.stdout: "log_level": { 2024-03-21T10:35:33.602 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_level", 2024-03-21T10:35:33.602 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T10:35:33.602 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:33.602 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T10:35:33.602 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-03-21T10:35:33.602 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:33.602 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:33.602 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-03-21T10:35:33.602 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-03-21T10:35:33.602 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-03-21T10:35:33.602 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-03-21T10:35:33.602 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-03-21T10:35:33.603 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-03-21T10:35:33.603 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-03-21T10:35:33.603 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-03-21T10:35:33.603 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T10:35:33.603 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:33.603 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:33.603 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:33.603 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:33.603 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster": { 2024-03-21T10:35:33.603 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster", 2024-03-21T10:35:33.603 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T10:35:33.603 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:33.603 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T10:35:33.603 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-03-21T10:35:33.604 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:33.604 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:33.604 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:33.604 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T10:35:33.604 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:33.604 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:33.604 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:33.604 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:33.604 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster_level": { 2024-03-21T10:35:33.604 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster_level", 2024-03-21T10:35:33.604 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T10:35:33.604 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:33.604 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T10:35:33.604 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "info", 2024-03-21T10:35:33.605 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:33.605 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:33.605 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-03-21T10:35:33.605 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-03-21T10:35:33.605 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-03-21T10:35:33.605 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-03-21T10:35:33.605 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-03-21T10:35:33.605 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-03-21T10:35:33.605 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-03-21T10:35:33.605 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-03-21T10:35:33.605 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T10:35:33.605 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:33.605 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:33.605 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:33.605 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:33.606 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_file": { 2024-03-21T10:35:33.606 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_file", 2024-03-21T10:35:33.606 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T10:35:33.606 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:33.606 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T10:35:33.606 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-03-21T10:35:33.606 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:33.606 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:33.606 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:33.606 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T10:35:33.606 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:33.606 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:33.606 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:33.606 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-21T10:35:33.607 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-21T10:35:33.607 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:33.607 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-03-21T10:35:33.607 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "localpool", 2024-03-21T10:35:33.607 INFO:teuthology.orchestra.run.smithi047.stdout: "can_run": true, 2024-03-21T10:35:33.607 INFO:teuthology.orchestra.run.smithi047.stdout: "error_string": "", 2024-03-21T10:35:33.607 INFO:teuthology.orchestra.run.smithi047.stdout: "module_options": { 2024-03-21T10:35:33.607 INFO:teuthology.orchestra.run.smithi047.stdout: "failure_domain": { 2024-03-21T10:35:33.607 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "failure_domain", 2024-03-21T10:35:33.607 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T10:35:33.607 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:33.607 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T10:35:33.607 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "host", 2024-03-21T10:35:33.607 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:33.607 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:33.608 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:33.608 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "failure domain for any created local pool", 2024-03-21T10:35:33.608 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-03-21T10:35:33.608 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:33.608 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:33.608 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:33.608 INFO:teuthology.orchestra.run.smithi047.stdout: "log_level": { 2024-03-21T10:35:33.608 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_level", 2024-03-21T10:35:33.608 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T10:35:33.608 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:33.608 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T10:35:33.608 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-03-21T10:35:33.608 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:33.608 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:33.608 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-03-21T10:35:33.609 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-03-21T10:35:33.609 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-03-21T10:35:33.609 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-03-21T10:35:33.609 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-03-21T10:35:33.609 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-03-21T10:35:33.609 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-03-21T10:35:33.609 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-03-21T10:35:33.609 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T10:35:33.609 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:33.609 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:33.609 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:33.609 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:33.609 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster": { 2024-03-21T10:35:33.609 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster", 2024-03-21T10:35:33.609 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T10:35:33.610 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:33.610 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T10:35:33.610 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-03-21T10:35:33.610 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:33.610 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:33.610 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:33.610 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T10:35:33.610 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:33.610 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:33.610 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:33.610 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:33.610 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster_level": { 2024-03-21T10:35:33.610 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster_level", 2024-03-21T10:35:33.610 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T10:35:33.611 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:33.611 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T10:35:33.611 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "info", 2024-03-21T10:35:33.611 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:33.611 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:33.611 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-03-21T10:35:33.611 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-03-21T10:35:33.611 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-03-21T10:35:33.611 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-03-21T10:35:33.611 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-03-21T10:35:33.611 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-03-21T10:35:33.611 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-03-21T10:35:33.611 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-03-21T10:35:33.611 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T10:35:33.611 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:33.611 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:33.612 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:33.612 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:33.612 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_file": { 2024-03-21T10:35:33.612 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_file", 2024-03-21T10:35:33.612 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T10:35:33.612 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:33.612 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T10:35:33.612 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-03-21T10:35:33.612 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:33.612 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:33.612 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:33.612 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T10:35:33.612 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:33.612 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:33.613 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:33.613 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:33.613 INFO:teuthology.orchestra.run.smithi047.stdout: "min_size": { 2024-03-21T10:35:33.613 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "min_size", 2024-03-21T10:35:33.613 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "int", 2024-03-21T10:35:33.613 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:33.613 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T10:35:33.613 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-03-21T10:35:33.613 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:33.613 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:33.613 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:33.613 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "default min_size for any created local pool", 2024-03-21T10:35:33.613 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-03-21T10:35:33.613 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:33.613 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:33.614 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:33.614 INFO:teuthology.orchestra.run.smithi047.stdout: "num_rep": { 2024-03-21T10:35:33.614 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "num_rep", 2024-03-21T10:35:33.614 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "int", 2024-03-21T10:35:33.614 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:33.614 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T10:35:33.614 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "3", 2024-03-21T10:35:33.614 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:33.614 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:33.614 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:33.614 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "default replica count for any created local pool", 2024-03-21T10:35:33.614 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:33.614 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:33.615 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:33.615 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:33.615 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_num": { 2024-03-21T10:35:33.615 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "pg_num", 2024-03-21T10:35:33.615 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "int", 2024-03-21T10:35:33.615 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:33.615 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T10:35:33.615 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "128", 2024-03-21T10:35:33.615 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:33.615 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:33.615 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:33.615 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "default pg_num for any created local pool", 2024-03-21T10:35:33.615 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:33.615 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:33.615 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:33.615 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:33.615 INFO:teuthology.orchestra.run.smithi047.stdout: "prefix": { 2024-03-21T10:35:33.615 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "prefix", 2024-03-21T10:35:33.616 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T10:35:33.616 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:33.616 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T10:35:33.616 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-03-21T10:35:33.616 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:33.616 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:33.616 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:33.616 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "name prefix for any created local pool", 2024-03-21T10:35:33.616 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:33.616 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:33.616 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:33.616 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:33.616 INFO:teuthology.orchestra.run.smithi047.stdout: "subtree": { 2024-03-21T10:35:33.617 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "subtree", 2024-03-21T10:35:33.617 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T10:35:33.617 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:33.617 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T10:35:33.617 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "rack", 2024-03-21T10:35:33.617 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:33.617 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:33.617 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:33.617 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "CRUSH level for which to create a local pool", 2024-03-21T10:35:33.617 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-03-21T10:35:33.617 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:33.617 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:33.617 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-21T10:35:33.617 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-21T10:35:33.618 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:33.618 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-03-21T10:35:33.618 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "mds_autoscaler", 2024-03-21T10:35:33.618 INFO:teuthology.orchestra.run.smithi047.stdout: "can_run": true, 2024-03-21T10:35:33.618 INFO:teuthology.orchestra.run.smithi047.stdout: "error_string": "", 2024-03-21T10:35:33.618 INFO:teuthology.orchestra.run.smithi047.stdout: "module_options": { 2024-03-21T10:35:33.618 INFO:teuthology.orchestra.run.smithi047.stdout: "log_level": { 2024-03-21T10:35:33.618 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_level", 2024-03-21T10:35:33.618 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T10:35:33.618 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:33.618 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T10:35:33.618 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-03-21T10:35:33.618 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:33.619 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:33.619 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-03-21T10:35:33.619 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-03-21T10:35:33.619 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-03-21T10:35:33.619 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-03-21T10:35:33.619 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-03-21T10:35:33.619 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-03-21T10:35:33.619 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-03-21T10:35:33.619 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-03-21T10:35:33.619 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T10:35:33.619 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:33.619 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:33.619 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:33.619 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:33.619 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster": { 2024-03-21T10:35:33.620 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster", 2024-03-21T10:35:33.620 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T10:35:33.620 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:33.620 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T10:35:33.620 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-03-21T10:35:33.620 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:33.620 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:33.620 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:33.620 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T10:35:33.620 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:33.620 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:33.620 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:33.620 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:33.620 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster_level": { 2024-03-21T10:35:33.620 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster_level", 2024-03-21T10:35:33.620 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T10:35:33.621 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:33.621 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T10:35:33.621 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "info", 2024-03-21T10:35:33.621 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:33.621 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:33.621 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-03-21T10:35:33.621 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-03-21T10:35:33.621 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-03-21T10:35:33.621 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-03-21T10:35:33.621 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-03-21T10:35:33.621 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-03-21T10:35:33.621 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-03-21T10:35:33.621 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-03-21T10:35:33.621 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T10:35:33.621 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:33.622 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:33.622 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:33.622 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:33.622 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_file": { 2024-03-21T10:35:33.622 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_file", 2024-03-21T10:35:33.622 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T10:35:33.622 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:33.622 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T10:35:33.622 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-03-21T10:35:33.622 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:33.622 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:33.622 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:33.622 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T10:35:33.622 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:33.622 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:33.623 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:33.623 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-21T10:35:33.623 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-21T10:35:33.623 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:33.623 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-03-21T10:35:33.623 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "mirroring", 2024-03-21T10:35:33.623 INFO:teuthology.orchestra.run.smithi047.stdout: "can_run": true, 2024-03-21T10:35:33.623 INFO:teuthology.orchestra.run.smithi047.stdout: "error_string": "", 2024-03-21T10:35:33.623 INFO:teuthology.orchestra.run.smithi047.stdout: "module_options": { 2024-03-21T10:35:33.623 INFO:teuthology.orchestra.run.smithi047.stdout: "log_level": { 2024-03-21T10:35:33.623 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_level", 2024-03-21T10:35:33.623 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T10:35:33.623 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:33.623 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T10:35:33.623 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-03-21T10:35:33.624 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:33.624 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:33.624 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-03-21T10:35:33.624 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-03-21T10:35:33.624 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-03-21T10:35:33.624 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-03-21T10:35:33.624 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-03-21T10:35:33.624 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-03-21T10:35:33.624 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-03-21T10:35:33.624 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-03-21T10:35:33.624 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T10:35:33.624 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:33.624 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:33.624 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:33.624 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:33.625 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster": { 2024-03-21T10:35:33.625 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster", 2024-03-21T10:35:33.625 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T10:35:33.625 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:33.625 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T10:35:33.625 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-03-21T10:35:33.625 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:33.625 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:33.625 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:33.625 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T10:35:33.625 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:33.625 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:33.625 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:33.625 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:33.625 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster_level": { 2024-03-21T10:35:33.625 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster_level", 2024-03-21T10:35:33.626 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T10:35:33.626 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:33.626 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T10:35:33.626 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "info", 2024-03-21T10:35:33.626 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:33.626 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:33.626 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-03-21T10:35:33.626 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-03-21T10:35:33.626 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-03-21T10:35:33.626 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-03-21T10:35:33.626 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-03-21T10:35:33.626 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-03-21T10:35:33.626 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-03-21T10:35:33.627 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-03-21T10:35:33.627 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T10:35:33.627 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:33.627 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:33.627 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:33.627 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:33.627 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_file": { 2024-03-21T10:35:33.627 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_file", 2024-03-21T10:35:33.627 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T10:35:33.627 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:33.627 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T10:35:33.627 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-03-21T10:35:33.627 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:33.627 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:33.627 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:33.627 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T10:35:33.628 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:33.628 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:33.628 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:33.628 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-21T10:35:33.628 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-21T10:35:33.628 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:33.628 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-03-21T10:35:33.628 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "nfs", 2024-03-21T10:35:33.628 INFO:teuthology.orchestra.run.smithi047.stdout: "can_run": true, 2024-03-21T10:35:33.628 INFO:teuthology.orchestra.run.smithi047.stdout: "error_string": "", 2024-03-21T10:35:33.628 INFO:teuthology.orchestra.run.smithi047.stdout: "module_options": { 2024-03-21T10:35:33.628 INFO:teuthology.orchestra.run.smithi047.stdout: "log_level": { 2024-03-21T10:35:33.628 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_level", 2024-03-21T10:35:33.628 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T10:35:33.628 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:33.629 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T10:35:33.629 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-03-21T10:35:33.629 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:33.629 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:33.629 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-03-21T10:35:33.629 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-03-21T10:35:33.629 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-03-21T10:35:33.629 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-03-21T10:35:33.629 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-03-21T10:35:33.629 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-03-21T10:35:33.629 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-03-21T10:35:33.629 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-03-21T10:35:33.629 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T10:35:33.629 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:33.629 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:33.630 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:33.630 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:33.630 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster": { 2024-03-21T10:35:33.630 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster", 2024-03-21T10:35:33.630 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T10:35:33.630 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:33.630 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T10:35:33.630 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-03-21T10:35:33.630 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:33.630 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:33.630 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:33.630 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T10:35:33.630 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:33.630 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:33.630 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:33.631 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:33.631 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster_level": { 2024-03-21T10:35:33.631 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster_level", 2024-03-21T10:35:33.631 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T10:35:33.631 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:33.631 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T10:35:33.631 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "info", 2024-03-21T10:35:33.631 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:33.631 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:33.631 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-03-21T10:35:33.631 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-03-21T10:35:33.631 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-03-21T10:35:33.631 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-03-21T10:35:33.631 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-03-21T10:35:33.631 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-03-21T10:35:33.632 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-03-21T10:35:33.632 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-03-21T10:35:33.632 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T10:35:33.632 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:33.632 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:33.632 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:33.632 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:33.632 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_file": { 2024-03-21T10:35:33.632 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_file", 2024-03-21T10:35:33.632 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T10:35:33.632 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:33.632 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T10:35:33.632 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-03-21T10:35:33.632 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:33.632 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:33.633 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:33.633 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T10:35:33.633 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:33.633 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:33.633 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:33.633 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-21T10:35:33.633 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-21T10:35:33.633 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:33.633 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-03-21T10:35:33.633 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "orchestrator", 2024-03-21T10:35:33.633 INFO:teuthology.orchestra.run.smithi047.stdout: "can_run": true, 2024-03-21T10:35:33.633 INFO:teuthology.orchestra.run.smithi047.stdout: "error_string": "", 2024-03-21T10:35:33.633 INFO:teuthology.orchestra.run.smithi047.stdout: "module_options": { 2024-03-21T10:35:33.633 INFO:teuthology.orchestra.run.smithi047.stdout: "fail_fs": { 2024-03-21T10:35:33.633 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "fail_fs", 2024-03-21T10:35:33.633 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T10:35:33.634 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:33.634 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T10:35:33.634 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-03-21T10:35:33.634 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:33.634 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:33.634 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:33.634 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-03-21T10:35:33.634 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:33.634 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:33.634 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:33.634 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:33.634 INFO:teuthology.orchestra.run.smithi047.stdout: "log_level": { 2024-03-21T10:35:33.634 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_level", 2024-03-21T10:35:33.634 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T10:35:33.634 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:33.635 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T10:35:33.635 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-03-21T10:35:33.635 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:33.635 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:33.635 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-03-21T10:35:33.635 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-03-21T10:35:33.635 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-03-21T10:35:33.635 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-03-21T10:35:33.635 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-03-21T10:35:33.635 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-03-21T10:35:33.635 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-03-21T10:35:33.635 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-03-21T10:35:33.635 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T10:35:33.635 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:33.635 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:33.635 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:33.636 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:33.636 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster": { 2024-03-21T10:35:33.636 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster", 2024-03-21T10:35:33.636 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T10:35:33.636 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:33.636 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T10:35:33.636 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-03-21T10:35:33.636 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:33.636 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:33.636 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:33.636 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T10:35:33.636 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:33.636 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:33.636 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:33.637 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:33.637 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster_level": { 2024-03-21T10:35:33.637 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster_level", 2024-03-21T10:35:33.637 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T10:35:33.637 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:33.637 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T10:35:33.637 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "info", 2024-03-21T10:35:33.637 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:33.637 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:33.637 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-03-21T10:35:33.637 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-03-21T10:35:33.637 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-03-21T10:35:33.638 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-03-21T10:35:33.638 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-03-21T10:35:33.638 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-03-21T10:35:33.638 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-03-21T10:35:33.638 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-03-21T10:35:33.638 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T10:35:33.638 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:33.638 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:33.638 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:33.638 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:33.638 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_file": { 2024-03-21T10:35:33.638 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_file", 2024-03-21T10:35:33.639 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T10:35:33.639 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:33.639 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T10:35:33.639 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-03-21T10:35:33.639 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:33.639 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:33.639 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:33.639 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T10:35:33.639 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:33.639 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:33.639 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:33.639 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:33.639 INFO:teuthology.orchestra.run.smithi047.stdout: "orchestrator": { 2024-03-21T10:35:33.639 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "orchestrator", 2024-03-21T10:35:33.640 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T10:35:33.640 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:33.640 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T10:35:33.640 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-03-21T10:35:33.640 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:33.640 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:33.640 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-03-21T10:35:33.640 INFO:teuthology.orchestra.run.smithi047.stdout: "cephadm", 2024-03-21T10:35:33.640 INFO:teuthology.orchestra.run.smithi047.stdout: "rook", 2024-03-21T10:35:33.640 INFO:teuthology.orchestra.run.smithi047.stdout: "test_orchestrator" 2024-03-21T10:35:33.640 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-03-21T10:35:33.640 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Orchestrator backend", 2024-03-21T10:35:33.640 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:33.640 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:33.641 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:33.641 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-21T10:35:33.641 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-21T10:35:33.641 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:33.641 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-03-21T10:35:33.641 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "osd_perf_query", 2024-03-21T10:35:33.641 INFO:teuthology.orchestra.run.smithi047.stdout: "can_run": true, 2024-03-21T10:35:33.641 INFO:teuthology.orchestra.run.smithi047.stdout: "error_string": "", 2024-03-21T10:35:33.641 INFO:teuthology.orchestra.run.smithi047.stdout: "module_options": { 2024-03-21T10:35:33.641 INFO:teuthology.orchestra.run.smithi047.stdout: "log_level": { 2024-03-21T10:35:33.641 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_level", 2024-03-21T10:35:33.641 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T10:35:33.641 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:33.641 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T10:35:33.642 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-03-21T10:35:33.642 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:33.642 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:33.642 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-03-21T10:35:33.642 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-03-21T10:35:33.642 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-03-21T10:35:33.642 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-03-21T10:35:33.642 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-03-21T10:35:33.642 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-03-21T10:35:33.642 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-03-21T10:35:33.642 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-03-21T10:35:33.642 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T10:35:33.642 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:33.642 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:33.643 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:33.643 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:33.643 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster": { 2024-03-21T10:35:33.643 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster", 2024-03-21T10:35:33.643 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T10:35:33.643 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:33.643 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T10:35:33.643 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-03-21T10:35:33.643 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:33.643 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:33.643 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:33.643 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T10:35:33.643 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:33.643 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:33.643 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:33.644 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:33.644 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster_level": { 2024-03-21T10:35:33.644 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster_level", 2024-03-21T10:35:33.644 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T10:35:33.644 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:33.644 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T10:35:33.644 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "info", 2024-03-21T10:35:33.644 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:33.644 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:33.644 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-03-21T10:35:33.644 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-03-21T10:35:33.644 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-03-21T10:35:33.644 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-03-21T10:35:33.644 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-03-21T10:35:33.644 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-03-21T10:35:33.645 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-03-21T10:35:33.645 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-03-21T10:35:33.645 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T10:35:33.645 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:33.645 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:33.645 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:33.645 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:33.645 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_file": { 2024-03-21T10:35:33.645 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_file", 2024-03-21T10:35:33.645 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T10:35:33.645 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:33.645 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T10:35:33.645 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-03-21T10:35:33.646 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:33.646 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:33.646 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:33.646 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T10:35:33.646 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:33.646 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:33.646 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:33.646 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-21T10:35:33.646 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-21T10:35:33.646 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:33.646 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-03-21T10:35:33.646 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "osd_support", 2024-03-21T10:35:33.646 INFO:teuthology.orchestra.run.smithi047.stdout: "can_run": true, 2024-03-21T10:35:33.647 INFO:teuthology.orchestra.run.smithi047.stdout: "error_string": "", 2024-03-21T10:35:33.647 INFO:teuthology.orchestra.run.smithi047.stdout: "module_options": { 2024-03-21T10:35:33.647 INFO:teuthology.orchestra.run.smithi047.stdout: "log_level": { 2024-03-21T10:35:33.647 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_level", 2024-03-21T10:35:33.647 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T10:35:33.647 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:33.647 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T10:35:33.647 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-03-21T10:35:33.647 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:33.647 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:33.647 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-03-21T10:35:33.647 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-03-21T10:35:33.647 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-03-21T10:35:33.647 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-03-21T10:35:33.647 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-03-21T10:35:33.647 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-03-21T10:35:33.648 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-03-21T10:35:33.648 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-03-21T10:35:33.648 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T10:35:33.648 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:33.648 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:33.648 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:33.648 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:33.648 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster": { 2024-03-21T10:35:33.648 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster", 2024-03-21T10:35:33.648 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T10:35:33.648 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:33.648 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T10:35:33.648 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-03-21T10:35:33.648 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:33.648 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:33.648 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:33.649 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T10:35:33.649 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:33.649 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:33.649 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:33.649 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:33.649 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster_level": { 2024-03-21T10:35:33.649 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster_level", 2024-03-21T10:35:33.649 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T10:35:33.649 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:33.649 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T10:35:33.649 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "info", 2024-03-21T10:35:33.649 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:33.649 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:33.649 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-03-21T10:35:33.650 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-03-21T10:35:33.650 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-03-21T10:35:33.650 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-03-21T10:35:33.650 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-03-21T10:35:33.650 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-03-21T10:35:33.650 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-03-21T10:35:33.650 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-03-21T10:35:33.650 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T10:35:33.650 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:33.650 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:33.650 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:33.650 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:33.650 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_file": { 2024-03-21T10:35:33.650 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_file", 2024-03-21T10:35:33.651 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T10:35:33.651 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:33.651 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T10:35:33.651 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-03-21T10:35:33.651 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:33.651 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:33.651 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:33.651 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T10:35:33.651 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:33.651 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:33.651 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:33.651 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-21T10:35:33.651 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-21T10:35:33.651 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:33.651 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-03-21T10:35:33.651 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "pg_autoscaler", 2024-03-21T10:35:33.652 INFO:teuthology.orchestra.run.smithi047.stdout: "can_run": true, 2024-03-21T10:35:33.652 INFO:teuthology.orchestra.run.smithi047.stdout: "error_string": "", 2024-03-21T10:35:33.652 INFO:teuthology.orchestra.run.smithi047.stdout: "module_options": { 2024-03-21T10:35:33.652 INFO:teuthology.orchestra.run.smithi047.stdout: "log_level": { 2024-03-21T10:35:33.652 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_level", 2024-03-21T10:35:33.652 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T10:35:33.652 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:33.652 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T10:35:33.652 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-03-21T10:35:33.652 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:33.652 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:33.652 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-03-21T10:35:33.652 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-03-21T10:35:33.653 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-03-21T10:35:33.653 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-03-21T10:35:33.653 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-03-21T10:35:33.653 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-03-21T10:35:33.653 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-03-21T10:35:33.653 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-03-21T10:35:33.653 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T10:35:33.653 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:33.653 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:33.653 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:33.653 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:33.653 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster": { 2024-03-21T10:35:33.653 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster", 2024-03-21T10:35:33.653 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T10:35:33.654 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:33.654 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T10:35:33.654 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-03-21T10:35:33.654 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:33.654 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:33.654 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:33.654 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T10:35:33.654 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:33.654 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:33.654 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:33.654 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:33.654 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster_level": { 2024-03-21T10:35:33.654 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster_level", 2024-03-21T10:35:33.655 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T10:35:33.655 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:33.655 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T10:35:33.655 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "info", 2024-03-21T10:35:33.655 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:33.655 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:33.655 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-03-21T10:35:33.655 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-03-21T10:35:33.655 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-03-21T10:35:33.655 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-03-21T10:35:33.655 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-03-21T10:35:33.655 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-03-21T10:35:33.655 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-03-21T10:35:33.656 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-03-21T10:35:33.656 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T10:35:33.656 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:33.656 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:33.656 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:33.656 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:33.656 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_file": { 2024-03-21T10:35:33.656 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_file", 2024-03-21T10:35:33.656 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T10:35:33.656 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:33.656 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T10:35:33.656 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-03-21T10:35:33.656 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:33.656 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:33.657 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:33.657 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T10:35:33.657 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:33.657 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:33.657 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:33.657 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:33.657 INFO:teuthology.orchestra.run.smithi047.stdout: "sleep_interval": { 2024-03-21T10:35:33.657 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "sleep_interval", 2024-03-21T10:35:33.657 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "secs", 2024-03-21T10:35:33.657 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:33.657 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T10:35:33.657 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "60", 2024-03-21T10:35:33.657 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:33.657 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:33.657 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:33.657 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T10:35:33.658 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:33.658 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:33.658 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:33.658 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:33.658 INFO:teuthology.orchestra.run.smithi047.stdout: "threshold": { 2024-03-21T10:35:33.658 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "threshold", 2024-03-21T10:35:33.658 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "float", 2024-03-21T10:35:33.658 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:33.658 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T10:35:33.658 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "3.0", 2024-03-21T10:35:33.658 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "1.0", 2024-03-21T10:35:33.658 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:33.658 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:33.659 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "scaling threshold", 2024-03-21T10:35:33.659 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-03-21T10:35:33.659 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:33.659 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:33.659 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-21T10:35:33.659 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-21T10:35:33.659 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:33.659 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-03-21T10:35:33.659 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "progress", 2024-03-21T10:35:33.659 INFO:teuthology.orchestra.run.smithi047.stdout: "can_run": true, 2024-03-21T10:35:33.659 INFO:teuthology.orchestra.run.smithi047.stdout: "error_string": "", 2024-03-21T10:35:33.659 INFO:teuthology.orchestra.run.smithi047.stdout: "module_options": { 2024-03-21T10:35:33.659 INFO:teuthology.orchestra.run.smithi047.stdout: "allow_pg_recovery_event": { 2024-03-21T10:35:33.659 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "allow_pg_recovery_event", 2024-03-21T10:35:33.660 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T10:35:33.660 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:33.660 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T10:35:33.660 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-03-21T10:35:33.660 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:33.660 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:33.660 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:33.660 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "allow the module to show pg recovery progress", 2024-03-21T10:35:33.660 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:33.660 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:33.660 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:33.660 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:33.660 INFO:teuthology.orchestra.run.smithi047.stdout: "enabled": { 2024-03-21T10:35:33.660 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "enabled", 2024-03-21T10:35:33.661 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T10:35:33.661 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:33.661 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T10:35:33.661 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "True", 2024-03-21T10:35:33.661 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:33.661 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:33.661 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:33.661 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T10:35:33.661 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:33.661 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:33.661 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:33.661 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:33.661 INFO:teuthology.orchestra.run.smithi047.stdout: "log_level": { 2024-03-21T10:35:33.661 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_level", 2024-03-21T10:35:33.661 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T10:35:33.662 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:33.662 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T10:35:33.662 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-03-21T10:35:33.662 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:33.662 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:33.662 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-03-21T10:35:33.662 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-03-21T10:35:33.662 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-03-21T10:35:33.662 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-03-21T10:35:33.662 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-03-21T10:35:33.662 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-03-21T10:35:33.662 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-03-21T10:35:33.662 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-03-21T10:35:33.662 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T10:35:33.662 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:33.663 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:33.663 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:33.663 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:33.663 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster": { 2024-03-21T10:35:33.663 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster", 2024-03-21T10:35:33.663 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T10:35:33.663 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:33.663 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T10:35:33.663 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-03-21T10:35:33.663 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:33.663 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:33.663 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:33.663 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T10:35:33.663 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:33.663 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:33.664 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:33.664 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:33.664 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster_level": { 2024-03-21T10:35:33.664 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster_level", 2024-03-21T10:35:33.664 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T10:35:33.664 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:33.664 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T10:35:33.664 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "info", 2024-03-21T10:35:33.664 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:33.664 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:33.664 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-03-21T10:35:33.664 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-03-21T10:35:33.664 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-03-21T10:35:33.664 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-03-21T10:35:33.664 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-03-21T10:35:33.665 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-03-21T10:35:33.665 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-03-21T10:35:33.665 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-03-21T10:35:33.665 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T10:35:33.665 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:33.665 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:33.665 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:33.665 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:33.665 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_file": { 2024-03-21T10:35:33.665 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_file", 2024-03-21T10:35:33.665 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T10:35:33.665 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:33.665 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T10:35:33.665 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-03-21T10:35:33.665 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:33.665 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:33.666 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:33.666 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T10:35:33.666 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:33.666 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:33.666 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:33.666 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:33.666 INFO:teuthology.orchestra.run.smithi047.stdout: "max_completed_events": { 2024-03-21T10:35:33.666 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "max_completed_events", 2024-03-21T10:35:33.666 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "int", 2024-03-21T10:35:33.666 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:33.666 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T10:35:33.666 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "50", 2024-03-21T10:35:33.666 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:33.667 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:33.667 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:33.667 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "number of past completed events to remember", 2024-03-21T10:35:33.667 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:33.667 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:33.667 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:33.667 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:33.667 INFO:teuthology.orchestra.run.smithi047.stdout: "sleep_interval": { 2024-03-21T10:35:33.667 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "sleep_interval", 2024-03-21T10:35:33.667 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "secs", 2024-03-21T10:35:33.667 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:33.667 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T10:35:33.667 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "5", 2024-03-21T10:35:33.667 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:33.667 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:33.668 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:33.668 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "how long the module is going to sleep", 2024-03-21T10:35:33.668 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:33.668 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:33.668 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:33.668 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-21T10:35:33.668 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-21T10:35:33.668 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:33.668 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-03-21T10:35:33.668 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "prometheus", 2024-03-21T10:35:33.668 INFO:teuthology.orchestra.run.smithi047.stdout: "can_run": true, 2024-03-21T10:35:33.668 INFO:teuthology.orchestra.run.smithi047.stdout: "error_string": "", 2024-03-21T10:35:33.669 INFO:teuthology.orchestra.run.smithi047.stdout: "module_options": { 2024-03-21T10:35:33.669 INFO:teuthology.orchestra.run.smithi047.stdout: "cache": { 2024-03-21T10:35:33.669 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "cache", 2024-03-21T10:35:33.669 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T10:35:33.669 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:33.669 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T10:35:33.669 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "True", 2024-03-21T10:35:33.669 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:33.669 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:33.669 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:33.669 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T10:35:33.669 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:33.669 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:33.670 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:33.670 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:33.670 INFO:teuthology.orchestra.run.smithi047.stdout: "exclude_perf_counters": { 2024-03-21T10:35:33.670 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "exclude_perf_counters", 2024-03-21T10:35:33.670 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T10:35:33.670 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:33.670 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T10:35:33.670 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "True", 2024-03-21T10:35:33.670 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:33.670 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:33.670 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:33.670 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-03-21T10:35:33.670 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-03-21T10:35:33.670 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:33.671 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:33.671 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:33.671 INFO:teuthology.orchestra.run.smithi047.stdout: "log_level": { 2024-03-21T10:35:33.671 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_level", 2024-03-21T10:35:33.671 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T10:35:33.671 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:33.671 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T10:35:33.671 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-03-21T10:35:33.671 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:33.671 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:33.671 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-03-21T10:35:33.671 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-03-21T10:35:33.671 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-03-21T10:35:33.672 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-03-21T10:35:33.672 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-03-21T10:35:33.672 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-03-21T10:35:33.672 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-03-21T10:35:33.672 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-03-21T10:35:33.672 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T10:35:33.672 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:33.672 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:33.672 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:33.672 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:33.672 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster": { 2024-03-21T10:35:33.672 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster", 2024-03-21T10:35:33.672 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T10:35:33.672 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:33.673 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T10:35:33.673 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-03-21T10:35:33.673 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:33.673 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:33.673 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:33.673 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T10:35:33.673 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:33.673 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:33.673 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:33.673 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:33.673 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster_level": { 2024-03-21T10:35:33.673 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster_level", 2024-03-21T10:35:33.673 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T10:35:33.673 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:33.673 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T10:35:33.674 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "info", 2024-03-21T10:35:33.674 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:33.674 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:33.674 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-03-21T10:35:33.674 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-03-21T10:35:33.674 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-03-21T10:35:33.674 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-03-21T10:35:33.674 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-03-21T10:35:33.674 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-03-21T10:35:33.674 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-03-21T10:35:33.674 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-03-21T10:35:33.674 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T10:35:33.674 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:33.674 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:33.674 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:33.675 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:33.675 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_file": { 2024-03-21T10:35:33.675 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_file", 2024-03-21T10:35:33.675 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T10:35:33.675 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:33.675 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T10:35:33.675 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-03-21T10:35:33.675 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:33.675 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:33.675 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:33.675 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T10:35:33.675 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:33.675 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:33.675 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:33.675 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:33.676 INFO:teuthology.orchestra.run.smithi047.stdout: "rbd_stats_pools": { 2024-03-21T10:35:33.676 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "rbd_stats_pools", 2024-03-21T10:35:33.676 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T10:35:33.676 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:33.676 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T10:35:33.676 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-03-21T10:35:33.676 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:33.676 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:33.676 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:33.676 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T10:35:33.676 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:33.676 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:33.677 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:33.677 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:33.677 INFO:teuthology.orchestra.run.smithi047.stdout: "rbd_stats_pools_refresh_interval": { 2024-03-21T10:35:33.677 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-03-21T10:35:33.677 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "int", 2024-03-21T10:35:33.677 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:33.677 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T10:35:33.677 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "300", 2024-03-21T10:35:33.677 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:33.677 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:33.677 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:33.677 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T10:35:33.677 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:33.678 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:33.678 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:33.678 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:33.678 INFO:teuthology.orchestra.run.smithi047.stdout: "scrape_interval": { 2024-03-21T10:35:33.678 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "scrape_interval", 2024-03-21T10:35:33.678 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "float", 2024-03-21T10:35:33.678 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:33.678 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T10:35:33.678 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "15.0", 2024-03-21T10:35:33.678 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:33.678 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:33.678 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:33.678 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T10:35:33.678 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:33.679 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:33.679 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:33.679 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:33.679 INFO:teuthology.orchestra.run.smithi047.stdout: "server_addr": { 2024-03-21T10:35:33.679 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "server_addr", 2024-03-21T10:35:33.679 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T10:35:33.679 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:33.679 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T10:35:33.679 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "::", 2024-03-21T10:35:33.679 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:33.679 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:33.679 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:33.679 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-03-21T10:35:33.680 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:33.680 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:33.680 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:33.680 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:33.680 INFO:teuthology.orchestra.run.smithi047.stdout: "server_port": { 2024-03-21T10:35:33.680 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "server_port", 2024-03-21T10:35:33.680 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "int", 2024-03-21T10:35:33.680 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:33.680 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T10:35:33.680 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "9283", 2024-03-21T10:35:33.680 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:33.680 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:33.680 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:33.680 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-03-21T10:35:33.681 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:33.681 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:33.681 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:33.681 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:33.681 INFO:teuthology.orchestra.run.smithi047.stdout: "stale_cache_strategy": { 2024-03-21T10:35:33.681 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "stale_cache_strategy", 2024-03-21T10:35:33.681 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T10:35:33.681 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:33.681 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T10:35:33.681 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "log", 2024-03-21T10:35:33.681 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:33.681 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:33.681 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:33.681 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T10:35:33.681 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:33.682 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:33.682 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:33.682 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:33.682 INFO:teuthology.orchestra.run.smithi047.stdout: "standby_behaviour": { 2024-03-21T10:35:33.682 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "standby_behaviour", 2024-03-21T10:35:33.682 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T10:35:33.682 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:33.682 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T10:35:33.682 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "default", 2024-03-21T10:35:33.682 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:33.682 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:33.682 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-03-21T10:35:33.682 INFO:teuthology.orchestra.run.smithi047.stdout: "default", 2024-03-21T10:35:33.682 INFO:teuthology.orchestra.run.smithi047.stdout: "error" 2024-03-21T10:35:33.683 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-03-21T10:35:33.683 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T10:35:33.683 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:33.683 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:33.683 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:33.683 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:33.683 INFO:teuthology.orchestra.run.smithi047.stdout: "standby_error_status_code": { 2024-03-21T10:35:33.683 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "standby_error_status_code", 2024-03-21T10:35:33.683 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "int", 2024-03-21T10:35:33.683 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:33.683 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T10:35:33.683 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "500", 2024-03-21T10:35:33.683 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "400", 2024-03-21T10:35:33.683 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "599", 2024-03-21T10:35:33.683 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:33.684 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T10:35:33.684 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:33.684 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:33.684 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:33.684 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-21T10:35:33.684 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-21T10:35:33.684 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:33.684 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-03-21T10:35:33.684 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "rbd_support", 2024-03-21T10:35:33.684 INFO:teuthology.orchestra.run.smithi047.stdout: "can_run": true, 2024-03-21T10:35:33.684 INFO:teuthology.orchestra.run.smithi047.stdout: "error_string": "", 2024-03-21T10:35:33.684 INFO:teuthology.orchestra.run.smithi047.stdout: "module_options": { 2024-03-21T10:35:33.684 INFO:teuthology.orchestra.run.smithi047.stdout: "log_level": { 2024-03-21T10:35:33.684 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_level", 2024-03-21T10:35:33.684 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T10:35:33.685 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:33.685 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T10:35:33.685 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-03-21T10:35:33.685 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:33.685 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:33.685 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-03-21T10:35:33.685 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-03-21T10:35:33.685 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-03-21T10:35:33.685 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-03-21T10:35:33.685 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-03-21T10:35:33.685 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-03-21T10:35:33.685 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-03-21T10:35:33.685 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-03-21T10:35:33.685 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T10:35:33.686 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:33.686 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:33.686 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:33.686 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:33.686 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster": { 2024-03-21T10:35:33.686 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster", 2024-03-21T10:35:33.686 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T10:35:33.686 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:33.686 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T10:35:33.686 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-03-21T10:35:33.686 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:33.686 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:33.686 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:33.687 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T10:35:33.687 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:33.687 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:33.687 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:33.687 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:33.687 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster_level": { 2024-03-21T10:35:33.687 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster_level", 2024-03-21T10:35:33.687 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T10:35:33.687 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:33.687 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T10:35:33.687 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "info", 2024-03-21T10:35:33.687 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:33.687 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:33.687 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-03-21T10:35:33.687 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-03-21T10:35:33.688 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-03-21T10:35:33.688 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-03-21T10:35:33.688 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-03-21T10:35:33.688 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-03-21T10:35:33.688 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-03-21T10:35:33.688 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-03-21T10:35:33.688 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T10:35:33.688 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:33.688 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:33.688 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:33.688 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:33.688 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_file": { 2024-03-21T10:35:33.688 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_file", 2024-03-21T10:35:33.688 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T10:35:33.689 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:33.689 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T10:35:33.689 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-03-21T10:35:33.689 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:33.689 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:33.689 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:33.689 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T10:35:33.689 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:33.689 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:33.689 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:33.689 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:33.689 INFO:teuthology.orchestra.run.smithi047.stdout: "max_concurrent_snap_create": { 2024-03-21T10:35:33.689 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "max_concurrent_snap_create", 2024-03-21T10:35:33.689 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "int", 2024-03-21T10:35:33.690 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:33.690 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T10:35:33.690 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "10", 2024-03-21T10:35:33.690 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:33.690 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:33.690 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:33.690 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T10:35:33.690 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:33.690 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:33.690 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:33.690 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:33.690 INFO:teuthology.orchestra.run.smithi047.stdout: "mirror_snapshot_schedule": { 2024-03-21T10:35:33.690 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "mirror_snapshot_schedule", 2024-03-21T10:35:33.690 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T10:35:33.690 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:33.691 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T10:35:33.691 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-03-21T10:35:33.691 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:33.691 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:33.691 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:33.691 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T10:35:33.691 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:33.691 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:33.691 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:33.691 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:33.691 INFO:teuthology.orchestra.run.smithi047.stdout: "trash_purge_schedule": { 2024-03-21T10:35:33.691 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "trash_purge_schedule", 2024-03-21T10:35:33.691 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T10:35:33.691 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:33.691 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T10:35:33.692 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-03-21T10:35:33.692 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:33.692 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:33.692 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:33.692 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T10:35:33.692 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:33.692 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:33.692 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:33.692 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-21T10:35:33.692 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-21T10:35:33.692 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:33.692 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-03-21T10:35:33.692 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "restful", 2024-03-21T10:35:33.692 INFO:teuthology.orchestra.run.smithi047.stdout: "can_run": true, 2024-03-21T10:35:33.693 INFO:teuthology.orchestra.run.smithi047.stdout: "error_string": "", 2024-03-21T10:35:33.693 INFO:teuthology.orchestra.run.smithi047.stdout: "module_options": { 2024-03-21T10:35:33.693 INFO:teuthology.orchestra.run.smithi047.stdout: "enable_auth": { 2024-03-21T10:35:33.693 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "enable_auth", 2024-03-21T10:35:33.693 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T10:35:33.693 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:33.693 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T10:35:33.693 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "True", 2024-03-21T10:35:33.693 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:33.693 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:33.693 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:33.693 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T10:35:33.693 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:33.693 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:33.694 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:33.694 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:33.694 INFO:teuthology.orchestra.run.smithi047.stdout: "key_file": { 2024-03-21T10:35:33.694 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "key_file", 2024-03-21T10:35:33.694 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T10:35:33.694 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:33.694 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T10:35:33.694 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-03-21T10:35:33.694 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:33.694 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:33.694 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:33.694 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T10:35:33.694 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:33.694 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:33.695 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:33.695 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:33.695 INFO:teuthology.orchestra.run.smithi047.stdout: "log_level": { 2024-03-21T10:35:33.695 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_level", 2024-03-21T10:35:33.695 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T10:35:33.695 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:33.695 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T10:35:33.695 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-03-21T10:35:33.695 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:33.695 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:33.695 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-03-21T10:35:33.695 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-03-21T10:35:33.695 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-03-21T10:35:33.695 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-03-21T10:35:33.696 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-03-21T10:35:33.696 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-03-21T10:35:33.696 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-03-21T10:35:33.696 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-03-21T10:35:33.696 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T10:35:33.696 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:33.696 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:33.696 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:33.696 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:33.696 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster": { 2024-03-21T10:35:33.696 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster", 2024-03-21T10:35:33.696 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T10:35:33.696 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:33.697 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T10:35:33.697 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-03-21T10:35:33.697 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:33.697 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:33.697 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:33.697 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T10:35:33.697 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:33.697 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:33.697 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:33.697 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:33.697 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster_level": { 2024-03-21T10:35:33.697 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster_level", 2024-03-21T10:35:33.697 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T10:35:33.697 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:33.697 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T10:35:33.698 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "info", 2024-03-21T10:35:33.698 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:33.698 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:33.698 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-03-21T10:35:33.698 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-03-21T10:35:33.698 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-03-21T10:35:33.698 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-03-21T10:35:33.698 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-03-21T10:35:33.698 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-03-21T10:35:33.698 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-03-21T10:35:33.698 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-03-21T10:35:33.698 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T10:35:33.698 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:33.698 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:33.698 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:33.699 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:33.699 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_file": { 2024-03-21T10:35:33.699 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_file", 2024-03-21T10:35:33.699 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T10:35:33.699 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:33.699 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T10:35:33.699 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-03-21T10:35:33.699 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:33.699 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:33.699 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:33.699 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T10:35:33.699 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:33.699 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:33.699 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:33.700 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:33.700 INFO:teuthology.orchestra.run.smithi047.stdout: "server_addr": { 2024-03-21T10:35:33.700 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "server_addr", 2024-03-21T10:35:33.700 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T10:35:33.700 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:33.700 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T10:35:33.700 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-03-21T10:35:33.700 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:33.700 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:33.700 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:33.700 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T10:35:33.700 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:33.700 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:33.700 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:33.700 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:33.701 INFO:teuthology.orchestra.run.smithi047.stdout: "server_port": { 2024-03-21T10:35:33.701 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "server_port", 2024-03-21T10:35:33.701 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T10:35:33.701 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:33.701 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T10:35:33.701 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-03-21T10:35:33.701 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:33.701 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:33.701 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:33.701 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T10:35:33.701 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:33.701 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:33.701 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:33.701 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-21T10:35:33.701 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-21T10:35:33.702 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:33.702 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-03-21T10:35:33.702 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "rgw", 2024-03-21T10:35:33.702 INFO:teuthology.orchestra.run.smithi047.stdout: "can_run": true, 2024-03-21T10:35:33.702 INFO:teuthology.orchestra.run.smithi047.stdout: "error_string": "", 2024-03-21T10:35:33.702 INFO:teuthology.orchestra.run.smithi047.stdout: "module_options": { 2024-03-21T10:35:33.702 INFO:teuthology.orchestra.run.smithi047.stdout: "log_level": { 2024-03-21T10:35:33.702 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_level", 2024-03-21T10:35:33.702 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T10:35:33.702 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:33.702 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T10:35:33.702 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-03-21T10:35:33.702 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:33.702 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:33.703 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-03-21T10:35:33.703 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-03-21T10:35:33.703 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-03-21T10:35:33.703 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-03-21T10:35:33.703 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-03-21T10:35:33.703 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-03-21T10:35:33.703 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-03-21T10:35:33.703 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-03-21T10:35:33.703 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T10:35:33.703 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:33.703 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:33.703 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:33.703 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:33.703 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster": { 2024-03-21T10:35:33.703 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster", 2024-03-21T10:35:33.704 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T10:35:33.704 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:33.704 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T10:35:33.704 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-03-21T10:35:33.704 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:33.704 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:33.704 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:33.704 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T10:35:33.704 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:33.704 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:33.704 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:33.704 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:33.704 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster_level": { 2024-03-21T10:35:33.704 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster_level", 2024-03-21T10:35:33.705 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T10:35:33.705 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:33.705 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T10:35:33.705 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "info", 2024-03-21T10:35:33.705 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:33.705 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:33.705 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-03-21T10:35:33.705 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-03-21T10:35:33.705 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-03-21T10:35:33.705 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-03-21T10:35:33.705 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-03-21T10:35:33.705 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-03-21T10:35:33.705 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-03-21T10:35:33.705 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-03-21T10:35:33.705 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T10:35:33.706 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:33.706 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:33.706 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:33.706 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:33.706 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_file": { 2024-03-21T10:35:33.706 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_file", 2024-03-21T10:35:33.706 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T10:35:33.706 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:33.706 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T10:35:33.706 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-03-21T10:35:33.706 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:33.706 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:33.706 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:33.706 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T10:35:33.707 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:33.707 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:33.707 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:33.707 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-21T10:35:33.707 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-21T10:35:33.707 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:33.707 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-03-21T10:35:33.707 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "rook", 2024-03-21T10:35:33.707 INFO:teuthology.orchestra.run.smithi047.stdout: "can_run": true, 2024-03-21T10:35:33.707 INFO:teuthology.orchestra.run.smithi047.stdout: "error_string": "", 2024-03-21T10:35:33.707 INFO:teuthology.orchestra.run.smithi047.stdout: "module_options": { 2024-03-21T10:35:33.707 INFO:teuthology.orchestra.run.smithi047.stdout: "log_level": { 2024-03-21T10:35:33.707 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_level", 2024-03-21T10:35:33.707 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T10:35:33.707 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:33.708 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T10:35:33.708 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-03-21T10:35:33.708 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:33.708 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:33.708 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-03-21T10:35:33.708 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-03-21T10:35:33.708 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-03-21T10:35:33.708 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-03-21T10:35:33.708 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-03-21T10:35:33.708 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-03-21T10:35:33.708 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-03-21T10:35:33.708 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-03-21T10:35:33.708 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T10:35:33.708 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:33.709 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:33.709 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:33.709 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:33.709 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster": { 2024-03-21T10:35:33.709 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster", 2024-03-21T10:35:33.709 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T10:35:33.709 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:33.709 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T10:35:33.709 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-03-21T10:35:33.709 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:33.709 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:33.709 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:33.709 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T10:35:33.710 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:33.710 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:33.710 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:33.710 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:33.710 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster_level": { 2024-03-21T10:35:33.710 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster_level", 2024-03-21T10:35:33.710 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T10:35:33.710 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:33.710 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T10:35:33.710 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "info", 2024-03-21T10:35:33.710 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:33.710 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:33.710 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-03-21T10:35:33.710 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-03-21T10:35:33.711 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-03-21T10:35:33.711 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-03-21T10:35:33.711 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-03-21T10:35:33.711 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-03-21T10:35:33.711 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-03-21T10:35:33.711 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-03-21T10:35:33.711 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T10:35:33.711 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:33.711 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:33.711 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:33.711 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:33.711 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_file": { 2024-03-21T10:35:33.711 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_file", 2024-03-21T10:35:33.711 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T10:35:33.711 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:33.712 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T10:35:33.712 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-03-21T10:35:33.712 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:33.712 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:33.712 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:33.712 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T10:35:33.712 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:33.712 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:33.712 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:33.712 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:33.712 INFO:teuthology.orchestra.run.smithi047.stdout: "storage_class": { 2024-03-21T10:35:33.712 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "storage_class", 2024-03-21T10:35:33.712 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T10:35:33.712 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:33.713 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T10:35:33.713 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "local", 2024-03-21T10:35:33.713 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:33.713 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:33.713 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:33.713 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-03-21T10:35:33.713 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:33.713 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:33.713 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:33.713 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-21T10:35:33.713 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-21T10:35:33.713 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:33.713 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-03-21T10:35:33.713 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "selftest", 2024-03-21T10:35:33.714 INFO:teuthology.orchestra.run.smithi047.stdout: "can_run": true, 2024-03-21T10:35:33.714 INFO:teuthology.orchestra.run.smithi047.stdout: "error_string": "", 2024-03-21T10:35:33.714 INFO:teuthology.orchestra.run.smithi047.stdout: "module_options": { 2024-03-21T10:35:33.714 INFO:teuthology.orchestra.run.smithi047.stdout: "log_level": { 2024-03-21T10:35:33.714 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_level", 2024-03-21T10:35:33.714 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T10:35:33.714 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:33.714 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T10:35:33.714 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-03-21T10:35:33.714 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:33.714 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:33.714 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-03-21T10:35:33.714 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-03-21T10:35:33.714 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-03-21T10:35:33.714 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-03-21T10:35:33.715 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-03-21T10:35:33.715 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-03-21T10:35:33.715 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-03-21T10:35:33.715 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-03-21T10:35:33.715 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T10:35:33.715 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:33.715 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:33.715 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:33.715 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:33.715 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster": { 2024-03-21T10:35:33.715 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster", 2024-03-21T10:35:33.715 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T10:35:33.716 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:33.716 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T10:35:33.716 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-03-21T10:35:33.716 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:33.716 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:33.716 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:33.716 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T10:35:33.716 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:33.716 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:33.716 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:33.716 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:33.716 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster_level": { 2024-03-21T10:35:33.716 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster_level", 2024-03-21T10:35:33.716 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T10:35:33.717 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:33.717 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T10:35:33.717 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "info", 2024-03-21T10:35:33.717 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:33.717 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:33.717 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-03-21T10:35:33.717 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-03-21T10:35:33.717 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-03-21T10:35:33.717 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-03-21T10:35:33.717 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-03-21T10:35:33.717 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-03-21T10:35:33.717 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-03-21T10:35:33.717 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-03-21T10:35:33.717 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T10:35:33.717 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:33.718 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:33.718 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:33.718 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:33.718 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_file": { 2024-03-21T10:35:33.718 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_file", 2024-03-21T10:35:33.718 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T10:35:33.718 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:33.718 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T10:35:33.718 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-03-21T10:35:33.718 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:33.718 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:33.718 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:33.718 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T10:35:33.719 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:33.719 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:33.719 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:33.719 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:33.719 INFO:teuthology.orchestra.run.smithi047.stdout: "roption1": { 2024-03-21T10:35:33.719 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "roption1", 2024-03-21T10:35:33.719 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T10:35:33.719 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:33.719 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T10:35:33.719 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-03-21T10:35:33.719 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:33.719 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:33.719 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:33.719 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T10:35:33.720 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:33.720 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:33.720 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:33.720 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:33.720 INFO:teuthology.orchestra.run.smithi047.stdout: "roption2": { 2024-03-21T10:35:33.720 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "roption2", 2024-03-21T10:35:33.720 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T10:35:33.720 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:33.720 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T10:35:33.720 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "xyz", 2024-03-21T10:35:33.720 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:33.720 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:33.720 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:33.720 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T10:35:33.720 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:33.721 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:33.721 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:33.721 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:33.721 INFO:teuthology.orchestra.run.smithi047.stdout: "rwoption1": { 2024-03-21T10:35:33.721 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "rwoption1", 2024-03-21T10:35:33.721 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T10:35:33.721 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:33.721 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T10:35:33.721 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-03-21T10:35:33.721 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:33.721 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:33.721 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:33.722 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T10:35:33.722 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:33.722 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:33.722 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:33.722 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:33.722 INFO:teuthology.orchestra.run.smithi047.stdout: "rwoption2": { 2024-03-21T10:35:33.722 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "rwoption2", 2024-03-21T10:35:33.722 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "int", 2024-03-21T10:35:33.722 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:33.722 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T10:35:33.722 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-03-21T10:35:33.722 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:33.723 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:33.723 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:33.723 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T10:35:33.723 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:33.723 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:33.723 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:33.723 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:33.723 INFO:teuthology.orchestra.run.smithi047.stdout: "rwoption3": { 2024-03-21T10:35:33.723 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "rwoption3", 2024-03-21T10:35:33.723 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "float", 2024-03-21T10:35:33.723 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:33.723 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T10:35:33.723 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-03-21T10:35:33.724 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:33.724 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:33.724 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:33.724 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T10:35:33.724 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:33.724 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:33.724 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:33.724 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:33.724 INFO:teuthology.orchestra.run.smithi047.stdout: "rwoption4": { 2024-03-21T10:35:33.724 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "rwoption4", 2024-03-21T10:35:33.724 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T10:35:33.724 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:33.724 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T10:35:33.724 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-03-21T10:35:33.725 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:33.725 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:33.725 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:33.725 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T10:35:33.725 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:33.725 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:33.725 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:33.725 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:33.725 INFO:teuthology.orchestra.run.smithi047.stdout: "rwoption5": { 2024-03-21T10:35:33.725 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "rwoption5", 2024-03-21T10:35:33.725 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T10:35:33.725 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:33.725 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T10:35:33.725 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-03-21T10:35:33.726 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:33.726 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:33.726 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:33.726 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T10:35:33.726 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:33.726 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:33.726 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:33.726 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:33.726 INFO:teuthology.orchestra.run.smithi047.stdout: "rwoption6": { 2024-03-21T10:35:33.726 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "rwoption6", 2024-03-21T10:35:33.726 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T10:35:33.726 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:33.726 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T10:35:33.727 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "True", 2024-03-21T10:35:33.727 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:33.727 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:33.727 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:33.727 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T10:35:33.727 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:33.727 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:33.727 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:33.727 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:33.727 INFO:teuthology.orchestra.run.smithi047.stdout: "rwoption7": { 2024-03-21T10:35:33.727 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "rwoption7", 2024-03-21T10:35:33.727 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "int", 2024-03-21T10:35:33.727 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:33.728 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T10:35:33.728 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-03-21T10:35:33.728 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "1", 2024-03-21T10:35:33.728 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "42", 2024-03-21T10:35:33.728 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:33.728 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T10:35:33.728 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:33.728 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:33.728 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:33.728 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:33.728 INFO:teuthology.orchestra.run.smithi047.stdout: "testkey": { 2024-03-21T10:35:33.728 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "testkey", 2024-03-21T10:35:33.728 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T10:35:33.728 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:33.729 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T10:35:33.729 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-03-21T10:35:33.729 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:33.729 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:33.729 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:33.729 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T10:35:33.729 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:33.729 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:33.729 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:33.729 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:33.729 INFO:teuthology.orchestra.run.smithi047.stdout: "testlkey": { 2024-03-21T10:35:33.729 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "testlkey", 2024-03-21T10:35:33.729 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T10:35:33.729 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:33.730 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T10:35:33.730 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-03-21T10:35:33.730 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:33.730 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:33.730 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:33.730 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T10:35:33.730 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:33.730 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:33.730 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:33.730 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:33.730 INFO:teuthology.orchestra.run.smithi047.stdout: "testnewline": { 2024-03-21T10:35:33.730 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "testnewline", 2024-03-21T10:35:33.730 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T10:35:33.731 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:33.731 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T10:35:33.731 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-03-21T10:35:33.731 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:33.731 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:33.731 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:33.731 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T10:35:33.731 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:33.731 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:33.731 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:33.731 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-21T10:35:33.731 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-21T10:35:33.731 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:33.731 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-03-21T10:35:33.731 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "snap_schedule", 2024-03-21T10:35:33.732 INFO:teuthology.orchestra.run.smithi047.stdout: "can_run": true, 2024-03-21T10:35:33.732 INFO:teuthology.orchestra.run.smithi047.stdout: "error_string": "", 2024-03-21T10:35:33.732 INFO:teuthology.orchestra.run.smithi047.stdout: "module_options": { 2024-03-21T10:35:33.732 INFO:teuthology.orchestra.run.smithi047.stdout: "allow_m_granularity": { 2024-03-21T10:35:33.732 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "allow_m_granularity", 2024-03-21T10:35:33.732 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T10:35:33.732 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:33.732 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T10:35:33.732 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-03-21T10:35:33.732 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:33.732 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:33.732 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:33.732 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "allow minute scheduled snapshots", 2024-03-21T10:35:33.732 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:33.732 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:33.733 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:33.733 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:33.733 INFO:teuthology.orchestra.run.smithi047.stdout: "dump_on_update": { 2024-03-21T10:35:33.733 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "dump_on_update", 2024-03-21T10:35:33.733 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T10:35:33.733 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:33.733 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T10:35:33.733 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-03-21T10:35:33.733 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:33.733 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:33.733 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:33.733 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "dump database to debug log on update", 2024-03-21T10:35:33.733 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:33.733 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:33.734 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:33.734 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:33.734 INFO:teuthology.orchestra.run.smithi047.stdout: "log_level": { 2024-03-21T10:35:33.734 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_level", 2024-03-21T10:35:33.734 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T10:35:33.734 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:33.734 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T10:35:33.734 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-03-21T10:35:33.734 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:33.734 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:33.734 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-03-21T10:35:33.734 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-03-21T10:35:33.734 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-03-21T10:35:33.734 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-03-21T10:35:33.735 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-03-21T10:35:33.735 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-03-21T10:35:33.735 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-03-21T10:35:33.735 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-03-21T10:35:33.735 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T10:35:33.735 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:33.735 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:33.735 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:33.735 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:33.735 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster": { 2024-03-21T10:35:33.735 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster", 2024-03-21T10:35:33.735 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T10:35:33.735 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:33.735 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T10:35:33.735 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-03-21T10:35:33.736 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:33.736 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:33.736 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:33.736 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T10:35:33.736 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:33.736 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:33.736 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:33.736 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:33.736 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster_level": { 2024-03-21T10:35:33.736 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster_level", 2024-03-21T10:35:33.736 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T10:35:33.736 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:33.736 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T10:35:33.736 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "info", 2024-03-21T10:35:33.737 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:33.737 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:33.737 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-03-21T10:35:33.737 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-03-21T10:35:33.737 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-03-21T10:35:33.737 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-03-21T10:35:33.737 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-03-21T10:35:33.737 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-03-21T10:35:33.737 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-03-21T10:35:33.737 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-03-21T10:35:33.737 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T10:35:33.737 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:33.737 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:33.737 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:33.737 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:33.738 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_file": { 2024-03-21T10:35:33.738 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_file", 2024-03-21T10:35:33.738 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T10:35:33.738 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:33.738 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T10:35:33.738 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-03-21T10:35:33.738 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:33.738 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:33.738 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:33.738 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T10:35:33.738 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:33.738 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:33.738 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:33.739 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-21T10:35:33.739 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-21T10:35:33.739 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:33.739 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-03-21T10:35:33.739 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "stats", 2024-03-21T10:35:33.739 INFO:teuthology.orchestra.run.smithi047.stdout: "can_run": true, 2024-03-21T10:35:33.739 INFO:teuthology.orchestra.run.smithi047.stdout: "error_string": "", 2024-03-21T10:35:33.739 INFO:teuthology.orchestra.run.smithi047.stdout: "module_options": { 2024-03-21T10:35:33.739 INFO:teuthology.orchestra.run.smithi047.stdout: "log_level": { 2024-03-21T10:35:33.739 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_level", 2024-03-21T10:35:33.739 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T10:35:33.739 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:33.739 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T10:35:33.739 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-03-21T10:35:33.739 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:33.739 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:33.740 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-03-21T10:35:33.740 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-03-21T10:35:33.740 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-03-21T10:35:33.740 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-03-21T10:35:33.740 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-03-21T10:35:33.740 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-03-21T10:35:33.740 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-03-21T10:35:33.740 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-03-21T10:35:33.740 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T10:35:33.740 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:33.740 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:33.740 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:33.740 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:33.740 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster": { 2024-03-21T10:35:33.740 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster", 2024-03-21T10:35:33.740 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T10:35:33.741 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:33.741 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T10:35:33.741 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-03-21T10:35:33.741 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:33.741 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:33.741 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:33.741 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T10:35:33.741 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:33.741 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:33.741 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:33.741 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:33.741 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster_level": { 2024-03-21T10:35:33.741 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster_level", 2024-03-21T10:35:33.741 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T10:35:33.742 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:33.742 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T10:35:33.742 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "info", 2024-03-21T10:35:33.742 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:33.742 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:33.742 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-03-21T10:35:33.742 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-03-21T10:35:33.742 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-03-21T10:35:33.742 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-03-21T10:35:33.742 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-03-21T10:35:33.742 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-03-21T10:35:33.742 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-03-21T10:35:33.742 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-03-21T10:35:33.742 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T10:35:33.743 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:33.743 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:33.743 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:33.743 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:33.743 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_file": { 2024-03-21T10:35:33.743 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_file", 2024-03-21T10:35:33.743 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T10:35:33.743 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:33.743 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T10:35:33.743 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-03-21T10:35:33.743 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:33.743 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:33.743 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:33.743 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T10:35:33.743 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:33.744 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:33.744 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:33.744 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-21T10:35:33.744 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-21T10:35:33.744 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:33.744 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-03-21T10:35:33.744 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "status", 2024-03-21T10:35:33.744 INFO:teuthology.orchestra.run.smithi047.stdout: "can_run": true, 2024-03-21T10:35:33.744 INFO:teuthology.orchestra.run.smithi047.stdout: "error_string": "", 2024-03-21T10:35:33.744 INFO:teuthology.orchestra.run.smithi047.stdout: "module_options": { 2024-03-21T10:35:33.744 INFO:teuthology.orchestra.run.smithi047.stdout: "log_level": { 2024-03-21T10:35:33.744 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_level", 2024-03-21T10:35:33.744 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T10:35:33.744 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:33.744 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T10:35:33.745 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-03-21T10:35:33.745 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:33.745 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:33.745 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-03-21T10:35:33.745 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-03-21T10:35:33.745 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-03-21T10:35:33.745 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-03-21T10:35:33.745 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-03-21T10:35:33.745 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-03-21T10:35:33.745 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-03-21T10:35:33.745 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-03-21T10:35:33.745 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T10:35:33.745 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:33.745 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:33.745 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:33.746 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:33.746 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster": { 2024-03-21T10:35:33.746 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster", 2024-03-21T10:35:33.746 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T10:35:33.746 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:33.746 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T10:35:33.746 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-03-21T10:35:33.746 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:33.746 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:33.746 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:33.746 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T10:35:33.746 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:33.746 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:33.746 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:33.746 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:33.747 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster_level": { 2024-03-21T10:35:33.747 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster_level", 2024-03-21T10:35:33.747 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T10:35:33.747 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:33.747 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T10:35:33.747 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "info", 2024-03-21T10:35:33.747 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:33.747 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:33.747 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-03-21T10:35:33.747 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-03-21T10:35:33.747 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-03-21T10:35:33.747 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-03-21T10:35:33.747 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-03-21T10:35:33.747 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-03-21T10:35:33.748 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-03-21T10:35:33.748 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-03-21T10:35:33.748 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T10:35:33.748 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:33.748 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:33.748 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:33.748 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:33.748 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_file": { 2024-03-21T10:35:33.748 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_file", 2024-03-21T10:35:33.748 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T10:35:33.748 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:33.748 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T10:35:33.748 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-03-21T10:35:33.748 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:33.748 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:33.749 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:33.749 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T10:35:33.749 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:33.749 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:33.749 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:33.749 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-21T10:35:33.749 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-21T10:35:33.749 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:33.749 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-03-21T10:35:33.749 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "telegraf", 2024-03-21T10:35:33.749 INFO:teuthology.orchestra.run.smithi047.stdout: "can_run": true, 2024-03-21T10:35:33.749 INFO:teuthology.orchestra.run.smithi047.stdout: "error_string": "", 2024-03-21T10:35:33.749 INFO:teuthology.orchestra.run.smithi047.stdout: "module_options": { 2024-03-21T10:35:33.749 INFO:teuthology.orchestra.run.smithi047.stdout: "address": { 2024-03-21T10:35:33.750 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "address", 2024-03-21T10:35:33.750 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T10:35:33.750 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:33.750 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T10:35:33.750 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-03-21T10:35:33.750 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:33.750 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:33.750 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:33.750 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T10:35:33.750 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:33.750 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:33.750 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:33.750 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:33.750 INFO:teuthology.orchestra.run.smithi047.stdout: "interval": { 2024-03-21T10:35:33.751 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "interval", 2024-03-21T10:35:33.751 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "secs", 2024-03-21T10:35:33.751 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:33.751 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T10:35:33.751 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "15", 2024-03-21T10:35:33.751 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:33.751 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:33.751 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:33.751 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T10:35:33.751 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:33.751 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:33.751 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:33.751 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:33.751 INFO:teuthology.orchestra.run.smithi047.stdout: "log_level": { 2024-03-21T10:35:33.751 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_level", 2024-03-21T10:35:33.752 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T10:35:33.752 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:33.752 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T10:35:33.752 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-03-21T10:35:33.752 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:33.752 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:33.752 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-03-21T10:35:33.752 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-03-21T10:35:33.752 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-03-21T10:35:33.752 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-03-21T10:35:33.752 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-03-21T10:35:33.752 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-03-21T10:35:33.752 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-03-21T10:35:33.752 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-03-21T10:35:33.752 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T10:35:33.753 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:33.753 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:33.753 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:33.753 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:33.753 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster": { 2024-03-21T10:35:33.753 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster", 2024-03-21T10:35:33.753 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T10:35:33.753 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:33.753 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T10:35:33.753 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-03-21T10:35:33.753 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:33.753 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:33.753 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:33.754 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T10:35:33.754 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:33.754 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:33.754 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:33.754 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:33.754 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster_level": { 2024-03-21T10:35:33.754 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster_level", 2024-03-21T10:35:33.754 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T10:35:33.754 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:33.754 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T10:35:33.754 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "info", 2024-03-21T10:35:33.754 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:33.754 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:33.754 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-03-21T10:35:33.755 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-03-21T10:35:33.755 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-03-21T10:35:33.755 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-03-21T10:35:33.755 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-03-21T10:35:33.755 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-03-21T10:35:33.755 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-03-21T10:35:33.755 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-03-21T10:35:33.755 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T10:35:33.755 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:33.755 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:33.755 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:33.755 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:33.755 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_file": { 2024-03-21T10:35:33.755 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_file", 2024-03-21T10:35:33.756 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T10:35:33.756 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:33.756 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T10:35:33.756 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-03-21T10:35:33.756 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:33.756 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:33.756 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:33.756 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T10:35:33.756 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:33.756 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:33.756 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:33.756 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-21T10:35:33.756 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-21T10:35:33.756 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:33.757 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-03-21T10:35:33.757 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "telemetry", 2024-03-21T10:35:33.757 INFO:teuthology.orchestra.run.smithi047.stdout: "can_run": true, 2024-03-21T10:35:33.757 INFO:teuthology.orchestra.run.smithi047.stdout: "error_string": "", 2024-03-21T10:35:33.757 INFO:teuthology.orchestra.run.smithi047.stdout: "module_options": { 2024-03-21T10:35:33.757 INFO:teuthology.orchestra.run.smithi047.stdout: "channel_basic": { 2024-03-21T10:35:33.757 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "channel_basic", 2024-03-21T10:35:33.757 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T10:35:33.757 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:33.757 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T10:35:33.757 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "True", 2024-03-21T10:35:33.757 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:33.757 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:33.757 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:33.758 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Share basic cluster information (size, version)", 2024-03-21T10:35:33.758 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:33.758 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:33.758 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:33.758 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:33.758 INFO:teuthology.orchestra.run.smithi047.stdout: "channel_crash": { 2024-03-21T10:35:33.758 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "channel_crash", 2024-03-21T10:35:33.758 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T10:35:33.758 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:33.758 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T10:35:33.758 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "True", 2024-03-21T10:35:33.758 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:33.758 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:33.758 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:33.759 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-03-21T10:35:33.759 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:33.759 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:33.759 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:33.759 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:33.759 INFO:teuthology.orchestra.run.smithi047.stdout: "channel_device": { 2024-03-21T10:35:33.759 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "channel_device", 2024-03-21T10:35:33.759 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T10:35:33.759 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:33.759 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T10:35:33.759 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "True", 2024-03-21T10:35:33.759 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:33.759 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:33.759 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:33.760 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-03-21T10:35:33.760 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:33.760 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:33.760 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:33.760 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:33.760 INFO:teuthology.orchestra.run.smithi047.stdout: "channel_ident": { 2024-03-21T10:35:33.760 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "channel_ident", 2024-03-21T10:35:33.760 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T10:35:33.760 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:33.760 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T10:35:33.760 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-03-21T10:35:33.760 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:33.760 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:33.760 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:33.760 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-03-21T10:35:33.761 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:33.761 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:33.761 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:33.761 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:33.761 INFO:teuthology.orchestra.run.smithi047.stdout: "channel_perf": { 2024-03-21T10:35:33.761 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "channel_perf", 2024-03-21T10:35:33.761 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T10:35:33.761 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:33.761 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T10:35:33.761 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-03-21T10:35:33.761 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:33.761 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:33.761 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:33.761 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Share various performance metrics of a cluster", 2024-03-21T10:35:33.761 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:33.762 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:33.762 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:33.762 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:33.762 INFO:teuthology.orchestra.run.smithi047.stdout: "contact": { 2024-03-21T10:35:33.762 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "contact", 2024-03-21T10:35:33.762 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T10:35:33.762 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:33.762 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T10:35:33.762 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-03-21T10:35:33.762 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:33.762 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:33.762 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:33.762 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T10:35:33.762 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:33.763 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:33.763 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:33.763 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:33.763 INFO:teuthology.orchestra.run.smithi047.stdout: "description": { 2024-03-21T10:35:33.763 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "description", 2024-03-21T10:35:33.763 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T10:35:33.763 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:33.763 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T10:35:33.763 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-03-21T10:35:33.763 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:33.763 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:33.763 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:33.763 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T10:35:33.763 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:33.764 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:33.764 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:33.764 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:33.764 INFO:teuthology.orchestra.run.smithi047.stdout: "device_url": { 2024-03-21T10:35:33.764 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "device_url", 2024-03-21T10:35:33.764 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T10:35:33.764 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:33.764 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T10:35:33.764 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-03-21T10:35:33.764 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:33.764 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:33.764 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:33.764 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T10:35:33.764 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:33.764 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:33.764 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:33.765 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:33.765 INFO:teuthology.orchestra.run.smithi047.stdout: "enabled": { 2024-03-21T10:35:33.765 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "enabled", 2024-03-21T10:35:33.765 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T10:35:33.765 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:33.765 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T10:35:33.765 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-03-21T10:35:33.765 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:33.765 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:33.765 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:33.765 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T10:35:33.765 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:33.765 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:33.765 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:33.765 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:33.766 INFO:teuthology.orchestra.run.smithi047.stdout: "interval": { 2024-03-21T10:35:33.766 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "interval", 2024-03-21T10:35:33.766 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "int", 2024-03-21T10:35:33.766 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:33.766 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T10:35:33.766 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "24", 2024-03-21T10:35:33.766 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "8", 2024-03-21T10:35:33.766 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:33.766 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:33.766 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T10:35:33.766 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:33.766 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:33.766 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:33.767 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:33.767 INFO:teuthology.orchestra.run.smithi047.stdout: "last_opt_revision": { 2024-03-21T10:35:33.767 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "last_opt_revision", 2024-03-21T10:35:33.767 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "int", 2024-03-21T10:35:33.767 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:33.767 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T10:35:33.767 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "1", 2024-03-21T10:35:33.767 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:33.767 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:33.767 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:33.767 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T10:35:33.767 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:33.767 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:33.767 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:33.768 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:33.768 INFO:teuthology.orchestra.run.smithi047.stdout: "leaderboard": { 2024-03-21T10:35:33.768 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "leaderboard", 2024-03-21T10:35:33.768 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T10:35:33.768 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:33.768 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T10:35:33.768 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-03-21T10:35:33.768 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:33.768 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:33.768 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:33.768 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T10:35:33.768 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:33.768 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:33.768 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:33.768 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:33.769 INFO:teuthology.orchestra.run.smithi047.stdout: "leaderboard_description": { 2024-03-21T10:35:33.769 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "leaderboard_description", 2024-03-21T10:35:33.769 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T10:35:33.769 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:33.769 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T10:35:33.769 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-03-21T10:35:33.769 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:33.769 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:33.769 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:33.769 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T10:35:33.769 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:33.769 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:33.769 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:33.770 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:33.770 INFO:teuthology.orchestra.run.smithi047.stdout: "log_level": { 2024-03-21T10:35:33.770 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_level", 2024-03-21T10:35:33.770 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T10:35:33.770 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:33.770 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T10:35:33.770 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-03-21T10:35:33.770 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:33.770 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:33.770 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-03-21T10:35:33.770 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-03-21T10:35:33.770 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-03-21T10:35:33.770 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-03-21T10:35:33.771 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-03-21T10:35:33.771 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-03-21T10:35:33.771 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-03-21T10:35:33.771 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-03-21T10:35:33.771 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T10:35:33.771 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:33.771 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:33.771 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:33.771 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:33.771 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster": { 2024-03-21T10:35:33.771 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster", 2024-03-21T10:35:33.771 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T10:35:33.771 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:33.772 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T10:35:33.772 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-03-21T10:35:33.772 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:33.772 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:33.772 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:33.772 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T10:35:33.772 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:33.772 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:33.772 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:33.772 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:33.773 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster_level": { 2024-03-21T10:35:33.773 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster_level", 2024-03-21T10:35:33.773 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T10:35:33.773 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:33.773 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T10:35:33.773 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "info", 2024-03-21T10:35:33.773 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:33.773 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:33.773 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-03-21T10:35:33.773 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-03-21T10:35:33.774 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-03-21T10:35:33.774 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-03-21T10:35:33.774 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-03-21T10:35:33.774 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-03-21T10:35:33.774 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-03-21T10:35:33.774 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-03-21T10:35:33.775 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T10:35:33.775 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:33.775 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:33.775 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:33.775 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:33.775 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_file": { 2024-03-21T10:35:33.775 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_file", 2024-03-21T10:35:33.775 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T10:35:33.775 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:33.776 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T10:35:33.776 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-03-21T10:35:33.776 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:33.776 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:33.776 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:33.776 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T10:35:33.776 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:33.776 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:33.776 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:33.776 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:33.776 INFO:teuthology.orchestra.run.smithi047.stdout: "organization": { 2024-03-21T10:35:33.776 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "organization", 2024-03-21T10:35:33.776 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T10:35:33.776 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:33.777 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T10:35:33.777 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-03-21T10:35:33.777 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:33.777 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:33.777 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:33.777 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T10:35:33.777 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:33.777 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:33.777 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:33.777 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:33.777 INFO:teuthology.orchestra.run.smithi047.stdout: "proxy": { 2024-03-21T10:35:33.777 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "proxy", 2024-03-21T10:35:33.777 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T10:35:33.777 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:33.778 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T10:35:33.778 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-03-21T10:35:33.778 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:33.778 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:33.778 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:33.778 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T10:35:33.778 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:33.778 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:33.778 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:33.778 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:33.778 INFO:teuthology.orchestra.run.smithi047.stdout: "url": { 2024-03-21T10:35:33.778 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "url", 2024-03-21T10:35:33.778 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T10:35:33.778 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:33.779 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T10:35:33.779 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-03-21T10:35:33.779 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:33.779 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:33.779 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:33.779 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T10:35:33.779 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:33.779 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:33.779 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:33.779 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-21T10:35:33.779 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-21T10:35:33.779 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:33.779 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-03-21T10:35:33.779 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "test_orchestrator", 2024-03-21T10:35:33.780 INFO:teuthology.orchestra.run.smithi047.stdout: "can_run": true, 2024-03-21T10:35:33.780 INFO:teuthology.orchestra.run.smithi047.stdout: "error_string": "", 2024-03-21T10:35:33.780 INFO:teuthology.orchestra.run.smithi047.stdout: "module_options": { 2024-03-21T10:35:33.780 INFO:teuthology.orchestra.run.smithi047.stdout: "log_level": { 2024-03-21T10:35:33.780 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_level", 2024-03-21T10:35:33.780 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T10:35:33.780 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:33.780 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T10:35:33.780 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-03-21T10:35:33.780 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:33.780 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:33.780 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-03-21T10:35:33.780 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-03-21T10:35:33.781 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-03-21T10:35:33.781 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-03-21T10:35:33.781 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-03-21T10:35:33.781 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-03-21T10:35:33.781 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-03-21T10:35:33.781 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-03-21T10:35:33.781 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T10:35:33.781 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:33.781 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:33.781 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:33.781 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:33.781 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster": { 2024-03-21T10:35:33.781 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster", 2024-03-21T10:35:33.781 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T10:35:33.781 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:33.781 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T10:35:33.782 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-03-21T10:35:33.782 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:33.782 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:33.782 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:33.782 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T10:35:33.782 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:33.782 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:33.782 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:33.782 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:33.782 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster_level": { 2024-03-21T10:35:33.782 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster_level", 2024-03-21T10:35:33.782 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T10:35:33.782 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:33.783 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T10:35:33.783 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "info", 2024-03-21T10:35:33.783 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:33.783 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:33.783 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-03-21T10:35:33.783 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-03-21T10:35:33.783 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-03-21T10:35:33.783 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-03-21T10:35:33.783 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-03-21T10:35:33.783 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-03-21T10:35:33.783 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-03-21T10:35:33.783 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-03-21T10:35:33.783 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T10:35:33.783 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:33.783 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:33.784 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:33.784 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:33.784 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_file": { 2024-03-21T10:35:33.784 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_file", 2024-03-21T10:35:33.784 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T10:35:33.784 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:33.784 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T10:35:33.784 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-03-21T10:35:33.784 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:33.784 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:33.784 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:33.784 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T10:35:33.784 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:33.784 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:33.785 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:33.785 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-21T10:35:33.785 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-21T10:35:33.785 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:33.785 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-03-21T10:35:33.785 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "volumes", 2024-03-21T10:35:33.785 INFO:teuthology.orchestra.run.smithi047.stdout: "can_run": true, 2024-03-21T10:35:33.785 INFO:teuthology.orchestra.run.smithi047.stdout: "error_string": "", 2024-03-21T10:35:33.785 INFO:teuthology.orchestra.run.smithi047.stdout: "module_options": { 2024-03-21T10:35:33.785 INFO:teuthology.orchestra.run.smithi047.stdout: "log_level": { 2024-03-21T10:35:33.785 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_level", 2024-03-21T10:35:33.785 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T10:35:33.785 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:33.785 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T10:35:33.785 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-03-21T10:35:33.786 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:33.786 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:33.786 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-03-21T10:35:33.786 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-03-21T10:35:33.786 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-03-21T10:35:33.786 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-03-21T10:35:33.786 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-03-21T10:35:33.786 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-03-21T10:35:33.786 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-03-21T10:35:33.786 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-03-21T10:35:33.786 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T10:35:33.786 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:33.786 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:33.786 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:33.786 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:33.787 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster": { 2024-03-21T10:35:33.787 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster", 2024-03-21T10:35:33.787 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T10:35:33.787 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:33.787 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T10:35:33.787 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-03-21T10:35:33.787 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:33.787 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:33.787 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:33.787 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T10:35:33.787 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:33.787 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:33.787 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:33.787 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:33.788 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster_level": { 2024-03-21T10:35:33.788 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster_level", 2024-03-21T10:35:33.788 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T10:35:33.788 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:33.788 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T10:35:33.788 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "info", 2024-03-21T10:35:33.788 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:33.788 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:33.788 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-03-21T10:35:33.788 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-03-21T10:35:33.788 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-03-21T10:35:33.788 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-03-21T10:35:33.788 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-03-21T10:35:33.788 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-03-21T10:35:33.788 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-03-21T10:35:33.789 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-03-21T10:35:33.789 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T10:35:33.789 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:33.789 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:33.789 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:33.789 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:33.789 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_file": { 2024-03-21T10:35:33.789 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_file", 2024-03-21T10:35:33.789 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T10:35:33.789 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:33.789 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T10:35:33.789 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-03-21T10:35:33.789 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:33.790 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:33.790 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:33.790 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T10:35:33.790 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:33.790 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:33.790 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:33.790 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:33.790 INFO:teuthology.orchestra.run.smithi047.stdout: "max_concurrent_clones": { 2024-03-21T10:35:33.790 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "max_concurrent_clones", 2024-03-21T10:35:33.790 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "int", 2024-03-21T10:35:33.790 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:33.790 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T10:35:33.790 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "4", 2024-03-21T10:35:33.790 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:33.791 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:33.791 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:33.791 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Number of asynchronous cloner threads", 2024-03-21T10:35:33.791 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:33.791 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:33.791 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:33.791 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:33.791 INFO:teuthology.orchestra.run.smithi047.stdout: "periodic_async_work": { 2024-03-21T10:35:33.791 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "periodic_async_work", 2024-03-21T10:35:33.791 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T10:35:33.791 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:33.791 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T10:35:33.791 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-03-21T10:35:33.791 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:33.791 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:33.792 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:33.792 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Periodically check for async work", 2024-03-21T10:35:33.792 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:33.792 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:33.792 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:33.792 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:33.792 INFO:teuthology.orchestra.run.smithi047.stdout: "snapshot_clone_delay": { 2024-03-21T10:35:33.792 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "snapshot_clone_delay", 2024-03-21T10:35:33.792 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "int", 2024-03-21T10:35:33.792 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:33.792 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T10:35:33.792 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "0", 2024-03-21T10:35:33.792 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:33.792 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:33.792 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:33.793 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-03-21T10:35:33.793 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:33.793 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:33.793 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:33.793 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:33.793 INFO:teuthology.orchestra.run.smithi047.stdout: "snapshot_clone_no_wait": { 2024-03-21T10:35:33.793 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "snapshot_clone_no_wait", 2024-03-21T10:35:33.793 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T10:35:33.793 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:33.793 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T10:35:33.793 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "True", 2024-03-21T10:35:33.793 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:33.793 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:33.793 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:33.793 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-03-21T10:35:33.794 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:33.794 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:33.794 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:33.794 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-21T10:35:33.794 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-21T10:35:33.794 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:33.794 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-03-21T10:35:33.794 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "zabbix", 2024-03-21T10:35:33.794 INFO:teuthology.orchestra.run.smithi047.stdout: "can_run": true, 2024-03-21T10:35:33.794 INFO:teuthology.orchestra.run.smithi047.stdout: "error_string": "", 2024-03-21T10:35:33.794 INFO:teuthology.orchestra.run.smithi047.stdout: "module_options": { 2024-03-21T10:35:33.794 INFO:teuthology.orchestra.run.smithi047.stdout: "discovery_interval": { 2024-03-21T10:35:33.794 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "discovery_interval", 2024-03-21T10:35:33.794 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "uint", 2024-03-21T10:35:33.795 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:33.795 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T10:35:33.795 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "100", 2024-03-21T10:35:33.795 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:33.795 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:33.795 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:33.795 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T10:35:33.795 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:33.795 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:33.795 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:33.795 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:33.795 INFO:teuthology.orchestra.run.smithi047.stdout: "identifier": { 2024-03-21T10:35:33.795 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "identifier", 2024-03-21T10:35:33.795 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T10:35:33.796 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:33.796 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T10:35:33.796 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-03-21T10:35:33.796 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:33.796 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:33.796 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:33.796 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T10:35:33.796 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:33.796 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:33.796 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:33.796 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:33.796 INFO:teuthology.orchestra.run.smithi047.stdout: "interval": { 2024-03-21T10:35:33.796 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "interval", 2024-03-21T10:35:33.796 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "secs", 2024-03-21T10:35:33.796 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:33.797 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T10:35:33.797 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "60", 2024-03-21T10:35:33.797 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:33.797 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:33.797 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:33.797 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T10:35:33.797 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:33.797 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:33.797 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:33.797 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:33.797 INFO:teuthology.orchestra.run.smithi047.stdout: "log_level": { 2024-03-21T10:35:33.797 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_level", 2024-03-21T10:35:33.797 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T10:35:33.797 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:33.797 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T10:35:33.798 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-03-21T10:35:33.798 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:33.798 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:33.798 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-03-21T10:35:33.798 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-03-21T10:35:33.798 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-03-21T10:35:33.798 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-03-21T10:35:33.798 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-03-21T10:35:33.798 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-03-21T10:35:33.798 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-03-21T10:35:33.798 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-03-21T10:35:33.798 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T10:35:33.798 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:33.799 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:33.799 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:33.799 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:33.799 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster": { 2024-03-21T10:35:33.799 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster", 2024-03-21T10:35:33.799 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T10:35:33.799 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:33.799 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T10:35:33.799 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-03-21T10:35:33.799 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:33.799 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:33.799 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:33.799 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T10:35:33.799 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:33.800 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:33.800 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:33.800 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:33.800 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster_level": { 2024-03-21T10:35:33.800 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster_level", 2024-03-21T10:35:33.800 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T10:35:33.800 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:33.800 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T10:35:33.800 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "info", 2024-03-21T10:35:33.800 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:33.800 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:33.800 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-03-21T10:35:33.801 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-03-21T10:35:33.801 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-03-21T10:35:33.801 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-03-21T10:35:33.801 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-03-21T10:35:33.801 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-03-21T10:35:33.801 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-03-21T10:35:33.801 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-03-21T10:35:33.801 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T10:35:33.801 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:33.801 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:33.801 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:33.801 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:33.801 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_file": { 2024-03-21T10:35:33.801 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_file", 2024-03-21T10:35:33.802 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-03-21T10:35:33.802 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:33.802 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-03-21T10:35:33.802 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-03-21T10:35:33.802 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:33.802 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:33.802 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:33.802 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T10:35:33.802 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:33.802 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:33.802 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:33.802 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:33.802 INFO:teuthology.orchestra.run.smithi047.stdout: "zabbix_host": { 2024-03-21T10:35:33.803 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "zabbix_host", 2024-03-21T10:35:33.803 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T10:35:33.803 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:33.803 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T10:35:33.803 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-03-21T10:35:33.803 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:33.803 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:33.803 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:33.803 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T10:35:33.803 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:33.803 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:33.803 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:33.803 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:33.803 INFO:teuthology.orchestra.run.smithi047.stdout: "zabbix_port": { 2024-03-21T10:35:33.804 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "zabbix_port", 2024-03-21T10:35:33.804 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "int", 2024-03-21T10:35:33.804 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:33.804 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T10:35:33.804 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "10051", 2024-03-21T10:35:33.804 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:33.804 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:33.804 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:33.804 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T10:35:33.804 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:33.804 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:33.804 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:33.804 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:33.804 INFO:teuthology.orchestra.run.smithi047.stdout: "zabbix_sender": { 2024-03-21T10:35:33.805 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "zabbix_sender", 2024-03-21T10:35:33.805 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-03-21T10:35:33.805 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-03-21T10:35:33.805 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-03-21T10:35:33.805 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-03-21T10:35:33.805 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-03-21T10:35:33.805 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-03-21T10:35:33.805 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-03-21T10:35:33.805 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-03-21T10:35:33.805 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-03-21T10:35:33.805 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-03-21T10:35:33.805 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-03-21T10:35:33.805 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-21T10:35:33.805 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-21T10:35:33.805 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-21T10:35:33.806 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-03-21T10:35:33.806 INFO:teuthology.orchestra.run.smithi047.stdout: "services": { 2024-03-21T10:35:33.806 INFO:teuthology.orchestra.run.smithi047.stdout: "dashboard": "https://172.21.15.47:8443/" 2024-03-21T10:35:33.806 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:33.806 INFO:teuthology.orchestra.run.smithi047.stdout: "always_on_modules": { 2024-03-21T10:35:33.806 INFO:teuthology.orchestra.run.smithi047.stdout: "octopus": [ 2024-03-21T10:35:33.806 INFO:teuthology.orchestra.run.smithi047.stdout: "balancer", 2024-03-21T10:35:33.806 INFO:teuthology.orchestra.run.smithi047.stdout: "crash", 2024-03-21T10:35:33.806 INFO:teuthology.orchestra.run.smithi047.stdout: "devicehealth", 2024-03-21T10:35:33.806 INFO:teuthology.orchestra.run.smithi047.stdout: "orchestrator", 2024-03-21T10:35:33.806 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_autoscaler", 2024-03-21T10:35:33.806 INFO:teuthology.orchestra.run.smithi047.stdout: "progress", 2024-03-21T10:35:33.806 INFO:teuthology.orchestra.run.smithi047.stdout: "rbd_support", 2024-03-21T10:35:33.807 INFO:teuthology.orchestra.run.smithi047.stdout: "status", 2024-03-21T10:35:33.807 INFO:teuthology.orchestra.run.smithi047.stdout: "telemetry", 2024-03-21T10:35:33.807 INFO:teuthology.orchestra.run.smithi047.stdout: "volumes" 2024-03-21T10:35:33.807 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-03-21T10:35:33.807 INFO:teuthology.orchestra.run.smithi047.stdout: "pacific": [ 2024-03-21T10:35:33.807 INFO:teuthology.orchestra.run.smithi047.stdout: "balancer", 2024-03-21T10:35:33.807 INFO:teuthology.orchestra.run.smithi047.stdout: "crash", 2024-03-21T10:35:33.807 INFO:teuthology.orchestra.run.smithi047.stdout: "devicehealth", 2024-03-21T10:35:33.807 INFO:teuthology.orchestra.run.smithi047.stdout: "orchestrator", 2024-03-21T10:35:33.807 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_autoscaler", 2024-03-21T10:35:33.807 INFO:teuthology.orchestra.run.smithi047.stdout: "progress", 2024-03-21T10:35:33.807 INFO:teuthology.orchestra.run.smithi047.stdout: "rbd_support", 2024-03-21T10:35:33.807 INFO:teuthology.orchestra.run.smithi047.stdout: "status", 2024-03-21T10:35:33.807 INFO:teuthology.orchestra.run.smithi047.stdout: "telemetry", 2024-03-21T10:35:33.807 INFO:teuthology.orchestra.run.smithi047.stdout: "volumes" 2024-03-21T10:35:33.808 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-03-21T10:35:33.808 INFO:teuthology.orchestra.run.smithi047.stdout: "quincy": [ 2024-03-21T10:35:33.808 INFO:teuthology.orchestra.run.smithi047.stdout: "balancer", 2024-03-21T10:35:33.808 INFO:teuthology.orchestra.run.smithi047.stdout: "crash", 2024-03-21T10:35:33.808 INFO:teuthology.orchestra.run.smithi047.stdout: "devicehealth", 2024-03-21T10:35:33.808 INFO:teuthology.orchestra.run.smithi047.stdout: "orchestrator", 2024-03-21T10:35:33.808 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_autoscaler", 2024-03-21T10:35:33.808 INFO:teuthology.orchestra.run.smithi047.stdout: "progress", 2024-03-21T10:35:33.808 INFO:teuthology.orchestra.run.smithi047.stdout: "rbd_support", 2024-03-21T10:35:33.808 INFO:teuthology.orchestra.run.smithi047.stdout: "status", 2024-03-21T10:35:33.808 INFO:teuthology.orchestra.run.smithi047.stdout: "telemetry", 2024-03-21T10:35:33.808 INFO:teuthology.orchestra.run.smithi047.stdout: "volumes" 2024-03-21T10:35:33.808 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-03-21T10:35:33.808 INFO:teuthology.orchestra.run.smithi047.stdout: "reef": [ 2024-03-21T10:35:33.809 INFO:teuthology.orchestra.run.smithi047.stdout: "balancer", 2024-03-21T10:35:33.809 INFO:teuthology.orchestra.run.smithi047.stdout: "crash", 2024-03-21T10:35:33.809 INFO:teuthology.orchestra.run.smithi047.stdout: "devicehealth", 2024-03-21T10:35:33.809 INFO:teuthology.orchestra.run.smithi047.stdout: "orchestrator", 2024-03-21T10:35:33.809 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_autoscaler", 2024-03-21T10:35:33.809 INFO:teuthology.orchestra.run.smithi047.stdout: "progress", 2024-03-21T10:35:33.809 INFO:teuthology.orchestra.run.smithi047.stdout: "rbd_support", 2024-03-21T10:35:33.809 INFO:teuthology.orchestra.run.smithi047.stdout: "status", 2024-03-21T10:35:33.809 INFO:teuthology.orchestra.run.smithi047.stdout: "telemetry", 2024-03-21T10:35:33.809 INFO:teuthology.orchestra.run.smithi047.stdout: "volumes" 2024-03-21T10:35:33.809 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-03-21T10:35:33.809 INFO:teuthology.orchestra.run.smithi047.stdout: "squid": [ 2024-03-21T10:35:33.809 INFO:teuthology.orchestra.run.smithi047.stdout: "balancer", 2024-03-21T10:35:33.809 INFO:teuthology.orchestra.run.smithi047.stdout: "crash", 2024-03-21T10:35:33.810 INFO:teuthology.orchestra.run.smithi047.stdout: "devicehealth", 2024-03-21T10:35:33.810 INFO:teuthology.orchestra.run.smithi047.stdout: "orchestrator", 2024-03-21T10:35:33.810 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_autoscaler", 2024-03-21T10:35:33.810 INFO:teuthology.orchestra.run.smithi047.stdout: "progress", 2024-03-21T10:35:33.810 INFO:teuthology.orchestra.run.smithi047.stdout: "rbd_support", 2024-03-21T10:35:33.810 INFO:teuthology.orchestra.run.smithi047.stdout: "status", 2024-03-21T10:35:33.810 INFO:teuthology.orchestra.run.smithi047.stdout: "telemetry", 2024-03-21T10:35:33.810 INFO:teuthology.orchestra.run.smithi047.stdout: "volumes" 2024-03-21T10:35:33.810 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-03-21T10:35:33.810 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:33.810 INFO:teuthology.orchestra.run.smithi047.stdout: "last_failure_osd_epoch": 96, 2024-03-21T10:35:33.810 INFO:teuthology.orchestra.run.smithi047.stdout: "active_clients": [ 2024-03-21T10:35:33.810 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-03-21T10:35:33.810 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "devicehealth", 2024-03-21T10:35:33.811 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-03-21T10:35:33.811 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-03-21T10:35:33.811 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-03-21T10:35:33.811 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:0", 2024-03-21T10:35:33.811 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 2227562886 2024-03-21T10:35:33.811 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-21T10:35:33.811 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-03-21T10:35:33.811 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:33.811 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-03-21T10:35:33.811 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "libcephsqlite", 2024-03-21T10:35:33.811 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-03-21T10:35:33.811 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-03-21T10:35:33.811 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-03-21T10:35:33.811 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:0", 2024-03-21T10:35:33.811 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 1337904235 2024-03-21T10:35:33.811 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-21T10:35:33.811 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-03-21T10:35:33.812 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:33.812 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-03-21T10:35:33.812 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "rbd_support", 2024-03-21T10:35:33.812 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-03-21T10:35:33.812 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-03-21T10:35:33.812 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-03-21T10:35:33.812 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:0", 2024-03-21T10:35:33.812 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 2935075550 2024-03-21T10:35:33.812 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-21T10:35:33.812 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-03-21T10:35:33.812 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:33.812 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-03-21T10:35:33.812 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "snap_schedule", 2024-03-21T10:35:33.813 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-03-21T10:35:33.813 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-03-21T10:35:33.813 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-03-21T10:35:33.813 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:0", 2024-03-21T10:35:33.813 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 1721323879 2024-03-21T10:35:33.813 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-21T10:35:33.813 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-03-21T10:35:33.813 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:33.813 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-03-21T10:35:33.813 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "snap_schedule", 2024-03-21T10:35:33.813 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-03-21T10:35:33.813 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-03-21T10:35:33.813 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-03-21T10:35:33.813 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:0", 2024-03-21T10:35:33.813 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 431442223 2024-03-21T10:35:33.814 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-21T10:35:33.814 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-03-21T10:35:33.814 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:33.814 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-03-21T10:35:33.814 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "volumes", 2024-03-21T10:35:33.814 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-03-21T10:35:33.814 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-03-21T10:35:33.814 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-03-21T10:35:33.814 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:0", 2024-03-21T10:35:33.814 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 540962886 2024-03-21T10:35:33.814 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-21T10:35:33.814 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-03-21T10:35:33.814 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-21T10:35:33.814 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-03-21T10:35:33.814 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-03-21T10:35:33.816 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:35:33 smithi157 ceph-mon[35581]: pgmap v318: 129 pgs: 129 active+clean; 2.5 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 0 B/s wr, 14 op/s 2024-03-21T10:35:33.816 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:35:33 smithi157 ceph-mon[35581]: from='client.? 172.21.15.47:0/4165729570' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-03-21T10:35:33.816 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:35:33 smithi100 ceph-mon[35193]: pgmap v318: 129 pgs: 129 active+clean; 2.5 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 0 B/s wr, 14 op/s 2024-03-21T10:35:33.816 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:35:33 smithi100 ceph-mon[35193]: from='client.? 172.21.15.47:0/4165729570' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-03-21T10:35:33.817 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:32.540+0000 7f64f9a00640 1 -- 172.21.15.47:0/4165729570 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f64d80a3090 msgr2=0x7f64d80a5550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:35:33.817 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:32.540+0000 7f64f9a00640 1 --2- 172.21.15.47:0/4165729570 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f64d80a3090 0x7f64d80a5550 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f64e4009b60 tx=0x7f64e4005f90 comp rx=0 tx=0).stop 2024-03-21T10:35:33.817 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:32.540+0000 7f64f9a00640 1 -- 172.21.15.47:0/4165729570 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f64f410dcd0 msgr2=0x7f64f419e7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:35:33.817 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:32.540+0000 7f64f9a00640 1 --2- 172.21.15.47:0/4165729570 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f64f410dcd0 0x7f64f419e7b0 secure :-1 s=READY pgs=1328 cs=0 l=1 rev1=1 crypto rx=0x7f64ec00efd0 tx=0x7f64ec00c6a0 comp rx=0 tx=0).stop 2024-03-21T10:35:33.817 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:32.540+0000 7f64f9a00640 1 -- 172.21.15.47:0/4165729570 shutdown_connections 2024-03-21T10:35:33.817 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:32.540+0000 7f64f9a00640 1 --2- 172.21.15.47:0/4165729570 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f64f410dcd0 0x7f64f419e7b0 unknown :-1 s=CLOSED pgs=1328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:35:33.817 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:32.540+0000 7f64f9a00640 1 --2- 172.21.15.47:0/4165729570 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f64d80a3090 0x7f64d80a5550 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:35:33.818 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:32.540+0000 7f64f9a00640 1 --2- 172.21.15.47:0/4165729570 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f64f40743f0 0x7f64f419a430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:35:33.818 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:32.540+0000 7f64f9a00640 1 --2- 172.21.15.47:0/4165729570 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f64f4073ad0 0x7f64f4199ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:35:33.818 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:32.540+0000 7f64f9a00640 1 -- 172.21.15.47:0/4165729570 >> 172.21.15.47:0/4165729570 conn(0x7f64f40fcfa0 msgr2=0x7f64f40fdd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:35:33.818 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:32.541+0000 7f64f9a00640 1 -- 172.21.15.47:0/4165729570 shutdown_connections 2024-03-21T10:35:33.818 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:35:32.541+0000 7f64f9a00640 1 -- 172.21.15.47:0/4165729570 wait complete. 2024-03-21T10:35:33.825 DEBUG:tasks.check_counter:Getting stats from a 2024-03-21T10:35:33.825 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2edfee68ad7243bed422a049b6f5f58619300f11 shell --fsid 3056d51a-e76b-11ee-95cd-87774f69a715 -- ceph daemon mds.a perf dump 2024-03-21T10:35:33.826 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:35:32 smithi047 ceph-mon[31547]: from='client.? 172.21.15.47:0/3326126196' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-21T10:35:33.826 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:35:32 smithi047 ceph-mon[31547]: from='client.? 172.21.15.47:0/301756020' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-03-21T10:35:33.826 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:35:33 smithi047 ceph-mon[31547]: pgmap v318: 129 pgs: 129 active+clean; 2.5 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 0 B/s wr, 14 op/s 2024-03-21T10:35:33.826 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:35:33 smithi047 ceph-mon[31547]: from='client.? 172.21.15.47:0/4165729570' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-03-21T10:35:34.306 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/3056d51a-e76b-11ee-95cd-87774f69a715/mon.a/config 2024-03-21T10:35:35.711 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:35:35 smithi047 ceph-mon[31547]: pgmap v319: 129 pgs: 129 active+clean; 2.5 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 0 B/s wr, 10 op/s 2024-03-21T10:35:35.726 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:35:35 smithi100 ceph-mon[35193]: pgmap v319: 129 pgs: 129 active+clean; 2.5 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 0 B/s wr, 10 op/s 2024-03-21T10:35:35.736 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:35:35 smithi157 ceph-mon[35581]: pgmap v319: 129 pgs: 129 active+clean; 2.5 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 0 B/s wr, 10 op/s 2024-03-21T10:35:37.015 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-03-21T10:35:37.015 INFO:teuthology.orchestra.run.smithi047.stdout: "AsyncMessenger::Worker-0": { 2024-03-21T10:35:37.015 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_recv_messages": 32, 2024-03-21T10:35:37.016 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_send_messages": 177, 2024-03-21T10:35:37.016 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_recv_bytes": 7568, 2024-03-21T10:35:37.016 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_send_bytes": 344628, 2024-03-21T10:35:37.016 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_created_connections": 15, 2024-03-21T10:35:37.016 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_active_connections": 7, 2024-03-21T10:35:37.016 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_running_total_time": 0.054490547, 2024-03-21T10:35:37.016 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_running_send_time": 0.025420563, 2024-03-21T10:35:37.016 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_running_recv_time": 0.011648793, 2024-03-21T10:35:37.016 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_running_fast_dispatch_time": 0.000509443, 2024-03-21T10:35:37.016 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_send_messages_queue_lat": { 2024-03-21T10:35:37.016 INFO:teuthology.orchestra.run.smithi047.stdout: "avgcount": 170, 2024-03-21T10:35:37.016 INFO:teuthology.orchestra.run.smithi047.stdout: "sum": 0.102376697, 2024-03-21T10:35:37.016 INFO:teuthology.orchestra.run.smithi047.stdout: "avgtime": 0.000602215 2024-03-21T10:35:37.017 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:37.017 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_handle_ack_lat": { 2024-03-21T10:35:37.017 INFO:teuthology.orchestra.run.smithi047.stdout: "avgcount": 32, 2024-03-21T10:35:37.017 INFO:teuthology.orchestra.run.smithi047.stdout: "sum": 0.000014429, 2024-03-21T10:35:37.017 INFO:teuthology.orchestra.run.smithi047.stdout: "avgtime": 0.000000450 2024-03-21T10:35:37.017 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:37.017 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_recv_encrypted_bytes": 1728, 2024-03-21T10:35:37.017 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_send_encrypted_bytes": 312432 2024-03-21T10:35:37.017 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:37.017 INFO:teuthology.orchestra.run.smithi047.stdout: "AsyncMessenger::Worker-1": { 2024-03-21T10:35:37.017 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_recv_messages": 394, 2024-03-21T10:35:37.017 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_send_messages": 395, 2024-03-21T10:35:37.017 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_recv_bytes": 1818807, 2024-03-21T10:35:37.017 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_send_bytes": 640953, 2024-03-21T10:35:37.017 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_created_connections": 51, 2024-03-21T10:35:37.018 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_active_connections": 8, 2024-03-21T10:35:37.018 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_running_total_time": 0.161143293, 2024-03-21T10:35:37.018 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_running_send_time": 0.052719613, 2024-03-21T10:35:37.018 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_running_recv_time": 0.123548881, 2024-03-21T10:35:37.018 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_running_fast_dispatch_time": 0.009924480, 2024-03-21T10:35:37.018 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_send_messages_queue_lat": { 2024-03-21T10:35:37.018 INFO:teuthology.orchestra.run.smithi047.stdout: "avgcount": 390, 2024-03-21T10:35:37.018 INFO:teuthology.orchestra.run.smithi047.stdout: "sum": 0.068642010, 2024-03-21T10:35:37.018 INFO:teuthology.orchestra.run.smithi047.stdout: "avgtime": 0.000176005 2024-03-21T10:35:37.018 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:37.018 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_handle_ack_lat": { 2024-03-21T10:35:37.018 INFO:teuthology.orchestra.run.smithi047.stdout: "avgcount": 164, 2024-03-21T10:35:37.018 INFO:teuthology.orchestra.run.smithi047.stdout: "sum": 0.000067745, 2024-03-21T10:35:37.019 INFO:teuthology.orchestra.run.smithi047.stdout: "avgtime": 0.000000413 2024-03-21T10:35:37.019 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:37.019 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_recv_encrypted_bytes": 1782768, 2024-03-21T10:35:37.019 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_send_encrypted_bytes": 141264 2024-03-21T10:35:37.019 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:37.019 INFO:teuthology.orchestra.run.smithi047.stdout: "AsyncMessenger::Worker-2": { 2024-03-21T10:35:37.019 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_recv_messages": 493, 2024-03-21T10:35:37.019 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_send_messages": 702, 2024-03-21T10:35:37.019 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_recv_bytes": 169421, 2024-03-21T10:35:37.019 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_send_bytes": 506164, 2024-03-21T10:35:37.019 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_created_connections": 9, 2024-03-21T10:35:37.019 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_active_connections": 3, 2024-03-21T10:35:37.019 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_running_total_time": 0.142751034, 2024-03-21T10:35:37.019 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_running_send_time": 0.065311219, 2024-03-21T10:35:37.019 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_running_recv_time": 0.069854687, 2024-03-21T10:35:37.020 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_running_fast_dispatch_time": 0.006922161, 2024-03-21T10:35:37.020 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_send_messages_queue_lat": { 2024-03-21T10:35:37.020 INFO:teuthology.orchestra.run.smithi047.stdout: "avgcount": 709, 2024-03-21T10:35:37.020 INFO:teuthology.orchestra.run.smithi047.stdout: "sum": 0.135710167, 2024-03-21T10:35:37.020 INFO:teuthology.orchestra.run.smithi047.stdout: "avgtime": 0.000191410 2024-03-21T10:35:37.020 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:37.020 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_handle_ack_lat": { 2024-03-21T10:35:37.020 INFO:teuthology.orchestra.run.smithi047.stdout: "avgcount": 1095, 2024-03-21T10:35:37.020 INFO:teuthology.orchestra.run.smithi047.stdout: "sum": 0.000533461, 2024-03-21T10:35:37.020 INFO:teuthology.orchestra.run.smithi047.stdout: "avgtime": 0.000000487 2024-03-21T10:35:37.020 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:37.020 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_recv_encrypted_bytes": 3104, 2024-03-21T10:35:37.020 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_send_encrypted_bytes": 147520 2024-03-21T10:35:37.020 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:37.021 INFO:teuthology.orchestra.run.smithi047.stdout: "cct": { 2024-03-21T10:35:37.021 INFO:teuthology.orchestra.run.smithi047.stdout: "total_workers": 1, 2024-03-21T10:35:37.021 INFO:teuthology.orchestra.run.smithi047.stdout: "unhealthy_workers": 0 2024-03-21T10:35:37.021 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:37.021 INFO:teuthology.orchestra.run.smithi047.stdout: "finisher-MDSRank": { 2024-03-21T10:35:37.021 INFO:teuthology.orchestra.run.smithi047.stdout: "queue_len": 0, 2024-03-21T10:35:37.021 INFO:teuthology.orchestra.run.smithi047.stdout: "complete_latency": { 2024-03-21T10:35:37.021 INFO:teuthology.orchestra.run.smithi047.stdout: "avgcount": 47, 2024-03-21T10:35:37.021 INFO:teuthology.orchestra.run.smithi047.stdout: "sum": 0.001997460, 2024-03-21T10:35:37.021 INFO:teuthology.orchestra.run.smithi047.stdout: "avgtime": 0.000042499 2024-03-21T10:35:37.021 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-21T10:35:37.021 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:37.021 INFO:teuthology.orchestra.run.smithi047.stdout: "finisher-PurgeQueue": { 2024-03-21T10:35:37.022 INFO:teuthology.orchestra.run.smithi047.stdout: "queue_len": 0, 2024-03-21T10:35:37.022 INFO:teuthology.orchestra.run.smithi047.stdout: "complete_latency": { 2024-03-21T10:35:37.022 INFO:teuthology.orchestra.run.smithi047.stdout: "avgcount": 2, 2024-03-21T10:35:37.022 INFO:teuthology.orchestra.run.smithi047.stdout: "sum": 0.000011447, 2024-03-21T10:35:37.022 INFO:teuthology.orchestra.run.smithi047.stdout: "avgtime": 0.000005723 2024-03-21T10:35:37.022 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-21T10:35:37.022 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:37.022 INFO:teuthology.orchestra.run.smithi047.stdout: "mds": { 2024-03-21T10:35:37.022 INFO:teuthology.orchestra.run.smithi047.stdout: "request": 0, 2024-03-21T10:35:37.022 INFO:teuthology.orchestra.run.smithi047.stdout: "reply": 0, 2024-03-21T10:35:37.022 INFO:teuthology.orchestra.run.smithi047.stdout: "reply_latency": { 2024-03-21T10:35:37.022 INFO:teuthology.orchestra.run.smithi047.stdout: "avgcount": 0, 2024-03-21T10:35:37.022 INFO:teuthology.orchestra.run.smithi047.stdout: "sum": 0.000000000, 2024-03-21T10:35:37.023 INFO:teuthology.orchestra.run.smithi047.stdout: "avgtime": 0.000000000 2024-03-21T10:35:37.023 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:37.023 INFO:teuthology.orchestra.run.smithi047.stdout: "slow_reply": 0, 2024-03-21T10:35:37.023 INFO:teuthology.orchestra.run.smithi047.stdout: "forward": 0, 2024-03-21T10:35:37.023 INFO:teuthology.orchestra.run.smithi047.stdout: "dir_fetch_complete": 0, 2024-03-21T10:35:37.023 INFO:teuthology.orchestra.run.smithi047.stdout: "dir_fetch_keys": 0, 2024-03-21T10:35:37.023 INFO:teuthology.orchestra.run.smithi047.stdout: "dir_commit": 11, 2024-03-21T10:35:37.023 INFO:teuthology.orchestra.run.smithi047.stdout: "dir_split": 0, 2024-03-21T10:35:37.023 INFO:teuthology.orchestra.run.smithi047.stdout: "dir_merge": 0, 2024-03-21T10:35:37.023 INFO:teuthology.orchestra.run.smithi047.stdout: "inodes": 10, 2024-03-21T10:35:37.023 INFO:teuthology.orchestra.run.smithi047.stdout: "inodes_top": 0, 2024-03-21T10:35:37.023 INFO:teuthology.orchestra.run.smithi047.stdout: "inodes_bottom": 10, 2024-03-21T10:35:37.023 INFO:teuthology.orchestra.run.smithi047.stdout: "inodes_pin_tail": 0, 2024-03-21T10:35:37.023 INFO:teuthology.orchestra.run.smithi047.stdout: "inodes_pinned": 10, 2024-03-21T10:35:37.024 INFO:teuthology.orchestra.run.smithi047.stdout: "inodes_expired": 0, 2024-03-21T10:35:37.024 INFO:teuthology.orchestra.run.smithi047.stdout: "inodes_with_caps": 0, 2024-03-21T10:35:37.024 INFO:teuthology.orchestra.run.smithi047.stdout: "caps": 0, 2024-03-21T10:35:37.024 INFO:teuthology.orchestra.run.smithi047.stdout: "subtrees": 1, 2024-03-21T10:35:37.024 INFO:teuthology.orchestra.run.smithi047.stdout: "traverse": 0, 2024-03-21T10:35:37.024 INFO:teuthology.orchestra.run.smithi047.stdout: "traverse_hit": 0, 2024-03-21T10:35:37.024 INFO:teuthology.orchestra.run.smithi047.stdout: "traverse_forward": 0, 2024-03-21T10:35:37.024 INFO:teuthology.orchestra.run.smithi047.stdout: "traverse_discover": 0, 2024-03-21T10:35:37.024 INFO:teuthology.orchestra.run.smithi047.stdout: "traverse_dir_fetch": 0, 2024-03-21T10:35:37.024 INFO:teuthology.orchestra.run.smithi047.stdout: "traverse_remote_ino": 0, 2024-03-21T10:35:37.024 INFO:teuthology.orchestra.run.smithi047.stdout: "traverse_lock": 0, 2024-03-21T10:35:37.024 INFO:teuthology.orchestra.run.smithi047.stdout: "load_cent": 0, 2024-03-21T10:35:37.024 INFO:teuthology.orchestra.run.smithi047.stdout: "q": 0, 2024-03-21T10:35:37.024 INFO:teuthology.orchestra.run.smithi047.stdout: "exported": 0, 2024-03-21T10:35:37.025 INFO:teuthology.orchestra.run.smithi047.stdout: "exported_inodes": 0, 2024-03-21T10:35:37.025 INFO:teuthology.orchestra.run.smithi047.stdout: "imported": 0, 2024-03-21T10:35:37.025 INFO:teuthology.orchestra.run.smithi047.stdout: "imported_inodes": 0, 2024-03-21T10:35:37.025 INFO:teuthology.orchestra.run.smithi047.stdout: "openino_dir_fetch": 0, 2024-03-21T10:35:37.025 INFO:teuthology.orchestra.run.smithi047.stdout: "openino_backtrace_fetch": 0, 2024-03-21T10:35:37.025 INFO:teuthology.orchestra.run.smithi047.stdout: "openino_peer_discover": 0, 2024-03-21T10:35:37.025 INFO:teuthology.orchestra.run.smithi047.stdout: "root_rfiles": 4, 2024-03-21T10:35:37.025 INFO:teuthology.orchestra.run.smithi047.stdout: "root_rbytes": 234, 2024-03-21T10:35:37.025 INFO:teuthology.orchestra.run.smithi047.stdout: "root_rsnaps": 6, 2024-03-21T10:35:37.025 INFO:teuthology.orchestra.run.smithi047.stdout: "scrub_backtrace_fetch": 0, 2024-03-21T10:35:37.025 INFO:teuthology.orchestra.run.smithi047.stdout: "scrub_set_tag": 0, 2024-03-21T10:35:37.025 INFO:teuthology.orchestra.run.smithi047.stdout: "scrub_backtrace_repaired": 0, 2024-03-21T10:35:37.025 INFO:teuthology.orchestra.run.smithi047.stdout: "scrub_inotable_repaired": 0, 2024-03-21T10:35:37.025 INFO:teuthology.orchestra.run.smithi047.stdout: "scrub_dir_inodes": 0, 2024-03-21T10:35:37.026 INFO:teuthology.orchestra.run.smithi047.stdout: "scrub_dir_base_inodes": 0, 2024-03-21T10:35:37.026 INFO:teuthology.orchestra.run.smithi047.stdout: "scrub_dirfrag_rstats": 0, 2024-03-21T10:35:37.026 INFO:teuthology.orchestra.run.smithi047.stdout: "scrub_file_inodes": 0, 2024-03-21T10:35:37.026 INFO:teuthology.orchestra.run.smithi047.stdout: "handle_inode_file_caps": 0, 2024-03-21T10:35:37.026 INFO:teuthology.orchestra.run.smithi047.stdout: "ceph_cap_op_revoke": 0, 2024-03-21T10:35:37.026 INFO:teuthology.orchestra.run.smithi047.stdout: "ceph_cap_op_grant": 0, 2024-03-21T10:35:37.026 INFO:teuthology.orchestra.run.smithi047.stdout: "ceph_cap_op_trunc": 0, 2024-03-21T10:35:37.026 INFO:teuthology.orchestra.run.smithi047.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-03-21T10:35:37.026 INFO:teuthology.orchestra.run.smithi047.stdout: "ceph_cap_op_flush_ack": 0, 2024-03-21T10:35:37.026 INFO:teuthology.orchestra.run.smithi047.stdout: "handle_client_caps": 0, 2024-03-21T10:35:37.026 INFO:teuthology.orchestra.run.smithi047.stdout: "handle_client_caps_dirty": 0, 2024-03-21T10:35:37.026 INFO:teuthology.orchestra.run.smithi047.stdout: "handle_client_cap_release": 0, 2024-03-21T10:35:37.026 INFO:teuthology.orchestra.run.smithi047.stdout: "process_request_cap_release": 0 2024-03-21T10:35:37.026 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:37.027 INFO:teuthology.orchestra.run.smithi047.stdout: "mds_cache": { 2024-03-21T10:35:37.027 INFO:teuthology.orchestra.run.smithi047.stdout: "dir_update": 0, 2024-03-21T10:35:37.027 INFO:teuthology.orchestra.run.smithi047.stdout: "dir_update_receipt": 0, 2024-03-21T10:35:37.027 INFO:teuthology.orchestra.run.smithi047.stdout: "dir_try_discover": 0, 2024-03-21T10:35:37.027 INFO:teuthology.orchestra.run.smithi047.stdout: "dir_send_discover": 1, 2024-03-21T10:35:37.027 INFO:teuthology.orchestra.run.smithi047.stdout: "dir_handle_discover": 0, 2024-03-21T10:35:37.027 INFO:teuthology.orchestra.run.smithi047.stdout: "num_strays": 0, 2024-03-21T10:35:37.027 INFO:teuthology.orchestra.run.smithi047.stdout: "num_strays_delayed": 0, 2024-03-21T10:35:37.027 INFO:teuthology.orchestra.run.smithi047.stdout: "num_strays_enqueuing": 0, 2024-03-21T10:35:37.027 INFO:teuthology.orchestra.run.smithi047.stdout: "strays_created": 0, 2024-03-21T10:35:37.027 INFO:teuthology.orchestra.run.smithi047.stdout: "strays_enqueued": 0, 2024-03-21T10:35:37.027 INFO:teuthology.orchestra.run.smithi047.stdout: "strays_reintegrated": 0, 2024-03-21T10:35:37.027 INFO:teuthology.orchestra.run.smithi047.stdout: "strays_migrated": 0, 2024-03-21T10:35:37.027 INFO:teuthology.orchestra.run.smithi047.stdout: "num_recovering_processing": 0, 2024-03-21T10:35:37.027 INFO:teuthology.orchestra.run.smithi047.stdout: "num_recovering_enqueued": 0, 2024-03-21T10:35:37.027 INFO:teuthology.orchestra.run.smithi047.stdout: "num_recovering_prioritized": 0, 2024-03-21T10:35:37.028 INFO:teuthology.orchestra.run.smithi047.stdout: "recovery_started": 0, 2024-03-21T10:35:37.028 INFO:teuthology.orchestra.run.smithi047.stdout: "recovery_completed": 0, 2024-03-21T10:35:37.028 INFO:teuthology.orchestra.run.smithi047.stdout: "ireq_enqueue_scrub": 0, 2024-03-21T10:35:37.028 INFO:teuthology.orchestra.run.smithi047.stdout: "ireq_exportdir": 0, 2024-03-21T10:35:37.028 INFO:teuthology.orchestra.run.smithi047.stdout: "ireq_flush": 0, 2024-03-21T10:35:37.028 INFO:teuthology.orchestra.run.smithi047.stdout: "ireq_fragmentdir": 0, 2024-03-21T10:35:37.028 INFO:teuthology.orchestra.run.smithi047.stdout: "ireq_fragstats": 0, 2024-03-21T10:35:37.028 INFO:teuthology.orchestra.run.smithi047.stdout: "ireq_inodestats": 0 2024-03-21T10:35:37.028 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:37.028 INFO:teuthology.orchestra.run.smithi047.stdout: "mds_log": { 2024-03-21T10:35:37.028 INFO:teuthology.orchestra.run.smithi047.stdout: "evlrg": 0, 2024-03-21T10:35:37.028 INFO:teuthology.orchestra.run.smithi047.stdout: "evadd": 4, 2024-03-21T10:35:37.028 INFO:teuthology.orchestra.run.smithi047.stdout: "evex": 0, 2024-03-21T10:35:37.028 INFO:teuthology.orchestra.run.smithi047.stdout: "evtrm": 0, 2024-03-21T10:35:37.028 INFO:teuthology.orchestra.run.smithi047.stdout: "ev": 4, 2024-03-21T10:35:37.029 INFO:teuthology.orchestra.run.smithi047.stdout: "evexg": 0, 2024-03-21T10:35:37.029 INFO:teuthology.orchestra.run.smithi047.stdout: "evexd": 0, 2024-03-21T10:35:37.029 INFO:teuthology.orchestra.run.smithi047.stdout: "segadd": 2, 2024-03-21T10:35:37.029 INFO:teuthology.orchestra.run.smithi047.stdout: "segex": 0, 2024-03-21T10:35:37.029 INFO:teuthology.orchestra.run.smithi047.stdout: "segtrm": 0, 2024-03-21T10:35:37.029 INFO:teuthology.orchestra.run.smithi047.stdout: "seg": 2, 2024-03-21T10:35:37.029 INFO:teuthology.orchestra.run.smithi047.stdout: "segmjr": 2, 2024-03-21T10:35:37.029 INFO:teuthology.orchestra.run.smithi047.stdout: "segexg": 0, 2024-03-21T10:35:37.029 INFO:teuthology.orchestra.run.smithi047.stdout: "segexd": 0, 2024-03-21T10:35:37.029 INFO:teuthology.orchestra.run.smithi047.stdout: "expos": 4194304, 2024-03-21T10:35:37.029 INFO:teuthology.orchestra.run.smithi047.stdout: "wrpos": 4195776, 2024-03-21T10:35:37.029 INFO:teuthology.orchestra.run.smithi047.stdout: "rdpos": 0, 2024-03-21T10:35:37.029 INFO:teuthology.orchestra.run.smithi047.stdout: "jlat": { 2024-03-21T10:35:37.029 INFO:teuthology.orchestra.run.smithi047.stdout: "avgcount": 3, 2024-03-21T10:35:37.030 INFO:teuthology.orchestra.run.smithi047.stdout: "sum": 0.007153148, 2024-03-21T10:35:37.030 INFO:teuthology.orchestra.run.smithi047.stdout: "avgtime": 0.002384382 2024-03-21T10:35:37.030 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:37.030 INFO:teuthology.orchestra.run.smithi047.stdout: "replayed": 0 2024-03-21T10:35:37.030 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:37.030 INFO:teuthology.orchestra.run.smithi047.stdout: "mds_mem": { 2024-03-21T10:35:37.030 INFO:teuthology.orchestra.run.smithi047.stdout: "ino": 13, 2024-03-21T10:35:37.030 INFO:teuthology.orchestra.run.smithi047.stdout: "ino+": 13, 2024-03-21T10:35:37.030 INFO:teuthology.orchestra.run.smithi047.stdout: "ino-": 0, 2024-03-21T10:35:37.030 INFO:teuthology.orchestra.run.smithi047.stdout: "dir": 11, 2024-03-21T10:35:37.030 INFO:teuthology.orchestra.run.smithi047.stdout: "dir+": 12, 2024-03-21T10:35:37.030 INFO:teuthology.orchestra.run.smithi047.stdout: "dir-": 1, 2024-03-21T10:35:37.030 INFO:teuthology.orchestra.run.smithi047.stdout: "dn": 10, 2024-03-21T10:35:37.030 INFO:teuthology.orchestra.run.smithi047.stdout: "dn+": 10, 2024-03-21T10:35:37.031 INFO:teuthology.orchestra.run.smithi047.stdout: "dn-": 0, 2024-03-21T10:35:37.031 INFO:teuthology.orchestra.run.smithi047.stdout: "cap": 0, 2024-03-21T10:35:37.031 INFO:teuthology.orchestra.run.smithi047.stdout: "cap+": 0, 2024-03-21T10:35:37.031 INFO:teuthology.orchestra.run.smithi047.stdout: "cap-": 0, 2024-03-21T10:35:37.031 INFO:teuthology.orchestra.run.smithi047.stdout: "rss": 50688, 2024-03-21T10:35:37.031 INFO:teuthology.orchestra.run.smithi047.stdout: "heap": 223612 2024-03-21T10:35:37.031 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:37.031 INFO:teuthology.orchestra.run.smithi047.stdout: "mds_server": { 2024-03-21T10:35:37.031 INFO:teuthology.orchestra.run.smithi047.stdout: "dispatch_client_request": 0, 2024-03-21T10:35:37.031 INFO:teuthology.orchestra.run.smithi047.stdout: "dispatch_server_request": 0, 2024-03-21T10:35:37.031 INFO:teuthology.orchestra.run.smithi047.stdout: "handle_client_request": 0, 2024-03-21T10:35:37.031 INFO:teuthology.orchestra.run.smithi047.stdout: "handle_client_session": 17, 2024-03-21T10:35:37.031 INFO:teuthology.orchestra.run.smithi047.stdout: "handle_peer_request": 0, 2024-03-21T10:35:37.031 INFO:teuthology.orchestra.run.smithi047.stdout: "req_create_latency": { 2024-03-21T10:35:37.032 INFO:teuthology.orchestra.run.smithi047.stdout: "avgcount": 0, 2024-03-21T10:35:37.032 INFO:teuthology.orchestra.run.smithi047.stdout: "sum": 0.000000000, 2024-03-21T10:35:37.032 INFO:teuthology.orchestra.run.smithi047.stdout: "avgtime": 0.000000000 2024-03-21T10:35:37.032 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:37.032 INFO:teuthology.orchestra.run.smithi047.stdout: "req_getattr_latency": { 2024-03-21T10:35:37.032 INFO:teuthology.orchestra.run.smithi047.stdout: "avgcount": 0, 2024-03-21T10:35:37.032 INFO:teuthology.orchestra.run.smithi047.stdout: "sum": 0.000000000, 2024-03-21T10:35:37.032 INFO:teuthology.orchestra.run.smithi047.stdout: "avgtime": 0.000000000 2024-03-21T10:35:37.032 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:37.032 INFO:teuthology.orchestra.run.smithi047.stdout: "req_getfilelock_latency": { 2024-03-21T10:35:37.032 INFO:teuthology.orchestra.run.smithi047.stdout: "avgcount": 0, 2024-03-21T10:35:37.032 INFO:teuthology.orchestra.run.smithi047.stdout: "sum": 0.000000000, 2024-03-21T10:35:37.032 INFO:teuthology.orchestra.run.smithi047.stdout: "avgtime": 0.000000000 2024-03-21T10:35:37.033 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:37.033 INFO:teuthology.orchestra.run.smithi047.stdout: "req_link_latency": { 2024-03-21T10:35:37.033 INFO:teuthology.orchestra.run.smithi047.stdout: "avgcount": 0, 2024-03-21T10:35:37.033 INFO:teuthology.orchestra.run.smithi047.stdout: "sum": 0.000000000, 2024-03-21T10:35:37.033 INFO:teuthology.orchestra.run.smithi047.stdout: "avgtime": 0.000000000 2024-03-21T10:35:37.033 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:37.033 INFO:teuthology.orchestra.run.smithi047.stdout: "req_lookup_latency": { 2024-03-21T10:35:37.033 INFO:teuthology.orchestra.run.smithi047.stdout: "avgcount": 0, 2024-03-21T10:35:37.033 INFO:teuthology.orchestra.run.smithi047.stdout: "sum": 0.000000000, 2024-03-21T10:35:37.033 INFO:teuthology.orchestra.run.smithi047.stdout: "avgtime": 0.000000000 2024-03-21T10:35:37.033 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:37.033 INFO:teuthology.orchestra.run.smithi047.stdout: "req_lookuphash_latency": { 2024-03-21T10:35:37.033 INFO:teuthology.orchestra.run.smithi047.stdout: "avgcount": 0, 2024-03-21T10:35:37.033 INFO:teuthology.orchestra.run.smithi047.stdout: "sum": 0.000000000, 2024-03-21T10:35:37.033 INFO:teuthology.orchestra.run.smithi047.stdout: "avgtime": 0.000000000 2024-03-21T10:35:37.034 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:37.034 INFO:teuthology.orchestra.run.smithi047.stdout: "req_lookupino_latency": { 2024-03-21T10:35:37.034 INFO:teuthology.orchestra.run.smithi047.stdout: "avgcount": 0, 2024-03-21T10:35:37.034 INFO:teuthology.orchestra.run.smithi047.stdout: "sum": 0.000000000, 2024-03-21T10:35:37.034 INFO:teuthology.orchestra.run.smithi047.stdout: "avgtime": 0.000000000 2024-03-21T10:35:37.034 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:37.034 INFO:teuthology.orchestra.run.smithi047.stdout: "req_lookupname_latency": { 2024-03-21T10:35:37.034 INFO:teuthology.orchestra.run.smithi047.stdout: "avgcount": 0, 2024-03-21T10:35:37.034 INFO:teuthology.orchestra.run.smithi047.stdout: "sum": 0.000000000, 2024-03-21T10:35:37.034 INFO:teuthology.orchestra.run.smithi047.stdout: "avgtime": 0.000000000 2024-03-21T10:35:37.034 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:37.034 INFO:teuthology.orchestra.run.smithi047.stdout: "req_lookupparent_latency": { 2024-03-21T10:35:37.034 INFO:teuthology.orchestra.run.smithi047.stdout: "avgcount": 0, 2024-03-21T10:35:37.034 INFO:teuthology.orchestra.run.smithi047.stdout: "sum": 0.000000000, 2024-03-21T10:35:37.035 INFO:teuthology.orchestra.run.smithi047.stdout: "avgtime": 0.000000000 2024-03-21T10:35:37.035 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:37.035 INFO:teuthology.orchestra.run.smithi047.stdout: "req_lookupsnap_latency": { 2024-03-21T10:35:37.035 INFO:teuthology.orchestra.run.smithi047.stdout: "avgcount": 0, 2024-03-21T10:35:37.035 INFO:teuthology.orchestra.run.smithi047.stdout: "sum": 0.000000000, 2024-03-21T10:35:37.035 INFO:teuthology.orchestra.run.smithi047.stdout: "avgtime": 0.000000000 2024-03-21T10:35:37.035 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:37.035 INFO:teuthology.orchestra.run.smithi047.stdout: "req_lssnap_latency": { 2024-03-21T10:35:37.035 INFO:teuthology.orchestra.run.smithi047.stdout: "avgcount": 0, 2024-03-21T10:35:37.035 INFO:teuthology.orchestra.run.smithi047.stdout: "sum": 0.000000000, 2024-03-21T10:35:37.035 INFO:teuthology.orchestra.run.smithi047.stdout: "avgtime": 0.000000000 2024-03-21T10:35:37.035 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:37.035 INFO:teuthology.orchestra.run.smithi047.stdout: "req_mkdir_latency": { 2024-03-21T10:35:37.035 INFO:teuthology.orchestra.run.smithi047.stdout: "avgcount": 0, 2024-03-21T10:35:37.036 INFO:teuthology.orchestra.run.smithi047.stdout: "sum": 0.000000000, 2024-03-21T10:35:37.036 INFO:teuthology.orchestra.run.smithi047.stdout: "avgtime": 0.000000000 2024-03-21T10:35:37.036 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:37.036 INFO:teuthology.orchestra.run.smithi047.stdout: "req_mknod_latency": { 2024-03-21T10:35:37.036 INFO:teuthology.orchestra.run.smithi047.stdout: "avgcount": 0, 2024-03-21T10:35:37.037 INFO:teuthology.orchestra.run.smithi047.stdout: "sum": 0.000000000, 2024-03-21T10:35:37.037 INFO:teuthology.orchestra.run.smithi047.stdout: "avgtime": 0.000000000 2024-03-21T10:35:37.037 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:37.037 INFO:teuthology.orchestra.run.smithi047.stdout: "req_mksnap_latency": { 2024-03-21T10:35:37.037 INFO:teuthology.orchestra.run.smithi047.stdout: "avgcount": 0, 2024-03-21T10:35:37.037 INFO:teuthology.orchestra.run.smithi047.stdout: "sum": 0.000000000, 2024-03-21T10:35:37.037 INFO:teuthology.orchestra.run.smithi047.stdout: "avgtime": 0.000000000 2024-03-21T10:35:37.037 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:37.037 INFO:teuthology.orchestra.run.smithi047.stdout: "req_open_latency": { 2024-03-21T10:35:37.037 INFO:teuthology.orchestra.run.smithi047.stdout: "avgcount": 0, 2024-03-21T10:35:37.037 INFO:teuthology.orchestra.run.smithi047.stdout: "sum": 0.000000000, 2024-03-21T10:35:37.037 INFO:teuthology.orchestra.run.smithi047.stdout: "avgtime": 0.000000000 2024-03-21T10:35:37.037 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:37.038 INFO:teuthology.orchestra.run.smithi047.stdout: "req_readdir_latency": { 2024-03-21T10:35:37.038 INFO:teuthology.orchestra.run.smithi047.stdout: "avgcount": 0, 2024-03-21T10:35:37.038 INFO:teuthology.orchestra.run.smithi047.stdout: "sum": 0.000000000, 2024-03-21T10:35:37.038 INFO:teuthology.orchestra.run.smithi047.stdout: "avgtime": 0.000000000 2024-03-21T10:35:37.038 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:37.038 INFO:teuthology.orchestra.run.smithi047.stdout: "req_rename_latency": { 2024-03-21T10:35:37.038 INFO:teuthology.orchestra.run.smithi047.stdout: "avgcount": 0, 2024-03-21T10:35:37.038 INFO:teuthology.orchestra.run.smithi047.stdout: "sum": 0.000000000, 2024-03-21T10:35:37.038 INFO:teuthology.orchestra.run.smithi047.stdout: "avgtime": 0.000000000 2024-03-21T10:35:37.038 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:37.038 INFO:teuthology.orchestra.run.smithi047.stdout: "req_renamesnap_latency": { 2024-03-21T10:35:37.038 INFO:teuthology.orchestra.run.smithi047.stdout: "avgcount": 0, 2024-03-21T10:35:37.038 INFO:teuthology.orchestra.run.smithi047.stdout: "sum": 0.000000000, 2024-03-21T10:35:37.039 INFO:teuthology.orchestra.run.smithi047.stdout: "avgtime": 0.000000000 2024-03-21T10:35:37.039 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:37.039 INFO:teuthology.orchestra.run.smithi047.stdout: "req_snapdiff_latency": { 2024-03-21T10:35:37.039 INFO:teuthology.orchestra.run.smithi047.stdout: "avgcount": 0, 2024-03-21T10:35:37.039 INFO:teuthology.orchestra.run.smithi047.stdout: "sum": 0.000000000, 2024-03-21T10:35:37.039 INFO:teuthology.orchestra.run.smithi047.stdout: "avgtime": 0.000000000 2024-03-21T10:35:37.039 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:37.039 INFO:teuthology.orchestra.run.smithi047.stdout: "req_rmdir_latency": { 2024-03-21T10:35:37.039 INFO:teuthology.orchestra.run.smithi047.stdout: "avgcount": 0, 2024-03-21T10:35:37.039 INFO:teuthology.orchestra.run.smithi047.stdout: "sum": 0.000000000, 2024-03-21T10:35:37.039 INFO:teuthology.orchestra.run.smithi047.stdout: "avgtime": 0.000000000 2024-03-21T10:35:37.039 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:37.039 INFO:teuthology.orchestra.run.smithi047.stdout: "req_rmsnap_latency": { 2024-03-21T10:35:37.039 INFO:teuthology.orchestra.run.smithi047.stdout: "avgcount": 0, 2024-03-21T10:35:37.040 INFO:teuthology.orchestra.run.smithi047.stdout: "sum": 0.000000000, 2024-03-21T10:35:37.040 INFO:teuthology.orchestra.run.smithi047.stdout: "avgtime": 0.000000000 2024-03-21T10:35:37.040 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:37.040 INFO:teuthology.orchestra.run.smithi047.stdout: "req_rmxattr_latency": { 2024-03-21T10:35:37.040 INFO:teuthology.orchestra.run.smithi047.stdout: "avgcount": 0, 2024-03-21T10:35:37.040 INFO:teuthology.orchestra.run.smithi047.stdout: "sum": 0.000000000, 2024-03-21T10:35:37.040 INFO:teuthology.orchestra.run.smithi047.stdout: "avgtime": 0.000000000 2024-03-21T10:35:37.040 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:37.040 INFO:teuthology.orchestra.run.smithi047.stdout: "req_setattr_latency": { 2024-03-21T10:35:37.040 INFO:teuthology.orchestra.run.smithi047.stdout: "avgcount": 0, 2024-03-21T10:35:37.040 INFO:teuthology.orchestra.run.smithi047.stdout: "sum": 0.000000000, 2024-03-21T10:35:37.040 INFO:teuthology.orchestra.run.smithi047.stdout: "avgtime": 0.000000000 2024-03-21T10:35:37.040 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:37.040 INFO:teuthology.orchestra.run.smithi047.stdout: "req_setdirlayout_latency": { 2024-03-21T10:35:37.040 INFO:teuthology.orchestra.run.smithi047.stdout: "avgcount": 0, 2024-03-21T10:35:37.041 INFO:teuthology.orchestra.run.smithi047.stdout: "sum": 0.000000000, 2024-03-21T10:35:37.041 INFO:teuthology.orchestra.run.smithi047.stdout: "avgtime": 0.000000000 2024-03-21T10:35:37.041 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:37.041 INFO:teuthology.orchestra.run.smithi047.stdout: "req_setfilelock_latency": { 2024-03-21T10:35:37.041 INFO:teuthology.orchestra.run.smithi047.stdout: "avgcount": 0, 2024-03-21T10:35:37.041 INFO:teuthology.orchestra.run.smithi047.stdout: "sum": 0.000000000, 2024-03-21T10:35:37.041 INFO:teuthology.orchestra.run.smithi047.stdout: "avgtime": 0.000000000 2024-03-21T10:35:37.041 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:37.041 INFO:teuthology.orchestra.run.smithi047.stdout: "req_setlayout_latency": { 2024-03-21T10:35:37.041 INFO:teuthology.orchestra.run.smithi047.stdout: "avgcount": 0, 2024-03-21T10:35:37.041 INFO:teuthology.orchestra.run.smithi047.stdout: "sum": 0.000000000, 2024-03-21T10:35:37.041 INFO:teuthology.orchestra.run.smithi047.stdout: "avgtime": 0.000000000 2024-03-21T10:35:37.041 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:37.041 INFO:teuthology.orchestra.run.smithi047.stdout: "req_setxattr_latency": { 2024-03-21T10:35:37.042 INFO:teuthology.orchestra.run.smithi047.stdout: "avgcount": 0, 2024-03-21T10:35:37.042 INFO:teuthology.orchestra.run.smithi047.stdout: "sum": 0.000000000, 2024-03-21T10:35:37.042 INFO:teuthology.orchestra.run.smithi047.stdout: "avgtime": 0.000000000 2024-03-21T10:35:37.042 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:37.042 INFO:teuthology.orchestra.run.smithi047.stdout: "req_symlink_latency": { 2024-03-21T10:35:37.042 INFO:teuthology.orchestra.run.smithi047.stdout: "avgcount": 0, 2024-03-21T10:35:37.042 INFO:teuthology.orchestra.run.smithi047.stdout: "sum": 0.000000000, 2024-03-21T10:35:37.042 INFO:teuthology.orchestra.run.smithi047.stdout: "avgtime": 0.000000000 2024-03-21T10:35:37.042 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:37.042 INFO:teuthology.orchestra.run.smithi047.stdout: "req_unlink_latency": { 2024-03-21T10:35:37.042 INFO:teuthology.orchestra.run.smithi047.stdout: "avgcount": 0, 2024-03-21T10:35:37.042 INFO:teuthology.orchestra.run.smithi047.stdout: "sum": 0.000000000, 2024-03-21T10:35:37.042 INFO:teuthology.orchestra.run.smithi047.stdout: "avgtime": 0.000000000 2024-03-21T10:35:37.042 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:37.043 INFO:teuthology.orchestra.run.smithi047.stdout: "cap_revoke_eviction": 0, 2024-03-21T10:35:37.043 INFO:teuthology.orchestra.run.smithi047.stdout: "cap_acquisition_throttle": 0, 2024-03-21T10:35:37.043 INFO:teuthology.orchestra.run.smithi047.stdout: "req_getvxattr_latency": { 2024-03-21T10:35:37.043 INFO:teuthology.orchestra.run.smithi047.stdout: "avgcount": 0, 2024-03-21T10:35:37.043 INFO:teuthology.orchestra.run.smithi047.stdout: "sum": 0.000000000, 2024-03-21T10:35:37.043 INFO:teuthology.orchestra.run.smithi047.stdout: "avgtime": 0.000000000 2024-03-21T10:35:37.043 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-21T10:35:37.043 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:37.043 INFO:teuthology.orchestra.run.smithi047.stdout: "mds_sessions": { 2024-03-21T10:35:37.043 INFO:teuthology.orchestra.run.smithi047.stdout: "session_count": 2, 2024-03-21T10:35:37.043 INFO:teuthology.orchestra.run.smithi047.stdout: "session_add": 2, 2024-03-21T10:35:37.043 INFO:teuthology.orchestra.run.smithi047.stdout: "session_remove": 0, 2024-03-21T10:35:37.043 INFO:teuthology.orchestra.run.smithi047.stdout: "sessions_open": 2, 2024-03-21T10:35:37.043 INFO:teuthology.orchestra.run.smithi047.stdout: "sessions_stale": 0, 2024-03-21T10:35:37.044 INFO:teuthology.orchestra.run.smithi047.stdout: "total_load": 0, 2024-03-21T10:35:37.044 INFO:teuthology.orchestra.run.smithi047.stdout: "average_load": 0, 2024-03-21T10:35:37.044 INFO:teuthology.orchestra.run.smithi047.stdout: "avg_session_uptime": 606, 2024-03-21T10:35:37.044 INFO:teuthology.orchestra.run.smithi047.stdout: "mdthresh_evicted": 0 2024-03-21T10:35:37.044 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:37.044 INFO:teuthology.orchestra.run.smithi047.stdout: "mempool": { 2024-03-21T10:35:37.044 INFO:teuthology.orchestra.run.smithi047.stdout: "bloom_filter_bytes": 0, 2024-03-21T10:35:37.044 INFO:teuthology.orchestra.run.smithi047.stdout: "bloom_filter_items": 0, 2024-03-21T10:35:37.044 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_alloc_bytes": 0, 2024-03-21T10:35:37.044 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_alloc_items": 0, 2024-03-21T10:35:37.044 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_cache_data_bytes": 0, 2024-03-21T10:35:37.044 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_cache_data_items": 0, 2024-03-21T10:35:37.044 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_cache_onode_bytes": 0, 2024-03-21T10:35:37.044 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_cache_onode_items": 0, 2024-03-21T10:35:37.045 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_cache_meta_bytes": 0, 2024-03-21T10:35:37.045 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_cache_meta_items": 0, 2024-03-21T10:35:37.045 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_cache_other_bytes": 0, 2024-03-21T10:35:37.045 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_cache_other_items": 0, 2024-03-21T10:35:37.045 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_cache_buffer_bytes": 0, 2024-03-21T10:35:37.045 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_cache_buffer_items": 0, 2024-03-21T10:35:37.045 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_extent_bytes": 0, 2024-03-21T10:35:37.045 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_extent_items": 0, 2024-03-21T10:35:37.045 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_blob_bytes": 0, 2024-03-21T10:35:37.045 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_blob_items": 0, 2024-03-21T10:35:37.045 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_shared_blob_bytes": 0, 2024-03-21T10:35:37.045 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_shared_blob_items": 0, 2024-03-21T10:35:37.045 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_inline_bl_bytes": 0, 2024-03-21T10:35:37.045 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_inline_bl_items": 0, 2024-03-21T10:35:37.046 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_fsck_bytes": 0, 2024-03-21T10:35:37.046 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_fsck_items": 0, 2024-03-21T10:35:37.046 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_txc_bytes": 0, 2024-03-21T10:35:37.046 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_txc_items": 0, 2024-03-21T10:35:37.046 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_writing_deferred_bytes": 0, 2024-03-21T10:35:37.046 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_writing_deferred_items": 0, 2024-03-21T10:35:37.046 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_writing_bytes": 0, 2024-03-21T10:35:37.046 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_writing_items": 0, 2024-03-21T10:35:37.046 INFO:teuthology.orchestra.run.smithi047.stdout: "bluefs_bytes": 0, 2024-03-21T10:35:37.046 INFO:teuthology.orchestra.run.smithi047.stdout: "bluefs_items": 0, 2024-03-21T10:35:37.046 INFO:teuthology.orchestra.run.smithi047.stdout: "bluefs_file_reader_bytes": 0, 2024-03-21T10:35:37.046 INFO:teuthology.orchestra.run.smithi047.stdout: "bluefs_file_reader_items": 0, 2024-03-21T10:35:37.046 INFO:teuthology.orchestra.run.smithi047.stdout: "bluefs_file_writer_bytes": 0, 2024-03-21T10:35:37.046 INFO:teuthology.orchestra.run.smithi047.stdout: "bluefs_file_writer_items": 0, 2024-03-21T10:35:37.047 INFO:teuthology.orchestra.run.smithi047.stdout: "buffer_anon_bytes": 247191, 2024-03-21T10:35:37.047 INFO:teuthology.orchestra.run.smithi047.stdout: "buffer_anon_items": 86, 2024-03-21T10:35:37.047 INFO:teuthology.orchestra.run.smithi047.stdout: "buffer_meta_bytes": 0, 2024-03-21T10:35:37.047 INFO:teuthology.orchestra.run.smithi047.stdout: "buffer_meta_items": 0, 2024-03-21T10:35:37.047 INFO:teuthology.orchestra.run.smithi047.stdout: "osd_bytes": 0, 2024-03-21T10:35:37.047 INFO:teuthology.orchestra.run.smithi047.stdout: "osd_items": 0, 2024-03-21T10:35:37.047 INFO:teuthology.orchestra.run.smithi047.stdout: "osd_mapbl_bytes": 0, 2024-03-21T10:35:37.047 INFO:teuthology.orchestra.run.smithi047.stdout: "osd_mapbl_items": 0, 2024-03-21T10:35:37.047 INFO:teuthology.orchestra.run.smithi047.stdout: "osd_pglog_bytes": 0, 2024-03-21T10:35:37.047 INFO:teuthology.orchestra.run.smithi047.stdout: "osd_pglog_items": 0, 2024-03-21T10:35:37.047 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap_bytes": 9712, 2024-03-21T10:35:37.047 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap_items": 187, 2024-03-21T10:35:37.047 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap_mapping_bytes": 0, 2024-03-21T10:35:37.047 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap_mapping_items": 0, 2024-03-21T10:35:37.048 INFO:teuthology.orchestra.run.smithi047.stdout: "pgmap_bytes": 0, 2024-03-21T10:35:37.048 INFO:teuthology.orchestra.run.smithi047.stdout: "pgmap_items": 0, 2024-03-21T10:35:37.048 INFO:teuthology.orchestra.run.smithi047.stdout: "mds_co_bytes": 50040, 2024-03-21T10:35:37.048 INFO:teuthology.orchestra.run.smithi047.stdout: "mds_co_items": 198, 2024-03-21T10:35:37.048 INFO:teuthology.orchestra.run.smithi047.stdout: "unittest_1_bytes": 0, 2024-03-21T10:35:37.048 INFO:teuthology.orchestra.run.smithi047.stdout: "unittest_1_items": 0, 2024-03-21T10:35:37.048 INFO:teuthology.orchestra.run.smithi047.stdout: "unittest_2_bytes": 0, 2024-03-21T10:35:37.048 INFO:teuthology.orchestra.run.smithi047.stdout: "unittest_2_items": 0 2024-03-21T10:35:37.048 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:37.048 INFO:teuthology.orchestra.run.smithi047.stdout: "objecter": { 2024-03-21T10:35:37.048 INFO:teuthology.orchestra.run.smithi047.stdout: "op_active": 0, 2024-03-21T10:35:37.048 INFO:teuthology.orchestra.run.smithi047.stdout: "op_laggy": 0, 2024-03-21T10:35:37.048 INFO:teuthology.orchestra.run.smithi047.stdout: "op_send": 39, 2024-03-21T10:35:37.048 INFO:teuthology.orchestra.run.smithi047.stdout: "op_send_bytes": 11774, 2024-03-21T10:35:37.048 INFO:teuthology.orchestra.run.smithi047.stdout: "op_resend": 33, 2024-03-21T10:35:37.049 INFO:teuthology.orchestra.run.smithi047.stdout: "op_reply": 39, 2024-03-21T10:35:37.049 INFO:teuthology.orchestra.run.smithi047.stdout: "op_latency": { 2024-03-21T10:35:37.049 INFO:teuthology.orchestra.run.smithi047.stdout: "avgcount": 39, 2024-03-21T10:35:37.049 INFO:teuthology.orchestra.run.smithi047.stdout: "sum": 0.380991519, 2024-03-21T10:35:37.049 INFO:teuthology.orchestra.run.smithi047.stdout: "avgtime": 0.009769013 2024-03-21T10:35:37.049 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:37.049 INFO:teuthology.orchestra.run.smithi047.stdout: "op_inflight": 0, 2024-03-21T10:35:37.049 INFO:teuthology.orchestra.run.smithi047.stdout: "oplen_avg": { 2024-03-21T10:35:37.049 INFO:teuthology.orchestra.run.smithi047.stdout: "avgcount": 39, 2024-03-21T10:35:37.049 INFO:teuthology.orchestra.run.smithi047.stdout: "sum": 61 2024-03-21T10:35:37.049 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:37.049 INFO:teuthology.orchestra.run.smithi047.stdout: "op": 39, 2024-03-21T10:35:37.049 INFO:teuthology.orchestra.run.smithi047.stdout: "op_r": 0, 2024-03-21T10:35:37.049 INFO:teuthology.orchestra.run.smithi047.stdout: "op_w": 39, 2024-03-21T10:35:37.049 INFO:teuthology.orchestra.run.smithi047.stdout: "op_rmw": 0, 2024-03-21T10:35:37.050 INFO:teuthology.orchestra.run.smithi047.stdout: "op_pg": 0, 2024-03-21T10:35:37.050 INFO:teuthology.orchestra.run.smithi047.stdout: "osdop_stat": 0, 2024-03-21T10:35:37.050 INFO:teuthology.orchestra.run.smithi047.stdout: "osdop_create": 10, 2024-03-21T10:35:37.050 INFO:teuthology.orchestra.run.smithi047.stdout: "osdop_read": 0, 2024-03-21T10:35:37.050 INFO:teuthology.orchestra.run.smithi047.stdout: "osdop_write": 3, 2024-03-21T10:35:37.050 INFO:teuthology.orchestra.run.smithi047.stdout: "osdop_writefull": 7, 2024-03-21T10:35:37.050 INFO:teuthology.orchestra.run.smithi047.stdout: "osdop_writesame": 0, 2024-03-21T10:35:37.050 INFO:teuthology.orchestra.run.smithi047.stdout: "osdop_append": 0, 2024-03-21T10:35:37.050 INFO:teuthology.orchestra.run.smithi047.stdout: "osdop_zero": 0, 2024-03-21T10:35:37.050 INFO:teuthology.orchestra.run.smithi047.stdout: "osdop_truncate": 0, 2024-03-21T10:35:37.050 INFO:teuthology.orchestra.run.smithi047.stdout: "osdop_delete": 6, 2024-03-21T10:35:37.050 INFO:teuthology.orchestra.run.smithi047.stdout: "osdop_mapext": 0, 2024-03-21T10:35:37.050 INFO:teuthology.orchestra.run.smithi047.stdout: "osdop_sparse_read": 0, 2024-03-21T10:35:37.050 INFO:teuthology.orchestra.run.smithi047.stdout: "osdop_clonerange": 0, 2024-03-21T10:35:37.050 INFO:teuthology.orchestra.run.smithi047.stdout: "osdop_getxattr": 0, 2024-03-21T10:35:37.051 INFO:teuthology.orchestra.run.smithi047.stdout: "osdop_setxattr": 20, 2024-03-21T10:35:37.051 INFO:teuthology.orchestra.run.smithi047.stdout: "osdop_cmpxattr": 0, 2024-03-21T10:35:37.051 INFO:teuthology.orchestra.run.smithi047.stdout: "osdop_rmxattr": 0, 2024-03-21T10:35:37.051 INFO:teuthology.orchestra.run.smithi047.stdout: "osdop_resetxattrs": 0, 2024-03-21T10:35:37.051 INFO:teuthology.orchestra.run.smithi047.stdout: "osdop_call": 0, 2024-03-21T10:35:37.051 INFO:teuthology.orchestra.run.smithi047.stdout: "osdop_watch": 0, 2024-03-21T10:35:37.051 INFO:teuthology.orchestra.run.smithi047.stdout: "osdop_notify": 0, 2024-03-21T10:35:37.051 INFO:teuthology.orchestra.run.smithi047.stdout: "osdop_src_cmpxattr": 0, 2024-03-21T10:35:37.051 INFO:teuthology.orchestra.run.smithi047.stdout: "osdop_pgls": 0, 2024-03-21T10:35:37.051 INFO:teuthology.orchestra.run.smithi047.stdout: "osdop_pgls_filter": 0, 2024-03-21T10:35:37.051 INFO:teuthology.orchestra.run.smithi047.stdout: "osdop_other": 0, 2024-03-21T10:35:37.051 INFO:teuthology.orchestra.run.smithi047.stdout: "linger_active": 0, 2024-03-21T10:35:37.051 INFO:teuthology.orchestra.run.smithi047.stdout: "linger_send": 0, 2024-03-21T10:35:37.051 INFO:teuthology.orchestra.run.smithi047.stdout: "linger_resend": 0, 2024-03-21T10:35:37.051 INFO:teuthology.orchestra.run.smithi047.stdout: "linger_ping": 0, 2024-03-21T10:35:37.052 INFO:teuthology.orchestra.run.smithi047.stdout: "poolop_active": 0, 2024-03-21T10:35:37.052 INFO:teuthology.orchestra.run.smithi047.stdout: "poolop_send": 0, 2024-03-21T10:35:37.052 INFO:teuthology.orchestra.run.smithi047.stdout: "poolop_resend": 0, 2024-03-21T10:35:37.052 INFO:teuthology.orchestra.run.smithi047.stdout: "poolstat_active": 0, 2024-03-21T10:35:37.052 INFO:teuthology.orchestra.run.smithi047.stdout: "poolstat_send": 0, 2024-03-21T10:35:37.052 INFO:teuthology.orchestra.run.smithi047.stdout: "poolstat_resend": 0, 2024-03-21T10:35:37.052 INFO:teuthology.orchestra.run.smithi047.stdout: "statfs_active": 0, 2024-03-21T10:35:37.052 INFO:teuthology.orchestra.run.smithi047.stdout: "statfs_send": 0, 2024-03-21T10:35:37.052 INFO:teuthology.orchestra.run.smithi047.stdout: "statfs_resend": 0, 2024-03-21T10:35:37.052 INFO:teuthology.orchestra.run.smithi047.stdout: "command_active": 0, 2024-03-21T10:35:37.052 INFO:teuthology.orchestra.run.smithi047.stdout: "command_send": 0, 2024-03-21T10:35:37.052 INFO:teuthology.orchestra.run.smithi047.stdout: "command_resend": 0, 2024-03-21T10:35:37.052 INFO:teuthology.orchestra.run.smithi047.stdout: "map_epoch": 105, 2024-03-21T10:35:37.052 INFO:teuthology.orchestra.run.smithi047.stdout: "map_full": 0, 2024-03-21T10:35:37.052 INFO:teuthology.orchestra.run.smithi047.stdout: "map_inc": 12, 2024-03-21T10:35:37.053 INFO:teuthology.orchestra.run.smithi047.stdout: "osd_sessions": 10, 2024-03-21T10:35:37.053 INFO:teuthology.orchestra.run.smithi047.stdout: "osd_session_open": 10, 2024-03-21T10:35:37.053 INFO:teuthology.orchestra.run.smithi047.stdout: "osd_session_close": 0, 2024-03-21T10:35:37.053 INFO:teuthology.orchestra.run.smithi047.stdout: "osd_laggy": 0, 2024-03-21T10:35:37.053 INFO:teuthology.orchestra.run.smithi047.stdout: "omap_wr": 14, 2024-03-21T10:35:37.053 INFO:teuthology.orchestra.run.smithi047.stdout: "omap_rd": 0, 2024-03-21T10:35:37.053 INFO:teuthology.orchestra.run.smithi047.stdout: "omap_del": 1 2024-03-21T10:35:37.053 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:37.053 INFO:teuthology.orchestra.run.smithi047.stdout: "oft": { 2024-03-21T10:35:37.053 INFO:teuthology.orchestra.run.smithi047.stdout: "omap_total_objs": 1, 2024-03-21T10:35:37.053 INFO:teuthology.orchestra.run.smithi047.stdout: "omap_total_kv_pairs": 0, 2024-03-21T10:35:37.053 INFO:teuthology.orchestra.run.smithi047.stdout: "omap_total_updates": 0, 2024-03-21T10:35:37.053 INFO:teuthology.orchestra.run.smithi047.stdout: "omap_total_removes": 0 2024-03-21T10:35:37.053 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:37.053 INFO:teuthology.orchestra.run.smithi047.stdout: "osd-slow-ops": { 2024-03-21T10:35:37.054 INFO:teuthology.orchestra.run.smithi047.stdout: "slow_ops_count": 0 2024-03-21T10:35:37.054 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:37.054 INFO:teuthology.orchestra.run.smithi047.stdout: "purge_queue": { 2024-03-21T10:35:37.054 INFO:teuthology.orchestra.run.smithi047.stdout: "pq_executing_ops": 0, 2024-03-21T10:35:37.054 INFO:teuthology.orchestra.run.smithi047.stdout: "pq_executing_ops_high_water": 0, 2024-03-21T10:35:37.054 INFO:teuthology.orchestra.run.smithi047.stdout: "pq_executing": 0, 2024-03-21T10:35:37.054 INFO:teuthology.orchestra.run.smithi047.stdout: "pq_executing_high_water": 0, 2024-03-21T10:35:37.054 INFO:teuthology.orchestra.run.smithi047.stdout: "pq_executed_ops": 0, 2024-03-21T10:35:37.054 INFO:teuthology.orchestra.run.smithi047.stdout: "pq_executed": 0, 2024-03-21T10:35:37.054 INFO:teuthology.orchestra.run.smithi047.stdout: "pq_item_in_journal": 0 2024-03-21T10:35:37.054 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:37.054 INFO:teuthology.orchestra.run.smithi047.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-03-21T10:35:37.054 INFO:teuthology.orchestra.run.smithi047.stdout: "val": 0, 2024-03-21T10:35:37.055 INFO:teuthology.orchestra.run.smithi047.stdout: "max": 104857600, 2024-03-21T10:35:37.055 INFO:teuthology.orchestra.run.smithi047.stdout: "get_started": 0, 2024-03-21T10:35:37.055 INFO:teuthology.orchestra.run.smithi047.stdout: "get": 915, 2024-03-21T10:35:37.055 INFO:teuthology.orchestra.run.smithi047.stdout: "get_sum": 1895677, 2024-03-21T10:35:37.055 INFO:teuthology.orchestra.run.smithi047.stdout: "get_or_fail_fail": 0, 2024-03-21T10:35:37.055 INFO:teuthology.orchestra.run.smithi047.stdout: "get_or_fail_success": 915, 2024-03-21T10:35:37.055 INFO:teuthology.orchestra.run.smithi047.stdout: "take": 0, 2024-03-21T10:35:37.055 INFO:teuthology.orchestra.run.smithi047.stdout: "take_sum": 0, 2024-03-21T10:35:37.055 INFO:teuthology.orchestra.run.smithi047.stdout: "put": 915, 2024-03-21T10:35:37.055 INFO:teuthology.orchestra.run.smithi047.stdout: "put_sum": 1895677, 2024-03-21T10:35:37.055 INFO:teuthology.orchestra.run.smithi047.stdout: "wait": { 2024-03-21T10:35:37.055 INFO:teuthology.orchestra.run.smithi047.stdout: "avgcount": 0, 2024-03-21T10:35:37.055 INFO:teuthology.orchestra.run.smithi047.stdout: "sum": 0.000000000, 2024-03-21T10:35:37.055 INFO:teuthology.orchestra.run.smithi047.stdout: "avgtime": 0.000000000 2024-03-21T10:35:37.056 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-21T10:35:37.056 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:37.056 INFO:teuthology.orchestra.run.smithi047.stdout: "throttle-objecter_bytes": { 2024-03-21T10:35:37.056 INFO:teuthology.orchestra.run.smithi047.stdout: "val": 0, 2024-03-21T10:35:37.056 INFO:teuthology.orchestra.run.smithi047.stdout: "max": 104857600, 2024-03-21T10:35:37.056 INFO:teuthology.orchestra.run.smithi047.stdout: "get_started": 0, 2024-03-21T10:35:37.056 INFO:teuthology.orchestra.run.smithi047.stdout: "get": 0, 2024-03-21T10:35:37.056 INFO:teuthology.orchestra.run.smithi047.stdout: "get_sum": 0, 2024-03-21T10:35:37.056 INFO:teuthology.orchestra.run.smithi047.stdout: "get_or_fail_fail": 0, 2024-03-21T10:35:37.056 INFO:teuthology.orchestra.run.smithi047.stdout: "get_or_fail_success": 0, 2024-03-21T10:35:37.056 INFO:teuthology.orchestra.run.smithi047.stdout: "take": 39, 2024-03-21T10:35:37.056 INFO:teuthology.orchestra.run.smithi047.stdout: "take_sum": 11774, 2024-03-21T10:35:37.056 INFO:teuthology.orchestra.run.smithi047.stdout: "put": 39, 2024-03-21T10:35:37.056 INFO:teuthology.orchestra.run.smithi047.stdout: "put_sum": 11774, 2024-03-21T10:35:37.056 INFO:teuthology.orchestra.run.smithi047.stdout: "wait": { 2024-03-21T10:35:37.057 INFO:teuthology.orchestra.run.smithi047.stdout: "avgcount": 0, 2024-03-21T10:35:37.057 INFO:teuthology.orchestra.run.smithi047.stdout: "sum": 0.000000000, 2024-03-21T10:35:37.057 INFO:teuthology.orchestra.run.smithi047.stdout: "avgtime": 0.000000000 2024-03-21T10:35:37.057 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-21T10:35:37.057 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:37.057 INFO:teuthology.orchestra.run.smithi047.stdout: "throttle-objecter_ops": { 2024-03-21T10:35:37.057 INFO:teuthology.orchestra.run.smithi047.stdout: "val": 0, 2024-03-21T10:35:37.057 INFO:teuthology.orchestra.run.smithi047.stdout: "max": 1024, 2024-03-21T10:35:37.057 INFO:teuthology.orchestra.run.smithi047.stdout: "get_started": 0, 2024-03-21T10:35:37.057 INFO:teuthology.orchestra.run.smithi047.stdout: "get": 0, 2024-03-21T10:35:37.057 INFO:teuthology.orchestra.run.smithi047.stdout: "get_sum": 0, 2024-03-21T10:35:37.057 INFO:teuthology.orchestra.run.smithi047.stdout: "get_or_fail_fail": 0, 2024-03-21T10:35:37.057 INFO:teuthology.orchestra.run.smithi047.stdout: "get_or_fail_success": 0, 2024-03-21T10:35:37.057 INFO:teuthology.orchestra.run.smithi047.stdout: "take": 39, 2024-03-21T10:35:37.058 INFO:teuthology.orchestra.run.smithi047.stdout: "take_sum": 39, 2024-03-21T10:35:37.058 INFO:teuthology.orchestra.run.smithi047.stdout: "put": 39, 2024-03-21T10:35:37.058 INFO:teuthology.orchestra.run.smithi047.stdout: "put_sum": 39, 2024-03-21T10:35:37.058 INFO:teuthology.orchestra.run.smithi047.stdout: "wait": { 2024-03-21T10:35:37.058 INFO:teuthology.orchestra.run.smithi047.stdout: "avgcount": 0, 2024-03-21T10:35:37.058 INFO:teuthology.orchestra.run.smithi047.stdout: "sum": 0.000000000, 2024-03-21T10:35:37.058 INFO:teuthology.orchestra.run.smithi047.stdout: "avgtime": 0.000000000 2024-03-21T10:35:37.058 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-21T10:35:37.058 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:37.058 INFO:teuthology.orchestra.run.smithi047.stdout: "throttle-write_buf_throttle": { 2024-03-21T10:35:37.058 INFO:teuthology.orchestra.run.smithi047.stdout: "val": 0, 2024-03-21T10:35:37.058 INFO:teuthology.orchestra.run.smithi047.stdout: "max": 3758096384, 2024-03-21T10:35:37.058 INFO:teuthology.orchestra.run.smithi047.stdout: "get_started": 0, 2024-03-21T10:35:37.058 INFO:teuthology.orchestra.run.smithi047.stdout: "get": 0, 2024-03-21T10:35:37.059 INFO:teuthology.orchestra.run.smithi047.stdout: "get_sum": 0, 2024-03-21T10:35:37.059 INFO:teuthology.orchestra.run.smithi047.stdout: "get_or_fail_fail": 0, 2024-03-21T10:35:37.059 INFO:teuthology.orchestra.run.smithi047.stdout: "get_or_fail_success": 0, 2024-03-21T10:35:37.059 INFO:teuthology.orchestra.run.smithi047.stdout: "take": 0, 2024-03-21T10:35:37.059 INFO:teuthology.orchestra.run.smithi047.stdout: "take_sum": 0, 2024-03-21T10:35:37.059 INFO:teuthology.orchestra.run.smithi047.stdout: "put": 0, 2024-03-21T10:35:37.059 INFO:teuthology.orchestra.run.smithi047.stdout: "put_sum": 0, 2024-03-21T10:35:37.059 INFO:teuthology.orchestra.run.smithi047.stdout: "wait": { 2024-03-21T10:35:37.059 INFO:teuthology.orchestra.run.smithi047.stdout: "avgcount": 0, 2024-03-21T10:35:37.059 INFO:teuthology.orchestra.run.smithi047.stdout: "sum": 0.000000000, 2024-03-21T10:35:37.059 INFO:teuthology.orchestra.run.smithi047.stdout: "avgtime": 0.000000000 2024-03-21T10:35:37.059 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-21T10:35:37.059 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:37.059 INFO:teuthology.orchestra.run.smithi047.stdout: "throttle-write_buf_throttle-0x5629fdf54500": { 2024-03-21T10:35:37.059 INFO:teuthology.orchestra.run.smithi047.stdout: "val": 0, 2024-03-21T10:35:37.060 INFO:teuthology.orchestra.run.smithi047.stdout: "max": 3758096384, 2024-03-21T10:35:37.060 INFO:teuthology.orchestra.run.smithi047.stdout: "get_started": 0, 2024-03-21T10:35:37.060 INFO:teuthology.orchestra.run.smithi047.stdout: "get": 4, 2024-03-21T10:35:37.060 INFO:teuthology.orchestra.run.smithi047.stdout: "get_sum": 1472, 2024-03-21T10:35:37.060 INFO:teuthology.orchestra.run.smithi047.stdout: "get_or_fail_fail": 0, 2024-03-21T10:35:37.060 INFO:teuthology.orchestra.run.smithi047.stdout: "get_or_fail_success": 4, 2024-03-21T10:35:37.060 INFO:teuthology.orchestra.run.smithi047.stdout: "take": 0, 2024-03-21T10:35:37.060 INFO:teuthology.orchestra.run.smithi047.stdout: "take_sum": 0, 2024-03-21T10:35:37.060 INFO:teuthology.orchestra.run.smithi047.stdout: "put": 3, 2024-03-21T10:35:37.060 INFO:teuthology.orchestra.run.smithi047.stdout: "put_sum": 1472, 2024-03-21T10:35:37.060 INFO:teuthology.orchestra.run.smithi047.stdout: "wait": { 2024-03-21T10:35:37.060 INFO:teuthology.orchestra.run.smithi047.stdout: "avgcount": 0, 2024-03-21T10:35:37.060 INFO:teuthology.orchestra.run.smithi047.stdout: "sum": 0.000000000, 2024-03-21T10:35:37.060 INFO:teuthology.orchestra.run.smithi047.stdout: "avgtime": 0.000000000 2024-03-21T10:35:37.060 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-21T10:35:37.060 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-21T10:35:37.061 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-03-21T10:35:37.630 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:35:37 smithi047 ceph-mon[31547]: pgmap v320: 129 pgs: 129 active+clean; 2.5 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 KiB/s rd, 0 B/s wr, 14 op/s 2024-03-21T10:35:37.726 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:35:37 smithi100 ceph-mon[35193]: pgmap v320: 129 pgs: 129 active+clean; 2.5 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 KiB/s rd, 0 B/s wr, 14 op/s 2024-03-21T10:35:37.736 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:35:37 smithi157 ceph-mon[35581]: pgmap v320: 129 pgs: 129 active+clean; 2.5 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 KiB/s rd, 0 B/s wr, 14 op/s 2024-03-21T10:35:38.054 INFO:tasks.check_counter:Daemon mds.a mds.exported=0 2024-03-21T10:35:38.054 INFO:tasks.check_counter:Daemon mds.a mds.imported=0 2024-03-21T10:35:38.054 INFO:tasks.check_counter:Daemon mds.a mds.root_rsnaps=6 2024-03-21T10:35:38.054 INFO:tasks.check_counter:Daemon mds.a mds_server.req_mksnap_latency.avgcount=0 2024-03-21T10:35:38.054 INFO:tasks.check_counter:Daemon mds.a mds_server.req_rmsnap_latency.avgcount=0 2024-03-21T10:35:38.054 DEBUG:tasks.check_counter:Getting stats from d 2024-03-21T10:35:38.055 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2edfee68ad7243bed422a049b6f5f58619300f11 shell --fsid 3056d51a-e76b-11ee-95cd-87774f69a715 -- ceph daemon mds.d perf dump 2024-03-21T10:35:38.537 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/3056d51a-e76b-11ee-95cd-87774f69a715/mon.a/config 2024-03-21T10:35:39.711 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:35:39 smithi047 ceph-mon[31547]: pgmap v321: 129 pgs: 129 active+clean; 2.5 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 0 B/s wr, 11 op/s 2024-03-21T10:35:39.726 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:35:39 smithi100 ceph-mon[35193]: pgmap v321: 129 pgs: 129 active+clean; 2.5 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 0 B/s wr, 11 op/s 2024-03-21T10:35:39.737 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:35:39 smithi157 ceph-mon[35581]: pgmap v321: 129 pgs: 129 active+clean; 2.5 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 0 B/s wr, 11 op/s 2024-03-21T10:35:40.863 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-03-21T10:35:40.863 INFO:teuthology.orchestra.run.smithi047.stdout: "AsyncMessenger::Worker-0": { 2024-03-21T10:35:40.863 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_recv_messages": 241, 2024-03-21T10:35:40.864 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_send_messages": 229, 2024-03-21T10:35:40.864 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_recv_bytes": 1723920, 2024-03-21T10:35:40.864 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_send_bytes": 137888, 2024-03-21T10:35:40.864 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_created_connections": 4, 2024-03-21T10:35:40.864 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_active_connections": 0, 2024-03-21T10:35:40.864 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_running_total_time": 0.109475862, 2024-03-21T10:35:40.864 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_running_send_time": 0.039755120, 2024-03-21T10:35:40.864 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_running_recv_time": 0.076995080, 2024-03-21T10:35:40.864 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_running_fast_dispatch_time": 0.011280469, 2024-03-21T10:35:40.864 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_send_messages_queue_lat": { 2024-03-21T10:35:40.864 INFO:teuthology.orchestra.run.smithi047.stdout: "avgcount": 228, 2024-03-21T10:35:40.864 INFO:teuthology.orchestra.run.smithi047.stdout: "sum": 0.021848873, 2024-03-21T10:35:40.864 INFO:teuthology.orchestra.run.smithi047.stdout: "avgtime": 0.000095828 2024-03-21T10:35:40.864 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:40.865 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_handle_ack_lat": { 2024-03-21T10:35:40.865 INFO:teuthology.orchestra.run.smithi047.stdout: "avgcount": 0, 2024-03-21T10:35:40.865 INFO:teuthology.orchestra.run.smithi047.stdout: "sum": 0.000000000, 2024-03-21T10:35:40.865 INFO:teuthology.orchestra.run.smithi047.stdout: "avgtime": 0.000000000 2024-03-21T10:35:40.865 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:40.865 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_recv_encrypted_bytes": 1723920, 2024-03-21T10:35:40.865 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_send_encrypted_bytes": 137888 2024-03-21T10:35:40.865 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:40.865 INFO:teuthology.orchestra.run.smithi047.stdout: "AsyncMessenger::Worker-1": { 2024-03-21T10:35:40.865 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_recv_messages": 6, 2024-03-21T10:35:40.865 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_send_messages": 177, 2024-03-21T10:35:40.865 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_recv_bytes": 3264, 2024-03-21T10:35:40.865 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_send_bytes": 186272, 2024-03-21T10:35:40.865 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_created_connections": 5, 2024-03-21T10:35:40.865 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_active_connections": 3, 2024-03-21T10:35:40.866 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_running_total_time": 0.038716461, 2024-03-21T10:35:40.866 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_running_send_time": 0.028514939, 2024-03-21T10:35:40.866 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_running_recv_time": 0.004767480, 2024-03-21T10:35:40.866 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-03-21T10:35:40.866 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_send_messages_queue_lat": { 2024-03-21T10:35:40.866 INFO:teuthology.orchestra.run.smithi047.stdout: "avgcount": 175, 2024-03-21T10:35:40.866 INFO:teuthology.orchestra.run.smithi047.stdout: "sum": 0.019787814, 2024-03-21T10:35:40.866 INFO:teuthology.orchestra.run.smithi047.stdout: "avgtime": 0.000113073 2024-03-21T10:35:40.866 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:40.866 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_handle_ack_lat": { 2024-03-21T10:35:40.866 INFO:teuthology.orchestra.run.smithi047.stdout: "avgcount": 0, 2024-03-21T10:35:40.866 INFO:teuthology.orchestra.run.smithi047.stdout: "sum": 0.000000000, 2024-03-21T10:35:40.866 INFO:teuthology.orchestra.run.smithi047.stdout: "avgtime": 0.000000000 2024-03-21T10:35:40.867 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:40.867 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_recv_encrypted_bytes": 3264, 2024-03-21T10:35:40.867 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_send_encrypted_bytes": 186272 2024-03-21T10:35:40.867 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:40.867 INFO:teuthology.orchestra.run.smithi047.stdout: "AsyncMessenger::Worker-2": { 2024-03-21T10:35:40.867 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_recv_messages": 0, 2024-03-21T10:35:40.867 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_send_messages": 2, 2024-03-21T10:35:40.867 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_recv_bytes": 0, 2024-03-21T10:35:40.867 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_send_bytes": 0, 2024-03-21T10:35:40.867 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_created_connections": 2, 2024-03-21T10:35:40.867 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_active_connections": 1, 2024-03-21T10:35:40.867 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_running_total_time": 0.000886273, 2024-03-21T10:35:40.867 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_running_send_time": 0.000000000, 2024-03-21T10:35:40.867 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_running_recv_time": 0.000712019, 2024-03-21T10:35:40.867 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-03-21T10:35:40.867 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_send_messages_queue_lat": { 2024-03-21T10:35:40.868 INFO:teuthology.orchestra.run.smithi047.stdout: "avgcount": 0, 2024-03-21T10:35:40.868 INFO:teuthology.orchestra.run.smithi047.stdout: "sum": 0.000000000, 2024-03-21T10:35:40.868 INFO:teuthology.orchestra.run.smithi047.stdout: "avgtime": 0.000000000 2024-03-21T10:35:40.868 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:40.868 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_handle_ack_lat": { 2024-03-21T10:35:40.868 INFO:teuthology.orchestra.run.smithi047.stdout: "avgcount": 0, 2024-03-21T10:35:40.868 INFO:teuthology.orchestra.run.smithi047.stdout: "sum": 0.000000000, 2024-03-21T10:35:40.868 INFO:teuthology.orchestra.run.smithi047.stdout: "avgtime": 0.000000000 2024-03-21T10:35:40.868 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:40.868 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_recv_encrypted_bytes": 0, 2024-03-21T10:35:40.868 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_send_encrypted_bytes": 0 2024-03-21T10:35:40.868 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:40.868 INFO:teuthology.orchestra.run.smithi047.stdout: "cct": { 2024-03-21T10:35:40.868 INFO:teuthology.orchestra.run.smithi047.stdout: "total_workers": 0, 2024-03-21T10:35:40.868 INFO:teuthology.orchestra.run.smithi047.stdout: "unhealthy_workers": 0 2024-03-21T10:35:40.869 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:40.869 INFO:teuthology.orchestra.run.smithi047.stdout: "mempool": { 2024-03-21T10:35:40.869 INFO:teuthology.orchestra.run.smithi047.stdout: "bloom_filter_bytes": 0, 2024-03-21T10:35:40.869 INFO:teuthology.orchestra.run.smithi047.stdout: "bloom_filter_items": 0, 2024-03-21T10:35:40.869 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_alloc_bytes": 0, 2024-03-21T10:35:40.869 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_alloc_items": 0, 2024-03-21T10:35:40.869 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_cache_data_bytes": 0, 2024-03-21T10:35:40.869 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_cache_data_items": 0, 2024-03-21T10:35:40.869 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_cache_onode_bytes": 0, 2024-03-21T10:35:40.869 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_cache_onode_items": 0, 2024-03-21T10:35:40.869 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_cache_meta_bytes": 0, 2024-03-21T10:35:40.869 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_cache_meta_items": 0, 2024-03-21T10:35:40.869 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_cache_other_bytes": 0, 2024-03-21T10:35:40.869 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_cache_other_items": 0, 2024-03-21T10:35:40.869 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_cache_buffer_bytes": 0, 2024-03-21T10:35:40.869 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_cache_buffer_items": 0, 2024-03-21T10:35:40.869 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_extent_bytes": 0, 2024-03-21T10:35:40.870 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_extent_items": 0, 2024-03-21T10:35:40.870 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_blob_bytes": 0, 2024-03-21T10:35:40.870 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_blob_items": 0, 2024-03-21T10:35:40.870 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_shared_blob_bytes": 0, 2024-03-21T10:35:40.870 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_shared_blob_items": 0, 2024-03-21T10:35:40.870 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_inline_bl_bytes": 0, 2024-03-21T10:35:40.870 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_inline_bl_items": 0, 2024-03-21T10:35:40.870 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_fsck_bytes": 0, 2024-03-21T10:35:40.870 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_fsck_items": 0, 2024-03-21T10:35:40.870 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_txc_bytes": 0, 2024-03-21T10:35:40.870 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_txc_items": 0, 2024-03-21T10:35:40.870 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_writing_deferred_bytes": 0, 2024-03-21T10:35:40.870 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_writing_deferred_items": 0, 2024-03-21T10:35:40.870 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_writing_bytes": 0, 2024-03-21T10:35:40.871 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_writing_items": 0, 2024-03-21T10:35:40.871 INFO:teuthology.orchestra.run.smithi047.stdout: "bluefs_bytes": 0, 2024-03-21T10:35:40.871 INFO:teuthology.orchestra.run.smithi047.stdout: "bluefs_items": 0, 2024-03-21T10:35:40.871 INFO:teuthology.orchestra.run.smithi047.stdout: "bluefs_file_reader_bytes": 0, 2024-03-21T10:35:40.871 INFO:teuthology.orchestra.run.smithi047.stdout: "bluefs_file_reader_items": 0, 2024-03-21T10:35:40.871 INFO:teuthology.orchestra.run.smithi047.stdout: "bluefs_file_writer_bytes": 0, 2024-03-21T10:35:40.871 INFO:teuthology.orchestra.run.smithi047.stdout: "bluefs_file_writer_items": 0, 2024-03-21T10:35:40.871 INFO:teuthology.orchestra.run.smithi047.stdout: "buffer_anon_bytes": 148754, 2024-03-21T10:35:40.871 INFO:teuthology.orchestra.run.smithi047.stdout: "buffer_anon_items": 31, 2024-03-21T10:35:40.871 INFO:teuthology.orchestra.run.smithi047.stdout: "buffer_meta_bytes": 0, 2024-03-21T10:35:40.871 INFO:teuthology.orchestra.run.smithi047.stdout: "buffer_meta_items": 0, 2024-03-21T10:35:40.871 INFO:teuthology.orchestra.run.smithi047.stdout: "osd_bytes": 0, 2024-03-21T10:35:40.871 INFO:teuthology.orchestra.run.smithi047.stdout: "osd_items": 0, 2024-03-21T10:35:40.871 INFO:teuthology.orchestra.run.smithi047.stdout: "osd_mapbl_bytes": 0, 2024-03-21T10:35:40.871 INFO:teuthology.orchestra.run.smithi047.stdout: "osd_mapbl_items": 0, 2024-03-21T10:35:40.871 INFO:teuthology.orchestra.run.smithi047.stdout: "osd_pglog_bytes": 0, 2024-03-21T10:35:40.872 INFO:teuthology.orchestra.run.smithi047.stdout: "osd_pglog_items": 0, 2024-03-21T10:35:40.872 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap_bytes": 0, 2024-03-21T10:35:40.872 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap_items": 0, 2024-03-21T10:35:40.872 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap_mapping_bytes": 0, 2024-03-21T10:35:40.872 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap_mapping_items": 0, 2024-03-21T10:35:40.872 INFO:teuthology.orchestra.run.smithi047.stdout: "pgmap_bytes": 0, 2024-03-21T10:35:40.872 INFO:teuthology.orchestra.run.smithi047.stdout: "pgmap_items": 0, 2024-03-21T10:35:40.872 INFO:teuthology.orchestra.run.smithi047.stdout: "mds_co_bytes": 928, 2024-03-21T10:35:40.872 INFO:teuthology.orchestra.run.smithi047.stdout: "mds_co_items": 2, 2024-03-21T10:35:40.872 INFO:teuthology.orchestra.run.smithi047.stdout: "unittest_1_bytes": 0, 2024-03-21T10:35:40.872 INFO:teuthology.orchestra.run.smithi047.stdout: "unittest_1_items": 0, 2024-03-21T10:35:40.872 INFO:teuthology.orchestra.run.smithi047.stdout: "unittest_2_bytes": 0, 2024-03-21T10:35:40.872 INFO:teuthology.orchestra.run.smithi047.stdout: "unittest_2_items": 0 2024-03-21T10:35:40.872 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:40.872 INFO:teuthology.orchestra.run.smithi047.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-03-21T10:35:40.873 INFO:teuthology.orchestra.run.smithi047.stdout: "val": 0, 2024-03-21T10:35:40.873 INFO:teuthology.orchestra.run.smithi047.stdout: "max": 104857600, 2024-03-21T10:35:40.873 INFO:teuthology.orchestra.run.smithi047.stdout: "get_started": 0, 2024-03-21T10:35:40.873 INFO:teuthology.orchestra.run.smithi047.stdout: "get": 243, 2024-03-21T10:35:40.873 INFO:teuthology.orchestra.run.smithi047.stdout: "get_sum": 1689933, 2024-03-21T10:35:40.873 INFO:teuthology.orchestra.run.smithi047.stdout: "get_or_fail_fail": 0, 2024-03-21T10:35:40.873 INFO:teuthology.orchestra.run.smithi047.stdout: "get_or_fail_success": 243, 2024-03-21T10:35:40.873 INFO:teuthology.orchestra.run.smithi047.stdout: "take": 0, 2024-03-21T10:35:40.873 INFO:teuthology.orchestra.run.smithi047.stdout: "take_sum": 0, 2024-03-21T10:35:40.873 INFO:teuthology.orchestra.run.smithi047.stdout: "put": 243, 2024-03-21T10:35:40.873 INFO:teuthology.orchestra.run.smithi047.stdout: "put_sum": 1689933, 2024-03-21T10:35:40.873 INFO:teuthology.orchestra.run.smithi047.stdout: "wait": { 2024-03-21T10:35:40.873 INFO:teuthology.orchestra.run.smithi047.stdout: "avgcount": 0, 2024-03-21T10:35:40.873 INFO:teuthology.orchestra.run.smithi047.stdout: "sum": 0.000000000, 2024-03-21T10:35:40.873 INFO:teuthology.orchestra.run.smithi047.stdout: "avgtime": 0.000000000 2024-03-21T10:35:40.873 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-21T10:35:40.874 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-21T10:35:40.874 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-03-21T10:35:41.678 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:35:41 smithi047 ceph-mon[31547]: pgmap v322: 129 pgs: 129 active+clean; 2.5 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-21T10:35:41.726 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:35:41 smithi100 ceph-mon[35193]: pgmap v322: 129 pgs: 129 active+clean; 2.5 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-21T10:35:41.736 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:35:41 smithi157 ceph-mon[35581]: pgmap v322: 129 pgs: 129 active+clean; 2.5 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-21T10:35:41.761 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.d 2024-03-21T10:35:41.761 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.d 2024-03-21T10:35:41.761 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.d 2024-03-21T10:35:41.761 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.d 2024-03-21T10:35:41.761 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.d 2024-03-21T10:35:41.761 DEBUG:tasks.check_counter:Getting stats from g 2024-03-21T10:35:41.762 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2edfee68ad7243bed422a049b6f5f58619300f11 shell --fsid 3056d51a-e76b-11ee-95cd-87774f69a715 -- ceph daemon mds.g perf dump 2024-03-21T10:35:42.247 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/3056d51a-e76b-11ee-95cd-87774f69a715/mon.a/config 2024-03-21T10:35:43.710 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:35:43 smithi047 ceph-mon[31547]: pgmap v323: 129 pgs: 129 active+clean; 2.5 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-03-21T10:35:43.726 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:35:43 smithi100 ceph-mon[35193]: pgmap v323: 129 pgs: 129 active+clean; 2.5 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-03-21T10:35:43.738 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:35:43 smithi157 ceph-mon[35581]: pgmap v323: 129 pgs: 129 active+clean; 2.5 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-03-21T10:35:44.610 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-03-21T10:35:44.610 INFO:teuthology.orchestra.run.smithi047.stdout: "AsyncMessenger::Worker-0": { 2024-03-21T10:35:44.611 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_recv_messages": 1049, 2024-03-21T10:35:44.611 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_send_messages": 1034, 2024-03-21T10:35:44.611 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_recv_bytes": 2033248, 2024-03-21T10:35:44.611 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_send_bytes": 389886, 2024-03-21T10:35:44.611 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_created_connections": 5, 2024-03-21T10:35:44.611 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_active_connections": 1, 2024-03-21T10:35:44.611 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_running_total_time": 0.296841822, 2024-03-21T10:35:44.611 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_running_send_time": 0.121604294, 2024-03-21T10:35:44.611 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_running_recv_time": 0.105180900, 2024-03-21T10:35:44.612 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_running_fast_dispatch_time": 0.062973957, 2024-03-21T10:35:44.612 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_send_messages_queue_lat": { 2024-03-21T10:35:44.612 INFO:teuthology.orchestra.run.smithi047.stdout: "avgcount": 1033, 2024-03-21T10:35:44.612 INFO:teuthology.orchestra.run.smithi047.stdout: "sum": 0.103022130, 2024-03-21T10:35:44.612 INFO:teuthology.orchestra.run.smithi047.stdout: "avgtime": 0.000099731 2024-03-21T10:35:44.612 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:44.612 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_handle_ack_lat": { 2024-03-21T10:35:44.612 INFO:teuthology.orchestra.run.smithi047.stdout: "avgcount": 0, 2024-03-21T10:35:44.612 INFO:teuthology.orchestra.run.smithi047.stdout: "sum": 0.000000000, 2024-03-21T10:35:44.612 INFO:teuthology.orchestra.run.smithi047.stdout: "avgtime": 0.000000000 2024-03-21T10:35:44.613 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:44.613 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_recv_encrypted_bytes": 1766800, 2024-03-21T10:35:44.613 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_send_encrypted_bytes": 137712 2024-03-21T10:35:44.613 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:44.613 INFO:teuthology.orchestra.run.smithi047.stdout: "AsyncMessenger::Worker-1": { 2024-03-21T10:35:44.613 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_recv_messages": 1445, 2024-03-21T10:35:44.613 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_send_messages": 1614, 2024-03-21T10:35:44.613 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_recv_bytes": 369522, 2024-03-21T10:35:44.613 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_send_bytes": 913459, 2024-03-21T10:35:44.613 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_created_connections": 7, 2024-03-21T10:35:44.613 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_active_connections": 5, 2024-03-21T10:35:44.613 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_running_total_time": 0.271507783, 2024-03-21T10:35:44.613 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_running_send_time": 0.103425563, 2024-03-21T10:35:44.614 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_running_recv_time": 0.064136690, 2024-03-21T10:35:44.614 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_running_fast_dispatch_time": 0.079774707, 2024-03-21T10:35:44.614 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_send_messages_queue_lat": { 2024-03-21T10:35:44.614 INFO:teuthology.orchestra.run.smithi047.stdout: "avgcount": 1611, 2024-03-21T10:35:44.614 INFO:teuthology.orchestra.run.smithi047.stdout: "sum": 0.168991482, 2024-03-21T10:35:44.614 INFO:teuthology.orchestra.run.smithi047.stdout: "avgtime": 0.000104898 2024-03-21T10:35:44.614 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:44.614 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_handle_ack_lat": { 2024-03-21T10:35:44.614 INFO:teuthology.orchestra.run.smithi047.stdout: "avgcount": 0, 2024-03-21T10:35:44.614 INFO:teuthology.orchestra.run.smithi047.stdout: "sum": 0.000000000, 2024-03-21T10:35:44.614 INFO:teuthology.orchestra.run.smithi047.stdout: "avgtime": 0.000000000 2024-03-21T10:35:44.615 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:44.615 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_recv_encrypted_bytes": 320, 2024-03-21T10:35:44.615 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_send_encrypted_bytes": 454528 2024-03-21T10:35:44.615 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:44.615 INFO:teuthology.orchestra.run.smithi047.stdout: "AsyncMessenger::Worker-2": { 2024-03-21T10:35:44.615 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_recv_messages": 8, 2024-03-21T10:35:44.616 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_send_messages": 8, 2024-03-21T10:35:44.616 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_recv_bytes": 4573, 2024-03-21T10:35:44.616 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_send_bytes": 1732, 2024-03-21T10:35:44.616 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_created_connections": 5, 2024-03-21T10:35:44.616 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_active_connections": 4, 2024-03-21T10:35:44.616 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_running_total_time": 0.003159074, 2024-03-21T10:35:44.616 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_running_send_time": 0.000200438, 2024-03-21T10:35:44.616 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_running_recv_time": 0.002822736, 2024-03-21T10:35:44.616 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_running_fast_dispatch_time": 0.000076036, 2024-03-21T10:35:44.616 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_send_messages_queue_lat": { 2024-03-21T10:35:44.616 INFO:teuthology.orchestra.run.smithi047.stdout: "avgcount": 7, 2024-03-21T10:35:44.617 INFO:teuthology.orchestra.run.smithi047.stdout: "sum": 0.005544982, 2024-03-21T10:35:44.617 INFO:teuthology.orchestra.run.smithi047.stdout: "avgtime": 0.000792140 2024-03-21T10:35:44.617 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:44.617 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_handle_ack_lat": { 2024-03-21T10:35:44.617 INFO:teuthology.orchestra.run.smithi047.stdout: "avgcount": 0, 2024-03-21T10:35:44.617 INFO:teuthology.orchestra.run.smithi047.stdout: "sum": 0.000000000, 2024-03-21T10:35:44.617 INFO:teuthology.orchestra.run.smithi047.stdout: "avgtime": 0.000000000 2024-03-21T10:35:44.617 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:44.617 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_recv_encrypted_bytes": 2944, 2024-03-21T10:35:44.617 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_send_encrypted_bytes": 448 2024-03-21T10:35:44.617 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:44.617 INFO:teuthology.orchestra.run.smithi047.stdout: "cct": { 2024-03-21T10:35:44.618 INFO:teuthology.orchestra.run.smithi047.stdout: "total_workers": 1, 2024-03-21T10:35:44.618 INFO:teuthology.orchestra.run.smithi047.stdout: "unhealthy_workers": 0 2024-03-21T10:35:44.618 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:44.618 INFO:teuthology.orchestra.run.smithi047.stdout: "finisher-MDSRank": { 2024-03-21T10:35:44.618 INFO:teuthology.orchestra.run.smithi047.stdout: "queue_len": 0, 2024-03-21T10:35:44.618 INFO:teuthology.orchestra.run.smithi047.stdout: "complete_latency": { 2024-03-21T10:35:44.618 INFO:teuthology.orchestra.run.smithi047.stdout: "avgcount": 3815, 2024-03-21T10:35:44.618 INFO:teuthology.orchestra.run.smithi047.stdout: "sum": 0.192310676, 2024-03-21T10:35:44.618 INFO:teuthology.orchestra.run.smithi047.stdout: "avgtime": 0.000050409 2024-03-21T10:35:44.618 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-21T10:35:44.619 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:44.619 INFO:teuthology.orchestra.run.smithi047.stdout: "finisher-PurgeQueue": { 2024-03-21T10:35:44.619 INFO:teuthology.orchestra.run.smithi047.stdout: "queue_len": 0, 2024-03-21T10:35:44.619 INFO:teuthology.orchestra.run.smithi047.stdout: "complete_latency": { 2024-03-21T10:35:44.619 INFO:teuthology.orchestra.run.smithi047.stdout: "avgcount": 0, 2024-03-21T10:35:44.619 INFO:teuthology.orchestra.run.smithi047.stdout: "sum": 0.000000000, 2024-03-21T10:35:44.619 INFO:teuthology.orchestra.run.smithi047.stdout: "avgtime": 0.000000000 2024-03-21T10:35:44.619 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-21T10:35:44.619 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:44.619 INFO:teuthology.orchestra.run.smithi047.stdout: "mds": { 2024-03-21T10:35:44.619 INFO:teuthology.orchestra.run.smithi047.stdout: "request": 0, 2024-03-21T10:35:44.619 INFO:teuthology.orchestra.run.smithi047.stdout: "reply": 0, 2024-03-21T10:35:44.619 INFO:teuthology.orchestra.run.smithi047.stdout: "reply_latency": { 2024-03-21T10:35:44.620 INFO:teuthology.orchestra.run.smithi047.stdout: "avgcount": 0, 2024-03-21T10:35:44.620 INFO:teuthology.orchestra.run.smithi047.stdout: "sum": 0.000000000, 2024-03-21T10:35:44.620 INFO:teuthology.orchestra.run.smithi047.stdout: "avgtime": 0.000000000 2024-03-21T10:35:44.620 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:44.620 INFO:teuthology.orchestra.run.smithi047.stdout: "slow_reply": 0, 2024-03-21T10:35:44.620 INFO:teuthology.orchestra.run.smithi047.stdout: "forward": 0, 2024-03-21T10:35:44.620 INFO:teuthology.orchestra.run.smithi047.stdout: "dir_fetch_complete": 0, 2024-03-21T10:35:44.620 INFO:teuthology.orchestra.run.smithi047.stdout: "dir_fetch_keys": 0, 2024-03-21T10:35:44.620 INFO:teuthology.orchestra.run.smithi047.stdout: "dir_commit": 0, 2024-03-21T10:35:44.620 INFO:teuthology.orchestra.run.smithi047.stdout: "dir_split": 0, 2024-03-21T10:35:44.620 INFO:teuthology.orchestra.run.smithi047.stdout: "dir_merge": 0, 2024-03-21T10:35:44.621 INFO:teuthology.orchestra.run.smithi047.stdout: "inodes": 0, 2024-03-21T10:35:44.621 INFO:teuthology.orchestra.run.smithi047.stdout: "inodes_top": 0, 2024-03-21T10:35:44.621 INFO:teuthology.orchestra.run.smithi047.stdout: "inodes_bottom": 0, 2024-03-21T10:35:44.621 INFO:teuthology.orchestra.run.smithi047.stdout: "inodes_pin_tail": 0, 2024-03-21T10:35:44.621 INFO:teuthology.orchestra.run.smithi047.stdout: "inodes_pinned": 0, 2024-03-21T10:35:44.621 INFO:teuthology.orchestra.run.smithi047.stdout: "inodes_expired": 0, 2024-03-21T10:35:44.621 INFO:teuthology.orchestra.run.smithi047.stdout: "inodes_with_caps": 0, 2024-03-21T10:35:44.621 INFO:teuthology.orchestra.run.smithi047.stdout: "caps": 0, 2024-03-21T10:35:44.621 INFO:teuthology.orchestra.run.smithi047.stdout: "subtrees": 1, 2024-03-21T10:35:44.621 INFO:teuthology.orchestra.run.smithi047.stdout: "traverse": 0, 2024-03-21T10:35:44.622 INFO:teuthology.orchestra.run.smithi047.stdout: "traverse_hit": 0, 2024-03-21T10:35:44.622 INFO:teuthology.orchestra.run.smithi047.stdout: "traverse_forward": 0, 2024-03-21T10:35:44.622 INFO:teuthology.orchestra.run.smithi047.stdout: "traverse_discover": 0, 2024-03-21T10:35:44.622 INFO:teuthology.orchestra.run.smithi047.stdout: "traverse_dir_fetch": 0, 2024-03-21T10:35:44.622 INFO:teuthology.orchestra.run.smithi047.stdout: "traverse_remote_ino": 0, 2024-03-21T10:35:44.622 INFO:teuthology.orchestra.run.smithi047.stdout: "traverse_lock": 0, 2024-03-21T10:35:44.622 INFO:teuthology.orchestra.run.smithi047.stdout: "load_cent": 0, 2024-03-21T10:35:44.622 INFO:teuthology.orchestra.run.smithi047.stdout: "q": 0, 2024-03-21T10:35:44.622 INFO:teuthology.orchestra.run.smithi047.stdout: "exported": 0, 2024-03-21T10:35:44.622 INFO:teuthology.orchestra.run.smithi047.stdout: "exported_inodes": 0, 2024-03-21T10:35:44.622 INFO:teuthology.orchestra.run.smithi047.stdout: "imported": 0, 2024-03-21T10:35:44.622 INFO:teuthology.orchestra.run.smithi047.stdout: "imported_inodes": 0, 2024-03-21T10:35:44.623 INFO:teuthology.orchestra.run.smithi047.stdout: "openino_dir_fetch": 0, 2024-03-21T10:35:44.623 INFO:teuthology.orchestra.run.smithi047.stdout: "openino_backtrace_fetch": 0, 2024-03-21T10:35:44.623 INFO:teuthology.orchestra.run.smithi047.stdout: "openino_peer_discover": 0, 2024-03-21T10:35:44.623 INFO:teuthology.orchestra.run.smithi047.stdout: "root_rfiles": 0, 2024-03-21T10:35:44.623 INFO:teuthology.orchestra.run.smithi047.stdout: "root_rbytes": 0, 2024-03-21T10:35:44.623 INFO:teuthology.orchestra.run.smithi047.stdout: "root_rsnaps": 0, 2024-03-21T10:35:44.623 INFO:teuthology.orchestra.run.smithi047.stdout: "scrub_backtrace_fetch": 0, 2024-03-21T10:35:44.623 INFO:teuthology.orchestra.run.smithi047.stdout: "scrub_set_tag": 0, 2024-03-21T10:35:44.623 INFO:teuthology.orchestra.run.smithi047.stdout: "scrub_backtrace_repaired": 0, 2024-03-21T10:35:44.623 INFO:teuthology.orchestra.run.smithi047.stdout: "scrub_inotable_repaired": 0, 2024-03-21T10:35:44.623 INFO:teuthology.orchestra.run.smithi047.stdout: "scrub_dir_inodes": 0, 2024-03-21T10:35:44.623 INFO:teuthology.orchestra.run.smithi047.stdout: "scrub_dir_base_inodes": 0, 2024-03-21T10:35:44.624 INFO:teuthology.orchestra.run.smithi047.stdout: "scrub_dirfrag_rstats": 0, 2024-03-21T10:35:44.624 INFO:teuthology.orchestra.run.smithi047.stdout: "scrub_file_inodes": 0, 2024-03-21T10:35:44.624 INFO:teuthology.orchestra.run.smithi047.stdout: "handle_inode_file_caps": 0, 2024-03-21T10:35:44.624 INFO:teuthology.orchestra.run.smithi047.stdout: "ceph_cap_op_revoke": 0, 2024-03-21T10:35:44.624 INFO:teuthology.orchestra.run.smithi047.stdout: "ceph_cap_op_grant": 0, 2024-03-21T10:35:44.624 INFO:teuthology.orchestra.run.smithi047.stdout: "ceph_cap_op_trunc": 0, 2024-03-21T10:35:44.624 INFO:teuthology.orchestra.run.smithi047.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-03-21T10:35:44.624 INFO:teuthology.orchestra.run.smithi047.stdout: "ceph_cap_op_flush_ack": 0, 2024-03-21T10:35:44.624 INFO:teuthology.orchestra.run.smithi047.stdout: "handle_client_caps": 0, 2024-03-21T10:35:44.624 INFO:teuthology.orchestra.run.smithi047.stdout: "handle_client_caps_dirty": 0, 2024-03-21T10:35:44.624 INFO:teuthology.orchestra.run.smithi047.stdout: "handle_client_cap_release": 0, 2024-03-21T10:35:44.625 INFO:teuthology.orchestra.run.smithi047.stdout: "process_request_cap_release": 0 2024-03-21T10:35:44.625 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:44.625 INFO:teuthology.orchestra.run.smithi047.stdout: "mds_cache": { 2024-03-21T10:35:44.625 INFO:teuthology.orchestra.run.smithi047.stdout: "dir_update": 0, 2024-03-21T10:35:44.625 INFO:teuthology.orchestra.run.smithi047.stdout: "dir_update_receipt": 0, 2024-03-21T10:35:44.625 INFO:teuthology.orchestra.run.smithi047.stdout: "dir_try_discover": 0, 2024-03-21T10:35:44.625 INFO:teuthology.orchestra.run.smithi047.stdout: "dir_send_discover": 0, 2024-03-21T10:35:44.625 INFO:teuthology.orchestra.run.smithi047.stdout: "dir_handle_discover": 0, 2024-03-21T10:35:44.625 INFO:teuthology.orchestra.run.smithi047.stdout: "num_strays": 0, 2024-03-21T10:35:44.625 INFO:teuthology.orchestra.run.smithi047.stdout: "num_strays_delayed": 0, 2024-03-21T10:35:44.625 INFO:teuthology.orchestra.run.smithi047.stdout: "num_strays_enqueuing": 0, 2024-03-21T10:35:44.625 INFO:teuthology.orchestra.run.smithi047.stdout: "strays_created": 0, 2024-03-21T10:35:44.625 INFO:teuthology.orchestra.run.smithi047.stdout: "strays_enqueued": 0, 2024-03-21T10:35:44.626 INFO:teuthology.orchestra.run.smithi047.stdout: "strays_reintegrated": 0, 2024-03-21T10:35:44.626 INFO:teuthology.orchestra.run.smithi047.stdout: "strays_migrated": 0, 2024-03-21T10:35:44.626 INFO:teuthology.orchestra.run.smithi047.stdout: "num_recovering_processing": 0, 2024-03-21T10:35:44.626 INFO:teuthology.orchestra.run.smithi047.stdout: "num_recovering_enqueued": 0, 2024-03-21T10:35:44.626 INFO:teuthology.orchestra.run.smithi047.stdout: "num_recovering_prioritized": 0, 2024-03-21T10:35:44.626 INFO:teuthology.orchestra.run.smithi047.stdout: "recovery_started": 0, 2024-03-21T10:35:44.626 INFO:teuthology.orchestra.run.smithi047.stdout: "recovery_completed": 0, 2024-03-21T10:35:44.626 INFO:teuthology.orchestra.run.smithi047.stdout: "ireq_enqueue_scrub": 0, 2024-03-21T10:35:44.626 INFO:teuthology.orchestra.run.smithi047.stdout: "ireq_exportdir": 0, 2024-03-21T10:35:44.626 INFO:teuthology.orchestra.run.smithi047.stdout: "ireq_flush": 0, 2024-03-21T10:35:44.626 INFO:teuthology.orchestra.run.smithi047.stdout: "ireq_fragmentdir": 0, 2024-03-21T10:35:44.626 INFO:teuthology.orchestra.run.smithi047.stdout: "ireq_fragstats": 0, 2024-03-21T10:35:44.627 INFO:teuthology.orchestra.run.smithi047.stdout: "ireq_inodestats": 0 2024-03-21T10:35:44.627 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:44.627 INFO:teuthology.orchestra.run.smithi047.stdout: "mds_log": { 2024-03-21T10:35:44.627 INFO:teuthology.orchestra.run.smithi047.stdout: "evlrg": 0, 2024-03-21T10:35:44.627 INFO:teuthology.orchestra.run.smithi047.stdout: "evadd": 0, 2024-03-21T10:35:44.627 INFO:teuthology.orchestra.run.smithi047.stdout: "evex": 0, 2024-03-21T10:35:44.627 INFO:teuthology.orchestra.run.smithi047.stdout: "evtrm": 0, 2024-03-21T10:35:44.627 INFO:teuthology.orchestra.run.smithi047.stdout: "ev": 4, 2024-03-21T10:35:44.627 INFO:teuthology.orchestra.run.smithi047.stdout: "evexg": 0, 2024-03-21T10:35:44.627 INFO:teuthology.orchestra.run.smithi047.stdout: "evexd": 0, 2024-03-21T10:35:44.627 INFO:teuthology.orchestra.run.smithi047.stdout: "segadd": 0, 2024-03-21T10:35:44.628 INFO:teuthology.orchestra.run.smithi047.stdout: "segex": 0, 2024-03-21T10:35:44.628 INFO:teuthology.orchestra.run.smithi047.stdout: "segtrm": 0, 2024-03-21T10:35:44.628 INFO:teuthology.orchestra.run.smithi047.stdout: "seg": 2, 2024-03-21T10:35:44.628 INFO:teuthology.orchestra.run.smithi047.stdout: "segmjr": 2, 2024-03-21T10:35:44.628 INFO:teuthology.orchestra.run.smithi047.stdout: "segexg": 0, 2024-03-21T10:35:44.628 INFO:teuthology.orchestra.run.smithi047.stdout: "segexd": 0, 2024-03-21T10:35:44.628 INFO:teuthology.orchestra.run.smithi047.stdout: "expos": 4194304, 2024-03-21T10:35:44.628 INFO:teuthology.orchestra.run.smithi047.stdout: "wrpos": 4195776, 2024-03-21T10:35:44.628 INFO:teuthology.orchestra.run.smithi047.stdout: "rdpos": 4195385, 2024-03-21T10:35:44.628 INFO:teuthology.orchestra.run.smithi047.stdout: "jlat": { 2024-03-21T10:35:44.628 INFO:teuthology.orchestra.run.smithi047.stdout: "avgcount": 0, 2024-03-21T10:35:44.628 INFO:teuthology.orchestra.run.smithi047.stdout: "sum": 0.000000000, 2024-03-21T10:35:44.629 INFO:teuthology.orchestra.run.smithi047.stdout: "avgtime": 0.000000000 2024-03-21T10:35:44.629 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:44.629 INFO:teuthology.orchestra.run.smithi047.stdout: "replayed": 4 2024-03-21T10:35:44.629 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:44.629 INFO:teuthology.orchestra.run.smithi047.stdout: "mds_mem": { 2024-03-21T10:35:44.629 INFO:teuthology.orchestra.run.smithi047.stdout: "ino": 3, 2024-03-21T10:35:44.629 INFO:teuthology.orchestra.run.smithi047.stdout: "ino+": 3, 2024-03-21T10:35:44.629 INFO:teuthology.orchestra.run.smithi047.stdout: "ino-": 0, 2024-03-21T10:35:44.629 INFO:teuthology.orchestra.run.smithi047.stdout: "dir": 1, 2024-03-21T10:35:44.629 INFO:teuthology.orchestra.run.smithi047.stdout: "dir+": 1, 2024-03-21T10:35:44.629 INFO:teuthology.orchestra.run.smithi047.stdout: "dir-": 0, 2024-03-21T10:35:44.629 INFO:teuthology.orchestra.run.smithi047.stdout: "dn": 0, 2024-03-21T10:35:44.629 INFO:teuthology.orchestra.run.smithi047.stdout: "dn+": 0, 2024-03-21T10:35:44.630 INFO:teuthology.orchestra.run.smithi047.stdout: "dn-": 0, 2024-03-21T10:35:44.630 INFO:teuthology.orchestra.run.smithi047.stdout: "cap": 0, 2024-03-21T10:35:44.630 INFO:teuthology.orchestra.run.smithi047.stdout: "cap+": 0, 2024-03-21T10:35:44.630 INFO:teuthology.orchestra.run.smithi047.stdout: "cap-": 0, 2024-03-21T10:35:44.630 INFO:teuthology.orchestra.run.smithi047.stdout: "rss": 49152, 2024-03-21T10:35:44.630 INFO:teuthology.orchestra.run.smithi047.stdout: "heap": 215420 2024-03-21T10:35:44.630 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:44.630 INFO:teuthology.orchestra.run.smithi047.stdout: "mds_server": { 2024-03-21T10:35:44.630 INFO:teuthology.orchestra.run.smithi047.stdout: "dispatch_client_request": 0, 2024-03-21T10:35:44.630 INFO:teuthology.orchestra.run.smithi047.stdout: "dispatch_server_request": 0, 2024-03-21T10:35:44.630 INFO:teuthology.orchestra.run.smithi047.stdout: "handle_client_request": 0, 2024-03-21T10:35:44.631 INFO:teuthology.orchestra.run.smithi047.stdout: "handle_client_session": 0, 2024-03-21T10:35:44.631 INFO:teuthology.orchestra.run.smithi047.stdout: "handle_peer_request": 0, 2024-03-21T10:35:44.631 INFO:teuthology.orchestra.run.smithi047.stdout: "req_create_latency": { 2024-03-21T10:35:44.631 INFO:teuthology.orchestra.run.smithi047.stdout: "avgcount": 0, 2024-03-21T10:35:44.631 INFO:teuthology.orchestra.run.smithi047.stdout: "sum": 0.000000000, 2024-03-21T10:35:44.631 INFO:teuthology.orchestra.run.smithi047.stdout: "avgtime": 0.000000000 2024-03-21T10:35:44.631 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:44.631 INFO:teuthology.orchestra.run.smithi047.stdout: "req_getattr_latency": { 2024-03-21T10:35:44.631 INFO:teuthology.orchestra.run.smithi047.stdout: "avgcount": 0, 2024-03-21T10:35:44.631 INFO:teuthology.orchestra.run.smithi047.stdout: "sum": 0.000000000, 2024-03-21T10:35:44.631 INFO:teuthology.orchestra.run.smithi047.stdout: "avgtime": 0.000000000 2024-03-21T10:35:44.631 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:44.631 INFO:teuthology.orchestra.run.smithi047.stdout: "req_getfilelock_latency": { 2024-03-21T10:35:44.632 INFO:teuthology.orchestra.run.smithi047.stdout: "avgcount": 0, 2024-03-21T10:35:44.632 INFO:teuthology.orchestra.run.smithi047.stdout: "sum": 0.000000000, 2024-03-21T10:35:44.632 INFO:teuthology.orchestra.run.smithi047.stdout: "avgtime": 0.000000000 2024-03-21T10:35:44.632 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:44.632 INFO:teuthology.orchestra.run.smithi047.stdout: "req_link_latency": { 2024-03-21T10:35:44.632 INFO:teuthology.orchestra.run.smithi047.stdout: "avgcount": 0, 2024-03-21T10:35:44.632 INFO:teuthology.orchestra.run.smithi047.stdout: "sum": 0.000000000, 2024-03-21T10:35:44.632 INFO:teuthology.orchestra.run.smithi047.stdout: "avgtime": 0.000000000 2024-03-21T10:35:44.632 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:44.632 INFO:teuthology.orchestra.run.smithi047.stdout: "req_lookup_latency": { 2024-03-21T10:35:44.632 INFO:teuthology.orchestra.run.smithi047.stdout: "avgcount": 0, 2024-03-21T10:35:44.632 INFO:teuthology.orchestra.run.smithi047.stdout: "sum": 0.000000000, 2024-03-21T10:35:44.632 INFO:teuthology.orchestra.run.smithi047.stdout: "avgtime": 0.000000000 2024-03-21T10:35:44.633 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:44.633 INFO:teuthology.orchestra.run.smithi047.stdout: "req_lookuphash_latency": { 2024-03-21T10:35:44.633 INFO:teuthology.orchestra.run.smithi047.stdout: "avgcount": 0, 2024-03-21T10:35:44.633 INFO:teuthology.orchestra.run.smithi047.stdout: "sum": 0.000000000, 2024-03-21T10:35:44.633 INFO:teuthology.orchestra.run.smithi047.stdout: "avgtime": 0.000000000 2024-03-21T10:35:44.633 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:44.633 INFO:teuthology.orchestra.run.smithi047.stdout: "req_lookupino_latency": { 2024-03-21T10:35:44.633 INFO:teuthology.orchestra.run.smithi047.stdout: "avgcount": 0, 2024-03-21T10:35:44.633 INFO:teuthology.orchestra.run.smithi047.stdout: "sum": 0.000000000, 2024-03-21T10:35:44.634 INFO:teuthology.orchestra.run.smithi047.stdout: "avgtime": 0.000000000 2024-03-21T10:35:44.634 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:44.634 INFO:teuthology.orchestra.run.smithi047.stdout: "req_lookupname_latency": { 2024-03-21T10:35:44.634 INFO:teuthology.orchestra.run.smithi047.stdout: "avgcount": 0, 2024-03-21T10:35:44.634 INFO:teuthology.orchestra.run.smithi047.stdout: "sum": 0.000000000, 2024-03-21T10:35:44.634 INFO:teuthology.orchestra.run.smithi047.stdout: "avgtime": 0.000000000 2024-03-21T10:35:44.634 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:44.634 INFO:teuthology.orchestra.run.smithi047.stdout: "req_lookupparent_latency": { 2024-03-21T10:35:44.634 INFO:teuthology.orchestra.run.smithi047.stdout: "avgcount": 0, 2024-03-21T10:35:44.634 INFO:teuthology.orchestra.run.smithi047.stdout: "sum": 0.000000000, 2024-03-21T10:35:44.634 INFO:teuthology.orchestra.run.smithi047.stdout: "avgtime": 0.000000000 2024-03-21T10:35:44.634 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:44.635 INFO:teuthology.orchestra.run.smithi047.stdout: "req_lookupsnap_latency": { 2024-03-21T10:35:44.635 INFO:teuthology.orchestra.run.smithi047.stdout: "avgcount": 0, 2024-03-21T10:35:44.635 INFO:teuthology.orchestra.run.smithi047.stdout: "sum": 0.000000000, 2024-03-21T10:35:44.635 INFO:teuthology.orchestra.run.smithi047.stdout: "avgtime": 0.000000000 2024-03-21T10:35:44.635 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:44.635 INFO:teuthology.orchestra.run.smithi047.stdout: "req_lssnap_latency": { 2024-03-21T10:35:44.635 INFO:teuthology.orchestra.run.smithi047.stdout: "avgcount": 0, 2024-03-21T10:35:44.635 INFO:teuthology.orchestra.run.smithi047.stdout: "sum": 0.000000000, 2024-03-21T10:35:44.635 INFO:teuthology.orchestra.run.smithi047.stdout: "avgtime": 0.000000000 2024-03-21T10:35:44.635 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:44.635 INFO:teuthology.orchestra.run.smithi047.stdout: "req_mkdir_latency": { 2024-03-21T10:35:44.635 INFO:teuthology.orchestra.run.smithi047.stdout: "avgcount": 0, 2024-03-21T10:35:44.635 INFO:teuthology.orchestra.run.smithi047.stdout: "sum": 0.000000000, 2024-03-21T10:35:44.636 INFO:teuthology.orchestra.run.smithi047.stdout: "avgtime": 0.000000000 2024-03-21T10:35:44.636 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:44.636 INFO:teuthology.orchestra.run.smithi047.stdout: "req_mknod_latency": { 2024-03-21T10:35:44.636 INFO:teuthology.orchestra.run.smithi047.stdout: "avgcount": 0, 2024-03-21T10:35:44.637 INFO:teuthology.orchestra.run.smithi047.stdout: "sum": 0.000000000, 2024-03-21T10:35:44.637 INFO:teuthology.orchestra.run.smithi047.stdout: "avgtime": 0.000000000 2024-03-21T10:35:44.637 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:44.637 INFO:teuthology.orchestra.run.smithi047.stdout: "req_mksnap_latency": { 2024-03-21T10:35:44.637 INFO:teuthology.orchestra.run.smithi047.stdout: "avgcount": 0, 2024-03-21T10:35:44.637 INFO:teuthology.orchestra.run.smithi047.stdout: "sum": 0.000000000, 2024-03-21T10:35:44.637 INFO:teuthology.orchestra.run.smithi047.stdout: "avgtime": 0.000000000 2024-03-21T10:35:44.637 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:44.638 INFO:teuthology.orchestra.run.smithi047.stdout: "req_open_latency": { 2024-03-21T10:35:44.638 INFO:teuthology.orchestra.run.smithi047.stdout: "avgcount": 0, 2024-03-21T10:35:44.638 INFO:teuthology.orchestra.run.smithi047.stdout: "sum": 0.000000000, 2024-03-21T10:35:44.638 INFO:teuthology.orchestra.run.smithi047.stdout: "avgtime": 0.000000000 2024-03-21T10:35:44.638 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:44.638 INFO:teuthology.orchestra.run.smithi047.stdout: "req_readdir_latency": { 2024-03-21T10:35:44.638 INFO:teuthology.orchestra.run.smithi047.stdout: "avgcount": 0, 2024-03-21T10:35:44.638 INFO:teuthology.orchestra.run.smithi047.stdout: "sum": 0.000000000, 2024-03-21T10:35:44.638 INFO:teuthology.orchestra.run.smithi047.stdout: "avgtime": 0.000000000 2024-03-21T10:35:44.638 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:44.638 INFO:teuthology.orchestra.run.smithi047.stdout: "req_rename_latency": { 2024-03-21T10:35:44.639 INFO:teuthology.orchestra.run.smithi047.stdout: "avgcount": 0, 2024-03-21T10:35:44.639 INFO:teuthology.orchestra.run.smithi047.stdout: "sum": 0.000000000, 2024-03-21T10:35:44.639 INFO:teuthology.orchestra.run.smithi047.stdout: "avgtime": 0.000000000 2024-03-21T10:35:44.639 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:44.639 INFO:teuthology.orchestra.run.smithi047.stdout: "req_renamesnap_latency": { 2024-03-21T10:35:44.639 INFO:teuthology.orchestra.run.smithi047.stdout: "avgcount": 0, 2024-03-21T10:35:44.639 INFO:teuthology.orchestra.run.smithi047.stdout: "sum": 0.000000000, 2024-03-21T10:35:44.639 INFO:teuthology.orchestra.run.smithi047.stdout: "avgtime": 0.000000000 2024-03-21T10:35:44.639 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:44.639 INFO:teuthology.orchestra.run.smithi047.stdout: "req_snapdiff_latency": { 2024-03-21T10:35:44.639 INFO:teuthology.orchestra.run.smithi047.stdout: "avgcount": 0, 2024-03-21T10:35:44.639 INFO:teuthology.orchestra.run.smithi047.stdout: "sum": 0.000000000, 2024-03-21T10:35:44.640 INFO:teuthology.orchestra.run.smithi047.stdout: "avgtime": 0.000000000 2024-03-21T10:35:44.640 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:44.640 INFO:teuthology.orchestra.run.smithi047.stdout: "req_rmdir_latency": { 2024-03-21T10:35:44.640 INFO:teuthology.orchestra.run.smithi047.stdout: "avgcount": 0, 2024-03-21T10:35:44.640 INFO:teuthology.orchestra.run.smithi047.stdout: "sum": 0.000000000, 2024-03-21T10:35:44.640 INFO:teuthology.orchestra.run.smithi047.stdout: "avgtime": 0.000000000 2024-03-21T10:35:44.640 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:44.640 INFO:teuthology.orchestra.run.smithi047.stdout: "req_rmsnap_latency": { 2024-03-21T10:35:44.640 INFO:teuthology.orchestra.run.smithi047.stdout: "avgcount": 0, 2024-03-21T10:35:44.640 INFO:teuthology.orchestra.run.smithi047.stdout: "sum": 0.000000000, 2024-03-21T10:35:44.640 INFO:teuthology.orchestra.run.smithi047.stdout: "avgtime": 0.000000000 2024-03-21T10:35:44.640 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:44.641 INFO:teuthology.orchestra.run.smithi047.stdout: "req_rmxattr_latency": { 2024-03-21T10:35:44.641 INFO:teuthology.orchestra.run.smithi047.stdout: "avgcount": 0, 2024-03-21T10:35:44.641 INFO:teuthology.orchestra.run.smithi047.stdout: "sum": 0.000000000, 2024-03-21T10:35:44.641 INFO:teuthology.orchestra.run.smithi047.stdout: "avgtime": 0.000000000 2024-03-21T10:35:44.641 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:44.641 INFO:teuthology.orchestra.run.smithi047.stdout: "req_setattr_latency": { 2024-03-21T10:35:44.641 INFO:teuthology.orchestra.run.smithi047.stdout: "avgcount": 0, 2024-03-21T10:35:44.641 INFO:teuthology.orchestra.run.smithi047.stdout: "sum": 0.000000000, 2024-03-21T10:35:44.641 INFO:teuthology.orchestra.run.smithi047.stdout: "avgtime": 0.000000000 2024-03-21T10:35:44.641 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:44.641 INFO:teuthology.orchestra.run.smithi047.stdout: "req_setdirlayout_latency": { 2024-03-21T10:35:44.641 INFO:teuthology.orchestra.run.smithi047.stdout: "avgcount": 0, 2024-03-21T10:35:44.642 INFO:teuthology.orchestra.run.smithi047.stdout: "sum": 0.000000000, 2024-03-21T10:35:44.642 INFO:teuthology.orchestra.run.smithi047.stdout: "avgtime": 0.000000000 2024-03-21T10:35:44.642 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:44.642 INFO:teuthology.orchestra.run.smithi047.stdout: "req_setfilelock_latency": { 2024-03-21T10:35:44.642 INFO:teuthology.orchestra.run.smithi047.stdout: "avgcount": 0, 2024-03-21T10:35:44.642 INFO:teuthology.orchestra.run.smithi047.stdout: "sum": 0.000000000, 2024-03-21T10:35:44.642 INFO:teuthology.orchestra.run.smithi047.stdout: "avgtime": 0.000000000 2024-03-21T10:35:44.642 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:44.642 INFO:teuthology.orchestra.run.smithi047.stdout: "req_setlayout_latency": { 2024-03-21T10:35:44.642 INFO:teuthology.orchestra.run.smithi047.stdout: "avgcount": 0, 2024-03-21T10:35:44.642 INFO:teuthology.orchestra.run.smithi047.stdout: "sum": 0.000000000, 2024-03-21T10:35:44.643 INFO:teuthology.orchestra.run.smithi047.stdout: "avgtime": 0.000000000 2024-03-21T10:35:44.643 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:44.643 INFO:teuthology.orchestra.run.smithi047.stdout: "req_setxattr_latency": { 2024-03-21T10:35:44.643 INFO:teuthology.orchestra.run.smithi047.stdout: "avgcount": 0, 2024-03-21T10:35:44.643 INFO:teuthology.orchestra.run.smithi047.stdout: "sum": 0.000000000, 2024-03-21T10:35:44.643 INFO:teuthology.orchestra.run.smithi047.stdout: "avgtime": 0.000000000 2024-03-21T10:35:44.643 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:44.643 INFO:teuthology.orchestra.run.smithi047.stdout: "req_symlink_latency": { 2024-03-21T10:35:44.643 INFO:teuthology.orchestra.run.smithi047.stdout: "avgcount": 0, 2024-03-21T10:35:44.644 INFO:teuthology.orchestra.run.smithi047.stdout: "sum": 0.000000000, 2024-03-21T10:35:44.644 INFO:teuthology.orchestra.run.smithi047.stdout: "avgtime": 0.000000000 2024-03-21T10:35:44.644 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:44.644 INFO:teuthology.orchestra.run.smithi047.stdout: "req_unlink_latency": { 2024-03-21T10:35:44.644 INFO:teuthology.orchestra.run.smithi047.stdout: "avgcount": 0, 2024-03-21T10:35:44.644 INFO:teuthology.orchestra.run.smithi047.stdout: "sum": 0.000000000, 2024-03-21T10:35:44.644 INFO:teuthology.orchestra.run.smithi047.stdout: "avgtime": 0.000000000 2024-03-21T10:35:44.644 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:44.644 INFO:teuthology.orchestra.run.smithi047.stdout: "cap_revoke_eviction": 0, 2024-03-21T10:35:44.644 INFO:teuthology.orchestra.run.smithi047.stdout: "cap_acquisition_throttle": 0, 2024-03-21T10:35:44.644 INFO:teuthology.orchestra.run.smithi047.stdout: "req_getvxattr_latency": { 2024-03-21T10:35:44.644 INFO:teuthology.orchestra.run.smithi047.stdout: "avgcount": 0, 2024-03-21T10:35:44.645 INFO:teuthology.orchestra.run.smithi047.stdout: "sum": 0.000000000, 2024-03-21T10:35:44.645 INFO:teuthology.orchestra.run.smithi047.stdout: "avgtime": 0.000000000 2024-03-21T10:35:44.645 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-21T10:35:44.645 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:44.645 INFO:teuthology.orchestra.run.smithi047.stdout: "mds_sessions": { 2024-03-21T10:35:44.645 INFO:teuthology.orchestra.run.smithi047.stdout: "session_count": 2, 2024-03-21T10:35:44.645 INFO:teuthology.orchestra.run.smithi047.stdout: "session_add": 2, 2024-03-21T10:35:44.645 INFO:teuthology.orchestra.run.smithi047.stdout: "session_remove": 0, 2024-03-21T10:35:44.645 INFO:teuthology.orchestra.run.smithi047.stdout: "sessions_open": 2, 2024-03-21T10:35:44.645 INFO:teuthology.orchestra.run.smithi047.stdout: "sessions_stale": 0, 2024-03-21T10:35:44.645 INFO:teuthology.orchestra.run.smithi047.stdout: "total_load": 0, 2024-03-21T10:35:44.645 INFO:teuthology.orchestra.run.smithi047.stdout: "average_load": 0, 2024-03-21T10:35:44.645 INFO:teuthology.orchestra.run.smithi047.stdout: "avg_session_uptime": 2121, 2024-03-21T10:35:44.646 INFO:teuthology.orchestra.run.smithi047.stdout: "mdthresh_evicted": 0 2024-03-21T10:35:44.646 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:44.646 INFO:teuthology.orchestra.run.smithi047.stdout: "mempool": { 2024-03-21T10:35:44.646 INFO:teuthology.orchestra.run.smithi047.stdout: "bloom_filter_bytes": 0, 2024-03-21T10:35:44.646 INFO:teuthology.orchestra.run.smithi047.stdout: "bloom_filter_items": 0, 2024-03-21T10:35:44.646 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_alloc_bytes": 0, 2024-03-21T10:35:44.646 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_alloc_items": 0, 2024-03-21T10:35:44.646 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_cache_data_bytes": 0, 2024-03-21T10:35:44.646 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_cache_data_items": 0, 2024-03-21T10:35:44.646 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_cache_onode_bytes": 0, 2024-03-21T10:35:44.646 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_cache_onode_items": 0, 2024-03-21T10:35:44.646 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_cache_meta_bytes": 0, 2024-03-21T10:35:44.647 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_cache_meta_items": 0, 2024-03-21T10:35:44.647 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_cache_other_bytes": 0, 2024-03-21T10:35:44.647 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_cache_other_items": 0, 2024-03-21T10:35:44.647 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_cache_buffer_bytes": 0, 2024-03-21T10:35:44.647 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_cache_buffer_items": 0, 2024-03-21T10:35:44.647 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_extent_bytes": 0, 2024-03-21T10:35:44.647 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_extent_items": 0, 2024-03-21T10:35:44.647 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_blob_bytes": 0, 2024-03-21T10:35:44.647 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_blob_items": 0, 2024-03-21T10:35:44.647 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_shared_blob_bytes": 0, 2024-03-21T10:35:44.647 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_shared_blob_items": 0, 2024-03-21T10:35:44.647 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_inline_bl_bytes": 0, 2024-03-21T10:35:44.647 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_inline_bl_items": 0, 2024-03-21T10:35:44.648 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_fsck_bytes": 0, 2024-03-21T10:35:44.648 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_fsck_items": 0, 2024-03-21T10:35:44.648 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_txc_bytes": 0, 2024-03-21T10:35:44.648 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_txc_items": 0, 2024-03-21T10:35:44.648 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_writing_deferred_bytes": 0, 2024-03-21T10:35:44.648 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_writing_deferred_items": 0, 2024-03-21T10:35:44.648 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_writing_bytes": 0, 2024-03-21T10:35:44.648 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_writing_items": 0, 2024-03-21T10:35:44.648 INFO:teuthology.orchestra.run.smithi047.stdout: "bluefs_bytes": 0, 2024-03-21T10:35:44.648 INFO:teuthology.orchestra.run.smithi047.stdout: "bluefs_items": 0, 2024-03-21T10:35:44.648 INFO:teuthology.orchestra.run.smithi047.stdout: "bluefs_file_reader_bytes": 0, 2024-03-21T10:35:44.648 INFO:teuthology.orchestra.run.smithi047.stdout: "bluefs_file_reader_items": 0, 2024-03-21T10:35:44.649 INFO:teuthology.orchestra.run.smithi047.stdout: "bluefs_file_writer_bytes": 0, 2024-03-21T10:35:44.649 INFO:teuthology.orchestra.run.smithi047.stdout: "bluefs_file_writer_items": 0, 2024-03-21T10:35:44.649 INFO:teuthology.orchestra.run.smithi047.stdout: "buffer_anon_bytes": 198241, 2024-03-21T10:35:44.649 INFO:teuthology.orchestra.run.smithi047.stdout: "buffer_anon_items": 56, 2024-03-21T10:35:44.649 INFO:teuthology.orchestra.run.smithi047.stdout: "buffer_meta_bytes": 0, 2024-03-21T10:35:44.649 INFO:teuthology.orchestra.run.smithi047.stdout: "buffer_meta_items": 0, 2024-03-21T10:35:44.649 INFO:teuthology.orchestra.run.smithi047.stdout: "osd_bytes": 0, 2024-03-21T10:35:44.649 INFO:teuthology.orchestra.run.smithi047.stdout: "osd_items": 0, 2024-03-21T10:35:44.649 INFO:teuthology.orchestra.run.smithi047.stdout: "osd_mapbl_bytes": 0, 2024-03-21T10:35:44.649 INFO:teuthology.orchestra.run.smithi047.stdout: "osd_mapbl_items": 0, 2024-03-21T10:35:44.649 INFO:teuthology.orchestra.run.smithi047.stdout: "osd_pglog_bytes": 0, 2024-03-21T10:35:44.649 INFO:teuthology.orchestra.run.smithi047.stdout: "osd_pglog_items": 0, 2024-03-21T10:35:44.649 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap_bytes": 9712, 2024-03-21T10:35:44.649 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap_items": 187, 2024-03-21T10:35:44.650 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap_mapping_bytes": 0, 2024-03-21T10:35:44.650 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap_mapping_items": 0, 2024-03-21T10:35:44.650 INFO:teuthology.orchestra.run.smithi047.stdout: "pgmap_bytes": 0, 2024-03-21T10:35:44.650 INFO:teuthology.orchestra.run.smithi047.stdout: "pgmap_items": 0, 2024-03-21T10:35:44.650 INFO:teuthology.orchestra.run.smithi047.stdout: "mds_co_bytes": 8320, 2024-03-21T10:35:44.650 INFO:teuthology.orchestra.run.smithi047.stdout: "mds_co_items": 13, 2024-03-21T10:35:44.650 INFO:teuthology.orchestra.run.smithi047.stdout: "unittest_1_bytes": 0, 2024-03-21T10:35:44.650 INFO:teuthology.orchestra.run.smithi047.stdout: "unittest_1_items": 0, 2024-03-21T10:35:44.650 INFO:teuthology.orchestra.run.smithi047.stdout: "unittest_2_bytes": 0, 2024-03-21T10:35:44.650 INFO:teuthology.orchestra.run.smithi047.stdout: "unittest_2_items": 0 2024-03-21T10:35:44.650 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:44.650 INFO:teuthology.orchestra.run.smithi047.stdout: "objecter": { 2024-03-21T10:35:44.651 INFO:teuthology.orchestra.run.smithi047.stdout: "op_active": 0, 2024-03-21T10:35:44.651 INFO:teuthology.orchestra.run.smithi047.stdout: "op_laggy": 0, 2024-03-21T10:35:44.651 INFO:teuthology.orchestra.run.smithi047.stdout: "op_send": 2240, 2024-03-21T10:35:44.651 INFO:teuthology.orchestra.run.smithi047.stdout: "op_send_bytes": 21, 2024-03-21T10:35:44.651 INFO:teuthology.orchestra.run.smithi047.stdout: "op_resend": 3, 2024-03-21T10:35:44.651 INFO:teuthology.orchestra.run.smithi047.stdout: "op_reply": 2240, 2024-03-21T10:35:44.651 INFO:teuthology.orchestra.run.smithi047.stdout: "op_latency": { 2024-03-21T10:35:44.651 INFO:teuthology.orchestra.run.smithi047.stdout: "avgcount": 2240, 2024-03-21T10:35:44.651 INFO:teuthology.orchestra.run.smithi047.stdout: "sum": 3.215928903, 2024-03-21T10:35:44.651 INFO:teuthology.orchestra.run.smithi047.stdout: "avgtime": 0.001435682 2024-03-21T10:35:44.651 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:44.651 INFO:teuthology.orchestra.run.smithi047.stdout: "op_inflight": 0, 2024-03-21T10:35:44.652 INFO:teuthology.orchestra.run.smithi047.stdout: "oplen_avg": { 2024-03-21T10:35:44.652 INFO:teuthology.orchestra.run.smithi047.stdout: "avgcount": 2240, 2024-03-21T10:35:44.652 INFO:teuthology.orchestra.run.smithi047.stdout: "sum": 2241 2024-03-21T10:35:44.652 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:44.652 INFO:teuthology.orchestra.run.smithi047.stdout: "op": 2240, 2024-03-21T10:35:44.652 INFO:teuthology.orchestra.run.smithi047.stdout: "op_r": 2240, 2024-03-21T10:35:44.652 INFO:teuthology.orchestra.run.smithi047.stdout: "op_w": 0, 2024-03-21T10:35:44.652 INFO:teuthology.orchestra.run.smithi047.stdout: "op_rmw": 0, 2024-03-21T10:35:44.652 INFO:teuthology.orchestra.run.smithi047.stdout: "op_pg": 0, 2024-03-21T10:35:44.652 INFO:teuthology.orchestra.run.smithi047.stdout: "osdop_stat": 1439, 2024-03-21T10:35:44.652 INFO:teuthology.orchestra.run.smithi047.stdout: "osdop_create": 0, 2024-03-21T10:35:44.652 INFO:teuthology.orchestra.run.smithi047.stdout: "osdop_read": 799, 2024-03-21T10:35:44.653 INFO:teuthology.orchestra.run.smithi047.stdout: "osdop_write": 0, 2024-03-21T10:35:44.653 INFO:teuthology.orchestra.run.smithi047.stdout: "osdop_writefull": 0, 2024-03-21T10:35:44.653 INFO:teuthology.orchestra.run.smithi047.stdout: "osdop_writesame": 0, 2024-03-21T10:35:44.653 INFO:teuthology.orchestra.run.smithi047.stdout: "osdop_append": 0, 2024-03-21T10:35:44.653 INFO:teuthology.orchestra.run.smithi047.stdout: "osdop_zero": 0, 2024-03-21T10:35:44.653 INFO:teuthology.orchestra.run.smithi047.stdout: "osdop_truncate": 0, 2024-03-21T10:35:44.653 INFO:teuthology.orchestra.run.smithi047.stdout: "osdop_delete": 0, 2024-03-21T10:35:44.653 INFO:teuthology.orchestra.run.smithi047.stdout: "osdop_mapext": 0, 2024-03-21T10:35:44.653 INFO:teuthology.orchestra.run.smithi047.stdout: "osdop_sparse_read": 0, 2024-03-21T10:35:44.653 INFO:teuthology.orchestra.run.smithi047.stdout: "osdop_clonerange": 0, 2024-03-21T10:35:44.653 INFO:teuthology.orchestra.run.smithi047.stdout: "osdop_getxattr": 1, 2024-03-21T10:35:44.653 INFO:teuthology.orchestra.run.smithi047.stdout: "osdop_setxattr": 0, 2024-03-21T10:35:44.653 INFO:teuthology.orchestra.run.smithi047.stdout: "osdop_cmpxattr": 0, 2024-03-21T10:35:44.654 INFO:teuthology.orchestra.run.smithi047.stdout: "osdop_rmxattr": 0, 2024-03-21T10:35:44.654 INFO:teuthology.orchestra.run.smithi047.stdout: "osdop_resetxattrs": 0, 2024-03-21T10:35:44.654 INFO:teuthology.orchestra.run.smithi047.stdout: "osdop_call": 0, 2024-03-21T10:35:44.654 INFO:teuthology.orchestra.run.smithi047.stdout: "osdop_watch": 0, 2024-03-21T10:35:44.654 INFO:teuthology.orchestra.run.smithi047.stdout: "osdop_notify": 0, 2024-03-21T10:35:44.654 INFO:teuthology.orchestra.run.smithi047.stdout: "osdop_src_cmpxattr": 0, 2024-03-21T10:35:44.654 INFO:teuthology.orchestra.run.smithi047.stdout: "osdop_pgls": 0, 2024-03-21T10:35:44.654 INFO:teuthology.orchestra.run.smithi047.stdout: "osdop_pgls_filter": 0, 2024-03-21T10:35:44.654 INFO:teuthology.orchestra.run.smithi047.stdout: "osdop_other": 0, 2024-03-21T10:35:44.654 INFO:teuthology.orchestra.run.smithi047.stdout: "linger_active": 0, 2024-03-21T10:35:44.654 INFO:teuthology.orchestra.run.smithi047.stdout: "linger_send": 0, 2024-03-21T10:35:44.655 INFO:teuthology.orchestra.run.smithi047.stdout: "linger_resend": 0, 2024-03-21T10:35:44.655 INFO:teuthology.orchestra.run.smithi047.stdout: "linger_ping": 0, 2024-03-21T10:35:44.655 INFO:teuthology.orchestra.run.smithi047.stdout: "poolop_active": 0, 2024-03-21T10:35:44.655 INFO:teuthology.orchestra.run.smithi047.stdout: "poolop_send": 0, 2024-03-21T10:35:44.655 INFO:teuthology.orchestra.run.smithi047.stdout: "poolop_resend": 0, 2024-03-21T10:35:44.655 INFO:teuthology.orchestra.run.smithi047.stdout: "poolstat_active": 0, 2024-03-21T10:35:44.655 INFO:teuthology.orchestra.run.smithi047.stdout: "poolstat_send": 0, 2024-03-21T10:35:44.655 INFO:teuthology.orchestra.run.smithi047.stdout: "poolstat_resend": 0, 2024-03-21T10:35:44.655 INFO:teuthology.orchestra.run.smithi047.stdout: "statfs_active": 0, 2024-03-21T10:35:44.655 INFO:teuthology.orchestra.run.smithi047.stdout: "statfs_send": 0, 2024-03-21T10:35:44.655 INFO:teuthology.orchestra.run.smithi047.stdout: "statfs_resend": 0, 2024-03-21T10:35:44.655 INFO:teuthology.orchestra.run.smithi047.stdout: "command_active": 0, 2024-03-21T10:35:44.655 INFO:teuthology.orchestra.run.smithi047.stdout: "command_send": 0, 2024-03-21T10:35:44.656 INFO:teuthology.orchestra.run.smithi047.stdout: "command_resend": 0, 2024-03-21T10:35:44.656 INFO:teuthology.orchestra.run.smithi047.stdout: "map_epoch": 105, 2024-03-21T10:35:44.656 INFO:teuthology.orchestra.run.smithi047.stdout: "map_full": 0, 2024-03-21T10:35:44.656 INFO:teuthology.orchestra.run.smithi047.stdout: "map_inc": 12, 2024-03-21T10:35:44.656 INFO:teuthology.orchestra.run.smithi047.stdout: "osd_sessions": 6, 2024-03-21T10:35:44.656 INFO:teuthology.orchestra.run.smithi047.stdout: "osd_session_open": 6, 2024-03-21T10:35:44.656 INFO:teuthology.orchestra.run.smithi047.stdout: "osd_session_close": 0, 2024-03-21T10:35:44.656 INFO:teuthology.orchestra.run.smithi047.stdout: "osd_laggy": 0, 2024-03-21T10:35:44.656 INFO:teuthology.orchestra.run.smithi047.stdout: "omap_wr": 0, 2024-03-21T10:35:44.656 INFO:teuthology.orchestra.run.smithi047.stdout: "omap_rd": 2, 2024-03-21T10:35:44.656 INFO:teuthology.orchestra.run.smithi047.stdout: "omap_del": 0 2024-03-21T10:35:44.656 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:44.657 INFO:teuthology.orchestra.run.smithi047.stdout: "oft": { 2024-03-21T10:35:44.657 INFO:teuthology.orchestra.run.smithi047.stdout: "omap_total_objs": 0, 2024-03-21T10:35:44.657 INFO:teuthology.orchestra.run.smithi047.stdout: "omap_total_kv_pairs": 0, 2024-03-21T10:35:44.657 INFO:teuthology.orchestra.run.smithi047.stdout: "omap_total_updates": 0, 2024-03-21T10:35:44.657 INFO:teuthology.orchestra.run.smithi047.stdout: "omap_total_removes": 0 2024-03-21T10:35:44.657 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:44.657 INFO:teuthology.orchestra.run.smithi047.stdout: "osd-slow-ops": { 2024-03-21T10:35:44.657 INFO:teuthology.orchestra.run.smithi047.stdout: "slow_ops_count": 0 2024-03-21T10:35:44.657 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:44.657 INFO:teuthology.orchestra.run.smithi047.stdout: "purge_queue": { 2024-03-21T10:35:44.657 INFO:teuthology.orchestra.run.smithi047.stdout: "pq_executing_ops": 0, 2024-03-21T10:35:44.657 INFO:teuthology.orchestra.run.smithi047.stdout: "pq_executing_ops_high_water": 0, 2024-03-21T10:35:44.657 INFO:teuthology.orchestra.run.smithi047.stdout: "pq_executing": 0, 2024-03-21T10:35:44.658 INFO:teuthology.orchestra.run.smithi047.stdout: "pq_executing_high_water": 0, 2024-03-21T10:35:44.658 INFO:teuthology.orchestra.run.smithi047.stdout: "pq_executed_ops": 0, 2024-03-21T10:35:44.658 INFO:teuthology.orchestra.run.smithi047.stdout: "pq_executed": 0, 2024-03-21T10:35:44.658 INFO:teuthology.orchestra.run.smithi047.stdout: "pq_item_in_journal": 0 2024-03-21T10:35:44.658 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:44.658 INFO:teuthology.orchestra.run.smithi047.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-03-21T10:35:44.658 INFO:teuthology.orchestra.run.smithi047.stdout: "val": 0, 2024-03-21T10:35:44.658 INFO:teuthology.orchestra.run.smithi047.stdout: "max": 104857600, 2024-03-21T10:35:44.658 INFO:teuthology.orchestra.run.smithi047.stdout: "get_started": 0, 2024-03-21T10:35:44.658 INFO:teuthology.orchestra.run.smithi047.stdout: "get": 2498, 2024-03-21T10:35:44.658 INFO:teuthology.orchestra.run.smithi047.stdout: "get_sum": 2166462, 2024-03-21T10:35:44.658 INFO:teuthology.orchestra.run.smithi047.stdout: "get_or_fail_fail": 0, 2024-03-21T10:35:44.659 INFO:teuthology.orchestra.run.smithi047.stdout: "get_or_fail_success": 2498, 2024-03-21T10:35:44.659 INFO:teuthology.orchestra.run.smithi047.stdout: "take": 0, 2024-03-21T10:35:44.659 INFO:teuthology.orchestra.run.smithi047.stdout: "take_sum": 0, 2024-03-21T10:35:44.659 INFO:teuthology.orchestra.run.smithi047.stdout: "put": 2498, 2024-03-21T10:35:44.659 INFO:teuthology.orchestra.run.smithi047.stdout: "put_sum": 2166462, 2024-03-21T10:35:44.659 INFO:teuthology.orchestra.run.smithi047.stdout: "wait": { 2024-03-21T10:35:44.659 INFO:teuthology.orchestra.run.smithi047.stdout: "avgcount": 0, 2024-03-21T10:35:44.659 INFO:teuthology.orchestra.run.smithi047.stdout: "sum": 0.000000000, 2024-03-21T10:35:44.659 INFO:teuthology.orchestra.run.smithi047.stdout: "avgtime": 0.000000000 2024-03-21T10:35:44.659 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-21T10:35:44.659 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:44.659 INFO:teuthology.orchestra.run.smithi047.stdout: "throttle-objecter_bytes": { 2024-03-21T10:35:44.659 INFO:teuthology.orchestra.run.smithi047.stdout: "val": 0, 2024-03-21T10:35:44.660 INFO:teuthology.orchestra.run.smithi047.stdout: "max": 104857600, 2024-03-21T10:35:44.660 INFO:teuthology.orchestra.run.smithi047.stdout: "get_started": 0, 2024-03-21T10:35:44.660 INFO:teuthology.orchestra.run.smithi047.stdout: "get": 0, 2024-03-21T10:35:44.660 INFO:teuthology.orchestra.run.smithi047.stdout: "get_sum": 0, 2024-03-21T10:35:44.660 INFO:teuthology.orchestra.run.smithi047.stdout: "get_or_fail_fail": 0, 2024-03-21T10:35:44.660 INFO:teuthology.orchestra.run.smithi047.stdout: "get_or_fail_success": 0, 2024-03-21T10:35:44.660 INFO:teuthology.orchestra.run.smithi047.stdout: "take": 2240, 2024-03-21T10:35:44.660 INFO:teuthology.orchestra.run.smithi047.stdout: "take_sum": 1477, 2024-03-21T10:35:44.660 INFO:teuthology.orchestra.run.smithi047.stdout: "put": 2240, 2024-03-21T10:35:44.660 INFO:teuthology.orchestra.run.smithi047.stdout: "put_sum": 1477, 2024-03-21T10:35:44.660 INFO:teuthology.orchestra.run.smithi047.stdout: "wait": { 2024-03-21T10:35:44.660 INFO:teuthology.orchestra.run.smithi047.stdout: "avgcount": 0, 2024-03-21T10:35:44.660 INFO:teuthology.orchestra.run.smithi047.stdout: "sum": 0.000000000, 2024-03-21T10:35:44.661 INFO:teuthology.orchestra.run.smithi047.stdout: "avgtime": 0.000000000 2024-03-21T10:35:44.661 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-21T10:35:44.661 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:44.661 INFO:teuthology.orchestra.run.smithi047.stdout: "throttle-objecter_ops": { 2024-03-21T10:35:44.661 INFO:teuthology.orchestra.run.smithi047.stdout: "val": 0, 2024-03-21T10:35:44.661 INFO:teuthology.orchestra.run.smithi047.stdout: "max": 1024, 2024-03-21T10:35:44.661 INFO:teuthology.orchestra.run.smithi047.stdout: "get_started": 0, 2024-03-21T10:35:44.661 INFO:teuthology.orchestra.run.smithi047.stdout: "get": 0, 2024-03-21T10:35:44.661 INFO:teuthology.orchestra.run.smithi047.stdout: "get_sum": 0, 2024-03-21T10:35:44.661 INFO:teuthology.orchestra.run.smithi047.stdout: "get_or_fail_fail": 0, 2024-03-21T10:35:44.661 INFO:teuthology.orchestra.run.smithi047.stdout: "get_or_fail_success": 0, 2024-03-21T10:35:44.661 INFO:teuthology.orchestra.run.smithi047.stdout: "take": 2240, 2024-03-21T10:35:44.662 INFO:teuthology.orchestra.run.smithi047.stdout: "take_sum": 2240, 2024-03-21T10:35:44.662 INFO:teuthology.orchestra.run.smithi047.stdout: "put": 2240, 2024-03-21T10:35:44.662 INFO:teuthology.orchestra.run.smithi047.stdout: "put_sum": 2240, 2024-03-21T10:35:44.662 INFO:teuthology.orchestra.run.smithi047.stdout: "wait": { 2024-03-21T10:35:44.662 INFO:teuthology.orchestra.run.smithi047.stdout: "avgcount": 0, 2024-03-21T10:35:44.662 INFO:teuthology.orchestra.run.smithi047.stdout: "sum": 0.000000000, 2024-03-21T10:35:44.662 INFO:teuthology.orchestra.run.smithi047.stdout: "avgtime": 0.000000000 2024-03-21T10:35:44.662 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-21T10:35:44.662 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:44.662 INFO:teuthology.orchestra.run.smithi047.stdout: "throttle-write_buf_throttle": { 2024-03-21T10:35:44.662 INFO:teuthology.orchestra.run.smithi047.stdout: "val": 0, 2024-03-21T10:35:44.662 INFO:teuthology.orchestra.run.smithi047.stdout: "max": 3758096384, 2024-03-21T10:35:44.662 INFO:teuthology.orchestra.run.smithi047.stdout: "get_started": 0, 2024-03-21T10:35:44.663 INFO:teuthology.orchestra.run.smithi047.stdout: "get": 0, 2024-03-21T10:35:44.663 INFO:teuthology.orchestra.run.smithi047.stdout: "get_sum": 0, 2024-03-21T10:35:44.663 INFO:teuthology.orchestra.run.smithi047.stdout: "get_or_fail_fail": 0, 2024-03-21T10:35:44.663 INFO:teuthology.orchestra.run.smithi047.stdout: "get_or_fail_success": 0, 2024-03-21T10:35:44.663 INFO:teuthology.orchestra.run.smithi047.stdout: "take": 0, 2024-03-21T10:35:44.663 INFO:teuthology.orchestra.run.smithi047.stdout: "take_sum": 0, 2024-03-21T10:35:44.663 INFO:teuthology.orchestra.run.smithi047.stdout: "put": 0, 2024-03-21T10:35:44.663 INFO:teuthology.orchestra.run.smithi047.stdout: "put_sum": 0, 2024-03-21T10:35:44.663 INFO:teuthology.orchestra.run.smithi047.stdout: "wait": { 2024-03-21T10:35:44.663 INFO:teuthology.orchestra.run.smithi047.stdout: "avgcount": 0, 2024-03-21T10:35:44.663 INFO:teuthology.orchestra.run.smithi047.stdout: "sum": 0.000000000, 2024-03-21T10:35:44.663 INFO:teuthology.orchestra.run.smithi047.stdout: "avgtime": 0.000000000 2024-03-21T10:35:44.663 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-21T10:35:44.664 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:44.664 INFO:teuthology.orchestra.run.smithi047.stdout: "throttle-write_buf_throttle-0x56390ff7e780": { 2024-03-21T10:35:44.664 INFO:teuthology.orchestra.run.smithi047.stdout: "val": 0, 2024-03-21T10:35:44.664 INFO:teuthology.orchestra.run.smithi047.stdout: "max": 3758096384, 2024-03-21T10:35:44.664 INFO:teuthology.orchestra.run.smithi047.stdout: "get_started": 0, 2024-03-21T10:35:44.664 INFO:teuthology.orchestra.run.smithi047.stdout: "get": 0, 2024-03-21T10:35:44.664 INFO:teuthology.orchestra.run.smithi047.stdout: "get_sum": 0, 2024-03-21T10:35:44.664 INFO:teuthology.orchestra.run.smithi047.stdout: "get_or_fail_fail": 0, 2024-03-21T10:35:44.664 INFO:teuthology.orchestra.run.smithi047.stdout: "get_or_fail_success": 0, 2024-03-21T10:35:44.664 INFO:teuthology.orchestra.run.smithi047.stdout: "take": 0, 2024-03-21T10:35:44.664 INFO:teuthology.orchestra.run.smithi047.stdout: "take_sum": 0, 2024-03-21T10:35:44.664 INFO:teuthology.orchestra.run.smithi047.stdout: "put": 0, 2024-03-21T10:35:44.665 INFO:teuthology.orchestra.run.smithi047.stdout: "put_sum": 0, 2024-03-21T10:35:44.665 INFO:teuthology.orchestra.run.smithi047.stdout: "wait": { 2024-03-21T10:35:44.665 INFO:teuthology.orchestra.run.smithi047.stdout: "avgcount": 0, 2024-03-21T10:35:44.665 INFO:teuthology.orchestra.run.smithi047.stdout: "sum": 0.000000000, 2024-03-21T10:35:44.665 INFO:teuthology.orchestra.run.smithi047.stdout: "avgtime": 0.000000000 2024-03-21T10:35:44.665 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-21T10:35:44.665 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-21T10:35:44.665 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-03-21T10:35:45.461 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:35:45 smithi047 ceph-mon[31547]: pgmap v324: 129 pgs: 129 active+clean; 2.5 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-21T10:35:45.605 INFO:tasks.check_counter:Daemon mds.g mds.exported=0 2024-03-21T10:35:45.605 INFO:tasks.check_counter:Daemon mds.g mds.imported=0 2024-03-21T10:35:45.605 INFO:tasks.check_counter:Daemon mds.g mds.root_rsnaps=0 2024-03-21T10:35:45.605 INFO:tasks.check_counter:Daemon mds.g mds_server.req_mksnap_latency.avgcount=0 2024-03-21T10:35:45.605 INFO:tasks.check_counter:Daemon mds.g mds_server.req_rmsnap_latency.avgcount=0 2024-03-21T10:35:45.605 DEBUG:tasks.check_counter:Getting stats from j 2024-03-21T10:35:45.606 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2edfee68ad7243bed422a049b6f5f58619300f11 shell --fsid 3056d51a-e76b-11ee-95cd-87774f69a715 -- ceph daemon mds.j perf dump 2024-03-21T10:35:45.726 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:35:45 smithi100 ceph-mon[35193]: pgmap v324: 129 pgs: 129 active+clean; 2.5 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-21T10:35:45.736 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:35:45 smithi157 ceph-mon[35581]: pgmap v324: 129 pgs: 129 active+clean; 2.5 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-21T10:35:46.050 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/3056d51a-e76b-11ee-95cd-87774f69a715/mon.a/config 2024-03-21T10:35:47.460 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:35:47 smithi047 ceph-mon[31547]: pgmap v325: 129 pgs: 129 active+clean; 2.5 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 12 op/s 2024-03-21T10:35:47.726 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:35:47 smithi100 ceph-mon[35193]: pgmap v325: 129 pgs: 129 active+clean; 2.5 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 12 op/s 2024-03-21T10:35:47.736 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:35:47 smithi157 ceph-mon[35581]: pgmap v325: 129 pgs: 129 active+clean; 2.5 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 12 op/s 2024-03-21T10:35:48.509 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-03-21T10:35:48.509 INFO:teuthology.orchestra.run.smithi047.stdout: "AsyncMessenger::Worker-0": { 2024-03-21T10:35:48.509 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_recv_messages": 4, 2024-03-21T10:35:48.509 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_send_messages": 4, 2024-03-21T10:35:48.509 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_recv_bytes": 2944, 2024-03-21T10:35:48.509 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_send_bytes": 448, 2024-03-21T10:35:48.509 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_created_connections": 4, 2024-03-21T10:35:48.509 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_active_connections": 0, 2024-03-21T10:35:48.510 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_running_total_time": 0.004672034, 2024-03-21T10:35:48.510 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_running_send_time": 0.000093549, 2024-03-21T10:35:48.510 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_running_recv_time": 0.001619098, 2024-03-21T10:35:48.510 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-03-21T10:35:48.510 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_send_messages_queue_lat": { 2024-03-21T10:35:48.510 INFO:teuthology.orchestra.run.smithi047.stdout: "avgcount": 3, 2024-03-21T10:35:48.510 INFO:teuthology.orchestra.run.smithi047.stdout: "sum": 0.002088633, 2024-03-21T10:35:48.510 INFO:teuthology.orchestra.run.smithi047.stdout: "avgtime": 0.000696211 2024-03-21T10:35:48.510 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:48.510 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_handle_ack_lat": { 2024-03-21T10:35:48.510 INFO:teuthology.orchestra.run.smithi047.stdout: "avgcount": 0, 2024-03-21T10:35:48.510 INFO:teuthology.orchestra.run.smithi047.stdout: "sum": 0.000000000, 2024-03-21T10:35:48.511 INFO:teuthology.orchestra.run.smithi047.stdout: "avgtime": 0.000000000 2024-03-21T10:35:48.511 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:48.511 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_recv_encrypted_bytes": 2944, 2024-03-21T10:35:48.511 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_send_encrypted_bytes": 448 2024-03-21T10:35:48.511 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:48.511 INFO:teuthology.orchestra.run.smithi047.stdout: "AsyncMessenger::Worker-1": { 2024-03-21T10:35:48.511 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_recv_messages": 2, 2024-03-21T10:35:48.511 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_send_messages": 178, 2024-03-21T10:35:48.511 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_recv_bytes": 320, 2024-03-21T10:35:48.511 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_send_bytes": 186352, 2024-03-21T10:35:48.511 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_created_connections": 5, 2024-03-21T10:35:48.511 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_active_connections": 3, 2024-03-21T10:35:48.511 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_running_total_time": 0.036910690, 2024-03-21T10:35:48.511 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_running_send_time": 0.027563539, 2024-03-21T10:35:48.511 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_running_recv_time": 0.003858092, 2024-03-21T10:35:48.512 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-03-21T10:35:48.512 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_send_messages_queue_lat": { 2024-03-21T10:35:48.512 INFO:teuthology.orchestra.run.smithi047.stdout: "avgcount": 175, 2024-03-21T10:35:48.512 INFO:teuthology.orchestra.run.smithi047.stdout: "sum": 0.040349967, 2024-03-21T10:35:48.512 INFO:teuthology.orchestra.run.smithi047.stdout: "avgtime": 0.000230571 2024-03-21T10:35:48.512 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:48.512 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_handle_ack_lat": { 2024-03-21T10:35:48.512 INFO:teuthology.orchestra.run.smithi047.stdout: "avgcount": 0, 2024-03-21T10:35:48.512 INFO:teuthology.orchestra.run.smithi047.stdout: "sum": 0.000000000, 2024-03-21T10:35:48.512 INFO:teuthology.orchestra.run.smithi047.stdout: "avgtime": 0.000000000 2024-03-21T10:35:48.512 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:48.512 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_recv_encrypted_bytes": 320, 2024-03-21T10:35:48.512 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_send_encrypted_bytes": 186352 2024-03-21T10:35:48.512 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:48.512 INFO:teuthology.orchestra.run.smithi047.stdout: "AsyncMessenger::Worker-2": { 2024-03-21T10:35:48.513 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_recv_messages": 244, 2024-03-21T10:35:48.513 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_send_messages": 232, 2024-03-21T10:35:48.513 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_recv_bytes": 1724736, 2024-03-21T10:35:48.513 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_send_bytes": 139760, 2024-03-21T10:35:48.513 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_created_connections": 2, 2024-03-21T10:35:48.513 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_active_connections": 1, 2024-03-21T10:35:48.513 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_running_total_time": 0.103895288, 2024-03-21T10:35:48.513 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_running_send_time": 0.037005261, 2024-03-21T10:35:48.513 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_running_recv_time": 0.075459566, 2024-03-21T10:35:48.513 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_running_fast_dispatch_time": 0.010843307, 2024-03-21T10:35:48.513 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_send_messages_queue_lat": { 2024-03-21T10:35:48.513 INFO:teuthology.orchestra.run.smithi047.stdout: "avgcount": 231, 2024-03-21T10:35:48.513 INFO:teuthology.orchestra.run.smithi047.stdout: "sum": 0.023330376, 2024-03-21T10:35:48.513 INFO:teuthology.orchestra.run.smithi047.stdout: "avgtime": 0.000100997 2024-03-21T10:35:48.513 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:48.513 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_handle_ack_lat": { 2024-03-21T10:35:48.514 INFO:teuthology.orchestra.run.smithi047.stdout: "avgcount": 0, 2024-03-21T10:35:48.514 INFO:teuthology.orchestra.run.smithi047.stdout: "sum": 0.000000000, 2024-03-21T10:35:48.514 INFO:teuthology.orchestra.run.smithi047.stdout: "avgtime": 0.000000000 2024-03-21T10:35:48.514 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:48.514 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_recv_encrypted_bytes": 1724736, 2024-03-21T10:35:48.514 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_send_encrypted_bytes": 139760 2024-03-21T10:35:48.514 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:48.514 INFO:teuthology.orchestra.run.smithi047.stdout: "cct": { 2024-03-21T10:35:48.514 INFO:teuthology.orchestra.run.smithi047.stdout: "total_workers": 0, 2024-03-21T10:35:48.515 INFO:teuthology.orchestra.run.smithi047.stdout: "unhealthy_workers": 0 2024-03-21T10:35:48.515 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:48.515 INFO:teuthology.orchestra.run.smithi047.stdout: "mempool": { 2024-03-21T10:35:48.515 INFO:teuthology.orchestra.run.smithi047.stdout: "bloom_filter_bytes": 0, 2024-03-21T10:35:48.515 INFO:teuthology.orchestra.run.smithi047.stdout: "bloom_filter_items": 0, 2024-03-21T10:35:48.515 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_alloc_bytes": 0, 2024-03-21T10:35:48.515 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_alloc_items": 0, 2024-03-21T10:35:48.515 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_cache_data_bytes": 0, 2024-03-21T10:35:48.515 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_cache_data_items": 0, 2024-03-21T10:35:48.515 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_cache_onode_bytes": 0, 2024-03-21T10:35:48.515 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_cache_onode_items": 0, 2024-03-21T10:35:48.515 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_cache_meta_bytes": 0, 2024-03-21T10:35:48.515 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_cache_meta_items": 0, 2024-03-21T10:35:48.515 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_cache_other_bytes": 0, 2024-03-21T10:35:48.516 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_cache_other_items": 0, 2024-03-21T10:35:48.516 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_cache_buffer_bytes": 0, 2024-03-21T10:35:48.516 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_cache_buffer_items": 0, 2024-03-21T10:35:48.516 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_extent_bytes": 0, 2024-03-21T10:35:48.516 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_extent_items": 0, 2024-03-21T10:35:48.516 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_blob_bytes": 0, 2024-03-21T10:35:48.516 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_blob_items": 0, 2024-03-21T10:35:48.516 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_shared_blob_bytes": 0, 2024-03-21T10:35:48.516 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_shared_blob_items": 0, 2024-03-21T10:35:48.516 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_inline_bl_bytes": 0, 2024-03-21T10:35:48.516 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_inline_bl_items": 0, 2024-03-21T10:35:48.516 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_fsck_bytes": 0, 2024-03-21T10:35:48.516 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_fsck_items": 0, 2024-03-21T10:35:48.516 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_txc_bytes": 0, 2024-03-21T10:35:48.516 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_txc_items": 0, 2024-03-21T10:35:48.517 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_writing_deferred_bytes": 0, 2024-03-21T10:35:48.517 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_writing_deferred_items": 0, 2024-03-21T10:35:48.517 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_writing_bytes": 0, 2024-03-21T10:35:48.517 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_writing_items": 0, 2024-03-21T10:35:48.517 INFO:teuthology.orchestra.run.smithi047.stdout: "bluefs_bytes": 0, 2024-03-21T10:35:48.517 INFO:teuthology.orchestra.run.smithi047.stdout: "bluefs_items": 0, 2024-03-21T10:35:48.517 INFO:teuthology.orchestra.run.smithi047.stdout: "bluefs_file_reader_bytes": 0, 2024-03-21T10:35:48.517 INFO:teuthology.orchestra.run.smithi047.stdout: "bluefs_file_reader_items": 0, 2024-03-21T10:35:48.517 INFO:teuthology.orchestra.run.smithi047.stdout: "bluefs_file_writer_bytes": 0, 2024-03-21T10:35:48.517 INFO:teuthology.orchestra.run.smithi047.stdout: "bluefs_file_writer_items": 0, 2024-03-21T10:35:48.517 INFO:teuthology.orchestra.run.smithi047.stdout: "buffer_anon_bytes": 148794, 2024-03-21T10:35:48.517 INFO:teuthology.orchestra.run.smithi047.stdout: "buffer_anon_items": 32, 2024-03-21T10:35:48.517 INFO:teuthology.orchestra.run.smithi047.stdout: "buffer_meta_bytes": 0, 2024-03-21T10:35:48.517 INFO:teuthology.orchestra.run.smithi047.stdout: "buffer_meta_items": 0, 2024-03-21T10:35:48.517 INFO:teuthology.orchestra.run.smithi047.stdout: "osd_bytes": 0, 2024-03-21T10:35:48.518 INFO:teuthology.orchestra.run.smithi047.stdout: "osd_items": 0, 2024-03-21T10:35:48.518 INFO:teuthology.orchestra.run.smithi047.stdout: "osd_mapbl_bytes": 0, 2024-03-21T10:35:48.518 INFO:teuthology.orchestra.run.smithi047.stdout: "osd_mapbl_items": 0, 2024-03-21T10:35:48.518 INFO:teuthology.orchestra.run.smithi047.stdout: "osd_pglog_bytes": 0, 2024-03-21T10:35:48.518 INFO:teuthology.orchestra.run.smithi047.stdout: "osd_pglog_items": 0, 2024-03-21T10:35:48.518 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap_bytes": 0, 2024-03-21T10:35:48.518 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap_items": 0, 2024-03-21T10:35:48.518 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap_mapping_bytes": 0, 2024-03-21T10:35:48.518 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap_mapping_items": 0, 2024-03-21T10:35:48.518 INFO:teuthology.orchestra.run.smithi047.stdout: "pgmap_bytes": 0, 2024-03-21T10:35:48.518 INFO:teuthology.orchestra.run.smithi047.stdout: "pgmap_items": 0, 2024-03-21T10:35:48.518 INFO:teuthology.orchestra.run.smithi047.stdout: "mds_co_bytes": 928, 2024-03-21T10:35:48.518 INFO:teuthology.orchestra.run.smithi047.stdout: "mds_co_items": 2, 2024-03-21T10:35:48.518 INFO:teuthology.orchestra.run.smithi047.stdout: "unittest_1_bytes": 0, 2024-03-21T10:35:48.518 INFO:teuthology.orchestra.run.smithi047.stdout: "unittest_1_items": 0, 2024-03-21T10:35:48.519 INFO:teuthology.orchestra.run.smithi047.stdout: "unittest_2_bytes": 0, 2024-03-21T10:35:48.519 INFO:teuthology.orchestra.run.smithi047.stdout: "unittest_2_items": 0 2024-03-21T10:35:48.519 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-03-21T10:35:48.519 INFO:teuthology.orchestra.run.smithi047.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-03-21T10:35:48.519 INFO:teuthology.orchestra.run.smithi047.stdout: "val": 0, 2024-03-21T10:35:48.519 INFO:teuthology.orchestra.run.smithi047.stdout: "max": 104857600, 2024-03-21T10:35:48.519 INFO:teuthology.orchestra.run.smithi047.stdout: "get_started": 0, 2024-03-21T10:35:48.519 INFO:teuthology.orchestra.run.smithi047.stdout: "get": 246, 2024-03-21T10:35:48.519 INFO:teuthology.orchestra.run.smithi047.stdout: "get_sum": 1690323, 2024-03-21T10:35:48.519 INFO:teuthology.orchestra.run.smithi047.stdout: "get_or_fail_fail": 0, 2024-03-21T10:35:48.519 INFO:teuthology.orchestra.run.smithi047.stdout: "get_or_fail_success": 246, 2024-03-21T10:35:48.519 INFO:teuthology.orchestra.run.smithi047.stdout: "take": 0, 2024-03-21T10:35:48.519 INFO:teuthology.orchestra.run.smithi047.stdout: "take_sum": 0, 2024-03-21T10:35:48.519 INFO:teuthology.orchestra.run.smithi047.stdout: "put": 246, 2024-03-21T10:35:48.519 INFO:teuthology.orchestra.run.smithi047.stdout: "put_sum": 1690323, 2024-03-21T10:35:48.520 INFO:teuthology.orchestra.run.smithi047.stdout: "wait": { 2024-03-21T10:35:48.520 INFO:teuthology.orchestra.run.smithi047.stdout: "avgcount": 0, 2024-03-21T10:35:48.520 INFO:teuthology.orchestra.run.smithi047.stdout: "sum": 0.000000000, 2024-03-21T10:35:48.520 INFO:teuthology.orchestra.run.smithi047.stdout: "avgtime": 0.000000000 2024-03-21T10:35:48.520 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-21T10:35:48.520 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-03-21T10:35:48.520 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-03-21T10:35:49.352 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.j 2024-03-21T10:35:49.352 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.j 2024-03-21T10:35:49.352 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.j 2024-03-21T10:35:49.352 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.j 2024-03-21T10:35:49.353 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.j 2024-03-21T10:35:49.353 DEBUG:tasks.check_counter:Getting stats from b 2024-03-21T10:35:49.353 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2edfee68ad7243bed422a049b6f5f58619300f11 shell --fsid 3056d51a-e76b-11ee-95cd-87774f69a715 -- ceph daemon mds.b perf dump 2024-03-21T10:35:49.710 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:35:49 smithi047 ceph-mon[31547]: pgmap v326: 129 pgs: 129 active+clean; 2.5 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-03-21T10:35:49.711 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:35:49 smithi047 ceph-mon[31547]: from='mgr.15687 172.21.15.47:0/8730661' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T10:35:49.726 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:35:49 smithi100 ceph-mon[35193]: pgmap v326: 129 pgs: 129 active+clean; 2.5 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-03-21T10:35:49.726 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:35:49 smithi100 ceph-mon[35193]: from='mgr.15687 172.21.15.47:0/8730661' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T10:35:49.736 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:35:49 smithi157 ceph-mon[35581]: pgmap v326: 129 pgs: 129 active+clean; 2.5 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-03-21T10:35:49.736 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:35:49 smithi157 ceph-mon[35581]: from='mgr.15687 172.21.15.47:0/8730661' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T10:35:50.237 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/3056d51a-e76b-11ee-95cd-87774f69a715/mon.b/config 2024-03-21T10:35:51.710 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:35:51 smithi047 ceph-mon[31547]: pgmap v327: 129 pgs: 129 active+clean; 2.5 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-21T10:35:51.711 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:35:51 smithi047 ceph-mon[31547]: from='mgr.15687 172.21.15.47:0/8730661' entity='mgr.x' 2024-03-21T10:35:51.711 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:35:51 smithi047 ceph-mon[31547]: from='mgr.15687 172.21.15.47:0/8730661' entity='mgr.x' 2024-03-21T10:35:51.726 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:35:51 smithi100 ceph-mon[35193]: pgmap v327: 129 pgs: 129 active+clean; 2.5 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-21T10:35:51.726 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:35:51 smithi100 ceph-mon[35193]: from='mgr.15687 172.21.15.47:0/8730661' entity='mgr.x' 2024-03-21T10:35:51.726 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:35:51 smithi100 ceph-mon[35193]: from='mgr.15687 172.21.15.47:0/8730661' entity='mgr.x' 2024-03-21T10:35:51.737 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:35:51 smithi157 ceph-mon[35581]: pgmap v327: 129 pgs: 129 active+clean; 2.5 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-21T10:35:51.737 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:35:51 smithi157 ceph-mon[35581]: from='mgr.15687 172.21.15.47:0/8730661' entity='mgr.x' 2024-03-21T10:35:51.737 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:35:51 smithi157 ceph-mon[35581]: from='mgr.15687 172.21.15.47:0/8730661' entity='mgr.x' 2024-03-21T10:35:52.627 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:35:52 smithi100 ceph-mon[35193]: from='mgr.15687 172.21.15.47:0/8730661' entity='mgr.x' 2024-03-21T10:35:52.628 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:35:52 smithi100 ceph-mon[35193]: from='mgr.15687 172.21.15.47:0/8730661' entity='mgr.x' 2024-03-21T10:35:52.628 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:35:52 smithi100 ceph-mon[35193]: from='mgr.15687 172.21.15.47:0/8730661' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-21T10:35:52.628 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:35:52 smithi100 ceph-mon[35193]: from='mgr.15687 172.21.15.47:0/8730661' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-21T10:35:52.628 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:35:52 smithi100 ceph-mon[35193]: from='mgr.15687 172.21.15.47:0/8730661' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-03-21T10:35:52.628 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:35:52 smithi100 ceph-mon[35193]: from='mgr.15687 172.21.15.47:0/8730661' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-03-21T10:35:52.884 INFO:teuthology.orchestra.run.smithi100.stdout:{ 2024-03-21T10:35:52.884 INFO:teuthology.orchestra.run.smithi100.stdout: "AsyncMessenger::Worker-0": { 2024-03-21T10:35:52.884 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_recv_messages": 2081, 2024-03-21T10:35:52.884 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_send_messages": 1358, 2024-03-21T10:35:52.884 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_recv_bytes": 882571, 2024-03-21T10:35:52.885 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_send_bytes": 2117933, 2024-03-21T10:35:52.885 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_created_connections": 18, 2024-03-21T10:35:52.885 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_active_connections": 4, 2024-03-21T10:35:52.885 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_running_total_time": 0.372970994, 2024-03-21T10:35:52.885 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_running_send_time": 0.100347043, 2024-03-21T10:35:52.885 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_running_recv_time": 0.109482525, 2024-03-21T10:35:52.885 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_running_fast_dispatch_time": 0.106502782, 2024-03-21T10:35:52.885 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_send_messages_queue_lat": { 2024-03-21T10:35:52.885 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 1357, 2024-03-21T10:35:52.885 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.496797226, 2024-03-21T10:35:52.885 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000366099 2024-03-21T10:35:52.885 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-21T10:35:52.885 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_handle_ack_lat": { 2024-03-21T10:35:52.885 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 2059, 2024-03-21T10:35:52.886 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.000482531, 2024-03-21T10:35:52.886 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000000234 2024-03-21T10:35:52.886 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-21T10:35:52.886 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_recv_encrypted_bytes": 60688, 2024-03-21T10:35:52.886 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_send_encrypted_bytes": 313968 2024-03-21T10:35:52.886 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-21T10:35:52.886 INFO:teuthology.orchestra.run.smithi100.stdout: "AsyncMessenger::Worker-1": { 2024-03-21T10:35:52.886 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_recv_messages": 2025, 2024-03-21T10:35:52.886 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_send_messages": 1261, 2024-03-21T10:35:52.886 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_recv_bytes": 2567803, 2024-03-21T10:35:52.886 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_send_bytes": 678980, 2024-03-21T10:35:52.886 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_created_connections": 22, 2024-03-21T10:35:52.886 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_active_connections": 7, 2024-03-21T10:35:52.887 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_running_total_time": 0.371126561, 2024-03-21T10:35:52.887 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_running_send_time": 0.142188486, 2024-03-21T10:35:52.887 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_running_recv_time": 0.167579555, 2024-03-21T10:35:52.887 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_running_fast_dispatch_time": 0.080805922, 2024-03-21T10:35:52.887 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_send_messages_queue_lat": { 2024-03-21T10:35:52.887 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 1260, 2024-03-21T10:35:52.887 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.762244733, 2024-03-21T10:35:52.887 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000604956 2024-03-21T10:35:52.887 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-21T10:35:52.887 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_handle_ack_lat": { 2024-03-21T10:35:52.887 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 1821, 2024-03-21T10:35:52.887 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.000453862, 2024-03-21T10:35:52.887 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000000249 2024-03-21T10:35:52.888 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-21T10:35:52.888 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_recv_encrypted_bytes": 2132400, 2024-03-21T10:35:52.888 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_send_encrypted_bytes": 254768 2024-03-21T10:35:52.888 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-21T10:35:52.888 INFO:teuthology.orchestra.run.smithi100.stdout: "AsyncMessenger::Worker-2": { 2024-03-21T10:35:52.888 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_recv_messages": 1931, 2024-03-21T10:35:52.888 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_send_messages": 1495, 2024-03-21T10:35:52.888 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_recv_bytes": 801514, 2024-03-21T10:35:52.888 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_send_bytes": 2491306, 2024-03-21T10:35:52.888 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_created_connections": 187, 2024-03-21T10:35:52.888 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_active_connections": 5, 2024-03-21T10:35:52.888 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_running_total_time": 0.474932562, 2024-03-21T10:35:52.888 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_running_send_time": 0.184082417, 2024-03-21T10:35:52.889 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_running_recv_time": 0.219338471, 2024-03-21T10:35:52.889 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_running_fast_dispatch_time": 0.062102302, 2024-03-21T10:35:52.889 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_send_messages_queue_lat": { 2024-03-21T10:35:52.889 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 1492, 2024-03-21T10:35:52.889 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.416673192, 2024-03-21T10:35:52.889 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000279271 2024-03-21T10:35:52.889 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-21T10:35:52.889 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_handle_ack_lat": { 2024-03-21T10:35:52.889 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 2470, 2024-03-21T10:35:52.889 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.000630778, 2024-03-21T10:35:52.889 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000000255 2024-03-21T10:35:52.889 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-21T10:35:52.889 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_recv_encrypted_bytes": 84608, 2024-03-21T10:35:52.889 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_send_encrypted_bytes": 157456 2024-03-21T10:35:52.889 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-21T10:35:52.890 INFO:teuthology.orchestra.run.smithi100.stdout: "cct": { 2024-03-21T10:35:52.890 INFO:teuthology.orchestra.run.smithi100.stdout: "total_workers": 1, 2024-03-21T10:35:52.890 INFO:teuthology.orchestra.run.smithi100.stdout: "unhealthy_workers": 0 2024-03-21T10:35:52.890 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-21T10:35:52.890 INFO:teuthology.orchestra.run.smithi100.stdout: "finisher-MDSRank": { 2024-03-21T10:35:52.890 INFO:teuthology.orchestra.run.smithi100.stdout: "queue_len": 0, 2024-03-21T10:35:52.890 INFO:teuthology.orchestra.run.smithi100.stdout: "complete_latency": { 2024-03-21T10:35:52.890 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 866, 2024-03-21T10:35:52.890 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.320905047, 2024-03-21T10:35:52.890 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000370560 2024-03-21T10:35:52.890 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-03-21T10:35:52.890 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-21T10:35:52.890 INFO:teuthology.orchestra.run.smithi100.stdout: "finisher-PurgeQueue": { 2024-03-21T10:35:52.891 INFO:teuthology.orchestra.run.smithi100.stdout: "queue_len": 0, 2024-03-21T10:35:52.891 INFO:teuthology.orchestra.run.smithi100.stdout: "complete_latency": { 2024-03-21T10:35:52.891 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 77, 2024-03-21T10:35:52.891 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.004188436, 2024-03-21T10:35:52.891 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000054395 2024-03-21T10:35:52.891 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-03-21T10:35:52.891 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-21T10:35:52.891 INFO:teuthology.orchestra.run.smithi100.stdout: "mds": { 2024-03-21T10:35:52.891 INFO:teuthology.orchestra.run.smithi100.stdout: "request": 169, 2024-03-21T10:35:52.891 INFO:teuthology.orchestra.run.smithi100.stdout: "reply": 169, 2024-03-21T10:35:52.891 INFO:teuthology.orchestra.run.smithi100.stdout: "reply_latency": { 2024-03-21T10:35:52.891 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 169, 2024-03-21T10:35:52.891 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.282387791, 2024-03-21T10:35:52.891 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.001670933 2024-03-21T10:35:52.892 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-21T10:35:52.892 INFO:teuthology.orchestra.run.smithi100.stdout: "slow_reply": 0, 2024-03-21T10:35:52.892 INFO:teuthology.orchestra.run.smithi100.stdout: "forward": 0, 2024-03-21T10:35:52.892 INFO:teuthology.orchestra.run.smithi100.stdout: "dir_fetch_complete": 0, 2024-03-21T10:35:52.892 INFO:teuthology.orchestra.run.smithi100.stdout: "dir_fetch_keys": 0, 2024-03-21T10:35:52.892 INFO:teuthology.orchestra.run.smithi100.stdout: "dir_commit": 78, 2024-03-21T10:35:52.892 INFO:teuthology.orchestra.run.smithi100.stdout: "dir_split": 0, 2024-03-21T10:35:52.892 INFO:teuthology.orchestra.run.smithi100.stdout: "dir_merge": 0, 2024-03-21T10:35:52.892 INFO:teuthology.orchestra.run.smithi100.stdout: "inodes": 39, 2024-03-21T10:35:52.892 INFO:teuthology.orchestra.run.smithi100.stdout: "inodes_top": 7, 2024-03-21T10:35:52.892 INFO:teuthology.orchestra.run.smithi100.stdout: "inodes_bottom": 32, 2024-03-21T10:35:52.892 INFO:teuthology.orchestra.run.smithi100.stdout: "inodes_pin_tail": 0, 2024-03-21T10:35:52.892 INFO:teuthology.orchestra.run.smithi100.stdout: "inodes_pinned": 28, 2024-03-21T10:35:52.892 INFO:teuthology.orchestra.run.smithi100.stdout: "inodes_expired": 0, 2024-03-21T10:35:52.892 INFO:teuthology.orchestra.run.smithi100.stdout: "inodes_with_caps": 3, 2024-03-21T10:35:52.893 INFO:teuthology.orchestra.run.smithi100.stdout: "caps": 3, 2024-03-21T10:35:52.893 INFO:teuthology.orchestra.run.smithi100.stdout: "subtrees": 2, 2024-03-21T10:35:52.893 INFO:teuthology.orchestra.run.smithi100.stdout: "traverse": 236, 2024-03-21T10:35:52.893 INFO:teuthology.orchestra.run.smithi100.stdout: "traverse_hit": 228, 2024-03-21T10:35:52.893 INFO:teuthology.orchestra.run.smithi100.stdout: "traverse_forward": 0, 2024-03-21T10:35:52.893 INFO:teuthology.orchestra.run.smithi100.stdout: "traverse_discover": 0, 2024-03-21T10:35:52.893 INFO:teuthology.orchestra.run.smithi100.stdout: "traverse_dir_fetch": 0, 2024-03-21T10:35:52.893 INFO:teuthology.orchestra.run.smithi100.stdout: "traverse_remote_ino": 0, 2024-03-21T10:35:52.893 INFO:teuthology.orchestra.run.smithi100.stdout: "traverse_lock": 0, 2024-03-21T10:35:52.893 INFO:teuthology.orchestra.run.smithi100.stdout: "load_cent": 0, 2024-03-21T10:35:52.893 INFO:teuthology.orchestra.run.smithi100.stdout: "q": 0, 2024-03-21T10:35:52.893 INFO:teuthology.orchestra.run.smithi100.stdout: "exported": 0, 2024-03-21T10:35:52.893 INFO:teuthology.orchestra.run.smithi100.stdout: "exported_inodes": 0, 2024-03-21T10:35:52.893 INFO:teuthology.orchestra.run.smithi100.stdout: "imported": 0, 2024-03-21T10:35:52.893 INFO:teuthology.orchestra.run.smithi100.stdout: "imported_inodes": 0, 2024-03-21T10:35:52.894 INFO:teuthology.orchestra.run.smithi100.stdout: "openino_dir_fetch": 0, 2024-03-21T10:35:52.894 INFO:teuthology.orchestra.run.smithi100.stdout: "openino_backtrace_fetch": 240, 2024-03-21T10:35:52.894 INFO:teuthology.orchestra.run.smithi100.stdout: "openino_peer_discover": 0, 2024-03-21T10:35:52.894 INFO:teuthology.orchestra.run.smithi100.stdout: "root_rfiles": 4, 2024-03-21T10:35:52.894 INFO:teuthology.orchestra.run.smithi100.stdout: "root_rbytes": 234, 2024-03-21T10:35:52.894 INFO:teuthology.orchestra.run.smithi100.stdout: "root_rsnaps": 6, 2024-03-21T10:35:52.894 INFO:teuthology.orchestra.run.smithi100.stdout: "scrub_backtrace_fetch": 240, 2024-03-21T10:35:52.894 INFO:teuthology.orchestra.run.smithi100.stdout: "scrub_set_tag": 0, 2024-03-21T10:35:52.894 INFO:teuthology.orchestra.run.smithi100.stdout: "scrub_backtrace_repaired": 0, 2024-03-21T10:35:52.894 INFO:teuthology.orchestra.run.smithi100.stdout: "scrub_inotable_repaired": 0, 2024-03-21T10:35:52.894 INFO:teuthology.orchestra.run.smithi100.stdout: "scrub_dir_inodes": 149, 2024-03-21T10:35:52.894 INFO:teuthology.orchestra.run.smithi100.stdout: "scrub_dir_base_inodes": 20, 2024-03-21T10:35:52.894 INFO:teuthology.orchestra.run.smithi100.stdout: "scrub_dirfrag_rstats": 129, 2024-03-21T10:35:52.894 INFO:teuthology.orchestra.run.smithi100.stdout: "scrub_file_inodes": 91, 2024-03-21T10:35:52.894 INFO:teuthology.orchestra.run.smithi100.stdout: "handle_inode_file_caps": 0, 2024-03-21T10:35:52.895 INFO:teuthology.orchestra.run.smithi100.stdout: "ceph_cap_op_revoke": 21, 2024-03-21T10:35:52.895 INFO:teuthology.orchestra.run.smithi100.stdout: "ceph_cap_op_grant": 31, 2024-03-21T10:35:52.895 INFO:teuthology.orchestra.run.smithi100.stdout: "ceph_cap_op_trunc": 0, 2024-03-21T10:35:52.895 INFO:teuthology.orchestra.run.smithi100.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-03-21T10:35:52.895 INFO:teuthology.orchestra.run.smithi100.stdout: "ceph_cap_op_flush_ack": 0, 2024-03-21T10:35:52.895 INFO:teuthology.orchestra.run.smithi100.stdout: "handle_client_caps": 100, 2024-03-21T10:35:52.895 INFO:teuthology.orchestra.run.smithi100.stdout: "handle_client_caps_dirty": 76, 2024-03-21T10:35:52.895 INFO:teuthology.orchestra.run.smithi100.stdout: "handle_client_cap_release": 3, 2024-03-21T10:35:52.895 INFO:teuthology.orchestra.run.smithi100.stdout: "process_request_cap_release": 35 2024-03-21T10:35:52.895 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-21T10:35:52.895 INFO:teuthology.orchestra.run.smithi100.stdout: "mds_cache": { 2024-03-21T10:35:52.895 INFO:teuthology.orchestra.run.smithi100.stdout: "dir_update": 0, 2024-03-21T10:35:52.895 INFO:teuthology.orchestra.run.smithi100.stdout: "dir_update_receipt": 0, 2024-03-21T10:35:52.895 INFO:teuthology.orchestra.run.smithi100.stdout: "dir_try_discover": 0, 2024-03-21T10:35:52.895 INFO:teuthology.orchestra.run.smithi100.stdout: "dir_send_discover": 0, 2024-03-21T10:35:52.896 INFO:teuthology.orchestra.run.smithi100.stdout: "dir_handle_discover": 9, 2024-03-21T10:35:52.896 INFO:teuthology.orchestra.run.smithi100.stdout: "num_strays": 2, 2024-03-21T10:35:52.896 INFO:teuthology.orchestra.run.smithi100.stdout: "num_strays_delayed": 0, 2024-03-21T10:35:52.896 INFO:teuthology.orchestra.run.smithi100.stdout: "num_strays_enqueuing": 0, 2024-03-21T10:35:52.896 INFO:teuthology.orchestra.run.smithi100.stdout: "strays_created": 6, 2024-03-21T10:35:52.896 INFO:teuthology.orchestra.run.smithi100.stdout: "strays_enqueued": 4, 2024-03-21T10:35:52.896 INFO:teuthology.orchestra.run.smithi100.stdout: "strays_reintegrated": 0, 2024-03-21T10:35:52.896 INFO:teuthology.orchestra.run.smithi100.stdout: "strays_migrated": 0, 2024-03-21T10:35:52.896 INFO:teuthology.orchestra.run.smithi100.stdout: "num_recovering_processing": 0, 2024-03-21T10:35:52.896 INFO:teuthology.orchestra.run.smithi100.stdout: "num_recovering_enqueued": 0, 2024-03-21T10:35:52.896 INFO:teuthology.orchestra.run.smithi100.stdout: "num_recovering_prioritized": 0, 2024-03-21T10:35:52.896 INFO:teuthology.orchestra.run.smithi100.stdout: "recovery_started": 0, 2024-03-21T10:35:52.896 INFO:teuthology.orchestra.run.smithi100.stdout: "recovery_completed": 0, 2024-03-21T10:35:52.896 INFO:teuthology.orchestra.run.smithi100.stdout: "ireq_enqueue_scrub": 20, 2024-03-21T10:35:52.897 INFO:teuthology.orchestra.run.smithi100.stdout: "ireq_exportdir": 0, 2024-03-21T10:35:52.897 INFO:teuthology.orchestra.run.smithi100.stdout: "ireq_flush": 0, 2024-03-21T10:35:52.897 INFO:teuthology.orchestra.run.smithi100.stdout: "ireq_fragmentdir": 0, 2024-03-21T10:35:52.897 INFO:teuthology.orchestra.run.smithi100.stdout: "ireq_fragstats": 0, 2024-03-21T10:35:52.897 INFO:teuthology.orchestra.run.smithi100.stdout: "ireq_inodestats": 0 2024-03-21T10:35:52.897 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-21T10:35:52.897 INFO:teuthology.orchestra.run.smithi100.stdout: "mds_log": { 2024-03-21T10:35:52.897 INFO:teuthology.orchestra.run.smithi100.stdout: "evlrg": 0, 2024-03-21T10:35:52.897 INFO:teuthology.orchestra.run.smithi100.stdout: "evadd": 250, 2024-03-21T10:35:52.897 INFO:teuthology.orchestra.run.smithi100.stdout: "evex": 169, 2024-03-21T10:35:52.897 INFO:teuthology.orchestra.run.smithi100.stdout: "evtrm": 169, 2024-03-21T10:35:52.897 INFO:teuthology.orchestra.run.smithi100.stdout: "ev": 81, 2024-03-21T10:35:52.897 INFO:teuthology.orchestra.run.smithi100.stdout: "evexg": 0, 2024-03-21T10:35:52.897 INFO:teuthology.orchestra.run.smithi100.stdout: "evexd": 20, 2024-03-21T10:35:52.897 INFO:teuthology.orchestra.run.smithi100.stdout: "segadd": 16, 2024-03-21T10:35:52.898 INFO:teuthology.orchestra.run.smithi100.stdout: "segex": 15, 2024-03-21T10:35:52.898 INFO:teuthology.orchestra.run.smithi100.stdout: "segtrm": 15, 2024-03-21T10:35:52.898 INFO:teuthology.orchestra.run.smithi100.stdout: "seg": 1, 2024-03-21T10:35:52.898 INFO:teuthology.orchestra.run.smithi100.stdout: "segmjr": 1, 2024-03-21T10:35:52.898 INFO:teuthology.orchestra.run.smithi100.stdout: "segexg": 0, 2024-03-21T10:35:52.898 INFO:teuthology.orchestra.run.smithi100.stdout: "segexd": 1, 2024-03-21T10:35:52.898 INFO:teuthology.orchestra.run.smithi100.stdout: "expos": 5259645, 2024-03-21T10:35:52.898 INFO:teuthology.orchestra.run.smithi100.stdout: "wrpos": 5518046, 2024-03-21T10:35:52.898 INFO:teuthology.orchestra.run.smithi100.stdout: "rdpos": 0, 2024-03-21T10:35:52.898 INFO:teuthology.orchestra.run.smithi100.stdout: "jlat": { 2024-03-21T10:35:52.898 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 205, 2024-03-21T10:35:52.898 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 1.810331023, 2024-03-21T10:35:52.898 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.008830883 2024-03-21T10:35:52.898 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-21T10:35:52.898 INFO:teuthology.orchestra.run.smithi100.stdout: "replayed": 0 2024-03-21T10:35:52.899 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-21T10:35:52.899 INFO:teuthology.orchestra.run.smithi100.stdout: "mds_mem": { 2024-03-21T10:35:52.899 INFO:teuthology.orchestra.run.smithi100.stdout: "ino": 30, 2024-03-21T10:35:52.899 INFO:teuthology.orchestra.run.smithi100.stdout: "ino+": 54, 2024-03-21T10:35:52.899 INFO:teuthology.orchestra.run.smithi100.stdout: "ino-": 24, 2024-03-21T10:35:52.899 INFO:teuthology.orchestra.run.smithi100.stdout: "dir": 20, 2024-03-21T10:35:52.899 INFO:teuthology.orchestra.run.smithi100.stdout: "dir+": 20, 2024-03-21T10:35:52.899 INFO:teuthology.orchestra.run.smithi100.stdout: "dir-": 0, 2024-03-21T10:35:52.899 INFO:teuthology.orchestra.run.smithi100.stdout: "dn": 39, 2024-03-21T10:35:52.899 INFO:teuthology.orchestra.run.smithi100.stdout: "dn+": 43, 2024-03-21T10:35:52.899 INFO:teuthology.orchestra.run.smithi100.stdout: "dn-": 4, 2024-03-21T10:35:52.899 INFO:teuthology.orchestra.run.smithi100.stdout: "cap": 3, 2024-03-21T10:35:52.899 INFO:teuthology.orchestra.run.smithi100.stdout: "cap+": 21, 2024-03-21T10:35:52.900 INFO:teuthology.orchestra.run.smithi100.stdout: "cap-": 18, 2024-03-21T10:35:52.900 INFO:teuthology.orchestra.run.smithi100.stdout: "rss": 55876, 2024-03-21T10:35:52.900 INFO:teuthology.orchestra.run.smithi100.stdout: "heap": 231804 2024-03-21T10:35:52.900 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-21T10:35:52.900 INFO:teuthology.orchestra.run.smithi100.stdout: "mds_server": { 2024-03-21T10:35:52.900 INFO:teuthology.orchestra.run.smithi100.stdout: "dispatch_client_request": 218, 2024-03-21T10:35:52.900 INFO:teuthology.orchestra.run.smithi100.stdout: "dispatch_server_request": 1, 2024-03-21T10:35:52.900 INFO:teuthology.orchestra.run.smithi100.stdout: "handle_client_request": 169, 2024-03-21T10:35:52.900 INFO:teuthology.orchestra.run.smithi100.stdout: "handle_client_session": 36, 2024-03-21T10:35:52.900 INFO:teuthology.orchestra.run.smithi100.stdout: "handle_peer_request": 2, 2024-03-21T10:35:52.900 INFO:teuthology.orchestra.run.smithi100.stdout: "req_create_latency": { 2024-03-21T10:35:52.900 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 8, 2024-03-21T10:35:52.900 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.013481841, 2024-03-21T10:35:52.900 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.001685230 2024-03-21T10:35:52.900 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-21T10:35:52.901 INFO:teuthology.orchestra.run.smithi100.stdout: "req_getattr_latency": { 2024-03-21T10:35:52.901 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 109, 2024-03-21T10:35:52.901 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.038942936, 2024-03-21T10:35:52.901 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000357274 2024-03-21T10:35:52.901 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-21T10:35:52.901 INFO:teuthology.orchestra.run.smithi100.stdout: "req_getfilelock_latency": { 2024-03-21T10:35:52.901 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 0, 2024-03-21T10:35:52.901 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.000000000, 2024-03-21T10:35:52.901 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000000000 2024-03-21T10:35:52.901 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-21T10:35:52.901 INFO:teuthology.orchestra.run.smithi100.stdout: "req_link_latency": { 2024-03-21T10:35:52.901 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 0, 2024-03-21T10:35:52.901 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.000000000, 2024-03-21T10:35:52.901 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000000000 2024-03-21T10:35:52.901 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-21T10:35:52.902 INFO:teuthology.orchestra.run.smithi100.stdout: "req_lookup_latency": { 2024-03-21T10:35:52.902 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 0, 2024-03-21T10:35:52.902 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.000000000, 2024-03-21T10:35:52.902 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000000000 2024-03-21T10:35:52.902 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-21T10:35:52.902 INFO:teuthology.orchestra.run.smithi100.stdout: "req_lookuphash_latency": { 2024-03-21T10:35:52.902 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 0, 2024-03-21T10:35:52.902 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.000000000, 2024-03-21T10:35:52.902 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000000000 2024-03-21T10:35:52.902 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-21T10:35:52.902 INFO:teuthology.orchestra.run.smithi100.stdout: "req_lookupino_latency": { 2024-03-21T10:35:52.902 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 0, 2024-03-21T10:35:52.902 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.000000000, 2024-03-21T10:35:52.902 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000000000 2024-03-21T10:35:52.903 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-21T10:35:52.903 INFO:teuthology.orchestra.run.smithi100.stdout: "req_lookupname_latency": { 2024-03-21T10:35:52.903 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 0, 2024-03-21T10:35:52.903 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.000000000, 2024-03-21T10:35:52.903 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000000000 2024-03-21T10:35:52.903 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-21T10:35:52.903 INFO:teuthology.orchestra.run.smithi100.stdout: "req_lookupparent_latency": { 2024-03-21T10:35:52.903 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 0, 2024-03-21T10:35:52.903 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.000000000, 2024-03-21T10:35:52.903 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000000000 2024-03-21T10:35:52.903 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-21T10:35:52.903 INFO:teuthology.orchestra.run.smithi100.stdout: "req_lookupsnap_latency": { 2024-03-21T10:35:52.903 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 0, 2024-03-21T10:35:52.903 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.000000000, 2024-03-21T10:35:52.903 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000000000 2024-03-21T10:35:52.904 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-21T10:35:52.904 INFO:teuthology.orchestra.run.smithi100.stdout: "req_lssnap_latency": { 2024-03-21T10:35:52.904 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 10, 2024-03-21T10:35:52.904 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.003022045, 2024-03-21T10:35:52.904 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000302204 2024-03-21T10:35:52.904 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-21T10:35:52.904 INFO:teuthology.orchestra.run.smithi100.stdout: "req_mkdir_latency": { 2024-03-21T10:35:52.904 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 8, 2024-03-21T10:35:52.904 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.025835465, 2024-03-21T10:35:52.904 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.003229433 2024-03-21T10:35:52.904 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-21T10:35:52.904 INFO:teuthology.orchestra.run.smithi100.stdout: "req_mknod_latency": { 2024-03-21T10:35:52.904 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 0, 2024-03-21T10:35:52.904 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.000000000, 2024-03-21T10:35:52.905 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000000000 2024-03-21T10:35:52.905 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-21T10:35:52.905 INFO:teuthology.orchestra.run.smithi100.stdout: "req_mksnap_latency": { 2024-03-21T10:35:52.905 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 10, 2024-03-21T10:35:52.905 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.099137352, 2024-03-21T10:35:52.905 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.009913735 2024-03-21T10:35:52.905 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-21T10:35:52.905 INFO:teuthology.orchestra.run.smithi100.stdout: "req_open_latency": { 2024-03-21T10:35:52.905 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 0, 2024-03-21T10:35:52.905 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.000000000, 2024-03-21T10:35:52.905 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000000000 2024-03-21T10:35:52.905 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-21T10:35:52.905 INFO:teuthology.orchestra.run.smithi100.stdout: "req_readdir_latency": { 2024-03-21T10:35:52.905 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 2, 2024-03-21T10:35:52.905 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.006060090, 2024-03-21T10:35:52.906 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.003030045 2024-03-21T10:35:52.906 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-21T10:35:52.906 INFO:teuthology.orchestra.run.smithi100.stdout: "req_rename_latency": { 2024-03-21T10:35:52.906 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 2, 2024-03-21T10:35:52.906 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.003667314, 2024-03-21T10:35:52.906 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.001833657 2024-03-21T10:35:52.906 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-21T10:35:52.906 INFO:teuthology.orchestra.run.smithi100.stdout: "req_renamesnap_latency": { 2024-03-21T10:35:52.906 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 0, 2024-03-21T10:35:52.906 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.000000000, 2024-03-21T10:35:52.906 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000000000 2024-03-21T10:35:52.906 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-21T10:35:52.906 INFO:teuthology.orchestra.run.smithi100.stdout: "req_snapdiff_latency": { 2024-03-21T10:35:52.906 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 0, 2024-03-21T10:35:52.907 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.000000000, 2024-03-21T10:35:52.907 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000000000 2024-03-21T10:35:52.907 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-21T10:35:52.907 INFO:teuthology.orchestra.run.smithi100.stdout: "req_rmdir_latency": { 2024-03-21T10:35:52.907 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 2, 2024-03-21T10:35:52.907 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.018091611, 2024-03-21T10:35:52.907 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.009045805 2024-03-21T10:35:52.907 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-21T10:35:52.907 INFO:teuthology.orchestra.run.smithi100.stdout: "req_rmsnap_latency": { 2024-03-21T10:35:52.907 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 4, 2024-03-21T10:35:52.907 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.030919030, 2024-03-21T10:35:52.907 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.007729757 2024-03-21T10:35:52.908 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-21T10:35:52.908 INFO:teuthology.orchestra.run.smithi100.stdout: "req_rmxattr_latency": { 2024-03-21T10:35:52.908 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 0, 2024-03-21T10:35:52.908 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.000000000, 2024-03-21T10:35:52.908 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000000000 2024-03-21T10:35:52.908 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-21T10:35:52.908 INFO:teuthology.orchestra.run.smithi100.stdout: "req_setattr_latency": { 2024-03-21T10:35:52.908 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 2, 2024-03-21T10:35:52.908 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.002792102, 2024-03-21T10:35:52.908 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.001396051 2024-03-21T10:35:52.908 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-21T10:35:52.908 INFO:teuthology.orchestra.run.smithi100.stdout: "req_setdirlayout_latency": { 2024-03-21T10:35:52.908 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 0, 2024-03-21T10:35:52.908 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.000000000, 2024-03-21T10:35:52.909 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000000000 2024-03-21T10:35:52.909 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-21T10:35:52.909 INFO:teuthology.orchestra.run.smithi100.stdout: "req_setfilelock_latency": { 2024-03-21T10:35:52.909 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 0, 2024-03-21T10:35:52.909 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.000000000, 2024-03-21T10:35:52.909 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000000000 2024-03-21T10:35:52.909 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-21T10:35:52.909 INFO:teuthology.orchestra.run.smithi100.stdout: "req_setlayout_latency": { 2024-03-21T10:35:52.909 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 0, 2024-03-21T10:35:52.909 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.000000000, 2024-03-21T10:35:52.909 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000000000 2024-03-21T10:35:52.909 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-21T10:35:52.909 INFO:teuthology.orchestra.run.smithi100.stdout: "req_setxattr_latency": { 2024-03-21T10:35:52.909 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 8, 2024-03-21T10:35:52.909 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.034239880, 2024-03-21T10:35:52.910 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.004279985 2024-03-21T10:35:52.910 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-21T10:35:52.910 INFO:teuthology.orchestra.run.smithi100.stdout: "req_symlink_latency": { 2024-03-21T10:35:52.910 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 0, 2024-03-21T10:35:52.910 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.000000000, 2024-03-21T10:35:52.910 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000000000 2024-03-21T10:35:52.910 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-21T10:35:52.910 INFO:teuthology.orchestra.run.smithi100.stdout: "req_unlink_latency": { 2024-03-21T10:35:52.910 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 4, 2024-03-21T10:35:52.910 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.006198125, 2024-03-21T10:35:52.910 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.001549531 2024-03-21T10:35:52.910 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-21T10:35:52.910 INFO:teuthology.orchestra.run.smithi100.stdout: "cap_revoke_eviction": 0, 2024-03-21T10:35:52.911 INFO:teuthology.orchestra.run.smithi100.stdout: "cap_acquisition_throttle": 0, 2024-03-21T10:35:52.911 INFO:teuthology.orchestra.run.smithi100.stdout: "req_getvxattr_latency": { 2024-03-21T10:35:52.911 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 0, 2024-03-21T10:35:52.911 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.000000000, 2024-03-21T10:35:52.911 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000000000 2024-03-21T10:35:52.911 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-03-21T10:35:52.911 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-21T10:35:52.911 INFO:teuthology.orchestra.run.smithi100.stdout: "mds_sessions": { 2024-03-21T10:35:52.911 INFO:teuthology.orchestra.run.smithi100.stdout: "session_count": 2, 2024-03-21T10:35:52.911 INFO:teuthology.orchestra.run.smithi100.stdout: "session_add": 6, 2024-03-21T10:35:52.911 INFO:teuthology.orchestra.run.smithi100.stdout: "session_remove": 4, 2024-03-21T10:35:52.911 INFO:teuthology.orchestra.run.smithi100.stdout: "sessions_open": 2, 2024-03-21T10:35:52.911 INFO:teuthology.orchestra.run.smithi100.stdout: "sessions_stale": 0, 2024-03-21T10:35:52.911 INFO:teuthology.orchestra.run.smithi100.stdout: "total_load": 27, 2024-03-21T10:35:52.912 INFO:teuthology.orchestra.run.smithi100.stdout: "average_load": 13, 2024-03-21T10:35:52.912 INFO:teuthology.orchestra.run.smithi100.stdout: "avg_session_uptime": 620, 2024-03-21T10:35:52.912 INFO:teuthology.orchestra.run.smithi100.stdout: "mdthresh_evicted": 0 2024-03-21T10:35:52.912 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-21T10:35:52.912 INFO:teuthology.orchestra.run.smithi100.stdout: "mempool": { 2024-03-21T10:35:52.912 INFO:teuthology.orchestra.run.smithi100.stdout: "bloom_filter_bytes": 0, 2024-03-21T10:35:52.912 INFO:teuthology.orchestra.run.smithi100.stdout: "bloom_filter_items": 0, 2024-03-21T10:35:52.912 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_alloc_bytes": 0, 2024-03-21T10:35:52.912 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_alloc_items": 0, 2024-03-21T10:35:52.912 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_cache_data_bytes": 0, 2024-03-21T10:35:52.912 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_cache_data_items": 0, 2024-03-21T10:35:52.912 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_cache_onode_bytes": 0, 2024-03-21T10:35:52.913 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_cache_onode_items": 0, 2024-03-21T10:35:52.913 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_cache_meta_bytes": 0, 2024-03-21T10:35:52.913 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_cache_meta_items": 0, 2024-03-21T10:35:52.913 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_cache_other_bytes": 0, 2024-03-21T10:35:52.913 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_cache_other_items": 0, 2024-03-21T10:35:52.913 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_cache_buffer_bytes": 0, 2024-03-21T10:35:52.913 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_cache_buffer_items": 0, 2024-03-21T10:35:52.913 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_extent_bytes": 0, 2024-03-21T10:35:52.913 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_extent_items": 0, 2024-03-21T10:35:52.913 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_blob_bytes": 0, 2024-03-21T10:35:52.913 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_blob_items": 0, 2024-03-21T10:35:52.913 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_shared_blob_bytes": 0, 2024-03-21T10:35:52.914 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_shared_blob_items": 0, 2024-03-21T10:35:52.914 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_inline_bl_bytes": 0, 2024-03-21T10:35:52.914 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_inline_bl_items": 0, 2024-03-21T10:35:52.914 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_fsck_bytes": 0, 2024-03-21T10:35:52.914 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_fsck_items": 0, 2024-03-21T10:35:52.914 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_txc_bytes": 0, 2024-03-21T10:35:52.914 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_txc_items": 0, 2024-03-21T10:35:52.914 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_writing_deferred_bytes": 0, 2024-03-21T10:35:52.914 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_writing_deferred_items": 0, 2024-03-21T10:35:52.914 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_writing_bytes": 0, 2024-03-21T10:35:52.914 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_writing_items": 0, 2024-03-21T10:35:52.914 INFO:teuthology.orchestra.run.smithi100.stdout: "bluefs_bytes": 0, 2024-03-21T10:35:52.914 INFO:teuthology.orchestra.run.smithi100.stdout: "bluefs_items": 0, 2024-03-21T10:35:52.915 INFO:teuthology.orchestra.run.smithi100.stdout: "bluefs_file_reader_bytes": 0, 2024-03-21T10:35:52.915 INFO:teuthology.orchestra.run.smithi100.stdout: "bluefs_file_reader_items": 0, 2024-03-21T10:35:52.915 INFO:teuthology.orchestra.run.smithi100.stdout: "bluefs_file_writer_bytes": 0, 2024-03-21T10:35:52.915 INFO:teuthology.orchestra.run.smithi100.stdout: "bluefs_file_writer_items": 0, 2024-03-21T10:35:52.915 INFO:teuthology.orchestra.run.smithi100.stdout: "buffer_anon_bytes": 376169, 2024-03-21T10:35:52.915 INFO:teuthology.orchestra.run.smithi100.stdout: "buffer_anon_items": 147, 2024-03-21T10:35:52.915 INFO:teuthology.orchestra.run.smithi100.stdout: "buffer_meta_bytes": 352, 2024-03-21T10:35:52.915 INFO:teuthology.orchestra.run.smithi100.stdout: "buffer_meta_items": 4, 2024-03-21T10:35:52.915 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_bytes": 0, 2024-03-21T10:35:52.915 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_items": 0, 2024-03-21T10:35:52.915 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_mapbl_bytes": 0, 2024-03-21T10:35:52.915 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_mapbl_items": 0, 2024-03-21T10:35:52.915 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_pglog_bytes": 0, 2024-03-21T10:35:52.916 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_pglog_items": 0, 2024-03-21T10:35:52.916 INFO:teuthology.orchestra.run.smithi100.stdout: "osdmap_bytes": 9712, 2024-03-21T10:35:52.916 INFO:teuthology.orchestra.run.smithi100.stdout: "osdmap_items": 187, 2024-03-21T10:35:52.916 INFO:teuthology.orchestra.run.smithi100.stdout: "osdmap_mapping_bytes": 0, 2024-03-21T10:35:52.916 INFO:teuthology.orchestra.run.smithi100.stdout: "osdmap_mapping_items": 0, 2024-03-21T10:35:52.916 INFO:teuthology.orchestra.run.smithi100.stdout: "pgmap_bytes": 0, 2024-03-21T10:35:52.916 INFO:teuthology.orchestra.run.smithi100.stdout: "pgmap_items": 0, 2024-03-21T10:35:52.916 INFO:teuthology.orchestra.run.smithi100.stdout: "mds_co_bytes": 137490, 2024-03-21T10:35:52.916 INFO:teuthology.orchestra.run.smithi100.stdout: "mds_co_items": 1059, 2024-03-21T10:35:52.916 INFO:teuthology.orchestra.run.smithi100.stdout: "unittest_1_bytes": 0, 2024-03-21T10:35:52.916 INFO:teuthology.orchestra.run.smithi100.stdout: "unittest_1_items": 0, 2024-03-21T10:35:52.916 INFO:teuthology.orchestra.run.smithi100.stdout: "unittest_2_bytes": 0, 2024-03-21T10:35:52.916 INFO:teuthology.orchestra.run.smithi100.stdout: "unittest_2_items": 0 2024-03-21T10:35:52.916 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-21T10:35:52.917 INFO:teuthology.orchestra.run.smithi100.stdout: "objecter": { 2024-03-21T10:35:52.917 INFO:teuthology.orchestra.run.smithi100.stdout: "op_active": 0, 2024-03-21T10:35:52.917 INFO:teuthology.orchestra.run.smithi100.stdout: "op_laggy": 0, 2024-03-21T10:35:52.917 INFO:teuthology.orchestra.run.smithi100.stdout: "op_send": 1297, 2024-03-21T10:35:52.917 INFO:teuthology.orchestra.run.smithi100.stdout: "op_send_bytes": 1480652, 2024-03-21T10:35:52.917 INFO:teuthology.orchestra.run.smithi100.stdout: "op_resend": 37, 2024-03-21T10:35:52.917 INFO:teuthology.orchestra.run.smithi100.stdout: "op_reply": 1297, 2024-03-21T10:35:52.917 INFO:teuthology.orchestra.run.smithi100.stdout: "op_latency": { 2024-03-21T10:35:52.917 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 1297, 2024-03-21T10:35:52.917 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 37.510020582, 2024-03-21T10:35:52.917 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.028920601 2024-03-21T10:35:52.917 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-21T10:35:52.917 INFO:teuthology.orchestra.run.smithi100.stdout: "op_inflight": 0, 2024-03-21T10:35:52.917 INFO:teuthology.orchestra.run.smithi100.stdout: "oplen_avg": { 2024-03-21T10:35:52.918 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 1297, 2024-03-21T10:35:52.918 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 1514 2024-03-21T10:35:52.918 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-21T10:35:52.918 INFO:teuthology.orchestra.run.smithi100.stdout: "op": 1297, 2024-03-21T10:35:52.918 INFO:teuthology.orchestra.run.smithi100.stdout: "op_r": 281, 2024-03-21T10:35:52.918 INFO:teuthology.orchestra.run.smithi100.stdout: "op_w": 1016, 2024-03-21T10:35:52.918 INFO:teuthology.orchestra.run.smithi100.stdout: "op_rmw": 0, 2024-03-21T10:35:52.918 INFO:teuthology.orchestra.run.smithi100.stdout: "op_pg": 0, 2024-03-21T10:35:52.918 INFO:teuthology.orchestra.run.smithi100.stdout: "osdop_stat": 58, 2024-03-21T10:35:52.918 INFO:teuthology.orchestra.run.smithi100.stdout: "osdop_create": 36, 2024-03-21T10:35:52.918 INFO:teuthology.orchestra.run.smithi100.stdout: "osdop_read": 21, 2024-03-21T10:35:52.918 INFO:teuthology.orchestra.run.smithi100.stdout: "osdop_write": 206, 2024-03-21T10:35:52.918 INFO:teuthology.orchestra.run.smithi100.stdout: "osdop_writefull": 68, 2024-03-21T10:35:52.918 INFO:teuthology.orchestra.run.smithi100.stdout: "osdop_writesame": 0, 2024-03-21T10:35:52.919 INFO:teuthology.orchestra.run.smithi100.stdout: "osdop_append": 0, 2024-03-21T10:35:52.919 INFO:teuthology.orchestra.run.smithi100.stdout: "osdop_zero": 0, 2024-03-21T10:35:52.919 INFO:teuthology.orchestra.run.smithi100.stdout: "osdop_truncate": 0, 2024-03-21T10:35:52.919 INFO:teuthology.orchestra.run.smithi100.stdout: "osdop_delete": 583, 2024-03-21T10:35:52.919 INFO:teuthology.orchestra.run.smithi100.stdout: "osdop_mapext": 0, 2024-03-21T10:35:52.919 INFO:teuthology.orchestra.run.smithi100.stdout: "osdop_sparse_read": 0, 2024-03-21T10:35:52.919 INFO:teuthology.orchestra.run.smithi100.stdout: "osdop_clonerange": 0, 2024-03-21T10:35:52.919 INFO:teuthology.orchestra.run.smithi100.stdout: "osdop_getxattr": 260, 2024-03-21T10:35:52.919 INFO:teuthology.orchestra.run.smithi100.stdout: "osdop_setxattr": 64, 2024-03-21T10:35:52.919 INFO:teuthology.orchestra.run.smithi100.stdout: "osdop_cmpxattr": 0, 2024-03-21T10:35:52.919 INFO:teuthology.orchestra.run.smithi100.stdout: "osdop_rmxattr": 0, 2024-03-21T10:35:52.919 INFO:teuthology.orchestra.run.smithi100.stdout: "osdop_resetxattrs": 0, 2024-03-21T10:35:52.919 INFO:teuthology.orchestra.run.smithi100.stdout: "osdop_call": 0, 2024-03-21T10:35:52.919 INFO:teuthology.orchestra.run.smithi100.stdout: "osdop_watch": 0, 2024-03-21T10:35:52.919 INFO:teuthology.orchestra.run.smithi100.stdout: "osdop_notify": 0, 2024-03-21T10:35:52.920 INFO:teuthology.orchestra.run.smithi100.stdout: "osdop_src_cmpxattr": 0, 2024-03-21T10:35:52.920 INFO:teuthology.orchestra.run.smithi100.stdout: "osdop_pgls": 0, 2024-03-21T10:35:52.920 INFO:teuthology.orchestra.run.smithi100.stdout: "osdop_pgls_filter": 0, 2024-03-21T10:35:52.920 INFO:teuthology.orchestra.run.smithi100.stdout: "osdop_other": 0, 2024-03-21T10:35:52.920 INFO:teuthology.orchestra.run.smithi100.stdout: "linger_active": 0, 2024-03-21T10:35:52.920 INFO:teuthology.orchestra.run.smithi100.stdout: "linger_send": 0, 2024-03-21T10:35:52.920 INFO:teuthology.orchestra.run.smithi100.stdout: "linger_resend": 0, 2024-03-21T10:35:52.920 INFO:teuthology.orchestra.run.smithi100.stdout: "linger_ping": 0, 2024-03-21T10:35:52.920 INFO:teuthology.orchestra.run.smithi100.stdout: "poolop_active": 0, 2024-03-21T10:35:52.920 INFO:teuthology.orchestra.run.smithi100.stdout: "poolop_send": 0, 2024-03-21T10:35:52.920 INFO:teuthology.orchestra.run.smithi100.stdout: "poolop_resend": 0, 2024-03-21T10:35:52.920 INFO:teuthology.orchestra.run.smithi100.stdout: "poolstat_active": 0, 2024-03-21T10:35:52.920 INFO:teuthology.orchestra.run.smithi100.stdout: "poolstat_send": 0, 2024-03-21T10:35:52.920 INFO:teuthology.orchestra.run.smithi100.stdout: "poolstat_resend": 0, 2024-03-21T10:35:52.920 INFO:teuthology.orchestra.run.smithi100.stdout: "statfs_active": 0, 2024-03-21T10:35:52.920 INFO:teuthology.orchestra.run.smithi100.stdout: "statfs_send": 0, 2024-03-21T10:35:52.921 INFO:teuthology.orchestra.run.smithi100.stdout: "statfs_resend": 0, 2024-03-21T10:35:52.921 INFO:teuthology.orchestra.run.smithi100.stdout: "command_active": 0, 2024-03-21T10:35:52.921 INFO:teuthology.orchestra.run.smithi100.stdout: "command_send": 0, 2024-03-21T10:35:52.921 INFO:teuthology.orchestra.run.smithi100.stdout: "command_resend": 0, 2024-03-21T10:35:52.921 INFO:teuthology.orchestra.run.smithi100.stdout: "map_epoch": 105, 2024-03-21T10:35:52.921 INFO:teuthology.orchestra.run.smithi100.stdout: "map_full": 0, 2024-03-21T10:35:52.921 INFO:teuthology.orchestra.run.smithi100.stdout: "map_inc": 19, 2024-03-21T10:35:52.921 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_sessions": 12, 2024-03-21T10:35:52.921 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_session_open": 12, 2024-03-21T10:35:52.921 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_session_close": 0, 2024-03-21T10:35:52.921 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_laggy": 0, 2024-03-21T10:35:52.921 INFO:teuthology.orchestra.run.smithi100.stdout: "omap_wr": 205, 2024-03-21T10:35:52.921 INFO:teuthology.orchestra.run.smithi100.stdout: "omap_rd": 0, 2024-03-21T10:35:52.921 INFO:teuthology.orchestra.run.smithi100.stdout: "omap_del": 13 2024-03-21T10:35:52.921 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-21T10:35:52.921 INFO:teuthology.orchestra.run.smithi100.stdout: "oft": { 2024-03-21T10:35:52.922 INFO:teuthology.orchestra.run.smithi100.stdout: "omap_total_objs": 1, 2024-03-21T10:35:52.922 INFO:teuthology.orchestra.run.smithi100.stdout: "omap_total_kv_pairs": 0, 2024-03-21T10:35:52.922 INFO:teuthology.orchestra.run.smithi100.stdout: "omap_total_updates": 0, 2024-03-21T10:35:52.922 INFO:teuthology.orchestra.run.smithi100.stdout: "omap_total_removes": 0 2024-03-21T10:35:52.922 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-21T10:35:52.922 INFO:teuthology.orchestra.run.smithi100.stdout: "osd-slow-ops": { 2024-03-21T10:35:52.922 INFO:teuthology.orchestra.run.smithi100.stdout: "slow_ops_count": 0 2024-03-21T10:35:52.922 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-21T10:35:52.922 INFO:teuthology.orchestra.run.smithi100.stdout: "purge_queue": { 2024-03-21T10:35:52.922 INFO:teuthology.orchestra.run.smithi100.stdout: "pq_executing_ops": 0, 2024-03-21T10:35:52.922 INFO:teuthology.orchestra.run.smithi100.stdout: "pq_executing_ops_high_water": 69, 2024-03-21T10:35:52.922 INFO:teuthology.orchestra.run.smithi100.stdout: "pq_executing": 0, 2024-03-21T10:35:52.922 INFO:teuthology.orchestra.run.smithi100.stdout: "pq_executing_high_water": 3, 2024-03-21T10:35:52.922 INFO:teuthology.orchestra.run.smithi100.stdout: "pq_executed_ops": 71, 2024-03-21T10:35:52.923 INFO:teuthology.orchestra.run.smithi100.stdout: "pq_executed": 4, 2024-03-21T10:35:52.923 INFO:teuthology.orchestra.run.smithi100.stdout: "pq_item_in_journal": 0 2024-03-21T10:35:52.923 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-21T10:35:52.923 INFO:teuthology.orchestra.run.smithi100.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-03-21T10:35:52.923 INFO:teuthology.orchestra.run.smithi100.stdout: "val": 0, 2024-03-21T10:35:52.923 INFO:teuthology.orchestra.run.smithi100.stdout: "max": 104857600, 2024-03-21T10:35:52.923 INFO:teuthology.orchestra.run.smithi100.stdout: "get_started": 0, 2024-03-21T10:35:52.923 INFO:teuthology.orchestra.run.smithi100.stdout: "get": 6033, 2024-03-21T10:35:52.923 INFO:teuthology.orchestra.run.smithi100.stdout: "get_sum": 3641267, 2024-03-21T10:35:52.923 INFO:teuthology.orchestra.run.smithi100.stdout: "get_or_fail_fail": 0, 2024-03-21T10:35:52.923 INFO:teuthology.orchestra.run.smithi100.stdout: "get_or_fail_success": 6033, 2024-03-21T10:35:52.923 INFO:teuthology.orchestra.run.smithi100.stdout: "take": 0, 2024-03-21T10:35:52.923 INFO:teuthology.orchestra.run.smithi100.stdout: "take_sum": 0, 2024-03-21T10:35:52.923 INFO:teuthology.orchestra.run.smithi100.stdout: "put": 6033, 2024-03-21T10:35:52.923 INFO:teuthology.orchestra.run.smithi100.stdout: "put_sum": 3641267, 2024-03-21T10:35:52.924 INFO:teuthology.orchestra.run.smithi100.stdout: "wait": { 2024-03-21T10:35:52.924 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 0, 2024-03-21T10:35:52.924 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.000000000, 2024-03-21T10:35:52.924 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000000000 2024-03-21T10:35:52.924 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-03-21T10:35:52.924 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-21T10:35:52.924 INFO:teuthology.orchestra.run.smithi100.stdout: "throttle-objecter_bytes": { 2024-03-21T10:35:52.924 INFO:teuthology.orchestra.run.smithi100.stdout: "val": 0, 2024-03-21T10:35:52.924 INFO:teuthology.orchestra.run.smithi100.stdout: "max": 104857600, 2024-03-21T10:35:52.924 INFO:teuthology.orchestra.run.smithi100.stdout: "get_started": 0, 2024-03-21T10:35:52.924 INFO:teuthology.orchestra.run.smithi100.stdout: "get": 0, 2024-03-21T10:35:52.924 INFO:teuthology.orchestra.run.smithi100.stdout: "get_sum": 0, 2024-03-21T10:35:52.924 INFO:teuthology.orchestra.run.smithi100.stdout: "get_or_fail_fail": 0, 2024-03-21T10:35:52.924 INFO:teuthology.orchestra.run.smithi100.stdout: "get_or_fail_success": 0, 2024-03-21T10:35:52.924 INFO:teuthology.orchestra.run.smithi100.stdout: "take": 1297, 2024-03-21T10:35:52.925 INFO:teuthology.orchestra.run.smithi100.stdout: "take_sum": 1481144, 2024-03-21T10:35:52.925 INFO:teuthology.orchestra.run.smithi100.stdout: "put": 1297, 2024-03-21T10:35:52.925 INFO:teuthology.orchestra.run.smithi100.stdout: "put_sum": 1481144, 2024-03-21T10:35:52.925 INFO:teuthology.orchestra.run.smithi100.stdout: "wait": { 2024-03-21T10:35:52.925 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 0, 2024-03-21T10:35:52.926 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.000000000, 2024-03-21T10:35:52.926 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000000000 2024-03-21T10:35:52.926 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-03-21T10:35:52.926 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-21T10:35:52.926 INFO:teuthology.orchestra.run.smithi100.stdout: "throttle-objecter_ops": { 2024-03-21T10:35:52.926 INFO:teuthology.orchestra.run.smithi100.stdout: "val": 0, 2024-03-21T10:35:52.926 INFO:teuthology.orchestra.run.smithi100.stdout: "max": 1024, 2024-03-21T10:35:52.926 INFO:teuthology.orchestra.run.smithi100.stdout: "get_started": 0, 2024-03-21T10:35:52.926 INFO:teuthology.orchestra.run.smithi100.stdout: "get": 0, 2024-03-21T10:35:52.926 INFO:teuthology.orchestra.run.smithi100.stdout: "get_sum": 0, 2024-03-21T10:35:52.926 INFO:teuthology.orchestra.run.smithi100.stdout: "get_or_fail_fail": 0, 2024-03-21T10:35:52.926 INFO:teuthology.orchestra.run.smithi100.stdout: "get_or_fail_success": 0, 2024-03-21T10:35:52.926 INFO:teuthology.orchestra.run.smithi100.stdout: "take": 1297, 2024-03-21T10:35:52.927 INFO:teuthology.orchestra.run.smithi100.stdout: "take_sum": 1297, 2024-03-21T10:35:52.927 INFO:teuthology.orchestra.run.smithi100.stdout: "put": 1297, 2024-03-21T10:35:52.927 INFO:teuthology.orchestra.run.smithi100.stdout: "put_sum": 1297, 2024-03-21T10:35:52.927 INFO:teuthology.orchestra.run.smithi100.stdout: "wait": { 2024-03-21T10:35:52.927 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 0, 2024-03-21T10:35:52.927 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.000000000, 2024-03-21T10:35:52.927 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000000000 2024-03-21T10:35:52.927 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-03-21T10:35:52.927 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-21T10:35:52.927 INFO:teuthology.orchestra.run.smithi100.stdout: "throttle-write_buf_throttle": { 2024-03-21T10:35:52.927 INFO:teuthology.orchestra.run.smithi100.stdout: "val": 0, 2024-03-21T10:35:52.927 INFO:teuthology.orchestra.run.smithi100.stdout: "max": 3758096384, 2024-03-21T10:35:52.927 INFO:teuthology.orchestra.run.smithi100.stdout: "get_started": 0, 2024-03-21T10:35:52.927 INFO:teuthology.orchestra.run.smithi100.stdout: "get": 4, 2024-03-21T10:35:52.927 INFO:teuthology.orchestra.run.smithi100.stdout: "get_sum": 492, 2024-03-21T10:35:52.927 INFO:teuthology.orchestra.run.smithi100.stdout: "get_or_fail_fail": 0, 2024-03-21T10:35:52.928 INFO:teuthology.orchestra.run.smithi100.stdout: "get_or_fail_success": 4, 2024-03-21T10:35:52.928 INFO:teuthology.orchestra.run.smithi100.stdout: "take": 0, 2024-03-21T10:35:52.928 INFO:teuthology.orchestra.run.smithi100.stdout: "take_sum": 0, 2024-03-21T10:35:52.928 INFO:teuthology.orchestra.run.smithi100.stdout: "put": 1, 2024-03-21T10:35:52.928 INFO:teuthology.orchestra.run.smithi100.stdout: "put_sum": 492, 2024-03-21T10:35:52.928 INFO:teuthology.orchestra.run.smithi100.stdout: "wait": { 2024-03-21T10:35:52.928 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 0, 2024-03-21T10:35:52.928 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.000000000, 2024-03-21T10:35:52.928 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000000000 2024-03-21T10:35:52.928 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-03-21T10:35:52.928 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-21T10:35:52.928 INFO:teuthology.orchestra.run.smithi100.stdout: "throttle-write_buf_throttle-0x55819974c500": { 2024-03-21T10:35:52.928 INFO:teuthology.orchestra.run.smithi100.stdout: "val": 0, 2024-03-21T10:35:52.928 INFO:teuthology.orchestra.run.smithi100.stdout: "max": 3758096384, 2024-03-21T10:35:52.928 INFO:teuthology.orchestra.run.smithi100.stdout: "get_started": 0, 2024-03-21T10:35:52.929 INFO:teuthology.orchestra.run.smithi100.stdout: "get": 250, 2024-03-21T10:35:52.929 INFO:teuthology.orchestra.run.smithi100.stdout: "get_sum": 1323742, 2024-03-21T10:35:52.929 INFO:teuthology.orchestra.run.smithi100.stdout: "get_or_fail_fail": 0, 2024-03-21T10:35:52.929 INFO:teuthology.orchestra.run.smithi100.stdout: "get_or_fail_success": 250, 2024-03-21T10:35:52.929 INFO:teuthology.orchestra.run.smithi100.stdout: "take": 0, 2024-03-21T10:35:52.929 INFO:teuthology.orchestra.run.smithi100.stdout: "take_sum": 0, 2024-03-21T10:35:52.929 INFO:teuthology.orchestra.run.smithi100.stdout: "put": 205, 2024-03-21T10:35:52.929 INFO:teuthology.orchestra.run.smithi100.stdout: "put_sum": 1323742, 2024-03-21T10:35:52.929 INFO:teuthology.orchestra.run.smithi100.stdout: "wait": { 2024-03-21T10:35:52.929 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 0, 2024-03-21T10:35:52.929 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.000000000, 2024-03-21T10:35:52.929 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000000000 2024-03-21T10:35:52.929 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-03-21T10:35:52.929 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-03-21T10:35:52.929 INFO:teuthology.orchestra.run.smithi100.stdout:} 2024-03-21T10:35:52.961 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:35:52 smithi047 ceph-mon[31547]: from='mgr.15687 172.21.15.47:0/8730661' entity='mgr.x' 2024-03-21T10:35:52.961 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:35:52 smithi047 ceph-mon[31547]: from='mgr.15687 172.21.15.47:0/8730661' entity='mgr.x' 2024-03-21T10:35:52.961 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:35:52 smithi047 ceph-mon[31547]: from='mgr.15687 172.21.15.47:0/8730661' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-21T10:35:52.961 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:35:52 smithi047 ceph-mon[31547]: from='mgr.15687 172.21.15.47:0/8730661' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-21T10:35:52.961 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:35:52 smithi047 ceph-mon[31547]: from='mgr.15687 172.21.15.47:0/8730661' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-03-21T10:35:52.961 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:35:52 smithi047 ceph-mon[31547]: from='mgr.15687 172.21.15.47:0/8730661' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-03-21T10:35:52.986 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:35:52 smithi157 ceph-mon[35581]: from='mgr.15687 172.21.15.47:0/8730661' entity='mgr.x' 2024-03-21T10:35:52.986 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:35:52 smithi157 ceph-mon[35581]: from='mgr.15687 172.21.15.47:0/8730661' entity='mgr.x' 2024-03-21T10:35:52.987 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:35:52 smithi157 ceph-mon[35581]: from='mgr.15687 172.21.15.47:0/8730661' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-21T10:35:52.987 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:35:52 smithi157 ceph-mon[35581]: from='mgr.15687 172.21.15.47:0/8730661' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-21T10:35:52.987 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:35:52 smithi157 ceph-mon[35581]: from='mgr.15687 172.21.15.47:0/8730661' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-03-21T10:35:52.987 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:35:52 smithi157 ceph-mon[35581]: from='mgr.15687 172.21.15.47:0/8730661' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-03-21T10:35:53.726 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:35:53 smithi100 ceph-mon[35193]: pgmap v328: 129 pgs: 129 active+clean; 2.5 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-03-21T10:35:53.726 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:35:53 smithi100 ceph-mon[35193]: Adjusting osd_memory_target on smithi157 to 176.8M 2024-03-21T10:35:53.726 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:35:53 smithi100 ceph-mon[35193]: Unable to set osd_memory_target on smithi157 to 185389568: error parsing value: Value '185389568' is below minimum 939524096 2024-03-21T10:35:53.727 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:35:53 smithi100 ceph-mon[35193]: from='mgr.15687 172.21.15.47:0/8730661' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-21T10:35:53.727 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:35:53 smithi100 ceph-mon[35193]: from='mgr.15687 172.21.15.47:0/8730661' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-21T10:35:53.727 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:35:53 smithi100 ceph-mon[35193]: from='mgr.15687 172.21.15.47:0/8730661' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-21T10:35:53.727 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:35:53 smithi100 ceph-mon[35193]: from='mgr.15687 172.21.15.47:0/8730661' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-03-21T10:35:53.829 INFO:tasks.check_counter:Daemon mds.b mds.exported=0 2024-03-21T10:35:53.829 INFO:tasks.check_counter:Daemon mds.b mds.imported=0 2024-03-21T10:35:53.829 INFO:tasks.check_counter:Daemon mds.b mds.root_rsnaps=6 2024-03-21T10:35:53.829 INFO:tasks.check_counter:Daemon mds.b mds_server.req_mksnap_latency.avgcount=10 2024-03-21T10:35:53.829 INFO:tasks.check_counter:Daemon mds.b mds_server.req_rmsnap_latency.avgcount=4 2024-03-21T10:35:53.829 DEBUG:tasks.check_counter:Getting stats from e 2024-03-21T10:35:53.830 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2edfee68ad7243bed422a049b6f5f58619300f11 shell --fsid 3056d51a-e76b-11ee-95cd-87774f69a715 -- ceph daemon mds.e perf dump 2024-03-21T10:35:53.863 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:35:53 smithi047 ceph-mon[31547]: pgmap v328: 129 pgs: 129 active+clean; 2.5 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-03-21T10:35:53.863 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:35:53 smithi047 ceph-mon[31547]: Adjusting osd_memory_target on smithi157 to 176.8M 2024-03-21T10:35:53.863 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:35:53 smithi047 ceph-mon[31547]: Unable to set osd_memory_target on smithi157 to 185389568: error parsing value: Value '185389568' is below minimum 939524096 2024-03-21T10:35:53.864 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:35:53 smithi047 ceph-mon[31547]: from='mgr.15687 172.21.15.47:0/8730661' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-21T10:35:53.864 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:35:53 smithi047 ceph-mon[31547]: from='mgr.15687 172.21.15.47:0/8730661' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-21T10:35:53.864 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:35:53 smithi047 ceph-mon[31547]: from='mgr.15687 172.21.15.47:0/8730661' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-21T10:35:53.864 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:35:53 smithi047 ceph-mon[31547]: from='mgr.15687 172.21.15.47:0/8730661' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-03-21T10:35:53.987 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:35:53 smithi157 ceph-mon[35581]: pgmap v328: 129 pgs: 129 active+clean; 2.5 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-03-21T10:35:53.987 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:35:53 smithi157 ceph-mon[35581]: Adjusting osd_memory_target on smithi157 to 176.8M 2024-03-21T10:35:53.987 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:35:53 smithi157 ceph-mon[35581]: Unable to set osd_memory_target on smithi157 to 185389568: error parsing value: Value '185389568' is below minimum 939524096 2024-03-21T10:35:53.987 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:35:53 smithi157 ceph-mon[35581]: from='mgr.15687 172.21.15.47:0/8730661' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-21T10:35:53.987 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:35:53 smithi157 ceph-mon[35581]: from='mgr.15687 172.21.15.47:0/8730661' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-21T10:35:53.987 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:35:53 smithi157 ceph-mon[35581]: from='mgr.15687 172.21.15.47:0/8730661' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-21T10:35:53.987 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:35:53 smithi157 ceph-mon[35581]: from='mgr.15687 172.21.15.47:0/8730661' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-03-21T10:35:54.312 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/3056d51a-e76b-11ee-95cd-87774f69a715/mon.b/config 2024-03-21T10:35:54.799 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:35:54 smithi100 ceph-mon[35193]: Adjusting osd_memory_target on smithi047 to 177.8M 2024-03-21T10:35:54.799 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:35:54 smithi100 ceph-mon[35193]: Unable to set osd_memory_target on smithi047 to 186497740: error parsing value: Value '186497740' is below minimum 939524096 2024-03-21T10:35:54.799 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:35:54 smithi100 ceph-mon[35193]: from='mgr.15687 172.21.15.47:0/8730661' entity='mgr.x' 2024-03-21T10:35:54.799 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:35:54 smithi100 ceph-mon[35193]: from='mgr.15687 172.21.15.47:0/8730661' entity='mgr.x' 2024-03-21T10:35:54.897 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:35:54 smithi047 ceph-mon[31547]: Adjusting osd_memory_target on smithi047 to 177.8M 2024-03-21T10:35:54.897 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:35:54 smithi047 ceph-mon[31547]: Unable to set osd_memory_target on smithi047 to 186497740: error parsing value: Value '186497740' is below minimum 939524096 2024-03-21T10:35:54.897 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:35:54 smithi047 ceph-mon[31547]: from='mgr.15687 172.21.15.47:0/8730661' entity='mgr.x' 2024-03-21T10:35:54.898 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:35:54 smithi047 ceph-mon[31547]: from='mgr.15687 172.21.15.47:0/8730661' entity='mgr.x' 2024-03-21T10:35:54.987 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:35:54 smithi157 ceph-mon[35581]: Adjusting osd_memory_target on smithi047 to 177.8M 2024-03-21T10:35:54.987 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:35:54 smithi157 ceph-mon[35581]: Unable to set osd_memory_target on smithi047 to 186497740: error parsing value: Value '186497740' is below minimum 939524096 2024-03-21T10:35:54.987 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:35:54 smithi157 ceph-mon[35581]: from='mgr.15687 172.21.15.47:0/8730661' entity='mgr.x' 2024-03-21T10:35:54.987 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:35:54 smithi157 ceph-mon[35581]: from='mgr.15687 172.21.15.47:0/8730661' entity='mgr.x' 2024-03-21T10:35:55.652 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:35:55 smithi047 ceph-mon[31547]: pgmap v329: 129 pgs: 129 active+clean; 2.5 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-21T10:35:55.652 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:35:55 smithi047 ceph-mon[31547]: from='mgr.15687 172.21.15.47:0/8730661' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-21T10:35:55.652 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:35:55 smithi047 ceph-mon[31547]: from='mgr.15687 172.21.15.47:0/8730661' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-21T10:35:55.652 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:35:55 smithi047 ceph-mon[31547]: from='mgr.15687 172.21.15.47:0/8730661' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-21T10:35:55.652 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:35:55 smithi047 ceph-mon[31547]: from='mgr.15687 172.21.15.47:0/8730661' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-03-21T10:35:55.652 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:35:55 smithi047 ceph-mon[31547]: from='mgr.15687 172.21.15.47:0/8730661' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T10:35:55.652 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:35:55 smithi047 ceph-mon[31547]: from='mgr.15687 172.21.15.47:0/8730661' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T10:35:55.652 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:35:55 smithi047 ceph-mon[31547]: from='mgr.15687 172.21.15.47:0/8730661' entity='mgr.x' 2024-03-21T10:35:55.727 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:35:55 smithi100 ceph-mon[35193]: pgmap v329: 129 pgs: 129 active+clean; 2.5 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-21T10:35:55.727 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:35:55 smithi100 ceph-mon[35193]: from='mgr.15687 172.21.15.47:0/8730661' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-21T10:35:55.727 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:35:55 smithi100 ceph-mon[35193]: from='mgr.15687 172.21.15.47:0/8730661' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-21T10:35:55.727 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:35:55 smithi100 ceph-mon[35193]: from='mgr.15687 172.21.15.47:0/8730661' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-21T10:35:55.728 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:35:55 smithi100 ceph-mon[35193]: from='mgr.15687 172.21.15.47:0/8730661' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-03-21T10:35:55.728 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:35:55 smithi100 ceph-mon[35193]: from='mgr.15687 172.21.15.47:0/8730661' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T10:35:55.728 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:35:55 smithi100 ceph-mon[35193]: from='mgr.15687 172.21.15.47:0/8730661' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T10:35:55.728 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:35:55 smithi100 ceph-mon[35193]: from='mgr.15687 172.21.15.47:0/8730661' entity='mgr.x' 2024-03-21T10:35:55.987 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:35:55 smithi157 ceph-mon[35581]: pgmap v329: 129 pgs: 129 active+clean; 2.5 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-21T10:35:55.987 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:35:55 smithi157 ceph-mon[35581]: from='mgr.15687 172.21.15.47:0/8730661' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-21T10:35:55.987 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:35:55 smithi157 ceph-mon[35581]: from='mgr.15687 172.21.15.47:0/8730661' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-21T10:35:55.987 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:35:55 smithi157 ceph-mon[35581]: from='mgr.15687 172.21.15.47:0/8730661' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-21T10:35:55.987 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:35:55 smithi157 ceph-mon[35581]: from='mgr.15687 172.21.15.47:0/8730661' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-03-21T10:35:55.987 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:35:55 smithi157 ceph-mon[35581]: from='mgr.15687 172.21.15.47:0/8730661' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T10:35:55.987 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:35:55 smithi157 ceph-mon[35581]: from='mgr.15687 172.21.15.47:0/8730661' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T10:35:55.987 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:35:55 smithi157 ceph-mon[35581]: from='mgr.15687 172.21.15.47:0/8730661' entity='mgr.x' 2024-03-21T10:35:56.575 INFO:teuthology.orchestra.run.smithi100.stdout:{ 2024-03-21T10:35:56.575 INFO:teuthology.orchestra.run.smithi100.stdout: "AsyncMessenger::Worker-0": { 2024-03-21T10:35:56.575 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_recv_messages": 131, 2024-03-21T10:35:56.576 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_send_messages": 164, 2024-03-21T10:35:56.576 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_recv_bytes": 42062, 2024-03-21T10:35:56.576 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_send_bytes": 507253, 2024-03-21T10:35:56.576 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_created_connections": 54, 2024-03-21T10:35:56.576 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_active_connections": 6, 2024-03-21T10:35:56.576 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_running_total_time": 0.063303545, 2024-03-21T10:35:56.576 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_running_send_time": 0.010614228, 2024-03-21T10:35:56.576 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_running_recv_time": 0.040155031, 2024-03-21T10:35:56.576 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_running_fast_dispatch_time": 0.000627733, 2024-03-21T10:35:56.576 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_send_messages_queue_lat": { 2024-03-21T10:35:56.576 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 158, 2024-03-21T10:35:56.576 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.103297325, 2024-03-21T10:35:56.576 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000653780 2024-03-21T10:35:56.576 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-21T10:35:56.577 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_handle_ack_lat": { 2024-03-21T10:35:56.577 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 169, 2024-03-21T10:35:56.577 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.000573316, 2024-03-21T10:35:56.577 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000003392 2024-03-21T10:35:56.577 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-21T10:35:56.577 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_recv_encrypted_bytes": 928, 2024-03-21T10:35:56.577 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_send_encrypted_bytes": 736 2024-03-21T10:35:56.577 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-21T10:35:56.577 INFO:teuthology.orchestra.run.smithi100.stdout: "AsyncMessenger::Worker-1": { 2024-03-21T10:35:56.577 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_recv_messages": 478, 2024-03-21T10:35:56.577 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_send_messages": 826, 2024-03-21T10:35:56.577 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_recv_bytes": 149385, 2024-03-21T10:35:56.577 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_send_bytes": 762928, 2024-03-21T10:35:56.578 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_created_connections": 11, 2024-03-21T10:35:56.578 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_active_connections": 4, 2024-03-21T10:35:56.578 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_running_total_time": 0.141097095, 2024-03-21T10:35:56.578 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_running_send_time": 0.081125989, 2024-03-21T10:35:56.578 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_running_recv_time": 0.051839991, 2024-03-21T10:35:56.578 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_running_fast_dispatch_time": 0.004873746, 2024-03-21T10:35:56.578 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_send_messages_queue_lat": { 2024-03-21T10:35:56.578 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 827, 2024-03-21T10:35:56.578 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.212707635, 2024-03-21T10:35:56.578 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000257203 2024-03-21T10:35:56.578 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-21T10:35:56.579 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_handle_ack_lat": { 2024-03-21T10:35:56.579 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 1062, 2024-03-21T10:35:56.579 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.000333108, 2024-03-21T10:35:56.579 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000000313 2024-03-21T10:35:56.579 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-21T10:35:56.579 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_recv_encrypted_bytes": 320, 2024-03-21T10:35:56.579 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_send_encrypted_bytes": 397328 2024-03-21T10:35:56.579 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-21T10:35:56.579 INFO:teuthology.orchestra.run.smithi100.stdout: "AsyncMessenger::Worker-2": { 2024-03-21T10:35:56.579 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_recv_messages": 383, 2024-03-21T10:35:56.579 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_send_messages": 385, 2024-03-21T10:35:56.579 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_recv_bytes": 1838395, 2024-03-21T10:35:56.579 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_send_bytes": 217165, 2024-03-21T10:35:56.580 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_created_connections": 10, 2024-03-21T10:35:56.580 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_active_connections": 5, 2024-03-21T10:35:56.580 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_running_total_time": 0.099029888, 2024-03-21T10:35:56.580 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_running_send_time": 0.032619133, 2024-03-21T10:35:56.580 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_running_recv_time": 0.108357365, 2024-03-21T10:35:56.580 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_running_fast_dispatch_time": 0.007709347, 2024-03-21T10:35:56.580 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_send_messages_queue_lat": { 2024-03-21T10:35:56.580 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 385, 2024-03-21T10:35:56.580 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.105646198, 2024-03-21T10:35:56.580 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000274405 2024-03-21T10:35:56.580 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-21T10:35:56.580 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_handle_ack_lat": { 2024-03-21T10:35:56.581 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 132, 2024-03-21T10:35:56.581 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.000036679, 2024-03-21T10:35:56.581 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000000277 2024-03-21T10:35:56.581 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-21T10:35:56.581 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_recv_encrypted_bytes": 1808464, 2024-03-21T10:35:56.581 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_send_encrypted_bytes": 159824 2024-03-21T10:35:56.581 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-21T10:35:56.581 INFO:teuthology.orchestra.run.smithi100.stdout: "cct": { 2024-03-21T10:35:56.581 INFO:teuthology.orchestra.run.smithi100.stdout: "total_workers": 1, 2024-03-21T10:35:56.581 INFO:teuthology.orchestra.run.smithi100.stdout: "unhealthy_workers": 0 2024-03-21T10:35:56.581 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-21T10:35:56.582 INFO:teuthology.orchestra.run.smithi100.stdout: "finisher-MDSRank": { 2024-03-21T10:35:56.582 INFO:teuthology.orchestra.run.smithi100.stdout: "queue_len": 0, 2024-03-21T10:35:56.582 INFO:teuthology.orchestra.run.smithi100.stdout: "complete_latency": { 2024-03-21T10:35:56.582 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 53, 2024-03-21T10:35:56.582 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.004163825, 2024-03-21T10:35:56.582 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000078562 2024-03-21T10:35:56.582 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-03-21T10:35:56.582 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-21T10:35:56.582 INFO:teuthology.orchestra.run.smithi100.stdout: "finisher-PurgeQueue": { 2024-03-21T10:35:56.582 INFO:teuthology.orchestra.run.smithi100.stdout: "queue_len": 0, 2024-03-21T10:35:56.582 INFO:teuthology.orchestra.run.smithi100.stdout: "complete_latency": { 2024-03-21T10:35:56.582 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 2, 2024-03-21T10:35:56.583 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.000007798, 2024-03-21T10:35:56.583 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000003899 2024-03-21T10:35:56.583 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-03-21T10:35:56.583 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-21T10:35:56.583 INFO:teuthology.orchestra.run.smithi100.stdout: "mds": { 2024-03-21T10:35:56.583 INFO:teuthology.orchestra.run.smithi100.stdout: "request": 6, 2024-03-21T10:35:56.583 INFO:teuthology.orchestra.run.smithi100.stdout: "reply": 4, 2024-03-21T10:35:56.583 INFO:teuthology.orchestra.run.smithi100.stdout: "reply_latency": { 2024-03-21T10:35:56.583 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 4, 2024-03-21T10:35:56.583 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.001849530, 2024-03-21T10:35:56.583 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000462382 2024-03-21T10:35:56.584 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-21T10:35:56.584 INFO:teuthology.orchestra.run.smithi100.stdout: "slow_reply": 0, 2024-03-21T10:35:56.584 INFO:teuthology.orchestra.run.smithi100.stdout: "forward": 2, 2024-03-21T10:35:56.584 INFO:teuthology.orchestra.run.smithi100.stdout: "dir_fetch_complete": 0, 2024-03-21T10:35:56.584 INFO:teuthology.orchestra.run.smithi100.stdout: "dir_fetch_keys": 0, 2024-03-21T10:35:56.584 INFO:teuthology.orchestra.run.smithi100.stdout: "dir_commit": 11, 2024-03-21T10:35:56.584 INFO:teuthology.orchestra.run.smithi100.stdout: "dir_split": 0, 2024-03-21T10:35:56.584 INFO:teuthology.orchestra.run.smithi100.stdout: "dir_merge": 0, 2024-03-21T10:35:56.584 INFO:teuthology.orchestra.run.smithi100.stdout: "inodes": 11, 2024-03-21T10:35:56.584 INFO:teuthology.orchestra.run.smithi100.stdout: "inodes_top": 0, 2024-03-21T10:35:56.584 INFO:teuthology.orchestra.run.smithi100.stdout: "inodes_bottom": 11, 2024-03-21T10:35:56.584 INFO:teuthology.orchestra.run.smithi100.stdout: "inodes_pin_tail": 0, 2024-03-21T10:35:56.584 INFO:teuthology.orchestra.run.smithi100.stdout: "inodes_pinned": 11, 2024-03-21T10:35:56.585 INFO:teuthology.orchestra.run.smithi100.stdout: "inodes_expired": 0, 2024-03-21T10:35:56.585 INFO:teuthology.orchestra.run.smithi100.stdout: "inodes_with_caps": 2, 2024-03-21T10:35:56.585 INFO:teuthology.orchestra.run.smithi100.stdout: "caps": 2, 2024-03-21T10:35:56.585 INFO:teuthology.orchestra.run.smithi100.stdout: "subtrees": 2, 2024-03-21T10:35:56.585 INFO:teuthology.orchestra.run.smithi100.stdout: "traverse": 9, 2024-03-21T10:35:56.585 INFO:teuthology.orchestra.run.smithi100.stdout: "traverse_hit": 7, 2024-03-21T10:35:56.585 INFO:teuthology.orchestra.run.smithi100.stdout: "traverse_forward": 0, 2024-03-21T10:35:56.585 INFO:teuthology.orchestra.run.smithi100.stdout: "traverse_discover": 1, 2024-03-21T10:35:56.585 INFO:teuthology.orchestra.run.smithi100.stdout: "traverse_dir_fetch": 0, 2024-03-21T10:35:56.585 INFO:teuthology.orchestra.run.smithi100.stdout: "traverse_remote_ino": 0, 2024-03-21T10:35:56.585 INFO:teuthology.orchestra.run.smithi100.stdout: "traverse_lock": 0, 2024-03-21T10:35:56.585 INFO:teuthology.orchestra.run.smithi100.stdout: "load_cent": 0, 2024-03-21T10:35:56.585 INFO:teuthology.orchestra.run.smithi100.stdout: "q": 0, 2024-03-21T10:35:56.586 INFO:teuthology.orchestra.run.smithi100.stdout: "exported": 0, 2024-03-21T10:35:56.586 INFO:teuthology.orchestra.run.smithi100.stdout: "exported_inodes": 0, 2024-03-21T10:35:56.586 INFO:teuthology.orchestra.run.smithi100.stdout: "imported": 0, 2024-03-21T10:35:56.586 INFO:teuthology.orchestra.run.smithi100.stdout: "imported_inodes": 0, 2024-03-21T10:35:56.586 INFO:teuthology.orchestra.run.smithi100.stdout: "openino_dir_fetch": 0, 2024-03-21T10:35:56.586 INFO:teuthology.orchestra.run.smithi100.stdout: "openino_backtrace_fetch": 0, 2024-03-21T10:35:56.586 INFO:teuthology.orchestra.run.smithi100.stdout: "openino_peer_discover": 0, 2024-03-21T10:35:56.586 INFO:teuthology.orchestra.run.smithi100.stdout: "root_rfiles": 4, 2024-03-21T10:35:56.586 INFO:teuthology.orchestra.run.smithi100.stdout: "root_rbytes": 234, 2024-03-21T10:35:56.586 INFO:teuthology.orchestra.run.smithi100.stdout: "root_rsnaps": 6, 2024-03-21T10:35:56.586 INFO:teuthology.orchestra.run.smithi100.stdout: "scrub_backtrace_fetch": 0, 2024-03-21T10:35:56.586 INFO:teuthology.orchestra.run.smithi100.stdout: "scrub_set_tag": 0, 2024-03-21T10:35:56.586 INFO:teuthology.orchestra.run.smithi100.stdout: "scrub_backtrace_repaired": 0, 2024-03-21T10:35:56.586 INFO:teuthology.orchestra.run.smithi100.stdout: "scrub_inotable_repaired": 0, 2024-03-21T10:35:56.587 INFO:teuthology.orchestra.run.smithi100.stdout: "scrub_dir_inodes": 0, 2024-03-21T10:35:56.587 INFO:teuthology.orchestra.run.smithi100.stdout: "scrub_dir_base_inodes": 0, 2024-03-21T10:35:56.587 INFO:teuthology.orchestra.run.smithi100.stdout: "scrub_dirfrag_rstats": 0, 2024-03-21T10:35:56.587 INFO:teuthology.orchestra.run.smithi100.stdout: "scrub_file_inodes": 0, 2024-03-21T10:35:56.587 INFO:teuthology.orchestra.run.smithi100.stdout: "handle_inode_file_caps": 0, 2024-03-21T10:35:56.587 INFO:teuthology.orchestra.run.smithi100.stdout: "ceph_cap_op_revoke": 0, 2024-03-21T10:35:56.587 INFO:teuthology.orchestra.run.smithi100.stdout: "ceph_cap_op_grant": 0, 2024-03-21T10:35:56.587 INFO:teuthology.orchestra.run.smithi100.stdout: "ceph_cap_op_trunc": 0, 2024-03-21T10:35:56.587 INFO:teuthology.orchestra.run.smithi100.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-03-21T10:35:56.587 INFO:teuthology.orchestra.run.smithi100.stdout: "ceph_cap_op_flush_ack": 0, 2024-03-21T10:35:56.587 INFO:teuthology.orchestra.run.smithi100.stdout: "handle_client_caps": 0, 2024-03-21T10:35:56.587 INFO:teuthology.orchestra.run.smithi100.stdout: "handle_client_caps_dirty": 0, 2024-03-21T10:35:56.587 INFO:teuthology.orchestra.run.smithi100.stdout: "handle_client_cap_release": 0, 2024-03-21T10:35:56.588 INFO:teuthology.orchestra.run.smithi100.stdout: "process_request_cap_release": 0 2024-03-21T10:35:56.588 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-21T10:35:56.588 INFO:teuthology.orchestra.run.smithi100.stdout: "mds_cache": { 2024-03-21T10:35:56.588 INFO:teuthology.orchestra.run.smithi100.stdout: "dir_update": 0, 2024-03-21T10:35:56.588 INFO:teuthology.orchestra.run.smithi100.stdout: "dir_update_receipt": 0, 2024-03-21T10:35:56.588 INFO:teuthology.orchestra.run.smithi100.stdout: "dir_try_discover": 0, 2024-03-21T10:35:56.588 INFO:teuthology.orchestra.run.smithi100.stdout: "dir_send_discover": 2, 2024-03-21T10:35:56.588 INFO:teuthology.orchestra.run.smithi100.stdout: "dir_handle_discover": 0, 2024-03-21T10:35:56.588 INFO:teuthology.orchestra.run.smithi100.stdout: "num_strays": 0, 2024-03-21T10:35:56.588 INFO:teuthology.orchestra.run.smithi100.stdout: "num_strays_delayed": 0, 2024-03-21T10:35:56.588 INFO:teuthology.orchestra.run.smithi100.stdout: "num_strays_enqueuing": 0, 2024-03-21T10:35:56.588 INFO:teuthology.orchestra.run.smithi100.stdout: "strays_created": 0, 2024-03-21T10:35:56.588 INFO:teuthology.orchestra.run.smithi100.stdout: "strays_enqueued": 0, 2024-03-21T10:35:56.588 INFO:teuthology.orchestra.run.smithi100.stdout: "strays_reintegrated": 0, 2024-03-21T10:35:56.589 INFO:teuthology.orchestra.run.smithi100.stdout: "strays_migrated": 0, 2024-03-21T10:35:56.589 INFO:teuthology.orchestra.run.smithi100.stdout: "num_recovering_processing": 0, 2024-03-21T10:35:56.589 INFO:teuthology.orchestra.run.smithi100.stdout: "num_recovering_enqueued": 0, 2024-03-21T10:35:56.589 INFO:teuthology.orchestra.run.smithi100.stdout: "num_recovering_prioritized": 0, 2024-03-21T10:35:56.589 INFO:teuthology.orchestra.run.smithi100.stdout: "recovery_started": 0, 2024-03-21T10:35:56.589 INFO:teuthology.orchestra.run.smithi100.stdout: "recovery_completed": 0, 2024-03-21T10:35:56.589 INFO:teuthology.orchestra.run.smithi100.stdout: "ireq_enqueue_scrub": 0, 2024-03-21T10:35:56.589 INFO:teuthology.orchestra.run.smithi100.stdout: "ireq_exportdir": 0, 2024-03-21T10:35:56.589 INFO:teuthology.orchestra.run.smithi100.stdout: "ireq_flush": 0, 2024-03-21T10:35:56.589 INFO:teuthology.orchestra.run.smithi100.stdout: "ireq_fragmentdir": 0, 2024-03-21T10:35:56.589 INFO:teuthology.orchestra.run.smithi100.stdout: "ireq_fragstats": 0, 2024-03-21T10:35:56.589 INFO:teuthology.orchestra.run.smithi100.stdout: "ireq_inodestats": 0 2024-03-21T10:35:56.589 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-21T10:35:56.590 INFO:teuthology.orchestra.run.smithi100.stdout: "mds_log": { 2024-03-21T10:35:56.590 INFO:teuthology.orchestra.run.smithi100.stdout: "evlrg": 0, 2024-03-21T10:35:56.590 INFO:teuthology.orchestra.run.smithi100.stdout: "evadd": 6, 2024-03-21T10:35:56.590 INFO:teuthology.orchestra.run.smithi100.stdout: "evex": 0, 2024-03-21T10:35:56.590 INFO:teuthology.orchestra.run.smithi100.stdout: "evtrm": 0, 2024-03-21T10:35:56.590 INFO:teuthology.orchestra.run.smithi100.stdout: "ev": 6, 2024-03-21T10:35:56.590 INFO:teuthology.orchestra.run.smithi100.stdout: "evexg": 0, 2024-03-21T10:35:56.590 INFO:teuthology.orchestra.run.smithi100.stdout: "evexd": 0, 2024-03-21T10:35:56.590 INFO:teuthology.orchestra.run.smithi100.stdout: "segadd": 2, 2024-03-21T10:35:56.590 INFO:teuthology.orchestra.run.smithi100.stdout: "segex": 0, 2024-03-21T10:35:56.590 INFO:teuthology.orchestra.run.smithi100.stdout: "segtrm": 0, 2024-03-21T10:35:56.590 INFO:teuthology.orchestra.run.smithi100.stdout: "seg": 2, 2024-03-21T10:35:56.590 INFO:teuthology.orchestra.run.smithi100.stdout: "segmjr": 2, 2024-03-21T10:35:56.591 INFO:teuthology.orchestra.run.smithi100.stdout: "segexg": 0, 2024-03-21T10:35:56.591 INFO:teuthology.orchestra.run.smithi100.stdout: "segexd": 0, 2024-03-21T10:35:56.591 INFO:teuthology.orchestra.run.smithi100.stdout: "expos": 4194304, 2024-03-21T10:35:56.591 INFO:teuthology.orchestra.run.smithi100.stdout: "wrpos": 4196308, 2024-03-21T10:35:56.591 INFO:teuthology.orchestra.run.smithi100.stdout: "rdpos": 0, 2024-03-21T10:35:56.591 INFO:teuthology.orchestra.run.smithi100.stdout: "jlat": { 2024-03-21T10:35:56.591 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 5, 2024-03-21T10:35:56.591 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.018218446, 2024-03-21T10:35:56.591 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.003643689 2024-03-21T10:35:56.591 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-21T10:35:56.591 INFO:teuthology.orchestra.run.smithi100.stdout: "replayed": 0 2024-03-21T10:35:56.591 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-21T10:35:56.592 INFO:teuthology.orchestra.run.smithi100.stdout: "mds_mem": { 2024-03-21T10:35:56.592 INFO:teuthology.orchestra.run.smithi100.stdout: "ino": 14, 2024-03-21T10:35:56.592 INFO:teuthology.orchestra.run.smithi100.stdout: "ino+": 14, 2024-03-21T10:35:56.592 INFO:teuthology.orchestra.run.smithi100.stdout: "ino-": 0, 2024-03-21T10:35:56.592 INFO:teuthology.orchestra.run.smithi100.stdout: "dir": 12, 2024-03-21T10:35:56.592 INFO:teuthology.orchestra.run.smithi100.stdout: "dir+": 13, 2024-03-21T10:35:56.592 INFO:teuthology.orchestra.run.smithi100.stdout: "dir-": 1, 2024-03-21T10:35:56.592 INFO:teuthology.orchestra.run.smithi100.stdout: "dn": 11, 2024-03-21T10:35:56.592 INFO:teuthology.orchestra.run.smithi100.stdout: "dn+": 11, 2024-03-21T10:35:56.592 INFO:teuthology.orchestra.run.smithi100.stdout: "dn-": 0, 2024-03-21T10:35:56.592 INFO:teuthology.orchestra.run.smithi100.stdout: "cap": 2, 2024-03-21T10:35:56.592 INFO:teuthology.orchestra.run.smithi100.stdout: "cap+": 2, 2024-03-21T10:35:56.593 INFO:teuthology.orchestra.run.smithi100.stdout: "cap-": 0, 2024-03-21T10:35:56.593 INFO:teuthology.orchestra.run.smithi100.stdout: "rss": 51328, 2024-03-21T10:35:56.593 INFO:teuthology.orchestra.run.smithi100.stdout: "heap": 223612 2024-03-21T10:35:56.593 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-21T10:35:56.593 INFO:teuthology.orchestra.run.smithi100.stdout: "mds_server": { 2024-03-21T10:35:56.593 INFO:teuthology.orchestra.run.smithi100.stdout: "dispatch_client_request": 7, 2024-03-21T10:35:56.593 INFO:teuthology.orchestra.run.smithi100.stdout: "dispatch_server_request": 0, 2024-03-21T10:35:56.593 INFO:teuthology.orchestra.run.smithi100.stdout: "handle_client_request": 6, 2024-03-21T10:35:56.593 INFO:teuthology.orchestra.run.smithi100.stdout: "handle_client_session": 21, 2024-03-21T10:35:56.593 INFO:teuthology.orchestra.run.smithi100.stdout: "handle_peer_request": 0, 2024-03-21T10:35:56.593 INFO:teuthology.orchestra.run.smithi100.stdout: "req_create_latency": { 2024-03-21T10:35:56.593 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 0, 2024-03-21T10:35:56.594 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.000000000, 2024-03-21T10:35:56.594 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000000000 2024-03-21T10:35:56.594 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-21T10:35:56.594 INFO:teuthology.orchestra.run.smithi100.stdout: "req_getattr_latency": { 2024-03-21T10:35:56.594 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 4, 2024-03-21T10:35:56.594 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.001849530, 2024-03-21T10:35:56.594 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000462382 2024-03-21T10:35:56.594 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-21T10:35:56.594 INFO:teuthology.orchestra.run.smithi100.stdout: "req_getfilelock_latency": { 2024-03-21T10:35:56.594 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 0, 2024-03-21T10:35:56.594 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.000000000, 2024-03-21T10:35:56.594 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000000000 2024-03-21T10:35:56.595 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-21T10:35:56.595 INFO:teuthology.orchestra.run.smithi100.stdout: "req_link_latency": { 2024-03-21T10:35:56.595 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 0, 2024-03-21T10:35:56.595 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.000000000, 2024-03-21T10:35:56.595 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000000000 2024-03-21T10:35:56.595 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-21T10:35:56.595 INFO:teuthology.orchestra.run.smithi100.stdout: "req_lookup_latency": { 2024-03-21T10:35:56.595 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 0, 2024-03-21T10:35:56.595 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.000000000, 2024-03-21T10:35:56.595 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000000000 2024-03-21T10:35:56.595 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-21T10:35:56.595 INFO:teuthology.orchestra.run.smithi100.stdout: "req_lookuphash_latency": { 2024-03-21T10:35:56.596 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 0, 2024-03-21T10:35:56.596 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.000000000, 2024-03-21T10:35:56.596 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000000000 2024-03-21T10:35:56.596 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-21T10:35:56.596 INFO:teuthology.orchestra.run.smithi100.stdout: "req_lookupino_latency": { 2024-03-21T10:35:56.596 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 0, 2024-03-21T10:35:56.596 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.000000000, 2024-03-21T10:35:56.596 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000000000 2024-03-21T10:35:56.596 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-21T10:35:56.596 INFO:teuthology.orchestra.run.smithi100.stdout: "req_lookupname_latency": { 2024-03-21T10:35:56.596 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 0, 2024-03-21T10:35:56.596 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.000000000, 2024-03-21T10:35:56.597 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000000000 2024-03-21T10:35:56.597 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-21T10:35:56.597 INFO:teuthology.orchestra.run.smithi100.stdout: "req_lookupparent_latency": { 2024-03-21T10:35:56.597 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 0, 2024-03-21T10:35:56.597 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.000000000, 2024-03-21T10:35:56.597 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000000000 2024-03-21T10:35:56.597 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-21T10:35:56.597 INFO:teuthology.orchestra.run.smithi100.stdout: "req_lookupsnap_latency": { 2024-03-21T10:35:56.597 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 0, 2024-03-21T10:35:56.597 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.000000000, 2024-03-21T10:35:56.597 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000000000 2024-03-21T10:35:56.597 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-21T10:35:56.598 INFO:teuthology.orchestra.run.smithi100.stdout: "req_lssnap_latency": { 2024-03-21T10:35:56.598 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 0, 2024-03-21T10:35:56.598 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.000000000, 2024-03-21T10:35:56.598 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000000000 2024-03-21T10:35:56.598 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-21T10:35:56.598 INFO:teuthology.orchestra.run.smithi100.stdout: "req_mkdir_latency": { 2024-03-21T10:35:56.598 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 0, 2024-03-21T10:35:56.598 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.000000000, 2024-03-21T10:35:56.598 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000000000 2024-03-21T10:35:56.599 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-21T10:35:56.599 INFO:teuthology.orchestra.run.smithi100.stdout: "req_mknod_latency": { 2024-03-21T10:35:56.599 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 0, 2024-03-21T10:35:56.599 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.000000000, 2024-03-21T10:35:56.599 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000000000 2024-03-21T10:35:56.599 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-21T10:35:56.599 INFO:teuthology.orchestra.run.smithi100.stdout: "req_mksnap_latency": { 2024-03-21T10:35:56.600 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 0, 2024-03-21T10:35:56.600 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.000000000, 2024-03-21T10:35:56.600 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000000000 2024-03-21T10:35:56.600 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-21T10:35:56.600 INFO:teuthology.orchestra.run.smithi100.stdout: "req_open_latency": { 2024-03-21T10:35:56.600 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 0, 2024-03-21T10:35:56.600 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.000000000, 2024-03-21T10:35:56.600 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000000000 2024-03-21T10:35:56.600 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-21T10:35:56.600 INFO:teuthology.orchestra.run.smithi100.stdout: "req_readdir_latency": { 2024-03-21T10:35:56.600 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 0, 2024-03-21T10:35:56.601 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.000000000, 2024-03-21T10:35:56.601 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000000000 2024-03-21T10:35:56.601 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-21T10:35:56.601 INFO:teuthology.orchestra.run.smithi100.stdout: "req_rename_latency": { 2024-03-21T10:35:56.601 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 0, 2024-03-21T10:35:56.601 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.000000000, 2024-03-21T10:35:56.601 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000000000 2024-03-21T10:35:56.601 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-21T10:35:56.601 INFO:teuthology.orchestra.run.smithi100.stdout: "req_renamesnap_latency": { 2024-03-21T10:35:56.601 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 0, 2024-03-21T10:35:56.602 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.000000000, 2024-03-21T10:35:56.602 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000000000 2024-03-21T10:35:56.602 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-21T10:35:56.602 INFO:teuthology.orchestra.run.smithi100.stdout: "req_snapdiff_latency": { 2024-03-21T10:35:56.602 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 0, 2024-03-21T10:35:56.602 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.000000000, 2024-03-21T10:35:56.602 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000000000 2024-03-21T10:35:56.602 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-21T10:35:56.602 INFO:teuthology.orchestra.run.smithi100.stdout: "req_rmdir_latency": { 2024-03-21T10:35:56.602 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 0, 2024-03-21T10:35:56.602 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.000000000, 2024-03-21T10:35:56.603 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000000000 2024-03-21T10:35:56.603 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-21T10:35:56.603 INFO:teuthology.orchestra.run.smithi100.stdout: "req_rmsnap_latency": { 2024-03-21T10:35:56.603 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 0, 2024-03-21T10:35:56.603 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.000000000, 2024-03-21T10:35:56.603 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000000000 2024-03-21T10:35:56.603 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-21T10:35:56.603 INFO:teuthology.orchestra.run.smithi100.stdout: "req_rmxattr_latency": { 2024-03-21T10:35:56.603 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 0, 2024-03-21T10:35:56.603 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.000000000, 2024-03-21T10:35:56.603 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000000000 2024-03-21T10:35:56.604 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-21T10:35:56.604 INFO:teuthology.orchestra.run.smithi100.stdout: "req_setattr_latency": { 2024-03-21T10:35:56.604 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 0, 2024-03-21T10:35:56.604 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.000000000, 2024-03-21T10:35:56.604 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000000000 2024-03-21T10:35:56.604 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-21T10:35:56.604 INFO:teuthology.orchestra.run.smithi100.stdout: "req_setdirlayout_latency": { 2024-03-21T10:35:56.604 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 0, 2024-03-21T10:35:56.604 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.000000000, 2024-03-21T10:35:56.604 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000000000 2024-03-21T10:35:56.604 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-21T10:35:56.604 INFO:teuthology.orchestra.run.smithi100.stdout: "req_setfilelock_latency": { 2024-03-21T10:35:56.605 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 0, 2024-03-21T10:35:56.605 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.000000000, 2024-03-21T10:35:56.605 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000000000 2024-03-21T10:35:56.605 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-21T10:35:56.605 INFO:teuthology.orchestra.run.smithi100.stdout: "req_setlayout_latency": { 2024-03-21T10:35:56.605 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 0, 2024-03-21T10:35:56.605 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.000000000, 2024-03-21T10:35:56.605 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000000000 2024-03-21T10:35:56.605 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-21T10:35:56.605 INFO:teuthology.orchestra.run.smithi100.stdout: "req_setxattr_latency": { 2024-03-21T10:35:56.605 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 0, 2024-03-21T10:35:56.606 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.000000000, 2024-03-21T10:35:56.606 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000000000 2024-03-21T10:35:56.606 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-21T10:35:56.606 INFO:teuthology.orchestra.run.smithi100.stdout: "req_symlink_latency": { 2024-03-21T10:35:56.606 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 0, 2024-03-21T10:35:56.606 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.000000000, 2024-03-21T10:35:56.606 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000000000 2024-03-21T10:35:56.606 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-21T10:35:56.606 INFO:teuthology.orchestra.run.smithi100.stdout: "req_unlink_latency": { 2024-03-21T10:35:56.606 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 0, 2024-03-21T10:35:56.606 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.000000000, 2024-03-21T10:35:56.606 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000000000 2024-03-21T10:35:56.607 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-21T10:35:56.607 INFO:teuthology.orchestra.run.smithi100.stdout: "cap_revoke_eviction": 0, 2024-03-21T10:35:56.607 INFO:teuthology.orchestra.run.smithi100.stdout: "cap_acquisition_throttle": 0, 2024-03-21T10:35:56.607 INFO:teuthology.orchestra.run.smithi100.stdout: "req_getvxattr_latency": { 2024-03-21T10:35:56.607 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 0, 2024-03-21T10:35:56.607 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.000000000, 2024-03-21T10:35:56.607 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000000000 2024-03-21T10:35:56.607 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-03-21T10:35:56.607 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-21T10:35:56.607 INFO:teuthology.orchestra.run.smithi100.stdout: "mds_sessions": { 2024-03-21T10:35:56.607 INFO:teuthology.orchestra.run.smithi100.stdout: "session_count": 2, 2024-03-21T10:35:56.608 INFO:teuthology.orchestra.run.smithi100.stdout: "session_add": 3, 2024-03-21T10:35:56.608 INFO:teuthology.orchestra.run.smithi100.stdout: "session_remove": 1, 2024-03-21T10:35:56.608 INFO:teuthology.orchestra.run.smithi100.stdout: "sessions_open": 2, 2024-03-21T10:35:56.608 INFO:teuthology.orchestra.run.smithi100.stdout: "sessions_stale": 0, 2024-03-21T10:35:56.608 INFO:teuthology.orchestra.run.smithi100.stdout: "total_load": 3, 2024-03-21T10:35:56.608 INFO:teuthology.orchestra.run.smithi100.stdout: "average_load": 1, 2024-03-21T10:35:56.608 INFO:teuthology.orchestra.run.smithi100.stdout: "avg_session_uptime": 627, 2024-03-21T10:35:56.608 INFO:teuthology.orchestra.run.smithi100.stdout: "mdthresh_evicted": 0 2024-03-21T10:35:56.608 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-21T10:35:56.608 INFO:teuthology.orchestra.run.smithi100.stdout: "mempool": { 2024-03-21T10:35:56.608 INFO:teuthology.orchestra.run.smithi100.stdout: "bloom_filter_bytes": 0, 2024-03-21T10:35:56.608 INFO:teuthology.orchestra.run.smithi100.stdout: "bloom_filter_items": 0, 2024-03-21T10:35:56.608 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_alloc_bytes": 0, 2024-03-21T10:35:56.609 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_alloc_items": 0, 2024-03-21T10:35:56.609 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_cache_data_bytes": 0, 2024-03-21T10:35:56.609 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_cache_data_items": 0, 2024-03-21T10:35:56.609 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_cache_onode_bytes": 0, 2024-03-21T10:35:56.609 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_cache_onode_items": 0, 2024-03-21T10:35:56.609 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_cache_meta_bytes": 0, 2024-03-21T10:35:56.609 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_cache_meta_items": 0, 2024-03-21T10:35:56.609 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_cache_other_bytes": 0, 2024-03-21T10:35:56.609 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_cache_other_items": 0, 2024-03-21T10:35:56.609 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_cache_buffer_bytes": 0, 2024-03-21T10:35:56.609 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_cache_buffer_items": 0, 2024-03-21T10:35:56.610 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_extent_bytes": 0, 2024-03-21T10:35:56.610 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_extent_items": 0, 2024-03-21T10:35:56.610 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_blob_bytes": 0, 2024-03-21T10:35:56.610 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_blob_items": 0, 2024-03-21T10:35:56.610 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_shared_blob_bytes": 0, 2024-03-21T10:35:56.610 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_shared_blob_items": 0, 2024-03-21T10:35:56.610 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_inline_bl_bytes": 0, 2024-03-21T10:35:56.610 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_inline_bl_items": 0, 2024-03-21T10:35:56.610 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_fsck_bytes": 0, 2024-03-21T10:35:56.610 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_fsck_items": 0, 2024-03-21T10:35:56.610 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_txc_bytes": 0, 2024-03-21T10:35:56.610 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_txc_items": 0, 2024-03-21T10:35:56.610 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_writing_deferred_bytes": 0, 2024-03-21T10:35:56.611 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_writing_deferred_items": 0, 2024-03-21T10:35:56.611 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_writing_bytes": 0, 2024-03-21T10:35:56.611 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_writing_items": 0, 2024-03-21T10:35:56.611 INFO:teuthology.orchestra.run.smithi100.stdout: "bluefs_bytes": 0, 2024-03-21T10:35:56.611 INFO:teuthology.orchestra.run.smithi100.stdout: "bluefs_items": 0, 2024-03-21T10:35:56.611 INFO:teuthology.orchestra.run.smithi100.stdout: "bluefs_file_reader_bytes": 0, 2024-03-21T10:35:56.611 INFO:teuthology.orchestra.run.smithi100.stdout: "bluefs_file_reader_items": 0, 2024-03-21T10:35:56.611 INFO:teuthology.orchestra.run.smithi100.stdout: "bluefs_file_writer_bytes": 0, 2024-03-21T10:35:56.611 INFO:teuthology.orchestra.run.smithi100.stdout: "bluefs_file_writer_items": 0, 2024-03-21T10:35:56.611 INFO:teuthology.orchestra.run.smithi100.stdout: "buffer_anon_bytes": 272518, 2024-03-21T10:35:56.611 INFO:teuthology.orchestra.run.smithi100.stdout: "buffer_anon_items": 102, 2024-03-21T10:35:56.611 INFO:teuthology.orchestra.run.smithi100.stdout: "buffer_meta_bytes": 88, 2024-03-21T10:35:56.611 INFO:teuthology.orchestra.run.smithi100.stdout: "buffer_meta_items": 1, 2024-03-21T10:35:56.612 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_bytes": 0, 2024-03-21T10:35:56.612 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_items": 0, 2024-03-21T10:35:56.612 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_mapbl_bytes": 0, 2024-03-21T10:35:56.612 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_mapbl_items": 0, 2024-03-21T10:35:56.612 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_pglog_bytes": 0, 2024-03-21T10:35:56.612 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_pglog_items": 0, 2024-03-21T10:35:56.612 INFO:teuthology.orchestra.run.smithi100.stdout: "osdmap_bytes": 9712, 2024-03-21T10:35:56.612 INFO:teuthology.orchestra.run.smithi100.stdout: "osdmap_items": 187, 2024-03-21T10:35:56.612 INFO:teuthology.orchestra.run.smithi100.stdout: "osdmap_mapping_bytes": 0, 2024-03-21T10:35:56.612 INFO:teuthology.orchestra.run.smithi100.stdout: "osdmap_mapping_items": 0, 2024-03-21T10:35:56.612 INFO:teuthology.orchestra.run.smithi100.stdout: "pgmap_bytes": 0, 2024-03-21T10:35:56.612 INFO:teuthology.orchestra.run.smithi100.stdout: "pgmap_items": 0, 2024-03-21T10:35:56.612 INFO:teuthology.orchestra.run.smithi100.stdout: "mds_co_bytes": 54928, 2024-03-21T10:35:56.613 INFO:teuthology.orchestra.run.smithi100.stdout: "mds_co_items": 218, 2024-03-21T10:35:56.613 INFO:teuthology.orchestra.run.smithi100.stdout: "unittest_1_bytes": 0, 2024-03-21T10:35:56.613 INFO:teuthology.orchestra.run.smithi100.stdout: "unittest_1_items": 0, 2024-03-21T10:35:56.613 INFO:teuthology.orchestra.run.smithi100.stdout: "unittest_2_bytes": 0, 2024-03-21T10:35:56.613 INFO:teuthology.orchestra.run.smithi100.stdout: "unittest_2_items": 0 2024-03-21T10:35:56.613 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-21T10:35:56.613 INFO:teuthology.orchestra.run.smithi100.stdout: "objecter": { 2024-03-21T10:35:56.613 INFO:teuthology.orchestra.run.smithi100.stdout: "op_active": 0, 2024-03-21T10:35:56.613 INFO:teuthology.orchestra.run.smithi100.stdout: "op_laggy": 0, 2024-03-21T10:35:56.613 INFO:teuthology.orchestra.run.smithi100.stdout: "op_send": 43, 2024-03-21T10:35:56.613 INFO:teuthology.orchestra.run.smithi100.stdout: "op_send_bytes": 12490, 2024-03-21T10:35:56.613 INFO:teuthology.orchestra.run.smithi100.stdout: "op_resend": 33, 2024-03-21T10:35:56.614 INFO:teuthology.orchestra.run.smithi100.stdout: "op_reply": 43, 2024-03-21T10:35:56.614 INFO:teuthology.orchestra.run.smithi100.stdout: "op_latency": { 2024-03-21T10:35:56.614 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 43, 2024-03-21T10:35:56.614 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 6.797821338, 2024-03-21T10:35:56.614 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.158088868 2024-03-21T10:35:56.614 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-21T10:35:56.614 INFO:teuthology.orchestra.run.smithi100.stdout: "op_inflight": 0, 2024-03-21T10:35:56.614 INFO:teuthology.orchestra.run.smithi100.stdout: "oplen_avg": { 2024-03-21T10:35:56.614 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 43, 2024-03-21T10:35:56.614 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 66 2024-03-21T10:35:56.615 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-21T10:35:56.615 INFO:teuthology.orchestra.run.smithi100.stdout: "op": 43, 2024-03-21T10:35:56.615 INFO:teuthology.orchestra.run.smithi100.stdout: "op_r": 0, 2024-03-21T10:35:56.615 INFO:teuthology.orchestra.run.smithi100.stdout: "op_w": 43, 2024-03-21T10:35:56.615 INFO:teuthology.orchestra.run.smithi100.stdout: "op_rmw": 0, 2024-03-21T10:35:56.615 INFO:teuthology.orchestra.run.smithi100.stdout: "op_pg": 0, 2024-03-21T10:35:56.615 INFO:teuthology.orchestra.run.smithi100.stdout: "osdop_stat": 0, 2024-03-21T10:35:56.615 INFO:teuthology.orchestra.run.smithi100.stdout: "osdop_create": 10, 2024-03-21T10:35:56.615 INFO:teuthology.orchestra.run.smithi100.stdout: "osdop_read": 0, 2024-03-21T10:35:56.615 INFO:teuthology.orchestra.run.smithi100.stdout: "osdop_write": 5, 2024-03-21T10:35:56.615 INFO:teuthology.orchestra.run.smithi100.stdout: "osdop_writefull": 8, 2024-03-21T10:35:56.615 INFO:teuthology.orchestra.run.smithi100.stdout: "osdop_writesame": 0, 2024-03-21T10:35:56.616 INFO:teuthology.orchestra.run.smithi100.stdout: "osdop_append": 0, 2024-03-21T10:35:56.616 INFO:teuthology.orchestra.run.smithi100.stdout: "osdop_zero": 0, 2024-03-21T10:35:56.616 INFO:teuthology.orchestra.run.smithi100.stdout: "osdop_truncate": 0, 2024-03-21T10:35:56.616 INFO:teuthology.orchestra.run.smithi100.stdout: "osdop_delete": 6, 2024-03-21T10:35:56.616 INFO:teuthology.orchestra.run.smithi100.stdout: "osdop_mapext": 0, 2024-03-21T10:35:56.616 INFO:teuthology.orchestra.run.smithi100.stdout: "osdop_sparse_read": 0, 2024-03-21T10:35:56.616 INFO:teuthology.orchestra.run.smithi100.stdout: "osdop_clonerange": 0, 2024-03-21T10:35:56.616 INFO:teuthology.orchestra.run.smithi100.stdout: "osdop_getxattr": 0, 2024-03-21T10:35:56.616 INFO:teuthology.orchestra.run.smithi100.stdout: "osdop_setxattr": 20, 2024-03-21T10:35:56.616 INFO:teuthology.orchestra.run.smithi100.stdout: "osdop_cmpxattr": 0, 2024-03-21T10:35:56.616 INFO:teuthology.orchestra.run.smithi100.stdout: "osdop_rmxattr": 0, 2024-03-21T10:35:56.617 INFO:teuthology.orchestra.run.smithi100.stdout: "osdop_resetxattrs": 0, 2024-03-21T10:35:56.617 INFO:teuthology.orchestra.run.smithi100.stdout: "osdop_call": 0, 2024-03-21T10:35:56.617 INFO:teuthology.orchestra.run.smithi100.stdout: "osdop_watch": 0, 2024-03-21T10:35:56.617 INFO:teuthology.orchestra.run.smithi100.stdout: "osdop_notify": 0, 2024-03-21T10:35:56.617 INFO:teuthology.orchestra.run.smithi100.stdout: "osdop_src_cmpxattr": 0, 2024-03-21T10:35:56.617 INFO:teuthology.orchestra.run.smithi100.stdout: "osdop_pgls": 0, 2024-03-21T10:35:56.617 INFO:teuthology.orchestra.run.smithi100.stdout: "osdop_pgls_filter": 0, 2024-03-21T10:35:56.617 INFO:teuthology.orchestra.run.smithi100.stdout: "osdop_other": 0, 2024-03-21T10:35:56.617 INFO:teuthology.orchestra.run.smithi100.stdout: "linger_active": 0, 2024-03-21T10:35:56.617 INFO:teuthology.orchestra.run.smithi100.stdout: "linger_send": 0, 2024-03-21T10:35:56.617 INFO:teuthology.orchestra.run.smithi100.stdout: "linger_resend": 0, 2024-03-21T10:35:56.617 INFO:teuthology.orchestra.run.smithi100.stdout: "linger_ping": 0, 2024-03-21T10:35:56.618 INFO:teuthology.orchestra.run.smithi100.stdout: "poolop_active": 0, 2024-03-21T10:35:56.618 INFO:teuthology.orchestra.run.smithi100.stdout: "poolop_send": 0, 2024-03-21T10:35:56.618 INFO:teuthology.orchestra.run.smithi100.stdout: "poolop_resend": 0, 2024-03-21T10:35:56.618 INFO:teuthology.orchestra.run.smithi100.stdout: "poolstat_active": 0, 2024-03-21T10:35:56.618 INFO:teuthology.orchestra.run.smithi100.stdout: "poolstat_send": 0, 2024-03-21T10:35:56.618 INFO:teuthology.orchestra.run.smithi100.stdout: "poolstat_resend": 0, 2024-03-21T10:35:56.618 INFO:teuthology.orchestra.run.smithi100.stdout: "statfs_active": 0, 2024-03-21T10:35:56.618 INFO:teuthology.orchestra.run.smithi100.stdout: "statfs_send": 0, 2024-03-21T10:35:56.618 INFO:teuthology.orchestra.run.smithi100.stdout: "statfs_resend": 0, 2024-03-21T10:35:56.618 INFO:teuthology.orchestra.run.smithi100.stdout: "command_active": 0, 2024-03-21T10:35:56.618 INFO:teuthology.orchestra.run.smithi100.stdout: "command_send": 0, 2024-03-21T10:35:56.618 INFO:teuthology.orchestra.run.smithi100.stdout: "command_resend": 0, 2024-03-21T10:35:56.619 INFO:teuthology.orchestra.run.smithi100.stdout: "map_epoch": 105, 2024-03-21T10:35:56.619 INFO:teuthology.orchestra.run.smithi100.stdout: "map_full": 0, 2024-03-21T10:35:56.619 INFO:teuthology.orchestra.run.smithi100.stdout: "map_inc": 13, 2024-03-21T10:35:56.619 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_sessions": 9, 2024-03-21T10:35:56.619 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_session_open": 9, 2024-03-21T10:35:56.619 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_session_close": 0, 2024-03-21T10:35:56.619 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_laggy": 0, 2024-03-21T10:35:56.619 INFO:teuthology.orchestra.run.smithi100.stdout: "omap_wr": 16, 2024-03-21T10:35:56.619 INFO:teuthology.orchestra.run.smithi100.stdout: "omap_rd": 0, 2024-03-21T10:35:56.619 INFO:teuthology.orchestra.run.smithi100.stdout: "omap_del": 1 2024-03-21T10:35:56.619 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-21T10:35:56.619 INFO:teuthology.orchestra.run.smithi100.stdout: "oft": { 2024-03-21T10:35:56.619 INFO:teuthology.orchestra.run.smithi100.stdout: "omap_total_objs": 1, 2024-03-21T10:35:56.620 INFO:teuthology.orchestra.run.smithi100.stdout: "omap_total_kv_pairs": 0, 2024-03-21T10:35:56.620 INFO:teuthology.orchestra.run.smithi100.stdout: "omap_total_updates": 0, 2024-03-21T10:35:56.620 INFO:teuthology.orchestra.run.smithi100.stdout: "omap_total_removes": 0 2024-03-21T10:35:56.620 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-21T10:35:56.620 INFO:teuthology.orchestra.run.smithi100.stdout: "osd-slow-ops": { 2024-03-21T10:35:56.620 INFO:teuthology.orchestra.run.smithi100.stdout: "slow_ops_count": 0 2024-03-21T10:35:56.620 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-21T10:35:56.620 INFO:teuthology.orchestra.run.smithi100.stdout: "purge_queue": { 2024-03-21T10:35:56.620 INFO:teuthology.orchestra.run.smithi100.stdout: "pq_executing_ops": 0, 2024-03-21T10:35:56.620 INFO:teuthology.orchestra.run.smithi100.stdout: "pq_executing_ops_high_water": 0, 2024-03-21T10:35:56.620 INFO:teuthology.orchestra.run.smithi100.stdout: "pq_executing": 0, 2024-03-21T10:35:56.620 INFO:teuthology.orchestra.run.smithi100.stdout: "pq_executing_high_water": 0, 2024-03-21T10:35:56.621 INFO:teuthology.orchestra.run.smithi100.stdout: "pq_executed_ops": 0, 2024-03-21T10:35:56.621 INFO:teuthology.orchestra.run.smithi100.stdout: "pq_executed": 0, 2024-03-21T10:35:56.621 INFO:teuthology.orchestra.run.smithi100.stdout: "pq_item_in_journal": 0 2024-03-21T10:35:56.621 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-21T10:35:56.621 INFO:teuthology.orchestra.run.smithi100.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-03-21T10:35:56.621 INFO:teuthology.orchestra.run.smithi100.stdout: "val": 0, 2024-03-21T10:35:56.621 INFO:teuthology.orchestra.run.smithi100.stdout: "max": 104857600, 2024-03-21T10:35:56.621 INFO:teuthology.orchestra.run.smithi100.stdout: "get_started": 0, 2024-03-21T10:35:56.621 INFO:teuthology.orchestra.run.smithi100.stdout: "get": 988, 2024-03-21T10:35:56.621 INFO:teuthology.orchestra.run.smithi100.stdout: "get_sum": 1921835, 2024-03-21T10:35:56.621 INFO:teuthology.orchestra.run.smithi100.stdout: "get_or_fail_fail": 0, 2024-03-21T10:35:56.622 INFO:teuthology.orchestra.run.smithi100.stdout: "get_or_fail_success": 988, 2024-03-21T10:35:56.622 INFO:teuthology.orchestra.run.smithi100.stdout: "take": 0, 2024-03-21T10:35:56.622 INFO:teuthology.orchestra.run.smithi100.stdout: "take_sum": 0, 2024-03-21T10:35:56.622 INFO:teuthology.orchestra.run.smithi100.stdout: "put": 988, 2024-03-21T10:35:56.622 INFO:teuthology.orchestra.run.smithi100.stdout: "put_sum": 1921835, 2024-03-21T10:35:56.622 INFO:teuthology.orchestra.run.smithi100.stdout: "wait": { 2024-03-21T10:35:56.622 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 0, 2024-03-21T10:35:56.622 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.000000000, 2024-03-21T10:35:56.622 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000000000 2024-03-21T10:35:56.622 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-03-21T10:35:56.622 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-21T10:35:56.622 INFO:teuthology.orchestra.run.smithi100.stdout: "throttle-objecter_bytes": { 2024-03-21T10:35:56.623 INFO:teuthology.orchestra.run.smithi100.stdout: "val": 0, 2024-03-21T10:35:56.623 INFO:teuthology.orchestra.run.smithi100.stdout: "max": 104857600, 2024-03-21T10:35:56.623 INFO:teuthology.orchestra.run.smithi100.stdout: "get_started": 0, 2024-03-21T10:35:56.623 INFO:teuthology.orchestra.run.smithi100.stdout: "get": 0, 2024-03-21T10:35:56.623 INFO:teuthology.orchestra.run.smithi100.stdout: "get_sum": 0, 2024-03-21T10:35:56.623 INFO:teuthology.orchestra.run.smithi100.stdout: "get_or_fail_fail": 0, 2024-03-21T10:35:56.623 INFO:teuthology.orchestra.run.smithi100.stdout: "get_or_fail_success": 0, 2024-03-21T10:35:56.623 INFO:teuthology.orchestra.run.smithi100.stdout: "take": 43, 2024-03-21T10:35:56.623 INFO:teuthology.orchestra.run.smithi100.stdout: "take_sum": 12490, 2024-03-21T10:35:56.623 INFO:teuthology.orchestra.run.smithi100.stdout: "put": 43, 2024-03-21T10:35:56.623 INFO:teuthology.orchestra.run.smithi100.stdout: "put_sum": 12490, 2024-03-21T10:35:56.623 INFO:teuthology.orchestra.run.smithi100.stdout: "wait": { 2024-03-21T10:35:56.624 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 0, 2024-03-21T10:35:56.624 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.000000000, 2024-03-21T10:35:56.624 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000000000 2024-03-21T10:35:56.624 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-03-21T10:35:56.624 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-21T10:35:56.624 INFO:teuthology.orchestra.run.smithi100.stdout: "throttle-objecter_ops": { 2024-03-21T10:35:56.624 INFO:teuthology.orchestra.run.smithi100.stdout: "val": 0, 2024-03-21T10:35:56.624 INFO:teuthology.orchestra.run.smithi100.stdout: "max": 1024, 2024-03-21T10:35:56.624 INFO:teuthology.orchestra.run.smithi100.stdout: "get_started": 0, 2024-03-21T10:35:56.624 INFO:teuthology.orchestra.run.smithi100.stdout: "get": 0, 2024-03-21T10:35:56.624 INFO:teuthology.orchestra.run.smithi100.stdout: "get_sum": 0, 2024-03-21T10:35:56.624 INFO:teuthology.orchestra.run.smithi100.stdout: "get_or_fail_fail": 0, 2024-03-21T10:35:56.625 INFO:teuthology.orchestra.run.smithi100.stdout: "get_or_fail_success": 0, 2024-03-21T10:35:56.625 INFO:teuthology.orchestra.run.smithi100.stdout: "take": 43, 2024-03-21T10:35:56.625 INFO:teuthology.orchestra.run.smithi100.stdout: "take_sum": 43, 2024-03-21T10:35:56.625 INFO:teuthology.orchestra.run.smithi100.stdout: "put": 43, 2024-03-21T10:35:56.625 INFO:teuthology.orchestra.run.smithi100.stdout: "put_sum": 43, 2024-03-21T10:35:56.625 INFO:teuthology.orchestra.run.smithi100.stdout: "wait": { 2024-03-21T10:35:56.625 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 0, 2024-03-21T10:35:56.625 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.000000000, 2024-03-21T10:35:56.625 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000000000 2024-03-21T10:35:56.625 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-03-21T10:35:56.625 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-21T10:35:56.625 INFO:teuthology.orchestra.run.smithi100.stdout: "throttle-write_buf_throttle": { 2024-03-21T10:35:56.626 INFO:teuthology.orchestra.run.smithi100.stdout: "val": 0, 2024-03-21T10:35:56.626 INFO:teuthology.orchestra.run.smithi100.stdout: "max": 3758096384, 2024-03-21T10:35:56.626 INFO:teuthology.orchestra.run.smithi100.stdout: "get_started": 0, 2024-03-21T10:35:56.626 INFO:teuthology.orchestra.run.smithi100.stdout: "get": 0, 2024-03-21T10:35:56.626 INFO:teuthology.orchestra.run.smithi100.stdout: "get_sum": 0, 2024-03-21T10:35:56.626 INFO:teuthology.orchestra.run.smithi100.stdout: "get_or_fail_fail": 0, 2024-03-21T10:35:56.626 INFO:teuthology.orchestra.run.smithi100.stdout: "get_or_fail_success": 0, 2024-03-21T10:35:56.626 INFO:teuthology.orchestra.run.smithi100.stdout: "take": 0, 2024-03-21T10:35:56.626 INFO:teuthology.orchestra.run.smithi100.stdout: "take_sum": 0, 2024-03-21T10:35:56.626 INFO:teuthology.orchestra.run.smithi100.stdout: "put": 0, 2024-03-21T10:35:56.626 INFO:teuthology.orchestra.run.smithi100.stdout: "put_sum": 0, 2024-03-21T10:35:56.626 INFO:teuthology.orchestra.run.smithi100.stdout: "wait": { 2024-03-21T10:35:56.627 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 0, 2024-03-21T10:35:56.627 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.000000000, 2024-03-21T10:35:56.627 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000000000 2024-03-21T10:35:56.627 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-03-21T10:35:56.627 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-21T10:35:56.627 INFO:teuthology.orchestra.run.smithi100.stdout: "throttle-write_buf_throttle-0x55862850a500": { 2024-03-21T10:35:56.627 INFO:teuthology.orchestra.run.smithi100.stdout: "val": 0, 2024-03-21T10:35:56.627 INFO:teuthology.orchestra.run.smithi100.stdout: "max": 3758096384, 2024-03-21T10:35:56.627 INFO:teuthology.orchestra.run.smithi100.stdout: "get_started": 0, 2024-03-21T10:35:56.627 INFO:teuthology.orchestra.run.smithi100.stdout: "get": 6, 2024-03-21T10:35:56.627 INFO:teuthology.orchestra.run.smithi100.stdout: "get_sum": 2004, 2024-03-21T10:35:56.627 INFO:teuthology.orchestra.run.smithi100.stdout: "get_or_fail_fail": 0, 2024-03-21T10:35:56.627 INFO:teuthology.orchestra.run.smithi100.stdout: "get_or_fail_success": 6, 2024-03-21T10:35:56.627 INFO:teuthology.orchestra.run.smithi100.stdout: "take": 0, 2024-03-21T10:35:56.628 INFO:teuthology.orchestra.run.smithi100.stdout: "take_sum": 0, 2024-03-21T10:35:56.628 INFO:teuthology.orchestra.run.smithi100.stdout: "put": 5, 2024-03-21T10:35:56.628 INFO:teuthology.orchestra.run.smithi100.stdout: "put_sum": 2004, 2024-03-21T10:35:56.628 INFO:teuthology.orchestra.run.smithi100.stdout: "wait": { 2024-03-21T10:35:56.628 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 0, 2024-03-21T10:35:56.628 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.000000000, 2024-03-21T10:35:56.628 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000000000 2024-03-21T10:35:56.628 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-03-21T10:35:56.628 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-03-21T10:35:56.628 INFO:teuthology.orchestra.run.smithi100.stdout:} 2024-03-21T10:35:56.629 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:35:56 smithi100 ceph-mon[35193]: Adjusting osd_memory_target on smithi100 to 176.7M 2024-03-21T10:35:56.629 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:35:56 smithi100 ceph-mon[35193]: Unable to set osd_memory_target on smithi100 to 185329356: error parsing value: Value '185329356' is below minimum 939524096 2024-03-21T10:35:56.961 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:35:56 smithi047 ceph-mon[31547]: Adjusting osd_memory_target on smithi100 to 176.7M 2024-03-21T10:35:56.961 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:35:56 smithi047 ceph-mon[31547]: Unable to set osd_memory_target on smithi100 to 185329356: error parsing value: Value '185329356' is below minimum 939524096 2024-03-21T10:35:56.986 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:35:56 smithi157 ceph-mon[35581]: Adjusting osd_memory_target on smithi100 to 176.7M 2024-03-21T10:35:56.986 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:35:56 smithi157 ceph-mon[35581]: Unable to set osd_memory_target on smithi100 to 185329356: error parsing value: Value '185329356' is below minimum 939524096 2024-03-21T10:35:57.474 INFO:tasks.check_counter:Daemon mds.e mds.exported=0 2024-03-21T10:35:57.474 INFO:tasks.check_counter:Daemon mds.e mds.imported=0 2024-03-21T10:35:57.474 INFO:tasks.check_counter:Daemon mds.e mds.root_rsnaps=6 2024-03-21T10:35:57.474 INFO:tasks.check_counter:Daemon mds.e mds_server.req_mksnap_latency.avgcount=0 2024-03-21T10:35:57.474 INFO:tasks.check_counter:Daemon mds.e mds_server.req_rmsnap_latency.avgcount=0 2024-03-21T10:35:57.474 DEBUG:tasks.check_counter:Getting stats from h 2024-03-21T10:35:57.475 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2edfee68ad7243bed422a049b6f5f58619300f11 shell --fsid 3056d51a-e76b-11ee-95cd-87774f69a715 -- ceph daemon mds.h perf dump 2024-03-21T10:35:57.726 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:35:57 smithi100 ceph-mon[35193]: pgmap v330: 129 pgs: 129 active+clean; 2.5 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 12 op/s 2024-03-21T10:35:57.961 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:35:57 smithi047 ceph-mon[31547]: pgmap v330: 129 pgs: 129 active+clean; 2.5 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 12 op/s 2024-03-21T10:35:57.986 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:35:57 smithi157 ceph-mon[35581]: pgmap v330: 129 pgs: 129 active+clean; 2.5 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 12 op/s 2024-03-21T10:35:57.998 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/3056d51a-e76b-11ee-95cd-87774f69a715/mon.b/config 2024-03-21T10:35:59.639 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:35:59 smithi100 ceph-mon[35193]: pgmap v331: 129 pgs: 129 active+clean; 2.5 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-03-21T10:35:59.960 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:35:59 smithi047 ceph-mon[31547]: pgmap v331: 129 pgs: 129 active+clean; 2.5 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-03-21T10:35:59.986 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:35:59 smithi157 ceph-mon[35581]: pgmap v331: 129 pgs: 129 active+clean; 2.5 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-03-21T10:36:00.269 INFO:teuthology.orchestra.run.smithi100.stdout:{ 2024-03-21T10:36:00.270 INFO:teuthology.orchestra.run.smithi100.stdout: "AsyncMessenger::Worker-0": { 2024-03-21T10:36:00.270 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_recv_messages": 95, 2024-03-21T10:36:00.270 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_send_messages": 129, 2024-03-21T10:36:00.270 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_recv_bytes": 36256, 2024-03-21T10:36:00.270 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_send_bytes": 157115, 2024-03-21T10:36:00.271 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_created_connections": 21, 2024-03-21T10:36:00.271 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_active_connections": 4, 2024-03-21T10:36:00.271 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_running_total_time": 0.038399907, 2024-03-21T10:36:00.271 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_running_send_time": 0.009154480, 2024-03-21T10:36:00.271 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_running_recv_time": 0.017715854, 2024-03-21T10:36:00.271 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_running_fast_dispatch_time": 0.000660305, 2024-03-21T10:36:00.272 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_send_messages_queue_lat": { 2024-03-21T10:36:00.272 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 123, 2024-03-21T10:36:00.272 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.054652090, 2024-03-21T10:36:00.272 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000444325 2024-03-21T10:36:00.272 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-21T10:36:00.272 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_handle_ack_lat": { 2024-03-21T10:36:00.273 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 134, 2024-03-21T10:36:00.273 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.000036897, 2024-03-21T10:36:00.273 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000000275 2024-03-21T10:36:00.273 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-21T10:36:00.273 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_recv_encrypted_bytes": 0, 2024-03-21T10:36:00.273 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_send_encrypted_bytes": 0 2024-03-21T10:36:00.274 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-21T10:36:00.274 INFO:teuthology.orchestra.run.smithi100.stdout: "AsyncMessenger::Worker-1": { 2024-03-21T10:36:00.274 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_recv_messages": 547, 2024-03-21T10:36:00.274 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_send_messages": 761, 2024-03-21T10:36:00.274 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_recv_bytes": 165147, 2024-03-21T10:36:00.274 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_send_bytes": 755655, 2024-03-21T10:36:00.275 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_created_connections": 45, 2024-03-21T10:36:00.275 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_active_connections": 5, 2024-03-21T10:36:00.275 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_running_total_time": 0.163547526, 2024-03-21T10:36:00.275 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_running_send_time": 0.078326036, 2024-03-21T10:36:00.275 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_running_recv_time": 0.075824129, 2024-03-21T10:36:00.275 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_running_fast_dispatch_time": 0.005143834, 2024-03-21T10:36:00.276 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_send_messages_queue_lat": { 2024-03-21T10:36:00.276 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 763, 2024-03-21T10:36:00.276 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.122933169, 2024-03-21T10:36:00.276 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000161118 2024-03-21T10:36:00.276 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-21T10:36:00.276 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_handle_ack_lat": { 2024-03-21T10:36:00.277 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 1164, 2024-03-21T10:36:00.277 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.000614148, 2024-03-21T10:36:00.277 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000000527 2024-03-21T10:36:00.277 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-21T10:36:00.277 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_recv_encrypted_bytes": 5280, 2024-03-21T10:36:00.277 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_send_encrypted_bytes": 119280 2024-03-21T10:36:00.277 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-21T10:36:00.277 INFO:teuthology.orchestra.run.smithi100.stdout: "AsyncMessenger::Worker-2": { 2024-03-21T10:36:00.278 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_recv_messages": 367, 2024-03-21T10:36:00.278 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_send_messages": 471, 2024-03-21T10:36:00.278 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_recv_bytes": 1835913, 2024-03-21T10:36:00.278 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_send_bytes": 466856, 2024-03-21T10:36:00.278 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_created_connections": 8, 2024-03-21T10:36:00.278 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_active_connections": 6, 2024-03-21T10:36:00.278 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_running_total_time": 0.102693119, 2024-03-21T10:36:00.279 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_running_send_time": 0.042049452, 2024-03-21T10:36:00.279 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_running_recv_time": 0.067914461, 2024-03-21T10:36:00.279 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_running_fast_dispatch_time": 0.008451500, 2024-03-21T10:36:00.279 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_send_messages_queue_lat": { 2024-03-21T10:36:00.279 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 470, 2024-03-21T10:36:00.279 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.074604587, 2024-03-21T10:36:00.279 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000158733 2024-03-21T10:36:00.280 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-21T10:36:00.280 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_handle_ack_lat": { 2024-03-21T10:36:00.280 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 79, 2024-03-21T10:36:00.280 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.000021655, 2024-03-21T10:36:00.280 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000000274 2024-03-21T10:36:00.280 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-21T10:36:00.280 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_recv_encrypted_bytes": 1805968, 2024-03-21T10:36:00.280 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_send_encrypted_bytes": 429536 2024-03-21T10:36:00.280 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-21T10:36:00.280 INFO:teuthology.orchestra.run.smithi100.stdout: "cct": { 2024-03-21T10:36:00.280 INFO:teuthology.orchestra.run.smithi100.stdout: "total_workers": 1, 2024-03-21T10:36:00.281 INFO:teuthology.orchestra.run.smithi100.stdout: "unhealthy_workers": 0 2024-03-21T10:36:00.281 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-21T10:36:00.281 INFO:teuthology.orchestra.run.smithi100.stdout: "finisher-MDSRank": { 2024-03-21T10:36:00.281 INFO:teuthology.orchestra.run.smithi100.stdout: "queue_len": 0, 2024-03-21T10:36:00.281 INFO:teuthology.orchestra.run.smithi100.stdout: "complete_latency": { 2024-03-21T10:36:00.281 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 60, 2024-03-21T10:36:00.281 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.002316396, 2024-03-21T10:36:00.281 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000038606 2024-03-21T10:36:00.281 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-03-21T10:36:00.281 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-21T10:36:00.281 INFO:teuthology.orchestra.run.smithi100.stdout: "finisher-PurgeQueue": { 2024-03-21T10:36:00.281 INFO:teuthology.orchestra.run.smithi100.stdout: "queue_len": 0, 2024-03-21T10:36:00.281 INFO:teuthology.orchestra.run.smithi100.stdout: "complete_latency": { 2024-03-21T10:36:00.281 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 2, 2024-03-21T10:36:00.281 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.000010585, 2024-03-21T10:36:00.282 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000005292 2024-03-21T10:36:00.282 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-03-21T10:36:00.282 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-21T10:36:00.282 INFO:teuthology.orchestra.run.smithi100.stdout: "mds": { 2024-03-21T10:36:00.282 INFO:teuthology.orchestra.run.smithi100.stdout: "request": 6, 2024-03-21T10:36:00.282 INFO:teuthology.orchestra.run.smithi100.stdout: "reply": 5, 2024-03-21T10:36:00.282 INFO:teuthology.orchestra.run.smithi100.stdout: "reply_latency": { 2024-03-21T10:36:00.282 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 5, 2024-03-21T10:36:00.282 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.005558525, 2024-03-21T10:36:00.282 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.001111705 2024-03-21T10:36:00.282 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-21T10:36:00.282 INFO:teuthology.orchestra.run.smithi100.stdout: "slow_reply": 0, 2024-03-21T10:36:00.282 INFO:teuthology.orchestra.run.smithi100.stdout: "forward": 1, 2024-03-21T10:36:00.283 INFO:teuthology.orchestra.run.smithi100.stdout: "dir_fetch_complete": 0, 2024-03-21T10:36:00.283 INFO:teuthology.orchestra.run.smithi100.stdout: "dir_fetch_keys": 0, 2024-03-21T10:36:00.283 INFO:teuthology.orchestra.run.smithi100.stdout: "dir_commit": 11, 2024-03-21T10:36:00.283 INFO:teuthology.orchestra.run.smithi100.stdout: "dir_split": 0, 2024-03-21T10:36:00.283 INFO:teuthology.orchestra.run.smithi100.stdout: "dir_merge": 0, 2024-03-21T10:36:00.283 INFO:teuthology.orchestra.run.smithi100.stdout: "inodes": 12, 2024-03-21T10:36:00.283 INFO:teuthology.orchestra.run.smithi100.stdout: "inodes_top": 0, 2024-03-21T10:36:00.283 INFO:teuthology.orchestra.run.smithi100.stdout: "inodes_bottom": 12, 2024-03-21T10:36:00.283 INFO:teuthology.orchestra.run.smithi100.stdout: "inodes_pin_tail": 0, 2024-03-21T10:36:00.283 INFO:teuthology.orchestra.run.smithi100.stdout: "inodes_pinned": 12, 2024-03-21T10:36:00.283 INFO:teuthology.orchestra.run.smithi100.stdout: "inodes_expired": 1, 2024-03-21T10:36:00.283 INFO:teuthology.orchestra.run.smithi100.stdout: "inodes_with_caps": 2, 2024-03-21T10:36:00.283 INFO:teuthology.orchestra.run.smithi100.stdout: "caps": 2, 2024-03-21T10:36:00.283 INFO:teuthology.orchestra.run.smithi100.stdout: "subtrees": 2, 2024-03-21T10:36:00.283 INFO:teuthology.orchestra.run.smithi100.stdout: "traverse": 12, 2024-03-21T10:36:00.284 INFO:teuthology.orchestra.run.smithi100.stdout: "traverse_hit": 7, 2024-03-21T10:36:00.284 INFO:teuthology.orchestra.run.smithi100.stdout: "traverse_forward": 1, 2024-03-21T10:36:00.284 INFO:teuthology.orchestra.run.smithi100.stdout: "traverse_discover": 2, 2024-03-21T10:36:00.284 INFO:teuthology.orchestra.run.smithi100.stdout: "traverse_dir_fetch": 0, 2024-03-21T10:36:00.284 INFO:teuthology.orchestra.run.smithi100.stdout: "traverse_remote_ino": 0, 2024-03-21T10:36:00.284 INFO:teuthology.orchestra.run.smithi100.stdout: "traverse_lock": 0, 2024-03-21T10:36:00.284 INFO:teuthology.orchestra.run.smithi100.stdout: "load_cent": 0, 2024-03-21T10:36:00.284 INFO:teuthology.orchestra.run.smithi100.stdout: "q": 0, 2024-03-21T10:36:00.284 INFO:teuthology.orchestra.run.smithi100.stdout: "exported": 0, 2024-03-21T10:36:00.284 INFO:teuthology.orchestra.run.smithi100.stdout: "exported_inodes": 0, 2024-03-21T10:36:00.284 INFO:teuthology.orchestra.run.smithi100.stdout: "imported": 0, 2024-03-21T10:36:00.284 INFO:teuthology.orchestra.run.smithi100.stdout: "imported_inodes": 0, 2024-03-21T10:36:00.284 INFO:teuthology.orchestra.run.smithi100.stdout: "openino_dir_fetch": 0, 2024-03-21T10:36:00.284 INFO:teuthology.orchestra.run.smithi100.stdout: "openino_backtrace_fetch": 0, 2024-03-21T10:36:00.284 INFO:teuthology.orchestra.run.smithi100.stdout: "openino_peer_discover": 0, 2024-03-21T10:36:00.285 INFO:teuthology.orchestra.run.smithi100.stdout: "root_rfiles": 4, 2024-03-21T10:36:00.285 INFO:teuthology.orchestra.run.smithi100.stdout: "root_rbytes": 234, 2024-03-21T10:36:00.285 INFO:teuthology.orchestra.run.smithi100.stdout: "root_rsnaps": 6, 2024-03-21T10:36:00.285 INFO:teuthology.orchestra.run.smithi100.stdout: "scrub_backtrace_fetch": 0, 2024-03-21T10:36:00.285 INFO:teuthology.orchestra.run.smithi100.stdout: "scrub_set_tag": 0, 2024-03-21T10:36:00.285 INFO:teuthology.orchestra.run.smithi100.stdout: "scrub_backtrace_repaired": 0, 2024-03-21T10:36:00.285 INFO:teuthology.orchestra.run.smithi100.stdout: "scrub_inotable_repaired": 0, 2024-03-21T10:36:00.285 INFO:teuthology.orchestra.run.smithi100.stdout: "scrub_dir_inodes": 0, 2024-03-21T10:36:00.285 INFO:teuthology.orchestra.run.smithi100.stdout: "scrub_dir_base_inodes": 0, 2024-03-21T10:36:00.285 INFO:teuthology.orchestra.run.smithi100.stdout: "scrub_dirfrag_rstats": 0, 2024-03-21T10:36:00.285 INFO:teuthology.orchestra.run.smithi100.stdout: "scrub_file_inodes": 0, 2024-03-21T10:36:00.285 INFO:teuthology.orchestra.run.smithi100.stdout: "handle_inode_file_caps": 0, 2024-03-21T10:36:00.285 INFO:teuthology.orchestra.run.smithi100.stdout: "ceph_cap_op_revoke": 0, 2024-03-21T10:36:00.285 INFO:teuthology.orchestra.run.smithi100.stdout: "ceph_cap_op_grant": 0, 2024-03-21T10:36:00.285 INFO:teuthology.orchestra.run.smithi100.stdout: "ceph_cap_op_trunc": 0, 2024-03-21T10:36:00.286 INFO:teuthology.orchestra.run.smithi100.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-03-21T10:36:00.286 INFO:teuthology.orchestra.run.smithi100.stdout: "ceph_cap_op_flush_ack": 0, 2024-03-21T10:36:00.286 INFO:teuthology.orchestra.run.smithi100.stdout: "handle_client_caps": 0, 2024-03-21T10:36:00.286 INFO:teuthology.orchestra.run.smithi100.stdout: "handle_client_caps_dirty": 0, 2024-03-21T10:36:00.286 INFO:teuthology.orchestra.run.smithi100.stdout: "handle_client_cap_release": 0, 2024-03-21T10:36:00.286 INFO:teuthology.orchestra.run.smithi100.stdout: "process_request_cap_release": 0 2024-03-21T10:36:00.286 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-21T10:36:00.286 INFO:teuthology.orchestra.run.smithi100.stdout: "mds_cache": { 2024-03-21T10:36:00.286 INFO:teuthology.orchestra.run.smithi100.stdout: "dir_update": 0, 2024-03-21T10:36:00.286 INFO:teuthology.orchestra.run.smithi100.stdout: "dir_update_receipt": 0, 2024-03-21T10:36:00.286 INFO:teuthology.orchestra.run.smithi100.stdout: "dir_try_discover": 0, 2024-03-21T10:36:00.286 INFO:teuthology.orchestra.run.smithi100.stdout: "dir_send_discover": 3, 2024-03-21T10:36:00.286 INFO:teuthology.orchestra.run.smithi100.stdout: "dir_handle_discover": 0, 2024-03-21T10:36:00.286 INFO:teuthology.orchestra.run.smithi100.stdout: "num_strays": 0, 2024-03-21T10:36:00.287 INFO:teuthology.orchestra.run.smithi100.stdout: "num_strays_delayed": 0, 2024-03-21T10:36:00.287 INFO:teuthology.orchestra.run.smithi100.stdout: "num_strays_enqueuing": 0, 2024-03-21T10:36:00.287 INFO:teuthology.orchestra.run.smithi100.stdout: "strays_created": 0, 2024-03-21T10:36:00.287 INFO:teuthology.orchestra.run.smithi100.stdout: "strays_enqueued": 0, 2024-03-21T10:36:00.287 INFO:teuthology.orchestra.run.smithi100.stdout: "strays_reintegrated": 0, 2024-03-21T10:36:00.287 INFO:teuthology.orchestra.run.smithi100.stdout: "strays_migrated": 0, 2024-03-21T10:36:00.287 INFO:teuthology.orchestra.run.smithi100.stdout: "num_recovering_processing": 0, 2024-03-21T10:36:00.287 INFO:teuthology.orchestra.run.smithi100.stdout: "num_recovering_enqueued": 0, 2024-03-21T10:36:00.287 INFO:teuthology.orchestra.run.smithi100.stdout: "num_recovering_prioritized": 0, 2024-03-21T10:36:00.287 INFO:teuthology.orchestra.run.smithi100.stdout: "recovery_started": 0, 2024-03-21T10:36:00.287 INFO:teuthology.orchestra.run.smithi100.stdout: "recovery_completed": 0, 2024-03-21T10:36:00.287 INFO:teuthology.orchestra.run.smithi100.stdout: "ireq_enqueue_scrub": 0, 2024-03-21T10:36:00.287 INFO:teuthology.orchestra.run.smithi100.stdout: "ireq_exportdir": 0, 2024-03-21T10:36:00.287 INFO:teuthology.orchestra.run.smithi100.stdout: "ireq_flush": 0, 2024-03-21T10:36:00.287 INFO:teuthology.orchestra.run.smithi100.stdout: "ireq_fragmentdir": 0, 2024-03-21T10:36:00.287 INFO:teuthology.orchestra.run.smithi100.stdout: "ireq_fragstats": 0, 2024-03-21T10:36:00.288 INFO:teuthology.orchestra.run.smithi100.stdout: "ireq_inodestats": 0 2024-03-21T10:36:00.288 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-21T10:36:00.288 INFO:teuthology.orchestra.run.smithi100.stdout: "mds_log": { 2024-03-21T10:36:00.288 INFO:teuthology.orchestra.run.smithi100.stdout: "evlrg": 0, 2024-03-21T10:36:00.288 INFO:teuthology.orchestra.run.smithi100.stdout: "evadd": 5, 2024-03-21T10:36:00.288 INFO:teuthology.orchestra.run.smithi100.stdout: "evex": 0, 2024-03-21T10:36:00.288 INFO:teuthology.orchestra.run.smithi100.stdout: "evtrm": 0, 2024-03-21T10:36:00.288 INFO:teuthology.orchestra.run.smithi100.stdout: "ev": 5, 2024-03-21T10:36:00.288 INFO:teuthology.orchestra.run.smithi100.stdout: "evexg": 0, 2024-03-21T10:36:00.288 INFO:teuthology.orchestra.run.smithi100.stdout: "evexd": 0, 2024-03-21T10:36:00.288 INFO:teuthology.orchestra.run.smithi100.stdout: "segadd": 2, 2024-03-21T10:36:00.288 INFO:teuthology.orchestra.run.smithi100.stdout: "segex": 0, 2024-03-21T10:36:00.288 INFO:teuthology.orchestra.run.smithi100.stdout: "segtrm": 0, 2024-03-21T10:36:00.288 INFO:teuthology.orchestra.run.smithi100.stdout: "seg": 2, 2024-03-21T10:36:00.288 INFO:teuthology.orchestra.run.smithi100.stdout: "segmjr": 2, 2024-03-21T10:36:00.289 INFO:teuthology.orchestra.run.smithi100.stdout: "segexg": 0, 2024-03-21T10:36:00.289 INFO:teuthology.orchestra.run.smithi100.stdout: "segexd": 0, 2024-03-21T10:36:00.289 INFO:teuthology.orchestra.run.smithi100.stdout: "expos": 4194304, 2024-03-21T10:36:00.289 INFO:teuthology.orchestra.run.smithi100.stdout: "wrpos": 4195917, 2024-03-21T10:36:00.289 INFO:teuthology.orchestra.run.smithi100.stdout: "rdpos": 0, 2024-03-21T10:36:00.289 INFO:teuthology.orchestra.run.smithi100.stdout: "jlat": { 2024-03-21T10:36:00.289 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 4, 2024-03-21T10:36:00.289 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.015094163, 2024-03-21T10:36:00.289 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.003773540 2024-03-21T10:36:00.289 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-21T10:36:00.289 INFO:teuthology.orchestra.run.smithi100.stdout: "replayed": 0 2024-03-21T10:36:00.289 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-21T10:36:00.290 INFO:teuthology.orchestra.run.smithi100.stdout: "mds_mem": { 2024-03-21T10:36:00.290 INFO:teuthology.orchestra.run.smithi100.stdout: "ino": 15, 2024-03-21T10:36:00.290 INFO:teuthology.orchestra.run.smithi100.stdout: "ino+": 15, 2024-03-21T10:36:00.290 INFO:teuthology.orchestra.run.smithi100.stdout: "ino-": 0, 2024-03-21T10:36:00.290 INFO:teuthology.orchestra.run.smithi100.stdout: "dir": 13, 2024-03-21T10:36:00.290 INFO:teuthology.orchestra.run.smithi100.stdout: "dir+": 14, 2024-03-21T10:36:00.290 INFO:teuthology.orchestra.run.smithi100.stdout: "dir-": 1, 2024-03-21T10:36:00.290 INFO:teuthology.orchestra.run.smithi100.stdout: "dn": 12, 2024-03-21T10:36:00.290 INFO:teuthology.orchestra.run.smithi100.stdout: "dn+": 13, 2024-03-21T10:36:00.290 INFO:teuthology.orchestra.run.smithi100.stdout: "dn-": 1, 2024-03-21T10:36:00.290 INFO:teuthology.orchestra.run.smithi100.stdout: "cap": 2, 2024-03-21T10:36:00.290 INFO:teuthology.orchestra.run.smithi100.stdout: "cap+": 4, 2024-03-21T10:36:00.290 INFO:teuthology.orchestra.run.smithi100.stdout: "cap-": 2, 2024-03-21T10:36:00.291 INFO:teuthology.orchestra.run.smithi100.stdout: "rss": 51456, 2024-03-21T10:36:00.291 INFO:teuthology.orchestra.run.smithi100.stdout: "heap": 223612 2024-03-21T10:36:00.291 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-21T10:36:00.291 INFO:teuthology.orchestra.run.smithi100.stdout: "mds_server": { 2024-03-21T10:36:00.291 INFO:teuthology.orchestra.run.smithi100.stdout: "dispatch_client_request": 11, 2024-03-21T10:36:00.291 INFO:teuthology.orchestra.run.smithi100.stdout: "dispatch_server_request": 0, 2024-03-21T10:36:00.291 INFO:teuthology.orchestra.run.smithi100.stdout: "handle_client_request": 6, 2024-03-21T10:36:00.291 INFO:teuthology.orchestra.run.smithi100.stdout: "handle_client_session": 13, 2024-03-21T10:36:00.291 INFO:teuthology.orchestra.run.smithi100.stdout: "handle_peer_request": 1, 2024-03-21T10:36:00.291 INFO:teuthology.orchestra.run.smithi100.stdout: "req_create_latency": { 2024-03-21T10:36:00.291 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 0, 2024-03-21T10:36:00.291 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.000000000, 2024-03-21T10:36:00.291 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000000000 2024-03-21T10:36:00.291 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-21T10:36:00.291 INFO:teuthology.orchestra.run.smithi100.stdout: "req_getattr_latency": { 2024-03-21T10:36:00.292 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 3, 2024-03-21T10:36:00.292 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.004677297, 2024-03-21T10:36:00.292 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.001559099 2024-03-21T10:36:00.292 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-21T10:36:00.292 INFO:teuthology.orchestra.run.smithi100.stdout: "req_getfilelock_latency": { 2024-03-21T10:36:00.292 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 0, 2024-03-21T10:36:00.292 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.000000000, 2024-03-21T10:36:00.292 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000000000 2024-03-21T10:36:00.292 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-21T10:36:00.292 INFO:teuthology.orchestra.run.smithi100.stdout: "req_link_latency": { 2024-03-21T10:36:00.292 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 0, 2024-03-21T10:36:00.292 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.000000000, 2024-03-21T10:36:00.292 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000000000 2024-03-21T10:36:00.292 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-21T10:36:00.293 INFO:teuthology.orchestra.run.smithi100.stdout: "req_lookup_latency": { 2024-03-21T10:36:00.293 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 2, 2024-03-21T10:36:00.293 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.000881228, 2024-03-21T10:36:00.293 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000440614 2024-03-21T10:36:00.293 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-21T10:36:00.293 INFO:teuthology.orchestra.run.smithi100.stdout: "req_lookuphash_latency": { 2024-03-21T10:36:00.293 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 0, 2024-03-21T10:36:00.293 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.000000000, 2024-03-21T10:36:00.293 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000000000 2024-03-21T10:36:00.293 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-21T10:36:00.293 INFO:teuthology.orchestra.run.smithi100.stdout: "req_lookupino_latency": { 2024-03-21T10:36:00.294 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 0, 2024-03-21T10:36:00.294 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.000000000, 2024-03-21T10:36:00.294 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000000000 2024-03-21T10:36:00.294 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-21T10:36:00.294 INFO:teuthology.orchestra.run.smithi100.stdout: "req_lookupname_latency": { 2024-03-21T10:36:00.294 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 0, 2024-03-21T10:36:00.294 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.000000000, 2024-03-21T10:36:00.294 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000000000 2024-03-21T10:36:00.294 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-21T10:36:00.294 INFO:teuthology.orchestra.run.smithi100.stdout: "req_lookupparent_latency": { 2024-03-21T10:36:00.294 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 0, 2024-03-21T10:36:00.294 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.000000000, 2024-03-21T10:36:00.295 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000000000 2024-03-21T10:36:00.295 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-21T10:36:00.295 INFO:teuthology.orchestra.run.smithi100.stdout: "req_lookupsnap_latency": { 2024-03-21T10:36:00.295 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 0, 2024-03-21T10:36:00.295 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.000000000, 2024-03-21T10:36:00.295 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000000000 2024-03-21T10:36:00.295 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-21T10:36:00.295 INFO:teuthology.orchestra.run.smithi100.stdout: "req_lssnap_latency": { 2024-03-21T10:36:00.295 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 0, 2024-03-21T10:36:00.295 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.000000000, 2024-03-21T10:36:00.295 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000000000 2024-03-21T10:36:00.295 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-21T10:36:00.295 INFO:teuthology.orchestra.run.smithi100.stdout: "req_mkdir_latency": { 2024-03-21T10:36:00.296 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 0, 2024-03-21T10:36:00.296 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.000000000, 2024-03-21T10:36:00.296 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000000000 2024-03-21T10:36:00.296 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-21T10:36:00.297 INFO:teuthology.orchestra.run.smithi100.stdout: "req_mknod_latency": { 2024-03-21T10:36:00.297 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 0, 2024-03-21T10:36:00.297 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.000000000, 2024-03-21T10:36:00.297 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000000000 2024-03-21T10:36:00.297 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-21T10:36:00.297 INFO:teuthology.orchestra.run.smithi100.stdout: "req_mksnap_latency": { 2024-03-21T10:36:00.297 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 0, 2024-03-21T10:36:00.297 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.000000000, 2024-03-21T10:36:00.297 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000000000 2024-03-21T10:36:00.297 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-21T10:36:00.297 INFO:teuthology.orchestra.run.smithi100.stdout: "req_open_latency": { 2024-03-21T10:36:00.297 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 0, 2024-03-21T10:36:00.297 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.000000000, 2024-03-21T10:36:00.298 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000000000 2024-03-21T10:36:00.298 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-21T10:36:00.298 INFO:teuthology.orchestra.run.smithi100.stdout: "req_readdir_latency": { 2024-03-21T10:36:00.298 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 0, 2024-03-21T10:36:00.298 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.000000000, 2024-03-21T10:36:00.298 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000000000 2024-03-21T10:36:00.299 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-21T10:36:00.299 INFO:teuthology.orchestra.run.smithi100.stdout: "req_rename_latency": { 2024-03-21T10:36:00.299 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 0, 2024-03-21T10:36:00.299 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.000000000, 2024-03-21T10:36:00.299 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000000000 2024-03-21T10:36:00.299 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-21T10:36:00.299 INFO:teuthology.orchestra.run.smithi100.stdout: "req_renamesnap_latency": { 2024-03-21T10:36:00.300 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 0, 2024-03-21T10:36:00.300 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.000000000, 2024-03-21T10:36:00.300 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000000000 2024-03-21T10:36:00.300 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-21T10:36:00.300 INFO:teuthology.orchestra.run.smithi100.stdout: "req_snapdiff_latency": { 2024-03-21T10:36:00.300 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 0, 2024-03-21T10:36:00.300 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.000000000, 2024-03-21T10:36:00.300 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000000000 2024-03-21T10:36:00.300 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-21T10:36:00.301 INFO:teuthology.orchestra.run.smithi100.stdout: "req_rmdir_latency": { 2024-03-21T10:36:00.301 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 0, 2024-03-21T10:36:00.301 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.000000000, 2024-03-21T10:36:00.301 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000000000 2024-03-21T10:36:00.301 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-21T10:36:00.301 INFO:teuthology.orchestra.run.smithi100.stdout: "req_rmsnap_latency": { 2024-03-21T10:36:00.301 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 0, 2024-03-21T10:36:00.301 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.000000000, 2024-03-21T10:36:00.301 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000000000 2024-03-21T10:36:00.301 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-21T10:36:00.301 INFO:teuthology.orchestra.run.smithi100.stdout: "req_rmxattr_latency": { 2024-03-21T10:36:00.301 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 0, 2024-03-21T10:36:00.301 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.000000000, 2024-03-21T10:36:00.302 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000000000 2024-03-21T10:36:00.302 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-21T10:36:00.302 INFO:teuthology.orchestra.run.smithi100.stdout: "req_setattr_latency": { 2024-03-21T10:36:00.302 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 0, 2024-03-21T10:36:00.302 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.000000000, 2024-03-21T10:36:00.302 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000000000 2024-03-21T10:36:00.302 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-21T10:36:00.302 INFO:teuthology.orchestra.run.smithi100.stdout: "req_setdirlayout_latency": { 2024-03-21T10:36:00.302 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 0, 2024-03-21T10:36:00.302 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.000000000, 2024-03-21T10:36:00.302 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000000000 2024-03-21T10:36:00.303 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-21T10:36:00.303 INFO:teuthology.orchestra.run.smithi100.stdout: "req_setfilelock_latency": { 2024-03-21T10:36:00.303 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 0, 2024-03-21T10:36:00.303 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.000000000, 2024-03-21T10:36:00.303 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000000000 2024-03-21T10:36:00.303 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-21T10:36:00.303 INFO:teuthology.orchestra.run.smithi100.stdout: "req_setlayout_latency": { 2024-03-21T10:36:00.303 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 0, 2024-03-21T10:36:00.303 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.000000000, 2024-03-21T10:36:00.303 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000000000 2024-03-21T10:36:00.303 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-21T10:36:00.303 INFO:teuthology.orchestra.run.smithi100.stdout: "req_setxattr_latency": { 2024-03-21T10:36:00.305 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 0, 2024-03-21T10:36:00.305 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.000000000, 2024-03-21T10:36:00.305 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000000000 2024-03-21T10:36:00.305 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-21T10:36:00.305 INFO:teuthology.orchestra.run.smithi100.stdout: "req_symlink_latency": { 2024-03-21T10:36:00.305 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 0, 2024-03-21T10:36:00.305 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.000000000, 2024-03-21T10:36:00.306 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000000000 2024-03-21T10:36:00.306 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-21T10:36:00.306 INFO:teuthology.orchestra.run.smithi100.stdout: "req_unlink_latency": { 2024-03-21T10:36:00.306 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 0, 2024-03-21T10:36:00.306 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.000000000, 2024-03-21T10:36:00.306 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000000000 2024-03-21T10:36:00.306 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-21T10:36:00.306 INFO:teuthology.orchestra.run.smithi100.stdout: "cap_revoke_eviction": 0, 2024-03-21T10:36:00.306 INFO:teuthology.orchestra.run.smithi100.stdout: "cap_acquisition_throttle": 0, 2024-03-21T10:36:00.306 INFO:teuthology.orchestra.run.smithi100.stdout: "req_getvxattr_latency": { 2024-03-21T10:36:00.306 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 0, 2024-03-21T10:36:00.306 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.000000000, 2024-03-21T10:36:00.306 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000000000 2024-03-21T10:36:00.307 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-03-21T10:36:00.307 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-21T10:36:00.307 INFO:teuthology.orchestra.run.smithi100.stdout: "mds_sessions": { 2024-03-21T10:36:00.307 INFO:teuthology.orchestra.run.smithi100.stdout: "session_count": 1, 2024-03-21T10:36:00.307 INFO:teuthology.orchestra.run.smithi100.stdout: "session_add": 2, 2024-03-21T10:36:00.307 INFO:teuthology.orchestra.run.smithi100.stdout: "session_remove": 1, 2024-03-21T10:36:00.307 INFO:teuthology.orchestra.run.smithi100.stdout: "sessions_open": 1, 2024-03-21T10:36:00.307 INFO:teuthology.orchestra.run.smithi100.stdout: "sessions_stale": 0, 2024-03-21T10:36:00.307 INFO:teuthology.orchestra.run.smithi100.stdout: "total_load": 4, 2024-03-21T10:36:00.307 INFO:teuthology.orchestra.run.smithi100.stdout: "average_load": 2, 2024-03-21T10:36:00.307 INFO:teuthology.orchestra.run.smithi100.stdout: "avg_session_uptime": 661, 2024-03-21T10:36:00.307 INFO:teuthology.orchestra.run.smithi100.stdout: "mdthresh_evicted": 0 2024-03-21T10:36:00.307 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-21T10:36:00.307 INFO:teuthology.orchestra.run.smithi100.stdout: "mempool": { 2024-03-21T10:36:00.308 INFO:teuthology.orchestra.run.smithi100.stdout: "bloom_filter_bytes": 0, 2024-03-21T10:36:00.308 INFO:teuthology.orchestra.run.smithi100.stdout: "bloom_filter_items": 0, 2024-03-21T10:36:00.308 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_alloc_bytes": 0, 2024-03-21T10:36:00.308 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_alloc_items": 0, 2024-03-21T10:36:00.308 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_cache_data_bytes": 0, 2024-03-21T10:36:00.308 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_cache_data_items": 0, 2024-03-21T10:36:00.308 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_cache_onode_bytes": 0, 2024-03-21T10:36:00.308 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_cache_onode_items": 0, 2024-03-21T10:36:00.308 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_cache_meta_bytes": 0, 2024-03-21T10:36:00.308 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_cache_meta_items": 0, 2024-03-21T10:36:00.308 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_cache_other_bytes": 0, 2024-03-21T10:36:00.308 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_cache_other_items": 0, 2024-03-21T10:36:00.308 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_cache_buffer_bytes": 0, 2024-03-21T10:36:00.308 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_cache_buffer_items": 0, 2024-03-21T10:36:00.309 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_extent_bytes": 0, 2024-03-21T10:36:00.309 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_extent_items": 0, 2024-03-21T10:36:00.309 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_blob_bytes": 0, 2024-03-21T10:36:00.309 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_blob_items": 0, 2024-03-21T10:36:00.309 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_shared_blob_bytes": 0, 2024-03-21T10:36:00.309 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_shared_blob_items": 0, 2024-03-21T10:36:00.309 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_inline_bl_bytes": 0, 2024-03-21T10:36:00.309 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_inline_bl_items": 0, 2024-03-21T10:36:00.309 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_fsck_bytes": 0, 2024-03-21T10:36:00.310 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_fsck_items": 0, 2024-03-21T10:36:00.310 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_txc_bytes": 0, 2024-03-21T10:36:00.310 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_txc_items": 0, 2024-03-21T10:36:00.310 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_writing_deferred_bytes": 0, 2024-03-21T10:36:00.310 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_writing_deferred_items": 0, 2024-03-21T10:36:00.310 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_writing_bytes": 0, 2024-03-21T10:36:00.311 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_writing_items": 0, 2024-03-21T10:36:00.311 INFO:teuthology.orchestra.run.smithi100.stdout: "bluefs_bytes": 0, 2024-03-21T10:36:00.311 INFO:teuthology.orchestra.run.smithi100.stdout: "bluefs_items": 0, 2024-03-21T10:36:00.311 INFO:teuthology.orchestra.run.smithi100.stdout: "bluefs_file_reader_bytes": 0, 2024-03-21T10:36:00.311 INFO:teuthology.orchestra.run.smithi100.stdout: "bluefs_file_reader_items": 0, 2024-03-21T10:36:00.311 INFO:teuthology.orchestra.run.smithi100.stdout: "bluefs_file_writer_bytes": 0, 2024-03-21T10:36:00.312 INFO:teuthology.orchestra.run.smithi100.stdout: "bluefs_file_writer_items": 0, 2024-03-21T10:36:00.312 INFO:teuthology.orchestra.run.smithi100.stdout: "buffer_anon_bytes": 264449, 2024-03-21T10:36:00.312 INFO:teuthology.orchestra.run.smithi100.stdout: "buffer_anon_items": 98, 2024-03-21T10:36:00.312 INFO:teuthology.orchestra.run.smithi100.stdout: "buffer_meta_bytes": 0, 2024-03-21T10:36:00.312 INFO:teuthology.orchestra.run.smithi100.stdout: "buffer_meta_items": 0, 2024-03-21T10:36:00.312 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_bytes": 0, 2024-03-21T10:36:00.313 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_items": 0, 2024-03-21T10:36:00.313 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_mapbl_bytes": 0, 2024-03-21T10:36:00.313 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_mapbl_items": 0, 2024-03-21T10:36:00.313 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_pglog_bytes": 0, 2024-03-21T10:36:00.313 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_pglog_items": 0, 2024-03-21T10:36:00.313 INFO:teuthology.orchestra.run.smithi100.stdout: "osdmap_bytes": 9712, 2024-03-21T10:36:00.314 INFO:teuthology.orchestra.run.smithi100.stdout: "osdmap_items": 187, 2024-03-21T10:36:00.314 INFO:teuthology.orchestra.run.smithi100.stdout: "osdmap_mapping_bytes": 0, 2024-03-21T10:36:00.314 INFO:teuthology.orchestra.run.smithi100.stdout: "osdmap_mapping_items": 0, 2024-03-21T10:36:00.314 INFO:teuthology.orchestra.run.smithi100.stdout: "pgmap_bytes": 0, 2024-03-21T10:36:00.314 INFO:teuthology.orchestra.run.smithi100.stdout: "pgmap_items": 0, 2024-03-21T10:36:00.314 INFO:teuthology.orchestra.run.smithi100.stdout: "mds_co_bytes": 59120, 2024-03-21T10:36:00.314 INFO:teuthology.orchestra.run.smithi100.stdout: "mds_co_items": 239, 2024-03-21T10:36:00.314 INFO:teuthology.orchestra.run.smithi100.stdout: "unittest_1_bytes": 0, 2024-03-21T10:36:00.314 INFO:teuthology.orchestra.run.smithi100.stdout: "unittest_1_items": 0, 2024-03-21T10:36:00.315 INFO:teuthology.orchestra.run.smithi100.stdout: "unittest_2_bytes": 0, 2024-03-21T10:36:00.315 INFO:teuthology.orchestra.run.smithi100.stdout: "unittest_2_items": 0 2024-03-21T10:36:00.315 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-21T10:36:00.315 INFO:teuthology.orchestra.run.smithi100.stdout: "objecter": { 2024-03-21T10:36:00.315 INFO:teuthology.orchestra.run.smithi100.stdout: "op_active": 0, 2024-03-21T10:36:00.315 INFO:teuthology.orchestra.run.smithi100.stdout: "op_laggy": 0, 2024-03-21T10:36:00.315 INFO:teuthology.orchestra.run.smithi100.stdout: "op_send": 44, 2024-03-21T10:36:00.316 INFO:teuthology.orchestra.run.smithi100.stdout: "op_send_bytes": 12344, 2024-03-21T10:36:00.316 INFO:teuthology.orchestra.run.smithi100.stdout: "op_resend": 33, 2024-03-21T10:36:00.316 INFO:teuthology.orchestra.run.smithi100.stdout: "op_reply": 44, 2024-03-21T10:36:00.316 INFO:teuthology.orchestra.run.smithi100.stdout: "op_latency": { 2024-03-21T10:36:00.316 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 44, 2024-03-21T10:36:00.316 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.537985871, 2024-03-21T10:36:00.317 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.012226951 2024-03-21T10:36:00.317 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-21T10:36:00.317 INFO:teuthology.orchestra.run.smithi100.stdout: "op_inflight": 0, 2024-03-21T10:36:00.317 INFO:teuthology.orchestra.run.smithi100.stdout: "oplen_avg": { 2024-03-21T10:36:00.317 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 44, 2024-03-21T10:36:00.317 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 69 2024-03-21T10:36:00.318 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-21T10:36:00.318 INFO:teuthology.orchestra.run.smithi100.stdout: "op": 44, 2024-03-21T10:36:00.318 INFO:teuthology.orchestra.run.smithi100.stdout: "op_r": 0, 2024-03-21T10:36:00.318 INFO:teuthology.orchestra.run.smithi100.stdout: "op_w": 44, 2024-03-21T10:36:00.318 INFO:teuthology.orchestra.run.smithi100.stdout: "op_rmw": 0, 2024-03-21T10:36:00.319 INFO:teuthology.orchestra.run.smithi100.stdout: "op_pg": 0, 2024-03-21T10:36:00.319 INFO:teuthology.orchestra.run.smithi100.stdout: "osdop_stat": 0, 2024-03-21T10:36:00.319 INFO:teuthology.orchestra.run.smithi100.stdout: "osdop_create": 10, 2024-03-21T10:36:00.319 INFO:teuthology.orchestra.run.smithi100.stdout: "osdop_read": 0, 2024-03-21T10:36:00.319 INFO:teuthology.orchestra.run.smithi100.stdout: "osdop_write": 4, 2024-03-21T10:36:00.319 INFO:teuthology.orchestra.run.smithi100.stdout: "osdop_writefull": 8, 2024-03-21T10:36:00.320 INFO:teuthology.orchestra.run.smithi100.stdout: "osdop_writesame": 0, 2024-03-21T10:36:00.320 INFO:teuthology.orchestra.run.smithi100.stdout: "osdop_append": 0, 2024-03-21T10:36:00.320 INFO:teuthology.orchestra.run.smithi100.stdout: "osdop_zero": 0, 2024-03-21T10:36:00.320 INFO:teuthology.orchestra.run.smithi100.stdout: "osdop_truncate": 0, 2024-03-21T10:36:00.320 INFO:teuthology.orchestra.run.smithi100.stdout: "osdop_delete": 6, 2024-03-21T10:36:00.321 INFO:teuthology.orchestra.run.smithi100.stdout: "osdop_mapext": 0, 2024-03-21T10:36:00.321 INFO:teuthology.orchestra.run.smithi100.stdout: "osdop_sparse_read": 0, 2024-03-21T10:36:00.321 INFO:teuthology.orchestra.run.smithi100.stdout: "osdop_clonerange": 0, 2024-03-21T10:36:00.321 INFO:teuthology.orchestra.run.smithi100.stdout: "osdop_getxattr": 0, 2024-03-21T10:36:00.321 INFO:teuthology.orchestra.run.smithi100.stdout: "osdop_setxattr": 20, 2024-03-21T10:36:00.321 INFO:teuthology.orchestra.run.smithi100.stdout: "osdop_cmpxattr": 0, 2024-03-21T10:36:00.322 INFO:teuthology.orchestra.run.smithi100.stdout: "osdop_rmxattr": 0, 2024-03-21T10:36:00.322 INFO:teuthology.orchestra.run.smithi100.stdout: "osdop_resetxattrs": 0, 2024-03-21T10:36:00.322 INFO:teuthology.orchestra.run.smithi100.stdout: "osdop_call": 0, 2024-03-21T10:36:00.322 INFO:teuthology.orchestra.run.smithi100.stdout: "osdop_watch": 0, 2024-03-21T10:36:00.322 INFO:teuthology.orchestra.run.smithi100.stdout: "osdop_notify": 0, 2024-03-21T10:36:00.322 INFO:teuthology.orchestra.run.smithi100.stdout: "osdop_src_cmpxattr": 0, 2024-03-21T10:36:00.323 INFO:teuthology.orchestra.run.smithi100.stdout: "osdop_pgls": 0, 2024-03-21T10:36:00.323 INFO:teuthology.orchestra.run.smithi100.stdout: "osdop_pgls_filter": 0, 2024-03-21T10:36:00.323 INFO:teuthology.orchestra.run.smithi100.stdout: "osdop_other": 0, 2024-03-21T10:36:00.323 INFO:teuthology.orchestra.run.smithi100.stdout: "linger_active": 0, 2024-03-21T10:36:00.323 INFO:teuthology.orchestra.run.smithi100.stdout: "linger_send": 0, 2024-03-21T10:36:00.324 INFO:teuthology.orchestra.run.smithi100.stdout: "linger_resend": 0, 2024-03-21T10:36:00.324 INFO:teuthology.orchestra.run.smithi100.stdout: "linger_ping": 0, 2024-03-21T10:36:00.324 INFO:teuthology.orchestra.run.smithi100.stdout: "poolop_active": 0, 2024-03-21T10:36:00.324 INFO:teuthology.orchestra.run.smithi100.stdout: "poolop_send": 0, 2024-03-21T10:36:00.324 INFO:teuthology.orchestra.run.smithi100.stdout: "poolop_resend": 0, 2024-03-21T10:36:00.324 INFO:teuthology.orchestra.run.smithi100.stdout: "poolstat_active": 0, 2024-03-21T10:36:00.325 INFO:teuthology.orchestra.run.smithi100.stdout: "poolstat_send": 0, 2024-03-21T10:36:00.325 INFO:teuthology.orchestra.run.smithi100.stdout: "poolstat_resend": 0, 2024-03-21T10:36:00.325 INFO:teuthology.orchestra.run.smithi100.stdout: "statfs_active": 0, 2024-03-21T10:36:00.325 INFO:teuthology.orchestra.run.smithi100.stdout: "statfs_send": 0, 2024-03-21T10:36:00.325 INFO:teuthology.orchestra.run.smithi100.stdout: "statfs_resend": 0, 2024-03-21T10:36:00.325 INFO:teuthology.orchestra.run.smithi100.stdout: "command_active": 0, 2024-03-21T10:36:00.326 INFO:teuthology.orchestra.run.smithi100.stdout: "command_send": 0, 2024-03-21T10:36:00.326 INFO:teuthology.orchestra.run.smithi100.stdout: "command_resend": 0, 2024-03-21T10:36:00.326 INFO:teuthology.orchestra.run.smithi100.stdout: "map_epoch": 105, 2024-03-21T10:36:00.326 INFO:teuthology.orchestra.run.smithi100.stdout: "map_full": 0, 2024-03-21T10:36:00.326 INFO:teuthology.orchestra.run.smithi100.stdout: "map_inc": 14, 2024-03-21T10:36:00.326 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_sessions": 9, 2024-03-21T10:36:00.327 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_session_open": 9, 2024-03-21T10:36:00.327 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_session_close": 0, 2024-03-21T10:36:00.327 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_laggy": 0, 2024-03-21T10:36:00.327 INFO:teuthology.orchestra.run.smithi100.stdout: "omap_wr": 20, 2024-03-21T10:36:00.327 INFO:teuthology.orchestra.run.smithi100.stdout: "omap_rd": 0, 2024-03-21T10:36:00.328 INFO:teuthology.orchestra.run.smithi100.stdout: "omap_del": 1 2024-03-21T10:36:00.328 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-21T10:36:00.328 INFO:teuthology.orchestra.run.smithi100.stdout: "oft": { 2024-03-21T10:36:00.328 INFO:teuthology.orchestra.run.smithi100.stdout: "omap_total_objs": 1, 2024-03-21T10:36:00.328 INFO:teuthology.orchestra.run.smithi100.stdout: "omap_total_kv_pairs": 0, 2024-03-21T10:36:00.328 INFO:teuthology.orchestra.run.smithi100.stdout: "omap_total_updates": 0, 2024-03-21T10:36:00.329 INFO:teuthology.orchestra.run.smithi100.stdout: "omap_total_removes": 0 2024-03-21T10:36:00.329 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-21T10:36:00.329 INFO:teuthology.orchestra.run.smithi100.stdout: "osd-slow-ops": { 2024-03-21T10:36:00.329 INFO:teuthology.orchestra.run.smithi100.stdout: "slow_ops_count": 0 2024-03-21T10:36:00.329 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-21T10:36:00.329 INFO:teuthology.orchestra.run.smithi100.stdout: "purge_queue": { 2024-03-21T10:36:00.330 INFO:teuthology.orchestra.run.smithi100.stdout: "pq_executing_ops": 0, 2024-03-21T10:36:00.330 INFO:teuthology.orchestra.run.smithi100.stdout: "pq_executing_ops_high_water": 0, 2024-03-21T10:36:00.330 INFO:teuthology.orchestra.run.smithi100.stdout: "pq_executing": 0, 2024-03-21T10:36:00.330 INFO:teuthology.orchestra.run.smithi100.stdout: "pq_executing_high_water": 0, 2024-03-21T10:36:00.330 INFO:teuthology.orchestra.run.smithi100.stdout: "pq_executed_ops": 0, 2024-03-21T10:36:00.331 INFO:teuthology.orchestra.run.smithi100.stdout: "pq_executed": 0, 2024-03-21T10:36:00.331 INFO:teuthology.orchestra.run.smithi100.stdout: "pq_item_in_journal": 0 2024-03-21T10:36:00.331 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-21T10:36:00.331 INFO:teuthology.orchestra.run.smithi100.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-03-21T10:36:00.331 INFO:teuthology.orchestra.run.smithi100.stdout: "val": 0, 2024-03-21T10:36:00.331 INFO:teuthology.orchestra.run.smithi100.stdout: "max": 104857600, 2024-03-21T10:36:00.331 INFO:teuthology.orchestra.run.smithi100.stdout: "get_started": 0, 2024-03-21T10:36:00.332 INFO:teuthology.orchestra.run.smithi100.stdout: "get": 1005, 2024-03-21T10:36:00.332 INFO:teuthology.orchestra.run.smithi100.stdout: "get_sum": 1928170, 2024-03-21T10:36:00.332 INFO:teuthology.orchestra.run.smithi100.stdout: "get_or_fail_fail": 0, 2024-03-21T10:36:00.332 INFO:teuthology.orchestra.run.smithi100.stdout: "get_or_fail_success": 1005, 2024-03-21T10:36:00.332 INFO:teuthology.orchestra.run.smithi100.stdout: "take": 0, 2024-03-21T10:36:00.332 INFO:teuthology.orchestra.run.smithi100.stdout: "take_sum": 0, 2024-03-21T10:36:00.332 INFO:teuthology.orchestra.run.smithi100.stdout: "put": 1005, 2024-03-21T10:36:00.332 INFO:teuthology.orchestra.run.smithi100.stdout: "put_sum": 1928170, 2024-03-21T10:36:00.332 INFO:teuthology.orchestra.run.smithi100.stdout: "wait": { 2024-03-21T10:36:00.332 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 0, 2024-03-21T10:36:00.332 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.000000000, 2024-03-21T10:36:00.332 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000000000 2024-03-21T10:36:00.332 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-03-21T10:36:00.332 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-21T10:36:00.332 INFO:teuthology.orchestra.run.smithi100.stdout: "throttle-objecter_bytes": { 2024-03-21T10:36:00.333 INFO:teuthology.orchestra.run.smithi100.stdout: "val": 0, 2024-03-21T10:36:00.333 INFO:teuthology.orchestra.run.smithi100.stdout: "max": 104857600, 2024-03-21T10:36:00.333 INFO:teuthology.orchestra.run.smithi100.stdout: "get_started": 0, 2024-03-21T10:36:00.333 INFO:teuthology.orchestra.run.smithi100.stdout: "get": 0, 2024-03-21T10:36:00.333 INFO:teuthology.orchestra.run.smithi100.stdout: "get_sum": 0, 2024-03-21T10:36:00.333 INFO:teuthology.orchestra.run.smithi100.stdout: "get_or_fail_fail": 0, 2024-03-21T10:36:00.333 INFO:teuthology.orchestra.run.smithi100.stdout: "get_or_fail_success": 0, 2024-03-21T10:36:00.333 INFO:teuthology.orchestra.run.smithi100.stdout: "take": 44, 2024-03-21T10:36:00.333 INFO:teuthology.orchestra.run.smithi100.stdout: "take_sum": 12344, 2024-03-21T10:36:00.333 INFO:teuthology.orchestra.run.smithi100.stdout: "put": 44, 2024-03-21T10:36:00.333 INFO:teuthology.orchestra.run.smithi100.stdout: "put_sum": 12344, 2024-03-21T10:36:00.333 INFO:teuthology.orchestra.run.smithi100.stdout: "wait": { 2024-03-21T10:36:00.333 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 0, 2024-03-21T10:36:00.333 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.000000000, 2024-03-21T10:36:00.333 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000000000 2024-03-21T10:36:00.333 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-03-21T10:36:00.334 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-21T10:36:00.334 INFO:teuthology.orchestra.run.smithi100.stdout: "throttle-objecter_ops": { 2024-03-21T10:36:00.334 INFO:teuthology.orchestra.run.smithi100.stdout: "val": 0, 2024-03-21T10:36:00.334 INFO:teuthology.orchestra.run.smithi100.stdout: "max": 1024, 2024-03-21T10:36:00.334 INFO:teuthology.orchestra.run.smithi100.stdout: "get_started": 0, 2024-03-21T10:36:00.334 INFO:teuthology.orchestra.run.smithi100.stdout: "get": 0, 2024-03-21T10:36:00.334 INFO:teuthology.orchestra.run.smithi100.stdout: "get_sum": 0, 2024-03-21T10:36:00.334 INFO:teuthology.orchestra.run.smithi100.stdout: "get_or_fail_fail": 0, 2024-03-21T10:36:00.334 INFO:teuthology.orchestra.run.smithi100.stdout: "get_or_fail_success": 0, 2024-03-21T10:36:00.334 INFO:teuthology.orchestra.run.smithi100.stdout: "take": 44, 2024-03-21T10:36:00.334 INFO:teuthology.orchestra.run.smithi100.stdout: "take_sum": 44, 2024-03-21T10:36:00.334 INFO:teuthology.orchestra.run.smithi100.stdout: "put": 44, 2024-03-21T10:36:00.334 INFO:teuthology.orchestra.run.smithi100.stdout: "put_sum": 44, 2024-03-21T10:36:00.335 INFO:teuthology.orchestra.run.smithi100.stdout: "wait": { 2024-03-21T10:36:00.335 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 0, 2024-03-21T10:36:00.335 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.000000000, 2024-03-21T10:36:00.335 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000000000 2024-03-21T10:36:00.335 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-03-21T10:36:00.335 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-21T10:36:00.335 INFO:teuthology.orchestra.run.smithi100.stdout: "throttle-write_buf_throttle": { 2024-03-21T10:36:00.335 INFO:teuthology.orchestra.run.smithi100.stdout: "val": 0, 2024-03-21T10:36:00.335 INFO:teuthology.orchestra.run.smithi100.stdout: "max": 3758096384, 2024-03-21T10:36:00.335 INFO:teuthology.orchestra.run.smithi100.stdout: "get_started": 0, 2024-03-21T10:36:00.335 INFO:teuthology.orchestra.run.smithi100.stdout: "get": 0, 2024-03-21T10:36:00.335 INFO:teuthology.orchestra.run.smithi100.stdout: "get_sum": 0, 2024-03-21T10:36:00.335 INFO:teuthology.orchestra.run.smithi100.stdout: "get_or_fail_fail": 0, 2024-03-21T10:36:00.335 INFO:teuthology.orchestra.run.smithi100.stdout: "get_or_fail_success": 0, 2024-03-21T10:36:00.336 INFO:teuthology.orchestra.run.smithi100.stdout: "take": 0, 2024-03-21T10:36:00.336 INFO:teuthology.orchestra.run.smithi100.stdout: "take_sum": 0, 2024-03-21T10:36:00.336 INFO:teuthology.orchestra.run.smithi100.stdout: "put": 0, 2024-03-21T10:36:00.336 INFO:teuthology.orchestra.run.smithi100.stdout: "put_sum": 0, 2024-03-21T10:36:00.336 INFO:teuthology.orchestra.run.smithi100.stdout: "wait": { 2024-03-21T10:36:00.336 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 0, 2024-03-21T10:36:00.336 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.000000000, 2024-03-21T10:36:00.336 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000000000 2024-03-21T10:36:00.336 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-03-21T10:36:00.336 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-21T10:36:00.336 INFO:teuthology.orchestra.run.smithi100.stdout: "throttle-write_buf_throttle-0x55eb08954500": { 2024-03-21T10:36:00.336 INFO:teuthology.orchestra.run.smithi100.stdout: "val": 0, 2024-03-21T10:36:00.336 INFO:teuthology.orchestra.run.smithi100.stdout: "max": 3758096384, 2024-03-21T10:36:00.336 INFO:teuthology.orchestra.run.smithi100.stdout: "get_started": 0, 2024-03-21T10:36:00.336 INFO:teuthology.orchestra.run.smithi100.stdout: "get": 5, 2024-03-21T10:36:00.337 INFO:teuthology.orchestra.run.smithi100.stdout: "get_sum": 1613, 2024-03-21T10:36:00.337 INFO:teuthology.orchestra.run.smithi100.stdout: "get_or_fail_fail": 0, 2024-03-21T10:36:00.337 INFO:teuthology.orchestra.run.smithi100.stdout: "get_or_fail_success": 5, 2024-03-21T10:36:00.337 INFO:teuthology.orchestra.run.smithi100.stdout: "take": 0, 2024-03-21T10:36:00.337 INFO:teuthology.orchestra.run.smithi100.stdout: "take_sum": 0, 2024-03-21T10:36:00.337 INFO:teuthology.orchestra.run.smithi100.stdout: "put": 4, 2024-03-21T10:36:00.337 INFO:teuthology.orchestra.run.smithi100.stdout: "put_sum": 1613, 2024-03-21T10:36:00.337 INFO:teuthology.orchestra.run.smithi100.stdout: "wait": { 2024-03-21T10:36:00.337 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 0, 2024-03-21T10:36:00.337 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.000000000, 2024-03-21T10:36:00.337 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000000000 2024-03-21T10:36:00.337 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-03-21T10:36:00.337 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-03-21T10:36:00.337 INFO:teuthology.orchestra.run.smithi100.stdout:} 2024-03-21T10:36:01.168 INFO:tasks.check_counter:Daemon mds.h mds.exported=0 2024-03-21T10:36:01.168 INFO:tasks.check_counter:Daemon mds.h mds.imported=0 2024-03-21T10:36:01.168 INFO:tasks.check_counter:Daemon mds.h mds.root_rsnaps=6 2024-03-21T10:36:01.168 INFO:tasks.check_counter:Daemon mds.h mds_server.req_mksnap_latency.avgcount=0 2024-03-21T10:36:01.168 INFO:tasks.check_counter:Daemon mds.h mds_server.req_rmsnap_latency.avgcount=0 2024-03-21T10:36:01.168 DEBUG:tasks.check_counter:Getting stats from k 2024-03-21T10:36:01.168 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2edfee68ad7243bed422a049b6f5f58619300f11 shell --fsid 3056d51a-e76b-11ee-95cd-87774f69a715 -- ceph daemon mds.k perf dump 2024-03-21T10:36:01.690 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/3056d51a-e76b-11ee-95cd-87774f69a715/mon.b/config 2024-03-21T10:36:01.843 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:36:01 smithi100 ceph-mon[35193]: pgmap v332: 129 pgs: 129 active+clean; 2.5 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-21T10:36:01.960 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:36:01 smithi047 ceph-mon[31547]: pgmap v332: 129 pgs: 129 active+clean; 2.5 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-21T10:36:01.986 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:36:01 smithi157 ceph-mon[35581]: pgmap v332: 129 pgs: 129 active+clean; 2.5 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-21T10:36:03.726 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:36:03 smithi100 ceph-mon[35193]: pgmap v333: 129 pgs: 129 active+clean; 2.5 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 33 KiB/s wr, 28 op/s 2024-03-21T10:36:03.960 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:36:03 smithi047 ceph-mon[31547]: pgmap v333: 129 pgs: 129 active+clean; 2.5 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 33 KiB/s wr, 28 op/s 2024-03-21T10:36:03.979 INFO:teuthology.orchestra.run.smithi100.stdout:{ 2024-03-21T10:36:03.979 INFO:teuthology.orchestra.run.smithi100.stdout: "AsyncMessenger::Worker-0": { 2024-03-21T10:36:03.979 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_recv_messages": 138, 2024-03-21T10:36:03.979 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_send_messages": 173, 2024-03-21T10:36:03.980 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_recv_bytes": 62062, 2024-03-21T10:36:03.980 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_send_bytes": 122775, 2024-03-21T10:36:03.980 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_created_connections": 15, 2024-03-21T10:36:03.980 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_active_connections": 5, 2024-03-21T10:36:03.980 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_running_total_time": 0.041269378, 2024-03-21T10:36:03.980 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_running_send_time": 0.012671706, 2024-03-21T10:36:03.980 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_running_recv_time": 0.017205705, 2024-03-21T10:36:03.980 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_running_fast_dispatch_time": 0.000433726, 2024-03-21T10:36:03.980 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_send_messages_queue_lat": { 2024-03-21T10:36:03.980 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 170, 2024-03-21T10:36:03.980 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.083491577, 2024-03-21T10:36:03.980 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000491126 2024-03-21T10:36:03.980 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-21T10:36:03.980 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_handle_ack_lat": { 2024-03-21T10:36:03.981 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 224, 2024-03-21T10:36:03.981 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.000054400, 2024-03-21T10:36:03.981 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000000242 2024-03-21T10:36:03.981 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-21T10:36:03.981 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_recv_encrypted_bytes": 0, 2024-03-21T10:36:03.981 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_send_encrypted_bytes": 0 2024-03-21T10:36:03.981 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-21T10:36:03.981 INFO:teuthology.orchestra.run.smithi100.stdout: "AsyncMessenger::Worker-1": { 2024-03-21T10:36:03.981 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_recv_messages": 835, 2024-03-21T10:36:03.981 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_send_messages": 960, 2024-03-21T10:36:03.981 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_recv_bytes": 1948890, 2024-03-21T10:36:03.981 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_send_bytes": 550058, 2024-03-21T10:36:03.981 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_created_connections": 12, 2024-03-21T10:36:03.981 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_active_connections": 5, 2024-03-21T10:36:03.981 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_running_total_time": 0.208582218, 2024-03-21T10:36:03.981 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_running_send_time": 0.096494656, 2024-03-21T10:36:03.982 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_running_recv_time": 0.117624757, 2024-03-21T10:36:03.982 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_running_fast_dispatch_time": 0.013782242, 2024-03-21T10:36:03.982 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_send_messages_queue_lat": { 2024-03-21T10:36:03.982 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 960, 2024-03-21T10:36:03.982 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.140619191, 2024-03-21T10:36:03.982 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000146478 2024-03-21T10:36:03.982 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-21T10:36:03.982 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_handle_ack_lat": { 2024-03-21T10:36:03.982 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 1125, 2024-03-21T10:36:03.982 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.000415601, 2024-03-21T10:36:03.982 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000000369 2024-03-21T10:36:03.982 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-21T10:36:03.982 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_recv_encrypted_bytes": 1799408, 2024-03-21T10:36:03.982 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_send_encrypted_bytes": 159680 2024-03-21T10:36:03.983 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-21T10:36:03.983 INFO:teuthology.orchestra.run.smithi100.stdout: "AsyncMessenger::Worker-2": { 2024-03-21T10:36:03.983 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_recv_messages": 105, 2024-03-21T10:36:03.983 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_send_messages": 284, 2024-03-21T10:36:03.983 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_recv_bytes": 38172, 2024-03-21T10:36:03.983 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_send_bytes": 847652, 2024-03-21T10:36:03.983 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_created_connections": 49, 2024-03-21T10:36:03.983 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_active_connections": 8, 2024-03-21T10:36:03.983 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_running_total_time": 0.059452282, 2024-03-21T10:36:03.983 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_running_send_time": 0.018998519, 2024-03-21T10:36:03.983 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_running_recv_time": 0.035185249, 2024-03-21T10:36:03.983 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_running_fast_dispatch_time": 0.000155807, 2024-03-21T10:36:03.983 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_send_messages_queue_lat": { 2024-03-21T10:36:03.984 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 282, 2024-03-21T10:36:03.984 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.077666596, 2024-03-21T10:36:03.984 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000275413 2024-03-21T10:36:03.984 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-21T10:36:03.984 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_handle_ack_lat": { 2024-03-21T10:36:03.984 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 134, 2024-03-21T10:36:03.984 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.000041608, 2024-03-21T10:36:03.984 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000000310 2024-03-21T10:36:03.984 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-21T10:36:03.984 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_recv_encrypted_bytes": 4192, 2024-03-21T10:36:03.984 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_send_encrypted_bytes": 402656 2024-03-21T10:36:03.984 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-21T10:36:03.984 INFO:teuthology.orchestra.run.smithi100.stdout: "cct": { 2024-03-21T10:36:03.984 INFO:teuthology.orchestra.run.smithi100.stdout: "total_workers": 1, 2024-03-21T10:36:03.984 INFO:teuthology.orchestra.run.smithi100.stdout: "unhealthy_workers": 0 2024-03-21T10:36:03.985 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-21T10:36:03.985 INFO:teuthology.orchestra.run.smithi100.stdout: "finisher-MDSRank": { 2024-03-21T10:36:03.985 INFO:teuthology.orchestra.run.smithi100.stdout: "queue_len": 0, 2024-03-21T10:36:03.985 INFO:teuthology.orchestra.run.smithi100.stdout: "complete_latency": { 2024-03-21T10:36:03.985 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 59, 2024-03-21T10:36:03.985 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.002102801, 2024-03-21T10:36:03.985 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000035640 2024-03-21T10:36:03.985 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-03-21T10:36:03.985 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-21T10:36:03.985 INFO:teuthology.orchestra.run.smithi100.stdout: "finisher-PurgeQueue": { 2024-03-21T10:36:03.985 INFO:teuthology.orchestra.run.smithi100.stdout: "queue_len": 0, 2024-03-21T10:36:03.985 INFO:teuthology.orchestra.run.smithi100.stdout: "complete_latency": { 2024-03-21T10:36:03.985 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 2, 2024-03-21T10:36:03.985 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.000006372, 2024-03-21T10:36:03.986 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000003186 2024-03-21T10:36:03.986 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-03-21T10:36:03.986 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-21T10:36:03.986 INFO:teuthology.orchestra.run.smithi100.stdout: "mds": { 2024-03-21T10:36:03.986 INFO:teuthology.orchestra.run.smithi100.stdout: "request": 6, 2024-03-21T10:36:03.986 INFO:teuthology.orchestra.run.smithi100.stdout: "reply": 4, 2024-03-21T10:36:03.986 INFO:teuthology.orchestra.run.smithi100.stdout: "reply_latency": { 2024-03-21T10:36:03.986 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 4, 2024-03-21T10:36:03.986 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.002984780, 2024-03-21T10:36:03.986 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000746195 2024-03-21T10:36:03.986 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-21T10:36:03.986 INFO:teuthology.orchestra.run.smithi100.stdout: "slow_reply": 0, 2024-03-21T10:36:03.986 INFO:teuthology.orchestra.run.smithi100.stdout: "forward": 2, 2024-03-21T10:36:03.986 INFO:teuthology.orchestra.run.smithi100.stdout: "dir_fetch_complete": 0, 2024-03-21T10:36:03.987 INFO:teuthology.orchestra.run.smithi100.stdout: "dir_fetch_keys": 0, 2024-03-21T10:36:03.987 INFO:teuthology.orchestra.run.smithi100.stdout: "dir_commit": 11, 2024-03-21T10:36:03.987 INFO:teuthology.orchestra.run.smithi100.stdout: "dir_split": 0, 2024-03-21T10:36:03.987 INFO:teuthology.orchestra.run.smithi100.stdout: "dir_merge": 0, 2024-03-21T10:36:03.987 INFO:teuthology.orchestra.run.smithi100.stdout: "inodes": 14, 2024-03-21T10:36:03.987 INFO:teuthology.orchestra.run.smithi100.stdout: "inodes_top": 0, 2024-03-21T10:36:03.987 INFO:teuthology.orchestra.run.smithi100.stdout: "inodes_bottom": 14, 2024-03-21T10:36:03.987 INFO:teuthology.orchestra.run.smithi100.stdout: "inodes_pin_tail": 0, 2024-03-21T10:36:03.987 INFO:teuthology.orchestra.run.smithi100.stdout: "inodes_pinned": 14, 2024-03-21T10:36:03.987 INFO:teuthology.orchestra.run.smithi100.stdout: "inodes_expired": 1, 2024-03-21T10:36:03.987 INFO:teuthology.orchestra.run.smithi100.stdout: "inodes_with_caps": 1, 2024-03-21T10:36:03.987 INFO:teuthology.orchestra.run.smithi100.stdout: "caps": 1, 2024-03-21T10:36:03.987 INFO:teuthology.orchestra.run.smithi100.stdout: "subtrees": 2, 2024-03-21T10:36:03.987 INFO:teuthology.orchestra.run.smithi100.stdout: "traverse": 11, 2024-03-21T10:36:03.987 INFO:teuthology.orchestra.run.smithi100.stdout: "traverse_hit": 5, 2024-03-21T10:36:03.988 INFO:teuthology.orchestra.run.smithi100.stdout: "traverse_forward": 1, 2024-03-21T10:36:03.988 INFO:teuthology.orchestra.run.smithi100.stdout: "traverse_discover": 2, 2024-03-21T10:36:03.988 INFO:teuthology.orchestra.run.smithi100.stdout: "traverse_dir_fetch": 0, 2024-03-21T10:36:03.988 INFO:teuthology.orchestra.run.smithi100.stdout: "traverse_remote_ino": 0, 2024-03-21T10:36:03.988 INFO:teuthology.orchestra.run.smithi100.stdout: "traverse_lock": 0, 2024-03-21T10:36:03.988 INFO:teuthology.orchestra.run.smithi100.stdout: "load_cent": 0, 2024-03-21T10:36:03.988 INFO:teuthology.orchestra.run.smithi100.stdout: "q": 0, 2024-03-21T10:36:03.988 INFO:teuthology.orchestra.run.smithi100.stdout: "exported": 0, 2024-03-21T10:36:03.988 INFO:teuthology.orchestra.run.smithi100.stdout: "exported_inodes": 0, 2024-03-21T10:36:03.988 INFO:teuthology.orchestra.run.smithi100.stdout: "imported": 0, 2024-03-21T10:36:03.988 INFO:teuthology.orchestra.run.smithi100.stdout: "imported_inodes": 0, 2024-03-21T10:36:03.988 INFO:teuthology.orchestra.run.smithi100.stdout: "openino_dir_fetch": 0, 2024-03-21T10:36:03.988 INFO:teuthology.orchestra.run.smithi100.stdout: "openino_backtrace_fetch": 0, 2024-03-21T10:36:03.988 INFO:teuthology.orchestra.run.smithi100.stdout: "openino_peer_discover": 0, 2024-03-21T10:36:03.988 INFO:teuthology.orchestra.run.smithi100.stdout: "root_rfiles": 4, 2024-03-21T10:36:03.988 INFO:teuthology.orchestra.run.smithi100.stdout: "root_rbytes": 234, 2024-03-21T10:36:03.989 INFO:teuthology.orchestra.run.smithi100.stdout: "root_rsnaps": 6, 2024-03-21T10:36:03.989 INFO:teuthology.orchestra.run.smithi100.stdout: "scrub_backtrace_fetch": 0, 2024-03-21T10:36:03.989 INFO:teuthology.orchestra.run.smithi100.stdout: "scrub_set_tag": 0, 2024-03-21T10:36:03.989 INFO:teuthology.orchestra.run.smithi100.stdout: "scrub_backtrace_repaired": 0, 2024-03-21T10:36:03.989 INFO:teuthology.orchestra.run.smithi100.stdout: "scrub_inotable_repaired": 0, 2024-03-21T10:36:03.989 INFO:teuthology.orchestra.run.smithi100.stdout: "scrub_dir_inodes": 0, 2024-03-21T10:36:03.989 INFO:teuthology.orchestra.run.smithi100.stdout: "scrub_dir_base_inodes": 0, 2024-03-21T10:36:03.989 INFO:teuthology.orchestra.run.smithi100.stdout: "scrub_dirfrag_rstats": 0, 2024-03-21T10:36:03.989 INFO:teuthology.orchestra.run.smithi100.stdout: "scrub_file_inodes": 0, 2024-03-21T10:36:03.989 INFO:teuthology.orchestra.run.smithi100.stdout: "handle_inode_file_caps": 0, 2024-03-21T10:36:03.989 INFO:teuthology.orchestra.run.smithi100.stdout: "ceph_cap_op_revoke": 1, 2024-03-21T10:36:03.989 INFO:teuthology.orchestra.run.smithi100.stdout: "ceph_cap_op_grant": 0, 2024-03-21T10:36:03.989 INFO:teuthology.orchestra.run.smithi100.stdout: "ceph_cap_op_trunc": 0, 2024-03-21T10:36:03.989 INFO:teuthology.orchestra.run.smithi100.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-03-21T10:36:03.989 INFO:teuthology.orchestra.run.smithi100.stdout: "ceph_cap_op_flush_ack": 0, 2024-03-21T10:36:03.989 INFO:teuthology.orchestra.run.smithi100.stdout: "handle_client_caps": 1, 2024-03-21T10:36:03.990 INFO:teuthology.orchestra.run.smithi100.stdout: "handle_client_caps_dirty": 0, 2024-03-21T10:36:03.990 INFO:teuthology.orchestra.run.smithi100.stdout: "handle_client_cap_release": 0, 2024-03-21T10:36:03.990 INFO:teuthology.orchestra.run.smithi100.stdout: "process_request_cap_release": 1 2024-03-21T10:36:03.990 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-21T10:36:03.990 INFO:teuthology.orchestra.run.smithi100.stdout: "mds_cache": { 2024-03-21T10:36:03.990 INFO:teuthology.orchestra.run.smithi100.stdout: "dir_update": 0, 2024-03-21T10:36:03.990 INFO:teuthology.orchestra.run.smithi100.stdout: "dir_update_receipt": 0, 2024-03-21T10:36:03.990 INFO:teuthology.orchestra.run.smithi100.stdout: "dir_try_discover": 0, 2024-03-21T10:36:03.990 INFO:teuthology.orchestra.run.smithi100.stdout: "dir_send_discover": 3, 2024-03-21T10:36:03.990 INFO:teuthology.orchestra.run.smithi100.stdout: "dir_handle_discover": 0, 2024-03-21T10:36:03.990 INFO:teuthology.orchestra.run.smithi100.stdout: "num_strays": 0, 2024-03-21T10:36:03.990 INFO:teuthology.orchestra.run.smithi100.stdout: "num_strays_delayed": 0, 2024-03-21T10:36:03.990 INFO:teuthology.orchestra.run.smithi100.stdout: "num_strays_enqueuing": 0, 2024-03-21T10:36:03.990 INFO:teuthology.orchestra.run.smithi100.stdout: "strays_created": 0, 2024-03-21T10:36:03.991 INFO:teuthology.orchestra.run.smithi100.stdout: "strays_enqueued": 0, 2024-03-21T10:36:03.991 INFO:teuthology.orchestra.run.smithi100.stdout: "strays_reintegrated": 0, 2024-03-21T10:36:03.991 INFO:teuthology.orchestra.run.smithi100.stdout: "strays_migrated": 0, 2024-03-21T10:36:03.991 INFO:teuthology.orchestra.run.smithi100.stdout: "num_recovering_processing": 0, 2024-03-21T10:36:03.991 INFO:teuthology.orchestra.run.smithi100.stdout: "num_recovering_enqueued": 0, 2024-03-21T10:36:03.991 INFO:teuthology.orchestra.run.smithi100.stdout: "num_recovering_prioritized": 0, 2024-03-21T10:36:03.991 INFO:teuthology.orchestra.run.smithi100.stdout: "recovery_started": 0, 2024-03-21T10:36:03.991 INFO:teuthology.orchestra.run.smithi100.stdout: "recovery_completed": 0, 2024-03-21T10:36:03.991 INFO:teuthology.orchestra.run.smithi100.stdout: "ireq_enqueue_scrub": 0, 2024-03-21T10:36:03.991 INFO:teuthology.orchestra.run.smithi100.stdout: "ireq_exportdir": 0, 2024-03-21T10:36:03.991 INFO:teuthology.orchestra.run.smithi100.stdout: "ireq_flush": 0, 2024-03-21T10:36:03.991 INFO:teuthology.orchestra.run.smithi100.stdout: "ireq_fragmentdir": 0, 2024-03-21T10:36:03.991 INFO:teuthology.orchestra.run.smithi100.stdout: "ireq_fragstats": 0, 2024-03-21T10:36:03.991 INFO:teuthology.orchestra.run.smithi100.stdout: "ireq_inodestats": 0 2024-03-21T10:36:03.991 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-21T10:36:03.992 INFO:teuthology.orchestra.run.smithi100.stdout: "mds_log": { 2024-03-21T10:36:03.992 INFO:teuthology.orchestra.run.smithi100.stdout: "evlrg": 0, 2024-03-21T10:36:03.992 INFO:teuthology.orchestra.run.smithi100.stdout: "evadd": 8, 2024-03-21T10:36:03.992 INFO:teuthology.orchestra.run.smithi100.stdout: "evex": 0, 2024-03-21T10:36:03.992 INFO:teuthology.orchestra.run.smithi100.stdout: "evtrm": 0, 2024-03-21T10:36:03.992 INFO:teuthology.orchestra.run.smithi100.stdout: "ev": 8, 2024-03-21T10:36:03.992 INFO:teuthology.orchestra.run.smithi100.stdout: "evexg": 0, 2024-03-21T10:36:03.992 INFO:teuthology.orchestra.run.smithi100.stdout: "evexd": 0, 2024-03-21T10:36:03.992 INFO:teuthology.orchestra.run.smithi100.stdout: "segadd": 2, 2024-03-21T10:36:03.992 INFO:teuthology.orchestra.run.smithi100.stdout: "segex": 0, 2024-03-21T10:36:03.992 INFO:teuthology.orchestra.run.smithi100.stdout: "segtrm": 0, 2024-03-21T10:36:03.992 INFO:teuthology.orchestra.run.smithi100.stdout: "seg": 2, 2024-03-21T10:36:03.992 INFO:teuthology.orchestra.run.smithi100.stdout: "segmjr": 2, 2024-03-21T10:36:03.992 INFO:teuthology.orchestra.run.smithi100.stdout: "segexg": 0, 2024-03-21T10:36:03.992 INFO:teuthology.orchestra.run.smithi100.stdout: "segexd": 0, 2024-03-21T10:36:03.992 INFO:teuthology.orchestra.run.smithi100.stdout: "expos": 4194304, 2024-03-21T10:36:03.993 INFO:teuthology.orchestra.run.smithi100.stdout: "wrpos": 4196840, 2024-03-21T10:36:03.993 INFO:teuthology.orchestra.run.smithi100.stdout: "rdpos": 0, 2024-03-21T10:36:03.993 INFO:teuthology.orchestra.run.smithi100.stdout: "jlat": { 2024-03-21T10:36:03.993 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 7, 2024-03-21T10:36:03.993 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.031047809, 2024-03-21T10:36:03.993 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.004435401 2024-03-21T10:36:03.993 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-21T10:36:03.993 INFO:teuthology.orchestra.run.smithi100.stdout: "replayed": 0 2024-03-21T10:36:03.993 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-21T10:36:03.993 INFO:teuthology.orchestra.run.smithi100.stdout: "mds_mem": { 2024-03-21T10:36:03.993 INFO:teuthology.orchestra.run.smithi100.stdout: "ino": 17, 2024-03-21T10:36:03.993 INFO:teuthology.orchestra.run.smithi100.stdout: "ino+": 17, 2024-03-21T10:36:03.993 INFO:teuthology.orchestra.run.smithi100.stdout: "ino-": 0, 2024-03-21T10:36:03.993 INFO:teuthology.orchestra.run.smithi100.stdout: "dir": 15, 2024-03-21T10:36:03.993 INFO:teuthology.orchestra.run.smithi100.stdout: "dir+": 16, 2024-03-21T10:36:03.994 INFO:teuthology.orchestra.run.smithi100.stdout: "dir-": 1, 2024-03-21T10:36:03.994 INFO:teuthology.orchestra.run.smithi100.stdout: "dn": 14, 2024-03-21T10:36:03.994 INFO:teuthology.orchestra.run.smithi100.stdout: "dn+": 15, 2024-03-21T10:36:03.994 INFO:teuthology.orchestra.run.smithi100.stdout: "dn-": 1, 2024-03-21T10:36:03.994 INFO:teuthology.orchestra.run.smithi100.stdout: "cap": 1, 2024-03-21T10:36:03.994 INFO:teuthology.orchestra.run.smithi100.stdout: "cap+": 3, 2024-03-21T10:36:03.994 INFO:teuthology.orchestra.run.smithi100.stdout: "cap-": 2, 2024-03-21T10:36:03.994 INFO:teuthology.orchestra.run.smithi100.stdout: "rss": 51584, 2024-03-21T10:36:03.994 INFO:teuthology.orchestra.run.smithi100.stdout: "heap": 223612 2024-03-21T10:36:03.994 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-21T10:36:03.994 INFO:teuthology.orchestra.run.smithi100.stdout: "mds_server": { 2024-03-21T10:36:03.994 INFO:teuthology.orchestra.run.smithi100.stdout: "dispatch_client_request": 8, 2024-03-21T10:36:03.994 INFO:teuthology.orchestra.run.smithi100.stdout: "dispatch_server_request": 0, 2024-03-21T10:36:03.994 INFO:teuthology.orchestra.run.smithi100.stdout: "handle_client_request": 6, 2024-03-21T10:36:03.995 INFO:teuthology.orchestra.run.smithi100.stdout: "handle_client_session": 26, 2024-03-21T10:36:03.995 INFO:teuthology.orchestra.run.smithi100.stdout: "handle_peer_request": 0, 2024-03-21T10:36:03.995 INFO:teuthology.orchestra.run.smithi100.stdout: "req_create_latency": { 2024-03-21T10:36:03.995 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 0, 2024-03-21T10:36:03.995 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.000000000, 2024-03-21T10:36:03.995 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000000000 2024-03-21T10:36:03.995 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-21T10:36:03.995 INFO:teuthology.orchestra.run.smithi100.stdout: "req_getattr_latency": { 2024-03-21T10:36:03.995 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 2, 2024-03-21T10:36:03.995 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.001939641, 2024-03-21T10:36:03.995 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000969820 2024-03-21T10:36:03.995 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-21T10:36:03.995 INFO:teuthology.orchestra.run.smithi100.stdout: "req_getfilelock_latency": { 2024-03-21T10:36:03.995 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 0, 2024-03-21T10:36:03.995 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.000000000, 2024-03-21T10:36:03.996 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000000000 2024-03-21T10:36:03.996 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-21T10:36:03.996 INFO:teuthology.orchestra.run.smithi100.stdout: "req_link_latency": { 2024-03-21T10:36:03.996 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 0, 2024-03-21T10:36:03.996 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.000000000, 2024-03-21T10:36:03.996 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000000000 2024-03-21T10:36:03.996 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-21T10:36:03.996 INFO:teuthology.orchestra.run.smithi100.stdout: "req_lookup_latency": { 2024-03-21T10:36:03.996 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 2, 2024-03-21T10:36:03.996 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.001045139, 2024-03-21T10:36:03.996 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000522569 2024-03-21T10:36:03.996 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-21T10:36:03.996 INFO:teuthology.orchestra.run.smithi100.stdout: "req_lookuphash_latency": { 2024-03-21T10:36:03.996 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 0, 2024-03-21T10:36:03.996 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.000000000, 2024-03-21T10:36:03.997 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000000000 2024-03-21T10:36:03.997 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-21T10:36:03.997 INFO:teuthology.orchestra.run.smithi100.stdout: "req_lookupino_latency": { 2024-03-21T10:36:03.997 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 0, 2024-03-21T10:36:03.997 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.000000000, 2024-03-21T10:36:03.997 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000000000 2024-03-21T10:36:03.997 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-21T10:36:03.997 INFO:teuthology.orchestra.run.smithi100.stdout: "req_lookupname_latency": { 2024-03-21T10:36:03.997 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 0, 2024-03-21T10:36:03.997 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.000000000, 2024-03-21T10:36:03.997 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000000000 2024-03-21T10:36:03.997 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-21T10:36:03.997 INFO:teuthology.orchestra.run.smithi100.stdout: "req_lookupparent_latency": { 2024-03-21T10:36:03.997 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 0, 2024-03-21T10:36:03.997 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.000000000, 2024-03-21T10:36:03.997 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000000000 2024-03-21T10:36:03.998 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-21T10:36:03.998 INFO:teuthology.orchestra.run.smithi100.stdout: "req_lookupsnap_latency": { 2024-03-21T10:36:03.998 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 0, 2024-03-21T10:36:03.998 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.000000000, 2024-03-21T10:36:03.998 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000000000 2024-03-21T10:36:03.998 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-21T10:36:03.998 INFO:teuthology.orchestra.run.smithi100.stdout: "req_lssnap_latency": { 2024-03-21T10:36:03.998 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 0, 2024-03-21T10:36:03.998 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.000000000, 2024-03-21T10:36:03.998 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000000000 2024-03-21T10:36:03.998 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-21T10:36:03.998 INFO:teuthology.orchestra.run.smithi100.stdout: "req_mkdir_latency": { 2024-03-21T10:36:03.998 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 0, 2024-03-21T10:36:03.998 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.000000000, 2024-03-21T10:36:03.998 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000000000 2024-03-21T10:36:03.999 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:36:03 smithi157 ceph-mon[35581]: pgmap v333: 129 pgs: 129 active+clean; 2.5 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 33 KiB/s wr, 28 op/s 2024-03-21T10:36:03.999 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-21T10:36:03.999 INFO:teuthology.orchestra.run.smithi100.stdout: "req_mknod_latency": { 2024-03-21T10:36:04.000 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 0, 2024-03-21T10:36:04.000 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.000000000, 2024-03-21T10:36:04.000 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000000000 2024-03-21T10:36:04.000 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-21T10:36:04.000 INFO:teuthology.orchestra.run.smithi100.stdout: "req_mksnap_latency": { 2024-03-21T10:36:04.000 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 0, 2024-03-21T10:36:04.000 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.000000000, 2024-03-21T10:36:04.000 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000000000 2024-03-21T10:36:04.000 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-21T10:36:04.000 INFO:teuthology.orchestra.run.smithi100.stdout: "req_open_latency": { 2024-03-21T10:36:04.000 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 0, 2024-03-21T10:36:04.000 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.000000000, 2024-03-21T10:36:04.000 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000000000 2024-03-21T10:36:04.000 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-21T10:36:04.001 INFO:teuthology.orchestra.run.smithi100.stdout: "req_readdir_latency": { 2024-03-21T10:36:04.001 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 0, 2024-03-21T10:36:04.001 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.000000000, 2024-03-21T10:36:04.001 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000000000 2024-03-21T10:36:04.001 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-21T10:36:04.001 INFO:teuthology.orchestra.run.smithi100.stdout: "req_rename_latency": { 2024-03-21T10:36:04.001 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 0, 2024-03-21T10:36:04.001 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.000000000, 2024-03-21T10:36:04.001 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000000000 2024-03-21T10:36:04.001 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-21T10:36:04.001 INFO:teuthology.orchestra.run.smithi100.stdout: "req_renamesnap_latency": { 2024-03-21T10:36:04.001 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 0, 2024-03-21T10:36:04.001 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.000000000, 2024-03-21T10:36:04.001 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000000000 2024-03-21T10:36:04.001 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-21T10:36:04.002 INFO:teuthology.orchestra.run.smithi100.stdout: "req_snapdiff_latency": { 2024-03-21T10:36:04.002 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 0, 2024-03-21T10:36:04.002 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.000000000, 2024-03-21T10:36:04.002 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000000000 2024-03-21T10:36:04.002 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-21T10:36:04.002 INFO:teuthology.orchestra.run.smithi100.stdout: "req_rmdir_latency": { 2024-03-21T10:36:04.002 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 0, 2024-03-21T10:36:04.002 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.000000000, 2024-03-21T10:36:04.002 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000000000 2024-03-21T10:36:04.002 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-21T10:36:04.002 INFO:teuthology.orchestra.run.smithi100.stdout: "req_rmsnap_latency": { 2024-03-21T10:36:04.002 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 0, 2024-03-21T10:36:04.002 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.000000000, 2024-03-21T10:36:04.002 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000000000 2024-03-21T10:36:04.003 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-21T10:36:04.003 INFO:teuthology.orchestra.run.smithi100.stdout: "req_rmxattr_latency": { 2024-03-21T10:36:04.003 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 0, 2024-03-21T10:36:04.003 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.000000000, 2024-03-21T10:36:04.003 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000000000 2024-03-21T10:36:04.003 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-21T10:36:04.003 INFO:teuthology.orchestra.run.smithi100.stdout: "req_setattr_latency": { 2024-03-21T10:36:04.003 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 0, 2024-03-21T10:36:04.003 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.000000000, 2024-03-21T10:36:04.003 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000000000 2024-03-21T10:36:04.003 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-21T10:36:04.003 INFO:teuthology.orchestra.run.smithi100.stdout: "req_setdirlayout_latency": { 2024-03-21T10:36:04.003 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 0, 2024-03-21T10:36:04.003 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.000000000, 2024-03-21T10:36:04.004 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000000000 2024-03-21T10:36:04.004 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-21T10:36:04.004 INFO:teuthology.orchestra.run.smithi100.stdout: "req_setfilelock_latency": { 2024-03-21T10:36:04.004 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 0, 2024-03-21T10:36:04.004 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.000000000, 2024-03-21T10:36:04.004 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000000000 2024-03-21T10:36:04.004 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-21T10:36:04.004 INFO:teuthology.orchestra.run.smithi100.stdout: "req_setlayout_latency": { 2024-03-21T10:36:04.004 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 0, 2024-03-21T10:36:04.004 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.000000000, 2024-03-21T10:36:04.004 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000000000 2024-03-21T10:36:04.004 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-21T10:36:04.004 INFO:teuthology.orchestra.run.smithi100.stdout: "req_setxattr_latency": { 2024-03-21T10:36:04.004 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 0, 2024-03-21T10:36:04.004 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.000000000, 2024-03-21T10:36:04.005 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000000000 2024-03-21T10:36:04.005 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-21T10:36:04.005 INFO:teuthology.orchestra.run.smithi100.stdout: "req_symlink_latency": { 2024-03-21T10:36:04.005 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 0, 2024-03-21T10:36:04.005 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.000000000, 2024-03-21T10:36:04.005 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000000000 2024-03-21T10:36:04.005 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-21T10:36:04.005 INFO:teuthology.orchestra.run.smithi100.stdout: "req_unlink_latency": { 2024-03-21T10:36:04.005 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 0, 2024-03-21T10:36:04.005 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.000000000, 2024-03-21T10:36:04.005 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000000000 2024-03-21T10:36:04.005 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-21T10:36:04.005 INFO:teuthology.orchestra.run.smithi100.stdout: "cap_revoke_eviction": 0, 2024-03-21T10:36:04.005 INFO:teuthology.orchestra.run.smithi100.stdout: "cap_acquisition_throttle": 0, 2024-03-21T10:36:04.006 INFO:teuthology.orchestra.run.smithi100.stdout: "req_getvxattr_latency": { 2024-03-21T10:36:04.006 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 0, 2024-03-21T10:36:04.006 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.000000000, 2024-03-21T10:36:04.006 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000000000 2024-03-21T10:36:04.006 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-03-21T10:36:04.006 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-21T10:36:04.006 INFO:teuthology.orchestra.run.smithi100.stdout: "mds_sessions": { 2024-03-21T10:36:04.006 INFO:teuthology.orchestra.run.smithi100.stdout: "session_count": 2, 2024-03-21T10:36:04.006 INFO:teuthology.orchestra.run.smithi100.stdout: "session_add": 4, 2024-03-21T10:36:04.006 INFO:teuthology.orchestra.run.smithi100.stdout: "session_remove": 2, 2024-03-21T10:36:04.006 INFO:teuthology.orchestra.run.smithi100.stdout: "sessions_open": 2, 2024-03-21T10:36:04.006 INFO:teuthology.orchestra.run.smithi100.stdout: "sessions_stale": 0, 2024-03-21T10:36:04.006 INFO:teuthology.orchestra.run.smithi100.stdout: "total_load": 3, 2024-03-21T10:36:04.006 INFO:teuthology.orchestra.run.smithi100.stdout: "average_load": 1, 2024-03-21T10:36:04.007 INFO:teuthology.orchestra.run.smithi100.stdout: "avg_session_uptime": 634, 2024-03-21T10:36:04.007 INFO:teuthology.orchestra.run.smithi100.stdout: "mdthresh_evicted": 0 2024-03-21T10:36:04.007 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-21T10:36:04.007 INFO:teuthology.orchestra.run.smithi100.stdout: "mempool": { 2024-03-21T10:36:04.007 INFO:teuthology.orchestra.run.smithi100.stdout: "bloom_filter_bytes": 0, 2024-03-21T10:36:04.007 INFO:teuthology.orchestra.run.smithi100.stdout: "bloom_filter_items": 0, 2024-03-21T10:36:04.007 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_alloc_bytes": 0, 2024-03-21T10:36:04.007 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_alloc_items": 0, 2024-03-21T10:36:04.007 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_cache_data_bytes": 0, 2024-03-21T10:36:04.007 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_cache_data_items": 0, 2024-03-21T10:36:04.007 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_cache_onode_bytes": 0, 2024-03-21T10:36:04.007 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_cache_onode_items": 0, 2024-03-21T10:36:04.007 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_cache_meta_bytes": 0, 2024-03-21T10:36:04.007 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_cache_meta_items": 0, 2024-03-21T10:36:04.007 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_cache_other_bytes": 0, 2024-03-21T10:36:04.008 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_cache_other_items": 0, 2024-03-21T10:36:04.008 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_cache_buffer_bytes": 0, 2024-03-21T10:36:04.008 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_cache_buffer_items": 0, 2024-03-21T10:36:04.008 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_extent_bytes": 0, 2024-03-21T10:36:04.008 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_extent_items": 0, 2024-03-21T10:36:04.008 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_blob_bytes": 0, 2024-03-21T10:36:04.008 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_blob_items": 0, 2024-03-21T10:36:04.008 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_shared_blob_bytes": 0, 2024-03-21T10:36:04.008 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_shared_blob_items": 0, 2024-03-21T10:36:04.008 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_inline_bl_bytes": 0, 2024-03-21T10:36:04.008 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_inline_bl_items": 0, 2024-03-21T10:36:04.008 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_fsck_bytes": 0, 2024-03-21T10:36:04.008 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_fsck_items": 0, 2024-03-21T10:36:04.008 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_txc_bytes": 0, 2024-03-21T10:36:04.009 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_txc_items": 0, 2024-03-21T10:36:04.009 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_writing_deferred_bytes": 0, 2024-03-21T10:36:04.009 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_writing_deferred_items": 0, 2024-03-21T10:36:04.009 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_writing_bytes": 0, 2024-03-21T10:36:04.009 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_writing_items": 0, 2024-03-21T10:36:04.009 INFO:teuthology.orchestra.run.smithi100.stdout: "bluefs_bytes": 0, 2024-03-21T10:36:04.009 INFO:teuthology.orchestra.run.smithi100.stdout: "bluefs_items": 0, 2024-03-21T10:36:04.009 INFO:teuthology.orchestra.run.smithi100.stdout: "bluefs_file_reader_bytes": 0, 2024-03-21T10:36:04.009 INFO:teuthology.orchestra.run.smithi100.stdout: "bluefs_file_reader_items": 0, 2024-03-21T10:36:04.009 INFO:teuthology.orchestra.run.smithi100.stdout: "bluefs_file_writer_bytes": 0, 2024-03-21T10:36:04.009 INFO:teuthology.orchestra.run.smithi100.stdout: "bluefs_file_writer_items": 0, 2024-03-21T10:36:04.009 INFO:teuthology.orchestra.run.smithi100.stdout: "buffer_anon_bytes": 272857, 2024-03-21T10:36:04.009 INFO:teuthology.orchestra.run.smithi100.stdout: "buffer_anon_items": 104, 2024-03-21T10:36:04.009 INFO:teuthology.orchestra.run.smithi100.stdout: "buffer_meta_bytes": 0, 2024-03-21T10:36:04.009 INFO:teuthology.orchestra.run.smithi100.stdout: "buffer_meta_items": 0, 2024-03-21T10:36:04.009 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_bytes": 0, 2024-03-21T10:36:04.010 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_items": 0, 2024-03-21T10:36:04.010 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_mapbl_bytes": 0, 2024-03-21T10:36:04.010 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_mapbl_items": 0, 2024-03-21T10:36:04.010 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_pglog_bytes": 0, 2024-03-21T10:36:04.010 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_pglog_items": 0, 2024-03-21T10:36:04.010 INFO:teuthology.orchestra.run.smithi100.stdout: "osdmap_bytes": 9712, 2024-03-21T10:36:04.010 INFO:teuthology.orchestra.run.smithi100.stdout: "osdmap_items": 187, 2024-03-21T10:36:04.010 INFO:teuthology.orchestra.run.smithi100.stdout: "osdmap_mapping_bytes": 0, 2024-03-21T10:36:04.010 INFO:teuthology.orchestra.run.smithi100.stdout: "osdmap_mapping_items": 0, 2024-03-21T10:36:04.010 INFO:teuthology.orchestra.run.smithi100.stdout: "pgmap_bytes": 0, 2024-03-21T10:36:04.010 INFO:teuthology.orchestra.run.smithi100.stdout: "pgmap_items": 0, 2024-03-21T10:36:04.010 INFO:teuthology.orchestra.run.smithi100.stdout: "mds_co_bytes": 67013, 2024-03-21T10:36:04.010 INFO:teuthology.orchestra.run.smithi100.stdout: "mds_co_items": 296, 2024-03-21T10:36:04.010 INFO:teuthology.orchestra.run.smithi100.stdout: "unittest_1_bytes": 0, 2024-03-21T10:36:04.011 INFO:teuthology.orchestra.run.smithi100.stdout: "unittest_1_items": 0, 2024-03-21T10:36:04.011 INFO:teuthology.orchestra.run.smithi100.stdout: "unittest_2_bytes": 0, 2024-03-21T10:36:04.011 INFO:teuthology.orchestra.run.smithi100.stdout: "unittest_2_items": 0 2024-03-21T10:36:04.011 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-21T10:36:04.011 INFO:teuthology.orchestra.run.smithi100.stdout: "objecter": { 2024-03-21T10:36:04.011 INFO:teuthology.orchestra.run.smithi100.stdout: "op_active": 0, 2024-03-21T10:36:04.011 INFO:teuthology.orchestra.run.smithi100.stdout: "op_laggy": 0, 2024-03-21T10:36:04.011 INFO:teuthology.orchestra.run.smithi100.stdout: "op_send": 48, 2024-03-21T10:36:04.011 INFO:teuthology.orchestra.run.smithi100.stdout: "op_send_bytes": 13471, 2024-03-21T10:36:04.011 INFO:teuthology.orchestra.run.smithi100.stdout: "op_resend": 33, 2024-03-21T10:36:04.011 INFO:teuthology.orchestra.run.smithi100.stdout: "op_reply": 48, 2024-03-21T10:36:04.011 INFO:teuthology.orchestra.run.smithi100.stdout: "op_latency": { 2024-03-21T10:36:04.011 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 48, 2024-03-21T10:36:04.011 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.602984169, 2024-03-21T10:36:04.011 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.012562170 2024-03-21T10:36:04.011 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-21T10:36:04.012 INFO:teuthology.orchestra.run.smithi100.stdout: "op_inflight": 0, 2024-03-21T10:36:04.012 INFO:teuthology.orchestra.run.smithi100.stdout: "oplen_avg": { 2024-03-21T10:36:04.012 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 48, 2024-03-21T10:36:04.012 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 73 2024-03-21T10:36:04.012 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-21T10:36:04.012 INFO:teuthology.orchestra.run.smithi100.stdout: "op": 48, 2024-03-21T10:36:04.012 INFO:teuthology.orchestra.run.smithi100.stdout: "op_r": 0, 2024-03-21T10:36:04.012 INFO:teuthology.orchestra.run.smithi100.stdout: "op_w": 48, 2024-03-21T10:36:04.012 INFO:teuthology.orchestra.run.smithi100.stdout: "op_rmw": 0, 2024-03-21T10:36:04.012 INFO:teuthology.orchestra.run.smithi100.stdout: "op_pg": 0, 2024-03-21T10:36:04.012 INFO:teuthology.orchestra.run.smithi100.stdout: "osdop_stat": 0, 2024-03-21T10:36:04.012 INFO:teuthology.orchestra.run.smithi100.stdout: "osdop_create": 10, 2024-03-21T10:36:04.012 INFO:teuthology.orchestra.run.smithi100.stdout: "osdop_read": 0, 2024-03-21T10:36:04.012 INFO:teuthology.orchestra.run.smithi100.stdout: "osdop_write": 7, 2024-03-21T10:36:04.012 INFO:teuthology.orchestra.run.smithi100.stdout: "osdop_writefull": 9, 2024-03-21T10:36:04.012 INFO:teuthology.orchestra.run.smithi100.stdout: "osdop_writesame": 0, 2024-03-21T10:36:04.013 INFO:teuthology.orchestra.run.smithi100.stdout: "osdop_append": 0, 2024-03-21T10:36:04.013 INFO:teuthology.orchestra.run.smithi100.stdout: "osdop_zero": 0, 2024-03-21T10:36:04.013 INFO:teuthology.orchestra.run.smithi100.stdout: "osdop_truncate": 0, 2024-03-21T10:36:04.013 INFO:teuthology.orchestra.run.smithi100.stdout: "osdop_delete": 6, 2024-03-21T10:36:04.013 INFO:teuthology.orchestra.run.smithi100.stdout: "osdop_mapext": 0, 2024-03-21T10:36:04.013 INFO:teuthology.orchestra.run.smithi100.stdout: "osdop_sparse_read": 0, 2024-03-21T10:36:04.013 INFO:teuthology.orchestra.run.smithi100.stdout: "osdop_clonerange": 0, 2024-03-21T10:36:04.013 INFO:teuthology.orchestra.run.smithi100.stdout: "osdop_getxattr": 0, 2024-03-21T10:36:04.013 INFO:teuthology.orchestra.run.smithi100.stdout: "osdop_setxattr": 20, 2024-03-21T10:36:04.013 INFO:teuthology.orchestra.run.smithi100.stdout: "osdop_cmpxattr": 0, 2024-03-21T10:36:04.013 INFO:teuthology.orchestra.run.smithi100.stdout: "osdop_rmxattr": 0, 2024-03-21T10:36:04.013 INFO:teuthology.orchestra.run.smithi100.stdout: "osdop_resetxattrs": 0, 2024-03-21T10:36:04.013 INFO:teuthology.orchestra.run.smithi100.stdout: "osdop_call": 0, 2024-03-21T10:36:04.013 INFO:teuthology.orchestra.run.smithi100.stdout: "osdop_watch": 0, 2024-03-21T10:36:04.013 INFO:teuthology.orchestra.run.smithi100.stdout: "osdop_notify": 0, 2024-03-21T10:36:04.014 INFO:teuthology.orchestra.run.smithi100.stdout: "osdop_src_cmpxattr": 0, 2024-03-21T10:36:04.014 INFO:teuthology.orchestra.run.smithi100.stdout: "osdop_pgls": 0, 2024-03-21T10:36:04.014 INFO:teuthology.orchestra.run.smithi100.stdout: "osdop_pgls_filter": 0, 2024-03-21T10:36:04.014 INFO:teuthology.orchestra.run.smithi100.stdout: "osdop_other": 0, 2024-03-21T10:36:04.014 INFO:teuthology.orchestra.run.smithi100.stdout: "linger_active": 0, 2024-03-21T10:36:04.014 INFO:teuthology.orchestra.run.smithi100.stdout: "linger_send": 0, 2024-03-21T10:36:04.014 INFO:teuthology.orchestra.run.smithi100.stdout: "linger_resend": 0, 2024-03-21T10:36:04.014 INFO:teuthology.orchestra.run.smithi100.stdout: "linger_ping": 0, 2024-03-21T10:36:04.014 INFO:teuthology.orchestra.run.smithi100.stdout: "poolop_active": 0, 2024-03-21T10:36:04.014 INFO:teuthology.orchestra.run.smithi100.stdout: "poolop_send": 0, 2024-03-21T10:36:04.014 INFO:teuthology.orchestra.run.smithi100.stdout: "poolop_resend": 0, 2024-03-21T10:36:04.014 INFO:teuthology.orchestra.run.smithi100.stdout: "poolstat_active": 0, 2024-03-21T10:36:04.014 INFO:teuthology.orchestra.run.smithi100.stdout: "poolstat_send": 0, 2024-03-21T10:36:04.014 INFO:teuthology.orchestra.run.smithi100.stdout: "poolstat_resend": 0, 2024-03-21T10:36:04.014 INFO:teuthology.orchestra.run.smithi100.stdout: "statfs_active": 0, 2024-03-21T10:36:04.015 INFO:teuthology.orchestra.run.smithi100.stdout: "statfs_send": 0, 2024-03-21T10:36:04.015 INFO:teuthology.orchestra.run.smithi100.stdout: "statfs_resend": 0, 2024-03-21T10:36:04.015 INFO:teuthology.orchestra.run.smithi100.stdout: "command_active": 0, 2024-03-21T10:36:04.015 INFO:teuthology.orchestra.run.smithi100.stdout: "command_send": 0, 2024-03-21T10:36:04.015 INFO:teuthology.orchestra.run.smithi100.stdout: "command_resend": 0, 2024-03-21T10:36:04.015 INFO:teuthology.orchestra.run.smithi100.stdout: "map_epoch": 105, 2024-03-21T10:36:04.015 INFO:teuthology.orchestra.run.smithi100.stdout: "map_full": 0, 2024-03-21T10:36:04.015 INFO:teuthology.orchestra.run.smithi100.stdout: "map_inc": 12, 2024-03-21T10:36:04.015 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_sessions": 9, 2024-03-21T10:36:04.015 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_session_open": 9, 2024-03-21T10:36:04.015 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_session_close": 0, 2024-03-21T10:36:04.015 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_laggy": 0, 2024-03-21T10:36:04.015 INFO:teuthology.orchestra.run.smithi100.stdout: "omap_wr": 20, 2024-03-21T10:36:04.015 INFO:teuthology.orchestra.run.smithi100.stdout: "omap_rd": 0, 2024-03-21T10:36:04.016 INFO:teuthology.orchestra.run.smithi100.stdout: "omap_del": 1 2024-03-21T10:36:04.016 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-21T10:36:04.016 INFO:teuthology.orchestra.run.smithi100.stdout: "oft": { 2024-03-21T10:36:04.016 INFO:teuthology.orchestra.run.smithi100.stdout: "omap_total_objs": 1, 2024-03-21T10:36:04.016 INFO:teuthology.orchestra.run.smithi100.stdout: "omap_total_kv_pairs": 0, 2024-03-21T10:36:04.016 INFO:teuthology.orchestra.run.smithi100.stdout: "omap_total_updates": 0, 2024-03-21T10:36:04.016 INFO:teuthology.orchestra.run.smithi100.stdout: "omap_total_removes": 0 2024-03-21T10:36:04.016 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-21T10:36:04.016 INFO:teuthology.orchestra.run.smithi100.stdout: "osd-slow-ops": { 2024-03-21T10:36:04.016 INFO:teuthology.orchestra.run.smithi100.stdout: "slow_ops_count": 0 2024-03-21T10:36:04.016 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-21T10:36:04.016 INFO:teuthology.orchestra.run.smithi100.stdout: "purge_queue": { 2024-03-21T10:36:04.016 INFO:teuthology.orchestra.run.smithi100.stdout: "pq_executing_ops": 0, 2024-03-21T10:36:04.016 INFO:teuthology.orchestra.run.smithi100.stdout: "pq_executing_ops_high_water": 0, 2024-03-21T10:36:04.016 INFO:teuthology.orchestra.run.smithi100.stdout: "pq_executing": 0, 2024-03-21T10:36:04.017 INFO:teuthology.orchestra.run.smithi100.stdout: "pq_executing_high_water": 0, 2024-03-21T10:36:04.017 INFO:teuthology.orchestra.run.smithi100.stdout: "pq_executed_ops": 0, 2024-03-21T10:36:04.017 INFO:teuthology.orchestra.run.smithi100.stdout: "pq_executed": 0, 2024-03-21T10:36:04.017 INFO:teuthology.orchestra.run.smithi100.stdout: "pq_item_in_journal": 0 2024-03-21T10:36:04.017 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-21T10:36:04.017 INFO:teuthology.orchestra.run.smithi100.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-03-21T10:36:04.017 INFO:teuthology.orchestra.run.smithi100.stdout: "val": 0, 2024-03-21T10:36:04.017 INFO:teuthology.orchestra.run.smithi100.stdout: "max": 104857600, 2024-03-21T10:36:04.017 INFO:teuthology.orchestra.run.smithi100.stdout: "get_started": 0, 2024-03-21T10:36:04.017 INFO:teuthology.orchestra.run.smithi100.stdout: "get": 1074, 2024-03-21T10:36:04.017 INFO:teuthology.orchestra.run.smithi100.stdout: "get_sum": 1933210, 2024-03-21T10:36:04.017 INFO:teuthology.orchestra.run.smithi100.stdout: "get_or_fail_fail": 0, 2024-03-21T10:36:04.017 INFO:teuthology.orchestra.run.smithi100.stdout: "get_or_fail_success": 1074, 2024-03-21T10:36:04.017 INFO:teuthology.orchestra.run.smithi100.stdout: "take": 0, 2024-03-21T10:36:04.017 INFO:teuthology.orchestra.run.smithi100.stdout: "take_sum": 0, 2024-03-21T10:36:04.017 INFO:teuthology.orchestra.run.smithi100.stdout: "put": 1074, 2024-03-21T10:36:04.018 INFO:teuthology.orchestra.run.smithi100.stdout: "put_sum": 1933210, 2024-03-21T10:36:04.018 INFO:teuthology.orchestra.run.smithi100.stdout: "wait": { 2024-03-21T10:36:04.018 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 0, 2024-03-21T10:36:04.018 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.000000000, 2024-03-21T10:36:04.018 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000000000 2024-03-21T10:36:04.018 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-03-21T10:36:04.018 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-21T10:36:04.018 INFO:teuthology.orchestra.run.smithi100.stdout: "throttle-objecter_bytes": { 2024-03-21T10:36:04.018 INFO:teuthology.orchestra.run.smithi100.stdout: "val": 0, 2024-03-21T10:36:04.018 INFO:teuthology.orchestra.run.smithi100.stdout: "max": 104857600, 2024-03-21T10:36:04.018 INFO:teuthology.orchestra.run.smithi100.stdout: "get_started": 0, 2024-03-21T10:36:04.018 INFO:teuthology.orchestra.run.smithi100.stdout: "get": 0, 2024-03-21T10:36:04.018 INFO:teuthology.orchestra.run.smithi100.stdout: "get_sum": 0, 2024-03-21T10:36:04.018 INFO:teuthology.orchestra.run.smithi100.stdout: "get_or_fail_fail": 0, 2024-03-21T10:36:04.019 INFO:teuthology.orchestra.run.smithi100.stdout: "get_or_fail_success": 0, 2024-03-21T10:36:04.019 INFO:teuthology.orchestra.run.smithi100.stdout: "take": 48, 2024-03-21T10:36:04.019 INFO:teuthology.orchestra.run.smithi100.stdout: "take_sum": 13471, 2024-03-21T10:36:04.019 INFO:teuthology.orchestra.run.smithi100.stdout: "put": 48, 2024-03-21T10:36:04.019 INFO:teuthology.orchestra.run.smithi100.stdout: "put_sum": 13471, 2024-03-21T10:36:04.019 INFO:teuthology.orchestra.run.smithi100.stdout: "wait": { 2024-03-21T10:36:04.019 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 0, 2024-03-21T10:36:04.019 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.000000000, 2024-03-21T10:36:04.019 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000000000 2024-03-21T10:36:04.019 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-03-21T10:36:04.019 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-21T10:36:04.019 INFO:teuthology.orchestra.run.smithi100.stdout: "throttle-objecter_ops": { 2024-03-21T10:36:04.019 INFO:teuthology.orchestra.run.smithi100.stdout: "val": 0, 2024-03-21T10:36:04.019 INFO:teuthology.orchestra.run.smithi100.stdout: "max": 1024, 2024-03-21T10:36:04.019 INFO:teuthology.orchestra.run.smithi100.stdout: "get_started": 0, 2024-03-21T10:36:04.019 INFO:teuthology.orchestra.run.smithi100.stdout: "get": 0, 2024-03-21T10:36:04.020 INFO:teuthology.orchestra.run.smithi100.stdout: "get_sum": 0, 2024-03-21T10:36:04.020 INFO:teuthology.orchestra.run.smithi100.stdout: "get_or_fail_fail": 0, 2024-03-21T10:36:04.020 INFO:teuthology.orchestra.run.smithi100.stdout: "get_or_fail_success": 0, 2024-03-21T10:36:04.020 INFO:teuthology.orchestra.run.smithi100.stdout: "take": 48, 2024-03-21T10:36:04.020 INFO:teuthology.orchestra.run.smithi100.stdout: "take_sum": 48, 2024-03-21T10:36:04.020 INFO:teuthology.orchestra.run.smithi100.stdout: "put": 48, 2024-03-21T10:36:04.020 INFO:teuthology.orchestra.run.smithi100.stdout: "put_sum": 48, 2024-03-21T10:36:04.020 INFO:teuthology.orchestra.run.smithi100.stdout: "wait": { 2024-03-21T10:36:04.020 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 0, 2024-03-21T10:36:04.020 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.000000000, 2024-03-21T10:36:04.020 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000000000 2024-03-21T10:36:04.020 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-03-21T10:36:04.020 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-21T10:36:04.020 INFO:teuthology.orchestra.run.smithi100.stdout: "throttle-write_buf_throttle": { 2024-03-21T10:36:04.020 INFO:teuthology.orchestra.run.smithi100.stdout: "val": 0, 2024-03-21T10:36:04.020 INFO:teuthology.orchestra.run.smithi100.stdout: "max": 3758096384, 2024-03-21T10:36:04.021 INFO:teuthology.orchestra.run.smithi100.stdout: "get_started": 0, 2024-03-21T10:36:04.021 INFO:teuthology.orchestra.run.smithi100.stdout: "get": 0, 2024-03-21T10:36:04.021 INFO:teuthology.orchestra.run.smithi100.stdout: "get_sum": 0, 2024-03-21T10:36:04.021 INFO:teuthology.orchestra.run.smithi100.stdout: "get_or_fail_fail": 0, 2024-03-21T10:36:04.021 INFO:teuthology.orchestra.run.smithi100.stdout: "get_or_fail_success": 0, 2024-03-21T10:36:04.021 INFO:teuthology.orchestra.run.smithi100.stdout: "take": 0, 2024-03-21T10:36:04.021 INFO:teuthology.orchestra.run.smithi100.stdout: "take_sum": 0, 2024-03-21T10:36:04.021 INFO:teuthology.orchestra.run.smithi100.stdout: "put": 0, 2024-03-21T10:36:04.021 INFO:teuthology.orchestra.run.smithi100.stdout: "put_sum": 0, 2024-03-21T10:36:04.021 INFO:teuthology.orchestra.run.smithi100.stdout: "wait": { 2024-03-21T10:36:04.021 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 0, 2024-03-21T10:36:04.021 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.000000000, 2024-03-21T10:36:04.021 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000000000 2024-03-21T10:36:04.021 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-03-21T10:36:04.021 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-21T10:36:04.022 INFO:teuthology.orchestra.run.smithi100.stdout: "throttle-write_buf_throttle-0x56475815c500": { 2024-03-21T10:36:04.022 INFO:teuthology.orchestra.run.smithi100.stdout: "val": 0, 2024-03-21T10:36:04.022 INFO:teuthology.orchestra.run.smithi100.stdout: "max": 3758096384, 2024-03-21T10:36:04.022 INFO:teuthology.orchestra.run.smithi100.stdout: "get_started": 0, 2024-03-21T10:36:04.022 INFO:teuthology.orchestra.run.smithi100.stdout: "get": 8, 2024-03-21T10:36:04.022 INFO:teuthology.orchestra.run.smithi100.stdout: "get_sum": 2536, 2024-03-21T10:36:04.022 INFO:teuthology.orchestra.run.smithi100.stdout: "get_or_fail_fail": 0, 2024-03-21T10:36:04.022 INFO:teuthology.orchestra.run.smithi100.stdout: "get_or_fail_success": 8, 2024-03-21T10:36:04.022 INFO:teuthology.orchestra.run.smithi100.stdout: "take": 0, 2024-03-21T10:36:04.022 INFO:teuthology.orchestra.run.smithi100.stdout: "take_sum": 0, 2024-03-21T10:36:04.022 INFO:teuthology.orchestra.run.smithi100.stdout: "put": 7, 2024-03-21T10:36:04.022 INFO:teuthology.orchestra.run.smithi100.stdout: "put_sum": 2536, 2024-03-21T10:36:04.022 INFO:teuthology.orchestra.run.smithi100.stdout: "wait": { 2024-03-21T10:36:04.022 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 0, 2024-03-21T10:36:04.022 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.000000000, 2024-03-21T10:36:04.023 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000000000 2024-03-21T10:36:04.023 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-03-21T10:36:04.023 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-03-21T10:36:04.023 INFO:teuthology.orchestra.run.smithi100.stdout:} 2024-03-21T10:36:04.921 INFO:tasks.check_counter:Daemon mds.k mds.exported=0 2024-03-21T10:36:04.921 INFO:tasks.check_counter:Daemon mds.k mds.imported=0 2024-03-21T10:36:04.921 INFO:tasks.check_counter:Daemon mds.k mds.root_rsnaps=6 2024-03-21T10:36:04.921 INFO:tasks.check_counter:Daemon mds.k mds_server.req_mksnap_latency.avgcount=0 2024-03-21T10:36:04.921 INFO:tasks.check_counter:Daemon mds.k mds_server.req_rmsnap_latency.avgcount=0 2024-03-21T10:36:04.921 DEBUG:tasks.check_counter:Getting stats from c 2024-03-21T10:36:04.921 DEBUG:teuthology.orchestra.run.smithi157:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2edfee68ad7243bed422a049b6f5f58619300f11 shell --fsid 3056d51a-e76b-11ee-95cd-87774f69a715 -- ceph daemon mds.c perf dump 2024-03-21T10:36:05.367 INFO:teuthology.orchestra.run.smithi157.stderr:Inferring config /var/lib/ceph/3056d51a-e76b-11ee-95cd-87774f69a715/mon.c/config 2024-03-21T10:36:05.830 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:36:05 smithi157 ceph-mon[35581]: pgmap v334: 129 pgs: 129 active+clean; 2.5 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 33 KiB/s wr, 24 op/s 2024-03-21T10:36:05.960 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:36:05 smithi047 ceph-mon[31547]: pgmap v334: 129 pgs: 129 active+clean; 2.5 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 33 KiB/s wr, 24 op/s 2024-03-21T10:36:05.976 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:36:05 smithi100 ceph-mon[35193]: pgmap v334: 129 pgs: 129 active+clean; 2.5 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 33 KiB/s wr, 24 op/s 2024-03-21T10:36:07.653 INFO:teuthology.orchestra.run.smithi157.stdout:{ 2024-03-21T10:36:07.653 INFO:teuthology.orchestra.run.smithi157.stdout: "AsyncMessenger::Worker-0": { 2024-03-21T10:36:07.653 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_recv_messages": 1086, 2024-03-21T10:36:07.653 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_send_messages": 1068, 2024-03-21T10:36:07.654 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_recv_bytes": 2061088, 2024-03-21T10:36:07.654 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_send_bytes": 403294, 2024-03-21T10:36:07.654 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_created_connections": 5, 2024-03-21T10:36:07.654 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_active_connections": 1, 2024-03-21T10:36:07.654 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_running_total_time": 0.283082260, 2024-03-21T10:36:07.654 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_running_send_time": 0.092221090, 2024-03-21T10:36:07.654 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_running_recv_time": 0.138119010, 2024-03-21T10:36:07.654 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_running_fast_dispatch_time": 0.060169949, 2024-03-21T10:36:07.654 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_send_messages_queue_lat": { 2024-03-21T10:36:07.654 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 1067, 2024-03-21T10:36:07.654 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.110725508, 2024-03-21T10:36:07.654 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000103772 2024-03-21T10:36:07.654 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-03-21T10:36:07.654 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_handle_ack_lat": { 2024-03-21T10:36:07.655 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-03-21T10:36:07.655 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-03-21T10:36:07.655 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-03-21T10:36:07.655 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-03-21T10:36:07.655 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_recv_encrypted_bytes": 1786576, 2024-03-21T10:36:07.655 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_send_encrypted_bytes": 143488 2024-03-21T10:36:07.655 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-03-21T10:36:07.655 INFO:teuthology.orchestra.run.smithi157.stdout: "AsyncMessenger::Worker-1": { 2024-03-21T10:36:07.655 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_recv_messages": 8, 2024-03-21T10:36:07.655 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_send_messages": 183, 2024-03-21T10:36:07.655 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_recv_bytes": 3820, 2024-03-21T10:36:07.655 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_send_bytes": 403450, 2024-03-21T10:36:07.655 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_created_connections": 6, 2024-03-21T10:36:07.655 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_active_connections": 4, 2024-03-21T10:36:07.656 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_running_total_time": 0.029865109, 2024-03-21T10:36:07.656 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_running_send_time": 0.016584246, 2024-03-21T10:36:07.656 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_running_recv_time": 0.007822864, 2024-03-21T10:36:07.656 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_running_fast_dispatch_time": 0.000068689, 2024-03-21T10:36:07.656 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_send_messages_queue_lat": { 2024-03-21T10:36:07.656 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 181, 2024-03-21T10:36:07.656 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.028759079, 2024-03-21T10:36:07.656 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000158889 2024-03-21T10:36:07.656 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-03-21T10:36:07.656 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_handle_ack_lat": { 2024-03-21T10:36:07.656 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-03-21T10:36:07.656 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-03-21T10:36:07.656 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-03-21T10:36:07.656 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-03-21T10:36:07.657 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_recv_encrypted_bytes": 3264, 2024-03-21T10:36:07.657 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_send_encrypted_bytes": 402752 2024-03-21T10:36:07.657 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-03-21T10:36:07.657 INFO:teuthology.orchestra.run.smithi157.stdout: "AsyncMessenger::Worker-2": { 2024-03-21T10:36:07.657 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_recv_messages": 1493, 2024-03-21T10:36:07.657 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_send_messages": 1495, 2024-03-21T10:36:07.657 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_recv_bytes": 382608, 2024-03-21T10:36:07.657 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_send_bytes": 474781, 2024-03-21T10:36:07.657 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_created_connections": 4, 2024-03-21T10:36:07.657 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_active_connections": 3, 2024-03-21T10:36:07.657 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_running_total_time": 0.266167533, 2024-03-21T10:36:07.657 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_running_send_time": 0.094743103, 2024-03-21T10:36:07.657 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_running_recv_time": 0.070024942, 2024-03-21T10:36:07.657 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_running_fast_dispatch_time": 0.082793775, 2024-03-21T10:36:07.657 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_send_messages_queue_lat": { 2024-03-21T10:36:07.658 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 1493, 2024-03-21T10:36:07.658 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.135887247, 2024-03-21T10:36:07.658 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000091016 2024-03-21T10:36:07.658 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-03-21T10:36:07.658 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_handle_ack_lat": { 2024-03-21T10:36:07.658 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-03-21T10:36:07.658 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-03-21T10:36:07.658 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-03-21T10:36:07.658 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-03-21T10:36:07.658 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_recv_encrypted_bytes": 0, 2024-03-21T10:36:07.658 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_send_encrypted_bytes": 0 2024-03-21T10:36:07.658 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-03-21T10:36:07.658 INFO:teuthology.orchestra.run.smithi157.stdout: "cct": { 2024-03-21T10:36:07.659 INFO:teuthology.orchestra.run.smithi157.stdout: "total_workers": 1, 2024-03-21T10:36:07.659 INFO:teuthology.orchestra.run.smithi157.stdout: "unhealthy_workers": 0 2024-03-21T10:36:07.659 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-03-21T10:36:07.659 INFO:teuthology.orchestra.run.smithi157.stdout: "finisher-MDSRank": { 2024-03-21T10:36:07.659 INFO:teuthology.orchestra.run.smithi157.stdout: "queue_len": 0, 2024-03-21T10:36:07.659 INFO:teuthology.orchestra.run.smithi157.stdout: "complete_latency": { 2024-03-21T10:36:07.659 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 3942, 2024-03-21T10:36:07.659 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.197315884, 2024-03-21T10:36:07.659 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000050054 2024-03-21T10:36:07.659 INFO:teuthology.orchestra.run.smithi157.stdout: } 2024-03-21T10:36:07.659 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-03-21T10:36:07.659 INFO:teuthology.orchestra.run.smithi157.stdout: "finisher-PurgeQueue": { 2024-03-21T10:36:07.659 INFO:teuthology.orchestra.run.smithi157.stdout: "queue_len": 0, 2024-03-21T10:36:07.659 INFO:teuthology.orchestra.run.smithi157.stdout: "complete_latency": { 2024-03-21T10:36:07.659 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-03-21T10:36:07.660 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-03-21T10:36:07.660 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-03-21T10:36:07.660 INFO:teuthology.orchestra.run.smithi157.stdout: } 2024-03-21T10:36:07.660 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-03-21T10:36:07.660 INFO:teuthology.orchestra.run.smithi157.stdout: "mds": { 2024-03-21T10:36:07.660 INFO:teuthology.orchestra.run.smithi157.stdout: "request": 0, 2024-03-21T10:36:07.660 INFO:teuthology.orchestra.run.smithi157.stdout: "reply": 0, 2024-03-21T10:36:07.660 INFO:teuthology.orchestra.run.smithi157.stdout: "reply_latency": { 2024-03-21T10:36:07.660 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-03-21T10:36:07.660 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-03-21T10:36:07.660 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-03-21T10:36:07.660 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-03-21T10:36:07.660 INFO:teuthology.orchestra.run.smithi157.stdout: "slow_reply": 0, 2024-03-21T10:36:07.660 INFO:teuthology.orchestra.run.smithi157.stdout: "forward": 0, 2024-03-21T10:36:07.660 INFO:teuthology.orchestra.run.smithi157.stdout: "dir_fetch_complete": 0, 2024-03-21T10:36:07.661 INFO:teuthology.orchestra.run.smithi157.stdout: "dir_fetch_keys": 0, 2024-03-21T10:36:07.661 INFO:teuthology.orchestra.run.smithi157.stdout: "dir_commit": 0, 2024-03-21T10:36:07.661 INFO:teuthology.orchestra.run.smithi157.stdout: "dir_split": 0, 2024-03-21T10:36:07.661 INFO:teuthology.orchestra.run.smithi157.stdout: "dir_merge": 0, 2024-03-21T10:36:07.661 INFO:teuthology.orchestra.run.smithi157.stdout: "inodes": 0, 2024-03-21T10:36:07.661 INFO:teuthology.orchestra.run.smithi157.stdout: "inodes_top": 0, 2024-03-21T10:36:07.661 INFO:teuthology.orchestra.run.smithi157.stdout: "inodes_bottom": 0, 2024-03-21T10:36:07.661 INFO:teuthology.orchestra.run.smithi157.stdout: "inodes_pin_tail": 0, 2024-03-21T10:36:07.661 INFO:teuthology.orchestra.run.smithi157.stdout: "inodes_pinned": 0, 2024-03-21T10:36:07.661 INFO:teuthology.orchestra.run.smithi157.stdout: "inodes_expired": 0, 2024-03-21T10:36:07.661 INFO:teuthology.orchestra.run.smithi157.stdout: "inodes_with_caps": 0, 2024-03-21T10:36:07.661 INFO:teuthology.orchestra.run.smithi157.stdout: "caps": 0, 2024-03-21T10:36:07.661 INFO:teuthology.orchestra.run.smithi157.stdout: "subtrees": 1, 2024-03-21T10:36:07.662 INFO:teuthology.orchestra.run.smithi157.stdout: "traverse": 0, 2024-03-21T10:36:07.662 INFO:teuthology.orchestra.run.smithi157.stdout: "traverse_hit": 0, 2024-03-21T10:36:07.662 INFO:teuthology.orchestra.run.smithi157.stdout: "traverse_forward": 0, 2024-03-21T10:36:07.662 INFO:teuthology.orchestra.run.smithi157.stdout: "traverse_discover": 0, 2024-03-21T10:36:07.662 INFO:teuthology.orchestra.run.smithi157.stdout: "traverse_dir_fetch": 0, 2024-03-21T10:36:07.662 INFO:teuthology.orchestra.run.smithi157.stdout: "traverse_remote_ino": 0, 2024-03-21T10:36:07.662 INFO:teuthology.orchestra.run.smithi157.stdout: "traverse_lock": 0, 2024-03-21T10:36:07.662 INFO:teuthology.orchestra.run.smithi157.stdout: "load_cent": 0, 2024-03-21T10:36:07.662 INFO:teuthology.orchestra.run.smithi157.stdout: "q": 0, 2024-03-21T10:36:07.662 INFO:teuthology.orchestra.run.smithi157.stdout: "exported": 0, 2024-03-21T10:36:07.662 INFO:teuthology.orchestra.run.smithi157.stdout: "exported_inodes": 0, 2024-03-21T10:36:07.662 INFO:teuthology.orchestra.run.smithi157.stdout: "imported": 0, 2024-03-21T10:36:07.662 INFO:teuthology.orchestra.run.smithi157.stdout: "imported_inodes": 0, 2024-03-21T10:36:07.662 INFO:teuthology.orchestra.run.smithi157.stdout: "openino_dir_fetch": 0, 2024-03-21T10:36:07.663 INFO:teuthology.orchestra.run.smithi157.stdout: "openino_backtrace_fetch": 0, 2024-03-21T10:36:07.663 INFO:teuthology.orchestra.run.smithi157.stdout: "openino_peer_discover": 0, 2024-03-21T10:36:07.663 INFO:teuthology.orchestra.run.smithi157.stdout: "root_rfiles": 0, 2024-03-21T10:36:07.663 INFO:teuthology.orchestra.run.smithi157.stdout: "root_rbytes": 0, 2024-03-21T10:36:07.663 INFO:teuthology.orchestra.run.smithi157.stdout: "root_rsnaps": 0, 2024-03-21T10:36:07.663 INFO:teuthology.orchestra.run.smithi157.stdout: "scrub_backtrace_fetch": 0, 2024-03-21T10:36:07.663 INFO:teuthology.orchestra.run.smithi157.stdout: "scrub_set_tag": 0, 2024-03-21T10:36:07.663 INFO:teuthology.orchestra.run.smithi157.stdout: "scrub_backtrace_repaired": 0, 2024-03-21T10:36:07.663 INFO:teuthology.orchestra.run.smithi157.stdout: "scrub_inotable_repaired": 0, 2024-03-21T10:36:07.663 INFO:teuthology.orchestra.run.smithi157.stdout: "scrub_dir_inodes": 0, 2024-03-21T10:36:07.663 INFO:teuthology.orchestra.run.smithi157.stdout: "scrub_dir_base_inodes": 0, 2024-03-21T10:36:07.663 INFO:teuthology.orchestra.run.smithi157.stdout: "scrub_dirfrag_rstats": 0, 2024-03-21T10:36:07.663 INFO:teuthology.orchestra.run.smithi157.stdout: "scrub_file_inodes": 0, 2024-03-21T10:36:07.663 INFO:teuthology.orchestra.run.smithi157.stdout: "handle_inode_file_caps": 0, 2024-03-21T10:36:07.663 INFO:teuthology.orchestra.run.smithi157.stdout: "ceph_cap_op_revoke": 0, 2024-03-21T10:36:07.664 INFO:teuthology.orchestra.run.smithi157.stdout: "ceph_cap_op_grant": 0, 2024-03-21T10:36:07.664 INFO:teuthology.orchestra.run.smithi157.stdout: "ceph_cap_op_trunc": 0, 2024-03-21T10:36:07.664 INFO:teuthology.orchestra.run.smithi157.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-03-21T10:36:07.664 INFO:teuthology.orchestra.run.smithi157.stdout: "ceph_cap_op_flush_ack": 0, 2024-03-21T10:36:07.664 INFO:teuthology.orchestra.run.smithi157.stdout: "handle_client_caps": 0, 2024-03-21T10:36:07.664 INFO:teuthology.orchestra.run.smithi157.stdout: "handle_client_caps_dirty": 0, 2024-03-21T10:36:07.664 INFO:teuthology.orchestra.run.smithi157.stdout: "handle_client_cap_release": 0, 2024-03-21T10:36:07.664 INFO:teuthology.orchestra.run.smithi157.stdout: "process_request_cap_release": 0 2024-03-21T10:36:07.664 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-03-21T10:36:07.664 INFO:teuthology.orchestra.run.smithi157.stdout: "mds_cache": { 2024-03-21T10:36:07.664 INFO:teuthology.orchestra.run.smithi157.stdout: "dir_update": 0, 2024-03-21T10:36:07.664 INFO:teuthology.orchestra.run.smithi157.stdout: "dir_update_receipt": 0, 2024-03-21T10:36:07.664 INFO:teuthology.orchestra.run.smithi157.stdout: "dir_try_discover": 0, 2024-03-21T10:36:07.664 INFO:teuthology.orchestra.run.smithi157.stdout: "dir_send_discover": 0, 2024-03-21T10:36:07.664 INFO:teuthology.orchestra.run.smithi157.stdout: "dir_handle_discover": 0, 2024-03-21T10:36:07.664 INFO:teuthology.orchestra.run.smithi157.stdout: "num_strays": 0, 2024-03-21T10:36:07.665 INFO:teuthology.orchestra.run.smithi157.stdout: "num_strays_delayed": 0, 2024-03-21T10:36:07.665 INFO:teuthology.orchestra.run.smithi157.stdout: "num_strays_enqueuing": 0, 2024-03-21T10:36:07.665 INFO:teuthology.orchestra.run.smithi157.stdout: "strays_created": 0, 2024-03-21T10:36:07.665 INFO:teuthology.orchestra.run.smithi157.stdout: "strays_enqueued": 0, 2024-03-21T10:36:07.665 INFO:teuthology.orchestra.run.smithi157.stdout: "strays_reintegrated": 0, 2024-03-21T10:36:07.665 INFO:teuthology.orchestra.run.smithi157.stdout: "strays_migrated": 0, 2024-03-21T10:36:07.665 INFO:teuthology.orchestra.run.smithi157.stdout: "num_recovering_processing": 0, 2024-03-21T10:36:07.665 INFO:teuthology.orchestra.run.smithi157.stdout: "num_recovering_enqueued": 0, 2024-03-21T10:36:07.665 INFO:teuthology.orchestra.run.smithi157.stdout: "num_recovering_prioritized": 0, 2024-03-21T10:36:07.665 INFO:teuthology.orchestra.run.smithi157.stdout: "recovery_started": 0, 2024-03-21T10:36:07.665 INFO:teuthology.orchestra.run.smithi157.stdout: "recovery_completed": 0, 2024-03-21T10:36:07.665 INFO:teuthology.orchestra.run.smithi157.stdout: "ireq_enqueue_scrub": 0, 2024-03-21T10:36:07.665 INFO:teuthology.orchestra.run.smithi157.stdout: "ireq_exportdir": 0, 2024-03-21T10:36:07.665 INFO:teuthology.orchestra.run.smithi157.stdout: "ireq_flush": 0, 2024-03-21T10:36:07.665 INFO:teuthology.orchestra.run.smithi157.stdout: "ireq_fragmentdir": 0, 2024-03-21T10:36:07.666 INFO:teuthology.orchestra.run.smithi157.stdout: "ireq_fragstats": 0, 2024-03-21T10:36:07.666 INFO:teuthology.orchestra.run.smithi157.stdout: "ireq_inodestats": 0 2024-03-21T10:36:07.666 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-03-21T10:36:07.666 INFO:teuthology.orchestra.run.smithi157.stdout: "mds_log": { 2024-03-21T10:36:07.666 INFO:teuthology.orchestra.run.smithi157.stdout: "evlrg": 0, 2024-03-21T10:36:07.666 INFO:teuthology.orchestra.run.smithi157.stdout: "evadd": 0, 2024-03-21T10:36:07.666 INFO:teuthology.orchestra.run.smithi157.stdout: "evex": 0, 2024-03-21T10:36:07.666 INFO:teuthology.orchestra.run.smithi157.stdout: "evtrm": 0, 2024-03-21T10:36:07.666 INFO:teuthology.orchestra.run.smithi157.stdout: "ev": 5, 2024-03-21T10:36:07.666 INFO:teuthology.orchestra.run.smithi157.stdout: "evexg": 0, 2024-03-21T10:36:07.666 INFO:teuthology.orchestra.run.smithi157.stdout: "evexd": 0, 2024-03-21T10:36:07.666 INFO:teuthology.orchestra.run.smithi157.stdout: "segadd": 0, 2024-03-21T10:36:07.666 INFO:teuthology.orchestra.run.smithi157.stdout: "segex": 0, 2024-03-21T10:36:07.666 INFO:teuthology.orchestra.run.smithi157.stdout: "segtrm": 0, 2024-03-21T10:36:07.666 INFO:teuthology.orchestra.run.smithi157.stdout: "seg": 2, 2024-03-21T10:36:07.667 INFO:teuthology.orchestra.run.smithi157.stdout: "segmjr": 2, 2024-03-21T10:36:07.667 INFO:teuthology.orchestra.run.smithi157.stdout: "segexg": 0, 2024-03-21T10:36:07.667 INFO:teuthology.orchestra.run.smithi157.stdout: "segexd": 0, 2024-03-21T10:36:07.667 INFO:teuthology.orchestra.run.smithi157.stdout: "expos": 4194304, 2024-03-21T10:36:07.667 INFO:teuthology.orchestra.run.smithi157.stdout: "wrpos": 4195917, 2024-03-21T10:36:07.667 INFO:teuthology.orchestra.run.smithi157.stdout: "rdpos": 4195776, 2024-03-21T10:36:07.667 INFO:teuthology.orchestra.run.smithi157.stdout: "jlat": { 2024-03-21T10:36:07.667 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-03-21T10:36:07.667 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-03-21T10:36:07.667 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-03-21T10:36:07.667 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-03-21T10:36:07.667 INFO:teuthology.orchestra.run.smithi157.stdout: "replayed": 5 2024-03-21T10:36:07.667 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-03-21T10:36:07.667 INFO:teuthology.orchestra.run.smithi157.stdout: "mds_mem": { 2024-03-21T10:36:07.668 INFO:teuthology.orchestra.run.smithi157.stdout: "ino": 3, 2024-03-21T10:36:07.668 INFO:teuthology.orchestra.run.smithi157.stdout: "ino+": 3, 2024-03-21T10:36:07.668 INFO:teuthology.orchestra.run.smithi157.stdout: "ino-": 0, 2024-03-21T10:36:07.668 INFO:teuthology.orchestra.run.smithi157.stdout: "dir": 1, 2024-03-21T10:36:07.668 INFO:teuthology.orchestra.run.smithi157.stdout: "dir+": 1, 2024-03-21T10:36:07.668 INFO:teuthology.orchestra.run.smithi157.stdout: "dir-": 0, 2024-03-21T10:36:07.668 INFO:teuthology.orchestra.run.smithi157.stdout: "dn": 0, 2024-03-21T10:36:07.668 INFO:teuthology.orchestra.run.smithi157.stdout: "dn+": 0, 2024-03-21T10:36:07.668 INFO:teuthology.orchestra.run.smithi157.stdout: "dn-": 0, 2024-03-21T10:36:07.668 INFO:teuthology.orchestra.run.smithi157.stdout: "cap": 0, 2024-03-21T10:36:07.668 INFO:teuthology.orchestra.run.smithi157.stdout: "cap+": 0, 2024-03-21T10:36:07.668 INFO:teuthology.orchestra.run.smithi157.stdout: "cap-": 0, 2024-03-21T10:36:07.668 INFO:teuthology.orchestra.run.smithi157.stdout: "rss": 48692, 2024-03-21T10:36:07.668 INFO:teuthology.orchestra.run.smithi157.stdout: "heap": 215420 2024-03-21T10:36:07.668 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-03-21T10:36:07.668 INFO:teuthology.orchestra.run.smithi157.stdout: "mds_server": { 2024-03-21T10:36:07.669 INFO:teuthology.orchestra.run.smithi157.stdout: "dispatch_client_request": 0, 2024-03-21T10:36:07.669 INFO:teuthology.orchestra.run.smithi157.stdout: "dispatch_server_request": 0, 2024-03-21T10:36:07.669 INFO:teuthology.orchestra.run.smithi157.stdout: "handle_client_request": 0, 2024-03-21T10:36:07.669 INFO:teuthology.orchestra.run.smithi157.stdout: "handle_client_session": 0, 2024-03-21T10:36:07.669 INFO:teuthology.orchestra.run.smithi157.stdout: "handle_peer_request": 0, 2024-03-21T10:36:07.669 INFO:teuthology.orchestra.run.smithi157.stdout: "req_create_latency": { 2024-03-21T10:36:07.669 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-03-21T10:36:07.669 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-03-21T10:36:07.669 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-03-21T10:36:07.669 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-03-21T10:36:07.669 INFO:teuthology.orchestra.run.smithi157.stdout: "req_getattr_latency": { 2024-03-21T10:36:07.669 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-03-21T10:36:07.669 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-03-21T10:36:07.669 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-03-21T10:36:07.670 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-03-21T10:36:07.670 INFO:teuthology.orchestra.run.smithi157.stdout: "req_getfilelock_latency": { 2024-03-21T10:36:07.670 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-03-21T10:36:07.670 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-03-21T10:36:07.670 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-03-21T10:36:07.670 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-03-21T10:36:07.670 INFO:teuthology.orchestra.run.smithi157.stdout: "req_link_latency": { 2024-03-21T10:36:07.670 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-03-21T10:36:07.670 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-03-21T10:36:07.670 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-03-21T10:36:07.670 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-03-21T10:36:07.670 INFO:teuthology.orchestra.run.smithi157.stdout: "req_lookup_latency": { 2024-03-21T10:36:07.670 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-03-21T10:36:07.670 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-03-21T10:36:07.671 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-03-21T10:36:07.671 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-03-21T10:36:07.671 INFO:teuthology.orchestra.run.smithi157.stdout: "req_lookuphash_latency": { 2024-03-21T10:36:07.671 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-03-21T10:36:07.671 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-03-21T10:36:07.671 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-03-21T10:36:07.671 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-03-21T10:36:07.671 INFO:teuthology.orchestra.run.smithi157.stdout: "req_lookupino_latency": { 2024-03-21T10:36:07.671 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-03-21T10:36:07.671 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-03-21T10:36:07.671 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-03-21T10:36:07.671 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-03-21T10:36:07.671 INFO:teuthology.orchestra.run.smithi157.stdout: "req_lookupname_latency": { 2024-03-21T10:36:07.671 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-03-21T10:36:07.671 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-03-21T10:36:07.672 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-03-21T10:36:07.672 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-03-21T10:36:07.672 INFO:teuthology.orchestra.run.smithi157.stdout: "req_lookupparent_latency": { 2024-03-21T10:36:07.672 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-03-21T10:36:07.672 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-03-21T10:36:07.672 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-03-21T10:36:07.672 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-03-21T10:36:07.672 INFO:teuthology.orchestra.run.smithi157.stdout: "req_lookupsnap_latency": { 2024-03-21T10:36:07.673 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-03-21T10:36:07.673 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-03-21T10:36:07.673 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-03-21T10:36:07.673 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-03-21T10:36:07.673 INFO:teuthology.orchestra.run.smithi157.stdout: "req_lssnap_latency": { 2024-03-21T10:36:07.673 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-03-21T10:36:07.673 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-03-21T10:36:07.673 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-03-21T10:36:07.673 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-03-21T10:36:07.673 INFO:teuthology.orchestra.run.smithi157.stdout: "req_mkdir_latency": { 2024-03-21T10:36:07.674 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-03-21T10:36:07.674 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-03-21T10:36:07.674 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-03-21T10:36:07.674 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-03-21T10:36:07.674 INFO:teuthology.orchestra.run.smithi157.stdout: "req_mknod_latency": { 2024-03-21T10:36:07.675 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-03-21T10:36:07.675 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-03-21T10:36:07.675 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-03-21T10:36:07.675 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-03-21T10:36:07.675 INFO:teuthology.orchestra.run.smithi157.stdout: "req_mksnap_latency": { 2024-03-21T10:36:07.675 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-03-21T10:36:07.675 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-03-21T10:36:07.675 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-03-21T10:36:07.675 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-03-21T10:36:07.675 INFO:teuthology.orchestra.run.smithi157.stdout: "req_open_latency": { 2024-03-21T10:36:07.675 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-03-21T10:36:07.675 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-03-21T10:36:07.675 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-03-21T10:36:07.675 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-03-21T10:36:07.675 INFO:teuthology.orchestra.run.smithi157.stdout: "req_readdir_latency": { 2024-03-21T10:36:07.676 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-03-21T10:36:07.676 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-03-21T10:36:07.676 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-03-21T10:36:07.676 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-03-21T10:36:07.676 INFO:teuthology.orchestra.run.smithi157.stdout: "req_rename_latency": { 2024-03-21T10:36:07.676 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-03-21T10:36:07.676 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-03-21T10:36:07.676 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-03-21T10:36:07.676 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-03-21T10:36:07.676 INFO:teuthology.orchestra.run.smithi157.stdout: "req_renamesnap_latency": { 2024-03-21T10:36:07.676 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-03-21T10:36:07.676 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-03-21T10:36:07.676 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-03-21T10:36:07.676 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-03-21T10:36:07.676 INFO:teuthology.orchestra.run.smithi157.stdout: "req_snapdiff_latency": { 2024-03-21T10:36:07.676 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-03-21T10:36:07.676 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-03-21T10:36:07.677 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-03-21T10:36:07.677 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-03-21T10:36:07.677 INFO:teuthology.orchestra.run.smithi157.stdout: "req_rmdir_latency": { 2024-03-21T10:36:07.677 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-03-21T10:36:07.677 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-03-21T10:36:07.677 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-03-21T10:36:07.677 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-03-21T10:36:07.677 INFO:teuthology.orchestra.run.smithi157.stdout: "req_rmsnap_latency": { 2024-03-21T10:36:07.677 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-03-21T10:36:07.677 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-03-21T10:36:07.677 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-03-21T10:36:07.677 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-03-21T10:36:07.677 INFO:teuthology.orchestra.run.smithi157.stdout: "req_rmxattr_latency": { 2024-03-21T10:36:07.677 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-03-21T10:36:07.677 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-03-21T10:36:07.677 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-03-21T10:36:07.678 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-03-21T10:36:07.678 INFO:teuthology.orchestra.run.smithi157.stdout: "req_setattr_latency": { 2024-03-21T10:36:07.678 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-03-21T10:36:07.678 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-03-21T10:36:07.678 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-03-21T10:36:07.678 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-03-21T10:36:07.678 INFO:teuthology.orchestra.run.smithi157.stdout: "req_setdirlayout_latency": { 2024-03-21T10:36:07.678 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-03-21T10:36:07.678 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-03-21T10:36:07.678 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-03-21T10:36:07.678 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-03-21T10:36:07.678 INFO:teuthology.orchestra.run.smithi157.stdout: "req_setfilelock_latency": { 2024-03-21T10:36:07.678 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-03-21T10:36:07.678 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-03-21T10:36:07.679 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-03-21T10:36:07.679 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-03-21T10:36:07.679 INFO:teuthology.orchestra.run.smithi157.stdout: "req_setlayout_latency": { 2024-03-21T10:36:07.679 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-03-21T10:36:07.679 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-03-21T10:36:07.679 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-03-21T10:36:07.679 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-03-21T10:36:07.679 INFO:teuthology.orchestra.run.smithi157.stdout: "req_setxattr_latency": { 2024-03-21T10:36:07.679 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-03-21T10:36:07.679 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-03-21T10:36:07.679 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-03-21T10:36:07.679 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-03-21T10:36:07.679 INFO:teuthology.orchestra.run.smithi157.stdout: "req_symlink_latency": { 2024-03-21T10:36:07.679 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-03-21T10:36:07.679 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-03-21T10:36:07.679 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-03-21T10:36:07.679 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-03-21T10:36:07.680 INFO:teuthology.orchestra.run.smithi157.stdout: "req_unlink_latency": { 2024-03-21T10:36:07.680 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-03-21T10:36:07.680 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-03-21T10:36:07.680 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-03-21T10:36:07.680 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-03-21T10:36:07.680 INFO:teuthology.orchestra.run.smithi157.stdout: "cap_revoke_eviction": 0, 2024-03-21T10:36:07.680 INFO:teuthology.orchestra.run.smithi157.stdout: "cap_acquisition_throttle": 0, 2024-03-21T10:36:07.680 INFO:teuthology.orchestra.run.smithi157.stdout: "req_getvxattr_latency": { 2024-03-21T10:36:07.680 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-03-21T10:36:07.680 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-03-21T10:36:07.680 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-03-21T10:36:07.680 INFO:teuthology.orchestra.run.smithi157.stdout: } 2024-03-21T10:36:07.680 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-03-21T10:36:07.680 INFO:teuthology.orchestra.run.smithi157.stdout: "mds_sessions": { 2024-03-21T10:36:07.680 INFO:teuthology.orchestra.run.smithi157.stdout: "session_count": 1, 2024-03-21T10:36:07.681 INFO:teuthology.orchestra.run.smithi157.stdout: "session_add": 2, 2024-03-21T10:36:07.681 INFO:teuthology.orchestra.run.smithi157.stdout: "session_remove": 1, 2024-03-21T10:36:07.681 INFO:teuthology.orchestra.run.smithi157.stdout: "sessions_open": 2, 2024-03-21T10:36:07.681 INFO:teuthology.orchestra.run.smithi157.stdout: "sessions_stale": 0, 2024-03-21T10:36:07.681 INFO:teuthology.orchestra.run.smithi157.stdout: "total_load": 0, 2024-03-21T10:36:07.681 INFO:teuthology.orchestra.run.smithi157.stdout: "average_load": 0, 2024-03-21T10:36:07.681 INFO:teuthology.orchestra.run.smithi157.stdout: "avg_session_uptime": 3590, 2024-03-21T10:36:07.681 INFO:teuthology.orchestra.run.smithi157.stdout: "mdthresh_evicted": 0 2024-03-21T10:36:07.681 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-03-21T10:36:07.681 INFO:teuthology.orchestra.run.smithi157.stdout: "mempool": { 2024-03-21T10:36:07.681 INFO:teuthology.orchestra.run.smithi157.stdout: "bloom_filter_bytes": 0, 2024-03-21T10:36:07.681 INFO:teuthology.orchestra.run.smithi157.stdout: "bloom_filter_items": 0, 2024-03-21T10:36:07.681 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_alloc_bytes": 0, 2024-03-21T10:36:07.681 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_alloc_items": 0, 2024-03-21T10:36:07.681 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_cache_data_bytes": 0, 2024-03-21T10:36:07.681 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_cache_data_items": 0, 2024-03-21T10:36:07.682 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_cache_onode_bytes": 0, 2024-03-21T10:36:07.682 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_cache_onode_items": 0, 2024-03-21T10:36:07.682 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_cache_meta_bytes": 0, 2024-03-21T10:36:07.682 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_cache_meta_items": 0, 2024-03-21T10:36:07.682 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_cache_other_bytes": 0, 2024-03-21T10:36:07.682 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_cache_other_items": 0, 2024-03-21T10:36:07.682 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_cache_buffer_bytes": 0, 2024-03-21T10:36:07.682 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_cache_buffer_items": 0, 2024-03-21T10:36:07.682 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_extent_bytes": 0, 2024-03-21T10:36:07.682 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_extent_items": 0, 2024-03-21T10:36:07.682 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_blob_bytes": 0, 2024-03-21T10:36:07.682 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_blob_items": 0, 2024-03-21T10:36:07.683 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_shared_blob_bytes": 0, 2024-03-21T10:36:07.683 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_shared_blob_items": 0, 2024-03-21T10:36:07.683 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_inline_bl_bytes": 0, 2024-03-21T10:36:07.683 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_inline_bl_items": 0, 2024-03-21T10:36:07.683 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_fsck_bytes": 0, 2024-03-21T10:36:07.683 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_fsck_items": 0, 2024-03-21T10:36:07.683 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_txc_bytes": 0, 2024-03-21T10:36:07.683 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_txc_items": 0, 2024-03-21T10:36:07.683 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_writing_deferred_bytes": 0, 2024-03-21T10:36:07.683 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_writing_deferred_items": 0, 2024-03-21T10:36:07.683 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_writing_bytes": 0, 2024-03-21T10:36:07.683 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_writing_items": 0, 2024-03-21T10:36:07.683 INFO:teuthology.orchestra.run.smithi157.stdout: "bluefs_bytes": 0, 2024-03-21T10:36:07.683 INFO:teuthology.orchestra.run.smithi157.stdout: "bluefs_items": 0, 2024-03-21T10:36:07.683 INFO:teuthology.orchestra.run.smithi157.stdout: "bluefs_file_reader_bytes": 0, 2024-03-21T10:36:07.684 INFO:teuthology.orchestra.run.smithi157.stdout: "bluefs_file_reader_items": 0, 2024-03-21T10:36:07.684 INFO:teuthology.orchestra.run.smithi157.stdout: "bluefs_file_writer_bytes": 0, 2024-03-21T10:36:07.684 INFO:teuthology.orchestra.run.smithi157.stdout: "bluefs_file_writer_items": 0, 2024-03-21T10:36:07.684 INFO:teuthology.orchestra.run.smithi157.stdout: "buffer_anon_bytes": 182081, 2024-03-21T10:36:07.684 INFO:teuthology.orchestra.run.smithi157.stdout: "buffer_anon_items": 48, 2024-03-21T10:36:07.684 INFO:teuthology.orchestra.run.smithi157.stdout: "buffer_meta_bytes": 0, 2024-03-21T10:36:07.684 INFO:teuthology.orchestra.run.smithi157.stdout: "buffer_meta_items": 0, 2024-03-21T10:36:07.684 INFO:teuthology.orchestra.run.smithi157.stdout: "osd_bytes": 0, 2024-03-21T10:36:07.684 INFO:teuthology.orchestra.run.smithi157.stdout: "osd_items": 0, 2024-03-21T10:36:07.684 INFO:teuthology.orchestra.run.smithi157.stdout: "osd_mapbl_bytes": 0, 2024-03-21T10:36:07.684 INFO:teuthology.orchestra.run.smithi157.stdout: "osd_mapbl_items": 0, 2024-03-21T10:36:07.684 INFO:teuthology.orchestra.run.smithi157.stdout: "osd_pglog_bytes": 0, 2024-03-21T10:36:07.684 INFO:teuthology.orchestra.run.smithi157.stdout: "osd_pglog_items": 0, 2024-03-21T10:36:07.684 INFO:teuthology.orchestra.run.smithi157.stdout: "osdmap_bytes": 9712, 2024-03-21T10:36:07.684 INFO:teuthology.orchestra.run.smithi157.stdout: "osdmap_items": 187, 2024-03-21T10:36:07.684 INFO:teuthology.orchestra.run.smithi157.stdout: "osdmap_mapping_bytes": 0, 2024-03-21T10:36:07.684 INFO:teuthology.orchestra.run.smithi157.stdout: "osdmap_mapping_items": 0, 2024-03-21T10:36:07.685 INFO:teuthology.orchestra.run.smithi157.stdout: "pgmap_bytes": 0, 2024-03-21T10:36:07.685 INFO:teuthology.orchestra.run.smithi157.stdout: "pgmap_items": 0, 2024-03-21T10:36:07.685 INFO:teuthology.orchestra.run.smithi157.stdout: "mds_co_bytes": 8320, 2024-03-21T10:36:07.685 INFO:teuthology.orchestra.run.smithi157.stdout: "mds_co_items": 13, 2024-03-21T10:36:07.685 INFO:teuthology.orchestra.run.smithi157.stdout: "unittest_1_bytes": 0, 2024-03-21T10:36:07.685 INFO:teuthology.orchestra.run.smithi157.stdout: "unittest_1_items": 0, 2024-03-21T10:36:07.685 INFO:teuthology.orchestra.run.smithi157.stdout: "unittest_2_bytes": 0, 2024-03-21T10:36:07.685 INFO:teuthology.orchestra.run.smithi157.stdout: "unittest_2_items": 0 2024-03-21T10:36:07.685 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-03-21T10:36:07.685 INFO:teuthology.orchestra.run.smithi157.stdout: "objecter": { 2024-03-21T10:36:07.685 INFO:teuthology.orchestra.run.smithi157.stdout: "op_active": 0, 2024-03-21T10:36:07.685 INFO:teuthology.orchestra.run.smithi157.stdout: "op_laggy": 0, 2024-03-21T10:36:07.685 INFO:teuthology.orchestra.run.smithi157.stdout: "op_send": 2312, 2024-03-21T10:36:07.685 INFO:teuthology.orchestra.run.smithi157.stdout: "op_send_bytes": 21, 2024-03-21T10:36:07.686 INFO:teuthology.orchestra.run.smithi157.stdout: "op_resend": 3, 2024-03-21T10:36:07.686 INFO:teuthology.orchestra.run.smithi157.stdout: "op_reply": 2312, 2024-03-21T10:36:07.686 INFO:teuthology.orchestra.run.smithi157.stdout: "op_latency": { 2024-03-21T10:36:07.686 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 2312, 2024-03-21T10:36:07.686 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 2.521954133, 2024-03-21T10:36:07.686 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.001090810 2024-03-21T10:36:07.686 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-03-21T10:36:07.686 INFO:teuthology.orchestra.run.smithi157.stdout: "op_inflight": 0, 2024-03-21T10:36:07.686 INFO:teuthology.orchestra.run.smithi157.stdout: "oplen_avg": { 2024-03-21T10:36:07.686 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 2312, 2024-03-21T10:36:07.686 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 2313 2024-03-21T10:36:07.686 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-03-21T10:36:07.686 INFO:teuthology.orchestra.run.smithi157.stdout: "op": 2312, 2024-03-21T10:36:07.686 INFO:teuthology.orchestra.run.smithi157.stdout: "op_r": 2312, 2024-03-21T10:36:07.686 INFO:teuthology.orchestra.run.smithi157.stdout: "op_w": 0, 2024-03-21T10:36:07.687 INFO:teuthology.orchestra.run.smithi157.stdout: "op_rmw": 0, 2024-03-21T10:36:07.687 INFO:teuthology.orchestra.run.smithi157.stdout: "op_pg": 0, 2024-03-21T10:36:07.687 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_stat": 1486, 2024-03-21T10:36:07.687 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_create": 0, 2024-03-21T10:36:07.687 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_read": 824, 2024-03-21T10:36:07.687 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_write": 0, 2024-03-21T10:36:07.687 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_writefull": 0, 2024-03-21T10:36:07.687 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_writesame": 0, 2024-03-21T10:36:07.687 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_append": 0, 2024-03-21T10:36:07.687 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_zero": 0, 2024-03-21T10:36:07.687 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_truncate": 0, 2024-03-21T10:36:07.687 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_delete": 0, 2024-03-21T10:36:07.687 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_mapext": 0, 2024-03-21T10:36:07.687 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_sparse_read": 0, 2024-03-21T10:36:07.687 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_clonerange": 0, 2024-03-21T10:36:07.688 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_getxattr": 1, 2024-03-21T10:36:07.688 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_setxattr": 0, 2024-03-21T10:36:07.688 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_cmpxattr": 0, 2024-03-21T10:36:07.688 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_rmxattr": 0, 2024-03-21T10:36:07.688 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_resetxattrs": 0, 2024-03-21T10:36:07.688 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_call": 0, 2024-03-21T10:36:07.688 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_watch": 0, 2024-03-21T10:36:07.688 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_notify": 0, 2024-03-21T10:36:07.688 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_src_cmpxattr": 0, 2024-03-21T10:36:07.688 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_pgls": 0, 2024-03-21T10:36:07.688 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_pgls_filter": 0, 2024-03-21T10:36:07.688 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_other": 0, 2024-03-21T10:36:07.688 INFO:teuthology.orchestra.run.smithi157.stdout: "linger_active": 0, 2024-03-21T10:36:07.688 INFO:teuthology.orchestra.run.smithi157.stdout: "linger_send": 0, 2024-03-21T10:36:07.688 INFO:teuthology.orchestra.run.smithi157.stdout: "linger_resend": 0, 2024-03-21T10:36:07.689 INFO:teuthology.orchestra.run.smithi157.stdout: "linger_ping": 0, 2024-03-21T10:36:07.689 INFO:teuthology.orchestra.run.smithi157.stdout: "poolop_active": 0, 2024-03-21T10:36:07.689 INFO:teuthology.orchestra.run.smithi157.stdout: "poolop_send": 0, 2024-03-21T10:36:07.689 INFO:teuthology.orchestra.run.smithi157.stdout: "poolop_resend": 0, 2024-03-21T10:36:07.689 INFO:teuthology.orchestra.run.smithi157.stdout: "poolstat_active": 0, 2024-03-21T10:36:07.689 INFO:teuthology.orchestra.run.smithi157.stdout: "poolstat_send": 0, 2024-03-21T10:36:07.689 INFO:teuthology.orchestra.run.smithi157.stdout: "poolstat_resend": 0, 2024-03-21T10:36:07.689 INFO:teuthology.orchestra.run.smithi157.stdout: "statfs_active": 0, 2024-03-21T10:36:07.689 INFO:teuthology.orchestra.run.smithi157.stdout: "statfs_send": 0, 2024-03-21T10:36:07.689 INFO:teuthology.orchestra.run.smithi157.stdout: "statfs_resend": 0, 2024-03-21T10:36:07.689 INFO:teuthology.orchestra.run.smithi157.stdout: "command_active": 0, 2024-03-21T10:36:07.689 INFO:teuthology.orchestra.run.smithi157.stdout: "command_send": 0, 2024-03-21T10:36:07.689 INFO:teuthology.orchestra.run.smithi157.stdout: "command_resend": 0, 2024-03-21T10:36:07.689 INFO:teuthology.orchestra.run.smithi157.stdout: "map_epoch": 105, 2024-03-21T10:36:07.689 INFO:teuthology.orchestra.run.smithi157.stdout: "map_full": 0, 2024-03-21T10:36:07.689 INFO:teuthology.orchestra.run.smithi157.stdout: "map_inc": 12, 2024-03-21T10:36:07.689 INFO:teuthology.orchestra.run.smithi157.stdout: "osd_sessions": 4, 2024-03-21T10:36:07.690 INFO:teuthology.orchestra.run.smithi157.stdout: "osd_session_open": 4, 2024-03-21T10:36:07.690 INFO:teuthology.orchestra.run.smithi157.stdout: "osd_session_close": 0, 2024-03-21T10:36:07.690 INFO:teuthology.orchestra.run.smithi157.stdout: "osd_laggy": 0, 2024-03-21T10:36:07.690 INFO:teuthology.orchestra.run.smithi157.stdout: "omap_wr": 0, 2024-03-21T10:36:07.690 INFO:teuthology.orchestra.run.smithi157.stdout: "omap_rd": 2, 2024-03-21T10:36:07.690 INFO:teuthology.orchestra.run.smithi157.stdout: "omap_del": 0 2024-03-21T10:36:07.690 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-03-21T10:36:07.690 INFO:teuthology.orchestra.run.smithi157.stdout: "oft": { 2024-03-21T10:36:07.690 INFO:teuthology.orchestra.run.smithi157.stdout: "omap_total_objs": 0, 2024-03-21T10:36:07.690 INFO:teuthology.orchestra.run.smithi157.stdout: "omap_total_kv_pairs": 0, 2024-03-21T10:36:07.690 INFO:teuthology.orchestra.run.smithi157.stdout: "omap_total_updates": 0, 2024-03-21T10:36:07.690 INFO:teuthology.orchestra.run.smithi157.stdout: "omap_total_removes": 0 2024-03-21T10:36:07.690 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-03-21T10:36:07.690 INFO:teuthology.orchestra.run.smithi157.stdout: "osd-slow-ops": { 2024-03-21T10:36:07.690 INFO:teuthology.orchestra.run.smithi157.stdout: "slow_ops_count": 0 2024-03-21T10:36:07.691 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-03-21T10:36:07.691 INFO:teuthology.orchestra.run.smithi157.stdout: "purge_queue": { 2024-03-21T10:36:07.691 INFO:teuthology.orchestra.run.smithi157.stdout: "pq_executing_ops": 0, 2024-03-21T10:36:07.691 INFO:teuthology.orchestra.run.smithi157.stdout: "pq_executing_ops_high_water": 0, 2024-03-21T10:36:07.691 INFO:teuthology.orchestra.run.smithi157.stdout: "pq_executing": 0, 2024-03-21T10:36:07.691 INFO:teuthology.orchestra.run.smithi157.stdout: "pq_executing_high_water": 0, 2024-03-21T10:36:07.691 INFO:teuthology.orchestra.run.smithi157.stdout: "pq_executed_ops": 0, 2024-03-21T10:36:07.691 INFO:teuthology.orchestra.run.smithi157.stdout: "pq_executed": 0, 2024-03-21T10:36:07.691 INFO:teuthology.orchestra.run.smithi157.stdout: "pq_item_in_journal": 0 2024-03-21T10:36:07.691 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-03-21T10:36:07.691 INFO:teuthology.orchestra.run.smithi157.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-03-21T10:36:07.691 INFO:teuthology.orchestra.run.smithi157.stdout: "val": 0, 2024-03-21T10:36:07.691 INFO:teuthology.orchestra.run.smithi157.stdout: "max": 104857600, 2024-03-21T10:36:07.691 INFO:teuthology.orchestra.run.smithi157.stdout: "get_started": 0, 2024-03-21T10:36:07.691 INFO:teuthology.orchestra.run.smithi157.stdout: "get": 2583, 2024-03-21T10:36:07.691 INFO:teuthology.orchestra.run.smithi157.stdout: "get_sum": 2198324, 2024-03-21T10:36:07.691 INFO:teuthology.orchestra.run.smithi157.stdout: "get_or_fail_fail": 0, 2024-03-21T10:36:07.692 INFO:teuthology.orchestra.run.smithi157.stdout: "get_or_fail_success": 2583, 2024-03-21T10:36:07.692 INFO:teuthology.orchestra.run.smithi157.stdout: "take": 0, 2024-03-21T10:36:07.692 INFO:teuthology.orchestra.run.smithi157.stdout: "take_sum": 0, 2024-03-21T10:36:07.692 INFO:teuthology.orchestra.run.smithi157.stdout: "put": 2583, 2024-03-21T10:36:07.692 INFO:teuthology.orchestra.run.smithi157.stdout: "put_sum": 2198324, 2024-03-21T10:36:07.692 INFO:teuthology.orchestra.run.smithi157.stdout: "wait": { 2024-03-21T10:36:07.692 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-03-21T10:36:07.692 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-03-21T10:36:07.692 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-03-21T10:36:07.692 INFO:teuthology.orchestra.run.smithi157.stdout: } 2024-03-21T10:36:07.692 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-03-21T10:36:07.692 INFO:teuthology.orchestra.run.smithi157.stdout: "throttle-objecter_bytes": { 2024-03-21T10:36:07.692 INFO:teuthology.orchestra.run.smithi157.stdout: "val": 0, 2024-03-21T10:36:07.692 INFO:teuthology.orchestra.run.smithi157.stdout: "max": 104857600, 2024-03-21T10:36:07.692 INFO:teuthology.orchestra.run.smithi157.stdout: "get_started": 0, 2024-03-21T10:36:07.693 INFO:teuthology.orchestra.run.smithi157.stdout: "get": 0, 2024-03-21T10:36:07.693 INFO:teuthology.orchestra.run.smithi157.stdout: "get_sum": 0, 2024-03-21T10:36:07.693 INFO:teuthology.orchestra.run.smithi157.stdout: "get_or_fail_fail": 0, 2024-03-21T10:36:07.693 INFO:teuthology.orchestra.run.smithi157.stdout: "get_or_fail_success": 0, 2024-03-21T10:36:07.693 INFO:teuthology.orchestra.run.smithi157.stdout: "take": 2312, 2024-03-21T10:36:07.693 INFO:teuthology.orchestra.run.smithi157.stdout: "take_sum": 1618, 2024-03-21T10:36:07.693 INFO:teuthology.orchestra.run.smithi157.stdout: "put": 2312, 2024-03-21T10:36:07.693 INFO:teuthology.orchestra.run.smithi157.stdout: "put_sum": 1618, 2024-03-21T10:36:07.693 INFO:teuthology.orchestra.run.smithi157.stdout: "wait": { 2024-03-21T10:36:07.693 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-03-21T10:36:07.693 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-03-21T10:36:07.693 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-03-21T10:36:07.693 INFO:teuthology.orchestra.run.smithi157.stdout: } 2024-03-21T10:36:07.693 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-03-21T10:36:07.693 INFO:teuthology.orchestra.run.smithi157.stdout: "throttle-objecter_ops": { 2024-03-21T10:36:07.694 INFO:teuthology.orchestra.run.smithi157.stdout: "val": 0, 2024-03-21T10:36:07.694 INFO:teuthology.orchestra.run.smithi157.stdout: "max": 1024, 2024-03-21T10:36:07.694 INFO:teuthology.orchestra.run.smithi157.stdout: "get_started": 0, 2024-03-21T10:36:07.694 INFO:teuthology.orchestra.run.smithi157.stdout: "get": 0, 2024-03-21T10:36:07.694 INFO:teuthology.orchestra.run.smithi157.stdout: "get_sum": 0, 2024-03-21T10:36:07.694 INFO:teuthology.orchestra.run.smithi157.stdout: "get_or_fail_fail": 0, 2024-03-21T10:36:07.694 INFO:teuthology.orchestra.run.smithi157.stdout: "get_or_fail_success": 0, 2024-03-21T10:36:07.694 INFO:teuthology.orchestra.run.smithi157.stdout: "take": 2312, 2024-03-21T10:36:07.694 INFO:teuthology.orchestra.run.smithi157.stdout: "take_sum": 2312, 2024-03-21T10:36:07.694 INFO:teuthology.orchestra.run.smithi157.stdout: "put": 2312, 2024-03-21T10:36:07.694 INFO:teuthology.orchestra.run.smithi157.stdout: "put_sum": 2312, 2024-03-21T10:36:07.694 INFO:teuthology.orchestra.run.smithi157.stdout: "wait": { 2024-03-21T10:36:07.694 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-03-21T10:36:07.694 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-03-21T10:36:07.695 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-03-21T10:36:07.695 INFO:teuthology.orchestra.run.smithi157.stdout: } 2024-03-21T10:36:07.695 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-03-21T10:36:07.695 INFO:teuthology.orchestra.run.smithi157.stdout: "throttle-write_buf_throttle": { 2024-03-21T10:36:07.695 INFO:teuthology.orchestra.run.smithi157.stdout: "val": 0, 2024-03-21T10:36:07.695 INFO:teuthology.orchestra.run.smithi157.stdout: "max": 3758096384, 2024-03-21T10:36:07.695 INFO:teuthology.orchestra.run.smithi157.stdout: "get_started": 0, 2024-03-21T10:36:07.695 INFO:teuthology.orchestra.run.smithi157.stdout: "get": 0, 2024-03-21T10:36:07.695 INFO:teuthology.orchestra.run.smithi157.stdout: "get_sum": 0, 2024-03-21T10:36:07.695 INFO:teuthology.orchestra.run.smithi157.stdout: "get_or_fail_fail": 0, 2024-03-21T10:36:07.695 INFO:teuthology.orchestra.run.smithi157.stdout: "get_or_fail_success": 0, 2024-03-21T10:36:07.695 INFO:teuthology.orchestra.run.smithi157.stdout: "take": 0, 2024-03-21T10:36:07.695 INFO:teuthology.orchestra.run.smithi157.stdout: "take_sum": 0, 2024-03-21T10:36:07.695 INFO:teuthology.orchestra.run.smithi157.stdout: "put": 0, 2024-03-21T10:36:07.696 INFO:teuthology.orchestra.run.smithi157.stdout: "put_sum": 0, 2024-03-21T10:36:07.696 INFO:teuthology.orchestra.run.smithi157.stdout: "wait": { 2024-03-21T10:36:07.696 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-03-21T10:36:07.696 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-03-21T10:36:07.696 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-03-21T10:36:07.696 INFO:teuthology.orchestra.run.smithi157.stdout: } 2024-03-21T10:36:07.696 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-03-21T10:36:07.696 INFO:teuthology.orchestra.run.smithi157.stdout: "throttle-write_buf_throttle-0x562a063dc780": { 2024-03-21T10:36:07.696 INFO:teuthology.orchestra.run.smithi157.stdout: "val": 0, 2024-03-21T10:36:07.696 INFO:teuthology.orchestra.run.smithi157.stdout: "max": 3758096384, 2024-03-21T10:36:07.696 INFO:teuthology.orchestra.run.smithi157.stdout: "get_started": 0, 2024-03-21T10:36:07.696 INFO:teuthology.orchestra.run.smithi157.stdout: "get": 0, 2024-03-21T10:36:07.696 INFO:teuthology.orchestra.run.smithi157.stdout: "get_sum": 0, 2024-03-21T10:36:07.696 INFO:teuthology.orchestra.run.smithi157.stdout: "get_or_fail_fail": 0, 2024-03-21T10:36:07.696 INFO:teuthology.orchestra.run.smithi157.stdout: "get_or_fail_success": 0, 2024-03-21T10:36:07.696 INFO:teuthology.orchestra.run.smithi157.stdout: "take": 0, 2024-03-21T10:36:07.697 INFO:teuthology.orchestra.run.smithi157.stdout: "take_sum": 0, 2024-03-21T10:36:07.697 INFO:teuthology.orchestra.run.smithi157.stdout: "put": 0, 2024-03-21T10:36:07.697 INFO:teuthology.orchestra.run.smithi157.stdout: "put_sum": 0, 2024-03-21T10:36:07.697 INFO:teuthology.orchestra.run.smithi157.stdout: "wait": { 2024-03-21T10:36:07.697 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-03-21T10:36:07.697 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-03-21T10:36:07.697 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-03-21T10:36:07.697 INFO:teuthology.orchestra.run.smithi157.stdout: } 2024-03-21T10:36:07.697 INFO:teuthology.orchestra.run.smithi157.stdout: } 2024-03-21T10:36:07.697 INFO:teuthology.orchestra.run.smithi157.stdout:} 2024-03-21T10:36:07.960 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:36:07 smithi047 ceph-mon[31547]: pgmap v335: 129 pgs: 129 active+clean; 2.5 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 44 KiB/s wr, 29 op/s 2024-03-21T10:36:07.976 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:36:07 smithi100 ceph-mon[35193]: pgmap v335: 129 pgs: 129 active+clean; 2.5 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 44 KiB/s wr, 29 op/s 2024-03-21T10:36:07.986 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:36:07 smithi157 ceph-mon[35581]: pgmap v335: 129 pgs: 129 active+clean; 2.5 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 44 KiB/s wr, 29 op/s 2024-03-21T10:36:08.650 INFO:tasks.check_counter:Daemon mds.c mds.exported=0 2024-03-21T10:36:08.650 INFO:tasks.check_counter:Daemon mds.c mds.imported=0 2024-03-21T10:36:08.650 INFO:tasks.check_counter:Daemon mds.c mds.root_rsnaps=0 2024-03-21T10:36:08.650 INFO:tasks.check_counter:Daemon mds.c mds_server.req_mksnap_latency.avgcount=0 2024-03-21T10:36:08.650 INFO:tasks.check_counter:Daemon mds.c mds_server.req_rmsnap_latency.avgcount=0 2024-03-21T10:36:08.650 DEBUG:tasks.check_counter:Getting stats from f 2024-03-21T10:36:08.650 DEBUG:teuthology.orchestra.run.smithi157:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2edfee68ad7243bed422a049b6f5f58619300f11 shell --fsid 3056d51a-e76b-11ee-95cd-87774f69a715 -- ceph daemon mds.f perf dump 2024-03-21T10:36:09.078 INFO:teuthology.orchestra.run.smithi157.stderr:Inferring config /var/lib/ceph/3056d51a-e76b-11ee-95cd-87774f69a715/mon.c/config 2024-03-21T10:36:09.960 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:36:09 smithi047 ceph-mon[31547]: pgmap v336: 129 pgs: 129 active+clean; 2.5 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 44 KiB/s wr, 26 op/s 2024-03-21T10:36:09.976 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:36:09 smithi100 ceph-mon[35193]: pgmap v336: 129 pgs: 129 active+clean; 2.5 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 44 KiB/s wr, 26 op/s 2024-03-21T10:36:09.987 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:36:09 smithi157 ceph-mon[35581]: pgmap v336: 129 pgs: 129 active+clean; 2.5 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 44 KiB/s wr, 26 op/s 2024-03-21T10:36:10.824 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:36:10 smithi157 ceph-mon[35581]: osdmap e106: 12 total, 12 up, 12 in 2024-03-21T10:36:10.961 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:36:10 smithi047 ceph-mon[31547]: osdmap e106: 12 total, 12 up, 12 in 2024-03-21T10:36:10.976 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:36:10 smithi100 ceph-mon[35193]: osdmap e106: 12 total, 12 up, 12 in 2024-03-21T10:36:11.358 INFO:teuthology.orchestra.run.smithi157.stdout:{ 2024-03-21T10:36:11.358 INFO:teuthology.orchestra.run.smithi157.stdout: "AsyncMessenger::Worker-0": { 2024-03-21T10:36:11.358 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_recv_messages": 828, 2024-03-21T10:36:11.358 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_send_messages": 830, 2024-03-21T10:36:11.358 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_recv_bytes": 218927, 2024-03-21T10:36:11.358 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_send_bytes": 263367, 2024-03-21T10:36:11.358 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_created_connections": 6, 2024-03-21T10:36:11.358 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_active_connections": 2, 2024-03-21T10:36:11.359 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_running_total_time": 0.170564510, 2024-03-21T10:36:11.359 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_running_send_time": 0.060227226, 2024-03-21T10:36:11.359 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_running_recv_time": 0.043277550, 2024-03-21T10:36:11.359 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_running_fast_dispatch_time": 0.045748194, 2024-03-21T10:36:11.359 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_send_messages_queue_lat": { 2024-03-21T10:36:11.359 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 828, 2024-03-21T10:36:11.359 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.082826311, 2024-03-21T10:36:11.359 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000100031 2024-03-21T10:36:11.359 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-03-21T10:36:11.359 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_handle_ack_lat": { 2024-03-21T10:36:11.359 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-03-21T10:36:11.359 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-03-21T10:36:11.359 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-03-21T10:36:11.359 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-03-21T10:36:11.359 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_recv_encrypted_bytes": 0, 2024-03-21T10:36:11.360 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_send_encrypted_bytes": 0 2024-03-21T10:36:11.360 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-03-21T10:36:11.360 INFO:teuthology.orchestra.run.smithi157.stdout: "AsyncMessenger::Worker-1": { 2024-03-21T10:36:11.360 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_recv_messages": 4, 2024-03-21T10:36:11.360 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_send_messages": 183, 2024-03-21T10:36:11.360 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_recv_bytes": 1422, 2024-03-21T10:36:11.360 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_send_bytes": 404898, 2024-03-21T10:36:11.360 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_created_connections": 7, 2024-03-21T10:36:11.360 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_active_connections": 5, 2024-03-21T10:36:11.360 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_running_total_time": 0.029317430, 2024-03-21T10:36:11.360 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_running_send_time": 0.015327084, 2024-03-21T10:36:11.360 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_running_recv_time": 0.008271779, 2024-03-21T10:36:11.360 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_running_fast_dispatch_time": 0.000064015, 2024-03-21T10:36:11.360 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_send_messages_queue_lat": { 2024-03-21T10:36:11.361 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 180, 2024-03-21T10:36:11.361 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.035482172, 2024-03-21T10:36:11.361 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000197123 2024-03-21T10:36:11.361 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-03-21T10:36:11.361 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_handle_ack_lat": { 2024-03-21T10:36:11.361 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-03-21T10:36:11.361 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-03-21T10:36:11.361 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-03-21T10:36:11.361 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-03-21T10:36:11.361 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_recv_encrypted_bytes": 320, 2024-03-21T10:36:11.361 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_send_encrypted_bytes": 404256 2024-03-21T10:36:11.361 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-03-21T10:36:11.361 INFO:teuthology.orchestra.run.smithi157.stdout: "AsyncMessenger::Worker-2": { 2024-03-21T10:36:11.361 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_recv_messages": 1830, 2024-03-21T10:36:11.362 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_send_messages": 1810, 2024-03-21T10:36:11.362 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_recv_bytes": 2247550, 2024-03-21T10:36:11.362 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_send_bytes": 639532, 2024-03-21T10:36:11.362 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_created_connections": 4, 2024-03-21T10:36:11.362 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_active_connections": 3, 2024-03-21T10:36:11.362 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_running_total_time": 0.435566370, 2024-03-21T10:36:11.362 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_running_send_time": 0.140298189, 2024-03-21T10:36:11.362 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_running_recv_time": 0.182192406, 2024-03-21T10:36:11.362 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_running_fast_dispatch_time": 0.111869104, 2024-03-21T10:36:11.362 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_send_messages_queue_lat": { 2024-03-21T10:36:11.362 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 1810, 2024-03-21T10:36:11.362 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.161812220, 2024-03-21T10:36:11.362 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000089399 2024-03-21T10:36:11.362 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-03-21T10:36:11.363 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_handle_ack_lat": { 2024-03-21T10:36:11.363 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-03-21T10:36:11.363 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-03-21T10:36:11.363 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-03-21T10:36:11.363 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-03-21T10:36:11.363 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_recv_encrypted_bytes": 1791376, 2024-03-21T10:36:11.363 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_send_encrypted_bytes": 145360 2024-03-21T10:36:11.363 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-03-21T10:36:11.363 INFO:teuthology.orchestra.run.smithi157.stdout: "cct": { 2024-03-21T10:36:11.363 INFO:teuthology.orchestra.run.smithi157.stdout: "total_workers": 1, 2024-03-21T10:36:11.363 INFO:teuthology.orchestra.run.smithi157.stdout: "unhealthy_workers": 0 2024-03-21T10:36:11.363 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-03-21T10:36:11.363 INFO:teuthology.orchestra.run.smithi157.stdout: "finisher-MDSRank": { 2024-03-21T10:36:11.363 INFO:teuthology.orchestra.run.smithi157.stdout: "queue_len": 0, 2024-03-21T10:36:11.363 INFO:teuthology.orchestra.run.smithi157.stdout: "complete_latency": { 2024-03-21T10:36:11.364 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 3937, 2024-03-21T10:36:11.364 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.215708044, 2024-03-21T10:36:11.364 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000054789 2024-03-21T10:36:11.364 INFO:teuthology.orchestra.run.smithi157.stdout: } 2024-03-21T10:36:11.364 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-03-21T10:36:11.364 INFO:teuthology.orchestra.run.smithi157.stdout: "finisher-PurgeQueue": { 2024-03-21T10:36:11.364 INFO:teuthology.orchestra.run.smithi157.stdout: "queue_len": 0, 2024-03-21T10:36:11.364 INFO:teuthology.orchestra.run.smithi157.stdout: "complete_latency": { 2024-03-21T10:36:11.364 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-03-21T10:36:11.364 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-03-21T10:36:11.364 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-03-21T10:36:11.364 INFO:teuthology.orchestra.run.smithi157.stdout: } 2024-03-21T10:36:11.364 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-03-21T10:36:11.365 INFO:teuthology.orchestra.run.smithi157.stdout: "mds": { 2024-03-21T10:36:11.365 INFO:teuthology.orchestra.run.smithi157.stdout: "request": 0, 2024-03-21T10:36:11.365 INFO:teuthology.orchestra.run.smithi157.stdout: "reply": 0, 2024-03-21T10:36:11.365 INFO:teuthology.orchestra.run.smithi157.stdout: "reply_latency": { 2024-03-21T10:36:11.365 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-03-21T10:36:11.365 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-03-21T10:36:11.365 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-03-21T10:36:11.365 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-03-21T10:36:11.365 INFO:teuthology.orchestra.run.smithi157.stdout: "slow_reply": 0, 2024-03-21T10:36:11.365 INFO:teuthology.orchestra.run.smithi157.stdout: "forward": 0, 2024-03-21T10:36:11.365 INFO:teuthology.orchestra.run.smithi157.stdout: "dir_fetch_complete": 0, 2024-03-21T10:36:11.365 INFO:teuthology.orchestra.run.smithi157.stdout: "dir_fetch_keys": 0, 2024-03-21T10:36:11.365 INFO:teuthology.orchestra.run.smithi157.stdout: "dir_commit": 0, 2024-03-21T10:36:11.365 INFO:teuthology.orchestra.run.smithi157.stdout: "dir_split": 0, 2024-03-21T10:36:11.366 INFO:teuthology.orchestra.run.smithi157.stdout: "dir_merge": 0, 2024-03-21T10:36:11.366 INFO:teuthology.orchestra.run.smithi157.stdout: "inodes": 0, 2024-03-21T10:36:11.366 INFO:teuthology.orchestra.run.smithi157.stdout: "inodes_top": 0, 2024-03-21T10:36:11.366 INFO:teuthology.orchestra.run.smithi157.stdout: "inodes_bottom": 0, 2024-03-21T10:36:11.366 INFO:teuthology.orchestra.run.smithi157.stdout: "inodes_pin_tail": 0, 2024-03-21T10:36:11.366 INFO:teuthology.orchestra.run.smithi157.stdout: "inodes_pinned": 0, 2024-03-21T10:36:11.366 INFO:teuthology.orchestra.run.smithi157.stdout: "inodes_expired": 0, 2024-03-21T10:36:11.366 INFO:teuthology.orchestra.run.smithi157.stdout: "inodes_with_caps": 0, 2024-03-21T10:36:11.366 INFO:teuthology.orchestra.run.smithi157.stdout: "caps": 0, 2024-03-21T10:36:11.366 INFO:teuthology.orchestra.run.smithi157.stdout: "subtrees": 1, 2024-03-21T10:36:11.366 INFO:teuthology.orchestra.run.smithi157.stdout: "traverse": 0, 2024-03-21T10:36:11.366 INFO:teuthology.orchestra.run.smithi157.stdout: "traverse_hit": 0, 2024-03-21T10:36:11.366 INFO:teuthology.orchestra.run.smithi157.stdout: "traverse_forward": 0, 2024-03-21T10:36:11.366 INFO:teuthology.orchestra.run.smithi157.stdout: "traverse_discover": 0, 2024-03-21T10:36:11.367 INFO:teuthology.orchestra.run.smithi157.stdout: "traverse_dir_fetch": 0, 2024-03-21T10:36:11.367 INFO:teuthology.orchestra.run.smithi157.stdout: "traverse_remote_ino": 0, 2024-03-21T10:36:11.367 INFO:teuthology.orchestra.run.smithi157.stdout: "traverse_lock": 0, 2024-03-21T10:36:11.367 INFO:teuthology.orchestra.run.smithi157.stdout: "load_cent": 0, 2024-03-21T10:36:11.367 INFO:teuthology.orchestra.run.smithi157.stdout: "q": 0, 2024-03-21T10:36:11.367 INFO:teuthology.orchestra.run.smithi157.stdout: "exported": 0, 2024-03-21T10:36:11.367 INFO:teuthology.orchestra.run.smithi157.stdout: "exported_inodes": 0, 2024-03-21T10:36:11.367 INFO:teuthology.orchestra.run.smithi157.stdout: "imported": 0, 2024-03-21T10:36:11.367 INFO:teuthology.orchestra.run.smithi157.stdout: "imported_inodes": 0, 2024-03-21T10:36:11.367 INFO:teuthology.orchestra.run.smithi157.stdout: "openino_dir_fetch": 0, 2024-03-21T10:36:11.367 INFO:teuthology.orchestra.run.smithi157.stdout: "openino_backtrace_fetch": 0, 2024-03-21T10:36:11.367 INFO:teuthology.orchestra.run.smithi157.stdout: "openino_peer_discover": 0, 2024-03-21T10:36:11.367 INFO:teuthology.orchestra.run.smithi157.stdout: "root_rfiles": 0, 2024-03-21T10:36:11.367 INFO:teuthology.orchestra.run.smithi157.stdout: "root_rbytes": 0, 2024-03-21T10:36:11.367 INFO:teuthology.orchestra.run.smithi157.stdout: "root_rsnaps": 0, 2024-03-21T10:36:11.369 INFO:teuthology.orchestra.run.smithi157.stdout: "scrub_backtrace_fetch": 0, 2024-03-21T10:36:11.369 INFO:teuthology.orchestra.run.smithi157.stdout: "scrub_set_tag": 0, 2024-03-21T10:36:11.369 INFO:teuthology.orchestra.run.smithi157.stdout: "scrub_backtrace_repaired": 0, 2024-03-21T10:36:11.369 INFO:teuthology.orchestra.run.smithi157.stdout: "scrub_inotable_repaired": 0, 2024-03-21T10:36:11.369 INFO:teuthology.orchestra.run.smithi157.stdout: "scrub_dir_inodes": 0, 2024-03-21T10:36:11.369 INFO:teuthology.orchestra.run.smithi157.stdout: "scrub_dir_base_inodes": 0, 2024-03-21T10:36:11.369 INFO:teuthology.orchestra.run.smithi157.stdout: "scrub_dirfrag_rstats": 0, 2024-03-21T10:36:11.369 INFO:teuthology.orchestra.run.smithi157.stdout: "scrub_file_inodes": 0, 2024-03-21T10:36:11.370 INFO:teuthology.orchestra.run.smithi157.stdout: "handle_inode_file_caps": 0, 2024-03-21T10:36:11.370 INFO:teuthology.orchestra.run.smithi157.stdout: "ceph_cap_op_revoke": 0, 2024-03-21T10:36:11.370 INFO:teuthology.orchestra.run.smithi157.stdout: "ceph_cap_op_grant": 0, 2024-03-21T10:36:11.370 INFO:teuthology.orchestra.run.smithi157.stdout: "ceph_cap_op_trunc": 0, 2024-03-21T10:36:11.370 INFO:teuthology.orchestra.run.smithi157.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-03-21T10:36:11.370 INFO:teuthology.orchestra.run.smithi157.stdout: "ceph_cap_op_flush_ack": 0, 2024-03-21T10:36:11.370 INFO:teuthology.orchestra.run.smithi157.stdout: "handle_client_caps": 0, 2024-03-21T10:36:11.370 INFO:teuthology.orchestra.run.smithi157.stdout: "handle_client_caps_dirty": 0, 2024-03-21T10:36:11.370 INFO:teuthology.orchestra.run.smithi157.stdout: "handle_client_cap_release": 0, 2024-03-21T10:36:11.370 INFO:teuthology.orchestra.run.smithi157.stdout: "process_request_cap_release": 0 2024-03-21T10:36:11.370 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-03-21T10:36:11.370 INFO:teuthology.orchestra.run.smithi157.stdout: "mds_cache": { 2024-03-21T10:36:11.370 INFO:teuthology.orchestra.run.smithi157.stdout: "dir_update": 0, 2024-03-21T10:36:11.370 INFO:teuthology.orchestra.run.smithi157.stdout: "dir_update_receipt": 0, 2024-03-21T10:36:11.370 INFO:teuthology.orchestra.run.smithi157.stdout: "dir_try_discover": 0, 2024-03-21T10:36:11.371 INFO:teuthology.orchestra.run.smithi157.stdout: "dir_send_discover": 0, 2024-03-21T10:36:11.371 INFO:teuthology.orchestra.run.smithi157.stdout: "dir_handle_discover": 0, 2024-03-21T10:36:11.371 INFO:teuthology.orchestra.run.smithi157.stdout: "num_strays": 0, 2024-03-21T10:36:11.371 INFO:teuthology.orchestra.run.smithi157.stdout: "num_strays_delayed": 0, 2024-03-21T10:36:11.371 INFO:teuthology.orchestra.run.smithi157.stdout: "num_strays_enqueuing": 0, 2024-03-21T10:36:11.371 INFO:teuthology.orchestra.run.smithi157.stdout: "strays_created": 0, 2024-03-21T10:36:11.371 INFO:teuthology.orchestra.run.smithi157.stdout: "strays_enqueued": 0, 2024-03-21T10:36:11.371 INFO:teuthology.orchestra.run.smithi157.stdout: "strays_reintegrated": 0, 2024-03-21T10:36:11.371 INFO:teuthology.orchestra.run.smithi157.stdout: "strays_migrated": 0, 2024-03-21T10:36:11.371 INFO:teuthology.orchestra.run.smithi157.stdout: "num_recovering_processing": 0, 2024-03-21T10:36:11.372 INFO:teuthology.orchestra.run.smithi157.stdout: "num_recovering_enqueued": 0, 2024-03-21T10:36:11.372 INFO:teuthology.orchestra.run.smithi157.stdout: "num_recovering_prioritized": 0, 2024-03-21T10:36:11.372 INFO:teuthology.orchestra.run.smithi157.stdout: "recovery_started": 0, 2024-03-21T10:36:11.372 INFO:teuthology.orchestra.run.smithi157.stdout: "recovery_completed": 0, 2024-03-21T10:36:11.372 INFO:teuthology.orchestra.run.smithi157.stdout: "ireq_enqueue_scrub": 0, 2024-03-21T10:36:11.372 INFO:teuthology.orchestra.run.smithi157.stdout: "ireq_exportdir": 0, 2024-03-21T10:36:11.372 INFO:teuthology.orchestra.run.smithi157.stdout: "ireq_flush": 0, 2024-03-21T10:36:11.372 INFO:teuthology.orchestra.run.smithi157.stdout: "ireq_fragmentdir": 0, 2024-03-21T10:36:11.372 INFO:teuthology.orchestra.run.smithi157.stdout: "ireq_fragstats": 0, 2024-03-21T10:36:11.372 INFO:teuthology.orchestra.run.smithi157.stdout: "ireq_inodestats": 0 2024-03-21T10:36:11.372 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-03-21T10:36:11.372 INFO:teuthology.orchestra.run.smithi157.stdout: "mds_log": { 2024-03-21T10:36:11.372 INFO:teuthology.orchestra.run.smithi157.stdout: "evlrg": 0, 2024-03-21T10:36:11.372 INFO:teuthology.orchestra.run.smithi157.stdout: "evadd": 0, 2024-03-21T10:36:11.372 INFO:teuthology.orchestra.run.smithi157.stdout: "evex": 0, 2024-03-21T10:36:11.373 INFO:teuthology.orchestra.run.smithi157.stdout: "evtrm": 0, 2024-03-21T10:36:11.373 INFO:teuthology.orchestra.run.smithi157.stdout: "ev": 6, 2024-03-21T10:36:11.373 INFO:teuthology.orchestra.run.smithi157.stdout: "evexg": 0, 2024-03-21T10:36:11.373 INFO:teuthology.orchestra.run.smithi157.stdout: "evexd": 0, 2024-03-21T10:36:11.373 INFO:teuthology.orchestra.run.smithi157.stdout: "segadd": 0, 2024-03-21T10:36:11.373 INFO:teuthology.orchestra.run.smithi157.stdout: "segex": 0, 2024-03-21T10:36:11.373 INFO:teuthology.orchestra.run.smithi157.stdout: "segtrm": 0, 2024-03-21T10:36:11.373 INFO:teuthology.orchestra.run.smithi157.stdout: "seg": 2, 2024-03-21T10:36:11.373 INFO:teuthology.orchestra.run.smithi157.stdout: "segmjr": 2, 2024-03-21T10:36:11.373 INFO:teuthology.orchestra.run.smithi157.stdout: "segexg": 0, 2024-03-21T10:36:11.373 INFO:teuthology.orchestra.run.smithi157.stdout: "segexd": 0, 2024-03-21T10:36:11.373 INFO:teuthology.orchestra.run.smithi157.stdout: "expos": 4194304, 2024-03-21T10:36:11.373 INFO:teuthology.orchestra.run.smithi157.stdout: "wrpos": 4196308, 2024-03-21T10:36:11.373 INFO:teuthology.orchestra.run.smithi157.stdout: "rdpos": 4195917, 2024-03-21T10:36:11.373 INFO:teuthology.orchestra.run.smithi157.stdout: "jlat": { 2024-03-21T10:36:11.373 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-03-21T10:36:11.374 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-03-21T10:36:11.374 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-03-21T10:36:11.374 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-03-21T10:36:11.374 INFO:teuthology.orchestra.run.smithi157.stdout: "replayed": 6 2024-03-21T10:36:11.374 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-03-21T10:36:11.374 INFO:teuthology.orchestra.run.smithi157.stdout: "mds_mem": { 2024-03-21T10:36:11.374 INFO:teuthology.orchestra.run.smithi157.stdout: "ino": 3, 2024-03-21T10:36:11.374 INFO:teuthology.orchestra.run.smithi157.stdout: "ino+": 3, 2024-03-21T10:36:11.374 INFO:teuthology.orchestra.run.smithi157.stdout: "ino-": 0, 2024-03-21T10:36:11.374 INFO:teuthology.orchestra.run.smithi157.stdout: "dir": 1, 2024-03-21T10:36:11.374 INFO:teuthology.orchestra.run.smithi157.stdout: "dir+": 1, 2024-03-21T10:36:11.374 INFO:teuthology.orchestra.run.smithi157.stdout: "dir-": 0, 2024-03-21T10:36:11.374 INFO:teuthology.orchestra.run.smithi157.stdout: "dn": 0, 2024-03-21T10:36:11.374 INFO:teuthology.orchestra.run.smithi157.stdout: "dn+": 0, 2024-03-21T10:36:11.374 INFO:teuthology.orchestra.run.smithi157.stdout: "dn-": 0, 2024-03-21T10:36:11.375 INFO:teuthology.orchestra.run.smithi157.stdout: "cap": 0, 2024-03-21T10:36:11.375 INFO:teuthology.orchestra.run.smithi157.stdout: "cap+": 0, 2024-03-21T10:36:11.375 INFO:teuthology.orchestra.run.smithi157.stdout: "cap-": 0, 2024-03-21T10:36:11.375 INFO:teuthology.orchestra.run.smithi157.stdout: "rss": 48484, 2024-03-21T10:36:11.375 INFO:teuthology.orchestra.run.smithi157.stdout: "heap": 215420 2024-03-21T10:36:11.375 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-03-21T10:36:11.375 INFO:teuthology.orchestra.run.smithi157.stdout: "mds_server": { 2024-03-21T10:36:11.375 INFO:teuthology.orchestra.run.smithi157.stdout: "dispatch_client_request": 0, 2024-03-21T10:36:11.375 INFO:teuthology.orchestra.run.smithi157.stdout: "dispatch_server_request": 0, 2024-03-21T10:36:11.375 INFO:teuthology.orchestra.run.smithi157.stdout: "handle_client_request": 0, 2024-03-21T10:36:11.375 INFO:teuthology.orchestra.run.smithi157.stdout: "handle_client_session": 0, 2024-03-21T10:36:11.375 INFO:teuthology.orchestra.run.smithi157.stdout: "handle_peer_request": 0, 2024-03-21T10:36:11.375 INFO:teuthology.orchestra.run.smithi157.stdout: "req_create_latency": { 2024-03-21T10:36:11.375 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-03-21T10:36:11.375 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-03-21T10:36:11.376 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-03-21T10:36:11.376 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-03-21T10:36:11.376 INFO:teuthology.orchestra.run.smithi157.stdout: "req_getattr_latency": { 2024-03-21T10:36:11.376 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-03-21T10:36:11.376 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-03-21T10:36:11.376 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-03-21T10:36:11.376 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-03-21T10:36:11.376 INFO:teuthology.orchestra.run.smithi157.stdout: "req_getfilelock_latency": { 2024-03-21T10:36:11.376 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-03-21T10:36:11.376 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-03-21T10:36:11.376 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-03-21T10:36:11.376 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-03-21T10:36:11.376 INFO:teuthology.orchestra.run.smithi157.stdout: "req_link_latency": { 2024-03-21T10:36:11.376 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-03-21T10:36:11.376 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-03-21T10:36:11.377 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-03-21T10:36:11.377 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-03-21T10:36:11.377 INFO:teuthology.orchestra.run.smithi157.stdout: "req_lookup_latency": { 2024-03-21T10:36:11.377 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-03-21T10:36:11.377 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-03-21T10:36:11.377 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-03-21T10:36:11.377 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-03-21T10:36:11.377 INFO:teuthology.orchestra.run.smithi157.stdout: "req_lookuphash_latency": { 2024-03-21T10:36:11.377 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-03-21T10:36:11.377 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-03-21T10:36:11.377 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-03-21T10:36:11.377 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-03-21T10:36:11.377 INFO:teuthology.orchestra.run.smithi157.stdout: "req_lookupino_latency": { 2024-03-21T10:36:11.377 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-03-21T10:36:11.377 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-03-21T10:36:11.378 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-03-21T10:36:11.378 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-03-21T10:36:11.378 INFO:teuthology.orchestra.run.smithi157.stdout: "req_lookupname_latency": { 2024-03-21T10:36:11.378 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-03-21T10:36:11.378 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-03-21T10:36:11.378 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-03-21T10:36:11.378 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-03-21T10:36:11.378 INFO:teuthology.orchestra.run.smithi157.stdout: "req_lookupparent_latency": { 2024-03-21T10:36:11.378 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-03-21T10:36:11.378 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-03-21T10:36:11.378 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-03-21T10:36:11.378 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-03-21T10:36:11.378 INFO:teuthology.orchestra.run.smithi157.stdout: "req_lookupsnap_latency": { 2024-03-21T10:36:11.378 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-03-21T10:36:11.378 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-03-21T10:36:11.379 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-03-21T10:36:11.379 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-03-21T10:36:11.379 INFO:teuthology.orchestra.run.smithi157.stdout: "req_lssnap_latency": { 2024-03-21T10:36:11.379 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-03-21T10:36:11.379 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-03-21T10:36:11.379 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-03-21T10:36:11.379 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-03-21T10:36:11.379 INFO:teuthology.orchestra.run.smithi157.stdout: "req_mkdir_latency": { 2024-03-21T10:36:11.379 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-03-21T10:36:11.379 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-03-21T10:36:11.379 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-03-21T10:36:11.379 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-03-21T10:36:11.380 INFO:teuthology.orchestra.run.smithi157.stdout: "req_mknod_latency": { 2024-03-21T10:36:11.380 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-03-21T10:36:11.380 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-03-21T10:36:11.380 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-03-21T10:36:11.380 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-03-21T10:36:11.380 INFO:teuthology.orchestra.run.smithi157.stdout: "req_mksnap_latency": { 2024-03-21T10:36:11.381 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-03-21T10:36:11.381 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-03-21T10:36:11.381 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-03-21T10:36:11.381 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-03-21T10:36:11.381 INFO:teuthology.orchestra.run.smithi157.stdout: "req_open_latency": { 2024-03-21T10:36:11.381 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-03-21T10:36:11.381 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-03-21T10:36:11.381 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-03-21T10:36:11.381 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-03-21T10:36:11.381 INFO:teuthology.orchestra.run.smithi157.stdout: "req_readdir_latency": { 2024-03-21T10:36:11.381 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-03-21T10:36:11.381 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-03-21T10:36:11.381 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-03-21T10:36:11.381 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-03-21T10:36:11.381 INFO:teuthology.orchestra.run.smithi157.stdout: "req_rename_latency": { 2024-03-21T10:36:11.382 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-03-21T10:36:11.382 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-03-21T10:36:11.382 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-03-21T10:36:11.382 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-03-21T10:36:11.382 INFO:teuthology.orchestra.run.smithi157.stdout: "req_renamesnap_latency": { 2024-03-21T10:36:11.382 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-03-21T10:36:11.382 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-03-21T10:36:11.382 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-03-21T10:36:11.382 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-03-21T10:36:11.382 INFO:teuthology.orchestra.run.smithi157.stdout: "req_snapdiff_latency": { 2024-03-21T10:36:11.382 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-03-21T10:36:11.382 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-03-21T10:36:11.382 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-03-21T10:36:11.383 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-03-21T10:36:11.383 INFO:teuthology.orchestra.run.smithi157.stdout: "req_rmdir_latency": { 2024-03-21T10:36:11.383 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-03-21T10:36:11.383 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-03-21T10:36:11.383 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-03-21T10:36:11.383 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-03-21T10:36:11.383 INFO:teuthology.orchestra.run.smithi157.stdout: "req_rmsnap_latency": { 2024-03-21T10:36:11.383 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-03-21T10:36:11.383 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-03-21T10:36:11.383 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-03-21T10:36:11.383 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-03-21T10:36:11.383 INFO:teuthology.orchestra.run.smithi157.stdout: "req_rmxattr_latency": { 2024-03-21T10:36:11.384 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-03-21T10:36:11.384 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-03-21T10:36:11.384 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-03-21T10:36:11.384 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-03-21T10:36:11.384 INFO:teuthology.orchestra.run.smithi157.stdout: "req_setattr_latency": { 2024-03-21T10:36:11.384 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-03-21T10:36:11.384 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-03-21T10:36:11.384 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-03-21T10:36:11.384 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-03-21T10:36:11.384 INFO:teuthology.orchestra.run.smithi157.stdout: "req_setdirlayout_latency": { 2024-03-21T10:36:11.384 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-03-21T10:36:11.384 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-03-21T10:36:11.385 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-03-21T10:36:11.385 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-03-21T10:36:11.385 INFO:teuthology.orchestra.run.smithi157.stdout: "req_setfilelock_latency": { 2024-03-21T10:36:11.385 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-03-21T10:36:11.385 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-03-21T10:36:11.385 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-03-21T10:36:11.385 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-03-21T10:36:11.385 INFO:teuthology.orchestra.run.smithi157.stdout: "req_setlayout_latency": { 2024-03-21T10:36:11.385 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-03-21T10:36:11.385 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-03-21T10:36:11.385 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-03-21T10:36:11.385 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-03-21T10:36:11.385 INFO:teuthology.orchestra.run.smithi157.stdout: "req_setxattr_latency": { 2024-03-21T10:36:11.385 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-03-21T10:36:11.385 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-03-21T10:36:11.386 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-03-21T10:36:11.386 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-03-21T10:36:11.386 INFO:teuthology.orchestra.run.smithi157.stdout: "req_symlink_latency": { 2024-03-21T10:36:11.386 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-03-21T10:36:11.386 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-03-21T10:36:11.386 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-03-21T10:36:11.386 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-03-21T10:36:11.386 INFO:teuthology.orchestra.run.smithi157.stdout: "req_unlink_latency": { 2024-03-21T10:36:11.386 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-03-21T10:36:11.386 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-03-21T10:36:11.386 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-03-21T10:36:11.386 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-03-21T10:36:11.386 INFO:teuthology.orchestra.run.smithi157.stdout: "cap_revoke_eviction": 0, 2024-03-21T10:36:11.387 INFO:teuthology.orchestra.run.smithi157.stdout: "cap_acquisition_throttle": 0, 2024-03-21T10:36:11.387 INFO:teuthology.orchestra.run.smithi157.stdout: "req_getvxattr_latency": { 2024-03-21T10:36:11.387 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-03-21T10:36:11.387 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-03-21T10:36:11.387 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-03-21T10:36:11.387 INFO:teuthology.orchestra.run.smithi157.stdout: } 2024-03-21T10:36:11.387 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-03-21T10:36:11.387 INFO:teuthology.orchestra.run.smithi157.stdout: "mds_sessions": { 2024-03-21T10:36:11.387 INFO:teuthology.orchestra.run.smithi157.stdout: "session_count": 2, 2024-03-21T10:36:11.387 INFO:teuthology.orchestra.run.smithi157.stdout: "session_add": 3, 2024-03-21T10:36:11.387 INFO:teuthology.orchestra.run.smithi157.stdout: "session_remove": 1, 2024-03-21T10:36:11.387 INFO:teuthology.orchestra.run.smithi157.stdout: "sessions_open": 2, 2024-03-21T10:36:11.387 INFO:teuthology.orchestra.run.smithi157.stdout: "sessions_stale": 0, 2024-03-21T10:36:11.387 INFO:teuthology.orchestra.run.smithi157.stdout: "total_load": 0, 2024-03-21T10:36:11.388 INFO:teuthology.orchestra.run.smithi157.stdout: "average_load": 0, 2024-03-21T10:36:11.388 INFO:teuthology.orchestra.run.smithi157.stdout: "avg_session_uptime": 2122, 2024-03-21T10:36:11.388 INFO:teuthology.orchestra.run.smithi157.stdout: "mdthresh_evicted": 0 2024-03-21T10:36:11.388 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-03-21T10:36:11.388 INFO:teuthology.orchestra.run.smithi157.stdout: "mempool": { 2024-03-21T10:36:11.388 INFO:teuthology.orchestra.run.smithi157.stdout: "bloom_filter_bytes": 0, 2024-03-21T10:36:11.388 INFO:teuthology.orchestra.run.smithi157.stdout: "bloom_filter_items": 0, 2024-03-21T10:36:11.388 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_alloc_bytes": 0, 2024-03-21T10:36:11.388 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_alloc_items": 0, 2024-03-21T10:36:11.388 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_cache_data_bytes": 0, 2024-03-21T10:36:11.388 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_cache_data_items": 0, 2024-03-21T10:36:11.388 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_cache_onode_bytes": 0, 2024-03-21T10:36:11.388 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_cache_onode_items": 0, 2024-03-21T10:36:11.388 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_cache_meta_bytes": 0, 2024-03-21T10:36:11.388 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_cache_meta_items": 0, 2024-03-21T10:36:11.389 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_cache_other_bytes": 0, 2024-03-21T10:36:11.389 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_cache_other_items": 0, 2024-03-21T10:36:11.389 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_cache_buffer_bytes": 0, 2024-03-21T10:36:11.389 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_cache_buffer_items": 0, 2024-03-21T10:36:11.389 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_extent_bytes": 0, 2024-03-21T10:36:11.389 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_extent_items": 0, 2024-03-21T10:36:11.389 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_blob_bytes": 0, 2024-03-21T10:36:11.389 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_blob_items": 0, 2024-03-21T10:36:11.389 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_shared_blob_bytes": 0, 2024-03-21T10:36:11.389 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_shared_blob_items": 0, 2024-03-21T10:36:11.389 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_inline_bl_bytes": 0, 2024-03-21T10:36:11.389 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_inline_bl_items": 0, 2024-03-21T10:36:11.389 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_fsck_bytes": 0, 2024-03-21T10:36:11.389 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_fsck_items": 0, 2024-03-21T10:36:11.389 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_txc_bytes": 0, 2024-03-21T10:36:11.390 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_txc_items": 0, 2024-03-21T10:36:11.390 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_writing_deferred_bytes": 0, 2024-03-21T10:36:11.390 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_writing_deferred_items": 0, 2024-03-21T10:36:11.390 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_writing_bytes": 0, 2024-03-21T10:36:11.390 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_writing_items": 0, 2024-03-21T10:36:11.390 INFO:teuthology.orchestra.run.smithi157.stdout: "bluefs_bytes": 0, 2024-03-21T10:36:11.390 INFO:teuthology.orchestra.run.smithi157.stdout: "bluefs_items": 0, 2024-03-21T10:36:11.390 INFO:teuthology.orchestra.run.smithi157.stdout: "bluefs_file_reader_bytes": 0, 2024-03-21T10:36:11.390 INFO:teuthology.orchestra.run.smithi157.stdout: "bluefs_file_reader_items": 0, 2024-03-21T10:36:11.390 INFO:teuthology.orchestra.run.smithi157.stdout: "bluefs_file_writer_bytes": 0, 2024-03-21T10:36:11.390 INFO:teuthology.orchestra.run.smithi157.stdout: "bluefs_file_writer_items": 0, 2024-03-21T10:36:11.390 INFO:teuthology.orchestra.run.smithi157.stdout: "buffer_anon_bytes": 198197, 2024-03-21T10:36:11.390 INFO:teuthology.orchestra.run.smithi157.stdout: "buffer_anon_items": 56, 2024-03-21T10:36:11.390 INFO:teuthology.orchestra.run.smithi157.stdout: "buffer_meta_bytes": 0, 2024-03-21T10:36:11.390 INFO:teuthology.orchestra.run.smithi157.stdout: "buffer_meta_items": 0, 2024-03-21T10:36:11.391 INFO:teuthology.orchestra.run.smithi157.stdout: "osd_bytes": 0, 2024-03-21T10:36:11.391 INFO:teuthology.orchestra.run.smithi157.stdout: "osd_items": 0, 2024-03-21T10:36:11.391 INFO:teuthology.orchestra.run.smithi157.stdout: "osd_mapbl_bytes": 0, 2024-03-21T10:36:11.391 INFO:teuthology.orchestra.run.smithi157.stdout: "osd_mapbl_items": 0, 2024-03-21T10:36:11.391 INFO:teuthology.orchestra.run.smithi157.stdout: "osd_pglog_bytes": 0, 2024-03-21T10:36:11.391 INFO:teuthology.orchestra.run.smithi157.stdout: "osd_pglog_items": 0, 2024-03-21T10:36:11.391 INFO:teuthology.orchestra.run.smithi157.stdout: "osdmap_bytes": 9712, 2024-03-21T10:36:11.391 INFO:teuthology.orchestra.run.smithi157.stdout: "osdmap_items": 187, 2024-03-21T10:36:11.391 INFO:teuthology.orchestra.run.smithi157.stdout: "osdmap_mapping_bytes": 0, 2024-03-21T10:36:11.391 INFO:teuthology.orchestra.run.smithi157.stdout: "osdmap_mapping_items": 0, 2024-03-21T10:36:11.391 INFO:teuthology.orchestra.run.smithi157.stdout: "pgmap_bytes": 0, 2024-03-21T10:36:11.391 INFO:teuthology.orchestra.run.smithi157.stdout: "pgmap_items": 0, 2024-03-21T10:36:11.391 INFO:teuthology.orchestra.run.smithi157.stdout: "mds_co_bytes": 8320, 2024-03-21T10:36:11.391 INFO:teuthology.orchestra.run.smithi157.stdout: "mds_co_items": 13, 2024-03-21T10:36:11.392 INFO:teuthology.orchestra.run.smithi157.stdout: "unittest_1_bytes": 0, 2024-03-21T10:36:11.392 INFO:teuthology.orchestra.run.smithi157.stdout: "unittest_1_items": 0, 2024-03-21T10:36:11.392 INFO:teuthology.orchestra.run.smithi157.stdout: "unittest_2_bytes": 0, 2024-03-21T10:36:11.392 INFO:teuthology.orchestra.run.smithi157.stdout: "unittest_2_items": 0 2024-03-21T10:36:11.392 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-03-21T10:36:11.392 INFO:teuthology.orchestra.run.smithi157.stdout: "objecter": { 2024-03-21T10:36:11.392 INFO:teuthology.orchestra.run.smithi157.stdout: "op_active": 0, 2024-03-21T10:36:11.392 INFO:teuthology.orchestra.run.smithi157.stdout: "op_laggy": 0, 2024-03-21T10:36:11.392 INFO:teuthology.orchestra.run.smithi157.stdout: "op_send": 2384, 2024-03-21T10:36:11.392 INFO:teuthology.orchestra.run.smithi157.stdout: "op_send_bytes": 21, 2024-03-21T10:36:11.392 INFO:teuthology.orchestra.run.smithi157.stdout: "op_resend": 3, 2024-03-21T10:36:11.392 INFO:teuthology.orchestra.run.smithi157.stdout: "op_reply": 2384, 2024-03-21T10:36:11.392 INFO:teuthology.orchestra.run.smithi157.stdout: "op_latency": { 2024-03-21T10:36:11.392 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 2384, 2024-03-21T10:36:11.392 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 2.674951350, 2024-03-21T10:36:11.393 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.001122043 2024-03-21T10:36:11.393 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-03-21T10:36:11.393 INFO:teuthology.orchestra.run.smithi157.stdout: "op_inflight": 0, 2024-03-21T10:36:11.393 INFO:teuthology.orchestra.run.smithi157.stdout: "oplen_avg": { 2024-03-21T10:36:11.393 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 2384, 2024-03-21T10:36:11.393 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 2385 2024-03-21T10:36:11.393 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-03-21T10:36:11.393 INFO:teuthology.orchestra.run.smithi157.stdout: "op": 2384, 2024-03-21T10:36:11.393 INFO:teuthology.orchestra.run.smithi157.stdout: "op_r": 2384, 2024-03-21T10:36:11.393 INFO:teuthology.orchestra.run.smithi157.stdout: "op_w": 0, 2024-03-21T10:36:11.393 INFO:teuthology.orchestra.run.smithi157.stdout: "op_rmw": 0, 2024-03-21T10:36:11.393 INFO:teuthology.orchestra.run.smithi157.stdout: "op_pg": 0, 2024-03-21T10:36:11.393 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_stat": 1554, 2024-03-21T10:36:11.393 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_create": 0, 2024-03-21T10:36:11.393 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_read": 828, 2024-03-21T10:36:11.393 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_write": 0, 2024-03-21T10:36:11.394 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_writefull": 0, 2024-03-21T10:36:11.394 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_writesame": 0, 2024-03-21T10:36:11.394 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_append": 0, 2024-03-21T10:36:11.394 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_zero": 0, 2024-03-21T10:36:11.394 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_truncate": 0, 2024-03-21T10:36:11.394 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_delete": 0, 2024-03-21T10:36:11.394 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_mapext": 0, 2024-03-21T10:36:11.394 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_sparse_read": 0, 2024-03-21T10:36:11.394 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_clonerange": 0, 2024-03-21T10:36:11.394 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_getxattr": 1, 2024-03-21T10:36:11.394 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_setxattr": 0, 2024-03-21T10:36:11.394 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_cmpxattr": 0, 2024-03-21T10:36:11.394 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_rmxattr": 0, 2024-03-21T10:36:11.394 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_resetxattrs": 0, 2024-03-21T10:36:11.394 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_call": 0, 2024-03-21T10:36:11.395 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_watch": 0, 2024-03-21T10:36:11.395 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_notify": 0, 2024-03-21T10:36:11.395 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_src_cmpxattr": 0, 2024-03-21T10:36:11.395 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_pgls": 0, 2024-03-21T10:36:11.395 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_pgls_filter": 0, 2024-03-21T10:36:11.395 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_other": 0, 2024-03-21T10:36:11.395 INFO:teuthology.orchestra.run.smithi157.stdout: "linger_active": 0, 2024-03-21T10:36:11.395 INFO:teuthology.orchestra.run.smithi157.stdout: "linger_send": 0, 2024-03-21T10:36:11.395 INFO:teuthology.orchestra.run.smithi157.stdout: "linger_resend": 0, 2024-03-21T10:36:11.395 INFO:teuthology.orchestra.run.smithi157.stdout: "linger_ping": 0, 2024-03-21T10:36:11.395 INFO:teuthology.orchestra.run.smithi157.stdout: "poolop_active": 0, 2024-03-21T10:36:11.395 INFO:teuthology.orchestra.run.smithi157.stdout: "poolop_send": 0, 2024-03-21T10:36:11.395 INFO:teuthology.orchestra.run.smithi157.stdout: "poolop_resend": 0, 2024-03-21T10:36:11.395 INFO:teuthology.orchestra.run.smithi157.stdout: "poolstat_active": 0, 2024-03-21T10:36:11.396 INFO:teuthology.orchestra.run.smithi157.stdout: "poolstat_send": 0, 2024-03-21T10:36:11.396 INFO:teuthology.orchestra.run.smithi157.stdout: "poolstat_resend": 0, 2024-03-21T10:36:11.396 INFO:teuthology.orchestra.run.smithi157.stdout: "statfs_active": 0, 2024-03-21T10:36:11.396 INFO:teuthology.orchestra.run.smithi157.stdout: "statfs_send": 0, 2024-03-21T10:36:11.396 INFO:teuthology.orchestra.run.smithi157.stdout: "statfs_resend": 0, 2024-03-21T10:36:11.396 INFO:teuthology.orchestra.run.smithi157.stdout: "command_active": 0, 2024-03-21T10:36:11.396 INFO:teuthology.orchestra.run.smithi157.stdout: "command_send": 0, 2024-03-21T10:36:11.396 INFO:teuthology.orchestra.run.smithi157.stdout: "command_resend": 0, 2024-03-21T10:36:11.396 INFO:teuthology.orchestra.run.smithi157.stdout: "map_epoch": 106, 2024-03-21T10:36:11.396 INFO:teuthology.orchestra.run.smithi157.stdout: "map_full": 0, 2024-03-21T10:36:11.396 INFO:teuthology.orchestra.run.smithi157.stdout: "map_inc": 13, 2024-03-21T10:36:11.396 INFO:teuthology.orchestra.run.smithi157.stdout: "osd_sessions": 6, 2024-03-21T10:36:11.396 INFO:teuthology.orchestra.run.smithi157.stdout: "osd_session_open": 6, 2024-03-21T10:36:11.396 INFO:teuthology.orchestra.run.smithi157.stdout: "osd_session_close": 0, 2024-03-21T10:36:11.396 INFO:teuthology.orchestra.run.smithi157.stdout: "osd_laggy": 0, 2024-03-21T10:36:11.397 INFO:teuthology.orchestra.run.smithi157.stdout: "omap_wr": 0, 2024-03-21T10:36:11.397 INFO:teuthology.orchestra.run.smithi157.stdout: "omap_rd": 2, 2024-03-21T10:36:11.397 INFO:teuthology.orchestra.run.smithi157.stdout: "omap_del": 0 2024-03-21T10:36:11.397 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-03-21T10:36:11.397 INFO:teuthology.orchestra.run.smithi157.stdout: "oft": { 2024-03-21T10:36:11.397 INFO:teuthology.orchestra.run.smithi157.stdout: "omap_total_objs": 0, 2024-03-21T10:36:11.397 INFO:teuthology.orchestra.run.smithi157.stdout: "omap_total_kv_pairs": 0, 2024-03-21T10:36:11.397 INFO:teuthology.orchestra.run.smithi157.stdout: "omap_total_updates": 0, 2024-03-21T10:36:11.397 INFO:teuthology.orchestra.run.smithi157.stdout: "omap_total_removes": 0 2024-03-21T10:36:11.397 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-03-21T10:36:11.397 INFO:teuthology.orchestra.run.smithi157.stdout: "osd-slow-ops": { 2024-03-21T10:36:11.397 INFO:teuthology.orchestra.run.smithi157.stdout: "slow_ops_count": 0 2024-03-21T10:36:11.397 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-03-21T10:36:11.397 INFO:teuthology.orchestra.run.smithi157.stdout: "purge_queue": { 2024-03-21T10:36:11.397 INFO:teuthology.orchestra.run.smithi157.stdout: "pq_executing_ops": 0, 2024-03-21T10:36:11.398 INFO:teuthology.orchestra.run.smithi157.stdout: "pq_executing_ops_high_water": 0, 2024-03-21T10:36:11.398 INFO:teuthology.orchestra.run.smithi157.stdout: "pq_executing": 0, 2024-03-21T10:36:11.398 INFO:teuthology.orchestra.run.smithi157.stdout: "pq_executing_high_water": 0, 2024-03-21T10:36:11.398 INFO:teuthology.orchestra.run.smithi157.stdout: "pq_executed_ops": 0, 2024-03-21T10:36:11.398 INFO:teuthology.orchestra.run.smithi157.stdout: "pq_executed": 0, 2024-03-21T10:36:11.398 INFO:teuthology.orchestra.run.smithi157.stdout: "pq_item_in_journal": 0 2024-03-21T10:36:11.398 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-03-21T10:36:11.398 INFO:teuthology.orchestra.run.smithi157.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-03-21T10:36:11.398 INFO:teuthology.orchestra.run.smithi157.stdout: "val": 0, 2024-03-21T10:36:11.398 INFO:teuthology.orchestra.run.smithi157.stdout: "max": 104857600, 2024-03-21T10:36:11.398 INFO:teuthology.orchestra.run.smithi157.stdout: "get_started": 0, 2024-03-21T10:36:11.398 INFO:teuthology.orchestra.run.smithi157.stdout: "get": 2658, 2024-03-21T10:36:11.398 INFO:teuthology.orchestra.run.smithi157.stdout: "get_sum": 2211803, 2024-03-21T10:36:11.398 INFO:teuthology.orchestra.run.smithi157.stdout: "get_or_fail_fail": 0, 2024-03-21T10:36:11.399 INFO:teuthology.orchestra.run.smithi157.stdout: "get_or_fail_success": 2658, 2024-03-21T10:36:11.399 INFO:teuthology.orchestra.run.smithi157.stdout: "take": 0, 2024-03-21T10:36:11.399 INFO:teuthology.orchestra.run.smithi157.stdout: "take_sum": 0, 2024-03-21T10:36:11.399 INFO:teuthology.orchestra.run.smithi157.stdout: "put": 2658, 2024-03-21T10:36:11.399 INFO:teuthology.orchestra.run.smithi157.stdout: "put_sum": 2211803, 2024-03-21T10:36:11.399 INFO:teuthology.orchestra.run.smithi157.stdout: "wait": { 2024-03-21T10:36:11.399 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-03-21T10:36:11.399 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-03-21T10:36:11.399 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-03-21T10:36:11.399 INFO:teuthology.orchestra.run.smithi157.stdout: } 2024-03-21T10:36:11.399 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-03-21T10:36:11.399 INFO:teuthology.orchestra.run.smithi157.stdout: "throttle-objecter_bytes": { 2024-03-21T10:36:11.399 INFO:teuthology.orchestra.run.smithi157.stdout: "val": 0, 2024-03-21T10:36:11.399 INFO:teuthology.orchestra.run.smithi157.stdout: "max": 104857600, 2024-03-21T10:36:11.399 INFO:teuthology.orchestra.run.smithi157.stdout: "get_started": 0, 2024-03-21T10:36:11.400 INFO:teuthology.orchestra.run.smithi157.stdout: "get": 0, 2024-03-21T10:36:11.400 INFO:teuthology.orchestra.run.smithi157.stdout: "get_sum": 0, 2024-03-21T10:36:11.400 INFO:teuthology.orchestra.run.smithi157.stdout: "get_or_fail_fail": 0, 2024-03-21T10:36:11.400 INFO:teuthology.orchestra.run.smithi157.stdout: "get_or_fail_success": 0, 2024-03-21T10:36:11.400 INFO:teuthology.orchestra.run.smithi157.stdout: "take": 2384, 2024-03-21T10:36:11.400 INFO:teuthology.orchestra.run.smithi157.stdout: "take_sum": 2009, 2024-03-21T10:36:11.400 INFO:teuthology.orchestra.run.smithi157.stdout: "put": 2384, 2024-03-21T10:36:11.400 INFO:teuthology.orchestra.run.smithi157.stdout: "put_sum": 2009, 2024-03-21T10:36:11.400 INFO:teuthology.orchestra.run.smithi157.stdout: "wait": { 2024-03-21T10:36:11.400 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-03-21T10:36:11.400 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-03-21T10:36:11.400 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-03-21T10:36:11.400 INFO:teuthology.orchestra.run.smithi157.stdout: } 2024-03-21T10:36:11.400 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-03-21T10:36:11.400 INFO:teuthology.orchestra.run.smithi157.stdout: "throttle-objecter_ops": { 2024-03-21T10:36:11.401 INFO:teuthology.orchestra.run.smithi157.stdout: "val": 0, 2024-03-21T10:36:11.401 INFO:teuthology.orchestra.run.smithi157.stdout: "max": 1024, 2024-03-21T10:36:11.401 INFO:teuthology.orchestra.run.smithi157.stdout: "get_started": 0, 2024-03-21T10:36:11.401 INFO:teuthology.orchestra.run.smithi157.stdout: "get": 0, 2024-03-21T10:36:11.401 INFO:teuthology.orchestra.run.smithi157.stdout: "get_sum": 0, 2024-03-21T10:36:11.401 INFO:teuthology.orchestra.run.smithi157.stdout: "get_or_fail_fail": 0, 2024-03-21T10:36:11.401 INFO:teuthology.orchestra.run.smithi157.stdout: "get_or_fail_success": 0, 2024-03-21T10:36:11.401 INFO:teuthology.orchestra.run.smithi157.stdout: "take": 2384, 2024-03-21T10:36:11.401 INFO:teuthology.orchestra.run.smithi157.stdout: "take_sum": 2384, 2024-03-21T10:36:11.401 INFO:teuthology.orchestra.run.smithi157.stdout: "put": 2384, 2024-03-21T10:36:11.401 INFO:teuthology.orchestra.run.smithi157.stdout: "put_sum": 2384, 2024-03-21T10:36:11.401 INFO:teuthology.orchestra.run.smithi157.stdout: "wait": { 2024-03-21T10:36:11.401 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-03-21T10:36:11.401 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-03-21T10:36:11.401 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-03-21T10:36:11.402 INFO:teuthology.orchestra.run.smithi157.stdout: } 2024-03-21T10:36:11.402 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-03-21T10:36:11.402 INFO:teuthology.orchestra.run.smithi157.stdout: "throttle-write_buf_throttle": { 2024-03-21T10:36:11.402 INFO:teuthology.orchestra.run.smithi157.stdout: "val": 0, 2024-03-21T10:36:11.402 INFO:teuthology.orchestra.run.smithi157.stdout: "max": 3758096384, 2024-03-21T10:36:11.402 INFO:teuthology.orchestra.run.smithi157.stdout: "get_started": 0, 2024-03-21T10:36:11.402 INFO:teuthology.orchestra.run.smithi157.stdout: "get": 0, 2024-03-21T10:36:11.402 INFO:teuthology.orchestra.run.smithi157.stdout: "get_sum": 0, 2024-03-21T10:36:11.402 INFO:teuthology.orchestra.run.smithi157.stdout: "get_or_fail_fail": 0, 2024-03-21T10:36:11.402 INFO:teuthology.orchestra.run.smithi157.stdout: "get_or_fail_success": 0, 2024-03-21T10:36:11.402 INFO:teuthology.orchestra.run.smithi157.stdout: "take": 0, 2024-03-21T10:36:11.402 INFO:teuthology.orchestra.run.smithi157.stdout: "take_sum": 0, 2024-03-21T10:36:11.402 INFO:teuthology.orchestra.run.smithi157.stdout: "put": 0, 2024-03-21T10:36:11.402 INFO:teuthology.orchestra.run.smithi157.stdout: "put_sum": 0, 2024-03-21T10:36:11.402 INFO:teuthology.orchestra.run.smithi157.stdout: "wait": { 2024-03-21T10:36:11.403 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-03-21T10:36:11.403 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-03-21T10:36:11.403 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-03-21T10:36:11.403 INFO:teuthology.orchestra.run.smithi157.stdout: } 2024-03-21T10:36:11.403 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-03-21T10:36:11.403 INFO:teuthology.orchestra.run.smithi157.stdout: "throttle-write_buf_throttle-0x56528181a780": { 2024-03-21T10:36:11.403 INFO:teuthology.orchestra.run.smithi157.stdout: "val": 0, 2024-03-21T10:36:11.403 INFO:teuthology.orchestra.run.smithi157.stdout: "max": 3758096384, 2024-03-21T10:36:11.403 INFO:teuthology.orchestra.run.smithi157.stdout: "get_started": 0, 2024-03-21T10:36:11.403 INFO:teuthology.orchestra.run.smithi157.stdout: "get": 0, 2024-03-21T10:36:11.403 INFO:teuthology.orchestra.run.smithi157.stdout: "get_sum": 0, 2024-03-21T10:36:11.403 INFO:teuthology.orchestra.run.smithi157.stdout: "get_or_fail_fail": 0, 2024-03-21T10:36:11.403 INFO:teuthology.orchestra.run.smithi157.stdout: "get_or_fail_success": 0, 2024-03-21T10:36:11.403 INFO:teuthology.orchestra.run.smithi157.stdout: "take": 0, 2024-03-21T10:36:11.403 INFO:teuthology.orchestra.run.smithi157.stdout: "take_sum": 0, 2024-03-21T10:36:11.404 INFO:teuthology.orchestra.run.smithi157.stdout: "put": 0, 2024-03-21T10:36:11.404 INFO:teuthology.orchestra.run.smithi157.stdout: "put_sum": 0, 2024-03-21T10:36:11.404 INFO:teuthology.orchestra.run.smithi157.stdout: "wait": { 2024-03-21T10:36:11.404 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-03-21T10:36:11.404 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-03-21T10:36:11.404 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-03-21T10:36:11.404 INFO:teuthology.orchestra.run.smithi157.stdout: } 2024-03-21T10:36:11.404 INFO:teuthology.orchestra.run.smithi157.stdout: } 2024-03-21T10:36:11.404 INFO:teuthology.orchestra.run.smithi157.stdout:} 2024-03-21T10:36:11.961 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:36:11 smithi047 ceph-mon[31547]: pgmap v338: 129 pgs: 129 active+clean; 2.5 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 53 KiB/s wr, 30 op/s 2024-03-21T10:36:11.976 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:36:11 smithi100 ceph-mon[35193]: pgmap v338: 129 pgs: 129 active+clean; 2.5 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 53 KiB/s wr, 30 op/s 2024-03-21T10:36:11.986 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:36:11 smithi157 ceph-mon[35581]: pgmap v338: 129 pgs: 129 active+clean; 2.5 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 53 KiB/s wr, 30 op/s 2024-03-21T10:36:12.274 INFO:tasks.check_counter:Daemon mds.f mds.exported=0 2024-03-21T10:36:12.274 INFO:tasks.check_counter:Daemon mds.f mds.imported=0 2024-03-21T10:36:12.274 INFO:tasks.check_counter:Daemon mds.f mds.root_rsnaps=0 2024-03-21T10:36:12.274 INFO:tasks.check_counter:Daemon mds.f mds_server.req_mksnap_latency.avgcount=0 2024-03-21T10:36:12.274 INFO:tasks.check_counter:Daemon mds.f mds_server.req_rmsnap_latency.avgcount=0 2024-03-21T10:36:12.274 DEBUG:tasks.check_counter:Getting stats from i 2024-03-21T10:36:12.274 DEBUG:teuthology.orchestra.run.smithi157:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2edfee68ad7243bed422a049b6f5f58619300f11 shell --fsid 3056d51a-e76b-11ee-95cd-87774f69a715 -- ceph daemon mds.i perf dump 2024-03-21T10:36:12.728 INFO:teuthology.orchestra.run.smithi157.stderr:Inferring config /var/lib/ceph/3056d51a-e76b-11ee-95cd-87774f69a715/mon.c/config 2024-03-21T10:36:13.905 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:36:13 smithi157 ceph-mon[35581]: pgmap v339: 129 pgs: 129 active+clean; 2.5 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 14 KiB/s wr, 12 op/s 2024-03-21T10:36:13.960 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:36:13 smithi047 ceph-mon[31547]: pgmap v339: 129 pgs: 129 active+clean; 2.5 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 14 KiB/s wr, 12 op/s 2024-03-21T10:36:13.976 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:36:13 smithi100 ceph-mon[35193]: pgmap v339: 129 pgs: 129 active+clean; 2.5 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 14 KiB/s wr, 12 op/s 2024-03-21T10:36:14.901 INFO:teuthology.orchestra.run.smithi157.stdout:{ 2024-03-21T10:36:14.901 INFO:teuthology.orchestra.run.smithi157.stdout: "AsyncMessenger::Worker-0": { 2024-03-21T10:36:14.901 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_recv_messages": 1484, 2024-03-21T10:36:14.901 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_send_messages": 1482, 2024-03-21T10:36:14.901 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_recv_bytes": 1979726, 2024-03-21T10:36:14.901 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_send_bytes": 514102, 2024-03-21T10:36:14.901 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_created_connections": 5, 2024-03-21T10:36:14.901 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_active_connections": 1, 2024-03-21T10:36:14.901 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_running_total_time": 0.309575282, 2024-03-21T10:36:14.901 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_running_send_time": 0.106650501, 2024-03-21T10:36:14.901 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_running_recv_time": 0.104358467, 2024-03-21T10:36:14.902 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_running_fast_dispatch_time": 0.078035660, 2024-03-21T10:36:14.902 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_send_messages_queue_lat": { 2024-03-21T10:36:14.902 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 1481, 2024-03-21T10:36:14.902 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.122628871, 2024-03-21T10:36:14.902 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000082801 2024-03-21T10:36:14.902 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-03-21T10:36:14.902 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_handle_ack_lat": { 2024-03-21T10:36:14.902 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-03-21T10:36:14.902 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-03-21T10:36:14.902 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-03-21T10:36:14.902 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-03-21T10:36:14.902 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_recv_encrypted_bytes": 351248, 2024-03-21T10:36:14.902 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_send_encrypted_bytes": 99744 2024-03-21T10:36:14.902 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-03-21T10:36:14.902 INFO:teuthology.orchestra.run.smithi157.stdout: "AsyncMessenger::Worker-1": { 2024-03-21T10:36:14.903 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_recv_messages": 9, 2024-03-21T10:36:14.903 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_send_messages": 129, 2024-03-21T10:36:14.903 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_recv_bytes": 6394, 2024-03-21T10:36:14.903 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_send_bytes": 270346, 2024-03-21T10:36:14.903 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_created_connections": 5, 2024-03-21T10:36:14.903 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_active_connections": 4, 2024-03-21T10:36:14.903 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_running_total_time": 0.019768117, 2024-03-21T10:36:14.903 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_running_send_time": 0.011477307, 2024-03-21T10:36:14.903 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_running_recv_time": 0.005457511, 2024-03-21T10:36:14.903 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_running_fast_dispatch_time": 0.000167676, 2024-03-21T10:36:14.903 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_send_messages_queue_lat": { 2024-03-21T10:36:14.903 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 128, 2024-03-21T10:36:14.903 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.023359935, 2024-03-21T10:36:14.903 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000182499 2024-03-21T10:36:14.903 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-03-21T10:36:14.904 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_handle_ack_lat": { 2024-03-21T10:36:14.904 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-03-21T10:36:14.904 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-03-21T10:36:14.904 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-03-21T10:36:14.904 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-03-21T10:36:14.904 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_recv_encrypted_bytes": 3104, 2024-03-21T10:36:14.904 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_send_encrypted_bytes": 269008 2024-03-21T10:36:14.904 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-03-21T10:36:14.904 INFO:teuthology.orchestra.run.smithi157.stdout: "AsyncMessenger::Worker-2": { 2024-03-21T10:36:14.904 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_recv_messages": 599, 2024-03-21T10:36:14.904 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_send_messages": 601, 2024-03-21T10:36:14.904 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_recv_bytes": 147543, 2024-03-21T10:36:14.904 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_send_bytes": 190489, 2024-03-21T10:36:14.905 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_created_connections": 4, 2024-03-21T10:36:14.905 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_active_connections": 3, 2024-03-21T10:36:14.905 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_running_total_time": 0.140525895, 2024-03-21T10:36:14.905 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_running_send_time": 0.056899053, 2024-03-21T10:36:14.905 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_running_recv_time": 0.031137925, 2024-03-21T10:36:14.905 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_running_fast_dispatch_time": 0.038838503, 2024-03-21T10:36:14.905 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_send_messages_queue_lat": { 2024-03-21T10:36:14.905 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 599, 2024-03-21T10:36:14.905 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.059043790, 2024-03-21T10:36:14.905 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000098570 2024-03-21T10:36:14.905 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-03-21T10:36:14.905 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_handle_ack_lat": { 2024-03-21T10:36:14.905 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-03-21T10:36:14.905 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-03-21T10:36:14.906 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-03-21T10:36:14.906 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-03-21T10:36:14.906 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_recv_encrypted_bytes": 0, 2024-03-21T10:36:14.906 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_send_encrypted_bytes": 0 2024-03-21T10:36:14.906 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-03-21T10:36:14.906 INFO:teuthology.orchestra.run.smithi157.stdout: "cct": { 2024-03-21T10:36:14.906 INFO:teuthology.orchestra.run.smithi157.stdout: "total_workers": 1, 2024-03-21T10:36:14.906 INFO:teuthology.orchestra.run.smithi157.stdout: "unhealthy_workers": 0 2024-03-21T10:36:14.906 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-03-21T10:36:14.906 INFO:teuthology.orchestra.run.smithi157.stdout: "finisher-MDSRank": { 2024-03-21T10:36:14.906 INFO:teuthology.orchestra.run.smithi157.stdout: "queue_len": 0, 2024-03-21T10:36:14.906 INFO:teuthology.orchestra.run.smithi157.stdout: "complete_latency": { 2024-03-21T10:36:14.906 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 3197, 2024-03-21T10:36:14.906 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.164009242, 2024-03-21T10:36:14.907 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000051300 2024-03-21T10:36:14.907 INFO:teuthology.orchestra.run.smithi157.stdout: } 2024-03-21T10:36:14.907 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-03-21T10:36:14.907 INFO:teuthology.orchestra.run.smithi157.stdout: "finisher-PurgeQueue": { 2024-03-21T10:36:14.907 INFO:teuthology.orchestra.run.smithi157.stdout: "queue_len": 0, 2024-03-21T10:36:14.907 INFO:teuthology.orchestra.run.smithi157.stdout: "complete_latency": { 2024-03-21T10:36:14.907 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-03-21T10:36:14.907 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-03-21T10:36:14.907 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-03-21T10:36:14.907 INFO:teuthology.orchestra.run.smithi157.stdout: } 2024-03-21T10:36:14.907 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-03-21T10:36:14.907 INFO:teuthology.orchestra.run.smithi157.stdout: "mds": { 2024-03-21T10:36:14.907 INFO:teuthology.orchestra.run.smithi157.stdout: "request": 0, 2024-03-21T10:36:14.907 INFO:teuthology.orchestra.run.smithi157.stdout: "reply": 0, 2024-03-21T10:36:14.907 INFO:teuthology.orchestra.run.smithi157.stdout: "reply_latency": { 2024-03-21T10:36:14.908 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-03-21T10:36:14.908 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-03-21T10:36:14.908 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-03-21T10:36:14.908 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-03-21T10:36:14.908 INFO:teuthology.orchestra.run.smithi157.stdout: "slow_reply": 0, 2024-03-21T10:36:14.908 INFO:teuthology.orchestra.run.smithi157.stdout: "forward": 0, 2024-03-21T10:36:14.908 INFO:teuthology.orchestra.run.smithi157.stdout: "dir_fetch_complete": 0, 2024-03-21T10:36:14.908 INFO:teuthology.orchestra.run.smithi157.stdout: "dir_fetch_keys": 0, 2024-03-21T10:36:14.908 INFO:teuthology.orchestra.run.smithi157.stdout: "dir_commit": 0, 2024-03-21T10:36:14.908 INFO:teuthology.orchestra.run.smithi157.stdout: "dir_split": 0, 2024-03-21T10:36:14.908 INFO:teuthology.orchestra.run.smithi157.stdout: "dir_merge": 0, 2024-03-21T10:36:14.908 INFO:teuthology.orchestra.run.smithi157.stdout: "inodes": 24, 2024-03-21T10:36:14.908 INFO:teuthology.orchestra.run.smithi157.stdout: "inodes_top": 2, 2024-03-21T10:36:14.908 INFO:teuthology.orchestra.run.smithi157.stdout: "inodes_bottom": 22, 2024-03-21T10:36:14.908 INFO:teuthology.orchestra.run.smithi157.stdout: "inodes_pin_tail": 0, 2024-03-21T10:36:14.908 INFO:teuthology.orchestra.run.smithi157.stdout: "inodes_pinned": 22, 2024-03-21T10:36:14.909 INFO:teuthology.orchestra.run.smithi157.stdout: "inodes_expired": 0, 2024-03-21T10:36:14.909 INFO:teuthology.orchestra.run.smithi157.stdout: "inodes_with_caps": 0, 2024-03-21T10:36:14.909 INFO:teuthology.orchestra.run.smithi157.stdout: "caps": 0, 2024-03-21T10:36:14.909 INFO:teuthology.orchestra.run.smithi157.stdout: "subtrees": 2, 2024-03-21T10:36:14.909 INFO:teuthology.orchestra.run.smithi157.stdout: "traverse": 0, 2024-03-21T10:36:14.909 INFO:teuthology.orchestra.run.smithi157.stdout: "traverse_hit": 0, 2024-03-21T10:36:14.909 INFO:teuthology.orchestra.run.smithi157.stdout: "traverse_forward": 0, 2024-03-21T10:36:14.909 INFO:teuthology.orchestra.run.smithi157.stdout: "traverse_discover": 0, 2024-03-21T10:36:14.909 INFO:teuthology.orchestra.run.smithi157.stdout: "traverse_dir_fetch": 0, 2024-03-21T10:36:14.909 INFO:teuthology.orchestra.run.smithi157.stdout: "traverse_remote_ino": 0, 2024-03-21T10:36:14.909 INFO:teuthology.orchestra.run.smithi157.stdout: "traverse_lock": 0, 2024-03-21T10:36:14.909 INFO:teuthology.orchestra.run.smithi157.stdout: "load_cent": 0, 2024-03-21T10:36:14.909 INFO:teuthology.orchestra.run.smithi157.stdout: "q": 0, 2024-03-21T10:36:14.909 INFO:teuthology.orchestra.run.smithi157.stdout: "exported": 0, 2024-03-21T10:36:14.909 INFO:teuthology.orchestra.run.smithi157.stdout: "exported_inodes": 0, 2024-03-21T10:36:14.910 INFO:teuthology.orchestra.run.smithi157.stdout: "imported": 0, 2024-03-21T10:36:14.910 INFO:teuthology.orchestra.run.smithi157.stdout: "imported_inodes": 0, 2024-03-21T10:36:14.910 INFO:teuthology.orchestra.run.smithi157.stdout: "openino_dir_fetch": 0, 2024-03-21T10:36:14.910 INFO:teuthology.orchestra.run.smithi157.stdout: "openino_backtrace_fetch": 0, 2024-03-21T10:36:14.910 INFO:teuthology.orchestra.run.smithi157.stdout: "openino_peer_discover": 0, 2024-03-21T10:36:14.910 INFO:teuthology.orchestra.run.smithi157.stdout: "root_rfiles": 4, 2024-03-21T10:36:14.910 INFO:teuthology.orchestra.run.smithi157.stdout: "root_rbytes": 234, 2024-03-21T10:36:14.910 INFO:teuthology.orchestra.run.smithi157.stdout: "root_rsnaps": 6, 2024-03-21T10:36:14.910 INFO:teuthology.orchestra.run.smithi157.stdout: "scrub_backtrace_fetch": 0, 2024-03-21T10:36:14.910 INFO:teuthology.orchestra.run.smithi157.stdout: "scrub_set_tag": 0, 2024-03-21T10:36:14.910 INFO:teuthology.orchestra.run.smithi157.stdout: "scrub_backtrace_repaired": 0, 2024-03-21T10:36:14.910 INFO:teuthology.orchestra.run.smithi157.stdout: "scrub_inotable_repaired": 0, 2024-03-21T10:36:14.910 INFO:teuthology.orchestra.run.smithi157.stdout: "scrub_dir_inodes": 0, 2024-03-21T10:36:14.910 INFO:teuthology.orchestra.run.smithi157.stdout: "scrub_dir_base_inodes": 0, 2024-03-21T10:36:14.911 INFO:teuthology.orchestra.run.smithi157.stdout: "scrub_dirfrag_rstats": 0, 2024-03-21T10:36:14.911 INFO:teuthology.orchestra.run.smithi157.stdout: "scrub_file_inodes": 0, 2024-03-21T10:36:14.911 INFO:teuthology.orchestra.run.smithi157.stdout: "handle_inode_file_caps": 0, 2024-03-21T10:36:14.911 INFO:teuthology.orchestra.run.smithi157.stdout: "ceph_cap_op_revoke": 0, 2024-03-21T10:36:14.911 INFO:teuthology.orchestra.run.smithi157.stdout: "ceph_cap_op_grant": 0, 2024-03-21T10:36:14.911 INFO:teuthology.orchestra.run.smithi157.stdout: "ceph_cap_op_trunc": 0, 2024-03-21T10:36:14.911 INFO:teuthology.orchestra.run.smithi157.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-03-21T10:36:14.911 INFO:teuthology.orchestra.run.smithi157.stdout: "ceph_cap_op_flush_ack": 0, 2024-03-21T10:36:14.911 INFO:teuthology.orchestra.run.smithi157.stdout: "handle_client_caps": 0, 2024-03-21T10:36:14.911 INFO:teuthology.orchestra.run.smithi157.stdout: "handle_client_caps_dirty": 0, 2024-03-21T10:36:14.911 INFO:teuthology.orchestra.run.smithi157.stdout: "handle_client_cap_release": 0, 2024-03-21T10:36:14.911 INFO:teuthology.orchestra.run.smithi157.stdout: "process_request_cap_release": 0 2024-03-21T10:36:14.911 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-03-21T10:36:14.911 INFO:teuthology.orchestra.run.smithi157.stdout: "mds_cache": { 2024-03-21T10:36:14.911 INFO:teuthology.orchestra.run.smithi157.stdout: "dir_update": 0, 2024-03-21T10:36:14.912 INFO:teuthology.orchestra.run.smithi157.stdout: "dir_update_receipt": 0, 2024-03-21T10:36:14.912 INFO:teuthology.orchestra.run.smithi157.stdout: "dir_try_discover": 0, 2024-03-21T10:36:14.912 INFO:teuthology.orchestra.run.smithi157.stdout: "dir_send_discover": 0, 2024-03-21T10:36:14.912 INFO:teuthology.orchestra.run.smithi157.stdout: "dir_handle_discover": 0, 2024-03-21T10:36:14.912 INFO:teuthology.orchestra.run.smithi157.stdout: "num_strays": 0, 2024-03-21T10:36:14.912 INFO:teuthology.orchestra.run.smithi157.stdout: "num_strays_delayed": 0, 2024-03-21T10:36:14.912 INFO:teuthology.orchestra.run.smithi157.stdout: "num_strays_enqueuing": 0, 2024-03-21T10:36:14.912 INFO:teuthology.orchestra.run.smithi157.stdout: "strays_created": 0, 2024-03-21T10:36:14.912 INFO:teuthology.orchestra.run.smithi157.stdout: "strays_enqueued": 0, 2024-03-21T10:36:14.912 INFO:teuthology.orchestra.run.smithi157.stdout: "strays_reintegrated": 0, 2024-03-21T10:36:14.912 INFO:teuthology.orchestra.run.smithi157.stdout: "strays_migrated": 0, 2024-03-21T10:36:14.912 INFO:teuthology.orchestra.run.smithi157.stdout: "num_recovering_processing": 0, 2024-03-21T10:36:14.912 INFO:teuthology.orchestra.run.smithi157.stdout: "num_recovering_enqueued": 0, 2024-03-21T10:36:14.912 INFO:teuthology.orchestra.run.smithi157.stdout: "num_recovering_prioritized": 0, 2024-03-21T10:36:14.912 INFO:teuthology.orchestra.run.smithi157.stdout: "recovery_started": 0, 2024-03-21T10:36:14.913 INFO:teuthology.orchestra.run.smithi157.stdout: "recovery_completed": 0, 2024-03-21T10:36:14.913 INFO:teuthology.orchestra.run.smithi157.stdout: "ireq_enqueue_scrub": 0, 2024-03-21T10:36:14.913 INFO:teuthology.orchestra.run.smithi157.stdout: "ireq_exportdir": 0, 2024-03-21T10:36:14.913 INFO:teuthology.orchestra.run.smithi157.stdout: "ireq_flush": 0, 2024-03-21T10:36:14.913 INFO:teuthology.orchestra.run.smithi157.stdout: "ireq_fragmentdir": 0, 2024-03-21T10:36:14.913 INFO:teuthology.orchestra.run.smithi157.stdout: "ireq_fragstats": 0, 2024-03-21T10:36:14.913 INFO:teuthology.orchestra.run.smithi157.stdout: "ireq_inodestats": 0 2024-03-21T10:36:14.913 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-03-21T10:36:14.913 INFO:teuthology.orchestra.run.smithi157.stdout: "mds_log": { 2024-03-21T10:36:14.913 INFO:teuthology.orchestra.run.smithi157.stdout: "evlrg": 0, 2024-03-21T10:36:14.913 INFO:teuthology.orchestra.run.smithi157.stdout: "evadd": 0, 2024-03-21T10:36:14.913 INFO:teuthology.orchestra.run.smithi157.stdout: "evex": 0, 2024-03-21T10:36:14.913 INFO:teuthology.orchestra.run.smithi157.stdout: "evtrm": 0, 2024-03-21T10:36:14.913 INFO:teuthology.orchestra.run.smithi157.stdout: "ev": 209, 2024-03-21T10:36:14.913 INFO:teuthology.orchestra.run.smithi157.stdout: "evexg": 0, 2024-03-21T10:36:14.914 INFO:teuthology.orchestra.run.smithi157.stdout: "evexd": 0, 2024-03-21T10:36:14.914 INFO:teuthology.orchestra.run.smithi157.stdout: "segadd": 0, 2024-03-21T10:36:14.914 INFO:teuthology.orchestra.run.smithi157.stdout: "segex": 0, 2024-03-21T10:36:14.914 INFO:teuthology.orchestra.run.smithi157.stdout: "segtrm": 0, 2024-03-21T10:36:14.914 INFO:teuthology.orchestra.run.smithi157.stdout: "seg": 2, 2024-03-21T10:36:14.914 INFO:teuthology.orchestra.run.smithi157.stdout: "segmjr": 11, 2024-03-21T10:36:14.914 INFO:teuthology.orchestra.run.smithi157.stdout: "segexg": 0, 2024-03-21T10:36:14.914 INFO:teuthology.orchestra.run.smithi157.stdout: "segexd": 0, 2024-03-21T10:36:14.914 INFO:teuthology.orchestra.run.smithi157.stdout: "expos": 5259645, 2024-03-21T10:36:14.914 INFO:teuthology.orchestra.run.smithi157.stdout: "wrpos": 5521440, 2024-03-21T10:36:14.914 INFO:teuthology.orchestra.run.smithi157.stdout: "rdpos": 5521304, 2024-03-21T10:36:14.914 INFO:teuthology.orchestra.run.smithi157.stdout: "jlat": { 2024-03-21T10:36:14.914 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-03-21T10:36:14.914 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-03-21T10:36:14.915 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-03-21T10:36:14.915 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-03-21T10:36:14.915 INFO:teuthology.orchestra.run.smithi157.stdout: "replayed": 209 2024-03-21T10:36:14.915 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-03-21T10:36:14.915 INFO:teuthology.orchestra.run.smithi157.stdout: "mds_mem": { 2024-03-21T10:36:14.915 INFO:teuthology.orchestra.run.smithi157.stdout: "ino": 15, 2024-03-21T10:36:14.915 INFO:teuthology.orchestra.run.smithi157.stdout: "ino+": 19, 2024-03-21T10:36:14.915 INFO:teuthology.orchestra.run.smithi157.stdout: "ino-": 4, 2024-03-21T10:36:14.915 INFO:teuthology.orchestra.run.smithi157.stdout: "dir": 9, 2024-03-21T10:36:14.915 INFO:teuthology.orchestra.run.smithi157.stdout: "dir+": 9, 2024-03-21T10:36:14.915 INFO:teuthology.orchestra.run.smithi157.stdout: "dir-": 0, 2024-03-21T10:36:14.915 INFO:teuthology.orchestra.run.smithi157.stdout: "dn": 24, 2024-03-21T10:36:14.915 INFO:teuthology.orchestra.run.smithi157.stdout: "dn+": 24, 2024-03-21T10:36:14.915 INFO:teuthology.orchestra.run.smithi157.stdout: "dn-": 0, 2024-03-21T10:36:14.915 INFO:teuthology.orchestra.run.smithi157.stdout: "cap": 0, 2024-03-21T10:36:14.915 INFO:teuthology.orchestra.run.smithi157.stdout: "cap+": 0, 2024-03-21T10:36:14.916 INFO:teuthology.orchestra.run.smithi157.stdout: "cap-": 0, 2024-03-21T10:36:14.916 INFO:teuthology.orchestra.run.smithi157.stdout: "rss": 48472, 2024-03-21T10:36:14.916 INFO:teuthology.orchestra.run.smithi157.stdout: "heap": 215420 2024-03-21T10:36:14.916 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-03-21T10:36:14.916 INFO:teuthology.orchestra.run.smithi157.stdout: "mds_server": { 2024-03-21T10:36:14.916 INFO:teuthology.orchestra.run.smithi157.stdout: "dispatch_client_request": 0, 2024-03-21T10:36:14.916 INFO:teuthology.orchestra.run.smithi157.stdout: "dispatch_server_request": 0, 2024-03-21T10:36:14.916 INFO:teuthology.orchestra.run.smithi157.stdout: "handle_client_request": 0, 2024-03-21T10:36:14.916 INFO:teuthology.orchestra.run.smithi157.stdout: "handle_client_session": 0, 2024-03-21T10:36:14.916 INFO:teuthology.orchestra.run.smithi157.stdout: "handle_peer_request": 0, 2024-03-21T10:36:14.916 INFO:teuthology.orchestra.run.smithi157.stdout: "req_create_latency": { 2024-03-21T10:36:14.916 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-03-21T10:36:14.916 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-03-21T10:36:14.916 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-03-21T10:36:14.916 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-03-21T10:36:14.916 INFO:teuthology.orchestra.run.smithi157.stdout: "req_getattr_latency": { 2024-03-21T10:36:14.916 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-03-21T10:36:14.917 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-03-21T10:36:14.917 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-03-21T10:36:14.917 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-03-21T10:36:14.917 INFO:teuthology.orchestra.run.smithi157.stdout: "req_getfilelock_latency": { 2024-03-21T10:36:14.917 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-03-21T10:36:14.917 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-03-21T10:36:14.917 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-03-21T10:36:14.917 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-03-21T10:36:14.917 INFO:teuthology.orchestra.run.smithi157.stdout: "req_link_latency": { 2024-03-21T10:36:14.917 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-03-21T10:36:14.917 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-03-21T10:36:14.917 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-03-21T10:36:14.917 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-03-21T10:36:14.917 INFO:teuthology.orchestra.run.smithi157.stdout: "req_lookup_latency": { 2024-03-21T10:36:14.918 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-03-21T10:36:14.918 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-03-21T10:36:14.918 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-03-21T10:36:14.918 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-03-21T10:36:14.918 INFO:teuthology.orchestra.run.smithi157.stdout: "req_lookuphash_latency": { 2024-03-21T10:36:14.918 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-03-21T10:36:14.918 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-03-21T10:36:14.918 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-03-21T10:36:14.918 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-03-21T10:36:14.918 INFO:teuthology.orchestra.run.smithi157.stdout: "req_lookupino_latency": { 2024-03-21T10:36:14.918 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-03-21T10:36:14.918 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-03-21T10:36:14.918 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-03-21T10:36:14.918 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-03-21T10:36:14.919 INFO:teuthology.orchestra.run.smithi157.stdout: "req_lookupname_latency": { 2024-03-21T10:36:14.919 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-03-21T10:36:14.919 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-03-21T10:36:14.919 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-03-21T10:36:14.919 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-03-21T10:36:14.919 INFO:teuthology.orchestra.run.smithi157.stdout: "req_lookupparent_latency": { 2024-03-21T10:36:14.919 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-03-21T10:36:14.919 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-03-21T10:36:14.919 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-03-21T10:36:14.919 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-03-21T10:36:14.919 INFO:teuthology.orchestra.run.smithi157.stdout: "req_lookupsnap_latency": { 2024-03-21T10:36:14.919 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-03-21T10:36:14.919 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-03-21T10:36:14.919 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-03-21T10:36:14.919 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-03-21T10:36:14.920 INFO:teuthology.orchestra.run.smithi157.stdout: "req_lssnap_latency": { 2024-03-21T10:36:14.920 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-03-21T10:36:14.920 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-03-21T10:36:14.920 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-03-21T10:36:14.920 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-03-21T10:36:14.920 INFO:teuthology.orchestra.run.smithi157.stdout: "req_mkdir_latency": { 2024-03-21T10:36:14.920 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-03-21T10:36:14.920 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-03-21T10:36:14.920 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-03-21T10:36:14.920 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-03-21T10:36:14.920 INFO:teuthology.orchestra.run.smithi157.stdout: "req_mknod_latency": { 2024-03-21T10:36:14.921 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-03-21T10:36:14.921 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-03-21T10:36:14.921 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-03-21T10:36:14.921 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-03-21T10:36:14.921 INFO:teuthology.orchestra.run.smithi157.stdout: "req_mksnap_latency": { 2024-03-21T10:36:14.921 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-03-21T10:36:14.921 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-03-21T10:36:14.921 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-03-21T10:36:14.921 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-03-21T10:36:14.921 INFO:teuthology.orchestra.run.smithi157.stdout: "req_open_latency": { 2024-03-21T10:36:14.921 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-03-21T10:36:14.921 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-03-21T10:36:14.921 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-03-21T10:36:14.922 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-03-21T10:36:14.922 INFO:teuthology.orchestra.run.smithi157.stdout: "req_readdir_latency": { 2024-03-21T10:36:14.922 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-03-21T10:36:14.922 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-03-21T10:36:14.922 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-03-21T10:36:14.922 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-03-21T10:36:14.922 INFO:teuthology.orchestra.run.smithi157.stdout: "req_rename_latency": { 2024-03-21T10:36:14.922 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-03-21T10:36:14.922 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-03-21T10:36:14.922 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-03-21T10:36:14.922 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-03-21T10:36:14.922 INFO:teuthology.orchestra.run.smithi157.stdout: "req_renamesnap_latency": { 2024-03-21T10:36:14.922 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-03-21T10:36:14.922 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-03-21T10:36:14.923 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-03-21T10:36:14.923 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-03-21T10:36:14.923 INFO:teuthology.orchestra.run.smithi157.stdout: "req_snapdiff_latency": { 2024-03-21T10:36:14.923 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-03-21T10:36:14.923 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-03-21T10:36:14.923 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-03-21T10:36:14.923 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-03-21T10:36:14.923 INFO:teuthology.orchestra.run.smithi157.stdout: "req_rmdir_latency": { 2024-03-21T10:36:14.923 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-03-21T10:36:14.923 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-03-21T10:36:14.923 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-03-21T10:36:14.923 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-03-21T10:36:14.923 INFO:teuthology.orchestra.run.smithi157.stdout: "req_rmsnap_latency": { 2024-03-21T10:36:14.923 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-03-21T10:36:14.923 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-03-21T10:36:14.924 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-03-21T10:36:14.924 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-03-21T10:36:14.924 INFO:teuthology.orchestra.run.smithi157.stdout: "req_rmxattr_latency": { 2024-03-21T10:36:14.924 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-03-21T10:36:14.924 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-03-21T10:36:14.924 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-03-21T10:36:14.924 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-03-21T10:36:14.924 INFO:teuthology.orchestra.run.smithi157.stdout: "req_setattr_latency": { 2024-03-21T10:36:14.924 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-03-21T10:36:14.924 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-03-21T10:36:14.924 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-03-21T10:36:14.924 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-03-21T10:36:14.924 INFO:teuthology.orchestra.run.smithi157.stdout: "req_setdirlayout_latency": { 2024-03-21T10:36:14.924 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-03-21T10:36:14.924 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-03-21T10:36:14.924 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-03-21T10:36:14.925 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-03-21T10:36:14.925 INFO:teuthology.orchestra.run.smithi157.stdout: "req_setfilelock_latency": { 2024-03-21T10:36:14.925 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-03-21T10:36:14.925 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-03-21T10:36:14.925 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-03-21T10:36:14.925 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-03-21T10:36:14.925 INFO:teuthology.orchestra.run.smithi157.stdout: "req_setlayout_latency": { 2024-03-21T10:36:14.925 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-03-21T10:36:14.925 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-03-21T10:36:14.925 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-03-21T10:36:14.925 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-03-21T10:36:14.925 INFO:teuthology.orchestra.run.smithi157.stdout: "req_setxattr_latency": { 2024-03-21T10:36:14.925 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-03-21T10:36:14.925 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-03-21T10:36:14.925 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-03-21T10:36:14.926 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-03-21T10:36:14.926 INFO:teuthology.orchestra.run.smithi157.stdout: "req_symlink_latency": { 2024-03-21T10:36:14.926 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-03-21T10:36:14.926 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-03-21T10:36:14.926 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-03-21T10:36:14.926 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-03-21T10:36:14.926 INFO:teuthology.orchestra.run.smithi157.stdout: "req_unlink_latency": { 2024-03-21T10:36:14.926 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-03-21T10:36:14.926 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-03-21T10:36:14.926 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-03-21T10:36:14.926 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-03-21T10:36:14.926 INFO:teuthology.orchestra.run.smithi157.stdout: "cap_revoke_eviction": 0, 2024-03-21T10:36:14.926 INFO:teuthology.orchestra.run.smithi157.stdout: "cap_acquisition_throttle": 0, 2024-03-21T10:36:14.926 INFO:teuthology.orchestra.run.smithi157.stdout: "req_getvxattr_latency": { 2024-03-21T10:36:14.927 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-03-21T10:36:14.927 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-03-21T10:36:14.927 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-03-21T10:36:14.927 INFO:teuthology.orchestra.run.smithi157.stdout: } 2024-03-21T10:36:14.927 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-03-21T10:36:14.927 INFO:teuthology.orchestra.run.smithi157.stdout: "mds_sessions": { 2024-03-21T10:36:14.927 INFO:teuthology.orchestra.run.smithi157.stdout: "session_count": 2, 2024-03-21T10:36:14.927 INFO:teuthology.orchestra.run.smithi157.stdout: "session_add": 4, 2024-03-21T10:36:14.927 INFO:teuthology.orchestra.run.smithi157.stdout: "session_remove": 2, 2024-03-21T10:36:14.927 INFO:teuthology.orchestra.run.smithi157.stdout: "sessions_open": 0, 2024-03-21T10:36:14.927 INFO:teuthology.orchestra.run.smithi157.stdout: "sessions_stale": 0, 2024-03-21T10:36:14.927 INFO:teuthology.orchestra.run.smithi157.stdout: "total_load": 0, 2024-03-21T10:36:14.927 INFO:teuthology.orchestra.run.smithi157.stdout: "average_load": 0, 2024-03-21T10:36:14.927 INFO:teuthology.orchestra.run.smithi157.stdout: "avg_session_uptime": 3654, 2024-03-21T10:36:14.927 INFO:teuthology.orchestra.run.smithi157.stdout: "mdthresh_evicted": 0 2024-03-21T10:36:14.928 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-03-21T10:36:14.928 INFO:teuthology.orchestra.run.smithi157.stdout: "mempool": { 2024-03-21T10:36:14.928 INFO:teuthology.orchestra.run.smithi157.stdout: "bloom_filter_bytes": 0, 2024-03-21T10:36:14.928 INFO:teuthology.orchestra.run.smithi157.stdout: "bloom_filter_items": 0, 2024-03-21T10:36:14.928 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_alloc_bytes": 0, 2024-03-21T10:36:14.928 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_alloc_items": 0, 2024-03-21T10:36:14.928 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_cache_data_bytes": 0, 2024-03-21T10:36:14.928 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_cache_data_items": 0, 2024-03-21T10:36:14.928 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_cache_onode_bytes": 0, 2024-03-21T10:36:14.928 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_cache_onode_items": 0, 2024-03-21T10:36:14.928 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_cache_meta_bytes": 0, 2024-03-21T10:36:14.928 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_cache_meta_items": 0, 2024-03-21T10:36:14.928 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_cache_other_bytes": 0, 2024-03-21T10:36:14.928 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_cache_other_items": 0, 2024-03-21T10:36:14.928 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_cache_buffer_bytes": 0, 2024-03-21T10:36:14.928 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_cache_buffer_items": 0, 2024-03-21T10:36:14.929 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_extent_bytes": 0, 2024-03-21T10:36:14.929 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_extent_items": 0, 2024-03-21T10:36:14.929 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_blob_bytes": 0, 2024-03-21T10:36:14.929 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_blob_items": 0, 2024-03-21T10:36:14.929 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_shared_blob_bytes": 0, 2024-03-21T10:36:14.929 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_shared_blob_items": 0, 2024-03-21T10:36:14.929 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_inline_bl_bytes": 0, 2024-03-21T10:36:14.929 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_inline_bl_items": 0, 2024-03-21T10:36:14.929 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_fsck_bytes": 0, 2024-03-21T10:36:14.929 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_fsck_items": 0, 2024-03-21T10:36:14.929 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_txc_bytes": 0, 2024-03-21T10:36:14.929 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_txc_items": 0, 2024-03-21T10:36:14.929 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_writing_deferred_bytes": 0, 2024-03-21T10:36:14.929 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_writing_deferred_items": 0, 2024-03-21T10:36:14.929 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_writing_bytes": 0, 2024-03-21T10:36:14.930 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_writing_items": 0, 2024-03-21T10:36:14.930 INFO:teuthology.orchestra.run.smithi157.stdout: "bluefs_bytes": 0, 2024-03-21T10:36:14.930 INFO:teuthology.orchestra.run.smithi157.stdout: "bluefs_items": 0, 2024-03-21T10:36:14.930 INFO:teuthology.orchestra.run.smithi157.stdout: "bluefs_file_reader_bytes": 0, 2024-03-21T10:36:14.930 INFO:teuthology.orchestra.run.smithi157.stdout: "bluefs_file_reader_items": 0, 2024-03-21T10:36:14.930 INFO:teuthology.orchestra.run.smithi157.stdout: "bluefs_file_writer_bytes": 0, 2024-03-21T10:36:14.930 INFO:teuthology.orchestra.run.smithi157.stdout: "bluefs_file_writer_items": 0, 2024-03-21T10:36:14.930 INFO:teuthology.orchestra.run.smithi157.stdout: "buffer_anon_bytes": 190414, 2024-03-21T10:36:14.930 INFO:teuthology.orchestra.run.smithi157.stdout: "buffer_anon_items": 51, 2024-03-21T10:36:14.930 INFO:teuthology.orchestra.run.smithi157.stdout: "buffer_meta_bytes": 0, 2024-03-21T10:36:14.930 INFO:teuthology.orchestra.run.smithi157.stdout: "buffer_meta_items": 0, 2024-03-21T10:36:14.930 INFO:teuthology.orchestra.run.smithi157.stdout: "osd_bytes": 0, 2024-03-21T10:36:14.930 INFO:teuthology.orchestra.run.smithi157.stdout: "osd_items": 0, 2024-03-21T10:36:14.930 INFO:teuthology.orchestra.run.smithi157.stdout: "osd_mapbl_bytes": 0, 2024-03-21T10:36:14.930 INFO:teuthology.orchestra.run.smithi157.stdout: "osd_mapbl_items": 0, 2024-03-21T10:36:14.931 INFO:teuthology.orchestra.run.smithi157.stdout: "osd_pglog_bytes": 0, 2024-03-21T10:36:14.931 INFO:teuthology.orchestra.run.smithi157.stdout: "osd_pglog_items": 0, 2024-03-21T10:36:14.931 INFO:teuthology.orchestra.run.smithi157.stdout: "osdmap_bytes": 9952, 2024-03-21T10:36:14.931 INFO:teuthology.orchestra.run.smithi157.stdout: "osdmap_items": 193, 2024-03-21T10:36:14.931 INFO:teuthology.orchestra.run.smithi157.stdout: "osdmap_mapping_bytes": 0, 2024-03-21T10:36:14.931 INFO:teuthology.orchestra.run.smithi157.stdout: "osdmap_mapping_items": 0, 2024-03-21T10:36:14.931 INFO:teuthology.orchestra.run.smithi157.stdout: "pgmap_bytes": 0, 2024-03-21T10:36:14.931 INFO:teuthology.orchestra.run.smithi157.stdout: "pgmap_items": 0, 2024-03-21T10:36:14.931 INFO:teuthology.orchestra.run.smithi157.stdout: "mds_co_bytes": 76093, 2024-03-21T10:36:14.931 INFO:teuthology.orchestra.run.smithi157.stdout: "mds_co_items": 515, 2024-03-21T10:36:14.931 INFO:teuthology.orchestra.run.smithi157.stdout: "unittest_1_bytes": 0, 2024-03-21T10:36:14.931 INFO:teuthology.orchestra.run.smithi157.stdout: "unittest_1_items": 0, 2024-03-21T10:36:14.931 INFO:teuthology.orchestra.run.smithi157.stdout: "unittest_2_bytes": 0, 2024-03-21T10:36:14.931 INFO:teuthology.orchestra.run.smithi157.stdout: "unittest_2_items": 0 2024-03-21T10:36:14.931 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-03-21T10:36:14.931 INFO:teuthology.orchestra.run.smithi157.stdout: "objecter": { 2024-03-21T10:36:14.931 INFO:teuthology.orchestra.run.smithi157.stdout: "op_active": 0, 2024-03-21T10:36:14.932 INFO:teuthology.orchestra.run.smithi157.stdout: "op_laggy": 0, 2024-03-21T10:36:14.932 INFO:teuthology.orchestra.run.smithi157.stdout: "op_send": 1906, 2024-03-21T10:36:14.932 INFO:teuthology.orchestra.run.smithi157.stdout: "op_send_bytes": 26, 2024-03-21T10:36:14.932 INFO:teuthology.orchestra.run.smithi157.stdout: "op_resend": 4, 2024-03-21T10:36:14.932 INFO:teuthology.orchestra.run.smithi157.stdout: "op_reply": 1906, 2024-03-21T10:36:14.932 INFO:teuthology.orchestra.run.smithi157.stdout: "op_latency": { 2024-03-21T10:36:14.932 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 1906, 2024-03-21T10:36:14.932 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 2.198959983, 2024-03-21T10:36:14.932 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.001153704 2024-03-21T10:36:14.932 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-03-21T10:36:14.932 INFO:teuthology.orchestra.run.smithi157.stdout: "op_inflight": 0, 2024-03-21T10:36:14.932 INFO:teuthology.orchestra.run.smithi157.stdout: "oplen_avg": { 2024-03-21T10:36:14.932 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 1906, 2024-03-21T10:36:14.932 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 1907 2024-03-21T10:36:14.932 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-03-21T10:36:14.933 INFO:teuthology.orchestra.run.smithi157.stdout: "op": 1906, 2024-03-21T10:36:14.933 INFO:teuthology.orchestra.run.smithi157.stdout: "op_r": 1906, 2024-03-21T10:36:14.933 INFO:teuthology.orchestra.run.smithi157.stdout: "op_w": 0, 2024-03-21T10:36:14.933 INFO:teuthology.orchestra.run.smithi157.stdout: "op_rmw": 0, 2024-03-21T10:36:14.933 INFO:teuthology.orchestra.run.smithi157.stdout: "op_pg": 0, 2024-03-21T10:36:14.933 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_stat": 1192, 2024-03-21T10:36:14.933 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_create": 0, 2024-03-21T10:36:14.933 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_read": 711, 2024-03-21T10:36:14.933 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_write": 0, 2024-03-21T10:36:14.933 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_writefull": 0, 2024-03-21T10:36:14.933 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_writesame": 0, 2024-03-21T10:36:14.933 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_append": 0, 2024-03-21T10:36:14.933 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_zero": 0, 2024-03-21T10:36:14.933 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_truncate": 0, 2024-03-21T10:36:14.933 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_delete": 0, 2024-03-21T10:36:14.934 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_mapext": 0, 2024-03-21T10:36:14.934 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_sparse_read": 0, 2024-03-21T10:36:14.934 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_clonerange": 0, 2024-03-21T10:36:14.934 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_getxattr": 2, 2024-03-21T10:36:14.934 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_setxattr": 0, 2024-03-21T10:36:14.934 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_cmpxattr": 0, 2024-03-21T10:36:14.934 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_rmxattr": 0, 2024-03-21T10:36:14.934 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_resetxattrs": 0, 2024-03-21T10:36:14.934 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_call": 0, 2024-03-21T10:36:14.934 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_watch": 0, 2024-03-21T10:36:14.934 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_notify": 0, 2024-03-21T10:36:14.934 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_src_cmpxattr": 0, 2024-03-21T10:36:14.934 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_pgls": 0, 2024-03-21T10:36:14.934 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_pgls_filter": 0, 2024-03-21T10:36:14.934 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_other": 0, 2024-03-21T10:36:14.935 INFO:teuthology.orchestra.run.smithi157.stdout: "linger_active": 0, 2024-03-21T10:36:14.935 INFO:teuthology.orchestra.run.smithi157.stdout: "linger_send": 0, 2024-03-21T10:36:14.935 INFO:teuthology.orchestra.run.smithi157.stdout: "linger_resend": 0, 2024-03-21T10:36:14.935 INFO:teuthology.orchestra.run.smithi157.stdout: "linger_ping": 0, 2024-03-21T10:36:14.935 INFO:teuthology.orchestra.run.smithi157.stdout: "poolop_active": 0, 2024-03-21T10:36:14.935 INFO:teuthology.orchestra.run.smithi157.stdout: "poolop_send": 0, 2024-03-21T10:36:14.935 INFO:teuthology.orchestra.run.smithi157.stdout: "poolop_resend": 0, 2024-03-21T10:36:14.935 INFO:teuthology.orchestra.run.smithi157.stdout: "poolstat_active": 0, 2024-03-21T10:36:14.935 INFO:teuthology.orchestra.run.smithi157.stdout: "poolstat_send": 0, 2024-03-21T10:36:14.935 INFO:teuthology.orchestra.run.smithi157.stdout: "poolstat_resend": 0, 2024-03-21T10:36:14.935 INFO:teuthology.orchestra.run.smithi157.stdout: "statfs_active": 0, 2024-03-21T10:36:14.935 INFO:teuthology.orchestra.run.smithi157.stdout: "statfs_send": 0, 2024-03-21T10:36:14.935 INFO:teuthology.orchestra.run.smithi157.stdout: "statfs_resend": 0, 2024-03-21T10:36:14.935 INFO:teuthology.orchestra.run.smithi157.stdout: "command_active": 0, 2024-03-21T10:36:14.936 INFO:teuthology.orchestra.run.smithi157.stdout: "command_send": 0, 2024-03-21T10:36:14.936 INFO:teuthology.orchestra.run.smithi157.stdout: "command_resend": 0, 2024-03-21T10:36:14.936 INFO:teuthology.orchestra.run.smithi157.stdout: "map_epoch": 107, 2024-03-21T10:36:14.936 INFO:teuthology.orchestra.run.smithi157.stdout: "map_full": 0, 2024-03-21T10:36:14.936 INFO:teuthology.orchestra.run.smithi157.stdout: "map_inc": 10, 2024-03-21T10:36:14.936 INFO:teuthology.orchestra.run.smithi157.stdout: "osd_sessions": 5, 2024-03-21T10:36:14.936 INFO:teuthology.orchestra.run.smithi157.stdout: "osd_session_open": 5, 2024-03-21T10:36:14.936 INFO:teuthology.orchestra.run.smithi157.stdout: "osd_session_close": 0, 2024-03-21T10:36:14.936 INFO:teuthology.orchestra.run.smithi157.stdout: "osd_laggy": 0, 2024-03-21T10:36:14.936 INFO:teuthology.orchestra.run.smithi157.stdout: "omap_wr": 0, 2024-03-21T10:36:14.936 INFO:teuthology.orchestra.run.smithi157.stdout: "omap_rd": 2, 2024-03-21T10:36:14.936 INFO:teuthology.orchestra.run.smithi157.stdout: "omap_del": 0 2024-03-21T10:36:14.936 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-03-21T10:36:14.936 INFO:teuthology.orchestra.run.smithi157.stdout: "oft": { 2024-03-21T10:36:14.936 INFO:teuthology.orchestra.run.smithi157.stdout: "omap_total_objs": 0, 2024-03-21T10:36:14.936 INFO:teuthology.orchestra.run.smithi157.stdout: "omap_total_kv_pairs": 0, 2024-03-21T10:36:14.936 INFO:teuthology.orchestra.run.smithi157.stdout: "omap_total_updates": 0, 2024-03-21T10:36:14.937 INFO:teuthology.orchestra.run.smithi157.stdout: "omap_total_removes": 0 2024-03-21T10:36:14.937 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-03-21T10:36:14.937 INFO:teuthology.orchestra.run.smithi157.stdout: "osd-slow-ops": { 2024-03-21T10:36:14.937 INFO:teuthology.orchestra.run.smithi157.stdout: "slow_ops_count": 0 2024-03-21T10:36:14.937 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-03-21T10:36:14.937 INFO:teuthology.orchestra.run.smithi157.stdout: "purge_queue": { 2024-03-21T10:36:14.937 INFO:teuthology.orchestra.run.smithi157.stdout: "pq_executing_ops": 0, 2024-03-21T10:36:14.937 INFO:teuthology.orchestra.run.smithi157.stdout: "pq_executing_ops_high_water": 0, 2024-03-21T10:36:14.937 INFO:teuthology.orchestra.run.smithi157.stdout: "pq_executing": 0, 2024-03-21T10:36:14.937 INFO:teuthology.orchestra.run.smithi157.stdout: "pq_executing_high_water": 0, 2024-03-21T10:36:14.937 INFO:teuthology.orchestra.run.smithi157.stdout: "pq_executed_ops": 0, 2024-03-21T10:36:14.937 INFO:teuthology.orchestra.run.smithi157.stdout: "pq_executed": 0, 2024-03-21T10:36:14.937 INFO:teuthology.orchestra.run.smithi157.stdout: "pq_item_in_journal": 0 2024-03-21T10:36:14.937 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-03-21T10:36:14.938 INFO:teuthology.orchestra.run.smithi157.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-03-21T10:36:14.938 INFO:teuthology.orchestra.run.smithi157.stdout: "val": 0, 2024-03-21T10:36:14.938 INFO:teuthology.orchestra.run.smithi157.stdout: "max": 104857600, 2024-03-21T10:36:14.938 INFO:teuthology.orchestra.run.smithi157.stdout: "get_started": 0, 2024-03-21T10:36:14.938 INFO:teuthology.orchestra.run.smithi157.stdout: "get": 2088, 2024-03-21T10:36:14.938 INFO:teuthology.orchestra.run.smithi157.stdout: "get_sum": 1933507, 2024-03-21T10:36:14.938 INFO:teuthology.orchestra.run.smithi157.stdout: "get_or_fail_fail": 0, 2024-03-21T10:36:14.938 INFO:teuthology.orchestra.run.smithi157.stdout: "get_or_fail_success": 2088, 2024-03-21T10:36:14.938 INFO:teuthology.orchestra.run.smithi157.stdout: "take": 0, 2024-03-21T10:36:14.938 INFO:teuthology.orchestra.run.smithi157.stdout: "take_sum": 0, 2024-03-21T10:36:14.938 INFO:teuthology.orchestra.run.smithi157.stdout: "put": 2088, 2024-03-21T10:36:14.938 INFO:teuthology.orchestra.run.smithi157.stdout: "put_sum": 1933507, 2024-03-21T10:36:14.938 INFO:teuthology.orchestra.run.smithi157.stdout: "wait": { 2024-03-21T10:36:14.938 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-03-21T10:36:14.938 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-03-21T10:36:14.939 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-03-21T10:36:14.939 INFO:teuthology.orchestra.run.smithi157.stdout: } 2024-03-21T10:36:14.939 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-03-21T10:36:14.939 INFO:teuthology.orchestra.run.smithi157.stdout: "throttle-objecter_bytes": { 2024-03-21T10:36:14.939 INFO:teuthology.orchestra.run.smithi157.stdout: "val": 0, 2024-03-21T10:36:14.939 INFO:teuthology.orchestra.run.smithi157.stdout: "max": 104857600, 2024-03-21T10:36:14.939 INFO:teuthology.orchestra.run.smithi157.stdout: "get_started": 0, 2024-03-21T10:36:14.939 INFO:teuthology.orchestra.run.smithi157.stdout: "get": 0, 2024-03-21T10:36:14.939 INFO:teuthology.orchestra.run.smithi157.stdout: "get_sum": 0, 2024-03-21T10:36:14.939 INFO:teuthology.orchestra.run.smithi157.stdout: "get_or_fail_fail": 0, 2024-03-21T10:36:14.939 INFO:teuthology.orchestra.run.smithi157.stdout: "get_or_fail_success": 0, 2024-03-21T10:36:14.939 INFO:teuthology.orchestra.run.smithi157.stdout: "take": 1906, 2024-03-21T10:36:14.939 INFO:teuthology.orchestra.run.smithi157.stdout: "take_sum": 1244105, 2024-03-21T10:36:14.939 INFO:teuthology.orchestra.run.smithi157.stdout: "put": 1906, 2024-03-21T10:36:14.939 INFO:teuthology.orchestra.run.smithi157.stdout: "put_sum": 1244105, 2024-03-21T10:36:14.940 INFO:teuthology.orchestra.run.smithi157.stdout: "wait": { 2024-03-21T10:36:14.940 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-03-21T10:36:14.940 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-03-21T10:36:14.940 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-03-21T10:36:14.940 INFO:teuthology.orchestra.run.smithi157.stdout: } 2024-03-21T10:36:14.940 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-03-21T10:36:14.940 INFO:teuthology.orchestra.run.smithi157.stdout: "throttle-objecter_ops": { 2024-03-21T10:36:14.940 INFO:teuthology.orchestra.run.smithi157.stdout: "val": 0, 2024-03-21T10:36:14.940 INFO:teuthology.orchestra.run.smithi157.stdout: "max": 1024, 2024-03-21T10:36:14.940 INFO:teuthology.orchestra.run.smithi157.stdout: "get_started": 0, 2024-03-21T10:36:14.941 INFO:teuthology.orchestra.run.smithi157.stdout: "get": 0, 2024-03-21T10:36:14.941 INFO:teuthology.orchestra.run.smithi157.stdout: "get_sum": 0, 2024-03-21T10:36:14.941 INFO:teuthology.orchestra.run.smithi157.stdout: "get_or_fail_fail": 0, 2024-03-21T10:36:14.941 INFO:teuthology.orchestra.run.smithi157.stdout: "get_or_fail_success": 0, 2024-03-21T10:36:14.941 INFO:teuthology.orchestra.run.smithi157.stdout: "take": 1906, 2024-03-21T10:36:14.941 INFO:teuthology.orchestra.run.smithi157.stdout: "take_sum": 1906, 2024-03-21T10:36:14.941 INFO:teuthology.orchestra.run.smithi157.stdout: "put": 1906, 2024-03-21T10:36:14.941 INFO:teuthology.orchestra.run.smithi157.stdout: "put_sum": 1906, 2024-03-21T10:36:14.941 INFO:teuthology.orchestra.run.smithi157.stdout: "wait": { 2024-03-21T10:36:14.941 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-03-21T10:36:14.941 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-03-21T10:36:14.941 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-03-21T10:36:14.941 INFO:teuthology.orchestra.run.smithi157.stdout: } 2024-03-21T10:36:14.941 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-03-21T10:36:14.942 INFO:teuthology.orchestra.run.smithi157.stdout: "throttle-write_buf_throttle": { 2024-03-21T10:36:14.942 INFO:teuthology.orchestra.run.smithi157.stdout: "val": 0, 2024-03-21T10:36:14.942 INFO:teuthology.orchestra.run.smithi157.stdout: "max": 3758096384, 2024-03-21T10:36:14.942 INFO:teuthology.orchestra.run.smithi157.stdout: "get_started": 0, 2024-03-21T10:36:14.942 INFO:teuthology.orchestra.run.smithi157.stdout: "get": 0, 2024-03-21T10:36:14.942 INFO:teuthology.orchestra.run.smithi157.stdout: "get_sum": 0, 2024-03-21T10:36:14.942 INFO:teuthology.orchestra.run.smithi157.stdout: "get_or_fail_fail": 0, 2024-03-21T10:36:14.942 INFO:teuthology.orchestra.run.smithi157.stdout: "get_or_fail_success": 0, 2024-03-21T10:36:14.942 INFO:teuthology.orchestra.run.smithi157.stdout: "take": 0, 2024-03-21T10:36:14.942 INFO:teuthology.orchestra.run.smithi157.stdout: "take_sum": 0, 2024-03-21T10:36:14.942 INFO:teuthology.orchestra.run.smithi157.stdout: "put": 0, 2024-03-21T10:36:14.942 INFO:teuthology.orchestra.run.smithi157.stdout: "put_sum": 0, 2024-03-21T10:36:14.942 INFO:teuthology.orchestra.run.smithi157.stdout: "wait": { 2024-03-21T10:36:14.942 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-03-21T10:36:14.943 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-03-21T10:36:14.943 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-03-21T10:36:14.943 INFO:teuthology.orchestra.run.smithi157.stdout: } 2024-03-21T10:36:14.943 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-03-21T10:36:14.943 INFO:teuthology.orchestra.run.smithi157.stdout: "throttle-write_buf_throttle-0x55b02925a780": { 2024-03-21T10:36:14.943 INFO:teuthology.orchestra.run.smithi157.stdout: "val": 0, 2024-03-21T10:36:14.943 INFO:teuthology.orchestra.run.smithi157.stdout: "max": 3758096384, 2024-03-21T10:36:14.943 INFO:teuthology.orchestra.run.smithi157.stdout: "get_started": 0, 2024-03-21T10:36:14.943 INFO:teuthology.orchestra.run.smithi157.stdout: "get": 0, 2024-03-21T10:36:14.943 INFO:teuthology.orchestra.run.smithi157.stdout: "get_sum": 0, 2024-03-21T10:36:14.943 INFO:teuthology.orchestra.run.smithi157.stdout: "get_or_fail_fail": 0, 2024-03-21T10:36:14.943 INFO:teuthology.orchestra.run.smithi157.stdout: "get_or_fail_success": 0, 2024-03-21T10:36:14.943 INFO:teuthology.orchestra.run.smithi157.stdout: "take": 0, 2024-03-21T10:36:14.943 INFO:teuthology.orchestra.run.smithi157.stdout: "take_sum": 0, 2024-03-21T10:36:14.943 INFO:teuthology.orchestra.run.smithi157.stdout: "put": 0, 2024-03-21T10:36:14.944 INFO:teuthology.orchestra.run.smithi157.stdout: "put_sum": 0, 2024-03-21T10:36:14.944 INFO:teuthology.orchestra.run.smithi157.stdout: "wait": { 2024-03-21T10:36:14.944 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-03-21T10:36:14.944 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-03-21T10:36:14.944 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-03-21T10:36:14.944 INFO:teuthology.orchestra.run.smithi157.stdout: } 2024-03-21T10:36:14.944 INFO:teuthology.orchestra.run.smithi157.stdout: } 2024-03-21T10:36:14.944 INFO:teuthology.orchestra.run.smithi157.stdout:} 2024-03-21T10:36:15.210 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:36:14 smithi047 ceph-mon[31547]: osdmap e107: 12 total, 12 up, 12 in 2024-03-21T10:36:15.211 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:36:14 smithi047 ceph-mon[31547]: pgmap v341: 129 pgs: 129 active+clean; 2.5 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 127 B/s wr, 7 op/s 2024-03-21T10:36:15.226 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:36:14 smithi100 ceph-mon[35193]: osdmap e107: 12 total, 12 up, 12 in 2024-03-21T10:36:15.226 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:36:14 smithi100 ceph-mon[35193]: pgmap v341: 129 pgs: 129 active+clean; 2.5 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 127 B/s wr, 7 op/s 2024-03-21T10:36:15.236 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:36:14 smithi157 ceph-mon[35581]: osdmap e107: 12 total, 12 up, 12 in 2024-03-21T10:36:15.236 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:36:14 smithi157 ceph-mon[35581]: pgmap v341: 129 pgs: 129 active+clean; 2.5 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 127 B/s wr, 7 op/s 2024-03-21T10:36:15.831 INFO:tasks.check_counter:Daemon mds.i mds.exported=0 2024-03-21T10:36:15.831 INFO:tasks.check_counter:Daemon mds.i mds.imported=0 2024-03-21T10:36:15.831 INFO:tasks.check_counter:Daemon mds.i mds.root_rsnaps=6 2024-03-21T10:36:15.831 INFO:tasks.check_counter:Daemon mds.i mds_server.req_mksnap_latency.avgcount=0 2024-03-21T10:36:15.831 INFO:tasks.check_counter:Daemon mds.i mds_server.req_rmsnap_latency.avgcount=0 2024-03-21T10:36:15.831 DEBUG:tasks.check_counter:Getting stats from l 2024-03-21T10:36:15.832 DEBUG:teuthology.orchestra.run.smithi157:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2edfee68ad7243bed422a049b6f5f58619300f11 shell --fsid 3056d51a-e76b-11ee-95cd-87774f69a715 -- ceph daemon mds.l perf dump 2024-03-21T10:36:16.296 INFO:teuthology.orchestra.run.smithi157.stderr:Inferring config /var/lib/ceph/3056d51a-e76b-11ee-95cd-87774f69a715/mon.c/config 2024-03-21T10:36:17.487 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:36:17 smithi157 ceph-mon[35581]: pgmap v342: 129 pgs: 129 active+clean; 2.5 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 255 B/s wr, 12 op/s 2024-03-21T10:36:17.710 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:36:17 smithi047 ceph-mon[31547]: pgmap v342: 129 pgs: 129 active+clean; 2.5 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 255 B/s wr, 12 op/s 2024-03-21T10:36:17.726 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:36:17 smithi100 ceph-mon[35193]: pgmap v342: 129 pgs: 129 active+clean; 2.5 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 255 B/s wr, 12 op/s 2024-03-21T10:36:18.388 INFO:teuthology.orchestra.run.smithi157.stdout:{ 2024-03-21T10:36:18.388 INFO:teuthology.orchestra.run.smithi157.stdout: "AsyncMessenger::Worker-0": { 2024-03-21T10:36:18.388 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_recv_messages": 1109, 2024-03-21T10:36:18.388 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_send_messages": 1091, 2024-03-21T10:36:18.388 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_recv_bytes": 2069952, 2024-03-21T10:36:18.388 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_send_bytes": 413096, 2024-03-21T10:36:18.388 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_created_connections": 5, 2024-03-21T10:36:18.388 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_active_connections": 1, 2024-03-21T10:36:18.389 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_running_total_time": 0.284811533, 2024-03-21T10:36:18.389 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_running_send_time": 0.095250061, 2024-03-21T10:36:18.389 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_running_recv_time": 0.127320177, 2024-03-21T10:36:18.389 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_running_fast_dispatch_time": 0.063410734, 2024-03-21T10:36:18.389 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_send_messages_queue_lat": { 2024-03-21T10:36:18.389 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 1090, 2024-03-21T10:36:18.389 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.113143473, 2024-03-21T10:36:18.389 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000103801 2024-03-21T10:36:18.389 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-03-21T10:36:18.389 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_handle_ack_lat": { 2024-03-21T10:36:18.389 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-03-21T10:36:18.389 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-03-21T10:36:18.390 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-03-21T10:36:18.390 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-03-21T10:36:18.390 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_recv_encrypted_bytes": 1791744, 2024-03-21T10:36:18.390 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_send_encrypted_bytes": 149792 2024-03-21T10:36:18.390 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-03-21T10:36:18.390 INFO:teuthology.orchestra.run.smithi157.stdout: "AsyncMessenger::Worker-1": { 2024-03-21T10:36:18.390 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_recv_messages": 5, 2024-03-21T10:36:18.390 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_send_messages": 189, 2024-03-21T10:36:18.390 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_recv_bytes": 1710, 2024-03-21T10:36:18.390 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_send_bytes": 407758, 2024-03-21T10:36:18.390 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_created_connections": 7, 2024-03-21T10:36:18.390 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_active_connections": 5, 2024-03-21T10:36:18.391 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_running_total_time": 0.025882363, 2024-03-21T10:36:18.391 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_running_send_time": 0.015910454, 2024-03-21T10:36:18.391 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_running_recv_time": 0.004847481, 2024-03-21T10:36:18.391 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_running_fast_dispatch_time": 0.000122359, 2024-03-21T10:36:18.391 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_send_messages_queue_lat": { 2024-03-21T10:36:18.391 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 186, 2024-03-21T10:36:18.391 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.035740996, 2024-03-21T10:36:18.391 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000192155 2024-03-21T10:36:18.391 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-03-21T10:36:18.391 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_handle_ack_lat": { 2024-03-21T10:36:18.391 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-03-21T10:36:18.391 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-03-21T10:36:18.391 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-03-21T10:36:18.392 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-03-21T10:36:18.392 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_recv_encrypted_bytes": 320, 2024-03-21T10:36:18.392 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_send_encrypted_bytes": 406736 2024-03-21T10:36:18.392 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-03-21T10:36:18.392 INFO:teuthology.orchestra.run.smithi157.stdout: "AsyncMessenger::Worker-2": { 2024-03-21T10:36:18.392 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_recv_messages": 1519, 2024-03-21T10:36:18.392 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_send_messages": 1519, 2024-03-21T10:36:18.392 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_recv_bytes": 391475, 2024-03-21T10:36:18.392 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_send_bytes": 482219, 2024-03-21T10:36:18.392 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_created_connections": 5, 2024-03-21T10:36:18.392 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_active_connections": 4, 2024-03-21T10:36:18.392 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_running_total_time": 0.297494053, 2024-03-21T10:36:18.392 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_running_send_time": 0.122327681, 2024-03-21T10:36:18.393 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_running_recv_time": 0.072229983, 2024-03-21T10:36:18.393 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_running_fast_dispatch_time": 0.082488173, 2024-03-21T10:36:18.393 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_send_messages_queue_lat": { 2024-03-21T10:36:18.393 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 1518, 2024-03-21T10:36:18.393 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.168459541, 2024-03-21T10:36:18.393 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000110974 2024-03-21T10:36:18.393 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-03-21T10:36:18.393 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_handle_ack_lat": { 2024-03-21T10:36:18.393 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-03-21T10:36:18.393 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-03-21T10:36:18.393 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-03-21T10:36:18.393 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-03-21T10:36:18.394 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_recv_encrypted_bytes": 2944, 2024-03-21T10:36:18.394 INFO:teuthology.orchestra.run.smithi157.stdout: "msgr_send_encrypted_bytes": 448 2024-03-21T10:36:18.394 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-03-21T10:36:18.394 INFO:teuthology.orchestra.run.smithi157.stdout: "cct": { 2024-03-21T10:36:18.394 INFO:teuthology.orchestra.run.smithi157.stdout: "total_workers": 1, 2024-03-21T10:36:18.394 INFO:teuthology.orchestra.run.smithi157.stdout: "unhealthy_workers": 0 2024-03-21T10:36:18.394 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-03-21T10:36:18.394 INFO:teuthology.orchestra.run.smithi157.stdout: "finisher-MDSRank": { 2024-03-21T10:36:18.394 INFO:teuthology.orchestra.run.smithi157.stdout: "queue_len": 0, 2024-03-21T10:36:18.394 INFO:teuthology.orchestra.run.smithi157.stdout: "complete_latency": { 2024-03-21T10:36:18.394 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 3997, 2024-03-21T10:36:18.394 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.206431959, 2024-03-21T10:36:18.394 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000051646 2024-03-21T10:36:18.394 INFO:teuthology.orchestra.run.smithi157.stdout: } 2024-03-21T10:36:18.395 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-03-21T10:36:18.395 INFO:teuthology.orchestra.run.smithi157.stdout: "finisher-PurgeQueue": { 2024-03-21T10:36:18.395 INFO:teuthology.orchestra.run.smithi157.stdout: "queue_len": 0, 2024-03-21T10:36:18.395 INFO:teuthology.orchestra.run.smithi157.stdout: "complete_latency": { 2024-03-21T10:36:18.395 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-03-21T10:36:18.395 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-03-21T10:36:18.395 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-03-21T10:36:18.395 INFO:teuthology.orchestra.run.smithi157.stdout: } 2024-03-21T10:36:18.395 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-03-21T10:36:18.395 INFO:teuthology.orchestra.run.smithi157.stdout: "mds": { 2024-03-21T10:36:18.395 INFO:teuthology.orchestra.run.smithi157.stdout: "request": 0, 2024-03-21T10:36:18.395 INFO:teuthology.orchestra.run.smithi157.stdout: "reply": 0, 2024-03-21T10:36:18.395 INFO:teuthology.orchestra.run.smithi157.stdout: "reply_latency": { 2024-03-21T10:36:18.396 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-03-21T10:36:18.396 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-03-21T10:36:18.396 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-03-21T10:36:18.396 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-03-21T10:36:18.396 INFO:teuthology.orchestra.run.smithi157.stdout: "slow_reply": 0, 2024-03-21T10:36:18.396 INFO:teuthology.orchestra.run.smithi157.stdout: "forward": 0, 2024-03-21T10:36:18.396 INFO:teuthology.orchestra.run.smithi157.stdout: "dir_fetch_complete": 0, 2024-03-21T10:36:18.396 INFO:teuthology.orchestra.run.smithi157.stdout: "dir_fetch_keys": 0, 2024-03-21T10:36:18.396 INFO:teuthology.orchestra.run.smithi157.stdout: "dir_commit": 0, 2024-03-21T10:36:18.396 INFO:teuthology.orchestra.run.smithi157.stdout: "dir_split": 0, 2024-03-21T10:36:18.396 INFO:teuthology.orchestra.run.smithi157.stdout: "dir_merge": 0, 2024-03-21T10:36:18.396 INFO:teuthology.orchestra.run.smithi157.stdout: "inodes": 0, 2024-03-21T10:36:18.396 INFO:teuthology.orchestra.run.smithi157.stdout: "inodes_top": 0, 2024-03-21T10:36:18.396 INFO:teuthology.orchestra.run.smithi157.stdout: "inodes_bottom": 0, 2024-03-21T10:36:18.396 INFO:teuthology.orchestra.run.smithi157.stdout: "inodes_pin_tail": 0, 2024-03-21T10:36:18.397 INFO:teuthology.orchestra.run.smithi157.stdout: "inodes_pinned": 0, 2024-03-21T10:36:18.397 INFO:teuthology.orchestra.run.smithi157.stdout: "inodes_expired": 0, 2024-03-21T10:36:18.397 INFO:teuthology.orchestra.run.smithi157.stdout: "inodes_with_caps": 0, 2024-03-21T10:36:18.397 INFO:teuthology.orchestra.run.smithi157.stdout: "caps": 0, 2024-03-21T10:36:18.397 INFO:teuthology.orchestra.run.smithi157.stdout: "subtrees": 1, 2024-03-21T10:36:18.397 INFO:teuthology.orchestra.run.smithi157.stdout: "traverse": 0, 2024-03-21T10:36:18.397 INFO:teuthology.orchestra.run.smithi157.stdout: "traverse_hit": 0, 2024-03-21T10:36:18.397 INFO:teuthology.orchestra.run.smithi157.stdout: "traverse_forward": 0, 2024-03-21T10:36:18.397 INFO:teuthology.orchestra.run.smithi157.stdout: "traverse_discover": 0, 2024-03-21T10:36:18.397 INFO:teuthology.orchestra.run.smithi157.stdout: "traverse_dir_fetch": 0, 2024-03-21T10:36:18.397 INFO:teuthology.orchestra.run.smithi157.stdout: "traverse_remote_ino": 0, 2024-03-21T10:36:18.397 INFO:teuthology.orchestra.run.smithi157.stdout: "traverse_lock": 0, 2024-03-21T10:36:18.397 INFO:teuthology.orchestra.run.smithi157.stdout: "load_cent": 0, 2024-03-21T10:36:18.397 INFO:teuthology.orchestra.run.smithi157.stdout: "q": 0, 2024-03-21T10:36:18.397 INFO:teuthology.orchestra.run.smithi157.stdout: "exported": 0, 2024-03-21T10:36:18.398 INFO:teuthology.orchestra.run.smithi157.stdout: "exported_inodes": 0, 2024-03-21T10:36:18.398 INFO:teuthology.orchestra.run.smithi157.stdout: "imported": 0, 2024-03-21T10:36:18.398 INFO:teuthology.orchestra.run.smithi157.stdout: "imported_inodes": 0, 2024-03-21T10:36:18.398 INFO:teuthology.orchestra.run.smithi157.stdout: "openino_dir_fetch": 0, 2024-03-21T10:36:18.398 INFO:teuthology.orchestra.run.smithi157.stdout: "openino_backtrace_fetch": 0, 2024-03-21T10:36:18.398 INFO:teuthology.orchestra.run.smithi157.stdout: "openino_peer_discover": 0, 2024-03-21T10:36:18.398 INFO:teuthology.orchestra.run.smithi157.stdout: "root_rfiles": 0, 2024-03-21T10:36:18.398 INFO:teuthology.orchestra.run.smithi157.stdout: "root_rbytes": 0, 2024-03-21T10:36:18.398 INFO:teuthology.orchestra.run.smithi157.stdout: "root_rsnaps": 0, 2024-03-21T10:36:18.398 INFO:teuthology.orchestra.run.smithi157.stdout: "scrub_backtrace_fetch": 0, 2024-03-21T10:36:18.398 INFO:teuthology.orchestra.run.smithi157.stdout: "scrub_set_tag": 0, 2024-03-21T10:36:18.398 INFO:teuthology.orchestra.run.smithi157.stdout: "scrub_backtrace_repaired": 0, 2024-03-21T10:36:18.398 INFO:teuthology.orchestra.run.smithi157.stdout: "scrub_inotable_repaired": 0, 2024-03-21T10:36:18.399 INFO:teuthology.orchestra.run.smithi157.stdout: "scrub_dir_inodes": 0, 2024-03-21T10:36:18.399 INFO:teuthology.orchestra.run.smithi157.stdout: "scrub_dir_base_inodes": 0, 2024-03-21T10:36:18.399 INFO:teuthology.orchestra.run.smithi157.stdout: "scrub_dirfrag_rstats": 0, 2024-03-21T10:36:18.399 INFO:teuthology.orchestra.run.smithi157.stdout: "scrub_file_inodes": 0, 2024-03-21T10:36:18.399 INFO:teuthology.orchestra.run.smithi157.stdout: "handle_inode_file_caps": 0, 2024-03-21T10:36:18.399 INFO:teuthology.orchestra.run.smithi157.stdout: "ceph_cap_op_revoke": 0, 2024-03-21T10:36:18.399 INFO:teuthology.orchestra.run.smithi157.stdout: "ceph_cap_op_grant": 0, 2024-03-21T10:36:18.399 INFO:teuthology.orchestra.run.smithi157.stdout: "ceph_cap_op_trunc": 0, 2024-03-21T10:36:18.399 INFO:teuthology.orchestra.run.smithi157.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-03-21T10:36:18.399 INFO:teuthology.orchestra.run.smithi157.stdout: "ceph_cap_op_flush_ack": 0, 2024-03-21T10:36:18.399 INFO:teuthology.orchestra.run.smithi157.stdout: "handle_client_caps": 0, 2024-03-21T10:36:18.399 INFO:teuthology.orchestra.run.smithi157.stdout: "handle_client_caps_dirty": 0, 2024-03-21T10:36:18.399 INFO:teuthology.orchestra.run.smithi157.stdout: "handle_client_cap_release": 0, 2024-03-21T10:36:18.399 INFO:teuthology.orchestra.run.smithi157.stdout: "process_request_cap_release": 0 2024-03-21T10:36:18.399 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-03-21T10:36:18.399 INFO:teuthology.orchestra.run.smithi157.stdout: "mds_cache": { 2024-03-21T10:36:18.400 INFO:teuthology.orchestra.run.smithi157.stdout: "dir_update": 0, 2024-03-21T10:36:18.400 INFO:teuthology.orchestra.run.smithi157.stdout: "dir_update_receipt": 0, 2024-03-21T10:36:18.400 INFO:teuthology.orchestra.run.smithi157.stdout: "dir_try_discover": 0, 2024-03-21T10:36:18.400 INFO:teuthology.orchestra.run.smithi157.stdout: "dir_send_discover": 0, 2024-03-21T10:36:18.400 INFO:teuthology.orchestra.run.smithi157.stdout: "dir_handle_discover": 0, 2024-03-21T10:36:18.400 INFO:teuthology.orchestra.run.smithi157.stdout: "num_strays": 0, 2024-03-21T10:36:18.400 INFO:teuthology.orchestra.run.smithi157.stdout: "num_strays_delayed": 0, 2024-03-21T10:36:18.400 INFO:teuthology.orchestra.run.smithi157.stdout: "num_strays_enqueuing": 0, 2024-03-21T10:36:18.400 INFO:teuthology.orchestra.run.smithi157.stdout: "strays_created": 0, 2024-03-21T10:36:18.400 INFO:teuthology.orchestra.run.smithi157.stdout: "strays_enqueued": 0, 2024-03-21T10:36:18.400 INFO:teuthology.orchestra.run.smithi157.stdout: "strays_reintegrated": 0, 2024-03-21T10:36:18.400 INFO:teuthology.orchestra.run.smithi157.stdout: "strays_migrated": 0, 2024-03-21T10:36:18.400 INFO:teuthology.orchestra.run.smithi157.stdout: "num_recovering_processing": 0, 2024-03-21T10:36:18.400 INFO:teuthology.orchestra.run.smithi157.stdout: "num_recovering_enqueued": 0, 2024-03-21T10:36:18.400 INFO:teuthology.orchestra.run.smithi157.stdout: "num_recovering_prioritized": 0, 2024-03-21T10:36:18.400 INFO:teuthology.orchestra.run.smithi157.stdout: "recovery_started": 0, 2024-03-21T10:36:18.401 INFO:teuthology.orchestra.run.smithi157.stdout: "recovery_completed": 0, 2024-03-21T10:36:18.401 INFO:teuthology.orchestra.run.smithi157.stdout: "ireq_enqueue_scrub": 0, 2024-03-21T10:36:18.401 INFO:teuthology.orchestra.run.smithi157.stdout: "ireq_exportdir": 0, 2024-03-21T10:36:18.401 INFO:teuthology.orchestra.run.smithi157.stdout: "ireq_flush": 0, 2024-03-21T10:36:18.401 INFO:teuthology.orchestra.run.smithi157.stdout: "ireq_fragmentdir": 0, 2024-03-21T10:36:18.401 INFO:teuthology.orchestra.run.smithi157.stdout: "ireq_fragstats": 0, 2024-03-21T10:36:18.401 INFO:teuthology.orchestra.run.smithi157.stdout: "ireq_inodestats": 0 2024-03-21T10:36:18.401 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-03-21T10:36:18.401 INFO:teuthology.orchestra.run.smithi157.stdout: "mds_log": { 2024-03-21T10:36:18.401 INFO:teuthology.orchestra.run.smithi157.stdout: "evlrg": 0, 2024-03-21T10:36:18.401 INFO:teuthology.orchestra.run.smithi157.stdout: "evadd": 0, 2024-03-21T10:36:18.401 INFO:teuthology.orchestra.run.smithi157.stdout: "evex": 0, 2024-03-21T10:36:18.401 INFO:teuthology.orchestra.run.smithi157.stdout: "evtrm": 0, 2024-03-21T10:36:18.401 INFO:teuthology.orchestra.run.smithi157.stdout: "ev": 8, 2024-03-21T10:36:18.402 INFO:teuthology.orchestra.run.smithi157.stdout: "evexg": 0, 2024-03-21T10:36:18.402 INFO:teuthology.orchestra.run.smithi157.stdout: "evexd": 0, 2024-03-21T10:36:18.402 INFO:teuthology.orchestra.run.smithi157.stdout: "segadd": 0, 2024-03-21T10:36:18.402 INFO:teuthology.orchestra.run.smithi157.stdout: "segex": 0, 2024-03-21T10:36:18.402 INFO:teuthology.orchestra.run.smithi157.stdout: "segtrm": 0, 2024-03-21T10:36:18.402 INFO:teuthology.orchestra.run.smithi157.stdout: "seg": 2, 2024-03-21T10:36:18.402 INFO:teuthology.orchestra.run.smithi157.stdout: "segmjr": 2, 2024-03-21T10:36:18.402 INFO:teuthology.orchestra.run.smithi157.stdout: "segexg": 0, 2024-03-21T10:36:18.402 INFO:teuthology.orchestra.run.smithi157.stdout: "segexd": 0, 2024-03-21T10:36:18.402 INFO:teuthology.orchestra.run.smithi157.stdout: "expos": 4194304, 2024-03-21T10:36:18.402 INFO:teuthology.orchestra.run.smithi157.stdout: "wrpos": 4196840, 2024-03-21T10:36:18.402 INFO:teuthology.orchestra.run.smithi157.stdout: "rdpos": 4196699, 2024-03-21T10:36:18.402 INFO:teuthology.orchestra.run.smithi157.stdout: "jlat": { 2024-03-21T10:36:18.402 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-03-21T10:36:18.403 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-03-21T10:36:18.403 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-03-21T10:36:18.403 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-03-21T10:36:18.403 INFO:teuthology.orchestra.run.smithi157.stdout: "replayed": 8 2024-03-21T10:36:18.403 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-03-21T10:36:18.403 INFO:teuthology.orchestra.run.smithi157.stdout: "mds_mem": { 2024-03-21T10:36:18.403 INFO:teuthology.orchestra.run.smithi157.stdout: "ino": 3, 2024-03-21T10:36:18.403 INFO:teuthology.orchestra.run.smithi157.stdout: "ino+": 3, 2024-03-21T10:36:18.403 INFO:teuthology.orchestra.run.smithi157.stdout: "ino-": 0, 2024-03-21T10:36:18.403 INFO:teuthology.orchestra.run.smithi157.stdout: "dir": 1, 2024-03-21T10:36:18.403 INFO:teuthology.orchestra.run.smithi157.stdout: "dir+": 1, 2024-03-21T10:36:18.403 INFO:teuthology.orchestra.run.smithi157.stdout: "dir-": 0, 2024-03-21T10:36:18.403 INFO:teuthology.orchestra.run.smithi157.stdout: "dn": 0, 2024-03-21T10:36:18.403 INFO:teuthology.orchestra.run.smithi157.stdout: "dn+": 0, 2024-03-21T10:36:18.403 INFO:teuthology.orchestra.run.smithi157.stdout: "dn-": 0, 2024-03-21T10:36:18.403 INFO:teuthology.orchestra.run.smithi157.stdout: "cap": 0, 2024-03-21T10:36:18.404 INFO:teuthology.orchestra.run.smithi157.stdout: "cap+": 0, 2024-03-21T10:36:18.404 INFO:teuthology.orchestra.run.smithi157.stdout: "cap-": 0, 2024-03-21T10:36:18.404 INFO:teuthology.orchestra.run.smithi157.stdout: "rss": 48768, 2024-03-21T10:36:18.404 INFO:teuthology.orchestra.run.smithi157.stdout: "heap": 215420 2024-03-21T10:36:18.404 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-03-21T10:36:18.404 INFO:teuthology.orchestra.run.smithi157.stdout: "mds_server": { 2024-03-21T10:36:18.404 INFO:teuthology.orchestra.run.smithi157.stdout: "dispatch_client_request": 0, 2024-03-21T10:36:18.404 INFO:teuthology.orchestra.run.smithi157.stdout: "dispatch_server_request": 0, 2024-03-21T10:36:18.404 INFO:teuthology.orchestra.run.smithi157.stdout: "handle_client_request": 0, 2024-03-21T10:36:18.404 INFO:teuthology.orchestra.run.smithi157.stdout: "handle_client_session": 0, 2024-03-21T10:36:18.404 INFO:teuthology.orchestra.run.smithi157.stdout: "handle_peer_request": 0, 2024-03-21T10:36:18.404 INFO:teuthology.orchestra.run.smithi157.stdout: "req_create_latency": { 2024-03-21T10:36:18.404 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-03-21T10:36:18.404 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-03-21T10:36:18.404 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-03-21T10:36:18.404 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-03-21T10:36:18.405 INFO:teuthology.orchestra.run.smithi157.stdout: "req_getattr_latency": { 2024-03-21T10:36:18.405 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-03-21T10:36:18.405 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-03-21T10:36:18.405 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-03-21T10:36:18.405 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-03-21T10:36:18.405 INFO:teuthology.orchestra.run.smithi157.stdout: "req_getfilelock_latency": { 2024-03-21T10:36:18.405 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-03-21T10:36:18.405 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-03-21T10:36:18.405 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-03-21T10:36:18.405 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-03-21T10:36:18.405 INFO:teuthology.orchestra.run.smithi157.stdout: "req_link_latency": { 2024-03-21T10:36:18.405 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-03-21T10:36:18.405 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-03-21T10:36:18.405 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-03-21T10:36:18.406 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-03-21T10:36:18.406 INFO:teuthology.orchestra.run.smithi157.stdout: "req_lookup_latency": { 2024-03-21T10:36:18.406 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-03-21T10:36:18.406 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-03-21T10:36:18.406 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-03-21T10:36:18.406 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-03-21T10:36:18.406 INFO:teuthology.orchestra.run.smithi157.stdout: "req_lookuphash_latency": { 2024-03-21T10:36:18.406 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-03-21T10:36:18.406 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-03-21T10:36:18.406 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-03-21T10:36:18.406 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-03-21T10:36:18.406 INFO:teuthology.orchestra.run.smithi157.stdout: "req_lookupino_latency": { 2024-03-21T10:36:18.406 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-03-21T10:36:18.407 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-03-21T10:36:18.407 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-03-21T10:36:18.407 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-03-21T10:36:18.407 INFO:teuthology.orchestra.run.smithi157.stdout: "req_lookupname_latency": { 2024-03-21T10:36:18.407 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-03-21T10:36:18.407 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-03-21T10:36:18.407 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-03-21T10:36:18.407 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-03-21T10:36:18.407 INFO:teuthology.orchestra.run.smithi157.stdout: "req_lookupparent_latency": { 2024-03-21T10:36:18.407 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-03-21T10:36:18.407 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-03-21T10:36:18.407 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-03-21T10:36:18.407 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-03-21T10:36:18.407 INFO:teuthology.orchestra.run.smithi157.stdout: "req_lookupsnap_latency": { 2024-03-21T10:36:18.407 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-03-21T10:36:18.407 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-03-21T10:36:18.408 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-03-21T10:36:18.408 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-03-21T10:36:18.408 INFO:teuthology.orchestra.run.smithi157.stdout: "req_lssnap_latency": { 2024-03-21T10:36:18.408 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-03-21T10:36:18.408 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-03-21T10:36:18.408 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-03-21T10:36:18.408 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-03-21T10:36:18.408 INFO:teuthology.orchestra.run.smithi157.stdout: "req_mkdir_latency": { 2024-03-21T10:36:18.408 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-03-21T10:36:18.408 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-03-21T10:36:18.408 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-03-21T10:36:18.408 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-03-21T10:36:18.409 INFO:teuthology.orchestra.run.smithi157.stdout: "req_mknod_latency": { 2024-03-21T10:36:18.409 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-03-21T10:36:18.409 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-03-21T10:36:18.409 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-03-21T10:36:18.409 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-03-21T10:36:18.409 INFO:teuthology.orchestra.run.smithi157.stdout: "req_mksnap_latency": { 2024-03-21T10:36:18.409 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-03-21T10:36:18.409 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-03-21T10:36:18.410 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-03-21T10:36:18.410 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-03-21T10:36:18.410 INFO:teuthology.orchestra.run.smithi157.stdout: "req_open_latency": { 2024-03-21T10:36:18.410 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-03-21T10:36:18.410 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-03-21T10:36:18.410 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-03-21T10:36:18.410 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-03-21T10:36:18.410 INFO:teuthology.orchestra.run.smithi157.stdout: "req_readdir_latency": { 2024-03-21T10:36:18.410 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-03-21T10:36:18.410 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-03-21T10:36:18.410 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-03-21T10:36:18.410 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-03-21T10:36:18.411 INFO:teuthology.orchestra.run.smithi157.stdout: "req_rename_latency": { 2024-03-21T10:36:18.411 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-03-21T10:36:18.411 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-03-21T10:36:18.411 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-03-21T10:36:18.411 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-03-21T10:36:18.411 INFO:teuthology.orchestra.run.smithi157.stdout: "req_renamesnap_latency": { 2024-03-21T10:36:18.411 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-03-21T10:36:18.411 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-03-21T10:36:18.411 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-03-21T10:36:18.411 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-03-21T10:36:18.411 INFO:teuthology.orchestra.run.smithi157.stdout: "req_snapdiff_latency": { 2024-03-21T10:36:18.411 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-03-21T10:36:18.411 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-03-21T10:36:18.411 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-03-21T10:36:18.411 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-03-21T10:36:18.412 INFO:teuthology.orchestra.run.smithi157.stdout: "req_rmdir_latency": { 2024-03-21T10:36:18.412 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-03-21T10:36:18.412 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-03-21T10:36:18.412 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-03-21T10:36:18.412 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-03-21T10:36:18.412 INFO:teuthology.orchestra.run.smithi157.stdout: "req_rmsnap_latency": { 2024-03-21T10:36:18.412 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-03-21T10:36:18.412 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-03-21T10:36:18.412 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-03-21T10:36:18.412 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-03-21T10:36:18.412 INFO:teuthology.orchestra.run.smithi157.stdout: "req_rmxattr_latency": { 2024-03-21T10:36:18.412 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-03-21T10:36:18.412 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-03-21T10:36:18.412 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-03-21T10:36:18.413 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-03-21T10:36:18.413 INFO:teuthology.orchestra.run.smithi157.stdout: "req_setattr_latency": { 2024-03-21T10:36:18.413 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-03-21T10:36:18.413 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-03-21T10:36:18.413 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-03-21T10:36:18.413 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-03-21T10:36:18.413 INFO:teuthology.orchestra.run.smithi157.stdout: "req_setdirlayout_latency": { 2024-03-21T10:36:18.413 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-03-21T10:36:18.413 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-03-21T10:36:18.413 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-03-21T10:36:18.413 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-03-21T10:36:18.413 INFO:teuthology.orchestra.run.smithi157.stdout: "req_setfilelock_latency": { 2024-03-21T10:36:18.413 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-03-21T10:36:18.414 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-03-21T10:36:18.414 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-03-21T10:36:18.414 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-03-21T10:36:18.414 INFO:teuthology.orchestra.run.smithi157.stdout: "req_setlayout_latency": { 2024-03-21T10:36:18.414 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-03-21T10:36:18.414 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-03-21T10:36:18.414 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-03-21T10:36:18.414 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-03-21T10:36:18.414 INFO:teuthology.orchestra.run.smithi157.stdout: "req_setxattr_latency": { 2024-03-21T10:36:18.414 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-03-21T10:36:18.414 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-03-21T10:36:18.414 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-03-21T10:36:18.414 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-03-21T10:36:18.414 INFO:teuthology.orchestra.run.smithi157.stdout: "req_symlink_latency": { 2024-03-21T10:36:18.414 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-03-21T10:36:18.415 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-03-21T10:36:18.415 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-03-21T10:36:18.415 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-03-21T10:36:18.415 INFO:teuthology.orchestra.run.smithi157.stdout: "req_unlink_latency": { 2024-03-21T10:36:18.415 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-03-21T10:36:18.415 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-03-21T10:36:18.415 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-03-21T10:36:18.415 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-03-21T10:36:18.415 INFO:teuthology.orchestra.run.smithi157.stdout: "cap_revoke_eviction": 0, 2024-03-21T10:36:18.415 INFO:teuthology.orchestra.run.smithi157.stdout: "cap_acquisition_throttle": 0, 2024-03-21T10:36:18.415 INFO:teuthology.orchestra.run.smithi157.stdout: "req_getvxattr_latency": { 2024-03-21T10:36:18.415 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-03-21T10:36:18.415 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-03-21T10:36:18.416 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-03-21T10:36:18.416 INFO:teuthology.orchestra.run.smithi157.stdout: } 2024-03-21T10:36:18.416 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-03-21T10:36:18.416 INFO:teuthology.orchestra.run.smithi157.stdout: "mds_sessions": { 2024-03-21T10:36:18.416 INFO:teuthology.orchestra.run.smithi157.stdout: "session_count": 2, 2024-03-21T10:36:18.416 INFO:teuthology.orchestra.run.smithi157.stdout: "session_add": 4, 2024-03-21T10:36:18.416 INFO:teuthology.orchestra.run.smithi157.stdout: "session_remove": 2, 2024-03-21T10:36:18.416 INFO:teuthology.orchestra.run.smithi157.stdout: "sessions_open": 4, 2024-03-21T10:36:18.416 INFO:teuthology.orchestra.run.smithi157.stdout: "sessions_stale": 0, 2024-03-21T10:36:18.416 INFO:teuthology.orchestra.run.smithi157.stdout: "total_load": 0, 2024-03-21T10:36:18.416 INFO:teuthology.orchestra.run.smithi157.stdout: "average_load": 0, 2024-03-21T10:36:18.416 INFO:teuthology.orchestra.run.smithi157.stdout: "avg_session_uptime": 3601, 2024-03-21T10:36:18.416 INFO:teuthology.orchestra.run.smithi157.stdout: "mdthresh_evicted": 0 2024-03-21T10:36:18.416 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-03-21T10:36:18.416 INFO:teuthology.orchestra.run.smithi157.stdout: "mempool": { 2024-03-21T10:36:18.417 INFO:teuthology.orchestra.run.smithi157.stdout: "bloom_filter_bytes": 0, 2024-03-21T10:36:18.417 INFO:teuthology.orchestra.run.smithi157.stdout: "bloom_filter_items": 0, 2024-03-21T10:36:18.417 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_alloc_bytes": 0, 2024-03-21T10:36:18.417 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_alloc_items": 0, 2024-03-21T10:36:18.417 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_cache_data_bytes": 0, 2024-03-21T10:36:18.417 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_cache_data_items": 0, 2024-03-21T10:36:18.417 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_cache_onode_bytes": 0, 2024-03-21T10:36:18.417 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_cache_onode_items": 0, 2024-03-21T10:36:18.417 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_cache_meta_bytes": 0, 2024-03-21T10:36:18.417 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_cache_meta_items": 0, 2024-03-21T10:36:18.417 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_cache_other_bytes": 0, 2024-03-21T10:36:18.417 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_cache_other_items": 0, 2024-03-21T10:36:18.417 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_cache_buffer_bytes": 0, 2024-03-21T10:36:18.417 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_cache_buffer_items": 0, 2024-03-21T10:36:18.417 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_extent_bytes": 0, 2024-03-21T10:36:18.418 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_extent_items": 0, 2024-03-21T10:36:18.418 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_blob_bytes": 0, 2024-03-21T10:36:18.418 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_blob_items": 0, 2024-03-21T10:36:18.418 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_shared_blob_bytes": 0, 2024-03-21T10:36:18.418 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_shared_blob_items": 0, 2024-03-21T10:36:18.418 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_inline_bl_bytes": 0, 2024-03-21T10:36:18.418 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_inline_bl_items": 0, 2024-03-21T10:36:18.418 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_fsck_bytes": 0, 2024-03-21T10:36:18.418 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_fsck_items": 0, 2024-03-21T10:36:18.418 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_txc_bytes": 0, 2024-03-21T10:36:18.418 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_txc_items": 0, 2024-03-21T10:36:18.418 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_writing_deferred_bytes": 0, 2024-03-21T10:36:18.418 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_writing_deferred_items": 0, 2024-03-21T10:36:18.418 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_writing_bytes": 0, 2024-03-21T10:36:18.419 INFO:teuthology.orchestra.run.smithi157.stdout: "bluestore_writing_items": 0, 2024-03-21T10:36:18.419 INFO:teuthology.orchestra.run.smithi157.stdout: "bluefs_bytes": 0, 2024-03-21T10:36:18.419 INFO:teuthology.orchestra.run.smithi157.stdout: "bluefs_items": 0, 2024-03-21T10:36:18.419 INFO:teuthology.orchestra.run.smithi157.stdout: "bluefs_file_reader_bytes": 0, 2024-03-21T10:36:18.419 INFO:teuthology.orchestra.run.smithi157.stdout: "bluefs_file_reader_items": 0, 2024-03-21T10:36:18.419 INFO:teuthology.orchestra.run.smithi157.stdout: "bluefs_file_writer_bytes": 0, 2024-03-21T10:36:18.419 INFO:teuthology.orchestra.run.smithi157.stdout: "bluefs_file_writer_items": 0, 2024-03-21T10:36:18.419 INFO:teuthology.orchestra.run.smithi157.stdout: "buffer_anon_bytes": 198337, 2024-03-21T10:36:18.419 INFO:teuthology.orchestra.run.smithi157.stdout: "buffer_anon_items": 59, 2024-03-21T10:36:18.419 INFO:teuthology.orchestra.run.smithi157.stdout: "buffer_meta_bytes": 0, 2024-03-21T10:36:18.419 INFO:teuthology.orchestra.run.smithi157.stdout: "buffer_meta_items": 0, 2024-03-21T10:36:18.419 INFO:teuthology.orchestra.run.smithi157.stdout: "osd_bytes": 0, 2024-03-21T10:36:18.419 INFO:teuthology.orchestra.run.smithi157.stdout: "osd_items": 0, 2024-03-21T10:36:18.419 INFO:teuthology.orchestra.run.smithi157.stdout: "osd_mapbl_bytes": 0, 2024-03-21T10:36:18.419 INFO:teuthology.orchestra.run.smithi157.stdout: "osd_mapbl_items": 0, 2024-03-21T10:36:18.420 INFO:teuthology.orchestra.run.smithi157.stdout: "osd_pglog_bytes": 0, 2024-03-21T10:36:18.420 INFO:teuthology.orchestra.run.smithi157.stdout: "osd_pglog_items": 0, 2024-03-21T10:36:18.420 INFO:teuthology.orchestra.run.smithi157.stdout: "osdmap_bytes": 9712, 2024-03-21T10:36:18.420 INFO:teuthology.orchestra.run.smithi157.stdout: "osdmap_items": 187, 2024-03-21T10:36:18.420 INFO:teuthology.orchestra.run.smithi157.stdout: "osdmap_mapping_bytes": 0, 2024-03-21T10:36:18.420 INFO:teuthology.orchestra.run.smithi157.stdout: "osdmap_mapping_items": 0, 2024-03-21T10:36:18.420 INFO:teuthology.orchestra.run.smithi157.stdout: "pgmap_bytes": 0, 2024-03-21T10:36:18.420 INFO:teuthology.orchestra.run.smithi157.stdout: "pgmap_items": 0, 2024-03-21T10:36:18.420 INFO:teuthology.orchestra.run.smithi157.stdout: "mds_co_bytes": 8320, 2024-03-21T10:36:18.420 INFO:teuthology.orchestra.run.smithi157.stdout: "mds_co_items": 13, 2024-03-21T10:36:18.420 INFO:teuthology.orchestra.run.smithi157.stdout: "unittest_1_bytes": 0, 2024-03-21T10:36:18.420 INFO:teuthology.orchestra.run.smithi157.stdout: "unittest_1_items": 0, 2024-03-21T10:36:18.420 INFO:teuthology.orchestra.run.smithi157.stdout: "unittest_2_bytes": 0, 2024-03-21T10:36:18.420 INFO:teuthology.orchestra.run.smithi157.stdout: "unittest_2_items": 0 2024-03-21T10:36:18.420 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-03-21T10:36:18.420 INFO:teuthology.orchestra.run.smithi157.stdout: "objecter": { 2024-03-21T10:36:18.420 INFO:teuthology.orchestra.run.smithi157.stdout: "op_active": 0, 2024-03-21T10:36:18.421 INFO:teuthology.orchestra.run.smithi157.stdout: "op_laggy": 0, 2024-03-21T10:36:18.421 INFO:teuthology.orchestra.run.smithi157.stdout: "op_send": 2346, 2024-03-21T10:36:18.421 INFO:teuthology.orchestra.run.smithi157.stdout: "op_send_bytes": 21, 2024-03-21T10:36:18.421 INFO:teuthology.orchestra.run.smithi157.stdout: "op_resend": 3, 2024-03-21T10:36:18.421 INFO:teuthology.orchestra.run.smithi157.stdout: "op_reply": 2346, 2024-03-21T10:36:18.421 INFO:teuthology.orchestra.run.smithi157.stdout: "op_latency": { 2024-03-21T10:36:18.421 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 2346, 2024-03-21T10:36:18.421 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 2.406956228, 2024-03-21T10:36:18.421 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.001025983 2024-03-21T10:36:18.421 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-03-21T10:36:18.421 INFO:teuthology.orchestra.run.smithi157.stdout: "op_inflight": 0, 2024-03-21T10:36:18.421 INFO:teuthology.orchestra.run.smithi157.stdout: "oplen_avg": { 2024-03-21T10:36:18.421 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 2346, 2024-03-21T10:36:18.421 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 2347 2024-03-21T10:36:18.421 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-03-21T10:36:18.422 INFO:teuthology.orchestra.run.smithi157.stdout: "op": 2346, 2024-03-21T10:36:18.422 INFO:teuthology.orchestra.run.smithi157.stdout: "op_r": 2346, 2024-03-21T10:36:18.422 INFO:teuthology.orchestra.run.smithi157.stdout: "op_w": 0, 2024-03-21T10:36:18.422 INFO:teuthology.orchestra.run.smithi157.stdout: "op_rmw": 0, 2024-03-21T10:36:18.422 INFO:teuthology.orchestra.run.smithi157.stdout: "op_pg": 0, 2024-03-21T10:36:18.422 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_stat": 1508, 2024-03-21T10:36:18.422 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_create": 0, 2024-03-21T10:36:18.422 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_read": 836, 2024-03-21T10:36:18.422 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_write": 0, 2024-03-21T10:36:18.422 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_writefull": 0, 2024-03-21T10:36:18.422 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_writesame": 0, 2024-03-21T10:36:18.422 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_append": 0, 2024-03-21T10:36:18.422 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_zero": 0, 2024-03-21T10:36:18.422 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_truncate": 0, 2024-03-21T10:36:18.423 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_delete": 0, 2024-03-21T10:36:18.423 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_mapext": 0, 2024-03-21T10:36:18.423 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_sparse_read": 0, 2024-03-21T10:36:18.423 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_clonerange": 0, 2024-03-21T10:36:18.423 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_getxattr": 1, 2024-03-21T10:36:18.423 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_setxattr": 0, 2024-03-21T10:36:18.423 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_cmpxattr": 0, 2024-03-21T10:36:18.423 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_rmxattr": 0, 2024-03-21T10:36:18.423 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_resetxattrs": 0, 2024-03-21T10:36:18.423 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_call": 0, 2024-03-21T10:36:18.423 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_watch": 0, 2024-03-21T10:36:18.423 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_notify": 0, 2024-03-21T10:36:18.423 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_src_cmpxattr": 0, 2024-03-21T10:36:18.423 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_pgls": 0, 2024-03-21T10:36:18.423 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_pgls_filter": 0, 2024-03-21T10:36:18.423 INFO:teuthology.orchestra.run.smithi157.stdout: "osdop_other": 0, 2024-03-21T10:36:18.424 INFO:teuthology.orchestra.run.smithi157.stdout: "linger_active": 0, 2024-03-21T10:36:18.424 INFO:teuthology.orchestra.run.smithi157.stdout: "linger_send": 0, 2024-03-21T10:36:18.424 INFO:teuthology.orchestra.run.smithi157.stdout: "linger_resend": 0, 2024-03-21T10:36:18.424 INFO:teuthology.orchestra.run.smithi157.stdout: "linger_ping": 0, 2024-03-21T10:36:18.424 INFO:teuthology.orchestra.run.smithi157.stdout: "poolop_active": 0, 2024-03-21T10:36:18.424 INFO:teuthology.orchestra.run.smithi157.stdout: "poolop_send": 0, 2024-03-21T10:36:18.424 INFO:teuthology.orchestra.run.smithi157.stdout: "poolop_resend": 0, 2024-03-21T10:36:18.424 INFO:teuthology.orchestra.run.smithi157.stdout: "poolstat_active": 0, 2024-03-21T10:36:18.424 INFO:teuthology.orchestra.run.smithi157.stdout: "poolstat_send": 0, 2024-03-21T10:36:18.424 INFO:teuthology.orchestra.run.smithi157.stdout: "poolstat_resend": 0, 2024-03-21T10:36:18.424 INFO:teuthology.orchestra.run.smithi157.stdout: "statfs_active": 0, 2024-03-21T10:36:18.424 INFO:teuthology.orchestra.run.smithi157.stdout: "statfs_send": 0, 2024-03-21T10:36:18.424 INFO:teuthology.orchestra.run.smithi157.stdout: "statfs_resend": 0, 2024-03-21T10:36:18.424 INFO:teuthology.orchestra.run.smithi157.stdout: "command_active": 0, 2024-03-21T10:36:18.424 INFO:teuthology.orchestra.run.smithi157.stdout: "command_send": 0, 2024-03-21T10:36:18.424 INFO:teuthology.orchestra.run.smithi157.stdout: "command_resend": 0, 2024-03-21T10:36:18.425 INFO:teuthology.orchestra.run.smithi157.stdout: "map_epoch": 107, 2024-03-21T10:36:18.425 INFO:teuthology.orchestra.run.smithi157.stdout: "map_full": 0, 2024-03-21T10:36:18.425 INFO:teuthology.orchestra.run.smithi157.stdout: "map_inc": 14, 2024-03-21T10:36:18.425 INFO:teuthology.orchestra.run.smithi157.stdout: "osd_sessions": 6, 2024-03-21T10:36:18.425 INFO:teuthology.orchestra.run.smithi157.stdout: "osd_session_open": 6, 2024-03-21T10:36:18.425 INFO:teuthology.orchestra.run.smithi157.stdout: "osd_session_close": 0, 2024-03-21T10:36:18.425 INFO:teuthology.orchestra.run.smithi157.stdout: "osd_laggy": 0, 2024-03-21T10:36:18.425 INFO:teuthology.orchestra.run.smithi157.stdout: "omap_wr": 0, 2024-03-21T10:36:18.425 INFO:teuthology.orchestra.run.smithi157.stdout: "omap_rd": 2, 2024-03-21T10:36:18.425 INFO:teuthology.orchestra.run.smithi157.stdout: "omap_del": 0 2024-03-21T10:36:18.425 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-03-21T10:36:18.425 INFO:teuthology.orchestra.run.smithi157.stdout: "oft": { 2024-03-21T10:36:18.425 INFO:teuthology.orchestra.run.smithi157.stdout: "omap_total_objs": 0, 2024-03-21T10:36:18.425 INFO:teuthology.orchestra.run.smithi157.stdout: "omap_total_kv_pairs": 0, 2024-03-21T10:36:18.425 INFO:teuthology.orchestra.run.smithi157.stdout: "omap_total_updates": 0, 2024-03-21T10:36:18.425 INFO:teuthology.orchestra.run.smithi157.stdout: "omap_total_removes": 0 2024-03-21T10:36:18.425 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-03-21T10:36:18.426 INFO:teuthology.orchestra.run.smithi157.stdout: "osd-slow-ops": { 2024-03-21T10:36:18.426 INFO:teuthology.orchestra.run.smithi157.stdout: "slow_ops_count": 0 2024-03-21T10:36:18.426 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-03-21T10:36:18.426 INFO:teuthology.orchestra.run.smithi157.stdout: "purge_queue": { 2024-03-21T10:36:18.426 INFO:teuthology.orchestra.run.smithi157.stdout: "pq_executing_ops": 0, 2024-03-21T10:36:18.426 INFO:teuthology.orchestra.run.smithi157.stdout: "pq_executing_ops_high_water": 0, 2024-03-21T10:36:18.426 INFO:teuthology.orchestra.run.smithi157.stdout: "pq_executing": 0, 2024-03-21T10:36:18.426 INFO:teuthology.orchestra.run.smithi157.stdout: "pq_executing_high_water": 0, 2024-03-21T10:36:18.426 INFO:teuthology.orchestra.run.smithi157.stdout: "pq_executed_ops": 0, 2024-03-21T10:36:18.426 INFO:teuthology.orchestra.run.smithi157.stdout: "pq_executed": 0, 2024-03-21T10:36:18.426 INFO:teuthology.orchestra.run.smithi157.stdout: "pq_item_in_journal": 0 2024-03-21T10:36:18.426 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-03-21T10:36:18.426 INFO:teuthology.orchestra.run.smithi157.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-03-21T10:36:18.426 INFO:teuthology.orchestra.run.smithi157.stdout: "val": 0, 2024-03-21T10:36:18.427 INFO:teuthology.orchestra.run.smithi157.stdout: "max": 104857600, 2024-03-21T10:36:18.427 INFO:teuthology.orchestra.run.smithi157.stdout: "get_started": 0, 2024-03-21T10:36:18.427 INFO:teuthology.orchestra.run.smithi157.stdout: "get": 2629, 2024-03-21T10:36:18.427 INFO:teuthology.orchestra.run.smithi157.stdout: "get_sum": 2209202, 2024-03-21T10:36:18.427 INFO:teuthology.orchestra.run.smithi157.stdout: "get_or_fail_fail": 0, 2024-03-21T10:36:18.427 INFO:teuthology.orchestra.run.smithi157.stdout: "get_or_fail_success": 2629, 2024-03-21T10:36:18.427 INFO:teuthology.orchestra.run.smithi157.stdout: "take": 0, 2024-03-21T10:36:18.427 INFO:teuthology.orchestra.run.smithi157.stdout: "take_sum": 0, 2024-03-21T10:36:18.427 INFO:teuthology.orchestra.run.smithi157.stdout: "put": 2629, 2024-03-21T10:36:18.427 INFO:teuthology.orchestra.run.smithi157.stdout: "put_sum": 2209202, 2024-03-21T10:36:18.427 INFO:teuthology.orchestra.run.smithi157.stdout: "wait": { 2024-03-21T10:36:18.427 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-03-21T10:36:18.427 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-03-21T10:36:18.427 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-03-21T10:36:18.427 INFO:teuthology.orchestra.run.smithi157.stdout: } 2024-03-21T10:36:18.427 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-03-21T10:36:18.427 INFO:teuthology.orchestra.run.smithi157.stdout: "throttle-objecter_bytes": { 2024-03-21T10:36:18.428 INFO:teuthology.orchestra.run.smithi157.stdout: "val": 0, 2024-03-21T10:36:18.428 INFO:teuthology.orchestra.run.smithi157.stdout: "max": 104857600, 2024-03-21T10:36:18.428 INFO:teuthology.orchestra.run.smithi157.stdout: "get_started": 0, 2024-03-21T10:36:18.428 INFO:teuthology.orchestra.run.smithi157.stdout: "get": 0, 2024-03-21T10:36:18.428 INFO:teuthology.orchestra.run.smithi157.stdout: "get_sum": 0, 2024-03-21T10:36:18.428 INFO:teuthology.orchestra.run.smithi157.stdout: "get_or_fail_fail": 0, 2024-03-21T10:36:18.428 INFO:teuthology.orchestra.run.smithi157.stdout: "get_or_fail_success": 0, 2024-03-21T10:36:18.428 INFO:teuthology.orchestra.run.smithi157.stdout: "take": 2346, 2024-03-21T10:36:18.428 INFO:teuthology.orchestra.run.smithi157.stdout: "take_sum": 2541, 2024-03-21T10:36:18.428 INFO:teuthology.orchestra.run.smithi157.stdout: "put": 2346, 2024-03-21T10:36:18.428 INFO:teuthology.orchestra.run.smithi157.stdout: "put_sum": 2541, 2024-03-21T10:36:18.428 INFO:teuthology.orchestra.run.smithi157.stdout: "wait": { 2024-03-21T10:36:18.428 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-03-21T10:36:18.428 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-03-21T10:36:18.428 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-03-21T10:36:18.428 INFO:teuthology.orchestra.run.smithi157.stdout: } 2024-03-21T10:36:18.429 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-03-21T10:36:18.429 INFO:teuthology.orchestra.run.smithi157.stdout: "throttle-objecter_ops": { 2024-03-21T10:36:18.429 INFO:teuthology.orchestra.run.smithi157.stdout: "val": 0, 2024-03-21T10:36:18.429 INFO:teuthology.orchestra.run.smithi157.stdout: "max": 1024, 2024-03-21T10:36:18.429 INFO:teuthology.orchestra.run.smithi157.stdout: "get_started": 0, 2024-03-21T10:36:18.429 INFO:teuthology.orchestra.run.smithi157.stdout: "get": 0, 2024-03-21T10:36:18.429 INFO:teuthology.orchestra.run.smithi157.stdout: "get_sum": 0, 2024-03-21T10:36:18.429 INFO:teuthology.orchestra.run.smithi157.stdout: "get_or_fail_fail": 0, 2024-03-21T10:36:18.429 INFO:teuthology.orchestra.run.smithi157.stdout: "get_or_fail_success": 0, 2024-03-21T10:36:18.429 INFO:teuthology.orchestra.run.smithi157.stdout: "take": 2346, 2024-03-21T10:36:18.429 INFO:teuthology.orchestra.run.smithi157.stdout: "take_sum": 2346, 2024-03-21T10:36:18.429 INFO:teuthology.orchestra.run.smithi157.stdout: "put": 2346, 2024-03-21T10:36:18.429 INFO:teuthology.orchestra.run.smithi157.stdout: "put_sum": 2346, 2024-03-21T10:36:18.429 INFO:teuthology.orchestra.run.smithi157.stdout: "wait": { 2024-03-21T10:36:18.429 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-03-21T10:36:18.429 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-03-21T10:36:18.430 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-03-21T10:36:18.430 INFO:teuthology.orchestra.run.smithi157.stdout: } 2024-03-21T10:36:18.430 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-03-21T10:36:18.430 INFO:teuthology.orchestra.run.smithi157.stdout: "throttle-write_buf_throttle": { 2024-03-21T10:36:18.430 INFO:teuthology.orchestra.run.smithi157.stdout: "val": 0, 2024-03-21T10:36:18.430 INFO:teuthology.orchestra.run.smithi157.stdout: "max": 3758096384, 2024-03-21T10:36:18.430 INFO:teuthology.orchestra.run.smithi157.stdout: "get_started": 0, 2024-03-21T10:36:18.430 INFO:teuthology.orchestra.run.smithi157.stdout: "get": 0, 2024-03-21T10:36:18.430 INFO:teuthology.orchestra.run.smithi157.stdout: "get_sum": 0, 2024-03-21T10:36:18.430 INFO:teuthology.orchestra.run.smithi157.stdout: "get_or_fail_fail": 0, 2024-03-21T10:36:18.430 INFO:teuthology.orchestra.run.smithi157.stdout: "get_or_fail_success": 0, 2024-03-21T10:36:18.430 INFO:teuthology.orchestra.run.smithi157.stdout: "take": 0, 2024-03-21T10:36:18.430 INFO:teuthology.orchestra.run.smithi157.stdout: "take_sum": 0, 2024-03-21T10:36:18.430 INFO:teuthology.orchestra.run.smithi157.stdout: "put": 0, 2024-03-21T10:36:18.430 INFO:teuthology.orchestra.run.smithi157.stdout: "put_sum": 0, 2024-03-21T10:36:18.430 INFO:teuthology.orchestra.run.smithi157.stdout: "wait": { 2024-03-21T10:36:18.431 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-03-21T10:36:18.431 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-03-21T10:36:18.431 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-03-21T10:36:18.431 INFO:teuthology.orchestra.run.smithi157.stdout: } 2024-03-21T10:36:18.431 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-03-21T10:36:18.431 INFO:teuthology.orchestra.run.smithi157.stdout: "throttle-write_buf_throttle-0x5652c72ea780": { 2024-03-21T10:36:18.431 INFO:teuthology.orchestra.run.smithi157.stdout: "val": 0, 2024-03-21T10:36:18.431 INFO:teuthology.orchestra.run.smithi157.stdout: "max": 3758096384, 2024-03-21T10:36:18.431 INFO:teuthology.orchestra.run.smithi157.stdout: "get_started": 0, 2024-03-21T10:36:18.431 INFO:teuthology.orchestra.run.smithi157.stdout: "get": 0, 2024-03-21T10:36:18.431 INFO:teuthology.orchestra.run.smithi157.stdout: "get_sum": 0, 2024-03-21T10:36:18.431 INFO:teuthology.orchestra.run.smithi157.stdout: "get_or_fail_fail": 0, 2024-03-21T10:36:18.431 INFO:teuthology.orchestra.run.smithi157.stdout: "get_or_fail_success": 0, 2024-03-21T10:36:18.431 INFO:teuthology.orchestra.run.smithi157.stdout: "take": 0, 2024-03-21T10:36:18.431 INFO:teuthology.orchestra.run.smithi157.stdout: "take_sum": 0, 2024-03-21T10:36:18.432 INFO:teuthology.orchestra.run.smithi157.stdout: "put": 0, 2024-03-21T10:36:18.432 INFO:teuthology.orchestra.run.smithi157.stdout: "put_sum": 0, 2024-03-21T10:36:18.432 INFO:teuthology.orchestra.run.smithi157.stdout: "wait": { 2024-03-21T10:36:18.432 INFO:teuthology.orchestra.run.smithi157.stdout: "avgcount": 0, 2024-03-21T10:36:18.432 INFO:teuthology.orchestra.run.smithi157.stdout: "sum": 0.000000000, 2024-03-21T10:36:18.432 INFO:teuthology.orchestra.run.smithi157.stdout: "avgtime": 0.000000000 2024-03-21T10:36:18.432 INFO:teuthology.orchestra.run.smithi157.stdout: } 2024-03-21T10:36:18.432 INFO:teuthology.orchestra.run.smithi157.stdout: } 2024-03-21T10:36:18.432 INFO:teuthology.orchestra.run.smithi157.stdout:} 2024-03-21T10:36:19.316 INFO:tasks.check_counter:Daemon mds.l mds.exported=0 2024-03-21T10:36:19.316 INFO:tasks.check_counter:Daemon mds.l mds.imported=0 2024-03-21T10:36:19.316 INFO:tasks.check_counter:Daemon mds.l mds.root_rsnaps=0 2024-03-21T10:36:19.316 INFO:tasks.check_counter:Daemon mds.l mds_server.req_mksnap_latency.avgcount=0 2024-03-21T10:36:19.317 INFO:tasks.check_counter:Daemon mds.l mds_server.req_rmsnap_latency.avgcount=0 2024-03-21T10:36:19.317 DEBUG:teuthology.run_tasks:Unwinding manager ceph-fuse 2024-03-21T10:36:19.326 INFO:tasks.ceph_fuse:Unmounting ceph-fuse clients... 2024-03-21T10:36:19.327 DEBUG:teuthology.orchestra.run.smithi047:> set -ex 2024-03-21T10:36:19.327 DEBUG:teuthology.orchestra.run.smithi047:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-21T10:36:19.385 DEBUG:teuthology.orchestra.run.smithi047:> set -ex 2024-03-21T10:36:19.385 DEBUG:teuthology.orchestra.run.smithi047:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-21T10:36:19.432 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-21T10:36:19.707 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:19.708+0000 7efe36000640 1 -- 172.21.15.47:0/1583071666 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7efe300743f0 msgr2=0x7efe3010d790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:36:19.707 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:19.708+0000 7efe36000640 1 --2- 172.21.15.47:0/1583071666 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7efe300743f0 0x7efe3010d790 secure :-1 s=READY pgs=1329 cs=0 l=1 rev1=1 crypto rx=0x7efe20009900 tx=0x7efe2002bcc0 comp rx=0 tx=0).stop 2024-03-21T10:36:19.707 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:19.708+0000 7efe36000640 1 -- 172.21.15.47:0/1583071666 shutdown_connections 2024-03-21T10:36:19.707 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:19.708+0000 7efe36000640 1 --2- 172.21.15.47:0/1583071666 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7efe3010dcd0 0x7efe30110120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:36:19.707 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:19.708+0000 7efe36000640 1 --2- 172.21.15.47:0/1583071666 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7efe300743f0 0x7efe3010d790 unknown :-1 s=CLOSED pgs=1329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:36:19.707 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:19.708+0000 7efe36000640 1 --2- 172.21.15.47:0/1583071666 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7efe30073ad0 0x7efe30073eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:36:19.707 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:19.708+0000 7efe36000640 1 -- 172.21.15.47:0/1583071666 >> 172.21.15.47:0/1583071666 conn(0x7efe300fcfa0 msgr2=0x7efe300ff3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:36:19.708 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:19.708+0000 7efe36000640 1 -- 172.21.15.47:0/1583071666 shutdown_connections 2024-03-21T10:36:19.708 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:19.708+0000 7efe36000640 1 -- 172.21.15.47:0/1583071666 wait complete. 2024-03-21T10:36:19.708 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:19.709+0000 7efe36000640 1 Processor -- start 2024-03-21T10:36:19.709 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:19.709+0000 7efe36000640 1 -- start start 2024-03-21T10:36:19.709 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:19.710+0000 7efe36000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7efe30073ad0 0x7efe30199eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:36:19.709 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:19.710+0000 7efe36000640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7efe300743f0 0x7efe3019a3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:36:19.709 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:19.710+0000 7efe36000640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7efe3010dcd0 0x7efe3019e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:36:19.709 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:19.710+0000 7efe36000640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe30113930 con 0x7efe300743f0 2024-03-21T10:36:19.709 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:19.710+0000 7efe36000640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe301137b0 con 0x7efe30073ad0 2024-03-21T10:36:19.709 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:19.710+0000 7efe36000640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe30113ab0 con 0x7efe3010dcd0 2024-03-21T10:36:19.710 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:19.710+0000 7efe2fe00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7efe300743f0 0x7efe3019a3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:36:19.710 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:19.710+0000 7efe2fe00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7efe300743f0 0x7efe3019a3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:35554/0 (socket says 172.21.15.47:35554) 2024-03-21T10:36:19.710 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:19.710+0000 7efe2fe00640 1 -- 172.21.15.47:0/1432980451 learned_addr learned my addr 172.21.15.47:0/1432980451 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:36:19.710 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:19.710+0000 7efe34c00640 1 --2- 172.21.15.47:0/1432980451 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7efe30073ad0 0x7efe30199eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:36:19.710 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:19.710+0000 7efe35600640 1 --2- 172.21.15.47:0/1432980451 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7efe3010dcd0 0x7efe3019e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:36:19.710 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:19.711+0000 7efe2fe00640 1 -- 172.21.15.47:0/1432980451 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7efe3010dcd0 msgr2=0x7efe3019e770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:36:19.711 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:19.711+0000 7efe2fe00640 1 --2- 172.21.15.47:0/1432980451 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7efe3010dcd0 0x7efe3019e770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:36:19.711 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:19.711+0000 7efe2fe00640 1 -- 172.21.15.47:0/1432980451 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7efe30073ad0 msgr2=0x7efe30199eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:36:19.711 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:19.711+0000 7efe2fe00640 1 --2- 172.21.15.47:0/1432980451 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7efe30073ad0 0x7efe30199eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:36:19.711 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:19.711+0000 7efe2fe00640 1 -- 172.21.15.47:0/1432980451 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe3019eea0 con 0x7efe300743f0 2024-03-21T10:36:19.711 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:19.711+0000 7efe2fe00640 1 --2- 172.21.15.47:0/1432980451 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7efe300743f0 0x7efe3019a3f0 secure :-1 s=READY pgs=1330 cs=0 l=1 rev1=1 crypto rx=0x7efe20000c00 tx=0x7efe2002d880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:36:19.711 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:36:19 smithi047 ceph-mon[31547]: pgmap v343: 129 pgs: 129 active+clean; 2.5 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 236 B/s wr, 12 op/s 2024-03-21T10:36:19.712 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:19.712+0000 7efe2d600640 1 -- 172.21.15.47:0/1432980451 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe20043020 con 0x7efe300743f0 2024-03-21T10:36:19.712 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:19.712+0000 7efe36000640 1 -- 172.21.15.47:0/1432980451 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efe3019f130 con 0x7efe300743f0 2024-03-21T10:36:19.712 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:19.712+0000 7efe2d600640 1 -- 172.21.15.47:0/1432980451 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7efe2002dd40 con 0x7efe300743f0 2024-03-21T10:36:19.712 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:19.712+0000 7efe36000640 1 -- 172.21.15.47:0/1432980451 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efe301a6a20 con 0x7efe300743f0 2024-03-21T10:36:19.712 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:19.712+0000 7efe2d600640 1 -- 172.21.15.47:0/1432980451 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe20003a80 con 0x7efe300743f0 2024-03-21T10:36:19.712 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:19.713+0000 7efe36000640 1 -- 172.21.15.47:0/1432980451 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efe300683b0 con 0x7efe300743f0 2024-03-21T10:36:19.714 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:19.715+0000 7efe2d600640 1 -- 172.21.15.47:0/1432980451 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 27) v1 ==== 137985+0+0 (secure 0 0 0) 0x7efe20046020 con 0x7efe300743f0 2024-03-21T10:36:19.715 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:19.716+0000 7efe2d600640 1 --2- 172.21.15.47:0/1432980451 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7efe180a3090 0x7efe180a5550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:36:19.715 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:19.717+0000 7efe2d600640 1 -- 172.21.15.47:0/1432980451 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(107..107 src has 1..107) v4 ==== 9562+0+0 (secure 0 0 0) 0x7efe200f1c60 con 0x7efe300743f0 2024-03-21T10:36:19.715 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:19.717+0000 7efe34c00640 1 --2- 172.21.15.47:0/1432980451 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7efe180a3090 0x7efe180a5550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:36:19.716 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:19.718+0000 7efe34c00640 1 --2- 172.21.15.47:0/1432980451 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7efe180a3090 0x7efe180a5550 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7efe2400bbe0 tx=0x7efe2400b390 comp rx=0 tx=0).ready entity=mgr.15687 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:36:19.720 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:19.721+0000 7efe2d600640 1 -- 172.21.15.47:0/1432980451 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7efe2004d020 con 0x7efe300743f0 2024-03-21T10:36:19.726 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:36:19 smithi100 ceph-mon[35193]: pgmap v343: 129 pgs: 129 active+clean; 2.5 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 236 B/s wr, 12 op/s 2024-03-21T10:36:19.736 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:36:19 smithi157 ceph-mon[35581]: pgmap v343: 129 pgs: 129 active+clean; 2.5 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 236 B/s wr, 12 op/s 2024-03-21T10:36:19.923 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:19.924+0000 7efe36000640 1 -- 172.21.15.47:0/1432980451 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7efe301088c0 con 0x7efe300743f0 2024-03-21T10:36:19.925 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:19.927+0000 7efe2d600640 1 -- 172.21.15.47:0/1432980451 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v107) v1 ==== 74+0+25412 (secure 0 0 0) 0x7efe200be3f0 con 0x7efe300743f0 2024-03-21T10:36:19.926 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-21T10:36:19.926 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":107,"fsid":"3056d51a-e76b-11ee-95cd-87774f69a715","created":"2024-03-21T10:11:17.019965+0000","modified":"2024-03-21T10:36:13.887907+0000","last_up_change":"2024-03-21T10:20:52.675328+0000","last_in_change":"2024-03-21T10:20:27.648212+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-21T10:15:40.553114+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-21T10:21:56.399518+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-21T10:21:57.812940+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"106","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":17,"snap_epoch":106,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-21T10:22:01.822568+0000","flags":8197,"flags_names":"hashpspool,ec_overwrites,selfmanaged_snaps","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"106","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":17,"snap_epoch":106,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"b0c7a102-8d89-428b-b793-cbbe67151f50","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6802","nonce":775837313},{"type":"v1","addr":"172.21.15.47:6803","nonce":775837313}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6804","nonce":775837313},{"type":"v1","addr":"172.21.15.47:6805","nonce":775837313}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6808","nonce":775837313},{"type":"v1","addr":"172.21.15.47:6809","nonce":775837313}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6806","nonce":775837313},{"type":"v1","addr":"172.21.15.47:6807","nonce":775837313}]},"public_addr":"172.21.15.47:6803/775837313","cluster_addr":"172.21.15.47:6805/775837313","heartbeat_back_addr":"172.21.15.47:6809/775837313","heartbeat_front_addr":"172.21.15.47:6807/775837313","state":["exists","up"]},{"osd":1,"uuid":"07bb81bb-7cf3-4bbc-ac9e-802f95185d11","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6800","nonce":2923304710},{"type":"v1","addr":"172.21.15.100:6801","nonce":2923304710}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":2923304710},{"type":"v1","addr":"172.21.15.100:6803","nonce":2923304710}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":2923304710},{"type":"v1","addr":"172.21.15.100:6807","nonce":2923304710}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6804","nonce":2923304710},{"type":"v1","addr":"172.21.15.100:6805","nonce":2923304710}]},"public_addr":"172.21.15.100:6801/2923304710","cluster_addr":"172.21.15.100:6803/2923304710","heartbeat_back_addr":"172.21.15.100:6807/2923304710","heartbeat_front_addr":"172.21.15.100:6805/2923304710","state":["exists","up"]},{"osd":2,"uuid":"88ce76b9-6081-4836-aad0-8c249c56e8a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":171377005},{"type":"v1","addr":"172.21.15.157:6801","nonce":171377005}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":171377005},{"type":"v1","addr":"172.21.15.157:6803","nonce":171377005}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":171377005},{"type":"v1","addr":"172.21.15.157:6807","nonce":171377005}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":171377005},{"type":"v1","addr":"172.21.15.157:6805","nonce":171377005}]},"public_addr":"172.21.15.157:6801/171377005","cluster_addr":"172.21.15.157:6803/171377005","heartbeat_back_addr":"172.21.15.157:6807/171377005","heartbeat_front_addr":"172.21.15.157:6805/171377005","state":["exists","up"]},{"osd":3,"uuid":"518aa33d-263f-45e6-aab5-2201ceb40e74","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":27,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6810","nonce":640760101},{"type":"v1","addr":"172.21.15.47:6811","nonce":640760101}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6812","nonce":640760101},{"type":"v1","addr":"172.21.15.47:6813","nonce":640760101}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6816","nonce":640760101},{"type":"v1","addr":"172.21.15.47:6817","nonce":640760101}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6814","nonce":640760101},{"type":"v1","addr":"172.21.15.47:6815","nonce":640760101}]},"public_addr":"172.21.15.47:6811/640760101","cluster_addr":"172.21.15.47:6813/640760101","heartbeat_back_addr":"172.21.15.47:6817/640760101","heartbeat_front_addr":"172.21.15.47:6815/640760101","state":["exists","up"]},{"osd":4,"uuid":"987bde83-ecc6-42e5-bd56-c143b3de4226","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":35,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6808","nonce":2785478639},{"type":"v1","addr":"172.21.15.100:6809","nonce":2785478639}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6810","nonce":2785478639},{"type":"v1","addr":"172.21.15.100:6811","nonce":2785478639}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6814","nonce":2785478639},{"type":"v1","addr":"172.21.15.100:6815","nonce":2785478639}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":2785478639},{"type":"v1","addr":"172.21.15.100:6813","nonce":2785478639}]},"public_addr":"172.21.15.100:6809/2785478639","cluster_addr":"172.21.15.100:6811/2785478639","heartbeat_back_addr":"172.21.15.100:6815/2785478639","heartbeat_front_addr":"172.21.15.100:6813/2785478639","state":["exists","up"]},{"osd":5,"uuid":"012d2188-3a61-4a5f-8d01-2001c15ffbe2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":40,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":1525645410},{"type":"v1","addr":"172.21.15.157:6809","nonce":1525645410}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":1525645410},{"type":"v1","addr":"172.21.15.157:6811","nonce":1525645410}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":1525645410},{"type":"v1","addr":"172.21.15.157:6815","nonce":1525645410}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":1525645410},{"type":"v1","addr":"172.21.15.157:6813","nonce":1525645410}]},"public_addr":"172.21.15.157:6809/1525645410","cluster_addr":"172.21.15.157:6811/1525645410","heartbeat_back_addr":"172.21.15.157:6815/1525645410","heartbeat_front_addr":"172.21.15.157:6813/1525645410","state":["exists","up"]},{"osd":6,"uuid":"c12c14b4-85f0-4d88-937c-de4f556437de","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6818","nonce":2321642790},{"type":"v1","addr":"172.21.15.47:6819","nonce":2321642790}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6820","nonce":2321642790},{"type":"v1","addr":"172.21.15.47:6821","nonce":2321642790}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6824","nonce":2321642790},{"type":"v1","addr":"172.21.15.47:6825","nonce":2321642790}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6822","nonce":2321642790},{"type":"v1","addr":"172.21.15.47:6823","nonce":2321642790}]},"public_addr":"172.21.15.47:6819/2321642790","cluster_addr":"172.21.15.47:6821/2321642790","heartbeat_back_addr":"172.21.15.47:6825/2321642790","heartbeat_front_addr":"172.21.15.47:6823/2321642790","state":["exists","up"]},{"osd":7,"uuid":"3c87c6d0-1d1f-4294-9917-aaaad7dda824","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6816","nonce":4104727330},{"type":"v1","addr":"172.21.15.100:6817","nonce":4104727330}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":4104727330},{"type":"v1","addr":"172.21.15.100:6819","nonce":4104727330}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":4104727330},{"type":"v1","addr":"172.21.15.100:6823","nonce":4104727330}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6820","nonce":4104727330},{"type":"v1","addr":"172.21.15.100:6821","nonce":4104727330}]},"public_addr":"172.21.15.100:6817/4104727330","cluster_addr":"172.21.15.100:6819/4104727330","heartbeat_back_addr":"172.21.15.100:6823/4104727330","heartbeat_front_addr":"172.21.15.100:6821/4104727330","state":["exists","up"]},{"osd":8,"uuid":"502d65cf-53c8-4936-8de4-e11b2a6e1e18","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":61,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":4067715347},{"type":"v1","addr":"172.21.15.157:6817","nonce":4067715347}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":4067715347},{"type":"v1","addr":"172.21.15.157:6819","nonce":4067715347}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":4067715347},{"type":"v1","addr":"172.21.15.157:6823","nonce":4067715347}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":4067715347},{"type":"v1","addr":"172.21.15.157:6821","nonce":4067715347}]},"public_addr":"172.21.15.157:6817/4067715347","cluster_addr":"172.21.15.157:6819/4067715347","heartbeat_back_addr":"172.21.15.157:6823/4067715347","heartbeat_front_addr":"172.21.15.157:6821/4067715347","state":["exists","up"]},{"osd":9,"uuid":"5ea1f5be-8458-4c93-9e61-c2e867a514f3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6826","nonce":1982108738},{"type":"v1","addr":"172.21.15.47:6827","nonce":1982108738}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6828","nonce":1982108738},{"type":"v1","addr":"172.21.15.47:6829","nonce":1982108738}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6832","nonce":1982108738},{"type":"v1","addr":"172.21.15.47:6833","nonce":1982108738}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6830","nonce":1982108738},{"type":"v1","addr":"172.21.15.47:6831","nonce":1982108738}]},"public_addr":"172.21.15.47:6827/1982108738","cluster_addr":"172.21.15.47:6829/1982108738","heartbeat_back_addr":"172.21.15.47:6833/1982108738","heartbeat_front_addr":"172.21.15.47:6831/1982108738","state":["exists","up"]},{"osd":10,"uuid":"2da3da90-a0be-4a54-a9ff-b2d6c9b36870","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":74,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":396391028},{"type":"v1","addr":"172.21.15.100:6825","nonce":396391028}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":396391028},{"type":"v1","addr":"172.21.15.100:6827","nonce":396391028}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6830","nonce":396391028},{"type":"v1","addr":"172.21.15.100:6831","nonce":396391028}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":396391028},{"type":"v1","addr":"172.21.15.100:6829","nonce":396391028}]},"public_addr":"172.21.15.100:6825/396391028","cluster_addr":"172.21.15.100:6827/396391028","heartbeat_back_addr":"172.21.15.100:6831/396391028","heartbeat_front_addr":"172.21.15.100:6829/396391028","state":["exists","up"]},{"osd":11,"uuid":"11798d41-7f7e-4319-95b8-8d3d0946bde9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":82,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":2671029483},{"type":"v1","addr":"172.21.15.157:6825","nonce":2671029483}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":2671029483},{"type":"v1","addr":"172.21.15.157:6827","nonce":2671029483}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":2671029483},{"type":"v1","addr":"172.21.15.157:6831","nonce":2671029483}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":2671029483},{"type":"v1","addr":"172.21.15.157:6829","nonce":2671029483}]},"public_addr":"172.21.15.157:6825/2671029483","cluster_addr":"172.21.15.157:6827/2671029483","heartbeat_back_addr":"172.21.15.157:6831/2671029483","heartbeat_front_addr":"172.21.15.157:6829/2671029483","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T10:14:31.822399+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T10:15:04.175721+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T10:15:35.902595+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T10:16:12.088891+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T10:16:46.428704+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T10:17:20.158270+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T10:17:55.915253+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T10:18:30.064641+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T10:19:04.410004+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T10:19:40.605181+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T10:20:15.636605+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T10:20:50.151365+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.11","mappings":[{"from":7,"to":11}]},{"pgid":"2.14","mappings":[{"from":2,"to":4}]},{"pgid":"2.1a","mappings":[{"from":8,"to":1}]},{"pgid":"2.1b","mappings":[{"from":2,"to":4}]},{"pgid":"2.1d","mappings":[{"from":8,"to":11}]},{"pgid":"3.0","mappings":[{"from":4,"to":8},{"from":5,"to":9}]},{"pgid":"3.6","mappings":[{"from":3,"to":7}]},{"pgid":"3.1b","mappings":[{"from":4,"to":6}]},{"pgid":"3.33","mappings":[{"from":5,"to":9}]},{"pgid":"3.3c","mappings":[{"from":4,"to":9}]},{"pgid":"4.c","mappings":[{"from":9,"to":10}]},{"pgid":"4.f","mappings":[{"from":9,"to":8}]},{"pgid":"4.19","mappings":[{"from":9,"to":8}]},{"pgid":"4.1d","mappings":[{"from":5,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.47:0/936039355":"2024-03-22T10:25:19.912777+0000","172.21.15.47:0/2983192238":"2024-03-22T10:25:19.912777+0000","172.21.15.47:6801/2823885994":"2024-03-22T10:25:19.912777+0000","172.21.15.47:0/356822431":"2024-03-22T10:12:25.844530+0000","172.21.15.47:0/1167490879":"2024-03-22T10:12:25.844530+0000","172.21.15.47:0/1540718138":"2024-03-22T10:12:25.844530+0000","172.21.15.47:6801/3836903972":"2024-03-22T10:12:25.844530+0000","172.21.15.47:0/235436602":"2024-03-22T10:11:50.301838+0000","172.21.15.47:0/2052815572":"2024-03-22T10:11:50.301838+0000","172.21.15.47:0/1607664848":"2024-03-22T10:25:19.912777+0000","172.21.15.47:6800/3942658432":"2024-03-22T10:11:50.301838+0000","172.21.15.47:0/3101796817":"2024-03-22T10:25:19.912777+0000","172.21.15.47:6801/3942658432":"2024-03-22T10:11:50.301838+0000","172.21.15.47:6800/2823885994":"2024-03-22T10:25:19.912777+0000","172.21.15.47:0/1419516257":"2024-03-22T10:11:50.301838+0000","172.21.15.47:6800/3836903972":"2024-03-22T10:12:25.844530+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[{"pool":3,"snaps":[{"begin":6,"length":1},{"begin":17,"length":1}]},{"pool":4,"snaps":[{"begin":6,"length":1},{"begin":17,"length":1}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-21T10:36:19.929 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:19.929+0000 7efe36000640 1 -- 172.21.15.47:0/1432980451 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7efe180a3090 msgr2=0x7efe180a5550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:36:19.929 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:19.929+0000 7efe36000640 1 --2- 172.21.15.47:0/1432980451 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7efe180a3090 0x7efe180a5550 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7efe2400bbe0 tx=0x7efe2400b390 comp rx=0 tx=0).stop 2024-03-21T10:36:19.929 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:19.930+0000 7efe36000640 1 -- 172.21.15.47:0/1432980451 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7efe300743f0 msgr2=0x7efe3019a3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:36:19.929 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:19.930+0000 7efe36000640 1 --2- 172.21.15.47:0/1432980451 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7efe300743f0 0x7efe3019a3f0 secure :-1 s=READY pgs=1330 cs=0 l=1 rev1=1 crypto rx=0x7efe20000c00 tx=0x7efe2002d880 comp rx=0 tx=0).stop 2024-03-21T10:36:19.929 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:19.930+0000 7efe36000640 1 -- 172.21.15.47:0/1432980451 shutdown_connections 2024-03-21T10:36:19.929 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:19.930+0000 7efe36000640 1 --2- 172.21.15.47:0/1432980451 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7efe3010dcd0 0x7efe3019e770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:36:19.929 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:19.930+0000 7efe36000640 1 --2- 172.21.15.47:0/1432980451 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7efe300743f0 0x7efe3019a3f0 unknown :-1 s=CLOSED pgs=1330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:36:19.929 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:19.930+0000 7efe36000640 1 --2- 172.21.15.47:0/1432980451 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7efe180a3090 0x7efe180a5550 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:36:19.929 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:19.930+0000 7efe36000640 1 --2- 172.21.15.47:0/1432980451 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7efe30073ad0 0x7efe30199eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:36:19.929 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:19.930+0000 7efe36000640 1 -- 172.21.15.47:0/1432980451 >> 172.21.15.47:0/1432980451 conn(0x7efe300fcfa0 msgr2=0x7efe300fdcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:36:19.930 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:19.930+0000 7efe36000640 1 -- 172.21.15.47:0/1432980451 shutdown_connections 2024-03-21T10:36:19.930 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:19.930+0000 7efe36000640 1 -- 172.21.15.47:0/1432980451 wait complete. 2024-03-21T10:36:19.949 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-21T10:15:40.553114+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-21T10:21:56.399518+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '86', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-21T10:21:57.812940+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '106', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 17, 'snap_epoch': 106, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-03-21T10:22:01.822568+0000', 'flags': 8197, 'flags_names': 'hashpspool,ec_overwrites,selfmanaged_snaps', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '106', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 17, 'snap_epoch': 106, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-03-21T10:36:19.950 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-21T10:36:20.182 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:20.183+0000 7fef3e000640 1 -- 172.21.15.47:0/2493496476 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fef381040e0 msgr2=0x7fef381105f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:36:20.183 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:20.183+0000 7fef3e000640 1 --2- 172.21.15.47:0/2493496476 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fef381040e0 0x7fef381105f0 secure :-1 s=READY pgs=1331 cs=0 l=1 rev1=1 crypto rx=0x7fef30009cd0 tx=0x7fef3002c0e0 comp rx=0 tx=0).stop 2024-03-21T10:36:20.183 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:20.183+0000 7fef3e000640 1 -- 172.21.15.47:0/2493496476 shutdown_connections 2024-03-21T10:36:20.183 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:20.183+0000 7fef3e000640 1 --2- 172.21.15.47:0/2493496476 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fef381040e0 0x7fef381105f0 unknown :-1 s=CLOSED pgs=1331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:36:20.183 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:20.183+0000 7fef3e000640 1 --2- 172.21.15.47:0/2493496476 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fef38103720 0x7fef38103ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:36:20.183 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:20.183+0000 7fef3e000640 1 --2- 172.21.15.47:0/2493496476 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fef38109730 0x7fef38109b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:36:20.183 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:20.183+0000 7fef3e000640 1 -- 172.21.15.47:0/2493496476 >> 172.21.15.47:0/2493496476 conn(0x7fef380ff1d0 msgr2=0x7fef381015f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:36:20.183 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:20.183+0000 7fef3e000640 1 -- 172.21.15.47:0/2493496476 shutdown_connections 2024-03-21T10:36:20.183 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:20.183+0000 7fef3e000640 1 -- 172.21.15.47:0/2493496476 wait complete. 2024-03-21T10:36:20.185 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:20.184+0000 7fef3e000640 1 Processor -- start 2024-03-21T10:36:20.185 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:20.184+0000 7fef3e000640 1 -- start start 2024-03-21T10:36:20.185 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:20.185+0000 7fef3e000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fef38103720 0x7fef38199f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:36:20.185 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:20.185+0000 7fef3e000640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fef381040e0 0x7fef3819a490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:36:20.185 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:20.185+0000 7fef3e000640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fef38109730 0x7fef3819e7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:36:20.185 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:20.185+0000 7fef3e000640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef381139f0 con 0x7fef38109730 2024-03-21T10:36:20.185 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:20.185+0000 7fef3e000640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef38113870 con 0x7fef38103720 2024-03-21T10:36:20.185 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:20.185+0000 7fef3e000640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef38113b70 con 0x7fef381040e0 2024-03-21T10:36:20.186 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:20.185+0000 7fef3d600640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fef38109730 0x7fef3819e7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:36:20.186 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:20.185+0000 7fef3d600640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fef38109730 0x7fef3819e7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:35576/0 (socket says 172.21.15.47:35576) 2024-03-21T10:36:20.186 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:20.185+0000 7fef3d600640 1 -- 172.21.15.47:0/3353693930 learned_addr learned my addr 172.21.15.47:0/3353693930 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:36:20.186 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:20.185+0000 7fef3cc00640 1 --2- 172.21.15.47:0/3353693930 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fef38103720 0x7fef38199f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:36:20.186 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:20.185+0000 7fef37e00640 1 --2- 172.21.15.47:0/3353693930 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fef381040e0 0x7fef3819a490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:36:20.186 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:20.185+0000 7fef3d600640 1 -- 172.21.15.47:0/3353693930 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fef381040e0 msgr2=0x7fef3819a490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:36:20.186 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:20.186+0000 7fef3d600640 1 --2- 172.21.15.47:0/3353693930 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fef381040e0 0x7fef3819a490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:36:20.186 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:20.186+0000 7fef3d600640 1 -- 172.21.15.47:0/3353693930 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fef38103720 msgr2=0x7fef38199f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:36:20.186 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:20.186+0000 7fef3d600640 1 --2- 172.21.15.47:0/3353693930 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fef38103720 0x7fef38199f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:36:20.186 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:20.186+0000 7fef3d600640 1 -- 172.21.15.47:0/3353693930 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef3819eef0 con 0x7fef38109730 2024-03-21T10:36:20.186 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:20.186+0000 7fef3d600640 1 --2- 172.21.15.47:0/3353693930 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fef38109730 0x7fef3819e7c0 secure :-1 s=READY pgs=1332 cs=0 l=1 rev1=1 crypto rx=0x7fef30004af0 tx=0x7fef30035ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:36:20.186 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:20.186+0000 7fef35600640 1 -- 172.21.15.47:0/3353693930 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef30039040 con 0x7fef38109730 2024-03-21T10:36:20.186 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:20.186+0000 7fef35600640 1 -- 172.21.15.47:0/3353693930 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fef30044070 con 0x7fef38109730 2024-03-21T10:36:20.186 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:20.186+0000 7fef3e000640 1 -- 172.21.15.47:0/3353693930 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fef3819f180 con 0x7fef38109730 2024-03-21T10:36:20.187 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:20.186+0000 7fef35600640 1 -- 172.21.15.47:0/3353693930 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef300042a0 con 0x7fef38109730 2024-03-21T10:36:20.187 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:20.187+0000 7fef3e000640 1 -- 172.21.15.47:0/3353693930 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fef381a6a70 con 0x7fef38109730 2024-03-21T10:36:20.187 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:20.188+0000 7fef3e000640 1 -- 172.21.15.47:0/3353693930 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fef38068800 con 0x7fef38109730 2024-03-21T10:36:20.188 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:20.190+0000 7fef35600640 1 -- 172.21.15.47:0/3353693930 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 27) v1 ==== 137985+0+0 (secure 0 0 0) 0x7fef30004440 con 0x7fef38109730 2024-03-21T10:36:20.190 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:20.191+0000 7fef35600640 1 --2- 172.21.15.47:0/3353693930 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7fef200a3090 0x7fef200a5550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:36:20.190 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:20.191+0000 7fef35600640 1 -- 172.21.15.47:0/3353693930 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(107..107 src has 1..107) v4 ==== 9562+0+0 (secure 0 0 0) 0x7fef300f2090 con 0x7fef38109730 2024-03-21T10:36:20.190 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:20.192+0000 7fef3cc00640 1 --2- 172.21.15.47:0/3353693930 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7fef200a3090 0x7fef200a5550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:36:20.191 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:20.192+0000 7fef3cc00640 1 --2- 172.21.15.47:0/3353693930 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7fef200a3090 0x7fef200a5550 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fef2c005fd0 tx=0x7fef2c00b390 comp rx=0 tx=0).ready entity=mgr.15687 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:36:20.194 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:20.196+0000 7fef35600640 1 -- 172.21.15.47:0/3353693930 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fef300be820 con 0x7fef38109730 2024-03-21T10:36:20.452 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:20.453+0000 7fef3e000640 1 -- 172.21.15.47:0/3353693930 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fef3810bc90 con 0x7fef38109730 2024-03-21T10:36:20.452 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:20.454+0000 7fef35600640 1 -- 172.21.15.47:0/3353693930 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v107) v1 ==== 93+0+10 (secure 0 0 0) 0x7fef300be120 con 0x7fef38109730 2024-03-21T10:36:20.452 INFO:teuthology.orchestra.run.smithi047.stdout:pg_num: 1 2024-03-21T10:36:20.456 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:20.456+0000 7fef3e000640 1 -- 172.21.15.47:0/3353693930 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7fef200a3090 msgr2=0x7fef200a5550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:36:20.456 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:20.456+0000 7fef3e000640 1 --2- 172.21.15.47:0/3353693930 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7fef200a3090 0x7fef200a5550 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fef2c005fd0 tx=0x7fef2c00b390 comp rx=0 tx=0).stop 2024-03-21T10:36:20.456 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:20.457+0000 7fef3e000640 1 -- 172.21.15.47:0/3353693930 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fef38109730 msgr2=0x7fef3819e7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:36:20.456 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:20.457+0000 7fef3e000640 1 --2- 172.21.15.47:0/3353693930 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fef38109730 0x7fef3819e7c0 secure :-1 s=READY pgs=1332 cs=0 l=1 rev1=1 crypto rx=0x7fef30004af0 tx=0x7fef30035ce0 comp rx=0 tx=0).stop 2024-03-21T10:36:20.456 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:20.457+0000 7fef3e000640 1 -- 172.21.15.47:0/3353693930 shutdown_connections 2024-03-21T10:36:20.456 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:20.457+0000 7fef3e000640 1 --2- 172.21.15.47:0/3353693930 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fef38109730 0x7fef3819e7c0 unknown :-1 s=CLOSED pgs=1332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:36:20.456 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:20.457+0000 7fef3e000640 1 --2- 172.21.15.47:0/3353693930 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fef381040e0 0x7fef3819a490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:36:20.456 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:20.457+0000 7fef3e000640 1 --2- 172.21.15.47:0/3353693930 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7fef200a3090 0x7fef200a5550 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:36:20.456 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:20.457+0000 7fef3e000640 1 --2- 172.21.15.47:0/3353693930 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fef38103720 0x7fef38199f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:36:20.456 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:20.457+0000 7fef3e000640 1 -- 172.21.15.47:0/3353693930 >> 172.21.15.47:0/3353693930 conn(0x7fef380ff1d0 msgr2=0x7fef380ff980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:36:20.456 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:20.457+0000 7fef3e000640 1 -- 172.21.15.47:0/3353693930 shutdown_connections 2024-03-21T10:36:20.456 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:20.457+0000 7fef3e000640 1 -- 172.21.15.47:0/3353693930 wait complete. 2024-03-21T10:36:20.476 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-21T10:36:20.690 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:20.691+0000 7f4c03e00640 1 -- 172.21.15.47:0/2427181014 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4bf002cc80 con 0x7f4c040743f0 2024-03-21T10:36:20.691 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:20.692+0000 7f4c0ac00640 1 -- 172.21.15.47:0/2427181014 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4c040743f0 msgr2=0x7f4c0410d800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:36:20.691 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:20.692+0000 7f4c0ac00640 1 --2- 172.21.15.47:0/2427181014 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4c040743f0 0x7f4c0410d800 secure :-1 s=READY pgs=1333 cs=0 l=1 rev1=1 crypto rx=0x7f4bf0009900 tx=0x7f4bf002bcd0 comp rx=0 tx=0).stop 2024-03-21T10:36:20.691 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:20.692+0000 7f4c0ac00640 1 -- 172.21.15.47:0/2427181014 shutdown_connections 2024-03-21T10:36:20.691 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:20.692+0000 7f4c0ac00640 1 --2- 172.21.15.47:0/2427181014 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4c0410dd40 0x7f4c04110190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:36:20.691 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:20.692+0000 7f4c0ac00640 1 --2- 172.21.15.47:0/2427181014 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4c040743f0 0x7f4c0410d800 unknown :-1 s=CLOSED pgs=1333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:36:20.691 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:20.692+0000 7f4c0ac00640 1 --2- 172.21.15.47:0/2427181014 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4c04073ad0 0x7f4c04073eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:36:20.691 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:20.692+0000 7f4c0ac00640 1 -- 172.21.15.47:0/2427181014 >> 172.21.15.47:0/2427181014 conn(0x7f4c040fd010 msgr2=0x7f4c040ff430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:36:20.691 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:20.692+0000 7f4c0ac00640 1 -- 172.21.15.47:0/2427181014 shutdown_connections 2024-03-21T10:36:20.692 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:20.692+0000 7f4c0ac00640 1 -- 172.21.15.47:0/2427181014 wait complete. 2024-03-21T10:36:20.692 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:20.693+0000 7f4c0ac00640 1 Processor -- start 2024-03-21T10:36:20.692 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:20.694+0000 7f4c0ac00640 1 -- start start 2024-03-21T10:36:20.694 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:20.694+0000 7f4c0ac00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4c04073ad0 0x7f4c04199f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:36:20.694 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:20.694+0000 7f4c0ac00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4c040743f0 0x7f4c0419a4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:36:20.694 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:20.694+0000 7f4c0ac00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4c0410dd40 0x7f4c0419e830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:36:20.694 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:20.694+0000 7f4c0ac00640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c04113990 con 0x7f4c04073ad0 2024-03-21T10:36:20.694 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:20.694+0000 7f4c0ac00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c04113810 con 0x7f4c040743f0 2024-03-21T10:36:20.694 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:20.694+0000 7f4c0ac00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c04113b10 con 0x7f4c0410dd40 2024-03-21T10:36:20.694 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:20.694+0000 7f4c09800640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4c04073ad0 0x7f4c04199f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:36:20.694 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:20.694+0000 7f4c09800640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4c04073ad0 0x7f4c04199f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:35588/0 (socket says 172.21.15.47:35588) 2024-03-21T10:36:20.694 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:20.694+0000 7f4c09800640 1 -- 172.21.15.47:0/3431521705 learned_addr learned my addr 172.21.15.47:0/3431521705 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:36:20.694 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:20.695+0000 7f4c08e00640 1 --2- 172.21.15.47:0/3431521705 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4c040743f0 0x7f4c0419a4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:36:20.694 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:20.695+0000 7f4c0a200640 1 --2- 172.21.15.47:0/3431521705 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4c0410dd40 0x7f4c0419e830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:36:20.695 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:20.695+0000 7f4c09800640 1 -- 172.21.15.47:0/3431521705 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4c0410dd40 msgr2=0x7f4c0419e830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:36:20.695 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:20.695+0000 7f4c09800640 1 --2- 172.21.15.47:0/3431521705 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4c0410dd40 0x7f4c0419e830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:36:20.695 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:20.695+0000 7f4c09800640 1 -- 172.21.15.47:0/3431521705 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4c040743f0 msgr2=0x7f4c0419a4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:36:20.695 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:20.695+0000 7f4c09800640 1 --2- 172.21.15.47:0/3431521705 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4c040743f0 0x7f4c0419a4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:36:20.695 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:20.695+0000 7f4c09800640 1 -- 172.21.15.47:0/3431521705 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c0419ef60 con 0x7f4c04073ad0 2024-03-21T10:36:20.695 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:20.695+0000 7f4c08e00640 1 --2- 172.21.15.47:0/3431521705 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4c040743f0 0x7f4c0419a4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T10:36:20.695 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:20.695+0000 7f4c09800640 1 --2- 172.21.15.47:0/3431521705 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4c04073ad0 0x7f4c04199f70 secure :-1 s=READY pgs=1334 cs=0 l=1 rev1=1 crypto rx=0x7f4bfc00eb90 tx=0x7f4bfc00c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:36:20.696 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:20.695+0000 7f4c02000640 1 -- 172.21.15.47:0/3431521705 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4bfc004d20 con 0x7f4c04073ad0 2024-03-21T10:36:20.696 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:20.696+0000 7f4c0ac00640 1 -- 172.21.15.47:0/3431521705 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4c0419f250 con 0x7f4c04073ad0 2024-03-21T10:36:20.696 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:20.696+0000 7f4c02000640 1 -- 172.21.15.47:0/3431521705 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f4bfc00ee90 con 0x7f4c04073ad0 2024-03-21T10:36:20.696 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:20.696+0000 7f4c02000640 1 -- 172.21.15.47:0/3431521705 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4bfc010670 con 0x7f4c04073ad0 2024-03-21T10:36:20.696 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:20.696+0000 7f4c0ac00640 1 -- 172.21.15.47:0/3431521705 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4c041a6ae0 con 0x7f4c04073ad0 2024-03-21T10:36:20.696 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:20.697+0000 7f4c0ac00640 1 -- 172.21.15.47:0/3431521705 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4c040683b0 con 0x7f4c04073ad0 2024-03-21T10:36:20.697 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:20.699+0000 7f4c02000640 1 -- 172.21.15.47:0/3431521705 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 27) v1 ==== 137985+0+0 (secure 0 0 0) 0x7f4bfc005020 con 0x7f4c04073ad0 2024-03-21T10:36:20.699 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:20.700+0000 7f4c02000640 1 --2- 172.21.15.47:0/3431521705 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f4be80a3090 0x7f4be80a5550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:36:20.699 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:20.700+0000 7f4c02000640 1 -- 172.21.15.47:0/3431521705 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(107..107 src has 1..107) v4 ==== 9562+0+0 (secure 0 0 0) 0x7f4bfc0d03f0 con 0x7f4c04073ad0 2024-03-21T10:36:20.699 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:20.701+0000 7f4c08e00640 1 --2- 172.21.15.47:0/3431521705 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f4be80a3090 0x7f4be80a5550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:36:20.700 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:20.701+0000 7f4c08e00640 1 --2- 172.21.15.47:0/3431521705 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f4be80a3090 0x7f4be80a5550 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f4c0410f1c0 tx=0x7f4bf0005a20 comp rx=0 tx=0).ready entity=mgr.15687 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:36:20.703 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:20.705+0000 7f4c02000640 1 -- 172.21.15.47:0/3431521705 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f4bfc09ca70 con 0x7f4c04073ad0 2024-03-21T10:36:20.710 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:36:20 smithi047 ceph-mon[31547]: from='client.? 172.21.15.47:0/1432980451' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-21T10:36:20.726 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:36:20 smithi100 ceph-mon[35193]: from='client.? 172.21.15.47:0/1432980451' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-21T10:36:20.736 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:36:20 smithi157 ceph-mon[35581]: from='client.? 172.21.15.47:0/1432980451' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-21T10:36:20.907 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:20.908+0000 7f4c0ac00640 1 -- 172.21.15.47:0/3431521705 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f4c04108ec0 con 0x7f4c04073ad0 2024-03-21T10:36:20.908 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:20.909+0000 7f4c02000640 1 -- 172.21.15.47:0/3431521705 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v107) v1 ==== 104+0+11 (secure 0 0 0) 0x7f4bfc09c370 con 0x7f4c04073ad0 2024-03-21T10:36:20.908 INFO:teuthology.orchestra.run.smithi047.stdout:pg_num: 32 2024-03-21T10:36:20.913 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:20.913+0000 7f4c0ac00640 1 -- 172.21.15.47:0/3431521705 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f4be80a3090 msgr2=0x7f4be80a5550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:36:20.913 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:20.913+0000 7f4c0ac00640 1 --2- 172.21.15.47:0/3431521705 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f4be80a3090 0x7f4be80a5550 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f4c0410f1c0 tx=0x7f4bf0005a20 comp rx=0 tx=0).stop 2024-03-21T10:36:20.913 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:20.913+0000 7f4c0ac00640 1 -- 172.21.15.47:0/3431521705 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4c04073ad0 msgr2=0x7f4c04199f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:36:20.913 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:20.913+0000 7f4c0ac00640 1 --2- 172.21.15.47:0/3431521705 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4c04073ad0 0x7f4c04199f70 secure :-1 s=READY pgs=1334 cs=0 l=1 rev1=1 crypto rx=0x7f4bfc00eb90 tx=0x7f4bfc00c6a0 comp rx=0 tx=0).stop 2024-03-21T10:36:20.913 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:20.914+0000 7f4c0ac00640 1 -- 172.21.15.47:0/3431521705 shutdown_connections 2024-03-21T10:36:20.913 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:20.914+0000 7f4c0ac00640 1 --2- 172.21.15.47:0/3431521705 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4c0410dd40 0x7f4c0419e830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:36:20.913 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:20.914+0000 7f4c0ac00640 1 --2- 172.21.15.47:0/3431521705 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f4be80a3090 0x7f4be80a5550 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:36:20.913 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:20.914+0000 7f4c0ac00640 1 --2- 172.21.15.47:0/3431521705 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4c040743f0 0x7f4c0419a4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:36:20.913 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:20.914+0000 7f4c0ac00640 1 --2- 172.21.15.47:0/3431521705 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4c04073ad0 0x7f4c04199f70 unknown :-1 s=CLOSED pgs=1334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:36:20.913 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:20.914+0000 7f4c0ac00640 1 -- 172.21.15.47:0/3431521705 >> 172.21.15.47:0/3431521705 conn(0x7f4c040fd010 msgr2=0x7f4c040fde20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:36:20.913 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:20.914+0000 7f4c0ac00640 1 -- 172.21.15.47:0/3431521705 shutdown_connections 2024-03-21T10:36:20.913 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:20.914+0000 7f4c0ac00640 1 -- 172.21.15.47:0/3431521705 wait complete. 2024-03-21T10:36:20.932 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-21T10:36:21.145 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:21.146+0000 7f7866a00640 1 -- 172.21.15.47:0/1738655435 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f78601002b0 msgr2=0x7f7860100690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:36:21.145 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:21.146+0000 7f7866a00640 1 --2- 172.21.15.47:0/1738655435 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f78601002b0 0x7f7860100690 secure :-1 s=READY pgs=1335 cs=0 l=1 rev1=1 crypto rx=0x7f7854009900 tx=0x7f785402bc30 comp rx=0 tx=0).stop 2024-03-21T10:36:21.145 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:21.146+0000 7f7866a00640 1 -- 172.21.15.47:0/1738655435 shutdown_connections 2024-03-21T10:36:21.145 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:21.146+0000 7f7866a00640 1 --2- 172.21.15.47:0/1738655435 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f786010de60 0x7f7860110250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:36:21.145 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:21.146+0000 7f7866a00640 1 --2- 172.21.15.47:0/1738655435 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7860100c60 0x7f786010d920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:36:21.145 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:21.146+0000 7f7866a00640 1 --2- 172.21.15.47:0/1738655435 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f78601002b0 0x7f7860100690 unknown :-1 s=CLOSED pgs=1335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:36:21.145 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:21.146+0000 7f7866a00640 1 -- 172.21.15.47:0/1738655435 >> 172.21.15.47:0/1738655435 conn(0x7f78600fbef0 msgr2=0x7f78600fe310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:36:21.145 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:21.146+0000 7f7866a00640 1 -- 172.21.15.47:0/1738655435 shutdown_connections 2024-03-21T10:36:21.145 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:21.146+0000 7f7866a00640 1 -- 172.21.15.47:0/1738655435 wait complete. 2024-03-21T10:36:21.146 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:21.147+0000 7f7866a00640 1 Processor -- start 2024-03-21T10:36:21.146 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:21.147+0000 7f7866a00640 1 -- start start 2024-03-21T10:36:21.147 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:21.148+0000 7f7866a00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f78601002b0 0x7f7860199f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:36:21.147 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:21.148+0000 7f7866a00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7860100c60 0x7f786019a470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:36:21.147 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:21.148+0000 7f7866a00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f786010de60 0x7f786019e7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:36:21.148 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:21.148+0000 7f7866a00640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7860113a10 con 0x7f786010de60 2024-03-21T10:36:21.148 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:21.148+0000 7f7866a00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7860113890 con 0x7f78601002b0 2024-03-21T10:36:21.148 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:21.148+0000 7f7866a00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7860113b90 con 0x7f7860100c60 2024-03-21T10:36:21.148 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:21.148+0000 7f7865600640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f78601002b0 0x7f7860199f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:36:21.148 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:21.148+0000 7f7866000640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f786010de60 0x7f786019e7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:36:21.148 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:21.148+0000 7f7865600640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f78601002b0 0x7f7860199f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:56380/0 (socket says 172.21.15.47:56380) 2024-03-21T10:36:21.148 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:21.148+0000 7f7865600640 1 -- 172.21.15.47:0/3975140152 learned_addr learned my addr 172.21.15.47:0/3975140152 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:36:21.148 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:21.148+0000 7f7864c00640 1 --2- 172.21.15.47:0/3975140152 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7860100c60 0x7f786019a470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:36:21.148 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:21.149+0000 7f7866000640 1 -- 172.21.15.47:0/3975140152 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7860100c60 msgr2=0x7f786019a470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:36:21.148 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:21.149+0000 7f7866000640 1 --2- 172.21.15.47:0/3975140152 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7860100c60 0x7f786019a470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:36:21.148 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:21.149+0000 7f7866000640 1 -- 172.21.15.47:0/3975140152 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f78601002b0 msgr2=0x7f7860199f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:36:21.148 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:21.149+0000 7f7866000640 1 --2- 172.21.15.47:0/3975140152 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f78601002b0 0x7f7860199f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:36:21.148 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:21.149+0000 7f7866000640 1 -- 172.21.15.47:0/3975140152 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f786019ef20 con 0x7f786010de60 2024-03-21T10:36:21.148 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:21.149+0000 7f7865600640 1 --2- 172.21.15.47:0/3975140152 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f78601002b0 0x7f7860199f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T10:36:21.148 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:21.149+0000 7f7864c00640 1 --2- 172.21.15.47:0/3975140152 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7860100c60 0x7f786019a470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T10:36:21.149 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:21.149+0000 7f7866000640 1 --2- 172.21.15.47:0/3975140152 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f786010de60 0x7f786019e7f0 secure :-1 s=READY pgs=1336 cs=0 l=1 rev1=1 crypto rx=0x7f7858005fd0 tx=0x7f785800ef10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:36:21.149 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:21.149+0000 7f785e000640 1 -- 172.21.15.47:0/3975140152 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f785800cdf0 con 0x7f786010de60 2024-03-21T10:36:21.149 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:21.149+0000 7f785e000640 1 -- 172.21.15.47:0/3975140152 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f7858005020 con 0x7f786010de60 2024-03-21T10:36:21.150 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:21.149+0000 7f7866a00640 1 -- 172.21.15.47:0/3975140152 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f786019f210 con 0x7f786010de60 2024-03-21T10:36:21.150 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:21.149+0000 7f785e000640 1 -- 172.21.15.47:0/3975140152 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7858010690 con 0x7f786010de60 2024-03-21T10:36:21.150 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:21.149+0000 7f7866a00640 1 -- 172.21.15.47:0/3975140152 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f78601a6aa0 con 0x7f786010de60 2024-03-21T10:36:21.150 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:21.150+0000 7f7866a00640 1 -- 172.21.15.47:0/3975140152 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f78600683b0 con 0x7f786010de60 2024-03-21T10:36:21.151 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:21.153+0000 7f785e000640 1 -- 172.21.15.47:0/3975140152 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 27) v1 ==== 137985+0+0 (secure 0 0 0) 0x7f7858004a40 con 0x7f786010de60 2024-03-21T10:36:21.153 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:21.154+0000 7f785e000640 1 --2- 172.21.15.47:0/3975140152 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f78480a3090 0x7f78480a5550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:36:21.153 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:21.154+0000 7f785e000640 1 -- 172.21.15.47:0/3975140152 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(107..107 src has 1..107) v4 ==== 9562+0+0 (secure 0 0 0) 0x7f78580d0390 con 0x7f786010de60 2024-03-21T10:36:21.153 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:21.154+0000 7f7865600640 1 --2- 172.21.15.47:0/3975140152 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f78480a3090 0x7f78480a5550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:36:21.154 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:21.155+0000 7f7865600640 1 --2- 172.21.15.47:0/3975140152 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f78480a3090 0x7f78480a5550 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f78540096c0 tx=0x7f7854005a20 comp rx=0 tx=0).ready entity=mgr.15687 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:36:21.157 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:21.158+0000 7f785e000640 1 -- 172.21.15.47:0/3975140152 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f785809ca10 con 0x7f786010de60 2024-03-21T10:36:21.362 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:21.363+0000 7f7866a00640 1 -- 172.21.15.47:0/3975140152 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f7860108ff0 con 0x7f786010de60 2024-03-21T10:36:21.362 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:21.363+0000 7f785e000640 1 -- 172.21.15.47:0/3975140152 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v107) v1 ==== 100+0+11 (secure 0 0 0) 0x7f785809c310 con 0x7f786010de60 2024-03-21T10:36:21.362 INFO:teuthology.orchestra.run.smithi047.stdout:pg_num: 64 2024-03-21T10:36:21.365 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:21.366+0000 7f7866a00640 1 -- 172.21.15.47:0/3975140152 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f78480a3090 msgr2=0x7f78480a5550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:36:21.365 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:21.366+0000 7f7866a00640 1 --2- 172.21.15.47:0/3975140152 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f78480a3090 0x7f78480a5550 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f78540096c0 tx=0x7f7854005a20 comp rx=0 tx=0).stop 2024-03-21T10:36:21.365 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:21.366+0000 7f7866a00640 1 -- 172.21.15.47:0/3975140152 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f786010de60 msgr2=0x7f786019e7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:36:21.365 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:21.366+0000 7f7866a00640 1 --2- 172.21.15.47:0/3975140152 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f786010de60 0x7f786019e7f0 secure :-1 s=READY pgs=1336 cs=0 l=1 rev1=1 crypto rx=0x7f7858005fd0 tx=0x7f785800ef10 comp rx=0 tx=0).stop 2024-03-21T10:36:21.366 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:21.366+0000 7f7866a00640 1 -- 172.21.15.47:0/3975140152 shutdown_connections 2024-03-21T10:36:21.366 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:21.366+0000 7f7866a00640 1 --2- 172.21.15.47:0/3975140152 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f786010de60 0x7f786019e7f0 unknown :-1 s=CLOSED pgs=1336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:36:21.366 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:21.366+0000 7f7866a00640 1 --2- 172.21.15.47:0/3975140152 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7860100c60 0x7f786019a470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:36:21.366 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:21.366+0000 7f7866a00640 1 --2- 172.21.15.47:0/3975140152 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f78480a3090 0x7f78480a5550 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:36:21.366 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:21.366+0000 7f7866a00640 1 --2- 172.21.15.47:0/3975140152 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f78601002b0 0x7f7860199f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:36:21.366 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:21.366+0000 7f7866a00640 1 -- 172.21.15.47:0/3975140152 >> 172.21.15.47:0/3975140152 conn(0x7f78600fbef0 msgr2=0x7f78600fe2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:36:21.366 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:21.367+0000 7f7866a00640 1 -- 172.21.15.47:0/3975140152 shutdown_connections 2024-03-21T10:36:21.366 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:21.367+0000 7f7866a00640 1 -- 172.21.15.47:0/3975140152 wait complete. 2024-03-21T10:36:21.383 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-03-21T10:36:21.603 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:21.603+0000 7f4b6b200640 1 -- 172.21.15.47:0/1152326566 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4b6c10baf0 msgr2=0x7f4b6c10df90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:36:21.603 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:21.603+0000 7f4b6b200640 1 --2- 172.21.15.47:0/1152326566 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4b6c10baf0 0x7f4b6c10df90 secure :-1 s=READY pgs=1337 cs=0 l=1 rev1=1 crypto rx=0x7f4b64009900 tx=0x7f4b6402bcd0 comp rx=0 tx=0).stop 2024-03-21T10:36:21.603 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:21.604+0000 7f4b6b200640 1 -- 172.21.15.47:0/1152326566 shutdown_connections 2024-03-21T10:36:21.603 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:21.604+0000 7f4b6b200640 1 --2- 172.21.15.47:0/1152326566 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4b6c10baf0 0x7f4b6c10df90 unknown :-1 s=CLOSED pgs=1337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:36:21.603 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:21.604+0000 7f4b6b200640 1 --2- 172.21.15.47:0/1152326566 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4b6c102ff0 0x7f4b6c10b4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:36:21.603 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:21.604+0000 7f4b6b200640 1 --2- 172.21.15.47:0/1152326566 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4b6c1026d0 0x7f4b6c102ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:36:21.603 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:21.604+0000 7f4b6b200640 1 -- 172.21.15.47:0/1152326566 >> 172.21.15.47:0/1152326566 conn(0x7f4b6c0fbef0 msgr2=0x7f4b6c0fe310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:36:21.603 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:21.604+0000 7f4b6b200640 1 -- 172.21.15.47:0/1152326566 shutdown_connections 2024-03-21T10:36:21.603 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:21.604+0000 7f4b6b200640 1 -- 172.21.15.47:0/1152326566 wait complete. 2024-03-21T10:36:21.605 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:21.605+0000 7f4b6b200640 1 Processor -- start 2024-03-21T10:36:21.605 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:21.605+0000 7f4b6b200640 1 -- start start 2024-03-21T10:36:21.605 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:21.605+0000 7f4b6b200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4b6c1026d0 0x7f4b6c19da70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:36:21.605 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:21.605+0000 7f4b6b200640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4b6c102ff0 0x7f4b6c19dfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:36:21.606 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:21.605+0000 7f4b6b200640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4b6c10baf0 0x7f4b6c197b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:36:21.606 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:21.605+0000 7f4b6b200640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b6c111780 con 0x7f4b6c102ff0 2024-03-21T10:36:21.606 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:21.605+0000 7f4b6b200640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b6c111600 con 0x7f4b6c1026d0 2024-03-21T10:36:21.606 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:21.605+0000 7f4b6b200640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b6c111900 con 0x7f4b6c10baf0 2024-03-21T10:36:21.606 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:21.606+0000 7f4b61400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4b6c102ff0 0x7f4b6c19dfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:36:21.606 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:21.606+0000 7f4b69e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4b6c1026d0 0x7f4b6c19da70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:36:21.606 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:21.606+0000 7f4b61400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4b6c102ff0 0x7f4b6c19dfb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:35626/0 (socket says 172.21.15.47:35626) 2024-03-21T10:36:21.606 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:21.606+0000 7f4b61400640 1 -- 172.21.15.47:0/1808710356 learned_addr learned my addr 172.21.15.47:0/1808710356 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:36:21.606 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:21.606+0000 7f4b6a800640 1 --2- 172.21.15.47:0/1808710356 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4b6c10baf0 0x7f4b6c197b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:36:21.606 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:21.606+0000 7f4b61400640 1 -- 172.21.15.47:0/1808710356 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4b6c10baf0 msgr2=0x7f4b6c197b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:36:21.606 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:21.606+0000 7f4b61400640 1 --2- 172.21.15.47:0/1808710356 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4b6c10baf0 0x7f4b6c197b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:36:21.606 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:21.606+0000 7f4b61400640 1 -- 172.21.15.47:0/1808710356 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4b6c1026d0 msgr2=0x7f4b6c19da70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:36:21.606 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:21.606+0000 7f4b61400640 1 --2- 172.21.15.47:0/1808710356 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4b6c1026d0 0x7f4b6c19da70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:36:21.607 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:21.606+0000 7f4b61400640 1 -- 172.21.15.47:0/1808710356 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b6c198440 con 0x7f4b6c102ff0 2024-03-21T10:36:21.607 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:21.606+0000 7f4b69e00640 1 --2- 172.21.15.47:0/1808710356 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4b6c1026d0 0x7f4b6c19da70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T10:36:21.607 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:21.607+0000 7f4b61400640 1 --2- 172.21.15.47:0/1808710356 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4b6c102ff0 0x7f4b6c19dfb0 secure :-1 s=READY pgs=1338 cs=0 l=1 rev1=1 crypto rx=0x7f4b5400e9f0 tx=0x7f4b5400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:36:21.607 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:21.607+0000 7f4b63400640 1 -- 172.21.15.47:0/1808710356 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b5400ccc0 con 0x7f4b6c102ff0 2024-03-21T10:36:21.608 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:21.607+0000 7f4b63400640 1 -- 172.21.15.47:0/1808710356 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f4b540108e0 con 0x7f4b6c102ff0 2024-03-21T10:36:21.608 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:21.607+0000 7f4b63400640 1 -- 172.21.15.47:0/1808710356 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b540187c0 con 0x7f4b6c102ff0 2024-03-21T10:36:21.608 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:21.607+0000 7f4b6b200640 1 -- 172.21.15.47:0/1808710356 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4b6c198730 con 0x7f4b6c102ff0 2024-03-21T10:36:21.608 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:21.607+0000 7f4b6b200640 1 -- 172.21.15.47:0/1808710356 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4b6c1a49f0 con 0x7f4b6c102ff0 2024-03-21T10:36:21.608 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:21.608+0000 7f4b6b200640 1 -- 172.21.15.47:0/1808710356 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4b6c0690e0 con 0x7f4b6c102ff0 2024-03-21T10:36:21.611 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:21.612+0000 7f4b63400640 1 -- 172.21.15.47:0/1808710356 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 27) v1 ==== 137985+0+0 (secure 0 0 0) 0x7f4b54010430 con 0x7f4b6c102ff0 2024-03-21T10:36:21.612 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:21.613+0000 7f4b63400640 1 --2- 172.21.15.47:0/1808710356 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f4b500a3090 0x7f4b500a5550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:36:21.612 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:21.614+0000 7f4b63400640 1 -- 172.21.15.47:0/1808710356 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(107..107 src has 1..107) v4 ==== 9562+0+0 (secure 0 0 0) 0x7f4b540cff80 con 0x7f4b6c102ff0 2024-03-21T10:36:21.612 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:21.614+0000 7f4b69e00640 1 --2- 172.21.15.47:0/1808710356 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f4b500a3090 0x7f4b500a5550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:36:21.613 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:21.614+0000 7f4b69e00640 1 --2- 172.21.15.47:0/1808710356 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f4b500a3090 0x7f4b500a5550 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f4b5c009fd0 tx=0x7f4b5c009680 comp rx=0 tx=0).ready entity=mgr.15687 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:36:21.620 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:21.621+0000 7f4b63400640 1 -- 172.21.15.47:0/1808710356 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f4b5409c600 con 0x7f4b6c102ff0 2024-03-21T10:36:21.710 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:36:21 smithi047 ceph-mon[31547]: pgmap v344: 129 pgs: 129 active+clean; 2.5 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-03-21T10:36:21.710 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:36:21 smithi047 ceph-mon[31547]: from='client.? 172.21.15.47:0/3353693930' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-21T10:36:21.711 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:36:21 smithi047 ceph-mon[31547]: from='client.? 172.21.15.47:0/3431521705' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-21T10:36:21.726 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:36:21 smithi100 ceph-mon[35193]: pgmap v344: 129 pgs: 129 active+clean; 2.5 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-03-21T10:36:21.726 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:36:21 smithi100 ceph-mon[35193]: from='client.? 172.21.15.47:0/3353693930' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-21T10:36:21.726 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:36:21 smithi100 ceph-mon[35193]: from='client.? 172.21.15.47:0/3431521705' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-21T10:36:21.736 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:36:21 smithi157 ceph-mon[35581]: pgmap v344: 129 pgs: 129 active+clean; 2.5 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-03-21T10:36:21.737 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:36:21 smithi157 ceph-mon[35581]: from='client.? 172.21.15.47:0/3353693930' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-21T10:36:21.737 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:36:21 smithi157 ceph-mon[35581]: from='client.? 172.21.15.47:0/3431521705' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-21T10:36:21.827 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:21.828+0000 7f4b6b200640 1 -- 172.21.15.47:0/1808710356 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f4b6c198ff0 con 0x7f4b6c102ff0 2024-03-21T10:36:21.827 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:21.829+0000 7f4b63400640 1 -- 172.21.15.47:0/1808710356 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v107) v1 ==== 103+0+11 (secure 0 0 0) 0x7f4b540bc0f0 con 0x7f4b6c102ff0 2024-03-21T10:36:21.828 INFO:teuthology.orchestra.run.smithi047.stdout:pg_num: 32 2024-03-21T10:36:21.830 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:21.831+0000 7f4b6b200640 1 -- 172.21.15.47:0/1808710356 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f4b500a3090 msgr2=0x7f4b500a5550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:36:21.831 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:21.831+0000 7f4b6b200640 1 --2- 172.21.15.47:0/1808710356 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f4b500a3090 0x7f4b500a5550 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f4b5c009fd0 tx=0x7f4b5c009680 comp rx=0 tx=0).stop 2024-03-21T10:36:21.831 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:21.831+0000 7f4b6b200640 1 -- 172.21.15.47:0/1808710356 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4b6c102ff0 msgr2=0x7f4b6c19dfb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:36:21.831 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:21.831+0000 7f4b6b200640 1 --2- 172.21.15.47:0/1808710356 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4b6c102ff0 0x7f4b6c19dfb0 secure :-1 s=READY pgs=1338 cs=0 l=1 rev1=1 crypto rx=0x7f4b5400e9f0 tx=0x7f4b5400eeb0 comp rx=0 tx=0).stop 2024-03-21T10:36:21.831 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:21.831+0000 7f4b6b200640 1 -- 172.21.15.47:0/1808710356 shutdown_connections 2024-03-21T10:36:21.831 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:21.831+0000 7f4b6b200640 1 --2- 172.21.15.47:0/1808710356 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4b6c10baf0 0x7f4b6c197b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:36:21.831 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:21.831+0000 7f4b6b200640 1 --2- 172.21.15.47:0/1808710356 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4b6c102ff0 0x7f4b6c19dfb0 unknown :-1 s=CLOSED pgs=1338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:36:21.831 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:21.831+0000 7f4b6b200640 1 --2- 172.21.15.47:0/1808710356 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f4b500a3090 0x7f4b500a5550 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:36:21.831 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:21.831+0000 7f4b6b200640 1 --2- 172.21.15.47:0/1808710356 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4b6c1026d0 0x7f4b6c19da70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:36:21.831 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:21.831+0000 7f4b6b200640 1 -- 172.21.15.47:0/1808710356 >> 172.21.15.47:0/1808710356 conn(0x7f4b6c0fbef0 msgr2=0x7f4b6c0fcca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:36:21.831 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:21.832+0000 7f4b6b200640 1 -- 172.21.15.47:0/1808710356 shutdown_connections 2024-03-21T10:36:21.831 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:21.832+0000 7f4b6b200640 1 -- 172.21.15.47:0/1808710356 wait complete. 2024-03-21T10:36:21.853 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-03-21T10:36:22.065 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:22.066+0000 7fd815200640 1 -- 172.21.15.47:0/3626918511 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd810103200 msgr2=0x7fd81010b6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:36:22.066 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:22.066+0000 7fd815200640 1 --2- 172.21.15.47:0/3626918511 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd810103200 0x7fd81010b6e0 secure :-1 s=READY pgs=1339 cs=0 l=1 rev1=1 crypto rx=0x7fd7fc009900 tx=0x7fd7fc02bcd0 comp rx=0 tx=0).stop 2024-03-21T10:36:22.066 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:22.066+0000 7fd815200640 1 -- 172.21.15.47:0/3626918511 shutdown_connections 2024-03-21T10:36:22.066 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:22.066+0000 7fd815200640 1 --2- 172.21.15.47:0/3626918511 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd81010bcb0 0x7fd81010e0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:36:22.066 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:22.066+0000 7fd815200640 1 --2- 172.21.15.47:0/3626918511 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd810103200 0x7fd81010b6e0 unknown :-1 s=CLOSED pgs=1339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:36:22.066 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:22.066+0000 7fd815200640 1 --2- 172.21.15.47:0/3626918511 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd8101028e0 0x7fd810102cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:36:22.066 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:22.066+0000 7fd815200640 1 -- 172.21.15.47:0/3626918511 >> 172.21.15.47:0/3626918511 conn(0x7fd810076920 msgr2=0x7fd810076d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:36:22.066 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:22.066+0000 7fd815200640 1 -- 172.21.15.47:0/3626918511 shutdown_connections 2024-03-21T10:36:22.066 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:22.066+0000 7fd815200640 1 -- 172.21.15.47:0/3626918511 wait complete. 2024-03-21T10:36:22.068 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:22.067+0000 7fd815200640 1 Processor -- start 2024-03-21T10:36:22.069 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:22.067+0000 7fd815200640 1 -- start start 2024-03-21T10:36:22.069 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:22.068+0000 7fd815200640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd8101028e0 0x7fd810100530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:36:22.069 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:22.068+0000 7fd815200640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd810103200 0x7fd810100a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:36:22.069 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:22.068+0000 7fd815200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd81010bcb0 0x7fd810100fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:36:22.069 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:22.068+0000 7fd815200640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8101118f0 con 0x7fd810103200 2024-03-21T10:36:22.069 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:22.068+0000 7fd815200640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd810111770 con 0x7fd81010bcb0 2024-03-21T10:36:22.069 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:22.068+0000 7fd815200640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd810111a70 con 0x7fd8101028e0 2024-03-21T10:36:22.069 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:22.068+0000 7fd80d400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd810103200 0x7fd810100a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:36:22.069 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:22.068+0000 7fd80d400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd810103200 0x7fd810100a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:58184/0 (socket says 172.21.15.47:58184) 2024-03-21T10:36:22.069 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:22.068+0000 7fd80d400640 1 -- 172.21.15.47:0/73964892 learned_addr learned my addr 172.21.15.47:0/73964892 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:36:22.069 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:22.068+0000 7fd80e800640 1 --2- 172.21.15.47:0/73964892 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd81010bcb0 0x7fd810100fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:36:22.070 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:22.068+0000 7fd80de00640 1 --2- 172.21.15.47:0/73964892 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd8101028e0 0x7fd810100530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:36:22.070 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:22.069+0000 7fd80d400640 1 -- 172.21.15.47:0/73964892 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd8101028e0 msgr2=0x7fd810100530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:36:22.070 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:22.069+0000 7fd80d400640 1 --2- 172.21.15.47:0/73964892 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd8101028e0 0x7fd810100530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:36:22.070 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:22.069+0000 7fd80d400640 1 -- 172.21.15.47:0/73964892 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd81010bcb0 msgr2=0x7fd810100fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:36:22.070 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:22.069+0000 7fd80d400640 1 --2- 172.21.15.47:0/73964892 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd81010bcb0 0x7fd810100fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:36:22.070 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:22.069+0000 7fd80d400640 1 -- 172.21.15.47:0/73964892 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8101a63d0 con 0x7fd810103200 2024-03-21T10:36:22.070 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:22.069+0000 7fd80de00640 1 --2- 172.21.15.47:0/73964892 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd8101028e0 0x7fd810100530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T10:36:22.070 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:22.069+0000 7fd80e800640 1 --2- 172.21.15.47:0/73964892 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd81010bcb0 0x7fd810100fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T10:36:22.070 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:22.069+0000 7fd80d400640 1 --2- 172.21.15.47:0/73964892 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd810103200 0x7fd810100a70 secure :-1 s=READY pgs=1340 cs=0 l=1 rev1=1 crypto rx=0x7fd7fc005940 tx=0x7fd7fc0045e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:36:22.070 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:22.069+0000 7fd806a00640 1 -- 172.21.15.47:0/73964892 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7fc043020 con 0x7fd810103200 2024-03-21T10:36:22.070 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:22.069+0000 7fd806a00640 1 -- 172.21.15.47:0/73964892 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fd7fc02dba0 con 0x7fd810103200 2024-03-21T10:36:22.070 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:22.069+0000 7fd815200640 1 -- 172.21.15.47:0/73964892 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd8101a6660 con 0x7fd810103200 2024-03-21T10:36:22.070 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:22.070+0000 7fd806a00640 1 -- 172.21.15.47:0/73964892 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7fc003a10 con 0x7fd810103200 2024-03-21T10:36:22.071 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:22.070+0000 7fd815200640 1 -- 172.21.15.47:0/73964892 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd8101a6bb0 con 0x7fd810103200 2024-03-21T10:36:22.071 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:22.072+0000 7fd815200640 1 -- 172.21.15.47:0/73964892 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd81006a490 con 0x7fd810103200 2024-03-21T10:36:22.073 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:22.074+0000 7fd806a00640 1 -- 172.21.15.47:0/73964892 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 27) v1 ==== 137985+0+0 (secure 0 0 0) 0x7fd7fc046020 con 0x7fd810103200 2024-03-21T10:36:22.075 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:22.076+0000 7fd806a00640 1 --2- 172.21.15.47:0/73964892 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7fd7f40a3090 0x7fd7f40a5550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:36:22.075 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:22.076+0000 7fd806a00640 1 -- 172.21.15.47:0/73964892 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(107..107 src has 1..107) v4 ==== 9562+0+0 (secure 0 0 0) 0x7fd7fc0f22e0 con 0x7fd810103200 2024-03-21T10:36:22.075 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:22.076+0000 7fd80de00640 1 --2- 172.21.15.47:0/73964892 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7fd7f40a3090 0x7fd7f40a5550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:36:22.075 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:22.076+0000 7fd806a00640 1 -- 172.21.15.47:0/73964892 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fd7fc04d020 con 0x7fd810103200 2024-03-21T10:36:22.076 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:22.077+0000 7fd80de00640 1 --2- 172.21.15.47:0/73964892 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7fd7f40a3090 0x7fd7f40a5550 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fd8000058c0 tx=0x7fd800005850 comp rx=0 tx=0).ready entity=mgr.15687 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:36:22.295 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:22.297+0000 7fd815200640 1 -- 172.21.15.47:0/73964892 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7fd8100688e0 con 0x7fd810103200 2024-03-21T10:36:22.296 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:22.298+0000 7fd806a00640 1 -- 172.21.15.47:0/73964892 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 16 entries v107) v1 ==== 81+0+943 (secure 0 0 0) 0x7fd8100688e0 con 0x7fd810103200 2024-03-21T10:36:22.296 INFO:teuthology.orchestra.run.smithi047.stdout:172.21.15.47:0/936039355 2024-03-22T10:25:19.912777+0000 2024-03-21T10:36:22.296 INFO:teuthology.orchestra.run.smithi047.stdout:172.21.15.47:0/2983192238 2024-03-22T10:25:19.912777+0000 2024-03-21T10:36:22.296 INFO:teuthology.orchestra.run.smithi047.stdout:172.21.15.47:6801/2823885994 2024-03-22T10:25:19.912777+0000 2024-03-21T10:36:22.296 INFO:teuthology.orchestra.run.smithi047.stdout:172.21.15.47:0/356822431 2024-03-22T10:12:25.844530+0000 2024-03-21T10:36:22.296 INFO:teuthology.orchestra.run.smithi047.stdout:172.21.15.47:0/1167490879 2024-03-22T10:12:25.844530+0000 2024-03-21T10:36:22.296 INFO:teuthology.orchestra.run.smithi047.stdout:172.21.15.47:0/1540718138 2024-03-22T10:12:25.844530+0000 2024-03-21T10:36:22.297 INFO:teuthology.orchestra.run.smithi047.stdout:172.21.15.47:6801/3836903972 2024-03-22T10:12:25.844530+0000 2024-03-21T10:36:22.297 INFO:teuthology.orchestra.run.smithi047.stdout:172.21.15.47:0/235436602 2024-03-22T10:11:50.301838+0000 2024-03-21T10:36:22.297 INFO:teuthology.orchestra.run.smithi047.stdout:172.21.15.47:0/2052815572 2024-03-22T10:11:50.301838+0000 2024-03-21T10:36:22.297 INFO:teuthology.orchestra.run.smithi047.stdout:172.21.15.47:0/1607664848 2024-03-22T10:25:19.912777+0000 2024-03-21T10:36:22.297 INFO:teuthology.orchestra.run.smithi047.stdout:172.21.15.47:6800/3942658432 2024-03-22T10:11:50.301838+0000 2024-03-21T10:36:22.297 INFO:teuthology.orchestra.run.smithi047.stdout:172.21.15.47:0/3101796817 2024-03-22T10:25:19.912777+0000 2024-03-21T10:36:22.297 INFO:teuthology.orchestra.run.smithi047.stdout:172.21.15.47:6801/3942658432 2024-03-22T10:11:50.301838+0000 2024-03-21T10:36:22.297 INFO:teuthology.orchestra.run.smithi047.stdout:172.21.15.47:6800/2823885994 2024-03-22T10:25:19.912777+0000 2024-03-21T10:36:22.297 INFO:teuthology.orchestra.run.smithi047.stdout:172.21.15.47:0/1419516257 2024-03-22T10:11:50.301838+0000 2024-03-21T10:36:22.297 INFO:teuthology.orchestra.run.smithi047.stdout:172.21.15.47:6800/3836903972 2024-03-22T10:12:25.844530+0000 2024-03-21T10:36:22.297 INFO:teuthology.orchestra.run.smithi047.stderr:listed 16 entries 2024-03-21T10:36:22.299 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:22.301+0000 7fd815200640 1 -- 172.21.15.47:0/73964892 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7fd7f40a3090 msgr2=0x7fd7f40a5550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:36:22.299 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:22.301+0000 7fd815200640 1 --2- 172.21.15.47:0/73964892 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7fd7f40a3090 0x7fd7f40a5550 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fd8000058c0 tx=0x7fd800005850 comp rx=0 tx=0).stop 2024-03-21T10:36:22.299 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:22.301+0000 7fd815200640 1 -- 172.21.15.47:0/73964892 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd810103200 msgr2=0x7fd810100a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:36:22.299 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:22.301+0000 7fd815200640 1 --2- 172.21.15.47:0/73964892 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd810103200 0x7fd810100a70 secure :-1 s=READY pgs=1340 cs=0 l=1 rev1=1 crypto rx=0x7fd7fc005940 tx=0x7fd7fc0045e0 comp rx=0 tx=0).stop 2024-03-21T10:36:22.299 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:22.301+0000 7fd815200640 1 -- 172.21.15.47:0/73964892 shutdown_connections 2024-03-21T10:36:22.299 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:22.301+0000 7fd815200640 1 --2- 172.21.15.47:0/73964892 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7fd7f40a3090 0x7fd7f40a5550 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:36:22.300 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:22.301+0000 7fd815200640 1 --2- 172.21.15.47:0/73964892 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd81010bcb0 0x7fd810100fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:36:22.300 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:22.301+0000 7fd815200640 1 --2- 172.21.15.47:0/73964892 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd810103200 0x7fd810100a70 unknown :-1 s=CLOSED pgs=1340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:36:22.300 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:22.301+0000 7fd815200640 1 --2- 172.21.15.47:0/73964892 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd8101028e0 0x7fd810100530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:36:22.300 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:22.301+0000 7fd815200640 1 -- 172.21.15.47:0/73964892 >> 172.21.15.47:0/73964892 conn(0x7fd810076920 msgr2=0x7fd8100fd500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:36:22.300 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:22.302+0000 7fd815200640 1 -- 172.21.15.47:0/73964892 shutdown_connections 2024-03-21T10:36:22.300 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:22.302+0000 7fd815200640 1 -- 172.21.15.47:0/73964892 wait complete. 2024-03-21T10:36:22.319 DEBUG:teuthology.orchestra.run.smithi047:> set -ex 2024-03-21T10:36:22.319 DEBUG:teuthology.orchestra.run.smithi047:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-21T10:36:22.374 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-21T10:36:22.613 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:22.615+0000 7f16cbe00640 1 -- 172.21.15.47:0/3616501643 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f16cc107590 msgr2=0x7f16cc107970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:36:22.613 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:22.615+0000 7f16cbe00640 1 --2- 172.21.15.47:0/3616501643 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f16cc107590 0x7f16cc107970 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f16bc009930 tx=0x7f16bc02bc30 comp rx=0 tx=0).stop 2024-03-21T10:36:22.613 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:22.615+0000 7f16cbe00640 1 -- 172.21.15.47:0/3616501643 shutdown_connections 2024-03-21T10:36:22.613 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:22.615+0000 7f16cbe00640 1 --2- 172.21.15.47:0/3616501643 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f16cc0691c0 0x7f16cc10e310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:36:22.613 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:22.615+0000 7f16cbe00640 1 --2- 172.21.15.47:0/3616501643 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f16cc068800 0x7f16cc068c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:36:22.614 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:22.615+0000 7f16cbe00640 1 --2- 172.21.15.47:0/3616501643 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f16cc107590 0x7f16cc107970 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:36:22.614 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:22.615+0000 7f16cbe00640 1 -- 172.21.15.47:0/3616501643 >> 172.21.15.47:0/3616501643 conn(0x7f16cc075fb0 msgr2=0x7f16cc0763c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:36:22.614 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:22.615+0000 7f16cbe00640 1 -- 172.21.15.47:0/3616501643 shutdown_connections 2024-03-21T10:36:22.614 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:22.615+0000 7f16cbe00640 1 -- 172.21.15.47:0/3616501643 wait complete. 2024-03-21T10:36:22.616 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:22.616+0000 7f16cbe00640 1 Processor -- start 2024-03-21T10:36:22.616 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:22.616+0000 7f16cbe00640 1 -- start start 2024-03-21T10:36:22.616 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:22.617+0000 7f16cbe00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f16cc068800 0x7f16cc199ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:36:22.616 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:22.617+0000 7f16cbe00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f16cc0691c0 0x7f16cc19a400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:36:22.616 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:22.617+0000 7f16cbe00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f16cc107590 0x7f16cc19e780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:36:22.616 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:22.617+0000 7f16cbe00640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16cc111730 con 0x7f16cc068800 2024-03-21T10:36:22.616 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:22.617+0000 7f16cbe00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16cc1115b0 con 0x7f16cc107590 2024-03-21T10:36:22.616 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:22.617+0000 7f16cbe00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16cc1118b0 con 0x7f16cc0691c0 2024-03-21T10:36:22.616 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:22.617+0000 7f16caa00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f16cc068800 0x7f16cc199ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:36:22.617 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:22.617+0000 7f16cb400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f16cc107590 0x7f16cc19e780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:36:22.617 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:22.617+0000 7f16caa00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f16cc068800 0x7f16cc199ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:58204/0 (socket says 172.21.15.47:58204) 2024-03-21T10:36:22.617 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:22.617+0000 7f16cb400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f16cc107590 0x7f16cc19e780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:59202/0 (socket says 172.21.15.47:59202) 2024-03-21T10:36:22.617 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:22.617+0000 7f16caa00640 1 -- 172.21.15.47:0/376976068 learned_addr learned my addr 172.21.15.47:0/376976068 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:36:22.617 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:22.617+0000 7f16ca000640 1 --2- 172.21.15.47:0/376976068 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f16cc0691c0 0x7f16cc19a400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:36:22.617 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:22.618+0000 7f16caa00640 1 -- 172.21.15.47:0/376976068 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f16cc0691c0 msgr2=0x7f16cc19a400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:36:22.617 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:22.618+0000 7f16caa00640 1 --2- 172.21.15.47:0/376976068 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f16cc0691c0 0x7f16cc19a400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:36:22.617 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:22.618+0000 7f16caa00640 1 -- 172.21.15.47:0/376976068 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f16cc107590 msgr2=0x7f16cc19e780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:36:22.617 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:22.618+0000 7f16caa00640 1 --2- 172.21.15.47:0/376976068 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f16cc107590 0x7f16cc19e780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:36:22.617 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:22.618+0000 7f16caa00640 1 -- 172.21.15.47:0/376976068 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f16cc19eeb0 con 0x7f16cc068800 2024-03-21T10:36:22.617 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:22.618+0000 7f16cb400640 1 --2- 172.21.15.47:0/376976068 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f16cc107590 0x7f16cc19e780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T10:36:22.617 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:22.618+0000 7f16caa00640 1 --2- 172.21.15.47:0/376976068 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f16cc068800 0x7f16cc199ec0 secure :-1 s=READY pgs=1341 cs=0 l=1 rev1=1 crypto rx=0x7f16bc009230 tx=0x7f16bc02c710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:36:22.617 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:22.618+0000 7f16c3400640 1 -- 172.21.15.47:0/376976068 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16bc043020 con 0x7f16cc068800 2024-03-21T10:36:22.617 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:22.618+0000 7f16cbe00640 1 -- 172.21.15.47:0/376976068 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f16cc19f140 con 0x7f16cc068800 2024-03-21T10:36:22.618 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:22.618+0000 7f16c3400640 1 -- 172.21.15.47:0/376976068 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f16bc02caf0 con 0x7f16cc068800 2024-03-21T10:36:22.618 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:22.618+0000 7f16c3400640 1 -- 172.21.15.47:0/376976068 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16bc003ae0 con 0x7f16cc068800 2024-03-21T10:36:22.618 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:22.618+0000 7f16cbe00640 1 -- 172.21.15.47:0/376976068 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f16cc1a6a30 con 0x7f16cc068800 2024-03-21T10:36:22.618 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:22.620+0000 7f16cbe00640 1 -- 172.21.15.47:0/376976068 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f16cc0683b0 con 0x7f16cc068800 2024-03-21T10:36:22.619 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:22.621+0000 7f16c3400640 1 -- 172.21.15.47:0/376976068 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 27) v1 ==== 137985+0+0 (secure 0 0 0) 0x7f16bc046020 con 0x7f16cc068800 2024-03-21T10:36:22.621 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:22.623+0000 7f16c3400640 1 --2- 172.21.15.47:0/376976068 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f16b00a3090 0x7f16b00a5550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:36:22.621 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:22.623+0000 7f16c3400640 1 -- 172.21.15.47:0/376976068 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(107..107 src has 1..107) v4 ==== 9562+0+0 (secure 0 0 0) 0x7f16bc0f1ba0 con 0x7f16cc068800 2024-03-21T10:36:22.621 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:22.623+0000 7f16ca000640 1 --2- 172.21.15.47:0/376976068 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f16b00a3090 0x7f16b00a5550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:36:22.621 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:22.624+0000 7f16ca000640 1 --2- 172.21.15.47:0/376976068 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f16b00a3090 0x7f16b00a5550 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f16b80095f0 tx=0x7f16b8005f90 comp rx=0 tx=0).ready entity=mgr.15687 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:36:22.629 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:22.632+0000 7f16c3400640 1 -- 172.21.15.47:0/376976068 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f16bc04d020 con 0x7f16cc068800 2024-03-21T10:36:22.710 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:36:22 smithi047 ceph-mon[31547]: from='client.? 172.21.15.47:0/3975140152' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-21T10:36:22.710 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:36:22 smithi047 ceph-mon[31547]: from='client.? 172.21.15.47:0/1808710356' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-03-21T10:36:22.710 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:36:22 smithi047 ceph-mon[31547]: from='client.? 172.21.15.47:0/73964892' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-03-21T10:36:22.726 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:36:22 smithi100 ceph-mon[35193]: from='client.? 172.21.15.47:0/3975140152' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-21T10:36:22.726 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:36:22 smithi100 ceph-mon[35193]: from='client.? 172.21.15.47:0/1808710356' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-03-21T10:36:22.726 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:36:22 smithi100 ceph-mon[35193]: from='client.? 172.21.15.47:0/73964892' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-03-21T10:36:22.736 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:36:22 smithi157 ceph-mon[35581]: from='client.? 172.21.15.47:0/3975140152' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-21T10:36:22.736 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:36:22 smithi157 ceph-mon[35581]: from='client.? 172.21.15.47:0/1808710356' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-03-21T10:36:22.736 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:36:22 smithi157 ceph-mon[35581]: from='client.? 172.21.15.47:0/73964892' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-03-21T10:36:22.923 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:22.925+0000 7f16cbe00640 1 -- 172.21.15.47:0/376976068 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f16cc1a6c00 con 0x7f16cc068800 2024-03-21T10:36:22.925 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:22.927+0000 7f16c3400640 1 -- 172.21.15.47:0/376976068 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v107) v1 ==== 74+0+25412 (secure 0 0 0) 0x7f16bc0be330 con 0x7f16cc068800 2024-03-21T10:36:22.926 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-21T10:36:22.926 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":107,"fsid":"3056d51a-e76b-11ee-95cd-87774f69a715","created":"2024-03-21T10:11:17.019965+0000","modified":"2024-03-21T10:36:13.887907+0000","last_up_change":"2024-03-21T10:20:52.675328+0000","last_in_change":"2024-03-21T10:20:27.648212+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-21T10:15:40.553114+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-21T10:21:56.399518+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-21T10:21:57.812940+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"106","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":17,"snap_epoch":106,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-21T10:22:01.822568+0000","flags":8197,"flags_names":"hashpspool,ec_overwrites,selfmanaged_snaps","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"106","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":17,"snap_epoch":106,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"b0c7a102-8d89-428b-b793-cbbe67151f50","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6802","nonce":775837313},{"type":"v1","addr":"172.21.15.47:6803","nonce":775837313}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6804","nonce":775837313},{"type":"v1","addr":"172.21.15.47:6805","nonce":775837313}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6808","nonce":775837313},{"type":"v1","addr":"172.21.15.47:6809","nonce":775837313}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6806","nonce":775837313},{"type":"v1","addr":"172.21.15.47:6807","nonce":775837313}]},"public_addr":"172.21.15.47:6803/775837313","cluster_addr":"172.21.15.47:6805/775837313","heartbeat_back_addr":"172.21.15.47:6809/775837313","heartbeat_front_addr":"172.21.15.47:6807/775837313","state":["exists","up"]},{"osd":1,"uuid":"07bb81bb-7cf3-4bbc-ac9e-802f95185d11","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6800","nonce":2923304710},{"type":"v1","addr":"172.21.15.100:6801","nonce":2923304710}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":2923304710},{"type":"v1","addr":"172.21.15.100:6803","nonce":2923304710}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":2923304710},{"type":"v1","addr":"172.21.15.100:6807","nonce":2923304710}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6804","nonce":2923304710},{"type":"v1","addr":"172.21.15.100:6805","nonce":2923304710}]},"public_addr":"172.21.15.100:6801/2923304710","cluster_addr":"172.21.15.100:6803/2923304710","heartbeat_back_addr":"172.21.15.100:6807/2923304710","heartbeat_front_addr":"172.21.15.100:6805/2923304710","state":["exists","up"]},{"osd":2,"uuid":"88ce76b9-6081-4836-aad0-8c249c56e8a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":171377005},{"type":"v1","addr":"172.21.15.157:6801","nonce":171377005}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":171377005},{"type":"v1","addr":"172.21.15.157:6803","nonce":171377005}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":171377005},{"type":"v1","addr":"172.21.15.157:6807","nonce":171377005}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":171377005},{"type":"v1","addr":"172.21.15.157:6805","nonce":171377005}]},"public_addr":"172.21.15.157:6801/171377005","cluster_addr":"172.21.15.157:6803/171377005","heartbeat_back_addr":"172.21.15.157:6807/171377005","heartbeat_front_addr":"172.21.15.157:6805/171377005","state":["exists","up"]},{"osd":3,"uuid":"518aa33d-263f-45e6-aab5-2201ceb40e74","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":27,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6810","nonce":640760101},{"type":"v1","addr":"172.21.15.47:6811","nonce":640760101}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6812","nonce":640760101},{"type":"v1","addr":"172.21.15.47:6813","nonce":640760101}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6816","nonce":640760101},{"type":"v1","addr":"172.21.15.47:6817","nonce":640760101}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6814","nonce":640760101},{"type":"v1","addr":"172.21.15.47:6815","nonce":640760101}]},"public_addr":"172.21.15.47:6811/640760101","cluster_addr":"172.21.15.47:6813/640760101","heartbeat_back_addr":"172.21.15.47:6817/640760101","heartbeat_front_addr":"172.21.15.47:6815/640760101","state":["exists","up"]},{"osd":4,"uuid":"987bde83-ecc6-42e5-bd56-c143b3de4226","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":35,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6808","nonce":2785478639},{"type":"v1","addr":"172.21.15.100:6809","nonce":2785478639}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6810","nonce":2785478639},{"type":"v1","addr":"172.21.15.100:6811","nonce":2785478639}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6814","nonce":2785478639},{"type":"v1","addr":"172.21.15.100:6815","nonce":2785478639}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":2785478639},{"type":"v1","addr":"172.21.15.100:6813","nonce":2785478639}]},"public_addr":"172.21.15.100:6809/2785478639","cluster_addr":"172.21.15.100:6811/2785478639","heartbeat_back_addr":"172.21.15.100:6815/2785478639","heartbeat_front_addr":"172.21.15.100:6813/2785478639","state":["exists","up"]},{"osd":5,"uuid":"012d2188-3a61-4a5f-8d01-2001c15ffbe2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":40,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":1525645410},{"type":"v1","addr":"172.21.15.157:6809","nonce":1525645410}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":1525645410},{"type":"v1","addr":"172.21.15.157:6811","nonce":1525645410}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":1525645410},{"type":"v1","addr":"172.21.15.157:6815","nonce":1525645410}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":1525645410},{"type":"v1","addr":"172.21.15.157:6813","nonce":1525645410}]},"public_addr":"172.21.15.157:6809/1525645410","cluster_addr":"172.21.15.157:6811/1525645410","heartbeat_back_addr":"172.21.15.157:6815/1525645410","heartbeat_front_addr":"172.21.15.157:6813/1525645410","state":["exists","up"]},{"osd":6,"uuid":"c12c14b4-85f0-4d88-937c-de4f556437de","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6818","nonce":2321642790},{"type":"v1","addr":"172.21.15.47:6819","nonce":2321642790}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6820","nonce":2321642790},{"type":"v1","addr":"172.21.15.47:6821","nonce":2321642790}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6824","nonce":2321642790},{"type":"v1","addr":"172.21.15.47:6825","nonce":2321642790}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6822","nonce":2321642790},{"type":"v1","addr":"172.21.15.47:6823","nonce":2321642790}]},"public_addr":"172.21.15.47:6819/2321642790","cluster_addr":"172.21.15.47:6821/2321642790","heartbeat_back_addr":"172.21.15.47:6825/2321642790","heartbeat_front_addr":"172.21.15.47:6823/2321642790","state":["exists","up"]},{"osd":7,"uuid":"3c87c6d0-1d1f-4294-9917-aaaad7dda824","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6816","nonce":4104727330},{"type":"v1","addr":"172.21.15.100:6817","nonce":4104727330}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":4104727330},{"type":"v1","addr":"172.21.15.100:6819","nonce":4104727330}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":4104727330},{"type":"v1","addr":"172.21.15.100:6823","nonce":4104727330}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6820","nonce":4104727330},{"type":"v1","addr":"172.21.15.100:6821","nonce":4104727330}]},"public_addr":"172.21.15.100:6817/4104727330","cluster_addr":"172.21.15.100:6819/4104727330","heartbeat_back_addr":"172.21.15.100:6823/4104727330","heartbeat_front_addr":"172.21.15.100:6821/4104727330","state":["exists","up"]},{"osd":8,"uuid":"502d65cf-53c8-4936-8de4-e11b2a6e1e18","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":61,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":4067715347},{"type":"v1","addr":"172.21.15.157:6817","nonce":4067715347}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":4067715347},{"type":"v1","addr":"172.21.15.157:6819","nonce":4067715347}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":4067715347},{"type":"v1","addr":"172.21.15.157:6823","nonce":4067715347}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":4067715347},{"type":"v1","addr":"172.21.15.157:6821","nonce":4067715347}]},"public_addr":"172.21.15.157:6817/4067715347","cluster_addr":"172.21.15.157:6819/4067715347","heartbeat_back_addr":"172.21.15.157:6823/4067715347","heartbeat_front_addr":"172.21.15.157:6821/4067715347","state":["exists","up"]},{"osd":9,"uuid":"5ea1f5be-8458-4c93-9e61-c2e867a514f3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6826","nonce":1982108738},{"type":"v1","addr":"172.21.15.47:6827","nonce":1982108738}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6828","nonce":1982108738},{"type":"v1","addr":"172.21.15.47:6829","nonce":1982108738}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6832","nonce":1982108738},{"type":"v1","addr":"172.21.15.47:6833","nonce":1982108738}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6830","nonce":1982108738},{"type":"v1","addr":"172.21.15.47:6831","nonce":1982108738}]},"public_addr":"172.21.15.47:6827/1982108738","cluster_addr":"172.21.15.47:6829/1982108738","heartbeat_back_addr":"172.21.15.47:6833/1982108738","heartbeat_front_addr":"172.21.15.47:6831/1982108738","state":["exists","up"]},{"osd":10,"uuid":"2da3da90-a0be-4a54-a9ff-b2d6c9b36870","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":74,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":396391028},{"type":"v1","addr":"172.21.15.100:6825","nonce":396391028}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":396391028},{"type":"v1","addr":"172.21.15.100:6827","nonce":396391028}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6830","nonce":396391028},{"type":"v1","addr":"172.21.15.100:6831","nonce":396391028}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":396391028},{"type":"v1","addr":"172.21.15.100:6829","nonce":396391028}]},"public_addr":"172.21.15.100:6825/396391028","cluster_addr":"172.21.15.100:6827/396391028","heartbeat_back_addr":"172.21.15.100:6831/396391028","heartbeat_front_addr":"172.21.15.100:6829/396391028","state":["exists","up"]},{"osd":11,"uuid":"11798d41-7f7e-4319-95b8-8d3d0946bde9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":82,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":2671029483},{"type":"v1","addr":"172.21.15.157:6825","nonce":2671029483}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":2671029483},{"type":"v1","addr":"172.21.15.157:6827","nonce":2671029483}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":2671029483},{"type":"v1","addr":"172.21.15.157:6831","nonce":2671029483}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":2671029483},{"type":"v1","addr":"172.21.15.157:6829","nonce":2671029483}]},"public_addr":"172.21.15.157:6825/2671029483","cluster_addr":"172.21.15.157:6827/2671029483","heartbeat_back_addr":"172.21.15.157:6831/2671029483","heartbeat_front_addr":"172.21.15.157:6829/2671029483","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T10:14:31.822399+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T10:15:04.175721+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T10:15:35.902595+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T10:16:12.088891+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T10:16:46.428704+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T10:17:20.158270+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T10:17:55.915253+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T10:18:30.064641+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T10:19:04.410004+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T10:19:40.605181+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T10:20:15.636605+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T10:20:50.151365+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.11","mappings":[{"from":7,"to":11}]},{"pgid":"2.14","mappings":[{"from":2,"to":4}]},{"pgid":"2.1a","mappings":[{"from":8,"to":1}]},{"pgid":"2.1b","mappings":[{"from":2,"to":4}]},{"pgid":"2.1d","mappings":[{"from":8,"to":11}]},{"pgid":"3.0","mappings":[{"from":4,"to":8},{"from":5,"to":9}]},{"pgid":"3.6","mappings":[{"from":3,"to":7}]},{"pgid":"3.1b","mappings":[{"from":4,"to":6}]},{"pgid":"3.33","mappings":[{"from":5,"to":9}]},{"pgid":"3.3c","mappings":[{"from":4,"to":9}]},{"pgid":"4.c","mappings":[{"from":9,"to":10}]},{"pgid":"4.f","mappings":[{"from":9,"to":8}]},{"pgid":"4.19","mappings":[{"from":9,"to":8}]},{"pgid":"4.1d","mappings":[{"from":5,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.47:0/936039355":"2024-03-22T10:25:19.912777+0000","172.21.15.47:0/2983192238":"2024-03-22T10:25:19.912777+0000","172.21.15.47:6801/2823885994":"2024-03-22T10:25:19.912777+0000","172.21.15.47:0/356822431":"2024-03-22T10:12:25.844530+0000","172.21.15.47:0/1167490879":"2024-03-22T10:12:25.844530+0000","172.21.15.47:0/1540718138":"2024-03-22T10:12:25.844530+0000","172.21.15.47:6801/3836903972":"2024-03-22T10:12:25.844530+0000","172.21.15.47:0/235436602":"2024-03-22T10:11:50.301838+0000","172.21.15.47:0/2052815572":"2024-03-22T10:11:50.301838+0000","172.21.15.47:0/1607664848":"2024-03-22T10:25:19.912777+0000","172.21.15.47:6800/3942658432":"2024-03-22T10:11:50.301838+0000","172.21.15.47:0/3101796817":"2024-03-22T10:25:19.912777+0000","172.21.15.47:6801/3942658432":"2024-03-22T10:11:50.301838+0000","172.21.15.47:6800/2823885994":"2024-03-22T10:25:19.912777+0000","172.21.15.47:0/1419516257":"2024-03-22T10:11:50.301838+0000","172.21.15.47:6800/3836903972":"2024-03-22T10:12:25.844530+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[{"pool":3,"snaps":[{"begin":6,"length":1},{"begin":17,"length":1}]},{"pool":4,"snaps":[{"begin":6,"length":1},{"begin":17,"length":1}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-21T10:36:22.928 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:22.930+0000 7f16cbe00640 1 -- 172.21.15.47:0/376976068 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f16b00a3090 msgr2=0x7f16b00a5550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:36:22.928 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:22.930+0000 7f16cbe00640 1 --2- 172.21.15.47:0/376976068 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f16b00a3090 0x7f16b00a5550 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f16b80095f0 tx=0x7f16b8005f90 comp rx=0 tx=0).stop 2024-03-21T10:36:22.929 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:22.930+0000 7f16cbe00640 1 -- 172.21.15.47:0/376976068 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f16cc068800 msgr2=0x7f16cc199ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:36:22.929 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:22.930+0000 7f16cbe00640 1 --2- 172.21.15.47:0/376976068 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f16cc068800 0x7f16cc199ec0 secure :-1 s=READY pgs=1341 cs=0 l=1 rev1=1 crypto rx=0x7f16bc009230 tx=0x7f16bc02c710 comp rx=0 tx=0).stop 2024-03-21T10:36:22.929 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:22.930+0000 7f16cbe00640 1 -- 172.21.15.47:0/376976068 shutdown_connections 2024-03-21T10:36:22.929 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:22.930+0000 7f16cbe00640 1 --2- 172.21.15.47:0/376976068 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f16b00a3090 0x7f16b00a5550 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:36:22.929 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:22.930+0000 7f16cbe00640 1 --2- 172.21.15.47:0/376976068 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f16cc107590 0x7f16cc19e780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:36:22.929 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:22.930+0000 7f16cbe00640 1 --2- 172.21.15.47:0/376976068 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f16cc0691c0 0x7f16cc19a400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:36:22.929 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:22.930+0000 7f16cbe00640 1 --2- 172.21.15.47:0/376976068 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f16cc068800 0x7f16cc199ec0 unknown :-1 s=CLOSED pgs=1341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:36:22.929 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:22.930+0000 7f16cbe00640 1 -- 172.21.15.47:0/376976068 >> 172.21.15.47:0/376976068 conn(0x7f16cc075fb0 msgr2=0x7f16cc0ff110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:36:22.929 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:22.931+0000 7f16cbe00640 1 -- 172.21.15.47:0/376976068 shutdown_connections 2024-03-21T10:36:22.929 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:22.931+0000 7f16cbe00640 1 -- 172.21.15.47:0/376976068 wait complete. 2024-03-21T10:36:22.947 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-21T10:15:40.553114+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-21T10:21:56.399518+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '86', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-21T10:21:57.812940+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '106', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 17, 'snap_epoch': 106, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-03-21T10:22:01.822568+0000', 'flags': 8197, 'flags_names': 'hashpspool,ec_overwrites,selfmanaged_snaps', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '106', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 17, 'snap_epoch': 106, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-03-21T10:36:22.948 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-21T10:36:23.146 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:23.148+0000 7f40b7e00640 1 -- 172.21.15.47:0/4020973554 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f40b810baf0 msgr2=0x7f40b810df90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:36:23.146 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:23.148+0000 7f40b7e00640 1 --2- 172.21.15.47:0/4020973554 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f40b810baf0 0x7f40b810df90 secure :-1 s=READY pgs=1342 cs=0 l=1 rev1=1 crypto rx=0x7f40b0009ca0 tx=0x7f40b002c190 comp rx=0 tx=0).stop 2024-03-21T10:36:23.146 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:23.148+0000 7f40b7e00640 1 -- 172.21.15.47:0/4020973554 shutdown_connections 2024-03-21T10:36:23.147 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:23.148+0000 7f40b7e00640 1 --2- 172.21.15.47:0/4020973554 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f40b810baf0 0x7f40b810df90 unknown :-1 s=CLOSED pgs=1342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:36:23.147 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:23.148+0000 7f40b7e00640 1 --2- 172.21.15.47:0/4020973554 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f40b8102ff0 0x7f40b810b4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:36:23.147 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:23.148+0000 7f40b7e00640 1 --2- 172.21.15.47:0/4020973554 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f40b81026d0 0x7f40b8102ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:36:23.147 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:23.148+0000 7f40b7e00640 1 -- 172.21.15.47:0/4020973554 >> 172.21.15.47:0/4020973554 conn(0x7f40b80fbef0 msgr2=0x7f40b80fe310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:36:23.147 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:23.148+0000 7f40b7e00640 1 -- 172.21.15.47:0/4020973554 shutdown_connections 2024-03-21T10:36:23.147 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:23.148+0000 7f40b7e00640 1 -- 172.21.15.47:0/4020973554 wait complete. 2024-03-21T10:36:23.149 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:23.149+0000 7f40b7e00640 1 Processor -- start 2024-03-21T10:36:23.149 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:23.149+0000 7f40b7e00640 1 -- start start 2024-03-21T10:36:23.149 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:23.150+0000 7f40b7e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f40b81026d0 0x7f40b8100320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:36:23.149 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:23.150+0000 7f40b7e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f40b8102ff0 0x7f40b8100860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:36:23.150 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:23.150+0000 7f40b7e00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f40b810baf0 0x7f40b8100da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:36:23.150 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:23.150+0000 7f40b7e00640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40b8111780 con 0x7f40b8102ff0 2024-03-21T10:36:23.150 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:23.150+0000 7f40b7e00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40b8111600 con 0x7f40b81026d0 2024-03-21T10:36:23.150 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:23.150+0000 7f40b7e00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40b8111900 con 0x7f40b810baf0 2024-03-21T10:36:23.150 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:23.150+0000 7f40afe00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f40b8102ff0 0x7f40b8100860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:36:23.150 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:23.150+0000 7f40afe00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f40b8102ff0 0x7f40b8100860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:58216/0 (socket says 172.21.15.47:58216) 2024-03-21T10:36:23.150 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:23.150+0000 7f40afe00640 1 -- 172.21.15.47:0/1542307046 learned_addr learned my addr 172.21.15.47:0/1542307046 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:36:23.150 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:23.150+0000 7f40b6a00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f40b81026d0 0x7f40b8100320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:36:23.150 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:23.150+0000 7f40b7400640 1 --2- 172.21.15.47:0/1542307046 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f40b810baf0 0x7f40b8100da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:36:23.150 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:23.151+0000 7f40afe00640 1 -- 172.21.15.47:0/1542307046 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f40b810baf0 msgr2=0x7f40b8100da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:36:23.150 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:23.151+0000 7f40afe00640 1 --2- 172.21.15.47:0/1542307046 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f40b810baf0 0x7f40b8100da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:36:23.150 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:23.151+0000 7f40afe00640 1 -- 172.21.15.47:0/1542307046 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f40b81026d0 msgr2=0x7f40b8100320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:36:23.151 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:23.151+0000 7f40afe00640 1 --2- 172.21.15.47:0/1542307046 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f40b81026d0 0x7f40b8100320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:36:23.151 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:23.151+0000 7f40afe00640 1 -- 172.21.15.47:0/1542307046 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f40b81a6200 con 0x7f40b8102ff0 2024-03-21T10:36:23.151 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:23.151+0000 7f40afe00640 1 --2- 172.21.15.47:0/1542307046 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f40b8102ff0 0x7f40b8100860 secure :-1 s=READY pgs=1343 cs=0 l=1 rev1=1 crypto rx=0x7f40a000d950 tx=0x7f40a000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:36:23.151 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:23.151+0000 7f40af400640 1 -- 172.21.15.47:0/1542307046 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40a0017030 con 0x7f40b8102ff0 2024-03-21T10:36:23.151 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:23.151+0000 7f40af400640 1 -- 172.21.15.47:0/1542307046 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f40a0004020 con 0x7f40b8102ff0 2024-03-21T10:36:23.151 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:23.151+0000 7f40af400640 1 -- 172.21.15.47:0/1542307046 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40a00055e0 con 0x7f40b8102ff0 2024-03-21T10:36:23.151 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:23.151+0000 7f40b7e00640 1 -- 172.21.15.47:0/1542307046 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f40b81a64f0 con 0x7f40b8102ff0 2024-03-21T10:36:23.151 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:23.151+0000 7f40b7e00640 1 -- 172.21.15.47:0/1542307046 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f40b81a69f0 con 0x7f40b8102ff0 2024-03-21T10:36:23.151 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:23.153+0000 7f40b7e00640 1 -- 172.21.15.47:0/1542307046 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f40b80683b0 con 0x7f40b8102ff0 2024-03-21T10:36:23.152 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:23.155+0000 7f40af400640 1 -- 172.21.15.47:0/1542307046 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 27) v1 ==== 137985+0+0 (secure 0 0 0) 0x7f40a00041d0 con 0x7f40b8102ff0 2024-03-21T10:36:23.154 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:23.156+0000 7f40af400640 1 --2- 172.21.15.47:0/1542307046 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f409c0a3110 0x7f409c0a55d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:36:23.154 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:23.156+0000 7f40af400640 1 -- 172.21.15.47:0/1542307046 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(107..107 src has 1..107) v4 ==== 9562+0+0 (secure 0 0 0) 0x7f40a00cf490 con 0x7f40b8102ff0 2024-03-21T10:36:23.154 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:23.157+0000 7f40b6a00640 1 --2- 172.21.15.47:0/1542307046 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f409c0a3110 0x7f409c0a55d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:36:23.155 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:23.157+0000 7f40b6a00640 1 --2- 172.21.15.47:0/1542307046 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f409c0a3110 0x7f409c0a55d0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f40a8005fd0 tx=0x7f40a800b390 comp rx=0 tx=0).ready entity=mgr.15687 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:36:23.158 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:23.160+0000 7f40af400640 1 -- 172.21.15.47:0/1542307046 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f40a009bb10 con 0x7f40b8102ff0 2024-03-21T10:36:23.396 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:23.398+0000 7f40b7e00640 1 -- 172.21.15.47:0/1542307046 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f40b8106b90 con 0x7f40b8102ff0 2024-03-21T10:36:23.397 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:23.399+0000 7f40af400640 1 -- 172.21.15.47:0/1542307046 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v107) v1 ==== 93+0+10 (secure 0 0 0) 0x7f40a009b410 con 0x7f40b8102ff0 2024-03-21T10:36:23.397 INFO:teuthology.orchestra.run.smithi047.stdout:pg_num: 1 2024-03-21T10:36:23.399 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:23.401+0000 7f40b7e00640 1 -- 172.21.15.47:0/1542307046 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f409c0a3110 msgr2=0x7f409c0a55d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:36:23.399 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:23.401+0000 7f40b7e00640 1 --2- 172.21.15.47:0/1542307046 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f409c0a3110 0x7f409c0a55d0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f40a8005fd0 tx=0x7f40a800b390 comp rx=0 tx=0).stop 2024-03-21T10:36:23.399 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:23.401+0000 7f40b7e00640 1 -- 172.21.15.47:0/1542307046 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f40b8102ff0 msgr2=0x7f40b8100860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:36:23.400 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:23.401+0000 7f40b7e00640 1 --2- 172.21.15.47:0/1542307046 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f40b8102ff0 0x7f40b8100860 secure :-1 s=READY pgs=1343 cs=0 l=1 rev1=1 crypto rx=0x7f40a000d950 tx=0x7f40a000de10 comp rx=0 tx=0).stop 2024-03-21T10:36:23.400 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:23.401+0000 7f40b7e00640 1 -- 172.21.15.47:0/1542307046 shutdown_connections 2024-03-21T10:36:23.400 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:23.401+0000 7f40b7e00640 1 --2- 172.21.15.47:0/1542307046 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f40b810baf0 0x7f40b8100da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:36:23.400 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:23.401+0000 7f40b7e00640 1 --2- 172.21.15.47:0/1542307046 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f40b8102ff0 0x7f40b8100860 unknown :-1 s=CLOSED pgs=1343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:36:23.400 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:23.402+0000 7f40b7e00640 1 --2- 172.21.15.47:0/1542307046 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f409c0a3110 0x7f409c0a55d0 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:36:23.400 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:23.402+0000 7f40b7e00640 1 --2- 172.21.15.47:0/1542307046 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f40b81026d0 0x7f40b8100320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:36:23.400 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:23.402+0000 7f40b7e00640 1 -- 172.21.15.47:0/1542307046 >> 172.21.15.47:0/1542307046 conn(0x7f40b80fbef0 msgr2=0x7f40b80fcc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:36:23.400 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:23.402+0000 7f40b7e00640 1 -- 172.21.15.47:0/1542307046 shutdown_connections 2024-03-21T10:36:23.400 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:23.402+0000 7f40b7e00640 1 -- 172.21.15.47:0/1542307046 wait complete. 2024-03-21T10:36:23.420 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-21T10:36:23.499 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:36:23 smithi047 ceph-mon[31547]: pgmap v345: 129 pgs: 129 active+clean; 2.5 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 102 B/s wr, 9 op/s 2024-03-21T10:36:23.499 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:36:23 smithi047 ceph-mon[31547]: from='client.? 172.21.15.47:0/376976068' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-21T10:36:23.636 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:23.637+0000 7eff94000640 1 -- 172.21.15.47:0/4125408864 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7eff8c103840 msgr2=0x7eff8c103cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:36:23.636 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:23.637+0000 7eff94000640 1 --2- 172.21.15.47:0/4125408864 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7eff8c103840 0x7eff8c103cc0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7eff78009900 tx=0x7eff7802bc30 comp rx=0 tx=0).stop 2024-03-21T10:36:23.636 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:23.637+0000 7eff94000640 1 -- 172.21.15.47:0/4125408864 shutdown_connections 2024-03-21T10:36:23.636 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:23.637+0000 7eff94000640 1 --2- 172.21.15.47:0/4125408864 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7eff8c104200 0x7eff8c110710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:36:23.636 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:23.637+0000 7eff94000640 1 --2- 172.21.15.47:0/4125408864 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7eff8c103840 0x7eff8c103cc0 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:36:23.636 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:23.637+0000 7eff94000640 1 --2- 172.21.15.47:0/4125408864 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7eff8c109850 0x7eff8c109c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:36:23.636 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:23.637+0000 7eff94000640 1 -- 172.21.15.47:0/4125408864 >> 172.21.15.47:0/4125408864 conn(0x7eff8c0ff310 msgr2=0x7eff8c101730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:36:23.636 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:23.638+0000 7eff94000640 1 -- 172.21.15.47:0/4125408864 shutdown_connections 2024-03-21T10:36:23.636 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:23.638+0000 7eff94000640 1 -- 172.21.15.47:0/4125408864 wait complete. 2024-03-21T10:36:23.639 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:23.639+0000 7eff94000640 1 Processor -- start 2024-03-21T10:36:23.639 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:23.639+0000 7eff94000640 1 -- start start 2024-03-21T10:36:23.639 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:23.639+0000 7eff94000640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7eff8c103840 0x7eff8c19a0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:36:23.639 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:23.639+0000 7eff94000640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7eff8c104200 0x7eff8c19a5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:36:23.639 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:23.639+0000 7eff94000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7eff8c109850 0x7eff8c19e960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:36:23.639 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:23.639+0000 7eff94000640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff8c113b10 con 0x7eff8c103840 2024-03-21T10:36:23.639 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:23.639+0000 7eff94000640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff8c113990 con 0x7eff8c109850 2024-03-21T10:36:23.639 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:23.639+0000 7eff94000640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff8c113c90 con 0x7eff8c104200 2024-03-21T10:36:23.639 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:23.640+0000 7eff91600640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7eff8c103840 0x7eff8c19a0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:36:23.640 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:23.640+0000 7eff92000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7eff8c109850 0x7eff8c19e960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:36:23.640 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:23.640+0000 7eff91600640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7eff8c103840 0x7eff8c19a0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:58226/0 (socket says 172.21.15.47:58226) 2024-03-21T10:36:23.640 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:23.640+0000 7eff91600640 1 -- 172.21.15.47:0/1293107382 learned_addr learned my addr 172.21.15.47:0/1293107382 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:36:23.640 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:23.640+0000 7eff92000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7eff8c109850 0x7eff8c19e960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.47:59242/0 (socket says 172.21.15.47:59242) 2024-03-21T10:36:23.640 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:23.640+0000 7eff90c00640 1 --2- 172.21.15.47:0/1293107382 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7eff8c104200 0x7eff8c19a5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:36:23.640 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:23.640+0000 7eff91600640 1 -- 172.21.15.47:0/1293107382 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7eff8c104200 msgr2=0x7eff8c19a5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:36:23.640 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:23.640+0000 7eff91600640 1 --2- 172.21.15.47:0/1293107382 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7eff8c104200 0x7eff8c19a5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:36:23.640 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:23.640+0000 7eff91600640 1 -- 172.21.15.47:0/1293107382 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7eff8c109850 msgr2=0x7eff8c19e960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:36:23.640 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:23.640+0000 7eff91600640 1 --2- 172.21.15.47:0/1293107382 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7eff8c109850 0x7eff8c19e960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:36:23.640 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:23.640+0000 7eff91600640 1 -- 172.21.15.47:0/1293107382 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff8c19f090 con 0x7eff8c103840 2024-03-21T10:36:23.640 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:23.640+0000 7eff92000640 1 --2- 172.21.15.47:0/1293107382 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7eff8c109850 0x7eff8c19e960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T10:36:23.641 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:23.641+0000 7eff91600640 1 --2- 172.21.15.47:0/1293107382 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7eff8c103840 0x7eff8c19a0a0 secure :-1 s=READY pgs=1344 cs=0 l=1 rev1=1 crypto rx=0x7eff8000ea60 tx=0x7eff8000ef20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:36:23.641 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:23.641+0000 7eff8a000640 1 -- 172.21.15.47:0/1293107382 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff8000cde0 con 0x7eff8c103840 2024-03-21T10:36:23.641 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:23.641+0000 7eff8a000640 1 -- 172.21.15.47:0/1293107382 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7eff800035f0 con 0x7eff8c103840 2024-03-21T10:36:23.641 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:23.641+0000 7eff94000640 1 -- 172.21.15.47:0/1293107382 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7eff8c19f380 con 0x7eff8c103840 2024-03-21T10:36:23.641 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:23.641+0000 7eff8a000640 1 -- 172.21.15.47:0/1293107382 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff80010690 con 0x7eff8c103840 2024-03-21T10:36:23.641 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:23.641+0000 7eff94000640 1 -- 172.21.15.47:0/1293107382 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7eff8c1a6c10 con 0x7eff8c103840 2024-03-21T10:36:23.641 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:23.643+0000 7eff94000640 1 -- 172.21.15.47:0/1293107382 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7eff8c104f80 con 0x7eff8c103840 2024-03-21T10:36:23.642 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:23.644+0000 7eff8a000640 1 -- 172.21.15.47:0/1293107382 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 27) v1 ==== 137985+0+0 (secure 0 0 0) 0x7eff800037a0 con 0x7eff8c103840 2024-03-21T10:36:23.644 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:23.646+0000 7eff8a000640 1 --2- 172.21.15.47:0/1293107382 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7eff740a3090 0x7eff740a5550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:36:23.644 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:23.646+0000 7eff8a000640 1 -- 172.21.15.47:0/1293107382 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(107..107 src has 1..107) v4 ==== 9562+0+0 (secure 0 0 0) 0x7eff800cff30 con 0x7eff8c103840 2024-03-21T10:36:23.644 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:23.646+0000 7eff90c00640 1 --2- 172.21.15.47:0/1293107382 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7eff740a3090 0x7eff740a5550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:36:23.644 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:23.647+0000 7eff90c00640 1 --2- 172.21.15.47:0/1293107382 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7eff740a3090 0x7eff740a5550 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7eff8c108f40 tx=0x7eff78000bc0 comp rx=0 tx=0).ready entity=mgr.15687 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:36:23.652 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:23.654+0000 7eff8a000640 1 -- 172.21.15.47:0/1293107382 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7eff8009c5b0 con 0x7eff8c103840 2024-03-21T10:36:23.726 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:36:23 smithi100 ceph-mon[35193]: pgmap v345: 129 pgs: 129 active+clean; 2.5 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 102 B/s wr, 9 op/s 2024-03-21T10:36:23.726 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:36:23 smithi100 ceph-mon[35193]: from='client.? 172.21.15.47:0/376976068' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-21T10:36:23.736 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:36:23 smithi157 ceph-mon[35581]: pgmap v345: 129 pgs: 129 active+clean; 2.5 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 102 B/s wr, 9 op/s 2024-03-21T10:36:23.736 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:36:23 smithi157 ceph-mon[35581]: from='client.? 172.21.15.47:0/376976068' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-21T10:36:23.865 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:23.867+0000 7eff94000640 1 -- 172.21.15.47:0/1293107382 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7eff8c10be30 con 0x7eff8c103840 2024-03-21T10:36:23.866 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:23.868+0000 7eff8a000640 1 -- 172.21.15.47:0/1293107382 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v107) v1 ==== 104+0+11 (secure 0 0 0) 0x7eff8009beb0 con 0x7eff8c103840 2024-03-21T10:36:23.866 INFO:teuthology.orchestra.run.smithi047.stdout:pg_num: 32 2024-03-21T10:36:23.868 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:23.870+0000 7eff94000640 1 -- 172.21.15.47:0/1293107382 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7eff740a3090 msgr2=0x7eff740a5550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:36:23.868 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:23.870+0000 7eff94000640 1 --2- 172.21.15.47:0/1293107382 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7eff740a3090 0x7eff740a5550 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7eff8c108f40 tx=0x7eff78000bc0 comp rx=0 tx=0).stop 2024-03-21T10:36:23.868 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:23.870+0000 7eff94000640 1 -- 172.21.15.47:0/1293107382 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7eff8c103840 msgr2=0x7eff8c19a0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:36:23.868 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:23.870+0000 7eff94000640 1 --2- 172.21.15.47:0/1293107382 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7eff8c103840 0x7eff8c19a0a0 secure :-1 s=READY pgs=1344 cs=0 l=1 rev1=1 crypto rx=0x7eff8000ea60 tx=0x7eff8000ef20 comp rx=0 tx=0).stop 2024-03-21T10:36:23.869 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:23.870+0000 7eff94000640 1 -- 172.21.15.47:0/1293107382 shutdown_connections 2024-03-21T10:36:23.869 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:23.870+0000 7eff94000640 1 --2- 172.21.15.47:0/1293107382 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7eff740a3090 0x7eff740a5550 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:36:23.869 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:23.870+0000 7eff94000640 1 --2- 172.21.15.47:0/1293107382 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7eff8c109850 0x7eff8c19e960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:36:23.869 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:23.870+0000 7eff94000640 1 --2- 172.21.15.47:0/1293107382 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7eff8c104200 0x7eff8c19a5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:36:23.869 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:23.870+0000 7eff94000640 1 --2- 172.21.15.47:0/1293107382 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7eff8c103840 0x7eff8c19a0a0 unknown :-1 s=CLOSED pgs=1344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:36:23.869 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:23.870+0000 7eff94000640 1 -- 172.21.15.47:0/1293107382 >> 172.21.15.47:0/1293107382 conn(0x7eff8c0ff310 msgr2=0x7eff8c0ffb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:36:23.869 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:23.871+0000 7eff94000640 1 -- 172.21.15.47:0/1293107382 shutdown_connections 2024-03-21T10:36:23.869 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:23.871+0000 7eff94000640 1 -- 172.21.15.47:0/1293107382 wait complete. 2024-03-21T10:36:23.887 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-21T10:36:24.083 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:24.085+0000 7f03da200640 1 -- 172.21.15.47:0/3816095354 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f03d4100460 msgr2=0x7f03d4100840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:36:24.084 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:24.085+0000 7f03da200640 1 --2- 172.21.15.47:0/3816095354 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f03d4100460 0x7f03d4100840 secure :-1 s=READY pgs=1345 cs=0 l=1 rev1=1 crypto rx=0x7f03c4009900 tx=0x7f03c402bc30 comp rx=0 tx=0).stop 2024-03-21T10:36:24.084 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:24.085+0000 7f03da200640 1 -- 172.21.15.47:0/3816095354 shutdown_connections 2024-03-21T10:36:24.084 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:24.085+0000 7f03da200640 1 --2- 172.21.15.47:0/3816095354 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f03d410e010 0x7f03d4110400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:36:24.084 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:24.085+0000 7f03da200640 1 --2- 172.21.15.47:0/3816095354 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f03d4100d80 0x7f03d410dad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:36:24.084 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:24.085+0000 7f03da200640 1 --2- 172.21.15.47:0/3816095354 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f03d4100460 0x7f03d4100840 unknown :-1 s=CLOSED pgs=1345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:36:24.084 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:24.085+0000 7f03da200640 1 -- 172.21.15.47:0/3816095354 >> 172.21.15.47:0/3816095354 conn(0x7f03d4076920 msgr2=0x7f03d4076d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:36:24.084 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:24.085+0000 7f03da200640 1 -- 172.21.15.47:0/3816095354 shutdown_connections 2024-03-21T10:36:24.084 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:24.085+0000 7f03da200640 1 -- 172.21.15.47:0/3816095354 wait complete. 2024-03-21T10:36:24.087 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:24.086+0000 7f03da200640 1 Processor -- start 2024-03-21T10:36:24.087 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:24.087+0000 7f03da200640 1 -- start start 2024-03-21T10:36:24.087 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:24.087+0000 7f03da200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f03d4100460 0x7f03d419a070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:36:24.087 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:24.087+0000 7f03da200640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f03d4100d80 0x7f03d419a5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:36:24.087 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:24.087+0000 7f03da200640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f03d410e010 0x7f03d419e930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:36:24.087 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:24.087+0000 7f03da200640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03d4113b70 con 0x7f03d4100d80 2024-03-21T10:36:24.087 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:24.087+0000 7f03da200640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03d41139f0 con 0x7f03d4100460 2024-03-21T10:36:24.087 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:24.087+0000 7f03da200640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03d4113cf0 con 0x7f03d410e010 2024-03-21T10:36:24.087 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:24.087+0000 7f03d2a00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f03d4100d80 0x7f03d419a5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:36:24.087 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:24.087+0000 7f03d2a00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f03d4100d80 0x7f03d419a5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:58248/0 (socket says 172.21.15.47:58248) 2024-03-21T10:36:24.087 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:24.087+0000 7f03d2a00640 1 -- 172.21.15.47:0/374694752 learned_addr learned my addr 172.21.15.47:0/374694752 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:36:24.088 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:24.088+0000 7f03d3400640 1 --2- 172.21.15.47:0/374694752 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f03d4100460 0x7f03d419a070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:36:24.088 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:24.088+0000 7f03d3e00640 1 --2- 172.21.15.47:0/374694752 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f03d410e010 0x7f03d419e930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:36:24.088 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:24.088+0000 7f03d2a00640 1 -- 172.21.15.47:0/374694752 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f03d410e010 msgr2=0x7f03d419e930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:36:24.088 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:24.088+0000 7f03d2a00640 1 --2- 172.21.15.47:0/374694752 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f03d410e010 0x7f03d419e930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:36:24.088 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:24.088+0000 7f03d2a00640 1 -- 172.21.15.47:0/374694752 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f03d4100460 msgr2=0x7f03d419a070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:36:24.088 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:24.088+0000 7f03d2a00640 1 --2- 172.21.15.47:0/374694752 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f03d4100460 0x7f03d419a070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:36:24.088 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:24.088+0000 7f03d2a00640 1 -- 172.21.15.47:0/374694752 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f03d419f060 con 0x7f03d4100d80 2024-03-21T10:36:24.088 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:24.088+0000 7f03d3e00640 1 --2- 172.21.15.47:0/374694752 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f03d410e010 0x7f03d419e930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T10:36:24.088 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:24.088+0000 7f03d3400640 1 --2- 172.21.15.47:0/374694752 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f03d4100460 0x7f03d419a070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T10:36:24.088 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:24.088+0000 7f03d2a00640 1 --2- 172.21.15.47:0/374694752 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f03d4100d80 0x7f03d419a5b0 secure :-1 s=READY pgs=1346 cs=0 l=1 rev1=1 crypto rx=0x7f03c000ea60 tx=0x7f03c000ef20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:36:24.088 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:24.089+0000 7f03cbe00640 1 -- 172.21.15.47:0/374694752 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03c000cde0 con 0x7f03d4100d80 2024-03-21T10:36:24.088 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:24.089+0000 7f03cbe00640 1 -- 172.21.15.47:0/374694752 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f03c00035f0 con 0x7f03d4100d80 2024-03-21T10:36:24.088 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:24.089+0000 7f03da200640 1 -- 172.21.15.47:0/374694752 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f03d419f350 con 0x7f03d4100d80 2024-03-21T10:36:24.088 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:24.089+0000 7f03cbe00640 1 -- 172.21.15.47:0/374694752 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03c0010690 con 0x7f03d4100d80 2024-03-21T10:36:24.088 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:24.089+0000 7f03da200640 1 -- 172.21.15.47:0/374694752 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f03d41a6be0 con 0x7f03d4100d80 2024-03-21T10:36:24.089 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:24.090+0000 7f03da200640 1 -- 172.21.15.47:0/374694752 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f03d406a490 con 0x7f03d4100d80 2024-03-21T10:36:24.093 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:24.096+0000 7f03cbe00640 1 -- 172.21.15.47:0/374694752 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 27) v1 ==== 137985+0+0 (secure 0 0 0) 0x7f03c00037a0 con 0x7f03d4100d80 2024-03-21T10:36:24.095 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:24.097+0000 7f03cbe00640 1 --2- 172.21.15.47:0/374694752 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f03b80a3090 0x7f03b80a5550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:36:24.095 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:24.098+0000 7f03cbe00640 1 -- 172.21.15.47:0/374694752 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(107..107 src has 1..107) v4 ==== 9562+0+0 (secure 0 0 0) 0x7f03c00d0280 con 0x7f03d4100d80 2024-03-21T10:36:24.096 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:24.098+0000 7f03d3400640 1 --2- 172.21.15.47:0/374694752 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f03b80a3090 0x7f03b80a5550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:36:24.096 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:24.098+0000 7f03cbe00640 1 -- 172.21.15.47:0/374694752 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f03c00d0700 con 0x7f03d4100d80 2024-03-21T10:36:24.096 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:24.099+0000 7f03d3400640 1 --2- 172.21.15.47:0/374694752 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f03b80a3090 0x7f03b80a5550 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f03c4006010 tx=0x7f03c4005a20 comp rx=0 tx=0).ready entity=mgr.15687 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:36:24.309 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:24.311+0000 7f03da200640 1 -- 172.21.15.47:0/374694752 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f03d419b3c0 con 0x7f03d4100d80 2024-03-21T10:36:24.309 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:24.312+0000 7f03cbe00640 1 -- 172.21.15.47:0/374694752 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v107) v1 ==== 100+0+11 (secure 0 0 0) 0x7f03c001f330 con 0x7f03d4100d80 2024-03-21T10:36:24.309 INFO:teuthology.orchestra.run.smithi047.stdout:pg_num: 64 2024-03-21T10:36:24.312 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:24.314+0000 7f03da200640 1 -- 172.21.15.47:0/374694752 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f03b80a3090 msgr2=0x7f03b80a5550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:36:24.313 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:24.314+0000 7f03da200640 1 --2- 172.21.15.47:0/374694752 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f03b80a3090 0x7f03b80a5550 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f03c4006010 tx=0x7f03c4005a20 comp rx=0 tx=0).stop 2024-03-21T10:36:24.313 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:24.314+0000 7f03da200640 1 -- 172.21.15.47:0/374694752 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f03d4100d80 msgr2=0x7f03d419a5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:36:24.313 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:24.314+0000 7f03da200640 1 --2- 172.21.15.47:0/374694752 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f03d4100d80 0x7f03d419a5b0 secure :-1 s=READY pgs=1346 cs=0 l=1 rev1=1 crypto rx=0x7f03c000ea60 tx=0x7f03c000ef20 comp rx=0 tx=0).stop 2024-03-21T10:36:24.313 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:24.315+0000 7f03da200640 1 -- 172.21.15.47:0/374694752 shutdown_connections 2024-03-21T10:36:24.313 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:24.315+0000 7f03da200640 1 --2- 172.21.15.47:0/374694752 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f03d410e010 0x7f03d419e930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:36:24.313 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:24.315+0000 7f03da200640 1 --2- 172.21.15.47:0/374694752 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f03d4100d80 0x7f03d419a5b0 unknown :-1 s=CLOSED pgs=1346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:36:24.313 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:24.315+0000 7f03da200640 1 --2- 172.21.15.47:0/374694752 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f03b80a3090 0x7f03b80a5550 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:36:24.313 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:24.315+0000 7f03da200640 1 --2- 172.21.15.47:0/374694752 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f03d4100460 0x7f03d419a070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:36:24.313 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:24.315+0000 7f03da200640 1 -- 172.21.15.47:0/374694752 >> 172.21.15.47:0/374694752 conn(0x7f03d4076920 msgr2=0x7f03d410f5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:36:24.313 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:24.315+0000 7f03da200640 1 -- 172.21.15.47:0/374694752 shutdown_connections 2024-03-21T10:36:24.313 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:24.315+0000 7f03da200640 1 -- 172.21.15.47:0/374694752 wait complete. 2024-03-21T10:36:24.332 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-03-21T10:36:24.533 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:24.534+0000 7f7f95a00640 1 -- 172.21.15.47:0/3510211265 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7f90074360 msgr2=0x7f7f90111fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:36:24.533 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:24.534+0000 7f7f95a00640 1 --2- 172.21.15.47:0/3510211265 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7f90074360 0x7f7f90111fe0 secure :-1 s=READY pgs=1347 cs=0 l=1 rev1=1 crypto rx=0x7f7f78009900 tx=0x7f7f7802bcc0 comp rx=0 tx=0).stop 2024-03-21T10:36:24.533 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:24.535+0000 7f7f95a00640 1 -- 172.21.15.47:0/3510211265 shutdown_connections 2024-03-21T10:36:24.533 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:24.535+0000 7f7f95a00640 1 --2- 172.21.15.47:0/3510211265 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7f90112520 0x7f7f90114910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:36:24.533 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:24.535+0000 7f7f95a00640 1 --2- 172.21.15.47:0/3510211265 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7f90074360 0x7f7f90111fe0 unknown :-1 s=CLOSED pgs=1347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:36:24.533 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:24.535+0000 7f7f95a00640 1 --2- 172.21.15.47:0/3510211265 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7f90073a40 0x7f7f90073e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:36:24.533 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:24.535+0000 7f7f95a00640 1 -- 172.21.15.47:0/3510211265 >> 172.21.15.47:0/3510211265 conn(0x7f7f900fd040 msgr2=0x7f7f900ff460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:36:24.533 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:24.535+0000 7f7f95a00640 1 -- 172.21.15.47:0/3510211265 shutdown_connections 2024-03-21T10:36:24.533 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:24.535+0000 7f7f95a00640 1 -- 172.21.15.47:0/3510211265 wait complete. 2024-03-21T10:36:24.536 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:24.536+0000 7f7f95a00640 1 Processor -- start 2024-03-21T10:36:24.536 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:24.536+0000 7f7f95a00640 1 -- start start 2024-03-21T10:36:24.536 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:24.536+0000 7f7f95a00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7f90073a40 0x7f7f9019e360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:36:24.536 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:24.536+0000 7f7f95a00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7f90074360 0x7f7f9019e8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:36:24.536 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:24.536+0000 7f7f95a00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7f90112520 0x7f7f901a2c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:36:24.536 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:24.536+0000 7f7f95a00640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f90117e20 con 0x7f7f90073a40 2024-03-21T10:36:24.536 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:24.536+0000 7f7f95a00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f90117ca0 con 0x7f7f90112520 2024-03-21T10:36:24.536 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:24.536+0000 7f7f95a00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f90117fa0 con 0x7f7f90074360 2024-03-21T10:36:24.536 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:24.537+0000 7f7f8fe00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7f90073a40 0x7f7f9019e360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:36:24.536 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:24.537+0000 7f7f8fe00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7f90073a40 0x7f7f9019e360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:58266/0 (socket says 172.21.15.47:58266) 2024-03-21T10:36:24.536 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:24.537+0000 7f7f8fe00640 1 -- 172.21.15.47:0/3808079081 learned_addr learned my addr 172.21.15.47:0/3808079081 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:36:24.536 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:24.537+0000 7f7f95000640 1 --2- 172.21.15.47:0/3808079081 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7f90112520 0x7f7f901a2c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:36:24.536 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:24.537+0000 7f7f87e00640 1 --2- 172.21.15.47:0/3808079081 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7f90074360 0x7f7f9019e8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:36:24.537 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:24.537+0000 7f7f8fe00640 1 -- 172.21.15.47:0/3808079081 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7f90074360 msgr2=0x7f7f9019e8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:36:24.537 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:24.537+0000 7f7f8fe00640 1 --2- 172.21.15.47:0/3808079081 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7f90074360 0x7f7f9019e8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:36:24.537 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:24.537+0000 7f7f8fe00640 1 -- 172.21.15.47:0/3808079081 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7f90112520 msgr2=0x7f7f901a2c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:36:24.537 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:24.537+0000 7f7f8fe00640 1 --2- 172.21.15.47:0/3808079081 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7f90112520 0x7f7f901a2c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:36:24.537 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:24.537+0000 7f7f8fe00640 1 -- 172.21.15.47:0/3808079081 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f901a3350 con 0x7f7f90073a40 2024-03-21T10:36:24.537 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:24.537+0000 7f7f95000640 1 --2- 172.21.15.47:0/3808079081 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7f90112520 0x7f7f901a2c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T10:36:24.537 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:24.538+0000 7f7f8fe00640 1 --2- 172.21.15.47:0/3808079081 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7f90073a40 0x7f7f9019e360 secure :-1 s=READY pgs=1348 cs=0 l=1 rev1=1 crypto rx=0x7f7f8000ebc0 tx=0x7f7f8000c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:36:24.537 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:24.538+0000 7f7f8d600640 1 -- 172.21.15.47:0/3808079081 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f80010630 con 0x7f7f90073a40 2024-03-21T10:36:24.537 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:24.538+0000 7f7f8d600640 1 -- 172.21.15.47:0/3808079081 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f7f80010c80 con 0x7f7f90073a40 2024-03-21T10:36:24.537 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:24.538+0000 7f7f95a00640 1 -- 172.21.15.47:0/3808079081 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7f901a3640 con 0x7f7f90073a40 2024-03-21T10:36:24.537 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:24.538+0000 7f7f8d600640 1 -- 172.21.15.47:0/3808079081 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f80018b30 con 0x7f7f90073a40 2024-03-21T10:36:24.537 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:24.538+0000 7f7f95a00640 1 -- 172.21.15.47:0/3808079081 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7f901aaed0 con 0x7f7f90073a40 2024-03-21T10:36:24.538 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:24.539+0000 7f7f95a00640 1 -- 172.21.15.47:0/3808079081 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7f900683b0 con 0x7f7f90073a40 2024-03-21T10:36:24.539 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:24.541+0000 7f7f8d600640 1 -- 172.21.15.47:0/3808079081 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 27) v1 ==== 137985+0+0 (secure 0 0 0) 0x7f7f80018cd0 con 0x7f7f90073a40 2024-03-21T10:36:24.540 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:24.543+0000 7f7f8d600640 1 --2- 172.21.15.47:0/3808079081 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f7f740a3090 0x7f7f740a5550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:36:24.541 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:24.543+0000 7f7f8d600640 1 -- 172.21.15.47:0/3808079081 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(107..107 src has 1..107) v4 ==== 9562+0+0 (secure 0 0 0) 0x7f7f800d0300 con 0x7f7f90073a40 2024-03-21T10:36:24.541 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:24.543+0000 7f7f87e00640 1 --2- 172.21.15.47:0/3808079081 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f7f740a3090 0x7f7f740a5550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:36:24.541 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:24.544+0000 7f7f87e00640 1 --2- 172.21.15.47:0/3808079081 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f7f740a3090 0x7f7f740a5550 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f7f78006010 tx=0x7f7f78000bc0 comp rx=0 tx=0).ready entity=mgr.15687 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:36:24.548 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:24.551+0000 7f7f8d600640 1 -- 172.21.15.47:0/3808079081 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f7f8009c980 con 0x7f7f90073a40 2024-03-21T10:36:24.710 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:36:24 smithi047 ceph-mon[31547]: from='client.? 172.21.15.47:0/1542307046' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-21T10:36:24.710 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:36:24 smithi047 ceph-mon[31547]: from='client.? 172.21.15.47:0/1293107382' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-21T10:36:24.710 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:36:24 smithi047 ceph-mon[31547]: from='client.? 172.21.15.47:0/374694752' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-21T10:36:24.726 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:36:24 smithi100 ceph-mon[35193]: from='client.? 172.21.15.47:0/1542307046' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-21T10:36:24.726 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:36:24 smithi100 ceph-mon[35193]: from='client.? 172.21.15.47:0/1293107382' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-21T10:36:24.726 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:36:24 smithi100 ceph-mon[35193]: from='client.? 172.21.15.47:0/374694752' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-21T10:36:24.736 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:36:24 smithi157 ceph-mon[35581]: from='client.? 172.21.15.47:0/1542307046' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-21T10:36:24.736 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:36:24 smithi157 ceph-mon[35581]: from='client.? 172.21.15.47:0/1293107382' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-21T10:36:24.736 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:36:24 smithi157 ceph-mon[35581]: from='client.? 172.21.15.47:0/374694752' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-21T10:36:24.766 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:24.768+0000 7f7f95a00640 1 -- 172.21.15.47:0/3808079081 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f7f901049b0 con 0x7f7f90073a40 2024-03-21T10:36:24.766 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:24.769+0000 7f7f8d600640 1 -- 172.21.15.47:0/3808079081 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v107) v1 ==== 103+0+11 (secure 0 0 0) 0x7f7f8009c280 con 0x7f7f90073a40 2024-03-21T10:36:24.767 INFO:teuthology.orchestra.run.smithi047.stdout:pg_num: 32 2024-03-21T10:36:24.769 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:24.771+0000 7f7f95a00640 1 -- 172.21.15.47:0/3808079081 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f7f740a3090 msgr2=0x7f7f740a5550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:36:24.769 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:24.771+0000 7f7f95a00640 1 --2- 172.21.15.47:0/3808079081 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f7f740a3090 0x7f7f740a5550 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f7f78006010 tx=0x7f7f78000bc0 comp rx=0 tx=0).stop 2024-03-21T10:36:24.769 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:24.771+0000 7f7f95a00640 1 -- 172.21.15.47:0/3808079081 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7f90073a40 msgr2=0x7f7f9019e360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:36:24.770 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:24.771+0000 7f7f95a00640 1 --2- 172.21.15.47:0/3808079081 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7f90073a40 0x7f7f9019e360 secure :-1 s=READY pgs=1348 cs=0 l=1 rev1=1 crypto rx=0x7f7f8000ebc0 tx=0x7f7f8000c6a0 comp rx=0 tx=0).stop 2024-03-21T10:36:24.770 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:24.772+0000 7f7f95a00640 1 -- 172.21.15.47:0/3808079081 shutdown_connections 2024-03-21T10:36:24.770 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:24.772+0000 7f7f95a00640 1 --2- 172.21.15.47:0/3808079081 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f7f740a3090 0x7f7f740a5550 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:36:24.770 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:24.772+0000 7f7f95a00640 1 --2- 172.21.15.47:0/3808079081 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7f90112520 0x7f7f901a2c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:36:24.770 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:24.772+0000 7f7f95a00640 1 --2- 172.21.15.47:0/3808079081 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7f90074360 0x7f7f9019e8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:36:24.770 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:24.772+0000 7f7f95a00640 1 --2- 172.21.15.47:0/3808079081 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7f90073a40 0x7f7f9019e360 unknown :-1 s=CLOSED pgs=1348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:36:24.770 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:24.772+0000 7f7f95a00640 1 -- 172.21.15.47:0/3808079081 >> 172.21.15.47:0/3808079081 conn(0x7f7f900fd040 msgr2=0x7f7f900fe8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:36:24.770 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:24.772+0000 7f7f95a00640 1 -- 172.21.15.47:0/3808079081 shutdown_connections 2024-03-21T10:36:24.770 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:24.772+0000 7f7f95a00640 1 -- 172.21.15.47:0/3808079081 wait complete. 2024-03-21T10:36:24.788 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-03-21T10:36:24.993 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:24.995+0000 7f901d400640 1 -- 172.21.15.47:0/2037456518 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f90181004c0 msgr2=0x7f90181008a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:36:25.087 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:24.995+0000 7f901d400640 1 --2- 172.21.15.47:0/2037456518 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f90181004c0 0x7f90181008a0 secure :-1 s=READY pgs=1349 cs=0 l=1 rev1=1 crypto rx=0x7f9008009900 tx=0x7f900802bd10 comp rx=0 tx=0).stop 2024-03-21T10:36:25.087 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:24.995+0000 7f901d400640 1 -- 172.21.15.47:0/2037456518 shutdown_connections 2024-03-21T10:36:25.087 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:24.995+0000 7f901d400640 1 --2- 172.21.15.47:0/2037456518 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f90181058c0 0x7f9018105d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:36:25.087 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:24.995+0000 7f901d400640 1 --2- 172.21.15.47:0/2037456518 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9018100de0 0x7f90181052a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:36:25.087 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:24.995+0000 7f901d400640 1 --2- 172.21.15.47:0/2037456518 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f90181004c0 0x7f90181008a0 unknown :-1 s=CLOSED pgs=1349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:36:25.087 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:24.995+0000 7f901d400640 1 -- 172.21.15.47:0/2037456518 >> 172.21.15.47:0/2037456518 conn(0x7f90180fbef0 msgr2=0x7f90180fe310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:36:25.087 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:24.995+0000 7f901d400640 1 -- 172.21.15.47:0/2037456518 shutdown_connections 2024-03-21T10:36:25.087 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:24.995+0000 7f901d400640 1 -- 172.21.15.47:0/2037456518 wait complete. 2024-03-21T10:36:25.090 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:24.996+0000 7f901d400640 1 Processor -- start 2024-03-21T10:36:25.090 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:24.996+0000 7f901d400640 1 -- start start 2024-03-21T10:36:25.090 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:24.996+0000 7f901d400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f90181004c0 0x7f901819e310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:36:25.090 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:24.996+0000 7f901d400640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9018100de0 0x7f901819e850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:36:25.090 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:24.996+0000 7f901d400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f90181058c0 0x7f90181a2bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:36:25.091 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:24.996+0000 7f901d400640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9018117db0 con 0x7f90181058c0 2024-03-21T10:36:25.091 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:24.997+0000 7f901d400640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9018117c30 con 0x7f90181004c0 2024-03-21T10:36:25.091 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:24.997+0000 7f901d400640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9018117f30 con 0x7f9018100de0 2024-03-21T10:36:25.091 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:24.997+0000 7f901ca00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f90181058c0 0x7f90181a2bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:36:25.091 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:24.997+0000 7f901ca00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f90181058c0 0x7f90181a2bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:58282/0 (socket says 172.21.15.47:58282) 2024-03-21T10:36:25.091 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:24.997+0000 7f901ca00640 1 -- 172.21.15.47:0/2594436382 learned_addr learned my addr 172.21.15.47:0/2594436382 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:36:25.091 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:24.997+0000 7f9017e00640 1 --2- 172.21.15.47:0/2594436382 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f90181004c0 0x7f901819e310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:36:25.091 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:24.997+0000 7f900f400640 1 --2- 172.21.15.47:0/2594436382 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9018100de0 0x7f901819e850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:36:25.091 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:24.997+0000 7f901ca00640 1 -- 172.21.15.47:0/2594436382 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9018100de0 msgr2=0x7f901819e850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:36:25.091 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:24.997+0000 7f901ca00640 1 --2- 172.21.15.47:0/2594436382 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9018100de0 0x7f901819e850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:36:25.091 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:24.997+0000 7f901ca00640 1 -- 172.21.15.47:0/2594436382 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f90181004c0 msgr2=0x7f901819e310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:36:25.091 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:24.997+0000 7f901ca00640 1 --2- 172.21.15.47:0/2594436382 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f90181004c0 0x7f901819e310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:36:25.091 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:24.997+0000 7f901ca00640 1 -- 172.21.15.47:0/2594436382 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f90181a3300 con 0x7f90181058c0 2024-03-21T10:36:25.092 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:24.998+0000 7f901ca00640 1 --2- 172.21.15.47:0/2594436382 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f90181058c0 0x7f90181a2bd0 secure :-1 s=READY pgs=1350 cs=0 l=1 rev1=1 crypto rx=0x7f901000eaa0 tx=0x7f901000ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:36:25.092 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:24.998+0000 7f9015600640 1 -- 172.21.15.47:0/2594436382 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f901000cdb0 con 0x7f90181058c0 2024-03-21T10:36:25.092 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:24.998+0000 7f9015600640 1 -- 172.21.15.47:0/2594436382 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f9010005020 con 0x7f90181058c0 2024-03-21T10:36:25.092 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:24.998+0000 7f9015600640 1 -- 172.21.15.47:0/2594436382 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90100116d0 con 0x7f90181058c0 2024-03-21T10:36:25.092 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:24.998+0000 7f901d400640 1 -- 172.21.15.47:0/2594436382 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f90181a35f0 con 0x7f90181058c0 2024-03-21T10:36:25.092 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:24.998+0000 7f901d400640 1 -- 172.21.15.47:0/2594436382 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f90181aae80 con 0x7f90181058c0 2024-03-21T10:36:25.092 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:24.999+0000 7f901d400640 1 -- 172.21.15.47:0/2594436382 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f90180683b0 con 0x7f90181058c0 2024-03-21T10:36:25.092 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:25.001+0000 7f9015600640 1 -- 172.21.15.47:0/2594436382 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 27) v1 ==== 137985+0+0 (secure 0 0 0) 0x7f90100051d0 con 0x7f90181058c0 2024-03-21T10:36:25.093 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:25.002+0000 7f9015600640 1 --2- 172.21.15.47:0/2594436382 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f8ffc0a3110 0x7f8ffc0a55d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:36:25.093 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:25.003+0000 7f9015600640 1 -- 172.21.15.47:0/2594436382 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(107..107 src has 1..107) v4 ==== 9562+0+0 (secure 0 0 0) 0x7f90100cfd40 con 0x7f90181058c0 2024-03-21T10:36:25.093 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:25.003+0000 7f9017e00640 1 --2- 172.21.15.47:0/2594436382 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f8ffc0a3110 0x7f8ffc0a55d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:36:25.093 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:25.003+0000 7f9017e00640 1 --2- 172.21.15.47:0/2594436382 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f8ffc0a3110 0x7f8ffc0a55d0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f9008006010 tx=0x7f9008005e40 comp rx=0 tx=0).ready entity=mgr.15687 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:36:25.093 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:25.007+0000 7f9015600640 1 -- 172.21.15.47:0/2594436382 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f9010016030 con 0x7f90181058c0 2024-03-21T10:36:25.222 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:25.224+0000 7f901d400640 1 -- 172.21.15.47:0/2594436382 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f90181ab0e0 con 0x7f90181058c0 2024-03-21T10:36:25.223 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:25.225+0000 7f9015600640 1 -- 172.21.15.47:0/2594436382 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 16 entries v107) v1 ==== 81+0+943 (secure 0 0 0) 0x7f901009c3c0 con 0x7f90181058c0 2024-03-21T10:36:25.223 INFO:teuthology.orchestra.run.smithi047.stdout:172.21.15.47:0/936039355 2024-03-22T10:25:19.912777+0000 2024-03-21T10:36:25.223 INFO:teuthology.orchestra.run.smithi047.stdout:172.21.15.47:0/2983192238 2024-03-22T10:25:19.912777+0000 2024-03-21T10:36:25.223 INFO:teuthology.orchestra.run.smithi047.stdout:172.21.15.47:6801/2823885994 2024-03-22T10:25:19.912777+0000 2024-03-21T10:36:25.223 INFO:teuthology.orchestra.run.smithi047.stdout:172.21.15.47:0/356822431 2024-03-22T10:12:25.844530+0000 2024-03-21T10:36:25.223 INFO:teuthology.orchestra.run.smithi047.stdout:172.21.15.47:0/1167490879 2024-03-22T10:12:25.844530+0000 2024-03-21T10:36:25.223 INFO:teuthology.orchestra.run.smithi047.stdout:172.21.15.47:0/1540718138 2024-03-22T10:12:25.844530+0000 2024-03-21T10:36:25.224 INFO:teuthology.orchestra.run.smithi047.stdout:172.21.15.47:6801/3836903972 2024-03-22T10:12:25.844530+0000 2024-03-21T10:36:25.224 INFO:teuthology.orchestra.run.smithi047.stdout:172.21.15.47:0/235436602 2024-03-22T10:11:50.301838+0000 2024-03-21T10:36:25.224 INFO:teuthology.orchestra.run.smithi047.stdout:172.21.15.47:0/2052815572 2024-03-22T10:11:50.301838+0000 2024-03-21T10:36:25.224 INFO:teuthology.orchestra.run.smithi047.stdout:172.21.15.47:0/1607664848 2024-03-22T10:25:19.912777+0000 2024-03-21T10:36:25.224 INFO:teuthology.orchestra.run.smithi047.stdout:172.21.15.47:6800/3942658432 2024-03-22T10:11:50.301838+0000 2024-03-21T10:36:25.224 INFO:teuthology.orchestra.run.smithi047.stdout:172.21.15.47:0/3101796817 2024-03-22T10:25:19.912777+0000 2024-03-21T10:36:25.224 INFO:teuthology.orchestra.run.smithi047.stdout:172.21.15.47:6801/3942658432 2024-03-22T10:11:50.301838+0000 2024-03-21T10:36:25.224 INFO:teuthology.orchestra.run.smithi047.stdout:172.21.15.47:6800/2823885994 2024-03-22T10:25:19.912777+0000 2024-03-21T10:36:25.224 INFO:teuthology.orchestra.run.smithi047.stdout:172.21.15.47:0/1419516257 2024-03-22T10:11:50.301838+0000 2024-03-21T10:36:25.224 INFO:teuthology.orchestra.run.smithi047.stdout:172.21.15.47:6800/3836903972 2024-03-22T10:12:25.844530+0000 2024-03-21T10:36:25.225 INFO:teuthology.orchestra.run.smithi047.stderr:listed 16 entries 2024-03-21T10:36:25.226 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:25.227+0000 7f901d400640 1 -- 172.21.15.47:0/2594436382 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f8ffc0a3110 msgr2=0x7f8ffc0a55d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:36:25.226 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:25.227+0000 7f901d400640 1 --2- 172.21.15.47:0/2594436382 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f8ffc0a3110 0x7f8ffc0a55d0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f9008006010 tx=0x7f9008005e40 comp rx=0 tx=0).stop 2024-03-21T10:36:25.226 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:25.227+0000 7f901d400640 1 -- 172.21.15.47:0/2594436382 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f90181058c0 msgr2=0x7f90181a2bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:36:25.226 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:25.227+0000 7f901d400640 1 --2- 172.21.15.47:0/2594436382 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f90181058c0 0x7f90181a2bd0 secure :-1 s=READY pgs=1350 cs=0 l=1 rev1=1 crypto rx=0x7f901000eaa0 tx=0x7f901000ef60 comp rx=0 tx=0).stop 2024-03-21T10:36:25.226 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:25.228+0000 7f901d400640 1 -- 172.21.15.47:0/2594436382 shutdown_connections 2024-03-21T10:36:25.226 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:25.228+0000 7f901d400640 1 --2- 172.21.15.47:0/2594436382 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f90181058c0 0x7f90181a2bd0 unknown :-1 s=CLOSED pgs=1350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:36:25.226 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:25.228+0000 7f901d400640 1 --2- 172.21.15.47:0/2594436382 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9018100de0 0x7f901819e850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:36:25.226 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:25.228+0000 7f901d400640 1 --2- 172.21.15.47:0/2594436382 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f8ffc0a3110 0x7f8ffc0a55d0 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:36:25.227 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:25.228+0000 7f901d400640 1 --2- 172.21.15.47:0/2594436382 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f90181004c0 0x7f901819e310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:36:25.227 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:25.228+0000 7f901d400640 1 -- 172.21.15.47:0/2594436382 >> 172.21.15.47:0/2594436382 conn(0x7f90180fbef0 msgr2=0x7f9018103270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:36:25.227 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:25.228+0000 7f901d400640 1 -- 172.21.15.47:0/2594436382 shutdown_connections 2024-03-21T10:36:25.227 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:25.228+0000 7f901d400640 1 -- 172.21.15.47:0/2594436382 wait complete. 2024-03-21T10:36:25.243 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi047.front.sepia.ceph.com... 2024-03-21T10:36:25.243 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-21T10:36:25.243 DEBUG:teuthology.orchestra.run.smithi047:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2024-03-21T10:36:25.338 INFO:teuthology.orchestra.run:waiting for 300 2024-03-21T10:36:25.343 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi047.stderr:ceph-fuse[83961]: fuse finished with error 0 and tester_r 0 2024-03-21T10:36:25.710 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:36:25 smithi047 ceph-mon[31547]: pgmap v346: 129 pgs: 129 active+clean; 2.5 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 98 B/s wr, 9 op/s 2024-03-21T10:36:25.711 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:36:25 smithi047 ceph-mon[31547]: from='client.? 172.21.15.47:0/3808079081' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-03-21T10:36:25.711 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:36:25 smithi047 ceph-mon[31547]: from='client.? 172.21.15.47:0/2594436382' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-03-21T10:36:25.726 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:36:25 smithi100 ceph-mon[35193]: pgmap v346: 129 pgs: 129 active+clean; 2.5 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 98 B/s wr, 9 op/s 2024-03-21T10:36:25.726 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:36:25 smithi100 ceph-mon[35193]: from='client.? 172.21.15.47:0/3808079081' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-03-21T10:36:25.726 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:36:25 smithi100 ceph-mon[35193]: from='client.? 172.21.15.47:0/2594436382' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-03-21T10:36:25.736 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:36:25 smithi157 ceph-mon[35581]: pgmap v346: 129 pgs: 129 active+clean; 2.5 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 98 B/s wr, 9 op/s 2024-03-21T10:36:25.736 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:36:25 smithi157 ceph-mon[35581]: from='client.? 172.21.15.47:0/3808079081' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-03-21T10:36:25.736 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:36:25 smithi157 ceph-mon[35581]: from='client.? 172.21.15.47:0/2594436382' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-03-21T10:36:27.710 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:36:27 smithi047 ceph-mon[31547]: pgmap v347: 129 pgs: 129 active+clean; 2.5 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 938 B/s wr, 13 op/s 2024-03-21T10:36:27.726 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:36:27 smithi100 ceph-mon[35193]: pgmap v347: 129 pgs: 129 active+clean; 2.5 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 938 B/s wr, 13 op/s 2024-03-21T10:36:27.736 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:36:27 smithi157 ceph-mon[35581]: pgmap v347: 129 pgs: 129 active+clean; 2.5 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 938 B/s wr, 13 op/s 2024-03-21T10:36:29.710 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:36:29 smithi047 ceph-mon[31547]: pgmap v348: 129 pgs: 129 active+clean; 2.5 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 938 B/s wr, 9 op/s 2024-03-21T10:36:29.726 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:36:29 smithi100 ceph-mon[35193]: pgmap v348: 129 pgs: 129 active+clean; 2.5 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 938 B/s wr, 9 op/s 2024-03-21T10:36:29.736 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:36:29 smithi157 ceph-mon[35581]: pgmap v348: 129 pgs: 129 active+clean; 2.5 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 938 B/s wr, 9 op/s 2024-03-21T10:36:31.440 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi047.front.sepia.ceph.com 2024-03-21T10:36:31.440 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-21T10:36:31.440 DEBUG:teuthology.orchestra.run.smithi047:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-03-21T10:36:31.502 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-21T10:36:31.502 DEBUG:teuthology.orchestra.run.smithi047:> ip netns list 2024-03-21T10:36:31.553 INFO:teuthology.orchestra.run.smithi047.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-03-21T10:36:31.554 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-21T10:36:31.554 DEBUG:teuthology.orchestra.run.smithi047:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.0 2024-03-21T10:36:31.636 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:36:31 smithi047 ceph-mon[31547]: pgmap v349: 129 pgs: 129 active+clean; 2.5 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 938 B/s wr, 9 op/s 2024-03-21T10:36:31.653 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-21T10:36:31.653 DEBUG:teuthology.orchestra.run.smithi047:> sudo ip link delete ceph-brx 2024-03-21T10:36:31.726 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:36:31 smithi100 ceph-mon[35193]: pgmap v349: 129 pgs: 129 active+clean; 2.5 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 938 B/s wr, 9 op/s 2024-03-21T10:36:31.736 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:36:31 smithi157 ceph-mon[35581]: pgmap v349: 129 pgs: 129 active+clean; 2.5 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 938 B/s wr, 9 op/s 2024-03-21T10:36:31.763 DEBUG:teuthology.run_tasks:Unwinding manager cephadm 2024-03-21T10:36:31.774 INFO:tasks.cephadm:Teardown begin 2024-03-21T10:36:31.774 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-03-21T10:36:31.775 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-21T10:36:31.976 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:31.975+0000 7f2c64e00640 1 -- 172.21.15.47:0/733147598 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2c601014b0 msgr2=0x7f2c60101930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:36:31.976 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:31.975+0000 7f2c64e00640 1 --2- 172.21.15.47:0/733147598 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2c601014b0 0x7f2c60101930 secure :-1 s=READY pgs=1351 cs=0 l=1 rev1=1 crypto rx=0x7f2c4c009900 tx=0x7f2c4c02bc40 comp rx=0 tx=0).stop 2024-03-21T10:36:31.977 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:31.975+0000 7f2c64e00640 1 -- 172.21.15.47:0/733147598 shutdown_connections 2024-03-21T10:36:31.977 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:31.975+0000 7f2c64e00640 1 --2- 172.21.15.47:0/733147598 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2c60101e70 0x7f2c6010e330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:36:31.977 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:31.975+0000 7f2c64e00640 1 --2- 172.21.15.47:0/733147598 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2c601014b0 0x7f2c60101930 unknown :-1 s=CLOSED pgs=1351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:36:31.977 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:31.975+0000 7f2c64e00640 1 --2- 172.21.15.47:0/733147598 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2c60107570 0x7f2c60107950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:36:31.977 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:31.975+0000 7f2c64e00640 1 -- 172.21.15.47:0/733147598 >> 172.21.15.47:0/733147598 conn(0x7f2c600fcf80 msgr2=0x7f2c600ff3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:36:31.977 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:31.975+0000 7f2c64e00640 1 -- 172.21.15.47:0/733147598 shutdown_connections 2024-03-21T10:36:31.977 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:31.975+0000 7f2c64e00640 1 -- 172.21.15.47:0/733147598 wait complete. 2024-03-21T10:36:31.977 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:31.976+0000 7f2c64e00640 1 Processor -- start 2024-03-21T10:36:31.978 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:31.976+0000 7f2c64e00640 1 -- start start 2024-03-21T10:36:31.978 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:31.976+0000 7f2c64e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2c601014b0 0x7f2c600739a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:36:31.978 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:31.977+0000 7f2c64e00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2c60101e70 0x7f2c60073ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:36:31.978 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:31.977+0000 7f2c64e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2c60107570 0x7f2c60074420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:36:31.978 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:31.977+0000 7f2c64e00640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c60111750 con 0x7f2c601014b0 2024-03-21T10:36:31.978 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:31.977+0000 7f2c64e00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c601115d0 con 0x7f2c60107570 2024-03-21T10:36:31.978 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:31.977+0000 7f2c64e00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c601118d0 con 0x7f2c60101e70 2024-03-21T10:36:31.978 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:31.977+0000 7f2c5f400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2c601014b0 0x7f2c600739a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:36:31.978 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:31.977+0000 7f2c5f400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2c601014b0 0x7f2c600739a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:34638/0 (socket says 172.21.15.47:34638) 2024-03-21T10:36:31.978 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:31.977+0000 7f2c5f400640 1 -- 172.21.15.47:0/3059799533 learned_addr learned my addr 172.21.15.47:0/3059799533 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:36:31.978 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:31.977+0000 7f2c5fe00640 1 --2- 172.21.15.47:0/3059799533 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2c60107570 0x7f2c60074420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:36:31.979 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:31.977+0000 7f2c5ea00640 1 --2- 172.21.15.47:0/3059799533 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2c60101e70 0x7f2c60073ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:36:31.979 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:31.977+0000 7f2c5f400640 1 -- 172.21.15.47:0/3059799533 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2c60101e70 msgr2=0x7f2c60073ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:36:31.979 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:31.977+0000 7f2c5f400640 1 --2- 172.21.15.47:0/3059799533 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2c60101e70 0x7f2c60073ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:36:31.979 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:31.977+0000 7f2c5f400640 1 -- 172.21.15.47:0/3059799533 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2c60107570 msgr2=0x7f2c60074420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:36:31.979 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:31.977+0000 7f2c5f400640 1 --2- 172.21.15.47:0/3059799533 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2c60107570 0x7f2c60074420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:36:31.979 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:31.977+0000 7f2c5f400640 1 -- 172.21.15.47:0/3059799533 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c601a60d0 con 0x7f2c601014b0 2024-03-21T10:36:31.979 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:31.978+0000 7f2c5f400640 1 --2- 172.21.15.47:0/3059799533 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2c601014b0 0x7f2c600739a0 secure :-1 s=READY pgs=1352 cs=0 l=1 rev1=1 crypto rx=0x7f2c5000e9f0 tx=0x7f2c5000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:36:31.979 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:31.978+0000 7f2c57e00640 1 -- 172.21.15.47:0/3059799533 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c5000cd80 con 0x7f2c601014b0 2024-03-21T10:36:31.979 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:31.978+0000 7f2c57e00640 1 -- 172.21.15.47:0/3059799533 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f2c500108e0 con 0x7f2c601014b0 2024-03-21T10:36:31.979 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:31.978+0000 7f2c57e00640 1 -- 172.21.15.47:0/3059799533 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c50018920 con 0x7f2c601014b0 2024-03-21T10:36:31.979 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:31.978+0000 7f2c64e00640 1 -- 172.21.15.47:0/3059799533 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2c601a63c0 con 0x7f2c601014b0 2024-03-21T10:36:31.979 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:31.978+0000 7f2c64e00640 1 -- 172.21.15.47:0/3059799533 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2c601a6890 con 0x7f2c601014b0 2024-03-21T10:36:31.980 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:31.979+0000 7f2c64e00640 1 -- 172.21.15.47:0/3059799533 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2c600683b0 con 0x7f2c601014b0 2024-03-21T10:36:31.980 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:31.982+0000 7f2c57e00640 1 -- 172.21.15.47:0/3059799533 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 27) v1 ==== 137985+0+0 (secure 0 0 0) 0x7f2c50010a90 con 0x7f2c601014b0 2024-03-21T10:36:31.981 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:31.983+0000 7f2c57e00640 1 --2- 172.21.15.47:0/3059799533 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f2c440a3110 0x7f2c440a55d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:36:31.981 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:31.983+0000 7f2c57e00640 1 -- 172.21.15.47:0/3059799533 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(107..107 src has 1..107) v4 ==== 9562+0+0 (secure 0 0 0) 0x7f2c500d03a0 con 0x7f2c601014b0 2024-03-21T10:36:31.981 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:31.983+0000 7f2c5ea00640 1 --2- 172.21.15.47:0/3059799533 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f2c440a3110 0x7f2c440a55d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:36:31.981 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:31.984+0000 7f2c5ea00640 1 --2- 172.21.15.47:0/3059799533 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f2c440a3110 0x7f2c440a55d0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f2c4c006010 tx=0x7f2c4c000bc0 comp rx=0 tx=0).ready entity=mgr.15687 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:36:31.987 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:31.990+0000 7f2c57e00640 1 -- 172.21.15.47:0/3059799533 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f2c5009ca20 con 0x7f2c601014b0 2024-03-21T10:36:32.233 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:32.235+0000 7f2c64e00640 1 -- 172.21.15.47:0/3059799533 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f2c60074ed0 con 0x7f2c601014b0 2024-03-21T10:36:32.235 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:32.237+0000 7f2c57e00640 1 -- 172.21.15.47:0/3059799533 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 32 v32) v1 ==== 94+0+11471 (secure 0 0 0) 0x7f2c5009c320 con 0x7f2c601014b0 2024-03-21T10:36:32.235 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-21T10:36:32.235 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":32,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14511,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6835/735724037","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6834","nonce":735724037},{"type":"v1","addr":"172.21.15.47:6835","nonce":735724037}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14529,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6837/2248191724","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6836","nonce":2248191724},{"type":"v1","addr":"172.21.15.47:6837","nonce":2248191724}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":32,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-21T10:21:58.329063+0000","modified":"2024-03-21T10:26:17.169892+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":34535,"mds_1":34493,"mds_2":34454,"mds_3":34427,"mds_4":14619},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14604":{"gid":14604,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.47:6839/2881335356","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6838","nonce":2881335356},{"type":"v1","addr":"172.21.15.47:6839","nonce":2881335356}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_14619":{"gid":14619,"name":"a","rank":4,"incarnation":27,"state":"up:active","state_seq":11,"addr":"172.21.15.47:6841/3401578978","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6840","nonce":3401578978},{"type":"v1","addr":"172.21.15.47:6841","nonce":3401578978}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24406":{"gid":24406,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.157:6833/4000010136","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":4000010136},{"type":"v1","addr":"172.21.15.157:6833","nonce":4000010136}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24457":{"gid":24457,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.157:6835/3902868651","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":3902868651},{"type":"v1","addr":"172.21.15.157:6835","nonce":3902868651}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24475":{"gid":24475,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.157:6837/1908087204","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6836","nonce":1908087204},{"type":"v1","addr":"172.21.15.157:6837","nonce":1908087204}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_25729":{"gid":25729,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.157:6839/1998408416","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6838","nonce":1998408416},{"type":"v1","addr":"172.21.15.157:6839","nonce":1998408416}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_34427":{"gid":34427,"name":"k","rank":3,"incarnation":25,"state":"up:active","state_seq":20,"addr":"172.21.15.100:6833/2723302038","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6832","nonce":2723302038},{"type":"v1","addr":"172.21.15.100:6833","nonce":2723302038}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_34454":{"gid":34454,"name":"e","rank":2,"incarnation":22,"state":"up:active","state_seq":17,"addr":"172.21.15.100:6835/2597849957","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":2597849957},{"type":"v1","addr":"172.21.15.100:6835","nonce":2597849957}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_34493":{"gid":34493,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":13,"addr":"172.21.15.100:6837/3497644162","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6836","nonce":3497644162},{"type":"v1","addr":"172.21.15.100:6837","nonce":3497644162}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_34535":{"gid":34535,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.100:6839/4173168440","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6838","nonce":4173168440},{"type":"v1","addr":"172.21.15.100:6839","nonce":4173168440}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":34535,"qdb_cluster":[34427,14619,34454,34535,34493]},"id":1}]} 2024-03-21T10:36:32.236 INFO:teuthology.orchestra.run.smithi047.stderr:dumped fsmap epoch 32 2024-03-21T10:36:32.238 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:32.240+0000 7f2c64e00640 1 -- 172.21.15.47:0/3059799533 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f2c440a3110 msgr2=0x7f2c440a55d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:36:32.238 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:32.240+0000 7f2c64e00640 1 --2- 172.21.15.47:0/3059799533 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f2c440a3110 0x7f2c440a55d0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f2c4c006010 tx=0x7f2c4c000bc0 comp rx=0 tx=0).stop 2024-03-21T10:36:32.238 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:32.240+0000 7f2c64e00640 1 -- 172.21.15.47:0/3059799533 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2c601014b0 msgr2=0x7f2c600739a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:36:32.238 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:32.240+0000 7f2c64e00640 1 --2- 172.21.15.47:0/3059799533 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2c601014b0 0x7f2c600739a0 secure :-1 s=READY pgs=1352 cs=0 l=1 rev1=1 crypto rx=0x7f2c5000e9f0 tx=0x7f2c5000eeb0 comp rx=0 tx=0).stop 2024-03-21T10:36:32.238 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:32.240+0000 7f2c64e00640 1 -- 172.21.15.47:0/3059799533 shutdown_connections 2024-03-21T10:36:32.238 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:32.240+0000 7f2c64e00640 1 --2- 172.21.15.47:0/3059799533 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f2c440a3110 0x7f2c440a55d0 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:36:32.238 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:32.240+0000 7f2c64e00640 1 --2- 172.21.15.47:0/3059799533 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2c60107570 0x7f2c60074420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:36:32.239 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:32.240+0000 7f2c64e00640 1 --2- 172.21.15.47:0/3059799533 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2c60101e70 0x7f2c60073ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:36:32.239 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:32.240+0000 7f2c64e00640 1 --2- 172.21.15.47:0/3059799533 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2c601014b0 0x7f2c600739a0 unknown :-1 s=CLOSED pgs=1352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:36:32.239 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:32.240+0000 7f2c64e00640 1 -- 172.21.15.47:0/3059799533 >> 172.21.15.47:0/3059799533 conn(0x7f2c600fcf80 msgr2=0x7f2c600fe7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:36:32.239 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:32.240+0000 7f2c64e00640 1 -- 172.21.15.47:0/3059799533 shutdown_connections 2024-03-21T10:36:32.239 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:32.240+0000 7f2c64e00640 1 -- 172.21.15.47:0/3059799533 wait complete. 2024-03-21T10:36:32.258 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-21T10:36:32.486 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:32.488+0000 7f2f29c00640 1 -- 172.21.15.47:0/1620150814 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2f24069d50 msgr2=0x7f2f2406a1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:36:32.487 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:32.488+0000 7f2f29c00640 1 --2- 172.21.15.47:0/1620150814 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2f24069d50 0x7f2f2406a1b0 secure :-1 s=READY pgs=1353 cs=0 l=1 rev1=1 crypto rx=0x7f2f10009900 tx=0x7f2f1002bcd0 comp rx=0 tx=0).stop 2024-03-21T10:36:32.487 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:32.488+0000 7f2f29c00640 1 -- 172.21.15.47:0/1620150814 shutdown_connections 2024-03-21T10:36:32.487 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:32.488+0000 7f2f29c00640 1 --2- 172.21.15.47:0/1620150814 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2f2406a6f0 0x7f2f2410e4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:36:32.487 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:32.488+0000 7f2f29c00640 1 --2- 172.21.15.47:0/1620150814 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2f24069d50 0x7f2f2406a1b0 unknown :-1 s=CLOSED pgs=1353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:36:32.487 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:32.488+0000 7f2f29c00640 1 --2- 172.21.15.47:0/1620150814 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2f241076a0 0x7f2f24107a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:36:32.487 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:32.488+0000 7f2f29c00640 1 -- 172.21.15.47:0/1620150814 >> 172.21.15.47:0/1620150814 conn(0x7f2f24077640 msgr2=0x7f2f24077a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:36:32.487 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:32.488+0000 7f2f29c00640 1 -- 172.21.15.47:0/1620150814 shutdown_connections 2024-03-21T10:36:32.487 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:32.488+0000 7f2f29c00640 1 -- 172.21.15.47:0/1620150814 wait complete. 2024-03-21T10:36:32.490 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:32.489+0000 7f2f29c00640 1 Processor -- start 2024-03-21T10:36:32.490 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:32.490+0000 7f2f29c00640 1 -- start start 2024-03-21T10:36:32.490 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:32.490+0000 7f2f29c00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2f24069d50 0x7f2f241005d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:36:32.490 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:32.490+0000 7f2f29c00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2f2406a6f0 0x7f2f24100b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:36:32.490 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:32.490+0000 7f2f29c00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2f241076a0 0x7f2f24101050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:36:32.490 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:32.490+0000 7f2f29c00640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f24111920 con 0x7f2f24069d50 2024-03-21T10:36:32.490 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:32.490+0000 7f2f29c00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f241117a0 con 0x7f2f2406a6f0 2024-03-21T10:36:32.490 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:32.490+0000 7f2f29c00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f24111aa0 con 0x7f2f241076a0 2024-03-21T10:36:32.491 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:32.490+0000 7f2f22a00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2f24069d50 0x7f2f241005d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:36:32.491 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:32.490+0000 7f2f22a00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2f24069d50 0x7f2f241005d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:34642/0 (socket says 172.21.15.47:34642) 2024-03-21T10:36:32.491 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:32.490+0000 7f2f22a00640 1 -- 172.21.15.47:0/656710606 learned_addr learned my addr 172.21.15.47:0/656710606 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:36:32.491 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:32.490+0000 7f2f22000640 1 --2- 172.21.15.47:0/656710606 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2f2406a6f0 0x7f2f24100b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:36:32.491 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:32.491+0000 7f2f23400640 1 --2- 172.21.15.47:0/656710606 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2f241076a0 0x7f2f24101050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:36:32.491 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:32.491+0000 7f2f22a00640 1 -- 172.21.15.47:0/656710606 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2f241076a0 msgr2=0x7f2f24101050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:36:32.491 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:32.491+0000 7f2f22a00640 1 --2- 172.21.15.47:0/656710606 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2f241076a0 0x7f2f24101050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:36:32.491 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:32.491+0000 7f2f22a00640 1 -- 172.21.15.47:0/656710606 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2f2406a6f0 msgr2=0x7f2f24100b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:36:32.491 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:32.491+0000 7f2f22a00640 1 --2- 172.21.15.47:0/656710606 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2f2406a6f0 0x7f2f24100b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:36:32.491 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:32.491+0000 7f2f22a00640 1 -- 172.21.15.47:0/656710606 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f241a62a0 con 0x7f2f24069d50 2024-03-21T10:36:32.491 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:32.491+0000 7f2f23400640 1 --2- 172.21.15.47:0/656710606 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2f241076a0 0x7f2f24101050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T10:36:32.491 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:32.491+0000 7f2f22000640 1 --2- 172.21.15.47:0/656710606 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2f2406a6f0 0x7f2f24100b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T10:36:32.492 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:32.491+0000 7f2f22a00640 1 --2- 172.21.15.47:0/656710606 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2f24069d50 0x7f2f241005d0 secure :-1 s=READY pgs=1354 cs=0 l=1 rev1=1 crypto rx=0x7f2f1400cce0 tx=0x7f2f1400a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:36:32.492 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:32.491+0000 7f2f1b400640 1 -- 172.21.15.47:0/656710606 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f1400f650 con 0x7f2f24069d50 2024-03-21T10:36:32.492 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:32.492+0000 7f2f1b400640 1 -- 172.21.15.47:0/656710606 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f2f1400fca0 con 0x7f2f24069d50 2024-03-21T10:36:32.492 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:32.492+0000 7f2f29c00640 1 -- 172.21.15.47:0/656710606 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2f241a6590 con 0x7f2f24069d50 2024-03-21T10:36:32.492 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:32.492+0000 7f2f1b400640 1 -- 172.21.15.47:0/656710606 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f14018b90 con 0x7f2f24069d50 2024-03-21T10:36:32.492 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:32.492+0000 7f2f29c00640 1 -- 172.21.15.47:0/656710606 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2f241a6b00 con 0x7f2f24069d50 2024-03-21T10:36:32.493 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:32.493+0000 7f2f29c00640 1 -- 172.21.15.47:0/656710606 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2f240688e0 con 0x7f2f24069d50 2024-03-21T10:36:32.493 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:32.495+0000 7f2f1b400640 1 -- 172.21.15.47:0/656710606 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 27) v1 ==== 137985+0+0 (secure 0 0 0) 0x7f2f1400f7f0 con 0x7f2f24069d50 2024-03-21T10:36:32.494 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:32.496+0000 7f2f1b400640 1 --2- 172.21.15.47:0/656710606 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f2f080a3140 0x7f2f080a5600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:36:32.494 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:32.497+0000 7f2f1b400640 1 -- 172.21.15.47:0/656710606 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(107..107 src has 1..107) v4 ==== 9562+0+0 (secure 0 0 0) 0x7f2f140d0070 con 0x7f2f24069d50 2024-03-21T10:36:32.494 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:32.497+0000 7f2f22000640 1 --2- 172.21.15.47:0/656710606 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f2f080a3140 0x7f2f080a5600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:36:32.495 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:32.497+0000 7f2f22000640 1 --2- 172.21.15.47:0/656710606 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f2f080a3140 0x7f2f080a5600 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f2f10005ab0 tx=0x7f2f10005a20 comp rx=0 tx=0).ready entity=mgr.15687 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:36:32.501 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:32.504+0000 7f2f1b400640 1 -- 172.21.15.47:0/656710606 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f2f14014030 con 0x7f2f24069d50 2024-03-21T10:36:32.710 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:36:32 smithi047 ceph-mon[31547]: from='client.? 172.21.15.47:0/3059799533' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-21T10:36:32.726 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:36:32 smithi100 ceph-mon[35193]: from='client.? 172.21.15.47:0/3059799533' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-21T10:36:32.736 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:36:32 smithi157 ceph-mon[35581]: from='client.? 172.21.15.47:0/3059799533' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-21T10:36:32.750 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:32.752+0000 7f2f29c00640 1 -- 172.21.15.47:0/656710606 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f2f24109b40 con 0x7f2f24069d50 2024-03-21T10:36:32.751 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:32.753+0000 7f2f1b400640 1 -- 172.21.15.47:0/656710606 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 32 v32) v1 ==== 94+0+11471 (secure 0 0 0) 0x7f2f1409c6f0 con 0x7f2f24069d50 2024-03-21T10:36:32.751 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-21T10:36:32.751 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":32,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14511,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6835/735724037","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6834","nonce":735724037},{"type":"v1","addr":"172.21.15.47:6835","nonce":735724037}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14529,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6837/2248191724","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6836","nonce":2248191724},{"type":"v1","addr":"172.21.15.47:6837","nonce":2248191724}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":32,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-21T10:21:58.329063+0000","modified":"2024-03-21T10:26:17.169892+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":34535,"mds_1":34493,"mds_2":34454,"mds_3":34427,"mds_4":14619},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14604":{"gid":14604,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.47:6839/2881335356","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6838","nonce":2881335356},{"type":"v1","addr":"172.21.15.47:6839","nonce":2881335356}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_14619":{"gid":14619,"name":"a","rank":4,"incarnation":27,"state":"up:active","state_seq":11,"addr":"172.21.15.47:6841/3401578978","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6840","nonce":3401578978},{"type":"v1","addr":"172.21.15.47:6841","nonce":3401578978}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24406":{"gid":24406,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.157:6833/4000010136","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":4000010136},{"type":"v1","addr":"172.21.15.157:6833","nonce":4000010136}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24457":{"gid":24457,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.157:6835/3902868651","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":3902868651},{"type":"v1","addr":"172.21.15.157:6835","nonce":3902868651}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24475":{"gid":24475,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.157:6837/1908087204","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6836","nonce":1908087204},{"type":"v1","addr":"172.21.15.157:6837","nonce":1908087204}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_25729":{"gid":25729,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.157:6839/1998408416","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6838","nonce":1998408416},{"type":"v1","addr":"172.21.15.157:6839","nonce":1998408416}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_34427":{"gid":34427,"name":"k","rank":3,"incarnation":25,"state":"up:active","state_seq":20,"addr":"172.21.15.100:6833/2723302038","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6832","nonce":2723302038},{"type":"v1","addr":"172.21.15.100:6833","nonce":2723302038}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_34454":{"gid":34454,"name":"e","rank":2,"incarnation":22,"state":"up:active","state_seq":17,"addr":"172.21.15.100:6835/2597849957","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":2597849957},{"type":"v1","addr":"172.21.15.100:6835","nonce":2597849957}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_34493":{"gid":34493,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":13,"addr":"172.21.15.100:6837/3497644162","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6836","nonce":3497644162},{"type":"v1","addr":"172.21.15.100:6837","nonce":3497644162}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_34535":{"gid":34535,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.100:6839/4173168440","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6838","nonce":4173168440},{"type":"v1","addr":"172.21.15.100:6839","nonce":4173168440}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":34535,"qdb_cluster":[34427,14619,34454,34535,34493]},"id":1}]} 2024-03-21T10:36:32.752 INFO:teuthology.orchestra.run.smithi047.stderr:dumped fsmap epoch 32 2024-03-21T10:36:32.755 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:32.756+0000 7f2f29c00640 1 -- 172.21.15.47:0/656710606 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f2f080a3140 msgr2=0x7f2f080a5600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:36:32.755 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:32.756+0000 7f2f29c00640 1 --2- 172.21.15.47:0/656710606 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f2f080a3140 0x7f2f080a5600 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f2f10005ab0 tx=0x7f2f10005a20 comp rx=0 tx=0).stop 2024-03-21T10:36:32.755 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:32.756+0000 7f2f29c00640 1 -- 172.21.15.47:0/656710606 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2f24069d50 msgr2=0x7f2f241005d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:36:32.755 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:32.756+0000 7f2f29c00640 1 --2- 172.21.15.47:0/656710606 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2f24069d50 0x7f2f241005d0 secure :-1 s=READY pgs=1354 cs=0 l=1 rev1=1 crypto rx=0x7f2f1400cce0 tx=0x7f2f1400a620 comp rx=0 tx=0).stop 2024-03-21T10:36:32.755 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:32.757+0000 7f2f29c00640 1 -- 172.21.15.47:0/656710606 shutdown_connections 2024-03-21T10:36:32.755 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:32.757+0000 7f2f29c00640 1 --2- 172.21.15.47:0/656710606 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2f241076a0 0x7f2f24101050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:36:32.755 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:32.757+0000 7f2f29c00640 1 --2- 172.21.15.47:0/656710606 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f2f080a3140 0x7f2f080a5600 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:36:32.755 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:32.757+0000 7f2f29c00640 1 --2- 172.21.15.47:0/656710606 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2f2406a6f0 0x7f2f24100b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:36:32.755 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:32.757+0000 7f2f29c00640 1 --2- 172.21.15.47:0/656710606 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2f24069d50 0x7f2f241005d0 unknown :-1 s=CLOSED pgs=1354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:36:32.755 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:32.757+0000 7f2f29c00640 1 -- 172.21.15.47:0/656710606 >> 172.21.15.47:0/656710606 conn(0x7f2f24077640 msgr2=0x7f2f240ff1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:36:32.755 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:32.757+0000 7f2f29c00640 1 -- 172.21.15.47:0/656710606 shutdown_connections 2024-03-21T10:36:32.755 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:32.757+0000 7f2f29c00640 1 -- 172.21.15.47:0/656710606 wait complete. 2024-03-21T10:36:32.772 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-21T10:36:32.995 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:32.997+0000 7fe517200640 1 -- 172.21.15.47:0/327873967 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe518107590 msgr2=0x7fe518107970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:36:32.995 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:32.997+0000 7fe517200640 1 --2- 172.21.15.47:0/327873967 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe518107590 0x7fe518107970 secure :-1 s=READY pgs=1355 cs=0 l=1 rev1=1 crypto rx=0x7fe508009930 tx=0x7fe50802bc30 comp rx=0 tx=0).stop 2024-03-21T10:36:32.995 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:32.997+0000 7fe517200640 1 -- 172.21.15.47:0/327873967 shutdown_connections 2024-03-21T10:36:32.995 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:32.997+0000 7fe517200640 1 --2- 172.21.15.47:0/327873967 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe5180691c0 0x7fe51810e310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:36:32.995 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:32.997+0000 7fe517200640 1 --2- 172.21.15.47:0/327873967 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe518068800 0x7fe518068c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:36:32.995 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:32.997+0000 7fe517200640 1 --2- 172.21.15.47:0/327873967 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe518107590 0x7fe518107970 unknown :-1 s=CLOSED pgs=1355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:36:32.995 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:32.997+0000 7fe517200640 1 -- 172.21.15.47:0/327873967 >> 172.21.15.47:0/327873967 conn(0x7fe518075fb0 msgr2=0x7fe5180763c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:36:32.996 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:32.997+0000 7fe517200640 1 -- 172.21.15.47:0/327873967 shutdown_connections 2024-03-21T10:36:32.996 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:32.997+0000 7fe517200640 1 -- 172.21.15.47:0/327873967 wait complete. 2024-03-21T10:36:32.997 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:32.998+0000 7fe517200640 1 Processor -- start 2024-03-21T10:36:32.997 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:32.998+0000 7fe517200640 1 -- start start 2024-03-21T10:36:32.997 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:32.998+0000 7fe517200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe518068800 0x7fe518100720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:36:32.997 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:32.998+0000 7fe517200640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe5180691c0 0x7fe518100c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:36:32.997 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:32.999+0000 7fe517200640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe518107590 0x7fe5181011a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:36:32.997 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:32.999+0000 7fe517200640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe518111780 con 0x7fe5180691c0 2024-03-21T10:36:32.997 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:32.999+0000 7fe517200640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe518111600 con 0x7fe518068800 2024-03-21T10:36:32.997 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:32.999+0000 7fe517200640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe518111900 con 0x7fe518107590 2024-03-21T10:36:32.998 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:32.999+0000 7fe515400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe5180691c0 0x7fe518100c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:36:32.998 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:32.999+0000 7fe515400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe5180691c0 0x7fe518100c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:34660/0 (socket says 172.21.15.47:34660) 2024-03-21T10:36:32.998 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:32.999+0000 7fe515e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe518068800 0x7fe518100720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:36:32.998 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:32.999+0000 7fe515400640 1 -- 172.21.15.47:0/2593264157 learned_addr learned my addr 172.21.15.47:0/2593264157 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:36:32.998 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:32.999+0000 7fe516800640 1 --2- 172.21.15.47:0/2593264157 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe518107590 0x7fe5181011a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:36:32.998 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:32.999+0000 7fe515400640 1 -- 172.21.15.47:0/2593264157 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe518107590 msgr2=0x7fe5181011a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:36:32.998 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:32.999+0000 7fe515400640 1 --2- 172.21.15.47:0/2593264157 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe518107590 0x7fe5181011a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:36:32.998 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:32.999+0000 7fe515400640 1 -- 172.21.15.47:0/2593264157 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe518068800 msgr2=0x7fe518100720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:36:32.998 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:32.999+0000 7fe515400640 1 --2- 172.21.15.47:0/2593264157 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe518068800 0x7fe518100720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:36:32.998 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:32.999+0000 7fe515400640 1 -- 172.21.15.47:0/2593264157 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5181a6160 con 0x7fe5180691c0 2024-03-21T10:36:32.999 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:33.000+0000 7fe515400640 1 --2- 172.21.15.47:0/2593264157 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe5180691c0 0x7fe518100c60 secure :-1 s=READY pgs=1356 cs=0 l=1 rev1=1 crypto rx=0x7fe50400d950 tx=0x7fe50400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:36:32.999 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:33.000+0000 7fe50ea00640 1 -- 172.21.15.47:0/2593264157 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe504017070 con 0x7fe5180691c0 2024-03-21T10:36:32.999 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:33.000+0000 7fe50ea00640 1 -- 172.21.15.47:0/2593264157 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fe504009c30 con 0x7fe5180691c0 2024-03-21T10:36:32.999 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:33.000+0000 7fe517200640 1 -- 172.21.15.47:0/2593264157 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe5181a6450 con 0x7fe5180691c0 2024-03-21T10:36:32.999 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:33.000+0000 7fe50ea00640 1 -- 172.21.15.47:0/2593264157 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe50400f620 con 0x7fe5180691c0 2024-03-21T10:36:33.000 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:33.000+0000 7fe517200640 1 -- 172.21.15.47:0/2593264157 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe5181a69f0 con 0x7fe5180691c0 2024-03-21T10:36:33.000 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:33.001+0000 7fe517200640 1 -- 172.21.15.47:0/2593264157 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe5180683b0 con 0x7fe5180691c0 2024-03-21T10:36:33.002 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:33.004+0000 7fe50ea00640 1 -- 172.21.15.47:0/2593264157 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 27) v1 ==== 137985+0+0 (secure 0 0 0) 0x7fe5040109a0 con 0x7fe5180691c0 2024-03-21T10:36:33.005 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:33.007+0000 7fe50ea00640 1 --2- 172.21.15.47:0/2593264157 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7fe4fc0a3110 0x7fe4fc0a55d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:36:33.005 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:33.008+0000 7fe515e00640 1 --2- 172.21.15.47:0/2593264157 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7fe4fc0a3110 0x7fe4fc0a55d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:36:33.005 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:33.008+0000 7fe50ea00640 1 -- 172.21.15.47:0/2593264157 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(107..107 src has 1..107) v4 ==== 9562+0+0 (secure 0 0 0) 0x7fe5040cf8a0 con 0x7fe5180691c0 2024-03-21T10:36:33.006 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:33.009+0000 7fe515e00640 1 --2- 172.21.15.47:0/2593264157 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7fe4fc0a3110 0x7fe4fc0a55d0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fe508009900 tx=0x7fe508005a20 comp rx=0 tx=0).ready entity=mgr.15687 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:36:33.012 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:33.014+0000 7fe50ea00640 1 -- 172.21.15.47:0/2593264157 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fe504009de0 con 0x7fe5180691c0 2024-03-21T10:36:33.221 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:33.223+0000 7fe517200640 1 -- 172.21.15.47:0/2593264157 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fe518109420 con 0x7fe5180691c0 2024-03-21T10:36:33.226 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:33.228+0000 7fe50ea00640 1 -- 172.21.15.47:0/2593264157 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v107) v1 ==== 74+0+25412 (secure 0 0 0) 0x7fe50409bf20 con 0x7fe5180691c0 2024-03-21T10:36:33.226 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-03-21T10:36:33.226 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":107,"fsid":"3056d51a-e76b-11ee-95cd-87774f69a715","created":"2024-03-21T10:11:17.019965+0000","modified":"2024-03-21T10:36:13.887907+0000","last_up_change":"2024-03-21T10:20:52.675328+0000","last_in_change":"2024-03-21T10:20:27.648212+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-21T10:15:40.553114+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-21T10:21:56.399518+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-21T10:21:57.812940+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"106","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":17,"snap_epoch":106,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-21T10:22:01.822568+0000","flags":8197,"flags_names":"hashpspool,ec_overwrites,selfmanaged_snaps","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"106","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":17,"snap_epoch":106,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"b0c7a102-8d89-428b-b793-cbbe67151f50","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6802","nonce":775837313},{"type":"v1","addr":"172.21.15.47:6803","nonce":775837313}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6804","nonce":775837313},{"type":"v1","addr":"172.21.15.47:6805","nonce":775837313}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6808","nonce":775837313},{"type":"v1","addr":"172.21.15.47:6809","nonce":775837313}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6806","nonce":775837313},{"type":"v1","addr":"172.21.15.47:6807","nonce":775837313}]},"public_addr":"172.21.15.47:6803/775837313","cluster_addr":"172.21.15.47:6805/775837313","heartbeat_back_addr":"172.21.15.47:6809/775837313","heartbeat_front_addr":"172.21.15.47:6807/775837313","state":["exists","up"]},{"osd":1,"uuid":"07bb81bb-7cf3-4bbc-ac9e-802f95185d11","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6800","nonce":2923304710},{"type":"v1","addr":"172.21.15.100:6801","nonce":2923304710}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":2923304710},{"type":"v1","addr":"172.21.15.100:6803","nonce":2923304710}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":2923304710},{"type":"v1","addr":"172.21.15.100:6807","nonce":2923304710}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6804","nonce":2923304710},{"type":"v1","addr":"172.21.15.100:6805","nonce":2923304710}]},"public_addr":"172.21.15.100:6801/2923304710","cluster_addr":"172.21.15.100:6803/2923304710","heartbeat_back_addr":"172.21.15.100:6807/2923304710","heartbeat_front_addr":"172.21.15.100:6805/2923304710","state":["exists","up"]},{"osd":2,"uuid":"88ce76b9-6081-4836-aad0-8c249c56e8a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":171377005},{"type":"v1","addr":"172.21.15.157:6801","nonce":171377005}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":171377005},{"type":"v1","addr":"172.21.15.157:6803","nonce":171377005}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":171377005},{"type":"v1","addr":"172.21.15.157:6807","nonce":171377005}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":171377005},{"type":"v1","addr":"172.21.15.157:6805","nonce":171377005}]},"public_addr":"172.21.15.157:6801/171377005","cluster_addr":"172.21.15.157:6803/171377005","heartbeat_back_addr":"172.21.15.157:6807/171377005","heartbeat_front_addr":"172.21.15.157:6805/171377005","state":["exists","up"]},{"osd":3,"uuid":"518aa33d-263f-45e6-aab5-2201ceb40e74","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":27,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6810","nonce":640760101},{"type":"v1","addr":"172.21.15.47:6811","nonce":640760101}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6812","nonce":640760101},{"type":"v1","addr":"172.21.15.47:6813","nonce":640760101}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6816","nonce":640760101},{"type":"v1","addr":"172.21.15.47:6817","nonce":640760101}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6814","nonce":640760101},{"type":"v1","addr":"172.21.15.47:6815","nonce":640760101}]},"public_addr":"172.21.15.47:6811/640760101","cluster_addr":"172.21.15.47:6813/640760101","heartbeat_back_addr":"172.21.15.47:6817/640760101","heartbeat_front_addr":"172.21.15.47:6815/640760101","state":["exists","up"]},{"osd":4,"uuid":"987bde83-ecc6-42e5-bd56-c143b3de4226","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":35,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6808","nonce":2785478639},{"type":"v1","addr":"172.21.15.100:6809","nonce":2785478639}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6810","nonce":2785478639},{"type":"v1","addr":"172.21.15.100:6811","nonce":2785478639}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6814","nonce":2785478639},{"type":"v1","addr":"172.21.15.100:6815","nonce":2785478639}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":2785478639},{"type":"v1","addr":"172.21.15.100:6813","nonce":2785478639}]},"public_addr":"172.21.15.100:6809/2785478639","cluster_addr":"172.21.15.100:6811/2785478639","heartbeat_back_addr":"172.21.15.100:6815/2785478639","heartbeat_front_addr":"172.21.15.100:6813/2785478639","state":["exists","up"]},{"osd":5,"uuid":"012d2188-3a61-4a5f-8d01-2001c15ffbe2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":40,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":1525645410},{"type":"v1","addr":"172.21.15.157:6809","nonce":1525645410}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":1525645410},{"type":"v1","addr":"172.21.15.157:6811","nonce":1525645410}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":1525645410},{"type":"v1","addr":"172.21.15.157:6815","nonce":1525645410}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":1525645410},{"type":"v1","addr":"172.21.15.157:6813","nonce":1525645410}]},"public_addr":"172.21.15.157:6809/1525645410","cluster_addr":"172.21.15.157:6811/1525645410","heartbeat_back_addr":"172.21.15.157:6815/1525645410","heartbeat_front_addr":"172.21.15.157:6813/1525645410","state":["exists","up"]},{"osd":6,"uuid":"c12c14b4-85f0-4d88-937c-de4f556437de","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6818","nonce":2321642790},{"type":"v1","addr":"172.21.15.47:6819","nonce":2321642790}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6820","nonce":2321642790},{"type":"v1","addr":"172.21.15.47:6821","nonce":2321642790}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6824","nonce":2321642790},{"type":"v1","addr":"172.21.15.47:6825","nonce":2321642790}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6822","nonce":2321642790},{"type":"v1","addr":"172.21.15.47:6823","nonce":2321642790}]},"public_addr":"172.21.15.47:6819/2321642790","cluster_addr":"172.21.15.47:6821/2321642790","heartbeat_back_addr":"172.21.15.47:6825/2321642790","heartbeat_front_addr":"172.21.15.47:6823/2321642790","state":["exists","up"]},{"osd":7,"uuid":"3c87c6d0-1d1f-4294-9917-aaaad7dda824","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6816","nonce":4104727330},{"type":"v1","addr":"172.21.15.100:6817","nonce":4104727330}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":4104727330},{"type":"v1","addr":"172.21.15.100:6819","nonce":4104727330}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":4104727330},{"type":"v1","addr":"172.21.15.100:6823","nonce":4104727330}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6820","nonce":4104727330},{"type":"v1","addr":"172.21.15.100:6821","nonce":4104727330}]},"public_addr":"172.21.15.100:6817/4104727330","cluster_addr":"172.21.15.100:6819/4104727330","heartbeat_back_addr":"172.21.15.100:6823/4104727330","heartbeat_front_addr":"172.21.15.100:6821/4104727330","state":["exists","up"]},{"osd":8,"uuid":"502d65cf-53c8-4936-8de4-e11b2a6e1e18","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":61,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":4067715347},{"type":"v1","addr":"172.21.15.157:6817","nonce":4067715347}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":4067715347},{"type":"v1","addr":"172.21.15.157:6819","nonce":4067715347}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":4067715347},{"type":"v1","addr":"172.21.15.157:6823","nonce":4067715347}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":4067715347},{"type":"v1","addr":"172.21.15.157:6821","nonce":4067715347}]},"public_addr":"172.21.15.157:6817/4067715347","cluster_addr":"172.21.15.157:6819/4067715347","heartbeat_back_addr":"172.21.15.157:6823/4067715347","heartbeat_front_addr":"172.21.15.157:6821/4067715347","state":["exists","up"]},{"osd":9,"uuid":"5ea1f5be-8458-4c93-9e61-c2e867a514f3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6826","nonce":1982108738},{"type":"v1","addr":"172.21.15.47:6827","nonce":1982108738}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6828","nonce":1982108738},{"type":"v1","addr":"172.21.15.47:6829","nonce":1982108738}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6832","nonce":1982108738},{"type":"v1","addr":"172.21.15.47:6833","nonce":1982108738}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6830","nonce":1982108738},{"type":"v1","addr":"172.21.15.47:6831","nonce":1982108738}]},"public_addr":"172.21.15.47:6827/1982108738","cluster_addr":"172.21.15.47:6829/1982108738","heartbeat_back_addr":"172.21.15.47:6833/1982108738","heartbeat_front_addr":"172.21.15.47:6831/1982108738","state":["exists","up"]},{"osd":10,"uuid":"2da3da90-a0be-4a54-a9ff-b2d6c9b36870","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":74,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":396391028},{"type":"v1","addr":"172.21.15.100:6825","nonce":396391028}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":396391028},{"type":"v1","addr":"172.21.15.100:6827","nonce":396391028}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6830","nonce":396391028},{"type":"v1","addr":"172.21.15.100:6831","nonce":396391028}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":396391028},{"type":"v1","addr":"172.21.15.100:6829","nonce":396391028}]},"public_addr":"172.21.15.100:6825/396391028","cluster_addr":"172.21.15.100:6827/396391028","heartbeat_back_addr":"172.21.15.100:6831/396391028","heartbeat_front_addr":"172.21.15.100:6829/396391028","state":["exists","up"]},{"osd":11,"uuid":"11798d41-7f7e-4319-95b8-8d3d0946bde9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":82,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":2671029483},{"type":"v1","addr":"172.21.15.157:6825","nonce":2671029483}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":2671029483},{"type":"v1","addr":"172.21.15.157:6827","nonce":2671029483}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":2671029483},{"type":"v1","addr":"172.21.15.157:6831","nonce":2671029483}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":2671029483},{"type":"v1","addr":"172.21.15.157:6829","nonce":2671029483}]},"public_addr":"172.21.15.157:6825/2671029483","cluster_addr":"172.21.15.157:6827/2671029483","heartbeat_back_addr":"172.21.15.157:6831/2671029483","heartbeat_front_addr":"172.21.15.157:6829/2671029483","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T10:14:31.822399+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T10:15:04.175721+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T10:15:35.902595+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T10:16:12.088891+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T10:16:46.428704+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T10:17:20.158270+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T10:17:55.915253+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T10:18:30.064641+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T10:19:04.410004+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T10:19:40.605181+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T10:20:15.636605+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T10:20:50.151365+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.11","mappings":[{"from":7,"to":11}]},{"pgid":"2.14","mappings":[{"from":2,"to":4}]},{"pgid":"2.1a","mappings":[{"from":8,"to":1}]},{"pgid":"2.1b","mappings":[{"from":2,"to":4}]},{"pgid":"2.1d","mappings":[{"from":8,"to":11}]},{"pgid":"3.0","mappings":[{"from":4,"to":8},{"from":5,"to":9}]},{"pgid":"3.6","mappings":[{"from":3,"to":7}]},{"pgid":"3.1b","mappings":[{"from":4,"to":6}]},{"pgid":"3.33","mappings":[{"from":5,"to":9}]},{"pgid":"3.3c","mappings":[{"from":4,"to":9}]},{"pgid":"4.c","mappings":[{"from":9,"to":10}]},{"pgid":"4.f","mappings":[{"from":9,"to":8}]},{"pgid":"4.19","mappings":[{"from":9,"to":8}]},{"pgid":"4.1d","mappings":[{"from":5,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.47:0/936039355":"2024-03-22T10:25:19.912777+0000","172.21.15.47:0/2983192238":"2024-03-22T10:25:19.912777+0000","172.21.15.47:6801/2823885994":"2024-03-22T10:25:19.912777+0000","172.21.15.47:0/356822431":"2024-03-22T10:12:25.844530+0000","172.21.15.47:0/1167490879":"2024-03-22T10:12:25.844530+0000","172.21.15.47:0/1540718138":"2024-03-22T10:12:25.844530+0000","172.21.15.47:6801/3836903972":"2024-03-22T10:12:25.844530+0000","172.21.15.47:0/235436602":"2024-03-22T10:11:50.301838+0000","172.21.15.47:0/2052815572":"2024-03-22T10:11:50.301838+0000","172.21.15.47:0/1607664848":"2024-03-22T10:25:19.912777+0000","172.21.15.47:6800/3942658432":"2024-03-22T10:11:50.301838+0000","172.21.15.47:0/3101796817":"2024-03-22T10:25:19.912777+0000","172.21.15.47:6801/3942658432":"2024-03-22T10:11:50.301838+0000","172.21.15.47:6800/2823885994":"2024-03-22T10:25:19.912777+0000","172.21.15.47:0/1419516257":"2024-03-22T10:11:50.301838+0000","172.21.15.47:6800/3836903972":"2024-03-22T10:12:25.844530+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[{"pool":3,"snaps":[{"begin":6,"length":1},{"begin":17,"length":1}]},{"pool":4,"snaps":[{"begin":6,"length":1},{"begin":17,"length":1}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-21T10:36:33.229 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:33.231+0000 7fe517200640 1 -- 172.21.15.47:0/2593264157 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7fe4fc0a3110 msgr2=0x7fe4fc0a55d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:36:33.229 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:33.231+0000 7fe517200640 1 --2- 172.21.15.47:0/2593264157 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7fe4fc0a3110 0x7fe4fc0a55d0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fe508009900 tx=0x7fe508005a20 comp rx=0 tx=0).stop 2024-03-21T10:36:33.229 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:33.231+0000 7fe517200640 1 -- 172.21.15.47:0/2593264157 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe5180691c0 msgr2=0x7fe518100c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:36:33.229 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:33.231+0000 7fe517200640 1 --2- 172.21.15.47:0/2593264157 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe5180691c0 0x7fe518100c60 secure :-1 s=READY pgs=1356 cs=0 l=1 rev1=1 crypto rx=0x7fe50400d950 tx=0x7fe50400de10 comp rx=0 tx=0).stop 2024-03-21T10:36:33.229 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:33.231+0000 7fe517200640 1 -- 172.21.15.47:0/2593264157 shutdown_connections 2024-03-21T10:36:33.229 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:33.231+0000 7fe517200640 1 --2- 172.21.15.47:0/2593264157 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe518107590 0x7fe5181011a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:36:33.229 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:33.231+0000 7fe517200640 1 --2- 172.21.15.47:0/2593264157 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe5180691c0 0x7fe518100c60 unknown :-1 s=CLOSED pgs=1356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:36:33.229 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:33.231+0000 7fe517200640 1 --2- 172.21.15.47:0/2593264157 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7fe4fc0a3110 0x7fe4fc0a55d0 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:36:33.229 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:33.231+0000 7fe517200640 1 --2- 172.21.15.47:0/2593264157 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe518068800 0x7fe518100720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:36:33.229 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:33.231+0000 7fe517200640 1 -- 172.21.15.47:0/2593264157 >> 172.21.15.47:0/2593264157 conn(0x7fe518075fb0 msgr2=0x7fe5180ff290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:36:33.229 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:33.231+0000 7fe517200640 1 -- 172.21.15.47:0/2593264157 shutdown_connections 2024-03-21T10:36:33.229 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:33.231+0000 7fe517200640 1 -- 172.21.15.47:0/2593264157 wait complete. 2024-03-21T10:36:33.249 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-03-21T10:36:33.457 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:33.458+0000 7f3ae6a00640 1 -- 172.21.15.47:0/3601071252 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3ae0104200 msgr2=0x7f3ae0110710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:36:33.457 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:33.458+0000 7f3ae6a00640 1 --2- 172.21.15.47:0/3601071252 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3ae0104200 0x7f3ae0110710 secure :-1 s=READY pgs=1357 cs=0 l=1 rev1=1 crypto rx=0x7f3ad8009cd0 tx=0x7f3ad802c0f0 comp rx=0 tx=0).stop 2024-03-21T10:36:33.457 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:33.459+0000 7f3ae6a00640 1 -- 172.21.15.47:0/3601071252 shutdown_connections 2024-03-21T10:36:33.457 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:33.459+0000 7f3ae6a00640 1 --2- 172.21.15.47:0/3601071252 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3ae0104200 0x7f3ae0110710 unknown :-1 s=CLOSED pgs=1357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:36:33.457 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:33.459+0000 7f3ae6a00640 1 --2- 172.21.15.47:0/3601071252 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3ae0103840 0x7f3ae0103cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:36:33.457 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:33.459+0000 7f3ae6a00640 1 --2- 172.21.15.47:0/3601071252 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3ae0109850 0x7f3ae0109c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:36:33.457 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:33.459+0000 7f3ae6a00640 1 -- 172.21.15.47:0/3601071252 >> 172.21.15.47:0/3601071252 conn(0x7f3ae00ff310 msgr2=0x7f3ae0101730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:36:33.457 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:33.459+0000 7f3ae6a00640 1 -- 172.21.15.47:0/3601071252 shutdown_connections 2024-03-21T10:36:33.457 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:33.459+0000 7f3ae6a00640 1 -- 172.21.15.47:0/3601071252 wait complete. 2024-03-21T10:36:33.458 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:33.460+0000 7f3ae6a00640 1 Processor -- start 2024-03-21T10:36:33.458 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:33.460+0000 7f3ae6a00640 1 -- start start 2024-03-21T10:36:33.460 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:33.461+0000 7f3ae6a00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3ae0103840 0x7f3ae019a080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:36:33.460 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:33.461+0000 7f3ae6a00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3ae0104200 0x7f3ae019a5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:36:33.460 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:33.461+0000 7f3ae6a00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3ae0109850 0x7f3ae019e940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:36:33.460 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:33.461+0000 7f3ae6a00640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ae0113b10 con 0x7f3ae0104200 2024-03-21T10:36:33.460 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:33.461+0000 7f3ae6a00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ae0113990 con 0x7f3ae0103840 2024-03-21T10:36:33.460 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:33.461+0000 7f3ae6a00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ae0113c90 con 0x7f3ae0109850 2024-03-21T10:36:33.460 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:33.461+0000 7f3adf400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3ae0104200 0x7f3ae019a5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:36:33.460 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:33.461+0000 7f3adf400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3ae0104200 0x7f3ae019a5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:34690/0 (socket says 172.21.15.47:34690) 2024-03-21T10:36:33.460 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:33.461+0000 7f3adf400640 1 -- 172.21.15.47:0/556659138 learned_addr learned my addr 172.21.15.47:0/556659138 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:36:33.460 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:33.461+0000 7f3adfe00640 1 --2- 172.21.15.47:0/556659138 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3ae0103840 0x7f3ae019a080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:36:33.460 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:33.461+0000 7f3ae4a00640 1 --2- 172.21.15.47:0/556659138 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3ae0109850 0x7f3ae019e940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:36:33.460 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:33.462+0000 7f3adf400640 1 -- 172.21.15.47:0/556659138 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3ae0109850 msgr2=0x7f3ae019e940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:36:33.460 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:33.462+0000 7f3adf400640 1 --2- 172.21.15.47:0/556659138 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3ae0109850 0x7f3ae019e940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:36:33.460 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:33.462+0000 7f3adf400640 1 -- 172.21.15.47:0/556659138 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3ae0103840 msgr2=0x7f3ae019a080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:36:33.461 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:33.462+0000 7f3adf400640 1 --2- 172.21.15.47:0/556659138 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3ae0103840 0x7f3ae019a080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:36:33.461 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:33.462+0000 7f3adf400640 1 -- 172.21.15.47:0/556659138 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ae019f070 con 0x7f3ae0104200 2024-03-21T10:36:33.461 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:33.462+0000 7f3adf400640 1 --2- 172.21.15.47:0/556659138 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3ae0104200 0x7f3ae019a5c0 secure :-1 s=READY pgs=1358 cs=0 l=1 rev1=1 crypto rx=0x7f3acc00d950 tx=0x7f3acc00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:36:33.461 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:33.462+0000 7f3adcc00640 1 -- 172.21.15.47:0/556659138 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3acc017030 con 0x7f3ae0104200 2024-03-21T10:36:33.461 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:33.462+0000 7f3ae6a00640 1 -- 172.21.15.47:0/556659138 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3ae019f360 con 0x7f3ae0104200 2024-03-21T10:36:33.461 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:33.462+0000 7f3adcc00640 1 -- 172.21.15.47:0/556659138 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f3acc004020 con 0x7f3ae0104200 2024-03-21T10:36:33.461 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:33.462+0000 7f3adcc00640 1 -- 172.21.15.47:0/556659138 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3acc0047a0 con 0x7f3ae0104200 2024-03-21T10:36:33.461 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:33.462+0000 7f3ae6a00640 1 -- 172.21.15.47:0/556659138 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3ae01a6bf0 con 0x7f3ae0104200 2024-03-21T10:36:33.462 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:33.464+0000 7f3ae6a00640 1 -- 172.21.15.47:0/556659138 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3ae00688e0 con 0x7f3ae0104200 2024-03-21T10:36:33.464 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:33.466+0000 7f3adcc00640 1 -- 172.21.15.47:0/556659138 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 27) v1 ==== 137985+0+0 (secure 0 0 0) 0x7f3acc0041d0 con 0x7f3ae0104200 2024-03-21T10:36:33.465 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:33.468+0000 7f3adcc00640 1 --2- 172.21.15.47:0/556659138 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f3ac40a3090 0x7f3ac40a5550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:36:33.466 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:33.468+0000 7f3adcc00640 1 -- 172.21.15.47:0/556659138 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(107..107 src has 1..107) v4 ==== 9562+0+0 (secure 0 0 0) 0x7f3acc0cf700 con 0x7f3ae0104200 2024-03-21T10:36:33.466 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:33.468+0000 7f3adfe00640 1 --2- 172.21.15.47:0/556659138 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f3ac40a3090 0x7f3ac40a5550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:36:33.466 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:33.469+0000 7f3adfe00640 1 --2- 172.21.15.47:0/556659138 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f3ac40a3090 0x7f3ac40a5550 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f3ad0005fd0 tx=0x7f3ad000b390 comp rx=0 tx=0).ready entity=mgr.15687 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:36:33.470 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:33.472+0000 7f3adcc00640 1 -- 172.21.15.47:0/556659138 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f3acc09bd80 con 0x7f3ae0104200 2024-03-21T10:36:33.710 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:36:33 smithi047 ceph-mon[31547]: pgmap v350: 129 pgs: 129 active+clean; 2.5 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 938 B/s wr, 14 op/s 2024-03-21T10:36:33.711 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:36:33 smithi047 ceph-mon[31547]: from='client.? 172.21.15.47:0/656710606' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-21T10:36:33.711 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:36:33 smithi047 ceph-mon[31547]: from='client.? 172.21.15.47:0/2593264157' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-21T10:36:33.726 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:36:33 smithi100 ceph-mon[35193]: pgmap v350: 129 pgs: 129 active+clean; 2.5 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 938 B/s wr, 14 op/s 2024-03-21T10:36:33.726 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:36:33 smithi100 ceph-mon[35193]: from='client.? 172.21.15.47:0/656710606' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-21T10:36:33.726 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:36:33 smithi100 ceph-mon[35193]: from='client.? 172.21.15.47:0/2593264157' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-21T10:36:33.736 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:36:33 smithi157 ceph-mon[35581]: pgmap v350: 129 pgs: 129 active+clean; 2.5 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 938 B/s wr, 14 op/s 2024-03-21T10:36:33.736 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:36:33 smithi157 ceph-mon[35581]: from='client.? 172.21.15.47:0/656710606' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-21T10:36:33.736 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:36:33 smithi157 ceph-mon[35581]: from='client.? 172.21.15.47:0/2593264157' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-21T10:36:33.737 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:33.740+0000 7f3ae6a00640 1 -- 172.21.15.47:0/556659138 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f3ae010b820 con 0x7f3ae0104200 2024-03-21T10:36:33.742 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:33.745+0000 7f3adcc00640 1 -- 172.21.15.47:0/556659138 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v33) v1 ==== 161+0+0 (secure 0 0 0) 0x7f3acc09b680 con 0x7f3ae0104200 2024-03-21T10:36:33.743 INFO:teuthology.orchestra.run.smithi047.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-03-21T10:36:33.748 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:33.750+0000 7f3ae6a00640 1 -- 172.21.15.47:0/556659138 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f3ac40a3090 msgr2=0x7f3ac40a5550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:36:33.748 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:33.750+0000 7f3ae6a00640 1 --2- 172.21.15.47:0/556659138 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f3ac40a3090 0x7f3ac40a5550 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f3ad0005fd0 tx=0x7f3ad000b390 comp rx=0 tx=0).stop 2024-03-21T10:36:33.748 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:33.750+0000 7f3ae6a00640 1 -- 172.21.15.47:0/556659138 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3ae0104200 msgr2=0x7f3ae019a5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:36:33.748 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:33.750+0000 7f3ae6a00640 1 --2- 172.21.15.47:0/556659138 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3ae0104200 0x7f3ae019a5c0 secure :-1 s=READY pgs=1358 cs=0 l=1 rev1=1 crypto rx=0x7f3acc00d950 tx=0x7f3acc00de10 comp rx=0 tx=0).stop 2024-03-21T10:36:33.750 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:33.753+0000 7f3ae6a00640 1 -- 172.21.15.47:0/556659138 shutdown_connections 2024-03-21T10:36:33.750 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:33.753+0000 7f3ae6a00640 1 --2- 172.21.15.47:0/556659138 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3ae0109850 0x7f3ae019e940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:36:33.751 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:33.753+0000 7f3ae6a00640 1 --2- 172.21.15.47:0/556659138 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3ae0104200 0x7f3ae019a5c0 unknown :-1 s=CLOSED pgs=1358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:36:33.751 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:33.753+0000 7f3ae6a00640 1 --2- 172.21.15.47:0/556659138 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f3ac40a3090 0x7f3ac40a5550 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:36:33.751 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:33.753+0000 7f3ae6a00640 1 --2- 172.21.15.47:0/556659138 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3ae0103840 0x7f3ae019a080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:36:33.751 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:33.753+0000 7f3ae6a00640 1 -- 172.21.15.47:0/556659138 >> 172.21.15.47:0/556659138 conn(0x7f3ae00ff310 msgr2=0x7f3ae00ffc50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:36:33.751 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:33.753+0000 7f3ae6a00640 1 -- 172.21.15.47:0/556659138 shutdown_connections 2024-03-21T10:36:33.751 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:33.753+0000 7f3ae6a00640 1 -- 172.21.15.47:0/556659138 wait complete. 2024-03-21T10:36:33.773 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-03-21T10:36:33.974 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:33.975+0000 7f1adf000640 1 -- 172.21.15.47:0/363830714 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1ad8109850 msgr2=0x7f1ad8109c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:36:33.974 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:33.975+0000 7f1adf000640 1 --2- 172.21.15.47:0/363830714 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1ad8109850 0x7f1ad8109c30 secure :-1 s=READY pgs=1363 cs=0 l=1 rev1=1 crypto rx=0x7f1ac8009930 tx=0x7f1ac802bc20 comp rx=0 tx=0).stop 2024-03-21T10:36:33.974 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:33.976+0000 7f1adf000640 1 -- 172.21.15.47:0/363830714 shutdown_connections 2024-03-21T10:36:33.974 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:33.976+0000 7f1adf000640 1 --2- 172.21.15.47:0/363830714 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1ad8104200 0x7f1ad8110710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:36:33.974 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:33.976+0000 7f1adf000640 1 --2- 172.21.15.47:0/363830714 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1ad8103840 0x7f1ad8103cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:36:33.974 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:33.976+0000 7f1adf000640 1 --2- 172.21.15.47:0/363830714 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1ad8109850 0x7f1ad8109c30 unknown :-1 s=CLOSED pgs=1363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:36:33.974 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:33.976+0000 7f1adf000640 1 -- 172.21.15.47:0/363830714 >> 172.21.15.47:0/363830714 conn(0x7f1ad80ff2f0 msgr2=0x7f1ad8101710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:36:33.974 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:33.976+0000 7f1adf000640 1 -- 172.21.15.47:0/363830714 shutdown_connections 2024-03-21T10:36:33.974 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:33.976+0000 7f1adf000640 1 -- 172.21.15.47:0/363830714 wait complete. 2024-03-21T10:36:33.976 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:33.977+0000 7f1adf000640 1 Processor -- start 2024-03-21T10:36:33.976 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:33.977+0000 7f1adf000640 1 -- start start 2024-03-21T10:36:33.976 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:33.977+0000 7f1adf000640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1ad8103840 0x7f1ad819a0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:36:33.976 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:33.977+0000 7f1adf000640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1ad8104200 0x7f1ad819a5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:36:33.976 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:33.977+0000 7f1adf000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1ad8109850 0x7f1ad819e970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:36:33.976 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:33.977+0000 7f1adf000640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ad8113b10 con 0x7f1ad8104200 2024-03-21T10:36:33.976 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:33.977+0000 7f1adf000640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ad8113990 con 0x7f1ad8109850 2024-03-21T10:36:33.976 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:33.977+0000 7f1adf000640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ad8113c90 con 0x7f1ad8103840 2024-03-21T10:36:33.976 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:33.978+0000 7f1ad7400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1ad8104200 0x7f1ad819a5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:36:33.976 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:33.978+0000 7f1ad7400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1ad8104200 0x7f1ad819a5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:34736/0 (socket says 172.21.15.47:34736) 2024-03-21T10:36:33.977 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:33.978+0000 7f1ad7400640 1 -- 172.21.15.47:0/2310630323 learned_addr learned my addr 172.21.15.47:0/2310630323 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:36:33.977 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:33.978+0000 7f1ad7e00640 1 --2- 172.21.15.47:0/2310630323 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1ad8103840 0x7f1ad819a0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:36:33.977 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:33.978+0000 7f1ad7400640 1 -- 172.21.15.47:0/2310630323 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1ad8103840 msgr2=0x7f1ad819a0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:36:33.977 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:33.978+0000 7f1ad7400640 1 --2- 172.21.15.47:0/2310630323 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1ad8103840 0x7f1ad819a0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:36:33.977 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:33.978+0000 7f1ad7400640 1 -- 172.21.15.47:0/2310630323 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1ad8109850 msgr2=0x7f1ad819e970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:36:33.977 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:33.978+0000 7f1add000640 1 --2- 172.21.15.47:0/2310630323 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1ad8109850 0x7f1ad819e970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:36:33.977 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:33.978+0000 7f1ad7400640 1 --2- 172.21.15.47:0/2310630323 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1ad8109850 0x7f1ad819e970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:36:33.977 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:33.978+0000 7f1ad7400640 1 -- 172.21.15.47:0/2310630323 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ad819f0a0 con 0x7f1ad8104200 2024-03-21T10:36:33.977 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:33.978+0000 7f1ad7e00640 1 --2- 172.21.15.47:0/2310630323 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1ad8103840 0x7f1ad819a0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T10:36:33.977 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:33.978+0000 7f1add000640 1 --2- 172.21.15.47:0/2310630323 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1ad8109850 0x7f1ad819e970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T10:36:33.977 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:33.978+0000 7f1ad7400640 1 --2- 172.21.15.47:0/2310630323 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1ad8104200 0x7f1ad819a5f0 secure :-1 s=READY pgs=1364 cs=0 l=1 rev1=1 crypto rx=0x7f1ac400d950 tx=0x7f1ac400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:36:33.978 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:33.979+0000 7f1ad4c00640 1 -- 172.21.15.47:0/2310630323 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ac4017030 con 0x7f1ad8104200 2024-03-21T10:36:33.978 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:33.979+0000 7f1ad4c00640 1 -- 172.21.15.47:0/2310630323 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f1ac4004020 con 0x7f1ad8104200 2024-03-21T10:36:33.978 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:33.979+0000 7f1adf000640 1 -- 172.21.15.47:0/2310630323 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1ad819f390 con 0x7f1ad8104200 2024-03-21T10:36:33.978 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:33.979+0000 7f1ad4c00640 1 -- 172.21.15.47:0/2310630323 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ac40047a0 con 0x7f1ad8104200 2024-03-21T10:36:33.978 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:33.979+0000 7f1adf000640 1 -- 172.21.15.47:0/2310630323 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1ad81a6c20 con 0x7f1ad8104200 2024-03-21T10:36:33.978 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:33.980+0000 7f1adf000640 1 -- 172.21.15.47:0/2310630323 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1ad80688e0 con 0x7f1ad8104200 2024-03-21T10:36:33.980 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:33.983+0000 7f1ad4c00640 1 -- 172.21.15.47:0/2310630323 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 27) v1 ==== 137985+0+0 (secure 0 0 0) 0x7f1ac40041d0 con 0x7f1ad8104200 2024-03-21T10:36:33.982 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:33.984+0000 7f1ad4c00640 1 --2- 172.21.15.47:0/2310630323 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f1abc0a3090 0x7f1abc0a5550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:36:33.982 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:33.984+0000 7f1ad4c00640 1 -- 172.21.15.47:0/2310630323 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(108..108 src has 1..108) v4 ==== 9904+0+0 (secure 0 0 0) 0x7f1ac40cf8d0 con 0x7f1ad8104200 2024-03-21T10:36:33.982 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:33.985+0000 7f1ad7e00640 1 --2- 172.21.15.47:0/2310630323 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f1abc0a3090 0x7f1abc0a5550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:36:33.983 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:33.985+0000 7f1ad7e00640 1 --2- 172.21.15.47:0/2310630323 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f1abc0a3090 0x7f1abc0a5550 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f1ac8036040 tx=0x7f1ac8000bc0 comp rx=0 tx=0).ready entity=mgr.15687 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:36:33.987 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:33.989+0000 7f1ad4c00640 1 -- 172.21.15.47:0/2310630323 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f1ac409be20 con 0x7f1ad8104200 2024-03-21T10:36:34.210 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:36:33 smithi047 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-mon-a[31543]: 2024-03-21T10:36:33.741+0000 7f6080a00700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-03-21T10:36:34.211 INFO:journalctl@ceph.mds.a.smithi047.stdout:Mar 21 10:36:33 smithi047 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-mds-a[71185]: ignoring --setuser ceph since I am not root 2024-03-21T10:36:34.211 INFO:journalctl@ceph.mds.a.smithi047.stdout:Mar 21 10:36:33 smithi047 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-mds-a[71185]: ignoring --setgroup ceph since I am not root 2024-03-21T10:36:34.211 INFO:journalctl@ceph.mds.a.smithi047.stdout:Mar 21 10:36:33 smithi047 ceph-mds[71189]: -- 172.21.15.47:0/3084594394 <== mon.0 v2:172.21.15.47:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x556c4ee95a40 con 0x556c4ee68800 2024-03-21T10:36:34.211 INFO:journalctl@ceph.mds.a.smithi047.stdout:Mar 21 10:36:33 smithi047 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-mds-a[71185]: starting mds.a at 2024-03-21T10:36:34.211 INFO:journalctl@ceph.mds.g.smithi047.stdout:Mar 21 10:36:33 smithi047 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-mds-g[70784]: ignoring --setuser ceph since I am not root 2024-03-21T10:36:34.211 INFO:journalctl@ceph.mds.g.smithi047.stdout:Mar 21 10:36:33 smithi047 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-mds-g[70784]: ignoring --setgroup ceph since I am not root 2024-03-21T10:36:34.212 INFO:journalctl@ceph.mds.g.smithi047.stdout:Mar 21 10:36:33 smithi047 ceph-mds[70808]: -- 172.21.15.47:0/2608021879 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x563bf55c04e0 con 0x563bf5596800 2024-03-21T10:36:34.212 INFO:journalctl@ceph.mds.g.smithi047.stdout:Mar 21 10:36:33 smithi047 ceph-mds[70808]: -- 172.21.15.47:0/2608021879 <== mon.0 v2:172.21.15.47:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x563bf55c3a40 con 0x563bf5596800 2024-03-21T10:36:34.212 INFO:journalctl@ceph.mds.g.smithi047.stdout:Mar 21 10:36:33 smithi047 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-mds-g[70784]: starting mds.g at 2024-03-21T10:36:34.227 INFO:journalctl@ceph.mds.k.smithi100.stdout:Mar 21 10:36:33 smithi100 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-mds-k[64070]: ignoring --setuser ceph since I am not root 2024-03-21T10:36:34.227 INFO:journalctl@ceph.mds.k.smithi100.stdout:Mar 21 10:36:33 smithi100 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-mds-k[64070]: ignoring --setgroup ceph since I am not root 2024-03-21T10:36:34.227 INFO:journalctl@ceph.mds.k.smithi100.stdout:Mar 21 10:36:33 smithi100 ceph-mds[64076]: -- 172.21.15.100:0/30256887 <== mon.2 v2:172.21.15.100:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x561cf3827a40 con 0x561cf37fa800 2024-03-21T10:36:34.227 INFO:journalctl@ceph.mds.k.smithi100.stdout:Mar 21 10:36:33 smithi100 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-mds-k[64070]: starting mds.k at 2024-03-21T10:36:34.227 INFO:journalctl@ceph.mds.h.smithi100.stdout:Mar 21 10:36:33 smithi100 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-mds-h[64682]: ignoring --setuser ceph since I am not root 2024-03-21T10:36:34.227 INFO:journalctl@ceph.mds.h.smithi100.stdout:Mar 21 10:36:33 smithi100 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-mds-h[64682]: ignoring --setgroup ceph since I am not root 2024-03-21T10:36:34.227 INFO:journalctl@ceph.mds.h.smithi100.stdout:Mar 21 10:36:33 smithi100 ceph-mds[64686]: -- 172.21.15.100:0/2093349489 <== mon.1 v2:172.21.15.157:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x559c7021fa40 con 0x559c701f2c00 2024-03-21T10:36:34.228 INFO:journalctl@ceph.mds.h.smithi100.stdout:Mar 21 10:36:33 smithi100 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-mds-h[64682]: starting mds.h at 2024-03-21T10:36:34.228 INFO:journalctl@ceph.mds.e.smithi100.stdout:Mar 21 10:36:33 smithi100 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-mds-e[64379]: ignoring --setuser ceph since I am not root 2024-03-21T10:36:34.228 INFO:journalctl@ceph.mds.e.smithi100.stdout:Mar 21 10:36:33 smithi100 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-mds-e[64379]: ignoring --setgroup ceph since I am not root 2024-03-21T10:36:34.228 INFO:journalctl@ceph.mds.e.smithi100.stdout:Mar 21 10:36:33 smithi100 ceph-mds[64383]: -- 172.21.15.100:0/2990916668 <== mon.2 v2:172.21.15.100:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55a54e04ba40 con 0x55a54e01f000 2024-03-21T10:36:34.228 INFO:journalctl@ceph.mds.e.smithi100.stdout:Mar 21 10:36:33 smithi100 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-mds-e[64379]: starting mds.e at 2024-03-21T10:36:34.228 INFO:journalctl@ceph.mds.b.smithi100.stdout:Mar 21 10:36:33 smithi100 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-mds-b[64996]: ignoring --setuser ceph since I am not root 2024-03-21T10:36:34.228 INFO:journalctl@ceph.mds.b.smithi100.stdout:Mar 21 10:36:33 smithi100 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-mds-b[64996]: ignoring --setgroup ceph since I am not root 2024-03-21T10:36:34.228 INFO:journalctl@ceph.mds.b.smithi100.stdout:Mar 21 10:36:33 smithi100 ceph-mds[65000]: -- 172.21.15.100:0/3679900756 <== mon.2 v2:172.21.15.100:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x56006fa19a40 con 0x56006f9ecc00 2024-03-21T10:36:34.229 INFO:journalctl@ceph.mds.b.smithi100.stdout:Mar 21 10:36:33 smithi100 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-mds-b[64996]: starting mds.b at 2024-03-21T10:36:34.229 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:34.231+0000 7f1adf000640 1 -- 172.21.15.47:0/2310630323 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f1ad802d070 con 0x7f1ad8104200 2024-03-21T10:36:34.237 INFO:journalctl@ceph.mds.i.smithi157.stdout:Mar 21 10:36:33 smithi157 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-mds-i[70254]: ignoring --setuser ceph since I am not root 2024-03-21T10:36:34.237 INFO:journalctl@ceph.mds.i.smithi157.stdout:Mar 21 10:36:33 smithi157 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-mds-i[70254]: ignoring --setgroup ceph since I am not root 2024-03-21T10:36:34.237 INFO:journalctl@ceph.mds.i.smithi157.stdout:Mar 21 10:36:33 smithi157 ceph-mds[70258]: -- 172.21.15.157:0/2661593676 <== mon.2 v2:172.21.15.100:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55aaa69e9a40 con 0x55aaa69bc800 2024-03-21T10:36:34.237 INFO:journalctl@ceph.mds.i.smithi157.stdout:Mar 21 10:36:33 smithi157 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-mds-i[70254]: starting mds.i at 2024-03-21T10:36:34.237 INFO:journalctl@ceph.mds.l.smithi157.stdout:Mar 21 10:36:33 smithi157 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-mds-l[65659]: ignoring --setuser ceph since I am not root 2024-03-21T10:36:34.237 INFO:journalctl@ceph.mds.l.smithi157.stdout:Mar 21 10:36:33 smithi157 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-mds-l[65659]: ignoring --setgroup ceph since I am not root 2024-03-21T10:36:34.237 INFO:journalctl@ceph.mds.l.smithi157.stdout:Mar 21 10:36:33 smithi157 ceph-mds[65663]: -- 172.21.15.157:0/3703165373 <== mon.2 v2:172.21.15.100:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x5626cb9cba40 con 0x5626cb99e800 2024-03-21T10:36:34.238 INFO:journalctl@ceph.mds.l.smithi157.stdout:Mar 21 10:36:33 smithi157 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-mds-l[65659]: starting mds.l at 2024-03-21T10:36:34.238 INFO:journalctl@ceph.mds.c.smithi157.stdout:Mar 21 10:36:33 smithi157 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-mds-c[66294]: ignoring --setuser ceph since I am not root 2024-03-21T10:36:34.238 INFO:journalctl@ceph.mds.c.smithi157.stdout:Mar 21 10:36:33 smithi157 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-mds-c[66294]: ignoring --setgroup ceph since I am not root 2024-03-21T10:36:34.238 INFO:journalctl@ceph.mds.c.smithi157.stdout:Mar 21 10:36:33 smithi157 ceph-mds[66298]: -- 172.21.15.157:0/428971332 <== mon.1 v2:172.21.15.157:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x5593d62c5a40 con 0x5593d6298800 2024-03-21T10:36:34.238 INFO:journalctl@ceph.mds.c.smithi157.stdout:Mar 21 10:36:33 smithi157 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-mds-c[66294]: starting mds.c at 2024-03-21T10:36:34.238 INFO:journalctl@ceph.mds.f.smithi157.stdout:Mar 21 10:36:33 smithi157 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-mds-f[65979]: ignoring --setuser ceph since I am not root 2024-03-21T10:36:34.238 INFO:journalctl@ceph.mds.f.smithi157.stdout:Mar 21 10:36:33 smithi157 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-mds-f[65979]: ignoring --setgroup ceph since I am not root 2024-03-21T10:36:34.238 INFO:journalctl@ceph.mds.f.smithi157.stdout:Mar 21 10:36:33 smithi157 ceph-mds[65984]: -- 172.21.15.157:0/4065589461 <== mon.1 v2:172.21.15.157:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x555c508e5a40 con 0x555c508b9000 2024-03-21T10:36:34.239 INFO:journalctl@ceph.mds.f.smithi157.stdout:Mar 21 10:36:33 smithi157 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-mds-f[65979]: starting mds.f at 2024-03-21T10:36:34.711 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:36:34 smithi047 ceph-mon[31547]: from='client.? 172.21.15.47:0/556659138' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-03-21T10:36:34.711 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:36:34 smithi047 ceph-mon[31547]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-03-21T10:36:34.711 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:36:34 smithi047 ceph-mon[31547]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-03-21T10:36:34.711 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:36:34 smithi047 ceph-mon[31547]: from='client.? 172.21.15.47:0/556659138' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-03-21T10:36:34.711 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:36:34 smithi047 ceph-mon[31547]: osdmap e108: 12 total, 12 up, 12 in 2024-03-21T10:36:34.711 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:36:34 smithi047 ceph-mon[31547]: fsmap cephfs:0/5 2 up:standby, 5 failed 2024-03-21T10:36:34.711 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:36:34 smithi047 ceph-mon[31547]: from='client.? 172.21.15.47:0/2310630323' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-03-21T10:36:34.729 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:36:34 smithi100 ceph-mon[35193]: from='client.? 172.21.15.47:0/556659138' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-03-21T10:36:34.729 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:36:34 smithi100 ceph-mon[35193]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-03-21T10:36:34.729 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:36:34 smithi100 ceph-mon[35193]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-03-21T10:36:34.730 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:36:34 smithi100 ceph-mon[35193]: from='client.? 172.21.15.47:0/556659138' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-03-21T10:36:34.730 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:36:34 smithi100 ceph-mon[35193]: osdmap e108: 12 total, 12 up, 12 in 2024-03-21T10:36:34.730 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:36:34 smithi100 ceph-mon[35193]: fsmap cephfs:0/5 2 up:standby, 5 failed 2024-03-21T10:36:34.730 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:36:34 smithi100 ceph-mon[35193]: from='client.? 172.21.15.47:0/2310630323' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-03-21T10:36:34.736 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:36:34 smithi157 ceph-mon[35581]: from='client.? 172.21.15.47:0/556659138' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-03-21T10:36:34.737 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:36:34 smithi157 ceph-mon[35581]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-03-21T10:36:34.737 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:36:34 smithi157 ceph-mon[35581]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-03-21T10:36:34.737 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:36:34 smithi157 ceph-mon[35581]: from='client.? 172.21.15.47:0/556659138' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-03-21T10:36:34.737 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:36:34 smithi157 ceph-mon[35581]: osdmap e108: 12 total, 12 up, 12 in 2024-03-21T10:36:34.737 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:36:34 smithi157 ceph-mon[35581]: fsmap cephfs:0/5 2 up:standby, 5 failed 2024-03-21T10:36:34.737 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:36:34 smithi157 ceph-mon[35581]: from='client.? 172.21.15.47:0/2310630323' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-03-21T10:36:34.747 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:34.749+0000 7f1ad4c00640 1 -- 172.21.15.47:0/2310630323 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v34) v1 ==== 104+0+0 (secure 0 0 0) 0x7f1ac409b720 con 0x7f1ad8104200 2024-03-21T10:36:34.751 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:34.753+0000 7f1adf000640 1 -- 172.21.15.47:0/2310630323 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f1abc0a3090 msgr2=0x7f1abc0a5550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:36:34.751 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:34.753+0000 7f1adf000640 1 --2- 172.21.15.47:0/2310630323 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f1abc0a3090 0x7f1abc0a5550 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f1ac8036040 tx=0x7f1ac8000bc0 comp rx=0 tx=0).stop 2024-03-21T10:36:34.751 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:34.753+0000 7f1adf000640 1 -- 172.21.15.47:0/2310630323 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1ad8104200 msgr2=0x7f1ad819a5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:36:34.751 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:34.753+0000 7f1adf000640 1 --2- 172.21.15.47:0/2310630323 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1ad8104200 0x7f1ad819a5f0 secure :-1 s=READY pgs=1364 cs=0 l=1 rev1=1 crypto rx=0x7f1ac400d950 tx=0x7f1ac400de10 comp rx=0 tx=0).stop 2024-03-21T10:36:34.751 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:34.753+0000 7f1adf000640 1 -- 172.21.15.47:0/2310630323 shutdown_connections 2024-03-21T10:36:34.751 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:34.753+0000 7f1adf000640 1 --2- 172.21.15.47:0/2310630323 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f1abc0a3090 0x7f1abc0a5550 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:36:34.751 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:34.753+0000 7f1adf000640 1 --2- 172.21.15.47:0/2310630323 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1ad8109850 0x7f1ad819e970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:36:34.752 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:34.753+0000 7f1adf000640 1 --2- 172.21.15.47:0/2310630323 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1ad8104200 0x7f1ad819a5f0 unknown :-1 s=CLOSED pgs=1364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:36:34.752 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:34.753+0000 7f1adf000640 1 --2- 172.21.15.47:0/2310630323 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1ad8103840 0x7f1ad819a0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:36:34.752 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:34.753+0000 7f1adf000640 1 -- 172.21.15.47:0/2310630323 >> 172.21.15.47:0/2310630323 conn(0x7f1ad80ff2f0 msgr2=0x7f1ad80ffc30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:36:34.752 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:34.754+0000 7f1adf000640 1 -- 172.21.15.47:0/2310630323 shutdown_connections 2024-03-21T10:36:34.752 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:34.754+0000 7f1adf000640 1 -- 172.21.15.47:0/2310630323 wait complete. 2024-03-21T10:36:34.778 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-03-21T10:36:34.987 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:34.989+0000 7f96e6e00640 1 -- 172.21.15.47:0/2030592371 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f96e0101550 msgr2=0x7f96e01019d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:36:34.987 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:34.989+0000 7f96e6e00640 1 --2- 172.21.15.47:0/2030592371 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f96e0101550 0x7f96e01019d0 secure :-1 s=READY pgs=1365 cs=0 l=1 rev1=1 crypto rx=0x7f96cc009900 tx=0x7f96cc02bc30 comp rx=0 tx=0).stop 2024-03-21T10:36:34.987 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:34.989+0000 7f96e6e00640 1 -- 172.21.15.47:0/2030592371 shutdown_connections 2024-03-21T10:36:34.987 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:34.989+0000 7f96e6e00640 1 --2- 172.21.15.47:0/2030592371 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f96e0101f10 0x7f96e010e3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:36:34.987 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:34.989+0000 7f96e6e00640 1 --2- 172.21.15.47:0/2030592371 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f96e0101550 0x7f96e01019d0 unknown :-1 s=CLOSED pgs=1365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:36:34.987 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:34.989+0000 7f96e6e00640 1 --2- 172.21.15.47:0/2030592371 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f96e0107610 0x7f96e01079f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:36:34.988 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:34.989+0000 7f96e6e00640 1 -- 172.21.15.47:0/2030592371 >> 172.21.15.47:0/2030592371 conn(0x7f96e00fd040 msgr2=0x7f96e00ff460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:36:34.988 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:34.989+0000 7f96e6e00640 1 -- 172.21.15.47:0/2030592371 shutdown_connections 2024-03-21T10:36:34.988 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:34.989+0000 7f96e6e00640 1 -- 172.21.15.47:0/2030592371 wait complete. 2024-03-21T10:36:34.989 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:34.990+0000 7f96e6e00640 1 Processor -- start 2024-03-21T10:36:34.990 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:34.990+0000 7f96e6e00640 1 -- start start 2024-03-21T10:36:34.990 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:34.990+0000 7f96e6e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f96e0101550 0x7f96e0199e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:36:34.990 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:34.990+0000 7f96e6e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f96e0101f10 0x7f96e019a3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:36:34.990 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:34.990+0000 7f96e6e00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f96e0107610 0x7f96e019e740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:36:34.990 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:34.991+0000 7f96e6e00640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96e0111780 con 0x7f96e0101550 2024-03-21T10:36:34.990 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:34.991+0000 7f96e6e00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96e0111600 con 0x7f96e0101f10 2024-03-21T10:36:34.990 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:34.991+0000 7f96e6e00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96e0111900 con 0x7f96e0107610 2024-03-21T10:36:34.990 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:34.991+0000 7f96e5a00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f96e0101550 0x7f96e0199e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:36:34.990 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:34.991+0000 7f96e5a00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f96e0101550 0x7f96e0199e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:34754/0 (socket says 172.21.15.47:34754) 2024-03-21T10:36:34.990 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:34.991+0000 7f96e5a00640 1 -- 172.21.15.47:0/3900663626 learned_addr learned my addr 172.21.15.47:0/3900663626 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:36:34.990 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:34.991+0000 7f96e5000640 1 --2- 172.21.15.47:0/3900663626 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f96e0101f10 0x7f96e019a3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:36:34.990 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:34.991+0000 7f96e6400640 1 --2- 172.21.15.47:0/3900663626 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f96e0107610 0x7f96e019e740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:36:34.990 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:34.991+0000 7f96e5a00640 1 -- 172.21.15.47:0/3900663626 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f96e0107610 msgr2=0x7f96e019e740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:36:34.991 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:34.991+0000 7f96e5a00640 1 --2- 172.21.15.47:0/3900663626 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f96e0107610 0x7f96e019e740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:36:34.991 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:34.991+0000 7f96e5a00640 1 -- 172.21.15.47:0/3900663626 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f96e0101f10 msgr2=0x7f96e019a3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:36:34.991 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:34.991+0000 7f96e5a00640 1 --2- 172.21.15.47:0/3900663626 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f96e0101f10 0x7f96e019a3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:36:34.991 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:34.991+0000 7f96e5a00640 1 -- 172.21.15.47:0/3900663626 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96e019ee70 con 0x7f96e0101550 2024-03-21T10:36:34.991 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:34.991+0000 7f96e5000640 1 --2- 172.21.15.47:0/3900663626 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f96e0101f10 0x7f96e019a3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T10:36:34.991 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:34.992+0000 7f96e5a00640 1 --2- 172.21.15.47:0/3900663626 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f96e0101550 0x7f96e0199e80 secure :-1 s=READY pgs=1366 cs=0 l=1 rev1=1 crypto rx=0x7f96d400d950 tx=0x7f96d400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:36:34.992 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:34.993+0000 7f96de000640 1 -- 172.21.15.47:0/3900663626 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96d4017070 con 0x7f96e0101550 2024-03-21T10:36:34.992 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:34.993+0000 7f96de000640 1 -- 172.21.15.47:0/3900663626 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f96d4009c30 con 0x7f96e0101550 2024-03-21T10:36:34.992 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:34.993+0000 7f96e6e00640 1 -- 172.21.15.47:0/3900663626 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f96e019f160 con 0x7f96e0101550 2024-03-21T10:36:34.992 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:34.993+0000 7f96de000640 1 -- 172.21.15.47:0/3900663626 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96d400f620 con 0x7f96e0101550 2024-03-21T10:36:34.992 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:34.993+0000 7f96e6e00640 1 -- 172.21.15.47:0/3900663626 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f96e01a69f0 con 0x7f96e0101550 2024-03-21T10:36:34.992 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:34.994+0000 7f96e6e00640 1 -- 172.21.15.47:0/3900663626 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f96e0068800 con 0x7f96e0101550 2024-03-21T10:36:34.994 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:34.996+0000 7f96de000640 1 -- 172.21.15.47:0/3900663626 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 27) v1 ==== 137985+0+0 (secure 0 0 0) 0x7f96d40109a0 con 0x7f96e0101550 2024-03-21T10:36:34.995 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:34.997+0000 7f96de000640 1 --2- 172.21.15.47:0/3900663626 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f96c80a3110 0x7f96c80a55d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:36:34.995 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:34.998+0000 7f96de000640 1 -- 172.21.15.47:0/3900663626 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(108..108 src has 1..108) v4 ==== 9904+0+0 (secure 0 0 0) 0x7f96d40cfa10 con 0x7f96e0101550 2024-03-21T10:36:34.995 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:34.998+0000 7f96e5000640 1 --2- 172.21.15.47:0/3900663626 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f96c80a3110 0x7f96c80a55d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:36:34.996 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:34.999+0000 7f96e5000640 1 --2- 172.21.15.47:0/3900663626 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f96c80a3110 0x7f96c80a55d0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f96cc036040 tx=0x7f96cc000bc0 comp rx=0 tx=0).ready entity=mgr.15687 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:36:34.999 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:35.002+0000 7f96de000640 1 -- 172.21.15.47:0/3900663626 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f96d4009de0 con 0x7f96e0101550 2024-03-21T10:36:35.197 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:35.199+0000 7f96e6e00640 1 -- 172.21.15.47:0/3900663626 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f96e01a6c20 con 0x7f96e0101550 2024-03-21T10:36:35.711 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:36:35 smithi047 ceph-mon[31547]: pgmap v352: 129 pgs: 129 active+clean; 2.5 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 1.1 KiB/s wr, 12 op/s 2024-03-21T10:36:35.712 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:36:35 smithi047 ceph-mon[31547]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-03-21T10:36:35.712 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:36:35 smithi047 ceph-mon[31547]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-03-21T10:36:35.712 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:36:35 smithi047 ceph-mon[31547]: Cluster is now healthy 2024-03-21T10:36:35.712 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:36:35 smithi047 ceph-mon[31547]: from='client.? 172.21.15.47:0/2310630323' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-03-21T10:36:35.712 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:36:35 smithi047 ceph-mon[31547]: mds.? [v2:172.21.15.47:6838/3404370258,v1:172.21.15.47:6839/3404370258] up:boot 2024-03-21T10:36:35.712 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:36:35 smithi047 ceph-mon[31547]: mds.? [v2:172.21.15.47:6840/1277788103,v1:172.21.15.47:6841/1277788103] up:boot 2024-03-21T10:36:35.712 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:36:35 smithi047 ceph-mon[31547]: mds.? [v2:172.21.15.100:6832/249774652,v1:172.21.15.100:6833/249774652] up:boot 2024-03-21T10:36:35.712 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:36:35 smithi047 ceph-mon[31547]: mds.? [v2:172.21.15.100:6834/150109915,v1:172.21.15.100:6835/150109915] up:boot 2024-03-21T10:36:35.712 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:36:35 smithi047 ceph-mon[31547]: mds.? [v2:172.21.15.157:6832/2286478028,v1:172.21.15.157:6833/2286478028] up:boot 2024-03-21T10:36:35.712 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:36:35 smithi047 ceph-mon[31547]: mds.? [v2:172.21.15.100:6836/3797982324,v1:172.21.15.100:6837/3797982324] up:boot 2024-03-21T10:36:35.712 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:36:35 smithi047 ceph-mon[31547]: mds.? [v2:172.21.15.100:6838/2801987363,v1:172.21.15.100:6839/2801987363] up:boot 2024-03-21T10:36:35.712 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:36:35 smithi047 ceph-mon[31547]: mds.? [v2:172.21.15.157:6834/199575835,v1:172.21.15.157:6836/199575835] up:boot 2024-03-21T10:36:35.712 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:36:35 smithi047 ceph-mon[31547]: mds.? [v2:172.21.15.157:6835/1229672645,v1:172.21.15.157:6838/1229672645] up:boot 2024-03-21T10:36:35.713 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:36:35 smithi047 ceph-mon[31547]: mds.? [v2:172.21.15.157:6837/1659333233,v1:172.21.15.157:6839/1659333233] up:boot 2024-03-21T10:36:35.713 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:36:35 smithi047 ceph-mon[31547]: fsmap 12 up:standby 2024-03-21T10:36:35.713 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:36:35 smithi047 ceph-mon[31547]: from='mgr.15687 172.21.15.47:0/8730661' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-03-21T10:36:35.713 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:36:35 smithi047 ceph-mon[31547]: from='mgr.15687 172.21.15.47:0/8730661' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-03-21T10:36:35.713 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:36:35 smithi047 ceph-mon[31547]: from='mgr.15687 172.21.15.47:0/8730661' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-03-21T10:36:35.713 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:36:35 smithi047 ceph-mon[31547]: from='mgr.15687 172.21.15.47:0/8730661' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-03-21T10:36:35.713 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:36:35 smithi047 ceph-mon[31547]: from='mgr.15687 172.21.15.47:0/8730661' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-03-21T10:36:35.713 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:36:35 smithi047 ceph-mon[31547]: from='mgr.15687 172.21.15.47:0/8730661' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-03-21T10:36:35.713 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:36:35 smithi047 ceph-mon[31547]: from='mgr.15687 172.21.15.47:0/8730661' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-03-21T10:36:35.713 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:36:35 smithi047 ceph-mon[31547]: from='mgr.15687 172.21.15.47:0/8730661' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-03-21T10:36:35.713 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:36:35 smithi047 ceph-mon[31547]: from='mgr.15687 172.21.15.47:0/8730661' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-03-21T10:36:35.713 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:36:35 smithi047 ceph-mon[31547]: from='mgr.15687 172.21.15.47:0/8730661' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-03-21T10:36:35.713 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:36:35 smithi047 ceph-mon[31547]: from='client.? 172.21.15.47:0/3900663626' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-03-21T10:36:35.727 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:36:35 smithi100 ceph-mon[35193]: pgmap v352: 129 pgs: 129 active+clean; 2.5 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 1.1 KiB/s wr, 12 op/s 2024-03-21T10:36:35.727 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:36:35 smithi100 ceph-mon[35193]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-03-21T10:36:35.727 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:36:35 smithi100 ceph-mon[35193]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-03-21T10:36:35.728 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:36:35 smithi100 ceph-mon[35193]: Cluster is now healthy 2024-03-21T10:36:35.728 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:36:35 smithi100 ceph-mon[35193]: from='client.? 172.21.15.47:0/2310630323' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-03-21T10:36:35.728 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:36:35 smithi100 ceph-mon[35193]: mds.? [v2:172.21.15.47:6838/3404370258,v1:172.21.15.47:6839/3404370258] up:boot 2024-03-21T10:36:35.728 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:36:35 smithi100 ceph-mon[35193]: mds.? [v2:172.21.15.47:6840/1277788103,v1:172.21.15.47:6841/1277788103] up:boot 2024-03-21T10:36:35.728 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:36:35 smithi100 ceph-mon[35193]: mds.? [v2:172.21.15.100:6832/249774652,v1:172.21.15.100:6833/249774652] up:boot 2024-03-21T10:36:35.728 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:36:35 smithi100 ceph-mon[35193]: mds.? [v2:172.21.15.100:6834/150109915,v1:172.21.15.100:6835/150109915] up:boot 2024-03-21T10:36:35.728 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:36:35 smithi100 ceph-mon[35193]: mds.? [v2:172.21.15.157:6832/2286478028,v1:172.21.15.157:6833/2286478028] up:boot 2024-03-21T10:36:35.728 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:36:35 smithi100 ceph-mon[35193]: mds.? [v2:172.21.15.100:6836/3797982324,v1:172.21.15.100:6837/3797982324] up:boot 2024-03-21T10:36:35.728 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:36:35 smithi100 ceph-mon[35193]: mds.? [v2:172.21.15.100:6838/2801987363,v1:172.21.15.100:6839/2801987363] up:boot 2024-03-21T10:36:35.728 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:36:35 smithi100 ceph-mon[35193]: mds.? [v2:172.21.15.157:6834/199575835,v1:172.21.15.157:6836/199575835] up:boot 2024-03-21T10:36:35.728 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:36:35 smithi100 ceph-mon[35193]: mds.? [v2:172.21.15.157:6835/1229672645,v1:172.21.15.157:6838/1229672645] up:boot 2024-03-21T10:36:35.728 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:36:35 smithi100 ceph-mon[35193]: mds.? [v2:172.21.15.157:6837/1659333233,v1:172.21.15.157:6839/1659333233] up:boot 2024-03-21T10:36:35.728 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:36:35 smithi100 ceph-mon[35193]: fsmap 12 up:standby 2024-03-21T10:36:35.728 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:36:35 smithi100 ceph-mon[35193]: from='mgr.15687 172.21.15.47:0/8730661' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-03-21T10:36:35.728 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:36:35 smithi100 ceph-mon[35193]: from='mgr.15687 172.21.15.47:0/8730661' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-03-21T10:36:35.728 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:36:35 smithi100 ceph-mon[35193]: from='mgr.15687 172.21.15.47:0/8730661' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-03-21T10:36:35.729 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:36:35 smithi100 ceph-mon[35193]: from='mgr.15687 172.21.15.47:0/8730661' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-03-21T10:36:35.729 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:36:35 smithi100 ceph-mon[35193]: from='mgr.15687 172.21.15.47:0/8730661' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-03-21T10:36:35.729 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:36:35 smithi100 ceph-mon[35193]: from='mgr.15687 172.21.15.47:0/8730661' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-03-21T10:36:35.729 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:36:35 smithi100 ceph-mon[35193]: from='mgr.15687 172.21.15.47:0/8730661' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-03-21T10:36:35.729 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:36:35 smithi100 ceph-mon[35193]: from='mgr.15687 172.21.15.47:0/8730661' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-03-21T10:36:35.729 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:36:35 smithi100 ceph-mon[35193]: from='mgr.15687 172.21.15.47:0/8730661' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-03-21T10:36:35.729 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:36:35 smithi100 ceph-mon[35193]: from='mgr.15687 172.21.15.47:0/8730661' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-03-21T10:36:35.729 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:36:35 smithi100 ceph-mon[35193]: from='client.? 172.21.15.47:0/3900663626' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-03-21T10:36:35.737 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:36:35 smithi157 ceph-mon[35581]: pgmap v352: 129 pgs: 129 active+clean; 2.5 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 1.1 KiB/s wr, 12 op/s 2024-03-21T10:36:35.737 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:36:35 smithi157 ceph-mon[35581]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-03-21T10:36:35.737 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:36:35 smithi157 ceph-mon[35581]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-03-21T10:36:35.737 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:36:35 smithi157 ceph-mon[35581]: Cluster is now healthy 2024-03-21T10:36:35.738 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:36:35 smithi157 ceph-mon[35581]: from='client.? 172.21.15.47:0/2310630323' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-03-21T10:36:35.738 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:36:35 smithi157 ceph-mon[35581]: mds.? [v2:172.21.15.47:6838/3404370258,v1:172.21.15.47:6839/3404370258] up:boot 2024-03-21T10:36:35.738 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:36:35 smithi157 ceph-mon[35581]: mds.? [v2:172.21.15.47:6840/1277788103,v1:172.21.15.47:6841/1277788103] up:boot 2024-03-21T10:36:35.738 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:36:35 smithi157 ceph-mon[35581]: mds.? [v2:172.21.15.100:6832/249774652,v1:172.21.15.100:6833/249774652] up:boot 2024-03-21T10:36:35.738 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:36:35 smithi157 ceph-mon[35581]: mds.? [v2:172.21.15.100:6834/150109915,v1:172.21.15.100:6835/150109915] up:boot 2024-03-21T10:36:35.738 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:36:35 smithi157 ceph-mon[35581]: mds.? [v2:172.21.15.157:6832/2286478028,v1:172.21.15.157:6833/2286478028] up:boot 2024-03-21T10:36:35.738 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:36:35 smithi157 ceph-mon[35581]: mds.? [v2:172.21.15.100:6836/3797982324,v1:172.21.15.100:6837/3797982324] up:boot 2024-03-21T10:36:35.738 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:36:35 smithi157 ceph-mon[35581]: mds.? [v2:172.21.15.100:6838/2801987363,v1:172.21.15.100:6839/2801987363] up:boot 2024-03-21T10:36:35.738 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:36:35 smithi157 ceph-mon[35581]: mds.? [v2:172.21.15.157:6834/199575835,v1:172.21.15.157:6836/199575835] up:boot 2024-03-21T10:36:35.738 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:36:35 smithi157 ceph-mon[35581]: mds.? [v2:172.21.15.157:6835/1229672645,v1:172.21.15.157:6838/1229672645] up:boot 2024-03-21T10:36:35.738 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:36:35 smithi157 ceph-mon[35581]: mds.? [v2:172.21.15.157:6837/1659333233,v1:172.21.15.157:6839/1659333233] up:boot 2024-03-21T10:36:35.738 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:36:35 smithi157 ceph-mon[35581]: fsmap 12 up:standby 2024-03-21T10:36:35.738 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:36:35 smithi157 ceph-mon[35581]: from='mgr.15687 172.21.15.47:0/8730661' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-03-21T10:36:35.738 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:36:35 smithi157 ceph-mon[35581]: from='mgr.15687 172.21.15.47:0/8730661' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-03-21T10:36:35.738 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:36:35 smithi157 ceph-mon[35581]: from='mgr.15687 172.21.15.47:0/8730661' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-03-21T10:36:35.739 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:36:35 smithi157 ceph-mon[35581]: from='mgr.15687 172.21.15.47:0/8730661' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-03-21T10:36:35.739 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:36:35 smithi157 ceph-mon[35581]: from='mgr.15687 172.21.15.47:0/8730661' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-03-21T10:36:35.739 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:36:35 smithi157 ceph-mon[35581]: from='mgr.15687 172.21.15.47:0/8730661' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-03-21T10:36:35.739 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:36:35 smithi157 ceph-mon[35581]: from='mgr.15687 172.21.15.47:0/8730661' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-03-21T10:36:35.739 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:36:35 smithi157 ceph-mon[35581]: from='mgr.15687 172.21.15.47:0/8730661' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-03-21T10:36:35.739 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:36:35 smithi157 ceph-mon[35581]: from='mgr.15687 172.21.15.47:0/8730661' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-03-21T10:36:35.739 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:36:35 smithi157 ceph-mon[35581]: from='mgr.15687 172.21.15.47:0/8730661' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-03-21T10:36:35.739 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:36:35 smithi157 ceph-mon[35581]: from='client.? 172.21.15.47:0/3900663626' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-03-21T10:36:35.763 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:35.765+0000 7f96de000640 1 -- 172.21.15.47:0/3900663626 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v109) v1 ==== 181+0+0 (secure 0 0 0) 0x7f96d409bf60 con 0x7f96e0101550 2024-03-21T10:36:35.763 INFO:teuthology.orchestra.run.smithi047.stderr:pool 'cephfs_metadata' removed 2024-03-21T10:36:35.767 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:35.769+0000 7f96e6e00640 1 -- 172.21.15.47:0/3900663626 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f96c80a3110 msgr2=0x7f96c80a55d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:36:35.767 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:35.769+0000 7f96e6e00640 1 --2- 172.21.15.47:0/3900663626 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f96c80a3110 0x7f96c80a55d0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f96cc036040 tx=0x7f96cc000bc0 comp rx=0 tx=0).stop 2024-03-21T10:36:35.767 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:35.769+0000 7f96e6e00640 1 -- 172.21.15.47:0/3900663626 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f96e0101550 msgr2=0x7f96e0199e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:36:35.767 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:35.769+0000 7f96e6e00640 1 --2- 172.21.15.47:0/3900663626 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f96e0101550 0x7f96e0199e80 secure :-1 s=READY pgs=1366 cs=0 l=1 rev1=1 crypto rx=0x7f96d400d950 tx=0x7f96d400de10 comp rx=0 tx=0).stop 2024-03-21T10:36:35.767 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:35.770+0000 7f96e6e00640 1 -- 172.21.15.47:0/3900663626 shutdown_connections 2024-03-21T10:36:35.768 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:35.770+0000 7f96e6e00640 1 --2- 172.21.15.47:0/3900663626 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f96e0107610 0x7f96e019e740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:36:35.768 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:35.770+0000 7f96e6e00640 1 --2- 172.21.15.47:0/3900663626 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f96c80a3110 0x7f96c80a55d0 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:36:35.768 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:35.770+0000 7f96e6e00640 1 --2- 172.21.15.47:0/3900663626 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f96e0101f10 0x7f96e019a3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:36:35.768 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:35.770+0000 7f96e6e00640 1 --2- 172.21.15.47:0/3900663626 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f96e0101550 0x7f96e0199e80 unknown :-1 s=CLOSED pgs=1366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:36:35.768 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:35.770+0000 7f96e6e00640 1 -- 172.21.15.47:0/3900663626 >> 172.21.15.47:0/3900663626 conn(0x7f96e00fd040 msgr2=0x7f96e00fe5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:36:35.768 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:35.771+0000 7f96e6e00640 1 -- 172.21.15.47:0/3900663626 shutdown_connections 2024-03-21T10:36:35.769 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:35.772+0000 7f96e6e00640 1 -- 172.21.15.47:0/3900663626 wait complete. 2024-03-21T10:36:35.794 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-03-21T10:36:36.016 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:36.018+0000 7f2337200640 1 -- 172.21.15.47:0/3341069623 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2338109730 msgr2=0x7f2338109b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:36:36.017 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:36.018+0000 7f2337200640 1 --2- 172.21.15.47:0/3341069623 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2338109730 0x7f2338109b10 secure :-1 s=READY pgs=1367 cs=0 l=1 rev1=1 crypto rx=0x7f2328009930 tx=0x7f232802bc30 comp rx=0 tx=0).stop 2024-03-21T10:36:36.017 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:36.018+0000 7f2337200640 1 -- 172.21.15.47:0/3341069623 shutdown_connections 2024-03-21T10:36:36.017 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:36.018+0000 7f2337200640 1 --2- 172.21.15.47:0/3341069623 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f23381040e0 0x7f23381105f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:36:36.017 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:36.018+0000 7f2337200640 1 --2- 172.21.15.47:0/3341069623 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2338103720 0x7f2338103ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:36:36.017 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:36.018+0000 7f2337200640 1 --2- 172.21.15.47:0/3341069623 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2338109730 0x7f2338109b10 unknown :-1 s=CLOSED pgs=1367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:36:36.017 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:36.018+0000 7f2337200640 1 -- 172.21.15.47:0/3341069623 >> 172.21.15.47:0/3341069623 conn(0x7f23380ff1b0 msgr2=0x7f23381015d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:36:36.017 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:36.019+0000 7f2337200640 1 -- 172.21.15.47:0/3341069623 shutdown_connections 2024-03-21T10:36:36.017 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:36.019+0000 7f2337200640 1 -- 172.21.15.47:0/3341069623 wait complete. 2024-03-21T10:36:36.019 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:36.020+0000 7f2337200640 1 Processor -- start 2024-03-21T10:36:36.019 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:36.020+0000 7f2337200640 1 -- start start 2024-03-21T10:36:36.019 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:36.020+0000 7f2337200640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2338103720 0x7f233819a000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:36:36.019 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:36.020+0000 7f2337200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f23381040e0 0x7f233819a540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:36:36.019 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:36.020+0000 7f2337200640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2338109730 0x7f233819e8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:36:36.020 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:36.020+0000 7f2337200640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2338113a40 con 0x7f2338103720 2024-03-21T10:36:36.020 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:36.020+0000 7f2337200640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23381138c0 con 0x7f23381040e0 2024-03-21T10:36:36.020 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:36.020+0000 7f2337200640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2338113bc0 con 0x7f2338109730 2024-03-21T10:36:36.020 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:36.020+0000 7f2335e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2338103720 0x7f233819a000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:36:36.020 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:36.020+0000 7f2335e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2338103720 0x7f233819a000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:34766/0 (socket says 172.21.15.47:34766) 2024-03-21T10:36:36.020 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:36.020+0000 7f2335e00640 1 -- 172.21.15.47:0/120085467 learned_addr learned my addr 172.21.15.47:0/120085467 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:36:36.020 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:36.021+0000 7f2335400640 1 --2- 172.21.15.47:0/120085467 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f23381040e0 0x7f233819a540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:36:36.020 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:36.021+0000 7f2336800640 1 --2- 172.21.15.47:0/120085467 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2338109730 0x7f233819e8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:36:36.020 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:36.021+0000 7f2335e00640 1 -- 172.21.15.47:0/120085467 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2338109730 msgr2=0x7f233819e8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:36:36.020 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:36.021+0000 7f2335e00640 1 --2- 172.21.15.47:0/120085467 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2338109730 0x7f233819e8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:36:36.020 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:36.021+0000 7f2335e00640 1 -- 172.21.15.47:0/120085467 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f23381040e0 msgr2=0x7f233819a540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:36:36.020 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:36.021+0000 7f2335e00640 1 --2- 172.21.15.47:0/120085467 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f23381040e0 0x7f233819a540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:36:36.020 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:36.021+0000 7f2335e00640 1 -- 172.21.15.47:0/120085467 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f233819eff0 con 0x7f2338103720 2024-03-21T10:36:36.020 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:36.021+0000 7f2335400640 1 --2- 172.21.15.47:0/120085467 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f23381040e0 0x7f233819a540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T10:36:36.021 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:36.021+0000 7f2335e00640 1 --2- 172.21.15.47:0/120085467 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2338103720 0x7f233819a000 secure :-1 s=READY pgs=1368 cs=0 l=1 rev1=1 crypto rx=0x7f2328000c00 tx=0x7f232802c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:36:36.021 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:36.022+0000 7f232ea00640 1 -- 172.21.15.47:0/120085467 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2328043020 con 0x7f2338103720 2024-03-21T10:36:36.021 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:36.022+0000 7f232ea00640 1 -- 172.21.15.47:0/120085467 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f232802cc60 con 0x7f2338103720 2024-03-21T10:36:36.021 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:36.022+0000 7f232ea00640 1 -- 172.21.15.47:0/120085467 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2328003c30 con 0x7f2338103720 2024-03-21T10:36:36.021 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:36.022+0000 7f2337200640 1 -- 172.21.15.47:0/120085467 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f233819f280 con 0x7f2338103720 2024-03-21T10:36:36.021 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:36.022+0000 7f2337200640 1 -- 172.21.15.47:0/120085467 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f23381a6b70 con 0x7f2338103720 2024-03-21T10:36:36.021 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:36.023+0000 7f2337200640 1 -- 172.21.15.47:0/120085467 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f23380683b0 con 0x7f2338103720 2024-03-21T10:36:36.022 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:36.025+0000 7f232ea00640 1 -- 172.21.15.47:0/120085467 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 27) v1 ==== 137985+0+0 (secure 0 0 0) 0x7f2328046020 con 0x7f2338103720 2024-03-21T10:36:36.024 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:36.026+0000 7f232ea00640 1 --2- 172.21.15.47:0/120085467 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f231c0a3110 0x7f231c0a55d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:36:36.024 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:36.026+0000 7f232ea00640 1 -- 172.21.15.47:0/120085467 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(109..109 src has 1..109) v4 ==== 9321+0+0 (secure 0 0 0) 0x7f23280f1d50 con 0x7f2338103720 2024-03-21T10:36:36.024 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:36.027+0000 7f2335400640 1 --2- 172.21.15.47:0/120085467 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f231c0a3110 0x7f231c0a55d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:36:36.025 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:36.027+0000 7f2335400640 1 --2- 172.21.15.47:0/120085467 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f231c0a3110 0x7f231c0a55d0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f2338101930 tx=0x7f2324005e50 comp rx=0 tx=0).ready entity=mgr.15687 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:36:36.028 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:36.031+0000 7f232ea00640 1 -- 172.21.15.47:0/120085467 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f232804d020 con 0x7f2338103720 2024-03-21T10:36:36.233 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:36.236+0000 7f2337200640 1 -- 172.21.15.47:0/120085467 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f233810bce0 con 0x7f2338103720 2024-03-21T10:36:36.774 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:36.776+0000 7f232ea00640 1 -- 172.21.15.47:0/120085467 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v110) v1 ==== 169+0+0 (secure 0 0 0) 0x7f233810bce0 con 0x7f2338103720 2024-03-21T10:36:36.775 INFO:teuthology.orchestra.run.smithi047.stderr:pool 'cephfs_data' removed 2024-03-21T10:36:36.778 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:36.780+0000 7f2337200640 1 -- 172.21.15.47:0/120085467 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f231c0a3110 msgr2=0x7f231c0a55d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:36:36.778 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:36.780+0000 7f2337200640 1 --2- 172.21.15.47:0/120085467 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f231c0a3110 0x7f231c0a55d0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f2338101930 tx=0x7f2324005e50 comp rx=0 tx=0).stop 2024-03-21T10:36:36.778 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:36.780+0000 7f2337200640 1 -- 172.21.15.47:0/120085467 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2338103720 msgr2=0x7f233819a000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:36:36.778 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:36.780+0000 7f2337200640 1 --2- 172.21.15.47:0/120085467 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2338103720 0x7f233819a000 secure :-1 s=READY pgs=1368 cs=0 l=1 rev1=1 crypto rx=0x7f2328000c00 tx=0x7f232802c860 comp rx=0 tx=0).stop 2024-03-21T10:36:36.779 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:36.781+0000 7f2337200640 1 -- 172.21.15.47:0/120085467 shutdown_connections 2024-03-21T10:36:36.779 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:36.781+0000 7f2337200640 1 --2- 172.21.15.47:0/120085467 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2338109730 0x7f233819e8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:36:36.779 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:36.781+0000 7f2337200640 1 --2- 172.21.15.47:0/120085467 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7f231c0a3110 0x7f231c0a55d0 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:36:36.779 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:36.781+0000 7f2337200640 1 --2- 172.21.15.47:0/120085467 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f23381040e0 0x7f233819a540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:36:36.779 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:36.781+0000 7f2337200640 1 --2- 172.21.15.47:0/120085467 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2338103720 0x7f233819a000 unknown :-1 s=CLOSED pgs=1368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:36:36.779 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:36.781+0000 7f2337200640 1 -- 172.21.15.47:0/120085467 >> 172.21.15.47:0/120085467 conn(0x7f23380ff1b0 msgr2=0x7f23380ffb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:36:36.781 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:36.783+0000 7f2337200640 1 -- 172.21.15.47:0/120085467 shutdown_connections 2024-03-21T10:36:36.781 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:36.783+0000 7f2337200640 1 -- 172.21.15.47:0/120085467 wait complete. 2024-03-21T10:36:36.811 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data_ec cephfs_data_ec --yes-i-really-really-mean-it 2024-03-21T10:36:37.025 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:37.026+0000 7fca9e000640 1 -- 172.21.15.47:0/1672047051 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fca98074360 msgr2=0x7fca98112100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:36:37.025 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:37.026+0000 7fca9e000640 1 --2- 172.21.15.47:0/1672047051 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fca98074360 0x7fca98112100 secure :-1 s=READY pgs=1369 cs=0 l=1 rev1=1 crypto rx=0x7fca84009900 tx=0x7fca8402bcd0 comp rx=0 tx=0).stop 2024-03-21T10:36:37.025 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:37.027+0000 7fca9e000640 1 -- 172.21.15.47:0/1672047051 shutdown_connections 2024-03-21T10:36:37.025 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:37.027+0000 7fca9e000640 1 --2- 172.21.15.47:0/1672047051 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fca98112640 0x7fca98114a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:36:37.025 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:37.027+0000 7fca9e000640 1 --2- 172.21.15.47:0/1672047051 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fca98074360 0x7fca98112100 unknown :-1 s=CLOSED pgs=1369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:36:37.025 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:37.027+0000 7fca9e000640 1 --2- 172.21.15.47:0/1672047051 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fca98073a40 0x7fca98073e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:36:37.026 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:37.027+0000 7fca9e000640 1 -- 172.21.15.47:0/1672047051 >> 172.21.15.47:0/1672047051 conn(0x7fca98078aa0 msgr2=0x7fca98078eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:36:37.026 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:37.027+0000 7fca9e000640 1 -- 172.21.15.47:0/1672047051 shutdown_connections 2024-03-21T10:36:37.026 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:37.027+0000 7fca9e000640 1 -- 172.21.15.47:0/1672047051 wait complete. 2024-03-21T10:36:37.028 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:37.028+0000 7fca9e000640 1 Processor -- start 2024-03-21T10:36:37.028 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:37.028+0000 7fca9e000640 1 -- start start 2024-03-21T10:36:37.028 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:37.028+0000 7fca9e000640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fca98073a40 0x7fca9819e490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:36:37.028 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:37.028+0000 7fca9e000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fca98074360 0x7fca9819e9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:36:37.028 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:37.028+0000 7fca9e000640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fca98112640 0x7fca981a2d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:36:37.028 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:37.028+0000 7fca9e000640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca98117ef0 con 0x7fca98112640 2024-03-21T10:36:37.029 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:37.028+0000 7fca9e000640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca98117d70 con 0x7fca98074360 2024-03-21T10:36:37.029 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:37.028+0000 7fca9e000640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca98118070 con 0x7fca98073a40 2024-03-21T10:36:37.029 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:37.029+0000 7fca97e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fca98112640 0x7fca981a2d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:36:37.029 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:37.029+0000 7fca97e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fca98112640 0x7fca981a2d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:34774/0 (socket says 172.21.15.47:34774) 2024-03-21T10:36:37.029 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:37.029+0000 7fca97e00640 1 -- 172.21.15.47:0/4268834561 learned_addr learned my addr 172.21.15.47:0/4268834561 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-03-21T10:36:37.029 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:37.029+0000 7fca96a00640 1 --2- 172.21.15.47:0/4268834561 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fca98074360 0x7fca9819e9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:36:37.029 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:37.029+0000 7fca97400640 1 --2- 172.21.15.47:0/4268834561 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fca98073a40 0x7fca9819e490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:36:37.029 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:37.029+0000 7fca97e00640 1 -- 172.21.15.47:0/4268834561 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fca98073a40 msgr2=0x7fca9819e490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:36:37.029 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:37.029+0000 7fca97e00640 1 --2- 172.21.15.47:0/4268834561 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fca98073a40 0x7fca9819e490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:36:37.029 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:37.029+0000 7fca97e00640 1 -- 172.21.15.47:0/4268834561 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fca98074360 msgr2=0x7fca9819e9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:36:37.029 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:37.029+0000 7fca97e00640 1 --2- 172.21.15.47:0/4268834561 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fca98074360 0x7fca9819e9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:36:37.029 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:37.029+0000 7fca97e00640 1 -- 172.21.15.47:0/4268834561 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca981a3480 con 0x7fca98112640 2024-03-21T10:36:37.029 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:37.029+0000 7fca97400640 1 --2- 172.21.15.47:0/4268834561 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fca98073a40 0x7fca9819e490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T10:36:37.029 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:37.029+0000 7fca97e00640 1 --2- 172.21.15.47:0/4268834561 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fca98112640 0x7fca981a2d50 secure :-1 s=READY pgs=1370 cs=0 l=1 rev1=1 crypto rx=0x7fca9000cab0 tx=0x7fca9000cf70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:36:37.030 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:37.030+0000 7fca8fe00640 1 -- 172.21.15.47:0/4268834561 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca9000ae20 con 0x7fca98112640 2024-03-21T10:36:37.030 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:37.030+0000 7fca8fe00640 1 -- 172.21.15.47:0/4268834561 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fca9000f900 con 0x7fca98112640 2024-03-21T10:36:37.030 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:37.030+0000 7fca8fe00640 1 -- 172.21.15.47:0/4268834561 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca900189f0 con 0x7fca98112640 2024-03-21T10:36:37.030 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:37.030+0000 7fca9e000640 1 -- 172.21.15.47:0/4268834561 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fca981a3770 con 0x7fca98112640 2024-03-21T10:36:37.030 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:37.030+0000 7fca9e000640 1 -- 172.21.15.47:0/4268834561 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fca981ab000 con 0x7fca98112640 2024-03-21T10:36:37.030 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:37.031+0000 7fca9e000640 1 -- 172.21.15.47:0/4268834561 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fca980688e0 con 0x7fca98112640 2024-03-21T10:36:37.031 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:37.033+0000 7fca8fe00640 1 -- 172.21.15.47:0/4268834561 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 27) v1 ==== 137985+0+0 (secure 0 0 0) 0x7fca9000fab0 con 0x7fca98112640 2024-03-21T10:36:37.032 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:37.034+0000 7fca8fe00640 1 --2- 172.21.15.47:0/4268834561 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7fca7c0a3110 0x7fca7c0a55d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:36:37.032 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:37.034+0000 7fca8fe00640 1 -- 172.21.15.47:0/4268834561 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(110..110 src has 1..110) v4 ==== 8754+0+0 (secure 0 0 0) 0x7fca900d00a0 con 0x7fca98112640 2024-03-21T10:36:37.032 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:37.035+0000 7fca97400640 1 --2- 172.21.15.47:0/4268834561 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7fca7c0a3110 0x7fca7c0a55d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:36:37.033 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:37.035+0000 7fca97400640 1 --2- 172.21.15.47:0/4268834561 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7fca7c0a3110 0x7fca7c0a55d0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fca880058c0 tx=0x7fca88005d30 comp rx=0 tx=0).ready entity=mgr.15687 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:36:37.037 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:37.039+0000 7fca8fe00640 1 -- 172.21.15.47:0/4268834561 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fca90014030 con 0x7fca98112640 2024-03-21T10:36:37.211 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:36:36 smithi047 ceph-mon[31547]: from='client.? 172.21.15.47:0/3900663626' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-03-21T10:36:37.211 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:36:36 smithi047 ceph-mon[31547]: osdmap e109: 12 total, 12 up, 12 in 2024-03-21T10:36:37.211 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:36:36 smithi047 ceph-mon[31547]: from='client.? 172.21.15.47:0/120085467' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-03-21T10:36:37.211 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:36:36 smithi047 ceph-mon[31547]: pgmap v354: 97 pgs: 97 active+clean; 577 KiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail 2024-03-21T10:36:37.226 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:36:36 smithi100 ceph-mon[35193]: from='client.? 172.21.15.47:0/3900663626' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-03-21T10:36:37.227 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:36:36 smithi100 ceph-mon[35193]: osdmap e109: 12 total, 12 up, 12 in 2024-03-21T10:36:37.227 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:36:36 smithi100 ceph-mon[35193]: from='client.? 172.21.15.47:0/120085467' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-03-21T10:36:37.227 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:36:36 smithi100 ceph-mon[35193]: pgmap v354: 97 pgs: 97 active+clean; 577 KiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail 2024-03-21T10:36:37.236 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:36:36 smithi157 ceph-mon[35581]: from='client.? 172.21.15.47:0/3900663626' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-03-21T10:36:37.236 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:36:36 smithi157 ceph-mon[35581]: osdmap e109: 12 total, 12 up, 12 in 2024-03-21T10:36:37.237 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:36:36 smithi157 ceph-mon[35581]: from='client.? 172.21.15.47:0/120085467' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-03-21T10:36:37.237 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:36:36 smithi157 ceph-mon[35581]: pgmap v354: 97 pgs: 97 active+clean; 577 KiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail 2024-03-21T10:36:37.246 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:37.248+0000 7fca9e000640 1 -- 172.21.15.47:0/4268834561 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fca981ab300 con 0x7fca98112640 2024-03-21T10:36:37.789 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:37.791+0000 7fca8fe00640 1 -- 172.21.15.47:0/4268834561 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data_ec' removed v111) v1 ==== 178+0+0 (secure 0 0 0) 0x7fca9009ca40 con 0x7fca98112640 2024-03-21T10:36:37.789 INFO:teuthology.orchestra.run.smithi047.stderr:pool 'cephfs_data_ec' removed 2024-03-21T10:36:37.793 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:37.795+0000 7fca9e000640 1 -- 172.21.15.47:0/4268834561 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7fca7c0a3110 msgr2=0x7fca7c0a55d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:36:37.793 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:37.795+0000 7fca9e000640 1 --2- 172.21.15.47:0/4268834561 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7fca7c0a3110 0x7fca7c0a55d0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fca880058c0 tx=0x7fca88005d30 comp rx=0 tx=0).stop 2024-03-21T10:36:37.793 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:37.795+0000 7fca9e000640 1 -- 172.21.15.47:0/4268834561 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fca98112640 msgr2=0x7fca981a2d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:36:37.793 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:37.795+0000 7fca9e000640 1 --2- 172.21.15.47:0/4268834561 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fca98112640 0x7fca981a2d50 secure :-1 s=READY pgs=1370 cs=0 l=1 rev1=1 crypto rx=0x7fca9000cab0 tx=0x7fca9000cf70 comp rx=0 tx=0).stop 2024-03-21T10:36:37.793 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:37.795+0000 7fca9e000640 1 -- 172.21.15.47:0/4268834561 shutdown_connections 2024-03-21T10:36:37.793 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:37.795+0000 7fca9e000640 1 --2- 172.21.15.47:0/4268834561 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fca98112640 0x7fca981a2d50 unknown :-1 s=CLOSED pgs=1370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:36:37.793 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:37.795+0000 7fca9e000640 1 --2- 172.21.15.47:0/4268834561 >> [v2:172.21.15.47:6800/196172753,v1:172.21.15.47:6801/196172753] conn(0x7fca7c0a3110 0x7fca7c0a55d0 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:36:37.794 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:37.795+0000 7fca9e000640 1 --2- 172.21.15.47:0/4268834561 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fca98074360 0x7fca9819e9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:36:37.794 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:37.795+0000 7fca9e000640 1 --2- 172.21.15.47:0/4268834561 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fca98073a40 0x7fca9819e490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:36:37.794 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:37.795+0000 7fca9e000640 1 -- 172.21.15.47:0/4268834561 >> 172.21.15.47:0/4268834561 conn(0x7fca98078aa0 msgr2=0x7fca98111620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:36:37.794 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:37.796+0000 7fca9e000640 1 -- 172.21.15.47:0/4268834561 shutdown_connections 2024-03-21T10:36:37.794 INFO:teuthology.orchestra.run.smithi047.stderr:2024-03-21T10:36:37.796+0000 7fca9e000640 1 -- 172.21.15.47:0/4268834561 wait complete. 2024-03-21T10:36:37.828 DEBUG:teuthology.orchestra.run.smithi047:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-03-21T10:36:37.907 DEBUG:teuthology.orchestra.run.smithi100:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-03-21T10:36:38.008 DEBUG:teuthology.orchestra.run.smithi157:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-03-21T10:36:38.080 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:36:37 smithi157 ceph-mon[35581]: from='client.? 172.21.15.47:0/120085467' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2024-03-21T10:36:38.081 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:36:37 smithi157 ceph-mon[35581]: osdmap e110: 12 total, 12 up, 12 in 2024-03-21T10:36:38.081 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:36:37 smithi157 ceph-mon[35581]: from='client.? 172.21.15.47:0/4268834561' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]: dispatch 2024-03-21T10:36:38.098 INFO:tasks.cephadm:Cleaning up testdir ceph.* files... 2024-03-21T10:36:38.098 DEBUG:teuthology.orchestra.run.smithi047:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-03-21T10:36:38.162 DEBUG:teuthology.orchestra.run.smithi100:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-03-21T10:36:38.205 DEBUG:teuthology.orchestra.run.smithi157:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-03-21T10:36:38.210 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:36:37 smithi047 ceph-mon[31547]: from='client.? 172.21.15.47:0/120085467' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2024-03-21T10:36:38.210 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:36:37 smithi047 ceph-mon[31547]: osdmap e110: 12 total, 12 up, 12 in 2024-03-21T10:36:38.210 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:36:37 smithi047 ceph-mon[31547]: from='client.? 172.21.15.47:0/4268834561' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]: dispatch 2024-03-21T10:36:38.226 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:36:37 smithi100 ceph-mon[35193]: from='client.? 172.21.15.47:0/120085467' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2024-03-21T10:36:38.226 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:36:37 smithi100 ceph-mon[35193]: osdmap e110: 12 total, 12 up, 12 in 2024-03-21T10:36:38.226 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:36:37 smithi100 ceph-mon[35193]: from='client.? 172.21.15.47:0/4268834561' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]: dispatch 2024-03-21T10:36:38.250 INFO:tasks.cephadm:Stopping all daemons... 2024-03-21T10:36:38.251 INFO:tasks.cephadm.mon.a:Stopping mon.a... 2024-03-21T10:36:38.251 DEBUG:teuthology.orchestra.run.smithi047:> sudo systemctl stop ceph-3056d51a-e76b-11ee-95cd-87774f69a715@mon.a 2024-03-21T10:36:38.711 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:36:38 smithi047 systemd[1]: Stopping Ceph mon.a for 3056d51a-e76b-11ee-95cd-87774f69a715... 2024-03-21T10:36:39.211 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:36:38 smithi047 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-mon-a[31543]: 2024-03-21T10:36:38.780+0000 7f6087800700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-03-21T10:36:39.211 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:36:38 smithi047 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-mon-a[31543]: 2024-03-21T10:36:38.780+0000 7f6087800700 -1 mon.a@0(leader) e3 *** Got Signal Terminated *** 2024-03-21T10:36:39.212 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:36:38 smithi047 podman[102801]: 2024-03-21 10:36:38.894589927 +0000 UTC m=+0.286189483 container died e43a05e9bf7b17a59e76c9f2a93da51a6c01f99431f509aeb2d6d80f76fc6414 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2edfee68ad7243bed422a049b6f5f58619300f11, name=ceph-3056d51a-e76b-11ee-95cd-87774f69a715-mon-a, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240319, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, CEPH_POINT_RELEASE=, GIT_CLEAN=True) 2024-03-21T10:36:39.668 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:36:39 smithi047 podman[102801]: 2024-03-21 10:36:39.419003838 +0000 UTC m=+0.810603421 container cleanup e43a05e9bf7b17a59e76c9f2a93da51a6c01f99431f509aeb2d6d80f76fc6414 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2edfee68ad7243bed422a049b6f5f58619300f11, name=ceph-3056d51a-e76b-11ee-95cd-87774f69a715-mon-a, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240319, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, ceph=True) 2024-03-21T10:36:39.669 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:36:39 smithi047 bash[102801]: ceph-3056d51a-e76b-11ee-95cd-87774f69a715-mon-a 2024-03-21T10:36:39.960 INFO:journalctl@ceph.mon.a.smithi047.stdout:Mar 21 10:36:39 smithi047 podman[102815]: 2024-03-21 10:36:39.670261596 +0000 UTC m=+0.765714202 container remove e43a05e9bf7b17a59e76c9f2a93da51a6c01f99431f509aeb2d6d80f76fc6414 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2edfee68ad7243bed422a049b6f5f58619300f11, name=ceph-3056d51a-e76b-11ee-95cd-87774f69a715-mon-a, org.label-schema.vendor=CentOS, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240319, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6) 2024-03-21T10:36:40.164 DEBUG:teuthology.orchestra.run.smithi047:> sudo pkill -f 'journalctl -f -n 0 -u ceph-3056d51a-e76b-11ee-95cd-87774f69a715@mon.a.service' 2024-03-21T10:36:40.317 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-21T10:36:40.317 INFO:tasks.cephadm.mon.a:Stopped mon.a 2024-03-21T10:36:40.317 INFO:tasks.cephadm.mon.c:Stopping mon.b... 2024-03-21T10:36:40.317 DEBUG:teuthology.orchestra.run.smithi100:> sudo systemctl stop ceph-3056d51a-e76b-11ee-95cd-87774f69a715@mon.b 2024-03-21T10:36:40.726 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:36:40 smithi100 systemd[1]: Stopping Ceph mon.b for 3056d51a-e76b-11ee-95cd-87774f69a715... 2024-03-21T10:36:41.227 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:36:40 smithi100 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-mon-b[35189]: 2024-03-21T10:36:40.905+0000 7f5c20000700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-03-21T10:36:41.227 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:36:40 smithi100 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-mon-b[35189]: 2024-03-21T10:36:40.905+0000 7f5c20000700 -1 mon.b@2(peon) e3 *** Got Signal Terminated *** 2024-03-21T10:36:41.227 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:36:41 smithi100 podman[70795]: 2024-03-21 10:36:41.008278272 +0000 UTC m=+0.260063005 container died 5aa332c64f454ab905946166832a26da9695527b9633cf3a9379bdabea331d23 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-3056d51a-e76b-11ee-95cd-87774f69a715-mon-b, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.build-date=20240319, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True, GIT_BRANCH=HEAD) 2024-03-21T10:36:41.713 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:36:41 smithi100 podman[70795]: 2024-03-21 10:36:41.421796862 +0000 UTC m=+0.673581612 container cleanup 5aa332c64f454ab905946166832a26da9695527b9633cf3a9379bdabea331d23 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-3056d51a-e76b-11ee-95cd-87774f69a715-mon-b, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_CLEAN=True, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240319) 2024-03-21T10:36:41.713 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:36:41 smithi100 bash[70795]: ceph-3056d51a-e76b-11ee-95cd-87774f69a715-mon-b 2024-03-21T10:36:41.963 DEBUG:teuthology.orchestra.run.smithi100:> sudo pkill -f 'journalctl -f -n 0 -u ceph-3056d51a-e76b-11ee-95cd-87774f69a715@mon.b.service' 2024-03-21T10:36:41.976 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:36:41 smithi100 podman[70809]: 2024-03-21 10:36:41.714957521 +0000 UTC m=+0.684978334 container remove 5aa332c64f454ab905946166832a26da9695527b9633cf3a9379bdabea331d23 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-3056d51a-e76b-11ee-95cd-87774f69a715-mon-b, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_BRANCH=HEAD, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, ceph=True, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240319, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux ) 2024-03-21T10:36:41.976 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:36:41 smithi100 systemd[1]: ceph-3056d51a-e76b-11ee-95cd-87774f69a715@mon.b.service: Deactivated successfully. 2024-03-21T10:36:41.976 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:36:41 smithi100 systemd[1]: Stopped Ceph mon.b for 3056d51a-e76b-11ee-95cd-87774f69a715. 2024-03-21T10:36:41.976 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 10:36:41 smithi100 systemd[1]: ceph-3056d51a-e76b-11ee-95cd-87774f69a715@mon.b.service: Consumed 28.096s CPU time. 2024-03-21T10:36:42.094 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-21T10:36:42.095 INFO:tasks.cephadm.mon.c:Stopped mon.b 2024-03-21T10:36:42.095 INFO:tasks.cephadm.mon.c:Stopping mon.c... 2024-03-21T10:36:42.095 DEBUG:teuthology.orchestra.run.smithi157:> sudo systemctl stop ceph-3056d51a-e76b-11ee-95cd-87774f69a715@mon.c 2024-03-21T10:36:42.486 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:36:42 smithi157 systemd[1]: Stopping Ceph mon.c for 3056d51a-e76b-11ee-95cd-87774f69a715... 2024-03-21T10:36:42.901 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:36:42 smithi157 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-mon-c[35577]: 2024-03-21T10:36:42.675+0000 7f9040800700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-03-21T10:36:42.901 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:36:42 smithi157 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-mon-c[35577]: 2024-03-21T10:36:42.675+0000 7f9040800700 -1 mon.c@1(peon) e3 *** Got Signal Terminated *** 2024-03-21T10:36:43.208 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:36:42 smithi157 podman[72857]: 2024-03-21 10:36:42.902215316 +0000 UTC m=+0.358850527 container died d9e77d928b1edb6585269826c3f06d8b28c3edaf426be89e4bff287c4b2551af (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-3056d51a-e76b-11ee-95cd-87774f69a715-mon-c, maintainer=Guillaume Abrioux , GIT_CLEAN=True, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, ceph=True, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240319) 2024-03-21T10:36:43.487 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:36:43 smithi157 podman[72857]: 2024-03-21 10:36:43.336735503 +0000 UTC m=+0.793370704 container cleanup d9e77d928b1edb6585269826c3f06d8b28c3edaf426be89e4bff287c4b2551af (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-3056d51a-e76b-11ee-95cd-87774f69a715-mon-c, CEPH_POINT_RELEASE=, ceph=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.build-date=20240319) 2024-03-21T10:36:43.487 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:36:43 smithi157 bash[72857]: ceph-3056d51a-e76b-11ee-95cd-87774f69a715-mon-c 2024-03-21T10:36:43.810 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:36:43 smithi157 podman[72871]: 2024-03-21 10:36:43.605002073 +0000 UTC m=+0.682876922 container remove d9e77d928b1edb6585269826c3f06d8b28c3edaf426be89e4bff287c4b2551af (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-3056d51a-e76b-11ee-95cd-87774f69a715-mon-c, ceph=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, GIT_CLEAN=True, org.label-schema.build-date=20240319, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-21T10:36:44.005 DEBUG:teuthology.orchestra.run.smithi157:> sudo pkill -f 'journalctl -f -n 0 -u ceph-3056d51a-e76b-11ee-95cd-87774f69a715@mon.c.service' 2024-03-21T10:36:44.075 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:36:43 smithi157 systemd[1]: ceph-3056d51a-e76b-11ee-95cd-87774f69a715@mon.c.service: Deactivated successfully. 2024-03-21T10:36:44.076 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:36:43 smithi157 systemd[1]: Stopped Ceph mon.c for 3056d51a-e76b-11ee-95cd-87774f69a715. 2024-03-21T10:36:44.076 INFO:journalctl@ceph.mon.c.smithi157.stdout:Mar 21 10:36:43 smithi157 systemd[1]: ceph-3056d51a-e76b-11ee-95cd-87774f69a715@mon.c.service: Consumed 32.151s CPU time. 2024-03-21T10:36:44.119 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-21T10:36:44.120 INFO:tasks.cephadm.mon.c:Stopped mon.c 2024-03-21T10:36:44.120 INFO:tasks.cephadm.mgr.x:Stopping mgr.x... 2024-03-21T10:36:44.120 DEBUG:teuthology.orchestra.run.smithi047:> sudo systemctl stop ceph-3056d51a-e76b-11ee-95cd-87774f69a715@mgr.x 2024-03-21T10:36:44.711 INFO:journalctl@ceph.mgr.x.smithi047.stdout:Mar 21 10:36:44 smithi047 systemd[1]: Stopping Ceph mgr.x for 3056d51a-e76b-11ee-95cd-87774f69a715... 2024-03-21T10:36:45.203 INFO:journalctl@ceph.mgr.x.smithi047.stdout:Mar 21 10:36:44 smithi047 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-mgr-x[31773]: 2024-03-21T10:36:44.789+0000 7ffb14600700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mgr -n mgr.x -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-21T10:36:45.204 INFO:journalctl@ceph.mgr.x.smithi047.stdout:Mar 21 10:36:44 smithi047 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-mgr-x[31773]: 2024-03-21T10:36:44.789+0000 7ffb14600700 -1 mgr handle_mgr_signal *** Got signal Terminated *** 2024-03-21T10:36:45.204 INFO:journalctl@ceph.mgr.x.smithi047.stdout:Mar 21 10:36:44 smithi047 podman[102922]: 2024-03-21 10:36:44.884711537 +0000 UTC m=+0.280621689 container died d2faba9824e03dbc0ebbad478602be6bfd4e2c2dafcee4d67e10d3e703ebaea9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2edfee68ad7243bed422a049b6f5f58619300f11, name=ceph-3056d51a-e76b-11ee-95cd-87774f69a715-mgr-x, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240319, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, ceph=True, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_CLEAN=True, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, org.label-schema.vendor=CentOS) 2024-03-21T10:36:45.961 INFO:journalctl@ceph.mgr.x.smithi047.stdout:Mar 21 10:36:45 smithi047 podman[102922]: 2024-03-21 10:36:45.525137495 +0000 UTC m=+0.921047629 container cleanup d2faba9824e03dbc0ebbad478602be6bfd4e2c2dafcee4d67e10d3e703ebaea9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2edfee68ad7243bed422a049b6f5f58619300f11, name=ceph-3056d51a-e76b-11ee-95cd-87774f69a715-mgr-x, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.build-date=20240319, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30) 2024-03-21T10:36:45.961 INFO:journalctl@ceph.mgr.x.smithi047.stdout:Mar 21 10:36:45 smithi047 bash[102922]: ceph-3056d51a-e76b-11ee-95cd-87774f69a715-mgr-x 2024-03-21T10:36:45.961 INFO:journalctl@ceph.mgr.x.smithi047.stdout:Mar 21 10:36:45 smithi047 podman[102935]: 2024-03-21 10:36:45.751567079 +0000 UTC m=+0.851535571 container remove d2faba9824e03dbc0ebbad478602be6bfd4e2c2dafcee4d67e10d3e703ebaea9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2edfee68ad7243bed422a049b6f5f58619300f11, name=ceph-3056d51a-e76b-11ee-95cd-87774f69a715-mgr-x, org.label-schema.build-date=20240319, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_CLEAN=True, maintainer=Guillaume Abrioux , RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, org.label-schema.schema-version=1.0) 2024-03-21T10:36:46.264 DEBUG:teuthology.orchestra.run.smithi047:> sudo pkill -f 'journalctl -f -n 0 -u ceph-3056d51a-e76b-11ee-95cd-87774f69a715@mgr.x.service' 2024-03-21T10:36:46.386 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-21T10:36:46.387 INFO:tasks.cephadm.mgr.x:Stopped mgr.x 2024-03-21T10:36:46.387 INFO:tasks.cephadm.mgr.y:Stopping mgr.y... 2024-03-21T10:36:46.387 DEBUG:teuthology.orchestra.run.smithi100:> sudo systemctl stop ceph-3056d51a-e76b-11ee-95cd-87774f69a715@mgr.y 2024-03-21T10:36:46.728 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Mar 21 10:36:46 smithi100 systemd[1]: Stopping Ceph mgr.y for 3056d51a-e76b-11ee-95cd-87774f69a715... 2024-03-21T10:36:47.440 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Mar 21 10:36:47 smithi100 podman[70928]: 2024-03-21 10:36:47.091796576 +0000 UTC m=+0.232898197 container died f904fc9002ac42b49e0c43d8dea04a7a8bede22cdde5ec4ded2c9c1e81800c5d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-3056d51a-e76b-11ee-95cd-87774f69a715-mgr-y, org.label-schema.build-date=20240319, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_CLEAN=True, ceph=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-21T10:36:47.726 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Mar 21 10:36:47 smithi100 podman[70928]: 2024-03-21 10:36:47.61877745 +0000 UTC m=+0.759879132 container cleanup f904fc9002ac42b49e0c43d8dea04a7a8bede22cdde5ec4ded2c9c1e81800c5d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-3056d51a-e76b-11ee-95cd-87774f69a715-mgr-y, org.label-schema.build-date=20240319, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6) 2024-03-21T10:36:47.727 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Mar 21 10:36:47 smithi100 bash[70928]: ceph-3056d51a-e76b-11ee-95cd-87774f69a715-mgr-y 2024-03-21T10:36:48.227 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Mar 21 10:36:47 smithi100 podman[70942]: 2024-03-21 10:36:47.861853171 +0000 UTC m=+0.756871632 container remove f904fc9002ac42b49e0c43d8dea04a7a8bede22cdde5ec4ded2c9c1e81800c5d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-3056d51a-e76b-11ee-95cd-87774f69a715-mgr-y, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, org.label-schema.build-date=20240319, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, io.buildah.version=1.33.5) 2024-03-21T10:36:48.227 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Mar 21 10:36:47 smithi100 systemd[1]: ceph-3056d51a-e76b-11ee-95cd-87774f69a715@mgr.y.service: Main process exited, code=exited, status=143/n/a 2024-03-21T10:36:48.341 DEBUG:teuthology.orchestra.run.smithi100:> sudo pkill -f 'journalctl -f -n 0 -u ceph-3056d51a-e76b-11ee-95cd-87774f69a715@mgr.y.service' 2024-03-21T10:36:48.456 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-21T10:36:48.456 INFO:tasks.cephadm.mgr.y:Stopped mgr.y 2024-03-21T10:36:48.456 INFO:tasks.cephadm.mgr.z:Stopping mgr.z... 2024-03-21T10:36:48.456 DEBUG:teuthology.orchestra.run.smithi157:> sudo systemctl stop ceph-3056d51a-e76b-11ee-95cd-87774f69a715@mgr.z 2024-03-21T10:36:48.987 INFO:journalctl@ceph.mgr.z.smithi157.stdout:Mar 21 10:36:48 smithi157 systemd[1]: Stopping Ceph mgr.z for 3056d51a-e76b-11ee-95cd-87774f69a715... 2024-03-21T10:36:49.397 INFO:journalctl@ceph.mgr.z.smithi157.stdout:Mar 21 10:36:49 smithi157 podman[72989]: 2024-03-21 10:36:49.094842895 +0000 UTC m=+0.214303579 container died 7392304e277dddc346cec26193ccb280eea73c374221ec762f6f5d9fbe395f53 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-3056d51a-e76b-11ee-95cd-87774f69a715-mgr-z, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.license=GPLv2, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, io.buildah.version=1.33.5, ceph=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.build-date=20240319, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0) 2024-03-21T10:36:49.737 INFO:journalctl@ceph.mgr.z.smithi157.stdout:Mar 21 10:36:49 smithi157 podman[72989]: 2024-03-21 10:36:49.584186311 +0000 UTC m=+0.703646971 container cleanup 7392304e277dddc346cec26193ccb280eea73c374221ec762f6f5d9fbe395f53 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-3056d51a-e76b-11ee-95cd-87774f69a715-mgr-z, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, ceph=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.build-date=20240319, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0) 2024-03-21T10:36:49.737 INFO:journalctl@ceph.mgr.z.smithi157.stdout:Mar 21 10:36:49 smithi157 bash[72989]: ceph-3056d51a-e76b-11ee-95cd-87774f69a715-mgr-z 2024-03-21T10:36:50.237 INFO:journalctl@ceph.mgr.z.smithi157.stdout:Mar 21 10:36:49 smithi157 podman[73003]: 2024-03-21 10:36:49.833814219 +0000 UTC m=+0.726644344 container remove 7392304e277dddc346cec26193ccb280eea73c374221ec762f6f5d9fbe395f53 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-3056d51a-e76b-11ee-95cd-87774f69a715-mgr-z, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, ceph=True, maintainer=Guillaume Abrioux , RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.build-date=20240319, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5) 2024-03-21T10:36:50.237 INFO:journalctl@ceph.mgr.z.smithi157.stdout:Mar 21 10:36:49 smithi157 systemd[1]: ceph-3056d51a-e76b-11ee-95cd-87774f69a715@mgr.z.service: Main process exited, code=exited, status=143/n/a 2024-03-21T10:36:50.359 DEBUG:teuthology.orchestra.run.smithi157:> sudo pkill -f 'journalctl -f -n 0 -u ceph-3056d51a-e76b-11ee-95cd-87774f69a715@mgr.z.service' 2024-03-21T10:36:50.481 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-21T10:36:50.481 INFO:tasks.cephadm.mgr.z:Stopped mgr.z 2024-03-21T10:36:50.481 INFO:tasks.cephadm.osd.0:Stopping osd.0... 2024-03-21T10:36:50.481 DEBUG:teuthology.orchestra.run.smithi047:> sudo systemctl stop ceph-3056d51a-e76b-11ee-95cd-87774f69a715@osd.0 2024-03-21T10:36:50.960 INFO:journalctl@ceph.osd.0.smithi047.stdout:Mar 21 10:36:50 smithi047 systemd[1]: Stopping Ceph osd.0 for 3056d51a-e76b-11ee-95cd-87774f69a715... 2024-03-21T10:36:51.461 INFO:journalctl@ceph.osd.0.smithi047.stdout:Mar 21 10:36:51 smithi047 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-0[44057]: 2024-03-21T10:36:51.144+0000 7f04da800700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.0 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-21T10:36:51.461 INFO:journalctl@ceph.osd.0.smithi047.stdout:Mar 21 10:36:51 smithi047 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-0[44057]: 2024-03-21T10:36:51.144+0000 7f04da800700 -1 osd.0 111 *** Got signal Terminated *** 2024-03-21T10:36:51.461 INFO:journalctl@ceph.osd.0.smithi047.stdout:Mar 21 10:36:51 smithi047 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-0[44057]: 2024-03-21T10:36:51.144+0000 7f04da800700 -1 osd.0 111 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-21T10:36:56.961 INFO:journalctl@ceph.osd.0.smithi047.stdout:Mar 21 10:36:56 smithi047 podman[103042]: 2024-03-21 10:36:56.679538 +0000 UTC m=+5.710534950 container died dff9fdcb42dc8f270601ca1553ba569aa12f5626b1198d640e0ea029fbd7fed5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-0, ceph=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, org.label-schema.build-date=20240319, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_CLEAN=True, CEPH_POINT_RELEASE=) 2024-03-21T10:36:57.346 INFO:journalctl@ceph.osd.0.smithi047.stdout:Mar 21 10:36:57 smithi047 podman[103042]: 2024-03-21 10:36:57.146905515 +0000 UTC m=+6.177902509 container cleanup dff9fdcb42dc8f270601ca1553ba569aa12f5626b1198d640e0ea029fbd7fed5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-0, org.label-schema.build-date=20240319, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True) 2024-03-21T10:36:57.346 INFO:journalctl@ceph.osd.0.smithi047.stdout:Mar 21 10:36:57 smithi047 bash[103042]: ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-0 2024-03-21T10:36:57.710 INFO:journalctl@ceph.osd.0.smithi047.stdout:Mar 21 10:36:57 smithi047 podman[103072]: 2024-03-21 10:36:57.34835068 +0000 UTC m=+0.649990276 container remove dff9fdcb42dc8f270601ca1553ba569aa12f5626b1198d640e0ea029fbd7fed5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.build-date=20240319, GIT_CLEAN=True, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-21T10:36:58.461 INFO:journalctl@ceph.osd.0.smithi047.stdout:Mar 21 10:36:58 smithi047 podman[103159]: 2024-03-21 10:36:58.080191734 +0000 UTC m=+0.034283029 image pull 237e43c7a65ff48534d94b97c1bfdbe4b7be5672e3d9a613745083ea3702aa54 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232 2024-03-21T10:36:58.461 INFO:journalctl@ceph.osd.0.smithi047.stdout:Mar 21 10:36:58 smithi047 podman[103159]: 2024-03-21 10:36:58.226924661 +0000 UTC m=+0.181015994 container create 125b6c530fe8860a8b1b473ee8b79c83d0a0dbee378a2adcf4e3dd34ed33e982 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-0-deactivate, ceph=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240319, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, GIT_CLEAN=True) 2024-03-21T10:36:58.961 INFO:journalctl@ceph.osd.0.smithi047.stdout:Mar 21 10:36:58 smithi047 podman[103159]: 2024-03-21 10:36:58.545112727 +0000 UTC m=+0.499204067 container init 125b6c530fe8860a8b1b473ee8b79c83d0a0dbee378a2adcf4e3dd34ed33e982 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-0-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.build-date=20240319, GIT_CLEAN=True, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , ceph=True, GIT_BRANCH=HEAD) 2024-03-21T10:36:58.961 INFO:journalctl@ceph.osd.0.smithi047.stdout:Mar 21 10:36:58 smithi047 podman[103159]: 2024-03-21 10:36:58.557217221 +0000 UTC m=+0.511308524 container start 125b6c530fe8860a8b1b473ee8b79c83d0a0dbee378a2adcf4e3dd34ed33e982 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-0-deactivate, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240319, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_CLEAN=True, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.vendor=CentOS, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6) 2024-03-21T10:36:58.961 INFO:journalctl@ceph.osd.0.smithi047.stdout:Mar 21 10:36:58 smithi047 podman[103159]: 2024-03-21 10:36:58.643908273 +0000 UTC m=+0.597999662 container attach 125b6c530fe8860a8b1b473ee8b79c83d0a0dbee378a2adcf4e3dd34ed33e982 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-0-deactivate, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240319, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30) 2024-03-21T10:36:59.391 INFO:journalctl@ceph.osd.0.smithi047.stdout:Mar 21 10:36:59 smithi047 podman[103159]: 2024-03-21 10:36:59.016954608 +0000 UTC m=+0.971045947 container died 125b6c530fe8860a8b1b473ee8b79c83d0a0dbee378a2adcf4e3dd34ed33e982 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-0-deactivate, ceph=True, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.build-date=20240319, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6) 2024-03-21T10:36:59.867 DEBUG:teuthology.orchestra.run.smithi047:> sudo pkill -f 'journalctl -f -n 0 -u ceph-3056d51a-e76b-11ee-95cd-87774f69a715@osd.0.service' 2024-03-21T10:37:00.001 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-21T10:37:00.001 INFO:tasks.cephadm.osd.0:Stopped osd.0 2024-03-21T10:37:00.002 INFO:tasks.cephadm.osd.1:Stopping osd.1... 2024-03-21T10:37:00.002 DEBUG:teuthology.orchestra.run.smithi100:> sudo systemctl stop ceph-3056d51a-e76b-11ee-95cd-87774f69a715@osd.1 2024-03-21T10:37:00.477 INFO:journalctl@ceph.osd.1.smithi100.stdout:Mar 21 10:37:00 smithi100 systemd[1]: Stopping Ceph osd.1 for 3056d51a-e76b-11ee-95cd-87774f69a715... 2024-03-21T10:37:00.976 INFO:journalctl@ceph.osd.1.smithi100.stdout:Mar 21 10:37:00 smithi100 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-1[39797]: 2024-03-21T10:37:00.588+0000 7fa35fa00700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.1 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-21T10:37:00.977 INFO:journalctl@ceph.osd.1.smithi100.stdout:Mar 21 10:37:00 smithi100 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-1[39797]: 2024-03-21T10:37:00.588+0000 7fa35fa00700 -1 osd.1 111 *** Got signal Terminated *** 2024-03-21T10:37:00.977 INFO:journalctl@ceph.osd.1.smithi100.stdout:Mar 21 10:37:00 smithi100 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-1[39797]: 2024-03-21T10:37:00.588+0000 7fa35fa00700 -1 osd.1 111 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-21T10:37:06.579 INFO:journalctl@ceph.osd.1.smithi100.stdout:Mar 21 10:37:06 smithi100 podman[71064]: 2024-03-21 10:37:06.310608722 +0000 UTC m=+5.881807191 container died e02373ec2d739859e2cc27fe3ec135e95b428ce9e3883ab6cab4a962a20f9e32 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-1, ceph=True, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240319, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=) 2024-03-21T10:37:06.976 INFO:journalctl@ceph.osd.1.smithi100.stdout:Mar 21 10:37:06 smithi100 podman[71064]: 2024-03-21 10:37:06.789819306 +0000 UTC m=+6.361017784 container cleanup e02373ec2d739859e2cc27fe3ec135e95b428ce9e3883ab6cab4a962a20f9e32 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-1, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, ceph=True, GIT_CLEAN=True, org.label-schema.build-date=20240319, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, CEPH_POINT_RELEASE=, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-21T10:37:06.977 INFO:journalctl@ceph.osd.1.smithi100.stdout:Mar 21 10:37:06 smithi100 bash[71064]: ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-1 2024-03-21T10:37:07.476 INFO:journalctl@ceph.osd.1.smithi100.stdout:Mar 21 10:37:07 smithi100 podman[71082]: 2024-03-21 10:37:07.132752702 +0000 UTC m=+0.803398760 container remove e02373ec2d739859e2cc27fe3ec135e95b428ce9e3883ab6cab4a962a20f9e32 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-1, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.build-date=20240319, ceph=True, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6) 2024-03-21T10:37:07.976 INFO:journalctl@ceph.osd.1.smithi100.stdout:Mar 21 10:37:07 smithi100 podman[71172]: 2024-03-21 10:37:07.591319997 +0000 UTC m=+0.035937996 image pull 237e43c7a65ff48534d94b97c1bfdbe4b7be5672e3d9a613745083ea3702aa54 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232 2024-03-21T10:37:07.976 INFO:journalctl@ceph.osd.1.smithi100.stdout:Mar 21 10:37:07 smithi100 podman[71172]: 2024-03-21 10:37:07.716894037 +0000 UTC m=+0.161512054 container create e501187d0626b674d331e32d796aeaaa9db48f598db4d14796b1b75aa5f61ea3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-1-deactivate, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_BRANCH=HEAD, org.label-schema.build-date=20240319, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, ceph=True, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=) 2024-03-21T10:37:08.477 INFO:journalctl@ceph.osd.1.smithi100.stdout:Mar 21 10:37:08 smithi100 podman[71172]: 2024-03-21 10:37:08.039391843 +0000 UTC m=+0.484009904 container init e501187d0626b674d331e32d796aeaaa9db48f598db4d14796b1b75aa5f61ea3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-1-deactivate, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, ceph=True, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, org.label-schema.build-date=20240319, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_BRANCH=HEAD) 2024-03-21T10:37:08.477 INFO:journalctl@ceph.osd.1.smithi100.stdout:Mar 21 10:37:08 smithi100 podman[71172]: 2024-03-21 10:37:08.050951869 +0000 UTC m=+0.495569856 container start e501187d0626b674d331e32d796aeaaa9db48f598db4d14796b1b75aa5f61ea3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-1-deactivate, org.label-schema.build-date=20240319, org.label-schema.license=GPLv2, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, maintainer=Guillaume Abrioux , ceph=True, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_CLEAN=True, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0) 2024-03-21T10:37:08.477 INFO:journalctl@ceph.osd.1.smithi100.stdout:Mar 21 10:37:08 smithi100 podman[71172]: 2024-03-21 10:37:08.145026326 +0000 UTC m=+0.589644376 container attach e501187d0626b674d331e32d796aeaaa9db48f598db4d14796b1b75aa5f61ea3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-1-deactivate, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, ceph=True, org.label-schema.build-date=20240319, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-21T10:37:08.843 INFO:journalctl@ceph.osd.1.smithi100.stdout:Mar 21 10:37:08 smithi100 podman[71172]: 2024-03-21 10:37:08.481098793 +0000 UTC m=+0.925716787 container died e501187d0626b674d331e32d796aeaaa9db48f598db4d14796b1b75aa5f61ea3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-1-deactivate, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.build-date=20240319, CEPH_POINT_RELEASE=, ceph=True, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, org.label-schema.vendor=CentOS) 2024-03-21T10:37:09.255 DEBUG:teuthology.orchestra.run.smithi100:> sudo pkill -f 'journalctl -f -n 0 -u ceph-3056d51a-e76b-11ee-95cd-87774f69a715@osd.1.service' 2024-03-21T10:37:09.369 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-21T10:37:09.369 INFO:tasks.cephadm.osd.1:Stopped osd.1 2024-03-21T10:37:09.369 INFO:tasks.cephadm.osd.2:Stopping osd.2... 2024-03-21T10:37:09.369 DEBUG:teuthology.orchestra.run.smithi157:> sudo systemctl stop ceph-3056d51a-e76b-11ee-95cd-87774f69a715@osd.2 2024-03-21T10:37:09.736 INFO:journalctl@ceph.osd.2.smithi157.stdout:Mar 21 10:37:09 smithi157 systemd[1]: Stopping Ceph osd.2 for 3056d51a-e76b-11ee-95cd-87774f69a715... 2024-03-21T10:37:10.237 INFO:journalctl@ceph.osd.2.smithi157.stdout:Mar 21 10:37:09 smithi157 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-2[40607]: 2024-03-21T10:37:09.970+0000 7f4f78200700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.2 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-21T10:37:10.237 INFO:journalctl@ceph.osd.2.smithi157.stdout:Mar 21 10:37:09 smithi157 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-2[40607]: 2024-03-21T10:37:09.970+0000 7f4f78200700 -1 osd.2 111 *** Got signal Terminated *** 2024-03-21T10:37:10.237 INFO:journalctl@ceph.osd.2.smithi157.stdout:Mar 21 10:37:09 smithi157 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-2[40607]: 2024-03-21T10:37:09.970+0000 7f4f78200700 -1 osd.2 111 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-21T10:37:15.852 INFO:journalctl@ceph.osd.2.smithi157.stdout:Mar 21 10:37:15 smithi157 podman[73112]: 2024-03-21 10:37:15.575334459 +0000 UTC m=+5.761581098 container died 7205552e07a67f3ca4dfb4030dc364fafb243cd44c6d31ca68e8d15186d16f52 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-2, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_CLEAN=True, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, org.label-schema.build-date=20240319, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-21T10:37:16.192 INFO:journalctl@ceph.osd.2.smithi157.stdout:Mar 21 10:37:15 smithi157 podman[73112]: 2024-03-21 10:37:15.989634176 +0000 UTC m=+6.175880809 container cleanup 7205552e07a67f3ca4dfb4030dc364fafb243cd44c6d31ca68e8d15186d16f52 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, ceph=True, org.label-schema.build-date=20240319, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5) 2024-03-21T10:37:16.192 INFO:journalctl@ceph.osd.2.smithi157.stdout:Mar 21 10:37:15 smithi157 bash[73112]: ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-2 2024-03-21T10:37:16.486 INFO:journalctl@ceph.osd.2.smithi157.stdout:Mar 21 10:37:16 smithi157 podman[73130]: 2024-03-21 10:37:16.266086473 +0000 UTC m=+0.675903386 container remove 7205552e07a67f3ca4dfb4030dc364fafb243cd44c6d31ca68e8d15186d16f52 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-2, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.schema-version=1.0, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240319, GIT_BRANCH=HEAD, ceph=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS) 2024-03-21T10:37:17.078 INFO:journalctl@ceph.osd.2.smithi157.stdout:Mar 21 10:37:16 smithi157 podman[73226]: 2024-03-21 10:37:16.697075815 +0000 UTC m=+0.034334809 image pull 237e43c7a65ff48534d94b97c1bfdbe4b7be5672e3d9a613745083ea3702aa54 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232 2024-03-21T10:37:17.078 INFO:journalctl@ceph.osd.2.smithi157.stdout:Mar 21 10:37:16 smithi157 podman[73226]: 2024-03-21 10:37:16.816821099 +0000 UTC m=+0.154080097 container create 572b101940d4d66e441dd35828b2074748501aa9db29742810cb18107786d43a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-2-deactivate, GIT_CLEAN=True, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240319, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, io.buildah.version=1.33.5) 2024-03-21T10:37:17.431 INFO:journalctl@ceph.osd.2.smithi157.stdout:Mar 21 10:37:17 smithi157 podman[73226]: 2024-03-21 10:37:17.079768045 +0000 UTC m=+0.417027083 container init 572b101940d4d66e441dd35828b2074748501aa9db29742810cb18107786d43a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-2-deactivate, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, ceph=True, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.build-date=20240319, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30) 2024-03-21T10:37:17.431 INFO:journalctl@ceph.osd.2.smithi157.stdout:Mar 21 10:37:17 smithi157 podman[73226]: 2024-03-21 10:37:17.090432475 +0000 UTC m=+0.427691478 container start 572b101940d4d66e441dd35828b2074748501aa9db29742810cb18107786d43a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-2-deactivate, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240319, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30) 2024-03-21T10:37:17.431 INFO:journalctl@ceph.osd.2.smithi157.stdout:Mar 21 10:37:17 smithi157 podman[73226]: 2024-03-21 10:37:17.202967677 +0000 UTC m=+0.540226679 container attach 572b101940d4d66e441dd35828b2074748501aa9db29742810cb18107786d43a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-2-deactivate, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, ceph=True, maintainer=Guillaume Abrioux , GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.build-date=20240319) 2024-03-21T10:37:17.476 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 10:37:17 smithi100 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-10[59767]: 2024-03-21T10:37:17.127+0000 7f315da00700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-03-21T10:36:52.673512+0000 front 2024-03-21T10:36:52.673688+0000 (oldest deadline 2024-03-21T10:37:16.675039+0000) 2024-03-21T10:37:17.736 INFO:journalctl@ceph.osd.2.smithi157.stdout:Mar 21 10:37:17 smithi157 podman[73226]: 2024-03-21 10:37:17.508085154 +0000 UTC m=+0.845344192 container died 572b101940d4d66e441dd35828b2074748501aa9db29742810cb18107786d43a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-2-deactivate, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240319, ceph=True, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6) 2024-03-21T10:37:17.737 INFO:journalctl@ceph.osd.11.smithi157.stdout:Mar 21 10:37:17 smithi157 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-11[61259]: 2024-03-21T10:37:17.585+0000 7fd0f8400700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-03-21T10:36:54.398353+0000 front 2024-03-21T10:36:54.398068+0000 (oldest deadline 2024-03-21T10:37:16.698280+0000) 2024-03-21T10:37:18.259 DEBUG:teuthology.orchestra.run.smithi157:> sudo pkill -f 'journalctl -f -n 0 -u ceph-3056d51a-e76b-11ee-95cd-87774f69a715@osd.2.service' 2024-03-21T10:37:18.386 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-21T10:37:18.386 INFO:tasks.cephadm.osd.2:Stopped osd.2 2024-03-21T10:37:18.386 INFO:tasks.cephadm.osd.3:Stopping osd.3... 2024-03-21T10:37:18.386 DEBUG:teuthology.orchestra.run.smithi047:> sudo systemctl stop ceph-3056d51a-e76b-11ee-95cd-87774f69a715@osd.3 2024-03-21T10:37:18.476 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 10:37:18 smithi100 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-10[59767]: 2024-03-21T10:37:18.150+0000 7f315da00700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-03-21T10:36:52.673512+0000 front 2024-03-21T10:36:52.673688+0000 (oldest deadline 2024-03-21T10:37:16.675039+0000) 2024-03-21T10:37:18.961 INFO:journalctl@ceph.osd.3.smithi047.stdout:Mar 21 10:37:18 smithi047 systemd[1]: Stopping Ceph osd.3 for 3056d51a-e76b-11ee-95cd-87774f69a715... 2024-03-21T10:37:18.961 INFO:journalctl@ceph.osd.9.smithi047.stdout:Mar 21 10:37:18 smithi047 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-9[64137]: 2024-03-21T10:37:18.477+0000 7f02dc200700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-03-21T10:36:54.991613+0000 front 2024-03-21T10:36:54.991458+0000 (oldest deadline 2024-03-21T10:37:17.891147+0000) 2024-03-21T10:37:18.986 INFO:journalctl@ceph.osd.11.smithi157.stdout:Mar 21 10:37:18 smithi157 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-11[61259]: 2024-03-21T10:37:18.585+0000 7fd0f8400700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-03-21T10:36:54.398353+0000 front 2024-03-21T10:36:54.398068+0000 (oldest deadline 2024-03-21T10:37:16.698280+0000) 2024-03-21T10:37:19.461 INFO:journalctl@ceph.osd.3.smithi047.stdout:Mar 21 10:37:19 smithi047 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-3[50720]: 2024-03-21T10:37:19.016+0000 7f7843200700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.3 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-21T10:37:19.461 INFO:journalctl@ceph.osd.3.smithi047.stdout:Mar 21 10:37:19 smithi047 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-3[50720]: 2024-03-21T10:37:19.016+0000 7f7843200700 -1 osd.3 111 *** Got signal Terminated *** 2024-03-21T10:37:19.461 INFO:journalctl@ceph.osd.3.smithi047.stdout:Mar 21 10:37:19 smithi047 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-3[50720]: 2024-03-21T10:37:19.016+0000 7f7843200700 -1 osd.3 111 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-21T10:37:19.476 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 10:37:19 smithi100 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-10[59767]: 2024-03-21T10:37:19.195+0000 7f315da00700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-03-21T10:36:52.673512+0000 front 2024-03-21T10:36:52.673688+0000 (oldest deadline 2024-03-21T10:37:16.675039+0000) 2024-03-21T10:37:19.960 INFO:journalctl@ceph.osd.3.smithi047.stdout:Mar 21 10:37:19 smithi047 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-3[50720]: 2024-03-21T10:37:19.727+0000 7f7839a00700 -1 osd.3 111 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-03-21T10:36:55.928288+0000 front 2024-03-21T10:36:55.929070+0000 (oldest deadline 2024-03-21T10:37:18.827820+0000) 2024-03-21T10:37:19.961 INFO:journalctl@ceph.osd.9.smithi047.stdout:Mar 21 10:37:19 smithi047 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-9[64137]: 2024-03-21T10:37:19.509+0000 7f02dc200700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-03-21T10:36:54.991613+0000 front 2024-03-21T10:36:54.991458+0000 (oldest deadline 2024-03-21T10:37:17.891147+0000) 2024-03-21T10:37:19.986 INFO:journalctl@ceph.osd.11.smithi157.stdout:Mar 21 10:37:19 smithi157 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-11[61259]: 2024-03-21T10:37:19.620+0000 7fd0f8400700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-03-21T10:36:54.398353+0000 front 2024-03-21T10:36:54.398068+0000 (oldest deadline 2024-03-21T10:37:16.698280+0000) 2024-03-21T10:37:20.476 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 10:37:20 smithi100 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-10[59767]: 2024-03-21T10:37:20.207+0000 7f315da00700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-03-21T10:36:52.673512+0000 front 2024-03-21T10:36:52.673688+0000 (oldest deadline 2024-03-21T10:37:16.675039+0000) 2024-03-21T10:37:20.652 INFO:journalctl@ceph.osd.5.smithi157.stdout:Mar 21 10:37:20 smithi157 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-5[47667]: 2024-03-21T10:37:20.346+0000 7f242fa00700 -1 osd.5 111 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-03-21T10:36:55.338628+0000 front 2024-03-21T10:36:55.339174+0000 (oldest deadline 2024-03-21T10:37:19.438029+0000) 2024-03-21T10:37:20.960 INFO:journalctl@ceph.osd.9.smithi047.stdout:Mar 21 10:37:20 smithi047 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-9[64137]: 2024-03-21T10:37:20.521+0000 7f02dc200700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-03-21T10:36:54.991613+0000 front 2024-03-21T10:36:54.991458+0000 (oldest deadline 2024-03-21T10:37:17.891147+0000) 2024-03-21T10:37:20.961 INFO:journalctl@ceph.osd.3.smithi047.stdout:Mar 21 10:37:20 smithi047 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-3[50720]: 2024-03-21T10:37:20.736+0000 7f7839a00700 -1 osd.3 111 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-03-21T10:36:55.928288+0000 front 2024-03-21T10:36:55.929070+0000 (oldest deadline 2024-03-21T10:37:18.827820+0000) 2024-03-21T10:37:20.986 INFO:journalctl@ceph.osd.11.smithi157.stdout:Mar 21 10:37:20 smithi157 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-11[61259]: 2024-03-21T10:37:20.654+0000 7fd0f8400700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-03-21T10:36:54.398353+0000 front 2024-03-21T10:36:54.398068+0000 (oldest deadline 2024-03-21T10:37:16.698280+0000) 2024-03-21T10:37:21.641 INFO:journalctl@ceph.osd.5.smithi157.stdout:Mar 21 10:37:21 smithi157 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-5[47667]: 2024-03-21T10:37:21.379+0000 7f242fa00700 -1 osd.5 111 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-03-21T10:36:55.338628+0000 front 2024-03-21T10:36:55.339174+0000 (oldest deadline 2024-03-21T10:37:19.438029+0000) 2024-03-21T10:37:21.726 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 10:37:21 smithi100 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-10[59767]: 2024-03-21T10:37:21.229+0000 7f315da00700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-03-21T10:36:52.673512+0000 front 2024-03-21T10:36:52.673688+0000 (oldest deadline 2024-03-21T10:37:16.675039+0000) 2024-03-21T10:37:21.781 INFO:journalctl@ceph.osd.9.smithi047.stdout:Mar 21 10:37:21 smithi047 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-9[64137]: 2024-03-21T10:37:21.505+0000 7f02dc200700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-03-21T10:36:54.991613+0000 front 2024-03-21T10:36:54.991458+0000 (oldest deadline 2024-03-21T10:37:17.891147+0000) 2024-03-21T10:37:21.986 INFO:journalctl@ceph.osd.11.smithi157.stdout:Mar 21 10:37:21 smithi157 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-11[61259]: 2024-03-21T10:37:21.643+0000 7fd0f8400700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-03-21T10:36:54.398353+0000 front 2024-03-21T10:36:54.398068+0000 (oldest deadline 2024-03-21T10:37:16.698280+0000) 2024-03-21T10:37:22.210 INFO:journalctl@ceph.osd.3.smithi047.stdout:Mar 21 10:37:21 smithi047 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-3[50720]: 2024-03-21T10:37:21.783+0000 7f7839a00700 -1 osd.3 111 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-03-21T10:36:55.928288+0000 front 2024-03-21T10:36:55.929070+0000 (oldest deadline 2024-03-21T10:37:18.827820+0000) 2024-03-21T10:37:22.476 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 10:37:22 smithi100 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-10[59767]: 2024-03-21T10:37:22.197+0000 7f315da00700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-03-21T10:36:52.673512+0000 front 2024-03-21T10:36:52.673688+0000 (oldest deadline 2024-03-21T10:37:16.675039+0000) 2024-03-21T10:37:22.690 INFO:journalctl@ceph.osd.5.smithi157.stdout:Mar 21 10:37:22 smithi157 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-5[47667]: 2024-03-21T10:37:22.366+0000 7f242fa00700 -1 osd.5 111 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-03-21T10:36:55.338628+0000 front 2024-03-21T10:36:55.339174+0000 (oldest deadline 2024-03-21T10:37:19.438029+0000) 2024-03-21T10:37:22.827 INFO:journalctl@ceph.osd.9.smithi047.stdout:Mar 21 10:37:22 smithi047 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-9[64137]: 2024-03-21T10:37:22.494+0000 7f02dc200700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-03-21T10:36:54.991613+0000 front 2024-03-21T10:36:54.991458+0000 (oldest deadline 2024-03-21T10:37:17.891147+0000) 2024-03-21T10:37:22.986 INFO:journalctl@ceph.osd.11.smithi157.stdout:Mar 21 10:37:22 smithi157 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-11[61259]: 2024-03-21T10:37:22.692+0000 7fd0f8400700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-03-21T10:36:54.398353+0000 front 2024-03-21T10:36:54.398068+0000 (oldest deadline 2024-03-21T10:37:16.698280+0000) 2024-03-21T10:37:23.210 INFO:journalctl@ceph.osd.3.smithi047.stdout:Mar 21 10:37:22 smithi047 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-3[50720]: 2024-03-21T10:37:22.829+0000 7f7839a00700 -1 osd.3 111 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-03-21T10:36:55.928288+0000 front 2024-03-21T10:36:55.929070+0000 (oldest deadline 2024-03-21T10:37:18.827820+0000) 2024-03-21T10:37:23.476 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 10:37:23 smithi100 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-10[59767]: 2024-03-21T10:37:23.178+0000 7f315da00700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-03-21T10:36:52.673512+0000 front 2024-03-21T10:36:52.673688+0000 (oldest deadline 2024-03-21T10:37:16.675039+0000) 2024-03-21T10:37:23.694 INFO:journalctl@ceph.osd.5.smithi157.stdout:Mar 21 10:37:23 smithi157 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-5[47667]: 2024-03-21T10:37:23.337+0000 7f242fa00700 -1 osd.5 111 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-03-21T10:36:55.338628+0000 front 2024-03-21T10:36:55.339174+0000 (oldest deadline 2024-03-21T10:37:19.438029+0000) 2024-03-21T10:37:23.829 INFO:journalctl@ceph.osd.9.smithi047.stdout:Mar 21 10:37:23 smithi047 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-9[64137]: 2024-03-21T10:37:23.504+0000 7f02dc200700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-03-21T10:36:54.991613+0000 front 2024-03-21T10:36:54.991458+0000 (oldest deadline 2024-03-21T10:37:17.891147+0000) 2024-03-21T10:37:23.986 INFO:journalctl@ceph.osd.11.smithi157.stdout:Mar 21 10:37:23 smithi157 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-11[61259]: 2024-03-21T10:37:23.696+0000 7fd0f8400700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-03-21T10:36:54.398353+0000 front 2024-03-21T10:36:54.398068+0000 (oldest deadline 2024-03-21T10:37:16.698280+0000) 2024-03-21T10:37:24.210 INFO:journalctl@ceph.osd.3.smithi047.stdout:Mar 21 10:37:23 smithi047 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-3[50720]: 2024-03-21T10:37:23.831+0000 7f7839a00700 -1 osd.3 111 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-03-21T10:36:55.928288+0000 front 2024-03-21T10:36:55.929070+0000 (oldest deadline 2024-03-21T10:37:18.827820+0000) 2024-03-21T10:37:24.476 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 10:37:24 smithi100 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-10[59767]: 2024-03-21T10:37:24.202+0000 7f315da00700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-03-21T10:36:52.673512+0000 front 2024-03-21T10:36:52.673688+0000 (oldest deadline 2024-03-21T10:37:16.675039+0000) 2024-03-21T10:37:24.704 INFO:journalctl@ceph.osd.5.smithi157.stdout:Mar 21 10:37:24 smithi157 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-5[47667]: 2024-03-21T10:37:24.348+0000 7f242fa00700 -1 osd.5 111 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-03-21T10:36:55.338628+0000 front 2024-03-21T10:36:55.339174+0000 (oldest deadline 2024-03-21T10:37:19.438029+0000) 2024-03-21T10:37:24.961 INFO:journalctl@ceph.osd.3.smithi047.stdout:Mar 21 10:37:24 smithi047 podman[103266]: 2024-03-21 10:37:24.761161437 +0000 UTC m=+5.908692508 container died 042c8ec9454629e88fa48d612a95088b6c65c7cf17938338978dc5f32b2de43e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-3, org.label-schema.schema-version=1.0, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, org.label-schema.license=GPLv2, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, ceph=True, org.label-schema.build-date=20240319, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, CEPH_POINT_RELEASE=) 2024-03-21T10:37:24.961 INFO:journalctl@ceph.osd.9.smithi047.stdout:Mar 21 10:37:24 smithi047 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-9[64137]: 2024-03-21T10:37:24.530+0000 7f02dc200700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-03-21T10:36:54.991613+0000 front 2024-03-21T10:36:54.991458+0000 (oldest deadline 2024-03-21T10:37:17.891147+0000) 2024-03-21T10:37:24.986 INFO:journalctl@ceph.osd.11.smithi157.stdout:Mar 21 10:37:24 smithi157 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-11[61259]: 2024-03-21T10:37:24.706+0000 7fd0f8400700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-03-21T10:36:54.398353+0000 front 2024-03-21T10:36:54.398068+0000 (oldest deadline 2024-03-21T10:37:16.698280+0000) 2024-03-21T10:37:25.460 INFO:journalctl@ceph.osd.3.smithi047.stdout:Mar 21 10:37:25 smithi047 podman[103266]: 2024-03-21 10:37:25.300642267 +0000 UTC m=+6.448173294 container cleanup 042c8ec9454629e88fa48d612a95088b6c65c7cf17938338978dc5f32b2de43e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-3, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, ceph=True, io.buildah.version=1.33.5, org.label-schema.build-date=20240319, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS) 2024-03-21T10:37:25.461 INFO:journalctl@ceph.osd.3.smithi047.stdout:Mar 21 10:37:25 smithi047 bash[103266]: ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-3 2024-03-21T10:37:25.476 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 10:37:25 smithi100 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-10[59767]: 2024-03-21T10:37:25.200+0000 7f315da00700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-03-21T10:36:52.673512+0000 front 2024-03-21T10:36:52.673688+0000 (oldest deadline 2024-03-21T10:37:16.675039+0000) 2024-03-21T10:37:25.729 INFO:journalctl@ceph.osd.5.smithi157.stdout:Mar 21 10:37:25 smithi157 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-5[47667]: 2024-03-21T10:37:25.380+0000 7f242fa00700 -1 osd.5 111 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-03-21T10:36:55.338628+0000 front 2024-03-21T10:36:55.339174+0000 (oldest deadline 2024-03-21T10:37:19.438029+0000) 2024-03-21T10:37:25.960 INFO:journalctl@ceph.osd.9.smithi047.stdout:Mar 21 10:37:25 smithi047 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-9[64137]: 2024-03-21T10:37:25.571+0000 7f02dc200700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-03-21T10:36:54.991613+0000 front 2024-03-21T10:36:54.991458+0000 (oldest deadline 2024-03-21T10:37:17.891147+0000) 2024-03-21T10:37:25.961 INFO:journalctl@ceph.osd.3.smithi047.stdout:Mar 21 10:37:25 smithi047 podman[103283]: 2024-03-21 10:37:25.594369059 +0000 UTC m=+0.821315913 container remove 042c8ec9454629e88fa48d612a95088b6c65c7cf17938338978dc5f32b2de43e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-3, CEPH_POINT_RELEASE=, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, ceph=True, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240319, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True) 2024-03-21T10:37:25.986 INFO:journalctl@ceph.osd.11.smithi157.stdout:Mar 21 10:37:25 smithi157 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-11[61259]: 2024-03-21T10:37:25.731+0000 7fd0f8400700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-03-21T10:36:54.398353+0000 front 2024-03-21T10:36:54.398068+0000 (oldest deadline 2024-03-21T10:37:16.698280+0000) 2024-03-21T10:37:26.226 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 10:37:26 smithi100 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-10[59767]: 2024-03-21T10:37:26.155+0000 7f315da00700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-03-21T10:36:52.673512+0000 front 2024-03-21T10:36:52.673688+0000 (oldest deadline 2024-03-21T10:37:16.675039+0000) 2024-03-21T10:37:26.679 INFO:journalctl@ceph.osd.5.smithi157.stdout:Mar 21 10:37:26 smithi157 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-5[47667]: 2024-03-21T10:37:26.331+0000 7f242fa00700 -1 osd.5 111 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-03-21T10:36:55.338628+0000 front 2024-03-21T10:36:55.339174+0000 (oldest deadline 2024-03-21T10:37:19.438029+0000) 2024-03-21T10:37:26.711 INFO:journalctl@ceph.osd.9.smithi047.stdout:Mar 21 10:37:26 smithi047 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-9[64137]: 2024-03-21T10:37:26.567+0000 7f02dc200700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-03-21T10:36:54.991613+0000 front 2024-03-21T10:36:54.991458+0000 (oldest deadline 2024-03-21T10:37:17.891147+0000) 2024-03-21T10:37:26.711 INFO:journalctl@ceph.osd.3.smithi047.stdout:Mar 21 10:37:26 smithi047 podman[103371]: 2024-03-21 10:37:26.344593502 +0000 UTC m=+0.033381584 image pull 237e43c7a65ff48534d94b97c1bfdbe4b7be5672e3d9a613745083ea3702aa54 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232 2024-03-21T10:37:26.711 INFO:journalctl@ceph.osd.3.smithi047.stdout:Mar 21 10:37:26 smithi047 podman[103371]: 2024-03-21 10:37:26.445149126 +0000 UTC m=+0.133937248 container create fc930cd9828db35cdb735edf712c21d82aa99e82a8c14e7943f9164c2812ead8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-3-deactivate, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, io.buildah.version=1.33.5, org.label-schema.build-date=20240319, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-21T10:37:26.986 INFO:journalctl@ceph.osd.11.smithi157.stdout:Mar 21 10:37:26 smithi157 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-11[61259]: 2024-03-21T10:37:26.681+0000 7fd0f8400700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-03-21T10:36:54.398353+0000 front 2024-03-21T10:36:54.398068+0000 (oldest deadline 2024-03-21T10:37:16.698280+0000) 2024-03-21T10:37:27.210 INFO:journalctl@ceph.osd.6.smithi047.stdout:Mar 21 10:37:27 smithi047 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-6[57372]: 2024-03-21T10:37:27.099+0000 7f157e200700 -1 osd.6 111 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-03-21T10:37:03.517239+0000 front 2024-03-21T10:37:03.517159+0000 (oldest deadline 2024-03-21T10:37:26.417262+0000) 2024-03-21T10:37:27.211 INFO:journalctl@ceph.osd.3.smithi047.stdout:Mar 21 10:37:26 smithi047 podman[103371]: 2024-03-21 10:37:26.886161623 +0000 UTC m=+0.574949792 container init fc930cd9828db35cdb735edf712c21d82aa99e82a8c14e7943f9164c2812ead8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-3-deactivate, maintainer=Guillaume Abrioux , GIT_CLEAN=True, io.buildah.version=1.33.5, ceph=True, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240319, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS) 2024-03-21T10:37:27.211 INFO:journalctl@ceph.osd.3.smithi047.stdout:Mar 21 10:37:26 smithi047 podman[103371]: 2024-03-21 10:37:26.897715264 +0000 UTC m=+0.586503390 container start fc930cd9828db35cdb735edf712c21d82aa99e82a8c14e7943f9164c2812ead8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-3-deactivate, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_CLEAN=True, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, GIT_BRANCH=HEAD, org.label-schema.build-date=20240319, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True) 2024-03-21T10:37:27.211 INFO:journalctl@ceph.osd.3.smithi047.stdout:Mar 21 10:37:26 smithi047 podman[103371]: 2024-03-21 10:37:26.989420732 +0000 UTC m=+0.678208904 container attach fc930cd9828db35cdb735edf712c21d82aa99e82a8c14e7943f9164c2812ead8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-3-deactivate, ceph=True, org.label-schema.build-date=20240319, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, io.buildah.version=1.33.5, GIT_CLEAN=True) 2024-03-21T10:37:27.476 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 10:37:27 smithi100 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-10[59767]: 2024-03-21T10:37:27.166+0000 7f315da00700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-03-21T10:36:52.673512+0000 front 2024-03-21T10:36:52.673688+0000 (oldest deadline 2024-03-21T10:37:16.675039+0000) 2024-03-21T10:37:27.668 INFO:journalctl@ceph.osd.5.smithi157.stdout:Mar 21 10:37:27 smithi157 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-5[47667]: 2024-03-21T10:37:27.287+0000 7f242fa00700 -1 osd.5 111 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-03-21T10:36:55.338628+0000 front 2024-03-21T10:36:55.339174+0000 (oldest deadline 2024-03-21T10:37:19.438029+0000) 2024-03-21T10:37:27.711 INFO:journalctl@ceph.osd.3.smithi047.stdout:Mar 21 10:37:27 smithi047 podman[103397]: 2024-03-21 10:37:27.406465069 +0000 UTC m=+0.038527960 container died fc930cd9828db35cdb735edf712c21d82aa99e82a8c14e7943f9164c2812ead8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-3-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240319, GIT_BRANCH=HEAD, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, io.buildah.version=1.33.5, GIT_CLEAN=True) 2024-03-21T10:37:27.711 INFO:journalctl@ceph.osd.9.smithi047.stdout:Mar 21 10:37:27 smithi047 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-9[64137]: 2024-03-21T10:37:27.586+0000 7f02dc200700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-03-21T10:36:54.991613+0000 front 2024-03-21T10:36:54.991458+0000 (oldest deadline 2024-03-21T10:37:17.891147+0000) 2024-03-21T10:37:27.711 INFO:journalctl@ceph.osd.9.smithi047.stdout:Mar 21 10:37:27 smithi047 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-9[64137]: 2024-03-21T10:37:27.586+0000 7f02dc200700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-03-21T10:37:02.494627+0000 front 2024-03-21T10:37:02.494473+0000 (oldest deadline 2024-03-21T10:37:26.594304+0000) 2024-03-21T10:37:27.986 INFO:journalctl@ceph.osd.11.smithi157.stdout:Mar 21 10:37:27 smithi157 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-11[61259]: 2024-03-21T10:37:27.670+0000 7fd0f8400700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-03-21T10:36:54.398353+0000 front 2024-03-21T10:36:54.398068+0000 (oldest deadline 2024-03-21T10:37:16.698280+0000) 2024-03-21T10:37:27.987 INFO:journalctl@ceph.osd.8.smithi157.stdout:Mar 21 10:37:27 smithi157 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-8[54391]: 2024-03-21T10:37:27.794+0000 7fca85e00700 -1 osd.8 111 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-03-21T10:37:03.032692+0000 front 2024-03-21T10:37:03.032837+0000 (oldest deadline 2024-03-21T10:37:27.132936+0000) 2024-03-21T10:37:28.171 DEBUG:teuthology.orchestra.run.smithi047:> sudo pkill -f 'journalctl -f -n 0 -u ceph-3056d51a-e76b-11ee-95cd-87774f69a715@osd.3.service' 2024-03-21T10:37:28.285 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-21T10:37:28.285 INFO:tasks.cephadm.osd.3:Stopped osd.3 2024-03-21T10:37:28.285 INFO:tasks.cephadm.osd.4:Stopping osd.4... 2024-03-21T10:37:28.285 DEBUG:teuthology.orchestra.run.smithi100:> sudo systemctl stop ceph-3056d51a-e76b-11ee-95cd-87774f69a715@osd.4 2024-03-21T10:37:28.460 INFO:journalctl@ceph.osd.6.smithi047.stdout:Mar 21 10:37:28 smithi047 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-6[57372]: 2024-03-21T10:37:28.136+0000 7f157e200700 -1 osd.6 111 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-03-21T10:37:03.517239+0000 front 2024-03-21T10:37:03.517159+0000 (oldest deadline 2024-03-21T10:37:26.417262+0000) 2024-03-21T10:37:28.476 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 10:37:28 smithi100 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-10[59767]: 2024-03-21T10:37:28.194+0000 7f315da00700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-03-21T10:36:52.673512+0000 front 2024-03-21T10:36:52.673688+0000 (oldest deadline 2024-03-21T10:37:16.675039+0000) 2024-03-21T10:37:28.476 INFO:journalctl@ceph.osd.4.smithi100.stdout:Mar 21 10:37:28 smithi100 systemd[1]: Stopping Ceph osd.4 for 3056d51a-e76b-11ee-95cd-87774f69a715... 2024-03-21T10:37:28.638 INFO:journalctl@ceph.osd.5.smithi157.stdout:Mar 21 10:37:28 smithi157 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-5[47667]: 2024-03-21T10:37:28.253+0000 7f242fa00700 -1 osd.5 111 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-03-21T10:36:55.338628+0000 front 2024-03-21T10:36:55.339174+0000 (oldest deadline 2024-03-21T10:37:19.438029+0000) 2024-03-21T10:37:28.960 INFO:journalctl@ceph.osd.9.smithi047.stdout:Mar 21 10:37:28 smithi047 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-9[64137]: 2024-03-21T10:37:28.582+0000 7f02dc200700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-03-21T10:36:54.991613+0000 front 2024-03-21T10:36:54.991458+0000 (oldest deadline 2024-03-21T10:37:17.891147+0000) 2024-03-21T10:37:28.961 INFO:journalctl@ceph.osd.9.smithi047.stdout:Mar 21 10:37:28 smithi047 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-9[64137]: 2024-03-21T10:37:28.582+0000 7f02dc200700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-03-21T10:37:02.494627+0000 front 2024-03-21T10:37:02.494473+0000 (oldest deadline 2024-03-21T10:37:26.594304+0000) 2024-03-21T10:37:28.986 INFO:journalctl@ceph.osd.8.smithi157.stdout:Mar 21 10:37:28 smithi157 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-8[54391]: 2024-03-21T10:37:28.843+0000 7fca85e00700 -1 osd.8 111 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-03-21T10:37:03.032692+0000 front 2024-03-21T10:37:03.032837+0000 (oldest deadline 2024-03-21T10:37:27.132936+0000) 2024-03-21T10:37:28.986 INFO:journalctl@ceph.osd.11.smithi157.stdout:Mar 21 10:37:28 smithi157 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-11[61259]: 2024-03-21T10:37:28.640+0000 7fd0f8400700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-03-21T10:36:54.398353+0000 front 2024-03-21T10:36:54.398068+0000 (oldest deadline 2024-03-21T10:37:16.698280+0000) 2024-03-21T10:37:29.161 INFO:journalctl@ceph.osd.4.smithi100.stdout:Mar 21 10:37:28 smithi100 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-4[46429]: 2024-03-21T10:37:28.848+0000 7fa0c9600700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.4 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-21T10:37:29.161 INFO:journalctl@ceph.osd.4.smithi100.stdout:Mar 21 10:37:28 smithi100 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-4[46429]: 2024-03-21T10:37:28.848+0000 7fa0c9600700 -1 osd.4 111 *** Got signal Terminated *** 2024-03-21T10:37:29.161 INFO:journalctl@ceph.osd.4.smithi100.stdout:Mar 21 10:37:28 smithi100 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-4[46429]: 2024-03-21T10:37:28.848+0000 7fa0c9600700 -1 osd.4 111 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-21T10:37:29.161 INFO:journalctl@ceph.osd.4.smithi100.stdout:Mar 21 10:37:28 smithi100 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-4[46429]: 2024-03-21T10:37:28.905+0000 7fa0c4e00700 -1 osd.4 111 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-03-21T10:37:00.399137+0000 front 2024-03-21T10:37:00.399169+0000 (oldest deadline 2024-03-21T10:37:28.000866+0000) 2024-03-21T10:37:29.460 INFO:journalctl@ceph.osd.6.smithi047.stdout:Mar 21 10:37:29 smithi047 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-6[57372]: 2024-03-21T10:37:29.100+0000 7f157e200700 -1 osd.6 111 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-03-21T10:37:03.517239+0000 front 2024-03-21T10:37:03.517159+0000 (oldest deadline 2024-03-21T10:37:26.417262+0000) 2024-03-21T10:37:29.475 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 10:37:29 smithi100 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-10[59767]: 2024-03-21T10:37:29.161+0000 7f315da00700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-03-21T10:36:52.673512+0000 front 2024-03-21T10:36:52.673688+0000 (oldest deadline 2024-03-21T10:37:16.675039+0000) 2024-03-21T10:37:29.486 INFO:journalctl@ceph.osd.5.smithi157.stdout:Mar 21 10:37:29 smithi157 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-5[47667]: 2024-03-21T10:37:29.221+0000 7f242fa00700 -1 osd.5 111 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-03-21T10:36:55.338628+0000 front 2024-03-21T10:36:55.339174+0000 (oldest deadline 2024-03-21T10:37:19.438029+0000) 2024-03-21T10:37:29.726 INFO:journalctl@ceph.osd.7.smithi100.stdout:Mar 21 10:37:29 smithi100 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-7[53119]: 2024-03-21T10:37:29.476+0000 7fb6ed400700 -1 osd.7 111 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-03-21T10:37:04.136587+0000 front 2024-03-21T10:37:04.136839+0000 (oldest deadline 2024-03-21T10:37:29.436935+0000) 2024-03-21T10:37:29.960 INFO:journalctl@ceph.osd.9.smithi047.stdout:Mar 21 10:37:29 smithi047 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-9[64137]: 2024-03-21T10:37:29.574+0000 7f02dc200700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-03-21T10:36:54.991613+0000 front 2024-03-21T10:36:54.991458+0000 (oldest deadline 2024-03-21T10:37:17.891147+0000) 2024-03-21T10:37:29.961 INFO:journalctl@ceph.osd.9.smithi047.stdout:Mar 21 10:37:29 smithi047 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-9[64137]: 2024-03-21T10:37:29.574+0000 7f02dc200700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-03-21T10:37:02.494627+0000 front 2024-03-21T10:37:02.494473+0000 (oldest deadline 2024-03-21T10:37:26.594304+0000) 2024-03-21T10:37:29.986 INFO:journalctl@ceph.osd.8.smithi157.stdout:Mar 21 10:37:29 smithi157 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-8[54391]: 2024-03-21T10:37:29.817+0000 7fca85e00700 -1 osd.8 111 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-03-21T10:37:03.032692+0000 front 2024-03-21T10:37:03.032837+0000 (oldest deadline 2024-03-21T10:37:27.132936+0000) 2024-03-21T10:37:29.986 INFO:journalctl@ceph.osd.11.smithi157.stdout:Mar 21 10:37:29 smithi157 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-11[61259]: 2024-03-21T10:37:29.598+0000 7fd0f8400700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-03-21T10:36:54.398353+0000 front 2024-03-21T10:36:54.398068+0000 (oldest deadline 2024-03-21T10:37:16.698280+0000) 2024-03-21T10:37:29.986 INFO:journalctl@ceph.osd.11.smithi157.stdout:Mar 21 10:37:29 smithi157 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-11[61259]: 2024-03-21T10:37:29.598+0000 7fd0f8400700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-03-21T10:37:04.901925+0000 front 2024-03-21T10:37:04.902083+0000 (oldest deadline 2024-03-21T10:37:29.502349+0000) 2024-03-21T10:37:30.163 INFO:journalctl@ceph.osd.4.smithi100.stdout:Mar 21 10:37:29 smithi100 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-4[46429]: 2024-03-21T10:37:29.914+0000 7fa0c4e00700 -1 osd.4 111 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-03-21T10:37:00.399137+0000 front 2024-03-21T10:37:00.399169+0000 (oldest deadline 2024-03-21T10:37:28.000866+0000) 2024-03-21T10:37:30.163 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 10:37:30 smithi100 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-10[59767]: 2024-03-21T10:37:30.163+0000 7f315da00700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-03-21T10:36:52.673512+0000 front 2024-03-21T10:36:52.673688+0000 (oldest deadline 2024-03-21T10:37:16.675039+0000) 2024-03-21T10:37:30.163 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 10:37:30 smithi100 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-10[59767]: 2024-03-21T10:37:30.163+0000 7f315da00700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-03-21T10:37:05.379887+0000 front 2024-03-21T10:37:05.379536+0000 (oldest deadline 2024-03-21T10:37:30.079809+0000) 2024-03-21T10:37:30.460 INFO:journalctl@ceph.osd.6.smithi047.stdout:Mar 21 10:37:30 smithi047 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-6[57372]: 2024-03-21T10:37:30.085+0000 7f157e200700 -1 osd.6 111 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-03-21T10:37:03.517239+0000 front 2024-03-21T10:37:03.517159+0000 (oldest deadline 2024-03-21T10:37:26.417262+0000) 2024-03-21T10:37:30.486 INFO:journalctl@ceph.osd.5.smithi157.stdout:Mar 21 10:37:30 smithi157 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-5[47667]: 2024-03-21T10:37:30.173+0000 7f242fa00700 -1 osd.5 111 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-03-21T10:36:55.338628+0000 front 2024-03-21T10:36:55.339174+0000 (oldest deadline 2024-03-21T10:37:19.438029+0000) 2024-03-21T10:37:30.878 INFO:journalctl@ceph.osd.7.smithi100.stdout:Mar 21 10:37:30 smithi100 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-7[53119]: 2024-03-21T10:37:30.495+0000 7fb6ed400700 -1 osd.7 111 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-03-21T10:37:04.136587+0000 front 2024-03-21T10:37:04.136839+0000 (oldest deadline 2024-03-21T10:37:29.436935+0000) 2024-03-21T10:37:30.960 INFO:journalctl@ceph.osd.9.smithi047.stdout:Mar 21 10:37:30 smithi047 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-9[64137]: 2024-03-21T10:37:30.610+0000 7f02dc200700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-03-21T10:36:54.991613+0000 front 2024-03-21T10:36:54.991458+0000 (oldest deadline 2024-03-21T10:37:17.891147+0000) 2024-03-21T10:37:30.961 INFO:journalctl@ceph.osd.9.smithi047.stdout:Mar 21 10:37:30 smithi047 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-9[64137]: 2024-03-21T10:37:30.610+0000 7f02dc200700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-03-21T10:37:02.494627+0000 front 2024-03-21T10:37:02.494473+0000 (oldest deadline 2024-03-21T10:37:26.594304+0000) 2024-03-21T10:37:30.986 INFO:journalctl@ceph.osd.8.smithi157.stdout:Mar 21 10:37:30 smithi157 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-8[54391]: 2024-03-21T10:37:30.794+0000 7fca85e00700 -1 osd.8 111 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-03-21T10:37:03.032692+0000 front 2024-03-21T10:37:03.032837+0000 (oldest deadline 2024-03-21T10:37:27.132936+0000) 2024-03-21T10:37:30.986 INFO:journalctl@ceph.osd.11.smithi157.stdout:Mar 21 10:37:30 smithi157 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-11[61259]: 2024-03-21T10:37:30.588+0000 7fd0f8400700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-03-21T10:36:54.398353+0000 front 2024-03-21T10:36:54.398068+0000 (oldest deadline 2024-03-21T10:37:16.698280+0000) 2024-03-21T10:37:30.986 INFO:journalctl@ceph.osd.11.smithi157.stdout:Mar 21 10:37:30 smithi157 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-11[61259]: 2024-03-21T10:37:30.588+0000 7fd0f8400700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-03-21T10:37:04.901925+0000 front 2024-03-21T10:37:04.902083+0000 (oldest deadline 2024-03-21T10:37:29.502349+0000) 2024-03-21T10:37:31.129 INFO:journalctl@ceph.osd.4.smithi100.stdout:Mar 21 10:37:30 smithi100 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-4[46429]: 2024-03-21T10:37:30.879+0000 7fa0c4e00700 -1 osd.4 111 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-03-21T10:37:00.399137+0000 front 2024-03-21T10:37:00.399169+0000 (oldest deadline 2024-03-21T10:37:28.000866+0000) 2024-03-21T10:37:31.460 INFO:journalctl@ceph.osd.6.smithi047.stdout:Mar 21 10:37:31 smithi047 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-6[57372]: 2024-03-21T10:37:31.041+0000 7f157e200700 -1 osd.6 111 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-03-21T10:37:03.517239+0000 front 2024-03-21T10:37:03.517159+0000 (oldest deadline 2024-03-21T10:37:26.417262+0000) 2024-03-21T10:37:31.464 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 10:37:31 smithi100 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-10[59767]: 2024-03-21T10:37:31.131+0000 7f315da00700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-03-21T10:36:52.673512+0000 front 2024-03-21T10:36:52.673688+0000 (oldest deadline 2024-03-21T10:37:16.675039+0000) 2024-03-21T10:37:31.464 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 10:37:31 smithi100 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-10[59767]: 2024-03-21T10:37:31.131+0000 7f315da00700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-03-21T10:37:05.379887+0000 front 2024-03-21T10:37:05.379536+0000 (oldest deadline 2024-03-21T10:37:30.079809+0000) 2024-03-21T10:37:31.486 INFO:journalctl@ceph.osd.5.smithi157.stdout:Mar 21 10:37:31 smithi157 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-5[47667]: 2024-03-21T10:37:31.171+0000 7f242fa00700 -1 osd.5 111 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-03-21T10:36:55.338628+0000 front 2024-03-21T10:36:55.339174+0000 (oldest deadline 2024-03-21T10:37:19.438029+0000) 2024-03-21T10:37:31.726 INFO:journalctl@ceph.osd.7.smithi100.stdout:Mar 21 10:37:31 smithi100 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-7[53119]: 2024-03-21T10:37:31.465+0000 7fb6ed400700 -1 osd.7 111 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-03-21T10:37:04.136587+0000 front 2024-03-21T10:37:04.136839+0000 (oldest deadline 2024-03-21T10:37:29.436935+0000) 2024-03-21T10:37:31.840 INFO:journalctl@ceph.osd.11.smithi157.stdout:Mar 21 10:37:31 smithi157 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-11[61259]: 2024-03-21T10:37:31.585+0000 7fd0f8400700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-03-21T10:36:54.398353+0000 front 2024-03-21T10:36:54.398068+0000 (oldest deadline 2024-03-21T10:37:16.698280+0000) 2024-03-21T10:37:31.840 INFO:journalctl@ceph.osd.11.smithi157.stdout:Mar 21 10:37:31 smithi157 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-11[61259]: 2024-03-21T10:37:31.585+0000 7fd0f8400700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-03-21T10:37:04.901925+0000 front 2024-03-21T10:37:04.902083+0000 (oldest deadline 2024-03-21T10:37:29.502349+0000) 2024-03-21T10:37:31.960 INFO:journalctl@ceph.osd.9.smithi047.stdout:Mar 21 10:37:31 smithi047 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-9[64137]: 2024-03-21T10:37:31.576+0000 7f02dc200700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-03-21T10:36:54.991613+0000 front 2024-03-21T10:36:54.991458+0000 (oldest deadline 2024-03-21T10:37:17.891147+0000) 2024-03-21T10:37:31.960 INFO:journalctl@ceph.osd.9.smithi047.stdout:Mar 21 10:37:31 smithi047 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-9[64137]: 2024-03-21T10:37:31.576+0000 7f02dc200700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-03-21T10:37:02.494627+0000 front 2024-03-21T10:37:02.494473+0000 (oldest deadline 2024-03-21T10:37:26.594304+0000) 2024-03-21T10:37:32.150 INFO:journalctl@ceph.osd.8.smithi157.stdout:Mar 21 10:37:31 smithi157 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-8[54391]: 2024-03-21T10:37:31.842+0000 7fca85e00700 -1 osd.8 111 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-03-21T10:37:03.032692+0000 front 2024-03-21T10:37:03.032837+0000 (oldest deadline 2024-03-21T10:37:27.132936+0000) 2024-03-21T10:37:32.226 INFO:journalctl@ceph.osd.4.smithi100.stdout:Mar 21 10:37:31 smithi100 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-4[46429]: 2024-03-21T10:37:31.901+0000 7fa0c4e00700 -1 osd.4 111 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-03-21T10:37:00.399137+0000 front 2024-03-21T10:37:00.399169+0000 (oldest deadline 2024-03-21T10:37:28.000866+0000) 2024-03-21T10:37:32.227 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 10:37:32 smithi100 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-10[59767]: 2024-03-21T10:37:32.133+0000 7f315da00700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-03-21T10:36:52.673512+0000 front 2024-03-21T10:36:52.673688+0000 (oldest deadline 2024-03-21T10:37:16.675039+0000) 2024-03-21T10:37:32.227 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 10:37:32 smithi100 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-10[59767]: 2024-03-21T10:37:32.133+0000 7f315da00700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-03-21T10:37:05.379887+0000 front 2024-03-21T10:37:05.379536+0000 (oldest deadline 2024-03-21T10:37:30.079809+0000) 2024-03-21T10:37:32.460 INFO:journalctl@ceph.osd.6.smithi047.stdout:Mar 21 10:37:32 smithi047 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-6[57372]: 2024-03-21T10:37:32.086+0000 7f157e200700 -1 osd.6 111 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-03-21T10:37:03.517239+0000 front 2024-03-21T10:37:03.517159+0000 (oldest deadline 2024-03-21T10:37:26.417262+0000) 2024-03-21T10:37:32.486 INFO:journalctl@ceph.osd.5.smithi157.stdout:Mar 21 10:37:32 smithi157 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-5[47667]: 2024-03-21T10:37:32.152+0000 7f242fa00700 -1 osd.5 111 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-03-21T10:36:55.338628+0000 front 2024-03-21T10:36:55.339174+0000 (oldest deadline 2024-03-21T10:37:19.438029+0000) 2024-03-21T10:37:32.486 INFO:journalctl@ceph.osd.5.smithi157.stdout:Mar 21 10:37:32 smithi157 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-5[47667]: 2024-03-21T10:37:32.152+0000 7f242fa00700 -1 osd.5 111 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-03-21T10:37:04.140579+0000 front 2024-03-21T10:37:04.140414+0000 (oldest deadline 2024-03-21T10:37:31.741045+0000) 2024-03-21T10:37:32.726 INFO:journalctl@ceph.osd.7.smithi100.stdout:Mar 21 10:37:32 smithi100 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-7[53119]: 2024-03-21T10:37:32.476+0000 7fb6ed400700 -1 osd.7 111 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-03-21T10:37:04.136587+0000 front 2024-03-21T10:37:04.136839+0000 (oldest deadline 2024-03-21T10:37:29.436935+0000) 2024-03-21T10:37:32.886 INFO:journalctl@ceph.osd.11.smithi157.stdout:Mar 21 10:37:32 smithi157 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-11[61259]: 2024-03-21T10:37:32.583+0000 7fd0f8400700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-03-21T10:36:54.398353+0000 front 2024-03-21T10:36:54.398068+0000 (oldest deadline 2024-03-21T10:37:16.698280+0000) 2024-03-21T10:37:32.886 INFO:journalctl@ceph.osd.11.smithi157.stdout:Mar 21 10:37:32 smithi157 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-11[61259]: 2024-03-21T10:37:32.583+0000 7fd0f8400700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-03-21T10:37:04.901925+0000 front 2024-03-21T10:37:04.902083+0000 (oldest deadline 2024-03-21T10:37:29.502349+0000) 2024-03-21T10:37:32.960 INFO:journalctl@ceph.osd.9.smithi047.stdout:Mar 21 10:37:32 smithi047 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-9[64137]: 2024-03-21T10:37:32.565+0000 7f02dc200700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-03-21T10:36:54.991613+0000 front 2024-03-21T10:36:54.991458+0000 (oldest deadline 2024-03-21T10:37:17.891147+0000) 2024-03-21T10:37:32.960 INFO:journalctl@ceph.osd.9.smithi047.stdout:Mar 21 10:37:32 smithi047 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-9[64137]: 2024-03-21T10:37:32.565+0000 7f02dc200700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-03-21T10:37:02.494627+0000 front 2024-03-21T10:37:02.494473+0000 (oldest deadline 2024-03-21T10:37:26.594304+0000) 2024-03-21T10:37:33.148 INFO:journalctl@ceph.osd.8.smithi157.stdout:Mar 21 10:37:32 smithi157 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-8[54391]: 2024-03-21T10:37:32.888+0000 7fca85e00700 -1 osd.8 111 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-03-21T10:37:03.032692+0000 front 2024-03-21T10:37:03.032837+0000 (oldest deadline 2024-03-21T10:37:27.132936+0000) 2024-03-21T10:37:33.226 INFO:journalctl@ceph.osd.4.smithi100.stdout:Mar 21 10:37:32 smithi100 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-4[46429]: 2024-03-21T10:37:32.940+0000 7fa0c4e00700 -1 osd.4 111 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-03-21T10:37:00.399137+0000 front 2024-03-21T10:37:00.399169+0000 (oldest deadline 2024-03-21T10:37:28.000866+0000) 2024-03-21T10:37:33.226 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 10:37:33 smithi100 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-10[59767]: 2024-03-21T10:37:33.157+0000 7f315da00700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-03-21T10:36:52.673512+0000 front 2024-03-21T10:36:52.673688+0000 (oldest deadline 2024-03-21T10:37:16.675039+0000) 2024-03-21T10:37:33.226 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 10:37:33 smithi100 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-10[59767]: 2024-03-21T10:37:33.157+0000 7f315da00700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-03-21T10:37:05.379887+0000 front 2024-03-21T10:37:05.379536+0000 (oldest deadline 2024-03-21T10:37:30.079809+0000) 2024-03-21T10:37:33.460 INFO:journalctl@ceph.osd.6.smithi047.stdout:Mar 21 10:37:33 smithi047 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-6[57372]: 2024-03-21T10:37:33.113+0000 7f157e200700 -1 osd.6 111 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-03-21T10:37:03.517239+0000 front 2024-03-21T10:37:03.517159+0000 (oldest deadline 2024-03-21T10:37:26.417262+0000) 2024-03-21T10:37:33.486 INFO:journalctl@ceph.osd.5.smithi157.stdout:Mar 21 10:37:33 smithi157 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-5[47667]: 2024-03-21T10:37:33.151+0000 7f242fa00700 -1 osd.5 111 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-03-21T10:36:55.338628+0000 front 2024-03-21T10:36:55.339174+0000 (oldest deadline 2024-03-21T10:37:19.438029+0000) 2024-03-21T10:37:33.486 INFO:journalctl@ceph.osd.5.smithi157.stdout:Mar 21 10:37:33 smithi157 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-5[47667]: 2024-03-21T10:37:33.151+0000 7f242fa00700 -1 osd.5 111 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-03-21T10:37:04.140579+0000 front 2024-03-21T10:37:04.140414+0000 (oldest deadline 2024-03-21T10:37:31.741045+0000) 2024-03-21T10:37:33.895 INFO:journalctl@ceph.osd.11.smithi157.stdout:Mar 21 10:37:33 smithi157 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-11[61259]: 2024-03-21T10:37:33.623+0000 7fd0f8400700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-03-21T10:36:54.398353+0000 front 2024-03-21T10:36:54.398068+0000 (oldest deadline 2024-03-21T10:37:16.698280+0000) 2024-03-21T10:37:33.895 INFO:journalctl@ceph.osd.11.smithi157.stdout:Mar 21 10:37:33 smithi157 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-11[61259]: 2024-03-21T10:37:33.623+0000 7fd0f8400700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-03-21T10:37:04.901925+0000 front 2024-03-21T10:37:04.902083+0000 (oldest deadline 2024-03-21T10:37:29.502349+0000) 2024-03-21T10:37:33.960 INFO:journalctl@ceph.osd.9.smithi047.stdout:Mar 21 10:37:33 smithi047 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-9[64137]: 2024-03-21T10:37:33.563+0000 7f02dc200700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-03-21T10:36:54.991613+0000 front 2024-03-21T10:36:54.991458+0000 (oldest deadline 2024-03-21T10:37:17.891147+0000) 2024-03-21T10:37:33.961 INFO:journalctl@ceph.osd.9.smithi047.stdout:Mar 21 10:37:33 smithi047 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-9[64137]: 2024-03-21T10:37:33.563+0000 7f02dc200700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-03-21T10:37:02.494627+0000 front 2024-03-21T10:37:02.494473+0000 (oldest deadline 2024-03-21T10:37:26.594304+0000) 2024-03-21T10:37:33.976 INFO:journalctl@ceph.osd.7.smithi100.stdout:Mar 21 10:37:33 smithi100 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-7[53119]: 2024-03-21T10:37:33.490+0000 7fb6ed400700 -1 osd.7 111 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-03-21T10:37:04.136587+0000 front 2024-03-21T10:37:04.136839+0000 (oldest deadline 2024-03-21T10:37:29.436935+0000) 2024-03-21T10:37:34.236 INFO:journalctl@ceph.osd.8.smithi157.stdout:Mar 21 10:37:33 smithi157 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-8[54391]: 2024-03-21T10:37:33.897+0000 7fca85e00700 -1 osd.8 111 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-03-21T10:37:03.032692+0000 front 2024-03-21T10:37:03.032837+0000 (oldest deadline 2024-03-21T10:37:27.132936+0000) 2024-03-21T10:37:34.236 INFO:journalctl@ceph.osd.5.smithi157.stdout:Mar 21 10:37:34 smithi157 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-5[47667]: 2024-03-21T10:37:34.122+0000 7f242fa00700 -1 osd.5 111 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-03-21T10:36:55.338628+0000 front 2024-03-21T10:36:55.339174+0000 (oldest deadline 2024-03-21T10:37:19.438029+0000) 2024-03-21T10:37:34.237 INFO:journalctl@ceph.osd.5.smithi157.stdout:Mar 21 10:37:34 smithi157 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-5[47667]: 2024-03-21T10:37:34.122+0000 7f242fa00700 -1 osd.5 111 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-03-21T10:37:04.140579+0000 front 2024-03-21T10:37:04.140414+0000 (oldest deadline 2024-03-21T10:37:31.741045+0000) 2024-03-21T10:37:34.460 INFO:journalctl@ceph.osd.6.smithi047.stdout:Mar 21 10:37:34 smithi047 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-6[57372]: 2024-03-21T10:37:34.110+0000 7f157e200700 -1 osd.6 111 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-03-21T10:37:03.517239+0000 front 2024-03-21T10:37:03.517159+0000 (oldest deadline 2024-03-21T10:37:26.417262+0000) 2024-03-21T10:37:34.476 INFO:journalctl@ceph.osd.4.smithi100.stdout:Mar 21 10:37:34 smithi100 podman[71290]: 2024-03-21 10:37:34.366698817 +0000 UTC m=+5.663928278 container died e1dd02fadaabd39fe53e4b0411213b16e115f37ffc3c87e5373f3d20dd7b2d79 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-4, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_CLEAN=True, ceph=True, CEPH_POINT_RELEASE=, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.build-date=20240319, maintainer=Guillaume Abrioux , GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0) 2024-03-21T10:37:34.477 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 10:37:34 smithi100 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-10[59767]: 2024-03-21T10:37:34.189+0000 7f315da00700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-03-21T10:36:52.673512+0000 front 2024-03-21T10:36:52.673688+0000 (oldest deadline 2024-03-21T10:37:16.675039+0000) 2024-03-21T10:37:34.477 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 10:37:34 smithi100 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-10[59767]: 2024-03-21T10:37:34.189+0000 7f315da00700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-03-21T10:37:05.379887+0000 front 2024-03-21T10:37:05.379536+0000 (oldest deadline 2024-03-21T10:37:30.079809+0000) 2024-03-21T10:37:34.789 INFO:journalctl@ceph.osd.7.smithi100.stdout:Mar 21 10:37:34 smithi100 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-7[53119]: 2024-03-21T10:37:34.497+0000 7fb6ed400700 -1 osd.7 111 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-03-21T10:37:04.136587+0000 front 2024-03-21T10:37:04.136839+0000 (oldest deadline 2024-03-21T10:37:29.436935+0000) 2024-03-21T10:37:34.960 INFO:journalctl@ceph.osd.9.smithi047.stdout:Mar 21 10:37:34 smithi047 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-9[64137]: 2024-03-21T10:37:34.542+0000 7f02dc200700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-03-21T10:36:54.991613+0000 front 2024-03-21T10:36:54.991458+0000 (oldest deadline 2024-03-21T10:37:17.891147+0000) 2024-03-21T10:37:34.960 INFO:journalctl@ceph.osd.9.smithi047.stdout:Mar 21 10:37:34 smithi047 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-9[64137]: 2024-03-21T10:37:34.542+0000 7f02dc200700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-03-21T10:37:02.494627+0000 front 2024-03-21T10:37:02.494473+0000 (oldest deadline 2024-03-21T10:37:26.594304+0000) 2024-03-21T10:37:34.986 INFO:journalctl@ceph.osd.8.smithi157.stdout:Mar 21 10:37:34 smithi157 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-8[54391]: 2024-03-21T10:37:34.856+0000 7fca85e00700 -1 osd.8 111 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-03-21T10:37:03.032692+0000 front 2024-03-21T10:37:03.032837+0000 (oldest deadline 2024-03-21T10:37:27.132936+0000) 2024-03-21T10:37:34.986 INFO:journalctl@ceph.osd.11.smithi157.stdout:Mar 21 10:37:34 smithi157 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-11[61259]: 2024-03-21T10:37:34.661+0000 7fd0f8400700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-03-21T10:36:54.398353+0000 front 2024-03-21T10:36:54.398068+0000 (oldest deadline 2024-03-21T10:37:16.698280+0000) 2024-03-21T10:37:34.987 INFO:journalctl@ceph.osd.11.smithi157.stdout:Mar 21 10:37:34 smithi157 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-11[61259]: 2024-03-21T10:37:34.662+0000 7fd0f8400700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-03-21T10:37:04.901925+0000 front 2024-03-21T10:37:04.902083+0000 (oldest deadline 2024-03-21T10:37:29.502349+0000) 2024-03-21T10:37:35.083 INFO:journalctl@ceph.osd.4.smithi100.stdout:Mar 21 10:37:34 smithi100 podman[71290]: 2024-03-21 10:37:34.792090829 +0000 UTC m=+6.089320327 container cleanup e1dd02fadaabd39fe53e4b0411213b16e115f37ffc3c87e5373f3d20dd7b2d79 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-4, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, ceph=True, maintainer=Guillaume Abrioux , GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.build-date=20240319, org.label-schema.schema-version=1.0, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5) 2024-03-21T10:37:35.083 INFO:journalctl@ceph.osd.4.smithi100.stdout:Mar 21 10:37:34 smithi100 bash[71290]: ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-4 2024-03-21T10:37:35.391 INFO:journalctl@ceph.osd.4.smithi100.stdout:Mar 21 10:37:35 smithi100 podman[71310]: 2024-03-21 10:37:35.085271889 +0000 UTC m=+0.704377018 container remove e1dd02fadaabd39fe53e4b0411213b16e115f37ffc3c87e5373f3d20dd7b2d79 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-4, ceph=True, org.label-schema.build-date=20240319, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD) 2024-03-21T10:37:35.391 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 10:37:35 smithi100 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-10[59767]: 2024-03-21T10:37:35.178+0000 7f315da00700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-03-21T10:36:52.673512+0000 front 2024-03-21T10:36:52.673688+0000 (oldest deadline 2024-03-21T10:37:16.675039+0000) 2024-03-21T10:37:35.391 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 10:37:35 smithi100 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-10[59767]: 2024-03-21T10:37:35.178+0000 7f315da00700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-03-21T10:37:05.379887+0000 front 2024-03-21T10:37:05.379536+0000 (oldest deadline 2024-03-21T10:37:30.079809+0000) 2024-03-21T10:37:35.460 INFO:journalctl@ceph.osd.6.smithi047.stdout:Mar 21 10:37:35 smithi047 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-6[57372]: 2024-03-21T10:37:35.111+0000 7f157e200700 -1 osd.6 111 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-03-21T10:37:03.517239+0000 front 2024-03-21T10:37:03.517159+0000 (oldest deadline 2024-03-21T10:37:26.417262+0000) 2024-03-21T10:37:35.486 INFO:journalctl@ceph.osd.5.smithi157.stdout:Mar 21 10:37:35 smithi157 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-5[47667]: 2024-03-21T10:37:35.085+0000 7f242fa00700 -1 osd.5 111 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-03-21T10:36:55.338628+0000 front 2024-03-21T10:36:55.339174+0000 (oldest deadline 2024-03-21T10:37:19.438029+0000) 2024-03-21T10:37:35.486 INFO:journalctl@ceph.osd.5.smithi157.stdout:Mar 21 10:37:35 smithi157 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-5[47667]: 2024-03-21T10:37:35.085+0000 7f242fa00700 -1 osd.5 111 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-03-21T10:37:04.140579+0000 front 2024-03-21T10:37:04.140414+0000 (oldest deadline 2024-03-21T10:37:31.741045+0000) 2024-03-21T10:37:35.726 INFO:journalctl@ceph.osd.7.smithi100.stdout:Mar 21 10:37:35 smithi100 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-7[53119]: 2024-03-21T10:37:35.527+0000 7fb6ed400700 -1 osd.7 111 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-03-21T10:37:04.136587+0000 front 2024-03-21T10:37:04.136839+0000 (oldest deadline 2024-03-21T10:37:29.436935+0000) 2024-03-21T10:37:35.961 INFO:journalctl@ceph.osd.9.smithi047.stdout:Mar 21 10:37:35 smithi047 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-9[64137]: 2024-03-21T10:37:35.562+0000 7f02dc200700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-03-21T10:36:54.991613+0000 front 2024-03-21T10:36:54.991458+0000 (oldest deadline 2024-03-21T10:37:17.891147+0000) 2024-03-21T10:37:35.961 INFO:journalctl@ceph.osd.9.smithi047.stdout:Mar 21 10:37:35 smithi047 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-9[64137]: 2024-03-21T10:37:35.562+0000 7f02dc200700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-03-21T10:37:02.494627+0000 front 2024-03-21T10:37:02.494473+0000 (oldest deadline 2024-03-21T10:37:26.594304+0000) 2024-03-21T10:37:35.986 INFO:journalctl@ceph.osd.8.smithi157.stdout:Mar 21 10:37:35 smithi157 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-8[54391]: 2024-03-21T10:37:35.810+0000 7fca85e00700 -1 osd.8 111 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-03-21T10:37:03.032692+0000 front 2024-03-21T10:37:03.032837+0000 (oldest deadline 2024-03-21T10:37:27.132936+0000) 2024-03-21T10:37:35.986 INFO:journalctl@ceph.osd.11.smithi157.stdout:Mar 21 10:37:35 smithi157 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-11[61259]: 2024-03-21T10:37:35.692+0000 7fd0f8400700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-03-21T10:36:54.398353+0000 front 2024-03-21T10:36:54.398068+0000 (oldest deadline 2024-03-21T10:37:16.698280+0000) 2024-03-21T10:37:35.986 INFO:journalctl@ceph.osd.11.smithi157.stdout:Mar 21 10:37:35 smithi157 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-11[61259]: 2024-03-21T10:37:35.692+0000 7fd0f8400700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-03-21T10:37:04.901925+0000 front 2024-03-21T10:37:04.902083+0000 (oldest deadline 2024-03-21T10:37:29.502349+0000) 2024-03-21T10:37:36.213 INFO:journalctl@ceph.osd.4.smithi100.stdout:Mar 21 10:37:35 smithi100 podman[71405]: 2024-03-21 10:37:35.805263868 +0000 UTC m=+0.031761774 image pull 237e43c7a65ff48534d94b97c1bfdbe4b7be5672e3d9a613745083ea3702aa54 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232 2024-03-21T10:37:36.213 INFO:journalctl@ceph.osd.4.smithi100.stdout:Mar 21 10:37:35 smithi100 podman[71405]: 2024-03-21 10:37:35.927896819 +0000 UTC m=+0.154394761 container create e327db29f0d01c14d70c56c6f155afb54b67ef003254a69a62432afb57ed3222 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-4-deactivate, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240319, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_CLEAN=True, maintainer=Guillaume Abrioux ) 2024-03-21T10:37:36.460 INFO:journalctl@ceph.osd.6.smithi047.stdout:Mar 21 10:37:36 smithi047 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-6[57372]: 2024-03-21T10:37:36.076+0000 7f157e200700 -1 osd.6 111 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-03-21T10:37:03.517239+0000 front 2024-03-21T10:37:03.517159+0000 (oldest deadline 2024-03-21T10:37:26.417262+0000) 2024-03-21T10:37:36.476 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 10:37:36 smithi100 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-10[59767]: 2024-03-21T10:37:36.220+0000 7f315da00700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-03-21T10:36:52.673512+0000 front 2024-03-21T10:36:52.673688+0000 (oldest deadline 2024-03-21T10:37:16.675039+0000) 2024-03-21T10:37:36.477 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 10:37:36 smithi100 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-10[59767]: 2024-03-21T10:37:36.221+0000 7f315da00700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-03-21T10:37:05.379887+0000 front 2024-03-21T10:37:05.379536+0000 (oldest deadline 2024-03-21T10:37:30.079809+0000) 2024-03-21T10:37:36.477 INFO:journalctl@ceph.osd.4.smithi100.stdout:Mar 21 10:37:36 smithi100 podman[71405]: 2024-03-21 10:37:36.214835511 +0000 UTC m=+0.441333473 container init e327db29f0d01c14d70c56c6f155afb54b67ef003254a69a62432afb57ed3222 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-4-deactivate, maintainer=Guillaume Abrioux , GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240319, GIT_BRANCH=HEAD) 2024-03-21T10:37:36.477 INFO:journalctl@ceph.osd.4.smithi100.stdout:Mar 21 10:37:36 smithi100 podman[71405]: 2024-03-21 10:37:36.226320197 +0000 UTC m=+0.452818146 container start e327db29f0d01c14d70c56c6f155afb54b67ef003254a69a62432afb57ed3222 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-4-deactivate, org.label-schema.license=GPLv2, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240319, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD) 2024-03-21T10:37:36.477 INFO:journalctl@ceph.osd.4.smithi100.stdout:Mar 21 10:37:36 smithi100 podman[71405]: 2024-03-21 10:37:36.330269031 +0000 UTC m=+0.556766974 container attach e327db29f0d01c14d70c56c6f155afb54b67ef003254a69a62432afb57ed3222 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-4-deactivate, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True, org.label-schema.build-date=20240319, GIT_BRANCH=HEAD) 2024-03-21T10:37:36.486 INFO:journalctl@ceph.osd.5.smithi157.stdout:Mar 21 10:37:36 smithi157 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-5[47667]: 2024-03-21T10:37:36.105+0000 7f242fa00700 -1 osd.5 111 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-03-21T10:36:55.338628+0000 front 2024-03-21T10:36:55.339174+0000 (oldest deadline 2024-03-21T10:37:19.438029+0000) 2024-03-21T10:37:36.486 INFO:journalctl@ceph.osd.5.smithi157.stdout:Mar 21 10:37:36 smithi157 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-5[47667]: 2024-03-21T10:37:36.105+0000 7f242fa00700 -1 osd.5 111 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-03-21T10:37:04.140579+0000 front 2024-03-21T10:37:04.140414+0000 (oldest deadline 2024-03-21T10:37:31.741045+0000) 2024-03-21T10:37:36.961 INFO:journalctl@ceph.osd.9.smithi047.stdout:Mar 21 10:37:36 smithi047 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-9[64137]: 2024-03-21T10:37:36.567+0000 7f02dc200700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-03-21T10:36:54.991613+0000 front 2024-03-21T10:36:54.991458+0000 (oldest deadline 2024-03-21T10:37:17.891147+0000) 2024-03-21T10:37:36.961 INFO:journalctl@ceph.osd.9.smithi047.stdout:Mar 21 10:37:36 smithi047 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-9[64137]: 2024-03-21T10:37:36.567+0000 7f02dc200700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-03-21T10:37:02.494627+0000 front 2024-03-21T10:37:02.494473+0000 (oldest deadline 2024-03-21T10:37:26.594304+0000) 2024-03-21T10:37:36.968 INFO:journalctl@ceph.osd.7.smithi100.stdout:Mar 21 10:37:36 smithi100 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-7[53119]: 2024-03-21T10:37:36.552+0000 7fb6ed400700 -1 osd.7 111 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-03-21T10:37:04.136587+0000 front 2024-03-21T10:37:04.136839+0000 (oldest deadline 2024-03-21T10:37:29.436935+0000) 2024-03-21T10:37:36.968 INFO:journalctl@ceph.osd.4.smithi100.stdout:Mar 21 10:37:36 smithi100 podman[71405]: 2024-03-21 10:37:36.653515376 +0000 UTC m=+0.880013348 container died e327db29f0d01c14d70c56c6f155afb54b67ef003254a69a62432afb57ed3222 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-4-deactivate, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240319, GIT_CLEAN=True) 2024-03-21T10:37:36.986 INFO:journalctl@ceph.osd.8.smithi157.stdout:Mar 21 10:37:36 smithi157 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-8[54391]: 2024-03-21T10:37:36.847+0000 7fca85e00700 -1 osd.8 111 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-03-21T10:37:03.032692+0000 front 2024-03-21T10:37:03.032837+0000 (oldest deadline 2024-03-21T10:37:27.132936+0000) 2024-03-21T10:37:36.986 INFO:journalctl@ceph.osd.11.smithi157.stdout:Mar 21 10:37:36 smithi157 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-11[61259]: 2024-03-21T10:37:36.661+0000 7fd0f8400700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-03-21T10:36:54.398353+0000 front 2024-03-21T10:36:54.398068+0000 (oldest deadline 2024-03-21T10:37:16.698280+0000) 2024-03-21T10:37:36.986 INFO:journalctl@ceph.osd.11.smithi157.stdout:Mar 21 10:37:36 smithi157 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-11[61259]: 2024-03-21T10:37:36.661+0000 7fd0f8400700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-03-21T10:37:04.901925+0000 front 2024-03-21T10:37:04.902083+0000 (oldest deadline 2024-03-21T10:37:29.502349+0000) 2024-03-21T10:37:37.226 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 10:37:37 smithi100 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-10[59767]: 2024-03-21T10:37:37.194+0000 7f315da00700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-03-21T10:36:52.673512+0000 front 2024-03-21T10:36:52.673688+0000 (oldest deadline 2024-03-21T10:37:16.675039+0000) 2024-03-21T10:37:37.226 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 10:37:37 smithi100 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-10[59767]: 2024-03-21T10:37:37.194+0000 7f315da00700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-03-21T10:37:05.379887+0000 front 2024-03-21T10:37:05.379536+0000 (oldest deadline 2024-03-21T10:37:30.079809+0000) 2024-03-21T10:37:37.424 DEBUG:teuthology.orchestra.run.smithi100:> sudo pkill -f 'journalctl -f -n 0 -u ceph-3056d51a-e76b-11ee-95cd-87774f69a715@osd.4.service' 2024-03-21T10:37:37.460 INFO:journalctl@ceph.osd.6.smithi047.stdout:Mar 21 10:37:37 smithi047 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-6[57372]: 2024-03-21T10:37:37.099+0000 7f157e200700 -1 osd.6 111 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-03-21T10:37:03.517239+0000 front 2024-03-21T10:37:03.517159+0000 (oldest deadline 2024-03-21T10:37:26.417262+0000) 2024-03-21T10:37:37.486 INFO:journalctl@ceph.osd.5.smithi157.stdout:Mar 21 10:37:37 smithi157 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-5[47667]: 2024-03-21T10:37:37.096+0000 7f242fa00700 -1 osd.5 111 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-03-21T10:36:55.338628+0000 front 2024-03-21T10:36:55.339174+0000 (oldest deadline 2024-03-21T10:37:19.438029+0000) 2024-03-21T10:37:37.486 INFO:journalctl@ceph.osd.5.smithi157.stdout:Mar 21 10:37:37 smithi157 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-5[47667]: 2024-03-21T10:37:37.096+0000 7f242fa00700 -1 osd.5 111 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-03-21T10:37:04.140579+0000 front 2024-03-21T10:37:04.140414+0000 (oldest deadline 2024-03-21T10:37:31.741045+0000) 2024-03-21T10:37:37.546 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-21T10:37:37.547 INFO:tasks.cephadm.osd.4:Stopped osd.4 2024-03-21T10:37:37.547 INFO:tasks.cephadm.osd.5:Stopping osd.5... 2024-03-21T10:37:37.547 DEBUG:teuthology.orchestra.run.smithi157:> sudo systemctl stop ceph-3056d51a-e76b-11ee-95cd-87774f69a715@osd.5 2024-03-21T10:37:37.726 INFO:journalctl@ceph.osd.7.smithi100.stdout:Mar 21 10:37:37 smithi100 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-7[53119]: 2024-03-21T10:37:37.536+0000 7fb6ed400700 -1 osd.7 111 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-03-21T10:37:04.136587+0000 front 2024-03-21T10:37:04.136839+0000 (oldest deadline 2024-03-21T10:37:29.436935+0000) 2024-03-21T10:37:37.960 INFO:journalctl@ceph.osd.9.smithi047.stdout:Mar 21 10:37:37 smithi047 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-9[64137]: 2024-03-21T10:37:37.532+0000 7f02dc200700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-03-21T10:36:54.991613+0000 front 2024-03-21T10:36:54.991458+0000 (oldest deadline 2024-03-21T10:37:17.891147+0000) 2024-03-21T10:37:37.961 INFO:journalctl@ceph.osd.9.smithi047.stdout:Mar 21 10:37:37 smithi047 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-9[64137]: 2024-03-21T10:37:37.532+0000 7f02dc200700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-03-21T10:37:02.494627+0000 front 2024-03-21T10:37:02.494473+0000 (oldest deadline 2024-03-21T10:37:26.594304+0000) 2024-03-21T10:37:37.986 INFO:journalctl@ceph.osd.5.smithi157.stdout:Mar 21 10:37:37 smithi157 systemd[1]: Stopping Ceph osd.5 for 3056d51a-e76b-11ee-95cd-87774f69a715... 2024-03-21T10:37:37.986 INFO:journalctl@ceph.osd.8.smithi157.stdout:Mar 21 10:37:37 smithi157 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-8[54391]: 2024-03-21T10:37:37.875+0000 7fca85e00700 -1 osd.8 111 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-03-21T10:37:03.032692+0000 front 2024-03-21T10:37:03.032837+0000 (oldest deadline 2024-03-21T10:37:27.132936+0000) 2024-03-21T10:37:37.987 INFO:journalctl@ceph.osd.11.smithi157.stdout:Mar 21 10:37:37 smithi157 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-11[61259]: 2024-03-21T10:37:37.645+0000 7fd0f8400700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-03-21T10:36:54.398353+0000 front 2024-03-21T10:36:54.398068+0000 (oldest deadline 2024-03-21T10:37:16.698280+0000) 2024-03-21T10:37:37.987 INFO:journalctl@ceph.osd.11.smithi157.stdout:Mar 21 10:37:37 smithi157 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-11[61259]: 2024-03-21T10:37:37.645+0000 7fd0f8400700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-03-21T10:37:04.901925+0000 front 2024-03-21T10:37:04.902083+0000 (oldest deadline 2024-03-21T10:37:29.502349+0000) 2024-03-21T10:37:37.987 INFO:journalctl@ceph.osd.11.smithi157.stdout:Mar 21 10:37:37 smithi157 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-11[61259]: 2024-03-21T10:37:37.645+0000 7fd0f8400700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-03-21T10:37:11.804109+0000 front 2024-03-21T10:37:11.804464+0000 (oldest deadline 2024-03-21T10:37:37.104077+0000) 2024-03-21T10:37:38.423 INFO:journalctl@ceph.osd.6.smithi047.stdout:Mar 21 10:37:38 smithi047 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-6[57372]: 2024-03-21T10:37:38.136+0000 7f157e200700 -1 osd.6 111 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-03-21T10:37:03.517239+0000 front 2024-03-21T10:37:03.517159+0000 (oldest deadline 2024-03-21T10:37:26.417262+0000) 2024-03-21T10:37:38.476 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 10:37:38 smithi100 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-10[59767]: 2024-03-21T10:37:38.193+0000 7f315da00700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-03-21T10:36:52.673512+0000 front 2024-03-21T10:36:52.673688+0000 (oldest deadline 2024-03-21T10:37:16.675039+0000) 2024-03-21T10:37:38.476 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 10:37:38 smithi100 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-10[59767]: 2024-03-21T10:37:38.193+0000 7f315da00700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-03-21T10:37:05.379887+0000 front 2024-03-21T10:37:05.379536+0000 (oldest deadline 2024-03-21T10:37:30.079809+0000) 2024-03-21T10:37:38.486 INFO:journalctl@ceph.osd.5.smithi157.stdout:Mar 21 10:37:38 smithi157 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-5[47667]: 2024-03-21T10:37:38.117+0000 7f2439200700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.5 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-21T10:37:38.487 INFO:journalctl@ceph.osd.5.smithi157.stdout:Mar 21 10:37:38 smithi157 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-5[47667]: 2024-03-21T10:37:38.117+0000 7f2439200700 -1 osd.5 111 *** Got signal Terminated *** 2024-03-21T10:37:38.487 INFO:journalctl@ceph.osd.5.smithi157.stdout:Mar 21 10:37:38 smithi157 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-5[47667]: 2024-03-21T10:37:38.117+0000 7f2439200700 -1 osd.5 111 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-21T10:37:38.487 INFO:journalctl@ceph.osd.5.smithi157.stdout:Mar 21 10:37:38 smithi157 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-5[47667]: 2024-03-21T10:37:38.128+0000 7f242fa00700 -1 osd.5 111 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-03-21T10:36:55.338628+0000 front 2024-03-21T10:36:55.339174+0000 (oldest deadline 2024-03-21T10:37:19.438029+0000) 2024-03-21T10:37:38.487 INFO:journalctl@ceph.osd.5.smithi157.stdout:Mar 21 10:37:38 smithi157 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-5[47667]: 2024-03-21T10:37:38.128+0000 7f242fa00700 -1 osd.5 111 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-03-21T10:37:04.140579+0000 front 2024-03-21T10:37:04.140414+0000 (oldest deadline 2024-03-21T10:37:31.741045+0000) 2024-03-21T10:37:38.487 INFO:journalctl@ceph.osd.5.smithi157.stdout:Mar 21 10:37:38 smithi157 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-5[47667]: 2024-03-21T10:37:38.128+0000 7f242fa00700 -1 osd.5 111 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-03-21T10:37:11.742161+0000 front 2024-03-21T10:37:11.742253+0000 (oldest deadline 2024-03-21T10:37:37.642357+0000) 2024-03-21T10:37:38.710 INFO:journalctl@ceph.osd.9.smithi047.stdout:Mar 21 10:37:38 smithi047 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-9[64137]: 2024-03-21T10:37:38.521+0000 7f02dc200700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-03-21T10:36:54.991613+0000 front 2024-03-21T10:36:54.991458+0000 (oldest deadline 2024-03-21T10:37:17.891147+0000) 2024-03-21T10:37:38.711 INFO:journalctl@ceph.osd.9.smithi047.stdout:Mar 21 10:37:38 smithi047 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-9[64137]: 2024-03-21T10:37:38.521+0000 7f02dc200700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-03-21T10:37:02.494627+0000 front 2024-03-21T10:37:02.494473+0000 (oldest deadline 2024-03-21T10:37:26.594304+0000) 2024-03-21T10:37:38.711 INFO:journalctl@ceph.osd.9.smithi047.stdout:Mar 21 10:37:38 smithi047 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-9[64137]: 2024-03-21T10:37:38.521+0000 7f02dc200700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-03-21T10:37:13.597260+0000 front 2024-03-21T10:37:13.596840+0000 (oldest deadline 2024-03-21T10:37:37.696959+0000) 2024-03-21T10:37:38.899 INFO:journalctl@ceph.osd.11.smithi157.stdout:Mar 21 10:37:38 smithi157 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-11[61259]: 2024-03-21T10:37:38.605+0000 7fd0f8400700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-03-21T10:36:54.398353+0000 front 2024-03-21T10:36:54.398068+0000 (oldest deadline 2024-03-21T10:37:16.698280+0000) 2024-03-21T10:37:38.899 INFO:journalctl@ceph.osd.11.smithi157.stdout:Mar 21 10:37:38 smithi157 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-11[61259]: 2024-03-21T10:37:38.605+0000 7fd0f8400700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-03-21T10:37:04.901925+0000 front 2024-03-21T10:37:04.902083+0000 (oldest deadline 2024-03-21T10:37:29.502349+0000) 2024-03-21T10:37:38.899 INFO:journalctl@ceph.osd.11.smithi157.stdout:Mar 21 10:37:38 smithi157 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-11[61259]: 2024-03-21T10:37:38.605+0000 7fd0f8400700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-03-21T10:37:11.804109+0000 front 2024-03-21T10:37:11.804464+0000 (oldest deadline 2024-03-21T10:37:37.104077+0000) 2024-03-21T10:37:38.976 INFO:journalctl@ceph.osd.7.smithi100.stdout:Mar 21 10:37:38 smithi100 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-7[53119]: 2024-03-21T10:37:38.493+0000 7fb6ed400700 -1 osd.7 111 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-03-21T10:37:04.136587+0000 front 2024-03-21T10:37:04.136839+0000 (oldest deadline 2024-03-21T10:37:29.436935+0000) 2024-03-21T10:37:38.976 INFO:journalctl@ceph.osd.7.smithi100.stdout:Mar 21 10:37:38 smithi100 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-7[53119]: 2024-03-21T10:37:38.493+0000 7fb6ed400700 -1 osd.7 111 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-03-21T10:37:14.139283+0000 front 2024-03-21T10:37:14.139243+0000 (oldest deadline 2024-03-21T10:37:37.639505+0000) 2024-03-21T10:37:39.236 INFO:journalctl@ceph.osd.8.smithi157.stdout:Mar 21 10:37:38 smithi157 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-8[54391]: 2024-03-21T10:37:38.901+0000 7fca85e00700 -1 osd.8 111 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-03-21T10:37:03.032692+0000 front 2024-03-21T10:37:03.032837+0000 (oldest deadline 2024-03-21T10:37:27.132936+0000) 2024-03-21T10:37:39.236 INFO:journalctl@ceph.osd.5.smithi157.stdout:Mar 21 10:37:39 smithi157 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-5[47667]: 2024-03-21T10:37:39.119+0000 7f242fa00700 -1 osd.5 111 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-03-21T10:36:55.338628+0000 front 2024-03-21T10:36:55.339174+0000 (oldest deadline 2024-03-21T10:37:19.438029+0000) 2024-03-21T10:37:39.236 INFO:journalctl@ceph.osd.5.smithi157.stdout:Mar 21 10:37:39 smithi157 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-5[47667]: 2024-03-21T10:37:39.119+0000 7f242fa00700 -1 osd.5 111 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-03-21T10:37:04.140579+0000 front 2024-03-21T10:37:04.140414+0000 (oldest deadline 2024-03-21T10:37:31.741045+0000) 2024-03-21T10:37:39.237 INFO:journalctl@ceph.osd.5.smithi157.stdout:Mar 21 10:37:39 smithi157 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-5[47667]: 2024-03-21T10:37:39.119+0000 7f242fa00700 -1 osd.5 111 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-03-21T10:37:11.742161+0000 front 2024-03-21T10:37:11.742253+0000 (oldest deadline 2024-03-21T10:37:37.642357+0000) 2024-03-21T10:37:39.460 INFO:journalctl@ceph.osd.6.smithi047.stdout:Mar 21 10:37:39 smithi047 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-6[57372]: 2024-03-21T10:37:39.133+0000 7f157e200700 -1 osd.6 111 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-03-21T10:37:03.517239+0000 front 2024-03-21T10:37:03.517159+0000 (oldest deadline 2024-03-21T10:37:26.417262+0000) 2024-03-21T10:37:39.461 INFO:journalctl@ceph.osd.6.smithi047.stdout:Mar 21 10:37:39 smithi047 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-6[57372]: 2024-03-21T10:37:39.133+0000 7f157e200700 -1 osd.6 111 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-03-21T10:37:12.722409+0000 front 2024-03-21T10:37:12.722673+0000 (oldest deadline 2024-03-21T10:37:38.622523+0000) 2024-03-21T10:37:39.475 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 10:37:39 smithi100 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-10[59767]: 2024-03-21T10:37:39.173+0000 7f315da00700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-03-21T10:36:52.673512+0000 front 2024-03-21T10:36:52.673688+0000 (oldest deadline 2024-03-21T10:37:16.675039+0000) 2024-03-21T10:37:39.475 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 10:37:39 smithi100 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-10[59767]: 2024-03-21T10:37:39.173+0000 7f315da00700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-03-21T10:37:05.379887+0000 front 2024-03-21T10:37:05.379536+0000 (oldest deadline 2024-03-21T10:37:30.079809+0000) 2024-03-21T10:37:39.475 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 10:37:39 smithi100 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-10[59767]: 2024-03-21T10:37:39.173+0000 7f315da00700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-03-21T10:37:12.981593+0000 front 2024-03-21T10:37:12.981483+0000 (oldest deadline 2024-03-21T10:37:38.281183+0000) 2024-03-21T10:37:39.726 INFO:journalctl@ceph.osd.7.smithi100.stdout:Mar 21 10:37:39 smithi100 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-7[53119]: 2024-03-21T10:37:39.476+0000 7fb6ed400700 -1 osd.7 111 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-03-21T10:37:04.136587+0000 front 2024-03-21T10:37:04.136839+0000 (oldest deadline 2024-03-21T10:37:29.436935+0000) 2024-03-21T10:37:39.726 INFO:journalctl@ceph.osd.7.smithi100.stdout:Mar 21 10:37:39 smithi100 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-7[53119]: 2024-03-21T10:37:39.476+0000 7fb6ed400700 -1 osd.7 111 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-03-21T10:37:14.139283+0000 front 2024-03-21T10:37:14.139243+0000 (oldest deadline 2024-03-21T10:37:37.639505+0000) 2024-03-21T10:37:39.960 INFO:journalctl@ceph.osd.9.smithi047.stdout:Mar 21 10:37:39 smithi047 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-9[64137]: 2024-03-21T10:37:39.517+0000 7f02dc200700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-03-21T10:36:54.991613+0000 front 2024-03-21T10:36:54.991458+0000 (oldest deadline 2024-03-21T10:37:17.891147+0000) 2024-03-21T10:37:39.961 INFO:journalctl@ceph.osd.9.smithi047.stdout:Mar 21 10:37:39 smithi047 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-9[64137]: 2024-03-21T10:37:39.517+0000 7f02dc200700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-03-21T10:37:02.494627+0000 front 2024-03-21T10:37:02.494473+0000 (oldest deadline 2024-03-21T10:37:26.594304+0000) 2024-03-21T10:37:39.961 INFO:journalctl@ceph.osd.9.smithi047.stdout:Mar 21 10:37:39 smithi047 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-9[64137]: 2024-03-21T10:37:39.517+0000 7f02dc200700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-03-21T10:37:13.597260+0000 front 2024-03-21T10:37:13.596840+0000 (oldest deadline 2024-03-21T10:37:37.696959+0000) 2024-03-21T10:37:39.986 INFO:journalctl@ceph.osd.8.smithi157.stdout:Mar 21 10:37:39 smithi157 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-8[54391]: 2024-03-21T10:37:39.896+0000 7fca85e00700 -1 osd.8 111 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-03-21T10:37:03.032692+0000 front 2024-03-21T10:37:03.032837+0000 (oldest deadline 2024-03-21T10:37:27.132936+0000) 2024-03-21T10:37:39.986 INFO:journalctl@ceph.osd.8.smithi157.stdout:Mar 21 10:37:39 smithi157 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-8[54391]: 2024-03-21T10:37:39.896+0000 7fca85e00700 -1 osd.8 111 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-03-21T10:37:13.535744+0000 front 2024-03-21T10:37:13.536199+0000 (oldest deadline 2024-03-21T10:37:39.435978+0000) 2024-03-21T10:37:39.987 INFO:journalctl@ceph.osd.11.smithi157.stdout:Mar 21 10:37:39 smithi157 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-11[61259]: 2024-03-21T10:37:39.649+0000 7fd0f8400700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-03-21T10:36:54.398353+0000 front 2024-03-21T10:36:54.398068+0000 (oldest deadline 2024-03-21T10:37:16.698280+0000) 2024-03-21T10:37:39.987 INFO:journalctl@ceph.osd.11.smithi157.stdout:Mar 21 10:37:39 smithi157 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-11[61259]: 2024-03-21T10:37:39.649+0000 7fd0f8400700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-03-21T10:37:04.901925+0000 front 2024-03-21T10:37:04.902083+0000 (oldest deadline 2024-03-21T10:37:29.502349+0000) 2024-03-21T10:37:39.987 INFO:journalctl@ceph.osd.11.smithi157.stdout:Mar 21 10:37:39 smithi157 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-11[61259]: 2024-03-21T10:37:39.649+0000 7fd0f8400700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-03-21T10:37:11.804109+0000 front 2024-03-21T10:37:11.804464+0000 (oldest deadline 2024-03-21T10:37:37.104077+0000) 2024-03-21T10:37:40.460 INFO:journalctl@ceph.osd.6.smithi047.stdout:Mar 21 10:37:40 smithi047 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-6[57372]: 2024-03-21T10:37:40.168+0000 7f157e200700 -1 osd.6 111 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-03-21T10:37:03.517239+0000 front 2024-03-21T10:37:03.517159+0000 (oldest deadline 2024-03-21T10:37:26.417262+0000) 2024-03-21T10:37:40.460 INFO:journalctl@ceph.osd.6.smithi047.stdout:Mar 21 10:37:40 smithi047 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-6[57372]: 2024-03-21T10:37:40.168+0000 7f157e200700 -1 osd.6 111 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-03-21T10:37:12.722409+0000 front 2024-03-21T10:37:12.722673+0000 (oldest deadline 2024-03-21T10:37:38.622523+0000) 2024-03-21T10:37:40.466 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 10:37:40 smithi100 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-10[59767]: 2024-03-21T10:37:40.197+0000 7f315da00700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-03-21T10:36:52.673512+0000 front 2024-03-21T10:36:52.673688+0000 (oldest deadline 2024-03-21T10:37:16.675039+0000) 2024-03-21T10:37:40.467 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 10:37:40 smithi100 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-10[59767]: 2024-03-21T10:37:40.197+0000 7f315da00700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-03-21T10:37:05.379887+0000 front 2024-03-21T10:37:05.379536+0000 (oldest deadline 2024-03-21T10:37:30.079809+0000) 2024-03-21T10:37:40.467 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 10:37:40 smithi100 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-10[59767]: 2024-03-21T10:37:40.197+0000 7f315da00700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-03-21T10:37:12.981593+0000 front 2024-03-21T10:37:12.981483+0000 (oldest deadline 2024-03-21T10:37:38.281183+0000) 2024-03-21T10:37:40.486 INFO:journalctl@ceph.osd.5.smithi157.stdout:Mar 21 10:37:40 smithi157 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-5[47667]: 2024-03-21T10:37:40.154+0000 7f242fa00700 -1 osd.5 111 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-03-21T10:36:55.338628+0000 front 2024-03-21T10:36:55.339174+0000 (oldest deadline 2024-03-21T10:37:19.438029+0000) 2024-03-21T10:37:40.486 INFO:journalctl@ceph.osd.5.smithi157.stdout:Mar 21 10:37:40 smithi157 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-5[47667]: 2024-03-21T10:37:40.154+0000 7f242fa00700 -1 osd.5 111 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-03-21T10:37:04.140579+0000 front 2024-03-21T10:37:04.140414+0000 (oldest deadline 2024-03-21T10:37:31.741045+0000) 2024-03-21T10:37:40.486 INFO:journalctl@ceph.osd.5.smithi157.stdout:Mar 21 10:37:40 smithi157 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-5[47667]: 2024-03-21T10:37:40.154+0000 7f242fa00700 -1 osd.5 111 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-03-21T10:37:11.742161+0000 front 2024-03-21T10:37:11.742253+0000 (oldest deadline 2024-03-21T10:37:37.642357+0000) 2024-03-21T10:37:40.726 INFO:journalctl@ceph.osd.7.smithi100.stdout:Mar 21 10:37:40 smithi100 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-7[53119]: 2024-03-21T10:37:40.468+0000 7fb6ed400700 -1 osd.7 111 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-03-21T10:37:04.136587+0000 front 2024-03-21T10:37:04.136839+0000 (oldest deadline 2024-03-21T10:37:29.436935+0000) 2024-03-21T10:37:40.726 INFO:journalctl@ceph.osd.7.smithi100.stdout:Mar 21 10:37:40 smithi100 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-7[53119]: 2024-03-21T10:37:40.468+0000 7fb6ed400700 -1 osd.7 111 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-03-21T10:37:14.139283+0000 front 2024-03-21T10:37:14.139243+0000 (oldest deadline 2024-03-21T10:37:37.639505+0000) 2024-03-21T10:37:40.960 INFO:journalctl@ceph.osd.9.smithi047.stdout:Mar 21 10:37:40 smithi047 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-9[64137]: 2024-03-21T10:37:40.471+0000 7f02dc200700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-03-21T10:36:54.991613+0000 front 2024-03-21T10:36:54.991458+0000 (oldest deadline 2024-03-21T10:37:17.891147+0000) 2024-03-21T10:37:40.961 INFO:journalctl@ceph.osd.9.smithi047.stdout:Mar 21 10:37:40 smithi047 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-9[64137]: 2024-03-21T10:37:40.471+0000 7f02dc200700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-03-21T10:37:02.494627+0000 front 2024-03-21T10:37:02.494473+0000 (oldest deadline 2024-03-21T10:37:26.594304+0000) 2024-03-21T10:37:40.961 INFO:journalctl@ceph.osd.9.smithi047.stdout:Mar 21 10:37:40 smithi047 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-9[64137]: 2024-03-21T10:37:40.471+0000 7f02dc200700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-03-21T10:37:13.597260+0000 front 2024-03-21T10:37:13.596840+0000 (oldest deadline 2024-03-21T10:37:37.696959+0000) 2024-03-21T10:37:40.986 INFO:journalctl@ceph.osd.8.smithi157.stdout:Mar 21 10:37:40 smithi157 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-8[54391]: 2024-03-21T10:37:40.858+0000 7fca85e00700 -1 osd.8 111 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-03-21T10:37:03.032692+0000 front 2024-03-21T10:37:03.032837+0000 (oldest deadline 2024-03-21T10:37:27.132936+0000) 2024-03-21T10:37:40.986 INFO:journalctl@ceph.osd.8.smithi157.stdout:Mar 21 10:37:40 smithi157 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-8[54391]: 2024-03-21T10:37:40.858+0000 7fca85e00700 -1 osd.8 111 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-03-21T10:37:13.535744+0000 front 2024-03-21T10:37:13.536199+0000 (oldest deadline 2024-03-21T10:37:39.435978+0000) 2024-03-21T10:37:40.987 INFO:journalctl@ceph.osd.11.smithi157.stdout:Mar 21 10:37:40 smithi157 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-11[61259]: 2024-03-21T10:37:40.650+0000 7fd0f8400700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-03-21T10:36:54.398353+0000 front 2024-03-21T10:36:54.398068+0000 (oldest deadline 2024-03-21T10:37:16.698280+0000) 2024-03-21T10:37:40.987 INFO:journalctl@ceph.osd.11.smithi157.stdout:Mar 21 10:37:40 smithi157 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-11[61259]: 2024-03-21T10:37:40.650+0000 7fd0f8400700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-03-21T10:37:04.901925+0000 front 2024-03-21T10:37:04.902083+0000 (oldest deadline 2024-03-21T10:37:29.502349+0000) 2024-03-21T10:37:40.987 INFO:journalctl@ceph.osd.11.smithi157.stdout:Mar 21 10:37:40 smithi157 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-11[61259]: 2024-03-21T10:37:40.650+0000 7fd0f8400700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-03-21T10:37:11.804109+0000 front 2024-03-21T10:37:11.804464+0000 (oldest deadline 2024-03-21T10:37:37.104077+0000) 2024-03-21T10:37:41.445 INFO:journalctl@ceph.osd.6.smithi047.stdout:Mar 21 10:37:41 smithi047 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-6[57372]: 2024-03-21T10:37:41.181+0000 7f157e200700 -1 osd.6 111 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-03-21T10:37:03.517239+0000 front 2024-03-21T10:37:03.517159+0000 (oldest deadline 2024-03-21T10:37:26.417262+0000) 2024-03-21T10:37:41.445 INFO:journalctl@ceph.osd.6.smithi047.stdout:Mar 21 10:37:41 smithi047 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-6[57372]: 2024-03-21T10:37:41.181+0000 7f157e200700 -1 osd.6 111 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-03-21T10:37:12.722409+0000 front 2024-03-21T10:37:12.722673+0000 (oldest deadline 2024-03-21T10:37:38.622523+0000) 2024-03-21T10:37:41.456 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 10:37:41 smithi100 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-10[59767]: 2024-03-21T10:37:41.149+0000 7f315da00700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-03-21T10:36:52.673512+0000 front 2024-03-21T10:36:52.673688+0000 (oldest deadline 2024-03-21T10:37:16.675039+0000) 2024-03-21T10:37:41.456 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 10:37:41 smithi100 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-10[59767]: 2024-03-21T10:37:41.150+0000 7f315da00700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-03-21T10:37:05.379887+0000 front 2024-03-21T10:37:05.379536+0000 (oldest deadline 2024-03-21T10:37:30.079809+0000) 2024-03-21T10:37:41.457 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 10:37:41 smithi100 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-10[59767]: 2024-03-21T10:37:41.150+0000 7f315da00700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-03-21T10:37:12.981593+0000 front 2024-03-21T10:37:12.981483+0000 (oldest deadline 2024-03-21T10:37:38.281183+0000) 2024-03-21T10:37:41.486 INFO:journalctl@ceph.osd.5.smithi157.stdout:Mar 21 10:37:41 smithi157 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-5[47667]: 2024-03-21T10:37:41.140+0000 7f242fa00700 -1 osd.5 111 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-03-21T10:36:55.338628+0000 front 2024-03-21T10:36:55.339174+0000 (oldest deadline 2024-03-21T10:37:19.438029+0000) 2024-03-21T10:37:41.486 INFO:journalctl@ceph.osd.5.smithi157.stdout:Mar 21 10:37:41 smithi157 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-5[47667]: 2024-03-21T10:37:41.140+0000 7f242fa00700 -1 osd.5 111 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-03-21T10:37:04.140579+0000 front 2024-03-21T10:37:04.140414+0000 (oldest deadline 2024-03-21T10:37:31.741045+0000) 2024-03-21T10:37:41.486 INFO:journalctl@ceph.osd.5.smithi157.stdout:Mar 21 10:37:41 smithi157 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-5[47667]: 2024-03-21T10:37:41.140+0000 7f242fa00700 -1 osd.5 111 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-03-21T10:37:11.742161+0000 front 2024-03-21T10:37:11.742253+0000 (oldest deadline 2024-03-21T10:37:37.642357+0000) 2024-03-21T10:37:41.710 INFO:journalctl@ceph.osd.9.smithi047.stdout:Mar 21 10:37:41 smithi047 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-9[64137]: 2024-03-21T10:37:41.444+0000 7f02dc200700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-03-21T10:36:54.991613+0000 front 2024-03-21T10:36:54.991458+0000 (oldest deadline 2024-03-21T10:37:17.891147+0000) 2024-03-21T10:37:41.710 INFO:journalctl@ceph.osd.9.smithi047.stdout:Mar 21 10:37:41 smithi047 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-9[64137]: 2024-03-21T10:37:41.444+0000 7f02dc200700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-03-21T10:37:02.494627+0000 front 2024-03-21T10:37:02.494473+0000 (oldest deadline 2024-03-21T10:37:26.594304+0000) 2024-03-21T10:37:41.711 INFO:journalctl@ceph.osd.9.smithi047.stdout:Mar 21 10:37:41 smithi047 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-9[64137]: 2024-03-21T10:37:41.444+0000 7f02dc200700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-03-21T10:37:13.597260+0000 front 2024-03-21T10:37:13.596840+0000 (oldest deadline 2024-03-21T10:37:37.696959+0000) 2024-03-21T10:37:41.726 INFO:journalctl@ceph.osd.7.smithi100.stdout:Mar 21 10:37:41 smithi100 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-7[53119]: 2024-03-21T10:37:41.458+0000 7fb6ed400700 -1 osd.7 111 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-03-21T10:37:04.136587+0000 front 2024-03-21T10:37:04.136839+0000 (oldest deadline 2024-03-21T10:37:29.436935+0000) 2024-03-21T10:37:41.726 INFO:journalctl@ceph.osd.7.smithi100.stdout:Mar 21 10:37:41 smithi100 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-7[53119]: 2024-03-21T10:37:41.458+0000 7fb6ed400700 -1 osd.7 111 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-03-21T10:37:14.139283+0000 front 2024-03-21T10:37:14.139243+0000 (oldest deadline 2024-03-21T10:37:37.639505+0000) 2024-03-21T10:37:41.986 INFO:journalctl@ceph.osd.8.smithi157.stdout:Mar 21 10:37:41 smithi157 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-8[54391]: 2024-03-21T10:37:41.814+0000 7fca85e00700 -1 osd.8 111 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-03-21T10:37:03.032692+0000 front 2024-03-21T10:37:03.032837+0000 (oldest deadline 2024-03-21T10:37:27.132936+0000) 2024-03-21T10:37:41.987 INFO:journalctl@ceph.osd.8.smithi157.stdout:Mar 21 10:37:41 smithi157 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-8[54391]: 2024-03-21T10:37:41.814+0000 7fca85e00700 -1 osd.8 111 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-03-21T10:37:13.535744+0000 front 2024-03-21T10:37:13.536199+0000 (oldest deadline 2024-03-21T10:37:39.435978+0000) 2024-03-21T10:37:41.987 INFO:journalctl@ceph.osd.11.smithi157.stdout:Mar 21 10:37:41 smithi157 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-11[61259]: 2024-03-21T10:37:41.644+0000 7fd0f8400700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-03-21T10:36:54.398353+0000 front 2024-03-21T10:36:54.398068+0000 (oldest deadline 2024-03-21T10:37:16.698280+0000) 2024-03-21T10:37:41.987 INFO:journalctl@ceph.osd.11.smithi157.stdout:Mar 21 10:37:41 smithi157 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-11[61259]: 2024-03-21T10:37:41.644+0000 7fd0f8400700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-03-21T10:37:04.901925+0000 front 2024-03-21T10:37:04.902083+0000 (oldest deadline 2024-03-21T10:37:29.502349+0000) 2024-03-21T10:37:41.987 INFO:journalctl@ceph.osd.11.smithi157.stdout:Mar 21 10:37:41 smithi157 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-11[61259]: 2024-03-21T10:37:41.644+0000 7fd0f8400700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-03-21T10:37:11.804109+0000 front 2024-03-21T10:37:11.804464+0000 (oldest deadline 2024-03-21T10:37:37.104077+0000) 2024-03-21T10:37:42.438 INFO:journalctl@ceph.osd.6.smithi047.stdout:Mar 21 10:37:42 smithi047 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-6[57372]: 2024-03-21T10:37:42.180+0000 7f157e200700 -1 osd.6 111 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-03-21T10:37:03.517239+0000 front 2024-03-21T10:37:03.517159+0000 (oldest deadline 2024-03-21T10:37:26.417262+0000) 2024-03-21T10:37:42.438 INFO:journalctl@ceph.osd.6.smithi047.stdout:Mar 21 10:37:42 smithi047 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-6[57372]: 2024-03-21T10:37:42.180+0000 7f157e200700 -1 osd.6 111 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-03-21T10:37:12.722409+0000 front 2024-03-21T10:37:12.722673+0000 (oldest deadline 2024-03-21T10:37:38.622523+0000) 2024-03-21T10:37:42.441 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 10:37:42 smithi100 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-10[59767]: 2024-03-21T10:37:42.179+0000 7f315da00700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-03-21T10:36:52.673512+0000 front 2024-03-21T10:36:52.673688+0000 (oldest deadline 2024-03-21T10:37:16.675039+0000) 2024-03-21T10:37:42.442 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 10:37:42 smithi100 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-10[59767]: 2024-03-21T10:37:42.179+0000 7f315da00700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-03-21T10:37:05.379887+0000 front 2024-03-21T10:37:05.379536+0000 (oldest deadline 2024-03-21T10:37:30.079809+0000) 2024-03-21T10:37:42.442 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 10:37:42 smithi100 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-10[59767]: 2024-03-21T10:37:42.179+0000 7f315da00700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-03-21T10:37:12.981593+0000 front 2024-03-21T10:37:12.981483+0000 (oldest deadline 2024-03-21T10:37:38.281183+0000) 2024-03-21T10:37:42.486 INFO:journalctl@ceph.osd.5.smithi157.stdout:Mar 21 10:37:42 smithi157 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-5[47667]: 2024-03-21T10:37:42.142+0000 7f242fa00700 -1 osd.5 111 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-03-21T10:36:55.338628+0000 front 2024-03-21T10:36:55.339174+0000 (oldest deadline 2024-03-21T10:37:19.438029+0000) 2024-03-21T10:37:42.487 INFO:journalctl@ceph.osd.5.smithi157.stdout:Mar 21 10:37:42 smithi157 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-5[47667]: 2024-03-21T10:37:42.142+0000 7f242fa00700 -1 osd.5 111 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-03-21T10:37:04.140579+0000 front 2024-03-21T10:37:04.140414+0000 (oldest deadline 2024-03-21T10:37:31.741045+0000) 2024-03-21T10:37:42.487 INFO:journalctl@ceph.osd.5.smithi157.stdout:Mar 21 10:37:42 smithi157 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-5[47667]: 2024-03-21T10:37:42.142+0000 7f242fa00700 -1 osd.5 111 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-03-21T10:37:11.742161+0000 front 2024-03-21T10:37:11.742253+0000 (oldest deadline 2024-03-21T10:37:37.642357+0000) 2024-03-21T10:37:42.710 INFO:journalctl@ceph.osd.9.smithi047.stdout:Mar 21 10:37:42 smithi047 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-9[64137]: 2024-03-21T10:37:42.439+0000 7f02dc200700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-03-21T10:36:54.991613+0000 front 2024-03-21T10:36:54.991458+0000 (oldest deadline 2024-03-21T10:37:17.891147+0000) 2024-03-21T10:37:42.711 INFO:journalctl@ceph.osd.9.smithi047.stdout:Mar 21 10:37:42 smithi047 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-9[64137]: 2024-03-21T10:37:42.439+0000 7f02dc200700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-03-21T10:37:02.494627+0000 front 2024-03-21T10:37:02.494473+0000 (oldest deadline 2024-03-21T10:37:26.594304+0000) 2024-03-21T10:37:42.711 INFO:journalctl@ceph.osd.9.smithi047.stdout:Mar 21 10:37:42 smithi047 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-9[64137]: 2024-03-21T10:37:42.439+0000 7f02dc200700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-03-21T10:37:13.597260+0000 front 2024-03-21T10:37:13.596840+0000 (oldest deadline 2024-03-21T10:37:37.696959+0000) 2024-03-21T10:37:42.726 INFO:journalctl@ceph.osd.7.smithi100.stdout:Mar 21 10:37:42 smithi100 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-7[53119]: 2024-03-21T10:37:42.443+0000 7fb6ed400700 -1 osd.7 111 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-03-21T10:37:04.136587+0000 front 2024-03-21T10:37:04.136839+0000 (oldest deadline 2024-03-21T10:37:29.436935+0000) 2024-03-21T10:37:42.726 INFO:journalctl@ceph.osd.7.smithi100.stdout:Mar 21 10:37:42 smithi100 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-7[53119]: 2024-03-21T10:37:42.443+0000 7fb6ed400700 -1 osd.7 111 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-03-21T10:37:14.139283+0000 front 2024-03-21T10:37:14.139243+0000 (oldest deadline 2024-03-21T10:37:37.639505+0000) 2024-03-21T10:37:42.986 INFO:journalctl@ceph.osd.8.smithi157.stdout:Mar 21 10:37:42 smithi157 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-8[54391]: 2024-03-21T10:37:42.816+0000 7fca85e00700 -1 osd.8 111 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-03-21T10:37:03.032692+0000 front 2024-03-21T10:37:03.032837+0000 (oldest deadline 2024-03-21T10:37:27.132936+0000) 2024-03-21T10:37:42.986 INFO:journalctl@ceph.osd.8.smithi157.stdout:Mar 21 10:37:42 smithi157 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-8[54391]: 2024-03-21T10:37:42.816+0000 7fca85e00700 -1 osd.8 111 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-03-21T10:37:13.535744+0000 front 2024-03-21T10:37:13.536199+0000 (oldest deadline 2024-03-21T10:37:39.435978+0000) 2024-03-21T10:37:42.987 INFO:journalctl@ceph.osd.11.smithi157.stdout:Mar 21 10:37:42 smithi157 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-11[61259]: 2024-03-21T10:37:42.621+0000 7fd0f8400700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-03-21T10:36:54.398353+0000 front 2024-03-21T10:36:54.398068+0000 (oldest deadline 2024-03-21T10:37:16.698280+0000) 2024-03-21T10:37:42.987 INFO:journalctl@ceph.osd.11.smithi157.stdout:Mar 21 10:37:42 smithi157 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-11[61259]: 2024-03-21T10:37:42.621+0000 7fd0f8400700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-03-21T10:37:04.901925+0000 front 2024-03-21T10:37:04.902083+0000 (oldest deadline 2024-03-21T10:37:29.502349+0000) 2024-03-21T10:37:42.987 INFO:journalctl@ceph.osd.11.smithi157.stdout:Mar 21 10:37:42 smithi157 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-11[61259]: 2024-03-21T10:37:42.621+0000 7fd0f8400700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-03-21T10:37:11.804109+0000 front 2024-03-21T10:37:11.804464+0000 (oldest deadline 2024-03-21T10:37:37.104077+0000) 2024-03-21T10:37:43.455 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 10:37:43 smithi100 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-10[59767]: 2024-03-21T10:37:43.133+0000 7f315da00700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-03-21T10:36:52.673512+0000 front 2024-03-21T10:36:52.673688+0000 (oldest deadline 2024-03-21T10:37:16.675039+0000) 2024-03-21T10:37:43.455 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 10:37:43 smithi100 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-10[59767]: 2024-03-21T10:37:43.133+0000 7f315da00700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-03-21T10:37:05.379887+0000 front 2024-03-21T10:37:05.379536+0000 (oldest deadline 2024-03-21T10:37:30.079809+0000) 2024-03-21T10:37:43.455 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 10:37:43 smithi100 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-10[59767]: 2024-03-21T10:37:43.133+0000 7f315da00700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-03-21T10:37:12.981593+0000 front 2024-03-21T10:37:12.981483+0000 (oldest deadline 2024-03-21T10:37:38.281183+0000) 2024-03-21T10:37:43.460 INFO:journalctl@ceph.osd.6.smithi047.stdout:Mar 21 10:37:43 smithi047 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-6[57372]: 2024-03-21T10:37:43.190+0000 7f157e200700 -1 osd.6 111 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-03-21T10:37:03.517239+0000 front 2024-03-21T10:37:03.517159+0000 (oldest deadline 2024-03-21T10:37:26.417262+0000) 2024-03-21T10:37:43.460 INFO:journalctl@ceph.osd.6.smithi047.stdout:Mar 21 10:37:43 smithi047 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-6[57372]: 2024-03-21T10:37:43.190+0000 7f157e200700 -1 osd.6 111 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-03-21T10:37:12.722409+0000 front 2024-03-21T10:37:12.722673+0000 (oldest deadline 2024-03-21T10:37:38.622523+0000) 2024-03-21T10:37:43.726 INFO:journalctl@ceph.osd.7.smithi100.stdout:Mar 21 10:37:43 smithi100 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-7[53119]: 2024-03-21T10:37:43.456+0000 7fb6ed400700 -1 osd.7 111 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-03-21T10:37:04.136587+0000 front 2024-03-21T10:37:04.136839+0000 (oldest deadline 2024-03-21T10:37:29.436935+0000) 2024-03-21T10:37:43.726 INFO:journalctl@ceph.osd.7.smithi100.stdout:Mar 21 10:37:43 smithi100 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-7[53119]: 2024-03-21T10:37:43.456+0000 7fb6ed400700 -1 osd.7 111 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-03-21T10:37:14.139283+0000 front 2024-03-21T10:37:14.139243+0000 (oldest deadline 2024-03-21T10:37:37.639505+0000) 2024-03-21T10:37:43.841 INFO:journalctl@ceph.osd.5.smithi157.stdout:Mar 21 10:37:43 smithi157 podman[73334]: 2024-03-21 10:37:43.79446001 +0000 UTC m=+5.810939502 container died a4e2eea4f685ca42235c7a4f49e8a04408d174e1ec6308caf6c53a2fad284773 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-5, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, io.buildah.version=1.33.5, org.label-schema.build-date=20240319, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_BRANCH=HEAD) 2024-03-21T10:37:43.841 INFO:journalctl@ceph.osd.11.smithi157.stdout:Mar 21 10:37:43 smithi157 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-11[61259]: 2024-03-21T10:37:43.587+0000 7fd0f8400700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-03-21T10:36:54.398353+0000 front 2024-03-21T10:36:54.398068+0000 (oldest deadline 2024-03-21T10:37:16.698280+0000) 2024-03-21T10:37:43.842 INFO:journalctl@ceph.osd.11.smithi157.stdout:Mar 21 10:37:43 smithi157 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-11[61259]: 2024-03-21T10:37:43.587+0000 7fd0f8400700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-03-21T10:37:04.901925+0000 front 2024-03-21T10:37:04.902083+0000 (oldest deadline 2024-03-21T10:37:29.502349+0000) 2024-03-21T10:37:43.842 INFO:journalctl@ceph.osd.11.smithi157.stdout:Mar 21 10:37:43 smithi157 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-11[61259]: 2024-03-21T10:37:43.587+0000 7fd0f8400700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-03-21T10:37:11.804109+0000 front 2024-03-21T10:37:11.804464+0000 (oldest deadline 2024-03-21T10:37:37.104077+0000) 2024-03-21T10:37:43.960 INFO:journalctl@ceph.osd.9.smithi047.stdout:Mar 21 10:37:43 smithi047 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-9[64137]: 2024-03-21T10:37:43.469+0000 7f02dc200700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-03-21T10:36:54.991613+0000 front 2024-03-21T10:36:54.991458+0000 (oldest deadline 2024-03-21T10:37:17.891147+0000) 2024-03-21T10:37:43.961 INFO:journalctl@ceph.osd.9.smithi047.stdout:Mar 21 10:37:43 smithi047 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-9[64137]: 2024-03-21T10:37:43.469+0000 7f02dc200700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-03-21T10:37:02.494627+0000 front 2024-03-21T10:37:02.494473+0000 (oldest deadline 2024-03-21T10:37:26.594304+0000) 2024-03-21T10:37:43.961 INFO:journalctl@ceph.osd.9.smithi047.stdout:Mar 21 10:37:43 smithi047 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-9[64137]: 2024-03-21T10:37:43.469+0000 7f02dc200700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-03-21T10:37:13.597260+0000 front 2024-03-21T10:37:13.596840+0000 (oldest deadline 2024-03-21T10:37:37.696959+0000) 2024-03-21T10:37:44.098 INFO:journalctl@ceph.osd.8.smithi157.stdout:Mar 21 10:37:43 smithi157 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-8[54391]: 2024-03-21T10:37:43.843+0000 7fca85e00700 -1 osd.8 111 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-03-21T10:37:03.032692+0000 front 2024-03-21T10:37:03.032837+0000 (oldest deadline 2024-03-21T10:37:27.132936+0000) 2024-03-21T10:37:44.098 INFO:journalctl@ceph.osd.8.smithi157.stdout:Mar 21 10:37:43 smithi157 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-8[54391]: 2024-03-21T10:37:43.843+0000 7fca85e00700 -1 osd.8 111 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-03-21T10:37:13.535744+0000 front 2024-03-21T10:37:13.536199+0000 (oldest deadline 2024-03-21T10:37:39.435978+0000) 2024-03-21T10:37:44.412 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 10:37:44 smithi100 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-10[59767]: 2024-03-21T10:37:44.127+0000 7f315da00700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-03-21T10:36:52.673512+0000 front 2024-03-21T10:36:52.673688+0000 (oldest deadline 2024-03-21T10:37:16.675039+0000) 2024-03-21T10:37:44.412 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 10:37:44 smithi100 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-10[59767]: 2024-03-21T10:37:44.127+0000 7f315da00700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-03-21T10:37:05.379887+0000 front 2024-03-21T10:37:05.379536+0000 (oldest deadline 2024-03-21T10:37:30.079809+0000) 2024-03-21T10:37:44.412 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 10:37:44 smithi100 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-10[59767]: 2024-03-21T10:37:44.127+0000 7f315da00700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-03-21T10:37:12.981593+0000 front 2024-03-21T10:37:12.981483+0000 (oldest deadline 2024-03-21T10:37:38.281183+0000) 2024-03-21T10:37:44.460 INFO:journalctl@ceph.osd.6.smithi047.stdout:Mar 21 10:37:44 smithi047 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-6[57372]: 2024-03-21T10:37:44.191+0000 7f157e200700 -1 osd.6 111 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-03-21T10:37:03.517239+0000 front 2024-03-21T10:37:03.517159+0000 (oldest deadline 2024-03-21T10:37:26.417262+0000) 2024-03-21T10:37:44.461 INFO:journalctl@ceph.osd.6.smithi047.stdout:Mar 21 10:37:44 smithi047 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-6[57372]: 2024-03-21T10:37:44.191+0000 7f157e200700 -1 osd.6 111 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-03-21T10:37:12.722409+0000 front 2024-03-21T10:37:12.722673+0000 (oldest deadline 2024-03-21T10:37:38.622523+0000) 2024-03-21T10:37:44.486 INFO:journalctl@ceph.osd.5.smithi157.stdout:Mar 21 10:37:44 smithi157 podman[73334]: 2024-03-21 10:37:44.251861328 +0000 UTC m=+6.268340854 container cleanup a4e2eea4f685ca42235c7a4f49e8a04408d174e1ec6308caf6c53a2fad284773 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-5, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240319, GIT_CLEAN=True, ceph=True, maintainer=Guillaume Abrioux , GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, io.buildah.version=1.33.5) 2024-03-21T10:37:44.486 INFO:journalctl@ceph.osd.5.smithi157.stdout:Mar 21 10:37:44 smithi157 bash[73334]: ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-5 2024-03-21T10:37:44.726 INFO:journalctl@ceph.osd.7.smithi100.stdout:Mar 21 10:37:44 smithi100 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-7[53119]: 2024-03-21T10:37:44.413+0000 7fb6ed400700 -1 osd.7 111 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-03-21T10:37:04.136587+0000 front 2024-03-21T10:37:04.136839+0000 (oldest deadline 2024-03-21T10:37:29.436935+0000) 2024-03-21T10:37:44.726 INFO:journalctl@ceph.osd.7.smithi100.stdout:Mar 21 10:37:44 smithi100 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-7[53119]: 2024-03-21T10:37:44.413+0000 7fb6ed400700 -1 osd.7 111 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-03-21T10:37:14.139283+0000 front 2024-03-21T10:37:14.139243+0000 (oldest deadline 2024-03-21T10:37:37.639505+0000) 2024-03-21T10:37:44.886 INFO:journalctl@ceph.osd.5.smithi157.stdout:Mar 21 10:37:44 smithi157 podman[73355]: 2024-03-21 10:37:44.49474596 +0000 UTC m=+0.688567570 container remove a4e2eea4f685ca42235c7a4f49e8a04408d174e1ec6308caf6c53a2fad284773 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.build-date=20240319, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, ceph=True, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD) 2024-03-21T10:37:44.887 INFO:journalctl@ceph.osd.11.smithi157.stdout:Mar 21 10:37:44 smithi157 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-11[61259]: 2024-03-21T10:37:44.602+0000 7fd0f8400700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-03-21T10:36:54.398353+0000 front 2024-03-21T10:36:54.398068+0000 (oldest deadline 2024-03-21T10:37:16.698280+0000) 2024-03-21T10:37:44.887 INFO:journalctl@ceph.osd.11.smithi157.stdout:Mar 21 10:37:44 smithi157 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-11[61259]: 2024-03-21T10:37:44.602+0000 7fd0f8400700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-03-21T10:37:04.901925+0000 front 2024-03-21T10:37:04.902083+0000 (oldest deadline 2024-03-21T10:37:29.502349+0000) 2024-03-21T10:37:44.887 INFO:journalctl@ceph.osd.11.smithi157.stdout:Mar 21 10:37:44 smithi157 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-11[61259]: 2024-03-21T10:37:44.602+0000 7fd0f8400700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-03-21T10:37:11.804109+0000 front 2024-03-21T10:37:11.804464+0000 (oldest deadline 2024-03-21T10:37:37.104077+0000) 2024-03-21T10:37:44.960 INFO:journalctl@ceph.osd.9.smithi047.stdout:Mar 21 10:37:44 smithi047 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-9[64137]: 2024-03-21T10:37:44.476+0000 7f02dc200700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-03-21T10:36:54.991613+0000 front 2024-03-21T10:36:54.991458+0000 (oldest deadline 2024-03-21T10:37:17.891147+0000) 2024-03-21T10:37:44.961 INFO:journalctl@ceph.osd.9.smithi047.stdout:Mar 21 10:37:44 smithi047 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-9[64137]: 2024-03-21T10:37:44.476+0000 7f02dc200700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-03-21T10:37:02.494627+0000 front 2024-03-21T10:37:02.494473+0000 (oldest deadline 2024-03-21T10:37:26.594304+0000) 2024-03-21T10:37:44.961 INFO:journalctl@ceph.osd.9.smithi047.stdout:Mar 21 10:37:44 smithi047 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-9[64137]: 2024-03-21T10:37:44.476+0000 7f02dc200700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-03-21T10:37:13.597260+0000 front 2024-03-21T10:37:13.596840+0000 (oldest deadline 2024-03-21T10:37:37.696959+0000) 2024-03-21T10:37:45.236 INFO:journalctl@ceph.osd.8.smithi157.stdout:Mar 21 10:37:44 smithi157 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-8[54391]: 2024-03-21T10:37:44.888+0000 7fca85e00700 -1 osd.8 111 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-03-21T10:37:03.032692+0000 front 2024-03-21T10:37:03.032837+0000 (oldest deadline 2024-03-21T10:37:27.132936+0000) 2024-03-21T10:37:45.236 INFO:journalctl@ceph.osd.8.smithi157.stdout:Mar 21 10:37:44 smithi157 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-8[54391]: 2024-03-21T10:37:44.888+0000 7fca85e00700 -1 osd.8 111 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-03-21T10:37:13.535744+0000 front 2024-03-21T10:37:13.536199+0000 (oldest deadline 2024-03-21T10:37:39.435978+0000) 2024-03-21T10:37:45.441 INFO:journalctl@ceph.osd.6.smithi047.stdout:Mar 21 10:37:45 smithi047 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-6[57372]: 2024-03-21T10:37:45.164+0000 7f157e200700 -1 osd.6 111 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-03-21T10:37:03.517239+0000 front 2024-03-21T10:37:03.517159+0000 (oldest deadline 2024-03-21T10:37:26.417262+0000) 2024-03-21T10:37:45.441 INFO:journalctl@ceph.osd.6.smithi047.stdout:Mar 21 10:37:45 smithi047 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-6[57372]: 2024-03-21T10:37:45.164+0000 7f157e200700 -1 osd.6 111 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-03-21T10:37:12.722409+0000 front 2024-03-21T10:37:12.722673+0000 (oldest deadline 2024-03-21T10:37:38.622523+0000) 2024-03-21T10:37:45.476 INFO:journalctl@ceph.osd.7.smithi100.stdout:Mar 21 10:37:45 smithi100 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-7[53119]: 2024-03-21T10:37:45.387+0000 7fb6ed400700 -1 osd.7 111 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-03-21T10:37:04.136587+0000 front 2024-03-21T10:37:04.136839+0000 (oldest deadline 2024-03-21T10:37:29.436935+0000) 2024-03-21T10:37:45.476 INFO:journalctl@ceph.osd.7.smithi100.stdout:Mar 21 10:37:45 smithi100 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-7[53119]: 2024-03-21T10:37:45.387+0000 7fb6ed400700 -1 osd.7 111 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-03-21T10:37:14.139283+0000 front 2024-03-21T10:37:14.139243+0000 (oldest deadline 2024-03-21T10:37:37.639505+0000) 2024-03-21T10:37:45.477 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 10:37:45 smithi100 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-10[59767]: 2024-03-21T10:37:45.158+0000 7f315da00700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-03-21T10:36:52.673512+0000 front 2024-03-21T10:36:52.673688+0000 (oldest deadline 2024-03-21T10:37:16.675039+0000) 2024-03-21T10:37:45.477 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 10:37:45 smithi100 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-10[59767]: 2024-03-21T10:37:45.158+0000 7f315da00700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-03-21T10:37:05.379887+0000 front 2024-03-21T10:37:05.379536+0000 (oldest deadline 2024-03-21T10:37:30.079809+0000) 2024-03-21T10:37:45.477 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 10:37:45 smithi100 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-10[59767]: 2024-03-21T10:37:45.158+0000 7f315da00700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-03-21T10:37:12.981593+0000 front 2024-03-21T10:37:12.981483+0000 (oldest deadline 2024-03-21T10:37:38.281183+0000) 2024-03-21T10:37:45.582 INFO:journalctl@ceph.osd.5.smithi157.stdout:Mar 21 10:37:45 smithi157 podman[73442]: 2024-03-21 10:37:45.197710202 +0000 UTC m=+0.032789766 image pull 237e43c7a65ff48534d94b97c1bfdbe4b7be5672e3d9a613745083ea3702aa54 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232 2024-03-21T10:37:45.582 INFO:journalctl@ceph.osd.5.smithi157.stdout:Mar 21 10:37:45 smithi157 podman[73442]: 2024-03-21 10:37:45.321468231 +0000 UTC m=+0.156547828 container create 3f7d3c64a2a6174bfd7a086893397d92f99ff478dfa42ecdf4cad43c574b736f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-5-deactivate, GIT_CLEAN=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.build-date=20240319, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , ceph=True) 2024-03-21T10:37:45.582 INFO:journalctl@ceph.osd.5.smithi157.stdout:Mar 21 10:37:45 smithi157 podman[73442]: 2024-03-21 10:37:45.583527829 +0000 UTC m=+0.418607432 container init 3f7d3c64a2a6174bfd7a086893397d92f99ff478dfa42ecdf4cad43c574b736f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-5-deactivate, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.build-date=20240319, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, ceph=True) 2024-03-21T10:37:45.710 INFO:journalctl@ceph.osd.9.smithi047.stdout:Mar 21 10:37:45 smithi047 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-9[64137]: 2024-03-21T10:37:45.443+0000 7f02dc200700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-03-21T10:36:54.991613+0000 front 2024-03-21T10:36:54.991458+0000 (oldest deadline 2024-03-21T10:37:17.891147+0000) 2024-03-21T10:37:45.711 INFO:journalctl@ceph.osd.9.smithi047.stdout:Mar 21 10:37:45 smithi047 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-9[64137]: 2024-03-21T10:37:45.443+0000 7f02dc200700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-03-21T10:37:02.494627+0000 front 2024-03-21T10:37:02.494473+0000 (oldest deadline 2024-03-21T10:37:26.594304+0000) 2024-03-21T10:37:45.711 INFO:journalctl@ceph.osd.9.smithi047.stdout:Mar 21 10:37:45 smithi047 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-9[64137]: 2024-03-21T10:37:45.443+0000 7f02dc200700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-03-21T10:37:13.597260+0000 front 2024-03-21T10:37:13.596840+0000 (oldest deadline 2024-03-21T10:37:37.696959+0000) 2024-03-21T10:37:45.711 INFO:journalctl@ceph.osd.9.smithi047.stdout:Mar 21 10:37:45 smithi047 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-9[64137]: 2024-03-21T10:37:45.443+0000 7f02dc200700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.47:6814 osd.3 since back 2024-03-21T10:37:21.198652+0000 front 2024-03-21T10:37:21.199080+0000 (oldest deadline 2024-03-21T10:37:45.200271+0000) 2024-03-21T10:37:45.912 INFO:journalctl@ceph.osd.5.smithi157.stdout:Mar 21 10:37:45 smithi157 podman[73442]: 2024-03-21 10:37:45.595432413 +0000 UTC m=+0.430512020 container start 3f7d3c64a2a6174bfd7a086893397d92f99ff478dfa42ecdf4cad43c574b736f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-5-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.build-date=20240319, org.label-schema.schema-version=1.0, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True) 2024-03-21T10:37:45.912 INFO:journalctl@ceph.osd.5.smithi157.stdout:Mar 21 10:37:45 smithi157 podman[73442]: 2024-03-21 10:37:45.690780318 +0000 UTC m=+0.525859920 container attach 3f7d3c64a2a6174bfd7a086893397d92f99ff478dfa42ecdf4cad43c574b736f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-5-deactivate, org.label-schema.build-date=20240319, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, GIT_CLEAN=True) 2024-03-21T10:37:45.912 INFO:journalctl@ceph.osd.11.smithi157.stdout:Mar 21 10:37:45 smithi157 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-11[61259]: 2024-03-21T10:37:45.626+0000 7fd0f8400700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-03-21T10:36:54.398353+0000 front 2024-03-21T10:36:54.398068+0000 (oldest deadline 2024-03-21T10:37:16.698280+0000) 2024-03-21T10:37:45.912 INFO:journalctl@ceph.osd.11.smithi157.stdout:Mar 21 10:37:45 smithi157 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-11[61259]: 2024-03-21T10:37:45.626+0000 7fd0f8400700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-03-21T10:37:04.901925+0000 front 2024-03-21T10:37:04.902083+0000 (oldest deadline 2024-03-21T10:37:29.502349+0000) 2024-03-21T10:37:45.912 INFO:journalctl@ceph.osd.11.smithi157.stdout:Mar 21 10:37:45 smithi157 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-11[61259]: 2024-03-21T10:37:45.626+0000 7fd0f8400700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-03-21T10:37:11.804109+0000 front 2024-03-21T10:37:11.804464+0000 (oldest deadline 2024-03-21T10:37:37.104077+0000) 2024-03-21T10:37:46.237 INFO:journalctl@ceph.osd.5.smithi157.stdout:Mar 21 10:37:46 smithi157 podman[73442]: 2024-03-21 10:37:46.025810067 +0000 UTC m=+0.860889664 container died 3f7d3c64a2a6174bfd7a086893397d92f99ff478dfa42ecdf4cad43c574b736f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-5-deactivate, io.buildah.version=1.33.5, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.license=GPLv2, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.schema-version=1.0, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, org.label-schema.build-date=20240319) 2024-03-21T10:37:46.237 INFO:journalctl@ceph.osd.8.smithi157.stdout:Mar 21 10:37:45 smithi157 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-8[54391]: 2024-03-21T10:37:45.913+0000 7fca85e00700 -1 osd.8 111 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-03-21T10:37:03.032692+0000 front 2024-03-21T10:37:03.032837+0000 (oldest deadline 2024-03-21T10:37:27.132936+0000) 2024-03-21T10:37:46.237 INFO:journalctl@ceph.osd.8.smithi157.stdout:Mar 21 10:37:45 smithi157 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-8[54391]: 2024-03-21T10:37:45.913+0000 7fca85e00700 -1 osd.8 111 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-03-21T10:37:13.535744+0000 front 2024-03-21T10:37:13.536199+0000 (oldest deadline 2024-03-21T10:37:39.435978+0000) 2024-03-21T10:37:46.463 INFO:journalctl@ceph.osd.6.smithi047.stdout:Mar 21 10:37:46 smithi047 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-6[57372]: 2024-03-21T10:37:46.214+0000 7f157e200700 -1 osd.6 111 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-03-21T10:37:03.517239+0000 front 2024-03-21T10:37:03.517159+0000 (oldest deadline 2024-03-21T10:37:26.417262+0000) 2024-03-21T10:37:46.463 INFO:journalctl@ceph.osd.6.smithi047.stdout:Mar 21 10:37:46 smithi047 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-6[57372]: 2024-03-21T10:37:46.214+0000 7f157e200700 -1 osd.6 111 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-03-21T10:37:12.722409+0000 front 2024-03-21T10:37:12.722673+0000 (oldest deadline 2024-03-21T10:37:38.622523+0000) 2024-03-21T10:37:46.476 INFO:journalctl@ceph.osd.7.smithi100.stdout:Mar 21 10:37:46 smithi100 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-7[53119]: 2024-03-21T10:37:46.357+0000 7fb6ed400700 -1 osd.7 111 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-03-21T10:37:04.136587+0000 front 2024-03-21T10:37:04.136839+0000 (oldest deadline 2024-03-21T10:37:29.436935+0000) 2024-03-21T10:37:46.476 INFO:journalctl@ceph.osd.7.smithi100.stdout:Mar 21 10:37:46 smithi100 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-7[53119]: 2024-03-21T10:37:46.357+0000 7fb6ed400700 -1 osd.7 111 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-03-21T10:37:14.139283+0000 front 2024-03-21T10:37:14.139243+0000 (oldest deadline 2024-03-21T10:37:37.639505+0000) 2024-03-21T10:37:46.476 INFO:journalctl@ceph.osd.7.smithi100.stdout:Mar 21 10:37:46 smithi100 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-7[53119]: 2024-03-21T10:37:46.357+0000 7fb6ed400700 -1 osd.7 111 heartbeat_check: no reply from 172.21.15.47:6814 osd.3 since back 2024-03-21T10:37:22.243333+0000 front 2024-03-21T10:37:22.243357+0000 (oldest deadline 2024-03-21T10:37:45.742507+0000) 2024-03-21T10:37:46.477 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 10:37:46 smithi100 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-10[59767]: 2024-03-21T10:37:46.176+0000 7f315da00700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-03-21T10:36:52.673512+0000 front 2024-03-21T10:36:52.673688+0000 (oldest deadline 2024-03-21T10:37:16.675039+0000) 2024-03-21T10:37:46.477 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 10:37:46 smithi100 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-10[59767]: 2024-03-21T10:37:46.176+0000 7f315da00700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-03-21T10:37:05.379887+0000 front 2024-03-21T10:37:05.379536+0000 (oldest deadline 2024-03-21T10:37:30.079809+0000) 2024-03-21T10:37:46.477 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 10:37:46 smithi100 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-10[59767]: 2024-03-21T10:37:46.176+0000 7f315da00700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-03-21T10:37:12.981593+0000 front 2024-03-21T10:37:12.981483+0000 (oldest deadline 2024-03-21T10:37:38.281183+0000) 2024-03-21T10:37:46.477 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 10:37:46 smithi100 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-10[59767]: 2024-03-21T10:37:46.176+0000 7f315da00700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.47:6814 osd.3 since back 2024-03-21T10:37:23.583085+0000 front 2024-03-21T10:37:23.583243+0000 (oldest deadline 2024-03-21T10:37:45.283261+0000) 2024-03-21T10:37:46.702 INFO:journalctl@ceph.osd.11.smithi157.stdout:Mar 21 10:37:46 smithi157 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-11[61259]: 2024-03-21T10:37:46.628+0000 7fd0f8400700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-03-21T10:36:54.398353+0000 front 2024-03-21T10:36:54.398068+0000 (oldest deadline 2024-03-21T10:37:16.698280+0000) 2024-03-21T10:37:46.702 INFO:journalctl@ceph.osd.11.smithi157.stdout:Mar 21 10:37:46 smithi157 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-11[61259]: 2024-03-21T10:37:46.628+0000 7fd0f8400700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-03-21T10:37:04.901925+0000 front 2024-03-21T10:37:04.902083+0000 (oldest deadline 2024-03-21T10:37:29.502349+0000) 2024-03-21T10:37:46.702 INFO:journalctl@ceph.osd.11.smithi157.stdout:Mar 21 10:37:46 smithi157 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-11[61259]: 2024-03-21T10:37:46.628+0000 7fd0f8400700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-03-21T10:37:11.804109+0000 front 2024-03-21T10:37:11.804464+0000 (oldest deadline 2024-03-21T10:37:37.104077+0000) 2024-03-21T10:37:46.782 DEBUG:teuthology.orchestra.run.smithi157:> sudo pkill -f 'journalctl -f -n 0 -u ceph-3056d51a-e76b-11ee-95cd-87774f69a715@osd.5.service' 2024-03-21T10:37:46.878 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-21T10:37:46.879 INFO:tasks.cephadm.osd.5:Stopped osd.5 2024-03-21T10:37:46.879 INFO:tasks.cephadm.osd.6:Stopping osd.6... 2024-03-21T10:37:46.879 DEBUG:teuthology.orchestra.run.smithi047:> sudo systemctl stop ceph-3056d51a-e76b-11ee-95cd-87774f69a715@osd.6 2024-03-21T10:37:46.960 INFO:journalctl@ceph.osd.9.smithi047.stdout:Mar 21 10:37:46 smithi047 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-9[64137]: 2024-03-21T10:37:46.464+0000 7f02dc200700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-03-21T10:36:54.991613+0000 front 2024-03-21T10:36:54.991458+0000 (oldest deadline 2024-03-21T10:37:17.891147+0000) 2024-03-21T10:37:46.961 INFO:journalctl@ceph.osd.9.smithi047.stdout:Mar 21 10:37:46 smithi047 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-9[64137]: 2024-03-21T10:37:46.464+0000 7f02dc200700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-03-21T10:37:02.494627+0000 front 2024-03-21T10:37:02.494473+0000 (oldest deadline 2024-03-21T10:37:26.594304+0000) 2024-03-21T10:37:46.961 INFO:journalctl@ceph.osd.9.smithi047.stdout:Mar 21 10:37:46 smithi047 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-9[64137]: 2024-03-21T10:37:46.464+0000 7f02dc200700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-03-21T10:37:13.597260+0000 front 2024-03-21T10:37:13.596840+0000 (oldest deadline 2024-03-21T10:37:37.696959+0000) 2024-03-21T10:37:46.961 INFO:journalctl@ceph.osd.9.smithi047.stdout:Mar 21 10:37:46 smithi047 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-9[64137]: 2024-03-21T10:37:46.464+0000 7f02dc200700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.47:6814 osd.3 since back 2024-03-21T10:37:21.198652+0000 front 2024-03-21T10:37:21.199080+0000 (oldest deadline 2024-03-21T10:37:45.200271+0000) 2024-03-21T10:37:46.986 INFO:journalctl@ceph.osd.8.smithi157.stdout:Mar 21 10:37:46 smithi157 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-8[54391]: 2024-03-21T10:37:46.869+0000 7fca85e00700 -1 osd.8 111 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-03-21T10:37:03.032692+0000 front 2024-03-21T10:37:03.032837+0000 (oldest deadline 2024-03-21T10:37:27.132936+0000) 2024-03-21T10:37:46.986 INFO:journalctl@ceph.osd.8.smithi157.stdout:Mar 21 10:37:46 smithi157 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-8[54391]: 2024-03-21T10:37:46.869+0000 7fca85e00700 -1 osd.8 111 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-03-21T10:37:13.535744+0000 front 2024-03-21T10:37:13.536199+0000 (oldest deadline 2024-03-21T10:37:39.435978+0000) 2024-03-21T10:37:47.461 INFO:journalctl@ceph.osd.6.smithi047.stdout:Mar 21 10:37:47 smithi047 systemd[1]: Stopping Ceph osd.6 for 3056d51a-e76b-11ee-95cd-87774f69a715... 2024-03-21T10:37:47.461 INFO:journalctl@ceph.osd.6.smithi047.stdout:Mar 21 10:37:47 smithi047 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-6[57372]: 2024-03-21T10:37:47.190+0000 7f157e200700 -1 osd.6 111 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-03-21T10:37:03.517239+0000 front 2024-03-21T10:37:03.517159+0000 (oldest deadline 2024-03-21T10:37:26.417262+0000) 2024-03-21T10:37:47.461 INFO:journalctl@ceph.osd.6.smithi047.stdout:Mar 21 10:37:47 smithi047 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-6[57372]: 2024-03-21T10:37:47.190+0000 7f157e200700 -1 osd.6 111 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-03-21T10:37:12.722409+0000 front 2024-03-21T10:37:12.722673+0000 (oldest deadline 2024-03-21T10:37:38.622523+0000) 2024-03-21T10:37:47.461 INFO:journalctl@ceph.osd.6.smithi047.stdout:Mar 21 10:37:47 smithi047 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-6[57372]: 2024-03-21T10:37:47.190+0000 7f157e200700 -1 osd.6 111 heartbeat_check: no reply from 172.21.15.47:6814 osd.3 since back 2024-03-21T10:37:22.124841+0000 front 2024-03-21T10:37:22.124686+0000 (oldest deadline 2024-03-21T10:37:46.224686+0000) 2024-03-21T10:37:47.476 INFO:journalctl@ceph.osd.7.smithi100.stdout:Mar 21 10:37:47 smithi100 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-7[53119]: 2024-03-21T10:37:47.400+0000 7fb6ed400700 -1 osd.7 111 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-03-21T10:37:04.136587+0000 front 2024-03-21T10:37:04.136839+0000 (oldest deadline 2024-03-21T10:37:29.436935+0000) 2024-03-21T10:37:47.477 INFO:journalctl@ceph.osd.7.smithi100.stdout:Mar 21 10:37:47 smithi100 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-7[53119]: 2024-03-21T10:37:47.400+0000 7fb6ed400700 -1 osd.7 111 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-03-21T10:37:14.139283+0000 front 2024-03-21T10:37:14.139243+0000 (oldest deadline 2024-03-21T10:37:37.639505+0000) 2024-03-21T10:37:47.477 INFO:journalctl@ceph.osd.7.smithi100.stdout:Mar 21 10:37:47 smithi100 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-7[53119]: 2024-03-21T10:37:47.400+0000 7fb6ed400700 -1 osd.7 111 heartbeat_check: no reply from 172.21.15.47:6814 osd.3 since back 2024-03-21T10:37:22.243333+0000 front 2024-03-21T10:37:22.243357+0000 (oldest deadline 2024-03-21T10:37:45.742507+0000) 2024-03-21T10:37:47.477 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 10:37:47 smithi100 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-10[59767]: 2024-03-21T10:37:47.155+0000 7f315da00700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-03-21T10:36:52.673512+0000 front 2024-03-21T10:36:52.673688+0000 (oldest deadline 2024-03-21T10:37:16.675039+0000) 2024-03-21T10:37:47.477 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 10:37:47 smithi100 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-10[59767]: 2024-03-21T10:37:47.155+0000 7f315da00700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-03-21T10:37:05.379887+0000 front 2024-03-21T10:37:05.379536+0000 (oldest deadline 2024-03-21T10:37:30.079809+0000) 2024-03-21T10:37:47.477 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 10:37:47 smithi100 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-10[59767]: 2024-03-21T10:37:47.155+0000 7f315da00700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-03-21T10:37:12.981593+0000 front 2024-03-21T10:37:12.981483+0000 (oldest deadline 2024-03-21T10:37:38.281183+0000) 2024-03-21T10:37:47.477 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 10:37:47 smithi100 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-10[59767]: 2024-03-21T10:37:47.155+0000 7f315da00700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.47:6814 osd.3 since back 2024-03-21T10:37:23.583085+0000 front 2024-03-21T10:37:23.583243+0000 (oldest deadline 2024-03-21T10:37:45.283261+0000) 2024-03-21T10:37:47.829 INFO:journalctl@ceph.osd.11.smithi157.stdout:Mar 21 10:37:47 smithi157 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-11[61259]: 2024-03-21T10:37:47.591+0000 7fd0f8400700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-03-21T10:36:54.398353+0000 front 2024-03-21T10:36:54.398068+0000 (oldest deadline 2024-03-21T10:37:16.698280+0000) 2024-03-21T10:37:47.829 INFO:journalctl@ceph.osd.11.smithi157.stdout:Mar 21 10:37:47 smithi157 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-11[61259]: 2024-03-21T10:37:47.591+0000 7fd0f8400700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-03-21T10:37:04.901925+0000 front 2024-03-21T10:37:04.902083+0000 (oldest deadline 2024-03-21T10:37:29.502349+0000) 2024-03-21T10:37:47.829 INFO:journalctl@ceph.osd.11.smithi157.stdout:Mar 21 10:37:47 smithi157 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-11[61259]: 2024-03-21T10:37:47.591+0000 7fd0f8400700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-03-21T10:37:11.804109+0000 front 2024-03-21T10:37:11.804464+0000 (oldest deadline 2024-03-21T10:37:37.104077+0000) 2024-03-21T10:37:47.961 INFO:journalctl@ceph.osd.6.smithi047.stdout:Mar 21 10:37:47 smithi047 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-6[57372]: 2024-03-21T10:37:47.544+0000 7f1587a00700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.6 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-21T10:37:47.961 INFO:journalctl@ceph.osd.6.smithi047.stdout:Mar 21 10:37:47 smithi047 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-6[57372]: 2024-03-21T10:37:47.544+0000 7f1587a00700 -1 osd.6 111 *** Got signal Terminated *** 2024-03-21T10:37:47.961 INFO:journalctl@ceph.osd.6.smithi047.stdout:Mar 21 10:37:47 smithi047 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-6[57372]: 2024-03-21T10:37:47.544+0000 7f1587a00700 -1 osd.6 111 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-21T10:37:47.962 INFO:journalctl@ceph.osd.9.smithi047.stdout:Mar 21 10:37:47 smithi047 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-9[64137]: 2024-03-21T10:37:47.475+0000 7f02dc200700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-03-21T10:36:54.991613+0000 front 2024-03-21T10:36:54.991458+0000 (oldest deadline 2024-03-21T10:37:17.891147+0000) 2024-03-21T10:37:47.962 INFO:journalctl@ceph.osd.9.smithi047.stdout:Mar 21 10:37:47 smithi047 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-9[64137]: 2024-03-21T10:37:47.475+0000 7f02dc200700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-03-21T10:37:02.494627+0000 front 2024-03-21T10:37:02.494473+0000 (oldest deadline 2024-03-21T10:37:26.594304+0000) 2024-03-21T10:37:47.962 INFO:journalctl@ceph.osd.9.smithi047.stdout:Mar 21 10:37:47 smithi047 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-9[64137]: 2024-03-21T10:37:47.475+0000 7f02dc200700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-03-21T10:37:13.597260+0000 front 2024-03-21T10:37:13.596840+0000 (oldest deadline 2024-03-21T10:37:37.696959+0000) 2024-03-21T10:37:47.962 INFO:journalctl@ceph.osd.9.smithi047.stdout:Mar 21 10:37:47 smithi047 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-9[64137]: 2024-03-21T10:37:47.475+0000 7f02dc200700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.47:6814 osd.3 since back 2024-03-21T10:37:21.198652+0000 front 2024-03-21T10:37:21.199080+0000 (oldest deadline 2024-03-21T10:37:45.200271+0000) 2024-03-21T10:37:48.236 INFO:journalctl@ceph.osd.8.smithi157.stdout:Mar 21 10:37:47 smithi157 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-8[54391]: 2024-03-21T10:37:47.830+0000 7fca85e00700 -1 osd.8 111 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-03-21T10:37:03.032692+0000 front 2024-03-21T10:37:03.032837+0000 (oldest deadline 2024-03-21T10:37:27.132936+0000) 2024-03-21T10:37:48.236 INFO:journalctl@ceph.osd.8.smithi157.stdout:Mar 21 10:37:47 smithi157 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-8[54391]: 2024-03-21T10:37:47.830+0000 7fca85e00700 -1 osd.8 111 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-03-21T10:37:13.535744+0000 front 2024-03-21T10:37:13.536199+0000 (oldest deadline 2024-03-21T10:37:39.435978+0000) 2024-03-21T10:37:48.460 INFO:journalctl@ceph.osd.6.smithi047.stdout:Mar 21 10:37:48 smithi047 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-6[57372]: 2024-03-21T10:37:48.196+0000 7f157e200700 -1 osd.6 111 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-03-21T10:37:03.517239+0000 front 2024-03-21T10:37:03.517159+0000 (oldest deadline 2024-03-21T10:37:26.417262+0000) 2024-03-21T10:37:48.461 INFO:journalctl@ceph.osd.6.smithi047.stdout:Mar 21 10:37:48 smithi047 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-6[57372]: 2024-03-21T10:37:48.196+0000 7f157e200700 -1 osd.6 111 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-03-21T10:37:12.722409+0000 front 2024-03-21T10:37:12.722673+0000 (oldest deadline 2024-03-21T10:37:38.622523+0000) 2024-03-21T10:37:48.461 INFO:journalctl@ceph.osd.6.smithi047.stdout:Mar 21 10:37:48 smithi047 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-6[57372]: 2024-03-21T10:37:48.196+0000 7f157e200700 -1 osd.6 111 heartbeat_check: no reply from 172.21.15.47:6814 osd.3 since back 2024-03-21T10:37:22.124841+0000 front 2024-03-21T10:37:22.124686+0000 (oldest deadline 2024-03-21T10:37:46.224686+0000) 2024-03-21T10:37:48.476 INFO:journalctl@ceph.osd.7.smithi100.stdout:Mar 21 10:37:48 smithi100 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-7[53119]: 2024-03-21T10:37:48.430+0000 7fb6ed400700 -1 osd.7 111 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-03-21T10:37:04.136587+0000 front 2024-03-21T10:37:04.136839+0000 (oldest deadline 2024-03-21T10:37:29.436935+0000) 2024-03-21T10:37:48.476 INFO:journalctl@ceph.osd.7.smithi100.stdout:Mar 21 10:37:48 smithi100 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-7[53119]: 2024-03-21T10:37:48.430+0000 7fb6ed400700 -1 osd.7 111 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-03-21T10:37:14.139283+0000 front 2024-03-21T10:37:14.139243+0000 (oldest deadline 2024-03-21T10:37:37.639505+0000) 2024-03-21T10:37:48.477 INFO:journalctl@ceph.osd.7.smithi100.stdout:Mar 21 10:37:48 smithi100 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-7[53119]: 2024-03-21T10:37:48.430+0000 7fb6ed400700 -1 osd.7 111 heartbeat_check: no reply from 172.21.15.47:6814 osd.3 since back 2024-03-21T10:37:22.243333+0000 front 2024-03-21T10:37:22.243357+0000 (oldest deadline 2024-03-21T10:37:45.742507+0000) 2024-03-21T10:37:48.477 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 10:37:48 smithi100 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-10[59767]: 2024-03-21T10:37:48.203+0000 7f315da00700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-03-21T10:36:52.673512+0000 front 2024-03-21T10:36:52.673688+0000 (oldest deadline 2024-03-21T10:37:16.675039+0000) 2024-03-21T10:37:48.477 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 10:37:48 smithi100 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-10[59767]: 2024-03-21T10:37:48.203+0000 7f315da00700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-03-21T10:37:05.379887+0000 front 2024-03-21T10:37:05.379536+0000 (oldest deadline 2024-03-21T10:37:30.079809+0000) 2024-03-21T10:37:48.477 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 10:37:48 smithi100 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-10[59767]: 2024-03-21T10:37:48.203+0000 7f315da00700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-03-21T10:37:12.981593+0000 front 2024-03-21T10:37:12.981483+0000 (oldest deadline 2024-03-21T10:37:38.281183+0000) 2024-03-21T10:37:48.477 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 10:37:48 smithi100 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-10[59767]: 2024-03-21T10:37:48.203+0000 7f315da00700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.47:6814 osd.3 since back 2024-03-21T10:37:23.583085+0000 front 2024-03-21T10:37:23.583243+0000 (oldest deadline 2024-03-21T10:37:45.283261+0000) 2024-03-21T10:37:48.840 INFO:journalctl@ceph.osd.11.smithi157.stdout:Mar 21 10:37:48 smithi157 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-11[61259]: 2024-03-21T10:37:48.562+0000 7fd0f8400700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-03-21T10:36:54.398353+0000 front 2024-03-21T10:36:54.398068+0000 (oldest deadline 2024-03-21T10:37:16.698280+0000) 2024-03-21T10:37:48.840 INFO:journalctl@ceph.osd.11.smithi157.stdout:Mar 21 10:37:48 smithi157 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-11[61259]: 2024-03-21T10:37:48.562+0000 7fd0f8400700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-03-21T10:37:04.901925+0000 front 2024-03-21T10:37:04.902083+0000 (oldest deadline 2024-03-21T10:37:29.502349+0000) 2024-03-21T10:37:48.840 INFO:journalctl@ceph.osd.11.smithi157.stdout:Mar 21 10:37:48 smithi157 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-11[61259]: 2024-03-21T10:37:48.562+0000 7fd0f8400700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-03-21T10:37:11.804109+0000 front 2024-03-21T10:37:11.804464+0000 (oldest deadline 2024-03-21T10:37:37.104077+0000) 2024-03-21T10:37:48.960 INFO:journalctl@ceph.osd.9.smithi047.stdout:Mar 21 10:37:48 smithi047 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-9[64137]: 2024-03-21T10:37:48.509+0000 7f02dc200700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-03-21T10:36:54.991613+0000 front 2024-03-21T10:36:54.991458+0000 (oldest deadline 2024-03-21T10:37:17.891147+0000) 2024-03-21T10:37:48.961 INFO:journalctl@ceph.osd.9.smithi047.stdout:Mar 21 10:37:48 smithi047 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-9[64137]: 2024-03-21T10:37:48.509+0000 7f02dc200700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-03-21T10:37:02.494627+0000 front 2024-03-21T10:37:02.494473+0000 (oldest deadline 2024-03-21T10:37:26.594304+0000) 2024-03-21T10:37:48.961 INFO:journalctl@ceph.osd.9.smithi047.stdout:Mar 21 10:37:48 smithi047 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-9[64137]: 2024-03-21T10:37:48.509+0000 7f02dc200700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-03-21T10:37:13.597260+0000 front 2024-03-21T10:37:13.596840+0000 (oldest deadline 2024-03-21T10:37:37.696959+0000) 2024-03-21T10:37:48.961 INFO:journalctl@ceph.osd.9.smithi047.stdout:Mar 21 10:37:48 smithi047 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-9[64137]: 2024-03-21T10:37:48.509+0000 7f02dc200700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.47:6814 osd.3 since back 2024-03-21T10:37:21.198652+0000 front 2024-03-21T10:37:21.199080+0000 (oldest deadline 2024-03-21T10:37:45.200271+0000) 2024-03-21T10:37:49.236 INFO:journalctl@ceph.osd.8.smithi157.stdout:Mar 21 10:37:48 smithi157 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-8[54391]: 2024-03-21T10:37:48.841+0000 7fca85e00700 -1 osd.8 111 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-03-21T10:37:03.032692+0000 front 2024-03-21T10:37:03.032837+0000 (oldest deadline 2024-03-21T10:37:27.132936+0000) 2024-03-21T10:37:49.236 INFO:journalctl@ceph.osd.8.smithi157.stdout:Mar 21 10:37:48 smithi157 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-8[54391]: 2024-03-21T10:37:48.841+0000 7fca85e00700 -1 osd.8 111 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-03-21T10:37:13.535744+0000 front 2024-03-21T10:37:13.536199+0000 (oldest deadline 2024-03-21T10:37:39.435978+0000) 2024-03-21T10:37:49.236 INFO:journalctl@ceph.osd.8.smithi157.stdout:Mar 21 10:37:48 smithi157 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-8[54391]: 2024-03-21T10:37:48.841+0000 7fca85e00700 -1 osd.8 111 heartbeat_check: no reply from 172.21.15.47:6814 osd.3 since back 2024-03-21T10:37:19.437169+0000 front 2024-03-21T10:37:19.437298+0000 (oldest deadline 2024-03-21T10:37:48.838456+0000) 2024-03-21T10:37:49.460 INFO:journalctl@ceph.osd.6.smithi047.stdout:Mar 21 10:37:49 smithi047 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-6[57372]: 2024-03-21T10:37:49.196+0000 7f157e200700 -1 osd.6 111 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-03-21T10:37:03.517239+0000 front 2024-03-21T10:37:03.517159+0000 (oldest deadline 2024-03-21T10:37:26.417262+0000) 2024-03-21T10:37:49.461 INFO:journalctl@ceph.osd.6.smithi047.stdout:Mar 21 10:37:49 smithi047 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-6[57372]: 2024-03-21T10:37:49.196+0000 7f157e200700 -1 osd.6 111 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-03-21T10:37:12.722409+0000 front 2024-03-21T10:37:12.722673+0000 (oldest deadline 2024-03-21T10:37:38.622523+0000) 2024-03-21T10:37:49.461 INFO:journalctl@ceph.osd.6.smithi047.stdout:Mar 21 10:37:49 smithi047 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-6[57372]: 2024-03-21T10:37:49.196+0000 7f157e200700 -1 osd.6 111 heartbeat_check: no reply from 172.21.15.47:6814 osd.3 since back 2024-03-21T10:37:22.124841+0000 front 2024-03-21T10:37:22.124686+0000 (oldest deadline 2024-03-21T10:37:46.224686+0000) 2024-03-21T10:37:49.726 INFO:journalctl@ceph.osd.7.smithi100.stdout:Mar 21 10:37:49 smithi100 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-7[53119]: 2024-03-21T10:37:49.426+0000 7fb6ed400700 -1 osd.7 111 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-03-21T10:37:04.136587+0000 front 2024-03-21T10:37:04.136839+0000 (oldest deadline 2024-03-21T10:37:29.436935+0000) 2024-03-21T10:37:49.727 INFO:journalctl@ceph.osd.7.smithi100.stdout:Mar 21 10:37:49 smithi100 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-7[53119]: 2024-03-21T10:37:49.426+0000 7fb6ed400700 -1 osd.7 111 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-03-21T10:37:14.139283+0000 front 2024-03-21T10:37:14.139243+0000 (oldest deadline 2024-03-21T10:37:37.639505+0000) 2024-03-21T10:37:49.727 INFO:journalctl@ceph.osd.7.smithi100.stdout:Mar 21 10:37:49 smithi100 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-7[53119]: 2024-03-21T10:37:49.426+0000 7fb6ed400700 -1 osd.7 111 heartbeat_check: no reply from 172.21.15.47:6814 osd.3 since back 2024-03-21T10:37:22.243333+0000 front 2024-03-21T10:37:22.243357+0000 (oldest deadline 2024-03-21T10:37:45.742507+0000) 2024-03-21T10:37:49.727 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 10:37:49 smithi100 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-10[59767]: 2024-03-21T10:37:49.229+0000 7f315da00700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-03-21T10:36:52.673512+0000 front 2024-03-21T10:36:52.673688+0000 (oldest deadline 2024-03-21T10:37:16.675039+0000) 2024-03-21T10:37:49.727 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 10:37:49 smithi100 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-10[59767]: 2024-03-21T10:37:49.229+0000 7f315da00700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-03-21T10:37:05.379887+0000 front 2024-03-21T10:37:05.379536+0000 (oldest deadline 2024-03-21T10:37:30.079809+0000) 2024-03-21T10:37:49.727 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 10:37:49 smithi100 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-10[59767]: 2024-03-21T10:37:49.229+0000 7f315da00700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-03-21T10:37:12.981593+0000 front 2024-03-21T10:37:12.981483+0000 (oldest deadline 2024-03-21T10:37:38.281183+0000) 2024-03-21T10:37:49.727 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 10:37:49 smithi100 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-10[59767]: 2024-03-21T10:37:49.229+0000 7f315da00700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.47:6814 osd.3 since back 2024-03-21T10:37:23.583085+0000 front 2024-03-21T10:37:23.583243+0000 (oldest deadline 2024-03-21T10:37:45.283261+0000) 2024-03-21T10:37:49.818 INFO:journalctl@ceph.osd.11.smithi157.stdout:Mar 21 10:37:49 smithi157 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-11[61259]: 2024-03-21T10:37:49.524+0000 7fd0f8400700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-03-21T10:36:54.398353+0000 front 2024-03-21T10:36:54.398068+0000 (oldest deadline 2024-03-21T10:37:16.698280+0000) 2024-03-21T10:37:49.818 INFO:journalctl@ceph.osd.11.smithi157.stdout:Mar 21 10:37:49 smithi157 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-11[61259]: 2024-03-21T10:37:49.524+0000 7fd0f8400700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-03-21T10:37:04.901925+0000 front 2024-03-21T10:37:04.902083+0000 (oldest deadline 2024-03-21T10:37:29.502349+0000) 2024-03-21T10:37:49.818 INFO:journalctl@ceph.osd.11.smithi157.stdout:Mar 21 10:37:49 smithi157 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-11[61259]: 2024-03-21T10:37:49.524+0000 7fd0f8400700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-03-21T10:37:11.804109+0000 front 2024-03-21T10:37:11.804464+0000 (oldest deadline 2024-03-21T10:37:37.104077+0000) 2024-03-21T10:37:49.960 INFO:journalctl@ceph.osd.9.smithi047.stdout:Mar 21 10:37:49 smithi047 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-9[64137]: 2024-03-21T10:37:49.536+0000 7f02dc200700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-03-21T10:36:54.991613+0000 front 2024-03-21T10:36:54.991458+0000 (oldest deadline 2024-03-21T10:37:17.891147+0000) 2024-03-21T10:37:49.961 INFO:journalctl@ceph.osd.9.smithi047.stdout:Mar 21 10:37:49 smithi047 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-9[64137]: 2024-03-21T10:37:49.536+0000 7f02dc200700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-03-21T10:37:02.494627+0000 front 2024-03-21T10:37:02.494473+0000 (oldest deadline 2024-03-21T10:37:26.594304+0000) 2024-03-21T10:37:49.961 INFO:journalctl@ceph.osd.9.smithi047.stdout:Mar 21 10:37:49 smithi047 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-9[64137]: 2024-03-21T10:37:49.536+0000 7f02dc200700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-03-21T10:37:13.597260+0000 front 2024-03-21T10:37:13.596840+0000 (oldest deadline 2024-03-21T10:37:37.696959+0000) 2024-03-21T10:37:49.961 INFO:journalctl@ceph.osd.9.smithi047.stdout:Mar 21 10:37:49 smithi047 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-9[64137]: 2024-03-21T10:37:49.536+0000 7f02dc200700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.47:6814 osd.3 since back 2024-03-21T10:37:21.198652+0000 front 2024-03-21T10:37:21.199080+0000 (oldest deadline 2024-03-21T10:37:45.200271+0000) 2024-03-21T10:37:50.236 INFO:journalctl@ceph.osd.8.smithi157.stdout:Mar 21 10:37:49 smithi157 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-8[54391]: 2024-03-21T10:37:49.820+0000 7fca85e00700 -1 osd.8 111 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-03-21T10:37:03.032692+0000 front 2024-03-21T10:37:03.032837+0000 (oldest deadline 2024-03-21T10:37:27.132936+0000) 2024-03-21T10:37:50.236 INFO:journalctl@ceph.osd.8.smithi157.stdout:Mar 21 10:37:49 smithi157 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-8[54391]: 2024-03-21T10:37:49.820+0000 7fca85e00700 -1 osd.8 111 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-03-21T10:37:13.535744+0000 front 2024-03-21T10:37:13.536199+0000 (oldest deadline 2024-03-21T10:37:39.435978+0000) 2024-03-21T10:37:50.236 INFO:journalctl@ceph.osd.8.smithi157.stdout:Mar 21 10:37:49 smithi157 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-8[54391]: 2024-03-21T10:37:49.820+0000 7fca85e00700 -1 osd.8 111 heartbeat_check: no reply from 172.21.15.47:6814 osd.3 since back 2024-03-21T10:37:19.437169+0000 front 2024-03-21T10:37:19.437298+0000 (oldest deadline 2024-03-21T10:37:48.838456+0000) 2024-03-21T10:37:50.460 INFO:journalctl@ceph.osd.6.smithi047.stdout:Mar 21 10:37:50 smithi047 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-6[57372]: 2024-03-21T10:37:50.193+0000 7f157e200700 -1 osd.6 111 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-03-21T10:37:03.517239+0000 front 2024-03-21T10:37:03.517159+0000 (oldest deadline 2024-03-21T10:37:26.417262+0000) 2024-03-21T10:37:50.461 INFO:journalctl@ceph.osd.6.smithi047.stdout:Mar 21 10:37:50 smithi047 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-6[57372]: 2024-03-21T10:37:50.193+0000 7f157e200700 -1 osd.6 111 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-03-21T10:37:12.722409+0000 front 2024-03-21T10:37:12.722673+0000 (oldest deadline 2024-03-21T10:37:38.622523+0000) 2024-03-21T10:37:50.461 INFO:journalctl@ceph.osd.6.smithi047.stdout:Mar 21 10:37:50 smithi047 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-6[57372]: 2024-03-21T10:37:50.193+0000 7f157e200700 -1 osd.6 111 heartbeat_check: no reply from 172.21.15.47:6814 osd.3 since back 2024-03-21T10:37:22.124841+0000 front 2024-03-21T10:37:22.124686+0000 (oldest deadline 2024-03-21T10:37:46.224686+0000) 2024-03-21T10:37:50.476 INFO:journalctl@ceph.osd.7.smithi100.stdout:Mar 21 10:37:50 smithi100 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-7[53119]: 2024-03-21T10:37:50.410+0000 7fb6ed400700 -1 osd.7 111 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-03-21T10:37:04.136587+0000 front 2024-03-21T10:37:04.136839+0000 (oldest deadline 2024-03-21T10:37:29.436935+0000) 2024-03-21T10:37:50.476 INFO:journalctl@ceph.osd.7.smithi100.stdout:Mar 21 10:37:50 smithi100 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-7[53119]: 2024-03-21T10:37:50.410+0000 7fb6ed400700 -1 osd.7 111 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-03-21T10:37:14.139283+0000 front 2024-03-21T10:37:14.139243+0000 (oldest deadline 2024-03-21T10:37:37.639505+0000) 2024-03-21T10:37:50.476 INFO:journalctl@ceph.osd.7.smithi100.stdout:Mar 21 10:37:50 smithi100 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-7[53119]: 2024-03-21T10:37:50.410+0000 7fb6ed400700 -1 osd.7 111 heartbeat_check: no reply from 172.21.15.47:6814 osd.3 since back 2024-03-21T10:37:22.243333+0000 front 2024-03-21T10:37:22.243357+0000 (oldest deadline 2024-03-21T10:37:45.742507+0000) 2024-03-21T10:37:50.477 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 10:37:50 smithi100 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-10[59767]: 2024-03-21T10:37:50.197+0000 7f315da00700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-03-21T10:36:52.673512+0000 front 2024-03-21T10:36:52.673688+0000 (oldest deadline 2024-03-21T10:37:16.675039+0000) 2024-03-21T10:37:50.477 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 10:37:50 smithi100 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-10[59767]: 2024-03-21T10:37:50.197+0000 7f315da00700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-03-21T10:37:05.379887+0000 front 2024-03-21T10:37:05.379536+0000 (oldest deadline 2024-03-21T10:37:30.079809+0000) 2024-03-21T10:37:50.477 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 10:37:50 smithi100 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-10[59767]: 2024-03-21T10:37:50.197+0000 7f315da00700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-03-21T10:37:12.981593+0000 front 2024-03-21T10:37:12.981483+0000 (oldest deadline 2024-03-21T10:37:38.281183+0000) 2024-03-21T10:37:50.477 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 10:37:50 smithi100 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-10[59767]: 2024-03-21T10:37:50.197+0000 7f315da00700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.47:6814 osd.3 since back 2024-03-21T10:37:23.583085+0000 front 2024-03-21T10:37:23.583243+0000 (oldest deadline 2024-03-21T10:37:45.283261+0000) 2024-03-21T10:37:50.820 INFO:journalctl@ceph.osd.11.smithi157.stdout:Mar 21 10:37:50 smithi157 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-11[61259]: 2024-03-21T10:37:50.540+0000 7fd0f8400700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-03-21T10:36:54.398353+0000 front 2024-03-21T10:36:54.398068+0000 (oldest deadline 2024-03-21T10:37:16.698280+0000) 2024-03-21T10:37:50.821 INFO:journalctl@ceph.osd.11.smithi157.stdout:Mar 21 10:37:50 smithi157 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-11[61259]: 2024-03-21T10:37:50.540+0000 7fd0f8400700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-03-21T10:37:04.901925+0000 front 2024-03-21T10:37:04.902083+0000 (oldest deadline 2024-03-21T10:37:29.502349+0000) 2024-03-21T10:37:50.821 INFO:journalctl@ceph.osd.11.smithi157.stdout:Mar 21 10:37:50 smithi157 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-11[61259]: 2024-03-21T10:37:50.540+0000 7fd0f8400700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-03-21T10:37:11.804109+0000 front 2024-03-21T10:37:11.804464+0000 (oldest deadline 2024-03-21T10:37:37.104077+0000) 2024-03-21T10:37:50.960 INFO:journalctl@ceph.osd.9.smithi047.stdout:Mar 21 10:37:50 smithi047 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-9[64137]: 2024-03-21T10:37:50.583+0000 7f02dc200700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-03-21T10:36:54.991613+0000 front 2024-03-21T10:36:54.991458+0000 (oldest deadline 2024-03-21T10:37:17.891147+0000) 2024-03-21T10:37:50.961 INFO:journalctl@ceph.osd.9.smithi047.stdout:Mar 21 10:37:50 smithi047 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-9[64137]: 2024-03-21T10:37:50.583+0000 7f02dc200700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-03-21T10:37:02.494627+0000 front 2024-03-21T10:37:02.494473+0000 (oldest deadline 2024-03-21T10:37:26.594304+0000) 2024-03-21T10:37:50.961 INFO:journalctl@ceph.osd.9.smithi047.stdout:Mar 21 10:37:50 smithi047 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-9[64137]: 2024-03-21T10:37:50.583+0000 7f02dc200700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-03-21T10:37:13.597260+0000 front 2024-03-21T10:37:13.596840+0000 (oldest deadline 2024-03-21T10:37:37.696959+0000) 2024-03-21T10:37:50.961 INFO:journalctl@ceph.osd.9.smithi047.stdout:Mar 21 10:37:50 smithi047 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-9[64137]: 2024-03-21T10:37:50.583+0000 7f02dc200700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.47:6814 osd.3 since back 2024-03-21T10:37:21.198652+0000 front 2024-03-21T10:37:21.199080+0000 (oldest deadline 2024-03-21T10:37:45.200271+0000) 2024-03-21T10:37:51.236 INFO:journalctl@ceph.osd.8.smithi157.stdout:Mar 21 10:37:50 smithi157 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-8[54391]: 2024-03-21T10:37:50.822+0000 7fca85e00700 -1 osd.8 111 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-03-21T10:37:03.032692+0000 front 2024-03-21T10:37:03.032837+0000 (oldest deadline 2024-03-21T10:37:27.132936+0000) 2024-03-21T10:37:51.236 INFO:journalctl@ceph.osd.8.smithi157.stdout:Mar 21 10:37:50 smithi157 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-8[54391]: 2024-03-21T10:37:50.822+0000 7fca85e00700 -1 osd.8 111 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-03-21T10:37:13.535744+0000 front 2024-03-21T10:37:13.536199+0000 (oldest deadline 2024-03-21T10:37:39.435978+0000) 2024-03-21T10:37:51.237 INFO:journalctl@ceph.osd.8.smithi157.stdout:Mar 21 10:37:50 smithi157 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-8[54391]: 2024-03-21T10:37:50.822+0000 7fca85e00700 -1 osd.8 111 heartbeat_check: no reply from 172.21.15.47:6814 osd.3 since back 2024-03-21T10:37:19.437169+0000 front 2024-03-21T10:37:19.437298+0000 (oldest deadline 2024-03-21T10:37:48.838456+0000) 2024-03-21T10:37:51.460 INFO:journalctl@ceph.osd.6.smithi047.stdout:Mar 21 10:37:51 smithi047 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-6[57372]: 2024-03-21T10:37:51.155+0000 7f157e200700 -1 osd.6 111 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-03-21T10:37:03.517239+0000 front 2024-03-21T10:37:03.517159+0000 (oldest deadline 2024-03-21T10:37:26.417262+0000) 2024-03-21T10:37:51.460 INFO:journalctl@ceph.osd.6.smithi047.stdout:Mar 21 10:37:51 smithi047 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-6[57372]: 2024-03-21T10:37:51.155+0000 7f157e200700 -1 osd.6 111 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-03-21T10:37:12.722409+0000 front 2024-03-21T10:37:12.722673+0000 (oldest deadline 2024-03-21T10:37:38.622523+0000) 2024-03-21T10:37:51.461 INFO:journalctl@ceph.osd.6.smithi047.stdout:Mar 21 10:37:51 smithi047 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-6[57372]: 2024-03-21T10:37:51.155+0000 7f157e200700 -1 osd.6 111 heartbeat_check: no reply from 172.21.15.47:6814 osd.3 since back 2024-03-21T10:37:22.124841+0000 front 2024-03-21T10:37:22.124686+0000 (oldest deadline 2024-03-21T10:37:46.224686+0000) 2024-03-21T10:37:51.478 INFO:journalctl@ceph.osd.7.smithi100.stdout:Mar 21 10:37:51 smithi100 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-7[53119]: 2024-03-21T10:37:51.440+0000 7fb6ed400700 -1 osd.7 111 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-03-21T10:37:04.136587+0000 front 2024-03-21T10:37:04.136839+0000 (oldest deadline 2024-03-21T10:37:29.436935+0000) 2024-03-21T10:37:51.478 INFO:journalctl@ceph.osd.7.smithi100.stdout:Mar 21 10:37:51 smithi100 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-7[53119]: 2024-03-21T10:37:51.440+0000 7fb6ed400700 -1 osd.7 111 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-03-21T10:37:14.139283+0000 front 2024-03-21T10:37:14.139243+0000 (oldest deadline 2024-03-21T10:37:37.639505+0000) 2024-03-21T10:37:51.478 INFO:journalctl@ceph.osd.7.smithi100.stdout:Mar 21 10:37:51 smithi100 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-7[53119]: 2024-03-21T10:37:51.440+0000 7fb6ed400700 -1 osd.7 111 heartbeat_check: no reply from 172.21.15.47:6814 osd.3 since back 2024-03-21T10:37:22.243333+0000 front 2024-03-21T10:37:22.243357+0000 (oldest deadline 2024-03-21T10:37:45.742507+0000) 2024-03-21T10:37:51.478 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 10:37:51 smithi100 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-10[59767]: 2024-03-21T10:37:51.199+0000 7f315da00700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-03-21T10:36:52.673512+0000 front 2024-03-21T10:36:52.673688+0000 (oldest deadline 2024-03-21T10:37:16.675039+0000) 2024-03-21T10:37:51.478 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 10:37:51 smithi100 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-10[59767]: 2024-03-21T10:37:51.199+0000 7f315da00700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-03-21T10:37:05.379887+0000 front 2024-03-21T10:37:05.379536+0000 (oldest deadline 2024-03-21T10:37:30.079809+0000) 2024-03-21T10:37:51.479 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 10:37:51 smithi100 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-10[59767]: 2024-03-21T10:37:51.199+0000 7f315da00700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-03-21T10:37:12.981593+0000 front 2024-03-21T10:37:12.981483+0000 (oldest deadline 2024-03-21T10:37:38.281183+0000) 2024-03-21T10:37:51.479 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 10:37:51 smithi100 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-10[59767]: 2024-03-21T10:37:51.199+0000 7f315da00700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.47:6814 osd.3 since back 2024-03-21T10:37:23.583085+0000 front 2024-03-21T10:37:23.583243+0000 (oldest deadline 2024-03-21T10:37:45.283261+0000) 2024-03-21T10:37:51.862 INFO:journalctl@ceph.osd.11.smithi157.stdout:Mar 21 10:37:51 smithi157 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-11[61259]: 2024-03-21T10:37:51.551+0000 7fd0f8400700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-03-21T10:36:54.398353+0000 front 2024-03-21T10:36:54.398068+0000 (oldest deadline 2024-03-21T10:37:16.698280+0000) 2024-03-21T10:37:51.862 INFO:journalctl@ceph.osd.11.smithi157.stdout:Mar 21 10:37:51 smithi157 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-11[61259]: 2024-03-21T10:37:51.551+0000 7fd0f8400700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-03-21T10:37:04.901925+0000 front 2024-03-21T10:37:04.902083+0000 (oldest deadline 2024-03-21T10:37:29.502349+0000) 2024-03-21T10:37:51.862 INFO:journalctl@ceph.osd.11.smithi157.stdout:Mar 21 10:37:51 smithi157 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-11[61259]: 2024-03-21T10:37:51.551+0000 7fd0f8400700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-03-21T10:37:11.804109+0000 front 2024-03-21T10:37:11.804464+0000 (oldest deadline 2024-03-21T10:37:37.104077+0000) 2024-03-21T10:37:51.960 INFO:journalctl@ceph.osd.9.smithi047.stdout:Mar 21 10:37:51 smithi047 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-9[64137]: 2024-03-21T10:37:51.551+0000 7f02dc200700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-03-21T10:36:54.991613+0000 front 2024-03-21T10:36:54.991458+0000 (oldest deadline 2024-03-21T10:37:17.891147+0000) 2024-03-21T10:37:51.961 INFO:journalctl@ceph.osd.9.smithi047.stdout:Mar 21 10:37:51 smithi047 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-9[64137]: 2024-03-21T10:37:51.551+0000 7f02dc200700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-03-21T10:37:02.494627+0000 front 2024-03-21T10:37:02.494473+0000 (oldest deadline 2024-03-21T10:37:26.594304+0000) 2024-03-21T10:37:51.961 INFO:journalctl@ceph.osd.9.smithi047.stdout:Mar 21 10:37:51 smithi047 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-9[64137]: 2024-03-21T10:37:51.551+0000 7f02dc200700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-03-21T10:37:13.597260+0000 front 2024-03-21T10:37:13.596840+0000 (oldest deadline 2024-03-21T10:37:37.696959+0000) 2024-03-21T10:37:51.961 INFO:journalctl@ceph.osd.9.smithi047.stdout:Mar 21 10:37:51 smithi047 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-9[64137]: 2024-03-21T10:37:51.551+0000 7f02dc200700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.47:6814 osd.3 since back 2024-03-21T10:37:21.198652+0000 front 2024-03-21T10:37:21.199080+0000 (oldest deadline 2024-03-21T10:37:45.200271+0000) 2024-03-21T10:37:52.236 INFO:journalctl@ceph.osd.8.smithi157.stdout:Mar 21 10:37:51 smithi157 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-8[54391]: 2024-03-21T10:37:51.863+0000 7fca85e00700 -1 osd.8 111 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-03-21T10:37:03.032692+0000 front 2024-03-21T10:37:03.032837+0000 (oldest deadline 2024-03-21T10:37:27.132936+0000) 2024-03-21T10:37:52.236 INFO:journalctl@ceph.osd.8.smithi157.stdout:Mar 21 10:37:51 smithi157 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-8[54391]: 2024-03-21T10:37:51.863+0000 7fca85e00700 -1 osd.8 111 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-03-21T10:37:13.535744+0000 front 2024-03-21T10:37:13.536199+0000 (oldest deadline 2024-03-21T10:37:39.435978+0000) 2024-03-21T10:37:52.237 INFO:journalctl@ceph.osd.8.smithi157.stdout:Mar 21 10:37:51 smithi157 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-8[54391]: 2024-03-21T10:37:51.863+0000 7fca85e00700 -1 osd.8 111 heartbeat_check: no reply from 172.21.15.47:6814 osd.3 since back 2024-03-21T10:37:19.437169+0000 front 2024-03-21T10:37:19.437298+0000 (oldest deadline 2024-03-21T10:37:48.838456+0000) 2024-03-21T10:37:52.459 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 10:37:52 smithi100 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-10[59767]: 2024-03-21T10:37:52.157+0000 7f315da00700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-03-21T10:36:52.673512+0000 front 2024-03-21T10:36:52.673688+0000 (oldest deadline 2024-03-21T10:37:16.675039+0000) 2024-03-21T10:37:52.459 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 10:37:52 smithi100 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-10[59767]: 2024-03-21T10:37:52.157+0000 7f315da00700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-03-21T10:37:05.379887+0000 front 2024-03-21T10:37:05.379536+0000 (oldest deadline 2024-03-21T10:37:30.079809+0000) 2024-03-21T10:37:52.459 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 10:37:52 smithi100 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-10[59767]: 2024-03-21T10:37:52.157+0000 7f315da00700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-03-21T10:37:12.981593+0000 front 2024-03-21T10:37:12.981483+0000 (oldest deadline 2024-03-21T10:37:38.281183+0000) 2024-03-21T10:37:52.459 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 10:37:52 smithi100 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-10[59767]: 2024-03-21T10:37:52.157+0000 7f315da00700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.47:6814 osd.3 since back 2024-03-21T10:37:23.583085+0000 front 2024-03-21T10:37:23.583243+0000 (oldest deadline 2024-03-21T10:37:45.283261+0000) 2024-03-21T10:37:52.460 INFO:journalctl@ceph.osd.6.smithi047.stdout:Mar 21 10:37:52 smithi047 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-6[57372]: 2024-03-21T10:37:52.146+0000 7f157e200700 -1 osd.6 111 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-03-21T10:37:03.517239+0000 front 2024-03-21T10:37:03.517159+0000 (oldest deadline 2024-03-21T10:37:26.417262+0000) 2024-03-21T10:37:52.460 INFO:journalctl@ceph.osd.6.smithi047.stdout:Mar 21 10:37:52 smithi047 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-6[57372]: 2024-03-21T10:37:52.146+0000 7f157e200700 -1 osd.6 111 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-03-21T10:37:12.722409+0000 front 2024-03-21T10:37:12.722673+0000 (oldest deadline 2024-03-21T10:37:38.622523+0000) 2024-03-21T10:37:52.460 INFO:journalctl@ceph.osd.6.smithi047.stdout:Mar 21 10:37:52 smithi047 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-6[57372]: 2024-03-21T10:37:52.146+0000 7f157e200700 -1 osd.6 111 heartbeat_check: no reply from 172.21.15.47:6814 osd.3 since back 2024-03-21T10:37:22.124841+0000 front 2024-03-21T10:37:22.124686+0000 (oldest deadline 2024-03-21T10:37:46.224686+0000) 2024-03-21T10:37:52.726 INFO:journalctl@ceph.osd.7.smithi100.stdout:Mar 21 10:37:52 smithi100 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-7[53119]: 2024-03-21T10:37:52.460+0000 7fb6ed400700 -1 osd.7 111 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-03-21T10:37:04.136587+0000 front 2024-03-21T10:37:04.136839+0000 (oldest deadline 2024-03-21T10:37:29.436935+0000) 2024-03-21T10:37:52.726 INFO:journalctl@ceph.osd.7.smithi100.stdout:Mar 21 10:37:52 smithi100 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-7[53119]: 2024-03-21T10:37:52.460+0000 7fb6ed400700 -1 osd.7 111 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-03-21T10:37:14.139283+0000 front 2024-03-21T10:37:14.139243+0000 (oldest deadline 2024-03-21T10:37:37.639505+0000) 2024-03-21T10:37:52.726 INFO:journalctl@ceph.osd.7.smithi100.stdout:Mar 21 10:37:52 smithi100 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-7[53119]: 2024-03-21T10:37:52.460+0000 7fb6ed400700 -1 osd.7 111 heartbeat_check: no reply from 172.21.15.47:6814 osd.3 since back 2024-03-21T10:37:22.243333+0000 front 2024-03-21T10:37:22.243357+0000 (oldest deadline 2024-03-21T10:37:45.742507+0000) 2024-03-21T10:37:52.843 INFO:journalctl@ceph.osd.11.smithi157.stdout:Mar 21 10:37:52 smithi157 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-11[61259]: 2024-03-21T10:37:52.520+0000 7fd0f8400700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-03-21T10:36:54.398353+0000 front 2024-03-21T10:36:54.398068+0000 (oldest deadline 2024-03-21T10:37:16.698280+0000) 2024-03-21T10:37:52.843 INFO:journalctl@ceph.osd.11.smithi157.stdout:Mar 21 10:37:52 smithi157 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-11[61259]: 2024-03-21T10:37:52.520+0000 7fd0f8400700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-03-21T10:37:04.901925+0000 front 2024-03-21T10:37:04.902083+0000 (oldest deadline 2024-03-21T10:37:29.502349+0000) 2024-03-21T10:37:52.843 INFO:journalctl@ceph.osd.11.smithi157.stdout:Mar 21 10:37:52 smithi157 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-11[61259]: 2024-03-21T10:37:52.520+0000 7fd0f8400700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-03-21T10:37:11.804109+0000 front 2024-03-21T10:37:11.804464+0000 (oldest deadline 2024-03-21T10:37:37.104077+0000) 2024-03-21T10:37:52.960 INFO:journalctl@ceph.osd.9.smithi047.stdout:Mar 21 10:37:52 smithi047 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-9[64137]: 2024-03-21T10:37:52.510+0000 7f02dc200700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-03-21T10:36:54.991613+0000 front 2024-03-21T10:36:54.991458+0000 (oldest deadline 2024-03-21T10:37:17.891147+0000) 2024-03-21T10:37:52.961 INFO:journalctl@ceph.osd.9.smithi047.stdout:Mar 21 10:37:52 smithi047 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-9[64137]: 2024-03-21T10:37:52.510+0000 7f02dc200700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-03-21T10:37:02.494627+0000 front 2024-03-21T10:37:02.494473+0000 (oldest deadline 2024-03-21T10:37:26.594304+0000) 2024-03-21T10:37:52.961 INFO:journalctl@ceph.osd.9.smithi047.stdout:Mar 21 10:37:52 smithi047 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-9[64137]: 2024-03-21T10:37:52.510+0000 7f02dc200700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-03-21T10:37:13.597260+0000 front 2024-03-21T10:37:13.596840+0000 (oldest deadline 2024-03-21T10:37:37.696959+0000) 2024-03-21T10:37:52.961 INFO:journalctl@ceph.osd.9.smithi047.stdout:Mar 21 10:37:52 smithi047 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-9[64137]: 2024-03-21T10:37:52.510+0000 7f02dc200700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.47:6814 osd.3 since back 2024-03-21T10:37:21.198652+0000 front 2024-03-21T10:37:21.199080+0000 (oldest deadline 2024-03-21T10:37:45.200271+0000) 2024-03-21T10:37:53.236 INFO:journalctl@ceph.osd.8.smithi157.stdout:Mar 21 10:37:52 smithi157 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-8[54391]: 2024-03-21T10:37:52.845+0000 7fca85e00700 -1 osd.8 111 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-03-21T10:37:03.032692+0000 front 2024-03-21T10:37:03.032837+0000 (oldest deadline 2024-03-21T10:37:27.132936+0000) 2024-03-21T10:37:53.236 INFO:journalctl@ceph.osd.8.smithi157.stdout:Mar 21 10:37:52 smithi157 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-8[54391]: 2024-03-21T10:37:52.845+0000 7fca85e00700 -1 osd.8 111 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-03-21T10:37:13.535744+0000 front 2024-03-21T10:37:13.536199+0000 (oldest deadline 2024-03-21T10:37:39.435978+0000) 2024-03-21T10:37:53.236 INFO:journalctl@ceph.osd.8.smithi157.stdout:Mar 21 10:37:52 smithi157 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-8[54391]: 2024-03-21T10:37:52.845+0000 7fca85e00700 -1 osd.8 111 heartbeat_check: no reply from 172.21.15.47:6814 osd.3 since back 2024-03-21T10:37:19.437169+0000 front 2024-03-21T10:37:19.437298+0000 (oldest deadline 2024-03-21T10:37:48.838456+0000) 2024-03-21T10:37:53.460 INFO:journalctl@ceph.osd.6.smithi047.stdout:Mar 21 10:37:53 smithi047 podman[103488]: 2024-03-21 10:37:53.159369258 +0000 UTC m=+5.769257736 container died ef71a58e6ea5f8ed6326f70e2394b6e85323d77e0b5fbf87a8788cf0d08fd7c2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-6, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, GIT_CLEAN=True, ceph=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_BRANCH=HEAD, org.label-schema.build-date=20240319, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=) 2024-03-21T10:37:53.476 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 10:37:53 smithi100 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-10[59767]: 2024-03-21T10:37:53.124+0000 7f315da00700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-03-21T10:36:52.673512+0000 front 2024-03-21T10:36:52.673688+0000 (oldest deadline 2024-03-21T10:37:16.675039+0000) 2024-03-21T10:37:53.476 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 10:37:53 smithi100 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-10[59767]: 2024-03-21T10:37:53.124+0000 7f315da00700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-03-21T10:37:05.379887+0000 front 2024-03-21T10:37:05.379536+0000 (oldest deadline 2024-03-21T10:37:30.079809+0000) 2024-03-21T10:37:53.476 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 10:37:53 smithi100 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-10[59767]: 2024-03-21T10:37:53.124+0000 7f315da00700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-03-21T10:37:12.981593+0000 front 2024-03-21T10:37:12.981483+0000 (oldest deadline 2024-03-21T10:37:38.281183+0000) 2024-03-21T10:37:53.476 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 10:37:53 smithi100 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-10[59767]: 2024-03-21T10:37:53.124+0000 7f315da00700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.47:6814 osd.3 since back 2024-03-21T10:37:23.583085+0000 front 2024-03-21T10:37:23.583243+0000 (oldest deadline 2024-03-21T10:37:45.283261+0000) 2024-03-21T10:37:53.870 INFO:journalctl@ceph.osd.6.smithi047.stdout:Mar 21 10:37:53 smithi047 podman[103488]: 2024-03-21 10:37:53.612934612 +0000 UTC m=+6.222823113 container cleanup ef71a58e6ea5f8ed6326f70e2394b6e85323d77e0b5fbf87a8788cf0d08fd7c2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-6, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.build-date=20240319, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2) 2024-03-21T10:37:53.870 INFO:journalctl@ceph.osd.6.smithi047.stdout:Mar 21 10:37:53 smithi047 bash[103488]: ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-6 2024-03-21T10:37:53.870 INFO:journalctl@ceph.osd.9.smithi047.stdout:Mar 21 10:37:53 smithi047 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-9[64137]: 2024-03-21T10:37:53.463+0000 7f02dc200700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-03-21T10:36:54.991613+0000 front 2024-03-21T10:36:54.991458+0000 (oldest deadline 2024-03-21T10:37:17.891147+0000) 2024-03-21T10:37:53.870 INFO:journalctl@ceph.osd.9.smithi047.stdout:Mar 21 10:37:53 smithi047 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-9[64137]: 2024-03-21T10:37:53.463+0000 7f02dc200700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-03-21T10:37:02.494627+0000 front 2024-03-21T10:37:02.494473+0000 (oldest deadline 2024-03-21T10:37:26.594304+0000) 2024-03-21T10:37:53.871 INFO:journalctl@ceph.osd.9.smithi047.stdout:Mar 21 10:37:53 smithi047 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-9[64137]: 2024-03-21T10:37:53.463+0000 7f02dc200700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-03-21T10:37:13.597260+0000 front 2024-03-21T10:37:13.596840+0000 (oldest deadline 2024-03-21T10:37:37.696959+0000) 2024-03-21T10:37:53.871 INFO:journalctl@ceph.osd.9.smithi047.stdout:Mar 21 10:37:53 smithi047 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-9[64137]: 2024-03-21T10:37:53.463+0000 7f02dc200700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.47:6814 osd.3 since back 2024-03-21T10:37:21.198652+0000 front 2024-03-21T10:37:21.199080+0000 (oldest deadline 2024-03-21T10:37:45.200271+0000) 2024-03-21T10:37:53.891 INFO:journalctl@ceph.osd.11.smithi157.stdout:Mar 21 10:37:53 smithi157 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-11[61259]: 2024-03-21T10:37:53.508+0000 7fd0f8400700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-03-21T10:36:54.398353+0000 front 2024-03-21T10:36:54.398068+0000 (oldest deadline 2024-03-21T10:37:16.698280+0000) 2024-03-21T10:37:53.891 INFO:journalctl@ceph.osd.11.smithi157.stdout:Mar 21 10:37:53 smithi157 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-11[61259]: 2024-03-21T10:37:53.508+0000 7fd0f8400700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-03-21T10:37:04.901925+0000 front 2024-03-21T10:37:04.902083+0000 (oldest deadline 2024-03-21T10:37:29.502349+0000) 2024-03-21T10:37:53.891 INFO:journalctl@ceph.osd.11.smithi157.stdout:Mar 21 10:37:53 smithi157 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-11[61259]: 2024-03-21T10:37:53.508+0000 7fd0f8400700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-03-21T10:37:11.804109+0000 front 2024-03-21T10:37:11.804464+0000 (oldest deadline 2024-03-21T10:37:37.104077+0000) 2024-03-21T10:37:53.976 INFO:journalctl@ceph.osd.7.smithi100.stdout:Mar 21 10:37:53 smithi100 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-7[53119]: 2024-03-21T10:37:53.503+0000 7fb6ed400700 -1 osd.7 111 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-03-21T10:37:04.136587+0000 front 2024-03-21T10:37:04.136839+0000 (oldest deadline 2024-03-21T10:37:29.436935+0000) 2024-03-21T10:37:53.976 INFO:journalctl@ceph.osd.7.smithi100.stdout:Mar 21 10:37:53 smithi100 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-7[53119]: 2024-03-21T10:37:53.503+0000 7fb6ed400700 -1 osd.7 111 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-03-21T10:37:14.139283+0000 front 2024-03-21T10:37:14.139243+0000 (oldest deadline 2024-03-21T10:37:37.639505+0000) 2024-03-21T10:37:53.976 INFO:journalctl@ceph.osd.7.smithi100.stdout:Mar 21 10:37:53 smithi100 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-7[53119]: 2024-03-21T10:37:53.503+0000 7fb6ed400700 -1 osd.7 111 heartbeat_check: no reply from 172.21.15.47:6814 osd.3 since back 2024-03-21T10:37:22.243333+0000 front 2024-03-21T10:37:22.243357+0000 (oldest deadline 2024-03-21T10:37:45.742507+0000) 2024-03-21T10:37:54.210 INFO:journalctl@ceph.osd.6.smithi047.stdout:Mar 21 10:37:53 smithi047 podman[103507]: 2024-03-21 10:37:53.87236112 +0000 UTC m=+0.697775205 container remove ef71a58e6ea5f8ed6326f70e2394b6e85323d77e0b5fbf87a8788cf0d08fd7c2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-6, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, ceph=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240319, GIT_CLEAN=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30) 2024-03-21T10:37:54.236 INFO:journalctl@ceph.osd.8.smithi157.stdout:Mar 21 10:37:53 smithi157 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-8[54391]: 2024-03-21T10:37:53.892+0000 7fca85e00700 -1 osd.8 111 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-03-21T10:37:03.032692+0000 front 2024-03-21T10:37:03.032837+0000 (oldest deadline 2024-03-21T10:37:27.132936+0000) 2024-03-21T10:37:54.237 INFO:journalctl@ceph.osd.8.smithi157.stdout:Mar 21 10:37:53 smithi157 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-8[54391]: 2024-03-21T10:37:53.892+0000 7fca85e00700 -1 osd.8 111 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-03-21T10:37:13.535744+0000 front 2024-03-21T10:37:13.536199+0000 (oldest deadline 2024-03-21T10:37:39.435978+0000) 2024-03-21T10:37:54.237 INFO:journalctl@ceph.osd.8.smithi157.stdout:Mar 21 10:37:53 smithi157 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-8[54391]: 2024-03-21T10:37:53.892+0000 7fca85e00700 -1 osd.8 111 heartbeat_check: no reply from 172.21.15.47:6814 osd.3 since back 2024-03-21T10:37:19.437169+0000 front 2024-03-21T10:37:19.437298+0000 (oldest deadline 2024-03-21T10:37:48.838456+0000) 2024-03-21T10:37:54.476 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 10:37:54 smithi100 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-10[59767]: 2024-03-21T10:37:54.093+0000 7f315da00700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-03-21T10:36:52.673512+0000 front 2024-03-21T10:36:52.673688+0000 (oldest deadline 2024-03-21T10:37:16.675039+0000) 2024-03-21T10:37:54.477 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 10:37:54 smithi100 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-10[59767]: 2024-03-21T10:37:54.093+0000 7f315da00700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-03-21T10:37:05.379887+0000 front 2024-03-21T10:37:05.379536+0000 (oldest deadline 2024-03-21T10:37:30.079809+0000) 2024-03-21T10:37:54.477 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 10:37:54 smithi100 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-10[59767]: 2024-03-21T10:37:54.093+0000 7f315da00700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-03-21T10:37:12.981593+0000 front 2024-03-21T10:37:12.981483+0000 (oldest deadline 2024-03-21T10:37:38.281183+0000) 2024-03-21T10:37:54.477 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 10:37:54 smithi100 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-10[59767]: 2024-03-21T10:37:54.093+0000 7f315da00700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.47:6814 osd.3 since back 2024-03-21T10:37:23.583085+0000 front 2024-03-21T10:37:23.583243+0000 (oldest deadline 2024-03-21T10:37:45.283261+0000) 2024-03-21T10:37:54.717 INFO:journalctl@ceph.osd.9.smithi047.stdout:Mar 21 10:37:54 smithi047 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-9[64137]: 2024-03-21T10:37:54.466+0000 7f02dc200700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-03-21T10:36:54.991613+0000 front 2024-03-21T10:36:54.991458+0000 (oldest deadline 2024-03-21T10:37:17.891147+0000) 2024-03-21T10:37:54.717 INFO:journalctl@ceph.osd.9.smithi047.stdout:Mar 21 10:37:54 smithi047 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-9[64137]: 2024-03-21T10:37:54.466+0000 7f02dc200700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-03-21T10:37:02.494627+0000 front 2024-03-21T10:37:02.494473+0000 (oldest deadline 2024-03-21T10:37:26.594304+0000) 2024-03-21T10:37:54.717 INFO:journalctl@ceph.osd.9.smithi047.stdout:Mar 21 10:37:54 smithi047 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-9[64137]: 2024-03-21T10:37:54.466+0000 7f02dc200700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-03-21T10:37:13.597260+0000 front 2024-03-21T10:37:13.596840+0000 (oldest deadline 2024-03-21T10:37:37.696959+0000) 2024-03-21T10:37:54.717 INFO:journalctl@ceph.osd.9.smithi047.stdout:Mar 21 10:37:54 smithi047 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-9[64137]: 2024-03-21T10:37:54.466+0000 7f02dc200700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.47:6814 osd.3 since back 2024-03-21T10:37:21.198652+0000 front 2024-03-21T10:37:21.199080+0000 (oldest deadline 2024-03-21T10:37:45.200271+0000) 2024-03-21T10:37:54.871 INFO:journalctl@ceph.osd.11.smithi157.stdout:Mar 21 10:37:54 smithi157 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-11[61259]: 2024-03-21T10:37:54.551+0000 7fd0f8400700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-03-21T10:36:54.398353+0000 front 2024-03-21T10:36:54.398068+0000 (oldest deadline 2024-03-21T10:37:16.698280+0000) 2024-03-21T10:37:54.871 INFO:journalctl@ceph.osd.11.smithi157.stdout:Mar 21 10:37:54 smithi157 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-11[61259]: 2024-03-21T10:37:54.551+0000 7fd0f8400700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-03-21T10:37:04.901925+0000 front 2024-03-21T10:37:04.902083+0000 (oldest deadline 2024-03-21T10:37:29.502349+0000) 2024-03-21T10:37:54.871 INFO:journalctl@ceph.osd.11.smithi157.stdout:Mar 21 10:37:54 smithi157 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-11[61259]: 2024-03-21T10:37:54.551+0000 7fd0f8400700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-03-21T10:37:11.804109+0000 front 2024-03-21T10:37:11.804464+0000 (oldest deadline 2024-03-21T10:37:37.104077+0000) 2024-03-21T10:37:54.976 INFO:journalctl@ceph.osd.7.smithi100.stdout:Mar 21 10:37:54 smithi100 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-7[53119]: 2024-03-21T10:37:54.500+0000 7fb6ed400700 -1 osd.7 111 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-03-21T10:37:04.136587+0000 front 2024-03-21T10:37:04.136839+0000 (oldest deadline 2024-03-21T10:37:29.436935+0000) 2024-03-21T10:37:54.976 INFO:journalctl@ceph.osd.7.smithi100.stdout:Mar 21 10:37:54 smithi100 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-7[53119]: 2024-03-21T10:37:54.500+0000 7fb6ed400700 -1 osd.7 111 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-03-21T10:37:14.139283+0000 front 2024-03-21T10:37:14.139243+0000 (oldest deadline 2024-03-21T10:37:37.639505+0000) 2024-03-21T10:37:54.976 INFO:journalctl@ceph.osd.7.smithi100.stdout:Mar 21 10:37:54 smithi100 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-7[53119]: 2024-03-21T10:37:54.500+0000 7fb6ed400700 -1 osd.7 111 heartbeat_check: no reply from 172.21.15.47:6814 osd.3 since back 2024-03-21T10:37:22.243333+0000 front 2024-03-21T10:37:22.243357+0000 (oldest deadline 2024-03-21T10:37:45.742507+0000) 2024-03-21T10:37:55.093 INFO:journalctl@ceph.osd.6.smithi047.stdout:Mar 21 10:37:54 smithi047 podman[103594]: 2024-03-21 10:37:54.625528658 +0000 UTC m=+0.037867631 image pull 237e43c7a65ff48534d94b97c1bfdbe4b7be5672e3d9a613745083ea3702aa54 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232 2024-03-21T10:37:55.093 INFO:journalctl@ceph.osd.6.smithi047.stdout:Mar 21 10:37:54 smithi047 podman[103594]: 2024-03-21 10:37:54.773325112 +0000 UTC m=+0.185664119 container create 43d854a2f48043c98632af1a0731a10ea351eb4d4a69f725da394bdca5fbc09f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-6-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, ceph=True, org.label-schema.build-date=20240319, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0) 2024-03-21T10:37:55.236 INFO:journalctl@ceph.osd.8.smithi157.stdout:Mar 21 10:37:54 smithi157 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-8[54391]: 2024-03-21T10:37:54.872+0000 7fca85e00700 -1 osd.8 111 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-03-21T10:37:03.032692+0000 front 2024-03-21T10:37:03.032837+0000 (oldest deadline 2024-03-21T10:37:27.132936+0000) 2024-03-21T10:37:55.236 INFO:journalctl@ceph.osd.8.smithi157.stdout:Mar 21 10:37:54 smithi157 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-8[54391]: 2024-03-21T10:37:54.872+0000 7fca85e00700 -1 osd.8 111 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-03-21T10:37:13.535744+0000 front 2024-03-21T10:37:13.536199+0000 (oldest deadline 2024-03-21T10:37:39.435978+0000) 2024-03-21T10:37:55.236 INFO:journalctl@ceph.osd.8.smithi157.stdout:Mar 21 10:37:54 smithi157 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-8[54391]: 2024-03-21T10:37:54.872+0000 7fca85e00700 -1 osd.8 111 heartbeat_check: no reply from 172.21.15.47:6814 osd.3 since back 2024-03-21T10:37:19.437169+0000 front 2024-03-21T10:37:19.437298+0000 (oldest deadline 2024-03-21T10:37:48.838456+0000) 2024-03-21T10:37:55.454 INFO:journalctl@ceph.osd.6.smithi047.stdout:Mar 21 10:37:55 smithi047 podman[103594]: 2024-03-21 10:37:55.235383529 +0000 UTC m=+0.647722542 container init 43d854a2f48043c98632af1a0731a10ea351eb4d4a69f725da394bdca5fbc09f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-6-deactivate, GIT_CLEAN=True, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.build-date=20240319, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30) 2024-03-21T10:37:55.455 INFO:journalctl@ceph.osd.6.smithi047.stdout:Mar 21 10:37:55 smithi047 podman[103594]: 2024-03-21 10:37:55.246174689 +0000 UTC m=+0.658513670 container start 43d854a2f48043c98632af1a0731a10ea351eb4d4a69f725da394bdca5fbc09f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-6-deactivate, GIT_CLEAN=True, GIT_BRANCH=HEAD, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240319, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-21T10:37:55.455 INFO:journalctl@ceph.osd.6.smithi047.stdout:Mar 21 10:37:55 smithi047 podman[103594]: 2024-03-21 10:37:55.317571748 +0000 UTC m=+0.729910747 container attach 43d854a2f48043c98632af1a0731a10ea351eb4d4a69f725da394bdca5fbc09f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-6-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240319, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-21T10:37:55.476 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 10:37:55 smithi100 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-10[59767]: 2024-03-21T10:37:55.093+0000 7f315da00700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-03-21T10:36:52.673512+0000 front 2024-03-21T10:36:52.673688+0000 (oldest deadline 2024-03-21T10:37:16.675039+0000) 2024-03-21T10:37:55.476 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 10:37:55 smithi100 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-10[59767]: 2024-03-21T10:37:55.093+0000 7f315da00700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-03-21T10:37:05.379887+0000 front 2024-03-21T10:37:05.379536+0000 (oldest deadline 2024-03-21T10:37:30.079809+0000) 2024-03-21T10:37:55.476 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 10:37:55 smithi100 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-10[59767]: 2024-03-21T10:37:55.093+0000 7f315da00700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-03-21T10:37:12.981593+0000 front 2024-03-21T10:37:12.981483+0000 (oldest deadline 2024-03-21T10:37:38.281183+0000) 2024-03-21T10:37:55.476 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 10:37:55 smithi100 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-10[59767]: 2024-03-21T10:37:55.093+0000 7f315da00700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.47:6814 osd.3 since back 2024-03-21T10:37:23.583085+0000 front 2024-03-21T10:37:23.583243+0000 (oldest deadline 2024-03-21T10:37:45.283261+0000) 2024-03-21T10:37:55.710 INFO:journalctl@ceph.osd.6.smithi047.stdout:Mar 21 10:37:55 smithi047 podman[103594]: 2024-03-21 10:37:55.683413243 +0000 UTC m=+1.095752257 container died 43d854a2f48043c98632af1a0731a10ea351eb4d4a69f725da394bdca5fbc09f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-6-deactivate, org.label-schema.schema-version=1.0, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, org.label-schema.license=GPLv2, GIT_CLEAN=True, ceph=True, org.label-schema.build-date=20240319, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-21T10:37:55.710 INFO:journalctl@ceph.osd.9.smithi047.stdout:Mar 21 10:37:55 smithi047 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-9[64137]: 2024-03-21T10:37:55.514+0000 7f02dc200700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-03-21T10:36:54.991613+0000 front 2024-03-21T10:36:54.991458+0000 (oldest deadline 2024-03-21T10:37:17.891147+0000) 2024-03-21T10:37:55.710 INFO:journalctl@ceph.osd.9.smithi047.stdout:Mar 21 10:37:55 smithi047 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-9[64137]: 2024-03-21T10:37:55.514+0000 7f02dc200700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-03-21T10:37:02.494627+0000 front 2024-03-21T10:37:02.494473+0000 (oldest deadline 2024-03-21T10:37:26.594304+0000) 2024-03-21T10:37:55.710 INFO:journalctl@ceph.osd.9.smithi047.stdout:Mar 21 10:37:55 smithi047 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-9[64137]: 2024-03-21T10:37:55.514+0000 7f02dc200700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-03-21T10:37:13.597260+0000 front 2024-03-21T10:37:13.596840+0000 (oldest deadline 2024-03-21T10:37:37.696959+0000) 2024-03-21T10:37:55.711 INFO:journalctl@ceph.osd.9.smithi047.stdout:Mar 21 10:37:55 smithi047 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-9[64137]: 2024-03-21T10:37:55.514+0000 7f02dc200700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.47:6814 osd.3 since back 2024-03-21T10:37:21.198652+0000 front 2024-03-21T10:37:21.199080+0000 (oldest deadline 2024-03-21T10:37:45.200271+0000) 2024-03-21T10:37:55.711 INFO:journalctl@ceph.osd.9.smithi047.stdout:Mar 21 10:37:55 smithi047 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-9[64137]: 2024-03-21T10:37:55.514+0000 7f02dc200700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.100:6812 osd.4 since back 2024-03-21T10:37:31.503983+0000 front 2024-03-21T10:37:31.503595+0000 (oldest deadline 2024-03-21T10:37:55.003462+0000) 2024-03-21T10:37:55.839 INFO:journalctl@ceph.osd.7.smithi100.stdout:Mar 21 10:37:55 smithi100 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-7[53119]: 2024-03-21T10:37:55.535+0000 7fb6ed400700 -1 osd.7 111 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-03-21T10:37:04.136587+0000 front 2024-03-21T10:37:04.136839+0000 (oldest deadline 2024-03-21T10:37:29.436935+0000) 2024-03-21T10:37:55.839 INFO:journalctl@ceph.osd.7.smithi100.stdout:Mar 21 10:37:55 smithi100 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-7[53119]: 2024-03-21T10:37:55.535+0000 7fb6ed400700 -1 osd.7 111 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-03-21T10:37:14.139283+0000 front 2024-03-21T10:37:14.139243+0000 (oldest deadline 2024-03-21T10:37:37.639505+0000) 2024-03-21T10:37:55.839 INFO:journalctl@ceph.osd.7.smithi100.stdout:Mar 21 10:37:55 smithi100 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-7[53119]: 2024-03-21T10:37:55.535+0000 7fb6ed400700 -1 osd.7 111 heartbeat_check: no reply from 172.21.15.47:6814 osd.3 since back 2024-03-21T10:37:22.243333+0000 front 2024-03-21T10:37:22.243357+0000 (oldest deadline 2024-03-21T10:37:45.742507+0000) 2024-03-21T10:37:55.906 INFO:journalctl@ceph.osd.11.smithi157.stdout:Mar 21 10:37:55 smithi157 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-11[61259]: 2024-03-21T10:37:55.507+0000 7fd0f8400700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-03-21T10:36:54.398353+0000 front 2024-03-21T10:36:54.398068+0000 (oldest deadline 2024-03-21T10:37:16.698280+0000) 2024-03-21T10:37:55.907 INFO:journalctl@ceph.osd.11.smithi157.stdout:Mar 21 10:37:55 smithi157 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-11[61259]: 2024-03-21T10:37:55.507+0000 7fd0f8400700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-03-21T10:37:04.901925+0000 front 2024-03-21T10:37:04.902083+0000 (oldest deadline 2024-03-21T10:37:29.502349+0000) 2024-03-21T10:37:55.907 INFO:journalctl@ceph.osd.11.smithi157.stdout:Mar 21 10:37:55 smithi157 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-11[61259]: 2024-03-21T10:37:55.507+0000 7fd0f8400700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-03-21T10:37:11.804109+0000 front 2024-03-21T10:37:11.804464+0000 (oldest deadline 2024-03-21T10:37:37.104077+0000) 2024-03-21T10:37:55.907 INFO:journalctl@ceph.osd.11.smithi157.stdout:Mar 21 10:37:55 smithi157 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-11[61259]: 2024-03-21T10:37:55.507+0000 7fd0f8400700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.100:6812 osd.4 since back 2024-03-21T10:37:33.508959+0000 front 2024-03-21T10:37:33.509595+0000 (oldest deadline 2024-03-21T10:37:54.609171+0000) 2024-03-21T10:37:56.236 INFO:journalctl@ceph.osd.8.smithi157.stdout:Mar 21 10:37:55 smithi157 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-8[54391]: 2024-03-21T10:37:55.908+0000 7fca85e00700 -1 osd.8 111 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-03-21T10:37:03.032692+0000 front 2024-03-21T10:37:03.032837+0000 (oldest deadline 2024-03-21T10:37:27.132936+0000) 2024-03-21T10:37:56.236 INFO:journalctl@ceph.osd.8.smithi157.stdout:Mar 21 10:37:55 smithi157 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-8[54391]: 2024-03-21T10:37:55.908+0000 7fca85e00700 -1 osd.8 111 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-03-21T10:37:13.535744+0000 front 2024-03-21T10:37:13.536199+0000 (oldest deadline 2024-03-21T10:37:39.435978+0000) 2024-03-21T10:37:56.236 INFO:journalctl@ceph.osd.8.smithi157.stdout:Mar 21 10:37:55 smithi157 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-8[54391]: 2024-03-21T10:37:55.908+0000 7fca85e00700 -1 osd.8 111 heartbeat_check: no reply from 172.21.15.47:6814 osd.3 since back 2024-03-21T10:37:19.437169+0000 front 2024-03-21T10:37:19.437298+0000 (oldest deadline 2024-03-21T10:37:48.838456+0000) 2024-03-21T10:37:56.476 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 10:37:56 smithi100 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-10[59767]: 2024-03-21T10:37:56.128+0000 7f315da00700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-03-21T10:36:52.673512+0000 front 2024-03-21T10:36:52.673688+0000 (oldest deadline 2024-03-21T10:37:16.675039+0000) 2024-03-21T10:37:56.476 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 10:37:56 smithi100 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-10[59767]: 2024-03-21T10:37:56.128+0000 7f315da00700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-03-21T10:37:05.379887+0000 front 2024-03-21T10:37:05.379536+0000 (oldest deadline 2024-03-21T10:37:30.079809+0000) 2024-03-21T10:37:56.476 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 10:37:56 smithi100 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-10[59767]: 2024-03-21T10:37:56.128+0000 7f315da00700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-03-21T10:37:12.981593+0000 front 2024-03-21T10:37:12.981483+0000 (oldest deadline 2024-03-21T10:37:38.281183+0000) 2024-03-21T10:37:56.476 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 10:37:56 smithi100 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-10[59767]: 2024-03-21T10:37:56.128+0000 7f315da00700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.47:6814 osd.3 since back 2024-03-21T10:37:23.583085+0000 front 2024-03-21T10:37:23.583243+0000 (oldest deadline 2024-03-21T10:37:45.283261+0000) 2024-03-21T10:37:56.476 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 10:37:56 smithi100 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-10[59767]: 2024-03-21T10:37:56.128+0000 7f315da00700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.100:6812 osd.4 since back 2024-03-21T10:37:28.185438+0000 front 2024-03-21T10:37:28.185490+0000 (oldest deadline 2024-03-21T10:37:55.786600+0000) 2024-03-21T10:37:56.649 DEBUG:teuthology.orchestra.run.smithi047:> sudo pkill -f 'journalctl -f -n 0 -u ceph-3056d51a-e76b-11ee-95cd-87774f69a715@osd.6.service' 2024-03-21T10:37:56.736 INFO:journalctl@ceph.osd.11.smithi157.stdout:Mar 21 10:37:56 smithi157 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-11[61259]: 2024-03-21T10:37:56.458+0000 7fd0f8400700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-03-21T10:36:54.398353+0000 front 2024-03-21T10:36:54.398068+0000 (oldest deadline 2024-03-21T10:37:16.698280+0000) 2024-03-21T10:37:56.736 INFO:journalctl@ceph.osd.11.smithi157.stdout:Mar 21 10:37:56 smithi157 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-11[61259]: 2024-03-21T10:37:56.458+0000 7fd0f8400700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-03-21T10:37:04.901925+0000 front 2024-03-21T10:37:04.902083+0000 (oldest deadline 2024-03-21T10:37:29.502349+0000) 2024-03-21T10:37:56.736 INFO:journalctl@ceph.osd.11.smithi157.stdout:Mar 21 10:37:56 smithi157 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-11[61259]: 2024-03-21T10:37:56.458+0000 7fd0f8400700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-03-21T10:37:11.804109+0000 front 2024-03-21T10:37:11.804464+0000 (oldest deadline 2024-03-21T10:37:37.104077+0000) 2024-03-21T10:37:56.736 INFO:journalctl@ceph.osd.11.smithi157.stdout:Mar 21 10:37:56 smithi157 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-11[61259]: 2024-03-21T10:37:56.458+0000 7fd0f8400700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.100:6812 osd.4 since back 2024-03-21T10:37:33.508959+0000 front 2024-03-21T10:37:33.509595+0000 (oldest deadline 2024-03-21T10:37:54.609171+0000) 2024-03-21T10:37:56.757 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-21T10:37:56.758 INFO:tasks.cephadm.osd.6:Stopped osd.6 2024-03-21T10:37:56.758 INFO:tasks.cephadm.osd.7:Stopping osd.7... 2024-03-21T10:37:56.758 DEBUG:teuthology.orchestra.run.smithi100:> sudo systemctl stop ceph-3056d51a-e76b-11ee-95cd-87774f69a715@osd.7 2024-03-21T10:37:56.834 INFO:journalctl@ceph.osd.7.smithi100.stdout:Mar 21 10:37:56 smithi100 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-7[53119]: 2024-03-21T10:37:56.580+0000 7fb6ed400700 -1 osd.7 111 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-03-21T10:37:04.136587+0000 front 2024-03-21T10:37:04.136839+0000 (oldest deadline 2024-03-21T10:37:29.436935+0000) 2024-03-21T10:37:56.834 INFO:journalctl@ceph.osd.7.smithi100.stdout:Mar 21 10:37:56 smithi100 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-7[53119]: 2024-03-21T10:37:56.580+0000 7fb6ed400700 -1 osd.7 111 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-03-21T10:37:14.139283+0000 front 2024-03-21T10:37:14.139243+0000 (oldest deadline 2024-03-21T10:37:37.639505+0000) 2024-03-21T10:37:56.835 INFO:journalctl@ceph.osd.7.smithi100.stdout:Mar 21 10:37:56 smithi100 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-7[53119]: 2024-03-21T10:37:56.580+0000 7fb6ed400700 -1 osd.7 111 heartbeat_check: no reply from 172.21.15.47:6814 osd.3 since back 2024-03-21T10:37:22.243333+0000 front 2024-03-21T10:37:22.243357+0000 (oldest deadline 2024-03-21T10:37:45.742507+0000) 2024-03-21T10:37:56.961 INFO:journalctl@ceph.osd.9.smithi047.stdout:Mar 21 10:37:56 smithi047 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-9[64137]: 2024-03-21T10:37:56.558+0000 7f02dc200700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-03-21T10:36:54.991613+0000 front 2024-03-21T10:36:54.991458+0000 (oldest deadline 2024-03-21T10:37:17.891147+0000) 2024-03-21T10:37:56.961 INFO:journalctl@ceph.osd.9.smithi047.stdout:Mar 21 10:37:56 smithi047 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-9[64137]: 2024-03-21T10:37:56.558+0000 7f02dc200700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-03-21T10:37:02.494627+0000 front 2024-03-21T10:37:02.494473+0000 (oldest deadline 2024-03-21T10:37:26.594304+0000) 2024-03-21T10:37:56.961 INFO:journalctl@ceph.osd.9.smithi047.stdout:Mar 21 10:37:56 smithi047 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-9[64137]: 2024-03-21T10:37:56.558+0000 7f02dc200700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-03-21T10:37:13.597260+0000 front 2024-03-21T10:37:13.596840+0000 (oldest deadline 2024-03-21T10:37:37.696959+0000) 2024-03-21T10:37:56.961 INFO:journalctl@ceph.osd.9.smithi047.stdout:Mar 21 10:37:56 smithi047 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-9[64137]: 2024-03-21T10:37:56.558+0000 7f02dc200700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.47:6814 osd.3 since back 2024-03-21T10:37:21.198652+0000 front 2024-03-21T10:37:21.199080+0000 (oldest deadline 2024-03-21T10:37:45.200271+0000) 2024-03-21T10:37:56.961 INFO:journalctl@ceph.osd.9.smithi047.stdout:Mar 21 10:37:56 smithi047 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-9[64137]: 2024-03-21T10:37:56.558+0000 7f02dc200700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.100:6812 osd.4 since back 2024-03-21T10:37:31.503983+0000 front 2024-03-21T10:37:31.503595+0000 (oldest deadline 2024-03-21T10:37:55.003462+0000) 2024-03-21T10:37:57.150 INFO:journalctl@ceph.osd.7.smithi100.stdout:Mar 21 10:37:56 smithi100 systemd[1]: Stopping Ceph osd.7 for 3056d51a-e76b-11ee-95cd-87774f69a715... 2024-03-21T10:37:57.236 INFO:journalctl@ceph.osd.8.smithi157.stdout:Mar 21 10:37:56 smithi157 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-8[54391]: 2024-03-21T10:37:56.883+0000 7fca85e00700 -1 osd.8 111 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-03-21T10:37:03.032692+0000 front 2024-03-21T10:37:03.032837+0000 (oldest deadline 2024-03-21T10:37:27.132936+0000) 2024-03-21T10:37:57.236 INFO:journalctl@ceph.osd.8.smithi157.stdout:Mar 21 10:37:56 smithi157 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-8[54391]: 2024-03-21T10:37:56.883+0000 7fca85e00700 -1 osd.8 111 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-03-21T10:37:13.535744+0000 front 2024-03-21T10:37:13.536199+0000 (oldest deadline 2024-03-21T10:37:39.435978+0000) 2024-03-21T10:37:57.236 INFO:journalctl@ceph.osd.8.smithi157.stdout:Mar 21 10:37:56 smithi157 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-8[54391]: 2024-03-21T10:37:56.883+0000 7fca85e00700 -1 osd.8 111 heartbeat_check: no reply from 172.21.15.47:6814 osd.3 since back 2024-03-21T10:37:19.437169+0000 front 2024-03-21T10:37:19.437298+0000 (oldest deadline 2024-03-21T10:37:48.838456+0000) 2024-03-21T10:37:57.476 INFO:journalctl@ceph.osd.7.smithi100.stdout:Mar 21 10:37:57 smithi100 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-7[53119]: 2024-03-21T10:37:57.351+0000 7fb6f1c00700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.7 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-21T10:37:57.476 INFO:journalctl@ceph.osd.7.smithi100.stdout:Mar 21 10:37:57 smithi100 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-7[53119]: 2024-03-21T10:37:57.351+0000 7fb6f1c00700 -1 osd.7 111 *** Got signal Terminated *** 2024-03-21T10:37:57.477 INFO:journalctl@ceph.osd.7.smithi100.stdout:Mar 21 10:37:57 smithi100 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-7[53119]: 2024-03-21T10:37:57.351+0000 7fb6f1c00700 -1 osd.7 111 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-21T10:37:57.477 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 10:37:57 smithi100 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-10[59767]: 2024-03-21T10:37:57.152+0000 7f315da00700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-03-21T10:36:52.673512+0000 front 2024-03-21T10:36:52.673688+0000 (oldest deadline 2024-03-21T10:37:16.675039+0000) 2024-03-21T10:37:57.477 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 10:37:57 smithi100 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-10[59767]: 2024-03-21T10:37:57.152+0000 7f315da00700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-03-21T10:37:05.379887+0000 front 2024-03-21T10:37:05.379536+0000 (oldest deadline 2024-03-21T10:37:30.079809+0000) 2024-03-21T10:37:57.477 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 10:37:57 smithi100 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-10[59767]: 2024-03-21T10:37:57.152+0000 7f315da00700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-03-21T10:37:12.981593+0000 front 2024-03-21T10:37:12.981483+0000 (oldest deadline 2024-03-21T10:37:38.281183+0000) 2024-03-21T10:37:57.477 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 10:37:57 smithi100 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-10[59767]: 2024-03-21T10:37:57.152+0000 7f315da00700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.47:6814 osd.3 since back 2024-03-21T10:37:23.583085+0000 front 2024-03-21T10:37:23.583243+0000 (oldest deadline 2024-03-21T10:37:45.283261+0000) 2024-03-21T10:37:57.477 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 10:37:57 smithi100 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-10[59767]: 2024-03-21T10:37:57.152+0000 7f315da00700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.100:6812 osd.4 since back 2024-03-21T10:37:28.185438+0000 front 2024-03-21T10:37:28.185490+0000 (oldest deadline 2024-03-21T10:37:55.786600+0000) 2024-03-21T10:37:57.685 INFO:journalctl@ceph.osd.11.smithi157.stdout:Mar 21 10:37:57 smithi157 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-11[61259]: 2024-03-21T10:37:57.421+0000 7fd0f8400700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-03-21T10:36:54.398353+0000 front 2024-03-21T10:36:54.398068+0000 (oldest deadline 2024-03-21T10:37:16.698280+0000) 2024-03-21T10:37:57.685 INFO:journalctl@ceph.osd.11.smithi157.stdout:Mar 21 10:37:57 smithi157 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-11[61259]: 2024-03-21T10:37:57.421+0000 7fd0f8400700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-03-21T10:37:04.901925+0000 front 2024-03-21T10:37:04.902083+0000 (oldest deadline 2024-03-21T10:37:29.502349+0000) 2024-03-21T10:37:57.685 INFO:journalctl@ceph.osd.11.smithi157.stdout:Mar 21 10:37:57 smithi157 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-11[61259]: 2024-03-21T10:37:57.421+0000 7fd0f8400700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-03-21T10:37:11.804109+0000 front 2024-03-21T10:37:11.804464+0000 (oldest deadline 2024-03-21T10:37:37.104077+0000) 2024-03-21T10:37:57.685 INFO:journalctl@ceph.osd.11.smithi157.stdout:Mar 21 10:37:57 smithi157 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-11[61259]: 2024-03-21T10:37:57.421+0000 7fd0f8400700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.100:6812 osd.4 since back 2024-03-21T10:37:33.508959+0000 front 2024-03-21T10:37:33.509595+0000 (oldest deadline 2024-03-21T10:37:54.609171+0000) 2024-03-21T10:37:57.960 INFO:journalctl@ceph.osd.9.smithi047.stdout:Mar 21 10:37:57 smithi047 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-9[64137]: 2024-03-21T10:37:57.544+0000 7f02dc200700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-03-21T10:36:54.991613+0000 front 2024-03-21T10:36:54.991458+0000 (oldest deadline 2024-03-21T10:37:17.891147+0000) 2024-03-21T10:37:57.961 INFO:journalctl@ceph.osd.9.smithi047.stdout:Mar 21 10:37:57 smithi047 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-9[64137]: 2024-03-21T10:37:57.544+0000 7f02dc200700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-03-21T10:37:02.494627+0000 front 2024-03-21T10:37:02.494473+0000 (oldest deadline 2024-03-21T10:37:26.594304+0000) 2024-03-21T10:37:57.961 INFO:journalctl@ceph.osd.9.smithi047.stdout:Mar 21 10:37:57 smithi047 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-9[64137]: 2024-03-21T10:37:57.544+0000 7f02dc200700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-03-21T10:37:13.597260+0000 front 2024-03-21T10:37:13.596840+0000 (oldest deadline 2024-03-21T10:37:37.696959+0000) 2024-03-21T10:37:57.961 INFO:journalctl@ceph.osd.9.smithi047.stdout:Mar 21 10:37:57 smithi047 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-9[64137]: 2024-03-21T10:37:57.544+0000 7f02dc200700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.47:6814 osd.3 since back 2024-03-21T10:37:21.198652+0000 front 2024-03-21T10:37:21.199080+0000 (oldest deadline 2024-03-21T10:37:45.200271+0000) 2024-03-21T10:37:57.961 INFO:journalctl@ceph.osd.9.smithi047.stdout:Mar 21 10:37:57 smithi047 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-9[64137]: 2024-03-21T10:37:57.544+0000 7f02dc200700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.100:6812 osd.4 since back 2024-03-21T10:37:31.503983+0000 front 2024-03-21T10:37:31.503595+0000 (oldest deadline 2024-03-21T10:37:55.003462+0000) 2024-03-21T10:37:57.976 INFO:journalctl@ceph.osd.7.smithi100.stdout:Mar 21 10:37:57 smithi100 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-7[53119]: 2024-03-21T10:37:57.570+0000 7fb6ed400700 -1 osd.7 111 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-03-21T10:37:04.136587+0000 front 2024-03-21T10:37:04.136839+0000 (oldest deadline 2024-03-21T10:37:29.436935+0000) 2024-03-21T10:37:57.976 INFO:journalctl@ceph.osd.7.smithi100.stdout:Mar 21 10:37:57 smithi100 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-7[53119]: 2024-03-21T10:37:57.570+0000 7fb6ed400700 -1 osd.7 111 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-03-21T10:37:14.139283+0000 front 2024-03-21T10:37:14.139243+0000 (oldest deadline 2024-03-21T10:37:37.639505+0000) 2024-03-21T10:37:57.976 INFO:journalctl@ceph.osd.7.smithi100.stdout:Mar 21 10:37:57 smithi100 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-7[53119]: 2024-03-21T10:37:57.570+0000 7fb6ed400700 -1 osd.7 111 heartbeat_check: no reply from 172.21.15.47:6814 osd.3 since back 2024-03-21T10:37:22.243333+0000 front 2024-03-21T10:37:22.243357+0000 (oldest deadline 2024-03-21T10:37:45.742507+0000) 2024-03-21T10:37:57.976 INFO:journalctl@ceph.osd.7.smithi100.stdout:Mar 21 10:37:57 smithi100 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-7[53119]: 2024-03-21T10:37:57.570+0000 7fb6ed400700 -1 osd.7 111 heartbeat_check: no reply from 172.21.15.100:6812 osd.4 since back 2024-03-21T10:37:31.545177+0000 front 2024-03-21T10:37:31.544862+0000 (oldest deadline 2024-03-21T10:37:56.844781+0000) 2024-03-21T10:37:57.986 INFO:journalctl@ceph.osd.8.smithi157.stdout:Mar 21 10:37:57 smithi157 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-8[54391]: 2024-03-21T10:37:57.901+0000 7fca85e00700 -1 osd.8 111 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-03-21T10:37:03.032692+0000 front 2024-03-21T10:37:03.032837+0000 (oldest deadline 2024-03-21T10:37:27.132936+0000) 2024-03-21T10:37:57.986 INFO:journalctl@ceph.osd.8.smithi157.stdout:Mar 21 10:37:57 smithi157 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-8[54391]: 2024-03-21T10:37:57.901+0000 7fca85e00700 -1 osd.8 111 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-03-21T10:37:13.535744+0000 front 2024-03-21T10:37:13.536199+0000 (oldest deadline 2024-03-21T10:37:39.435978+0000) 2024-03-21T10:37:57.986 INFO:journalctl@ceph.osd.8.smithi157.stdout:Mar 21 10:37:57 smithi157 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-8[54391]: 2024-03-21T10:37:57.901+0000 7fca85e00700 -1 osd.8 111 heartbeat_check: no reply from 172.21.15.47:6814 osd.3 since back 2024-03-21T10:37:19.437169+0000 front 2024-03-21T10:37:19.437298+0000 (oldest deadline 2024-03-21T10:37:48.838456+0000) 2024-03-21T10:37:58.476 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 10:37:58 smithi100 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-10[59767]: 2024-03-21T10:37:58.149+0000 7f315da00700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-03-21T10:36:52.673512+0000 front 2024-03-21T10:36:52.673688+0000 (oldest deadline 2024-03-21T10:37:16.675039+0000) 2024-03-21T10:37:58.476 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 10:37:58 smithi100 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-10[59767]: 2024-03-21T10:37:58.149+0000 7f315da00700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-03-21T10:37:05.379887+0000 front 2024-03-21T10:37:05.379536+0000 (oldest deadline 2024-03-21T10:37:30.079809+0000) 2024-03-21T10:37:58.476 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 10:37:58 smithi100 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-10[59767]: 2024-03-21T10:37:58.149+0000 7f315da00700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-03-21T10:37:12.981593+0000 front 2024-03-21T10:37:12.981483+0000 (oldest deadline 2024-03-21T10:37:38.281183+0000) 2024-03-21T10:37:58.476 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 10:37:58 smithi100 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-10[59767]: 2024-03-21T10:37:58.149+0000 7f315da00700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.47:6814 osd.3 since back 2024-03-21T10:37:23.583085+0000 front 2024-03-21T10:37:23.583243+0000 (oldest deadline 2024-03-21T10:37:45.283261+0000) 2024-03-21T10:37:58.477 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 10:37:58 smithi100 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-10[59767]: 2024-03-21T10:37:58.149+0000 7f315da00700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.100:6812 osd.4 since back 2024-03-21T10:37:28.185438+0000 front 2024-03-21T10:37:28.185490+0000 (oldest deadline 2024-03-21T10:37:55.786600+0000) 2024-03-21T10:37:58.736 INFO:journalctl@ceph.osd.11.smithi157.stdout:Mar 21 10:37:58 smithi157 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-11[61259]: 2024-03-21T10:37:58.469+0000 7fd0f8400700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-03-21T10:36:54.398353+0000 front 2024-03-21T10:36:54.398068+0000 (oldest deadline 2024-03-21T10:37:16.698280+0000) 2024-03-21T10:37:58.736 INFO:journalctl@ceph.osd.11.smithi157.stdout:Mar 21 10:37:58 smithi157 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-11[61259]: 2024-03-21T10:37:58.469+0000 7fd0f8400700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-03-21T10:37:04.901925+0000 front 2024-03-21T10:37:04.902083+0000 (oldest deadline 2024-03-21T10:37:29.502349+0000) 2024-03-21T10:37:58.737 INFO:journalctl@ceph.osd.11.smithi157.stdout:Mar 21 10:37:58 smithi157 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-11[61259]: 2024-03-21T10:37:58.469+0000 7fd0f8400700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-03-21T10:37:11.804109+0000 front 2024-03-21T10:37:11.804464+0000 (oldest deadline 2024-03-21T10:37:37.104077+0000) 2024-03-21T10:37:58.737 INFO:journalctl@ceph.osd.11.smithi157.stdout:Mar 21 10:37:58 smithi157 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-11[61259]: 2024-03-21T10:37:58.469+0000 7fd0f8400700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.100:6812 osd.4 since back 2024-03-21T10:37:33.508959+0000 front 2024-03-21T10:37:33.509595+0000 (oldest deadline 2024-03-21T10:37:54.609171+0000) 2024-03-21T10:37:58.960 INFO:journalctl@ceph.osd.9.smithi047.stdout:Mar 21 10:37:58 smithi047 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-9[64137]: 2024-03-21T10:37:58.523+0000 7f02dc200700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-03-21T10:36:54.991613+0000 front 2024-03-21T10:36:54.991458+0000 (oldest deadline 2024-03-21T10:37:17.891147+0000) 2024-03-21T10:37:58.961 INFO:journalctl@ceph.osd.9.smithi047.stdout:Mar 21 10:37:58 smithi047 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-9[64137]: 2024-03-21T10:37:58.523+0000 7f02dc200700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-03-21T10:37:02.494627+0000 front 2024-03-21T10:37:02.494473+0000 (oldest deadline 2024-03-21T10:37:26.594304+0000) 2024-03-21T10:37:58.961 INFO:journalctl@ceph.osd.9.smithi047.stdout:Mar 21 10:37:58 smithi047 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-9[64137]: 2024-03-21T10:37:58.523+0000 7f02dc200700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-03-21T10:37:13.597260+0000 front 2024-03-21T10:37:13.596840+0000 (oldest deadline 2024-03-21T10:37:37.696959+0000) 2024-03-21T10:37:58.961 INFO:journalctl@ceph.osd.9.smithi047.stdout:Mar 21 10:37:58 smithi047 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-9[64137]: 2024-03-21T10:37:58.523+0000 7f02dc200700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.47:6814 osd.3 since back 2024-03-21T10:37:21.198652+0000 front 2024-03-21T10:37:21.199080+0000 (oldest deadline 2024-03-21T10:37:45.200271+0000) 2024-03-21T10:37:58.961 INFO:journalctl@ceph.osd.9.smithi047.stdout:Mar 21 10:37:58 smithi047 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-9[64137]: 2024-03-21T10:37:58.523+0000 7f02dc200700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.100:6812 osd.4 since back 2024-03-21T10:37:31.503983+0000 front 2024-03-21T10:37:31.503595+0000 (oldest deadline 2024-03-21T10:37:55.003462+0000) 2024-03-21T10:37:58.976 INFO:journalctl@ceph.osd.7.smithi100.stdout:Mar 21 10:37:58 smithi100 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-7[53119]: 2024-03-21T10:37:58.573+0000 7fb6ed400700 -1 osd.7 111 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-03-21T10:37:04.136587+0000 front 2024-03-21T10:37:04.136839+0000 (oldest deadline 2024-03-21T10:37:29.436935+0000) 2024-03-21T10:37:58.976 INFO:journalctl@ceph.osd.7.smithi100.stdout:Mar 21 10:37:58 smithi100 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-7[53119]: 2024-03-21T10:37:58.573+0000 7fb6ed400700 -1 osd.7 111 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-03-21T10:37:14.139283+0000 front 2024-03-21T10:37:14.139243+0000 (oldest deadline 2024-03-21T10:37:37.639505+0000) 2024-03-21T10:37:58.976 INFO:journalctl@ceph.osd.7.smithi100.stdout:Mar 21 10:37:58 smithi100 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-7[53119]: 2024-03-21T10:37:58.573+0000 7fb6ed400700 -1 osd.7 111 heartbeat_check: no reply from 172.21.15.47:6814 osd.3 since back 2024-03-21T10:37:22.243333+0000 front 2024-03-21T10:37:22.243357+0000 (oldest deadline 2024-03-21T10:37:45.742507+0000) 2024-03-21T10:37:58.976 INFO:journalctl@ceph.osd.7.smithi100.stdout:Mar 21 10:37:58 smithi100 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-7[53119]: 2024-03-21T10:37:58.573+0000 7fb6ed400700 -1 osd.7 111 heartbeat_check: no reply from 172.21.15.100:6812 osd.4 since back 2024-03-21T10:37:31.545177+0000 front 2024-03-21T10:37:31.544862+0000 (oldest deadline 2024-03-21T10:37:56.844781+0000) 2024-03-21T10:37:59.236 INFO:journalctl@ceph.osd.8.smithi157.stdout:Mar 21 10:37:58 smithi157 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-8[54391]: 2024-03-21T10:37:58.871+0000 7fca85e00700 -1 osd.8 111 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-03-21T10:37:03.032692+0000 front 2024-03-21T10:37:03.032837+0000 (oldest deadline 2024-03-21T10:37:27.132936+0000) 2024-03-21T10:37:59.236 INFO:journalctl@ceph.osd.8.smithi157.stdout:Mar 21 10:37:58 smithi157 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-8[54391]: 2024-03-21T10:37:58.871+0000 7fca85e00700 -1 osd.8 111 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-03-21T10:37:13.535744+0000 front 2024-03-21T10:37:13.536199+0000 (oldest deadline 2024-03-21T10:37:39.435978+0000) 2024-03-21T10:37:59.236 INFO:journalctl@ceph.osd.8.smithi157.stdout:Mar 21 10:37:58 smithi157 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-8[54391]: 2024-03-21T10:37:58.871+0000 7fca85e00700 -1 osd.8 111 heartbeat_check: no reply from 172.21.15.47:6814 osd.3 since back 2024-03-21T10:37:19.437169+0000 front 2024-03-21T10:37:19.437298+0000 (oldest deadline 2024-03-21T10:37:48.838456+0000) 2024-03-21T10:37:59.476 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 10:37:59 smithi100 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-10[59767]: 2024-03-21T10:37:59.180+0000 7f315da00700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-03-21T10:36:52.673512+0000 front 2024-03-21T10:36:52.673688+0000 (oldest deadline 2024-03-21T10:37:16.675039+0000) 2024-03-21T10:37:59.476 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 10:37:59 smithi100 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-10[59767]: 2024-03-21T10:37:59.180+0000 7f315da00700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-03-21T10:37:05.379887+0000 front 2024-03-21T10:37:05.379536+0000 (oldest deadline 2024-03-21T10:37:30.079809+0000) 2024-03-21T10:37:59.476 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 10:37:59 smithi100 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-10[59767]: 2024-03-21T10:37:59.180+0000 7f315da00700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-03-21T10:37:12.981593+0000 front 2024-03-21T10:37:12.981483+0000 (oldest deadline 2024-03-21T10:37:38.281183+0000) 2024-03-21T10:37:59.476 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 10:37:59 smithi100 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-10[59767]: 2024-03-21T10:37:59.180+0000 7f315da00700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.47:6814 osd.3 since back 2024-03-21T10:37:23.583085+0000 front 2024-03-21T10:37:23.583243+0000 (oldest deadline 2024-03-21T10:37:45.283261+0000) 2024-03-21T10:37:59.477 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 10:37:59 smithi100 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-10[59767]: 2024-03-21T10:37:59.180+0000 7f315da00700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.100:6812 osd.4 since back 2024-03-21T10:37:28.185438+0000 front 2024-03-21T10:37:28.185490+0000 (oldest deadline 2024-03-21T10:37:55.786600+0000) 2024-03-21T10:37:59.736 INFO:journalctl@ceph.osd.11.smithi157.stdout:Mar 21 10:37:59 smithi157 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-11[61259]: 2024-03-21T10:37:59.483+0000 7fd0f8400700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-03-21T10:36:54.398353+0000 front 2024-03-21T10:36:54.398068+0000 (oldest deadline 2024-03-21T10:37:16.698280+0000) 2024-03-21T10:37:59.736 INFO:journalctl@ceph.osd.11.smithi157.stdout:Mar 21 10:37:59 smithi157 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-11[61259]: 2024-03-21T10:37:59.483+0000 7fd0f8400700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-03-21T10:37:04.901925+0000 front 2024-03-21T10:37:04.902083+0000 (oldest deadline 2024-03-21T10:37:29.502349+0000) 2024-03-21T10:37:59.737 INFO:journalctl@ceph.osd.11.smithi157.stdout:Mar 21 10:37:59 smithi157 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-11[61259]: 2024-03-21T10:37:59.483+0000 7fd0f8400700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-03-21T10:37:11.804109+0000 front 2024-03-21T10:37:11.804464+0000 (oldest deadline 2024-03-21T10:37:37.104077+0000) 2024-03-21T10:37:59.737 INFO:journalctl@ceph.osd.11.smithi157.stdout:Mar 21 10:37:59 smithi157 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-11[61259]: 2024-03-21T10:37:59.483+0000 7fd0f8400700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.100:6812 osd.4 since back 2024-03-21T10:37:33.508959+0000 front 2024-03-21T10:37:33.509595+0000 (oldest deadline 2024-03-21T10:37:54.609171+0000) 2024-03-21T10:37:59.960 INFO:journalctl@ceph.osd.9.smithi047.stdout:Mar 21 10:37:59 smithi047 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-9[64137]: 2024-03-21T10:37:59.532+0000 7f02dc200700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-03-21T10:36:54.991613+0000 front 2024-03-21T10:36:54.991458+0000 (oldest deadline 2024-03-21T10:37:17.891147+0000) 2024-03-21T10:37:59.961 INFO:journalctl@ceph.osd.9.smithi047.stdout:Mar 21 10:37:59 smithi047 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-9[64137]: 2024-03-21T10:37:59.532+0000 7f02dc200700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-03-21T10:37:02.494627+0000 front 2024-03-21T10:37:02.494473+0000 (oldest deadline 2024-03-21T10:37:26.594304+0000) 2024-03-21T10:37:59.961 INFO:journalctl@ceph.osd.9.smithi047.stdout:Mar 21 10:37:59 smithi047 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-9[64137]: 2024-03-21T10:37:59.532+0000 7f02dc200700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-03-21T10:37:13.597260+0000 front 2024-03-21T10:37:13.596840+0000 (oldest deadline 2024-03-21T10:37:37.696959+0000) 2024-03-21T10:37:59.961 INFO:journalctl@ceph.osd.9.smithi047.stdout:Mar 21 10:37:59 smithi047 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-9[64137]: 2024-03-21T10:37:59.532+0000 7f02dc200700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.47:6814 osd.3 since back 2024-03-21T10:37:21.198652+0000 front 2024-03-21T10:37:21.199080+0000 (oldest deadline 2024-03-21T10:37:45.200271+0000) 2024-03-21T10:37:59.961 INFO:journalctl@ceph.osd.9.smithi047.stdout:Mar 21 10:37:59 smithi047 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-9[64137]: 2024-03-21T10:37:59.532+0000 7f02dc200700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.100:6812 osd.4 since back 2024-03-21T10:37:31.503983+0000 front 2024-03-21T10:37:31.503595+0000 (oldest deadline 2024-03-21T10:37:55.003462+0000) 2024-03-21T10:37:59.977 INFO:journalctl@ceph.osd.7.smithi100.stdout:Mar 21 10:37:59 smithi100 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-7[53119]: 2024-03-21T10:37:59.608+0000 7fb6ed400700 -1 osd.7 111 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-03-21T10:37:04.136587+0000 front 2024-03-21T10:37:04.136839+0000 (oldest deadline 2024-03-21T10:37:29.436935+0000) 2024-03-21T10:37:59.977 INFO:journalctl@ceph.osd.7.smithi100.stdout:Mar 21 10:37:59 smithi100 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-7[53119]: 2024-03-21T10:37:59.608+0000 7fb6ed400700 -1 osd.7 111 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-03-21T10:37:14.139283+0000 front 2024-03-21T10:37:14.139243+0000 (oldest deadline 2024-03-21T10:37:37.639505+0000) 2024-03-21T10:37:59.977 INFO:journalctl@ceph.osd.7.smithi100.stdout:Mar 21 10:37:59 smithi100 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-7[53119]: 2024-03-21T10:37:59.608+0000 7fb6ed400700 -1 osd.7 111 heartbeat_check: no reply from 172.21.15.47:6814 osd.3 since back 2024-03-21T10:37:22.243333+0000 front 2024-03-21T10:37:22.243357+0000 (oldest deadline 2024-03-21T10:37:45.742507+0000) 2024-03-21T10:37:59.977 INFO:journalctl@ceph.osd.7.smithi100.stdout:Mar 21 10:37:59 smithi100 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-7[53119]: 2024-03-21T10:37:59.608+0000 7fb6ed400700 -1 osd.7 111 heartbeat_check: no reply from 172.21.15.100:6812 osd.4 since back 2024-03-21T10:37:31.545177+0000 front 2024-03-21T10:37:31.544862+0000 (oldest deadline 2024-03-21T10:37:56.844781+0000) 2024-03-21T10:38:00.236 INFO:journalctl@ceph.osd.8.smithi157.stdout:Mar 21 10:37:59 smithi157 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-8[54391]: 2024-03-21T10:37:59.825+0000 7fca85e00700 -1 osd.8 111 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-03-21T10:37:03.032692+0000 front 2024-03-21T10:37:03.032837+0000 (oldest deadline 2024-03-21T10:37:27.132936+0000) 2024-03-21T10:38:00.236 INFO:journalctl@ceph.osd.8.smithi157.stdout:Mar 21 10:37:59 smithi157 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-8[54391]: 2024-03-21T10:37:59.825+0000 7fca85e00700 -1 osd.8 111 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-03-21T10:37:13.535744+0000 front 2024-03-21T10:37:13.536199+0000 (oldest deadline 2024-03-21T10:37:39.435978+0000) 2024-03-21T10:38:00.236 INFO:journalctl@ceph.osd.8.smithi157.stdout:Mar 21 10:37:59 smithi157 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-8[54391]: 2024-03-21T10:37:59.825+0000 7fca85e00700 -1 osd.8 111 heartbeat_check: no reply from 172.21.15.47:6814 osd.3 since back 2024-03-21T10:37:19.437169+0000 front 2024-03-21T10:37:19.437298+0000 (oldest deadline 2024-03-21T10:37:48.838456+0000) 2024-03-21T10:38:00.476 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 10:38:00 smithi100 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-10[59767]: 2024-03-21T10:38:00.149+0000 7f315da00700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-03-21T10:36:52.673512+0000 front 2024-03-21T10:36:52.673688+0000 (oldest deadline 2024-03-21T10:37:16.675039+0000) 2024-03-21T10:38:00.476 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 10:38:00 smithi100 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-10[59767]: 2024-03-21T10:38:00.149+0000 7f315da00700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-03-21T10:37:05.379887+0000 front 2024-03-21T10:37:05.379536+0000 (oldest deadline 2024-03-21T10:37:30.079809+0000) 2024-03-21T10:38:00.476 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 10:38:00 smithi100 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-10[59767]: 2024-03-21T10:38:00.149+0000 7f315da00700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-03-21T10:37:12.981593+0000 front 2024-03-21T10:37:12.981483+0000 (oldest deadline 2024-03-21T10:37:38.281183+0000) 2024-03-21T10:38:00.477 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 10:38:00 smithi100 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-10[59767]: 2024-03-21T10:38:00.149+0000 7f315da00700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.47:6814 osd.3 since back 2024-03-21T10:37:23.583085+0000 front 2024-03-21T10:37:23.583243+0000 (oldest deadline 2024-03-21T10:37:45.283261+0000) 2024-03-21T10:38:00.477 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 10:38:00 smithi100 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-10[59767]: 2024-03-21T10:38:00.149+0000 7f315da00700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.100:6812 osd.4 since back 2024-03-21T10:37:28.185438+0000 front 2024-03-21T10:37:28.185490+0000 (oldest deadline 2024-03-21T10:37:55.786600+0000) 2024-03-21T10:38:00.736 INFO:journalctl@ceph.osd.11.smithi157.stdout:Mar 21 10:38:00 smithi157 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-11[61259]: 2024-03-21T10:38:00.459+0000 7fd0f8400700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-03-21T10:36:54.398353+0000 front 2024-03-21T10:36:54.398068+0000 (oldest deadline 2024-03-21T10:37:16.698280+0000) 2024-03-21T10:38:00.736 INFO:journalctl@ceph.osd.11.smithi157.stdout:Mar 21 10:38:00 smithi157 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-11[61259]: 2024-03-21T10:38:00.459+0000 7fd0f8400700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-03-21T10:37:04.901925+0000 front 2024-03-21T10:37:04.902083+0000 (oldest deadline 2024-03-21T10:37:29.502349+0000) 2024-03-21T10:38:00.737 INFO:journalctl@ceph.osd.11.smithi157.stdout:Mar 21 10:38:00 smithi157 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-11[61259]: 2024-03-21T10:38:00.459+0000 7fd0f8400700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-03-21T10:37:11.804109+0000 front 2024-03-21T10:37:11.804464+0000 (oldest deadline 2024-03-21T10:37:37.104077+0000) 2024-03-21T10:38:00.737 INFO:journalctl@ceph.osd.11.smithi157.stdout:Mar 21 10:38:00 smithi157 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-11[61259]: 2024-03-21T10:38:00.459+0000 7fd0f8400700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.100:6812 osd.4 since back 2024-03-21T10:37:33.508959+0000 front 2024-03-21T10:37:33.509595+0000 (oldest deadline 2024-03-21T10:37:54.609171+0000) 2024-03-21T10:38:00.960 INFO:journalctl@ceph.osd.9.smithi047.stdout:Mar 21 10:38:00 smithi047 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-9[64137]: 2024-03-21T10:38:00.562+0000 7f02dc200700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-03-21T10:36:54.991613+0000 front 2024-03-21T10:36:54.991458+0000 (oldest deadline 2024-03-21T10:37:17.891147+0000) 2024-03-21T10:38:00.961 INFO:journalctl@ceph.osd.9.smithi047.stdout:Mar 21 10:38:00 smithi047 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-9[64137]: 2024-03-21T10:38:00.562+0000 7f02dc200700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-03-21T10:37:02.494627+0000 front 2024-03-21T10:37:02.494473+0000 (oldest deadline 2024-03-21T10:37:26.594304+0000) 2024-03-21T10:38:00.961 INFO:journalctl@ceph.osd.9.smithi047.stdout:Mar 21 10:38:00 smithi047 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-9[64137]: 2024-03-21T10:38:00.562+0000 7f02dc200700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-03-21T10:37:13.597260+0000 front 2024-03-21T10:37:13.596840+0000 (oldest deadline 2024-03-21T10:37:37.696959+0000) 2024-03-21T10:38:00.961 INFO:journalctl@ceph.osd.9.smithi047.stdout:Mar 21 10:38:00 smithi047 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-9[64137]: 2024-03-21T10:38:00.562+0000 7f02dc200700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.47:6814 osd.3 since back 2024-03-21T10:37:21.198652+0000 front 2024-03-21T10:37:21.199080+0000 (oldest deadline 2024-03-21T10:37:45.200271+0000) 2024-03-21T10:38:00.961 INFO:journalctl@ceph.osd.9.smithi047.stdout:Mar 21 10:38:00 smithi047 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-9[64137]: 2024-03-21T10:38:00.562+0000 7f02dc200700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.100:6812 osd.4 since back 2024-03-21T10:37:31.503983+0000 front 2024-03-21T10:37:31.503595+0000 (oldest deadline 2024-03-21T10:37:55.003462+0000) 2024-03-21T10:38:00.976 INFO:journalctl@ceph.osd.7.smithi100.stdout:Mar 21 10:38:00 smithi100 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-7[53119]: 2024-03-21T10:38:00.632+0000 7fb6ed400700 -1 osd.7 111 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-03-21T10:37:04.136587+0000 front 2024-03-21T10:37:04.136839+0000 (oldest deadline 2024-03-21T10:37:29.436935+0000) 2024-03-21T10:38:00.976 INFO:journalctl@ceph.osd.7.smithi100.stdout:Mar 21 10:38:00 smithi100 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-7[53119]: 2024-03-21T10:38:00.632+0000 7fb6ed400700 -1 osd.7 111 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-03-21T10:37:14.139283+0000 front 2024-03-21T10:37:14.139243+0000 (oldest deadline 2024-03-21T10:37:37.639505+0000) 2024-03-21T10:38:00.976 INFO:journalctl@ceph.osd.7.smithi100.stdout:Mar 21 10:38:00 smithi100 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-7[53119]: 2024-03-21T10:38:00.632+0000 7fb6ed400700 -1 osd.7 111 heartbeat_check: no reply from 172.21.15.47:6814 osd.3 since back 2024-03-21T10:37:22.243333+0000 front 2024-03-21T10:37:22.243357+0000 (oldest deadline 2024-03-21T10:37:45.742507+0000) 2024-03-21T10:38:00.976 INFO:journalctl@ceph.osd.7.smithi100.stdout:Mar 21 10:38:00 smithi100 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-7[53119]: 2024-03-21T10:38:00.632+0000 7fb6ed400700 -1 osd.7 111 heartbeat_check: no reply from 172.21.15.100:6812 osd.4 since back 2024-03-21T10:37:31.545177+0000 front 2024-03-21T10:37:31.544862+0000 (oldest deadline 2024-03-21T10:37:56.844781+0000) 2024-03-21T10:38:01.236 INFO:journalctl@ceph.osd.8.smithi157.stdout:Mar 21 10:38:00 smithi157 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-8[54391]: 2024-03-21T10:38:00.802+0000 7fca85e00700 -1 osd.8 111 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-03-21T10:37:03.032692+0000 front 2024-03-21T10:37:03.032837+0000 (oldest deadline 2024-03-21T10:37:27.132936+0000) 2024-03-21T10:38:01.237 INFO:journalctl@ceph.osd.8.smithi157.stdout:Mar 21 10:38:00 smithi157 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-8[54391]: 2024-03-21T10:38:00.802+0000 7fca85e00700 -1 osd.8 111 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-03-21T10:37:13.535744+0000 front 2024-03-21T10:37:13.536199+0000 (oldest deadline 2024-03-21T10:37:39.435978+0000) 2024-03-21T10:38:01.237 INFO:journalctl@ceph.osd.8.smithi157.stdout:Mar 21 10:38:00 smithi157 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-8[54391]: 2024-03-21T10:38:00.802+0000 7fca85e00700 -1 osd.8 111 heartbeat_check: no reply from 172.21.15.47:6814 osd.3 since back 2024-03-21T10:37:19.437169+0000 front 2024-03-21T10:37:19.437298+0000 (oldest deadline 2024-03-21T10:37:48.838456+0000) 2024-03-21T10:38:01.237 INFO:journalctl@ceph.osd.8.smithi157.stdout:Mar 21 10:38:00 smithi157 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-8[54391]: 2024-03-21T10:38:00.802+0000 7fca85e00700 -1 osd.8 111 heartbeat_check: no reply from 172.21.15.100:6812 osd.4 since back 2024-03-21T10:37:28.839776+0000 front 2024-03-21T10:37:28.839421+0000 (oldest deadline 2024-03-21T10:38:00.040369+0000) 2024-03-21T10:38:01.476 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 10:38:01 smithi100 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-10[59767]: 2024-03-21T10:38:01.146+0000 7f315da00700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-03-21T10:36:52.673512+0000 front 2024-03-21T10:36:52.673688+0000 (oldest deadline 2024-03-21T10:37:16.675039+0000) 2024-03-21T10:38:01.476 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 10:38:01 smithi100 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-10[59767]: 2024-03-21T10:38:01.146+0000 7f315da00700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-03-21T10:37:05.379887+0000 front 2024-03-21T10:37:05.379536+0000 (oldest deadline 2024-03-21T10:37:30.079809+0000) 2024-03-21T10:38:01.476 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 10:38:01 smithi100 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-10[59767]: 2024-03-21T10:38:01.146+0000 7f315da00700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-03-21T10:37:12.981593+0000 front 2024-03-21T10:37:12.981483+0000 (oldest deadline 2024-03-21T10:37:38.281183+0000) 2024-03-21T10:38:01.476 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 10:38:01 smithi100 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-10[59767]: 2024-03-21T10:38:01.146+0000 7f315da00700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.47:6814 osd.3 since back 2024-03-21T10:37:23.583085+0000 front 2024-03-21T10:37:23.583243+0000 (oldest deadline 2024-03-21T10:37:45.283261+0000) 2024-03-21T10:38:01.477 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 10:38:01 smithi100 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-10[59767]: 2024-03-21T10:38:01.146+0000 7f315da00700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.100:6812 osd.4 since back 2024-03-21T10:37:28.185438+0000 front 2024-03-21T10:37:28.185490+0000 (oldest deadline 2024-03-21T10:37:55.786600+0000) 2024-03-21T10:38:01.736 INFO:journalctl@ceph.osd.11.smithi157.stdout:Mar 21 10:38:01 smithi157 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-11[61259]: 2024-03-21T10:38:01.438+0000 7fd0f8400700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-03-21T10:36:54.398353+0000 front 2024-03-21T10:36:54.398068+0000 (oldest deadline 2024-03-21T10:37:16.698280+0000) 2024-03-21T10:38:01.736 INFO:journalctl@ceph.osd.11.smithi157.stdout:Mar 21 10:38:01 smithi157 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-11[61259]: 2024-03-21T10:38:01.438+0000 7fd0f8400700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-03-21T10:37:04.901925+0000 front 2024-03-21T10:37:04.902083+0000 (oldest deadline 2024-03-21T10:37:29.502349+0000) 2024-03-21T10:38:01.737 INFO:journalctl@ceph.osd.11.smithi157.stdout:Mar 21 10:38:01 smithi157 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-11[61259]: 2024-03-21T10:38:01.438+0000 7fd0f8400700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-03-21T10:37:11.804109+0000 front 2024-03-21T10:37:11.804464+0000 (oldest deadline 2024-03-21T10:37:37.104077+0000) 2024-03-21T10:38:01.737 INFO:journalctl@ceph.osd.11.smithi157.stdout:Mar 21 10:38:01 smithi157 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-11[61259]: 2024-03-21T10:38:01.438+0000 7fd0f8400700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.100:6812 osd.4 since back 2024-03-21T10:37:33.508959+0000 front 2024-03-21T10:37:33.509595+0000 (oldest deadline 2024-03-21T10:37:54.609171+0000) 2024-03-21T10:38:01.960 INFO:journalctl@ceph.osd.9.smithi047.stdout:Mar 21 10:38:01 smithi047 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-9[64137]: 2024-03-21T10:38:01.575+0000 7f02dc200700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-03-21T10:36:54.991613+0000 front 2024-03-21T10:36:54.991458+0000 (oldest deadline 2024-03-21T10:37:17.891147+0000) 2024-03-21T10:38:01.961 INFO:journalctl@ceph.osd.9.smithi047.stdout:Mar 21 10:38:01 smithi047 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-9[64137]: 2024-03-21T10:38:01.575+0000 7f02dc200700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-03-21T10:37:02.494627+0000 front 2024-03-21T10:37:02.494473+0000 (oldest deadline 2024-03-21T10:37:26.594304+0000) 2024-03-21T10:38:01.961 INFO:journalctl@ceph.osd.9.smithi047.stdout:Mar 21 10:38:01 smithi047 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-9[64137]: 2024-03-21T10:38:01.575+0000 7f02dc200700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-03-21T10:37:13.597260+0000 front 2024-03-21T10:37:13.596840+0000 (oldest deadline 2024-03-21T10:37:37.696959+0000) 2024-03-21T10:38:01.961 INFO:journalctl@ceph.osd.9.smithi047.stdout:Mar 21 10:38:01 smithi047 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-9[64137]: 2024-03-21T10:38:01.575+0000 7f02dc200700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.47:6814 osd.3 since back 2024-03-21T10:37:21.198652+0000 front 2024-03-21T10:37:21.199080+0000 (oldest deadline 2024-03-21T10:37:45.200271+0000) 2024-03-21T10:38:01.961 INFO:journalctl@ceph.osd.9.smithi047.stdout:Mar 21 10:38:01 smithi047 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-9[64137]: 2024-03-21T10:38:01.575+0000 7f02dc200700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.100:6812 osd.4 since back 2024-03-21T10:37:31.503983+0000 front 2024-03-21T10:37:31.503595+0000 (oldest deadline 2024-03-21T10:37:55.003462+0000) 2024-03-21T10:38:01.976 INFO:journalctl@ceph.osd.7.smithi100.stdout:Mar 21 10:38:01 smithi100 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-7[53119]: 2024-03-21T10:38:01.652+0000 7fb6ed400700 -1 osd.7 111 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-03-21T10:37:04.136587+0000 front 2024-03-21T10:37:04.136839+0000 (oldest deadline 2024-03-21T10:37:29.436935+0000) 2024-03-21T10:38:01.976 INFO:journalctl@ceph.osd.7.smithi100.stdout:Mar 21 10:38:01 smithi100 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-7[53119]: 2024-03-21T10:38:01.652+0000 7fb6ed400700 -1 osd.7 111 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-03-21T10:37:14.139283+0000 front 2024-03-21T10:37:14.139243+0000 (oldest deadline 2024-03-21T10:37:37.639505+0000) 2024-03-21T10:38:01.976 INFO:journalctl@ceph.osd.7.smithi100.stdout:Mar 21 10:38:01 smithi100 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-7[53119]: 2024-03-21T10:38:01.652+0000 7fb6ed400700 -1 osd.7 111 heartbeat_check: no reply from 172.21.15.47:6814 osd.3 since back 2024-03-21T10:37:22.243333+0000 front 2024-03-21T10:37:22.243357+0000 (oldest deadline 2024-03-21T10:37:45.742507+0000) 2024-03-21T10:38:01.976 INFO:journalctl@ceph.osd.7.smithi100.stdout:Mar 21 10:38:01 smithi100 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-7[53119]: 2024-03-21T10:38:01.652+0000 7fb6ed400700 -1 osd.7 111 heartbeat_check: no reply from 172.21.15.100:6812 osd.4 since back 2024-03-21T10:37:31.545177+0000 front 2024-03-21T10:37:31.544862+0000 (oldest deadline 2024-03-21T10:37:56.844781+0000) 2024-03-21T10:38:02.236 INFO:journalctl@ceph.osd.8.smithi157.stdout:Mar 21 10:38:01 smithi157 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-8[54391]: 2024-03-21T10:38:01.847+0000 7fca85e00700 -1 osd.8 111 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-03-21T10:37:03.032692+0000 front 2024-03-21T10:37:03.032837+0000 (oldest deadline 2024-03-21T10:37:27.132936+0000) 2024-03-21T10:38:02.236 INFO:journalctl@ceph.osd.8.smithi157.stdout:Mar 21 10:38:01 smithi157 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-8[54391]: 2024-03-21T10:38:01.847+0000 7fca85e00700 -1 osd.8 111 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-03-21T10:37:13.535744+0000 front 2024-03-21T10:37:13.536199+0000 (oldest deadline 2024-03-21T10:37:39.435978+0000) 2024-03-21T10:38:02.237 INFO:journalctl@ceph.osd.8.smithi157.stdout:Mar 21 10:38:01 smithi157 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-8[54391]: 2024-03-21T10:38:01.847+0000 7fca85e00700 -1 osd.8 111 heartbeat_check: no reply from 172.21.15.47:6814 osd.3 since back 2024-03-21T10:37:19.437169+0000 front 2024-03-21T10:37:19.437298+0000 (oldest deadline 2024-03-21T10:37:48.838456+0000) 2024-03-21T10:38:02.237 INFO:journalctl@ceph.osd.8.smithi157.stdout:Mar 21 10:38:01 smithi157 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-8[54391]: 2024-03-21T10:38:01.847+0000 7fca85e00700 -1 osd.8 111 heartbeat_check: no reply from 172.21.15.100:6812 osd.4 since back 2024-03-21T10:37:28.839776+0000 front 2024-03-21T10:37:28.839421+0000 (oldest deadline 2024-03-21T10:38:00.040369+0000) 2024-03-21T10:38:02.476 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 10:38:02 smithi100 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-10[59767]: 2024-03-21T10:38:02.125+0000 7f315da00700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-03-21T10:36:52.673512+0000 front 2024-03-21T10:36:52.673688+0000 (oldest deadline 2024-03-21T10:37:16.675039+0000) 2024-03-21T10:38:02.476 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 10:38:02 smithi100 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-10[59767]: 2024-03-21T10:38:02.125+0000 7f315da00700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-03-21T10:37:05.379887+0000 front 2024-03-21T10:37:05.379536+0000 (oldest deadline 2024-03-21T10:37:30.079809+0000) 2024-03-21T10:38:02.476 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 10:38:02 smithi100 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-10[59767]: 2024-03-21T10:38:02.125+0000 7f315da00700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-03-21T10:37:12.981593+0000 front 2024-03-21T10:37:12.981483+0000 (oldest deadline 2024-03-21T10:37:38.281183+0000) 2024-03-21T10:38:02.477 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 10:38:02 smithi100 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-10[59767]: 2024-03-21T10:38:02.125+0000 7f315da00700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.47:6814 osd.3 since back 2024-03-21T10:37:23.583085+0000 front 2024-03-21T10:37:23.583243+0000 (oldest deadline 2024-03-21T10:37:45.283261+0000) 2024-03-21T10:38:02.477 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 10:38:02 smithi100 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-10[59767]: 2024-03-21T10:38:02.125+0000 7f315da00700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.100:6812 osd.4 since back 2024-03-21T10:37:28.185438+0000 front 2024-03-21T10:37:28.185490+0000 (oldest deadline 2024-03-21T10:37:55.786600+0000) 2024-03-21T10:38:02.736 INFO:journalctl@ceph.osd.11.smithi157.stdout:Mar 21 10:38:02 smithi157 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-11[61259]: 2024-03-21T10:38:02.439+0000 7fd0f8400700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-03-21T10:36:54.398353+0000 front 2024-03-21T10:36:54.398068+0000 (oldest deadline 2024-03-21T10:37:16.698280+0000) 2024-03-21T10:38:02.736 INFO:journalctl@ceph.osd.11.smithi157.stdout:Mar 21 10:38:02 smithi157 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-11[61259]: 2024-03-21T10:38:02.439+0000 7fd0f8400700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-03-21T10:37:04.901925+0000 front 2024-03-21T10:37:04.902083+0000 (oldest deadline 2024-03-21T10:37:29.502349+0000) 2024-03-21T10:38:02.737 INFO:journalctl@ceph.osd.11.smithi157.stdout:Mar 21 10:38:02 smithi157 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-11[61259]: 2024-03-21T10:38:02.439+0000 7fd0f8400700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-03-21T10:37:11.804109+0000 front 2024-03-21T10:37:11.804464+0000 (oldest deadline 2024-03-21T10:37:37.104077+0000) 2024-03-21T10:38:02.737 INFO:journalctl@ceph.osd.11.smithi157.stdout:Mar 21 10:38:02 smithi157 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-11[61259]: 2024-03-21T10:38:02.439+0000 7fd0f8400700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.100:6812 osd.4 since back 2024-03-21T10:37:33.508959+0000 front 2024-03-21T10:37:33.509595+0000 (oldest deadline 2024-03-21T10:37:54.609171+0000) 2024-03-21T10:38:02.960 INFO:journalctl@ceph.osd.9.smithi047.stdout:Mar 21 10:38:02 smithi047 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-9[64137]: 2024-03-21T10:38:02.546+0000 7f02dc200700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-03-21T10:36:54.991613+0000 front 2024-03-21T10:36:54.991458+0000 (oldest deadline 2024-03-21T10:37:17.891147+0000) 2024-03-21T10:38:02.961 INFO:journalctl@ceph.osd.9.smithi047.stdout:Mar 21 10:38:02 smithi047 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-9[64137]: 2024-03-21T10:38:02.546+0000 7f02dc200700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-03-21T10:37:02.494627+0000 front 2024-03-21T10:37:02.494473+0000 (oldest deadline 2024-03-21T10:37:26.594304+0000) 2024-03-21T10:38:02.961 INFO:journalctl@ceph.osd.9.smithi047.stdout:Mar 21 10:38:02 smithi047 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-9[64137]: 2024-03-21T10:38:02.546+0000 7f02dc200700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-03-21T10:37:13.597260+0000 front 2024-03-21T10:37:13.596840+0000 (oldest deadline 2024-03-21T10:37:37.696959+0000) 2024-03-21T10:38:02.961 INFO:journalctl@ceph.osd.9.smithi047.stdout:Mar 21 10:38:02 smithi047 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-9[64137]: 2024-03-21T10:38:02.546+0000 7f02dc200700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.47:6814 osd.3 since back 2024-03-21T10:37:21.198652+0000 front 2024-03-21T10:37:21.199080+0000 (oldest deadline 2024-03-21T10:37:45.200271+0000) 2024-03-21T10:38:02.961 INFO:journalctl@ceph.osd.9.smithi047.stdout:Mar 21 10:38:02 smithi047 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-9[64137]: 2024-03-21T10:38:02.546+0000 7f02dc200700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.100:6812 osd.4 since back 2024-03-21T10:37:31.503983+0000 front 2024-03-21T10:37:31.503595+0000 (oldest deadline 2024-03-21T10:37:55.003462+0000) 2024-03-21T10:38:03.236 INFO:journalctl@ceph.osd.8.smithi157.stdout:Mar 21 10:38:02 smithi157 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-8[54391]: 2024-03-21T10:38:02.878+0000 7fca85e00700 -1 osd.8 111 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-03-21T10:37:03.032692+0000 front 2024-03-21T10:37:03.032837+0000 (oldest deadline 2024-03-21T10:37:27.132936+0000) 2024-03-21T10:38:03.237 INFO:journalctl@ceph.osd.8.smithi157.stdout:Mar 21 10:38:02 smithi157 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-8[54391]: 2024-03-21T10:38:02.878+0000 7fca85e00700 -1 osd.8 111 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-03-21T10:37:13.535744+0000 front 2024-03-21T10:37:13.536199+0000 (oldest deadline 2024-03-21T10:37:39.435978+0000) 2024-03-21T10:38:03.237 INFO:journalctl@ceph.osd.8.smithi157.stdout:Mar 21 10:38:02 smithi157 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-8[54391]: 2024-03-21T10:38:02.878+0000 7fca85e00700 -1 osd.8 111 heartbeat_check: no reply from 172.21.15.47:6814 osd.3 since back 2024-03-21T10:37:19.437169+0000 front 2024-03-21T10:37:19.437298+0000 (oldest deadline 2024-03-21T10:37:48.838456+0000) 2024-03-21T10:38:03.237 INFO:journalctl@ceph.osd.8.smithi157.stdout:Mar 21 10:38:02 smithi157 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-8[54391]: 2024-03-21T10:38:02.878+0000 7fca85e00700 -1 osd.8 111 heartbeat_check: no reply from 172.21.15.100:6812 osd.4 since back 2024-03-21T10:37:28.839776+0000 front 2024-03-21T10:37:28.839421+0000 (oldest deadline 2024-03-21T10:38:00.040369+0000) 2024-03-21T10:38:03.324 INFO:journalctl@ceph.osd.7.smithi100.stdout:Mar 21 10:38:03 smithi100 podman[71521]: 2024-03-21 10:38:03.034174388 +0000 UTC m=+5.839113801 container died a1542225a4f57ead7f21a995f73a9dc1449bc08e9361d7ed6758ab85671be329 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-7, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.build-date=20240319, ceph=True, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5) 2024-03-21T10:38:03.325 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 10:38:03 smithi100 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-10[59767]: 2024-03-21T10:38:03.117+0000 7f315da00700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-03-21T10:36:52.673512+0000 front 2024-03-21T10:36:52.673688+0000 (oldest deadline 2024-03-21T10:37:16.675039+0000) 2024-03-21T10:38:03.325 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 10:38:03 smithi100 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-10[59767]: 2024-03-21T10:38:03.117+0000 7f315da00700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-03-21T10:37:05.379887+0000 front 2024-03-21T10:37:05.379536+0000 (oldest deadline 2024-03-21T10:37:30.079809+0000) 2024-03-21T10:38:03.325 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 10:38:03 smithi100 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-10[59767]: 2024-03-21T10:38:03.117+0000 7f315da00700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-03-21T10:37:12.981593+0000 front 2024-03-21T10:37:12.981483+0000 (oldest deadline 2024-03-21T10:37:38.281183+0000) 2024-03-21T10:38:03.325 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 10:38:03 smithi100 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-10[59767]: 2024-03-21T10:38:03.117+0000 7f315da00700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.47:6814 osd.3 since back 2024-03-21T10:37:23.583085+0000 front 2024-03-21T10:37:23.583243+0000 (oldest deadline 2024-03-21T10:37:45.283261+0000) 2024-03-21T10:38:03.325 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 10:38:03 smithi100 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-10[59767]: 2024-03-21T10:38:03.117+0000 7f315da00700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.100:6812 osd.4 since back 2024-03-21T10:37:28.185438+0000 front 2024-03-21T10:37:28.185490+0000 (oldest deadline 2024-03-21T10:37:55.786600+0000) 2024-03-21T10:38:03.718 INFO:journalctl@ceph.osd.7.smithi100.stdout:Mar 21 10:38:03 smithi100 podman[71521]: 2024-03-21 10:38:03.46101663 +0000 UTC m=+6.265956040 container cleanup a1542225a4f57ead7f21a995f73a9dc1449bc08e9361d7ed6758ab85671be329 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-7, org.label-schema.build-date=20240319, GIT_CLEAN=True, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30) 2024-03-21T10:38:03.718 INFO:journalctl@ceph.osd.7.smithi100.stdout:Mar 21 10:38:03 smithi100 bash[71521]: ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-7 2024-03-21T10:38:03.736 INFO:journalctl@ceph.osd.11.smithi157.stdout:Mar 21 10:38:03 smithi157 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-11[61259]: 2024-03-21T10:38:03.411+0000 7fd0f8400700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-03-21T10:36:54.398353+0000 front 2024-03-21T10:36:54.398068+0000 (oldest deadline 2024-03-21T10:37:16.698280+0000) 2024-03-21T10:38:03.736 INFO:journalctl@ceph.osd.11.smithi157.stdout:Mar 21 10:38:03 smithi157 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-11[61259]: 2024-03-21T10:38:03.411+0000 7fd0f8400700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-03-21T10:37:04.901925+0000 front 2024-03-21T10:37:04.902083+0000 (oldest deadline 2024-03-21T10:37:29.502349+0000) 2024-03-21T10:38:03.736 INFO:journalctl@ceph.osd.11.smithi157.stdout:Mar 21 10:38:03 smithi157 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-11[61259]: 2024-03-21T10:38:03.411+0000 7fd0f8400700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-03-21T10:37:11.804109+0000 front 2024-03-21T10:37:11.804464+0000 (oldest deadline 2024-03-21T10:37:37.104077+0000) 2024-03-21T10:38:03.736 INFO:journalctl@ceph.osd.11.smithi157.stdout:Mar 21 10:38:03 smithi157 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-11[61259]: 2024-03-21T10:38:03.411+0000 7fd0f8400700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.100:6812 osd.4 since back 2024-03-21T10:37:33.508959+0000 front 2024-03-21T10:37:33.509595+0000 (oldest deadline 2024-03-21T10:37:54.609171+0000) 2024-03-21T10:38:03.960 INFO:journalctl@ceph.osd.9.smithi047.stdout:Mar 21 10:38:03 smithi047 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-9[64137]: 2024-03-21T10:38:03.574+0000 7f02dc200700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-03-21T10:36:54.991613+0000 front 2024-03-21T10:36:54.991458+0000 (oldest deadline 2024-03-21T10:37:17.891147+0000) 2024-03-21T10:38:03.961 INFO:journalctl@ceph.osd.9.smithi047.stdout:Mar 21 10:38:03 smithi047 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-9[64137]: 2024-03-21T10:38:03.574+0000 7f02dc200700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-03-21T10:37:02.494627+0000 front 2024-03-21T10:37:02.494473+0000 (oldest deadline 2024-03-21T10:37:26.594304+0000) 2024-03-21T10:38:03.961 INFO:journalctl@ceph.osd.9.smithi047.stdout:Mar 21 10:38:03 smithi047 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-9[64137]: 2024-03-21T10:38:03.574+0000 7f02dc200700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-03-21T10:37:13.597260+0000 front 2024-03-21T10:37:13.596840+0000 (oldest deadline 2024-03-21T10:37:37.696959+0000) 2024-03-21T10:38:03.961 INFO:journalctl@ceph.osd.9.smithi047.stdout:Mar 21 10:38:03 smithi047 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-9[64137]: 2024-03-21T10:38:03.574+0000 7f02dc200700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.47:6814 osd.3 since back 2024-03-21T10:37:21.198652+0000 front 2024-03-21T10:37:21.199080+0000 (oldest deadline 2024-03-21T10:37:45.200271+0000) 2024-03-21T10:38:03.961 INFO:journalctl@ceph.osd.9.smithi047.stdout:Mar 21 10:38:03 smithi047 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-9[64137]: 2024-03-21T10:38:03.574+0000 7f02dc200700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.100:6812 osd.4 since back 2024-03-21T10:37:31.503983+0000 front 2024-03-21T10:37:31.503595+0000 (oldest deadline 2024-03-21T10:37:55.003462+0000) 2024-03-21T10:38:03.976 INFO:journalctl@ceph.osd.7.smithi100.stdout:Mar 21 10:38:03 smithi100 podman[71540]: 2024-03-21 10:38:03.720231218 +0000 UTC m=+0.665464370 container remove a1542225a4f57ead7f21a995f73a9dc1449bc08e9361d7ed6758ab85671be329 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-7, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240319, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_BRANCH=HEAD) 2024-03-21T10:38:04.236 INFO:journalctl@ceph.osd.8.smithi157.stdout:Mar 21 10:38:03 smithi157 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-8[54391]: 2024-03-21T10:38:03.893+0000 7fca85e00700 -1 osd.8 111 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-03-21T10:37:03.032692+0000 front 2024-03-21T10:37:03.032837+0000 (oldest deadline 2024-03-21T10:37:27.132936+0000) 2024-03-21T10:38:04.236 INFO:journalctl@ceph.osd.8.smithi157.stdout:Mar 21 10:38:03 smithi157 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-8[54391]: 2024-03-21T10:38:03.893+0000 7fca85e00700 -1 osd.8 111 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-03-21T10:37:13.535744+0000 front 2024-03-21T10:37:13.536199+0000 (oldest deadline 2024-03-21T10:37:39.435978+0000) 2024-03-21T10:38:04.237 INFO:journalctl@ceph.osd.8.smithi157.stdout:Mar 21 10:38:03 smithi157 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-8[54391]: 2024-03-21T10:38:03.893+0000 7fca85e00700 -1 osd.8 111 heartbeat_check: no reply from 172.21.15.47:6814 osd.3 since back 2024-03-21T10:37:19.437169+0000 front 2024-03-21T10:37:19.437298+0000 (oldest deadline 2024-03-21T10:37:48.838456+0000) 2024-03-21T10:38:04.237 INFO:journalctl@ceph.osd.8.smithi157.stdout:Mar 21 10:38:03 smithi157 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-8[54391]: 2024-03-21T10:38:03.893+0000 7fca85e00700 -1 osd.8 111 heartbeat_check: no reply from 172.21.15.100:6812 osd.4 since back 2024-03-21T10:37:28.839776+0000 front 2024-03-21T10:37:28.839421+0000 (oldest deadline 2024-03-21T10:38:00.040369+0000) 2024-03-21T10:38:04.476 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 10:38:04 smithi100 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-10[59767]: 2024-03-21T10:38:04.089+0000 7f315da00700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-03-21T10:36:52.673512+0000 front 2024-03-21T10:36:52.673688+0000 (oldest deadline 2024-03-21T10:37:16.675039+0000) 2024-03-21T10:38:04.477 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 10:38:04 smithi100 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-10[59767]: 2024-03-21T10:38:04.089+0000 7f315da00700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-03-21T10:37:05.379887+0000 front 2024-03-21T10:37:05.379536+0000 (oldest deadline 2024-03-21T10:37:30.079809+0000) 2024-03-21T10:38:04.477 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 10:38:04 smithi100 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-10[59767]: 2024-03-21T10:38:04.089+0000 7f315da00700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-03-21T10:37:12.981593+0000 front 2024-03-21T10:37:12.981483+0000 (oldest deadline 2024-03-21T10:37:38.281183+0000) 2024-03-21T10:38:04.477 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 10:38:04 smithi100 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-10[59767]: 2024-03-21T10:38:04.090+0000 7f315da00700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.47:6814 osd.3 since back 2024-03-21T10:37:23.583085+0000 front 2024-03-21T10:37:23.583243+0000 (oldest deadline 2024-03-21T10:37:45.283261+0000) 2024-03-21T10:38:04.477 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 10:38:04 smithi100 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-10[59767]: 2024-03-21T10:38:04.090+0000 7f315da00700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.100:6812 osd.4 since back 2024-03-21T10:37:28.185438+0000 front 2024-03-21T10:37:28.185490+0000 (oldest deadline 2024-03-21T10:37:55.786600+0000) 2024-03-21T10:38:04.477 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 10:38:04 smithi100 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-10[59767]: 2024-03-21T10:38:04.090+0000 7f315da00700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.157:6812 osd.5 since back 2024-03-21T10:37:39.888144+0000 front 2024-03-21T10:37:39.888262+0000 (oldest deadline 2024-03-21T10:38:03.888699+0000) 2024-03-21T10:38:04.736 INFO:journalctl@ceph.osd.11.smithi157.stdout:Mar 21 10:38:04 smithi157 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-11[61259]: 2024-03-21T10:38:04.421+0000 7fd0f8400700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-03-21T10:36:54.398353+0000 front 2024-03-21T10:36:54.398068+0000 (oldest deadline 2024-03-21T10:37:16.698280+0000) 2024-03-21T10:38:04.736 INFO:journalctl@ceph.osd.11.smithi157.stdout:Mar 21 10:38:04 smithi157 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-11[61259]: 2024-03-21T10:38:04.421+0000 7fd0f8400700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-03-21T10:37:04.901925+0000 front 2024-03-21T10:37:04.902083+0000 (oldest deadline 2024-03-21T10:37:29.502349+0000) 2024-03-21T10:38:04.737 INFO:journalctl@ceph.osd.11.smithi157.stdout:Mar 21 10:38:04 smithi157 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-11[61259]: 2024-03-21T10:38:04.421+0000 7fd0f8400700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-03-21T10:37:11.804109+0000 front 2024-03-21T10:37:11.804464+0000 (oldest deadline 2024-03-21T10:37:37.104077+0000) 2024-03-21T10:38:04.737 INFO:journalctl@ceph.osd.11.smithi157.stdout:Mar 21 10:38:04 smithi157 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-11[61259]: 2024-03-21T10:38:04.421+0000 7fd0f8400700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.100:6812 osd.4 since back 2024-03-21T10:37:33.508959+0000 front 2024-03-21T10:37:33.509595+0000 (oldest deadline 2024-03-21T10:37:54.609171+0000) 2024-03-21T10:38:04.881 INFO:journalctl@ceph.osd.7.smithi100.stdout:Mar 21 10:38:04 smithi100 podman[71630]: 2024-03-21 10:38:04.470425744 +0000 UTC m=+0.031980493 image pull 237e43c7a65ff48534d94b97c1bfdbe4b7be5672e3d9a613745083ea3702aa54 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232 2024-03-21T10:38:04.881 INFO:journalctl@ceph.osd.7.smithi100.stdout:Mar 21 10:38:04 smithi100 podman[71630]: 2024-03-21 10:38:04.612847152 +0000 UTC m=+0.174401938 container create f5a3f72d91a76ddfc8cc500658000587c7528c6cae83801067f31d0fca7d27ea (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-7-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True, io.buildah.version=1.33.5, org.label-schema.build-date=20240319, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, CEPH_POINT_RELEASE=) 2024-03-21T10:38:04.881 INFO:journalctl@ceph.osd.7.smithi100.stdout:Mar 21 10:38:04 smithi100 podman[71630]: 2024-03-21 10:38:04.883010427 +0000 UTC m=+0.444565219 container init f5a3f72d91a76ddfc8cc500658000587c7528c6cae83801067f31d0fca7d27ea (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-7-deactivate, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.build-date=20240319) 2024-03-21T10:38:04.960 INFO:journalctl@ceph.osd.9.smithi047.stdout:Mar 21 10:38:04 smithi047 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-9[64137]: 2024-03-21T10:38:04.541+0000 7f02dc200700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-03-21T10:36:54.991613+0000 front 2024-03-21T10:36:54.991458+0000 (oldest deadline 2024-03-21T10:37:17.891147+0000) 2024-03-21T10:38:04.961 INFO:journalctl@ceph.osd.9.smithi047.stdout:Mar 21 10:38:04 smithi047 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-9[64137]: 2024-03-21T10:38:04.541+0000 7f02dc200700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-03-21T10:37:02.494627+0000 front 2024-03-21T10:37:02.494473+0000 (oldest deadline 2024-03-21T10:37:26.594304+0000) 2024-03-21T10:38:04.961 INFO:journalctl@ceph.osd.9.smithi047.stdout:Mar 21 10:38:04 smithi047 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-9[64137]: 2024-03-21T10:38:04.541+0000 7f02dc200700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-03-21T10:37:13.597260+0000 front 2024-03-21T10:37:13.596840+0000 (oldest deadline 2024-03-21T10:37:37.696959+0000) 2024-03-21T10:38:04.961 INFO:journalctl@ceph.osd.9.smithi047.stdout:Mar 21 10:38:04 smithi047 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-9[64137]: 2024-03-21T10:38:04.541+0000 7f02dc200700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.47:6814 osd.3 since back 2024-03-21T10:37:21.198652+0000 front 2024-03-21T10:37:21.199080+0000 (oldest deadline 2024-03-21T10:37:45.200271+0000) 2024-03-21T10:38:04.961 INFO:journalctl@ceph.osd.9.smithi047.stdout:Mar 21 10:38:04 smithi047 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-9[64137]: 2024-03-21T10:38:04.541+0000 7f02dc200700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.100:6812 osd.4 since back 2024-03-21T10:37:31.503983+0000 front 2024-03-21T10:37:31.503595+0000 (oldest deadline 2024-03-21T10:37:55.003462+0000) 2024-03-21T10:38:05.226 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 10:38:05 smithi100 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-10[59767]: 2024-03-21T10:38:05.080+0000 7f315da00700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-03-21T10:36:52.673512+0000 front 2024-03-21T10:36:52.673688+0000 (oldest deadline 2024-03-21T10:37:16.675039+0000) 2024-03-21T10:38:05.226 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 10:38:05 smithi100 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-10[59767]: 2024-03-21T10:38:05.080+0000 7f315da00700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-03-21T10:37:05.379887+0000 front 2024-03-21T10:37:05.379536+0000 (oldest deadline 2024-03-21T10:37:30.079809+0000) 2024-03-21T10:38:05.226 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 10:38:05 smithi100 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-10[59767]: 2024-03-21T10:38:05.080+0000 7f315da00700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-03-21T10:37:12.981593+0000 front 2024-03-21T10:37:12.981483+0000 (oldest deadline 2024-03-21T10:37:38.281183+0000) 2024-03-21T10:38:05.226 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 10:38:05 smithi100 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-10[59767]: 2024-03-21T10:38:05.080+0000 7f315da00700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.47:6814 osd.3 since back 2024-03-21T10:37:23.583085+0000 front 2024-03-21T10:37:23.583243+0000 (oldest deadline 2024-03-21T10:37:45.283261+0000) 2024-03-21T10:38:05.227 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 10:38:05 smithi100 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-10[59767]: 2024-03-21T10:38:05.080+0000 7f315da00700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.100:6812 osd.4 since back 2024-03-21T10:37:28.185438+0000 front 2024-03-21T10:37:28.185490+0000 (oldest deadline 2024-03-21T10:37:55.786600+0000) 2024-03-21T10:38:05.227 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 10:38:05 smithi100 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-10[59767]: 2024-03-21T10:38:05.080+0000 7f315da00700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.157:6812 osd.5 since back 2024-03-21T10:37:39.888144+0000 front 2024-03-21T10:37:39.888262+0000 (oldest deadline 2024-03-21T10:38:03.888699+0000) 2024-03-21T10:38:05.227 INFO:journalctl@ceph.osd.7.smithi100.stdout:Mar 21 10:38:04 smithi100 podman[71630]: 2024-03-21 10:38:04.894731043 +0000 UTC m=+0.456285859 container start f5a3f72d91a76ddfc8cc500658000587c7528c6cae83801067f31d0fca7d27ea (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-7-deactivate, io.buildah.version=1.33.5, GIT_CLEAN=True, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, org.label-schema.build-date=20240319, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True) 2024-03-21T10:38:05.227 INFO:journalctl@ceph.osd.7.smithi100.stdout:Mar 21 10:38:04 smithi100 podman[71630]: 2024-03-21 10:38:04.998615885 +0000 UTC m=+0.560170688 container attach f5a3f72d91a76ddfc8cc500658000587c7528c6cae83801067f31d0fca7d27ea (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-7-deactivate, org.label-schema.build-date=20240319, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, ceph=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_CLEAN=True) 2024-03-21T10:38:05.236 INFO:journalctl@ceph.osd.8.smithi157.stdout:Mar 21 10:38:04 smithi157 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-8[54391]: 2024-03-21T10:38:04.863+0000 7fca85e00700 -1 osd.8 111 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-03-21T10:37:03.032692+0000 front 2024-03-21T10:37:03.032837+0000 (oldest deadline 2024-03-21T10:37:27.132936+0000) 2024-03-21T10:38:05.236 INFO:journalctl@ceph.osd.8.smithi157.stdout:Mar 21 10:38:04 smithi157 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-8[54391]: 2024-03-21T10:38:04.863+0000 7fca85e00700 -1 osd.8 111 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-03-21T10:37:13.535744+0000 front 2024-03-21T10:37:13.536199+0000 (oldest deadline 2024-03-21T10:37:39.435978+0000) 2024-03-21T10:38:05.236 INFO:journalctl@ceph.osd.8.smithi157.stdout:Mar 21 10:38:04 smithi157 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-8[54391]: 2024-03-21T10:38:04.863+0000 7fca85e00700 -1 osd.8 111 heartbeat_check: no reply from 172.21.15.47:6814 osd.3 since back 2024-03-21T10:37:19.437169+0000 front 2024-03-21T10:37:19.437298+0000 (oldest deadline 2024-03-21T10:37:48.838456+0000) 2024-03-21T10:38:05.236 INFO:journalctl@ceph.osd.8.smithi157.stdout:Mar 21 10:38:04 smithi157 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-8[54391]: 2024-03-21T10:38:04.863+0000 7fca85e00700 -1 osd.8 111 heartbeat_check: no reply from 172.21.15.100:6812 osd.4 since back 2024-03-21T10:37:28.839776+0000 front 2024-03-21T10:37:28.839421+0000 (oldest deadline 2024-03-21T10:38:00.040369+0000) 2024-03-21T10:38:05.666 INFO:journalctl@ceph.osd.7.smithi100.stdout:Mar 21 10:38:05 smithi100 podman[71630]: 2024-03-21 10:38:05.321131226 +0000 UTC m=+0.882686002 container died f5a3f72d91a76ddfc8cc500658000587c7528c6cae83801067f31d0fca7d27ea (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-7-deactivate, io.buildah.version=1.33.5, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.schema-version=1.0, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240319, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=) 2024-03-21T10:38:05.736 INFO:journalctl@ceph.osd.11.smithi157.stdout:Mar 21 10:38:05 smithi157 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-11[61259]: 2024-03-21T10:38:05.377+0000 7fd0f8400700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-03-21T10:36:54.398353+0000 front 2024-03-21T10:36:54.398068+0000 (oldest deadline 2024-03-21T10:37:16.698280+0000) 2024-03-21T10:38:05.736 INFO:journalctl@ceph.osd.11.smithi157.stdout:Mar 21 10:38:05 smithi157 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-11[61259]: 2024-03-21T10:38:05.377+0000 7fd0f8400700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-03-21T10:37:04.901925+0000 front 2024-03-21T10:37:04.902083+0000 (oldest deadline 2024-03-21T10:37:29.502349+0000) 2024-03-21T10:38:05.737 INFO:journalctl@ceph.osd.11.smithi157.stdout:Mar 21 10:38:05 smithi157 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-11[61259]: 2024-03-21T10:38:05.377+0000 7fd0f8400700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-03-21T10:37:11.804109+0000 front 2024-03-21T10:37:11.804464+0000 (oldest deadline 2024-03-21T10:37:37.104077+0000) 2024-03-21T10:38:05.737 INFO:journalctl@ceph.osd.11.smithi157.stdout:Mar 21 10:38:05 smithi157 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-11[61259]: 2024-03-21T10:38:05.377+0000 7fd0f8400700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.100:6812 osd.4 since back 2024-03-21T10:37:33.508959+0000 front 2024-03-21T10:37:33.509595+0000 (oldest deadline 2024-03-21T10:37:54.609171+0000) 2024-03-21T10:38:05.737 INFO:journalctl@ceph.osd.11.smithi157.stdout:Mar 21 10:38:05 smithi157 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-11[61259]: 2024-03-21T10:38:05.377+0000 7fd0f8400700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.157:6812 osd.5 since back 2024-03-21T10:37:41.512925+0000 front 2024-03-21T10:37:41.513037+0000 (oldest deadline 2024-03-21T10:38:05.013049+0000) 2024-03-21T10:38:05.960 INFO:journalctl@ceph.osd.9.smithi047.stdout:Mar 21 10:38:05 smithi047 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-9[64137]: 2024-03-21T10:38:05.505+0000 7f02dc200700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-03-21T10:36:54.991613+0000 front 2024-03-21T10:36:54.991458+0000 (oldest deadline 2024-03-21T10:37:17.891147+0000) 2024-03-21T10:38:05.961 INFO:journalctl@ceph.osd.9.smithi047.stdout:Mar 21 10:38:05 smithi047 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-9[64137]: 2024-03-21T10:38:05.505+0000 7f02dc200700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-03-21T10:37:02.494627+0000 front 2024-03-21T10:37:02.494473+0000 (oldest deadline 2024-03-21T10:37:26.594304+0000) 2024-03-21T10:38:05.961 INFO:journalctl@ceph.osd.9.smithi047.stdout:Mar 21 10:38:05 smithi047 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-9[64137]: 2024-03-21T10:38:05.505+0000 7f02dc200700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-03-21T10:37:13.597260+0000 front 2024-03-21T10:37:13.596840+0000 (oldest deadline 2024-03-21T10:37:37.696959+0000) 2024-03-21T10:38:05.961 INFO:journalctl@ceph.osd.9.smithi047.stdout:Mar 21 10:38:05 smithi047 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-9[64137]: 2024-03-21T10:38:05.505+0000 7f02dc200700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.47:6814 osd.3 since back 2024-03-21T10:37:21.198652+0000 front 2024-03-21T10:37:21.199080+0000 (oldest deadline 2024-03-21T10:37:45.200271+0000) 2024-03-21T10:38:05.961 INFO:journalctl@ceph.osd.9.smithi047.stdout:Mar 21 10:38:05 smithi047 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-9[64137]: 2024-03-21T10:38:05.505+0000 7f02dc200700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.100:6812 osd.4 since back 2024-03-21T10:37:31.503983+0000 front 2024-03-21T10:37:31.503595+0000 (oldest deadline 2024-03-21T10:37:55.003462+0000) 2024-03-21T10:38:05.961 INFO:journalctl@ceph.osd.9.smithi047.stdout:Mar 21 10:38:05 smithi047 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-9[64137]: 2024-03-21T10:38:05.505+0000 7f02dc200700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.157:6812 osd.5 since back 2024-03-21T10:37:39.105588+0000 front 2024-03-21T10:37:39.105888+0000 (oldest deadline 2024-03-21T10:38:05.005588+0000) 2024-03-21T10:38:06.063 DEBUG:teuthology.orchestra.run.smithi100:> sudo pkill -f 'journalctl -f -n 0 -u ceph-3056d51a-e76b-11ee-95cd-87774f69a715@osd.7.service' 2024-03-21T10:38:06.171 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-21T10:38:06.171 INFO:tasks.cephadm.osd.7:Stopped osd.7 2024-03-21T10:38:06.171 INFO:tasks.cephadm.osd.8:Stopping osd.8... 2024-03-21T10:38:06.172 DEBUG:teuthology.orchestra.run.smithi157:> sudo systemctl stop ceph-3056d51a-e76b-11ee-95cd-87774f69a715@osd.8 2024-03-21T10:38:06.236 INFO:journalctl@ceph.osd.8.smithi157.stdout:Mar 21 10:38:05 smithi157 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-8[54391]: 2024-03-21T10:38:05.834+0000 7fca85e00700 -1 osd.8 111 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-03-21T10:37:03.032692+0000 front 2024-03-21T10:37:03.032837+0000 (oldest deadline 2024-03-21T10:37:27.132936+0000) 2024-03-21T10:38:06.236 INFO:journalctl@ceph.osd.8.smithi157.stdout:Mar 21 10:38:05 smithi157 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-8[54391]: 2024-03-21T10:38:05.834+0000 7fca85e00700 -1 osd.8 111 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-03-21T10:37:13.535744+0000 front 2024-03-21T10:37:13.536199+0000 (oldest deadline 2024-03-21T10:37:39.435978+0000) 2024-03-21T10:38:06.236 INFO:journalctl@ceph.osd.8.smithi157.stdout:Mar 21 10:38:05 smithi157 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-8[54391]: 2024-03-21T10:38:05.834+0000 7fca85e00700 -1 osd.8 111 heartbeat_check: no reply from 172.21.15.47:6814 osd.3 since back 2024-03-21T10:37:19.437169+0000 front 2024-03-21T10:37:19.437298+0000 (oldest deadline 2024-03-21T10:37:48.838456+0000) 2024-03-21T10:38:06.236 INFO:journalctl@ceph.osd.8.smithi157.stdout:Mar 21 10:38:05 smithi157 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-8[54391]: 2024-03-21T10:38:05.834+0000 7fca85e00700 -1 osd.8 111 heartbeat_check: no reply from 172.21.15.100:6812 osd.4 since back 2024-03-21T10:37:28.839776+0000 front 2024-03-21T10:37:28.839421+0000 (oldest deadline 2024-03-21T10:38:00.040369+0000) 2024-03-21T10:38:06.237 INFO:journalctl@ceph.osd.8.smithi157.stdout:Mar 21 10:38:05 smithi157 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-8[54391]: 2024-03-21T10:38:05.834+0000 7fca85e00700 -1 osd.8 111 heartbeat_check: no reply from 172.21.15.157:6812 osd.5 since back 2024-03-21T10:37:40.041222+0000 front 2024-03-21T10:37:40.041302+0000 (oldest deadline 2024-03-21T10:38:05.242684+0000) 2024-03-21T10:38:06.476 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 10:38:06 smithi100 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-10[59767]: 2024-03-21T10:38:06.096+0000 7f315da00700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-03-21T10:36:52.673512+0000 front 2024-03-21T10:36:52.673688+0000 (oldest deadline 2024-03-21T10:37:16.675039+0000) 2024-03-21T10:38:06.477 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 10:38:06 smithi100 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-10[59767]: 2024-03-21T10:38:06.096+0000 7f315da00700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-03-21T10:37:05.379887+0000 front 2024-03-21T10:37:05.379536+0000 (oldest deadline 2024-03-21T10:37:30.079809+0000) 2024-03-21T10:38:06.477 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 10:38:06 smithi100 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-10[59767]: 2024-03-21T10:38:06.096+0000 7f315da00700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-03-21T10:37:12.981593+0000 front 2024-03-21T10:37:12.981483+0000 (oldest deadline 2024-03-21T10:37:38.281183+0000) 2024-03-21T10:38:06.477 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 10:38:06 smithi100 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-10[59767]: 2024-03-21T10:38:06.097+0000 7f315da00700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.47:6814 osd.3 since back 2024-03-21T10:37:23.583085+0000 front 2024-03-21T10:37:23.583243+0000 (oldest deadline 2024-03-21T10:37:45.283261+0000) 2024-03-21T10:38:06.477 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 10:38:06 smithi100 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-10[59767]: 2024-03-21T10:38:06.097+0000 7f315da00700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.100:6812 osd.4 since back 2024-03-21T10:37:28.185438+0000 front 2024-03-21T10:37:28.185490+0000 (oldest deadline 2024-03-21T10:37:55.786600+0000) 2024-03-21T10:38:06.477 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 10:38:06 smithi100 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-10[59767]: 2024-03-21T10:38:06.097+0000 7f315da00700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.157:6812 osd.5 since back 2024-03-21T10:37:39.888144+0000 front 2024-03-21T10:37:39.888262+0000 (oldest deadline 2024-03-21T10:38:03.888699+0000) 2024-03-21T10:38:06.736 INFO:journalctl@ceph.osd.8.smithi157.stdout:Mar 21 10:38:06 smithi157 systemd[1]: Stopping Ceph osd.8 for 3056d51a-e76b-11ee-95cd-87774f69a715... 2024-03-21T10:38:06.736 INFO:journalctl@ceph.osd.11.smithi157.stdout:Mar 21 10:38:06 smithi157 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-11[61259]: 2024-03-21T10:38:06.391+0000 7fd0f8400700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-03-21T10:36:54.398353+0000 front 2024-03-21T10:36:54.398068+0000 (oldest deadline 2024-03-21T10:37:16.698280+0000) 2024-03-21T10:38:06.737 INFO:journalctl@ceph.osd.11.smithi157.stdout:Mar 21 10:38:06 smithi157 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-11[61259]: 2024-03-21T10:38:06.391+0000 7fd0f8400700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-03-21T10:37:04.901925+0000 front 2024-03-21T10:37:04.902083+0000 (oldest deadline 2024-03-21T10:37:29.502349+0000) 2024-03-21T10:38:06.737 INFO:journalctl@ceph.osd.11.smithi157.stdout:Mar 21 10:38:06 smithi157 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-11[61259]: 2024-03-21T10:38:06.391+0000 7fd0f8400700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-03-21T10:37:11.804109+0000 front 2024-03-21T10:37:11.804464+0000 (oldest deadline 2024-03-21T10:37:37.104077+0000) 2024-03-21T10:38:06.737 INFO:journalctl@ceph.osd.11.smithi157.stdout:Mar 21 10:38:06 smithi157 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-11[61259]: 2024-03-21T10:38:06.391+0000 7fd0f8400700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.100:6812 osd.4 since back 2024-03-21T10:37:33.508959+0000 front 2024-03-21T10:37:33.509595+0000 (oldest deadline 2024-03-21T10:37:54.609171+0000) 2024-03-21T10:38:06.737 INFO:journalctl@ceph.osd.11.smithi157.stdout:Mar 21 10:38:06 smithi157 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-11[61259]: 2024-03-21T10:38:06.391+0000 7fd0f8400700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.157:6812 osd.5 since back 2024-03-21T10:37:41.512925+0000 front 2024-03-21T10:37:41.513037+0000 (oldest deadline 2024-03-21T10:38:05.013049+0000) 2024-03-21T10:38:06.961 INFO:journalctl@ceph.osd.9.smithi047.stdout:Mar 21 10:38:06 smithi047 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-9[64137]: 2024-03-21T10:38:06.515+0000 7f02dc200700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-03-21T10:36:54.991613+0000 front 2024-03-21T10:36:54.991458+0000 (oldest deadline 2024-03-21T10:37:17.891147+0000) 2024-03-21T10:38:06.961 INFO:journalctl@ceph.osd.9.smithi047.stdout:Mar 21 10:38:06 smithi047 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-9[64137]: 2024-03-21T10:38:06.515+0000 7f02dc200700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-03-21T10:37:02.494627+0000 front 2024-03-21T10:37:02.494473+0000 (oldest deadline 2024-03-21T10:37:26.594304+0000) 2024-03-21T10:38:06.961 INFO:journalctl@ceph.osd.9.smithi047.stdout:Mar 21 10:38:06 smithi047 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-9[64137]: 2024-03-21T10:38:06.515+0000 7f02dc200700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-03-21T10:37:13.597260+0000 front 2024-03-21T10:37:13.596840+0000 (oldest deadline 2024-03-21T10:37:37.696959+0000) 2024-03-21T10:38:06.961 INFO:journalctl@ceph.osd.9.smithi047.stdout:Mar 21 10:38:06 smithi047 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-9[64137]: 2024-03-21T10:38:06.515+0000 7f02dc200700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.47:6814 osd.3 since back 2024-03-21T10:37:21.198652+0000 front 2024-03-21T10:37:21.199080+0000 (oldest deadline 2024-03-21T10:37:45.200271+0000) 2024-03-21T10:38:06.961 INFO:journalctl@ceph.osd.9.smithi047.stdout:Mar 21 10:38:06 smithi047 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-9[64137]: 2024-03-21T10:38:06.515+0000 7f02dc200700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.100:6812 osd.4 since back 2024-03-21T10:37:31.503983+0000 front 2024-03-21T10:37:31.503595+0000 (oldest deadline 2024-03-21T10:37:55.003462+0000) 2024-03-21T10:38:06.961 INFO:journalctl@ceph.osd.9.smithi047.stdout:Mar 21 10:38:06 smithi047 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-9[64137]: 2024-03-21T10:38:06.515+0000 7f02dc200700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.157:6812 osd.5 since back 2024-03-21T10:37:39.105588+0000 front 2024-03-21T10:37:39.105888+0000 (oldest deadline 2024-03-21T10:38:05.005588+0000) 2024-03-21T10:38:07.237 INFO:journalctl@ceph.osd.8.smithi157.stdout:Mar 21 10:38:06 smithi157 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-8[54391]: 2024-03-21T10:38:06.744+0000 7fca8f600700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.8 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-21T10:38:07.237 INFO:journalctl@ceph.osd.8.smithi157.stdout:Mar 21 10:38:06 smithi157 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-8[54391]: 2024-03-21T10:38:06.744+0000 7fca8f600700 -1 osd.8 111 *** Got signal Terminated *** 2024-03-21T10:38:07.237 INFO:journalctl@ceph.osd.8.smithi157.stdout:Mar 21 10:38:06 smithi157 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-8[54391]: 2024-03-21T10:38:06.744+0000 7fca8f600700 -1 osd.8 111 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-21T10:38:07.237 INFO:journalctl@ceph.osd.8.smithi157.stdout:Mar 21 10:38:06 smithi157 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-8[54391]: 2024-03-21T10:38:06.809+0000 7fca85e00700 -1 osd.8 111 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-03-21T10:37:03.032692+0000 front 2024-03-21T10:37:03.032837+0000 (oldest deadline 2024-03-21T10:37:27.132936+0000) 2024-03-21T10:38:07.237 INFO:journalctl@ceph.osd.8.smithi157.stdout:Mar 21 10:38:06 smithi157 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-8[54391]: 2024-03-21T10:38:06.809+0000 7fca85e00700 -1 osd.8 111 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-03-21T10:37:13.535744+0000 front 2024-03-21T10:37:13.536199+0000 (oldest deadline 2024-03-21T10:37:39.435978+0000) 2024-03-21T10:38:07.237 INFO:journalctl@ceph.osd.8.smithi157.stdout:Mar 21 10:38:06 smithi157 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-8[54391]: 2024-03-21T10:38:06.809+0000 7fca85e00700 -1 osd.8 111 heartbeat_check: no reply from 172.21.15.47:6814 osd.3 since back 2024-03-21T10:37:19.437169+0000 front 2024-03-21T10:37:19.437298+0000 (oldest deadline 2024-03-21T10:37:48.838456+0000) 2024-03-21T10:38:07.237 INFO:journalctl@ceph.osd.8.smithi157.stdout:Mar 21 10:38:06 smithi157 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-8[54391]: 2024-03-21T10:38:06.809+0000 7fca85e00700 -1 osd.8 111 heartbeat_check: no reply from 172.21.15.100:6812 osd.4 since back 2024-03-21T10:37:28.839776+0000 front 2024-03-21T10:37:28.839421+0000 (oldest deadline 2024-03-21T10:38:00.040369+0000) 2024-03-21T10:38:07.237 INFO:journalctl@ceph.osd.8.smithi157.stdout:Mar 21 10:38:06 smithi157 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-8[54391]: 2024-03-21T10:38:06.809+0000 7fca85e00700 -1 osd.8 111 heartbeat_check: no reply from 172.21.15.157:6812 osd.5 since back 2024-03-21T10:37:40.041222+0000 front 2024-03-21T10:37:40.041302+0000 (oldest deadline 2024-03-21T10:38:05.242684+0000) 2024-03-21T10:38:07.476 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 10:38:07 smithi100 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-10[59767]: 2024-03-21T10:38:07.113+0000 7f315da00700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-03-21T10:36:52.673512+0000 front 2024-03-21T10:36:52.673688+0000 (oldest deadline 2024-03-21T10:37:16.675039+0000) 2024-03-21T10:38:07.477 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 10:38:07 smithi100 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-10[59767]: 2024-03-21T10:38:07.113+0000 7f315da00700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-03-21T10:37:05.379887+0000 front 2024-03-21T10:37:05.379536+0000 (oldest deadline 2024-03-21T10:37:30.079809+0000) 2024-03-21T10:38:07.477 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 10:38:07 smithi100 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-10[59767]: 2024-03-21T10:38:07.113+0000 7f315da00700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-03-21T10:37:12.981593+0000 front 2024-03-21T10:37:12.981483+0000 (oldest deadline 2024-03-21T10:37:38.281183+0000) 2024-03-21T10:38:07.477 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 10:38:07 smithi100 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-10[59767]: 2024-03-21T10:38:07.113+0000 7f315da00700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.47:6814 osd.3 since back 2024-03-21T10:37:23.583085+0000 front 2024-03-21T10:37:23.583243+0000 (oldest deadline 2024-03-21T10:37:45.283261+0000) 2024-03-21T10:38:07.477 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 10:38:07 smithi100 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-10[59767]: 2024-03-21T10:38:07.113+0000 7f315da00700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.100:6812 osd.4 since back 2024-03-21T10:37:28.185438+0000 front 2024-03-21T10:37:28.185490+0000 (oldest deadline 2024-03-21T10:37:55.786600+0000) 2024-03-21T10:38:07.477 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 10:38:07 smithi100 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-10[59767]: 2024-03-21T10:38:07.113+0000 7f315da00700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.157:6812 osd.5 since back 2024-03-21T10:37:39.888144+0000 front 2024-03-21T10:37:39.888262+0000 (oldest deadline 2024-03-21T10:38:03.888699+0000) 2024-03-21T10:38:07.737 INFO:journalctl@ceph.osd.11.smithi157.stdout:Mar 21 10:38:07 smithi157 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-11[61259]: 2024-03-21T10:38:07.350+0000 7fd0f8400700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-03-21T10:36:54.398353+0000 front 2024-03-21T10:36:54.398068+0000 (oldest deadline 2024-03-21T10:37:16.698280+0000) 2024-03-21T10:38:07.737 INFO:journalctl@ceph.osd.11.smithi157.stdout:Mar 21 10:38:07 smithi157 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-11[61259]: 2024-03-21T10:38:07.350+0000 7fd0f8400700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-03-21T10:37:04.901925+0000 front 2024-03-21T10:37:04.902083+0000 (oldest deadline 2024-03-21T10:37:29.502349+0000) 2024-03-21T10:38:07.737 INFO:journalctl@ceph.osd.11.smithi157.stdout:Mar 21 10:38:07 smithi157 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-11[61259]: 2024-03-21T10:38:07.350+0000 7fd0f8400700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-03-21T10:37:11.804109+0000 front 2024-03-21T10:37:11.804464+0000 (oldest deadline 2024-03-21T10:37:37.104077+0000) 2024-03-21T10:38:07.737 INFO:journalctl@ceph.osd.11.smithi157.stdout:Mar 21 10:38:07 smithi157 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-11[61259]: 2024-03-21T10:38:07.350+0000 7fd0f8400700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.100:6812 osd.4 since back 2024-03-21T10:37:33.508959+0000 front 2024-03-21T10:37:33.509595+0000 (oldest deadline 2024-03-21T10:37:54.609171+0000) 2024-03-21T10:38:07.737 INFO:journalctl@ceph.osd.11.smithi157.stdout:Mar 21 10:38:07 smithi157 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-11[61259]: 2024-03-21T10:38:07.350+0000 7fd0f8400700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.157:6812 osd.5 since back 2024-03-21T10:37:41.512925+0000 front 2024-03-21T10:37:41.513037+0000 (oldest deadline 2024-03-21T10:38:05.013049+0000) 2024-03-21T10:38:07.960 INFO:journalctl@ceph.osd.9.smithi047.stdout:Mar 21 10:38:07 smithi047 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-9[64137]: 2024-03-21T10:38:07.514+0000 7f02dc200700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-03-21T10:36:54.991613+0000 front 2024-03-21T10:36:54.991458+0000 (oldest deadline 2024-03-21T10:37:17.891147+0000) 2024-03-21T10:38:07.961 INFO:journalctl@ceph.osd.9.smithi047.stdout:Mar 21 10:38:07 smithi047 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-9[64137]: 2024-03-21T10:38:07.514+0000 7f02dc200700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-03-21T10:37:02.494627+0000 front 2024-03-21T10:37:02.494473+0000 (oldest deadline 2024-03-21T10:37:26.594304+0000) 2024-03-21T10:38:07.961 INFO:journalctl@ceph.osd.9.smithi047.stdout:Mar 21 10:38:07 smithi047 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-9[64137]: 2024-03-21T10:38:07.514+0000 7f02dc200700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-03-21T10:37:13.597260+0000 front 2024-03-21T10:37:13.596840+0000 (oldest deadline 2024-03-21T10:37:37.696959+0000) 2024-03-21T10:38:07.961 INFO:journalctl@ceph.osd.9.smithi047.stdout:Mar 21 10:38:07 smithi047 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-9[64137]: 2024-03-21T10:38:07.514+0000 7f02dc200700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.47:6814 osd.3 since back 2024-03-21T10:37:21.198652+0000 front 2024-03-21T10:37:21.199080+0000 (oldest deadline 2024-03-21T10:37:45.200271+0000) 2024-03-21T10:38:07.961 INFO:journalctl@ceph.osd.9.smithi047.stdout:Mar 21 10:38:07 smithi047 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-9[64137]: 2024-03-21T10:38:07.514+0000 7f02dc200700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.100:6812 osd.4 since back 2024-03-21T10:37:31.503983+0000 front 2024-03-21T10:37:31.503595+0000 (oldest deadline 2024-03-21T10:37:55.003462+0000) 2024-03-21T10:38:07.961 INFO:journalctl@ceph.osd.9.smithi047.stdout:Mar 21 10:38:07 smithi047 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-9[64137]: 2024-03-21T10:38:07.514+0000 7f02dc200700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.157:6812 osd.5 since back 2024-03-21T10:37:39.105588+0000 front 2024-03-21T10:37:39.105888+0000 (oldest deadline 2024-03-21T10:38:05.005588+0000) 2024-03-21T10:38:08.236 INFO:journalctl@ceph.osd.8.smithi157.stdout:Mar 21 10:38:07 smithi157 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-8[54391]: 2024-03-21T10:38:07.814+0000 7fca85e00700 -1 osd.8 111 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-03-21T10:37:03.032692+0000 front 2024-03-21T10:37:03.032837+0000 (oldest deadline 2024-03-21T10:37:27.132936+0000) 2024-03-21T10:38:08.237 INFO:journalctl@ceph.osd.8.smithi157.stdout:Mar 21 10:38:07 smithi157 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-8[54391]: 2024-03-21T10:38:07.814+0000 7fca85e00700 -1 osd.8 111 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-03-21T10:37:13.535744+0000 front 2024-03-21T10:37:13.536199+0000 (oldest deadline 2024-03-21T10:37:39.435978+0000) 2024-03-21T10:38:08.237 INFO:journalctl@ceph.osd.8.smithi157.stdout:Mar 21 10:38:07 smithi157 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-8[54391]: 2024-03-21T10:38:07.814+0000 7fca85e00700 -1 osd.8 111 heartbeat_check: no reply from 172.21.15.47:6814 osd.3 since back 2024-03-21T10:37:19.437169+0000 front 2024-03-21T10:37:19.437298+0000 (oldest deadline 2024-03-21T10:37:48.838456+0000) 2024-03-21T10:38:08.237 INFO:journalctl@ceph.osd.8.smithi157.stdout:Mar 21 10:38:07 smithi157 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-8[54391]: 2024-03-21T10:38:07.814+0000 7fca85e00700 -1 osd.8 111 heartbeat_check: no reply from 172.21.15.100:6812 osd.4 since back 2024-03-21T10:37:28.839776+0000 front 2024-03-21T10:37:28.839421+0000 (oldest deadline 2024-03-21T10:38:00.040369+0000) 2024-03-21T10:38:08.237 INFO:journalctl@ceph.osd.8.smithi157.stdout:Mar 21 10:38:07 smithi157 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-8[54391]: 2024-03-21T10:38:07.814+0000 7fca85e00700 -1 osd.8 111 heartbeat_check: no reply from 172.21.15.157:6812 osd.5 since back 2024-03-21T10:37:40.041222+0000 front 2024-03-21T10:37:40.041302+0000 (oldest deadline 2024-03-21T10:38:05.242684+0000) 2024-03-21T10:38:08.477 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 10:38:08 smithi100 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-10[59767]: 2024-03-21T10:38:08.140+0000 7f315da00700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-03-21T10:36:52.673512+0000 front 2024-03-21T10:36:52.673688+0000 (oldest deadline 2024-03-21T10:37:16.675039+0000) 2024-03-21T10:38:08.477 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 10:38:08 smithi100 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-10[59767]: 2024-03-21T10:38:08.140+0000 7f315da00700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-03-21T10:37:05.379887+0000 front 2024-03-21T10:37:05.379536+0000 (oldest deadline 2024-03-21T10:37:30.079809+0000) 2024-03-21T10:38:08.477 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 10:38:08 smithi100 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-10[59767]: 2024-03-21T10:38:08.140+0000 7f315da00700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-03-21T10:37:12.981593+0000 front 2024-03-21T10:37:12.981483+0000 (oldest deadline 2024-03-21T10:37:38.281183+0000) 2024-03-21T10:38:08.477 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 10:38:08 smithi100 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-10[59767]: 2024-03-21T10:38:08.140+0000 7f315da00700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.47:6814 osd.3 since back 2024-03-21T10:37:23.583085+0000 front 2024-03-21T10:37:23.583243+0000 (oldest deadline 2024-03-21T10:37:45.283261+0000) 2024-03-21T10:38:08.477 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 10:38:08 smithi100 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-10[59767]: 2024-03-21T10:38:08.140+0000 7f315da00700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.100:6812 osd.4 since back 2024-03-21T10:37:28.185438+0000 front 2024-03-21T10:37:28.185490+0000 (oldest deadline 2024-03-21T10:37:55.786600+0000) 2024-03-21T10:38:08.477 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 10:38:08 smithi100 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-10[59767]: 2024-03-21T10:38:08.140+0000 7f315da00700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.157:6812 osd.5 since back 2024-03-21T10:37:39.888144+0000 front 2024-03-21T10:37:39.888262+0000 (oldest deadline 2024-03-21T10:38:03.888699+0000) 2024-03-21T10:38:08.737 INFO:journalctl@ceph.osd.11.smithi157.stdout:Mar 21 10:38:08 smithi157 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-11[61259]: 2024-03-21T10:38:08.352+0000 7fd0f8400700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-03-21T10:36:54.398353+0000 front 2024-03-21T10:36:54.398068+0000 (oldest deadline 2024-03-21T10:37:16.698280+0000) 2024-03-21T10:38:08.737 INFO:journalctl@ceph.osd.11.smithi157.stdout:Mar 21 10:38:08 smithi157 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-11[61259]: 2024-03-21T10:38:08.352+0000 7fd0f8400700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-03-21T10:37:04.901925+0000 front 2024-03-21T10:37:04.902083+0000 (oldest deadline 2024-03-21T10:37:29.502349+0000) 2024-03-21T10:38:08.737 INFO:journalctl@ceph.osd.11.smithi157.stdout:Mar 21 10:38:08 smithi157 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-11[61259]: 2024-03-21T10:38:08.352+0000 7fd0f8400700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-03-21T10:37:11.804109+0000 front 2024-03-21T10:37:11.804464+0000 (oldest deadline 2024-03-21T10:37:37.104077+0000) 2024-03-21T10:38:08.737 INFO:journalctl@ceph.osd.11.smithi157.stdout:Mar 21 10:38:08 smithi157 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-11[61259]: 2024-03-21T10:38:08.352+0000 7fd0f8400700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.100:6812 osd.4 since back 2024-03-21T10:37:33.508959+0000 front 2024-03-21T10:37:33.509595+0000 (oldest deadline 2024-03-21T10:37:54.609171+0000) 2024-03-21T10:38:08.737 INFO:journalctl@ceph.osd.11.smithi157.stdout:Mar 21 10:38:08 smithi157 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-11[61259]: 2024-03-21T10:38:08.352+0000 7fd0f8400700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.157:6812 osd.5 since back 2024-03-21T10:37:41.512925+0000 front 2024-03-21T10:37:41.513037+0000 (oldest deadline 2024-03-21T10:38:05.013049+0000) 2024-03-21T10:38:08.961 INFO:journalctl@ceph.osd.9.smithi047.stdout:Mar 21 10:38:08 smithi047 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-9[64137]: 2024-03-21T10:38:08.537+0000 7f02dc200700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-03-21T10:36:54.991613+0000 front 2024-03-21T10:36:54.991458+0000 (oldest deadline 2024-03-21T10:37:17.891147+0000) 2024-03-21T10:38:08.961 INFO:journalctl@ceph.osd.9.smithi047.stdout:Mar 21 10:38:08 smithi047 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-9[64137]: 2024-03-21T10:38:08.537+0000 7f02dc200700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-03-21T10:37:02.494627+0000 front 2024-03-21T10:37:02.494473+0000 (oldest deadline 2024-03-21T10:37:26.594304+0000) 2024-03-21T10:38:08.961 INFO:journalctl@ceph.osd.9.smithi047.stdout:Mar 21 10:38:08 smithi047 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-9[64137]: 2024-03-21T10:38:08.537+0000 7f02dc200700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-03-21T10:37:13.597260+0000 front 2024-03-21T10:37:13.596840+0000 (oldest deadline 2024-03-21T10:37:37.696959+0000) 2024-03-21T10:38:08.961 INFO:journalctl@ceph.osd.9.smithi047.stdout:Mar 21 10:38:08 smithi047 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-9[64137]: 2024-03-21T10:38:08.537+0000 7f02dc200700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.47:6814 osd.3 since back 2024-03-21T10:37:21.198652+0000 front 2024-03-21T10:37:21.199080+0000 (oldest deadline 2024-03-21T10:37:45.200271+0000) 2024-03-21T10:38:08.961 INFO:journalctl@ceph.osd.9.smithi047.stdout:Mar 21 10:38:08 smithi047 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-9[64137]: 2024-03-21T10:38:08.537+0000 7f02dc200700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.100:6812 osd.4 since back 2024-03-21T10:37:31.503983+0000 front 2024-03-21T10:37:31.503595+0000 (oldest deadline 2024-03-21T10:37:55.003462+0000) 2024-03-21T10:38:08.961 INFO:journalctl@ceph.osd.9.smithi047.stdout:Mar 21 10:38:08 smithi047 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-9[64137]: 2024-03-21T10:38:08.537+0000 7f02dc200700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.157:6812 osd.5 since back 2024-03-21T10:37:39.105588+0000 front 2024-03-21T10:37:39.105888+0000 (oldest deadline 2024-03-21T10:38:05.005588+0000) 2024-03-21T10:38:09.237 INFO:journalctl@ceph.osd.8.smithi157.stdout:Mar 21 10:38:08 smithi157 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-8[54391]: 2024-03-21T10:38:08.851+0000 7fca85e00700 -1 osd.8 111 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-03-21T10:37:03.032692+0000 front 2024-03-21T10:37:03.032837+0000 (oldest deadline 2024-03-21T10:37:27.132936+0000) 2024-03-21T10:38:09.237 INFO:journalctl@ceph.osd.8.smithi157.stdout:Mar 21 10:38:08 smithi157 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-8[54391]: 2024-03-21T10:38:08.851+0000 7fca85e00700 -1 osd.8 111 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-03-21T10:37:13.535744+0000 front 2024-03-21T10:37:13.536199+0000 (oldest deadline 2024-03-21T10:37:39.435978+0000) 2024-03-21T10:38:09.237 INFO:journalctl@ceph.osd.8.smithi157.stdout:Mar 21 10:38:08 smithi157 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-8[54391]: 2024-03-21T10:38:08.851+0000 7fca85e00700 -1 osd.8 111 heartbeat_check: no reply from 172.21.15.47:6814 osd.3 since back 2024-03-21T10:37:19.437169+0000 front 2024-03-21T10:37:19.437298+0000 (oldest deadline 2024-03-21T10:37:48.838456+0000) 2024-03-21T10:38:09.237 INFO:journalctl@ceph.osd.8.smithi157.stdout:Mar 21 10:38:08 smithi157 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-8[54391]: 2024-03-21T10:38:08.851+0000 7fca85e00700 -1 osd.8 111 heartbeat_check: no reply from 172.21.15.100:6812 osd.4 since back 2024-03-21T10:37:28.839776+0000 front 2024-03-21T10:37:28.839421+0000 (oldest deadline 2024-03-21T10:38:00.040369+0000) 2024-03-21T10:38:09.237 INFO:journalctl@ceph.osd.8.smithi157.stdout:Mar 21 10:38:08 smithi157 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-8[54391]: 2024-03-21T10:38:08.851+0000 7fca85e00700 -1 osd.8 111 heartbeat_check: no reply from 172.21.15.157:6812 osd.5 since back 2024-03-21T10:37:40.041222+0000 front 2024-03-21T10:37:40.041302+0000 (oldest deadline 2024-03-21T10:38:05.242684+0000) 2024-03-21T10:38:09.477 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 10:38:09 smithi100 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-10[59767]: 2024-03-21T10:38:09.174+0000 7f315da00700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-03-21T10:36:52.673512+0000 front 2024-03-21T10:36:52.673688+0000 (oldest deadline 2024-03-21T10:37:16.675039+0000) 2024-03-21T10:38:09.477 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 10:38:09 smithi100 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-10[59767]: 2024-03-21T10:38:09.174+0000 7f315da00700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-03-21T10:37:05.379887+0000 front 2024-03-21T10:37:05.379536+0000 (oldest deadline 2024-03-21T10:37:30.079809+0000) 2024-03-21T10:38:09.477 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 10:38:09 smithi100 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-10[59767]: 2024-03-21T10:38:09.174+0000 7f315da00700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-03-21T10:37:12.981593+0000 front 2024-03-21T10:37:12.981483+0000 (oldest deadline 2024-03-21T10:37:38.281183+0000) 2024-03-21T10:38:09.477 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 10:38:09 smithi100 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-10[59767]: 2024-03-21T10:38:09.174+0000 7f315da00700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.47:6814 osd.3 since back 2024-03-21T10:37:23.583085+0000 front 2024-03-21T10:37:23.583243+0000 (oldest deadline 2024-03-21T10:37:45.283261+0000) 2024-03-21T10:38:09.477 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 10:38:09 smithi100 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-10[59767]: 2024-03-21T10:38:09.174+0000 7f315da00700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.100:6812 osd.4 since back 2024-03-21T10:37:28.185438+0000 front 2024-03-21T10:37:28.185490+0000 (oldest deadline 2024-03-21T10:37:55.786600+0000) 2024-03-21T10:38:09.477 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 10:38:09 smithi100 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-10[59767]: 2024-03-21T10:38:09.174+0000 7f315da00700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.157:6812 osd.5 since back 2024-03-21T10:37:39.888144+0000 front 2024-03-21T10:37:39.888262+0000 (oldest deadline 2024-03-21T10:38:03.888699+0000) 2024-03-21T10:38:09.737 INFO:journalctl@ceph.osd.11.smithi157.stdout:Mar 21 10:38:09 smithi157 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-11[61259]: 2024-03-21T10:38:09.363+0000 7fd0f8400700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-03-21T10:36:54.398353+0000 front 2024-03-21T10:36:54.398068+0000 (oldest deadline 2024-03-21T10:37:16.698280+0000) 2024-03-21T10:38:09.737 INFO:journalctl@ceph.osd.11.smithi157.stdout:Mar 21 10:38:09 smithi157 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-11[61259]: 2024-03-21T10:38:09.363+0000 7fd0f8400700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-03-21T10:37:04.901925+0000 front 2024-03-21T10:37:04.902083+0000 (oldest deadline 2024-03-21T10:37:29.502349+0000) 2024-03-21T10:38:09.737 INFO:journalctl@ceph.osd.11.smithi157.stdout:Mar 21 10:38:09 smithi157 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-11[61259]: 2024-03-21T10:38:09.363+0000 7fd0f8400700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-03-21T10:37:11.804109+0000 front 2024-03-21T10:37:11.804464+0000 (oldest deadline 2024-03-21T10:37:37.104077+0000) 2024-03-21T10:38:09.737 INFO:journalctl@ceph.osd.11.smithi157.stdout:Mar 21 10:38:09 smithi157 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-11[61259]: 2024-03-21T10:38:09.363+0000 7fd0f8400700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.100:6812 osd.4 since back 2024-03-21T10:37:33.508959+0000 front 2024-03-21T10:37:33.509595+0000 (oldest deadline 2024-03-21T10:37:54.609171+0000) 2024-03-21T10:38:09.737 INFO:journalctl@ceph.osd.11.smithi157.stdout:Mar 21 10:38:09 smithi157 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-11[61259]: 2024-03-21T10:38:09.363+0000 7fd0f8400700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.157:6812 osd.5 since back 2024-03-21T10:37:41.512925+0000 front 2024-03-21T10:37:41.513037+0000 (oldest deadline 2024-03-21T10:38:05.013049+0000) 2024-03-21T10:38:09.961 INFO:journalctl@ceph.osd.9.smithi047.stdout:Mar 21 10:38:09 smithi047 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-9[64137]: 2024-03-21T10:38:09.523+0000 7f02dc200700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-03-21T10:36:54.991613+0000 front 2024-03-21T10:36:54.991458+0000 (oldest deadline 2024-03-21T10:37:17.891147+0000) 2024-03-21T10:38:09.961 INFO:journalctl@ceph.osd.9.smithi047.stdout:Mar 21 10:38:09 smithi047 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-9[64137]: 2024-03-21T10:38:09.523+0000 7f02dc200700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-03-21T10:37:02.494627+0000 front 2024-03-21T10:37:02.494473+0000 (oldest deadline 2024-03-21T10:37:26.594304+0000) 2024-03-21T10:38:09.961 INFO:journalctl@ceph.osd.9.smithi047.stdout:Mar 21 10:38:09 smithi047 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-9[64137]: 2024-03-21T10:38:09.523+0000 7f02dc200700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-03-21T10:37:13.597260+0000 front 2024-03-21T10:37:13.596840+0000 (oldest deadline 2024-03-21T10:37:37.696959+0000) 2024-03-21T10:38:09.961 INFO:journalctl@ceph.osd.9.smithi047.stdout:Mar 21 10:38:09 smithi047 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-9[64137]: 2024-03-21T10:38:09.523+0000 7f02dc200700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.47:6814 osd.3 since back 2024-03-21T10:37:21.198652+0000 front 2024-03-21T10:37:21.199080+0000 (oldest deadline 2024-03-21T10:37:45.200271+0000) 2024-03-21T10:38:09.961 INFO:journalctl@ceph.osd.9.smithi047.stdout:Mar 21 10:38:09 smithi047 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-9[64137]: 2024-03-21T10:38:09.523+0000 7f02dc200700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.100:6812 osd.4 since back 2024-03-21T10:37:31.503983+0000 front 2024-03-21T10:37:31.503595+0000 (oldest deadline 2024-03-21T10:37:55.003462+0000) 2024-03-21T10:38:09.961 INFO:journalctl@ceph.osd.9.smithi047.stdout:Mar 21 10:38:09 smithi047 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-9[64137]: 2024-03-21T10:38:09.523+0000 7f02dc200700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.157:6812 osd.5 since back 2024-03-21T10:37:39.105588+0000 front 2024-03-21T10:37:39.105888+0000 (oldest deadline 2024-03-21T10:38:05.005588+0000) 2024-03-21T10:38:10.236 INFO:journalctl@ceph.osd.8.smithi157.stdout:Mar 21 10:38:09 smithi157 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-8[54391]: 2024-03-21T10:38:09.805+0000 7fca85e00700 -1 osd.8 111 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-03-21T10:37:03.032692+0000 front 2024-03-21T10:37:03.032837+0000 (oldest deadline 2024-03-21T10:37:27.132936+0000) 2024-03-21T10:38:10.237 INFO:journalctl@ceph.osd.8.smithi157.stdout:Mar 21 10:38:09 smithi157 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-8[54391]: 2024-03-21T10:38:09.805+0000 7fca85e00700 -1 osd.8 111 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-03-21T10:37:13.535744+0000 front 2024-03-21T10:37:13.536199+0000 (oldest deadline 2024-03-21T10:37:39.435978+0000) 2024-03-21T10:38:10.237 INFO:journalctl@ceph.osd.8.smithi157.stdout:Mar 21 10:38:09 smithi157 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-8[54391]: 2024-03-21T10:38:09.805+0000 7fca85e00700 -1 osd.8 111 heartbeat_check: no reply from 172.21.15.47:6814 osd.3 since back 2024-03-21T10:37:19.437169+0000 front 2024-03-21T10:37:19.437298+0000 (oldest deadline 2024-03-21T10:37:48.838456+0000) 2024-03-21T10:38:10.237 INFO:journalctl@ceph.osd.8.smithi157.stdout:Mar 21 10:38:09 smithi157 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-8[54391]: 2024-03-21T10:38:09.805+0000 7fca85e00700 -1 osd.8 111 heartbeat_check: no reply from 172.21.15.100:6812 osd.4 since back 2024-03-21T10:37:28.839776+0000 front 2024-03-21T10:37:28.839421+0000 (oldest deadline 2024-03-21T10:38:00.040369+0000) 2024-03-21T10:38:10.237 INFO:journalctl@ceph.osd.8.smithi157.stdout:Mar 21 10:38:09 smithi157 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-8[54391]: 2024-03-21T10:38:09.805+0000 7fca85e00700 -1 osd.8 111 heartbeat_check: no reply from 172.21.15.157:6812 osd.5 since back 2024-03-21T10:37:40.041222+0000 front 2024-03-21T10:37:40.041302+0000 (oldest deadline 2024-03-21T10:38:05.242684+0000) 2024-03-21T10:38:10.476 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 10:38:10 smithi100 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-10[59767]: 2024-03-21T10:38:10.153+0000 7f315da00700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-03-21T10:36:52.673512+0000 front 2024-03-21T10:36:52.673688+0000 (oldest deadline 2024-03-21T10:37:16.675039+0000) 2024-03-21T10:38:10.476 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 10:38:10 smithi100 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-10[59767]: 2024-03-21T10:38:10.153+0000 7f315da00700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-03-21T10:37:05.379887+0000 front 2024-03-21T10:37:05.379536+0000 (oldest deadline 2024-03-21T10:37:30.079809+0000) 2024-03-21T10:38:10.477 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 10:38:10 smithi100 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-10[59767]: 2024-03-21T10:38:10.153+0000 7f315da00700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-03-21T10:37:12.981593+0000 front 2024-03-21T10:37:12.981483+0000 (oldest deadline 2024-03-21T10:37:38.281183+0000) 2024-03-21T10:38:10.477 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 10:38:10 smithi100 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-10[59767]: 2024-03-21T10:38:10.153+0000 7f315da00700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.47:6814 osd.3 since back 2024-03-21T10:37:23.583085+0000 front 2024-03-21T10:37:23.583243+0000 (oldest deadline 2024-03-21T10:37:45.283261+0000) 2024-03-21T10:38:10.477 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 10:38:10 smithi100 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-10[59767]: 2024-03-21T10:38:10.153+0000 7f315da00700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.100:6812 osd.4 since back 2024-03-21T10:37:28.185438+0000 front 2024-03-21T10:37:28.185490+0000 (oldest deadline 2024-03-21T10:37:55.786600+0000) 2024-03-21T10:38:10.477 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 10:38:10 smithi100 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-10[59767]: 2024-03-21T10:38:10.153+0000 7f315da00700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.157:6812 osd.5 since back 2024-03-21T10:37:39.888144+0000 front 2024-03-21T10:37:39.888262+0000 (oldest deadline 2024-03-21T10:38:03.888699+0000) 2024-03-21T10:38:10.736 INFO:journalctl@ceph.osd.11.smithi157.stdout:Mar 21 10:38:10 smithi157 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-11[61259]: 2024-03-21T10:38:10.354+0000 7fd0f8400700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-03-21T10:36:54.398353+0000 front 2024-03-21T10:36:54.398068+0000 (oldest deadline 2024-03-21T10:37:16.698280+0000) 2024-03-21T10:38:10.737 INFO:journalctl@ceph.osd.11.smithi157.stdout:Mar 21 10:38:10 smithi157 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-11[61259]: 2024-03-21T10:38:10.354+0000 7fd0f8400700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-03-21T10:37:04.901925+0000 front 2024-03-21T10:37:04.902083+0000 (oldest deadline 2024-03-21T10:37:29.502349+0000) 2024-03-21T10:38:10.737 INFO:journalctl@ceph.osd.11.smithi157.stdout:Mar 21 10:38:10 smithi157 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-11[61259]: 2024-03-21T10:38:10.354+0000 7fd0f8400700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-03-21T10:37:11.804109+0000 front 2024-03-21T10:37:11.804464+0000 (oldest deadline 2024-03-21T10:37:37.104077+0000) 2024-03-21T10:38:10.737 INFO:journalctl@ceph.osd.11.smithi157.stdout:Mar 21 10:38:10 smithi157 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-11[61259]: 2024-03-21T10:38:10.354+0000 7fd0f8400700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.100:6812 osd.4 since back 2024-03-21T10:37:33.508959+0000 front 2024-03-21T10:37:33.509595+0000 (oldest deadline 2024-03-21T10:37:54.609171+0000) 2024-03-21T10:38:10.737 INFO:journalctl@ceph.osd.11.smithi157.stdout:Mar 21 10:38:10 smithi157 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-11[61259]: 2024-03-21T10:38:10.354+0000 7fd0f8400700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.157:6812 osd.5 since back 2024-03-21T10:37:41.512925+0000 front 2024-03-21T10:37:41.513037+0000 (oldest deadline 2024-03-21T10:38:05.013049+0000) 2024-03-21T10:38:10.960 INFO:journalctl@ceph.osd.9.smithi047.stdout:Mar 21 10:38:10 smithi047 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-9[64137]: 2024-03-21T10:38:10.532+0000 7f02dc200700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-03-21T10:36:54.991613+0000 front 2024-03-21T10:36:54.991458+0000 (oldest deadline 2024-03-21T10:37:17.891147+0000) 2024-03-21T10:38:10.961 INFO:journalctl@ceph.osd.9.smithi047.stdout:Mar 21 10:38:10 smithi047 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-9[64137]: 2024-03-21T10:38:10.532+0000 7f02dc200700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-03-21T10:37:02.494627+0000 front 2024-03-21T10:37:02.494473+0000 (oldest deadline 2024-03-21T10:37:26.594304+0000) 2024-03-21T10:38:10.961 INFO:journalctl@ceph.osd.9.smithi047.stdout:Mar 21 10:38:10 smithi047 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-9[64137]: 2024-03-21T10:38:10.532+0000 7f02dc200700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-03-21T10:37:13.597260+0000 front 2024-03-21T10:37:13.596840+0000 (oldest deadline 2024-03-21T10:37:37.696959+0000) 2024-03-21T10:38:10.961 INFO:journalctl@ceph.osd.9.smithi047.stdout:Mar 21 10:38:10 smithi047 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-9[64137]: 2024-03-21T10:38:10.532+0000 7f02dc200700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.47:6814 osd.3 since back 2024-03-21T10:37:21.198652+0000 front 2024-03-21T10:37:21.199080+0000 (oldest deadline 2024-03-21T10:37:45.200271+0000) 2024-03-21T10:38:10.961 INFO:journalctl@ceph.osd.9.smithi047.stdout:Mar 21 10:38:10 smithi047 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-9[64137]: 2024-03-21T10:38:10.532+0000 7f02dc200700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.100:6812 osd.4 since back 2024-03-21T10:37:31.503983+0000 front 2024-03-21T10:37:31.503595+0000 (oldest deadline 2024-03-21T10:37:55.003462+0000) 2024-03-21T10:38:10.961 INFO:journalctl@ceph.osd.9.smithi047.stdout:Mar 21 10:38:10 smithi047 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-9[64137]: 2024-03-21T10:38:10.532+0000 7f02dc200700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.157:6812 osd.5 since back 2024-03-21T10:37:39.105588+0000 front 2024-03-21T10:37:39.105888+0000 (oldest deadline 2024-03-21T10:38:05.005588+0000) 2024-03-21T10:38:11.236 INFO:journalctl@ceph.osd.8.smithi157.stdout:Mar 21 10:38:10 smithi157 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-8[54391]: 2024-03-21T10:38:10.791+0000 7fca85e00700 -1 osd.8 111 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-03-21T10:37:03.032692+0000 front 2024-03-21T10:37:03.032837+0000 (oldest deadline 2024-03-21T10:37:27.132936+0000) 2024-03-21T10:38:11.237 INFO:journalctl@ceph.osd.8.smithi157.stdout:Mar 21 10:38:10 smithi157 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-8[54391]: 2024-03-21T10:38:10.791+0000 7fca85e00700 -1 osd.8 111 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-03-21T10:37:13.535744+0000 front 2024-03-21T10:37:13.536199+0000 (oldest deadline 2024-03-21T10:37:39.435978+0000) 2024-03-21T10:38:11.237 INFO:journalctl@ceph.osd.8.smithi157.stdout:Mar 21 10:38:10 smithi157 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-8[54391]: 2024-03-21T10:38:10.791+0000 7fca85e00700 -1 osd.8 111 heartbeat_check: no reply from 172.21.15.47:6814 osd.3 since back 2024-03-21T10:37:19.437169+0000 front 2024-03-21T10:37:19.437298+0000 (oldest deadline 2024-03-21T10:37:48.838456+0000) 2024-03-21T10:38:11.237 INFO:journalctl@ceph.osd.8.smithi157.stdout:Mar 21 10:38:10 smithi157 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-8[54391]: 2024-03-21T10:38:10.791+0000 7fca85e00700 -1 osd.8 111 heartbeat_check: no reply from 172.21.15.100:6812 osd.4 since back 2024-03-21T10:37:28.839776+0000 front 2024-03-21T10:37:28.839421+0000 (oldest deadline 2024-03-21T10:38:00.040369+0000) 2024-03-21T10:38:11.237 INFO:journalctl@ceph.osd.8.smithi157.stdout:Mar 21 10:38:10 smithi157 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-8[54391]: 2024-03-21T10:38:10.791+0000 7fca85e00700 -1 osd.8 111 heartbeat_check: no reply from 172.21.15.157:6812 osd.5 since back 2024-03-21T10:37:40.041222+0000 front 2024-03-21T10:37:40.041302+0000 (oldest deadline 2024-03-21T10:38:05.242684+0000) 2024-03-21T10:38:11.476 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 10:38:11 smithi100 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-10[59767]: 2024-03-21T10:38:11.150+0000 7f315da00700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-03-21T10:36:52.673512+0000 front 2024-03-21T10:36:52.673688+0000 (oldest deadline 2024-03-21T10:37:16.675039+0000) 2024-03-21T10:38:11.476 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 10:38:11 smithi100 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-10[59767]: 2024-03-21T10:38:11.150+0000 7f315da00700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-03-21T10:37:05.379887+0000 front 2024-03-21T10:37:05.379536+0000 (oldest deadline 2024-03-21T10:37:30.079809+0000) 2024-03-21T10:38:11.477 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 10:38:11 smithi100 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-10[59767]: 2024-03-21T10:38:11.150+0000 7f315da00700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-03-21T10:37:12.981593+0000 front 2024-03-21T10:37:12.981483+0000 (oldest deadline 2024-03-21T10:37:38.281183+0000) 2024-03-21T10:38:11.477 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 10:38:11 smithi100 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-10[59767]: 2024-03-21T10:38:11.150+0000 7f315da00700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.47:6814 osd.3 since back 2024-03-21T10:37:23.583085+0000 front 2024-03-21T10:37:23.583243+0000 (oldest deadline 2024-03-21T10:37:45.283261+0000) 2024-03-21T10:38:11.477 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 10:38:11 smithi100 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-10[59767]: 2024-03-21T10:38:11.150+0000 7f315da00700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.100:6812 osd.4 since back 2024-03-21T10:37:28.185438+0000 front 2024-03-21T10:37:28.185490+0000 (oldest deadline 2024-03-21T10:37:55.786600+0000) 2024-03-21T10:38:11.477 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 10:38:11 smithi100 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-10[59767]: 2024-03-21T10:38:11.150+0000 7f315da00700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.157:6812 osd.5 since back 2024-03-21T10:37:39.888144+0000 front 2024-03-21T10:37:39.888262+0000 (oldest deadline 2024-03-21T10:38:03.888699+0000) 2024-03-21T10:38:11.736 INFO:journalctl@ceph.osd.11.smithi157.stdout:Mar 21 10:38:11 smithi157 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-11[61259]: 2024-03-21T10:38:11.339+0000 7fd0f8400700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-03-21T10:36:54.398353+0000 front 2024-03-21T10:36:54.398068+0000 (oldest deadline 2024-03-21T10:37:16.698280+0000) 2024-03-21T10:38:11.737 INFO:journalctl@ceph.osd.11.smithi157.stdout:Mar 21 10:38:11 smithi157 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-11[61259]: 2024-03-21T10:38:11.339+0000 7fd0f8400700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-03-21T10:37:04.901925+0000 front 2024-03-21T10:37:04.902083+0000 (oldest deadline 2024-03-21T10:37:29.502349+0000) 2024-03-21T10:38:11.737 INFO:journalctl@ceph.osd.11.smithi157.stdout:Mar 21 10:38:11 smithi157 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-11[61259]: 2024-03-21T10:38:11.339+0000 7fd0f8400700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-03-21T10:37:11.804109+0000 front 2024-03-21T10:37:11.804464+0000 (oldest deadline 2024-03-21T10:37:37.104077+0000) 2024-03-21T10:38:11.737 INFO:journalctl@ceph.osd.11.smithi157.stdout:Mar 21 10:38:11 smithi157 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-11[61259]: 2024-03-21T10:38:11.339+0000 7fd0f8400700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.100:6812 osd.4 since back 2024-03-21T10:37:33.508959+0000 front 2024-03-21T10:37:33.509595+0000 (oldest deadline 2024-03-21T10:37:54.609171+0000) 2024-03-21T10:38:11.737 INFO:journalctl@ceph.osd.11.smithi157.stdout:Mar 21 10:38:11 smithi157 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-11[61259]: 2024-03-21T10:38:11.339+0000 7fd0f8400700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.157:6812 osd.5 since back 2024-03-21T10:37:41.512925+0000 front 2024-03-21T10:37:41.513037+0000 (oldest deadline 2024-03-21T10:38:05.013049+0000) 2024-03-21T10:38:11.961 INFO:journalctl@ceph.osd.9.smithi047.stdout:Mar 21 10:38:11 smithi047 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-9[64137]: 2024-03-21T10:38:11.532+0000 7f02dc200700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-03-21T10:36:54.991613+0000 front 2024-03-21T10:36:54.991458+0000 (oldest deadline 2024-03-21T10:37:17.891147+0000) 2024-03-21T10:38:11.961 INFO:journalctl@ceph.osd.9.smithi047.stdout:Mar 21 10:38:11 smithi047 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-9[64137]: 2024-03-21T10:38:11.532+0000 7f02dc200700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-03-21T10:37:02.494627+0000 front 2024-03-21T10:37:02.494473+0000 (oldest deadline 2024-03-21T10:37:26.594304+0000) 2024-03-21T10:38:11.961 INFO:journalctl@ceph.osd.9.smithi047.stdout:Mar 21 10:38:11 smithi047 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-9[64137]: 2024-03-21T10:38:11.532+0000 7f02dc200700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-03-21T10:37:13.597260+0000 front 2024-03-21T10:37:13.596840+0000 (oldest deadline 2024-03-21T10:37:37.696959+0000) 2024-03-21T10:38:11.961 INFO:journalctl@ceph.osd.9.smithi047.stdout:Mar 21 10:38:11 smithi047 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-9[64137]: 2024-03-21T10:38:11.532+0000 7f02dc200700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.47:6814 osd.3 since back 2024-03-21T10:37:21.198652+0000 front 2024-03-21T10:37:21.199080+0000 (oldest deadline 2024-03-21T10:37:45.200271+0000) 2024-03-21T10:38:11.961 INFO:journalctl@ceph.osd.9.smithi047.stdout:Mar 21 10:38:11 smithi047 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-9[64137]: 2024-03-21T10:38:11.532+0000 7f02dc200700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.100:6812 osd.4 since back 2024-03-21T10:37:31.503983+0000 front 2024-03-21T10:37:31.503595+0000 (oldest deadline 2024-03-21T10:37:55.003462+0000) 2024-03-21T10:38:11.961 INFO:journalctl@ceph.osd.9.smithi047.stdout:Mar 21 10:38:11 smithi047 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-9[64137]: 2024-03-21T10:38:11.532+0000 7f02dc200700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.157:6812 osd.5 since back 2024-03-21T10:37:39.105588+0000 front 2024-03-21T10:37:39.105888+0000 (oldest deadline 2024-03-21T10:38:05.005588+0000) 2024-03-21T10:38:12.476 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 10:38:12 smithi100 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-10[59767]: 2024-03-21T10:38:12.116+0000 7f315da00700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-03-21T10:36:52.673512+0000 front 2024-03-21T10:36:52.673688+0000 (oldest deadline 2024-03-21T10:37:16.675039+0000) 2024-03-21T10:38:12.476 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 10:38:12 smithi100 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-10[59767]: 2024-03-21T10:38:12.116+0000 7f315da00700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-03-21T10:37:05.379887+0000 front 2024-03-21T10:37:05.379536+0000 (oldest deadline 2024-03-21T10:37:30.079809+0000) 2024-03-21T10:38:12.476 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 10:38:12 smithi100 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-10[59767]: 2024-03-21T10:38:12.116+0000 7f315da00700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-03-21T10:37:12.981593+0000 front 2024-03-21T10:37:12.981483+0000 (oldest deadline 2024-03-21T10:37:38.281183+0000) 2024-03-21T10:38:12.476 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 10:38:12 smithi100 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-10[59767]: 2024-03-21T10:38:12.116+0000 7f315da00700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.47:6814 osd.3 since back 2024-03-21T10:37:23.583085+0000 front 2024-03-21T10:37:23.583243+0000 (oldest deadline 2024-03-21T10:37:45.283261+0000) 2024-03-21T10:38:12.477 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 10:38:12 smithi100 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-10[59767]: 2024-03-21T10:38:12.116+0000 7f315da00700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.100:6812 osd.4 since back 2024-03-21T10:37:28.185438+0000 front 2024-03-21T10:37:28.185490+0000 (oldest deadline 2024-03-21T10:37:55.786600+0000) 2024-03-21T10:38:12.477 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 10:38:12 smithi100 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-10[59767]: 2024-03-21T10:38:12.116+0000 7f315da00700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.157:6812 osd.5 since back 2024-03-21T10:37:39.888144+0000 front 2024-03-21T10:37:39.888262+0000 (oldest deadline 2024-03-21T10:38:03.888699+0000) 2024-03-21T10:38:12.598 INFO:journalctl@ceph.osd.8.smithi157.stdout:Mar 21 10:38:12 smithi157 podman[73560]: 2024-03-21 10:38:12.342919438 +0000 UTC m=+5.742771047 container died 5255a1eb55e07c5e668a6430b57640e61a4d19854d9d5d2e13e1d45e3a9dd482 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-8, org.label-schema.vendor=CentOS, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240319, CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True, GIT_CLEAN=True, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-21T10:38:12.600 INFO:journalctl@ceph.osd.11.smithi157.stdout:Mar 21 10:38:12 smithi157 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-11[61259]: 2024-03-21T10:38:12.374+0000 7fd0f8400700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-03-21T10:36:54.398353+0000 front 2024-03-21T10:36:54.398068+0000 (oldest deadline 2024-03-21T10:37:16.698280+0000) 2024-03-21T10:38:12.600 INFO:journalctl@ceph.osd.11.smithi157.stdout:Mar 21 10:38:12 smithi157 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-11[61259]: 2024-03-21T10:38:12.374+0000 7fd0f8400700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-03-21T10:37:04.901925+0000 front 2024-03-21T10:37:04.902083+0000 (oldest deadline 2024-03-21T10:37:29.502349+0000) 2024-03-21T10:38:12.600 INFO:journalctl@ceph.osd.11.smithi157.stdout:Mar 21 10:38:12 smithi157 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-11[61259]: 2024-03-21T10:38:12.374+0000 7fd0f8400700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-03-21T10:37:11.804109+0000 front 2024-03-21T10:37:11.804464+0000 (oldest deadline 2024-03-21T10:37:37.104077+0000) 2024-03-21T10:38:12.600 INFO:journalctl@ceph.osd.11.smithi157.stdout:Mar 21 10:38:12 smithi157 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-11[61259]: 2024-03-21T10:38:12.374+0000 7fd0f8400700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.100:6812 osd.4 since back 2024-03-21T10:37:33.508959+0000 front 2024-03-21T10:37:33.509595+0000 (oldest deadline 2024-03-21T10:37:54.609171+0000) 2024-03-21T10:38:12.600 INFO:journalctl@ceph.osd.11.smithi157.stdout:Mar 21 10:38:12 smithi157 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-11[61259]: 2024-03-21T10:38:12.374+0000 7fd0f8400700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.157:6812 osd.5 since back 2024-03-21T10:37:41.512925+0000 front 2024-03-21T10:37:41.513037+0000 (oldest deadline 2024-03-21T10:38:05.013049+0000) 2024-03-21T10:38:12.930 INFO:journalctl@ceph.osd.8.smithi157.stdout:Mar 21 10:38:12 smithi157 podman[73560]: 2024-03-21 10:38:12.713424058 +0000 UTC m=+6.113275702 container cleanup 5255a1eb55e07c5e668a6430b57640e61a4d19854d9d5d2e13e1d45e3a9dd482 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-8, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_CLEAN=True, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.build-date=20240319, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-21T10:38:12.930 INFO:journalctl@ceph.osd.8.smithi157.stdout:Mar 21 10:38:12 smithi157 bash[73560]: ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-8 2024-03-21T10:38:12.961 INFO:journalctl@ceph.osd.9.smithi047.stdout:Mar 21 10:38:12 smithi047 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-9[64137]: 2024-03-21T10:38:12.505+0000 7f02dc200700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-03-21T10:36:54.991613+0000 front 2024-03-21T10:36:54.991458+0000 (oldest deadline 2024-03-21T10:37:17.891147+0000) 2024-03-21T10:38:12.961 INFO:journalctl@ceph.osd.9.smithi047.stdout:Mar 21 10:38:12 smithi047 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-9[64137]: 2024-03-21T10:38:12.505+0000 7f02dc200700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-03-21T10:37:02.494627+0000 front 2024-03-21T10:37:02.494473+0000 (oldest deadline 2024-03-21T10:37:26.594304+0000) 2024-03-21T10:38:12.961 INFO:journalctl@ceph.osd.9.smithi047.stdout:Mar 21 10:38:12 smithi047 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-9[64137]: 2024-03-21T10:38:12.505+0000 7f02dc200700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-03-21T10:37:13.597260+0000 front 2024-03-21T10:37:13.596840+0000 (oldest deadline 2024-03-21T10:37:37.696959+0000) 2024-03-21T10:38:12.961 INFO:journalctl@ceph.osd.9.smithi047.stdout:Mar 21 10:38:12 smithi047 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-9[64137]: 2024-03-21T10:38:12.505+0000 7f02dc200700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.47:6814 osd.3 since back 2024-03-21T10:37:21.198652+0000 front 2024-03-21T10:37:21.199080+0000 (oldest deadline 2024-03-21T10:37:45.200271+0000) 2024-03-21T10:38:12.961 INFO:journalctl@ceph.osd.9.smithi047.stdout:Mar 21 10:38:12 smithi047 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-9[64137]: 2024-03-21T10:38:12.505+0000 7f02dc200700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.100:6812 osd.4 since back 2024-03-21T10:37:31.503983+0000 front 2024-03-21T10:37:31.503595+0000 (oldest deadline 2024-03-21T10:37:55.003462+0000) 2024-03-21T10:38:12.961 INFO:journalctl@ceph.osd.9.smithi047.stdout:Mar 21 10:38:12 smithi047 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-9[64137]: 2024-03-21T10:38:12.505+0000 7f02dc200700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.157:6812 osd.5 since back 2024-03-21T10:37:39.105588+0000 front 2024-03-21T10:37:39.105888+0000 (oldest deadline 2024-03-21T10:38:05.005588+0000) 2024-03-21T10:38:13.237 INFO:journalctl@ceph.osd.8.smithi157.stdout:Mar 21 10:38:12 smithi157 podman[73578]: 2024-03-21 10:38:12.930849824 +0000 UTC m=+0.570868803 container remove 5255a1eb55e07c5e668a6430b57640e61a4d19854d9d5d2e13e1d45e3a9dd482 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-8, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.build-date=20240319, GIT_BRANCH=HEAD, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, io.buildah.version=1.33.5, ceph=True, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-21T10:38:13.476 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 10:38:13 smithi100 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-10[59767]: 2024-03-21T10:38:13.164+0000 7f315da00700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-03-21T10:36:52.673512+0000 front 2024-03-21T10:36:52.673688+0000 (oldest deadline 2024-03-21T10:37:16.675039+0000) 2024-03-21T10:38:13.476 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 10:38:13 smithi100 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-10[59767]: 2024-03-21T10:38:13.164+0000 7f315da00700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-03-21T10:37:05.379887+0000 front 2024-03-21T10:37:05.379536+0000 (oldest deadline 2024-03-21T10:37:30.079809+0000) 2024-03-21T10:38:13.476 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 10:38:13 smithi100 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-10[59767]: 2024-03-21T10:38:13.164+0000 7f315da00700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-03-21T10:37:12.981593+0000 front 2024-03-21T10:37:12.981483+0000 (oldest deadline 2024-03-21T10:37:38.281183+0000) 2024-03-21T10:38:13.477 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 10:38:13 smithi100 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-10[59767]: 2024-03-21T10:38:13.164+0000 7f315da00700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.47:6814 osd.3 since back 2024-03-21T10:37:23.583085+0000 front 2024-03-21T10:37:23.583243+0000 (oldest deadline 2024-03-21T10:37:45.283261+0000) 2024-03-21T10:38:13.477 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 10:38:13 smithi100 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-10[59767]: 2024-03-21T10:38:13.164+0000 7f315da00700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.100:6812 osd.4 since back 2024-03-21T10:37:28.185438+0000 front 2024-03-21T10:37:28.185490+0000 (oldest deadline 2024-03-21T10:37:55.786600+0000) 2024-03-21T10:38:13.477 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 10:38:13 smithi100 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-10[59767]: 2024-03-21T10:38:13.164+0000 7f315da00700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.157:6812 osd.5 since back 2024-03-21T10:37:39.888144+0000 front 2024-03-21T10:37:39.888262+0000 (oldest deadline 2024-03-21T10:38:03.888699+0000) 2024-03-21T10:38:13.711 INFO:journalctl@ceph.osd.9.smithi047.stdout:Mar 21 10:38:13 smithi047 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-9[64137]: 2024-03-21T10:38:13.457+0000 7f02dc200700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-03-21T10:36:54.991613+0000 front 2024-03-21T10:36:54.991458+0000 (oldest deadline 2024-03-21T10:37:17.891147+0000) 2024-03-21T10:38:13.711 INFO:journalctl@ceph.osd.9.smithi047.stdout:Mar 21 10:38:13 smithi047 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-9[64137]: 2024-03-21T10:38:13.457+0000 7f02dc200700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-03-21T10:37:02.494627+0000 front 2024-03-21T10:37:02.494473+0000 (oldest deadline 2024-03-21T10:37:26.594304+0000) 2024-03-21T10:38:13.711 INFO:journalctl@ceph.osd.9.smithi047.stdout:Mar 21 10:38:13 smithi047 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-9[64137]: 2024-03-21T10:38:13.457+0000 7f02dc200700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-03-21T10:37:13.597260+0000 front 2024-03-21T10:37:13.596840+0000 (oldest deadline 2024-03-21T10:37:37.696959+0000) 2024-03-21T10:38:13.711 INFO:journalctl@ceph.osd.9.smithi047.stdout:Mar 21 10:38:13 smithi047 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-9[64137]: 2024-03-21T10:38:13.457+0000 7f02dc200700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.47:6814 osd.3 since back 2024-03-21T10:37:21.198652+0000 front 2024-03-21T10:37:21.199080+0000 (oldest deadline 2024-03-21T10:37:45.200271+0000) 2024-03-21T10:38:13.711 INFO:journalctl@ceph.osd.9.smithi047.stdout:Mar 21 10:38:13 smithi047 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-9[64137]: 2024-03-21T10:38:13.457+0000 7f02dc200700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.100:6812 osd.4 since back 2024-03-21T10:37:31.503983+0000 front 2024-03-21T10:37:31.503595+0000 (oldest deadline 2024-03-21T10:37:55.003462+0000) 2024-03-21T10:38:13.711 INFO:journalctl@ceph.osd.9.smithi047.stdout:Mar 21 10:38:13 smithi047 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-9[64137]: 2024-03-21T10:38:13.457+0000 7f02dc200700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.157:6812 osd.5 since back 2024-03-21T10:37:39.105588+0000 front 2024-03-21T10:37:39.105888+0000 (oldest deadline 2024-03-21T10:38:05.005588+0000) 2024-03-21T10:38:13.717 INFO:journalctl@ceph.osd.11.smithi157.stdout:Mar 21 10:38:13 smithi157 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-11[61259]: 2024-03-21T10:38:13.406+0000 7fd0f8400700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-03-21T10:36:54.398353+0000 front 2024-03-21T10:36:54.398068+0000 (oldest deadline 2024-03-21T10:37:16.698280+0000) 2024-03-21T10:38:13.717 INFO:journalctl@ceph.osd.11.smithi157.stdout:Mar 21 10:38:13 smithi157 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-11[61259]: 2024-03-21T10:38:13.406+0000 7fd0f8400700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-03-21T10:37:04.901925+0000 front 2024-03-21T10:37:04.902083+0000 (oldest deadline 2024-03-21T10:37:29.502349+0000) 2024-03-21T10:38:13.717 INFO:journalctl@ceph.osd.11.smithi157.stdout:Mar 21 10:38:13 smithi157 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-11[61259]: 2024-03-21T10:38:13.406+0000 7fd0f8400700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-03-21T10:37:11.804109+0000 front 2024-03-21T10:37:11.804464+0000 (oldest deadline 2024-03-21T10:37:37.104077+0000) 2024-03-21T10:38:13.717 INFO:journalctl@ceph.osd.11.smithi157.stdout:Mar 21 10:38:13 smithi157 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-11[61259]: 2024-03-21T10:38:13.406+0000 7fd0f8400700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.100:6812 osd.4 since back 2024-03-21T10:37:33.508959+0000 front 2024-03-21T10:37:33.509595+0000 (oldest deadline 2024-03-21T10:37:54.609171+0000) 2024-03-21T10:38:13.717 INFO:journalctl@ceph.osd.11.smithi157.stdout:Mar 21 10:38:13 smithi157 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-11[61259]: 2024-03-21T10:38:13.406+0000 7fd0f8400700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.157:6812 osd.5 since back 2024-03-21T10:37:41.512925+0000 front 2024-03-21T10:37:41.513037+0000 (oldest deadline 2024-03-21T10:38:05.013049+0000) 2024-03-21T10:38:13.987 INFO:journalctl@ceph.osd.8.smithi157.stdout:Mar 21 10:38:13 smithi157 podman[73664]: 2024-03-21 10:38:13.623798523 +0000 UTC m=+0.034651532 image pull 237e43c7a65ff48534d94b97c1bfdbe4b7be5672e3d9a613745083ea3702aa54 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232 2024-03-21T10:38:13.987 INFO:journalctl@ceph.osd.8.smithi157.stdout:Mar 21 10:38:13 smithi157 podman[73664]: 2024-03-21 10:38:13.748470544 +0000 UTC m=+0.159323584 container create fc590bbabc01e0c47de2e23e79bdd6774f66adfbb24c49f06256ec93d0bee915 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-8-deactivate, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, ceph=True, maintainer=Guillaume Abrioux , GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240319, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, GIT_CLEAN=True, org.label-schema.license=GPLv2) 2024-03-21T10:38:14.384 INFO:journalctl@ceph.osd.8.smithi157.stdout:Mar 21 10:38:14 smithi157 podman[73664]: 2024-03-21 10:38:14.044087772 +0000 UTC m=+0.454940819 container init fc590bbabc01e0c47de2e23e79bdd6774f66adfbb24c49f06256ec93d0bee915 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-8-deactivate, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, CEPH_POINT_RELEASE=, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, org.label-schema.build-date=20240319, ceph=True, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-21T10:38:14.385 INFO:journalctl@ceph.osd.8.smithi157.stdout:Mar 21 10:38:14 smithi157 podman[73664]: 2024-03-21 10:38:14.058070635 +0000 UTC m=+0.468923645 container start fc590bbabc01e0c47de2e23e79bdd6774f66adfbb24c49f06256ec93d0bee915 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-8-deactivate, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.build-date=20240319) 2024-03-21T10:38:14.385 INFO:journalctl@ceph.osd.8.smithi157.stdout:Mar 21 10:38:14 smithi157 podman[73664]: 2024-03-21 10:38:14.150937527 +0000 UTC m=+0.561790597 container attach fc590bbabc01e0c47de2e23e79bdd6774f66adfbb24c49f06256ec93d0bee915 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-8-deactivate, maintainer=Guillaume Abrioux , GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.schema-version=1.0, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.build-date=20240319, GIT_CLEAN=True) 2024-03-21T10:38:14.476 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 10:38:14 smithi100 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-10[59767]: 2024-03-21T10:38:14.132+0000 7f315da00700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-03-21T10:36:52.673512+0000 front 2024-03-21T10:36:52.673688+0000 (oldest deadline 2024-03-21T10:37:16.675039+0000) 2024-03-21T10:38:14.476 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 10:38:14 smithi100 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-10[59767]: 2024-03-21T10:38:14.132+0000 7f315da00700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-03-21T10:37:05.379887+0000 front 2024-03-21T10:37:05.379536+0000 (oldest deadline 2024-03-21T10:37:30.079809+0000) 2024-03-21T10:38:14.477 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 10:38:14 smithi100 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-10[59767]: 2024-03-21T10:38:14.132+0000 7f315da00700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-03-21T10:37:12.981593+0000 front 2024-03-21T10:37:12.981483+0000 (oldest deadline 2024-03-21T10:37:38.281183+0000) 2024-03-21T10:38:14.477 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 10:38:14 smithi100 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-10[59767]: 2024-03-21T10:38:14.132+0000 7f315da00700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.47:6814 osd.3 since back 2024-03-21T10:37:23.583085+0000 front 2024-03-21T10:37:23.583243+0000 (oldest deadline 2024-03-21T10:37:45.283261+0000) 2024-03-21T10:38:14.477 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 10:38:14 smithi100 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-10[59767]: 2024-03-21T10:38:14.132+0000 7f315da00700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.100:6812 osd.4 since back 2024-03-21T10:37:28.185438+0000 front 2024-03-21T10:37:28.185490+0000 (oldest deadline 2024-03-21T10:37:55.786600+0000) 2024-03-21T10:38:14.477 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 10:38:14 smithi100 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-10[59767]: 2024-03-21T10:38:14.132+0000 7f315da00700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.157:6812 osd.5 since back 2024-03-21T10:37:39.888144+0000 front 2024-03-21T10:37:39.888262+0000 (oldest deadline 2024-03-21T10:38:03.888699+0000) 2024-03-21T10:38:14.710 INFO:journalctl@ceph.osd.9.smithi047.stdout:Mar 21 10:38:14 smithi047 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-9[64137]: 2024-03-21T10:38:14.460+0000 7f02dc200700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-03-21T10:36:54.991613+0000 front 2024-03-21T10:36:54.991458+0000 (oldest deadline 2024-03-21T10:37:17.891147+0000) 2024-03-21T10:38:14.711 INFO:journalctl@ceph.osd.9.smithi047.stdout:Mar 21 10:38:14 smithi047 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-9[64137]: 2024-03-21T10:38:14.460+0000 7f02dc200700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-03-21T10:37:02.494627+0000 front 2024-03-21T10:37:02.494473+0000 (oldest deadline 2024-03-21T10:37:26.594304+0000) 2024-03-21T10:38:14.711 INFO:journalctl@ceph.osd.9.smithi047.stdout:Mar 21 10:38:14 smithi047 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-9[64137]: 2024-03-21T10:38:14.460+0000 7f02dc200700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-03-21T10:37:13.597260+0000 front 2024-03-21T10:37:13.596840+0000 (oldest deadline 2024-03-21T10:37:37.696959+0000) 2024-03-21T10:38:14.711 INFO:journalctl@ceph.osd.9.smithi047.stdout:Mar 21 10:38:14 smithi047 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-9[64137]: 2024-03-21T10:38:14.460+0000 7f02dc200700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.47:6814 osd.3 since back 2024-03-21T10:37:21.198652+0000 front 2024-03-21T10:37:21.199080+0000 (oldest deadline 2024-03-21T10:37:45.200271+0000) 2024-03-21T10:38:14.711 INFO:journalctl@ceph.osd.9.smithi047.stdout:Mar 21 10:38:14 smithi047 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-9[64137]: 2024-03-21T10:38:14.460+0000 7f02dc200700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.100:6812 osd.4 since back 2024-03-21T10:37:31.503983+0000 front 2024-03-21T10:37:31.503595+0000 (oldest deadline 2024-03-21T10:37:55.003462+0000) 2024-03-21T10:38:14.711 INFO:journalctl@ceph.osd.9.smithi047.stdout:Mar 21 10:38:14 smithi047 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-9[64137]: 2024-03-21T10:38:14.460+0000 7f02dc200700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.157:6812 osd.5 since back 2024-03-21T10:37:39.105588+0000 front 2024-03-21T10:37:39.105888+0000 (oldest deadline 2024-03-21T10:38:05.005588+0000) 2024-03-21T10:38:14.736 INFO:journalctl@ceph.osd.8.smithi157.stdout:Mar 21 10:38:14 smithi157 podman[73664]: 2024-03-21 10:38:14.483761744 +0000 UTC m=+0.894614748 container died fc590bbabc01e0c47de2e23e79bdd6774f66adfbb24c49f06256ec93d0bee915 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-8-deactivate, GIT_CLEAN=True, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.build-date=20240319, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, ceph=True, GIT_BRANCH=HEAD, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2) 2024-03-21T10:38:14.736 INFO:journalctl@ceph.osd.11.smithi157.stdout:Mar 21 10:38:14 smithi157 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-11[61259]: 2024-03-21T10:38:14.386+0000 7fd0f8400700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-03-21T10:36:54.398353+0000 front 2024-03-21T10:36:54.398068+0000 (oldest deadline 2024-03-21T10:37:16.698280+0000) 2024-03-21T10:38:14.737 INFO:journalctl@ceph.osd.11.smithi157.stdout:Mar 21 10:38:14 smithi157 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-11[61259]: 2024-03-21T10:38:14.386+0000 7fd0f8400700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-03-21T10:37:04.901925+0000 front 2024-03-21T10:37:04.902083+0000 (oldest deadline 2024-03-21T10:37:29.502349+0000) 2024-03-21T10:38:14.737 INFO:journalctl@ceph.osd.11.smithi157.stdout:Mar 21 10:38:14 smithi157 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-11[61259]: 2024-03-21T10:38:14.386+0000 7fd0f8400700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-03-21T10:37:11.804109+0000 front 2024-03-21T10:37:11.804464+0000 (oldest deadline 2024-03-21T10:37:37.104077+0000) 2024-03-21T10:38:14.737 INFO:journalctl@ceph.osd.11.smithi157.stdout:Mar 21 10:38:14 smithi157 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-11[61259]: 2024-03-21T10:38:14.386+0000 7fd0f8400700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.100:6812 osd.4 since back 2024-03-21T10:37:33.508959+0000 front 2024-03-21T10:37:33.509595+0000 (oldest deadline 2024-03-21T10:37:54.609171+0000) 2024-03-21T10:38:14.737 INFO:journalctl@ceph.osd.11.smithi157.stdout:Mar 21 10:38:14 smithi157 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-11[61259]: 2024-03-21T10:38:14.386+0000 7fd0f8400700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.157:6812 osd.5 since back 2024-03-21T10:37:41.512925+0000 front 2024-03-21T10:37:41.513037+0000 (oldest deadline 2024-03-21T10:38:05.013049+0000) 2024-03-21T10:38:15.223 DEBUG:teuthology.orchestra.run.smithi157:> sudo pkill -f 'journalctl -f -n 0 -u ceph-3056d51a-e76b-11ee-95cd-87774f69a715@osd.8.service' 2024-03-21T10:38:15.348 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-21T10:38:15.348 INFO:tasks.cephadm.osd.8:Stopped osd.8 2024-03-21T10:38:15.348 INFO:tasks.cephadm.osd.9:Stopping osd.9... 2024-03-21T10:38:15.348 DEBUG:teuthology.orchestra.run.smithi047:> sudo systemctl stop ceph-3056d51a-e76b-11ee-95cd-87774f69a715@osd.9 2024-03-21T10:38:15.476 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 10:38:15 smithi100 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-10[59767]: 2024-03-21T10:38:15.114+0000 7f315da00700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-03-21T10:36:52.673512+0000 front 2024-03-21T10:36:52.673688+0000 (oldest deadline 2024-03-21T10:37:16.675039+0000) 2024-03-21T10:38:15.477 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 10:38:15 smithi100 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-10[59767]: 2024-03-21T10:38:15.114+0000 7f315da00700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-03-21T10:37:05.379887+0000 front 2024-03-21T10:37:05.379536+0000 (oldest deadline 2024-03-21T10:37:30.079809+0000) 2024-03-21T10:38:15.477 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 10:38:15 smithi100 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-10[59767]: 2024-03-21T10:38:15.114+0000 7f315da00700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-03-21T10:37:12.981593+0000 front 2024-03-21T10:37:12.981483+0000 (oldest deadline 2024-03-21T10:37:38.281183+0000) 2024-03-21T10:38:15.477 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 10:38:15 smithi100 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-10[59767]: 2024-03-21T10:38:15.114+0000 7f315da00700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.47:6814 osd.3 since back 2024-03-21T10:37:23.583085+0000 front 2024-03-21T10:37:23.583243+0000 (oldest deadline 2024-03-21T10:37:45.283261+0000) 2024-03-21T10:38:15.477 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 10:38:15 smithi100 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-10[59767]: 2024-03-21T10:38:15.114+0000 7f315da00700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.100:6812 osd.4 since back 2024-03-21T10:37:28.185438+0000 front 2024-03-21T10:37:28.185490+0000 (oldest deadline 2024-03-21T10:37:55.786600+0000) 2024-03-21T10:38:15.477 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 10:38:15 smithi100 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-10[59767]: 2024-03-21T10:38:15.114+0000 7f315da00700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.157:6812 osd.5 since back 2024-03-21T10:37:39.888144+0000 front 2024-03-21T10:37:39.888262+0000 (oldest deadline 2024-03-21T10:38:03.888699+0000) 2024-03-21T10:38:15.477 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 10:38:15 smithi100 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-10[59767]: 2024-03-21T10:38:15.114+0000 7f315da00700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.47:6822 osd.6 since back 2024-03-21T10:37:51.491091+0000 front 2024-03-21T10:37:51.491288+0000 (oldest deadline 2024-03-21T10:38:14.390851+0000) 2024-03-21T10:38:15.486 INFO:journalctl@ceph.osd.11.smithi157.stdout:Mar 21 10:38:15 smithi157 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-11[61259]: 2024-03-21T10:38:15.366+0000 7fd0f8400700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-03-21T10:36:54.398353+0000 front 2024-03-21T10:36:54.398068+0000 (oldest deadline 2024-03-21T10:37:16.698280+0000) 2024-03-21T10:38:15.487 INFO:journalctl@ceph.osd.11.smithi157.stdout:Mar 21 10:38:15 smithi157 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-11[61259]: 2024-03-21T10:38:15.366+0000 7fd0f8400700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-03-21T10:37:04.901925+0000 front 2024-03-21T10:37:04.902083+0000 (oldest deadline 2024-03-21T10:37:29.502349+0000) 2024-03-21T10:38:15.487 INFO:journalctl@ceph.osd.11.smithi157.stdout:Mar 21 10:38:15 smithi157 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-11[61259]: 2024-03-21T10:38:15.366+0000 7fd0f8400700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-03-21T10:37:11.804109+0000 front 2024-03-21T10:37:11.804464+0000 (oldest deadline 2024-03-21T10:37:37.104077+0000) 2024-03-21T10:38:15.487 INFO:journalctl@ceph.osd.11.smithi157.stdout:Mar 21 10:38:15 smithi157 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-11[61259]: 2024-03-21T10:38:15.366+0000 7fd0f8400700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.100:6812 osd.4 since back 2024-03-21T10:37:33.508959+0000 front 2024-03-21T10:37:33.509595+0000 (oldest deadline 2024-03-21T10:37:54.609171+0000) 2024-03-21T10:38:15.487 INFO:journalctl@ceph.osd.11.smithi157.stdout:Mar 21 10:38:15 smithi157 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-11[61259]: 2024-03-21T10:38:15.366+0000 7fd0f8400700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.157:6812 osd.5 since back 2024-03-21T10:37:41.512925+0000 front 2024-03-21T10:37:41.513037+0000 (oldest deadline 2024-03-21T10:38:05.013049+0000) 2024-03-21T10:38:15.710 INFO:journalctl@ceph.osd.9.smithi047.stdout:Mar 21 10:38:15 smithi047 systemd[1]: Stopping Ceph osd.9 for 3056d51a-e76b-11ee-95cd-87774f69a715... 2024-03-21T10:38:15.711 INFO:journalctl@ceph.osd.9.smithi047.stdout:Mar 21 10:38:15 smithi047 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-9[64137]: 2024-03-21T10:38:15.507+0000 7f02dc200700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-03-21T10:36:54.991613+0000 front 2024-03-21T10:36:54.991458+0000 (oldest deadline 2024-03-21T10:37:17.891147+0000) 2024-03-21T10:38:15.711 INFO:journalctl@ceph.osd.9.smithi047.stdout:Mar 21 10:38:15 smithi047 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-9[64137]: 2024-03-21T10:38:15.507+0000 7f02dc200700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-03-21T10:37:02.494627+0000 front 2024-03-21T10:37:02.494473+0000 (oldest deadline 2024-03-21T10:37:26.594304+0000) 2024-03-21T10:38:15.711 INFO:journalctl@ceph.osd.9.smithi047.stdout:Mar 21 10:38:15 smithi047 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-9[64137]: 2024-03-21T10:38:15.507+0000 7f02dc200700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-03-21T10:37:13.597260+0000 front 2024-03-21T10:37:13.596840+0000 (oldest deadline 2024-03-21T10:37:37.696959+0000) 2024-03-21T10:38:15.711 INFO:journalctl@ceph.osd.9.smithi047.stdout:Mar 21 10:38:15 smithi047 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-9[64137]: 2024-03-21T10:38:15.507+0000 7f02dc200700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.47:6814 osd.3 since back 2024-03-21T10:37:21.198652+0000 front 2024-03-21T10:37:21.199080+0000 (oldest deadline 2024-03-21T10:37:45.200271+0000) 2024-03-21T10:38:15.711 INFO:journalctl@ceph.osd.9.smithi047.stdout:Mar 21 10:38:15 smithi047 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-9[64137]: 2024-03-21T10:38:15.507+0000 7f02dc200700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.100:6812 osd.4 since back 2024-03-21T10:37:31.503983+0000 front 2024-03-21T10:37:31.503595+0000 (oldest deadline 2024-03-21T10:37:55.003462+0000) 2024-03-21T10:38:15.711 INFO:journalctl@ceph.osd.9.smithi047.stdout:Mar 21 10:38:15 smithi047 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-9[64137]: 2024-03-21T10:38:15.507+0000 7f02dc200700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.157:6812 osd.5 since back 2024-03-21T10:37:39.105588+0000 front 2024-03-21T10:37:39.105888+0000 (oldest deadline 2024-03-21T10:38:05.005588+0000) 2024-03-21T10:38:16.210 INFO:journalctl@ceph.osd.9.smithi047.stdout:Mar 21 10:38:15 smithi047 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-9[64137]: 2024-03-21T10:38:15.930+0000 7f02e0a00700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.9 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-21T10:38:16.211 INFO:journalctl@ceph.osd.9.smithi047.stdout:Mar 21 10:38:15 smithi047 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-9[64137]: 2024-03-21T10:38:15.930+0000 7f02e0a00700 -1 osd.9 111 *** Got signal Terminated *** 2024-03-21T10:38:16.211 INFO:journalctl@ceph.osd.9.smithi047.stdout:Mar 21 10:38:15 smithi047 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-9[64137]: 2024-03-21T10:38:15.930+0000 7f02e0a00700 -1 osd.9 111 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-21T10:38:16.476 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 10:38:16 smithi100 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-10[59767]: 2024-03-21T10:38:16.144+0000 7f315da00700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-03-21T10:36:52.673512+0000 front 2024-03-21T10:36:52.673688+0000 (oldest deadline 2024-03-21T10:37:16.675039+0000) 2024-03-21T10:38:16.477 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 10:38:16 smithi100 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-10[59767]: 2024-03-21T10:38:16.144+0000 7f315da00700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-03-21T10:37:05.379887+0000 front 2024-03-21T10:37:05.379536+0000 (oldest deadline 2024-03-21T10:37:30.079809+0000) 2024-03-21T10:38:16.477 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 10:38:16 smithi100 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-10[59767]: 2024-03-21T10:38:16.144+0000 7f315da00700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-03-21T10:37:12.981593+0000 front 2024-03-21T10:37:12.981483+0000 (oldest deadline 2024-03-21T10:37:38.281183+0000) 2024-03-21T10:38:16.477 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 10:38:16 smithi100 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-10[59767]: 2024-03-21T10:38:16.144+0000 7f315da00700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.47:6814 osd.3 since back 2024-03-21T10:37:23.583085+0000 front 2024-03-21T10:37:23.583243+0000 (oldest deadline 2024-03-21T10:37:45.283261+0000) 2024-03-21T10:38:16.477 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 10:38:16 smithi100 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-10[59767]: 2024-03-21T10:38:16.144+0000 7f315da00700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.100:6812 osd.4 since back 2024-03-21T10:37:28.185438+0000 front 2024-03-21T10:37:28.185490+0000 (oldest deadline 2024-03-21T10:37:55.786600+0000) 2024-03-21T10:38:16.477 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 10:38:16 smithi100 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-10[59767]: 2024-03-21T10:38:16.144+0000 7f315da00700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.157:6812 osd.5 since back 2024-03-21T10:37:39.888144+0000 front 2024-03-21T10:37:39.888262+0000 (oldest deadline 2024-03-21T10:38:03.888699+0000) 2024-03-21T10:38:16.477 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 10:38:16 smithi100 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-10[59767]: 2024-03-21T10:38:16.144+0000 7f315da00700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.47:6822 osd.6 since back 2024-03-21T10:37:51.491091+0000 front 2024-03-21T10:37:51.491288+0000 (oldest deadline 2024-03-21T10:38:14.390851+0000) 2024-03-21T10:38:16.719 INFO:journalctl@ceph.osd.11.smithi157.stdout:Mar 21 10:38:16 smithi157 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-11[61259]: 2024-03-21T10:38:16.332+0000 7fd0f8400700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-03-21T10:36:54.398353+0000 front 2024-03-21T10:36:54.398068+0000 (oldest deadline 2024-03-21T10:37:16.698280+0000) 2024-03-21T10:38:16.719 INFO:journalctl@ceph.osd.11.smithi157.stdout:Mar 21 10:38:16 smithi157 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-11[61259]: 2024-03-21T10:38:16.332+0000 7fd0f8400700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-03-21T10:37:04.901925+0000 front 2024-03-21T10:37:04.902083+0000 (oldest deadline 2024-03-21T10:37:29.502349+0000) 2024-03-21T10:38:16.720 INFO:journalctl@ceph.osd.11.smithi157.stdout:Mar 21 10:38:16 smithi157 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-11[61259]: 2024-03-21T10:38:16.332+0000 7fd0f8400700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-03-21T10:37:11.804109+0000 front 2024-03-21T10:37:11.804464+0000 (oldest deadline 2024-03-21T10:37:37.104077+0000) 2024-03-21T10:38:16.720 INFO:journalctl@ceph.osd.11.smithi157.stdout:Mar 21 10:38:16 smithi157 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-11[61259]: 2024-03-21T10:38:16.332+0000 7fd0f8400700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.100:6812 osd.4 since back 2024-03-21T10:37:33.508959+0000 front 2024-03-21T10:37:33.509595+0000 (oldest deadline 2024-03-21T10:37:54.609171+0000) 2024-03-21T10:38:16.720 INFO:journalctl@ceph.osd.11.smithi157.stdout:Mar 21 10:38:16 smithi157 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-11[61259]: 2024-03-21T10:38:16.332+0000 7fd0f8400700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.157:6812 osd.5 since back 2024-03-21T10:37:41.512925+0000 front 2024-03-21T10:37:41.513037+0000 (oldest deadline 2024-03-21T10:38:05.013049+0000) 2024-03-21T10:38:16.960 INFO:journalctl@ceph.osd.9.smithi047.stdout:Mar 21 10:38:16 smithi047 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-9[64137]: 2024-03-21T10:38:16.470+0000 7f02dc200700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-03-21T10:36:54.991613+0000 front 2024-03-21T10:36:54.991458+0000 (oldest deadline 2024-03-21T10:37:17.891147+0000) 2024-03-21T10:38:16.961 INFO:journalctl@ceph.osd.9.smithi047.stdout:Mar 21 10:38:16 smithi047 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-9[64137]: 2024-03-21T10:38:16.470+0000 7f02dc200700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-03-21T10:37:02.494627+0000 front 2024-03-21T10:37:02.494473+0000 (oldest deadline 2024-03-21T10:37:26.594304+0000) 2024-03-21T10:38:16.961 INFO:journalctl@ceph.osd.9.smithi047.stdout:Mar 21 10:38:16 smithi047 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-9[64137]: 2024-03-21T10:38:16.470+0000 7f02dc200700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-03-21T10:37:13.597260+0000 front 2024-03-21T10:37:13.596840+0000 (oldest deadline 2024-03-21T10:37:37.696959+0000) 2024-03-21T10:38:16.961 INFO:journalctl@ceph.osd.9.smithi047.stdout:Mar 21 10:38:16 smithi047 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-9[64137]: 2024-03-21T10:38:16.470+0000 7f02dc200700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.47:6814 osd.3 since back 2024-03-21T10:37:21.198652+0000 front 2024-03-21T10:37:21.199080+0000 (oldest deadline 2024-03-21T10:37:45.200271+0000) 2024-03-21T10:38:16.961 INFO:journalctl@ceph.osd.9.smithi047.stdout:Mar 21 10:38:16 smithi047 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-9[64137]: 2024-03-21T10:38:16.470+0000 7f02dc200700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.100:6812 osd.4 since back 2024-03-21T10:37:31.503983+0000 front 2024-03-21T10:37:31.503595+0000 (oldest deadline 2024-03-21T10:37:55.003462+0000) 2024-03-21T10:38:16.961 INFO:journalctl@ceph.osd.9.smithi047.stdout:Mar 21 10:38:16 smithi047 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-9[64137]: 2024-03-21T10:38:16.470+0000 7f02dc200700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.157:6812 osd.5 since back 2024-03-21T10:37:39.105588+0000 front 2024-03-21T10:37:39.105888+0000 (oldest deadline 2024-03-21T10:38:05.005588+0000) 2024-03-21T10:38:17.477 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 10:38:17 smithi100 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-10[59767]: 2024-03-21T10:38:17.128+0000 7f315da00700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-03-21T10:36:52.673512+0000 front 2024-03-21T10:36:52.673688+0000 (oldest deadline 2024-03-21T10:37:16.675039+0000) 2024-03-21T10:38:17.477 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 10:38:17 smithi100 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-10[59767]: 2024-03-21T10:38:17.128+0000 7f315da00700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-03-21T10:37:05.379887+0000 front 2024-03-21T10:37:05.379536+0000 (oldest deadline 2024-03-21T10:37:30.079809+0000) 2024-03-21T10:38:17.477 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 10:38:17 smithi100 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-10[59767]: 2024-03-21T10:38:17.128+0000 7f315da00700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-03-21T10:37:12.981593+0000 front 2024-03-21T10:37:12.981483+0000 (oldest deadline 2024-03-21T10:37:38.281183+0000) 2024-03-21T10:38:17.477 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 10:38:17 smithi100 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-10[59767]: 2024-03-21T10:38:17.128+0000 7f315da00700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.47:6814 osd.3 since back 2024-03-21T10:37:23.583085+0000 front 2024-03-21T10:37:23.583243+0000 (oldest deadline 2024-03-21T10:37:45.283261+0000) 2024-03-21T10:38:17.478 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 10:38:17 smithi100 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-10[59767]: 2024-03-21T10:38:17.128+0000 7f315da00700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.100:6812 osd.4 since back 2024-03-21T10:37:28.185438+0000 front 2024-03-21T10:37:28.185490+0000 (oldest deadline 2024-03-21T10:37:55.786600+0000) 2024-03-21T10:38:17.478 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 10:38:17 smithi100 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-10[59767]: 2024-03-21T10:38:17.128+0000 7f315da00700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.157:6812 osd.5 since back 2024-03-21T10:37:39.888144+0000 front 2024-03-21T10:37:39.888262+0000 (oldest deadline 2024-03-21T10:38:03.888699+0000) 2024-03-21T10:38:17.478 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 10:38:17 smithi100 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-10[59767]: 2024-03-21T10:38:17.128+0000 7f315da00700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.47:6822 osd.6 since back 2024-03-21T10:37:51.491091+0000 front 2024-03-21T10:37:51.491288+0000 (oldest deadline 2024-03-21T10:38:14.390851+0000) 2024-03-21T10:38:17.711 INFO:journalctl@ceph.osd.9.smithi047.stdout:Mar 21 10:38:17 smithi047 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-9[64137]: 2024-03-21T10:38:17.422+0000 7f02dc200700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-03-21T10:36:54.991613+0000 front 2024-03-21T10:36:54.991458+0000 (oldest deadline 2024-03-21T10:37:17.891147+0000) 2024-03-21T10:38:17.711 INFO:journalctl@ceph.osd.9.smithi047.stdout:Mar 21 10:38:17 smithi047 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-9[64137]: 2024-03-21T10:38:17.422+0000 7f02dc200700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-03-21T10:37:02.494627+0000 front 2024-03-21T10:37:02.494473+0000 (oldest deadline 2024-03-21T10:37:26.594304+0000) 2024-03-21T10:38:17.711 INFO:journalctl@ceph.osd.9.smithi047.stdout:Mar 21 10:38:17 smithi047 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-9[64137]: 2024-03-21T10:38:17.422+0000 7f02dc200700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-03-21T10:37:13.597260+0000 front 2024-03-21T10:37:13.596840+0000 (oldest deadline 2024-03-21T10:37:37.696959+0000) 2024-03-21T10:38:17.711 INFO:journalctl@ceph.osd.9.smithi047.stdout:Mar 21 10:38:17 smithi047 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-9[64137]: 2024-03-21T10:38:17.422+0000 7f02dc200700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.47:6814 osd.3 since back 2024-03-21T10:37:21.198652+0000 front 2024-03-21T10:37:21.199080+0000 (oldest deadline 2024-03-21T10:37:45.200271+0000) 2024-03-21T10:38:17.711 INFO:journalctl@ceph.osd.9.smithi047.stdout:Mar 21 10:38:17 smithi047 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-9[64137]: 2024-03-21T10:38:17.422+0000 7f02dc200700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.100:6812 osd.4 since back 2024-03-21T10:37:31.503983+0000 front 2024-03-21T10:37:31.503595+0000 (oldest deadline 2024-03-21T10:37:55.003462+0000) 2024-03-21T10:38:17.711 INFO:journalctl@ceph.osd.9.smithi047.stdout:Mar 21 10:38:17 smithi047 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-9[64137]: 2024-03-21T10:38:17.422+0000 7f02dc200700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.157:6812 osd.5 since back 2024-03-21T10:37:39.105588+0000 front 2024-03-21T10:37:39.105888+0000 (oldest deadline 2024-03-21T10:38:05.005588+0000) 2024-03-21T10:38:17.736 INFO:journalctl@ceph.osd.11.smithi157.stdout:Mar 21 10:38:17 smithi157 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-11[61259]: 2024-03-21T10:38:17.319+0000 7fd0f8400700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-03-21T10:36:54.398353+0000 front 2024-03-21T10:36:54.398068+0000 (oldest deadline 2024-03-21T10:37:16.698280+0000) 2024-03-21T10:38:17.737 INFO:journalctl@ceph.osd.11.smithi157.stdout:Mar 21 10:38:17 smithi157 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-11[61259]: 2024-03-21T10:38:17.319+0000 7fd0f8400700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-03-21T10:37:04.901925+0000 front 2024-03-21T10:37:04.902083+0000 (oldest deadline 2024-03-21T10:37:29.502349+0000) 2024-03-21T10:38:17.737 INFO:journalctl@ceph.osd.11.smithi157.stdout:Mar 21 10:38:17 smithi157 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-11[61259]: 2024-03-21T10:38:17.319+0000 7fd0f8400700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-03-21T10:37:11.804109+0000 front 2024-03-21T10:37:11.804464+0000 (oldest deadline 2024-03-21T10:37:37.104077+0000) 2024-03-21T10:38:17.737 INFO:journalctl@ceph.osd.11.smithi157.stdout:Mar 21 10:38:17 smithi157 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-11[61259]: 2024-03-21T10:38:17.319+0000 7fd0f8400700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.100:6812 osd.4 since back 2024-03-21T10:37:33.508959+0000 front 2024-03-21T10:37:33.509595+0000 (oldest deadline 2024-03-21T10:37:54.609171+0000) 2024-03-21T10:38:17.737 INFO:journalctl@ceph.osd.11.smithi157.stdout:Mar 21 10:38:17 smithi157 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-11[61259]: 2024-03-21T10:38:17.319+0000 7fd0f8400700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.157:6812 osd.5 since back 2024-03-21T10:37:41.512925+0000 front 2024-03-21T10:37:41.513037+0000 (oldest deadline 2024-03-21T10:38:05.013049+0000) 2024-03-21T10:38:17.737 INFO:journalctl@ceph.osd.11.smithi157.stdout:Mar 21 10:38:17 smithi157 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-11[61259]: 2024-03-21T10:38:17.319+0000 7fd0f8400700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.47:6822 osd.6 since back 2024-03-21T10:37:52.015740+0000 front 2024-03-21T10:37:52.015675+0000 (oldest deadline 2024-03-21T10:38:17.315784+0000) 2024-03-21T10:38:18.476 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 10:38:18 smithi100 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-10[59767]: 2024-03-21T10:38:18.079+0000 7f315da00700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-03-21T10:36:52.673512+0000 front 2024-03-21T10:36:52.673688+0000 (oldest deadline 2024-03-21T10:37:16.675039+0000) 2024-03-21T10:38:18.477 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 10:38:18 smithi100 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-10[59767]: 2024-03-21T10:38:18.080+0000 7f315da00700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-03-21T10:37:05.379887+0000 front 2024-03-21T10:37:05.379536+0000 (oldest deadline 2024-03-21T10:37:30.079809+0000) 2024-03-21T10:38:18.477 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 10:38:18 smithi100 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-10[59767]: 2024-03-21T10:38:18.080+0000 7f315da00700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-03-21T10:37:12.981593+0000 front 2024-03-21T10:37:12.981483+0000 (oldest deadline 2024-03-21T10:37:38.281183+0000) 2024-03-21T10:38:18.477 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 10:38:18 smithi100 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-10[59767]: 2024-03-21T10:38:18.080+0000 7f315da00700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.47:6814 osd.3 since back 2024-03-21T10:37:23.583085+0000 front 2024-03-21T10:37:23.583243+0000 (oldest deadline 2024-03-21T10:37:45.283261+0000) 2024-03-21T10:38:18.477 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 10:38:18 smithi100 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-10[59767]: 2024-03-21T10:38:18.080+0000 7f315da00700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.100:6812 osd.4 since back 2024-03-21T10:37:28.185438+0000 front 2024-03-21T10:37:28.185490+0000 (oldest deadline 2024-03-21T10:37:55.786600+0000) 2024-03-21T10:38:18.477 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 10:38:18 smithi100 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-10[59767]: 2024-03-21T10:38:18.080+0000 7f315da00700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.157:6812 osd.5 since back 2024-03-21T10:37:39.888144+0000 front 2024-03-21T10:37:39.888262+0000 (oldest deadline 2024-03-21T10:38:03.888699+0000) 2024-03-21T10:38:18.477 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 10:38:18 smithi100 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-10[59767]: 2024-03-21T10:38:18.080+0000 7f315da00700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.47:6822 osd.6 since back 2024-03-21T10:37:51.491091+0000 front 2024-03-21T10:37:51.491288+0000 (oldest deadline 2024-03-21T10:38:14.390851+0000) 2024-03-21T10:38:18.711 INFO:journalctl@ceph.osd.9.smithi047.stdout:Mar 21 10:38:18 smithi047 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-9[64137]: 2024-03-21T10:38:18.417+0000 7f02dc200700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-03-21T10:36:54.991613+0000 front 2024-03-21T10:36:54.991458+0000 (oldest deadline 2024-03-21T10:37:17.891147+0000) 2024-03-21T10:38:18.711 INFO:journalctl@ceph.osd.9.smithi047.stdout:Mar 21 10:38:18 smithi047 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-9[64137]: 2024-03-21T10:38:18.417+0000 7f02dc200700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-03-21T10:37:02.494627+0000 front 2024-03-21T10:37:02.494473+0000 (oldest deadline 2024-03-21T10:37:26.594304+0000) 2024-03-21T10:38:18.711 INFO:journalctl@ceph.osd.9.smithi047.stdout:Mar 21 10:38:18 smithi047 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-9[64137]: 2024-03-21T10:38:18.417+0000 7f02dc200700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-03-21T10:37:13.597260+0000 front 2024-03-21T10:37:13.596840+0000 (oldest deadline 2024-03-21T10:37:37.696959+0000) 2024-03-21T10:38:18.711 INFO:journalctl@ceph.osd.9.smithi047.stdout:Mar 21 10:38:18 smithi047 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-9[64137]: 2024-03-21T10:38:18.417+0000 7f02dc200700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.47:6814 osd.3 since back 2024-03-21T10:37:21.198652+0000 front 2024-03-21T10:37:21.199080+0000 (oldest deadline 2024-03-21T10:37:45.200271+0000) 2024-03-21T10:38:18.711 INFO:journalctl@ceph.osd.9.smithi047.stdout:Mar 21 10:38:18 smithi047 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-9[64137]: 2024-03-21T10:38:18.417+0000 7f02dc200700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.100:6812 osd.4 since back 2024-03-21T10:37:31.503983+0000 front 2024-03-21T10:37:31.503595+0000 (oldest deadline 2024-03-21T10:37:55.003462+0000) 2024-03-21T10:38:18.711 INFO:journalctl@ceph.osd.9.smithi047.stdout:Mar 21 10:38:18 smithi047 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-9[64137]: 2024-03-21T10:38:18.417+0000 7f02dc200700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.157:6812 osd.5 since back 2024-03-21T10:37:39.105588+0000 front 2024-03-21T10:37:39.105888+0000 (oldest deadline 2024-03-21T10:38:05.005588+0000) 2024-03-21T10:38:18.736 INFO:journalctl@ceph.osd.11.smithi157.stdout:Mar 21 10:38:18 smithi157 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-11[61259]: 2024-03-21T10:38:18.369+0000 7fd0f8400700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-03-21T10:36:54.398353+0000 front 2024-03-21T10:36:54.398068+0000 (oldest deadline 2024-03-21T10:37:16.698280+0000) 2024-03-21T10:38:18.736 INFO:journalctl@ceph.osd.11.smithi157.stdout:Mar 21 10:38:18 smithi157 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-11[61259]: 2024-03-21T10:38:18.369+0000 7fd0f8400700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-03-21T10:37:04.901925+0000 front 2024-03-21T10:37:04.902083+0000 (oldest deadline 2024-03-21T10:37:29.502349+0000) 2024-03-21T10:38:18.737 INFO:journalctl@ceph.osd.11.smithi157.stdout:Mar 21 10:38:18 smithi157 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-11[61259]: 2024-03-21T10:38:18.369+0000 7fd0f8400700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-03-21T10:37:11.804109+0000 front 2024-03-21T10:37:11.804464+0000 (oldest deadline 2024-03-21T10:37:37.104077+0000) 2024-03-21T10:38:18.737 INFO:journalctl@ceph.osd.11.smithi157.stdout:Mar 21 10:38:18 smithi157 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-11[61259]: 2024-03-21T10:38:18.369+0000 7fd0f8400700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.100:6812 osd.4 since back 2024-03-21T10:37:33.508959+0000 front 2024-03-21T10:37:33.509595+0000 (oldest deadline 2024-03-21T10:37:54.609171+0000) 2024-03-21T10:38:18.737 INFO:journalctl@ceph.osd.11.smithi157.stdout:Mar 21 10:38:18 smithi157 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-11[61259]: 2024-03-21T10:38:18.369+0000 7fd0f8400700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.157:6812 osd.5 since back 2024-03-21T10:37:41.512925+0000 front 2024-03-21T10:37:41.513037+0000 (oldest deadline 2024-03-21T10:38:05.013049+0000) 2024-03-21T10:38:18.737 INFO:journalctl@ceph.osd.11.smithi157.stdout:Mar 21 10:38:18 smithi157 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-11[61259]: 2024-03-21T10:38:18.369+0000 7fd0f8400700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.47:6822 osd.6 since back 2024-03-21T10:37:52.015740+0000 front 2024-03-21T10:37:52.015675+0000 (oldest deadline 2024-03-21T10:38:17.315784+0000) 2024-03-21T10:38:19.226 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 10:38:19 smithi100 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-10[59767]: 2024-03-21T10:38:19.031+0000 7f315da00700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-03-21T10:36:52.673512+0000 front 2024-03-21T10:36:52.673688+0000 (oldest deadline 2024-03-21T10:37:16.675039+0000) 2024-03-21T10:38:19.227 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 10:38:19 smithi100 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-10[59767]: 2024-03-21T10:38:19.031+0000 7f315da00700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-03-21T10:37:05.379887+0000 front 2024-03-21T10:37:05.379536+0000 (oldest deadline 2024-03-21T10:37:30.079809+0000) 2024-03-21T10:38:19.227 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 10:38:19 smithi100 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-10[59767]: 2024-03-21T10:38:19.031+0000 7f315da00700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-03-21T10:37:12.981593+0000 front 2024-03-21T10:37:12.981483+0000 (oldest deadline 2024-03-21T10:37:38.281183+0000) 2024-03-21T10:38:19.227 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 10:38:19 smithi100 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-10[59767]: 2024-03-21T10:38:19.031+0000 7f315da00700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.47:6814 osd.3 since back 2024-03-21T10:37:23.583085+0000 front 2024-03-21T10:37:23.583243+0000 (oldest deadline 2024-03-21T10:37:45.283261+0000) 2024-03-21T10:38:19.227 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 10:38:19 smithi100 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-10[59767]: 2024-03-21T10:38:19.031+0000 7f315da00700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.100:6812 osd.4 since back 2024-03-21T10:37:28.185438+0000 front 2024-03-21T10:37:28.185490+0000 (oldest deadline 2024-03-21T10:37:55.786600+0000) 2024-03-21T10:38:19.227 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 10:38:19 smithi100 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-10[59767]: 2024-03-21T10:38:19.031+0000 7f315da00700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.157:6812 osd.5 since back 2024-03-21T10:37:39.888144+0000 front 2024-03-21T10:37:39.888262+0000 (oldest deadline 2024-03-21T10:38:03.888699+0000) 2024-03-21T10:38:19.227 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 10:38:19 smithi100 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-10[59767]: 2024-03-21T10:38:19.031+0000 7f315da00700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.47:6822 osd.6 since back 2024-03-21T10:37:51.491091+0000 front 2024-03-21T10:37:51.491288+0000 (oldest deadline 2024-03-21T10:38:14.390851+0000) 2024-03-21T10:38:19.710 INFO:journalctl@ceph.osd.9.smithi047.stdout:Mar 21 10:38:19 smithi047 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-9[64137]: 2024-03-21T10:38:19.447+0000 7f02dc200700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-03-21T10:36:54.991613+0000 front 2024-03-21T10:36:54.991458+0000 (oldest deadline 2024-03-21T10:37:17.891147+0000) 2024-03-21T10:38:19.711 INFO:journalctl@ceph.osd.9.smithi047.stdout:Mar 21 10:38:19 smithi047 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-9[64137]: 2024-03-21T10:38:19.447+0000 7f02dc200700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-03-21T10:37:02.494627+0000 front 2024-03-21T10:37:02.494473+0000 (oldest deadline 2024-03-21T10:37:26.594304+0000) 2024-03-21T10:38:19.711 INFO:journalctl@ceph.osd.9.smithi047.stdout:Mar 21 10:38:19 smithi047 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-9[64137]: 2024-03-21T10:38:19.447+0000 7f02dc200700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-03-21T10:37:13.597260+0000 front 2024-03-21T10:37:13.596840+0000 (oldest deadline 2024-03-21T10:37:37.696959+0000) 2024-03-21T10:38:19.711 INFO:journalctl@ceph.osd.9.smithi047.stdout:Mar 21 10:38:19 smithi047 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-9[64137]: 2024-03-21T10:38:19.447+0000 7f02dc200700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.47:6814 osd.3 since back 2024-03-21T10:37:21.198652+0000 front 2024-03-21T10:37:21.199080+0000 (oldest deadline 2024-03-21T10:37:45.200271+0000) 2024-03-21T10:38:19.711 INFO:journalctl@ceph.osd.9.smithi047.stdout:Mar 21 10:38:19 smithi047 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-9[64137]: 2024-03-21T10:38:19.447+0000 7f02dc200700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.100:6812 osd.4 since back 2024-03-21T10:37:31.503983+0000 front 2024-03-21T10:37:31.503595+0000 (oldest deadline 2024-03-21T10:37:55.003462+0000) 2024-03-21T10:38:19.711 INFO:journalctl@ceph.osd.9.smithi047.stdout:Mar 21 10:38:19 smithi047 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-9[64137]: 2024-03-21T10:38:19.447+0000 7f02dc200700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.157:6812 osd.5 since back 2024-03-21T10:37:39.105588+0000 front 2024-03-21T10:37:39.105888+0000 (oldest deadline 2024-03-21T10:38:05.005588+0000) 2024-03-21T10:38:19.736 INFO:journalctl@ceph.osd.11.smithi157.stdout:Mar 21 10:38:19 smithi157 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-11[61259]: 2024-03-21T10:38:19.402+0000 7fd0f8400700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-03-21T10:36:54.398353+0000 front 2024-03-21T10:36:54.398068+0000 (oldest deadline 2024-03-21T10:37:16.698280+0000) 2024-03-21T10:38:19.736 INFO:journalctl@ceph.osd.11.smithi157.stdout:Mar 21 10:38:19 smithi157 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-11[61259]: 2024-03-21T10:38:19.402+0000 7fd0f8400700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-03-21T10:37:04.901925+0000 front 2024-03-21T10:37:04.902083+0000 (oldest deadline 2024-03-21T10:37:29.502349+0000) 2024-03-21T10:38:19.737 INFO:journalctl@ceph.osd.11.smithi157.stdout:Mar 21 10:38:19 smithi157 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-11[61259]: 2024-03-21T10:38:19.402+0000 7fd0f8400700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-03-21T10:37:11.804109+0000 front 2024-03-21T10:37:11.804464+0000 (oldest deadline 2024-03-21T10:37:37.104077+0000) 2024-03-21T10:38:19.737 INFO:journalctl@ceph.osd.11.smithi157.stdout:Mar 21 10:38:19 smithi157 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-11[61259]: 2024-03-21T10:38:19.402+0000 7fd0f8400700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.100:6812 osd.4 since back 2024-03-21T10:37:33.508959+0000 front 2024-03-21T10:37:33.509595+0000 (oldest deadline 2024-03-21T10:37:54.609171+0000) 2024-03-21T10:38:19.737 INFO:journalctl@ceph.osd.11.smithi157.stdout:Mar 21 10:38:19 smithi157 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-11[61259]: 2024-03-21T10:38:19.402+0000 7fd0f8400700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.157:6812 osd.5 since back 2024-03-21T10:37:41.512925+0000 front 2024-03-21T10:37:41.513037+0000 (oldest deadline 2024-03-21T10:38:05.013049+0000) 2024-03-21T10:38:19.737 INFO:journalctl@ceph.osd.11.smithi157.stdout:Mar 21 10:38:19 smithi157 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-11[61259]: 2024-03-21T10:38:19.402+0000 7fd0f8400700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.47:6822 osd.6 since back 2024-03-21T10:37:52.015740+0000 front 2024-03-21T10:37:52.015675+0000 (oldest deadline 2024-03-21T10:38:17.315784+0000) 2024-03-21T10:38:20.476 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 10:38:19 smithi100 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-10[59767]: 2024-03-21T10:38:19.993+0000 7f315da00700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-03-21T10:36:52.673512+0000 front 2024-03-21T10:36:52.673688+0000 (oldest deadline 2024-03-21T10:37:16.675039+0000) 2024-03-21T10:38:20.477 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 10:38:19 smithi100 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-10[59767]: 2024-03-21T10:38:19.993+0000 7f315da00700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-03-21T10:37:05.379887+0000 front 2024-03-21T10:37:05.379536+0000 (oldest deadline 2024-03-21T10:37:30.079809+0000) 2024-03-21T10:38:20.477 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 10:38:19 smithi100 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-10[59767]: 2024-03-21T10:38:19.993+0000 7f315da00700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-03-21T10:37:12.981593+0000 front 2024-03-21T10:37:12.981483+0000 (oldest deadline 2024-03-21T10:37:38.281183+0000) 2024-03-21T10:38:20.477 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 10:38:19 smithi100 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-10[59767]: 2024-03-21T10:38:19.993+0000 7f315da00700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.47:6814 osd.3 since back 2024-03-21T10:37:23.583085+0000 front 2024-03-21T10:37:23.583243+0000 (oldest deadline 2024-03-21T10:37:45.283261+0000) 2024-03-21T10:38:20.477 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 10:38:19 smithi100 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-10[59767]: 2024-03-21T10:38:19.993+0000 7f315da00700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.100:6812 osd.4 since back 2024-03-21T10:37:28.185438+0000 front 2024-03-21T10:37:28.185490+0000 (oldest deadline 2024-03-21T10:37:55.786600+0000) 2024-03-21T10:38:20.477 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 10:38:19 smithi100 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-10[59767]: 2024-03-21T10:38:19.993+0000 7f315da00700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.157:6812 osd.5 since back 2024-03-21T10:37:39.888144+0000 front 2024-03-21T10:37:39.888262+0000 (oldest deadline 2024-03-21T10:38:03.888699+0000) 2024-03-21T10:38:20.477 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 10:38:19 smithi100 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-10[59767]: 2024-03-21T10:38:19.993+0000 7f315da00700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.47:6822 osd.6 since back 2024-03-21T10:37:51.491091+0000 front 2024-03-21T10:37:51.491288+0000 (oldest deadline 2024-03-21T10:38:14.390851+0000) 2024-03-21T10:38:20.710 INFO:journalctl@ceph.osd.9.smithi047.stdout:Mar 21 10:38:20 smithi047 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-9[64137]: 2024-03-21T10:38:20.406+0000 7f02dc200700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-03-21T10:36:54.991613+0000 front 2024-03-21T10:36:54.991458+0000 (oldest deadline 2024-03-21T10:37:17.891147+0000) 2024-03-21T10:38:20.711 INFO:journalctl@ceph.osd.9.smithi047.stdout:Mar 21 10:38:20 smithi047 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-9[64137]: 2024-03-21T10:38:20.406+0000 7f02dc200700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-03-21T10:37:02.494627+0000 front 2024-03-21T10:37:02.494473+0000 (oldest deadline 2024-03-21T10:37:26.594304+0000) 2024-03-21T10:38:20.711 INFO:journalctl@ceph.osd.9.smithi047.stdout:Mar 21 10:38:20 smithi047 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-9[64137]: 2024-03-21T10:38:20.406+0000 7f02dc200700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-03-21T10:37:13.597260+0000 front 2024-03-21T10:37:13.596840+0000 (oldest deadline 2024-03-21T10:37:37.696959+0000) 2024-03-21T10:38:20.711 INFO:journalctl@ceph.osd.9.smithi047.stdout:Mar 21 10:38:20 smithi047 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-9[64137]: 2024-03-21T10:38:20.406+0000 7f02dc200700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.47:6814 osd.3 since back 2024-03-21T10:37:21.198652+0000 front 2024-03-21T10:37:21.199080+0000 (oldest deadline 2024-03-21T10:37:45.200271+0000) 2024-03-21T10:38:20.711 INFO:journalctl@ceph.osd.9.smithi047.stdout:Mar 21 10:38:20 smithi047 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-9[64137]: 2024-03-21T10:38:20.406+0000 7f02dc200700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.100:6812 osd.4 since back 2024-03-21T10:37:31.503983+0000 front 2024-03-21T10:37:31.503595+0000 (oldest deadline 2024-03-21T10:37:55.003462+0000) 2024-03-21T10:38:20.711 INFO:journalctl@ceph.osd.9.smithi047.stdout:Mar 21 10:38:20 smithi047 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-9[64137]: 2024-03-21T10:38:20.406+0000 7f02dc200700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.157:6812 osd.5 since back 2024-03-21T10:37:39.105588+0000 front 2024-03-21T10:37:39.105888+0000 (oldest deadline 2024-03-21T10:38:05.005588+0000) 2024-03-21T10:38:20.736 INFO:journalctl@ceph.osd.11.smithi157.stdout:Mar 21 10:38:20 smithi157 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-11[61259]: 2024-03-21T10:38:20.416+0000 7fd0f8400700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-03-21T10:36:54.398353+0000 front 2024-03-21T10:36:54.398068+0000 (oldest deadline 2024-03-21T10:37:16.698280+0000) 2024-03-21T10:38:20.736 INFO:journalctl@ceph.osd.11.smithi157.stdout:Mar 21 10:38:20 smithi157 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-11[61259]: 2024-03-21T10:38:20.416+0000 7fd0f8400700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-03-21T10:37:04.901925+0000 front 2024-03-21T10:37:04.902083+0000 (oldest deadline 2024-03-21T10:37:29.502349+0000) 2024-03-21T10:38:20.736 INFO:journalctl@ceph.osd.11.smithi157.stdout:Mar 21 10:38:20 smithi157 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-11[61259]: 2024-03-21T10:38:20.416+0000 7fd0f8400700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-03-21T10:37:11.804109+0000 front 2024-03-21T10:37:11.804464+0000 (oldest deadline 2024-03-21T10:37:37.104077+0000) 2024-03-21T10:38:20.737 INFO:journalctl@ceph.osd.11.smithi157.stdout:Mar 21 10:38:20 smithi157 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-11[61259]: 2024-03-21T10:38:20.416+0000 7fd0f8400700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.100:6812 osd.4 since back 2024-03-21T10:37:33.508959+0000 front 2024-03-21T10:37:33.509595+0000 (oldest deadline 2024-03-21T10:37:54.609171+0000) 2024-03-21T10:38:20.737 INFO:journalctl@ceph.osd.11.smithi157.stdout:Mar 21 10:38:20 smithi157 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-11[61259]: 2024-03-21T10:38:20.416+0000 7fd0f8400700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.157:6812 osd.5 since back 2024-03-21T10:37:41.512925+0000 front 2024-03-21T10:37:41.513037+0000 (oldest deadline 2024-03-21T10:38:05.013049+0000) 2024-03-21T10:38:20.737 INFO:journalctl@ceph.osd.11.smithi157.stdout:Mar 21 10:38:20 smithi157 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-11[61259]: 2024-03-21T10:38:20.416+0000 7fd0f8400700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.47:6822 osd.6 since back 2024-03-21T10:37:52.015740+0000 front 2024-03-21T10:37:52.015675+0000 (oldest deadline 2024-03-21T10:38:17.315784+0000) 2024-03-21T10:38:21.476 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 10:38:20 smithi100 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-10[59767]: 2024-03-21T10:38:20.983+0000 7f315da00700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-03-21T10:36:52.673512+0000 front 2024-03-21T10:36:52.673688+0000 (oldest deadline 2024-03-21T10:37:16.675039+0000) 2024-03-21T10:38:21.477 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 10:38:20 smithi100 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-10[59767]: 2024-03-21T10:38:20.983+0000 7f315da00700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-03-21T10:37:05.379887+0000 front 2024-03-21T10:37:05.379536+0000 (oldest deadline 2024-03-21T10:37:30.079809+0000) 2024-03-21T10:38:21.477 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 10:38:20 smithi100 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-10[59767]: 2024-03-21T10:38:20.983+0000 7f315da00700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-03-21T10:37:12.981593+0000 front 2024-03-21T10:37:12.981483+0000 (oldest deadline 2024-03-21T10:37:38.281183+0000) 2024-03-21T10:38:21.477 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 10:38:20 smithi100 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-10[59767]: 2024-03-21T10:38:20.983+0000 7f315da00700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.47:6814 osd.3 since back 2024-03-21T10:37:23.583085+0000 front 2024-03-21T10:37:23.583243+0000 (oldest deadline 2024-03-21T10:37:45.283261+0000) 2024-03-21T10:38:21.477 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 10:38:20 smithi100 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-10[59767]: 2024-03-21T10:38:20.983+0000 7f315da00700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.100:6812 osd.4 since back 2024-03-21T10:37:28.185438+0000 front 2024-03-21T10:37:28.185490+0000 (oldest deadline 2024-03-21T10:37:55.786600+0000) 2024-03-21T10:38:21.477 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 10:38:20 smithi100 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-10[59767]: 2024-03-21T10:38:20.983+0000 7f315da00700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.157:6812 osd.5 since back 2024-03-21T10:37:39.888144+0000 front 2024-03-21T10:37:39.888262+0000 (oldest deadline 2024-03-21T10:38:03.888699+0000) 2024-03-21T10:38:21.477 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 10:38:20 smithi100 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-10[59767]: 2024-03-21T10:38:20.983+0000 7f315da00700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.47:6822 osd.6 since back 2024-03-21T10:37:51.491091+0000 front 2024-03-21T10:37:51.491288+0000 (oldest deadline 2024-03-21T10:38:14.390851+0000) 2024-03-21T10:38:21.737 INFO:journalctl@ceph.osd.11.smithi157.stdout:Mar 21 10:38:21 smithi157 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-11[61259]: 2024-03-21T10:38:21.402+0000 7fd0f8400700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-03-21T10:36:54.398353+0000 front 2024-03-21T10:36:54.398068+0000 (oldest deadline 2024-03-21T10:37:16.698280+0000) 2024-03-21T10:38:21.737 INFO:journalctl@ceph.osd.11.smithi157.stdout:Mar 21 10:38:21 smithi157 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-11[61259]: 2024-03-21T10:38:21.402+0000 7fd0f8400700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-03-21T10:37:04.901925+0000 front 2024-03-21T10:37:04.902083+0000 (oldest deadline 2024-03-21T10:37:29.502349+0000) 2024-03-21T10:38:21.737 INFO:journalctl@ceph.osd.11.smithi157.stdout:Mar 21 10:38:21 smithi157 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-11[61259]: 2024-03-21T10:38:21.402+0000 7fd0f8400700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-03-21T10:37:11.804109+0000 front 2024-03-21T10:37:11.804464+0000 (oldest deadline 2024-03-21T10:37:37.104077+0000) 2024-03-21T10:38:21.737 INFO:journalctl@ceph.osd.11.smithi157.stdout:Mar 21 10:38:21 smithi157 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-11[61259]: 2024-03-21T10:38:21.402+0000 7fd0f8400700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.100:6812 osd.4 since back 2024-03-21T10:37:33.508959+0000 front 2024-03-21T10:37:33.509595+0000 (oldest deadline 2024-03-21T10:37:54.609171+0000) 2024-03-21T10:38:21.737 INFO:journalctl@ceph.osd.11.smithi157.stdout:Mar 21 10:38:21 smithi157 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-11[61259]: 2024-03-21T10:38:21.402+0000 7fd0f8400700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.157:6812 osd.5 since back 2024-03-21T10:37:41.512925+0000 front 2024-03-21T10:37:41.513037+0000 (oldest deadline 2024-03-21T10:38:05.013049+0000) 2024-03-21T10:38:21.737 INFO:journalctl@ceph.osd.11.smithi157.stdout:Mar 21 10:38:21 smithi157 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-11[61259]: 2024-03-21T10:38:21.402+0000 7fd0f8400700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.47:6822 osd.6 since back 2024-03-21T10:37:52.015740+0000 front 2024-03-21T10:37:52.015675+0000 (oldest deadline 2024-03-21T10:38:17.315784+0000) 2024-03-21T10:38:21.943 INFO:journalctl@ceph.osd.9.smithi047.stdout:Mar 21 10:38:21 smithi047 podman[103713]: 2024-03-21 10:38:21.662968019 +0000 UTC m=+5.871536453 container died 831d8f0ad59fc7985f9f8f89cc7f1f9c3e3ae875196dbd5f95eb2089616f098f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-9, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240319, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5) 2024-03-21T10:38:22.210 INFO:journalctl@ceph.osd.9.smithi047.stdout:Mar 21 10:38:22 smithi047 podman[103713]: 2024-03-21 10:38:22.131853244 +0000 UTC m=+6.340421699 container cleanup 831d8f0ad59fc7985f9f8f89cc7f1f9c3e3ae875196dbd5f95eb2089616f098f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-9, org.label-schema.build-date=20240319, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-21T10:38:22.210 INFO:journalctl@ceph.osd.9.smithi047.stdout:Mar 21 10:38:22 smithi047 bash[103713]: ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-9 2024-03-21T10:38:22.476 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 10:38:22 smithi100 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-10[59767]: 2024-03-21T10:38:22.007+0000 7f315da00700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-03-21T10:36:52.673512+0000 front 2024-03-21T10:36:52.673688+0000 (oldest deadline 2024-03-21T10:37:16.675039+0000) 2024-03-21T10:38:22.476 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 10:38:22 smithi100 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-10[59767]: 2024-03-21T10:38:22.007+0000 7f315da00700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-03-21T10:37:05.379887+0000 front 2024-03-21T10:37:05.379536+0000 (oldest deadline 2024-03-21T10:37:30.079809+0000) 2024-03-21T10:38:22.476 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 10:38:22 smithi100 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-10[59767]: 2024-03-21T10:38:22.007+0000 7f315da00700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-03-21T10:37:12.981593+0000 front 2024-03-21T10:37:12.981483+0000 (oldest deadline 2024-03-21T10:37:38.281183+0000) 2024-03-21T10:38:22.477 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 10:38:22 smithi100 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-10[59767]: 2024-03-21T10:38:22.007+0000 7f315da00700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.47:6814 osd.3 since back 2024-03-21T10:37:23.583085+0000 front 2024-03-21T10:37:23.583243+0000 (oldest deadline 2024-03-21T10:37:45.283261+0000) 2024-03-21T10:38:22.477 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 10:38:22 smithi100 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-10[59767]: 2024-03-21T10:38:22.007+0000 7f315da00700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.100:6812 osd.4 since back 2024-03-21T10:37:28.185438+0000 front 2024-03-21T10:37:28.185490+0000 (oldest deadline 2024-03-21T10:37:55.786600+0000) 2024-03-21T10:38:22.477 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 10:38:22 smithi100 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-10[59767]: 2024-03-21T10:38:22.007+0000 7f315da00700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.157:6812 osd.5 since back 2024-03-21T10:37:39.888144+0000 front 2024-03-21T10:37:39.888262+0000 (oldest deadline 2024-03-21T10:38:03.888699+0000) 2024-03-21T10:38:22.477 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 10:38:22 smithi100 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-10[59767]: 2024-03-21T10:38:22.007+0000 7f315da00700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.47:6822 osd.6 since back 2024-03-21T10:37:51.491091+0000 front 2024-03-21T10:37:51.491288+0000 (oldest deadline 2024-03-21T10:38:14.390851+0000) 2024-03-21T10:38:22.711 INFO:journalctl@ceph.osd.9.smithi047.stdout:Mar 21 10:38:22 smithi047 podman[103731]: 2024-03-21 10:38:22.357935697 +0000 UTC m=+0.679519321 container remove 831d8f0ad59fc7985f9f8f89cc7f1f9c3e3ae875196dbd5f95eb2089616f098f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-9, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240319, org.label-schema.schema-version=1.0, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, ceph=True, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-21T10:38:22.736 INFO:journalctl@ceph.osd.11.smithi157.stdout:Mar 21 10:38:22 smithi157 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-11[61259]: 2024-03-21T10:38:22.399+0000 7fd0f8400700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-03-21T10:36:54.398353+0000 front 2024-03-21T10:36:54.398068+0000 (oldest deadline 2024-03-21T10:37:16.698280+0000) 2024-03-21T10:38:22.737 INFO:journalctl@ceph.osd.11.smithi157.stdout:Mar 21 10:38:22 smithi157 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-11[61259]: 2024-03-21T10:38:22.399+0000 7fd0f8400700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-03-21T10:37:04.901925+0000 front 2024-03-21T10:37:04.902083+0000 (oldest deadline 2024-03-21T10:37:29.502349+0000) 2024-03-21T10:38:22.737 INFO:journalctl@ceph.osd.11.smithi157.stdout:Mar 21 10:38:22 smithi157 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-11[61259]: 2024-03-21T10:38:22.399+0000 7fd0f8400700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-03-21T10:37:11.804109+0000 front 2024-03-21T10:37:11.804464+0000 (oldest deadline 2024-03-21T10:37:37.104077+0000) 2024-03-21T10:38:22.737 INFO:journalctl@ceph.osd.11.smithi157.stdout:Mar 21 10:38:22 smithi157 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-11[61259]: 2024-03-21T10:38:22.399+0000 7fd0f8400700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.100:6812 osd.4 since back 2024-03-21T10:37:33.508959+0000 front 2024-03-21T10:37:33.509595+0000 (oldest deadline 2024-03-21T10:37:54.609171+0000) 2024-03-21T10:38:22.737 INFO:journalctl@ceph.osd.11.smithi157.stdout:Mar 21 10:38:22 smithi157 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-11[61259]: 2024-03-21T10:38:22.399+0000 7fd0f8400700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.157:6812 osd.5 since back 2024-03-21T10:37:41.512925+0000 front 2024-03-21T10:37:41.513037+0000 (oldest deadline 2024-03-21T10:38:05.013049+0000) 2024-03-21T10:38:22.737 INFO:journalctl@ceph.osd.11.smithi157.stdout:Mar 21 10:38:22 smithi157 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-11[61259]: 2024-03-21T10:38:22.399+0000 7fd0f8400700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.47:6822 osd.6 since back 2024-03-21T10:37:52.015740+0000 front 2024-03-21T10:37:52.015675+0000 (oldest deadline 2024-03-21T10:38:17.315784+0000) 2024-03-21T10:38:23.476 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 10:38:23 smithi100 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-10[59767]: 2024-03-21T10:38:23.055+0000 7f315da00700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-03-21T10:36:52.673512+0000 front 2024-03-21T10:36:52.673688+0000 (oldest deadline 2024-03-21T10:37:16.675039+0000) 2024-03-21T10:38:23.477 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 10:38:23 smithi100 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-10[59767]: 2024-03-21T10:38:23.055+0000 7f315da00700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-03-21T10:37:05.379887+0000 front 2024-03-21T10:37:05.379536+0000 (oldest deadline 2024-03-21T10:37:30.079809+0000) 2024-03-21T10:38:23.477 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 10:38:23 smithi100 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-10[59767]: 2024-03-21T10:38:23.055+0000 7f315da00700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-03-21T10:37:12.981593+0000 front 2024-03-21T10:37:12.981483+0000 (oldest deadline 2024-03-21T10:37:38.281183+0000) 2024-03-21T10:38:23.477 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 10:38:23 smithi100 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-10[59767]: 2024-03-21T10:38:23.055+0000 7f315da00700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.47:6814 osd.3 since back 2024-03-21T10:37:23.583085+0000 front 2024-03-21T10:37:23.583243+0000 (oldest deadline 2024-03-21T10:37:45.283261+0000) 2024-03-21T10:38:23.477 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 10:38:23 smithi100 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-10[59767]: 2024-03-21T10:38:23.055+0000 7f315da00700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.100:6812 osd.4 since back 2024-03-21T10:37:28.185438+0000 front 2024-03-21T10:37:28.185490+0000 (oldest deadline 2024-03-21T10:37:55.786600+0000) 2024-03-21T10:38:23.477 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 10:38:23 smithi100 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-10[59767]: 2024-03-21T10:38:23.055+0000 7f315da00700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.157:6812 osd.5 since back 2024-03-21T10:37:39.888144+0000 front 2024-03-21T10:37:39.888262+0000 (oldest deadline 2024-03-21T10:38:03.888699+0000) 2024-03-21T10:38:23.477 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 10:38:23 smithi100 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-10[59767]: 2024-03-21T10:38:23.055+0000 7f315da00700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.47:6822 osd.6 since back 2024-03-21T10:37:51.491091+0000 front 2024-03-21T10:37:51.491288+0000 (oldest deadline 2024-03-21T10:38:14.390851+0000) 2024-03-21T10:38:23.535 INFO:journalctl@ceph.osd.9.smithi047.stdout:Mar 21 10:38:23 smithi047 podman[103821]: 2024-03-21 10:38:23.127456208 +0000 UTC m=+0.036956360 image pull 237e43c7a65ff48534d94b97c1bfdbe4b7be5672e3d9a613745083ea3702aa54 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232 2024-03-21T10:38:23.535 INFO:journalctl@ceph.osd.9.smithi047.stdout:Mar 21 10:38:23 smithi047 podman[103821]: 2024-03-21 10:38:23.266928601 +0000 UTC m=+0.176428780 container create 99bf16d7ae6c5c7ddc6de3560a73060f69a59416e56259891f5fca64907b5f5c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-9-deactivate, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240319, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=) 2024-03-21T10:38:23.535 INFO:journalctl@ceph.osd.9.smithi047.stdout:Mar 21 10:38:23 smithi047 podman[103821]: 2024-03-21 10:38:23.537388679 +0000 UTC m=+0.446888865 container init 99bf16d7ae6c5c7ddc6de3560a73060f69a59416e56259891f5fca64907b5f5c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-9-deactivate, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240319, CEPH_POINT_RELEASE=, ceph=True, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-21T10:38:23.736 INFO:journalctl@ceph.osd.11.smithi157.stdout:Mar 21 10:38:23 smithi157 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-11[61259]: 2024-03-21T10:38:23.407+0000 7fd0f8400700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-03-21T10:36:54.398353+0000 front 2024-03-21T10:36:54.398068+0000 (oldest deadline 2024-03-21T10:37:16.698280+0000) 2024-03-21T10:38:23.736 INFO:journalctl@ceph.osd.11.smithi157.stdout:Mar 21 10:38:23 smithi157 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-11[61259]: 2024-03-21T10:38:23.407+0000 7fd0f8400700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-03-21T10:37:04.901925+0000 front 2024-03-21T10:37:04.902083+0000 (oldest deadline 2024-03-21T10:37:29.502349+0000) 2024-03-21T10:38:23.737 INFO:journalctl@ceph.osd.11.smithi157.stdout:Mar 21 10:38:23 smithi157 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-11[61259]: 2024-03-21T10:38:23.407+0000 7fd0f8400700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-03-21T10:37:11.804109+0000 front 2024-03-21T10:37:11.804464+0000 (oldest deadline 2024-03-21T10:37:37.104077+0000) 2024-03-21T10:38:23.737 INFO:journalctl@ceph.osd.11.smithi157.stdout:Mar 21 10:38:23 smithi157 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-11[61259]: 2024-03-21T10:38:23.407+0000 7fd0f8400700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.100:6812 osd.4 since back 2024-03-21T10:37:33.508959+0000 front 2024-03-21T10:37:33.509595+0000 (oldest deadline 2024-03-21T10:37:54.609171+0000) 2024-03-21T10:38:23.737 INFO:journalctl@ceph.osd.11.smithi157.stdout:Mar 21 10:38:23 smithi157 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-11[61259]: 2024-03-21T10:38:23.407+0000 7fd0f8400700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.157:6812 osd.5 since back 2024-03-21T10:37:41.512925+0000 front 2024-03-21T10:37:41.513037+0000 (oldest deadline 2024-03-21T10:38:05.013049+0000) 2024-03-21T10:38:23.737 INFO:journalctl@ceph.osd.11.smithi157.stdout:Mar 21 10:38:23 smithi157 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-11[61259]: 2024-03-21T10:38:23.407+0000 7fd0f8400700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.47:6822 osd.6 since back 2024-03-21T10:37:52.015740+0000 front 2024-03-21T10:37:52.015675+0000 (oldest deadline 2024-03-21T10:38:17.315784+0000) 2024-03-21T10:38:23.960 INFO:journalctl@ceph.osd.9.smithi047.stdout:Mar 21 10:38:23 smithi047 podman[103821]: 2024-03-21 10:38:23.548811817 +0000 UTC m=+0.458311968 container start 99bf16d7ae6c5c7ddc6de3560a73060f69a59416e56259891f5fca64907b5f5c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-9-deactivate, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, org.label-schema.build-date=20240319, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, CEPH_POINT_RELEASE=) 2024-03-21T10:38:23.960 INFO:journalctl@ceph.osd.9.smithi047.stdout:Mar 21 10:38:23 smithi047 podman[103821]: 2024-03-21 10:38:23.769084677 +0000 UTC m=+0.678584896 container attach 99bf16d7ae6c5c7ddc6de3560a73060f69a59416e56259891f5fca64907b5f5c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-9-deactivate, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, org.label-schema.build-date=20240319, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, CEPH_POINT_RELEASE=, ceph=True, GIT_BRANCH=HEAD, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, GIT_CLEAN=True) 2024-03-21T10:38:24.331 INFO:journalctl@ceph.osd.9.smithi047.stdout:Mar 21 10:38:23 smithi047 podman[103821]: 2024-03-21 10:38:23.996298026 +0000 UTC m=+0.905798168 container died 99bf16d7ae6c5c7ddc6de3560a73060f69a59416e56259891f5fca64907b5f5c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-9-deactivate, org.label-schema.build-date=20240319, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.license=GPLv2) 2024-03-21T10:38:24.379 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 10:38:24 smithi100 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-10[59767]: 2024-03-21T10:38:24.075+0000 7f315da00700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-03-21T10:36:52.673512+0000 front 2024-03-21T10:36:52.673688+0000 (oldest deadline 2024-03-21T10:37:16.675039+0000) 2024-03-21T10:38:24.379 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 10:38:24 smithi100 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-10[59767]: 2024-03-21T10:38:24.075+0000 7f315da00700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-03-21T10:37:05.379887+0000 front 2024-03-21T10:37:05.379536+0000 (oldest deadline 2024-03-21T10:37:30.079809+0000) 2024-03-21T10:38:24.379 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 10:38:24 smithi100 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-10[59767]: 2024-03-21T10:38:24.075+0000 7f315da00700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-03-21T10:37:12.981593+0000 front 2024-03-21T10:37:12.981483+0000 (oldest deadline 2024-03-21T10:37:38.281183+0000) 2024-03-21T10:38:24.379 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 10:38:24 smithi100 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-10[59767]: 2024-03-21T10:38:24.075+0000 7f315da00700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.47:6814 osd.3 since back 2024-03-21T10:37:23.583085+0000 front 2024-03-21T10:37:23.583243+0000 (oldest deadline 2024-03-21T10:37:45.283261+0000) 2024-03-21T10:38:24.379 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 10:38:24 smithi100 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-10[59767]: 2024-03-21T10:38:24.075+0000 7f315da00700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.100:6812 osd.4 since back 2024-03-21T10:37:28.185438+0000 front 2024-03-21T10:37:28.185490+0000 (oldest deadline 2024-03-21T10:37:55.786600+0000) 2024-03-21T10:38:24.379 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 10:38:24 smithi100 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-10[59767]: 2024-03-21T10:38:24.075+0000 7f315da00700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.157:6812 osd.5 since back 2024-03-21T10:37:39.888144+0000 front 2024-03-21T10:37:39.888262+0000 (oldest deadline 2024-03-21T10:38:03.888699+0000) 2024-03-21T10:38:24.379 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 10:38:24 smithi100 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-10[59767]: 2024-03-21T10:38:24.075+0000 7f315da00700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.47:6822 osd.6 since back 2024-03-21T10:37:51.491091+0000 front 2024-03-21T10:37:51.491288+0000 (oldest deadline 2024-03-21T10:38:14.390851+0000) 2024-03-21T10:38:24.704 DEBUG:teuthology.orchestra.run.smithi047:> sudo pkill -f 'journalctl -f -n 0 -u ceph-3056d51a-e76b-11ee-95cd-87774f69a715@osd.9.service' 2024-03-21T10:38:24.737 INFO:journalctl@ceph.osd.11.smithi157.stdout:Mar 21 10:38:24 smithi157 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-11[61259]: 2024-03-21T10:38:24.444+0000 7fd0f8400700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-03-21T10:36:54.398353+0000 front 2024-03-21T10:36:54.398068+0000 (oldest deadline 2024-03-21T10:37:16.698280+0000) 2024-03-21T10:38:24.737 INFO:journalctl@ceph.osd.11.smithi157.stdout:Mar 21 10:38:24 smithi157 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-11[61259]: 2024-03-21T10:38:24.444+0000 7fd0f8400700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-03-21T10:37:04.901925+0000 front 2024-03-21T10:37:04.902083+0000 (oldest deadline 2024-03-21T10:37:29.502349+0000) 2024-03-21T10:38:24.737 INFO:journalctl@ceph.osd.11.smithi157.stdout:Mar 21 10:38:24 smithi157 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-11[61259]: 2024-03-21T10:38:24.444+0000 7fd0f8400700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-03-21T10:37:11.804109+0000 front 2024-03-21T10:37:11.804464+0000 (oldest deadline 2024-03-21T10:37:37.104077+0000) 2024-03-21T10:38:24.737 INFO:journalctl@ceph.osd.11.smithi157.stdout:Mar 21 10:38:24 smithi157 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-11[61259]: 2024-03-21T10:38:24.444+0000 7fd0f8400700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.100:6812 osd.4 since back 2024-03-21T10:37:33.508959+0000 front 2024-03-21T10:37:33.509595+0000 (oldest deadline 2024-03-21T10:37:54.609171+0000) 2024-03-21T10:38:24.737 INFO:journalctl@ceph.osd.11.smithi157.stdout:Mar 21 10:38:24 smithi157 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-11[61259]: 2024-03-21T10:38:24.444+0000 7fd0f8400700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.157:6812 osd.5 since back 2024-03-21T10:37:41.512925+0000 front 2024-03-21T10:37:41.513037+0000 (oldest deadline 2024-03-21T10:38:05.013049+0000) 2024-03-21T10:38:24.737 INFO:journalctl@ceph.osd.11.smithi157.stdout:Mar 21 10:38:24 smithi157 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-11[61259]: 2024-03-21T10:38:24.444+0000 7fd0f8400700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.47:6822 osd.6 since back 2024-03-21T10:37:52.015740+0000 front 2024-03-21T10:37:52.015675+0000 (oldest deadline 2024-03-21T10:38:17.315784+0000) 2024-03-21T10:38:24.839 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-21T10:38:24.839 INFO:tasks.cephadm.osd.9:Stopped osd.9 2024-03-21T10:38:24.839 INFO:tasks.cephadm.osd.10:Stopping osd.10... 2024-03-21T10:38:24.839 DEBUG:teuthology.orchestra.run.smithi100:> sudo systemctl stop ceph-3056d51a-e76b-11ee-95cd-87774f69a715@osd.10 2024-03-21T10:38:25.226 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 10:38:24 smithi100 systemd[1]: Stopping Ceph osd.10 for 3056d51a-e76b-11ee-95cd-87774f69a715... 2024-03-21T10:38:25.226 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 10:38:25 smithi100 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-10[59767]: 2024-03-21T10:38:25.119+0000 7f315da00700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-03-21T10:36:52.673512+0000 front 2024-03-21T10:36:52.673688+0000 (oldest deadline 2024-03-21T10:37:16.675039+0000) 2024-03-21T10:38:25.226 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 10:38:25 smithi100 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-10[59767]: 2024-03-21T10:38:25.119+0000 7f315da00700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-03-21T10:37:05.379887+0000 front 2024-03-21T10:37:05.379536+0000 (oldest deadline 2024-03-21T10:37:30.079809+0000) 2024-03-21T10:38:25.226 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 10:38:25 smithi100 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-10[59767]: 2024-03-21T10:38:25.119+0000 7f315da00700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-03-21T10:37:12.981593+0000 front 2024-03-21T10:37:12.981483+0000 (oldest deadline 2024-03-21T10:37:38.281183+0000) 2024-03-21T10:38:25.226 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 10:38:25 smithi100 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-10[59767]: 2024-03-21T10:38:25.119+0000 7f315da00700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.47:6814 osd.3 since back 2024-03-21T10:37:23.583085+0000 front 2024-03-21T10:37:23.583243+0000 (oldest deadline 2024-03-21T10:37:45.283261+0000) 2024-03-21T10:38:25.227 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 10:38:25 smithi100 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-10[59767]: 2024-03-21T10:38:25.119+0000 7f315da00700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.100:6812 osd.4 since back 2024-03-21T10:37:28.185438+0000 front 2024-03-21T10:37:28.185490+0000 (oldest deadline 2024-03-21T10:37:55.786600+0000) 2024-03-21T10:38:25.227 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 10:38:25 smithi100 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-10[59767]: 2024-03-21T10:38:25.119+0000 7f315da00700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.157:6812 osd.5 since back 2024-03-21T10:37:39.888144+0000 front 2024-03-21T10:37:39.888262+0000 (oldest deadline 2024-03-21T10:38:03.888699+0000) 2024-03-21T10:38:25.227 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 10:38:25 smithi100 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-10[59767]: 2024-03-21T10:38:25.119+0000 7f315da00700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.47:6822 osd.6 since back 2024-03-21T10:37:51.491091+0000 front 2024-03-21T10:37:51.491288+0000 (oldest deadline 2024-03-21T10:38:14.390851+0000) 2024-03-21T10:38:25.726 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 10:38:25 smithi100 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-10[59767]: 2024-03-21T10:38:25.402+0000 7f3167200700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.10 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-21T10:38:25.726 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 10:38:25 smithi100 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-10[59767]: 2024-03-21T10:38:25.402+0000 7f3167200700 -1 osd.10 111 *** Got signal Terminated *** 2024-03-21T10:38:25.727 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 10:38:25 smithi100 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-10[59767]: 2024-03-21T10:38:25.402+0000 7f3167200700 -1 osd.10 111 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-21T10:38:25.736 INFO:journalctl@ceph.osd.11.smithi157.stdout:Mar 21 10:38:25 smithi157 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-11[61259]: 2024-03-21T10:38:25.413+0000 7fd0f8400700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-03-21T10:36:54.398353+0000 front 2024-03-21T10:36:54.398068+0000 (oldest deadline 2024-03-21T10:37:16.698280+0000) 2024-03-21T10:38:25.736 INFO:journalctl@ceph.osd.11.smithi157.stdout:Mar 21 10:38:25 smithi157 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-11[61259]: 2024-03-21T10:38:25.413+0000 7fd0f8400700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-03-21T10:37:04.901925+0000 front 2024-03-21T10:37:04.902083+0000 (oldest deadline 2024-03-21T10:37:29.502349+0000) 2024-03-21T10:38:25.737 INFO:journalctl@ceph.osd.11.smithi157.stdout:Mar 21 10:38:25 smithi157 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-11[61259]: 2024-03-21T10:38:25.413+0000 7fd0f8400700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-03-21T10:37:11.804109+0000 front 2024-03-21T10:37:11.804464+0000 (oldest deadline 2024-03-21T10:37:37.104077+0000) 2024-03-21T10:38:25.737 INFO:journalctl@ceph.osd.11.smithi157.stdout:Mar 21 10:38:25 smithi157 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-11[61259]: 2024-03-21T10:38:25.413+0000 7fd0f8400700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.100:6812 osd.4 since back 2024-03-21T10:37:33.508959+0000 front 2024-03-21T10:37:33.509595+0000 (oldest deadline 2024-03-21T10:37:54.609171+0000) 2024-03-21T10:38:25.737 INFO:journalctl@ceph.osd.11.smithi157.stdout:Mar 21 10:38:25 smithi157 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-11[61259]: 2024-03-21T10:38:25.413+0000 7fd0f8400700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.157:6812 osd.5 since back 2024-03-21T10:37:41.512925+0000 front 2024-03-21T10:37:41.513037+0000 (oldest deadline 2024-03-21T10:38:05.013049+0000) 2024-03-21T10:38:25.737 INFO:journalctl@ceph.osd.11.smithi157.stdout:Mar 21 10:38:25 smithi157 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-11[61259]: 2024-03-21T10:38:25.413+0000 7fd0f8400700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.47:6822 osd.6 since back 2024-03-21T10:37:52.015740+0000 front 2024-03-21T10:37:52.015675+0000 (oldest deadline 2024-03-21T10:38:17.315784+0000) 2024-03-21T10:38:26.477 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 10:38:26 smithi100 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-10[59767]: 2024-03-21T10:38:26.157+0000 7f315da00700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-03-21T10:36:52.673512+0000 front 2024-03-21T10:36:52.673688+0000 (oldest deadline 2024-03-21T10:37:16.675039+0000) 2024-03-21T10:38:26.477 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 10:38:26 smithi100 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-10[59767]: 2024-03-21T10:38:26.157+0000 7f315da00700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-03-21T10:37:05.379887+0000 front 2024-03-21T10:37:05.379536+0000 (oldest deadline 2024-03-21T10:37:30.079809+0000) 2024-03-21T10:38:26.477 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 10:38:26 smithi100 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-10[59767]: 2024-03-21T10:38:26.157+0000 7f315da00700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-03-21T10:37:12.981593+0000 front 2024-03-21T10:37:12.981483+0000 (oldest deadline 2024-03-21T10:37:38.281183+0000) 2024-03-21T10:38:26.477 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 10:38:26 smithi100 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-10[59767]: 2024-03-21T10:38:26.157+0000 7f315da00700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.47:6814 osd.3 since back 2024-03-21T10:37:23.583085+0000 front 2024-03-21T10:37:23.583243+0000 (oldest deadline 2024-03-21T10:37:45.283261+0000) 2024-03-21T10:38:26.477 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 10:38:26 smithi100 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-10[59767]: 2024-03-21T10:38:26.157+0000 7f315da00700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.100:6812 osd.4 since back 2024-03-21T10:37:28.185438+0000 front 2024-03-21T10:37:28.185490+0000 (oldest deadline 2024-03-21T10:37:55.786600+0000) 2024-03-21T10:38:26.477 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 10:38:26 smithi100 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-10[59767]: 2024-03-21T10:38:26.157+0000 7f315da00700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.157:6812 osd.5 since back 2024-03-21T10:37:39.888144+0000 front 2024-03-21T10:37:39.888262+0000 (oldest deadline 2024-03-21T10:38:03.888699+0000) 2024-03-21T10:38:26.478 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 10:38:26 smithi100 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-10[59767]: 2024-03-21T10:38:26.157+0000 7f315da00700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.47:6822 osd.6 since back 2024-03-21T10:37:51.491091+0000 front 2024-03-21T10:37:51.491288+0000 (oldest deadline 2024-03-21T10:38:14.390851+0000) 2024-03-21T10:38:26.736 INFO:journalctl@ceph.osd.11.smithi157.stdout:Mar 21 10:38:26 smithi157 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-11[61259]: 2024-03-21T10:38:26.376+0000 7fd0f8400700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-03-21T10:36:54.398353+0000 front 2024-03-21T10:36:54.398068+0000 (oldest deadline 2024-03-21T10:37:16.698280+0000) 2024-03-21T10:38:26.737 INFO:journalctl@ceph.osd.11.smithi157.stdout:Mar 21 10:38:26 smithi157 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-11[61259]: 2024-03-21T10:38:26.376+0000 7fd0f8400700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-03-21T10:37:04.901925+0000 front 2024-03-21T10:37:04.902083+0000 (oldest deadline 2024-03-21T10:37:29.502349+0000) 2024-03-21T10:38:26.737 INFO:journalctl@ceph.osd.11.smithi157.stdout:Mar 21 10:38:26 smithi157 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-11[61259]: 2024-03-21T10:38:26.376+0000 7fd0f8400700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-03-21T10:37:11.804109+0000 front 2024-03-21T10:37:11.804464+0000 (oldest deadline 2024-03-21T10:37:37.104077+0000) 2024-03-21T10:38:26.737 INFO:journalctl@ceph.osd.11.smithi157.stdout:Mar 21 10:38:26 smithi157 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-11[61259]: 2024-03-21T10:38:26.376+0000 7fd0f8400700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.100:6812 osd.4 since back 2024-03-21T10:37:33.508959+0000 front 2024-03-21T10:37:33.509595+0000 (oldest deadline 2024-03-21T10:37:54.609171+0000) 2024-03-21T10:38:26.737 INFO:journalctl@ceph.osd.11.smithi157.stdout:Mar 21 10:38:26 smithi157 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-11[61259]: 2024-03-21T10:38:26.376+0000 7fd0f8400700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.157:6812 osd.5 since back 2024-03-21T10:37:41.512925+0000 front 2024-03-21T10:37:41.513037+0000 (oldest deadline 2024-03-21T10:38:05.013049+0000) 2024-03-21T10:38:26.737 INFO:journalctl@ceph.osd.11.smithi157.stdout:Mar 21 10:38:26 smithi157 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-11[61259]: 2024-03-21T10:38:26.376+0000 7fd0f8400700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.47:6822 osd.6 since back 2024-03-21T10:37:52.015740+0000 front 2024-03-21T10:37:52.015675+0000 (oldest deadline 2024-03-21T10:38:17.315784+0000) 2024-03-21T10:38:26.737 INFO:journalctl@ceph.osd.11.smithi157.stdout:Mar 21 10:38:26 smithi157 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-11[61259]: 2024-03-21T10:38:26.376+0000 7fd0f8400700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.100:6820 osd.7 since back 2024-03-21T10:37:57.316871+0000 front 2024-03-21T10:37:57.316918+0000 (oldest deadline 2024-03-21T10:38:25.517713+0000) 2024-03-21T10:38:27.476 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 10:38:27 smithi100 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-10[59767]: 2024-03-21T10:38:27.187+0000 7f315da00700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-03-21T10:36:52.673512+0000 front 2024-03-21T10:36:52.673688+0000 (oldest deadline 2024-03-21T10:37:16.675039+0000) 2024-03-21T10:38:27.477 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 10:38:27 smithi100 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-10[59767]: 2024-03-21T10:38:27.187+0000 7f315da00700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-03-21T10:37:05.379887+0000 front 2024-03-21T10:37:05.379536+0000 (oldest deadline 2024-03-21T10:37:30.079809+0000) 2024-03-21T10:38:27.477 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 10:38:27 smithi100 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-10[59767]: 2024-03-21T10:38:27.187+0000 7f315da00700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-03-21T10:37:12.981593+0000 front 2024-03-21T10:37:12.981483+0000 (oldest deadline 2024-03-21T10:37:38.281183+0000) 2024-03-21T10:38:27.477 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 10:38:27 smithi100 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-10[59767]: 2024-03-21T10:38:27.187+0000 7f315da00700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.47:6814 osd.3 since back 2024-03-21T10:37:23.583085+0000 front 2024-03-21T10:37:23.583243+0000 (oldest deadline 2024-03-21T10:37:45.283261+0000) 2024-03-21T10:38:27.477 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 10:38:27 smithi100 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-10[59767]: 2024-03-21T10:38:27.187+0000 7f315da00700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.100:6812 osd.4 since back 2024-03-21T10:37:28.185438+0000 front 2024-03-21T10:37:28.185490+0000 (oldest deadline 2024-03-21T10:37:55.786600+0000) 2024-03-21T10:38:27.477 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 10:38:27 smithi100 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-10[59767]: 2024-03-21T10:38:27.187+0000 7f315da00700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.157:6812 osd.5 since back 2024-03-21T10:37:39.888144+0000 front 2024-03-21T10:37:39.888262+0000 (oldest deadline 2024-03-21T10:38:03.888699+0000) 2024-03-21T10:38:27.477 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 10:38:27 smithi100 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-10[59767]: 2024-03-21T10:38:27.187+0000 7f315da00700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.47:6822 osd.6 since back 2024-03-21T10:37:51.491091+0000 front 2024-03-21T10:37:51.491288+0000 (oldest deadline 2024-03-21T10:38:14.390851+0000) 2024-03-21T10:38:27.737 INFO:journalctl@ceph.osd.11.smithi157.stdout:Mar 21 10:38:27 smithi157 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-11[61259]: 2024-03-21T10:38:27.364+0000 7fd0f8400700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-03-21T10:36:54.398353+0000 front 2024-03-21T10:36:54.398068+0000 (oldest deadline 2024-03-21T10:37:16.698280+0000) 2024-03-21T10:38:27.737 INFO:journalctl@ceph.osd.11.smithi157.stdout:Mar 21 10:38:27 smithi157 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-11[61259]: 2024-03-21T10:38:27.364+0000 7fd0f8400700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-03-21T10:37:04.901925+0000 front 2024-03-21T10:37:04.902083+0000 (oldest deadline 2024-03-21T10:37:29.502349+0000) 2024-03-21T10:38:27.737 INFO:journalctl@ceph.osd.11.smithi157.stdout:Mar 21 10:38:27 smithi157 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-11[61259]: 2024-03-21T10:38:27.364+0000 7fd0f8400700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-03-21T10:37:11.804109+0000 front 2024-03-21T10:37:11.804464+0000 (oldest deadline 2024-03-21T10:37:37.104077+0000) 2024-03-21T10:38:27.737 INFO:journalctl@ceph.osd.11.smithi157.stdout:Mar 21 10:38:27 smithi157 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-11[61259]: 2024-03-21T10:38:27.364+0000 7fd0f8400700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.100:6812 osd.4 since back 2024-03-21T10:37:33.508959+0000 front 2024-03-21T10:37:33.509595+0000 (oldest deadline 2024-03-21T10:37:54.609171+0000) 2024-03-21T10:38:27.737 INFO:journalctl@ceph.osd.11.smithi157.stdout:Mar 21 10:38:27 smithi157 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-11[61259]: 2024-03-21T10:38:27.364+0000 7fd0f8400700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.157:6812 osd.5 since back 2024-03-21T10:37:41.512925+0000 front 2024-03-21T10:37:41.513037+0000 (oldest deadline 2024-03-21T10:38:05.013049+0000) 2024-03-21T10:38:27.737 INFO:journalctl@ceph.osd.11.smithi157.stdout:Mar 21 10:38:27 smithi157 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-11[61259]: 2024-03-21T10:38:27.364+0000 7fd0f8400700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.47:6822 osd.6 since back 2024-03-21T10:37:52.015740+0000 front 2024-03-21T10:37:52.015675+0000 (oldest deadline 2024-03-21T10:38:17.315784+0000) 2024-03-21T10:38:27.737 INFO:journalctl@ceph.osd.11.smithi157.stdout:Mar 21 10:38:27 smithi157 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-11[61259]: 2024-03-21T10:38:27.364+0000 7fd0f8400700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.100:6820 osd.7 since back 2024-03-21T10:37:57.316871+0000 front 2024-03-21T10:37:57.316918+0000 (oldest deadline 2024-03-21T10:38:25.517713+0000) 2024-03-21T10:38:28.476 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 10:38:28 smithi100 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-10[59767]: 2024-03-21T10:38:28.186+0000 7f315da00700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-03-21T10:36:52.673512+0000 front 2024-03-21T10:36:52.673688+0000 (oldest deadline 2024-03-21T10:37:16.675039+0000) 2024-03-21T10:38:28.476 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 10:38:28 smithi100 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-10[59767]: 2024-03-21T10:38:28.186+0000 7f315da00700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-03-21T10:37:05.379887+0000 front 2024-03-21T10:37:05.379536+0000 (oldest deadline 2024-03-21T10:37:30.079809+0000) 2024-03-21T10:38:28.476 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 10:38:28 smithi100 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-10[59767]: 2024-03-21T10:38:28.186+0000 7f315da00700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-03-21T10:37:12.981593+0000 front 2024-03-21T10:37:12.981483+0000 (oldest deadline 2024-03-21T10:37:38.281183+0000) 2024-03-21T10:38:28.477 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 10:38:28 smithi100 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-10[59767]: 2024-03-21T10:38:28.186+0000 7f315da00700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.47:6814 osd.3 since back 2024-03-21T10:37:23.583085+0000 front 2024-03-21T10:37:23.583243+0000 (oldest deadline 2024-03-21T10:37:45.283261+0000) 2024-03-21T10:38:28.477 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 10:38:28 smithi100 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-10[59767]: 2024-03-21T10:38:28.186+0000 7f315da00700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.100:6812 osd.4 since back 2024-03-21T10:37:28.185438+0000 front 2024-03-21T10:37:28.185490+0000 (oldest deadline 2024-03-21T10:37:55.786600+0000) 2024-03-21T10:38:28.477 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 10:38:28 smithi100 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-10[59767]: 2024-03-21T10:38:28.186+0000 7f315da00700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.157:6812 osd.5 since back 2024-03-21T10:37:39.888144+0000 front 2024-03-21T10:37:39.888262+0000 (oldest deadline 2024-03-21T10:38:03.888699+0000) 2024-03-21T10:38:28.477 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 10:38:28 smithi100 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-10[59767]: 2024-03-21T10:38:28.186+0000 7f315da00700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.47:6822 osd.6 since back 2024-03-21T10:37:51.491091+0000 front 2024-03-21T10:37:51.491288+0000 (oldest deadline 2024-03-21T10:38:14.390851+0000) 2024-03-21T10:38:28.486 INFO:journalctl@ceph.osd.11.smithi157.stdout:Mar 21 10:38:28 smithi157 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-11[61259]: 2024-03-21T10:38:28.316+0000 7fd0f8400700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-03-21T10:36:54.398353+0000 front 2024-03-21T10:36:54.398068+0000 (oldest deadline 2024-03-21T10:37:16.698280+0000) 2024-03-21T10:38:28.486 INFO:journalctl@ceph.osd.11.smithi157.stdout:Mar 21 10:38:28 smithi157 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-11[61259]: 2024-03-21T10:38:28.316+0000 7fd0f8400700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-03-21T10:37:04.901925+0000 front 2024-03-21T10:37:04.902083+0000 (oldest deadline 2024-03-21T10:37:29.502349+0000) 2024-03-21T10:38:28.486 INFO:journalctl@ceph.osd.11.smithi157.stdout:Mar 21 10:38:28 smithi157 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-11[61259]: 2024-03-21T10:38:28.316+0000 7fd0f8400700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-03-21T10:37:11.804109+0000 front 2024-03-21T10:37:11.804464+0000 (oldest deadline 2024-03-21T10:37:37.104077+0000) 2024-03-21T10:38:28.487 INFO:journalctl@ceph.osd.11.smithi157.stdout:Mar 21 10:38:28 smithi157 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-11[61259]: 2024-03-21T10:38:28.316+0000 7fd0f8400700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.100:6812 osd.4 since back 2024-03-21T10:37:33.508959+0000 front 2024-03-21T10:37:33.509595+0000 (oldest deadline 2024-03-21T10:37:54.609171+0000) 2024-03-21T10:38:28.487 INFO:journalctl@ceph.osd.11.smithi157.stdout:Mar 21 10:38:28 smithi157 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-11[61259]: 2024-03-21T10:38:28.316+0000 7fd0f8400700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.157:6812 osd.5 since back 2024-03-21T10:37:41.512925+0000 front 2024-03-21T10:37:41.513037+0000 (oldest deadline 2024-03-21T10:38:05.013049+0000) 2024-03-21T10:38:28.487 INFO:journalctl@ceph.osd.11.smithi157.stdout:Mar 21 10:38:28 smithi157 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-11[61259]: 2024-03-21T10:38:28.316+0000 7fd0f8400700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.47:6822 osd.6 since back 2024-03-21T10:37:52.015740+0000 front 2024-03-21T10:37:52.015675+0000 (oldest deadline 2024-03-21T10:38:17.315784+0000) 2024-03-21T10:38:28.487 INFO:journalctl@ceph.osd.11.smithi157.stdout:Mar 21 10:38:28 smithi157 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-11[61259]: 2024-03-21T10:38:28.316+0000 7fd0f8400700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.100:6820 osd.7 since back 2024-03-21T10:37:57.316871+0000 front 2024-03-21T10:37:57.316918+0000 (oldest deadline 2024-03-21T10:38:25.517713+0000) 2024-03-21T10:38:29.476 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 10:38:29 smithi100 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-10[59767]: 2024-03-21T10:38:29.177+0000 7f315da00700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-03-21T10:36:52.673512+0000 front 2024-03-21T10:36:52.673688+0000 (oldest deadline 2024-03-21T10:37:16.675039+0000) 2024-03-21T10:38:29.476 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 10:38:29 smithi100 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-10[59767]: 2024-03-21T10:38:29.177+0000 7f315da00700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-03-21T10:37:05.379887+0000 front 2024-03-21T10:37:05.379536+0000 (oldest deadline 2024-03-21T10:37:30.079809+0000) 2024-03-21T10:38:29.477 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 10:38:29 smithi100 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-10[59767]: 2024-03-21T10:38:29.177+0000 7f315da00700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-03-21T10:37:12.981593+0000 front 2024-03-21T10:37:12.981483+0000 (oldest deadline 2024-03-21T10:37:38.281183+0000) 2024-03-21T10:38:29.477 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 10:38:29 smithi100 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-10[59767]: 2024-03-21T10:38:29.177+0000 7f315da00700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.47:6814 osd.3 since back 2024-03-21T10:37:23.583085+0000 front 2024-03-21T10:37:23.583243+0000 (oldest deadline 2024-03-21T10:37:45.283261+0000) 2024-03-21T10:38:29.477 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 10:38:29 smithi100 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-10[59767]: 2024-03-21T10:38:29.177+0000 7f315da00700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.100:6812 osd.4 since back 2024-03-21T10:37:28.185438+0000 front 2024-03-21T10:37:28.185490+0000 (oldest deadline 2024-03-21T10:37:55.786600+0000) 2024-03-21T10:38:29.477 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 10:38:29 smithi100 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-10[59767]: 2024-03-21T10:38:29.177+0000 7f315da00700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.157:6812 osd.5 since back 2024-03-21T10:37:39.888144+0000 front 2024-03-21T10:37:39.888262+0000 (oldest deadline 2024-03-21T10:38:03.888699+0000) 2024-03-21T10:38:29.477 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 10:38:29 smithi100 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-10[59767]: 2024-03-21T10:38:29.177+0000 7f315da00700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.47:6822 osd.6 since back 2024-03-21T10:37:51.491091+0000 front 2024-03-21T10:37:51.491288+0000 (oldest deadline 2024-03-21T10:38:14.390851+0000) 2024-03-21T10:38:29.736 INFO:journalctl@ceph.osd.11.smithi157.stdout:Mar 21 10:38:29 smithi157 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-11[61259]: 2024-03-21T10:38:29.314+0000 7fd0f8400700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-03-21T10:36:54.398353+0000 front 2024-03-21T10:36:54.398068+0000 (oldest deadline 2024-03-21T10:37:16.698280+0000) 2024-03-21T10:38:29.737 INFO:journalctl@ceph.osd.11.smithi157.stdout:Mar 21 10:38:29 smithi157 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-11[61259]: 2024-03-21T10:38:29.314+0000 7fd0f8400700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-03-21T10:37:04.901925+0000 front 2024-03-21T10:37:04.902083+0000 (oldest deadline 2024-03-21T10:37:29.502349+0000) 2024-03-21T10:38:29.737 INFO:journalctl@ceph.osd.11.smithi157.stdout:Mar 21 10:38:29 smithi157 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-11[61259]: 2024-03-21T10:38:29.314+0000 7fd0f8400700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-03-21T10:37:11.804109+0000 front 2024-03-21T10:37:11.804464+0000 (oldest deadline 2024-03-21T10:37:37.104077+0000) 2024-03-21T10:38:29.737 INFO:journalctl@ceph.osd.11.smithi157.stdout:Mar 21 10:38:29 smithi157 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-11[61259]: 2024-03-21T10:38:29.314+0000 7fd0f8400700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.100:6812 osd.4 since back 2024-03-21T10:37:33.508959+0000 front 2024-03-21T10:37:33.509595+0000 (oldest deadline 2024-03-21T10:37:54.609171+0000) 2024-03-21T10:38:29.737 INFO:journalctl@ceph.osd.11.smithi157.stdout:Mar 21 10:38:29 smithi157 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-11[61259]: 2024-03-21T10:38:29.314+0000 7fd0f8400700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.157:6812 osd.5 since back 2024-03-21T10:37:41.512925+0000 front 2024-03-21T10:37:41.513037+0000 (oldest deadline 2024-03-21T10:38:05.013049+0000) 2024-03-21T10:38:29.737 INFO:journalctl@ceph.osd.11.smithi157.stdout:Mar 21 10:38:29 smithi157 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-11[61259]: 2024-03-21T10:38:29.314+0000 7fd0f8400700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.47:6822 osd.6 since back 2024-03-21T10:37:52.015740+0000 front 2024-03-21T10:37:52.015675+0000 (oldest deadline 2024-03-21T10:38:17.315784+0000) 2024-03-21T10:38:29.737 INFO:journalctl@ceph.osd.11.smithi157.stdout:Mar 21 10:38:29 smithi157 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-11[61259]: 2024-03-21T10:38:29.314+0000 7fd0f8400700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.100:6820 osd.7 since back 2024-03-21T10:37:57.316871+0000 front 2024-03-21T10:37:57.316918+0000 (oldest deadline 2024-03-21T10:38:25.517713+0000) 2024-03-21T10:38:30.476 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 10:38:30 smithi100 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-10[59767]: 2024-03-21T10:38:30.191+0000 7f315da00700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-03-21T10:36:52.673512+0000 front 2024-03-21T10:36:52.673688+0000 (oldest deadline 2024-03-21T10:37:16.675039+0000) 2024-03-21T10:38:30.476 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 10:38:30 smithi100 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-10[59767]: 2024-03-21T10:38:30.191+0000 7f315da00700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-03-21T10:37:05.379887+0000 front 2024-03-21T10:37:05.379536+0000 (oldest deadline 2024-03-21T10:37:30.079809+0000) 2024-03-21T10:38:30.477 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 10:38:30 smithi100 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-10[59767]: 2024-03-21T10:38:30.191+0000 7f315da00700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-03-21T10:37:12.981593+0000 front 2024-03-21T10:37:12.981483+0000 (oldest deadline 2024-03-21T10:37:38.281183+0000) 2024-03-21T10:38:30.477 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 10:38:30 smithi100 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-10[59767]: 2024-03-21T10:38:30.191+0000 7f315da00700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.47:6814 osd.3 since back 2024-03-21T10:37:23.583085+0000 front 2024-03-21T10:37:23.583243+0000 (oldest deadline 2024-03-21T10:37:45.283261+0000) 2024-03-21T10:38:30.477 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 10:38:30 smithi100 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-10[59767]: 2024-03-21T10:38:30.191+0000 7f315da00700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.100:6812 osd.4 since back 2024-03-21T10:37:28.185438+0000 front 2024-03-21T10:37:28.185490+0000 (oldest deadline 2024-03-21T10:37:55.786600+0000) 2024-03-21T10:38:30.477 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 10:38:30 smithi100 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-10[59767]: 2024-03-21T10:38:30.191+0000 7f315da00700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.157:6812 osd.5 since back 2024-03-21T10:37:39.888144+0000 front 2024-03-21T10:37:39.888262+0000 (oldest deadline 2024-03-21T10:38:03.888699+0000) 2024-03-21T10:38:30.477 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 10:38:30 smithi100 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-10[59767]: 2024-03-21T10:38:30.191+0000 7f315da00700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.47:6822 osd.6 since back 2024-03-21T10:37:51.491091+0000 front 2024-03-21T10:37:51.491288+0000 (oldest deadline 2024-03-21T10:38:14.390851+0000) 2024-03-21T10:38:30.736 INFO:journalctl@ceph.osd.11.smithi157.stdout:Mar 21 10:38:30 smithi157 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-11[61259]: 2024-03-21T10:38:30.312+0000 7fd0f8400700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-03-21T10:36:54.398353+0000 front 2024-03-21T10:36:54.398068+0000 (oldest deadline 2024-03-21T10:37:16.698280+0000) 2024-03-21T10:38:30.737 INFO:journalctl@ceph.osd.11.smithi157.stdout:Mar 21 10:38:30 smithi157 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-11[61259]: 2024-03-21T10:38:30.312+0000 7fd0f8400700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-03-21T10:37:04.901925+0000 front 2024-03-21T10:37:04.902083+0000 (oldest deadline 2024-03-21T10:37:29.502349+0000) 2024-03-21T10:38:30.737 INFO:journalctl@ceph.osd.11.smithi157.stdout:Mar 21 10:38:30 smithi157 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-11[61259]: 2024-03-21T10:38:30.312+0000 7fd0f8400700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-03-21T10:37:11.804109+0000 front 2024-03-21T10:37:11.804464+0000 (oldest deadline 2024-03-21T10:37:37.104077+0000) 2024-03-21T10:38:30.737 INFO:journalctl@ceph.osd.11.smithi157.stdout:Mar 21 10:38:30 smithi157 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-11[61259]: 2024-03-21T10:38:30.312+0000 7fd0f8400700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.100:6812 osd.4 since back 2024-03-21T10:37:33.508959+0000 front 2024-03-21T10:37:33.509595+0000 (oldest deadline 2024-03-21T10:37:54.609171+0000) 2024-03-21T10:38:30.737 INFO:journalctl@ceph.osd.11.smithi157.stdout:Mar 21 10:38:30 smithi157 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-11[61259]: 2024-03-21T10:38:30.312+0000 7fd0f8400700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.157:6812 osd.5 since back 2024-03-21T10:37:41.512925+0000 front 2024-03-21T10:37:41.513037+0000 (oldest deadline 2024-03-21T10:38:05.013049+0000) 2024-03-21T10:38:30.737 INFO:journalctl@ceph.osd.11.smithi157.stdout:Mar 21 10:38:30 smithi157 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-11[61259]: 2024-03-21T10:38:30.312+0000 7fd0f8400700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.47:6822 osd.6 since back 2024-03-21T10:37:52.015740+0000 front 2024-03-21T10:37:52.015675+0000 (oldest deadline 2024-03-21T10:38:17.315784+0000) 2024-03-21T10:38:30.737 INFO:journalctl@ceph.osd.11.smithi157.stdout:Mar 21 10:38:30 smithi157 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-11[61259]: 2024-03-21T10:38:30.312+0000 7fd0f8400700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.100:6820 osd.7 since back 2024-03-21T10:37:57.316871+0000 front 2024-03-21T10:37:57.316918+0000 (oldest deadline 2024-03-21T10:38:25.517713+0000) 2024-03-21T10:38:31.282 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 10:38:30 smithi100 podman[71746]: 2024-03-21 10:38:30.997769268 +0000 UTC m=+5.756733418 container died d27e4913dd062337785f7bace9b9a50da5d996ea847c06e1c65b1ae492063788 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-10, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240319, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux ) 2024-03-21T10:38:31.560 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 10:38:31 smithi100 podman[71746]: 2024-03-21 10:38:31.387043515 +0000 UTC m=+6.146007625 container cleanup d27e4913dd062337785f7bace9b9a50da5d996ea847c06e1c65b1ae492063788 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-10, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, io.buildah.version=1.33.5, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.build-date=20240319, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-21T10:38:31.561 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 10:38:31 smithi100 bash[71746]: ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-10 2024-03-21T10:38:31.736 INFO:journalctl@ceph.osd.11.smithi157.stdout:Mar 21 10:38:31 smithi157 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-11[61259]: 2024-03-21T10:38:31.361+0000 7fd0f8400700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-03-21T10:36:54.398353+0000 front 2024-03-21T10:36:54.398068+0000 (oldest deadline 2024-03-21T10:37:16.698280+0000) 2024-03-21T10:38:31.737 INFO:journalctl@ceph.osd.11.smithi157.stdout:Mar 21 10:38:31 smithi157 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-11[61259]: 2024-03-21T10:38:31.361+0000 7fd0f8400700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-03-21T10:37:04.901925+0000 front 2024-03-21T10:37:04.902083+0000 (oldest deadline 2024-03-21T10:37:29.502349+0000) 2024-03-21T10:38:31.737 INFO:journalctl@ceph.osd.11.smithi157.stdout:Mar 21 10:38:31 smithi157 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-11[61259]: 2024-03-21T10:38:31.361+0000 7fd0f8400700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-03-21T10:37:11.804109+0000 front 2024-03-21T10:37:11.804464+0000 (oldest deadline 2024-03-21T10:37:37.104077+0000) 2024-03-21T10:38:31.737 INFO:journalctl@ceph.osd.11.smithi157.stdout:Mar 21 10:38:31 smithi157 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-11[61259]: 2024-03-21T10:38:31.361+0000 7fd0f8400700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.100:6812 osd.4 since back 2024-03-21T10:37:33.508959+0000 front 2024-03-21T10:37:33.509595+0000 (oldest deadline 2024-03-21T10:37:54.609171+0000) 2024-03-21T10:38:31.737 INFO:journalctl@ceph.osd.11.smithi157.stdout:Mar 21 10:38:31 smithi157 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-11[61259]: 2024-03-21T10:38:31.361+0000 7fd0f8400700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.157:6812 osd.5 since back 2024-03-21T10:37:41.512925+0000 front 2024-03-21T10:37:41.513037+0000 (oldest deadline 2024-03-21T10:38:05.013049+0000) 2024-03-21T10:38:31.737 INFO:journalctl@ceph.osd.11.smithi157.stdout:Mar 21 10:38:31 smithi157 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-11[61259]: 2024-03-21T10:38:31.361+0000 7fd0f8400700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.47:6822 osd.6 since back 2024-03-21T10:37:52.015740+0000 front 2024-03-21T10:37:52.015675+0000 (oldest deadline 2024-03-21T10:38:17.315784+0000) 2024-03-21T10:38:31.737 INFO:journalctl@ceph.osd.11.smithi157.stdout:Mar 21 10:38:31 smithi157 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-11[61259]: 2024-03-21T10:38:31.361+0000 7fd0f8400700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.100:6820 osd.7 since back 2024-03-21T10:37:57.316871+0000 front 2024-03-21T10:37:57.316918+0000 (oldest deadline 2024-03-21T10:38:25.517713+0000) 2024-03-21T10:38:31.976 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 10:38:31 smithi100 podman[71765]: 2024-03-21 10:38:31.562730122 +0000 UTC m=+0.548350632 container remove d27e4913dd062337785f7bace9b9a50da5d996ea847c06e1c65b1ae492063788 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-10, org.label-schema.vendor=CentOS, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, io.buildah.version=1.33.5, ceph=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.build-date=20240319, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux ) 2024-03-21T10:38:32.640 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 10:38:32 smithi100 podman[71854]: 2024-03-21 10:38:32.268052596 +0000 UTC m=+0.034514749 image pull 237e43c7a65ff48534d94b97c1bfdbe4b7be5672e3d9a613745083ea3702aa54 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232 2024-03-21T10:38:32.640 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 10:38:32 smithi100 podman[71854]: 2024-03-21 10:38:32.371916159 +0000 UTC m=+0.138378349 container create 1496c53ead2307bb80b2c4e40abc3cd7140b6fdd24b17bd8ae822eaf3725dda8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-10-deactivate, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240319, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0) 2024-03-21T10:38:32.640 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 10:38:32 smithi100 podman[71854]: 2024-03-21 10:38:32.642263299 +0000 UTC m=+0.408725496 container init 1496c53ead2307bb80b2c4e40abc3cd7140b6fdd24b17bd8ae822eaf3725dda8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-10-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240319, org.label-schema.vendor=CentOS, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, ceph=True, maintainer=Guillaume Abrioux ) 2024-03-21T10:38:32.736 INFO:journalctl@ceph.osd.11.smithi157.stdout:Mar 21 10:38:32 smithi157 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-11[61259]: 2024-03-21T10:38:32.333+0000 7fd0f8400700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-03-21T10:36:54.398353+0000 front 2024-03-21T10:36:54.398068+0000 (oldest deadline 2024-03-21T10:37:16.698280+0000) 2024-03-21T10:38:32.737 INFO:journalctl@ceph.osd.11.smithi157.stdout:Mar 21 10:38:32 smithi157 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-11[61259]: 2024-03-21T10:38:32.333+0000 7fd0f8400700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-03-21T10:37:04.901925+0000 front 2024-03-21T10:37:04.902083+0000 (oldest deadline 2024-03-21T10:37:29.502349+0000) 2024-03-21T10:38:32.737 INFO:journalctl@ceph.osd.11.smithi157.stdout:Mar 21 10:38:32 smithi157 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-11[61259]: 2024-03-21T10:38:32.333+0000 7fd0f8400700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-03-21T10:37:11.804109+0000 front 2024-03-21T10:37:11.804464+0000 (oldest deadline 2024-03-21T10:37:37.104077+0000) 2024-03-21T10:38:32.737 INFO:journalctl@ceph.osd.11.smithi157.stdout:Mar 21 10:38:32 smithi157 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-11[61259]: 2024-03-21T10:38:32.333+0000 7fd0f8400700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.100:6812 osd.4 since back 2024-03-21T10:37:33.508959+0000 front 2024-03-21T10:37:33.509595+0000 (oldest deadline 2024-03-21T10:37:54.609171+0000) 2024-03-21T10:38:32.737 INFO:journalctl@ceph.osd.11.smithi157.stdout:Mar 21 10:38:32 smithi157 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-11[61259]: 2024-03-21T10:38:32.333+0000 7fd0f8400700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.157:6812 osd.5 since back 2024-03-21T10:37:41.512925+0000 front 2024-03-21T10:37:41.513037+0000 (oldest deadline 2024-03-21T10:38:05.013049+0000) 2024-03-21T10:38:32.737 INFO:journalctl@ceph.osd.11.smithi157.stdout:Mar 21 10:38:32 smithi157 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-11[61259]: 2024-03-21T10:38:32.333+0000 7fd0f8400700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.47:6822 osd.6 since back 2024-03-21T10:37:52.015740+0000 front 2024-03-21T10:37:52.015675+0000 (oldest deadline 2024-03-21T10:38:17.315784+0000) 2024-03-21T10:38:32.737 INFO:journalctl@ceph.osd.11.smithi157.stdout:Mar 21 10:38:32 smithi157 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-11[61259]: 2024-03-21T10:38:32.333+0000 7fd0f8400700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.100:6820 osd.7 since back 2024-03-21T10:37:57.316871+0000 front 2024-03-21T10:37:57.316918+0000 (oldest deadline 2024-03-21T10:38:25.517713+0000) 2024-03-21T10:38:32.976 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 10:38:32 smithi100 podman[71854]: 2024-03-21 10:38:32.653763984 +0000 UTC m=+0.420226204 container start 1496c53ead2307bb80b2c4e40abc3cd7140b6fdd24b17bd8ae822eaf3725dda8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-10-deactivate, org.label-schema.build-date=20240319, org.label-schema.vendor=CentOS, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0) 2024-03-21T10:38:32.976 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 10:38:32 smithi100 podman[71854]: 2024-03-21 10:38:32.740651604 +0000 UTC m=+0.507113777 container attach 1496c53ead2307bb80b2c4e40abc3cd7140b6fdd24b17bd8ae822eaf3725dda8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-10-deactivate, GIT_CLEAN=True, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, org.label-schema.build-date=20240319, ceph=True, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS) 2024-03-21T10:38:33.463 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 10:38:33 smithi100 podman[71854]: 2024-03-21 10:38:33.085106022 +0000 UTC m=+0.851568184 container died 1496c53ead2307bb80b2c4e40abc3cd7140b6fdd24b17bd8ae822eaf3725dda8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-10-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_CLEAN=True, maintainer=Guillaume Abrioux , ceph=True, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.build-date=20240319, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-21T10:38:33.736 INFO:journalctl@ceph.osd.11.smithi157.stdout:Mar 21 10:38:33 smithi157 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-11[61259]: 2024-03-21T10:38:33.365+0000 7fd0f8400700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-03-21T10:36:54.398353+0000 front 2024-03-21T10:36:54.398068+0000 (oldest deadline 2024-03-21T10:37:16.698280+0000) 2024-03-21T10:38:33.737 INFO:journalctl@ceph.osd.11.smithi157.stdout:Mar 21 10:38:33 smithi157 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-11[61259]: 2024-03-21T10:38:33.365+0000 7fd0f8400700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-03-21T10:37:04.901925+0000 front 2024-03-21T10:37:04.902083+0000 (oldest deadline 2024-03-21T10:37:29.502349+0000) 2024-03-21T10:38:33.737 INFO:journalctl@ceph.osd.11.smithi157.stdout:Mar 21 10:38:33 smithi157 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-11[61259]: 2024-03-21T10:38:33.365+0000 7fd0f8400700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-03-21T10:37:11.804109+0000 front 2024-03-21T10:37:11.804464+0000 (oldest deadline 2024-03-21T10:37:37.104077+0000) 2024-03-21T10:38:33.737 INFO:journalctl@ceph.osd.11.smithi157.stdout:Mar 21 10:38:33 smithi157 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-11[61259]: 2024-03-21T10:38:33.365+0000 7fd0f8400700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.100:6812 osd.4 since back 2024-03-21T10:37:33.508959+0000 front 2024-03-21T10:37:33.509595+0000 (oldest deadline 2024-03-21T10:37:54.609171+0000) 2024-03-21T10:38:33.737 INFO:journalctl@ceph.osd.11.smithi157.stdout:Mar 21 10:38:33 smithi157 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-11[61259]: 2024-03-21T10:38:33.365+0000 7fd0f8400700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.157:6812 osd.5 since back 2024-03-21T10:37:41.512925+0000 front 2024-03-21T10:37:41.513037+0000 (oldest deadline 2024-03-21T10:38:05.013049+0000) 2024-03-21T10:38:33.737 INFO:journalctl@ceph.osd.11.smithi157.stdout:Mar 21 10:38:33 smithi157 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-11[61259]: 2024-03-21T10:38:33.365+0000 7fd0f8400700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.47:6822 osd.6 since back 2024-03-21T10:37:52.015740+0000 front 2024-03-21T10:37:52.015675+0000 (oldest deadline 2024-03-21T10:38:17.315784+0000) 2024-03-21T10:38:33.737 INFO:journalctl@ceph.osd.11.smithi157.stdout:Mar 21 10:38:33 smithi157 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-11[61259]: 2024-03-21T10:38:33.365+0000 7fd0f8400700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.100:6820 osd.7 since back 2024-03-21T10:37:57.316871+0000 front 2024-03-21T10:37:57.316918+0000 (oldest deadline 2024-03-21T10:38:25.517713+0000) 2024-03-21T10:38:33.866 DEBUG:teuthology.orchestra.run.smithi100:> sudo pkill -f 'journalctl -f -n 0 -u ceph-3056d51a-e76b-11ee-95cd-87774f69a715@osd.10.service' 2024-03-21T10:38:33.993 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-21T10:38:33.993 INFO:tasks.cephadm.osd.10:Stopped osd.10 2024-03-21T10:38:33.993 INFO:tasks.cephadm.osd.11:Stopping osd.11... 2024-03-21T10:38:33.993 DEBUG:teuthology.orchestra.run.smithi157:> sudo systemctl stop ceph-3056d51a-e76b-11ee-95cd-87774f69a715@osd.11 2024-03-21T10:38:34.337 INFO:journalctl@ceph.osd.11.smithi157.stdout:Mar 21 10:38:34 smithi157 systemd[1]: Stopping Ceph osd.11 for 3056d51a-e76b-11ee-95cd-87774f69a715... 2024-03-21T10:38:34.737 INFO:journalctl@ceph.osd.11.smithi157.stdout:Mar 21 10:38:34 smithi157 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-11[61259]: 2024-03-21T10:38:34.340+0000 7fd0f8400700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-03-21T10:36:54.398353+0000 front 2024-03-21T10:36:54.398068+0000 (oldest deadline 2024-03-21T10:37:16.698280+0000) 2024-03-21T10:38:34.737 INFO:journalctl@ceph.osd.11.smithi157.stdout:Mar 21 10:38:34 smithi157 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-11[61259]: 2024-03-21T10:38:34.340+0000 7fd0f8400700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-03-21T10:37:04.901925+0000 front 2024-03-21T10:37:04.902083+0000 (oldest deadline 2024-03-21T10:37:29.502349+0000) 2024-03-21T10:38:34.737 INFO:journalctl@ceph.osd.11.smithi157.stdout:Mar 21 10:38:34 smithi157 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-11[61259]: 2024-03-21T10:38:34.340+0000 7fd0f8400700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-03-21T10:37:11.804109+0000 front 2024-03-21T10:37:11.804464+0000 (oldest deadline 2024-03-21T10:37:37.104077+0000) 2024-03-21T10:38:34.737 INFO:journalctl@ceph.osd.11.smithi157.stdout:Mar 21 10:38:34 smithi157 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-11[61259]: 2024-03-21T10:38:34.340+0000 7fd0f8400700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.100:6812 osd.4 since back 2024-03-21T10:37:33.508959+0000 front 2024-03-21T10:37:33.509595+0000 (oldest deadline 2024-03-21T10:37:54.609171+0000) 2024-03-21T10:38:34.737 INFO:journalctl@ceph.osd.11.smithi157.stdout:Mar 21 10:38:34 smithi157 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-11[61259]: 2024-03-21T10:38:34.340+0000 7fd0f8400700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.157:6812 osd.5 since back 2024-03-21T10:37:41.512925+0000 front 2024-03-21T10:37:41.513037+0000 (oldest deadline 2024-03-21T10:38:05.013049+0000) 2024-03-21T10:38:34.737 INFO:journalctl@ceph.osd.11.smithi157.stdout:Mar 21 10:38:34 smithi157 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-11[61259]: 2024-03-21T10:38:34.340+0000 7fd0f8400700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.47:6822 osd.6 since back 2024-03-21T10:37:52.015740+0000 front 2024-03-21T10:37:52.015675+0000 (oldest deadline 2024-03-21T10:38:17.315784+0000) 2024-03-21T10:38:34.737 INFO:journalctl@ceph.osd.11.smithi157.stdout:Mar 21 10:38:34 smithi157 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-11[61259]: 2024-03-21T10:38:34.340+0000 7fd0f8400700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.100:6820 osd.7 since back 2024-03-21T10:37:57.316871+0000 front 2024-03-21T10:37:57.316918+0000 (oldest deadline 2024-03-21T10:38:25.517713+0000) 2024-03-21T10:38:34.738 INFO:journalctl@ceph.osd.11.smithi157.stdout:Mar 21 10:38:34 smithi157 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-11[61259]: 2024-03-21T10:38:34.554+0000 7fd101c00700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.11 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-21T10:38:34.738 INFO:journalctl@ceph.osd.11.smithi157.stdout:Mar 21 10:38:34 smithi157 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-11[61259]: 2024-03-21T10:38:34.554+0000 7fd101c00700 -1 osd.11 111 *** Got signal Terminated *** 2024-03-21T10:38:34.738 INFO:journalctl@ceph.osd.11.smithi157.stdout:Mar 21 10:38:34 smithi157 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-11[61259]: 2024-03-21T10:38:34.554+0000 7fd101c00700 -1 osd.11 111 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-21T10:38:35.736 INFO:journalctl@ceph.osd.11.smithi157.stdout:Mar 21 10:38:35 smithi157 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-11[61259]: 2024-03-21T10:38:35.335+0000 7fd0f8400700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-03-21T10:36:54.398353+0000 front 2024-03-21T10:36:54.398068+0000 (oldest deadline 2024-03-21T10:37:16.698280+0000) 2024-03-21T10:38:35.737 INFO:journalctl@ceph.osd.11.smithi157.stdout:Mar 21 10:38:35 smithi157 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-11[61259]: 2024-03-21T10:38:35.335+0000 7fd0f8400700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-03-21T10:37:04.901925+0000 front 2024-03-21T10:37:04.902083+0000 (oldest deadline 2024-03-21T10:37:29.502349+0000) 2024-03-21T10:38:35.737 INFO:journalctl@ceph.osd.11.smithi157.stdout:Mar 21 10:38:35 smithi157 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-11[61259]: 2024-03-21T10:38:35.335+0000 7fd0f8400700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-03-21T10:37:11.804109+0000 front 2024-03-21T10:37:11.804464+0000 (oldest deadline 2024-03-21T10:37:37.104077+0000) 2024-03-21T10:38:35.737 INFO:journalctl@ceph.osd.11.smithi157.stdout:Mar 21 10:38:35 smithi157 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-11[61259]: 2024-03-21T10:38:35.335+0000 7fd0f8400700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.100:6812 osd.4 since back 2024-03-21T10:37:33.508959+0000 front 2024-03-21T10:37:33.509595+0000 (oldest deadline 2024-03-21T10:37:54.609171+0000) 2024-03-21T10:38:35.737 INFO:journalctl@ceph.osd.11.smithi157.stdout:Mar 21 10:38:35 smithi157 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-11[61259]: 2024-03-21T10:38:35.335+0000 7fd0f8400700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.157:6812 osd.5 since back 2024-03-21T10:37:41.512925+0000 front 2024-03-21T10:37:41.513037+0000 (oldest deadline 2024-03-21T10:38:05.013049+0000) 2024-03-21T10:38:35.737 INFO:journalctl@ceph.osd.11.smithi157.stdout:Mar 21 10:38:35 smithi157 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-11[61259]: 2024-03-21T10:38:35.335+0000 7fd0f8400700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.47:6822 osd.6 since back 2024-03-21T10:37:52.015740+0000 front 2024-03-21T10:37:52.015675+0000 (oldest deadline 2024-03-21T10:38:17.315784+0000) 2024-03-21T10:38:35.737 INFO:journalctl@ceph.osd.11.smithi157.stdout:Mar 21 10:38:35 smithi157 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-11[61259]: 2024-03-21T10:38:35.335+0000 7fd0f8400700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.100:6820 osd.7 since back 2024-03-21T10:37:57.316871+0000 front 2024-03-21T10:37:57.316918+0000 (oldest deadline 2024-03-21T10:38:25.517713+0000) 2024-03-21T10:38:36.737 INFO:journalctl@ceph.osd.11.smithi157.stdout:Mar 21 10:38:36 smithi157 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-11[61259]: 2024-03-21T10:38:36.298+0000 7fd0f8400700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-03-21T10:36:54.398353+0000 front 2024-03-21T10:36:54.398068+0000 (oldest deadline 2024-03-21T10:37:16.698280+0000) 2024-03-21T10:38:36.737 INFO:journalctl@ceph.osd.11.smithi157.stdout:Mar 21 10:38:36 smithi157 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-11[61259]: 2024-03-21T10:38:36.298+0000 7fd0f8400700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-03-21T10:37:04.901925+0000 front 2024-03-21T10:37:04.902083+0000 (oldest deadline 2024-03-21T10:37:29.502349+0000) 2024-03-21T10:38:36.737 INFO:journalctl@ceph.osd.11.smithi157.stdout:Mar 21 10:38:36 smithi157 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-11[61259]: 2024-03-21T10:38:36.298+0000 7fd0f8400700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-03-21T10:37:11.804109+0000 front 2024-03-21T10:37:11.804464+0000 (oldest deadline 2024-03-21T10:37:37.104077+0000) 2024-03-21T10:38:36.737 INFO:journalctl@ceph.osd.11.smithi157.stdout:Mar 21 10:38:36 smithi157 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-11[61259]: 2024-03-21T10:38:36.298+0000 7fd0f8400700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.100:6812 osd.4 since back 2024-03-21T10:37:33.508959+0000 front 2024-03-21T10:37:33.509595+0000 (oldest deadline 2024-03-21T10:37:54.609171+0000) 2024-03-21T10:38:36.737 INFO:journalctl@ceph.osd.11.smithi157.stdout:Mar 21 10:38:36 smithi157 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-11[61259]: 2024-03-21T10:38:36.298+0000 7fd0f8400700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.157:6812 osd.5 since back 2024-03-21T10:37:41.512925+0000 front 2024-03-21T10:37:41.513037+0000 (oldest deadline 2024-03-21T10:38:05.013049+0000) 2024-03-21T10:38:36.738 INFO:journalctl@ceph.osd.11.smithi157.stdout:Mar 21 10:38:36 smithi157 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-11[61259]: 2024-03-21T10:38:36.298+0000 7fd0f8400700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.47:6822 osd.6 since back 2024-03-21T10:37:52.015740+0000 front 2024-03-21T10:37:52.015675+0000 (oldest deadline 2024-03-21T10:38:17.315784+0000) 2024-03-21T10:38:36.738 INFO:journalctl@ceph.osd.11.smithi157.stdout:Mar 21 10:38:36 smithi157 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-11[61259]: 2024-03-21T10:38:36.298+0000 7fd0f8400700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.100:6820 osd.7 since back 2024-03-21T10:37:57.316871+0000 front 2024-03-21T10:37:57.316918+0000 (oldest deadline 2024-03-21T10:38:25.517713+0000) 2024-03-21T10:38:36.738 INFO:journalctl@ceph.osd.11.smithi157.stdout:Mar 21 10:38:36 smithi157 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-11[61259]: 2024-03-21T10:38:36.298+0000 7fd0f8400700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.157:6820 osd.8 since back 2024-03-21T10:38:11.320500+0000 front 2024-03-21T10:38:11.320632+0000 (oldest deadline 2024-03-21T10:38:35.420446+0000) 2024-03-21T10:38:37.737 INFO:journalctl@ceph.osd.11.smithi157.stdout:Mar 21 10:38:37 smithi157 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-11[61259]: 2024-03-21T10:38:37.342+0000 7fd0f8400700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-03-21T10:36:54.398353+0000 front 2024-03-21T10:36:54.398068+0000 (oldest deadline 2024-03-21T10:37:16.698280+0000) 2024-03-21T10:38:37.737 INFO:journalctl@ceph.osd.11.smithi157.stdout:Mar 21 10:38:37 smithi157 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-11[61259]: 2024-03-21T10:38:37.342+0000 7fd0f8400700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-03-21T10:37:04.901925+0000 front 2024-03-21T10:37:04.902083+0000 (oldest deadline 2024-03-21T10:37:29.502349+0000) 2024-03-21T10:38:37.737 INFO:journalctl@ceph.osd.11.smithi157.stdout:Mar 21 10:38:37 smithi157 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-11[61259]: 2024-03-21T10:38:37.342+0000 7fd0f8400700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-03-21T10:37:11.804109+0000 front 2024-03-21T10:37:11.804464+0000 (oldest deadline 2024-03-21T10:37:37.104077+0000) 2024-03-21T10:38:37.737 INFO:journalctl@ceph.osd.11.smithi157.stdout:Mar 21 10:38:37 smithi157 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-11[61259]: 2024-03-21T10:38:37.342+0000 7fd0f8400700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.100:6812 osd.4 since back 2024-03-21T10:37:33.508959+0000 front 2024-03-21T10:37:33.509595+0000 (oldest deadline 2024-03-21T10:37:54.609171+0000) 2024-03-21T10:38:37.737 INFO:journalctl@ceph.osd.11.smithi157.stdout:Mar 21 10:38:37 smithi157 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-11[61259]: 2024-03-21T10:38:37.342+0000 7fd0f8400700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.157:6812 osd.5 since back 2024-03-21T10:37:41.512925+0000 front 2024-03-21T10:37:41.513037+0000 (oldest deadline 2024-03-21T10:38:05.013049+0000) 2024-03-21T10:38:37.737 INFO:journalctl@ceph.osd.11.smithi157.stdout:Mar 21 10:38:37 smithi157 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-11[61259]: 2024-03-21T10:38:37.342+0000 7fd0f8400700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.47:6822 osd.6 since back 2024-03-21T10:37:52.015740+0000 front 2024-03-21T10:37:52.015675+0000 (oldest deadline 2024-03-21T10:38:17.315784+0000) 2024-03-21T10:38:37.737 INFO:journalctl@ceph.osd.11.smithi157.stdout:Mar 21 10:38:37 smithi157 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-11[61259]: 2024-03-21T10:38:37.342+0000 7fd0f8400700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.100:6820 osd.7 since back 2024-03-21T10:37:57.316871+0000 front 2024-03-21T10:37:57.316918+0000 (oldest deadline 2024-03-21T10:38:25.517713+0000) 2024-03-21T10:38:37.737 INFO:journalctl@ceph.osd.11.smithi157.stdout:Mar 21 10:38:37 smithi157 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-11[61259]: 2024-03-21T10:38:37.342+0000 7fd0f8400700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.157:6820 osd.8 since back 2024-03-21T10:38:11.320500+0000 front 2024-03-21T10:38:11.320632+0000 (oldest deadline 2024-03-21T10:38:35.420446+0000) 2024-03-21T10:38:38.737 INFO:journalctl@ceph.osd.11.smithi157.stdout:Mar 21 10:38:38 smithi157 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-11[61259]: 2024-03-21T10:38:38.386+0000 7fd0f8400700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-03-21T10:36:54.398353+0000 front 2024-03-21T10:36:54.398068+0000 (oldest deadline 2024-03-21T10:37:16.698280+0000) 2024-03-21T10:38:38.737 INFO:journalctl@ceph.osd.11.smithi157.stdout:Mar 21 10:38:38 smithi157 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-11[61259]: 2024-03-21T10:38:38.386+0000 7fd0f8400700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-03-21T10:37:04.901925+0000 front 2024-03-21T10:37:04.902083+0000 (oldest deadline 2024-03-21T10:37:29.502349+0000) 2024-03-21T10:38:38.737 INFO:journalctl@ceph.osd.11.smithi157.stdout:Mar 21 10:38:38 smithi157 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-11[61259]: 2024-03-21T10:38:38.387+0000 7fd0f8400700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-03-21T10:37:11.804109+0000 front 2024-03-21T10:37:11.804464+0000 (oldest deadline 2024-03-21T10:37:37.104077+0000) 2024-03-21T10:38:38.737 INFO:journalctl@ceph.osd.11.smithi157.stdout:Mar 21 10:38:38 smithi157 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-11[61259]: 2024-03-21T10:38:38.387+0000 7fd0f8400700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.100:6812 osd.4 since back 2024-03-21T10:37:33.508959+0000 front 2024-03-21T10:37:33.509595+0000 (oldest deadline 2024-03-21T10:37:54.609171+0000) 2024-03-21T10:38:38.737 INFO:journalctl@ceph.osd.11.smithi157.stdout:Mar 21 10:38:38 smithi157 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-11[61259]: 2024-03-21T10:38:38.387+0000 7fd0f8400700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.157:6812 osd.5 since back 2024-03-21T10:37:41.512925+0000 front 2024-03-21T10:37:41.513037+0000 (oldest deadline 2024-03-21T10:38:05.013049+0000) 2024-03-21T10:38:38.737 INFO:journalctl@ceph.osd.11.smithi157.stdout:Mar 21 10:38:38 smithi157 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-11[61259]: 2024-03-21T10:38:38.387+0000 7fd0f8400700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.47:6822 osd.6 since back 2024-03-21T10:37:52.015740+0000 front 2024-03-21T10:37:52.015675+0000 (oldest deadline 2024-03-21T10:38:17.315784+0000) 2024-03-21T10:38:38.737 INFO:journalctl@ceph.osd.11.smithi157.stdout:Mar 21 10:38:38 smithi157 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-11[61259]: 2024-03-21T10:38:38.387+0000 7fd0f8400700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.100:6820 osd.7 since back 2024-03-21T10:37:57.316871+0000 front 2024-03-21T10:37:57.316918+0000 (oldest deadline 2024-03-21T10:38:25.517713+0000) 2024-03-21T10:38:38.737 INFO:journalctl@ceph.osd.11.smithi157.stdout:Mar 21 10:38:38 smithi157 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-11[61259]: 2024-03-21T10:38:38.387+0000 7fd0f8400700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.157:6820 osd.8 since back 2024-03-21T10:38:11.320500+0000 front 2024-03-21T10:38:11.320632+0000 (oldest deadline 2024-03-21T10:38:35.420446+0000) 2024-03-21T10:38:39.737 INFO:journalctl@ceph.osd.11.smithi157.stdout:Mar 21 10:38:39 smithi157 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-11[61259]: 2024-03-21T10:38:39.379+0000 7fd0f8400700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-03-21T10:36:54.398353+0000 front 2024-03-21T10:36:54.398068+0000 (oldest deadline 2024-03-21T10:37:16.698280+0000) 2024-03-21T10:38:39.737 INFO:journalctl@ceph.osd.11.smithi157.stdout:Mar 21 10:38:39 smithi157 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-11[61259]: 2024-03-21T10:38:39.379+0000 7fd0f8400700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-03-21T10:37:04.901925+0000 front 2024-03-21T10:37:04.902083+0000 (oldest deadline 2024-03-21T10:37:29.502349+0000) 2024-03-21T10:38:39.737 INFO:journalctl@ceph.osd.11.smithi157.stdout:Mar 21 10:38:39 smithi157 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-11[61259]: 2024-03-21T10:38:39.379+0000 7fd0f8400700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.157:6804 osd.2 since back 2024-03-21T10:37:11.804109+0000 front 2024-03-21T10:37:11.804464+0000 (oldest deadline 2024-03-21T10:37:37.104077+0000) 2024-03-21T10:38:39.737 INFO:journalctl@ceph.osd.11.smithi157.stdout:Mar 21 10:38:39 smithi157 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-11[61259]: 2024-03-21T10:38:39.379+0000 7fd0f8400700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.100:6812 osd.4 since back 2024-03-21T10:37:33.508959+0000 front 2024-03-21T10:37:33.509595+0000 (oldest deadline 2024-03-21T10:37:54.609171+0000) 2024-03-21T10:38:39.737 INFO:journalctl@ceph.osd.11.smithi157.stdout:Mar 21 10:38:39 smithi157 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-11[61259]: 2024-03-21T10:38:39.379+0000 7fd0f8400700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.157:6812 osd.5 since back 2024-03-21T10:37:41.512925+0000 front 2024-03-21T10:37:41.513037+0000 (oldest deadline 2024-03-21T10:38:05.013049+0000) 2024-03-21T10:38:39.737 INFO:journalctl@ceph.osd.11.smithi157.stdout:Mar 21 10:38:39 smithi157 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-11[61259]: 2024-03-21T10:38:39.380+0000 7fd0f8400700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.47:6822 osd.6 since back 2024-03-21T10:37:52.015740+0000 front 2024-03-21T10:37:52.015675+0000 (oldest deadline 2024-03-21T10:38:17.315784+0000) 2024-03-21T10:38:39.737 INFO:journalctl@ceph.osd.11.smithi157.stdout:Mar 21 10:38:39 smithi157 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-11[61259]: 2024-03-21T10:38:39.380+0000 7fd0f8400700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.100:6820 osd.7 since back 2024-03-21T10:37:57.316871+0000 front 2024-03-21T10:37:57.316918+0000 (oldest deadline 2024-03-21T10:38:25.517713+0000) 2024-03-21T10:38:39.737 INFO:journalctl@ceph.osd.11.smithi157.stdout:Mar 21 10:38:39 smithi157 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-11[61259]: 2024-03-21T10:38:39.380+0000 7fd0f8400700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.157:6820 osd.8 since back 2024-03-21T10:38:11.320500+0000 front 2024-03-21T10:38:11.320632+0000 (oldest deadline 2024-03-21T10:38:35.420446+0000) 2024-03-21T10:38:40.486 INFO:journalctl@ceph.osd.11.smithi157.stdout:Mar 21 10:38:40 smithi157 podman[73781]: 2024-03-21 10:38:40.226895083 +0000 UTC m=+5.800803036 container died 32f5dd1c5653369157da75a9ae26d9892db548d6b3205a939ed3f89920ff44eb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-11, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.build-date=20240319, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, ceph=True, GIT_CLEAN=True, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6) 2024-03-21T10:38:40.954 INFO:journalctl@ceph.osd.11.smithi157.stdout:Mar 21 10:38:40 smithi157 podman[73781]: 2024-03-21 10:38:40.622057867 +0000 UTC m=+6.195965819 container cleanup 32f5dd1c5653369157da75a9ae26d9892db548d6b3205a939ed3f89920ff44eb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-11, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.build-date=20240319, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True, io.buildah.version=1.33.5, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-21T10:38:40.954 INFO:journalctl@ceph.osd.11.smithi157.stdout:Mar 21 10:38:40 smithi157 bash[73781]: ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-11 2024-03-21T10:38:41.236 INFO:journalctl@ceph.osd.11.smithi157.stdout:Mar 21 10:38:40 smithi157 podman[73801]: 2024-03-21 10:38:40.956464731 +0000 UTC m=+0.712085075 container remove 32f5dd1c5653369157da75a9ae26d9892db548d6b3205a939ed3f89920ff44eb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-11, CEPH_POINT_RELEASE=, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240319, org.label-schema.license=GPLv2, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , ceph=True) 2024-03-21T10:38:41.817 INFO:journalctl@ceph.osd.11.smithi157.stdout:Mar 21 10:38:41 smithi157 podman[73889]: 2024-03-21 10:38:41.463997955 +0000 UTC m=+0.040824043 image pull 237e43c7a65ff48534d94b97c1bfdbe4b7be5672e3d9a613745083ea3702aa54 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232 2024-03-21T10:38:41.817 INFO:journalctl@ceph.osd.11.smithi157.stdout:Mar 21 10:38:41 smithi157 podman[73889]: 2024-03-21 10:38:41.573836826 +0000 UTC m=+0.150662925 container create f60260f73c9416829aa2a2e594b7e7cba8a45340c3abdf11095a201308d1c48d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-11-deactivate, CEPH_POINT_RELEASE=, GIT_CLEAN=True, maintainer=Guillaume Abrioux , ceph=True, GIT_BRANCH=HEAD, org.label-schema.build-date=20240319, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.license=GPLv2) 2024-03-21T10:38:41.817 INFO:journalctl@ceph.osd.11.smithi157.stdout:Mar 21 10:38:41 smithi157 podman[73889]: 2024-03-21 10:38:41.819229749 +0000 UTC m=+0.396055852 container init f60260f73c9416829aa2a2e594b7e7cba8a45340c3abdf11095a201308d1c48d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-11-deactivate, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240319, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, org.label-schema.license=GPLv2, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD) 2024-03-21T10:38:42.237 INFO:journalctl@ceph.osd.11.smithi157.stdout:Mar 21 10:38:41 smithi157 podman[73889]: 2024-03-21 10:38:41.831959593 +0000 UTC m=+0.408785706 container start f60260f73c9416829aa2a2e594b7e7cba8a45340c3abdf11095a201308d1c48d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-11-deactivate, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, org.label-schema.schema-version=1.0, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_CLEAN=True, ceph=True, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.build-date=20240319) 2024-03-21T10:38:42.237 INFO:journalctl@ceph.osd.11.smithi157.stdout:Mar 21 10:38:41 smithi157 podman[73889]: 2024-03-21 10:38:41.962935332 +0000 UTC m=+0.539761458 container attach f60260f73c9416829aa2a2e594b7e7cba8a45340c3abdf11095a201308d1c48d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-11-deactivate, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240319, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0) 2024-03-21T10:38:42.559 INFO:journalctl@ceph.osd.11.smithi157.stdout:Mar 21 10:38:42 smithi157 podman[73889]: 2024-03-21 10:38:42.260535039 +0000 UTC m=+0.837361142 container died f60260f73c9416829aa2a2e594b7e7cba8a45340c3abdf11095a201308d1c48d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-3056d51a-e76b-11ee-95cd-87774f69a715-osd-11-deactivate, org.label-schema.schema-version=1.0, GIT_CLEAN=True, io.buildah.version=1.33.5, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240319) 2024-03-21T10:38:42.877 DEBUG:teuthology.orchestra.run.smithi157:> sudo pkill -f 'journalctl -f -n 0 -u ceph-3056d51a-e76b-11ee-95cd-87774f69a715@osd.11.service' 2024-03-21T10:38:42.997 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-21T10:38:42.997 INFO:tasks.cephadm.osd.11:Stopped osd.11 2024-03-21T10:38:42.997 INFO:tasks.cephadm.mds.a:Stopping mds.a... 2024-03-21T10:38:42.997 DEBUG:teuthology.orchestra.run.smithi047:> sudo systemctl stop ceph-3056d51a-e76b-11ee-95cd-87774f69a715@mds.a 2024-03-21T10:38:43.460 INFO:journalctl@ceph.mds.a.smithi047.stdout:Mar 21 10:38:43 smithi047 systemd[1]: Stopping Ceph mds.a for 3056d51a-e76b-11ee-95cd-87774f69a715... 2024-03-21T10:38:43.822 INFO:journalctl@ceph.mds.a.smithi047.stdout:Mar 21 10:38:43 smithi047 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-mds-a[71185]: 2024-03-21T10:38:43.547+0000 7f7af9e00700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-21T10:38:43.823 INFO:journalctl@ceph.mds.a.smithi047.stdout:Mar 21 10:38:43 smithi047 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-mds-a[71185]: 2024-03-21T10:38:43.547+0000 7f7af9e00700 -1 mds.a *** got signal Terminated *** 2024-03-21T10:38:46.143 INFO:journalctl@ceph.mds.a.smithi047.stdout:Mar 21 10:38:45 smithi047 podman[103924]: 2024-03-21 10:38:45.859805537 +0000 UTC m=+2.462420975 container died 4ecabcd2ea3456885d498ce3e1b521700a3d4ad37ce5f34b4ed1b814f05dc46c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-3056d51a-e76b-11ee-95cd-87774f69a715-mds-a, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.build-date=20240319, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, io.buildah.version=1.33.5, ceph=True, org.label-schema.license=GPLv2) 2024-03-21T10:38:46.460 INFO:journalctl@ceph.mds.a.smithi047.stdout:Mar 21 10:38:46 smithi047 podman[103924]: 2024-03-21 10:38:46.365326263 +0000 UTC m=+2.967941698 container cleanup 4ecabcd2ea3456885d498ce3e1b521700a3d4ad37ce5f34b4ed1b814f05dc46c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-3056d51a-e76b-11ee-95cd-87774f69a715-mds-a, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.build-date=20240319) 2024-03-21T10:38:46.460 INFO:journalctl@ceph.mds.a.smithi047.stdout:Mar 21 10:38:46 smithi047 bash[103924]: ceph-3056d51a-e76b-11ee-95cd-87774f69a715-mds-a 2024-03-21T10:38:46.923 INFO:journalctl@ceph.mds.a.smithi047.stdout:Mar 21 10:38:46 smithi047 podman[103950]: 2024-03-21 10:38:46.674797844 +0000 UTC m=+0.803652892 container remove 4ecabcd2ea3456885d498ce3e1b521700a3d4ad37ce5f34b4ed1b814f05dc46c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-3056d51a-e76b-11ee-95cd-87774f69a715-mds-a, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.build-date=20240319, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux ) 2024-03-21T10:38:46.949 DEBUG:teuthology.orchestra.run.smithi047:> sudo pkill -f 'journalctl -f -n 0 -u ceph-3056d51a-e76b-11ee-95cd-87774f69a715@mds.a.service' 2024-03-21T10:38:47.071 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-21T10:38:47.072 INFO:tasks.cephadm.mds.a:Stopped mds.a 2024-03-21T10:38:47.072 INFO:tasks.cephadm.mds.d:Stopping mds.d... 2024-03-21T10:38:47.072 DEBUG:teuthology.orchestra.run.smithi047:> sudo systemctl stop ceph-3056d51a-e76b-11ee-95cd-87774f69a715@mds.d 2024-03-21T10:38:47.210 INFO:journalctl@ceph.mds.d.smithi047.stdout:Mar 21 10:38:47 smithi047 systemd[1]: Stopping Ceph mds.d for 3056d51a-e76b-11ee-95cd-87774f69a715... 2024-03-21T10:38:47.960 INFO:journalctl@ceph.mds.d.smithi047.stdout:Mar 21 10:38:47 smithi047 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-mds-d[70102]: 2024-03-21T10:38:47.617+0000 7f5003200700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.d -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-21T10:38:47.960 INFO:journalctl@ceph.mds.d.smithi047.stdout:Mar 21 10:38:47 smithi047 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-mds-d[70102]: 2024-03-21T10:38:47.617+0000 7f5003200700 -1 mds.d *** got signal Terminated *** 2024-03-21T10:38:49.887 INFO:journalctl@ceph.mds.d.smithi047.stdout:Mar 21 10:38:49 smithi047 podman[104054]: 2024-03-21 10:38:49.620933468 +0000 UTC m=+2.128942635 container died 4ae97ca5479a46279de4f2018d9df8327e903391c120cbc5d5fbb89dc48ae6e8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-3056d51a-e76b-11ee-95cd-87774f69a715-mds-d, maintainer=Guillaume Abrioux , GIT_CLEAN=True, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.build-date=20240319, GIT_BRANCH=HEAD, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30) 2024-03-21T10:38:50.210 INFO:journalctl@ceph.mds.d.smithi047.stdout:Mar 21 10:38:49 smithi047 podman[104054]: 2024-03-21 10:38:49.993857399 +0000 UTC m=+2.501866603 container cleanup 4ae97ca5479a46279de4f2018d9df8327e903391c120cbc5d5fbb89dc48ae6e8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-3056d51a-e76b-11ee-95cd-87774f69a715-mds-d, io.buildah.version=1.33.5, org.label-schema.build-date=20240319, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True) 2024-03-21T10:38:50.210 INFO:journalctl@ceph.mds.d.smithi047.stdout:Mar 21 10:38:49 smithi047 bash[104054]: ceph-3056d51a-e76b-11ee-95cd-87774f69a715-mds-d 2024-03-21T10:38:50.697 INFO:journalctl@ceph.mds.d.smithi047.stdout:Mar 21 10:38:50 smithi047 podman[104068]: 2024-03-21 10:38:50.262249051 +0000 UTC m=+0.625990436 container remove 4ae97ca5479a46279de4f2018d9df8327e903391c120cbc5d5fbb89dc48ae6e8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-3056d51a-e76b-11ee-95cd-87774f69a715-mds-d, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, ceph=True, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_CLEAN=True, org.label-schema.build-date=20240319, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6) 2024-03-21T10:38:50.728 DEBUG:teuthology.orchestra.run.smithi047:> sudo pkill -f 'journalctl -f -n 0 -u ceph-3056d51a-e76b-11ee-95cd-87774f69a715@mds.d.service' 2024-03-21T10:38:50.853 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-21T10:38:50.853 INFO:tasks.cephadm.mds.d:Stopped mds.d 2024-03-21T10:38:50.853 INFO:tasks.cephadm.mds.g:Stopping mds.g... 2024-03-21T10:38:50.854 DEBUG:teuthology.orchestra.run.smithi047:> sudo systemctl stop ceph-3056d51a-e76b-11ee-95cd-87774f69a715@mds.g 2024-03-21T10:38:51.419 INFO:journalctl@ceph.mds.g.smithi047.stdout:Mar 21 10:38:50 smithi047 systemd[1]: Stopping Ceph mds.g for 3056d51a-e76b-11ee-95cd-87774f69a715... 2024-03-21T10:38:51.710 INFO:journalctl@ceph.mds.g.smithi047.stdout:Mar 21 10:38:51 smithi047 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-mds-g[70784]: 2024-03-21T10:38:51.421+0000 7f165dc00700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.g -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-21T10:38:51.710 INFO:journalctl@ceph.mds.g.smithi047.stdout:Mar 21 10:38:51 smithi047 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-mds-g[70784]: 2024-03-21T10:38:51.421+0000 7f165dc00700 -1 mds.g *** got signal Terminated *** 2024-03-21T10:38:54.183 INFO:journalctl@ceph.mds.g.smithi047.stdout:Mar 21 10:38:53 smithi047 podman[104173]: 2024-03-21 10:38:53.877811189 +0000 UTC m=+2.585785916 container died b018904203b9a1d608f6ca4316601ca6e5a1896b34b908959c9eebd0f12f0dcc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-3056d51a-e76b-11ee-95cd-87774f69a715-mds-g, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.build-date=20240319, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True, GIT_CLEAN=True, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6) 2024-03-21T10:38:54.460 INFO:journalctl@ceph.mds.g.smithi047.stdout:Mar 21 10:38:54 smithi047 podman[104173]: 2024-03-21 10:38:54.306473034 +0000 UTC m=+3.014447773 container cleanup b018904203b9a1d608f6ca4316601ca6e5a1896b34b908959c9eebd0f12f0dcc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-3056d51a-e76b-11ee-95cd-87774f69a715-mds-g, ceph=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, org.label-schema.build-date=20240319) 2024-03-21T10:38:54.460 INFO:journalctl@ceph.mds.g.smithi047.stdout:Mar 21 10:38:54 smithi047 bash[104173]: ceph-3056d51a-e76b-11ee-95cd-87774f69a715-mds-g 2024-03-21T10:38:54.960 INFO:journalctl@ceph.mds.g.smithi047.stdout:Mar 21 10:38:54 smithi047 podman[104187]: 2024-03-21 10:38:54.582701624 +0000 UTC m=+0.686677324 container remove b018904203b9a1d608f6ca4316601ca6e5a1896b34b908959c9eebd0f12f0dcc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-3056d51a-e76b-11ee-95cd-87774f69a715-mds-g, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, ceph=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.build-date=20240319, maintainer=Guillaume Abrioux , GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.vendor=CentOS) 2024-03-21T10:38:55.132 DEBUG:teuthology.orchestra.run.smithi047:> sudo pkill -f 'journalctl -f -n 0 -u ceph-3056d51a-e76b-11ee-95cd-87774f69a715@mds.g.service' 2024-03-21T10:38:55.245 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-21T10:38:55.245 INFO:tasks.cephadm.mds.g:Stopped mds.g 2024-03-21T10:38:55.245 INFO:tasks.cephadm.mds.j:Stopping mds.j... 2024-03-21T10:38:55.245 DEBUG:teuthology.orchestra.run.smithi047:> sudo systemctl stop ceph-3056d51a-e76b-11ee-95cd-87774f69a715@mds.j 2024-03-21T10:38:55.710 INFO:journalctl@ceph.mds.j.smithi047.stdout:Mar 21 10:38:55 smithi047 systemd[1]: Stopping Ceph mds.j for 3056d51a-e76b-11ee-95cd-87774f69a715... 2024-03-21T10:38:56.210 INFO:journalctl@ceph.mds.j.smithi047.stdout:Mar 21 10:38:55 smithi047 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-mds-j[69717]: 2024-03-21T10:38:55.842+0000 7fa342000700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.j -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-21T10:38:56.210 INFO:journalctl@ceph.mds.j.smithi047.stdout:Mar 21 10:38:55 smithi047 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-mds-j[69717]: 2024-03-21T10:38:55.842+0000 7fa342000700 -1 mds.j *** got signal Terminated *** 2024-03-21T10:38:57.705 INFO:journalctl@ceph.mds.j.smithi047.stdout:Mar 21 10:38:57 smithi047 podman[104293]: 2024-03-21 10:38:57.457286232 +0000 UTC m=+1.731613085 container died 2488f82e2f505c307e52eb483a7879dd4fe69698767cb9fd3c2f7479ff5b920d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-3056d51a-e76b-11ee-95cd-87774f69a715-mds-j, GIT_CLEAN=True, ceph=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, org.label-schema.build-date=20240319, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD) 2024-03-21T10:38:57.960 INFO:journalctl@ceph.mds.j.smithi047.stdout:Mar 21 10:38:57 smithi047 podman[104293]: 2024-03-21 10:38:57.810445417 +0000 UTC m=+2.084772314 container cleanup 2488f82e2f505c307e52eb483a7879dd4fe69698767cb9fd3c2f7479ff5b920d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-3056d51a-e76b-11ee-95cd-87774f69a715-mds-j, GIT_CLEAN=True, org.label-schema.build-date=20240319, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0) 2024-03-21T10:38:57.960 INFO:journalctl@ceph.mds.j.smithi047.stdout:Mar 21 10:38:57 smithi047 bash[104293]: ceph-3056d51a-e76b-11ee-95cd-87774f69a715-mds-j 2024-03-21T10:38:58.460 INFO:journalctl@ceph.mds.j.smithi047.stdout:Mar 21 10:38:58 smithi047 podman[104307]: 2024-03-21 10:38:58.069892113 +0000 UTC m=+0.591855423 container remove 2488f82e2f505c307e52eb483a7879dd4fe69698767cb9fd3c2f7479ff5b920d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-3056d51a-e76b-11ee-95cd-87774f69a715-mds-j, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_BRANCH=HEAD, ceph=True, org.label-schema.build-date=20240319, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0) 2024-03-21T10:38:58.603 DEBUG:teuthology.orchestra.run.smithi047:> sudo pkill -f 'journalctl -f -n 0 -u ceph-3056d51a-e76b-11ee-95cd-87774f69a715@mds.j.service' 2024-03-21T10:38:58.716 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-21T10:38:58.716 INFO:tasks.cephadm.mds.j:Stopped mds.j 2024-03-21T10:38:58.716 INFO:tasks.cephadm.mds.b:Stopping mds.b... 2024-03-21T10:38:58.716 DEBUG:teuthology.orchestra.run.smithi100:> sudo systemctl stop ceph-3056d51a-e76b-11ee-95cd-87774f69a715@mds.b 2024-03-21T10:38:59.226 INFO:journalctl@ceph.mds.b.smithi100.stdout:Mar 21 10:38:58 smithi100 systemd[1]: Stopping Ceph mds.b for 3056d51a-e76b-11ee-95cd-87774f69a715... 2024-03-21T10:38:59.578 INFO:journalctl@ceph.mds.b.smithi100.stdout:Mar 21 10:38:59 smithi100 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-mds-b[64996]: 2024-03-21T10:38:59.333+0000 7fe44a600700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-21T10:38:59.578 INFO:journalctl@ceph.mds.b.smithi100.stdout:Mar 21 10:38:59 smithi100 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-mds-b[64996]: 2024-03-21T10:38:59.333+0000 7fe44a600700 -1 mds.b *** got signal Terminated *** 2024-03-21T10:39:02.204 INFO:journalctl@ceph.mds.b.smithi100.stdout:Mar 21 10:39:01 smithi100 podman[71958]: 2024-03-21 10:39:01.880961122 +0000 UTC m=+2.703638150 container died 030eeb395aa27d3eea1e0e1c2836227117026c1daa13ad4cea4a6af0a1076b62 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-3056d51a-e76b-11ee-95cd-87774f69a715-mds-b, ceph=True, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240319, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS) 2024-03-21T10:39:02.476 INFO:journalctl@ceph.mds.b.smithi100.stdout:Mar 21 10:39:02 smithi100 podman[71958]: 2024-03-21 10:39:02.326990399 +0000 UTC m=+3.149667424 container cleanup 030eeb395aa27d3eea1e0e1c2836227117026c1daa13ad4cea4a6af0a1076b62 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-3056d51a-e76b-11ee-95cd-87774f69a715-mds-b, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, ceph=True, CEPH_POINT_RELEASE=, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.build-date=20240319, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True) 2024-03-21T10:39:02.476 INFO:journalctl@ceph.mds.b.smithi100.stdout:Mar 21 10:39:02 smithi100 bash[71958]: ceph-3056d51a-e76b-11ee-95cd-87774f69a715-mds-b 2024-03-21T10:39:02.976 INFO:journalctl@ceph.mds.b.smithi100.stdout:Mar 21 10:39:02 smithi100 podman[71984]: 2024-03-21 10:39:02.578235123 +0000 UTC m=+0.676159884 container remove 030eeb395aa27d3eea1e0e1c2836227117026c1daa13ad4cea4a6af0a1076b62 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-3056d51a-e76b-11ee-95cd-87774f69a715-mds-b, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240319, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.vendor=CentOS) 2024-03-21T10:39:03.042 DEBUG:teuthology.orchestra.run.smithi100:> sudo pkill -f 'journalctl -f -n 0 -u ceph-3056d51a-e76b-11ee-95cd-87774f69a715@mds.b.service' 2024-03-21T10:39:03.163 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-21T10:39:03.164 INFO:tasks.cephadm.mds.b:Stopped mds.b 2024-03-21T10:39:03.164 INFO:tasks.cephadm.mds.e:Stopping mds.e... 2024-03-21T10:39:03.164 DEBUG:teuthology.orchestra.run.smithi100:> sudo systemctl stop ceph-3056d51a-e76b-11ee-95cd-87774f69a715@mds.e 2024-03-21T10:39:03.476 INFO:journalctl@ceph.mds.e.smithi100.stdout:Mar 21 10:39:03 smithi100 systemd[1]: Stopping Ceph mds.e for 3056d51a-e76b-11ee-95cd-87774f69a715... 2024-03-21T10:39:04.226 INFO:journalctl@ceph.mds.e.smithi100.stdout:Mar 21 10:39:03 smithi100 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-mds-e[64379]: 2024-03-21T10:39:03.729+0000 7fa432800700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.e -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-21T10:39:04.226 INFO:journalctl@ceph.mds.e.smithi100.stdout:Mar 21 10:39:03 smithi100 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-mds-e[64379]: 2024-03-21T10:39:03.729+0000 7fa432800700 -1 mds.e *** got signal Terminated *** 2024-03-21T10:39:06.175 INFO:journalctl@ceph.mds.e.smithi100.stdout:Mar 21 10:39:05 smithi100 podman[72089]: 2024-03-21 10:39:05.894385671 +0000 UTC m=+2.295907124 container died bce3049fc80580804e3e6d25b54faf5cb48c904d63c538381b47310e4d1ff714 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-3056d51a-e76b-11ee-95cd-87774f69a715-mds-e, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.build-date=20240319, CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30) 2024-03-21T10:39:06.476 INFO:journalctl@ceph.mds.e.smithi100.stdout:Mar 21 10:39:06 smithi100 podman[72089]: 2024-03-21 10:39:06.289436187 +0000 UTC m=+2.690957637 container cleanup bce3049fc80580804e3e6d25b54faf5cb48c904d63c538381b47310e4d1ff714 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-3056d51a-e76b-11ee-95cd-87774f69a715-mds-e, ceph=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240319, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0) 2024-03-21T10:39:06.476 INFO:journalctl@ceph.mds.e.smithi100.stdout:Mar 21 10:39:06 smithi100 bash[72089]: ceph-3056d51a-e76b-11ee-95cd-87774f69a715-mds-e 2024-03-21T10:39:06.976 INFO:journalctl@ceph.mds.e.smithi100.stdout:Mar 21 10:39:06 smithi100 podman[72102]: 2024-03-21 10:39:06.498676343 +0000 UTC m=+0.592623587 container remove bce3049fc80580804e3e6d25b54faf5cb48c904d63c538381b47310e4d1ff714 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-3056d51a-e76b-11ee-95cd-87774f69a715-mds-e, ceph=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.build-date=20240319, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-21T10:39:07.035 DEBUG:teuthology.orchestra.run.smithi100:> sudo pkill -f 'journalctl -f -n 0 -u ceph-3056d51a-e76b-11ee-95cd-87774f69a715@mds.e.service' 2024-03-21T10:39:07.154 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-21T10:39:07.154 INFO:tasks.cephadm.mds.e:Stopped mds.e 2024-03-21T10:39:07.154 INFO:tasks.cephadm.mds.h:Stopping mds.h... 2024-03-21T10:39:07.154 DEBUG:teuthology.orchestra.run.smithi100:> sudo systemctl stop ceph-3056d51a-e76b-11ee-95cd-87774f69a715@mds.h 2024-03-21T10:39:07.726 INFO:journalctl@ceph.mds.h.smithi100.stdout:Mar 21 10:39:07 smithi100 systemd[1]: Stopping Ceph mds.h for 3056d51a-e76b-11ee-95cd-87774f69a715... 2024-03-21T10:39:08.226 INFO:journalctl@ceph.mds.h.smithi100.stdout:Mar 21 10:39:07 smithi100 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-mds-h[64682]: 2024-03-21T10:39:07.765+0000 7f462de00700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.h -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-21T10:39:08.226 INFO:journalctl@ceph.mds.h.smithi100.stdout:Mar 21 10:39:07 smithi100 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-mds-h[64682]: 2024-03-21T10:39:07.765+0000 7f462de00700 -1 mds.h *** got signal Terminated *** 2024-03-21T10:39:10.210 INFO:journalctl@ceph.mds.h.smithi100.stdout:Mar 21 10:39:09 smithi100 podman[72207]: 2024-03-21 10:39:09.900423119 +0000 UTC m=+2.285248406 container died 8a41fac6864248e992b6e5b139cdeac025974f3418e722dbcb048f7eda8e8524 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-3056d51a-e76b-11ee-95cd-87774f69a715-mds-h, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240319, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux ) 2024-03-21T10:39:10.476 INFO:journalctl@ceph.mds.h.smithi100.stdout:Mar 21 10:39:10 smithi100 podman[72207]: 2024-03-21 10:39:10.334700161 +0000 UTC m=+2.719525461 container cleanup 8a41fac6864248e992b6e5b139cdeac025974f3418e722dbcb048f7eda8e8524 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-3056d51a-e76b-11ee-95cd-87774f69a715-mds-h, org.label-schema.build-date=20240319, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_CLEAN=True, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, ceph=True, io.buildah.version=1.33.5, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.license=GPLv2) 2024-03-21T10:39:10.476 INFO:journalctl@ceph.mds.h.smithi100.stdout:Mar 21 10:39:10 smithi100 bash[72207]: ceph-3056d51a-e76b-11ee-95cd-87774f69a715-mds-h 2024-03-21T10:39:10.887 INFO:journalctl@ceph.mds.h.smithi100.stdout:Mar 21 10:39:10 smithi100 podman[72220]: 2024-03-21 10:39:10.637192195 +0000 UTC m=+0.720860676 container remove 8a41fac6864248e992b6e5b139cdeac025974f3418e722dbcb048f7eda8e8524 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-3056d51a-e76b-11ee-95cd-87774f69a715-mds-h, maintainer=Guillaume Abrioux , RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, GIT_CLEAN=True, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True, org.label-schema.build-date=20240319) 2024-03-21T10:39:10.941 DEBUG:teuthology.orchestra.run.smithi100:> sudo pkill -f 'journalctl -f -n 0 -u ceph-3056d51a-e76b-11ee-95cd-87774f69a715@mds.h.service' 2024-03-21T10:39:11.049 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-21T10:39:11.049 INFO:tasks.cephadm.mds.h:Stopped mds.h 2024-03-21T10:39:11.049 INFO:tasks.cephadm.mds.k:Stopping mds.k... 2024-03-21T10:39:11.049 DEBUG:teuthology.orchestra.run.smithi100:> sudo systemctl stop ceph-3056d51a-e76b-11ee-95cd-87774f69a715@mds.k 2024-03-21T10:39:11.475 INFO:journalctl@ceph.mds.k.smithi100.stdout:Mar 21 10:39:11 smithi100 systemd[1]: Stopping Ceph mds.k for 3056d51a-e76b-11ee-95cd-87774f69a715... 2024-03-21T10:39:11.976 INFO:journalctl@ceph.mds.k.smithi100.stdout:Mar 21 10:39:11 smithi100 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-mds-k[64070]: 2024-03-21T10:39:11.621+0000 7fc0ed400700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.k -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-21T10:39:11.976 INFO:journalctl@ceph.mds.k.smithi100.stdout:Mar 21 10:39:11 smithi100 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-mds-k[64070]: 2024-03-21T10:39:11.621+0000 7fc0ed400700 -1 mds.k *** got signal Terminated *** 2024-03-21T10:39:14.162 INFO:journalctl@ceph.mds.k.smithi100.stdout:Mar 21 10:39:13 smithi100 podman[72325]: 2024-03-21 10:39:13.909915932 +0000 UTC m=+2.410536275 container died 6867ea9076c0ab6dc16b0e288d2479cb09856d735987273937c2a3c421815794 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-3056d51a-e76b-11ee-95cd-87774f69a715-mds-k, GIT_CLEAN=True, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240319, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0) 2024-03-21T10:39:14.476 INFO:journalctl@ceph.mds.k.smithi100.stdout:Mar 21 10:39:14 smithi100 podman[72325]: 2024-03-21 10:39:14.265203982 +0000 UTC m=+2.765824355 container cleanup 6867ea9076c0ab6dc16b0e288d2479cb09856d735987273937c2a3c421815794 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-3056d51a-e76b-11ee-95cd-87774f69a715-mds-k, ceph=True, org.label-schema.vendor=CentOS, GIT_CLEAN=True, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.build-date=20240319) 2024-03-21T10:39:14.476 INFO:journalctl@ceph.mds.k.smithi100.stdout:Mar 21 10:39:14 smithi100 bash[72325]: ceph-3056d51a-e76b-11ee-95cd-87774f69a715-mds-k 2024-03-21T10:39:14.976 INFO:journalctl@ceph.mds.k.smithi100.stdout:Mar 21 10:39:14 smithi100 podman[72339]: 2024-03-21 10:39:14.482528591 +0000 UTC m=+0.555466727 container remove 6867ea9076c0ab6dc16b0e288d2479cb09856d735987273937c2a3c421815794 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-3056d51a-e76b-11ee-95cd-87774f69a715-mds-k, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_CLEAN=True, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, org.label-schema.build-date=20240319, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_BRANCH=HEAD, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, maintainer=Guillaume Abrioux ) 2024-03-21T10:39:15.019 DEBUG:teuthology.orchestra.run.smithi100:> sudo pkill -f 'journalctl -f -n 0 -u ceph-3056d51a-e76b-11ee-95cd-87774f69a715@mds.k.service' 2024-03-21T10:39:15.133 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-21T10:39:15.133 INFO:tasks.cephadm.mds.k:Stopped mds.k 2024-03-21T10:39:15.133 INFO:tasks.cephadm.mds.c:Stopping mds.c... 2024-03-21T10:39:15.134 DEBUG:teuthology.orchestra.run.smithi157:> sudo systemctl stop ceph-3056d51a-e76b-11ee-95cd-87774f69a715@mds.c 2024-03-21T10:39:15.486 INFO:journalctl@ceph.mds.c.smithi157.stdout:Mar 21 10:39:15 smithi157 systemd[1]: Stopping Ceph mds.c for 3056d51a-e76b-11ee-95cd-87774f69a715... 2024-03-21T10:39:15.986 INFO:journalctl@ceph.mds.c.smithi157.stdout:Mar 21 10:39:15 smithi157 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-mds-c[66294]: 2024-03-21T10:39:15.699+0000 7f721c600700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-21T10:39:15.986 INFO:journalctl@ceph.mds.c.smithi157.stdout:Mar 21 10:39:15 smithi157 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-mds-c[66294]: 2024-03-21T10:39:15.699+0000 7f721c600700 -1 mds.c *** got signal Terminated *** 2024-03-21T10:39:18.188 INFO:journalctl@ceph.mds.c.smithi157.stdout:Mar 21 10:39:17 smithi157 podman[73994]: 2024-03-21 10:39:17.912035823 +0000 UTC m=+2.356003253 container died 55d8b6b1a13b2ba9ec9abc3ec6ed4dc7eb96680b2a2fe637a89a64f1cbd2542a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-3056d51a-e76b-11ee-95cd-87774f69a715-mds-c, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240319, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, io.buildah.version=1.33.5, GIT_CLEAN=True, ceph=True) 2024-03-21T10:39:18.486 INFO:journalctl@ceph.mds.c.smithi157.stdout:Mar 21 10:39:18 smithi157 podman[73994]: 2024-03-21 10:39:18.385698339 +0000 UTC m=+2.829665742 container cleanup 55d8b6b1a13b2ba9ec9abc3ec6ed4dc7eb96680b2a2fe637a89a64f1cbd2542a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-3056d51a-e76b-11ee-95cd-87774f69a715-mds-c, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240319) 2024-03-21T10:39:18.486 INFO:journalctl@ceph.mds.c.smithi157.stdout:Mar 21 10:39:18 smithi157 bash[73994]: ceph-3056d51a-e76b-11ee-95cd-87774f69a715-mds-c 2024-03-21T10:39:18.986 INFO:journalctl@ceph.mds.c.smithi157.stdout:Mar 21 10:39:18 smithi157 podman[74017]: 2024-03-21 10:39:18.720065265 +0000 UTC m=+0.791021450 container remove 55d8b6b1a13b2ba9ec9abc3ec6ed4dc7eb96680b2a2fe637a89a64f1cbd2542a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-3056d51a-e76b-11ee-95cd-87774f69a715-mds-c, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240319, org.label-schema.vendor=CentOS, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux ) 2024-03-21T10:39:19.042 DEBUG:teuthology.orchestra.run.smithi157:> sudo pkill -f 'journalctl -f -n 0 -u ceph-3056d51a-e76b-11ee-95cd-87774f69a715@mds.c.service' 2024-03-21T10:39:19.159 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-21T10:39:19.159 INFO:tasks.cephadm.mds.c:Stopped mds.c 2024-03-21T10:39:19.159 INFO:tasks.cephadm.mds.f:Stopping mds.f... 2024-03-21T10:39:19.159 DEBUG:teuthology.orchestra.run.smithi157:> sudo systemctl stop ceph-3056d51a-e76b-11ee-95cd-87774f69a715@mds.f 2024-03-21T10:39:19.736 INFO:journalctl@ceph.mds.f.smithi157.stdout:Mar 21 10:39:19 smithi157 systemd[1]: Stopping Ceph mds.f for 3056d51a-e76b-11ee-95cd-87774f69a715... 2024-03-21T10:39:20.236 INFO:journalctl@ceph.mds.f.smithi157.stdout:Mar 21 10:39:19 smithi157 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-mds-f[65979]: 2024-03-21T10:39:19.753+0000 7fb318800700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.f -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-21T10:39:20.236 INFO:journalctl@ceph.mds.f.smithi157.stdout:Mar 21 10:39:19 smithi157 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-mds-f[65979]: 2024-03-21T10:39:19.753+0000 7fb318800700 -1 mds.f *** got signal Terminated *** 2024-03-21T10:39:22.201 INFO:journalctl@ceph.mds.f.smithi157.stdout:Mar 21 10:39:21 smithi157 podman[74127]: 2024-03-21 10:39:21.895147065 +0000 UTC m=+2.265610427 container died 96a8befe1f795cbf5899d1331296e8a3d373ed092f34602632f159c40d0285eb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-3056d51a-e76b-11ee-95cd-87774f69a715-mds-f, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.build-date=20240319, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0) 2024-03-21T10:39:22.486 INFO:journalctl@ceph.mds.f.smithi157.stdout:Mar 21 10:39:22 smithi157 podman[74127]: 2024-03-21 10:39:22.322775881 +0000 UTC m=+2.693239233 container cleanup 96a8befe1f795cbf5899d1331296e8a3d373ed092f34602632f159c40d0285eb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-3056d51a-e76b-11ee-95cd-87774f69a715-mds-f, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240319, org.label-schema.license=GPLv2, GIT_CLEAN=True, ceph=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD) 2024-03-21T10:39:22.486 INFO:journalctl@ceph.mds.f.smithi157.stdout:Mar 21 10:39:22 smithi157 bash[74127]: ceph-3056d51a-e76b-11ee-95cd-87774f69a715-mds-f 2024-03-21T10:39:22.986 INFO:journalctl@ceph.mds.f.smithi157.stdout:Mar 21 10:39:22 smithi157 podman[74142]: 2024-03-21 10:39:22.623914204 +0000 UTC m=+0.710661359 container remove 96a8befe1f795cbf5899d1331296e8a3d373ed092f34602632f159c40d0285eb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-3056d51a-e76b-11ee-95cd-87774f69a715-mds-f, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, io.buildah.version=1.33.5, org.label-schema.build-date=20240319, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6) 2024-03-21T10:39:23.124 DEBUG:teuthology.orchestra.run.smithi157:> sudo pkill -f 'journalctl -f -n 0 -u ceph-3056d51a-e76b-11ee-95cd-87774f69a715@mds.f.service' 2024-03-21T10:39:23.239 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-21T10:39:23.240 INFO:tasks.cephadm.mds.f:Stopped mds.f 2024-03-21T10:39:23.240 INFO:tasks.cephadm.mds.i:Stopping mds.i... 2024-03-21T10:39:23.240 DEBUG:teuthology.orchestra.run.smithi157:> sudo systemctl stop ceph-3056d51a-e76b-11ee-95cd-87774f69a715@mds.i 2024-03-21T10:39:23.736 INFO:journalctl@ceph.mds.i.smithi157.stdout:Mar 21 10:39:23 smithi157 systemd[1]: Stopping Ceph mds.i for 3056d51a-e76b-11ee-95cd-87774f69a715... 2024-03-21T10:39:24.236 INFO:journalctl@ceph.mds.i.smithi157.stdout:Mar 21 10:39:23 smithi157 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-mds-i[70254]: 2024-03-21T10:39:23.850+0000 7fd236600700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.i -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-21T10:39:24.236 INFO:journalctl@ceph.mds.i.smithi157.stdout:Mar 21 10:39:23 smithi157 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-mds-i[70254]: 2024-03-21T10:39:23.850+0000 7fd236600700 -1 mds.i *** got signal Terminated *** 2024-03-21T10:39:26.203 INFO:journalctl@ceph.mds.i.smithi157.stdout:Mar 21 10:39:25 smithi157 podman[74248]: 2024-03-21 10:39:25.913628521 +0000 UTC m=+2.186063459 container died ab79b2d30930c056f856b8a2c0014be41d4218687dffd31341cb4ba04a345c1b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-3056d51a-e76b-11ee-95cd-87774f69a715-mds-i, org.label-schema.build-date=20240319, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5) 2024-03-21T10:39:26.486 INFO:journalctl@ceph.mds.i.smithi157.stdout:Mar 21 10:39:26 smithi157 podman[74248]: 2024-03-21 10:39:26.318345873 +0000 UTC m=+2.590780791 container cleanup ab79b2d30930c056f856b8a2c0014be41d4218687dffd31341cb4ba04a345c1b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-3056d51a-e76b-11ee-95cd-87774f69a715-mds-i, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, org.label-schema.build-date=20240319, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_CLEAN=True, ceph=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30) 2024-03-21T10:39:26.486 INFO:journalctl@ceph.mds.i.smithi157.stdout:Mar 21 10:39:26 smithi157 bash[74248]: ceph-3056d51a-e76b-11ee-95cd-87774f69a715-mds-i 2024-03-21T10:39:26.986 INFO:journalctl@ceph.mds.i.smithi157.stdout:Mar 21 10:39:26 smithi157 podman[74262]: 2024-03-21 10:39:26.58631834 +0000 UTC m=+0.657975191 container remove ab79b2d30930c056f856b8a2c0014be41d4218687dffd31341cb4ba04a345c1b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-3056d51a-e76b-11ee-95cd-87774f69a715-mds-i, GIT_BRANCH=HEAD, org.label-schema.build-date=20240319, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, ceph=True) 2024-03-21T10:39:27.092 DEBUG:teuthology.orchestra.run.smithi157:> sudo pkill -f 'journalctl -f -n 0 -u ceph-3056d51a-e76b-11ee-95cd-87774f69a715@mds.i.service' 2024-03-21T10:39:27.200 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-21T10:39:27.201 INFO:tasks.cephadm.mds.i:Stopped mds.i 2024-03-21T10:39:27.201 INFO:tasks.cephadm.mds.l:Stopping mds.l... 2024-03-21T10:39:27.201 DEBUG:teuthology.orchestra.run.smithi157:> sudo systemctl stop ceph-3056d51a-e76b-11ee-95cd-87774f69a715@mds.l 2024-03-21T10:39:27.736 INFO:journalctl@ceph.mds.l.smithi157.stdout:Mar 21 10:39:27 smithi157 systemd[1]: Stopping Ceph mds.l for 3056d51a-e76b-11ee-95cd-87774f69a715... 2024-03-21T10:39:28.236 INFO:journalctl@ceph.mds.l.smithi157.stdout:Mar 21 10:39:27 smithi157 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-mds-l[65659]: 2024-03-21T10:39:27.829+0000 7fac66400700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.l -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-21T10:39:28.236 INFO:journalctl@ceph.mds.l.smithi157.stdout:Mar 21 10:39:27 smithi157 ceph-3056d51a-e76b-11ee-95cd-87774f69a715-mds-l[65659]: 2024-03-21T10:39:27.829+0000 7fac66400700 -1 mds.l *** got signal Terminated *** 2024-03-21T10:39:30.197 INFO:journalctl@ceph.mds.l.smithi157.stdout:Mar 21 10:39:29 smithi157 podman[74367]: 2024-03-21 10:39:29.922463565 +0000 UTC m=+2.253808912 container died 49f83b9ee10028688a9908fe705d11794e6453616d82420c5ce2eb932178549c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-3056d51a-e76b-11ee-95cd-87774f69a715-mds-l, ceph=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.build-date=20240319, org.label-schema.license=GPLv2, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.schema-version=1.0) 2024-03-21T10:39:30.486 INFO:journalctl@ceph.mds.l.smithi157.stdout:Mar 21 10:39:30 smithi157 podman[74367]: 2024-03-21 10:39:30.313723386 +0000 UTC m=+2.645068700 container cleanup 49f83b9ee10028688a9908fe705d11794e6453616d82420c5ce2eb932178549c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-3056d51a-e76b-11ee-95cd-87774f69a715-mds-l, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, ceph=True, org.label-schema.build-date=20240319, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-21T10:39:30.486 INFO:journalctl@ceph.mds.l.smithi157.stdout:Mar 21 10:39:30 smithi157 bash[74367]: ceph-3056d51a-e76b-11ee-95cd-87774f69a715-mds-l 2024-03-21T10:39:30.838 INFO:journalctl@ceph.mds.l.smithi157.stdout:Mar 21 10:39:30 smithi157 podman[74381]: 2024-03-21 10:39:30.589712159 +0000 UTC m=+0.653633904 container remove 49f83b9ee10028688a9908fe705d11794e6453616d82420c5ce2eb932178549c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-3056d51a-e76b-11ee-95cd-87774f69a715-mds-l, org.label-schema.vendor=CentOS, GIT_CLEAN=True, ceph=True, io.buildah.version=1.33.5, org.label-schema.build-date=20240319, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, CEPH_POINT_RELEASE=) 2024-03-21T10:39:30.838 INFO:journalctl@ceph.mds.l.smithi157.stdout:Mar 21 10:39:30 smithi157 systemd[1]: ceph-3056d51a-e76b-11ee-95cd-87774f69a715@mds.l.service: Deactivated successfully. 2024-03-21T10:39:30.838 INFO:journalctl@ceph.mds.l.smithi157.stdout:Mar 21 10:39:30 smithi157 systemd[1]: Stopped Ceph mds.l for 3056d51a-e76b-11ee-95cd-87774f69a715. 2024-03-21T10:39:30.850 DEBUG:teuthology.orchestra.run.smithi157:> sudo pkill -f 'journalctl -f -n 0 -u ceph-3056d51a-e76b-11ee-95cd-87774f69a715@mds.l.service' 2024-03-21T10:39:30.964 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-21T10:39:30.965 INFO:tasks.cephadm.mds.l:Stopped mds.l 2024-03-21T10:39:30.965 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 3056d51a-e76b-11ee-95cd-87774f69a715 --force --keep-logs 2024-03-21T10:39:31.305 INFO:teuthology.orchestra.run.smithi047.stdout:Deleting cluster with fsid: 3056d51a-e76b-11ee-95cd-87774f69a715 2024-03-21T10:39:36.448 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 3056d51a-e76b-11ee-95cd-87774f69a715 --force --keep-logs 2024-03-21T10:39:36.764 INFO:teuthology.orchestra.run.smithi100.stdout:Deleting cluster with fsid: 3056d51a-e76b-11ee-95cd-87774f69a715 2024-03-21T10:39:41.542 DEBUG:teuthology.orchestra.run.smithi157:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 3056d51a-e76b-11ee-95cd-87774f69a715 --force --keep-logs 2024-03-21T10:39:41.866 INFO:teuthology.orchestra.run.smithi157.stdout:Deleting cluster with fsid: 3056d51a-e76b-11ee-95cd-87774f69a715 2024-03-21T10:39:46.636 DEBUG:teuthology.orchestra.run.smithi047:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-03-21T10:39:46.747 DEBUG:teuthology.orchestra.run.smithi100:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-03-21T10:39:46.851 DEBUG:teuthology.orchestra.run.smithi157:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-03-21T10:39:46.931 INFO:tasks.cephadm:Archiving crash dumps... 2024-03-21T10:39:46.934 DEBUG:teuthology.misc:Transferring archived files from smithi047:/var/lib/ceph/3056d51a-e76b-11ee-95cd-87774f69a715/crash to /home/teuthworker/archive/vshankar-2024-03-20_11:21:18-fs-wip-vshankar-testing-20240320.053002-testing-default-smithi/7612221/remote/smithi047/crash 2024-03-21T10:39:46.934 DEBUG:teuthology.orchestra.run.smithi047:> sudo tar c -f - -C /var/lib/ceph/3056d51a-e76b-11ee-95cd-87774f69a715/crash -- . 2024-03-21T10:39:47.043 INFO:teuthology.orchestra.run.smithi047.stderr:tar: /var/lib/ceph/3056d51a-e76b-11ee-95cd-87774f69a715/crash: Cannot open: No such file or directory 2024-03-21T10:39:47.043 INFO:teuthology.orchestra.run.smithi047.stderr:tar: Error is not recoverable: exiting now 2024-03-21T10:39:47.049 DEBUG:teuthology.misc:Transferring archived files from smithi100:/var/lib/ceph/3056d51a-e76b-11ee-95cd-87774f69a715/crash to /home/teuthworker/archive/vshankar-2024-03-20_11:21:18-fs-wip-vshankar-testing-20240320.053002-testing-default-smithi/7612221/remote/smithi100/crash 2024-03-21T10:39:47.049 DEBUG:teuthology.orchestra.run.smithi100:> sudo tar c -f - -C /var/lib/ceph/3056d51a-e76b-11ee-95cd-87774f69a715/crash -- . 2024-03-21T10:39:47.160 INFO:teuthology.orchestra.run.smithi100.stderr:tar: /var/lib/ceph/3056d51a-e76b-11ee-95cd-87774f69a715/crash: Cannot open: No such file or directory 2024-03-21T10:39:47.160 INFO:teuthology.orchestra.run.smithi100.stderr:tar: Error is not recoverable: exiting now 2024-03-21T10:39:47.166 DEBUG:teuthology.misc:Transferring archived files from smithi157:/var/lib/ceph/3056d51a-e76b-11ee-95cd-87774f69a715/crash to /home/teuthworker/archive/vshankar-2024-03-20_11:21:18-fs-wip-vshankar-testing-20240320.053002-testing-default-smithi/7612221/remote/smithi157/crash 2024-03-21T10:39:47.167 DEBUG:teuthology.orchestra.run.smithi157:> sudo tar c -f - -C /var/lib/ceph/3056d51a-e76b-11ee-95cd-87774f69a715/crash -- . 2024-03-21T10:39:47.271 INFO:teuthology.orchestra.run.smithi157.stderr:tar: /var/lib/ceph/3056d51a-e76b-11ee-95cd-87774f69a715/crash: Cannot open: No such file or directory 2024-03-21T10:39:47.271 INFO:teuthology.orchestra.run.smithi157.stderr:tar: Error is not recoverable: exiting now 2024-03-21T10:39:47.276 INFO:tasks.cephadm:Checking cluster log for badness... 2024-03-21T10:39:47.276 DEBUG:teuthology.orchestra.run.smithi047:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/3056d51a-e76b-11ee-95cd-87774f69a715/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v 'filesystem is online with fewer MDS than max_mds' | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'deprecated feature inline_data' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v MON_DOWN | egrep -v 'slow metadata IO' | egrep -v SLOW_OPS | egrep -v 'slow request' | head -n 1 2024-03-21T10:39:47.393 INFO:teuthology.orchestra.run.smithi047.stdout:2024-03-21T10:13:44.902312+0000 mon.a (mon.0) 183 : cluster [WRN] Health detail: HEALTH_WARN 1/3 mons down, quorum a,c 2024-03-21T10:39:47.394 WARNING:tasks.cephadm:Found errors (ERR|WRN|SEC) in cluster log 2024-03-21T10:39:47.394 DEBUG:teuthology.orchestra.run.smithi047:> sudo egrep '\[SEC\]' /var/log/ceph/3056d51a-e76b-11ee-95cd-87774f69a715/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v 'filesystem is online with fewer MDS than max_mds' | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'deprecated feature inline_data' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v MON_DOWN | egrep -v 'slow metadata IO' | egrep -v SLOW_OPS | egrep -v 'slow request' | head -n 1 2024-03-21T10:39:47.489 DEBUG:teuthology.orchestra.run.smithi047:> sudo egrep '\[ERR\]' /var/log/ceph/3056d51a-e76b-11ee-95cd-87774f69a715/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v 'filesystem is online with fewer MDS than max_mds' | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'deprecated feature inline_data' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v MON_DOWN | egrep -v 'slow metadata IO' | egrep -v SLOW_OPS | egrep -v 'slow request' | head -n 1 2024-03-21T10:39:47.581 DEBUG:teuthology.orchestra.run.smithi047:> sudo egrep '\[WRN\]' /var/log/ceph/3056d51a-e76b-11ee-95cd-87774f69a715/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v 'filesystem is online with fewer MDS than max_mds' | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'deprecated feature inline_data' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v MON_DOWN | egrep -v 'slow metadata IO' | egrep -v SLOW_OPS | egrep -v 'slow request' | head -n 1 2024-03-21T10:39:47.675 INFO:teuthology.orchestra.run.smithi047.stdout:2024-03-21T10:13:44.902312+0000 mon.a (mon.0) 183 : cluster [WRN] Health detail: HEALTH_WARN 1/3 mons down, quorum a,c 2024-03-21T10:39:47.676 INFO:tasks.cephadm:Compressing logs... 2024-03-21T10:39:47.676 DEBUG:teuthology.orchestra.run.smithi047:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-03-21T10:39:47.681 DEBUG:teuthology.orchestra.run.smithi100:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-03-21T10:39:47.689 DEBUG:teuthology.orchestra.run.smithi157:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-03-21T10:39:47.751 INFO:teuthology.orchestra.run.smithi047.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-03-21T10:39:47.751 INFO:teuthology.orchestra.run.smithi047.stderr:: No such file or directory 2024-03-21T10:39:47.752 INFO:teuthology.orchestra.run.smithi047.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.log 2024-03-21T10:39:47.753 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/cephadm.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.log 2024-03-21T10:39:47.754 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.0.log: gzip -5 --verbose -- /var/log/ceph/3056d51a-e76b-11ee-95cd-87774f69a715/ceph-osd.6.log 2024-03-21T10:39:47.755 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.log: gzip -5 --verbose -- /var/log/ceph/3056d51a-e76b-11ee-95cd-87774f69a715/ceph-mon.a.log 2024-03-21T10:39:47.757 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/3056d51a-e76b-11ee-95cd-87774f69a715/ceph-osd.6.log: gzip -5 --verbose -- /var/log/ceph/3056d51a-e76b-11ee-95cd-87774f69a715/ceph.audit.log 2024-03-21T10:39:47.758 INFO:teuthology.orchestra.run.smithi047.stderr: 92.3% -- replaced with /var/log/ceph/ceph-client.admin.log.gz 2024-03-21T10:39:47.759 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/3056d51a-e76b-11ee-95cd-87774f69a715/ceph-mon.a.log: gzip -5 --verbose -- /var/log/ceph/3056d51a-e76b-11ee-95cd-87774f69a715/ceph-mds.g.log 2024-03-21T10:39:47.760 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/3056d51a-e76b-11ee-95cd-87774f69a715/ceph.audit.log: 93.2% -- replaced with /var/log/ceph/cephadm.log.gz 2024-03-21T10:39:47.761 INFO:teuthology.orchestra.run.smithi047.stderr:gzip -5 --verbose -- /var/log/ceph/3056d51a-e76b-11ee-95cd-87774f69a715/ceph-mds.j.log 2024-03-21T10:39:47.762 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/3056d51a-e76b-11ee-95cd-87774f69a715/ceph-mds.g.log: gzip -5 --verbose -- /var/log/ceph/3056d51a-e76b-11ee-95cd-87774f69a715/ceph-osd.3.log 2024-03-21T10:39:47.764 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/3056d51a-e76b-11ee-95cd-87774f69a715/ceph-mds.j.log: gzip -5 --verbose -- /var/log/ceph/3056d51a-e76b-11ee-95cd-87774f69a715/ceph-mgr.x.log 2024-03-21T10:39:47.764 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/3056d51a-e76b-11ee-95cd-87774f69a715/ceph-osd.3.log: 90.1% -- replaced with /var/log/ceph/3056d51a-e76b-11ee-95cd-87774f69a715/ceph.audit.log.gz 2024-03-21T10:39:47.768 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/3056d51a-e76b-11ee-95cd-87774f69a715/ceph-mgr.x.log: gzip -5 --verbose -- /var/log/ceph/3056d51a-e76b-11ee-95cd-87774f69a715/ceph-osd.0.log 2024-03-21T10:39:47.770 INFO:teuthology.orchestra.run.smithi047.stderr:gzip -5 --verbose -- /var/log/ceph/3056d51a-e76b-11ee-95cd-87774f69a715/ceph.cephadm.log 2024-03-21T10:39:47.770 INFO:teuthology.orchestra.run.smithi047.stderr: 94.5% -- replaced with /var/log/ceph/3056d51a-e76b-11ee-95cd-87774f69a715/ceph-mds.j.log.gz 2024-03-21T10:39:47.771 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/3056d51a-e76b-11ee-95cd-87774f69a715/ceph-osd.0.log: gzip -5 --verbose -- /var/log/ceph/3056d51a-e76b-11ee-95cd-87774f69a715/ceph.log 2024-03-21T10:39:47.772 INFO:teuthology.orchestra.run.smithi157.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-03-21T10:39:47.772 INFO:teuthology.orchestra.run.smithi157.stderr:: No such file or directory 2024-03-21T10:39:47.773 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/3056d51a-e76b-11ee-95cd-87774f69a715/ceph.cephadm.log: gzip -5 --verbose -- /var/log/ceph/3056d51a-e76b-11ee-95cd-87774f69a715/ceph-volume.log 2024-03-21T10:39:47.773 INFO:teuthology.orchestra.run.smithi157.stderr:gzip -5 --verbose -- /var/log/ceph/3056d51a-e76b-11ee-95cd-87774f69a715/ceph-mds.l.log 2024-03-21T10:39:47.774 INFO:teuthology.orchestra.run.smithi157.stderr:/var/log/ceph/cephadm.log: gzip -5 --verbose -- /var/log/ceph/3056d51a-e76b-11ee-95cd-87774f69a715/ceph.audit.log 2024-03-21T10:39:47.776 INFO:teuthology.orchestra.run.smithi100.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose --: No such file or directory /var/log/ceph/cephadm.log 2024-03-21T10:39:47.776 INFO:teuthology.orchestra.run.smithi100.stderr: 2024-03-21T10:39:47.776 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/3056d51a-e76b-11ee-95cd-87774f69a715/ceph-osd.10.log 2024-03-21T10:39:47.776 INFO:teuthology.orchestra.run.smithi157.stderr:/var/log/ceph/3056d51a-e76b-11ee-95cd-87774f69a715/ceph-mds.l.log: gzip -5 --verbose -- /var/log/ceph/3056d51a-e76b-11ee-95cd-87774f69a715/ceph-mds.i.log 2024-03-21T10:39:47.776 INFO:teuthology.orchestra.run.smithi157.stderr:/var/log/ceph/3056d51a-e76b-11ee-95cd-87774f69a715/ceph.audit.log: 92.3% -- replaced with /var/log/ceph/cephadm.log.gz 2024-03-21T10:39:47.777 INFO:teuthology.orchestra.run.smithi157.stderr:gzip -5 --verbose -- /var/log/ceph/3056d51a-e76b-11ee-95cd-87774f69a715/ceph-osd.11.log 2024-03-21T10:39:47.779 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/cephadm.log: gzip -5 --verbose -- /var/log/ceph/3056d51a-e76b-11ee-95cd-87774f69a715/ceph.audit.log 2024-03-21T10:39:47.779 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/3056d51a-e76b-11ee-95cd-87774f69a715/ceph-osd.10.log: 92.0% -- replaced with /var/log/ceph/cephadm.log.gz 2024-03-21T10:39:47.779 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/3056d51a-e76b-11ee-95cd-87774f69a715/ceph-mds.b.log 2024-03-21T10:39:47.779 INFO:teuthology.orchestra.run.smithi047.stderr: 83.8%/var/log/ceph/3056d51a-e76b-11ee-95cd-87774f69a715/ceph.log: -- replaced with /var/log/ceph/3056d51a-e76b-11ee-95cd-87774f69a715/ceph.cephadm.log.gz 2024-03-21T10:39:47.779 INFO:teuthology.orchestra.run.smithi047.stderr:gzip -5 --verbose -- /var/log/ceph/3056d51a-e76b-11ee-95cd-87774f69a715/ceph-osd.9.log 2024-03-21T10:39:47.781 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/3056d51a-e76b-11ee-95cd-87774f69a715/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/3056d51a-e76b-11ee-95cd-87774f69a715/ceph-osd.4.log 2024-03-21T10:39:47.781 INFO:teuthology.orchestra.run.smithi157.stderr:/var/log/ceph/3056d51a-e76b-11ee-95cd-87774f69a715/ceph-mds.i.log: gzip -5 --verbose -- /var/log/ceph/3056d51a-e76b-11ee-95cd-87774f69a715/ceph-mgr.z.log 2024-03-21T10:39:47.781 INFO:teuthology.orchestra.run.smithi157.stderr:/var/log/ceph/3056d51a-e76b-11ee-95cd-87774f69a715/ceph-osd.11.log: gzip -5 --verbose -- /var/log/ceph/3056d51a-e76b-11ee-95cd-87774f69a715/ceph-osd.8.log 2024-03-21T10:39:47.782 INFO:teuthology.orchestra.run.smithi157.stderr: 90.2% -- replaced with /var/log/ceph/3056d51a-e76b-11ee-95cd-87774f69a715/ceph.audit.log.gz 2024-03-21T10:39:47.782 INFO:teuthology.orchestra.run.smithi157.stderr:/var/log/ceph/3056d51a-e76b-11ee-95cd-87774f69a715/ceph-mgr.z.log: gzip -5 --verbose -- /var/log/ceph/3056d51a-e76b-11ee-95cd-87774f69a715/ceph-osd.5.log 2024-03-21T10:39:47.783 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/3056d51a-e76b-11ee-95cd-87774f69a715/ceph-volume.log: gzip/var/log/ceph/3056d51a-e76b-11ee-95cd-87774f69a715/ceph-osd.9.log: 87.3% -- replaced with /var/log/ceph/3056d51a-e76b-11ee-95cd-87774f69a715/ceph.log.gz 2024-03-21T10:39:47.783 INFO:teuthology.orchestra.run.smithi047.stderr: -5 --verbose -- /var/log/ceph/3056d51a-e76b-11ee-95cd-87774f69a715/ceph-mds.d.log 2024-03-21T10:39:47.783 INFO:teuthology.orchestra.run.smithi157.stderr:/var/log/ceph/3056d51a-e76b-11ee-95cd-87774f69a715/ceph-osd.8.log: gzip -5 --verbose -- /var/log/ceph/3056d51a-e76b-11ee-95cd-87774f69a715/ceph-mds.f.log 2024-03-21T10:39:47.784 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/3056d51a-e76b-11ee-95cd-87774f69a715/ceph-mds.b.log: gzip -5 --verbose -- /var/log/ceph/3056d51a-e76b-11ee-95cd-87774f69a715/ceph-osd.1.log 2024-03-21T10:39:47.784 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/3056d51a-e76b-11ee-95cd-87774f69a715/ceph-osd.4.log: 90.2% -- replaced with /var/log/ceph/3056d51a-e76b-11ee-95cd-87774f69a715/ceph.audit.log.gz 2024-03-21T10:39:47.784 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/3056d51a-e76b-11ee-95cd-87774f69a715/ceph-mds.k.log 2024-03-21T10:39:47.784 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/3056d51a-e76b-11ee-95cd-87774f69a715/ceph-mds.h.log 2024-03-21T10:39:47.785 INFO:teuthology.orchestra.run.smithi047.stderr:gzip -5 --verbose -- /var/log/ceph/3056d51a-e76b-11ee-95cd-87774f69a715/ceph-mds.a.log 2024-03-21T10:39:47.786 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/3056d51a-e76b-11ee-95cd-87774f69a715/ceph-osd.1.log: /var/log/ceph/3056d51a-e76b-11ee-95cd-87774f69a715/ceph-mds.k.log: gzip -5 --verbose -- /var/log/ceph/3056d51a-e76b-11ee-95cd-87774f69a715/ceph-osd.7.log 2024-03-21T10:39:47.786 INFO:teuthology.orchestra.run.smithi157.stderr:/var/log/ceph/3056d51a-e76b-11ee-95cd-87774f69a715/ceph-osd.5.log: /var/log/ceph/3056d51a-e76b-11ee-95cd-87774f69a715/ceph-mds.f.log: gzip -5 --verbose -- /var/log/ceph/3056d51a-e76b-11ee-95cd-87774f69a715/ceph.cephadm.log 2024-03-21T10:39:47.787 INFO:teuthology.orchestra.run.smithi157.stderr:gzip -5 --verbose -- /var/log/ceph/3056d51a-e76b-11ee-95cd-87774f69a715/ceph.log 2024-03-21T10:39:47.787 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/3056d51a-e76b-11ee-95cd-87774f69a715/ceph-mds.h.log: gzip -5 --verbose -- /var/log/ceph/3056d51a-e76b-11ee-95cd-87774f69a715/ceph-mds.e.log 2024-03-21T10:39:47.789 INFO:teuthology.orchestra.run.smithi157.stderr:/var/log/ceph/3056d51a-e76b-11ee-95cd-87774f69a715/ceph.cephadm.log: 83.6% -- replaced with /var/log/ceph/3056d51a-e76b-11ee-95cd-87774f69a715/ceph.cephadm.log.gz 2024-03-21T10:39:47.790 INFO:teuthology.orchestra.run.smithi157.stderr:gzip -5 --verbose -- /var/log/ceph/3056d51a-e76b-11ee-95cd-87774f69a715/ceph-osd.2.log 2024-03-21T10:39:47.791 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/3056d51a-e76b-11ee-95cd-87774f69a715/ceph-osd.7.log: /var/log/ceph/3056d51a-e76b-11ee-95cd-87774f69a715/ceph-mds.e.log: gzip -5 --verbose -- /var/log/ceph/3056d51a-e76b-11ee-95cd-87774f69a715/ceph-mgr.y.log 2024-03-21T10:39:47.792 INFO:teuthology.orchestra.run.smithi157.stderr: 92.1% -- replaced with /var/log/ceph/3056d51a-e76b-11ee-95cd-87774f69a715/ceph-mgr.z.log.gz 2024-03-21T10:39:47.792 INFO:teuthology.orchestra.run.smithi157.stderr:gzip -5 --verbose -- /var/log/ceph/3056d51a-e76b-11ee-95cd-87774f69a715/ceph-mds.c.log 2024-03-21T10:39:47.792 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/3056d51a-e76b-11ee-95cd-87774f69a715/ceph.cephadm.log 2024-03-21T10:39:47.793 INFO:teuthology.orchestra.run.smithi157.stderr:gzip -5 --verbose --/var/log/ceph/3056d51a-e76b-11ee-95cd-87774f69a715/ceph.log: /var/log/ceph/3056d51a-e76b-11ee-95cd-87774f69a715/ceph-volume.log 2024-03-21T10:39:47.794 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/3056d51a-e76b-11ee-95cd-87774f69a715/ceph-mgr.y.log: gzip -5 --verbose -- /var/log/ceph/3056d51a-e76b-11ee-95cd-87774f69a715/ceph.log 2024-03-21T10:39:47.795 INFO:teuthology.orchestra.run.smithi157.stderr:/var/log/ceph/3056d51a-e76b-11ee-95cd-87774f69a715/ceph-mds.c.log: gzip -5 --verbose -- /var/log/ceph/3056d51a-e76b-11ee-95cd-87774f69a715/ceph-mon.c.log 2024-03-21T10:39:47.796 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/3056d51a-e76b-11ee-95cd-87774f69a715/ceph.cephadm.log: 83.7% -- replaced with /var/log/ceph/3056d51a-e76b-11ee-95cd-87774f69a715/ceph.cephadm.log.gz 2024-03-21T10:39:47.799 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/3056d51a-e76b-11ee-95cd-87774f69a715/ceph-mds.d.log: /var/log/ceph/3056d51a-e76b-11ee-95cd-87774f69a715/ceph-mds.a.log: 94.5% -- replaced with /var/log/ceph/3056d51a-e76b-11ee-95cd-87774f69a715/ceph-mds.d.log.gz 2024-03-21T10:39:47.800 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/3056d51a-e76b-11ee-95cd-87774f69a715/ceph.log: gzip -5 --verbose -- /var/log/ceph/3056d51a-e76b-11ee-95cd-87774f69a715/ceph-volume.log 2024-03-21T10:39:47.802 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/3056d51a-e76b-11ee-95cd-87774f69a715/ceph-mon.b.log 2024-03-21T10:39:47.803 INFO:teuthology.orchestra.run.smithi157.stderr:/var/log/ceph/3056d51a-e76b-11ee-95cd-87774f69a715/ceph-volume.log: /var/log/ceph/3056d51a-e76b-11ee-95cd-87774f69a715/ceph-osd.2.log: /var/log/ceph/3056d51a-e76b-11ee-95cd-87774f69a715/ceph-mon.c.log: 87.4% -- replaced with /var/log/ceph/3056d51a-e76b-11ee-95cd-87774f69a715/ceph.log.gz 2024-03-21T10:39:47.804 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/3056d51a-e76b-11ee-95cd-87774f69a715/ceph-volume.log: 87.4% -- replaced with /var/log/ceph/3056d51a-e76b-11ee-95cd-87774f69a715/ceph.log.gz 2024-03-21T10:39:47.807 INFO:teuthology.orchestra.run.smithi047.stderr: 90.8% -- replaced with /var/log/ceph/3056d51a-e76b-11ee-95cd-87774f69a715/ceph-volume.log.gz 2024-03-21T10:39:47.818 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/3056d51a-e76b-11ee-95cd-87774f69a715/ceph-mon.b.log: 92.0% -- replaced with /var/log/ceph/3056d51a-e76b-11ee-95cd-87774f69a715/ceph-mgr.y.log.gz 2024-03-21T10:39:47.830 INFO:teuthology.orchestra.run.smithi157.stderr: 90.8% -- replaced with /var/log/ceph/3056d51a-e76b-11ee-95cd-87774f69a715/ceph-volume.log.gz 2024-03-21T10:39:47.835 INFO:teuthology.orchestra.run.smithi100.stderr: 90.8% -- replaced with /var/log/ceph/3056d51a-e76b-11ee-95cd-87774f69a715/ceph-volume.log.gz 2024-03-21T10:39:47.842 INFO:teuthology.orchestra.run.smithi047.stderr: 94.5% -- replaced with /var/log/ceph/3056d51a-e76b-11ee-95cd-87774f69a715/ceph-mds.g.log.gz 2024-03-21T10:39:47.850 INFO:teuthology.orchestra.run.smithi157.stderr: 94.4% -- replaced with /var/log/ceph/3056d51a-e76b-11ee-95cd-87774f69a715/ceph-mds.l.log.gz 2024-03-21T10:39:47.870 INFO:teuthology.orchestra.run.smithi100.stderr: 92.6% -- replaced with /var/log/ceph/3056d51a-e76b-11ee-95cd-87774f69a715/ceph-mds.k.log.gz 2024-03-21T10:39:47.872 INFO:teuthology.orchestra.run.smithi157.stderr: 94.6% -- replaced with /var/log/ceph/3056d51a-e76b-11ee-95cd-87774f69a715/ceph-mds.c.log.gz 2024-03-21T10:39:47.874 INFO:teuthology.orchestra.run.smithi100.stderr: 92.7% -- replaced with /var/log/ceph/3056d51a-e76b-11ee-95cd-87774f69a715/ceph-mds.h.log.gz 2024-03-21T10:39:47.878 INFO:teuthology.orchestra.run.smithi047.stderr: 92.7% -- replaced with /var/log/ceph/3056d51a-e76b-11ee-95cd-87774f69a715/ceph-mds.a.log.gz 2024-03-21T10:39:47.886 INFO:teuthology.orchestra.run.smithi100.stderr: 92.7% -- replaced with /var/log/ceph/3056d51a-e76b-11ee-95cd-87774f69a715/ceph-mds.e.log.gz 2024-03-21T10:39:47.900 INFO:teuthology.orchestra.run.smithi157.stderr: 94.4% -- replaced with /var/log/ceph/3056d51a-e76b-11ee-95cd-87774f69a715/ceph-mds.f.log.gz 2024-03-21T10:39:47.942 INFO:teuthology.orchestra.run.smithi157.stderr: 93.5% -- replaced with /var/log/ceph/3056d51a-e76b-11ee-95cd-87774f69a715/ceph-mds.i.log.gz 2024-03-21T10:39:48.168 INFO:teuthology.orchestra.run.smithi047.stderr: 89.9% -- replaced with /var/log/ceph/3056d51a-e76b-11ee-95cd-87774f69a715/ceph-mgr.x.log.gz 2024-03-21T10:39:48.213 INFO:teuthology.orchestra.run.smithi100.stderr: 92.8% -- replaced with /var/log/ceph/3056d51a-e76b-11ee-95cd-87774f69a715/ceph-mds.b.log.gz 2024-03-21T10:39:48.248 INFO:teuthology.orchestra.run.smithi157.stderr: 91.9% -- replaced with /var/log/ceph/3056d51a-e76b-11ee-95cd-87774f69a715/ceph-mon.c.log.gz 2024-03-21T10:39:48.342 INFO:teuthology.orchestra.run.smithi100.stderr: 91.9% -- replaced with /var/log/ceph/3056d51a-e76b-11ee-95cd-87774f69a715/ceph-mon.b.log.gz 2024-03-21T10:39:49.342 INFO:teuthology.orchestra.run.smithi047.stderr: 90.5% -- replaced with /var/log/ceph/3056d51a-e76b-11ee-95cd-87774f69a715/ceph-mon.a.log.gz 2024-03-21T10:39:57.220 INFO:teuthology.orchestra.run.smithi047.stderr: 96.0% -- replaced with /var/log/ceph/ceph-client.0.log.gz 2024-03-21T10:39:58.961 INFO:teuthology.orchestra.run.smithi157.stderr: 91.4% -- replaced with /var/log/ceph/3056d51a-e76b-11ee-95cd-87774f69a715/ceph-osd.11.log.gz 2024-03-21T10:39:59.442 INFO:teuthology.orchestra.run.smithi100.stderr: 91.2% -- replaced with /var/log/ceph/3056d51a-e76b-11ee-95cd-87774f69a715/ceph-osd.10.log.gz 2024-03-21T10:39:59.618 INFO:teuthology.orchestra.run.smithi100.stderr: 91.2% -- replaced with /var/log/ceph/3056d51a-e76b-11ee-95cd-87774f69a715/ceph-osd.1.log.gz 2024-03-21T10:40:00.147 INFO:teuthology.orchestra.run.smithi157.stderr: 91.2% -- replaced with /var/log/ceph/3056d51a-e76b-11ee-95cd-87774f69a715/ceph-osd.8.log.gz 2024-03-21T10:40:00.808 INFO:teuthology.orchestra.run.smithi047.stderr: 91.1% -- replaced with /var/log/ceph/3056d51a-e76b-11ee-95cd-87774f69a715/ceph-osd.6.log.gz 2024-03-21T10:40:02.676 INFO:teuthology.orchestra.run.smithi157.stderr: 91.2% -- replaced with /var/log/ceph/3056d51a-e76b-11ee-95cd-87774f69a715/ceph-osd.2.log.gz 2024-03-21T10:40:02.755 INFO:teuthology.orchestra.run.smithi047.stderr: 90.9% -- replaced with /var/log/ceph/3056d51a-e76b-11ee-95cd-87774f69a715/ceph-osd.0.log.gz 2024-03-21T10:40:03.348 INFO:teuthology.orchestra.run.smithi100.stderr: 90.9% -- replaced with /var/log/ceph/3056d51a-e76b-11ee-95cd-87774f69a715/ceph-osd.7.log.gz 2024-03-21T10:40:04.980 INFO:teuthology.orchestra.run.smithi100.stderr: 91.0% -- replaced with /var/log/ceph/3056d51a-e76b-11ee-95cd-87774f69a715/ceph-osd.4.log.gz 2024-03-21T10:40:04.985 INFO:teuthology.orchestra.run.smithi100.stderr: 2024-03-21T10:40:04.986 INFO:teuthology.orchestra.run.smithi100.stderr:real 0m17.251s 2024-03-21T10:40:04.986 INFO:teuthology.orchestra.run.smithi100.stderr:user 0m53.463s 2024-03-21T10:40:04.986 INFO:teuthology.orchestra.run.smithi100.stderr:sys 0m3.852s 2024-03-21T10:40:08.537 INFO:teuthology.orchestra.run.smithi157.stderr: 91.0% -- replaced with /var/log/ceph/3056d51a-e76b-11ee-95cd-87774f69a715/ceph-osd.5.log.gz 2024-03-21T10:40:08.542 INFO:teuthology.orchestra.run.smithi157.stderr: 2024-03-21T10:40:08.542 INFO:teuthology.orchestra.run.smithi157.stderr:real 0m20.803s 2024-03-21T10:40:08.542 INFO:teuthology.orchestra.run.smithi157.stderr:user 0m55.909s 2024-03-21T10:40:08.542 INFO:teuthology.orchestra.run.smithi157.stderr:sys 0m4.039s 2024-03-21T10:40:09.493 INFO:teuthology.orchestra.run.smithi047.stderr: 91.0% -- replaced with /var/log/ceph/3056d51a-e76b-11ee-95cd-87774f69a715/ceph-osd.3.log.gz 2024-03-21T10:40:10.804 INFO:teuthology.orchestra.run.smithi047.stderr: 90.8% -- replaced with /var/log/ceph/3056d51a-e76b-11ee-95cd-87774f69a715/ceph-osd.9.log.gz 2024-03-21T10:40:10.808 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-03-21T10:40:10.808 INFO:teuthology.orchestra.run.smithi047.stderr:real 0m23.094s 2024-03-21T10:40:10.809 INFO:teuthology.orchestra.run.smithi047.stderr:user 1m18.291s 2024-03-21T10:40:10.809 INFO:teuthology.orchestra.run.smithi047.stderr:sys 0m5.418s 2024-03-21T10:40:10.810 INFO:tasks.cephadm:Archiving logs... 2024-03-21T10:40:10.810 DEBUG:teuthology.misc:Transferring archived files from smithi047:/var/log/ceph to /home/teuthworker/archive/vshankar-2024-03-20_11:21:18-fs-wip-vshankar-testing-20240320.053002-testing-default-smithi/7612221/remote/smithi047/log 2024-03-21T10:40:10.811 DEBUG:teuthology.orchestra.run.smithi047:> sudo tar c -f - -C /var/log/ceph -- . 2024-03-21T10:40:22.311 DEBUG:teuthology.misc:Transferring archived files from smithi100:/var/log/ceph to /home/teuthworker/archive/vshankar-2024-03-20_11:21:18-fs-wip-vshankar-testing-20240320.053002-testing-default-smithi/7612221/remote/smithi100/log 2024-03-21T10:40:22.312 DEBUG:teuthology.orchestra.run.smithi100:> sudo tar c -f - -C /var/log/ceph -- . 2024-03-21T10:40:31.313 DEBUG:teuthology.misc:Transferring archived files from smithi157:/var/log/ceph to /home/teuthworker/archive/vshankar-2024-03-20_11:21:18-fs-wip-vshankar-testing-20240320.053002-testing-default-smithi/7612221/remote/smithi157/log 2024-03-21T10:40:31.314 DEBUG:teuthology.orchestra.run.smithi157:> sudo tar c -f - -C /var/log/ceph -- . 2024-03-21T10:40:40.636 INFO:tasks.cephadm:Removing cluster... 2024-03-21T10:40:40.637 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 3056d51a-e76b-11ee-95cd-87774f69a715 --force 2024-03-21T10:40:40.974 INFO:teuthology.orchestra.run.smithi047.stdout:Deleting cluster with fsid: 3056d51a-e76b-11ee-95cd-87774f69a715 2024-03-21T10:40:41.740 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 3056d51a-e76b-11ee-95cd-87774f69a715 --force 2024-03-21T10:40:42.065 INFO:teuthology.orchestra.run.smithi100.stdout:Deleting cluster with fsid: 3056d51a-e76b-11ee-95cd-87774f69a715 2024-03-21T10:40:42.864 DEBUG:teuthology.orchestra.run.smithi157:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 3056d51a-e76b-11ee-95cd-87774f69a715 --force 2024-03-21T10:40:43.189 INFO:teuthology.orchestra.run.smithi157.stdout:Deleting cluster with fsid: 3056d51a-e76b-11ee-95cd-87774f69a715 2024-03-21T10:40:43.950 INFO:tasks.cephadm:Removing cephadm ... 2024-03-21T10:40:43.950 DEBUG:teuthology.orchestra.run.smithi047:> rm -rf /home/ubuntu/cephtest/cephadm 2024-03-21T10:40:44.012 DEBUG:teuthology.orchestra.run.smithi100:> rm -rf /home/ubuntu/cephtest/cephadm 2024-03-21T10:40:44.069 DEBUG:teuthology.orchestra.run.smithi157:> rm -rf /home/ubuntu/cephtest/cephadm 2024-03-21T10:40:44.127 INFO:tasks.cephadm:Teardown complete 2024-03-21T10:40:44.127 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-03-21T10:40:44.155 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-03-21T10:40:44.156 DEBUG:teuthology.orchestra.run.smithi047:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-03-21T10:40:44.163 DEBUG:teuthology.orchestra.run.smithi100:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-03-21T10:40:44.171 DEBUG:teuthology.orchestra.run.smithi157:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-03-21T10:40:44.338 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-03-21T10:40:44.354 INFO:teuthology.task.clock:Checking final clock skew... 2024-03-21T10:40:44.355 DEBUG:teuthology.orchestra.run.smithi047:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-03-21T10:40:44.363 DEBUG:teuthology.orchestra.run.smithi100:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-03-21T10:40:44.371 DEBUG:teuthology.orchestra.run.smithi157:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-03-21T10:40:44.413 INFO:teuthology.orchestra.run.smithi047.stderr:bash: line 1: ntpq: command not found 2024-03-21T10:40:44.417 INFO:teuthology.orchestra.run.smithi100.stderr:bash: line 1: ntpq: command not found 2024-03-21T10:40:44.425 INFO:teuthology.orchestra.run.smithi047.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-03-21T10:40:44.425 INFO:teuthology.orchestra.run.smithi047.stdout:=============================================================================== 2024-03-21T10:40:44.425 INFO:teuthology.orchestra.run.smithi047.stdout:^+ hv01.front.sepia.ceph.com 2 6 377 43 +284us[ +308us] +/- 72ms 2024-03-21T10:40:44.425 INFO:teuthology.orchestra.run.smithi047.stdout:^* hv02.front.sepia.ceph.com 3 6 377 43 -907us[ -883us] +/- 78ms 2024-03-21T10:40:44.425 INFO:teuthology.orchestra.run.smithi047.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 45 +667us[ +691us] +/- 84ms 2024-03-21T10:40:44.425 INFO:teuthology.orchestra.run.smithi047.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-03-21T10:40:44.426 INFO:teuthology.orchestra.run.smithi100.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-03-21T10:40:44.426 INFO:teuthology.orchestra.run.smithi100.stdout:=============================================================================== 2024-03-21T10:40:44.426 INFO:teuthology.orchestra.run.smithi100.stdout:^+ hv01.front.sepia.ceph.com 2 6 377 44 +316us[ +316us] +/- 72ms 2024-03-21T10:40:44.426 INFO:teuthology.orchestra.run.smithi100.stdout:^* hv02.front.sepia.ceph.com 3 6 377 45 -864us[ -818us] +/- 78ms 2024-03-21T10:40:44.426 INFO:teuthology.orchestra.run.smithi100.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 44 +700us[ +700us] +/- 84ms 2024-03-21T10:40:44.426 INFO:teuthology.orchestra.run.smithi100.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-03-21T10:40:44.427 INFO:teuthology.orchestra.run.smithi157.stderr:bash: line 1: ntpq: command not found 2024-03-21T10:40:44.437 INFO:teuthology.orchestra.run.smithi157.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-03-21T10:40:44.437 INFO:teuthology.orchestra.run.smithi157.stdout:=============================================================================== 2024-03-21T10:40:44.437 INFO:teuthology.orchestra.run.smithi157.stdout:^+ hv01.front.sepia.ceph.com 2 6 377 42 +324us[ +324us] +/- 72ms 2024-03-21T10:40:44.437 INFO:teuthology.orchestra.run.smithi157.stdout:^* hv02.front.sepia.ceph.com 3 7 377 44 -837us[ -812us] +/- 78ms 2024-03-21T10:40:44.437 INFO:teuthology.orchestra.run.smithi157.stdout:^+ hv03.front.sepia.ceph.com 3 7 377 111 +748us[ +767us] +/- 84ms 2024-03-21T10:40:44.437 INFO:teuthology.orchestra.run.smithi157.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-03-21T10:40:44.439 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-03-21T10:40:44.450 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-03-21T10:40:44.450 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-03-21T10:40:44.464 DEBUG:teuthology.orchestra.run.smithi047:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-03-21T10:40:44.726 DEBUG:teuthology.orchestra.run.smithi100:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-03-21T10:40:44.946 DEBUG:teuthology.orchestra.run.smithi157:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-03-21T10:40:45.165 DEBUG:teuthology.orchestra.run.smithi047:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-03-21T10:40:45.272 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-21T10:40:45.273 DEBUG:teuthology.orchestra.run.smithi100:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-03-21T10:40:45.372 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-21T10:40:45.373 DEBUG:teuthology.orchestra.run.smithi157:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-03-21T10:40:45.474 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-21T10:40:45.475 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-03-21T10:40:45.486 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-03-21T10:40:45.501 INFO:teuthology.task.internal:Duration was 2366.391366 seconds 2024-03-21T10:40:45.501 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-03-21T10:40:45.515 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-03-21T10:40:45.515 DEBUG:teuthology.orchestra.run.smithi047:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-03-21T10:40:45.523 DEBUG:teuthology.orchestra.run.smithi100:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-03-21T10:40:45.530 DEBUG:teuthology.orchestra.run.smithi157:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-03-21T10:40:45.689 INFO:teuthology.orchestra.run.smithi100.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-03-21T10:40:45.695 INFO:teuthology.orchestra.run.smithi157.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-03-21T10:40:45.695 INFO:teuthology.orchestra.run.smithi047.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-03-21T10:40:46.047 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-03-21T10:40:46.047 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi047.front.sepia.ceph.com 2024-03-21T10:40:46.048 DEBUG:teuthology.orchestra.run.smithi047:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-03-21T10:40:46.114 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi100.front.sepia.ceph.com 2024-03-21T10:40:46.115 DEBUG:teuthology.orchestra.run.smithi100:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-03-21T10:40:46.191 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi157.front.sepia.ceph.com 2024-03-21T10:40:46.191 DEBUG:teuthology.orchestra.run.smithi157:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-03-21T10:40:46.275 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-03-21T10:40:46.276 DEBUG:teuthology.orchestra.run.smithi047:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-03-21T10:40:46.284 DEBUG:teuthology.orchestra.run.smithi100:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-03-21T10:40:46.292 DEBUG:teuthology.orchestra.run.smithi157:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-03-21T10:40:46.385 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-03-21T10:40:46.385 DEBUG:teuthology.orchestra.run.smithi047:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-03-21T10:40:46.391 DEBUG:teuthology.orchestra.run.smithi100:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-03-21T10:40:46.396 DEBUG:teuthology.orchestra.run.smithi157:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-03-21T10:40:46.938 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-03-21T10:40:46.953 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-03-21T10:40:46.953 DEBUG:teuthology.orchestra.run.smithi047:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-03-21T10:40:47.055 DEBUG:teuthology.orchestra.run.smithi100:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-03-21T10:40:47.156 DEBUG:teuthology.orchestra.run.smithi157:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-03-21T10:40:47.257 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-03-21T10:40:47.268 DEBUG:teuthology.orchestra.run.smithi047:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-03-21T10:40:47.276 DEBUG:teuthology.orchestra.run.smithi100:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-03-21T10:40:47.284 DEBUG:teuthology.orchestra.run.smithi157:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-03-21T10:40:47.386 INFO:teuthology.orchestra.run.smithi047.stdout:kernel.core_pattern = core 2024-03-21T10:40:47.389 INFO:teuthology.orchestra.run.smithi100.stdout:kernel.core_pattern = core 2024-03-21T10:40:47.391 INFO:teuthology.orchestra.run.smithi157.stdout:kernel.core_pattern = core 2024-03-21T10:40:47.456 DEBUG:teuthology.orchestra.run.smithi047:> test -e /home/ubuntu/cephtest/archive/coredump 2024-03-21T10:40:47.510 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-21T10:40:47.510 DEBUG:teuthology.orchestra.run.smithi100:> test -e /home/ubuntu/cephtest/archive/coredump 2024-03-21T10:40:47.563 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-21T10:40:47.564 DEBUG:teuthology.orchestra.run.smithi157:> test -e /home/ubuntu/cephtest/archive/coredump 2024-03-21T10:40:47.618 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-21T10:40:47.618 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-03-21T10:40:47.630 INFO:teuthology.task.internal:Transferring archived files... 2024-03-21T10:40:47.631 DEBUG:teuthology.misc:Transferring archived files from smithi047:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/vshankar-2024-03-20_11:21:18-fs-wip-vshankar-testing-20240320.053002-testing-default-smithi/7612221/remote/smithi047 2024-03-21T10:40:47.631 DEBUG:teuthology.orchestra.run.smithi047:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-03-21T10:40:47.751 DEBUG:teuthology.misc:Transferring archived files from smithi100:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/vshankar-2024-03-20_11:21:18-fs-wip-vshankar-testing-20240320.053002-testing-default-smithi/7612221/remote/smithi100 2024-03-21T10:40:47.751 DEBUG:teuthology.orchestra.run.smithi100:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-03-21T10:40:47.857 DEBUG:teuthology.misc:Transferring archived files from smithi157:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/vshankar-2024-03-20_11:21:18-fs-wip-vshankar-testing-20240320.053002-testing-default-smithi/7612221/remote/smithi157 2024-03-21T10:40:47.857 DEBUG:teuthology.orchestra.run.smithi157:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-03-21T10:40:47.965 INFO:teuthology.task.internal:Removing archive directory... 2024-03-21T10:40:47.965 DEBUG:teuthology.orchestra.run.smithi047:> rm -rf -- /home/ubuntu/cephtest/archive 2024-03-21T10:40:47.973 DEBUG:teuthology.orchestra.run.smithi100:> rm -rf -- /home/ubuntu/cephtest/archive 2024-03-21T10:40:47.980 DEBUG:teuthology.orchestra.run.smithi157:> rm -rf -- /home/ubuntu/cephtest/archive 2024-03-21T10:40:48.036 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-03-21T10:40:48.050 INFO:teuthology.task.internal:Not uploading archives. 2024-03-21T10:40:48.050 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-03-21T10:40:48.063 INFO:teuthology.task.internal:Tidying up after the test... 2024-03-21T10:40:48.063 DEBUG:teuthology.orchestra.run.smithi047:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-03-21T10:40:48.070 DEBUG:teuthology.orchestra.run.smithi100:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-03-21T10:40:48.078 DEBUG:teuthology.orchestra.run.smithi157:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-03-21T10:40:48.123 INFO:teuthology.orchestra.run.smithi047.stdout: 393229 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Mar 21 10:40 /home/ubuntu/cephtest 2024-03-21T10:40:48.129 INFO:teuthology.orchestra.run.smithi100.stdout: 393229 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Mar 21 10:40 /home/ubuntu/cephtest 2024-03-21T10:40:48.134 INFO:teuthology.orchestra.run.smithi157.stdout: 393724 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Mar 21 10:40 /home/ubuntu/cephtest 2024-03-21T10:40:48.137 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-03-21T10:40:48.149 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-03-21T10:40:48.174 INFO:teuthology.nuke:Checking targets against current locks 2024-03-21T10:40:48.201 DEBUG:teuthology.nuke:shortname: smithi047 2024-03-21T10:40:48.201 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-03-21T10:40:48.226 DEBUG:teuthology.nuke:shortname: smithi100 2024-03-21T10:40:48.227 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-03-21T10:40:48.232 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi047.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/vshankar-2024-03-20_11:21:18-fs-wip-vshankar-testing-20240320.053002-testing-default-smithi/7612221', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-03-21 09:51:20.906566', 'locked_by': 'scheduled_vshankar@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBHNXJSw2oJjrIRoEo1Osgd8J/rbxEni4kT0rPJ7iyFCCmLmOeHp7qFWpLHBfaLStn9uQ5h8B5wjIeEzYPyxoawU='} 2024-03-21T10:40:48.250 DEBUG:teuthology.nuke:shortname: smithi157 2024-03-21T10:40:48.250 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-03-21T10:40:48.253 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi100.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/vshankar-2024-03-20_11:21:18-fs-wip-vshankar-testing-20240320.053002-testing-default-smithi/7612221', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-03-21 09:51:20.904946', 'locked_by': 'scheduled_vshankar@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBCtiuUJf14dOt1EXUNXu5LKRKerqT+HKFTSH10265CedGplUrp3qL+/yj4PXM8//sBfj+Pra2spb7RAgx1LOEBw='} 2024-03-21T10:40:48.273 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi157.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/vshankar-2024-03-20_11:21:18-fs-wip-vshankar-testing-20240320.053002-testing-default-smithi/7612221', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-03-21 09:51:20.907815', 'locked_by': 'scheduled_vshankar@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBM/llVo/dKuUfYnnYg3GbzDRHmUgFjiTlkAr3eSJvGV/iWSij9WSYdIjABt+AMkcLr9HLbGhSL7qCEIaxaDwTmk='} 2024-03-21T10:40:48.289 INFO:teuthology.orchestra.console.smithi047:Power off 2024-03-21T10:40:48.289 DEBUG:teuthology.orchestra.console.smithi047:pexpect command: ipmitool -H smithi047.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-03-21T10:40:48.314 DEBUG:teuthology.orchestra.console.smithi047:power off output: Chassis Power Control: Down/Off 2024-03-21T10:40:48.314 DEBUG:teuthology.orchestra.console.smithi047:pexpect command: ipmitool -H smithi047.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-21T10:40:48.330 INFO:teuthology.orchestra.console.smithi157:Power off 2024-03-21T10:40:48.330 DEBUG:teuthology.orchestra.console.smithi157:pexpect command: ipmitool -H smithi157.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-03-21T10:40:48.342 INFO:teuthology.orchestra.console.smithi100:Power off 2024-03-21T10:40:48.343 DEBUG:teuthology.orchestra.console.smithi100:pexpect command: ipmitool -H smithi100.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-03-21T10:40:48.351 DEBUG:teuthology.orchestra.console.smithi157:power off output: Chassis Power Control: Down/Off 2024-03-21T10:40:48.352 DEBUG:teuthology.orchestra.console.smithi157:pexpect command: ipmitool -H smithi157.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-21T10:40:48.359 DEBUG:teuthology.orchestra.console.smithi047:check power output: Chassis Power is on 2024-03-21T10:40:48.364 DEBUG:teuthology.orchestra.console.smithi100:power off output: Chassis Power Control: Down/Off 2024-03-21T10:40:48.364 DEBUG:teuthology.orchestra.console.smithi100:pexpect command: ipmitool -H smithi100.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-21T10:40:48.373 DEBUG:teuthology.orchestra.console.smithi157:check power output: Chassis Power is on 2024-03-21T10:40:48.386 DEBUG:teuthology.orchestra.console.smithi100:check power output: Chassis Power is on 2024-03-21T10:40:52.360 DEBUG:teuthology.orchestra.console.smithi047:pexpect command: ipmitool -H smithi047.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-21T10:40:52.374 DEBUG:teuthology.orchestra.console.smithi157:pexpect command: ipmitool -H smithi157.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-21T10:40:52.387 DEBUG:teuthology.orchestra.console.smithi100:pexpect command: ipmitool -H smithi100.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-21T10:40:52.472 DEBUG:teuthology.orchestra.console.smithi047:check power output: Chassis Power is on 2024-03-21T10:40:52.483 DEBUG:teuthology.orchestra.console.smithi157:check power output: Chassis Power is on 2024-03-21T10:40:52.495 DEBUG:teuthology.orchestra.console.smithi100:check power output: Chassis Power is on 2024-03-21T10:40:56.476 DEBUG:teuthology.orchestra.console.smithi047:pexpect command: ipmitool -H smithi047.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-21T10:40:56.486 DEBUG:teuthology.orchestra.console.smithi157:pexpect command: ipmitool -H smithi157.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-21T10:40:56.497 DEBUG:teuthology.orchestra.console.smithi100:pexpect command: ipmitool -H smithi100.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-21T10:40:56.588 DEBUG:teuthology.orchestra.console.smithi047:check power output: Chassis Power is off 2024-03-21T10:40:56.598 DEBUG:teuthology.orchestra.console.smithi157:check power output: Chassis Power is off 2024-03-21T10:40:56.608 DEBUG:teuthology.orchestra.console.smithi100:check power output: Chassis Power is on 2024-03-21T10:40:56.690 INFO:teuthology.orchestra.console.smithi047:Power off completed 2024-03-21T10:40:56.699 INFO:teuthology.orchestra.console.smithi157:Power off completed 2024-03-21T10:41:00.610 DEBUG:teuthology.orchestra.console.smithi100:pexpect command: ipmitool -H smithi100.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-21T10:41:00.719 DEBUG:teuthology.orchestra.console.smithi100:check power output: Chassis Power is off 2024-03-21T10:41:00.820 INFO:teuthology.orchestra.console.smithi100:Power off completed 2024-03-21T10:41:00.925 INFO:teuthology.run:Summary data: description: fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate} clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} mount/fuse objectstore-ec/bluestore-ec-root omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/automatic export-check n/5 replication/default} standby-replay tasks/{0-subvolume/{with-namespace-isolated} 1-check-counter 2-scrub/yes 3-snaps/yes 4-flush/yes 5-workunit/suites/fsync-tester}} duration: 2366.391366481781 failure_reason: '"2024-03-21T10:13:44.902312+0000 mon.a (mon.0) 183 : cluster [WRN] Health detail: HEALTH_WARN 1/3 mons down, quorum a,c" in cluster log' flavor: default mon.a-kernel-sha1: 5ec3e6e88b2e2783e09498588bb1f433ae9c8986 mon.b-kernel-sha1: 5ec3e6e88b2e2783e09498588bb1f433ae9c8986 mon.c-kernel-sha1: 5ec3e6e88b2e2783e09498588bb1f433ae9c8986 owner: scheduled_vshankar@teuthology success: false 2024-03-21T10:41:00.926 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-03-21T10:41:01.031 INFO:teuthology.run:FAIL